2024-01-24T13:44:52.205 INFO:root:teuthology version: 0.0.1.dev251+gc856e0f 2024-01-24T13:44:52.205 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993 --name vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi --verbose --owner scheduled_vshankar@teuthology --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} -- /home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993/orig.config.yaml 2024-01-24T13:44:52.250 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T13:44:52.325 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993 branch: wip-vshankar-testing-pacific-020240119.072505 description: fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: vshankar@redhat.com first_in_suite: false job_id: '7530993' kernel: client: sha1: distro flavor: default kdb: true sha1: f2d3460408951339ac0933340ad449487c9efddc last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-vshankar-testing-pacific-020240119.072505 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 9854ba8e5f82e4476d4c6c06dd8feefca92b8b32 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9854ba8e5f82e4476d4c6c06dd8feefca92b8b32 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-pacific-020240119.072505 sha1: 9854ba8e5f82e4476d4c6c06dd8feefca92b8b32 owner: scheduled_vshankar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2977 sha1: 9854ba8e5f82e4476d4c6c06dd8feefca92b8b32 sleep_before_teardown: 0 subset: 749/1024 suite: fs suite_branch: wip-vshankar-testing-pacific-020240119.072505 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9854ba8e5f82e4476d4c6c06dd8feefca92b8b32 targets: smithi029.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI6PZ2wPbShQTiM/sTKlfYNw8H9btmfys69InEuhp0dLIpq2XsOxNNeYRu+9JOp4KpOsakfk4NvM3/d57tFr91M= smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG7uo/iwSJdGQ71ymsdoikagQTGpJfY2j1wVDZ+MX9OUV8Jev1J0swzSpQDMJ1RkC3+idQGjoMx1olU7dutxTKM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: c856e0f9244dd29a50697123b480d63f16a4ed81 timestamp: 2024-01-24_13:20:07 tube: smithi use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-24T13:44:52.326 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/qa; will attempt to use it 2024-01-24T13:44:52.326 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/qa/tasks 2024-01-24T13:44:52.326 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-24T13:44:52.328 INFO:teuthology.task.internal:Checking packages... 2024-01-24T13:44:52.352 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '9854ba8e5f82e4476d4c6c06dd8feefca92b8b32' 2024-01-24T13:44:52.352 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-24T13:44:52.352 INFO:teuthology.packaging:ref: None 2024-01-24T13:44:52.352 INFO:teuthology.packaging:tag: None 2024-01-24T13:44:52.352 INFO:teuthology.packaging:branch: wip-vshankar-testing-pacific-020240119.072505 2024-01-24T13:44:52.352 INFO:teuthology.packaging:sha1: 9854ba8e5f82e4476d4c6c06dd8feefca92b8b32 2024-01-24T13:44:52.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing-pacific-020240119.072505 2024-01-24T13:44:52.573 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-452.g9854ba8e 2024-01-24T13:44:52.575 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-24T13:44:52.584 INFO:teuthology.task.internal:no buildpackages task found 2024-01-24T13:44:52.584 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-24T13:44:52.610 INFO:teuthology.task.internal:Saving configuration 2024-01-24T13:44:52.626 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-24T13:44:52.638 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-24T13:44:52.663 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-24 13:39:10.629982', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI6PZ2wPbShQTiM/sTKlfYNw8H9btmfys69InEuhp0dLIpq2XsOxNNeYRu+9JOp4KpOsakfk4NvM3/d57tFr91M='} 2024-01-24T13:44:52.689 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-24 13:39:10.627900', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG7uo/iwSJdGQ71ymsdoikagQTGpJfY2j1wVDZ+MX9OUV8Jev1J0swzSpQDMJ1RkC3+idQGjoMx1olU7dutxTKM='} 2024-01-24T13:44:52.689 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-24T13:44:52.701 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-01-24T13:44:52.701 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-01-24T13:44:52.701 INFO:teuthology.run_tasks:Running task console_log... 2024-01-24T13:44:52.791 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3571e500d0>, signals=[15]) 2024-01-24T13:44:52.792 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-24T13:44:52.805 INFO:teuthology.task.internal:Opening connections... 2024-01-24T13:44:52.805 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T13:44:52.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T13:44:52.886 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T13:44:52.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T13:44:52.973 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-24T13:44:52.982 DEBUG:teuthology.orchestra.run.smithi029:> uname -m 2024-01-24T13:44:53.005 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2024-01-24T13:44:53.005 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2024-01-24T13:44:53.066 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="Red Hat Enterprise Linux" 2024-01-24T13:44:53.066 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="8.6 (Ootpa)" 2024-01-24T13:44:53.066 INFO:teuthology.orchestra.run.smithi029.stdout:ID="rhel" 2024-01-24T13:44:53.066 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="fedora" 2024-01-24T13:44:53.066 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="8.6" 2024-01-24T13:44:53.066 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el8" 2024-01-24T13:44:53.067 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-24T13:44:53.067 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2024-01-24T13:44:53.067 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-24T13:44:53.067 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://www.redhat.com/" 2024-01-24T13:44:53.067 INFO:teuthology.orchestra.run.smithi029.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-24T13:44:53.067 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-24T13:44:53.067 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:44:53.067 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-24T13:44:53.067 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-24T13:44:53.067 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-24T13:44:53.067 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-24T13:44:53.068 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2024-01-24T13:44:53.108 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2024-01-24T13:44:53.130 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2024-01-24T13:44:53.130 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2024-01-24T13:44:53.189 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="Red Hat Enterprise Linux" 2024-01-24T13:44:53.189 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="8.6 (Ootpa)" 2024-01-24T13:44:53.189 INFO:teuthology.orchestra.run.smithi163.stdout:ID="rhel" 2024-01-24T13:44:53.189 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="fedora" 2024-01-24T13:44:53.189 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="8.6" 2024-01-24T13:44:53.189 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el8" 2024-01-24T13:44:53.189 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-24T13:44:53.190 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2024-01-24T13:44:53.190 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-24T13:44:53.190 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://www.redhat.com/" 2024-01-24T13:44:53.190 INFO:teuthology.orchestra.run.smithi163.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-24T13:44:53.190 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-24T13:44:53.190 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:44:53.190 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-24T13:44:53.190 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-24T13:44:53.190 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-24T13:44:53.190 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-24T13:44:53.190 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2024-01-24T13:44:53.224 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-24T13:44:53.236 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-24T13:44:53.257 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-24T13:44:53.258 DEBUG:teuthology.orchestra.run.smithi029:> test '!' -e /home/ubuntu/cephtest 2024-01-24T13:44:53.260 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2024-01-24T13:44:53.279 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-24T13:44:53.290 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-24T13:44:53.290 DEBUG:teuthology.orchestra.run.smithi029:> test -z $(ls -A /var/lib/ceph) 2024-01-24T13:44:53.320 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2024-01-24T13:44:53.348 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-24T13:44:53.403 INFO:teuthology.run_tasks:Running task kernel... 2024-01-24T13:44:53.424 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'f2d3460408951339ac0933340ad449487c9efddc'} 2024-01-24T13:44:53.424 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-01-24T13:44:53.424 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-01-24T13:44:53.424 DEBUG:teuthology.orchestra.run.smithi029:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-24T13:44:53.425 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-24T13:44:53.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T13:44:53.444 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2024-01-24T13:44:53.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T13:44:53.445 DEBUG:teuthology.orchestra.run.smithi029:> uname -r 2024-01-24T13:44:53.505 INFO:teuthology.orchestra.run.smithi163.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-01-24T13:44:53.506 INFO:teuthology.task.kernel:Running kernel on smithi163: 4.18.0-513.9.1.el8_9.x86_64 2024-01-24T13:44:53.506 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum install -y kernel 2024-01-24T13:44:53.508 INFO:teuthology.orchestra.run.smithi029.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-01-24T13:44:53.509 INFO:teuthology.task.kernel:Running kernel on smithi029: 4.18.0-513.9.1.el8_9.x86_64 2024-01-24T13:44:53.509 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum install -y kernel 2024-01-24T13:44:54.950 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2024-01-24T13:44:54.951 INFO:teuthology.orchestra.run.smithi163.stdout:Unable to read consumer identity 2024-01-24T13:44:55.217 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:44:55.217 INFO:teuthology.orchestra.run.smithi163.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-24T13:44:55.217 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:44:55.637 INFO:teuthology.orchestra.run.smithi029.stdout:Updating Subscription Management repositories. 2024-01-24T13:44:55.638 INFO:teuthology.orchestra.run.smithi029.stdout:Unable to read consumer identity 2024-01-24T13:44:55.908 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:44:55.909 INFO:teuthology.orchestra.run.smithi029.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-24T13:44:55.909 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:44:56.966 INFO:teuthology.orchestra.run.smithi163.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2024-01-24T13:44:57.942 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2024-01-24T13:44:59.421 INFO:teuthology.orchestra.run.smithi029.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-01-24T13:45:00.280 INFO:teuthology.orchestra.run.smithi029.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2024-01-24T13:45:01.534 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 403 kB/s | 24 kB 00:00 2024-01-24T13:45:02.286 INFO:teuthology.orchestra.run.smithi163.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-01-24T13:45:02.286 INFO:teuthology.orchestra.run.smithi163.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-01-24T13:45:02.376 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-01-24T13:45:02.377 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do. 2024-01-24T13:45:02.377 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-01-24T13:45:02.510 DEBUG:teuthology.orchestra.run.smithi163:> echo no | sudo yum reinstall kernel || true 2024-01-24T13:45:02.845 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2024-01-24T13:45:02.845 INFO:teuthology.orchestra.run.smithi163.stdout:Unable to read consumer identity 2024-01-24T13:45:03.110 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:45:03.111 INFO:teuthology.orchestra.run.smithi163.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-24T13:45:03.111 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:45:03.422 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:00:02 ago on Wed 24 Jan 2024 01:45:01 PM UTC. 2024-01-24T13:45:03.503 INFO:teuthology.orchestra.run.smithi163.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-24T13:45:03.508 INFO:teuthology.orchestra.run.smithi163.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-24T13:45:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:Error: No packages marked for reinstall. 2024-01-24T13:45:03.593 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum reinstall -y kernel || true 2024-01-24T13:45:03.928 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2024-01-24T13:45:03.928 INFO:teuthology.orchestra.run.smithi163.stdout:Unable to read consumer identity 2024-01-24T13:45:04.130 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:45:04.130 INFO:teuthology.orchestra.run.smithi163.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-24T13:45:04.131 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:45:04.348 INFO:teuthology.orchestra.run.smithi029.stdout:lab-extras 388 kB/s | 24 kB 00:00 2024-01-24T13:45:04.439 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Jan 2024 01:45:01 PM UTC. 2024-01-24T13:45:04.506 INFO:teuthology.orchestra.run.smithi163.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-24T13:45:04.506 INFO:teuthology.orchestra.run.smithi163.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-24T13:45:04.544 INFO:teuthology.orchestra.run.smithi163.stderr:Error: No packages marked for reinstall. 2024-01-24T13:45:04.596 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q kernel | sort -rV | head -n 1 2024-01-24T13:45:04.759 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-01-24T13:45:04.760 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-01-24T13:45:04.760 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-01-24T13:45:04.760 DEBUG:teuthology.parallel:result is None 2024-01-24T13:45:05.187 INFO:teuthology.orchestra.run.smithi029.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-01-24T13:45:05.188 INFO:teuthology.orchestra.run.smithi029.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-01-24T13:45:05.301 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-01-24T13:45:05.303 INFO:teuthology.orchestra.run.smithi029.stdout:Nothing to do. 2024-01-24T13:45:05.303 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-01-24T13:45:05.467 DEBUG:teuthology.orchestra.run.smithi029:> echo no | sudo yum reinstall kernel || true 2024-01-24T13:45:05.858 INFO:teuthology.orchestra.run.smithi029.stdout:Updating Subscription Management repositories. 2024-01-24T13:45:05.858 INFO:teuthology.orchestra.run.smithi029.stdout:Unable to read consumer identity 2024-01-24T13:45:06.121 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:45:06.121 INFO:teuthology.orchestra.run.smithi029.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-24T13:45:06.121 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:45:06.488 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:00:02 ago on Wed 24 Jan 2024 01:45:04 PM UTC. 2024-01-24T13:45:06.593 INFO:teuthology.orchestra.run.smithi029.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-24T13:45:06.604 INFO:teuthology.orchestra.run.smithi029.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-24T13:45:06.655 INFO:teuthology.orchestra.run.smithi029.stderr:Error: No packages marked for reinstall. 2024-01-24T13:45:06.711 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum reinstall -y kernel || true 2024-01-24T13:45:07.138 INFO:teuthology.orchestra.run.smithi029.stdout:Updating Subscription Management repositories. 2024-01-24T13:45:07.138 INFO:teuthology.orchestra.run.smithi029.stdout:Unable to read consumer identity 2024-01-24T13:45:07.335 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:45:07.336 INFO:teuthology.orchestra.run.smithi029.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-24T13:45:07.336 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:45:07.691 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Jan 2024 01:45:04 PM UTC. 2024-01-24T13:45:07.760 INFO:teuthology.orchestra.run.smithi029.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-24T13:45:07.760 INFO:teuthology.orchestra.run.smithi029.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-24T13:45:07.793 INFO:teuthology.orchestra.run.smithi029.stderr:Error: No packages marked for reinstall. 2024-01-24T13:45:07.848 DEBUG:teuthology.orchestra.run.smithi029:> rpm -q kernel | sort -rV | head -n 1 2024-01-24T13:45:08.018 INFO:teuthology.orchestra.run.smithi029.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-01-24T13:45:08.019 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-01-24T13:45:08.019 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-01-24T13:45:08.020 DEBUG:teuthology.parallel:result is None 2024-01-24T13:45:08.020 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-24T13:45:08.030 INFO:teuthology.task.internal:Creating test directory... 2024-01-24T13:45:08.030 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-24T13:45:08.033 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-24T13:45:08.054 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-24T13:45:08.063 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-24T13:45:08.104 INFO:teuthology.task.internal:Creating archive directory... 2024-01-24T13:45:08.104 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-24T13:45:08.106 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-24T13:45:08.148 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-24T13:45:08.156 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-24T13:45:08.157 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-24T13:45:08.177 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-24T13:45:08.218 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-24T13:45:08.223 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-24T13:45:08.240 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-24T13:45:08.241 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-24T13:45:08.243 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-24T13:45:08.250 INFO:teuthology.task.internal:Configuring sudo... 2024-01-24T13:45:08.251 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-24T13:45:08.285 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-24T13:45:08.318 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-24T13:45:08.332 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-24T13:45:08.332 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-24T13:45:08.361 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-24T13:45:08.380 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-24T13:45:08.470 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-24T13:45:08.526 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-24T13:45:08.615 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-24T13:45:08.665 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:45:08.665 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-24T13:45:08.739 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-24T13:45:08.785 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-24T13:45:08.855 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-24T13:45:08.903 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-24T13:45:08.944 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:45:08.945 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-24T13:45:09.017 DEBUG:teuthology.orchestra.run.smithi029:> sudo service rsyslog restart 2024-01-24T13:45:09.019 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2024-01-24T13:45:09.081 INFO:teuthology.orchestra.run.smithi029.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-24T13:45:09.111 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-24T13:45:09.521 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-24T13:45:09.530 INFO:teuthology.task.internal:Starting timer... 2024-01-24T13:45:09.530 INFO:teuthology.run_tasks:Running task pcp... 2024-01-24T13:45:09.579 INFO:teuthology.run_tasks:Running task selinux... 2024-01-24T13:45:09.609 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-01-24T13:45:09.610 DEBUG:teuthology.orchestra.run.smithi029:> sudo service auditd rotate 2024-01-24T13:45:09.698 INFO:teuthology.orchestra.run.smithi029.stdout:Rotating logs: 2024-01-24T13:45:09.700 DEBUG:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2024-01-24T13:45:09.775 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: 2024-01-24T13:45:09.777 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-24T13:45:09.777 DEBUG:teuthology.orchestra.run.smithi029:> /usr/sbin/getenforce 2024-01-24T13:45:09.805 INFO:teuthology.orchestra.run.smithi029.stdout:Permissive 2024-01-24T13:45:09.805 DEBUG:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2024-01-24T13:45:09.826 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2024-01-24T13:45:09.827 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi029.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi163.front.sepia.ceph.com': 'permissive'} 2024-01-24T13:45:09.827 DEBUG:teuthology.orchestra.run.smithi029:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-24T13:45:09.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T13:45:09.877 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-24T13:45:09.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T13:45:09.904 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-24T13:45:09.905 DEBUG:teuthology.orchestra.run.smithi029:> sudo /usr/sbin/setenforce permissive 2024-01-24T13:45:09.949 DEBUG:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2024-01-24T13:45:09.979 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-24T13:45:09.989 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-24T13:45:10.006 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-24T13:45:10.007 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi029.front.sepia.ceph.com,smithi163.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-24T13:54:29.757 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi029.front.sepia.ceph.com'), Remote(name='ubuntu@smithi163.front.sepia.ceph.com')] 2024-01-24T13:54:29.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-01-24T13:54:29.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T13:54:29.845 DEBUG:teuthology.orchestra.run.smithi029:> true 2024-01-24T13:54:29.922 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-01-24T13:54:29.923 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-01-24T13:54:29.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T13:54:30.002 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-01-24T13:54:30.073 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-01-24T13:54:30.073 INFO:teuthology.run_tasks:Running task clock... 2024-01-24T13:54:30.085 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-24T13:54:30.086 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-24T13:54:30.086 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-24T13:54:30.089 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-24T13:54:30.089 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-24T13:54:30.127 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-24T13:54:30.145 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-24T13:54:30.147 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-24T13:54:30.163 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-24T13:54:30.221 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: ntpd: command not found 2024-01-24T13:54:30.235 INFO:teuthology.orchestra.run.smithi163.stderr:sudo: ntpd: command not found 2024-01-24T13:54:30.236 INFO:teuthology.orchestra.run.smithi029.stdout:506 Cannot talk to daemon 2024-01-24T13:54:30.247 INFO:teuthology.orchestra.run.smithi163.stdout:506 Cannot talk to daemon 2024-01-24T13:54:30.254 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-24T13:54:30.262 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-24T13:54:30.271 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-24T13:54:30.277 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-24T13:54:30.343 INFO:teuthology.orchestra.run.smithi163.stderr:bash: ntpq: command not found 2024-01-24T13:54:30.346 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-24T13:54:30.346 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-01-24T13:54:30.346 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-24T13:54:30.346 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-24T13:54:30.346 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-24T13:54:30.347 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-24T13:54:30.363 INFO:teuthology.orchestra.run.smithi029.stderr:bash: ntpq: command not found 2024-01-24T13:54:30.367 INFO:teuthology.orchestra.run.smithi029.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-24T13:54:30.367 INFO:teuthology.orchestra.run.smithi029.stdout:=============================================================================== 2024-01-24T13:54:30.367 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-24T13:54:30.367 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-24T13:54:30.367 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-24T13:54:30.367 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-24T13:54:30.368 INFO:teuthology.run_tasks:Running task install... 2024-01-24T13:54:30.376 DEBUG:teuthology.task.install:project ceph 2024-01-24T13:54:30.376 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9854ba8e5f82e4476d4c6c06dd8feefca92b8b32'}} 2024-01-24T13:54:30.376 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '9854ba8e5f82e4476d4c6c06dd8feefca92b8b32'} 2024-01-24T13:54:30.376 INFO:teuthology.task.install:Using flavor: default 2024-01-24T13:54:30.382 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-24T13:54:30.382 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-01-24T13:54:30.382 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9854ba8e5f82e4476d4c6c06dd8feefca92b8b32', 'tag': None, 'wait_for_package': False} 2024-01-24T13:54:30.383 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9854ba8e5f82e4476d4c6c06dd8feefca92b8b32 2024-01-24T13:54:30.385 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9854ba8e5f82e4476d4c6c06dd8feefca92b8b32', 'tag': None, 'wait_for_package': False} 2024-01-24T13:54:30.385 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9854ba8e5f82e4476d4c6c06dd8feefca92b8b32 2024-01-24T13:54:30.532 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-020240119.072505/9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/centos/8/flavors/default/ 2024-01-24T13:54:30.533 INFO:teuthology.task.install.rpm:Package version is 16.2.14-452.g9854ba8e 2024-01-24T13:54:30.633 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-020240119.072505/9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/centos/8/flavors/default/ 2024-01-24T13:54:30.633 INFO:teuthology.task.install.rpm:Package version is 16.2.14-452.g9854ba8e 2024-01-24T13:54:30.774 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-020240119.072505/9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-020240119.072505/9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-020240119.072505/9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-24T13:54:30.775 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:54:30.775 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-24T13:54:30.803 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2024-01-24T13:54:30.803 DEBUG:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-24T13:54:30.872 DEBUG:teuthology.orchestra.run.smithi163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-24T13:54:30.951 DEBUG:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-24T13:54:31.017 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2024-01-24T13:54:31.019 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum clean all 2024-01-24T13:54:31.031 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-020240119.072505/9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-020240119.072505/9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-020240119.072505/9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-24T13:54:31.031 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:54:31.031 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-24T13:54:31.066 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2024-01-24T13:54:31.066 DEBUG:teuthology.orchestra.run.smithi029:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-24T13:54:31.142 DEBUG:teuthology.orchestra.run.smithi029:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-24T13:54:31.231 DEBUG:teuthology.orchestra.run.smithi029:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-24T13:54:31.263 INFO:teuthology.orchestra.run.smithi029.stdout:check_obsoletes = 1 2024-01-24T13:54:31.265 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum clean all 2024-01-24T13:54:31.395 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2024-01-24T13:54:31.704 INFO:teuthology.orchestra.run.smithi029.stdout:Updating Subscription Management repositories. 2024-01-24T13:54:31.723 INFO:teuthology.orchestra.run.smithi163.stdout:57 files removed 2024-01-24T13:54:31.776 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2024-01-24T13:54:32.016 INFO:teuthology.orchestra.run.smithi029.stdout:57 files removed 2024-01-24T13:54:32.069 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2024-01-24T13:54:32.110 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2024-01-24T13:54:32.455 INFO:teuthology.orchestra.run.smithi029.stdout:Updating Subscription Management repositories. 2024-01-24T13:54:33.180 INFO:teuthology.orchestra.run.smithi163.stdout:ceph packages for x86_64 129 kB/s | 76 kB 00:00 2024-01-24T13:54:33.428 INFO:teuthology.orchestra.run.smithi029.stdout:ceph packages for x86_64 154 kB/s | 76 kB 00:00 2024-01-24T13:54:33.690 INFO:teuthology.orchestra.run.smithi163.stdout:ceph noarch packages 32 kB/s | 15 kB 00:00 2024-01-24T13:54:34.004 INFO:teuthology.orchestra.run.smithi029.stdout:ceph noarch packages 29 kB/s | 15 kB 00:00 2024-01-24T13:54:34.232 INFO:teuthology.orchestra.run.smithi163.stdout:ceph source packages 2.8 kB/s | 1.4 kB 00:00 2024-01-24T13:54:34.416 INFO:teuthology.orchestra.run.smithi163.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-01-24T13:54:34.472 INFO:teuthology.orchestra.run.smithi029.stdout:ceph source packages 3.3 kB/s | 1.4 kB 00:00 2024-01-24T13:54:34.639 INFO:teuthology.orchestra.run.smithi029.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-01-24T13:54:35.633 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2024-01-24T13:54:39.217 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 409 kB/s | 24 kB 00:00 2024-01-24T13:54:40.277 INFO:teuthology.orchestra.run.smithi163.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 59 MB 00:00 2024-01-24T13:54:40.487 INFO:teuthology.orchestra.run.smithi029.stdout:Extra Packages for Enterprise Linux 2.8 MB/s | 16 MB 00:05 2024-01-24T13:54:44.452 INFO:teuthology.orchestra.run.smithi029.stdout:lab-extras 426 kB/s | 24 kB 00:00 2024-01-24T13:54:46.172 INFO:teuthology.orchestra.run.smithi029.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 MB/s | 59 MB 00:01 2024-01-24T13:54:51.269 INFO:teuthology.orchestra.run.smithi163.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 65 MB 00:01 2024-01-24T13:54:57.535 INFO:teuthology.orchestra.run.smithi029.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 20 MB/s | 8.8 MB 00:00 2024-01-24T13:55:00.976 INFO:teuthology.orchestra.run.smithi029.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 39 MB/s | 65 MB 00:01 2024-01-24T13:55:01.369 INFO:teuthology.orchestra.run.smithi163.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 27 MB/s | 8.8 MB 00:00 2024-01-24T13:55:08.496 INFO:teuthology.orchestra.run.smithi163.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-24T13:55:08.498 INFO:teuthology.orchestra.run.smithi163.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-24T13:55:08.516 INFO:teuthology.orchestra.run.smithi163.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-01-24T13:55:08.518 INFO:teuthology.orchestra.run.smithi163.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-01-24T13:55:08.519 INFO:teuthology.orchestra.run.smithi163.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-01-24T13:55:08.521 INFO:teuthology.orchestra.run.smithi163.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-01-24T13:55:08.524 INFO:teuthology.orchestra.run.smithi163.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-01-24T13:55:08.528 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-01-24T13:55:08.529 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-01-24T13:55:08.731 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-01-24T13:55:08.744 INFO:teuthology.orchestra.run.smithi163.stdout:=========================================================================================================================== 2024-01-24T13:55:08.744 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-01-24T13:55:08.744 INFO:teuthology.orchestra.run.smithi163.stdout:=========================================================================================================================== 2024-01-24T13:55:08.744 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-01-24T13:55:08.745 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-01-24T13:55:08.745 INFO:teuthology.orchestra.run.smithi163.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-01-24T13:55:08.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 6.4 k 2024-01-24T13:55:08.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 6.1 M 2024-01-24T13:55:08.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 914 k 2024-01-24T13:55:08.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 161 k 2024-01-24T13:55:08.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 1.6 M 2024-01-24T13:55:08.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 116 k 2024-01-24T13:55:08.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 3.1 M 2024-01-24T13:55:08.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 8.6 M 2024-01-24T13:55:08.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 27 k 2024-01-24T13:55:08.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 12 M 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 52 M 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 85 k 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 257 k 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 19 k 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 25 k 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 828 k 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 131 k 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-01-24T13:55:08.746 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 215 k 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 389 k 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 369 k 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 114 k 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 88 k 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 3.7 M 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 175 k 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-24T13:55:08.747 INFO:teuthology.orchestra.run.smithi163.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 3.8 M 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 4.2 M 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-01-24T13:55:08.748 INFO:teuthology.orchestra.run.smithi163.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 24 M 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 24 k 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 2.4 M 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 245 k 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 5.0 M 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 18 M 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 15 k 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 26 k 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-01-24T13:55:08.749 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 174 k 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 494 k 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 3.8 M 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-01-24T13:55:08.750 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-01-24T13:55:08.751 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 45 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 99 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-01-24T13:55:08.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-01-24T13:55:08.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-01-24T13:55:08.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout:=========================================================================================================================== 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout:Install 124 Packages 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 16 Packages 2024-01-24T13:55:08.755 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:55:08.756 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 211 M 2024-01-24T13:55:08.756 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-01-24T13:55:08.979 INFO:teuthology.orchestra.run.smithi163.stdout:(1/140): ceph-16.2.14-452.g9854ba8e.el8.x86_64. 34 kB/s | 6.4 kB 00:00 2024-01-24T13:55:09.487 INFO:teuthology.orchestra.run.smithi163.stdout:(2/140): ceph-base-16.2.14-452.g9854ba8e.el8.x8 8.7 MB/s | 6.1 MB 00:00 2024-01-24T13:55:09.562 INFO:teuthology.orchestra.run.smithi163.stdout:(3/140): ceph-immutable-object-cache-16.2.14-45 2.1 MB/s | 161 kB 00:00 2024-01-24T13:55:09.821 INFO:teuthology.orchestra.run.smithi163.stdout:(4/140): ceph-mds-16.2.14-452.g9854ba8e.el8.x86 9.3 MB/s | 2.4 MB 00:00 2024-01-24T13:55:09.971 INFO:teuthology.orchestra.run.smithi163.stdout:(5/140): ceph-fuse-16.2.14-452.g9854ba8e.el8.x8 921 kB/s | 914 kB 00:00 2024-01-24T13:55:10.055 INFO:teuthology.orchestra.run.smithi163.stdout:(6/140): ceph-mgr-16.2.14-452.g9854ba8e.el8.x86 7.0 MB/s | 1.6 MB 00:00 2024-01-24T13:55:10.531 INFO:teuthology.orchestra.run.smithi163.stdout:(7/140): ceph-common-16.2.14-452.g9854ba8e.el8. 14 MB/s | 24 MB 00:01 2024-01-24T13:55:11.364 INFO:teuthology.orchestra.run.smithi163.stdout:(8/140): ceph-radosgw-16.2.14-452.g9854ba8e.el8 14 MB/s | 12 MB 00:00 2024-01-24T13:55:11.415 INFO:teuthology.orchestra.run.smithi163.stdout:(9/140): ceph-selinux-16.2.14-452.g9854ba8e.el8 514 kB/s | 26 kB 00:00 2024-01-24T13:55:11.965 INFO:teuthology.orchestra.run.smithi163.stdout:(10/140): ceph-osd-16.2.14-452.g9854ba8e.el8.x8 9.6 MB/s | 18 MB 00:01 2024-01-24T13:55:12.015 INFO:teuthology.orchestra.run.smithi163.stdout:(11/140): cephfs-mirror-16.2.14-452.g9854ba8e.e 5.0 MB/s | 257 kB 00:00 2024-01-24T13:55:12.073 INFO:teuthology.orchestra.run.smithi163.stdout:(12/140): libcephfs-devel-16.2.14-452.g9854ba8e 438 kB/s | 25 kB 00:00 2024-01-24T13:55:12.216 INFO:teuthology.orchestra.run.smithi163.stdout:(13/140): libcephfs2-16.2.14-452.g9854ba8e.el8. 5.7 MB/s | 828 kB 00:00 2024-01-24T13:55:12.291 INFO:teuthology.orchestra.run.smithi163.stdout:(14/140): libcephsqlite-16.2.14-452.g9854ba8e.e 2.3 MB/s | 174 kB 00:00 2024-01-24T13:55:12.368 INFO:teuthology.orchestra.run.smithi163.stdout:(15/140): librados-devel-16.2.14-452.g9854ba8e. 1.7 MB/s | 131 kB 00:00 2024-01-24T13:55:12.450 INFO:teuthology.orchestra.run.smithi163.stdout:(16/140): libradosstriper1-16.2.14-452.g9854ba8 5.8 MB/s | 494 kB 00:00 2024-01-24T13:55:12.792 INFO:teuthology.orchestra.run.smithi163.stdout:(17/140): ceph-mon-16.2.14-452.g9854ba8e.el8.x8 1.8 MB/s | 5.0 MB 00:02 2024-01-24T13:55:12.909 INFO:teuthology.orchestra.run.smithi163.stdout:(18/140): librgw2-16.2.14-452.g9854ba8e.el8.x86 8.3 MB/s | 3.8 MB 00:00 2024-01-24T13:55:12.934 INFO:teuthology.orchestra.run.smithi163.stdout:(19/140): python3-ceph-argparse-16.2.14-452.g98 317 kB/s | 45 kB 00:00 2024-01-24T13:55:12.976 INFO:teuthology.orchestra.run.smithi163.stdout:(20/140): python3-ceph-common-16.2.14-452.g9854 1.4 MB/s | 99 kB 00:00 2024-01-24T13:55:13.060 INFO:teuthology.orchestra.run.smithi163.stdout:(21/140): python3-rados-16.2.14-452.g9854ba8e.e 4.6 MB/s | 389 kB 00:00 2024-01-24T13:55:13.093 INFO:teuthology.orchestra.run.smithi163.stdout:(22/140): python3-cephfs-16.2.14-452.g9854ba8e. 1.3 MB/s | 215 kB 00:00 2024-01-24T13:55:13.144 INFO:teuthology.orchestra.run.smithi163.stdout:(23/140): python3-rbd-16.2.14-452.g9854ba8e.el8 4.3 MB/s | 369 kB 00:00 2024-01-24T13:55:13.194 INFO:teuthology.orchestra.run.smithi163.stdout:(24/140): python3-rgw-16.2.14-452.g9854ba8e.el8 1.1 MB/s | 114 kB 00:00 2024-01-24T13:55:13.219 INFO:teuthology.orchestra.run.smithi163.stdout:(25/140): rbd-fuse-16.2.14-452.g9854ba8e.el8.x8 1.1 MB/s | 88 kB 00:00 2024-01-24T13:55:13.303 INFO:teuthology.orchestra.run.smithi163.stdout:(26/140): rbd-nbd-16.2.14-452.g9854ba8e.el8.x86 2.1 MB/s | 175 kB 00:00 2024-01-24T13:55:13.371 INFO:teuthology.orchestra.run.smithi163.stdout:(27/140): ceph-grafana-dashboards-16.2.14-452.g 365 kB/s | 24 kB 00:00 2024-01-24T13:55:13.437 INFO:teuthology.orchestra.run.smithi163.stdout:(28/140): ceph-mgr-cephadm-16.2.14-452.g9854ba8 1.7 MB/s | 116 kB 00:00 2024-01-24T13:55:13.837 INFO:teuthology.orchestra.run.smithi163.stdout:(29/140): ceph-mgr-dashboard-16.2.14-452.g9854b 7.8 MB/s | 3.1 MB 00:00 2024-01-24T13:55:14.304 INFO:teuthology.orchestra.run.smithi163.stdout:(30/140): ceph-test-16.2.14-452.g9854ba8e.el8.x 18 MB/s | 52 MB 00:02 2024-01-24T13:55:14.363 INFO:teuthology.orchestra.run.smithi163.stdout:(31/140): ceph-mgr-modules-core-16.2.14-452.g98 4.1 MB/s | 245 kB 00:00 2024-01-24T13:55:14.430 INFO:teuthology.orchestra.run.smithi163.stdout:(32/140): ceph-mgr-rook-16.2.14-452.g9854ba8e.e 411 kB/s | 27 kB 00:00 2024-01-24T13:55:14.505 INFO:teuthology.orchestra.run.smithi163.stdout:(33/140): ceph-prometheus-alerts-16.2.14-452.g9 198 kB/s | 15 kB 00:00 2024-01-24T13:55:14.564 INFO:teuthology.orchestra.run.smithi163.stdout:(34/140): cephadm-16.2.14-452.g9854ba8e.el8.noa 1.4 MB/s | 85 kB 00:00 2024-01-24T13:55:14.656 INFO:teuthology.orchestra.run.smithi163.stdout:(35/140): ceph-mgr-diskprediction-local-16.2.14 10 MB/s | 8.6 MB 00:00 2024-01-24T13:55:14.673 INFO:teuthology.orchestra.run.smithi163.stdout:(36/140): cephfs-top-16.2.14-452.g9854ba8e.el8. 172 kB/s | 19 kB 00:00 2024-01-24T13:55:14.882 INFO:teuthology.orchestra.run.smithi163.stdout:(37/140): rbd-mirror-16.2.14-452.g9854ba8e.el8. 2.2 MB/s | 3.7 MB 00:01 2024-01-24T13:55:15.440 INFO:teuthology.orchestra.run.smithi163.stdout:(38/140): liboath-2.6.2-3.el8.x86_64.rpm 105 kB/s | 59 kB 00:00 2024-01-24T13:55:15.457 INFO:teuthology.orchestra.run.smithi163.stdout:(39/140): leveldb-1.22-1.el8.x86_64.rpm 231 kB/s | 181 kB 00:00 2024-01-24T13:55:15.491 INFO:teuthology.orchestra.run.smithi163.stdout:(40/140): gperftools-libs-2.7-9.el8.x86_64.rpm 367 kB/s | 306 kB 00:00 2024-01-24T13:55:15.574 INFO:teuthology.orchestra.run.smithi163.stdout:(41/140): libunwind-1.3.1-3.el8.x86_64.rpm 561 kB/s | 75 kB 00:00 2024-01-24T13:55:15.591 INFO:teuthology.orchestra.run.smithi163.stdout:(42/140): python3-apipkg-1.5-6.el8.noarch.rpm 164 kB/s | 21 kB 00:00 2024-01-24T13:55:15.617 INFO:teuthology.orchestra.run.smithi163.stdout:(43/140): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 362 kB/s | 44 kB 00:00 2024-01-24T13:55:15.675 INFO:teuthology.orchestra.run.smithi163.stdout:(44/140): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2024-01-24T13:55:15.767 INFO:teuthology.orchestra.run.smithi163.stdout:(45/140): python3-certifi-2018.10.15-7.el8.noar 112 kB/s | 16 kB 00:00 2024-01-24T13:55:15.801 INFO:teuthology.orchestra.run.smithi163.stdout:(46/140): python3-cachetools-3.1.1-4.el8.noarch 161 kB/s | 33 kB 00:00 2024-01-24T13:55:15.909 INFO:teuthology.orchestra.run.smithi163.stdout:(47/140): python3-cheroot-8.5.2-1.el8.noarch.rp 739 kB/s | 173 kB 00:00 2024-01-24T13:55:15.951 INFO:teuthology.orchestra.run.smithi163.stdout:(48/140): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2024-01-24T13:55:15.977 INFO:teuthology.orchestra.run.smithi163.stdout:(49/140): python3-execnet-1.7.1-1.el8.noarch.rp 2.0 MB/s | 355 kB 00:00 2024-01-24T13:55:16.035 INFO:teuthology.orchestra.run.smithi163.stdout:(50/140): python3-google-auth-1.1.1-10.el8.noar 649 kB/s | 81 kB 00:00 2024-01-24T13:55:16.061 INFO:teuthology.orchestra.run.smithi163.stdout:(51/140): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2024-01-24T13:55:16.086 INFO:teuthology.orchestra.run.smithi163.stdout:(52/140): python3-jaraco-functools-2.0-4.el8.no 168 kB/s | 18 kB 00:00 2024-01-24T13:55:16.211 INFO:teuthology.orchestra.run.smithi163.stdout:(53/140): python3-more-itertools-7.2.0-3.el8.no 490 kB/s | 59 kB 00:00 2024-01-24T13:55:16.237 INFO:teuthology.orchestra.run.smithi163.stdout:(54/140): python3-logutils-0.3.5-11.el8.noarch. 285 kB/s | 49 kB 00:00 2024-01-24T13:55:16.312 INFO:teuthology.orchestra.run.smithi163.stdout:(55/140): python3-kubernetes-11.0.0-6.el8.noarc 4.2 MB/s | 1.2 MB 00:00 2024-01-24T13:55:16.354 INFO:teuthology.orchestra.run.smithi163.stdout:(56/140): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2024-01-24T13:55:16.379 INFO:teuthology.orchestra.run.smithi163.stdout:(57/140): python3-portend-2.6-1.el8.noarch.rpm 113 kB/s | 16 kB 00:00 2024-01-24T13:55:16.429 INFO:teuthology.orchestra.run.smithi163.stdout:(58/140): python3-remoto-1.2.1-1.el8.noarch.rpm 420 kB/s | 49 kB 00:00 2024-01-24T13:55:16.465 INFO:teuthology.orchestra.run.smithi163.stdout:(59/140): python3-repoze-lru-0.7-6.el8.noarch.r 310 kB/s | 34 kB 00:00 2024-01-24T13:55:16.514 INFO:teuthology.orchestra.run.smithi163.stdout:(60/140): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2024-01-24T13:55:16.547 INFO:teuthology.orchestra.run.smithi163.stdout:(61/140): python3-rsa-4.9-2.el8.noarch.rpm 549 kB/s | 64 kB 00:00 2024-01-24T13:55:16.573 INFO:teuthology.orchestra.run.smithi163.stdout:(62/140): python3-simplegeneric-0.8.1-17.el8.no 179 kB/s | 19 kB 00:00 2024-01-24T13:55:16.623 INFO:teuthology.orchestra.run.smithi163.stdout:(63/140): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2024-01-24T13:55:16.665 INFO:teuthology.orchestra.run.smithi163.stdout:(64/140): python3-tempora-1.14.1-5.el8.noarch.r 252 kB/s | 29 kB 00:00 2024-01-24T13:55:16.690 INFO:teuthology.orchestra.run.smithi163.stdout:(65/140): python3-trustme-0.6.0-4.el8.noarch.rp 244 kB/s | 27 kB 00:00 2024-01-24T13:55:16.799 INFO:teuthology.orchestra.run.smithi163.stdout:(66/140): python3-waitress-1.4.3-1.el8.noarch.r 1.4 MB/s | 255 kB 00:00 2024-01-24T13:55:16.824 INFO:teuthology.orchestra.run.smithi163.stdout:(67/140): python3-websocket-client-0.56.0-5.el8 475 kB/s | 61 kB 00:00 2024-01-24T13:55:16.858 INFO:teuthology.orchestra.run.smithi163.stdout:(68/140): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2024-01-24T13:55:16.892 INFO:teuthology.orchestra.run.smithi163.stdout:(69/140): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.4 MB/s | 70 kB 00:00 2024-01-24T13:55:16.941 INFO:teuthology.orchestra.run.smithi163.stdout:(70/140): python3-jsonpointer-1.10-11.el8.noarc 407 kB/s | 20 kB 00:00 2024-01-24T13:55:16.967 INFO:teuthology.orchestra.run.smithi163.stdout:(71/140): python3-webtest-2.0.33-1.el8.noarch.r 510 kB/s | 85 kB 00:00 2024-01-24T13:55:17.000 INFO:teuthology.orchestra.run.smithi163.stdout:(72/140): python3-zc-lockfile-2.0-2.el8.noarch. 134 kB/s | 23 kB 00:00 2024-01-24T13:55:17.026 INFO:teuthology.orchestra.run.smithi163.stdout:(73/140): python3-prettytable-0.7.2-14.el8.noar 528 kB/s | 44 kB 00:00 2024-01-24T13:55:17.060 INFO:teuthology.orchestra.run.smithi163.stdout:(74/140): python3-pyasn1-modules-0.3.7-6.el8.no 1.2 MB/s | 110 kB 00:00 2024-01-24T13:55:17.085 INFO:teuthology.orchestra.run.smithi163.stdout:(75/140): python3-jsonpatch-1.21-2.el8.noarch.r 338 kB/s | 27 kB 00:00 2024-01-24T13:55:17.110 INFO:teuthology.orchestra.run.smithi163.stdout:(76/140): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.6 MB/s | 126 kB 00:00 2024-01-24T13:55:17.144 INFO:teuthology.orchestra.run.smithi163.stdout:(77/140): python3-werkzeug-0.12.2-4.el8.noarch. 5.3 MB/s | 457 kB 00:00 2024-01-24T13:55:17.194 INFO:teuthology.orchestra.run.smithi163.stdout:(78/140): flex-2.6.1-9.el8.x86_64.rpm 3.7 MB/s | 320 kB 00:00 2024-01-24T13:55:17.220 INFO:teuthology.orchestra.run.smithi163.stdout:(79/140): bison-3.0.4-10.el8.x86_64.rpm 5.0 MB/s | 688 kB 00:00 2024-01-24T13:55:17.245 INFO:teuthology.orchestra.run.smithi163.stdout:(80/140): python3-markupsafe-0.23-19.el8.x86_64 771 kB/s | 39 kB 00:00 2024-01-24T13:55:17.278 INFO:teuthology.orchestra.run.smithi163.stdout:(81/140): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2024-01-24T13:55:17.304 INFO:teuthology.orchestra.run.smithi163.stdout:(82/140): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-01-24T13:55:17.320 INFO:teuthology.orchestra.run.smithi163.stdout:(83/140): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2024-01-24T13:55:17.354 INFO:teuthology.orchestra.run.smithi163.stdout:(84/140): libtool-2.4.6-25.el8.x86_64.rpm 3.3 MB/s | 709 kB 00:00 2024-01-24T13:55:17.388 INFO:teuthology.orchestra.run.smithi163.stdout:(85/140): python3-jinja2-2.10.1-3.el8.noarch.rp 7.8 MB/s | 538 kB 00:00 2024-01-24T13:55:17.438 INFO:teuthology.orchestra.run.smithi163.stdout:(86/140): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2024-01-24T13:55:17.480 INFO:teuthology.orchestra.run.smithi163.stdout:(87/140): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2024-01-24T13:55:17.522 INFO:teuthology.orchestra.run.smithi163.stdout:(88/140): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2024-01-24T13:55:17.552 INFO:teuthology.orchestra.run.smithi029.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-24T13:55:17.553 INFO:teuthology.orchestra.run.smithi029.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-24T13:55:17.576 INFO:teuthology.orchestra.run.smithi029.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-01-24T13:55:17.577 INFO:teuthology.orchestra.run.smithi029.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-01-24T13:55:17.579 INFO:teuthology.orchestra.run.smithi029.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-01-24T13:55:17.581 INFO:teuthology.orchestra.run.smithi029.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-01-24T13:55:17.581 INFO:teuthology.orchestra.run.smithi163.stdout:(89/140): python3-numpy-1.14.3-10.el8.x86_64.rp 16 MB/s | 3.7 MB 00:00 2024-01-24T13:55:17.584 INFO:teuthology.orchestra.run.smithi029.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-01-24T13:55:17.589 INFO:teuthology.orchestra.run.smithi029.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-01-24T13:55:17.590 INFO:teuthology.orchestra.run.smithi029.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-01-24T13:55:17.656 INFO:teuthology.orchestra.run.smithi163.stdout:(90/140): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2024-01-24T13:55:17.673 INFO:teuthology.orchestra.run.smithi163.stdout:(91/140): python3-mako-1.0.6-14.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2024-01-24T13:55:17.724 INFO:teuthology.orchestra.run.smithi163.stdout:(92/140): automake-1.16.1-8.el8.noarch.rpm 14 MB/s | 714 kB 00:00 2024-01-24T13:55:17.766 INFO:teuthology.orchestra.run.smithi163.stdout:(93/140): jq-1.6-7.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2024-01-24T13:55:17.799 INFO:teuthology.orchestra.run.smithi163.stdout:(94/140): python3-pytz-2017.2-11.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2024-01-24T13:55:17.826 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-01-24T13:55:17.841 INFO:teuthology.orchestra.run.smithi029.stdout:=========================================================================================================================== 2024-01-24T13:55:17.842 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-01-24T13:55:17.842 INFO:teuthology.orchestra.run.smithi029.stdout:=========================================================================================================================== 2024-01-24T13:55:17.842 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-01-24T13:55:17.842 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-01-24T13:55:17.842 INFO:teuthology.orchestra.run.smithi029.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-01-24T13:55:17.842 INFO:teuthology.orchestra.run.smithi029.stdout: ceph x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 6.4 k 2024-01-24T13:55:17.842 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 6.1 M 2024-01-24T13:55:17.842 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-fuse x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 914 k 2024-01-24T13:55:17.842 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 161 k 2024-01-24T13:55:17.842 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 1.6 M 2024-01-24T13:55:17.842 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-cephadm noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 116 k 2024-01-24T13:55:17.842 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 3.1 M 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 8.6 M 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-rook noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 27 k 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-radosgw x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 12 M 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-test x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 52 M 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: cephadm noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 85 k 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-mirror x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 257 k 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-top noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 19 k 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs-devel x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 25 k 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs2 x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 828 k 2024-01-24T13:55:17.843 INFO:teuthology.orchestra.run.smithi029.stdout: librados-devel x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 131 k 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cephfs x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 215 k 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 389 k 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rbd x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 369 k 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rgw x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 114 k 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-fuse x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 88 k 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-mirror x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 3.7 M 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-nbd x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 175 k 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout:Upgrading: 2024-01-24T13:55:17.844 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: librados2 x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 3.8 M 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: librbd1 x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 4.2 M 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-01-24T13:55:17.845 INFO:teuthology.orchestra.run.smithi029.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-common x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 24 M 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-grafana-dashboards noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 24 k 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mds x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 2.4 M 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 245 k 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mon x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 5.0 M 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-osd x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 18 M 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-prometheus-alerts noarch 2:16.2.14-452.g9854ba8e.el8 ceph-noarch 15 k 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-selinux x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 26 k 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-01-24T13:55:17.846 INFO:teuthology.orchestra.run.smithi029.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: libcephsqlite x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 174 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1 x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 494 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: librgw2 x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 3.8 M 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-01-24T13:55:17.847 INFO:teuthology.orchestra.run.smithi029.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-01-24T13:55:17.848 INFO:teuthology.orchestra.run.smithi029.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 45 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common x86_64 2:16.2.14-452.g9854ba8e.el8 ceph 99 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-01-24T13:55:17.849 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-01-24T13:55:17.850 INFO:teuthology.orchestra.run.smithi029.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-01-24T13:55:17.851 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout:=========================================================================================================================== 2024-01-24T13:55:17.852 INFO:teuthology.orchestra.run.smithi029.stdout:Install 124 Packages 2024-01-24T13:55:17.853 INFO:teuthology.orchestra.run.smithi029.stdout:Upgrade 16 Packages 2024-01-24T13:55:17.853 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:55:17.853 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 211 M 2024-01-24T13:55:17.853 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-01-24T13:55:17.883 INFO:teuthology.orchestra.run.smithi163.stdout:(95/140): openblas-threads-0.3.15-6.el8.x86_64. 14 MB/s | 5.2 MB 00:00 2024-01-24T13:55:17.958 INFO:teuthology.orchestra.run.smithi163.stdout:(96/140): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-01-24T13:55:17.975 INFO:teuthology.orchestra.run.smithi163.stdout:(97/140): libverto-devel-0.3.0-5.el8.x86_64.rpm 194 kB/s | 18 kB 00:00 2024-01-24T13:55:18.000 INFO:teuthology.orchestra.run.smithi163.stdout:(98/140): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2024-01-24T13:55:18.026 INFO:teuthology.orchestra.run.smithi163.stdout:(99/140): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2024-01-24T13:55:18.043 INFO:teuthology.orchestra.run.smithi163.stdout:(100/140): python3-pycparser-2.14-14.el8.noarch 2.5 MB/s | 109 kB 00:00 2024-01-24T13:55:18.087 INFO:teuthology.orchestra.run.smithi163.stdout:(101/140): python3-requests-oauthlib-1.0.0-1.el 696 kB/s | 43 kB 00:00 2024-01-24T13:55:18.110 INFO:teuthology.orchestra.run.smithi163.stdout:(102/140): libaio-devel-0.3.112-1.el8.x86_64.rp 279 kB/s | 19 kB 00:00 2024-01-24T13:55:18.127 INFO:teuthology.orchestra.run.smithi163.stdout:(103/140): libzstd-devel-1.4.4-1.el8.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-01-24T13:55:18.160 INFO:teuthology.orchestra.run.smithi163.stdout:(104/140): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2024-01-24T13:55:18.186 INFO:teuthology.orchestra.run.smithi163.stdout:(105/140): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2024-01-24T13:55:18.192 INFO:teuthology.orchestra.run.smithi029.stdout:(1/140): ceph-16.2.14-452.g9854ba8e.el8.x86_64. 32 kB/s | 6.4 kB 00:00 2024-01-24T13:55:18.211 INFO:teuthology.orchestra.run.smithi163.stdout:(106/140): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2024-01-24T13:55:18.236 INFO:teuthology.orchestra.run.smithi163.stdout:(107/140): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2024-01-24T13:55:18.253 INFO:teuthology.orchestra.run.smithi163.stdout:(108/140): keyutils-libs-devel-1.5.10-9.el8.x86 1.1 MB/s | 48 kB 00:00 2024-01-24T13:55:18.270 INFO:teuthology.orchestra.run.smithi163.stdout:(109/140): libsepol-devel-2.9-3.el8.x86_64.rpm 2.5 MB/s | 87 kB 00:00 2024-01-24T13:55:18.296 INFO:teuthology.orchestra.run.smithi163.stdout:(110/140): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2024-01-24T13:55:18.321 INFO:teuthology.orchestra.run.smithi163.stdout:(111/140): libcom_err-devel-1.45.6-4.el8.x86_64 767 kB/s | 39 kB 00:00 2024-01-24T13:55:18.338 INFO:teuthology.orchestra.run.smithi163.stdout:(112/140): fuse-2.9.7-17.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2024-01-24T13:55:18.372 INFO:teuthology.orchestra.run.smithi163.stdout:(113/140): libgfortran-8.5.0-20.el8.x86_64.rpm 12 MB/s | 645 kB 00:00 2024-01-24T13:55:18.389 INFO:teuthology.orchestra.run.smithi163.stdout:(114/140): libquadmath-8.5.0-20.el8.x86_64.rpm 3.4 MB/s | 172 kB 00:00 2024-01-24T13:55:18.414 INFO:teuthology.orchestra.run.smithi163.stdout:(115/140): libuuid-devel-2.32.1-43.el8.x86_64.r 2.3 MB/s | 100 kB 00:00 2024-01-24T13:55:18.440 INFO:teuthology.orchestra.run.smithi163.stdout:(116/140): python3-cffi-1.11.5-6.el8.x86_64.rpm 4.6 MB/s | 238 kB 00:00 2024-01-24T13:55:18.482 INFO:teuthology.orchestra.run.smithi163.stdout:(117/140): python3-cryptography-3.2.1-6.el8.x86 8.2 MB/s | 559 kB 00:00 2024-01-24T13:55:18.499 INFO:teuthology.orchestra.run.smithi163.stdout:(118/140): xfsprogs-devel-5.0.0-12.el8.x86_64.r 1.2 MB/s | 73 kB 00:00 2024-01-24T13:55:18.524 INFO:teuthology.orchestra.run.smithi163.stdout:(119/140): elfutils-libelf-devel-0.189-3.el8.x8 1.4 MB/s | 62 kB 00:00 2024-01-24T13:55:18.549 INFO:teuthology.orchestra.run.smithi163.stdout:(120/140): librabbitmq-0.9.0-4.el8.x86_64.rpm 935 kB/s | 47 kB 00:00 2024-01-24T13:55:18.583 INFO:teuthology.orchestra.run.smithi163.stdout:(121/140): krb5-devel-1.18.2-26.el8_9.x86_64.rp 9.4 MB/s | 562 kB 00:00 2024-01-24T13:55:18.625 INFO:teuthology.orchestra.run.smithi163.stdout:(122/140): NetworkManager-initscripts-updown-1. 3.4 MB/s | 144 kB 00:00 2024-01-24T13:55:18.684 INFO:teuthology.orchestra.run.smithi163.stdout:(123/140): openssl-devel-1.1.1k-12.el8_9.x86_64 17 MB/s | 2.3 MB 00:00 2024-01-24T13:55:18.759 INFO:teuthology.orchestra.run.smithi029.stdout:(2/140): ceph-base-16.2.14-452.g9854ba8e.el8.x8 7.9 MB/s | 6.1 MB 00:00 2024-01-24T13:55:18.834 INFO:teuthology.orchestra.run.smithi029.stdout:(3/140): ceph-immutable-object-cache-16.2.14-45 2.1 MB/s | 161 kB 00:00 2024-01-24T13:55:18.876 INFO:teuthology.orchestra.run.smithi163.stdout:(124/140): python3-scipy-1.0.0-21.module+el8.9. 13 MB/s | 14 MB 00:01 2024-01-24T13:55:18.911 INFO:teuthology.orchestra.run.smithi163.stdout:(125/140): elfutils-libelf-0.189-3.el8.x86_64.r 6.8 MB/s | 232 kB 00:00 2024-01-24T13:55:18.973 INFO:teuthology.orchestra.run.smithi163.stdout:(126/140): elfutils-libs-0.189-3.el8.x86_64.rpm 4.7 MB/s | 303 kB 00:00 2024-01-24T13:55:19.018 INFO:teuthology.orchestra.run.smithi163.stdout:(127/140): libmount-2.32.1-43.el8.x86_64.rpm 5.1 MB/s | 237 kB 00:00 2024-01-24T13:55:19.052 INFO:teuthology.orchestra.run.smithi163.stdout:(128/140): libsmartcols-2.32.1-43.el8.x86_64.rp 5.3 MB/s | 179 kB 00:00 2024-01-24T13:55:19.085 INFO:teuthology.orchestra.run.smithi163.stdout:(129/140): libuuid-2.32.1-43.el8.x86_64.rpm 2.9 MB/s | 99 kB 00:00 2024-01-24T13:55:19.152 INFO:teuthology.orchestra.run.smithi029.stdout:(4/140): ceph-fuse-16.2.14-452.g9854ba8e.el8.x8 953 kB/s | 914 kB 00:00 2024-01-24T13:55:19.169 INFO:teuthology.orchestra.run.smithi163.stdout:(130/140): util-linux-2.32.1-43.el8.x86_64.rpm 30 MB/s | 2.5 MB 00:00 2024-01-24T13:55:19.203 INFO:teuthology.orchestra.run.smithi163.stdout:(131/140): util-linux-user-2.32.1-43.el8.x86_64 3.0 MB/s | 103 kB 00:00 2024-01-24T13:55:19.210 INFO:teuthology.orchestra.run.smithi029.stdout:(5/140): ceph-mds-16.2.14-452.g9854ba8e.el8.x86 6.4 MB/s | 2.4 MB 00:00 2024-01-24T13:55:19.261 INFO:teuthology.orchestra.run.smithi163.stdout:(132/140): iproute-6.2.0-5.el8_9.x86_64.rpm 14 MB/s | 854 kB 00:00 2024-01-24T13:55:19.303 INFO:teuthology.orchestra.run.smithi163.stdout:(133/140): libblkid-2.32.1-43.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2024-01-24T13:55:19.346 INFO:teuthology.orchestra.run.smithi163.stdout:(134/140): libfdisk-2.32.1-43.el8.x86_64.rpm 6.0 MB/s | 254 kB 00:00 2024-01-24T13:55:19.396 INFO:teuthology.orchestra.run.smithi163.stdout:(135/140): xfsprogs-5.0.0-12.el8.x86_64.rpm 21 MB/s | 1.1 MB 00:00 2024-01-24T13:55:19.438 INFO:teuthology.orchestra.run.smithi163.stdout:(136/140): openssl-1.1.1k-12.el8_9.x86_64.rpm 17 MB/s | 711 kB 00:00 2024-01-24T13:55:19.496 INFO:teuthology.orchestra.run.smithi163.stdout:(137/140): openssl-libs-1.1.1k-12.el8_9.x86_64. 25 MB/s | 1.5 MB 00:00 2024-01-24T13:55:19.538 INFO:teuthology.orchestra.run.smithi163.stdout:(138/140): xfsdump-3.1.8-7.el8_9.x86_64.rpm 7.8 MB/s | 333 kB 00:00 2024-01-24T13:55:19.906 INFO:teuthology.orchestra.run.smithi163.stdout:(139/140): librbd1-16.2.14-452.g9854ba8e.el8.x8 3.4 MB/s | 4.2 MB 00:01 2024-01-24T13:55:20.059 INFO:teuthology.orchestra.run.smithi029.stdout:(6/140): ceph-common-16.2.14-452.g9854ba8e.el8. 12 MB/s | 24 MB 00:02 2024-01-24T13:55:20.120 INFO:teuthology.orchestra.run.smithi029.stdout:(7/140): ceph-mgr-16.2.14-452.g9854ba8e.el8.x86 1.7 MB/s | 1.6 MB 00:00 2024-01-24T13:55:20.229 INFO:teuthology.orchestra.run.smithi029.stdout:(8/140): ceph-mon-16.2.14-452.g9854ba8e.el8.x86 4.9 MB/s | 5.0 MB 00:01 2024-01-24T13:55:20.304 INFO:teuthology.orchestra.run.smithi029.stdout:(9/140): ceph-selinux-16.2.14-452.g9854ba8e.el8 342 kB/s | 26 kB 00:00 2024-01-24T13:55:20.356 INFO:teuthology.orchestra.run.smithi163.stdout:(140/140): librados2-16.2.14-452.g9854ba8e.el8. 2.2 MB/s | 3.8 MB 00:01 2024-01-24T13:55:20.359 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-01-24T13:55:20.359 INFO:teuthology.orchestra.run.smithi163.stdout:Total 18 MB/s | 211 MB 00:11 2024-01-24T13:55:21.341 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-01-24T13:55:21.506 INFO:teuthology.orchestra.run.smithi029.stdout:(10/140): ceph-osd-16.2.14-452.g9854ba8e.el8.x8 13 MB/s | 18 MB 00:01 2024-01-24T13:55:21.576 INFO:teuthology.orchestra.run.smithi029.stdout:(11/140): cephfs-mirror-16.2.14-452.g9854ba8e.e 3.8 MB/s | 257 kB 00:00 2024-01-24T13:55:21.649 INFO:teuthology.orchestra.run.smithi029.stdout:(12/140): libcephfs-devel-16.2.14-452.g9854ba8e 340 kB/s | 25 kB 00:00 2024-01-24T13:55:21.736 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-01-24T13:55:21.736 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-01-24T13:55:21.816 INFO:teuthology.orchestra.run.smithi029.stdout:(13/140): libcephfs2-16.2.14-452.g9854ba8e.el8. 4.9 MB/s | 828 kB 00:00 2024-01-24T13:55:21.891 INFO:teuthology.orchestra.run.smithi029.stdout:(14/140): libcephsqlite-16.2.14-452.g9854ba8e.e 2.3 MB/s | 174 kB 00:00 2024-01-24T13:55:21.975 INFO:teuthology.orchestra.run.smithi029.stdout:(15/140): librados-devel-16.2.14-452.g9854ba8e. 1.5 MB/s | 131 kB 00:00 2024-01-24T13:55:22.068 INFO:teuthology.orchestra.run.smithi029.stdout:(16/140): libradosstriper1-16.2.14-452.g9854ba8 5.3 MB/s | 494 kB 00:00 2024-01-24T13:55:22.535 INFO:teuthology.orchestra.run.smithi029.stdout:(17/140): ceph-radosgw-16.2.14-452.g9854ba8e.el 4.8 MB/s | 12 MB 00:02 2024-01-24T13:55:22.800 INFO:teuthology.orchestra.run.smithi029.stdout:(18/140): librgw2-16.2.14-452.g9854ba8e.el8.x86 5.2 MB/s | 3.8 MB 00:00 2024-01-24T13:55:22.836 INFO:teuthology.orchestra.run.smithi029.stdout:(19/140): python3-ceph-argparse-16.2.14-452.g98 150 kB/s | 45 kB 00:00 2024-01-24T13:55:22.870 INFO:teuthology.orchestra.run.smithi029.stdout:(20/140): python3-ceph-common-16.2.14-452.g9854 1.3 MB/s | 99 kB 00:00 2024-01-24T13:55:22.900 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-01-24T13:55:22.908 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-01-24T13:55:22.946 INFO:teuthology.orchestra.run.smithi029.stdout:(21/140): python3-rados-16.2.14-452.g9854ba8e.e 5.1 MB/s | 389 kB 00:00 2024-01-24T13:55:22.979 INFO:teuthology.orchestra.run.smithi029.stdout:(22/140): python3-cephfs-16.2.14-452.g9854ba8e. 1.5 MB/s | 215 kB 00:00 2024-01-24T13:55:23.013 INFO:teuthology.orchestra.run.smithi029.stdout:(23/140): python3-rbd-16.2.14-452.g9854ba8e.el8 5.4 MB/s | 369 kB 00:00 2024-01-24T13:55:23.055 INFO:teuthology.orchestra.run.smithi029.stdout:(24/140): python3-rgw-16.2.14-452.g9854ba8e.el8 1.5 MB/s | 114 kB 00:00 2024-01-24T13:55:23.082 INFO:teuthology.orchestra.run.smithi029.stdout:(25/140): rbd-fuse-16.2.14-452.g9854ba8e.el8.x8 1.3 MB/s | 88 kB 00:00 2024-01-24T13:55:23.150 INFO:teuthology.orchestra.run.smithi029.stdout:(26/140): rbd-nbd-16.2.14-452.g9854ba8e.el8.x86 2.6 MB/s | 175 kB 00:00 2024-01-24T13:55:23.200 INFO:teuthology.orchestra.run.smithi029.stdout:(27/140): ceph-grafana-dashboards-16.2.14-452.g 488 kB/s | 24 kB 00:00 2024-01-24T13:55:23.267 INFO:teuthology.orchestra.run.smithi029.stdout:(28/140): ceph-mgr-cephadm-16.2.14-452.g9854ba8 1.7 MB/s | 116 kB 00:00 2024-01-24T13:55:23.643 INFO:teuthology.orchestra.run.smithi029.stdout:(29/140): rbd-mirror-16.2.14-452.g9854ba8e.el8. 6.3 MB/s | 3.7 MB 00:00 2024-01-24T13:55:23.718 INFO:teuthology.orchestra.run.smithi029.stdout:(30/140): ceph-mgr-dashboard-16.2.14-452.g9854b 6.9 MB/s | 3.1 MB 00:00 2024-01-24T13:55:23.794 INFO:teuthology.orchestra.run.smithi029.stdout:(31/140): ceph-mgr-modules-core-16.2.14-452.g98 3.2 MB/s | 245 kB 00:00 2024-01-24T13:55:23.861 INFO:teuthology.orchestra.run.smithi029.stdout:(32/140): ceph-mgr-rook-16.2.14-452.g9854ba8e.e 410 kB/s | 27 kB 00:00 2024-01-24T13:55:23.920 INFO:teuthology.orchestra.run.smithi029.stdout:(33/140): ceph-prometheus-alerts-16.2.14-452.g9 254 kB/s | 15 kB 00:00 2024-01-24T13:55:23.987 INFO:teuthology.orchestra.run.smithi029.stdout:(34/140): cephadm-16.2.14-452.g9854ba8e.el8.noa 1.3 MB/s | 85 kB 00:00 2024-01-24T13:55:24.088 INFO:teuthology.orchestra.run.smithi029.stdout:(35/140): cephfs-top-16.2.14-452.g9854ba8e.el8. 187 kB/s | 19 kB 00:00 2024-01-24T13:55:25.163 INFO:teuthology.orchestra.run.smithi029.stdout:(36/140): ceph-test-16.2.14-452.g9854ba8e.el8.x 11 MB/s | 52 MB 00:04 2024-01-24T13:55:25.331 INFO:teuthology.orchestra.run.smithi029.stdout:(37/140): ceph-mgr-diskprediction-local-16.2.14 5.1 MB/s | 8.6 MB 00:01 2024-01-24T13:55:25.356 INFO:teuthology.orchestra.run.smithi029.stdout:(38/140): gperftools-libs-2.7-9.el8.x86_64.rpm 241 kB/s | 306 kB 00:01 2024-01-24T13:55:25.523 INFO:teuthology.orchestra.run.smithi029.stdout:(39/140): libunwind-1.3.1-3.el8.x86_64.rpm 450 kB/s | 75 kB 00:00 2024-01-24T13:55:25.650 INFO:teuthology.orchestra.run.smithi029.stdout:(40/140): python3-apipkg-1.5-6.el8.noarch.rpm 167 kB/s | 21 kB 00:00 2024-01-24T13:55:25.676 INFO:teuthology.orchestra.run.smithi029.stdout:(41/140): liboath-2.6.2-3.el8.x86_64.rpm 171 kB/s | 59 kB 00:00 2024-01-24T13:55:25.735 INFO:teuthology.orchestra.run.smithi029.stdout:(42/140): leveldb-1.22-1.el8.x86_64.rpm 317 kB/s | 181 kB 00:00 2024-01-24T13:55:25.794 INFO:teuthology.orchestra.run.smithi029.stdout:(43/140): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 305 kB/s | 44 kB 00:00 2024-01-24T13:55:25.844 INFO:teuthology.orchestra.run.smithi029.stdout:(44/140): python3-cachetools-3.1.1-4.el8.noarch 302 kB/s | 33 kB 00:00 2024-01-24T13:55:25.911 INFO:teuthology.orchestra.run.smithi029.stdout:(45/140): python3-certifi-2018.10.15-7.el8.noar 139 kB/s | 16 kB 00:00 2024-01-24T13:55:25.937 INFO:teuthology.orchestra.run.smithi029.stdout:(46/140): python3-beautifulsoup4-4.6.3-2.el8.1. 710 kB/s | 185 kB 00:00 2024-01-24T13:55:25.996 INFO:teuthology.orchestra.run.smithi029.stdout:(47/140): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2024-01-24T13:55:26.121 INFO:teuthology.orchestra.run.smithi029.stdout:(48/140): python3-google-auth-1.1.1-10.el8.noar 650 kB/s | 81 kB 00:00 2024-01-24T13:55:26.194 INFO:teuthology.orchestra.run.smithi029.stdout:(49/140): python3-execnet-1.7.1-1.el8.noarch.rp 1.4 MB/s | 355 kB 00:00 2024-01-24T13:55:26.231 INFO:teuthology.orchestra.run.smithi029.stdout:(50/140): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2024-01-24T13:55:26.290 INFO:teuthology.orchestra.run.smithi029.stdout:(51/140): python3-jaraco-functools-2.0-4.el8.no 185 kB/s | 18 kB 00:00 2024-01-24T13:55:26.323 INFO:teuthology.orchestra.run.smithi029.stdout:(52/140): python3-cherrypy-18.4.0-1.el8.noarch. 934 kB/s | 384 kB 00:00 2024-01-24T13:55:26.415 INFO:teuthology.orchestra.run.smithi029.stdout:(53/140): python3-logutils-0.3.5-11.el8.noarch. 388 kB/s | 49 kB 00:00 2024-01-24T13:55:26.441 INFO:teuthology.orchestra.run.smithi029.stdout:(54/140): python3-more-itertools-7.2.0-3.el8.no 500 kB/s | 59 kB 00:00 2024-01-24T13:55:26.533 INFO:teuthology.orchestra.run.smithi029.stdout:(55/140): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2024-01-24T13:55:26.577 INFO:teuthology.orchestra.run.smithi029.stdout:(56/140): python3-kubernetes-11.0.0-6.el8.noarc 3.4 MB/s | 1.2 MB 00:00 2024-01-24T13:55:26.636 INFO:teuthology.orchestra.run.smithi029.stdout:(57/140): python3-remoto-1.2.1-1.el8.noarch.rpm 477 kB/s | 49 kB 00:00 2024-01-24T13:55:26.678 INFO:teuthology.orchestra.run.smithi029.stdout:(58/140): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2024-01-24T13:55:26.704 INFO:teuthology.orchestra.run.smithi029.stdout:(59/140): python3-repoze-lru-0.7-6.el8.noarch.r 266 kB/s | 34 kB 00:00 2024-01-24T13:55:26.779 INFO:teuthology.orchestra.run.smithi029.stdout:(60/140): python3-routes-2.4.1-12.el8.noarch.rp 1.3 MB/s | 196 kB 00:00 2024-01-24T13:55:26.805 INFO:teuthology.orchestra.run.smithi029.stdout:(61/140): python3-simplegeneric-0.8.1-17.el8.no 185 kB/s | 19 kB 00:00 2024-01-24T13:55:26.830 INFO:teuthology.orchestra.run.smithi029.stdout:(62/140): python3-rsa-4.9-2.el8.noarch.rpm 423 kB/s | 64 kB 00:00 2024-01-24T13:55:26.881 INFO:teuthology.orchestra.run.smithi029.stdout:(63/140): python3-singledispatch-3.4.0.3-18.el8 236 kB/s | 24 kB 00:00 2024-01-24T13:55:26.906 INFO:teuthology.orchestra.run.smithi029.stdout:(64/140): python3-tempora-1.14.1-5.el8.noarch.r 289 kB/s | 29 kB 00:00 2024-01-24T13:55:26.940 INFO:teuthology.orchestra.run.smithi029.stdout:(65/140): python3-trustme-0.6.0-4.el8.noarch.rp 251 kB/s | 27 kB 00:00 2024-01-24T13:55:27.007 INFO:teuthology.orchestra.run.smithi029.stdout:(66/140): python3-waitress-1.4.3-1.el8.noarch.r 2.0 MB/s | 255 kB 00:00 2024-01-24T13:55:27.033 INFO:teuthology.orchestra.run.smithi029.stdout:(67/140): python3-webob-1.8.5-1.el8.1.noarch.rp 1.9 MB/s | 251 kB 00:00 2024-01-24T13:55:27.067 INFO:teuthology.orchestra.run.smithi029.stdout:(68/140): python3-websocket-client-0.56.0-5.el8 484 kB/s | 61 kB 00:00 2024-01-24T13:55:27.092 INFO:teuthology.orchestra.run.smithi029.stdout:(69/140): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-01-24T13:55:27.118 INFO:teuthology.orchestra.run.smithi029.stdout:(70/140): python3-webtest-2.0.33-1.el8.noarch.r 775 kB/s | 85 kB 00:00 2024-01-24T13:55:27.145 INFO:teuthology.orchestra.run.smithi029.stdout:(71/140): python3-zc-lockfile-2.0-2.el8.noarch. 205 kB/s | 23 kB 00:00 2024-01-24T13:55:27.196 INFO:teuthology.orchestra.run.smithi029.stdout:(72/140): python3-jsonpointer-1.10-11.el8.noarc 194 kB/s | 20 kB 00:00 2024-01-24T13:55:27.229 INFO:teuthology.orchestra.run.smithi029.stdout:(73/140): python3-prettytable-0.7.2-14.el8.noar 396 kB/s | 44 kB 00:00 2024-01-24T13:55:27.255 INFO:teuthology.orchestra.run.smithi029.stdout:(74/140): python3-jsonpatch-1.21-2.el8.noarch.r 527 kB/s | 27 kB 00:00 2024-01-24T13:55:27.280 INFO:teuthology.orchestra.run.smithi029.stdout:(75/140): python3-pyasn1-modules-0.3.7-6.el8.no 936 kB/s | 110 kB 00:00 2024-01-24T13:55:27.339 INFO:teuthology.orchestra.run.smithi029.stdout:(76/140): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2024-01-24T13:55:27.373 INFO:teuthology.orchestra.run.smithi029.stdout:(77/140): python3-werkzeug-0.12.2-4.el8.noarch. 3.8 MB/s | 457 kB 00:00 2024-01-24T13:55:27.415 INFO:teuthology.orchestra.run.smithi029.stdout:(78/140): bison-3.0.4-10.el8.x86_64.rpm 5.0 MB/s | 688 kB 00:00 2024-01-24T13:55:27.449 INFO:teuthology.orchestra.run.smithi029.stdout:(79/140): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2024-01-24T13:55:27.475 INFO:teuthology.orchestra.run.smithi029.stdout:(80/140): python3-markupsafe-0.23-19.el8.x86_64 658 kB/s | 39 kB 00:00 2024-01-24T13:55:27.500 INFO:teuthology.orchestra.run.smithi029.stdout:(81/140): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2024-01-24T13:55:27.526 INFO:teuthology.orchestra.run.smithi029.stdout:(82/140): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2024-01-24T13:55:27.559 INFO:teuthology.orchestra.run.smithi029.stdout:(83/140): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.7 MB/s | 103 kB 00:00 2024-01-24T13:55:27.593 INFO:teuthology.orchestra.run.smithi029.stdout:(84/140): libtool-2.4.6-25.el8.x86_64.rpm 3.1 MB/s | 709 kB 00:00 2024-01-24T13:55:27.635 INFO:teuthology.orchestra.run.smithi029.stdout:(85/140): python3-jinja2-2.10.1-3.el8.noarch.rp 6.9 MB/s | 538 kB 00:00 2024-01-24T13:55:27.686 INFO:teuthology.orchestra.run.smithi029.stdout:(86/140): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2024-01-24T13:55:27.896 INFO:teuthology.orchestra.run.smithi029.stdout:(87/140): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.0 MB/s | 225 kB 00:00 2024-01-24T13:55:28.227 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-01-24T13:55:28.363 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-01-24T13:55:28.413 INFO:teuthology.orchestra.run.smithi029.stdout:(88/140): librdkafka-0.11.4-3.el8.x86_64.rpm 684 kB/s | 354 kB 00:00 2024-01-24T13:55:28.592 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2024-01-24T13:55:28.669 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2024-01-24T13:55:28.806 INFO:teuthology.orchestra.run.smithi029.stdout:(89/140): python3-numpy-1.14.3-10.el8.x86_64.rp 3.1 MB/s | 3.7 MB 00:01 2024-01-24T13:55:28.835 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 2/156 2024-01-24T13:55:28.907 INFO:teuthology.orchestra.run.smithi029.stdout:(90/140): python3-babel-2.5.1-7.el8.noarch.rpm 3.5 MB/s | 4.8 MB 00:01 2024-01-24T13:55:28.932 INFO:teuthology.orchestra.run.smithi029.stdout:(91/140): python3-mako-1.0.6-14.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2024-01-24T13:55:28.948 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 2/156 2024-01-24T13:55:29.000 INFO:teuthology.orchestra.run.smithi029.stdout:(92/140): automake-1.16.1-8.el8.noarch.rpm 10 MB/s | 714 kB 00:00 2024-01-24T13:55:29.042 INFO:teuthology.orchestra.run.smithi029.stdout:(93/140): jq-1.6-7.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2024-01-24T13:55:29.084 INFO:teuthology.orchestra.run.smithi029.stdout:(94/140): python3-pytz-2017.2-11.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2024-01-24T13:55:29.119 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 3/156 2024-01-24T13:55:29.192 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 3/156 2024-01-24T13:55:29.209 INFO:teuthology.orchestra.run.smithi029.stdout:(95/140): openblas-threads-0.3.15-6.el8.x86_64. 6.5 MB/s | 5.2 MB 00:00 2024-01-24T13:55:29.368 INFO:teuthology.orchestra.run.smithi029.stdout:(96/140): openblas-0.3.15-6.el8.x86_64.rpm 11 MB/s | 5.0 MB 00:00 2024-01-24T13:55:29.377 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/156 2024-01-24T13:55:29.385 INFO:teuthology.orchestra.run.smithi029.stdout:(97/140): libverto-devel-0.3.0-5.el8.x86_64.rpm 102 kB/s | 18 kB 00:00 2024-01-24T13:55:29.411 INFO:teuthology.orchestra.run.smithi029.stdout:(98/140): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2024-01-24T13:55:29.437 INFO:teuthology.orchestra.run.smithi029.stdout:(99/140): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2024-01-24T13:55:29.470 INFO:teuthology.orchestra.run.smithi029.stdout:(100/140): python3-pycparser-2.14-14.el8.noarch 1.8 MB/s | 109 kB 00:00 2024-01-24T13:55:29.497 INFO:teuthology.orchestra.run.smithi029.stdout:(101/140): python3-requests-oauthlib-1.0.0-1.el 711 kB/s | 43 kB 00:00 2024-01-24T13:55:29.523 INFO:teuthology.orchestra.run.smithi029.stdout:(102/140): libaio-devel-0.3.112-1.el8.x86_64.rp 357 kB/s | 19 kB 00:00 2024-01-24T13:55:29.548 INFO:teuthology.orchestra.run.smithi029.stdout:(103/140): libzstd-devel-1.4.4-1.el8.x86_64.rpm 864 kB/s | 44 kB 00:00 2024-01-24T13:55:29.593 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : leveldb-1.22-1.el8.x86_64 5/156 2024-01-24T13:55:29.599 INFO:teuthology.orchestra.run.smithi029.stdout:(104/140): pcre2-devel-10.32-2.el8.x86_64.rpm 7.8 MB/s | 605 kB 00:00 2024-01-24T13:55:29.633 INFO:teuthology.orchestra.run.smithi029.stdout:(105/140): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2024-01-24T13:55:29.659 INFO:teuthology.orchestra.run.smithi029.stdout:(106/140): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2024-01-24T13:55:29.684 INFO:teuthology.orchestra.run.smithi029.stdout:(107/140): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2024-01-24T13:55:29.710 INFO:teuthology.orchestra.run.smithi029.stdout:(108/140): keyutils-libs-devel-1.5.10-9.el8.x86 949 kB/s | 48 kB 00:00 2024-01-24T13:55:29.744 INFO:teuthology.orchestra.run.smithi029.stdout:(109/140): libsepol-devel-2.9-3.el8.x86_64.rpm 1.4 MB/s | 87 kB 00:00 2024-01-24T13:55:29.769 INFO:teuthology.orchestra.run.smithi029.stdout:(110/140): zlib-devel-1.2.11-18.el8_5.x86_64.rp 977 kB/s | 58 kB 00:00 2024-01-24T13:55:29.795 INFO:teuthology.orchestra.run.smithi029.stdout:(111/140): libcom_err-devel-1.45.6-4.el8.x86_64 761 kB/s | 39 kB 00:00 2024-01-24T13:55:29.802 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/156 2024-01-24T13:55:29.820 INFO:teuthology.orchestra.run.smithi029.stdout:(112/140): fuse-2.9.7-17.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2024-01-24T13:55:29.877 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/156 2024-01-24T13:55:29.921 INFO:teuthology.orchestra.run.smithi029.stdout:(113/140): libgfortran-8.5.0-20.el8.x86_64.rpm 5.0 MB/s | 645 kB 00:00 2024-01-24T13:55:29.946 INFO:teuthology.orchestra.run.smithi029.stdout:(114/140): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-01-24T13:55:29.972 INFO:teuthology.orchestra.run.smithi029.stdout:(115/140): libuuid-devel-2.32.1-43.el8.x86_64.r 1.9 MB/s | 100 kB 00:00 2024-01-24T13:55:30.008 INFO:teuthology.orchestra.run.smithi029.stdout:(116/140): python3-cffi-1.11.5-6.el8.x86_64.rpm 3.8 MB/s | 238 kB 00:00 2024-01-24T13:55:30.050 INFO:teuthology.orchestra.run.smithi029.stdout:(117/140): python3-cryptography-3.2.1-6.el8.x86 7.1 MB/s | 559 kB 00:00 2024-01-24T13:55:30.070 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/156 2024-01-24T13:55:30.075 INFO:teuthology.orchestra.run.smithi029.stdout:(118/140): xfsprogs-devel-5.0.0-12.el8.x86_64.r 1.1 MB/s | 73 kB 00:00 2024-01-24T13:55:30.101 INFO:teuthology.orchestra.run.smithi029.stdout:(119/140): elfutils-libelf-devel-0.189-3.el8.x8 1.2 MB/s | 62 kB 00:00 2024-01-24T13:55:30.126 INFO:teuthology.orchestra.run.smithi029.stdout:(120/140): librabbitmq-0.9.0-4.el8.x86_64.rpm 929 kB/s | 47 kB 00:00 2024-01-24T13:55:30.169 INFO:teuthology.orchestra.run.smithi029.stdout:(121/140): krb5-devel-1.18.2-26.el8_9.x86_64.rp 8.2 MB/s | 562 kB 00:00 2024-01-24T13:55:30.211 INFO:teuthology.orchestra.run.smithi029.stdout:(122/140): NetworkManager-initscripts-updown-1. 3.4 MB/s | 144 kB 00:00 2024-01-24T13:55:30.260 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/156 2024-01-24T13:55:30.295 INFO:teuthology.orchestra.run.smithi029.stdout:(123/140): openssl-devel-1.1.1k-12.el8_9.x86_64 14 MB/s | 2.3 MB 00:00 2024-01-24T13:55:30.653 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/156 2024-01-24T13:55:30.662 INFO:teuthology.orchestra.run.smithi029.stdout:(124/140): python3-scipy-1.0.0-21.module+el8.9. 8.7 MB/s | 14 MB 00:01 2024-01-24T13:55:30.713 INFO:teuthology.orchestra.run.smithi029.stdout:(125/140): elfutils-libelf-0.189-3.el8.x86_64.r 4.5 MB/s | 232 kB 00:00 2024-01-24T13:55:30.763 INFO:teuthology.orchestra.run.smithi029.stdout:(126/140): elfutils-libs-0.189-3.el8.x86_64.rpm 5.9 MB/s | 303 kB 00:00 2024-01-24T13:55:30.805 INFO:teuthology.orchestra.run.smithi029.stdout:(127/140): libmount-2.32.1-43.el8.x86_64.rpm 5.5 MB/s | 237 kB 00:00 2024-01-24T13:55:30.869 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librados2-2:16.2.14-452.g9854ba8e.el8.x86_64 9/156 2024-01-24T13:55:30.906 INFO:teuthology.orchestra.run.smithi029.stdout:(128/140): librbd1-16.2.14-452.g9854ba8e.el8.x8 6.8 MB/s | 4.2 MB 00:00 2024-01-24T13:55:30.998 INFO:teuthology.orchestra.run.smithi029.stdout:(129/140): librados2-16.2.14-452.g9854ba8e.el8. 4.8 MB/s | 3.8 MB 00:00 2024-01-24T13:55:31.024 INFO:teuthology.orchestra.run.smithi029.stdout:(130/140): libsmartcols-2.32.1-43.el8.x86_64.rp 823 kB/s | 179 kB 00:00 2024-01-24T13:55:31.050 INFO:teuthology.orchestra.run.smithi029.stdout:(131/140): libuuid-2.32.1-43.el8.x86_64.rpm 685 kB/s | 99 kB 00:00 2024-01-24T13:55:31.084 INFO:teuthology.orchestra.run.smithi029.stdout:(132/140): util-linux-user-2.32.1-43.el8.x86_64 1.7 MB/s | 103 kB 00:00 2024-01-24T13:55:31.179 INFO:teuthology.orchestra.run.smithi029.stdout:(133/140): iproute-6.2.0-5.el8_9.x86_64.rpm 6.5 MB/s | 854 kB 00:00 2024-01-24T13:55:31.219 INFO:teuthology.orchestra.run.smithi029.stdout:(134/140): libblkid-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 220 kB 00:00 2024-01-24T13:55:31.253 INFO:teuthology.orchestra.run.smithi029.stdout:(135/140): libfdisk-2.32.1-43.el8.x86_64.rpm 3.4 MB/s | 254 kB 00:00 2024-01-24T13:55:31.268 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:16.2.14-452.g9854ba8e.el8.x86_64 9/156 2024-01-24T13:55:31.311 INFO:teuthology.orchestra.run.smithi029.stdout:(136/140): xfsprogs-5.0.0-12.el8.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-01-24T13:55:31.379 INFO:teuthology.orchestra.run.smithi029.stdout:(137/140): util-linux-2.32.1-43.el8.x86_64.rpm 6.5 MB/s | 2.5 MB 00:00 2024-01-24T13:55:31.421 INFO:teuthology.orchestra.run.smithi029.stdout:(138/140): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.1 MB/s | 711 kB 00:00 2024-01-24T13:55:31.455 INFO:teuthology.orchestra.run.smithi029.stdout:(139/140): xfsdump-3.1.8-7.el8_9.x86_64.rpm 4.3 MB/s | 333 kB 00:00 2024-01-24T13:55:31.469 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librbd1-2:16.2.14-452.g9854ba8e.el8.x86_64 10/156 2024-01-24T13:55:31.513 INFO:teuthology.orchestra.run.smithi029.stdout:(140/140): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.3 MB/s | 1.5 MB 00:00 2024-01-24T13:55:31.517 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-01-24T13:55:31.517 INFO:teuthology.orchestra.run.smithi029.stdout:Total 15 MB/s | 211 MB 00:13 2024-01-24T13:55:31.594 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:16.2.14-452.g9854ba8e.el8.x86_64 10/156 2024-01-24T13:55:31.977 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs2-2:16.2.14-452.g9854ba8e.el8.x86_64 11/156 2024-01-24T13:55:32.074 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs2-2:16.2.14-452.g9854ba8e.el8.x86_64 11/156 2024-01-24T13:55:32.533 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-01-24T13:55:32.534 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rados-2:16.2.14-452.g9854ba8e.el8.x86_64 12/156 2024-01-24T13:55:32.660 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librgw2-2:16.2.14-452.g9854ba8e.el8.x86_64 13/156 2024-01-24T13:55:32.844 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librgw2-2:16.2.14-452.g9854ba8e.el8.x86_64 13/156 2024-01-24T13:55:32.979 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-01-24T13:55:32.979 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-01-24T13:55:33.060 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/156 2024-01-24T13:55:33.149 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/156 2024-01-24T13:55:33.372 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/156 2024-01-24T13:55:33.563 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/156 2024-01-24T13:55:33.708 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/156 2024-01-24T13:55:33.836 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephsqlite-2:16.2.14-452.g9854ba8e.el8.x86_64 18/156 2024-01-24T13:55:33.928 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephsqlite-2:16.2.14-452.g9854ba8e.el8.x86_64 18/156 2024-01-24T13:55:34.051 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libradosstriper1-2:16.2.14-452.g9854ba8e.el8.x86 19/156 2024-01-24T13:55:34.117 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libradosstriper1-2:16.2.14-452.g9854ba8e.el8.x86 19/156 2024-01-24T13:55:34.266 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-01-24T13:55:34.277 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-01-24T13:55:34.310 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/156 2024-01-24T13:55:34.440 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/156 2024-01-24T13:55:34.604 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/156 2024-01-24T13:55:35.227 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/156 2024-01-24T13:55:35.393 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 22/156 2024-01-24T13:55:36.001 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 22/156 2024-01-24T13:55:36.218 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-0.3.15-6.el8.x86_64 23/156 2024-01-24T13:55:36.911 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 23/156 2024-01-24T13:55:37.174 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/156 2024-01-24T13:55:37.407 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/156 2024-01-24T13:55:37.564 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 26/156 2024-01-24T13:55:37.831 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 27/156 2024-01-24T13:55:38.116 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 28/156 2024-01-24T13:55:38.372 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 29/156 2024-01-24T13:55:38.532 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-common-2:16.2.14-452.g9854ba8e.el8. 30/156 2024-01-24T13:55:38.672 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-argparse-2:16.2.14-452.g9854ba8e.el 31/156 2024-01-24T13:55:38.937 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cephfs-2:16.2.14-452.g9854ba8e.el8.x86_6 32/156 2024-01-24T13:55:39.898 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/156 2024-01-24T13:55:39.947 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-01-24T13:55:40.097 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-01-24T13:55:40.199 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/156 2024-01-24T13:55:40.434 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2024-01-24T13:55:40.497 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/156 2024-01-24T13:55:40.528 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2024-01-24T13:55:40.682 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 2/156 2024-01-24T13:55:40.700 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/156 2024-01-24T13:55:40.763 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 2/156 2024-01-24T13:55:41.007 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 3/156 2024-01-24T13:55:41.066 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 3/156 2024-01-24T13:55:41.268 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/156 2024-01-24T13:55:41.476 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : leveldb-1.22-1.el8.x86_64 5/156 2024-01-24T13:55:41.724 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/156 2024-01-24T13:55:41.795 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/156 2024-01-24T13:55:42.031 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/156 2024-01-24T13:55:42.290 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/156 2024-01-24T13:55:42.458 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/156 2024-01-24T13:55:42.716 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/156 2024-01-24T13:55:42.768 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 37/156 2024-01-24T13:55:42.940 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : librados2-2:16.2.14-452.g9854ba8e.el8.x86_64 9/156 2024-01-24T13:55:42.986 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/156 2024-01-24T13:55:43.162 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 39/156 2024-01-24T13:55:43.396 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados2-2:16.2.14-452.g9854ba8e.el8.x86_64 9/156 2024-01-24T13:55:43.426 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rgw-2:16.2.14-452.g9854ba8e.el8.x86_64 40/156 2024-01-24T13:55:43.590 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : librbd1-2:16.2.14-452.g9854ba8e.el8.x86_64 10/156 2024-01-24T13:55:43.621 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rbd-2:16.2.14-452.g9854ba8e.el8.x86_64 41/156 2024-01-24T13:55:43.724 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librbd1-2:16.2.14-452.g9854ba8e.el8.x86_64 10/156 2024-01-24T13:55:43.814 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librados-devel-2:16.2.14-452.g9854ba8e.el8.x86_6 42/156 2024-01-24T13:55:43.948 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephfs2-2:16.2.14-452.g9854ba8e.el8.x86_64 11/156 2024-01-24T13:55:44.025 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/156 2024-01-24T13:55:44.051 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephfs2-2:16.2.14-452.g9854ba8e.el8.x86_64 11/156 2024-01-24T13:55:44.092 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/156 2024-01-24T13:55:44.325 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/156 2024-01-24T13:55:44.393 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/156 2024-01-24T13:55:44.587 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/156 2024-01-24T13:55:44.830 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/156 2024-01-24T13:55:45.033 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/156 2024-01-24T13:55:45.112 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rados-2:16.2.14-452.g9854ba8e.el8.x86_64 12/156 2024-01-24T13:55:45.265 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librgw2-2:16.2.14-452.g9854ba8e.el8.x86_64 13/156 2024-01-24T13:55:45.463 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librgw2-2:16.2.14-452.g9854ba8e.el8.x86_64 13/156 2024-01-24T13:55:45.483 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/156 2024-01-24T13:55:45.748 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/156 2024-01-24T13:55:45.817 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/156 2024-01-24T13:55:45.834 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/156 2024-01-24T13:55:45.845 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/156 2024-01-24T13:55:46.072 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/156 2024-01-24T13:55:46.087 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/156 2024-01-24T13:55:46.278 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/156 2024-01-24T13:55:46.321 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/156 2024-01-24T13:55:46.453 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/156 2024-01-24T13:55:46.498 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/156 2024-01-24T13:55:46.642 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/156 2024-01-24T13:55:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephsqlite-2:16.2.14-452.g9854ba8e.el8.x86_64 18/156 2024-01-24T13:55:46.769 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephsqlite-2:16.2.14-452.g9854ba8e.el8.x86_64 18/156 2024-01-24T13:55:46.861 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/156 2024-01-24T13:55:46.956 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libradosstriper1-2:16.2.14-452.g9854ba8e.el8.x86 19/156 2024-01-24T13:55:47.041 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libradosstriper1-2:16.2.14-452.g9854ba8e.el8.x86 19/156 2024-01-24T13:55:47.200 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/156 2024-01-24T13:55:47.214 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/156 2024-01-24T13:55:47.362 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/156 2024-01-24T13:55:47.479 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/156 2024-01-24T13:55:47.585 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/156 2024-01-24T13:55:47.696 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/156 2024-01-24T13:55:47.863 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/156 2024-01-24T13:55:48.103 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/156 2024-01-24T13:55:48.281 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/156 2024-01-24T13:55:48.442 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/156 2024-01-24T13:55:48.473 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 22/156 2024-01-24T13:55:48.727 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 60/156 2024-01-24T13:55:48.942 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/156 2024-01-24T13:55:49.189 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 22/156 2024-01-24T13:55:49.194 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 62/156 2024-01-24T13:55:49.389 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openblas-0.3.15-6.el8.x86_64 23/156 2024-01-24T13:55:49.445 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 63/156 2024-01-24T13:55:49.642 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 64/156 2024-01-24T13:55:49.816 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 65/156 2024-01-24T13:55:49.993 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 66/156 2024-01-24T13:55:50.148 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 23/156 2024-01-24T13:55:50.171 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 67/156 2024-01-24T13:55:50.485 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 68/156 2024-01-24T13:55:50.505 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/156 2024-01-24T13:55:50.657 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/156 2024-01-24T13:55:50.733 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : automake-1.16.1-8.el8.noarch 69/156 2024-01-24T13:55:50.846 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 26/156 2024-01-24T13:55:50.907 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 70/156 2024-01-24T13:55:50.992 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 70/156 2024-01-24T13:55:51.148 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : jq-1.6-7.el8.x86_64 71/156 2024-01-24T13:55:51.187 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 27/156 2024-01-24T13:55:51.306 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 72/156 2024-01-24T13:55:51.447 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 28/156 2024-01-24T13:55:51.464 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 73/156 2024-01-24T13:55:51.650 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 74/156 2024-01-24T13:55:51.767 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 29/156 2024-01-24T13:55:51.799 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 75/156 2024-01-24T13:55:51.946 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ceph-common-2:16.2.14-452.g9854ba8e.el8. 30/156 2024-01-24T13:55:51.971 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 76/156 2024-01-24T13:55:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ceph-argparse-2:16.2.14-452.g9854ba8e.el 31/156 2024-01-24T13:55:52.212 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 77/156 2024-01-24T13:55:52.404 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cephfs-2:16.2.14-452.g9854ba8e.el8.x86_6 32/156 2024-01-24T13:55:52.422 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 78/156 2024-01-24T13:55:52.653 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 79/156 2024-01-24T13:55:52.819 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 80/156 2024-01-24T13:55:53.017 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 81/156 2024-01-24T13:55:53.210 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 82/156 2024-01-24T13:55:53.402 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 83/156 2024-01-24T13:55:53.507 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/156 2024-01-24T13:55:53.588 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 84/156 2024-01-24T13:55:53.827 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 85/156 2024-01-24T13:55:53.880 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/156 2024-01-24T13:55:54.045 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 86/156 2024-01-24T13:55:54.157 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/156 2024-01-24T13:55:54.204 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 87/156 2024-01-24T13:55:54.355 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/156 2024-01-24T13:55:54.477 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-portend-2.6-1.el8.noarch 88/156 2024-01-24T13:55:54.665 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 89/156 2024-01-24T13:55:54.853 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 90/156 2024-01-24T13:55:55.082 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 91/156 2024-01-24T13:55:56.161 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 92/156 2024-01-24T13:55:56.364 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/156 2024-01-24T13:55:56.476 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 93/156 2024-01-24T13:55:56.681 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 94/156 2024-01-24T13:55:56.796 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 37/156 2024-01-24T13:55:56.965 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 95/156 2024-01-24T13:55:57.051 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/156 2024-01-24T13:55:57.229 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 96/156 2024-01-24T13:55:57.236 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 39/156 2024-01-24T13:55:57.483 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 97/156 2024-01-24T13:55:57.569 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rgw-2:16.2.14-452.g9854ba8e.el8.x86_64 40/156 2024-01-24T13:55:57.803 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rbd-2:16.2.14-452.g9854ba8e.el8.x86_64 41/156 2024-01-24T13:55:57.891 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 98/156 2024-01-24T13:55:58.047 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librados-devel-2:16.2.14-452.g9854ba8e.el8.x86_6 42/156 2024-01-24T13:55:58.255 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/156 2024-01-24T13:55:58.256 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 99/156 2024-01-24T13:55:58.335 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/156 2024-01-24T13:55:58.448 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 100/156 2024-01-24T13:55:58.630 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/156 2024-01-24T13:55:58.640 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 101/156 2024-01-24T13:55:58.696 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/156 2024-01-24T13:55:58.923 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/156 2024-01-24T13:55:59.246 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/156 2024-01-24T13:55:59.471 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/156 2024-01-24T13:55:59.952 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/156 2024-01-24T13:56:00.814 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/156 2024-01-24T13:56:00.835 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/156 2024-01-24T13:56:01.077 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/156 2024-01-24T13:56:01.290 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/156 2024-01-24T13:56:01.495 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/156 2024-01-24T13:56:01.704 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/156 2024-01-24T13:56:01.961 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/156 2024-01-24T13:56:02.223 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:16.2.14-452.g9854ba8e.el8.x86_64 102/156 2024-01-24T13:56:02.280 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/156 2024-01-24T13:56:02.447 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-common-2:16.2.14-452.g9854ba8e.el8.x86_64 102/156 2024-01-24T13:56:02.644 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/156 2024-01-24T13:56:02.869 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/156 2024-01-24T13:56:03.107 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/156 2024-01-24T13:56:03.127 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:16.2.14-452.g9854ba8e.el8.x86_64 102/156 2024-01-24T13:56:03.341 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/156 2024-01-24T13:56:03.732 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/156 2024-01-24T13:56:04.036 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 60/156 2024-01-24T13:56:04.045 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-base-2:16.2.14-452.g9854ba8e.el8.x86_64 103/156 2024-01-24T13:56:04.134 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-base-2:16.2.14-452.g9854ba8e.el8.x86_64 103/156 2024-01-24T13:56:04.288 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-selinux-2:16.2.14-452.g9854ba8e.el8.x86_64 104/156 2024-01-24T13:56:04.303 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/156 2024-01-24T13:56:04.606 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 62/156 2024-01-24T13:56:04.923 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 63/156 2024-01-24T13:56:05.163 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 64/156 2024-01-24T13:56:05.407 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 65/156 2024-01-24T13:56:05.691 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 66/156 2024-01-24T13:56:05.903 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 67/156 2024-01-24T13:56:06.257 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 68/156 2024-01-24T13:56:06.561 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : automake-1.16.1-8.el8.noarch 69/156 2024-01-24T13:56:06.720 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 70/156 2024-01-24T13:56:06.812 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 70/156 2024-01-24T13:56:07.014 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : jq-1.6-7.el8.x86_64 71/156 2024-01-24T13:56:07.255 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 72/156 2024-01-24T13:56:07.438 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 73/156 2024-01-24T13:56:07.682 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 74/156 2024-01-24T13:56:07.880 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 75/156 2024-01-24T13:56:08.078 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 76/156 2024-01-24T13:56:08.383 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 77/156 2024-01-24T13:56:08.604 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 78/156 2024-01-24T13:56:08.817 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 79/156 2024-01-24T13:56:09.025 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 80/156 2024-01-24T13:56:09.284 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 81/156 2024-01-24T13:56:09.559 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 82/156 2024-01-24T13:56:09.766 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 83/156 2024-01-24T13:56:09.985 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 84/156 2024-01-24T13:56:10.256 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 85/156 2024-01-24T13:56:10.547 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 86/156 2024-01-24T13:56:10.785 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 87/156 2024-01-24T13:56:11.061 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-portend-2.6-1.el8.noarch 88/156 2024-01-24T13:56:11.329 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 89/156 2024-01-24T13:56:11.554 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 90/156 2024-01-24T13:56:11.862 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 91/156 2024-01-24T13:56:13.062 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 92/156 2024-01-24T13:56:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 93/156 2024-01-24T13:56:13.717 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 94/156 2024-01-24T13:56:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 95/156 2024-01-24T13:56:14.337 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 96/156 2024-01-24T13:56:14.678 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 97/156 2024-01-24T13:56:14.989 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 98/156 2024-01-24T13:56:15.280 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 99/156 2024-01-24T13:56:15.492 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 100/156 2024-01-24T13:56:15.737 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 101/156 2024-01-24T13:56:20.383 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-common-2:16.2.14-452.g9854ba8e.el8.x86_64 102/156 2024-01-24T13:56:20.670 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-common-2:16.2.14-452.g9854ba8e.el8.x86_64 102/156 2024-01-24T13:56:21.427 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-common-2:16.2.14-452.g9854ba8e.el8.x86_64 102/156 2024-01-24T13:56:21.821 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-base-2:16.2.14-452.g9854ba8e.el8.x86_64 103/156 2024-01-24T13:56:21.910 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-base-2:16.2.14-452.g9854ba8e.el8.x86_64 103/156 2024-01-24T13:56:22.579 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-selinux-2:16.2.14-452.g9854ba8e.el8.x86_64 104/156 2024-01-24T13:56:23.659 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-selinux-2:16.2.14-452.g9854ba8e.el8.x86_64 104/156 2024-01-24T13:56:23.659 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /sys 2024-01-24T13:56:23.659 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /proc 2024-01-24T13:56:23.659 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /dev 2024-01-24T13:56:23.659 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /run 2024-01-24T13:56:23.659 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /mnt 2024-01-24T13:56:23.659 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /var/tmp 2024-01-24T13:56:23.659 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /home 2024-01-24T13:56:23.660 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /tmp 2024-01-24T13:56:23.660 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /dev 2024-01-24T13:56:23.660 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:56:24.043 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mds-2:16.2.14-452.g9854ba8e.el8.x86_64 105/156 2024-01-24T13:56:24.470 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mds-2:16.2.14-452.g9854ba8e.el8.x86_64 105/156 2024-01-24T13:56:24.618 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mon-2:16.2.14-452.g9854ba8e.el8.x86_64 106/156 2024-01-24T13:56:26.108 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mon-2:16.2.14-452.g9854ba8e.el8.x86_64 106/156 2024-01-24T13:56:26.260 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-osd-2:16.2.14-452.g9854ba8e.el8.x86_64 107/156 2024-01-24T13:56:26.283 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-osd-2:16.2.14-452.g9854ba8e.el8.x86_64 107/156 2024-01-24T13:56:27.931 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:16.2.14-452.g9854ba8e.el8.noarch 108/156 2024-01-24T13:56:28.077 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephadm-2:16.2.14-452.g9854ba8e.el8.noarch 108/156 2024-01-24T13:56:28.232 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-452.g9854ba8e.e 109/156 2024-01-24T13:56:28.435 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-452.g9854ba8e. 110/156 2024-01-24T13:56:28.642 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-452.g9854ba8e.el8.noa 111/156 2024-01-24T13:56:29.563 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-452.g9854ba8e.el8.noa 111/156 2024-01-24T13:56:29.816 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-452.g9854ba8e.el8.n 112/156 2024-01-24T13:56:30.862 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-452.g9854ba8e.el8.n 112/156 2024-01-24T13:56:31.034 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-452.g985 113/156 2024-01-24T13:56:31.124 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-452.g985 113/156 2024-01-24T13:56:31.484 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-452.g9854ba8e.el 114/156 2024-01-24T13:56:31.641 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-2:16.2.14-452.g9854ba8e.el8.x86_64 115/156 2024-01-24T13:56:31.667 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-2:16.2.14-452.g9854ba8e.el8.x86_64 115/156 2024-01-24T13:56:31.809 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-rook-2:16.2.14-452.g9854ba8e.el8.noarch 116/156 2024-01-24T13:56:31.821 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-452.g9854ba8e.el8.noarch 116/156 2024-01-24T13:56:32.880 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-2:16.2.14-452.g9854ba8e.el8.x86_64 117/156 2024-01-24T13:56:33.199 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-radosgw-2:16.2.14-452.g9854ba8e.el8.x86_64 118/156 2024-01-24T13:56:33.312 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-452.g9854ba8e.el8.x86_64 118/156 2024-01-24T13:56:33.459 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-452.g9854b 119/156 2024-01-24T13:56:33.502 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-452.g9854b 119/156 2024-01-24T13:56:33.642 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 120/156 2024-01-24T13:56:33.988 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephfs-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 120/156 2024-01-24T13:56:34.125 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 121/156 2024-01-24T13:56:38.636 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 121/156 2024-01-24T13:56:38.942 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-test-2:16.2.14-452.g9854ba8e.el8.x86_64 122/156 2024-01-24T13:56:39.090 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libtool-2.4.6-25.el8.x86_64 123/156 2024-01-24T13:56:39.774 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 123/156 2024-01-24T13:56:40.079 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 124/156 2024-01-24T13:56:40.349 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 125/156 2024-01-24T13:56:40.547 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-fuse-2:16.2.14-452.g9854ba8e.el8.x86_64 126/156 2024-01-24T13:56:40.752 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 127/156 2024-01-24T13:56:40.919 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 128/156 2024-01-24T13:56:41.114 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs-devel-2:16.2.14-452.g9854ba8e.el8.x86_ 129/156 2024-01-24T13:56:41.464 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 130/156 2024-01-24T13:56:41.790 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 131/156 2024-01-24T13:56:42.020 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 132/156 2024-01-24T13:56:42.207 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-top-2:16.2.14-452.g9854ba8e.el8.noarch 133/156 2024-01-24T13:56:42.423 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-fuse-2:16.2.14-452.g9854ba8e.el8.x86_64 134/156 2024-01-24T13:56:42.694 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-nbd-2:16.2.14-452.g9854ba8e.el8.x86_64 135/156 2024-01-24T13:56:42.925 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 136/156 2024-01-24T13:56:43.082 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 137/156 2024-01-24T13:56:43.092 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 137/156 2024-01-24T13:56:43.286 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 138/156 2024-01-24T13:56:43.489 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flex-2.6.1-9.el8.x86_64 139/156 2024-01-24T13:56:43.571 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 139/156 2024-01-24T13:56:43.773 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : bison-3.0.4-10.el8.x86_64 140/156 2024-01-24T13:56:43.787 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 140/156 2024-01-24T13:56:43.934 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 141/156 2024-01-24T13:56:44.125 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 142/156 2024-01-24T13:56:44.277 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 143/156 2024-01-24T13:56:44.277 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 144/156 2024-01-24T13:56:44.543 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 144/156 2024-01-24T13:56:44.543 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 145/156 2024-01-24T13:56:44.760 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 145/156 2024-01-24T13:56:44.760 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/156 2024-01-24T13:56:45.038 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/156 2024-01-24T13:56:45.250 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 147/156 2024-01-24T13:56:45.417 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 148/156 2024-01-24T13:56:45.633 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 149/156 2024-01-24T13:56:45.633 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-24T13:56:45.900 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-24T13:56:45.901 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 151/156 2024-01-24T13:56:46.150 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 151/156 2024-01-24T13:56:46.150 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 152/156 2024-01-24T13:56:46.375 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 152/156 2024-01-24T13:56:46.376 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 153/156 2024-01-24T13:56:46.451 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-selinux-2:16.2.14-452.g9854ba8e.el8.x86_64 104/156 2024-01-24T13:56:46.451 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /sys 2024-01-24T13:56:46.451 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /proc 2024-01-24T13:56:46.451 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /dev 2024-01-24T13:56:46.451 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /run 2024-01-24T13:56:46.451 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /mnt 2024-01-24T13:56:46.452 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /var/tmp 2024-01-24T13:56:46.452 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /home 2024-01-24T13:56:46.452 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /tmp 2024-01-24T13:56:46.452 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /dev 2024-01-24T13:56:46.452 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:56:46.592 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 153/156 2024-01-24T13:56:46.592 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 154/156 2024-01-24T13:56:46.850 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 154/156 2024-01-24T13:56:46.891 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mds-2:16.2.14-452.g9854ba8e.el8.x86_64 105/156 2024-01-24T13:56:46.983 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 155/156 2024-01-24T13:56:46.984 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-24T13:56:47.364 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mds-2:16.2.14-452.g9854ba8e.el8.x86_64 105/156 2024-01-24T13:56:47.541 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mon-2:16.2.14-452.g9854ba8e.el8.x86_64 106/156 2024-01-24T13:56:47.833 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-24T13:56:47.833 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-2:16.2.14-452.g9854ba8e.el8.x86_64 1/156 2024-01-24T13:56:47.833 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-base-2:16.2.14-452.g9854ba8e.el8.x86_64 2/156 2024-01-24T13:56:47.833 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-common-2:16.2.14-452.g9854ba8e.el8.x86_64 3/156 2024-01-24T13:56:47.833 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-fuse-2:16.2.14-452.g9854ba8e.el8.x86_64 4/156 2024-01-24T13:56:47.833 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-452.g9854b 5/156 2024-01-24T13:56:47.833 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mds-2:16.2.14-452.g9854ba8e.el8.x86_64 6/156 2024-01-24T13:56:47.833 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-2:16.2.14-452.g9854ba8e.el8.x86_64 7/156 2024-01-24T13:56:47.833 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mon-2:16.2.14-452.g9854ba8e.el8.x86_64 8/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-osd-2:16.2.14-452.g9854ba8e.el8.x86_64 9/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-radosgw-2:16.2.14-452.g9854ba8e.el8.x86_64 10/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-selinux-2:16.2.14-452.g9854ba8e.el8.x86_64 11/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-test-2:16.2.14-452.g9854ba8e.el8.x86_64 12/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 13/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs-devel-2:16.2.14-452.g9854ba8e.el8.x86_ 14/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs2-2:16.2.14-452.g9854ba8e.el8.x86_64 15/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephsqlite-2:16.2.14-452.g9854ba8e.el8.x86_64 16/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados-devel-2:16.2.14-452.g9854ba8e.el8.x86_6 17/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libradosstriper1-2:16.2.14-452.g9854ba8e.el8.x86 18/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librgw2-2:16.2.14-452.g9854ba8e.el8.x86_64 19/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-argparse-2:16.2.14-452.g9854ba8e.el 20/156 2024-01-24T13:56:47.834 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-common-2:16.2.14-452.g9854ba8e.el8. 21/156 2024-01-24T13:56:47.835 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cephfs-2:16.2.14-452.g9854ba8e.el8.x86_6 22/156 2024-01-24T13:56:47.835 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rados-2:16.2.14-452.g9854ba8e.el8.x86_64 23/156 2024-01-24T13:56:47.835 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rbd-2:16.2.14-452.g9854ba8e.el8.x86_64 24/156 2024-01-24T13:56:47.835 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rgw-2:16.2.14-452.g9854ba8e.el8.x86_64 25/156 2024-01-24T13:56:47.835 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-fuse-2:16.2.14-452.g9854ba8e.el8.x86_64 26/156 2024-01-24T13:56:47.835 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 27/156 2024-01-24T13:56:47.835 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-nbd-2:16.2.14-452.g9854ba8e.el8.x86_64 28/156 2024-01-24T13:56:47.835 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-452.g9854ba8e. 29/156 2024-01-24T13:56:47.835 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-452.g9854ba8e.el8.noa 30/156 2024-01-24T13:56:47.847 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-452.g9854ba8e.el8.n 31/156 2024-01-24T13:56:47.847 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-452.g985 32/156 2024-01-24T13:56:47.847 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-452.g9854ba8e.el 33/156 2024-01-24T13:56:47.847 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-rook-2:16.2.14-452.g9854ba8e.el8.noarch 34/156 2024-01-24T13:56:47.847 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-452.g9854ba8e.e 35/156 2024-01-24T13:56:47.847 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephadm-2:16.2.14-452.g9854ba8e.el8.noarch 36/156 2024-01-24T13:56:47.847 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-top-2:16.2.14-452.g9854ba8e.el8.noarch 37/156 2024-01-24T13:56:47.847 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 38/156 2024-01-24T13:56:47.847 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : leveldb-1.22-1.el8.x86_64 39/156 2024-01-24T13:56:47.847 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/156 2024-01-24T13:56:47.847 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 42/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 49/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/156 2024-01-24T13:56:47.848 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/156 2024-01-24T13:56:47.849 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/156 2024-01-24T13:56:47.849 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/156 2024-01-24T13:56:47.849 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 58/156 2024-01-24T13:56:47.849 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/156 2024-01-24T13:56:47.849 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/156 2024-01-24T13:56:47.849 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/156 2024-01-24T13:56:47.849 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/156 2024-01-24T13:56:47.849 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/156 2024-01-24T13:56:47.849 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/156 2024-01-24T13:56:47.849 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/156 2024-01-24T13:56:47.849 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/156 2024-01-24T13:56:47.849 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/156 2024-01-24T13:56:47.850 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 83/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : jq-1.6-7.el8.x86_64 95/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 96/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 97/156 2024-01-24T13:56:47.851 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 98/156 2024-01-24T13:56:47.852 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 99/156 2024-01-24T13:56:47.852 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 100/156 2024-01-24T13:56:47.852 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 101/156 2024-01-24T13:56:47.852 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 102/156 2024-01-24T13:56:47.852 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 103/156 2024-01-24T13:56:47.852 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 104/156 2024-01-24T13:56:47.852 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 105/156 2024-01-24T13:56:47.852 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 106/156 2024-01-24T13:56:47.852 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 107/156 2024-01-24T13:56:47.852 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 108/156 2024-01-24T13:56:47.852 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 109/156 2024-01-24T13:56:47.853 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 110/156 2024-01-24T13:56:47.853 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 111/156 2024-01-24T13:56:47.853 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 112/156 2024-01-24T13:56:47.853 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 113/156 2024-01-24T13:56:47.853 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 114/156 2024-01-24T13:56:47.853 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 115/156 2024-01-24T13:56:47.853 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 116/156 2024-01-24T13:56:47.853 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 117/156 2024-01-24T13:56:47.853 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 118/156 2024-01-24T13:56:47.853 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 119/156 2024-01-24T13:56:47.853 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 120/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 121/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 122/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 123/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 124/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:16.2.14-452.g9854ba8e.el8.x86_64 125/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 126/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:16.2.14-452.g9854ba8e.el8.x86_64 127/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 128/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 129/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 130/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 131/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 132/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 133/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 134/156 2024-01-24T13:56:47.854 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 135/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 136/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 137/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 138/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 139/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 140/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 141/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 142/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 143/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 144/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 145/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 146/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 147/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 148/156 2024-01-24T13:56:47.855 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 149/156 2024-01-24T13:56:47.856 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 150/156 2024-01-24T13:56:47.856 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 151/156 2024-01-24T13:56:47.856 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 152/156 2024-01-24T13:56:47.856 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 153/156 2024-01-24T13:56:47.856 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 154/156 2024-01-24T13:56:47.856 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 155/156 2024-01-24T13:56:49.192 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mon-2:16.2.14-452.g9854ba8e.el8.x86_64 106/156 2024-01-24T13:56:49.391 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-osd-2:16.2.14-452.g9854ba8e.el8.x86_64 107/156 2024-01-24T13:56:49.418 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-osd-2:16.2.14-452.g9854ba8e.el8.x86_64 107/156 2024-01-24T13:56:51.443 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: cephadm-2:16.2.14-452.g9854ba8e.el8.noarch 108/156 2024-01-24T13:56:51.612 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephadm-2:16.2.14-452.g9854ba8e.el8.noarch 108/156 2024-01-24T13:56:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-452.g9854ba8e.e 109/156 2024-01-24T13:56:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-452.g9854ba8e. 110/156 2024-01-24T13:56:52.332 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-452.g9854ba8e.el8.noa 111/156 2024-01-24T13:56:53.363 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-452.g9854ba8e.el8.noa 111/156 2024-01-24T13:56:53.700 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-452.g9854ba8e.el8.n 112/156 2024-01-24T13:56:54.247 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 156/156 2024-01-24T13:56:54.247 INFO:teuthology.orchestra.run.smithi163.stdout:Installed products updated. 2024-01-24T13:56:54.840 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:56:54.840 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2024-01-24T13:56:54.840 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-01-24T13:56:54.840 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-01-24T13:56:54.840 INFO:teuthology.orchestra.run.smithi163.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-01-24T13:56:54.840 INFO:teuthology.orchestra.run.smithi163.stdout: libblkid-2.32.1-43.el8.x86_64 2024-01-24T13:56:54.840 INFO:teuthology.orchestra.run.smithi163.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-01-24T13:56:54.840 INFO:teuthology.orchestra.run.smithi163.stdout: libmount-2.32.1-43.el8.x86_64 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: librados2-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-2.32.1-43.el8.x86_64 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-2.32.1-43.el8.x86_64 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: automake-1.16.1-8.el8.noarch 2024-01-24T13:56:54.841 INFO:teuthology.orchestra.run.smithi163.stdout: bison-3.0.4-10.el8.x86_64 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: flex-2.6.1-9.el8.x86_64 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: fuse-2.9.7-17.el8.x86_64 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: jq-1.6-7.el8.x86_64 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-01-24T13:56:54.843 INFO:teuthology.orchestra.run.smithi163.stdout: leveldb-1.22-1.el8.x86_64 2024-01-24T13:56:54.844 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-24T13:56:54.844 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.844 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.844 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.844 INFO:teuthology.orchestra.run.smithi163.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-01-24T13:56:54.844 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-01-24T13:56:54.844 INFO:teuthology.orchestra.run.smithi163.stdout: liboath-2.6.2-3.el8.x86_64 2024-01-24T13:56:54.844 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-01-24T13:56:54.844 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-01-24T13:56:54.844 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.844 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: libtool-2.4.6-25.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind-1.3.1-3.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-0.3.15-6.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-01-24T13:56:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: python3-apipkg-1.5-6.el8.noarch 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel-2.5.1-7.el8.noarch 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-01-24T13:56:54.846 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-execnet-1.7.1-1.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-6.2-6.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako-1.0.6-14.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-01-24T13:56:54.847 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend-2.6-1.el8.noarch 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser-2.14-14.el8.noarch 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz-2017.2-11.el8.noarch 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-remoto-1.2.1-1.el8.noarch 2024-01-24T13:56:54.848 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes-2.4.1-12.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa-4.9-2.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-01-24T13:56:54.849 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.850 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.850 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:56:54.850 INFO:teuthology.orchestra.run.smithi163.stdout: socat-1.7.4.1-1.el8.x86_64 2024-01-24T13:56:54.850 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-01-24T13:56:54.850 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-01-24T13:56:54.850 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-01-24T13:56:54.850 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:56:54.850 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-01-24T13:56:54.861 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-452.g9854ba8e.el8.n 112/156 2024-01-24T13:56:55.032 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-452.g985 113/156 2024-01-24T13:56:55.047 DEBUG:teuthology.parallel:result is None 2024-01-24T13:56:55.137 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-452.g985 113/156 2024-01-24T13:56:55.593 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-452.g9854ba8e.el 114/156 2024-01-24T13:56:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-2:16.2.14-452.g9854ba8e.el8.x86_64 115/156 2024-01-24T13:56:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-2:16.2.14-452.g9854ba8e.el8.x86_64 115/156 2024-01-24T13:56:56.032 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-rook-2:16.2.14-452.g9854ba8e.el8.noarch 116/156 2024-01-24T13:56:56.047 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-452.g9854ba8e.el8.noarch 116/156 2024-01-24T13:56:57.208 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-2:16.2.14-452.g9854ba8e.el8.x86_64 117/156 2024-01-24T13:56:57.417 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-radosgw-2:16.2.14-452.g9854ba8e.el8.x86_64 118/156 2024-01-24T13:56:57.566 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-452.g9854ba8e.el8.x86_64 118/156 2024-01-24T13:56:57.724 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-452.g9854b 119/156 2024-01-24T13:56:57.773 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-452.g9854b 119/156 2024-01-24T13:56:57.940 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 120/156 2024-01-24T13:56:58.325 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: cephfs-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 120/156 2024-01-24T13:56:58.498 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 121/156 2024-01-24T13:57:03.493 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: rbd-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 121/156 2024-01-24T13:57:03.816 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-test-2:16.2.14-452.g9854ba8e.el8.x86_64 122/156 2024-01-24T13:57:04.022 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libtool-2.4.6-25.el8.x86_64 123/156 2024-01-24T13:57:04.833 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 123/156 2024-01-24T13:57:05.187 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 124/156 2024-01-24T13:57:05.447 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 125/156 2024-01-24T13:57:05.738 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-fuse-2:16.2.14-452.g9854ba8e.el8.x86_64 126/156 2024-01-24T13:57:05.995 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 127/156 2024-01-24T13:57:06.234 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 128/156 2024-01-24T13:57:06.439 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephfs-devel-2:16.2.14-452.g9854ba8e.el8.x86_ 129/156 2024-01-24T13:57:06.860 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 130/156 2024-01-24T13:57:07.201 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 131/156 2024-01-24T13:57:07.486 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 132/156 2024-01-24T13:57:07.675 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-top-2:16.2.14-452.g9854ba8e.el8.noarch 133/156 2024-01-24T13:57:07.956 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-fuse-2:16.2.14-452.g9854ba8e.el8.x86_64 134/156 2024-01-24T13:57:08.267 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-nbd-2:16.2.14-452.g9854ba8e.el8.x86_64 135/156 2024-01-24T13:57:08.549 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 136/156 2024-01-24T13:57:08.738 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 137/156 2024-01-24T13:57:08.750 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 137/156 2024-01-24T13:57:08.939 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 138/156 2024-01-24T13:57:09.188 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flex-2.6.1-9.el8.x86_64 139/156 2024-01-24T13:57:09.275 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 139/156 2024-01-24T13:57:09.505 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : bison-3.0.4-10.el8.x86_64 140/156 2024-01-24T13:57:09.521 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 140/156 2024-01-24T13:57:09.771 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 141/156 2024-01-24T13:57:09.974 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 142/156 2024-01-24T13:57:10.200 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 143/156 2024-01-24T13:57:10.201 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 144/156 2024-01-24T13:57:10.475 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 144/156 2024-01-24T13:57:10.475 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 145/156 2024-01-24T13:57:10.709 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 145/156 2024-01-24T13:57:10.709 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/156 2024-01-24T13:57:11.004 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/156 2024-01-24T13:57:11.232 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 147/156 2024-01-24T13:57:11.477 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 148/156 2024-01-24T13:57:11.724 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 149/156 2024-01-24T13:57:11.724 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-24T13:57:12.025 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-24T13:57:12.025 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 151/156 2024-01-24T13:57:12.283 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 151/156 2024-01-24T13:57:12.283 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 152/156 2024-01-24T13:57:12.558 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 152/156 2024-01-24T13:57:12.558 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 153/156 2024-01-24T13:57:12.799 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 153/156 2024-01-24T13:57:12.799 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 154/156 2024-01-24T13:57:13.082 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 154/156 2024-01-24T13:57:13.249 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 155/156 2024-01-24T13:57:13.249 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-24T13:57:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-24T13:57:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-2:16.2.14-452.g9854ba8e.el8.x86_64 1/156 2024-01-24T13:57:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-base-2:16.2.14-452.g9854ba8e.el8.x86_64 2/156 2024-01-24T13:57:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-common-2:16.2.14-452.g9854ba8e.el8.x86_64 3/156 2024-01-24T13:57:14.223 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-fuse-2:16.2.14-452.g9854ba8e.el8.x86_64 4/156 2024-01-24T13:57:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-452.g9854b 5/156 2024-01-24T13:57:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mds-2:16.2.14-452.g9854ba8e.el8.x86_64 6/156 2024-01-24T13:57:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-2:16.2.14-452.g9854ba8e.el8.x86_64 7/156 2024-01-24T13:57:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mon-2:16.2.14-452.g9854ba8e.el8.x86_64 8/156 2024-01-24T13:57:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-osd-2:16.2.14-452.g9854ba8e.el8.x86_64 9/156 2024-01-24T13:57:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-radosgw-2:16.2.14-452.g9854ba8e.el8.x86_64 10/156 2024-01-24T13:57:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-selinux-2:16.2.14-452.g9854ba8e.el8.x86_64 11/156 2024-01-24T13:57:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-test-2:16.2.14-452.g9854ba8e.el8.x86_64 12/156 2024-01-24T13:57:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 13/156 2024-01-24T13:57:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephfs-devel-2:16.2.14-452.g9854ba8e.el8.x86_ 14/156 2024-01-24T13:57:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephfs2-2:16.2.14-452.g9854ba8e.el8.x86_64 15/156 2024-01-24T13:57:14.224 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephsqlite-2:16.2.14-452.g9854ba8e.el8.x86_64 16/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados-devel-2:16.2.14-452.g9854ba8e.el8.x86_6 17/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libradosstriper1-2:16.2.14-452.g9854ba8e.el8.x86 18/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librgw2-2:16.2.14-452.g9854ba8e.el8.x86_64 19/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ceph-argparse-2:16.2.14-452.g9854ba8e.el 20/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ceph-common-2:16.2.14-452.g9854ba8e.el8. 21/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cephfs-2:16.2.14-452.g9854ba8e.el8.x86_6 22/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rados-2:16.2.14-452.g9854ba8e.el8.x86_64 23/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rbd-2:16.2.14-452.g9854ba8e.el8.x86_64 24/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rgw-2:16.2.14-452.g9854ba8e.el8.x86_64 25/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-fuse-2:16.2.14-452.g9854ba8e.el8.x86_64 26/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 27/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-nbd-2:16.2.14-452.g9854ba8e.el8.x86_64 28/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-452.g9854ba8e. 29/156 2024-01-24T13:57:14.225 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-452.g9854ba8e.el8.noa 30/156 2024-01-24T13:57:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-452.g9854ba8e.el8.n 31/156 2024-01-24T13:57:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-452.g985 32/156 2024-01-24T13:57:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-452.g9854ba8e.el 33/156 2024-01-24T13:57:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-rook-2:16.2.14-452.g9854ba8e.el8.noarch 34/156 2024-01-24T13:57:14.241 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-452.g9854ba8e.e 35/156 2024-01-24T13:57:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephadm-2:16.2.14-452.g9854ba8e.el8.noarch 36/156 2024-01-24T13:57:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-top-2:16.2.14-452.g9854ba8e.el8.noarch 37/156 2024-01-24T13:57:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 38/156 2024-01-24T13:57:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : leveldb-1.22-1.el8.x86_64 39/156 2024-01-24T13:57:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/156 2024-01-24T13:57:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/156 2024-01-24T13:57:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 42/156 2024-01-24T13:57:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/156 2024-01-24T13:57:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/156 2024-01-24T13:57:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/156 2024-01-24T13:57:14.242 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/156 2024-01-24T13:57:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/156 2024-01-24T13:57:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/156 2024-01-24T13:57:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 49/156 2024-01-24T13:57:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/156 2024-01-24T13:57:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/156 2024-01-24T13:57:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/156 2024-01-24T13:57:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/156 2024-01-24T13:57:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/156 2024-01-24T13:57:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/156 2024-01-24T13:57:14.243 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/156 2024-01-24T13:57:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/156 2024-01-24T13:57:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 58/156 2024-01-24T13:57:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/156 2024-01-24T13:57:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/156 2024-01-24T13:57:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/156 2024-01-24T13:57:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/156 2024-01-24T13:57:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/156 2024-01-24T13:57:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/156 2024-01-24T13:57:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/156 2024-01-24T13:57:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/156 2024-01-24T13:57:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/156 2024-01-24T13:57:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/156 2024-01-24T13:57:14.245 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 83/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/156 2024-01-24T13:57:14.246 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : jq-1.6-7.el8.x86_64 95/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 96/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 97/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 98/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 99/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 100/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 101/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 102/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 103/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 104/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 105/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 106/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 107/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 108/156 2024-01-24T13:57:14.247 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 109/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 110/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 111/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 112/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 113/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 114/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 115/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 116/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 117/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 118/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 119/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 120/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 121/156 2024-01-24T13:57:14.248 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 122/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 123/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 124/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados2-2:16.2.14-452.g9854ba8e.el8.x86_64 125/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 126/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librbd1-2:16.2.14-452.g9854ba8e.el8.x86_64 127/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 128/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 129/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 130/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 131/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 132/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 133/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 134/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 135/156 2024-01-24T13:57:14.249 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 136/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 137/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 138/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 139/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 140/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 141/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 142/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 143/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 144/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 145/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 146/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 147/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 148/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 149/156 2024-01-24T13:57:14.250 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 150/156 2024-01-24T13:57:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 151/156 2024-01-24T13:57:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 152/156 2024-01-24T13:57:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 153/156 2024-01-24T13:57:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 154/156 2024-01-24T13:57:14.251 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 155/156 2024-01-24T13:57:23.416 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 156/156 2024-01-24T13:57:23.416 INFO:teuthology.orchestra.run.smithi029.stdout:Installed products updated. 2024-01-24T13:57:24.094 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:24.094 INFO:teuthology.orchestra.run.smithi029.stdout:Upgraded: 2024-01-24T13:57:24.094 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-01-24T13:57:24.094 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-01-24T13:57:24.094 INFO:teuthology.orchestra.run.smithi029.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-01-24T13:57:24.094 INFO:teuthology.orchestra.run.smithi029.stdout: libblkid-2.32.1-43.el8.x86_64 2024-01-24T13:57:24.094 INFO:teuthology.orchestra.run.smithi029.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-01-24T13:57:24.094 INFO:teuthology.orchestra.run.smithi029.stdout: libmount-2.32.1-43.el8.x86_64 2024-01-24T13:57:24.094 INFO:teuthology.orchestra.run.smithi029.stdout: librados2-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.095 INFO:teuthology.orchestra.run.smithi029.stdout: librbd1-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.095 INFO:teuthology.orchestra.run.smithi029.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-01-24T13:57:24.095 INFO:teuthology.orchestra.run.smithi029.stdout: libuuid-2.32.1-43.el8.x86_64 2024-01-24T13:57:24.095 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-01-24T13:57:24.095 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-01-24T13:57:24.095 INFO:teuthology.orchestra.run.smithi029.stdout: util-linux-2.32.1-43.el8.x86_64 2024-01-24T13:57:24.095 INFO:teuthology.orchestra.run.smithi029.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-01-24T13:57:24.095 INFO:teuthology.orchestra.run.smithi029.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-01-24T13:57:24.095 INFO:teuthology.orchestra.run.smithi029.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-01-24T13:57:24.095 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-01-24T13:57:24.095 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-01-24T13:57:24.095 INFO:teuthology.orchestra.run.smithi029.stdout: automake-1.16.1-8.el8.noarch 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: bison-3.0.4-10.el8.x86_64 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-common-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-fuse-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-grafana-dashboards-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-immutable-object-cache-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mds-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-cephadm-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-diskprediction-local-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:57:24.096 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-rook-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mon-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-osd-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-prometheus-alerts-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-radosgw-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-selinux-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-test-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: cephadm-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-top-2:16.2.14-452.g9854ba8e.el8.noarch 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: flex-2.6.1-9.el8.x86_64 2024-01-24T13:57:24.097 INFO:teuthology.orchestra.run.smithi029.stdout: fuse-2.9.7-17.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: jq-1.6-7.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: leveldb-1.22-1.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs-devel-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs2-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: libcephsqlite-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: liboath-2.6.2-3.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-01-24T13:57:24.098 INFO:teuthology.orchestra.run.smithi029.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: librados-devel-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: librgw2-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: libtool-2.4.6-25.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: libunwind-1.3.1-3.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-01-24T13:57:24.099 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-0.3.15-6.el8.x86_64 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: python3-apipkg-1.5-6.el8.noarch 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: python3-babel-2.5.1-7.el8.noarch 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cephfs-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.100 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-execnet-1.7.1-1.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-6.2-6.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako-1.0.6-14.el8.noarch 2024-01-24T13:57:24.101 INFO:teuthology.orchestra.run.smithi029.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-portend-2.6-1.el8.noarch 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pycparser-2.14-14.el8.noarch 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytz-2017.2-11.el8.noarch 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.102 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rbd-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-remoto-1.2.1-1.el8.noarch 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rgw-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-routes-2.4.1-12.el8.noarch 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa-4.9-2.el8.noarch 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-01-24T13:57:24.103 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-01-24T13:57:24.104 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-01-24T13:57:24.104 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-01-24T13:57:24.104 INFO:teuthology.orchestra.run.smithi029.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-01-24T13:57:24.104 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-fuse-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.104 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-mirror-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.104 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-nbd-2:16.2.14-452.g9854ba8e.el8.x86_64 2024-01-24T13:57:24.104 INFO:teuthology.orchestra.run.smithi029.stdout: socat-1.7.4.1-1.el8.x86_64 2024-01-24T13:57:24.104 INFO:teuthology.orchestra.run.smithi029.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-01-24T13:57:24.104 INFO:teuthology.orchestra.run.smithi029.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-01-24T13:57:24.104 INFO:teuthology.orchestra.run.smithi029.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-01-24T13:57:24.105 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:24.105 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-01-24T13:57:24.309 DEBUG:teuthology.parallel:result is None 2024-01-24T13:57:24.309 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9854ba8e5f82e4476d4c6c06dd8feefca92b8b32 2024-01-24T13:57:24.527 DEBUG:teuthology.orchestra.run.smithi029:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-24T13:57:24.556 INFO:teuthology.orchestra.run.smithi029.stdout:16.2.14-452.g9854ba8e.el8 2024-01-24T13:57:24.557 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-452.g9854ba8e.el8 2024-01-24T13:57:24.557 INFO:teuthology.task.install:The correct ceph version 16.2.14-452.g9854ba8e is installed. 2024-01-24T13:57:24.559 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9854ba8e5f82e4476d4c6c06dd8feefca92b8b32 2024-01-24T13:57:24.706 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-24T13:57:24.735 INFO:teuthology.orchestra.run.smithi163.stdout:16.2.14-452.g9854ba8e.el8 2024-01-24T13:57:24.735 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-452.g9854ba8e.el8 2024-01-24T13:57:24.735 INFO:teuthology.task.install:The correct ceph version 16.2.14-452.g9854ba8e is installed. 2024-01-24T13:57:24.737 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-24T13:57:24.737 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:24.737 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-24T13:57:24.791 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:24.791 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-24T13:57:24.839 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-24T13:57:24.840 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:24.840 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/daemon-helper 2024-01-24T13:57:24.874 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-24T13:57:24.948 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:24.948 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/daemon-helper 2024-01-24T13:57:24.979 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-24T13:57:25.048 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-24T13:57:25.049 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:25.049 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-24T13:57:25.083 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-24T13:57:25.158 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:25.158 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-24T13:57:25.186 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-24T13:57:25.254 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-24T13:57:25.255 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:25.255 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/stdin-killer 2024-01-24T13:57:25.289 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-24T13:57:25.364 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:25.364 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/stdin-killer 2024-01-24T13:57:25.392 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-24T13:57:25.460 INFO:teuthology.run_tasks:Running task ceph... 2024-01-24T13:57:25.522 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-24T13:57:25.522 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 777 /var/log/ceph 2024-01-24T13:57:25.525 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /var/log/ceph 2024-01-24T13:57:25.556 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-24T13:57:25.557 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-24T13:57:25.600 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-24T13:57:25.631 INFO:tasks.ceph:Creating extra log directories... 2024-01-24T13:57:25.631 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-24T13:57:25.674 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-24T13:57:25.712 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-01-24T13:57:25.712 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-01-24T13:57:25.712 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-01-24T13:57:25.713 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:25.713 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-24T13:57:25.767 DEBUG:teuthology.orchestra.run.smithi029:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-24T13:57:25.897 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-24T13:57:25.933 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:25.933 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-24T13:57:25.951 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-24T13:57:26.065 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-24T13:57:26.097 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-24T13:57:26.097 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-01-24T13:57:26.098 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993', 'branch': 'wip-vshankar-testing-pacific-020240119.072505', 'description': 'fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7530993', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'f2d3460408951339ac0933340ad449487c9efddc'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-pacific-020240119.072505'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '9854ba8e5f82e4476d4c6c06dd8feefca92b8b32'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9854ba8e5f82e4476d4c6c06dd8feefca92b8b32'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-pacific-020240119.072505', 'sha1': '9854ba8e5f82e4476d4c6c06dd8feefca92b8b32'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2977, 'sha1': '9854ba8e5f82e4476d4c6c06dd8feefca92b8b32', 'sleep_before_teardown': 0, 'subset': '749/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-pacific-020240119.072505', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_9854ba8e5f82e4476d4c6c06dd8feefca92b8b32/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '9854ba8e5f82e4476d4c6c06dd8feefca92b8b32', 'targets': {'smithi029.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI6PZ2wPbShQTiM/sTKlfYNw8H9btmfys69InEuhp0dLIpq2XsOxNNeYRu+9JOp4KpOsakfk4NvM3/d57tFr91M=', 'smithi163.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG7uo/iwSJdGQ71ymsdoikagQTGpJfY2j1wVDZ+MX9OUV8Jev1J0swzSpQDMJ1RkC3+idQGjoMx1olU7dutxTKM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'f2d3460408951339ac0933340ad449487c9efddc'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '9854ba8e5f82e4476d4c6c06dd8feefca92b8b32'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'c856e0f9244dd29a50697123b480d63f16a4ed81', 'timestamp': '2024-01-24_13:20:07', 'tube': 'smithi', 'use_shaman': True, 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-24T13:57:26.098 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-24T13:57:26.100 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-24T13:57:26.149 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m0777 -- /var/run/ceph 2024-01-24T13:57:26.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/run/ceph 2024-01-24T13:57:26.216 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:26.216 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/scratch_devs of=/dev/stdout 2024-01-24T13:57:26.256 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-24T13:57:26.256 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_1 2024-01-24T13:57:26.318 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-24T13:57:26.318 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T13:57:26.318 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 6h/6d Inode: 175763 Links: 1 2024-01-24T13:57:26.319 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T13:57:26.319 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-01-24T13:57:26.319 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-01-24 13:57:13.832807554 +0000 2024-01-24T13:57:26.319 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-01-24 13:53:59.848669728 +0000 2024-01-24T13:57:26.319 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-01-24 13:53:59.848669728 +0000 2024-01-24T13:57:26.319 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2024-01-24T13:57:26.319 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-24T13:57:26.390 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-01-24T13:57:26.390 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-01-24T13:57:26.391 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000124337 s, 4.1 MB/s 2024-01-24T13:57:26.392 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-24T13:57:26.455 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_2 2024-01-24T13:57:26.517 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-24T13:57:26.517 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T13:57:26.517 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 6h/6d Inode: 175868 Links: 1 2024-01-24T13:57:26.517 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T13:57:26.517 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-01-24T13:57:26.517 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-01-24 13:57:13.832807554 +0000 2024-01-24T13:57:26.517 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-01-24 13:54:00.323661788 +0000 2024-01-24T13:57:26.517 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-01-24 13:54:00.323661788 +0000 2024-01-24T13:57:26.517 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2024-01-24T13:57:26.517 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-24T13:57:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-01-24T13:57:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-01-24T13:57:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.00012937 s, 4.0 MB/s 2024-01-24T13:57:26.591 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-24T13:57:26.654 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_3 2024-01-24T13:57:26.716 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-24T13:57:26.716 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T13:57:26.716 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 6h/6d Inode: 174491 Links: 1 2024-01-24T13:57:26.716 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T13:57:26.716 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-01-24T13:57:26.716 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-01-24 13:57:13.832807554 +0000 2024-01-24T13:57:26.716 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-01-24 13:54:00.693655604 +0000 2024-01-24T13:57:26.716 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-01-24 13:54:00.693655604 +0000 2024-01-24T13:57:26.717 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2024-01-24T13:57:26.717 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-24T13:57:26.788 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-01-24T13:57:26.788 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-01-24T13:57:26.788 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000128193 s, 4.0 MB/s 2024-01-24T13:57:26.789 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-24T13:57:26.853 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_4 2024-01-24T13:57:26.917 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-24T13:57:26.917 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T13:57:26.917 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 6h/6d Inode: 174586 Links: 1 2024-01-24T13:57:26.917 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T13:57:26.917 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-01-24T13:57:26.917 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-01-24 13:57:13.832807554 +0000 2024-01-24T13:57:26.917 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-01-24 13:54:01.073649252 +0000 2024-01-24T13:57:26.917 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-01-24 13:54:01.073649252 +0000 2024-01-24T13:57:26.917 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2024-01-24T13:57:26.917 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-24T13:57:26.989 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-01-24T13:57:26.989 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-01-24T13:57:26.990 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000122065 s, 4.2 MB/s 2024-01-24T13:57:26.991 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-24T13:57:27.053 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-24T13:57:27.053 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:27.053 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2024-01-24T13:57:27.074 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-24T13:57:27.074 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 2024-01-24T13:57:27.135 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-24T13:57:27.135 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T13:57:27.135 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 6h/6d Inode: 172727 Links: 1 2024-01-24T13:57:27.135 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T13:57:27.135 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-01-24T13:57:27.135 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-01-24 13:56:47.485352212 +0000 2024-01-24T13:57:27.135 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-01-24 13:53:02.360154598 +0000 2024-01-24T13:57:27.136 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-01-24 13:53:02.360154598 +0000 2024-01-24T13:57:27.136 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2024-01-24T13:57:27.136 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-24T13:57:27.206 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-01-24T13:57:27.206 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-01-24T13:57:27.206 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.00022537 s, 2.3 MB/s 2024-01-24T13:57:27.207 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-24T13:57:27.269 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 2024-01-24T13:57:27.330 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-24T13:57:27.330 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T13:57:27.330 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 6h/6d Inode: 172840 Links: 1 2024-01-24T13:57:27.330 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T13:57:27.330 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-01-24T13:57:27.330 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-01-24 13:56:47.486352188 +0000 2024-01-24T13:57:27.330 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-01-24 13:53:02.711146211 +0000 2024-01-24T13:57:27.331 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-01-24 13:53:02.711146211 +0000 2024-01-24T13:57:27.331 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2024-01-24T13:57:27.331 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-24T13:57:27.401 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-01-24T13:57:27.401 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-01-24T13:57:27.401 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000232752 s, 2.2 MB/s 2024-01-24T13:57:27.402 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-24T13:57:27.463 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 2024-01-24T13:57:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-24T13:57:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T13:57:27.523 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 6h/6d Inode: 170807 Links: 1 2024-01-24T13:57:27.523 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T13:57:27.523 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-01-24T13:57:27.523 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-01-24 13:56:47.486352188 +0000 2024-01-24T13:57:27.523 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-01-24 13:53:03.088137203 +0000 2024-01-24T13:57:27.523 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-01-24 13:53:03.088137203 +0000 2024-01-24T13:57:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2024-01-24T13:57:27.523 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-24T13:57:27.591 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-01-24T13:57:27.591 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-01-24T13:57:27.591 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000225064 s, 2.3 MB/s 2024-01-24T13:57:27.592 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-24T13:57:27.653 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 2024-01-24T13:57:27.712 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-24T13:57:27.713 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T13:57:27.713 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 6h/6d Inode: 173985 Links: 1 2024-01-24T13:57:27.713 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T13:57:27.713 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-01-24T13:57:27.713 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-01-24 13:56:47.486352188 +0000 2024-01-24T13:57:27.713 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-01-24 13:53:03.419129295 +0000 2024-01-24T13:57:27.713 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-01-24 13:53:03.419129295 +0000 2024-01-24T13:57:27.713 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2024-01-24T13:57:27.713 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-24T13:57:27.780 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-01-24T13:57:27.780 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-01-24T13:57:27.780 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.0002377 s, 2.2 MB/s 2024-01-24T13:57:27.781 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-24T13:57:27.840 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-24T13:57:27.841 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi029.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-24T13:57:27.841 INFO:tasks.ceph:Generating config... 2024-01-24T13:57:27.842 INFO:tasks.ceph:[client] client mount timeout = 600 2024-01-24T13:57:27.842 INFO:tasks.ceph:[client] debug client = 20 2024-01-24T13:57:27.842 INFO:tasks.ceph:[client] debug ms = 1 2024-01-24T13:57:27.842 INFO:tasks.ceph:[client] pid file = 2024-01-24T13:57:27.842 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-01-24T13:57:27.842 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-01-24T13:57:27.842 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-01-24T13:57:27.842 INFO:tasks.ceph:[mds] debug mds = 20 2024-01-24T13:57:27.842 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mds] debug ms = 1 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mds] mds debug frag = True 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mgr] debug client = 20 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-24T13:57:27.843 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-01-24T13:57:27.844 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-01-24T13:57:27.845 INFO:tasks.ceph:Setting up mon.a... 2024-01-24T13:57:27.845 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-24T13:57:27.910 INFO:teuthology.orchestra.run.smithi029.stdout:creating /etc/ceph/ceph.keyring 2024-01-24T13:57:27.913 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-24T13:57:27.963 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-24T13:57:27.995 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.29'), ('mon.b', '172.21.15.163'), ('mon.c', '[v2:172.21.15.163:3301,v1:172.21.15.163:6790]')] 2024-01-24T13:57:27.995 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.29,172.21.15.163,[v2:172.21.15.163:3301,v1:172.21.15.163:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-01-24T13:57:27.996 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:27.996 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-24T13:57:28.056 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.29 --add b 172.21.15.163 --addv c '[v2:172.21.15.163:3301,v1:172.21.15.163:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:57:28.147 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:57:28.148 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool: generated fsid b1c88d29-1963-443e-8a67-63a822ae75dd 2024-01-24T13:57:28.148 INFO:teuthology.orchestra.run.smithi029.stdout:epoch 0 2024-01-24T13:57:28.148 INFO:teuthology.orchestra.run.smithi029.stdout:fsid b1c88d29-1963-443e-8a67-63a822ae75dd 2024-01-24T13:57:28.148 INFO:teuthology.orchestra.run.smithi029.stdout:last_changed 2024-01-24T13:57:28.148565+0000 2024-01-24T13:57:28.148 INFO:teuthology.orchestra.run.smithi029.stdout:created 2024-01-24T13:57:28.148565+0000 2024-01-24T13:57:28.148 INFO:teuthology.orchestra.run.smithi029.stdout:min_mon_release 0 (unknown) 2024-01-24T13:57:28.148 INFO:teuthology.orchestra.run.smithi029.stdout:election_strategy: 1 2024-01-24T13:57:28.148 INFO:teuthology.orchestra.run.smithi029.stdout:0: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.a 2024-01-24T13:57:28.148 INFO:teuthology.orchestra.run.smithi029.stdout:1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.b 2024-01-24T13:57:28.148 INFO:teuthology.orchestra.run.smithi029.stdout:2: [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] mon.c 2024-01-24T13:57:28.148 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-01-24T13:57:28.149 DEBUG:teuthology.orchestra.run.smithi029:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-24T13:57:28.169 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b1c88d29-1963-443e-8a67-63a822ae75dd... 2024-01-24T13:57:28.170 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-24T13:57:28.212 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-24T13:57:28.266 INFO:teuthology.orchestra.run.smithi163.stdout:[global] 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: chdir = "" 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: auth supported = cephx 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: filestore xattr use omap = true 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: mon clock drift allowed = 1.000 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: osd crush chooseleaf type = 0 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: auth debug = true 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on old message = true 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on bug = true 2024-01-24T13:57:28.267 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: mon pg warn max object skew = 0 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: # disable pg_autoscaler by default for new pools 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default size = 2 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow primary affinity = true 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow pg remap = true 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on legacy crush tunables = false 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on crush straw calc version zero = false 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on no sortbitwise = false 2024-01-24T13:57:28.268 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on osd down out interval zero = false 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on too few osds = false 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: mon_allow_pool_size_one = true 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: osd default data pool replay window = 5 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: mon allow pool delete = true 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: mon cluster log file level = debug 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: debug asserts on shutdown = true 2024-01-24T13:57:28.269 INFO:teuthology.orchestra.run.smithi163.stdout: mon health detail to clog = false 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: mon host = "172.21.15.29,172.21.15.163,[v2:172.21.15.163:3301,v1:172.21.15.163:6790]" 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on skipped message = False 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: fsid = b1c88d29-1963-443e-8a67-63a822ae75dd 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout:[osd] 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: osd journal size = 100 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub load threshold = 5.0 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub max interval = 600 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: osd recover clone overlap = true 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: osd recovery max chunk = 1048576 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug shutdown = true 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug op order = true 2024-01-24T13:57:28.270 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug verify stray on activate = true 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: osd open classes on start = true 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug pg log writeout = true 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: osd deep scrub update digest min age = 30 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: osd map max advance = 10 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: journal zero on create = true 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: filestore ondisk finisher threads = 3 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: filestore apply finisher threads = 3 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.271 INFO:teuthology.orchestra.run.smithi163.stdout: bdev debug aio = true 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug misdirected ops = true 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore block size = 96636764160 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore fsck on mount = True 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluefs = 20 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluestore = 20 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: debug osd = 20 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: debug rocksdb = 10 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd backfillfull_ratio = 0.85 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd full ratio = 0.9 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd nearfull ratio = 0.8 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: osd failsafe full ratio = 0.95 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: osd objectstore = bluestore 2024-01-24T13:57:28.272 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr] 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout: debug mgr = 20 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout: mgr/telemetry/nag = false 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout: debug client = 20 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout: debug finisher = 20 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout:[mon] 2024-01-24T13:57:28.273 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: debug paxos = 20 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon data avail warn = 5 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon mgr mkfs grace = 240 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd reporter subtree level = osd 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd prime pg temp = true 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: auth mon ticket ttl = 660 # 11m 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: auth service ticket ttl = 240 # 4m 2024-01-24T13:57:28.274 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: # don't complain about insecure global_id in the test suite 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: mon op complaint time = 120 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout:[client] 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: rgw cache enabled = true 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable ops log = true 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable usage log = true 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: client mount timeout = 600 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: debug client = 20 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-01-24T13:57:28.275 INFO:teuthology.orchestra.run.smithi163.stdout: pid file = "" 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.a] 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.b] 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.c] 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.d] 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout:[mds] 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout: debug mds = 20 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout: debug mds balancer = 20 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug frag = True 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug scatterstat = True 2024-01-24T13:57:28.276 INFO:teuthology.orchestra.run.smithi163.stdout: mds op complaint time = 180 2024-01-24T13:57:28.277 INFO:teuthology.orchestra.run.smithi163.stdout: mds verify scatter = True 2024-01-24T13:57:28.277 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2024-01-24T13:57:28.277 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2024-01-24T13:57:28.277 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2024-01-24T13:57:28.277 INFO:teuthology.orchestra.run.smithi029.stdout:[global] 2024-01-24T13:57:28.277 INFO:teuthology.orchestra.run.smithi029.stdout: chdir = "" 2024-01-24T13:57:28.277 INFO:teuthology.orchestra.run.smithi029.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-24T13:57:28.277 INFO:teuthology.orchestra.run.smithi029.stdout: auth supported = cephx 2024-01-24T13:57:28.277 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: filestore xattr use omap = true 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: mon clock drift allowed = 1.000 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: osd crush chooseleaf type = 0 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: auth debug = true 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: ms die on old message = true 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: ms die on bug = true 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: mon pg warn max object skew = 0 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: # disable pg_autoscaler by default for new pools 2024-01-24T13:57:28.278 INFO:teuthology.orchestra.run.smithi029.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: osd pool default size = 2 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd allow primary affinity = true 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd allow pg remap = true 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on legacy crush tunables = false 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on crush straw calc version zero = false 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on no sortbitwise = false 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on osd down out interval zero = false 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on too few osds = false 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: mon_allow_pool_size_one = true 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2024-01-24T13:57:28.279 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: osd default data pool replay window = 5 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: mon allow pool delete = true 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: mon cluster log file level = debug 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: debug asserts on shutdown = true 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: mon health detail to clog = false 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: mon host = "172.21.15.29,172.21.15.163,[v2:172.21.15.163:3301,v1:172.21.15.163:6790]" 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: ms die on skipped message = False 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: fsid = b1c88d29-1963-443e-8a67-63a822ae75dd 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout:[osd] 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: osd journal size = 100 2024-01-24T13:57:28.280 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: osd scrub load threshold = 5.0 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: osd scrub max interval = 600 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: osd recover clone overlap = true 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: osd recovery max chunk = 1048576 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug shutdown = true 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug op order = true 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug verify stray on activate = true 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: osd open classes on start = true 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug pg log writeout = true 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: osd deep scrub update digest min age = 30 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.281 INFO:teuthology.orchestra.run.smithi029.stdout: osd map max advance = 10 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: journal zero on create = true 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: filestore ondisk finisher threads = 3 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: filestore apply finisher threads = 3 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: bdev debug aio = true 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug misdirected ops = true 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore block size = 96636764160 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore fsck on mount = True 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: debug bluefs = 20 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: debug bluestore = 20 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: debug osd = 20 2024-01-24T13:57:28.282 INFO:teuthology.orchestra.run.smithi029.stdout: debug rocksdb = 10 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd backfillfull_ratio = 0.85 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd full ratio = 0.9 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd nearfull ratio = 0.8 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: osd failsafe full ratio = 0.95 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: osd objectstore = bluestore 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: osd op complaint time = 180 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr] 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: debug mgr = 20 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: debug mon = 20 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: debug auth = 20 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min pgs per osd = 4 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min bytes per osd = 10 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: mgr/telemetry/nag = false 2024-01-24T13:57:28.283 INFO:teuthology.orchestra.run.smithi029.stdout: debug client = 20 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: debug finisher = 20 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout:[mon] 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: debug mon = 20 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: debug paxos = 20 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: debug auth = 20 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: mon data avail warn = 5 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: mon mgr mkfs grace = 240 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min pgs per osd = 4 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd reporter subtree level = osd 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd prime pg temp = true 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min bytes per osd = 10 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.284 INFO:teuthology.orchestra.run.smithi029.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: auth mon ticket ttl = 660 # 11m 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: auth service ticket ttl = 240 # 4m 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: # don't complain about insecure global_id in the test suite 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: mon op complaint time = 120 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout:[client] 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: rgw cache enabled = true 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: rgw enable ops log = true 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: rgw enable usage log = true 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: client mount timeout = 600 2024-01-24T13:57:28.285 INFO:teuthology.orchestra.run.smithi029.stdout: debug client = 20 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout: pid file = "" 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout: rados mon op timeout = 900 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout: rados osd op timeout = 900 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.a] 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.b] 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.c] 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.a] 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.c] 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.b] 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.d] 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout:[mds] 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout: debug mds = 20 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout: debug mds balancer = 20 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-01-24T13:57:28.286 INFO:teuthology.orchestra.run.smithi029.stdout: mds debug frag = True 2024-01-24T13:57:28.287 INFO:teuthology.orchestra.run.smithi029.stdout: mds debug scatterstat = True 2024-01-24T13:57:28.287 INFO:teuthology.orchestra.run.smithi029.stdout: mds op complaint time = 180 2024-01-24T13:57:28.287 INFO:teuthology.orchestra.run.smithi029.stdout: mds verify scatter = True 2024-01-24T13:57:28.287 INFO:teuthology.orchestra.run.smithi029.stdout: osd op complaint time = 180 2024-01-24T13:57:28.287 INFO:teuthology.orchestra.run.smithi029.stdout: rados mon op timeout = 900 2024-01-24T13:57:28.287 INFO:teuthology.orchestra.run.smithi029.stdout: rados osd op timeout = 900 2024-01-24T13:57:28.288 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-24T13:57:28.288 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-24T13:57:28.393 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-24T13:57:28.393 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:28.393 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-24T13:57:28.413 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:28.413 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-24T13:57:28.474 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T13:57:28.474 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:28.474 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-24T13:57:28.474 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-24T13:57:28.561 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:28.562 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:57:28.756 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T13:57:28.757 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:28.757 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-24T13:57:28.757 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-24T13:57:28.801 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:28.801 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:57:28.860 INFO:tasks.ceph:Setting up mon nodes... 2024-01-24T13:57:28.861 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-24T13:57:28.861 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-01-24T13:57:28.924 INFO:teuthology.orchestra.run.smithi029.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-01-24T13:57:28.926 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-24T13:57:28.993 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-24T13:57:28.996 INFO:tasks.ceph:Setting up mds nodes... 2024-01-24T13:57:28.996 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-01-24T13:57:29.059 INFO:teuthology.orchestra.run.smithi029.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-01-24T13:57:29.062 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-01-24T13:57:29.093 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-01-24T13:57:29.196 INFO:teuthology.orchestra.run.smithi029.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-01-24T13:57:29.198 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-01-24T13:57:29.229 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-01-24T13:57:29.286 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-01-24T13:57:29.288 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-01-24T13:57:29.317 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-01-24T13:57:29.408 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-01-24T13:57:29.410 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-01-24T13:57:29.437 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-24T13:57:29.437 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-24T13:57:29.488 INFO:teuthology.orchestra.run.smithi029.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-24T13:57:29.504 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-01-24T13:57:29.551 INFO:teuthology.orchestra.run.smithi029.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-01-24T13:57:29.578 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-01-24T13:57:29.618 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-01-24T13:57:29.631 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-01-24T13:57:29.710 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-01-24T13:57:29.723 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-24T13:57:29.723 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi029.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-24T13:57:29.723 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-24T13:57:29.758 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-24T13:57:29.758 INFO:tasks.ceph:role: osd.0 2024-01-24T13:57:29.758 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T13:57:29.758 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-24T13:57:29.831 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T13:57:29.831 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T13:57:29.831 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T13:57:29.832 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T13:57:29.832 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T13:57:29.832 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-01-24T13:57:29.832 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T13:57:29.832 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T13:57:29.832 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T13:57:29.832 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T13:57:29.854 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-01-24T13:57:29.856 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-01-24T13:57:29.856 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-24T13:57:29.944 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-24T13:57:29.981 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-24T13:57:30.057 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-24T13:57:30.057 INFO:tasks.ceph:role: osd.1 2024-01-24T13:57:30.057 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T13:57:30.057 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-24T13:57:30.132 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T13:57:30.132 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T13:57:30.132 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T13:57:30.132 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T13:57:30.132 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T13:57:30.132 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-01-24T13:57:30.132 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T13:57:30.132 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T13:57:30.132 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T13:57:30.132 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T13:57:30.155 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-01-24T13:57:30.157 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-01-24T13:57:30.157 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-24T13:57:30.245 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-24T13:57:30.282 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-24T13:57:30.356 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-24T13:57:30.356 INFO:tasks.ceph:role: osd.2 2024-01-24T13:57:30.356 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T13:57:30.357 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-24T13:57:30.431 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T13:57:30.431 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T13:57:30.431 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T13:57:30.431 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T13:57:30.431 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T13:57:30.431 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-01-24T13:57:30.432 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T13:57:30.432 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T13:57:30.432 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T13:57:30.432 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T13:57:30.454 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-01-24T13:57:30.456 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-01-24T13:57:30.457 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-01-24T13:57:30.544 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-24T13:57:30.580 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-24T13:57:30.655 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-24T13:57:30.655 INFO:tasks.ceph:role: osd.3 2024-01-24T13:57:30.655 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T13:57:30.655 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-24T13:57:30.729 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T13:57:30.729 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T13:57:30.729 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T13:57:30.730 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T13:57:30.730 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T13:57:30.730 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-01-24T13:57:30.730 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T13:57:30.730 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T13:57:30.730 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T13:57:30.730 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T13:57:30.752 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-01-24T13:57:30.755 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-01-24T13:57:30.755 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-01-24T13:57:30.842 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-24T13:57:30.879 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:57:31.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:31.011+0000 7fc8d2fce380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-01-24T13:57:31.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:31.011+0000 7fc8d2fce380 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-01-24T13:57:31.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:31.011+0000 7fc8d2fce380 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-24T13:57:31.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:31.011+0000 7fc8d2fce380 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-24T13:57:31.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:31.011+0000 7fc8d2fce380 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-24T13:57:31.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:31.012+0000 7fc8d2fce380 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-01-24T13:57:33.850 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-24T13:57:33.885 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:57:33.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:33.980+0000 7f667053c380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-01-24T13:57:33.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:33.980+0000 7f667053c380 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-01-24T13:57:33.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:33.980+0000 7f667053c380 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-24T13:57:33.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:33.980+0000 7f667053c380 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-24T13:57:33.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:33.980+0000 7f667053c380 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-24T13:57:33.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:33.980+0000 7f667053c380 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-01-24T13:57:36.828 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-24T13:57:36.863 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:57:36.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:36.958+0000 7f6f8aedc380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-01-24T13:57:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:36.958+0000 7f6f8aedc380 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-01-24T13:57:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:36.958+0000 7f6f8aedc380 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-24T13:57:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:36.958+0000 7f6f8aedc380 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-24T13:57:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:36.958+0000 7f6f8aedc380 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-24T13:57:36.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:36.958+0000 7f6f8aedc380 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-01-24T13:57:39.809 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-24T13:57:39.843 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:57:39.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:39.936+0000 7f59d3a2a380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-01-24T13:57:39.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:39.936+0000 7f59d3a2a380 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-01-24T13:57:39.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:39.937+0000 7f59d3a2a380 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-24T13:57:39.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:39.937+0000 7f59d3a2a380 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-24T13:57:39.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:39.937+0000 7f59d3a2a380 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-24T13:57:39.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:57:39.937+0000 7f59d3a2a380 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-01-24T13:57:42.785 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-24T13:57:42.819 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-01-24T13:57:42.852 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-24T13:57:42.852 INFO:tasks.ceph:role: osd.4 2024-01-24T13:57:42.853 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T13:57:42.853 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-24T13:57:42.930 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T13:57:42.930 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T13:57:42.930 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T13:57:42.930 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T13:57:42.930 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T13:57:42.931 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-01-24T13:57:42.931 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T13:57:42.931 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T13:57:42.931 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T13:57:42.931 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T13:57:43.366 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-01-24T13:57:43.368 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-01-24T13:57:43.368 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-01-24T13:57:43.542 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-01-24T13:57:43.580 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-01-24T13:57:43.651 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-24T13:57:43.651 INFO:tasks.ceph:role: osd.5 2024-01-24T13:57:43.651 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T13:57:43.651 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-24T13:57:43.727 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T13:57:43.727 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T13:57:43.727 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T13:57:43.728 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T13:57:43.728 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T13:57:43.728 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-01-24T13:57:43.728 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T13:57:43.728 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T13:57:43.728 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T13:57:43.728 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T13:57:44.149 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-01-24T13:57:44.151 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-01-24T13:57:44.151 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-01-24T13:57:44.263 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-01-24T13:57:44.296 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-01-24T13:57:44.365 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-24T13:57:44.365 INFO:tasks.ceph:role: osd.6 2024-01-24T13:57:44.365 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T13:57:44.365 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-24T13:57:44.440 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T13:57:44.440 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T13:57:44.440 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T13:57:44.440 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T13:57:44.441 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T13:57:44.441 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-01-24T13:57:44.441 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T13:57:44.441 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T13:57:44.441 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T13:57:44.441 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T13:57:44.882 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-01-24T13:57:44.884 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-01-24T13:57:44.884 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-01-24T13:57:45.045 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-01-24T13:57:45.079 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-01-24T13:57:45.147 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-24T13:57:45.147 INFO:tasks.ceph:role: osd.7 2024-01-24T13:57:45.148 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T13:57:45.148 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-24T13:57:45.222 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T13:57:45.222 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T13:57:45.222 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T13:57:45.222 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T13:57:45.222 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T13:57:45.222 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-01-24T13:57:45.222 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T13:57:45.222 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T13:57:45.222 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T13:57:45.222 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T13:57:45.665 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-01-24T13:57:45.667 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-01-24T13:57:45.667 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-01-24T13:57:45.859 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-01-24T13:57:45.895 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:57:45.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:45.980+0000 7fec4ae76380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-01-24T13:57:45.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:45.980+0000 7fec4ae76380 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-01-24T13:57:45.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:45.980+0000 7fec4ae76380 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-24T13:57:45.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:45.980+0000 7fec4ae76380 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-24T13:57:45.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:45.980+0000 7fec4ae76380 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-24T13:57:45.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:45.981+0000 7fec4ae76380 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-01-24T13:57:48.825 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-01-24T13:57:48.857 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:57:48.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:48.944+0000 7f4985d95380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-01-24T13:57:48.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:48.944+0000 7f4985d95380 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-01-24T13:57:48.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:48.944+0000 7f4985d95380 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-24T13:57:48.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:48.944+0000 7f4985d95380 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-24T13:57:48.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:48.944+0000 7f4985d95380 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-24T13:57:48.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:48.945+0000 7f4985d95380 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-01-24T13:57:51.787 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-01-24T13:57:51.820 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:57:51.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:51.912+0000 7f5698e60380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-01-24T13:57:51.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:51.912+0000 7f5698e60380 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-01-24T13:57:51.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:51.912+0000 7f5698e60380 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-24T13:57:51.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:51.912+0000 7f5698e60380 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-24T13:57:51.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:51.912+0000 7f5698e60380 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-24T13:57:51.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:51.913+0000 7f5698e60380 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-01-24T13:57:54.752 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-01-24T13:57:54.782 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:57:54.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:54.866+0000 7f9f91c16380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-01-24T13:57:54.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:54.867+0000 7f9f91c16380 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-01-24T13:57:54.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:54.867+0000 7f9f91c16380 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-24T13:57:54.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:54.867+0000 7f9f91c16380 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-24T13:57:54.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:54.867+0000 7f9f91c16380 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-24T13:57:54.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:57:54.868+0000 7f9f91c16380 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-01-24T13:57:56.099 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T13:57:56.102 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T13:57:57.736 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-01-24T13:57:57.765 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-24T13:57:57.766 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:57.766 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-01-24T13:57:57.801 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:57.801 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-01-24T13:57:57.876 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:57.876 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-01-24T13:57:57.950 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:57.950 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-24T13:57:58.026 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:58.026 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-24T13:57:58.101 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:58.101 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-24T13:57:58.175 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:58.175 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-24T13:57:58.249 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:58.249 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-24T13:57:58.281 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:58.281 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-01-24T13:57:58.352 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:58.352 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-01-24T13:57:58.422 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:58.422 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-01-24T13:57:58.492 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:58.492 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-01-24T13:57:58.561 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:58.561 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-01-24T13:57:58.628 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:58.628 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-01-24T13:57:58.695 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:58.695 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-24T13:57:58.717 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:57:58.717 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-01-24T13:57:58.781 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:58.781 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-01-24T13:57:58.798 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:57:58.798 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-01-24T13:57:58.855 INFO:tasks.ceph:Adding keys to all mons... 2024-01-24T13:57:58.856 DEBUG:teuthology.orchestra.run.smithi029:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-24T13:57:58.858 DEBUG:teuthology.orchestra.run.smithi163:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-24T13:57:58.899 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr.y] 2024-01-24T13:57:58.899 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA== 2024-01-24T13:57:58.899 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.a] 2024-01-24T13:57:58.899 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew== 2024-01-24T13:57:58.899 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.c] 2024-01-24T13:57:58.899 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw== 2024-01-24T13:57:58.899 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.0] 2024-01-24T13:57:58.899 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ== 2024-01-24T13:57:58.899 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.1] 2024-01-24T13:57:58.899 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA== 2024-01-24T13:57:58.900 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.2] 2024-01-24T13:57:58.900 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw== 2024-01-24T13:57:58.900 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.3] 2024-01-24T13:57:58.900 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ== 2024-01-24T13:57:58.900 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr.x] 2024-01-24T13:57:58.900 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw== 2024-01-24T13:57:58.900 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.b] 2024-01-24T13:57:58.900 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ== 2024-01-24T13:57:58.900 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.d] 2024-01-24T13:57:58.900 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ== 2024-01-24T13:57:58.900 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.4] 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw== 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.5] 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ== 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.6] 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A== 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.7] 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ== 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw== 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout:[client.2] 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g== 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout:[client.1] 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA== 2024-01-24T13:57:58.901 INFO:teuthology.orchestra.run.smithi029.stdout:[client.3] 2024-01-24T13:57:58.902 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ== 2024-01-24T13:57:58.921 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.y] 2024-01-24T13:57:58.921 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA== 2024-01-24T13:57:58.921 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2024-01-24T13:57:58.921 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew== 2024-01-24T13:57:58.921 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2024-01-24T13:57:58.921 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw== 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.0] 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ== 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.1] 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA== 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.2] 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw== 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.3] 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ== 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.x] 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw== 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ== 2024-01-24T13:57:58.922 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.d] 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ== 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.4] 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw== 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.5] 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ== 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.6] 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A== 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.7] 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ== 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout:[client.0] 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw== 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout:[client.2] 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g== 2024-01-24T13:57:58.923 INFO:teuthology.orchestra.run.smithi163.stdout:[client.1] 2024-01-24T13:57:58.924 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA== 2024-01-24T13:57:58.924 INFO:teuthology.orchestra.run.smithi163.stdout:[client.3] 2024-01-24T13:57:58.924 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ== 2024-01-24T13:57:58.925 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-24T13:57:58.946 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-24T13:57:59.013 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-24T13:57:59.015 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-24T13:57:59.073 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-24T13:57:59.076 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-24T13:57:59.133 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.135 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.189 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.191 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.243 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.245 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.297 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.351 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-24T13:57:59.354 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-24T13:57:59.406 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-24T13:57:59.408 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-24T13:57:59.464 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-24T13:57:59.466 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-24T13:57:59.521 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.523 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.575 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.577 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.628 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.631 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.683 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.685 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T13:57:59.737 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-24T13:57:59.739 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-24T13:57:59.793 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-24T13:57:59.795 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-24T13:57:59.851 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-24T13:57:59.853 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-24T13:57:59.908 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-24T13:57:59.910 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-24T13:57:59.965 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-24T13:57:59.965 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-24T13:57:59.995 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-24T13:58:00.097 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-24T13:58:00.128 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-01-24T13:58:00.157 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-24T13:58:00.251 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-01-24T13:58:00.279 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-01-24T13:58:00.345 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-24T13:58:00.439 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-01-24T13:58:00.465 DEBUG:teuthology.orchestra.run.smithi029:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:58:00.468 DEBUG:teuthology.orchestra.run.smithi163:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-24T13:58:00.522 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-24T13:58:00.522 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-24T13:58:00.523 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-24T13:58:00.529 INFO:tasks.ceph.mon.a:Started 2024-01-24T13:58:00.529 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-24T13:58:00.530 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-24T13:58:00.565 INFO:tasks.ceph.mon.b:Started 2024-01-24T13:58:00.565 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-24T13:58:00.565 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-01-24T13:58:00.567 INFO:tasks.ceph.mon.c:Started 2024-01-24T13:58:00.568 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-24T13:58:00.568 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-01-24T13:58:00.568 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-01-24T13:58:00.570 INFO:tasks.ceph.mgr.y:Started 2024-01-24T13:58:00.571 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-24T13:58:00.571 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-01-24T13:58:00.573 INFO:tasks.ceph.mgr.x:Started 2024-01-24T13:58:00.573 INFO:tasks.ceph:Setting crush tunables to default 2024-01-24T13:58:00.573 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd crush tunables default 2024-01-24T13:58:00.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.707+0000 7f0342cb8700 1 Processor -- start 2024-01-24T13:58:00.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.707+0000 7f0342cb8700 1 -- start start 2024-01-24T13:58:00.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.708+0000 7f0342cb8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c05e1f0 0x7f033c155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:00.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.708+0000 7f0342cb8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c158960 con 0x7f033c14e530 2024-01-24T13:58:00.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.708+0000 7f0342cb8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c155650 con 0x7f033c0c0e20 2024-01-24T13:58:00.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.708+0000 7f0342cb8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0c1b90 con 0x7f033c05e1f0 2024-01-24T13:58:00.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.709+0000 7f0340a54700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f033c14e530 0x7f033c0c0710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36326/0 (socket says 172.21.15.29:36326) 2024-01-24T13:58:00.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.709+0000 7f0340a54700 1 -- 172.21.15.29:0/327418691 learned_addr learned my addr 172.21.15.29:0/327418691 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:00.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.709+0000 7f0341255700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c05e1f0 0x7f033c155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:00.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.709+0000 7f0341255700 1 -- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c05e1f0 msgr2=0x7f033c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-24T13:58:00.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.709+0000 7f0341255700 1 -- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c05e1f0 msgr2=0x7f033c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-24T13:58:00.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.709+0000 7f0341255700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c05e1f0 0x7f033c155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-01-24T13:58:00.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.709+0000 7f0341255700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c05e1f0 0x7f033c155160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T13:58:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.894+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234149685 0 0) 0x7f033c158960 con 0x7f033c14e530 2024-01-24T13:58:00.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.894+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03240033f0 con 0x7f033c14e530 2024-01-24T13:58:00.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2044433337 0 0) 0x7f03240033f0 con 0x7f033c14e530 2024-01-24T13:58:00.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c05e1f0 msgr2=0x7f033c155160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c05e1f0 0x7f033c155160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 >> v1:172.21.15.163:6789/0 conn(0x7f033c0c0e20 legacy=0x7f033c068a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c157d50 con 0x7f033c14e530 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f032c001470 con 0x7f033c14e530 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 >> v1:172.21.15.29:6789/0 conn(0x7f033c14e530 legacy=0x7f033c0c0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0324003910 0x7f0324003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c05e1f0 0x7f0324004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0324004b80 0x7f0324005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0c1b90 con 0x7f0324003910 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032400ba60 con 0x7f0324004b80 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.895+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032400bc20 con 0x7f033c05e1f0 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.896+0000 7f0340a54700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0324003910 0x7f0324003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.896+0000 7f033bfff700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c05e1f0 0x7f0324004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:00.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.896+0000 7f0341255700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0324004b80 0x7f0324005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.896+0000 7f0340a54700 1 -- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c05e1f0 msgr2=0x7f0324004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.896+0000 7f0340a54700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c05e1f0 0x7f0324004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.896+0000 7f0340a54700 1 -- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0324004b80 msgr2=0x7f0324005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.896+0000 7f0340a54700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0324004b80 0x7f0324005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.896+0000 7f0340a54700 1 -- 172.21.15.29:0/327418691 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03240033f0 con 0x7f0324003910 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.896+0000 7f0340a54700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0324003910 0x7f0324003d30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f032c00c190 tx=0x7f032c001e50).ready entity=mon.0 client_cookie=637ed9e8ac2ea502 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.896+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c007a20 con 0x7f0324003910 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.896+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c00cac0 con 0x7f0324003910 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.896+0000 7f033b7fe700 1 -- 172.21.15.29:0/327418691 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00ce40 con 0x7f0324003910 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.897+0000 7f0342cb8700 1 -- 172.21.15.29:0/327418691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0324003910 msgr2=0x7f0324003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.897+0000 7f0342cb8700 1 --2- 172.21.15.29:0/327418691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0324003910 0x7f0324003d30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f032c00c190 tx=0x7f032c001e50).stop 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.897+0000 7f0341255700 1 -- 172.21.15.29:0/327418691 reap_dead start 2024-01-24T13:58:00.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.897+0000 7f0342cb8700 1 -- 172.21.15.29:0/327418691 shutdown_connections 2024-01-24T13:58:00.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.897+0000 7f0342cb8700 1 -- 172.21.15.29:0/327418691 >> 172.21.15.29:0/327418691 conn(0x7f033c0c7d40 msgr2=0x7f033c0c3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:00.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.897+0000 7f0342cb8700 1 -- 172.21.15.29:0/327418691 shutdown_connections 2024-01-24T13:58:00.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.897+0000 7f0342cb8700 1 -- 172.21.15.29:0/327418691 wait complete. 2024-01-24T13:58:00.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.899+0000 7f0342cb8700 1 Processor -- start 2024-01-24T13:58:00.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.899+0000 7f0342cb8700 1 -- start start 2024-01-24T13:58:00.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.900+0000 7f0342cb8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f033c13bbc0 0x7f033c167990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:00.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.900+0000 7f0342cb8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c13bfa0 0x7f033c167e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.900+0000 7f0342cb8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c0c5b60 0x7f033c0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.900+0000 7f0342cb8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0c64b0 con 0x7f033c13bbc0 2024-01-24T13:58:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.900+0000 7f0342cb8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c168500 con 0x7f033c13bfa0 2024-01-24T13:58:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.900+0000 7f0342cb8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c1687e0 con 0x7f033c0c5b60 2024-01-24T13:58:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.900+0000 7f0340a54700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f033c13bbc0 0x7f033c167990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.900+0000 7f033bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c13bfa0 0x7f033c167e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.900+0000 7f0340a54700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f033c13bbc0 0x7f033c167990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50420/0 (socket says 172.21.15.29:50420) 2024-01-24T13:58:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.900+0000 7f0340a54700 1 -- 172.21.15.29:0/4235050556 learned_addr learned my addr 172.21.15.29:0/4235050556 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.900+0000 7f0341255700 1 --2- 172.21.15.29:0/4235050556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c0c5b60 0x7f033c0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.901+0000 7f033bfff700 1 -- 172.21.15.29:0/4235050556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c0c5b60 msgr2=0x7f033c0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.901+0000 7f033bfff700 1 --2- 172.21.15.29:0/4235050556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c0c5b60 0x7f033c0c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:00.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.901+0000 7f033bfff700 1 -- 172.21.15.29:0/4235050556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f033c13bbc0 msgr2=0x7f033c167990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:00.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.901+0000 7f033bfff700 1 --2- 172.21.15.29:0/4235050556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f033c13bbc0 0x7f033c167990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:00.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.901+0000 7f033bfff700 1 -- 172.21.15.29:0/4235050556 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c165590 con 0x7f033c13bfa0 2024-01-24T13:58:00.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.901+0000 7f0340a54700 1 --2- 172.21.15.29:0/4235050556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f033c13bbc0 0x7f033c167990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:00.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.901+0000 7f033bfff700 1 --2- 172.21.15.29:0/4235050556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c13bfa0 0x7f033c167e80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f033c0c1220 tx=0x7f0330007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:00.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.905+0000 7f0338ff9700 1 -- 172.21.15.29:0/4235050556 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0330007240 con 0x7f033c13bfa0 2024-01-24T13:58:00.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.905+0000 7f0338ff9700 1 -- 172.21.15.29:0/4235050556 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03300073a0 con 0x7f033c13bfa0 2024-01-24T13:58:00.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.905+0000 7f0342cb8700 1 -- 172.21.15.29:0/4235050556 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033c165870 con 0x7f033c13bfa0 2024-01-24T13:58:00.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.905+0000 7f0338ff9700 1 -- 172.21.15.29:0/4235050556 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300066f0 con 0x7f033c13bfa0 2024-01-24T13:58:00.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.905+0000 7f0342cb8700 1 -- 172.21.15.29:0/4235050556 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c14a2f0 con 0x7f033c13bfa0 2024-01-24T13:58:00.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.905+0000 7f0338ff9700 1 -- 172.21.15.29:0/4235050556 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f0330006ae0 con 0x7f033c13bfa0 2024-01-24T13:58:00.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.905+0000 7f0338ff9700 1 -- 172.21.15.29:0/4235050556 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f033001fba0 con 0x7f033c13bfa0 2024-01-24T13:58:00.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.907+0000 7f0342cb8700 1 -- 172.21.15.29:0/4235050556 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c165d90 con 0x7f033c13bfa0 2024-01-24T13:58:00.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.909+0000 7f0338ff9700 1 -- 172.21.15.29:0/4235050556 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f033c165d90 con 0x7f033c13bfa0 2024-01-24T13:58:00.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.992+0000 7f0342cb8700 1 -- 172.21.15.29:0/4235050556 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f033c0475e0 con 0x7f033c13bfa0 2024-01-24T13:58:00.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.997+0000 7f0338ff9700 1 -- 172.21.15.29:0/4235050556 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0330015410 con 0x7f033c13bfa0 2024-01-24T13:58:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.999+0000 7f0342cb8700 1 -- 172.21.15.29:0/4235050556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c13bfa0 msgr2=0x7f033c167e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.999+0000 7f0342cb8700 1 --2- 172.21.15.29:0/4235050556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c13bfa0 0x7f033c167e80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f033c0c1220 tx=0x7f0330007c00).stop 2024-01-24T13:58:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.999+0000 7f0342cb8700 1 -- 172.21.15.29:0/4235050556 shutdown_connections 2024-01-24T13:58:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.999+0000 7f0342cb8700 1 --2- 172.21.15.29:0/4235050556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f033c0c5b60 0x7f033c0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.999+0000 7f0342cb8700 1 --2- 172.21.15.29:0/4235050556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f033c13bbc0 0x7f033c167990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.999+0000 7f0342cb8700 1 --2- 172.21.15.29:0/4235050556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c13bfa0 0x7f033c167e80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:00.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.999+0000 7f0342cb8700 1 -- 172.21.15.29:0/4235050556 >> 172.21.15.29:0/4235050556 conn(0x7f033c0c7d40 msgr2=0x7f033c14daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:01.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.999+0000 7f0342cb8700 1 -- 172.21.15.29:0/4235050556 shutdown_connections 2024-01-24T13:58:01.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:00.999+0000 7f0342cb8700 1 -- 172.21.15.29:0/4235050556 wait complete. 2024-01-24T13:58:01.000 INFO:teuthology.orchestra.run.smithi029.stderr:adjusted tunables profile to default 2024-01-24T13:58:01.011 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-24T13:58:01.011 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:58:01.011 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-24T13:58:01.035 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:01.035+0000 7fc516bdc680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T13:58:01.041 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:58:01.041 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-24T13:58:01.085 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:01.086+0000 7f1a5db3b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T13:58:01.103 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:01.104+0000 7fc516bdc680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T13:58:01.112 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:58:01.112 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-24T13:58:01.166 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:01.167+0000 7f1a5db3b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T13:58:01.182 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-01-24T13:58:01.182 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-24T13:58:01.252 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:58:01.252 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-01-24T13:58:01.279 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:58:01.279 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-01-24T13:58:01.346 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:58:01.346 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-01-24T13:58:01.413 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-01-24T13:58:01.413 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-01-24T13:58:01.480 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 12bc8013-1170-4506-9682-f768047e31e4 0 2024-01-24T13:58:01.584 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:01.584+0000 7fc516bdc680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T13:58:01.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.638+0000 7f88a2442700 1 Processor -- start 2024-01-24T13:58:01.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.638+0000 7f88a2442700 1 -- start start 2024-01-24T13:58:01.640 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f88a2442700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f889c14b140 0x7f889c155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:01.640 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f88a2442700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c062ec0 0x7f889c0bce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f88a2442700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889c154cd0 con 0x7f889c0bca30 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f88a2442700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889c1546a0 con 0x7f889c14b140 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f88a2442700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0c81e0 con 0x7f889c062ec0 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f88a09df700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c062ec0 0x7f889c0bce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f889b7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f889c0bca30 0x7f889c0627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57466/0 (socket says 172.21.15.163:57466) 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f889b7fe700 1 -- 172.21.15.163:0/179476676 learned_addr learned my addr 172.21.15.163:0/179476676 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f889bfff700 1 --2- 172.21.15.163:0/179476676 >> v2:172.21.15.29:3300/0 conn(0x7f889c14b140 0x7f889c155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f88a09df700 1 -- 172.21.15.163:0/179476676 >> v2:172.21.15.29:3300/0 conn(0x7f889c14b140 msgr2=0x7f889c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f88a09df700 1 --2- 172.21.15.163:0/179476676 >> v2:172.21.15.29:3300/0 conn(0x7f889c14b140 0x7f889c155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f88a09df700 1 -- 172.21.15.163:0/179476676 >> v1:172.21.15.163:6789/0 conn(0x7f889c0bca30 legacy=0x7f889c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f889affd700 1 -- 172.21.15.163:0/179476676 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774817092 0 0) 0x7f889c154cd0 con 0x7f889c0bca30 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f88a09df700 1 -- 172.21.15.163:0/179476676 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c0c84c0 con 0x7f889c062ec0 2024-01-24T13:58:01.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f88a09df700 1 --2- 172.21.15.163:0/179476676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c062ec0 0x7f889c0bce90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f8894009e60 tx=0x7f8894009780).ready entity=mon.2 client_cookie=9ebbc7ec79983e3a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:01.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.639+0000 7f889affd700 1 -- 172.21.15.163:0/179476676 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889400e070 con 0x7f889c062ec0 2024-01-24T13:58:01.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.640+0000 7f889affd700 1 -- 172.21.15.163:0/179476676 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889400bc30 con 0x7f889c062ec0 2024-01-24T13:58:01.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.640+0000 7f889affd700 1 -- 172.21.15.163:0/179476676 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894012410 con 0x7f889c062ec0 2024-01-24T13:58:01.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.640+0000 7f88a2442700 1 -- 172.21.15.163:0/179476676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c062ec0 msgr2=0x7f889c0bce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:01.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.640+0000 7f88a2442700 1 --2- 172.21.15.163:0/179476676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c062ec0 0x7f889c0bce90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f8894009e60 tx=0x7f8894009780).stop 2024-01-24T13:58:01.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.640+0000 7f88a2442700 1 -- 172.21.15.163:0/179476676 shutdown_connections 2024-01-24T13:58:01.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.640+0000 7f88a2442700 1 --2- 172.21.15.163:0/179476676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c062ec0 0x7f889c0bce90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.640+0000 7f88a2442700 1 --2- 172.21.15.163:0/179476676 >> v2:172.21.15.29:3300/0 conn(0x7f889c14b140 0x7f889c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.640+0000 7f88a2442700 1 -- 172.21.15.163:0/179476676 >> 172.21.15.163:0/179476676 conn(0x7f889c064390 msgr2=0x7f889c0c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:01.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.640+0000 7f88a2442700 1 -- 172.21.15.163:0/179476676 shutdown_connections 2024-01-24T13:58:01.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.640+0000 7f88a2442700 1 -- 172.21.15.163:0/179476676 wait complete. 2024-01-24T13:58:01.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.642+0000 7f88a2442700 1 Processor -- start 2024-01-24T13:58:01.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.642+0000 7f88a2442700 1 -- start start 2024-01-24T13:58:01.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a2442700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f889c062ec0 0x7f889c0a9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:01.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a2442700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f889c0bca30 0x7f889c14f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:01.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a2442700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c14b140 0x7f889c1512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:01.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a2442700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0ab470 con 0x7f889c0bca30 2024-01-24T13:58:01.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a2442700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0a9750 con 0x7f889c062ec0 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a2442700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c14f900 con 0x7f889c14b140 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a09df700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c14b140 0x7f889c1512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f889bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f889c062ec0 0x7f889c0a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f889bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f889c062ec0 0x7f889c0a9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53464/0 (socket says 172.21.15.163:53464) 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a09df700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c14b140 0x7f889c1512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:32928/0 (socket says 172.21.15.163:32928) 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f889bfff700 1 -- 172.21.15.163:0/1408396132 learned_addr learned my addr 172.21.15.163:0/1408396132 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f889b7fe700 1 --2- 172.21.15.163:0/1408396132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f889c0bca30 0x7f889c14f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a09df700 1 -- 172.21.15.163:0/1408396132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f889c062ec0 msgr2=0x7f889c0a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a09df700 1 --2- 172.21.15.163:0/1408396132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f889c062ec0 0x7f889c0a9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a09df700 1 -- 172.21.15.163:0/1408396132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f889c0bca30 msgr2=0x7f889c14f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a09df700 1 --2- 172.21.15.163:0/1408396132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f889c0bca30 0x7f889c14f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f88a09df700 1 -- 172.21.15.163:0/1408396132 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894009a20 con 0x7f889c14b140 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.643+0000 7f889bfff700 1 --2- 172.21.15.163:0/1408396132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f889c062ec0 0x7f889c0a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:01.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.644+0000 7f88a09df700 1 --2- 172.21.15.163:0/1408396132 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c14b140 0x7f889c1512a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f8894014000 tx=0x7f8894009780).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:01.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.644+0000 7f8883fff700 1 -- 172.21.15.163:0/1408396132 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894012410 con 0x7f889c14b140 2024-01-24T13:58:01.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.644+0000 7f8883fff700 1 -- 172.21.15.163:0/1408396132 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889400b6f0 con 0x7f889c14b140 2024-01-24T13:58:01.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.644+0000 7f88a2442700 1 -- 172.21.15.163:0/1408396132 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889c0a99d0 con 0x7f889c14b140 2024-01-24T13:58:01.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.644+0000 7f8883fff700 1 -- 172.21.15.163:0/1408396132 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894011c80 con 0x7f889c14b140 2024-01-24T13:58:01.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.644+0000 7f88a2442700 1 -- 172.21.15.163:0/1408396132 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c0aae20 con 0x7f889c14b140 2024-01-24T13:58:01.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.644+0000 7f8883fff700 1 -- 172.21.15.163:0/1408396132 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f8894003040 con 0x7f889c14b140 2024-01-24T13:58:01.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.644+0000 7f8883fff700 1 -- 172.21.15.163:0/1408396132 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f8894013e00 con 0x7f889c14b140 2024-01-24T13:58:01.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.645+0000 7f88a2442700 1 -- 172.21.15.163:0/1408396132 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889c0475e0 con 0x7f889c14b140 2024-01-24T13:58:01.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.647+0000 7f8883fff700 1 -- 172.21.15.163:0/1408396132 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f8894017070 con 0x7f889c14b140 2024-01-24T13:58:01.653 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:01.654+0000 7f1a5db3b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T13:58:01.742 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.742+0000 7f88a2442700 1 -- 172.21.15.163:0/1408396132 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "12bc8013-1170-4506-9682-f768047e31e4", "id": 0} v 0) v1 -- 0x7f889c06a440 con 0x7f889c14b140 2024-01-24T13:58:01.747 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.748+0000 7f8883fff700 1 -- 172.21.15.163:0/1408396132 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "12bc8013-1170-4506-9682-f768047e31e4", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f88940360b0 con 0x7f889c14b140 2024-01-24T13:58:01.747 INFO:teuthology.orchestra.run.smithi163.stdout:0 2024-01-24T13:58:01.749 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.749+0000 7f88a2442700 1 -- 172.21.15.163:0/1408396132 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c14b140 msgr2=0x7f889c1512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:01.749 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.749+0000 7f88a2442700 1 --2- 172.21.15.163:0/1408396132 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c14b140 0x7f889c1512a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f8894014000 tx=0x7f8894009780).stop 2024-01-24T13:58:01.749 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.749+0000 7f88a2442700 1 -- 172.21.15.163:0/1408396132 shutdown_connections 2024-01-24T13:58:01.749 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.749+0000 7f88a2442700 1 --2- 172.21.15.163:0/1408396132 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f889c14b140 0x7f889c1512a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.749 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.749+0000 7f88a2442700 1 --2- 172.21.15.163:0/1408396132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f889c062ec0 0x7f889c0a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.749 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.749+0000 7f88a2442700 1 --2- 172.21.15.163:0/1408396132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f889c0bca30 0x7f889c14f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.749 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.749+0000 7f88a2442700 1 -- 172.21.15.163:0/1408396132 >> 172.21.15.163:0/1408396132 conn(0x7f889c064390 msgr2=0x7f889c0c3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:01.749 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.749+0000 7f88a2442700 1 -- 172.21.15.163:0/1408396132 shutdown_connections 2024-01-24T13:58:01.749 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.749+0000 7f88a2442700 1 -- 172.21.15.163:0/1408396132 wait complete. 2024-01-24T13:58:01.760 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 998bacf2-ab7f-4cca-8df3-3d88ada9d0d4 1 2024-01-24T13:58:01.879 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.879+0000 7fa87551d700 1 Processor -- start 2024-01-24T13:58:01.879 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.879+0000 7fa87551d700 1 -- start start 2024-01-24T13:58:01.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.880+0000 7fa87551d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8700663b0 0x7fa870152f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:01.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.880+0000 7fa87551d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa8700be4b0 0x7fa870069b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:01.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.880+0000 7fa87551d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa870069690 con 0x7fa87006a010 2024-01-24T13:58:01.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.880+0000 7fa87551d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700be930 con 0x7fa8700be4b0 2024-01-24T13:58:01.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.880+0000 7fa87551d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700bca30 con 0x7fa8700663b0 2024-01-24T13:58:01.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.880+0000 7fa86effd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8700663b0 0x7fa870152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:01.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.880+0000 7fa86effd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8700663b0 0x7fa870152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53468/0 (socket says 172.21.15.163:53468) 2024-01-24T13:58:01.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.880+0000 7fa86effd700 1 -- 172.21.15.163:0/3244441467 learned_addr learned my addr 172.21.15.163:0/3244441467 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:01.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.881+0000 7fa86e7fc700 1 --2- 172.21.15.163:0/3244441467 >> v2:172.21.15.29:3300/0 conn(0x7fa8700be4b0 0x7fa870069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:01.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.881+0000 7fa86effd700 1 -- 172.21.15.163:0/3244441467 >> v2:172.21.15.29:3300/0 conn(0x7fa8700be4b0 msgr2=0x7fa870069b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.881+0000 7fa86effd700 1 --2- 172.21.15.163:0/3244441467 >> v2:172.21.15.29:3300/0 conn(0x7fa8700be4b0 0x7fa870069b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.881+0000 7fa86effd700 1 -- 172.21.15.163:0/3244441467 >> v1:172.21.15.29:6789/0 conn(0x7fa87006a010 legacy=0x7fa870065420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.881+0000 7fa86effd700 1 -- 172.21.15.163:0/3244441467 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700bcc40 con 0x7fa8700663b0 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.881+0000 7fa86effd700 1 --2- 172.21.15.163:0/3244441467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8700663b0 0x7fa870152f20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fa8700c4350 tx=0x7fa8600090f0).ready entity=mon.1 client_cookie=a3f2fc1c3b41e974 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.881+0000 7fa86dffb700 1 -- 172.21.15.163:0/3244441467 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000b520 con 0x7fa8700663b0 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.881+0000 7fa86dffb700 1 -- 172.21.15.163:0/3244441467 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86000b680 con 0x7fa8700663b0 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.881+0000 7fa86dffb700 1 -- 172.21.15.163:0/3244441467 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000ba20 con 0x7fa8700663b0 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.881+0000 7fa87551d700 1 -- 172.21.15.163:0/3244441467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8700663b0 msgr2=0x7fa870152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.881+0000 7fa87551d700 1 --2- 172.21.15.163:0/3244441467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8700663b0 0x7fa870152f20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fa8700c4350 tx=0x7fa8600090f0).stop 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.882+0000 7fa87551d700 1 -- 172.21.15.163:0/3244441467 shutdown_connections 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.882+0000 7fa87551d700 1 --2- 172.21.15.163:0/3244441467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8700663b0 0x7fa870152f20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.882+0000 7fa87551d700 1 --2- 172.21.15.163:0/3244441467 >> v2:172.21.15.29:3300/0 conn(0x7fa8700be4b0 0x7fa870069b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.882+0000 7fa87551d700 1 -- 172.21.15.163:0/3244441467 >> 172.21.15.163:0/3244441467 conn(0x7fa870064390 msgr2=0x7fa8700c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:01.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.882+0000 7fa87551d700 1 -- 172.21.15.163:0/3244441467 shutdown_connections 2024-01-24T13:58:01.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.882+0000 7fa87551d700 1 -- 172.21.15.163:0/3244441467 wait complete. 2024-01-24T13:58:01.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.883+0000 7fa87551d700 1 Processor -- start 2024-01-24T13:58:01.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.883+0000 7fa87551d700 1 -- start start 2024-01-24T13:58:01.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa87551d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700663b0 0x7fa87013a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:01.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa87551d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87006a010 0x7fa8700c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa87551d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8700be4b0 0x7fa8700c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa87551d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700c5100 con 0x7fa8700be4b0 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa87551d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa870139ce0 con 0x7fa87006a010 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa87551d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700c05d0 con 0x7fa8700663b0 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86effd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700663b0 0x7fa87013a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86e7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87006a010 0x7fa8700c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86effd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700663b0 0x7fa87013a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:32930/0 (socket says 172.21.15.163:32930) 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86effd700 1 -- 172.21.15.163:0/4280020757 learned_addr learned my addr 172.21.15.163:0/4280020757 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86f7fe700 1 --2- 172.21.15.163:0/4280020757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8700be4b0 0x7fa8700c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86effd700 1 -- 172.21.15.163:0/4280020757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87006a010 msgr2=0x7fa8700c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86effd700 1 --2- 172.21.15.163:0/4280020757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87006a010 0x7fa8700c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86effd700 1 -- 172.21.15.163:0/4280020757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8700be4b0 msgr2=0x7fa8700c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86effd700 1 --2- 172.21.15.163:0/4280020757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8700be4b0 0x7fa8700c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86effd700 1 -- 172.21.15.163:0/4280020757 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860009370 con 0x7fa8700663b0 2024-01-24T13:58:01.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86e7fc700 1 --2- 172.21.15.163:0/4280020757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87006a010 0x7fa8700c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:01.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86f7fe700 1 --2- 172.21.15.163:0/4280020757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8700be4b0 0x7fa8700c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:01.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa86effd700 1 --2- 172.21.15.163:0/4280020757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700663b0 0x7fa87013a780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fa860002f30 tx=0x7fa860017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:01.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa856ffd700 1 -- 172.21.15.163:0/4280020757 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860006990 con 0x7fa8700663b0 2024-01-24T13:58:01.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa856ffd700 1 -- 172.21.15.163:0/4280020757 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860000bc0 con 0x7fa8700663b0 2024-01-24T13:58:01.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa87551d700 1 -- 172.21.15.163:0/4280020757 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8700bfce0 con 0x7fa8700663b0 2024-01-24T13:58:01.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa856ffd700 1 -- 172.21.15.163:0/4280020757 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86001c420 con 0x7fa8700663b0 2024-01-24T13:58:01.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.884+0000 7fa87551d700 1 -- 172.21.15.163:0/4280020757 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8700c0260 con 0x7fa8700663b0 2024-01-24T13:58:01.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.885+0000 7fa856ffd700 1 -- 172.21.15.163:0/4280020757 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fa86001ca70 con 0x7fa8700663b0 2024-01-24T13:58:01.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.885+0000 7fa856ffd700 1 -- 172.21.15.163:0/4280020757 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fa860025450 con 0x7fa8700663b0 2024-01-24T13:58:01.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.885+0000 7fa87551d700 1 -- 172.21.15.163:0/4280020757 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8700475e0 con 0x7fa8700663b0 2024-01-24T13:58:01.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.888+0000 7fa856ffd700 1 -- 172.21.15.163:0/4280020757 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7fa860044e50 con 0x7fa8700663b0 2024-01-24T13:58:01.968 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.968+0000 7fa87551d700 1 -- 172.21.15.163:0/4280020757 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "998bacf2-ab7f-4cca-8df3-3d88ada9d0d4", "id": 1} v 0) v1 -- 0x7fa8700bcf20 con 0x7fa8700663b0 2024-01-24T13:58:01.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.977+0000 7fa856ffd700 1 -- 172.21.15.163:0/4280020757 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "998bacf2-ab7f-4cca-8df3-3d88ada9d0d4", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa860033030 con 0x7fa8700663b0 2024-01-24T13:58:01.978 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-01-24T13:58:01.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.978+0000 7fa87551d700 1 -- 172.21.15.163:0/4280020757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700663b0 msgr2=0x7fa87013a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:01.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.978+0000 7fa87551d700 1 --2- 172.21.15.163:0/4280020757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700663b0 0x7fa87013a780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fa860002f30 tx=0x7fa860017000).stop 2024-01-24T13:58:01.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.978+0000 7fa87551d700 1 -- 172.21.15.163:0/4280020757 shutdown_connections 2024-01-24T13:58:01.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.978+0000 7fa87551d700 1 --2- 172.21.15.163:0/4280020757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8700be4b0 0x7fa8700c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.978+0000 7fa87551d700 1 --2- 172.21.15.163:0/4280020757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700663b0 0x7fa87013a780 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.978+0000 7fa87551d700 1 --2- 172.21.15.163:0/4280020757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87006a010 0x7fa8700c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:01.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.978+0000 7fa87551d700 1 -- 172.21.15.163:0/4280020757 >> 172.21.15.163:0/4280020757 conn(0x7fa870064390 msgr2=0x7fa8700627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:01.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.978+0000 7fa87551d700 1 -- 172.21.15.163:0/4280020757 shutdown_connections 2024-01-24T13:58:01.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:01.978+0000 7fa87551d700 1 -- 172.21.15.163:0/4280020757 wait complete. 2024-01-24T13:58:01.989 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new f7a975b6-cab3-43be-a47b-178d2b65f18e 2 2024-01-24T13:58:02.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.107+0000 7fc1bc063700 1 Processor -- start 2024-01-24T13:58:02.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.107+0000 7fc1bc063700 1 -- start start 2024-01-24T13:58:02.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1bc063700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc1b4155310 0x7fc1b41556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1bc063700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b40ac8a0 con 0x7fc1b40aac00 2024-01-24T13:58:02.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1bc063700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b40acb80 con 0x7fc1b40c4420 2024-01-24T13:58:02.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1bc063700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b40abc60 con 0x7fc1b4155310 2024-01-24T13:58:02.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1b95fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc1b40c4420 0x7fc1b40aa4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57468/0 (socket says 172.21.15.163:57468) 2024-01-24T13:58:02.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1b95fe700 1 -- 172.21.15.163:0/19943789 learned_addr learned my addr 172.21.15.163:0/19943789 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-01-24T13:58:02.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1b9dff700 1 --2- 172.21.15.163:0/19943789 >> v2:172.21.15.29:3300/0 conn(0x7fc1b4155310 0x7fc1b41556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801712520 0 0) 0x7fc1b40acb80 con 0x7fc1b40c4420 2024-01-24T13:58:02.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1a00033f0 con 0x7fc1b40c4420 2024-01-24T13:58:02.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 <== mon.1 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3365853061 0 0) 0x7fc1a00033f0 con 0x7fc1b40c4420 2024-01-24T13:58:02.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 >> v2:172.21.15.29:3300/0 conn(0x7fc1b4155310 msgr2=0x7fc1b41556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:02.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1b8dfd700 1 --2- 172.21.15.163:0/19943789 >> v2:172.21.15.29:3300/0 conn(0x7fc1b4155310 0x7fc1b41556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.108+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 >> v1:172.21.15.29:6789/0 conn(0x7fc1b40aac00 legacy=0x7fc1b4062a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b40ace60 con 0x7fc1b40c4420 2024-01-24T13:58:02.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 <== mon.1 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fc1a4001470 con 0x7fc1b40c4420 2024-01-24T13:58:02.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 >> v1:172.21.15.163:6789/0 conn(0x7fc1b40c4420 legacy=0x7fc1b40aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b8dfd700 1 --2- 172.21.15.163:0/19943789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1a00038e0 0x7fc1a0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b8dfd700 1 --2- 172.21.15.163:0/19943789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1a0004690 0x7fc1a0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b8dfd700 1 --2- 172.21.15.163:0/19943789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a0005000 0x7fc1a000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40abc60 con 0x7fc1a0004690 2024-01-24T13:58:02.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a000bf60 con 0x7fc1a0005000 2024-01-24T13:58:02.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a000c0f0 con 0x7fc1a00038e0 2024-01-24T13:58:02.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b9dff700 1 --2- 172.21.15.163:0/19943789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1a00038e0 0x7fc1a0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1ba600700 1 --2- 172.21.15.163:0/19943789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a0005000 0x7fc1a000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b95fe700 1 --2- 172.21.15.163:0/19943789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1a0004690 0x7fc1a0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1ba600700 1 -- 172.21.15.163:0/19943789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1a00038e0 msgr2=0x7fc1a0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:02.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1ba600700 1 --2- 172.21.15.163:0/19943789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1a00038e0 0x7fc1a0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1ba600700 1 -- 172.21.15.163:0/19943789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1a0004690 msgr2=0x7fc1a0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:02.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1ba600700 1 --2- 172.21.15.163:0/19943789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1a0004690 0x7fc1a0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1ba600700 1 -- 172.21.15.163:0/19943789 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00033f0 con 0x7fc1a0005000 2024-01-24T13:58:02.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b9dff700 1 --2- 172.21.15.163:0/19943789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1a00038e0 0x7fc1a0003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:02.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1ba600700 1 -- 172.21.15.163:0/19943789 reap_dead start 2024-01-24T13:58:02.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1ba600700 1 --2- 172.21.15.163:0/19943789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a0005000 0x7fc1a000b8e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fc1ac001e00 tx=0x7fc1ac007f30).ready entity=mon.1 client_cookie=153d1f066a8ba2d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac006a20 con 0x7fc1a0005000 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac006b80 con 0x7fc1a0005000 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.109+0000 7fc1b8dfd700 1 -- 172.21.15.163:0/19943789 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00c5d0 con 0x7fc1a0005000 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.110+0000 7fc1bc063700 1 -- 172.21.15.163:0/19943789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a0005000 msgr2=0x7fc1a000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.110+0000 7fc1bc063700 1 --2- 172.21.15.163:0/19943789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a0005000 0x7fc1a000b8e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fc1ac001e00 tx=0x7fc1ac007f30).stop 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.110+0000 7fc1bc063700 1 -- 172.21.15.163:0/19943789 shutdown_connections 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.110+0000 7fc1bc063700 1 --2- 172.21.15.163:0/19943789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a0005000 0x7fc1a000b8e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.110+0000 7fc1bc063700 1 -- 172.21.15.163:0/19943789 >> 172.21.15.163:0/19943789 conn(0x7fc1b40c7d40 msgr2=0x7fc1b40c3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.110+0000 7fc1bc063700 1 -- 172.21.15.163:0/19943789 shutdown_connections 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.110+0000 7fc1bc063700 1 -- 172.21.15.163:0/19943789 wait complete. 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.111+0000 7fc1bc063700 1 Processor -- start 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.111+0000 7fc1bc063700 1 -- start start 2024-01-24T13:58:02.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.112+0000 7fc1bc063700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1a0005000 0x7fc1b4165980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.112+0000 7fc1bc063700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1b4150030 0x7fc1b414f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.112+0000 7fc1bc063700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1b41533e0 0x7fc1b4153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.112+0000 7fc1bc063700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b4166840 con 0x7fc1a0005000 2024-01-24T13:58:02.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.112+0000 7fc1bc063700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40bd860 con 0x7fc1b4150030 2024-01-24T13:58:02.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.112+0000 7fc1bc063700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b4153d50 con 0x7fc1b41533e0 2024-01-24T13:58:02.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.112+0000 7fc1b95fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1b4150030 0x7fc1b414f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.112+0000 7fc1ba600700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1b41533e0 0x7fc1b4153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.112+0000 7fc1b95fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1b4150030 0x7fc1b414f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53498/0 (socket says 172.21.15.163:53498) 2024-01-24T13:58:02.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.112+0000 7fc1b95fe700 1 -- 172.21.15.163:0/767788626 learned_addr learned my addr 172.21.15.163:0/767788626 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:02.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.112+0000 7fc1b9dff700 1 --2- 172.21.15.163:0/767788626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1a0005000 0x7fc1b4165980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1b95fe700 1 -- 172.21.15.163:0/767788626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1b41533e0 msgr2=0x7fc1b4153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1b95fe700 1 --2- 172.21.15.163:0/767788626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1b41533e0 0x7fc1b4153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1b95fe700 1 -- 172.21.15.163:0/767788626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1a0005000 msgr2=0x7fc1b4165980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1b95fe700 1 --2- 172.21.15.163:0/767788626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1a0005000 0x7fc1b4165980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1b95fe700 1 -- 172.21.15.163:0/767788626 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b4166280 con 0x7fc1b4150030 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1ba600700 1 --2- 172.21.15.163:0/767788626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1b41533e0 0x7fc1b4153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1b95fe700 1 --2- 172.21.15.163:0/767788626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1b4150030 0x7fc1b414f1a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fc1a4004a40 tx=0x7fc1a4001e50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1a9ffb700 1 -- 172.21.15.163:0/767788626 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a400f8d0 con 0x7fc1b4150030 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1a9ffb700 1 -- 172.21.15.163:0/767788626 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a400fa30 con 0x7fc1b4150030 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1bc063700 1 -- 172.21.15.163:0/767788626 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b4166560 con 0x7fc1b4150030 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1a9ffb700 1 -- 172.21.15.163:0/767788626 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a400fdb0 con 0x7fc1b4150030 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1bc063700 1 -- 172.21.15.163:0/767788626 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b4165e70 con 0x7fc1b4150030 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1a9ffb700 1 -- 172.21.15.163:0/767788626 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fc1a4004d50 con 0x7fc1b4150030 2024-01-24T13:58:02.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.113+0000 7fc1a9ffb700 1 -- 172.21.15.163:0/767788626 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7fc1a401e830 con 0x7fc1b4150030 2024-01-24T13:58:02.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.115+0000 7fc1bc063700 1 -- 172.21.15.163:0/767788626 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b40475e0 con 0x7fc1b4150030 2024-01-24T13:58:02.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.117+0000 7fc1a9ffb700 1 -- 172.21.15.163:0/767788626 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7fc1a4015b00 con 0x7fc1b4150030 2024-01-24T13:58:02.206 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.207+0000 7fc1bc063700 1 -- 172.21.15.163:0/767788626 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f7a975b6-cab3-43be-a47b-178d2b65f18e", "id": 2} v 0) v1 -- 0x7fc1b40bd9a0 con 0x7fc1b4150030 2024-01-24T13:58:02.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.213+0000 7fc1a9ffb700 1 -- 172.21.15.163:0/767788626 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f7a975b6-cab3-43be-a47b-178d2b65f18e", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc1a4015ce0 con 0x7fc1b4150030 2024-01-24T13:58:02.212 INFO:teuthology.orchestra.run.smithi163.stdout:2 2024-01-24T13:58:02.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.214+0000 7fc1bc063700 1 -- 172.21.15.163:0/767788626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1b4150030 msgr2=0x7fc1b414f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.214+0000 7fc1bc063700 1 --2- 172.21.15.163:0/767788626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1b4150030 0x7fc1b414f1a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fc1a4004a40 tx=0x7fc1a4001e50).stop 2024-01-24T13:58:02.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.214+0000 7fc1bc063700 1 -- 172.21.15.163:0/767788626 shutdown_connections 2024-01-24T13:58:02.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.214+0000 7fc1bc063700 1 --2- 172.21.15.163:0/767788626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1b41533e0 0x7fc1b4153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.214+0000 7fc1bc063700 1 --2- 172.21.15.163:0/767788626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1a0005000 0x7fc1b4165980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.214+0000 7fc1bc063700 1 --2- 172.21.15.163:0/767788626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1b4150030 0x7fc1b414f1a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.214+0000 7fc1bc063700 1 -- 172.21.15.163:0/767788626 >> 172.21.15.163:0/767788626 conn(0x7fc1b40c7d40 msgr2=0x7fc1b4151040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:02.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.214+0000 7fc1bc063700 1 -- 172.21.15.163:0/767788626 shutdown_connections 2024-01-24T13:58:02.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.214+0000 7fc1bc063700 1 -- 172.21.15.163:0/767788626 wait complete. 2024-01-24T13:58:02.225 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6 3 2024-01-24T13:58:02.357 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:02.358+0000 7fc516bdc680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T13:58:02.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.366+0000 7f5bab8b8700 1 Processor -- start 2024-01-24T13:58:02.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.366+0000 7f5bab8b8700 1 -- start start 2024-01-24T13:58:02.373 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.367+0000 7f5bab8b8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ba40663b0 0x7f5ba4152f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.373 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.367+0000 7f5bab8b8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ba40be4b0 0x7f5ba4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.373 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.367+0000 7f5bab8b8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ba406a010 0x7f5ba4065420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.373 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.367+0000 7f5bab8b8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba40be930 con 0x7f5ba406a010 2024-01-24T13:58:02.373 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.367+0000 7f5bab8b8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba40bca30 con 0x7f5ba40663b0 2024-01-24T13:58:02.373 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.367+0000 7f5bab8b8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba4069690 con 0x7f5ba40be4b0 2024-01-24T13:58:02.373 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.367+0000 7f5ba9654700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ba40663b0 0x7f5ba4152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.367+0000 7f5ba8e53700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ba40be4b0 0x7f5ba4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.367+0000 7f5ba9654700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ba40663b0 0x7f5ba4152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53510/0 (socket says 172.21.15.163:53510) 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.367+0000 7f5ba9654700 1 -- 172.21.15.163:0/1716744197 learned_addr learned my addr 172.21.15.163:0/1716744197 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.367+0000 7f5ba9e55700 1 --2- 172.21.15.163:0/1716744197 >> v2:172.21.15.29:3300/0 conn(0x7f5ba406a010 0x7f5ba4065420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5ba8e53700 1 -- 172.21.15.163:0/1716744197 >> v2:172.21.15.163:3300/0 conn(0x7f5ba40663b0 msgr2=0x7f5ba4152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5ba8e53700 1 --2- 172.21.15.163:0/1716744197 >> v2:172.21.15.163:3300/0 conn(0x7f5ba40663b0 0x7f5ba4152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5ba8e53700 1 -- 172.21.15.163:0/1716744197 >> v2:172.21.15.29:3300/0 conn(0x7f5ba406a010 msgr2=0x7f5ba4065420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5ba8e53700 1 --2- 172.21.15.163:0/1716744197 >> v2:172.21.15.29:3300/0 conn(0x7f5ba406a010 0x7f5ba4065420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5ba8e53700 1 -- 172.21.15.163:0/1716744197 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba40bcc40 con 0x7f5ba40be4b0 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5ba8e53700 1 --2- 172.21.15.163:0/1716744197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ba40be4b0 0x7f5ba4069b20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f5ba40c4350 tx=0x7f5b94009060).ready entity=mon.2 client_cookie=75a4b12c4846bd48 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5b9bfff700 1 -- 172.21.15.163:0/1716744197 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940090d0 con 0x7f5ba40be4b0 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5b9bfff700 1 -- 172.21.15.163:0/1716744197 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9400d4a0 con 0x7f5ba40be4b0 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5b9bfff700 1 -- 172.21.15.163:0/1716744197 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400d820 con 0x7f5ba40be4b0 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1716744197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ba40be4b0 msgr2=0x7f5ba4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5bab8b8700 1 --2- 172.21.15.163:0/1716744197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ba40be4b0 0x7f5ba4069b20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f5ba40c4350 tx=0x7f5b94009060).stop 2024-01-24T13:58:02.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1716744197 shutdown_connections 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5bab8b8700 1 --2- 172.21.15.163:0/1716744197 >> v2:172.21.15.29:3300/0 conn(0x7f5ba406a010 0x7f5ba4065420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5bab8b8700 1 --2- 172.21.15.163:0/1716744197 >> v2:172.21.15.163:3300/0 conn(0x7f5ba40663b0 0x7f5ba4152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5bab8b8700 1 --2- 172.21.15.163:0/1716744197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ba40be4b0 0x7f5ba4069b20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.368+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1716744197 >> 172.21.15.163:0/1716744197 conn(0x7f5ba4064390 msgr2=0x7f5ba40c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.369+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1716744197 shutdown_connections 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.369+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1716744197 wait complete. 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.370+0000 7f5bab8b8700 1 Processor -- start 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.370+0000 7f5bab8b8700 1 -- start start 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5bab8b8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ba40663b0 0x7f5ba413b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5bab8b8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ba406a010 0x7f5ba40ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5bab8b8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ba40be4b0 0x7f5ba4151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5bab8b8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba413a260 con 0x7f5ba40be4b0 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5bab8b8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40a76a0 con 0x7f5ba40663b0 2024-01-24T13:58:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5bab8b8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba413bb10 con 0x7f5ba406a010 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5ba9654700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ba40663b0 0x7f5ba413b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5ba8e53700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ba406a010 0x7f5ba40ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5ba9654700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ba40663b0 0x7f5ba413b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53526/0 (socket says 172.21.15.163:53526) 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5ba9654700 1 -- 172.21.15.163:0/1265786899 learned_addr learned my addr 172.21.15.163:0/1265786899 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5ba9e55700 1 --2- 172.21.15.163:0/1265786899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ba40be4b0 0x7f5ba4151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5ba9654700 1 -- 172.21.15.163:0/1265786899 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ba406a010 msgr2=0x7f5ba40ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5ba9654700 1 --2- 172.21.15.163:0/1265786899 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ba406a010 0x7f5ba40ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5ba9654700 1 -- 172.21.15.163:0/1265786899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ba40be4b0 msgr2=0x7f5ba4151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5ba9654700 1 --2- 172.21.15.163:0/1265786899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ba40be4b0 0x7f5ba4151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5ba9654700 1 -- 172.21.15.163:0/1265786899 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b940092a0 con 0x7f5ba40663b0 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5ba9654700 1 --2- 172.21.15.163:0/1265786899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ba40663b0 0x7f5ba413b620 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f5ba000e930 tx=0x7f5ba00090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5b997fa700 1 -- 172.21.15.163:0/1265786899 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0012600 con 0x7f5ba40663b0 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5b997fa700 1 -- 172.21.15.163:0/1265786899 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba0012760 con 0x7f5ba40663b0 2024-01-24T13:58:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5b997fa700 1 -- 172.21.15.163:0/1265786899 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0012ae0 con 0x7f5ba40663b0 2024-01-24T13:58:02.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.371+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1265786899 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba4138b90 con 0x7f5ba40663b0 2024-01-24T13:58:02.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.372+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1265786899 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba4139430 con 0x7f5ba40663b0 2024-01-24T13:58:02.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.372+0000 7f5b997fa700 1 -- 172.21.15.163:0/1265786899 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f5ba001b660 con 0x7f5ba40663b0 2024-01-24T13:58:02.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.372+0000 7f5b997fa700 1 -- 172.21.15.163:0/1265786899 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f5ba000c3e0 con 0x7f5ba40663b0 2024-01-24T13:58:02.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.372+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1265786899 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba40a7980 con 0x7f5ba40663b0 2024-01-24T13:58:02.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.375+0000 7f5b997fa700 1 -- 172.21.15.163:0/1265786899 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f5ba003ae50 con 0x7f5ba40663b0 2024-01-24T13:58:02.429 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:02.429+0000 7fc516bdc680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T13:58:02.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.455+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1265786899 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6", "id": 3} v 0) v1 -- 0x7f5ba40475e0 con 0x7f5ba40663b0 2024-01-24T13:58:02.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.464+0000 7f5b997fa700 1 -- 172.21.15.163:0/1265786899 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5ba0029030 con 0x7f5ba40663b0 2024-01-24T13:58:02.464 INFO:teuthology.orchestra.run.smithi163.stdout:3 2024-01-24T13:58:02.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.465+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1265786899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ba40663b0 msgr2=0x7f5ba413b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.465+0000 7f5bab8b8700 1 --2- 172.21.15.163:0/1265786899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ba40663b0 0x7f5ba413b620 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f5ba000e930 tx=0x7f5ba00090f0).stop 2024-01-24T13:58:02.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.465+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1265786899 shutdown_connections 2024-01-24T13:58:02.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.465+0000 7f5bab8b8700 1 --2- 172.21.15.163:0/1265786899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ba40be4b0 0x7f5ba4151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.465+0000 7f5bab8b8700 1 --2- 172.21.15.163:0/1265786899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ba40663b0 0x7f5ba413b620 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.465+0000 7f5bab8b8700 1 --2- 172.21.15.163:0/1265786899 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ba406a010 0x7f5ba40ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.465+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1265786899 >> 172.21.15.163:0/1265786899 conn(0x7f5ba4064390 msgr2=0x7f5ba40c4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:02.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.465+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1265786899 shutdown_connections 2024-01-24T13:58:02.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.465+0000 7f5bab8b8700 1 -- 172.21.15.163:0/1265786899 wait complete. 2024-01-24T13:58:02.477 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new a9ba71db-c0f6-4c63-a4e2-cf7f94184237 4 2024-01-24T13:58:02.492 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:02.492+0000 7f1a5db3b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T13:58:02.568 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:02.568+0000 7f1a5db3b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T13:58:02.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.595+0000 7f99d7549700 1 Processor -- start 2024-01-24T13:58:02.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.595+0000 7f99d7549700 1 -- start start 2024-01-24T13:58:02.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.596+0000 7f99d7549700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f99d014e530 0x7f99d0155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.596+0000 7f99d7549700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d0158960 con 0x7f99d00c0e20 2024-01-24T13:58:02.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.596+0000 7f99d7549700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d0155650 con 0x7f99d005e1f0 2024-01-24T13:58:02.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.596+0000 7f99d7549700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d00c1b90 con 0x7f99d014e530 2024-01-24T13:58:02.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.596+0000 7f99d52e5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f99d014e530 0x7f99d0155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.597+0000 7f99d4ae4700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f99d005e1f0 0x7f99d00c0710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57472/0 (socket says 172.21.15.163:57472) 2024-01-24T13:58:02.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.597+0000 7f99d4ae4700 1 -- 172.21.15.163:0/1134046319 learned_addr learned my addr 172.21.15.163:0/1134046319 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-01-24T13:58:02.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.597+0000 7f99d52e5700 1 -- 172.21.15.163:0/1134046319 >> v1:172.21.15.163:6789/0 conn(0x7f99d005e1f0 legacy=0x7f99d00c0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.597+0000 7f99c7fff700 1 -- 172.21.15.163:0/1134046319 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257482277 0 0) 0x7f99d0155650 con 0x7f99d005e1f0 2024-01-24T13:58:02.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.597+0000 7f99d52e5700 1 -- 172.21.15.163:0/1134046319 >> v1:172.21.15.29:6789/0 conn(0x7f99d00c0e20 legacy=0x7f99d0068a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.597+0000 7f99d52e5700 1 -- 172.21.15.163:0/1134046319 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d0157d50 con 0x7f99d014e530 2024-01-24T13:58:02.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.597+0000 7f99d52e5700 1 --2- 172.21.15.163:0/1134046319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014e530 0x7f99d0155160 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f99cc009760 tx=0x7f99cc009060).ready entity=mon.1 client_cookie=98f1bfd06b0b8c32 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:02.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.597+0000 7f99c7fff700 1 -- 172.21.15.163:0/1134046319 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00b580 con 0x7f99d014e530 2024-01-24T13:58:02.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.597+0000 7f99c7fff700 1 -- 172.21.15.163:0/1134046319 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc00b6e0 con 0x7f99d014e530 2024-01-24T13:58:02.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.597+0000 7f99c7fff700 1 -- 172.21.15.163:0/1134046319 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00ba60 con 0x7f99d014e530 2024-01-24T13:58:02.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.598+0000 7f99d7549700 1 -- 172.21.15.163:0/1134046319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014e530 msgr2=0x7f99d0155160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.598+0000 7f99d7549700 1 --2- 172.21.15.163:0/1134046319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014e530 0x7f99d0155160 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f99cc009760 tx=0x7f99cc009060).stop 2024-01-24T13:58:02.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.598+0000 7f99d7549700 1 -- 172.21.15.163:0/1134046319 shutdown_connections 2024-01-24T13:58:02.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.598+0000 7f99d7549700 1 --2- 172.21.15.163:0/1134046319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014e530 0x7f99d0155160 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.598+0000 7f99d7549700 1 -- 172.21.15.163:0/1134046319 >> 172.21.15.163:0/1134046319 conn(0x7f99d00c7d40 msgr2=0x7f99d00c3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:02.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.598+0000 7f99d7549700 1 -- 172.21.15.163:0/1134046319 shutdown_connections 2024-01-24T13:58:02.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.598+0000 7f99d7549700 1 -- 172.21.15.163:0/1134046319 wait complete. 2024-01-24T13:58:02.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.599+0000 7f99d7549700 1 Processor -- start 2024-01-24T13:58:02.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.599+0000 7f99d7549700 1 -- start start 2024-01-24T13:58:02.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d7549700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99d005e1f0 0x7f99d013b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d7549700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00c0e20 0x7f99d00c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d7549700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014e530 0x7f99d00c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d7549700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d00c5530 con 0x7f99d005e1f0 2024-01-24T13:58:02.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d7549700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d01526d0 con 0x7f99d014e530 2024-01-24T13:58:02.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d7549700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d0152240 con 0x7f99d00c0e20 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d5ae6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014e530 0x7f99d00c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d5ae6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014e530 0x7f99d00c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53538/0 (socket says 172.21.15.163:53538) 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d5ae6700 1 -- 172.21.15.163:0/2736895257 learned_addr learned my addr 172.21.15.163:0/2736895257 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d52e5700 1 --2- 172.21.15.163:0/2736895257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99d005e1f0 0x7f99d013b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d4ae4700 1 --2- 172.21.15.163:0/2736895257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00c0e20 0x7f99d00c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d5ae6700 1 -- 172.21.15.163:0/2736895257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00c0e20 msgr2=0x7f99d00c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d5ae6700 1 --2- 172.21.15.163:0/2736895257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00c0e20 0x7f99d00c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d5ae6700 1 -- 172.21.15.163:0/2736895257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99d005e1f0 msgr2=0x7f99d013b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d5ae6700 1 --2- 172.21.15.163:0/2736895257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99d005e1f0 0x7f99d013b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d5ae6700 1 -- 172.21.15.163:0/2736895257 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc0092a0 con 0x7f99d014e530 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.600+0000 7f99d5ae6700 1 --2- 172.21.15.163:0/2736895257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014e530 0x7f99d00c5040 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f99c800c7c0 tx=0x7f99c8001530).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.601+0000 7f99c57fa700 1 -- 172.21.15.163:0/2736895257 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c800e4a0 con 0x7f99d014e530 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.601+0000 7f99c57fa700 1 -- 172.21.15.163:0/2736895257 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c800e600 con 0x7f99d014e530 2024-01-24T13:58:02.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.601+0000 7f99d7549700 1 -- 172.21.15.163:0/2736895257 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d0151630 con 0x7f99d014e530 2024-01-24T13:58:02.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.601+0000 7f99c57fa700 1 -- 172.21.15.163:0/2736895257 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c800e980 con 0x7f99d014e530 2024-01-24T13:58:02.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.601+0000 7f99d7549700 1 -- 172.21.15.163:0/2736895257 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d0151bf0 con 0x7f99d014e530 2024-01-24T13:58:02.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.601+0000 7f99c57fa700 1 -- 172.21.15.163:0/2736895257 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f99c801c440 con 0x7f99d014e530 2024-01-24T13:58:02.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.601+0000 7f99c57fa700 1 -- 172.21.15.163:0/2736895257 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f99c801b8e0 con 0x7f99d014e530 2024-01-24T13:58:02.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.602+0000 7f99d7549700 1 -- 172.21.15.163:0/2736895257 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d01523b0 con 0x7f99d014e530 2024-01-24T13:58:02.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.604+0000 7f99c57fa700 1 -- 172.21.15.163:0/2736895257 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f99c8021060 con 0x7f99d014e530 2024-01-24T13:58:02.631 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:02.632+0000 7fc516bdc680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T13:58:02.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.684+0000 7f99d7549700 1 -- 172.21.15.163:0/2736895257 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a9ba71db-c0f6-4c63-a4e2-cf7f94184237", "id": 4} v 0) v1 -- 0x7f99d00475e0 con 0x7f99d014e530 2024-01-24T13:58:02.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.690+0000 7f99c57fa700 1 -- 172.21.15.163:0/2736895257 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a9ba71db-c0f6-4c63-a4e2-cf7f94184237", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f99c8017070 con 0x7f99d014e530 2024-01-24T13:58:02.690 INFO:teuthology.orchestra.run.smithi163.stdout:4 2024-01-24T13:58:02.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.691+0000 7f99d7549700 1 -- 172.21.15.163:0/2736895257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014e530 msgr2=0x7f99d00c5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.691+0000 7f99d7549700 1 --2- 172.21.15.163:0/2736895257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014e530 0x7f99d00c5040 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f99c800c7c0 tx=0x7f99c8001530).stop 2024-01-24T13:58:02.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.691+0000 7f99d7549700 1 -- 172.21.15.163:0/2736895257 shutdown_connections 2024-01-24T13:58:02.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.691+0000 7f99d7549700 1 --2- 172.21.15.163:0/2736895257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014e530 0x7f99d00c5040 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.691+0000 7f99d7549700 1 --2- 172.21.15.163:0/2736895257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99d005e1f0 0x7f99d013b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.691+0000 7f99d7549700 1 --2- 172.21.15.163:0/2736895257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00c0e20 0x7f99d00c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.691+0000 7f99d7549700 1 -- 172.21.15.163:0/2736895257 >> 172.21.15.163:0/2736895257 conn(0x7f99d00c7d40 msgr2=0x7f99d0155830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.692+0000 7f99d7549700 1 -- 172.21.15.163:0/2736895257 shutdown_connections 2024-01-24T13:58:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.692+0000 7f99d7549700 1 -- 172.21.15.163:0/2736895257 wait complete. 2024-01-24T13:58:02.703 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 3768b6d0-201b-490d-9990-a3312db4d9c1 5 2024-01-24T13:58:02.783 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:02.783+0000 7f1a5db3b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T13:58:02.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.823+0000 7f32d5beb700 1 Processor -- start 2024-01-24T13:58:02.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.823+0000 7f32d5beb700 1 -- start start 2024-01-24T13:58:02.825 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32d5beb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f32d00be5a0 0x7f32d0069f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.825 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32d5beb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f32d00aa190 0x7f32d0062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.825 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32d5beb700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00bca30 con 0x7f32d0062970 2024-01-24T13:58:02.825 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32d5beb700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d0069b50 con 0x7f32d00aa190 2024-01-24T13:58:02.825 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32d5beb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d00bcff0 con 0x7f32d00be5a0 2024-01-24T13:58:02.825 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32ceffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f32d00be5a0 0x7f32d0069f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.825 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32ceffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f32d00be5a0 0x7f32d0069f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53554/0 (socket says 172.21.15.163:53554) 2024-01-24T13:58:02.825 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32ceffd700 1 -- 172.21.15.163:0/1663675337 learned_addr learned my addr 172.21.15.163:0/1663675337 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:02.825 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32ceffd700 1 -- 172.21.15.163:0/1663675337 >> v2:172.21.15.29:3300/0 conn(0x7f32d00aa190 msgr2=0x7f32d0062d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32ceffd700 1 --2- 172.21.15.163:0/1663675337 >> v2:172.21.15.29:3300/0 conn(0x7f32d00aa190 0x7f32d0062d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32ceffd700 1 -- 172.21.15.163:0/1663675337 >> v1:172.21.15.29:6789/0 conn(0x7f32d0062970 legacy=0x7f32d00a9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32ceffd700 1 -- 172.21.15.163:0/1663675337 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00bcd10 con 0x7f32d00be5a0 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32ceffd700 1 --2- 172.21.15.163:0/1663675337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d00be5a0 0x7f32d0069f70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f32d006af90 tx=0x7f32c4009060).ready entity=mon.1 client_cookie=5f583c84367ab433 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32ce7fc700 1 -- 172.21.15.163:0/1663675337 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c4005880 con 0x7f32d00be5a0 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32ce7fc700 1 -- 172.21.15.163:0/1663675337 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c40059e0 con 0x7f32d00be5a0 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.824+0000 7f32ce7fc700 1 -- 172.21.15.163:0/1663675337 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c4005d60 con 0x7f32d00be5a0 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.825+0000 7f32d5beb700 1 -- 172.21.15.163:0/1663675337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d00be5a0 msgr2=0x7f32d0069f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.825+0000 7f32d5beb700 1 --2- 172.21.15.163:0/1663675337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d00be5a0 0x7f32d0069f70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f32d006af90 tx=0x7f32c4009060).stop 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.825+0000 7f32d5beb700 1 -- 172.21.15.163:0/1663675337 shutdown_connections 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.825+0000 7f32d5beb700 1 --2- 172.21.15.163:0/1663675337 >> v2:172.21.15.29:3300/0 conn(0x7f32d00aa190 0x7f32d0062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.825+0000 7f32d5beb700 1 --2- 172.21.15.163:0/1663675337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d00be5a0 0x7f32d0069f70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.825+0000 7f32d5beb700 1 -- 172.21.15.163:0/1663675337 >> 172.21.15.163:0/1663675337 conn(0x7f32d0064390 msgr2=0x7f32d00c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:02.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.825+0000 7f32d5beb700 1 -- 172.21.15.163:0/1663675337 shutdown_connections 2024-01-24T13:58:02.827 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.825+0000 7f32d5beb700 1 -- 172.21.15.163:0/1663675337 wait complete. 2024-01-24T13:58:02.829 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.826+0000 7f32d5beb700 1 Processor -- start 2024-01-24T13:58:02.829 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.826+0000 7f32d5beb700 1 -- start start 2024-01-24T13:58:02.829 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.827+0000 7f32d5beb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d0062970 0x7f32d0167d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.829 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.827+0000 7f32d5beb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32d00aa190 0x7f32d0168270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.829 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.827+0000 7f32d5beb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32d00be5a0 0x7f32d0138990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.827+0000 7f32d5beb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0168760 con 0x7f32d00be5a0 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.827+0000 7f32d5beb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00c27c0 con 0x7f32d0062970 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.827+0000 7f32d5beb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0139740 con 0x7f32d00aa190 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.827+0000 7f32cf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d0062970 0x7f32d0167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.827+0000 7f32ceffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32d00aa190 0x7f32d0168270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.827+0000 7f32cf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d0062970 0x7f32d0167d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53570/0 (socket says 172.21.15.163:53570) 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.827+0000 7f32cf7fe700 1 -- 172.21.15.163:0/3238799527 learned_addr learned my addr 172.21.15.163:0/3238799527 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.827+0000 7f32cffff700 1 --2- 172.21.15.163:0/3238799527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32d00be5a0 0x7f32d0138990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32cf7fe700 1 -- 172.21.15.163:0/3238799527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32d00aa190 msgr2=0x7f32d0168270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32cf7fe700 1 --2- 172.21.15.163:0/3238799527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32d00aa190 0x7f32d0168270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32cf7fe700 1 -- 172.21.15.163:0/3238799527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32d00be5a0 msgr2=0x7f32d0138990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32cf7fe700 1 --2- 172.21.15.163:0/3238799527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32d00be5a0 0x7f32d0138990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32cf7fe700 1 -- 172.21.15.163:0/3238799527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c4009300 con 0x7f32d0062970 2024-01-24T13:58:02.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32ceffd700 1 --2- 172.21.15.163:0/3238799527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32d00aa190 0x7f32d0168270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:02.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32cf7fe700 1 --2- 172.21.15.163:0/3238799527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d0062970 0x7f32d0167d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f32c000c4c0 tx=0x7f32c0007e00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:02.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32b77fe700 1 -- 172.21.15.163:0/3238799527 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000ce70 con 0x7f32d0062970 2024-01-24T13:58:02.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32b77fe700 1 -- 172.21.15.163:0/3238799527 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0001e90 con 0x7f32d0062970 2024-01-24T13:58:02.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32d5beb700 1 -- 172.21.15.163:0/3238799527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d0139a20 con 0x7f32d0062970 2024-01-24T13:58:02.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32b77fe700 1 -- 172.21.15.163:0/3238799527 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0007250 con 0x7f32d0062970 2024-01-24T13:58:02.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32d5beb700 1 -- 172.21.15.163:0/3238799527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d00c2450 con 0x7f32d0062970 2024-01-24T13:58:02.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32b77fe700 1 -- 172.21.15.163:0/3238799527 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f32c0006900 con 0x7f32d0062970 2024-01-24T13:58:02.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.828+0000 7f32b77fe700 1 -- 172.21.15.163:0/3238799527 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f32c000fbb0 con 0x7f32d0062970 2024-01-24T13:58:02.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.829+0000 7f32d5beb700 1 -- 172.21.15.163:0/3238799527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d00475e0 con 0x7f32d0062970 2024-01-24T13:58:02.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.831+0000 7f32b77fe700 1 -- 172.21.15.163:0/3238799527 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f32c0006de0 con 0x7f32d0062970 2024-01-24T13:58:02.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.914+0000 7f32d5beb700 1 -- 172.21.15.163:0/3238799527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3768b6d0-201b-490d-9990-a3312db4d9c1", "id": 5} v 0) v1 -- 0x7f32d0069760 con 0x7f32d0062970 2024-01-24T13:58:02.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.917+0000 7f32b77fe700 1 -- 172.21.15.163:0/3238799527 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3768b6d0-201b-490d-9990-a3312db4d9c1", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f32c000f600 con 0x7f32d0062970 2024-01-24T13:58:02.917 INFO:teuthology.orchestra.run.smithi163.stdout:5 2024-01-24T13:58:02.918 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.918+0000 7f32d5beb700 1 -- 172.21.15.163:0/3238799527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d0062970 msgr2=0x7f32d0167d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:02.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.918+0000 7f32d5beb700 1 --2- 172.21.15.163:0/3238799527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d0062970 0x7f32d0167d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f32c000c4c0 tx=0x7f32c0007e00).stop 2024-01-24T13:58:02.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.919+0000 7f32d5beb700 1 -- 172.21.15.163:0/3238799527 shutdown_connections 2024-01-24T13:58:02.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.919+0000 7f32d5beb700 1 --2- 172.21.15.163:0/3238799527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32d00be5a0 0x7f32d0138990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.919+0000 7f32d5beb700 1 --2- 172.21.15.163:0/3238799527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d0062970 0x7f32d0167d80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.919+0000 7f32d5beb700 1 --2- 172.21.15.163:0/3238799527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32d00aa190 0x7f32d0168270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:02.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.919+0000 7f32d5beb700 1 -- 172.21.15.163:0/3238799527 >> 172.21.15.163:0/3238799527 conn(0x7f32d0064390 msgr2=0x7f32d013b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:02.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.919+0000 7f32d5beb700 1 -- 172.21.15.163:0/3238799527 shutdown_connections 2024-01-24T13:58:02.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:02.919+0000 7f32d5beb700 1 -- 172.21.15.163:0/3238799527 wait complete. 2024-01-24T13:58:02.930 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 555eb732-dd59-49c4-a326-e5f75cbb17d2 6 2024-01-24T13:58:03.047 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.047+0000 7f9383432700 1 Processor -- start 2024-01-24T13:58:03.047 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.047+0000 7f9383432700 1 -- start start 2024-01-24T13:58:03.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.048+0000 7f9383432700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f937c064740 0x7f937c064b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.048+0000 7f9383432700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937c0c6820 con 0x7f937c0bfd20 2024-01-24T13:58:03.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.048+0000 7f9383432700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937c155220 con 0x7f937c0c5af0 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.048+0000 7f9383432700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f937c1556b0 con 0x7f937c064740 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.048+0000 7f93811ce700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f937c064740 0x7f937c064b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.048+0000 7f93809cd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f937c0c5af0 0x7f937c0c5f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57486/0 (socket says 172.21.15.163:57486) 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.048+0000 7f93809cd700 1 -- 172.21.15.163:0/1836003929 learned_addr learned my addr 172.21.15.163:0/1836003929 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.049+0000 7f93811ce700 1 -- 172.21.15.163:0/1836003929 >> v1:172.21.15.163:6789/0 conn(0x7f937c0c5af0 legacy=0x7f937c0c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.049+0000 7f93811ce700 1 -- 172.21.15.163:0/1836003929 >> v1:172.21.15.29:6789/0 conn(0x7f937c0bfd20 legacy=0x7f937c0c0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.049+0000 7f93811ce700 1 -- 172.21.15.163:0/1836003929 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c154680 con 0x7f937c064740 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.049+0000 7f93811ce700 1 --2- 172.21.15.163:0/1836003929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f937c064740 0x7f937c064b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9378009760 tx=0x7f9378009060).ready entity=mon.1 client_cookie=5ad3944d860deaaa server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.049+0000 7f9373fff700 1 -- 172.21.15.163:0/1836003929 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937800b580 con 0x7f937c064740 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.049+0000 7f9373fff700 1 -- 172.21.15.163:0/1836003929 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937800b6e0 con 0x7f937c064740 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.049+0000 7f9373fff700 1 -- 172.21.15.163:0/1836003929 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937800ba60 con 0x7f937c064740 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.049+0000 7f9383432700 1 -- 172.21.15.163:0/1836003929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f937c064740 msgr2=0x7f937c064b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.049+0000 7f9383432700 1 --2- 172.21.15.163:0/1836003929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f937c064740 0x7f937c064b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9378009760 tx=0x7f9378009060).stop 2024-01-24T13:58:03.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.050+0000 7f9383432700 1 -- 172.21.15.163:0/1836003929 shutdown_connections 2024-01-24T13:58:03.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.050+0000 7f9383432700 1 --2- 172.21.15.163:0/1836003929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f937c064740 0x7f937c064b20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.050+0000 7f9383432700 1 -- 172.21.15.163:0/1836003929 >> 172.21.15.163:0/1836003929 conn(0x7f937c0cb890 msgr2=0x7f937c0c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:03.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.050+0000 7f9383432700 1 -- 172.21.15.163:0/1836003929 shutdown_connections 2024-01-24T13:58:03.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.050+0000 7f9383432700 1 -- 172.21.15.163:0/1836003929 wait complete. 2024-01-24T13:58:03.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.051+0000 7f9383432700 1 Processor -- start 2024-01-24T13:58:03.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.051+0000 7f9383432700 1 -- start start 2024-01-24T13:58:03.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f9383432700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f937c064740 0x7f937c1558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f9383432700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f937c0bfd20 0x7f937c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f9383432700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f937c0c5af0 0x7f937c1688c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f9383432700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0af170 con 0x7f937c064740 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f9383432700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c155da0 con 0x7f937c0bfd20 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f9383432700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0ae560 con 0x7f937c0c5af0 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93809cd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f937c0bfd20 0x7f937c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93819cf700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f937c0c5af0 0x7f937c1688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93809cd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f937c0bfd20 0x7f937c159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53588/0 (socket says 172.21.15.163:53588) 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93809cd700 1 -- 172.21.15.163:0/1725602131 learned_addr learned my addr 172.21.15.163:0/1725602131 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93811ce700 1 --2- 172.21.15.163:0/1725602131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f937c064740 0x7f937c1558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93819cf700 1 -- 172.21.15.163:0/1725602131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f937c0bfd20 msgr2=0x7f937c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93819cf700 1 --2- 172.21.15.163:0/1725602131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f937c0bfd20 0x7f937c159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93819cf700 1 -- 172.21.15.163:0/1725602131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f937c064740 msgr2=0x7f937c1558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93819cf700 1 --2- 172.21.15.163:0/1725602131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f937c064740 0x7f937c1558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93819cf700 1 -- 172.21.15.163:0/1725602131 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93780092a0 con 0x7f937c0c5af0 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93809cd700 1 --2- 172.21.15.163:0/1725602131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f937c0bfd20 0x7f937c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:03.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93819cf700 1 --2- 172.21.15.163:0/1725602131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f937c0c5af0 0x7f937c1688c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f937c0aaa00 tx=0x7f9374001480).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:03.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.052+0000 7f93717fa700 1 -- 172.21.15.163:0/1725602131 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937400f470 con 0x7f937c0c5af0 2024-01-24T13:58:03.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.053+0000 7f93717fa700 1 -- 172.21.15.163:0/1725602131 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937400f5d0 con 0x7f937c0c5af0 2024-01-24T13:58:03.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.053+0000 7f9383432700 1 -- 172.21.15.163:0/1725602131 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937c0ae840 con 0x7f937c0c5af0 2024-01-24T13:58:03.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.053+0000 7f93717fa700 1 -- 172.21.15.163:0/1725602131 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937400f970 con 0x7f937c0c5af0 2024-01-24T13:58:03.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.053+0000 7f9383432700 1 -- 172.21.15.163:0/1725602131 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937c0aee00 con 0x7f937c0c5af0 2024-01-24T13:58:03.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.053+0000 7f93717fa700 1 -- 172.21.15.163:0/1725602131 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f937401b3f0 con 0x7f937c0c5af0 2024-01-24T13:58:03.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.053+0000 7f93717fa700 1 -- 172.21.15.163:0/1725602131 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7f9374012a10 con 0x7f937c0c5af0 2024-01-24T13:58:03.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.053+0000 7f9383432700 1 -- 172.21.15.163:0/1725602131 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937c0c78d0 con 0x7f937c0c5af0 2024-01-24T13:58:03.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.056+0000 7f93717fa700 1 -- 172.21.15.163:0/1725602131 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f9374020060 con 0x7f937c0c5af0 2024-01-24T13:58:03.104 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:03.105+0000 7fc516bdc680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T13:58:03.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.135+0000 7f9383432700 1 -- 172.21.15.163:0/1725602131 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "555eb732-dd59-49c4-a326-e5f75cbb17d2", "id": 6} v 0) v1 -- 0x7f937c0475e0 con 0x7f937c0c5af0 2024-01-24T13:58:03.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.141+0000 7f93717fa700 1 -- 172.21.15.163:0/1725602131 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "555eb732-dd59-49c4-a326-e5f75cbb17d2", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f937c0c6820 con 0x7f937c0c5af0 2024-01-24T13:58:03.141 INFO:teuthology.orchestra.run.smithi163.stdout:6 2024-01-24T13:58:03.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.142+0000 7f9383432700 1 -- 172.21.15.163:0/1725602131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f937c0c5af0 msgr2=0x7f937c1688c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.142+0000 7f9383432700 1 --2- 172.21.15.163:0/1725602131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f937c0c5af0 0x7f937c1688c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f937c0aaa00 tx=0x7f9374001480).stop 2024-01-24T13:58:03.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.142+0000 7f9383432700 1 -- 172.21.15.163:0/1725602131 shutdown_connections 2024-01-24T13:58:03.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.142+0000 7f9383432700 1 --2- 172.21.15.163:0/1725602131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f937c0c5af0 0x7f937c1688c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.142+0000 7f9383432700 1 --2- 172.21.15.163:0/1725602131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f937c064740 0x7f937c1558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.142+0000 7f9383432700 1 --2- 172.21.15.163:0/1725602131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f937c0bfd20 0x7f937c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.142+0000 7f9383432700 1 -- 172.21.15.163:0/1725602131 >> 172.21.15.163:0/1725602131 conn(0x7f937c0cb890 msgr2=0x7f937c0c8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:03.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.143+0000 7f9383432700 1 -- 172.21.15.163:0/1725602131 shutdown_connections 2024-01-24T13:58:03.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.143+0000 7f9383432700 1 -- 172.21.15.163:0/1725602131 wait complete. 2024-01-24T13:58:03.154 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 4935e576-9e41-4af8-9c65-30b80ddb2b6e 7 2024-01-24T13:58:03.216 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:03.216+0000 7fc516bdc680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T13:58:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.271+0000 7ff1ed898700 1 Processor -- start 2024-01-24T13:58:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.271+0000 7ff1ed898700 1 -- start start 2024-01-24T13:58:03.274 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.272+0000 7ff1ed898700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff1e80c5af0 0x7ff1e80c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.274 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.272+0000 7ff1ed898700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff1e80bfd20 0x7ff1e80c0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.274 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.272+0000 7ff1ed898700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e80c6820 con 0x7ff1e8064740 2024-01-24T13:58:03.274 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.272+0000 7ff1ed898700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e81556b0 con 0x7ff1e80c5af0 2024-01-24T13:58:03.274 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.272+0000 7ff1ed898700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e8155220 con 0x7ff1e80bfd20 2024-01-24T13:58:03.274 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.272+0000 7ff1e6ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff1e8064740 0x7ff1e8064b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57502/0 (socket says 172.21.15.163:57502) 2024-01-24T13:58:03.274 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.272+0000 7ff1e6ffd700 1 -- 172.21.15.163:0/1034758661 learned_addr learned my addr 172.21.15.163:0/1034758661 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-01-24T13:58:03.274 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.272+0000 7ff1e77fe700 1 --2- 172.21.15.163:0/1034758661 >> v2:172.21.15.163:3300/0 conn(0x7ff1e80bfd20 0x7ff1e80c0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.274 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.272+0000 7ff1e67fc700 1 --2- 172.21.15.163:0/1034758661 >> v2:172.21.15.29:3300/0 conn(0x7ff1e80c5af0 0x7ff1e80c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.274 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.273+0000 7ff1e5ffb700 1 -- 172.21.15.163:0/1034758661 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038132913 0 0) 0x7ff1dc001110 con 0x7ff1e8064740 2024-01-24T13:58:03.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.273+0000 7ff1e5ffb700 1 -- 172.21.15.163:0/1034758661 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1d00033f0 con 0x7ff1e8064740 2024-01-24T13:58:03.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.273+0000 7ff1e77fe700 1 -- 172.21.15.163:0/1034758661 >> v2:172.21.15.29:3300/0 conn(0x7ff1e80c5af0 msgr2=0x7ff1e80c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:03.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.273+0000 7ff1e77fe700 1 --2- 172.21.15.163:0/1034758661 >> v2:172.21.15.29:3300/0 conn(0x7ff1e80c5af0 0x7ff1e80c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.273+0000 7ff1e77fe700 1 -- 172.21.15.163:0/1034758661 >> v1:172.21.15.163:6789/0 conn(0x7ff1e8064740 legacy=0x7ff1e8064b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.273+0000 7ff1e77fe700 1 -- 172.21.15.163:0/1034758661 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e8154680 con 0x7ff1e80bfd20 2024-01-24T13:58:03.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.273+0000 7ff1e77fe700 1 --2- 172.21.15.163:0/1034758661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1e80bfd20 0x7ff1e80c0170 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7ff1e80aaa00 tx=0x7ff1e0009780).ready entity=mon.1 client_cookie=5f839e60761891f7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:03.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.273+0000 7ff1e5ffb700 1 -- 172.21.15.163:0/1034758661 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000e070 con 0x7ff1e80bfd20 2024-01-24T13:58:03.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.273+0000 7ff1e5ffb700 1 -- 172.21.15.163:0/1034758661 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e000ba80 con 0x7ff1e80bfd20 2024-01-24T13:58:03.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.273+0000 7ff1e5ffb700 1 -- 172.21.15.163:0/1034758661 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000be00 con 0x7ff1e80bfd20 2024-01-24T13:58:03.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.273+0000 7ff1ed898700 1 -- 172.21.15.163:0/1034758661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1e80bfd20 msgr2=0x7ff1e80c0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.273+0000 7ff1ed898700 1 --2- 172.21.15.163:0/1034758661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1e80bfd20 0x7ff1e80c0170 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7ff1e80aaa00 tx=0x7ff1e0009780).stop 2024-01-24T13:58:03.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.274+0000 7ff1ed898700 1 -- 172.21.15.163:0/1034758661 shutdown_connections 2024-01-24T13:58:03.276 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.274+0000 7ff1ed898700 1 --2- 172.21.15.163:0/1034758661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1e80bfd20 0x7ff1e80c0170 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.276 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.274+0000 7ff1ed898700 1 --2- 172.21.15.163:0/1034758661 >> v2:172.21.15.29:3300/0 conn(0x7ff1e80c5af0 0x7ff1e80c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.276 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.274+0000 7ff1ed898700 1 -- 172.21.15.163:0/1034758661 >> 172.21.15.163:0/1034758661 conn(0x7ff1e80cb890 msgr2=0x7ff1e80c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:03.276 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.274+0000 7ff1ed898700 1 -- 172.21.15.163:0/1034758661 shutdown_connections 2024-01-24T13:58:03.276 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.274+0000 7ff1ed898700 1 -- 172.21.15.163:0/1034758661 wait complete. 2024-01-24T13:58:03.278 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.275+0000 7ff1ed898700 1 Processor -- start 2024-01-24T13:58:03.278 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.275+0000 7ff1ed898700 1 -- start start 2024-01-24T13:58:03.278 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1ed898700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1e8064740 0x7ff1e814f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.279 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1ed898700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1e80bfd20 0x7ff1e80ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.279 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1ed898700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1e80c5af0 0x7ff1e8164760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.279 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1ed898700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80c2a70 con 0x7ff1e80c5af0 2024-01-24T13:58:03.279 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1ed898700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80c1e60 con 0x7ff1e8064740 2024-01-24T13:58:03.279 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1ed898700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80c2140 con 0x7ff1e80bfd20 2024-01-24T13:58:03.279 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1e6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1e8064740 0x7ff1e814f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.279 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1e6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1e8064740 0x7ff1e814f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53604/0 (socket says 172.21.15.163:53604) 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1e67fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1e80bfd20 0x7ff1e80ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1e6ffd700 1 -- 172.21.15.163:0/4043610652 learned_addr learned my addr 172.21.15.163:0/4043610652 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1e77fe700 1 --2- 172.21.15.163:0/4043610652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1e80c5af0 0x7ff1e8164760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1e6ffd700 1 -- 172.21.15.163:0/4043610652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1e80bfd20 msgr2=0x7ff1e80ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1e6ffd700 1 --2- 172.21.15.163:0/4043610652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1e80bfd20 0x7ff1e80ae740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1e6ffd700 1 -- 172.21.15.163:0/4043610652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1e80c5af0 msgr2=0x7ff1e8164760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1e6ffd700 1 --2- 172.21.15.163:0/4043610652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1e80c5af0 0x7ff1e8164760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1e6ffd700 1 -- 172.21.15.163:0/4043610652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e0009a20 con 0x7ff1e8064740 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1e67fc700 1 --2- 172.21.15.163:0/4043610652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1e80bfd20 0x7ff1e80ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1e6ffd700 1 --2- 172.21.15.163:0/4043610652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1e8064740 0x7ff1e814f8a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7ff1dc00c4a0 tx=0x7ff1dc007af0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1ceffd700 1 -- 172.21.15.163:0/4043610652 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00ce50 con 0x7ff1e8064740 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1ceffd700 1 -- 172.21.15.163:0/4043610652 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc0072e0 con 0x7ff1e8064740 2024-01-24T13:58:03.280 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1ed898700 1 -- 172.21.15.163:0/4043610652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e80c2420 con 0x7ff1e8064740 2024-01-24T13:58:03.281 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1ceffd700 1 -- 172.21.15.163:0/4043610652 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0057a0 con 0x7ff1e8064740 2024-01-24T13:58:03.281 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.276+0000 7ff1ed898700 1 -- 172.21.15.163:0/4043610652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e814fd90 con 0x7ff1e8064740 2024-01-24T13:58:03.281 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.277+0000 7ff1ceffd700 1 -- 172.21.15.163:0/4043610652 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7ff1dc013450 con 0x7ff1e8064740 2024-01-24T13:58:03.281 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.277+0000 7ff1ceffd700 1 -- 172.21.15.163:0/4043610652 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7ff1dc005e30 con 0x7ff1e8064740 2024-01-24T13:58:03.281 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.277+0000 7ff1ed898700 1 -- 172.21.15.163:0/4043610652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e80475e0 con 0x7ff1e8064740 2024-01-24T13:58:03.281 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.280+0000 7ff1ceffd700 1 -- 172.21.15.163:0/4043610652 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7ff1dc018070 con 0x7ff1e8064740 2024-01-24T13:58:03.284 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:03.284+0000 7f1a5db3b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T13:58:03.361 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.362+0000 7ff1ed898700 1 -- 172.21.15.163:0/4043610652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4935e576-9e41-4af8-9c65-30b80ddb2b6e", "id": 7} v 0) v1 -- 0x7ff1e81524f0 con 0x7ff1e8064740 2024-01-24T13:58:03.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.367+0000 7ff1ceffd700 1 -- 172.21.15.163:0/4043610652 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4935e576-9e41-4af8-9c65-30b80ddb2b6e", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff1e80c6820 con 0x7ff1e8064740 2024-01-24T13:58:03.367 INFO:teuthology.orchestra.run.smithi163.stdout:7 2024-01-24T13:58:03.368 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.368+0000 7ff1ed898700 1 -- 172.21.15.163:0/4043610652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1e8064740 msgr2=0x7ff1e814f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.368 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.368+0000 7ff1ed898700 1 --2- 172.21.15.163:0/4043610652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1e8064740 0x7ff1e814f8a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7ff1dc00c4a0 tx=0x7ff1dc007af0).stop 2024-01-24T13:58:03.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.368+0000 7ff1ed898700 1 -- 172.21.15.163:0/4043610652 shutdown_connections 2024-01-24T13:58:03.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.368+0000 7ff1ed898700 1 --2- 172.21.15.163:0/4043610652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1e80c5af0 0x7ff1e8164760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.368+0000 7ff1ed898700 1 --2- 172.21.15.163:0/4043610652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1e8064740 0x7ff1e814f8a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.368+0000 7ff1ed898700 1 --2- 172.21.15.163:0/4043610652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1e80bfd20 0x7ff1e80ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.368+0000 7ff1ed898700 1 -- 172.21.15.163:0/4043610652 >> 172.21.15.163:0/4043610652 conn(0x7ff1e80cb890 msgr2=0x7ff1e80c1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:03.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.368+0000 7ff1ed898700 1 -- 172.21.15.163:0/4043610652 shutdown_connections 2024-01-24T13:58:03.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:03.368+0000 7ff1ed898700 1 -- 172.21.15.163:0/4043610652 wait complete. 2024-01-24T13:58:03.379 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:03.379+0000 7fc516bdc680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T13:58:03.380 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-24T13:58:03.380 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-01-24T13:58:03.382 INFO:tasks.ceph.osd.0:Started 2024-01-24T13:58:03.382 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-24T13:58:03.383 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-01-24T13:58:03.384 INFO:tasks.ceph.osd.1:Started 2024-01-24T13:58:03.385 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-24T13:58:03.385 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-01-24T13:58:03.387 INFO:tasks.ceph.osd.2:Started 2024-01-24T13:58:03.387 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-24T13:58:03.387 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-01-24T13:58:03.389 INFO:tasks.ceph.osd.3:Started 2024-01-24T13:58:03.389 INFO:tasks.ceph.osd.4:Restarting daemon 2024-01-24T13:58:03.390 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-01-24T13:58:03.392 INFO:tasks.ceph.osd.4:Started 2024-01-24T13:58:03.392 INFO:tasks.ceph.osd.5:Restarting daemon 2024-01-24T13:58:03.392 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-01-24T13:58:03.394 INFO:tasks.ceph.osd.5:Started 2024-01-24T13:58:03.394 INFO:tasks.ceph.osd.6:Restarting daemon 2024-01-24T13:58:03.394 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-01-24T13:58:03.396 INFO:tasks.ceph.osd.6:Started 2024-01-24T13:58:03.396 INFO:tasks.ceph.osd.7:Restarting daemon 2024-01-24T13:58:03.396 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-01-24T13:58:03.398 INFO:tasks.ceph.osd.7:Started 2024-01-24T13:58:03.398 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:03.408 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:03.408+0000 7f1a5db3b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T13:58:03.473 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:03.473+0000 7fc516bdc680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T13:58:03.552 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:03.552+0000 7fc516bdc680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T13:58:03.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.566+0000 7f735124f700 1 Processor -- start 2024-01-24T13:58:03.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.566+0000 7f735124f700 1 -- start start 2024-01-24T13:58:03.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.567+0000 7f735124f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c134cc0 0x7f734c0c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.567+0000 7f735124f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f734c1350a0 0x7f734c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.567+0000 7f735124f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734c068360 con 0x7f734c0664f0 2024-01-24T13:58:03.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.567+0000 7f735124f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734c153360 con 0x7f734c1350a0 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.567+0000 7f735124f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0c3820 con 0x7f734c134cc0 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.567+0000 7f734a59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f734c1350a0 0x7f734c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.567+0000 7f734b59e700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f734c0664f0 0x7f734c155a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36544/0 (socket says 172.21.15.29:36544) 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.567+0000 7f734b59e700 1 -- 172.21.15.29:0/3816361559 learned_addr learned my addr 172.21.15.29:0/3816361559 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.567+0000 7f734ad9d700 1 --2- 172.21.15.29:0/3816361559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c134cc0 0x7f734c0c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.568+0000 7f734a59c700 1 -- 172.21.15.29:0/3816361559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c134cc0 msgr2=0x7f734c0c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.568+0000 7f734a59c700 1 --2- 172.21.15.29:0/3816361559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c134cc0 0x7f734c0c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.568+0000 7f734a59c700 1 -- 172.21.15.29:0/3816361559 >> v1:172.21.15.29:6789/0 conn(0x7f734c0664f0 legacy=0x7f734c155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.568+0000 7f7349d9b700 1 -- 172.21.15.29:0/3816361559 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938600213 0 0) 0x7f734c068360 con 0x7f734c0664f0 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.568+0000 7f734a59c700 1 -- 172.21.15.29:0/3816361559 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c067a30 con 0x7f734c1350a0 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.568+0000 7f734a59c700 1 --2- 172.21.15.29:0/3816361559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c1350a0 0x7f734c152e70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f7334009760 tx=0x7f7334009060).ready entity=mon.0 client_cookie=5c7a3cea488300be server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.568+0000 7f7349d9b700 1 -- 172.21.15.29:0/3816361559 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733400f510 con 0x7f734c1350a0 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.568+0000 7f7349d9b700 1 -- 172.21.15.29:0/3816361559 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733400f670 con 0x7f734c1350a0 2024-01-24T13:58:03.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.568+0000 7f7349d9b700 1 -- 172.21.15.29:0/3816361559 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733400f9f0 con 0x7f734c1350a0 2024-01-24T13:58:03.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.569+0000 7f735124f700 1 -- 172.21.15.29:0/3816361559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c1350a0 msgr2=0x7f734c152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.569+0000 7f735124f700 1 --2- 172.21.15.29:0/3816361559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c1350a0 0x7f734c152e70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f7334009760 tx=0x7f7334009060).stop 2024-01-24T13:58:03.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.569+0000 7f735124f700 1 -- 172.21.15.29:0/3816361559 shutdown_connections 2024-01-24T13:58:03.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.569+0000 7f735124f700 1 --2- 172.21.15.29:0/3816361559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c134cc0 0x7f734c0c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.569+0000 7f735124f700 1 --2- 172.21.15.29:0/3816361559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c1350a0 0x7f734c152e70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.569+0000 7f735124f700 1 -- 172.21.15.29:0/3816361559 >> 172.21.15.29:0/3816361559 conn(0x7f734c0c8bb0 msgr2=0x7f734c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:03.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.569+0000 7f735124f700 1 -- 172.21.15.29:0/3816361559 shutdown_connections 2024-01-24T13:58:03.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.569+0000 7f735124f700 1 -- 172.21.15.29:0/3816361559 wait complete. 2024-01-24T13:58:03.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.570+0000 7f735124f700 1 Processor -- start 2024-01-24T13:58:03.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.570+0000 7f735124f700 1 -- start start 2024-01-24T13:58:03.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.571+0000 7f735124f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0664f0 0x7f734c065440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.571+0000 7f735124f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c134cc0 0x7f734c0c0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.571+0000 7f735124f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c1350a0 0x7f734c151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.571+0000 7f735124f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c065930 con 0x7f734c0664f0 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.571+0000 7f735124f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0c0b50 con 0x7f734c134cc0 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.571+0000 7f735124f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0c0dd0 con 0x7f734c1350a0 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.571+0000 7f734ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0664f0 0x7f734c065440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.571+0000 7f734ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0664f0 0x7f734c065440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59558/0 (socket says 172.21.15.29:59558) 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.571+0000 7f734ad9d700 1 -- 172.21.15.29:0/2387351250 learned_addr learned my addr 172.21.15.29:0/2387351250 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.571+0000 7f734a59c700 1 --2- 172.21.15.29:0/2387351250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c134cc0 0x7f734c0c0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.571+0000 7f734b59e700 1 --2- 172.21.15.29:0/2387351250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c1350a0 0x7f734c151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.572+0000 7f734ad9d700 1 -- 172.21.15.29:0/2387351250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c1350a0 msgr2=0x7f734c151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.572+0000 7f734ad9d700 1 --2- 172.21.15.29:0/2387351250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c1350a0 0x7f734c151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.572+0000 7f734ad9d700 1 -- 172.21.15.29:0/2387351250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c134cc0 msgr2=0x7f734c0c0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.572+0000 7f734ad9d700 1 --2- 172.21.15.29:0/2387351250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c134cc0 0x7f734c0c0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.572+0000 7f734ad9d700 1 -- 172.21.15.29:0/2387351250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73340092a0 con 0x7f734c0664f0 2024-01-24T13:58:03.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.572+0000 7f734a59c700 1 --2- 172.21.15.29:0/2387351250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c134cc0 0x7f734c0c0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:03.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.572+0000 7f734ad9d700 1 --2- 172.21.15.29:0/2387351250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0664f0 0x7f734c065440 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f733c009ef0 tx=0x7f733c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:03.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.572+0000 7f7342ffd700 1 -- 172.21.15.29:0/2387351250 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c00cb70 con 0x7f734c0664f0 2024-01-24T13:58:03.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.572+0000 7f7342ffd700 1 -- 172.21.15.29:0/2387351250 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733c00f040 con 0x7f734c0664f0 2024-01-24T13:58:03.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.572+0000 7f735124f700 1 -- 172.21.15.29:0/2387351250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734c0c10b0 con 0x7f734c0664f0 2024-01-24T13:58:03.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.572+0000 7f7342ffd700 1 -- 172.21.15.29:0/2387351250 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c013400 con 0x7f734c0664f0 2024-01-24T13:58:03.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.572+0000 7f735124f700 1 -- 172.21.15.29:0/2387351250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734c0c16f0 con 0x7f734c0664f0 2024-01-24T13:58:03.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.574+0000 7f7342ffd700 1 -- 172.21.15.29:0/2387351250 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f733c0139d0 con 0x7f734c0664f0 2024-01-24T13:58:03.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.574+0000 7f7342ffd700 1 -- 172.21.15.29:0/2387351250 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f733c01dca0 con 0x7f734c0664f0 2024-01-24T13:58:03.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.574+0000 7f735124f700 1 -- 172.21.15.29:0/2387351250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734c0475e0 con 0x7f734c0664f0 2024-01-24T13:58:03.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.576+0000 7f7342ffd700 1 -- 172.21.15.29:0/2387351250 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f733c01d5a0 con 0x7f734c0664f0 2024-01-24T13:58:03.623 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:03.623+0000 7f1a5db3b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T13:58:03.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.648+0000 7f735124f700 1 -- 172.21.15.29:0/2387351250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f734c14b410 con 0x7f734c0664f0 2024-01-24T13:58:03.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.649+0000 7f7342ffd700 1 -- 172.21.15.29:0/2387351250 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7045 (secure 0 0 0) 0x7f733c01d740 con 0x7f734c0664f0 2024-01-24T13:58:03.649 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:03.649 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":10,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:03.363720+0000","last_up_change":"0.000000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:03.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.650+0000 7f735124f700 1 -- 172.21.15.29:0/2387351250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0664f0 msgr2=0x7f734c065440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:03.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.650+0000 7f735124f700 1 --2- 172.21.15.29:0/2387351250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0664f0 0x7f734c065440 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f733c009ef0 tx=0x7f733c00b040).stop 2024-01-24T13:58:03.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.650+0000 7f735124f700 1 -- 172.21.15.29:0/2387351250 shutdown_connections 2024-01-24T13:58:03.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.650+0000 7f735124f700 1 --2- 172.21.15.29:0/2387351250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c1350a0 0x7f734c151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.650+0000 7f735124f700 1 --2- 172.21.15.29:0/2387351250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0664f0 0x7f734c065440 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.650+0000 7f735124f700 1 --2- 172.21.15.29:0/2387351250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c134cc0 0x7f734c0c0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:03.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.650+0000 7f735124f700 1 -- 172.21.15.29:0/2387351250 >> 172.21.15.29:0/2387351250 conn(0x7f734c0c8bb0 msgr2=0x7f734c14bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:03.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.650+0000 7f735124f700 1 -- 172.21.15.29:0/2387351250 shutdown_connections 2024-01-24T13:58:03.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:03.650+0000 7f735124f700 1 -- 172.21.15.29:0/2387351250 wait complete. 2024-01-24T13:58:03.669 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-24T13:58:03.669 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-24T13:58:03.718 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:03.717+0000 7f1a5db3b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T13:58:03.750 INFO:tasks.ceph.osd.4.smithi163.stderr:2024-01-24T13:58:03.750+0000 7fc744d4f380 -1 Falling back to public interface 2024-01-24T13:58:03.798 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:03.798+0000 7f1a5db3b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T13:58:03.818 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:03.818+0000 7fc516bdc680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T13:58:03.992 INFO:tasks.ceph.osd.6.smithi163.stderr:2024-01-24T13:58:03.992+0000 7f954bd43380 -1 Falling back to public interface 2024-01-24T13:58:04.002 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-01-24T13:58:04.001+0000 7efcb724b380 -1 Falling back to public interface 2024-01-24T13:58:04.013 INFO:tasks.ceph.osd.3.smithi029.stderr:2024-01-24T13:58:04.013+0000 7ff8da9bf380 -1 Falling back to public interface 2024-01-24T13:58:04.017 INFO:tasks.ceph.osd.5.smithi163.stderr:2024-01-24T13:58:04.017+0000 7f6c91a57380 -1 Falling back to public interface 2024-01-24T13:58:04.017 INFO:tasks.ceph.osd.7.smithi163.stderr:2024-01-24T13:58:04.017+0000 7fd995c89380 -1 Falling back to public interface 2024-01-24T13:58:04.018 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:04.018+0000 7fc516bdc680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T13:58:04.024 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-01-24T13:58:04.024+0000 7fdf6ac98380 -1 Falling back to public interface 2024-01-24T13:58:04.034 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-01-24T13:58:04.034+0000 7fc76bc67380 -1 Falling back to public interface 2024-01-24T13:58:04.102 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:04.102+0000 7f1a5db3b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T13:58:04.328 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:04.327+0000 7f1a5db3b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T13:58:04.475 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:04.596 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.596+0000 7f4456719700 1 Processor -- start 2024-01-24T13:58:04.596 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.596+0000 7f4456719700 1 -- start start 2024-01-24T13:58:04.599 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.597+0000 7f4456719700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f44500627b0 0x7f4450062c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:04.599 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.597+0000 7f4456719700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4450065770 con 0x7f4450138710 2024-01-24T13:58:04.599 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.597+0000 7f4456719700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4450063130 con 0x7f4450138330 2024-01-24T13:58:04.599 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.597+0000 7f4456719700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4450069cf0 con 0x7f44500627b0 2024-01-24T13:58:04.599 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.597+0000 7f444f7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4450138710 0x7f44500be3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36554/0 (socket says 172.21.15.29:36554) 2024-01-24T13:58:04.599 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.597+0000 7f444f7fe700 1 -- 172.21.15.29:0/3236374080 learned_addr learned my addr 172.21.15.29:0/3236374080 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.597+0000 7f4454cb6700 1 --2- 172.21.15.29:0/3236374080 >> v2:172.21.15.163:3300/0 conn(0x7f44500627b0 0x7f4450062c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.597+0000 7f444effd700 1 -- 172.21.15.29:0/3236374080 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420139490 0 0) 0x7f4450065770 con 0x7f4450138710 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.597+0000 7f444effd700 1 -- 172.21.15.29:0/3236374080 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44380033f0 con 0x7f4450138710 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.597+0000 7f444effd700 1 -- 172.21.15.29:0/3236374080 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518656158 0 0) 0x7f4450063130 con 0x7f4450138330 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.597+0000 7f444effd700 1 -- 172.21.15.29:0/3236374080 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4450065770 con 0x7f4450138330 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.598+0000 7f4454cb6700 1 -- 172.21.15.29:0/3236374080 >> v1:172.21.15.163:6789/0 conn(0x7f4450138330 legacy=0x7f44500bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.598+0000 7f444effd700 1 -- 172.21.15.29:0/3236374080 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1222490096 0 0) 0x7f44380033f0 con 0x7f4450138710 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.598+0000 7f4454cb6700 1 -- 172.21.15.29:0/3236374080 >> v1:172.21.15.29:6789/0 conn(0x7f4450138710 legacy=0x7f44500be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.598+0000 7f4454cb6700 1 -- 172.21.15.29:0/3236374080 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4450069f60 con 0x7f44500627b0 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.598+0000 7f4454cb6700 1 --2- 172.21.15.29:0/3236374080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44500627b0 0x7f4450062c40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f4448009e60 tx=0x7f4448009780).ready entity=mon.1 client_cookie=66f4a5acf3811cdc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.598+0000 7f444effd700 1 -- 172.21.15.29:0/3236374080 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444800e070 con 0x7f44500627b0 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.598+0000 7f444effd700 1 -- 172.21.15.29:0/3236374080 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444800b7a0 con 0x7f44500627b0 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.598+0000 7f444effd700 1 -- 172.21.15.29:0/3236374080 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444001470 con 0x7f44500627b0 2024-01-24T13:58:04.600 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.599+0000 7f4456719700 1 -- 172.21.15.29:0/3236374080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44500627b0 msgr2=0x7f4450062c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:04.601 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.599+0000 7f4456719700 1 --2- 172.21.15.29:0/3236374080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44500627b0 0x7f4450062c40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f4448009e60 tx=0x7f4448009780).stop 2024-01-24T13:58:04.601 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.599+0000 7f4456719700 1 -- 172.21.15.29:0/3236374080 shutdown_connections 2024-01-24T13:58:04.601 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.599+0000 7f4456719700 1 --2- 172.21.15.29:0/3236374080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44500627b0 0x7f4450062c40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:04.601 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.599+0000 7f4456719700 1 -- 172.21.15.29:0/3236374080 >> 172.21.15.29:0/3236374080 conn(0x7f44500c8580 msgr2=0x7f44500c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:04.601 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.599+0000 7f4456719700 1 -- 172.21.15.29:0/3236374080 shutdown_connections 2024-01-24T13:58:04.601 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.599+0000 7f4456719700 1 -- 172.21.15.29:0/3236374080 wait complete. 2024-01-24T13:58:04.604 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.600+0000 7f4456719700 1 Processor -- start 2024-01-24T13:58:04.604 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.600+0000 7f4456719700 1 -- start start 2024-01-24T13:58:04.604 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.601+0000 7f4456719700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44500627b0 0x7f44500bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:04.604 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.601+0000 7f4456719700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4450138330 0x7f44501373f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:04.604 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.601+0000 7f4456719700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4450138710 0x7f4450165bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:04.604 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.601+0000 7f4456719700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44500bfe60 con 0x7f4450138710 2024-01-24T13:58:04.604 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.601+0000 7f4456719700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44500aae10 con 0x7f44500627b0 2024-01-24T13:58:04.604 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.601+0000 7f4456719700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44500ab0f0 con 0x7f4450138330 2024-01-24T13:58:04.604 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.601+0000 7f4454cb6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4450138710 0x7f4450165bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:04.604 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.601+0000 7f4454cb6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4450138710 0x7f4450165bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59566/0 (socket says 172.21.15.29:59566) 2024-01-24T13:58:04.604 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.601+0000 7f4454cb6700 1 -- 172.21.15.29:0/847805879 learned_addr learned my addr 172.21.15.29:0/847805879 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:04.604 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.601+0000 7f444f7fe700 1 --2- 172.21.15.29:0/847805879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4450138330 0x7f44501373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.601+0000 7f444ffff700 1 --2- 172.21.15.29:0/847805879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44500627b0 0x7f44500bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4454cb6700 1 -- 172.21.15.29:0/847805879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4450138330 msgr2=0x7f44501373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4454cb6700 1 --2- 172.21.15.29:0/847805879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4450138330 0x7f44501373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4454cb6700 1 -- 172.21.15.29:0/847805879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44500627b0 msgr2=0x7f44500bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4454cb6700 1 --2- 172.21.15.29:0/847805879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44500627b0 0x7f44500bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4454cb6700 1 -- 172.21.15.29:0/847805879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4448009a20 con 0x7f4450138710 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f444ffff700 1 --2- 172.21.15.29:0/847805879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44500627b0 0x7f44500bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4454cb6700 1 --2- 172.21.15.29:0/847805879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4450138710 0x7f4450165bf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f444800b610 tx=0x7f4448009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4437fff700 1 -- 172.21.15.29:0/847805879 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444801d440 con 0x7f4450138710 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4437fff700 1 -- 172.21.15.29:0/847805879 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444801d5a0 con 0x7f4450138710 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4456719700 1 -- 172.21.15.29:0/847805879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44500ab3d0 con 0x7f4450138710 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4437fff700 1 -- 172.21.15.29:0/847805879 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444801d940 con 0x7f4450138710 2024-01-24T13:58:04.605 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4456719700 1 -- 172.21.15.29:0/847805879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44500aaaa0 con 0x7f4450138710 2024-01-24T13:58:04.606 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4437fff700 1 -- 172.21.15.29:0/847805879 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f4448027440 con 0x7f4450138710 2024-01-24T13:58:04.606 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.602+0000 7f4437fff700 1 -- 172.21.15.29:0/847805879 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f4448026ab0 con 0x7f4450138710 2024-01-24T13:58:04.606 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.603+0000 7f4456719700 1 -- 172.21.15.29:0/847805879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44500c0030 con 0x7f4450138710 2024-01-24T13:58:04.606 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.606+0000 7f4437fff700 1 -- 172.21.15.29:0/847805879 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f44500c0030 con 0x7f4450138710 2024-01-24T13:58:04.677 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.677+0000 7f4456719700 1 -- 172.21.15.29:0/847805879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4450069a60 con 0x7f4450138710 2024-01-24T13:58:04.678 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.677+0000 7f4437fff700 1 -- 172.21.15.29:0/847805879 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7045 (secure 0 0 0) 0x7f4448022070 con 0x7f4450138710 2024-01-24T13:58:04.678 INFO:teuthology.misc.health.smithi029.stdout: 2024-01-24T13:58:04.678 INFO:teuthology.misc.health.smithi029.stdout:{"epoch":10,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:03.363720+0000","last_up_change":"0.000000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:04.679 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.678+0000 7f4456719700 1 -- 172.21.15.29:0/847805879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4450138710 msgr2=0x7f4450165bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:04.679 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.678+0000 7f4456719700 1 --2- 172.21.15.29:0/847805879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4450138710 0x7f4450165bf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f444800b610 tx=0x7f4448009780).stop 2024-01-24T13:58:04.679 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.678+0000 7f4456719700 1 -- 172.21.15.29:0/847805879 shutdown_connections 2024-01-24T13:58:04.679 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.678+0000 7f4456719700 1 --2- 172.21.15.29:0/847805879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4450138710 0x7f4450165bf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:04.679 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.678+0000 7f4456719700 1 --2- 172.21.15.29:0/847805879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44500627b0 0x7f44500bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:04.679 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.678+0000 7f4456719700 1 --2- 172.21.15.29:0/847805879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4450138330 0x7f44501373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:04.679 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.678+0000 7f4456719700 1 -- 172.21.15.29:0/847805879 >> 172.21.15.29:0/847805879 conn(0x7f44500c8580 msgr2=0x7f44500a7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:04.680 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.679+0000 7f4456719700 1 -- 172.21.15.29:0/847805879 shutdown_connections 2024-01-24T13:58:04.680 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:04.679+0000 7f4456719700 1 -- 172.21.15.29:0/847805879 wait complete. 2024-01-24T13:58:04.691 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-24T13:58:04.696 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:04.696+0000 7fc516bdc680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T13:58:04.765 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:04.765+0000 7fc516bdc680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T13:58:04.846 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:04.846+0000 7fc516bdc680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T13:58:04.996 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:04.997+0000 7fc516bdc680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T13:58:05.068 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:05.068+0000 7fc516bdc680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T13:58:05.104 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:05.104+0000 7f1a5db3b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T13:58:05.183 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:05.183+0000 7f1a5db3b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T13:58:05.188 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:05.188+0000 7fc516bdc680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T13:58:05.272 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:05.272+0000 7f1a5db3b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T13:58:05.296 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:05.296+0000 7fc516bdc680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T13:58:05.441 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:05.441+0000 7f1a5db3b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T13:58:05.522 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:05.522+0000 7f1a5db3b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T13:58:05.624 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:05.624+0000 7fc516bdc680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T13:58:05.655 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:05.655+0000 7f1a5db3b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T13:58:05.699 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:05.699+0000 7fc516bdc680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T13:58:05.780 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:05.780+0000 7f1a5db3b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T13:58:06.148 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:06.148+0000 7f1a5db3b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T13:58:06.229 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:58:06.229+0000 7f1a5db3b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T13:58:06.728 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:06.728+0000 7fc4cc3ef700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-24T13:58:06.728 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:58:06.728+0000 7fc4cc3ef700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-24T13:58:07.257 INFO:tasks.ceph.osd.4.smithi163.stderr:2024-01-24T13:58:07.257+0000 7fc744d4f380 -1 osd.4 0 log_to_monitors {default=true} 2024-01-24T13:58:07.432 INFO:tasks.ceph.osd.6.smithi163.stderr:2024-01-24T13:58:07.432+0000 7f954bd43380 -1 osd.6 0 log_to_monitors {default=true} 2024-01-24T13:58:07.481 INFO:tasks.ceph.osd.5.smithi163.stderr:2024-01-24T13:58:07.482+0000 7f6c91a57380 -1 osd.5 0 log_to_monitors {default=true} 2024-01-24T13:58:07.519 INFO:tasks.ceph.osd.3.smithi029.stderr:2024-01-24T13:58:07.519+0000 7ff8da9bf380 -1 osd.3 0 log_to_monitors {default=true} 2024-01-24T13:58:07.522 INFO:tasks.ceph.osd.7.smithi163.stderr:2024-01-24T13:58:07.522+0000 7fd995c89380 -1 osd.7 0 log_to_monitors {default=true} 2024-01-24T13:58:07.536 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-01-24T13:58:07.536+0000 7fdf6ac98380 -1 osd.2 0 log_to_monitors {default=true} 2024-01-24T13:58:07.716 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-01-24T13:58:07.716+0000 7efcb724b380 -1 osd.1 0 log_to_monitors {default=true} 2024-01-24T13:58:07.784 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-01-24T13:58:07.783+0000 7fc76bc67380 -1 osd.0 0 log_to_monitors {default=true} 2024-01-24T13:58:08.732 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-01-24T13:58:08.732+0000 7efcadc00700 -1 osd.1 0 waiting for initial osdmap 2024-01-24T13:58:08.733 INFO:tasks.ceph.osd.3.smithi029.stderr:2024-01-24T13:58:08.732+0000 7ff8d1374700 -1 osd.3 0 waiting for initial osdmap 2024-01-24T13:58:08.733 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-01-24T13:58:08.733+0000 7fdf6164d700 -1 osd.2 0 waiting for initial osdmap 2024-01-24T13:58:08.733 INFO:tasks.ceph.osd.5.smithi163.stderr:2024-01-24T13:58:08.733+0000 7f6c8840c700 -1 osd.5 0 waiting for initial osdmap 2024-01-24T13:58:08.734 INFO:tasks.ceph.osd.4.smithi163.stderr:2024-01-24T13:58:08.734+0000 7fc73b704700 -1 osd.4 0 waiting for initial osdmap 2024-01-24T13:58:08.734 INFO:tasks.ceph.osd.6.smithi163.stderr:2024-01-24T13:58:08.734+0000 7f95426f8700 -1 osd.6 0 waiting for initial osdmap 2024-01-24T13:58:08.734 INFO:tasks.ceph.osd.7.smithi163.stderr:2024-01-24T13:58:08.734+0000 7fd98c63e700 -1 osd.7 0 waiting for initial osdmap 2024-01-24T13:58:08.738 INFO:tasks.ceph.osd.2.smithi029.stderr:2024-01-24T13:58:08.738+0000 7fdf5ddc9700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T13:58:08.739 INFO:tasks.ceph.osd.3.smithi029.stderr:2024-01-24T13:58:08.738+0000 7ff8cdaf0700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T13:58:08.739 INFO:tasks.ceph.osd.1.smithi029.stderr:2024-01-24T13:58:08.738+0000 7efcaa37c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T13:58:08.740 INFO:tasks.ceph.osd.4.smithi163.stderr:2024-01-24T13:58:08.740+0000 7fc737e80700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T13:58:08.741 INFO:tasks.ceph.osd.6.smithi163.stderr:2024-01-24T13:58:08.741+0000 7f953ee74700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T13:58:08.741 INFO:tasks.ceph.osd.5.smithi163.stderr:2024-01-24T13:58:08.741+0000 7f6c84b88700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T13:58:08.742 INFO:tasks.ceph.osd.7.smithi163.stderr:2024-01-24T13:58:08.741+0000 7fd988dba700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T13:58:09.738 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-01-24T13:58:09.738+0000 7fc76261c700 -1 osd.0 0 waiting for initial osdmap 2024-01-24T13:58:09.744 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-01-24T13:58:09.744+0000 7fc75ed98700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T13:58:11.497 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:11.625 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.625+0000 7f350613f700 1 Processor -- start 2024-01-24T13:58:11.626 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.625+0000 7f350613f700 1 -- start start 2024-01-24T13:58:11.628 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.626+0000 7f350613f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3500062970 0x7f3500062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:11.628 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.626+0000 7f350613f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3500066b40 0x7f3500158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:11.628 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.626+0000 7f350613f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35000bcf60 con 0x7f35000c4da0 2024-01-24T13:58:11.628 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.626+0000 7f350613f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35000bdba0 con 0x7f3500062970 2024-01-24T13:58:11.628 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.626+0000 7f350613f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35000a7480 con 0x7f3500066b40 2024-01-24T13:58:11.628 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.626+0000 7f34ff7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3500062970 0x7f3500062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:11.628 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.626+0000 7f34ff7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3500062970 0x7f3500062d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59618/0 (socket says 172.21.15.29:59618) 2024-01-24T13:58:11.628 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.626+0000 7f34fffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f35000c4da0 0x7f3500069240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36568/0 (socket says 172.21.15.29:36568) 2024-01-24T13:58:11.628 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.626+0000 7f34ff7fe700 1 -- 172.21.15.29:0/1845900639 learned_addr learned my addr 172.21.15.29:0/1845900639 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:11.628 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.626+0000 7f34feffd700 1 --2- 172.21.15.29:0/1845900639 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3500066b40 0x7f3500158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.627+0000 7f34ff7fe700 1 -- 172.21.15.29:0/1845900639 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3500066b40 msgr2=0x7f3500158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.627+0000 7f34ff7fe700 1 --2- 172.21.15.29:0/1845900639 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3500066b40 0x7f3500158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.627+0000 7f34ff7fe700 1 -- 172.21.15.29:0/1845900639 >> v1:172.21.15.29:6789/0 conn(0x7f35000c4da0 legacy=0x7f3500069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.627+0000 7f34fe7fc700 1 -- 172.21.15.29:0/1845900639 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195104807 0 0) 0x7f35000bcf60 con 0x7f35000c4da0 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.627+0000 7f34ff7fe700 1 -- 172.21.15.29:0/1845900639 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3500158af0 con 0x7f3500062970 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.627+0000 7f34ff7fe700 1 --2- 172.21.15.29:0/1845900639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3500062970 0x7f3500062d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f34f0006fd0 tx=0x7f34f00090f0).ready entity=mon.0 client_cookie=20cd3e8282517e2b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.627+0000 7f34fe7fc700 1 -- 172.21.15.29:0/1845900639 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f000b580 con 0x7f3500062970 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.627+0000 7f34fe7fc700 1 -- 172.21.15.29:0/1845900639 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f000b6e0 con 0x7f3500062970 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.627+0000 7f34fe7fc700 1 -- 172.21.15.29:0/1845900639 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f000ba60 con 0x7f3500062970 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.627+0000 7f350613f700 1 -- 172.21.15.29:0/1845900639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3500062970 msgr2=0x7f3500062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.627+0000 7f350613f700 1 --2- 172.21.15.29:0/1845900639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3500062970 0x7f3500062d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f34f0006fd0 tx=0x7f34f00090f0).stop 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.628+0000 7f350613f700 1 -- 172.21.15.29:0/1845900639 shutdown_connections 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.628+0000 7f350613f700 1 --2- 172.21.15.29:0/1845900639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3500062970 0x7f3500062d50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.629 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.628+0000 7f350613f700 1 --2- 172.21.15.29:0/1845900639 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3500066b40 0x7f3500158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.630 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.628+0000 7f350613f700 1 -- 172.21.15.29:0/1845900639 >> 172.21.15.29:0/1845900639 conn(0x7f35000c8580 msgr2=0x7f35000c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:11.630 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.628+0000 7f350613f700 1 -- 172.21.15.29:0/1845900639 shutdown_connections 2024-01-24T13:58:11.630 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.628+0000 7f350613f700 1 -- 172.21.15.29:0/1845900639 wait complete. 2024-01-24T13:58:11.632 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.629+0000 7f350613f700 1 Processor -- start 2024-01-24T13:58:11.633 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.629+0000 7f350613f700 1 -- start start 2024-01-24T13:58:11.633 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f350613f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3500062970 0x7f350014c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:11.633 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f350613f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3500066b40 0x7f35000c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:11.633 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f350613f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35000c4da0 0x7f35000aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:11.633 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f350613f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35000c7240 con 0x7f35000c4da0 2024-01-24T13:58:11.633 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f350613f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350014fad0 con 0x7f3500062970 2024-01-24T13:58:11.633 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f350613f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35000c6c80 con 0x7f3500066b40 2024-01-24T13:58:11.633 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f34fffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35000c4da0 0x7f35000aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:11.633 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f34fffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35000c4da0 0x7f35000aaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59634/0 (socket says 172.21.15.29:59634) 2024-01-24T13:58:11.633 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f34fffff700 1 -- 172.21.15.29:0/1680138511 learned_addr learned my addr 172.21.15.29:0/1680138511 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:11.633 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f34feffd700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3500066b40 0x7f35000c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:11.633 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f34ff7fe700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3500062970 0x7f350014c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f34fffff700 1 -- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3500066b40 msgr2=0x7f35000c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f34fffff700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3500066b40 0x7f35000c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f34fffff700 1 -- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3500062970 msgr2=0x7f350014c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f34fffff700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3500062970 0x7f350014c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f34fffff700 1 -- 172.21.15.29:0/1680138511 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f0009370 con 0x7f35000c4da0 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.630+0000 7f34fffff700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35000c4da0 0x7f35000aaee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f34f800c760 tx=0x7f34f8001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.631+0000 7f34f77fe700 1 -- 172.21.15.29:0/1680138511 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8005c70 con 0x7f35000c4da0 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.631+0000 7f34f77fe700 1 -- 172.21.15.29:0/1680138511 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f8005dd0 con 0x7f35000c4da0 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.631+0000 7f350613f700 1 -- 172.21.15.29:0/1680138511 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35000c6f60 con 0x7f35000c4da0 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.631+0000 7f34f77fe700 1 -- 172.21.15.29:0/1680138511 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8007300 con 0x7f35000c4da0 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.631+0000 7f350613f700 1 -- 172.21.15.29:0/1680138511 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350014ca50 con 0x7f35000c4da0 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.632+0000 7f350613f700 1 -- 172.21.15.29:0/1680138511 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350014fc60 con 0x7f35000c4da0 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.632+0000 7f34f77fe700 1 -- 172.21.15.29:0/1680138511 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f34f80076b0 con 0x7f35000c4da0 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.632+0000 7f34f77fe700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f34e005f150 0x7f34e0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:11.634 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.632+0000 7f34f77fe700 1 -- 172.21.15.29:0/1680138511 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4892+0+0 (secure 0 0 0) 0x7f34f8079700 con 0x7f35000c4da0 2024-01-24T13:58:11.635 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.633+0000 7f34ff7fe700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f34e005f150 0x7f34e0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:11.635 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.633+0000 7f34ff7fe700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f34e005f150 0x7f34e0061610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f34f0009fd0 tx=0x7f34f0017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:11.637 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.637+0000 7f34f77fe700 1 -- 172.21.15.29:0/1680138511 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f34f804aea0 con 0x7f35000c4da0 2024-01-24T13:58:11.821 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.820+0000 7f350613f700 1 -- 172.21.15.29:0/1680138511 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35000c5700 con 0x7f35000c4da0 2024-01-24T13:58:11.821 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.821+0000 7f34f77fe700 1 -- 172.21.15.29:0/1680138511 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+12202 (secure 0 0 0) 0x7f34f8018030 con 0x7f35000c4da0 2024-01-24T13:58:11.822 INFO:teuthology.misc.health.smithi029.stdout: 2024-01-24T13:58:11.822 INFO:teuthology.misc.health.smithi029.stdout:{"epoch":15,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:11.743171+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:11.824 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.823+0000 7f350613f700 1 -- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f34e005f150 msgr2=0x7f34e0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:11.824 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.823+0000 7f350613f700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f34e005f150 0x7f34e0061610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f34f0009fd0 tx=0x7f34f0017000).stop 2024-01-24T13:58:11.824 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.823+0000 7f350613f700 1 -- 172.21.15.29:0/1680138511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35000c4da0 msgr2=0x7f35000aaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:11.824 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.823+0000 7f350613f700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35000c4da0 0x7f35000aaee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f34f800c760 tx=0x7f34f8001520).stop 2024-01-24T13:58:11.824 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.823+0000 7f350613f700 1 -- 172.21.15.29:0/1680138511 shutdown_connections 2024-01-24T13:58:11.824 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.823+0000 7f350613f700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f34e005f150 0x7f34e0061610 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.824 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.823+0000 7f350613f700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35000c4da0 0x7f35000aaee0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.825 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.823+0000 7f350613f700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3500062970 0x7f350014c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.825 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.823+0000 7f350613f700 1 --2- 172.21.15.29:0/1680138511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3500066b40 0x7f35000c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.825 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.823+0000 7f350613f700 1 -- 172.21.15.29:0/1680138511 >> 172.21.15.29:0/1680138511 conn(0x7f35000c8580 msgr2=0x7f35000638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:11.825 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.823+0000 7f350613f700 1 -- 172.21.15.29:0/1680138511 shutdown_connections 2024-01-24T13:58:11.825 INFO:teuthology.misc.health.smithi029.stderr:2024-01-24T13:58:11.823+0000 7f350613f700 1 -- 172.21.15.29:0/1680138511 wait complete. 2024-01-24T13:58:11.836 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-01-24T13:58:11.836 INFO:tasks.ceph:Creating RBD pool 2024-01-24T13:58:11.836 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd pool create rbd 8 2024-01-24T13:58:11.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.965+0000 7f8e9e498700 1 Processor -- start 2024-01-24T13:58:11.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.966+0000 7f8e9e498700 1 -- start start 2024-01-24T13:58:11.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.966+0000 7f8e9e498700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e98157380 0x7f8e98063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:11.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.966+0000 7f8e9e498700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8e98157760 0x7f8e98157c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.966+0000 7f8e9e498700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e980627b0 con 0x7f8e98158110 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.966+0000 7f8e9e498700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e98062d70 con 0x7f8e98157380 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.966+0000 7f8e9e498700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e981586e0 con 0x7f8e98157760 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e97fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e98157380 0x7f8e98063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e9ca35700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8e98158110 0x7f8e98064b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36582/0 (socket says 172.21.15.29:36582) 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e9ca35700 1 -- 172.21.15.29:0/1076701891 learned_addr learned my addr 172.21.15.29:0/1076701891 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e977fe700 1 --2- 172.21.15.29:0/1076701891 >> v2:172.21.15.163:3300/0 conn(0x7f8e98157760 0x7f8e98157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e97fff700 1 -- 172.21.15.29:0/1076701891 >> v2:172.21.15.163:3300/0 conn(0x7f8e98157760 msgr2=0x7f8e98157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e97fff700 1 --2- 172.21.15.29:0/1076701891 >> v2:172.21.15.163:3300/0 conn(0x7f8e98157760 0x7f8e98157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e97fff700 1 -- 172.21.15.29:0/1076701891 >> v1:172.21.15.29:6789/0 conn(0x7f8e98158110 legacy=0x7f8e98064b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e97fff700 1 -- 172.21.15.29:0/1076701891 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e981589c0 con 0x7f8e98157380 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e96ffd700 1 -- 172.21.15.29:0/1076701891 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252544317 0 0) 0x7f8e980627b0 con 0x7f8e98158110 2024-01-24T13:58:11.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e97fff700 1 --2- 172.21.15.29:0/1076701891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e98157380 0x7f8e98063200 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f8e980be350 tx=0x7f8e88009060).ready entity=mon.0 client_cookie=582e231d194f5e6e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e96ffd700 1 -- 172.21.15.29:0/1076701891 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e880090d0 con 0x7f8e98157380 2024-01-24T13:58:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e96ffd700 1 -- 172.21.15.29:0/1076701891 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8800f510 con 0x7f8e98157380 2024-01-24T13:58:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.967+0000 7f8e96ffd700 1 -- 172.21.15.29:0/1076701891 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8800f890 con 0x7f8e98157380 2024-01-24T13:58:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.968+0000 7f8e9e498700 1 -- 172.21.15.29:0/1076701891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e98157380 msgr2=0x7f8e98063200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.968+0000 7f8e9e498700 1 --2- 172.21.15.29:0/1076701891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e98157380 0x7f8e98063200 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f8e980be350 tx=0x7f8e88009060).stop 2024-01-24T13:58:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.968+0000 7f8e9e498700 1 -- 172.21.15.29:0/1076701891 shutdown_connections 2024-01-24T13:58:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.968+0000 7f8e9e498700 1 --2- 172.21.15.29:0/1076701891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e98157380 0x7f8e98063200 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.968+0000 7f8e9e498700 1 --2- 172.21.15.29:0/1076701891 >> v2:172.21.15.163:3300/0 conn(0x7f8e98157760 0x7f8e98157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.968+0000 7f8e9e498700 1 -- 172.21.15.29:0/1076701891 >> 172.21.15.29:0/1076701891 conn(0x7f8e980c7260 msgr2=0x7f8e980c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.968+0000 7f8e9e498700 1 -- 172.21.15.29:0/1076701891 shutdown_connections 2024-01-24T13:58:11.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.968+0000 7f8e9e498700 1 -- 172.21.15.29:0/1076701891 wait complete. 2024-01-24T13:58:11.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.969+0000 7f8e9e498700 1 Processor -- start 2024-01-24T13:58:11.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.969+0000 7f8e9e498700 1 -- start start 2024-01-24T13:58:11.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.970+0000 7f8e9e498700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e98157380 0x7f8e980a8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:11.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.970+0000 7f8e9e498700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e98157760 0x7f8e98143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:11.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.970+0000 7f8e9e498700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e98158110 0x7f8e98140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.970+0000 7f8e9e498700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e98143c10 con 0x7f8e98157760 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.970+0000 7f8e9e498700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e98142030 con 0x7f8e98158110 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.970+0000 7f8e9e498700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e98141420 con 0x7f8e98157380 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.970+0000 7f8e977fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e98157760 0x7f8e98143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.970+0000 7f8e977fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e98157760 0x7f8e98143720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59652/0 (socket says 172.21.15.29:59652) 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.970+0000 7f8e977fe700 1 -- 172.21.15.29:0/3079893914 learned_addr learned my addr 172.21.15.29:0/3079893914 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.970+0000 7f8e97fff700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e98157380 0x7f8e980a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.970+0000 7f8e9ca35700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e98158110 0x7f8e98140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.971+0000 7f8e977fe700 1 -- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e98157380 msgr2=0x7f8e980a8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.971+0000 7f8e977fe700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e98157380 0x7f8e980a8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.971+0000 7f8e977fe700 1 -- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e98158110 msgr2=0x7f8e98140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.971+0000 7f8e977fe700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e98158110 0x7f8e98140ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.971+0000 7f8e977fe700 1 -- 172.21.15.29:0/3079893914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e880092a0 con 0x7f8e98157760 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.971+0000 7f8e977fe700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e98157760 0x7f8e98143720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f8e8c00e900 tx=0x7f8e8c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.971+0000 7f8e7ffff700 1 -- 172.21.15.29:0/3079893914 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c012600 con 0x7f8e98157760 2024-01-24T13:58:11.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.971+0000 7f8e7ffff700 1 -- 172.21.15.29:0/3079893914 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8c012760 con 0x7f8e98157760 2024-01-24T13:58:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.971+0000 7f8e9e498700 1 -- 172.21.15.29:0/3079893914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e98141700 con 0x7f8e98157760 2024-01-24T13:58:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.971+0000 7f8e7ffff700 1 -- 172.21.15.29:0/3079893914 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c012ae0 con 0x7f8e98157760 2024-01-24T13:58:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.971+0000 7f8e9e498700 1 -- 172.21.15.29:0/3079893914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e98141cc0 con 0x7f8e98157760 2024-01-24T13:58:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.972+0000 7f8e9e498700 1 -- 172.21.15.29:0/3079893914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e981421c0 con 0x7f8e98157760 2024-01-24T13:58:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.972+0000 7f8e7ffff700 1 -- 172.21.15.29:0/3079893914 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8e8c01b400 con 0x7f8e98157760 2024-01-24T13:58:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.973+0000 7f8e7ffff700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e8405f150 0x7f8e84061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.973+0000 7f8e7ffff700 1 -- 172.21.15.29:0/3079893914 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4916+0+0 (secure 0 0 0) 0x7f8e8c07a140 con 0x7f8e98157760 2024-01-24T13:58:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.973+0000 7f8e97fff700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e8405f150 0x7f8e84061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:11.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.973+0000 7f8e97fff700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e8405f150 0x7f8e84061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8e980be350 tx=0x7f8e88017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:11.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:11.977+0000 7f8e7ffff700 1 -- 172.21.15.29:0/3079893914 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8e8c04b860 con 0x7f8e98157760 2024-01-24T13:58:12.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.158+0000 7f8e9e498700 1 -- 172.21.15.29:0/3079893914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f8e98062a00 con 0x7f8e98157760 2024-01-24T13:58:12.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.751+0000 7f8e7ffff700 1 -- 172.21.15.29:0/3079893914 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7f8e8c016070 con 0x7f8e98157760 2024-01-24T13:58:12.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.754+0000 7f8e9e498700 1 -- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e8405f150 msgr2=0x7f8e84061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:12.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.754+0000 7f8e9e498700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e8405f150 0x7f8e84061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8e980be350 tx=0x7f8e88017000).stop 2024-01-24T13:58:12.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.754+0000 7f8e9e498700 1 -- 172.21.15.29:0/3079893914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e98157760 msgr2=0x7f8e98143720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:12.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.754+0000 7f8e9e498700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e98157760 0x7f8e98143720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f8e8c00e900 tx=0x7f8e8c009060).stop 2024-01-24T13:58:12.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.754+0000 7f8e9e498700 1 -- 172.21.15.29:0/3079893914 shutdown_connections 2024-01-24T13:58:12.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.754+0000 7f8e9e498700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e98158110 0x7f8e98140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:12.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.754+0000 7f8e9e498700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e98157380 0x7f8e980a8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:12.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.754+0000 7f8e9e498700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e8405f150 0x7f8e84061610 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:12.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.754+0000 7f8e9e498700 1 --2- 172.21.15.29:0/3079893914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e98157760 0x7f8e98143720 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:12.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.754+0000 7f8e9e498700 1 -- 172.21.15.29:0/3079893914 >> 172.21.15.29:0/3079893914 conn(0x7f8e980c7260 msgr2=0x7f8e98152240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:12.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.754+0000 7f8e9e498700 1 -- 172.21.15.29:0/3079893914 shutdown_connections 2024-01-24T13:58:12.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.754+0000 7f8e9e498700 1 -- 172.21.15.29:0/3079893914 wait complete. 2024-01-24T13:58:12.756 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'rbd' created 2024-01-24T13:58:12.770 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-01-24T13:58:12.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.904+0000 7f8e00cd9700 1 Processor -- start 2024-01-24T13:58:12.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.905+0000 7f8e00cd9700 1 -- start start 2024-01-24T13:58:12.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8e00cd9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8dfc1513d0 0x7f8dfc063180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:12.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8e00cd9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8dfc1517b0 0x7f8dfc151c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:12.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8e00cd9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dfc069930 con 0x7f8dfc0aa420 2024-01-24T13:58:12.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8e00cd9700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dfc068b10 con 0x7f8dfc1513d0 2024-01-24T13:58:12.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8e00cd9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dfc069dc0 con 0x7f8dfc1517b0 2024-01-24T13:58:12.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8dfa59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8dfc1513d0 0x7f8dfc063180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:12.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8dfa59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8dfc1513d0 0x7f8dfc063180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59660/0 (socket says 172.21.15.29:59660) 2024-01-24T13:58:12.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8dfa59c700 1 -- 172.21.15.29:0/235807674 learned_addr learned my addr 172.21.15.29:0/235807674 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:12.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8df9d9b700 1 --2- 172.21.15.29:0/235807674 >> v2:172.21.15.163:3300/0 conn(0x7f8dfc1517b0 0x7f8dfc151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:12.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8dfa59c700 1 -- 172.21.15.29:0/235807674 >> v2:172.21.15.163:3300/0 conn(0x7f8dfc1517b0 msgr2=0x7f8dfc151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:12.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8dfa59c700 1 --2- 172.21.15.29:0/235807674 >> v2:172.21.15.163:3300/0 conn(0x7f8dfc1517b0 0x7f8dfc151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:12.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8dfa59c700 1 -- 172.21.15.29:0/235807674 >> v1:172.21.15.163:6789/0 conn(0x7f8dfc0aa420 legacy=0x7f8dfc0aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:12.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8dfa59c700 1 -- 172.21.15.29:0/235807674 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc068d20 con 0x7f8dfc1513d0 2024-01-24T13:58:12.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8dfa59c700 1 --2- 172.21.15.29:0/235807674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dfc1513d0 0x7f8dfc063180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f8dfc066330 tx=0x7f8dec0090f0).ready entity=mon.0 client_cookie=84b13cfc8d732a24 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:12.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8df959a700 1 -- 172.21.15.29:0/235807674 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec00b520 con 0x7f8dfc1513d0 2024-01-24T13:58:12.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8df959a700 1 -- 172.21.15.29:0/235807674 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec00b680 con 0x7f8dfc1513d0 2024-01-24T13:58:12.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.906+0000 7f8df959a700 1 -- 172.21.15.29:0/235807674 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec00ba00 con 0x7f8dfc1513d0 2024-01-24T13:58:12.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.907+0000 7f8e00cd9700 1 -- 172.21.15.29:0/235807674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dfc1513d0 msgr2=0x7f8dfc063180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:12.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.907+0000 7f8e00cd9700 1 --2- 172.21.15.29:0/235807674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dfc1513d0 0x7f8dfc063180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f8dfc066330 tx=0x7f8dec0090f0).stop 2024-01-24T13:58:12.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.907+0000 7f8e00cd9700 1 -- 172.21.15.29:0/235807674 shutdown_connections 2024-01-24T13:58:12.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.907+0000 7f8e00cd9700 1 --2- 172.21.15.29:0/235807674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dfc1513d0 0x7f8dfc063180 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:12.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.907+0000 7f8e00cd9700 1 --2- 172.21.15.29:0/235807674 >> v2:172.21.15.163:3300/0 conn(0x7f8dfc1517b0 0x7f8dfc151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:12.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.907+0000 7f8e00cd9700 1 -- 172.21.15.29:0/235807674 >> 172.21.15.29:0/235807674 conn(0x7f8dfc0c7280 msgr2=0x7f8dfc0c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:12.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.907+0000 7f8e00cd9700 1 -- 172.21.15.29:0/235807674 shutdown_connections 2024-01-24T13:58:12.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.907+0000 7f8e00cd9700 1 -- 172.21.15.29:0/235807674 wait complete. 2024-01-24T13:58:12.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.908+0000 7f8e00cd9700 1 Processor -- start 2024-01-24T13:58:12.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.908+0000 7f8e00cd9700 1 -- start start 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.909+0000 7f8e00cd9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dfc0aa420 0x7f8dfc0c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.909+0000 7f8e00cd9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dfc1513d0 0x7f8dfc0c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.909+0000 7f8e00cd9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8dfc1517b0 0x7f8dfc0c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.909+0000 7f8e00cd9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc0c46c0 con 0x7f8dfc1513d0 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.909+0000 7f8e00cd9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc0c6a10 con 0x7f8dfc0aa420 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.909+0000 7f8e00cd9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc1554b0 con 0x7f8dfc1517b0 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.909+0000 7f8df9d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dfc1513d0 0x7f8dfc0c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.909+0000 7f8df9d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dfc1513d0 0x7f8dfc0c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59664/0 (socket says 172.21.15.29:59664) 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.909+0000 7f8df9d9b700 1 -- 172.21.15.29:0/1574590936 learned_addr learned my addr 172.21.15.29:0/1574590936 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.909+0000 7f8dfa59c700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dfc0aa420 0x7f8dfc0c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.909+0000 7f8dfad9d700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8dfc1517b0 0x7f8dfc0c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.910+0000 7f8df9d9b700 1 -- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8dfc1517b0 msgr2=0x7f8dfc0c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.910+0000 7f8df9d9b700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8dfc1517b0 0x7f8dfc0c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:12.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.910+0000 7f8df9d9b700 1 -- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dfc0aa420 msgr2=0x7f8dfc0c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.910+0000 7f8df9d9b700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dfc0aa420 0x7f8dfc0c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.910+0000 7f8df9d9b700 1 -- 172.21.15.29:0/1574590936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec009370 con 0x7f8dfc1513d0 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.910+0000 7f8df9d9b700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dfc1513d0 0x7f8dfc0c6030 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8df000e900 tx=0x7f8df0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.910+0000 7f8dea7fc700 1 -- 172.21.15.29:0/1574590936 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0012600 con 0x7f8dfc1513d0 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.910+0000 7f8dea7fc700 1 -- 172.21.15.29:0/1574590936 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df0012760 con 0x7f8dfc1513d0 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.910+0000 7f8e00cd9700 1 -- 172.21.15.29:0/1574590936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dfc1548a0 con 0x7f8dfc1513d0 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.910+0000 7f8dea7fc700 1 -- 172.21.15.29:0/1574590936 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0012ae0 con 0x7f8dfc1513d0 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.910+0000 7f8e00cd9700 1 -- 172.21.15.29:0/1574590936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dfc154e60 con 0x7f8dfc1513d0 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.911+0000 7f8e00cd9700 1 -- 172.21.15.29:0/1574590936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dfc0475e0 con 0x7f8dfc1513d0 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.912+0000 7f8dea7fc700 1 -- 172.21.15.29:0/1574590936 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8df001b400 con 0x7f8dfc1513d0 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.912+0000 7f8dea7fc700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8ddc05f150 0x7f8ddc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.912+0000 7f8dea7fc700 1 -- 172.21.15.29:0/1574590936 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5258+0+0 (secure 0 0 0) 0x7f8df007a0f0 con 0x7f8dfc1513d0 2024-01-24T13:58:12.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.912+0000 7f8dfa59c700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8ddc05f150 0x7f8ddc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:12.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.913+0000 7f8dfa59c700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8ddc05f150 0x7f8ddc061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f8dec009130 tx=0x7f8dec017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:12.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:12.917+0000 7f8dea7fc700 1 -- 172.21.15.29:0/1574590936 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8df004b6c0 con 0x7f8dfc1513d0 2024-01-24T13:58:13.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.097+0000 7f8e00cd9700 1 -- 172.21.15.29:0/1574590936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8dfc06b390 con 0x7f8dfc1513d0 2024-01-24T13:58:13.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.757+0000 7f8dea7fc700 1 -- 172.21.15.29:0/1574590936 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8df004ece0 con 0x7f8dfc1513d0 2024-01-24T13:58:13.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.760+0000 7f8e00cd9700 1 -- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8ddc05f150 msgr2=0x7f8ddc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:13.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.760+0000 7f8e00cd9700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8ddc05f150 0x7f8ddc061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f8dec009130 tx=0x7f8dec017000).stop 2024-01-24T13:58:13.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.760+0000 7f8e00cd9700 1 -- 172.21.15.29:0/1574590936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dfc1513d0 msgr2=0x7f8dfc0c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:13.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.760+0000 7f8e00cd9700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dfc1513d0 0x7f8dfc0c6030 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8df000e900 tx=0x7f8df0009060).stop 2024-01-24T13:58:13.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.760+0000 7f8e00cd9700 1 -- 172.21.15.29:0/1574590936 shutdown_connections 2024-01-24T13:58:13.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.760+0000 7f8e00cd9700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8dfc1517b0 0x7f8dfc0c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:13.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.760+0000 7f8e00cd9700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dfc0aa420 0x7f8dfc0c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:13.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.760+0000 7f8e00cd9700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8ddc05f150 0x7f8ddc061610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:13.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.760+0000 7f8e00cd9700 1 --2- 172.21.15.29:0/1574590936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8dfc1513d0 0x7f8dfc0c6030 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:13.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.760+0000 7f8e00cd9700 1 -- 172.21.15.29:0/1574590936 >> 172.21.15.29:0/1574590936 conn(0x7f8dfc0c7280 msgr2=0x7f8dfc0ab6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:13.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.761+0000 7f8e00cd9700 1 -- 172.21.15.29:0/1574590936 shutdown_connections 2024-01-24T13:58:13.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.761+0000 7f8e00cd9700 1 -- 172.21.15.29:0/1574590936 wait complete. 2024-01-24T13:58:13.762 INFO:teuthology.orchestra.run.smithi029.stderr:enabled application 'rbd' on pool 'rbd' 2024-01-24T13:58:13.778 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-24T13:58:13.778 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-24T13:58:13.778 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-24T13:58:13.781 INFO:tasks.ceph.mds.a:Started 2024-01-24T13:58:13.781 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-24T13:58:13.781 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-24T13:58:13.783 INFO:tasks.ceph.mds.c:Started 2024-01-24T13:58:13.783 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-24T13:58:13.783 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-24T13:58:13.786 INFO:tasks.ceph.mds.b:Started 2024-01-24T13:58:13.786 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-24T13:58:13.786 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-24T13:58:13.788 INFO:tasks.ceph.mds.d:Started 2024-01-24T13:58:13.788 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-01-24T13:58:13.789 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:13.866 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-01-24T13:58:13.867 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T13:58:13.871 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T13:58:13.871 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T13:58:13.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.929+0000 7fc3080d5700 1 Processor -- start 2024-01-24T13:58:13.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.929+0000 7fc3080d5700 1 -- start start 2024-01-24T13:58:13.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc3080d5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc300134cc0 0x7fc3000c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:13.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc3080d5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc300066c00 0x7fc300152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:13.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc3080d5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc300068360 con 0x7fc3001350a0 2024-01-24T13:58:13.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc3080d5700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc300153360 con 0x7fc300134cc0 2024-01-24T13:58:13.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc3080d5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000c3820 con 0x7fc300066c00 2024-01-24T13:58:13.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc305e71700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc300134cc0 0x7fc3000c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:13.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc305e71700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc300134cc0 0x7fc3000c31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39116/0 (socket says 172.21.15.29:39116) 2024-01-24T13:58:13.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc305e71700 1 -- 172.21.15.29:0/1965497152 learned_addr learned my addr 172.21.15.29:0/1965497152 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:13.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc306672700 1 --2- 172.21.15.29:0/1965497152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc300066c00 0x7fc300152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:13.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc305e71700 1 -- 172.21.15.29:0/1965497152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc300066c00 msgr2=0x7fc300152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:13.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc305e71700 1 --2- 172.21.15.29:0/1965497152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc300066c00 0x7fc300152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc305e71700 1 -- 172.21.15.29:0/1965497152 >> v1:172.21.15.163:6789/0 conn(0x7fc3001350a0 legacy=0x7fc3000664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.930+0000 7fc305e71700 1 -- 172.21.15.29:0/1965497152 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc300067a30 con 0x7fc300134cc0 2024-01-24T13:58:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.931+0000 7fc305e71700 1 --2- 172.21.15.29:0/1965497152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc300134cc0 0x7fc3000c31b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fc2fc006fd0 tx=0x7fc2fc0090f0).ready entity=mon.0 client_cookie=c3b8ae40149077c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.931+0000 7fc304e6f700 1 -- 172.21.15.29:0/1965497152 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00b580 con 0x7fc300134cc0 2024-01-24T13:58:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.931+0000 7fc304e6f700 1 -- 172.21.15.29:0/1965497152 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc00b6e0 con 0x7fc300134cc0 2024-01-24T13:58:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.931+0000 7fc304e6f700 1 -- 172.21.15.29:0/1965497152 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00ba60 con 0x7fc300134cc0 2024-01-24T13:58:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.931+0000 7fc3080d5700 1 -- 172.21.15.29:0/1965497152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc300134cc0 msgr2=0x7fc3000c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.931+0000 7fc3080d5700 1 --2- 172.21.15.29:0/1965497152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc300134cc0 0x7fc3000c31b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fc2fc006fd0 tx=0x7fc2fc0090f0).stop 2024-01-24T13:58:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.931+0000 7fc3080d5700 1 -- 172.21.15.29:0/1965497152 shutdown_connections 2024-01-24T13:58:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.931+0000 7fc3080d5700 1 --2- 172.21.15.29:0/1965497152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc300066c00 0x7fc300152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.931+0000 7fc3080d5700 1 --2- 172.21.15.29:0/1965497152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc300134cc0 0x7fc3000c31b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:13.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.931+0000 7fc3080d5700 1 -- 172.21.15.29:0/1965497152 >> 172.21.15.29:0/1965497152 conn(0x7fc3000c8bb0 msgr2=0x7fc3000c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.931+0000 7fc3080d5700 1 -- 172.21.15.29:0/1965497152 shutdown_connections 2024-01-24T13:58:13.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.931+0000 7fc3080d5700 1 -- 172.21.15.29:0/1965497152 wait complete. 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.933+0000 7fc3080d5700 1 Processor -- start 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.933+0000 7fc3080d5700 1 -- start start 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.933+0000 7fc3080d5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc300066c00 0x7fc300137190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.933+0000 7fc3080d5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc300134cc0 0x7fc30014b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.933+0000 7fc3080d5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3001350a0 0x7fc3000ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.933+0000 7fc3080d5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc300137680 con 0x7fc300134cc0 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.933+0000 7fc3080d5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc300135710 con 0x7fc3001350a0 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.933+0000 7fc3080d5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc300137890 con 0x7fc300066c00 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc305670700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc300134cc0 0x7fc30014b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc305670700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc300134cc0 0x7fc30014b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39132/0 (socket says 172.21.15.29:39132) 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc305670700 1 -- 172.21.15.29:0/2044590933 learned_addr learned my addr 172.21.15.29:0/2044590933 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc305e71700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc300066c00 0x7fc300137190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc306672700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3001350a0 0x7fc3000ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:13.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc305670700 1 -- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc300066c00 msgr2=0x7fc300137190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc305670700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc300066c00 0x7fc300137190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc305670700 1 -- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3001350a0 msgr2=0x7fc3000ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc305670700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3001350a0 0x7fc3000ad3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc305670700 1 -- 172.21.15.29:0/2044590933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc009370 con 0x7fc300134cc0 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc306672700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3001350a0 0x7fc3000ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc305670700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc300134cc0 0x7fc30014b740 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc2f000c440 tx=0x7fc2f0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc2f5ffb700 1 -- 172.21.15.29:0/2044590933 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0007020 con 0x7fc300134cc0 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc2f5ffb700 1 -- 172.21.15.29:0/2044590933 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f0007180 con 0x7fc300134cc0 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc2f5ffb700 1 -- 172.21.15.29:0/2044590933 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0007520 con 0x7fc300134cc0 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc3080d5700 1 -- 172.21.15.29:0/2044590933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc300137b70 con 0x7fc300134cc0 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.934+0000 7fc3080d5700 1 -- 172.21.15.29:0/2044590933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc300138130 con 0x7fc300134cc0 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.935+0000 7fc3080d5700 1 -- 172.21.15.29:0/2044590933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc300138480 con 0x7fc300134cc0 2024-01-24T13:58:13.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.936+0000 7fc2f5ffb700 1 -- 172.21.15.29:0/2044590933 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc2f00068b0 con 0x7fc300134cc0 2024-01-24T13:58:13.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.936+0000 7fc2f5ffb700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2e405f150 0x7fc2e4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:13.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.936+0000 7fc2f5ffb700 1 -- 172.21.15.29:0/2044590933 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc2f0079220 con 0x7fc300134cc0 2024-01-24T13:58:13.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.936+0000 7fc305e71700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2e405f150 0x7fc2e4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:13.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.936+0000 7fc305e71700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2e405f150 0x7fc2e4061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fc2fc009fd0 tx=0x7fc2fc017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:13.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:13.940+0000 7fc2f5ffb700 1 -- 172.21.15.29:0/2044590933 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc2f0047b00 con 0x7fc300134cc0 2024-01-24T13:58:14.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.121+0000 7fc3080d5700 1 -- 172.21.15.29:0/2044590933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3001382c0 con 0x7fc300134cc0 2024-01-24T13:58:14.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.122+0000 7fc2f5ffb700 1 -- 172.21.15.29:0/2044590933 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13722 (secure 0 0 0) 0x7fc2f004a840 con 0x7fc300134cc0 2024-01-24T13:58:14.122 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:14.122 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":17,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:13.751761+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:14.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.124+0000 7fc3080d5700 1 -- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2e405f150 msgr2=0x7fc2e4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.124+0000 7fc3080d5700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2e405f150 0x7fc2e4061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fc2fc009fd0 tx=0x7fc2fc017000).stop 2024-01-24T13:58:14.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.124+0000 7fc3080d5700 1 -- 172.21.15.29:0/2044590933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc300134cc0 msgr2=0x7fc30014b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.124+0000 7fc3080d5700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc300134cc0 0x7fc30014b740 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc2f000c440 tx=0x7fc2f0007e00).stop 2024-01-24T13:58:14.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.124+0000 7fc3080d5700 1 -- 172.21.15.29:0/2044590933 shutdown_connections 2024-01-24T13:58:14.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.124+0000 7fc3080d5700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3001350a0 0x7fc3000ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.124+0000 7fc3080d5700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc300066c00 0x7fc300137190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.124+0000 7fc3080d5700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2e405f150 0x7fc2e4061610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.124+0000 7fc3080d5700 1 --2- 172.21.15.29:0/2044590933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc300134cc0 0x7fc30014b740 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.124+0000 7fc3080d5700 1 -- 172.21.15.29:0/2044590933 >> 172.21.15.29:0/2044590933 conn(0x7fc3000c8bb0 msgr2=0x7fc300152230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:14.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.124+0000 7fc3080d5700 1 -- 172.21.15.29:0/2044590933 shutdown_connections 2024-01-24T13:58:14.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.124+0000 7fc3080d5700 1 -- 172.21.15.29:0/2044590933 wait complete. 2024-01-24T13:58:14.138 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T13:58:14.138 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T13:58:14.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.271+0000 7fb47079b700 1 Processor -- start 2024-01-24T13:58:14.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.271+0000 7fb47079b700 1 -- start start 2024-01-24T13:58:14.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.272+0000 7fb47079b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46813a5e0 0x7fb4680639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.272+0000 7fb47079b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb468066bd0 0x7fb4680bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.272+0000 7fb47079b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4680679f0 con 0x7fb468063eb0 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb47079b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb468067cd0 con 0x7fb468066bd0 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb47079b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb468067fb0 con 0x7fb46813a5e0 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46dd36700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb468063eb0 0x7fb4680664c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58784/0 (socket says 172.21.15.29:58784) 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46dd36700 1 -- 172.21.15.29:0/1424287843 learned_addr learned my addr 172.21.15.29:0/1424287843 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46e537700 1 --2- 172.21.15.29:0/1424287843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46813a5e0 0x7fb4680639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46ed38700 1 --2- 172.21.15.29:0/1424287843 >> v2:172.21.15.163:3300/0 conn(0x7fb468066bd0 0x7fb4680bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46d535700 1 -- 172.21.15.29:0/1424287843 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107826540 0 0) 0x7fb4680679f0 con 0x7fb468063eb0 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46d535700 1 -- 172.21.15.29:0/1424287843 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4540033f0 con 0x7fb468063eb0 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46ed38700 1 -- 172.21.15.29:0/1424287843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46813a5e0 msgr2=0x7fb4680639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46ed38700 1 --2- 172.21.15.29:0/1424287843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46813a5e0 0x7fb4680639c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46d535700 1 -- 172.21.15.29:0/1424287843 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 377091972 0 0) 0x7fb4540033f0 con 0x7fb468063eb0 2024-01-24T13:58:14.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46ed38700 1 -- 172.21.15.29:0/1424287843 >> v1:172.21.15.29:6789/0 conn(0x7fb468063eb0 legacy=0x7fb4680664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46ed38700 1 -- 172.21.15.29:0/1424287843 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468068290 con 0x7fb468066bd0 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46e537700 1 --2- 172.21.15.29:0/1424287843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46813a5e0 0x7fb4680639c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.273+0000 7fb46ed38700 1 --2- 172.21.15.29:0/1424287843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb468066bd0 0x7fb4680bdb30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb460009e60 tx=0x7fb460009780).ready entity=mon.1 client_cookie=767b0d67b76fd9d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.274+0000 7fb46d535700 1 -- 172.21.15.29:0/1424287843 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46000e070 con 0x7fb468066bd0 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.274+0000 7fb46d535700 1 -- 172.21.15.29:0/1424287843 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46000b7a0 con 0x7fb468066bd0 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.274+0000 7fb46d535700 1 -- 172.21.15.29:0/1424287843 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb464001470 con 0x7fb468066bd0 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.274+0000 7fb47079b700 1 -- 172.21.15.29:0/1424287843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb468066bd0 msgr2=0x7fb4680bdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.274+0000 7fb47079b700 1 --2- 172.21.15.29:0/1424287843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb468066bd0 0x7fb4680bdb30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb460009e60 tx=0x7fb460009780).stop 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.274+0000 7fb47079b700 1 -- 172.21.15.29:0/1424287843 shutdown_connections 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.274+0000 7fb47079b700 1 --2- 172.21.15.29:0/1424287843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb468066bd0 0x7fb4680bdb30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.274+0000 7fb47079b700 1 --2- 172.21.15.29:0/1424287843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46813a5e0 0x7fb4680639c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.274+0000 7fb47079b700 1 -- 172.21.15.29:0/1424287843 >> 172.21.15.29:0/1424287843 conn(0x7fb4680c8bd0 msgr2=0x7fb4680c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:14.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.274+0000 7fb47079b700 1 -- 172.21.15.29:0/1424287843 shutdown_connections 2024-01-24T13:58:14.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.274+0000 7fb47079b700 1 -- 172.21.15.29:0/1424287843 wait complete. 2024-01-24T13:58:14.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.275+0000 7fb47079b700 1 Processor -- start 2024-01-24T13:58:14.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.276+0000 7fb47079b700 1 -- start start 2024-01-24T13:58:14.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.276+0000 7fb47079b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb468063eb0 0x7fb4681387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.276+0000 7fb47079b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb468066bd0 0x7fb468155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.276+0000 7fb47079b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46813a5e0 0x7fb468155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.276+0000 7fb47079b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb468155020 con 0x7fb468063eb0 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.276+0000 7fb47079b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680ae900 con 0x7fb468066bd0 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.276+0000 7fb47079b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb468138cc0 con 0x7fb46813a5e0 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.276+0000 7fb46e537700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb468063eb0 0x7fb4681387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.276+0000 7fb46e537700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb468063eb0 0x7fb4681387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39146/0 (socket says 172.21.15.29:39146) 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.276+0000 7fb46e537700 1 -- 172.21.15.29:0/4240962312 learned_addr learned my addr 172.21.15.29:0/4240962312 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.276+0000 7fb46ed38700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46813a5e0 0x7fb468155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.277+0000 7fb46dd36700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb468066bd0 0x7fb468155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.277+0000 7fb46e537700 1 -- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46813a5e0 msgr2=0x7fb468155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.277+0000 7fb46e537700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46813a5e0 0x7fb468155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.277+0000 7fb46e537700 1 -- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb468066bd0 msgr2=0x7fb468155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.277+0000 7fb46e537700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb468066bd0 0x7fb468155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.277+0000 7fb46e537700 1 -- 172.21.15.29:0/4240962312 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4580092a0 con 0x7fb468063eb0 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.277+0000 7fb46e537700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb468063eb0 0x7fb4681387d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb45800e980 tx=0x7fb458009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:14.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.277+0000 7fb45e7fc700 1 -- 172.21.15.29:0/4240962312 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458012600 con 0x7fb468063eb0 2024-01-24T13:58:14.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.277+0000 7fb47079b700 1 -- 172.21.15.29:0/4240962312 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb460009a20 con 0x7fb468063eb0 2024-01-24T13:58:14.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.277+0000 7fb45e7fc700 1 -- 172.21.15.29:0/4240962312 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb458012760 con 0x7fb468063eb0 2024-01-24T13:58:14.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.277+0000 7fb45e7fc700 1 -- 172.21.15.29:0/4240962312 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458012ae0 con 0x7fb468063eb0 2024-01-24T13:58:14.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.277+0000 7fb47079b700 1 -- 172.21.15.29:0/4240962312 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb468157470 con 0x7fb468063eb0 2024-01-24T13:58:14.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.278+0000 7fb47079b700 1 -- 172.21.15.29:0/4240962312 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4681555e0 con 0x7fb468063eb0 2024-01-24T13:58:14.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.279+0000 7fb45e7fc700 1 -- 172.21.15.29:0/4240962312 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb45801b400 con 0x7fb468063eb0 2024-01-24T13:58:14.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.279+0000 7fb45e7fc700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb44c05f150 0x7fb44c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.279+0000 7fb45e7fc700 1 -- 172.21.15.29:0/4240962312 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fb45807a350 con 0x7fb468063eb0 2024-01-24T13:58:14.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.279+0000 7fb46dd36700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb44c05f150 0x7fb44c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.280+0000 7fb46dd36700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb44c05f150 0x7fb44c061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb468167430 tx=0x7fb464001a10).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:14.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.284+0000 7fb45e7fc700 1 -- 172.21.15.29:0/4240962312 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb45804b910 con 0x7fb468063eb0 2024-01-24T13:58:14.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.463+0000 7fb47079b700 1 -- 172.21.15.29:0/4240962312 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb4680475e0 con 0x7fb468063eb0 2024-01-24T13:58:14.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.464+0000 7fb45e7fc700 1 -- 172.21.15.29:0/4240962312 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb45804ef30 con 0x7fb468063eb0 2024-01-24T13:58:14.464 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T13:58:14.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.466+0000 7fb47079b700 1 -- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb44c05f150 msgr2=0x7fb44c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.466+0000 7fb47079b700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb44c05f150 0x7fb44c061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb468167430 tx=0x7fb464001a10).stop 2024-01-24T13:58:14.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.466+0000 7fb47079b700 1 -- 172.21.15.29:0/4240962312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb468063eb0 msgr2=0x7fb4681387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.466+0000 7fb47079b700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb468063eb0 0x7fb4681387d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb45800e980 tx=0x7fb458009060).stop 2024-01-24T13:58:14.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.466+0000 7fb47079b700 1 -- 172.21.15.29:0/4240962312 shutdown_connections 2024-01-24T13:58:14.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.466+0000 7fb47079b700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46813a5e0 0x7fb468155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.466+0000 7fb47079b700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb44c05f150 0x7fb44c061610 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.466+0000 7fb47079b700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb468063eb0 0x7fb4681387d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.466+0000 7fb47079b700 1 --2- 172.21.15.29:0/4240962312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb468066bd0 0x7fb468155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.466+0000 7fb47079b700 1 -- 172.21.15.29:0/4240962312 >> 172.21.15.29:0/4240962312 conn(0x7fb4680c8bd0 msgr2=0x7fb4680be500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:14.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.466+0000 7fb47079b700 1 -- 172.21.15.29:0/4240962312 shutdown_connections 2024-01-24T13:58:14.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.466+0000 7fb47079b700 1 -- 172.21.15.29:0/4240962312 wait complete. 2024-01-24T13:58:14.479 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T13:58:14.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.612+0000 7f09feb5e700 1 Processor -- start 2024-01-24T13:58:14.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.612+0000 7f09feb5e700 1 -- start start 2024-01-24T13:58:14.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09feb5e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f09f8066940 0x7f09f80be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09feb5e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f80686a0 con 0x7f09f80664c0 2024-01-24T13:58:14.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09feb5e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f80c38e0 con 0x7f09f8134ea0 2024-01-24T13:58:14.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09feb5e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f8068b30 con 0x7f09f8066940 2024-01-24T13:58:14.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09f7fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f09f80664c0 0x7f09f80bdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58798/0 (socket says 172.21.15.29:58798) 2024-01-24T13:58:14.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09f7fff700 1 -- 172.21.15.29:0/1648630554 learned_addr learned my addr 172.21.15.29:0/1648630554 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09fd0fb700 1 --2- 172.21.15.29:0/1648630554 >> v2:172.21.15.163:3300/0 conn(0x7f09f8066940 0x7f09f80be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378972983 0 0) 0x7f09f80686a0 con 0x7f09f80664c0 2024-01-24T13:58:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09e80033f0 con 0x7f09f80664c0 2024-01-24T13:58:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 399425410 0 0) 0x7f09e80033f0 con 0x7f09f80664c0 2024-01-24T13:58:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 >> v2:172.21.15.163:3300/0 conn(0x7f09f8066940 msgr2=0x7f09f80be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09f77fe700 1 --2- 172.21.15.29:0/1648630554 >> v2:172.21.15.163:3300/0 conn(0x7f09f8066940 0x7f09f80be240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 >> v1:172.21.15.163:6789/0 conn(0x7f09f8134ea0 legacy=0x7f09f80c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f8066dd0 con 0x7f09f80664c0 2024-01-24T13:58:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f09e4001470 con 0x7f09f80664c0 2024-01-24T13:58:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.613+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 >> v1:172.21.15.29:6789/0 conn(0x7f09f80664c0 legacy=0x7f09f80bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09f77fe700 1 --2- 172.21.15.29:0/1648630554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09e80038e0 0x7f09e8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09f77fe700 1 --2- 172.21.15.29:0/1648630554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e8004690 0x7f09e8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09f77fe700 1 --2- 172.21.15.29:0/1648630554 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09e8005000 0x7f09e800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f8068b30 con 0x7f09e80038e0 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e800beb0 con 0x7f09e8004690 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e800c070 con 0x7f09e8005000 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09fc8fa700 1 --2- 172.21.15.29:0/1648630554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09e80038e0 0x7f09e8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09f7fff700 1 --2- 172.21.15.29:0/1648630554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e8004690 0x7f09e8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09fd0fb700 1 --2- 172.21.15.29:0/1648630554 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09e8005000 0x7f09e800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09fc8fa700 1 -- 172.21.15.29:0/1648630554 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09e8005000 msgr2=0x7f09e800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09fc8fa700 1 --2- 172.21.15.29:0/1648630554 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09e8005000 0x7f09e800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09fc8fa700 1 -- 172.21.15.29:0/1648630554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e8004690 msgr2=0x7f09e8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09fc8fa700 1 --2- 172.21.15.29:0/1648630554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e8004690 0x7f09e8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09fd0fb700 1 -- 172.21.15.29:0/1648630554 reap_dead start 2024-01-24T13:58:14.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09fc8fa700 1 -- 172.21.15.29:0/1648630554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e80033f0 con 0x7f09e80038e0 2024-01-24T13:58:14.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09fc8fa700 1 --2- 172.21.15.29:0/1648630554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09e80038e0 0x7f09e8003d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f09ec001730 tx=0x7f09ec007bb0).ready entity=mon.0 client_cookie=90ea4a0cc7a06ef3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:14.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec007020 con 0x7f09e80038e0 2024-01-24T13:58:14.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec007180 con 0x7f09e80038e0 2024-01-24T13:58:14.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.614+0000 7f09f77fe700 1 -- 172.21.15.29:0/1648630554 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec007500 con 0x7f09e80038e0 2024-01-24T13:58:14.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.615+0000 7f09feb5e700 1 -- 172.21.15.29:0/1648630554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09e80038e0 msgr2=0x7f09e8003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.615+0000 7f09feb5e700 1 --2- 172.21.15.29:0/1648630554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09e80038e0 0x7f09e8003d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f09ec001730 tx=0x7f09ec007bb0).stop 2024-01-24T13:58:14.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.615+0000 7f09feb5e700 1 -- 172.21.15.29:0/1648630554 shutdown_connections 2024-01-24T13:58:14.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.615+0000 7f09feb5e700 1 --2- 172.21.15.29:0/1648630554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09e80038e0 0x7f09e8003d00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.615+0000 7f09feb5e700 1 -- 172.21.15.29:0/1648630554 >> 172.21.15.29:0/1648630554 conn(0x7f09f80c8bd0 msgr2=0x7f09f80c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:14.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.615+0000 7f09feb5e700 1 -- 172.21.15.29:0/1648630554 shutdown_connections 2024-01-24T13:58:14.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.615+0000 7f09feb5e700 1 -- 172.21.15.29:0/1648630554 wait complete. 2024-01-24T13:58:14.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.616+0000 7f09feb5e700 1 Processor -- start 2024-01-24T13:58:14.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.616+0000 7f09feb5e700 1 -- start start 2024-01-24T13:58:14.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.617+0000 7f09feb5e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09e80038e0 0x7f09f80aab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.617+0000 7f09feb5e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09f8168310 0x7f09f80ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.617+0000 7f09feb5e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09f80ac4f0 0x7f09f80ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.617+0000 7f09feb5e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f80ab010 con 0x7f09f8168310 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.617+0000 7f09feb5e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f80a86b0 con 0x7f09f80ac4f0 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.617+0000 7f09feb5e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f80a9230 con 0x7f09e80038e0 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.617+0000 7f09f7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09f8168310 0x7f09f80ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.617+0000 7f09f7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09f8168310 0x7f09f80ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39168/0 (socket says 172.21.15.29:39168) 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.617+0000 7f09f7fff700 1 -- 172.21.15.29:0/1340217877 learned_addr learned my addr 172.21.15.29:0/1340217877 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.617+0000 7f09fc8fa700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09e80038e0 0x7f09f80aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.617+0000 7f09fd0fb700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09f80ac4f0 0x7f09f80ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.618+0000 7f09f7fff700 1 -- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09e80038e0 msgr2=0x7f09f80aab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.618+0000 7f09f7fff700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09e80038e0 0x7f09f80aab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.618+0000 7f09f7fff700 1 -- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09f80ac4f0 msgr2=0x7f09f80ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.618+0000 7f09f7fff700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09f80ac4f0 0x7f09f80ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.618+0000 7f09f7fff700 1 -- 172.21.15.29:0/1340217877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f80ae940 con 0x7f09f8168310 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.618+0000 7f09fc8fa700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09e80038e0 0x7f09f80aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.618+0000 7f09f7fff700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09f8168310 0x7f09f80ae450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f09e4007720 tx=0x7f09e4001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.618+0000 7f09f4ff9700 1 -- 172.21.15.29:0/1340217877 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4011590 con 0x7f09f8168310 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.618+0000 7f09f4ff9700 1 -- 172.21.15.29:0/1340217877 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e40116f0 con 0x7f09f8168310 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.618+0000 7f09feb5e700 1 -- 172.21.15.29:0/1340217877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f8168790 con 0x7f09f8168310 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.618+0000 7f09f4ff9700 1 -- 172.21.15.29:0/1340217877 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4011a70 con 0x7f09f8168310 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.618+0000 7f09feb5e700 1 -- 172.21.15.29:0/1340217877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f80a8be0 con 0x7f09f8168310 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.619+0000 7f09feb5e700 1 -- 172.21.15.29:0/1340217877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f80475e0 con 0x7f09f8168310 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.619+0000 7f09f4ff9700 1 -- 172.21.15.29:0/1340217877 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f09e401d3b0 con 0x7f09f8168310 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.620+0000 7f09f4ff9700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f09dc05f150 0x7f09dc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.620+0000 7f09f4ff9700 1 -- 172.21.15.29:0/1340217877 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f09e4079a20 con 0x7f09f8168310 2024-01-24T13:58:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.620+0000 7f09fc8fa700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f09dc05f150 0x7f09dc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.621+0000 7f09fc8fa700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f09dc05f150 0x7f09dc061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f09ec001010 tx=0x7f09ec015000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:14.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.624+0000 7f09f4ff9700 1 -- 172.21.15.29:0/1340217877 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f09e404af20 con 0x7f09f8168310 2024-01-24T13:58:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.807+0000 7f09feb5e700 1 -- 172.21.15.29:0/1340217877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f09f8067a90 con 0x7f09f8168310 2024-01-24T13:58:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.808+0000 7f09f4ff9700 1 -- 172.21.15.29:0/1340217877 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f09e4019020 con 0x7f09f8168310 2024-01-24T13:58:14.808 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T13:58:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.810+0000 7f09feb5e700 1 -- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f09dc05f150 msgr2=0x7f09dc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.810+0000 7f09feb5e700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f09dc05f150 0x7f09dc061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f09ec001010 tx=0x7f09ec015000).stop 2024-01-24T13:58:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.810+0000 7f09feb5e700 1 -- 172.21.15.29:0/1340217877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09f8168310 msgr2=0x7f09f80ae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.810+0000 7f09feb5e700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09f8168310 0x7f09f80ae450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f09e4007720 tx=0x7f09e4001e50).stop 2024-01-24T13:58:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.810+0000 7f09feb5e700 1 -- 172.21.15.29:0/1340217877 shutdown_connections 2024-01-24T13:58:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.810+0000 7f09feb5e700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09f80ac4f0 0x7f09f80ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.810+0000 7f09feb5e700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09e80038e0 0x7f09f80aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.810+0000 7f09feb5e700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f09dc05f150 0x7f09dc061610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.810+0000 7f09feb5e700 1 --2- 172.21.15.29:0/1340217877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09f8168310 0x7f09f80ae450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.810+0000 7f09feb5e700 1 -- 172.21.15.29:0/1340217877 >> 172.21.15.29:0/1340217877 conn(0x7f09f80c8bd0 msgr2=0x7f09f8150050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.810+0000 7f09feb5e700 1 -- 172.21.15.29:0/1340217877 shutdown_connections 2024-01-24T13:58:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.810+0000 7f09feb5e700 1 -- 172.21.15.29:0/1340217877 wait complete. 2024-01-24T13:58:14.824 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:58:14.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.955+0000 7f698d2c1700 1 Processor -- start 2024-01-24T13:58:14.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.956+0000 7f698d2c1700 1 -- start start 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.956+0000 7f698d2c1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6988154370 0x7f69880c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.956+0000 7f698d2c1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6988154750 0x7f69880631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.956+0000 7f698d2c1700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69880aa6c0 con 0x7f69880aa0e0 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.956+0000 7f698d2c1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6988064240 con 0x7f6988154370 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.956+0000 7f698d2c1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69880cb5c0 con 0x7f6988154750 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f69867fc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f69880aa0e0 0x7f69880ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58800/0 (socket says 172.21.15.29:58800) 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f69867fc700 1 -- 172.21.15.29:0/651757605 learned_addr learned my addr 172.21.15.29:0/651757605 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f69877fe700 1 --2- 172.21.15.29:0/651757605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6988154750 0x7f69880631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f6986ffd700 1 --2- 172.21.15.29:0/651757605 >> v2:172.21.15.163:3300/0 conn(0x7f6988154370 0x7f69880c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f6985ffb700 1 -- 172.21.15.29:0/651757605 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944541213 0 0) 0x7f69880aa6c0 con 0x7f69880aa0e0 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f6985ffb700 1 -- 172.21.15.29:0/651757605 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69700033f0 con 0x7f69880aa0e0 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f69877fe700 1 -- 172.21.15.29:0/651757605 >> v2:172.21.15.163:3300/0 conn(0x7f6988154370 msgr2=0x7f69880c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:14.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f69877fe700 1 --2- 172.21.15.29:0/651757605 >> v2:172.21.15.163:3300/0 conn(0x7f6988154370 0x7f69880c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f69877fe700 1 -- 172.21.15.29:0/651757605 >> v1:172.21.15.29:6789/0 conn(0x7f69880aa0e0 legacy=0x7f69880ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f6985ffb700 1 -- 172.21.15.29:0/651757605 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 532188048 0 0) 0x7f69700033f0 con 0x7f69880aa0e0 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f69877fe700 1 -- 172.21.15.29:0/651757605 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69880cb8a0 con 0x7f6988154750 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f6986ffd700 1 --2- 172.21.15.29:0/651757605 >> v2:172.21.15.163:3300/0 conn(0x7f6988154370 0x7f69880c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.957+0000 7f69877fe700 1 --2- 172.21.15.29:0/651757605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6988154750 0x7f69880631d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f6980009e60 tx=0x7f6980009780).ready entity=mon.2 client_cookie=fee489b4208b0345 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.958+0000 7f6985ffb700 1 -- 172.21.15.29:0/651757605 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698000e070 con 0x7f6988154750 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.958+0000 7f6985ffb700 1 -- 172.21.15.29:0/651757605 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698000b7a0 con 0x7f6988154750 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.958+0000 7f6985ffb700 1 -- 172.21.15.29:0/651757605 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c001470 con 0x7f6988154750 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.958+0000 7f698d2c1700 1 -- 172.21.15.29:0/651757605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6988154750 msgr2=0x7f69880631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.958+0000 7f698d2c1700 1 --2- 172.21.15.29:0/651757605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6988154750 0x7f69880631d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f6980009e60 tx=0x7f6980009780).stop 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.958+0000 7f698d2c1700 1 -- 172.21.15.29:0/651757605 shutdown_connections 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.958+0000 7f698d2c1700 1 --2- 172.21.15.29:0/651757605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6988154750 0x7f69880631d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.958+0000 7f698d2c1700 1 --2- 172.21.15.29:0/651757605 >> v2:172.21.15.163:3300/0 conn(0x7f6988154370 0x7f69880c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.958+0000 7f698d2c1700 1 -- 172.21.15.29:0/651757605 >> 172.21.15.29:0/651757605 conn(0x7f698805ec80 msgr2=0x7f69880cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:14.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.958+0000 7f698d2c1700 1 -- 172.21.15.29:0/651757605 shutdown_connections 2024-01-24T13:58:14.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.958+0000 7f698d2c1700 1 -- 172.21.15.29:0/651757605 wait complete. 2024-01-24T13:58:14.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.960+0000 7f698d2c1700 1 Processor -- start 2024-01-24T13:58:14.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.960+0000 7f698d2c1700 1 -- start start 2024-01-24T13:58:14.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.961+0000 7f698d2c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880aa0e0 0x7f698814b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.961+0000 7f698d2c1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6988154370 0x7f69881559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.961+0000 7f698d2c1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6988154750 0x7f6988155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.961+0000 7f698d2c1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69881581e0 con 0x7f6988154370 2024-01-24T13:58:14.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.961+0000 7f698d2c1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69881564a0 con 0x7f69880aa0e0 2024-01-24T13:58:14.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.961+0000 7f698d2c1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6988156780 con 0x7f6988154750 2024-01-24T13:58:14.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.961+0000 7f69867fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6988154370 0x7f69881559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.961+0000 7f69867fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6988154370 0x7f69881559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39178/0 (socket says 172.21.15.29:39178) 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.961+0000 7f69867fc700 1 -- 172.21.15.29:0/2831758493 learned_addr learned my addr 172.21.15.29:0/2831758493 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.961+0000 7f6986ffd700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880aa0e0 0x7f698814b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.962+0000 7f69877fe700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6988154750 0x7f6988155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.962+0000 7f69867fc700 1 -- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6988154750 msgr2=0x7f6988155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.962+0000 7f69867fc700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6988154750 0x7f6988155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.962+0000 7f69867fc700 1 -- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880aa0e0 msgr2=0x7f698814b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.962+0000 7f69867fc700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880aa0e0 0x7f698814b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.962+0000 7f69867fc700 1 -- 172.21.15.29:0/2831758493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6978009250 con 0x7f6988154370 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.962+0000 7f69867fc700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6988154370 0x7f69881559c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f697c00c810 tx=0x7f697c001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.962+0000 7f696effd700 1 -- 172.21.15.29:0/2831758493 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c007170 con 0x7f6988154370 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.962+0000 7f696effd700 1 -- 172.21.15.29:0/2831758493 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c0072d0 con 0x7f6988154370 2024-01-24T13:58:14.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.962+0000 7f696effd700 1 -- 172.21.15.29:0/2831758493 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c0048c0 con 0x7f6988154370 2024-01-24T13:58:14.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.962+0000 7f698d2c1700 1 -- 172.21.15.29:0/2831758493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6980009a20 con 0x7f6988154370 2024-01-24T13:58:14.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.962+0000 7f698d2c1700 1 -- 172.21.15.29:0/2831758493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6988156d40 con 0x7f6988154370 2024-01-24T13:58:14.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.964+0000 7f698d2c1700 1 -- 172.21.15.29:0/2831758493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69880475e0 con 0x7f6988154370 2024-01-24T13:58:14.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.964+0000 7f696effd700 1 -- 172.21.15.29:0/2831758493 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f697c004c70 con 0x7f6988154370 2024-01-24T13:58:14.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.964+0000 7f696effd700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f697405f150 0x7f6974061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:14.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.964+0000 7f696effd700 1 -- 172.21.15.29:0/2831758493 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f697c079930 con 0x7f6988154370 2024-01-24T13:58:14.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.965+0000 7f6986ffd700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f697405f150 0x7f6974061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:14.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.965+0000 7f6986ffd700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f697405f150 0x7f6974061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f6978006fd0 tx=0x7f6978006f40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:14.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:14.969+0000 7f696effd700 1 -- 172.21.15.29:0/2831758493 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f697c04af50 con 0x7f6988154370 2024-01-24T13:58:15.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.192+0000 7f698d2c1700 1 -- 172.21.15.29:0/2831758493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6988158320 con 0x7f6988154370 2024-01-24T13:58:15.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.193+0000 7f696effd700 1 -- 172.21.15.29:0/2831758493 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f697c019de0 con 0x7f6988154370 2024-01-24T13:58:15.193 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:15.193 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2038321939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2038321939},{"type":"v1","addr":"172.21.15.163:6836","nonce":2038321939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3675672874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3675672874},{"type":"v1","addr":"172.21.15.163:6837","nonce":3675672874}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1532435957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532435957},{"type":"v1","addr":"172.21.15.29:6834","nonce":1532435957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1328130656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1328130656},{"type":"v1","addr":"172.21.15.29:6835","nonce":1328130656}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-01-24T13:58:15.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.195+0000 7f698d2c1700 1 -- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f697405f150 msgr2=0x7f6974061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.195+0000 7f698d2c1700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f697405f150 0x7f6974061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f6978006fd0 tx=0x7f6978006f40).stop 2024-01-24T13:58:15.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.195+0000 7f698d2c1700 1 -- 172.21.15.29:0/2831758493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6988154370 msgr2=0x7f69881559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.195+0000 7f698d2c1700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6988154370 0x7f69881559c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f697c00c810 tx=0x7f697c001b50).stop 2024-01-24T13:58:15.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.195+0000 7f698d2c1700 1 -- 172.21.15.29:0/2831758493 shutdown_connections 2024-01-24T13:58:15.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.195+0000 7f698d2c1700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6988154750 0x7f6988155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.195+0000 7f698d2c1700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880aa0e0 0x7f698814b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.195+0000 7f698d2c1700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f697405f150 0x7f6974061610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.195+0000 7f698d2c1700 1 --2- 172.21.15.29:0/2831758493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6988154370 0x7f69881559c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.195+0000 7f698d2c1700 1 -- 172.21.15.29:0/2831758493 >> 172.21.15.29:0/2831758493 conn(0x7f698805ec80 msgr2=0x7f69880bfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:15.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.195+0000 7f698d2c1700 1 -- 172.21.15.29:0/2831758493 shutdown_connections 2024-01-24T13:58:15.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.195+0000 7f698d2c1700 1 -- 172.21.15.29:0/2831758493 wait complete. 2024-01-24T13:58:15.197 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 3 2024-01-24T13:58:15.209 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:15.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.338+0000 7fe2932ff700 1 Processor -- start 2024-01-24T13:58:15.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.338+0000 7fe2932ff700 1 -- start start 2024-01-24T13:58:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.339+0000 7fe2932ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe28c066c00 0x7fe28c0bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.339+0000 7fe2932ff700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe28c1582b0 con 0x7fe28c0c4a10 2024-01-24T13:58:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.339+0000 7fe2932ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe28c157ca0 con 0x7fe28c0670a0 2024-01-24T13:58:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.339+0000 7fe2932ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe28c063c60 con 0x7fe28c066c00 2024-01-24T13:58:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.339+0000 7fe29109b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe28c0c4a10 0x7fe28c0664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58816/0 (socket says 172.21.15.29:58816) 2024-01-24T13:58:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.339+0000 7fe29109b700 1 -- 172.21.15.29:0/158418691 learned_addr learned my addr 172.21.15.29:0/158418691 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.339+0000 7fe29089a700 1 --2- 172.21.15.29:0/158418691 >> v2:172.21.15.163:3300/0 conn(0x7fe28c066c00 0x7fe28c0bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.340+0000 7fe27ffff700 1 -- 172.21.15.29:0/158418691 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76445521 0 0) 0x7fe28c1582b0 con 0x7fe28c0c4a10 2024-01-24T13:58:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.340+0000 7fe27ffff700 1 -- 172.21.15.29:0/158418691 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2780033f0 con 0x7fe28c0c4a10 2024-01-24T13:58:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.340+0000 7fe29089a700 1 -- 172.21.15.29:0/158418691 >> v1:172.21.15.163:6789/0 conn(0x7fe28c0670a0 legacy=0x7fe28c0bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.340+0000 7fe27ffff700 1 -- 172.21.15.29:0/158418691 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222844034 0 0) 0x7fe28c157ca0 con 0x7fe28c0670a0 2024-01-24T13:58:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.340+0000 7fe29089a700 1 -- 172.21.15.29:0/158418691 >> v1:172.21.15.29:6789/0 conn(0x7fe28c0c4a10 legacy=0x7fe28c0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.340+0000 7fe29089a700 1 -- 172.21.15.29:0/158418691 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c157f40 con 0x7fe28c066c00 2024-01-24T13:58:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.340+0000 7fe29089a700 1 --2- 172.21.15.29:0/158418691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c066c00 0x7fe28c0bf6d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fe288009760 tx=0x7fe288009060).ready entity=mon.1 client_cookie=8c144df6e1c649a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.340+0000 7fe27ffff700 1 -- 172.21.15.29:0/158418691 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800f510 con 0x7fe28c066c00 2024-01-24T13:58:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.340+0000 7fe27ffff700 1 -- 172.21.15.29:0/158418691 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28800f670 con 0x7fe28c066c00 2024-01-24T13:58:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.340+0000 7fe27ffff700 1 -- 172.21.15.29:0/158418691 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800f9f0 con 0x7fe28c066c00 2024-01-24T13:58:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.341+0000 7fe2932ff700 1 -- 172.21.15.29:0/158418691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c066c00 msgr2=0x7fe28c0bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.341+0000 7fe2932ff700 1 --2- 172.21.15.29:0/158418691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c066c00 0x7fe28c0bf6d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fe288009760 tx=0x7fe288009060).stop 2024-01-24T13:58:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.341+0000 7fe2932ff700 1 -- 172.21.15.29:0/158418691 shutdown_connections 2024-01-24T13:58:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.341+0000 7fe2932ff700 1 --2- 172.21.15.29:0/158418691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c066c00 0x7fe28c0bf6d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.341+0000 7fe2932ff700 1 -- 172.21.15.29:0/158418691 >> 172.21.15.29:0/158418691 conn(0x7fe28c0c8bb0 msgr2=0x7fe28c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.341+0000 7fe2932ff700 1 -- 172.21.15.29:0/158418691 shutdown_connections 2024-01-24T13:58:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.341+0000 7fe2932ff700 1 -- 172.21.15.29:0/158418691 wait complete. 2024-01-24T13:58:15.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.342+0000 7fe2932ff700 1 Processor -- start 2024-01-24T13:58:15.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.342+0000 7fe2932ff700 1 -- start start 2024-01-24T13:58:15.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.343+0000 7fe2932ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c066c00 0x7fe28c144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:15.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.343+0000 7fe2932ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe28c0670a0 0x7fe28c0a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.343+0000 7fe2932ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe28c0c4a10 0x7fe28c0a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.343+0000 7fe2932ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c0a5260 con 0x7fe28c0670a0 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.343+0000 7fe2932ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c0a9de0 con 0x7fe28c066c00 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.343+0000 7fe2932ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c0a94f0 con 0x7fe28c0c4a10 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.343+0000 7fe29089a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe28c0670a0 0x7fe28c0a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.343+0000 7fe29089a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe28c0670a0 0x7fe28c0a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39194/0 (socket says 172.21.15.29:39194) 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.343+0000 7fe29089a700 1 -- 172.21.15.29:0/2693637087 learned_addr learned my addr 172.21.15.29:0/2693637087 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.343+0000 7fe29109b700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c066c00 0x7fe28c144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.343+0000 7fe29189c700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe28c0c4a10 0x7fe28c0a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.343+0000 7fe29089a700 1 -- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe28c0c4a10 msgr2=0x7fe28c0a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.344+0000 7fe29089a700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe28c0c4a10 0x7fe28c0a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.344+0000 7fe29089a700 1 -- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c066c00 msgr2=0x7fe28c144590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.344+0000 7fe29089a700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c066c00 0x7fe28c144590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.344+0000 7fe29089a700 1 -- 172.21.15.29:0/2693637087 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880092a0 con 0x7fe28c0670a0 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.344+0000 7fe29089a700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe28c0670a0 0x7fe28c0a4880 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe288009e00 tx=0x7fe288017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.344+0000 7fe27d7fa700 1 -- 172.21.15.29:0/2693637087 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800f7e0 con 0x7fe28c0670a0 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.344+0000 7fe27d7fa700 1 -- 172.21.15.29:0/2693637087 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28800f940 con 0x7fe28c0670a0 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.344+0000 7fe2932ff700 1 -- 172.21.15.29:0/2693637087 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28c0a9790 con 0x7fe28c0670a0 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.344+0000 7fe27d7fa700 1 -- 172.21.15.29:0/2693637087 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28801b660 con 0x7fe28c0670a0 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.344+0000 7fe2932ff700 1 -- 172.21.15.29:0/2693637087 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28c157100 con 0x7fe28c0670a0 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.345+0000 7fe2932ff700 1 -- 172.21.15.29:0/2693637087 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28c0a9f90 con 0x7fe28c0670a0 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.345+0000 7fe27d7fa700 1 -- 172.21.15.29:0/2693637087 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe28801b9d0 con 0x7fe28c0670a0 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.346+0000 7fe27d7fa700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe27005f150 0x7fe270061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.346+0000 7fe27d7fa700 1 -- 172.21.15.29:0/2693637087 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fe28807b860 con 0x7fe28c0670a0 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.346+0000 7fe29109b700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe27005f150 0x7fe270061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:15.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.346+0000 7fe29109b700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe27005f150 0x7fe270061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fe280005750 tx=0x7fe280009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:15.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.350+0000 7fe27d7fa700 1 -- 172.21.15.29:0/2693637087 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe28c0a9f90 con 0x7fe28c0670a0 2024-01-24T13:58:15.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.533+0000 7fe2932ff700 1 -- 172.21.15.29:0/2693637087 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe28c0a9f50 con 0x7fe28c0670a0 2024-01-24T13:58:15.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.534+0000 7fe27d7fa700 1 -- 172.21.15.29:0/2693637087 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13709 (secure 0 0 0) 0x7fe28804ce80 con 0x7fe28c0670a0 2024-01-24T13:58:15.535 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:15.535 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":18,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:14.756128+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.536+0000 7fe2932ff700 1 -- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe27005f150 msgr2=0x7fe270061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.536+0000 7fe2932ff700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe27005f150 0x7fe270061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fe280005750 tx=0x7fe280009040).stop 2024-01-24T13:58:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.536+0000 7fe2932ff700 1 -- 172.21.15.29:0/2693637087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe28c0670a0 msgr2=0x7fe28c0a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.536+0000 7fe2932ff700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe28c0670a0 0x7fe28c0a4880 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe288009e00 tx=0x7fe288017000).stop 2024-01-24T13:58:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.537+0000 7fe2932ff700 1 -- 172.21.15.29:0/2693637087 shutdown_connections 2024-01-24T13:58:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.537+0000 7fe2932ff700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe28c0c4a10 0x7fe28c0a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.537+0000 7fe2932ff700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c066c00 0x7fe28c144590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.537+0000 7fe2932ff700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe27005f150 0x7fe270061610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.537+0000 7fe2932ff700 1 --2- 172.21.15.29:0/2693637087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe28c0670a0 0x7fe28c0a4880 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.537+0000 7fe2932ff700 1 -- 172.21.15.29:0/2693637087 >> 172.21.15.29:0/2693637087 conn(0x7fe28c0c8bb0 msgr2=0x7fe28c0664f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.537+0000 7fe2932ff700 1 -- 172.21.15.29:0/2693637087 shutdown_connections 2024-01-24T13:58:15.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.537+0000 7fe2932ff700 1 -- 172.21.15.29:0/2693637087 wait complete. 2024-01-24T13:58:15.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T13:58:15.551 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T13:58:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.684+0000 7f429a490700 1 Processor -- start 2024-01-24T13:58:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.685+0000 7f429a490700 1 -- start start 2024-01-24T13:58:15.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.685+0000 7f429a490700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4294153f40 0x7f42940a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:15.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.685+0000 7f429a490700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4294062ec0 0x7f4294065440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:15.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.685+0000 7f429a490700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429406a360 con 0x7f42940c8290 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.685+0000 7f429a490700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42940bcef0 con 0x7f4294062ec0 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.685+0000 7f429a490700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4294069de0 con 0x7f4294153f40 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4298a2d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4294062ec0 0x7f4294065440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4298a2d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4294062ec0 0x7f4294065440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39202/0 (socket says 172.21.15.29:39202) 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4298a2d700 1 -- 172.21.15.29:0/2924293450 learned_addr learned my addr 172.21.15.29:0/2924293450 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f42937fe700 1 --1- 172.21.15.29:0/2924293450 >> v1:172.21.15.29:6789/0 conn(0x7f42940c8290 0x7f42940627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58832/0 (socket says 172.21.15.29:58832) 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4293fff700 1 --2- 172.21.15.29:0/2924293450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4294153f40 0x7f42940a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4298a2d700 1 -- 172.21.15.29:0/2924293450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4294153f40 msgr2=0x7f42940a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4298a2d700 1 --2- 172.21.15.29:0/2924293450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4294153f40 0x7f42940a6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4298a2d700 1 -- 172.21.15.29:0/2924293450 >> v1:172.21.15.29:6789/0 conn(0x7f42940c8290 legacy=0x7f42940627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4298a2d700 1 -- 172.21.15.29:0/2924293450 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429406a080 con 0x7f4294062ec0 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4292ffd700 1 -- 172.21.15.29:0/2924293450 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539467492 0 0) 0x7f429406a360 con 0x7f42940c8290 2024-01-24T13:58:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4298a2d700 1 --2- 172.21.15.29:0/2924293450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4294062ec0 0x7f4294065440 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f428c009f20 tx=0x7f428c009840).ready entity=mon.0 client_cookie=4202cb2a02c68273 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4292ffd700 1 -- 172.21.15.29:0/2924293450 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c00e070 con 0x7f4294062ec0 2024-01-24T13:58:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4292ffd700 1 -- 172.21.15.29:0/2924293450 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428c00b9d0 con 0x7f4294062ec0 2024-01-24T13:58:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.686+0000 7f4292ffd700 1 -- 172.21.15.29:0/2924293450 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c00bd50 con 0x7f4294062ec0 2024-01-24T13:58:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.687+0000 7f429a490700 1 -- 172.21.15.29:0/2924293450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4294062ec0 msgr2=0x7f4294065440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.687+0000 7f429a490700 1 --2- 172.21.15.29:0/2924293450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4294062ec0 0x7f4294065440 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f428c009f20 tx=0x7f428c009840).stop 2024-01-24T13:58:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.687+0000 7f429a490700 1 -- 172.21.15.29:0/2924293450 shutdown_connections 2024-01-24T13:58:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.687+0000 7f429a490700 1 --2- 172.21.15.29:0/2924293450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4294062ec0 0x7f4294065440 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.687+0000 7f429a490700 1 --2- 172.21.15.29:0/2924293450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4294153f40 0x7f42940a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.687+0000 7f429a490700 1 -- 172.21.15.29:0/2924293450 >> 172.21.15.29:0/2924293450 conn(0x7f4294064390 msgr2=0x7f42940c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.687+0000 7f429a490700 1 -- 172.21.15.29:0/2924293450 shutdown_connections 2024-01-24T13:58:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.687+0000 7f429a490700 1 -- 172.21.15.29:0/2924293450 wait complete. 2024-01-24T13:58:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.688+0000 7f429a490700 1 Processor -- start 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.688+0000 7f429a490700 1 -- start start 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.689+0000 7f429a490700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4294062ec0 0x7f42940bec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.689+0000 7f429a490700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42940c8290 0x7f42940a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.689+0000 7f429a490700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4294153f40 0x7f42941420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.689+0000 7f429a490700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42940a7650 con 0x7f4294062ec0 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.689+0000 7f429a490700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42940a84b0 con 0x7f4294153f40 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.689+0000 7f429a490700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42940a7b80 con 0x7f42940c8290 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.689+0000 7f4293fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4294062ec0 0x7f42940bec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.689+0000 7f4293fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4294062ec0 0x7f42940bec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39206/0 (socket says 172.21.15.29:39206) 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.689+0000 7f4293fff700 1 -- 172.21.15.29:0/422993823 learned_addr learned my addr 172.21.15.29:0/422993823 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.690+0000 7f4293fff700 1 -- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42940c8290 msgr2=0x7f42940a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.690+0000 7f4293fff700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42940c8290 0x7f42940a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T13:58:15.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.690+0000 7f4298a2d700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4294153f40 0x7f42941420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.690+0000 7f4293fff700 1 -- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4294153f40 msgr2=0x7f42941420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.690+0000 7f4293fff700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4294153f40 0x7f42941420d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.690+0000 7f4293fff700 1 -- 172.21.15.29:0/422993823 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c009ae0 con 0x7f4294062ec0 2024-01-24T13:58:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.690+0000 7f4293fff700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4294062ec0 0x7f42940bec00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f428400e900 tx=0x7f4284009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.690+0000 7f427bfff700 1 -- 172.21.15.29:0/422993823 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428400c640 con 0x7f4294062ec0 2024-01-24T13:58:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.690+0000 7f427bfff700 1 -- 172.21.15.29:0/422993823 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428400c7a0 con 0x7f4294062ec0 2024-01-24T13:58:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.690+0000 7f429a490700 1 -- 172.21.15.29:0/422993823 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42940a7e60 con 0x7f4294062ec0 2024-01-24T13:58:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.690+0000 7f427bfff700 1 -- 172.21.15.29:0/422993823 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428400cb20 con 0x7f4294062ec0 2024-01-24T13:58:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.690+0000 7f429a490700 1 -- 172.21.15.29:0/422993823 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4294142ae0 con 0x7f4294062ec0 2024-01-24T13:58:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.691+0000 7f429a490700 1 -- 172.21.15.29:0/422993823 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42940a8790 con 0x7f4294062ec0 2024-01-24T13:58:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.691+0000 7f427bfff700 1 -- 172.21.15.29:0/422993823 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f428401b400 con 0x7f4294062ec0 2024-01-24T13:58:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.692+0000 7f427bfff700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f428005f150 0x7f4280061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.692+0000 7f427bfff700 1 -- 172.21.15.29:0/422993823 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f428407a2b0 con 0x7f4294062ec0 2024-01-24T13:58:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.692+0000 7f42937fe700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f428005f150 0x7f4280061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.692+0000 7f42937fe700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f428005f150 0x7f4280061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f42880011e0 tx=0x7f4288007830).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:15.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.696+0000 7f427bfff700 1 -- 172.21.15.29:0/422993823 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f428404b870 con 0x7f4294062ec0 2024-01-24T13:58:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.881+0000 7f429a490700 1 -- 172.21.15.29:0/422993823 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f42940475e0 con 0x7f4294062ec0 2024-01-24T13:58:15.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.881+0000 7f427bfff700 1 -- 172.21.15.29:0/422993823 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4284016070 con 0x7f4294062ec0 2024-01-24T13:58:15.882 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T13:58:15.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.884+0000 7f429a490700 1 -- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f428005f150 msgr2=0x7f4280061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.884+0000 7f429a490700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f428005f150 0x7f4280061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f42880011e0 tx=0x7f4288007830).stop 2024-01-24T13:58:15.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.884+0000 7f429a490700 1 -- 172.21.15.29:0/422993823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4294062ec0 msgr2=0x7f42940bec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:15.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.884+0000 7f429a490700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4294062ec0 0x7f42940bec00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f428400e900 tx=0x7f4284009060).stop 2024-01-24T13:58:15.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.884+0000 7f429a490700 1 -- 172.21.15.29:0/422993823 shutdown_connections 2024-01-24T13:58:15.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.884+0000 7f429a490700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4294153f40 0x7f42941420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.884+0000 7f429a490700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f428005f150 0x7f4280061610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.884+0000 7f429a490700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4294062ec0 0x7f42940bec00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:15.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.884+0000 7f429a490700 1 --2- 172.21.15.29:0/422993823 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42940c8290 0x7f42940a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T13:58:15.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.884+0000 7f429a490700 1 -- 172.21.15.29:0/422993823 >> 172.21.15.29:0/422993823 conn(0x7f4294064390 msgr2=0x7f42940c5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:15.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.884+0000 7f429a490700 1 -- 172.21.15.29:0/422993823 shutdown_connections 2024-01-24T13:58:15.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:15.884+0000 7f429a490700 1 -- 172.21.15.29:0/422993823 wait complete. 2024-01-24T13:58:15.898 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T13:58:16.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.032+0000 7f4abd300700 1 Processor -- start 2024-01-24T13:58:16.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.032+0000 7f4abd300700 1 -- start start 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.033+0000 7f4abd300700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ab8153d20 0x7f4ab8154100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.033+0000 7f4abd300700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8158a40 0x7f4ab80c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.033+0000 7f4abd300700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab80c21e0 con 0x7f4ab80bc370 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.033+0000 7f4abd300700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab80bcc60 con 0x7f4ab8153d20 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.033+0000 7f4abd300700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab81545f0 con 0x7f4ab8158a40 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab67fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8158a40 0x7f4ab80c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ab8153d20 0x7f4ab8154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab77fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4ab80bc370 0x7f4ab80c33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:47834/0 (socket says 172.21.15.29:47834) 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab77fe700 1 -- 172.21.15.29:0/174468352 learned_addr learned my addr 172.21.15.29:0/174468352 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ab8153d20 0x7f4ab8154100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:38796/0 (socket says 172.21.15.29:38796) 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab67fc700 1 --2- 172.21.15.29:0/174468352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8158a40 0x7f4ab80c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:44244/0 (socket says 172.21.15.29:44244) 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab6ffd700 1 -- 172.21.15.29:0/174468352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8158a40 msgr2=0x7f4ab80c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:16.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab6ffd700 1 --2- 172.21.15.29:0/174468352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8158a40 0x7f4ab80c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab6ffd700 1 -- 172.21.15.29:0/174468352 >> v1:172.21.15.163:6789/0 conn(0x7f4ab80bc370 legacy=0x7f4ab80c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab6ffd700 1 -- 172.21.15.29:0/174468352 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab80c18b0 con 0x7f4ab8153d20 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab5ffb700 1 -- 172.21.15.29:0/174468352 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2718809913 0 0) 0x7f4ab80c21e0 con 0x7f4ab80bc370 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab67fc700 1 --2- 172.21.15.29:0/174468352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8158a40 0x7f4ab80c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab6ffd700 1 --2- 172.21.15.29:0/174468352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab8153d20 0x7f4ab8154100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f4aa0009760 tx=0x7f4aa0009060).ready entity=mon.1 client_cookie=327e39f73026dd7d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab5ffb700 1 -- 172.21.15.29:0/174468352 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000f510 con 0x7f4ab8153d20 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.034+0000 7f4ab5ffb700 1 -- 172.21.15.29:0/174468352 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa000f670 con 0x7f4ab8153d20 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.035+0000 7f4ab5ffb700 1 -- 172.21.15.29:0/174468352 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000f9f0 con 0x7f4ab8153d20 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.035+0000 7f4abd300700 1 -- 172.21.15.29:0/174468352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab8153d20 msgr2=0x7f4ab8154100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.035+0000 7f4abd300700 1 --2- 172.21.15.29:0/174468352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab8153d20 0x7f4ab8154100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f4aa0009760 tx=0x7f4aa0009060).stop 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.035+0000 7f4abd300700 1 -- 172.21.15.29:0/174468352 shutdown_connections 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.035+0000 7f4abd300700 1 --2- 172.21.15.29:0/174468352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab8153d20 0x7f4ab8154100 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.035+0000 7f4abd300700 1 --2- 172.21.15.29:0/174468352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8158a40 0x7f4ab80c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.035+0000 7f4abd300700 1 -- 172.21.15.29:0/174468352 >> 172.21.15.29:0/174468352 conn(0x7f4ab80627b0 msgr2=0x7f4ab8062bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:16.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.035+0000 7f4abd300700 1 -- 172.21.15.29:0/174468352 shutdown_connections 2024-01-24T13:58:16.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.035+0000 7f4abd300700 1 -- 172.21.15.29:0/174468352 wait complete. 2024-01-24T13:58:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.037+0000 7f4abd300700 1 Processor -- start 2024-01-24T13:58:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.037+0000 7f4abd300700 1 -- start start 2024-01-24T13:58:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.038+0000 7f4abd300700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ab80bc370 0x7f4ab8165760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.038+0000 7f4abd300700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8153d20 0x7f4ab8064a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.038+0000 7f4abd300700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab8158a40 0x7f4ab80c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.038+0000 7f4abd300700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab8167b60 con 0x7f4ab80bc370 2024-01-24T13:58:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.038+0000 7f4abd300700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab81689c0 con 0x7f4ab8158a40 2024-01-24T13:58:16.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.038+0000 7f4abd300700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab8167db0 con 0x7f4ab8153d20 2024-01-24T13:58:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.038+0000 7f4ab6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ab80bc370 0x7f4ab8165760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.038+0000 7f4ab6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ab80bc370 0x7f4ab8165760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39220/0 (socket says 172.21.15.29:39220) 2024-01-24T13:58:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.038+0000 7f4ab6ffd700 1 -- 172.21.15.29:0/1080028114 learned_addr learned my addr 172.21.15.29:0/1080028114 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.038+0000 7f4ab67fc700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8153d20 0x7f4ab8064a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.038+0000 7f4ab77fe700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab8158a40 0x7f4ab80c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.039+0000 7f4ab6ffd700 1 -- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8153d20 msgr2=0x7f4ab8064a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.039+0000 7f4ab6ffd700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8153d20 0x7f4ab8064a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.039+0000 7f4ab6ffd700 1 -- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab8158a40 msgr2=0x7f4ab80c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.039+0000 7f4ab6ffd700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab8158a40 0x7f4ab80c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.039+0000 7f4ab6ffd700 1 -- 172.21.15.29:0/1080028114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa80092a0 con 0x7f4ab80bc370 2024-01-24T13:58:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.039+0000 7f4ab67fc700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8153d20 0x7f4ab8064a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:16.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.039+0000 7f4ab6ffd700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ab80bc370 0x7f4ab8165760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f4ab80663d0 tx=0x7f4aa0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:16.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.039+0000 7f4aaeffd700 1 -- 172.21.15.29:0/1080028114 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000f7e0 con 0x7f4ab80bc370 2024-01-24T13:58:16.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.039+0000 7f4aaeffd700 1 -- 172.21.15.29:0/1080028114 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa000f940 con 0x7f4ab80bc370 2024-01-24T13:58:16.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.039+0000 7f4aaeffd700 1 -- 172.21.15.29:0/1080028114 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa001b660 con 0x7f4ab80bc370 2024-01-24T13:58:16.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.039+0000 7f4abd300700 1 -- 172.21.15.29:0/1080028114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa00092a0 con 0x7f4ab80bc370 2024-01-24T13:58:16.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.039+0000 7f4abd300700 1 -- 172.21.15.29:0/1080028114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab8168370 con 0x7f4ab80bc370 2024-01-24T13:58:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.040+0000 7f4abd300700 1 -- 172.21.15.29:0/1080028114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab80475e0 con 0x7f4ab80bc370 2024-01-24T13:58:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.041+0000 7f4aaeffd700 1 -- 172.21.15.29:0/1080028114 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4aa001ba50 con 0x7f4ab80bc370 2024-01-24T13:58:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.041+0000 7f4aaeffd700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4a9805f150 0x7f4a98061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.041+0000 7f4aaeffd700 1 -- 172.21.15.29:0/1080028114 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f4aa007b810 con 0x7f4ab80bc370 2024-01-24T13:58:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.041+0000 7f4ab67fc700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4a9805f150 0x7f4a98061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.042+0000 7f4ab67fc700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4a9805f150 0x7f4a98061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4ab8158870 tx=0x7f4aa8009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:16.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.046+0000 7f4aaeffd700 1 -- 172.21.15.29:0/1080028114 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4aa004ce30 con 0x7f4ab80bc370 2024-01-24T13:58:16.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.224+0000 7f4abd300700 1 -- 172.21.15.29:0/1080028114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4ab80c1b90 con 0x7f4ab80bc370 2024-01-24T13:58:16.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.225+0000 7f4aaeffd700 1 -- 172.21.15.29:0/1080028114 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4aa0028070 con 0x7f4ab80bc370 2024-01-24T13:58:16.225 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T13:58:16.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.227+0000 7f4abd300700 1 -- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4a9805f150 msgr2=0x7f4a98061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.227+0000 7f4abd300700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4a9805f150 0x7f4a98061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4ab8158870 tx=0x7f4aa8009060).stop 2024-01-24T13:58:16.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.227+0000 7f4abd300700 1 -- 172.21.15.29:0/1080028114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ab80bc370 msgr2=0x7f4ab8165760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.227+0000 7f4abd300700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ab80bc370 0x7f4ab8165760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f4ab80663d0 tx=0x7f4aa0017000).stop 2024-01-24T13:58:16.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.227+0000 7f4abd300700 1 -- 172.21.15.29:0/1080028114 shutdown_connections 2024-01-24T13:58:16.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.227+0000 7f4abd300700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab8158a40 0x7f4ab80c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.227+0000 7f4abd300700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4a9805f150 0x7f4a98061610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.227+0000 7f4abd300700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ab80bc370 0x7f4ab8165760 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.227+0000 7f4abd300700 1 --2- 172.21.15.29:0/1080028114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab8153d20 0x7f4ab8064a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.227+0000 7f4abd300700 1 -- 172.21.15.29:0/1080028114 >> 172.21.15.29:0/1080028114 conn(0x7f4ab80627b0 msgr2=0x7f4ab806a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:16.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.227+0000 7f4abd300700 1 -- 172.21.15.29:0/1080028114 shutdown_connections 2024-01-24T13:58:16.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.227+0000 7f4abd300700 1 -- 172.21.15.29:0/1080028114 wait complete. 2024-01-24T13:58:16.241 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-24T13:58:16.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.370+0000 7f9a0811e700 1 Processor -- start 2024-01-24T13:58:16.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.370+0000 7f9a0811e700 1 -- start start 2024-01-24T13:58:16.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.371+0000 7f9a0811e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a0014c890 0x7f9a000627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.371+0000 7f9a0811e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a00063e30 0x7f9a000689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.371+0000 7f9a0811e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a000639a0 con 0x7f9a0014c4b0 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.371+0000 7f9a0811e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a00062d90 con 0x7f9a0014c890 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.371+0000 7f9a0811e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a00063070 con 0x7f9a00063e30 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a05eba700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9a0014c4b0 0x7f9a00153060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58834/0 (socket says 172.21.15.29:58834) 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a05eba700 1 -- 172.21.15.29:0/4150862601 learned_addr learned my addr 172.21.15.29:0/4150862601 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a066bb700 1 --2- 172.21.15.29:0/4150862601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a00063e30 0x7f9a000689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a056b9700 1 --2- 172.21.15.29:0/4150862601 >> v2:172.21.15.163:3300/0 conn(0x7f9a0014c890 0x7f9a000627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a04eb8700 1 -- 172.21.15.29:0/4150862601 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039438764 0 0) 0x7f9a000639a0 con 0x7f9a0014c4b0 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a04eb8700 1 -- 172.21.15.29:0/4150862601 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99ec0033f0 con 0x7f9a0014c4b0 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a066bb700 1 -- 172.21.15.29:0/4150862601 >> v2:172.21.15.163:3300/0 conn(0x7f9a0014c890 msgr2=0x7f9a000627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a066bb700 1 --2- 172.21.15.29:0/4150862601 >> v2:172.21.15.163:3300/0 conn(0x7f9a0014c890 0x7f9a000627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a066bb700 1 -- 172.21.15.29:0/4150862601 >> v1:172.21.15.29:6789/0 conn(0x7f9a0014c4b0 legacy=0x7f9a00153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a066bb700 1 -- 172.21.15.29:0/4150862601 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00063350 con 0x7f9a00063e30 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a056b9700 1 --2- 172.21.15.29:0/4150862601 >> v2:172.21.15.163:3300/0 conn(0x7f9a0014c890 0x7f9a000627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:16.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a066bb700 1 --2- 172.21.15.29:0/4150862601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a00063e30 0x7f9a000689a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f99f8009e60 tx=0x7f99f8009780).ready entity=mon.2 client_cookie=3a6c1d1236219e13 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:16.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a04eb8700 1 -- 172.21.15.29:0/4150862601 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800e070 con 0x7f9a00063e30 2024-01-24T13:58:16.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a04eb8700 1 -- 172.21.15.29:0/4150862601 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f800bab0 con 0x7f9a00063e30 2024-01-24T13:58:16.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.372+0000 7f9a04eb8700 1 -- 172.21.15.29:0/4150862601 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800be30 con 0x7f9a00063e30 2024-01-24T13:58:16.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.373+0000 7f9a0811e700 1 -- 172.21.15.29:0/4150862601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a00063e30 msgr2=0x7f9a000689a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.373+0000 7f9a0811e700 1 --2- 172.21.15.29:0/4150862601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a00063e30 0x7f9a000689a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f99f8009e60 tx=0x7f99f8009780).stop 2024-01-24T13:58:16.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.373+0000 7f9a0811e700 1 -- 172.21.15.29:0/4150862601 shutdown_connections 2024-01-24T13:58:16.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.373+0000 7f9a0811e700 1 --2- 172.21.15.29:0/4150862601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a00063e30 0x7f9a000689a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.373+0000 7f9a0811e700 1 --2- 172.21.15.29:0/4150862601 >> v2:172.21.15.163:3300/0 conn(0x7f9a0014c890 0x7f9a000627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.373+0000 7f9a0811e700 1 -- 172.21.15.29:0/4150862601 >> 172.21.15.29:0/4150862601 conn(0x7f9a00064aa0 msgr2=0x7f9a00064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:16.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.373+0000 7f9a0811e700 1 -- 172.21.15.29:0/4150862601 shutdown_connections 2024-01-24T13:58:16.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.373+0000 7f9a0811e700 1 -- 172.21.15.29:0/4150862601 wait complete. 2024-01-24T13:58:16.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.374+0000 7f9a0811e700 1 Processor -- start 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.374+0000 7f9a0811e700 1 -- start start 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.375+0000 7f9a0811e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a00063e30 0x7f9a0014b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.375+0000 7f9a0811e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0014c4b0 0x7f9a000ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.375+0000 7f9a0811e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a0014c890 0x7f9a000abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.375+0000 7f9a0811e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000c1370 con 0x7f9a0014c4b0 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.375+0000 7f9a0811e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000ac240 con 0x7f9a00063e30 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.375+0000 7f9a0811e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000ac520 con 0x7f9a0014c890 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.375+0000 7f9a056b9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0014c4b0 0x7f9a000ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.375+0000 7f9a056b9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0014c4b0 0x7f9a000ab6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39228/0 (socket says 172.21.15.29:39228) 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.375+0000 7f9a066bb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a0014c890 0x7f9a000abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.375+0000 7f9a056b9700 1 -- 172.21.15.29:0/2606958249 learned_addr learned my addr 172.21.15.29:0/2606958249 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.375+0000 7f9a05eba700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a00063e30 0x7f9a0014b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.376+0000 7f9a056b9700 1 -- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a0014c890 msgr2=0x7f9a000abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.376+0000 7f9a056b9700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a0014c890 0x7f9a000abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.376+0000 7f9a056b9700 1 -- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a00063e30 msgr2=0x7f9a0014b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.376+0000 7f9a056b9700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a00063e30 0x7f9a0014b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.376+0000 7f9a056b9700 1 -- 172.21.15.29:0/2606958249 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc0092a0 con 0x7f9a0014c4b0 2024-01-24T13:58:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.376+0000 7f9a066bb700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a0014c890 0x7f9a000abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.376+0000 7f9a056b9700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0014c4b0 0x7f9a000ab6d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f99fc00ea20 tx=0x7f99fc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.376+0000 7f99f1ffb700 1 -- 172.21.15.29:0/2606958249 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00ee60 con 0x7f9a0014c4b0 2024-01-24T13:58:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.376+0000 7f99f1ffb700 1 -- 172.21.15.29:0/2606958249 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc00c850 con 0x7f9a0014c4b0 2024-01-24T13:58:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.376+0000 7f9a0811e700 1 -- 172.21.15.29:0/2606958249 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f8009a20 con 0x7f9a0014c4b0 2024-01-24T13:58:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.376+0000 7f99f1ffb700 1 -- 172.21.15.29:0/2606958249 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00cbd0 con 0x7f9a0014c4b0 2024-01-24T13:58:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.376+0000 7f9a0811e700 1 -- 172.21.15.29:0/2606958249 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a000acae0 con 0x7f9a0014c4b0 2024-01-24T13:58:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.377+0000 7f9a0811e700 1 -- 172.21.15.29:0/2606958249 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a000475e0 con 0x7f9a0014c4b0 2024-01-24T13:58:16.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.377+0000 7f99f1ffb700 1 -- 172.21.15.29:0/2606958249 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f99fc01b400 con 0x7f9a0014c4b0 2024-01-24T13:58:16.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.378+0000 7f99f1ffb700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f99e405f150 0x7f99e4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.378+0000 7f99f1ffb700 1 -- 172.21.15.29:0/2606958249 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f99fc07a380 con 0x7f9a0014c4b0 2024-01-24T13:58:16.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.378+0000 7f9a05eba700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f99e405f150 0x7f99e4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.378+0000 7f9a05eba700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f99e405f150 0x7f99e4061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f99f4007fd0 tx=0x7f99f4009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:16.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.382+0000 7f99f1ffb700 1 -- 172.21.15.29:0/2606958249 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f99fc04b9f0 con 0x7f9a0014c4b0 2024-01-24T13:58:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.566+0000 7f9a0811e700 1 -- 172.21.15.29:0/2606958249 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9a00064440 con 0x7f9a0014c4b0 2024-01-24T13:58:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.567+0000 7f99f1ffb700 1 -- 172.21.15.29:0/2606958249 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+143 (secure 0 0 0) 0x7f99fc04f010 con 0x7f9a0014c4b0 2024-01-24T13:58:16.567 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:16.567 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T13:58:16.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:58:16.568 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-01-24T13:58:16.568 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "device_health_metrics" 2024-01-24T13:58:16.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:58:16.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:58:16.568 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 2, 2024-01-24T13:58:16.568 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "rbd" 2024-01-24T13:58:16.568 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:58:16.568 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T13:58:16.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.569+0000 7f9a0811e700 1 -- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f99e405f150 msgr2=0x7f99e4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.569+0000 7f9a0811e700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f99e405f150 0x7f99e4061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f99f4007fd0 tx=0x7f99f4009040).stop 2024-01-24T13:58:16.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.569+0000 7f9a0811e700 1 -- 172.21.15.29:0/2606958249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0014c4b0 msgr2=0x7f9a000ab6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.569+0000 7f9a0811e700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0014c4b0 0x7f9a000ab6d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f99fc00ea20 tx=0x7f99fc009060).stop 2024-01-24T13:58:16.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.569+0000 7f9a0811e700 1 -- 172.21.15.29:0/2606958249 shutdown_connections 2024-01-24T13:58:16.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.569+0000 7f9a0811e700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a0014c890 0x7f9a000abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.569+0000 7f9a0811e700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a00063e30 0x7f9a0014b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.569+0000 7f9a0811e700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f99e405f150 0x7f99e4061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.569+0000 7f9a0811e700 1 --2- 172.21.15.29:0/2606958249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0014c4b0 0x7f9a000ab6d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.569+0000 7f9a0811e700 1 -- 172.21.15.29:0/2606958249 >> 172.21.15.29:0/2606958249 conn(0x7f9a00064aa0 msgr2=0x7f9a000bc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:16.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.569+0000 7f9a0811e700 1 -- 172.21.15.29:0/2606958249 shutdown_connections 2024-01-24T13:58:16.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.569+0000 7f9a0811e700 1 -- 172.21.15.29:0/2606958249 wait complete. 2024-01-24T13:58:16.582 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-24T13:58:16.583 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2024-01-24T13:58:16.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.718+0000 7f9abc4c7700 1 Processor -- start 2024-01-24T13:58:16.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.718+0000 7f9abc4c7700 1 -- start start 2024-01-24T13:58:16.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9abc4c7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9ab40c3660 0x7f9ab414b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9abc4c7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9ab40c4b00 0x7f9ab4154a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9abc4c7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab41557e0 con 0x7f9ab414b0a0 2024-01-24T13:58:16.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9abc4c7700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab40c28a0 con 0x7f9ab40c4b00 2024-01-24T13:58:16.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9abc4c7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab40c2b80 con 0x7f9ab40c3660 2024-01-24T13:58:16.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9abaa64700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9ab40c4b00 0x7f9ab4154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9abaa64700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9ab40c4b00 0x7f9ab4154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39244/0 (socket says 172.21.15.29:39244) 2024-01-24T13:58:16.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9abaa64700 1 -- 172.21.15.29:0/4247070038 learned_addr learned my addr 172.21.15.29:0/4247070038 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:16.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9ab9a62700 1 --2- 172.21.15.29:0/4247070038 >> v2:172.21.15.163:3300/0 conn(0x7f9ab40c3660 0x7f9ab414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9abaa64700 1 -- 172.21.15.29:0/4247070038 >> v2:172.21.15.163:3300/0 conn(0x7f9ab40c3660 msgr2=0x7f9ab414b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:16.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9abaa64700 1 --2- 172.21.15.29:0/4247070038 >> v2:172.21.15.163:3300/0 conn(0x7f9ab40c3660 0x7f9ab414b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9abaa64700 1 -- 172.21.15.29:0/4247070038 >> v1:172.21.15.163:6789/0 conn(0x7f9ab414b0a0 legacy=0x7f9ab40c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9abaa64700 1 -- 172.21.15.29:0/4247070038 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab40c2e60 con 0x7f9ab40c4b00 2024-01-24T13:58:16.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.719+0000 7f9ab9a62700 1 --2- 172.21.15.29:0/4247070038 >> v2:172.21.15.163:3300/0 conn(0x7f9ab40c3660 0x7f9ab414b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:16.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.720+0000 7f9abaa64700 1 --2- 172.21.15.29:0/4247070038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ab40c4b00 0x7f9ab4154a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9ab406a370 tx=0x7f9aac00ba50).ready entity=mon.0 client_cookie=f9ecadcb5b5d31d2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:16.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.720+0000 7f9ab9261700 1 -- 172.21.15.29:0/4247070038 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac010070 con 0x7f9ab40c4b00 2024-01-24T13:58:16.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.720+0000 7f9ab9261700 1 -- 172.21.15.29:0/4247070038 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac00de60 con 0x7f9ab40c4b00 2024-01-24T13:58:16.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.720+0000 7f9ab9261700 1 -- 172.21.15.29:0/4247070038 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac00b2e0 con 0x7f9ab40c4b00 2024-01-24T13:58:16.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.720+0000 7f9abc4c7700 1 -- 172.21.15.29:0/4247070038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ab40c4b00 msgr2=0x7f9ab4154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.720+0000 7f9abc4c7700 1 --2- 172.21.15.29:0/4247070038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ab40c4b00 0x7f9ab4154a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9ab406a370 tx=0x7f9aac00ba50).stop 2024-01-24T13:58:16.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.720+0000 7f9abc4c7700 1 -- 172.21.15.29:0/4247070038 shutdown_connections 2024-01-24T13:58:16.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.720+0000 7f9abc4c7700 1 --2- 172.21.15.29:0/4247070038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ab40c4b00 0x7f9ab4154a70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.720+0000 7f9abc4c7700 1 --2- 172.21.15.29:0/4247070038 >> v2:172.21.15.163:3300/0 conn(0x7f9ab40c3660 0x7f9ab414b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.720+0000 7f9abc4c7700 1 -- 172.21.15.29:0/4247070038 >> 172.21.15.29:0/4247070038 conn(0x7f9ab4067b20 msgr2=0x7f9ab40638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:16.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.720+0000 7f9abc4c7700 1 -- 172.21.15.29:0/4247070038 shutdown_connections 2024-01-24T13:58:16.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.720+0000 7f9abc4c7700 1 -- 172.21.15.29:0/4247070038 wait complete. 2024-01-24T13:58:16.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.722+0000 7f9abc4c7700 1 Processor -- start 2024-01-24T13:58:16.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.722+0000 7f9abc4c7700 1 -- start start 2024-01-24T13:58:16.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.722+0000 7f9abc4c7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab40c3660 0x7f9ab4147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.722+0000 7f9abc4c7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ab40c4b00 0x7f9ab40bebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.722+0000 7f9abc4c7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ab414b0a0 0x7f9ab4147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.722+0000 7f9abc4c7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40bc8a0 con 0x7f9ab414b0a0 2024-01-24T13:58:16.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.722+0000 7f9abc4c7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40c19e0 con 0x7f9ab40c3660 2024-01-24T13:58:16.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9abc4c7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab4147670 con 0x7f9ab40c4b00 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9abaa64700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ab414b0a0 0x7f9ab4147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9abaa64700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ab414b0a0 0x7f9ab4147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39252/0 (socket says 172.21.15.29:39252) 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9abaa64700 1 -- 172.21.15.29:0/244345378 learned_addr learned my addr 172.21.15.29:0/244345378 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9ab9a62700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ab40c4b00 0x7f9ab40bebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9aba263700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab40c3660 0x7f9ab4147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9abaa64700 1 -- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ab40c4b00 msgr2=0x7f9ab40bebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9abaa64700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ab40c4b00 0x7f9ab40bebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9abaa64700 1 -- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab40c3660 msgr2=0x7f9ab4147180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9abaa64700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab40c3660 0x7f9ab4147180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9abaa64700 1 -- 172.21.15.29:0/244345378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac00bcf0 con 0x7f9ab414b0a0 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9aba263700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab40c3660 0x7f9ab4147180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9abaa64700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ab414b0a0 0x7f9ab4147b10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f9aac00ba90 tx=0x7f9ab4151350).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:16.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9aaa7fc700 1 -- 172.21.15.29:0/244345378 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac00b220 con 0x7f9ab414b0a0 2024-01-24T13:58:16.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9aaa7fc700 1 -- 172.21.15.29:0/244345378 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac015420 con 0x7f9ab414b0a0 2024-01-24T13:58:16.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9abc4c7700 1 -- 172.21.15.29:0/244345378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab40bf0e0 con 0x7f9ab414b0a0 2024-01-24T13:58:16.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9aaa7fc700 1 -- 172.21.15.29:0/244345378 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac0157c0 con 0x7f9ab414b0a0 2024-01-24T13:58:16.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.723+0000 7f9abc4c7700 1 -- 172.21.15.29:0/244345378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab40bf690 con 0x7f9ab414b0a0 2024-01-24T13:58:16.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.725+0000 7f9abc4c7700 1 -- 172.21.15.29:0/244345378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab40bca30 con 0x7f9ab414b0a0 2024-01-24T13:58:16.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.725+0000 7f9aaa7fc700 1 -- 172.21.15.29:0/244345378 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9aac003040 con 0x7f9ab414b0a0 2024-01-24T13:58:16.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.725+0000 7f9aaa7fc700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a9805f150 0x7f9a98061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:16.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.725+0000 7f9aaa7fc700 1 -- 172.21.15.29:0/244345378 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f9aac07b080 con 0x7f9ab414b0a0 2024-01-24T13:58:16.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.725+0000 7f9aba263700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a9805f150 0x7f9a98061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:16.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.726+0000 7f9aba263700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a9805f150 0x7f9a98061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9ab0001690 tx=0x7f9ab0009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:16.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.729+0000 7f9aaa7fc700 1 -- 172.21.15.29:0/244345378 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9aac04c6a0 con 0x7f9ab414b0a0 2024-01-24T13:58:16.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:16.912+0000 7f9abc4c7700 1 -- 172.21.15.29:0/244345378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f9ab40475e0 con 0x7f9ab414b0a0 2024-01-24T13:58:17.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.772+0000 7f9aaa7fc700 1 -- 172.21.15.29:0/244345378 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 152+0+0 (secure 0 0 0) 0x7f9aac023070 con 0x7f9ab414b0a0 2024-01-24T13:58:17.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.777+0000 7f9abc4c7700 1 -- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a9805f150 msgr2=0x7f9a98061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:17.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.777+0000 7f9abc4c7700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a9805f150 0x7f9a98061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9ab0001690 tx=0x7f9ab0009040).stop 2024-01-24T13:58:17.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.777+0000 7f9abc4c7700 1 -- 172.21.15.29:0/244345378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ab414b0a0 msgr2=0x7f9ab4147b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:17.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.777+0000 7f9abc4c7700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ab414b0a0 0x7f9ab4147b10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f9aac00ba90 tx=0x7f9ab4151350).stop 2024-01-24T13:58:17.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.778+0000 7f9abc4c7700 1 -- 172.21.15.29:0/244345378 shutdown_connections 2024-01-24T13:58:17.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.778+0000 7f9abc4c7700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a9805f150 0x7f9a98061610 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0x7f9ab0001690 tx=0x7f9ab0009040).stop 2024-01-24T13:58:17.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.778+0000 7f9abc4c7700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ab414b0a0 0x7f9ab4147b10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:17.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.778+0000 7f9abc4c7700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab40c3660 0x7f9ab4147180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:17.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.778+0000 7f9abc4c7700 1 --2- 172.21.15.29:0/244345378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ab40c4b00 0x7f9ab40bebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:17.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.778+0000 7f9abc4c7700 1 -- 172.21.15.29:0/244345378 >> 172.21.15.29:0/244345378 conn(0x7f9ab4067b20 msgr2=0x7f9ab4063aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:17.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.779+0000 7f9abc4c7700 1 -- 172.21.15.29:0/244345378 shutdown_connections 2024-01-24T13:58:17.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.779+0000 7f9abc4c7700 1 -- 172.21.15.29:0/244345378 wait complete. 2024-01-24T13:58:17.783 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-01-24T13:58:17.800 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-01-24T13:58:17.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.942+0000 7f99d87db700 1 Processor -- start 2024-01-24T13:58:17.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.942+0000 7f99d87db700 1 -- start start 2024-01-24T13:58:17.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.943+0000 7f99d87db700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00c3e60 0x7f99d0065f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:17.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.943+0000 7f99d87db700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f99d00627b0 0x7f99d0062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:17.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.943+0000 7f99d87db700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d0063150 con 0x7f99d014e5f0 2024-01-24T13:58:17.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.943+0000 7f99d87db700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d006a4b0 con 0x7f99d00627b0 2024-01-24T13:58:17.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.943+0000 7f99d87db700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d0155230 con 0x7f99d00c3e60 2024-01-24T13:58:17.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.943+0000 7f99d6577700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f99d014e5f0 0x7f99d00c2260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58848/0 (socket says 172.21.15.29:58848) 2024-01-24T13:58:17.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.943+0000 7f99d6577700 1 -- 172.21.15.29:0/2622297800 learned_addr learned my addr 172.21.15.29:0/2622297800 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:17.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.943+0000 7f99d6d78700 1 --2- 172.21.15.29:0/2622297800 >> v2:172.21.15.163:3300/0 conn(0x7f99d00627b0 0x7f99d0062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:17.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.943+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173046999 0 0) 0x7f99cc001110 con 0x7f99d014e5f0 2024-01-24T13:58:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.943+0000 7f99d5d76700 1 --2- 172.21.15.29:0/2622297800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00c3e60 0x7f99d0065f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.943+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99bc0033f0 con 0x7f99d014e5f0 2024-01-24T13:58:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2461598488 0 0) 0x7f99bc0033f0 con 0x7f99d014e5f0 2024-01-24T13:58:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00c3e60 msgr2=0x7f99d0065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 --2- 172.21.15.29:0/2622297800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00c3e60 0x7f99d0065f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 >> v2:172.21.15.163:3300/0 conn(0x7f99d00627b0 msgr2=0x7f99d0062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 --2- 172.21.15.29:0/2622297800 >> v2:172.21.15.163:3300/0 conn(0x7f99d00627b0 0x7f99d0062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d0154640 con 0x7f99d014e5f0 2024-01-24T13:58:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f99cc001600 con 0x7f99d014e5f0 2024-01-24T13:58:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 >> v1:172.21.15.29:6789/0 conn(0x7f99d014e5f0 legacy=0x7f99d00c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 --2- 172.21.15.29:0/2622297800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99bc0038e0 0x7f99bc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 --2- 172.21.15.29:0/2622297800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99bc004690 0x7f99bc004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 --2- 172.21.15.29:0/2622297800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99bc004fc0 0x7f99bc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d006a4b0 con 0x7f99bc0038e0 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d0155230 con 0x7f99bc004fc0 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc004430 con 0x7f99bc004690 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d6577700 1 --2- 172.21.15.29:0/2622297800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99bc0038e0 0x7f99bc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5d76700 1 --2- 172.21.15.29:0/2622297800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99bc004690 0x7f99bc004ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d6d78700 1 --2- 172.21.15.29:0/2622297800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99bc004fc0 0x7f99bc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d6577700 1 -- 172.21.15.29:0/2622297800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99bc004690 msgr2=0x7f99bc004ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d6577700 1 --2- 172.21.15.29:0/2622297800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99bc004690 0x7f99bc004ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d6577700 1 -- 172.21.15.29:0/2622297800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99bc004fc0 msgr2=0x7f99bc00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d6577700 1 --2- 172.21.15.29:0/2622297800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99bc004fc0 0x7f99bc00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d6577700 1 -- 172.21.15.29:0/2622297800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc0033f0 con 0x7f99bc0038e0 2024-01-24T13:58:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d6577700 1 --2- 172.21.15.29:0/2622297800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99bc0038e0 0x7f99bc003d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f99cc00c340 tx=0x7f99cc007900).ready entity=mon.0 client_cookie=1802c435db34b3e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.944+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc007bf0 con 0x7f99bc0038e0 2024-01-24T13:58:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.945+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc00cc70 con 0x7f99bc0038e0 2024-01-24T13:58:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.945+0000 7f99d5575700 1 -- 172.21.15.29:0/2622297800 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0070d0 con 0x7f99bc0038e0 2024-01-24T13:58:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.945+0000 7f99d87db700 1 -- 172.21.15.29:0/2622297800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99bc0038e0 msgr2=0x7f99bc003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.945+0000 7f99d87db700 1 --2- 172.21.15.29:0/2622297800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99bc0038e0 0x7f99bc003d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f99cc00c340 tx=0x7f99cc007900).stop 2024-01-24T13:58:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.945+0000 7f99d6d78700 1 -- 172.21.15.29:0/2622297800 reap_dead start 2024-01-24T13:58:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.945+0000 7f99d87db700 1 -- 172.21.15.29:0/2622297800 shutdown_connections 2024-01-24T13:58:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.945+0000 7f99d87db700 1 -- 172.21.15.29:0/2622297800 >> 172.21.15.29:0/2622297800 conn(0x7f99d00638c0 msgr2=0x7f99d0063cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.945+0000 7f99d87db700 1 -- 172.21.15.29:0/2622297800 shutdown_connections 2024-01-24T13:58:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.945+0000 7f99d87db700 1 -- 172.21.15.29:0/2622297800 wait complete. 2024-01-24T13:58:17.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.946+0000 7f99d87db700 1 Processor -- start 2024-01-24T13:58:17.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.946+0000 7f99d87db700 1 -- start start 2024-01-24T13:58:17.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.947+0000 7f99d87db700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99bc0038e0 0x7f99d00a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:17.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.947+0000 7f99d87db700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014caf0 0x7f99d014cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:17.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.947+0000 7f99d87db700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00a77d0 0x7f99d00a7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:17.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.947+0000 7f99d87db700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d014d460 con 0x7f99bc0038e0 2024-01-24T13:58:17.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.947+0000 7f99d87db700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d00aa020 con 0x7f99d014caf0 2024-01-24T13:58:17.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.947+0000 7f99d87db700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d00a8140 con 0x7f99d00a77d0 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.947+0000 7f99d6577700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99bc0038e0 0x7f99d00a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.947+0000 7f99d6577700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99bc0038e0 0x7f99d00a8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39274/0 (socket says 172.21.15.29:39274) 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.947+0000 7f99d6577700 1 -- 172.21.15.29:0/3755236504 learned_addr learned my addr 172.21.15.29:0/3755236504 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.947+0000 7f99d6d78700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00a77d0 0x7f99d00a7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.947+0000 7f99d5d76700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014caf0 0x7f99d014cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.948+0000 7f99d6577700 1 -- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00a77d0 msgr2=0x7f99d00a7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.948+0000 7f99d6577700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00a77d0 0x7f99d00a7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.948+0000 7f99d6577700 1 -- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014caf0 msgr2=0x7f99d014cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.948+0000 7f99d6577700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014caf0 0x7f99d014cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.948+0000 7f99d6577700 1 -- 172.21.15.29:0/3755236504 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d00a9410 con 0x7f99bc0038e0 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.948+0000 7f99d5d76700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014caf0 0x7f99d014cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.948+0000 7f99d6577700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99bc0038e0 0x7f99d00a8e50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f99cc007d20 tx=0x7f99cc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.948+0000 7f99c27fc700 1 -- 172.21.15.29:0/3755236504 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0019a0 con 0x7f99bc0038e0 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.948+0000 7f99c27fc700 1 -- 172.21.15.29:0/3755236504 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc0046b0 con 0x7f99bc0038e0 2024-01-24T13:58:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.948+0000 7f99c27fc700 1 -- 172.21.15.29:0/3755236504 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc004a30 con 0x7f99bc0038e0 2024-01-24T13:58:17.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.948+0000 7f99d87db700 1 -- 172.21.15.29:0/3755236504 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d00a96f0 con 0x7f99bc0038e0 2024-01-24T13:58:17.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.948+0000 7f99d87db700 1 -- 172.21.15.29:0/3755236504 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d00a9cb0 con 0x7f99bc0038e0 2024-01-24T13:58:17.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.949+0000 7f99d87db700 1 -- 172.21.15.29:0/3755236504 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d00bc7c0 con 0x7f99bc0038e0 2024-01-24T13:58:17.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.949+0000 7f99c27fc700 1 -- 172.21.15.29:0/3755236504 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f99cc013450 con 0x7f99bc0038e0 2024-01-24T13:58:17.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.950+0000 7f99c27fc700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f99b405f120 0x7f99b40615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:17.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.950+0000 7f99c27fc700 1 -- 172.21.15.29:0/3755236504 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f99cc07ab30 con 0x7f99bc0038e0 2024-01-24T13:58:17.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.950+0000 7f99d5d76700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f99b405f120 0x7f99b40615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:17.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.951+0000 7f99d5d76700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f99b405f120 0x7f99b40615e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f99d00c2050 tx=0x7f99c4008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:17.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:17.955+0000 7f99c27fc700 1 -- 172.21.15.29:0/3755236504 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f99cc04bf80 con 0x7f99bc0038e0 2024-01-24T13:58:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.136+0000 7f99d87db700 1 -- 172.21.15.29:0/3755236504 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f99d0063150 con 0x7f99bc0038e0 2024-01-24T13:58:18.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.778+0000 7f99c27fc700 1 -- 172.21.15.29:0/3755236504 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) v1 ==== 171+0+0 (secure 0 0 0) 0x7f99d0063150 con 0x7f99bc0038e0 2024-01-24T13:58:18.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.783+0000 7f99d87db700 1 -- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f99b405f120 msgr2=0x7f99b40615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:18.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.783+0000 7f99d87db700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f99b405f120 0x7f99b40615e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f99d00c2050 tx=0x7f99c4008040).stop 2024-01-24T13:58:18.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.783+0000 7f99d87db700 1 -- 172.21.15.29:0/3755236504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99bc0038e0 msgr2=0x7f99d00a8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:18.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.783+0000 7f99d87db700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99bc0038e0 0x7f99d00a8e50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f99cc007d20 tx=0x7f99cc015000).stop 2024-01-24T13:58:18.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.785+0000 7f99d87db700 1 -- 172.21.15.29:0/3755236504 shutdown_connections 2024-01-24T13:58:18.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.785+0000 7f99d87db700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99d00a77d0 0x7f99d00a7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:18.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.785+0000 7f99d87db700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f99b405f120 0x7f99b40615e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:18.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.785+0000 7f99d87db700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99bc0038e0 0x7f99d00a8e50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:18.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.785+0000 7f99d87db700 1 --2- 172.21.15.29:0/3755236504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99d014caf0 0x7f99d014cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:18.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.785+0000 7f99d87db700 1 -- 172.21.15.29:0/3755236504 >> 172.21.15.29:0/3755236504 conn(0x7f99d00638c0 msgr2=0x7f99d014d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:18.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.786+0000 7f99d87db700 1 -- 172.21.15.29:0/3755236504 shutdown_connections 2024-01-24T13:58:18.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.788+0000 7f99d87db700 1 -- 172.21.15.29:0/3755236504 wait complete. 2024-01-24T13:58:18.797 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-01-24T13:58:18.816 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-24T13:58:18.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.951+0000 7fe8622d8700 1 Processor -- start 2024-01-24T13:58:18.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.951+0000 7fe8622d8700 1 -- start start 2024-01-24T13:58:18.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.952+0000 7fe8622d8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe85c153f40 0x7fe85c0a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:18.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.952+0000 7fe8622d8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85c06a360 con 0x7fe85c062ec0 2024-01-24T13:58:18.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.952+0000 7fe8622d8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85c0bcef0 con 0x7fe85c0c8290 2024-01-24T13:58:18.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.952+0000 7fe8622d8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c069de0 con 0x7fe85c153f40 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.952+0000 7fe860875700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe85c062ec0 0x7fe85c065440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58856/0 (socket says 172.21.15.29:58856) 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.952+0000 7fe860875700 1 -- 172.21.15.29:0/2158315250 learned_addr learned my addr 172.21.15.29:0/2158315250 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.952+0000 7fe85bfff700 1 --2- 172.21.15.29:0/2158315250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe85c153f40 0x7fe85c0a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.953+0000 7fe85affd700 1 -- 172.21.15.29:0/2158315250 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616235837 0 0) 0x7fe85c06a360 con 0x7fe85c062ec0 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.953+0000 7fe85affd700 1 -- 172.21.15.29:0/2158315250 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8440033f0 con 0x7fe85c062ec0 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.953+0000 7fe85affd700 1 -- 172.21.15.29:0/2158315250 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298948669 0 0) 0x7fe85c0bcef0 con 0x7fe85c0c8290 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.953+0000 7fe85affd700 1 -- 172.21.15.29:0/2158315250 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe85c06a360 con 0x7fe85c0c8290 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.953+0000 7fe85bfff700 1 -- 172.21.15.29:0/2158315250 >> v1:172.21.15.163:6789/0 conn(0x7fe85c0c8290 legacy=0x7fe85c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.953+0000 7fe85bfff700 1 -- 172.21.15.29:0/2158315250 >> v1:172.21.15.29:6789/0 conn(0x7fe85c062ec0 legacy=0x7fe85c065440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.953+0000 7fe85bfff700 1 -- 172.21.15.29:0/2158315250 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c06a080 con 0x7fe85c153f40 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.953+0000 7fe85bfff700 1 --2- 172.21.15.29:0/2158315250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe85c153f40 0x7fe85c0a6db0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fe84c009790 tx=0x7fe84c009060).ready entity=mon.2 client_cookie=ed9667e824a986e7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.953+0000 7fe85affd700 1 -- 172.21.15.29:0/2158315250 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0090d0 con 0x7fe85c153f40 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.953+0000 7fe85affd700 1 -- 172.21.15.29:0/2158315250 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c00f510 con 0x7fe85c153f40 2024-01-24T13:58:18.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.953+0000 7fe85affd700 1 -- 172.21.15.29:0/2158315250 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00f890 con 0x7fe85c153f40 2024-01-24T13:58:18.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.954+0000 7fe8622d8700 1 -- 172.21.15.29:0/2158315250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe85c153f40 msgr2=0x7fe85c0a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:18.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.954+0000 7fe8622d8700 1 --2- 172.21.15.29:0/2158315250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe85c153f40 0x7fe85c0a6db0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fe84c009790 tx=0x7fe84c009060).stop 2024-01-24T13:58:18.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.954+0000 7fe8622d8700 1 -- 172.21.15.29:0/2158315250 shutdown_connections 2024-01-24T13:58:18.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.954+0000 7fe8622d8700 1 --2- 172.21.15.29:0/2158315250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe85c153f40 0x7fe85c0a6db0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:18.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.954+0000 7fe8622d8700 1 -- 172.21.15.29:0/2158315250 >> 172.21.15.29:0/2158315250 conn(0x7fe85c064390 msgr2=0x7fe85c0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:18.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.954+0000 7fe8622d8700 1 -- 172.21.15.29:0/2158315250 shutdown_connections 2024-01-24T13:58:18.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.954+0000 7fe8622d8700 1 -- 172.21.15.29:0/2158315250 wait complete. 2024-01-24T13:58:18.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.955+0000 7fe8622d8700 1 Processor -- start 2024-01-24T13:58:18.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.955+0000 7fe8622d8700 1 -- start start 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.956+0000 7fe8622d8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe85c062ec0 0x7fe85c0c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.956+0000 7fe8622d8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe85c0c8290 0x7fe85c1513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.956+0000 7fe8622d8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe85c153f40 0x7fe85c1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.956+0000 7fe8622d8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c152bf0 con 0x7fe85c062ec0 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.956+0000 7fe8622d8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c151d90 con 0x7fe85c0c8290 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.956+0000 7fe8622d8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c1647b0 con 0x7fe85c153f40 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.956+0000 7fe85bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe85c062ec0 0x7fe85c0c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.956+0000 7fe85bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe85c062ec0 0x7fe85c0c34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39284/0 (socket says 172.21.15.29:39284) 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.956+0000 7fe85bfff700 1 -- 172.21.15.29:0/780595381 learned_addr learned my addr 172.21.15.29:0/780595381 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.956+0000 7fe85b7fe700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe85c0c8290 0x7fe85c1513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.956+0000 7fe860875700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe85c153f40 0x7fe85c1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.957+0000 7fe85bfff700 1 -- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe85c153f40 msgr2=0x7fe85c1518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.957+0000 7fe85bfff700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe85c153f40 0x7fe85c1518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:18.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.957+0000 7fe85bfff700 1 -- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe85c0c8290 msgr2=0x7fe85c1513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:18.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.957+0000 7fe85bfff700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe85c0c8290 0x7fe85c1513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:18.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.957+0000 7fe85bfff700 1 -- 172.21.15.29:0/780595381 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c0092a0 con 0x7fe85c062ec0 2024-01-24T13:58:18.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.957+0000 7fe860875700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe85c153f40 0x7fe85c1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:18.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.957+0000 7fe85bfff700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe85c062ec0 0x7fe85c0c34b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fe85c064910 tx=0x7fe84c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:18.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.957+0000 7fe843fff700 1 -- 172.21.15.29:0/780595381 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00f890 con 0x7fe85c062ec0 2024-01-24T13:58:18.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.957+0000 7fe843fff700 1 -- 172.21.15.29:0/780595381 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c000cc0 con 0x7fe85c062ec0 2024-01-24T13:58:18.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.957+0000 7fe8622d8700 1 -- 172.21.15.29:0/780595381 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85c151f70 con 0x7fe85c062ec0 2024-01-24T13:58:18.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.957+0000 7fe843fff700 1 -- 172.21.15.29:0/780595381 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00fea0 con 0x7fe85c062ec0 2024-01-24T13:58:18.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.957+0000 7fe8622d8700 1 -- 172.21.15.29:0/780595381 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c152530 con 0x7fe85c062ec0 2024-01-24T13:58:18.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.958+0000 7fe8622d8700 1 -- 172.21.15.29:0/780595381 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85c152e60 con 0x7fe85c062ec0 2024-01-24T13:58:18.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.958+0000 7fe843fff700 1 -- 172.21.15.29:0/780595381 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe84c01b620 con 0x7fe85c062ec0 2024-01-24T13:58:18.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.959+0000 7fe843fff700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe84805f150 0x7fe848061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:18.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.959+0000 7fe843fff700 1 -- 172.21.15.29:0/780595381 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6021+0+0 (secure 0 0 0) 0x7fe84c07b7f0 con 0x7fe85c062ec0 2024-01-24T13:58:18.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.959+0000 7fe85b7fe700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe84805f150 0x7fe848061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:18.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.960+0000 7fe85b7fe700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe84805f150 0x7fe848061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe85c0bfb70 tx=0x7fe850007bd0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:18.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:18.963+0000 7fe843fff700 1 -- 172.21.15.29:0/780595381 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe84c04cb20 con 0x7fe85c062ec0 2024-01-24T13:58:19.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.188+0000 7fe8622d8700 1 -- 172.21.15.29:0/780595381 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe85c0475e0 con 0x7fe85c062ec0 2024-01-24T13:58:19.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.194+0000 7fe843fff700 1 -- 172.21.15.29:0/780595381 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe84c029020 con 0x7fe85c062ec0 2024-01-24T13:58:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.206+0000 7fe8622d8700 1 -- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe84805f150 msgr2=0x7fe848061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.206+0000 7fe8622d8700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe84805f150 0x7fe848061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe85c0bfb70 tx=0x7fe850007bd0).stop 2024-01-24T13:58:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.206+0000 7fe8622d8700 1 -- 172.21.15.29:0/780595381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe85c062ec0 msgr2=0x7fe85c0c34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.206+0000 7fe8622d8700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe85c062ec0 0x7fe85c0c34b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fe85c064910 tx=0x7fe84c017000).stop 2024-01-24T13:58:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.206+0000 7fe8622d8700 1 -- 172.21.15.29:0/780595381 shutdown_connections 2024-01-24T13:58:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.206+0000 7fe8622d8700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe85c153f40 0x7fe85c1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.206+0000 7fe8622d8700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe84805f150 0x7fe848061610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.206+0000 7fe8622d8700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe85c062ec0 0x7fe85c0c34b0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0x7fe85c064910 tx=0x7fe84c017000).stop 2024-01-24T13:58:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.206+0000 7fe8622d8700 1 --2- 172.21.15.29:0/780595381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe85c0c8290 0x7fe85c1513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:19.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.206+0000 7fe8622d8700 1 -- 172.21.15.29:0/780595381 >> 172.21.15.29:0/780595381 conn(0x7fe85c064390 msgr2=0x7fe85c065350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:19.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.207+0000 7fe8622d8700 1 -- 172.21.15.29:0/780595381 shutdown_connections 2024-01-24T13:58:19.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.207+0000 7fe8622d8700 1 -- 172.21.15.29:0/780595381 wait complete. 2024-01-24T13:58:19.225 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 3 and data pool 4 2024-01-24T13:58:19.237 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-01-24T13:58:19.237 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-01-24T13:58:19.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.371+0000 7fecf2749700 1 Processor -- start 2024-01-24T13:58:19.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.371+0000 7fecf2749700 1 -- start start 2024-01-24T13:58:19.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.372+0000 7fecf2749700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fecec14f8f0 0x7fecec152490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:19.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.372+0000 7fecf2749700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecec14fcd0 0x7fecec156190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:19.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.372+0000 7fecf2749700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec06a690 con 0x7fecec156680 2024-01-24T13:58:19.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.372+0000 7fecf2749700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecec155a60 con 0x7fecec14f8f0 2024-01-24T13:58:19.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.372+0000 7fecf2749700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec155d00 con 0x7fecec14fcd0 2024-01-24T13:58:19.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.372+0000 7fecf0ce6700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fecec156680 0x7fecec065410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58864/0 (socket says 172.21.15.29:58864) 2024-01-24T13:58:19.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.372+0000 7fecf0ce6700 1 -- 172.21.15.29:0/135050935 learned_addr learned my addr 172.21.15.29:0/135050935 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:19.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.372+0000 7fecebfff700 1 --2- 172.21.15.29:0/135050935 >> v2:172.21.15.163:3300/0 conn(0x7fecec14f8f0 0x7fecec152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.372+0000 7feceb7fe700 1 --2- 172.21.15.29:0/135050935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecec14fcd0 0x7fecec156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.372+0000 7feceaffd700 1 -- 172.21.15.29:0/135050935 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774839174 0 0) 0x7fece4001490 con 0x7fecec156680 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.373+0000 7feceaffd700 1 -- 172.21.15.29:0/135050935 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd40033f0 con 0x7fecec156680 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.373+0000 7feceaffd700 1 -- 172.21.15.29:0/135050935 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 44907118 0 0) 0x7fecd40033f0 con 0x7fecec156680 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.373+0000 7fecebfff700 1 -- 172.21.15.29:0/135050935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecec14fcd0 msgr2=0x7fecec156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.373+0000 7fecebfff700 1 --2- 172.21.15.29:0/135050935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecec14fcd0 0x7fecec156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.373+0000 7fecebfff700 1 -- 172.21.15.29:0/135050935 >> v1:172.21.15.29:6789/0 conn(0x7fecec156680 legacy=0x7fecec065410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.373+0000 7fecebfff700 1 -- 172.21.15.29:0/135050935 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec152cf0 con 0x7fecec14f8f0 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.373+0000 7fecebfff700 1 --2- 172.21.15.29:0/135050935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecec14f8f0 0x7fecec152490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fecdc009760 tx=0x7fecdc009060).ready entity=mon.1 client_cookie=f0606b85d2e86f70 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.373+0000 7feceaffd700 1 -- 172.21.15.29:0/135050935 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4001980 con 0x7fecec14f8f0 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.373+0000 7feceaffd700 1 -- 172.21.15.29:0/135050935 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc011400 con 0x7fecec14f8f0 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.373+0000 7feceaffd700 1 -- 172.21.15.29:0/135050935 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc011780 con 0x7fecec14f8f0 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.374+0000 7fecf2749700 1 -- 172.21.15.29:0/135050935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecec14f8f0 msgr2=0x7fecec152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:19.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.374+0000 7fecf2749700 1 --2- 172.21.15.29:0/135050935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecec14f8f0 0x7fecec152490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fecdc009760 tx=0x7fecdc009060).stop 2024-01-24T13:58:19.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.374+0000 7fecf2749700 1 -- 172.21.15.29:0/135050935 shutdown_connections 2024-01-24T13:58:19.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.374+0000 7fecf2749700 1 --2- 172.21.15.29:0/135050935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecec14f8f0 0x7fecec152490 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:19.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.374+0000 7fecf2749700 1 --2- 172.21.15.29:0/135050935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecec14fcd0 0x7fecec156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:19.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.374+0000 7fecf2749700 1 -- 172.21.15.29:0/135050935 >> 172.21.15.29:0/135050935 conn(0x7fecec067b20 msgr2=0x7fecec0638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:19.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.374+0000 7fecf2749700 1 -- 172.21.15.29:0/135050935 shutdown_connections 2024-01-24T13:58:19.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.374+0000 7fecf2749700 1 -- 172.21.15.29:0/135050935 wait complete. 2024-01-24T13:58:19.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.375+0000 7fecf2749700 1 Processor -- start 2024-01-24T13:58:19.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.375+0000 7fecf2749700 1 -- start start 2024-01-24T13:58:19.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf2749700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecec14f8f0 0x7fecec146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:19.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf2749700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecec14fcd0 0x7fecec0c0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:19.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf2749700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecec156680 0x7fecec14d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:19.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf2749700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0c0170 con 0x7fecec156680 2024-01-24T13:58:19.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf2749700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0bfb60 con 0x7fecec14f8f0 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf2749700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec164ff0 con 0x7fecec14fcd0 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf0ce6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecec156680 0x7fecec14d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf0ce6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecec156680 0x7fecec14d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39298/0 (socket says 172.21.15.29:39298) 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf0ce6700 1 -- 172.21.15.29:0/3072074791 learned_addr learned my addr 172.21.15.29:0/3072074791 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecebfff700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecec14f8f0 0x7fecec146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7feceb7fe700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecec14fcd0 0x7fecec0c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf0ce6700 1 -- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecec14fcd0 msgr2=0x7fecec0c0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf0ce6700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecec14fcd0 0x7fecec0c0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf0ce6700 1 -- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecec14f8f0 msgr2=0x7fecec146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf0ce6700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecec14f8f0 0x7fecec146fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecf0ce6700 1 -- 172.21.15.29:0/3072074791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc0092a0 con 0x7fecec156680 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.376+0000 7fecebfff700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecec14f8f0 0x7fecec146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.377+0000 7fecf0ce6700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecec156680 0x7fecec14d830 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fece400cb90 tx=0x7fece4002190).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:19.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.377+0000 7fecd3fff700 1 -- 172.21.15.29:0/3072074791 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4004c40 con 0x7fecec156680 2024-01-24T13:58:19.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.377+0000 7fecd3fff700 1 -- 172.21.15.29:0/3072074791 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece4004da0 con 0x7fecec156680 2024-01-24T13:58:19.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.377+0000 7fecf2749700 1 -- 172.21.15.29:0/3072074791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec0bfe00 con 0x7fecec156680 2024-01-24T13:58:19.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.377+0000 7fecd3fff700 1 -- 172.21.15.29:0/3072074791 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4007380 con 0x7fecec156680 2024-01-24T13:58:19.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.377+0000 7fecf2749700 1 -- 172.21.15.29:0/3072074791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec1652d0 con 0x7fecec156680 2024-01-24T13:58:19.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.378+0000 7fecf2749700 1 -- 172.21.15.29:0/3072074791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec0475e0 con 0x7fecec156680 2024-01-24T13:58:19.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.378+0000 7fecd3fff700 1 -- 172.21.15.29:0/3072074791 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fece4007770 con 0x7fecec156680 2024-01-24T13:58:19.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.379+0000 7fecd3fff700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fecd805f150 0x7fecd8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:19.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.379+0000 7fecd3fff700 1 -- 172.21.15.29:0/3072074791 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fece4079f30 con 0x7fecec156680 2024-01-24T13:58:19.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.379+0000 7fecebfff700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fecd805f150 0x7fecd8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:19.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.379+0000 7fecebfff700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fecd805f150 0x7fecd8061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fecec0bb210 tx=0x7fecdc016000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:19.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.383+0000 7fecd3fff700 1 -- 172.21.15.29:0/3072074791 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fece404b200 con 0x7fecec156680 2024-01-24T13:58:19.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:19.578+0000 7fecf2749700 1 -- 172.21.15.29:0/3072074791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fecec06a690 con 0x7fecec156680 2024-01-24T13:58:20.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.196+0000 7fecd3fff700 1 -- 172.21.15.29:0/3072074791 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) v1 ==== 157+0+0 (secure 0 0 0) 0x7fecec06a690 con 0x7fecec156680 2024-01-24T13:58:20.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.199+0000 7fecf2749700 1 -- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fecd805f150 msgr2=0x7fecd8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:20.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.199+0000 7fecf2749700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fecd805f150 0x7fecd8061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fecec0bb210 tx=0x7fecdc016000).stop 2024-01-24T13:58:20.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.199+0000 7fecf2749700 1 -- 172.21.15.29:0/3072074791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecec156680 msgr2=0x7fecec14d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:20.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.199+0000 7fecf2749700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecec156680 0x7fecec14d830 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fece400cb90 tx=0x7fece4002190).stop 2024-01-24T13:58:20.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.199+0000 7fecf2749700 1 -- 172.21.15.29:0/3072074791 shutdown_connections 2024-01-24T13:58:20.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.199+0000 7fecf2749700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fecd805f150 0x7fecd8061610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:20.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.199+0000 7fecf2749700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecec156680 0x7fecec14d830 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:20.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.199+0000 7fecf2749700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecec14f8f0 0x7fecec146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:20.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.199+0000 7fecf2749700 1 --2- 172.21.15.29:0/3072074791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecec14fcd0 0x7fecec0c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:20.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.199+0000 7fecf2749700 1 -- 172.21.15.29:0/3072074791 >> 172.21.15.29:0/3072074791 conn(0x7fecec067b20 msgr2=0x7fecec14eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:20.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.199+0000 7fecf2749700 1 -- 172.21.15.29:0/3072074791 shutdown_connections 2024-01-24T13:58:20.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.200+0000 7fecf2749700 1 -- 172.21.15.29:0/3072074791 wait complete. 2024-01-24T13:58:20.215 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-01-24T13:58:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.357+0000 7f0bd27dc700 1 Processor -- start 2024-01-24T13:58:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.357+0000 7f0bd27dc700 1 -- start start 2024-01-24T13:58:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.358+0000 7f0bd27dc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bcc0c3bd0 0x7f0bcc066770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.358+0000 7f0bd27dc700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bcc153530 con 0x7f0bcc1353a0 2024-01-24T13:58:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.358+0000 7f0bd27dc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bcc152c00 con 0x7f0bcc066c60 2024-01-24T13:58:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.358+0000 7f0bd27dc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc152ee0 con 0x7f0bcc0c3bd0 2024-01-24T13:58:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.358+0000 7f0bcbfff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0bcc1353a0 0x7f0bcc0c34c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58874/0 (socket says 172.21.15.29:58874) 2024-01-24T13:58:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.358+0000 7f0bcbfff700 1 -- 172.21.15.29:0/905744787 learned_addr learned my addr 172.21.15.29:0/905744787 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.358+0000 7f0bcb7fe700 1 --2- 172.21.15.29:0/905744787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bcc0c3bd0 0x7f0bcc066770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.358+0000 7f0bcaffd700 1 -- 172.21.15.29:0/905744787 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215692608 0 0) 0x7f0bcc153530 con 0x7f0bcc1353a0 2024-01-24T13:58:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.358+0000 7f0bcaffd700 1 -- 172.21.15.29:0/905744787 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bb80033f0 con 0x7f0bcc1353a0 2024-01-24T13:58:20.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.358+0000 7f0bcaffd700 1 -- 172.21.15.29:0/905744787 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4184857978 0 0) 0x7f0bcc152c00 con 0x7f0bcc066c60 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.358+0000 7f0bcaffd700 1 -- 172.21.15.29:0/905744787 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bcc153530 con 0x7f0bcc066c60 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bcaffd700 1 -- 172.21.15.29:0/905744787 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4196538673 0 0) 0x7f0bb80033f0 con 0x7f0bcc1353a0 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bcb7fe700 1 -- 172.21.15.29:0/905744787 >> v1:172.21.15.163:6789/0 conn(0x7f0bcc066c60 legacy=0x7f0bcc0bde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bcb7fe700 1 -- 172.21.15.29:0/905744787 >> v1:172.21.15.29:6789/0 conn(0x7f0bcc1353a0 legacy=0x7f0bcc0c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bcb7fe700 1 -- 172.21.15.29:0/905744787 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc1531c0 con 0x7f0bcc0c3bd0 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bcb7fe700 1 --2- 172.21.15.29:0/905744787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bcc0c3bd0 0x7f0bcc066770 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f0bb4009790 tx=0x7f0bb4009060).ready entity=mon.2 client_cookie=9256a55023a61ac6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bcaffd700 1 -- 172.21.15.29:0/905744787 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc001470 con 0x7f0bcc0c3bd0 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bcaffd700 1 -- 172.21.15.29:0/905744787 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb4011400 con 0x7f0bcc0c3bd0 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bcaffd700 1 -- 172.21.15.29:0/905744787 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4011780 con 0x7f0bcc0c3bd0 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bd27dc700 1 -- 172.21.15.29:0/905744787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bcc0c3bd0 msgr2=0x7f0bcc066770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bd27dc700 1 --2- 172.21.15.29:0/905744787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bcc0c3bd0 0x7f0bcc066770 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f0bb4009790 tx=0x7f0bb4009060).stop 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bd27dc700 1 -- 172.21.15.29:0/905744787 shutdown_connections 2024-01-24T13:58:20.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bd27dc700 1 --2- 172.21.15.29:0/905744787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bcc0c3bd0 0x7f0bcc066770 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:20.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.359+0000 7f0bd27dc700 1 -- 172.21.15.29:0/905744787 >> 172.21.15.29:0/905744787 conn(0x7f0bcc05ec80 msgr2=0x7f0bcc0c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:20.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.360+0000 7f0bd27dc700 1 -- 172.21.15.29:0/905744787 shutdown_connections 2024-01-24T13:58:20.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.360+0000 7f0bd27dc700 1 -- 172.21.15.29:0/905744787 wait complete. 2024-01-24T13:58:20.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.361+0000 7f0bd27dc700 1 Processor -- start 2024-01-24T13:58:20.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.361+0000 7f0bd27dc700 1 -- start start 2024-01-24T13:58:20.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd27dc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0bcc066c60 0x7f0bcc065550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:20.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd27dc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bcc0c3bd0 0x7f0bcc0638c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd27dc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc1353a0 0x7f0bcc063ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd27dc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc1652f0 con 0x7f0bcc1353a0 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd27dc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc064dc0 con 0x7f0bcc066c60 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd27dc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc164670 con 0x7f0bcc0c3bd0 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd0d79700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc1353a0 0x7f0bcc063ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd0d79700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc1353a0 0x7f0bcc063ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39306/0 (socket says 172.21.15.29:39306) 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd0d79700 1 -- 172.21.15.29:0/2879326272 learned_addr learned my addr 172.21.15.29:0/2879326272 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bcb7fe700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bcc0c3bd0 0x7f0bcc0638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bcbfff700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0bcc066c60 0x7f0bcc065550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd0d79700 1 -- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bcc0c3bd0 msgr2=0x7f0bcc0638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd0d79700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bcc0c3bd0 0x7f0bcc0638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd0d79700 1 -- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0bcc066c60 msgr2=0x7f0bcc065550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd0d79700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0bcc066c60 0x7f0bcc065550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:20.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd0d79700 1 -- 172.21.15.29:0/2879326272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb40092a0 con 0x7f0bcc1353a0 2024-01-24T13:58:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bcb7fe700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bcc0c3bd0 0x7f0bcc0638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.362+0000 7f0bd0d79700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc1353a0 0x7f0bcc063ef0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f0bc400c710 tx=0x7f0bc400cad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.363+0000 7f0bc3fff700 1 -- 172.21.15.29:0/2879326272 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4010510 con 0x7f0bcc1353a0 2024-01-24T13:58:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.363+0000 7f0bc3fff700 1 -- 172.21.15.29:0/2879326272 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc4010670 con 0x7f0bcc1353a0 2024-01-24T13:58:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.363+0000 7f0bd27dc700 1 -- 172.21.15.29:0/2879326272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc164950 con 0x7f0bcc1353a0 2024-01-24T13:58:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.363+0000 7f0bc3fff700 1 -- 172.21.15.29:0/2879326272 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40109f0 con 0x7f0bcc1353a0 2024-01-24T13:58:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.363+0000 7f0bd27dc700 1 -- 172.21.15.29:0/2879326272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc164f10 con 0x7f0bcc1353a0 2024-01-24T13:58:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.364+0000 7f0bd27dc700 1 -- 172.21.15.29:0/2879326272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bcc165430 con 0x7f0bcc1353a0 2024-01-24T13:58:20.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.364+0000 7f0bc3fff700 1 -- 172.21.15.29:0/2879326272 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0bc401b400 con 0x7f0bcc1353a0 2024-01-24T13:58:20.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.365+0000 7f0bc3fff700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0bac05f150 0x7f0bac061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:20.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.365+0000 7f0bc3fff700 1 -- 172.21.15.29:0/2879326272 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f0bc407a1a0 con 0x7f0bcc1353a0 2024-01-24T13:58:20.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.365+0000 7f0bcbfff700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0bac05f150 0x7f0bac061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:20.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.365+0000 7f0bcbfff700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0bac05f150 0x7f0bac061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0bbc001ed0 tx=0x7f0bbc000bc0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:20.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.369+0000 7f0bc3fff700 1 -- 172.21.15.29:0/2879326272 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0bc404b310 con 0x7f0bcc1353a0 2024-01-24T13:58:20.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:20.551+0000 7f0bd27dc700 1 -- 172.21.15.29:0/2879326272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0bcc05fda0 con 0x7f0bcc1353a0 2024-01-24T13:58:22.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.203+0000 7f0bc3fff700 1 -- 172.21.15.29:0/2879326272 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) v1 ==== 228+0+0 (secure 0 0 0) 0x7f0bc404e930 con 0x7f0bcc1353a0 2024-01-24T13:58:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.209+0000 7f0bd27dc700 1 -- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0bac05f150 msgr2=0x7f0bac061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.209+0000 7f0bd27dc700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0bac05f150 0x7f0bac061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0bbc001ed0 tx=0x7f0bbc000bc0).stop 2024-01-24T13:58:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.209+0000 7f0bd27dc700 1 -- 172.21.15.29:0/2879326272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc1353a0 msgr2=0x7f0bcc063ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:22.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.209+0000 7f0bd27dc700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc1353a0 0x7f0bcc063ef0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f0bc400c710 tx=0x7f0bc400cad0).stop 2024-01-24T13:58:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.218+0000 7f0bd27dc700 1 -- 172.21.15.29:0/2879326272 shutdown_connections 2024-01-24T13:58:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.218+0000 7f0bd27dc700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0bac05f150 0x7f0bac061610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.218+0000 7f0bd27dc700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bcc1353a0 0x7f0bcc063ef0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.218+0000 7f0bd27dc700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0bcc066c60 0x7f0bcc065550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.218+0000 7f0bd27dc700 1 --2- 172.21.15.29:0/2879326272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bcc0c3bd0 0x7f0bcc0638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.218+0000 7f0bd27dc700 1 -- 172.21.15.29:0/2879326272 >> 172.21.15.29:0/2879326272 conn(0x7f0bcc05ec80 msgr2=0x7f0bcc0c6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.218+0000 7f0bd27dc700 1 -- 172.21.15.29:0/2879326272 shutdown_connections 2024-01-24T13:58:22.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.218+0000 7f0bd27dc700 1 -- 172.21.15.29:0/2879326272 wait complete. 2024-01-24T13:58:22.220 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data_ec' created 2024-01-24T13:58:22.236 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-01-24T13:58:22.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.372+0000 7fc11b6fa700 1 Processor -- start 2024-01-24T13:58:22.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.372+0000 7fc11b6fa700 1 -- start start 2024-01-24T13:58:22.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.373+0000 7fc11b6fa700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc1140bf870 0x7fc114068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:22.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.373+0000 7fc11b6fa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc114066450 0x7fc1140668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:22.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.373+0000 7fc11b6fa700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1140643f0 con 0x7fc11413a6f0 2024-01-24T13:58:22.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.373+0000 7fc11b6fa700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1140bdb30 con 0x7fc1140bf870 2024-01-24T13:58:22.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.373+0000 7fc11b6fa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1140be3d0 con 0x7fc114066450 2024-01-24T13:58:22.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.373+0000 7fc118c95700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc1140bf870 0x7fc114068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:22.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.373+0000 7fc118c95700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc1140bf870 0x7fc114068660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39314/0 (socket says 172.21.15.29:39314) 2024-01-24T13:58:22.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.373+0000 7fc118c95700 1 -- 172.21.15.29:0/2527214625 learned_addr learned my addr 172.21.15.29:0/2527214625 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:22.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.373+0000 7fc119c97700 1 --2- 172.21.15.29:0/2527214625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc114066450 0x7fc1140668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:22.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.374+0000 7fc118c95700 1 -- 172.21.15.29:0/2527214625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc114066450 msgr2=0x7fc1140668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:22.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.374+0000 7fc118c95700 1 --2- 172.21.15.29:0/2527214625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc114066450 0x7fc1140668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.374+0000 7fc118c95700 1 -- 172.21.15.29:0/2527214625 >> v1:172.21.15.163:6789/0 conn(0x7fc11413a6f0 legacy=0x7fc1140638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.374+0000 7fc118c95700 1 -- 172.21.15.29:0/2527214625 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc114066dd0 con 0x7fc1140bf870 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.374+0000 7fc118c95700 1 --2- 172.21.15.29:0/2527214625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1140bf870 0x7fc114068660 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fc110006fd0 tx=0x7fc1100090f0).ready entity=mon.0 client_cookie=45bf3b9504c6499 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.374+0000 7fc107fff700 1 -- 172.21.15.29:0/2527214625 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000b600 con 0x7fc1140bf870 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.374+0000 7fc107fff700 1 -- 172.21.15.29:0/2527214625 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11000b760 con 0x7fc1140bf870 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.374+0000 7fc107fff700 1 -- 172.21.15.29:0/2527214625 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000bae0 con 0x7fc1140bf870 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.374+0000 7fc11b6fa700 1 -- 172.21.15.29:0/2527214625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1140bf870 msgr2=0x7fc114068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.374+0000 7fc11b6fa700 1 --2- 172.21.15.29:0/2527214625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1140bf870 0x7fc114068660 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fc110006fd0 tx=0x7fc1100090f0).stop 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.375+0000 7fc11b6fa700 1 -- 172.21.15.29:0/2527214625 shutdown_connections 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.375+0000 7fc11b6fa700 1 --2- 172.21.15.29:0/2527214625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc114066450 0x7fc1140668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.375+0000 7fc11b6fa700 1 --2- 172.21.15.29:0/2527214625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1140bf870 0x7fc114068660 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.375+0000 7fc11b6fa700 1 -- 172.21.15.29:0/2527214625 >> 172.21.15.29:0/2527214625 conn(0x7fc1140c8bd0 msgr2=0x7fc1140c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.375+0000 7fc11b6fa700 1 -- 172.21.15.29:0/2527214625 shutdown_connections 2024-01-24T13:58:22.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.375+0000 7fc11b6fa700 1 -- 172.21.15.29:0/2527214625 wait complete. 2024-01-24T13:58:22.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.376+0000 7fc11b6fa700 1 Processor -- start 2024-01-24T13:58:22.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.376+0000 7fc11b6fa700 1 -- start start 2024-01-24T13:58:22.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc11b6fa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc114066450 0x7fc11413c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:22.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc11b6fa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1140bf870 0x7fc114168640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:22.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc11b6fa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11413a6f0 0x7fc1140a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:22.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc11b6fa700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11413cd70 con 0x7fc114066450 2024-01-24T13:58:22.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc11b6fa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11413db70 con 0x7fc1140bf870 2024-01-24T13:58:22.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc11b6fa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc114168b30 con 0x7fc11413a6f0 2024-01-24T13:58:22.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc119496700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc114066450 0x7fc11413c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:22.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc119496700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc114066450 0x7fc11413c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39330/0 (socket says 172.21.15.29:39330) 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc119496700 1 -- 172.21.15.29:0/908539890 learned_addr learned my addr 172.21.15.29:0/908539890 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc119c97700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11413a6f0 0x7fc1140a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc118c95700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1140bf870 0x7fc114168640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc119496700 1 -- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11413a6f0 msgr2=0x7fc1140a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc119496700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11413a6f0 0x7fc1140a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc119496700 1 -- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1140bf870 msgr2=0x7fc114168640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc119496700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1140bf870 0x7fc114168640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc119496700 1 -- 172.21.15.29:0/908539890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110009370 con 0x7fc114066450 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.377+0000 7fc118c95700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1140bf870 0x7fc114168640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.378+0000 7fc119496700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc114066450 0x7fc11413c880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fc10800c440 tx=0x7fc108007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.378+0000 7fc1057fa700 1 -- 172.21.15.29:0/908539890 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108012400 con 0x7fc114066450 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.378+0000 7fc1057fa700 1 -- 172.21.15.29:0/908539890 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc108012560 con 0x7fc114066450 2024-01-24T13:58:22.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.378+0000 7fc11b6fa700 1 -- 172.21.15.29:0/908539890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11413cf60 con 0x7fc114066450 2024-01-24T13:58:22.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.378+0000 7fc1057fa700 1 -- 172.21.15.29:0/908539890 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108012900 con 0x7fc114066450 2024-01-24T13:58:22.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.378+0000 7fc11b6fa700 1 -- 172.21.15.29:0/908539890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11413d520 con 0x7fc114066450 2024-01-24T13:58:22.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.379+0000 7fc11b6fa700 1 -- 172.21.15.29:0/908539890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1141544d0 con 0x7fc114066450 2024-01-24T13:58:22.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.379+0000 7fc1057fa700 1 -- 172.21.15.29:0/908539890 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc10801b3b0 con 0x7fc114066450 2024-01-24T13:58:22.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.380+0000 7fc1057fa700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc0f805f150 0x7fc0f8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:22.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.380+0000 7fc1057fa700 1 -- 172.21.15.29:0/908539890 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6873+0+0 (secure 0 0 0) 0x7fc108079c80 con 0x7fc114066450 2024-01-24T13:58:22.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.380+0000 7fc118c95700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc0f805f150 0x7fc0f8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:22.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.380+0000 7fc118c95700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc0f805f150 0x7fc0f8061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc110009130 tx=0x7fc110017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:22.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.384+0000 7fc1057fa700 1 -- 172.21.15.29:0/908539890 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc10804ac00 con 0x7fc114066450 2024-01-24T13:58:22.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:22.562+0000 7fc11b6fa700 1 -- 172.21.15.29:0/908539890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fc1140c54e0 con 0x7fc114066450 2024-01-24T13:58:23.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.209+0000 7fc1057fa700 1 -- 172.21.15.29:0/908539890 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc10804e220 con 0x7fc114066450 2024-01-24T13:58:23.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.217+0000 7fc11b6fa700 1 -- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc0f805f150 msgr2=0x7fc0f8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:23.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.217+0000 7fc11b6fa700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc0f805f150 0x7fc0f8061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc110009130 tx=0x7fc110017000).stop 2024-01-24T13:58:23.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.217+0000 7fc11b6fa700 1 -- 172.21.15.29:0/908539890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc114066450 msgr2=0x7fc11413c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:23.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.217+0000 7fc11b6fa700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc114066450 0x7fc11413c880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fc10800c440 tx=0x7fc108007db0).stop 2024-01-24T13:58:23.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.227+0000 7fc11b6fa700 1 -- 172.21.15.29:0/908539890 shutdown_connections 2024-01-24T13:58:23.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.227+0000 7fc11b6fa700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11413a6f0 0x7fc1140a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.227+0000 7fc11b6fa700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc0f805f150 0x7fc0f8061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.227+0000 7fc11b6fa700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc114066450 0x7fc11413c880 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.227+0000 7fc11b6fa700 1 --2- 172.21.15.29:0/908539890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1140bf870 0x7fc114168640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.227+0000 7fc11b6fa700 1 -- 172.21.15.29:0/908539890 >> 172.21.15.29:0/908539890 conn(0x7fc1140c8bd0 msgr2=0x7fc1141391e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.230+0000 7fc11b6fa700 1 -- 172.21.15.29:0/908539890 shutdown_connections 2024-01-24T13:58:23.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.230+0000 7fc11b6fa700 1 -- 172.21.15.29:0/908539890 wait complete. 2024-01-24T13:58:23.238 INFO:teuthology.orchestra.run.smithi029.stderr:set pool 5 allow_ec_overwrites to true 2024-01-24T13:58:23.252 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-01-24T13:58:23.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.385+0000 7fa4ee6be700 1 Processor -- start 2024-01-24T13:58:23.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.385+0000 7fa4ee6be700 1 -- start start 2024-01-24T13:58:23.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.386+0000 7fa4ee6be700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa4e814e530 0x7fa4e8155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:23.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.386+0000 7fa4ee6be700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa4e805e1f0 0x7fa4e80c0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:23.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.386+0000 7fa4ee6be700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e8158960 con 0x7fa4e814e910 2024-01-24T13:58:23.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.386+0000 7fa4ee6be700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e80c1b90 con 0x7fa4e814e530 2024-01-24T13:58:23.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.386+0000 7fa4ee6be700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e80c4760 con 0x7fa4e805e1f0 2024-01-24T13:58:23.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.386+0000 7fa4e7fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa4e814e530 0x7fa4e8155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:23.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.386+0000 7fa4e7fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa4e814e530 0x7fa4e8155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39342/0 (socket says 172.21.15.29:39342) 2024-01-24T13:58:23.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.386+0000 7fa4e7fff700 1 -- 172.21.15.29:0/1879549671 learned_addr learned my addr 172.21.15.29:0/1879549671 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.386+0000 7fa4e77fe700 1 --2- 172.21.15.29:0/1879549671 >> v2:172.21.15.163:3300/0 conn(0x7fa4e805e1f0 0x7fa4e80c0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.386+0000 7fa4e7fff700 1 -- 172.21.15.29:0/1879549671 >> v2:172.21.15.163:3300/0 conn(0x7fa4e805e1f0 msgr2=0x7fa4e80c0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.386+0000 7fa4e7fff700 1 --2- 172.21.15.29:0/1879549671 >> v2:172.21.15.163:3300/0 conn(0x7fa4e805e1f0 0x7fa4e80c0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.387+0000 7fa4e7fff700 1 -- 172.21.15.29:0/1879549671 >> v1:172.21.15.163:6789/0 conn(0x7fa4e814e910 legacy=0x7fa4e8062cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.387+0000 7fa4e7fff700 1 -- 172.21.15.29:0/1879549671 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e8155650 con 0x7fa4e814e530 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.387+0000 7fa4e7fff700 1 --2- 172.21.15.29:0/1879549671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e814e530 0x7fa4e8155160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa4e80c3ce0 tx=0x7fa4d80090f0).ready entity=mon.0 client_cookie=bbbb49809ac3100c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.387+0000 7fa4e6ffd700 1 -- 172.21.15.29:0/1879549671 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800b600 con 0x7fa4e814e530 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.387+0000 7fa4e6ffd700 1 -- 172.21.15.29:0/1879549671 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d800b760 con 0x7fa4e814e530 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.387+0000 7fa4e6ffd700 1 -- 172.21.15.29:0/1879549671 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800bae0 con 0x7fa4e814e530 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.387+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1879549671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e814e530 msgr2=0x7fa4e8155160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.387+0000 7fa4ee6be700 1 --2- 172.21.15.29:0/1879549671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e814e530 0x7fa4e8155160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa4e80c3ce0 tx=0x7fa4d80090f0).stop 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.387+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1879549671 shutdown_connections 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.387+0000 7fa4ee6be700 1 --2- 172.21.15.29:0/1879549671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e814e530 0x7fa4e8155160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.387+0000 7fa4ee6be700 1 --2- 172.21.15.29:0/1879549671 >> v2:172.21.15.163:3300/0 conn(0x7fa4e805e1f0 0x7fa4e80c0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.387+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1879549671 >> 172.21.15.29:0/1879549671 conn(0x7fa4e80c7d40 msgr2=0x7fa4e80c3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:23.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.388+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1879549671 shutdown_connections 2024-01-24T13:58:23.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.388+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1879549671 wait complete. 2024-01-24T13:58:23.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.389+0000 7fa4ee6be700 1 Processor -- start 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.389+0000 7fa4ee6be700 1 -- start start 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4ee6be700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e805e1f0 0x7fa4e80aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4ee6be700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4e814e530 0x7fa4e81648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4ee6be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4e814e910 0x7fa4e8156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4ee6be700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e8165810 con 0x7fa4e805e1f0 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4ee6be700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e8165250 con 0x7fa4e814e910 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4ee6be700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e8165530 con 0x7fa4e814e530 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4e7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e805e1f0 0x7fa4e80aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4e7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e805e1f0 0x7fa4e80aad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39354/0 (socket says 172.21.15.29:39354) 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4e7fff700 1 -- 172.21.15.29:0/1572145028 learned_addr learned my addr 172.21.15.29:0/1572145028 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4e77fe700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4e814e530 0x7fa4e81648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4ecc5b700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4e814e910 0x7fa4e8156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:23.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4e7fff700 1 -- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4e814e530 msgr2=0x7fa4e81648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4e7fff700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4e814e530 0x7fa4e81648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4e7fff700 1 -- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4e814e910 msgr2=0x7fa4e8156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4e7fff700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4e814e910 0x7fa4e8156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4e7fff700 1 -- 172.21.15.29:0/1572145028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d8009370 con 0x7fa4e805e1f0 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4e77fe700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4e814e530 0x7fa4e81648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4ecc5b700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4e814e910 0x7fa4e8156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.390+0000 7fa4e7fff700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e805e1f0 0x7fa4e80aad50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa4d8009130 tx=0x7fa4d8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.391+0000 7fa4cffff700 1 -- 172.21.15.29:0/1572145028 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800b8d0 con 0x7fa4e805e1f0 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.391+0000 7fa4cffff700 1 -- 172.21.15.29:0/1572145028 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d800ba30 con 0x7fa4e805e1f0 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.391+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1572145028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e80aa550 con 0x7fa4e805e1f0 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.391+0000 7fa4cffff700 1 -- 172.21.15.29:0/1572145028 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d801c670 con 0x7fa4e805e1f0 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.391+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1572145028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e80ab240 con 0x7fa4e805e1f0 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.392+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1572145028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e80aaa70 con 0x7fa4e805e1f0 2024-01-24T13:58:23.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.392+0000 7fa4cffff700 1 -- 172.21.15.29:0/1572145028 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa4d801c9e0 con 0x7fa4e805e1f0 2024-01-24T13:58:23.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.393+0000 7fa4cffff700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa4d405f150 0x7fa4d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:23.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.393+0000 7fa4cffff700 1 -- 172.21.15.29:0/1572145028 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6873+0+0 (secure 0 0 0) 0x7fa4d807bd10 con 0x7fa4e805e1f0 2024-01-24T13:58:23.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.393+0000 7fa4e77fe700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa4d405f150 0x7fa4d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:23.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.393+0000 7fa4e77fe700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa4d405f150 0x7fa4d4061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fa4dc006fd0 tx=0x7fa4dc008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:23.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.397+0000 7fa4cffff700 1 -- 172.21.15.29:0/1572145028 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa4e80aaa70 con 0x7fa4e805e1f0 2024-01-24T13:58:23.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.622+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1572145028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fa4e80475e0 con 0x7fa4e805e1f0 2024-01-24T13:58:23.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.627+0000 7fa4cffff700 1 -- 172.21.15.29:0/1572145028 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fa4d8020020 con 0x7fa4e805e1f0 2024-01-24T13:58:23.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.629+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa4d405f150 msgr2=0x7fa4d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:23.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.629+0000 7fa4ee6be700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa4d405f150 0x7fa4d4061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fa4dc006fd0 tx=0x7fa4dc008040).stop 2024-01-24T13:58:23.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.629+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1572145028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e805e1f0 msgr2=0x7fa4e80aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:23.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.629+0000 7fa4ee6be700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e805e1f0 0x7fa4e80aad50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa4d8009130 tx=0x7fa4d8017000).stop 2024-01-24T13:58:23.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.629+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1572145028 shutdown_connections 2024-01-24T13:58:23.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.629+0000 7fa4ee6be700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4e814e910 0x7fa4e8156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.629+0000 7fa4ee6be700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa4d405f150 0x7fa4d4061610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.629+0000 7fa4ee6be700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4e805e1f0 0x7fa4e80aad50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.629+0000 7fa4ee6be700 1 --2- 172.21.15.29:0/1572145028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4e814e530 0x7fa4e81648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.629+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1572145028 >> 172.21.15.29:0/1572145028 conn(0x7fa4e80c7d40 msgr2=0x7fa4e814bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:23.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.629+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1572145028 shutdown_connections 2024-01-24T13:58:23.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.629+0000 7fa4ee6be700 1 -- 172.21.15.29:0/1572145028 wait complete. 2024-01-24T13:58:23.631 INFO:teuthology.orchestra.run.smithi029.stderr:added data pool 5 to fsmap 2024-01-24T13:58:23.643 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:58:23.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.772+0000 7f8420c3a700 1 Processor -- start 2024-01-24T13:58:23.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.772+0000 7f8420c3a700 1 -- start start 2024-01-24T13:58:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f8420c3a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f841c134cc0 0x7f841c0c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f8420c3a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f841c066c00 0x7f841c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f8420c3a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841c068360 con 0x7f841c1350a0 2024-01-24T13:58:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f8420c3a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841c153360 con 0x7f841c066c00 2024-01-24T13:58:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f8420c3a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841c0c3820 con 0x7f841c134cc0 2024-01-24T13:58:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f841ad9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f841c066c00 0x7f841c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f841ad9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f841c066c00 0x7f841c152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51166/0 (socket says 172.21.15.29:51166) 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f841ad9d700 1 -- 172.21.15.29:0/1831305542 learned_addr learned my addr 172.21.15.29:0/1831305542 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f841a59c700 1 --2- 172.21.15.29:0/1831305542 >> v2:172.21.15.163:3300/0 conn(0x7f841c134cc0 0x7f841c0c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f841ad9d700 1 -- 172.21.15.29:0/1831305542 >> v2:172.21.15.163:3300/0 conn(0x7f841c134cc0 msgr2=0x7f841c0c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f841ad9d700 1 --2- 172.21.15.29:0/1831305542 >> v2:172.21.15.163:3300/0 conn(0x7f841c134cc0 0x7f841c0c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f841ad9d700 1 -- 172.21.15.29:0/1831305542 >> v1:172.21.15.163:6789/0 conn(0x7f841c1350a0 legacy=0x7f841c0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.773+0000 7f841ad9d700 1 -- 172.21.15.29:0/1831305542 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c067a30 con 0x7f841c066c00 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.774+0000 7f841ad9d700 1 --2- 172.21.15.29:0/1831305542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c066c00 0x7f841c152e70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f841c0aee60 tx=0x7f8414009840).ready entity=mon.0 client_cookie=bb0a4c59560eab6b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.774+0000 7f841959a700 1 -- 172.21.15.29:0/1831305542 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841400e070 con 0x7f841c066c00 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.774+0000 7f841959a700 1 -- 172.21.15.29:0/1831305542 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841400ba10 con 0x7f841c066c00 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.774+0000 7f841959a700 1 -- 172.21.15.29:0/1831305542 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841400bd90 con 0x7f841c066c00 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.774+0000 7f8420c3a700 1 -- 172.21.15.29:0/1831305542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c066c00 msgr2=0x7f841c152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.774+0000 7f8420c3a700 1 --2- 172.21.15.29:0/1831305542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c066c00 0x7f841c152e70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f841c0aee60 tx=0x7f8414009840).stop 2024-01-24T13:58:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.774+0000 7f8420c3a700 1 -- 172.21.15.29:0/1831305542 shutdown_connections 2024-01-24T13:58:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.774+0000 7f8420c3a700 1 --2- 172.21.15.29:0/1831305542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c066c00 0x7f841c152e70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.774+0000 7f8420c3a700 1 --2- 172.21.15.29:0/1831305542 >> v2:172.21.15.163:3300/0 conn(0x7f841c134cc0 0x7f841c0c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.774+0000 7f8420c3a700 1 -- 172.21.15.29:0/1831305542 >> 172.21.15.29:0/1831305542 conn(0x7f841c0c8bb0 msgr2=0x7f841c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.774+0000 7f8420c3a700 1 -- 172.21.15.29:0/1831305542 shutdown_connections 2024-01-24T13:58:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.774+0000 7f8420c3a700 1 -- 172.21.15.29:0/1831305542 wait complete. 2024-01-24T13:58:23.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.776+0000 7f8420c3a700 1 Processor -- start 2024-01-24T13:58:23.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.776+0000 7f8420c3a700 1 -- start start 2024-01-24T13:58:23.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f8420c3a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f841c066c00 0x7f841c134560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:23.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f8420c3a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f841c134cc0 0x7f841c13b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:23.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f8420c3a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c1350a0 0x7f841c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:23.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f8420c3a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c13ba40 con 0x7f841c1350a0 2024-01-24T13:58:23.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f8420c3a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c1374e0 con 0x7f841c066c00 2024-01-24T13:58:23.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f8420c3a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841c1389b0 con 0x7f841c134cc0 2024-01-24T13:58:23.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f841ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c1350a0 0x7f841c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:23.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f841ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c1350a0 0x7f841c13b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51174/0 (socket says 172.21.15.29:51174) 2024-01-24T13:58:23.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f841ad9d700 1 -- 172.21.15.29:0/1742087481 learned_addr learned my addr 172.21.15.29:0/1742087481 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f8419d9b700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f841c134cc0 0x7f841c13b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f841a59c700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f841c066c00 0x7f841c134560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f841ad9d700 1 -- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f841c134cc0 msgr2=0x7f841c13b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f841ad9d700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f841c134cc0 0x7f841c13b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f841ad9d700 1 -- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f841c066c00 msgr2=0x7f841c134560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f841ad9d700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f841c066c00 0x7f841c134560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f841ad9d700 1 -- 172.21.15.29:0/1742087481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8414009ae0 con 0x7f841c1350a0 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f841ad9d700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c1350a0 0x7f841c13b550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f8414010000 tx=0x7f841c0682d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f840a7fc700 1 -- 172.21.15.29:0/1742087481 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414013e60 con 0x7f841c1350a0 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f840a7fc700 1 -- 172.21.15.29:0/1742087481 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84140092e0 con 0x7f841c1350a0 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f840a7fc700 1 -- 172.21.15.29:0/1742087481 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841401d640 con 0x7f841c1350a0 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f8420c3a700 1 -- 172.21.15.29:0/1742087481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841c137730 con 0x7f841c1350a0 2024-01-24T13:58:23.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.777+0000 7f8420c3a700 1 -- 172.21.15.29:0/1742087481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c137cf0 con 0x7f841c1350a0 2024-01-24T13:58:23.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.779+0000 7f8420c3a700 1 -- 172.21.15.29:0/1742087481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841c0475e0 con 0x7f841c1350a0 2024-01-24T13:58:23.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.779+0000 7f840a7fc700 1 -- 172.21.15.29:0/1742087481 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f841401d9b0 con 0x7f841c1350a0 2024-01-24T13:58:23.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.779+0000 7f840a7fc700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f83fc05f150 0x7f83fc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:23.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.779+0000 7f840a7fc700 1 -- 172.21.15.29:0/1742087481 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f841407bec0 con 0x7f841c1350a0 2024-01-24T13:58:23.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.780+0000 7f841a59c700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f83fc05f150 0x7f83fc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:23.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.780+0000 7f841a59c700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f83fc05f150 0x7f83fc061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f840c00bbc0 tx=0x7f840c008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:23.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:23.784+0000 7f840a7fc700 1 -- 172.21.15.29:0/1742087481 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f841404ce80 con 0x7f841c1350a0 2024-01-24T13:58:24.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.009+0000 7f8420c3a700 1 -- 172.21.15.29:0/1742087481 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f841c0475e0 con 0x7f841c1350a0 2024-01-24T13:58:24.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.010+0000 7f840a7fc700 1 -- 172.21.15.29:0/1742087481 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4491 (secure 0 0 0) 0x7f84140504a0 con 0x7f841c1350a0 2024-01-24T13:58:24.011 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:24.011 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2038321939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2038321939},{"type":"v1","addr":"172.21.15.163:6836","nonce":2038321939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3675672874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3675672874},{"type":"v1","addr":"172.21.15.163:6837","nonce":3675672874}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1532435957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532435957},{"type":"v1","addr":"172.21.15.29:6834","nonce":1532435957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:58:19.190134+0000","modified":"2024-01-24T13:58:23.623801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1328130656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1328130656},{"type":"v1","addr":"172.21.15.29:6835","nonce":1328130656}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-01-24T13:58:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.012+0000 7f8420c3a700 1 -- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f83fc05f150 msgr2=0x7f83fc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.012+0000 7f8420c3a700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f83fc05f150 0x7f83fc061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f840c00bbc0 tx=0x7f840c008040).stop 2024-01-24T13:58:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.012+0000 7f8420c3a700 1 -- 172.21.15.29:0/1742087481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c1350a0 msgr2=0x7f841c13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.012+0000 7f8420c3a700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c1350a0 0x7f841c13b550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f8414010000 tx=0x7f841c0682d0).stop 2024-01-24T13:58:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.012+0000 7f8420c3a700 1 -- 172.21.15.29:0/1742087481 shutdown_connections 2024-01-24T13:58:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.012+0000 7f8420c3a700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f83fc05f150 0x7f83fc061610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.012+0000 7f8420c3a700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841c1350a0 0x7f841c13b550 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.012+0000 7f8420c3a700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f841c066c00 0x7f841c134560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.012+0000 7f8420c3a700 1 --2- 172.21.15.29:0/1742087481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f841c134cc0 0x7f841c13b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.012+0000 7f8420c3a700 1 -- 172.21.15.29:0/1742087481 >> 172.21.15.29:0/1742087481 conn(0x7f841c0c8bb0 msgr2=0x7f841c0c76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.012+0000 7f8420c3a700 1 -- 172.21.15.29:0/1742087481 shutdown_connections 2024-01-24T13:58:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.012+0000 7f8420c3a700 1 -- 172.21.15.29:0/1742087481 wait complete. 2024-01-24T13:58:24.014 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 7 2024-01-24T13:58:24.026 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:24.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.155+0000 7ff8b6e94700 1 Processor -- start 2024-01-24T13:58:24.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.155+0000 7ff8b6e94700 1 -- start start 2024-01-24T13:58:24.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.156+0000 7ff8b6e94700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff8b0063fd0 0x7ff8b0151570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.156+0000 7ff8b6e94700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b01522c0 0x7ff8b00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.156+0000 7ff8b6e94700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b014bb00 con 0x7ff8b0151ee0 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.156+0000 7ff8b6e94700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b006aa30 con 0x7ff8b0063fd0 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.156+0000 7ff8b6e94700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b0069ba0 con 0x7ff8b01522c0 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.156+0000 7ff8b5431700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b01522c0 0x7ff8b00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.156+0000 7ff8affff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff8b0063fd0 0x7ff8b0151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.156+0000 7ff8b5431700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b01522c0 0x7ff8b00696b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:47194/0 (socket says 172.21.15.29:47194) 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.156+0000 7ff8affff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff8b0063fd0 0x7ff8b0151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:55592/0 (socket says 172.21.15.29:55592) 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.156+0000 7ff8b5431700 1 -- 172.21.15.29:0/1153195877 learned_addr learned my addr 172.21.15.29:0/1153195877 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.157+0000 7ff8affff700 1 -- 172.21.15.29:0/1153195877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b01522c0 msgr2=0x7ff8b00696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.157+0000 7ff8affff700 1 --2- 172.21.15.29:0/1153195877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b01522c0 0x7ff8b00696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.157+0000 7ff8affff700 1 -- 172.21.15.29:0/1153195877 >> v1:172.21.15.163:6789/0 conn(0x7ff8b0151ee0 legacy=0x7ff8b00638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.157+0000 7ff8affff700 1 -- 172.21.15.29:0/1153195877 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b00c1870 con 0x7ff8b0063fd0 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.157+0000 7ff8af7fe700 1 -- 172.21.15.29:0/1153195877 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560162258 0 0) 0x7ff8b014bb00 con 0x7ff8b0151ee0 2024-01-24T13:58:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.157+0000 7ff8affff700 1 --2- 172.21.15.29:0/1153195877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b0063fd0 0x7ff8b0151570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff898009760 tx=0x7ff898009060).ready entity=mon.1 client_cookie=d5f430454d4cf08b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:24.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.157+0000 7ff8af7fe700 1 -- 172.21.15.29:0/1153195877 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800f510 con 0x7ff8b0063fd0 2024-01-24T13:58:24.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.157+0000 7ff8af7fe700 1 -- 172.21.15.29:0/1153195877 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89800f670 con 0x7ff8b0063fd0 2024-01-24T13:58:24.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.157+0000 7ff8af7fe700 1 -- 172.21.15.29:0/1153195877 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800f9f0 con 0x7ff8b0063fd0 2024-01-24T13:58:24.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.158+0000 7ff8b6e94700 1 -- 172.21.15.29:0/1153195877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b0063fd0 msgr2=0x7ff8b0151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.158+0000 7ff8b6e94700 1 --2- 172.21.15.29:0/1153195877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b0063fd0 0x7ff8b0151570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff898009760 tx=0x7ff898009060).stop 2024-01-24T13:58:24.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.158+0000 7ff8b6e94700 1 -- 172.21.15.29:0/1153195877 shutdown_connections 2024-01-24T13:58:24.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.158+0000 7ff8b6e94700 1 --2- 172.21.15.29:0/1153195877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b01522c0 0x7ff8b00696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.158+0000 7ff8b6e94700 1 --2- 172.21.15.29:0/1153195877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b0063fd0 0x7ff8b0151570 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.158+0000 7ff8b6e94700 1 -- 172.21.15.29:0/1153195877 >> 172.21.15.29:0/1153195877 conn(0x7ff8b0064aa0 msgr2=0x7ff8b0064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:24.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.158+0000 7ff8b6e94700 1 -- 172.21.15.29:0/1153195877 shutdown_connections 2024-01-24T13:58:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.158+0000 7ff8b6e94700 1 -- 172.21.15.29:0/1153195877 wait complete. 2024-01-24T13:58:24.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.159+0000 7ff8b6e94700 1 Processor -- start 2024-01-24T13:58:24.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.160+0000 7ff8b6e94700 1 -- start start 2024-01-24T13:58:24.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b6e94700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b0063fd0 0x7ff8b0166320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b6e94700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b0151ee0 0x7ff8b00bfff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b6e94700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b01522c0 0x7ff8b0161a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b6e94700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b00c14e0 con 0x7ff8b0063fd0 2024-01-24T13:58:24.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b6e94700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b00c08d0 con 0x7ff8b01522c0 2024-01-24T13:58:24.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b6e94700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b00c0bb0 con 0x7ff8b0151ee0 2024-01-24T13:58:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b4c30700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b0063fd0 0x7ff8b0166320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b4c30700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b0063fd0 0x7ff8b0166320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51184/0 (socket says 172.21.15.29:51184) 2024-01-24T13:58:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b4c30700 1 -- 172.21.15.29:0/3470536297 learned_addr learned my addr 172.21.15.29:0/3470536297 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b5431700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b01522c0 0x7ff8b0161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8affff700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b0151ee0 0x7ff8b00bfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b4c30700 1 -- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b0151ee0 msgr2=0x7ff8b00bfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b4c30700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b0151ee0 0x7ff8b00bfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b4c30700 1 -- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b01522c0 msgr2=0x7ff8b0161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b4c30700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b01522c0 0x7ff8b0161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b4c30700 1 -- 172.21.15.29:0/3470536297 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8980092a0 con 0x7ff8b0063fd0 2024-01-24T13:58:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b4c30700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b0063fd0 0x7ff8b0166320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7ff8a0001eb0 tx=0x7ff8a0007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:24.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8acff9700 1 -- 172.21.15.29:0/3470536297 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a000ce60 con 0x7ff8b0063fd0 2024-01-24T13:58:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8acff9700 1 -- 172.21.15.29:0/3470536297 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a000f470 con 0x7ff8b0063fd0 2024-01-24T13:58:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.161+0000 7ff8b6e94700 1 -- 172.21.15.29:0/3470536297 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b00c0e90 con 0x7ff8b0063fd0 2024-01-24T13:58:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.162+0000 7ff8acff9700 1 -- 172.21.15.29:0/3470536297 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a000f810 con 0x7ff8b0063fd0 2024-01-24T13:58:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.162+0000 7ff8b6e94700 1 -- 172.21.15.29:0/3470536297 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b00c04e0 con 0x7ff8b0063fd0 2024-01-24T13:58:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.163+0000 7ff8b6e94700 1 -- 172.21.15.29:0/3470536297 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b00475e0 con 0x7ff8b0063fd0 2024-01-24T13:58:24.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.163+0000 7ff8acff9700 1 -- 172.21.15.29:0/3470536297 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff8a000fb80 con 0x7ff8b0063fd0 2024-01-24T13:58:24.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.163+0000 7ff8acff9700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff89005f150 0x7ff890061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.163+0000 7ff8acff9700 1 -- 172.21.15.29:0/3470536297 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7ff8a0079a10 con 0x7ff8b0063fd0 2024-01-24T13:58:24.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.164+0000 7ff8affff700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff89005f150 0x7ff890061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.164+0000 7ff8affff700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff89005f150 0x7ff890061610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff898009760 tx=0x7ff898017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:24.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.168+0000 7ff8acff9700 1 -- 172.21.15.29:0/3470536297 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff8a004a9d0 con 0x7ff8b0063fd0 2024-01-24T13:58:24.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.350+0000 7ff8b6e94700 1 -- 172.21.15.29:0/3470536297 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8b013afe0 con 0x7ff8b0063fd0 2024-01-24T13:58:24.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.351+0000 7ff8acff9700 1 -- 172.21.15.29:0/3470536297 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7ff8a004dff0 con 0x7ff8b0063fd0 2024-01-24T13:58:24.352 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:24.353 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:24.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.353+0000 7ff8b6e94700 1 -- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff89005f150 msgr2=0x7ff890061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.353+0000 7ff8b6e94700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff89005f150 0x7ff890061610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff898009760 tx=0x7ff898017000).stop 2024-01-24T13:58:24.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.353+0000 7ff8b6e94700 1 -- 172.21.15.29:0/3470536297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b0063fd0 msgr2=0x7ff8b0166320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.353+0000 7ff8b6e94700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b0063fd0 0x7ff8b0166320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7ff8a0001eb0 tx=0x7ff8a0007c70).stop 2024-01-24T13:58:24.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.353+0000 7ff8b6e94700 1 -- 172.21.15.29:0/3470536297 shutdown_connections 2024-01-24T13:58:24.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.353+0000 7ff8b6e94700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b01522c0 0x7ff8b0161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.353+0000 7ff8b6e94700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff89005f150 0x7ff890061610 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.353+0000 7ff8b6e94700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b0063fd0 0x7ff8b0166320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.353+0000 7ff8b6e94700 1 --2- 172.21.15.29:0/3470536297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b0151ee0 0x7ff8b00bfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.353+0000 7ff8b6e94700 1 -- 172.21.15.29:0/3470536297 >> 172.21.15.29:0/3470536297 conn(0x7ff8b0064aa0 msgr2=0x7ff8b014afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:24.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.354+0000 7ff8b6e94700 1 -- 172.21.15.29:0/3470536297 shutdown_connections 2024-01-24T13:58:24.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.354+0000 7ff8b6e94700 1 -- 172.21.15.29:0/3470536297 wait complete. 2024-01-24T13:58:24.368 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:24.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.496+0000 7f4f0ca80700 1 Processor -- start 2024-01-24T13:58:24.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.496+0000 7f4f0ca80700 1 -- start start 2024-01-24T13:58:24.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.497+0000 7f4f0ca80700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4f08138330 0x7f4f08138710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0ca80700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f080bca50 0x7f4f080be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0ca80700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f08065770 con 0x7f4f080627b0 2024-01-24T13:58:24.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0ca80700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f08069cf0 con 0x7f4f08138330 2024-01-24T13:58:24.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0ca80700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08069f60 con 0x7f4f080bca50 2024-01-24T13:58:24.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4f08138330 0x7f4f08138710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4f08138330 0x7f4f08138710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51196/0 (socket says 172.21.15.29:51196) 2024-01-24T13:58:24.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0659c700 1 -- 172.21.15.29:0/687605088 learned_addr learned my addr 172.21.15.29:0/687605088 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:24.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f05d9b700 1 --2- 172.21.15.29:0/687605088 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f080bca50 0x7f4f080be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0659c700 1 -- 172.21.15.29:0/687605088 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f080bca50 msgr2=0x7f4f080be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:24.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0659c700 1 --2- 172.21.15.29:0/687605088 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f080bca50 0x7f4f080be3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0659c700 1 -- 172.21.15.29:0/687605088 >> v1:172.21.15.163:6789/0 conn(0x7f4f080627b0 legacy=0x7f4f08155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0659c700 1 -- 172.21.15.29:0/687605088 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0806a240 con 0x7f4f08138330 2024-01-24T13:58:24.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0659c700 1 --2- 172.21.15.29:0/687605088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f08138330 0x7f4f08138710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f4ef8006fd0 tx=0x7f4ef80090f0).ready entity=mon.0 client_cookie=a51bedfe84debd9e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:24.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0559a700 1 -- 172.21.15.29:0/687605088 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800b580 con 0x7f4f08138330 2024-01-24T13:58:24.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0559a700 1 -- 172.21.15.29:0/687605088 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef800b6e0 con 0x7f4f08138330 2024-01-24T13:58:24.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.498+0000 7f4f0559a700 1 -- 172.21.15.29:0/687605088 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800ba60 con 0x7f4f08138330 2024-01-24T13:58:24.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.499+0000 7f4f0ca80700 1 -- 172.21.15.29:0/687605088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f08138330 msgr2=0x7f4f08138710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.499+0000 7f4f0ca80700 1 --2- 172.21.15.29:0/687605088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f08138330 0x7f4f08138710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f4ef8006fd0 tx=0x7f4ef80090f0).stop 2024-01-24T13:58:24.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.499+0000 7f4f0ca80700 1 -- 172.21.15.29:0/687605088 shutdown_connections 2024-01-24T13:58:24.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.499+0000 7f4f0ca80700 1 --2- 172.21.15.29:0/687605088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f08138330 0x7f4f08138710 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.499+0000 7f4f0ca80700 1 --2- 172.21.15.29:0/687605088 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f080bca50 0x7f4f080be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.499+0000 7f4f0ca80700 1 -- 172.21.15.29:0/687605088 >> 172.21.15.29:0/687605088 conn(0x7f4f080c8580 msgr2=0x7f4f080c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:24.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.499+0000 7f4f0ca80700 1 -- 172.21.15.29:0/687605088 shutdown_connections 2024-01-24T13:58:24.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.499+0000 7f4f0ca80700 1 -- 172.21.15.29:0/687605088 wait complete. 2024-01-24T13:58:24.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.500+0000 7f4f0ca80700 1 Processor -- start 2024-01-24T13:58:24.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.500+0000 7f4f0ca80700 1 -- start start 2024-01-24T13:58:24.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.501+0000 7f4f0ca80700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f080627b0 0x7f4f081552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.501+0000 7f4f0ca80700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f080bca50 0x7f4f08137320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.501+0000 7f4f0ca80700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f08138330 0x7f4f08156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.501+0000 7f4f0ca80700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f080aaaa0 con 0x7f4f080627b0 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.501+0000 7f4f0ca80700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f080ab060 con 0x7f4f08138330 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.501+0000 7f4f0ca80700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08155c60 con 0x7f4f080bca50 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.501+0000 7f4f0659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f080627b0 0x7f4f081552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.501+0000 7f4f0659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f080627b0 0x7f4f081552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51198/0 (socket says 172.21.15.29:51198) 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.501+0000 7f4f0659c700 1 -- 172.21.15.29:0/3280393476 learned_addr learned my addr 172.21.15.29:0/3280393476 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.501+0000 7f4f05d9b700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f080bca50 0x7f4f08137320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.501+0000 7f4f06d9d700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f08138330 0x7f4f08156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.502+0000 7f4f0659c700 1 -- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f080bca50 msgr2=0x7f4f08137320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.502+0000 7f4f0659c700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f080bca50 0x7f4f08137320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.502+0000 7f4f0659c700 1 -- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f08138330 msgr2=0x7f4f08156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.502+0000 7f4f0659c700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f08138330 0x7f4f08156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.502+0000 7f4f0659c700 1 -- 172.21.15.29:0/3280393476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef8009370 con 0x7f4f080627b0 2024-01-24T13:58:24.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.502+0000 7f4f05d9b700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f080bca50 0x7f4f08137320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:24.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.502+0000 7f4f0659c700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f080627b0 0x7f4f081552f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f4ef8009130 tx=0x7f4ef8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:24.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.502+0000 7f4ef67fc700 1 -- 172.21.15.29:0/3280393476 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8000bc0 con 0x7f4f080627b0 2024-01-24T13:58:24.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.502+0000 7f4ef67fc700 1 -- 172.21.15.29:0/3280393476 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef800b850 con 0x7f4f080627b0 2024-01-24T13:58:24.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.502+0000 7f4f0ca80700 1 -- 172.21.15.29:0/3280393476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f08155f40 con 0x7f4f080627b0 2024-01-24T13:58:24.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.502+0000 7f4ef67fc700 1 -- 172.21.15.29:0/3280393476 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8000d20 con 0x7f4f080627b0 2024-01-24T13:58:24.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.502+0000 7f4f0ca80700 1 -- 172.21.15.29:0/3280393476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f081557e0 con 0x7f4f080627b0 2024-01-24T13:58:24.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.503+0000 7f4f0ca80700 1 -- 172.21.15.29:0/3280393476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f08156100 con 0x7f4f080627b0 2024-01-24T13:58:24.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.504+0000 7f4ef67fc700 1 -- 172.21.15.29:0/3280393476 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4ef801bb00 con 0x7f4f080627b0 2024-01-24T13:58:24.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.504+0000 7f4ef67fc700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ee805f120 0x7f4ee80615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.504+0000 7f4ef67fc700 1 -- 172.21.15.29:0/3280393476 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f4ef807bec0 con 0x7f4f080627b0 2024-01-24T13:58:24.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.504+0000 7f4f05d9b700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ee805f120 0x7f4ee80615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.505+0000 7f4f05d9b700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ee805f120 0x7f4ee80615e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f4f080acc40 tx=0x7f4efc009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:24.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.508+0000 7f4ef67fc700 1 -- 172.21.15.29:0/3280393476 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4f08156100 con 0x7f4f080627b0 2024-01-24T13:58:24.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.693+0000 7f4f0ca80700 1 -- 172.21.15.29:0/3280393476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f08156100 con 0x7f4f080627b0 2024-01-24T13:58:24.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.694+0000 7f4ef67fc700 1 -- 172.21.15.29:0/3280393476 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7f4f08156100 con 0x7f4f080627b0 2024-01-24T13:58:24.694 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:24.694 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:24.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.695+0000 7f4f0ca80700 1 -- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ee805f120 msgr2=0x7f4ee80615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.695+0000 7f4f0ca80700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ee805f120 0x7f4ee80615e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f4f080acc40 tx=0x7f4efc009060).stop 2024-01-24T13:58:24.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.695+0000 7f4f0ca80700 1 -- 172.21.15.29:0/3280393476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f080627b0 msgr2=0x7f4f081552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.695+0000 7f4f0ca80700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f080627b0 0x7f4f081552f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f4ef8009130 tx=0x7f4ef8017000).stop 2024-01-24T13:58:24.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.696+0000 7f4f0ca80700 1 -- 172.21.15.29:0/3280393476 shutdown_connections 2024-01-24T13:58:24.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.696+0000 7f4f0ca80700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f08138330 0x7f4f08156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.696+0000 7f4f0ca80700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ee805f120 0x7f4ee80615e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.696+0000 7f4f0ca80700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f080627b0 0x7f4f081552f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.696+0000 7f4f0ca80700 1 --2- 172.21.15.29:0/3280393476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f080bca50 0x7f4f08137320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.696+0000 7f4f0ca80700 1 -- 172.21.15.29:0/3280393476 >> 172.21.15.29:0/3280393476 conn(0x7f4f080c8580 msgr2=0x7f4f080a7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:24.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.696+0000 7f4f0ca80700 1 -- 172.21.15.29:0/3280393476 shutdown_connections 2024-01-24T13:58:24.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.696+0000 7f4f0ca80700 1 -- 172.21.15.29:0/3280393476 wait complete. 2024-01-24T13:58:24.723 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:58:24.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.852+0000 7f4408945700 1 Processor -- start 2024-01-24T13:58:24.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.852+0000 7f4408945700 1 -- start start 2024-01-24T13:58:24.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.853+0000 7f4408945700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4404150d40 0x7f4404069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.853+0000 7f4408945700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4404065ec0 0x7f4404154930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f4408945700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44040c2ce0 con 0x7f4404154e20 2024-01-24T13:58:24.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f4408945700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4404155260 con 0x7f4404150d40 2024-01-24T13:58:24.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f4408945700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4404069890 con 0x7f4404065ec0 2024-01-24T13:58:24.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f440259c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4404150d40 0x7f4404069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f440259c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4404150d40 0x7f4404069240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51200/0 (socket says 172.21.15.29:51200) 2024-01-24T13:58:24.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f440259c700 1 -- 172.21.15.29:0/2625333424 learned_addr learned my addr 172.21.15.29:0/2625333424 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:24.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f4402d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4404154e20 0x7f4404151120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33630/0 (socket says 172.21.15.29:33630) 2024-01-24T13:58:24.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f4401d9b700 1 --2- 172.21.15.29:0/2625333424 >> v2:172.21.15.163:3300/0 conn(0x7f4404065ec0 0x7f4404154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f440259c700 1 -- 172.21.15.29:0/2625333424 >> v2:172.21.15.163:3300/0 conn(0x7f4404065ec0 msgr2=0x7f4404154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f440259c700 1 --2- 172.21.15.29:0/2625333424 >> v2:172.21.15.163:3300/0 conn(0x7f4404065ec0 0x7f4404154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f440259c700 1 -- 172.21.15.29:0/2625333424 >> v1:172.21.15.29:6789/0 conn(0x7f4404154e20 legacy=0x7f4404151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f440259c700 1 -- 172.21.15.29:0/2625333424 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4404069b70 con 0x7f4404150d40 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f440159a700 1 -- 172.21.15.29:0/2625333424 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033205370 0 0) 0x7f44040c2ce0 con 0x7f4404154e20 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f440259c700 1 --2- 172.21.15.29:0/2625333424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4404150d40 0x7f4404069240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4404068cd0 tx=0x7f43f40090f0).ready entity=mon.0 client_cookie=f560e86b84f5ff68 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f440159a700 1 -- 172.21.15.29:0/2625333424 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f400b520 con 0x7f4404150d40 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f440159a700 1 -- 172.21.15.29:0/2625333424 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f400b680 con 0x7f4404150d40 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.854+0000 7f440159a700 1 -- 172.21.15.29:0/2625333424 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f400ba00 con 0x7f4404150d40 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.855+0000 7f4408945700 1 -- 172.21.15.29:0/2625333424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4404150d40 msgr2=0x7f4404069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.855+0000 7f4408945700 1 --2- 172.21.15.29:0/2625333424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4404150d40 0x7f4404069240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4404068cd0 tx=0x7f43f40090f0).stop 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.855+0000 7f4408945700 1 -- 172.21.15.29:0/2625333424 shutdown_connections 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.855+0000 7f4408945700 1 --2- 172.21.15.29:0/2625333424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4404150d40 0x7f4404069240 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.855+0000 7f4408945700 1 --2- 172.21.15.29:0/2625333424 >> v2:172.21.15.163:3300/0 conn(0x7f4404065ec0 0x7f4404154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.855+0000 7f4408945700 1 -- 172.21.15.29:0/2625333424 >> 172.21.15.29:0/2625333424 conn(0x7f44040c8580 msgr2=0x7f44040c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:24.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.855+0000 7f4408945700 1 -- 172.21.15.29:0/2625333424 shutdown_connections 2024-01-24T13:58:24.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.855+0000 7f4408945700 1 -- 172.21.15.29:0/2625333424 wait complete. 2024-01-24T13:58:24.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.856+0000 7f4408945700 1 Processor -- start 2024-01-24T13:58:24.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.856+0000 7f4408945700 1 -- start start 2024-01-24T13:58:24.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.857+0000 7f4408945700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4404065ec0 0x7f4404166510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.857+0000 7f4408945700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4404150d40 0x7f4404154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.857+0000 7f4408945700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4404154e20 0x7f440415eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.857+0000 7f4408945700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440415fd40 con 0x7f4404154e20 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.857+0000 7f4408945700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4404166080 con 0x7f4404150d40 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.857+0000 7f4408945700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44040ace20 con 0x7f4404065ec0 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.857+0000 7f4402d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4404154e20 0x7f440415eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.857+0000 7f4402d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4404154e20 0x7f440415eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51208/0 (socket says 172.21.15.29:51208) 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.857+0000 7f4402d9d700 1 -- 172.21.15.29:0/857385506 learned_addr learned my addr 172.21.15.29:0/857385506 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f440259c700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4404065ec0 0x7f4404166510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f4401d9b700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4404150d40 0x7f4404154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f4402d9d700 1 -- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4404065ec0 msgr2=0x7f4404166510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f4402d9d700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4404065ec0 0x7f4404166510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f4402d9d700 1 -- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4404150d40 msgr2=0x7f4404154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f4402d9d700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4404150d40 0x7f4404154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f4402d9d700 1 -- 172.21.15.29:0/857385506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f4009370 con 0x7f4404154e20 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f4402d9d700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4404154e20 0x7f440415eef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f43fc00c810 tx=0x7f43fc007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f43f27fc700 1 -- 172.21.15.29:0/857385506 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc005c70 con 0x7f4404154e20 2024-01-24T13:58:24.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f43f27fc700 1 -- 172.21.15.29:0/857385506 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc005df0 con 0x7f4404154e20 2024-01-24T13:58:24.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f4408945700 1 -- 172.21.15.29:0/857385506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4404154580 con 0x7f4404154e20 2024-01-24T13:58:24.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f43f27fc700 1 -- 172.21.15.29:0/857385506 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc007360 con 0x7f4404154e20 2024-01-24T13:58:24.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.858+0000 7f4408945700 1 -- 172.21.15.29:0/857385506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4404165a30 con 0x7f4404154e20 2024-01-24T13:58:24.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.859+0000 7f4408945700 1 -- 172.21.15.29:0/857385506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44041661f0 con 0x7f4404154e20 2024-01-24T13:58:24.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.860+0000 7f43f27fc700 1 -- 172.21.15.29:0/857385506 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f43fc007750 con 0x7f4404154e20 2024-01-24T13:58:24.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.860+0000 7f43f27fc700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43e405f120 0x7f43e40615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:24.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.860+0000 7f43f27fc700 1 -- 172.21.15.29:0/857385506 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f43fc07a4f0 con 0x7f4404154e20 2024-01-24T13:58:24.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.860+0000 7f440259c700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43e405f120 0x7f43e40615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:24.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.861+0000 7f440259c700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43e405f120 0x7f43e40615e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f43f4009130 tx=0x7f43f4017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:24.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:24.865+0000 7f43f27fc700 1 -- 172.21.15.29:0/857385506 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f43fc04b4b0 con 0x7f4404154e20 2024-01-24T13:58:24.878 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-01-24T13:58:24.878+0000 7fe714eb4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T13:58:25.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.091+0000 7f4408945700 1 -- 172.21.15.29:0/857385506 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44040475e0 con 0x7f4404154e20 2024-01-24T13:58:25.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.092+0000 7f43f27fc700 1 -- 172.21.15.29:0/857385506 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4491 (secure 0 0 0) 0x7f43fc04ead0 con 0x7f4404154e20 2024-01-24T13:58:25.092 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:25.092 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2038321939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2038321939},{"type":"v1","addr":"172.21.15.163:6836","nonce":2038321939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3675672874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3675672874},{"type":"v1","addr":"172.21.15.163:6837","nonce":3675672874}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1532435957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532435957},{"type":"v1","addr":"172.21.15.29:6834","nonce":1532435957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:58:19.190134+0000","modified":"2024-01-24T13:58:23.623801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1328130656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1328130656},{"type":"v1","addr":"172.21.15.29:6835","nonce":1328130656}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-01-24T13:58:25.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.094+0000 7f4408945700 1 -- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43e405f120 msgr2=0x7f43e40615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.094+0000 7f4408945700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43e405f120 0x7f43e40615e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f43f4009130 tx=0x7f43f4017000).stop 2024-01-24T13:58:25.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.094+0000 7f4408945700 1 -- 172.21.15.29:0/857385506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4404154e20 msgr2=0x7f440415eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.094+0000 7f4408945700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4404154e20 0x7f440415eef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f43fc00c810 tx=0x7f43fc007f10).stop 2024-01-24T13:58:25.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.094+0000 7f4408945700 1 -- 172.21.15.29:0/857385506 shutdown_connections 2024-01-24T13:58:25.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.094+0000 7f4408945700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43e405f120 0x7f43e40615e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.094+0000 7f4408945700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4404154e20 0x7f440415eef0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.094+0000 7f4408945700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4404065ec0 0x7f4404166510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.094+0000 7f4408945700 1 --2- 172.21.15.29:0/857385506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4404150d40 0x7f4404154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.094+0000 7f4408945700 1 -- 172.21.15.29:0/857385506 >> 172.21.15.29:0/857385506 conn(0x7f44040c8580 msgr2=0x7f44040627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:25.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.094+0000 7f4408945700 1 -- 172.21.15.29:0/857385506 shutdown_connections 2024-01-24T13:58:25.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.094+0000 7f4408945700 1 -- 172.21.15.29:0/857385506 wait complete. 2024-01-24T13:58:25.096 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 7 2024-01-24T13:58:25.108 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:25.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.238+0000 7fd8b7588700 1 Processor -- start 2024-01-24T13:58:25.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.238+0000 7fd8b7588700 1 -- start start 2024-01-24T13:58:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.239+0000 7fd8b7588700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0062970 0x7fd8b0062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.239+0000 7fd8b7588700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b00bcf60 con 0x7fd8b0158600 2024-01-24T13:58:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.239+0000 7fd8b7588700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b00a6ec0 con 0x7fd8b0066b40 2024-01-24T13:58:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.239+0000 7fd8b7588700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00bdba0 con 0x7fd8b0062970 2024-01-24T13:58:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.239+0000 7fd8b5b25700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd8b0158600 0x7fd8b0154d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33634/0 (socket says 172.21.15.29:33634) 2024-01-24T13:58:25.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.239+0000 7fd8b5b25700 1 -- 172.21.15.29:0/2509391599 learned_addr learned my addr 172.21.15.29:0/2509391599 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.239+0000 7fd8b5324700 1 --2- 172.21.15.29:0/2509391599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0062970 0x7fd8b0062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.239+0000 7fd8a7fff700 1 -- 172.21.15.29:0/2509391599 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751750566 0 0) 0x7fd8b00bcf60 con 0x7fd8b0158600 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.239+0000 7fd8a7fff700 1 -- 172.21.15.29:0/2509391599 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd89c0033f0 con 0x7fd8b0158600 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.240+0000 7fd8a7fff700 1 -- 172.21.15.29:0/2509391599 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684762119 0 0) 0x7fd8b00a6ec0 con 0x7fd8b0066b40 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.240+0000 7fd8a7fff700 1 -- 172.21.15.29:0/2509391599 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b00bcf60 con 0x7fd8b0066b40 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.240+0000 7fd8a7fff700 1 -- 172.21.15.29:0/2509391599 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2534775673 0 0) 0x7fd89c0033f0 con 0x7fd8b0158600 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.240+0000 7fd8b5324700 1 -- 172.21.15.29:0/2509391599 >> v1:172.21.15.163:6789/0 conn(0x7fd8b0066b40 legacy=0x7fd8b00c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.240+0000 7fd8b5324700 1 -- 172.21.15.29:0/2509391599 >> v1:172.21.15.29:6789/0 conn(0x7fd8b0158600 legacy=0x7fd8b0154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.240+0000 7fd8b5324700 1 -- 172.21.15.29:0/2509391599 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b00a71a0 con 0x7fd8b0062970 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.240+0000 7fd8b5324700 1 --2- 172.21.15.29:0/2509391599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0062970 0x7fd8b0062d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fd8ac009790 tx=0x7fd8ac009060).ready entity=mon.2 client_cookie=a7eff63e5661a15f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.240+0000 7fd8a7fff700 1 -- 172.21.15.29:0/2509391599 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80017f0 con 0x7fd8b0062970 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.240+0000 7fd8a7fff700 1 -- 172.21.15.29:0/2509391599 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac011400 con 0x7fd8b0062970 2024-01-24T13:58:25.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.240+0000 7fd8a7fff700 1 -- 172.21.15.29:0/2509391599 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac011780 con 0x7fd8b0062970 2024-01-24T13:58:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.241+0000 7fd8b7588700 1 -- 172.21.15.29:0/2509391599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0062970 msgr2=0x7fd8b0062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.241+0000 7fd8b7588700 1 --2- 172.21.15.29:0/2509391599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0062970 0x7fd8b0062d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fd8ac009790 tx=0x7fd8ac009060).stop 2024-01-24T13:58:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.241+0000 7fd8b7588700 1 -- 172.21.15.29:0/2509391599 shutdown_connections 2024-01-24T13:58:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.241+0000 7fd8b7588700 1 --2- 172.21.15.29:0/2509391599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0062970 0x7fd8b0062d50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.241+0000 7fd8b7588700 1 -- 172.21.15.29:0/2509391599 >> 172.21.15.29:0/2509391599 conn(0x7fd8b00c8580 msgr2=0x7fd8b00c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.241+0000 7fd8b7588700 1 -- 172.21.15.29:0/2509391599 shutdown_connections 2024-01-24T13:58:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.241+0000 7fd8b7588700 1 -- 172.21.15.29:0/2509391599 wait complete. 2024-01-24T13:58:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.242+0000 7fd8b7588700 1 Processor -- start 2024-01-24T13:58:25.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.242+0000 7fd8b7588700 1 -- start start 2024-01-24T13:58:25.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b7588700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b0062970 0x7fd8b00c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:25.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b7588700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8b0066b40 0x7fd8b00c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b7588700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0158600 0x7fd8b0157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b7588700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00a7480 con 0x7fd8b0062970 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b7588700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00c72d0 con 0x7fd8b0066b40 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b7588700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00bf840 con 0x7fd8b0158600 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b5324700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b0062970 0x7fd8b00c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b5324700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b0062970 0x7fd8b00c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51224/0 (socket says 172.21.15.29:51224) 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b5324700 1 -- 172.21.15.29:0/4131965409 learned_addr learned my addr 172.21.15.29:0/4131965409 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b5b25700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0158600 0x7fd8b0157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b4b23700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8b0066b40 0x7fd8b00c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b5324700 1 -- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0158600 msgr2=0x7fd8b0157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b5324700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0158600 0x7fd8b0157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b5324700 1 -- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8b0066b40 msgr2=0x7fd8b00c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b5324700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8b0066b40 0x7fd8b00c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b5324700 1 -- 172.21.15.29:0/4131965409 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac0092a0 con 0x7fd8b0062970 2024-01-24T13:58:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b4b23700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8b0066b40 0x7fd8b00c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b5324700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b0062970 0x7fd8b00c68f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fd8ac009d80 tx=0x7fd8ac016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8a57fa700 1 -- 172.21.15.29:0/4131965409 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0108a0 con 0x7fd8b0062970 2024-01-24T13:58:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8a57fa700 1 -- 172.21.15.29:0/4131965409 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac010a00 con 0x7fd8b0062970 2024-01-24T13:58:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8b7588700 1 -- 172.21.15.29:0/4131965409 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b00bec30 con 0x7fd8b0062970 2024-01-24T13:58:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.244+0000 7fd8a57fa700 1 -- 172.21.15.29:0/4131965409 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac010da0 con 0x7fd8b0062970 2024-01-24T13:58:25.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.245+0000 7fd8b7588700 1 -- 172.21.15.29:0/4131965409 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b00bf1f0 con 0x7fd8b0062970 2024-01-24T13:58:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.246+0000 7fd8b7588700 1 -- 172.21.15.29:0/4131965409 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b00475e0 con 0x7fd8b0062970 2024-01-24T13:58:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.246+0000 7fd8a57fa700 1 -- 172.21.15.29:0/4131965409 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd8ac018070 con 0x7fd8b0062970 2024-01-24T13:58:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.246+0000 7fd8a57fa700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd89405f150 0x7fd894061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.247+0000 7fd8a57fa700 1 -- 172.21.15.29:0/4131965409 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fd8ac07b5f0 con 0x7fd8b0062970 2024-01-24T13:58:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.247+0000 7fd8b4b23700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd89405f150 0x7fd894061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:25.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.247+0000 7fd8b4b23700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd89405f150 0x7fd894061610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fd8b00a7e40 tx=0x7fd8a0009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:25.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.251+0000 7fd8a57fa700 1 -- 172.21.15.29:0/4131965409 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd8ac04c5b0 con 0x7fd8b0062970 2024-01-24T13:58:25.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.432+0000 7fd8b7588700 1 -- 172.21.15.29:0/4131965409 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8b00bffd0 con 0x7fd8b0062970 2024-01-24T13:58:25.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.433+0000 7fd8a57fa700 1 -- 172.21.15.29:0/4131965409 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7fd8ac021020 con 0x7fd8b0062970 2024-01-24T13:58:25.434 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:25.434 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.435+0000 7fd8b7588700 1 -- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd89405f150 msgr2=0x7fd894061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.435+0000 7fd8b7588700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd89405f150 0x7fd894061610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fd8b00a7e40 tx=0x7fd8a0009040).stop 2024-01-24T13:58:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.435+0000 7fd8b7588700 1 -- 172.21.15.29:0/4131965409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b0062970 msgr2=0x7fd8b00c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.435+0000 7fd8b7588700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b0062970 0x7fd8b00c68f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fd8ac009d80 tx=0x7fd8ac016000).stop 2024-01-24T13:58:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.436+0000 7fd8b7588700 1 -- 172.21.15.29:0/4131965409 shutdown_connections 2024-01-24T13:58:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.436+0000 7fd8b7588700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0158600 0x7fd8b0157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.436+0000 7fd8b7588700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd89405f150 0x7fd894061610 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.436+0000 7fd8b7588700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b0062970 0x7fd8b00c68f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.436+0000 7fd8b7588700 1 --2- 172.21.15.29:0/4131965409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8b0066b40 0x7fd8b00c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.436+0000 7fd8b7588700 1 -- 172.21.15.29:0/4131965409 >> 172.21.15.29:0/4131965409 conn(0x7fd8b00c8580 msgr2=0x7fd8b00c4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:25.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.436+0000 7fd8b7588700 1 -- 172.21.15.29:0/4131965409 shutdown_connections 2024-01-24T13:58:25.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.436+0000 7fd8b7588700 1 -- 172.21.15.29:0/4131965409 wait complete. 2024-01-24T13:58:25.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T13:58:19.190134+0000', 'modified': '2024-01-24T13:58:23.623801+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/1328130656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 1328130656}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 1328130656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 1} 2024-01-24T13:58:25.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T13:58:25.852 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-24T13:58:25.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.991+0000 7fe8eed38700 1 Processor -- start 2024-01-24T13:58:25.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.991+0000 7fe8eed38700 1 -- start start 2024-01-24T13:58:25.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.992+0000 7fe8eed38700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe8e8062ec0 0x7fe8e815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:25.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.992+0000 7fe8eed38700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e80a73f0 con 0x7fe8e8151aa0 2024-01-24T13:58:25.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.992+0000 7fe8eed38700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e8065360 con 0x7fe8e8151e80 2024-01-24T13:58:25.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.992+0000 7fe8eed38700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e80bcab0 con 0x7fe8e8062ec0 2024-01-24T13:58:25.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.992+0000 7fe8ed2d5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe8e8062ec0 0x7fe8e815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:25.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.992+0000 7fe8ecad4700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe8e8151aa0 0x7fe8e80a6ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33642/0 (socket says 172.21.15.29:33642) 2024-01-24T13:58:25.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.992+0000 7fe8ed2d5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe8e8062ec0 0x7fe8e815efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51232/0 (socket says 172.21.15.29:51232) 2024-01-24T13:58:25.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.992+0000 7fe8ecad4700 1 -- 172.21.15.29:0/1374634868 learned_addr learned my addr 172.21.15.29:0/1374634868 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.992+0000 7fe8ed2d5700 1 -- 172.21.15.29:0/1374634868 >> v1:172.21.15.163:6789/0 conn(0x7fe8e8151e80 legacy=0x7fe8e80627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.992+0000 7fe8ed2d5700 1 -- 172.21.15.29:0/1374634868 >> v1:172.21.15.29:6789/0 conn(0x7fe8e8151aa0 legacy=0x7fe8e80a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.993+0000 7fe8ed2d5700 1 -- 172.21.15.29:0/1374634868 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e80bcd90 con 0x7fe8e8062ec0 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.992+0000 7fe8e77fe700 1 -- 172.21.15.29:0/1374634868 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499336851 0 0) 0x7fe8e80a73f0 con 0x7fe8e8151aa0 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.993+0000 7fe8ed2d5700 1 --2- 172.21.15.29:0/1374634868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e8062ec0 0x7fe8e815efc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fe8e0009e60 tx=0x7fe8e0009780).ready entity=mon.0 client_cookie=2aeae8011ca0429c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.993+0000 7fe8e77fe700 1 -- 172.21.15.29:0/1374634868 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000e070 con 0x7fe8e8062ec0 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.993+0000 7fe8e77fe700 1 -- 172.21.15.29:0/1374634868 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e000bc30 con 0x7fe8e8062ec0 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.993+0000 7fe8e77fe700 1 -- 172.21.15.29:0/1374634868 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0012410 con 0x7fe8e8062ec0 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.993+0000 7fe8eed38700 1 -- 172.21.15.29:0/1374634868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e8062ec0 msgr2=0x7fe8e815efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.993+0000 7fe8eed38700 1 --2- 172.21.15.29:0/1374634868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e8062ec0 0x7fe8e815efc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fe8e0009e60 tx=0x7fe8e0009780).stop 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.993+0000 7fe8eed38700 1 -- 172.21.15.29:0/1374634868 shutdown_connections 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.993+0000 7fe8eed38700 1 --2- 172.21.15.29:0/1374634868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e8062ec0 0x7fe8e815efc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.993+0000 7fe8eed38700 1 -- 172.21.15.29:0/1374634868 >> 172.21.15.29:0/1374634868 conn(0x7fe8e80c8580 msgr2=0x7fe8e80c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.993+0000 7fe8eed38700 1 -- 172.21.15.29:0/1374634868 shutdown_connections 2024-01-24T13:58:25.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.993+0000 7fe8eed38700 1 -- 172.21.15.29:0/1374634868 wait complete. 2024-01-24T13:58:25.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.995+0000 7fe8eed38700 1 Processor -- start 2024-01-24T13:58:25.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.995+0000 7fe8eed38700 1 -- start start 2024-01-24T13:58:25.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8eed38700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e8062ec0 0x7fe8e814e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:25.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8eed38700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8e8151aa0 0x7fe8e80a75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:25.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8eed38700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8e8151e80 0x7fe8e8065920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:25.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8eed38700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e80a7370 con 0x7fe8e8062ec0 2024-01-24T13:58:25.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8eed38700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e80aae10 con 0x7fe8e8151e80 2024-01-24T13:58:25.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8eed38700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e80a8030 con 0x7fe8e8151aa0 2024-01-24T13:58:25.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8ecad4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e8062ec0 0x7fe8e814e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:25.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8ecad4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e8062ec0 0x7fe8e814e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51248/0 (socket says 172.21.15.29:51248) 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8ecad4700 1 -- 172.21.15.29:0/1626058427 learned_addr learned my addr 172.21.15.29:0/1626058427 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8ed2d5700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8e8151e80 0x7fe8e8065920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8e7fff700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8e8151aa0 0x7fe8e80a75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8ecad4700 1 -- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8e8151aa0 msgr2=0x7fe8e80a75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8e8151aa0 0x7fe8e80a75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8ecad4700 1 -- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8e8151e80 msgr2=0x7fe8e8065920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8e8151e80 0x7fe8e8065920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8ecad4700 1 -- 172.21.15.29:0/1626058427 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0009a20 con 0x7fe8e8062ec0 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8ed2d5700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8e8151e80 0x7fe8e8065920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e8062ec0 0x7fe8e814e6b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fe8dc00c4c0 tx=0x7fe8dc007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8e4ff9700 1 -- 172.21.15.29:0/1626058427 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0072e0 con 0x7fe8e8062ec0 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8e4ff9700 1 -- 172.21.15.29:0/1626058427 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc007440 con 0x7fe8e8062ec0 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8eed38700 1 -- 172.21.15.29:0/1626058427 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e80a8310 con 0x7fe8e8062ec0 2024-01-24T13:58:25.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.996+0000 7fe8e4ff9700 1 -- 172.21.15.29:0/1626058427 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0057a0 con 0x7fe8e8062ec0 2024-01-24T13:58:26.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.997+0000 7fe8eed38700 1 -- 172.21.15.29:0/1626058427 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e80aaaa0 con 0x7fe8e8062ec0 2024-01-24T13:58:26.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.998+0000 7fe8e4ff9700 1 -- 172.21.15.29:0/1626058427 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe8dc005b90 con 0x7fe8e8062ec0 2024-01-24T13:58:26.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.998+0000 7fe8e4ff9700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe8cc05f120 0x7fe8cc0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.998+0000 7fe8e4ff9700 1 -- 172.21.15.29:0/1626058427 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe8dc079de0 con 0x7fe8e8062ec0 2024-01-24T13:58:26.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.998+0000 7fe8e7fff700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe8cc05f120 0x7fe8cc0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:25.999+0000 7fe8e7fff700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe8cc05f120 0x7fe8cc0615e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fe8e80c3a70 tx=0x7fe8d4009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:26.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.001+0000 7fe900703b80 1 Processor -- start 2024-01-24T13:58:26.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.001+0000 7fe900703b80 1 -- start start 2024-01-24T13:58:26.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.001+0000 7fe900703b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feeee620 0x5564feeeea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.001+0000 7fe900703b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564feee9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.001+0000 7fe900703b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feee9df0 0x5564feeea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.001+0000 7fe900703b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564fee653d0 con 0x5564fef372c0 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.001+0000 7fe900703b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564fee2abe0 con 0x5564feee9df0 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.001+0000 7fe900703b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564fee40260 con 0x5564feeee620 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.001+0000 7fe8e7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564feee9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8e7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564feee9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51260/0 (socket says 172.21.15.29:51260) 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8e7fff700 1 -- 172.21.15.29:0/2444753176 learned_addr learned my addr 172.21.15.29:0/2444753176 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/2444753176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feeee620 0x5564feeeea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8e7fff700 1 -- 172.21.15.29:0/2444753176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feeee620 msgr2=0x5564feeeea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8e7fff700 1 --2- 172.21.15.29:0/2444753176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feeee620 0x5564feeeea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8ed2d5700 1 --2- 172.21.15.29:0/2444753176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feee9df0 0x5564feeea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8e7fff700 1 -- 172.21.15.29:0/2444753176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feee9df0 msgr2=0x5564feeea270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:26.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8e7fff700 1 --2- 172.21.15.29:0/2444753176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feee9df0 0x5564feeea270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8e7fff700 1 -- 172.21.15.29:0/2444753176 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564fee392f0 con 0x5564fef372c0 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8ed2d5700 1 --2- 172.21.15.29:0/2444753176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feee9df0 0x5564feeea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8e7fff700 1 --2- 172.21.15.29:0/2444753176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564feee9900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fe8d4017660 tx=0x7fe8d4012000).ready entity=mon.0 client_cookie=289f5f954bfc93b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8d1ffb700 1 -- 172.21.15.29:0/2444753176 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d401c070 con 0x5564fef372c0 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8d1ffb700 1 -- 172.21.15.29:0/2444753176 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d40170d0 con 0x5564fef372c0 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.002+0000 7fe8d1ffb700 1 -- 172.21.15.29:0/2444753176 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4014680 con 0x5564fef372c0 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.003+0000 7fe900703b80 1 -- 172.21.15.29:0/2444753176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 msgr2=0x5564feee9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.003+0000 7fe900703b80 1 --2- 172.21.15.29:0/2444753176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564feee9900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fe8d4017660 tx=0x7fe8d4012000).stop 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.003+0000 7fe900703b80 1 -- 172.21.15.29:0/2444753176 shutdown_connections 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.003+0000 7fe900703b80 1 --2- 172.21.15.29:0/2444753176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feee9df0 0x5564feeea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.003+0000 7fe900703b80 1 --2- 172.21.15.29:0/2444753176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feeee620 0x5564feeeea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.003+0000 7fe900703b80 1 --2- 172.21.15.29:0/2444753176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564feee9900 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.003+0000 7fe900703b80 1 -- 172.21.15.29:0/2444753176 >> 172.21.15.29:0/2444753176 conn(0x5564fed572e0 msgr2=0x5564fef37750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.003+0000 7fe900703b80 1 -- 172.21.15.29:0/2444753176 shutdown_connections 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.003+0000 7fe900703b80 1 -- 172.21.15.29:0/2444753176 wait complete. 2024-01-24T13:58:26.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.003+0000 7fe900703b80 1 -- start start 2024-01-24T13:58:26.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.004+0000 7fe900703b80 1 Processor -- start 2024-01-24T13:58:26.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe900703b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feee9df0 0x5564fef33390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe900703b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feeee620 0x5564fef16f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe900703b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564fef15dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe900703b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564fee403a0 con 0x5564fef372c0 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe900703b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564fee40620 con 0x5564feeee620 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe900703b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564fee0b990 con 0x5564feee9df0 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe900703b80 10 client.0 ll_register_callbacks cb 0x5564feeae610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8ed2d5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564fef15dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8ed2d5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564fef15dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51268/0 (socket says 172.21.15.29:51268) 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8ed2d5700 1 -- 172.21.15.29:0/4019460430 learned_addr learned my addr 172.21.15.29:0/4019460430 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feee9df0 0x5564fef33390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8e7fff700 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feeee620 0x5564fef16f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8ed2d5700 1 -- 172.21.15.29:0/4019460430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feee9df0 msgr2=0x5564fef33390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8ed2d5700 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feee9df0 0x5564fef33390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8ed2d5700 1 -- 172.21.15.29:0/4019460430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feeee620 msgr2=0x5564fef16f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8ed2d5700 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feeee620 0x5564fef16f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8ed2d5700 1 -- 172.21.15.29:0/4019460430 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d400f040 con 0x5564fef372c0 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feee9df0 0x5564fef33390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8ed2d5700 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564fef15dd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fe8e000b6c0 tx=0x7fe8e0009780).ready entity=mon.0 client_cookie=6a26e37715a55a87 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8c6ffd700 1 -- 172.21.15.29:0/4019460430 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e001d600 con 0x5564fef372c0 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe900703b80 1 -- 172.21.15.29:0/4019460430 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564feee8df0 con 0x5564fef372c0 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.005+0000 7fe8c6ffd700 1 -- 172.21.15.29:0/4019460430 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe8e001d760 con 0x5564fef372c0 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe8c6ffd700 1 -- 172.21.15.29:0/4019460430 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8e001db30 con 0x5564fef372c0 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe900703b80 10 client.4416 fetch_fsmap learned FSMap version 7 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe900703b80 1 -- 172.21.15.29:0/4019460430 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5564fee34520 con 0x5564fef372c0 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe8c6ffd700 1 -- 172.21.15.29:0/4019460430 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5564feee8df0 con 0x5564fef372c0 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe8c6ffd700 1 -- 172.21.15.29:0/4019460430 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 7) v1 ==== 3108+0+0 (secure 0 0 0) 0x7fe8e001dc90 con 0x5564fef372c0 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe900703b80 10 client.4416 fetch_fsmap finished waiting for FSMap version 7 2024-01-24T13:58:26.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe900703b80 10 client.4416 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe900703b80 20 client.4416 populate_metadata read hostname 'smithi029' 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe900703b80 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x5564fef46a70 0x5564fef46e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe900703b80 4 client.4416 mds_command: new command op to 4281 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe900703b80 1 -- 172.21.15.29:0/4019460430 --> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564fee35d60 con 0x5564fef46a70 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x5564fef46a70 0x5564fef46e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe8c6ffd700 10 client.4416 ms_handle_connect on v2:172.21.15.29:6833/1328130656 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.006+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x5564fef46a70 0x5564fef46e50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a615f2ccbd998c2a server_cookie=656d6371cb838397 in_seq=0 out_seq=0 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.007+0000 7fe8c6ffd700 1 -- 172.21.15.29:0/4019460430 <== mds.0 v2:172.21.15.29:6833/1328130656 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x5564fee35d60 con 0x5564fef46a70 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.007+0000 7fe8c6ffd700 10 client.4416 handle_command_reply: tid=0 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.007+0000 7fe900703b80 1 client.4416 shutdown 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.007+0000 7fe900703b80 1 -- 172.21.15.29:0/4019460430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 msgr2=0x5564fef15dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.007+0000 7fe900703b80 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564fef15dd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fe8e000b6c0 tx=0x7fe8e0009780).stop 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.007+0000 7fe900703b80 1 -- 172.21.15.29:0/4019460430 shutdown_connections 2024-01-24T13:58:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.007+0000 7fe900703b80 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x5564fef46a70 0x5564fef46e50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.007+0000 7fe900703b80 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564fef15dd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.007+0000 7fe900703b80 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feee9df0 0x5564fef33390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.007+0000 7fe900703b80 1 --2- 172.21.15.29:0/4019460430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feeee620 0x5564fef16f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.007+0000 7fe8c6ffd700 0 client.4416 ms_handle_reset on v2:172.21.15.29:6833/1328130656 2024-01-24T13:58:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.007+0000 7fe900703b80 1 -- 172.21.15.29:0/4019460430 >> 172.21.15.29:0/4019460430 conn(0x5564fed572e0 msgr2=0x5564feeec690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.008+0000 7fe900703b80 1 -- 172.21.15.29:0/4019460430 shutdown_connections 2024-01-24T13:58:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.008+0000 7fe900703b80 1 -- 172.21.15.29:0/4019460430 wait complete. 2024-01-24T13:58:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.008+0000 7fe900703b80 20 client.4416 trim_cache size 0 max 16384 2024-01-24T13:58:26.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.025+0000 7fe900703b80 1 Processor -- start 2024-01-24T13:58:26.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.025+0000 7fe900703b80 1 -- start start 2024-01-24T13:58:26.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.026+0000 7fe900703b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564feee9df0 0x5564feeea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.026+0000 7fe900703b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feeee620 0x5564feeea6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.026+0000 7fe900703b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564fef372c0 0x5564feee5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.026+0000 7fe900703b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564feeeabb0 con 0x5564feee9df0 2024-01-24T13:58:26.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.026+0000 7fe900703b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564fef37750 con 0x5564fef372c0 2024-01-24T13:58:26.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.026+0000 7fe900703b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564feeeea00 con 0x5564feeee620 2024-01-24T13:58:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.026+0000 7fe8ecad4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564feee9df0 0x5564feeea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.026+0000 7fe8ecad4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564feee9df0 0x5564feeea1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51270/0 (socket says 172.21.15.29:51270) 2024-01-24T13:58:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.026+0000 7fe8ecad4700 1 -- 172.21.15.29:0/366471617 learned_addr learned my addr 172.21.15.29:0/366471617 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.026+0000 7fe8e7fff700 1 --2- 172.21.15.29:0/366471617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feeee620 0x5564feeea6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.026+0000 7fe8ed2d5700 1 --2- 172.21.15.29:0/366471617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564fef372c0 0x5564feee5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe8ecad4700 1 -- 172.21.15.29:0/366471617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feeee620 msgr2=0x5564feeea6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/366471617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feeee620 0x5564feeea6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe8ecad4700 1 -- 172.21.15.29:0/366471617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564fef372c0 msgr2=0x5564feee5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/366471617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564fef372c0 0x5564feee5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe8ecad4700 1 -- 172.21.15.29:0/366471617 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564fee34270 con 0x5564feee9df0 2024-01-24T13:58:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/366471617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564feee9df0 0x5564feeea1d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fe8dc04e990 tx=0x7fe8dc052040).ready entity=mon.0 client_cookie=fe7501f2edd5c063 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe8d27fc700 1 -- 172.21.15.29:0/366471617 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc054130 con 0x5564feee9df0 2024-01-24T13:58:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe8d27fc700 1 -- 172.21.15.29:0/366471617 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc04cba0 con 0x5564feee9df0 2024-01-24T13:58:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe8d27fc700 1 -- 172.21.15.29:0/366471617 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc04b170 con 0x5564feee9df0 2024-01-24T13:58:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe900703b80 1 -- 172.21.15.29:0/366471617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564feee9df0 msgr2=0x5564feeea1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe900703b80 1 --2- 172.21.15.29:0/366471617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564feee9df0 0x5564feeea1d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fe8dc04e990 tx=0x7fe8dc052040).stop 2024-01-24T13:58:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe900703b80 1 -- 172.21.15.29:0/366471617 shutdown_connections 2024-01-24T13:58:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe900703b80 1 --2- 172.21.15.29:0/366471617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564fef372c0 0x5564feee5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe900703b80 1 --2- 172.21.15.29:0/366471617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564feee9df0 0x5564feeea1d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe900703b80 1 --2- 172.21.15.29:0/366471617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feeee620 0x5564feeea6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe900703b80 1 -- 172.21.15.29:0/366471617 >> 172.21.15.29:0/366471617 conn(0x5564fed572e0 msgr2=0x5564fef331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.027+0000 7fe900703b80 1 -- 172.21.15.29:0/366471617 shutdown_connections 2024-01-24T13:58:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.028+0000 7fe900703b80 1 -- 172.21.15.29:0/366471617 wait complete. 2024-01-24T13:58:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.028+0000 7fe900703b80 1 -- start start 2024-01-24T13:58:26.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.028+0000 7fe900703b80 1 Processor -- start 2024-01-24T13:58:26.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe900703b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feee9df0 0x5564fef3bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe900703b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feeee620 0x5564feee5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe900703b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564feee5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe900703b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564feee5a00 con 0x5564fef372c0 2024-01-24T13:58:26.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe900703b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564fef9cad0 con 0x5564feeee620 2024-01-24T13:58:26.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe900703b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564fef4d3a0 con 0x5564feee9df0 2024-01-24T13:58:26.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe900703b80 10 client.0 ll_register_callbacks cb 0x5564feeae610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T13:58:26.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe8ed2d5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564feee5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe8ed2d5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564feee5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51274/0 (socket says 172.21.15.29:51274) 2024-01-24T13:58:26.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe8ed2d5700 1 -- 172.21.15.29:0/1961782318 learned_addr learned my addr 172.21.15.29:0/1961782318 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feee9df0 0x5564fef3bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe8e7fff700 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feeee620 0x5564feee5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe8ed2d5700 1 -- 172.21.15.29:0/1961782318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feee9df0 msgr2=0x5564fef3bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe8ed2d5700 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feee9df0 0x5564fef3bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe8ed2d5700 1 -- 172.21.15.29:0/1961782318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feeee620 msgr2=0x5564feee5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe8ed2d5700 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feeee620 0x5564feee5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe8ed2d5700 1 -- 172.21.15.29:0/1961782318 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e80aaaa0 con 0x5564fef372c0 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe8ed2d5700 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564feee5510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x5564fef31250 tx=0x7fe8e001e000).ready entity=mon.0 client_cookie=94cdd46f5736dae9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.030+0000 7fe8c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe8c6ffd700 1 -- 172.21.15.29:0/1961782318 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000bed0 con 0x5564fef372c0 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe8c6ffd700 1 -- 172.21.15.29:0/1961782318 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe8e001db30 con 0x5564fef372c0 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe900703b80 1 -- 172.21.15.29:0/1961782318 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8d40170d0 con 0x5564fef372c0 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe8c6ffd700 1 -- 172.21.15.29:0/1961782318 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8e00264c0 con 0x5564fef372c0 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe900703b80 10 client.4422 fetch_fsmap learned FSMap version 7 2024-01-24T13:58:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe900703b80 1 -- 172.21.15.29:0/1961782318 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5564fecadf80 con 0x5564fef372c0 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe8c6ffd700 1 -- 172.21.15.29:0/1961782318 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d40170d0 con 0x5564fef372c0 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe8c6ffd700 1 -- 172.21.15.29:0/1961782318 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 7) v1 ==== 3108+0+0 (secure 0 0 0) 0x7fe8e0026620 con 0x5564fef372c0 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe900703b80 10 client.4422 fetch_fsmap finished waiting for FSMap version 7 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe900703b80 10 client.4422 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe900703b80 20 client.4422 populate_metadata read hostname 'smithi029' 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe900703b80 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x5564fef33fe0 0x5564fef343c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe900703b80 4 client.4422 mds_command: new command op to 4281 tid=0[{"prefix": "status"}] 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe900703b80 1 -- 172.21.15.29:0/1961782318 --> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5564fee2c7d0 con 0x5564fef33fe0 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x5564fef33fe0 0x5564fef343c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe8c6ffd700 10 client.4422 ms_handle_connect on v2:172.21.15.29:6833/1328130656 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.031+0000 7fe8ecad4700 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x5564fef33fe0 0x5564fef343c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=20a15ffa82c392ae server_cookie=2e1ef07729ff1a1 in_seq=0 out_seq=0 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe8c6ffd700 1 -- 172.21.15.29:0/1961782318 <== mds.0 v2:172.21.15.29:6833/1328130656 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x5564fee2c7d0 con 0x5564fef33fe0 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe8c6ffd700 10 client.4422 handle_command_reply: tid=0 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe900703b80 1 client.4422 shutdown 2024-01-24T13:58:26.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe900703b80 1 -- 172.21.15.29:0/1961782318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 msgr2=0x5564feee5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:26.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe900703b80 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564feee5510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x5564fef31250 tx=0x7fe8e001e000).stop 2024-01-24T13:58:26.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe900703b80 1 -- 172.21.15.29:0/1961782318 shutdown_connections 2024-01-24T13:58:26.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe900703b80 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x5564fef33fe0 0x5564fef343c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe900703b80 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5564fef372c0 0x5564feee5510 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe900703b80 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5564feee9df0 0x5564fef3bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe8c6ffd700 0 client.4422 ms_handle_reset on v2:172.21.15.29:6833/1328130656 2024-01-24T13:58:26.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe900703b80 1 --2- 172.21.15.29:0/1961782318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564feeee620 0x5564feee5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe900703b80 1 -- 172.21.15.29:0/1961782318 >> 172.21.15.29:0/1961782318 conn(0x5564fed572e0 msgr2=0x5564fef390b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:26.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe900703b80 1 -- 172.21.15.29:0/1961782318 shutdown_connections 2024-01-24T13:58:26.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe900703b80 1 -- 172.21.15.29:0/1961782318 wait complete. 2024-01-24T13:58:26.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.032+0000 7fe900703b80 20 client.4422 trim_cache size 0 max 16384 2024-01-24T13:58:26.037 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T13:58:26.038 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T13:58:26.038 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T13:58:26.038 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 4281, 2024-01-24T13:58:26.038 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T13:58:26.038 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T13:58:26.038 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T13:58:26.038 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 6.841179468, 2024-01-24T13:58:26.038 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 7, 2024-01-24T13:58:26.038 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 26, 2024-01-24T13:58:26.038 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 25, 2024-01-24T13:58:26.038 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 12.158367807999999 2024-01-24T13:58:26.038 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T13:58:26.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.033+0000 7fe8eed38700 1 -- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe8cc05f120 msgr2=0x7fe8cc0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:26.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.033+0000 7fe8eed38700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe8cc05f120 0x7fe8cc0615e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fe8e80c3a70 tx=0x7fe8d4009040).stop 2024-01-24T13:58:26.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.033+0000 7fe8eed38700 1 -- 172.21.15.29:0/1626058427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e8062ec0 msgr2=0x7fe8e814e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:26.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.033+0000 7fe8eed38700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e8062ec0 0x7fe8e814e6b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fe8dc00c4c0 tx=0x7fe8dc007c10).stop 2024-01-24T13:58:26.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.033+0000 7fe8eed38700 1 -- 172.21.15.29:0/1626058427 shutdown_connections 2024-01-24T13:58:26.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.033+0000 7fe8eed38700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8e8151e80 0x7fe8e8065920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.033+0000 7fe8eed38700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe8cc05f120 0x7fe8cc0615e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.033+0000 7fe8eed38700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e8062ec0 0x7fe8e814e6b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.033+0000 7fe8eed38700 1 --2- 172.21.15.29:0/1626058427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8e8151aa0 0x7fe8e80a75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.033+0000 7fe8eed38700 1 -- 172.21.15.29:0/1626058427 >> 172.21.15.29:0/1626058427 conn(0x7fe8e80c8580 msgr2=0x7fe8e80c58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:26.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.033+0000 7fe8eed38700 1 -- 172.21.15.29:0/1626058427 shutdown_connections 2024-01-24T13:58:26.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.033+0000 7fe8eed38700 1 -- 172.21.15.29:0/1626058427 wait complete. 2024-01-24T13:58:26.046 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T13:58:26.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.161+0000 7f1de208b700 1 Processor -- start 2024-01-24T13:58:26.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.161+0000 7f1de208b700 1 -- start start 2024-01-24T13:58:26.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1de208b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1ddc154050 0x7f1ddc0c0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1de208b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ddc0c0b40 0x7f1ddc154430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1de208b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1ddc154920 0x7f1ddc154db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1de208b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ddc06aa10 con 0x7f1ddc154050 2024-01-24T13:58:26.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1de208b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ddc0a82f0 con 0x7f1ddc154920 2024-01-24T13:58:26.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1de208b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0c0fc0 con 0x7f1ddc0c0b40 2024-01-24T13:58:26.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1ddb7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1ddc154050 0x7f1ddc0c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1ddb7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1ddc154050 0x7f1ddc0c0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51276/0 (socket says 172.21.15.29:51276) 2024-01-24T13:58:26.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1ddb7fe700 1 -- 172.21.15.29:0/433391142 learned_addr learned my addr 172.21.15.29:0/433391142 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:26.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1ddaffd700 1 --2- 172.21.15.29:0/433391142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ddc0c0b40 0x7f1ddc154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1ddbfff700 1 --2- 172.21.15.29:0/433391142 >> v2:172.21.15.163:3300/0 conn(0x7f1ddc154920 0x7f1ddc154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1ddb7fe700 1 -- 172.21.15.29:0/433391142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ddc0c0b40 msgr2=0x7f1ddc154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1ddb7fe700 1 --2- 172.21.15.29:0/433391142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ddc0c0b40 0x7f1ddc154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1ddb7fe700 1 -- 172.21.15.29:0/433391142 >> v2:172.21.15.163:3300/0 conn(0x7f1ddc154920 msgr2=0x7f1ddc154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1ddb7fe700 1 --2- 172.21.15.29:0/433391142 >> v2:172.21.15.163:3300/0 conn(0x7f1ddc154920 0x7f1ddc154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1ddb7fe700 1 -- 172.21.15.29:0/433391142 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc155360 con 0x7f1ddc154050 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.162+0000 7f1ddbfff700 1 --2- 172.21.15.29:0/433391142 >> v2:172.21.15.163:3300/0 conn(0x7f1ddc154920 0x7f1ddc154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1ddb7fe700 1 --2- 172.21.15.29:0/433391142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ddc154050 0x7f1ddc0c0650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f1ddc083940 tx=0x7f1dcc0090f0).ready entity=mon.0 client_cookie=eaf5d903b92148a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1dda7fc700 1 -- 172.21.15.29:0/433391142 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00b520 con 0x7f1ddc154050 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1dda7fc700 1 -- 172.21.15.29:0/433391142 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dcc00b680 con 0x7f1ddc154050 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1dda7fc700 1 -- 172.21.15.29:0/433391142 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00ba00 con 0x7f1ddc154050 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1de208b700 1 -- 172.21.15.29:0/433391142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ddc154050 msgr2=0x7f1ddc0c0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1de208b700 1 --2- 172.21.15.29:0/433391142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ddc154050 0x7f1ddc0c0650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f1ddc083940 tx=0x7f1dcc0090f0).stop 2024-01-24T13:58:26.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1de208b700 1 -- 172.21.15.29:0/433391142 shutdown_connections 2024-01-24T13:58:26.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1de208b700 1 --2- 172.21.15.29:0/433391142 >> v2:172.21.15.163:3300/0 conn(0x7f1ddc154920 0x7f1ddc154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1de208b700 1 --2- 172.21.15.29:0/433391142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ddc154050 0x7f1ddc0c0650 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1de208b700 1 --2- 172.21.15.29:0/433391142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ddc0c0b40 0x7f1ddc154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1de208b700 1 -- 172.21.15.29:0/433391142 >> 172.21.15.29:0/433391142 conn(0x7f1ddc084bb0 msgr2=0x7f1ddc0944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:26.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1de208b700 1 -- 172.21.15.29:0/433391142 shutdown_connections 2024-01-24T13:58:26.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.163+0000 7f1de208b700 1 -- 172.21.15.29:0/433391142 wait complete. 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.165+0000 7f1de208b700 1 Processor -- start 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.165+0000 7f1de208b700 1 -- start start 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.165+0000 7f1de208b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ddc0c0b40 0x7f1ddc0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1de208b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ddc154050 0x7f1ddc14fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1de208b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ddc154920 0x7f1ddc1437e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1de208b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0ae190 con 0x7f1ddc154920 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1de208b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc152550 con 0x7f1ddc154050 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1de208b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc152830 con 0x7f1ddc0c0b40 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1ddbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ddc154920 0x7f1ddc1437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1ddbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ddc154920 0x7f1ddc1437e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51280/0 (socket says 172.21.15.29:51280) 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1ddbfff700 1 -- 172.21.15.29:0/4147499971 learned_addr learned my addr 172.21.15.29:0/4147499971 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1ddb7fe700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ddc0c0b40 0x7f1ddc0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1ddaffd700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ddc154050 0x7f1ddc14fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1ddbfff700 1 -- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ddc0c0b40 msgr2=0x7f1ddc0b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1ddbfff700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ddc0c0b40 0x7f1ddc0b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1ddbfff700 1 -- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ddc154050 msgr2=0x7f1ddc14fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1ddbfff700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ddc154050 0x7f1ddc14fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1ddbfff700 1 -- 172.21.15.29:0/4147499971 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc009370 con 0x7f1ddc154920 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1ddbfff700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ddc154920 0x7f1ddc1437e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f1dd4009fc0 tx=0x7f1dd40099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1dd37fe700 1 -- 172.21.15.29:0/4147499971 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd400ddd0 con 0x7f1ddc154920 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1dd37fe700 1 -- 172.21.15.29:0/4147499971 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd400f040 con 0x7f1ddc154920 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1de208b700 1 -- 172.21.15.29:0/4147499971 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ddc152b10 con 0x7f1ddc154920 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1dd37fe700 1 -- 172.21.15.29:0/4147499971 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4013600 con 0x7f1ddc154920 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.166+0000 7f1de208b700 1 -- 172.21.15.29:0/4147499971 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ddc150320 con 0x7f1ddc154920 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.167+0000 7f1de208b700 1 -- 172.21.15.29:0/4147499971 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ddc0ae300 con 0x7f1ddc154920 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.168+0000 7f1dd37fe700 1 -- 172.21.15.29:0/4147499971 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1dd40139f0 con 0x7f1ddc154920 2024-01-24T13:58:26.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.168+0000 7f1dd37fe700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1dbc05f150 0x7f1dbc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:26.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.168+0000 7f1dd37fe700 1 -- 172.21.15.29:0/4147499971 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f1dd407ab80 con 0x7f1ddc154920 2024-01-24T13:58:26.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.168+0000 7f1ddb7fe700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1dbc05f150 0x7f1dbc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:26.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.169+0000 7f1ddb7fe700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1dbc05f150 0x7f1dbc061610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1dcc009130 tx=0x7f1dcc017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:26.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.173+0000 7f1dd37fe700 1 -- 172.21.15.29:0/4147499971 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1ddc0ae300 con 0x7f1ddc154920 2024-01-24T13:58:26.183 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T13:58:26.185 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T13:58:26.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.401+0000 7f1de208b700 1 -- 172.21.15.29:0/4147499971 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ddc0ae2d0 con 0x7f1ddc154920 2024-01-24T13:58:26.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.401+0000 7f1dd37fe700 1 -- 172.21.15.29:0/4147499971 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1dd407ae60 con 0x7f1ddc154920 2024-01-24T13:58:26.402 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-24T13:58:26.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.403+0000 7f1de208b700 1 -- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1dbc05f150 msgr2=0x7f1dbc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:26.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.403+0000 7f1de208b700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1dbc05f150 0x7f1dbc061610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1dcc009130 tx=0x7f1dcc017000).stop 2024-01-24T13:58:26.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.403+0000 7f1de208b700 1 -- 172.21.15.29:0/4147499971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ddc154920 msgr2=0x7f1ddc1437e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:26.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.403+0000 7f1de208b700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ddc154920 0x7f1ddc1437e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f1dd4009fc0 tx=0x7f1dd40099b0).stop 2024-01-24T13:58:26.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.403+0000 7f1de208b700 1 -- 172.21.15.29:0/4147499971 shutdown_connections 2024-01-24T13:58:26.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.403+0000 7f1de208b700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1dbc05f150 0x7f1dbc061610 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.403+0000 7f1de208b700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ddc154920 0x7f1ddc1437e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.403+0000 7f1de208b700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ddc0c0b40 0x7f1ddc0b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.403+0000 7f1de208b700 1 --2- 172.21.15.29:0/4147499971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ddc154050 0x7f1ddc14fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:26.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.403+0000 7f1de208b700 1 -- 172.21.15.29:0/4147499971 >> 172.21.15.29:0/4147499971 conn(0x7f1ddc084bb0 msgr2=0x7f1ddc064180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:26.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.403+0000 7f1de208b700 1 -- 172.21.15.29:0/4147499971 shutdown_connections 2024-01-24T13:58:26.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:26.403+0000 7f1de208b700 1 -- 172.21.15.29:0/4147499971 wait complete. 2024-01-24T13:58:26.416 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-24T13:58:26.416 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T13:58:26.416 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T13:58:26.416 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-01-24T13:58:26.416 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-01-24T13:58:26.416 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-24T13:58:26.416 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-24T13:58:26.416 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T13:58:26.416 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-01-24T13:58:26.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:26.416 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T13:58:26.436 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T13:58:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T13:58:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T13:58:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T13:58:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:26.436 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T13:58:26.436 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T13:58:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:26.436 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T13:58:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T13:58:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42344sec preferred_lft 42344sec 2024-01-24T13:58:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T13:58:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:26.437 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T13:58:26.437 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:26.437 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T13:58:26.438 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-24T13:58:26.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:26.438 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:26.438 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:26.438 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add name ceph-brx type bridge 2024-01-24T13:58:26.438 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip addr flush dev ceph-brx 2024-01-24T13:58:26.438 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set ceph-brx up 2024-01-24T13:58:26.438 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-24T13:58:26.438 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:26.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:26.616 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T13:58:26.650 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T13:58:26.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:26.652 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T13:58:26.725 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T13:58:26.725 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T13:58:26.725 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T13:58:26.725 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T13:58:26.725 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T13:58:26.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:26.725 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:26.725 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:26.726 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T13:58:26.726 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T13:58:26.726 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T13:58:26.726 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:26.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:26.975 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T13:58:26.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:26.996 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list-id 2024-01-24T13:58:27.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:27.057 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:27.057 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:27.057 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-24T13:58:27.057 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-01-24T13:58:27.057 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:27.168 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-01-24T13:58:27.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:27.168 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:27.168 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:27.169 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-01-24T13:58:27.169 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-24T13:58:27.169 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-01-24T13:58:27.169 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-01-24T13:58:27.169 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-01-24T13:58:27.169 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:27.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:27.414 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:27.414 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:27.414 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T13:58:27.414 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set dev brx.0 master ceph-brx 2024-01-24T13:58:27.415 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:27.527 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-01-24T13:58:27.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:58:27.527 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -m 0000 -p -v /home/ubuntu/cephtest/mnt.admin 2024-01-24T13:58:27.546 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-01-24T13:58:27.546 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-01-24T13:58:27.621 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-01-24T13:58:27.621 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-01-24T13:58:27.695 INFO:teuthology.orchestra.run.smithi029.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-01-24T13:58:27.697 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-01-24T13:58:27.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:58:27.698 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-01-24T13:58:27.760 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-24T13:58:27.760 DEBUG:teuthology.orchestra.run.smithi029:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' --id admin --client_fs=cephfs /home/ubuntu/cephtest/mnt.admin 2024-01-24T13:58:27.802 DEBUG:teuthology.orchestra.run.smithi029:> sudo modprobe fuse 2024-01-24T13:58:27.837 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-01-24T13:58:27.837 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-01-24T13:58:27.906 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi029.stderr:2024-01-24T13:58:27.906+0000 7f474b6bb3c0 -1 init, newargv = 0x557041f94180 newargc=15 2024-01-24T13:58:27.906 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi029.stderr:ceph-fuse[48922]: starting ceph client 2024-01-24T13:58:27.909 INFO:teuthology.orchestra.run.smithi029.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-01-24T13:58:27.910 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-01-24T13:58:27.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:58:27.911 DEBUG:teuthology.orchestra.run.smithi029:> ls /sys/fs/fuse/connections 2024-01-24T13:58:27.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi029.stderr:ceph-fuse[48922]: starting fuse 2024-01-24T13:58:27.932 INFO:teuthology.orchestra.run.smithi029.stdout:47 2024-01-24T13:58:27.932 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2024-01-24T13:58:27.932 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-01-24T13:58:27.932 DEBUG:teuthology.orchestra.run.smithi029:> import glob 2024-01-24T13:58:27.932 DEBUG:teuthology.orchestra.run.smithi029:> import re 2024-01-24T13:58:27.932 DEBUG:teuthology.orchestra.run.smithi029:> import os 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> import subprocess 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> def _find_admin_socket(client_name): 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> files = glob.glob(asok_path) 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> # Given a non-glob path, it better be there 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> if "*" not in asok_path: 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> assert(len(files) == 1) 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> return files[0] 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> for f in files: 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-24T13:58:27.933 DEBUG:teuthology.orchestra.run.smithi029:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-24T13:58:27.934 DEBUG:teuthology.orchestra.run.smithi029:> contents = proc_f.read() 2024-01-24T13:58:27.934 DEBUG:teuthology.orchestra.run.smithi029:> if mountpoint in contents: 2024-01-24T13:58:27.934 DEBUG:teuthology.orchestra.run.smithi029:> return f 2024-01-24T13:58:27.934 DEBUG:teuthology.orchestra.run.smithi029:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-24T13:58:27.934 DEBUG:teuthology.orchestra.run.smithi029:> 2024-01-24T13:58:27.934 DEBUG:teuthology.orchestra.run.smithi029:> print(_find_admin_socket("client.admin")) 2024-01-24T13:58:27.934 DEBUG:teuthology.orchestra.run.smithi029:> ' 2024-01-24T13:58:28.069 INFO:teuthology.orchestra.run.smithi029.stdout:/var/run/ceph/ceph-client.admin.48922.asok 2024-01-24T13:58:28.251 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.48922.asok 2024-01-24T13:58:28.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:58:28.251 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.48922.asok status 2024-01-24T13:58:28.426 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T13:58:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "metadata": { 2024-01-24T13:58:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_sha1": "9854ba8e5f82e4476d4c6c06dd8feefca92b8b32", 2024-01-24T13:58:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 16.2.14-452-g9854ba8e (9854ba8e5f82e4476d4c6c06dd8feefca92b8b32) pacific (stable)", 2024-01-24T13:58:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "entity_id": "admin", 2024-01-24T13:58:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "hostname": "smithi029", 2024-01-24T13:58:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-01-24T13:58:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "pid": "48922", 2024-01-24T13:58:28.426 INFO:teuthology.orchestra.run.smithi029.stdout: "root": "/" 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_count": 0, 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "dentry_pinned_count": 0, 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 4434, 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "inst": { 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "name": { 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "client", 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "num": 4434 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1260644475 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:58:28.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": { 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "192.168.0.1:0", 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 1260644475 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "inst_str": "client.4434 192.168.0.1:0/1260644475", 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "addr_str": "192.168.0.1:0/1260644475", 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "inode_count": 1, 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_epoch": 7, 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch": 26, 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_epoch_barrier": 0, 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklisted": false, 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs" 2024-01-24T13:58:28.428 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T13:58:28.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:58:28.437 DEBUG:teuthology.orchestra.run.smithi029:> stat --file-system '--printf=%T 2024-01-24T13:58:28.437 DEBUG:teuthology.orchestra.run.smithi029:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-01-24T13:58:28.456 INFO:teuthology.orchestra.run.smithi029.stdout:fuseblk 2024-01-24T13:58:28.457 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-01-24T13:58:28.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:58:28.457 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-01-24T13:58:28.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:58:28.530 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-01-24T13:58:28.638 INFO:teuthology.orchestra.run.smithi029.stdout:# file: . 2024-01-24T13:58:28.638 INFO:teuthology.orchestra.run.smithi029.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-01-24T13:58:28.638 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:28.638 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi029.front.sepia.ceph.com... 2024-01-24T13:58:28.639 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T13:58:28.639 DEBUG:teuthology.orchestra.run.smithi029:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-01-24T13:58:28.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi029.stderr:ceph-fuse[48922]: fuse finished with error 0 and tester_r 0 2024-01-24T13:58:28.681 INFO:teuthology.orchestra.run:waiting for 900 2024-01-24T13:58:28.698 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T13:58:28.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:28.698 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-01-24T13:58:28.771 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:28.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.950+0000 7f5aab6e3700 1 Processor -- start 2024-01-24T13:58:28.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.950+0000 7f5aab6e3700 1 -- start start 2024-01-24T13:58:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.951+0000 7f5aab6e3700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5aa414aac0 0x7f5aa40654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.951+0000 7f5aab6e3700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa4151920 con 0x7f5aa4065990 2024-01-24T13:58:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.951+0000 7f5aab6e3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa4150d10 con 0x7f5aa40c2040 2024-01-24T13:58:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.951+0000 7f5aab6e3700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa4150b00 con 0x7f5aa414aac0 2024-01-24T13:58:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.951+0000 7f5aa947f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5aa414aac0 0x7f5aa40654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:28.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.951+0000 7f5aa947f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5aa414aac0 0x7f5aa40654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51296/0 (socket says 172.21.15.29:51296) 2024-01-24T13:58:28.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.951+0000 7f5aa947f700 1 -- 172.21.15.29:0/950943221 learned_addr learned my addr 172.21.15.29:0/950943221 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:28.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.951+0000 7f5aa947f700 1 -- 172.21.15.29:0/950943221 >> v1:172.21.15.163:6789/0 conn(0x7f5aa40c2040 legacy=0x7f5aa4068860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:28.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.951+0000 7f5aa947f700 1 -- 172.21.15.29:0/950943221 >> v1:172.21.15.29:6789/0 conn(0x7f5aa4065990 legacy=0x7f5aa40c1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:28.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.951+0000 7f5aa947f700 1 -- 172.21.15.29:0/950943221 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa4150ff0 con 0x7f5aa414aac0 2024-01-24T13:58:28.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.952+0000 7f5aa947f700 1 --2- 172.21.15.29:0/950943221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5aa414aac0 0x7f5aa40654a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f5aa0006fd0 tx=0x7f5aa000a2b0).ready entity=mon.0 client_cookie=1458c10ecc72d1dc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:28.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.952+0000 7f5a9bfff700 1 -- 172.21.15.29:0/950943221 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa000c930 con 0x7f5aa414aac0 2024-01-24T13:58:28.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.952+0000 7f5a9bfff700 1 -- 172.21.15.29:0/950943221 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa000ca90 con 0x7f5aa414aac0 2024-01-24T13:58:28.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.952+0000 7f5a9bfff700 1 -- 172.21.15.29:0/950943221 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa000ce10 con 0x7f5aa414aac0 2024-01-24T13:58:28.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.952+0000 7f5aab6e3700 1 -- 172.21.15.29:0/950943221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5aa414aac0 msgr2=0x7f5aa40654a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:28.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.952+0000 7f5aab6e3700 1 --2- 172.21.15.29:0/950943221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5aa414aac0 0x7f5aa40654a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f5aa0006fd0 tx=0x7f5aa000a2b0).stop 2024-01-24T13:58:28.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.952+0000 7f5aab6e3700 1 -- 172.21.15.29:0/950943221 shutdown_connections 2024-01-24T13:58:28.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.952+0000 7f5aab6e3700 1 --2- 172.21.15.29:0/950943221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5aa414aac0 0x7f5aa40654a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:28.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.952+0000 7f5aab6e3700 1 -- 172.21.15.29:0/950943221 >> 172.21.15.29:0/950943221 conn(0x7f5aa40c69f0 msgr2=0x7f5aa40c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:28.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.952+0000 7f5aab6e3700 1 -- 172.21.15.29:0/950943221 shutdown_connections 2024-01-24T13:58:28.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.952+0000 7f5aab6e3700 1 -- 172.21.15.29:0/950943221 wait complete. 2024-01-24T13:58:28.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.954+0000 7f5aab6e3700 1 Processor -- start 2024-01-24T13:58:28.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.954+0000 7f5aab6e3700 1 -- start start 2024-01-24T13:58:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aab6e3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5aa40c2040 0x7f5aa40be4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aab6e3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5aa414aac0 0x7f5aa414fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aab6e3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5aa40be990 0x7f5aa4141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aab6e3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa40ac2a0 con 0x7f5aa40c2040 2024-01-24T13:58:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aab6e3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa40a9b60 con 0x7f5aa414aac0 2024-01-24T13:58:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aab6e3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa415bdf0 con 0x7f5aa40be990 2024-01-24T13:58:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa947f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5aa40c2040 0x7f5aa40be4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa947f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5aa40c2040 0x7f5aa40be4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51310/0 (socket says 172.21.15.29:51310) 2024-01-24T13:58:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa947f700 1 -- 172.21.15.29:0/2967128866 learned_addr learned my addr 172.21.15.29:0/2967128866 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:28.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa8c7e700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5aa414aac0 0x7f5aa414fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa9c80700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5aa40be990 0x7f5aa4141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa947f700 1 -- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5aa40be990 msgr2=0x7f5aa4141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa947f700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5aa40be990 0x7f5aa4141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa947f700 1 -- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5aa414aac0 msgr2=0x7f5aa414fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa947f700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5aa414aac0 0x7f5aa414fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa947f700 1 -- 172.21.15.29:0/2967128866 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa000a510 con 0x7f5aa40c2040 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa9c80700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5aa40be990 0x7f5aa4141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa8c7e700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5aa414aac0 0x7f5aa414fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5aa947f700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5aa40c2040 0x7f5aa40be4a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f5aa000c4d0 tx=0x7f5aa000a2b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.955+0000 7f5a997fa700 1 -- 172.21.15.29:0/2967128866 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa000ce10 con 0x7f5aa40c2040 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.956+0000 7f5a997fa700 1 -- 172.21.15.29:0/2967128866 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa0023430 con 0x7f5aa40c2040 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.956+0000 7f5aab6e3700 1 -- 172.21.15.29:0/2967128866 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa40a9e00 con 0x7f5aa40c2040 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.956+0000 7f5a997fa700 1 -- 172.21.15.29:0/2967128866 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00237d0 con 0x7f5aa40c2040 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.956+0000 7f5aab6e3700 1 -- 172.21.15.29:0/2967128866 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa40bf440 con 0x7f5aa40c2040 2024-01-24T13:58:28.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.957+0000 7f5aab6e3700 1 -- 172.21.15.29:0/2967128866 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa40aa0e0 con 0x7f5aa40c2040 2024-01-24T13:58:28.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.957+0000 7f5a997fa700 1 -- 172.21.15.29:0/2967128866 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5aa0019420 con 0x7f5aa40c2040 2024-01-24T13:58:28.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.958+0000 7f5a997fa700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5a8805f150 0x7f5a88061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:28.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.958+0000 7f5a997fa700 1 -- 172.21.15.29:0/2967128866 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f5aa007b9f0 con 0x7f5aa40c2040 2024-01-24T13:58:28.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.958+0000 7f5aa8c7e700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5a8805f150 0x7f5a88061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:28.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.958+0000 7f5aa8c7e700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5a8805f150 0x7f5a88061610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5a94001710 tx=0x7f5a94009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:28.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:28.962+0000 7f5a997fa700 1 -- 172.21.15.29:0/2967128866 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5aa004c890 con 0x7f5aa40c2040 2024-01-24T13:58:29.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.146+0000 7f5aab6e3700 1 -- 172.21.15.29:0/2967128866 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5aa41512d0 con 0x7f5aa40c2040 2024-01-24T13:58:29.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.147+0000 7f5a997fa700 1 -- 172.21.15.29:0/2967128866 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7f5aa0028090 con 0x7f5aa40c2040 2024-01-24T13:58:29.148 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:29.148 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.149+0000 7f5aab6e3700 1 -- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5a8805f150 msgr2=0x7f5a88061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.149+0000 7f5aab6e3700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5a8805f150 0x7f5a88061610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5a94001710 tx=0x7f5a94009040).stop 2024-01-24T13:58:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.149+0000 7f5aab6e3700 1 -- 172.21.15.29:0/2967128866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5aa40c2040 msgr2=0x7f5aa40be4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.149+0000 7f5aab6e3700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5aa40c2040 0x7f5aa40be4a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f5aa000c4d0 tx=0x7f5aa000a2b0).stop 2024-01-24T13:58:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.149+0000 7f5aab6e3700 1 -- 172.21.15.29:0/2967128866 shutdown_connections 2024-01-24T13:58:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.149+0000 7f5aab6e3700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5aa40be990 0x7f5aa4141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.149+0000 7f5aab6e3700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5a8805f150 0x7f5a88061610 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.149+0000 7f5aab6e3700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5aa40c2040 0x7f5aa40be4a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.149+0000 7f5aab6e3700 1 --2- 172.21.15.29:0/2967128866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5aa414aac0 0x7f5aa414fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.149+0000 7f5aab6e3700 1 -- 172.21.15.29:0/2967128866 >> 172.21.15.29:0/2967128866 conn(0x7f5aa40c69f0 msgr2=0x7f5aa40bc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.149+0000 7f5aab6e3700 1 -- 172.21.15.29:0/2967128866 shutdown_connections 2024-01-24T13:58:29.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.149+0000 7f5aab6e3700 1 -- 172.21.15.29:0/2967128866 wait complete. 2024-01-24T13:58:29.163 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:29.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.293+0000 7f5252230700 1 Processor -- start 2024-01-24T13:58:29.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.293+0000 7f5252230700 1 -- start start 2024-01-24T13:58:29.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.294+0000 7f5252230700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524c062d50 0x7f524c158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:29.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.294+0000 7f5252230700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524c0bcf60 con 0x7f524c062970 2024-01-24T13:58:29.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.294+0000 7f5252230700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524c158af0 con 0x7f524c066b40 2024-01-24T13:58:29.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.294+0000 7f5252230700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0bdba0 con 0x7f524c062d50 2024-01-24T13:58:29.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.294+0000 7f524b7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f524c062970 0x7f524c0c4da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33664/0 (socket says 172.21.15.29:33664) 2024-01-24T13:58:29.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.294+0000 7f524b7fe700 1 -- 172.21.15.29:0/4237099052 learned_addr learned my addr 172.21.15.29:0/4237099052 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:29.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.294+0000 7f524bfff700 1 --2- 172.21.15.29:0/4237099052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524c062d50 0x7f524c158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:29.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.294+0000 7f524a7fc700 1 -- 172.21.15.29:0/4237099052 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019028104 0 0) 0x7f524c0bcf60 con 0x7f524c062970 2024-01-24T13:58:29.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.294+0000 7f524a7fc700 1 -- 172.21.15.29:0/4237099052 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52380033f0 con 0x7f524c062970 2024-01-24T13:58:29.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.295+0000 7f524a7fc700 1 -- 172.21.15.29:0/4237099052 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218601097 0 0) 0x7f524c158af0 con 0x7f524c066b40 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.295+0000 7f524a7fc700 1 -- 172.21.15.29:0/4237099052 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f524c0bcf60 con 0x7f524c066b40 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.295+0000 7f524bfff700 1 -- 172.21.15.29:0/4237099052 >> v1:172.21.15.163:6789/0 conn(0x7f524c066b40 legacy=0x7f524c154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.295+0000 7f524a7fc700 1 -- 172.21.15.29:0/4237099052 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 438054911 0 0) 0x7f52380033f0 con 0x7f524c062970 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.295+0000 7f524bfff700 1 -- 172.21.15.29:0/4237099052 >> v1:172.21.15.29:6789/0 conn(0x7f524c062970 legacy=0x7f524c0c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.295+0000 7f524bfff700 1 -- 172.21.15.29:0/4237099052 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c0a6ec0 con 0x7f524c062d50 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.295+0000 7f524bfff700 1 --2- 172.21.15.29:0/4237099052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524c062d50 0x7f524c158600 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f5244009e90 tx=0x7f5244009780).ready entity=mon.2 client_cookie=3b49c4fce321283f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.295+0000 7f524a7fc700 1 -- 172.21.15.29:0/4237099052 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524400e070 con 0x7f524c062d50 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.295+0000 7f524a7fc700 1 -- 172.21.15.29:0/4237099052 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524400b7a0 con 0x7f524c062d50 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.295+0000 7f524a7fc700 1 -- 172.21.15.29:0/4237099052 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c001470 con 0x7f524c062d50 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.295+0000 7f5252230700 1 -- 172.21.15.29:0/4237099052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524c062d50 msgr2=0x7f524c158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.295+0000 7f5252230700 1 --2- 172.21.15.29:0/4237099052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524c062d50 0x7f524c158600 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f5244009e90 tx=0x7f5244009780).stop 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.296+0000 7f5252230700 1 -- 172.21.15.29:0/4237099052 shutdown_connections 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.296+0000 7f5252230700 1 --2- 172.21.15.29:0/4237099052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524c062d50 0x7f524c158600 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.296+0000 7f5252230700 1 -- 172.21.15.29:0/4237099052 >> 172.21.15.29:0/4237099052 conn(0x7f524c0c8580 msgr2=0x7f524c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:29.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.296+0000 7f5252230700 1 -- 172.21.15.29:0/4237099052 shutdown_connections 2024-01-24T13:58:29.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.296+0000 7f5252230700 1 -- 172.21.15.29:0/4237099052 wait complete. 2024-01-24T13:58:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.297+0000 7f5252230700 1 Processor -- start 2024-01-24T13:58:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.297+0000 7f5252230700 1 -- start start 2024-01-24T13:58:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.298+0000 7f5252230700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524c062970 0x7f524c1679d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.298+0000 7f5252230700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524c062d50 0x7f524c0ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.298+0000 7f5252230700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f524c066b40 0x7f524c0ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.298+0000 7f5252230700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0631b0 con 0x7f524c062970 2024-01-24T13:58:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.298+0000 7f5252230700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0a7480 con 0x7f524c066b40 2024-01-24T13:58:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.298+0000 7f5252230700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0a8b50 con 0x7f524c062d50 2024-01-24T13:58:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.298+0000 7f524b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524c062970 0x7f524c1679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.298+0000 7f524b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524c062970 0x7f524c1679d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51316/0 (socket says 172.21.15.29:51316) 2024-01-24T13:58:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.298+0000 7f524b7fe700 1 -- 172.21.15.29:0/2200419162 learned_addr learned my addr 172.21.15.29:0/2200419162 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:29.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.298+0000 7f524bfff700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f524c066b40 0x7f524c0ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.298+0000 7f524affd700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524c062d50 0x7f524c0ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.299+0000 7f524b7fe700 1 -- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524c062d50 msgr2=0x7f524c0ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.299+0000 7f524b7fe700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524c062d50 0x7f524c0ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.299+0000 7f524b7fe700 1 -- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f524c066b40 msgr2=0x7f524c0ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.299+0000 7f524b7fe700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f524c066b40 0x7f524c0ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.299+0000 7f524b7fe700 1 -- 172.21.15.29:0/2200419162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5244009a20 con 0x7f524c062970 2024-01-24T13:58:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.299+0000 7f524b7fe700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524c062970 0x7f524c1679d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f523c00c800 tx=0x7f523c001ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.299+0000 7f52437fe700 1 -- 172.21.15.29:0/2200419162 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c007170 con 0x7f524c062970 2024-01-24T13:58:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.299+0000 7f52437fe700 1 -- 172.21.15.29:0/2200419162 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c0072d0 con 0x7f524c062970 2024-01-24T13:58:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.299+0000 7f5252230700 1 -- 172.21.15.29:0/2200419162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c168500 con 0x7f524c062970 2024-01-24T13:58:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.299+0000 7f52437fe700 1 -- 172.21.15.29:0/2200419162 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c0048c0 con 0x7f524c062970 2024-01-24T13:58:29.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.299+0000 7f5252230700 1 -- 172.21.15.29:0/2200419162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c0a8860 con 0x7f524c062970 2024-01-24T13:58:29.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.300+0000 7f5252230700 1 -- 172.21.15.29:0/2200419162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c167f80 con 0x7f524c062970 2024-01-24T13:58:29.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.301+0000 7f52437fe700 1 -- 172.21.15.29:0/2200419162 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f523c004c30 con 0x7f524c062970 2024-01-24T13:58:29.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.301+0000 7f52437fe700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f522c05f120 0x7f522c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:29.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.301+0000 7f52437fe700 1 -- 172.21.15.29:0/2200419162 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f523c079dc0 con 0x7f524c062970 2024-01-24T13:58:29.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.301+0000 7f524affd700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f522c05f120 0x7f522c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:29.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.302+0000 7f524affd700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f522c05f120 0x7f522c0615e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f524c0c2060 tx=0x7f5234007bd0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:29.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.305+0000 7f52437fe700 1 -- 172.21.15.29:0/2200419162 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f523c04ad80 con 0x7f524c062970 2024-01-24T13:58:29.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.486+0000 7f5252230700 1 -- 172.21.15.29:0/2200419162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f524c0475e0 con 0x7f524c062970 2024-01-24T13:58:29.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.487+0000 7f52437fe700 1 -- 172.21.15.29:0/2200419162 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7f523c019020 con 0x7f524c062970 2024-01-24T13:58:29.488 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:29.488 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:29.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.489+0000 7f5252230700 1 -- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f522c05f120 msgr2=0x7f522c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.489+0000 7f5252230700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f522c05f120 0x7f522c0615e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f524c0c2060 tx=0x7f5234007bd0).stop 2024-01-24T13:58:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.489+0000 7f5252230700 1 -- 172.21.15.29:0/2200419162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524c062970 msgr2=0x7f524c1679d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.489+0000 7f5252230700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524c062970 0x7f524c1679d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f523c00c800 tx=0x7f523c001ab0).stop 2024-01-24T13:58:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.489+0000 7f5252230700 1 -- 172.21.15.29:0/2200419162 shutdown_connections 2024-01-24T13:58:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.489+0000 7f5252230700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f524c066b40 0x7f524c0ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.489+0000 7f5252230700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f522c05f120 0x7f522c0615e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.489+0000 7f5252230700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f524c062970 0x7f524c1679d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.489+0000 7f5252230700 1 --2- 172.21.15.29:0/2200419162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524c062d50 0x7f524c0ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.489+0000 7f5252230700 1 -- 172.21.15.29:0/2200419162 >> 172.21.15.29:0/2200419162 conn(0x7f524c0c8580 msgr2=0x7f524c0c4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.489+0000 7f5252230700 1 -- 172.21.15.29:0/2200419162 shutdown_connections 2024-01-24T13:58:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.489+0000 7f5252230700 1 -- 172.21.15.29:0/2200419162 wait complete. 2024-01-24T13:58:29.503 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-24T13:58:29.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.636+0000 7ff665b88700 1 Processor -- start 2024-01-24T13:58:29.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.636+0000 7ff665b88700 1 -- start start 2024-01-24T13:58:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff665b88700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6601384f0 0x7ff66006a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff665b88700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6600bcda0 con 0x7ff6600c8270 2024-01-24T13:58:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff665b88700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff660065f80 con 0x7ff660062ec0 2024-01-24T13:58:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff665b88700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff660066260 con 0x7ff6601384f0 2024-01-24T13:58:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff65effd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff6600c8270 0x7ff6600627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33680/0 (socket says 172.21.15.29:33680) 2024-01-24T13:58:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff65effd700 1 -- 172.21.15.29:0/3616139199 learned_addr learned my addr 172.21.15.29:0/3616139199 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff65f7fe700 1 --2- 172.21.15.29:0/3616139199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6601384f0 0x7ff66006a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff65e7fc700 1 -- 172.21.15.29:0/3616139199 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817878652 0 0) 0x7ff6600bcda0 con 0x7ff6600c8270 2024-01-24T13:58:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff65e7fc700 1 -- 172.21.15.29:0/3616139199 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6480033f0 con 0x7ff6600c8270 2024-01-24T13:58:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff65e7fc700 1 -- 172.21.15.29:0/3616139199 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673799492 0 0) 0x7ff660065f80 con 0x7ff660062ec0 2024-01-24T13:58:29.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff65e7fc700 1 -- 172.21.15.29:0/3616139199 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6600bcda0 con 0x7ff660062ec0 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff65f7fe700 1 -- 172.21.15.29:0/3616139199 >> v1:172.21.15.163:6789/0 conn(0x7ff660062ec0 legacy=0x7ff6601391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff65f7fe700 1 -- 172.21.15.29:0/3616139199 >> v1:172.21.15.29:6789/0 conn(0x7ff6600c8270 legacy=0x7ff6600627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.637+0000 7ff65f7fe700 1 -- 172.21.15.29:0/3616139199 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6600bca50 con 0x7ff6601384f0 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.638+0000 7ff65f7fe700 1 --2- 172.21.15.29:0/3616139199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6601384f0 0x7ff66006a460 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff650009790 tx=0x7ff650009060).ready entity=mon.2 client_cookie=2072b0e791ae3e00 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.638+0000 7ff65e7fc700 1 -- 172.21.15.29:0/3616139199 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6500090d0 con 0x7ff6601384f0 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.638+0000 7ff65e7fc700 1 -- 172.21.15.29:0/3616139199 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65000f510 con 0x7ff6601384f0 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.638+0000 7ff65e7fc700 1 -- 172.21.15.29:0/3616139199 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000f890 con 0x7ff6601384f0 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.638+0000 7ff665b88700 1 -- 172.21.15.29:0/3616139199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6601384f0 msgr2=0x7ff66006a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.638+0000 7ff665b88700 1 --2- 172.21.15.29:0/3616139199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6601384f0 0x7ff66006a460 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff650009790 tx=0x7ff650009060).stop 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.638+0000 7ff665b88700 1 -- 172.21.15.29:0/3616139199 shutdown_connections 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.638+0000 7ff665b88700 1 --2- 172.21.15.29:0/3616139199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6601384f0 0x7ff66006a460 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.638+0000 7ff665b88700 1 -- 172.21.15.29:0/3616139199 >> 172.21.15.29:0/3616139199 conn(0x7ff660064390 msgr2=0x7ff6600c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:29.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.638+0000 7ff665b88700 1 -- 172.21.15.29:0/3616139199 shutdown_connections 2024-01-24T13:58:29.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.638+0000 7ff665b88700 1 -- 172.21.15.29:0/3616139199 wait complete. 2024-01-24T13:58:29.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.640+0000 7ff665b88700 1 Processor -- start 2024-01-24T13:58:29.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.640+0000 7ff665b88700 1 -- start start 2024-01-24T13:58:29.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff665b88700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff660062ec0 0x7ff6601519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:29.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff665b88700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6600c8270 0x7ff6600bf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:29.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff665b88700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6601384f0 0x7ff6600a9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:29.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff665b88700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600abab0 con 0x7ff660062ec0 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff665b88700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600bfea0 con 0x7ff6601384f0 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff665b88700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600c0180 con 0x7ff6600c8270 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff65f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff660062ec0 0x7ff6601519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff65f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff660062ec0 0x7ff6601519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51330/0 (socket says 172.21.15.29:51330) 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff65f7fe700 1 -- 172.21.15.29:0/2782426724 learned_addr learned my addr 172.21.15.29:0/2782426724 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff65effd700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6600c8270 0x7ff6600bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff65ffff700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6601384f0 0x7ff6600a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff65f7fe700 1 -- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6600c8270 msgr2=0x7ff6600bf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff65f7fe700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6600c8270 0x7ff6600bf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff65f7fe700 1 -- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6601384f0 msgr2=0x7ff6600a9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff65f7fe700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6601384f0 0x7ff6600a9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff65f7fe700 1 -- 172.21.15.29:0/2782426724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6500092a0 con 0x7ff660062ec0 2024-01-24T13:58:29.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff65ffff700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6601384f0 0x7ff6600a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:29.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff65f7fe700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff660062ec0 0x7ff6601519c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7ff660064910 tx=0x7ff650017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:29.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.641+0000 7ff6477fe700 1 -- 172.21.15.29:0/2782426724 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000f890 con 0x7ff660062ec0 2024-01-24T13:58:29.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.642+0000 7ff6477fe700 1 -- 172.21.15.29:0/2782426724 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff650000cc0 con 0x7ff660062ec0 2024-01-24T13:58:29.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.642+0000 7ff6477fe700 1 -- 172.21.15.29:0/2782426724 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000fea0 con 0x7ff660062ec0 2024-01-24T13:58:29.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.642+0000 7ff665b88700 1 -- 172.21.15.29:0/2782426724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6600c0460 con 0x7ff660062ec0 2024-01-24T13:58:29.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.642+0000 7ff665b88700 1 -- 172.21.15.29:0/2782426724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6600ab640 con 0x7ff660062ec0 2024-01-24T13:58:29.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.643+0000 7ff665b88700 1 -- 172.21.15.29:0/2782426724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6600ab860 con 0x7ff660062ec0 2024-01-24T13:58:29.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.643+0000 7ff6477fe700 1 -- 172.21.15.29:0/2782426724 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff65001b620 con 0x7ff660062ec0 2024-01-24T13:58:29.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.643+0000 7ff6477fe700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff64c05f150 0x7ff64c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:29.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.644+0000 7ff6477fe700 1 -- 172.21.15.29:0/2782426724 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7ff65007bc30 con 0x7ff660062ec0 2024-01-24T13:58:29.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.644+0000 7ff65effd700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff64c05f150 0x7ff64c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:29.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.644+0000 7ff65effd700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff64c05f150 0x7ff64c061610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7ff6600adda0 tx=0x7ff654009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:29.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.648+0000 7ff6477fe700 1 -- 172.21.15.29:0/2782426724 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff65004cbf0 con 0x7ff660062ec0 2024-01-24T13:58:29.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:29.879+0000 7ff665b88700 1 -- 172.21.15.29:0/2782426724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff6600475e0 con 0x7ff660062ec0 2024-01-24T13:58:30.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.232+0000 7ff6477fe700 1 -- 172.21.15.29:0/2782426724 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff650029020 con 0x7ff660062ec0 2024-01-24T13:58:30.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.234+0000 7ff665b88700 1 -- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff64c05f150 msgr2=0x7ff64c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:30.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.234+0000 7ff665b88700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff64c05f150 0x7ff64c061610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7ff6600adda0 tx=0x7ff654009040).stop 2024-01-24T13:58:30.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.234+0000 7ff665b88700 1 -- 172.21.15.29:0/2782426724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff660062ec0 msgr2=0x7ff6601519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:30.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.234+0000 7ff665b88700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff660062ec0 0x7ff6601519c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7ff660064910 tx=0x7ff650017000).stop 2024-01-24T13:58:30.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.234+0000 7ff665b88700 1 -- 172.21.15.29:0/2782426724 shutdown_connections 2024-01-24T13:58:30.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.234+0000 7ff665b88700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6601384f0 0x7ff6600a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:30.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.234+0000 7ff665b88700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff64c05f150 0x7ff64c061610 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:30.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.234+0000 7ff665b88700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff660062ec0 0x7ff6601519c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:30.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.234+0000 7ff665b88700 1 --2- 172.21.15.29:0/2782426724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6600c8270 0x7ff6600bf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.234+0000 7ff665b88700 1 -- 172.21.15.29:0/2782426724 >> 172.21.15.29:0/2782426724 conn(0x7ff660064390 msgr2=0x7ff6600627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.234+0000 7ff665b88700 1 -- 172.21.15.29:0/2782426724 shutdown_connections 2024-01-24T13:58:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.235+0000 7ff665b88700 1 -- 172.21.15.29:0/2782426724 wait complete. 2024-01-24T13:58:30.250 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-01-24T13:58:30.251 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-24T13:58:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.386+0000 7f9069848700 1 Processor -- start 2024-01-24T13:58:30.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.386+0000 7f9069848700 1 -- start start 2024-01-24T13:58:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.387+0000 7f9069848700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90640664c0 0x7f9064066920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.387+0000 7f9069848700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90640640d0 con 0x7f906413a5e0 2024-01-24T13:58:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.387+0000 7f9069848700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9064066e10 con 0x7f90640bdb30 2024-01-24T13:58:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.387+0000 7f9069848700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90640679f0 con 0x7f90640664c0 2024-01-24T13:58:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.387+0000 7f9062ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f906413a5e0 0x7f90640639c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33690/0 (socket says 172.21.15.29:33690) 2024-01-24T13:58:30.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.387+0000 7f9062ffd700 1 -- 172.21.15.29:0/1079991852 learned_addr learned my addr 172.21.15.29:0/1079991852 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.387+0000 7f90627fc700 1 --2- 172.21.15.29:0/1079991852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90640664c0 0x7f9064066920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.388+0000 7f9061ffb700 1 -- 172.21.15.29:0/1079991852 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357633596 0 0) 0x7f90640640d0 con 0x7f906413a5e0 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.388+0000 7f9061ffb700 1 -- 172.21.15.29:0/1079991852 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90500033f0 con 0x7f906413a5e0 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.388+0000 7f9061ffb700 1 -- 172.21.15.29:0/1079991852 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3872334837 0 0) 0x7f9064066e10 con 0x7f90640bdb30 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.388+0000 7f9061ffb700 1 -- 172.21.15.29:0/1079991852 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90640640d0 con 0x7f90640bdb30 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.388+0000 7f9061ffb700 1 -- 172.21.15.29:0/1079991852 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1790652143 0 0) 0x7f90500033f0 con 0x7f906413a5e0 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.388+0000 7f90627fc700 1 -- 172.21.15.29:0/1079991852 >> v1:172.21.15.163:6789/0 conn(0x7f90640bdb30 legacy=0x7f90640bdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.388+0000 7f90627fc700 1 -- 172.21.15.29:0/1079991852 >> v1:172.21.15.29:6789/0 conn(0x7f906413a5e0 legacy=0x7f90640639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.388+0000 7f90627fc700 1 -- 172.21.15.29:0/1079991852 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9064067cd0 con 0x7f90640664c0 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.388+0000 7f90627fc700 1 --2- 172.21.15.29:0/1079991852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90640664c0 0x7f9064066920 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f904c009790 tx=0x7f904c009060).ready entity=mon.2 client_cookie=f9091c1626057397 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.388+0000 7f9061ffb700 1 -- 172.21.15.29:0/1079991852 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054001470 con 0x7f90640664c0 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.388+0000 7f9061ffb700 1 -- 172.21.15.29:0/1079991852 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c011400 con 0x7f90640664c0 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.388+0000 7f9061ffb700 1 -- 172.21.15.29:0/1079991852 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c011780 con 0x7f90640664c0 2024-01-24T13:58:30.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.389+0000 7f9069848700 1 -- 172.21.15.29:0/1079991852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90640664c0 msgr2=0x7f9064066920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:30.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.389+0000 7f9069848700 1 --2- 172.21.15.29:0/1079991852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90640664c0 0x7f9064066920 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f904c009790 tx=0x7f904c009060).stop 2024-01-24T13:58:30.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.389+0000 7f9069848700 1 -- 172.21.15.29:0/1079991852 shutdown_connections 2024-01-24T13:58:30.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.389+0000 7f9069848700 1 --2- 172.21.15.29:0/1079991852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90640664c0 0x7f9064066920 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:30.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.389+0000 7f9069848700 1 -- 172.21.15.29:0/1079991852 >> 172.21.15.29:0/1079991852 conn(0x7f90640c8bd0 msgr2=0x7f90640c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:30.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.389+0000 7f9069848700 1 -- 172.21.15.29:0/1079991852 shutdown_connections 2024-01-24T13:58:30.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.389+0000 7f9069848700 1 -- 172.21.15.29:0/1079991852 wait complete. 2024-01-24T13:58:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.390+0000 7f9069848700 1 Processor -- start 2024-01-24T13:58:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.390+0000 7f9069848700 1 -- start start 2024-01-24T13:58:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.391+0000 7f9069848700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90640664c0 0x7f9064137200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.391+0000 7f9069848700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90640bdb30 0x7f90640ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.391+0000 7f9069848700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f906413a5e0 0x7f90640ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.391+0000 7f9069848700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90641643c0 con 0x7f90640664c0 2024-01-24T13:58:30.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.391+0000 7f9069848700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9064165210 con 0x7f906413a5e0 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.391+0000 7f9069848700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9064164600 con 0x7f90640bdb30 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.391+0000 7f9062ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90640664c0 0x7f9064137200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.391+0000 7f9062ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90640664c0 0x7f9064137200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51344/0 (socket says 172.21.15.29:51344) 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.391+0000 7f9062ffd700 1 -- 172.21.15.29:0/2340800028 learned_addr learned my addr 172.21.15.29:0/2340800028 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.391+0000 7f90627fc700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90640bdb30 0x7f90640ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.391+0000 7f90637fe700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f906413a5e0 0x7f90640ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.392+0000 7f9062ffd700 1 -- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90640bdb30 msgr2=0x7f90640ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.392+0000 7f9062ffd700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90640bdb30 0x7f90640ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.392+0000 7f9062ffd700 1 -- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f906413a5e0 msgr2=0x7f90640ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.392+0000 7f9062ffd700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f906413a5e0 0x7f90640ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.392+0000 7f9062ffd700 1 -- 172.21.15.29:0/2340800028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c0092a0 con 0x7f90640664c0 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.392+0000 7f9062ffd700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90640664c0 0x7f9064137200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f905400c800 tx=0x7f9054001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.392+0000 7f905affd700 1 -- 172.21.15.29:0/2340800028 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054007170 con 0x7f90640664c0 2024-01-24T13:58:30.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.392+0000 7f905affd700 1 -- 172.21.15.29:0/2340800028 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90540072d0 con 0x7f90640664c0 2024-01-24T13:58:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.392+0000 7f9069848700 1 -- 172.21.15.29:0/2340800028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90641648e0 con 0x7f90640664c0 2024-01-24T13:58:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.392+0000 7f905affd700 1 -- 172.21.15.29:0/2340800028 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540048c0 con 0x7f90640664c0 2024-01-24T13:58:30.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.392+0000 7f9069848700 1 -- 172.21.15.29:0/2340800028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9064164ea0 con 0x7f90640664c0 2024-01-24T13:58:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.393+0000 7f9069848700 1 -- 172.21.15.29:0/2340800028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90640475e0 con 0x7f90640664c0 2024-01-24T13:58:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.393+0000 7f905affd700 1 -- 172.21.15.29:0/2340800028 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9054004c30 con 0x7f90640664c0 2024-01-24T13:58:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.394+0000 7f905affd700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f904405f150 0x7f9044061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.394+0000 7f905affd700 1 -- 172.21.15.29:0/2340800028 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f9054079e50 con 0x7f90640664c0 2024-01-24T13:58:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.394+0000 7f90627fc700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f904405f150 0x7f9044061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:30.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.395+0000 7f90627fc700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f904405f150 0x7f9044061610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f904c0096e0 tx=0x7f904c016000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:30.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.398+0000 7f905affd700 1 -- 172.21.15.29:0/2340800028 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f905404ae10 con 0x7f90640664c0 2024-01-24T13:58:30.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:30.627+0000 7f9069848700 1 -- 172.21.15.29:0/2340800028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9064067fb0 con 0x7f90640664c0 2024-01-24T13:58:31.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.237+0000 7f905affd700 1 -- 172.21.15.29:0/2340800028 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9054019020 con 0x7f90640664c0 2024-01-24T13:58:31.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.239+0000 7f9069848700 1 -- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f904405f150 msgr2=0x7f9044061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.239+0000 7f9069848700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f904405f150 0x7f9044061610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f904c0096e0 tx=0x7f904c016000).stop 2024-01-24T13:58:31.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.239+0000 7f9069848700 1 -- 172.21.15.29:0/2340800028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90640664c0 msgr2=0x7f9064137200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.239+0000 7f9069848700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90640664c0 0x7f9064137200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f905400c800 tx=0x7f9054001b50).stop 2024-01-24T13:58:31.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.239+0000 7f9069848700 1 -- 172.21.15.29:0/2340800028 shutdown_connections 2024-01-24T13:58:31.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.239+0000 7f9069848700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f906413a5e0 0x7f90640ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.240+0000 7f9069848700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f904405f150 0x7f9044061610 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.240+0000 7f9069848700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90640664c0 0x7f9064137200 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.240+0000 7f9069848700 1 --2- 172.21.15.29:0/2340800028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90640bdb30 0x7f90640ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.240+0000 7f9069848700 1 -- 172.21.15.29:0/2340800028 >> 172.21.15.29:0/2340800028 conn(0x7f90640c8bd0 msgr2=0x7f90641567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:31.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.240+0000 7f9069848700 1 -- 172.21.15.29:0/2340800028 shutdown_connections 2024-01-24T13:58:31.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.240+0000 7f9069848700 1 -- 172.21.15.29:0/2340800028 wait complete. 2024-01-24T13:58:31.256 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-24T13:58:31.256 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:58:31.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.389+0000 7fad26a3c700 1 Processor -- start 2024-01-24T13:58:31.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.389+0000 7fad26a3c700 1 -- start start 2024-01-24T13:58:31.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad26a3c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fad200c4a10 0x7fad200bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad26a3c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fad200670a0 0x7fad20158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad26a3c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad201582b0 con 0x7fad200664f0 2024-01-24T13:58:31.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad26a3c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad20157ca0 con 0x7fad200c4a10 2024-01-24T13:58:31.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad26a3c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad20063c60 con 0x7fad200670a0 2024-01-24T13:58:31.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad1ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fad200c4a10 0x7fad200bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad1ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fad200c4a10 0x7fad200bf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51346/0 (socket says 172.21.15.29:51346) 2024-01-24T13:58:31.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad24fd9700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fad200664f0 0x7fad200bdb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33702/0 (socket says 172.21.15.29:33702) 2024-01-24T13:58:31.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad1ffff700 1 -- 172.21.15.29:0/1641079748 learned_addr learned my addr 172.21.15.29:0/1641079748 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad1f7fe700 1 --2- 172.21.15.29:0/1641079748 >> v2:172.21.15.163:3300/0 conn(0x7fad200670a0 0x7fad20158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad1ffff700 1 -- 172.21.15.29:0/1641079748 >> v2:172.21.15.163:3300/0 conn(0x7fad200670a0 msgr2=0x7fad20158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad1ffff700 1 --2- 172.21.15.29:0/1641079748 >> v2:172.21.15.163:3300/0 conn(0x7fad200670a0 0x7fad20158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.390+0000 7fad1ffff700 1 -- 172.21.15.29:0/1641079748 >> v1:172.21.15.29:6789/0 conn(0x7fad200664f0 legacy=0x7fad200bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad1ffff700 1 -- 172.21.15.29:0/1641079748 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad20157f40 con 0x7fad200c4a10 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad1f7fe700 1 --2- 172.21.15.29:0/1641079748 >> v2:172.21.15.163:3300/0 conn(0x7fad200670a0 0x7fad20158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad1ffff700 1 --2- 172.21.15.29:0/1641079748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad200c4a10 0x7fad200bf6d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fad20068cb0 tx=0x7fad100090f0).ready entity=mon.0 client_cookie=82d0e97cb7e08537 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad1effd700 1 -- 172.21.15.29:0/1641079748 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1000b600 con 0x7fad200c4a10 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad1effd700 1 -- 172.21.15.29:0/1641079748 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1000b760 con 0x7fad200c4a10 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad1effd700 1 -- 172.21.15.29:0/1641079748 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1000bae0 con 0x7fad200c4a10 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad26a3c700 1 -- 172.21.15.29:0/1641079748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad200c4a10 msgr2=0x7fad200bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad26a3c700 1 --2- 172.21.15.29:0/1641079748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad200c4a10 0x7fad200bf6d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fad20068cb0 tx=0x7fad100090f0).stop 2024-01-24T13:58:31.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad26a3c700 1 -- 172.21.15.29:0/1641079748 shutdown_connections 2024-01-24T13:58:31.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad26a3c700 1 --2- 172.21.15.29:0/1641079748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad200c4a10 0x7fad200bf6d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad26a3c700 1 --2- 172.21.15.29:0/1641079748 >> v2:172.21.15.163:3300/0 conn(0x7fad200670a0 0x7fad20158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad26a3c700 1 -- 172.21.15.29:0/1641079748 >> 172.21.15.29:0/1641079748 conn(0x7fad200c8bb0 msgr2=0x7fad200c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:31.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.391+0000 7fad26a3c700 1 -- 172.21.15.29:0/1641079748 shutdown_connections 2024-01-24T13:58:31.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.392+0000 7fad26a3c700 1 -- 172.21.15.29:0/1641079748 wait complete. 2024-01-24T13:58:31.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.393+0000 7fad26a3c700 1 Processor -- start 2024-01-24T13:58:31.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.393+0000 7fad26a3c700 1 -- start start 2024-01-24T13:58:31.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad26a3c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad200664f0 0x7fad201647c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad26a3c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad200670a0 0x7fad200bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad26a3c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad200c4a10 0x7fad200c2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad26a3c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200c16a0 con 0x7fad200670a0 2024-01-24T13:58:31.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad26a3c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad20164cb0 con 0x7fad200c4a10 2024-01-24T13:58:31.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad26a3c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad200c07d0 con 0x7fad200664f0 2024-01-24T13:58:31.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad1f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad200670a0 0x7fad200bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad1f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad200670a0 0x7fad200bfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51356/0 (socket says 172.21.15.29:51356) 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad1f7fe700 1 -- 172.21.15.29:0/3105361613 learned_addr learned my addr 172.21.15.29:0/3105361613 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad1ffff700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad200664f0 0x7fad201647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad24fd9700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad200c4a10 0x7fad200c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad1f7fe700 1 -- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad200664f0 msgr2=0x7fad201647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad1f7fe700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad200664f0 0x7fad201647c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad1f7fe700 1 -- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad200c4a10 msgr2=0x7fad200c2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad1f7fe700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad200c4a10 0x7fad200c2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.394+0000 7fad1f7fe700 1 -- 172.21.15.29:0/3105361613 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad10009370 con 0x7fad200670a0 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.395+0000 7fad1f7fe700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad200670a0 0x7fad200bfe40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fad1400d860 tx=0x7fad14008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.395+0000 7fad07fff700 1 -- 172.21.15.29:0/3105361613 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1400f520 con 0x7fad200670a0 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.395+0000 7fad07fff700 1 -- 172.21.15.29:0/3105361613 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1400f680 con 0x7fad200670a0 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.395+0000 7fad26a3c700 1 -- 172.21.15.29:0/3105361613 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad200c0a90 con 0x7fad200670a0 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.395+0000 7fad07fff700 1 -- 172.21.15.29:0/3105361613 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1400fa00 con 0x7fad200670a0 2024-01-24T13:58:31.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.395+0000 7fad26a3c700 1 -- 172.21.15.29:0/3105361613 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad200c1050 con 0x7fad200670a0 2024-01-24T13:58:31.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.396+0000 7fad26a3c700 1 -- 172.21.15.29:0/3105361613 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad200c1810 con 0x7fad200670a0 2024-01-24T13:58:31.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.396+0000 7fad07fff700 1 -- 172.21.15.29:0/3105361613 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fad14013480 con 0x7fad200670a0 2024-01-24T13:58:31.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.397+0000 7fad07fff700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad0c05f150 0x7fad0c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.397+0000 7fad07fff700 1 -- 172.21.15.29:0/3105361613 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fad1407a100 con 0x7fad200670a0 2024-01-24T13:58:31.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.397+0000 7fad1ffff700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad0c05f150 0x7fad0c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.398+0000 7fad1ffff700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad0c05f150 0x7fad0c061610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fad10009fd0 tx=0x7fad10017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:31.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.401+0000 7fad07fff700 1 -- 172.21.15.29:0/3105361613 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fad1404b060 con 0x7fad200670a0 2024-01-24T13:58:31.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.632+0000 7fad26a3c700 1 -- 172.21.15.29:0/3105361613 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad200475e0 con 0x7fad200670a0 2024-01-24T13:58:31.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.632+0000 7fad07fff700 1 -- 172.21.15.29:0/3105361613 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4491 (secure 0 0 0) 0x7fad1400b3f0 con 0x7fad200670a0 2024-01-24T13:58:31.633 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:31.633 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2038321939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2038321939},{"type":"v1","addr":"172.21.15.163:6836","nonce":2038321939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3675672874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3675672874},{"type":"v1","addr":"172.21.15.163:6837","nonce":3675672874}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1532435957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532435957},{"type":"v1","addr":"172.21.15.29:6834","nonce":1532435957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:58:19.190134+0000","modified":"2024-01-24T13:58:31.231527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1328130656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1328130656},{"type":"v1","addr":"172.21.15.29:6835","nonce":1328130656}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-24T13:58:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.635+0000 7fad26a3c700 1 -- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad0c05f150 msgr2=0x7fad0c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.635+0000 7fad26a3c700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad0c05f150 0x7fad0c061610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fad10009fd0 tx=0x7fad10017000).stop 2024-01-24T13:58:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.635+0000 7fad26a3c700 1 -- 172.21.15.29:0/3105361613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad200670a0 msgr2=0x7fad200bfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.635+0000 7fad26a3c700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad200670a0 0x7fad200bfe40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fad1400d860 tx=0x7fad14008040).stop 2024-01-24T13:58:31.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.635+0000 7fad26a3c700 1 -- 172.21.15.29:0/3105361613 shutdown_connections 2024-01-24T13:58:31.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.635+0000 7fad26a3c700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad200c4a10 0x7fad200c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.635+0000 7fad26a3c700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad200664f0 0x7fad201647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.635+0000 7fad26a3c700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad0c05f150 0x7fad0c061610 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.635+0000 7fad26a3c700 1 --2- 172.21.15.29:0/3105361613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad200670a0 0x7fad200bfe40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.635+0000 7fad26a3c700 1 -- 172.21.15.29:0/3105361613 >> 172.21.15.29:0/3105361613 conn(0x7fad200c8bb0 msgr2=0x7fad20066980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:31.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.635+0000 7fad26a3c700 1 -- 172.21.15.29:0/3105361613 shutdown_connections 2024-01-24T13:58:31.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.635+0000 7fad26a3c700 1 -- 172.21.15.29:0/3105361613 wait complete. 2024-01-24T13:58:31.636 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 9 2024-01-24T13:58:31.649 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:31.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.779+0000 7fc4e8a31700 1 Processor -- start 2024-01-24T13:58:31.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.779+0000 7fc4e8a31700 1 -- start start 2024-01-24T13:58:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e8a31700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc4e4062970 0x7fc4e4062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e8a31700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4e4066b40 0x7fc4e4158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e8a31700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e40c4da0 0x7fc4e4069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e8a31700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e40bdba0 con 0x7fc4e4066b40 2024-01-24T13:58:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e8a31700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e40a7480 con 0x7fc4e4062970 2024-01-24T13:58:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e8a31700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40bcf60 con 0x7fc4e40c4da0 2024-01-24T13:58:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e1d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4e4066b40 0x7fc4e4158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e1d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc4e4066b40 0x7fc4e4158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51358/0 (socket says 172.21.15.29:51358) 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e1d9b700 1 -- 172.21.15.29:0/3675893423 learned_addr learned my addr 172.21.15.29:0/3675893423 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e2d9d700 1 --2- 172.21.15.29:0/3675893423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e40c4da0 0x7fc4e4069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e259c700 1 --2- 172.21.15.29:0/3675893423 >> v2:172.21.15.163:3300/0 conn(0x7fc4e4062970 0x7fc4e4062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e1d9b700 1 -- 172.21.15.29:0/3675893423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e40c4da0 msgr2=0x7fc4e4069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e1d9b700 1 --2- 172.21.15.29:0/3675893423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e40c4da0 0x7fc4e4069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e1d9b700 1 -- 172.21.15.29:0/3675893423 >> v2:172.21.15.163:3300/0 conn(0x7fc4e4062970 msgr2=0x7fc4e4062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e1d9b700 1 --2- 172.21.15.29:0/3675893423 >> v2:172.21.15.163:3300/0 conn(0x7fc4e4062970 0x7fc4e4062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.780+0000 7fc4e1d9b700 1 -- 172.21.15.29:0/3675893423 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e4158af0 con 0x7fc4e4066b40 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e1d9b700 1 --2- 172.21.15.29:0/3675893423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4e4066b40 0x7fc4e4158600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fc4e4069140 tx=0x7fc4d80090f0).ready entity=mon.0 client_cookie=28654fc5051b8aae server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e159a700 1 -- 172.21.15.29:0/3675893423 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800f510 con 0x7fc4e4066b40 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e159a700 1 -- 172.21.15.29:0/3675893423 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d800f670 con 0x7fc4e4066b40 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e159a700 1 -- 172.21.15.29:0/3675893423 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800fa10 con 0x7fc4e4066b40 2024-01-24T13:58:31.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e8a31700 1 -- 172.21.15.29:0/3675893423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4e4066b40 msgr2=0x7fc4e4158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e8a31700 1 --2- 172.21.15.29:0/3675893423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4e4066b40 0x7fc4e4158600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fc4e4069140 tx=0x7fc4d80090f0).stop 2024-01-24T13:58:31.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e8a31700 1 -- 172.21.15.29:0/3675893423 shutdown_connections 2024-01-24T13:58:31.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e8a31700 1 --2- 172.21.15.29:0/3675893423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e40c4da0 0x7fc4e4069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e8a31700 1 --2- 172.21.15.29:0/3675893423 >> v2:172.21.15.163:3300/0 conn(0x7fc4e4062970 0x7fc4e4062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e8a31700 1 --2- 172.21.15.29:0/3675893423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4e4066b40 0x7fc4e4158600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e8a31700 1 -- 172.21.15.29:0/3675893423 >> 172.21.15.29:0/3675893423 conn(0x7fc4e40c8580 msgr2=0x7fc4e40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:31.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e8a31700 1 -- 172.21.15.29:0/3675893423 shutdown_connections 2024-01-24T13:58:31.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.781+0000 7fc4e8a31700 1 -- 172.21.15.29:0/3675893423 wait complete. 2024-01-24T13:58:31.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.783+0000 7fc4e8a31700 1 Processor -- start 2024-01-24T13:58:31.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.783+0000 7fc4e8a31700 1 -- start start 2024-01-24T13:58:31.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.783+0000 7fc4e8a31700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4e4062970 0x7fc4e4143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.783+0000 7fc4e8a31700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4e4066b40 0x7fc4e40bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.783+0000 7fc4e8a31700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e40c4da0 0x7fc4e40c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.783+0000 7fc4e8a31700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e4144420 con 0x7fc4e4066b40 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.783+0000 7fc4e8a31700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40a8090 con 0x7fc4e4062970 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.783+0000 7fc4e8a31700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40c0690 con 0x7fc4e40c4da0 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e1d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4e4066b40 0x7fc4e40bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e1d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4e4066b40 0x7fc4e40bfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51362/0 (socket says 172.21.15.29:51362) 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e1d9b700 1 -- 172.21.15.29:0/2471395934 learned_addr learned my addr 172.21.15.29:0/2471395934 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e2d9d700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e40c4da0 0x7fc4e40c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e259c700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4e4062970 0x7fc4e4143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e1d9b700 1 -- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e40c4da0 msgr2=0x7fc4e40c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e1d9b700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e40c4da0 0x7fc4e40c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e1d9b700 1 -- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4e4062970 msgr2=0x7fc4e4143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e1d9b700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4e4062970 0x7fc4e4143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e1d9b700 1 -- 172.21.15.29:0/2471395934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d8009370 con 0x7fc4e4066b40 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e259c700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4e4062970 0x7fc4e4143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e1d9b700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4e4066b40 0x7fc4e40bfdb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fc4e4069030 tx=0x7fc4d8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4d27fc700 1 -- 172.21.15.29:0/2471395934 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800fea0 con 0x7fc4e4066b40 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4d27fc700 1 -- 172.21.15.29:0/2471395934 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d800f7e0 con 0x7fc4e4066b40 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e8a31700 1 -- 172.21.15.29:0/2471395934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e40c0970 con 0x7fc4e4066b40 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4d27fc700 1 -- 172.21.15.29:0/2471395934 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8024470 con 0x7fc4e4066b40 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.784+0000 7fc4e8a31700 1 -- 172.21.15.29:0/2471395934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e40c0f30 con 0x7fc4e4066b40 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.785+0000 7fc4e8a31700 1 -- 172.21.15.29:0/2471395934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e40475e0 con 0x7fc4e4066b40 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.786+0000 7fc4d27fc700 1 -- 172.21.15.29:0/2471395934 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc4d8021070 con 0x7fc4e4066b40 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.786+0000 7fc4d27fc700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc4c405f150 0x7fc4c4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.786+0000 7fc4d27fc700 1 -- 172.21.15.29:0/2471395934 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fc4d807bf00 con 0x7fc4e4066b40 2024-01-24T13:58:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.786+0000 7fc4e259c700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc4c405f150 0x7fc4c4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:31.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.787+0000 7fc4e259c700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc4c405f150 0x7fc4c4061610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fc4d4006d80 tx=0x7fc4d4008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:31.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.791+0000 7fc4d27fc700 1 -- 172.21.15.29:0/2471395934 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc4d804cec0 con 0x7fc4e4066b40 2024-01-24T13:58:31.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.973+0000 7fc4e8a31700 1 -- 172.21.15.29:0/2471395934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4e40c5700 con 0x7fc4e4066b40 2024-01-24T13:58:31.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.974+0000 7fc4d27fc700 1 -- 172.21.15.29:0/2471395934 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7fc4d80504e0 con 0x7fc4e4066b40 2024-01-24T13:58:31.974 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:31.975 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:31.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.976+0000 7fc4e8a31700 1 -- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc4c405f150 msgr2=0x7fc4c4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.976+0000 7fc4e8a31700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc4c405f150 0x7fc4c4061610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fc4d4006d80 tx=0x7fc4d4008040).stop 2024-01-24T13:58:31.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.976+0000 7fc4e8a31700 1 -- 172.21.15.29:0/2471395934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4e4066b40 msgr2=0x7fc4e40bfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:31.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.976+0000 7fc4e8a31700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4e4066b40 0x7fc4e40bfdb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fc4e4069030 tx=0x7fc4d8017000).stop 2024-01-24T13:58:31.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.976+0000 7fc4e8a31700 1 -- 172.21.15.29:0/2471395934 shutdown_connections 2024-01-24T13:58:31.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.976+0000 7fc4e8a31700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e40c4da0 0x7fc4e40c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.976+0000 7fc4e8a31700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4e4062970 0x7fc4e4143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.976+0000 7fc4e8a31700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc4c405f150 0x7fc4c4061610 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.976+0000 7fc4e8a31700 1 --2- 172.21.15.29:0/2471395934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4e4066b40 0x7fc4e40bfdb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:31.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.976+0000 7fc4e8a31700 1 -- 172.21.15.29:0/2471395934 >> 172.21.15.29:0/2471395934 conn(0x7fc4e40c8580 msgr2=0x7fc4e4144c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:31.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.976+0000 7fc4e8a31700 1 -- 172.21.15.29:0/2471395934 shutdown_connections 2024-01-24T13:58:31.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:31.976+0000 7fc4e8a31700 1 -- 172.21.15.29:0/2471395934 wait complete. 2024-01-24T13:58:31.990 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-24T13:58:31.991 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T13:58:32.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.120+0000 7fd7c25b3700 1 Processor -- start 2024-01-24T13:58:32.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.121+0000 7fd7c25b3700 1 -- start start 2024-01-24T13:58:32.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.121+0000 7fd7c25b3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7bc0c1f80 0x7fd7bc0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.121+0000 7fd7c25b3700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7bc062ca0 0x7fd7bc0bc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.121+0000 7fd7c25b3700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7bc0640f0 con 0x7fd7bc14e840 2024-01-24T13:58:32.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.121+0000 7fd7c25b3700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7bc0634e0 con 0x7fd7bc062ca0 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.121+0000 7fd7c25b3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc0637c0 con 0x7fd7bc0c1f80 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7c0b50700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7bc062ca0 0x7fd7bc0bc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7c0b50700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7bc062ca0 0x7fd7bc0bc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51376/0 (socket says 172.21.15.29:51376) 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7c0b50700 1 -- 172.21.15.29:0/2194273841 learned_addr learned my addr 172.21.15.29:0/2194273841 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7bbfff700 1 --1- 172.21.15.29:0/2194273841 >> v1:172.21.15.29:6789/0 conn(0x7fd7bc14e840 0x7fd7bc0c1870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33708/0 (socket says 172.21.15.29:33708) 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7bb7fe700 1 --2- 172.21.15.29:0/2194273841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7bc0c1f80 0x7fd7bc0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7c0b50700 1 -- 172.21.15.29:0/2194273841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7bc0c1f80 msgr2=0x7fd7bc0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7c0b50700 1 --2- 172.21.15.29:0/2194273841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7bc0c1f80 0x7fd7bc0627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7c0b50700 1 -- 172.21.15.29:0/2194273841 >> v1:172.21.15.29:6789/0 conn(0x7fd7bc14e840 legacy=0x7fd7bc0c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7c0b50700 1 -- 172.21.15.29:0/2194273841 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc063aa0 con 0x7fd7bc062ca0 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7baffd700 1 -- 172.21.15.29:0/2194273841 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758800958 0 0) 0x7fd7bc0640f0 con 0x7fd7bc14e840 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7c0b50700 1 --2- 172.21.15.29:0/2194273841 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7bc062ca0 0x7fd7bc0bc4b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fd7b4009f20 tx=0x7fd7b4009840).ready entity=mon.0 client_cookie=9ad49de70e454422 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:32.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7baffd700 1 -- 172.21.15.29:0/2194273841 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b400e070 con 0x7fd7bc062ca0 2024-01-24T13:58:32.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7baffd700 1 -- 172.21.15.29:0/2194273841 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b400ba80 con 0x7fd7bc062ca0 2024-01-24T13:58:32.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.122+0000 7fd7baffd700 1 -- 172.21.15.29:0/2194273841 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b400be00 con 0x7fd7bc062ca0 2024-01-24T13:58:32.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.123+0000 7fd7c25b3700 1 -- 172.21.15.29:0/2194273841 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7bc062ca0 msgr2=0x7fd7bc0bc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.123+0000 7fd7c25b3700 1 --2- 172.21.15.29:0/2194273841 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7bc062ca0 0x7fd7bc0bc4b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fd7b4009f20 tx=0x7fd7b4009840).stop 2024-01-24T13:58:32.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.123+0000 7fd7c25b3700 1 -- 172.21.15.29:0/2194273841 shutdown_connections 2024-01-24T13:58:32.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.123+0000 7fd7c25b3700 1 --2- 172.21.15.29:0/2194273841 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7bc062ca0 0x7fd7bc0bc4b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.123+0000 7fd7c25b3700 1 --2- 172.21.15.29:0/2194273841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7bc0c1f80 0x7fd7bc0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.123+0000 7fd7c25b3700 1 -- 172.21.15.29:0/2194273841 >> 172.21.15.29:0/2194273841 conn(0x7fd7bc064aa0 msgr2=0x7fd7bc064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:32.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.123+0000 7fd7c25b3700 1 -- 172.21.15.29:0/2194273841 shutdown_connections 2024-01-24T13:58:32.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.123+0000 7fd7c25b3700 1 -- 172.21.15.29:0/2194273841 wait complete. 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.124+0000 7fd7c25b3700 1 Processor -- start 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.124+0000 7fd7c25b3700 1 -- start start 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.125+0000 7fd7c25b3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7bc062ca0 0x7fd7bc0a85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.125+0000 7fd7c25b3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7bc0c1f80 0x7fd7bc0a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.125+0000 7fd7c25b3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7bc14e840 0x7fd7bc0a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.125+0000 7fd7c25b3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc0a8fb0 con 0x7fd7bc14e840 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.125+0000 7fd7c25b3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc069190 con 0x7fd7bc062ca0 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.125+0000 7fd7c25b3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc0a7bf0 con 0x7fd7bc0c1f80 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.125+0000 7fd7c0b50700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7bc14e840 0x7fd7bc0a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.125+0000 7fd7c0b50700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7bc14e840 0x7fd7bc0a6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51384/0 (socket says 172.21.15.29:51384) 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.125+0000 7fd7c0b50700 1 -- 172.21.15.29:0/3014746845 learned_addr learned my addr 172.21.15.29:0/3014746845 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.125+0000 7fd7bb7fe700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7bc0c1f80 0x7fd7bc0a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7bbfff700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7bc062ca0 0x7fd7bc0a85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7c0b50700 1 -- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7bc0c1f80 msgr2=0x7fd7bc0a8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7c0b50700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7bc0c1f80 0x7fd7bc0a8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7c0b50700 1 -- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7bc062ca0 msgr2=0x7fd7bc0a85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7c0b50700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7bc062ca0 0x7fd7bc0a85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7c0b50700 1 -- 172.21.15.29:0/3014746845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b4009ae0 con 0x7fd7bc14e840 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7bb7fe700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7bc0c1f80 0x7fd7bc0a8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7c0b50700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7bc14e840 0x7fd7bc0a6e70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fd7bc064360 tx=0x7fd7b4009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7a3fff700 1 -- 172.21.15.29:0/3014746845 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b400be00 con 0x7fd7bc14e840 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7a3fff700 1 -- 172.21.15.29:0/3014746845 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b401d460 con 0x7fd7bc14e840 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7c25b3700 1 -- 172.21.15.29:0/3014746845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc068b40 con 0x7fd7bc14e840 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7a3fff700 1 -- 172.21.15.29:0/3014746845 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b401d820 con 0x7fd7bc14e840 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.126+0000 7fd7c25b3700 1 -- 172.21.15.29:0/3014746845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc137170 con 0x7fd7bc14e840 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.127+0000 7fd7c25b3700 1 -- 172.21.15.29:0/3014746845 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7bc0689a0 con 0x7fd7bc14e840 2024-01-24T13:58:32.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.128+0000 7fd7a3fff700 1 -- 172.21.15.29:0/3014746845 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd7b4003040 con 0x7fd7bc14e840 2024-01-24T13:58:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.128+0000 7fd7a3fff700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd7a805f150 0x7fd7a8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.128+0000 7fd7a3fff700 1 -- 172.21.15.29:0/3014746845 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fd7b407b8a0 con 0x7fd7bc14e840 2024-01-24T13:58:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.128+0000 7fd7bbfff700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd7a805f150 0x7fd7a8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.129+0000 7fd7bbfff700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd7a805f150 0x7fd7a8061610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd7ac0011e0 tx=0x7fd7ac007180).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:32.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.133+0000 7fd7a3fff700 1 -- 172.21.15.29:0/3014746845 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd7bc0689a0 con 0x7fd7bc14e840 2024-01-24T13:58:32.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.310+0000 7fd7c25b3700 1 -- 172.21.15.29:0/3014746845 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7bc0689a0 con 0x7fd7a805f150 2024-01-24T13:58:32.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.323+0000 7fd7a3fff700 1 -- 172.21.15.29:0/3014746845 <== mgr.4103 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+410603 (secure 0 0 0) 0x7fd7bc0689a0 con 0x7fd7a805f150 2024-01-24T13:58:32.329 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.327+0000 7fd7c25b3700 1 -- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd7a805f150 msgr2=0x7fd7a8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.327+0000 7fd7c25b3700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd7a805f150 0x7fd7a8061610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd7ac0011e0 tx=0x7fd7ac007180).stop 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.327+0000 7fd7c25b3700 1 -- 172.21.15.29:0/3014746845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7bc14e840 msgr2=0x7fd7bc0a6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.327+0000 7fd7c25b3700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7bc14e840 0x7fd7bc0a6e70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fd7bc064360 tx=0x7fd7b4009840).stop 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.327+0000 7fd7c25b3700 1 -- 172.21.15.29:0/3014746845 shutdown_connections 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.327+0000 7fd7c25b3700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd7a805f150 0x7fd7a8061610 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.327+0000 7fd7c25b3700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7bc14e840 0x7fd7bc0a6e70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.327+0000 7fd7c25b3700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7bc062ca0 0x7fd7bc0a85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.327+0000 7fd7c25b3700 1 --2- 172.21.15.29:0/3014746845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7bc0c1f80 0x7fd7bc0a8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.327+0000 7fd7c25b3700 1 -- 172.21.15.29:0/3014746845 >> 172.21.15.29:0/3014746845 conn(0x7fd7bc064aa0 msgr2=0x7fd7bc157da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.327+0000 7fd7c25b3700 1 -- 172.21.15.29:0/3014746845 shutdown_connections 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.327+0000 7fd7c25b3700 1 -- 172.21.15.29:0/3014746845 wait complete. 2024-01-24T13:58:32.334 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T13:58:32.341 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-01-24T13:58:31.726139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":402,"acting":402,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":402,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379004,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752595716,"statfs":{"total":773094113280,"available":770658013184,"internally_reserved":0,"allocated":2289664,"data_stored":941041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2179,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.519916"},"pg_stats":[{"pgid":"3.3f","version":"21'3","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.218432+0000","last_change":"2024-01-24T13:58:18.804112+0000","last_active":"2024-01-24T13:58:23.218432+0000","last_peered":"2024-01-24T13:58:23.218432+0000","last_clean":"2024-01-24T13:58:23.218432+0000","last_became_active":"2024-01-24T13:58:18.803739+0000","last_became_peered":"2024-01-24T13:58:18.803739+0000","last_unstale":"2024-01-24T13:58:23.218432+0000","last_undegraded":"2024-01-24T13:58:23.218432+0000","last_fullsized":"2024-01-24T13:58:23.218432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224809+0000","last_change":"2024-01-24T13:58:19.214526+0000","last_active":"2024-01-24T13:58:23.224809+0000","last_peered":"2024-01-24T13:58:23.224809+0000","last_clean":"2024-01-24T13:58:23.224809+0000","last_became_active":"2024-01-24T13:58:19.214261+0000","last_became_peered":"2024-01-24T13:58:19.214261+0000","last_unstale":"2024-01-24T13:58:23.224809+0000","last_undegraded":"2024-01-24T13:58:23.224809+0000","last_fullsized":"2024-01-24T13:58:23.224809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211257+0000","last_change":"2024-01-24T13:58:18.813731+0000","last_active":"2024-01-24T13:58:23.211257+0000","last_peered":"2024-01-24T13:58:23.211257+0000","last_clean":"2024-01-24T13:58:23.211257+0000","last_became_active":"2024-01-24T13:58:18.813399+0000","last_became_peered":"2024-01-24T13:58:18.813399+0000","last_unstale":"2024-01-24T13:58:23.211257+0000","last_undegraded":"2024-01-24T13:58:23.211257+0000","last_fullsized":"2024-01-24T13:58:23.211257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216628+0000","last_change":"2024-01-24T13:58:19.214934+0000","last_active":"2024-01-24T13:58:23.216628+0000","last_peered":"2024-01-24T13:58:23.216628+0000","last_clean":"2024-01-24T13:58:23.216628+0000","last_became_active":"2024-01-24T13:58:19.214473+0000","last_became_peered":"2024-01-24T13:58:19.214473+0000","last_unstale":"2024-01-24T13:58:23.216628+0000","last_undegraded":"2024-01-24T13:58:23.216628+0000","last_fullsized":"2024-01-24T13:58:23.216628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235905+0000","last_change":"2024-01-24T13:58:18.810864+0000","last_active":"2024-01-24T13:58:23.235905+0000","last_peered":"2024-01-24T13:58:23.235905+0000","last_clean":"2024-01-24T13:58:23.235905+0000","last_became_active":"2024-01-24T13:58:18.810538+0000","last_became_peered":"2024-01-24T13:58:18.810538+0000","last_unstale":"2024-01-24T13:58:23.235905+0000","last_undegraded":"2024-01-24T13:58:23.235905+0000","last_fullsized":"2024-01-24T13:58:23.235905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209261+0000","last_change":"2024-01-24T13:58:19.215436+0000","last_active":"2024-01-24T13:58:23.209261+0000","last_peered":"2024-01-24T13:58:23.209261+0000","last_clean":"2024-01-24T13:58:23.209261+0000","last_became_active":"2024-01-24T13:58:19.215145+0000","last_became_peered":"2024-01-24T13:58:19.215145+0000","last_unstale":"2024-01-24T13:58:23.209261+0000","last_undegraded":"2024-01-24T13:58:23.209261+0000","last_fullsized":"2024-01-24T13:58:23.209261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227753+0000","last_change":"2024-01-24T13:58:18.803776+0000","last_active":"2024-01-24T13:58:23.227753+0000","last_peered":"2024-01-24T13:58:23.227753+0000","last_clean":"2024-01-24T13:58:23.227753+0000","last_became_active":"2024-01-24T13:58:18.803485+0000","last_became_peered":"2024-01-24T13:58:18.803485+0000","last_unstale":"2024-01-24T13:58:23.227753+0000","last_undegraded":"2024-01-24T13:58:23.227753+0000","last_fullsized":"2024-01-24T13:58:23.227753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.234859+0000","last_change":"2024-01-24T13:58:19.224975+0000","last_active":"2024-01-24T13:58:23.234859+0000","last_peered":"2024-01-24T13:58:23.234859+0000","last_clean":"2024-01-24T13:58:23.234859+0000","last_became_active":"2024-01-24T13:58:19.224568+0000","last_became_peered":"2024-01-24T13:58:19.224568+0000","last_unstale":"2024-01-24T13:58:23.234859+0000","last_undegraded":"2024-01-24T13:58:23.234859+0000","last_fullsized":"2024-01-24T13:58:23.234859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217357+0000","last_change":"2024-01-24T13:58:18.797963+0000","last_active":"2024-01-24T13:58:23.217357+0000","last_peered":"2024-01-24T13:58:23.217357+0000","last_clean":"2024-01-24T13:58:23.217357+0000","last_became_active":"2024-01-24T13:58:18.797320+0000","last_became_peered":"2024-01-24T13:58:18.797320+0000","last_unstale":"2024-01-24T13:58:23.217357+0000","last_undegraded":"2024-01-24T13:58:23.217357+0000","last_fullsized":"2024-01-24T13:58:23.217357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213193+0000","last_change":"2024-01-24T13:58:19.213147+0000","last_active":"2024-01-24T13:58:23.213193+0000","last_peered":"2024-01-24T13:58:23.213193+0000","last_clean":"2024-01-24T13:58:23.213193+0000","last_became_active":"2024-01-24T13:58:19.212854+0000","last_became_peered":"2024-01-24T13:58:19.212854+0000","last_unstale":"2024-01-24T13:58:23.213193+0000","last_undegraded":"2024-01-24T13:58:23.213193+0000","last_fullsized":"2024-01-24T13:58:23.213193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.228193+0000","last_change":"2024-01-24T13:58:18.812721+0000","last_active":"2024-01-24T13:58:23.228193+0000","last_peered":"2024-01-24T13:58:23.228193+0000","last_clean":"2024-01-24T13:58:23.228193+0000","last_became_active":"2024-01-24T13:58:18.812439+0000","last_became_peered":"2024-01-24T13:58:18.812439+0000","last_unstale":"2024-01-24T13:58:23.228193+0000","last_undegraded":"2024-01-24T13:58:23.228193+0000","last_fullsized":"2024-01-24T13:58:23.228193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209659+0000","last_change":"2024-01-24T13:58:19.213672+0000","last_active":"2024-01-24T13:58:23.209659+0000","last_peered":"2024-01-24T13:58:23.209659+0000","last_clean":"2024-01-24T13:58:23.209659+0000","last_became_active":"2024-01-24T13:58:19.213378+0000","last_became_peered":"2024-01-24T13:58:19.213378+0000","last_unstale":"2024-01-24T13:58:23.209659+0000","last_undegraded":"2024-01-24T13:58:23.209659+0000","last_fullsized":"2024-01-24T13:58:23.209659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216204+0000","last_change":"2024-01-24T13:58:18.806273+0000","last_active":"2024-01-24T13:58:23.216204+0000","last_peered":"2024-01-24T13:58:23.216204+0000","last_clean":"2024-01-24T13:58:23.216204+0000","last_became_active":"2024-01-24T13:58:18.805970+0000","last_became_peered":"2024-01-24T13:58:18.805970+0000","last_unstale":"2024-01-24T13:58:23.216204+0000","last_undegraded":"2024-01-24T13:58:23.216204+0000","last_fullsized":"2024-01-24T13:58:23.216204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227834+0000","last_change":"2024-01-24T13:58:19.208876+0000","last_active":"2024-01-24T13:58:23.227834+0000","last_peered":"2024-01-24T13:58:23.227834+0000","last_clean":"2024-01-24T13:58:23.227834+0000","last_became_active":"2024-01-24T13:58:19.208537+0000","last_became_peered":"2024-01-24T13:58:19.208537+0000","last_unstale":"2024-01-24T13:58:23.227834+0000","last_undegraded":"2024-01-24T13:58:23.227834+0000","last_fullsized":"2024-01-24T13:58:23.227834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208114+0000","last_change":"2024-01-24T13:58:18.801957+0000","last_active":"2024-01-24T13:58:23.208114+0000","last_peered":"2024-01-24T13:58:23.208114+0000","last_clean":"2024-01-24T13:58:23.208114+0000","last_became_active":"2024-01-24T13:58:18.801641+0000","last_became_peered":"2024-01-24T13:58:18.801641+0000","last_unstale":"2024-01-24T13:58:23.208114+0000","last_undegraded":"2024-01-24T13:58:23.208114+0000","last_fullsized":"2024-01-24T13:58:23.208114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.222487+0000","last_change":"2024-01-24T13:58:19.217744+0000","last_active":"2024-01-24T13:58:23.222487+0000","last_peered":"2024-01-24T13:58:23.222487+0000","last_clean":"2024-01-24T13:58:23.222487+0000","last_became_active":"2024-01-24T13:58:19.217212+0000","last_became_peered":"2024-01-24T13:58:19.217212+0000","last_unstale":"2024-01-24T13:58:23.222487+0000","last_undegraded":"2024-01-24T13:58:23.222487+0000","last_fullsized":"2024-01-24T13:58:23.222487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.218321+0000","last_change":"2024-01-24T13:58:18.797573+0000","last_active":"2024-01-24T13:58:23.218321+0000","last_peered":"2024-01-24T13:58:23.218321+0000","last_clean":"2024-01-24T13:58:23.218321+0000","last_became_active":"2024-01-24T13:58:18.797100+0000","last_became_peered":"2024-01-24T13:58:18.797100+0000","last_unstale":"2024-01-24T13:58:23.218321+0000","last_undegraded":"2024-01-24T13:58:23.218321+0000","last_fullsized":"2024-01-24T13:58:23.218321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209573+0000","last_change":"2024-01-24T13:58:19.211019+0000","last_active":"2024-01-24T13:58:23.209573+0000","last_peered":"2024-01-24T13:58:23.209573+0000","last_clean":"2024-01-24T13:58:23.209573+0000","last_became_active":"2024-01-24T13:58:19.210424+0000","last_became_peered":"2024-01-24T13:58:19.210424+0000","last_unstale":"2024-01-24T13:58:23.209573+0000","last_undegraded":"2024-01-24T13:58:23.209573+0000","last_fullsized":"2024-01-24T13:58:23.209573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213015+0000","last_change":"2024-01-24T13:58:18.800935+0000","last_active":"2024-01-24T13:58:23.213015+0000","last_peered":"2024-01-24T13:58:23.213015+0000","last_clean":"2024-01-24T13:58:23.213015+0000","last_became_active":"2024-01-24T13:58:18.800629+0000","last_became_peered":"2024-01-24T13:58:18.800629+0000","last_unstale":"2024-01-24T13:58:23.213015+0000","last_undegraded":"2024-01-24T13:58:23.213015+0000","last_fullsized":"2024-01-24T13:58:23.213015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216389+0000","last_change":"2024-01-24T13:58:19.214756+0000","last_active":"2024-01-24T13:58:23.216389+0000","last_peered":"2024-01-24T13:58:23.216389+0000","last_clean":"2024-01-24T13:58:23.216389+0000","last_became_active":"2024-01-24T13:58:19.214127+0000","last_became_peered":"2024-01-24T13:58:19.214127+0000","last_unstale":"2024-01-24T13:58:23.216389+0000","last_undegraded":"2024-01-24T13:58:23.216389+0000","last_fullsized":"2024-01-24T13:58:23.216389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213067+0000","last_change":"2024-01-24T13:58:18.804647+0000","last_active":"2024-01-24T13:58:23.213067+0000","last_peered":"2024-01-24T13:58:23.213067+0000","last_clean":"2024-01-24T13:58:23.213067+0000","last_became_active":"2024-01-24T13:58:18.803996+0000","last_became_peered":"2024-01-24T13:58:18.803996+0000","last_unstale":"2024-01-24T13:58:23.213067+0000","last_undegraded":"2024-01-24T13:58:23.213067+0000","last_fullsized":"2024-01-24T13:58:23.213067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226664+0000","last_change":"2024-01-24T13:58:19.210603+0000","last_active":"2024-01-24T13:58:23.226664+0000","last_peered":"2024-01-24T13:58:23.226664+0000","last_clean":"2024-01-24T13:58:23.226664+0000","last_became_active":"2024-01-24T13:58:19.210245+0000","last_became_peered":"2024-01-24T13:58:19.210245+0000","last_unstale":"2024-01-24T13:58:23.226664+0000","last_undegraded":"2024-01-24T13:58:23.226664+0000","last_fullsized":"2024-01-24T13:58:23.226664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.34","version":"26'7","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-24T13:58:28.635902+0000","last_change":"2024-01-24T13:58:18.809410+0000","last_active":"2024-01-24T13:58:28.635902+0000","last_peered":"2024-01-24T13:58:28.635902+0000","last_clean":"2024-01-24T13:58:28.635902+0000","last_became_active":"2024-01-24T13:58:18.809115+0000","last_became_peered":"2024-01-24T13:58:18.809115+0000","last_unstale":"2024-01-24T13:58:28.635902+0000","last_undegraded":"2024-01-24T13:58:28.635902+0000","last_fullsized":"2024-01-24T13:58:28.635902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3071,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220937+0000","last_change":"2024-01-24T13:58:19.217910+0000","last_active":"2024-01-24T13:58:23.220937+0000","last_peered":"2024-01-24T13:58:23.220937+0000","last_clean":"2024-01-24T13:58:23.220937+0000","last_became_active":"2024-01-24T13:58:19.216908+0000","last_became_peered":"2024-01-24T13:58:19.216908+0000","last_unstale":"2024-01-24T13:58:23.220937+0000","last_undegraded":"2024-01-24T13:58:23.220937+0000","last_fullsized":"2024-01-24T13:58:23.220937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208119+0000","last_change":"2024-01-24T13:58:18.807547+0000","last_active":"2024-01-24T13:58:23.208119+0000","last_peered":"2024-01-24T13:58:23.208119+0000","last_clean":"2024-01-24T13:58:23.208119+0000","last_became_active":"2024-01-24T13:58:18.806845+0000","last_became_peered":"2024-01-24T13:58:18.806845+0000","last_unstale":"2024-01-24T13:58:23.208119+0000","last_undegraded":"2024-01-24T13:58:23.208119+0000","last_fullsized":"2024-01-24T13:58:23.208119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224004+0000","last_change":"2024-01-24T13:58:19.219483+0000","last_active":"2024-01-24T13:58:23.224004+0000","last_peered":"2024-01-24T13:58:23.224004+0000","last_clean":"2024-01-24T13:58:23.224004+0000","last_became_active":"2024-01-24T13:58:19.219182+0000","last_became_peered":"2024-01-24T13:58:19.219182+0000","last_unstale":"2024-01-24T13:58:23.224004+0000","last_undegraded":"2024-01-24T13:58:23.224004+0000","last_fullsized":"2024-01-24T13:58:23.224004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232615+0000","last_change":"2024-01-24T13:58:18.804144+0000","last_active":"2024-01-24T13:58:23.232615+0000","last_peered":"2024-01-24T13:58:23.232615+0000","last_clean":"2024-01-24T13:58:23.232615+0000","last_became_active":"2024-01-24T13:58:18.803873+0000","last_became_peered":"2024-01-24T13:58:18.803873+0000","last_unstale":"2024-01-24T13:58:23.232615+0000","last_undegraded":"2024-01-24T13:58:23.232615+0000","last_fullsized":"2024-01-24T13:58:23.232615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209346+0000","last_change":"2024-01-24T13:58:19.221162+0000","last_active":"2024-01-24T13:58:23.209346+0000","last_peered":"2024-01-24T13:58:23.209346+0000","last_clean":"2024-01-24T13:58:23.209346+0000","last_became_active":"2024-01-24T13:58:19.220665+0000","last_became_peered":"2024-01-24T13:58:19.220665+0000","last_unstale":"2024-01-24T13:58:23.209346+0000","last_undegraded":"2024-01-24T13:58:23.209346+0000","last_fullsized":"2024-01-24T13:58:23.209346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.210124+0000","last_change":"2024-01-24T13:58:18.799003+0000","last_active":"2024-01-24T13:58:23.210124+0000","last_peered":"2024-01-24T13:58:23.210124+0000","last_clean":"2024-01-24T13:58:23.210124+0000","last_became_active":"2024-01-24T13:58:18.796638+0000","last_became_peered":"2024-01-24T13:58:18.796638+0000","last_unstale":"2024-01-24T13:58:23.210124+0000","last_undegraded":"2024-01-24T13:58:23.210124+0000","last_fullsized":"2024-01-24T13:58:23.210124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227756+0000","last_change":"2024-01-24T13:58:19.213039+0000","last_active":"2024-01-24T13:58:23.227756+0000","last_peered":"2024-01-24T13:58:23.227756+0000","last_clean":"2024-01-24T13:58:23.227756+0000","last_became_active":"2024-01-24T13:58:19.212755+0000","last_became_peered":"2024-01-24T13:58:19.212755+0000","last_unstale":"2024-01-24T13:58:23.227756+0000","last_undegraded":"2024-01-24T13:58:23.227756+0000","last_fullsized":"2024-01-24T13:58:23.227756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"21'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.225127+0000","last_change":"2024-01-24T13:58:18.805645+0000","last_active":"2024-01-24T13:58:23.225127+0000","last_peered":"2024-01-24T13:58:23.225127+0000","last_clean":"2024-01-24T13:58:23.225127+0000","last_became_active":"2024-01-24T13:58:18.805375+0000","last_became_peered":"2024-01-24T13:58:18.805375+0000","last_unstale":"2024-01-24T13:58:23.225127+0000","last_undegraded":"2024-01-24T13:58:23.225127+0000","last_fullsized":"2024-01-24T13:58:23.225127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237611+0000","last_change":"2024-01-24T13:58:19.225874+0000","last_active":"2024-01-24T13:58:23.237611+0000","last_peered":"2024-01-24T13:58:23.237611+0000","last_clean":"2024-01-24T13:58:23.237611+0000","last_became_active":"2024-01-24T13:58:19.225220+0000","last_became_peered":"2024-01-24T13:58:19.225220+0000","last_unstale":"2024-01-24T13:58:23.237611+0000","last_undegraded":"2024-01-24T13:58:23.237611+0000","last_fullsized":"2024-01-24T13:58:23.237611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.233153+0000","last_change":"2024-01-24T13:58:18.797127+0000","last_active":"2024-01-24T13:58:23.233153+0000","last_peered":"2024-01-24T13:58:23.233153+0000","last_clean":"2024-01-24T13:58:23.233153+0000","last_became_active":"2024-01-24T13:58:18.796816+0000","last_became_peered":"2024-01-24T13:58:18.796816+0000","last_unstale":"2024-01-24T13:58:23.233153+0000","last_undegraded":"2024-01-24T13:58:23.233153+0000","last_fullsized":"2024-01-24T13:58:23.233153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216434+0000","last_change":"2024-01-24T13:58:19.219593+0000","last_active":"2024-01-24T13:58:23.216434+0000","last_peered":"2024-01-24T13:58:23.216434+0000","last_clean":"2024-01-24T13:58:23.216434+0000","last_became_active":"2024-01-24T13:58:19.219267+0000","last_became_peered":"2024-01-24T13:58:19.219267+0000","last_unstale":"2024-01-24T13:58:23.216434+0000","last_undegraded":"2024-01-24T13:58:23.216434+0000","last_fullsized":"2024-01-24T13:58:23.216434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220392+0000","last_change":"2024-01-24T13:58:18.805694+0000","last_active":"2024-01-24T13:58:23.220392+0000","last_peered":"2024-01-24T13:58:23.220392+0000","last_clean":"2024-01-24T13:58:23.220392+0000","last_became_active":"2024-01-24T13:58:18.805335+0000","last_became_peered":"2024-01-24T13:58:18.805335+0000","last_unstale":"2024-01-24T13:58:23.220392+0000","last_undegraded":"2024-01-24T13:58:23.220392+0000","last_fullsized":"2024-01-24T13:58:23.220392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207983+0000","last_change":"2024-01-24T13:58:19.213662+0000","last_active":"2024-01-24T13:58:23.207983+0000","last_peered":"2024-01-24T13:58:23.207983+0000","last_clean":"2024-01-24T13:58:23.207983+0000","last_became_active":"2024-01-24T13:58:19.213340+0000","last_became_peered":"2024-01-24T13:58:19.213340+0000","last_unstale":"2024-01-24T13:58:23.207983+0000","last_undegraded":"2024-01-24T13:58:23.207983+0000","last_fullsized":"2024-01-24T13:58:23.207983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2d","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217312+0000","last_change":"2024-01-24T13:58:18.798262+0000","last_active":"2024-01-24T13:58:23.217312+0000","last_peered":"2024-01-24T13:58:23.217312+0000","last_clean":"2024-01-24T13:58:23.217312+0000","last_became_active":"2024-01-24T13:58:18.797856+0000","last_became_peered":"2024-01-24T13:58:18.797856+0000","last_unstale":"2024-01-24T13:58:23.217312+0000","last_undegraded":"2024-01-24T13:58:23.217312+0000","last_fullsized":"2024-01-24T13:58:23.217312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232445+0000","last_change":"2024-01-24T13:58:19.223736+0000","last_active":"2024-01-24T13:58:23.232445+0000","last_peered":"2024-01-24T13:58:23.232445+0000","last_clean":"2024-01-24T13:58:23.232445+0000","last_became_active":"2024-01-24T13:58:19.223500+0000","last_became_peered":"2024-01-24T13:58:19.223500+0000","last_unstale":"2024-01-24T13:58:23.232445+0000","last_undegraded":"2024-01-24T13:58:23.232445+0000","last_fullsized":"2024-01-24T13:58:23.232445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213312+0000","last_change":"2024-01-24T13:58:18.802360+0000","last_active":"2024-01-24T13:58:23.213312+0000","last_peered":"2024-01-24T13:58:23.213312+0000","last_clean":"2024-01-24T13:58:23.213312+0000","last_became_active":"2024-01-24T13:58:18.802051+0000","last_became_peered":"2024-01-24T13:58:18.802051+0000","last_unstale":"2024-01-24T13:58:23.213312+0000","last_undegraded":"2024-01-24T13:58:23.213312+0000","last_fullsized":"2024-01-24T13:58:23.213312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217156+0000","last_change":"2024-01-24T13:58:19.218929+0000","last_active":"2024-01-24T13:58:23.217156+0000","last_peered":"2024-01-24T13:58:23.217156+0000","last_clean":"2024-01-24T13:58:23.217156+0000","last_became_active":"2024-01-24T13:58:19.218350+0000","last_became_peered":"2024-01-24T13:58:19.218350+0000","last_unstale":"2024-01-24T13:58:23.217156+0000","last_undegraded":"2024-01-24T13:58:23.217156+0000","last_fullsized":"2024-01-24T13:58:23.217156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2b","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235138+0000","last_change":"2024-01-24T13:58:18.810840+0000","last_active":"2024-01-24T13:58:23.235138+0000","last_peered":"2024-01-24T13:58:23.235138+0000","last_clean":"2024-01-24T13:58:23.235138+0000","last_became_active":"2024-01-24T13:58:18.809023+0000","last_became_peered":"2024-01-24T13:58:18.809023+0000","last_unstale":"2024-01-24T13:58:23.235138+0000","last_undegraded":"2024-01-24T13:58:23.235138+0000","last_fullsized":"2024-01-24T13:58:23.235138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209489+0000","last_change":"2024-01-24T13:58:19.217855+0000","last_active":"2024-01-24T13:58:23.209489+0000","last_peered":"2024-01-24T13:58:23.209489+0000","last_clean":"2024-01-24T13:58:23.209489+0000","last_became_active":"2024-01-24T13:58:19.217559+0000","last_became_peered":"2024-01-24T13:58:19.217559+0000","last_unstale":"2024-01-24T13:58:23.209489+0000","last_undegraded":"2024-01-24T13:58:23.209489+0000","last_fullsized":"2024-01-24T13:58:23.209489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226871+0000","last_change":"2024-01-24T13:58:18.798611+0000","last_active":"2024-01-24T13:58:23.226871+0000","last_peered":"2024-01-24T13:58:23.226871+0000","last_clean":"2024-01-24T13:58:23.226871+0000","last_became_active":"2024-01-24T13:58:18.798325+0000","last_became_peered":"2024-01-24T13:58:18.798325+0000","last_unstale":"2024-01-24T13:58:23.226871+0000","last_undegraded":"2024-01-24T13:58:23.226871+0000","last_fullsized":"2024-01-24T13:58:23.226871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.222261+0000","last_change":"2024-01-24T13:58:19.206846+0000","last_active":"2024-01-24T13:58:23.222261+0000","last_peered":"2024-01-24T13:58:23.222261+0000","last_clean":"2024-01-24T13:58:23.222261+0000","last_became_active":"2024-01-24T13:58:19.206552+0000","last_became_peered":"2024-01-24T13:58:19.206552+0000","last_unstale":"2024-01-24T13:58:23.222261+0000","last_undegraded":"2024-01-24T13:58:23.222261+0000","last_fullsized":"2024-01-24T13:58:23.222261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207449+0000","last_change":"2024-01-24T13:58:18.801628+0000","last_active":"2024-01-24T13:58:23.207449+0000","last_peered":"2024-01-24T13:58:23.207449+0000","last_clean":"2024-01-24T13:58:23.207449+0000","last_became_active":"2024-01-24T13:58:18.801291+0000","last_became_peered":"2024-01-24T13:58:18.801291+0000","last_unstale":"2024-01-24T13:58:23.207449+0000","last_undegraded":"2024-01-24T13:58:23.207449+0000","last_fullsized":"2024-01-24T13:58:23.207449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209637+0000","last_change":"2024-01-24T13:58:19.211347+0000","last_active":"2024-01-24T13:58:23.209637+0000","last_peered":"2024-01-24T13:58:23.209637+0000","last_clean":"2024-01-24T13:58:23.209637+0000","last_became_active":"2024-01-24T13:58:19.210298+0000","last_became_peered":"2024-01-24T13:58:19.210298+0000","last_unstale":"2024-01-24T13:58:23.209637+0000","last_undegraded":"2024-01-24T13:58:23.209637+0000","last_fullsized":"2024-01-24T13:58:23.209637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224993+0000","last_change":"2024-01-24T13:58:18.803399+0000","last_active":"2024-01-24T13:58:23.224993+0000","last_peered":"2024-01-24T13:58:23.224993+0000","last_clean":"2024-01-24T13:58:23.224993+0000","last_became_active":"2024-01-24T13:58:18.803109+0000","last_became_peered":"2024-01-24T13:58:18.803109+0000","last_unstale":"2024-01-24T13:58:23.224993+0000","last_undegraded":"2024-01-24T13:58:23.224993+0000","last_fullsized":"2024-01-24T13:58:23.224993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238205+0000","last_change":"2024-01-24T13:58:19.225710+0000","last_active":"2024-01-24T13:58:23.238205+0000","last_peered":"2024-01-24T13:58:23.238205+0000","last_clean":"2024-01-24T13:58:23.238205+0000","last_became_active":"2024-01-24T13:58:19.224844+0000","last_became_peered":"2024-01-24T13:58:19.224844+0000","last_unstale":"2024-01-24T13:58:23.238205+0000","last_undegraded":"2024-01-24T13:58:23.238205+0000","last_fullsized":"2024-01-24T13:58:23.238205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208117+0000","last_change":"2024-01-24T13:58:18.801623+0000","last_active":"2024-01-24T13:58:23.208117+0000","last_peered":"2024-01-24T13:58:23.208117+0000","last_clean":"2024-01-24T13:58:23.208117+0000","last_became_active":"2024-01-24T13:58:18.801272+0000","last_became_peered":"2024-01-24T13:58:18.801272+0000","last_unstale":"2024-01-24T13:58:23.208117+0000","last_undegraded":"2024-01-24T13:58:23.208117+0000","last_fullsized":"2024-01-24T13:58:23.208117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212308+0000","last_change":"2024-01-24T13:58:19.207457+0000","last_active":"2024-01-24T13:58:23.212308+0000","last_peered":"2024-01-24T13:58:23.212308+0000","last_clean":"2024-01-24T13:58:23.212308+0000","last_became_active":"2024-01-24T13:58:19.207133+0000","last_became_peered":"2024-01-24T13:58:19.207133+0000","last_unstale":"2024-01-24T13:58:23.212308+0000","last_undegraded":"2024-01-24T13:58:23.212308+0000","last_fullsized":"2024-01-24T13:58:23.212308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"26'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.880144+0000","last_change":"2024-01-24T13:58:18.800410+0000","last_active":"2024-01-24T13:58:23.880144+0000","last_peered":"2024-01-24T13:58:23.880144+0000","last_clean":"2024-01-24T13:58:23.880144+0000","last_became_active":"2024-01-24T13:58:18.800073+0000","last_became_peered":"2024-01-24T13:58:18.800073+0000","last_unstale":"2024-01-24T13:58:23.880144+0000","last_undegraded":"2024-01-24T13:58:23.880144+0000","last_fullsized":"2024-01-24T13:58:23.880144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207934+0000","last_change":"2024-01-24T13:58:19.221135+0000","last_active":"2024-01-24T13:58:23.207934+0000","last_peered":"2024-01-24T13:58:23.207934+0000","last_clean":"2024-01-24T13:58:23.207934+0000","last_became_active":"2024-01-24T13:58:19.220863+0000","last_became_peered":"2024-01-24T13:58:19.220863+0000","last_unstale":"2024-01-24T13:58:23.207934+0000","last_undegraded":"2024-01-24T13:58:23.207934+0000","last_fullsized":"2024-01-24T13:58:23.207934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211332+0000","last_change":"2024-01-24T13:58:18.800232+0000","last_active":"2024-01-24T13:58:23.211332+0000","last_peered":"2024-01-24T13:58:23.211332+0000","last_clean":"2024-01-24T13:58:23.211332+0000","last_became_active":"2024-01-24T13:58:18.799722+0000","last_became_peered":"2024-01-24T13:58:18.799722+0000","last_unstale":"2024-01-24T13:58:23.211332+0000","last_undegraded":"2024-01-24T13:58:23.211332+0000","last_fullsized":"2024-01-24T13:58:23.211332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209141+0000","last_change":"2024-01-24T13:58:19.215326+0000","last_active":"2024-01-24T13:58:23.209141+0000","last_peered":"2024-01-24T13:58:23.209141+0000","last_clean":"2024-01-24T13:58:23.209141+0000","last_became_active":"2024-01-24T13:58:19.214852+0000","last_became_peered":"2024-01-24T13:58:19.214852+0000","last_unstale":"2024-01-24T13:58:23.209141+0000","last_undegraded":"2024-01-24T13:58:23.209141+0000","last_fullsized":"2024-01-24T13:58:23.209141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207199+0000","last_change":"2024-01-24T13:58:18.804258+0000","last_active":"2024-01-24T13:58:23.207199+0000","last_peered":"2024-01-24T13:58:23.207199+0000","last_clean":"2024-01-24T13:58:23.207199+0000","last_became_active":"2024-01-24T13:58:18.802440+0000","last_became_peered":"2024-01-24T13:58:18.802440+0000","last_unstale":"2024-01-24T13:58:23.207199+0000","last_undegraded":"2024-01-24T13:58:23.207199+0000","last_fullsized":"2024-01-24T13:58:23.207199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208000+0000","last_change":"2024-01-24T13:58:19.214888+0000","last_active":"2024-01-24T13:58:23.208000+0000","last_peered":"2024-01-24T13:58:23.208000+0000","last_clean":"2024-01-24T13:58:23.208000+0000","last_became_active":"2024-01-24T13:58:19.214607+0000","last_became_peered":"2024-01-24T13:58:19.214607+0000","last_unstale":"2024-01-24T13:58:23.208000+0000","last_undegraded":"2024-01-24T13:58:23.208000+0000","last_fullsized":"2024-01-24T13:58:23.208000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235037+0000","last_change":"2024-01-24T13:58:18.813467+0000","last_active":"2024-01-24T13:58:23.235037+0000","last_peered":"2024-01-24T13:58:23.235037+0000","last_clean":"2024-01-24T13:58:23.235037+0000","last_became_active":"2024-01-24T13:58:18.813164+0000","last_became_peered":"2024-01-24T13:58:18.813164+0000","last_unstale":"2024-01-24T13:58:23.235037+0000","last_undegraded":"2024-01-24T13:58:23.235037+0000","last_fullsized":"2024-01-24T13:58:23.235037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211738+0000","last_change":"2024-01-24T13:58:19.212196+0000","last_active":"2024-01-24T13:58:23.211738+0000","last_peered":"2024-01-24T13:58:23.211738+0000","last_clean":"2024-01-24T13:58:23.211738+0000","last_became_active":"2024-01-24T13:58:19.211820+0000","last_became_peered":"2024-01-24T13:58:19.211820+0000","last_unstale":"2024-01-24T13:58:23.211738+0000","last_undegraded":"2024-01-24T13:58:23.211738+0000","last_fullsized":"2024-01-24T13:58:23.211738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226611+0000","last_change":"2024-01-24T13:58:18.802179+0000","last_active":"2024-01-24T13:58:23.226611+0000","last_peered":"2024-01-24T13:58:23.226611+0000","last_clean":"2024-01-24T13:58:23.226611+0000","last_became_active":"2024-01-24T13:58:18.801891+0000","last_became_peered":"2024-01-24T13:58:18.801891+0000","last_unstale":"2024-01-24T13:58:23.226611+0000","last_undegraded":"2024-01-24T13:58:23.226611+0000","last_fullsized":"2024-01-24T13:58:23.226611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227791+0000","last_change":"2024-01-24T13:58:19.217207+0000","last_active":"2024-01-24T13:58:23.227791+0000","last_peered":"2024-01-24T13:58:23.227791+0000","last_clean":"2024-01-24T13:58:23.227791+0000","last_became_active":"2024-01-24T13:58:19.216920+0000","last_became_peered":"2024-01-24T13:58:19.216920+0000","last_unstale":"2024-01-24T13:58:23.227791+0000","last_undegraded":"2024-01-24T13:58:23.227791+0000","last_fullsized":"2024-01-24T13:58:23.227791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220454+0000","last_change":"2024-01-24T13:58:18.793887+0000","last_active":"2024-01-24T13:58:23.220454+0000","last_peered":"2024-01-24T13:58:23.220454+0000","last_clean":"2024-01-24T13:58:23.220454+0000","last_became_active":"2024-01-24T13:58:18.793542+0000","last_became_peered":"2024-01-24T13:58:18.793542+0000","last_unstale":"2024-01-24T13:58:23.220454+0000","last_undegraded":"2024-01-24T13:58:23.220454+0000","last_fullsized":"2024-01-24T13:58:23.220454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211617+0000","last_change":"2024-01-24T13:58:19.216131+0000","last_active":"2024-01-24T13:58:23.211617+0000","last_peered":"2024-01-24T13:58:23.211617+0000","last_clean":"2024-01-24T13:58:23.211617+0000","last_became_active":"2024-01-24T13:58:19.215566+0000","last_became_peered":"2024-01-24T13:58:19.215566+0000","last_unstale":"2024-01-24T13:58:23.211617+0000","last_undegraded":"2024-01-24T13:58:23.211617+0000","last_fullsized":"2024-01-24T13:58:23.211617+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212581+0000","last_change":"2024-01-24T13:58:18.808952+0000","last_active":"2024-01-24T13:58:23.212581+0000","last_peered":"2024-01-24T13:58:23.212581+0000","last_clean":"2024-01-24T13:58:23.212581+0000","last_became_active":"2024-01-24T13:58:18.808672+0000","last_became_peered":"2024-01-24T13:58:18.808672+0000","last_unstale":"2024-01-24T13:58:23.212581+0000","last_undegraded":"2024-01-24T13:58:23.212581+0000","last_fullsized":"2024-01-24T13:58:23.212581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211211+0000","last_change":"2024-01-24T13:58:19.213798+0000","last_active":"2024-01-24T13:58:23.211211+0000","last_peered":"2024-01-24T13:58:23.211211+0000","last_clean":"2024-01-24T13:58:23.211211+0000","last_became_active":"2024-01-24T13:58:19.212458+0000","last_became_peered":"2024-01-24T13:58:19.212458+0000","last_unstale":"2024-01-24T13:58:23.211211+0000","last_undegraded":"2024-01-24T13:58:23.211211+0000","last_fullsized":"2024-01-24T13:58:23.211211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238030+0000","last_change":"2024-01-24T13:58:18.811471+0000","last_active":"2024-01-24T13:58:23.238030+0000","last_peered":"2024-01-24T13:58:23.238030+0000","last_clean":"2024-01-24T13:58:23.238030+0000","last_became_active":"2024-01-24T13:58:18.811159+0000","last_became_peered":"2024-01-24T13:58:18.811159+0000","last_unstale":"2024-01-24T13:58:23.238030+0000","last_undegraded":"2024-01-24T13:58:23.238030+0000","last_fullsized":"2024-01-24T13:58:23.238030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209454+0000","last_change":"2024-01-24T13:58:19.219125+0000","last_active":"2024-01-24T13:58:23.209454+0000","last_peered":"2024-01-24T13:58:23.209454+0000","last_clean":"2024-01-24T13:58:23.209454+0000","last_became_active":"2024-01-24T13:58:19.218823+0000","last_became_peered":"2024-01-24T13:58:19.218823+0000","last_unstale":"2024-01-24T13:58:23.209454+0000","last_undegraded":"2024-01-24T13:58:23.209454+0000","last_fullsized":"2024-01-24T13:58:23.209454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241886+0000","last_change":"2024-01-24T13:58:23.236176+0000","last_active":"2024-01-24T13:58:23.241886+0000","last_peered":"2024-01-24T13:58:23.241886+0000","last_clean":"2024-01-24T13:58:23.241886+0000","last_became_active":"2024-01-24T13:58:23.235658+0000","last_became_peered":"2024-01-24T13:58:23.235658+0000","last_unstale":"2024-01-24T13:58:23.241886+0000","last_undegraded":"2024-01-24T13:58:23.241886+0000","last_fullsized":"2024-01-24T13:58:23.241886+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207471+0000","last_change":"2024-01-24T13:58:18.806459+0000","last_active":"2024-01-24T13:58:23.207471+0000","last_peered":"2024-01-24T13:58:23.207471+0000","last_clean":"2024-01-24T13:58:23.207471+0000","last_became_active":"2024-01-24T13:58:18.806150+0000","last_became_peered":"2024-01-24T13:58:18.806150+0000","last_unstale":"2024-01-24T13:58:23.207471+0000","last_undegraded":"2024-01-24T13:58:23.207471+0000","last_fullsized":"2024-01-24T13:58:23.207471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207748+0000","last_change":"2024-01-24T13:58:19.211910+0000","last_active":"2024-01-24T13:58:23.207748+0000","last_peered":"2024-01-24T13:58:23.207748+0000","last_clean":"2024-01-24T13:58:23.207748+0000","last_became_active":"2024-01-24T13:58:19.210919+0000","last_became_peered":"2024-01-24T13:58:19.210919+0000","last_unstale":"2024-01-24T13:58:23.207748+0000","last_undegraded":"2024-01-24T13:58:23.207748+0000","last_fullsized":"2024-01-24T13:58:23.207748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232409+0000","last_change":"2024-01-24T13:58:23.223847+0000","last_active":"2024-01-24T13:58:23.232409+0000","last_peered":"2024-01-24T13:58:23.232409+0000","last_clean":"2024-01-24T13:58:23.232409+0000","last_became_active":"2024-01-24T13:58:23.222064+0000","last_became_peered":"2024-01-24T13:58:23.222064+0000","last_unstale":"2024-01-24T13:58:23.232409+0000","last_undegraded":"2024-01-24T13:58:23.232409+0000","last_fullsized":"2024-01-24T13:58:23.232409+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227575+0000","last_change":"2024-01-24T13:58:18.798060+0000","last_active":"2024-01-24T13:58:23.227575+0000","last_peered":"2024-01-24T13:58:23.227575+0000","last_clean":"2024-01-24T13:58:23.227575+0000","last_became_active":"2024-01-24T13:58:18.797774+0000","last_became_peered":"2024-01-24T13:58:18.797774+0000","last_unstale":"2024-01-24T13:58:23.227575+0000","last_undegraded":"2024-01-24T13:58:23.227575+0000","last_fullsized":"2024-01-24T13:58:23.227575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208950+0000","last_change":"2024-01-24T13:58:19.215145+0000","last_active":"2024-01-24T13:58:23.208950+0000","last_peered":"2024-01-24T13:58:23.208950+0000","last_clean":"2024-01-24T13:58:23.208950+0000","last_became_active":"2024-01-24T13:58:19.214840+0000","last_became_peered":"2024-01-24T13:58:19.214840+0000","last_unstale":"2024-01-24T13:58:23.208950+0000","last_undegraded":"2024-01-24T13:58:23.208950+0000","last_fullsized":"2024-01-24T13:58:23.208950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237348+0000","last_change":"2024-01-24T13:58:23.234486+0000","last_active":"2024-01-24T13:58:23.237348+0000","last_peered":"2024-01-24T13:58:23.237348+0000","last_clean":"2024-01-24T13:58:23.237348+0000","last_became_active":"2024-01-24T13:58:23.233948+0000","last_became_peered":"2024-01-24T13:58:23.233948+0000","last_unstale":"2024-01-24T13:58:23.237348+0000","last_undegraded":"2024-01-24T13:58:23.237348+0000","last_fullsized":"2024-01-24T13:58:23.237348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227976+0000","last_change":"2024-01-24T13:58:18.797763+0000","last_active":"2024-01-24T13:58:23.227976+0000","last_peered":"2024-01-24T13:58:23.227976+0000","last_clean":"2024-01-24T13:58:23.227976+0000","last_became_active":"2024-01-24T13:58:18.797478+0000","last_became_peered":"2024-01-24T13:58:18.797478+0000","last_unstale":"2024-01-24T13:58:23.227976+0000","last_undegraded":"2024-01-24T13:58:23.227976+0000","last_fullsized":"2024-01-24T13:58:23.227976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209496+0000","last_change":"2024-01-24T13:58:19.217185+0000","last_active":"2024-01-24T13:58:23.209496+0000","last_peered":"2024-01-24T13:58:23.209496+0000","last_clean":"2024-01-24T13:58:23.209496+0000","last_became_active":"2024-01-24T13:58:19.216628+0000","last_became_peered":"2024-01-24T13:58:19.216628+0000","last_unstale":"2024-01-24T13:58:23.209496+0000","last_undegraded":"2024-01-24T13:58:23.209496+0000","last_fullsized":"2024-01-24T13:58:23.209496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.236747+0000","last_change":"2024-01-24T13:58:23.230969+0000","last_active":"2024-01-24T13:58:23.236747+0000","last_peered":"2024-01-24T13:58:23.236747+0000","last_clean":"2024-01-24T13:58:23.236747+0000","last_became_active":"2024-01-24T13:58:23.230450+0000","last_became_peered":"2024-01-24T13:58:23.230450+0000","last_unstale":"2024-01-24T13:58:23.236747+0000","last_undegraded":"2024-01-24T13:58:23.236747+0000","last_fullsized":"2024-01-24T13:58:23.236747+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241684+0000","last_change":"2024-01-24T13:58:23.231930+0000","last_active":"2024-01-24T13:58:23.241684+0000","last_peered":"2024-01-24T13:58:23.241684+0000","last_clean":"2024-01-24T13:58:23.241684+0000","last_became_active":"2024-01-24T13:58:23.231494+0000","last_became_peered":"2024-01-24T13:58:23.231494+0000","last_unstale":"2024-01-24T13:58:23.241684+0000","last_undegraded":"2024-01-24T13:58:23.241684+0000","last_fullsized":"2024-01-24T13:58:23.241684+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235293+0000","last_change":"2024-01-24T13:58:18.811358+0000","last_active":"2024-01-24T13:58:23.235293+0000","last_peered":"2024-01-24T13:58:23.235293+0000","last_clean":"2024-01-24T13:58:23.235293+0000","last_became_active":"2024-01-24T13:58:18.811087+0000","last_became_peered":"2024-01-24T13:58:18.811087+0000","last_unstale":"2024-01-24T13:58:23.235293+0000","last_undegraded":"2024-01-24T13:58:23.235293+0000","last_fullsized":"2024-01-24T13:58:23.235293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224192+0000","last_change":"2024-01-24T13:58:19.213488+0000","last_active":"2024-01-24T13:58:23.224192+0000","last_peered":"2024-01-24T13:58:23.224192+0000","last_clean":"2024-01-24T13:58:23.224192+0000","last_became_active":"2024-01-24T13:58:19.213211+0000","last_became_peered":"2024-01-24T13:58:19.213211+0000","last_unstale":"2024-01-24T13:58:23.224192+0000","last_undegraded":"2024-01-24T13:58:23.224192+0000","last_fullsized":"2024-01-24T13:58:23.224192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209928+0000","last_change":"2024-01-24T13:58:19.216228+0000","last_active":"2024-01-24T13:58:23.209928+0000","last_peered":"2024-01-24T13:58:23.209928+0000","last_clean":"2024-01-24T13:58:23.209928+0000","last_became_active":"2024-01-24T13:58:19.215952+0000","last_became_peered":"2024-01-24T13:58:19.215952+0000","last_unstale":"2024-01-24T13:58:23.209928+0000","last_undegraded":"2024-01-24T13:58:23.209928+0000","last_fullsized":"2024-01-24T13:58:23.209928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237483+0000","last_change":"2024-01-24T13:58:18.813343+0000","last_active":"2024-01-24T13:58:23.237483+0000","last_peered":"2024-01-24T13:58:23.237483+0000","last_clean":"2024-01-24T13:58:23.237483+0000","last_became_active":"2024-01-24T13:58:18.812987+0000","last_became_peered":"2024-01-24T13:58:18.812987+0000","last_unstale":"2024-01-24T13:58:23.237483+0000","last_undegraded":"2024-01-24T13:58:23.237483+0000","last_fullsized":"2024-01-24T13:58:23.237483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213473+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T13:58:23.213473+0000","last_peered":"2024-01-24T13:58:23.213473+0000","last_clean":"2024-01-24T13:58:23.213473+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T13:58:23.213473+0000","last_undegraded":"2024-01-24T13:58:23.213473+0000","last_fullsized":"2024-01-24T13:58:23.213473+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241577+0000","last_change":"2024-01-24T13:58:23.235347+0000","last_active":"2024-01-24T13:58:23.241577+0000","last_peered":"2024-01-24T13:58:23.241577+0000","last_clean":"2024-01-24T13:58:23.241577+0000","last_became_active":"2024-01-24T13:58:23.234902+0000","last_became_peered":"2024-01-24T13:58:23.234902+0000","last_unstale":"2024-01-24T13:58:23.241577+0000","last_undegraded":"2024-01-24T13:58:23.241577+0000","last_fullsized":"2024-01-24T13:58:23.241577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207482+0000","last_change":"2024-01-24T13:58:19.218418+0000","last_active":"2024-01-24T13:58:23.207482+0000","last_peered":"2024-01-24T13:58:23.207482+0000","last_clean":"2024-01-24T13:58:23.207482+0000","last_became_active":"2024-01-24T13:58:19.210988+0000","last_became_peered":"2024-01-24T13:58:19.210988+0000","last_unstale":"2024-01-24T13:58:23.207482+0000","last_undegraded":"2024-01-24T13:58:23.207482+0000","last_fullsized":"2024-01-24T13:58:23.207482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211503+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T13:58:23.211503+0000","last_peered":"2024-01-24T13:58:23.211503+0000","last_clean":"2024-01-24T13:58:23.211503+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T13:58:23.211503+0000","last_undegraded":"2024-01-24T13:58:23.211503+0000","last_fullsized":"2024-01-24T13:58:23.211503+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207918+0000","last_change":"2024-01-24T13:58:18.798154+0000","last_active":"2024-01-24T13:58:23.207918+0000","last_peered":"2024-01-24T13:58:23.207918+0000","last_clean":"2024-01-24T13:58:23.207918+0000","last_became_active":"2024-01-24T13:58:18.797819+0000","last_became_peered":"2024-01-24T13:58:18.797819+0000","last_unstale":"2024-01-24T13:58:23.207918+0000","last_undegraded":"2024-01-24T13:58:23.207918+0000","last_fullsized":"2024-01-24T13:58:23.207918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232475+0000","last_change":"2024-01-24T13:58:23.230154+0000","last_active":"2024-01-24T13:58:23.232475+0000","last_peered":"2024-01-24T13:58:23.232475+0000","last_clean":"2024-01-24T13:58:23.232475+0000","last_became_active":"2024-01-24T13:58:23.226934+0000","last_became_peered":"2024-01-24T13:58:23.226934+0000","last_unstale":"2024-01-24T13:58:23.232475+0000","last_undegraded":"2024-01-24T13:58:23.232475+0000","last_fullsized":"2024-01-24T13:58:23.232475+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235379+0000","last_change":"2024-01-24T13:58:19.223917+0000","last_active":"2024-01-24T13:58:23.235379+0000","last_peered":"2024-01-24T13:58:23.235379+0000","last_clean":"2024-01-24T13:58:23.235379+0000","last_became_active":"2024-01-24T13:58:19.223624+0000","last_became_peered":"2024-01-24T13:58:19.223624+0000","last_unstale":"2024-01-24T13:58:23.235379+0000","last_undegraded":"2024-01-24T13:58:23.235379+0000","last_fullsized":"2024-01-24T13:58:23.235379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211872+0000","last_change":"2024-01-24T13:58:18.799495+0000","last_active":"2024-01-24T13:58:23.211872+0000","last_peered":"2024-01-24T13:58:23.211872+0000","last_clean":"2024-01-24T13:58:23.211872+0000","last_became_active":"2024-01-24T13:58:18.796442+0000","last_became_peered":"2024-01-24T13:58:18.796442+0000","last_unstale":"2024-01-24T13:58:23.211872+0000","last_undegraded":"2024-01-24T13:58:23.211872+0000","last_fullsized":"2024-01-24T13:58:23.211872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217101+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T13:58:23.217101+0000","last_peered":"2024-01-24T13:58:23.217101+0000","last_clean":"2024-01-24T13:58:23.217101+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T13:58:23.217101+0000","last_undegraded":"2024-01-24T13:58:23.217101+0000","last_fullsized":"2024-01-24T13:58:23.217101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.230573+0000","last_change":"2024-01-24T13:58:23.228846+0000","last_active":"2024-01-24T13:58:23.230573+0000","last_peered":"2024-01-24T13:58:23.230573+0000","last_clean":"2024-01-24T13:58:23.230573+0000","last_became_active":"2024-01-24T13:58:23.228366+0000","last_became_peered":"2024-01-24T13:58:23.228366+0000","last_unstale":"2024-01-24T13:58:23.230573+0000","last_undegraded":"2024-01-24T13:58:23.230573+0000","last_fullsized":"2024-01-24T13:58:23.230573+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232638+0000","last_change":"2024-01-24T13:58:18.810910+0000","last_active":"2024-01-24T13:58:23.232638+0000","last_peered":"2024-01-24T13:58:23.232638+0000","last_clean":"2024-01-24T13:58:23.232638+0000","last_became_active":"2024-01-24T13:58:18.810566+0000","last_became_peered":"2024-01-24T13:58:18.810566+0000","last_unstale":"2024-01-24T13:58:23.232638+0000","last_undegraded":"2024-01-24T13:58:23.232638+0000","last_fullsized":"2024-01-24T13:58:23.232638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216517+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T13:58:23.216517+0000","last_peered":"2024-01-24T13:58:23.216517+0000","last_clean":"2024-01-24T13:58:23.216517+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T13:58:23.216517+0000","last_undegraded":"2024-01-24T13:58:23.216517+0000","last_fullsized":"2024-01-24T13:58:23.216517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237683+0000","last_change":"2024-01-24T13:58:19.209151+0000","last_active":"2024-01-24T13:58:23.237683+0000","last_peered":"2024-01-24T13:58:23.237683+0000","last_clean":"2024-01-24T13:58:23.237683+0000","last_became_active":"2024-01-24T13:58:19.208680+0000","last_became_peered":"2024-01-24T13:58:19.208680+0000","last_unstale":"2024-01-24T13:58:23.237683+0000","last_undegraded":"2024-01-24T13:58:23.237683+0000","last_fullsized":"2024-01-24T13:58:23.237683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237878+0000","last_change":"2024-01-24T13:58:23.235219+0000","last_active":"2024-01-24T13:58:23.237878+0000","last_peered":"2024-01-24T13:58:23.237878+0000","last_clean":"2024-01-24T13:58:23.237878+0000","last_became_active":"2024-01-24T13:58:23.234797+0000","last_became_peered":"2024-01-24T13:58:23.234797+0000","last_unstale":"2024-01-24T13:58:23.237878+0000","last_undegraded":"2024-01-24T13:58:23.237878+0000","last_fullsized":"2024-01-24T13:58:23.237878+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207091+0000","last_change":"2024-01-24T13:58:18.802046+0000","last_active":"2024-01-24T13:58:23.207091+0000","last_peered":"2024-01-24T13:58:23.207091+0000","last_clean":"2024-01-24T13:58:23.207091+0000","last_became_active":"2024-01-24T13:58:18.801757+0000","last_became_peered":"2024-01-24T13:58:18.801757+0000","last_unstale":"2024-01-24T13:58:23.207091+0000","last_undegraded":"2024-01-24T13:58:23.207091+0000","last_fullsized":"2024-01-24T13:58:23.207091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216324+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T13:58:23.216324+0000","last_peered":"2024-01-24T13:58:23.216324+0000","last_clean":"2024-01-24T13:58:23.216324+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T13:58:23.216324+0000","last_undegraded":"2024-01-24T13:58:23.216324+0000","last_fullsized":"2024-01-24T13:58:23.216324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227454+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T13:58:23.227454+0000","last_peered":"2024-01-24T13:58:23.227454+0000","last_clean":"2024-01-24T13:58:23.227454+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T13:58:23.227454+0000","last_undegraded":"2024-01-24T13:58:23.227454+0000","last_fullsized":"2024-01-24T13:58:23.227454+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212923+0000","last_change":"2024-01-24T13:58:19.211323+0000","last_active":"2024-01-24T13:58:23.212923+0000","last_peered":"2024-01-24T13:58:23.212923+0000","last_clean":"2024-01-24T13:58:23.212923+0000","last_became_active":"2024-01-24T13:58:19.211057+0000","last_became_peered":"2024-01-24T13:58:19.211057+0000","last_unstale":"2024-01-24T13:58:23.212923+0000","last_undegraded":"2024-01-24T13:58:23.212923+0000","last_fullsized":"2024-01-24T13:58:23.212923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229871+0000","last_change":"2024-01-24T13:58:23.227723+0000","last_active":"2024-01-24T13:58:23.229871+0000","last_peered":"2024-01-24T13:58:23.229871+0000","last_clean":"2024-01-24T13:58:23.229871+0000","last_became_active":"2024-01-24T13:58:23.227339+0000","last_became_peered":"2024-01-24T13:58:23.227339+0000","last_unstale":"2024-01-24T13:58:23.229871+0000","last_undegraded":"2024-01-24T13:58:23.229871+0000","last_fullsized":"2024-01-24T13:58:23.229871+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.210205+0000","last_change":"2024-01-24T13:58:18.798895+0000","last_active":"2024-01-24T13:58:23.210205+0000","last_peered":"2024-01-24T13:58:23.210205+0000","last_clean":"2024-01-24T13:58:23.210205+0000","last_became_active":"2024-01-24T13:58:18.796782+0000","last_became_peered":"2024-01-24T13:58:18.796782+0000","last_unstale":"2024-01-24T13:58:23.210205+0000","last_undegraded":"2024-01-24T13:58:23.210205+0000","last_fullsized":"2024-01-24T13:58:23.210205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220366+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T13:58:23.220366+0000","last_peered":"2024-01-24T13:58:23.220366+0000","last_clean":"2024-01-24T13:58:23.220366+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T13:58:23.220366+0000","last_undegraded":"2024-01-24T13:58:23.220366+0000","last_fullsized":"2024-01-24T13:58:23.220366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211623+0000","last_change":"2024-01-24T13:58:19.212508+0000","last_active":"2024-01-24T13:58:23.211623+0000","last_peered":"2024-01-24T13:58:23.211623+0000","last_clean":"2024-01-24T13:58:23.211623+0000","last_became_active":"2024-01-24T13:58:19.212211+0000","last_became_peered":"2024-01-24T13:58:19.212211+0000","last_unstale":"2024-01-24T13:58:23.211623+0000","last_undegraded":"2024-01-24T13:58:23.211623+0000","last_fullsized":"2024-01-24T13:58:23.211623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241355+0000","last_change":"2024-01-24T13:58:23.238939+0000","last_active":"2024-01-24T13:58:23.241355+0000","last_peered":"2024-01-24T13:58:23.241355+0000","last_clean":"2024-01-24T13:58:23.241355+0000","last_became_active":"2024-01-24T13:58:23.234940+0000","last_became_peered":"2024-01-24T13:58:23.234940+0000","last_unstale":"2024-01-24T13:58:23.241355+0000","last_undegraded":"2024-01-24T13:58:23.241355+0000","last_fullsized":"2024-01-24T13:58:23.241355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211492+0000","last_change":"2024-01-24T13:58:19.213076+0000","last_active":"2024-01-24T13:58:23.211492+0000","last_peered":"2024-01-24T13:58:23.211492+0000","last_clean":"2024-01-24T13:58:23.211492+0000","last_became_active":"2024-01-24T13:58:19.208454+0000","last_became_peered":"2024-01-24T13:58:19.208454+0000","last_unstale":"2024-01-24T13:58:23.211492+0000","last_undegraded":"2024-01-24T13:58:23.211492+0000","last_fullsized":"2024-01-24T13:58:23.211492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209311+0000","last_change":"2024-01-24T13:58:18.806850+0000","last_active":"2024-01-24T13:58:23.209311+0000","last_peered":"2024-01-24T13:58:23.209311+0000","last_clean":"2024-01-24T13:58:23.209311+0000","last_became_active":"2024-01-24T13:58:18.806456+0000","last_became_peered":"2024-01-24T13:58:18.806456+0000","last_unstale":"2024-01-24T13:58:23.209311+0000","last_undegraded":"2024-01-24T13:58:23.209311+0000","last_fullsized":"2024-01-24T13:58:23.209311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226775+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T13:58:23.226775+0000","last_peered":"2024-01-24T13:58:23.226775+0000","last_clean":"2024-01-24T13:58:23.226775+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T13:58:23.226775+0000","last_undegraded":"2024-01-24T13:58:23.226775+0000","last_fullsized":"2024-01-24T13:58:23.226775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238433+0000","last_change":"2024-01-24T13:58:23.237234+0000","last_active":"2024-01-24T13:58:23.238433+0000","last_peered":"2024-01-24T13:58:23.238433+0000","last_clean":"2024-01-24T13:58:23.238433+0000","last_became_active":"2024-01-24T13:58:23.236720+0000","last_became_peered":"2024-01-24T13:58:23.236720+0000","last_unstale":"2024-01-24T13:58:23.238433+0000","last_undegraded":"2024-01-24T13:58:23.238433+0000","last_fullsized":"2024-01-24T13:58:23.238433+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212103+0000","last_change":"2024-01-24T13:58:19.217116+0000","last_active":"2024-01-24T13:58:23.212103+0000","last_peered":"2024-01-24T13:58:23.212103+0000","last_clean":"2024-01-24T13:58:23.212103+0000","last_became_active":"2024-01-24T13:58:19.216731+0000","last_became_peered":"2024-01-24T13:58:19.216731+0000","last_unstale":"2024-01-24T13:58:23.212103+0000","last_undegraded":"2024-01-24T13:58:23.212103+0000","last_fullsized":"2024-01-24T13:58:23.212103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227689+0000","last_change":"2024-01-24T13:58:18.803400+0000","last_active":"2024-01-24T13:58:23.227689+0000","last_peered":"2024-01-24T13:58:23.227689+0000","last_clean":"2024-01-24T13:58:23.227689+0000","last_became_active":"2024-01-24T13:58:18.802706+0000","last_became_peered":"2024-01-24T13:58:18.802706+0000","last_unstale":"2024-01-24T13:58:23.227689+0000","last_undegraded":"2024-01-24T13:58:23.227689+0000","last_fullsized":"2024-01-24T13:58:23.227689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211632+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T13:58:23.211632+0000","last_peered":"2024-01-24T13:58:23.211632+0000","last_clean":"2024-01-24T13:58:23.211632+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T13:58:23.211632+0000","last_undegraded":"2024-01-24T13:58:23.211632+0000","last_fullsized":"2024-01-24T13:58:23.211632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237685+0000","last_change":"2024-01-24T13:58:23.235636+0000","last_active":"2024-01-24T13:58:23.237685+0000","last_peered":"2024-01-24T13:58:23.237685+0000","last_clean":"2024-01-24T13:58:23.237685+0000","last_became_active":"2024-01-24T13:58:23.234531+0000","last_became_peered":"2024-01-24T13:58:23.234531+0000","last_unstale":"2024-01-24T13:58:23.237685+0000","last_undegraded":"2024-01-24T13:58:23.237685+0000","last_fullsized":"2024-01-24T13:58:23.237685+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207375+0000","last_change":"2024-01-24T13:58:18.806700+0000","last_active":"2024-01-24T13:58:23.207375+0000","last_peered":"2024-01-24T13:58:23.207375+0000","last_clean":"2024-01-24T13:58:23.207375+0000","last_became_active":"2024-01-24T13:58:18.805917+0000","last_became_peered":"2024-01-24T13:58:18.805917+0000","last_unstale":"2024-01-24T13:58:23.207375+0000","last_undegraded":"2024-01-24T13:58:23.207375+0000","last_fullsized":"2024-01-24T13:58:23.207375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211671+0000","last_change":"2024-01-24T13:58:19.215828+0000","last_active":"2024-01-24T13:58:23.211671+0000","last_peered":"2024-01-24T13:58:23.211671+0000","last_clean":"2024-01-24T13:58:23.211671+0000","last_became_active":"2024-01-24T13:58:19.215015+0000","last_became_peered":"2024-01-24T13:58:19.215015+0000","last_unstale":"2024-01-24T13:58:23.211671+0000","last_undegraded":"2024-01-24T13:58:23.211671+0000","last_fullsized":"2024-01-24T13:58:23.211671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238346+0000","last_change":"2024-01-24T13:58:23.236631+0000","last_active":"2024-01-24T13:58:23.238346+0000","last_peered":"2024-01-24T13:58:23.238346+0000","last_clean":"2024-01-24T13:58:23.238346+0000","last_became_active":"2024-01-24T13:58:23.236200+0000","last_became_peered":"2024-01-24T13:58:23.236200+0000","last_unstale":"2024-01-24T13:58:23.238346+0000","last_undegraded":"2024-01-24T13:58:23.238346+0000","last_fullsized":"2024-01-24T13:58:23.238346+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209848+0000","last_change":"2024-01-24T13:58:18.803901+0000","last_active":"2024-01-24T13:58:23.209848+0000","last_peered":"2024-01-24T13:58:23.209848+0000","last_clean":"2024-01-24T13:58:23.209848+0000","last_became_active":"2024-01-24T13:58:18.803605+0000","last_became_peered":"2024-01-24T13:58:18.803605+0000","last_unstale":"2024-01-24T13:58:23.209848+0000","last_undegraded":"2024-01-24T13:58:23.209848+0000","last_fullsized":"2024-01-24T13:58:23.209848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209809+0000","last_change":"2024-01-24T13:58:19.216716+0000","last_active":"2024-01-24T13:58:23.209809+0000","last_peered":"2024-01-24T13:58:23.209809+0000","last_clean":"2024-01-24T13:58:23.209809+0000","last_became_active":"2024-01-24T13:58:19.216444+0000","last_became_peered":"2024-01-24T13:58:19.216444+0000","last_unstale":"2024-01-24T13:58:23.209809+0000","last_undegraded":"2024-01-24T13:58:23.209809+0000","last_fullsized":"2024-01-24T13:58:23.209809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.242821+0000","last_change":"2024-01-24T13:58:23.242205+0000","last_active":"2024-01-24T13:58:23.242821+0000","last_peered":"2024-01-24T13:58:23.242821+0000","last_clean":"2024-01-24T13:58:23.242821+0000","last_became_active":"2024-01-24T13:58:23.241764+0000","last_became_peered":"2024-01-24T13:58:23.241764+0000","last_unstale":"2024-01-24T13:58:23.242821+0000","last_undegraded":"2024-01-24T13:58:23.242821+0000","last_fullsized":"2024-01-24T13:58:23.242821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212581+0000","last_change":"2024-01-24T13:58:18.808313+0000","last_active":"2024-01-24T13:58:23.212581+0000","last_peered":"2024-01-24T13:58:23.212581+0000","last_clean":"2024-01-24T13:58:23.212581+0000","last_became_active":"2024-01-24T13:58:18.808096+0000","last_became_peered":"2024-01-24T13:58:18.808096+0000","last_unstale":"2024-01-24T13:58:23.212581+0000","last_undegraded":"2024-01-24T13:58:23.212581+0000","last_fullsized":"2024-01-24T13:58:23.212581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209286+0000","last_change":"2024-01-24T13:58:19.209979+0000","last_active":"2024-01-24T13:58:23.209286+0000","last_peered":"2024-01-24T13:58:23.209286+0000","last_clean":"2024-01-24T13:58:23.209286+0000","last_became_active":"2024-01-24T13:58:19.209672+0000","last_became_peered":"2024-01-24T13:58:19.209672+0000","last_unstale":"2024-01-24T13:58:23.209286+0000","last_undegraded":"2024-01-24T13:58:23.209286+0000","last_fullsized":"2024-01-24T13:58:23.209286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.240684+0000","last_change":"2024-01-24T13:58:23.229668+0000","last_active":"2024-01-24T13:58:23.240684+0000","last_peered":"2024-01-24T13:58:23.240684+0000","last_clean":"2024-01-24T13:58:23.240684+0000","last_became_active":"2024-01-24T13:58:23.229053+0000","last_became_peered":"2024-01-24T13:58:23.229053+0000","last_unstale":"2024-01-24T13:58:23.240684+0000","last_undegraded":"2024-01-24T13:58:23.240684+0000","last_fullsized":"2024-01-24T13:58:23.240684+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207919+0000","last_change":"2024-01-24T13:58:18.807736+0000","last_active":"2024-01-24T13:58:23.207919+0000","last_peered":"2024-01-24T13:58:23.207919+0000","last_clean":"2024-01-24T13:58:23.207919+0000","last_became_active":"2024-01-24T13:58:18.807228+0000","last_became_peered":"2024-01-24T13:58:18.807228+0000","last_unstale":"2024-01-24T13:58:23.207919+0000","last_undegraded":"2024-01-24T13:58:23.207919+0000","last_fullsized":"2024-01-24T13:58:23.207919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209909+0000","last_change":"2024-01-24T13:58:19.218038+0000","last_active":"2024-01-24T13:58:23.209909+0000","last_peered":"2024-01-24T13:58:23.209909+0000","last_clean":"2024-01-24T13:58:23.209909+0000","last_became_active":"2024-01-24T13:58:19.210668+0000","last_became_peered":"2024-01-24T13:58:19.210668+0000","last_unstale":"2024-01-24T13:58:23.209909+0000","last_undegraded":"2024-01-24T13:58:23.209909+0000","last_fullsized":"2024-01-24T13:58:23.209909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241201+0000","last_change":"2024-01-24T13:58:23.234754+0000","last_active":"2024-01-24T13:58:23.241201+0000","last_peered":"2024-01-24T13:58:23.241201+0000","last_clean":"2024-01-24T13:58:23.241201+0000","last_became_active":"2024-01-24T13:58:23.233898+0000","last_became_peered":"2024-01-24T13:58:23.233898+0000","last_unstale":"2024-01-24T13:58:23.241201+0000","last_undegraded":"2024-01-24T13:58:23.241201+0000","last_fullsized":"2024-01-24T13:58:23.241201+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227851+0000","last_change":"2024-01-24T13:58:18.802179+0000","last_active":"2024-01-24T13:58:23.227851+0000","last_peered":"2024-01-24T13:58:23.227851+0000","last_clean":"2024-01-24T13:58:23.227851+0000","last_became_active":"2024-01-24T13:58:18.801898+0000","last_became_peered":"2024-01-24T13:58:18.801898+0000","last_unstale":"2024-01-24T13:58:23.227851+0000","last_undegraded":"2024-01-24T13:58:23.227851+0000","last_fullsized":"2024-01-24T13:58:23.227851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.234948+0000","last_change":"2024-01-24T13:58:19.224275+0000","last_active":"2024-01-24T13:58:23.234948+0000","last_peered":"2024-01-24T13:58:23.234948+0000","last_clean":"2024-01-24T13:58:23.234948+0000","last_became_active":"2024-01-24T13:58:19.224006+0000","last_became_peered":"2024-01-24T13:58:19.224006+0000","last_unstale":"2024-01-24T13:58:23.234948+0000","last_undegraded":"2024-01-24T13:58:23.234948+0000","last_fullsized":"2024-01-24T13:58:23.234948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241374+0000","last_change":"2024-01-24T13:58:23.239977+0000","last_active":"2024-01-24T13:58:23.241374+0000","last_peered":"2024-01-24T13:58:23.241374+0000","last_clean":"2024-01-24T13:58:23.241374+0000","last_became_active":"2024-01-24T13:58:23.234358+0000","last_became_peered":"2024-01-24T13:58:23.234358+0000","last_unstale":"2024-01-24T13:58:23.241374+0000","last_undegraded":"2024-01-24T13:58:23.241374+0000","last_fullsized":"2024-01-24T13:58:23.241374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217009+0000","last_change":"2024-01-24T13:58:18.804317+0000","last_active":"2024-01-24T13:58:23.217009+0000","last_peered":"2024-01-24T13:58:23.217009+0000","last_clean":"2024-01-24T13:58:23.217009+0000","last_became_active":"2024-01-24T13:58:18.804046+0000","last_became_peered":"2024-01-24T13:58:18.804046+0000","last_unstale":"2024-01-24T13:58:23.217009+0000","last_undegraded":"2024-01-24T13:58:23.217009+0000","last_fullsized":"2024-01-24T13:58:23.217009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212503+0000","last_change":"2024-01-24T13:58:19.217782+0000","last_active":"2024-01-24T13:58:23.212503+0000","last_peered":"2024-01-24T13:58:23.212503+0000","last_clean":"2024-01-24T13:58:23.212503+0000","last_became_active":"2024-01-24T13:58:19.217509+0000","last_became_peered":"2024-01-24T13:58:19.217509+0000","last_unstale":"2024-01-24T13:58:23.212503+0000","last_undegraded":"2024-01-24T13:58:23.212503+0000","last_fullsized":"2024-01-24T13:58:23.212503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241249+0000","last_change":"2024-01-24T13:58:23.237049+0000","last_active":"2024-01-24T13:58:23.241249+0000","last_peered":"2024-01-24T13:58:23.241249+0000","last_clean":"2024-01-24T13:58:23.241249+0000","last_became_active":"2024-01-24T13:58:23.235154+0000","last_became_peered":"2024-01-24T13:58:23.235154+0000","last_unstale":"2024-01-24T13:58:23.241249+0000","last_undegraded":"2024-01-24T13:58:23.241249+0000","last_fullsized":"2024-01-24T13:58:23.241249+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217784+0000","last_change":"2024-01-24T13:58:18.801841+0000","last_active":"2024-01-24T13:58:23.217784+0000","last_peered":"2024-01-24T13:58:23.217784+0000","last_clean":"2024-01-24T13:58:23.217784+0000","last_became_active":"2024-01-24T13:58:18.801535+0000","last_became_peered":"2024-01-24T13:58:18.801535+0000","last_unstale":"2024-01-24T13:58:23.217784+0000","last_undegraded":"2024-01-24T13:58:23.217784+0000","last_fullsized":"2024-01-24T13:58:23.217784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209722+0000","last_change":"2024-01-24T13:58:19.213673+0000","last_active":"2024-01-24T13:58:23.209722+0000","last_peered":"2024-01-24T13:58:23.209722+0000","last_clean":"2024-01-24T13:58:23.209722+0000","last_became_active":"2024-01-24T13:58:19.213457+0000","last_became_peered":"2024-01-24T13:58:19.213457+0000","last_unstale":"2024-01-24T13:58:23.209722+0000","last_undegraded":"2024-01-24T13:58:23.209722+0000","last_fullsized":"2024-01-24T13:58:23.209722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238263+0000","last_change":"2024-01-24T13:58:23.234302+0000","last_active":"2024-01-24T13:58:23.238263+0000","last_peered":"2024-01-24T13:58:23.238263+0000","last_clean":"2024-01-24T13:58:23.238263+0000","last_became_active":"2024-01-24T13:58:23.233692+0000","last_became_peered":"2024-01-24T13:58:23.233692+0000","last_unstale":"2024-01-24T13:58:23.238263+0000","last_undegraded":"2024-01-24T13:58:23.238263+0000","last_fullsized":"2024-01-24T13:58:23.238263+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.218145+0000","last_change":"2024-01-24T13:58:18.798076+0000","last_active":"2024-01-24T13:58:23.218145+0000","last_peered":"2024-01-24T13:58:23.218145+0000","last_clean":"2024-01-24T13:58:23.218145+0000","last_became_active":"2024-01-24T13:58:18.797403+0000","last_became_peered":"2024-01-24T13:58:18.797403+0000","last_unstale":"2024-01-24T13:58:23.218145+0000","last_undegraded":"2024-01-24T13:58:23.218145+0000","last_fullsized":"2024-01-24T13:58:23.218145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224903+0000","last_change":"2024-01-24T13:58:19.210870+0000","last_active":"2024-01-24T13:58:23.224903+0000","last_peered":"2024-01-24T13:58:23.224903+0000","last_clean":"2024-01-24T13:58:23.224903+0000","last_became_active":"2024-01-24T13:58:19.210226+0000","last_became_peered":"2024-01-24T13:58:19.210226+0000","last_unstale":"2024-01-24T13:58:23.224903+0000","last_undegraded":"2024-01-24T13:58:23.224903+0000","last_fullsized":"2024-01-24T13:58:23.224903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232630+0000","last_change":"2024-01-24T13:58:23.230340+0000","last_active":"2024-01-24T13:58:23.232630+0000","last_peered":"2024-01-24T13:58:23.232630+0000","last_clean":"2024-01-24T13:58:23.232630+0000","last_became_active":"2024-01-24T13:58:23.229909+0000","last_became_peered":"2024-01-24T13:58:23.229909+0000","last_unstale":"2024-01-24T13:58:23.232630+0000","last_undegraded":"2024-01-24T13:58:23.232630+0000","last_fullsized":"2024-01-24T13:58:23.232630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"21'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212395+0000","last_change":"2024-01-24T13:58:18.813040+0000","last_active":"2024-01-24T13:58:23.212395+0000","last_peered":"2024-01-24T13:58:23.212395+0000","last_clean":"2024-01-24T13:58:23.212395+0000","last_became_active":"2024-01-24T13:58:18.812822+0000","last_became_peered":"2024-01-24T13:58:18.812822+0000","last_unstale":"2024-01-24T13:58:23.212395+0000","last_undegraded":"2024-01-24T13:58:23.212395+0000","last_fullsized":"2024-01-24T13:58:23.212395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208215+0000","last_change":"2024-01-24T13:58:19.216414+0000","last_active":"2024-01-24T13:58:23.208215+0000","last_peered":"2024-01-24T13:58:23.208215+0000","last_clean":"2024-01-24T13:58:23.208215+0000","last_became_active":"2024-01-24T13:58:19.216111+0000","last_became_peered":"2024-01-24T13:58:19.216111+0000","last_unstale":"2024-01-24T13:58:23.208215+0000","last_undegraded":"2024-01-24T13:58:23.208215+0000","last_fullsized":"2024-01-24T13:58:23.208215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237430+0000","last_change":"2024-01-24T13:58:23.234462+0000","last_active":"2024-01-24T13:58:23.237430+0000","last_peered":"2024-01-24T13:58:23.237430+0000","last_clean":"2024-01-24T13:58:23.237430+0000","last_became_active":"2024-01-24T13:58:23.233834+0000","last_became_peered":"2024-01-24T13:58:23.233834+0000","last_unstale":"2024-01-24T13:58:23.237430+0000","last_undegraded":"2024-01-24T13:58:23.237430+0000","last_fullsized":"2024-01-24T13:58:23.237430+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216341+0000","last_change":"2024-01-24T13:58:18.801184+0000","last_active":"2024-01-24T13:58:23.216341+0000","last_peered":"2024-01-24T13:58:23.216341+0000","last_clean":"2024-01-24T13:58:23.216341+0000","last_became_active":"2024-01-24T13:58:18.800877+0000","last_became_peered":"2024-01-24T13:58:18.800877+0000","last_unstale":"2024-01-24T13:58:23.216341+0000","last_undegraded":"2024-01-24T13:58:23.216341+0000","last_fullsized":"2024-01-24T13:58:23.216341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237737+0000","last_change":"2024-01-24T13:58:19.209281+0000","last_active":"2024-01-24T13:58:23.237737+0000","last_peered":"2024-01-24T13:58:23.237737+0000","last_clean":"2024-01-24T13:58:23.237737+0000","last_became_active":"2024-01-24T13:58:19.208890+0000","last_became_peered":"2024-01-24T13:58:19.208890+0000","last_unstale":"2024-01-24T13:58:23.237737+0000","last_undegraded":"2024-01-24T13:58:23.237737+0000","last_fullsized":"2024-01-24T13:58:23.237737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.230375+0000","last_change":"2024-01-24T13:58:23.228668+0000","last_active":"2024-01-24T13:58:23.230375+0000","last_peered":"2024-01-24T13:58:23.230375+0000","last_clean":"2024-01-24T13:58:23.230375+0000","last_became_active":"2024-01-24T13:58:23.228266+0000","last_became_peered":"2024-01-24T13:58:23.228266+0000","last_unstale":"2024-01-24T13:58:23.230375+0000","last_undegraded":"2024-01-24T13:58:23.230375+0000","last_fullsized":"2024-01-24T13:58:23.230375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232549+0000","last_change":"2024-01-24T13:58:18.809056+0000","last_active":"2024-01-24T13:58:23.232549+0000","last_peered":"2024-01-24T13:58:23.232549+0000","last_clean":"2024-01-24T13:58:23.232549+0000","last_became_active":"2024-01-24T13:58:18.808790+0000","last_became_peered":"2024-01-24T13:58:18.808790+0000","last_unstale":"2024-01-24T13:58:23.232549+0000","last_undegraded":"2024-01-24T13:58:23.232549+0000","last_fullsized":"2024-01-24T13:58:23.232549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227958+0000","last_change":"2024-01-24T13:58:19.217272+0000","last_active":"2024-01-24T13:58:23.227958+0000","last_peered":"2024-01-24T13:58:23.227958+0000","last_clean":"2024-01-24T13:58:23.227958+0000","last_became_active":"2024-01-24T13:58:19.216991+0000","last_became_peered":"2024-01-24T13:58:19.216991+0000","last_unstale":"2024-01-24T13:58:23.227958+0000","last_undegraded":"2024-01-24T13:58:23.227958+0000","last_fullsized":"2024-01-24T13:58:23.227958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.231091+0000","last_change":"2024-01-24T13:58:23.228760+0000","last_active":"2024-01-24T13:58:23.231091+0000","last_peered":"2024-01-24T13:58:23.231091+0000","last_clean":"2024-01-24T13:58:23.231091+0000","last_became_active":"2024-01-24T13:58:23.227345+0000","last_became_peered":"2024-01-24T13:58:23.227345+0000","last_unstale":"2024-01-24T13:58:23.231091+0000","last_undegraded":"2024-01-24T13:58:23.231091+0000","last_fullsized":"2024-01-24T13:58:23.231091+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217256+0000","last_change":"2024-01-24T13:58:18.798145+0000","last_active":"2024-01-24T13:58:23.217256+0000","last_peered":"2024-01-24T13:58:23.217256+0000","last_clean":"2024-01-24T13:58:23.217256+0000","last_became_active":"2024-01-24T13:58:18.797671+0000","last_became_peered":"2024-01-24T13:58:18.797671+0000","last_unstale":"2024-01-24T13:58:23.217256+0000","last_undegraded":"2024-01-24T13:58:23.217256+0000","last_fullsized":"2024-01-24T13:58:23.217256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207234+0000","last_change":"2024-01-24T13:58:19.221287+0000","last_active":"2024-01-24T13:58:23.207234+0000","last_peered":"2024-01-24T13:58:23.207234+0000","last_clean":"2024-01-24T13:58:23.207234+0000","last_became_active":"2024-01-24T13:58:19.220970+0000","last_became_peered":"2024-01-24T13:58:19.220970+0000","last_unstale":"2024-01-24T13:58:23.207234+0000","last_undegraded":"2024-01-24T13:58:23.207234+0000","last_fullsized":"2024-01-24T13:58:23.207234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237365+0000","last_change":"2024-01-24T13:58:23.234514+0000","last_active":"2024-01-24T13:58:23.237365+0000","last_peered":"2024-01-24T13:58:23.237365+0000","last_clean":"2024-01-24T13:58:23.237365+0000","last_became_active":"2024-01-24T13:58:23.233824+0000","last_became_peered":"2024-01-24T13:58:23.233824+0000","last_unstale":"2024-01-24T13:58:23.237365+0000","last_undegraded":"2024-01-24T13:58:23.237365+0000","last_fullsized":"2024-01-24T13:58:23.237365+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209685+0000","last_change":"2024-01-24T13:58:18.801316+0000","last_active":"2024-01-24T13:58:23.209685+0000","last_peered":"2024-01-24T13:58:23.209685+0000","last_clean":"2024-01-24T13:58:23.209685+0000","last_became_active":"2024-01-24T13:58:18.800089+0000","last_became_peered":"2024-01-24T13:58:18.800089+0000","last_unstale":"2024-01-24T13:58:23.209685+0000","last_undegraded":"2024-01-24T13:58:23.209685+0000","last_fullsized":"2024-01-24T13:58:23.209685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209210+0000","last_change":"2024-01-24T13:58:19.209604+0000","last_active":"2024-01-24T13:58:23.209210+0000","last_peered":"2024-01-24T13:58:23.209210+0000","last_clean":"2024-01-24T13:58:23.209210+0000","last_became_active":"2024-01-24T13:58:19.209306+0000","last_became_peered":"2024-01-24T13:58:19.209306+0000","last_unstale":"2024-01-24T13:58:23.209210+0000","last_undegraded":"2024-01-24T13:58:23.209210+0000","last_fullsized":"2024-01-24T13:58:23.209210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229399+0000","last_change":"2024-01-24T13:58:23.225921+0000","last_active":"2024-01-24T13:58:23.229399+0000","last_peered":"2024-01-24T13:58:23.229399+0000","last_clean":"2024-01-24T13:58:23.229399+0000","last_became_active":"2024-01-24T13:58:23.225413+0000","last_became_peered":"2024-01-24T13:58:23.225413+0000","last_unstale":"2024-01-24T13:58:23.229399+0000","last_undegraded":"2024-01-24T13:58:23.229399+0000","last_fullsized":"2024-01-24T13:58:23.229399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217204+0000","last_change":"2024-01-24T13:58:18.799227+0000","last_active":"2024-01-24T13:58:23.217204+0000","last_peered":"2024-01-24T13:58:23.217204+0000","last_clean":"2024-01-24T13:58:23.217204+0000","last_became_active":"2024-01-24T13:58:18.798927+0000","last_became_peered":"2024-01-24T13:58:18.798927+0000","last_unstale":"2024-01-24T13:58:23.217204+0000","last_undegraded":"2024-01-24T13:58:23.217204+0000","last_fullsized":"2024-01-24T13:58:23.217204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212286+0000","last_change":"2024-01-24T13:58:19.217668+0000","last_active":"2024-01-24T13:58:23.212286+0000","last_peered":"2024-01-24T13:58:23.212286+0000","last_clean":"2024-01-24T13:58:23.212286+0000","last_became_active":"2024-01-24T13:58:19.217323+0000","last_became_peered":"2024-01-24T13:58:19.217323+0000","last_unstale":"2024-01-24T13:58:23.212286+0000","last_undegraded":"2024-01-24T13:58:23.212286+0000","last_fullsized":"2024-01-24T13:58:23.212286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.230948+0000","last_change":"2024-01-24T13:58:23.229837+0000","last_active":"2024-01-24T13:58:23.230948+0000","last_peered":"2024-01-24T13:58:23.230948+0000","last_clean":"2024-01-24T13:58:23.230948+0000","last_became_active":"2024-01-24T13:58:23.229398+0000","last_became_peered":"2024-01-24T13:58:23.229398+0000","last_unstale":"2024-01-24T13:58:23.230948+0000","last_undegraded":"2024-01-24T13:58:23.230948+0000","last_fullsized":"2024-01-24T13:58:23.230948+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227679+0000","last_change":"2024-01-24T13:58:18.798763+0000","last_active":"2024-01-24T13:58:23.227679+0000","last_peered":"2024-01-24T13:58:23.227679+0000","last_clean":"2024-01-24T13:58:23.227679+0000","last_became_active":"2024-01-24T13:58:18.798451+0000","last_became_peered":"2024-01-24T13:58:18.798451+0000","last_unstale":"2024-01-24T13:58:23.227679+0000","last_undegraded":"2024-01-24T13:58:23.227679+0000","last_fullsized":"2024-01-24T13:58:23.227679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207643+0000","last_change":"2024-01-24T13:58:19.221011+0000","last_active":"2024-01-24T13:58:23.207643+0000","last_peered":"2024-01-24T13:58:23.207643+0000","last_clean":"2024-01-24T13:58:23.207643+0000","last_became_active":"2024-01-24T13:58:19.220727+0000","last_became_peered":"2024-01-24T13:58:19.220727+0000","last_unstale":"2024-01-24T13:58:23.207643+0000","last_undegraded":"2024-01-24T13:58:23.207643+0000","last_fullsized":"2024-01-24T13:58:23.207643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237556+0000","last_change":"2024-01-24T13:58:23.235211+0000","last_active":"2024-01-24T13:58:23.237556+0000","last_peered":"2024-01-24T13:58:23.237556+0000","last_clean":"2024-01-24T13:58:23.237556+0000","last_became_active":"2024-01-24T13:58:23.234791+0000","last_became_peered":"2024-01-24T13:58:23.234791+0000","last_unstale":"2024-01-24T13:58:23.237556+0000","last_undegraded":"2024-01-24T13:58:23.237556+0000","last_fullsized":"2024-01-24T13:58:23.237556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237921+0000","last_change":"2024-01-24T13:58:18.815292+0000","last_active":"2024-01-24T13:58:23.237921+0000","last_peered":"2024-01-24T13:58:23.237921+0000","last_clean":"2024-01-24T13:58:23.237921+0000","last_became_active":"2024-01-24T13:58:18.811889+0000","last_became_peered":"2024-01-24T13:58:18.811889+0000","last_unstale":"2024-01-24T13:58:23.237921+0000","last_undegraded":"2024-01-24T13:58:23.237921+0000","last_fullsized":"2024-01-24T13:58:23.237921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207065+0000","last_change":"2024-01-24T13:58:19.218174+0000","last_active":"2024-01-24T13:58:23.207065+0000","last_peered":"2024-01-24T13:58:23.207065+0000","last_clean":"2024-01-24T13:58:23.207065+0000","last_became_active":"2024-01-24T13:58:19.211059+0000","last_became_peered":"2024-01-24T13:58:19.211059+0000","last_unstale":"2024-01-24T13:58:23.207065+0000","last_undegraded":"2024-01-24T13:58:23.207065+0000","last_fullsized":"2024-01-24T13:58:23.207065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229596+0000","last_change":"2024-01-24T13:58:23.225806+0000","last_active":"2024-01-24T13:58:23.229596+0000","last_peered":"2024-01-24T13:58:23.229596+0000","last_clean":"2024-01-24T13:58:23.229596+0000","last_became_active":"2024-01-24T13:58:23.225425+0000","last_became_peered":"2024-01-24T13:58:23.225425+0000","last_unstale":"2024-01-24T13:58:23.229596+0000","last_undegraded":"2024-01-24T13:58:23.229596+0000","last_fullsized":"2024-01-24T13:58:23.229596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235654+0000","last_change":"2024-01-24T13:58:23.226907+0000","last_active":"2024-01-24T13:58:23.235654+0000","last_peered":"2024-01-24T13:58:23.235654+0000","last_clean":"2024-01-24T13:58:23.235654+0000","last_became_active":"2024-01-24T13:58:23.226413+0000","last_became_peered":"2024-01-24T13:58:23.226413+0000","last_unstale":"2024-01-24T13:58:23.235654+0000","last_undegraded":"2024-01-24T13:58:23.235654+0000","last_fullsized":"2024-01-24T13:58:23.235654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.206957+0000","last_change":"2024-01-24T13:58:18.806830+0000","last_active":"2024-01-24T13:58:23.206957+0000","last_peered":"2024-01-24T13:58:23.206957+0000","last_clean":"2024-01-24T13:58:23.206957+0000","last_became_active":"2024-01-24T13:58:18.806385+0000","last_became_peered":"2024-01-24T13:58:18.806385+0000","last_unstale":"2024-01-24T13:58:23.206957+0000","last_undegraded":"2024-01-24T13:58:23.206957+0000","last_fullsized":"2024-01-24T13:58:23.206957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212967+0000","last_change":"2024-01-24T13:58:19.210472+0000","last_active":"2024-01-24T13:58:23.212967+0000","last_peered":"2024-01-24T13:58:23.212967+0000","last_clean":"2024-01-24T13:58:23.212967+0000","last_became_active":"2024-01-24T13:58:19.210211+0000","last_became_peered":"2024-01-24T13:58:19.210211+0000","last_unstale":"2024-01-24T13:58:23.212967+0000","last_undegraded":"2024-01-24T13:58:23.212967+0000","last_fullsized":"2024-01-24T13:58:23.212967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229446+0000","last_change":"2024-01-24T13:58:23.223133+0000","last_active":"2024-01-24T13:58:23.229446+0000","last_peered":"2024-01-24T13:58:23.229446+0000","last_clean":"2024-01-24T13:58:23.229446+0000","last_became_active":"2024-01-24T13:58:23.222741+0000","last_became_peered":"2024-01-24T13:58:23.222741+0000","last_unstale":"2024-01-24T13:58:23.229446+0000","last_undegraded":"2024-01-24T13:58:23.229446+0000","last_fullsized":"2024-01-24T13:58:23.229446+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.210236+0000","last_change":"2024-01-24T13:58:18.800359+0000","last_active":"2024-01-24T13:58:23.210236+0000","last_peered":"2024-01-24T13:58:23.210236+0000","last_clean":"2024-01-24T13:58:23.210236+0000","last_became_active":"2024-01-24T13:58:18.799774+0000","last_became_peered":"2024-01-24T13:58:18.799774+0000","last_unstale":"2024-01-24T13:58:23.210236+0000","last_undegraded":"2024-01-24T13:58:23.210236+0000","last_fullsized":"2024-01-24T13:58:23.210236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237579+0000","last_change":"2024-01-24T13:58:19.210774+0000","last_active":"2024-01-24T13:58:23.237579+0000","last_peered":"2024-01-24T13:58:23.237579+0000","last_clean":"2024-01-24T13:58:23.237579+0000","last_became_active":"2024-01-24T13:58:19.210508+0000","last_became_peered":"2024-01-24T13:58:19.210508+0000","last_unstale":"2024-01-24T13:58:23.237579+0000","last_undegraded":"2024-01-24T13:58:23.237579+0000","last_fullsized":"2024-01-24T13:58:23.237579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209129+0000","last_change":"2024-01-24T13:58:18.803254+0000","last_active":"2024-01-24T13:58:23.209129+0000","last_peered":"2024-01-24T13:58:23.209129+0000","last_clean":"2024-01-24T13:58:23.209129+0000","last_became_active":"2024-01-24T13:58:18.802965+0000","last_became_peered":"2024-01-24T13:58:18.802965+0000","last_unstale":"2024-01-24T13:58:23.209129+0000","last_undegraded":"2024-01-24T13:58:23.209129+0000","last_fullsized":"2024-01-24T13:58:23.209129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208494+0000","last_change":"2024-01-24T13:58:19.221404+0000","last_active":"2024-01-24T13:58:23.208494+0000","last_peered":"2024-01-24T13:58:23.208494+0000","last_clean":"2024-01-24T13:58:23.208494+0000","last_became_active":"2024-01-24T13:58:19.221024+0000","last_became_peered":"2024-01-24T13:58:19.221024+0000","last_unstale":"2024-01-24T13:58:23.208494+0000","last_undegraded":"2024-01-24T13:58:23.208494+0000","last_fullsized":"2024-01-24T13:58:23.208494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232448+0000","last_change":"2024-01-24T13:58:23.229756+0000","last_active":"2024-01-24T13:58:23.232448+0000","last_peered":"2024-01-24T13:58:23.232448+0000","last_clean":"2024-01-24T13:58:23.232448+0000","last_became_active":"2024-01-24T13:58:23.229328+0000","last_became_peered":"2024-01-24T13:58:23.229328+0000","last_unstale":"2024-01-24T13:58:23.232448+0000","last_undegraded":"2024-01-24T13:58:23.232448+0000","last_fullsized":"2024-01-24T13:58:23.232448+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297364,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074476,"statfs":{"total":96636764160,"available":96332263424,"internally_reserved":0,"allocated":274432,"data_stored":115668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297396,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074444,"statfs":{"total":96636764160,"available":96332230656,"internally_reserved":0,"allocated":307200,"data_stored":118043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297372,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074468,"statfs":{"total":96636764160,"available":96332255232,"internally_reserved":0,"allocated":282624,"data_stored":116699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297368,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074472,"statfs":{"total":96636764160,"available":96332259328,"internally_reserved":0,"allocated":278528,"data_stored":116629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297372,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074468,"statfs":{"total":96636764160,"available":96332255232,"internally_reserved":0,"allocated":282624,"data_stored":116719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297388,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074452,"statfs":{"total":96636764160,"available":96332238848,"internally_reserved":0,"allocated":299008,"data_stored":120400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297368,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074472,"statfs":{"total":96636764160,"available":96332259328,"internally_reserved":0,"allocated":278528,"data_stored":116629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297376,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074464,"statfs":{"total":96636764160,"available":96332251136,"internally_reserved":0,"allocated":286720,"data_stored":120254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":3625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T13:58:32.348 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T13:58:32.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.515+0000 7ff132a6d700 1 Processor -- start 2024-01-24T13:58:32.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.515+0000 7ff132a6d700 1 -- start start 2024-01-24T13:58:32.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.516+0000 7ff132a6d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c1543c0 0x7ff12c0bcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.516+0000 7ff132a6d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12c05e850 con 0x7ff12c153fe0 2024-01-24T13:58:32.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.516+0000 7ff132a6d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12c0bd5b0 con 0x7ff12c0bd120 2024-01-24T13:58:32.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.516+0000 7ff132a6d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c0a4f90 con 0x7ff12c1543c0 2024-01-24T13:58:32.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.516+0000 7ff130809700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff12c153fe0 0x7ff12c0a4880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33716/0 (socket says 172.21.15.29:33716) 2024-01-24T13:58:32.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.516+0000 7ff130809700 1 -- 172.21.15.29:0/427047847 learned_addr learned my addr 172.21.15.29:0/427047847 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:32.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.516+0000 7ff12bfff700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c1543c0 0x7ff12c0bcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.516+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806633043 0 0) 0x7ff12c05e850 con 0x7ff12c153fe0 2024-01-24T13:58:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.516+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff11c0033f0 con 0x7ff12c153fe0 2024-01-24T13:58:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 472470666 0 0) 0x7ff11c0033f0 con 0x7ff12c153fe0 2024-01-24T13:58:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c1543c0 msgr2=0x7ff12c0bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c1543c0 0x7ff12c0bcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 >> v1:172.21.15.163:6789/0 conn(0x7ff12c0bd120 legacy=0x7ff12c0c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c0631f0 con 0x7ff12c153fe0 2024-01-24T13:58:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7ff120001470 con 0x7ff12c153fe0 2024-01-24T13:58:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 >> v1:172.21.15.29:6789/0 conn(0x7ff12c153fe0 legacy=0x7ff12c0a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c1543c0 0x7ff11c003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff11c003e40 0x7ff11c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff11c004b80 0x7ff11c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c0a4f90 con 0x7ff11c004b80 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c00b990 con 0x7ff11c003e40 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c00bb50 con 0x7ff12c1543c0 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff130809700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c1543c0 0x7ff11c003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff13100a700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff11c004b80 0x7ff11c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12bfff700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff11c003e40 0x7ff11c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff130809700 1 -- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff11c003e40 msgr2=0x7ff11c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff130809700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff11c003e40 0x7ff11c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff130809700 1 -- 172.21.15.29:0/427047847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff11c004b80 msgr2=0x7ff11c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff130809700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff11c004b80 0x7ff11c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff130809700 1 -- 172.21.15.29:0/427047847 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c0033f0 con 0x7ff12c1543c0 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff13100a700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff11c004b80 0x7ff11c005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff12bfff700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff11c003e40 0x7ff11c004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:32.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.517+0000 7ff130809700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c1543c0 0x7ff11c003910 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ff12000c210 tx=0x7ff12c063160).ready entity=mon.2 client_cookie=3e13542bd5a99bb1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.518+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12000cb40 con 0x7ff12c1543c0 2024-01-24T13:58:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.518+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12000cca0 con 0x7ff12c1543c0 2024-01-24T13:58:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.518+0000 7ff12b7fe700 1 -- 172.21.15.29:0/427047847 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120007320 con 0x7ff12c1543c0 2024-01-24T13:58:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.518+0000 7ff132a6d700 1 -- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c1543c0 msgr2=0x7ff11c003910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.518+0000 7ff132a6d700 1 --2- 172.21.15.29:0/427047847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c1543c0 0x7ff11c003910 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ff12000c210 tx=0x7ff12c063160).stop 2024-01-24T13:58:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.518+0000 7ff13100a700 1 -- 172.21.15.29:0/427047847 reap_dead start 2024-01-24T13:58:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.518+0000 7ff132a6d700 1 -- 172.21.15.29:0/427047847 shutdown_connections 2024-01-24T13:58:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.518+0000 7ff132a6d700 1 -- 172.21.15.29:0/427047847 >> 172.21.15.29:0/427047847 conn(0x7ff12c064aa0 msgr2=0x7ff12c064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.518+0000 7ff132a6d700 1 -- 172.21.15.29:0/427047847 shutdown_connections 2024-01-24T13:58:32.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.518+0000 7ff132a6d700 1 -- 172.21.15.29:0/427047847 wait complete. 2024-01-24T13:58:32.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.519+0000 7ff132a6d700 1 Processor -- start 2024-01-24T13:58:32.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.520+0000 7ff132a6d700 1 -- start start 2024-01-24T13:58:32.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.520+0000 7ff132a6d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff12c1543c0 0x7ff12c150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.520+0000 7ff132a6d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12c154960 0x7ff12c154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.520+0000 7ff132a6d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c0c1850 0x7ff12c0c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.520+0000 7ff132a6d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c0a6c10 con 0x7ff12c154960 2024-01-24T13:58:32.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.520+0000 7ff132a6d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c156030 con 0x7ff12c1543c0 2024-01-24T13:58:32.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.520+0000 7ff132a6d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c0c21c0 con 0x7ff12c0c1850 2024-01-24T13:58:32.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff12bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12c154960 0x7ff12c154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff12bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12c154960 0x7ff12c154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51406/0 (socket says 172.21.15.29:51406) 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff12bfff700 1 -- 172.21.15.29:0/2090034944 learned_addr learned my addr 172.21.15.29:0/2090034944 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff130809700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff12c1543c0 0x7ff12c150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff13100a700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c0c1850 0x7ff12c0c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff12bfff700 1 -- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c0c1850 msgr2=0x7ff12c0c1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff12bfff700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c0c1850 0x7ff12c0c1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff12bfff700 1 -- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff12c1543c0 msgr2=0x7ff12c150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff12bfff700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff12c1543c0 0x7ff12c150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff12bfff700 1 -- 172.21.15.29:0/2090034944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c150700 con 0x7ff12c154960 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff12bfff700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12c154960 0x7ff12c154de0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7ff118011990 tx=0x7ff118008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff128ff9700 1 -- 172.21.15.29:0/2090034944 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11800f6d0 con 0x7ff12c154960 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff128ff9700 1 -- 172.21.15.29:0/2090034944 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11800f830 con 0x7ff12c154960 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff132a6d700 1 -- 172.21.15.29:0/2090034944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c155420 con 0x7ff12c154960 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff128ff9700 1 -- 172.21.15.29:0/2090034944 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11800fbb0 con 0x7ff12c154960 2024-01-24T13:58:32.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.521+0000 7ff132a6d700 1 -- 172.21.15.29:0/2090034944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c1559e0 con 0x7ff12c154960 2024-01-24T13:58:32.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.522+0000 7ff132a6d700 1 -- 172.21.15.29:0/2090034944 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c1561e0 con 0x7ff12c154960 2024-01-24T13:58:32.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.523+0000 7ff128ff9700 1 -- 172.21.15.29:0/2090034944 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff118015480 con 0x7ff12c154960 2024-01-24T13:58:32.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.523+0000 7ff128ff9700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff11005f150 0x7ff110061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.523+0000 7ff128ff9700 1 -- 172.21.15.29:0/2090034944 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7ff11807ad30 con 0x7ff12c154960 2024-01-24T13:58:32.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.523+0000 7ff130809700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff11005f150 0x7ff110061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.524+0000 7ff130809700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff11005f150 0x7ff110061610 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7ff1200078f0 tx=0x7ff120013000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:32.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.528+0000 7ff128ff9700 1 -- 172.21.15.29:0/2090034944 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff12c1561e0 con 0x7ff12c154960 2024-01-24T13:58:32.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.704+0000 7ff132a6d700 1 -- 172.21.15.29:0/2090034944 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff12c1561e0 con 0x7ff11005f150 2024-01-24T13:58:32.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.716+0000 7ff128ff9700 1 -- 172.21.15.29:0/2090034944 <== mgr.4103 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+410603 (secure 0 0 0) 0x7ff12c1561e0 con 0x7ff11005f150 2024-01-24T13:58:32.721 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:32.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.720+0000 7ff132a6d700 1 -- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff11005f150 msgr2=0x7ff110061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.720+0000 7ff132a6d700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff11005f150 0x7ff110061610 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7ff1200078f0 tx=0x7ff120013000).stop 2024-01-24T13:58:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.720+0000 7ff132a6d700 1 -- 172.21.15.29:0/2090034944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12c154960 msgr2=0x7ff12c154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.720+0000 7ff132a6d700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12c154960 0x7ff12c154de0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7ff118011990 tx=0x7ff118008040).stop 2024-01-24T13:58:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.720+0000 7ff132a6d700 1 -- 172.21.15.29:0/2090034944 shutdown_connections 2024-01-24T13:58:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.720+0000 7ff132a6d700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff12c0c1850 0x7ff12c0c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.720+0000 7ff132a6d700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff12c1543c0 0x7ff12c150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.720+0000 7ff132a6d700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff11005f150 0x7ff110061610 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.720+0000 7ff132a6d700 1 --2- 172.21.15.29:0/2090034944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff12c154960 0x7ff12c154de0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.720+0000 7ff132a6d700 1 -- 172.21.15.29:0/2090034944 >> 172.21.15.29:0/2090034944 conn(0x7ff12c064aa0 msgr2=0x7ff12c150f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.720+0000 7ff132a6d700 1 -- 172.21.15.29:0/2090034944 shutdown_connections 2024-01-24T13:58:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.720+0000 7ff132a6d700 1 -- 172.21.15.29:0/2090034944 wait complete. 2024-01-24T13:58:32.725 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T13:58:32.734 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-01-24T13:58:31.726139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":402,"acting":402,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":402,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379004,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752595716,"statfs":{"total":773094113280,"available":770658013184,"internally_reserved":0,"allocated":2289664,"data_stored":941041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2179,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.519916"},"pg_stats":[{"pgid":"3.3f","version":"21'3","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.218432+0000","last_change":"2024-01-24T13:58:18.804112+0000","last_active":"2024-01-24T13:58:23.218432+0000","last_peered":"2024-01-24T13:58:23.218432+0000","last_clean":"2024-01-24T13:58:23.218432+0000","last_became_active":"2024-01-24T13:58:18.803739+0000","last_became_peered":"2024-01-24T13:58:18.803739+0000","last_unstale":"2024-01-24T13:58:23.218432+0000","last_undegraded":"2024-01-24T13:58:23.218432+0000","last_fullsized":"2024-01-24T13:58:23.218432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224809+0000","last_change":"2024-01-24T13:58:19.214526+0000","last_active":"2024-01-24T13:58:23.224809+0000","last_peered":"2024-01-24T13:58:23.224809+0000","last_clean":"2024-01-24T13:58:23.224809+0000","last_became_active":"2024-01-24T13:58:19.214261+0000","last_became_peered":"2024-01-24T13:58:19.214261+0000","last_unstale":"2024-01-24T13:58:23.224809+0000","last_undegraded":"2024-01-24T13:58:23.224809+0000","last_fullsized":"2024-01-24T13:58:23.224809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211257+0000","last_change":"2024-01-24T13:58:18.813731+0000","last_active":"2024-01-24T13:58:23.211257+0000","last_peered":"2024-01-24T13:58:23.211257+0000","last_clean":"2024-01-24T13:58:23.211257+0000","last_became_active":"2024-01-24T13:58:18.813399+0000","last_became_peered":"2024-01-24T13:58:18.813399+0000","last_unstale":"2024-01-24T13:58:23.211257+0000","last_undegraded":"2024-01-24T13:58:23.211257+0000","last_fullsized":"2024-01-24T13:58:23.211257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216628+0000","last_change":"2024-01-24T13:58:19.214934+0000","last_active":"2024-01-24T13:58:23.216628+0000","last_peered":"2024-01-24T13:58:23.216628+0000","last_clean":"2024-01-24T13:58:23.216628+0000","last_became_active":"2024-01-24T13:58:19.214473+0000","last_became_peered":"2024-01-24T13:58:19.214473+0000","last_unstale":"2024-01-24T13:58:23.216628+0000","last_undegraded":"2024-01-24T13:58:23.216628+0000","last_fullsized":"2024-01-24T13:58:23.216628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235905+0000","last_change":"2024-01-24T13:58:18.810864+0000","last_active":"2024-01-24T13:58:23.235905+0000","last_peered":"2024-01-24T13:58:23.235905+0000","last_clean":"2024-01-24T13:58:23.235905+0000","last_became_active":"2024-01-24T13:58:18.810538+0000","last_became_peered":"2024-01-24T13:58:18.810538+0000","last_unstale":"2024-01-24T13:58:23.235905+0000","last_undegraded":"2024-01-24T13:58:23.235905+0000","last_fullsized":"2024-01-24T13:58:23.235905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209261+0000","last_change":"2024-01-24T13:58:19.215436+0000","last_active":"2024-01-24T13:58:23.209261+0000","last_peered":"2024-01-24T13:58:23.209261+0000","last_clean":"2024-01-24T13:58:23.209261+0000","last_became_active":"2024-01-24T13:58:19.215145+0000","last_became_peered":"2024-01-24T13:58:19.215145+0000","last_unstale":"2024-01-24T13:58:23.209261+0000","last_undegraded":"2024-01-24T13:58:23.209261+0000","last_fullsized":"2024-01-24T13:58:23.209261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227753+0000","last_change":"2024-01-24T13:58:18.803776+0000","last_active":"2024-01-24T13:58:23.227753+0000","last_peered":"2024-01-24T13:58:23.227753+0000","last_clean":"2024-01-24T13:58:23.227753+0000","last_became_active":"2024-01-24T13:58:18.803485+0000","last_became_peered":"2024-01-24T13:58:18.803485+0000","last_unstale":"2024-01-24T13:58:23.227753+0000","last_undegraded":"2024-01-24T13:58:23.227753+0000","last_fullsized":"2024-01-24T13:58:23.227753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.234859+0000","last_change":"2024-01-24T13:58:19.224975+0000","last_active":"2024-01-24T13:58:23.234859+0000","last_peered":"2024-01-24T13:58:23.234859+0000","last_clean":"2024-01-24T13:58:23.234859+0000","last_became_active":"2024-01-24T13:58:19.224568+0000","last_became_peered":"2024-01-24T13:58:19.224568+0000","last_unstale":"2024-01-24T13:58:23.234859+0000","last_undegraded":"2024-01-24T13:58:23.234859+0000","last_fullsized":"2024-01-24T13:58:23.234859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217357+0000","last_change":"2024-01-24T13:58:18.797963+0000","last_active":"2024-01-24T13:58:23.217357+0000","last_peered":"2024-01-24T13:58:23.217357+0000","last_clean":"2024-01-24T13:58:23.217357+0000","last_became_active":"2024-01-24T13:58:18.797320+0000","last_became_peered":"2024-01-24T13:58:18.797320+0000","last_unstale":"2024-01-24T13:58:23.217357+0000","last_undegraded":"2024-01-24T13:58:23.217357+0000","last_fullsized":"2024-01-24T13:58:23.217357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213193+0000","last_change":"2024-01-24T13:58:19.213147+0000","last_active":"2024-01-24T13:58:23.213193+0000","last_peered":"2024-01-24T13:58:23.213193+0000","last_clean":"2024-01-24T13:58:23.213193+0000","last_became_active":"2024-01-24T13:58:19.212854+0000","last_became_peered":"2024-01-24T13:58:19.212854+0000","last_unstale":"2024-01-24T13:58:23.213193+0000","last_undegraded":"2024-01-24T13:58:23.213193+0000","last_fullsized":"2024-01-24T13:58:23.213193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.228193+0000","last_change":"2024-01-24T13:58:18.812721+0000","last_active":"2024-01-24T13:58:23.228193+0000","last_peered":"2024-01-24T13:58:23.228193+0000","last_clean":"2024-01-24T13:58:23.228193+0000","last_became_active":"2024-01-24T13:58:18.812439+0000","last_became_peered":"2024-01-24T13:58:18.812439+0000","last_unstale":"2024-01-24T13:58:23.228193+0000","last_undegraded":"2024-01-24T13:58:23.228193+0000","last_fullsized":"2024-01-24T13:58:23.228193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209659+0000","last_change":"2024-01-24T13:58:19.213672+0000","last_active":"2024-01-24T13:58:23.209659+0000","last_peered":"2024-01-24T13:58:23.209659+0000","last_clean":"2024-01-24T13:58:23.209659+0000","last_became_active":"2024-01-24T13:58:19.213378+0000","last_became_peered":"2024-01-24T13:58:19.213378+0000","last_unstale":"2024-01-24T13:58:23.209659+0000","last_undegraded":"2024-01-24T13:58:23.209659+0000","last_fullsized":"2024-01-24T13:58:23.209659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216204+0000","last_change":"2024-01-24T13:58:18.806273+0000","last_active":"2024-01-24T13:58:23.216204+0000","last_peered":"2024-01-24T13:58:23.216204+0000","last_clean":"2024-01-24T13:58:23.216204+0000","last_became_active":"2024-01-24T13:58:18.805970+0000","last_became_peered":"2024-01-24T13:58:18.805970+0000","last_unstale":"2024-01-24T13:58:23.216204+0000","last_undegraded":"2024-01-24T13:58:23.216204+0000","last_fullsized":"2024-01-24T13:58:23.216204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227834+0000","last_change":"2024-01-24T13:58:19.208876+0000","last_active":"2024-01-24T13:58:23.227834+0000","last_peered":"2024-01-24T13:58:23.227834+0000","last_clean":"2024-01-24T13:58:23.227834+0000","last_became_active":"2024-01-24T13:58:19.208537+0000","last_became_peered":"2024-01-24T13:58:19.208537+0000","last_unstale":"2024-01-24T13:58:23.227834+0000","last_undegraded":"2024-01-24T13:58:23.227834+0000","last_fullsized":"2024-01-24T13:58:23.227834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208114+0000","last_change":"2024-01-24T13:58:18.801957+0000","last_active":"2024-01-24T13:58:23.208114+0000","last_peered":"2024-01-24T13:58:23.208114+0000","last_clean":"2024-01-24T13:58:23.208114+0000","last_became_active":"2024-01-24T13:58:18.801641+0000","last_became_peered":"2024-01-24T13:58:18.801641+0000","last_unstale":"2024-01-24T13:58:23.208114+0000","last_undegraded":"2024-01-24T13:58:23.208114+0000","last_fullsized":"2024-01-24T13:58:23.208114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.222487+0000","last_change":"2024-01-24T13:58:19.217744+0000","last_active":"2024-01-24T13:58:23.222487+0000","last_peered":"2024-01-24T13:58:23.222487+0000","last_clean":"2024-01-24T13:58:23.222487+0000","last_became_active":"2024-01-24T13:58:19.217212+0000","last_became_peered":"2024-01-24T13:58:19.217212+0000","last_unstale":"2024-01-24T13:58:23.222487+0000","last_undegraded":"2024-01-24T13:58:23.222487+0000","last_fullsized":"2024-01-24T13:58:23.222487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.218321+0000","last_change":"2024-01-24T13:58:18.797573+0000","last_active":"2024-01-24T13:58:23.218321+0000","last_peered":"2024-01-24T13:58:23.218321+0000","last_clean":"2024-01-24T13:58:23.218321+0000","last_became_active":"2024-01-24T13:58:18.797100+0000","last_became_peered":"2024-01-24T13:58:18.797100+0000","last_unstale":"2024-01-24T13:58:23.218321+0000","last_undegraded":"2024-01-24T13:58:23.218321+0000","last_fullsized":"2024-01-24T13:58:23.218321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209573+0000","last_change":"2024-01-24T13:58:19.211019+0000","last_active":"2024-01-24T13:58:23.209573+0000","last_peered":"2024-01-24T13:58:23.209573+0000","last_clean":"2024-01-24T13:58:23.209573+0000","last_became_active":"2024-01-24T13:58:19.210424+0000","last_became_peered":"2024-01-24T13:58:19.210424+0000","last_unstale":"2024-01-24T13:58:23.209573+0000","last_undegraded":"2024-01-24T13:58:23.209573+0000","last_fullsized":"2024-01-24T13:58:23.209573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213015+0000","last_change":"2024-01-24T13:58:18.800935+0000","last_active":"2024-01-24T13:58:23.213015+0000","last_peered":"2024-01-24T13:58:23.213015+0000","last_clean":"2024-01-24T13:58:23.213015+0000","last_became_active":"2024-01-24T13:58:18.800629+0000","last_became_peered":"2024-01-24T13:58:18.800629+0000","last_unstale":"2024-01-24T13:58:23.213015+0000","last_undegraded":"2024-01-24T13:58:23.213015+0000","last_fullsized":"2024-01-24T13:58:23.213015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216389+0000","last_change":"2024-01-24T13:58:19.214756+0000","last_active":"2024-01-24T13:58:23.216389+0000","last_peered":"2024-01-24T13:58:23.216389+0000","last_clean":"2024-01-24T13:58:23.216389+0000","last_became_active":"2024-01-24T13:58:19.214127+0000","last_became_peered":"2024-01-24T13:58:19.214127+0000","last_unstale":"2024-01-24T13:58:23.216389+0000","last_undegraded":"2024-01-24T13:58:23.216389+0000","last_fullsized":"2024-01-24T13:58:23.216389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213067+0000","last_change":"2024-01-24T13:58:18.804647+0000","last_active":"2024-01-24T13:58:23.213067+0000","last_peered":"2024-01-24T13:58:23.213067+0000","last_clean":"2024-01-24T13:58:23.213067+0000","last_became_active":"2024-01-24T13:58:18.803996+0000","last_became_peered":"2024-01-24T13:58:18.803996+0000","last_unstale":"2024-01-24T13:58:23.213067+0000","last_undegraded":"2024-01-24T13:58:23.213067+0000","last_fullsized":"2024-01-24T13:58:23.213067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226664+0000","last_change":"2024-01-24T13:58:19.210603+0000","last_active":"2024-01-24T13:58:23.226664+0000","last_peered":"2024-01-24T13:58:23.226664+0000","last_clean":"2024-01-24T13:58:23.226664+0000","last_became_active":"2024-01-24T13:58:19.210245+0000","last_became_peered":"2024-01-24T13:58:19.210245+0000","last_unstale":"2024-01-24T13:58:23.226664+0000","last_undegraded":"2024-01-24T13:58:23.226664+0000","last_fullsized":"2024-01-24T13:58:23.226664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.34","version":"26'7","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-24T13:58:28.635902+0000","last_change":"2024-01-24T13:58:18.809410+0000","last_active":"2024-01-24T13:58:28.635902+0000","last_peered":"2024-01-24T13:58:28.635902+0000","last_clean":"2024-01-24T13:58:28.635902+0000","last_became_active":"2024-01-24T13:58:18.809115+0000","last_became_peered":"2024-01-24T13:58:18.809115+0000","last_unstale":"2024-01-24T13:58:28.635902+0000","last_undegraded":"2024-01-24T13:58:28.635902+0000","last_fullsized":"2024-01-24T13:58:28.635902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3071,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220937+0000","last_change":"2024-01-24T13:58:19.217910+0000","last_active":"2024-01-24T13:58:23.220937+0000","last_peered":"2024-01-24T13:58:23.220937+0000","last_clean":"2024-01-24T13:58:23.220937+0000","last_became_active":"2024-01-24T13:58:19.216908+0000","last_became_peered":"2024-01-24T13:58:19.216908+0000","last_unstale":"2024-01-24T13:58:23.220937+0000","last_undegraded":"2024-01-24T13:58:23.220937+0000","last_fullsized":"2024-01-24T13:58:23.220937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208119+0000","last_change":"2024-01-24T13:58:18.807547+0000","last_active":"2024-01-24T13:58:23.208119+0000","last_peered":"2024-01-24T13:58:23.208119+0000","last_clean":"2024-01-24T13:58:23.208119+0000","last_became_active":"2024-01-24T13:58:18.806845+0000","last_became_peered":"2024-01-24T13:58:18.806845+0000","last_unstale":"2024-01-24T13:58:23.208119+0000","last_undegraded":"2024-01-24T13:58:23.208119+0000","last_fullsized":"2024-01-24T13:58:23.208119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224004+0000","last_change":"2024-01-24T13:58:19.219483+0000","last_active":"2024-01-24T13:58:23.224004+0000","last_peered":"2024-01-24T13:58:23.224004+0000","last_clean":"2024-01-24T13:58:23.224004+0000","last_became_active":"2024-01-24T13:58:19.219182+0000","last_became_peered":"2024-01-24T13:58:19.219182+0000","last_unstale":"2024-01-24T13:58:23.224004+0000","last_undegraded":"2024-01-24T13:58:23.224004+0000","last_fullsized":"2024-01-24T13:58:23.224004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232615+0000","last_change":"2024-01-24T13:58:18.804144+0000","last_active":"2024-01-24T13:58:23.232615+0000","last_peered":"2024-01-24T13:58:23.232615+0000","last_clean":"2024-01-24T13:58:23.232615+0000","last_became_active":"2024-01-24T13:58:18.803873+0000","last_became_peered":"2024-01-24T13:58:18.803873+0000","last_unstale":"2024-01-24T13:58:23.232615+0000","last_undegraded":"2024-01-24T13:58:23.232615+0000","last_fullsized":"2024-01-24T13:58:23.232615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209346+0000","last_change":"2024-01-24T13:58:19.221162+0000","last_active":"2024-01-24T13:58:23.209346+0000","last_peered":"2024-01-24T13:58:23.209346+0000","last_clean":"2024-01-24T13:58:23.209346+0000","last_became_active":"2024-01-24T13:58:19.220665+0000","last_became_peered":"2024-01-24T13:58:19.220665+0000","last_unstale":"2024-01-24T13:58:23.209346+0000","last_undegraded":"2024-01-24T13:58:23.209346+0000","last_fullsized":"2024-01-24T13:58:23.209346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.210124+0000","last_change":"2024-01-24T13:58:18.799003+0000","last_active":"2024-01-24T13:58:23.210124+0000","last_peered":"2024-01-24T13:58:23.210124+0000","last_clean":"2024-01-24T13:58:23.210124+0000","last_became_active":"2024-01-24T13:58:18.796638+0000","last_became_peered":"2024-01-24T13:58:18.796638+0000","last_unstale":"2024-01-24T13:58:23.210124+0000","last_undegraded":"2024-01-24T13:58:23.210124+0000","last_fullsized":"2024-01-24T13:58:23.210124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227756+0000","last_change":"2024-01-24T13:58:19.213039+0000","last_active":"2024-01-24T13:58:23.227756+0000","last_peered":"2024-01-24T13:58:23.227756+0000","last_clean":"2024-01-24T13:58:23.227756+0000","last_became_active":"2024-01-24T13:58:19.212755+0000","last_became_peered":"2024-01-24T13:58:19.212755+0000","last_unstale":"2024-01-24T13:58:23.227756+0000","last_undegraded":"2024-01-24T13:58:23.227756+0000","last_fullsized":"2024-01-24T13:58:23.227756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"21'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.225127+0000","last_change":"2024-01-24T13:58:18.805645+0000","last_active":"2024-01-24T13:58:23.225127+0000","last_peered":"2024-01-24T13:58:23.225127+0000","last_clean":"2024-01-24T13:58:23.225127+0000","last_became_active":"2024-01-24T13:58:18.805375+0000","last_became_peered":"2024-01-24T13:58:18.805375+0000","last_unstale":"2024-01-24T13:58:23.225127+0000","last_undegraded":"2024-01-24T13:58:23.225127+0000","last_fullsized":"2024-01-24T13:58:23.225127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237611+0000","last_change":"2024-01-24T13:58:19.225874+0000","last_active":"2024-01-24T13:58:23.237611+0000","last_peered":"2024-01-24T13:58:23.237611+0000","last_clean":"2024-01-24T13:58:23.237611+0000","last_became_active":"2024-01-24T13:58:19.225220+0000","last_became_peered":"2024-01-24T13:58:19.225220+0000","last_unstale":"2024-01-24T13:58:23.237611+0000","last_undegraded":"2024-01-24T13:58:23.237611+0000","last_fullsized":"2024-01-24T13:58:23.237611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.233153+0000","last_change":"2024-01-24T13:58:18.797127+0000","last_active":"2024-01-24T13:58:23.233153+0000","last_peered":"2024-01-24T13:58:23.233153+0000","last_clean":"2024-01-24T13:58:23.233153+0000","last_became_active":"2024-01-24T13:58:18.796816+0000","last_became_peered":"2024-01-24T13:58:18.796816+0000","last_unstale":"2024-01-24T13:58:23.233153+0000","last_undegraded":"2024-01-24T13:58:23.233153+0000","last_fullsized":"2024-01-24T13:58:23.233153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216434+0000","last_change":"2024-01-24T13:58:19.219593+0000","last_active":"2024-01-24T13:58:23.216434+0000","last_peered":"2024-01-24T13:58:23.216434+0000","last_clean":"2024-01-24T13:58:23.216434+0000","last_became_active":"2024-01-24T13:58:19.219267+0000","last_became_peered":"2024-01-24T13:58:19.219267+0000","last_unstale":"2024-01-24T13:58:23.216434+0000","last_undegraded":"2024-01-24T13:58:23.216434+0000","last_fullsized":"2024-01-24T13:58:23.216434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220392+0000","last_change":"2024-01-24T13:58:18.805694+0000","last_active":"2024-01-24T13:58:23.220392+0000","last_peered":"2024-01-24T13:58:23.220392+0000","last_clean":"2024-01-24T13:58:23.220392+0000","last_became_active":"2024-01-24T13:58:18.805335+0000","last_became_peered":"2024-01-24T13:58:18.805335+0000","last_unstale":"2024-01-24T13:58:23.220392+0000","last_undegraded":"2024-01-24T13:58:23.220392+0000","last_fullsized":"2024-01-24T13:58:23.220392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207983+0000","last_change":"2024-01-24T13:58:19.213662+0000","last_active":"2024-01-24T13:58:23.207983+0000","last_peered":"2024-01-24T13:58:23.207983+0000","last_clean":"2024-01-24T13:58:23.207983+0000","last_became_active":"2024-01-24T13:58:19.213340+0000","last_became_peered":"2024-01-24T13:58:19.213340+0000","last_unstale":"2024-01-24T13:58:23.207983+0000","last_undegraded":"2024-01-24T13:58:23.207983+0000","last_fullsized":"2024-01-24T13:58:23.207983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2d","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217312+0000","last_change":"2024-01-24T13:58:18.798262+0000","last_active":"2024-01-24T13:58:23.217312+0000","last_peered":"2024-01-24T13:58:23.217312+0000","last_clean":"2024-01-24T13:58:23.217312+0000","last_became_active":"2024-01-24T13:58:18.797856+0000","last_became_peered":"2024-01-24T13:58:18.797856+0000","last_unstale":"2024-01-24T13:58:23.217312+0000","last_undegraded":"2024-01-24T13:58:23.217312+0000","last_fullsized":"2024-01-24T13:58:23.217312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232445+0000","last_change":"2024-01-24T13:58:19.223736+0000","last_active":"2024-01-24T13:58:23.232445+0000","last_peered":"2024-01-24T13:58:23.232445+0000","last_clean":"2024-01-24T13:58:23.232445+0000","last_became_active":"2024-01-24T13:58:19.223500+0000","last_became_peered":"2024-01-24T13:58:19.223500+0000","last_unstale":"2024-01-24T13:58:23.232445+0000","last_undegraded":"2024-01-24T13:58:23.232445+0000","last_fullsized":"2024-01-24T13:58:23.232445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213312+0000","last_change":"2024-01-24T13:58:18.802360+0000","last_active":"2024-01-24T13:58:23.213312+0000","last_peered":"2024-01-24T13:58:23.213312+0000","last_clean":"2024-01-24T13:58:23.213312+0000","last_became_active":"2024-01-24T13:58:18.802051+0000","last_became_peered":"2024-01-24T13:58:18.802051+0000","last_unstale":"2024-01-24T13:58:23.213312+0000","last_undegraded":"2024-01-24T13:58:23.213312+0000","last_fullsized":"2024-01-24T13:58:23.213312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217156+0000","last_change":"2024-01-24T13:58:19.218929+0000","last_active":"2024-01-24T13:58:23.217156+0000","last_peered":"2024-01-24T13:58:23.217156+0000","last_clean":"2024-01-24T13:58:23.217156+0000","last_became_active":"2024-01-24T13:58:19.218350+0000","last_became_peered":"2024-01-24T13:58:19.218350+0000","last_unstale":"2024-01-24T13:58:23.217156+0000","last_undegraded":"2024-01-24T13:58:23.217156+0000","last_fullsized":"2024-01-24T13:58:23.217156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2b","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235138+0000","last_change":"2024-01-24T13:58:18.810840+0000","last_active":"2024-01-24T13:58:23.235138+0000","last_peered":"2024-01-24T13:58:23.235138+0000","last_clean":"2024-01-24T13:58:23.235138+0000","last_became_active":"2024-01-24T13:58:18.809023+0000","last_became_peered":"2024-01-24T13:58:18.809023+0000","last_unstale":"2024-01-24T13:58:23.235138+0000","last_undegraded":"2024-01-24T13:58:23.235138+0000","last_fullsized":"2024-01-24T13:58:23.235138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209489+0000","last_change":"2024-01-24T13:58:19.217855+0000","last_active":"2024-01-24T13:58:23.209489+0000","last_peered":"2024-01-24T13:58:23.209489+0000","last_clean":"2024-01-24T13:58:23.209489+0000","last_became_active":"2024-01-24T13:58:19.217559+0000","last_became_peered":"2024-01-24T13:58:19.217559+0000","last_unstale":"2024-01-24T13:58:23.209489+0000","last_undegraded":"2024-01-24T13:58:23.209489+0000","last_fullsized":"2024-01-24T13:58:23.209489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226871+0000","last_change":"2024-01-24T13:58:18.798611+0000","last_active":"2024-01-24T13:58:23.226871+0000","last_peered":"2024-01-24T13:58:23.226871+0000","last_clean":"2024-01-24T13:58:23.226871+0000","last_became_active":"2024-01-24T13:58:18.798325+0000","last_became_peered":"2024-01-24T13:58:18.798325+0000","last_unstale":"2024-01-24T13:58:23.226871+0000","last_undegraded":"2024-01-24T13:58:23.226871+0000","last_fullsized":"2024-01-24T13:58:23.226871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.222261+0000","last_change":"2024-01-24T13:58:19.206846+0000","last_active":"2024-01-24T13:58:23.222261+0000","last_peered":"2024-01-24T13:58:23.222261+0000","last_clean":"2024-01-24T13:58:23.222261+0000","last_became_active":"2024-01-24T13:58:19.206552+0000","last_became_peered":"2024-01-24T13:58:19.206552+0000","last_unstale":"2024-01-24T13:58:23.222261+0000","last_undegraded":"2024-01-24T13:58:23.222261+0000","last_fullsized":"2024-01-24T13:58:23.222261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207449+0000","last_change":"2024-01-24T13:58:18.801628+0000","last_active":"2024-01-24T13:58:23.207449+0000","last_peered":"2024-01-24T13:58:23.207449+0000","last_clean":"2024-01-24T13:58:23.207449+0000","last_became_active":"2024-01-24T13:58:18.801291+0000","last_became_peered":"2024-01-24T13:58:18.801291+0000","last_unstale":"2024-01-24T13:58:23.207449+0000","last_undegraded":"2024-01-24T13:58:23.207449+0000","last_fullsized":"2024-01-24T13:58:23.207449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209637+0000","last_change":"2024-01-24T13:58:19.211347+0000","last_active":"2024-01-24T13:58:23.209637+0000","last_peered":"2024-01-24T13:58:23.209637+0000","last_clean":"2024-01-24T13:58:23.209637+0000","last_became_active":"2024-01-24T13:58:19.210298+0000","last_became_peered":"2024-01-24T13:58:19.210298+0000","last_unstale":"2024-01-24T13:58:23.209637+0000","last_undegraded":"2024-01-24T13:58:23.209637+0000","last_fullsized":"2024-01-24T13:58:23.209637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224993+0000","last_change":"2024-01-24T13:58:18.803399+0000","last_active":"2024-01-24T13:58:23.224993+0000","last_peered":"2024-01-24T13:58:23.224993+0000","last_clean":"2024-01-24T13:58:23.224993+0000","last_became_active":"2024-01-24T13:58:18.803109+0000","last_became_peered":"2024-01-24T13:58:18.803109+0000","last_unstale":"2024-01-24T13:58:23.224993+0000","last_undegraded":"2024-01-24T13:58:23.224993+0000","last_fullsized":"2024-01-24T13:58:23.224993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238205+0000","last_change":"2024-01-24T13:58:19.225710+0000","last_active":"2024-01-24T13:58:23.238205+0000","last_peered":"2024-01-24T13:58:23.238205+0000","last_clean":"2024-01-24T13:58:23.238205+0000","last_became_active":"2024-01-24T13:58:19.224844+0000","last_became_peered":"2024-01-24T13:58:19.224844+0000","last_unstale":"2024-01-24T13:58:23.238205+0000","last_undegraded":"2024-01-24T13:58:23.238205+0000","last_fullsized":"2024-01-24T13:58:23.238205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208117+0000","last_change":"2024-01-24T13:58:18.801623+0000","last_active":"2024-01-24T13:58:23.208117+0000","last_peered":"2024-01-24T13:58:23.208117+0000","last_clean":"2024-01-24T13:58:23.208117+0000","last_became_active":"2024-01-24T13:58:18.801272+0000","last_became_peered":"2024-01-24T13:58:18.801272+0000","last_unstale":"2024-01-24T13:58:23.208117+0000","last_undegraded":"2024-01-24T13:58:23.208117+0000","last_fullsized":"2024-01-24T13:58:23.208117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212308+0000","last_change":"2024-01-24T13:58:19.207457+0000","last_active":"2024-01-24T13:58:23.212308+0000","last_peered":"2024-01-24T13:58:23.212308+0000","last_clean":"2024-01-24T13:58:23.212308+0000","last_became_active":"2024-01-24T13:58:19.207133+0000","last_became_peered":"2024-01-24T13:58:19.207133+0000","last_unstale":"2024-01-24T13:58:23.212308+0000","last_undegraded":"2024-01-24T13:58:23.212308+0000","last_fullsized":"2024-01-24T13:58:23.212308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"26'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.880144+0000","last_change":"2024-01-24T13:58:18.800410+0000","last_active":"2024-01-24T13:58:23.880144+0000","last_peered":"2024-01-24T13:58:23.880144+0000","last_clean":"2024-01-24T13:58:23.880144+0000","last_became_active":"2024-01-24T13:58:18.800073+0000","last_became_peered":"2024-01-24T13:58:18.800073+0000","last_unstale":"2024-01-24T13:58:23.880144+0000","last_undegraded":"2024-01-24T13:58:23.880144+0000","last_fullsized":"2024-01-24T13:58:23.880144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207934+0000","last_change":"2024-01-24T13:58:19.221135+0000","last_active":"2024-01-24T13:58:23.207934+0000","last_peered":"2024-01-24T13:58:23.207934+0000","last_clean":"2024-01-24T13:58:23.207934+0000","last_became_active":"2024-01-24T13:58:19.220863+0000","last_became_peered":"2024-01-24T13:58:19.220863+0000","last_unstale":"2024-01-24T13:58:23.207934+0000","last_undegraded":"2024-01-24T13:58:23.207934+0000","last_fullsized":"2024-01-24T13:58:23.207934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211332+0000","last_change":"2024-01-24T13:58:18.800232+0000","last_active":"2024-01-24T13:58:23.211332+0000","last_peered":"2024-01-24T13:58:23.211332+0000","last_clean":"2024-01-24T13:58:23.211332+0000","last_became_active":"2024-01-24T13:58:18.799722+0000","last_became_peered":"2024-01-24T13:58:18.799722+0000","last_unstale":"2024-01-24T13:58:23.211332+0000","last_undegraded":"2024-01-24T13:58:23.211332+0000","last_fullsized":"2024-01-24T13:58:23.211332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209141+0000","last_change":"2024-01-24T13:58:19.215326+0000","last_active":"2024-01-24T13:58:23.209141+0000","last_peered":"2024-01-24T13:58:23.209141+0000","last_clean":"2024-01-24T13:58:23.209141+0000","last_became_active":"2024-01-24T13:58:19.214852+0000","last_became_peered":"2024-01-24T13:58:19.214852+0000","last_unstale":"2024-01-24T13:58:23.209141+0000","last_undegraded":"2024-01-24T13:58:23.209141+0000","last_fullsized":"2024-01-24T13:58:23.209141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207199+0000","last_change":"2024-01-24T13:58:18.804258+0000","last_active":"2024-01-24T13:58:23.207199+0000","last_peered":"2024-01-24T13:58:23.207199+0000","last_clean":"2024-01-24T13:58:23.207199+0000","last_became_active":"2024-01-24T13:58:18.802440+0000","last_became_peered":"2024-01-24T13:58:18.802440+0000","last_unstale":"2024-01-24T13:58:23.207199+0000","last_undegraded":"2024-01-24T13:58:23.207199+0000","last_fullsized":"2024-01-24T13:58:23.207199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208000+0000","last_change":"2024-01-24T13:58:19.214888+0000","last_active":"2024-01-24T13:58:23.208000+0000","last_peered":"2024-01-24T13:58:23.208000+0000","last_clean":"2024-01-24T13:58:23.208000+0000","last_became_active":"2024-01-24T13:58:19.214607+0000","last_became_peered":"2024-01-24T13:58:19.214607+0000","last_unstale":"2024-01-24T13:58:23.208000+0000","last_undegraded":"2024-01-24T13:58:23.208000+0000","last_fullsized":"2024-01-24T13:58:23.208000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235037+0000","last_change":"2024-01-24T13:58:18.813467+0000","last_active":"2024-01-24T13:58:23.235037+0000","last_peered":"2024-01-24T13:58:23.235037+0000","last_clean":"2024-01-24T13:58:23.235037+0000","last_became_active":"2024-01-24T13:58:18.813164+0000","last_became_peered":"2024-01-24T13:58:18.813164+0000","last_unstale":"2024-01-24T13:58:23.235037+0000","last_undegraded":"2024-01-24T13:58:23.235037+0000","last_fullsized":"2024-01-24T13:58:23.235037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211738+0000","last_change":"2024-01-24T13:58:19.212196+0000","last_active":"2024-01-24T13:58:23.211738+0000","last_peered":"2024-01-24T13:58:23.211738+0000","last_clean":"2024-01-24T13:58:23.211738+0000","last_became_active":"2024-01-24T13:58:19.211820+0000","last_became_peered":"2024-01-24T13:58:19.211820+0000","last_unstale":"2024-01-24T13:58:23.211738+0000","last_undegraded":"2024-01-24T13:58:23.211738+0000","last_fullsized":"2024-01-24T13:58:23.211738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226611+0000","last_change":"2024-01-24T13:58:18.802179+0000","last_active":"2024-01-24T13:58:23.226611+0000","last_peered":"2024-01-24T13:58:23.226611+0000","last_clean":"2024-01-24T13:58:23.226611+0000","last_became_active":"2024-01-24T13:58:18.801891+0000","last_became_peered":"2024-01-24T13:58:18.801891+0000","last_unstale":"2024-01-24T13:58:23.226611+0000","last_undegraded":"2024-01-24T13:58:23.226611+0000","last_fullsized":"2024-01-24T13:58:23.226611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227791+0000","last_change":"2024-01-24T13:58:19.217207+0000","last_active":"2024-01-24T13:58:23.227791+0000","last_peered":"2024-01-24T13:58:23.227791+0000","last_clean":"2024-01-24T13:58:23.227791+0000","last_became_active":"2024-01-24T13:58:19.216920+0000","last_became_peered":"2024-01-24T13:58:19.216920+0000","last_unstale":"2024-01-24T13:58:23.227791+0000","last_undegraded":"2024-01-24T13:58:23.227791+0000","last_fullsized":"2024-01-24T13:58:23.227791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220454+0000","last_change":"2024-01-24T13:58:18.793887+0000","last_active":"2024-01-24T13:58:23.220454+0000","last_peered":"2024-01-24T13:58:23.220454+0000","last_clean":"2024-01-24T13:58:23.220454+0000","last_became_active":"2024-01-24T13:58:18.793542+0000","last_became_peered":"2024-01-24T13:58:18.793542+0000","last_unstale":"2024-01-24T13:58:23.220454+0000","last_undegraded":"2024-01-24T13:58:23.220454+0000","last_fullsized":"2024-01-24T13:58:23.220454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211617+0000","last_change":"2024-01-24T13:58:19.216131+0000","last_active":"2024-01-24T13:58:23.211617+0000","last_peered":"2024-01-24T13:58:23.211617+0000","last_clean":"2024-01-24T13:58:23.211617+0000","last_became_active":"2024-01-24T13:58:19.215566+0000","last_became_peered":"2024-01-24T13:58:19.215566+0000","last_unstale":"2024-01-24T13:58:23.211617+0000","last_undegraded":"2024-01-24T13:58:23.211617+0000","last_fullsized":"2024-01-24T13:58:23.211617+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212581+0000","last_change":"2024-01-24T13:58:18.808952+0000","last_active":"2024-01-24T13:58:23.212581+0000","last_peered":"2024-01-24T13:58:23.212581+0000","last_clean":"2024-01-24T13:58:23.212581+0000","last_became_active":"2024-01-24T13:58:18.808672+0000","last_became_peered":"2024-01-24T13:58:18.808672+0000","last_unstale":"2024-01-24T13:58:23.212581+0000","last_undegraded":"2024-01-24T13:58:23.212581+0000","last_fullsized":"2024-01-24T13:58:23.212581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211211+0000","last_change":"2024-01-24T13:58:19.213798+0000","last_active":"2024-01-24T13:58:23.211211+0000","last_peered":"2024-01-24T13:58:23.211211+0000","last_clean":"2024-01-24T13:58:23.211211+0000","last_became_active":"2024-01-24T13:58:19.212458+0000","last_became_peered":"2024-01-24T13:58:19.212458+0000","last_unstale":"2024-01-24T13:58:23.211211+0000","last_undegraded":"2024-01-24T13:58:23.211211+0000","last_fullsized":"2024-01-24T13:58:23.211211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238030+0000","last_change":"2024-01-24T13:58:18.811471+0000","last_active":"2024-01-24T13:58:23.238030+0000","last_peered":"2024-01-24T13:58:23.238030+0000","last_clean":"2024-01-24T13:58:23.238030+0000","last_became_active":"2024-01-24T13:58:18.811159+0000","last_became_peered":"2024-01-24T13:58:18.811159+0000","last_unstale":"2024-01-24T13:58:23.238030+0000","last_undegraded":"2024-01-24T13:58:23.238030+0000","last_fullsized":"2024-01-24T13:58:23.238030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209454+0000","last_change":"2024-01-24T13:58:19.219125+0000","last_active":"2024-01-24T13:58:23.209454+0000","last_peered":"2024-01-24T13:58:23.209454+0000","last_clean":"2024-01-24T13:58:23.209454+0000","last_became_active":"2024-01-24T13:58:19.218823+0000","last_became_peered":"2024-01-24T13:58:19.218823+0000","last_unstale":"2024-01-24T13:58:23.209454+0000","last_undegraded":"2024-01-24T13:58:23.209454+0000","last_fullsized":"2024-01-24T13:58:23.209454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241886+0000","last_change":"2024-01-24T13:58:23.236176+0000","last_active":"2024-01-24T13:58:23.241886+0000","last_peered":"2024-01-24T13:58:23.241886+0000","last_clean":"2024-01-24T13:58:23.241886+0000","last_became_active":"2024-01-24T13:58:23.235658+0000","last_became_peered":"2024-01-24T13:58:23.235658+0000","last_unstale":"2024-01-24T13:58:23.241886+0000","last_undegraded":"2024-01-24T13:58:23.241886+0000","last_fullsized":"2024-01-24T13:58:23.241886+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207471+0000","last_change":"2024-01-24T13:58:18.806459+0000","last_active":"2024-01-24T13:58:23.207471+0000","last_peered":"2024-01-24T13:58:23.207471+0000","last_clean":"2024-01-24T13:58:23.207471+0000","last_became_active":"2024-01-24T13:58:18.806150+0000","last_became_peered":"2024-01-24T13:58:18.806150+0000","last_unstale":"2024-01-24T13:58:23.207471+0000","last_undegraded":"2024-01-24T13:58:23.207471+0000","last_fullsized":"2024-01-24T13:58:23.207471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207748+0000","last_change":"2024-01-24T13:58:19.211910+0000","last_active":"2024-01-24T13:58:23.207748+0000","last_peered":"2024-01-24T13:58:23.207748+0000","last_clean":"2024-01-24T13:58:23.207748+0000","last_became_active":"2024-01-24T13:58:19.210919+0000","last_became_peered":"2024-01-24T13:58:19.210919+0000","last_unstale":"2024-01-24T13:58:23.207748+0000","last_undegraded":"2024-01-24T13:58:23.207748+0000","last_fullsized":"2024-01-24T13:58:23.207748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232409+0000","last_change":"2024-01-24T13:58:23.223847+0000","last_active":"2024-01-24T13:58:23.232409+0000","last_peered":"2024-01-24T13:58:23.232409+0000","last_clean":"2024-01-24T13:58:23.232409+0000","last_became_active":"2024-01-24T13:58:23.222064+0000","last_became_peered":"2024-01-24T13:58:23.222064+0000","last_unstale":"2024-01-24T13:58:23.232409+0000","last_undegraded":"2024-01-24T13:58:23.232409+0000","last_fullsized":"2024-01-24T13:58:23.232409+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227575+0000","last_change":"2024-01-24T13:58:18.798060+0000","last_active":"2024-01-24T13:58:23.227575+0000","last_peered":"2024-01-24T13:58:23.227575+0000","last_clean":"2024-01-24T13:58:23.227575+0000","last_became_active":"2024-01-24T13:58:18.797774+0000","last_became_peered":"2024-01-24T13:58:18.797774+0000","last_unstale":"2024-01-24T13:58:23.227575+0000","last_undegraded":"2024-01-24T13:58:23.227575+0000","last_fullsized":"2024-01-24T13:58:23.227575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208950+0000","last_change":"2024-01-24T13:58:19.215145+0000","last_active":"2024-01-24T13:58:23.208950+0000","last_peered":"2024-01-24T13:58:23.208950+0000","last_clean":"2024-01-24T13:58:23.208950+0000","last_became_active":"2024-01-24T13:58:19.214840+0000","last_became_peered":"2024-01-24T13:58:19.214840+0000","last_unstale":"2024-01-24T13:58:23.208950+0000","last_undegraded":"2024-01-24T13:58:23.208950+0000","last_fullsized":"2024-01-24T13:58:23.208950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237348+0000","last_change":"2024-01-24T13:58:23.234486+0000","last_active":"2024-01-24T13:58:23.237348+0000","last_peered":"2024-01-24T13:58:23.237348+0000","last_clean":"2024-01-24T13:58:23.237348+0000","last_became_active":"2024-01-24T13:58:23.233948+0000","last_became_peered":"2024-01-24T13:58:23.233948+0000","last_unstale":"2024-01-24T13:58:23.237348+0000","last_undegraded":"2024-01-24T13:58:23.237348+0000","last_fullsized":"2024-01-24T13:58:23.237348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227976+0000","last_change":"2024-01-24T13:58:18.797763+0000","last_active":"2024-01-24T13:58:23.227976+0000","last_peered":"2024-01-24T13:58:23.227976+0000","last_clean":"2024-01-24T13:58:23.227976+0000","last_became_active":"2024-01-24T13:58:18.797478+0000","last_became_peered":"2024-01-24T13:58:18.797478+0000","last_unstale":"2024-01-24T13:58:23.227976+0000","last_undegraded":"2024-01-24T13:58:23.227976+0000","last_fullsized":"2024-01-24T13:58:23.227976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209496+0000","last_change":"2024-01-24T13:58:19.217185+0000","last_active":"2024-01-24T13:58:23.209496+0000","last_peered":"2024-01-24T13:58:23.209496+0000","last_clean":"2024-01-24T13:58:23.209496+0000","last_became_active":"2024-01-24T13:58:19.216628+0000","last_became_peered":"2024-01-24T13:58:19.216628+0000","last_unstale":"2024-01-24T13:58:23.209496+0000","last_undegraded":"2024-01-24T13:58:23.209496+0000","last_fullsized":"2024-01-24T13:58:23.209496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.236747+0000","last_change":"2024-01-24T13:58:23.230969+0000","last_active":"2024-01-24T13:58:23.236747+0000","last_peered":"2024-01-24T13:58:23.236747+0000","last_clean":"2024-01-24T13:58:23.236747+0000","last_became_active":"2024-01-24T13:58:23.230450+0000","last_became_peered":"2024-01-24T13:58:23.230450+0000","last_unstale":"2024-01-24T13:58:23.236747+0000","last_undegraded":"2024-01-24T13:58:23.236747+0000","last_fullsized":"2024-01-24T13:58:23.236747+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241684+0000","last_change":"2024-01-24T13:58:23.231930+0000","last_active":"2024-01-24T13:58:23.241684+0000","last_peered":"2024-01-24T13:58:23.241684+0000","last_clean":"2024-01-24T13:58:23.241684+0000","last_became_active":"2024-01-24T13:58:23.231494+0000","last_became_peered":"2024-01-24T13:58:23.231494+0000","last_unstale":"2024-01-24T13:58:23.241684+0000","last_undegraded":"2024-01-24T13:58:23.241684+0000","last_fullsized":"2024-01-24T13:58:23.241684+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235293+0000","last_change":"2024-01-24T13:58:18.811358+0000","last_active":"2024-01-24T13:58:23.235293+0000","last_peered":"2024-01-24T13:58:23.235293+0000","last_clean":"2024-01-24T13:58:23.235293+0000","last_became_active":"2024-01-24T13:58:18.811087+0000","last_became_peered":"2024-01-24T13:58:18.811087+0000","last_unstale":"2024-01-24T13:58:23.235293+0000","last_undegraded":"2024-01-24T13:58:23.235293+0000","last_fullsized":"2024-01-24T13:58:23.235293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224192+0000","last_change":"2024-01-24T13:58:19.213488+0000","last_active":"2024-01-24T13:58:23.224192+0000","last_peered":"2024-01-24T13:58:23.224192+0000","last_clean":"2024-01-24T13:58:23.224192+0000","last_became_active":"2024-01-24T13:58:19.213211+0000","last_became_peered":"2024-01-24T13:58:19.213211+0000","last_unstale":"2024-01-24T13:58:23.224192+0000","last_undegraded":"2024-01-24T13:58:23.224192+0000","last_fullsized":"2024-01-24T13:58:23.224192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209928+0000","last_change":"2024-01-24T13:58:19.216228+0000","last_active":"2024-01-24T13:58:23.209928+0000","last_peered":"2024-01-24T13:58:23.209928+0000","last_clean":"2024-01-24T13:58:23.209928+0000","last_became_active":"2024-01-24T13:58:19.215952+0000","last_became_peered":"2024-01-24T13:58:19.215952+0000","last_unstale":"2024-01-24T13:58:23.209928+0000","last_undegraded":"2024-01-24T13:58:23.209928+0000","last_fullsized":"2024-01-24T13:58:23.209928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237483+0000","last_change":"2024-01-24T13:58:18.813343+0000","last_active":"2024-01-24T13:58:23.237483+0000","last_peered":"2024-01-24T13:58:23.237483+0000","last_clean":"2024-01-24T13:58:23.237483+0000","last_became_active":"2024-01-24T13:58:18.812987+0000","last_became_peered":"2024-01-24T13:58:18.812987+0000","last_unstale":"2024-01-24T13:58:23.237483+0000","last_undegraded":"2024-01-24T13:58:23.237483+0000","last_fullsized":"2024-01-24T13:58:23.237483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213473+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T13:58:23.213473+0000","last_peered":"2024-01-24T13:58:23.213473+0000","last_clean":"2024-01-24T13:58:23.213473+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T13:58:23.213473+0000","last_undegraded":"2024-01-24T13:58:23.213473+0000","last_fullsized":"2024-01-24T13:58:23.213473+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241577+0000","last_change":"2024-01-24T13:58:23.235347+0000","last_active":"2024-01-24T13:58:23.241577+0000","last_peered":"2024-01-24T13:58:23.241577+0000","last_clean":"2024-01-24T13:58:23.241577+0000","last_became_active":"2024-01-24T13:58:23.234902+0000","last_became_peered":"2024-01-24T13:58:23.234902+0000","last_unstale":"2024-01-24T13:58:23.241577+0000","last_undegraded":"2024-01-24T13:58:23.241577+0000","last_fullsized":"2024-01-24T13:58:23.241577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207482+0000","last_change":"2024-01-24T13:58:19.218418+0000","last_active":"2024-01-24T13:58:23.207482+0000","last_peered":"2024-01-24T13:58:23.207482+0000","last_clean":"2024-01-24T13:58:23.207482+0000","last_became_active":"2024-01-24T13:58:19.210988+0000","last_became_peered":"2024-01-24T13:58:19.210988+0000","last_unstale":"2024-01-24T13:58:23.207482+0000","last_undegraded":"2024-01-24T13:58:23.207482+0000","last_fullsized":"2024-01-24T13:58:23.207482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211503+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T13:58:23.211503+0000","last_peered":"2024-01-24T13:58:23.211503+0000","last_clean":"2024-01-24T13:58:23.211503+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T13:58:23.211503+0000","last_undegraded":"2024-01-24T13:58:23.211503+0000","last_fullsized":"2024-01-24T13:58:23.211503+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207918+0000","last_change":"2024-01-24T13:58:18.798154+0000","last_active":"2024-01-24T13:58:23.207918+0000","last_peered":"2024-01-24T13:58:23.207918+0000","last_clean":"2024-01-24T13:58:23.207918+0000","last_became_active":"2024-01-24T13:58:18.797819+0000","last_became_peered":"2024-01-24T13:58:18.797819+0000","last_unstale":"2024-01-24T13:58:23.207918+0000","last_undegraded":"2024-01-24T13:58:23.207918+0000","last_fullsized":"2024-01-24T13:58:23.207918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232475+0000","last_change":"2024-01-24T13:58:23.230154+0000","last_active":"2024-01-24T13:58:23.232475+0000","last_peered":"2024-01-24T13:58:23.232475+0000","last_clean":"2024-01-24T13:58:23.232475+0000","last_became_active":"2024-01-24T13:58:23.226934+0000","last_became_peered":"2024-01-24T13:58:23.226934+0000","last_unstale":"2024-01-24T13:58:23.232475+0000","last_undegraded":"2024-01-24T13:58:23.232475+0000","last_fullsized":"2024-01-24T13:58:23.232475+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235379+0000","last_change":"2024-01-24T13:58:19.223917+0000","last_active":"2024-01-24T13:58:23.235379+0000","last_peered":"2024-01-24T13:58:23.235379+0000","last_clean":"2024-01-24T13:58:23.235379+0000","last_became_active":"2024-01-24T13:58:19.223624+0000","last_became_peered":"2024-01-24T13:58:19.223624+0000","last_unstale":"2024-01-24T13:58:23.235379+0000","last_undegraded":"2024-01-24T13:58:23.235379+0000","last_fullsized":"2024-01-24T13:58:23.235379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211872+0000","last_change":"2024-01-24T13:58:18.799495+0000","last_active":"2024-01-24T13:58:23.211872+0000","last_peered":"2024-01-24T13:58:23.211872+0000","last_clean":"2024-01-24T13:58:23.211872+0000","last_became_active":"2024-01-24T13:58:18.796442+0000","last_became_peered":"2024-01-24T13:58:18.796442+0000","last_unstale":"2024-01-24T13:58:23.211872+0000","last_undegraded":"2024-01-24T13:58:23.211872+0000","last_fullsized":"2024-01-24T13:58:23.211872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217101+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T13:58:23.217101+0000","last_peered":"2024-01-24T13:58:23.217101+0000","last_clean":"2024-01-24T13:58:23.217101+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T13:58:23.217101+0000","last_undegraded":"2024-01-24T13:58:23.217101+0000","last_fullsized":"2024-01-24T13:58:23.217101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.230573+0000","last_change":"2024-01-24T13:58:23.228846+0000","last_active":"2024-01-24T13:58:23.230573+0000","last_peered":"2024-01-24T13:58:23.230573+0000","last_clean":"2024-01-24T13:58:23.230573+0000","last_became_active":"2024-01-24T13:58:23.228366+0000","last_became_peered":"2024-01-24T13:58:23.228366+0000","last_unstale":"2024-01-24T13:58:23.230573+0000","last_undegraded":"2024-01-24T13:58:23.230573+0000","last_fullsized":"2024-01-24T13:58:23.230573+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232638+0000","last_change":"2024-01-24T13:58:18.810910+0000","last_active":"2024-01-24T13:58:23.232638+0000","last_peered":"2024-01-24T13:58:23.232638+0000","last_clean":"2024-01-24T13:58:23.232638+0000","last_became_active":"2024-01-24T13:58:18.810566+0000","last_became_peered":"2024-01-24T13:58:18.810566+0000","last_unstale":"2024-01-24T13:58:23.232638+0000","last_undegraded":"2024-01-24T13:58:23.232638+0000","last_fullsized":"2024-01-24T13:58:23.232638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216517+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T13:58:23.216517+0000","last_peered":"2024-01-24T13:58:23.216517+0000","last_clean":"2024-01-24T13:58:23.216517+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T13:58:23.216517+0000","last_undegraded":"2024-01-24T13:58:23.216517+0000","last_fullsized":"2024-01-24T13:58:23.216517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237683+0000","last_change":"2024-01-24T13:58:19.209151+0000","last_active":"2024-01-24T13:58:23.237683+0000","last_peered":"2024-01-24T13:58:23.237683+0000","last_clean":"2024-01-24T13:58:23.237683+0000","last_became_active":"2024-01-24T13:58:19.208680+0000","last_became_peered":"2024-01-24T13:58:19.208680+0000","last_unstale":"2024-01-24T13:58:23.237683+0000","last_undegraded":"2024-01-24T13:58:23.237683+0000","last_fullsized":"2024-01-24T13:58:23.237683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237878+0000","last_change":"2024-01-24T13:58:23.235219+0000","last_active":"2024-01-24T13:58:23.237878+0000","last_peered":"2024-01-24T13:58:23.237878+0000","last_clean":"2024-01-24T13:58:23.237878+0000","last_became_active":"2024-01-24T13:58:23.234797+0000","last_became_peered":"2024-01-24T13:58:23.234797+0000","last_unstale":"2024-01-24T13:58:23.237878+0000","last_undegraded":"2024-01-24T13:58:23.237878+0000","last_fullsized":"2024-01-24T13:58:23.237878+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207091+0000","last_change":"2024-01-24T13:58:18.802046+0000","last_active":"2024-01-24T13:58:23.207091+0000","last_peered":"2024-01-24T13:58:23.207091+0000","last_clean":"2024-01-24T13:58:23.207091+0000","last_became_active":"2024-01-24T13:58:18.801757+0000","last_became_peered":"2024-01-24T13:58:18.801757+0000","last_unstale":"2024-01-24T13:58:23.207091+0000","last_undegraded":"2024-01-24T13:58:23.207091+0000","last_fullsized":"2024-01-24T13:58:23.207091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216324+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T13:58:23.216324+0000","last_peered":"2024-01-24T13:58:23.216324+0000","last_clean":"2024-01-24T13:58:23.216324+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T13:58:23.216324+0000","last_undegraded":"2024-01-24T13:58:23.216324+0000","last_fullsized":"2024-01-24T13:58:23.216324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227454+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T13:58:23.227454+0000","last_peered":"2024-01-24T13:58:23.227454+0000","last_clean":"2024-01-24T13:58:23.227454+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T13:58:23.227454+0000","last_undegraded":"2024-01-24T13:58:23.227454+0000","last_fullsized":"2024-01-24T13:58:23.227454+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212923+0000","last_change":"2024-01-24T13:58:19.211323+0000","last_active":"2024-01-24T13:58:23.212923+0000","last_peered":"2024-01-24T13:58:23.212923+0000","last_clean":"2024-01-24T13:58:23.212923+0000","last_became_active":"2024-01-24T13:58:19.211057+0000","last_became_peered":"2024-01-24T13:58:19.211057+0000","last_unstale":"2024-01-24T13:58:23.212923+0000","last_undegraded":"2024-01-24T13:58:23.212923+0000","last_fullsized":"2024-01-24T13:58:23.212923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229871+0000","last_change":"2024-01-24T13:58:23.227723+0000","last_active":"2024-01-24T13:58:23.229871+0000","last_peered":"2024-01-24T13:58:23.229871+0000","last_clean":"2024-01-24T13:58:23.229871+0000","last_became_active":"2024-01-24T13:58:23.227339+0000","last_became_peered":"2024-01-24T13:58:23.227339+0000","last_unstale":"2024-01-24T13:58:23.229871+0000","last_undegraded":"2024-01-24T13:58:23.229871+0000","last_fullsized":"2024-01-24T13:58:23.229871+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.210205+0000","last_change":"2024-01-24T13:58:18.798895+0000","last_active":"2024-01-24T13:58:23.210205+0000","last_peered":"2024-01-24T13:58:23.210205+0000","last_clean":"2024-01-24T13:58:23.210205+0000","last_became_active":"2024-01-24T13:58:18.796782+0000","last_became_peered":"2024-01-24T13:58:18.796782+0000","last_unstale":"2024-01-24T13:58:23.210205+0000","last_undegraded":"2024-01-24T13:58:23.210205+0000","last_fullsized":"2024-01-24T13:58:23.210205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220366+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T13:58:23.220366+0000","last_peered":"2024-01-24T13:58:23.220366+0000","last_clean":"2024-01-24T13:58:23.220366+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T13:58:23.220366+0000","last_undegraded":"2024-01-24T13:58:23.220366+0000","last_fullsized":"2024-01-24T13:58:23.220366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211623+0000","last_change":"2024-01-24T13:58:19.212508+0000","last_active":"2024-01-24T13:58:23.211623+0000","last_peered":"2024-01-24T13:58:23.211623+0000","last_clean":"2024-01-24T13:58:23.211623+0000","last_became_active":"2024-01-24T13:58:19.212211+0000","last_became_peered":"2024-01-24T13:58:19.212211+0000","last_unstale":"2024-01-24T13:58:23.211623+0000","last_undegraded":"2024-01-24T13:58:23.211623+0000","last_fullsized":"2024-01-24T13:58:23.211623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241355+0000","last_change":"2024-01-24T13:58:23.238939+0000","last_active":"2024-01-24T13:58:23.241355+0000","last_peered":"2024-01-24T13:58:23.241355+0000","last_clean":"2024-01-24T13:58:23.241355+0000","last_became_active":"2024-01-24T13:58:23.234940+0000","last_became_peered":"2024-01-24T13:58:23.234940+0000","last_unstale":"2024-01-24T13:58:23.241355+0000","last_undegraded":"2024-01-24T13:58:23.241355+0000","last_fullsized":"2024-01-24T13:58:23.241355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211492+0000","last_change":"2024-01-24T13:58:19.213076+0000","last_active":"2024-01-24T13:58:23.211492+0000","last_peered":"2024-01-24T13:58:23.211492+0000","last_clean":"2024-01-24T13:58:23.211492+0000","last_became_active":"2024-01-24T13:58:19.208454+0000","last_became_peered":"2024-01-24T13:58:19.208454+0000","last_unstale":"2024-01-24T13:58:23.211492+0000","last_undegraded":"2024-01-24T13:58:23.211492+0000","last_fullsized":"2024-01-24T13:58:23.211492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209311+0000","last_change":"2024-01-24T13:58:18.806850+0000","last_active":"2024-01-24T13:58:23.209311+0000","last_peered":"2024-01-24T13:58:23.209311+0000","last_clean":"2024-01-24T13:58:23.209311+0000","last_became_active":"2024-01-24T13:58:18.806456+0000","last_became_peered":"2024-01-24T13:58:18.806456+0000","last_unstale":"2024-01-24T13:58:23.209311+0000","last_undegraded":"2024-01-24T13:58:23.209311+0000","last_fullsized":"2024-01-24T13:58:23.209311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226775+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T13:58:23.226775+0000","last_peered":"2024-01-24T13:58:23.226775+0000","last_clean":"2024-01-24T13:58:23.226775+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T13:58:23.226775+0000","last_undegraded":"2024-01-24T13:58:23.226775+0000","last_fullsized":"2024-01-24T13:58:23.226775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238433+0000","last_change":"2024-01-24T13:58:23.237234+0000","last_active":"2024-01-24T13:58:23.238433+0000","last_peered":"2024-01-24T13:58:23.238433+0000","last_clean":"2024-01-24T13:58:23.238433+0000","last_became_active":"2024-01-24T13:58:23.236720+0000","last_became_peered":"2024-01-24T13:58:23.236720+0000","last_unstale":"2024-01-24T13:58:23.238433+0000","last_undegraded":"2024-01-24T13:58:23.238433+0000","last_fullsized":"2024-01-24T13:58:23.238433+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212103+0000","last_change":"2024-01-24T13:58:19.217116+0000","last_active":"2024-01-24T13:58:23.212103+0000","last_peered":"2024-01-24T13:58:23.212103+0000","last_clean":"2024-01-24T13:58:23.212103+0000","last_became_active":"2024-01-24T13:58:19.216731+0000","last_became_peered":"2024-01-24T13:58:19.216731+0000","last_unstale":"2024-01-24T13:58:23.212103+0000","last_undegraded":"2024-01-24T13:58:23.212103+0000","last_fullsized":"2024-01-24T13:58:23.212103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227689+0000","last_change":"2024-01-24T13:58:18.803400+0000","last_active":"2024-01-24T13:58:23.227689+0000","last_peered":"2024-01-24T13:58:23.227689+0000","last_clean":"2024-01-24T13:58:23.227689+0000","last_became_active":"2024-01-24T13:58:18.802706+0000","last_became_peered":"2024-01-24T13:58:18.802706+0000","last_unstale":"2024-01-24T13:58:23.227689+0000","last_undegraded":"2024-01-24T13:58:23.227689+0000","last_fullsized":"2024-01-24T13:58:23.227689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211632+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T13:58:23.211632+0000","last_peered":"2024-01-24T13:58:23.211632+0000","last_clean":"2024-01-24T13:58:23.211632+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T13:58:23.211632+0000","last_undegraded":"2024-01-24T13:58:23.211632+0000","last_fullsized":"2024-01-24T13:58:23.211632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237685+0000","last_change":"2024-01-24T13:58:23.235636+0000","last_active":"2024-01-24T13:58:23.237685+0000","last_peered":"2024-01-24T13:58:23.237685+0000","last_clean":"2024-01-24T13:58:23.237685+0000","last_became_active":"2024-01-24T13:58:23.234531+0000","last_became_peered":"2024-01-24T13:58:23.234531+0000","last_unstale":"2024-01-24T13:58:23.237685+0000","last_undegraded":"2024-01-24T13:58:23.237685+0000","last_fullsized":"2024-01-24T13:58:23.237685+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207375+0000","last_change":"2024-01-24T13:58:18.806700+0000","last_active":"2024-01-24T13:58:23.207375+0000","last_peered":"2024-01-24T13:58:23.207375+0000","last_clean":"2024-01-24T13:58:23.207375+0000","last_became_active":"2024-01-24T13:58:18.805917+0000","last_became_peered":"2024-01-24T13:58:18.805917+0000","last_unstale":"2024-01-24T13:58:23.207375+0000","last_undegraded":"2024-01-24T13:58:23.207375+0000","last_fullsized":"2024-01-24T13:58:23.207375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211671+0000","last_change":"2024-01-24T13:58:19.215828+0000","last_active":"2024-01-24T13:58:23.211671+0000","last_peered":"2024-01-24T13:58:23.211671+0000","last_clean":"2024-01-24T13:58:23.211671+0000","last_became_active":"2024-01-24T13:58:19.215015+0000","last_became_peered":"2024-01-24T13:58:19.215015+0000","last_unstale":"2024-01-24T13:58:23.211671+0000","last_undegraded":"2024-01-24T13:58:23.211671+0000","last_fullsized":"2024-01-24T13:58:23.211671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238346+0000","last_change":"2024-01-24T13:58:23.236631+0000","last_active":"2024-01-24T13:58:23.238346+0000","last_peered":"2024-01-24T13:58:23.238346+0000","last_clean":"2024-01-24T13:58:23.238346+0000","last_became_active":"2024-01-24T13:58:23.236200+0000","last_became_peered":"2024-01-24T13:58:23.236200+0000","last_unstale":"2024-01-24T13:58:23.238346+0000","last_undegraded":"2024-01-24T13:58:23.238346+0000","last_fullsized":"2024-01-24T13:58:23.238346+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209848+0000","last_change":"2024-01-24T13:58:18.803901+0000","last_active":"2024-01-24T13:58:23.209848+0000","last_peered":"2024-01-24T13:58:23.209848+0000","last_clean":"2024-01-24T13:58:23.209848+0000","last_became_active":"2024-01-24T13:58:18.803605+0000","last_became_peered":"2024-01-24T13:58:18.803605+0000","last_unstale":"2024-01-24T13:58:23.209848+0000","last_undegraded":"2024-01-24T13:58:23.209848+0000","last_fullsized":"2024-01-24T13:58:23.209848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209809+0000","last_change":"2024-01-24T13:58:19.216716+0000","last_active":"2024-01-24T13:58:23.209809+0000","last_peered":"2024-01-24T13:58:23.209809+0000","last_clean":"2024-01-24T13:58:23.209809+0000","last_became_active":"2024-01-24T13:58:19.216444+0000","last_became_peered":"2024-01-24T13:58:19.216444+0000","last_unstale":"2024-01-24T13:58:23.209809+0000","last_undegraded":"2024-01-24T13:58:23.209809+0000","last_fullsized":"2024-01-24T13:58:23.209809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.242821+0000","last_change":"2024-01-24T13:58:23.242205+0000","last_active":"2024-01-24T13:58:23.242821+0000","last_peered":"2024-01-24T13:58:23.242821+0000","last_clean":"2024-01-24T13:58:23.242821+0000","last_became_active":"2024-01-24T13:58:23.241764+0000","last_became_peered":"2024-01-24T13:58:23.241764+0000","last_unstale":"2024-01-24T13:58:23.242821+0000","last_undegraded":"2024-01-24T13:58:23.242821+0000","last_fullsized":"2024-01-24T13:58:23.242821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212581+0000","last_change":"2024-01-24T13:58:18.808313+0000","last_active":"2024-01-24T13:58:23.212581+0000","last_peered":"2024-01-24T13:58:23.212581+0000","last_clean":"2024-01-24T13:58:23.212581+0000","last_became_active":"2024-01-24T13:58:18.808096+0000","last_became_peered":"2024-01-24T13:58:18.808096+0000","last_unstale":"2024-01-24T13:58:23.212581+0000","last_undegraded":"2024-01-24T13:58:23.212581+0000","last_fullsized":"2024-01-24T13:58:23.212581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209286+0000","last_change":"2024-01-24T13:58:19.209979+0000","last_active":"2024-01-24T13:58:23.209286+0000","last_peered":"2024-01-24T13:58:23.209286+0000","last_clean":"2024-01-24T13:58:23.209286+0000","last_became_active":"2024-01-24T13:58:19.209672+0000","last_became_peered":"2024-01-24T13:58:19.209672+0000","last_unstale":"2024-01-24T13:58:23.209286+0000","last_undegraded":"2024-01-24T13:58:23.209286+0000","last_fullsized":"2024-01-24T13:58:23.209286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.240684+0000","last_change":"2024-01-24T13:58:23.229668+0000","last_active":"2024-01-24T13:58:23.240684+0000","last_peered":"2024-01-24T13:58:23.240684+0000","last_clean":"2024-01-24T13:58:23.240684+0000","last_became_active":"2024-01-24T13:58:23.229053+0000","last_became_peered":"2024-01-24T13:58:23.229053+0000","last_unstale":"2024-01-24T13:58:23.240684+0000","last_undegraded":"2024-01-24T13:58:23.240684+0000","last_fullsized":"2024-01-24T13:58:23.240684+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207919+0000","last_change":"2024-01-24T13:58:18.807736+0000","last_active":"2024-01-24T13:58:23.207919+0000","last_peered":"2024-01-24T13:58:23.207919+0000","last_clean":"2024-01-24T13:58:23.207919+0000","last_became_active":"2024-01-24T13:58:18.807228+0000","last_became_peered":"2024-01-24T13:58:18.807228+0000","last_unstale":"2024-01-24T13:58:23.207919+0000","last_undegraded":"2024-01-24T13:58:23.207919+0000","last_fullsized":"2024-01-24T13:58:23.207919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209909+0000","last_change":"2024-01-24T13:58:19.218038+0000","last_active":"2024-01-24T13:58:23.209909+0000","last_peered":"2024-01-24T13:58:23.209909+0000","last_clean":"2024-01-24T13:58:23.209909+0000","last_became_active":"2024-01-24T13:58:19.210668+0000","last_became_peered":"2024-01-24T13:58:19.210668+0000","last_unstale":"2024-01-24T13:58:23.209909+0000","last_undegraded":"2024-01-24T13:58:23.209909+0000","last_fullsized":"2024-01-24T13:58:23.209909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241201+0000","last_change":"2024-01-24T13:58:23.234754+0000","last_active":"2024-01-24T13:58:23.241201+0000","last_peered":"2024-01-24T13:58:23.241201+0000","last_clean":"2024-01-24T13:58:23.241201+0000","last_became_active":"2024-01-24T13:58:23.233898+0000","last_became_peered":"2024-01-24T13:58:23.233898+0000","last_unstale":"2024-01-24T13:58:23.241201+0000","last_undegraded":"2024-01-24T13:58:23.241201+0000","last_fullsized":"2024-01-24T13:58:23.241201+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227851+0000","last_change":"2024-01-24T13:58:18.802179+0000","last_active":"2024-01-24T13:58:23.227851+0000","last_peered":"2024-01-24T13:58:23.227851+0000","last_clean":"2024-01-24T13:58:23.227851+0000","last_became_active":"2024-01-24T13:58:18.801898+0000","last_became_peered":"2024-01-24T13:58:18.801898+0000","last_unstale":"2024-01-24T13:58:23.227851+0000","last_undegraded":"2024-01-24T13:58:23.227851+0000","last_fullsized":"2024-01-24T13:58:23.227851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.234948+0000","last_change":"2024-01-24T13:58:19.224275+0000","last_active":"2024-01-24T13:58:23.234948+0000","last_peered":"2024-01-24T13:58:23.234948+0000","last_clean":"2024-01-24T13:58:23.234948+0000","last_became_active":"2024-01-24T13:58:19.224006+0000","last_became_peered":"2024-01-24T13:58:19.224006+0000","last_unstale":"2024-01-24T13:58:23.234948+0000","last_undegraded":"2024-01-24T13:58:23.234948+0000","last_fullsized":"2024-01-24T13:58:23.234948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241374+0000","last_change":"2024-01-24T13:58:23.239977+0000","last_active":"2024-01-24T13:58:23.241374+0000","last_peered":"2024-01-24T13:58:23.241374+0000","last_clean":"2024-01-24T13:58:23.241374+0000","last_became_active":"2024-01-24T13:58:23.234358+0000","last_became_peered":"2024-01-24T13:58:23.234358+0000","last_unstale":"2024-01-24T13:58:23.241374+0000","last_undegraded":"2024-01-24T13:58:23.241374+0000","last_fullsized":"2024-01-24T13:58:23.241374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217009+0000","last_change":"2024-01-24T13:58:18.804317+0000","last_active":"2024-01-24T13:58:23.217009+0000","last_peered":"2024-01-24T13:58:23.217009+0000","last_clean":"2024-01-24T13:58:23.217009+0000","last_became_active":"2024-01-24T13:58:18.804046+0000","last_became_peered":"2024-01-24T13:58:18.804046+0000","last_unstale":"2024-01-24T13:58:23.217009+0000","last_undegraded":"2024-01-24T13:58:23.217009+0000","last_fullsized":"2024-01-24T13:58:23.217009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212503+0000","last_change":"2024-01-24T13:58:19.217782+0000","last_active":"2024-01-24T13:58:23.212503+0000","last_peered":"2024-01-24T13:58:23.212503+0000","last_clean":"2024-01-24T13:58:23.212503+0000","last_became_active":"2024-01-24T13:58:19.217509+0000","last_became_peered":"2024-01-24T13:58:19.217509+0000","last_unstale":"2024-01-24T13:58:23.212503+0000","last_undegraded":"2024-01-24T13:58:23.212503+0000","last_fullsized":"2024-01-24T13:58:23.212503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241249+0000","last_change":"2024-01-24T13:58:23.237049+0000","last_active":"2024-01-24T13:58:23.241249+0000","last_peered":"2024-01-24T13:58:23.241249+0000","last_clean":"2024-01-24T13:58:23.241249+0000","last_became_active":"2024-01-24T13:58:23.235154+0000","last_became_peered":"2024-01-24T13:58:23.235154+0000","last_unstale":"2024-01-24T13:58:23.241249+0000","last_undegraded":"2024-01-24T13:58:23.241249+0000","last_fullsized":"2024-01-24T13:58:23.241249+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217784+0000","last_change":"2024-01-24T13:58:18.801841+0000","last_active":"2024-01-24T13:58:23.217784+0000","last_peered":"2024-01-24T13:58:23.217784+0000","last_clean":"2024-01-24T13:58:23.217784+0000","last_became_active":"2024-01-24T13:58:18.801535+0000","last_became_peered":"2024-01-24T13:58:18.801535+0000","last_unstale":"2024-01-24T13:58:23.217784+0000","last_undegraded":"2024-01-24T13:58:23.217784+0000","last_fullsized":"2024-01-24T13:58:23.217784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209722+0000","last_change":"2024-01-24T13:58:19.213673+0000","last_active":"2024-01-24T13:58:23.209722+0000","last_peered":"2024-01-24T13:58:23.209722+0000","last_clean":"2024-01-24T13:58:23.209722+0000","last_became_active":"2024-01-24T13:58:19.213457+0000","last_became_peered":"2024-01-24T13:58:19.213457+0000","last_unstale":"2024-01-24T13:58:23.209722+0000","last_undegraded":"2024-01-24T13:58:23.209722+0000","last_fullsized":"2024-01-24T13:58:23.209722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238263+0000","last_change":"2024-01-24T13:58:23.234302+0000","last_active":"2024-01-24T13:58:23.238263+0000","last_peered":"2024-01-24T13:58:23.238263+0000","last_clean":"2024-01-24T13:58:23.238263+0000","last_became_active":"2024-01-24T13:58:23.233692+0000","last_became_peered":"2024-01-24T13:58:23.233692+0000","last_unstale":"2024-01-24T13:58:23.238263+0000","last_undegraded":"2024-01-24T13:58:23.238263+0000","last_fullsized":"2024-01-24T13:58:23.238263+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.218145+0000","last_change":"2024-01-24T13:58:18.798076+0000","last_active":"2024-01-24T13:58:23.218145+0000","last_peered":"2024-01-24T13:58:23.218145+0000","last_clean":"2024-01-24T13:58:23.218145+0000","last_became_active":"2024-01-24T13:58:18.797403+0000","last_became_peered":"2024-01-24T13:58:18.797403+0000","last_unstale":"2024-01-24T13:58:23.218145+0000","last_undegraded":"2024-01-24T13:58:23.218145+0000","last_fullsized":"2024-01-24T13:58:23.218145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224903+0000","last_change":"2024-01-24T13:58:19.210870+0000","last_active":"2024-01-24T13:58:23.224903+0000","last_peered":"2024-01-24T13:58:23.224903+0000","last_clean":"2024-01-24T13:58:23.224903+0000","last_became_active":"2024-01-24T13:58:19.210226+0000","last_became_peered":"2024-01-24T13:58:19.210226+0000","last_unstale":"2024-01-24T13:58:23.224903+0000","last_undegraded":"2024-01-24T13:58:23.224903+0000","last_fullsized":"2024-01-24T13:58:23.224903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232630+0000","last_change":"2024-01-24T13:58:23.230340+0000","last_active":"2024-01-24T13:58:23.232630+0000","last_peered":"2024-01-24T13:58:23.232630+0000","last_clean":"2024-01-24T13:58:23.232630+0000","last_became_active":"2024-01-24T13:58:23.229909+0000","last_became_peered":"2024-01-24T13:58:23.229909+0000","last_unstale":"2024-01-24T13:58:23.232630+0000","last_undegraded":"2024-01-24T13:58:23.232630+0000","last_fullsized":"2024-01-24T13:58:23.232630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"21'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212395+0000","last_change":"2024-01-24T13:58:18.813040+0000","last_active":"2024-01-24T13:58:23.212395+0000","last_peered":"2024-01-24T13:58:23.212395+0000","last_clean":"2024-01-24T13:58:23.212395+0000","last_became_active":"2024-01-24T13:58:18.812822+0000","last_became_peered":"2024-01-24T13:58:18.812822+0000","last_unstale":"2024-01-24T13:58:23.212395+0000","last_undegraded":"2024-01-24T13:58:23.212395+0000","last_fullsized":"2024-01-24T13:58:23.212395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208215+0000","last_change":"2024-01-24T13:58:19.216414+0000","last_active":"2024-01-24T13:58:23.208215+0000","last_peered":"2024-01-24T13:58:23.208215+0000","last_clean":"2024-01-24T13:58:23.208215+0000","last_became_active":"2024-01-24T13:58:19.216111+0000","last_became_peered":"2024-01-24T13:58:19.216111+0000","last_unstale":"2024-01-24T13:58:23.208215+0000","last_undegraded":"2024-01-24T13:58:23.208215+0000","last_fullsized":"2024-01-24T13:58:23.208215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237430+0000","last_change":"2024-01-24T13:58:23.234462+0000","last_active":"2024-01-24T13:58:23.237430+0000","last_peered":"2024-01-24T13:58:23.237430+0000","last_clean":"2024-01-24T13:58:23.237430+0000","last_became_active":"2024-01-24T13:58:23.233834+0000","last_became_peered":"2024-01-24T13:58:23.233834+0000","last_unstale":"2024-01-24T13:58:23.237430+0000","last_undegraded":"2024-01-24T13:58:23.237430+0000","last_fullsized":"2024-01-24T13:58:23.237430+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216341+0000","last_change":"2024-01-24T13:58:18.801184+0000","last_active":"2024-01-24T13:58:23.216341+0000","last_peered":"2024-01-24T13:58:23.216341+0000","last_clean":"2024-01-24T13:58:23.216341+0000","last_became_active":"2024-01-24T13:58:18.800877+0000","last_became_peered":"2024-01-24T13:58:18.800877+0000","last_unstale":"2024-01-24T13:58:23.216341+0000","last_undegraded":"2024-01-24T13:58:23.216341+0000","last_fullsized":"2024-01-24T13:58:23.216341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237737+0000","last_change":"2024-01-24T13:58:19.209281+0000","last_active":"2024-01-24T13:58:23.237737+0000","last_peered":"2024-01-24T13:58:23.237737+0000","last_clean":"2024-01-24T13:58:23.237737+0000","last_became_active":"2024-01-24T13:58:19.208890+0000","last_became_peered":"2024-01-24T13:58:19.208890+0000","last_unstale":"2024-01-24T13:58:23.237737+0000","last_undegraded":"2024-01-24T13:58:23.237737+0000","last_fullsized":"2024-01-24T13:58:23.237737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.230375+0000","last_change":"2024-01-24T13:58:23.228668+0000","last_active":"2024-01-24T13:58:23.230375+0000","last_peered":"2024-01-24T13:58:23.230375+0000","last_clean":"2024-01-24T13:58:23.230375+0000","last_became_active":"2024-01-24T13:58:23.228266+0000","last_became_peered":"2024-01-24T13:58:23.228266+0000","last_unstale":"2024-01-24T13:58:23.230375+0000","last_undegraded":"2024-01-24T13:58:23.230375+0000","last_fullsized":"2024-01-24T13:58:23.230375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232549+0000","last_change":"2024-01-24T13:58:18.809056+0000","last_active":"2024-01-24T13:58:23.232549+0000","last_peered":"2024-01-24T13:58:23.232549+0000","last_clean":"2024-01-24T13:58:23.232549+0000","last_became_active":"2024-01-24T13:58:18.808790+0000","last_became_peered":"2024-01-24T13:58:18.808790+0000","last_unstale":"2024-01-24T13:58:23.232549+0000","last_undegraded":"2024-01-24T13:58:23.232549+0000","last_fullsized":"2024-01-24T13:58:23.232549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227958+0000","last_change":"2024-01-24T13:58:19.217272+0000","last_active":"2024-01-24T13:58:23.227958+0000","last_peered":"2024-01-24T13:58:23.227958+0000","last_clean":"2024-01-24T13:58:23.227958+0000","last_became_active":"2024-01-24T13:58:19.216991+0000","last_became_peered":"2024-01-24T13:58:19.216991+0000","last_unstale":"2024-01-24T13:58:23.227958+0000","last_undegraded":"2024-01-24T13:58:23.227958+0000","last_fullsized":"2024-01-24T13:58:23.227958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.231091+0000","last_change":"2024-01-24T13:58:23.228760+0000","last_active":"2024-01-24T13:58:23.231091+0000","last_peered":"2024-01-24T13:58:23.231091+0000","last_clean":"2024-01-24T13:58:23.231091+0000","last_became_active":"2024-01-24T13:58:23.227345+0000","last_became_peered":"2024-01-24T13:58:23.227345+0000","last_unstale":"2024-01-24T13:58:23.231091+0000","last_undegraded":"2024-01-24T13:58:23.231091+0000","last_fullsized":"2024-01-24T13:58:23.231091+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217256+0000","last_change":"2024-01-24T13:58:18.798145+0000","last_active":"2024-01-24T13:58:23.217256+0000","last_peered":"2024-01-24T13:58:23.217256+0000","last_clean":"2024-01-24T13:58:23.217256+0000","last_became_active":"2024-01-24T13:58:18.797671+0000","last_became_peered":"2024-01-24T13:58:18.797671+0000","last_unstale":"2024-01-24T13:58:23.217256+0000","last_undegraded":"2024-01-24T13:58:23.217256+0000","last_fullsized":"2024-01-24T13:58:23.217256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207234+0000","last_change":"2024-01-24T13:58:19.221287+0000","last_active":"2024-01-24T13:58:23.207234+0000","last_peered":"2024-01-24T13:58:23.207234+0000","last_clean":"2024-01-24T13:58:23.207234+0000","last_became_active":"2024-01-24T13:58:19.220970+0000","last_became_peered":"2024-01-24T13:58:19.220970+0000","last_unstale":"2024-01-24T13:58:23.207234+0000","last_undegraded":"2024-01-24T13:58:23.207234+0000","last_fullsized":"2024-01-24T13:58:23.207234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237365+0000","last_change":"2024-01-24T13:58:23.234514+0000","last_active":"2024-01-24T13:58:23.237365+0000","last_peered":"2024-01-24T13:58:23.237365+0000","last_clean":"2024-01-24T13:58:23.237365+0000","last_became_active":"2024-01-24T13:58:23.233824+0000","last_became_peered":"2024-01-24T13:58:23.233824+0000","last_unstale":"2024-01-24T13:58:23.237365+0000","last_undegraded":"2024-01-24T13:58:23.237365+0000","last_fullsized":"2024-01-24T13:58:23.237365+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209685+0000","last_change":"2024-01-24T13:58:18.801316+0000","last_active":"2024-01-24T13:58:23.209685+0000","last_peered":"2024-01-24T13:58:23.209685+0000","last_clean":"2024-01-24T13:58:23.209685+0000","last_became_active":"2024-01-24T13:58:18.800089+0000","last_became_peered":"2024-01-24T13:58:18.800089+0000","last_unstale":"2024-01-24T13:58:23.209685+0000","last_undegraded":"2024-01-24T13:58:23.209685+0000","last_fullsized":"2024-01-24T13:58:23.209685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209210+0000","last_change":"2024-01-24T13:58:19.209604+0000","last_active":"2024-01-24T13:58:23.209210+0000","last_peered":"2024-01-24T13:58:23.209210+0000","last_clean":"2024-01-24T13:58:23.209210+0000","last_became_active":"2024-01-24T13:58:19.209306+0000","last_became_peered":"2024-01-24T13:58:19.209306+0000","last_unstale":"2024-01-24T13:58:23.209210+0000","last_undegraded":"2024-01-24T13:58:23.209210+0000","last_fullsized":"2024-01-24T13:58:23.209210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229399+0000","last_change":"2024-01-24T13:58:23.225921+0000","last_active":"2024-01-24T13:58:23.229399+0000","last_peered":"2024-01-24T13:58:23.229399+0000","last_clean":"2024-01-24T13:58:23.229399+0000","last_became_active":"2024-01-24T13:58:23.225413+0000","last_became_peered":"2024-01-24T13:58:23.225413+0000","last_unstale":"2024-01-24T13:58:23.229399+0000","last_undegraded":"2024-01-24T13:58:23.229399+0000","last_fullsized":"2024-01-24T13:58:23.229399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217204+0000","last_change":"2024-01-24T13:58:18.799227+0000","last_active":"2024-01-24T13:58:23.217204+0000","last_peered":"2024-01-24T13:58:23.217204+0000","last_clean":"2024-01-24T13:58:23.217204+0000","last_became_active":"2024-01-24T13:58:18.798927+0000","last_became_peered":"2024-01-24T13:58:18.798927+0000","last_unstale":"2024-01-24T13:58:23.217204+0000","last_undegraded":"2024-01-24T13:58:23.217204+0000","last_fullsized":"2024-01-24T13:58:23.217204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212286+0000","last_change":"2024-01-24T13:58:19.217668+0000","last_active":"2024-01-24T13:58:23.212286+0000","last_peered":"2024-01-24T13:58:23.212286+0000","last_clean":"2024-01-24T13:58:23.212286+0000","last_became_active":"2024-01-24T13:58:19.217323+0000","last_became_peered":"2024-01-24T13:58:19.217323+0000","last_unstale":"2024-01-24T13:58:23.212286+0000","last_undegraded":"2024-01-24T13:58:23.212286+0000","last_fullsized":"2024-01-24T13:58:23.212286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.230948+0000","last_change":"2024-01-24T13:58:23.229837+0000","last_active":"2024-01-24T13:58:23.230948+0000","last_peered":"2024-01-24T13:58:23.230948+0000","last_clean":"2024-01-24T13:58:23.230948+0000","last_became_active":"2024-01-24T13:58:23.229398+0000","last_became_peered":"2024-01-24T13:58:23.229398+0000","last_unstale":"2024-01-24T13:58:23.230948+0000","last_undegraded":"2024-01-24T13:58:23.230948+0000","last_fullsized":"2024-01-24T13:58:23.230948+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227679+0000","last_change":"2024-01-24T13:58:18.798763+0000","last_active":"2024-01-24T13:58:23.227679+0000","last_peered":"2024-01-24T13:58:23.227679+0000","last_clean":"2024-01-24T13:58:23.227679+0000","last_became_active":"2024-01-24T13:58:18.798451+0000","last_became_peered":"2024-01-24T13:58:18.798451+0000","last_unstale":"2024-01-24T13:58:23.227679+0000","last_undegraded":"2024-01-24T13:58:23.227679+0000","last_fullsized":"2024-01-24T13:58:23.227679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207643+0000","last_change":"2024-01-24T13:58:19.221011+0000","last_active":"2024-01-24T13:58:23.207643+0000","last_peered":"2024-01-24T13:58:23.207643+0000","last_clean":"2024-01-24T13:58:23.207643+0000","last_became_active":"2024-01-24T13:58:19.220727+0000","last_became_peered":"2024-01-24T13:58:19.220727+0000","last_unstale":"2024-01-24T13:58:23.207643+0000","last_undegraded":"2024-01-24T13:58:23.207643+0000","last_fullsized":"2024-01-24T13:58:23.207643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237556+0000","last_change":"2024-01-24T13:58:23.235211+0000","last_active":"2024-01-24T13:58:23.237556+0000","last_peered":"2024-01-24T13:58:23.237556+0000","last_clean":"2024-01-24T13:58:23.237556+0000","last_became_active":"2024-01-24T13:58:23.234791+0000","last_became_peered":"2024-01-24T13:58:23.234791+0000","last_unstale":"2024-01-24T13:58:23.237556+0000","last_undegraded":"2024-01-24T13:58:23.237556+0000","last_fullsized":"2024-01-24T13:58:23.237556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237921+0000","last_change":"2024-01-24T13:58:18.815292+0000","last_active":"2024-01-24T13:58:23.237921+0000","last_peered":"2024-01-24T13:58:23.237921+0000","last_clean":"2024-01-24T13:58:23.237921+0000","last_became_active":"2024-01-24T13:58:18.811889+0000","last_became_peered":"2024-01-24T13:58:18.811889+0000","last_unstale":"2024-01-24T13:58:23.237921+0000","last_undegraded":"2024-01-24T13:58:23.237921+0000","last_fullsized":"2024-01-24T13:58:23.237921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207065+0000","last_change":"2024-01-24T13:58:19.218174+0000","last_active":"2024-01-24T13:58:23.207065+0000","last_peered":"2024-01-24T13:58:23.207065+0000","last_clean":"2024-01-24T13:58:23.207065+0000","last_became_active":"2024-01-24T13:58:19.211059+0000","last_became_peered":"2024-01-24T13:58:19.211059+0000","last_unstale":"2024-01-24T13:58:23.207065+0000","last_undegraded":"2024-01-24T13:58:23.207065+0000","last_fullsized":"2024-01-24T13:58:23.207065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229596+0000","last_change":"2024-01-24T13:58:23.225806+0000","last_active":"2024-01-24T13:58:23.229596+0000","last_peered":"2024-01-24T13:58:23.229596+0000","last_clean":"2024-01-24T13:58:23.229596+0000","last_became_active":"2024-01-24T13:58:23.225425+0000","last_became_peered":"2024-01-24T13:58:23.225425+0000","last_unstale":"2024-01-24T13:58:23.229596+0000","last_undegraded":"2024-01-24T13:58:23.229596+0000","last_fullsized":"2024-01-24T13:58:23.229596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235654+0000","last_change":"2024-01-24T13:58:23.226907+0000","last_active":"2024-01-24T13:58:23.235654+0000","last_peered":"2024-01-24T13:58:23.235654+0000","last_clean":"2024-01-24T13:58:23.235654+0000","last_became_active":"2024-01-24T13:58:23.226413+0000","last_became_peered":"2024-01-24T13:58:23.226413+0000","last_unstale":"2024-01-24T13:58:23.235654+0000","last_undegraded":"2024-01-24T13:58:23.235654+0000","last_fullsized":"2024-01-24T13:58:23.235654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.206957+0000","last_change":"2024-01-24T13:58:18.806830+0000","last_active":"2024-01-24T13:58:23.206957+0000","last_peered":"2024-01-24T13:58:23.206957+0000","last_clean":"2024-01-24T13:58:23.206957+0000","last_became_active":"2024-01-24T13:58:18.806385+0000","last_became_peered":"2024-01-24T13:58:18.806385+0000","last_unstale":"2024-01-24T13:58:23.206957+0000","last_undegraded":"2024-01-24T13:58:23.206957+0000","last_fullsized":"2024-01-24T13:58:23.206957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212967+0000","last_change":"2024-01-24T13:58:19.210472+0000","last_active":"2024-01-24T13:58:23.212967+0000","last_peered":"2024-01-24T13:58:23.212967+0000","last_clean":"2024-01-24T13:58:23.212967+0000","last_became_active":"2024-01-24T13:58:19.210211+0000","last_became_peered":"2024-01-24T13:58:19.210211+0000","last_unstale":"2024-01-24T13:58:23.212967+0000","last_undegraded":"2024-01-24T13:58:23.212967+0000","last_fullsized":"2024-01-24T13:58:23.212967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229446+0000","last_change":"2024-01-24T13:58:23.223133+0000","last_active":"2024-01-24T13:58:23.229446+0000","last_peered":"2024-01-24T13:58:23.229446+0000","last_clean":"2024-01-24T13:58:23.229446+0000","last_became_active":"2024-01-24T13:58:23.222741+0000","last_became_peered":"2024-01-24T13:58:23.222741+0000","last_unstale":"2024-01-24T13:58:23.229446+0000","last_undegraded":"2024-01-24T13:58:23.229446+0000","last_fullsized":"2024-01-24T13:58:23.229446+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.210236+0000","last_change":"2024-01-24T13:58:18.800359+0000","last_active":"2024-01-24T13:58:23.210236+0000","last_peered":"2024-01-24T13:58:23.210236+0000","last_clean":"2024-01-24T13:58:23.210236+0000","last_became_active":"2024-01-24T13:58:18.799774+0000","last_became_peered":"2024-01-24T13:58:18.799774+0000","last_unstale":"2024-01-24T13:58:23.210236+0000","last_undegraded":"2024-01-24T13:58:23.210236+0000","last_fullsized":"2024-01-24T13:58:23.210236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237579+0000","last_change":"2024-01-24T13:58:19.210774+0000","last_active":"2024-01-24T13:58:23.237579+0000","last_peered":"2024-01-24T13:58:23.237579+0000","last_clean":"2024-01-24T13:58:23.237579+0000","last_became_active":"2024-01-24T13:58:19.210508+0000","last_became_peered":"2024-01-24T13:58:19.210508+0000","last_unstale":"2024-01-24T13:58:23.237579+0000","last_undegraded":"2024-01-24T13:58:23.237579+0000","last_fullsized":"2024-01-24T13:58:23.237579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209129+0000","last_change":"2024-01-24T13:58:18.803254+0000","last_active":"2024-01-24T13:58:23.209129+0000","last_peered":"2024-01-24T13:58:23.209129+0000","last_clean":"2024-01-24T13:58:23.209129+0000","last_became_active":"2024-01-24T13:58:18.802965+0000","last_became_peered":"2024-01-24T13:58:18.802965+0000","last_unstale":"2024-01-24T13:58:23.209129+0000","last_undegraded":"2024-01-24T13:58:23.209129+0000","last_fullsized":"2024-01-24T13:58:23.209129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208494+0000","last_change":"2024-01-24T13:58:19.221404+0000","last_active":"2024-01-24T13:58:23.208494+0000","last_peered":"2024-01-24T13:58:23.208494+0000","last_clean":"2024-01-24T13:58:23.208494+0000","last_became_active":"2024-01-24T13:58:19.221024+0000","last_became_peered":"2024-01-24T13:58:19.221024+0000","last_unstale":"2024-01-24T13:58:23.208494+0000","last_undegraded":"2024-01-24T13:58:23.208494+0000","last_fullsized":"2024-01-24T13:58:23.208494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232448+0000","last_change":"2024-01-24T13:58:23.229756+0000","last_active":"2024-01-24T13:58:23.232448+0000","last_peered":"2024-01-24T13:58:23.232448+0000","last_clean":"2024-01-24T13:58:23.232448+0000","last_became_active":"2024-01-24T13:58:23.229328+0000","last_became_peered":"2024-01-24T13:58:23.229328+0000","last_unstale":"2024-01-24T13:58:23.232448+0000","last_undegraded":"2024-01-24T13:58:23.232448+0000","last_fullsized":"2024-01-24T13:58:23.232448+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297364,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074476,"statfs":{"total":96636764160,"available":96332263424,"internally_reserved":0,"allocated":274432,"data_stored":115668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297396,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074444,"statfs":{"total":96636764160,"available":96332230656,"internally_reserved":0,"allocated":307200,"data_stored":118043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297372,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074468,"statfs":{"total":96636764160,"available":96332255232,"internally_reserved":0,"allocated":282624,"data_stored":116699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297368,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074472,"statfs":{"total":96636764160,"available":96332259328,"internally_reserved":0,"allocated":278528,"data_stored":116629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297372,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074468,"statfs":{"total":96636764160,"available":96332255232,"internally_reserved":0,"allocated":282624,"data_stored":116719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297388,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074452,"statfs":{"total":96636764160,"available":96332238848,"internally_reserved":0,"allocated":299008,"data_stored":120400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297368,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074472,"statfs":{"total":96636764160,"available":96332259328,"internally_reserved":0,"allocated":278528,"data_stored":116629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297376,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074464,"statfs":{"total":96636764160,"available":96332251136,"internally_reserved":0,"allocated":286720,"data_stored":120254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":3625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T13:58:32.740 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-24T13:58:32.741 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-24T13:58:32.741 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-24T13:58:32.741 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-24T13:58:32.741 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-24T13:58:32.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.908+0000 7f603d850700 1 Processor -- start 2024-01-24T13:58:32.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.908+0000 7f603d850700 1 -- start start 2024-01-24T13:58:32.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.909+0000 7f603d850700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60381560c0 0x7f60380c5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.909+0000 7f603d850700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6038066720 con 0x7f603814faf0 2024-01-24T13:58:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.909+0000 7f603d850700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60380be220 con 0x7f603814f710 2024-01-24T13:58:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.909+0000 7f603d850700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6038068b20 con 0x7f60381560c0 2024-01-24T13:58:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.909+0000 7f60367fc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f603814faf0 0x7f60381559b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33726/0 (socket says 172.21.15.29:33726) 2024-01-24T13:58:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.909+0000 7f60367fc700 1 -- 172.21.15.29:0/3131517173 learned_addr learned my addr 172.21.15.29:0/3131517173 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.909+0000 7f60377fe700 1 --2- 172.21.15.29:0/3131517173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60381560c0 0x7f60380c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.910+0000 7f6035ffb700 1 -- 172.21.15.29:0/3131517173 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963616259 0 0) 0x7f6038066720 con 0x7f603814faf0 2024-01-24T13:58:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.910+0000 7f6035ffb700 1 -- 172.21.15.29:0/3131517173 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60240033f0 con 0x7f603814faf0 2024-01-24T13:58:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.910+0000 7f60377fe700 1 -- 172.21.15.29:0/3131517173 >> v1:172.21.15.163:6789/0 conn(0x7f603814f710 legacy=0x7f60380bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.910+0000 7f6035ffb700 1 -- 172.21.15.29:0/3131517173 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408906028 0 0) 0x7f60380be220 con 0x7f603814f710 2024-01-24T13:58:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.910+0000 7f60377fe700 1 -- 172.21.15.29:0/3131517173 >> v1:172.21.15.29:6789/0 conn(0x7f603814faf0 legacy=0x7f60381559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.910+0000 7f60377fe700 1 -- 172.21.15.29:0/3131517173 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60380c3350 con 0x7f60381560c0 2024-01-24T13:58:32.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.910+0000 7f60377fe700 1 --2- 172.21.15.29:0/3131517173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60381560c0 0x7f60380c5000 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f6030009e90 tx=0x7f6030009780).ready entity=mon.2 client_cookie=68b6d468538c51b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:32.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.910+0000 7f6035ffb700 1 -- 172.21.15.29:0/3131517173 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603000e070 con 0x7f60381560c0 2024-01-24T13:58:32.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.910+0000 7f6035ffb700 1 -- 172.21.15.29:0/3131517173 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603000bb60 con 0x7f60381560c0 2024-01-24T13:58:32.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.910+0000 7f6035ffb700 1 -- 172.21.15.29:0/3131517173 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603000bee0 con 0x7f60381560c0 2024-01-24T13:58:32.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.911+0000 7f603d850700 1 -- 172.21.15.29:0/3131517173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60381560c0 msgr2=0x7f60380c5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.911+0000 7f603d850700 1 --2- 172.21.15.29:0/3131517173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60381560c0 0x7f60380c5000 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f6030009e90 tx=0x7f6030009780).stop 2024-01-24T13:58:32.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.911+0000 7f603d850700 1 -- 172.21.15.29:0/3131517173 shutdown_connections 2024-01-24T13:58:32.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.911+0000 7f603d850700 1 --2- 172.21.15.29:0/3131517173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60381560c0 0x7f60380c5000 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.911+0000 7f603d850700 1 -- 172.21.15.29:0/3131517173 >> 172.21.15.29:0/3131517173 conn(0x7f60380c8bb0 msgr2=0x7f60380c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:32.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.911+0000 7f603d850700 1 -- 172.21.15.29:0/3131517173 shutdown_connections 2024-01-24T13:58:32.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.911+0000 7f603d850700 1 -- 172.21.15.29:0/3131517173 wait complete. 2024-01-24T13:58:32.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.912+0000 7f603d850700 1 Processor -- start 2024-01-24T13:58:32.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.912+0000 7f603d850700 1 -- start start 2024-01-24T13:58:32.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.913+0000 7f603d850700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f603814f710 0x7f6038149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.913+0000 7f603d850700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f603814faf0 0x7f6038161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.913+0000 7f603d850700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60381560c0 0x7f60380bfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.913+0000 7f603d850700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60380c11d0 con 0x7f603814faf0 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.913+0000 7f603d850700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6038149e10 con 0x7f603814f710 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.913+0000 7f603d850700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60380c05c0 con 0x7f60381560c0 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.913+0000 7f60367fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f603814faf0 0x7f6038161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.913+0000 7f6036ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f603814f710 0x7f6038149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.913+0000 7f60367fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f603814faf0 0x7f6038161ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51422/0 (socket says 172.21.15.29:51422) 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.913+0000 7f60367fc700 1 -- 172.21.15.29:0/327031839 learned_addr learned my addr 172.21.15.29:0/327031839 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.913+0000 7f60377fe700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60381560c0 0x7f60380bfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.914+0000 7f60367fc700 1 -- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60381560c0 msgr2=0x7f60380bfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.914+0000 7f60367fc700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60381560c0 0x7f60380bfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.914+0000 7f60367fc700 1 -- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f603814f710 msgr2=0x7f6038149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.914+0000 7f60367fc700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f603814f710 0x7f6038149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.914+0000 7f60367fc700 1 -- 172.21.15.29:0/327031839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6030009a20 con 0x7f603814faf0 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.914+0000 7f6036ffd700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f603814f710 0x7f6038149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.914+0000 7f60367fc700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f603814faf0 0x7f6038161ac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f6028001ed0 tx=0x7f6028007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.914+0000 7f602effd700 1 -- 172.21.15.29:0/327031839 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800ce10 con 0x7f603814faf0 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.914+0000 7f602effd700 1 -- 172.21.15.29:0/327031839 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60280072e0 con 0x7f603814faf0 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.914+0000 7f603d850700 1 -- 172.21.15.29:0/327031839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60380c08a0 con 0x7f603814faf0 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.914+0000 7f602effd700 1 -- 172.21.15.29:0/327031839 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60280057a0 con 0x7f603814faf0 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.914+0000 7f603d850700 1 -- 172.21.15.29:0/327031839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60380c0e60 con 0x7f603814faf0 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.915+0000 7f603d850700 1 -- 172.21.15.29:0/327031839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60380c1380 con 0x7f603814faf0 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.916+0000 7f602effd700 1 -- 172.21.15.29:0/327031839 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6028005b50 con 0x7f603814faf0 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.916+0000 7f602effd700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f601805f150 0x7f6018061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.916+0000 7f602effd700 1 -- 172.21.15.29:0/327031839 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f6028079aa0 con 0x7f603814faf0 2024-01-24T13:58:32.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.916+0000 7f6036ffd700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f601805f150 0x7f6018061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:32.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.917+0000 7f6036ffd700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f601805f150 0x7f6018061610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f603814c8d0 tx=0x7f6020009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:32.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:32.920+0000 7f602effd700 1 -- 172.21.15.29:0/327031839 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f60380c1380 con 0x7f603814faf0 2024-01-24T13:58:33.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.151+0000 7f603d850700 1 -- 172.21.15.29:0/327031839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f60380475e0 con 0x7f603814faf0 2024-01-24T13:58:33.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.154+0000 7f602effd700 1 -- 172.21.15.29:0/327031839 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+148952 (secure 0 0 0) 0x7f602804aa60 con 0x7f603814faf0 2024-01-24T13:58:33.156 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:33.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.157+0000 7f603d850700 1 -- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f601805f150 msgr2=0x7f6018061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.157+0000 7f603d850700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f601805f150 0x7f6018061610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f603814c8d0 tx=0x7f6020009040).stop 2024-01-24T13:58:33.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.157+0000 7f603d850700 1 -- 172.21.15.29:0/327031839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f603814faf0 msgr2=0x7f6038161ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.157+0000 7f603d850700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f603814faf0 0x7f6038161ac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f6028001ed0 tx=0x7f6028007c70).stop 2024-01-24T13:58:33.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.157+0000 7f603d850700 1 -- 172.21.15.29:0/327031839 shutdown_connections 2024-01-24T13:58:33.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.157+0000 7f603d850700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60381560c0 0x7f60380bfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.157+0000 7f603d850700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f603814f710 0x7f6038149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.157+0000 7f603d850700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f601805f150 0x7f6018061610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.157+0000 7f603d850700 1 --2- 172.21.15.29:0/327031839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f603814faf0 0x7f6038161ac0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.157+0000 7f603d850700 1 -- 172.21.15.29:0/327031839 >> 172.21.15.29:0/327031839 conn(0x7f60380c8bb0 msgr2=0x7f603814e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:33.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.157+0000 7f603d850700 1 -- 172.21.15.29:0/327031839 shutdown_connections 2024-01-24T13:58:33.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.157+0000 7f603d850700 1 -- 172.21.15.29:0/327031839 wait complete. 2024-01-24T13:58:33.170 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":4,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":43241},{"type":"v1","addr":"172.21.15.163:6833","nonce":43241}]},"active_addr":"172.21.15.163:6833/43241","active_change":"2024-01-24T13:58:05.713147+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":233457199}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":46643247}]}]} 2024-01-24T13:58:33.174 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-24T13:58:33.174 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-24T13:58:33.174 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:33.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.304+0000 7f158b760700 1 Processor -- start 2024-01-24T13:58:33.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.305+0000 7f158b760700 1 -- start start 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.305+0000 7f158b760700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1584062970 0x7f1584062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.305+0000 7f158b760700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1584158600 0x7f1584154d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.305+0000 7f158b760700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15840bcf60 con 0x7f1584066b40 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.305+0000 7f158b760700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15840bdba0 con 0x7f1584158600 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.305+0000 7f158b760700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15840a7480 con 0x7f1584062970 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.306+0000 7f1589cfd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1584158600 0x7f1584154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.306+0000 7f1588cfb700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1584066b40 0x7f15840c4da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33740/0 (socket says 172.21.15.29:33740) 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.306+0000 7f1588cfb700 1 -- 172.21.15.29:0/620832936 learned_addr learned my addr 172.21.15.29:0/620832936 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.306+0000 7f15894fc700 1 --2- 172.21.15.29:0/620832936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1584062970 0x7f1584062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.306+0000 7f1589cfd700 1 -- 172.21.15.29:0/620832936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1584062970 msgr2=0x7f1584062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.306+0000 7f1589cfd700 1 --2- 172.21.15.29:0/620832936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1584062970 0x7f1584062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.306+0000 7f1589cfd700 1 -- 172.21.15.29:0/620832936 >> v1:172.21.15.29:6789/0 conn(0x7f1584066b40 legacy=0x7f15840c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.306+0000 7f1589cfd700 1 -- 172.21.15.29:0/620832936 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15840a6ec0 con 0x7f1584158600 2024-01-24T13:58:33.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.306+0000 7f1589cfd700 1 --2- 172.21.15.29:0/620832936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1584158600 0x7f1584154d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f157c009e60 tx=0x7f157c009780).ready entity=mon.0 client_cookie=30e44f23767ec036 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:33.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.306+0000 7f157bfff700 1 -- 172.21.15.29:0/620832936 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c00e070 con 0x7f1584158600 2024-01-24T13:58:33.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.306+0000 7f157bfff700 1 -- 172.21.15.29:0/620832936 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c00bb60 con 0x7f1584158600 2024-01-24T13:58:33.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.306+0000 7f157bfff700 1 -- 172.21.15.29:0/620832936 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c012410 con 0x7f1584158600 2024-01-24T13:58:33.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.307+0000 7f158b760700 1 -- 172.21.15.29:0/620832936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1584158600 msgr2=0x7f1584154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.307+0000 7f158b760700 1 --2- 172.21.15.29:0/620832936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1584158600 0x7f1584154d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f157c009e60 tx=0x7f157c009780).stop 2024-01-24T13:58:33.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.307+0000 7f158b760700 1 -- 172.21.15.29:0/620832936 shutdown_connections 2024-01-24T13:58:33.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.307+0000 7f158b760700 1 --2- 172.21.15.29:0/620832936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1584158600 0x7f1584154d90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.307+0000 7f158b760700 1 --2- 172.21.15.29:0/620832936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1584062970 0x7f1584062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.307+0000 7f158b760700 1 -- 172.21.15.29:0/620832936 >> 172.21.15.29:0/620832936 conn(0x7f15840c8580 msgr2=0x7f15840c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:33.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.307+0000 7f158b760700 1 -- 172.21.15.29:0/620832936 shutdown_connections 2024-01-24T13:58:33.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.307+0000 7f158b760700 1 -- 172.21.15.29:0/620832936 wait complete. 2024-01-24T13:58:33.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.308+0000 7f158b760700 1 Processor -- start 2024-01-24T13:58:33.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.308+0000 7f158b760700 1 -- start start 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.309+0000 7f158b760700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1584062970 0x7f158414c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.309+0000 7f158b760700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1584066b40 0x7f15840abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.309+0000 7f158b760700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1584158600 0x7f15840ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.309+0000 7f158b760700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15840aa640 con 0x7f1584158600 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.309+0000 7f158b760700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158414ce90 con 0x7f1584062970 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.309+0000 7f158b760700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15840bf660 con 0x7f1584066b40 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.309+0000 7f1589cfd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1584158600 0x7f15840ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.309+0000 7f1589cfd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1584158600 0x7f15840ac2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51452/0 (socket says 172.21.15.29:51452) 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.309+0000 7f1589cfd700 1 -- 172.21.15.29:0/170757647 learned_addr learned my addr 172.21.15.29:0/170757647 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f1588cfb700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1584066b40 0x7f15840abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f15894fc700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1584062970 0x7f158414c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f1589cfd700 1 -- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1584066b40 msgr2=0x7f15840abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f1589cfd700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1584066b40 0x7f15840abdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f1589cfd700 1 -- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1584062970 msgr2=0x7f158414c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f1589cfd700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1584062970 0x7f158414c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f1589cfd700 1 -- 172.21.15.29:0/170757647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c009a20 con 0x7f1584158600 2024-01-24T13:58:33.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f15894fc700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1584062970 0x7f158414c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:33.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f1589cfd700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1584158600 0x7f15840ac2b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f157c00c000 tx=0x7f157c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:33.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f15797fa700 1 -- 172.21.15.29:0/170757647 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c021070 con 0x7f1584158600 2024-01-24T13:58:33.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f15797fa700 1 -- 172.21.15.29:0/170757647 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c01d700 con 0x7f1584158600 2024-01-24T13:58:33.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f15797fa700 1 -- 172.21.15.29:0/170757647 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c01dac0 con 0x7f1584158600 2024-01-24T13:58:33.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f158b760700 1 -- 172.21.15.29:0/170757647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15840bed30 con 0x7f1584158600 2024-01-24T13:58:33.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.310+0000 7f158b760700 1 -- 172.21.15.29:0/170757647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15840bf2f0 con 0x7f1584158600 2024-01-24T13:58:33.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.311+0000 7f158b760700 1 -- 172.21.15.29:0/170757647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15840bf7f0 con 0x7f1584158600 2024-01-24T13:58:33.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.312+0000 7f15797fa700 1 -- 172.21.15.29:0/170757647 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f157c003040 con 0x7f1584158600 2024-01-24T13:58:33.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.312+0000 7f15797fa700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f156805f150 0x7f1568061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.312+0000 7f15797fa700 1 -- 172.21.15.29:0/170757647 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f157c07c440 con 0x7f1584158600 2024-01-24T13:58:33.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.312+0000 7f15894fc700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f156805f150 0x7f1568061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.313+0000 7f15894fc700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f156805f150 0x7f1568061610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f1580006d80 tx=0x7f1580008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:33.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.316+0000 7f15797fa700 1 -- 172.21.15.29:0/170757647 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f157c04d400 con 0x7f1584158600 2024-01-24T13:58:33.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.497+0000 7f158b760700 1 -- 172.21.15.29:0/170757647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15840475e0 con 0x7f1584158600 2024-01-24T13:58:33.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.498+0000 7f15797fa700 1 -- 172.21.15.29:0/170757647 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7f157c050a20 con 0x7f1584158600 2024-01-24T13:58:33.499 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:33.499 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:33.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.500+0000 7f158b760700 1 -- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f156805f150 msgr2=0x7f1568061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.500+0000 7f158b760700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f156805f150 0x7f1568061610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f1580006d80 tx=0x7f1580008040).stop 2024-01-24T13:58:33.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.500+0000 7f158b760700 1 -- 172.21.15.29:0/170757647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1584158600 msgr2=0x7f15840ac2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.500+0000 7f158b760700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1584158600 0x7f15840ac2b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f157c00c000 tx=0x7f157c009780).stop 2024-01-24T13:58:33.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.500+0000 7f158b760700 1 -- 172.21.15.29:0/170757647 shutdown_connections 2024-01-24T13:58:33.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.500+0000 7f158b760700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f156805f150 0x7f1568061610 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.500+0000 7f158b760700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1584158600 0x7f15840ac2b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.500+0000 7f158b760700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1584062970 0x7f158414c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.500+0000 7f158b760700 1 --2- 172.21.15.29:0/170757647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1584066b40 0x7f15840abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.500+0000 7f158b760700 1 -- 172.21.15.29:0/170757647 >> 172.21.15.29:0/170757647 conn(0x7f15840c8580 msgr2=0x7f15840a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:33.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.500+0000 7f158b760700 1 -- 172.21.15.29:0/170757647 shutdown_connections 2024-01-24T13:58:33.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.500+0000 7f158b760700 1 -- 172.21.15.29:0/170757647 wait complete. 2024-01-24T13:58:33.514 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-24T13:58:33.514 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:33.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.643+0000 7f285ce42700 1 Processor -- start 2024-01-24T13:58:33.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.643+0000 7f285ce42700 1 -- start start 2024-01-24T13:58:33.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.644+0000 7f285ce42700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f285814df80 0x7f285814e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285ce42700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28580bcfd0 0x7f28580c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285ce42700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28580c29e0 con 0x7f28580c2040 2024-01-24T13:58:33.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285ce42700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28580bd410 con 0x7f285814df80 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285ce42700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2858062840 con 0x7f28580bcfd0 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f285814df80 0x7f285814e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f2855d9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f28580c2040 0x7f28580bc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42082/0 (socket says 172.21.15.29:42082) 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f2855d9b700 1 -- 172.21.15.29:0/1074901682 learned_addr learned my addr 172.21.15.29:0/1074901682 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f2856d9d700 1 --2- 172.21.15.29:0/1074901682 >> v2:172.21.15.163:3300/0 conn(0x7f28580bcfd0 0x7f28580c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285659c700 1 -- 172.21.15.29:0/1074901682 >> v2:172.21.15.163:3300/0 conn(0x7f28580bcfd0 msgr2=0x7f28580c24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285659c700 1 --2- 172.21.15.29:0/1074901682 >> v2:172.21.15.163:3300/0 conn(0x7f28580bcfd0 0x7f28580c24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285659c700 1 -- 172.21.15.29:0/1074901682 >> v1:172.21.15.29:6789/0 conn(0x7f28580c2040 legacy=0x7f28580bc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285559a700 1 -- 172.21.15.29:0/1074901682 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139474223 0 0) 0x7f2840001110 con 0x7f28580c2040 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285659c700 1 -- 172.21.15.29:0/1074901682 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28580bd630 con 0x7f285814df80 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285659c700 1 --2- 172.21.15.29:0/1074901682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285814df80 0x7f285814e360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f2858068520 tx=0x7f284800a200).ready entity=mon.0 client_cookie=70db830ff386cea2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285559a700 1 -- 172.21.15.29:0/1074901682 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284800ae70 con 0x7f285814df80 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285559a700 1 -- 172.21.15.29:0/1074901682 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284800c970 con 0x7f285814df80 2024-01-24T13:58:33.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.645+0000 7f285559a700 1 -- 172.21.15.29:0/1074901682 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284800ccf0 con 0x7f285814df80 2024-01-24T13:58:33.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.646+0000 7f285ce42700 1 -- 172.21.15.29:0/1074901682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285814df80 msgr2=0x7f285814e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.646+0000 7f285ce42700 1 --2- 172.21.15.29:0/1074901682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285814df80 0x7f285814e360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f2858068520 tx=0x7f284800a200).stop 2024-01-24T13:58:33.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.646+0000 7f285ce42700 1 -- 172.21.15.29:0/1074901682 shutdown_connections 2024-01-24T13:58:33.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.646+0000 7f285ce42700 1 --2- 172.21.15.29:0/1074901682 >> v2:172.21.15.163:3300/0 conn(0x7f28580bcfd0 0x7f28580c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.646+0000 7f285ce42700 1 --2- 172.21.15.29:0/1074901682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285814df80 0x7f285814e360 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.646+0000 7f285ce42700 1 -- 172.21.15.29:0/1074901682 >> 172.21.15.29:0/1074901682 conn(0x7f2858064aa0 msgr2=0x7f2858064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:33.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.646+0000 7f285ce42700 1 -- 172.21.15.29:0/1074901682 shutdown_connections 2024-01-24T13:58:33.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.646+0000 7f285ce42700 1 -- 172.21.15.29:0/1074901682 wait complete. 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.648+0000 7f285ce42700 1 Processor -- start 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.648+0000 7f285ce42700 1 -- start start 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f285ce42700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28580bcfd0 0x7f285814b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f285ce42700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28580c2040 0x7f28580c09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f285ce42700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f285814df80 0x7f285813ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f285ce42700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2858166ac0 con 0x7f28580c2040 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f285ce42700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2858166630 con 0x7f28580bcfd0 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f285ce42700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2858165d00 con 0x7f285814df80 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f2855d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28580c2040 0x7f28580c09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f2855d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28580c2040 0x7f28580c09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41160/0 (socket says 172.21.15.29:41160) 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f2855d9b700 1 -- 172.21.15.29:0/524807440 learned_addr learned my addr 172.21.15.29:0/524807440 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f2856d9d700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f285814df80 0x7f285813ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f285659c700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28580bcfd0 0x7f285814b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f2855d9b700 1 -- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f285814df80 msgr2=0x7f285813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f2855d9b700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f285814df80 0x7f285813ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f2855d9b700 1 -- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28580bcfd0 msgr2=0x7f285814b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f2855d9b700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28580bcfd0 0x7f285814b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f2855d9b700 1 -- 172.21.15.29:0/524807440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284800a440 con 0x7f28580c2040 2024-01-24T13:58:33.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f285659c700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28580bcfd0 0x7f285814b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:33.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.649+0000 7f2855d9b700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28580c2040 0x7f28580c09f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f284000c510 tx=0x7f2840007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:33.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.650+0000 7f284e7fc700 1 -- 172.21.15.29:0/524807440 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28400072e0 con 0x7f28580c2040 2024-01-24T13:58:33.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.650+0000 7f284e7fc700 1 -- 172.21.15.29:0/524807440 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2840007440 con 0x7f28580c2040 2024-01-24T13:58:33.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.650+0000 7f285ce42700 1 -- 172.21.15.29:0/524807440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2858165fe0 con 0x7f28580c2040 2024-01-24T13:58:33.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.650+0000 7f284e7fc700 1 -- 172.21.15.29:0/524807440 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28400057a0 con 0x7f28580c2040 2024-01-24T13:58:33.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.650+0000 7f285ce42700 1 -- 172.21.15.29:0/524807440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28580c0ee0 con 0x7f28580c2040 2024-01-24T13:58:33.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.651+0000 7f285ce42700 1 -- 172.21.15.29:0/524807440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28580c34b0 con 0x7f28580c2040 2024-01-24T13:58:33.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.651+0000 7f284e7fc700 1 -- 172.21.15.29:0/524807440 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2840005b90 con 0x7f28580c2040 2024-01-24T13:58:33.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.651+0000 7f284e7fc700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f283805f150 0x7f2838061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.651+0000 7f284e7fc700 1 -- 172.21.15.29:0/524807440 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f2840079c40 con 0x7f28580c2040 2024-01-24T13:58:33.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.652+0000 7f285659c700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f283805f150 0x7f2838061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.652+0000 7f285659c700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f283805f150 0x7f2838061610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f284800afd0 tx=0x7f284800a200).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:33.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.656+0000 7f284e7fc700 1 -- 172.21.15.29:0/524807440 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f284004ac00 con 0x7f28580c2040 2024-01-24T13:58:33.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.839+0000 7f285ce42700 1 -- 172.21.15.29:0/524807440 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28580475e0 con 0x7f28580c2040 2024-01-24T13:58:33.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.840+0000 7f284e7fc700 1 -- 172.21.15.29:0/524807440 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7f2840018030 con 0x7f28580c2040 2024-01-24T13:58:33.840 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:33.840 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:33.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.841+0000 7f285ce42700 1 -- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f283805f150 msgr2=0x7f2838061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.842+0000 7f285ce42700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f283805f150 0x7f2838061610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f284800afd0 tx=0x7f284800a200).stop 2024-01-24T13:58:33.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.842+0000 7f285ce42700 1 -- 172.21.15.29:0/524807440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28580c2040 msgr2=0x7f28580c09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.842+0000 7f285ce42700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28580c2040 0x7f28580c09f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f284000c510 tx=0x7f2840007ad0).stop 2024-01-24T13:58:33.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.842+0000 7f285ce42700 1 -- 172.21.15.29:0/524807440 shutdown_connections 2024-01-24T13:58:33.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.842+0000 7f285ce42700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f285814df80 0x7f285813ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.842+0000 7f285ce42700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28580bcfd0 0x7f285814b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.842+0000 7f285ce42700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f283805f150 0x7f2838061610 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.842+0000 7f285ce42700 1 --2- 172.21.15.29:0/524807440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28580c2040 0x7f28580c09f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.842+0000 7f285ce42700 1 -- 172.21.15.29:0/524807440 >> 172.21.15.29:0/524807440 conn(0x7f2858064aa0 msgr2=0x7f28580c5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:33.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.842+0000 7f285ce42700 1 -- 172.21.15.29:0/524807440 shutdown_connections 2024-01-24T13:58:33.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.842+0000 7f285ce42700 1 -- 172.21.15.29:0/524807440 wait complete. 2024-01-24T13:58:33.856 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-24T13:58:33.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.986+0000 7fc6e4417700 1 Processor -- start 2024-01-24T13:58:33.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.986+0000 7fc6e4417700 1 -- start start 2024-01-24T13:58:33.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.987+0000 7fc6e4417700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc6dc069240 0x7fc6dc150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.987+0000 7fc6e4417700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc06a350 con 0x7fc6dc1514a0 2024-01-24T13:58:33.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.987+0000 7fc6e4417700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc069740 con 0x7fc6dc150bd0 2024-01-24T13:58:33.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.987+0000 7fc6e4417700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc069a20 con 0x7fc6dc069240 2024-01-24T13:58:33.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.988+0000 7fc6e29b4700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc6dc1514a0 0x7fc6dc0bcd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42088/0 (socket says 172.21.15.29:42088) 2024-01-24T13:58:33.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.988+0000 7fc6e29b4700 1 -- 172.21.15.29:0/3874878524 learned_addr learned my addr 172.21.15.29:0/3874878524 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:33.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.988+0000 7fc6e19b2700 1 --2- 172.21.15.29:0/3874878524 >> v2:172.21.15.163:3300/0 conn(0x7fc6dc069240 0x7fc6dc150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.988+0000 7fc6e11b1700 1 -- 172.21.15.29:0/3874878524 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250557231 0 0) 0x7fc6dc06a350 con 0x7fc6dc1514a0 2024-01-24T13:58:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.988+0000 7fc6e11b1700 1 -- 172.21.15.29:0/3874878524 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6c80033f0 con 0x7fc6dc1514a0 2024-01-24T13:58:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.988+0000 7fc6e11b1700 1 -- 172.21.15.29:0/3874878524 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513146484 0 0) 0x7fc6dc069740 con 0x7fc6dc150bd0 2024-01-24T13:58:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.988+0000 7fc6e11b1700 1 -- 172.21.15.29:0/3874878524 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6dc06a350 con 0x7fc6dc150bd0 2024-01-24T13:58:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.988+0000 7fc6e11b1700 1 -- 172.21.15.29:0/3874878524 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3502003272 0 0) 0x7fc6c80033f0 con 0x7fc6dc1514a0 2024-01-24T13:58:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.988+0000 7fc6e19b2700 1 -- 172.21.15.29:0/3874878524 >> v1:172.21.15.163:6789/0 conn(0x7fc6dc150bd0 legacy=0x7fc6dc065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.988+0000 7fc6e19b2700 1 -- 172.21.15.29:0/3874878524 >> v1:172.21.15.29:6789/0 conn(0x7fc6dc1514a0 legacy=0x7fc6dc0bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.988+0000 7fc6e19b2700 1 -- 172.21.15.29:0/3874878524 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc069d00 con 0x7fc6dc069240 2024-01-24T13:58:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.988+0000 7fc6e19b2700 1 --2- 172.21.15.29:0/3874878524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6dc069240 0x7fc6dc150fb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fc6d8009760 tx=0x7fc6d8009060).ready entity=mon.1 client_cookie=1a640ba752e271a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.989+0000 7fc6e11b1700 1 -- 172.21.15.29:0/3874878524 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d40017f0 con 0x7fc6dc069240 2024-01-24T13:58:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.989+0000 7fc6e11b1700 1 -- 172.21.15.29:0/3874878524 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d8011400 con 0x7fc6dc069240 2024-01-24T13:58:33.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.989+0000 7fc6e11b1700 1 -- 172.21.15.29:0/3874878524 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8011780 con 0x7fc6dc069240 2024-01-24T13:58:33.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.989+0000 7fc6e4417700 1 -- 172.21.15.29:0/3874878524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6dc069240 msgr2=0x7fc6dc150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.989+0000 7fc6e4417700 1 --2- 172.21.15.29:0/3874878524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6dc069240 0x7fc6dc150fb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fc6d8009760 tx=0x7fc6d8009060).stop 2024-01-24T13:58:33.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.989+0000 7fc6e4417700 1 -- 172.21.15.29:0/3874878524 shutdown_connections 2024-01-24T13:58:33.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.989+0000 7fc6e4417700 1 --2- 172.21.15.29:0/3874878524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6dc069240 0x7fc6dc150fb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.989+0000 7fc6e4417700 1 -- 172.21.15.29:0/3874878524 >> 172.21.15.29:0/3874878524 conn(0x7fc6dc0c8580 msgr2=0x7fc6dc0c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:33.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.989+0000 7fc6e4417700 1 -- 172.21.15.29:0/3874878524 shutdown_connections 2024-01-24T13:58:33.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.989+0000 7fc6e4417700 1 -- 172.21.15.29:0/3874878524 wait complete. 2024-01-24T13:58:33.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.991+0000 7fc6e4417700 1 Processor -- start 2024-01-24T13:58:33.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.991+0000 7fc6e4417700 1 -- start start 2024-01-24T13:58:33.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.991+0000 7fc6e4417700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6dc069240 0x7fc6dc13b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.991+0000 7fc6e4417700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6dc150bd0 0x7fc6dc0aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.991+0000 7fc6e4417700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6dc1514a0 0x7fc6dc0ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.991+0000 7fc6e4417700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0ac830 con 0x7fc6dc069240 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.991+0000 7fc6e4417700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0acdb0 con 0x7fc6dc1514a0 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.991+0000 7fc6e4417700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc13b780 con 0x7fc6dc150bd0 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e21b3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6dc069240 0x7fc6dc13b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e21b3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6dc069240 0x7fc6dc13b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41166/0 (socket says 172.21.15.29:41166) 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e21b3700 1 -- 172.21.15.29:0/2461052774 learned_addr learned my addr 172.21.15.29:0/2461052774 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e19b2700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6dc150bd0 0x7fc6dc0aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e29b4700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6dc1514a0 0x7fc6dc0ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e21b3700 1 -- 172.21.15.29:0/2461052774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6dc150bd0 msgr2=0x7fc6dc0aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e21b3700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6dc150bd0 0x7fc6dc0aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e21b3700 1 -- 172.21.15.29:0/2461052774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6dc1514a0 msgr2=0x7fc6dc0ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e21b3700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6dc1514a0 0x7fc6dc0ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e21b3700 1 -- 172.21.15.29:0/2461052774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80092a0 con 0x7fc6dc069240 2024-01-24T13:58:33.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e21b3700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6dc069240 0x7fc6dc13b290 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fc6d000c490 tx=0x7fc6d0007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6ce7fc700 1 -- 172.21.15.29:0/2461052774 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d000f470 con 0x7fc6dc069240 2024-01-24T13:58:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6ce7fc700 1 -- 172.21.15.29:0/2461052774 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d000f5d0 con 0x7fc6dc069240 2024-01-24T13:58:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e4417700 1 -- 172.21.15.29:0/2461052774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc14d880 con 0x7fc6dc069240 2024-01-24T13:58:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6ce7fc700 1 -- 172.21.15.29:0/2461052774 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d000f970 con 0x7fc6dc069240 2024-01-24T13:58:33.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.992+0000 7fc6e4417700 1 -- 172.21.15.29:0/2461052774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc0aca40 con 0x7fc6dc069240 2024-01-24T13:58:33.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.993+0000 7fc6e4417700 1 -- 172.21.15.29:0/2461052774 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc6dc0475e0 con 0x7fc6dc069240 2024-01-24T13:58:33.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.994+0000 7fc6ce7fc700 1 -- 172.21.15.29:0/2461052774 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc6d000fd20 con 0x7fc6dc069240 2024-01-24T13:58:33.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.994+0000 7fc6ce7fc700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc6c005f120 0x7fc6c00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.994+0000 7fc6ce7fc700 1 -- 172.21.15.29:0/2461052774 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fc6d0079bd0 con 0x7fc6dc069240 2024-01-24T13:58:33.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.995+0000 7fc6ce7fc700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] conn(0x7fc6c00698f0 0x7fc6c006bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:33.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.995+0000 7fc6ce7fc700 1 -- 172.21.15.29:0/2461052774 --> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6c006c410 con 0x7fc6c00698f0 2024-01-24T13:58:33.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.995+0000 7fc6ce7fc700 1 -- 172.21.15.29:0/2461052774 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6d007c1f0 con 0x7fc6dc069240 2024-01-24T13:58:33.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.995+0000 7fc6e19b2700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc6c005f120 0x7fc6c00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.995+0000 7fc6e29b4700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] conn(0x7fc6c00698f0 0x7fc6c006bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:33.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.995+0000 7fc6e29b4700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] conn(0x7fc6c00698f0 0x7fc6c006bd90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:33.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.995+0000 7fc6e19b2700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc6c005f120 0x7fc6c00615e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fc6d8009760 tx=0x7fc6d8016000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:33.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:33.996+0000 7fc6ce7fc700 1 -- 172.21.15.29:0/2461052774 <== osd.0 v2:172.21.15.29:6824/43714 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc6c006c410 con 0x7fc6c00698f0 2024-01-24T13:58:34.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.011+0000 7fc6e4417700 1 -- 172.21.15.29:0/2461052774 --> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc6dc0ad090 con 0x7fc6c00698f0 2024-01-24T13:58:34.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.011+0000 7fc6ce7fc700 1 -- 172.21.15.29:0/2461052774 <== osd.0 v2:172.21.15.29:6824/43714 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc6dc0ad090 con 0x7fc6c00698f0 2024-01-24T13:58:34.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.012+0000 7fc6e4417700 1 -- 172.21.15.29:0/2461052774 >> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] conn(0x7fc6c00698f0 msgr2=0x7fc6c006bd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.012+0000 7fc6e4417700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] conn(0x7fc6c00698f0 0x7fc6c006bd90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.012+0000 7fc6e4417700 1 -- 172.21.15.29:0/2461052774 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc6c005f120 msgr2=0x7fc6c00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.012+0000 7fc6e4417700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc6c005f120 0x7fc6c00615e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fc6d8009760 tx=0x7fc6d8016000).stop 2024-01-24T13:58:34.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.012+0000 7fc6e4417700 1 -- 172.21.15.29:0/2461052774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6dc069240 msgr2=0x7fc6dc13b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.012+0000 7fc6e4417700 1 --2- 172.21.15.29:0/2461052774 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6dc069240 0x7fc6dc13b290 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fc6d000c490 tx=0x7fc6d0007c90).stop 2024-01-24T13:58:34.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.012+0000 7fc6e29b4700 1 -- 172.21.15.29:0/2461052774 reap_dead start 2024-01-24T13:58:34.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.012+0000 7fc6e4417700 1 -- 172.21.15.29:0/2461052774 shutdown_connections 2024-01-24T13:58:34.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.012+0000 7fc6e4417700 1 -- 172.21.15.29:0/2461052774 >> 172.21.15.29:0/2461052774 conn(0x7fc6dc0c8580 msgr2=0x7fc6dc0627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:34.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.012+0000 7fc6e4417700 1 -- 172.21.15.29:0/2461052774 shutdown_connections 2024-01-24T13:58:34.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.012+0000 7fc6e4417700 1 -- 172.21.15.29:0/2461052774 wait complete. 2024-01-24T13:58:34.025 INFO:teuthology.orchestra.run.smithi029.stdout:60129542152 2024-01-24T13:58:34.025 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-24T13:58:34.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.156+0000 7f1c9e31a700 1 Processor -- start 2024-01-24T13:58:34.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.156+0000 7f1c9e31a700 1 -- start start 2024-01-24T13:58:34.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c9e31a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1c98151aa0 0x7f1c980a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c9e31a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c98065360 con 0x7f1c98151e80 2024-01-24T13:58:34.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c9e31a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c980bcab0 con 0x7f1c98062ec0 2024-01-24T13:58:34.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c9e31a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c980bcd90 con 0x7f1c98151aa0 2024-01-24T13:58:34.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c977fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1c98151e80 0x7f1c980627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42102/0 (socket says 172.21.15.29:42102) 2024-01-24T13:58:34.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c977fe700 1 -- 172.21.15.29:0/1556943055 learned_addr learned my addr 172.21.15.29:0/1556943055 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:34.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c97fff700 1 --2- 172.21.15.29:0/1556943055 >> v2:172.21.15.163:3300/0 conn(0x7f1c98151aa0 0x7f1c980a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500556451 0 0) 0x7f1c98065360 con 0x7f1c98151e80 2024-01-24T13:58:34.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c800033f0 con 0x7f1c98151e80 2024-01-24T13:58:34.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2575406219 0 0) 0x7f1c800033f0 con 0x7f1c98151e80 2024-01-24T13:58:34.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 >> v2:172.21.15.163:3300/0 conn(0x7f1c98151aa0 msgr2=0x7f1c980a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:34.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c96ffd700 1 --2- 172.21.15.29:0/1556943055 >> v2:172.21.15.163:3300/0 conn(0x7f1c98151aa0 0x7f1c980a6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 >> v1:172.21.15.163:6789/0 conn(0x7f1c98062ec0 legacy=0x7f1c98069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c980bd070 con 0x7f1c98151e80 2024-01-24T13:58:34.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f1c8c001470 con 0x7f1c98151e80 2024-01-24T13:58:34.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.157+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 >> v1:172.21.15.29:6789/0 conn(0x7f1c98151e80 legacy=0x7f1c980627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c96ffd700 1 --2- 172.21.15.29:0/1556943055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c800038e0 0x7f1c80003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c96ffd700 1 --2- 172.21.15.29:0/1556943055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c80004690 0x7f1c80004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c96ffd700 1 --2- 172.21.15.29:0/1556943055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c80005000 0x7f1c8000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980bcd90 con 0x7f1c80005000 2024-01-24T13:58:34.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8000beb0 con 0x7f1c800038e0 2024-01-24T13:58:34.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8000c070 con 0x7f1c80004690 2024-01-24T13:58:34.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c97fff700 1 --2- 172.21.15.29:0/1556943055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c800038e0 0x7f1c80003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c9c8b7700 1 --2- 172.21.15.29:0/1556943055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c80005000 0x7f1c8000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c977fe700 1 --2- 172.21.15.29:0/1556943055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c80004690 0x7f1c80004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c97fff700 1 -- 172.21.15.29:0/1556943055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c80004690 msgr2=0x7f1c80004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:34.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c97fff700 1 --2- 172.21.15.29:0/1556943055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c80004690 0x7f1c80004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c97fff700 1 -- 172.21.15.29:0/1556943055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c80005000 msgr2=0x7f1c8000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:34.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c97fff700 1 --2- 172.21.15.29:0/1556943055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c80005000 0x7f1c8000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c97fff700 1 -- 172.21.15.29:0/1556943055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c800033f0 con 0x7f1c800038e0 2024-01-24T13:58:34.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c9c8b7700 1 --2- 172.21.15.29:0/1556943055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c80005000 0x7f1c8000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:34.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c9c8b7700 1 -- 172.21.15.29:0/1556943055 reap_dead start 2024-01-24T13:58:34.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c97fff700 1 --2- 172.21.15.29:0/1556943055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c800038e0 0x7f1c80003d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1c88009a80 tx=0x7f1c88009060).ready entity=mon.1 client_cookie=8c6f31707e73379d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88009430 con 0x7f1c800038e0 2024-01-24T13:58:34.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8800e640 con 0x7f1c800038e0 2024-01-24T13:58:34.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.158+0000 7f1c96ffd700 1 -- 172.21.15.29:0/1556943055 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8800e9c0 con 0x7f1c800038e0 2024-01-24T13:58:34.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.159+0000 7f1c9e31a700 1 -- 172.21.15.29:0/1556943055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c800038e0 msgr2=0x7f1c80003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.159+0000 7f1c9e31a700 1 --2- 172.21.15.29:0/1556943055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c800038e0 0x7f1c80003d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1c88009a80 tx=0x7f1c88009060).stop 2024-01-24T13:58:34.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.159+0000 7f1c9e31a700 1 -- 172.21.15.29:0/1556943055 shutdown_connections 2024-01-24T13:58:34.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.159+0000 7f1c9e31a700 1 --2- 172.21.15.29:0/1556943055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c800038e0 0x7f1c80003d00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.159+0000 7f1c9e31a700 1 -- 172.21.15.29:0/1556943055 >> 172.21.15.29:0/1556943055 conn(0x7f1c980c8580 msgr2=0x7f1c980c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:34.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.159+0000 7f1c9e31a700 1 -- 172.21.15.29:0/1556943055 shutdown_connections 2024-01-24T13:58:34.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.159+0000 7f1c9e31a700 1 -- 172.21.15.29:0/1556943055 wait complete. 2024-01-24T13:58:34.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.160+0000 7f1c9e31a700 1 Processor -- start 2024-01-24T13:58:34.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.160+0000 7f1c9e31a700 1 -- start start 2024-01-24T13:58:34.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.161+0000 7f1c9e31a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c800038e0 0x7f1c980ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.161+0000 7f1c9e31a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c98165360 0x7f1c980a81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.161+0000 7f1c9e31a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c98150340 0x7f1c9814e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.161+0000 7f1c9e31a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980a6180 con 0x7f1c98150340 2024-01-24T13:58:34.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.161+0000 7f1c9e31a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98150d20 con 0x7f1c98165360 2024-01-24T13:58:34.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.161+0000 7f1c9e31a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980a5850 con 0x7f1c800038e0 2024-01-24T13:58:34.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.161+0000 7f1c9c8b7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c98150340 0x7f1c9814e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.161+0000 7f1c9c8b7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c98150340 0x7f1c9814e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41172/0 (socket says 172.21.15.29:41172) 2024-01-24T13:58:34.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.161+0000 7f1c9c8b7700 1 -- 172.21.15.29:0/2483653145 learned_addr learned my addr 172.21.15.29:0/2483653145 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:34.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.161+0000 7f1c97fff700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c800038e0 0x7f1c980ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.161+0000 7f1c977fe700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c98165360 0x7f1c980a81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.162+0000 7f1c9c8b7700 1 -- 172.21.15.29:0/2483653145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c800038e0 msgr2=0x7f1c980ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.162+0000 7f1c9c8b7700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c800038e0 0x7f1c980ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.162+0000 7f1c9c8b7700 1 -- 172.21.15.29:0/2483653145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c98165360 msgr2=0x7f1c980a81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.162+0000 7f1c9c8b7700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c98165360 0x7f1c980a81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.162+0000 7f1c9c8b7700 1 -- 172.21.15.29:0/2483653145 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c980a5b30 con 0x7f1c98150340 2024-01-24T13:58:34.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.162+0000 7f1c9c8b7700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c98150340 0x7f1c9814e4a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f1c98068ba0 tx=0x7f1c90009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.162+0000 7f1c7ffff700 1 -- 172.21.15.29:0/2483653145 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90011590 con 0x7f1c98150340 2024-01-24T13:58:34.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.162+0000 7f1c7ffff700 1 -- 172.21.15.29:0/2483653145 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c900116f0 con 0x7f1c98150340 2024-01-24T13:58:34.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.162+0000 7f1c9e31a700 1 -- 172.21.15.29:0/2483653145 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c980a5e10 con 0x7f1c98150340 2024-01-24T13:58:34.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.162+0000 7f1c7ffff700 1 -- 172.21.15.29:0/2483653145 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90011a90 con 0x7f1c98150340 2024-01-24T13:58:34.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.162+0000 7f1c9e31a700 1 -- 172.21.15.29:0/2483653145 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c981507c0 con 0x7f1c98150340 2024-01-24T13:58:34.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.163+0000 7f1c9e31a700 1 -- 172.21.15.29:0/2483653145 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c980475e0 con 0x7f1c98150340 2024-01-24T13:58:34.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.163+0000 7f1c7ffff700 1 -- 172.21.15.29:0/2483653145 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1c90003070 con 0x7f1c98150340 2024-01-24T13:58:34.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.164+0000 7f1c7ffff700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c8405f180 0x7f1c84061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.164+0000 7f1c7ffff700 1 -- 172.21.15.29:0/2483653145 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f1c90079fc0 con 0x7f1c98150340 2024-01-24T13:58:34.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.164+0000 7f1c7ffff700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] conn(0x7f1c840699d0 0x7f1c8406be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.164+0000 7f1c7ffff700 1 -- 172.21.15.29:0/2483653145 --> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c8406c4f0 con 0x7f1c840699d0 2024-01-24T13:58:34.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.164+0000 7f1c7ffff700 1 -- 172.21.15.29:0/2483653145 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c9007c580 con 0x7f1c98150340 2024-01-24T13:58:34.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.164+0000 7f1c97fff700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c8405f180 0x7f1c84061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.164+0000 7f1c977fe700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] conn(0x7f1c840699d0 0x7f1c8406be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.165+0000 7f1c977fe700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] conn(0x7f1c840699d0 0x7f1c8406be70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.165+0000 7f1c97fff700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c8405f180 0x7f1c84061640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f1c88006d50 tx=0x7f1c8801a000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.165+0000 7f1c7ffff700 1 -- 172.21.15.29:0/2483653145 <== osd.1 v2:172.21.15.29:6800/43696 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1c8406c4f0 con 0x7f1c840699d0 2024-01-24T13:58:34.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.180+0000 7f1c9e31a700 1 -- 172.21.15.29:0/2483653145 --> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c9806a5f0 con 0x7f1c840699d0 2024-01-24T13:58:34.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.181+0000 7f1c7ffff700 1 -- 172.21.15.29:0/2483653145 <== osd.1 v2:172.21.15.29:6800/43696 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1c9806a5f0 con 0x7f1c840699d0 2024-01-24T13:58:34.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.181+0000 7f1c9e31a700 1 -- 172.21.15.29:0/2483653145 >> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] conn(0x7f1c840699d0 msgr2=0x7f1c8406be70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.181+0000 7f1c9e31a700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] conn(0x7f1c840699d0 0x7f1c8406be70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.181+0000 7f1c9e31a700 1 -- 172.21.15.29:0/2483653145 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c8405f180 msgr2=0x7f1c84061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.181+0000 7f1c9e31a700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c8405f180 0x7f1c84061640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f1c88006d50 tx=0x7f1c8801a000).stop 2024-01-24T13:58:34.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.181+0000 7f1c9e31a700 1 -- 172.21.15.29:0/2483653145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c98150340 msgr2=0x7f1c9814e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.181+0000 7f1c9e31a700 1 --2- 172.21.15.29:0/2483653145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c98150340 0x7f1c9814e4a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f1c98068ba0 tx=0x7f1c90009040).stop 2024-01-24T13:58:34.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.181+0000 7f1c9c8b7700 1 -- 172.21.15.29:0/2483653145 reap_dead start 2024-01-24T13:58:34.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.181+0000 7f1c9e31a700 1 -- 172.21.15.29:0/2483653145 shutdown_connections 2024-01-24T13:58:34.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.181+0000 7f1c9e31a700 1 -- 172.21.15.29:0/2483653145 >> 172.21.15.29:0/2483653145 conn(0x7f1c980c8580 msgr2=0x7f1c980c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:34.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.181+0000 7f1c9e31a700 1 -- 172.21.15.29:0/2483653145 shutdown_connections 2024-01-24T13:58:34.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.181+0000 7f1c9e31a700 1 -- 172.21.15.29:0/2483653145 wait complete. 2024-01-24T13:58:34.195 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-01-24T13:58:34.195 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-24T13:58:34.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.327+0000 7f5d71854700 1 Processor -- start 2024-01-24T13:58:34.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.327+0000 7f5d71854700 1 -- start start 2024-01-24T13:58:34.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d71854700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d6c0bcbd0 0x7f5d6c0bcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d71854700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5d6c14b020 0x7f5d6c14b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d71854700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5d6c0bd630 0x7f5d6c0bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d71854700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c066470 con 0x7f5d6c14b020 2024-01-24T13:58:34.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d71854700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c0653d0 con 0x7f5d6c0bd630 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d71854700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c0656b0 con 0x7f5d6c0bcbd0 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d63fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5d6c14b020 0x7f5d6c14b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d63fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5d6c14b020 0x7f5d6c14b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41184/0 (socket says 172.21.15.29:41184) 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d63fff700 1 -- 172.21.15.29:0/3792108706 learned_addr learned my addr 172.21.15.29:0/3792108706 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d6affd700 1 --2- 172.21.15.29:0/3792108706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d6c0bcbd0 0x7f5d6c0bcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d6b7fe700 1 --2- 172.21.15.29:0/3792108706 >> v2:172.21.15.163:3300/0 conn(0x7f5d6c0bd630 0x7f5d6c0bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d63fff700 1 -- 172.21.15.29:0/3792108706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d6c0bcbd0 msgr2=0x7f5d6c0bcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d63fff700 1 --2- 172.21.15.29:0/3792108706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d6c0bcbd0 0x7f5d6c0bcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d63fff700 1 -- 172.21.15.29:0/3792108706 >> v2:172.21.15.163:3300/0 conn(0x7f5d6c0bd630 msgr2=0x7f5d6c0bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d63fff700 1 --2- 172.21.15.29:0/3792108706 >> v2:172.21.15.163:3300/0 conn(0x7f5d6c0bd630 0x7f5d6c0bdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d63fff700 1 -- 172.21.15.29:0/3792108706 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c065990 con 0x7f5d6c14b020 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.328+0000 7f5d6b7fe700 1 --2- 172.21.15.29:0/3792108706 >> v2:172.21.15.163:3300/0 conn(0x7f5d6c0bd630 0x7f5d6c0bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.329+0000 7f5d63fff700 1 --2- 172.21.15.29:0/3792108706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d6c14b020 0x7f5d6c14b4a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f5d6c0be360 tx=0x7f5d5400b3b0).ready entity=mon.0 client_cookie=1685c7fa780156aa server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.329+0000 7f5d6a7fc700 1 -- 172.21.15.29:0/3792108706 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400db10 con 0x7f5d6c14b020 2024-01-24T13:58:34.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.329+0000 7f5d6a7fc700 1 -- 172.21.15.29:0/3792108706 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5400dc70 con 0x7f5d6c14b020 2024-01-24T13:58:34.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.329+0000 7f5d6a7fc700 1 -- 172.21.15.29:0/3792108706 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54010470 con 0x7f5d6c14b020 2024-01-24T13:58:34.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.329+0000 7f5d71854700 1 -- 172.21.15.29:0/3792108706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d6c14b020 msgr2=0x7f5d6c14b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.329+0000 7f5d71854700 1 --2- 172.21.15.29:0/3792108706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d6c14b020 0x7f5d6c14b4a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f5d6c0be360 tx=0x7f5d5400b3b0).stop 2024-01-24T13:58:34.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.329+0000 7f5d71854700 1 -- 172.21.15.29:0/3792108706 shutdown_connections 2024-01-24T13:58:34.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.329+0000 7f5d71854700 1 --2- 172.21.15.29:0/3792108706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d6c0bcbd0 0x7f5d6c0bcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.329+0000 7f5d71854700 1 --2- 172.21.15.29:0/3792108706 >> v2:172.21.15.163:3300/0 conn(0x7f5d6c0bd630 0x7f5d6c0bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.329+0000 7f5d71854700 1 --2- 172.21.15.29:0/3792108706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d6c14b020 0x7f5d6c14b4a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.329+0000 7f5d71854700 1 -- 172.21.15.29:0/3792108706 >> 172.21.15.29:0/3792108706 conn(0x7f5d6c0c5020 msgr2=0x7f5d6c0c7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.330+0000 7f5d71854700 1 -- 172.21.15.29:0/3792108706 shutdown_connections 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.330+0000 7f5d71854700 1 -- 172.21.15.29:0/3792108706 wait complete. 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.331+0000 7f5d71854700 1 Processor -- start 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.331+0000 7f5d71854700 1 -- start start 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d71854700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d6c0bcbd0 0x7f5d6c0ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d71854700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d6c0bd630 0x7f5d6c138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d71854700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d6c14b020 0x7f5d6c146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d71854700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c149e60 con 0x7f5d6c0bd630 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d71854700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c149250 con 0x7f5d6c0bcbd0 2024-01-24T13:58:34.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d71854700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c149530 con 0x7f5d6c14b020 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d63fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d6c0bd630 0x7f5d6c138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d63fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d6c0bd630 0x7f5d6c138660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41196/0 (socket says 172.21.15.29:41196) 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d63fff700 1 -- 172.21.15.29:0/2576745936 learned_addr learned my addr 172.21.15.29:0/2576745936 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d6affd700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d6c0bcbd0 0x7f5d6c0ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d6b7fe700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d6c14b020 0x7f5d6c146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d63fff700 1 -- 172.21.15.29:0/2576745936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d6c14b020 msgr2=0x7f5d6c146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d63fff700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d6c14b020 0x7f5d6c146e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d63fff700 1 -- 172.21.15.29:0/2576745936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d6c0bcbd0 msgr2=0x7f5d6c0ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d63fff700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d6c0bcbd0 0x7f5d6c0ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d63fff700 1 -- 172.21.15.29:0/2576745936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5400b610 con 0x7f5d6c0bd630 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d63fff700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d6c0bd630 0x7f5d6c138660 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f5d54013000 tx=0x7f5d5400b3b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d62ffd700 1 -- 172.21.15.29:0/2576745936 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54000d10 con 0x7f5d6c0bd630 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d62ffd700 1 -- 172.21.15.29:0/2576745936 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d54019660 con 0x7f5d6c0bd630 2024-01-24T13:58:34.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d71854700 1 -- 172.21.15.29:0/2576745936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6c149810 con 0x7f5d6c0bd630 2024-01-24T13:58:34.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.332+0000 7f5d62ffd700 1 -- 172.21.15.29:0/2576745936 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d540199e0 con 0x7f5d6c0bd630 2024-01-24T13:58:34.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.333+0000 7f5d71854700 1 -- 172.21.15.29:0/2576745936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6c0ac640 con 0x7f5d6c0bd630 2024-01-24T13:58:34.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.333+0000 7f5d71854700 1 -- 172.21.15.29:0/2576745936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d6c0475e0 con 0x7f5d6c0bd630 2024-01-24T13:58:34.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.334+0000 7f5d62ffd700 1 -- 172.21.15.29:0/2576745936 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5d54023430 con 0x7f5d6c0bd630 2024-01-24T13:58:34.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.334+0000 7f5d62ffd700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5d4c05f150 0x7f5d4c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.334+0000 7f5d62ffd700 1 -- 172.21.15.29:0/2576745936 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f5d5407b9a0 con 0x7f5d6c0bd630 2024-01-24T13:58:34.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.334+0000 7f5d62ffd700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] conn(0x7f5d4c069830 0x7f5d4c06bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.334+0000 7f5d62ffd700 1 -- 172.21.15.29:0/2576745936 --> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d4c06c350 con 0x7f5d4c069830 2024-01-24T13:58:34.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.334+0000 7f5d62ffd700 1 -- 172.21.15.29:0/2576745936 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d54080220 con 0x7f5d6c0bd630 2024-01-24T13:58:34.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.334+0000 7f5d6affd700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5d4c05f150 0x7f5d4c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.335+0000 7f5d6b7fe700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] conn(0x7f5d4c069830 0x7f5d4c06bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.335+0000 7f5d6affd700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5d4c05f150 0x7f5d4c061610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f5d5c00bb20 tx=0x7f5d5c008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.335+0000 7f5d6b7fe700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] conn(0x7f5d4c069830 0x7f5d4c06bcd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.336+0000 7f5d62ffd700 1 -- 172.21.15.29:0/2576745936 <== osd.2 v2:172.21.15.29:6816/43698 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5d4c06c350 con 0x7f5d4c069830 2024-01-24T13:58:34.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.351+0000 7f5d71854700 1 -- 172.21.15.29:0/2576745936 --> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d6c154850 con 0x7f5d4c069830 2024-01-24T13:58:34.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.351+0000 7f5d62ffd700 1 -- 172.21.15.29:0/2576745936 <== osd.2 v2:172.21.15.29:6816/43698 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5d6c154850 con 0x7f5d4c069830 2024-01-24T13:58:34.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.352+0000 7f5d71854700 1 -- 172.21.15.29:0/2576745936 >> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] conn(0x7f5d4c069830 msgr2=0x7f5d4c06bcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.352+0000 7f5d71854700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] conn(0x7f5d4c069830 0x7f5d4c06bcd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.352+0000 7f5d71854700 1 -- 172.21.15.29:0/2576745936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5d4c05f150 msgr2=0x7f5d4c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.352+0000 7f5d71854700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5d4c05f150 0x7f5d4c061610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f5d5c00bb20 tx=0x7f5d5c008040).stop 2024-01-24T13:58:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.352+0000 7f5d71854700 1 -- 172.21.15.29:0/2576745936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d6c0bd630 msgr2=0x7f5d6c138660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.352+0000 7f5d71854700 1 --2- 172.21.15.29:0/2576745936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d6c0bd630 0x7f5d6c138660 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f5d54013000 tx=0x7f5d5400b3b0).stop 2024-01-24T13:58:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.352+0000 7f5d6b7fe700 1 -- 172.21.15.29:0/2576745936 reap_dead start 2024-01-24T13:58:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.352+0000 7f5d71854700 1 -- 172.21.15.29:0/2576745936 shutdown_connections 2024-01-24T13:58:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.352+0000 7f5d71854700 1 -- 172.21.15.29:0/2576745936 >> 172.21.15.29:0/2576745936 conn(0x7f5d6c0c5020 msgr2=0x7f5d6c0c6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.352+0000 7f5d71854700 1 -- 172.21.15.29:0/2576745936 shutdown_connections 2024-01-24T13:58:34.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.352+0000 7f5d71854700 1 -- 172.21.15.29:0/2576745936 wait complete. 2024-01-24T13:58:34.365 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-01-24T13:58:34.365 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-24T13:58:34.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.496+0000 7f8433180700 1 Processor -- start 2024-01-24T13:58:34.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.496+0000 7f8433180700 1 -- start start 2024-01-24T13:58:34.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.497+0000 7f8433180700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f842c069240 0x7f842c150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.497+0000 7f8433180700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842c06a350 con 0x7f842c150bd0 2024-01-24T13:58:34.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.497+0000 7f8433180700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842c069740 con 0x7f842c1514a0 2024-01-24T13:58:34.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.497+0000 7f8433180700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c069a20 con 0x7f842c069240 2024-01-24T13:58:34.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.497+0000 7f842bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f842c069240 0x7f842c150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.497+0000 7f842bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f842c069240 0x7f842c150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41202/0 (socket says 172.21.15.29:41202) 2024-01-24T13:58:34.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.497+0000 7f8430f1c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f842c150bd0 0x7f842c065e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42118/0 (socket says 172.21.15.29:42118) 2024-01-24T13:58:34.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.497+0000 7f842bfff700 1 -- 172.21.15.29:0/2687191697 learned_addr learned my addr 172.21.15.29:0/2687191697 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:34.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f842bfff700 1 -- 172.21.15.29:0/2687191697 >> v1:172.21.15.163:6789/0 conn(0x7f842c1514a0 legacy=0x7f842c0bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f842bfff700 1 -- 172.21.15.29:0/2687191697 >> v1:172.21.15.29:6789/0 conn(0x7f842c150bd0 legacy=0x7f842c065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f842b7fe700 1 -- 172.21.15.29:0/2687191697 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420223910 0 0) 0x7f842c06a350 con 0x7f842c150bd0 2024-01-24T13:58:34.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f842bfff700 1 -- 172.21.15.29:0/2687191697 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c069d00 con 0x7f842c069240 2024-01-24T13:58:34.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f842bfff700 1 --2- 172.21.15.29:0/2687191697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c069240 0x7f842c150fb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f8418006fd0 tx=0x7f84180090f0).ready entity=mon.0 client_cookie=5a0a2c01bb772189 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f842b7fe700 1 -- 172.21.15.29:0/2687191697 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800b600 con 0x7f842c069240 2024-01-24T13:58:34.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f842b7fe700 1 -- 172.21.15.29:0/2687191697 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841800b760 con 0x7f842c069240 2024-01-24T13:58:34.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f842b7fe700 1 -- 172.21.15.29:0/2687191697 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800bae0 con 0x7f842c069240 2024-01-24T13:58:34.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f8433180700 1 -- 172.21.15.29:0/2687191697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c069240 msgr2=0x7f842c150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f8433180700 1 --2- 172.21.15.29:0/2687191697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c069240 0x7f842c150fb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f8418006fd0 tx=0x7f84180090f0).stop 2024-01-24T13:58:34.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f8433180700 1 -- 172.21.15.29:0/2687191697 shutdown_connections 2024-01-24T13:58:34.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f8433180700 1 --2- 172.21.15.29:0/2687191697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c069240 0x7f842c150fb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f8433180700 1 -- 172.21.15.29:0/2687191697 >> 172.21.15.29:0/2687191697 conn(0x7f842c0c8580 msgr2=0x7f842c0c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.498+0000 7f8433180700 1 -- 172.21.15.29:0/2687191697 shutdown_connections 2024-01-24T13:58:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.499+0000 7f8433180700 1 -- 172.21.15.29:0/2687191697 wait complete. 2024-01-24T13:58:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.500+0000 7f8433180700 1 Processor -- start 2024-01-24T13:58:34.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.500+0000 7f8433180700 1 -- start start 2024-01-24T13:58:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8433180700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c069240 0x7f842c0c1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8433180700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f842c150bd0 0x7f842c0ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8433180700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f842c1514a0 0x7f842c144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8433180700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c165220 con 0x7f842c069240 2024-01-24T13:58:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8433180700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0c0d00 con 0x7f842c1514a0 2024-01-24T13:58:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8433180700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0adaa0 con 0x7f842c150bd0 2024-01-24T13:58:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8430f1c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c069240 0x7f842c0c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8430f1c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c069240 0x7f842c0c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41212/0 (socket says 172.21.15.29:41212) 2024-01-24T13:58:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8430f1c700 1 -- 172.21.15.29:0/1454755403 learned_addr learned my addr 172.21.15.29:0/1454755403 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f842bfff700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f842c150bd0 0x7f842c0ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f843171d700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f842c1514a0 0x7f842c144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8430f1c700 1 -- 172.21.15.29:0/1454755403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f842c150bd0 msgr2=0x7f842c0ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8430f1c700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f842c150bd0 0x7f842c0ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8430f1c700 1 -- 172.21.15.29:0/1454755403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f842c1514a0 msgr2=0x7f842c144680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8430f1c700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f842c1514a0 0x7f842c144680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8430f1c700 1 -- 172.21.15.29:0/1454755403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8418009370 con 0x7f842c069240 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f843171d700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f842c1514a0 0x7f842c144680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8430f1c700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c069240 0x7f842c0c1190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f842000c4c0 tx=0x7f8420007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8428ff9700 1 -- 172.21.15.29:0/1454755403 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84200072e0 con 0x7f842c069240 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8428ff9700 1 -- 172.21.15.29:0/1454755403 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8420007440 con 0x7f842c069240 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.501+0000 7f8433180700 1 -- 172.21.15.29:0/1454755403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c0ab110 con 0x7f842c069240 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.502+0000 7f8428ff9700 1 -- 172.21.15.29:0/1454755403 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420005930 con 0x7f842c069240 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.502+0000 7f8433180700 1 -- 172.21.15.29:0/1454755403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c0c0990 con 0x7f842c069240 2024-01-24T13:58:34.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.502+0000 7f8433180700 1 -- 172.21.15.29:0/1454755403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f842c0c0500 con 0x7f842c069240 2024-01-24T13:58:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.503+0000 7f8428ff9700 1 -- 172.21.15.29:0/1454755403 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8420005ce0 con 0x7f842c069240 2024-01-24T13:58:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.503+0000 7f8428ff9700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f841005f150 0x7f8410061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.504+0000 7f8428ff9700 1 -- 172.21.15.29:0/1454755403 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f842007a680 con 0x7f842c069240 2024-01-24T13:58:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.504+0000 7f8428ff9700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] conn(0x7f84100699a0 0x7f841006be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.504+0000 7f8428ff9700 1 -- 172.21.15.29:0/1454755403 --> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f841006c4c0 con 0x7f84100699a0 2024-01-24T13:58:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.504+0000 7f8428ff9700 1 -- 172.21.15.29:0/1454755403 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f842007aa30 con 0x7f842c069240 2024-01-24T13:58:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.504+0000 7f842bfff700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f841005f150 0x7f8410061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.504+0000 7f843171d700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] conn(0x7f84100699a0 0x7f841006be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.504+0000 7f843171d700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] conn(0x7f84100699a0 0x7f841006be40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.504+0000 7f842bfff700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f841005f150 0x7f8410061610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f8418006fd0 tx=0x7f8418017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.505+0000 7f8428ff9700 1 -- 172.21.15.29:0/1454755403 <== osd.3 v2:172.21.15.29:6808/43697 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f841006c4c0 con 0x7f84100699a0 2024-01-24T13:58:34.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.520+0000 7f8433180700 1 -- 172.21.15.29:0/1454755403 --> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f842c0c0fe0 con 0x7f84100699a0 2024-01-24T13:58:34.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.520+0000 7f8428ff9700 1 -- 172.21.15.29:0/1454755403 <== osd.3 v2:172.21.15.29:6808/43697 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f842c0c0fe0 con 0x7f84100699a0 2024-01-24T13:58:34.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.521+0000 7f8433180700 1 -- 172.21.15.29:0/1454755403 >> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] conn(0x7f84100699a0 msgr2=0x7f841006be40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.521+0000 7f8433180700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] conn(0x7f84100699a0 0x7f841006be40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.521+0000 7f8433180700 1 -- 172.21.15.29:0/1454755403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f841005f150 msgr2=0x7f8410061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.521+0000 7f8433180700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f841005f150 0x7f8410061610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f8418006fd0 tx=0x7f8418017000).stop 2024-01-24T13:58:34.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.521+0000 7f8433180700 1 -- 172.21.15.29:0/1454755403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c069240 msgr2=0x7f842c0c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.521+0000 7f8433180700 1 --2- 172.21.15.29:0/1454755403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c069240 0x7f842c0c1190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f842000c4c0 tx=0x7f8420007c10).stop 2024-01-24T13:58:34.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.521+0000 7f843171d700 1 -- 172.21.15.29:0/1454755403 reap_dead start 2024-01-24T13:58:34.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.521+0000 7f8433180700 1 -- 172.21.15.29:0/1454755403 shutdown_connections 2024-01-24T13:58:34.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.521+0000 7f8433180700 1 -- 172.21.15.29:0/1454755403 >> 172.21.15.29:0/1454755403 conn(0x7f842c0c8580 msgr2=0x7f842c0bd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:34.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.521+0000 7f8433180700 1 -- 172.21.15.29:0/1454755403 shutdown_connections 2024-01-24T13:58:34.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.521+0000 7f8433180700 1 -- 172.21.15.29:0/1454755403 wait complete. 2024-01-24T13:58:34.534 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-01-24T13:58:34.534 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-01-24T13:58:34.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.665+0000 7f9968a0f700 1 Processor -- start 2024-01-24T13:58:34.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.665+0000 7f9968a0f700 1 -- start start 2024-01-24T13:58:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.666+0000 7f9968a0f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9964151de0 0x7f9964155470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.666+0000 7f9968a0f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9964066bb0 0x7f996413d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.666+0000 7f9968a0f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9964158800 con 0x7f99640638c0 2024-01-24T13:58:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.666+0000 7f9968a0f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9964157ed0 con 0x7f9964066bb0 2024-01-24T13:58:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.666+0000 7f9968a0f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99641581b0 con 0x7f9964151de0 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.666+0000 7f9962d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9964066bb0 0x7f996413d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.666+0000 7f996259c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9964151de0 0x7f9964155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.666+0000 7f9962d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9964066bb0 0x7f996413d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:53594/0 (socket says 172.21.15.29:53594) 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.666+0000 7f9962d9d700 1 -- 172.21.15.29:0/1194222427 learned_addr learned my addr 172.21.15.29:0/1194222427 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.666+0000 7f9961d9b700 1 --1- 172.21.15.29:0/1194222427 >> v1:172.21.15.163:6789/0 conn(0x7f99640638c0 0x7f99640664a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:41522/0 (socket says 172.21.15.29:41522) 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.667+0000 7f996259c700 1 -- 172.21.15.29:0/1194222427 >> v2:172.21.15.163:3300/0 conn(0x7f9964066bb0 msgr2=0x7f996413d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.667+0000 7f996259c700 1 --2- 172.21.15.29:0/1194222427 >> v2:172.21.15.163:3300/0 conn(0x7f9964066bb0 0x7f996413d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.667+0000 7f996259c700 1 -- 172.21.15.29:0/1194222427 >> v1:172.21.15.163:6789/0 conn(0x7f99640638c0 legacy=0x7f99640664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.667+0000 7f996259c700 1 -- 172.21.15.29:0/1194222427 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9964158490 con 0x7f9964151de0 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.667+0000 7f996159a700 1 -- 172.21.15.29:0/1194222427 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644670076 0 0) 0x7f9964158800 con 0x7f99640638c0 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.667+0000 7f9962d9d700 1 --2- 172.21.15.29:0/1194222427 >> v2:172.21.15.163:3300/0 conn(0x7f9964066bb0 0x7f996413d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.667+0000 7f996259c700 1 --2- 172.21.15.29:0/1194222427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9964151de0 0x7f9964155470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f9958009760 tx=0x7f9958009060).ready entity=mon.2 client_cookie=d4134352a908175f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.667+0000 7f996159a700 1 -- 172.21.15.29:0/1194222427 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995800f510 con 0x7f9964151de0 2024-01-24T13:58:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.667+0000 7f996159a700 1 -- 172.21.15.29:0/1194222427 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995800f670 con 0x7f9964151de0 2024-01-24T13:58:34.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.667+0000 7f996159a700 1 -- 172.21.15.29:0/1194222427 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995800f9f0 con 0x7f9964151de0 2024-01-24T13:58:34.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.668+0000 7f9968a0f700 1 -- 172.21.15.29:0/1194222427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9964151de0 msgr2=0x7f9964155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.668+0000 7f9968a0f700 1 --2- 172.21.15.29:0/1194222427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9964151de0 0x7f9964155470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f9958009760 tx=0x7f9958009060).stop 2024-01-24T13:58:34.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.668+0000 7f9968a0f700 1 -- 172.21.15.29:0/1194222427 shutdown_connections 2024-01-24T13:58:34.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.668+0000 7f9968a0f700 1 --2- 172.21.15.29:0/1194222427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9964151de0 0x7f9964155470 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.668+0000 7f9968a0f700 1 --2- 172.21.15.29:0/1194222427 >> v2:172.21.15.163:3300/0 conn(0x7f9964066bb0 0x7f996413d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.668+0000 7f9968a0f700 1 -- 172.21.15.29:0/1194222427 >> 172.21.15.29:0/1194222427 conn(0x7f99640c5640 msgr2=0x7f99640c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.668+0000 7f9968a0f700 1 -- 172.21.15.29:0/1194222427 shutdown_connections 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.668+0000 7f9968a0f700 1 -- 172.21.15.29:0/1194222427 wait complete. 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.669+0000 7f9968a0f700 1 Processor -- start 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.669+0000 7f9968a0f700 1 -- start start 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9968a0f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99640638c0 0x7f99640ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9968a0f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9964066bb0 0x7f99640a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9968a0f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9964151de0 0x7f99640a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9968a0f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99640a7950 con 0x7f9964151de0 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9968a0f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99640ab5b0 con 0x7f9964066bb0 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9968a0f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99640a6d40 con 0x7f99640638c0 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9962d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9964151de0 0x7f99640a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9962d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9964151de0 0x7f99640a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41218/0 (socket says 172.21.15.29:41218) 2024-01-24T13:58:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9962d9d700 1 -- 172.21.15.29:0/3207354424 learned_addr learned my addr 172.21.15.29:0/3207354424 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f996259c700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99640638c0 0x7f99640ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9961d9b700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9964066bb0 0x7f99640a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9962d9d700 1 -- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99640638c0 msgr2=0x7f99640ab0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9962d9d700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99640638c0 0x7f99640ab0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9962d9d700 1 -- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9964066bb0 msgr2=0x7f99640a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9962d9d700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9964066bb0 0x7f99640a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.670+0000 7f9962d9d700 1 -- 172.21.15.29:0/3207354424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c009ae0 con 0x7f9964151de0 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.671+0000 7f9962d9d700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9964151de0 0x7f99640a6690 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f995c009fc0 tx=0x7f995c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.671+0000 7f99527fc700 1 -- 172.21.15.29:0/3207354424 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c00c960 con 0x7f9964151de0 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.671+0000 7f99527fc700 1 -- 172.21.15.29:0/3207354424 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c00f040 con 0x7f9964151de0 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.671+0000 7f9968a0f700 1 -- 172.21.15.29:0/3207354424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99580092a0 con 0x7f9964151de0 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.671+0000 7f99527fc700 1 -- 172.21.15.29:0/3207354424 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c00ccd0 con 0x7f9964151de0 2024-01-24T13:58:34.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.671+0000 7f9968a0f700 1 -- 172.21.15.29:0/3207354424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99640a7300 con 0x7f9964151de0 2024-01-24T13:58:34.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.672+0000 7f9968a0f700 1 -- 172.21.15.29:0/3207354424 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f99640475e0 con 0x7f9964151de0 2024-01-24T13:58:34.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.672+0000 7f99527fc700 1 -- 172.21.15.29:0/3207354424 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f995c014670 con 0x7f9964151de0 2024-01-24T13:58:34.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.673+0000 7f99527fc700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f994405f150 0x7f9944061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.673+0000 7f99527fc700 1 -- 172.21.15.29:0/3207354424 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f995c07aff0 con 0x7f9964151de0 2024-01-24T13:58:34.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.673+0000 7f99527fc700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] conn(0x7f99440699a0 0x7f994406be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.673+0000 7f99527fc700 1 -- 172.21.15.29:0/3207354424 --> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f994406c4c0 con 0x7f99440699a0 2024-01-24T13:58:34.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.673+0000 7f99527fc700 1 -- 172.21.15.29:0/3207354424 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f995c07b3a0 con 0x7f9964151de0 2024-01-24T13:58:34.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.673+0000 7f996259c700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f994405f150 0x7f9944061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.673+0000 7f9961d9b700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] conn(0x7f99440699a0 0x7f994406be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.673+0000 7f996259c700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f994405f150 0x7f9944061610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f9958009fd0 tx=0x7f9958017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.674+0000 7f9961d9b700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] conn(0x7f99440699a0 0x7f994406be40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.674+0000 7f99527fc700 1 -- 172.21.15.29:0/3207354424 <== osd.4 v2:172.21.15.163:6800/43896 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f994406c4c0 con 0x7f99440699a0 2024-01-24T13:58:34.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.690+0000 7f9968a0f700 1 -- 172.21.15.29:0/3207354424 --> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f996413e020 con 0x7f99440699a0 2024-01-24T13:58:34.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.690+0000 7f99527fc700 1 -- 172.21.15.29:0/3207354424 <== osd.4 v2:172.21.15.163:6800/43896 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f996413e020 con 0x7f99440699a0 2024-01-24T13:58:34.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.691+0000 7f9968a0f700 1 -- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] conn(0x7f99440699a0 msgr2=0x7f994406be40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.691+0000 7f9968a0f700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] conn(0x7f99440699a0 0x7f994406be40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.691+0000 7f9968a0f700 1 -- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f994405f150 msgr2=0x7f9944061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.691+0000 7f9968a0f700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f994405f150 0x7f9944061610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f9958009fd0 tx=0x7f9958017000).stop 2024-01-24T13:58:34.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.691+0000 7f9968a0f700 1 -- 172.21.15.29:0/3207354424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9964151de0 msgr2=0x7f99640a6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.691+0000 7f9968a0f700 1 --2- 172.21.15.29:0/3207354424 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9964151de0 0x7f99640a6690 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f995c009fc0 tx=0x7f995c00b040).stop 2024-01-24T13:58:34.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.691+0000 7f9962d9d700 1 -- 172.21.15.29:0/3207354424 reap_dead start 2024-01-24T13:58:34.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.691+0000 7f9968a0f700 1 -- 172.21.15.29:0/3207354424 shutdown_connections 2024-01-24T13:58:34.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.691+0000 7f9968a0f700 1 -- 172.21.15.29:0/3207354424 >> 172.21.15.29:0/3207354424 conn(0x7f99640c5640 msgr2=0x7f99641547a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:34.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.691+0000 7f9968a0f700 1 -- 172.21.15.29:0/3207354424 shutdown_connections 2024-01-24T13:58:34.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.691+0000 7f9968a0f700 1 -- 172.21.15.29:0/3207354424 wait complete. 2024-01-24T13:58:34.704 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-01-24T13:58:34.704 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-01-24T13:58:34.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.837+0000 7f6951f2f700 1 Processor -- start 2024-01-24T13:58:34.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.837+0000 7f6951f2f700 1 -- start start 2024-01-24T13:58:34.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f6951f2f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f694c154240 0x7f694c064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f6951f2f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f694c0a9d20 0x7f694c0aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f6951f2f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694c1550e0 con 0x7f694c157cf0 2024-01-24T13:58:34.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f6951f2f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694c0c0d10 con 0x7f694c154240 2024-01-24T13:58:34.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f6951f2f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694c0aa670 con 0x7f694c0a9d20 2024-01-24T13:58:34.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f694b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f694c154240 0x7f694c064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f694b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f694c154240 0x7f694c064780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41224/0 (socket says 172.21.15.29:41224) 2024-01-24T13:58:34.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f694b7fe700 1 -- 172.21.15.29:0/2214011556 learned_addr learned my addr 172.21.15.29:0/2214011556 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:34.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f694affd700 1 --2- 172.21.15.29:0/2214011556 >> v2:172.21.15.163:3300/0 conn(0x7f694c0a9d20 0x7f694c0aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f694b7fe700 1 -- 172.21.15.29:0/2214011556 >> v2:172.21.15.163:3300/0 conn(0x7f694c0a9d20 msgr2=0x7f694c0aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:34.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f694b7fe700 1 --2- 172.21.15.29:0/2214011556 >> v2:172.21.15.163:3300/0 conn(0x7f694c0a9d20 0x7f694c0aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f694b7fe700 1 -- 172.21.15.29:0/2214011556 >> v1:172.21.15.163:6789/0 conn(0x7f694c157cf0 legacy=0x7f694c158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f694b7fe700 1 -- 172.21.15.29:0/2214011556 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c0650d0 con 0x7f694c154240 2024-01-24T13:58:34.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f694b7fe700 1 --2- 172.21.15.29:0/2214011556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f694c154240 0x7f694c064780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f694c084090 tx=0x7f693c0090f0).ready entity=mon.0 client_cookie=f5f5f04e7313709f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.838+0000 7f694a7fc700 1 -- 172.21.15.29:0/2214011556 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c00b520 con 0x7f694c154240 2024-01-24T13:58:34.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.839+0000 7f694a7fc700 1 -- 172.21.15.29:0/2214011556 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c00b680 con 0x7f694c154240 2024-01-24T13:58:34.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.839+0000 7f694a7fc700 1 -- 172.21.15.29:0/2214011556 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c00ba00 con 0x7f694c154240 2024-01-24T13:58:34.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.839+0000 7f6951f2f700 1 -- 172.21.15.29:0/2214011556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f694c154240 msgr2=0x7f694c064780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.839+0000 7f6951f2f700 1 --2- 172.21.15.29:0/2214011556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f694c154240 0x7f694c064780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f694c084090 tx=0x7f693c0090f0).stop 2024-01-24T13:58:34.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.839+0000 7f6951f2f700 1 -- 172.21.15.29:0/2214011556 shutdown_connections 2024-01-24T13:58:34.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.839+0000 7f6951f2f700 1 --2- 172.21.15.29:0/2214011556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f694c154240 0x7f694c064780 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.839+0000 7f6951f2f700 1 --2- 172.21.15.29:0/2214011556 >> v2:172.21.15.163:3300/0 conn(0x7f694c0a9d20 0x7f694c0aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.839+0000 7f6951f2f700 1 -- 172.21.15.29:0/2214011556 >> 172.21.15.29:0/2214011556 conn(0x7f694c0c8310 msgr2=0x7f694c05e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:34.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.839+0000 7f6951f2f700 1 -- 172.21.15.29:0/2214011556 shutdown_connections 2024-01-24T13:58:34.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.839+0000 7f6951f2f700 1 -- 172.21.15.29:0/2214011556 wait complete. 2024-01-24T13:58:34.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.841+0000 7f6951f2f700 1 Processor -- start 2024-01-24T13:58:34.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.841+0000 7f6951f2f700 1 -- start start 2024-01-24T13:58:34.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.841+0000 7f6951f2f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f694c0a9d20 0x7f694c0aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.841+0000 7f6951f2f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f694c154240 0x7f694c0af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.841+0000 7f6951f2f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f694c157cf0 0x7f694c146670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.841+0000 7f6951f2f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c0af960 con 0x7f694c154240 2024-01-24T13:58:34.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.841+0000 7f6951f2f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c0afc40 con 0x7f694c157cf0 2024-01-24T13:58:34.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f6951f2f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694c0aff20 con 0x7f694c0a9d20 2024-01-24T13:58:34.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f694affd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f694c154240 0x7f694c0af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f694affd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f694c154240 0x7f694c0af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41240/0 (socket says 172.21.15.29:41240) 2024-01-24T13:58:34.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f694affd700 1 -- 172.21.15.29:0/3920595611 learned_addr learned my addr 172.21.15.29:0/3920595611 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:34.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f694b7fe700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f694c0a9d20 0x7f694c0aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f694bfff700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f694c157cf0 0x7f694c146670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f694affd700 1 -- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f694c0a9d20 msgr2=0x7f694c0aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f694affd700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f694c0a9d20 0x7f694c0aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f694affd700 1 -- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f694c157cf0 msgr2=0x7f694c146670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f694affd700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f694c157cf0 0x7f694c146670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f694affd700 1 -- 172.21.15.29:0/3920595611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693c009370 con 0x7f694c154240 2024-01-24T13:58:34.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f694affd700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f694c154240 0x7f694c0af110 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f694000e950 tx=0x7f6940009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f69337fe700 1 -- 172.21.15.29:0/3920595611 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940012600 con 0x7f694c154240 2024-01-24T13:58:34.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f69337fe700 1 -- 172.21.15.29:0/3920595611 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6940012760 con 0x7f694c154240 2024-01-24T13:58:34.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f6951f2f700 1 -- 172.21.15.29:0/3920595611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694c0b0200 con 0x7f694c154240 2024-01-24T13:58:34.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f69337fe700 1 -- 172.21.15.29:0/3920595611 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940012ae0 con 0x7f694c154240 2024-01-24T13:58:34.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.842+0000 7f6951f2f700 1 -- 172.21.15.29:0/3920595611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694c0af600 con 0x7f694c154240 2024-01-24T13:58:34.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.843+0000 7f6951f2f700 1 -- 172.21.15.29:0/3920595611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f694c0afdb0 con 0x7f694c154240 2024-01-24T13:58:34.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.844+0000 7f69337fe700 1 -- 172.21.15.29:0/3920595611 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f694001b400 con 0x7f694c154240 2024-01-24T13:58:34.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.844+0000 7f69337fe700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f693805f150 0x7f6938061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.844+0000 7f69337fe700 1 -- 172.21.15.29:0/3920595611 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f694007a9d0 con 0x7f694c154240 2024-01-24T13:58:34.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.844+0000 7f69337fe700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] conn(0x7f69380699a0 0x7f693806be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:34.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.844+0000 7f69337fe700 1 -- 172.21.15.29:0/3920595611 --> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f693806c4c0 con 0x7f69380699a0 2024-01-24T13:58:34.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.844+0000 7f694b7fe700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f693805f150 0x7f6938061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.844+0000 7f69337fe700 1 -- 172.21.15.29:0/3920595611 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f694007f220 con 0x7f694c154240 2024-01-24T13:58:34.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.845+0000 7f694bfff700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] conn(0x7f69380699a0 0x7f693806be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:34.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.845+0000 7f694b7fe700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f693805f150 0x7f6938061610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f693c009130 tx=0x7f693c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.845+0000 7f694bfff700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] conn(0x7f69380699a0 0x7f693806be40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:34.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.846+0000 7f69337fe700 1 -- 172.21.15.29:0/3920595611 <== osd.5 v2:172.21.15.163:6817/43926 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f693806c4c0 con 0x7f69380699a0 2024-01-24T13:58:34.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.861+0000 7f6951f2f700 1 -- 172.21.15.29:0/3920595611 --> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f694c0475e0 con 0x7f69380699a0 2024-01-24T13:58:34.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.862+0000 7f69337fe700 1 -- 172.21.15.29:0/3920595611 <== osd.5 v2:172.21.15.163:6817/43926 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f694c0475e0 con 0x7f69380699a0 2024-01-24T13:58:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.862+0000 7f6951f2f700 1 -- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] conn(0x7f69380699a0 msgr2=0x7f693806be40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.862+0000 7f6951f2f700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] conn(0x7f69380699a0 0x7f693806be40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.862+0000 7f6951f2f700 1 -- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f693805f150 msgr2=0x7f6938061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.862+0000 7f6951f2f700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f693805f150 0x7f6938061610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f693c009130 tx=0x7f693c017000).stop 2024-01-24T13:58:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.862+0000 7f6951f2f700 1 -- 172.21.15.29:0/3920595611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f694c154240 msgr2=0x7f694c0af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.862+0000 7f6951f2f700 1 --2- 172.21.15.29:0/3920595611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f694c154240 0x7f694c0af110 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f694000e950 tx=0x7f6940009060).stop 2024-01-24T13:58:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.862+0000 7f694bfff700 1 -- 172.21.15.29:0/3920595611 reap_dead start 2024-01-24T13:58:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.862+0000 7f6951f2f700 1 -- 172.21.15.29:0/3920595611 shutdown_connections 2024-01-24T13:58:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.862+0000 7f6951f2f700 1 -- 172.21.15.29:0/3920595611 >> 172.21.15.29:0/3920595611 conn(0x7f694c0c8310 msgr2=0x7f694c1535a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.862+0000 7f6951f2f700 1 -- 172.21.15.29:0/3920595611 shutdown_connections 2024-01-24T13:58:34.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:34.862+0000 7f6951f2f700 1 -- 172.21.15.29:0/3920595611 wait complete. 2024-01-24T13:58:34.875 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-01-24T13:58:34.876 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-01-24T13:58:35.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.006+0000 7f832e885700 1 Processor -- start 2024-01-24T13:58:35.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.006+0000 7f832e885700 1 -- start start 2024-01-24T13:58:35.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.006+0000 7f832e885700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f83280c2ca0 0x7f8328151150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f832e885700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8328151640 0x7f8328069340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f832e885700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83280c3eb0 con 0x7f8328150d70 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f832e885700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83280c32a0 con 0x7f83280c2ca0 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f832e885700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83280c3580 con 0x7f8328151640 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f83277fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f83280c2ca0 0x7f8328151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f83277fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f83280c2ca0 0x7f8328151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41242/0 (socket says 172.21.15.29:41242) 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f8327fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8328150d70 0x7f8328065e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42124/0 (socket says 172.21.15.29:42124) 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f83277fe700 1 -- 172.21.15.29:0/3156893925 learned_addr learned my addr 172.21.15.29:0/3156893925 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f832ce22700 1 --2- 172.21.15.29:0/3156893925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8328151640 0x7f8328069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f83277fe700 1 -- 172.21.15.29:0/3156893925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8328151640 msgr2=0x7f8328069340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f83277fe700 1 --2- 172.21.15.29:0/3156893925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8328151640 0x7f8328069340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f83277fe700 1 -- 172.21.15.29:0/3156893925 >> v1:172.21.15.29:6789/0 conn(0x7f8328150d70 legacy=0x7f8328065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f83277fe700 1 -- 172.21.15.29:0/3156893925 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83280c3860 con 0x7f83280c2ca0 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f8326ffd700 1 -- 172.21.15.29:0/3156893925 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900907646 0 0) 0x7f83280c3eb0 con 0x7f8328150d70 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f83277fe700 1 --2- 172.21.15.29:0/3156893925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83280c2ca0 0x7f8328151150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f831c006fd0 tx=0x7f831c0090f0).ready entity=mon.0 client_cookie=3f540a7a4faacb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f8326ffd700 1 -- 172.21.15.29:0/3156893925 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00b600 con 0x7f83280c2ca0 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f8326ffd700 1 -- 172.21.15.29:0/3156893925 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c00b760 con 0x7f83280c2ca0 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.007+0000 7f8326ffd700 1 -- 172.21.15.29:0/3156893925 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00bae0 con 0x7f83280c2ca0 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.008+0000 7f832e885700 1 -- 172.21.15.29:0/3156893925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83280c2ca0 msgr2=0x7f8328151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.008+0000 7f832e885700 1 --2- 172.21.15.29:0/3156893925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83280c2ca0 0x7f8328151150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f831c006fd0 tx=0x7f831c0090f0).stop 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.008+0000 7f832e885700 1 -- 172.21.15.29:0/3156893925 shutdown_connections 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.008+0000 7f832e885700 1 --2- 172.21.15.29:0/3156893925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8328151640 0x7f8328069340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.008+0000 7f832e885700 1 --2- 172.21.15.29:0/3156893925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83280c2ca0 0x7f8328151150 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.008+0000 7f832e885700 1 -- 172.21.15.29:0/3156893925 >> 172.21.15.29:0/3156893925 conn(0x7f83280c8580 msgr2=0x7f83280c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.008+0000 7f832e885700 1 -- 172.21.15.29:0/3156893925 shutdown_connections 2024-01-24T13:58:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.008+0000 7f832e885700 1 -- 172.21.15.29:0/3156893925 wait complete. 2024-01-24T13:58:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.009+0000 7f832e885700 1 Processor -- start 2024-01-24T13:58:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.010+0000 7f832e885700 1 -- start start 2024-01-24T13:58:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.010+0000 7f832e885700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83280c2ca0 0x7f83280ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.010+0000 7f832e885700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8328150d70 0x7f8328148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.010+0000 7f832e885700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8328151640 0x7f83280c1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.010+0000 7f832e885700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8328148c30 con 0x7f8328150d70 2024-01-24T13:58:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.010+0000 7f832e885700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83280ad0a0 con 0x7f83280c2ca0 2024-01-24T13:58:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.010+0000 7f832e885700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83280bf2d0 con 0x7f8328151640 2024-01-24T13:58:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f83277fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8328150d70 0x7f8328148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f83277fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8328150d70 0x7f8328148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41250/0 (socket says 172.21.15.29:41250) 2024-01-24T13:58:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f83277fe700 1 -- 172.21.15.29:0/1624987033 learned_addr learned my addr 172.21.15.29:0/1624987033 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f832ce22700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8328151640 0x7f83280c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f8327fff700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83280c2ca0 0x7f83280ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f83277fe700 1 -- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8328151640 msgr2=0x7f83280c1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f83277fe700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8328151640 0x7f83280c1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f83277fe700 1 -- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83280c2ca0 msgr2=0x7f83280ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f83277fe700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83280c2ca0 0x7f83280ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f83277fe700 1 -- 172.21.15.29:0/1624987033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c009370 con 0x7f8328150d70 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f8327fff700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83280c2ca0 0x7f83280ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f83277fe700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8328150d70 0x7f8328148740 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f8328069030 tx=0x7f831c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f830ffff700 1 -- 172.21.15.29:0/1624987033 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00b8d0 con 0x7f8328150d70 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f830ffff700 1 -- 172.21.15.29:0/1624987033 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c00ba30 con 0x7f8328150d70 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f832e885700 1 -- 172.21.15.29:0/1624987033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83280acdc0 con 0x7f8328150d70 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f830ffff700 1 -- 172.21.15.29:0/1624987033 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c01c670 con 0x7f8328150d70 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.011+0000 7f832e885700 1 -- 172.21.15.29:0/1624987033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83280bef60 con 0x7f8328150d70 2024-01-24T13:58:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.012+0000 7f832e885700 1 -- 172.21.15.29:0/1624987033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83280475e0 con 0x7f8328150d70 2024-01-24T13:58:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.013+0000 7f830ffff700 1 -- 172.21.15.29:0/1624987033 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f831c01ca20 con 0x7f8328150d70 2024-01-24T13:58:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.013+0000 7f830ffff700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f831405f150 0x7f8314061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.013+0000 7f830ffff700 1 -- 172.21.15.29:0/1624987033 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f831c07c2d0 con 0x7f8328150d70 2024-01-24T13:58:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.013+0000 7f8327fff700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f831405f150 0x7f8314061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.013+0000 7f830ffff700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] conn(0x7f83140699a0 0x7f831406be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.013+0000 7f830ffff700 1 -- 172.21.15.29:0/1624987033 --> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f831406c4c0 con 0x7f83140699a0 2024-01-24T13:58:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.013+0000 7f830ffff700 1 -- 172.21.15.29:0/1624987033 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f831c07e8f0 con 0x7f8328150d70 2024-01-24T13:58:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.013+0000 7f832ce22700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] conn(0x7f83140699a0 0x7f831406be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.014+0000 7f8327fff700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f831405f150 0x7f8314061610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f83180011e0 tx=0x7f8318007b10).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.014+0000 7f832ce22700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] conn(0x7f83140699a0 0x7f831406be40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:35.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.015+0000 7f830ffff700 1 -- 172.21.15.29:0/1624987033 <== osd.6 v2:172.21.15.163:6808/43897 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f831406c4c0 con 0x7f83140699a0 2024-01-24T13:58:35.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.030+0000 7f832e885700 1 -- 172.21.15.29:0/1624987033 --> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f83280bf440 con 0x7f83140699a0 2024-01-24T13:58:35.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.030+0000 7f830ffff700 1 -- 172.21.15.29:0/1624987033 <== osd.6 v2:172.21.15.163:6808/43897 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f83280bf440 con 0x7f83140699a0 2024-01-24T13:58:35.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.030+0000 7f832e885700 1 -- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] conn(0x7f83140699a0 msgr2=0x7f831406be40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.030+0000 7f832e885700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] conn(0x7f83140699a0 0x7f831406be40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.030+0000 7f832e885700 1 -- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f831405f150 msgr2=0x7f8314061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.030+0000 7f832e885700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f831405f150 0x7f8314061610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f83180011e0 tx=0x7f8318007b10).stop 2024-01-24T13:58:35.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.030+0000 7f832e885700 1 -- 172.21.15.29:0/1624987033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8328150d70 msgr2=0x7f8328148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.030+0000 7f832e885700 1 --2- 172.21.15.29:0/1624987033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8328150d70 0x7f8328148740 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f8328069030 tx=0x7f831c017000).stop 2024-01-24T13:58:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.031+0000 7f832ce22700 1 -- 172.21.15.29:0/1624987033 reap_dead start 2024-01-24T13:58:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.031+0000 7f832e885700 1 -- 172.21.15.29:0/1624987033 shutdown_connections 2024-01-24T13:58:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.031+0000 7f832e885700 1 -- 172.21.15.29:0/1624987033 >> 172.21.15.29:0/1624987033 conn(0x7f83280c8580 msgr2=0x7f83280bda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.031+0000 7f832e885700 1 -- 172.21.15.29:0/1624987033 shutdown_connections 2024-01-24T13:58:35.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.031+0000 7f832e885700 1 -- 172.21.15.29:0/1624987033 wait complete. 2024-01-24T13:58:35.044 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-01-24T13:58:35.044 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-01-24T13:58:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.174+0000 7f90684e0700 1 Processor -- start 2024-01-24T13:58:35.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.175+0000 7f90684e0700 1 -- start start 2024-01-24T13:58:35.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f90684e0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9060069240 0x7f9060150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f90684e0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90601514a0 0x7f9060151950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f90684e0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906006a350 con 0x7f9060150bd0 2024-01-24T13:58:35.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f90684e0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9060069740 con 0x7f9060069240 2024-01-24T13:58:35.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f90684e0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9060069a20 con 0x7f90601514a0 2024-01-24T13:58:35.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f9065a7b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9060069240 0x7f9060150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f9066a7d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90601514a0 0x7f9060151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f9065a7b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9060069240 0x7f9060150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41262/0 (socket says 172.21.15.29:41262) 2024-01-24T13:58:35.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f9066a7d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90601514a0 0x7f9060151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:53656/0 (socket says 172.21.15.29:53656) 2024-01-24T13:58:35.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f9065a7b700 1 -- 172.21.15.29:0/1612894582 learned_addr learned my addr 172.21.15.29:0/1612894582 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:35.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f9065a7b700 1 -- 172.21.15.29:0/1612894582 >> v2:172.21.15.163:3300/0 conn(0x7f90601514a0 msgr2=0x7f9060151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:35.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f9065a7b700 1 --2- 172.21.15.29:0/1612894582 >> v2:172.21.15.163:3300/0 conn(0x7f90601514a0 0x7f9060151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f9065a7b700 1 -- 172.21.15.29:0/1612894582 >> v1:172.21.15.163:6789/0 conn(0x7f9060150bd0 legacy=0x7f9060065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f9065a7b700 1 -- 172.21.15.29:0/1612894582 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060069d00 con 0x7f9060069240 2024-01-24T13:58:35.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f9065a7b700 1 --2- 172.21.15.29:0/1612894582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9060069240 0x7f9060150fb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f90600be950 tx=0x7f90500090f0).ready entity=mon.0 client_cookie=84d6e67e5e136e39 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:35.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f906527a700 1 -- 172.21.15.29:0/1612894582 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905000b520 con 0x7f9060069240 2024-01-24T13:58:35.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.176+0000 7f906527a700 1 -- 172.21.15.29:0/1612894582 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905000b680 con 0x7f9060069240 2024-01-24T13:58:35.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.177+0000 7f906527a700 1 -- 172.21.15.29:0/1612894582 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905000ba00 con 0x7f9060069240 2024-01-24T13:58:35.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.177+0000 7f90684e0700 1 -- 172.21.15.29:0/1612894582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9060069240 msgr2=0x7f9060150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.177+0000 7f90684e0700 1 --2- 172.21.15.29:0/1612894582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9060069240 0x7f9060150fb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f90600be950 tx=0x7f90500090f0).stop 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.177+0000 7f90684e0700 1 -- 172.21.15.29:0/1612894582 shutdown_connections 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.177+0000 7f90684e0700 1 --2- 172.21.15.29:0/1612894582 >> v2:172.21.15.163:3300/0 conn(0x7f90601514a0 0x7f9060151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.177+0000 7f90684e0700 1 --2- 172.21.15.29:0/1612894582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9060069240 0x7f9060150fb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.177+0000 7f90684e0700 1 -- 172.21.15.29:0/1612894582 >> 172.21.15.29:0/1612894582 conn(0x7f90600c8580 msgr2=0x7f90600c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.177+0000 7f90684e0700 1 -- 172.21.15.29:0/1612894582 shutdown_connections 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.177+0000 7f90684e0700 1 -- 172.21.15.29:0/1612894582 wait complete. 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.178+0000 7f90684e0700 1 Processor -- start 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.178+0000 7f90684e0700 1 -- start start 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.179+0000 7f90684e0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9060069240 0x7f90600c1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.179+0000 7f90684e0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9060150bd0 0x7f906013b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.179+0000 7f90684e0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90601514a0 0x7f906014cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.179+0000 7f90684e0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90600c0690 con 0x7f90601514a0 2024-01-24T13:58:35.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.179+0000 7f90684e0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90600c0970 con 0x7f9060069240 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.179+0000 7f90684e0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90600c0c50 con 0x7f9060150bd0 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.179+0000 7f9066a7d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90601514a0 0x7f906014cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.179+0000 7f9066a7d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90601514a0 0x7f906014cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41268/0 (socket says 172.21.15.29:41268) 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.179+0000 7f9066a7d700 1 -- 172.21.15.29:0/1639005380 learned_addr learned my addr 172.21.15.29:0/1639005380 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.179+0000 7f906627c700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9060069240 0x7f90600c1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.179+0000 7f9065a7b700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9060150bd0 0x7f906013b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.180+0000 7f9066a7d700 1 -- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9060150bd0 msgr2=0x7f906013b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.180+0000 7f9066a7d700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9060150bd0 0x7f906013b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.180+0000 7f9066a7d700 1 -- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9060069240 msgr2=0x7f90600c1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.180+0000 7f9066a7d700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9060069240 0x7f90600c1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.180+0000 7f9066a7d700 1 -- 172.21.15.29:0/1639005380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9050009370 con 0x7f90601514a0 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.180+0000 7f9066a7d700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90601514a0 0x7f906014cf80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9058009df0 tx=0x7f905800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.180+0000 7f90567fc700 1 -- 172.21.15.29:0/1639005380 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905800caa0 con 0x7f90601514a0 2024-01-24T13:58:35.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.180+0000 7f90567fc700 1 -- 172.21.15.29:0/1639005380 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905800f040 con 0x7f90601514a0 2024-01-24T13:58:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.180+0000 7f90684e0700 1 -- 172.21.15.29:0/1639005380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90600c0f30 con 0x7f90601514a0 2024-01-24T13:58:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.180+0000 7f90567fc700 1 -- 172.21.15.29:0/1639005380 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905800ce10 con 0x7f90601514a0 2024-01-24T13:58:35.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.180+0000 7f90684e0700 1 -- 172.21.15.29:0/1639005380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906013b950 con 0x7f90601514a0 2024-01-24T13:58:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.181+0000 7f90684e0700 1 -- 172.21.15.29:0/1639005380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f90600c0800 con 0x7f90601514a0 2024-01-24T13:58:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.181+0000 7f90567fc700 1 -- 172.21.15.29:0/1639005380 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9058013610 con 0x7f90601514a0 2024-01-24T13:58:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.182+0000 7f90567fc700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f904405f150 0x7f9044061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.182+0000 7f90567fc700 1 -- 172.21.15.29:0/1639005380 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f905807acc0 con 0x7f90601514a0 2024-01-24T13:58:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.182+0000 7f906627c700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f904405f150 0x7f9044061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.182+0000 7f90567fc700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] conn(0x7f9044069a20 0x7f904406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.182+0000 7f90567fc700 1 -- 172.21.15.29:0/1639005380 --> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f904406c540 con 0x7f9044069a20 2024-01-24T13:58:35.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.182+0000 7f90567fc700 1 -- 172.21.15.29:0/1639005380 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9058080210 con 0x7f90601514a0 2024-01-24T13:58:35.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.182+0000 7f9065a7b700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] conn(0x7f9044069a20 0x7f904406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.182+0000 7f906627c700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f904405f150 0x7f9044061610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f905c003900 tx=0x7f905c007d40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:35.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.183+0000 7f9065a7b700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] conn(0x7f9044069a20 0x7f904406bec0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:35.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.183+0000 7f90567fc700 1 -- 172.21.15.29:0/1639005380 <== osd.7 v2:172.21.15.163:6816/43927 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f904406c540 con 0x7f9044069a20 2024-01-24T13:58:35.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.199+0000 7f90684e0700 1 -- 172.21.15.29:0/1639005380 --> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f90600c0ae0 con 0x7f9044069a20 2024-01-24T13:58:35.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.199+0000 7f90567fc700 1 -- 172.21.15.29:0/1639005380 <== osd.7 v2:172.21.15.163:6816/43927 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f90600c0ae0 con 0x7f9044069a20 2024-01-24T13:58:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.200+0000 7f90684e0700 1 -- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] conn(0x7f9044069a20 msgr2=0x7f904406bec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.200+0000 7f90684e0700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] conn(0x7f9044069a20 0x7f904406bec0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.200+0000 7f90684e0700 1 -- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f904405f150 msgr2=0x7f9044061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.200+0000 7f90684e0700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f904405f150 0x7f9044061610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f905c003900 tx=0x7f905c007d40).stop 2024-01-24T13:58:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.200+0000 7f90684e0700 1 -- 172.21.15.29:0/1639005380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90601514a0 msgr2=0x7f906014cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.200+0000 7f90684e0700 1 --2- 172.21.15.29:0/1639005380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90601514a0 0x7f906014cf80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9058009df0 tx=0x7f905800b040).stop 2024-01-24T13:58:35.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.200+0000 7f9066a7d700 1 -- 172.21.15.29:0/1639005380 reap_dead start 2024-01-24T13:58:35.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.200+0000 7f90684e0700 1 -- 172.21.15.29:0/1639005380 shutdown_connections 2024-01-24T13:58:35.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.200+0000 7f90684e0700 1 -- 172.21.15.29:0/1639005380 >> 172.21.15.29:0/1639005380 conn(0x7f90600c8580 msgr2=0x7f90600627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:35.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.200+0000 7f90684e0700 1 -- 172.21.15.29:0/1639005380 shutdown_connections 2024-01-24T13:58:35.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.200+0000 7f90684e0700 1 -- 172.21.15.29:0/1639005380 wait complete. 2024-01-24T13:58:35.214 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-01-24T13:58:35.214 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-24T13:58:35.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.344+0000 7f51f7500700 1 Processor -- start 2024-01-24T13:58:35.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.345+0000 7f51f7500700 1 -- start start 2024-01-24T13:58:35.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.345+0000 7f51f7500700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51f0151de0 0x7f51f0155470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.345+0000 7f51f7500700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f0066bb0 0x7f51f013d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.345+0000 7f51f7500700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f0158800 con 0x7f51f00638c0 2024-01-24T13:58:35.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.345+0000 7f51f7500700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f0157ed0 con 0x7f51f0151de0 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51f7500700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f01581b0 con 0x7f51f0066bb0 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51f529c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51f0151de0 0x7f51f0155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51f529c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51f0151de0 0x7f51f0155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41270/0 (socket says 172.21.15.29:41270) 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51f529c700 1 -- 172.21.15.29:0/816190242 learned_addr learned my addr 172.21.15.29:0/816190242 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51f5a9d700 1 --2- 172.21.15.29:0/816190242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f0066bb0 0x7f51f013d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51f529c700 1 -- 172.21.15.29:0/816190242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f0066bb0 msgr2=0x7f51f013d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51f529c700 1 --2- 172.21.15.29:0/816190242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f0066bb0 0x7f51f013d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51f529c700 1 -- 172.21.15.29:0/816190242 >> v1:172.21.15.163:6789/0 conn(0x7f51f00638c0 legacy=0x7f51f00664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51f529c700 1 -- 172.21.15.29:0/816190242 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0158490 con 0x7f51f0151de0 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51f529c700 1 --2- 172.21.15.29:0/816190242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51f0151de0 0x7f51f0155470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f51ec006fd0 tx=0x7f51ec0090f0).ready entity=mon.0 client_cookie=194f077a41ac3486 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51e3fff700 1 -- 172.21.15.29:0/816190242 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00b600 con 0x7f51f0151de0 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51e3fff700 1 -- 172.21.15.29:0/816190242 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec00b760 con 0x7f51f0151de0 2024-01-24T13:58:35.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.346+0000 7f51e3fff700 1 -- 172.21.15.29:0/816190242 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00bae0 con 0x7f51f0151de0 2024-01-24T13:58:35.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.347+0000 7f51f7500700 1 -- 172.21.15.29:0/816190242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51f0151de0 msgr2=0x7f51f0155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.347+0000 7f51f7500700 1 --2- 172.21.15.29:0/816190242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51f0151de0 0x7f51f0155470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f51ec006fd0 tx=0x7f51ec0090f0).stop 2024-01-24T13:58:35.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.347+0000 7f51f7500700 1 -- 172.21.15.29:0/816190242 shutdown_connections 2024-01-24T13:58:35.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.347+0000 7f51f7500700 1 --2- 172.21.15.29:0/816190242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f0066bb0 0x7f51f013d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.347+0000 7f51f7500700 1 --2- 172.21.15.29:0/816190242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51f0151de0 0x7f51f0155470 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.347+0000 7f51f7500700 1 -- 172.21.15.29:0/816190242 >> 172.21.15.29:0/816190242 conn(0x7f51f00c5640 msgr2=0x7f51f00c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:35.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.347+0000 7f51f7500700 1 -- 172.21.15.29:0/816190242 shutdown_connections 2024-01-24T13:58:35.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.347+0000 7f51f7500700 1 -- 172.21.15.29:0/816190242 wait complete. 2024-01-24T13:58:35.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.348+0000 7f51f7500700 1 Processor -- start 2024-01-24T13:58:35.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.348+0000 7f51f7500700 1 -- start start 2024-01-24T13:58:35.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.349+0000 7f51f7500700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f00638c0 0x7f51f015a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.349+0000 7f51f7500700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f0066bb0 0x7f51f00a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.349+0000 7f51f7500700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51f0151de0 0x7f51f013d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.349+0000 7f51f7500700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f00ac360 con 0x7f51f0151de0 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.349+0000 7f51f7500700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f00abb50 con 0x7f51f0066bb0 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.349+0000 7f51f7500700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f00adaf0 con 0x7f51f00638c0 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.349+0000 7f51f5a9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51f0151de0 0x7f51f013d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.349+0000 7f51f5a9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51f0151de0 0x7f51f013d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41278/0 (socket says 172.21.15.29:41278) 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.349+0000 7f51f5a9d700 1 -- 172.21.15.29:0/2655377338 learned_addr learned my addr 172.21.15.29:0/2655377338 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.349+0000 7f51f529c700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f00638c0 0x7f51f015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.350+0000 7f51f4a9b700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f0066bb0 0x7f51f00a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.350+0000 7f51f5a9d700 1 -- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f00638c0 msgr2=0x7f51f015a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.350+0000 7f51f5a9d700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f00638c0 0x7f51f015a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.350+0000 7f51f5a9d700 1 -- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f0066bb0 msgr2=0x7f51f00a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.350+0000 7f51f5a9d700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f0066bb0 0x7f51f00a9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.350+0000 7f51f5a9d700 1 -- 172.21.15.29:0/2655377338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec009370 con 0x7f51f0151de0 2024-01-24T13:58:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.350+0000 7f51f5a9d700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51f0151de0 0x7f51f013d790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f51e8007f60 tx=0x7f51e80098e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.350+0000 7f51e17fa700 1 -- 172.21.15.29:0/2655377338 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e800eb90 con 0x7f51f0151de0 2024-01-24T13:58:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.350+0000 7f51e17fa700 1 -- 172.21.15.29:0/2655377338 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e800ecf0 con 0x7f51f0151de0 2024-01-24T13:58:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.350+0000 7f51f7500700 1 -- 172.21.15.29:0/2655377338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f00abda0 con 0x7f51f0151de0 2024-01-24T13:58:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.350+0000 7f51e17fa700 1 -- 172.21.15.29:0/2655377338 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8014610 con 0x7f51f0151de0 2024-01-24T13:58:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.350+0000 7f51f7500700 1 -- 172.21.15.29:0/2655377338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f00ad750 con 0x7f51f0151de0 2024-01-24T13:58:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.351+0000 7f51f7500700 1 -- 172.21.15.29:0/2655377338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f0151580 con 0x7f51f0151de0 2024-01-24T13:58:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.352+0000 7f51e17fa700 1 -- 172.21.15.29:0/2655377338 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f51e8014a40 con 0x7f51f0151de0 2024-01-24T13:58:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.352+0000 7f51e17fa700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f51d405f150 0x7f51d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.352+0000 7f51e17fa700 1 -- 172.21.15.29:0/2655377338 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f51e807a4f0 con 0x7f51f0151de0 2024-01-24T13:58:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.352+0000 7f51f529c700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f51d405f150 0x7f51d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.353+0000 7f51f529c700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f51d405f150 0x7f51d4061610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f51ec009130 tx=0x7f51ec017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:35.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.356+0000 7f51e17fa700 1 -- 172.21.15.29:0/2655377338 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f51e8049b00 con 0x7f51f0151de0 2024-01-24T13:58:35.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.539+0000 7f51f7500700 1 -- 172.21.15.29:0/2655377338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f51f00475e0 con 0x7f51f0151de0 2024-01-24T13:58:35.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.539+0000 7f51e17fa700 1 -- 172.21.15.29:0/2655377338 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f51e8019070 con 0x7f51f0151de0 2024-01-24T13:58:35.540 INFO:teuthology.orchestra.run.smithi029.stdout:60129542151 2024-01-24T13:58:35.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.541+0000 7f51f7500700 1 -- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f51d405f150 msgr2=0x7f51d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.541+0000 7f51f7500700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f51d405f150 0x7f51d4061610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f51ec009130 tx=0x7f51ec017000).stop 2024-01-24T13:58:35.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.541+0000 7f51f7500700 1 -- 172.21.15.29:0/2655377338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51f0151de0 msgr2=0x7f51f013d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:35.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.541+0000 7f51f7500700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51f0151de0 0x7f51f013d790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f51e8007f60 tx=0x7f51e80098e0).stop 2024-01-24T13:58:35.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.541+0000 7f51f7500700 1 -- 172.21.15.29:0/2655377338 shutdown_connections 2024-01-24T13:58:35.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.541+0000 7f51f7500700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f51d405f150 0x7f51d4061610 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.541+0000 7f51f7500700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51f0151de0 0x7f51f013d790 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.541+0000 7f51f7500700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f00638c0 0x7f51f015a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.541+0000 7f51f7500700 1 --2- 172.21.15.29:0/2655377338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f0066bb0 0x7f51f00a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:35.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.541+0000 7f51f7500700 1 -- 172.21.15.29:0/2655377338 >> 172.21.15.29:0/2655377338 conn(0x7f51f00c5640 msgr2=0x7f51f00c5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:35.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.542+0000 7f51f7500700 1 -- 172.21.15.29:0/2655377338 shutdown_connections 2024-01-24T13:58:35.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:35.542+0000 7f51f7500700 1 -- 172.21.15.29:0/2655377338 wait complete. 2024-01-24T13:58:35.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.0 2024-01-24T13:58:36.556 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-24T13:58:36.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.699+0000 7fc655fae700 1 Processor -- start 2024-01-24T13:58:36.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.699+0000 7fc655fae700 1 -- start start 2024-01-24T13:58:36.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.700+0000 7fc655fae700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc650154240 0x7fc650064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:36.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.700+0000 7fc655fae700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6501550e0 con 0x7fc6500a9d20 2024-01-24T13:58:36.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.700+0000 7fc655fae700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc650158b10 con 0x7fc6500aa180 2024-01-24T13:58:36.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.700+0000 7fc655fae700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6500c0d10 con 0x7fc650154240 2024-01-24T13:58:36.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.700+0000 7fc64effd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc6500a9d20 0x7fc650157cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42136/0 (socket says 172.21.15.29:42136) 2024-01-24T13:58:36.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.700+0000 7fc64effd700 1 -- 172.21.15.29:0/606534655 learned_addr learned my addr 172.21.15.29:0/606534655 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:36.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.700+0000 7fc64f7fe700 1 --2- 172.21.15.29:0/606534655 >> v2:172.21.15.163:3300/0 conn(0x7fc650154240 0x7fc650064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:36.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.700+0000 7fc64e7fc700 1 -- 172.21.15.29:0/606534655 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942771227 0 0) 0x7fc6501550e0 con 0x7fc6500a9d20 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.700+0000 7fc64e7fc700 1 -- 172.21.15.29:0/606534655 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc63c0033f0 con 0x7fc6500a9d20 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.701+0000 7fc64e7fc700 1 -- 172.21.15.29:0/606534655 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894432647 0 0) 0x7fc650158b10 con 0x7fc6500aa180 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.701+0000 7fc64e7fc700 1 -- 172.21.15.29:0/606534655 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6501550e0 con 0x7fc6500aa180 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.701+0000 7fc64f7fe700 1 -- 172.21.15.29:0/606534655 >> v1:172.21.15.163:6789/0 conn(0x7fc6500aa180 legacy=0x7fc650158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.701+0000 7fc64f7fe700 1 -- 172.21.15.29:0/606534655 >> v1:172.21.15.29:6789/0 conn(0x7fc6500a9d20 legacy=0x7fc650157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.701+0000 7fc64e7fc700 1 -- 172.21.15.29:0/606534655 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 210213332 0 0) 0x7fc63c0033f0 con 0x7fc6500a9d20 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.701+0000 7fc64f7fe700 1 -- 172.21.15.29:0/606534655 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6500650d0 con 0x7fc650154240 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.701+0000 7fc64f7fe700 1 --2- 172.21.15.29:0/606534655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc650154240 0x7fc650064780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fc640009760 tx=0x7fc640009060).ready entity=mon.1 client_cookie=4ab6762e7739482d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.701+0000 7fc64e7fc700 1 -- 172.21.15.29:0/606534655 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638001470 con 0x7fc650154240 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.701+0000 7fc64e7fc700 1 -- 172.21.15.29:0/606534655 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc640011400 con 0x7fc650154240 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.701+0000 7fc64e7fc700 1 -- 172.21.15.29:0/606534655 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640011780 con 0x7fc650154240 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.702+0000 7fc655fae700 1 -- 172.21.15.29:0/606534655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc650154240 msgr2=0x7fc650064780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.702+0000 7fc655fae700 1 --2- 172.21.15.29:0/606534655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc650154240 0x7fc650064780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fc640009760 tx=0x7fc640009060).stop 2024-01-24T13:58:36.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.702+0000 7fc655fae700 1 -- 172.21.15.29:0/606534655 shutdown_connections 2024-01-24T13:58:36.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.702+0000 7fc655fae700 1 --2- 172.21.15.29:0/606534655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc650154240 0x7fc650064780 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:36.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.702+0000 7fc655fae700 1 -- 172.21.15.29:0/606534655 >> 172.21.15.29:0/606534655 conn(0x7fc6500c8310 msgr2=0x7fc65005e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:36.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.702+0000 7fc655fae700 1 -- 172.21.15.29:0/606534655 shutdown_connections 2024-01-24T13:58:36.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.702+0000 7fc655fae700 1 -- 172.21.15.29:0/606534655 wait complete. 2024-01-24T13:58:36.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.703+0000 7fc655fae700 1 Processor -- start 2024-01-24T13:58:36.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.703+0000 7fc655fae700 1 -- start start 2024-01-24T13:58:36.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc655fae700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6500a9d20 0x7fc65014ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:36.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc655fae700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6500aa180 0x7fc6500add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:36.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc655fae700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc650154240 0x7fc6500ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:36.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc655fae700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65014bf30 con 0x7fc650154240 2024-01-24T13:58:36.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc655fae700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500ae730 con 0x7fc6500aa180 2024-01-24T13:58:36.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc655fae700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500b03d0 con 0x7fc6500a9d20 2024-01-24T13:58:36.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc64ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc650154240 0x7fc6500ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc64f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6500a9d20 0x7fc65014ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc64effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6500aa180 0x7fc6500add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc64f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6500a9d20 0x7fc65014ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:40446/0 (socket says 172.21.15.29:40446) 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc64f7fe700 1 -- 172.21.15.29:0/1016959518 learned_addr learned my addr 172.21.15.29:0/1016959518 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc64ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc650154240 0x7fc6500ae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41290/0 (socket says 172.21.15.29:41290) 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc64ffff700 1 -- 172.21.15.29:0/1016959518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6500a9d20 msgr2=0x7fc65014ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc64ffff700 1 --2- 172.21.15.29:0/1016959518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6500a9d20 0x7fc65014ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc64ffff700 1 -- 172.21.15.29:0/1016959518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6500aa180 msgr2=0x7fc6500add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.704+0000 7fc64ffff700 1 --2- 172.21.15.29:0/1016959518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6500aa180 0x7fc6500add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.705+0000 7fc64ffff700 1 -- 172.21.15.29:0/1016959518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6400092a0 con 0x7fc650154240 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.705+0000 7fc64ffff700 1 --2- 172.21.15.29:0/1016959518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc650154240 0x7fc6500ae240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fc64800c790 tx=0x7fc648007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.705+0000 7fc6477fe700 1 -- 172.21.15.29:0/1016959518 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64800e4a0 con 0x7fc650154240 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.705+0000 7fc6477fe700 1 -- 172.21.15.29:0/1016959518 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64800e600 con 0x7fc650154240 2024-01-24T13:58:36.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.705+0000 7fc655fae700 1 -- 172.21.15.29:0/1016959518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6500afe10 con 0x7fc650154240 2024-01-24T13:58:36.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.705+0000 7fc6477fe700 1 -- 172.21.15.29:0/1016959518 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64800e980 con 0x7fc650154240 2024-01-24T13:58:36.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.705+0000 7fc655fae700 1 -- 172.21.15.29:0/1016959518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6500c2050 con 0x7fc650154240 2024-01-24T13:58:36.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.706+0000 7fc655fae700 1 -- 172.21.15.29:0/1016959518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6500c1ea0 con 0x7fc650154240 2024-01-24T13:58:36.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.706+0000 7fc6477fe700 1 -- 172.21.15.29:0/1016959518 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc648012400 con 0x7fc650154240 2024-01-24T13:58:36.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.707+0000 7fc6477fe700 1 --2- 172.21.15.29:0/1016959518 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc63005f120 0x7fc6300615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:36.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.707+0000 7fc6477fe700 1 -- 172.21.15.29:0/1016959518 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fc648079b70 con 0x7fc650154240 2024-01-24T13:58:36.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.707+0000 7fc64f7fe700 1 --2- 172.21.15.29:0/1016959518 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc63005f120 0x7fc6300615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:36.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.708+0000 7fc64f7fe700 1 --2- 172.21.15.29:0/1016959518 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc63005f120 0x7fc6300615e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fc650083ff0 tx=0x7fc640016000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:36.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.711+0000 7fc6477fe700 1 -- 172.21.15.29:0/1016959518 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc648048b00 con 0x7fc650154240 2024-01-24T13:58:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.894+0000 7fc655fae700 1 -- 172.21.15.29:0/1016959518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc650154a90 con 0x7fc650154240 2024-01-24T13:58:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.894+0000 7fc6477fe700 1 -- 172.21.15.29:0/1016959518 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc648017070 con 0x7fc650154240 2024-01-24T13:58:36.895 INFO:teuthology.orchestra.run.smithi029.stdout:60129542152 2024-01-24T13:58:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.896+0000 7fc655fae700 1 -- 172.21.15.29:0/1016959518 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc63005f120 msgr2=0x7fc6300615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.896+0000 7fc655fae700 1 --2- 172.21.15.29:0/1016959518 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc63005f120 0x7fc6300615e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fc650083ff0 tx=0x7fc640016000).stop 2024-01-24T13:58:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.896+0000 7fc655fae700 1 -- 172.21.15.29:0/1016959518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc650154240 msgr2=0x7fc6500ae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.896+0000 7fc655fae700 1 --2- 172.21.15.29:0/1016959518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc650154240 0x7fc6500ae240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fc64800c790 tx=0x7fc648007ef0).stop 2024-01-24T13:58:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.896+0000 7fc655fae700 1 -- 172.21.15.29:0/1016959518 shutdown_connections 2024-01-24T13:58:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.896+0000 7fc655fae700 1 --2- 172.21.15.29:0/1016959518 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc63005f120 0x7fc6300615e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.896+0000 7fc655fae700 1 --2- 172.21.15.29:0/1016959518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc650154240 0x7fc6500ae240 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.896+0000 7fc655fae700 1 --2- 172.21.15.29:0/1016959518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6500a9d20 0x7fc65014ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.896+0000 7fc655fae700 1 --2- 172.21.15.29:0/1016959518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6500aa180 0x7fc6500add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.896+0000 7fc655fae700 1 -- 172.21.15.29:0/1016959518 >> 172.21.15.29:0/1016959518 conn(0x7fc6500c8310 msgr2=0x7fc6500ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.896+0000 7fc655fae700 1 -- 172.21.15.29:0/1016959518 shutdown_connections 2024-01-24T13:58:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:36.896+0000 7fc655fae700 1 -- 172.21.15.29:0/1016959518 wait complete. 2024-01-24T13:58:36.910 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.0 2024-01-24T13:58:36.910 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-24T13:58:37.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.041+0000 7f275e5b6700 1 Processor -- start 2024-01-24T13:58:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.041+0000 7f275e5b6700 1 -- start start 2024-01-24T13:58:37.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.042+0000 7f275e5b6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2758151de0 0x7f2758155470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.042+0000 7f275e5b6700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2758158800 con 0x7f27580638c0 2024-01-24T13:58:37.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.042+0000 7f275e5b6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2758157ed0 con 0x7f2758066bb0 2024-01-24T13:58:37.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.042+0000 7f275e5b6700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27581581b0 con 0x7f2758151de0 2024-01-24T13:58:37.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.042+0000 7f2757fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2758151de0 0x7f2758155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.042+0000 7f2757fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2758151de0 0x7f2758155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41294/0 (socket says 172.21.15.29:41294) 2024-01-24T13:58:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.042+0000 7f2757fff700 1 -- 172.21.15.29:0/667222523 learned_addr learned my addr 172.21.15.29:0/667222523 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.042+0000 7f27577fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f27580638c0 0x7f27580664a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42152/0 (socket says 172.21.15.29:42152) 2024-01-24T13:58:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f2757fff700 1 -- 172.21.15.29:0/667222523 >> v1:172.21.15.163:6789/0 conn(0x7f2758066bb0 legacy=0x7f275813d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f2757fff700 1 -- 172.21.15.29:0/667222523 >> v1:172.21.15.29:6789/0 conn(0x7f27580638c0 legacy=0x7f27580664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f2757fff700 1 -- 172.21.15.29:0/667222523 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2758158490 con 0x7f2758151de0 2024-01-24T13:58:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f2756ffd700 1 -- 172.21.15.29:0/667222523 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827000287 0 0) 0x7f2758158800 con 0x7f27580638c0 2024-01-24T13:58:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f2757fff700 1 --2- 172.21.15.29:0/667222523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2758151de0 0x7f2758155470 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f2740006fd0 tx=0x7f27400090f0).ready entity=mon.0 client_cookie=a8a84062d997be29 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f2756ffd700 1 -- 172.21.15.29:0/667222523 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274000b600 con 0x7f2758151de0 2024-01-24T13:58:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f2756ffd700 1 -- 172.21.15.29:0/667222523 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274000b760 con 0x7f2758151de0 2024-01-24T13:58:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f2756ffd700 1 -- 172.21.15.29:0/667222523 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274000bae0 con 0x7f2758151de0 2024-01-24T13:58:37.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f275e5b6700 1 -- 172.21.15.29:0/667222523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2758151de0 msgr2=0x7f2758155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f275e5b6700 1 --2- 172.21.15.29:0/667222523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2758151de0 0x7f2758155470 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f2740006fd0 tx=0x7f27400090f0).stop 2024-01-24T13:58:37.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f275e5b6700 1 -- 172.21.15.29:0/667222523 shutdown_connections 2024-01-24T13:58:37.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f275e5b6700 1 --2- 172.21.15.29:0/667222523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2758151de0 0x7f2758155470 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.043+0000 7f275e5b6700 1 -- 172.21.15.29:0/667222523 >> 172.21.15.29:0/667222523 conn(0x7f27580c5640 msgr2=0x7f27580c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:37.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.044+0000 7f275e5b6700 1 -- 172.21.15.29:0/667222523 shutdown_connections 2024-01-24T13:58:37.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.044+0000 7f275e5b6700 1 -- 172.21.15.29:0/667222523 wait complete. 2024-01-24T13:58:37.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.045+0000 7f275e5b6700 1 Processor -- start 2024-01-24T13:58:37.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.045+0000 7f275e5b6700 1 -- start start 2024-01-24T13:58:37.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f275e5b6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27580638c0 0x7f275813d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f275e5b6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2758066bb0 0x7f27580aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f275e5b6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2758151de0 0x7f2758156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f275e5b6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2758165770 con 0x7f2758066bb0 2024-01-24T13:58:37.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f275e5b6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27580ae9b0 con 0x7f27580638c0 2024-01-24T13:58:37.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f275e5b6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2758164b60 con 0x7f2758151de0 2024-01-24T13:58:37.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f27577fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2758066bb0 0x7f27580aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f27577fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2758066bb0 0x7f27580aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41298/0 (socket says 172.21.15.29:41298) 2024-01-24T13:58:37.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f27577fe700 1 -- 172.21.15.29:0/4056002749 learned_addr learned my addr 172.21.15.29:0/4056002749 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f2757fff700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27580638c0 0x7f275813d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f275cb53700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2758151de0 0x7f2758156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f27577fe700 1 -- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2758151de0 msgr2=0x7f2758156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f27577fe700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2758151de0 0x7f2758156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f27577fe700 1 -- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27580638c0 msgr2=0x7f275813d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f27577fe700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27580638c0 0x7f275813d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f27577fe700 1 -- 172.21.15.29:0/4056002749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2740009370 con 0x7f2758066bb0 2024-01-24T13:58:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f2757fff700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27580638c0 0x7f275813d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.046+0000 7f27577fe700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2758066bb0 0x7f27580aae40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f274800c4c0 tx=0x7f2748007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.047+0000 7f274ffff700 1 -- 172.21.15.29:0/4056002749 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27480072e0 con 0x7f2758066bb0 2024-01-24T13:58:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.047+0000 7f274ffff700 1 -- 172.21.15.29:0/4056002749 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2748007440 con 0x7f2758066bb0 2024-01-24T13:58:37.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.047+0000 7f274ffff700 1 -- 172.21.15.29:0/4056002749 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748005930 con 0x7f2758066bb0 2024-01-24T13:58:37.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.047+0000 7f275e5b6700 1 -- 172.21.15.29:0/4056002749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2758164e40 con 0x7f2758066bb0 2024-01-24T13:58:37.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.047+0000 7f275e5b6700 1 -- 172.21.15.29:0/4056002749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2758165400 con 0x7f2758066bb0 2024-01-24T13:58:37.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.048+0000 7f275e5b6700 1 -- 172.21.15.29:0/4056002749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27580475e0 con 0x7f2758066bb0 2024-01-24T13:58:37.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.048+0000 7f274ffff700 1 -- 172.21.15.29:0/4056002749 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2748005ce0 con 0x7f2758066bb0 2024-01-24T13:58:37.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.049+0000 7f274ffff700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f273805f150 0x7f2738061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.049+0000 7f274ffff700 1 -- 172.21.15.29:0/4056002749 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f2748079d10 con 0x7f2758066bb0 2024-01-24T13:58:37.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.049+0000 7f2757fff700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f273805f150 0x7f2738061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.049+0000 7f2757fff700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f273805f150 0x7f2738061610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f2740009130 tx=0x7f2740017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:37.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.053+0000 7f274ffff700 1 -- 172.21.15.29:0/4056002749 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f274804acd0 con 0x7f2758066bb0 2024-01-24T13:58:37.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.236+0000 7f275e5b6700 1 -- 172.21.15.29:0/4056002749 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f275813e020 con 0x7f2758066bb0 2024-01-24T13:58:37.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.236+0000 7f274ffff700 1 -- 172.21.15.29:0/4056002749 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f274804e2f0 con 0x7f2758066bb0 2024-01-24T13:58:37.237 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-01-24T13:58:37.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.238+0000 7f275e5b6700 1 -- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f273805f150 msgr2=0x7f2738061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.238+0000 7f275e5b6700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f273805f150 0x7f2738061610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f2740009130 tx=0x7f2740017000).stop 2024-01-24T13:58:37.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.238+0000 7f275e5b6700 1 -- 172.21.15.29:0/4056002749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2758066bb0 msgr2=0x7f27580aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.238+0000 7f275e5b6700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2758066bb0 0x7f27580aae40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f274800c4c0 tx=0x7f2748007c10).stop 2024-01-24T13:58:37.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.238+0000 7f275e5b6700 1 -- 172.21.15.29:0/4056002749 shutdown_connections 2024-01-24T13:58:37.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.238+0000 7f275e5b6700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2758151de0 0x7f2758156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.238+0000 7f275e5b6700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27580638c0 0x7f275813d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.238+0000 7f275e5b6700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f273805f150 0x7f2738061610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.238+0000 7f275e5b6700 1 --2- 172.21.15.29:0/4056002749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2758066bb0 0x7f27580aae40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.238+0000 7f275e5b6700 1 -- 172.21.15.29:0/4056002749 >> 172.21.15.29:0/4056002749 conn(0x7f27580c5640 msgr2=0x7f2758153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:37.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.238+0000 7f275e5b6700 1 -- 172.21.15.29:0/4056002749 shutdown_connections 2024-01-24T13:58:37.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.238+0000 7f275e5b6700 1 -- 172.21.15.29:0/4056002749 wait complete. 2024-01-24T13:58:37.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-01-24T13:58:37.252 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-24T13:58:37.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.383+0000 7effaaf44700 1 Processor -- start 2024-01-24T13:58:37.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.383+0000 7effaaf44700 1 -- start start 2024-01-24T13:58:37.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effaaf44700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7effa4150bd0 0x7effa4065e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effaaf44700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa4069240 0x7effa4150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effaaf44700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7effa41514a0 0x7effa4151930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effaaf44700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa406a350 con 0x7effa4150bd0 2024-01-24T13:58:37.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effaaf44700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa4069740 con 0x7effa41514a0 2024-01-24T13:58:37.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effaaf44700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa4069a20 con 0x7effa4069240 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effa8ce0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7effa4150bd0 0x7effa4065e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effa8ce0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7effa4150bd0 0x7effa4065e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41312/0 (socket says 172.21.15.29:41312) 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effa8ce0700 1 -- 172.21.15.29:0/3717024119 learned_addr learned my addr 172.21.15.29:0/3717024119 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effa3fff700 1 --2- 172.21.15.29:0/3717024119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa4069240 0x7effa4150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effa94e1700 1 --2- 172.21.15.29:0/3717024119 >> v2:172.21.15.163:3300/0 conn(0x7effa41514a0 0x7effa4151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effa8ce0700 1 -- 172.21.15.29:0/3717024119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa4069240 msgr2=0x7effa4150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effa8ce0700 1 --2- 172.21.15.29:0/3717024119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa4069240 0x7effa4150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effa8ce0700 1 -- 172.21.15.29:0/3717024119 >> v2:172.21.15.163:3300/0 conn(0x7effa41514a0 msgr2=0x7effa4151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effa8ce0700 1 --2- 172.21.15.29:0/3717024119 >> v2:172.21.15.163:3300/0 conn(0x7effa41514a0 0x7effa4151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.384+0000 7effa8ce0700 1 -- 172.21.15.29:0/3717024119 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa4069d00 con 0x7effa4150bd0 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.385+0000 7effa8ce0700 1 --2- 172.21.15.29:0/3717024119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa4150bd0 0x7effa4065e50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7effa4154840 tx=0x7eff980090f0).ready entity=mon.0 client_cookie=7783e24488ea3823 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.385+0000 7effa37fe700 1 -- 172.21.15.29:0/3717024119 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800f510 con 0x7effa4150bd0 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.385+0000 7effa37fe700 1 -- 172.21.15.29:0/3717024119 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9800f670 con 0x7effa4150bd0 2024-01-24T13:58:37.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.385+0000 7effa37fe700 1 -- 172.21.15.29:0/3717024119 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800f9f0 con 0x7effa4150bd0 2024-01-24T13:58:37.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.385+0000 7effaaf44700 1 -- 172.21.15.29:0/3717024119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa4150bd0 msgr2=0x7effa4065e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.385+0000 7effaaf44700 1 --2- 172.21.15.29:0/3717024119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa4150bd0 0x7effa4065e50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7effa4154840 tx=0x7eff980090f0).stop 2024-01-24T13:58:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.385+0000 7effaaf44700 1 -- 172.21.15.29:0/3717024119 shutdown_connections 2024-01-24T13:58:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.385+0000 7effaaf44700 1 --2- 172.21.15.29:0/3717024119 >> v2:172.21.15.163:3300/0 conn(0x7effa41514a0 0x7effa4151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.385+0000 7effaaf44700 1 --2- 172.21.15.29:0/3717024119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa4150bd0 0x7effa4065e50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.385+0000 7effaaf44700 1 --2- 172.21.15.29:0/3717024119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa4069240 0x7effa4150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.385+0000 7effaaf44700 1 -- 172.21.15.29:0/3717024119 >> 172.21.15.29:0/3717024119 conn(0x7effa40c8580 msgr2=0x7effa40c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.385+0000 7effaaf44700 1 -- 172.21.15.29:0/3717024119 shutdown_connections 2024-01-24T13:58:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.386+0000 7effaaf44700 1 -- 172.21.15.29:0/3717024119 wait complete. 2024-01-24T13:58:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.387+0000 7effaaf44700 1 Processor -- start 2024-01-24T13:58:37.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.387+0000 7effaaf44700 1 -- start start 2024-01-24T13:58:37.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effaaf44700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa4069240 0x7effa40bf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effaaf44700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4150bd0 0x7effa4147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effaaf44700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa41514a0 0x7effa40aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effaaf44700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa40c05c0 con 0x7effa41514a0 2024-01-24T13:58:37.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effaaf44700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa414ba40 con 0x7effa4150bd0 2024-01-24T13:58:37.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effaaf44700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa4149160 con 0x7effa4069240 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa94e1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa41514a0 0x7effa40aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa94e1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa41514a0 0x7effa40aaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41322/0 (socket says 172.21.15.29:41322) 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa94e1700 1 -- 172.21.15.29:0/1572056788 learned_addr learned my addr 172.21.15.29:0/1572056788 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa8ce0700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa4069240 0x7effa40bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa3fff700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4150bd0 0x7effa4147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa94e1700 1 -- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa4069240 msgr2=0x7effa40bf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa94e1700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa4069240 0x7effa40bf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa94e1700 1 -- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4150bd0 msgr2=0x7effa4147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa94e1700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4150bd0 0x7effa4147ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa94e1700 1 -- 172.21.15.29:0/1572056788 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff98009370 con 0x7effa41514a0 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa94e1700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa41514a0 0x7effa40aaaa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7eff9c009e90 tx=0x7eff9c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa0ff9700 1 -- 172.21.15.29:0/1572056788 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c009bf0 con 0x7effa41514a0 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.388+0000 7effa0ff9700 1 -- 172.21.15.29:0/1572056788 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c00f040 con 0x7effa41514a0 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.389+0000 7effaaf44700 1 -- 172.21.15.29:0/1572056788 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa40bfab0 con 0x7effa41514a0 2024-01-24T13:58:37.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.389+0000 7effa0ff9700 1 -- 172.21.15.29:0/1572056788 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00ce30 con 0x7effa41514a0 2024-01-24T13:58:37.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.389+0000 7effaaf44700 1 -- 172.21.15.29:0/1572056788 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa40c02e0 con 0x7effa41514a0 2024-01-24T13:58:37.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.390+0000 7effaaf44700 1 -- 172.21.15.29:0/1572056788 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa40bfdb0 con 0x7effa41514a0 2024-01-24T13:58:37.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.390+0000 7effa0ff9700 1 -- 172.21.15.29:0/1572056788 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7eff9c013610 con 0x7effa41514a0 2024-01-24T13:58:37.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.390+0000 7effa0ff9700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff8805f150 0x7eff88061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.391+0000 7effa0ff9700 1 -- 172.21.15.29:0/1572056788 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7eff9c07ae90 con 0x7effa41514a0 2024-01-24T13:58:37.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.391+0000 7effa8ce0700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff8805f150 0x7eff88061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.391+0000 7effa8ce0700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff8805f150 0x7eff88061610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7eff98009130 tx=0x7eff98017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:37.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.395+0000 7effa0ff9700 1 -- 172.21.15.29:0/1572056788 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7eff9c04bd90 con 0x7effa41514a0 2024-01-24T13:58:37.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.577+0000 7effaaf44700 1 -- 172.21.15.29:0/1572056788 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7effa40c5140 con 0x7effa41514a0 2024-01-24T13:58:37.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.577+0000 7effa0ff9700 1 -- 172.21.15.29:0/1572056788 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff9c07c070 con 0x7effa41514a0 2024-01-24T13:58:37.577 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-01-24T13:58:37.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.579+0000 7effaaf44700 1 -- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff8805f150 msgr2=0x7eff88061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.579+0000 7effaaf44700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff8805f150 0x7eff88061610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7eff98009130 tx=0x7eff98017000).stop 2024-01-24T13:58:37.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.579+0000 7effaaf44700 1 -- 172.21.15.29:0/1572056788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa41514a0 msgr2=0x7effa40aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.579+0000 7effaaf44700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa41514a0 0x7effa40aaaa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7eff9c009e90 tx=0x7eff9c00b040).stop 2024-01-24T13:58:37.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.579+0000 7effaaf44700 1 -- 172.21.15.29:0/1572056788 shutdown_connections 2024-01-24T13:58:37.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.579+0000 7effaaf44700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff8805f150 0x7eff88061610 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.579+0000 7effaaf44700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa41514a0 0x7effa40aaaa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.579+0000 7effaaf44700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa4069240 0x7effa40bf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.579+0000 7effaaf44700 1 --2- 172.21.15.29:0/1572056788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4150bd0 0x7effa4147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.579+0000 7effaaf44700 1 -- 172.21.15.29:0/1572056788 >> 172.21.15.29:0/1572056788 conn(0x7effa40c8580 msgr2=0x7effa4153f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:37.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.579+0000 7effaaf44700 1 -- 172.21.15.29:0/1572056788 shutdown_connections 2024-01-24T13:58:37.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.579+0000 7effaaf44700 1 -- 172.21.15.29:0/1572056788 wait complete. 2024-01-24T13:58:37.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-01-24T13:58:37.593 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-24T13:58:37.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.724+0000 7fe372c50700 1 Processor -- start 2024-01-24T13:58:37.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.724+0000 7fe372c50700 1 -- start start 2024-01-24T13:58:37.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.725+0000 7fe372c50700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe36c154240 0x7fe36c064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.725+0000 7fe372c50700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe36c0a9d20 0x7fe36c0aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.725+0000 7fe372c50700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c1550e0 con 0x7fe36c157cf0 2024-01-24T13:58:37.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.725+0000 7fe372c50700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c0c0d10 con 0x7fe36c0a9d20 2024-01-24T13:58:37.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.725+0000 7fe372c50700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0aa670 con 0x7fe36c154240 2024-01-24T13:58:37.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.725+0000 7fe3711ed700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe36c157cf0 0x7fe36c158180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42166/0 (socket says 172.21.15.29:42166) 2024-01-24T13:58:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.725+0000 7fe3711ed700 1 -- 172.21.15.29:0/3865504279 learned_addr learned my addr 172.21.15.29:0/3865504279 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.725+0000 7fe3709ec700 1 --2- 172.21.15.29:0/3865504279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe36c154240 0x7fe36c064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.725+0000 7fe36bfff700 1 --2- 172.21.15.29:0/3865504279 >> v2:172.21.15.163:3300/0 conn(0x7fe36c0a9d20 0x7fe36c0aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.725+0000 7fe36b7fe700 1 -- 172.21.15.29:0/3865504279 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378447434 0 0) 0x7fe36c1550e0 con 0x7fe36c157cf0 2024-01-24T13:58:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.725+0000 7fe36b7fe700 1 -- 172.21.15.29:0/3865504279 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe35c0033f0 con 0x7fe36c157cf0 2024-01-24T13:58:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.726+0000 7fe36bfff700 1 -- 172.21.15.29:0/3865504279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe36c154240 msgr2=0x7fe36c064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.726+0000 7fe36bfff700 1 --2- 172.21.15.29:0/3865504279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe36c154240 0x7fe36c064780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.726+0000 7fe36bfff700 1 -- 172.21.15.29:0/3865504279 >> v1:172.21.15.29:6789/0 conn(0x7fe36c157cf0 legacy=0x7fe36c158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.726+0000 7fe36bfff700 1 -- 172.21.15.29:0/3865504279 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0650d0 con 0x7fe36c0a9d20 2024-01-24T13:58:37.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.726+0000 7fe3709ec700 1 --2- 172.21.15.29:0/3865504279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe36c154240 0x7fe36c064780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:37.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.726+0000 7fe36bfff700 1 --2- 172.21.15.29:0/3865504279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36c0a9d20 0x7fe36c0aa180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fe358009760 tx=0x7fe358009060).ready entity=mon.1 client_cookie=c71738106e6a02fa server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:37.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.726+0000 7fe36b7fe700 1 -- 172.21.15.29:0/3865504279 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800d4a0 con 0x7fe36c0a9d20 2024-01-24T13:58:37.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.726+0000 7fe36b7fe700 1 -- 172.21.15.29:0/3865504279 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35800d600 con 0x7fe36c0a9d20 2024-01-24T13:58:37.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.726+0000 7fe36b7fe700 1 -- 172.21.15.29:0/3865504279 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800d980 con 0x7fe36c0a9d20 2024-01-24T13:58:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.726+0000 7fe372c50700 1 -- 172.21.15.29:0/3865504279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36c0a9d20 msgr2=0x7fe36c0aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.726+0000 7fe372c50700 1 --2- 172.21.15.29:0/3865504279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36c0a9d20 0x7fe36c0aa180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fe358009760 tx=0x7fe358009060).stop 2024-01-24T13:58:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.727+0000 7fe372c50700 1 -- 172.21.15.29:0/3865504279 shutdown_connections 2024-01-24T13:58:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.727+0000 7fe372c50700 1 --2- 172.21.15.29:0/3865504279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe36c154240 0x7fe36c064780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.727+0000 7fe372c50700 1 --2- 172.21.15.29:0/3865504279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36c0a9d20 0x7fe36c0aa180 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.727+0000 7fe372c50700 1 -- 172.21.15.29:0/3865504279 >> 172.21.15.29:0/3865504279 conn(0x7fe36c0c8310 msgr2=0x7fe36c05e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:37.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.727+0000 7fe372c50700 1 -- 172.21.15.29:0/3865504279 shutdown_connections 2024-01-24T13:58:37.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.727+0000 7fe372c50700 1 -- 172.21.15.29:0/3865504279 wait complete. 2024-01-24T13:58:37.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.728+0000 7fe372c50700 1 Processor -- start 2024-01-24T13:58:37.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.729+0000 7fe372c50700 1 -- start start 2024-01-24T13:58:37.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe372c50700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c0a9d20 0x7fe36c0add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe372c50700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe36c154240 0x7fe36c0ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe372c50700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36c157cf0 0x7fe36c0afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe372c50700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0ae710 con 0x7fe36c0a9d20 2024-01-24T13:58:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe372c50700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c14bca0 con 0x7fe36c157cf0 2024-01-24T13:58:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe372c50700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c151da0 con 0x7fe36c154240 2024-01-24T13:58:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe3709ec700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c0a9d20 0x7fe36c0add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe3709ec700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c0a9d20 0x7fe36c0add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41328/0 (socket says 172.21.15.29:41328) 2024-01-24T13:58:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe3709ec700 1 -- 172.21.15.29:0/194109894 learned_addr learned my addr 172.21.15.29:0/194109894 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe36bfff700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe36c154240 0x7fe36c0ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe3709ec700 1 -- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe36c154240 msgr2=0x7fe36c0ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe3709ec700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe36c154240 0x7fe36c0ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe3711ed700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36c157cf0 0x7fe36c0afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe3709ec700 1 -- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36c157cf0 msgr2=0x7fe36c0afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe3709ec700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36c157cf0 0x7fe36c0afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe3709ec700 1 -- 172.21.15.29:0/194109894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3600092a0 con 0x7fe36c0a9d20 2024-01-24T13:58:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe36bfff700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe36c154240 0x7fe36c0ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe3709ec700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c0a9d20 0x7fe36c0add30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fe36000ea20 tx=0x7fe360009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe368ff9700 1 -- 172.21.15.29:0/194109894 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36000c850 con 0x7fe36c0a9d20 2024-01-24T13:58:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe368ff9700 1 -- 172.21.15.29:0/194109894 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36000c9b0 con 0x7fe36c0a9d20 2024-01-24T13:58:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe372c50700 1 -- 172.21.15.29:0/194109894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3580092a0 con 0x7fe36c0a9d20 2024-01-24T13:58:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe368ff9700 1 -- 172.21.15.29:0/194109894 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36000cd30 con 0x7fe36c0a9d20 2024-01-24T13:58:37.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.730+0000 7fe372c50700 1 -- 172.21.15.29:0/194109894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c0ab140 con 0x7fe36c0a9d20 2024-01-24T13:58:37.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.732+0000 7fe372c50700 1 -- 172.21.15.29:0/194109894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c0475e0 con 0x7fe36c0a9d20 2024-01-24T13:58:37.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.732+0000 7fe368ff9700 1 -- 172.21.15.29:0/194109894 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe360012600 con 0x7fe36c0a9d20 2024-01-24T13:58:37.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.732+0000 7fe368ff9700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe35005f150 0x7fe350061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:37.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.733+0000 7fe368ff9700 1 -- 172.21.15.29:0/194109894 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe36007ab30 con 0x7fe36c0a9d20 2024-01-24T13:58:37.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.733+0000 7fe36bfff700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe35005f150 0x7fe350061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:37.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.733+0000 7fe36bfff700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe35005f150 0x7fe350061610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fe358009760 tx=0x7fe358016000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:37.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.737+0000 7fe368ff9700 1 -- 172.21.15.29:0/194109894 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe36004baf0 con 0x7fe36c0a9d20 2024-01-24T13:58:37.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.919+0000 7fe372c50700 1 -- 172.21.15.29:0/194109894 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe36c154a90 con 0x7fe36c0a9d20 2024-01-24T13:58:37.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.920+0000 7fe368ff9700 1 -- 172.21.15.29:0/194109894 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe36004f110 con 0x7fe36c0a9d20 2024-01-24T13:58:37.920 INFO:teuthology.orchestra.run.smithi029.stdout:55834574854 2024-01-24T13:58:37.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.921+0000 7fe372c50700 1 -- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe35005f150 msgr2=0x7fe350061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.921+0000 7fe372c50700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe35005f150 0x7fe350061610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fe358009760 tx=0x7fe358016000).stop 2024-01-24T13:58:37.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.921+0000 7fe372c50700 1 -- 172.21.15.29:0/194109894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c0a9d20 msgr2=0x7fe36c0add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:37.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.921+0000 7fe372c50700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c0a9d20 0x7fe36c0add30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fe36000ea20 tx=0x7fe360009060).stop 2024-01-24T13:58:37.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.922+0000 7fe372c50700 1 -- 172.21.15.29:0/194109894 shutdown_connections 2024-01-24T13:58:37.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.922+0000 7fe372c50700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36c157cf0 0x7fe36c0afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.922+0000 7fe372c50700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe35005f150 0x7fe350061610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.922+0000 7fe372c50700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe36c0a9d20 0x7fe36c0add30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.922+0000 7fe372c50700 1 --2- 172.21.15.29:0/194109894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe36c154240 0x7fe36c0ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:37.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.922+0000 7fe372c50700 1 -- 172.21.15.29:0/194109894 >> 172.21.15.29:0/194109894 conn(0x7fe36c0c8310 msgr2=0x7fe36c155470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:37.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.922+0000 7fe372c50700 1 -- 172.21.15.29:0/194109894 shutdown_connections 2024-01-24T13:58:37.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:37.922+0000 7fe372c50700 1 -- 172.21.15.29:0/194109894 wait complete. 2024-01-24T13:58:37.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-01-24T13:58:37.936 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-24T13:58:38.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.065+0000 7ff0d00bc700 1 Processor -- start 2024-01-24T13:58:38.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.066+0000 7ff0d00bc700 1 -- start start 2024-01-24T13:58:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0d00bc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff0c8069240 0x7ff0c8150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0d00bc700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c806a350 con 0x7ff0c8150bd0 2024-01-24T13:58:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0d00bc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c8069740 con 0x7ff0c81514a0 2024-01-24T13:58:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0d00bc700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c8069a20 con 0x7ff0c8069240 2024-01-24T13:58:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0cd657700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff0c8069240 0x7ff0c8150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0cd657700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff0c8069240 0x7ff0c8150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41340/0 (socket says 172.21.15.29:41340) 2024-01-24T13:58:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0cde58700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff0c8150bd0 0x7ff0c8065e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42174/0 (socket says 172.21.15.29:42174) 2024-01-24T13:58:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0cd657700 1 -- 172.21.15.29:0/1243698903 learned_addr learned my addr 172.21.15.29:0/1243698903 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0cd657700 1 -- 172.21.15.29:0/1243698903 >> v1:172.21.15.163:6789/0 conn(0x7ff0c81514a0 legacy=0x7ff0c80bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0cd657700 1 -- 172.21.15.29:0/1243698903 >> v1:172.21.15.29:6789/0 conn(0x7ff0c8150bd0 legacy=0x7ff0c8065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0cd657700 1 -- 172.21.15.29:0/1243698903 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c8069d00 con 0x7ff0c8069240 2024-01-24T13:58:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0cce56700 1 -- 172.21.15.29:0/1243698903 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586596112 0 0) 0x7ff0c806a350 con 0x7ff0c8150bd0 2024-01-24T13:58:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0cd657700 1 --2- 172.21.15.29:0/1243698903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0c8069240 0x7ff0c8150fb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7ff0b8006fd0 tx=0x7ff0b800b3b0).ready entity=mon.0 client_cookie=9b1f3a02d1b8b887 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.067+0000 7ff0cce56700 1 -- 172.21.15.29:0/1243698903 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b800db40 con 0x7ff0c8069240 2024-01-24T13:58:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.068+0000 7ff0cce56700 1 -- 172.21.15.29:0/1243698903 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b800dca0 con 0x7ff0c8069240 2024-01-24T13:58:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.068+0000 7ff0cce56700 1 -- 172.21.15.29:0/1243698903 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8010680 con 0x7ff0c8069240 2024-01-24T13:58:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.068+0000 7ff0d00bc700 1 -- 172.21.15.29:0/1243698903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0c8069240 msgr2=0x7ff0c8150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.068+0000 7ff0d00bc700 1 --2- 172.21.15.29:0/1243698903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0c8069240 0x7ff0c8150fb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7ff0b8006fd0 tx=0x7ff0b800b3b0).stop 2024-01-24T13:58:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.068+0000 7ff0d00bc700 1 -- 172.21.15.29:0/1243698903 shutdown_connections 2024-01-24T13:58:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.068+0000 7ff0d00bc700 1 --2- 172.21.15.29:0/1243698903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0c8069240 0x7ff0c8150fb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.068+0000 7ff0d00bc700 1 -- 172.21.15.29:0/1243698903 >> 172.21.15.29:0/1243698903 conn(0x7ff0c80c8580 msgr2=0x7ff0c80c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.068+0000 7ff0d00bc700 1 -- 172.21.15.29:0/1243698903 shutdown_connections 2024-01-24T13:58:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.068+0000 7ff0d00bc700 1 -- 172.21.15.29:0/1243698903 wait complete. 2024-01-24T13:58:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.069+0000 7ff0d00bc700 1 Processor -- start 2024-01-24T13:58:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.069+0000 7ff0d00bc700 1 -- start start 2024-01-24T13:58:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.070+0000 7ff0d00bc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0c8069240 0x7ff0c80beb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.070+0000 7ff0d00bc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0c8150bd0 0x7ff0c80aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.070+0000 7ff0d00bc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0c81514a0 0x7ff0c813e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.070+0000 7ff0d00bc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80c0280 con 0x7ff0c8069240 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.070+0000 7ff0d00bc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80bf170 con 0x7ff0c8150bd0 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.070+0000 7ff0d00bc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80bf450 con 0x7ff0c81514a0 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.070+0000 7ff0cde58700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0c8069240 0x7ff0c80beb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.070+0000 7ff0cde58700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0c8069240 0x7ff0c80beb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41346/0 (socket says 172.21.15.29:41346) 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.070+0000 7ff0cde58700 1 -- 172.21.15.29:0/4001654980 learned_addr learned my addr 172.21.15.29:0/4001654980 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0cd657700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0c8150bd0 0x7ff0c80aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.070+0000 7ff0ce659700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0c81514a0 0x7ff0c813e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0cde58700 1 -- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0c81514a0 msgr2=0x7ff0c813e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0cde58700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0c81514a0 0x7ff0c813e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0cde58700 1 -- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0c8150bd0 msgr2=0x7ff0c80aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0cde58700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0c8150bd0 0x7ff0c80aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0cde58700 1 -- 172.21.15.29:0/4001654980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b800b610 con 0x7ff0c8069240 2024-01-24T13:58:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0cd657700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0c8150bd0 0x7ff0c80aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0cde58700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0c8069240 0x7ff0c80beb50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7ff0c400c4c0 tx=0x7ff0c4007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0bdffb700 1 -- 172.21.15.29:0/4001654980 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40072e0 con 0x7ff0c8069240 2024-01-24T13:58:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0bdffb700 1 -- 172.21.15.29:0/4001654980 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c4007460 con 0x7ff0c8069240 2024-01-24T13:58:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0d00bc700 1 -- 172.21.15.29:0/4001654980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c80bf730 con 0x7ff0c8069240 2024-01-24T13:58:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0bdffb700 1 -- 172.21.15.29:0/4001654980 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4005930 con 0x7ff0c8069240 2024-01-24T13:58:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.071+0000 7ff0d00bc700 1 -- 172.21.15.29:0/4001654980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c80aaf90 con 0x7ff0c8069240 2024-01-24T13:58:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.072+0000 7ff0d00bc700 1 -- 172.21.15.29:0/4001654980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c80475e0 con 0x7ff0c8069240 2024-01-24T13:58:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.072+0000 7ff0bdffb700 1 -- 172.21.15.29:0/4001654980 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff0c4005ce0 con 0x7ff0c8069240 2024-01-24T13:58:38.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.073+0000 7ff0bdffb700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff0ac05f150 0x7ff0ac061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.073+0000 7ff0bdffb700 1 -- 172.21.15.29:0/4001654980 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7ff0c4079c40 con 0x7ff0c8069240 2024-01-24T13:58:38.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.073+0000 7ff0cd657700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff0ac05f150 0x7ff0ac061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.073+0000 7ff0cd657700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff0ac05f150 0x7ff0ac061610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff0b8006fd0 tx=0x7ff0b800b3b0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:38.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.077+0000 7ff0bdffb700 1 -- 172.21.15.29:0/4001654980 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff0c404ac00 con 0x7ff0c8069240 2024-01-24T13:58:38.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.264+0000 7ff0d00bc700 1 -- 172.21.15.29:0/4001654980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff0c80c04f0 con 0x7ff0c8069240 2024-01-24T13:58:38.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.264+0000 7ff0bdffb700 1 -- 172.21.15.29:0/4001654980 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0c404e220 con 0x7ff0c8069240 2024-01-24T13:58:38.265 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-01-24T13:58:38.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.266+0000 7ff0d00bc700 1 -- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff0ac05f150 msgr2=0x7ff0ac061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.266+0000 7ff0d00bc700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff0ac05f150 0x7ff0ac061610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff0b8006fd0 tx=0x7ff0b800b3b0).stop 2024-01-24T13:58:38.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.266+0000 7ff0d00bc700 1 -- 172.21.15.29:0/4001654980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0c8069240 msgr2=0x7ff0c80beb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.266+0000 7ff0d00bc700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0c8069240 0x7ff0c80beb50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7ff0c400c4c0 tx=0x7ff0c4007c10).stop 2024-01-24T13:58:38.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.266+0000 7ff0d00bc700 1 -- 172.21.15.29:0/4001654980 shutdown_connections 2024-01-24T13:58:38.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.266+0000 7ff0d00bc700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0c81514a0 0x7ff0c813e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.266+0000 7ff0d00bc700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff0ac05f150 0x7ff0ac061610 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.266+0000 7ff0d00bc700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0c8069240 0x7ff0c80beb50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.266+0000 7ff0d00bc700 1 --2- 172.21.15.29:0/4001654980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0c8150bd0 0x7ff0c80aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.266+0000 7ff0d00bc700 1 -- 172.21.15.29:0/4001654980 >> 172.21.15.29:0/4001654980 conn(0x7ff0c80c8580 msgr2=0x7ff0c80c2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:38.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.267+0000 7ff0d00bc700 1 -- 172.21.15.29:0/4001654980 shutdown_connections 2024-01-24T13:58:38.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.267+0000 7ff0d00bc700 1 -- 172.21.15.29:0/4001654980 wait complete. 2024-01-24T13:58:38.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-01-24T13:58:38.280 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-24T13:58:38.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.411+0000 7fb359812700 1 Processor -- start 2024-01-24T13:58:38.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.411+0000 7fb359812700 1 -- start start 2024-01-24T13:58:38.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.412+0000 7fb359812700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb354154240 0x7fb354064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.412+0000 7fb359812700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb35405eeb0 0x7fb3540a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.412+0000 7fb359812700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3541550d0 con 0x7fb354064c70 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.412+0000 7fb359812700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb354154620 con 0x7fb35405eeb0 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.412+0000 7fb359812700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540c0d10 con 0x7fb354154240 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb3527fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb35405eeb0 0x7fb3540a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb3537fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb354064c70 0x7fb354157cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42190/0 (socket says 172.21.15.29:42190) 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb3537fe700 1 -- 172.21.15.29:0/1858482249 learned_addr learned my addr 172.21.15.29:0/1858482249 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb352ffd700 1 --2- 172.21.15.29:0/1858482249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb354154240 0x7fb354064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb3527fc700 1 -- 172.21.15.29:0/1858482249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb354154240 msgr2=0x7fb354064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb3527fc700 1 --2- 172.21.15.29:0/1858482249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb354154240 0x7fb354064780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb3527fc700 1 -- 172.21.15.29:0/1858482249 >> v1:172.21.15.29:6789/0 conn(0x7fb354064c70 legacy=0x7fb354157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb3527fc700 1 -- 172.21.15.29:0/1858482249 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb354154810 con 0x7fb35405eeb0 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb351ffb700 1 -- 172.21.15.29:0/1858482249 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569519350 0 0) 0x7fb3541550d0 con 0x7fb354064c70 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb3527fc700 1 --2- 172.21.15.29:0/1858482249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb35405eeb0 0x7fb3540a9d20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fb348009760 tx=0x7fb348009060).ready entity=mon.0 client_cookie=1419fb909b3dff8a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:38.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb351ffb700 1 -- 172.21.15.29:0/1858482249 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34800f510 con 0x7fb35405eeb0 2024-01-24T13:58:38.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb351ffb700 1 -- 172.21.15.29:0/1858482249 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34800f670 con 0x7fb35405eeb0 2024-01-24T13:58:38.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.413+0000 7fb351ffb700 1 -- 172.21.15.29:0/1858482249 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34800f9f0 con 0x7fb35405eeb0 2024-01-24T13:58:38.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.414+0000 7fb359812700 1 -- 172.21.15.29:0/1858482249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb35405eeb0 msgr2=0x7fb3540a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.414+0000 7fb359812700 1 --2- 172.21.15.29:0/1858482249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb35405eeb0 0x7fb3540a9d20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fb348009760 tx=0x7fb348009060).stop 2024-01-24T13:58:38.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.414+0000 7fb359812700 1 -- 172.21.15.29:0/1858482249 shutdown_connections 2024-01-24T13:58:38.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.414+0000 7fb359812700 1 --2- 172.21.15.29:0/1858482249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb354154240 0x7fb354064780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.414+0000 7fb359812700 1 --2- 172.21.15.29:0/1858482249 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb35405eeb0 0x7fb3540a9d20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.414+0000 7fb359812700 1 -- 172.21.15.29:0/1858482249 >> 172.21.15.29:0/1858482249 conn(0x7fb3540c8310 msgr2=0x7fb35405e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:38.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.414+0000 7fb359812700 1 -- 172.21.15.29:0/1858482249 shutdown_connections 2024-01-24T13:58:38.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.414+0000 7fb359812700 1 -- 172.21.15.29:0/1858482249 wait complete. 2024-01-24T13:58:38.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.415+0000 7fb359812700 1 Processor -- start 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.415+0000 7fb359812700 1 -- start start 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.416+0000 7fb359812700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb354064c70 0x7fb3540a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.416+0000 7fb359812700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb354154240 0x7fb354144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.416+0000 7fb359812700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb354145020 0x7fb3540a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.416+0000 7fb359812700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540a3d30 con 0x7fb354145020 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.416+0000 7fb359812700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540a3120 con 0x7fb354064c70 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.416+0000 7fb359812700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540a3400 con 0x7fb354154240 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.416+0000 7fb3537fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb354145020 0x7fb3540a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.416+0000 7fb3537fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb354145020 0x7fb3540a24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41364/0 (socket says 172.21.15.29:41364) 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.416+0000 7fb3537fe700 1 -- 172.21.15.29:0/2406349650 learned_addr learned my addr 172.21.15.29:0/2406349650 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.416+0000 7fb3527fc700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb354154240 0x7fb354144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.416+0000 7fb352ffd700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb354064c70 0x7fb3540a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.417+0000 7fb3537fe700 1 -- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb354154240 msgr2=0x7fb354144b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.417+0000 7fb3537fe700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb354154240 0x7fb354144b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.417+0000 7fb3537fe700 1 -- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb354064c70 msgr2=0x7fb3540a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.417+0000 7fb3537fe700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb354064c70 0x7fb3540a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.417+0000 7fb3537fe700 1 -- 172.21.15.29:0/2406349650 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3480092a0 con 0x7fb354145020 2024-01-24T13:58:38.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.417+0000 7fb3537fe700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb354145020 0x7fb3540a24a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fb34c00c7e0 tx=0x7fb34c007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:38.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.417+0000 7fb33affd700 1 -- 172.21.15.29:0/2406349650 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c005d30 con 0x7fb354145020 2024-01-24T13:58:38.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.417+0000 7fb33affd700 1 -- 172.21.15.29:0/2406349650 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c005e90 con 0x7fb354145020 2024-01-24T13:58:38.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.417+0000 7fb359812700 1 -- 172.21.15.29:0/2406349650 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3540a36e0 con 0x7fb354145020 2024-01-24T13:58:38.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.417+0000 7fb33affd700 1 -- 172.21.15.29:0/2406349650 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c007320 con 0x7fb354145020 2024-01-24T13:58:38.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.417+0000 7fb359812700 1 -- 172.21.15.29:0/2406349650 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3540a2990 con 0x7fb354145020 2024-01-24T13:58:38.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.418+0000 7fb359812700 1 -- 172.21.15.29:0/2406349650 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3540a3ea0 con 0x7fb354145020 2024-01-24T13:58:38.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.418+0000 7fb33affd700 1 -- 172.21.15.29:0/2406349650 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb34c007710 con 0x7fb354145020 2024-01-24T13:58:38.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.419+0000 7fb33affd700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb34005f150 0x7fb340061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.419+0000 7fb33affd700 1 -- 172.21.15.29:0/2406349650 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fb34c07a470 con 0x7fb354145020 2024-01-24T13:58:38.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.419+0000 7fb352ffd700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb34005f150 0x7fb340061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.420+0000 7fb352ffd700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb34005f150 0x7fb340061610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fb344009b90 tx=0x7fb344009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:38.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.423+0000 7fb33affd700 1 -- 172.21.15.29:0/2406349650 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb34c04b3d0 con 0x7fb354145020 2024-01-24T13:58:38.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.605+0000 7fb359812700 1 -- 172.21.15.29:0/2406349650 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb3540475e0 con 0x7fb354145020 2024-01-24T13:58:38.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.605+0000 7fb33affd700 1 -- 172.21.15.29:0/2406349650 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb34c04e9f0 con 0x7fb354145020 2024-01-24T13:58:38.606 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-01-24T13:58:38.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.607+0000 7fb359812700 1 -- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb34005f150 msgr2=0x7fb340061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.607+0000 7fb359812700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb34005f150 0x7fb340061610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fb344009b90 tx=0x7fb344009060).stop 2024-01-24T13:58:38.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.607+0000 7fb359812700 1 -- 172.21.15.29:0/2406349650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb354145020 msgr2=0x7fb3540a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.607+0000 7fb359812700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb354145020 0x7fb3540a24a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fb34c00c7e0 tx=0x7fb34c007f90).stop 2024-01-24T13:58:38.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.607+0000 7fb359812700 1 -- 172.21.15.29:0/2406349650 shutdown_connections 2024-01-24T13:58:38.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.607+0000 7fb359812700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb34005f150 0x7fb340061610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.607+0000 7fb359812700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb354145020 0x7fb3540a24a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.607+0000 7fb359812700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb354064c70 0x7fb3540a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.607+0000 7fb359812700 1 --2- 172.21.15.29:0/2406349650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb354154240 0x7fb354144b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.607+0000 7fb359812700 1 -- 172.21.15.29:0/2406349650 >> 172.21.15.29:0/2406349650 conn(0x7fb3540c8310 msgr2=0x7fb3540c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:38.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.607+0000 7fb359812700 1 -- 172.21.15.29:0/2406349650 shutdown_connections 2024-01-24T13:58:38.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.607+0000 7fb359812700 1 -- 172.21.15.29:0/2406349650 wait complete. 2024-01-24T13:58:38.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-01-24T13:58:38.621 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-24T13:58:38.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.753+0000 7f2bf93a5700 1 Processor -- start 2024-01-24T13:58:38.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.753+0000 7f2bf93a5700 1 -- start start 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf93a5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2bf4133c90 0x7f2bf40699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf93a5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2bf4134070 0x7f2bf40bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf93a5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40bcf10 0x7f2bf41549a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf93a5700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf40691b0 con 0x7f2bf4133c90 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf93a5700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf40bd340 con 0x7f2bf4134070 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf93a5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf40685a0 con 0x7f2bf40bcf10 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf2ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2bf4133c90 0x7f2bf40699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf2ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2bf4133c90 0x7f2bf40699f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41366/0 (socket says 172.21.15.29:41366) 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf2ffd700 1 -- 172.21.15.29:0/1285267824 learned_addr learned my addr 172.21.15.29:0/1285267824 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf37fe700 1 --2- 172.21.15.29:0/1285267824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40bcf10 0x7f2bf41549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf27fc700 1 --2- 172.21.15.29:0/1285267824 >> v2:172.21.15.163:3300/0 conn(0x7f2bf4134070 0x7f2bf40bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf2ffd700 1 -- 172.21.15.29:0/1285267824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40bcf10 msgr2=0x7f2bf41549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf2ffd700 1 --2- 172.21.15.29:0/1285267824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40bcf10 0x7f2bf41549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf2ffd700 1 -- 172.21.15.29:0/1285267824 >> v2:172.21.15.163:3300/0 conn(0x7f2bf4134070 msgr2=0x7f2bf40bca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:38.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf2ffd700 1 --2- 172.21.15.29:0/1285267824 >> v2:172.21.15.163:3300/0 conn(0x7f2bf4134070 0x7f2bf40bca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf2ffd700 1 -- 172.21.15.29:0/1285267824 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf4068880 con 0x7f2bf4133c90 2024-01-24T13:58:38.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf2ffd700 1 --2- 172.21.15.29:0/1285267824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bf4133c90 0x7f2bf40699f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f2bf414b9f0 tx=0x7f2be40090f0).ready entity=mon.0 client_cookie=2e152512a415e16a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:38.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.754+0000 7f2bf1ffb700 1 -- 172.21.15.29:0/1285267824 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be400f510 con 0x7f2bf4133c90 2024-01-24T13:58:38.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.755+0000 7f2bf1ffb700 1 -- 172.21.15.29:0/1285267824 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be400f670 con 0x7f2bf4133c90 2024-01-24T13:58:38.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.755+0000 7f2bf1ffb700 1 -- 172.21.15.29:0/1285267824 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be400fa10 con 0x7f2bf4133c90 2024-01-24T13:58:38.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.755+0000 7f2bf93a5700 1 -- 172.21.15.29:0/1285267824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bf4133c90 msgr2=0x7f2bf40699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.755+0000 7f2bf93a5700 1 --2- 172.21.15.29:0/1285267824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bf4133c90 0x7f2bf40699f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f2bf414b9f0 tx=0x7f2be40090f0).stop 2024-01-24T13:58:38.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.755+0000 7f2bf93a5700 1 -- 172.21.15.29:0/1285267824 shutdown_connections 2024-01-24T13:58:38.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.755+0000 7f2bf93a5700 1 --2- 172.21.15.29:0/1285267824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40bcf10 0x7f2bf41549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.755+0000 7f2bf93a5700 1 --2- 172.21.15.29:0/1285267824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bf4133c90 0x7f2bf40699f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.755+0000 7f2bf93a5700 1 --2- 172.21.15.29:0/1285267824 >> v2:172.21.15.163:3300/0 conn(0x7f2bf4134070 0x7f2bf40bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.755+0000 7f2bf93a5700 1 -- 172.21.15.29:0/1285267824 >> 172.21.15.29:0/1285267824 conn(0x7f2bf40c7480 msgr2=0x7f2bf40c3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:38.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.755+0000 7f2bf93a5700 1 -- 172.21.15.29:0/1285267824 shutdown_connections 2024-01-24T13:58:38.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.755+0000 7f2bf93a5700 1 -- 172.21.15.29:0/1285267824 wait complete. 2024-01-24T13:58:38.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.757+0000 7f2bf93a5700 1 Processor -- start 2024-01-24T13:58:38.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.757+0000 7f2bf93a5700 1 -- start start 2024-01-24T13:58:38.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.757+0000 7f2bf93a5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40bcf10 0x7f2bf4132a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.757+0000 7f2bf93a5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf4133c90 0x7f2bf414e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.757+0000 7f2bf93a5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bf4134070 0x7f2bf414ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.757+0000 7f2bf93a5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf4166580 con 0x7f2bf4134070 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.757+0000 7f2bf93a5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf4134480 con 0x7f2bf4133c90 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.757+0000 7f2bf93a5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf4165970 con 0x7f2bf40bcf10 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bf4134070 0x7f2bf414ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bf4134070 0x7f2bf414ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41380/0 (socket says 172.21.15.29:41380) 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf37fe700 1 -- 172.21.15.29:0/143431539 learned_addr learned my addr 172.21.15.29:0/143431539 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf2ffd700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40bcf10 0x7f2bf4132a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf27fc700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf4133c90 0x7f2bf414e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf37fe700 1 -- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40bcf10 msgr2=0x7f2bf4132a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf37fe700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40bcf10 0x7f2bf4132a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf37fe700 1 -- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf4133c90 msgr2=0x7f2bf414e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf37fe700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf4133c90 0x7f2bf414e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf37fe700 1 -- 172.21.15.29:0/143431539 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be4009370 con 0x7f2bf4134070 2024-01-24T13:58:38.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf37fe700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bf4134070 0x7f2bf414ef60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f2bec009ec0 tx=0x7f2bec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:38.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bdaffd700 1 -- 172.21.15.29:0/143431539 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec00cb70 con 0x7f2bf4134070 2024-01-24T13:58:38.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bdaffd700 1 -- 172.21.15.29:0/143431539 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bec00f040 con 0x7f2bf4134070 2024-01-24T13:58:38.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf93a5700 1 -- 172.21.15.29:0/143431539 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf4165c50 con 0x7f2bf4134070 2024-01-24T13:58:38.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bdaffd700 1 -- 172.21.15.29:0/143431539 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec00cee0 con 0x7f2bf4134070 2024-01-24T13:58:38.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.758+0000 7f2bf93a5700 1 -- 172.21.15.29:0/143431539 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf4166210 con 0x7f2bf4134070 2024-01-24T13:58:38.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.759+0000 7f2bf93a5700 1 -- 172.21.15.29:0/143431539 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf40475e0 con 0x7f2bf4134070 2024-01-24T13:58:38.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.760+0000 7f2bdaffd700 1 -- 172.21.15.29:0/143431539 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2bec013610 con 0x7f2bf4134070 2024-01-24T13:58:38.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.760+0000 7f2bdaffd700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2be005f120 0x7f2be00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:38.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.760+0000 7f2bdaffd700 1 -- 172.21.15.29:0/143431539 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f2bec07ac90 con 0x7f2bf4134070 2024-01-24T13:58:38.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.760+0000 7f2bf2ffd700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2be005f120 0x7f2be00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:38.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.761+0000 7f2bf2ffd700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2be005f120 0x7f2be00615e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f2bf414db50 tx=0x7f2be4017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:38.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.765+0000 7f2bdaffd700 1 -- 172.21.15.29:0/143431539 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2bec04bc50 con 0x7f2bf4134070 2024-01-24T13:58:38.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.946+0000 7f2bf93a5700 1 -- 172.21.15.29:0/143431539 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2bf41546b0 con 0x7f2bf4134070 2024-01-24T13:58:38.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.947+0000 7f2bdaffd700 1 -- 172.21.15.29:0/143431539 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2bec04f270 con 0x7f2bf4134070 2024-01-24T13:58:38.947 INFO:teuthology.orchestra.run.smithi029.stdout:55834574856 2024-01-24T13:58:38.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.948+0000 7f2bf93a5700 1 -- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2be005f120 msgr2=0x7f2be00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.948+0000 7f2bf93a5700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2be005f120 0x7f2be00615e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f2bf414db50 tx=0x7f2be4017000).stop 2024-01-24T13:58:38.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.948+0000 7f2bf93a5700 1 -- 172.21.15.29:0/143431539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bf4134070 msgr2=0x7f2bf414ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:38.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.948+0000 7f2bf93a5700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bf4134070 0x7f2bf414ef60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f2bec009ec0 tx=0x7f2bec00b040).stop 2024-01-24T13:58:38.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.948+0000 7f2bf93a5700 1 -- 172.21.15.29:0/143431539 shutdown_connections 2024-01-24T13:58:38.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.948+0000 7f2bf93a5700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2be005f120 0x7f2be00615e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.948+0000 7f2bf93a5700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bf4134070 0x7f2bf414ef60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.948+0000 7f2bf93a5700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40bcf10 0x7f2bf4132a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.948+0000 7f2bf93a5700 1 --2- 172.21.15.29:0/143431539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf4133c90 0x7f2bf414e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:38.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.949+0000 7f2bf93a5700 1 -- 172.21.15.29:0/143431539 >> 172.21.15.29:0/143431539 conn(0x7f2bf40c7480 msgr2=0x7f2bf41509e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:38.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.949+0000 7f2bf93a5700 1 -- 172.21.15.29:0/143431539 shutdown_connections 2024-01-24T13:58:38.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:38.949+0000 7f2bf93a5700 1 -- 172.21.15.29:0/143431539 wait complete. 2024-01-24T13:58:38.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.6 2024-01-24T13:58:38.962 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-24T13:58:39.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.093+0000 7f9c43bac700 1 Processor -- start 2024-01-24T13:58:39.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.093+0000 7f9c43bac700 1 -- start start 2024-01-24T13:58:39.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.094+0000 7f9c43bac700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c3c151de0 0x7f9c3c155470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.094+0000 7f9c43bac700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9c3c066bb0 0x7f9c3c13d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.094+0000 7f9c43bac700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c158800 con 0x7f9c3c0638c0 2024-01-24T13:58:39.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.094+0000 7f9c43bac700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c157ed0 con 0x7f9c3c066bb0 2024-01-24T13:58:39.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.094+0000 7f9c43bac700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c1581b0 con 0x7f9c3c151de0 2024-01-24T13:58:39.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.094+0000 7f9c42149700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9c3c066bb0 0x7f9c3c13d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.094+0000 7f9c42149700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9c3c066bb0 0x7f9c3c13d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41390/0 (socket says 172.21.15.29:41390) 2024-01-24T13:58:39.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.094+0000 7f9c42149700 1 -- 172.21.15.29:0/3078456947 learned_addr learned my addr 172.21.15.29:0/3078456947 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:39.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.094+0000 7f9c41948700 1 --2- 172.21.15.29:0/3078456947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c3c151de0 0x7f9c3c155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c42149700 1 -- 172.21.15.29:0/3078456947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c3c151de0 msgr2=0x7f9c3c155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:39.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c42149700 1 --2- 172.21.15.29:0/3078456947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c3c151de0 0x7f9c3c155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c42149700 1 -- 172.21.15.29:0/3078456947 >> v1:172.21.15.163:6789/0 conn(0x7f9c3c0638c0 legacy=0x7f9c3c0664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c42149700 1 -- 172.21.15.29:0/3078456947 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c158490 con 0x7f9c3c066bb0 2024-01-24T13:58:39.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c42149700 1 --2- 172.21.15.29:0/3078456947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3c066bb0 0x7f9c3c13d150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f9c34009f20 tx=0x7f9c34009840).ready entity=mon.0 client_cookie=9b7ef3cc697689a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c40946700 1 -- 172.21.15.29:0/3078456947 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400e070 con 0x7f9c3c066bb0 2024-01-24T13:58:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c40946700 1 -- 172.21.15.29:0/3078456947 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3400b9d0 con 0x7f9c3c066bb0 2024-01-24T13:58:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c40946700 1 -- 172.21.15.29:0/3078456947 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400bd50 con 0x7f9c3c066bb0 2024-01-24T13:58:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c43bac700 1 -- 172.21.15.29:0/3078456947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3c066bb0 msgr2=0x7f9c3c13d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c43bac700 1 --2- 172.21.15.29:0/3078456947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3c066bb0 0x7f9c3c13d150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f9c34009f20 tx=0x7f9c34009840).stop 2024-01-24T13:58:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c43bac700 1 -- 172.21.15.29:0/3078456947 shutdown_connections 2024-01-24T13:58:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c43bac700 1 --2- 172.21.15.29:0/3078456947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c3c151de0 0x7f9c3c155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c43bac700 1 --2- 172.21.15.29:0/3078456947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3c066bb0 0x7f9c3c13d150 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.095+0000 7f9c43bac700 1 -- 172.21.15.29:0/3078456947 >> 172.21.15.29:0/3078456947 conn(0x7f9c3c0c5640 msgr2=0x7f9c3c0c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.096+0000 7f9c43bac700 1 -- 172.21.15.29:0/3078456947 shutdown_connections 2024-01-24T13:58:39.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.096+0000 7f9c43bac700 1 -- 172.21.15.29:0/3078456947 wait complete. 2024-01-24T13:58:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.097+0000 7f9c43bac700 1 Processor -- start 2024-01-24T13:58:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.097+0000 7f9c43bac700 1 -- start start 2024-01-24T13:58:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c43bac700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c3c0638c0 0x7f9c3c0c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c43bac700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3c066bb0 0x7f9c3c0ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c43bac700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c3c151de0 0x7f9c3c0c0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c43bac700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0c39c0 con 0x7f9c3c066bb0 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c43bac700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c165920 con 0x7f9c3c0638c0 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c43bac700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0a60d0 con 0x7f9c3c151de0 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c41147700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3c066bb0 0x7f9c3c0ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c41147700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3c066bb0 0x7f9c3c0ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41404/0 (socket says 172.21.15.29:41404) 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c41147700 1 -- 172.21.15.29:0/3844559839 learned_addr learned my addr 172.21.15.29:0/3844559839 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c42149700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c3c151de0 0x7f9c3c0c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c41948700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c3c0638c0 0x7f9c3c0c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c41147700 1 -- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c3c151de0 msgr2=0x7f9c3c0c0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c41147700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c3c151de0 0x7f9c3c0c0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c41147700 1 -- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c3c0638c0 msgr2=0x7f9c3c0c34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c41147700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c3c0638c0 0x7f9c3c0c34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c41147700 1 -- 172.21.15.29:0/3844559839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c34009ae0 con 0x7f9c3c066bb0 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c41147700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3c066bb0 0x7f9c3c0ae120 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f9c3800c4c0 tx=0x7f9c38007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:39.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.098+0000 7f9c31ffb700 1 -- 172.21.15.29:0/3844559839 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800ce10 con 0x7f9c3c066bb0 2024-01-24T13:58:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.099+0000 7f9c31ffb700 1 -- 172.21.15.29:0/3844559839 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c38007020 con 0x7f9c3c066bb0 2024-01-24T13:58:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.099+0000 7f9c43bac700 1 -- 172.21.15.29:0/3844559839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c0a63b0 con 0x7f9c3c066bb0 2024-01-24T13:58:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.099+0000 7f9c31ffb700 1 -- 172.21.15.29:0/3844559839 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c380073c0 con 0x7f9c3c066bb0 2024-01-24T13:58:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.099+0000 7f9c43bac700 1 -- 172.21.15.29:0/3844559839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c0ae610 con 0x7f9c3c066bb0 2024-01-24T13:58:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.100+0000 7f9c43bac700 1 -- 172.21.15.29:0/3844559839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c0a5f30 con 0x7f9c3c066bb0 2024-01-24T13:58:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.100+0000 7f9c31ffb700 1 -- 172.21.15.29:0/3844559839 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9c380068b0 con 0x7f9c3c066bb0 2024-01-24T13:58:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.100+0000 7f9c31ffb700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9c2005f150 0x7f9c20061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.100+0000 7f9c31ffb700 1 -- 172.21.15.29:0/3844559839 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f9c380797d0 con 0x7f9c3c066bb0 2024-01-24T13:58:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.101+0000 7f9c41948700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9c2005f150 0x7f9c20061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.101+0000 7f9c41948700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9c2005f150 0x7f9c20061610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f9c2c009b90 tx=0x7f9c2c009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:39.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.105+0000 7f9c31ffb700 1 -- 172.21.15.29:0/3844559839 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9c38047b00 con 0x7f9c3c066bb0 2024-01-24T13:58:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.287+0000 7f9c43bac700 1 -- 172.21.15.29:0/3844559839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9c3c0475e0 con 0x7f9c3c066bb0 2024-01-24T13:58:39.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.288+0000 7f9c31ffb700 1 -- 172.21.15.29:0/3844559839 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9c3c0a5f30 con 0x7f9c3c066bb0 2024-01-24T13:58:39.288 INFO:teuthology.orchestra.run.smithi029.stdout:55834574855 2024-01-24T13:58:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.290+0000 7f9c43bac700 1 -- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9c2005f150 msgr2=0x7f9c20061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.290+0000 7f9c43bac700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9c2005f150 0x7f9c20061610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f9c2c009b90 tx=0x7f9c2c009060).stop 2024-01-24T13:58:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.290+0000 7f9c43bac700 1 -- 172.21.15.29:0/3844559839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3c066bb0 msgr2=0x7f9c3c0ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.290+0000 7f9c43bac700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3c066bb0 0x7f9c3c0ae120 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f9c3800c4c0 tx=0x7f9c38007dd0).stop 2024-01-24T13:58:39.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.290+0000 7f9c43bac700 1 -- 172.21.15.29:0/3844559839 shutdown_connections 2024-01-24T13:58:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.290+0000 7f9c43bac700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c3c151de0 0x7f9c3c0c0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.290+0000 7f9c43bac700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c3c0638c0 0x7f9c3c0c34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.290+0000 7f9c43bac700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9c2005f150 0x7f9c20061610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.290+0000 7f9c43bac700 1 --2- 172.21.15.29:0/3844559839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3c066bb0 0x7f9c3c0ae120 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.290+0000 7f9c43bac700 1 -- 172.21.15.29:0/3844559839 >> 172.21.15.29:0/3844559839 conn(0x7f9c3c0c5640 msgr2=0x7f9c3c0bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.290+0000 7f9c43bac700 1 -- 172.21.15.29:0/3844559839 shutdown_connections 2024-01-24T13:58:39.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.290+0000 7f9c43bac700 1 -- 172.21.15.29:0/3844559839 wait complete. 2024-01-24T13:58:39.303 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.7 2024-01-24T13:58:39.304 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-24T13:58:39.304 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T13:58:39.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.433+0000 7fb213c7c700 1 Processor -- start 2024-01-24T13:58:39.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.433+0000 7fb213c7c700 1 -- start start 2024-01-24T13:58:39.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.434+0000 7fb213c7c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb20c0bd4c0 0x7fb20c062ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.434+0000 7fb213c7c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20c0640f0 con 0x7fb20c0c1f80 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.434+0000 7fb213c7c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20c0634e0 con 0x7fb20c14e840 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.434+0000 7fb213c7c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0637c0 con 0x7fb20c0bd4c0 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.434+0000 7fb211217700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb20c0c1f80 0x7fb20c0627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42194/0 (socket says 172.21.15.29:42194) 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.434+0000 7fb211217700 1 -- 172.21.15.29:0/1243986155 learned_addr learned my addr 172.21.15.29:0/1243986155 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.435+0000 7fb212219700 1 --2- 172.21.15.29:0/1243986155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb20c0bd4c0 0x7fb20c062ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.435+0000 7fb210a16700 1 -- 172.21.15.29:0/1243986155 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041502103 0 0) 0x7fb20c0640f0 con 0x7fb20c0c1f80 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.435+0000 7fb210a16700 1 -- 172.21.15.29:0/1243986155 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1f80033f0 con 0x7fb20c0c1f80 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.435+0000 7fb212219700 1 -- 172.21.15.29:0/1243986155 >> v1:172.21.15.163:6789/0 conn(0x7fb20c14e840 legacy=0x7fb20c0c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.435+0000 7fb210a16700 1 -- 172.21.15.29:0/1243986155 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078645082 0 0) 0x7fb20c0634e0 con 0x7fb20c14e840 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.435+0000 7fb212219700 1 -- 172.21.15.29:0/1243986155 >> v1:172.21.15.29:6789/0 conn(0x7fb20c0c1f80 legacy=0x7fb20c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.435+0000 7fb212219700 1 -- 172.21.15.29:0/1243986155 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c063aa0 con 0x7fb20c0bd4c0 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.435+0000 7fb212219700 1 --2- 172.21.15.29:0/1243986155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb20c0bd4c0 0x7fb20c062ec0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fb204009e90 tx=0x7fb204009780).ready entity=mon.2 client_cookie=62a9d5e96d920cb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.435+0000 7fb210a16700 1 -- 172.21.15.29:0/1243986155 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20400e070 con 0x7fb20c0bd4c0 2024-01-24T13:58:39.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.435+0000 7fb210a16700 1 -- 172.21.15.29:0/1243986155 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20400bb60 con 0x7fb20c0bd4c0 2024-01-24T13:58:39.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.435+0000 7fb210a16700 1 -- 172.21.15.29:0/1243986155 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20400bee0 con 0x7fb20c0bd4c0 2024-01-24T13:58:39.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.436+0000 7fb213c7c700 1 -- 172.21.15.29:0/1243986155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb20c0bd4c0 msgr2=0x7fb20c062ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.436+0000 7fb213c7c700 1 --2- 172.21.15.29:0/1243986155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb20c0bd4c0 0x7fb20c062ec0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fb204009e90 tx=0x7fb204009780).stop 2024-01-24T13:58:39.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.436+0000 7fb213c7c700 1 -- 172.21.15.29:0/1243986155 shutdown_connections 2024-01-24T13:58:39.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.436+0000 7fb213c7c700 1 --2- 172.21.15.29:0/1243986155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb20c0bd4c0 0x7fb20c062ec0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.436+0000 7fb213c7c700 1 -- 172.21.15.29:0/1243986155 >> 172.21.15.29:0/1243986155 conn(0x7fb20c064aa0 msgr2=0x7fb20c064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:39.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.436+0000 7fb213c7c700 1 -- 172.21.15.29:0/1243986155 shutdown_connections 2024-01-24T13:58:39.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.436+0000 7fb213c7c700 1 -- 172.21.15.29:0/1243986155 wait complete. 2024-01-24T13:58:39.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.437+0000 7fb213c7c700 1 Processor -- start 2024-01-24T13:58:39.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.437+0000 7fb213c7c700 1 -- start start 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb213c7c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb20c0bd4c0 0x7fb20c1537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb213c7c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb20c0c1f80 0x7fb20c13d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb213c7c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb20c14e840 0x7fb20c13d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb213c7c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0c5780 con 0x7fb20c0c1f80 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb213c7c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c13dd10 con 0x7fb20c14e840 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb213c7c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0c4e50 con 0x7fb20c0bd4c0 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb211217700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb20c0c1f80 0x7fb20c13d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb211217700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb20c0c1f80 0x7fb20c13d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41414/0 (socket says 172.21.15.29:41414) 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb211217700 1 -- 172.21.15.29:0/3601134526 learned_addr learned my addr 172.21.15.29:0/3601134526 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb211a18700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb20c0bd4c0 0x7fb20c1537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb212219700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb20c14e840 0x7fb20c13d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb211217700 1 -- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb20c0bd4c0 msgr2=0x7fb20c1537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.438+0000 7fb211217700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb20c0bd4c0 0x7fb20c1537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.439+0000 7fb211217700 1 -- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb20c14e840 msgr2=0x7fb20c13d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.439+0000 7fb211217700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb20c14e840 0x7fb20c13d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.439+0000 7fb211217700 1 -- 172.21.15.29:0/3601134526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb204009a20 con 0x7fb20c0c1f80 2024-01-24T13:58:39.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.439+0000 7fb211217700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb20c0c1f80 0x7fb20c13d180 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fb1fc00c420 tx=0x7fb1fc007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:39.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.439+0000 7fb201ffb700 1 -- 172.21.15.29:0/3601134526 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc00ce20 con 0x7fb20c0c1f80 2024-01-24T13:58:39.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.439+0000 7fb201ffb700 1 -- 172.21.15.29:0/3601134526 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1fc0072e0 con 0x7fb20c0c1f80 2024-01-24T13:58:39.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.439+0000 7fb213c7c700 1 -- 172.21.15.29:0/3601134526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20c0c5130 con 0x7fb20c0c1f80 2024-01-24T13:58:39.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.439+0000 7fb201ffb700 1 -- 172.21.15.29:0/3601134526 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc0057a0 con 0x7fb20c0c1f80 2024-01-24T13:58:39.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.439+0000 7fb213c7c700 1 -- 172.21.15.29:0/3601134526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20c13b120 con 0x7fb20c0c1f80 2024-01-24T13:58:39.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.440+0000 7fb213c7c700 1 -- 172.21.15.29:0/3601134526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20c0c58f0 con 0x7fb20c0c1f80 2024-01-24T13:58:39.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.441+0000 7fb201ffb700 1 -- 172.21.15.29:0/3601134526 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb1fc005b50 con 0x7fb20c0c1f80 2024-01-24T13:58:39.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.441+0000 7fb201ffb700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1f005f180 0x7fb1f0061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.441+0000 7fb201ffb700 1 -- 172.21.15.29:0/3601134526 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fb1fc079a80 con 0x7fb20c0c1f80 2024-01-24T13:58:39.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.441+0000 7fb211a18700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1f005f180 0x7fb1f0061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.442+0000 7fb211a18700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1f005f180 0x7fb1f0061640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fb208007fd0 tx=0x7fb208007bb0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:39.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.445+0000 7fb201ffb700 1 -- 172.21.15.29:0/3601134526 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb1fc04aa40 con 0x7fb20c0c1f80 2024-01-24T13:58:39.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.621+0000 7fb213c7c700 1 -- 172.21.15.29:0/3601134526 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb20c0475e0 con 0x7fb1f005f180 2024-01-24T13:58:39.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.634+0000 7fb201ffb700 1 -- 172.21.15.29:0/3601134526 <== mgr.4103 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+410634 (secure 0 0 0) 0x7fb20c0475e0 con 0x7fb1f005f180 2024-01-24T13:58:39.639 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:39.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.638+0000 7fb213c7c700 1 -- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1f005f180 msgr2=0x7fb1f0061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.638+0000 7fb213c7c700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1f005f180 0x7fb1f0061640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fb208007fd0 tx=0x7fb208007bb0).stop 2024-01-24T13:58:39.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.638+0000 7fb213c7c700 1 -- 172.21.15.29:0/3601134526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb20c0c1f80 msgr2=0x7fb20c13d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.638+0000 7fb213c7c700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb20c0c1f80 0x7fb20c13d180 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fb1fc00c420 tx=0x7fb1fc007c10).stop 2024-01-24T13:58:39.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.638+0000 7fb213c7c700 1 -- 172.21.15.29:0/3601134526 shutdown_connections 2024-01-24T13:58:39.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.638+0000 7fb213c7c700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb20c14e840 0x7fb20c13d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.638+0000 7fb213c7c700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb20c0bd4c0 0x7fb20c1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.638+0000 7fb213c7c700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1f005f180 0x7fb1f0061640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.638+0000 7fb213c7c700 1 --2- 172.21.15.29:0/3601134526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb20c0c1f80 0x7fb20c13d180 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.638+0000 7fb213c7c700 1 -- 172.21.15.29:0/3601134526 >> 172.21.15.29:0/3601134526 conn(0x7fb20c064aa0 msgr2=0x7fb20c157030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:39.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.638+0000 7fb213c7c700 1 -- 172.21.15.29:0/3601134526 shutdown_connections 2024-01-24T13:58:39.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.638+0000 7fb213c7c700 1 -- 172.21.15.29:0/3601134526 wait complete. 2024-01-24T13:58:39.643 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T13:58:39.652 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-01-24T13:58:37.728237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":402,"acting":402,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":402,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379004,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752595716,"statfs":{"total":773094113280,"available":770658013184,"internally_reserved":0,"allocated":2289664,"data_stored":941041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2179,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003965"},"pg_stats":[{"pgid":"3.3f","version":"21'3","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.218432+0000","last_change":"2024-01-24T13:58:18.804112+0000","last_active":"2024-01-24T13:58:23.218432+0000","last_peered":"2024-01-24T13:58:23.218432+0000","last_clean":"2024-01-24T13:58:23.218432+0000","last_became_active":"2024-01-24T13:58:18.803739+0000","last_became_peered":"2024-01-24T13:58:18.803739+0000","last_unstale":"2024-01-24T13:58:23.218432+0000","last_undegraded":"2024-01-24T13:58:23.218432+0000","last_fullsized":"2024-01-24T13:58:23.218432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224809+0000","last_change":"2024-01-24T13:58:19.214526+0000","last_active":"2024-01-24T13:58:23.224809+0000","last_peered":"2024-01-24T13:58:23.224809+0000","last_clean":"2024-01-24T13:58:23.224809+0000","last_became_active":"2024-01-24T13:58:19.214261+0000","last_became_peered":"2024-01-24T13:58:19.214261+0000","last_unstale":"2024-01-24T13:58:23.224809+0000","last_undegraded":"2024-01-24T13:58:23.224809+0000","last_fullsized":"2024-01-24T13:58:23.224809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211257+0000","last_change":"2024-01-24T13:58:18.813731+0000","last_active":"2024-01-24T13:58:23.211257+0000","last_peered":"2024-01-24T13:58:23.211257+0000","last_clean":"2024-01-24T13:58:23.211257+0000","last_became_active":"2024-01-24T13:58:18.813399+0000","last_became_peered":"2024-01-24T13:58:18.813399+0000","last_unstale":"2024-01-24T13:58:23.211257+0000","last_undegraded":"2024-01-24T13:58:23.211257+0000","last_fullsized":"2024-01-24T13:58:23.211257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216628+0000","last_change":"2024-01-24T13:58:19.214934+0000","last_active":"2024-01-24T13:58:23.216628+0000","last_peered":"2024-01-24T13:58:23.216628+0000","last_clean":"2024-01-24T13:58:23.216628+0000","last_became_active":"2024-01-24T13:58:19.214473+0000","last_became_peered":"2024-01-24T13:58:19.214473+0000","last_unstale":"2024-01-24T13:58:23.216628+0000","last_undegraded":"2024-01-24T13:58:23.216628+0000","last_fullsized":"2024-01-24T13:58:23.216628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235905+0000","last_change":"2024-01-24T13:58:18.810864+0000","last_active":"2024-01-24T13:58:23.235905+0000","last_peered":"2024-01-24T13:58:23.235905+0000","last_clean":"2024-01-24T13:58:23.235905+0000","last_became_active":"2024-01-24T13:58:18.810538+0000","last_became_peered":"2024-01-24T13:58:18.810538+0000","last_unstale":"2024-01-24T13:58:23.235905+0000","last_undegraded":"2024-01-24T13:58:23.235905+0000","last_fullsized":"2024-01-24T13:58:23.235905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209261+0000","last_change":"2024-01-24T13:58:19.215436+0000","last_active":"2024-01-24T13:58:23.209261+0000","last_peered":"2024-01-24T13:58:23.209261+0000","last_clean":"2024-01-24T13:58:23.209261+0000","last_became_active":"2024-01-24T13:58:19.215145+0000","last_became_peered":"2024-01-24T13:58:19.215145+0000","last_unstale":"2024-01-24T13:58:23.209261+0000","last_undegraded":"2024-01-24T13:58:23.209261+0000","last_fullsized":"2024-01-24T13:58:23.209261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227753+0000","last_change":"2024-01-24T13:58:18.803776+0000","last_active":"2024-01-24T13:58:23.227753+0000","last_peered":"2024-01-24T13:58:23.227753+0000","last_clean":"2024-01-24T13:58:23.227753+0000","last_became_active":"2024-01-24T13:58:18.803485+0000","last_became_peered":"2024-01-24T13:58:18.803485+0000","last_unstale":"2024-01-24T13:58:23.227753+0000","last_undegraded":"2024-01-24T13:58:23.227753+0000","last_fullsized":"2024-01-24T13:58:23.227753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.234859+0000","last_change":"2024-01-24T13:58:19.224975+0000","last_active":"2024-01-24T13:58:23.234859+0000","last_peered":"2024-01-24T13:58:23.234859+0000","last_clean":"2024-01-24T13:58:23.234859+0000","last_became_active":"2024-01-24T13:58:19.224568+0000","last_became_peered":"2024-01-24T13:58:19.224568+0000","last_unstale":"2024-01-24T13:58:23.234859+0000","last_undegraded":"2024-01-24T13:58:23.234859+0000","last_fullsized":"2024-01-24T13:58:23.234859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217357+0000","last_change":"2024-01-24T13:58:18.797963+0000","last_active":"2024-01-24T13:58:23.217357+0000","last_peered":"2024-01-24T13:58:23.217357+0000","last_clean":"2024-01-24T13:58:23.217357+0000","last_became_active":"2024-01-24T13:58:18.797320+0000","last_became_peered":"2024-01-24T13:58:18.797320+0000","last_unstale":"2024-01-24T13:58:23.217357+0000","last_undegraded":"2024-01-24T13:58:23.217357+0000","last_fullsized":"2024-01-24T13:58:23.217357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213193+0000","last_change":"2024-01-24T13:58:19.213147+0000","last_active":"2024-01-24T13:58:23.213193+0000","last_peered":"2024-01-24T13:58:23.213193+0000","last_clean":"2024-01-24T13:58:23.213193+0000","last_became_active":"2024-01-24T13:58:19.212854+0000","last_became_peered":"2024-01-24T13:58:19.212854+0000","last_unstale":"2024-01-24T13:58:23.213193+0000","last_undegraded":"2024-01-24T13:58:23.213193+0000","last_fullsized":"2024-01-24T13:58:23.213193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.228193+0000","last_change":"2024-01-24T13:58:18.812721+0000","last_active":"2024-01-24T13:58:23.228193+0000","last_peered":"2024-01-24T13:58:23.228193+0000","last_clean":"2024-01-24T13:58:23.228193+0000","last_became_active":"2024-01-24T13:58:18.812439+0000","last_became_peered":"2024-01-24T13:58:18.812439+0000","last_unstale":"2024-01-24T13:58:23.228193+0000","last_undegraded":"2024-01-24T13:58:23.228193+0000","last_fullsized":"2024-01-24T13:58:23.228193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209659+0000","last_change":"2024-01-24T13:58:19.213672+0000","last_active":"2024-01-24T13:58:23.209659+0000","last_peered":"2024-01-24T13:58:23.209659+0000","last_clean":"2024-01-24T13:58:23.209659+0000","last_became_active":"2024-01-24T13:58:19.213378+0000","last_became_peered":"2024-01-24T13:58:19.213378+0000","last_unstale":"2024-01-24T13:58:23.209659+0000","last_undegraded":"2024-01-24T13:58:23.209659+0000","last_fullsized":"2024-01-24T13:58:23.209659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216204+0000","last_change":"2024-01-24T13:58:18.806273+0000","last_active":"2024-01-24T13:58:23.216204+0000","last_peered":"2024-01-24T13:58:23.216204+0000","last_clean":"2024-01-24T13:58:23.216204+0000","last_became_active":"2024-01-24T13:58:18.805970+0000","last_became_peered":"2024-01-24T13:58:18.805970+0000","last_unstale":"2024-01-24T13:58:23.216204+0000","last_undegraded":"2024-01-24T13:58:23.216204+0000","last_fullsized":"2024-01-24T13:58:23.216204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227834+0000","last_change":"2024-01-24T13:58:19.208876+0000","last_active":"2024-01-24T13:58:23.227834+0000","last_peered":"2024-01-24T13:58:23.227834+0000","last_clean":"2024-01-24T13:58:23.227834+0000","last_became_active":"2024-01-24T13:58:19.208537+0000","last_became_peered":"2024-01-24T13:58:19.208537+0000","last_unstale":"2024-01-24T13:58:23.227834+0000","last_undegraded":"2024-01-24T13:58:23.227834+0000","last_fullsized":"2024-01-24T13:58:23.227834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208114+0000","last_change":"2024-01-24T13:58:18.801957+0000","last_active":"2024-01-24T13:58:23.208114+0000","last_peered":"2024-01-24T13:58:23.208114+0000","last_clean":"2024-01-24T13:58:23.208114+0000","last_became_active":"2024-01-24T13:58:18.801641+0000","last_became_peered":"2024-01-24T13:58:18.801641+0000","last_unstale":"2024-01-24T13:58:23.208114+0000","last_undegraded":"2024-01-24T13:58:23.208114+0000","last_fullsized":"2024-01-24T13:58:23.208114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.222487+0000","last_change":"2024-01-24T13:58:19.217744+0000","last_active":"2024-01-24T13:58:23.222487+0000","last_peered":"2024-01-24T13:58:23.222487+0000","last_clean":"2024-01-24T13:58:23.222487+0000","last_became_active":"2024-01-24T13:58:19.217212+0000","last_became_peered":"2024-01-24T13:58:19.217212+0000","last_unstale":"2024-01-24T13:58:23.222487+0000","last_undegraded":"2024-01-24T13:58:23.222487+0000","last_fullsized":"2024-01-24T13:58:23.222487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.218321+0000","last_change":"2024-01-24T13:58:18.797573+0000","last_active":"2024-01-24T13:58:23.218321+0000","last_peered":"2024-01-24T13:58:23.218321+0000","last_clean":"2024-01-24T13:58:23.218321+0000","last_became_active":"2024-01-24T13:58:18.797100+0000","last_became_peered":"2024-01-24T13:58:18.797100+0000","last_unstale":"2024-01-24T13:58:23.218321+0000","last_undegraded":"2024-01-24T13:58:23.218321+0000","last_fullsized":"2024-01-24T13:58:23.218321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209573+0000","last_change":"2024-01-24T13:58:19.211019+0000","last_active":"2024-01-24T13:58:23.209573+0000","last_peered":"2024-01-24T13:58:23.209573+0000","last_clean":"2024-01-24T13:58:23.209573+0000","last_became_active":"2024-01-24T13:58:19.210424+0000","last_became_peered":"2024-01-24T13:58:19.210424+0000","last_unstale":"2024-01-24T13:58:23.209573+0000","last_undegraded":"2024-01-24T13:58:23.209573+0000","last_fullsized":"2024-01-24T13:58:23.209573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213015+0000","last_change":"2024-01-24T13:58:18.800935+0000","last_active":"2024-01-24T13:58:23.213015+0000","last_peered":"2024-01-24T13:58:23.213015+0000","last_clean":"2024-01-24T13:58:23.213015+0000","last_became_active":"2024-01-24T13:58:18.800629+0000","last_became_peered":"2024-01-24T13:58:18.800629+0000","last_unstale":"2024-01-24T13:58:23.213015+0000","last_undegraded":"2024-01-24T13:58:23.213015+0000","last_fullsized":"2024-01-24T13:58:23.213015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216389+0000","last_change":"2024-01-24T13:58:19.214756+0000","last_active":"2024-01-24T13:58:23.216389+0000","last_peered":"2024-01-24T13:58:23.216389+0000","last_clean":"2024-01-24T13:58:23.216389+0000","last_became_active":"2024-01-24T13:58:19.214127+0000","last_became_peered":"2024-01-24T13:58:19.214127+0000","last_unstale":"2024-01-24T13:58:23.216389+0000","last_undegraded":"2024-01-24T13:58:23.216389+0000","last_fullsized":"2024-01-24T13:58:23.216389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213067+0000","last_change":"2024-01-24T13:58:18.804647+0000","last_active":"2024-01-24T13:58:23.213067+0000","last_peered":"2024-01-24T13:58:23.213067+0000","last_clean":"2024-01-24T13:58:23.213067+0000","last_became_active":"2024-01-24T13:58:18.803996+0000","last_became_peered":"2024-01-24T13:58:18.803996+0000","last_unstale":"2024-01-24T13:58:23.213067+0000","last_undegraded":"2024-01-24T13:58:23.213067+0000","last_fullsized":"2024-01-24T13:58:23.213067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226664+0000","last_change":"2024-01-24T13:58:19.210603+0000","last_active":"2024-01-24T13:58:23.226664+0000","last_peered":"2024-01-24T13:58:23.226664+0000","last_clean":"2024-01-24T13:58:23.226664+0000","last_became_active":"2024-01-24T13:58:19.210245+0000","last_became_peered":"2024-01-24T13:58:19.210245+0000","last_unstale":"2024-01-24T13:58:23.226664+0000","last_undegraded":"2024-01-24T13:58:23.226664+0000","last_fullsized":"2024-01-24T13:58:23.226664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.34","version":"26'7","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-24T13:58:28.635902+0000","last_change":"2024-01-24T13:58:18.809410+0000","last_active":"2024-01-24T13:58:28.635902+0000","last_peered":"2024-01-24T13:58:28.635902+0000","last_clean":"2024-01-24T13:58:28.635902+0000","last_became_active":"2024-01-24T13:58:18.809115+0000","last_became_peered":"2024-01-24T13:58:18.809115+0000","last_unstale":"2024-01-24T13:58:28.635902+0000","last_undegraded":"2024-01-24T13:58:28.635902+0000","last_fullsized":"2024-01-24T13:58:28.635902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3071,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220937+0000","last_change":"2024-01-24T13:58:19.217910+0000","last_active":"2024-01-24T13:58:23.220937+0000","last_peered":"2024-01-24T13:58:23.220937+0000","last_clean":"2024-01-24T13:58:23.220937+0000","last_became_active":"2024-01-24T13:58:19.216908+0000","last_became_peered":"2024-01-24T13:58:19.216908+0000","last_unstale":"2024-01-24T13:58:23.220937+0000","last_undegraded":"2024-01-24T13:58:23.220937+0000","last_fullsized":"2024-01-24T13:58:23.220937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208119+0000","last_change":"2024-01-24T13:58:18.807547+0000","last_active":"2024-01-24T13:58:23.208119+0000","last_peered":"2024-01-24T13:58:23.208119+0000","last_clean":"2024-01-24T13:58:23.208119+0000","last_became_active":"2024-01-24T13:58:18.806845+0000","last_became_peered":"2024-01-24T13:58:18.806845+0000","last_unstale":"2024-01-24T13:58:23.208119+0000","last_undegraded":"2024-01-24T13:58:23.208119+0000","last_fullsized":"2024-01-24T13:58:23.208119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224004+0000","last_change":"2024-01-24T13:58:19.219483+0000","last_active":"2024-01-24T13:58:23.224004+0000","last_peered":"2024-01-24T13:58:23.224004+0000","last_clean":"2024-01-24T13:58:23.224004+0000","last_became_active":"2024-01-24T13:58:19.219182+0000","last_became_peered":"2024-01-24T13:58:19.219182+0000","last_unstale":"2024-01-24T13:58:23.224004+0000","last_undegraded":"2024-01-24T13:58:23.224004+0000","last_fullsized":"2024-01-24T13:58:23.224004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232615+0000","last_change":"2024-01-24T13:58:18.804144+0000","last_active":"2024-01-24T13:58:23.232615+0000","last_peered":"2024-01-24T13:58:23.232615+0000","last_clean":"2024-01-24T13:58:23.232615+0000","last_became_active":"2024-01-24T13:58:18.803873+0000","last_became_peered":"2024-01-24T13:58:18.803873+0000","last_unstale":"2024-01-24T13:58:23.232615+0000","last_undegraded":"2024-01-24T13:58:23.232615+0000","last_fullsized":"2024-01-24T13:58:23.232615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209346+0000","last_change":"2024-01-24T13:58:19.221162+0000","last_active":"2024-01-24T13:58:23.209346+0000","last_peered":"2024-01-24T13:58:23.209346+0000","last_clean":"2024-01-24T13:58:23.209346+0000","last_became_active":"2024-01-24T13:58:19.220665+0000","last_became_peered":"2024-01-24T13:58:19.220665+0000","last_unstale":"2024-01-24T13:58:23.209346+0000","last_undegraded":"2024-01-24T13:58:23.209346+0000","last_fullsized":"2024-01-24T13:58:23.209346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.210124+0000","last_change":"2024-01-24T13:58:18.799003+0000","last_active":"2024-01-24T13:58:23.210124+0000","last_peered":"2024-01-24T13:58:23.210124+0000","last_clean":"2024-01-24T13:58:23.210124+0000","last_became_active":"2024-01-24T13:58:18.796638+0000","last_became_peered":"2024-01-24T13:58:18.796638+0000","last_unstale":"2024-01-24T13:58:23.210124+0000","last_undegraded":"2024-01-24T13:58:23.210124+0000","last_fullsized":"2024-01-24T13:58:23.210124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227756+0000","last_change":"2024-01-24T13:58:19.213039+0000","last_active":"2024-01-24T13:58:23.227756+0000","last_peered":"2024-01-24T13:58:23.227756+0000","last_clean":"2024-01-24T13:58:23.227756+0000","last_became_active":"2024-01-24T13:58:19.212755+0000","last_became_peered":"2024-01-24T13:58:19.212755+0000","last_unstale":"2024-01-24T13:58:23.227756+0000","last_undegraded":"2024-01-24T13:58:23.227756+0000","last_fullsized":"2024-01-24T13:58:23.227756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"21'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.225127+0000","last_change":"2024-01-24T13:58:18.805645+0000","last_active":"2024-01-24T13:58:23.225127+0000","last_peered":"2024-01-24T13:58:23.225127+0000","last_clean":"2024-01-24T13:58:23.225127+0000","last_became_active":"2024-01-24T13:58:18.805375+0000","last_became_peered":"2024-01-24T13:58:18.805375+0000","last_unstale":"2024-01-24T13:58:23.225127+0000","last_undegraded":"2024-01-24T13:58:23.225127+0000","last_fullsized":"2024-01-24T13:58:23.225127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237611+0000","last_change":"2024-01-24T13:58:19.225874+0000","last_active":"2024-01-24T13:58:23.237611+0000","last_peered":"2024-01-24T13:58:23.237611+0000","last_clean":"2024-01-24T13:58:23.237611+0000","last_became_active":"2024-01-24T13:58:19.225220+0000","last_became_peered":"2024-01-24T13:58:19.225220+0000","last_unstale":"2024-01-24T13:58:23.237611+0000","last_undegraded":"2024-01-24T13:58:23.237611+0000","last_fullsized":"2024-01-24T13:58:23.237611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.233153+0000","last_change":"2024-01-24T13:58:18.797127+0000","last_active":"2024-01-24T13:58:23.233153+0000","last_peered":"2024-01-24T13:58:23.233153+0000","last_clean":"2024-01-24T13:58:23.233153+0000","last_became_active":"2024-01-24T13:58:18.796816+0000","last_became_peered":"2024-01-24T13:58:18.796816+0000","last_unstale":"2024-01-24T13:58:23.233153+0000","last_undegraded":"2024-01-24T13:58:23.233153+0000","last_fullsized":"2024-01-24T13:58:23.233153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216434+0000","last_change":"2024-01-24T13:58:19.219593+0000","last_active":"2024-01-24T13:58:23.216434+0000","last_peered":"2024-01-24T13:58:23.216434+0000","last_clean":"2024-01-24T13:58:23.216434+0000","last_became_active":"2024-01-24T13:58:19.219267+0000","last_became_peered":"2024-01-24T13:58:19.219267+0000","last_unstale":"2024-01-24T13:58:23.216434+0000","last_undegraded":"2024-01-24T13:58:23.216434+0000","last_fullsized":"2024-01-24T13:58:23.216434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220392+0000","last_change":"2024-01-24T13:58:18.805694+0000","last_active":"2024-01-24T13:58:23.220392+0000","last_peered":"2024-01-24T13:58:23.220392+0000","last_clean":"2024-01-24T13:58:23.220392+0000","last_became_active":"2024-01-24T13:58:18.805335+0000","last_became_peered":"2024-01-24T13:58:18.805335+0000","last_unstale":"2024-01-24T13:58:23.220392+0000","last_undegraded":"2024-01-24T13:58:23.220392+0000","last_fullsized":"2024-01-24T13:58:23.220392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207983+0000","last_change":"2024-01-24T13:58:19.213662+0000","last_active":"2024-01-24T13:58:23.207983+0000","last_peered":"2024-01-24T13:58:23.207983+0000","last_clean":"2024-01-24T13:58:23.207983+0000","last_became_active":"2024-01-24T13:58:19.213340+0000","last_became_peered":"2024-01-24T13:58:19.213340+0000","last_unstale":"2024-01-24T13:58:23.207983+0000","last_undegraded":"2024-01-24T13:58:23.207983+0000","last_fullsized":"2024-01-24T13:58:23.207983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2d","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217312+0000","last_change":"2024-01-24T13:58:18.798262+0000","last_active":"2024-01-24T13:58:23.217312+0000","last_peered":"2024-01-24T13:58:23.217312+0000","last_clean":"2024-01-24T13:58:23.217312+0000","last_became_active":"2024-01-24T13:58:18.797856+0000","last_became_peered":"2024-01-24T13:58:18.797856+0000","last_unstale":"2024-01-24T13:58:23.217312+0000","last_undegraded":"2024-01-24T13:58:23.217312+0000","last_fullsized":"2024-01-24T13:58:23.217312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232445+0000","last_change":"2024-01-24T13:58:19.223736+0000","last_active":"2024-01-24T13:58:23.232445+0000","last_peered":"2024-01-24T13:58:23.232445+0000","last_clean":"2024-01-24T13:58:23.232445+0000","last_became_active":"2024-01-24T13:58:19.223500+0000","last_became_peered":"2024-01-24T13:58:19.223500+0000","last_unstale":"2024-01-24T13:58:23.232445+0000","last_undegraded":"2024-01-24T13:58:23.232445+0000","last_fullsized":"2024-01-24T13:58:23.232445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213312+0000","last_change":"2024-01-24T13:58:18.802360+0000","last_active":"2024-01-24T13:58:23.213312+0000","last_peered":"2024-01-24T13:58:23.213312+0000","last_clean":"2024-01-24T13:58:23.213312+0000","last_became_active":"2024-01-24T13:58:18.802051+0000","last_became_peered":"2024-01-24T13:58:18.802051+0000","last_unstale":"2024-01-24T13:58:23.213312+0000","last_undegraded":"2024-01-24T13:58:23.213312+0000","last_fullsized":"2024-01-24T13:58:23.213312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217156+0000","last_change":"2024-01-24T13:58:19.218929+0000","last_active":"2024-01-24T13:58:23.217156+0000","last_peered":"2024-01-24T13:58:23.217156+0000","last_clean":"2024-01-24T13:58:23.217156+0000","last_became_active":"2024-01-24T13:58:19.218350+0000","last_became_peered":"2024-01-24T13:58:19.218350+0000","last_unstale":"2024-01-24T13:58:23.217156+0000","last_undegraded":"2024-01-24T13:58:23.217156+0000","last_fullsized":"2024-01-24T13:58:23.217156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2b","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235138+0000","last_change":"2024-01-24T13:58:18.810840+0000","last_active":"2024-01-24T13:58:23.235138+0000","last_peered":"2024-01-24T13:58:23.235138+0000","last_clean":"2024-01-24T13:58:23.235138+0000","last_became_active":"2024-01-24T13:58:18.809023+0000","last_became_peered":"2024-01-24T13:58:18.809023+0000","last_unstale":"2024-01-24T13:58:23.235138+0000","last_undegraded":"2024-01-24T13:58:23.235138+0000","last_fullsized":"2024-01-24T13:58:23.235138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209489+0000","last_change":"2024-01-24T13:58:19.217855+0000","last_active":"2024-01-24T13:58:23.209489+0000","last_peered":"2024-01-24T13:58:23.209489+0000","last_clean":"2024-01-24T13:58:23.209489+0000","last_became_active":"2024-01-24T13:58:19.217559+0000","last_became_peered":"2024-01-24T13:58:19.217559+0000","last_unstale":"2024-01-24T13:58:23.209489+0000","last_undegraded":"2024-01-24T13:58:23.209489+0000","last_fullsized":"2024-01-24T13:58:23.209489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226871+0000","last_change":"2024-01-24T13:58:18.798611+0000","last_active":"2024-01-24T13:58:23.226871+0000","last_peered":"2024-01-24T13:58:23.226871+0000","last_clean":"2024-01-24T13:58:23.226871+0000","last_became_active":"2024-01-24T13:58:18.798325+0000","last_became_peered":"2024-01-24T13:58:18.798325+0000","last_unstale":"2024-01-24T13:58:23.226871+0000","last_undegraded":"2024-01-24T13:58:23.226871+0000","last_fullsized":"2024-01-24T13:58:23.226871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.222261+0000","last_change":"2024-01-24T13:58:19.206846+0000","last_active":"2024-01-24T13:58:23.222261+0000","last_peered":"2024-01-24T13:58:23.222261+0000","last_clean":"2024-01-24T13:58:23.222261+0000","last_became_active":"2024-01-24T13:58:19.206552+0000","last_became_peered":"2024-01-24T13:58:19.206552+0000","last_unstale":"2024-01-24T13:58:23.222261+0000","last_undegraded":"2024-01-24T13:58:23.222261+0000","last_fullsized":"2024-01-24T13:58:23.222261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207449+0000","last_change":"2024-01-24T13:58:18.801628+0000","last_active":"2024-01-24T13:58:23.207449+0000","last_peered":"2024-01-24T13:58:23.207449+0000","last_clean":"2024-01-24T13:58:23.207449+0000","last_became_active":"2024-01-24T13:58:18.801291+0000","last_became_peered":"2024-01-24T13:58:18.801291+0000","last_unstale":"2024-01-24T13:58:23.207449+0000","last_undegraded":"2024-01-24T13:58:23.207449+0000","last_fullsized":"2024-01-24T13:58:23.207449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209637+0000","last_change":"2024-01-24T13:58:19.211347+0000","last_active":"2024-01-24T13:58:23.209637+0000","last_peered":"2024-01-24T13:58:23.209637+0000","last_clean":"2024-01-24T13:58:23.209637+0000","last_became_active":"2024-01-24T13:58:19.210298+0000","last_became_peered":"2024-01-24T13:58:19.210298+0000","last_unstale":"2024-01-24T13:58:23.209637+0000","last_undegraded":"2024-01-24T13:58:23.209637+0000","last_fullsized":"2024-01-24T13:58:23.209637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224993+0000","last_change":"2024-01-24T13:58:18.803399+0000","last_active":"2024-01-24T13:58:23.224993+0000","last_peered":"2024-01-24T13:58:23.224993+0000","last_clean":"2024-01-24T13:58:23.224993+0000","last_became_active":"2024-01-24T13:58:18.803109+0000","last_became_peered":"2024-01-24T13:58:18.803109+0000","last_unstale":"2024-01-24T13:58:23.224993+0000","last_undegraded":"2024-01-24T13:58:23.224993+0000","last_fullsized":"2024-01-24T13:58:23.224993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238205+0000","last_change":"2024-01-24T13:58:19.225710+0000","last_active":"2024-01-24T13:58:23.238205+0000","last_peered":"2024-01-24T13:58:23.238205+0000","last_clean":"2024-01-24T13:58:23.238205+0000","last_became_active":"2024-01-24T13:58:19.224844+0000","last_became_peered":"2024-01-24T13:58:19.224844+0000","last_unstale":"2024-01-24T13:58:23.238205+0000","last_undegraded":"2024-01-24T13:58:23.238205+0000","last_fullsized":"2024-01-24T13:58:23.238205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208117+0000","last_change":"2024-01-24T13:58:18.801623+0000","last_active":"2024-01-24T13:58:23.208117+0000","last_peered":"2024-01-24T13:58:23.208117+0000","last_clean":"2024-01-24T13:58:23.208117+0000","last_became_active":"2024-01-24T13:58:18.801272+0000","last_became_peered":"2024-01-24T13:58:18.801272+0000","last_unstale":"2024-01-24T13:58:23.208117+0000","last_undegraded":"2024-01-24T13:58:23.208117+0000","last_fullsized":"2024-01-24T13:58:23.208117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212308+0000","last_change":"2024-01-24T13:58:19.207457+0000","last_active":"2024-01-24T13:58:23.212308+0000","last_peered":"2024-01-24T13:58:23.212308+0000","last_clean":"2024-01-24T13:58:23.212308+0000","last_became_active":"2024-01-24T13:58:19.207133+0000","last_became_peered":"2024-01-24T13:58:19.207133+0000","last_unstale":"2024-01-24T13:58:23.212308+0000","last_undegraded":"2024-01-24T13:58:23.212308+0000","last_fullsized":"2024-01-24T13:58:23.212308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"26'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.880144+0000","last_change":"2024-01-24T13:58:18.800410+0000","last_active":"2024-01-24T13:58:23.880144+0000","last_peered":"2024-01-24T13:58:23.880144+0000","last_clean":"2024-01-24T13:58:23.880144+0000","last_became_active":"2024-01-24T13:58:18.800073+0000","last_became_peered":"2024-01-24T13:58:18.800073+0000","last_unstale":"2024-01-24T13:58:23.880144+0000","last_undegraded":"2024-01-24T13:58:23.880144+0000","last_fullsized":"2024-01-24T13:58:23.880144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207934+0000","last_change":"2024-01-24T13:58:19.221135+0000","last_active":"2024-01-24T13:58:23.207934+0000","last_peered":"2024-01-24T13:58:23.207934+0000","last_clean":"2024-01-24T13:58:23.207934+0000","last_became_active":"2024-01-24T13:58:19.220863+0000","last_became_peered":"2024-01-24T13:58:19.220863+0000","last_unstale":"2024-01-24T13:58:23.207934+0000","last_undegraded":"2024-01-24T13:58:23.207934+0000","last_fullsized":"2024-01-24T13:58:23.207934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211332+0000","last_change":"2024-01-24T13:58:18.800232+0000","last_active":"2024-01-24T13:58:23.211332+0000","last_peered":"2024-01-24T13:58:23.211332+0000","last_clean":"2024-01-24T13:58:23.211332+0000","last_became_active":"2024-01-24T13:58:18.799722+0000","last_became_peered":"2024-01-24T13:58:18.799722+0000","last_unstale":"2024-01-24T13:58:23.211332+0000","last_undegraded":"2024-01-24T13:58:23.211332+0000","last_fullsized":"2024-01-24T13:58:23.211332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209141+0000","last_change":"2024-01-24T13:58:19.215326+0000","last_active":"2024-01-24T13:58:23.209141+0000","last_peered":"2024-01-24T13:58:23.209141+0000","last_clean":"2024-01-24T13:58:23.209141+0000","last_became_active":"2024-01-24T13:58:19.214852+0000","last_became_peered":"2024-01-24T13:58:19.214852+0000","last_unstale":"2024-01-24T13:58:23.209141+0000","last_undegraded":"2024-01-24T13:58:23.209141+0000","last_fullsized":"2024-01-24T13:58:23.209141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207199+0000","last_change":"2024-01-24T13:58:18.804258+0000","last_active":"2024-01-24T13:58:23.207199+0000","last_peered":"2024-01-24T13:58:23.207199+0000","last_clean":"2024-01-24T13:58:23.207199+0000","last_became_active":"2024-01-24T13:58:18.802440+0000","last_became_peered":"2024-01-24T13:58:18.802440+0000","last_unstale":"2024-01-24T13:58:23.207199+0000","last_undegraded":"2024-01-24T13:58:23.207199+0000","last_fullsized":"2024-01-24T13:58:23.207199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208000+0000","last_change":"2024-01-24T13:58:19.214888+0000","last_active":"2024-01-24T13:58:23.208000+0000","last_peered":"2024-01-24T13:58:23.208000+0000","last_clean":"2024-01-24T13:58:23.208000+0000","last_became_active":"2024-01-24T13:58:19.214607+0000","last_became_peered":"2024-01-24T13:58:19.214607+0000","last_unstale":"2024-01-24T13:58:23.208000+0000","last_undegraded":"2024-01-24T13:58:23.208000+0000","last_fullsized":"2024-01-24T13:58:23.208000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235037+0000","last_change":"2024-01-24T13:58:18.813467+0000","last_active":"2024-01-24T13:58:23.235037+0000","last_peered":"2024-01-24T13:58:23.235037+0000","last_clean":"2024-01-24T13:58:23.235037+0000","last_became_active":"2024-01-24T13:58:18.813164+0000","last_became_peered":"2024-01-24T13:58:18.813164+0000","last_unstale":"2024-01-24T13:58:23.235037+0000","last_undegraded":"2024-01-24T13:58:23.235037+0000","last_fullsized":"2024-01-24T13:58:23.235037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211738+0000","last_change":"2024-01-24T13:58:19.212196+0000","last_active":"2024-01-24T13:58:23.211738+0000","last_peered":"2024-01-24T13:58:23.211738+0000","last_clean":"2024-01-24T13:58:23.211738+0000","last_became_active":"2024-01-24T13:58:19.211820+0000","last_became_peered":"2024-01-24T13:58:19.211820+0000","last_unstale":"2024-01-24T13:58:23.211738+0000","last_undegraded":"2024-01-24T13:58:23.211738+0000","last_fullsized":"2024-01-24T13:58:23.211738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226611+0000","last_change":"2024-01-24T13:58:18.802179+0000","last_active":"2024-01-24T13:58:23.226611+0000","last_peered":"2024-01-24T13:58:23.226611+0000","last_clean":"2024-01-24T13:58:23.226611+0000","last_became_active":"2024-01-24T13:58:18.801891+0000","last_became_peered":"2024-01-24T13:58:18.801891+0000","last_unstale":"2024-01-24T13:58:23.226611+0000","last_undegraded":"2024-01-24T13:58:23.226611+0000","last_fullsized":"2024-01-24T13:58:23.226611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227791+0000","last_change":"2024-01-24T13:58:19.217207+0000","last_active":"2024-01-24T13:58:23.227791+0000","last_peered":"2024-01-24T13:58:23.227791+0000","last_clean":"2024-01-24T13:58:23.227791+0000","last_became_active":"2024-01-24T13:58:19.216920+0000","last_became_peered":"2024-01-24T13:58:19.216920+0000","last_unstale":"2024-01-24T13:58:23.227791+0000","last_undegraded":"2024-01-24T13:58:23.227791+0000","last_fullsized":"2024-01-24T13:58:23.227791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220454+0000","last_change":"2024-01-24T13:58:18.793887+0000","last_active":"2024-01-24T13:58:23.220454+0000","last_peered":"2024-01-24T13:58:23.220454+0000","last_clean":"2024-01-24T13:58:23.220454+0000","last_became_active":"2024-01-24T13:58:18.793542+0000","last_became_peered":"2024-01-24T13:58:18.793542+0000","last_unstale":"2024-01-24T13:58:23.220454+0000","last_undegraded":"2024-01-24T13:58:23.220454+0000","last_fullsized":"2024-01-24T13:58:23.220454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211617+0000","last_change":"2024-01-24T13:58:19.216131+0000","last_active":"2024-01-24T13:58:23.211617+0000","last_peered":"2024-01-24T13:58:23.211617+0000","last_clean":"2024-01-24T13:58:23.211617+0000","last_became_active":"2024-01-24T13:58:19.215566+0000","last_became_peered":"2024-01-24T13:58:19.215566+0000","last_unstale":"2024-01-24T13:58:23.211617+0000","last_undegraded":"2024-01-24T13:58:23.211617+0000","last_fullsized":"2024-01-24T13:58:23.211617+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212581+0000","last_change":"2024-01-24T13:58:18.808952+0000","last_active":"2024-01-24T13:58:23.212581+0000","last_peered":"2024-01-24T13:58:23.212581+0000","last_clean":"2024-01-24T13:58:23.212581+0000","last_became_active":"2024-01-24T13:58:18.808672+0000","last_became_peered":"2024-01-24T13:58:18.808672+0000","last_unstale":"2024-01-24T13:58:23.212581+0000","last_undegraded":"2024-01-24T13:58:23.212581+0000","last_fullsized":"2024-01-24T13:58:23.212581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211211+0000","last_change":"2024-01-24T13:58:19.213798+0000","last_active":"2024-01-24T13:58:23.211211+0000","last_peered":"2024-01-24T13:58:23.211211+0000","last_clean":"2024-01-24T13:58:23.211211+0000","last_became_active":"2024-01-24T13:58:19.212458+0000","last_became_peered":"2024-01-24T13:58:19.212458+0000","last_unstale":"2024-01-24T13:58:23.211211+0000","last_undegraded":"2024-01-24T13:58:23.211211+0000","last_fullsized":"2024-01-24T13:58:23.211211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238030+0000","last_change":"2024-01-24T13:58:18.811471+0000","last_active":"2024-01-24T13:58:23.238030+0000","last_peered":"2024-01-24T13:58:23.238030+0000","last_clean":"2024-01-24T13:58:23.238030+0000","last_became_active":"2024-01-24T13:58:18.811159+0000","last_became_peered":"2024-01-24T13:58:18.811159+0000","last_unstale":"2024-01-24T13:58:23.238030+0000","last_undegraded":"2024-01-24T13:58:23.238030+0000","last_fullsized":"2024-01-24T13:58:23.238030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209454+0000","last_change":"2024-01-24T13:58:19.219125+0000","last_active":"2024-01-24T13:58:23.209454+0000","last_peered":"2024-01-24T13:58:23.209454+0000","last_clean":"2024-01-24T13:58:23.209454+0000","last_became_active":"2024-01-24T13:58:19.218823+0000","last_became_peered":"2024-01-24T13:58:19.218823+0000","last_unstale":"2024-01-24T13:58:23.209454+0000","last_undegraded":"2024-01-24T13:58:23.209454+0000","last_fullsized":"2024-01-24T13:58:23.209454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241886+0000","last_change":"2024-01-24T13:58:23.236176+0000","last_active":"2024-01-24T13:58:23.241886+0000","last_peered":"2024-01-24T13:58:23.241886+0000","last_clean":"2024-01-24T13:58:23.241886+0000","last_became_active":"2024-01-24T13:58:23.235658+0000","last_became_peered":"2024-01-24T13:58:23.235658+0000","last_unstale":"2024-01-24T13:58:23.241886+0000","last_undegraded":"2024-01-24T13:58:23.241886+0000","last_fullsized":"2024-01-24T13:58:23.241886+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207471+0000","last_change":"2024-01-24T13:58:18.806459+0000","last_active":"2024-01-24T13:58:23.207471+0000","last_peered":"2024-01-24T13:58:23.207471+0000","last_clean":"2024-01-24T13:58:23.207471+0000","last_became_active":"2024-01-24T13:58:18.806150+0000","last_became_peered":"2024-01-24T13:58:18.806150+0000","last_unstale":"2024-01-24T13:58:23.207471+0000","last_undegraded":"2024-01-24T13:58:23.207471+0000","last_fullsized":"2024-01-24T13:58:23.207471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207748+0000","last_change":"2024-01-24T13:58:19.211910+0000","last_active":"2024-01-24T13:58:23.207748+0000","last_peered":"2024-01-24T13:58:23.207748+0000","last_clean":"2024-01-24T13:58:23.207748+0000","last_became_active":"2024-01-24T13:58:19.210919+0000","last_became_peered":"2024-01-24T13:58:19.210919+0000","last_unstale":"2024-01-24T13:58:23.207748+0000","last_undegraded":"2024-01-24T13:58:23.207748+0000","last_fullsized":"2024-01-24T13:58:23.207748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232409+0000","last_change":"2024-01-24T13:58:23.223847+0000","last_active":"2024-01-24T13:58:23.232409+0000","last_peered":"2024-01-24T13:58:23.232409+0000","last_clean":"2024-01-24T13:58:23.232409+0000","last_became_active":"2024-01-24T13:58:23.222064+0000","last_became_peered":"2024-01-24T13:58:23.222064+0000","last_unstale":"2024-01-24T13:58:23.232409+0000","last_undegraded":"2024-01-24T13:58:23.232409+0000","last_fullsized":"2024-01-24T13:58:23.232409+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227575+0000","last_change":"2024-01-24T13:58:18.798060+0000","last_active":"2024-01-24T13:58:23.227575+0000","last_peered":"2024-01-24T13:58:23.227575+0000","last_clean":"2024-01-24T13:58:23.227575+0000","last_became_active":"2024-01-24T13:58:18.797774+0000","last_became_peered":"2024-01-24T13:58:18.797774+0000","last_unstale":"2024-01-24T13:58:23.227575+0000","last_undegraded":"2024-01-24T13:58:23.227575+0000","last_fullsized":"2024-01-24T13:58:23.227575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208950+0000","last_change":"2024-01-24T13:58:19.215145+0000","last_active":"2024-01-24T13:58:23.208950+0000","last_peered":"2024-01-24T13:58:23.208950+0000","last_clean":"2024-01-24T13:58:23.208950+0000","last_became_active":"2024-01-24T13:58:19.214840+0000","last_became_peered":"2024-01-24T13:58:19.214840+0000","last_unstale":"2024-01-24T13:58:23.208950+0000","last_undegraded":"2024-01-24T13:58:23.208950+0000","last_fullsized":"2024-01-24T13:58:23.208950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237348+0000","last_change":"2024-01-24T13:58:23.234486+0000","last_active":"2024-01-24T13:58:23.237348+0000","last_peered":"2024-01-24T13:58:23.237348+0000","last_clean":"2024-01-24T13:58:23.237348+0000","last_became_active":"2024-01-24T13:58:23.233948+0000","last_became_peered":"2024-01-24T13:58:23.233948+0000","last_unstale":"2024-01-24T13:58:23.237348+0000","last_undegraded":"2024-01-24T13:58:23.237348+0000","last_fullsized":"2024-01-24T13:58:23.237348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227976+0000","last_change":"2024-01-24T13:58:18.797763+0000","last_active":"2024-01-24T13:58:23.227976+0000","last_peered":"2024-01-24T13:58:23.227976+0000","last_clean":"2024-01-24T13:58:23.227976+0000","last_became_active":"2024-01-24T13:58:18.797478+0000","last_became_peered":"2024-01-24T13:58:18.797478+0000","last_unstale":"2024-01-24T13:58:23.227976+0000","last_undegraded":"2024-01-24T13:58:23.227976+0000","last_fullsized":"2024-01-24T13:58:23.227976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209496+0000","last_change":"2024-01-24T13:58:19.217185+0000","last_active":"2024-01-24T13:58:23.209496+0000","last_peered":"2024-01-24T13:58:23.209496+0000","last_clean":"2024-01-24T13:58:23.209496+0000","last_became_active":"2024-01-24T13:58:19.216628+0000","last_became_peered":"2024-01-24T13:58:19.216628+0000","last_unstale":"2024-01-24T13:58:23.209496+0000","last_undegraded":"2024-01-24T13:58:23.209496+0000","last_fullsized":"2024-01-24T13:58:23.209496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.236747+0000","last_change":"2024-01-24T13:58:23.230969+0000","last_active":"2024-01-24T13:58:23.236747+0000","last_peered":"2024-01-24T13:58:23.236747+0000","last_clean":"2024-01-24T13:58:23.236747+0000","last_became_active":"2024-01-24T13:58:23.230450+0000","last_became_peered":"2024-01-24T13:58:23.230450+0000","last_unstale":"2024-01-24T13:58:23.236747+0000","last_undegraded":"2024-01-24T13:58:23.236747+0000","last_fullsized":"2024-01-24T13:58:23.236747+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241684+0000","last_change":"2024-01-24T13:58:23.231930+0000","last_active":"2024-01-24T13:58:23.241684+0000","last_peered":"2024-01-24T13:58:23.241684+0000","last_clean":"2024-01-24T13:58:23.241684+0000","last_became_active":"2024-01-24T13:58:23.231494+0000","last_became_peered":"2024-01-24T13:58:23.231494+0000","last_unstale":"2024-01-24T13:58:23.241684+0000","last_undegraded":"2024-01-24T13:58:23.241684+0000","last_fullsized":"2024-01-24T13:58:23.241684+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235293+0000","last_change":"2024-01-24T13:58:18.811358+0000","last_active":"2024-01-24T13:58:23.235293+0000","last_peered":"2024-01-24T13:58:23.235293+0000","last_clean":"2024-01-24T13:58:23.235293+0000","last_became_active":"2024-01-24T13:58:18.811087+0000","last_became_peered":"2024-01-24T13:58:18.811087+0000","last_unstale":"2024-01-24T13:58:23.235293+0000","last_undegraded":"2024-01-24T13:58:23.235293+0000","last_fullsized":"2024-01-24T13:58:23.235293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224192+0000","last_change":"2024-01-24T13:58:19.213488+0000","last_active":"2024-01-24T13:58:23.224192+0000","last_peered":"2024-01-24T13:58:23.224192+0000","last_clean":"2024-01-24T13:58:23.224192+0000","last_became_active":"2024-01-24T13:58:19.213211+0000","last_became_peered":"2024-01-24T13:58:19.213211+0000","last_unstale":"2024-01-24T13:58:23.224192+0000","last_undegraded":"2024-01-24T13:58:23.224192+0000","last_fullsized":"2024-01-24T13:58:23.224192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209928+0000","last_change":"2024-01-24T13:58:19.216228+0000","last_active":"2024-01-24T13:58:23.209928+0000","last_peered":"2024-01-24T13:58:23.209928+0000","last_clean":"2024-01-24T13:58:23.209928+0000","last_became_active":"2024-01-24T13:58:19.215952+0000","last_became_peered":"2024-01-24T13:58:19.215952+0000","last_unstale":"2024-01-24T13:58:23.209928+0000","last_undegraded":"2024-01-24T13:58:23.209928+0000","last_fullsized":"2024-01-24T13:58:23.209928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237483+0000","last_change":"2024-01-24T13:58:18.813343+0000","last_active":"2024-01-24T13:58:23.237483+0000","last_peered":"2024-01-24T13:58:23.237483+0000","last_clean":"2024-01-24T13:58:23.237483+0000","last_became_active":"2024-01-24T13:58:18.812987+0000","last_became_peered":"2024-01-24T13:58:18.812987+0000","last_unstale":"2024-01-24T13:58:23.237483+0000","last_undegraded":"2024-01-24T13:58:23.237483+0000","last_fullsized":"2024-01-24T13:58:23.237483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213473+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T13:58:23.213473+0000","last_peered":"2024-01-24T13:58:23.213473+0000","last_clean":"2024-01-24T13:58:23.213473+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T13:58:23.213473+0000","last_undegraded":"2024-01-24T13:58:23.213473+0000","last_fullsized":"2024-01-24T13:58:23.213473+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241577+0000","last_change":"2024-01-24T13:58:23.235347+0000","last_active":"2024-01-24T13:58:23.241577+0000","last_peered":"2024-01-24T13:58:23.241577+0000","last_clean":"2024-01-24T13:58:23.241577+0000","last_became_active":"2024-01-24T13:58:23.234902+0000","last_became_peered":"2024-01-24T13:58:23.234902+0000","last_unstale":"2024-01-24T13:58:23.241577+0000","last_undegraded":"2024-01-24T13:58:23.241577+0000","last_fullsized":"2024-01-24T13:58:23.241577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207482+0000","last_change":"2024-01-24T13:58:19.218418+0000","last_active":"2024-01-24T13:58:23.207482+0000","last_peered":"2024-01-24T13:58:23.207482+0000","last_clean":"2024-01-24T13:58:23.207482+0000","last_became_active":"2024-01-24T13:58:19.210988+0000","last_became_peered":"2024-01-24T13:58:19.210988+0000","last_unstale":"2024-01-24T13:58:23.207482+0000","last_undegraded":"2024-01-24T13:58:23.207482+0000","last_fullsized":"2024-01-24T13:58:23.207482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211503+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T13:58:23.211503+0000","last_peered":"2024-01-24T13:58:23.211503+0000","last_clean":"2024-01-24T13:58:23.211503+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T13:58:23.211503+0000","last_undegraded":"2024-01-24T13:58:23.211503+0000","last_fullsized":"2024-01-24T13:58:23.211503+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207918+0000","last_change":"2024-01-24T13:58:18.798154+0000","last_active":"2024-01-24T13:58:23.207918+0000","last_peered":"2024-01-24T13:58:23.207918+0000","last_clean":"2024-01-24T13:58:23.207918+0000","last_became_active":"2024-01-24T13:58:18.797819+0000","last_became_peered":"2024-01-24T13:58:18.797819+0000","last_unstale":"2024-01-24T13:58:23.207918+0000","last_undegraded":"2024-01-24T13:58:23.207918+0000","last_fullsized":"2024-01-24T13:58:23.207918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232475+0000","last_change":"2024-01-24T13:58:23.230154+0000","last_active":"2024-01-24T13:58:23.232475+0000","last_peered":"2024-01-24T13:58:23.232475+0000","last_clean":"2024-01-24T13:58:23.232475+0000","last_became_active":"2024-01-24T13:58:23.226934+0000","last_became_peered":"2024-01-24T13:58:23.226934+0000","last_unstale":"2024-01-24T13:58:23.232475+0000","last_undegraded":"2024-01-24T13:58:23.232475+0000","last_fullsized":"2024-01-24T13:58:23.232475+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235379+0000","last_change":"2024-01-24T13:58:19.223917+0000","last_active":"2024-01-24T13:58:23.235379+0000","last_peered":"2024-01-24T13:58:23.235379+0000","last_clean":"2024-01-24T13:58:23.235379+0000","last_became_active":"2024-01-24T13:58:19.223624+0000","last_became_peered":"2024-01-24T13:58:19.223624+0000","last_unstale":"2024-01-24T13:58:23.235379+0000","last_undegraded":"2024-01-24T13:58:23.235379+0000","last_fullsized":"2024-01-24T13:58:23.235379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211872+0000","last_change":"2024-01-24T13:58:18.799495+0000","last_active":"2024-01-24T13:58:23.211872+0000","last_peered":"2024-01-24T13:58:23.211872+0000","last_clean":"2024-01-24T13:58:23.211872+0000","last_became_active":"2024-01-24T13:58:18.796442+0000","last_became_peered":"2024-01-24T13:58:18.796442+0000","last_unstale":"2024-01-24T13:58:23.211872+0000","last_undegraded":"2024-01-24T13:58:23.211872+0000","last_fullsized":"2024-01-24T13:58:23.211872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217101+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T13:58:23.217101+0000","last_peered":"2024-01-24T13:58:23.217101+0000","last_clean":"2024-01-24T13:58:23.217101+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T13:58:23.217101+0000","last_undegraded":"2024-01-24T13:58:23.217101+0000","last_fullsized":"2024-01-24T13:58:23.217101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.230573+0000","last_change":"2024-01-24T13:58:23.228846+0000","last_active":"2024-01-24T13:58:23.230573+0000","last_peered":"2024-01-24T13:58:23.230573+0000","last_clean":"2024-01-24T13:58:23.230573+0000","last_became_active":"2024-01-24T13:58:23.228366+0000","last_became_peered":"2024-01-24T13:58:23.228366+0000","last_unstale":"2024-01-24T13:58:23.230573+0000","last_undegraded":"2024-01-24T13:58:23.230573+0000","last_fullsized":"2024-01-24T13:58:23.230573+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232638+0000","last_change":"2024-01-24T13:58:18.810910+0000","last_active":"2024-01-24T13:58:23.232638+0000","last_peered":"2024-01-24T13:58:23.232638+0000","last_clean":"2024-01-24T13:58:23.232638+0000","last_became_active":"2024-01-24T13:58:18.810566+0000","last_became_peered":"2024-01-24T13:58:18.810566+0000","last_unstale":"2024-01-24T13:58:23.232638+0000","last_undegraded":"2024-01-24T13:58:23.232638+0000","last_fullsized":"2024-01-24T13:58:23.232638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216517+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T13:58:23.216517+0000","last_peered":"2024-01-24T13:58:23.216517+0000","last_clean":"2024-01-24T13:58:23.216517+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T13:58:23.216517+0000","last_undegraded":"2024-01-24T13:58:23.216517+0000","last_fullsized":"2024-01-24T13:58:23.216517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237683+0000","last_change":"2024-01-24T13:58:19.209151+0000","last_active":"2024-01-24T13:58:23.237683+0000","last_peered":"2024-01-24T13:58:23.237683+0000","last_clean":"2024-01-24T13:58:23.237683+0000","last_became_active":"2024-01-24T13:58:19.208680+0000","last_became_peered":"2024-01-24T13:58:19.208680+0000","last_unstale":"2024-01-24T13:58:23.237683+0000","last_undegraded":"2024-01-24T13:58:23.237683+0000","last_fullsized":"2024-01-24T13:58:23.237683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237878+0000","last_change":"2024-01-24T13:58:23.235219+0000","last_active":"2024-01-24T13:58:23.237878+0000","last_peered":"2024-01-24T13:58:23.237878+0000","last_clean":"2024-01-24T13:58:23.237878+0000","last_became_active":"2024-01-24T13:58:23.234797+0000","last_became_peered":"2024-01-24T13:58:23.234797+0000","last_unstale":"2024-01-24T13:58:23.237878+0000","last_undegraded":"2024-01-24T13:58:23.237878+0000","last_fullsized":"2024-01-24T13:58:23.237878+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207091+0000","last_change":"2024-01-24T13:58:18.802046+0000","last_active":"2024-01-24T13:58:23.207091+0000","last_peered":"2024-01-24T13:58:23.207091+0000","last_clean":"2024-01-24T13:58:23.207091+0000","last_became_active":"2024-01-24T13:58:18.801757+0000","last_became_peered":"2024-01-24T13:58:18.801757+0000","last_unstale":"2024-01-24T13:58:23.207091+0000","last_undegraded":"2024-01-24T13:58:23.207091+0000","last_fullsized":"2024-01-24T13:58:23.207091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216324+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T13:58:23.216324+0000","last_peered":"2024-01-24T13:58:23.216324+0000","last_clean":"2024-01-24T13:58:23.216324+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T13:58:23.216324+0000","last_undegraded":"2024-01-24T13:58:23.216324+0000","last_fullsized":"2024-01-24T13:58:23.216324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227454+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T13:58:23.227454+0000","last_peered":"2024-01-24T13:58:23.227454+0000","last_clean":"2024-01-24T13:58:23.227454+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T13:58:23.227454+0000","last_undegraded":"2024-01-24T13:58:23.227454+0000","last_fullsized":"2024-01-24T13:58:23.227454+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212923+0000","last_change":"2024-01-24T13:58:19.211323+0000","last_active":"2024-01-24T13:58:23.212923+0000","last_peered":"2024-01-24T13:58:23.212923+0000","last_clean":"2024-01-24T13:58:23.212923+0000","last_became_active":"2024-01-24T13:58:19.211057+0000","last_became_peered":"2024-01-24T13:58:19.211057+0000","last_unstale":"2024-01-24T13:58:23.212923+0000","last_undegraded":"2024-01-24T13:58:23.212923+0000","last_fullsized":"2024-01-24T13:58:23.212923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229871+0000","last_change":"2024-01-24T13:58:23.227723+0000","last_active":"2024-01-24T13:58:23.229871+0000","last_peered":"2024-01-24T13:58:23.229871+0000","last_clean":"2024-01-24T13:58:23.229871+0000","last_became_active":"2024-01-24T13:58:23.227339+0000","last_became_peered":"2024-01-24T13:58:23.227339+0000","last_unstale":"2024-01-24T13:58:23.229871+0000","last_undegraded":"2024-01-24T13:58:23.229871+0000","last_fullsized":"2024-01-24T13:58:23.229871+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.210205+0000","last_change":"2024-01-24T13:58:18.798895+0000","last_active":"2024-01-24T13:58:23.210205+0000","last_peered":"2024-01-24T13:58:23.210205+0000","last_clean":"2024-01-24T13:58:23.210205+0000","last_became_active":"2024-01-24T13:58:18.796782+0000","last_became_peered":"2024-01-24T13:58:18.796782+0000","last_unstale":"2024-01-24T13:58:23.210205+0000","last_undegraded":"2024-01-24T13:58:23.210205+0000","last_fullsized":"2024-01-24T13:58:23.210205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220366+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T13:58:23.220366+0000","last_peered":"2024-01-24T13:58:23.220366+0000","last_clean":"2024-01-24T13:58:23.220366+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T13:58:23.220366+0000","last_undegraded":"2024-01-24T13:58:23.220366+0000","last_fullsized":"2024-01-24T13:58:23.220366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211623+0000","last_change":"2024-01-24T13:58:19.212508+0000","last_active":"2024-01-24T13:58:23.211623+0000","last_peered":"2024-01-24T13:58:23.211623+0000","last_clean":"2024-01-24T13:58:23.211623+0000","last_became_active":"2024-01-24T13:58:19.212211+0000","last_became_peered":"2024-01-24T13:58:19.212211+0000","last_unstale":"2024-01-24T13:58:23.211623+0000","last_undegraded":"2024-01-24T13:58:23.211623+0000","last_fullsized":"2024-01-24T13:58:23.211623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241355+0000","last_change":"2024-01-24T13:58:23.238939+0000","last_active":"2024-01-24T13:58:23.241355+0000","last_peered":"2024-01-24T13:58:23.241355+0000","last_clean":"2024-01-24T13:58:23.241355+0000","last_became_active":"2024-01-24T13:58:23.234940+0000","last_became_peered":"2024-01-24T13:58:23.234940+0000","last_unstale":"2024-01-24T13:58:23.241355+0000","last_undegraded":"2024-01-24T13:58:23.241355+0000","last_fullsized":"2024-01-24T13:58:23.241355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211492+0000","last_change":"2024-01-24T13:58:19.213076+0000","last_active":"2024-01-24T13:58:23.211492+0000","last_peered":"2024-01-24T13:58:23.211492+0000","last_clean":"2024-01-24T13:58:23.211492+0000","last_became_active":"2024-01-24T13:58:19.208454+0000","last_became_peered":"2024-01-24T13:58:19.208454+0000","last_unstale":"2024-01-24T13:58:23.211492+0000","last_undegraded":"2024-01-24T13:58:23.211492+0000","last_fullsized":"2024-01-24T13:58:23.211492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209311+0000","last_change":"2024-01-24T13:58:18.806850+0000","last_active":"2024-01-24T13:58:23.209311+0000","last_peered":"2024-01-24T13:58:23.209311+0000","last_clean":"2024-01-24T13:58:23.209311+0000","last_became_active":"2024-01-24T13:58:18.806456+0000","last_became_peered":"2024-01-24T13:58:18.806456+0000","last_unstale":"2024-01-24T13:58:23.209311+0000","last_undegraded":"2024-01-24T13:58:23.209311+0000","last_fullsized":"2024-01-24T13:58:23.209311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226775+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T13:58:23.226775+0000","last_peered":"2024-01-24T13:58:23.226775+0000","last_clean":"2024-01-24T13:58:23.226775+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T13:58:23.226775+0000","last_undegraded":"2024-01-24T13:58:23.226775+0000","last_fullsized":"2024-01-24T13:58:23.226775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238433+0000","last_change":"2024-01-24T13:58:23.237234+0000","last_active":"2024-01-24T13:58:23.238433+0000","last_peered":"2024-01-24T13:58:23.238433+0000","last_clean":"2024-01-24T13:58:23.238433+0000","last_became_active":"2024-01-24T13:58:23.236720+0000","last_became_peered":"2024-01-24T13:58:23.236720+0000","last_unstale":"2024-01-24T13:58:23.238433+0000","last_undegraded":"2024-01-24T13:58:23.238433+0000","last_fullsized":"2024-01-24T13:58:23.238433+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212103+0000","last_change":"2024-01-24T13:58:19.217116+0000","last_active":"2024-01-24T13:58:23.212103+0000","last_peered":"2024-01-24T13:58:23.212103+0000","last_clean":"2024-01-24T13:58:23.212103+0000","last_became_active":"2024-01-24T13:58:19.216731+0000","last_became_peered":"2024-01-24T13:58:19.216731+0000","last_unstale":"2024-01-24T13:58:23.212103+0000","last_undegraded":"2024-01-24T13:58:23.212103+0000","last_fullsized":"2024-01-24T13:58:23.212103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227689+0000","last_change":"2024-01-24T13:58:18.803400+0000","last_active":"2024-01-24T13:58:23.227689+0000","last_peered":"2024-01-24T13:58:23.227689+0000","last_clean":"2024-01-24T13:58:23.227689+0000","last_became_active":"2024-01-24T13:58:18.802706+0000","last_became_peered":"2024-01-24T13:58:18.802706+0000","last_unstale":"2024-01-24T13:58:23.227689+0000","last_undegraded":"2024-01-24T13:58:23.227689+0000","last_fullsized":"2024-01-24T13:58:23.227689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211632+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T13:58:23.211632+0000","last_peered":"2024-01-24T13:58:23.211632+0000","last_clean":"2024-01-24T13:58:23.211632+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T13:58:23.211632+0000","last_undegraded":"2024-01-24T13:58:23.211632+0000","last_fullsized":"2024-01-24T13:58:23.211632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237685+0000","last_change":"2024-01-24T13:58:23.235636+0000","last_active":"2024-01-24T13:58:23.237685+0000","last_peered":"2024-01-24T13:58:23.237685+0000","last_clean":"2024-01-24T13:58:23.237685+0000","last_became_active":"2024-01-24T13:58:23.234531+0000","last_became_peered":"2024-01-24T13:58:23.234531+0000","last_unstale":"2024-01-24T13:58:23.237685+0000","last_undegraded":"2024-01-24T13:58:23.237685+0000","last_fullsized":"2024-01-24T13:58:23.237685+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207375+0000","last_change":"2024-01-24T13:58:18.806700+0000","last_active":"2024-01-24T13:58:23.207375+0000","last_peered":"2024-01-24T13:58:23.207375+0000","last_clean":"2024-01-24T13:58:23.207375+0000","last_became_active":"2024-01-24T13:58:18.805917+0000","last_became_peered":"2024-01-24T13:58:18.805917+0000","last_unstale":"2024-01-24T13:58:23.207375+0000","last_undegraded":"2024-01-24T13:58:23.207375+0000","last_fullsized":"2024-01-24T13:58:23.207375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211671+0000","last_change":"2024-01-24T13:58:19.215828+0000","last_active":"2024-01-24T13:58:23.211671+0000","last_peered":"2024-01-24T13:58:23.211671+0000","last_clean":"2024-01-24T13:58:23.211671+0000","last_became_active":"2024-01-24T13:58:19.215015+0000","last_became_peered":"2024-01-24T13:58:19.215015+0000","last_unstale":"2024-01-24T13:58:23.211671+0000","last_undegraded":"2024-01-24T13:58:23.211671+0000","last_fullsized":"2024-01-24T13:58:23.211671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238346+0000","last_change":"2024-01-24T13:58:23.236631+0000","last_active":"2024-01-24T13:58:23.238346+0000","last_peered":"2024-01-24T13:58:23.238346+0000","last_clean":"2024-01-24T13:58:23.238346+0000","last_became_active":"2024-01-24T13:58:23.236200+0000","last_became_peered":"2024-01-24T13:58:23.236200+0000","last_unstale":"2024-01-24T13:58:23.238346+0000","last_undegraded":"2024-01-24T13:58:23.238346+0000","last_fullsized":"2024-01-24T13:58:23.238346+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209848+0000","last_change":"2024-01-24T13:58:18.803901+0000","last_active":"2024-01-24T13:58:23.209848+0000","last_peered":"2024-01-24T13:58:23.209848+0000","last_clean":"2024-01-24T13:58:23.209848+0000","last_became_active":"2024-01-24T13:58:18.803605+0000","last_became_peered":"2024-01-24T13:58:18.803605+0000","last_unstale":"2024-01-24T13:58:23.209848+0000","last_undegraded":"2024-01-24T13:58:23.209848+0000","last_fullsized":"2024-01-24T13:58:23.209848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209809+0000","last_change":"2024-01-24T13:58:19.216716+0000","last_active":"2024-01-24T13:58:23.209809+0000","last_peered":"2024-01-24T13:58:23.209809+0000","last_clean":"2024-01-24T13:58:23.209809+0000","last_became_active":"2024-01-24T13:58:19.216444+0000","last_became_peered":"2024-01-24T13:58:19.216444+0000","last_unstale":"2024-01-24T13:58:23.209809+0000","last_undegraded":"2024-01-24T13:58:23.209809+0000","last_fullsized":"2024-01-24T13:58:23.209809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.242821+0000","last_change":"2024-01-24T13:58:23.242205+0000","last_active":"2024-01-24T13:58:23.242821+0000","last_peered":"2024-01-24T13:58:23.242821+0000","last_clean":"2024-01-24T13:58:23.242821+0000","last_became_active":"2024-01-24T13:58:23.241764+0000","last_became_peered":"2024-01-24T13:58:23.241764+0000","last_unstale":"2024-01-24T13:58:23.242821+0000","last_undegraded":"2024-01-24T13:58:23.242821+0000","last_fullsized":"2024-01-24T13:58:23.242821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212581+0000","last_change":"2024-01-24T13:58:18.808313+0000","last_active":"2024-01-24T13:58:23.212581+0000","last_peered":"2024-01-24T13:58:23.212581+0000","last_clean":"2024-01-24T13:58:23.212581+0000","last_became_active":"2024-01-24T13:58:18.808096+0000","last_became_peered":"2024-01-24T13:58:18.808096+0000","last_unstale":"2024-01-24T13:58:23.212581+0000","last_undegraded":"2024-01-24T13:58:23.212581+0000","last_fullsized":"2024-01-24T13:58:23.212581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209286+0000","last_change":"2024-01-24T13:58:19.209979+0000","last_active":"2024-01-24T13:58:23.209286+0000","last_peered":"2024-01-24T13:58:23.209286+0000","last_clean":"2024-01-24T13:58:23.209286+0000","last_became_active":"2024-01-24T13:58:19.209672+0000","last_became_peered":"2024-01-24T13:58:19.209672+0000","last_unstale":"2024-01-24T13:58:23.209286+0000","last_undegraded":"2024-01-24T13:58:23.209286+0000","last_fullsized":"2024-01-24T13:58:23.209286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.240684+0000","last_change":"2024-01-24T13:58:23.229668+0000","last_active":"2024-01-24T13:58:23.240684+0000","last_peered":"2024-01-24T13:58:23.240684+0000","last_clean":"2024-01-24T13:58:23.240684+0000","last_became_active":"2024-01-24T13:58:23.229053+0000","last_became_peered":"2024-01-24T13:58:23.229053+0000","last_unstale":"2024-01-24T13:58:23.240684+0000","last_undegraded":"2024-01-24T13:58:23.240684+0000","last_fullsized":"2024-01-24T13:58:23.240684+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207919+0000","last_change":"2024-01-24T13:58:18.807736+0000","last_active":"2024-01-24T13:58:23.207919+0000","last_peered":"2024-01-24T13:58:23.207919+0000","last_clean":"2024-01-24T13:58:23.207919+0000","last_became_active":"2024-01-24T13:58:18.807228+0000","last_became_peered":"2024-01-24T13:58:18.807228+0000","last_unstale":"2024-01-24T13:58:23.207919+0000","last_undegraded":"2024-01-24T13:58:23.207919+0000","last_fullsized":"2024-01-24T13:58:23.207919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209909+0000","last_change":"2024-01-24T13:58:19.218038+0000","last_active":"2024-01-24T13:58:23.209909+0000","last_peered":"2024-01-24T13:58:23.209909+0000","last_clean":"2024-01-24T13:58:23.209909+0000","last_became_active":"2024-01-24T13:58:19.210668+0000","last_became_peered":"2024-01-24T13:58:19.210668+0000","last_unstale":"2024-01-24T13:58:23.209909+0000","last_undegraded":"2024-01-24T13:58:23.209909+0000","last_fullsized":"2024-01-24T13:58:23.209909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241201+0000","last_change":"2024-01-24T13:58:23.234754+0000","last_active":"2024-01-24T13:58:23.241201+0000","last_peered":"2024-01-24T13:58:23.241201+0000","last_clean":"2024-01-24T13:58:23.241201+0000","last_became_active":"2024-01-24T13:58:23.233898+0000","last_became_peered":"2024-01-24T13:58:23.233898+0000","last_unstale":"2024-01-24T13:58:23.241201+0000","last_undegraded":"2024-01-24T13:58:23.241201+0000","last_fullsized":"2024-01-24T13:58:23.241201+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227851+0000","last_change":"2024-01-24T13:58:18.802179+0000","last_active":"2024-01-24T13:58:23.227851+0000","last_peered":"2024-01-24T13:58:23.227851+0000","last_clean":"2024-01-24T13:58:23.227851+0000","last_became_active":"2024-01-24T13:58:18.801898+0000","last_became_peered":"2024-01-24T13:58:18.801898+0000","last_unstale":"2024-01-24T13:58:23.227851+0000","last_undegraded":"2024-01-24T13:58:23.227851+0000","last_fullsized":"2024-01-24T13:58:23.227851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.234948+0000","last_change":"2024-01-24T13:58:19.224275+0000","last_active":"2024-01-24T13:58:23.234948+0000","last_peered":"2024-01-24T13:58:23.234948+0000","last_clean":"2024-01-24T13:58:23.234948+0000","last_became_active":"2024-01-24T13:58:19.224006+0000","last_became_peered":"2024-01-24T13:58:19.224006+0000","last_unstale":"2024-01-24T13:58:23.234948+0000","last_undegraded":"2024-01-24T13:58:23.234948+0000","last_fullsized":"2024-01-24T13:58:23.234948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241374+0000","last_change":"2024-01-24T13:58:23.239977+0000","last_active":"2024-01-24T13:58:23.241374+0000","last_peered":"2024-01-24T13:58:23.241374+0000","last_clean":"2024-01-24T13:58:23.241374+0000","last_became_active":"2024-01-24T13:58:23.234358+0000","last_became_peered":"2024-01-24T13:58:23.234358+0000","last_unstale":"2024-01-24T13:58:23.241374+0000","last_undegraded":"2024-01-24T13:58:23.241374+0000","last_fullsized":"2024-01-24T13:58:23.241374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217009+0000","last_change":"2024-01-24T13:58:18.804317+0000","last_active":"2024-01-24T13:58:23.217009+0000","last_peered":"2024-01-24T13:58:23.217009+0000","last_clean":"2024-01-24T13:58:23.217009+0000","last_became_active":"2024-01-24T13:58:18.804046+0000","last_became_peered":"2024-01-24T13:58:18.804046+0000","last_unstale":"2024-01-24T13:58:23.217009+0000","last_undegraded":"2024-01-24T13:58:23.217009+0000","last_fullsized":"2024-01-24T13:58:23.217009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212503+0000","last_change":"2024-01-24T13:58:19.217782+0000","last_active":"2024-01-24T13:58:23.212503+0000","last_peered":"2024-01-24T13:58:23.212503+0000","last_clean":"2024-01-24T13:58:23.212503+0000","last_became_active":"2024-01-24T13:58:19.217509+0000","last_became_peered":"2024-01-24T13:58:19.217509+0000","last_unstale":"2024-01-24T13:58:23.212503+0000","last_undegraded":"2024-01-24T13:58:23.212503+0000","last_fullsized":"2024-01-24T13:58:23.212503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241249+0000","last_change":"2024-01-24T13:58:23.237049+0000","last_active":"2024-01-24T13:58:23.241249+0000","last_peered":"2024-01-24T13:58:23.241249+0000","last_clean":"2024-01-24T13:58:23.241249+0000","last_became_active":"2024-01-24T13:58:23.235154+0000","last_became_peered":"2024-01-24T13:58:23.235154+0000","last_unstale":"2024-01-24T13:58:23.241249+0000","last_undegraded":"2024-01-24T13:58:23.241249+0000","last_fullsized":"2024-01-24T13:58:23.241249+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217784+0000","last_change":"2024-01-24T13:58:18.801841+0000","last_active":"2024-01-24T13:58:23.217784+0000","last_peered":"2024-01-24T13:58:23.217784+0000","last_clean":"2024-01-24T13:58:23.217784+0000","last_became_active":"2024-01-24T13:58:18.801535+0000","last_became_peered":"2024-01-24T13:58:18.801535+0000","last_unstale":"2024-01-24T13:58:23.217784+0000","last_undegraded":"2024-01-24T13:58:23.217784+0000","last_fullsized":"2024-01-24T13:58:23.217784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209722+0000","last_change":"2024-01-24T13:58:19.213673+0000","last_active":"2024-01-24T13:58:23.209722+0000","last_peered":"2024-01-24T13:58:23.209722+0000","last_clean":"2024-01-24T13:58:23.209722+0000","last_became_active":"2024-01-24T13:58:19.213457+0000","last_became_peered":"2024-01-24T13:58:19.213457+0000","last_unstale":"2024-01-24T13:58:23.209722+0000","last_undegraded":"2024-01-24T13:58:23.209722+0000","last_fullsized":"2024-01-24T13:58:23.209722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238263+0000","last_change":"2024-01-24T13:58:23.234302+0000","last_active":"2024-01-24T13:58:23.238263+0000","last_peered":"2024-01-24T13:58:23.238263+0000","last_clean":"2024-01-24T13:58:23.238263+0000","last_became_active":"2024-01-24T13:58:23.233692+0000","last_became_peered":"2024-01-24T13:58:23.233692+0000","last_unstale":"2024-01-24T13:58:23.238263+0000","last_undegraded":"2024-01-24T13:58:23.238263+0000","last_fullsized":"2024-01-24T13:58:23.238263+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.218145+0000","last_change":"2024-01-24T13:58:18.798076+0000","last_active":"2024-01-24T13:58:23.218145+0000","last_peered":"2024-01-24T13:58:23.218145+0000","last_clean":"2024-01-24T13:58:23.218145+0000","last_became_active":"2024-01-24T13:58:18.797403+0000","last_became_peered":"2024-01-24T13:58:18.797403+0000","last_unstale":"2024-01-24T13:58:23.218145+0000","last_undegraded":"2024-01-24T13:58:23.218145+0000","last_fullsized":"2024-01-24T13:58:23.218145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224903+0000","last_change":"2024-01-24T13:58:19.210870+0000","last_active":"2024-01-24T13:58:23.224903+0000","last_peered":"2024-01-24T13:58:23.224903+0000","last_clean":"2024-01-24T13:58:23.224903+0000","last_became_active":"2024-01-24T13:58:19.210226+0000","last_became_peered":"2024-01-24T13:58:19.210226+0000","last_unstale":"2024-01-24T13:58:23.224903+0000","last_undegraded":"2024-01-24T13:58:23.224903+0000","last_fullsized":"2024-01-24T13:58:23.224903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232630+0000","last_change":"2024-01-24T13:58:23.230340+0000","last_active":"2024-01-24T13:58:23.232630+0000","last_peered":"2024-01-24T13:58:23.232630+0000","last_clean":"2024-01-24T13:58:23.232630+0000","last_became_active":"2024-01-24T13:58:23.229909+0000","last_became_peered":"2024-01-24T13:58:23.229909+0000","last_unstale":"2024-01-24T13:58:23.232630+0000","last_undegraded":"2024-01-24T13:58:23.232630+0000","last_fullsized":"2024-01-24T13:58:23.232630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"21'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212395+0000","last_change":"2024-01-24T13:58:18.813040+0000","last_active":"2024-01-24T13:58:23.212395+0000","last_peered":"2024-01-24T13:58:23.212395+0000","last_clean":"2024-01-24T13:58:23.212395+0000","last_became_active":"2024-01-24T13:58:18.812822+0000","last_became_peered":"2024-01-24T13:58:18.812822+0000","last_unstale":"2024-01-24T13:58:23.212395+0000","last_undegraded":"2024-01-24T13:58:23.212395+0000","last_fullsized":"2024-01-24T13:58:23.212395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208215+0000","last_change":"2024-01-24T13:58:19.216414+0000","last_active":"2024-01-24T13:58:23.208215+0000","last_peered":"2024-01-24T13:58:23.208215+0000","last_clean":"2024-01-24T13:58:23.208215+0000","last_became_active":"2024-01-24T13:58:19.216111+0000","last_became_peered":"2024-01-24T13:58:19.216111+0000","last_unstale":"2024-01-24T13:58:23.208215+0000","last_undegraded":"2024-01-24T13:58:23.208215+0000","last_fullsized":"2024-01-24T13:58:23.208215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237430+0000","last_change":"2024-01-24T13:58:23.234462+0000","last_active":"2024-01-24T13:58:23.237430+0000","last_peered":"2024-01-24T13:58:23.237430+0000","last_clean":"2024-01-24T13:58:23.237430+0000","last_became_active":"2024-01-24T13:58:23.233834+0000","last_became_peered":"2024-01-24T13:58:23.233834+0000","last_unstale":"2024-01-24T13:58:23.237430+0000","last_undegraded":"2024-01-24T13:58:23.237430+0000","last_fullsized":"2024-01-24T13:58:23.237430+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216341+0000","last_change":"2024-01-24T13:58:18.801184+0000","last_active":"2024-01-24T13:58:23.216341+0000","last_peered":"2024-01-24T13:58:23.216341+0000","last_clean":"2024-01-24T13:58:23.216341+0000","last_became_active":"2024-01-24T13:58:18.800877+0000","last_became_peered":"2024-01-24T13:58:18.800877+0000","last_unstale":"2024-01-24T13:58:23.216341+0000","last_undegraded":"2024-01-24T13:58:23.216341+0000","last_fullsized":"2024-01-24T13:58:23.216341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237737+0000","last_change":"2024-01-24T13:58:19.209281+0000","last_active":"2024-01-24T13:58:23.237737+0000","last_peered":"2024-01-24T13:58:23.237737+0000","last_clean":"2024-01-24T13:58:23.237737+0000","last_became_active":"2024-01-24T13:58:19.208890+0000","last_became_peered":"2024-01-24T13:58:19.208890+0000","last_unstale":"2024-01-24T13:58:23.237737+0000","last_undegraded":"2024-01-24T13:58:23.237737+0000","last_fullsized":"2024-01-24T13:58:23.237737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.230375+0000","last_change":"2024-01-24T13:58:23.228668+0000","last_active":"2024-01-24T13:58:23.230375+0000","last_peered":"2024-01-24T13:58:23.230375+0000","last_clean":"2024-01-24T13:58:23.230375+0000","last_became_active":"2024-01-24T13:58:23.228266+0000","last_became_peered":"2024-01-24T13:58:23.228266+0000","last_unstale":"2024-01-24T13:58:23.230375+0000","last_undegraded":"2024-01-24T13:58:23.230375+0000","last_fullsized":"2024-01-24T13:58:23.230375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232549+0000","last_change":"2024-01-24T13:58:18.809056+0000","last_active":"2024-01-24T13:58:23.232549+0000","last_peered":"2024-01-24T13:58:23.232549+0000","last_clean":"2024-01-24T13:58:23.232549+0000","last_became_active":"2024-01-24T13:58:18.808790+0000","last_became_peered":"2024-01-24T13:58:18.808790+0000","last_unstale":"2024-01-24T13:58:23.232549+0000","last_undegraded":"2024-01-24T13:58:23.232549+0000","last_fullsized":"2024-01-24T13:58:23.232549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227958+0000","last_change":"2024-01-24T13:58:19.217272+0000","last_active":"2024-01-24T13:58:23.227958+0000","last_peered":"2024-01-24T13:58:23.227958+0000","last_clean":"2024-01-24T13:58:23.227958+0000","last_became_active":"2024-01-24T13:58:19.216991+0000","last_became_peered":"2024-01-24T13:58:19.216991+0000","last_unstale":"2024-01-24T13:58:23.227958+0000","last_undegraded":"2024-01-24T13:58:23.227958+0000","last_fullsized":"2024-01-24T13:58:23.227958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.231091+0000","last_change":"2024-01-24T13:58:23.228760+0000","last_active":"2024-01-24T13:58:23.231091+0000","last_peered":"2024-01-24T13:58:23.231091+0000","last_clean":"2024-01-24T13:58:23.231091+0000","last_became_active":"2024-01-24T13:58:23.227345+0000","last_became_peered":"2024-01-24T13:58:23.227345+0000","last_unstale":"2024-01-24T13:58:23.231091+0000","last_undegraded":"2024-01-24T13:58:23.231091+0000","last_fullsized":"2024-01-24T13:58:23.231091+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217256+0000","last_change":"2024-01-24T13:58:18.798145+0000","last_active":"2024-01-24T13:58:23.217256+0000","last_peered":"2024-01-24T13:58:23.217256+0000","last_clean":"2024-01-24T13:58:23.217256+0000","last_became_active":"2024-01-24T13:58:18.797671+0000","last_became_peered":"2024-01-24T13:58:18.797671+0000","last_unstale":"2024-01-24T13:58:23.217256+0000","last_undegraded":"2024-01-24T13:58:23.217256+0000","last_fullsized":"2024-01-24T13:58:23.217256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207234+0000","last_change":"2024-01-24T13:58:19.221287+0000","last_active":"2024-01-24T13:58:23.207234+0000","last_peered":"2024-01-24T13:58:23.207234+0000","last_clean":"2024-01-24T13:58:23.207234+0000","last_became_active":"2024-01-24T13:58:19.220970+0000","last_became_peered":"2024-01-24T13:58:19.220970+0000","last_unstale":"2024-01-24T13:58:23.207234+0000","last_undegraded":"2024-01-24T13:58:23.207234+0000","last_fullsized":"2024-01-24T13:58:23.207234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237365+0000","last_change":"2024-01-24T13:58:23.234514+0000","last_active":"2024-01-24T13:58:23.237365+0000","last_peered":"2024-01-24T13:58:23.237365+0000","last_clean":"2024-01-24T13:58:23.237365+0000","last_became_active":"2024-01-24T13:58:23.233824+0000","last_became_peered":"2024-01-24T13:58:23.233824+0000","last_unstale":"2024-01-24T13:58:23.237365+0000","last_undegraded":"2024-01-24T13:58:23.237365+0000","last_fullsized":"2024-01-24T13:58:23.237365+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209685+0000","last_change":"2024-01-24T13:58:18.801316+0000","last_active":"2024-01-24T13:58:23.209685+0000","last_peered":"2024-01-24T13:58:23.209685+0000","last_clean":"2024-01-24T13:58:23.209685+0000","last_became_active":"2024-01-24T13:58:18.800089+0000","last_became_peered":"2024-01-24T13:58:18.800089+0000","last_unstale":"2024-01-24T13:58:23.209685+0000","last_undegraded":"2024-01-24T13:58:23.209685+0000","last_fullsized":"2024-01-24T13:58:23.209685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209210+0000","last_change":"2024-01-24T13:58:19.209604+0000","last_active":"2024-01-24T13:58:23.209210+0000","last_peered":"2024-01-24T13:58:23.209210+0000","last_clean":"2024-01-24T13:58:23.209210+0000","last_became_active":"2024-01-24T13:58:19.209306+0000","last_became_peered":"2024-01-24T13:58:19.209306+0000","last_unstale":"2024-01-24T13:58:23.209210+0000","last_undegraded":"2024-01-24T13:58:23.209210+0000","last_fullsized":"2024-01-24T13:58:23.209210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229399+0000","last_change":"2024-01-24T13:58:23.225921+0000","last_active":"2024-01-24T13:58:23.229399+0000","last_peered":"2024-01-24T13:58:23.229399+0000","last_clean":"2024-01-24T13:58:23.229399+0000","last_became_active":"2024-01-24T13:58:23.225413+0000","last_became_peered":"2024-01-24T13:58:23.225413+0000","last_unstale":"2024-01-24T13:58:23.229399+0000","last_undegraded":"2024-01-24T13:58:23.229399+0000","last_fullsized":"2024-01-24T13:58:23.229399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217204+0000","last_change":"2024-01-24T13:58:18.799227+0000","last_active":"2024-01-24T13:58:23.217204+0000","last_peered":"2024-01-24T13:58:23.217204+0000","last_clean":"2024-01-24T13:58:23.217204+0000","last_became_active":"2024-01-24T13:58:18.798927+0000","last_became_peered":"2024-01-24T13:58:18.798927+0000","last_unstale":"2024-01-24T13:58:23.217204+0000","last_undegraded":"2024-01-24T13:58:23.217204+0000","last_fullsized":"2024-01-24T13:58:23.217204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212286+0000","last_change":"2024-01-24T13:58:19.217668+0000","last_active":"2024-01-24T13:58:23.212286+0000","last_peered":"2024-01-24T13:58:23.212286+0000","last_clean":"2024-01-24T13:58:23.212286+0000","last_became_active":"2024-01-24T13:58:19.217323+0000","last_became_peered":"2024-01-24T13:58:19.217323+0000","last_unstale":"2024-01-24T13:58:23.212286+0000","last_undegraded":"2024-01-24T13:58:23.212286+0000","last_fullsized":"2024-01-24T13:58:23.212286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.230948+0000","last_change":"2024-01-24T13:58:23.229837+0000","last_active":"2024-01-24T13:58:23.230948+0000","last_peered":"2024-01-24T13:58:23.230948+0000","last_clean":"2024-01-24T13:58:23.230948+0000","last_became_active":"2024-01-24T13:58:23.229398+0000","last_became_peered":"2024-01-24T13:58:23.229398+0000","last_unstale":"2024-01-24T13:58:23.230948+0000","last_undegraded":"2024-01-24T13:58:23.230948+0000","last_fullsized":"2024-01-24T13:58:23.230948+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227679+0000","last_change":"2024-01-24T13:58:18.798763+0000","last_active":"2024-01-24T13:58:23.227679+0000","last_peered":"2024-01-24T13:58:23.227679+0000","last_clean":"2024-01-24T13:58:23.227679+0000","last_became_active":"2024-01-24T13:58:18.798451+0000","last_became_peered":"2024-01-24T13:58:18.798451+0000","last_unstale":"2024-01-24T13:58:23.227679+0000","last_undegraded":"2024-01-24T13:58:23.227679+0000","last_fullsized":"2024-01-24T13:58:23.227679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207643+0000","last_change":"2024-01-24T13:58:19.221011+0000","last_active":"2024-01-24T13:58:23.207643+0000","last_peered":"2024-01-24T13:58:23.207643+0000","last_clean":"2024-01-24T13:58:23.207643+0000","last_became_active":"2024-01-24T13:58:19.220727+0000","last_became_peered":"2024-01-24T13:58:19.220727+0000","last_unstale":"2024-01-24T13:58:23.207643+0000","last_undegraded":"2024-01-24T13:58:23.207643+0000","last_fullsized":"2024-01-24T13:58:23.207643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237556+0000","last_change":"2024-01-24T13:58:23.235211+0000","last_active":"2024-01-24T13:58:23.237556+0000","last_peered":"2024-01-24T13:58:23.237556+0000","last_clean":"2024-01-24T13:58:23.237556+0000","last_became_active":"2024-01-24T13:58:23.234791+0000","last_became_peered":"2024-01-24T13:58:23.234791+0000","last_unstale":"2024-01-24T13:58:23.237556+0000","last_undegraded":"2024-01-24T13:58:23.237556+0000","last_fullsized":"2024-01-24T13:58:23.237556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237921+0000","last_change":"2024-01-24T13:58:18.815292+0000","last_active":"2024-01-24T13:58:23.237921+0000","last_peered":"2024-01-24T13:58:23.237921+0000","last_clean":"2024-01-24T13:58:23.237921+0000","last_became_active":"2024-01-24T13:58:18.811889+0000","last_became_peered":"2024-01-24T13:58:18.811889+0000","last_unstale":"2024-01-24T13:58:23.237921+0000","last_undegraded":"2024-01-24T13:58:23.237921+0000","last_fullsized":"2024-01-24T13:58:23.237921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207065+0000","last_change":"2024-01-24T13:58:19.218174+0000","last_active":"2024-01-24T13:58:23.207065+0000","last_peered":"2024-01-24T13:58:23.207065+0000","last_clean":"2024-01-24T13:58:23.207065+0000","last_became_active":"2024-01-24T13:58:19.211059+0000","last_became_peered":"2024-01-24T13:58:19.211059+0000","last_unstale":"2024-01-24T13:58:23.207065+0000","last_undegraded":"2024-01-24T13:58:23.207065+0000","last_fullsized":"2024-01-24T13:58:23.207065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229596+0000","last_change":"2024-01-24T13:58:23.225806+0000","last_active":"2024-01-24T13:58:23.229596+0000","last_peered":"2024-01-24T13:58:23.229596+0000","last_clean":"2024-01-24T13:58:23.229596+0000","last_became_active":"2024-01-24T13:58:23.225425+0000","last_became_peered":"2024-01-24T13:58:23.225425+0000","last_unstale":"2024-01-24T13:58:23.229596+0000","last_undegraded":"2024-01-24T13:58:23.229596+0000","last_fullsized":"2024-01-24T13:58:23.229596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235654+0000","last_change":"2024-01-24T13:58:23.226907+0000","last_active":"2024-01-24T13:58:23.235654+0000","last_peered":"2024-01-24T13:58:23.235654+0000","last_clean":"2024-01-24T13:58:23.235654+0000","last_became_active":"2024-01-24T13:58:23.226413+0000","last_became_peered":"2024-01-24T13:58:23.226413+0000","last_unstale":"2024-01-24T13:58:23.235654+0000","last_undegraded":"2024-01-24T13:58:23.235654+0000","last_fullsized":"2024-01-24T13:58:23.235654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.206957+0000","last_change":"2024-01-24T13:58:18.806830+0000","last_active":"2024-01-24T13:58:23.206957+0000","last_peered":"2024-01-24T13:58:23.206957+0000","last_clean":"2024-01-24T13:58:23.206957+0000","last_became_active":"2024-01-24T13:58:18.806385+0000","last_became_peered":"2024-01-24T13:58:18.806385+0000","last_unstale":"2024-01-24T13:58:23.206957+0000","last_undegraded":"2024-01-24T13:58:23.206957+0000","last_fullsized":"2024-01-24T13:58:23.206957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212967+0000","last_change":"2024-01-24T13:58:19.210472+0000","last_active":"2024-01-24T13:58:23.212967+0000","last_peered":"2024-01-24T13:58:23.212967+0000","last_clean":"2024-01-24T13:58:23.212967+0000","last_became_active":"2024-01-24T13:58:19.210211+0000","last_became_peered":"2024-01-24T13:58:19.210211+0000","last_unstale":"2024-01-24T13:58:23.212967+0000","last_undegraded":"2024-01-24T13:58:23.212967+0000","last_fullsized":"2024-01-24T13:58:23.212967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229446+0000","last_change":"2024-01-24T13:58:23.223133+0000","last_active":"2024-01-24T13:58:23.229446+0000","last_peered":"2024-01-24T13:58:23.229446+0000","last_clean":"2024-01-24T13:58:23.229446+0000","last_became_active":"2024-01-24T13:58:23.222741+0000","last_became_peered":"2024-01-24T13:58:23.222741+0000","last_unstale":"2024-01-24T13:58:23.229446+0000","last_undegraded":"2024-01-24T13:58:23.229446+0000","last_fullsized":"2024-01-24T13:58:23.229446+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.210236+0000","last_change":"2024-01-24T13:58:18.800359+0000","last_active":"2024-01-24T13:58:23.210236+0000","last_peered":"2024-01-24T13:58:23.210236+0000","last_clean":"2024-01-24T13:58:23.210236+0000","last_became_active":"2024-01-24T13:58:18.799774+0000","last_became_peered":"2024-01-24T13:58:18.799774+0000","last_unstale":"2024-01-24T13:58:23.210236+0000","last_undegraded":"2024-01-24T13:58:23.210236+0000","last_fullsized":"2024-01-24T13:58:23.210236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237579+0000","last_change":"2024-01-24T13:58:19.210774+0000","last_active":"2024-01-24T13:58:23.237579+0000","last_peered":"2024-01-24T13:58:23.237579+0000","last_clean":"2024-01-24T13:58:23.237579+0000","last_became_active":"2024-01-24T13:58:19.210508+0000","last_became_peered":"2024-01-24T13:58:19.210508+0000","last_unstale":"2024-01-24T13:58:23.237579+0000","last_undegraded":"2024-01-24T13:58:23.237579+0000","last_fullsized":"2024-01-24T13:58:23.237579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209129+0000","last_change":"2024-01-24T13:58:18.803254+0000","last_active":"2024-01-24T13:58:23.209129+0000","last_peered":"2024-01-24T13:58:23.209129+0000","last_clean":"2024-01-24T13:58:23.209129+0000","last_became_active":"2024-01-24T13:58:18.802965+0000","last_became_peered":"2024-01-24T13:58:18.802965+0000","last_unstale":"2024-01-24T13:58:23.209129+0000","last_undegraded":"2024-01-24T13:58:23.209129+0000","last_fullsized":"2024-01-24T13:58:23.209129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208494+0000","last_change":"2024-01-24T13:58:19.221404+0000","last_active":"2024-01-24T13:58:23.208494+0000","last_peered":"2024-01-24T13:58:23.208494+0000","last_clean":"2024-01-24T13:58:23.208494+0000","last_became_active":"2024-01-24T13:58:19.221024+0000","last_became_peered":"2024-01-24T13:58:19.221024+0000","last_unstale":"2024-01-24T13:58:23.208494+0000","last_undegraded":"2024-01-24T13:58:23.208494+0000","last_fullsized":"2024-01-24T13:58:23.208494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232448+0000","last_change":"2024-01-24T13:58:23.229756+0000","last_active":"2024-01-24T13:58:23.232448+0000","last_peered":"2024-01-24T13:58:23.232448+0000","last_clean":"2024-01-24T13:58:23.232448+0000","last_became_active":"2024-01-24T13:58:23.229328+0000","last_became_peered":"2024-01-24T13:58:23.229328+0000","last_unstale":"2024-01-24T13:58:23.232448+0000","last_undegraded":"2024-01-24T13:58:23.232448+0000","last_fullsized":"2024-01-24T13:58:23.232448+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297364,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074476,"statfs":{"total":96636764160,"available":96332263424,"internally_reserved":0,"allocated":274432,"data_stored":115668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297396,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074444,"statfs":{"total":96636764160,"available":96332230656,"internally_reserved":0,"allocated":307200,"data_stored":118043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297372,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074468,"statfs":{"total":96636764160,"available":96332255232,"internally_reserved":0,"allocated":282624,"data_stored":116699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297368,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074472,"statfs":{"total":96636764160,"available":96332259328,"internally_reserved":0,"allocated":278528,"data_stored":116629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297372,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074468,"statfs":{"total":96636764160,"available":96332255232,"internally_reserved":0,"allocated":282624,"data_stored":116719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297388,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074452,"statfs":{"total":96636764160,"available":96332238848,"internally_reserved":0,"allocated":299008,"data_stored":120400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297368,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074472,"statfs":{"total":96636764160,"available":96332259328,"internally_reserved":0,"allocated":278528,"data_stored":116629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297376,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074464,"statfs":{"total":96636764160,"available":96332251136,"internally_reserved":0,"allocated":286720,"data_stored":120254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":3625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T13:58:39.657 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T13:58:39.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.824+0000 7fe563491700 1 Processor -- start 2024-01-24T13:58:39.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.825+0000 7fe563491700 1 -- start start 2024-01-24T13:58:39.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe563491700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe55c0bd340 0x7fe55c0c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe563491700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55c05e850 con 0x7fe55c153fe0 2024-01-24T13:58:39.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe563491700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55c0631f0 con 0x7fe55c1543c0 2024-01-24T13:58:39.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe563491700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c0a4f90 con 0x7fe55c0bd340 2024-01-24T13:58:39.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe56122d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe55c153fe0 0x7fe55c0a4880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42196/0 (socket says 172.21.15.29:42196) 2024-01-24T13:58:39.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe56122d700 1 -- 172.21.15.29:0/1468355776 learned_addr learned my addr 172.21.15.29:0/1468355776 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:39.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe561a2e700 1 --2- 172.21.15.29:0/1468355776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe55c0bd340 0x7fe55c0c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe54ffff700 1 -- 172.21.15.29:0/1468355776 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311617039 0 0) 0x7fe55c05e850 con 0x7fe55c153fe0 2024-01-24T13:58:39.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe54ffff700 1 -- 172.21.15.29:0/1468355776 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5480033f0 con 0x7fe55c153fe0 2024-01-24T13:58:39.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe54ffff700 1 -- 172.21.15.29:0/1468355776 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800991072 0 0) 0x7fe55c0631f0 con 0x7fe55c1543c0 2024-01-24T13:58:39.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe54ffff700 1 -- 172.21.15.29:0/1468355776 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe55c05e850 con 0x7fe55c1543c0 2024-01-24T13:58:39.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe561a2e700 1 -- 172.21.15.29:0/1468355776 >> v1:172.21.15.163:6789/0 conn(0x7fe55c1543c0 legacy=0x7fe55c0bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe561a2e700 1 -- 172.21.15.29:0/1468355776 >> v1:172.21.15.29:6789/0 conn(0x7fe55c153fe0 legacy=0x7fe55c0a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe54ffff700 1 -- 172.21.15.29:0/1468355776 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1788439294 0 0) 0x7fe5480033f0 con 0x7fe55c153fe0 2024-01-24T13:58:39.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe561a2e700 1 -- 172.21.15.29:0/1468355776 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c0bd7e0 con 0x7fe55c0bd340 2024-01-24T13:58:39.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.826+0000 7fe561a2e700 1 --2- 172.21.15.29:0/1468355776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe55c0bd340 0x7fe55c0c1870 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fe554009e90 tx=0x7fe554009780).ready entity=mon.2 client_cookie=6bda3e0b5ad788b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:39.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.827+0000 7fe54ffff700 1 -- 172.21.15.29:0/1468355776 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55400e070 con 0x7fe55c0bd340 2024-01-24T13:58:39.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.827+0000 7fe54ffff700 1 -- 172.21.15.29:0/1468355776 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55400b7a0 con 0x7fe55c0bd340 2024-01-24T13:58:39.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.827+0000 7fe54ffff700 1 -- 172.21.15.29:0/1468355776 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550001470 con 0x7fe55c0bd340 2024-01-24T13:58:39.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.827+0000 7fe563491700 1 -- 172.21.15.29:0/1468355776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe55c0bd340 msgr2=0x7fe55c0c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.827+0000 7fe563491700 1 --2- 172.21.15.29:0/1468355776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe55c0bd340 0x7fe55c0c1870 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fe554009e90 tx=0x7fe554009780).stop 2024-01-24T13:58:39.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.827+0000 7fe563491700 1 -- 172.21.15.29:0/1468355776 shutdown_connections 2024-01-24T13:58:39.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.827+0000 7fe563491700 1 --2- 172.21.15.29:0/1468355776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe55c0bd340 0x7fe55c0c1870 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.827+0000 7fe563491700 1 -- 172.21.15.29:0/1468355776 >> 172.21.15.29:0/1468355776 conn(0x7fe55c064aa0 msgr2=0x7fe55c064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.827+0000 7fe563491700 1 -- 172.21.15.29:0/1468355776 shutdown_connections 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.827+0000 7fe563491700 1 -- 172.21.15.29:0/1468355776 wait complete. 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.829+0000 7fe563491700 1 Processor -- start 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.829+0000 7fe563491700 1 -- start start 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe563491700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe55c0bd340 0x7fe55c156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe563491700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe55c153fe0 0x7fe55c145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe563491700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe55c1543c0 0x7fe55c146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe563491700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c146550 con 0x7fe55c0bd340 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe563491700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c154d00 con 0x7fe55c153fe0 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe563491700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c154a00 con 0x7fe55c1543c0 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe56122d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe55c0bd340 0x7fe55c156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe56122d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe55c0bd340 0x7fe55c156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41422/0 (socket says 172.21.15.29:41422) 2024-01-24T13:58:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe56122d700 1 -- 172.21.15.29:0/4273598974 learned_addr learned my addr 172.21.15.29:0/4273598974 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe561a2e700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe55c1543c0 0x7fe55c146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe560a2c700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe55c153fe0 0x7fe55c145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe56122d700 1 -- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe55c1543c0 msgr2=0x7fe55c146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe56122d700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe55c1543c0 0x7fe55c146060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe56122d700 1 -- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe55c153fe0 msgr2=0x7fe55c145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe56122d700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe55c153fe0 0x7fe55c145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe56122d700 1 -- 172.21.15.29:0/4273598974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe554009a20 con 0x7fe55c0bd340 2024-01-24T13:58:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe56122d700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe55c0bd340 0x7fe55c156f90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fe55000c7f0 tx=0x7fe550001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe54d7fa700 1 -- 172.21.15.29:0/4273598974 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550007170 con 0x7fe55c0bd340 2024-01-24T13:58:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe54d7fa700 1 -- 172.21.15.29:0/4273598974 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5500072f0 con 0x7fe55c0bd340 2024-01-24T13:58:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe563491700 1 -- 172.21.15.29:0/4273598974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c0a77d0 con 0x7fe55c0bd340 2024-01-24T13:58:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe54d7fa700 1 -- 172.21.15.29:0/4273598974 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5500048c0 con 0x7fe55c0bd340 2024-01-24T13:58:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.830+0000 7fe563491700 1 -- 172.21.15.29:0/4273598974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c0ab5c0 con 0x7fe55c0bd340 2024-01-24T13:58:39.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.831+0000 7fe563491700 1 -- 172.21.15.29:0/4273598974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55c154840 con 0x7fe55c0bd340 2024-01-24T13:58:39.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.832+0000 7fe54d7fa700 1 -- 172.21.15.29:0/4273598974 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe550004cf0 con 0x7fe55c0bd340 2024-01-24T13:58:39.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.832+0000 7fe54d7fa700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe54005f150 0x7fe540061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:39.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.832+0000 7fe54d7fa700 1 -- 172.21.15.29:0/4273598974 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe55007a1b0 con 0x7fe55c0bd340 2024-01-24T13:58:39.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.832+0000 7fe560a2c700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe54005f150 0x7fe540061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:39.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.833+0000 7fe560a2c700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe54005f150 0x7fe540061610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fe55c0a5cf0 tx=0x7fe558009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:39.837+0000 7fe54d7fa700 1 -- 172.21.15.29:0/4273598974 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe55004b170 con 0x7fe55c0bd340 2024-01-24T13:58:40.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.014+0000 7fe563491700 1 -- 172.21.15.29:0/4273598974 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe55c05e330 con 0x7fe54005f150 2024-01-24T13:58:40.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.026+0000 7fe54d7fa700 1 -- 172.21.15.29:0/4273598974 <== mgr.4103 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+410634 (secure 0 0 0) 0x7fe55c05e330 con 0x7fe54005f150 2024-01-24T13:58:40.031 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:40.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.030+0000 7fe563491700 1 -- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe54005f150 msgr2=0x7fe540061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.030+0000 7fe563491700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe54005f150 0x7fe540061610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fe55c0a5cf0 tx=0x7fe558009040).stop 2024-01-24T13:58:40.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.030+0000 7fe563491700 1 -- 172.21.15.29:0/4273598974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe55c0bd340 msgr2=0x7fe55c156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.030+0000 7fe563491700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe55c0bd340 0x7fe55c156f90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fe55000c7f0 tx=0x7fe550001b50).stop 2024-01-24T13:58:40.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.030+0000 7fe563491700 1 -- 172.21.15.29:0/4273598974 shutdown_connections 2024-01-24T13:58:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.030+0000 7fe563491700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe55c1543c0 0x7fe55c146060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.030+0000 7fe563491700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe54005f150 0x7fe540061610 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.030+0000 7fe563491700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe55c0bd340 0x7fe55c156f90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.030+0000 7fe563491700 1 --2- 172.21.15.29:0/4273598974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe55c153fe0 0x7fe55c145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.030+0000 7fe563491700 1 -- 172.21.15.29:0/4273598974 >> 172.21.15.29:0/4273598974 conn(0x7fe55c064aa0 msgr2=0x7fe55c0a4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.030+0000 7fe563491700 1 -- 172.21.15.29:0/4273598974 shutdown_connections 2024-01-24T13:58:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.030+0000 7fe563491700 1 -- 172.21.15.29:0/4273598974 wait complete. 2024-01-24T13:58:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T13:58:40.044 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-01-24T13:58:39.728937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":402,"acting":402,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":402,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379004,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752595716,"statfs":{"total":773094113280,"available":770658013184,"internally_reserved":0,"allocated":2289664,"data_stored":941041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2179,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004182"},"pg_stats":[{"pgid":"3.3f","version":"21'3","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.218432+0000","last_change":"2024-01-24T13:58:18.804112+0000","last_active":"2024-01-24T13:58:23.218432+0000","last_peered":"2024-01-24T13:58:23.218432+0000","last_clean":"2024-01-24T13:58:23.218432+0000","last_became_active":"2024-01-24T13:58:18.803739+0000","last_became_peered":"2024-01-24T13:58:18.803739+0000","last_unstale":"2024-01-24T13:58:23.218432+0000","last_undegraded":"2024-01-24T13:58:23.218432+0000","last_fullsized":"2024-01-24T13:58:23.218432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224809+0000","last_change":"2024-01-24T13:58:19.214526+0000","last_active":"2024-01-24T13:58:23.224809+0000","last_peered":"2024-01-24T13:58:23.224809+0000","last_clean":"2024-01-24T13:58:23.224809+0000","last_became_active":"2024-01-24T13:58:19.214261+0000","last_became_peered":"2024-01-24T13:58:19.214261+0000","last_unstale":"2024-01-24T13:58:23.224809+0000","last_undegraded":"2024-01-24T13:58:23.224809+0000","last_fullsized":"2024-01-24T13:58:23.224809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211257+0000","last_change":"2024-01-24T13:58:18.813731+0000","last_active":"2024-01-24T13:58:23.211257+0000","last_peered":"2024-01-24T13:58:23.211257+0000","last_clean":"2024-01-24T13:58:23.211257+0000","last_became_active":"2024-01-24T13:58:18.813399+0000","last_became_peered":"2024-01-24T13:58:18.813399+0000","last_unstale":"2024-01-24T13:58:23.211257+0000","last_undegraded":"2024-01-24T13:58:23.211257+0000","last_fullsized":"2024-01-24T13:58:23.211257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216628+0000","last_change":"2024-01-24T13:58:19.214934+0000","last_active":"2024-01-24T13:58:23.216628+0000","last_peered":"2024-01-24T13:58:23.216628+0000","last_clean":"2024-01-24T13:58:23.216628+0000","last_became_active":"2024-01-24T13:58:19.214473+0000","last_became_peered":"2024-01-24T13:58:19.214473+0000","last_unstale":"2024-01-24T13:58:23.216628+0000","last_undegraded":"2024-01-24T13:58:23.216628+0000","last_fullsized":"2024-01-24T13:58:23.216628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235905+0000","last_change":"2024-01-24T13:58:18.810864+0000","last_active":"2024-01-24T13:58:23.235905+0000","last_peered":"2024-01-24T13:58:23.235905+0000","last_clean":"2024-01-24T13:58:23.235905+0000","last_became_active":"2024-01-24T13:58:18.810538+0000","last_became_peered":"2024-01-24T13:58:18.810538+0000","last_unstale":"2024-01-24T13:58:23.235905+0000","last_undegraded":"2024-01-24T13:58:23.235905+0000","last_fullsized":"2024-01-24T13:58:23.235905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209261+0000","last_change":"2024-01-24T13:58:19.215436+0000","last_active":"2024-01-24T13:58:23.209261+0000","last_peered":"2024-01-24T13:58:23.209261+0000","last_clean":"2024-01-24T13:58:23.209261+0000","last_became_active":"2024-01-24T13:58:19.215145+0000","last_became_peered":"2024-01-24T13:58:19.215145+0000","last_unstale":"2024-01-24T13:58:23.209261+0000","last_undegraded":"2024-01-24T13:58:23.209261+0000","last_fullsized":"2024-01-24T13:58:23.209261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227753+0000","last_change":"2024-01-24T13:58:18.803776+0000","last_active":"2024-01-24T13:58:23.227753+0000","last_peered":"2024-01-24T13:58:23.227753+0000","last_clean":"2024-01-24T13:58:23.227753+0000","last_became_active":"2024-01-24T13:58:18.803485+0000","last_became_peered":"2024-01-24T13:58:18.803485+0000","last_unstale":"2024-01-24T13:58:23.227753+0000","last_undegraded":"2024-01-24T13:58:23.227753+0000","last_fullsized":"2024-01-24T13:58:23.227753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.234859+0000","last_change":"2024-01-24T13:58:19.224975+0000","last_active":"2024-01-24T13:58:23.234859+0000","last_peered":"2024-01-24T13:58:23.234859+0000","last_clean":"2024-01-24T13:58:23.234859+0000","last_became_active":"2024-01-24T13:58:19.224568+0000","last_became_peered":"2024-01-24T13:58:19.224568+0000","last_unstale":"2024-01-24T13:58:23.234859+0000","last_undegraded":"2024-01-24T13:58:23.234859+0000","last_fullsized":"2024-01-24T13:58:23.234859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217357+0000","last_change":"2024-01-24T13:58:18.797963+0000","last_active":"2024-01-24T13:58:23.217357+0000","last_peered":"2024-01-24T13:58:23.217357+0000","last_clean":"2024-01-24T13:58:23.217357+0000","last_became_active":"2024-01-24T13:58:18.797320+0000","last_became_peered":"2024-01-24T13:58:18.797320+0000","last_unstale":"2024-01-24T13:58:23.217357+0000","last_undegraded":"2024-01-24T13:58:23.217357+0000","last_fullsized":"2024-01-24T13:58:23.217357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213193+0000","last_change":"2024-01-24T13:58:19.213147+0000","last_active":"2024-01-24T13:58:23.213193+0000","last_peered":"2024-01-24T13:58:23.213193+0000","last_clean":"2024-01-24T13:58:23.213193+0000","last_became_active":"2024-01-24T13:58:19.212854+0000","last_became_peered":"2024-01-24T13:58:19.212854+0000","last_unstale":"2024-01-24T13:58:23.213193+0000","last_undegraded":"2024-01-24T13:58:23.213193+0000","last_fullsized":"2024-01-24T13:58:23.213193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.228193+0000","last_change":"2024-01-24T13:58:18.812721+0000","last_active":"2024-01-24T13:58:23.228193+0000","last_peered":"2024-01-24T13:58:23.228193+0000","last_clean":"2024-01-24T13:58:23.228193+0000","last_became_active":"2024-01-24T13:58:18.812439+0000","last_became_peered":"2024-01-24T13:58:18.812439+0000","last_unstale":"2024-01-24T13:58:23.228193+0000","last_undegraded":"2024-01-24T13:58:23.228193+0000","last_fullsized":"2024-01-24T13:58:23.228193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209659+0000","last_change":"2024-01-24T13:58:19.213672+0000","last_active":"2024-01-24T13:58:23.209659+0000","last_peered":"2024-01-24T13:58:23.209659+0000","last_clean":"2024-01-24T13:58:23.209659+0000","last_became_active":"2024-01-24T13:58:19.213378+0000","last_became_peered":"2024-01-24T13:58:19.213378+0000","last_unstale":"2024-01-24T13:58:23.209659+0000","last_undegraded":"2024-01-24T13:58:23.209659+0000","last_fullsized":"2024-01-24T13:58:23.209659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216204+0000","last_change":"2024-01-24T13:58:18.806273+0000","last_active":"2024-01-24T13:58:23.216204+0000","last_peered":"2024-01-24T13:58:23.216204+0000","last_clean":"2024-01-24T13:58:23.216204+0000","last_became_active":"2024-01-24T13:58:18.805970+0000","last_became_peered":"2024-01-24T13:58:18.805970+0000","last_unstale":"2024-01-24T13:58:23.216204+0000","last_undegraded":"2024-01-24T13:58:23.216204+0000","last_fullsized":"2024-01-24T13:58:23.216204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227834+0000","last_change":"2024-01-24T13:58:19.208876+0000","last_active":"2024-01-24T13:58:23.227834+0000","last_peered":"2024-01-24T13:58:23.227834+0000","last_clean":"2024-01-24T13:58:23.227834+0000","last_became_active":"2024-01-24T13:58:19.208537+0000","last_became_peered":"2024-01-24T13:58:19.208537+0000","last_unstale":"2024-01-24T13:58:23.227834+0000","last_undegraded":"2024-01-24T13:58:23.227834+0000","last_fullsized":"2024-01-24T13:58:23.227834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208114+0000","last_change":"2024-01-24T13:58:18.801957+0000","last_active":"2024-01-24T13:58:23.208114+0000","last_peered":"2024-01-24T13:58:23.208114+0000","last_clean":"2024-01-24T13:58:23.208114+0000","last_became_active":"2024-01-24T13:58:18.801641+0000","last_became_peered":"2024-01-24T13:58:18.801641+0000","last_unstale":"2024-01-24T13:58:23.208114+0000","last_undegraded":"2024-01-24T13:58:23.208114+0000","last_fullsized":"2024-01-24T13:58:23.208114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.222487+0000","last_change":"2024-01-24T13:58:19.217744+0000","last_active":"2024-01-24T13:58:23.222487+0000","last_peered":"2024-01-24T13:58:23.222487+0000","last_clean":"2024-01-24T13:58:23.222487+0000","last_became_active":"2024-01-24T13:58:19.217212+0000","last_became_peered":"2024-01-24T13:58:19.217212+0000","last_unstale":"2024-01-24T13:58:23.222487+0000","last_undegraded":"2024-01-24T13:58:23.222487+0000","last_fullsized":"2024-01-24T13:58:23.222487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.218321+0000","last_change":"2024-01-24T13:58:18.797573+0000","last_active":"2024-01-24T13:58:23.218321+0000","last_peered":"2024-01-24T13:58:23.218321+0000","last_clean":"2024-01-24T13:58:23.218321+0000","last_became_active":"2024-01-24T13:58:18.797100+0000","last_became_peered":"2024-01-24T13:58:18.797100+0000","last_unstale":"2024-01-24T13:58:23.218321+0000","last_undegraded":"2024-01-24T13:58:23.218321+0000","last_fullsized":"2024-01-24T13:58:23.218321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209573+0000","last_change":"2024-01-24T13:58:19.211019+0000","last_active":"2024-01-24T13:58:23.209573+0000","last_peered":"2024-01-24T13:58:23.209573+0000","last_clean":"2024-01-24T13:58:23.209573+0000","last_became_active":"2024-01-24T13:58:19.210424+0000","last_became_peered":"2024-01-24T13:58:19.210424+0000","last_unstale":"2024-01-24T13:58:23.209573+0000","last_undegraded":"2024-01-24T13:58:23.209573+0000","last_fullsized":"2024-01-24T13:58:23.209573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213015+0000","last_change":"2024-01-24T13:58:18.800935+0000","last_active":"2024-01-24T13:58:23.213015+0000","last_peered":"2024-01-24T13:58:23.213015+0000","last_clean":"2024-01-24T13:58:23.213015+0000","last_became_active":"2024-01-24T13:58:18.800629+0000","last_became_peered":"2024-01-24T13:58:18.800629+0000","last_unstale":"2024-01-24T13:58:23.213015+0000","last_undegraded":"2024-01-24T13:58:23.213015+0000","last_fullsized":"2024-01-24T13:58:23.213015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216389+0000","last_change":"2024-01-24T13:58:19.214756+0000","last_active":"2024-01-24T13:58:23.216389+0000","last_peered":"2024-01-24T13:58:23.216389+0000","last_clean":"2024-01-24T13:58:23.216389+0000","last_became_active":"2024-01-24T13:58:19.214127+0000","last_became_peered":"2024-01-24T13:58:19.214127+0000","last_unstale":"2024-01-24T13:58:23.216389+0000","last_undegraded":"2024-01-24T13:58:23.216389+0000","last_fullsized":"2024-01-24T13:58:23.216389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213067+0000","last_change":"2024-01-24T13:58:18.804647+0000","last_active":"2024-01-24T13:58:23.213067+0000","last_peered":"2024-01-24T13:58:23.213067+0000","last_clean":"2024-01-24T13:58:23.213067+0000","last_became_active":"2024-01-24T13:58:18.803996+0000","last_became_peered":"2024-01-24T13:58:18.803996+0000","last_unstale":"2024-01-24T13:58:23.213067+0000","last_undegraded":"2024-01-24T13:58:23.213067+0000","last_fullsized":"2024-01-24T13:58:23.213067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226664+0000","last_change":"2024-01-24T13:58:19.210603+0000","last_active":"2024-01-24T13:58:23.226664+0000","last_peered":"2024-01-24T13:58:23.226664+0000","last_clean":"2024-01-24T13:58:23.226664+0000","last_became_active":"2024-01-24T13:58:19.210245+0000","last_became_peered":"2024-01-24T13:58:19.210245+0000","last_unstale":"2024-01-24T13:58:23.226664+0000","last_undegraded":"2024-01-24T13:58:23.226664+0000","last_fullsized":"2024-01-24T13:58:23.226664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.34","version":"26'7","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-24T13:58:28.635902+0000","last_change":"2024-01-24T13:58:18.809410+0000","last_active":"2024-01-24T13:58:28.635902+0000","last_peered":"2024-01-24T13:58:28.635902+0000","last_clean":"2024-01-24T13:58:28.635902+0000","last_became_active":"2024-01-24T13:58:18.809115+0000","last_became_peered":"2024-01-24T13:58:18.809115+0000","last_unstale":"2024-01-24T13:58:28.635902+0000","last_undegraded":"2024-01-24T13:58:28.635902+0000","last_fullsized":"2024-01-24T13:58:28.635902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3071,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220937+0000","last_change":"2024-01-24T13:58:19.217910+0000","last_active":"2024-01-24T13:58:23.220937+0000","last_peered":"2024-01-24T13:58:23.220937+0000","last_clean":"2024-01-24T13:58:23.220937+0000","last_became_active":"2024-01-24T13:58:19.216908+0000","last_became_peered":"2024-01-24T13:58:19.216908+0000","last_unstale":"2024-01-24T13:58:23.220937+0000","last_undegraded":"2024-01-24T13:58:23.220937+0000","last_fullsized":"2024-01-24T13:58:23.220937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208119+0000","last_change":"2024-01-24T13:58:18.807547+0000","last_active":"2024-01-24T13:58:23.208119+0000","last_peered":"2024-01-24T13:58:23.208119+0000","last_clean":"2024-01-24T13:58:23.208119+0000","last_became_active":"2024-01-24T13:58:18.806845+0000","last_became_peered":"2024-01-24T13:58:18.806845+0000","last_unstale":"2024-01-24T13:58:23.208119+0000","last_undegraded":"2024-01-24T13:58:23.208119+0000","last_fullsized":"2024-01-24T13:58:23.208119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224004+0000","last_change":"2024-01-24T13:58:19.219483+0000","last_active":"2024-01-24T13:58:23.224004+0000","last_peered":"2024-01-24T13:58:23.224004+0000","last_clean":"2024-01-24T13:58:23.224004+0000","last_became_active":"2024-01-24T13:58:19.219182+0000","last_became_peered":"2024-01-24T13:58:19.219182+0000","last_unstale":"2024-01-24T13:58:23.224004+0000","last_undegraded":"2024-01-24T13:58:23.224004+0000","last_fullsized":"2024-01-24T13:58:23.224004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232615+0000","last_change":"2024-01-24T13:58:18.804144+0000","last_active":"2024-01-24T13:58:23.232615+0000","last_peered":"2024-01-24T13:58:23.232615+0000","last_clean":"2024-01-24T13:58:23.232615+0000","last_became_active":"2024-01-24T13:58:18.803873+0000","last_became_peered":"2024-01-24T13:58:18.803873+0000","last_unstale":"2024-01-24T13:58:23.232615+0000","last_undegraded":"2024-01-24T13:58:23.232615+0000","last_fullsized":"2024-01-24T13:58:23.232615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209346+0000","last_change":"2024-01-24T13:58:19.221162+0000","last_active":"2024-01-24T13:58:23.209346+0000","last_peered":"2024-01-24T13:58:23.209346+0000","last_clean":"2024-01-24T13:58:23.209346+0000","last_became_active":"2024-01-24T13:58:19.220665+0000","last_became_peered":"2024-01-24T13:58:19.220665+0000","last_unstale":"2024-01-24T13:58:23.209346+0000","last_undegraded":"2024-01-24T13:58:23.209346+0000","last_fullsized":"2024-01-24T13:58:23.209346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.210124+0000","last_change":"2024-01-24T13:58:18.799003+0000","last_active":"2024-01-24T13:58:23.210124+0000","last_peered":"2024-01-24T13:58:23.210124+0000","last_clean":"2024-01-24T13:58:23.210124+0000","last_became_active":"2024-01-24T13:58:18.796638+0000","last_became_peered":"2024-01-24T13:58:18.796638+0000","last_unstale":"2024-01-24T13:58:23.210124+0000","last_undegraded":"2024-01-24T13:58:23.210124+0000","last_fullsized":"2024-01-24T13:58:23.210124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227756+0000","last_change":"2024-01-24T13:58:19.213039+0000","last_active":"2024-01-24T13:58:23.227756+0000","last_peered":"2024-01-24T13:58:23.227756+0000","last_clean":"2024-01-24T13:58:23.227756+0000","last_became_active":"2024-01-24T13:58:19.212755+0000","last_became_peered":"2024-01-24T13:58:19.212755+0000","last_unstale":"2024-01-24T13:58:23.227756+0000","last_undegraded":"2024-01-24T13:58:23.227756+0000","last_fullsized":"2024-01-24T13:58:23.227756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"21'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.225127+0000","last_change":"2024-01-24T13:58:18.805645+0000","last_active":"2024-01-24T13:58:23.225127+0000","last_peered":"2024-01-24T13:58:23.225127+0000","last_clean":"2024-01-24T13:58:23.225127+0000","last_became_active":"2024-01-24T13:58:18.805375+0000","last_became_peered":"2024-01-24T13:58:18.805375+0000","last_unstale":"2024-01-24T13:58:23.225127+0000","last_undegraded":"2024-01-24T13:58:23.225127+0000","last_fullsized":"2024-01-24T13:58:23.225127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237611+0000","last_change":"2024-01-24T13:58:19.225874+0000","last_active":"2024-01-24T13:58:23.237611+0000","last_peered":"2024-01-24T13:58:23.237611+0000","last_clean":"2024-01-24T13:58:23.237611+0000","last_became_active":"2024-01-24T13:58:19.225220+0000","last_became_peered":"2024-01-24T13:58:19.225220+0000","last_unstale":"2024-01-24T13:58:23.237611+0000","last_undegraded":"2024-01-24T13:58:23.237611+0000","last_fullsized":"2024-01-24T13:58:23.237611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.233153+0000","last_change":"2024-01-24T13:58:18.797127+0000","last_active":"2024-01-24T13:58:23.233153+0000","last_peered":"2024-01-24T13:58:23.233153+0000","last_clean":"2024-01-24T13:58:23.233153+0000","last_became_active":"2024-01-24T13:58:18.796816+0000","last_became_peered":"2024-01-24T13:58:18.796816+0000","last_unstale":"2024-01-24T13:58:23.233153+0000","last_undegraded":"2024-01-24T13:58:23.233153+0000","last_fullsized":"2024-01-24T13:58:23.233153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216434+0000","last_change":"2024-01-24T13:58:19.219593+0000","last_active":"2024-01-24T13:58:23.216434+0000","last_peered":"2024-01-24T13:58:23.216434+0000","last_clean":"2024-01-24T13:58:23.216434+0000","last_became_active":"2024-01-24T13:58:19.219267+0000","last_became_peered":"2024-01-24T13:58:19.219267+0000","last_unstale":"2024-01-24T13:58:23.216434+0000","last_undegraded":"2024-01-24T13:58:23.216434+0000","last_fullsized":"2024-01-24T13:58:23.216434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220392+0000","last_change":"2024-01-24T13:58:18.805694+0000","last_active":"2024-01-24T13:58:23.220392+0000","last_peered":"2024-01-24T13:58:23.220392+0000","last_clean":"2024-01-24T13:58:23.220392+0000","last_became_active":"2024-01-24T13:58:18.805335+0000","last_became_peered":"2024-01-24T13:58:18.805335+0000","last_unstale":"2024-01-24T13:58:23.220392+0000","last_undegraded":"2024-01-24T13:58:23.220392+0000","last_fullsized":"2024-01-24T13:58:23.220392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207983+0000","last_change":"2024-01-24T13:58:19.213662+0000","last_active":"2024-01-24T13:58:23.207983+0000","last_peered":"2024-01-24T13:58:23.207983+0000","last_clean":"2024-01-24T13:58:23.207983+0000","last_became_active":"2024-01-24T13:58:19.213340+0000","last_became_peered":"2024-01-24T13:58:19.213340+0000","last_unstale":"2024-01-24T13:58:23.207983+0000","last_undegraded":"2024-01-24T13:58:23.207983+0000","last_fullsized":"2024-01-24T13:58:23.207983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2d","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217312+0000","last_change":"2024-01-24T13:58:18.798262+0000","last_active":"2024-01-24T13:58:23.217312+0000","last_peered":"2024-01-24T13:58:23.217312+0000","last_clean":"2024-01-24T13:58:23.217312+0000","last_became_active":"2024-01-24T13:58:18.797856+0000","last_became_peered":"2024-01-24T13:58:18.797856+0000","last_unstale":"2024-01-24T13:58:23.217312+0000","last_undegraded":"2024-01-24T13:58:23.217312+0000","last_fullsized":"2024-01-24T13:58:23.217312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232445+0000","last_change":"2024-01-24T13:58:19.223736+0000","last_active":"2024-01-24T13:58:23.232445+0000","last_peered":"2024-01-24T13:58:23.232445+0000","last_clean":"2024-01-24T13:58:23.232445+0000","last_became_active":"2024-01-24T13:58:19.223500+0000","last_became_peered":"2024-01-24T13:58:19.223500+0000","last_unstale":"2024-01-24T13:58:23.232445+0000","last_undegraded":"2024-01-24T13:58:23.232445+0000","last_fullsized":"2024-01-24T13:58:23.232445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213312+0000","last_change":"2024-01-24T13:58:18.802360+0000","last_active":"2024-01-24T13:58:23.213312+0000","last_peered":"2024-01-24T13:58:23.213312+0000","last_clean":"2024-01-24T13:58:23.213312+0000","last_became_active":"2024-01-24T13:58:18.802051+0000","last_became_peered":"2024-01-24T13:58:18.802051+0000","last_unstale":"2024-01-24T13:58:23.213312+0000","last_undegraded":"2024-01-24T13:58:23.213312+0000","last_fullsized":"2024-01-24T13:58:23.213312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217156+0000","last_change":"2024-01-24T13:58:19.218929+0000","last_active":"2024-01-24T13:58:23.217156+0000","last_peered":"2024-01-24T13:58:23.217156+0000","last_clean":"2024-01-24T13:58:23.217156+0000","last_became_active":"2024-01-24T13:58:19.218350+0000","last_became_peered":"2024-01-24T13:58:19.218350+0000","last_unstale":"2024-01-24T13:58:23.217156+0000","last_undegraded":"2024-01-24T13:58:23.217156+0000","last_fullsized":"2024-01-24T13:58:23.217156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2b","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235138+0000","last_change":"2024-01-24T13:58:18.810840+0000","last_active":"2024-01-24T13:58:23.235138+0000","last_peered":"2024-01-24T13:58:23.235138+0000","last_clean":"2024-01-24T13:58:23.235138+0000","last_became_active":"2024-01-24T13:58:18.809023+0000","last_became_peered":"2024-01-24T13:58:18.809023+0000","last_unstale":"2024-01-24T13:58:23.235138+0000","last_undegraded":"2024-01-24T13:58:23.235138+0000","last_fullsized":"2024-01-24T13:58:23.235138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209489+0000","last_change":"2024-01-24T13:58:19.217855+0000","last_active":"2024-01-24T13:58:23.209489+0000","last_peered":"2024-01-24T13:58:23.209489+0000","last_clean":"2024-01-24T13:58:23.209489+0000","last_became_active":"2024-01-24T13:58:19.217559+0000","last_became_peered":"2024-01-24T13:58:19.217559+0000","last_unstale":"2024-01-24T13:58:23.209489+0000","last_undegraded":"2024-01-24T13:58:23.209489+0000","last_fullsized":"2024-01-24T13:58:23.209489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226871+0000","last_change":"2024-01-24T13:58:18.798611+0000","last_active":"2024-01-24T13:58:23.226871+0000","last_peered":"2024-01-24T13:58:23.226871+0000","last_clean":"2024-01-24T13:58:23.226871+0000","last_became_active":"2024-01-24T13:58:18.798325+0000","last_became_peered":"2024-01-24T13:58:18.798325+0000","last_unstale":"2024-01-24T13:58:23.226871+0000","last_undegraded":"2024-01-24T13:58:23.226871+0000","last_fullsized":"2024-01-24T13:58:23.226871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.222261+0000","last_change":"2024-01-24T13:58:19.206846+0000","last_active":"2024-01-24T13:58:23.222261+0000","last_peered":"2024-01-24T13:58:23.222261+0000","last_clean":"2024-01-24T13:58:23.222261+0000","last_became_active":"2024-01-24T13:58:19.206552+0000","last_became_peered":"2024-01-24T13:58:19.206552+0000","last_unstale":"2024-01-24T13:58:23.222261+0000","last_undegraded":"2024-01-24T13:58:23.222261+0000","last_fullsized":"2024-01-24T13:58:23.222261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207449+0000","last_change":"2024-01-24T13:58:18.801628+0000","last_active":"2024-01-24T13:58:23.207449+0000","last_peered":"2024-01-24T13:58:23.207449+0000","last_clean":"2024-01-24T13:58:23.207449+0000","last_became_active":"2024-01-24T13:58:18.801291+0000","last_became_peered":"2024-01-24T13:58:18.801291+0000","last_unstale":"2024-01-24T13:58:23.207449+0000","last_undegraded":"2024-01-24T13:58:23.207449+0000","last_fullsized":"2024-01-24T13:58:23.207449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209637+0000","last_change":"2024-01-24T13:58:19.211347+0000","last_active":"2024-01-24T13:58:23.209637+0000","last_peered":"2024-01-24T13:58:23.209637+0000","last_clean":"2024-01-24T13:58:23.209637+0000","last_became_active":"2024-01-24T13:58:19.210298+0000","last_became_peered":"2024-01-24T13:58:19.210298+0000","last_unstale":"2024-01-24T13:58:23.209637+0000","last_undegraded":"2024-01-24T13:58:23.209637+0000","last_fullsized":"2024-01-24T13:58:23.209637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224993+0000","last_change":"2024-01-24T13:58:18.803399+0000","last_active":"2024-01-24T13:58:23.224993+0000","last_peered":"2024-01-24T13:58:23.224993+0000","last_clean":"2024-01-24T13:58:23.224993+0000","last_became_active":"2024-01-24T13:58:18.803109+0000","last_became_peered":"2024-01-24T13:58:18.803109+0000","last_unstale":"2024-01-24T13:58:23.224993+0000","last_undegraded":"2024-01-24T13:58:23.224993+0000","last_fullsized":"2024-01-24T13:58:23.224993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238205+0000","last_change":"2024-01-24T13:58:19.225710+0000","last_active":"2024-01-24T13:58:23.238205+0000","last_peered":"2024-01-24T13:58:23.238205+0000","last_clean":"2024-01-24T13:58:23.238205+0000","last_became_active":"2024-01-24T13:58:19.224844+0000","last_became_peered":"2024-01-24T13:58:19.224844+0000","last_unstale":"2024-01-24T13:58:23.238205+0000","last_undegraded":"2024-01-24T13:58:23.238205+0000","last_fullsized":"2024-01-24T13:58:23.238205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208117+0000","last_change":"2024-01-24T13:58:18.801623+0000","last_active":"2024-01-24T13:58:23.208117+0000","last_peered":"2024-01-24T13:58:23.208117+0000","last_clean":"2024-01-24T13:58:23.208117+0000","last_became_active":"2024-01-24T13:58:18.801272+0000","last_became_peered":"2024-01-24T13:58:18.801272+0000","last_unstale":"2024-01-24T13:58:23.208117+0000","last_undegraded":"2024-01-24T13:58:23.208117+0000","last_fullsized":"2024-01-24T13:58:23.208117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212308+0000","last_change":"2024-01-24T13:58:19.207457+0000","last_active":"2024-01-24T13:58:23.212308+0000","last_peered":"2024-01-24T13:58:23.212308+0000","last_clean":"2024-01-24T13:58:23.212308+0000","last_became_active":"2024-01-24T13:58:19.207133+0000","last_became_peered":"2024-01-24T13:58:19.207133+0000","last_unstale":"2024-01-24T13:58:23.212308+0000","last_undegraded":"2024-01-24T13:58:23.212308+0000","last_fullsized":"2024-01-24T13:58:23.212308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"26'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.880144+0000","last_change":"2024-01-24T13:58:18.800410+0000","last_active":"2024-01-24T13:58:23.880144+0000","last_peered":"2024-01-24T13:58:23.880144+0000","last_clean":"2024-01-24T13:58:23.880144+0000","last_became_active":"2024-01-24T13:58:18.800073+0000","last_became_peered":"2024-01-24T13:58:18.800073+0000","last_unstale":"2024-01-24T13:58:23.880144+0000","last_undegraded":"2024-01-24T13:58:23.880144+0000","last_fullsized":"2024-01-24T13:58:23.880144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207934+0000","last_change":"2024-01-24T13:58:19.221135+0000","last_active":"2024-01-24T13:58:23.207934+0000","last_peered":"2024-01-24T13:58:23.207934+0000","last_clean":"2024-01-24T13:58:23.207934+0000","last_became_active":"2024-01-24T13:58:19.220863+0000","last_became_peered":"2024-01-24T13:58:19.220863+0000","last_unstale":"2024-01-24T13:58:23.207934+0000","last_undegraded":"2024-01-24T13:58:23.207934+0000","last_fullsized":"2024-01-24T13:58:23.207934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211332+0000","last_change":"2024-01-24T13:58:18.800232+0000","last_active":"2024-01-24T13:58:23.211332+0000","last_peered":"2024-01-24T13:58:23.211332+0000","last_clean":"2024-01-24T13:58:23.211332+0000","last_became_active":"2024-01-24T13:58:18.799722+0000","last_became_peered":"2024-01-24T13:58:18.799722+0000","last_unstale":"2024-01-24T13:58:23.211332+0000","last_undegraded":"2024-01-24T13:58:23.211332+0000","last_fullsized":"2024-01-24T13:58:23.211332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209141+0000","last_change":"2024-01-24T13:58:19.215326+0000","last_active":"2024-01-24T13:58:23.209141+0000","last_peered":"2024-01-24T13:58:23.209141+0000","last_clean":"2024-01-24T13:58:23.209141+0000","last_became_active":"2024-01-24T13:58:19.214852+0000","last_became_peered":"2024-01-24T13:58:19.214852+0000","last_unstale":"2024-01-24T13:58:23.209141+0000","last_undegraded":"2024-01-24T13:58:23.209141+0000","last_fullsized":"2024-01-24T13:58:23.209141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207199+0000","last_change":"2024-01-24T13:58:18.804258+0000","last_active":"2024-01-24T13:58:23.207199+0000","last_peered":"2024-01-24T13:58:23.207199+0000","last_clean":"2024-01-24T13:58:23.207199+0000","last_became_active":"2024-01-24T13:58:18.802440+0000","last_became_peered":"2024-01-24T13:58:18.802440+0000","last_unstale":"2024-01-24T13:58:23.207199+0000","last_undegraded":"2024-01-24T13:58:23.207199+0000","last_fullsized":"2024-01-24T13:58:23.207199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208000+0000","last_change":"2024-01-24T13:58:19.214888+0000","last_active":"2024-01-24T13:58:23.208000+0000","last_peered":"2024-01-24T13:58:23.208000+0000","last_clean":"2024-01-24T13:58:23.208000+0000","last_became_active":"2024-01-24T13:58:19.214607+0000","last_became_peered":"2024-01-24T13:58:19.214607+0000","last_unstale":"2024-01-24T13:58:23.208000+0000","last_undegraded":"2024-01-24T13:58:23.208000+0000","last_fullsized":"2024-01-24T13:58:23.208000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235037+0000","last_change":"2024-01-24T13:58:18.813467+0000","last_active":"2024-01-24T13:58:23.235037+0000","last_peered":"2024-01-24T13:58:23.235037+0000","last_clean":"2024-01-24T13:58:23.235037+0000","last_became_active":"2024-01-24T13:58:18.813164+0000","last_became_peered":"2024-01-24T13:58:18.813164+0000","last_unstale":"2024-01-24T13:58:23.235037+0000","last_undegraded":"2024-01-24T13:58:23.235037+0000","last_fullsized":"2024-01-24T13:58:23.235037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211738+0000","last_change":"2024-01-24T13:58:19.212196+0000","last_active":"2024-01-24T13:58:23.211738+0000","last_peered":"2024-01-24T13:58:23.211738+0000","last_clean":"2024-01-24T13:58:23.211738+0000","last_became_active":"2024-01-24T13:58:19.211820+0000","last_became_peered":"2024-01-24T13:58:19.211820+0000","last_unstale":"2024-01-24T13:58:23.211738+0000","last_undegraded":"2024-01-24T13:58:23.211738+0000","last_fullsized":"2024-01-24T13:58:23.211738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226611+0000","last_change":"2024-01-24T13:58:18.802179+0000","last_active":"2024-01-24T13:58:23.226611+0000","last_peered":"2024-01-24T13:58:23.226611+0000","last_clean":"2024-01-24T13:58:23.226611+0000","last_became_active":"2024-01-24T13:58:18.801891+0000","last_became_peered":"2024-01-24T13:58:18.801891+0000","last_unstale":"2024-01-24T13:58:23.226611+0000","last_undegraded":"2024-01-24T13:58:23.226611+0000","last_fullsized":"2024-01-24T13:58:23.226611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227791+0000","last_change":"2024-01-24T13:58:19.217207+0000","last_active":"2024-01-24T13:58:23.227791+0000","last_peered":"2024-01-24T13:58:23.227791+0000","last_clean":"2024-01-24T13:58:23.227791+0000","last_became_active":"2024-01-24T13:58:19.216920+0000","last_became_peered":"2024-01-24T13:58:19.216920+0000","last_unstale":"2024-01-24T13:58:23.227791+0000","last_undegraded":"2024-01-24T13:58:23.227791+0000","last_fullsized":"2024-01-24T13:58:23.227791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220454+0000","last_change":"2024-01-24T13:58:18.793887+0000","last_active":"2024-01-24T13:58:23.220454+0000","last_peered":"2024-01-24T13:58:23.220454+0000","last_clean":"2024-01-24T13:58:23.220454+0000","last_became_active":"2024-01-24T13:58:18.793542+0000","last_became_peered":"2024-01-24T13:58:18.793542+0000","last_unstale":"2024-01-24T13:58:23.220454+0000","last_undegraded":"2024-01-24T13:58:23.220454+0000","last_fullsized":"2024-01-24T13:58:23.220454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211617+0000","last_change":"2024-01-24T13:58:19.216131+0000","last_active":"2024-01-24T13:58:23.211617+0000","last_peered":"2024-01-24T13:58:23.211617+0000","last_clean":"2024-01-24T13:58:23.211617+0000","last_became_active":"2024-01-24T13:58:19.215566+0000","last_became_peered":"2024-01-24T13:58:19.215566+0000","last_unstale":"2024-01-24T13:58:23.211617+0000","last_undegraded":"2024-01-24T13:58:23.211617+0000","last_fullsized":"2024-01-24T13:58:23.211617+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212581+0000","last_change":"2024-01-24T13:58:18.808952+0000","last_active":"2024-01-24T13:58:23.212581+0000","last_peered":"2024-01-24T13:58:23.212581+0000","last_clean":"2024-01-24T13:58:23.212581+0000","last_became_active":"2024-01-24T13:58:18.808672+0000","last_became_peered":"2024-01-24T13:58:18.808672+0000","last_unstale":"2024-01-24T13:58:23.212581+0000","last_undegraded":"2024-01-24T13:58:23.212581+0000","last_fullsized":"2024-01-24T13:58:23.212581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211211+0000","last_change":"2024-01-24T13:58:19.213798+0000","last_active":"2024-01-24T13:58:23.211211+0000","last_peered":"2024-01-24T13:58:23.211211+0000","last_clean":"2024-01-24T13:58:23.211211+0000","last_became_active":"2024-01-24T13:58:19.212458+0000","last_became_peered":"2024-01-24T13:58:19.212458+0000","last_unstale":"2024-01-24T13:58:23.211211+0000","last_undegraded":"2024-01-24T13:58:23.211211+0000","last_fullsized":"2024-01-24T13:58:23.211211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238030+0000","last_change":"2024-01-24T13:58:18.811471+0000","last_active":"2024-01-24T13:58:23.238030+0000","last_peered":"2024-01-24T13:58:23.238030+0000","last_clean":"2024-01-24T13:58:23.238030+0000","last_became_active":"2024-01-24T13:58:18.811159+0000","last_became_peered":"2024-01-24T13:58:18.811159+0000","last_unstale":"2024-01-24T13:58:23.238030+0000","last_undegraded":"2024-01-24T13:58:23.238030+0000","last_fullsized":"2024-01-24T13:58:23.238030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209454+0000","last_change":"2024-01-24T13:58:19.219125+0000","last_active":"2024-01-24T13:58:23.209454+0000","last_peered":"2024-01-24T13:58:23.209454+0000","last_clean":"2024-01-24T13:58:23.209454+0000","last_became_active":"2024-01-24T13:58:19.218823+0000","last_became_peered":"2024-01-24T13:58:19.218823+0000","last_unstale":"2024-01-24T13:58:23.209454+0000","last_undegraded":"2024-01-24T13:58:23.209454+0000","last_fullsized":"2024-01-24T13:58:23.209454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241886+0000","last_change":"2024-01-24T13:58:23.236176+0000","last_active":"2024-01-24T13:58:23.241886+0000","last_peered":"2024-01-24T13:58:23.241886+0000","last_clean":"2024-01-24T13:58:23.241886+0000","last_became_active":"2024-01-24T13:58:23.235658+0000","last_became_peered":"2024-01-24T13:58:23.235658+0000","last_unstale":"2024-01-24T13:58:23.241886+0000","last_undegraded":"2024-01-24T13:58:23.241886+0000","last_fullsized":"2024-01-24T13:58:23.241886+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207471+0000","last_change":"2024-01-24T13:58:18.806459+0000","last_active":"2024-01-24T13:58:23.207471+0000","last_peered":"2024-01-24T13:58:23.207471+0000","last_clean":"2024-01-24T13:58:23.207471+0000","last_became_active":"2024-01-24T13:58:18.806150+0000","last_became_peered":"2024-01-24T13:58:18.806150+0000","last_unstale":"2024-01-24T13:58:23.207471+0000","last_undegraded":"2024-01-24T13:58:23.207471+0000","last_fullsized":"2024-01-24T13:58:23.207471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207748+0000","last_change":"2024-01-24T13:58:19.211910+0000","last_active":"2024-01-24T13:58:23.207748+0000","last_peered":"2024-01-24T13:58:23.207748+0000","last_clean":"2024-01-24T13:58:23.207748+0000","last_became_active":"2024-01-24T13:58:19.210919+0000","last_became_peered":"2024-01-24T13:58:19.210919+0000","last_unstale":"2024-01-24T13:58:23.207748+0000","last_undegraded":"2024-01-24T13:58:23.207748+0000","last_fullsized":"2024-01-24T13:58:23.207748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232409+0000","last_change":"2024-01-24T13:58:23.223847+0000","last_active":"2024-01-24T13:58:23.232409+0000","last_peered":"2024-01-24T13:58:23.232409+0000","last_clean":"2024-01-24T13:58:23.232409+0000","last_became_active":"2024-01-24T13:58:23.222064+0000","last_became_peered":"2024-01-24T13:58:23.222064+0000","last_unstale":"2024-01-24T13:58:23.232409+0000","last_undegraded":"2024-01-24T13:58:23.232409+0000","last_fullsized":"2024-01-24T13:58:23.232409+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227575+0000","last_change":"2024-01-24T13:58:18.798060+0000","last_active":"2024-01-24T13:58:23.227575+0000","last_peered":"2024-01-24T13:58:23.227575+0000","last_clean":"2024-01-24T13:58:23.227575+0000","last_became_active":"2024-01-24T13:58:18.797774+0000","last_became_peered":"2024-01-24T13:58:18.797774+0000","last_unstale":"2024-01-24T13:58:23.227575+0000","last_undegraded":"2024-01-24T13:58:23.227575+0000","last_fullsized":"2024-01-24T13:58:23.227575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208950+0000","last_change":"2024-01-24T13:58:19.215145+0000","last_active":"2024-01-24T13:58:23.208950+0000","last_peered":"2024-01-24T13:58:23.208950+0000","last_clean":"2024-01-24T13:58:23.208950+0000","last_became_active":"2024-01-24T13:58:19.214840+0000","last_became_peered":"2024-01-24T13:58:19.214840+0000","last_unstale":"2024-01-24T13:58:23.208950+0000","last_undegraded":"2024-01-24T13:58:23.208950+0000","last_fullsized":"2024-01-24T13:58:23.208950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237348+0000","last_change":"2024-01-24T13:58:23.234486+0000","last_active":"2024-01-24T13:58:23.237348+0000","last_peered":"2024-01-24T13:58:23.237348+0000","last_clean":"2024-01-24T13:58:23.237348+0000","last_became_active":"2024-01-24T13:58:23.233948+0000","last_became_peered":"2024-01-24T13:58:23.233948+0000","last_unstale":"2024-01-24T13:58:23.237348+0000","last_undegraded":"2024-01-24T13:58:23.237348+0000","last_fullsized":"2024-01-24T13:58:23.237348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227976+0000","last_change":"2024-01-24T13:58:18.797763+0000","last_active":"2024-01-24T13:58:23.227976+0000","last_peered":"2024-01-24T13:58:23.227976+0000","last_clean":"2024-01-24T13:58:23.227976+0000","last_became_active":"2024-01-24T13:58:18.797478+0000","last_became_peered":"2024-01-24T13:58:18.797478+0000","last_unstale":"2024-01-24T13:58:23.227976+0000","last_undegraded":"2024-01-24T13:58:23.227976+0000","last_fullsized":"2024-01-24T13:58:23.227976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209496+0000","last_change":"2024-01-24T13:58:19.217185+0000","last_active":"2024-01-24T13:58:23.209496+0000","last_peered":"2024-01-24T13:58:23.209496+0000","last_clean":"2024-01-24T13:58:23.209496+0000","last_became_active":"2024-01-24T13:58:19.216628+0000","last_became_peered":"2024-01-24T13:58:19.216628+0000","last_unstale":"2024-01-24T13:58:23.209496+0000","last_undegraded":"2024-01-24T13:58:23.209496+0000","last_fullsized":"2024-01-24T13:58:23.209496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.236747+0000","last_change":"2024-01-24T13:58:23.230969+0000","last_active":"2024-01-24T13:58:23.236747+0000","last_peered":"2024-01-24T13:58:23.236747+0000","last_clean":"2024-01-24T13:58:23.236747+0000","last_became_active":"2024-01-24T13:58:23.230450+0000","last_became_peered":"2024-01-24T13:58:23.230450+0000","last_unstale":"2024-01-24T13:58:23.236747+0000","last_undegraded":"2024-01-24T13:58:23.236747+0000","last_fullsized":"2024-01-24T13:58:23.236747+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241684+0000","last_change":"2024-01-24T13:58:23.231930+0000","last_active":"2024-01-24T13:58:23.241684+0000","last_peered":"2024-01-24T13:58:23.241684+0000","last_clean":"2024-01-24T13:58:23.241684+0000","last_became_active":"2024-01-24T13:58:23.231494+0000","last_became_peered":"2024-01-24T13:58:23.231494+0000","last_unstale":"2024-01-24T13:58:23.241684+0000","last_undegraded":"2024-01-24T13:58:23.241684+0000","last_fullsized":"2024-01-24T13:58:23.241684+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235293+0000","last_change":"2024-01-24T13:58:18.811358+0000","last_active":"2024-01-24T13:58:23.235293+0000","last_peered":"2024-01-24T13:58:23.235293+0000","last_clean":"2024-01-24T13:58:23.235293+0000","last_became_active":"2024-01-24T13:58:18.811087+0000","last_became_peered":"2024-01-24T13:58:18.811087+0000","last_unstale":"2024-01-24T13:58:23.235293+0000","last_undegraded":"2024-01-24T13:58:23.235293+0000","last_fullsized":"2024-01-24T13:58:23.235293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224192+0000","last_change":"2024-01-24T13:58:19.213488+0000","last_active":"2024-01-24T13:58:23.224192+0000","last_peered":"2024-01-24T13:58:23.224192+0000","last_clean":"2024-01-24T13:58:23.224192+0000","last_became_active":"2024-01-24T13:58:19.213211+0000","last_became_peered":"2024-01-24T13:58:19.213211+0000","last_unstale":"2024-01-24T13:58:23.224192+0000","last_undegraded":"2024-01-24T13:58:23.224192+0000","last_fullsized":"2024-01-24T13:58:23.224192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209928+0000","last_change":"2024-01-24T13:58:19.216228+0000","last_active":"2024-01-24T13:58:23.209928+0000","last_peered":"2024-01-24T13:58:23.209928+0000","last_clean":"2024-01-24T13:58:23.209928+0000","last_became_active":"2024-01-24T13:58:19.215952+0000","last_became_peered":"2024-01-24T13:58:19.215952+0000","last_unstale":"2024-01-24T13:58:23.209928+0000","last_undegraded":"2024-01-24T13:58:23.209928+0000","last_fullsized":"2024-01-24T13:58:23.209928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237483+0000","last_change":"2024-01-24T13:58:18.813343+0000","last_active":"2024-01-24T13:58:23.237483+0000","last_peered":"2024-01-24T13:58:23.237483+0000","last_clean":"2024-01-24T13:58:23.237483+0000","last_became_active":"2024-01-24T13:58:18.812987+0000","last_became_peered":"2024-01-24T13:58:18.812987+0000","last_unstale":"2024-01-24T13:58:23.237483+0000","last_undegraded":"2024-01-24T13:58:23.237483+0000","last_fullsized":"2024-01-24T13:58:23.237483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.213473+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T13:58:23.213473+0000","last_peered":"2024-01-24T13:58:23.213473+0000","last_clean":"2024-01-24T13:58:23.213473+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T13:58:23.213473+0000","last_undegraded":"2024-01-24T13:58:23.213473+0000","last_fullsized":"2024-01-24T13:58:23.213473+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241577+0000","last_change":"2024-01-24T13:58:23.235347+0000","last_active":"2024-01-24T13:58:23.241577+0000","last_peered":"2024-01-24T13:58:23.241577+0000","last_clean":"2024-01-24T13:58:23.241577+0000","last_became_active":"2024-01-24T13:58:23.234902+0000","last_became_peered":"2024-01-24T13:58:23.234902+0000","last_unstale":"2024-01-24T13:58:23.241577+0000","last_undegraded":"2024-01-24T13:58:23.241577+0000","last_fullsized":"2024-01-24T13:58:23.241577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207482+0000","last_change":"2024-01-24T13:58:19.218418+0000","last_active":"2024-01-24T13:58:23.207482+0000","last_peered":"2024-01-24T13:58:23.207482+0000","last_clean":"2024-01-24T13:58:23.207482+0000","last_became_active":"2024-01-24T13:58:19.210988+0000","last_became_peered":"2024-01-24T13:58:19.210988+0000","last_unstale":"2024-01-24T13:58:23.207482+0000","last_undegraded":"2024-01-24T13:58:23.207482+0000","last_fullsized":"2024-01-24T13:58:23.207482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211503+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T13:58:23.211503+0000","last_peered":"2024-01-24T13:58:23.211503+0000","last_clean":"2024-01-24T13:58:23.211503+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T13:58:23.211503+0000","last_undegraded":"2024-01-24T13:58:23.211503+0000","last_fullsized":"2024-01-24T13:58:23.211503+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207918+0000","last_change":"2024-01-24T13:58:18.798154+0000","last_active":"2024-01-24T13:58:23.207918+0000","last_peered":"2024-01-24T13:58:23.207918+0000","last_clean":"2024-01-24T13:58:23.207918+0000","last_became_active":"2024-01-24T13:58:18.797819+0000","last_became_peered":"2024-01-24T13:58:18.797819+0000","last_unstale":"2024-01-24T13:58:23.207918+0000","last_undegraded":"2024-01-24T13:58:23.207918+0000","last_fullsized":"2024-01-24T13:58:23.207918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232475+0000","last_change":"2024-01-24T13:58:23.230154+0000","last_active":"2024-01-24T13:58:23.232475+0000","last_peered":"2024-01-24T13:58:23.232475+0000","last_clean":"2024-01-24T13:58:23.232475+0000","last_became_active":"2024-01-24T13:58:23.226934+0000","last_became_peered":"2024-01-24T13:58:23.226934+0000","last_unstale":"2024-01-24T13:58:23.232475+0000","last_undegraded":"2024-01-24T13:58:23.232475+0000","last_fullsized":"2024-01-24T13:58:23.232475+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235379+0000","last_change":"2024-01-24T13:58:19.223917+0000","last_active":"2024-01-24T13:58:23.235379+0000","last_peered":"2024-01-24T13:58:23.235379+0000","last_clean":"2024-01-24T13:58:23.235379+0000","last_became_active":"2024-01-24T13:58:19.223624+0000","last_became_peered":"2024-01-24T13:58:19.223624+0000","last_unstale":"2024-01-24T13:58:23.235379+0000","last_undegraded":"2024-01-24T13:58:23.235379+0000","last_fullsized":"2024-01-24T13:58:23.235379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211872+0000","last_change":"2024-01-24T13:58:18.799495+0000","last_active":"2024-01-24T13:58:23.211872+0000","last_peered":"2024-01-24T13:58:23.211872+0000","last_clean":"2024-01-24T13:58:23.211872+0000","last_became_active":"2024-01-24T13:58:18.796442+0000","last_became_peered":"2024-01-24T13:58:18.796442+0000","last_unstale":"2024-01-24T13:58:23.211872+0000","last_undegraded":"2024-01-24T13:58:23.211872+0000","last_fullsized":"2024-01-24T13:58:23.211872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217101+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T13:58:23.217101+0000","last_peered":"2024-01-24T13:58:23.217101+0000","last_clean":"2024-01-24T13:58:23.217101+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T13:58:23.217101+0000","last_undegraded":"2024-01-24T13:58:23.217101+0000","last_fullsized":"2024-01-24T13:58:23.217101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.230573+0000","last_change":"2024-01-24T13:58:23.228846+0000","last_active":"2024-01-24T13:58:23.230573+0000","last_peered":"2024-01-24T13:58:23.230573+0000","last_clean":"2024-01-24T13:58:23.230573+0000","last_became_active":"2024-01-24T13:58:23.228366+0000","last_became_peered":"2024-01-24T13:58:23.228366+0000","last_unstale":"2024-01-24T13:58:23.230573+0000","last_undegraded":"2024-01-24T13:58:23.230573+0000","last_fullsized":"2024-01-24T13:58:23.230573+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232638+0000","last_change":"2024-01-24T13:58:18.810910+0000","last_active":"2024-01-24T13:58:23.232638+0000","last_peered":"2024-01-24T13:58:23.232638+0000","last_clean":"2024-01-24T13:58:23.232638+0000","last_became_active":"2024-01-24T13:58:18.810566+0000","last_became_peered":"2024-01-24T13:58:18.810566+0000","last_unstale":"2024-01-24T13:58:23.232638+0000","last_undegraded":"2024-01-24T13:58:23.232638+0000","last_fullsized":"2024-01-24T13:58:23.232638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216517+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T13:58:23.216517+0000","last_peered":"2024-01-24T13:58:23.216517+0000","last_clean":"2024-01-24T13:58:23.216517+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T13:58:23.216517+0000","last_undegraded":"2024-01-24T13:58:23.216517+0000","last_fullsized":"2024-01-24T13:58:23.216517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237683+0000","last_change":"2024-01-24T13:58:19.209151+0000","last_active":"2024-01-24T13:58:23.237683+0000","last_peered":"2024-01-24T13:58:23.237683+0000","last_clean":"2024-01-24T13:58:23.237683+0000","last_became_active":"2024-01-24T13:58:19.208680+0000","last_became_peered":"2024-01-24T13:58:19.208680+0000","last_unstale":"2024-01-24T13:58:23.237683+0000","last_undegraded":"2024-01-24T13:58:23.237683+0000","last_fullsized":"2024-01-24T13:58:23.237683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237878+0000","last_change":"2024-01-24T13:58:23.235219+0000","last_active":"2024-01-24T13:58:23.237878+0000","last_peered":"2024-01-24T13:58:23.237878+0000","last_clean":"2024-01-24T13:58:23.237878+0000","last_became_active":"2024-01-24T13:58:23.234797+0000","last_became_peered":"2024-01-24T13:58:23.234797+0000","last_unstale":"2024-01-24T13:58:23.237878+0000","last_undegraded":"2024-01-24T13:58:23.237878+0000","last_fullsized":"2024-01-24T13:58:23.237878+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207091+0000","last_change":"2024-01-24T13:58:18.802046+0000","last_active":"2024-01-24T13:58:23.207091+0000","last_peered":"2024-01-24T13:58:23.207091+0000","last_clean":"2024-01-24T13:58:23.207091+0000","last_became_active":"2024-01-24T13:58:18.801757+0000","last_became_peered":"2024-01-24T13:58:18.801757+0000","last_unstale":"2024-01-24T13:58:23.207091+0000","last_undegraded":"2024-01-24T13:58:23.207091+0000","last_fullsized":"2024-01-24T13:58:23.207091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216324+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T13:58:23.216324+0000","last_peered":"2024-01-24T13:58:23.216324+0000","last_clean":"2024-01-24T13:58:23.216324+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T13:58:23.216324+0000","last_undegraded":"2024-01-24T13:58:23.216324+0000","last_fullsized":"2024-01-24T13:58:23.216324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227454+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T13:58:23.227454+0000","last_peered":"2024-01-24T13:58:23.227454+0000","last_clean":"2024-01-24T13:58:23.227454+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T13:58:23.227454+0000","last_undegraded":"2024-01-24T13:58:23.227454+0000","last_fullsized":"2024-01-24T13:58:23.227454+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212923+0000","last_change":"2024-01-24T13:58:19.211323+0000","last_active":"2024-01-24T13:58:23.212923+0000","last_peered":"2024-01-24T13:58:23.212923+0000","last_clean":"2024-01-24T13:58:23.212923+0000","last_became_active":"2024-01-24T13:58:19.211057+0000","last_became_peered":"2024-01-24T13:58:19.211057+0000","last_unstale":"2024-01-24T13:58:23.212923+0000","last_undegraded":"2024-01-24T13:58:23.212923+0000","last_fullsized":"2024-01-24T13:58:23.212923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229871+0000","last_change":"2024-01-24T13:58:23.227723+0000","last_active":"2024-01-24T13:58:23.229871+0000","last_peered":"2024-01-24T13:58:23.229871+0000","last_clean":"2024-01-24T13:58:23.229871+0000","last_became_active":"2024-01-24T13:58:23.227339+0000","last_became_peered":"2024-01-24T13:58:23.227339+0000","last_unstale":"2024-01-24T13:58:23.229871+0000","last_undegraded":"2024-01-24T13:58:23.229871+0000","last_fullsized":"2024-01-24T13:58:23.229871+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.210205+0000","last_change":"2024-01-24T13:58:18.798895+0000","last_active":"2024-01-24T13:58:23.210205+0000","last_peered":"2024-01-24T13:58:23.210205+0000","last_clean":"2024-01-24T13:58:23.210205+0000","last_became_active":"2024-01-24T13:58:18.796782+0000","last_became_peered":"2024-01-24T13:58:18.796782+0000","last_unstale":"2024-01-24T13:58:23.210205+0000","last_undegraded":"2024-01-24T13:58:23.210205+0000","last_fullsized":"2024-01-24T13:58:23.210205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.220366+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T13:58:23.220366+0000","last_peered":"2024-01-24T13:58:23.220366+0000","last_clean":"2024-01-24T13:58:23.220366+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T13:58:23.220366+0000","last_undegraded":"2024-01-24T13:58:23.220366+0000","last_fullsized":"2024-01-24T13:58:23.220366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211623+0000","last_change":"2024-01-24T13:58:19.212508+0000","last_active":"2024-01-24T13:58:23.211623+0000","last_peered":"2024-01-24T13:58:23.211623+0000","last_clean":"2024-01-24T13:58:23.211623+0000","last_became_active":"2024-01-24T13:58:19.212211+0000","last_became_peered":"2024-01-24T13:58:19.212211+0000","last_unstale":"2024-01-24T13:58:23.211623+0000","last_undegraded":"2024-01-24T13:58:23.211623+0000","last_fullsized":"2024-01-24T13:58:23.211623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241355+0000","last_change":"2024-01-24T13:58:23.238939+0000","last_active":"2024-01-24T13:58:23.241355+0000","last_peered":"2024-01-24T13:58:23.241355+0000","last_clean":"2024-01-24T13:58:23.241355+0000","last_became_active":"2024-01-24T13:58:23.234940+0000","last_became_peered":"2024-01-24T13:58:23.234940+0000","last_unstale":"2024-01-24T13:58:23.241355+0000","last_undegraded":"2024-01-24T13:58:23.241355+0000","last_fullsized":"2024-01-24T13:58:23.241355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211492+0000","last_change":"2024-01-24T13:58:19.213076+0000","last_active":"2024-01-24T13:58:23.211492+0000","last_peered":"2024-01-24T13:58:23.211492+0000","last_clean":"2024-01-24T13:58:23.211492+0000","last_became_active":"2024-01-24T13:58:19.208454+0000","last_became_peered":"2024-01-24T13:58:19.208454+0000","last_unstale":"2024-01-24T13:58:23.211492+0000","last_undegraded":"2024-01-24T13:58:23.211492+0000","last_fullsized":"2024-01-24T13:58:23.211492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209311+0000","last_change":"2024-01-24T13:58:18.806850+0000","last_active":"2024-01-24T13:58:23.209311+0000","last_peered":"2024-01-24T13:58:23.209311+0000","last_clean":"2024-01-24T13:58:23.209311+0000","last_became_active":"2024-01-24T13:58:18.806456+0000","last_became_peered":"2024-01-24T13:58:18.806456+0000","last_unstale":"2024-01-24T13:58:23.209311+0000","last_undegraded":"2024-01-24T13:58:23.209311+0000","last_fullsized":"2024-01-24T13:58:23.209311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.226775+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T13:58:23.226775+0000","last_peered":"2024-01-24T13:58:23.226775+0000","last_clean":"2024-01-24T13:58:23.226775+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T13:58:23.226775+0000","last_undegraded":"2024-01-24T13:58:23.226775+0000","last_fullsized":"2024-01-24T13:58:23.226775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238433+0000","last_change":"2024-01-24T13:58:23.237234+0000","last_active":"2024-01-24T13:58:23.238433+0000","last_peered":"2024-01-24T13:58:23.238433+0000","last_clean":"2024-01-24T13:58:23.238433+0000","last_became_active":"2024-01-24T13:58:23.236720+0000","last_became_peered":"2024-01-24T13:58:23.236720+0000","last_unstale":"2024-01-24T13:58:23.238433+0000","last_undegraded":"2024-01-24T13:58:23.238433+0000","last_fullsized":"2024-01-24T13:58:23.238433+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212103+0000","last_change":"2024-01-24T13:58:19.217116+0000","last_active":"2024-01-24T13:58:23.212103+0000","last_peered":"2024-01-24T13:58:23.212103+0000","last_clean":"2024-01-24T13:58:23.212103+0000","last_became_active":"2024-01-24T13:58:19.216731+0000","last_became_peered":"2024-01-24T13:58:19.216731+0000","last_unstale":"2024-01-24T13:58:23.212103+0000","last_undegraded":"2024-01-24T13:58:23.212103+0000","last_fullsized":"2024-01-24T13:58:23.212103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227689+0000","last_change":"2024-01-24T13:58:18.803400+0000","last_active":"2024-01-24T13:58:23.227689+0000","last_peered":"2024-01-24T13:58:23.227689+0000","last_clean":"2024-01-24T13:58:23.227689+0000","last_became_active":"2024-01-24T13:58:18.802706+0000","last_became_peered":"2024-01-24T13:58:18.802706+0000","last_unstale":"2024-01-24T13:58:23.227689+0000","last_undegraded":"2024-01-24T13:58:23.227689+0000","last_fullsized":"2024-01-24T13:58:23.227689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211632+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T13:58:23.211632+0000","last_peered":"2024-01-24T13:58:23.211632+0000","last_clean":"2024-01-24T13:58:23.211632+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T13:58:23.211632+0000","last_undegraded":"2024-01-24T13:58:23.211632+0000","last_fullsized":"2024-01-24T13:58:23.211632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237685+0000","last_change":"2024-01-24T13:58:23.235636+0000","last_active":"2024-01-24T13:58:23.237685+0000","last_peered":"2024-01-24T13:58:23.237685+0000","last_clean":"2024-01-24T13:58:23.237685+0000","last_became_active":"2024-01-24T13:58:23.234531+0000","last_became_peered":"2024-01-24T13:58:23.234531+0000","last_unstale":"2024-01-24T13:58:23.237685+0000","last_undegraded":"2024-01-24T13:58:23.237685+0000","last_fullsized":"2024-01-24T13:58:23.237685+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207375+0000","last_change":"2024-01-24T13:58:18.806700+0000","last_active":"2024-01-24T13:58:23.207375+0000","last_peered":"2024-01-24T13:58:23.207375+0000","last_clean":"2024-01-24T13:58:23.207375+0000","last_became_active":"2024-01-24T13:58:18.805917+0000","last_became_peered":"2024-01-24T13:58:18.805917+0000","last_unstale":"2024-01-24T13:58:23.207375+0000","last_undegraded":"2024-01-24T13:58:23.207375+0000","last_fullsized":"2024-01-24T13:58:23.207375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.211671+0000","last_change":"2024-01-24T13:58:19.215828+0000","last_active":"2024-01-24T13:58:23.211671+0000","last_peered":"2024-01-24T13:58:23.211671+0000","last_clean":"2024-01-24T13:58:23.211671+0000","last_became_active":"2024-01-24T13:58:19.215015+0000","last_became_peered":"2024-01-24T13:58:19.215015+0000","last_unstale":"2024-01-24T13:58:23.211671+0000","last_undegraded":"2024-01-24T13:58:23.211671+0000","last_fullsized":"2024-01-24T13:58:23.211671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238346+0000","last_change":"2024-01-24T13:58:23.236631+0000","last_active":"2024-01-24T13:58:23.238346+0000","last_peered":"2024-01-24T13:58:23.238346+0000","last_clean":"2024-01-24T13:58:23.238346+0000","last_became_active":"2024-01-24T13:58:23.236200+0000","last_became_peered":"2024-01-24T13:58:23.236200+0000","last_unstale":"2024-01-24T13:58:23.238346+0000","last_undegraded":"2024-01-24T13:58:23.238346+0000","last_fullsized":"2024-01-24T13:58:23.238346+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209848+0000","last_change":"2024-01-24T13:58:18.803901+0000","last_active":"2024-01-24T13:58:23.209848+0000","last_peered":"2024-01-24T13:58:23.209848+0000","last_clean":"2024-01-24T13:58:23.209848+0000","last_became_active":"2024-01-24T13:58:18.803605+0000","last_became_peered":"2024-01-24T13:58:18.803605+0000","last_unstale":"2024-01-24T13:58:23.209848+0000","last_undegraded":"2024-01-24T13:58:23.209848+0000","last_fullsized":"2024-01-24T13:58:23.209848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209809+0000","last_change":"2024-01-24T13:58:19.216716+0000","last_active":"2024-01-24T13:58:23.209809+0000","last_peered":"2024-01-24T13:58:23.209809+0000","last_clean":"2024-01-24T13:58:23.209809+0000","last_became_active":"2024-01-24T13:58:19.216444+0000","last_became_peered":"2024-01-24T13:58:19.216444+0000","last_unstale":"2024-01-24T13:58:23.209809+0000","last_undegraded":"2024-01-24T13:58:23.209809+0000","last_fullsized":"2024-01-24T13:58:23.209809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.242821+0000","last_change":"2024-01-24T13:58:23.242205+0000","last_active":"2024-01-24T13:58:23.242821+0000","last_peered":"2024-01-24T13:58:23.242821+0000","last_clean":"2024-01-24T13:58:23.242821+0000","last_became_active":"2024-01-24T13:58:23.241764+0000","last_became_peered":"2024-01-24T13:58:23.241764+0000","last_unstale":"2024-01-24T13:58:23.242821+0000","last_undegraded":"2024-01-24T13:58:23.242821+0000","last_fullsized":"2024-01-24T13:58:23.242821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212581+0000","last_change":"2024-01-24T13:58:18.808313+0000","last_active":"2024-01-24T13:58:23.212581+0000","last_peered":"2024-01-24T13:58:23.212581+0000","last_clean":"2024-01-24T13:58:23.212581+0000","last_became_active":"2024-01-24T13:58:18.808096+0000","last_became_peered":"2024-01-24T13:58:18.808096+0000","last_unstale":"2024-01-24T13:58:23.212581+0000","last_undegraded":"2024-01-24T13:58:23.212581+0000","last_fullsized":"2024-01-24T13:58:23.212581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209286+0000","last_change":"2024-01-24T13:58:19.209979+0000","last_active":"2024-01-24T13:58:23.209286+0000","last_peered":"2024-01-24T13:58:23.209286+0000","last_clean":"2024-01-24T13:58:23.209286+0000","last_became_active":"2024-01-24T13:58:19.209672+0000","last_became_peered":"2024-01-24T13:58:19.209672+0000","last_unstale":"2024-01-24T13:58:23.209286+0000","last_undegraded":"2024-01-24T13:58:23.209286+0000","last_fullsized":"2024-01-24T13:58:23.209286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.240684+0000","last_change":"2024-01-24T13:58:23.229668+0000","last_active":"2024-01-24T13:58:23.240684+0000","last_peered":"2024-01-24T13:58:23.240684+0000","last_clean":"2024-01-24T13:58:23.240684+0000","last_became_active":"2024-01-24T13:58:23.229053+0000","last_became_peered":"2024-01-24T13:58:23.229053+0000","last_unstale":"2024-01-24T13:58:23.240684+0000","last_undegraded":"2024-01-24T13:58:23.240684+0000","last_fullsized":"2024-01-24T13:58:23.240684+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207919+0000","last_change":"2024-01-24T13:58:18.807736+0000","last_active":"2024-01-24T13:58:23.207919+0000","last_peered":"2024-01-24T13:58:23.207919+0000","last_clean":"2024-01-24T13:58:23.207919+0000","last_became_active":"2024-01-24T13:58:18.807228+0000","last_became_peered":"2024-01-24T13:58:18.807228+0000","last_unstale":"2024-01-24T13:58:23.207919+0000","last_undegraded":"2024-01-24T13:58:23.207919+0000","last_fullsized":"2024-01-24T13:58:23.207919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209909+0000","last_change":"2024-01-24T13:58:19.218038+0000","last_active":"2024-01-24T13:58:23.209909+0000","last_peered":"2024-01-24T13:58:23.209909+0000","last_clean":"2024-01-24T13:58:23.209909+0000","last_became_active":"2024-01-24T13:58:19.210668+0000","last_became_peered":"2024-01-24T13:58:19.210668+0000","last_unstale":"2024-01-24T13:58:23.209909+0000","last_undegraded":"2024-01-24T13:58:23.209909+0000","last_fullsized":"2024-01-24T13:58:23.209909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241201+0000","last_change":"2024-01-24T13:58:23.234754+0000","last_active":"2024-01-24T13:58:23.241201+0000","last_peered":"2024-01-24T13:58:23.241201+0000","last_clean":"2024-01-24T13:58:23.241201+0000","last_became_active":"2024-01-24T13:58:23.233898+0000","last_became_peered":"2024-01-24T13:58:23.233898+0000","last_unstale":"2024-01-24T13:58:23.241201+0000","last_undegraded":"2024-01-24T13:58:23.241201+0000","last_fullsized":"2024-01-24T13:58:23.241201+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227851+0000","last_change":"2024-01-24T13:58:18.802179+0000","last_active":"2024-01-24T13:58:23.227851+0000","last_peered":"2024-01-24T13:58:23.227851+0000","last_clean":"2024-01-24T13:58:23.227851+0000","last_became_active":"2024-01-24T13:58:18.801898+0000","last_became_peered":"2024-01-24T13:58:18.801898+0000","last_unstale":"2024-01-24T13:58:23.227851+0000","last_undegraded":"2024-01-24T13:58:23.227851+0000","last_fullsized":"2024-01-24T13:58:23.227851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.234948+0000","last_change":"2024-01-24T13:58:19.224275+0000","last_active":"2024-01-24T13:58:23.234948+0000","last_peered":"2024-01-24T13:58:23.234948+0000","last_clean":"2024-01-24T13:58:23.234948+0000","last_became_active":"2024-01-24T13:58:19.224006+0000","last_became_peered":"2024-01-24T13:58:19.224006+0000","last_unstale":"2024-01-24T13:58:23.234948+0000","last_undegraded":"2024-01-24T13:58:23.234948+0000","last_fullsized":"2024-01-24T13:58:23.234948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241374+0000","last_change":"2024-01-24T13:58:23.239977+0000","last_active":"2024-01-24T13:58:23.241374+0000","last_peered":"2024-01-24T13:58:23.241374+0000","last_clean":"2024-01-24T13:58:23.241374+0000","last_became_active":"2024-01-24T13:58:23.234358+0000","last_became_peered":"2024-01-24T13:58:23.234358+0000","last_unstale":"2024-01-24T13:58:23.241374+0000","last_undegraded":"2024-01-24T13:58:23.241374+0000","last_fullsized":"2024-01-24T13:58:23.241374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217009+0000","last_change":"2024-01-24T13:58:18.804317+0000","last_active":"2024-01-24T13:58:23.217009+0000","last_peered":"2024-01-24T13:58:23.217009+0000","last_clean":"2024-01-24T13:58:23.217009+0000","last_became_active":"2024-01-24T13:58:18.804046+0000","last_became_peered":"2024-01-24T13:58:18.804046+0000","last_unstale":"2024-01-24T13:58:23.217009+0000","last_undegraded":"2024-01-24T13:58:23.217009+0000","last_fullsized":"2024-01-24T13:58:23.217009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212503+0000","last_change":"2024-01-24T13:58:19.217782+0000","last_active":"2024-01-24T13:58:23.212503+0000","last_peered":"2024-01-24T13:58:23.212503+0000","last_clean":"2024-01-24T13:58:23.212503+0000","last_became_active":"2024-01-24T13:58:19.217509+0000","last_became_peered":"2024-01-24T13:58:19.217509+0000","last_unstale":"2024-01-24T13:58:23.212503+0000","last_undegraded":"2024-01-24T13:58:23.212503+0000","last_fullsized":"2024-01-24T13:58:23.212503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.241249+0000","last_change":"2024-01-24T13:58:23.237049+0000","last_active":"2024-01-24T13:58:23.241249+0000","last_peered":"2024-01-24T13:58:23.241249+0000","last_clean":"2024-01-24T13:58:23.241249+0000","last_became_active":"2024-01-24T13:58:23.235154+0000","last_became_peered":"2024-01-24T13:58:23.235154+0000","last_unstale":"2024-01-24T13:58:23.241249+0000","last_undegraded":"2024-01-24T13:58:23.241249+0000","last_fullsized":"2024-01-24T13:58:23.241249+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217784+0000","last_change":"2024-01-24T13:58:18.801841+0000","last_active":"2024-01-24T13:58:23.217784+0000","last_peered":"2024-01-24T13:58:23.217784+0000","last_clean":"2024-01-24T13:58:23.217784+0000","last_became_active":"2024-01-24T13:58:18.801535+0000","last_became_peered":"2024-01-24T13:58:18.801535+0000","last_unstale":"2024-01-24T13:58:23.217784+0000","last_undegraded":"2024-01-24T13:58:23.217784+0000","last_fullsized":"2024-01-24T13:58:23.217784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209722+0000","last_change":"2024-01-24T13:58:19.213673+0000","last_active":"2024-01-24T13:58:23.209722+0000","last_peered":"2024-01-24T13:58:23.209722+0000","last_clean":"2024-01-24T13:58:23.209722+0000","last_became_active":"2024-01-24T13:58:19.213457+0000","last_became_peered":"2024-01-24T13:58:19.213457+0000","last_unstale":"2024-01-24T13:58:23.209722+0000","last_undegraded":"2024-01-24T13:58:23.209722+0000","last_fullsized":"2024-01-24T13:58:23.209722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.238263+0000","last_change":"2024-01-24T13:58:23.234302+0000","last_active":"2024-01-24T13:58:23.238263+0000","last_peered":"2024-01-24T13:58:23.238263+0000","last_clean":"2024-01-24T13:58:23.238263+0000","last_became_active":"2024-01-24T13:58:23.233692+0000","last_became_peered":"2024-01-24T13:58:23.233692+0000","last_unstale":"2024-01-24T13:58:23.238263+0000","last_undegraded":"2024-01-24T13:58:23.238263+0000","last_fullsized":"2024-01-24T13:58:23.238263+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.218145+0000","last_change":"2024-01-24T13:58:18.798076+0000","last_active":"2024-01-24T13:58:23.218145+0000","last_peered":"2024-01-24T13:58:23.218145+0000","last_clean":"2024-01-24T13:58:23.218145+0000","last_became_active":"2024-01-24T13:58:18.797403+0000","last_became_peered":"2024-01-24T13:58:18.797403+0000","last_unstale":"2024-01-24T13:58:23.218145+0000","last_undegraded":"2024-01-24T13:58:23.218145+0000","last_fullsized":"2024-01-24T13:58:23.218145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.224903+0000","last_change":"2024-01-24T13:58:19.210870+0000","last_active":"2024-01-24T13:58:23.224903+0000","last_peered":"2024-01-24T13:58:23.224903+0000","last_clean":"2024-01-24T13:58:23.224903+0000","last_became_active":"2024-01-24T13:58:19.210226+0000","last_became_peered":"2024-01-24T13:58:19.210226+0000","last_unstale":"2024-01-24T13:58:23.224903+0000","last_undegraded":"2024-01-24T13:58:23.224903+0000","last_fullsized":"2024-01-24T13:58:23.224903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232630+0000","last_change":"2024-01-24T13:58:23.230340+0000","last_active":"2024-01-24T13:58:23.232630+0000","last_peered":"2024-01-24T13:58:23.232630+0000","last_clean":"2024-01-24T13:58:23.232630+0000","last_became_active":"2024-01-24T13:58:23.229909+0000","last_became_peered":"2024-01-24T13:58:23.229909+0000","last_unstale":"2024-01-24T13:58:23.232630+0000","last_undegraded":"2024-01-24T13:58:23.232630+0000","last_fullsized":"2024-01-24T13:58:23.232630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"21'4","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212395+0000","last_change":"2024-01-24T13:58:18.813040+0000","last_active":"2024-01-24T13:58:23.212395+0000","last_peered":"2024-01-24T13:58:23.212395+0000","last_clean":"2024-01-24T13:58:23.212395+0000","last_became_active":"2024-01-24T13:58:18.812822+0000","last_became_peered":"2024-01-24T13:58:18.812822+0000","last_unstale":"2024-01-24T13:58:23.212395+0000","last_undegraded":"2024-01-24T13:58:23.212395+0000","last_fullsized":"2024-01-24T13:58:23.212395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208215+0000","last_change":"2024-01-24T13:58:19.216414+0000","last_active":"2024-01-24T13:58:23.208215+0000","last_peered":"2024-01-24T13:58:23.208215+0000","last_clean":"2024-01-24T13:58:23.208215+0000","last_became_active":"2024-01-24T13:58:19.216111+0000","last_became_peered":"2024-01-24T13:58:19.216111+0000","last_unstale":"2024-01-24T13:58:23.208215+0000","last_undegraded":"2024-01-24T13:58:23.208215+0000","last_fullsized":"2024-01-24T13:58:23.208215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237430+0000","last_change":"2024-01-24T13:58:23.234462+0000","last_active":"2024-01-24T13:58:23.237430+0000","last_peered":"2024-01-24T13:58:23.237430+0000","last_clean":"2024-01-24T13:58:23.237430+0000","last_became_active":"2024-01-24T13:58:23.233834+0000","last_became_peered":"2024-01-24T13:58:23.233834+0000","last_unstale":"2024-01-24T13:58:23.237430+0000","last_undegraded":"2024-01-24T13:58:23.237430+0000","last_fullsized":"2024-01-24T13:58:23.237430+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.216341+0000","last_change":"2024-01-24T13:58:18.801184+0000","last_active":"2024-01-24T13:58:23.216341+0000","last_peered":"2024-01-24T13:58:23.216341+0000","last_clean":"2024-01-24T13:58:23.216341+0000","last_became_active":"2024-01-24T13:58:18.800877+0000","last_became_peered":"2024-01-24T13:58:18.800877+0000","last_unstale":"2024-01-24T13:58:23.216341+0000","last_undegraded":"2024-01-24T13:58:23.216341+0000","last_fullsized":"2024-01-24T13:58:23.216341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237737+0000","last_change":"2024-01-24T13:58:19.209281+0000","last_active":"2024-01-24T13:58:23.237737+0000","last_peered":"2024-01-24T13:58:23.237737+0000","last_clean":"2024-01-24T13:58:23.237737+0000","last_became_active":"2024-01-24T13:58:19.208890+0000","last_became_peered":"2024-01-24T13:58:19.208890+0000","last_unstale":"2024-01-24T13:58:23.237737+0000","last_undegraded":"2024-01-24T13:58:23.237737+0000","last_fullsized":"2024-01-24T13:58:23.237737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.230375+0000","last_change":"2024-01-24T13:58:23.228668+0000","last_active":"2024-01-24T13:58:23.230375+0000","last_peered":"2024-01-24T13:58:23.230375+0000","last_clean":"2024-01-24T13:58:23.230375+0000","last_became_active":"2024-01-24T13:58:23.228266+0000","last_became_peered":"2024-01-24T13:58:23.228266+0000","last_unstale":"2024-01-24T13:58:23.230375+0000","last_undegraded":"2024-01-24T13:58:23.230375+0000","last_fullsized":"2024-01-24T13:58:23.230375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232549+0000","last_change":"2024-01-24T13:58:18.809056+0000","last_active":"2024-01-24T13:58:23.232549+0000","last_peered":"2024-01-24T13:58:23.232549+0000","last_clean":"2024-01-24T13:58:23.232549+0000","last_became_active":"2024-01-24T13:58:18.808790+0000","last_became_peered":"2024-01-24T13:58:18.808790+0000","last_unstale":"2024-01-24T13:58:23.232549+0000","last_undegraded":"2024-01-24T13:58:23.232549+0000","last_fullsized":"2024-01-24T13:58:23.232549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227958+0000","last_change":"2024-01-24T13:58:19.217272+0000","last_active":"2024-01-24T13:58:23.227958+0000","last_peered":"2024-01-24T13:58:23.227958+0000","last_clean":"2024-01-24T13:58:23.227958+0000","last_became_active":"2024-01-24T13:58:19.216991+0000","last_became_peered":"2024-01-24T13:58:19.216991+0000","last_unstale":"2024-01-24T13:58:23.227958+0000","last_undegraded":"2024-01-24T13:58:23.227958+0000","last_fullsized":"2024-01-24T13:58:23.227958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.231091+0000","last_change":"2024-01-24T13:58:23.228760+0000","last_active":"2024-01-24T13:58:23.231091+0000","last_peered":"2024-01-24T13:58:23.231091+0000","last_clean":"2024-01-24T13:58:23.231091+0000","last_became_active":"2024-01-24T13:58:23.227345+0000","last_became_peered":"2024-01-24T13:58:23.227345+0000","last_unstale":"2024-01-24T13:58:23.231091+0000","last_undegraded":"2024-01-24T13:58:23.231091+0000","last_fullsized":"2024-01-24T13:58:23.231091+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"21'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217256+0000","last_change":"2024-01-24T13:58:18.798145+0000","last_active":"2024-01-24T13:58:23.217256+0000","last_peered":"2024-01-24T13:58:23.217256+0000","last_clean":"2024-01-24T13:58:23.217256+0000","last_became_active":"2024-01-24T13:58:18.797671+0000","last_became_peered":"2024-01-24T13:58:18.797671+0000","last_unstale":"2024-01-24T13:58:23.217256+0000","last_undegraded":"2024-01-24T13:58:23.217256+0000","last_fullsized":"2024-01-24T13:58:23.217256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207234+0000","last_change":"2024-01-24T13:58:19.221287+0000","last_active":"2024-01-24T13:58:23.207234+0000","last_peered":"2024-01-24T13:58:23.207234+0000","last_clean":"2024-01-24T13:58:23.207234+0000","last_became_active":"2024-01-24T13:58:19.220970+0000","last_became_peered":"2024-01-24T13:58:19.220970+0000","last_unstale":"2024-01-24T13:58:23.207234+0000","last_undegraded":"2024-01-24T13:58:23.207234+0000","last_fullsized":"2024-01-24T13:58:23.207234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237365+0000","last_change":"2024-01-24T13:58:23.234514+0000","last_active":"2024-01-24T13:58:23.237365+0000","last_peered":"2024-01-24T13:58:23.237365+0000","last_clean":"2024-01-24T13:58:23.237365+0000","last_became_active":"2024-01-24T13:58:23.233824+0000","last_became_peered":"2024-01-24T13:58:23.233824+0000","last_unstale":"2024-01-24T13:58:23.237365+0000","last_undegraded":"2024-01-24T13:58:23.237365+0000","last_fullsized":"2024-01-24T13:58:23.237365+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"21'2","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209685+0000","last_change":"2024-01-24T13:58:18.801316+0000","last_active":"2024-01-24T13:58:23.209685+0000","last_peered":"2024-01-24T13:58:23.209685+0000","last_clean":"2024-01-24T13:58:23.209685+0000","last_became_active":"2024-01-24T13:58:18.800089+0000","last_became_peered":"2024-01-24T13:58:18.800089+0000","last_unstale":"2024-01-24T13:58:23.209685+0000","last_undegraded":"2024-01-24T13:58:23.209685+0000","last_fullsized":"2024-01-24T13:58:23.209685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209210+0000","last_change":"2024-01-24T13:58:19.209604+0000","last_active":"2024-01-24T13:58:23.209210+0000","last_peered":"2024-01-24T13:58:23.209210+0000","last_clean":"2024-01-24T13:58:23.209210+0000","last_became_active":"2024-01-24T13:58:19.209306+0000","last_became_peered":"2024-01-24T13:58:19.209306+0000","last_unstale":"2024-01-24T13:58:23.209210+0000","last_undegraded":"2024-01-24T13:58:23.209210+0000","last_fullsized":"2024-01-24T13:58:23.209210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229399+0000","last_change":"2024-01-24T13:58:23.225921+0000","last_active":"2024-01-24T13:58:23.229399+0000","last_peered":"2024-01-24T13:58:23.229399+0000","last_clean":"2024-01-24T13:58:23.229399+0000","last_became_active":"2024-01-24T13:58:23.225413+0000","last_became_peered":"2024-01-24T13:58:23.225413+0000","last_unstale":"2024-01-24T13:58:23.229399+0000","last_undegraded":"2024-01-24T13:58:23.229399+0000","last_fullsized":"2024-01-24T13:58:23.229399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.217204+0000","last_change":"2024-01-24T13:58:18.799227+0000","last_active":"2024-01-24T13:58:23.217204+0000","last_peered":"2024-01-24T13:58:23.217204+0000","last_clean":"2024-01-24T13:58:23.217204+0000","last_became_active":"2024-01-24T13:58:18.798927+0000","last_became_peered":"2024-01-24T13:58:18.798927+0000","last_unstale":"2024-01-24T13:58:23.217204+0000","last_undegraded":"2024-01-24T13:58:23.217204+0000","last_fullsized":"2024-01-24T13:58:23.217204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212286+0000","last_change":"2024-01-24T13:58:19.217668+0000","last_active":"2024-01-24T13:58:23.212286+0000","last_peered":"2024-01-24T13:58:23.212286+0000","last_clean":"2024-01-24T13:58:23.212286+0000","last_became_active":"2024-01-24T13:58:19.217323+0000","last_became_peered":"2024-01-24T13:58:19.217323+0000","last_unstale":"2024-01-24T13:58:23.212286+0000","last_undegraded":"2024-01-24T13:58:23.212286+0000","last_fullsized":"2024-01-24T13:58:23.212286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.230948+0000","last_change":"2024-01-24T13:58:23.229837+0000","last_active":"2024-01-24T13:58:23.230948+0000","last_peered":"2024-01-24T13:58:23.230948+0000","last_clean":"2024-01-24T13:58:23.230948+0000","last_became_active":"2024-01-24T13:58:23.229398+0000","last_became_peered":"2024-01-24T13:58:23.229398+0000","last_unstale":"2024-01-24T13:58:23.230948+0000","last_undegraded":"2024-01-24T13:58:23.230948+0000","last_fullsized":"2024-01-24T13:58:23.230948+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.227679+0000","last_change":"2024-01-24T13:58:18.798763+0000","last_active":"2024-01-24T13:58:23.227679+0000","last_peered":"2024-01-24T13:58:23.227679+0000","last_clean":"2024-01-24T13:58:23.227679+0000","last_became_active":"2024-01-24T13:58:18.798451+0000","last_became_peered":"2024-01-24T13:58:18.798451+0000","last_unstale":"2024-01-24T13:58:23.227679+0000","last_undegraded":"2024-01-24T13:58:23.227679+0000","last_fullsized":"2024-01-24T13:58:23.227679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207643+0000","last_change":"2024-01-24T13:58:19.221011+0000","last_active":"2024-01-24T13:58:23.207643+0000","last_peered":"2024-01-24T13:58:23.207643+0000","last_clean":"2024-01-24T13:58:23.207643+0000","last_became_active":"2024-01-24T13:58:19.220727+0000","last_became_peered":"2024-01-24T13:58:19.220727+0000","last_unstale":"2024-01-24T13:58:23.207643+0000","last_undegraded":"2024-01-24T13:58:23.207643+0000","last_fullsized":"2024-01-24T13:58:23.207643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237556+0000","last_change":"2024-01-24T13:58:23.235211+0000","last_active":"2024-01-24T13:58:23.237556+0000","last_peered":"2024-01-24T13:58:23.237556+0000","last_clean":"2024-01-24T13:58:23.237556+0000","last_became_active":"2024-01-24T13:58:23.234791+0000","last_became_peered":"2024-01-24T13:58:23.234791+0000","last_unstale":"2024-01-24T13:58:23.237556+0000","last_undegraded":"2024-01-24T13:58:23.237556+0000","last_fullsized":"2024-01-24T13:58:23.237556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237921+0000","last_change":"2024-01-24T13:58:18.815292+0000","last_active":"2024-01-24T13:58:23.237921+0000","last_peered":"2024-01-24T13:58:23.237921+0000","last_clean":"2024-01-24T13:58:23.237921+0000","last_became_active":"2024-01-24T13:58:18.811889+0000","last_became_peered":"2024-01-24T13:58:18.811889+0000","last_unstale":"2024-01-24T13:58:23.237921+0000","last_undegraded":"2024-01-24T13:58:23.237921+0000","last_fullsized":"2024-01-24T13:58:23.237921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.207065+0000","last_change":"2024-01-24T13:58:19.218174+0000","last_active":"2024-01-24T13:58:23.207065+0000","last_peered":"2024-01-24T13:58:23.207065+0000","last_clean":"2024-01-24T13:58:23.207065+0000","last_became_active":"2024-01-24T13:58:19.211059+0000","last_became_peered":"2024-01-24T13:58:19.211059+0000","last_unstale":"2024-01-24T13:58:23.207065+0000","last_undegraded":"2024-01-24T13:58:23.207065+0000","last_fullsized":"2024-01-24T13:58:23.207065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229596+0000","last_change":"2024-01-24T13:58:23.225806+0000","last_active":"2024-01-24T13:58:23.229596+0000","last_peered":"2024-01-24T13:58:23.229596+0000","last_clean":"2024-01-24T13:58:23.229596+0000","last_became_active":"2024-01-24T13:58:23.225425+0000","last_became_peered":"2024-01-24T13:58:23.225425+0000","last_unstale":"2024-01-24T13:58:23.229596+0000","last_undegraded":"2024-01-24T13:58:23.229596+0000","last_fullsized":"2024-01-24T13:58:23.229596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.235654+0000","last_change":"2024-01-24T13:58:23.226907+0000","last_active":"2024-01-24T13:58:23.235654+0000","last_peered":"2024-01-24T13:58:23.235654+0000","last_clean":"2024-01-24T13:58:23.235654+0000","last_became_active":"2024-01-24T13:58:23.226413+0000","last_became_peered":"2024-01-24T13:58:23.226413+0000","last_unstale":"2024-01-24T13:58:23.235654+0000","last_undegraded":"2024-01-24T13:58:23.235654+0000","last_fullsized":"2024-01-24T13:58:23.235654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.206957+0000","last_change":"2024-01-24T13:58:18.806830+0000","last_active":"2024-01-24T13:58:23.206957+0000","last_peered":"2024-01-24T13:58:23.206957+0000","last_clean":"2024-01-24T13:58:23.206957+0000","last_became_active":"2024-01-24T13:58:18.806385+0000","last_became_peered":"2024-01-24T13:58:18.806385+0000","last_unstale":"2024-01-24T13:58:23.206957+0000","last_undegraded":"2024-01-24T13:58:23.206957+0000","last_fullsized":"2024-01-24T13:58:23.206957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.212967+0000","last_change":"2024-01-24T13:58:19.210472+0000","last_active":"2024-01-24T13:58:23.212967+0000","last_peered":"2024-01-24T13:58:23.212967+0000","last_clean":"2024-01-24T13:58:23.212967+0000","last_became_active":"2024-01-24T13:58:19.210211+0000","last_became_peered":"2024-01-24T13:58:19.210211+0000","last_unstale":"2024-01-24T13:58:23.212967+0000","last_undegraded":"2024-01-24T13:58:23.212967+0000","last_fullsized":"2024-01-24T13:58:23.212967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.229446+0000","last_change":"2024-01-24T13:58:23.223133+0000","last_active":"2024-01-24T13:58:23.229446+0000","last_peered":"2024-01-24T13:58:23.229446+0000","last_clean":"2024-01-24T13:58:23.229446+0000","last_became_active":"2024-01-24T13:58:23.222741+0000","last_became_peered":"2024-01-24T13:58:23.222741+0000","last_unstale":"2024-01-24T13:58:23.229446+0000","last_undegraded":"2024-01-24T13:58:23.229446+0000","last_fullsized":"2024-01-24T13:58:23.229446+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.210236+0000","last_change":"2024-01-24T13:58:18.800359+0000","last_active":"2024-01-24T13:58:23.210236+0000","last_peered":"2024-01-24T13:58:23.210236+0000","last_clean":"2024-01-24T13:58:23.210236+0000","last_became_active":"2024-01-24T13:58:18.799774+0000","last_became_peered":"2024-01-24T13:58:18.799774+0000","last_unstale":"2024-01-24T13:58:23.210236+0000","last_undegraded":"2024-01-24T13:58:23.210236+0000","last_fullsized":"2024-01-24T13:58:23.210236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.237579+0000","last_change":"2024-01-24T13:58:19.210774+0000","last_active":"2024-01-24T13:58:23.237579+0000","last_peered":"2024-01-24T13:58:23.237579+0000","last_clean":"2024-01-24T13:58:23.237579+0000","last_became_active":"2024-01-24T13:58:19.210508+0000","last_became_peered":"2024-01-24T13:58:19.210508+0000","last_unstale":"2024-01-24T13:58:23.237579+0000","last_undegraded":"2024-01-24T13:58:23.237579+0000","last_fullsized":"2024-01-24T13:58:23.237579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.209129+0000","last_change":"2024-01-24T13:58:18.803254+0000","last_active":"2024-01-24T13:58:23.209129+0000","last_peered":"2024-01-24T13:58:23.209129+0000","last_clean":"2024-01-24T13:58:23.209129+0000","last_became_active":"2024-01-24T13:58:18.802965+0000","last_became_peered":"2024-01-24T13:58:18.802965+0000","last_unstale":"2024-01-24T13:58:23.209129+0000","last_undegraded":"2024-01-24T13:58:23.209129+0000","last_fullsized":"2024-01-24T13:58:23.209129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:17.765776+0000","last_clean_scrub_stamp":"2024-01-24T13:58:17.765776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.208494+0000","last_change":"2024-01-24T13:58:19.221404+0000","last_active":"2024-01-24T13:58:23.208494+0000","last_peered":"2024-01-24T13:58:23.208494+0000","last_clean":"2024-01-24T13:58:23.208494+0000","last_became_active":"2024-01-24T13:58:19.221024+0000","last_became_peered":"2024-01-24T13:58:19.221024+0000","last_unstale":"2024-01-24T13:58:23.208494+0000","last_undegraded":"2024-01-24T13:58:23.208494+0000","last_fullsized":"2024-01-24T13:58:23.208494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:18.771584+0000","last_clean_scrub_stamp":"2024-01-24T13:58:18.771584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-24T13:58:23.232448+0000","last_change":"2024-01-24T13:58:23.229756+0000","last_active":"2024-01-24T13:58:23.232448+0000","last_peered":"2024-01-24T13:58:23.232448+0000","last_clean":"2024-01-24T13:58:23.232448+0000","last_became_active":"2024-01-24T13:58:23.229328+0000","last_became_peered":"2024-01-24T13:58:23.229328+0000","last_unstale":"2024-01-24T13:58:23.232448+0000","last_undegraded":"2024-01-24T13:58:23.232448+0000","last_fullsized":"2024-01-24T13:58:23.232448+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:22.197940+0000","last_clean_scrub_stamp":"2024-01-24T13:58:22.197940+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297364,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074476,"statfs":{"total":96636764160,"available":96332263424,"internally_reserved":0,"allocated":274432,"data_stored":115668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297396,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074444,"statfs":{"total":96636764160,"available":96332230656,"internally_reserved":0,"allocated":307200,"data_stored":118043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297372,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074468,"statfs":{"total":96636764160,"available":96332255232,"internally_reserved":0,"allocated":282624,"data_stored":116699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297368,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074472,"statfs":{"total":96636764160,"available":96332259328,"internally_reserved":0,"allocated":278528,"data_stored":116629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297372,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074468,"statfs":{"total":96636764160,"available":96332255232,"internally_reserved":0,"allocated":282624,"data_stored":116719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297388,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074452,"statfs":{"total":96636764160,"available":96332238848,"internally_reserved":0,"allocated":299008,"data_stored":120400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297368,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074472,"statfs":{"total":96636764160,"available":96332259328,"internally_reserved":0,"allocated":278528,"data_stored":116629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297376,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074464,"statfs":{"total":96636764160,"available":96332251136,"internally_reserved":0,"allocated":286720,"data_stored":120254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":3625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T13:58:40.050 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-24T13:58:40.050 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-24T13:58:40.050 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-24T13:58:40.050 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-24T13:58:40.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.218+0000 7fd254532700 1 Processor -- start 2024-01-24T13:58:40.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.218+0000 7fd254532700 1 -- start start 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.219+0000 7fd254532700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd24c0a9e50 0x7fd24c063fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.219+0000 7fd254532700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c065bc0 con 0x7fd24c0627b0 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.219+0000 7fd254532700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c0645b0 con 0x7fd24c0678c0 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.219+0000 7fd254532700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c152430 con 0x7fd24c0a9e50 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.220+0000 7fd252acf700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd24c0627b0 0x7fd24c062bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42212/0 (socket says 172.21.15.29:42212) 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.220+0000 7fd252acf700 1 -- 172.21.15.29:0/354879334 learned_addr learned my addr 172.21.15.29:0/354879334 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.220+0000 7fd2522ce700 1 --2- 172.21.15.29:0/354879334 >> v2:172.21.15.163:3300/0 conn(0x7fd24c0a9e50 0x7fd24c063fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.220+0000 7fd2512cc700 1 -- 172.21.15.29:0/354879334 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949661879 0 0) 0x7fd24c065bc0 con 0x7fd24c0627b0 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.220+0000 7fd2512cc700 1 -- 172.21.15.29:0/354879334 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2380033f0 con 0x7fd24c0627b0 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.220+0000 7fd2522ce700 1 -- 172.21.15.29:0/354879334 >> v1:172.21.15.163:6789/0 conn(0x7fd24c0678c0 legacy=0x7fd24c153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.220+0000 7fd2522ce700 1 -- 172.21.15.29:0/354879334 >> v1:172.21.15.29:6789/0 conn(0x7fd24c0627b0 legacy=0x7fd24c062bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.220+0000 7fd2522ce700 1 -- 172.21.15.29:0/354879334 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c065e80 con 0x7fd24c0a9e50 2024-01-24T13:58:40.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.220+0000 7fd2522ce700 1 --2- 172.21.15.29:0/354879334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd24c0a9e50 0x7fd24c063fd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fd248009760 tx=0x7fd248009060).ready entity=mon.1 client_cookie=4f52167b6f3d0a29 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:40.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.220+0000 7fd2512cc700 1 -- 172.21.15.29:0/354879334 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24800f510 con 0x7fd24c0a9e50 2024-01-24T13:58:40.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.221+0000 7fd2512cc700 1 -- 172.21.15.29:0/354879334 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24800f670 con 0x7fd24c0a9e50 2024-01-24T13:58:40.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.221+0000 7fd2512cc700 1 -- 172.21.15.29:0/354879334 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24800f9f0 con 0x7fd24c0a9e50 2024-01-24T13:58:40.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.221+0000 7fd254532700 1 -- 172.21.15.29:0/354879334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd24c0a9e50 msgr2=0x7fd24c063fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.221+0000 7fd254532700 1 --2- 172.21.15.29:0/354879334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd24c0a9e50 0x7fd24c063fd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fd248009760 tx=0x7fd248009060).stop 2024-01-24T13:58:40.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.221+0000 7fd254532700 1 -- 172.21.15.29:0/354879334 shutdown_connections 2024-01-24T13:58:40.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.221+0000 7fd254532700 1 --2- 172.21.15.29:0/354879334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd24c0a9e50 0x7fd24c063fd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.221+0000 7fd254532700 1 -- 172.21.15.29:0/354879334 >> 172.21.15.29:0/354879334 conn(0x7fd24c0c5090 msgr2=0x7fd24c0638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:40.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.221+0000 7fd254532700 1 -- 172.21.15.29:0/354879334 shutdown_connections 2024-01-24T13:58:40.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.221+0000 7fd254532700 1 -- 172.21.15.29:0/354879334 wait complete. 2024-01-24T13:58:40.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.223+0000 7fd254532700 1 Processor -- start 2024-01-24T13:58:40.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.223+0000 7fd254532700 1 -- start start 2024-01-24T13:58:40.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd254532700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd24c0627b0 0x7fd24c0c17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd254532700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd24c0678c0 0x7fd24c0a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd254532700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd24c0a9e50 0x7fd24c162150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd254532700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0bf250 con 0x7fd24c0627b0 2024-01-24T13:58:40.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd254532700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0be960 con 0x7fd24c0678c0 2024-01-24T13:58:40.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd254532700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0a1690 con 0x7fd24c0a9e50 2024-01-24T13:58:40.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd2522ce700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd24c0627b0 0x7fd24c0c17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd2522ce700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd24c0627b0 0x7fd24c0c17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41434/0 (socket says 172.21.15.29:41434) 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd2522ce700 1 -- 172.21.15.29:0/3617564254 learned_addr learned my addr 172.21.15.29:0/3617564254 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd252acf700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd24c0a9e50 0x7fd24c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd251acd700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd24c0678c0 0x7fd24c0a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd2522ce700 1 -- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd24c0a9e50 msgr2=0x7fd24c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd2522ce700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd24c0a9e50 0x7fd24c162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd2522ce700 1 -- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd24c0678c0 msgr2=0x7fd24c0a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd2522ce700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd24c0678c0 0x7fd24c0a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd2522ce700 1 -- 172.21.15.29:0/3617564254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2480092a0 con 0x7fd24c0627b0 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd252acf700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd24c0a9e50 0x7fd24c162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd251acd700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd24c0678c0 0x7fd24c0a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd2522ce700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd24c0627b0 0x7fd24c0c17f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fd248009e00 tx=0x7fd248017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd2427fc700 1 -- 172.21.15.29:0/3617564254 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24800f7e0 con 0x7fd24c0627b0 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd2427fc700 1 -- 172.21.15.29:0/3617564254 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24800f940 con 0x7fd24c0627b0 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd2427fc700 1 -- 172.21.15.29:0/3617564254 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24801b690 con 0x7fd24c0627b0 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.224+0000 7fd254532700 1 -- 172.21.15.29:0/3617564254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c0bec00 con 0x7fd24c0627b0 2024-01-24T13:58:40.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.225+0000 7fd254532700 1 -- 172.21.15.29:0/3617564254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c0a1320 con 0x7fd24c0627b0 2024-01-24T13:58:40.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.226+0000 7fd254532700 1 -- 172.21.15.29:0/3617564254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24c0475e0 con 0x7fd24c0627b0 2024-01-24T13:58:40.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.226+0000 7fd2427fc700 1 -- 172.21.15.29:0/3617564254 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd24801ba40 con 0x7fd24c0627b0 2024-01-24T13:58:40.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.226+0000 7fd2427fc700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd23005f150 0x7fd230061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.226+0000 7fd2427fc700 1 -- 172.21.15.29:0/3617564254 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fd24807bb90 con 0x7fd24c0627b0 2024-01-24T13:58:40.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.226+0000 7fd251acd700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd23005f150 0x7fd230061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.227+0000 7fd251acd700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd23005f150 0x7fd230061610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fd24c0c2190 tx=0x7fd23c009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:40.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.231+0000 7fd2427fc700 1 -- 172.21.15.29:0/3617564254 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd24804cb50 con 0x7fd24c0627b0 2024-01-24T13:58:40.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.455+0000 7fd254532700 1 -- 172.21.15.29:0/3617564254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd24c0bf530 con 0x7fd24c0627b0 2024-01-24T13:58:40.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.455+0000 7fd2427fc700 1 -- 172.21.15.29:0/3617564254 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7fd248050170 con 0x7fd24c0627b0 2024-01-24T13:58:40.456 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:40.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.457+0000 7fd254532700 1 -- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd23005f150 msgr2=0x7fd230061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.457+0000 7fd254532700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd23005f150 0x7fd230061610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fd24c0c2190 tx=0x7fd23c009040).stop 2024-01-24T13:58:40.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.457+0000 7fd254532700 1 -- 172.21.15.29:0/3617564254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd24c0627b0 msgr2=0x7fd24c0c17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.457+0000 7fd254532700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd24c0627b0 0x7fd24c0c17f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fd248009e00 tx=0x7fd248017000).stop 2024-01-24T13:58:40.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.457+0000 7fd254532700 1 -- 172.21.15.29:0/3617564254 shutdown_connections 2024-01-24T13:58:40.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.457+0000 7fd254532700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd24c0a9e50 0x7fd24c162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.457+0000 7fd254532700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd23005f150 0x7fd230061610 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.457+0000 7fd254532700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd24c0627b0 0x7fd24c0c17f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.457+0000 7fd254532700 1 --2- 172.21.15.29:0/3617564254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd24c0678c0 0x7fd24c0a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.457+0000 7fd254532700 1 -- 172.21.15.29:0/3617564254 >> 172.21.15.29:0/3617564254 conn(0x7fd24c0c5090 msgr2=0x7fd24c0bd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:40.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.457+0000 7fd254532700 1 -- 172.21.15.29:0/3617564254 shutdown_connections 2024-01-24T13:58:40.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.457+0000 7fd254532700 1 -- 172.21.15.29:0/3617564254 wait complete. 2024-01-24T13:58:40.471 INFO:teuthology.orchestra.run.smithi029.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-24T13:58:40.471 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-24T13:58:40.471 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:40.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.600+0000 7ffacee60700 1 Processor -- start 2024-01-24T13:58:40.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.601+0000 7ffacee60700 1 -- start start 2024-01-24T13:58:40.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.601+0000 7ffacee60700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffac8154370 0x7ffac80c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.601+0000 7ffacee60700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffac80aa0e0 0x7ffac8154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffacee60700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac80aa6c0 con 0x7ffac80ab1f0 2024-01-24T13:58:40.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffacee60700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac8064240 con 0x7ffac80aa0e0 2024-01-24T13:58:40.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffacee60700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac80cb5c0 con 0x7ffac8154370 2024-01-24T13:58:40.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffac7fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffac80aa0e0 0x7ffac8154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffacd3fd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ffac80ab1f0 0x7ffac80631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42216/0 (socket says 172.21.15.29:42216) 2024-01-24T13:58:40.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffacd3fd700 1 -- 172.21.15.29:0/1560782306 learned_addr learned my addr 172.21.15.29:0/1560782306 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:40.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffaccbfc700 1 --2- 172.21.15.29:0/1560782306 >> v2:172.21.15.163:3300/0 conn(0x7ffac8154370 0x7ffac80c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffac7fff700 1 -- 172.21.15.29:0/1560782306 >> v2:172.21.15.163:3300/0 conn(0x7ffac8154370 msgr2=0x7ffac80c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:40.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffac7fff700 1 --2- 172.21.15.29:0/1560782306 >> v2:172.21.15.163:3300/0 conn(0x7ffac8154370 0x7ffac80c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffac7fff700 1 -- 172.21.15.29:0/1560782306 >> v1:172.21.15.29:6789/0 conn(0x7ffac80ab1f0 legacy=0x7ffac80631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffac7fff700 1 -- 172.21.15.29:0/1560782306 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac80cb8a0 con 0x7ffac80aa0e0 2024-01-24T13:58:40.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffac77fe700 1 -- 172.21.15.29:0/1560782306 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789452177 0 0) 0x7ffac80aa6c0 con 0x7ffac80ab1f0 2024-01-24T13:58:40.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffac7fff700 1 --2- 172.21.15.29:0/1560782306 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffac80aa0e0 0x7ffac8154750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7ffac8084050 tx=0x7ffab4009060).ready entity=mon.0 client_cookie=ef578971faa6d898 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:40.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffac77fe700 1 -- 172.21.15.29:0/1560782306 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab400f510 con 0x7ffac80aa0e0 2024-01-24T13:58:40.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffac77fe700 1 -- 172.21.15.29:0/1560782306 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab400f670 con 0x7ffac80aa0e0 2024-01-24T13:58:40.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.602+0000 7ffac77fe700 1 -- 172.21.15.29:0/1560782306 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab400f9f0 con 0x7ffac80aa0e0 2024-01-24T13:58:40.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.603+0000 7ffacee60700 1 -- 172.21.15.29:0/1560782306 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffac80aa0e0 msgr2=0x7ffac8154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.603+0000 7ffacee60700 1 --2- 172.21.15.29:0/1560782306 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffac80aa0e0 0x7ffac8154750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7ffac8084050 tx=0x7ffab4009060).stop 2024-01-24T13:58:40.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.603+0000 7ffacee60700 1 -- 172.21.15.29:0/1560782306 shutdown_connections 2024-01-24T13:58:40.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.603+0000 7ffacee60700 1 --2- 172.21.15.29:0/1560782306 >> v2:172.21.15.163:3300/0 conn(0x7ffac8154370 0x7ffac80c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.603+0000 7ffacee60700 1 --2- 172.21.15.29:0/1560782306 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffac80aa0e0 0x7ffac8154750 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.603+0000 7ffacee60700 1 -- 172.21.15.29:0/1560782306 >> 172.21.15.29:0/1560782306 conn(0x7ffac805ec80 msgr2=0x7ffac80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:40.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.603+0000 7ffacee60700 1 -- 172.21.15.29:0/1560782306 shutdown_connections 2024-01-24T13:58:40.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.603+0000 7ffacee60700 1 -- 172.21.15.29:0/1560782306 wait complete. 2024-01-24T13:58:40.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.604+0000 7ffacee60700 1 Processor -- start 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.605+0000 7ffacee60700 1 -- start start 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.605+0000 7ffacee60700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffac80aa0e0 0x7ffac8140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.605+0000 7ffacee60700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffac80ab1f0 0x7ffac814cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.605+0000 7ffacee60700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffac8154370 0x7ffac8154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.605+0000 7ffacee60700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8150800 con 0x7ffac8154370 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.605+0000 7ffacee60700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac814fbf0 con 0x7ffac80aa0e0 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.605+0000 7ffacee60700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac814fed0 con 0x7ffac80ab1f0 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffacd3fd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffac8154370 0x7ffac8154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffacd3fd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffac8154370 0x7ffac8154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41458/0 (socket says 172.21.15.29:41458) 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffacd3fd700 1 -- 172.21.15.29:0/957416114 learned_addr learned my addr 172.21.15.29:0/957416114 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffac7fff700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffac80ab1f0 0x7ffac814cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffaccbfc700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffac80aa0e0 0x7ffac8140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffacd3fd700 1 -- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffac80ab1f0 msgr2=0x7ffac814cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffacd3fd700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffac80ab1f0 0x7ffac814cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffacd3fd700 1 -- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffac80aa0e0 msgr2=0x7ffac8140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffacd3fd700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffac80aa0e0 0x7ffac8140980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffacd3fd700 1 -- 172.21.15.29:0/957416114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab40092a0 con 0x7ffac8154370 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffacd3fd700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffac8154370 0x7ffac8154ca0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7ffac000c7e0 tx=0x7ffac0007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffac4ff9700 1 -- 172.21.15.29:0/957416114 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0005d30 con 0x7ffac8154370 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffac4ff9700 1 -- 172.21.15.29:0/957416114 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac0005e90 con 0x7ffac8154370 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffac4ff9700 1 -- 172.21.15.29:0/957416114 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0007320 con 0x7ffac8154370 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffacee60700 1 -- 172.21.15.29:0/957416114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac81501b0 con 0x7ffac8154370 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.606+0000 7ffacee60700 1 -- 172.21.15.29:0/957416114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac80c4d00 con 0x7ffac8154370 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.607+0000 7ffacee60700 1 -- 172.21.15.29:0/957416114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac8150970 con 0x7ffac8154370 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.608+0000 7ffac4ff9700 1 -- 172.21.15.29:0/957416114 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ffac0007710 con 0x7ffac8154370 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.608+0000 7ffac4ff9700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffaac05f150 0x7ffaac061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.608+0000 7ffac4ff9700 1 -- 172.21.15.29:0/957416114 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7ffac007a590 con 0x7ffac8154370 2024-01-24T13:58:40.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.608+0000 7ffaccbfc700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffaac05f150 0x7ffaac061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.609+0000 7ffaccbfc700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffaac05f150 0x7ffaac061610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7ffabc009200 tx=0x7ffabc008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:40.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.613+0000 7ffac4ff9700 1 -- 172.21.15.29:0/957416114 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ffac004b550 con 0x7ffac8154370 2024-01-24T13:58:40.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.798+0000 7ffacee60700 1 -- 172.21.15.29:0/957416114 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffac80475e0 con 0x7ffac8154370 2024-01-24T13:58:40.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.799+0000 7ffac4ff9700 1 -- 172.21.15.29:0/957416114 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7ffac004eb70 con 0x7ffac8154370 2024-01-24T13:58:40.799 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:40.800 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:40.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.801+0000 7ffacee60700 1 -- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffaac05f150 msgr2=0x7ffaac061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.801+0000 7ffacee60700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffaac05f150 0x7ffaac061610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7ffabc009200 tx=0x7ffabc008040).stop 2024-01-24T13:58:40.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.801+0000 7ffacee60700 1 -- 172.21.15.29:0/957416114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffac8154370 msgr2=0x7ffac8154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.801+0000 7ffacee60700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffac8154370 0x7ffac8154ca0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7ffac000c7e0 tx=0x7ffac0007f90).stop 2024-01-24T13:58:40.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.801+0000 7ffacee60700 1 -- 172.21.15.29:0/957416114 shutdown_connections 2024-01-24T13:58:40.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.801+0000 7ffacee60700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffaac05f150 0x7ffaac061610 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.801+0000 7ffacee60700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffac8154370 0x7ffac8154ca0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.801+0000 7ffacee60700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffac80aa0e0 0x7ffac8140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.801+0000 7ffacee60700 1 --2- 172.21.15.29:0/957416114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffac80ab1f0 0x7ffac814cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.801+0000 7ffacee60700 1 -- 172.21.15.29:0/957416114 >> 172.21.15.29:0/957416114 conn(0x7ffac805ec80 msgr2=0x7ffac814d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:40.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.801+0000 7ffacee60700 1 -- 172.21.15.29:0/957416114 shutdown_connections 2024-01-24T13:58:40.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.801+0000 7ffacee60700 1 -- 172.21.15.29:0/957416114 wait complete. 2024-01-24T13:58:40.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T13:58:16.949817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T13:58:18.174460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-24T13:58:21.232882+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T13:58:40.815 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T13:58:40.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.949+0000 7fe8f7e46700 1 Processor -- start 2024-01-24T13:58:40.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.949+0000 7fe8f7e46700 1 -- start start 2024-01-24T13:58:40.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.950+0000 7fe8f7e46700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0134ea0 0x7fe8f00c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.950+0000 7fe8f7e46700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f00686a0 con 0x7fe8f00c36c0 2024-01-24T13:58:40.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.950+0000 7fe8f7e46700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f0068b30 con 0x7fe8f0066bd0 2024-01-24T13:58:40.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.950+0000 7fe8f7e46700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f0067a90 con 0x7fe8f0134ea0 2024-01-24T13:58:40.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.950+0000 7fe8f53e1700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe8f00c36c0 0x7fe8f00664c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42224/0 (socket says 172.21.15.29:42224) 2024-01-24T13:58:40.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.950+0000 7fe8f53e1700 1 -- 172.21.15.29:0/4130024743 learned_addr learned my addr 172.21.15.29:0/4130024743 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:40.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.950+0000 7fe8f5be2700 1 --2- 172.21.15.29:0/4130024743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0134ea0 0x7fe8f00c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.951+0000 7fe8f4be0700 1 -- 172.21.15.29:0/4130024743 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795301404 0 0) 0x7fe8f00686a0 con 0x7fe8f00c36c0 2024-01-24T13:58:40.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.951+0000 7fe8f4be0700 1 -- 172.21.15.29:0/4130024743 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8dc0033f0 con 0x7fe8f00c36c0 2024-01-24T13:58:40.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.951+0000 7fe8f4be0700 1 -- 172.21.15.29:0/4130024743 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289604088 0 0) 0x7fe8f0068b30 con 0x7fe8f0066bd0 2024-01-24T13:58:40.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.951+0000 7fe8f4be0700 1 -- 172.21.15.29:0/4130024743 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f00686a0 con 0x7fe8f0066bd0 2024-01-24T13:58:40.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.951+0000 7fe8f4be0700 1 -- 172.21.15.29:0/4130024743 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4202446277 0 0) 0x7fe8dc0033f0 con 0x7fe8f00c36c0 2024-01-24T13:58:40.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.951+0000 7fe8f5be2700 1 -- 172.21.15.29:0/4130024743 >> v1:172.21.15.163:6789/0 conn(0x7fe8f0066bd0 legacy=0x7fe8f00bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.951+0000 7fe8f5be2700 1 -- 172.21.15.29:0/4130024743 >> v1:172.21.15.29:6789/0 conn(0x7fe8f00c36c0 legacy=0x7fe8f00664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.951+0000 7fe8f5be2700 1 -- 172.21.15.29:0/4130024743 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f0067d70 con 0x7fe8f0134ea0 2024-01-24T13:58:40.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.951+0000 7fe8f5be2700 1 --2- 172.21.15.29:0/4130024743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0134ea0 0x7fe8f00c31d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe8e4009790 tx=0x7fe8e4009060).ready entity=mon.2 client_cookie=b6374dddf8d7e445 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:40.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.951+0000 7fe8f4be0700 1 -- 172.21.15.29:0/4130024743 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec001470 con 0x7fe8f0134ea0 2024-01-24T13:58:40.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.951+0000 7fe8f4be0700 1 -- 172.21.15.29:0/4130024743 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e4011400 con 0x7fe8f0134ea0 2024-01-24T13:58:40.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.951+0000 7fe8f4be0700 1 -- 172.21.15.29:0/4130024743 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4011780 con 0x7fe8f0134ea0 2024-01-24T13:58:40.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.952+0000 7fe8f7e46700 1 -- 172.21.15.29:0/4130024743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0134ea0 msgr2=0x7fe8f00c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.952+0000 7fe8f7e46700 1 --2- 172.21.15.29:0/4130024743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0134ea0 0x7fe8f00c31d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe8e4009790 tx=0x7fe8e4009060).stop 2024-01-24T13:58:40.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.952+0000 7fe8f7e46700 1 -- 172.21.15.29:0/4130024743 shutdown_connections 2024-01-24T13:58:40.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.952+0000 7fe8f7e46700 1 --2- 172.21.15.29:0/4130024743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0134ea0 0x7fe8f00c31d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.952+0000 7fe8f7e46700 1 -- 172.21.15.29:0/4130024743 >> 172.21.15.29:0/4130024743 conn(0x7fe8f00c8bd0 msgr2=0x7fe8f00c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:40.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.952+0000 7fe8f7e46700 1 -- 172.21.15.29:0/4130024743 shutdown_connections 2024-01-24T13:58:40.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.952+0000 7fe8f7e46700 1 -- 172.21.15.29:0/4130024743 wait complete. 2024-01-24T13:58:40.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.954+0000 7fe8f7e46700 1 Processor -- start 2024-01-24T13:58:40.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.954+0000 7fe8f7e46700 1 -- start start 2024-01-24T13:58:40.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.955+0000 7fe8f7e46700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0066bd0 0x7fe8f00ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.955+0000 7fe8f7e46700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f00c36c0 0x7fe8f00ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.955+0000 7fe8f7e46700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f0134ea0 0x7fe8f00abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.955+0000 7fe8f7e46700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00aa8a0 con 0x7fe8f00c36c0 2024-01-24T13:58:40.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.955+0000 7fe8f7e46700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00a9540 con 0x7fe8f0134ea0 2024-01-24T13:58:40.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.955+0000 7fe8f7e46700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00a9800 con 0x7fe8f0066bd0 2024-01-24T13:58:40.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.955+0000 7fe8f53e1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f00c36c0 0x7fe8f00ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8f53e1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f00c36c0 0x7fe8f00ae750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41474/0 (socket says 172.21.15.29:41474) 2024-01-24T13:58:40.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8f53e1700 1 -- 172.21.15.29:0/2831966245 learned_addr learned my addr 172.21.15.29:0/2831966245 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8f5be2700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0066bd0 0x7fe8f00ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8f63e3700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f0134ea0 0x7fe8f00abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8f53e1700 1 -- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0066bd0 msgr2=0x7fe8f00ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8f53e1700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0066bd0 0x7fe8f00ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8f53e1700 1 -- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f0134ea0 msgr2=0x7fe8f00abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8f53e1700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f0134ea0 0x7fe8f00abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8f53e1700 1 -- 172.21.15.29:0/2831966245 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e40092a0 con 0x7fe8f00c36c0 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8f63e3700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f0134ea0 0x7fe8f00abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8f53e1700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f00c36c0 0x7fe8f00ae750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fe8ec0011e0 tx=0x7fe8ec001a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8e1ffb700 1 -- 172.21.15.29:0/2831966245 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0072e0 con 0x7fe8f00c36c0 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8e1ffb700 1 -- 172.21.15.29:0/2831966245 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec007440 con 0x7fe8f00c36c0 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8f7e46700 1 -- 172.21.15.29:0/2831966245 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f00a9ae0 con 0x7fe8f00c36c0 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.956+0000 7fe8e1ffb700 1 -- 172.21.15.29:0/2831966245 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0057a0 con 0x7fe8f00c36c0 2024-01-24T13:58:40.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.957+0000 7fe8f7e46700 1 -- 172.21.15.29:0/2831966245 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f00aa0a0 con 0x7fe8f00c36c0 2024-01-24T13:58:40.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.958+0000 7fe8f7e46700 1 -- 172.21.15.29:0/2831966245 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f00475e0 con 0x7fe8f00c36c0 2024-01-24T13:58:40.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.958+0000 7fe8e1ffb700 1 -- 172.21.15.29:0/2831966245 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe8ec005b50 con 0x7fe8f00c36c0 2024-01-24T13:58:40.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.958+0000 7fe8e1ffb700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe8d405f150 0x7fe8d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:40.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.959+0000 7fe8e1ffb700 1 -- 172.21.15.29:0/2831966245 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe8ec079f80 con 0x7fe8f00c36c0 2024-01-24T13:58:40.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.959+0000 7fe8f5be2700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe8d405f150 0x7fe8d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:40.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.959+0000 7fe8f5be2700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe8d405f150 0x7fe8d4061610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fe8e40096e0 tx=0x7fe8e4016000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:40.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:40.963+0000 7fe8e1ffb700 1 -- 172.21.15.29:0/2831966245 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe8ec04ae80 con 0x7fe8f00c36c0 2024-01-24T13:58:41.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.144+0000 7fe8f7e46700 1 -- 172.21.15.29:0/2831966245 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe8f0068050 con 0x7fe8f00c36c0 2024-01-24T13:58:41.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.145+0000 7fe8e1ffb700 1 -- 172.21.15.29:0/2831966245 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe8ec04e4a0 con 0x7fe8f00c36c0 2024-01-24T13:58:41.145 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T13:58:41.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.147+0000 7fe8f7e46700 1 -- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe8d405f150 msgr2=0x7fe8d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.147+0000 7fe8f7e46700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe8d405f150 0x7fe8d4061610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fe8e40096e0 tx=0x7fe8e4016000).stop 2024-01-24T13:58:41.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.147+0000 7fe8f7e46700 1 -- 172.21.15.29:0/2831966245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f00c36c0 msgr2=0x7fe8f00ae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.147+0000 7fe8f7e46700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f00c36c0 0x7fe8f00ae750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fe8ec0011e0 tx=0x7fe8ec001a20).stop 2024-01-24T13:58:41.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.147+0000 7fe8f7e46700 1 -- 172.21.15.29:0/2831966245 shutdown_connections 2024-01-24T13:58:41.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.147+0000 7fe8f7e46700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f0134ea0 0x7fe8f00abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.147+0000 7fe8f7e46700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0066bd0 0x7fe8f00ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.147+0000 7fe8f7e46700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe8d405f150 0x7fe8d4061610 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.147+0000 7fe8f7e46700 1 --2- 172.21.15.29:0/2831966245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f00c36c0 0x7fe8f00ae750 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.147+0000 7fe8f7e46700 1 -- 172.21.15.29:0/2831966245 >> 172.21.15.29:0/2831966245 conn(0x7fe8f00c8bd0 msgr2=0x7fe8f0152490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:41.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.147+0000 7fe8f7e46700 1 -- 172.21.15.29:0/2831966245 shutdown_connections 2024-01-24T13:58:41.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.147+0000 7fe8f7e46700 1 -- 172.21.15.29:0/2831966245 wait complete. 2024-01-24T13:58:41.161 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T13:58:41.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.295+0000 7fa00d4ba700 1 Processor -- start 2024-01-24T13:58:41.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.295+0000 7fa00d4ba700 1 -- start start 2024-01-24T13:58:41.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.296+0000 7fa00d4ba700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0080c8490 0x7fa0080a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.296+0000 7fa00d4ba700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa0080bfd40 0x7fa0080c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.296+0000 7fa00d4ba700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa008152240 con 0x7fa008064760 2024-01-24T13:58:41.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.296+0000 7fa00d4ba700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0081586c0 con 0x7fa0080bfd40 2024-01-24T13:58:41.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.296+0000 7fa00d4ba700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0080aa670 con 0x7fa0080c8490 2024-01-24T13:58:41.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.296+0000 7fa0077fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa0080bfd40 0x7fa0080c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.296+0000 7fa006ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa008064760 0x7fa008064b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42240/0 (socket says 172.21.15.29:42240) 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.296+0000 7fa006ffd700 1 -- 172.21.15.29:0/12812822 learned_addr learned my addr 172.21.15.29:0/12812822 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.296+0000 7fa0077fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa0080bfd40 0x7fa0080c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41488/0 (socket says 172.21.15.29:41488) 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.296+0000 7fa0067fc700 1 --2- 172.21.15.29:0/12812822 >> v2:172.21.15.163:3300/0 conn(0x7fa0080c8490 0x7fa0080a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.297+0000 7fa0077fe700 1 -- 172.21.15.29:0/12812822 >> v2:172.21.15.163:3300/0 conn(0x7fa0080c8490 msgr2=0x7fa0080a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.297+0000 7fa0077fe700 1 --2- 172.21.15.29:0/12812822 >> v2:172.21.15.163:3300/0 conn(0x7fa0080c8490 0x7fa0080a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.297+0000 7fa0077fe700 1 -- 172.21.15.29:0/12812822 >> v1:172.21.15.29:6789/0 conn(0x7fa008064760 legacy=0x7fa008064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.297+0000 7fa0077fe700 1 -- 172.21.15.29:0/12812822 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa008157d90 con 0x7fa0080bfd40 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.297+0000 7fa005ffb700 1 -- 172.21.15.29:0/12812822 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506044104 0 0) 0x7fa008152240 con 0x7fa008064760 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.297+0000 7fa0077fe700 1 --2- 172.21.15.29:0/12812822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0080bfd40 0x7fa0080c01d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fa0080c83d0 tx=0x7fa000009780).ready entity=mon.0 client_cookie=f57647e3fafe41d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.297+0000 7fa005ffb700 1 -- 172.21.15.29:0/12812822 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00000e070 con 0x7fa0080bfd40 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.297+0000 7fa005ffb700 1 -- 172.21.15.29:0/12812822 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00000bbf0 con 0x7fa0080bfd40 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.297+0000 7fa005ffb700 1 -- 172.21.15.29:0/12812822 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000012410 con 0x7fa0080bfd40 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.297+0000 7fa00d4ba700 1 -- 172.21.15.29:0/12812822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0080bfd40 msgr2=0x7fa0080c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.297+0000 7fa00d4ba700 1 --2- 172.21.15.29:0/12812822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0080bfd40 0x7fa0080c01d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fa0080c83d0 tx=0x7fa000009780).stop 2024-01-24T13:58:41.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.298+0000 7fa00d4ba700 1 -- 172.21.15.29:0/12812822 shutdown_connections 2024-01-24T13:58:41.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.298+0000 7fa00d4ba700 1 --2- 172.21.15.29:0/12812822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0080bfd40 0x7fa0080c01d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.298+0000 7fa00d4ba700 1 --2- 172.21.15.29:0/12812822 >> v2:172.21.15.163:3300/0 conn(0x7fa0080c8490 0x7fa0080a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.298+0000 7fa00d4ba700 1 -- 172.21.15.29:0/12812822 >> 172.21.15.29:0/12812822 conn(0x7fa0080cb8b0 msgr2=0x7fa0080c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:41.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.298+0000 7fa00d4ba700 1 -- 172.21.15.29:0/12812822 shutdown_connections 2024-01-24T13:58:41.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.298+0000 7fa00d4ba700 1 -- 172.21.15.29:0/12812822 wait complete. 2024-01-24T13:58:41.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.299+0000 7fa00d4ba700 1 Processor -- start 2024-01-24T13:58:41.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.299+0000 7fa00d4ba700 1 -- start start 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa00d4ba700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa008064760 0x7fa00814fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa00d4ba700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0080bfd40 0x7fa008153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa00d4ba700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0080c8490 0x7fa0080c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa00d4ba700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00814ffb0 con 0x7fa008064760 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa00d4ba700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0080c34f0 con 0x7fa0080bfd40 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa00d4ba700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa008153720 con 0x7fa0080c8490 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa006ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa008064760 0x7fa00814fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa006ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa008064760 0x7fa00814fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41496/0 (socket says 172.21.15.29:41496) 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa006ffd700 1 -- 172.21.15.29:0/1540843148 learned_addr learned my addr 172.21.15.29:0/1540843148 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa0067fc700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0080bfd40 0x7fa008153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa0077fe700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0080c8490 0x7fa0080c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa006ffd700 1 -- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0080c8490 msgr2=0x7fa0080c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa006ffd700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0080c8490 0x7fa0080c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa006ffd700 1 -- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0080bfd40 msgr2=0x7fa008153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa006ffd700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0080bfd40 0x7fa008153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa006ffd700 1 -- 172.21.15.29:0/1540843148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa000009a20 con 0x7fa008064760 2024-01-24T13:58:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.300+0000 7fa0077fe700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0080c8490 0x7fa0080c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.301+0000 7fa006ffd700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa008064760 0x7fa00814fac0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f9ff800c4e0 tx=0x7f9ff8007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.301+0000 7f9feeffd700 1 -- 172.21.15.29:0/1540843148 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff80072e0 con 0x7fa008064760 2024-01-24T13:58:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.301+0000 7f9feeffd700 1 -- 172.21.15.29:0/1540843148 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff8007440 con 0x7fa008064760 2024-01-24T13:58:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.301+0000 7f9feeffd700 1 -- 172.21.15.29:0/1540843148 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff80057a0 con 0x7fa008064760 2024-01-24T13:58:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.301+0000 7fa00d4ba700 1 -- 172.21.15.29:0/1540843148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0080c2f30 con 0x7fa008064760 2024-01-24T13:58:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.301+0000 7fa00d4ba700 1 -- 172.21.15.29:0/1540843148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0081533b0 con 0x7fa008064760 2024-01-24T13:58:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.302+0000 7fa00d4ba700 1 -- 172.21.15.29:0/1540843148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa008153210 con 0x7fa008064760 2024-01-24T13:58:41.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.302+0000 7f9feeffd700 1 -- 172.21.15.29:0/1540843148 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9ff8005bd0 con 0x7fa008064760 2024-01-24T13:58:41.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.303+0000 7f9feeffd700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9ff405f150 0x7f9ff4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.303+0000 7f9feeffd700 1 -- 172.21.15.29:0/1540843148 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f9ff8079df0 con 0x7fa008064760 2024-01-24T13:58:41.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.303+0000 7fa0067fc700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9ff405f150 0x7f9ff4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.304+0000 7fa0067fc700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9ff405f150 0x7f9ff4061610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fa0081678c0 tx=0x7f9ffc008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:41.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.307+0000 7f9feeffd700 1 -- 172.21.15.29:0/1540843148 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa008153210 con 0x7fa008064760 2024-01-24T13:58:41.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.487+0000 7fa00d4ba700 1 -- 172.21.15.29:0/1540843148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa008153210 con 0x7fa008064760 2024-01-24T13:58:41.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.488+0000 7f9feeffd700 1 -- 172.21.15.29:0/1540843148 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa008153210 con 0x7fa008064760 2024-01-24T13:58:41.488 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T13:58:41.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.490+0000 7fa00d4ba700 1 -- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9ff405f150 msgr2=0x7f9ff4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.490+0000 7fa00d4ba700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9ff405f150 0x7f9ff4061610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fa0081678c0 tx=0x7f9ffc008040).stop 2024-01-24T13:58:41.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.490+0000 7fa00d4ba700 1 -- 172.21.15.29:0/1540843148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa008064760 msgr2=0x7fa00814fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.490+0000 7fa00d4ba700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa008064760 0x7fa00814fac0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f9ff800c4e0 tx=0x7f9ff8007c70).stop 2024-01-24T13:58:41.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.490+0000 7fa00d4ba700 1 -- 172.21.15.29:0/1540843148 shutdown_connections 2024-01-24T13:58:41.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.490+0000 7fa00d4ba700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0080c8490 0x7fa0080c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.490+0000 7fa00d4ba700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9ff405f150 0x7f9ff4061610 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.490+0000 7fa00d4ba700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa008064760 0x7fa00814fac0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.490+0000 7fa00d4ba700 1 --2- 172.21.15.29:0/1540843148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0080bfd40 0x7fa008153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.490+0000 7fa00d4ba700 1 -- 172.21.15.29:0/1540843148 >> 172.21.15.29:0/1540843148 conn(0x7fa0080cb8b0 msgr2=0x7fa008157460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:41.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.490+0000 7fa00d4ba700 1 -- 172.21.15.29:0/1540843148 shutdown_connections 2024-01-24T13:58:41.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.490+0000 7fa00d4ba700 1 -- 172.21.15.29:0/1540843148 wait complete. 2024-01-24T13:58:41.504 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T13:58:41.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.636+0000 7f50ba6aa700 1 Processor -- start 2024-01-24T13:58:41.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.636+0000 7f50ba6aa700 1 -- start start 2024-01-24T13:58:41.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.637+0000 7f50ba6aa700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f50b41599f0 0x7f50b415bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.637+0000 7f50ba6aa700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b40c3e70 con 0x7f50b40692a0 2024-01-24T13:58:41.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.637+0000 7f50ba6aa700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b40c4300 con 0x7f50b4069680 2024-01-24T13:58:41.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.637+0000 7f50ba6aa700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b40c3260 con 0x7f50b41599f0 2024-01-24T13:58:41.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.637+0000 7f50b3fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f50b40692a0 0x7f50b4065e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42246/0 (socket says 172.21.15.29:42246) 2024-01-24T13:58:41.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.637+0000 7f50b8c47700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f50b41599f0 0x7f50b415bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.637+0000 7f50b3fff700 1 -- 172.21.15.29:0/3832257057 learned_addr learned my addr 172.21.15.29:0/3832257057 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:41.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.638+0000 7f50b8c47700 1 -- 172.21.15.29:0/3832257057 >> v1:172.21.15.163:6789/0 conn(0x7f50b4069680 legacy=0x7f50b4069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:41.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.638+0000 7f50b8c47700 1 -- 172.21.15.29:0/3832257057 >> v1:172.21.15.29:6789/0 conn(0x7f50b40692a0 legacy=0x7f50b4065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.638+0000 7f50b2ffd700 1 -- 172.21.15.29:0/3832257057 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666031851 0 0) 0x7f50b40c3e70 con 0x7f50b40692a0 2024-01-24T13:58:41.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.638+0000 7f50b8c47700 1 -- 172.21.15.29:0/3832257057 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b40c3540 con 0x7f50b41599f0 2024-01-24T13:58:41.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.638+0000 7f50b8c47700 1 --2- 172.21.15.29:0/3832257057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50b41599f0 0x7f50b415bde0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f50ac009e60 tx=0x7f50ac009780).ready entity=mon.0 client_cookie=63478ffd515fc436 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:41.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.638+0000 7f50b2ffd700 1 -- 172.21.15.29:0/3832257057 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac00e070 con 0x7f50b41599f0 2024-01-24T13:58:41.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.638+0000 7f50b2ffd700 1 -- 172.21.15.29:0/3832257057 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ac00bc30 con 0x7f50b41599f0 2024-01-24T13:58:41.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.638+0000 7f50b2ffd700 1 -- 172.21.15.29:0/3832257057 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac012410 con 0x7f50b41599f0 2024-01-24T13:58:41.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.639+0000 7f50ba6aa700 1 -- 172.21.15.29:0/3832257057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50b41599f0 msgr2=0x7f50b415bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.639+0000 7f50ba6aa700 1 --2- 172.21.15.29:0/3832257057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50b41599f0 0x7f50b415bde0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f50ac009e60 tx=0x7f50ac009780).stop 2024-01-24T13:58:41.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.639+0000 7f50ba6aa700 1 -- 172.21.15.29:0/3832257057 shutdown_connections 2024-01-24T13:58:41.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.639+0000 7f50ba6aa700 1 --2- 172.21.15.29:0/3832257057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50b41599f0 0x7f50b415bde0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.639+0000 7f50ba6aa700 1 -- 172.21.15.29:0/3832257057 >> 172.21.15.29:0/3832257057 conn(0x7f50b4064390 msgr2=0x7f50b40c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:41.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.639+0000 7f50ba6aa700 1 -- 172.21.15.29:0/3832257057 shutdown_connections 2024-01-24T13:58:41.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.639+0000 7f50ba6aa700 1 -- 172.21.15.29:0/3832257057 wait complete. 2024-01-24T13:58:41.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.640+0000 7f50ba6aa700 1 Processor -- start 2024-01-24T13:58:41.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.640+0000 7f50ba6aa700 1 -- start start 2024-01-24T13:58:41.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50ba6aa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50b40692a0 0x7f50b4153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50ba6aa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50b4069680 0x7f50b40c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50ba6aa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50b41599f0 0x7f50b4162060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50ba6aa700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4161950 con 0x7f50b40692a0 2024-01-24T13:58:41.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50ba6aa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4161bd0 con 0x7f50b4069680 2024-01-24T13:58:41.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50ba6aa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b40aafa0 con 0x7f50b41599f0 2024-01-24T13:58:41.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50b3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50b40692a0 0x7f50b4153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50b3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50b40692a0 0x7f50b4153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41524/0 (socket says 172.21.15.29:41524) 2024-01-24T13:58:41.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50b3fff700 1 -- 172.21.15.29:0/2146275818 learned_addr learned my addr 172.21.15.29:0/2146275818 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:41.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50b37fe700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50b4069680 0x7f50b40c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50b8c47700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50b41599f0 0x7f50b4162060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50b3fff700 1 -- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50b41599f0 msgr2=0x7f50b4162060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50b3fff700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50b41599f0 0x7f50b4162060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50b3fff700 1 -- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50b4069680 msgr2=0x7f50b40c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50b3fff700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50b4069680 0x7f50b40c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.641+0000 7f50b3fff700 1 -- 172.21.15.29:0/2146275818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac009a20 con 0x7f50b40692a0 2024-01-24T13:58:41.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.642+0000 7f50b3fff700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50b40692a0 0x7f50b4153b10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f50a400c4c0 tx=0x7f50a4007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:41.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.642+0000 7f509bfff700 1 -- 172.21.15.29:0/2146275818 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a400ce30 con 0x7f50b40692a0 2024-01-24T13:58:41.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.642+0000 7f509bfff700 1 -- 172.21.15.29:0/2146275818 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a40072e0 con 0x7f50b40692a0 2024-01-24T13:58:41.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.642+0000 7f50ba6aa700 1 -- 172.21.15.29:0/2146275818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b40ab280 con 0x7f50b40692a0 2024-01-24T13:58:41.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.642+0000 7f509bfff700 1 -- 172.21.15.29:0/2146275818 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a40057a0 con 0x7f50b40692a0 2024-01-24T13:58:41.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.642+0000 7f50ba6aa700 1 -- 172.21.15.29:0/2146275818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b4154000 con 0x7f50b40692a0 2024-01-24T13:58:41.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.643+0000 7f50ba6aa700 1 -- 172.21.15.29:0/2146275818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b40ab7a0 con 0x7f50b40692a0 2024-01-24T13:58:41.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.643+0000 7f509bfff700 1 -- 172.21.15.29:0/2146275818 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f50a4005bd0 con 0x7f50b40692a0 2024-01-24T13:58:41.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.644+0000 7f509bfff700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f50a005f120 0x7f50a00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.644+0000 7f509bfff700 1 -- 172.21.15.29:0/2146275818 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f50a4079b50 con 0x7f50b40692a0 2024-01-24T13:58:41.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.644+0000 7f50b37fe700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f50a005f120 0x7f50a00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.644+0000 7f50b37fe700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f50a005f120 0x7f50a00615e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f50b413a510 tx=0x7f50a8007d40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:41.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.648+0000 7f509bfff700 1 -- 172.21.15.29:0/2146275818 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f50b40ab7a0 con 0x7f50b40692a0 2024-01-24T13:58:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.828+0000 7f50ba6aa700 1 -- 172.21.15.29:0/2146275818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f50b40ab7a0 con 0x7f50b40692a0 2024-01-24T13:58:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.828+0000 7f509bfff700 1 -- 172.21.15.29:0/2146275818 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f50b40ab7a0 con 0x7f50b40692a0 2024-01-24T13:58:41.828 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:58:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.830+0000 7f50ba6aa700 1 -- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f50a005f120 msgr2=0x7f50a00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.830+0000 7f50ba6aa700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f50a005f120 0x7f50a00615e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f50b413a510 tx=0x7f50a8007d40).stop 2024-01-24T13:58:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.830+0000 7f50ba6aa700 1 -- 172.21.15.29:0/2146275818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50b40692a0 msgr2=0x7f50b4153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.830+0000 7f50ba6aa700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50b40692a0 0x7f50b4153b10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f50a400c4c0 tx=0x7f50a4007c10).stop 2024-01-24T13:58:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.830+0000 7f50ba6aa700 1 -- 172.21.15.29:0/2146275818 shutdown_connections 2024-01-24T13:58:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.830+0000 7f50ba6aa700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50b41599f0 0x7f50b4162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.830+0000 7f50ba6aa700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f50a005f120 0x7f50a00615e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.830+0000 7f50ba6aa700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50b40692a0 0x7f50b4153b10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.830+0000 7f50ba6aa700 1 --2- 172.21.15.29:0/2146275818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50b4069680 0x7f50b40c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.830+0000 7f50ba6aa700 1 -- 172.21.15.29:0/2146275818 >> 172.21.15.29:0/2146275818 conn(0x7f50b4064390 msgr2=0x7f50b4069ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:41.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.830+0000 7f50ba6aa700 1 -- 172.21.15.29:0/2146275818 shutdown_connections 2024-01-24T13:58:41.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.830+0000 7f50ba6aa700 1 -- 172.21.15.29:0/2146275818 wait complete. 2024-01-24T13:58:41.844 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T13:58:41.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.976+0000 7f56ca1ff700 1 Processor -- start 2024-01-24T13:58:41.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.977+0000 7f56ca1ff700 1 -- start start 2024-01-24T13:58:41.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.977+0000 7f56ca1ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f56c4062ec0 0x7f56c406a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.977+0000 7f56ca1ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c40c8270 0x7f56c41391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.977+0000 7f56ca1ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c40bcda0 con 0x7f56c41384f0 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.977+0000 7f56ca1ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c4065f80 con 0x7f56c4062ec0 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56ca1ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4066260 con 0x7f56c40c8270 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56c3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c40c8270 0x7f56c41391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56c2ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f56c4062ec0 0x7f56c406a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56c3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c40c8270 0x7f56c41391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:40672/0 (socket says 172.21.15.29:40672) 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56c3fff700 1 -- 172.21.15.29:0/2237961054 learned_addr learned my addr 172.21.15.29:0/2237961054 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56c27fc700 1 -- 172.21.15.29:0/2237961054 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593066782 0 0) 0x7f56c40bcda0 con 0x7f56c41384f0 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56c27fc700 1 -- 172.21.15.29:0/2237961054 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56ac0033f0 con 0x7f56c41384f0 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56c3fff700 1 -- 172.21.15.29:0/2237961054 >> v2:172.21.15.163:3300/0 conn(0x7f56c4062ec0 msgr2=0x7f56c406a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56c3fff700 1 --2- 172.21.15.29:0/2237961054 >> v2:172.21.15.163:3300/0 conn(0x7f56c4062ec0 0x7f56c406a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56c3fff700 1 -- 172.21.15.29:0/2237961054 >> v1:172.21.15.163:6789/0 conn(0x7f56c41384f0 legacy=0x7f56c40627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56c3fff700 1 -- 172.21.15.29:0/2237961054 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c40bca50 con 0x7f56c40c8270 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56c2ffd700 1 --2- 172.21.15.29:0/2237961054 >> v2:172.21.15.163:3300/0 conn(0x7f56c4062ec0 0x7f56c406a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.978+0000 7f56c3fff700 1 --2- 172.21.15.29:0/2237961054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c40c8270 0x7f56c41391d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f56bc009f20 tx=0x7f56bc009840).ready entity=mon.2 client_cookie=abb7eff9c2fac7f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.979+0000 7f56c27fc700 1 -- 172.21.15.29:0/2237961054 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00e070 con 0x7f56c40c8270 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.979+0000 7f56c27fc700 1 -- 172.21.15.29:0/2237961054 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc00b970 con 0x7f56c40c8270 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.979+0000 7f56c27fc700 1 -- 172.21.15.29:0/2237961054 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00bcf0 con 0x7f56c40c8270 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.979+0000 7f56ca1ff700 1 -- 172.21.15.29:0/2237961054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c40c8270 msgr2=0x7f56c41391d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.979+0000 7f56ca1ff700 1 --2- 172.21.15.29:0/2237961054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c40c8270 0x7f56c41391d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f56bc009f20 tx=0x7f56bc009840).stop 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.979+0000 7f56ca1ff700 1 -- 172.21.15.29:0/2237961054 shutdown_connections 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.979+0000 7f56ca1ff700 1 --2- 172.21.15.29:0/2237961054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c40c8270 0x7f56c41391d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.979+0000 7f56ca1ff700 1 --2- 172.21.15.29:0/2237961054 >> v2:172.21.15.163:3300/0 conn(0x7f56c4062ec0 0x7f56c406a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.979+0000 7f56ca1ff700 1 -- 172.21.15.29:0/2237961054 >> 172.21.15.29:0/2237961054 conn(0x7f56c4064390 msgr2=0x7f56c40c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.979+0000 7f56ca1ff700 1 -- 172.21.15.29:0/2237961054 shutdown_connections 2024-01-24T13:58:41.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.979+0000 7f56ca1ff700 1 -- 172.21.15.29:0/2237961054 wait complete. 2024-01-24T13:58:41.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.981+0000 7f56ca1ff700 1 Processor -- start 2024-01-24T13:58:41.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.981+0000 7f56ca1ff700 1 -- start start 2024-01-24T13:58:41.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.982+0000 7f56ca1ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c4062ec0 0x7f56c4156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.982+0000 7f56ca1ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c40c8270 0x7f56c4162ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.982+0000 7f56ca1ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c41384f0 0x7f56c40692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.982+0000 7f56ca1ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c40ab6a0 con 0x7f56c4062ec0 2024-01-24T13:58:41.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.982+0000 7f56ca1ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c40abf40 con 0x7f56c41384f0 2024-01-24T13:58:41.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.982+0000 7f56ca1ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4162710 con 0x7f56c40c8270 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.982+0000 7f56c37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c4062ec0 0x7f56c4156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.982+0000 7f56c37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c4062ec0 0x7f56c4156fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41538/0 (socket says 172.21.15.29:41538) 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.982+0000 7f56c37fe700 1 -- 172.21.15.29:0/4152216172 learned_addr learned my addr 172.21.15.29:0/4152216172 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.982+0000 7f56c2ffd700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c40c8270 0x7f56c4162ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56c3fff700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c41384f0 0x7f56c40692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56c37fe700 1 -- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c40c8270 msgr2=0x7f56c4162ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56c37fe700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c40c8270 0x7f56c4162ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56c37fe700 1 -- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c41384f0 msgr2=0x7f56c40692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56c37fe700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c41384f0 0x7f56c40692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56c37fe700 1 -- 172.21.15.29:0/4152216172 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80092a0 con 0x7f56c4062ec0 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56c3fff700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c41384f0 0x7f56c40692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56c37fe700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c4062ec0 0x7f56c4156fb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f56b400c440 tx=0x7f56b4007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56ab7fe700 1 -- 172.21.15.29:0/4152216172 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b400cea0 con 0x7f56c4062ec0 2024-01-24T13:58:41.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56ab7fe700 1 -- 172.21.15.29:0/4152216172 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b400f470 con 0x7f56c4062ec0 2024-01-24T13:58:41.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56ca1ff700 1 -- 172.21.15.29:0/4152216172 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc009ae0 con 0x7f56c4062ec0 2024-01-24T13:58:41.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56ab7fe700 1 -- 172.21.15.29:0/4152216172 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b400f830 con 0x7f56c4062ec0 2024-01-24T13:58:41.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.983+0000 7f56ca1ff700 1 -- 172.21.15.29:0/4152216172 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c40abbd0 con 0x7f56c4062ec0 2024-01-24T13:58:41.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.984+0000 7f56ca1ff700 1 -- 172.21.15.29:0/4152216172 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c40475e0 con 0x7f56c4062ec0 2024-01-24T13:58:41.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.985+0000 7f56ab7fe700 1 -- 172.21.15.29:0/4152216172 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f56b400fbe0 con 0x7f56c4062ec0 2024-01-24T13:58:41.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.985+0000 7f56ab7fe700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56b005f150 0x7f56b0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:41.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.985+0000 7f56ab7fe700 1 -- 172.21.15.29:0/4152216172 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f56b40799c0 con 0x7f56c4062ec0 2024-01-24T13:58:41.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.985+0000 7f56c2ffd700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56b005f150 0x7f56b0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:41.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.986+0000 7f56c2ffd700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56b005f150 0x7f56b0061610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f56c40ad540 tx=0x7f56b8009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:41.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:41.990+0000 7f56ab7fe700 1 -- 172.21.15.29:0/4152216172 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f56b404a980 con 0x7f56c4062ec0 2024-01-24T13:58:42.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.170+0000 7f56ca1ff700 1 -- 172.21.15.29:0/4152216172 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f56c40c6ad0 con 0x7f56c4062ec0 2024-01-24T13:58:42.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.171+0000 7f56ab7fe700 1 -- 172.21.15.29:0/4152216172 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f56b404dfa0 con 0x7f56c4062ec0 2024-01-24T13:58:42.171 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:58:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.172+0000 7f56ca1ff700 1 -- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56b005f150 msgr2=0x7f56b0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.172+0000 7f56ca1ff700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56b005f150 0x7f56b0061610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f56c40ad540 tx=0x7f56b8009060).stop 2024-01-24T13:58:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.172+0000 7f56ca1ff700 1 -- 172.21.15.29:0/4152216172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c4062ec0 msgr2=0x7f56c4156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.172+0000 7f56ca1ff700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c4062ec0 0x7f56c4156fb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f56b400c440 tx=0x7f56b4007c90).stop 2024-01-24T13:58:42.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.172+0000 7f56ca1ff700 1 -- 172.21.15.29:0/4152216172 shutdown_connections 2024-01-24T13:58:42.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.173+0000 7f56ca1ff700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c41384f0 0x7f56c40692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.173+0000 7f56ca1ff700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56b005f150 0x7f56b0061610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.173+0000 7f56ca1ff700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c4062ec0 0x7f56c4156fb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.173+0000 7f56ca1ff700 1 --2- 172.21.15.29:0/4152216172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c40c8270 0x7f56c4162ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.173+0000 7f56ca1ff700 1 -- 172.21.15.29:0/4152216172 >> 172.21.15.29:0/4152216172 conn(0x7f56c4064390 msgr2=0x7f56c41374b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:42.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.173+0000 7f56ca1ff700 1 -- 172.21.15.29:0/4152216172 shutdown_connections 2024-01-24T13:58:42.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.173+0000 7f56ca1ff700 1 -- 172.21.15.29:0/4152216172 wait complete. 2024-01-24T13:58:42.186 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-24T13:58:42.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.318+0000 7f0e98305700 1 Processor -- start 2024-01-24T13:58:42.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.318+0000 7f0e98305700 1 -- start start 2024-01-24T13:58:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e98305700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e900664c0 0x7f0e90066950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e98305700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e900bdb30 0x7f0e900bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e98305700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e90066e40 con 0x7f0e9013a5e0 2024-01-24T13:58:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e98305700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e900640d0 con 0x7f0e900664c0 2024-01-24T13:58:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e98305700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e900679f0 con 0x7f0e900bdb30 2024-01-24T13:58:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e958a0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e900664c0 0x7f0e90066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e958a0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e900664c0 0x7f0e90066950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41552/0 (socket says 172.21.15.29:41552) 2024-01-24T13:58:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e960a1700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0e9013a5e0 0x7f0e900639c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42248/0 (socket says 172.21.15.29:42248) 2024-01-24T13:58:42.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e958a0700 1 -- 172.21.15.29:0/1309305412 learned_addr learned my addr 172.21.15.29:0/1309305412 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e968a2700 1 --2- 172.21.15.29:0/1309305412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e900bdb30 0x7f0e900bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e958a0700 1 -- 172.21.15.29:0/1309305412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e900bdb30 msgr2=0x7f0e900bdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e958a0700 1 --2- 172.21.15.29:0/1309305412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e900bdb30 0x7f0e900bdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e958a0700 1 -- 172.21.15.29:0/1309305412 >> v1:172.21.15.29:6789/0 conn(0x7f0e9013a5e0 legacy=0x7f0e900639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e958a0700 1 -- 172.21.15.29:0/1309305412 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e90067cd0 con 0x7f0e900664c0 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.319+0000 7f0e968a2700 1 --2- 172.21.15.29:0/1309305412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e900bdb30 0x7f0e900bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.320+0000 7f0e958a0700 1 --2- 172.21.15.29:0/1309305412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e900664c0 0x7f0e90066950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f0e8c006fd0 tx=0x7f0e8c0090f0).ready entity=mon.0 client_cookie=6977d3fc4e1eef45 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.320+0000 7f0e9509f700 1 -- 172.21.15.29:0/1309305412 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c00b600 con 0x7f0e900664c0 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.320+0000 7f0e9509f700 1 -- 172.21.15.29:0/1309305412 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c00b760 con 0x7f0e900664c0 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.320+0000 7f0e9509f700 1 -- 172.21.15.29:0/1309305412 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c00bae0 con 0x7f0e900664c0 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.320+0000 7f0e98305700 1 -- 172.21.15.29:0/1309305412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e900664c0 msgr2=0x7f0e90066950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.320+0000 7f0e98305700 1 --2- 172.21.15.29:0/1309305412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e900664c0 0x7f0e90066950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f0e8c006fd0 tx=0x7f0e8c0090f0).stop 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.320+0000 7f0e98305700 1 -- 172.21.15.29:0/1309305412 shutdown_connections 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.320+0000 7f0e98305700 1 --2- 172.21.15.29:0/1309305412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e900bdb30 0x7f0e900bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.320+0000 7f0e98305700 1 --2- 172.21.15.29:0/1309305412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e900664c0 0x7f0e90066950 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.320+0000 7f0e98305700 1 -- 172.21.15.29:0/1309305412 >> 172.21.15.29:0/1309305412 conn(0x7f0e900c8bd0 msgr2=0x7f0e900c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.320+0000 7f0e98305700 1 -- 172.21.15.29:0/1309305412 shutdown_connections 2024-01-24T13:58:42.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.320+0000 7f0e98305700 1 -- 172.21.15.29:0/1309305412 wait complete. 2024-01-24T13:58:42.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.322+0000 7f0e98305700 1 Processor -- start 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.322+0000 7f0e98305700 1 -- start start 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.323+0000 7f0e98305700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e900664c0 0x7f0e90138400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.323+0000 7f0e98305700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e900bdb30 0x7f0e900aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.323+0000 7f0e98305700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e9013a5e0 0x7f0e900abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.323+0000 7f0e98305700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e90168930 con 0x7f0e900664c0 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.323+0000 7f0e98305700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e901388f0 con 0x7f0e900bdb30 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.323+0000 7f0e98305700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e90167cb0 con 0x7f0e9013a5e0 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.323+0000 7f0e960a1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e900664c0 0x7f0e90138400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.323+0000 7f0e960a1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e900664c0 0x7f0e90138400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41566/0 (socket says 172.21.15.29:41566) 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.323+0000 7f0e960a1700 1 -- 172.21.15.29:0/2474607237 learned_addr learned my addr 172.21.15.29:0/2474607237 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e968a2700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e9013a5e0 0x7f0e900abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e958a0700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e900bdb30 0x7f0e900aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e960a1700 1 -- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e9013a5e0 msgr2=0x7f0e900abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e960a1700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e9013a5e0 0x7f0e900abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e960a1700 1 -- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e900bdb30 msgr2=0x7f0e900aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e960a1700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e900bdb30 0x7f0e900aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e960a1700 1 -- 172.21.15.29:0/2474607237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c009370 con 0x7f0e900664c0 2024-01-24T13:58:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e958a0700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e900bdb30 0x7f0e900aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e960a1700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e900664c0 0x7f0e90138400 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f0e84007fd0 tx=0x7f0e84007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e827fc700 1 -- 172.21.15.29:0/2474607237 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e840072e0 con 0x7f0e900664c0 2024-01-24T13:58:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e827fc700 1 -- 172.21.15.29:0/2474607237 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e84007440 con 0x7f0e900664c0 2024-01-24T13:58:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e827fc700 1 -- 172.21.15.29:0/2474607237 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e840058f0 con 0x7f0e900664c0 2024-01-24T13:58:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e98305700 1 -- 172.21.15.29:0/2474607237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e90167f90 con 0x7f0e900664c0 2024-01-24T13:58:42.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.324+0000 7f0e98305700 1 -- 172.21.15.29:0/2474607237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e90168550 con 0x7f0e900664c0 2024-01-24T13:58:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.325+0000 7f0e98305700 1 -- 172.21.15.29:0/2474607237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e900475e0 con 0x7f0e900664c0 2024-01-24T13:58:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.325+0000 7f0e827fc700 1 -- 172.21.15.29:0/2474607237 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0e84005d20 con 0x7f0e900664c0 2024-01-24T13:58:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.326+0000 7f0e827fc700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0e7405f150 0x7f0e74061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.326+0000 7f0e827fc700 1 -- 172.21.15.29:0/2474607237 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f0e84079c50 con 0x7f0e900664c0 2024-01-24T13:58:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.326+0000 7f0e958a0700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0e7405f150 0x7f0e74061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:42.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.326+0000 7f0e958a0700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0e7405f150 0x7f0e74061610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0e8c009130 tx=0x7f0e8c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:42.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.330+0000 7f0e827fc700 1 -- 172.21.15.29:0/2474607237 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0e8407f020 con 0x7f0e900664c0 2024-01-24T13:58:42.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.509+0000 7f0e98305700 1 -- 172.21.15.29:0/2474607237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0e90067fb0 con 0x7f0e900664c0 2024-01-24T13:58:42.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.509+0000 7f0e827fc700 1 -- 172.21.15.29:0/2474607237 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0e8404ac10 con 0x7f0e900664c0 2024-01-24T13:58:42.510 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-01-24T13:58:42.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.511+0000 7f0e98305700 1 -- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0e7405f150 msgr2=0x7f0e74061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.511+0000 7f0e98305700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0e7405f150 0x7f0e74061610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0e8c009130 tx=0x7f0e8c017000).stop 2024-01-24T13:58:42.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.511+0000 7f0e98305700 1 -- 172.21.15.29:0/2474607237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e900664c0 msgr2=0x7f0e90138400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.511+0000 7f0e98305700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e900664c0 0x7f0e90138400 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f0e84007fd0 tx=0x7f0e84007df0).stop 2024-01-24T13:58:42.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.511+0000 7f0e98305700 1 -- 172.21.15.29:0/2474607237 shutdown_connections 2024-01-24T13:58:42.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.511+0000 7f0e98305700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e9013a5e0 0x7f0e900abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.511+0000 7f0e98305700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0e7405f150 0x7f0e74061610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.511+0000 7f0e98305700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e900664c0 0x7f0e90138400 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.511+0000 7f0e98305700 1 --2- 172.21.15.29:0/2474607237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e900bdb30 0x7f0e900aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.511+0000 7f0e98305700 1 -- 172.21.15.29:0/2474607237 >> 172.21.15.29:0/2474607237 conn(0x7f0e900c8bd0 msgr2=0x7f0e90155a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:42.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.512+0000 7f0e98305700 1 -- 172.21.15.29:0/2474607237 shutdown_connections 2024-01-24T13:58:42.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.512+0000 7f0e98305700 1 -- 172.21.15.29:0/2474607237 wait complete. 2024-01-24T13:58:42.525 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:58:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.654+0000 7f8492653700 1 Processor -- start 2024-01-24T13:58:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.654+0000 7f8492653700 1 -- start start 2024-01-24T13:58:42.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.655+0000 7f8492653700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f848c0c4a10 0x7f848c0bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.655+0000 7f8492653700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f848c0670a0 0x7f848c158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.655+0000 7f8492653700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848c1582b0 con 0x7f848c0664f0 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.655+0000 7f8492653700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848c157ca0 con 0x7f848c0c4a10 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.655+0000 7f8492653700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c063c60 con 0x7f848c0670a0 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.655+0000 7f848bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f848c0c4a10 0x7f848c0bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.656+0000 7f848bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f848c0c4a10 0x7f848c0bf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41582/0 (socket says 172.21.15.29:41582) 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.656+0000 7f848bfff700 1 -- 172.21.15.29:0/3704809054 learned_addr learned my addr 172.21.15.29:0/3704809054 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.656+0000 7f848b7fe700 1 --2- 172.21.15.29:0/3704809054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f848c0670a0 0x7f848c158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.656+0000 7f848bfff700 1 -- 172.21.15.29:0/3704809054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f848c0670a0 msgr2=0x7f848c158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.656+0000 7f848bfff700 1 --2- 172.21.15.29:0/3704809054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f848c0670a0 0x7f848c158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.656+0000 7f848bfff700 1 -- 172.21.15.29:0/3704809054 >> v1:172.21.15.163:6789/0 conn(0x7f848c0664f0 legacy=0x7f848c0bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.656+0000 7f848bfff700 1 -- 172.21.15.29:0/3704809054 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c157f40 con 0x7f848c0c4a10 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.656+0000 7f848b7fe700 1 --2- 172.21.15.29:0/3704809054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f848c0670a0 0x7f848c158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.656+0000 7f848bfff700 1 --2- 172.21.15.29:0/3704809054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c0c4a10 0x7f848c0bf6d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f847c006fd0 tx=0x7f847c0090f0).ready entity=mon.0 client_cookie=1bc72ddcf40bb707 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:42.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.656+0000 7f848affd700 1 -- 172.21.15.29:0/3704809054 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c00b600 con 0x7f848c0c4a10 2024-01-24T13:58:42.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.656+0000 7f848affd700 1 -- 172.21.15.29:0/3704809054 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c00b760 con 0x7f848c0c4a10 2024-01-24T13:58:42.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.656+0000 7f848affd700 1 -- 172.21.15.29:0/3704809054 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c00bae0 con 0x7f848c0c4a10 2024-01-24T13:58:42.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.657+0000 7f8492653700 1 -- 172.21.15.29:0/3704809054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c0c4a10 msgr2=0x7f848c0bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.657+0000 7f8492653700 1 --2- 172.21.15.29:0/3704809054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c0c4a10 0x7f848c0bf6d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f847c006fd0 tx=0x7f847c0090f0).stop 2024-01-24T13:58:42.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.657+0000 7f8492653700 1 -- 172.21.15.29:0/3704809054 shutdown_connections 2024-01-24T13:58:42.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.657+0000 7f8492653700 1 --2- 172.21.15.29:0/3704809054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c0c4a10 0x7f848c0bf6d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.657+0000 7f8492653700 1 --2- 172.21.15.29:0/3704809054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f848c0670a0 0x7f848c158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.657+0000 7f8492653700 1 -- 172.21.15.29:0/3704809054 >> 172.21.15.29:0/3704809054 conn(0x7f848c0c8bb0 msgr2=0x7f848c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:42.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.657+0000 7f8492653700 1 -- 172.21.15.29:0/3704809054 shutdown_connections 2024-01-24T13:58:42.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.657+0000 7f8492653700 1 -- 172.21.15.29:0/3704809054 wait complete. 2024-01-24T13:58:42.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.658+0000 7f8492653700 1 Processor -- start 2024-01-24T13:58:42.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.659+0000 7f8492653700 1 -- start start 2024-01-24T13:58:42.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.659+0000 7f8492653700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f848c0664f0 0x7f848c14ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:42.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.659+0000 7f8492653700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f848c0670a0 0x7f848c0aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:42.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.659+0000 7f8492653700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c0c4a10 0x7f848c0ab060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:42.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.659+0000 7f8492653700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0ab550 con 0x7f848c0c4a10 2024-01-24T13:58:42.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.659+0000 7f8492653700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0c1730 con 0x7f848c0670a0 2024-01-24T13:58:42.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.659+0000 7f8492653700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0c08d0 con 0x7f848c0664f0 2024-01-24T13:58:42.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.659+0000 7f8490bf0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c0c4a10 0x7f848c0ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:42.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.659+0000 7f848bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f848c0664f0 0x7f848c14ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8490bf0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c0c4a10 0x7f848c0ab060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41594/0 (socket says 172.21.15.29:41594) 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8490bf0700 1 -- 172.21.15.29:0/1780395167 learned_addr learned my addr 172.21.15.29:0/1780395167 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f848b7fe700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f848c0670a0 0x7f848c0aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8490bf0700 1 -- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f848c0664f0 msgr2=0x7f848c14ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8490bf0700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f848c0664f0 0x7f848c14ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8490bf0700 1 -- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f848c0670a0 msgr2=0x7f848c0aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8490bf0700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f848c0670a0 0x7f848c0aab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8490bf0700 1 -- 172.21.15.29:0/1780395167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c009370 con 0x7f848c0c4a10 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8490bf0700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c0c4a10 0x7f848c0ab060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f8484002230 tx=0x7f84840015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8483fff700 1 -- 172.21.15.29:0/1780395167 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400e4a0 con 0x7f848c0c4a10 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8483fff700 1 -- 172.21.15.29:0/1780395167 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848400e600 con 0x7f848c0c4a10 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8483fff700 1 -- 172.21.15.29:0/1780395167 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400e980 con 0x7f848c0c4a10 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8492653700 1 -- 172.21.15.29:0/1780395167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c0c0bb0 con 0x7f848c0c4a10 2024-01-24T13:58:42.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.660+0000 7f8492653700 1 -- 172.21.15.29:0/1780395167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c14b0f0 con 0x7f848c0c4a10 2024-01-24T13:58:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.661+0000 7f8492653700 1 -- 172.21.15.29:0/1780395167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848c0c10d0 con 0x7f848c0c4a10 2024-01-24T13:58:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.662+0000 7f8483fff700 1 -- 172.21.15.29:0/1780395167 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f848400ed30 con 0x7f848c0c4a10 2024-01-24T13:58:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.662+0000 7f8483fff700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f846c05f150 0x7f846c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.663+0000 7f848bfff700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f846c05f150 0x7f846c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:42.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.663+0000 7f8483fff700 1 -- 172.21.15.29:0/1780395167 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f848404c990 con 0x7f848c0c4a10 2024-01-24T13:58:42.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.663+0000 7f848bfff700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f846c05f150 0x7f846c061610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f847c009130 tx=0x7f847c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:42.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.668+0000 7f8483fff700 1 -- 172.21.15.29:0/1780395167 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8484048b00 con 0x7f848c0c4a10 2024-01-24T13:58:42.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.899+0000 7f8492653700 1 -- 172.21.15.29:0/1780395167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f848c14b340 con 0x7f848c0c4a10 2024-01-24T13:58:42.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.900+0000 7f8483fff700 1 -- 172.21.15.29:0/1780395167 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4491 (secure 0 0 0) 0x7f848c0c10d0 con 0x7f848c0c4a10 2024-01-24T13:58:42.900 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:42.900 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2038321939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2038321939},{"type":"v1","addr":"172.21.15.163:6836","nonce":2038321939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3675672874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3675672874},{"type":"v1","addr":"172.21.15.163:6837","nonce":3675672874}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1532435957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532435957},{"type":"v1","addr":"172.21.15.29:6834","nonce":1532435957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:58:19.190134+0000","modified":"2024-01-24T13:58:31.231527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1328130656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1328130656},{"type":"v1","addr":"172.21.15.29:6835","nonce":1328130656}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-24T13:58:42.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.902+0000 7f8492653700 1 -- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f846c05f150 msgr2=0x7f846c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.902+0000 7f8492653700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f846c05f150 0x7f846c061610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f847c009130 tx=0x7f847c017000).stop 2024-01-24T13:58:42.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.902+0000 7f8492653700 1 -- 172.21.15.29:0/1780395167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c0c4a10 msgr2=0x7f848c0ab060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:42.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.902+0000 7f8492653700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c0c4a10 0x7f848c0ab060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f8484002230 tx=0x7f84840015e0).stop 2024-01-24T13:58:42.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.902+0000 7f8492653700 1 -- 172.21.15.29:0/1780395167 shutdown_connections 2024-01-24T13:58:42.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.902+0000 7f8492653700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f846c05f150 0x7f846c061610 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.902+0000 7f8492653700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f848c0c4a10 0x7f848c0ab060 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.902+0000 7f8492653700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f848c0664f0 0x7f848c14ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.902+0000 7f8492653700 1 --2- 172.21.15.29:0/1780395167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f848c0670a0 0x7f848c0aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:42.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.902+0000 7f8492653700 1 -- 172.21.15.29:0/1780395167 >> 172.21.15.29:0/1780395167 conn(0x7f848c0c8bb0 msgr2=0x7f848c0c6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:42.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.902+0000 7f8492653700 1 -- 172.21.15.29:0/1780395167 shutdown_connections 2024-01-24T13:58:42.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:42.902+0000 7f8492653700 1 -- 172.21.15.29:0/1780395167 wait complete. 2024-01-24T13:58:42.903 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 9 2024-01-24T13:58:42.916 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:43.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.045+0000 7fd22e94c700 1 Processor -- start 2024-01-24T13:58:43.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.045+0000 7fd22e94c700 1 -- start start 2024-01-24T13:58:43.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.046+0000 7fd22e94c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd228066b40 0x7fd228062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.046+0000 7fd22e94c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd228158600 0x7fd228154d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.046+0000 7fd22e94c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2280bcf60 con 0x7fd228062970 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.046+0000 7fd22e94c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2280bdba0 con 0x7fd228158600 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.046+0000 7fd22e94c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2280a7480 con 0x7fd228066b40 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.047+0000 7fd22cee9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd228158600 0x7fd228154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.047+0000 7fd22cee9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd228158600 0x7fd228154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41600/0 (socket says 172.21.15.29:41600) 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.047+0000 7fd22cee9700 1 -- 172.21.15.29:0/2848910988 learned_addr learned my addr 172.21.15.29:0/2848910988 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.047+0000 7fd2277fe700 1 --2- 172.21.15.29:0/2848910988 >> v2:172.21.15.163:3300/0 conn(0x7fd228066b40 0x7fd228062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.047+0000 7fd22cee9700 1 -- 172.21.15.29:0/2848910988 >> v2:172.21.15.163:3300/0 conn(0x7fd228066b40 msgr2=0x7fd228062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.047+0000 7fd22cee9700 1 --2- 172.21.15.29:0/2848910988 >> v2:172.21.15.163:3300/0 conn(0x7fd228066b40 0x7fd228062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.047+0000 7fd22cee9700 1 -- 172.21.15.29:0/2848910988 >> v1:172.21.15.163:6789/0 conn(0x7fd228062970 legacy=0x7fd2280c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.047+0000 7fd22cee9700 1 -- 172.21.15.29:0/2848910988 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2280a6ec0 con 0x7fd228158600 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.047+0000 7fd22cee9700 1 --2- 172.21.15.29:0/2848910988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd228158600 0x7fd228154d90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fd2280a75e0 tx=0x7fd220009840).ready entity=mon.0 client_cookie=2a3488ed154290c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.047+0000 7fd226ffd700 1 -- 172.21.15.29:0/2848910988 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22000e070 con 0x7fd228158600 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.047+0000 7fd226ffd700 1 -- 172.21.15.29:0/2848910988 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22000ba20 con 0x7fd228158600 2024-01-24T13:58:43.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.047+0000 7fd226ffd700 1 -- 172.21.15.29:0/2848910988 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22000bda0 con 0x7fd228158600 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.048+0000 7fd22e94c700 1 -- 172.21.15.29:0/2848910988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd228158600 msgr2=0x7fd228154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.048+0000 7fd22e94c700 1 --2- 172.21.15.29:0/2848910988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd228158600 0x7fd228154d90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fd2280a75e0 tx=0x7fd220009840).stop 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.048+0000 7fd22e94c700 1 -- 172.21.15.29:0/2848910988 shutdown_connections 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.048+0000 7fd22e94c700 1 --2- 172.21.15.29:0/2848910988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd228158600 0x7fd228154d90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.048+0000 7fd22e94c700 1 --2- 172.21.15.29:0/2848910988 >> v2:172.21.15.163:3300/0 conn(0x7fd228066b40 0x7fd228062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.048+0000 7fd22e94c700 1 -- 172.21.15.29:0/2848910988 >> 172.21.15.29:0/2848910988 conn(0x7fd2280c8580 msgr2=0x7fd2280c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.048+0000 7fd22e94c700 1 -- 172.21.15.29:0/2848910988 shutdown_connections 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.048+0000 7fd22e94c700 1 -- 172.21.15.29:0/2848910988 wait complete. 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.049+0000 7fd22e94c700 1 Processor -- start 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.049+0000 7fd22e94c700 1 -- start start 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.050+0000 7fd22e94c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228062970 0x7fd22814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.050+0000 7fd22e94c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd228066b40 0x7fd228065360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.050+0000 7fd22e94c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd228158600 0x7fd228065850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.050+0000 7fd22e94c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd228065d40 con 0x7fd228066b40 2024-01-24T13:58:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.050+0000 7fd22e94c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd228164850 con 0x7fd228062970 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.050+0000 7fd22e94c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd228156380 con 0x7fd228158600 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.050+0000 7fd2277fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd228066b40 0x7fd228065360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.050+0000 7fd2277fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd228066b40 0x7fd228065360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41604/0 (socket says 172.21.15.29:41604) 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.050+0000 7fd2277fe700 1 -- 172.21.15.29:0/889564886 learned_addr learned my addr 172.21.15.29:0/889564886 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.050+0000 7fd22cee9700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd228158600 0x7fd228065850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.050+0000 7fd227fff700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228062970 0x7fd22814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd2277fe700 1 -- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd228158600 msgr2=0x7fd228065850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd2277fe700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd228158600 0x7fd228065850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd2277fe700 1 -- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228062970 msgr2=0x7fd22814ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd2277fe700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228062970 0x7fd22814ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd2277fe700 1 -- 172.21.15.29:0/889564886 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd220009ae0 con 0x7fd228066b40 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd22cee9700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd228158600 0x7fd228065850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd227fff700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228062970 0x7fd22814ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd2277fe700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd228066b40 0x7fd228065360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fd21000e930 tx=0x7fd210009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:43.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd21ffff700 1 -- 172.21.15.29:0/889564886 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21000eed0 con 0x7fd228066b40 2024-01-24T13:58:43.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd21ffff700 1 -- 172.21.15.29:0/889564886 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21000c850 con 0x7fd228066b40 2024-01-24T13:58:43.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd22e94c700 1 -- 172.21.15.29:0/889564886 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2281560a0 con 0x7fd228066b40 2024-01-24T13:58:43.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd21ffff700 1 -- 172.21.15.29:0/889564886 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21000cbd0 con 0x7fd228066b40 2024-01-24T13:58:43.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.051+0000 7fd22e94c700 1 -- 172.21.15.29:0/889564886 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2281644e0 con 0x7fd228066b40 2024-01-24T13:58:43.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.052+0000 7fd22e94c700 1 -- 172.21.15.29:0/889564886 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2280475e0 con 0x7fd228066b40 2024-01-24T13:58:43.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.052+0000 7fd21ffff700 1 -- 172.21.15.29:0/889564886 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd21001b400 con 0x7fd228066b40 2024-01-24T13:58:43.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.053+0000 7fd21ffff700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd20805f180 0x7fd208061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.053+0000 7fd21ffff700 1 -- 172.21.15.29:0/889564886 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fd21007a9f0 con 0x7fd228066b40 2024-01-24T13:58:43.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.053+0000 7fd227fff700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd20805f180 0x7fd208061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.054+0000 7fd227fff700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd20805f180 0x7fd208061640 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fd218001690 tx=0x7fd218009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:43.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.057+0000 7fd21ffff700 1 -- 172.21.15.29:0/889564886 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd21004b950 con 0x7fd228066b40 2024-01-24T13:58:43.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.242+0000 7fd22e94c700 1 -- 172.21.15.29:0/889564886 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2280c5700 con 0x7fd228066b40 2024-01-24T13:58:43.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.243+0000 7fd21ffff700 1 -- 172.21.15.29:0/889564886 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7fd210016070 con 0x7fd228066b40 2024-01-24T13:58:43.244 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:43.244 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:43.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.245+0000 7fd22e94c700 1 -- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd20805f180 msgr2=0x7fd208061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.245+0000 7fd22e94c700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd20805f180 0x7fd208061640 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fd218001690 tx=0x7fd218009040).stop 2024-01-24T13:58:43.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.245+0000 7fd22e94c700 1 -- 172.21.15.29:0/889564886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd228066b40 msgr2=0x7fd228065360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.245+0000 7fd22e94c700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd228066b40 0x7fd228065360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fd21000e930 tx=0x7fd210009060).stop 2024-01-24T13:58:43.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.245+0000 7fd22e94c700 1 -- 172.21.15.29:0/889564886 shutdown_connections 2024-01-24T13:58:43.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.245+0000 7fd22e94c700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd228158600 0x7fd228065850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.245+0000 7fd22e94c700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228062970 0x7fd22814ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.245+0000 7fd22e94c700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd20805f180 0x7fd208061640 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.245+0000 7fd22e94c700 1 --2- 172.21.15.29:0/889564886 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd228066b40 0x7fd228065360 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.245+0000 7fd22e94c700 1 -- 172.21.15.29:0/889564886 >> 172.21.15.29:0/889564886 conn(0x7fd2280c8580 msgr2=0x7fd2280a4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:43.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.245+0000 7fd22e94c700 1 -- 172.21.15.29:0/889564886 shutdown_connections 2024-01-24T13:58:43.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.245+0000 7fd22e94c700 1 -- 172.21.15.29:0/889564886 wait complete. 2024-01-24T13:58:43.261 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T13:58:16.949817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T13:58:18.174460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-24T13:58:21.232882+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T13:58:43.262 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T13:58:43.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.395+0000 7fb4f9517700 1 Processor -- start 2024-01-24T13:58:43.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.395+0000 7fb4f9517700 1 -- start start 2024-01-24T13:58:43.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f9517700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb4f40664c0 0x7fb4f4066950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f9517700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f4066e40 con 0x7fb4f413a5e0 2024-01-24T13:58:43.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f9517700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f40640d0 con 0x7fb4f40bdb30 2024-01-24T13:58:43.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f9517700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f40679f0 con 0x7fb4f40664c0 2024-01-24T13:58:43.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f27fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb4f40664c0 0x7fb4f4066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f27fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb4f40664c0 0x7fb4f4066950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39480/0 (socket says 172.21.15.29:39480) 2024-01-24T13:58:43.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f2ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb4f413a5e0 0x7fb4f40639c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34426/0 (socket says 172.21.15.29:34426) 2024-01-24T13:58:43.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f27fc700 1 -- 172.21.15.29:0/966515690 learned_addr learned my addr 172.21.15.29:0/966515690 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f27fc700 1 -- 172.21.15.29:0/966515690 >> v1:172.21.15.163:6789/0 conn(0x7fb4f40bdb30 legacy=0x7fb4f40bdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f27fc700 1 -- 172.21.15.29:0/966515690 >> v1:172.21.15.29:6789/0 conn(0x7fb4f413a5e0 legacy=0x7fb4f40639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f27fc700 1 -- 172.21.15.29:0/966515690 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4067cd0 con 0x7fb4f40664c0 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f1ffb700 1 -- 172.21.15.29:0/966515690 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960785955 0 0) 0x7fb4f4066e40 con 0x7fb4f413a5e0 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.396+0000 7fb4f27fc700 1 --2- 172.21.15.29:0/966515690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4f40664c0 0x7fb4f4066950 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fb4e8006fd0 tx=0x7fb4e80090f0).ready entity=mon.0 client_cookie=19c7cf9232fa53f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.397+0000 7fb4f1ffb700 1 -- 172.21.15.29:0/966515690 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e800b600 con 0x7fb4f40664c0 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.397+0000 7fb4f1ffb700 1 -- 172.21.15.29:0/966515690 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e800b760 con 0x7fb4f40664c0 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.397+0000 7fb4f1ffb700 1 -- 172.21.15.29:0/966515690 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e800bae0 con 0x7fb4f40664c0 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.397+0000 7fb4f9517700 1 -- 172.21.15.29:0/966515690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4f40664c0 msgr2=0x7fb4f4066950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.397+0000 7fb4f9517700 1 --2- 172.21.15.29:0/966515690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4f40664c0 0x7fb4f4066950 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fb4e8006fd0 tx=0x7fb4e80090f0).stop 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.397+0000 7fb4f9517700 1 -- 172.21.15.29:0/966515690 shutdown_connections 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.397+0000 7fb4f9517700 1 --2- 172.21.15.29:0/966515690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4f40664c0 0x7fb4f4066950 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.397+0000 7fb4f9517700 1 -- 172.21.15.29:0/966515690 >> 172.21.15.29:0/966515690 conn(0x7fb4f40c8bd0 msgr2=0x7fb4f40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.397+0000 7fb4f9517700 1 -- 172.21.15.29:0/966515690 shutdown_connections 2024-01-24T13:58:43.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.397+0000 7fb4f9517700 1 -- 172.21.15.29:0/966515690 wait complete. 2024-01-24T13:58:43.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.399+0000 7fb4f9517700 1 Processor -- start 2024-01-24T13:58:43.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.399+0000 7fb4f9517700 1 -- start start 2024-01-24T13:58:43.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.399+0000 7fb4f9517700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40664c0 0x7fb4f40addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.399+0000 7fb4f9517700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f40bdb30 0x7fb4f4153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.399+0000 7fb4f9517700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4f413a5e0 0x7fb4f413d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.399+0000 7fb4f9517700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f4165b00 con 0x7fb4f413a5e0 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.399+0000 7fb4f9517700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f413b3f0 con 0x7fb4f40bdb30 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.399+0000 7fb4f9517700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f4153f10 con 0x7fb4f40664c0 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.399+0000 7fb4f37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4f413a5e0 0x7fb4f413d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4f37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4f413a5e0 0x7fb4f413d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39492/0 (socket says 172.21.15.29:39492) 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4f37fe700 1 -- 172.21.15.29:0/3802184967 learned_addr learned my addr 172.21.15.29:0/3802184967 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4f27fc700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f40bdb30 0x7fb4f4153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4f2ffd700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40664c0 0x7fb4f40addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4f37fe700 1 -- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40664c0 msgr2=0x7fb4f40addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4f37fe700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40664c0 0x7fb4f40addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4f37fe700 1 -- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f40bdb30 msgr2=0x7fb4f4153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4f37fe700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f40bdb30 0x7fb4f4153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4f37fe700 1 -- 172.21.15.29:0/3802184967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e8009370 con 0x7fb4f413a5e0 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4f37fe700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4f413a5e0 0x7fb4f413d6b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fb4ec00c810 tx=0x7fb4ec002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4daffd700 1 -- 172.21.15.29:0/3802184967 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec010510 con 0x7fb4f413a5e0 2024-01-24T13:58:43.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4daffd700 1 -- 172.21.15.29:0/3802184967 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0106b0 con 0x7fb4f413a5e0 2024-01-24T13:58:43.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4f9517700 1 -- 172.21.15.29:0/3802184967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f413ade0 con 0x7fb4f413a5e0 2024-01-24T13:58:43.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4daffd700 1 -- 172.21.15.29:0/3802184967 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec010a70 con 0x7fb4f413a5e0 2024-01-24T13:58:43.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.400+0000 7fb4f9517700 1 -- 172.21.15.29:0/3802184967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f413b080 con 0x7fb4f413a5e0 2024-01-24T13:58:43.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.401+0000 7fb4f9517700 1 -- 172.21.15.29:0/3802184967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f413b580 con 0x7fb4f413a5e0 2024-01-24T13:58:43.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.402+0000 7fb4daffd700 1 -- 172.21.15.29:0/3802184967 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb4ec012440 con 0x7fb4f413a5e0 2024-01-24T13:58:43.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.402+0000 7fb4daffd700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb4e005f150 0x7fb4e0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.402+0000 7fb4daffd700 1 -- 172.21.15.29:0/3802184967 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fb4ec079dd0 con 0x7fb4f413a5e0 2024-01-24T13:58:43.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.402+0000 7fb4f2ffd700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb4e005f150 0x7fb4e0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.403+0000 7fb4f2ffd700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb4e005f150 0x7fb4e0061610 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fb4e40011e0 tx=0x7fb4e4007bd0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:43.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.406+0000 7fb4daffd700 1 -- 172.21.15.29:0/3802184967 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb4ec048b00 con 0x7fb4f413a5e0 2024-01-24T13:58:43.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.588+0000 7fb4f9517700 1 -- 172.21.15.29:0/3802184967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb4f413dc00 con 0x7fb4f413a5e0 2024-01-24T13:58:43.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.588+0000 7fb4daffd700 1 -- 172.21.15.29:0/3802184967 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb4ec07b020 con 0x7fb4f413a5e0 2024-01-24T13:58:43.589 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T13:58:43.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.590+0000 7fb4f9517700 1 -- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb4e005f150 msgr2=0x7fb4e0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.590+0000 7fb4f9517700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb4e005f150 0x7fb4e0061610 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fb4e40011e0 tx=0x7fb4e4007bd0).stop 2024-01-24T13:58:43.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.590+0000 7fb4f9517700 1 -- 172.21.15.29:0/3802184967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4f413a5e0 msgr2=0x7fb4f413d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.590+0000 7fb4f9517700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4f413a5e0 0x7fb4f413d6b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fb4ec00c810 tx=0x7fb4ec002560).stop 2024-01-24T13:58:43.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.590+0000 7fb4f9517700 1 -- 172.21.15.29:0/3802184967 shutdown_connections 2024-01-24T13:58:43.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.590+0000 7fb4f9517700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb4e005f150 0x7fb4e0061610 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.590+0000 7fb4f9517700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4f413a5e0 0x7fb4f413d6b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.590+0000 7fb4f9517700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40664c0 0x7fb4f40addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.590+0000 7fb4f9517700 1 --2- 172.21.15.29:0/3802184967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f40bdb30 0x7fb4f4153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.590+0000 7fb4f9517700 1 -- 172.21.15.29:0/3802184967 >> 172.21.15.29:0/3802184967 conn(0x7fb4f40c8bd0 msgr2=0x7fb4f4155a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:43.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.590+0000 7fb4f9517700 1 -- 172.21.15.29:0/3802184967 shutdown_connections 2024-01-24T13:58:43.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.590+0000 7fb4f9517700 1 -- 172.21.15.29:0/3802184967 wait complete. 2024-01-24T13:58:43.604 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T13:58:43.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.738+0000 7fc5de8b0700 1 Processor -- start 2024-01-24T13:58:43.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.738+0000 7fc5de8b0700 1 -- start start 2024-01-24T13:58:43.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.739+0000 7fc5de8b0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d813a5e0 0x7fc5d80639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.739+0000 7fc5de8b0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc5d8063eb0 0x7fc5d80664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.739+0000 7fc5de8b0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc5d80669b0 0x7fc5d8157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.739+0000 7fc5de8b0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d8066e00 con 0x7fc5d80669b0 2024-01-24T13:58:43.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.739+0000 7fc5de8b0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d80679f0 con 0x7fc5d8063eb0 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.739+0000 7fc5de8b0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d8067cd0 con 0x7fc5d813a5e0 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.739+0000 7fc5dce4d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc5d80669b0 0x7fc5d8157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.739+0000 7fc5dce4d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc5d80669b0 0x7fc5d8157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39502/0 (socket says 172.21.15.29:39502) 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.739+0000 7fc5dce4d700 1 -- 172.21.15.29:0/1644567028 learned_addr learned my addr 172.21.15.29:0/1644567028 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.740+0000 7fc5d7fff700 1 --2- 172.21.15.29:0/1644567028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d813a5e0 0x7fc5d80639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.740+0000 7fc5d77fe700 1 --2- 172.21.15.29:0/1644567028 >> v2:172.21.15.163:3300/0 conn(0x7fc5d8063eb0 0x7fc5d80664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.740+0000 7fc5dce4d700 1 -- 172.21.15.29:0/1644567028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d813a5e0 msgr2=0x7fc5d80639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.740+0000 7fc5dce4d700 1 --2- 172.21.15.29:0/1644567028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d813a5e0 0x7fc5d80639c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.740+0000 7fc5dce4d700 1 -- 172.21.15.29:0/1644567028 >> v2:172.21.15.163:3300/0 conn(0x7fc5d8063eb0 msgr2=0x7fc5d80664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.740+0000 7fc5dce4d700 1 --2- 172.21.15.29:0/1644567028 >> v2:172.21.15.163:3300/0 conn(0x7fc5d8063eb0 0x7fc5d80664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.740+0000 7fc5dce4d700 1 -- 172.21.15.29:0/1644567028 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d8067fb0 con 0x7fc5d80669b0 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.740+0000 7fc5d77fe700 1 --2- 172.21.15.29:0/1644567028 >> v2:172.21.15.163:3300/0 conn(0x7fc5d8063eb0 0x7fc5d80664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.740+0000 7fc5dce4d700 1 --2- 172.21.15.29:0/1644567028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5d80669b0 0x7fc5d8157bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fc5d80698c0 tx=0x7fc5d0009840).ready entity=mon.0 client_cookie=70572add1423365c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.740+0000 7fc5d6ffd700 1 -- 172.21.15.29:0/1644567028 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d000e070 con 0x7fc5d80669b0 2024-01-24T13:58:43.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.740+0000 7fc5d6ffd700 1 -- 172.21.15.29:0/1644567028 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d000ba20 con 0x7fc5d80669b0 2024-01-24T13:58:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.740+0000 7fc5d6ffd700 1 -- 172.21.15.29:0/1644567028 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d000bda0 con 0x7fc5d80669b0 2024-01-24T13:58:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.741+0000 7fc5de8b0700 1 -- 172.21.15.29:0/1644567028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5d80669b0 msgr2=0x7fc5d8157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.741+0000 7fc5de8b0700 1 --2- 172.21.15.29:0/1644567028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5d80669b0 0x7fc5d8157bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fc5d80698c0 tx=0x7fc5d0009840).stop 2024-01-24T13:58:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.741+0000 7fc5de8b0700 1 -- 172.21.15.29:0/1644567028 shutdown_connections 2024-01-24T13:58:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.741+0000 7fc5de8b0700 1 --2- 172.21.15.29:0/1644567028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d813a5e0 0x7fc5d80639c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.741+0000 7fc5de8b0700 1 --2- 172.21.15.29:0/1644567028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5d80669b0 0x7fc5d8157bf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.741+0000 7fc5de8b0700 1 --2- 172.21.15.29:0/1644567028 >> v2:172.21.15.163:3300/0 conn(0x7fc5d8063eb0 0x7fc5d80664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.741+0000 7fc5de8b0700 1 -- 172.21.15.29:0/1644567028 >> 172.21.15.29:0/1644567028 conn(0x7fc5d80c8bd0 msgr2=0x7fc5d80c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.741+0000 7fc5de8b0700 1 -- 172.21.15.29:0/1644567028 shutdown_connections 2024-01-24T13:58:43.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.741+0000 7fc5de8b0700 1 -- 172.21.15.29:0/1644567028 wait complete. 2024-01-24T13:58:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.742+0000 7fc5de8b0700 1 Processor -- start 2024-01-24T13:58:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.742+0000 7fc5de8b0700 1 -- start start 2024-01-24T13:58:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.743+0000 7fc5de8b0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d8063eb0 0x7fc5d8153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.743+0000 7fc5de8b0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5d80669b0 0x7fc5d813b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.743+0000 7fc5de8b0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5d813a5e0 0x7fc5d813bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.743+0000 7fc5de8b0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d81540b0 con 0x7fc5d813a5e0 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.743+0000 7fc5de8b0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d813c0f0 con 0x7fc5d80669b0 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.743+0000 7fc5de8b0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d8139190 con 0x7fc5d8063eb0 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.743+0000 7fc5dce4d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5d813a5e0 0x7fc5d813bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.743+0000 7fc5dce4d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5d813a5e0 0x7fc5d813bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39506/0 (socket says 172.21.15.29:39506) 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.743+0000 7fc5dce4d700 1 -- 172.21.15.29:0/2663736673 learned_addr learned my addr 172.21.15.29:0/2663736673 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.743+0000 7fc5d7fff700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d8063eb0 0x7fc5d8153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.743+0000 7fc5d77fe700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5d80669b0 0x7fc5d813b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.744+0000 7fc5dce4d700 1 -- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d8063eb0 msgr2=0x7fc5d8153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.744+0000 7fc5dce4d700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d8063eb0 0x7fc5d8153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.744+0000 7fc5dce4d700 1 -- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5d80669b0 msgr2=0x7fc5d813b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.744+0000 7fc5dce4d700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5d80669b0 0x7fc5d813b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.744+0000 7fc5dce4d700 1 -- 172.21.15.29:0/2663736673 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d0009ae0 con 0x7fc5d813a5e0 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.744+0000 7fc5dce4d700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5d813a5e0 0x7fc5d813bc00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fc5d000b6c0 tx=0x7fc5d0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.744+0000 7fc5cffff700 1 -- 172.21.15.29:0/2663736673 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d00032f0 con 0x7fc5d813a5e0 2024-01-24T13:58:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.744+0000 7fc5cffff700 1 -- 172.21.15.29:0/2663736673 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d00092e0 con 0x7fc5d813a5e0 2024-01-24T13:58:43.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.744+0000 7fc5de8b0700 1 -- 172.21.15.29:0/2663736673 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d813cf00 con 0x7fc5d813a5e0 2024-01-24T13:58:43.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.744+0000 7fc5cffff700 1 -- 172.21.15.29:0/2663736673 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d001e6a0 con 0x7fc5d813a5e0 2024-01-24T13:58:43.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.744+0000 7fc5de8b0700 1 -- 172.21.15.29:0/2663736673 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d813c5d0 con 0x7fc5d813a5e0 2024-01-24T13:58:43.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.745+0000 7fc5de8b0700 1 -- 172.21.15.29:0/2663736673 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d813d090 con 0x7fc5d813a5e0 2024-01-24T13:58:43.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.745+0000 7fc5cffff700 1 -- 172.21.15.29:0/2663736673 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc5d0003040 con 0x7fc5d813a5e0 2024-01-24T13:58:43.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.746+0000 7fc5cffff700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc5b805f150 0x7fc5b8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:43.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.746+0000 7fc5cffff700 1 -- 172.21.15.29:0/2663736673 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fc5d007b830 con 0x7fc5d813a5e0 2024-01-24T13:58:43.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.746+0000 7fc5d7fff700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc5b805f150 0x7fc5b8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:43.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.747+0000 7fc5d7fff700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc5b805f150 0x7fc5b8061610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fc5c8009b90 tx=0x7fc5c8009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:43.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.750+0000 7fc5cffff700 1 -- 172.21.15.29:0/2663736673 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc5d004c8a0 con 0x7fc5d813a5e0 2024-01-24T13:58:43.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.929+0000 7fc5de8b0700 1 -- 172.21.15.29:0/2663736673 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc5d80475e0 con 0x7fc5d813a5e0 2024-01-24T13:58:43.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.929+0000 7fc5cffff700 1 -- 172.21.15.29:0/2663736673 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc5d0023070 con 0x7fc5d813a5e0 2024-01-24T13:58:43.930 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T13:58:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.931+0000 7fc5de8b0700 1 -- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc5b805f150 msgr2=0x7fc5b8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.931+0000 7fc5de8b0700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc5b805f150 0x7fc5b8061610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fc5c8009b90 tx=0x7fc5c8009060).stop 2024-01-24T13:58:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.931+0000 7fc5de8b0700 1 -- 172.21.15.29:0/2663736673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5d813a5e0 msgr2=0x7fc5d813bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.931+0000 7fc5de8b0700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5d813a5e0 0x7fc5d813bc00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fc5d000b6c0 tx=0x7fc5d0009840).stop 2024-01-24T13:58:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.931+0000 7fc5de8b0700 1 -- 172.21.15.29:0/2663736673 shutdown_connections 2024-01-24T13:58:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.931+0000 7fc5de8b0700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc5b805f150 0x7fc5b8061610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.931+0000 7fc5de8b0700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5d813a5e0 0x7fc5d813bc00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.931+0000 7fc5de8b0700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d8063eb0 0x7fc5d8153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.931+0000 7fc5de8b0700 1 --2- 172.21.15.29:0/2663736673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5d80669b0 0x7fc5d813b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.931+0000 7fc5de8b0700 1 -- 172.21.15.29:0/2663736673 >> 172.21.15.29:0/2663736673 conn(0x7fc5d80c8bd0 msgr2=0x7fc5d8156620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.931+0000 7fc5de8b0700 1 -- 172.21.15.29:0/2663736673 shutdown_connections 2024-01-24T13:58:43.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:43.931+0000 7fc5de8b0700 1 -- 172.21.15.29:0/2663736673 wait complete. 2024-01-24T13:58:43.945 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T13:58:44.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.078+0000 7f27d075a700 1 Processor -- start 2024-01-24T13:58:44.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.078+0000 7f27d075a700 1 -- start start 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27d075a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27c8155290 0x7f27c80bc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27d075a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f27c8156240 0x7f27c8062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27d075a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c8156920 con 0x7f27c8155670 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27d075a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c8065530 con 0x7f27c8155290 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27d075a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c8065710 con 0x7f27c8156240 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27ce4f6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27c8155290 0x7f27c80bc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27cdcf5700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f27c8155670 0x7f27c8155b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34430/0 (socket says 172.21.15.29:34430) 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27ce4f6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27c8155290 0x7f27c80bc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39518/0 (socket says 172.21.15.29:39518) 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27cdcf5700 1 -- 172.21.15.29:0/82494980 learned_addr learned my addr 172.21.15.29:0/82494980 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27cecf7700 1 --2- 172.21.15.29:0/82494980 >> v2:172.21.15.163:3300/0 conn(0x7f27c8156240 0x7f27c8062fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27ce4f6700 1 -- 172.21.15.29:0/82494980 >> v2:172.21.15.163:3300/0 conn(0x7f27c8156240 msgr2=0x7f27c8062fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27ce4f6700 1 --2- 172.21.15.29:0/82494980 >> v2:172.21.15.163:3300/0 conn(0x7f27c8156240 0x7f27c8062fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27ce4f6700 1 -- 172.21.15.29:0/82494980 >> v1:172.21.15.29:6789/0 conn(0x7f27c8155670 legacy=0x7f27c8155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27cd4f4700 1 -- 172.21.15.29:0/82494980 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219172868 0 0) 0x7f27c8156920 con 0x7f27c8155670 2024-01-24T13:58:44.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27ce4f6700 1 -- 172.21.15.29:0/82494980 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80659f0 con 0x7f27c8155290 2024-01-24T13:58:44.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.079+0000 7f27ce4f6700 1 --2- 172.21.15.29:0/82494980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c8155290 0x7f27c80bc2a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f27c8064430 tx=0x7f27c4009060).ready entity=mon.0 client_cookie=fe78461d1aa93430 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:44.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.080+0000 7f27cd4f4700 1 -- 172.21.15.29:0/82494980 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c40090d0 con 0x7f27c8155290 2024-01-24T13:58:44.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.080+0000 7f27cd4f4700 1 -- 172.21.15.29:0/82494980 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c400b580 con 0x7f27c8155290 2024-01-24T13:58:44.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.080+0000 7f27cd4f4700 1 -- 172.21.15.29:0/82494980 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c400b900 con 0x7f27c8155290 2024-01-24T13:58:44.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.080+0000 7f27d075a700 1 -- 172.21.15.29:0/82494980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c8155290 msgr2=0x7f27c80bc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.080+0000 7f27d075a700 1 --2- 172.21.15.29:0/82494980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c8155290 0x7f27c80bc2a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f27c8064430 tx=0x7f27c4009060).stop 2024-01-24T13:58:44.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.080+0000 7f27d075a700 1 -- 172.21.15.29:0/82494980 shutdown_connections 2024-01-24T13:58:44.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.080+0000 7f27d075a700 1 --2- 172.21.15.29:0/82494980 >> v2:172.21.15.163:3300/0 conn(0x7f27c8156240 0x7f27c8062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.080+0000 7f27d075a700 1 --2- 172.21.15.29:0/82494980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c8155290 0x7f27c80bc2a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.080+0000 7f27d075a700 1 -- 172.21.15.29:0/82494980 >> 172.21.15.29:0/82494980 conn(0x7f27c80c7260 msgr2=0x7f27c80c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:44.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.080+0000 7f27d075a700 1 -- 172.21.15.29:0/82494980 shutdown_connections 2024-01-24T13:58:44.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.080+0000 7f27d075a700 1 -- 172.21.15.29:0/82494980 wait complete. 2024-01-24T13:58:44.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.081+0000 7f27d075a700 1 Processor -- start 2024-01-24T13:58:44.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.082+0000 7f27d075a700 1 -- start start 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.082+0000 7f27d075a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27c8155290 0x7f27c814c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.082+0000 7f27d075a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c8155670 0x7f27c8148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.082+0000 7f27d075a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27c8156240 0x7f27c80bf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.082+0000 7f27d075a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c80c22d0 con 0x7f27c8155670 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.082+0000 7f27d075a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c80c2760 con 0x7f27c8156240 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.082+0000 7f27d075a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c814ca50 con 0x7f27c8155290 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.082+0000 7f27cdcf5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c8155670 0x7f27c8148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27cdcf5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c8155670 0x7f27c8148680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39532/0 (socket says 172.21.15.29:39532) 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27cdcf5700 1 -- 172.21.15.29:0/4147082907 learned_addr learned my addr 172.21.15.29:0/4147082907 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27cecf7700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27c8156240 0x7f27c80bf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27ce4f6700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27c8155290 0x7f27c814c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27cdcf5700 1 -- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27c8155290 msgr2=0x7f27c814c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27cdcf5700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27c8155290 0x7f27c814c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27cdcf5700 1 -- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27c8156240 msgr2=0x7f27c80bf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27cdcf5700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27c8156240 0x7f27c80bf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27cdcf5700 1 -- 172.21.15.29:0/4147082907 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c40092a0 con 0x7f27c8155670 2024-01-24T13:58:44.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27cecf7700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27c8156240 0x7f27c80bf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:44.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27ce4f6700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27c8155290 0x7f27c814c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:44.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27cdcf5700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c8155670 0x7f27c8148680 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f27bc00c4e0 tx=0x7f27bc007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:44.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27ba7fc700 1 -- 172.21.15.29:0/4147082907 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0072e0 con 0x7f27c8155670 2024-01-24T13:58:44.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27ba7fc700 1 -- 172.21.15.29:0/4147082907 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc007440 con 0x7f27c8155670 2024-01-24T13:58:44.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27d075a700 1 -- 172.21.15.29:0/4147082907 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c80c19a0 con 0x7f27c8155670 2024-01-24T13:58:44.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27ba7fc700 1 -- 172.21.15.29:0/4147082907 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0057a0 con 0x7f27c8155670 2024-01-24T13:58:44.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.083+0000 7f27d075a700 1 -- 172.21.15.29:0/4147082907 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c80c1f60 con 0x7f27c8155670 2024-01-24T13:58:44.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.084+0000 7f27d075a700 1 -- 172.21.15.29:0/4147082907 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c80c2480 con 0x7f27c8155670 2024-01-24T13:58:44.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.085+0000 7f27ba7fc700 1 -- 172.21.15.29:0/4147082907 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f27bc005bd0 con 0x7f27c8155670 2024-01-24T13:58:44.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.085+0000 7f27ba7fc700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f27ac05f150 0x7f27ac061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.085+0000 7f27ba7fc700 1 -- 172.21.15.29:0/4147082907 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f27bc079e30 con 0x7f27c8155670 2024-01-24T13:58:44.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.086+0000 7f27ce4f6700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f27ac05f150 0x7f27ac061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.086+0000 7f27ce4f6700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f27ac05f150 0x7f27ac061610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f27c4009fd0 tx=0x7f27c4017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:44.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.090+0000 7f27ba7fc700 1 -- 172.21.15.29:0/4147082907 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f27c80c2480 con 0x7f27c8155670 2024-01-24T13:58:44.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.269+0000 7f27d075a700 1 -- 172.21.15.29:0/4147082907 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27c80475e0 con 0x7f27c8155670 2024-01-24T13:58:44.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.270+0000 7f27ba7fc700 1 -- 172.21.15.29:0/4147082907 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f27bc04adf0 con 0x7f27c8155670 2024-01-24T13:58:44.270 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:58:44.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.272+0000 7f27d075a700 1 -- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f27ac05f150 msgr2=0x7f27ac061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.272+0000 7f27d075a700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f27ac05f150 0x7f27ac061610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f27c4009fd0 tx=0x7f27c4017000).stop 2024-01-24T13:58:44.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.272+0000 7f27d075a700 1 -- 172.21.15.29:0/4147082907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c8155670 msgr2=0x7f27c8148680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.272+0000 7f27d075a700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c8155670 0x7f27c8148680 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f27bc00c4e0 tx=0x7f27bc007c70).stop 2024-01-24T13:58:44.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.272+0000 7f27d075a700 1 -- 172.21.15.29:0/4147082907 shutdown_connections 2024-01-24T13:58:44.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.272+0000 7f27d075a700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27c8156240 0x7f27c80bf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.272+0000 7f27d075a700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27c8155290 0x7f27c814c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.272+0000 7f27d075a700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f27ac05f150 0x7f27ac061610 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.272+0000 7f27d075a700 1 --2- 172.21.15.29:0/4147082907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c8155670 0x7f27c8148680 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.272+0000 7f27d075a700 1 -- 172.21.15.29:0/4147082907 >> 172.21.15.29:0/4147082907 conn(0x7f27c80c7260 msgr2=0x7f27c80c5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:44.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.272+0000 7f27d075a700 1 -- 172.21.15.29:0/4147082907 shutdown_connections 2024-01-24T13:58:44.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.272+0000 7f27d075a700 1 -- 172.21.15.29:0/4147082907 wait complete. 2024-01-24T13:58:44.285 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T13:58:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.417+0000 7f90d26d6700 1 Processor -- start 2024-01-24T13:58:44.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.417+0000 7f90d26d6700 1 -- start start 2024-01-24T13:58:44.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.418+0000 7f90d26d6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f90cc066940 0x7f90cc0be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.418+0000 7f90d26d6700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90cc0686a0 con 0x7f90cc134ea0 2024-01-24T13:58:44.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.418+0000 7f90d26d6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90cc0c38e0 con 0x7f90cc0664c0 2024-01-24T13:58:44.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.418+0000 7f90d26d6700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90cc068b30 con 0x7f90cc066940 2024-01-24T13:58:44.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.418+0000 7f90d0c73700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f90cc066940 0x7f90cc0be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.418+0000 7f90cbfff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f90cc134ea0 0x7f90cc0c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34446/0 (socket says 172.21.15.29:34446) 2024-01-24T13:58:44.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.418+0000 7f90cbfff700 1 -- 172.21.15.29:0/2762521354 learned_addr learned my addr 172.21.15.29:0/2762521354 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:44.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.419+0000 7f90d0c73700 1 -- 172.21.15.29:0/2762521354 >> v1:172.21.15.163:6789/0 conn(0x7f90cc0664c0 legacy=0x7f90cc0bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.419+0000 7f90d0c73700 1 -- 172.21.15.29:0/2762521354 >> v1:172.21.15.29:6789/0 conn(0x7f90cc134ea0 legacy=0x7f90cc0c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.419+0000 7f90d0c73700 1 -- 172.21.15.29:0/2762521354 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc066dd0 con 0x7f90cc066940 2024-01-24T13:58:44.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.419+0000 7f90caffd700 1 -- 172.21.15.29:0/2762521354 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595825275 0 0) 0x7f90cc0686a0 con 0x7f90cc134ea0 2024-01-24T13:58:44.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.419+0000 7f90d0c73700 1 --2- 172.21.15.29:0/2762521354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90cc066940 0x7f90cc0be240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f90c4009e60 tx=0x7f90c4009780).ready entity=mon.0 client_cookie=46a2d81c3faa1b25 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:44.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.419+0000 7f90caffd700 1 -- 172.21.15.29:0/2762521354 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c400e070 con 0x7f90cc066940 2024-01-24T13:58:44.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.419+0000 7f90caffd700 1 -- 172.21.15.29:0/2762521354 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c400bc30 con 0x7f90cc066940 2024-01-24T13:58:44.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.419+0000 7f90caffd700 1 -- 172.21.15.29:0/2762521354 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4012410 con 0x7f90cc066940 2024-01-24T13:58:44.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.419+0000 7f90d26d6700 1 -- 172.21.15.29:0/2762521354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90cc066940 msgr2=0x7f90cc0be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.419+0000 7f90d26d6700 1 --2- 172.21.15.29:0/2762521354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90cc066940 0x7f90cc0be240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f90c4009e60 tx=0x7f90c4009780).stop 2024-01-24T13:58:44.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.420+0000 7f90d26d6700 1 -- 172.21.15.29:0/2762521354 shutdown_connections 2024-01-24T13:58:44.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.420+0000 7f90d26d6700 1 --2- 172.21.15.29:0/2762521354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90cc066940 0x7f90cc0be240 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.420+0000 7f90d26d6700 1 -- 172.21.15.29:0/2762521354 >> 172.21.15.29:0/2762521354 conn(0x7f90cc0c8bd0 msgr2=0x7f90cc0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:44.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.420+0000 7f90d26d6700 1 -- 172.21.15.29:0/2762521354 shutdown_connections 2024-01-24T13:58:44.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.420+0000 7f90d26d6700 1 -- 172.21.15.29:0/2762521354 wait complete. 2024-01-24T13:58:44.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.421+0000 7f90d26d6700 1 Processor -- start 2024-01-24T13:58:44.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.421+0000 7f90d26d6700 1 -- start start 2024-01-24T13:58:44.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90d26d6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90cc0664c0 0x7f90cc134490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90d26d6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc066940 0x7f90cc0638c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90d26d6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc134ea0 0x7f90cc1378b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90d26d6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc153450 con 0x7f90cc0664c0 2024-01-24T13:58:44.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90d26d6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0641c0 con 0x7f90cc134ea0 2024-01-24T13:58:44.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90d26d6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0644a0 con 0x7f90cc066940 2024-01-24T13:58:44.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90cbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90cc0664c0 0x7f90cc134490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90cbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90cc0664c0 0x7f90cc134490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39554/0 (socket says 172.21.15.29:39554) 2024-01-24T13:58:44.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90cbfff700 1 -- 172.21.15.29:0/962439685 learned_addr learned my addr 172.21.15.29:0/962439685 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:44.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90cb7fe700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc066940 0x7f90cc0638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90d0c73700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc134ea0 0x7f90cc1378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90cbfff700 1 -- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc066940 msgr2=0x7f90cc0638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90cbfff700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc066940 0x7f90cc0638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90cbfff700 1 -- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc134ea0 msgr2=0x7f90cc1378b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90cbfff700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc134ea0 0x7f90cc1378b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90cbfff700 1 -- 172.21.15.29:0/962439685 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c4009a20 con 0x7f90cc0664c0 2024-01-24T13:58:44.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.422+0000 7f90cb7fe700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc066940 0x7f90cc0638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:44.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.423+0000 7f90cbfff700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90cc0664c0 0x7f90cc134490 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f90bc00c4c0 tx=0x7f90bc007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:44.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.423+0000 7f90c3fff700 1 -- 172.21.15.29:0/962439685 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc0072e0 con 0x7f90cc0664c0 2024-01-24T13:58:44.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.423+0000 7f90c3fff700 1 -- 172.21.15.29:0/962439685 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc007440 con 0x7f90cc0664c0 2024-01-24T13:58:44.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.423+0000 7f90d26d6700 1 -- 172.21.15.29:0/962439685 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90cc064780 con 0x7f90cc0664c0 2024-01-24T13:58:44.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.423+0000 7f90c3fff700 1 -- 172.21.15.29:0/962439685 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc0057a0 con 0x7f90cc0664c0 2024-01-24T13:58:44.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.423+0000 7f90d26d6700 1 -- 172.21.15.29:0/962439685 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc134980 con 0x7f90cc0664c0 2024-01-24T13:58:44.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.424+0000 7f90d26d6700 1 -- 172.21.15.29:0/962439685 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc0475e0 con 0x7f90cc0664c0 2024-01-24T13:58:44.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.424+0000 7f90c3fff700 1 -- 172.21.15.29:0/962439685 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f90bc005b90 con 0x7f90cc0664c0 2024-01-24T13:58:44.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.425+0000 7f90c3fff700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f90ac05f120 0x7f90ac0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.425+0000 7f90c3fff700 1 -- 172.21.15.29:0/962439685 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f90bc079ea0 con 0x7f90cc0664c0 2024-01-24T13:58:44.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.425+0000 7f90cb7fe700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f90ac05f120 0x7f90ac0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.426+0000 7f90cb7fe700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f90ac05f120 0x7f90ac0615e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f90cc151410 tx=0x7f90b4009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:44.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.429+0000 7f90c3fff700 1 -- 172.21.15.29:0/962439685 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f90bc04ada0 con 0x7f90cc0664c0 2024-01-24T13:58:44.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.612+0000 7f90d26d6700 1 -- 172.21.15.29:0/962439685 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f90cc067a90 con 0x7f90cc0664c0 2024-01-24T13:58:44.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.613+0000 7f90c3fff700 1 -- 172.21.15.29:0/962439685 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f90bc018030 con 0x7f90cc0664c0 2024-01-24T13:58:44.613 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:58:44.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.614+0000 7f90d26d6700 1 -- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f90ac05f120 msgr2=0x7f90ac0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.614+0000 7f90d26d6700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f90ac05f120 0x7f90ac0615e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f90cc151410 tx=0x7f90b4009040).stop 2024-01-24T13:58:44.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.614+0000 7f90d26d6700 1 -- 172.21.15.29:0/962439685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90cc0664c0 msgr2=0x7f90cc134490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.614+0000 7f90d26d6700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90cc0664c0 0x7f90cc134490 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f90bc00c4c0 tx=0x7f90bc007c10).stop 2024-01-24T13:58:44.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.615+0000 7f90d26d6700 1 -- 172.21.15.29:0/962439685 shutdown_connections 2024-01-24T13:58:44.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.615+0000 7f90d26d6700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc134ea0 0x7f90cc1378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.615+0000 7f90d26d6700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f90ac05f120 0x7f90ac0615e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.615+0000 7f90d26d6700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90cc0664c0 0x7f90cc134490 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.615+0000 7f90d26d6700 1 --2- 172.21.15.29:0/962439685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc066940 0x7f90cc0638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.615+0000 7f90d26d6700 1 -- 172.21.15.29:0/962439685 >> 172.21.15.29:0/962439685 conn(0x7f90cc0c8bd0 msgr2=0x7f90cc0c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:44.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.615+0000 7f90d26d6700 1 -- 172.21.15.29:0/962439685 shutdown_connections 2024-01-24T13:58:44.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.615+0000 7f90d26d6700 1 -- 172.21.15.29:0/962439685 wait complete. 2024-01-24T13:58:44.628 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-24T13:58:44.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.761+0000 7f84c0ca6700 1 Processor -- start 2024-01-24T13:58:44.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.761+0000 7f84c0ca6700 1 -- start start 2024-01-24T13:58:44.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84c0ca6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84bc0664c0 0x7f84bc066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84c0ca6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f84bc0bdb30 0x7f84bc0bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84c0ca6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84bc0686a0 con 0x7f84bc134ea0 2024-01-24T13:58:44.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84c0ca6700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84bc066e30 con 0x7f84bc0664c0 2024-01-24T13:58:44.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84c0ca6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84bc0c38e0 con 0x7f84bc0bdb30 2024-01-24T13:58:44.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84b9d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84bc0664c0 0x7f84bc066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84b9d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84bc0664c0 0x7f84bc066940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39570/0 (socket says 172.21.15.29:39570) 2024-01-24T13:58:44.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84b9d9b700 1 -- 172.21.15.29:0/692426581 learned_addr learned my addr 172.21.15.29:0/692426581 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:44.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84bad9d700 1 --2- 172.21.15.29:0/692426581 >> v2:172.21.15.163:3300/0 conn(0x7f84bc0bdb30 0x7f84bc0bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84b9d9b700 1 -- 172.21.15.29:0/692426581 >> v2:172.21.15.163:3300/0 conn(0x7f84bc0bdb30 msgr2=0x7f84bc0bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84b9d9b700 1 --2- 172.21.15.29:0/692426581 >> v2:172.21.15.163:3300/0 conn(0x7f84bc0bdb30 0x7f84bc0bdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84b9d9b700 1 -- 172.21.15.29:0/692426581 >> v1:172.21.15.163:6789/0 conn(0x7f84bc134ea0 legacy=0x7f84bc0c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84b9d9b700 1 -- 172.21.15.29:0/692426581 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc068b30 con 0x7f84bc0664c0 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.763+0000 7f84b9d9b700 1 --2- 172.21.15.29:0/692426581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc0664c0 0x7f84bc066940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f84bc150020 tx=0x7f84a40090f0).ready entity=mon.0 client_cookie=1c68619c201fab88 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.764+0000 7f84b959a700 1 -- 172.21.15.29:0/692426581 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a400b520 con 0x7f84bc0664c0 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.764+0000 7f84b959a700 1 -- 172.21.15.29:0/692426581 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a400b680 con 0x7f84bc0664c0 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.764+0000 7f84b959a700 1 -- 172.21.15.29:0/692426581 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a400ba00 con 0x7f84bc0664c0 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.764+0000 7f84c0ca6700 1 -- 172.21.15.29:0/692426581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc0664c0 msgr2=0x7f84bc066940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.764+0000 7f84c0ca6700 1 --2- 172.21.15.29:0/692426581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc0664c0 0x7f84bc066940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f84bc150020 tx=0x7f84a40090f0).stop 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.764+0000 7f84c0ca6700 1 -- 172.21.15.29:0/692426581 shutdown_connections 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.764+0000 7f84c0ca6700 1 --2- 172.21.15.29:0/692426581 >> v2:172.21.15.163:3300/0 conn(0x7f84bc0bdb30 0x7f84bc0bdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.764+0000 7f84c0ca6700 1 --2- 172.21.15.29:0/692426581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc0664c0 0x7f84bc066940 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.764+0000 7f84c0ca6700 1 -- 172.21.15.29:0/692426581 >> 172.21.15.29:0/692426581 conn(0x7f84bc0c8bd0 msgr2=0x7f84bc0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:44.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.764+0000 7f84c0ca6700 1 -- 172.21.15.29:0/692426581 shutdown_connections 2024-01-24T13:58:44.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.764+0000 7f84c0ca6700 1 -- 172.21.15.29:0/692426581 wait complete. 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.766+0000 7f84c0ca6700 1 Processor -- start 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.766+0000 7f84c0ca6700 1 -- start start 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.766+0000 7f84c0ca6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84bc0664c0 0x7f84bc154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.766+0000 7f84c0ca6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84bc0bdb30 0x7f84bc153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.766+0000 7f84c0ca6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc134ea0 0x7f84bc153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.766+0000 7f84c0ca6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc153d00 con 0x7f84bc134ea0 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.766+0000 7f84c0ca6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc152750 con 0x7f84bc0664c0 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.766+0000 7f84c0ca6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc155470 con 0x7f84bc0bdb30 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84bad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc134ea0 0x7f84bc153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84bad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc134ea0 0x7f84bc153810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39586/0 (socket says 172.21.15.29:39586) 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84bad9d700 1 -- 172.21.15.29:0/642695351 learned_addr learned my addr 172.21.15.29:0/642695351 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84b9d9b700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84bc0bdb30 0x7f84bc153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84ba59c700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84bc0664c0 0x7f84bc154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84bad9d700 1 -- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84bc0bdb30 msgr2=0x7f84bc153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84bad9d700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84bc0bdb30 0x7f84bc153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84bad9d700 1 -- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84bc0664c0 msgr2=0x7f84bc154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84bad9d700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84bc0664c0 0x7f84bc154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84bad9d700 1 -- 172.21.15.29:0/642695351 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a4009370 con 0x7f84bc134ea0 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84ba59c700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84bc0664c0 0x7f84bc154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84b9d9b700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84bc0bdb30 0x7f84bc153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84bad9d700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc134ea0 0x7f84bc153810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f84b4009e90 tx=0x7f84b400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84b27fc700 1 -- 172.21.15.29:0/642695351 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b400cbc0 con 0x7f84bc134ea0 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84b27fc700 1 -- 172.21.15.29:0/642695351 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b400f040 con 0x7f84bc134ea0 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84c0ca6700 1 -- 172.21.15.29:0/642695351 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc153ee0 con 0x7f84bc134ea0 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84b27fc700 1 -- 172.21.15.29:0/642695351 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4013400 con 0x7f84bc134ea0 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.767+0000 7f84c0ca6700 1 -- 172.21.15.29:0/642695351 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc1544a0 con 0x7f84bc134ea0 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.768+0000 7f84c0ca6700 1 -- 172.21.15.29:0/642695351 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84bc0475e0 con 0x7f84bc134ea0 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.769+0000 7f84b27fc700 1 -- 172.21.15.29:0/642695351 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f84b40137b0 con 0x7f84bc134ea0 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.769+0000 7f84b27fc700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f849c05f150 0x7f849c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:44.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.769+0000 7f84b27fc700 1 -- 172.21.15.29:0/642695351 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f84b407ad50 con 0x7f84bc134ea0 2024-01-24T13:58:44.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.769+0000 7f84ba59c700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f849c05f150 0x7f849c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:44.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.770+0000 7f84ba59c700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f849c05f150 0x7f849c061610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f84ac001690 tx=0x7f84ac009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:44.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.773+0000 7f84b27fc700 1 -- 172.21.15.29:0/642695351 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f84b407e030 con 0x7f84bc134ea0 2024-01-24T13:58:44.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.952+0000 7f84c0ca6700 1 -- 172.21.15.29:0/642695351 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f84bc067a90 con 0x7f84bc134ea0 2024-01-24T13:58:44.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.953+0000 7f84b27fc700 1 -- 172.21.15.29:0/642695351 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f84b404bd10 con 0x7f84bc134ea0 2024-01-24T13:58:44.953 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-01-24T13:58:44.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.955+0000 7f84c0ca6700 1 -- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f849c05f150 msgr2=0x7f849c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.955+0000 7f84c0ca6700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f849c05f150 0x7f849c061610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f84ac001690 tx=0x7f84ac009040).stop 2024-01-24T13:58:44.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.955+0000 7f84c0ca6700 1 -- 172.21.15.29:0/642695351 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc134ea0 msgr2=0x7f84bc153810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:44.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.955+0000 7f84c0ca6700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc134ea0 0x7f84bc153810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f84b4009e90 tx=0x7f84b400b040).stop 2024-01-24T13:58:44.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.955+0000 7f84c0ca6700 1 -- 172.21.15.29:0/642695351 shutdown_connections 2024-01-24T13:58:44.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.955+0000 7f84c0ca6700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f849c05f150 0x7f849c061610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.955+0000 7f84c0ca6700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc134ea0 0x7f84bc153810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.955+0000 7f84c0ca6700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84bc0664c0 0x7f84bc154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.955+0000 7f84c0ca6700 1 --2- 172.21.15.29:0/642695351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84bc0bdb30 0x7f84bc153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:44.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.955+0000 7f84c0ca6700 1 -- 172.21.15.29:0/642695351 >> 172.21.15.29:0/642695351 conn(0x7f84bc0c8bd0 msgr2=0x7f84bc0bdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:44.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.955+0000 7f84c0ca6700 1 -- 172.21.15.29:0/642695351 shutdown_connections 2024-01-24T13:58:44.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:44.955+0000 7f84c0ca6700 1 -- 172.21.15.29:0/642695351 wait complete. 2024-01-24T13:58:44.969 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:58:45.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.098+0000 7f8f69bd4700 1 Processor -- start 2024-01-24T13:58:45.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.098+0000 7f8f69bd4700 1 -- start start 2024-01-24T13:58:45.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.100+0000 7f8f69bd4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8f6414c890 0x7f8f640627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.100+0000 7f8f69bd4700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f640639a0 con 0x7f8f64063e30 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.100+0000 7f8f69bd4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f64062d90 con 0x7f8f6414c4b0 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.100+0000 7f8f69bd4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f64063070 con 0x7f8f6414c890 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.100+0000 7f8f62ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8f6414c890 0x7f8f640627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.100+0000 7f8f62ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8f6414c890 0x7f8f640627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39592/0 (socket says 172.21.15.29:39592) 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.100+0000 7f8f62ffd700 1 -- 172.21.15.29:0/515978467 learned_addr learned my addr 172.21.15.29:0/515978467 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.100+0000 7f8f63fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8f64063e30 0x7f8f640689a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34456/0 (socket says 172.21.15.29:34456) 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.100+0000 7f8f62ffd700 1 -- 172.21.15.29:0/515978467 >> v1:172.21.15.163:6789/0 conn(0x7f8f6414c4b0 legacy=0x7f8f64153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.100+0000 7f8f62ffd700 1 -- 172.21.15.29:0/515978467 >> v1:172.21.15.29:6789/0 conn(0x7f8f64063e30 legacy=0x7f8f640689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.100+0000 7f8f627fc700 1 -- 172.21.15.29:0/515978467 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523170542 0 0) 0x7f8f640639a0 con 0x7f8f64063e30 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.100+0000 7f8f62ffd700 1 -- 172.21.15.29:0/515978467 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f64063350 con 0x7f8f6414c890 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.101+0000 7f8f62ffd700 1 --2- 172.21.15.29:0/515978467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f6414c890 0x7f8f640627b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f8f58006fd0 tx=0x7f8f580090f0).ready entity=mon.0 client_cookie=d81eb555afaedb09 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.101+0000 7f8f627fc700 1 -- 172.21.15.29:0/515978467 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800b600 con 0x7f8f6414c890 2024-01-24T13:58:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.101+0000 7f8f627fc700 1 -- 172.21.15.29:0/515978467 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5800b760 con 0x7f8f6414c890 2024-01-24T13:58:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.101+0000 7f8f627fc700 1 -- 172.21.15.29:0/515978467 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800bae0 con 0x7f8f6414c890 2024-01-24T13:58:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.101+0000 7f8f69bd4700 1 -- 172.21.15.29:0/515978467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f6414c890 msgr2=0x7f8f640627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.101+0000 7f8f69bd4700 1 --2- 172.21.15.29:0/515978467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f6414c890 0x7f8f640627b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f8f58006fd0 tx=0x7f8f580090f0).stop 2024-01-24T13:58:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.101+0000 7f8f69bd4700 1 -- 172.21.15.29:0/515978467 shutdown_connections 2024-01-24T13:58:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.101+0000 7f8f69bd4700 1 --2- 172.21.15.29:0/515978467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f6414c890 0x7f8f640627b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.101+0000 7f8f69bd4700 1 -- 172.21.15.29:0/515978467 >> 172.21.15.29:0/515978467 conn(0x7f8f64064aa0 msgr2=0x7f8f64064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.101+0000 7f8f69bd4700 1 -- 172.21.15.29:0/515978467 shutdown_connections 2024-01-24T13:58:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.101+0000 7f8f69bd4700 1 -- 172.21.15.29:0/515978467 wait complete. 2024-01-24T13:58:45.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.103+0000 7f8f69bd4700 1 Processor -- start 2024-01-24T13:58:45.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.103+0000 7f8f69bd4700 1 -- start start 2024-01-24T13:58:45.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.104+0000 7f8f69bd4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f64063e30 0x7f8f64162c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.104+0000 7f8f69bd4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f6414c4b0 0x7f8f64133750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.104+0000 7f8f69bd4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f6414c890 0x7f8f64154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.104+0000 7f8f69bd4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64163100 con 0x7f8f6414c4b0 2024-01-24T13:58:45.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.104+0000 7f8f69bd4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64133c40 con 0x7f8f6414c890 2024-01-24T13:58:45.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.104+0000 7f8f69bd4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640aa220 con 0x7f8f64063e30 2024-01-24T13:58:45.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.104+0000 7f8f62ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f6414c4b0 0x7f8f64133750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.104+0000 7f8f62ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f6414c4b0 0x7f8f64133750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39600/0 (socket says 172.21.15.29:39600) 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.104+0000 7f8f62ffd700 1 -- 172.21.15.29:0/214928225 learned_addr learned my addr 172.21.15.29:0/214928225 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f637fe700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f64063e30 0x7f8f64162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f63fff700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f6414c890 0x7f8f64154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f62ffd700 1 -- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f64063e30 msgr2=0x7f8f64162c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f62ffd700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f64063e30 0x7f8f64162c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f62ffd700 1 -- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f6414c890 msgr2=0x7f8f64154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f62ffd700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f6414c890 0x7f8f64154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f62ffd700 1 -- 172.21.15.29:0/214928225 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f58009370 con 0x7f8f6414c4b0 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f63fff700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f6414c890 0x7f8f64154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f62ffd700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f6414c4b0 0x7f8f64133750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f8f58009f50 tx=0x7f8f58017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f4b7fe700 1 -- 172.21.15.29:0/214928225 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800bae0 con 0x7f8f6414c4b0 2024-01-24T13:58:45.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f4b7fe700 1 -- 172.21.15.29:0/214928225 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f58000cc0 con 0x7f8f6414c4b0 2024-01-24T13:58:45.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f69bd4700 1 -- 172.21.15.29:0/214928225 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f640aa500 con 0x7f8f6414c4b0 2024-01-24T13:58:45.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f4b7fe700 1 -- 172.21.15.29:0/214928225 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5801c670 con 0x7f8f6414c4b0 2024-01-24T13:58:45.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.105+0000 7f8f69bd4700 1 -- 172.21.15.29:0/214928225 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f640aaac0 con 0x7f8f6414c4b0 2024-01-24T13:58:45.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.106+0000 7f8f69bd4700 1 -- 172.21.15.29:0/214928225 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f640475e0 con 0x7f8f6414c4b0 2024-01-24T13:58:45.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.107+0000 7f8f4b7fe700 1 -- 172.21.15.29:0/214928225 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8f5801c9e0 con 0x7f8f6414c4b0 2024-01-24T13:58:45.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.107+0000 7f8f4b7fe700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f5005f150 0x7f8f50061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.107+0000 7f8f4b7fe700 1 -- 172.21.15.29:0/214928225 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f8f5807c4e0 con 0x7f8f6414c4b0 2024-01-24T13:58:45.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.107+0000 7f8f637fe700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f5005f150 0x7f8f50061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.108+0000 7f8f637fe700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f5005f150 0x7f8f50061610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f8f640c10c0 tx=0x7f8f54009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:45.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.111+0000 7f8f4b7fe700 1 -- 172.21.15.29:0/214928225 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8f5804d4a0 con 0x7f8f6414c4b0 2024-01-24T13:58:45.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.338+0000 7f8f69bd4700 1 -- 172.21.15.29:0/214928225 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f64063630 con 0x7f8f6414c4b0 2024-01-24T13:58:45.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.339+0000 7f8f4b7fe700 1 -- 172.21.15.29:0/214928225 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4491 (secure 0 0 0) 0x7f8f58020020 con 0x7f8f6414c4b0 2024-01-24T13:58:45.339 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:45.339 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2038321939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2038321939},{"type":"v1","addr":"172.21.15.163:6836","nonce":2038321939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3675672874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3675672874},{"type":"v1","addr":"172.21.15.163:6837","nonce":3675672874}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1532435957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532435957},{"type":"v1","addr":"172.21.15.29:6834","nonce":1532435957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:58:19.190134+0000","modified":"2024-01-24T13:58:31.231527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1328130656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1328130656},{"type":"v1","addr":"172.21.15.29:6835","nonce":1328130656}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-24T13:58:45.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.341+0000 7f8f69bd4700 1 -- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f5005f150 msgr2=0x7f8f50061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.341+0000 7f8f69bd4700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f5005f150 0x7f8f50061610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f8f640c10c0 tx=0x7f8f54009040).stop 2024-01-24T13:58:45.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.341+0000 7f8f69bd4700 1 -- 172.21.15.29:0/214928225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f6414c4b0 msgr2=0x7f8f64133750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.341+0000 7f8f69bd4700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f6414c4b0 0x7f8f64133750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f8f58009f50 tx=0x7f8f58017000).stop 2024-01-24T13:58:45.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.341+0000 7f8f69bd4700 1 -- 172.21.15.29:0/214928225 shutdown_connections 2024-01-24T13:58:45.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.341+0000 7f8f69bd4700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f6414c890 0x7f8f64154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.341+0000 7f8f69bd4700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f64063e30 0x7f8f64162c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.341+0000 7f8f69bd4700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f5005f150 0x7f8f50061610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.341+0000 7f8f69bd4700 1 --2- 172.21.15.29:0/214928225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f6414c4b0 0x7f8f64133750 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.341+0000 7f8f69bd4700 1 -- 172.21.15.29:0/214928225 >> 172.21.15.29:0/214928225 conn(0x7f8f64064aa0 msgr2=0x7f8f640c6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:45.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.341+0000 7f8f69bd4700 1 -- 172.21.15.29:0/214928225 shutdown_connections 2024-01-24T13:58:45.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.341+0000 7f8f69bd4700 1 -- 172.21.15.29:0/214928225 wait complete. 2024-01-24T13:58:45.343 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 9 2024-01-24T13:58:45.355 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.484+0000 7f425bbca700 1 Processor -- start 2024-01-24T13:58:45.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.485+0000 7f425bbca700 1 -- start start 2024-01-24T13:58:45.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.485+0000 7f425bbca700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42541540a0 0x7f42540c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f425bbca700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42541550f0 0x7f42540627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f425bbca700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4254065360 con 0x7f4254062ca0 2024-01-24T13:58:45.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f425bbca700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42540a74f0 con 0x7f42541550f0 2024-01-24T13:58:45.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f425bbca700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42540a6ce0 con 0x7f42541540a0 2024-01-24T13:58:45.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f4259165700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42541550f0 0x7f42540627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f4259966700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42541540a0 0x7f42540c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f425a167700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4254062ca0 0x7f4254157c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:45794/0 (socket says 172.21.15.29:45794) 2024-01-24T13:58:45.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f4259966700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42541540a0 0x7f42540c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:45452/0 (socket says 172.21.15.29:45452) 2024-01-24T13:58:45.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f425a167700 1 -- 172.21.15.29:0/407928685 learned_addr learned my addr 172.21.15.29:0/407928685 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:45.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f4259165700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42541550f0 0x7f42540627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:58282/0 (socket says 172.21.15.29:58282) 2024-01-24T13:58:45.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f4259165700 1 -- 172.21.15.29:0/407928685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42541540a0 msgr2=0x7f42540c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:45.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f4259165700 1 --2- 172.21.15.29:0/407928685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42541540a0 0x7f42540c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f4259165700 1 -- 172.21.15.29:0/407928685 >> v1:172.21.15.163:6789/0 conn(0x7f4254062ca0 legacy=0x7f4254157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f4258964700 1 -- 172.21.15.29:0/407928685 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1833951422 0 0) 0x7f4254065360 con 0x7f4254062ca0 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f4259165700 1 -- 172.21.15.29:0/407928685 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4254063130 con 0x7f42541550f0 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f4259966700 1 --2- 172.21.15.29:0/407928685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42541540a0 0x7f42540c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.486+0000 7f4259165700 1 --2- 172.21.15.29:0/407928685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42541550f0 0x7f42540627b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f4244009760 tx=0x7f4244009060).ready entity=mon.1 client_cookie=a1881ad357146cdf server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.487+0000 7f4258964700 1 -- 172.21.15.29:0/407928685 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424400f510 con 0x7f42541550f0 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.487+0000 7f4258964700 1 -- 172.21.15.29:0/407928685 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424400f670 con 0x7f42541550f0 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.487+0000 7f4258964700 1 -- 172.21.15.29:0/407928685 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424400f9f0 con 0x7f42541550f0 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.487+0000 7f425bbca700 1 -- 172.21.15.29:0/407928685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42541550f0 msgr2=0x7f42540627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.487+0000 7f425bbca700 1 --2- 172.21.15.29:0/407928685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42541550f0 0x7f42540627b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f4244009760 tx=0x7f4244009060).stop 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.487+0000 7f425bbca700 1 -- 172.21.15.29:0/407928685 shutdown_connections 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.487+0000 7f425bbca700 1 --2- 172.21.15.29:0/407928685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42541540a0 0x7f42540c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.487+0000 7f425bbca700 1 --2- 172.21.15.29:0/407928685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42541550f0 0x7f42540627b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.487+0000 7f425bbca700 1 -- 172.21.15.29:0/407928685 >> 172.21.15.29:0/407928685 conn(0x7f42540c85c0 msgr2=0x7f42540c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:45.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.487+0000 7f425bbca700 1 -- 172.21.15.29:0/407928685 shutdown_connections 2024-01-24T13:58:45.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.487+0000 7f425bbca700 1 -- 172.21.15.29:0/407928685 wait complete. 2024-01-24T13:58:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.488+0000 7f425bbca700 1 Processor -- start 2024-01-24T13:58:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.489+0000 7f425bbca700 1 -- start start 2024-01-24T13:58:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.489+0000 7f425bbca700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4254062ca0 0x7f42541456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.489+0000 7f425bbca700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42541540a0 0x7f42540a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.489+0000 7f425bbca700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42541550f0 0x7f4254065cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.489+0000 7f425bbca700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4254145bc0 con 0x7f42541540a0 2024-01-24T13:58:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.489+0000 7f425bbca700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4254168d20 con 0x7f42541550f0 2024-01-24T13:58:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.489+0000 7f425bbca700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4254167e30 con 0x7f4254062ca0 2024-01-24T13:58:45.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.489+0000 7f4259165700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42541540a0 0x7f42540a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.489+0000 7f4259165700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42541540a0 0x7f42540a77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39604/0 (socket says 172.21.15.29:39604) 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f4259165700 1 -- 172.21.15.29:0/1420717888 learned_addr learned my addr 172.21.15.29:0/1420717888 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f4259966700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4254062ca0 0x7f42541456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f425a167700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42541550f0 0x7f4254065cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f4259165700 1 -- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4254062ca0 msgr2=0x7f42541456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f4259165700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4254062ca0 0x7f42541456d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f4259165700 1 -- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42541550f0 msgr2=0x7f4254065cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f4259165700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42541550f0 0x7f4254065cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f4259165700 1 -- 172.21.15.29:0/1420717888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42500092a0 con 0x7f42541540a0 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f4259165700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42541540a0 0x7f42540a77c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f4254069030 tx=0x7f4244017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f4249ffb700 1 -- 172.21.15.29:0/1420717888 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424400f7e0 con 0x7f42541540a0 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f4249ffb700 1 -- 172.21.15.29:0/1420717888 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424400f940 con 0x7f42541540a0 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f4249ffb700 1 -- 172.21.15.29:0/1420717888 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424401b660 con 0x7f42541540a0 2024-01-24T13:58:45.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.490+0000 7f425bbca700 1 -- 172.21.15.29:0/1420717888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42440092a0 con 0x7f42541540a0 2024-01-24T13:58:45.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.491+0000 7f425bbca700 1 -- 172.21.15.29:0/1420717888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42541683f0 con 0x7f42541540a0 2024-01-24T13:58:45.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.492+0000 7f425bbca700 1 -- 172.21.15.29:0/1420717888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42540475e0 con 0x7f42541540a0 2024-01-24T13:58:45.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.492+0000 7f4249ffb700 1 -- 172.21.15.29:0/1420717888 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f424401ba50 con 0x7f42541540a0 2024-01-24T13:58:45.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.493+0000 7f4249ffb700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f423805f150 0x7f4238061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.493+0000 7f4249ffb700 1 -- 172.21.15.29:0/1420717888 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f424407c050 con 0x7f42541540a0 2024-01-24T13:58:45.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.493+0000 7f4259966700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f423805f150 0x7f4238061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.493+0000 7f4259966700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f423805f150 0x7f4238061610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f4250009dc0 tx=0x7f4250009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:45.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.497+0000 7f4249ffb700 1 -- 172.21.15.29:0/1420717888 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f424404cfd0 con 0x7f42541540a0 2024-01-24T13:58:45.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.678+0000 7f425bbca700 1 -- 172.21.15.29:0/1420717888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f425406a5f0 con 0x7f42541540a0 2024-01-24T13:58:45.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.679+0000 7f4249ffb700 1 -- 172.21.15.29:0/1420717888 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7f424401f020 con 0x7f42541540a0 2024-01-24T13:58:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:45.679 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:45.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.681+0000 7f425bbca700 1 -- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f423805f150 msgr2=0x7f4238061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.681+0000 7f425bbca700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f423805f150 0x7f4238061610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f4250009dc0 tx=0x7f4250009060).stop 2024-01-24T13:58:45.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.681+0000 7f425bbca700 1 -- 172.21.15.29:0/1420717888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42541540a0 msgr2=0x7f42540a77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.681+0000 7f425bbca700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42541540a0 0x7f42540a77c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f4254069030 tx=0x7f4244017000).stop 2024-01-24T13:58:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.681+0000 7f425bbca700 1 -- 172.21.15.29:0/1420717888 shutdown_connections 2024-01-24T13:58:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.681+0000 7f425bbca700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42541550f0 0x7f4254065cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.681+0000 7f425bbca700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4254062ca0 0x7f42541456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.681+0000 7f425bbca700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f423805f150 0x7f4238061610 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.681+0000 7f425bbca700 1 --2- 172.21.15.29:0/1420717888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42541540a0 0x7f42540a77c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.681+0000 7f425bbca700 1 -- 172.21.15.29:0/1420717888 >> 172.21.15.29:0/1420717888 conn(0x7f42540c85c0 msgr2=0x7f4254153370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.681+0000 7f425bbca700 1 -- 172.21.15.29:0/1420717888 shutdown_connections 2024-01-24T13:58:45.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.681+0000 7f425bbca700 1 -- 172.21.15.29:0/1420717888 wait complete. 2024-01-24T13:58:45.695 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:58:45.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.825+0000 7fc949b79700 1 Processor -- start 2024-01-24T13:58:45.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.825+0000 7fc949b79700 1 -- start start 2024-01-24T13:58:45.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.826+0000 7fc949b79700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc94414c4b0 0x7fc9440627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.826+0000 7fc949b79700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc944063e30 0x7fc9440689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.826+0000 7fc949b79700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9440639a0 con 0x7fc94414c890 2024-01-24T13:58:45.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.826+0000 7fc949b79700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc944062d90 con 0x7fc94414c4b0 2024-01-24T13:58:45.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.826+0000 7fc949b79700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc944063070 con 0x7fc944063e30 2024-01-24T13:58:45.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.826+0000 7fc9437fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc94414c4b0 0x7fc9440627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.826+0000 7fc942ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc94414c890 0x7fc944153060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34458/0 (socket says 172.21.15.29:34458) 2024-01-24T13:58:45.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.826+0000 7fc9437fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc94414c4b0 0x7fc9440627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39616/0 (socket says 172.21.15.29:39616) 2024-01-24T13:58:45.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.826+0000 7fc942ffd700 1 -- 172.21.15.29:0/3147676891 learned_addr learned my addr 172.21.15.29:0/3147676891 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:45.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.827+0000 7fc943fff700 1 --2- 172.21.15.29:0/3147676891 >> v2:172.21.15.163:3300/0 conn(0x7fc944063e30 0x7fc9440689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.827+0000 7fc9437fe700 1 -- 172.21.15.29:0/3147676891 >> v2:172.21.15.163:3300/0 conn(0x7fc944063e30 msgr2=0x7fc9440689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:45.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.827+0000 7fc9437fe700 1 --2- 172.21.15.29:0/3147676891 >> v2:172.21.15.163:3300/0 conn(0x7fc944063e30 0x7fc9440689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.827+0000 7fc9437fe700 1 -- 172.21.15.29:0/3147676891 >> v1:172.21.15.29:6789/0 conn(0x7fc94414c890 legacy=0x7fc944153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.827+0000 7fc9437fe700 1 -- 172.21.15.29:0/3147676891 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944063350 con 0x7fc94414c4b0 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.827+0000 7fc9427fc700 1 -- 172.21.15.29:0/3147676891 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833291801 0 0) 0x7fc9440639a0 con 0x7fc94414c890 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.827+0000 7fc943fff700 1 --2- 172.21.15.29:0/3147676891 >> v2:172.21.15.163:3300/0 conn(0x7fc944063e30 0x7fc9440689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.827+0000 7fc9437fe700 1 --2- 172.21.15.29:0/3147676891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc94414c4b0 0x7fc9440627b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fc944062ca0 tx=0x7fc934009060).ready entity=mon.0 client_cookie=38281ed643616a68 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.827+0000 7fc9427fc700 1 -- 172.21.15.29:0/3147676891 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93400b580 con 0x7fc94414c4b0 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.827+0000 7fc9427fc700 1 -- 172.21.15.29:0/3147676891 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93400b6e0 con 0x7fc94414c4b0 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.827+0000 7fc9427fc700 1 -- 172.21.15.29:0/3147676891 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93400ba60 con 0x7fc94414c4b0 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.828+0000 7fc949b79700 1 -- 172.21.15.29:0/3147676891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc94414c4b0 msgr2=0x7fc9440627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.828+0000 7fc949b79700 1 --2- 172.21.15.29:0/3147676891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc94414c4b0 0x7fc9440627b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fc944062ca0 tx=0x7fc934009060).stop 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.828+0000 7fc949b79700 1 -- 172.21.15.29:0/3147676891 shutdown_connections 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.828+0000 7fc949b79700 1 --2- 172.21.15.29:0/3147676891 >> v2:172.21.15.163:3300/0 conn(0x7fc944063e30 0x7fc9440689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.828+0000 7fc949b79700 1 --2- 172.21.15.29:0/3147676891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc94414c4b0 0x7fc9440627b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.828+0000 7fc949b79700 1 -- 172.21.15.29:0/3147676891 >> 172.21.15.29:0/3147676891 conn(0x7fc944064aa0 msgr2=0x7fc944064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:45.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.828+0000 7fc949b79700 1 -- 172.21.15.29:0/3147676891 shutdown_connections 2024-01-24T13:58:45.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.828+0000 7fc949b79700 1 -- 172.21.15.29:0/3147676891 wait complete. 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.829+0000 7fc949b79700 1 Processor -- start 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.829+0000 7fc949b79700 1 -- start start 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc949b79700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc944063e30 0x7fc9440c0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc949b79700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc94414c4b0 0x7fc944134a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc949b79700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc94414c890 0x7fc944134f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc949b79700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc944135400 con 0x7fc94414c4b0 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc949b79700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc944168210 con 0x7fc94414c890 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc949b79700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc944167640 con 0x7fc944063e30 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc942ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc94414c4b0 0x7fc944134a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc942ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc94414c4b0 0x7fc944134a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39628/0 (socket says 172.21.15.29:39628) 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc942ffd700 1 -- 172.21.15.29:0/168055501 learned_addr learned my addr 172.21.15.29:0/168055501 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc943fff700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc94414c890 0x7fc944134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc9437fe700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc944063e30 0x7fc9440c0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc942ffd700 1 -- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc944063e30 msgr2=0x7fc9440c0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc942ffd700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc944063e30 0x7fc9440c0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc942ffd700 1 -- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc94414c890 msgr2=0x7fc944134f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc942ffd700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc94414c890 0x7fc944134f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.830+0000 7fc942ffd700 1 -- 172.21.15.29:0/168055501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9340092a0 con 0x7fc94414c4b0 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.831+0000 7fc942ffd700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc94414c4b0 0x7fc944134a20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fc938001eb0 tx=0x7fc938007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.831+0000 7fc92b7fe700 1 -- 172.21.15.29:0/168055501 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380070d0 con 0x7fc94414c4b0 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.831+0000 7fc92b7fe700 1 -- 172.21.15.29:0/168055501 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938007230 con 0x7fc94414c4b0 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.831+0000 7fc949b79700 1 -- 172.21.15.29:0/168055501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9441678e0 con 0x7fc94414c4b0 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.831+0000 7fc92b7fe700 1 -- 172.21.15.29:0/168055501 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380075d0 con 0x7fc94414c4b0 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.831+0000 7fc949b79700 1 -- 172.21.15.29:0/168055501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc944167ea0 con 0x7fc94414c4b0 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.832+0000 7fc949b79700 1 -- 172.21.15.29:0/168055501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9441685a0 con 0x7fc94414c4b0 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.832+0000 7fc92b7fe700 1 -- 172.21.15.29:0/168055501 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc938005bc0 con 0x7fc94414c4b0 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.833+0000 7fc92b7fe700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc93005f150 0x7fc930061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.833+0000 7fc92b7fe700 1 -- 172.21.15.29:0/168055501 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fc938079cf0 con 0x7fc94414c4b0 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.833+0000 7fc9437fe700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc93005f150 0x7fc930061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:45.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.833+0000 7fc9437fe700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc93005f150 0x7fc930061610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc934009fd0 tx=0x7fc934017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:45.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:45.837+0000 7fc92b7fe700 1 -- 172.21.15.29:0/168055501 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc93807e070 con 0x7fc94414c4b0 2024-01-24T13:58:46.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.063+0000 7fc949b79700 1 -- 172.21.15.29:0/168055501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9440475e0 con 0x7fc94414c4b0 2024-01-24T13:58:46.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.064+0000 7fc92b7fe700 1 -- 172.21.15.29:0/168055501 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4491 (secure 0 0 0) 0x7fc93804acb0 con 0x7fc94414c4b0 2024-01-24T13:58:46.064 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:46.065 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2038321939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2038321939},{"type":"v1","addr":"172.21.15.163:6836","nonce":2038321939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3675672874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3675672874},{"type":"v1","addr":"172.21.15.163:6837","nonce":3675672874}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1532435957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532435957},{"type":"v1","addr":"172.21.15.29:6834","nonce":1532435957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:58:19.190134+0000","modified":"2024-01-24T13:58:31.231527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1328130656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1328130656},{"type":"v1","addr":"172.21.15.29:6835","nonce":1328130656}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-24T13:58:46.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.066+0000 7fc949b79700 1 -- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc93005f150 msgr2=0x7fc930061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.066+0000 7fc949b79700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc93005f150 0x7fc930061610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc934009fd0 tx=0x7fc934017000).stop 2024-01-24T13:58:46.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.066+0000 7fc949b79700 1 -- 172.21.15.29:0/168055501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc94414c4b0 msgr2=0x7fc944134a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.066+0000 7fc949b79700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc94414c4b0 0x7fc944134a20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fc938001eb0 tx=0x7fc938007e90).stop 2024-01-24T13:58:46.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.066+0000 7fc949b79700 1 -- 172.21.15.29:0/168055501 shutdown_connections 2024-01-24T13:58:46.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.066+0000 7fc949b79700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc94414c890 0x7fc944134f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.066+0000 7fc949b79700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc944063e30 0x7fc9440c0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.066+0000 7fc949b79700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc93005f150 0x7fc930061610 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.066+0000 7fc949b79700 1 --2- 172.21.15.29:0/168055501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc94414c4b0 0x7fc944134a20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.066+0000 7fc949b79700 1 -- 172.21.15.29:0/168055501 >> 172.21.15.29:0/168055501 conn(0x7fc944064aa0 msgr2=0x7fc9440c65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:46.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.066+0000 7fc949b79700 1 -- 172.21.15.29:0/168055501 shutdown_connections 2024-01-24T13:58:46.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.066+0000 7fc949b79700 1 -- 172.21.15.29:0/168055501 wait complete. 2024-01-24T13:58:46.068 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 9 2024-01-24T13:58:46.483 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T13:58:19.190134+0000', 'modified': '2024-01-24T13:58:31.231527+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/1328130656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 1328130656}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 1328130656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-24T13:58:46.483 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T13:58:46.484 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-24T13:58:46.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.623+0000 7fcce0a4a700 1 Processor -- start 2024-01-24T13:58:46.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.623+0000 7fcce0a4a700 1 -- start start 2024-01-24T13:58:46.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.624+0000 7fcce0a4a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccdc13e220 0x7fccdc0639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.624+0000 7fcce0a4a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccdc0bd770 con 0x7fccdc13de40 2024-01-24T13:58:46.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.624+0000 7fcce0a4a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccdc063540 con 0x7fccdc063ec0 2024-01-24T13:58:46.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.624+0000 7fcce0a4a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc062930 con 0x7fccdc13e220 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.624+0000 7fccda59c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fccdc13de40 0x7fccdc0bd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34472/0 (socket says 172.21.15.29:34472) 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.624+0000 7fccda59c700 1 -- 172.21.15.29:0/3316846880 learned_addr learned my addr 172.21.15.29:0/3316846880 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.624+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/3316846880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccdc13e220 0x7fccdc0639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.624+0000 7fccd959a700 1 -- 172.21.15.29:0/3316846880 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254463058 0 0) 0x7fccdc0bd770 con 0x7fccdc13de40 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.624+0000 7fccd959a700 1 -- 172.21.15.29:0/3316846880 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccc40033f0 con 0x7fccdc13de40 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.625+0000 7fccd959a700 1 -- 172.21.15.29:0/3316846880 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408408522 0 0) 0x7fccdc063540 con 0x7fccdc063ec0 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.625+0000 7fccd959a700 1 -- 172.21.15.29:0/3316846880 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccdc0bd770 con 0x7fccdc063ec0 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.625+0000 7fccd9d9b700 1 -- 172.21.15.29:0/3316846880 >> v1:172.21.15.163:6789/0 conn(0x7fccdc063ec0 legacy=0x7fccdc0c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.625+0000 7fccd9d9b700 1 -- 172.21.15.29:0/3316846880 >> v1:172.21.15.29:6789/0 conn(0x7fccdc13de40 legacy=0x7fccdc0bd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.625+0000 7fccd9d9b700 1 -- 172.21.15.29:0/3316846880 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc062c10 con 0x7fccdc13e220 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.625+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/3316846880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccdc13e220 0x7fccdc0639d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fccd0009790 tx=0x7fccd0009060).ready entity=mon.2 client_cookie=49271c8d28ebb964 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.625+0000 7fccd959a700 1 -- 172.21.15.29:0/3316846880 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd000f510 con 0x7fccdc13e220 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.625+0000 7fccd959a700 1 -- 172.21.15.29:0/3316846880 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd000f670 con 0x7fccdc13e220 2024-01-24T13:58:46.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.625+0000 7fccd959a700 1 -- 172.21.15.29:0/3316846880 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd000f9f0 con 0x7fccdc13e220 2024-01-24T13:58:46.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.626+0000 7fcce0a4a700 1 -- 172.21.15.29:0/3316846880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccdc13e220 msgr2=0x7fccdc0639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.626+0000 7fcce0a4a700 1 --2- 172.21.15.29:0/3316846880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccdc13e220 0x7fccdc0639d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fccd0009790 tx=0x7fccd0009060).stop 2024-01-24T13:58:46.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.626+0000 7fcce0a4a700 1 -- 172.21.15.29:0/3316846880 shutdown_connections 2024-01-24T13:58:46.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.626+0000 7fcce0a4a700 1 --2- 172.21.15.29:0/3316846880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccdc13e220 0x7fccdc0639d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.626+0000 7fcce0a4a700 1 -- 172.21.15.29:0/3316846880 >> 172.21.15.29:0/3316846880 conn(0x7fccdc0c3cd0 msgr2=0x7fccdc0c40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:46.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.626+0000 7fcce0a4a700 1 -- 172.21.15.29:0/3316846880 shutdown_connections 2024-01-24T13:58:46.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.626+0000 7fcce0a4a700 1 -- 172.21.15.29:0/3316846880 wait complete. 2024-01-24T13:58:46.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.627+0000 7fcce0a4a700 1 Processor -- start 2024-01-24T13:58:46.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.627+0000 7fcce0a4a700 1 -- start start 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fcce0a4a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccdc063ec0 0x7fccdc0bfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fcce0a4a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccdc13de40 0x7fccdc0be2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fcce0a4a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccdc13e220 0x7fccdc1583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fcce0a4a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc0be7c0 con 0x7fccdc13e220 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fcce0a4a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc1588e0 con 0x7fccdc13de40 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fcce0a4a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc0be9d0 con 0x7fccdc063ec0 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fccdad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccdc13e220 0x7fccdc1583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fccdad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccdc13e220 0x7fccdc1583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39644/0 (socket says 172.21.15.29:39644) 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fccdad9d700 1 -- 172.21.15.29:0/359005035 learned_addr learned my addr 172.21.15.29:0/359005035 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fccda59c700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccdc063ec0 0x7fccdc0bfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccdc13de40 0x7fccdc0be2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fccdad9d700 1 -- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccdc063ec0 msgr2=0x7fccdc0bfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fccdad9d700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccdc063ec0 0x7fccdc0bfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fccdad9d700 1 -- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccdc13de40 msgr2=0x7fccdc0be2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.628+0000 7fccdad9d700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccdc13de40 0x7fccdc0be2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.629+0000 7fccdad9d700 1 -- 172.21.15.29:0/359005035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd00092a0 con 0x7fccdc13e220 2024-01-24T13:58:46.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.629+0000 7fccdad9d700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccdc13e220 0x7fccdc1583f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fccd400c7c0 tx=0x7fccd4007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:46.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.629+0000 7fccca7fc700 1 -- 172.21.15.29:0/359005035 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd400e4a0 con 0x7fccdc13e220 2024-01-24T13:58:46.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.629+0000 7fccca7fc700 1 -- 172.21.15.29:0/359005035 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd400e600 con 0x7fccdc13e220 2024-01-24T13:58:46.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.629+0000 7fcce0a4a700 1 -- 172.21.15.29:0/359005035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccdc0becb0 con 0x7fccdc13e220 2024-01-24T13:58:46.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.629+0000 7fccca7fc700 1 -- 172.21.15.29:0/359005035 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd400e980 con 0x7fccdc13e220 2024-01-24T13:58:46.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.629+0000 7fcce0a4a700 1 -- 172.21.15.29:0/359005035 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccdc0bf270 con 0x7fccdc13e220 2024-01-24T13:58:46.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.630+0000 7fccca7fc700 1 -- 172.21.15.29:0/359005035 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fccd400ed30 con 0x7fccdc13e220 2024-01-24T13:58:46.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.631+0000 7fccca7fc700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fccbc05f150 0x7fccbc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.631+0000 7fccca7fc700 1 -- 172.21.15.29:0/359005035 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fccd4079b50 con 0x7fccdc13e220 2024-01-24T13:58:46.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.631+0000 7fccda59c700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fccbc05f150 0x7fccbc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.631+0000 7fccda59c700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fccbc05f150 0x7fccbc061610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fcccc005750 tx=0x7fcccc009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:46.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.633+0000 7fccf2415b80 1 Processor -- start 2024-01-24T13:58:46.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.633+0000 7fccf2415b80 1 -- start start 2024-01-24T13:58:46.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccf2415b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 0x55c518083a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccf2415b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 0x55c51807e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccf2415b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c51807edf0 0x55c51807f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccf2415b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c517ffa3d0 con 0x55c518083620 2024-01-24T13:58:46.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccf2415b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c517fbfbe0 con 0x55c5180cc2c0 2024-01-24T13:58:46.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccf2415b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c517fd5260 con 0x55c51807edf0 2024-01-24T13:58:46.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccda59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 0x55c518083a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccda59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 0x55c518083a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39656/0 (socket says 172.21.15.29:39656) 2024-01-24T13:58:46.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccda59c700 1 -- 172.21.15.29:0/3076559040 learned_addr learned my addr 172.21.15.29:0/3076559040 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccdad9d700 1 --2- 172.21.15.29:0/3076559040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c51807edf0 0x55c51807f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/3076559040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 0x55c51807e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccda59c700 1 -- 172.21.15.29:0/3076559040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c51807edf0 msgr2=0x55c51807f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccda59c700 1 --2- 172.21.15.29:0/3076559040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c51807edf0 0x55c51807f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccda59c700 1 -- 172.21.15.29:0/3076559040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 msgr2=0x55c51807e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccda59c700 1 --2- 172.21.15.29:0/3076559040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 0x55c51807e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.634+0000 7fccda59c700 1 -- 172.21.15.29:0/3076559040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c517fce2f0 con 0x55c518083620 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.635+0000 7fccda59c700 1 --2- 172.21.15.29:0/3076559040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 0x55c518083a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fcccc00f870 tx=0x7fcccc013000).ready entity=mon.0 client_cookie=68a086eb63733624 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.635+0000 7fccbb7fe700 1 -- 172.21.15.29:0/3076559040 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc013600 con 0x55c518083620 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.635+0000 7fccbb7fe700 1 -- 172.21.15.29:0/3076559040 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc013760 con 0x55c518083620 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.635+0000 7fccbb7fe700 1 -- 172.21.15.29:0/3076559040 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc013ae0 con 0x55c518083620 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.635+0000 7fccf2415b80 1 -- 172.21.15.29:0/3076559040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 msgr2=0x55c518083a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.635+0000 7fccf2415b80 1 --2- 172.21.15.29:0/3076559040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 0x55c518083a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fcccc00f870 tx=0x7fcccc013000).stop 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.636+0000 7fccf2415b80 1 -- 172.21.15.29:0/3076559040 shutdown_connections 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.636+0000 7fccf2415b80 1 --2- 172.21.15.29:0/3076559040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c51807edf0 0x55c51807f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.636+0000 7fccf2415b80 1 --2- 172.21.15.29:0/3076559040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 0x55c518083a00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.636+0000 7fccf2415b80 1 --2- 172.21.15.29:0/3076559040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 0x55c51807e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.636+0000 7fccf2415b80 1 -- 172.21.15.29:0/3076559040 >> 172.21.15.29:0/3076559040 conn(0x55c517eec2e0 msgr2=0x55c5180cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:46.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.636+0000 7fccf2415b80 1 -- 172.21.15.29:0/3076559040 shutdown_connections 2024-01-24T13:58:46.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.636+0000 7fccf2415b80 1 -- 172.21.15.29:0/3076559040 wait complete. 2024-01-24T13:58:46.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.636+0000 7fccf2415b80 1 -- start start 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.636+0000 7fccf2415b80 1 Processor -- start 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.637+0000 7fccf2415b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 0x55c5180afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.637+0000 7fccf2415b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c518083620 0x55c5180c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.637+0000 7fccf2415b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c5180cc2c0 0x55c5180a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.637+0000 7fccf2415b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c517fd53a0 con 0x55c51807edf0 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.637+0000 7fccf2415b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c517fd5620 con 0x55c518083620 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.637+0000 7fccf2415b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c517fa0990 con 0x55c5180cc2c0 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.637+0000 7fccf2415b80 10 client.0 ll_register_callbacks cb 0x55c518043610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.637+0000 7fccda59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 0x55c5180afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccda59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 0x55c5180afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39658/0 (socket says 172.21.15.29:39658) 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccda59c700 1 -- 172.21.15.29:0/2919683496 learned_addr learned my addr 172.21.15.29:0/2919683496 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccdad9d700 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c5180cc2c0 0x55c5180a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c518083620 0x55c5180c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccda59c700 1 -- 172.21.15.29:0/2919683496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c5180cc2c0 msgr2=0x55c5180a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:46.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccda59c700 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c5180cc2c0 0x55c5180a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccda59c700 1 -- 172.21.15.29:0/2919683496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c518083620 msgr2=0x55c5180c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccda59c700 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c518083620 0x55c5180c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccda59c700 1 -- 172.21.15.29:0/2919683496 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc40033f0 con 0x55c51807edf0 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccdad9d700 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c5180cc2c0 0x55c5180a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c518083620 0x55c5180c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccda59c700 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 0x55c5180afb60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x55c5180cc720 tx=0x7fcccc019000).ready entity=mon.0 client_cookie=b8cccdc9b884a524 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccb8ff9700 1 -- 172.21.15.29:0/2919683496 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc005240 con 0x55c51807edf0 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccb8ff9700 1 -- 172.21.15.29:0/2919683496 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fcccc0053a0 con 0x55c51807edf0 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccf2415b80 1 -- 172.21.15.29:0/2919683496 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c51807ddf0 con 0x55c51807edf0 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccb8ff9700 1 -- 172.21.15.29:0/2919683496 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcccc005560 con 0x55c51807edf0 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccf2415b80 10 client.4776 fetch_fsmap learned FSMap version 9 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.638+0000 7fccf2415b80 1 -- 172.21.15.29:0/2919683496 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55c517fc9520 con 0x55c51807edf0 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.639+0000 7fccb8ff9700 1 -- 172.21.15.29:0/2919683496 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c51807ddf0 con 0x55c51807edf0 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.639+0000 7fccb8ff9700 1 -- 172.21.15.29:0/2919683496 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 9) v1 ==== 3108+0+0 (secure 0 0 0) 0x7fcccc0048c0 con 0x55c51807edf0 2024-01-24T13:58:46.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.639+0000 7fccf2415b80 10 client.4776 fetch_fsmap finished waiting for FSMap version 9 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.639+0000 7fccf2415b80 10 client.4776 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.639+0000 7fccf2415b80 20 client.4776 populate_metadata read hostname 'smithi029' 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.639+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x55c5180c74c0 0x55c5180c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.639+0000 7fccf2415b80 4 client.4776 mds_command: new command op to 4281 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.639+0000 7fccf2415b80 1 -- 172.21.15.29:0/2919683496 --> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c517fcad60 con 0x55c5180c74c0 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.639+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x55c5180c74c0 0x55c5180c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.639+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x55c5180c74c0 0x55c5180c78a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7c6389bde443d2c2 server_cookie=663ce1f6353be439 in_seq=0 out_seq=0 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.639+0000 7fccb8ff9700 10 client.4776 ms_handle_connect on v2:172.21.15.29:6833/1328130656 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccb8ff9700 1 -- 172.21.15.29:0/2919683496 <== mds.0 v2:172.21.15.29:6833/1328130656 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x55c517fcad60 con 0x55c5180c74c0 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccb8ff9700 10 client.4776 handle_command_reply: tid=0 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccf2415b80 1 client.4776 shutdown 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccf2415b80 1 -- 172.21.15.29:0/2919683496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 msgr2=0x55c5180afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 0x55c5180afb60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x55c5180cc720 tx=0x7fcccc019000).stop 2024-01-24T13:58:46.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccf2415b80 1 -- 172.21.15.29:0/2919683496 shutdown_connections 2024-01-24T13:58:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x55c5180c74c0 0x55c5180c78a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c5180cc2c0 0x55c5180a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 0x55c5180afb60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccb8ff9700 0 client.4776 ms_handle_reset on v2:172.21.15.29:6833/1328130656 2024-01-24T13:58:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2919683496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c518083620 0x55c5180c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccf2415b80 1 -- 172.21.15.29:0/2919683496 >> 172.21.15.29:0/2919683496 conn(0x55c517eec2e0 msgr2=0x55c518081690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccf2415b80 1 -- 172.21.15.29:0/2919683496 shutdown_connections 2024-01-24T13:58:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccf2415b80 1 -- 172.21.15.29:0/2919683496 wait complete. 2024-01-24T13:58:46.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.640+0000 7fccf2415b80 20 client.4776 trim_cache size 0 max 16384 2024-01-24T13:58:46.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.658+0000 7fccf2415b80 1 Processor -- start 2024-01-24T13:58:46.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.658+0000 7fccf2415b80 1 -- start start 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.658+0000 7fccf2415b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c51807edf0 0x55c5180cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.658+0000 7fccf2415b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 0x55c51807a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccf2415b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 0x55c5180aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccf2415b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5180b03f0 con 0x55c518083620 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccf2415b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5180859a0 con 0x55c5180cc2c0 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccf2415b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5180780e0 con 0x55c51807edf0 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccd9d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 0x55c51807a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccd9d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 0x55c51807a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39662/0 (socket says 172.21.15.29:39662) 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccd9d9b700 1 -- 172.21.15.29:0/2471228145 learned_addr learned my addr 172.21.15.29:0/2471228145 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccda59c700 1 --2- 172.21.15.29:0/2471228145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c51807edf0 0x55c5180cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccdad9d700 1 --2- 172.21.15.29:0/2471228145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 0x55c5180aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccd9d9b700 1 -- 172.21.15.29:0/2471228145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c51807edf0 msgr2=0x55c5180cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:46.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/2471228145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c51807edf0 0x55c5180cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccd9d9b700 1 -- 172.21.15.29:0/2471228145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 msgr2=0x55c5180aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/2471228145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 0x55c5180aa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccd9d9b700 1 -- 172.21.15.29:0/2471228145 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c517fc9270 con 0x55c518083620 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/2471228145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 0x55c51807a1f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fccd0011000 tx=0x7fccd001e040).ready entity=mon.0 client_cookie=54d0fac9f4bf39fd server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccbbfff700 1 -- 172.21.15.29:0/2471228145 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd000f7e0 con 0x55c518083620 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccbbfff700 1 -- 172.21.15.29:0/2471228145 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd0022040 con 0x55c518083620 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.659+0000 7fccbbfff700 1 -- 172.21.15.29:0/2471228145 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd000f960 con 0x55c518083620 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.660+0000 7fccf2415b80 1 -- 172.21.15.29:0/2471228145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 msgr2=0x55c51807a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.660+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2471228145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 0x55c51807a1f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fccd0011000 tx=0x7fccd001e040).stop 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.660+0000 7fccf2415b80 1 -- 172.21.15.29:0/2471228145 shutdown_connections 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.660+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2471228145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 0x55c5180aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.660+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2471228145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c51807edf0 0x55c5180cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.660+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2471228145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c518083620 0x55c51807a1f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.660+0000 7fccf2415b80 1 -- 172.21.15.29:0/2471228145 >> 172.21.15.29:0/2471228145 conn(0x55c517eec2e0 msgr2=0x55c5180c7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:46.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.660+0000 7fccf2415b80 1 -- 172.21.15.29:0/2471228145 shutdown_connections 2024-01-24T13:58:46.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.660+0000 7fccf2415b80 1 -- 172.21.15.29:0/2471228145 wait complete. 2024-01-24T13:58:46.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.661+0000 7fccf2415b80 1 -- start start 2024-01-24T13:58:46.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.661+0000 7fccf2415b80 1 Processor -- start 2024-01-24T13:58:46.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccf2415b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 0x55c5180d7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccf2415b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c518083620 0x55c51812da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccf2415b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 0x55c518083a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccf2415b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c51812e670 con 0x55c51807edf0 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccf2415b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5180a9210 con 0x55c5180cc2c0 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccf2415b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5180a9460 con 0x55c518083620 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccf2415b80 10 client.0 ll_register_callbacks cb 0x55c518043610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccda59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 0x55c5180d7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccda59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 0x55c5180d7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39670/0 (socket says 172.21.15.29:39670) 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccda59c700 1 -- 172.21.15.29:0/2451749400 learned_addr learned my addr 172.21.15.29:0/2451749400 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c518083620 0x55c51812da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccdad9d700 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 0x55c518083a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccda59c700 1 -- 172.21.15.29:0/2451749400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c518083620 msgr2=0x55c51812da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccda59c700 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c518083620 0x55c51812da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccda59c700 1 -- 172.21.15.29:0/2451749400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 msgr2=0x55c518083a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccda59c700 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 0x55c518083a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccda59c700 1 -- 172.21.15.29:0/2451749400 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccdc062c10 con 0x55c51807edf0 2024-01-24T13:58:46.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccda59c700 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 0x55c5180d7960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fcccc00f5c0 tx=0x7fcccc026000).ready entity=mon.0 client_cookie=d92ab779a23fe684 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccb8ff9700 1 -- 172.21.15.29:0/2451749400 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc024070 con 0x55c51807edf0 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.662+0000 7fccb8ff9700 1 -- 172.21.15.29:0/2451749400 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fcccc0048c0 con 0x55c51807edf0 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccf2415b80 1 -- 172.21.15.29:0/2451749400 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcccc013760 con 0x55c51807edf0 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccb8ff9700 1 -- 172.21.15.29:0/2451749400 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcccc005450 con 0x55c51807edf0 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccf2415b80 10 client.4782 fetch_fsmap learned FSMap version 9 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccf2415b80 1 -- 172.21.15.29:0/2451749400 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55c517e42f80 con 0x55c51807edf0 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccb8ff9700 1 -- 172.21.15.29:0/2451749400 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc013760 con 0x55c51807edf0 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccb8ff9700 1 -- 172.21.15.29:0/2451749400 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 9) v1 ==== 3108+0+0 (secure 0 0 0) 0x7fcccc020500 con 0x55c51807edf0 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccf2415b80 10 client.4782 fetch_fsmap finished waiting for FSMap version 9 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccf2415b80 10 client.4782 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccf2415b80 20 client.4782 populate_metadata read hostname 'smithi029' 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x55c5180a8c80 0x55c518083ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:46.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccf2415b80 4 client.4782 mds_command: new command op to 4281 tid=0[{"prefix": "status"}] 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccf2415b80 1 -- 172.21.15.29:0/2451749400 --> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c517fc17d0 con 0x55c5180a8c80 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x55c5180a8c80 0x55c518083ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccb8ff9700 10 client.4782 ms_handle_connect on v2:172.21.15.29:6833/1328130656 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.663+0000 7fccd9d9b700 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x55c5180a8c80 0x55c518083ef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=9590184ff81a5c15 server_cookie=17526038e50d550d in_seq=0 out_seq=0 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccb8ff9700 1 -- 172.21.15.29:0/2451749400 <== mds.0 v2:172.21.15.29:6833/1328130656 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55c517fc17d0 con 0x55c5180a8c80 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccb8ff9700 10 client.4782 handle_command_reply: tid=0 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccf2415b80 1 client.4782 shutdown 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccf2415b80 1 -- 172.21.15.29:0/2451749400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 msgr2=0x55c5180d7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 0x55c5180d7960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fcccc00f5c0 tx=0x7fcccc026000).stop 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccf2415b80 1 -- 172.21.15.29:0/2451749400 shutdown_connections 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.29:6833/1328130656,v1:172.21.15.29:6835/1328130656] conn(0x55c5180a8c80 0x55c518083ef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5180cc2c0 0x55c518083a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccb8ff9700 0 client.4782 ms_handle_reset on v2:172.21.15.29:6833/1328130656 2024-01-24T13:58:46.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c51807edf0 0x55c5180d7960 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccf2415b80 1 --2- 172.21.15.29:0/2451749400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c518083620 0x55c51812da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccf2415b80 1 -- 172.21.15.29:0/2451749400 >> 172.21.15.29:0/2451749400 conn(0x55c517eec2e0 msgr2=0x55c518080650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:46.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccf2415b80 1 -- 172.21.15.29:0/2451749400 shutdown_connections 2024-01-24T13:58:46.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccf2415b80 1 -- 172.21.15.29:0/2451749400 wait complete. 2024-01-24T13:58:46.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.664+0000 7fccf2415b80 20 client.4782 trim_cache size 0 max 16384 2024-01-24T13:58:46.670 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T13:58:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T13:58:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T13:58:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 4281, 2024-01-24T13:58:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T13:58:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T13:58:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T13:58:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 27.473151336000001, 2024-01-24T13:58:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 9, 2024-01-24T13:58:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 26, 2024-01-24T13:58:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 26, 2024-01-24T13:58:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 32.790337237000003 2024-01-24T13:58:46.671 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T13:58:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.665+0000 7fcce0a4a700 1 -- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fccbc05f150 msgr2=0x7fccbc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.665+0000 7fcce0a4a700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fccbc05f150 0x7fccbc061610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fcccc005750 tx=0x7fcccc009040).stop 2024-01-24T13:58:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.665+0000 7fcce0a4a700 1 -- 172.21.15.29:0/359005035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccdc13e220 msgr2=0x7fccdc1583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.665+0000 7fcce0a4a700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccdc13e220 0x7fccdc1583f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fccd400c7c0 tx=0x7fccd4007f50).stop 2024-01-24T13:58:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.665+0000 7fcce0a4a700 1 -- 172.21.15.29:0/359005035 shutdown_connections 2024-01-24T13:58:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.665+0000 7fcce0a4a700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fccbc05f150 0x7fccbc061610 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.665+0000 7fcce0a4a700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccdc13e220 0x7fccdc1583f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.665+0000 7fcce0a4a700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccdc063ec0 0x7fccdc0bfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.665+0000 7fcce0a4a700 1 --2- 172.21.15.29:0/359005035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccdc13de40 0x7fccdc0be2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.665+0000 7fcce0a4a700 1 -- 172.21.15.29:0/359005035 >> 172.21.15.29:0/359005035 conn(0x7fccdc0c3cd0 msgr2=0x7fccdc06a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.665+0000 7fcce0a4a700 1 -- 172.21.15.29:0/359005035 shutdown_connections 2024-01-24T13:58:46.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:46.665+0000 7fcce0a4a700 1 -- 172.21.15.29:0/359005035 wait complete. 2024-01-24T13:58:46.679 INFO:teuthology.run_tasks:Running task kclient... 2024-01-24T13:58:46.691 INFO:tasks.kclient:Mounting kernel clients... 2024-01-24T13:58:46.691 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-01-24T13:58:46.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:46.691 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T13:58:46.711 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-01-24T13:58:46.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:46.711 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-24T13:58:46.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:46.809 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link delete ceph-brx 2024-01-24T13:58:46.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:46.892 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T13:58:46.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:46.914 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link delete ceph-brx 2024-01-24T13:58:46.994 INFO:teuthology.orchestra.run.smithi029.stderr:Cannot find device "ceph-brx" 2024-01-24T13:58:46.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T13:58:46.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:46.996 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-01-24T13:58:47.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:47.018 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-01-24T13:58:47.092 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2024-01-24T13:58:47.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T13:58:47.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:47.094 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-01-24T13:58:47.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:47.154 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-01-24T13:58:47.225 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2024-01-24T13:58:47.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T13:58:47.226 INFO:tasks.kclient:client.0 config is {} 2024-01-24T13:58:47.226 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T13:58:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.349+0000 7f15c4e26700 1 Processor -- start 2024-01-24T13:58:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.349+0000 7f15c4e26700 1 -- start start 2024-01-24T13:58:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.350+0000 7f15c4e26700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f15c00be8a0 0x7f15c00c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.350+0000 7f15c4e26700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c00a7490 con 0x7f15c01520e0 2024-01-24T13:58:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.350+0000 7f15c4e26700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c00a8520 con 0x7f15c00be420 2024-01-24T13:58:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.350+0000 7f15c4e26700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c00a6880 con 0x7f15c00be8a0 2024-01-24T13:58:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.351+0000 7f15be59c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f15c01520e0 0x7f15c00a7e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34482/0 (socket says 172.21.15.29:34482) 2024-01-24T13:58:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.351+0000 7f15be59c700 1 -- 172.21.15.29:0/2583464980 learned_addr learned my addr 172.21.15.29:0/2583464980 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.351+0000 7f15bed9d700 1 --2- 172.21.15.29:0/2583464980 >> v2:172.21.15.163:3300/0 conn(0x7f15c00be8a0 0x7f15c00c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.351+0000 7f15bd59a700 1 -- 172.21.15.29:0/2583464980 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911678208 0 0) 0x7f15c00a7490 con 0x7f15c01520e0 2024-01-24T13:58:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.351+0000 7f15bd59a700 1 -- 172.21.15.29:0/2583464980 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15a80033f0 con 0x7f15c01520e0 2024-01-24T13:58:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.351+0000 7f15bed9d700 1 -- 172.21.15.29:0/2583464980 >> v1:172.21.15.163:6789/0 conn(0x7f15c00be420 legacy=0x7f15c00c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.351+0000 7f15bd59a700 1 -- 172.21.15.29:0/2583464980 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197864756 0 0) 0x7f15c00a8520 con 0x7f15c00be420 2024-01-24T13:58:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.351+0000 7f15bed9d700 1 -- 172.21.15.29:0/2583464980 >> v1:172.21.15.29:6789/0 conn(0x7f15c01520e0 legacy=0x7f15c00a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.351+0000 7f15bed9d700 1 -- 172.21.15.29:0/2583464980 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c00a6b60 con 0x7f15c00be8a0 2024-01-24T13:58:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.351+0000 7f15bed9d700 1 --2- 172.21.15.29:0/2583464980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15c00be8a0 0x7f15c00c7260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f15b8009e60 tx=0x7f15b8009780).ready entity=mon.1 client_cookie=5bf9db4771705e1c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.352+0000 7f15bd59a700 1 -- 172.21.15.29:0/2583464980 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b800e070 con 0x7f15c00be8a0 2024-01-24T13:58:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.352+0000 7f15bd59a700 1 -- 172.21.15.29:0/2583464980 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b800bb60 con 0x7f15c00be8a0 2024-01-24T13:58:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.352+0000 7f15bd59a700 1 -- 172.21.15.29:0/2583464980 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b800bee0 con 0x7f15c00be8a0 2024-01-24T13:58:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.352+0000 7f15c4e26700 1 -- 172.21.15.29:0/2583464980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15c00be8a0 msgr2=0x7f15c00c7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.352+0000 7f15c4e26700 1 --2- 172.21.15.29:0/2583464980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15c00be8a0 0x7f15c00c7260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f15b8009e60 tx=0x7f15b8009780).stop 2024-01-24T13:58:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.352+0000 7f15c4e26700 1 -- 172.21.15.29:0/2583464980 shutdown_connections 2024-01-24T13:58:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.352+0000 7f15c4e26700 1 --2- 172.21.15.29:0/2583464980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15c00be8a0 0x7f15c00c7260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.352+0000 7f15c4e26700 1 -- 172.21.15.29:0/2583464980 >> 172.21.15.29:0/2583464980 conn(0x7f15c00922f0 msgr2=0x7f15c0062f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.352+0000 7f15c4e26700 1 -- 172.21.15.29:0/2583464980 shutdown_connections 2024-01-24T13:58:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.352+0000 7f15c4e26700 1 -- 172.21.15.29:0/2583464980 wait complete. 2024-01-24T13:58:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.353+0000 7f15c4e26700 1 Processor -- start 2024-01-24T13:58:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.354+0000 7f15c4e26700 1 -- start start 2024-01-24T13:58:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.354+0000 7f15c4e26700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15c00be420 0x7f15c0150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.354+0000 7f15c4e26700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15c00be8a0 0x7f15c00c2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.354+0000 7f15c4e26700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15c01520e0 0x7f15c00adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.354+0000 7f15c4e26700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00ad5a0 con 0x7f15c00be420 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.354+0000 7f15c4e26700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c01509c0 con 0x7f15c01520e0 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.354+0000 7f15c4e26700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00adaf0 con 0x7f15c00be8a0 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.354+0000 7f15be59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15c00be420 0x7f15c0150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.354+0000 7f15be59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15c00be420 0x7f15c0150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39678/0 (socket says 172.21.15.29:39678) 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.354+0000 7f15be59c700 1 -- 172.21.15.29:0/38681849 learned_addr learned my addr 172.21.15.29:0/38681849 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15bed9d700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15c01520e0 0x7f15c00adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15bdd9b700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15c00be8a0 0x7f15c00c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15be59c700 1 -- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15c00be8a0 msgr2=0x7f15c00c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15be59c700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15c00be8a0 0x7f15c00c2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15be59c700 1 -- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15c01520e0 msgr2=0x7f15c00adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15be59c700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15c01520e0 0x7f15c00adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15be59c700 1 -- 172.21.15.29:0/38681849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b8009a20 con 0x7f15c00be420 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15bed9d700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15c01520e0 0x7f15c00adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:47.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15be59c700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15c00be420 0x7f15c0150240 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f15b000c440 tx=0x7f15b0007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:47.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15ae7fc700 1 -- 172.21.15.29:0/38681849 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b00072e0 con 0x7f15c00be420 2024-01-24T13:58:47.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15ae7fc700 1 -- 172.21.15.29:0/38681849 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b0007440 con 0x7f15c00be420 2024-01-24T13:58:47.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15c4e26700 1 -- 172.21.15.29:0/38681849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c0150c10 con 0x7f15c00be420 2024-01-24T13:58:47.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15ae7fc700 1 -- 172.21.15.29:0/38681849 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0005880 con 0x7f15c00be420 2024-01-24T13:58:47.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.355+0000 7f15c4e26700 1 -- 172.21.15.29:0/38681849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c00ad780 con 0x7f15c00be420 2024-01-24T13:58:47.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.356+0000 7f15c4e26700 1 -- 172.21.15.29:0/38681849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c005f400 con 0x7f15c00be420 2024-01-24T13:58:47.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.357+0000 7f15ae7fc700 1 -- 172.21.15.29:0/38681849 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f15b0005c70 con 0x7f15c00be420 2024-01-24T13:58:47.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.357+0000 7f15ae7fc700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f15a005f150 0x7f15a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:47.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.357+0000 7f15ae7fc700 1 -- 172.21.15.29:0/38681849 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f15b0079cf0 con 0x7f15c00be420 2024-01-24T13:58:47.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.357+0000 7f15bdd9b700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f15a005f150 0x7f15a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:47.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.358+0000 7f15bdd9b700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f15a005f150 0x7f15a0061610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f15c00c6520 tx=0x7f15b4009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:47.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.361+0000 7f15ae7fc700 1 -- 172.21.15.29:0/38681849 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f15b004ad50 con 0x7f15c00be420 2024-01-24T13:58:47.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.588+0000 7f15c4e26700 1 -- 172.21.15.29:0/38681849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f15c00ad940 con 0x7f15c00be420 2024-01-24T13:58:47.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.589+0000 7f15ae7fc700 1 -- 172.21.15.29:0/38681849 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f15b004e370 con 0x7f15c00be420 2024-01-24T13:58:47.589 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-24T13:58:47.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.591+0000 7f15c4e26700 1 -- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f15a005f150 msgr2=0x7f15a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:47.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.591+0000 7f15c4e26700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f15a005f150 0x7f15a0061610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f15c00c6520 tx=0x7f15b4009040).stop 2024-01-24T13:58:47.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.591+0000 7f15c4e26700 1 -- 172.21.15.29:0/38681849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15c00be420 msgr2=0x7f15c0150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:47.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.591+0000 7f15c4e26700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15c00be420 0x7f15c0150240 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f15b000c440 tx=0x7f15b0007ae0).stop 2024-01-24T13:58:47.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.591+0000 7f15c4e26700 1 -- 172.21.15.29:0/38681849 shutdown_connections 2024-01-24T13:58:47.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.591+0000 7f15c4e26700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15c01520e0 0x7f15c00adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:47.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.591+0000 7f15c4e26700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f15a005f150 0x7f15a0061610 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:47.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.591+0000 7f15c4e26700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15c00be420 0x7f15c0150240 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:47.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.591+0000 7f15c4e26700 1 --2- 172.21.15.29:0/38681849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15c00be8a0 0x7f15c00c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:47.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.591+0000 7f15c4e26700 1 -- 172.21.15.29:0/38681849 >> 172.21.15.29:0/38681849 conn(0x7f15c00922f0 msgr2=0x7f15c00c5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:47.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.591+0000 7f15c4e26700 1 -- 172.21.15.29:0/38681849 shutdown_connections 2024-01-24T13:58:47.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:47.591+0000 7f15c4e26700 1 -- 172.21.15.29:0/38681849 wait complete. 2024-01-24T13:58:47.603 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-24T13:58:47.603 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T13:58:47.603 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T13:58:47.603 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-24T13:58:47.603 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-24T13:58:47.603 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-24T13:58:47.603 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-24T13:58:47.603 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T13:58:47.604 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T13:58:47.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:47.604 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T13:58:47.623 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T13:58:47.623 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T13:58:47.623 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T13:58:47.623 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:47.623 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T13:58:47.623 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:47.623 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:47.623 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:47.623 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T13:58:47.624 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T13:58:47.624 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:47.624 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T13:58:47.624 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T13:58:47.624 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42323sec preferred_lft 42323sec 2024-01-24T13:58:47.624 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T13:58:47.624 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:47.624 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:47.624 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:47.624 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T13:58:47.624 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:47.624 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:47.625 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T13:58:47.625 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-24T13:58:47.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:47.625 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:47.625 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:47.625 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add name ceph-brx type bridge 2024-01-24T13:58:47.625 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip addr flush dev ceph-brx 2024-01-24T13:58:47.625 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set ceph-brx up 2024-01-24T13:58:47.625 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-24T13:58:47.625 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:47.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:47.757 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T13:58:47.789 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T13:58:47.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:47.791 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T13:58:47.854 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T13:58:47.854 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T13:58:47.854 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T13:58:47.854 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T13:58:47.855 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T13:58:47.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:47.855 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:47.855 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:47.855 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T13:58:47.855 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T13:58:47.855 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T13:58:47.855 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:47.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:47.982 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T13:58:48.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:48.002 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list-id 2024-01-24T13:58:48.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:48.064 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:48.064 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:48.064 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T13:58:48.064 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-24T13:58:48.064 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:48.175 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-24T13:58:48.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:48.176 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:48.176 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:48.176 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-24T13:58:48.176 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-24T13:58:48.176 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-24T13:58:48.176 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-24T13:58:48.176 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-24T13:58:48.176 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:48.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:48.385 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:48.385 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:48.385 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T13:58:48.385 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set dev brx.0 master ceph-brx 2024-01-24T13:58:48.386 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:48.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:48.498 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-01-24T13:58:48.517 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T13:58:48.517 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:58:48.608 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:58:48.609 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T13:58:48.711 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:58:48.711 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T13:58:48.714 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T13:58:48.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:48.714 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-24T13:58:48.744 INFO:tasks.kclient:client.2 config is {} 2024-01-24T13:58:48.744 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T13:58:48.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.904+0000 7f876ae6f700 1 Processor -- start 2024-01-24T13:58:48.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.905+0000 7f876ae6f700 1 -- start start 2024-01-24T13:58:48.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.905+0000 7f876ae6f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f876413a4b0 0x7f8764066740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:48.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.905+0000 7f876ae6f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8764069bb0 0x7f8764066c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:48.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.905+0000 7f876ae6f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8764067ea0 con 0x7f876413a890 2024-01-24T13:58:48.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.905+0000 7f876ae6f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8764067290 con 0x7f876413a4b0 2024-01-24T13:58:48.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.905+0000 7f876ae6f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8764067570 con 0x7f8764069bb0 2024-01-24T13:58:48.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.906+0000 7f8768c0b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f876413a4b0 0x7f8764066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:48.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.906+0000 7f8768c0b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f876413a4b0 0x7f8764066740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39692/0 (socket says 172.21.15.29:39692) 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.906+0000 7f8768c0b700 1 -- 172.21.15.29:0/507850234 learned_addr learned my addr 172.21.15.29:0/507850234 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.906+0000 7f876940c700 1 --2- 172.21.15.29:0/507850234 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8764069bb0 0x7f8764066c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.906+0000 7f8768c0b700 1 -- 172.21.15.29:0/507850234 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8764069bb0 msgr2=0x7f8764066c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.906+0000 7f8768c0b700 1 --2- 172.21.15.29:0/507850234 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8764069bb0 0x7f8764066c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.906+0000 7f8768c0b700 1 -- 172.21.15.29:0/507850234 >> v1:172.21.15.163:6789/0 conn(0x7f876413a890 legacy=0x7f87640694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.906+0000 7f8768c0b700 1 -- 172.21.15.29:0/507850234 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764067850 con 0x7f876413a4b0 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.906+0000 7f8768c0b700 1 --2- 172.21.15.29:0/507850234 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f876413a4b0 0x7f8764066740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f8758006fd0 tx=0x7f875800b3b0).ready entity=mon.0 client_cookie=901293fea71c6264 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.906+0000 7f87637fe700 1 -- 172.21.15.29:0/507850234 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875800da70 con 0x7f876413a4b0 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.906+0000 7f87637fe700 1 -- 172.21.15.29:0/507850234 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875800dbd0 con 0x7f876413a4b0 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.906+0000 7f87637fe700 1 -- 172.21.15.29:0/507850234 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758010470 con 0x7f876413a4b0 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.907+0000 7f876ae6f700 1 -- 172.21.15.29:0/507850234 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f876413a4b0 msgr2=0x7f8764066740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.907+0000 7f876ae6f700 1 --2- 172.21.15.29:0/507850234 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f876413a4b0 0x7f8764066740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f8758006fd0 tx=0x7f875800b3b0).stop 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.907+0000 7f876ae6f700 1 -- 172.21.15.29:0/507850234 shutdown_connections 2024-01-24T13:58:48.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.907+0000 7f876ae6f700 1 --2- 172.21.15.29:0/507850234 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8764069bb0 0x7f8764066c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.907+0000 7f876ae6f700 1 --2- 172.21.15.29:0/507850234 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f876413a4b0 0x7f8764066740 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.907+0000 7f876ae6f700 1 -- 172.21.15.29:0/507850234 >> 172.21.15.29:0/507850234 conn(0x7f87640c7b10 msgr2=0x7f87640c7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.907+0000 7f876ae6f700 1 -- 172.21.15.29:0/507850234 shutdown_connections 2024-01-24T13:58:48.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.907+0000 7f876ae6f700 1 -- 172.21.15.29:0/507850234 wait complete. 2024-01-24T13:58:48.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.908+0000 7f876ae6f700 1 Processor -- start 2024-01-24T13:58:48.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.909+0000 7f876ae6f700 1 -- start start 2024-01-24T13:58:48.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.909+0000 7f876ae6f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8764069bb0 0x7f87640ac450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:48.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.909+0000 7f876ae6f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f876413a4b0 0x7f87641687c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:48.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.909+0000 7f876ae6f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f876413a890 0x7f8764153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:48.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.909+0000 7f876ae6f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640adae0 con 0x7f876413a890 2024-01-24T13:58:48.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.909+0000 7f876ae6f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640bf680 con 0x7f8764069bb0 2024-01-24T13:58:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.909+0000 7f876ae6f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8764168cb0 con 0x7f876413a4b0 2024-01-24T13:58:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f876940c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f876413a890 0x7f8764153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f876940c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f876413a890 0x7f8764153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39696/0 (socket says 172.21.15.29:39696) 2024-01-24T13:58:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f876940c700 1 -- 172.21.15.29:0/4054331272 learned_addr learned my addr 172.21.15.29:0/4054331272 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f8768c0b700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8764069bb0 0x7f87640ac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f8763fff700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f876413a4b0 0x7f87641687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f876940c700 1 -- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f876413a4b0 msgr2=0x7f87641687c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f876940c700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f876413a4b0 0x7f87641687c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:48.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f876940c700 1 -- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8764069bb0 msgr2=0x7f87640ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:48.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f876940c700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8764069bb0 0x7f87640ac450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:48.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f876940c700 1 -- 172.21.15.29:0/4054331272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875800b610 con 0x7f876413a890 2024-01-24T13:58:48.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f876940c700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f876413a890 0x7f8764153790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f875c009ef0 tx=0x7f875c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:48.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f8760ff9700 1 -- 172.21.15.29:0/4054331272 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00cb10 con 0x7f876413a890 2024-01-24T13:58:48.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f8760ff9700 1 -- 172.21.15.29:0/4054331272 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00f040 con 0x7f876413a890 2024-01-24T13:58:48.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f876ae6f700 1 -- 172.21.15.29:0/4054331272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87640bea70 con 0x7f876413a890 2024-01-24T13:58:48.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f8760ff9700 1 -- 172.21.15.29:0/4054331272 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00ce80 con 0x7f876413a890 2024-01-24T13:58:48.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.910+0000 7f876ae6f700 1 -- 172.21.15.29:0/4054331272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87640bf030 con 0x7f876413a890 2024-01-24T13:58:48.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.911+0000 7f876ae6f700 1 -- 172.21.15.29:0/4054331272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87640bf830 con 0x7f876413a890 2024-01-24T13:58:48.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.912+0000 7f8760ff9700 1 -- 172.21.15.29:0/4054331272 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f875c013610 con 0x7f876413a890 2024-01-24T13:58:48.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.912+0000 7f8760ff9700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f874805f150 0x7f8748061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:48.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.912+0000 7f8760ff9700 1 -- 172.21.15.29:0/4054331272 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f875c07ac90 con 0x7f876413a890 2024-01-24T13:58:48.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.913+0000 7f8768c0b700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f874805f150 0x7f8748061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:48.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.913+0000 7f8768c0b700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f874805f150 0x7f8748061610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f875800bfd0 tx=0x7f875800b3b0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:48.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:48.917+0000 7f8760ff9700 1 -- 172.21.15.29:0/4054331272 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f87640bf830 con 0x7f876413a890 2024-01-24T13:58:49.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.148+0000 7f876ae6f700 1 -- 172.21.15.29:0/4054331272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f87640689c0 con 0x7f876413a890 2024-01-24T13:58:49.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.148+0000 7f8760ff9700 1 -- 172.21.15.29:0/4054331272 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f875c04bc50 con 0x7f876413a890 2024-01-24T13:58:49.149 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-24T13:58:49.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.150+0000 7f876ae6f700 1 -- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f874805f150 msgr2=0x7f8748061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:49.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.150+0000 7f876ae6f700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f874805f150 0x7f8748061610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f875800bfd0 tx=0x7f875800b3b0).stop 2024-01-24T13:58:49.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.150+0000 7f876ae6f700 1 -- 172.21.15.29:0/4054331272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f876413a890 msgr2=0x7f8764153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:49.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.150+0000 7f876ae6f700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f876413a890 0x7f8764153790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f875c009ef0 tx=0x7f875c00b040).stop 2024-01-24T13:58:49.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.150+0000 7f876ae6f700 1 -- 172.21.15.29:0/4054331272 shutdown_connections 2024-01-24T13:58:49.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.150+0000 7f876ae6f700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f874805f150 0x7f8748061610 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:49.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.150+0000 7f876ae6f700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f876413a890 0x7f8764153790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:49.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.150+0000 7f876ae6f700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8764069bb0 0x7f87640ac450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:49.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.150+0000 7f876ae6f700 1 --2- 172.21.15.29:0/4054331272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f876413a4b0 0x7f87641687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:49.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.150+0000 7f876ae6f700 1 -- 172.21.15.29:0/4054331272 >> 172.21.15.29:0/4054331272 conn(0x7f87640c7b10 msgr2=0x7f87640c3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:49.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.151+0000 7f876ae6f700 1 -- 172.21.15.29:0/4054331272 shutdown_connections 2024-01-24T13:58:49.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:49.151+0000 7f876ae6f700 1 -- 172.21.15.29:0/4054331272 wait complete. 2024-01-24T13:58:49.163 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-24T13:58:49.163 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T13:58:49.163 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T13:58:49.163 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-01-24T13:58:49.163 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-24T13:58:49.163 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-24T13:58:49.163 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-24T13:58:49.163 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T13:58:49.163 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-01-24T13:58:49.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:49.164 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T13:58:49.183 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T13:58:49.183 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T13:58:49.183 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T13:58:49.183 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.183 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T13:58:49.183 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.183 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:49.183 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42321sec preferred_lft 42321sec 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:49.184 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T13:58:49.185 INFO:teuthology.orchestra.run.smithi029.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T13:58:49.185 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:49.185 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T13:58:49.185 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.185 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::24bc:1cff:fe57:e704/64 scope link tentative 2024-01-24T13:58:49.185 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.185 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T13:58:49.185 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T13:58:49.185 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::2034:aaff:fef3:f12b/64 scope link tentative 2024-01-24T13:58:49.185 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:49.186 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T13:58:49.254 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T13:58:49.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:49.256 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T13:58:49.320 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T13:58:49.321 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T13:58:49.321 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T13:58:49.321 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T13:58:49.321 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T13:58:49.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:49.321 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:49.321 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:49.321 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T13:58:49.321 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T13:58:49.321 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T13:58:49.321 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:49.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:49.444 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T13:58:49.464 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T13:58:49.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:49.465 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list-id 2024-01-24T13:58:49.526 INFO:teuthology.orchestra.run.smithi029.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-01-24T13:58:49.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:49.527 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:49.527 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:49.527 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T13:58:49.527 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-01-24T13:58:49.527 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:49.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:49.638 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-24T13:58:49.678 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T13:58:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T13:58:49.678 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T13:58:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T13:58:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.679 INFO:teuthology.orchestra.run.smithi029.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T13:58:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 72:ff:db:fb:63:31 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-24T13:58:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-24T13:58:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::70ff:dbff:fefb:6331/64 scope link tentative 2024-01-24T13:58:49.679 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:49.692 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-24T13:58:49.780 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T13:58:49.781 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T13:58:49.781 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T13:58:49.781 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.781 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T13:58:49.781 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.781 INFO:teuthology.orchestra.run.smithi029.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T13:58:49.781 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 72:ff:db:fb:63:31 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-24T13:58:49.781 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-24T13:58:49.781 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.781 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::70ff:dbff:fefb:6331/64 scope link tentative 2024-01-24T13:58:49.781 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:49.796 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-01-24T13:58:49.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:49.796 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:49.796 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:49.796 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-01-24T13:58:49.796 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-24T13:58:49.796 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-01-24T13:58:49.796 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-01-24T13:58:49.796 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-01-24T13:58:49.796 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:50.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:50.033 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:58:50.034 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:58:50.034 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.1 up 2024-01-24T13:58:50.034 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set dev brx.1 master ceph-brx 2024-01-24T13:58:50.034 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:58:50.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:50.134 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.2 2024-01-24T13:58:50.154 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T13:58:50.154 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:58:50.246 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:58:50.246 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=2,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T13:58:50.268 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:58:50.268 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=2,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T13:58:50.271 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T13:58:50.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:50.271 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-01-24T13:58:50.302 INFO:tasks.kclient:client.1 config is {} 2024-01-24T13:58:50.303 DEBUG:teuthology.orchestra.run.smithi163:> ceph fs ls 2024-01-24T13:58:50.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.418+0000 7fd0789de700 1 Processor -- start 2024-01-24T13:58:50.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.418+0000 7fd0789de700 1 -- start start 2024-01-24T13:58:50.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd0789de700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd0740be8a0 0x7fd0740c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:50.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd0789de700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0740a7490 con 0x7fd0741520e0 2024-01-24T13:58:50.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd0789de700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0740a8520 con 0x7fd0740be420 2024-01-24T13:58:50.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd0789de700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0740a6880 con 0x7fd0740be8a0 2024-01-24T13:58:50.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd071d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd0740be420 0x7fd0740c6b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:47312/0 (socket says 172.21.15.163:47312) 2024-01-24T13:58:50.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd071d9b700 1 -- 172.21.15.163:0/3131704022 learned_addr learned my addr 172.21.15.163:0/3131704022 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-01-24T13:58:50.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd072d9d700 1 --2- 172.21.15.163:0/3131704022 >> v2:172.21.15.29:3300/0 conn(0x7fd0740be8a0 0x7fd0740c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:50.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618873726 0 0) 0x7fd0740a8520 con 0x7fd0740be420 2024-01-24T13:58:50.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd05c0033f0 con 0x7fd0740be420 2024-01-24T13:58:50.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 <== mon.1 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2477977326 0 0) 0x7fd05c0033f0 con 0x7fd0740be420 2024-01-24T13:58:50.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 >> v2:172.21.15.29:3300/0 conn(0x7fd0740be8a0 msgr2=0x7fd0740c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:50.432 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd07159a700 1 --2- 172.21.15.163:0/3131704022 >> v2:172.21.15.29:3300/0 conn(0x7fd0740be8a0 0x7fd0740c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:50.432 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.419+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 >> v1:172.21.15.29:6789/0 conn(0x7fd0741520e0 legacy=0x7fd0740a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:50.432 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0740a6b60 con 0x7fd0740be420 2024-01-24T13:58:50.432 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 <== mon.1 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fd068001470 con 0x7fd0740be420 2024-01-24T13:58:50.432 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 >> v1:172.21.15.163:6789/0 conn(0x7fd0740be420 legacy=0x7fd0740c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:50.432 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07159a700 1 --2- 172.21.15.163:0/3131704022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd05c0038e0 0x7fd05c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:50.433 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07159a700 1 --2- 172.21.15.163:0/3131704022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd05c004690 0x7fd05c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:50.433 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07159a700 1 --2- 172.21.15.163:0/3131704022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd05c005000 0x7fd05c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:50.433 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740a6880 con 0x7fd05c005000 2024-01-24T13:58:50.433 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c00beb0 con 0x7fd05c004690 2024-01-24T13:58:50.433 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c00c070 con 0x7fd05c0038e0 2024-01-24T13:58:50.434 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd071d9b700 1 --2- 172.21.15.163:0/3131704022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd05c004690 0x7fd05c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:50.434 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07259c700 1 --2- 172.21.15.163:0/3131704022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd05c0038e0 0x7fd05c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:50.434 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd072d9d700 1 --2- 172.21.15.163:0/3131704022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd05c005000 0x7fd05c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:50.434 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd071d9b700 1 -- 172.21.15.163:0/3131704022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd05c0038e0 msgr2=0x7fd05c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:50.434 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd071d9b700 1 --2- 172.21.15.163:0/3131704022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd05c0038e0 0x7fd05c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:50.434 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd071d9b700 1 -- 172.21.15.163:0/3131704022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd05c005000 msgr2=0x7fd05c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:50.435 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd071d9b700 1 --2- 172.21.15.163:0/3131704022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd05c005000 0x7fd05c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:50.435 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd071d9b700 1 -- 172.21.15.163:0/3131704022 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c0033f0 con 0x7fd05c004690 2024-01-24T13:58:50.435 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd072d9d700 1 -- 172.21.15.163:0/3131704022 reap_dead start 2024-01-24T13:58:50.435 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07259c700 1 --2- 172.21.15.163:0/3131704022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd05c0038e0 0x7fd05c003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:50.435 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd071d9b700 1 --2- 172.21.15.163:0/3131704022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd05c004690 0x7fd05c004b10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fd06800c190 tx=0x7fd068001e50).ready entity=mon.1 client_cookie=d3418a5d9ac340b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:50.435 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068007a20 con 0x7fd05c004690 2024-01-24T13:58:50.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06800ca40 con 0x7fd05c004690 2024-01-24T13:58:50.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.420+0000 7fd07159a700 1 -- 172.21.15.163:0/3131704022 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06800cdc0 con 0x7fd05c004690 2024-01-24T13:58:50.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.421+0000 7fd0789de700 1 -- 172.21.15.163:0/3131704022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd05c004690 msgr2=0x7fd05c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:50.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.421+0000 7fd0789de700 1 --2- 172.21.15.163:0/3131704022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd05c004690 0x7fd05c004b10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fd06800c190 tx=0x7fd068001e50).stop 2024-01-24T13:58:50.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.421+0000 7fd0789de700 1 -- 172.21.15.163:0/3131704022 shutdown_connections 2024-01-24T13:58:50.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.421+0000 7fd0789de700 1 --2- 172.21.15.163:0/3131704022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd05c004690 0x7fd05c004b10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:50.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.421+0000 7fd0789de700 1 -- 172.21.15.163:0/3131704022 >> 172.21.15.163:0/3131704022 conn(0x7fd0740922f0 msgr2=0x7fd074062f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:50.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.421+0000 7fd0789de700 1 -- 172.21.15.163:0/3131704022 shutdown_connections 2024-01-24T13:58:50.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.421+0000 7fd0789de700 1 -- 172.21.15.163:0/3131704022 wait complete. 2024-01-24T13:58:50.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.422+0000 7fd0789de700 1 Processor -- start 2024-01-24T13:58:50.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.422+0000 7fd0789de700 1 -- start start 2024-01-24T13:58:50.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd0789de700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd05c004690 0x7fd074149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:50.438 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd0789de700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd074162460 0x7fd0740a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:50.438 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd0789de700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0740a9270 0x7fd0740665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:50.438 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd0789de700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740ad480 con 0x7fd0740a9270 2024-01-24T13:58:50.438 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd0789de700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740ad150 con 0x7fd074162460 2024-01-24T13:58:50.438 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd0789de700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740a9760 con 0x7fd05c004690 2024-01-24T13:58:50.438 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd071d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd074162460 0x7fd0740a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:50.439 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd07259c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd05c004690 0x7fd074149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:50.439 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd071d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd074162460 0x7fd0740a8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53568/0 (socket says 172.21.15.163:53568) 2024-01-24T13:58:50.439 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd071d9b700 1 -- 172.21.15.163:0/110768891 learned_addr learned my addr 172.21.15.163:0/110768891 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:50.439 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd07259c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd05c004690 0x7fd074149730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:41792/0 (socket says 172.21.15.163:41792) 2024-01-24T13:58:50.439 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd072d9d700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0740a9270 0x7fd0740665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:50.439 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd071d9b700 1 -- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd05c004690 msgr2=0x7fd074149730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:50.439 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd071d9b700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd05c004690 0x7fd074149730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:50.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd071d9b700 1 -- 172.21.15.163:0/110768891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0740a9270 msgr2=0x7fd0740665c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:50.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd071d9b700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0740a9270 0x7fd0740665c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:50.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd071d9b700 1 -- 172.21.15.163:0/110768891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0740a9a40 con 0x7fd074162460 2024-01-24T13:58:50.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd072d9d700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0740a9270 0x7fd0740665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:50.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd07259c700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd05c004690 0x7fd074149730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:50.441 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.423+0000 7fd071d9b700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd074162460 0x7fd0740a8d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fd068001ed0 tx=0x7fd068015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:50.441 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.424+0000 7fd0627fc700 1 -- 172.21.15.163:0/110768891 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0680073b0 con 0x7fd074162460 2024-01-24T13:58:50.441 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.424+0000 7fd0627fc700 1 -- 172.21.15.163:0/110768891 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd068007510 con 0x7fd074162460 2024-01-24T13:58:50.441 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.424+0000 7fd0789de700 1 -- 172.21.15.163:0/110768891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0740a9d20 con 0x7fd074162460 2024-01-24T13:58:50.441 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.424+0000 7fd0627fc700 1 -- 172.21.15.163:0/110768891 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0680048e0 con 0x7fd074162460 2024-01-24T13:58:50.441 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.424+0000 7fd0789de700 1 -- 172.21.15.163:0/110768891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd074149c20 con 0x7fd074162460 2024-01-24T13:58:50.441 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.425+0000 7fd0789de700 1 -- 172.21.15.163:0/110768891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd074149e40 con 0x7fd074162460 2024-01-24T13:58:50.442 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.425+0000 7fd0627fc700 1 -- 172.21.15.163:0/110768891 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd06801c070 con 0x7fd074162460 2024-01-24T13:58:50.442 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.425+0000 7fd0627fc700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd05405f150 0x7fd054061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:50.442 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.425+0000 7fd0627fc700 1 -- 172.21.15.163:0/110768891 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fd06807aa60 con 0x7fd074162460 2024-01-24T13:58:50.442 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.425+0000 7fd07259c700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd05405f150 0x7fd054061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:50.442 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.426+0000 7fd07259c700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd05405f150 0x7fd054061610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fd064000c00 tx=0x7fd064009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:50.442 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.430+0000 7fd0627fc700 1 -- 172.21.15.163:0/110768891 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd074149e40 con 0x7fd074162460 2024-01-24T13:58:50.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.637+0000 7fd0789de700 1 -- 172.21.15.163:0/110768891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd074149e40 con 0x7fd074162460 2024-01-24T13:58:50.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.637+0000 7fd0627fc700 1 -- 172.21.15.163:0/110768891 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd074149e40 con 0x7fd074162460 2024-01-24T13:58:50.637 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-24T13:58:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.639+0000 7fd0789de700 1 -- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd05405f150 msgr2=0x7fd054061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.639+0000 7fd0789de700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd05405f150 0x7fd054061610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fd064000c00 tx=0x7fd064009040).stop 2024-01-24T13:58:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.639+0000 7fd0789de700 1 -- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd074162460 msgr2=0x7fd0740a8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.639+0000 7fd0789de700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd074162460 0x7fd0740a8d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fd068001ed0 tx=0x7fd068015000).stop 2024-01-24T13:58:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.639+0000 7fd0789de700 1 -- 172.21.15.163:0/110768891 shutdown_connections 2024-01-24T13:58:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.639+0000 7fd0789de700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd05405f150 0x7fd054061610 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.639+0000 7fd0789de700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0740a9270 0x7fd0740665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.639+0000 7fd0789de700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd05c004690 0x7fd074149730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.639+0000 7fd0789de700 1 --2- 172.21.15.163:0/110768891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd074162460 0x7fd0740a8d80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.639+0000 7fd0789de700 1 -- 172.21.15.163:0/110768891 >> 172.21.15.163:0/110768891 conn(0x7fd0740922f0 msgr2=0x7fd0740c7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:50.640 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.639+0000 7fd0789de700 1 -- 172.21.15.163:0/110768891 shutdown_connections 2024-01-24T13:58:50.640 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:50.639+0000 7fd0789de700 1 -- 172.21.15.163:0/110768891 wait complete. 2024-01-24T13:58:50.650 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-24T13:58:50.650 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T13:58:50.650 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2024-01-24T13:58:50.650 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-24T13:58:50.650 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-24T13:58:50.650 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-24T13:58:50.650 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-24T13:58:50.650 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T13:58:50.650 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-24T13:58:50.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:50.651 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2024-01-24T13:58:50.668 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:50.669 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:50.669 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T13:58:50.669 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:50.669 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-24T13:58:50.669 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42339sec preferred_lft 42339sec 2024-01-24T13:58:50.669 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::98ae:3aa9:42bd:294c/64 scope link noprefixroute 2024-01-24T13:58:50.669 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:50.669 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-24T13:58:50.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:50.669 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-01-24T13:58:50.669 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-01-24T13:58:50.670 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add name ceph-brx type bridge 2024-01-24T13:58:50.670 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr flush dev ceph-brx 2024-01-24T13:58:50.670 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set ceph-brx up 2024-01-24T13:58:50.670 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-24T13:58:50.670 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-01-24T13:58:50.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:50.846 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T13:58:50.874 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-01-24T13:58:50.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:50.876 DEBUG:teuthology.orchestra.run.smithi163:> route 2024-01-24T13:58:50.947 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2024-01-24T13:58:50.947 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T13:58:50.947 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-24T13:58:50.947 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-24T13:58:50.947 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T13:58:50.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:50.948 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-01-24T13:58:50.948 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-01-24T13:58:50.948 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-24T13:58:50.948 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-24T13:58:50.948 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-24T13:58:50.948 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-01-24T13:58:51.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:51.169 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-01-24T13:58:51.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:51.186 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list-id 2024-01-24T13:58:51.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:51.242 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-01-24T13:58:51.243 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-01-24T13:58:51.243 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-24T13:58:51.243 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-01-24T13:58:51.243 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-01-24T13:58:51.338 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-01-24T13:58:51.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:51.338 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-01-24T13:58:51.338 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-01-24T13:58:51.338 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-01-24T13:58:51.339 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-24T13:58:51.339 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-01-24T13:58:51.339 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-01-24T13:58:51.339 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-01-24T13:58:51.339 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-01-24T13:58:51.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:51.553 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-01-24T13:58:51.553 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-01-24T13:58:51.553 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2024-01-24T13:58:51.553 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set dev brx.0 master ceph-brx 2024-01-24T13:58:51.553 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-01-24T13:58:51.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:51.651 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p /home/ubuntu/cephtest/mnt.1 2024-01-24T13:58:51.668 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T13:58:51.668 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:58:51.751 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:58:51.751 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,name=1,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T13:58:51.841 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:58:51.841 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,name=1,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T13:58:51.844 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T13:58:51.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:51.844 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-24T13:58:51.910 INFO:tasks.kclient:client.3 config is {} 2024-01-24T13:58:51.911 DEBUG:teuthology.orchestra.run.smithi163:> ceph fs ls 2024-01-24T13:58:52.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.053+0000 7fa90c13f700 1 Processor -- start 2024-01-24T13:58:52.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.054+0000 7fa90c13f700 1 -- start start 2024-01-24T13:58:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa90c13f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 0x7fa904063d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa90c13f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa904065b00 con 0x7fa904063980 2024-01-24T13:58:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa90c13f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa904155850 con 0x7fa904069a70 2024-01-24T13:58:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa90c13f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa904064ef0 con 0x7fa904150ee0 2024-01-24T13:58:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa9096da700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa904069a70 0x7fa9041507d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:47316/0 (socket says 172.21.15.163:47316) 2024-01-24T13:58:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa9096da700 1 -- 172.21.15.163:0/3264913391 learned_addr learned my addr 172.21.15.163:0/3264913391 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-01-24T13:58:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa90a6dc700 1 --2- 172.21.15.163:0/3264913391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 0x7fa904063d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa908ed9700 1 -- 172.21.15.163:0/3264913391 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530794994 0 0) 0x7fa904155850 con 0x7fa904069a70 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa908ed9700 1 -- 172.21.15.163:0/3264913391 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f00033f0 con 0x7fa904069a70 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa90a6dc700 1 -- 172.21.15.163:0/3264913391 >> v1:172.21.15.163:6789/0 conn(0x7fa904069a70 legacy=0x7fa9041507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa90a6dc700 1 -- 172.21.15.163:0/3264913391 >> v1:172.21.15.29:6789/0 conn(0x7fa904063980 legacy=0x7fa904069360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa90a6dc700 1 -- 172.21.15.163:0/3264913391 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040651d0 con 0x7fa904150ee0 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa90a6dc700 1 --2- 172.21.15.163:0/3264913391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 0x7fa904063d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa8fc009e90 tx=0x7fa8fc009780).ready entity=mon.2 client_cookie=f278c46be36fd612 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.055+0000 7fa908ed9700 1 -- 172.21.15.163:0/3264913391 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00e070 con 0x7fa904150ee0 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.056+0000 7fa908ed9700 1 -- 172.21.15.163:0/3264913391 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc00bb60 con 0x7fa904150ee0 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.056+0000 7fa908ed9700 1 -- 172.21.15.163:0/3264913391 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc012410 con 0x7fa904150ee0 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.056+0000 7fa90c13f700 1 -- 172.21.15.163:0/3264913391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 msgr2=0x7fa904063d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.056+0000 7fa90c13f700 1 --2- 172.21.15.163:0/3264913391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 0x7fa904063d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa8fc009e90 tx=0x7fa8fc009780).stop 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.056+0000 7fa90c13f700 1 -- 172.21.15.163:0/3264913391 shutdown_connections 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.056+0000 7fa90c13f700 1 --2- 172.21.15.163:0/3264913391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 0x7fa904063d60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.056+0000 7fa90c13f700 1 -- 172.21.15.163:0/3264913391 >> 172.21.15.163:0/3264913391 conn(0x7fa9040c4810 msgr2=0x7fa904068c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:52.058 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.056+0000 7fa90c13f700 1 -- 172.21.15.163:0/3264913391 shutdown_connections 2024-01-24T13:58:52.058 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.056+0000 7fa90c13f700 1 -- 172.21.15.163:0/3264913391 wait complete. 2024-01-24T13:58:52.060 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.058+0000 7fa90c13f700 1 Processor -- start 2024-01-24T13:58:52.060 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.058+0000 7fa90c13f700 1 -- start start 2024-01-24T13:58:52.060 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90c13f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa904063980 0x7fa9040c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:52.060 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90c13f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa904069a70 0x7fa904154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:52.060 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90c13f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 0x7fa9040aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90c13f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040ab9a0 con 0x7fa904063980 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90c13f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9041552a0 con 0x7fa904069a70 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90c13f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa904154a00 con 0x7fa904150ee0 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa9096da700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa904069a70 0x7fa904154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90a6dc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 0x7fa9040aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa9096da700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa904069a70 0x7fa904154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53574/0 (socket says 172.21.15.163:53574) 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90a6dc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 0x7fa9040aa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:41810/0 (socket says 172.21.15.163:41810) 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa9096da700 1 -- 172.21.15.163:0/979673970 learned_addr learned my addr 172.21.15.163:0/979673970 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90a6dc700 1 -- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa904069a70 msgr2=0x7fa904154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90a6dc700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa904069a70 0x7fa904154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90a6dc700 1 -- 172.21.15.163:0/979673970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa904063980 msgr2=0x7fa9040c1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa909edb700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa904063980 0x7fa9040c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90a6dc700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa904063980 0x7fa9040c1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90a6dc700 1 -- 172.21.15.163:0/979673970 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc009a20 con 0x7fa904150ee0 2024-01-24T13:58:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa909edb700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa904063980 0x7fa9040c1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa9096da700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa904069a70 0x7fa904154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90a6dc700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 0x7fa9040aa3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa8fc014000 tx=0x7fa8fc009780).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa8f9ffb700 1 -- 172.21.15.163:0/979673970 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc01c660 con 0x7fa904150ee0 2024-01-24T13:58:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa8f9ffb700 1 -- 172.21.15.163:0/979673970 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc01c7c0 con 0x7fa904150ee0 2024-01-24T13:58:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa90c13f700 1 -- 172.21.15.163:0/979673970 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa904154ce0 con 0x7fa904150ee0 2024-01-24T13:58:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.059+0000 7fa8f9ffb700 1 -- 172.21.15.163:0/979673970 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc01cb80 con 0x7fa904150ee0 2024-01-24T13:58:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.060+0000 7fa90c13f700 1 -- 172.21.15.163:0/979673970 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9040c1fb0 con 0x7fa904150ee0 2024-01-24T13:58:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.061+0000 7fa90c13f700 1 -- 172.21.15.163:0/979673970 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9040abb10 con 0x7fa904150ee0 2024-01-24T13:58:52.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.061+0000 7fa8f9ffb700 1 -- 172.21.15.163:0/979673970 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa8fc003040 con 0x7fa904150ee0 2024-01-24T13:58:52.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.061+0000 7fa8f9ffb700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa8e805f150 0x7fa8e8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:52.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.061+0000 7fa8f9ffb700 1 -- 172.21.15.163:0/979673970 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fa8fc07c500 con 0x7fa904150ee0 2024-01-24T13:58:52.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.061+0000 7fa909edb700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa8e805f150 0x7fa8e8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:52.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.062+0000 7fa909edb700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa8e805f150 0x7fa8e8061610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fa8f4001710 tx=0x7fa8f4009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:52.065 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.066+0000 7fa8f9ffb700 1 -- 172.21.15.163:0/979673970 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa8fc04d400 con 0x7fa904150ee0 2024-01-24T13:58:52.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.272+0000 7fa90c13f700 1 -- 172.21.15.163:0/979673970 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa90406b230 con 0x7fa904150ee0 2024-01-24T13:58:52.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.273+0000 7fa8f9ffb700 1 -- 172.21.15.163:0/979673970 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa8fc050a20 con 0x7fa904150ee0 2024-01-24T13:58:52.273 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-24T13:58:52.274 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.275+0000 7fa90c13f700 1 -- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa8e805f150 msgr2=0x7fa8e8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.275+0000 7fa90c13f700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa8e805f150 0x7fa8e8061610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fa8f4001710 tx=0x7fa8f4009040).stop 2024-01-24T13:58:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.275+0000 7fa90c13f700 1 -- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 msgr2=0x7fa9040aa3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.275+0000 7fa90c13f700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 0x7fa9040aa3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa8fc014000 tx=0x7fa8fc009780).stop 2024-01-24T13:58:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.275+0000 7fa90c13f700 1 -- 172.21.15.163:0/979673970 shutdown_connections 2024-01-24T13:58:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.275+0000 7fa90c13f700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa904150ee0 0x7fa9040aa3e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.275+0000 7fa90c13f700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa8e805f150 0x7fa8e8061610 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.275+0000 7fa90c13f700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa904063980 0x7fa9040c1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.275+0000 7fa90c13f700 1 --2- 172.21.15.163:0/979673970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa904069a70 0x7fa904154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.275+0000 7fa90c13f700 1 -- 172.21.15.163:0/979673970 >> 172.21.15.163:0/979673970 conn(0x7fa9040c4810 msgr2=0x7fa9040627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.275+0000 7fa90c13f700 1 -- 172.21.15.163:0/979673970 shutdown_connections 2024-01-24T13:58:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-01-24T13:58:52.275+0000 7fa90c13f700 1 -- 172.21.15.163:0/979673970 wait complete. 2024-01-24T13:58:52.285 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-24T13:58:52.285 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T13:58:52.285 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2024-01-24T13:58:52.285 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-01-24T13:58:52.285 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-24T13:58:52.285 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-24T13:58:52.285 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-24T13:58:52.285 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T13:58:52.285 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-01-24T13:58:52.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:52.286 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2024-01-24T13:58:52.303 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T13:58:52.303 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T13:58:52.303 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:52.304 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42337sec preferred_lft 42337sec 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::98ae:3aa9:42bd:294c/64 scope link noprefixroute 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 02:8c:a2:7b:73:af brd ff:ff:ff:ff:ff:ff 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::e2:f3ff:fe13:3641/64 scope link tentative 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 02:8c:a2:7b:73:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::8c:a2ff:fe7b:73af/64 scope link tentative 2024-01-24T13:58:52.305 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:52.306 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T13:58:52.369 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-01-24T13:58:52.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:52.371 DEBUG:teuthology.orchestra.run.smithi163:> route 2024-01-24T13:58:52.430 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2024-01-24T13:58:52.430 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T13:58:52.430 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-24T13:58:52.430 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-24T13:58:52.430 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T13:58:52.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:52.430 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-01-24T13:58:52.431 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-01-24T13:58:52.431 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-24T13:58:52.431 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-24T13:58:52.431 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-24T13:58:52.431 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-01-24T13:58:52.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:52.536 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-01-24T13:58:52.552 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-24T13:58:52.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:52.553 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list-id 2024-01-24T13:58:52.609 INFO:teuthology.orchestra.run.smithi163.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-01-24T13:58:52.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:52.610 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-01-24T13:58:52.610 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-01-24T13:58:52.610 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-24T13:58:52.610 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-01-24T13:58:52.610 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-01-24T13:58:52.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:52.704 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-24T13:58:52.746 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T13:58:52.746 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T13:58:52.747 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T13:58:52.747 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.747 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2024-01-24T13:58:52.747 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.747 INFO:teuthology.orchestra.run.smithi163.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T13:58:52.747 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether ea:46:a1:2d:62:13 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-24T13:58:52.747 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-24T13:58:52.747 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.747 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::e846:a1ff:fe2d:6213/64 scope link tentative 2024-01-24T13:58:52.747 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:52.761 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-24T13:58:52.841 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T13:58:52.842 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T13:58:52.842 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T13:58:52.842 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.842 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2024-01-24T13:58:52.842 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.842 INFO:teuthology.orchestra.run.smithi163.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T13:58:52.842 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether ea:46:a1:2d:62:13 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-24T13:58:52.842 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-24T13:58:52.842 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.842 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::e846:a1ff:fe2d:6213/64 scope link tentative 2024-01-24T13:58:52.842 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:58:52.854 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-01-24T13:58:52.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:52.855 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-01-24T13:58:52.855 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-01-24T13:58:52.855 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-01-24T13:58:52.855 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-24T13:58:52.855 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-01-24T13:58:52.855 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-01-24T13:58:52.855 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-01-24T13:58:52.855 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-01-24T13:58:53.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:53.079 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-01-24T13:58:53.079 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-01-24T13:58:53.079 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.1 up 2024-01-24T13:58:53.080 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set dev brx.1 master ceph-brx 2024-01-24T13:58:53.080 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-01-24T13:58:53.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:53.166 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p /home/ubuntu/cephtest/mnt.3 2024-01-24T13:58:53.183 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T13:58:53.183 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:58:53.265 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:58:53.276 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,name=3,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T13:58:53.287 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:58:53.287 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,name=3,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T13:58:53.289 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T13:58:53.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:58:53.290 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-01-24T13:58:53.356 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-01-24T13:58:53.391 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:53.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.530+0000 7fbf24824700 1 Processor -- start 2024-01-24T13:58:53.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.530+0000 7fbf24824700 1 -- start start 2024-01-24T13:58:53.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.531+0000 7fbf24824700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf200be520 0x7fbf200664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:53.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.531+0000 7fbf24824700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf20064120 con 0x7fbf2013a5c0 2024-01-24T13:58:53.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.531+0000 7fbf24824700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf200bf6a0 con 0x7fbf200669e0 2024-01-24T13:58:53.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.531+0000 7fbf24824700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf20066e00 con 0x7fbf200be520 2024-01-24T13:58:53.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.531+0000 7fbf1dd9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf200be520 0x7fbf200664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:53.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.531+0000 7fbf1e59c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbf2013a5c0 0x7fbf20063a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46238/0 (socket says 172.21.15.29:46238) 2024-01-24T13:58:53.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.531+0000 7fbf1e59c700 1 -- 172.21.15.29:0/1034104309 learned_addr learned my addr 172.21.15.29:0/1034104309 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:53.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.532+0000 7fbf1dd9b700 1 -- 172.21.15.29:0/1034104309 >> v1:172.21.15.163:6789/0 conn(0x7fbf200669e0 legacy=0x7fbf200c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:53.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.532+0000 7fbf1dd9b700 1 -- 172.21.15.29:0/1034104309 >> v1:172.21.15.29:6789/0 conn(0x7fbf2013a5c0 legacy=0x7fbf20063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:53.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.532+0000 7fbf1dd9b700 1 -- 172.21.15.29:0/1034104309 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf200bea90 con 0x7fbf200be520 2024-01-24T13:58:53.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.532+0000 7fbf1d59a700 1 -- 172.21.15.29:0/1034104309 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805421270 0 0) 0x7fbf20064120 con 0x7fbf2013a5c0 2024-01-24T13:58:53.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.532+0000 7fbf1dd9b700 1 --2- 172.21.15.29:0/1034104309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf200be520 0x7fbf200664f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fbf10009760 tx=0x7fbf10009060).ready entity=mon.0 client_cookie=54d1b11fd24504b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:53.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.532+0000 7fbf1d59a700 1 -- 172.21.15.29:0/1034104309 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1000b580 con 0x7fbf200be520 2024-01-24T13:58:53.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.532+0000 7fbf1d59a700 1 -- 172.21.15.29:0/1034104309 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1000b6e0 con 0x7fbf200be520 2024-01-24T13:58:53.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.532+0000 7fbf1d59a700 1 -- 172.21.15.29:0/1034104309 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1000ba60 con 0x7fbf200be520 2024-01-24T13:58:53.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.532+0000 7fbf24824700 1 -- 172.21.15.29:0/1034104309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf200be520 msgr2=0x7fbf200664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:53.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.532+0000 7fbf24824700 1 --2- 172.21.15.29:0/1034104309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf200be520 0x7fbf200664f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fbf10009760 tx=0x7fbf10009060).stop 2024-01-24T13:58:53.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.533+0000 7fbf24824700 1 -- 172.21.15.29:0/1034104309 shutdown_connections 2024-01-24T13:58:53.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.533+0000 7fbf24824700 1 --2- 172.21.15.29:0/1034104309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf200be520 0x7fbf200664f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:53.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.533+0000 7fbf24824700 1 -- 172.21.15.29:0/1034104309 >> 172.21.15.29:0/1034104309 conn(0x7fbf200c8bb0 msgr2=0x7fbf200c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:53.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.533+0000 7fbf24824700 1 -- 172.21.15.29:0/1034104309 shutdown_connections 2024-01-24T13:58:53.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.533+0000 7fbf24824700 1 -- 172.21.15.29:0/1034104309 wait complete. 2024-01-24T13:58:53.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.534+0000 7fbf24824700 1 Processor -- start 2024-01-24T13:58:53.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.534+0000 7fbf24824700 1 -- start start 2024-01-24T13:58:53.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf24824700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf200669e0 0x7fbf20156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:53.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf24824700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf200be520 0x7fbf200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:53.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf24824700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf2013a5c0 0x7fbf200a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:53.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf24824700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200a55d0 con 0x7fbf2013a5c0 2024-01-24T13:58:53.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf24824700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200a63f0 con 0x7fbf200669e0 2024-01-24T13:58:53.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf24824700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200a9390 con 0x7fbf200be520 2024-01-24T13:58:53.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf1ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf2013a5c0 0x7fbf200a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:53.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf1ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf2013a5c0 0x7fbf200a8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33516/0 (socket says 172.21.15.29:33516) 2024-01-24T13:58:53.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf1ed9d700 1 -- 172.21.15.29:0/2210112817 learned_addr learned my addr 172.21.15.29:0/2210112817 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:53.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf1e59c700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf200669e0 0x7fbf20156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:53.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf1dd9b700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf200be520 0x7fbf200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:53.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf1ed9d700 1 -- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf200be520 msgr2=0x7fbf200a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:53.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf1ed9d700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf200be520 0x7fbf200a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:53.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf1ed9d700 1 -- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf200669e0 msgr2=0x7fbf20156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:53.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf1ed9d700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf200669e0 0x7fbf20156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:53.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf1ed9d700 1 -- 172.21.15.29:0/2210112817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100092a0 con 0x7fbf2013a5c0 2024-01-24T13:58:53.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.535+0000 7fbf1ed9d700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf2013a5c0 0x7fbf200a8ea0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fbf18002230 tx=0x7fbf180015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:53.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.536+0000 7fbf0e7fc700 1 -- 172.21.15.29:0/2210112817 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800e4a0 con 0x7fbf2013a5c0 2024-01-24T13:58:53.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.536+0000 7fbf0e7fc700 1 -- 172.21.15.29:0/2210112817 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1800e600 con 0x7fbf2013a5c0 2024-01-24T13:58:53.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.536+0000 7fbf0e7fc700 1 -- 172.21.15.29:0/2210112817 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800e980 con 0x7fbf2013a5c0 2024-01-24T13:58:53.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.536+0000 7fbf24824700 1 -- 172.21.15.29:0/2210112817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf200a57e0 con 0x7fbf2013a5c0 2024-01-24T13:58:53.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.536+0000 7fbf24824700 1 -- 172.21.15.29:0/2210112817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf200a5da0 con 0x7fbf2013a5c0 2024-01-24T13:58:53.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.537+0000 7fbf24824700 1 -- 172.21.15.29:0/2210112817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf200475e0 con 0x7fbf2013a5c0 2024-01-24T13:58:53.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.537+0000 7fbf0e7fc700 1 -- 172.21.15.29:0/2210112817 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fbf1801b3b0 con 0x7fbf2013a5c0 2024-01-24T13:58:53.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.537+0000 7fbf0e7fc700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0005f150 0x7fbf00061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:53.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.538+0000 7fbf0e7fc700 1 -- 172.21.15.29:0/2210112817 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fbf18079b00 con 0x7fbf2013a5c0 2024-01-24T13:58:53.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.538+0000 7fbf1e59c700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0005f150 0x7fbf00061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:53.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.538+0000 7fbf1e59c700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0005f150 0x7fbf00061610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fbf14007fd0 tx=0x7fbf14007db0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:53.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.542+0000 7fbf0e7fc700 1 -- 172.21.15.29:0/2210112817 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbf18048b00 con 0x7fbf2013a5c0 2024-01-24T13:58:53.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.721+0000 7fbf24824700 1 -- 172.21.15.29:0/2210112817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf200a6980 con 0x7fbf2013a5c0 2024-01-24T13:58:53.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.722+0000 7fbf0e7fc700 1 -- 172.21.15.29:0/2210112817 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7fbf1804ab10 con 0x7fbf2013a5c0 2024-01-24T13:58:53.723 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:53.723 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:53.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.724+0000 7fbf24824700 1 -- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0005f150 msgr2=0x7fbf00061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:53.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.724+0000 7fbf24824700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0005f150 0x7fbf00061610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fbf14007fd0 tx=0x7fbf14007db0).stop 2024-01-24T13:58:53.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.724+0000 7fbf24824700 1 -- 172.21.15.29:0/2210112817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf2013a5c0 msgr2=0x7fbf200a8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:53.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.724+0000 7fbf24824700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf2013a5c0 0x7fbf200a8ea0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fbf18002230 tx=0x7fbf180015e0).stop 2024-01-24T13:58:53.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.724+0000 7fbf24824700 1 -- 172.21.15.29:0/2210112817 shutdown_connections 2024-01-24T13:58:53.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.724+0000 7fbf24824700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0005f150 0x7fbf00061610 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:53.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.724+0000 7fbf24824700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf2013a5c0 0x7fbf200a8ea0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:53.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.725+0000 7fbf24824700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf200669e0 0x7fbf20156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:53.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.725+0000 7fbf24824700 1 --2- 172.21.15.29:0/2210112817 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf200be520 0x7fbf200a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:53.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.725+0000 7fbf24824700 1 -- 172.21.15.29:0/2210112817 >> 172.21.15.29:0/2210112817 conn(0x7fbf200c8bb0 msgr2=0x7fbf20063880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:53.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.725+0000 7fbf24824700 1 -- 172.21.15.29:0/2210112817 shutdown_connections 2024-01-24T13:58:53.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.725+0000 7fbf24824700 1 -- 172.21.15.29:0/2210112817 wait complete. 2024-01-24T13:58:53.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T13:58:16.949817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T13:58:18.174460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-24T13:58:21.232882+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T13:58:53.739 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T13:58:53.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.872+0000 7f36dd8e0700 1 Processor -- start 2024-01-24T13:58:53.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.873+0000 7f36dd8e0700 1 -- start start 2024-01-24T13:58:53.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.873+0000 7f36dd8e0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f36d80c8490 0x7f36d80a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:53.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36dd8e0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d80bfd40 0x7f36d80c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36dd8e0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d8152240 con 0x7f36d8064760 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36dd8e0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d81586c0 con 0x7f36d80c8490 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36dd8e0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d80aa670 con 0x7f36d80bfd40 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36d67fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f36d80c8490 0x7f36d80a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36d67fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f36d80c8490 0x7f36d80a9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33532/0 (socket says 172.21.15.29:33532) 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36d67fc700 1 -- 172.21.15.29:0/2297521253 learned_addr learned my addr 172.21.15.29:0/2297521253 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36d77fe700 1 --2- 172.21.15.29:0/2297521253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d80bfd40 0x7f36d80c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36d67fc700 1 -- 172.21.15.29:0/2297521253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d80bfd40 msgr2=0x7f36d80c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36d67fc700 1 --2- 172.21.15.29:0/2297521253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d80bfd40 0x7f36d80c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36d67fc700 1 -- 172.21.15.29:0/2297521253 >> v1:172.21.15.163:6789/0 conn(0x7f36d8064760 legacy=0x7f36d8064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36d67fc700 1 -- 172.21.15.29:0/2297521253 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d8157d90 con 0x7f36d80c8490 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36d67fc700 1 --2- 172.21.15.29:0/2297521253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36d80c8490 0x7f36d80a9d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f36cc006fd0 tx=0x7f36cc0090f0).ready entity=mon.0 client_cookie=e689680bf3f9f614 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36d5ffb700 1 -- 172.21.15.29:0/2297521253 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc00b690 con 0x7f36d80c8490 2024-01-24T13:58:53.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36d5ffb700 1 -- 172.21.15.29:0/2297521253 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36cc00b7f0 con 0x7f36d80c8490 2024-01-24T13:58:53.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.874+0000 7f36d5ffb700 1 -- 172.21.15.29:0/2297521253 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc00bb70 con 0x7f36d80c8490 2024-01-24T13:58:53.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.875+0000 7f36dd8e0700 1 -- 172.21.15.29:0/2297521253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36d80c8490 msgr2=0x7f36d80a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:53.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.875+0000 7f36dd8e0700 1 --2- 172.21.15.29:0/2297521253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36d80c8490 0x7f36d80a9d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f36cc006fd0 tx=0x7f36cc0090f0).stop 2024-01-24T13:58:53.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.875+0000 7f36dd8e0700 1 -- 172.21.15.29:0/2297521253 shutdown_connections 2024-01-24T13:58:53.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.875+0000 7f36dd8e0700 1 --2- 172.21.15.29:0/2297521253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d80bfd40 0x7f36d80c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:53.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.875+0000 7f36dd8e0700 1 --2- 172.21.15.29:0/2297521253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36d80c8490 0x7f36d80a9d90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:53.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.875+0000 7f36dd8e0700 1 -- 172.21.15.29:0/2297521253 >> 172.21.15.29:0/2297521253 conn(0x7f36d80cb8b0 msgr2=0x7f36d80c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:53.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.875+0000 7f36dd8e0700 1 -- 172.21.15.29:0/2297521253 shutdown_connections 2024-01-24T13:58:53.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.875+0000 7f36dd8e0700 1 -- 172.21.15.29:0/2297521253 wait complete. 2024-01-24T13:58:53.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.876+0000 7f36dd8e0700 1 Processor -- start 2024-01-24T13:58:53.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.877+0000 7f36dd8e0700 1 -- start start 2024-01-24T13:58:53.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.877+0000 7f36dd8e0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36d8064760 0x7f36d80c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:53.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.877+0000 7f36dd8e0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d80bfd40 0x7f36d8154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.877+0000 7f36dd8e0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36d80c8490 0x7f36d814d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.877+0000 7f36dd8e0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d80c4fe0 con 0x7f36d8064760 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.877+0000 7f36dd8e0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d8155140 con 0x7f36d80c8490 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.877+0000 7f36dd8e0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d81531f0 con 0x7f36d80bfd40 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.877+0000 7f36d6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36d8064760 0x7f36d80c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.877+0000 7f36d6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36d8064760 0x7f36d80c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33542/0 (socket says 172.21.15.29:33542) 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.877+0000 7f36d6ffd700 1 -- 172.21.15.29:0/191919590 learned_addr learned my addr 172.21.15.29:0/191919590 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36d67fc700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d80bfd40 0x7f36d8154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36d77fe700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36d80c8490 0x7f36d814d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36d6ffd700 1 -- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d80bfd40 msgr2=0x7f36d8154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36d6ffd700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d80bfd40 0x7f36d8154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36d6ffd700 1 -- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36d80c8490 msgr2=0x7f36d814d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:53.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36d6ffd700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36d80c8490 0x7f36d814d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36d6ffd700 1 -- 172.21.15.29:0/191919590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc009370 con 0x7f36d8064760 2024-01-24T13:58:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36d77fe700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36d80c8490 0x7f36d814d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36d6ffd700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36d8064760 0x7f36d80c4af0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f36c800c4c0 tx=0x7f36c8007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36beffd700 1 -- 172.21.15.29:0/191919590 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8007020 con 0x7f36d8064760 2024-01-24T13:58:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36beffd700 1 -- 172.21.15.29:0/191919590 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c8001e90 con 0x7f36d8064760 2024-01-24T13:58:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36dd8e0700 1 -- 172.21.15.29:0/191919590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d81534d0 con 0x7f36d8064760 2024-01-24T13:58:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36beffd700 1 -- 172.21.15.29:0/191919590 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c80073b0 con 0x7f36d8064760 2024-01-24T13:58:53.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.878+0000 7f36dd8e0700 1 -- 172.21.15.29:0/191919590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d8153a90 con 0x7f36d8064760 2024-01-24T13:58:53.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.879+0000 7f36dd8e0700 1 -- 172.21.15.29:0/191919590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d80475e0 con 0x7f36d8064760 2024-01-24T13:58:53.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.880+0000 7f36beffd700 1 -- 172.21.15.29:0/191919590 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f36c80068b0 con 0x7f36d8064760 2024-01-24T13:58:53.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.880+0000 7f36beffd700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f36c405f150 0x7f36c4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:53.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.880+0000 7f36beffd700 1 -- 172.21.15.29:0/191919590 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f36c8079e90 con 0x7f36d8064760 2024-01-24T13:58:53.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.880+0000 7f36d67fc700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f36c405f150 0x7f36c4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:53.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.881+0000 7f36d67fc700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f36c405f150 0x7f36c4061610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f36cc009130 tx=0x7f36cc017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:53.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:53.884+0000 7f36beffd700 1 -- 172.21.15.29:0/191919590 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f36c804ae50 con 0x7f36d8064760 2024-01-24T13:58:54.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.066+0000 7f36dd8e0700 1 -- 172.21.15.29:0/191919590 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f36d81540a0 con 0x7f36d8064760 2024-01-24T13:58:54.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.066+0000 7f36beffd700 1 -- 172.21.15.29:0/191919590 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f36c804e470 con 0x7f36d8064760 2024-01-24T13:58:54.067 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T13:58:54.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.069+0000 7f36dd8e0700 1 -- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f36c405f150 msgr2=0x7f36c4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.069+0000 7f36dd8e0700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f36c405f150 0x7f36c4061610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f36cc009130 tx=0x7f36cc017000).stop 2024-01-24T13:58:54.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.069+0000 7f36dd8e0700 1 -- 172.21.15.29:0/191919590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36d8064760 msgr2=0x7f36d80c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.069+0000 7f36dd8e0700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36d8064760 0x7f36d80c4af0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f36c800c4c0 tx=0x7f36c8007e00).stop 2024-01-24T13:58:54.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.069+0000 7f36dd8e0700 1 -- 172.21.15.29:0/191919590 shutdown_connections 2024-01-24T13:58:54.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.069+0000 7f36dd8e0700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36d80c8490 0x7f36d814d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.069+0000 7f36dd8e0700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f36c405f150 0x7f36c4061610 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.069+0000 7f36dd8e0700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36d8064760 0x7f36d80c4af0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.069+0000 7f36dd8e0700 1 --2- 172.21.15.29:0/191919590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d80bfd40 0x7f36d8154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.069+0000 7f36dd8e0700 1 -- 172.21.15.29:0/191919590 >> 172.21.15.29:0/191919590 conn(0x7f36d80cb8b0 msgr2=0x7f36d80a9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:54.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.069+0000 7f36dd8e0700 1 -- 172.21.15.29:0/191919590 shutdown_connections 2024-01-24T13:58:54.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.069+0000 7f36dd8e0700 1 -- 172.21.15.29:0/191919590 wait complete. 2024-01-24T13:58:54.082 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T13:58:54.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.215+0000 7fe9959cc700 1 Processor -- start 2024-01-24T13:58:54.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.215+0000 7fe9959cc700 1 -- start start 2024-01-24T13:58:54.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.216+0000 7fe9959cc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe99015d080 0x7fe99015f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.216+0000 7fe9959cc700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe990152240 con 0x7fe990064760 2024-01-24T13:58:54.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.216+0000 7fe9959cc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9901586c0 con 0x7fe9900c8490 2024-01-24T13:58:54.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.216+0000 7fe9959cc700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9900c0d50 con 0x7fe99015d080 2024-01-24T13:58:54.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.216+0000 7fe98f7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe99015d080 0x7fe99015f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.216+0000 7fe98effd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe990064760 0x7fe990064b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46240/0 (socket says 172.21.15.29:46240) 2024-01-24T13:58:54.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.216+0000 7fe98f7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe99015d080 0x7fe99015f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33544/0 (socket says 172.21.15.29:33544) 2024-01-24T13:58:54.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.216+0000 7fe98effd700 1 -- 172.21.15.29:0/1606081146 learned_addr learned my addr 172.21.15.29:0/1606081146 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:54.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe98f7fe700 1 -- 172.21.15.29:0/1606081146 >> v1:172.21.15.163:6789/0 conn(0x7fe9900c8490 legacy=0x7fe9900a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe98f7fe700 1 -- 172.21.15.29:0/1606081146 >> v1:172.21.15.29:6789/0 conn(0x7fe990064760 legacy=0x7fe990064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe98f7fe700 1 -- 172.21.15.29:0/1606081146 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9900aa670 con 0x7fe99015d080 2024-01-24T13:58:54.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe98e7fc700 1 -- 172.21.15.29:0/1606081146 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485244407 0 0) 0x7fe990152240 con 0x7fe990064760 2024-01-24T13:58:54.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe98f7fe700 1 --2- 172.21.15.29:0/1606081146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe99015d080 0x7fe99015f470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fe988009e60 tx=0x7fe988009780).ready entity=mon.0 client_cookie=b2659fa892162463 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:54.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe98e7fc700 1 -- 172.21.15.29:0/1606081146 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98800e070 con 0x7fe99015d080 2024-01-24T13:58:54.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe98e7fc700 1 -- 172.21.15.29:0/1606081146 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98800bcc0 con 0x7fe99015d080 2024-01-24T13:58:54.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe98e7fc700 1 -- 172.21.15.29:0/1606081146 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988012620 con 0x7fe99015d080 2024-01-24T13:58:54.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe9959cc700 1 -- 172.21.15.29:0/1606081146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe99015d080 msgr2=0x7fe99015f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe9959cc700 1 --2- 172.21.15.29:0/1606081146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe99015d080 0x7fe99015f470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fe988009e60 tx=0x7fe988009780).stop 2024-01-24T13:58:54.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe9959cc700 1 -- 172.21.15.29:0/1606081146 shutdown_connections 2024-01-24T13:58:54.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe9959cc700 1 --2- 172.21.15.29:0/1606081146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe99015d080 0x7fe99015f470 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.217+0000 7fe9959cc700 1 -- 172.21.15.29:0/1606081146 >> 172.21.15.29:0/1606081146 conn(0x7fe9900cb8b0 msgr2=0x7fe9900c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:54.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.218+0000 7fe9959cc700 1 -- 172.21.15.29:0/1606081146 shutdown_connections 2024-01-24T13:58:54.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.218+0000 7fe9959cc700 1 -- 172.21.15.29:0/1606081146 wait complete. 2024-01-24T13:58:54.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.219+0000 7fe9959cc700 1 Processor -- start 2024-01-24T13:58:54.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.219+0000 7fe9959cc700 1 -- start start 2024-01-24T13:58:54.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9959cc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe990064760 0x7fe990148830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9959cc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9900c8490 0x7fe9900ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9959cc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe99015d080 0x7fe990149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9959cc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9900aba60 con 0x7fe9900c8490 2024-01-24T13:58:54.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9959cc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9900ac2c0 con 0x7fe99015d080 2024-01-24T13:58:54.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9959cc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe990149a00 con 0x7fe990064760 2024-01-24T13:58:54.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9867fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9900c8490 0x7fe9900ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9867fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9900c8490 0x7fe9900ab260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33546/0 (socket says 172.21.15.29:33546) 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9867fc700 1 -- 172.21.15.29:0/2907158578 learned_addr learned my addr 172.21.15.29:0/2907158578 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe98f7fe700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe99015d080 0x7fe990149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe98effd700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe990064760 0x7fe990148830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9867fc700 1 -- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe990064760 msgr2=0x7fe990148830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9867fc700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe990064760 0x7fe990148830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9867fc700 1 -- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe99015d080 msgr2=0x7fe990149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9867fc700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe99015d080 0x7fe990149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9867fc700 1 -- 172.21.15.29:0/2907158578 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe988009a20 con 0x7fe9900c8490 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.220+0000 7fe9867fc700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9900c8490 0x7fe9900ab260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fe97800c4c0 tx=0x7fe978007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.221+0000 7fe9877fe700 1 -- 172.21.15.29:0/2907158578 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978007020 con 0x7fe9900c8490 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.221+0000 7fe9877fe700 1 -- 172.21.15.29:0/2907158578 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe978001e90 con 0x7fe9900c8490 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.221+0000 7fe9959cc700 1 -- 172.21.15.29:0/2907158578 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe990168840 con 0x7fe9900c8490 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.221+0000 7fe9877fe700 1 -- 172.21.15.29:0/2907158578 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780073d0 con 0x7fe9900c8490 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.221+0000 7fe9959cc700 1 -- 172.21.15.29:0/2907158578 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9900abf50 con 0x7fe9900c8490 2024-01-24T13:58:54.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.222+0000 7fe9959cc700 1 -- 172.21.15.29:0/2907158578 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9900475e0 con 0x7fe9900c8490 2024-01-24T13:58:54.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.222+0000 7fe9877fe700 1 -- 172.21.15.29:0/2907158578 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe9780068b0 con 0x7fe9900c8490 2024-01-24T13:58:54.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.223+0000 7fe9877fe700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe97005f120 0x7fe9700615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.223+0000 7fe9877fe700 1 -- 172.21.15.29:0/2907158578 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe978079e00 con 0x7fe9900c8490 2024-01-24T13:58:54.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.223+0000 7fe98effd700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe97005f120 0x7fe9700615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.223+0000 7fe98effd700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe97005f120 0x7fe9700615e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fe9800011e0 tx=0x7fe980009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:54.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.227+0000 7fe9877fe700 1 -- 172.21.15.29:0/2907158578 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe97804adc0 con 0x7fe9900c8490 2024-01-24T13:58:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.407+0000 7fe9959cc700 1 -- 172.21.15.29:0/2907158578 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe9900ab840 con 0x7fe9900c8490 2024-01-24T13:58:54.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.408+0000 7fe9877fe700 1 -- 172.21.15.29:0/2907158578 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe97804e3e0 con 0x7fe9900c8490 2024-01-24T13:58:54.408 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T13:58:54.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.410+0000 7fe9959cc700 1 -- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe97005f120 msgr2=0x7fe9700615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.410+0000 7fe9959cc700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe97005f120 0x7fe9700615e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fe9800011e0 tx=0x7fe980009040).stop 2024-01-24T13:58:54.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.410+0000 7fe9959cc700 1 -- 172.21.15.29:0/2907158578 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9900c8490 msgr2=0x7fe9900ab260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.410+0000 7fe9959cc700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9900c8490 0x7fe9900ab260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fe97800c4c0 tx=0x7fe978007e00).stop 2024-01-24T13:58:54.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.410+0000 7fe9959cc700 1 -- 172.21.15.29:0/2907158578 shutdown_connections 2024-01-24T13:58:54.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.410+0000 7fe9959cc700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe99015d080 0x7fe990149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.410+0000 7fe9959cc700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe990064760 0x7fe990148830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.410+0000 7fe9959cc700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe97005f120 0x7fe9700615e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.410+0000 7fe9959cc700 1 --2- 172.21.15.29:0/2907158578 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9900c8490 0x7fe9900ab260 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.410+0000 7fe9959cc700 1 -- 172.21.15.29:0/2907158578 >> 172.21.15.29:0/2907158578 conn(0x7fe9900cb8b0 msgr2=0x7fe99014f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:54.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.410+0000 7fe9959cc700 1 -- 172.21.15.29:0/2907158578 shutdown_connections 2024-01-24T13:58:54.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.410+0000 7fe9959cc700 1 -- 172.21.15.29:0/2907158578 wait complete. 2024-01-24T13:58:54.424 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T13:58:54.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.556+0000 7f1972a1a700 1 Processor -- start 2024-01-24T13:58:54.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.556+0000 7f1972a1a700 1 -- start start 2024-01-24T13:58:54.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.557+0000 7f1972a1a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f196c0664c0 0x7f196c066950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.557+0000 7f1972a1a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c066e40 con 0x7f196c0bdb30 2024-01-24T13:58:54.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.557+0000 7f1972a1a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0640d0 con 0x7f196c13a5e0 2024-01-24T13:58:54.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.557+0000 7f1972a1a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c0679f0 con 0x7f196c0664c0 2024-01-24T13:58:54.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.557+0000 7f196b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f196c0664c0 0x7f196c066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.557+0000 7f196b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f196c0664c0 0x7f196c066950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33556/0 (socket says 172.21.15.29:33556) 2024-01-24T13:58:54.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.557+0000 7f1970fb7700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f196c0bdb30 0x7f196c0bdf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46252/0 (socket says 172.21.15.29:46252) 2024-01-24T13:58:54.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.557+0000 7f196b7fe700 1 -- 172.21.15.29:0/813362682 learned_addr learned my addr 172.21.15.29:0/813362682 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:54.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.558+0000 7f196b7fe700 1 -- 172.21.15.29:0/813362682 >> v1:172.21.15.163:6789/0 conn(0x7f196c13a5e0 legacy=0x7f196c0639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.558+0000 7f196b7fe700 1 -- 172.21.15.29:0/813362682 >> v1:172.21.15.29:6789/0 conn(0x7f196c0bdb30 legacy=0x7f196c0bdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.558+0000 7f196b7fe700 1 -- 172.21.15.29:0/813362682 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c067cd0 con 0x7f196c0664c0 2024-01-24T13:58:54.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.558+0000 7f196b7fe700 1 --2- 172.21.15.29:0/813362682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c0664c0 0x7f196c066950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f1960006fd0 tx=0x7f19600090f0).ready entity=mon.0 client_cookie=b542fa22ab5efe3d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:54.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.558+0000 7f196affd700 1 -- 172.21.15.29:0/813362682 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000f510 con 0x7f196c0664c0 2024-01-24T13:58:54.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.558+0000 7f196affd700 1 -- 172.21.15.29:0/813362682 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196000f670 con 0x7f196c0664c0 2024-01-24T13:58:54.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.558+0000 7f196affd700 1 -- 172.21.15.29:0/813362682 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000f9f0 con 0x7f196c0664c0 2024-01-24T13:58:54.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.558+0000 7f1972a1a700 1 -- 172.21.15.29:0/813362682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c0664c0 msgr2=0x7f196c066950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.558+0000 7f1972a1a700 1 --2- 172.21.15.29:0/813362682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c0664c0 0x7f196c066950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f1960006fd0 tx=0x7f19600090f0).stop 2024-01-24T13:58:54.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.558+0000 7f1972a1a700 1 -- 172.21.15.29:0/813362682 shutdown_connections 2024-01-24T13:58:54.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.558+0000 7f1972a1a700 1 --2- 172.21.15.29:0/813362682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c0664c0 0x7f196c066950 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.559+0000 7f1972a1a700 1 -- 172.21.15.29:0/813362682 >> 172.21.15.29:0/813362682 conn(0x7f196c0c8bd0 msgr2=0x7f196c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:54.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.559+0000 7f1972a1a700 1 -- 172.21.15.29:0/813362682 shutdown_connections 2024-01-24T13:58:54.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.559+0000 7f1972a1a700 1 -- 172.21.15.29:0/813362682 wait complete. 2024-01-24T13:58:54.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.560+0000 7f1972a1a700 1 Processor -- start 2024-01-24T13:58:54.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.560+0000 7f1972a1a700 1 -- start start 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f1972a1a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f196c0664c0 0x7f196c0a8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f1972a1a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c0bdb30 0x7f196c1683a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f1972a1a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c13a5e0 0x7f196c0ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f1972a1a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0a9120 con 0x7f196c0bdb30 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f1972a1a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c168890 con 0x7f196c0664c0 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f1972a1a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0a9310 con 0x7f196c13a5e0 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f196b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c0bdb30 0x7f196c1683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f196b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c0bdb30 0x7f196c1683a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33570/0 (socket says 172.21.15.29:33570) 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f196b7fe700 1 -- 172.21.15.29:0/1226550680 learned_addr learned my addr 172.21.15.29:0/1226550680 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f196bfff700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f196c0664c0 0x7f196c0a8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f1970fb7700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c13a5e0 0x7f196c0ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f196b7fe700 1 -- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c13a5e0 msgr2=0x7f196c0ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f196b7fe700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c13a5e0 0x7f196c0ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f196b7fe700 1 -- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f196c0664c0 msgr2=0x7f196c0a8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f196b7fe700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f196c0664c0 0x7f196c0a8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f196b7fe700 1 -- 172.21.15.29:0/1226550680 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1960009370 con 0x7f196c0bdb30 2024-01-24T13:58:54.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f196bfff700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f196c0664c0 0x7f196c0a8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:54.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.561+0000 7f196b7fe700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c0bdb30 0x7f196c1683a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f196c0c66b0 tx=0x7f1960017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:54.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.562+0000 7f1953fff700 1 -- 172.21.15.29:0/1226550680 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000f9f0 con 0x7f196c0bdb30 2024-01-24T13:58:54.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.562+0000 7f1953fff700 1 -- 172.21.15.29:0/1226550680 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1960000c90 con 0x7f196c0bdb30 2024-01-24T13:58:54.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.562+0000 7f1972a1a700 1 -- 172.21.15.29:0/1226550680 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c0ac650 con 0x7f196c0bdb30 2024-01-24T13:58:54.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.562+0000 7f1953fff700 1 -- 172.21.15.29:0/1226550680 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196001c630 con 0x7f196c0bdb30 2024-01-24T13:58:54.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.562+0000 7f1972a1a700 1 -- 172.21.15.29:0/1226550680 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c0ac000 con 0x7f196c0bdb30 2024-01-24T13:58:54.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.563+0000 7f1972a1a700 1 -- 172.21.15.29:0/1226550680 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c0475e0 con 0x7f196c0bdb30 2024-01-24T13:58:54.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.563+0000 7f1953fff700 1 -- 172.21.15.29:0/1226550680 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f196001c9e0 con 0x7f196c0bdb30 2024-01-24T13:58:54.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.564+0000 7f1953fff700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f195805f150 0x7f1958061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.564+0000 7f1953fff700 1 -- 172.21.15.29:0/1226550680 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f196007c0c0 con 0x7f196c0bdb30 2024-01-24T13:58:54.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.564+0000 7f196bfff700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f195805f150 0x7f1958061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.565+0000 7f196bfff700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f195805f150 0x7f1958061610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f195c001770 tx=0x7f195c009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:54.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.568+0000 7f1953fff700 1 -- 172.21.15.29:0/1226550680 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f196004d080 con 0x7f196c0bdb30 2024-01-24T13:58:54.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.747+0000 7f1972a1a700 1 -- 172.21.15.29:0/1226550680 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f196c067fb0 con 0x7f196c0bdb30 2024-01-24T13:58:54.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.747+0000 7f1953fff700 1 -- 172.21.15.29:0/1226550680 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f19600506a0 con 0x7f196c0bdb30 2024-01-24T13:58:54.748 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:58:54.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.749+0000 7f1972a1a700 1 -- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f195805f150 msgr2=0x7f1958061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.749+0000 7f1972a1a700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f195805f150 0x7f1958061610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f195c001770 tx=0x7f195c009040).stop 2024-01-24T13:58:54.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.749+0000 7f1972a1a700 1 -- 172.21.15.29:0/1226550680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c0bdb30 msgr2=0x7f196c1683a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.749+0000 7f1972a1a700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c0bdb30 0x7f196c1683a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f196c0c66b0 tx=0x7f1960017000).stop 2024-01-24T13:58:54.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.749+0000 7f1972a1a700 1 -- 172.21.15.29:0/1226550680 shutdown_connections 2024-01-24T13:58:54.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.749+0000 7f1972a1a700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c13a5e0 0x7f196c0ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.749+0000 7f1972a1a700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f196c0664c0 0x7f196c0a8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.749+0000 7f1972a1a700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f195805f150 0x7f1958061610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.749+0000 7f1972a1a700 1 --2- 172.21.15.29:0/1226550680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c0bdb30 0x7f196c1683a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.749+0000 7f1972a1a700 1 -- 172.21.15.29:0/1226550680 >> 172.21.15.29:0/1226550680 conn(0x7f196c0c8bd0 msgr2=0x7f196c067000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:54.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.750+0000 7f1972a1a700 1 -- 172.21.15.29:0/1226550680 shutdown_connections 2024-01-24T13:58:54.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.750+0000 7f1972a1a700 1 -- 172.21.15.29:0/1226550680 wait complete. 2024-01-24T13:58:54.764 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T13:58:54.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.897+0000 7fc79617f700 1 Processor -- start 2024-01-24T13:58:54.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.897+0000 7fc79617f700 1 -- start start 2024-01-24T13:58:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.898+0000 7fc79617f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc790069440 0x7fc790069820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.898+0000 7fc79617f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79006a110 con 0x7fc790066290 2024-01-24T13:58:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.898+0000 7fc79617f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7900c3d40 con 0x7fc790065e30 2024-01-24T13:58:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.898+0000 7fc79617f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900c3130 con 0x7fc790069440 2024-01-24T13:58:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.898+0000 7fc78ffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc790066290 0x7fc7900bcc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46256/0 (socket says 172.21.15.29:46256) 2024-01-24T13:58:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.898+0000 7fc78ffff700 1 -- 172.21.15.29:0/2163118257 learned_addr learned my addr 172.21.15.29:0/2163118257 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.898+0000 7fc78f7fe700 1 --2- 172.21.15.29:0/2163118257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc790069440 0x7fc790069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.898+0000 7fc78e7fc700 1 -- 172.21.15.29:0/2163118257 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124992100 0 0) 0x7fc79006a110 con 0x7fc790066290 2024-01-24T13:58:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.899+0000 7fc78e7fc700 1 -- 172.21.15.29:0/2163118257 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7780033f0 con 0x7fc790066290 2024-01-24T13:58:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.899+0000 7fc78e7fc700 1 -- 172.21.15.29:0/2163118257 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386663637 0 0) 0x7fc7900c3d40 con 0x7fc790065e30 2024-01-24T13:58:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.899+0000 7fc78e7fc700 1 -- 172.21.15.29:0/2163118257 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc79006a110 con 0x7fc790065e30 2024-01-24T13:58:54.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.899+0000 7fc78f7fe700 1 -- 172.21.15.29:0/2163118257 >> v1:172.21.15.163:6789/0 conn(0x7fc790065e30 legacy=0x7fc790153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.899+0000 7fc78f7fe700 1 -- 172.21.15.29:0/2163118257 >> v1:172.21.15.29:6789/0 conn(0x7fc790066290 legacy=0x7fc7900bcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.899+0000 7fc78f7fe700 1 -- 172.21.15.29:0/2163118257 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7900c3410 con 0x7fc790069440 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.899+0000 7fc78f7fe700 1 --2- 172.21.15.29:0/2163118257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc790069440 0x7fc790069820 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fc780009790 tx=0x7fc780009060).ready entity=mon.2 client_cookie=c3c15bce07e24cd5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.899+0000 7fc78e7fc700 1 -- 172.21.15.29:0/2163118257 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78000f510 con 0x7fc790069440 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.899+0000 7fc78e7fc700 1 -- 172.21.15.29:0/2163118257 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78000f670 con 0x7fc790069440 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.899+0000 7fc78e7fc700 1 -- 172.21.15.29:0/2163118257 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78000f9f0 con 0x7fc790069440 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.900+0000 7fc79617f700 1 -- 172.21.15.29:0/2163118257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc790069440 msgr2=0x7fc790069820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.900+0000 7fc79617f700 1 --2- 172.21.15.29:0/2163118257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc790069440 0x7fc790069820 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fc780009790 tx=0x7fc780009060).stop 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.900+0000 7fc79617f700 1 -- 172.21.15.29:0/2163118257 shutdown_connections 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.900+0000 7fc79617f700 1 --2- 172.21.15.29:0/2163118257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc790069440 0x7fc790069820 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.900+0000 7fc79617f700 1 -- 172.21.15.29:0/2163118257 >> 172.21.15.29:0/2163118257 conn(0x7fc790064390 msgr2=0x7fc7900c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.900+0000 7fc79617f700 1 -- 172.21.15.29:0/2163118257 shutdown_connections 2024-01-24T13:58:54.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.900+0000 7fc79617f700 1 -- 172.21.15.29:0/2163118257 wait complete. 2024-01-24T13:58:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.901+0000 7fc79617f700 1 Processor -- start 2024-01-24T13:58:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.901+0000 7fc79617f700 1 -- start start 2024-01-24T13:58:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.902+0000 7fc79617f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc790065e30 0x7fc79013e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.902+0000 7fc79617f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc790066290 0x7fc790153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.902+0000 7fc79617f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc790069440 0x7fc7900bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.902+0000 7fc79617f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900c1240 con 0x7fc790066290 2024-01-24T13:58:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.902+0000 7fc79617f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79013eb50 con 0x7fc790065e30 2024-01-24T13:58:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.902+0000 7fc79617f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900c0630 con 0x7fc790069440 2024-01-24T13:58:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.902+0000 7fc78effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc790066290 0x7fc790153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.902+0000 7fc78effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc790066290 0x7fc790153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33572/0 (socket says 172.21.15.29:33572) 2024-01-24T13:58:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.902+0000 7fc78effd700 1 -- 172.21.15.29:0/557546289 learned_addr learned my addr 172.21.15.29:0/557546289 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.902+0000 7fc78ffff700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc790069440 0x7fc7900bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.902+0000 7fc78f7fe700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc790065e30 0x7fc79013e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc78effd700 1 -- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc790069440 msgr2=0x7fc7900bd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc78effd700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc790069440 0x7fc7900bd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc78effd700 1 -- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc790065e30 msgr2=0x7fc79013e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:54.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc78effd700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc790065e30 0x7fc79013e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:54.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc78effd700 1 -- 172.21.15.29:0/557546289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7800092a0 con 0x7fc790066290 2024-01-24T13:58:54.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc78ffff700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc790069440 0x7fc7900bd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:54.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc78f7fe700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc790065e30 0x7fc79013e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:54.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc78effd700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc790066290 0x7fc790153cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fc78400c490 tx=0x7fc784007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:54.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc7777fe700 1 -- 172.21.15.29:0/557546289 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78400f470 con 0x7fc790066290 2024-01-24T13:58:54.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc7777fe700 1 -- 172.21.15.29:0/557546289 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78400f5d0 con 0x7fc790066290 2024-01-24T13:58:54.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc79617f700 1 -- 172.21.15.29:0/557546289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7900c0910 con 0x7fc790066290 2024-01-24T13:58:54.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc7777fe700 1 -- 172.21.15.29:0/557546289 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78400f990 con 0x7fc790066290 2024-01-24T13:58:54.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.903+0000 7fc79617f700 1 -- 172.21.15.29:0/557546289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7900c0ed0 con 0x7fc790066290 2024-01-24T13:58:54.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.904+0000 7fc79617f700 1 -- 172.21.15.29:0/557546289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc790156ab0 con 0x7fc790066290 2024-01-24T13:58:54.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.904+0000 7fc7777fe700 1 -- 172.21.15.29:0/557546289 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc78400fd40 con 0x7fc790066290 2024-01-24T13:58:54.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.905+0000 7fc7777fe700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc77c05f150 0x7fc77c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:54.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.905+0000 7fc7777fe700 1 -- 172.21.15.29:0/557546289 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fc784079b50 con 0x7fc790066290 2024-01-24T13:58:54.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.905+0000 7fc78f7fe700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc77c05f150 0x7fc77c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:54.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.906+0000 7fc78f7fe700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc77c05f150 0x7fc77c061610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc780009fd0 tx=0x7fc780017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:54.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:54.910+0000 7fc7777fe700 1 -- 172.21.15.29:0/557546289 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc78404ab10 con 0x7fc790066290 2024-01-24T13:58:55.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.089+0000 7fc79617f700 1 -- 172.21.15.29:0/557546289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc7900475e0 con 0x7fc790066290 2024-01-24T13:58:55.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.090+0000 7fc7777fe700 1 -- 172.21.15.29:0/557546289 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc78404e130 con 0x7fc790066290 2024-01-24T13:58:55.090 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:58:55.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.092+0000 7fc79617f700 1 -- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc77c05f150 msgr2=0x7fc77c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.092+0000 7fc79617f700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc77c05f150 0x7fc77c061610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc780009fd0 tx=0x7fc780017000).stop 2024-01-24T13:58:55.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.092+0000 7fc79617f700 1 -- 172.21.15.29:0/557546289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc790066290 msgr2=0x7fc790153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.092+0000 7fc79617f700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc790066290 0x7fc790153cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fc78400c490 tx=0x7fc784007c60).stop 2024-01-24T13:58:55.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.092+0000 7fc79617f700 1 -- 172.21.15.29:0/557546289 shutdown_connections 2024-01-24T13:58:55.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.092+0000 7fc79617f700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc790069440 0x7fc7900bd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.092+0000 7fc79617f700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc790065e30 0x7fc79013e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.092+0000 7fc79617f700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc77c05f150 0x7fc77c061610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.092+0000 7fc79617f700 1 --2- 172.21.15.29:0/557546289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc790066290 0x7fc790153cc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.092+0000 7fc79617f700 1 -- 172.21.15.29:0/557546289 >> 172.21.15.29:0/557546289 conn(0x7fc790064390 msgr2=0x7fc7900bca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:55.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.092+0000 7fc79617f700 1 -- 172.21.15.29:0/557546289 shutdown_connections 2024-01-24T13:58:55.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.092+0000 7fc79617f700 1 -- 172.21.15.29:0/557546289 wait complete. 2024-01-24T13:58:55.106 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-24T13:58:55.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.240+0000 7f3bec221700 1 Processor -- start 2024-01-24T13:58:55.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.240+0000 7f3bec221700 1 -- start start 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.241+0000 7f3bec221700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3be40664c0 0x7f3be4066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.241+0000 7f3bec221700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be40686a0 con 0x7f3be40bdb30 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.241+0000 7f3bec221700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be4066e30 con 0x7f3be4134ea0 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.241+0000 7f3bec221700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be40c38e0 con 0x7f3be40664c0 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.242+0000 7f3be97bc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3be40664c0 0x7f3be4066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.242+0000 7f3be9fbd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3be4134ea0 0x7f3be40c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:46506/0 (socket says 172.21.15.29:46506) 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.242+0000 7f3be9fbd700 1 -- 172.21.15.29:0/3172159325 learned_addr learned my addr 172.21.15.29:0/3172159325 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.242+0000 7f3be97bc700 1 -- 172.21.15.29:0/3172159325 >> v1:172.21.15.163:6789/0 conn(0x7f3be4134ea0 legacy=0x7f3be40c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.242+0000 7f3be97bc700 1 -- 172.21.15.29:0/3172159325 >> v1:172.21.15.29:6789/0 conn(0x7f3be40bdb30 legacy=0x7f3be40bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.242+0000 7f3be8fbb700 1 -- 172.21.15.29:0/3172159325 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850274521 0 0) 0x7f3be4066e30 con 0x7f3be4134ea0 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.242+0000 7f3be97bc700 1 -- 172.21.15.29:0/3172159325 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be4068b30 con 0x7f3be40664c0 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.242+0000 7f3be97bc700 1 --2- 172.21.15.29:0/3172159325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be40664c0 0x7f3be4066940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f3bd8009760 tx=0x7f3bd8009060).ready entity=mon.0 client_cookie=786f71b5576882bf server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:55.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.242+0000 7f3be8fbb700 1 -- 172.21.15.29:0/3172159325 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800f510 con 0x7f3be40664c0 2024-01-24T13:58:55.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.242+0000 7f3be8fbb700 1 -- 172.21.15.29:0/3172159325 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd800f670 con 0x7f3be40664c0 2024-01-24T13:58:55.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.242+0000 7f3be8fbb700 1 -- 172.21.15.29:0/3172159325 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800f9f0 con 0x7f3be40664c0 2024-01-24T13:58:55.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.243+0000 7f3bec221700 1 -- 172.21.15.29:0/3172159325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be40664c0 msgr2=0x7f3be4066940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.243+0000 7f3bec221700 1 --2- 172.21.15.29:0/3172159325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be40664c0 0x7f3be4066940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f3bd8009760 tx=0x7f3bd8009060).stop 2024-01-24T13:58:55.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.243+0000 7f3bec221700 1 -- 172.21.15.29:0/3172159325 shutdown_connections 2024-01-24T13:58:55.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.243+0000 7f3bec221700 1 --2- 172.21.15.29:0/3172159325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be40664c0 0x7f3be4066940 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.243+0000 7f3bec221700 1 -- 172.21.15.29:0/3172159325 >> 172.21.15.29:0/3172159325 conn(0x7f3be40c8bd0 msgr2=0x7f3be40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:55.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.243+0000 7f3bec221700 1 -- 172.21.15.29:0/3172159325 shutdown_connections 2024-01-24T13:58:55.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.243+0000 7f3bec221700 1 -- 172.21.15.29:0/3172159325 wait complete. 2024-01-24T13:58:55.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.244+0000 7f3bec221700 1 Processor -- start 2024-01-24T13:58:55.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.244+0000 7f3bec221700 1 -- start start 2024-01-24T13:58:55.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.245+0000 7f3bec221700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be40664c0 0x7f3be4165060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.245+0000 7f3bec221700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be40bdb30 0x7f3be41378b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.245+0000 7f3bec221700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be4134ea0 0x7f3be4137da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.245+0000 7f3bec221700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be4134130 con 0x7f3be40664c0 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.245+0000 7f3bec221700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be40c1fd0 con 0x7f3be40bdb30 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.245+0000 7f3bec221700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be40c22b0 con 0x7f3be4134ea0 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.245+0000 7f3be9fbd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be40664c0 0x7f3be4165060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.245+0000 7f3be9fbd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be40664c0 0x7f3be4165060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33598/0 (socket says 172.21.15.29:33598) 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.245+0000 7f3be9fbd700 1 -- 172.21.15.29:0/1569636540 learned_addr learned my addr 172.21.15.29:0/1569636540 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.245+0000 7f3be97bc700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be40bdb30 0x7f3be41378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.245+0000 7f3bea7be700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be4134ea0 0x7f3be4137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.246+0000 7f3be9fbd700 1 -- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be4134ea0 msgr2=0x7f3be4137da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.246+0000 7f3be9fbd700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be4134ea0 0x7f3be4137da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.246+0000 7f3be9fbd700 1 -- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be40bdb30 msgr2=0x7f3be41378b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.246+0000 7f3be9fbd700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be40bdb30 0x7f3be41378b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.246+0000 7f3be9fbd700 1 -- 172.21.15.29:0/1569636540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd80092a0 con 0x7f3be40664c0 2024-01-24T13:58:55.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.246+0000 7f3be97bc700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be40bdb30 0x7f3be41378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:55.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.246+0000 7f3be9fbd700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be40664c0 0x7f3be4165060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f3bd400c590 tx=0x7f3bd4007ce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:55.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.246+0000 7f3be1ffb700 1 -- 172.21.15.29:0/1569636540 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd40072e0 con 0x7f3be40664c0 2024-01-24T13:58:55.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.246+0000 7f3be1ffb700 1 -- 172.21.15.29:0/1569636540 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd4007440 con 0x7f3be40664c0 2024-01-24T13:58:55.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.246+0000 7f3bec221700 1 -- 172.21.15.29:0/1569636540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be40c2590 con 0x7f3be40664c0 2024-01-24T13:58:55.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.246+0000 7f3be1ffb700 1 -- 172.21.15.29:0/1569636540 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd40057a0 con 0x7f3be40664c0 2024-01-24T13:58:55.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.246+0000 7f3bec221700 1 -- 172.21.15.29:0/1569636540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be4138290 con 0x7f3be40664c0 2024-01-24T13:58:55.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.247+0000 7f3bec221700 1 -- 172.21.15.29:0/1569636540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be40c2ab0 con 0x7f3be40664c0 2024-01-24T13:58:55.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.248+0000 7f3be1ffb700 1 -- 172.21.15.29:0/1569636540 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3bd4005bd0 con 0x7f3be40664c0 2024-01-24T13:58:55.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.248+0000 7f3be1ffb700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3bd005f150 0x7f3bd0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.248+0000 7f3be1ffb700 1 -- 172.21.15.29:0/1569636540 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f3bd4079e60 con 0x7f3be40664c0 2024-01-24T13:58:55.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.249+0000 7f3be97bc700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3bd005f150 0x7f3bd0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.249+0000 7f3be97bc700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3bd005f150 0x7f3bd0061610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f3bd8009760 tx=0x7f3bd8017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:55.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.253+0000 7f3be1ffb700 1 -- 172.21.15.29:0/1569636540 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3be40c2ab0 con 0x7f3be40664c0 2024-01-24T13:58:55.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.432+0000 7f3bec221700 1 -- 172.21.15.29:0/1569636540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3be40c2ab0 con 0x7f3be40664c0 2024-01-24T13:58:55.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.433+0000 7f3be1ffb700 1 -- 172.21.15.29:0/1569636540 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3be40c2ab0 con 0x7f3be40664c0 2024-01-24T13:58:55.434 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-01-24T13:58:55.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.435+0000 7f3bec221700 1 -- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3bd005f150 msgr2=0x7f3bd0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.435+0000 7f3bec221700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3bd005f150 0x7f3bd0061610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f3bd8009760 tx=0x7f3bd8017000).stop 2024-01-24T13:58:55.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.435+0000 7f3bec221700 1 -- 172.21.15.29:0/1569636540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be40664c0 msgr2=0x7f3be4165060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.435+0000 7f3bec221700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be40664c0 0x7f3be4165060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f3bd400c590 tx=0x7f3bd4007ce0).stop 2024-01-24T13:58:55.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.435+0000 7f3bec221700 1 -- 172.21.15.29:0/1569636540 shutdown_connections 2024-01-24T13:58:55.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.435+0000 7f3bec221700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be4134ea0 0x7f3be4137da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.435+0000 7f3bec221700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3bd005f150 0x7f3bd0061610 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.435+0000 7f3bec221700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be40664c0 0x7f3be4165060 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.435+0000 7f3bec221700 1 --2- 172.21.15.29:0/1569636540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be40bdb30 0x7f3be41378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.435+0000 7f3bec221700 1 -- 172.21.15.29:0/1569636540 >> 172.21.15.29:0/1569636540 conn(0x7f3be40c8bd0 msgr2=0x7f3be40c7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:55.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.435+0000 7f3bec221700 1 -- 172.21.15.29:0/1569636540 shutdown_connections 2024-01-24T13:58:55.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.435+0000 7f3bec221700 1 -- 172.21.15.29:0/1569636540 wait complete. 2024-01-24T13:58:55.450 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:55.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.579+0000 7f8d11814700 1 Processor -- start 2024-01-24T13:58:55.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.579+0000 7f8d11814700 1 -- start start 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.580+0000 7f8d11814700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8d0c1540a0 0x7f8d0c0c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.580+0000 7f8d11814700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d0c062ec0 0x7f8d0c069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.580+0000 7f8d11814700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0c065360 con 0x7f8d0c1550f0 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.580+0000 7f8d11814700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d0c0a74f0 con 0x7f8d0c1540a0 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.580+0000 7f8d11814700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0a6ce0 con 0x7f8d0c062ec0 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.581+0000 7f8d0a7fc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8d0c1550f0 0x7f8d0c0627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46278/0 (socket says 172.21.15.29:46278) 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.581+0000 7f8d0a7fc700 1 -- 172.21.15.29:0/3354926637 learned_addr learned my addr 172.21.15.29:0/3354926637 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.581+0000 7f8d0b7fe700 1 --2- 172.21.15.29:0/3354926637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d0c062ec0 0x7f8d0c069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.581+0000 7f8d0affd700 1 --2- 172.21.15.29:0/3354926637 >> v2:172.21.15.163:3300/0 conn(0x7f8d0c1540a0 0x7f8d0c0c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.581+0000 7f8d09ffb700 1 -- 172.21.15.29:0/3354926637 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144837590 0 0) 0x7f8d0c065360 con 0x7f8d0c1550f0 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.581+0000 7f8d09ffb700 1 -- 172.21.15.29:0/3354926637 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cf40033f0 con 0x7f8d0c1550f0 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.581+0000 7f8d0b7fe700 1 -- 172.21.15.29:0/3354926637 >> v2:172.21.15.163:3300/0 conn(0x7f8d0c1540a0 msgr2=0x7f8d0c0c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.581+0000 7f8d0b7fe700 1 --2- 172.21.15.29:0/3354926637 >> v2:172.21.15.163:3300/0 conn(0x7f8d0c1540a0 0x7f8d0c0c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.581+0000 7f8d0b7fe700 1 -- 172.21.15.29:0/3354926637 >> v1:172.21.15.29:6789/0 conn(0x7f8d0c1550f0 legacy=0x7f8d0c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.581+0000 7f8d0b7fe700 1 -- 172.21.15.29:0/3354926637 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c0636f0 con 0x7f8d0c062ec0 2024-01-24T13:58:55.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.581+0000 7f8d0b7fe700 1 --2- 172.21.15.29:0/3354926637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d0c062ec0 0x7f8d0c069240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f8d04009e60 tx=0x7f8d04009780).ready entity=mon.2 client_cookie=2227b7dbb2911e48 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:55.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.581+0000 7f8d09ffb700 1 -- 172.21.15.29:0/3354926637 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400e070 con 0x7f8d0c062ec0 2024-01-24T13:58:55.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.582+0000 7f8d09ffb700 1 -- 172.21.15.29:0/3354926637 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0400bab0 con 0x7f8d0c062ec0 2024-01-24T13:58:55.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.582+0000 7f8d09ffb700 1 -- 172.21.15.29:0/3354926637 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400be30 con 0x7f8d0c062ec0 2024-01-24T13:58:55.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.582+0000 7f8d11814700 1 -- 172.21.15.29:0/3354926637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d0c062ec0 msgr2=0x7f8d0c069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.582+0000 7f8d11814700 1 --2- 172.21.15.29:0/3354926637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d0c062ec0 0x7f8d0c069240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f8d04009e60 tx=0x7f8d04009780).stop 2024-01-24T13:58:55.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.582+0000 7f8d11814700 1 -- 172.21.15.29:0/3354926637 shutdown_connections 2024-01-24T13:58:55.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.582+0000 7f8d11814700 1 --2- 172.21.15.29:0/3354926637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d0c062ec0 0x7f8d0c069240 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.582+0000 7f8d11814700 1 --2- 172.21.15.29:0/3354926637 >> v2:172.21.15.163:3300/0 conn(0x7f8d0c1540a0 0x7f8d0c0c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.582+0000 7f8d11814700 1 -- 172.21.15.29:0/3354926637 >> 172.21.15.29:0/3354926637 conn(0x7f8d0c0c85c0 msgr2=0x7f8d0c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:55.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.582+0000 7f8d11814700 1 -- 172.21.15.29:0/3354926637 shutdown_connections 2024-01-24T13:58:55.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.582+0000 7f8d11814700 1 -- 172.21.15.29:0/3354926637 wait complete. 2024-01-24T13:58:55.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.584+0000 7f8d11814700 1 Processor -- start 2024-01-24T13:58:55.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.584+0000 7f8d11814700 1 -- start start 2024-01-24T13:58:55.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.584+0000 7f8d11814700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d0c062ec0 0x7f8d0c164680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.584+0000 7f8d11814700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d0c1540a0 0x7f8d0c0acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.584+0000 7f8d11814700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d0c1550f0 0x7f8d0c167240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.584+0000 7f8d11814700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c163c30 con 0x7f8d0c1550f0 2024-01-24T13:58:55.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.584+0000 7f8d11814700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c1641f0 con 0x7f8d0c1540a0 2024-01-24T13:58:55.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.584+0000 7f8d11814700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c167cc0 con 0x7f8d0c062ec0 2024-01-24T13:58:55.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d0b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d0c1550f0 0x7f8d0c167240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d0b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d0c1550f0 0x7f8d0c167240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33612/0 (socket says 172.21.15.29:33612) 2024-01-24T13:58:55.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d0b7fe700 1 -- 172.21.15.29:0/1959814577 learned_addr learned my addr 172.21.15.29:0/1959814577 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:55.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d0affd700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d0c062ec0 0x7f8d0c164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d0a7fc700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d0c1540a0 0x7f8d0c0acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d0b7fe700 1 -- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d0c062ec0 msgr2=0x7f8d0c164680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d0b7fe700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d0c062ec0 0x7f8d0c164680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d0b7fe700 1 -- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d0c1540a0 msgr2=0x7f8d0c0acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d0b7fe700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d0c1540a0 0x7f8d0c0acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d0b7fe700 1 -- 172.21.15.29:0/1959814577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d04009a20 con 0x7f8d0c1550f0 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d0b7fe700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d0c1550f0 0x7f8d0c167240 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f8d0400b610 tx=0x7f8d04009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8cf2ffd700 1 -- 172.21.15.29:0/1959814577 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0401d630 con 0x7f8d0c1550f0 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8cf2ffd700 1 -- 172.21.15.29:0/1959814577 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0401d7b0 con 0x7f8d0c1550f0 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d11814700 1 -- 172.21.15.29:0/1959814577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c167fa0 con 0x7f8d0c1550f0 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8cf2ffd700 1 -- 172.21.15.29:0/1959814577 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0401db70 con 0x7f8d0c1550f0 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.585+0000 7f8d11814700 1 -- 172.21.15.29:0/1959814577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c164b70 con 0x7f8d0c1550f0 2024-01-24T13:58:55.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.586+0000 7f8d11814700 1 -- 172.21.15.29:0/1959814577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0c164380 con 0x7f8d0c1550f0 2024-01-24T13:58:55.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.587+0000 7f8cf2ffd700 1 -- 172.21.15.29:0/1959814577 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8d04003040 con 0x7f8d0c1550f0 2024-01-24T13:58:55.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.587+0000 7f8cf2ffd700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8cf805f150 0x7f8cf8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.587+0000 7f8cf2ffd700 1 -- 172.21.15.29:0/1959814577 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f8d0407c260 con 0x7f8d0c1550f0 2024-01-24T13:58:55.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.587+0000 7f8d0affd700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8cf805f150 0x7f8cf8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.588+0000 7f8d0affd700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8cf805f150 0x7f8cf8061610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f8d0c154560 tx=0x7f8cfc008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:55.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.592+0000 7f8cf2ffd700 1 -- 172.21.15.29:0/1959814577 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8d0404d220 con 0x7f8d0c1550f0 2024-01-24T13:58:55.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.776+0000 7f8d11814700 1 -- 172.21.15.29:0/1959814577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d0c06a5f0 con 0x7f8d0c1550f0 2024-01-24T13:58:55.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.777+0000 7f8cf2ffd700 1 -- 172.21.15.29:0/1959814577 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7f8d04050840 con 0x7f8d0c1550f0 2024-01-24T13:58:55.777 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:55.777 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:55.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.779+0000 7f8d11814700 1 -- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8cf805f150 msgr2=0x7f8cf8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.779+0000 7f8d11814700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8cf805f150 0x7f8cf8061610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f8d0c154560 tx=0x7f8cfc008040).stop 2024-01-24T13:58:55.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.779+0000 7f8d11814700 1 -- 172.21.15.29:0/1959814577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d0c1550f0 msgr2=0x7f8d0c167240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.779+0000 7f8d11814700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d0c1550f0 0x7f8d0c167240 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f8d0400b610 tx=0x7f8d04009780).stop 2024-01-24T13:58:55.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.779+0000 7f8d11814700 1 -- 172.21.15.29:0/1959814577 shutdown_connections 2024-01-24T13:58:55.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.779+0000 7f8d11814700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8cf805f150 0x7f8cf8061610 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.779+0000 7f8d11814700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d0c1550f0 0x7f8d0c167240 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.779+0000 7f8d11814700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d0c062ec0 0x7f8d0c164680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.779+0000 7f8d11814700 1 --2- 172.21.15.29:0/1959814577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d0c1540a0 0x7f8d0c0acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.779+0000 7f8d11814700 1 -- 172.21.15.29:0/1959814577 >> 172.21.15.29:0/1959814577 conn(0x7f8d0c0c85c0 msgr2=0x7f8d0c0627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:55.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.779+0000 7f8d11814700 1 -- 172.21.15.29:0/1959814577 shutdown_connections 2024-01-24T13:58:55.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.779+0000 7f8d11814700 1 -- 172.21.15.29:0/1959814577 wait complete. 2024-01-24T13:58:55.793 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T13:58:16.949817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T13:58:18.174460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-24T13:58:21.232882+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T13:58:55.793 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T13:58:55.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.925+0000 7fe59761d700 1 Processor -- start 2024-01-24T13:58:55.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.925+0000 7fe59761d700 1 -- start start 2024-01-24T13:58:55.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.926+0000 7fe59761d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe590134ea0 0x7fe5900c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.926+0000 7fe59761d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe590066bd0 0x7fe5900bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.926+0000 7fe59761d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5900686a0 con 0x7fe5900c36c0 2024-01-24T13:58:55.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.926+0000 7fe59761d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe590068b30 con 0x7fe590066bd0 2024-01-24T13:58:55.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.926+0000 7fe59761d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe590067a90 con 0x7fe590134ea0 2024-01-24T13:58:55.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.926+0000 7fe595bba700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe590066bd0 0x7fe5900bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.926+0000 7fe594bb8700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe5900c36c0 0x7fe5900664c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46290/0 (socket says 172.21.15.29:46290) 2024-01-24T13:58:55.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.926+0000 7fe595bba700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe590066bd0 0x7fe5900bdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33628/0 (socket says 172.21.15.29:33628) 2024-01-24T13:58:55.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.926+0000 7fe594bb8700 1 -- 172.21.15.29:0/3890837241 learned_addr learned my addr 172.21.15.29:0/3890837241 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:55.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.927+0000 7fe5953b9700 1 --2- 172.21.15.29:0/3890837241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe590134ea0 0x7fe5900c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.927+0000 7fe595bba700 1 -- 172.21.15.29:0/3890837241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe590134ea0 msgr2=0x7fe5900c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:55.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.927+0000 7fe595bba700 1 --2- 172.21.15.29:0/3890837241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe590134ea0 0x7fe5900c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.927+0000 7fe595bba700 1 -- 172.21.15.29:0/3890837241 >> v1:172.21.15.29:6789/0 conn(0x7fe5900c36c0 legacy=0x7fe5900664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.927+0000 7fe595bba700 1 -- 172.21.15.29:0/3890837241 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590067d70 con 0x7fe590066bd0 2024-01-24T13:58:55.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.927+0000 7fe595bba700 1 --2- 172.21.15.29:0/3890837241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe590066bd0 0x7fe5900bdb30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fe588009e60 tx=0x7fe588009780).ready entity=mon.0 client_cookie=c9b1e308549da1a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:55.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.927+0000 7fe583fff700 1 -- 172.21.15.29:0/3890837241 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58800e070 con 0x7fe590066bd0 2024-01-24T13:58:55.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.927+0000 7fe583fff700 1 -- 172.21.15.29:0/3890837241 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58800bc30 con 0x7fe590066bd0 2024-01-24T13:58:55.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.927+0000 7fe583fff700 1 -- 172.21.15.29:0/3890837241 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588012410 con 0x7fe590066bd0 2024-01-24T13:58:55.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.928+0000 7fe59761d700 1 -- 172.21.15.29:0/3890837241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe590066bd0 msgr2=0x7fe5900bdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.928+0000 7fe59761d700 1 --2- 172.21.15.29:0/3890837241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe590066bd0 0x7fe5900bdb30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fe588009e60 tx=0x7fe588009780).stop 2024-01-24T13:58:55.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.928+0000 7fe59761d700 1 -- 172.21.15.29:0/3890837241 shutdown_connections 2024-01-24T13:58:55.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.928+0000 7fe59761d700 1 --2- 172.21.15.29:0/3890837241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe590066bd0 0x7fe5900bdb30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.928+0000 7fe59761d700 1 --2- 172.21.15.29:0/3890837241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe590134ea0 0x7fe5900c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.928+0000 7fe59761d700 1 -- 172.21.15.29:0/3890837241 >> 172.21.15.29:0/3890837241 conn(0x7fe5900c8bd0 msgr2=0x7fe5900c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:55.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.928+0000 7fe59761d700 1 -- 172.21.15.29:0/3890837241 shutdown_connections 2024-01-24T13:58:55.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.928+0000 7fe59761d700 1 -- 172.21.15.29:0/3890837241 wait complete. 2024-01-24T13:58:55.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.929+0000 7fe59761d700 1 Processor -- start 2024-01-24T13:58:55.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.929+0000 7fe59761d700 1 -- start start 2024-01-24T13:58:55.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.930+0000 7fe59761d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe590066bd0 0x7fe5900aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.930+0000 7fe59761d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5900c36c0 0x7fe590134520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.930+0000 7fe59761d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe590134ea0 0x7fe5900abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.930+0000 7fe59761d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe590064640 con 0x7fe590134ea0 2024-01-24T13:58:55.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.930+0000 7fe59761d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900ab110 con 0x7fe590066bd0 2024-01-24T13:58:55.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.930+0000 7fe59761d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe590064030 con 0x7fe5900c36c0 2024-01-24T13:58:55.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.930+0000 7fe595bba700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe590134ea0 0x7fe5900abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe595bba700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe590134ea0 0x7fe5900abb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33640/0 (socket says 172.21.15.29:33640) 2024-01-24T13:58:55.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe595bba700 1 -- 172.21.15.29:0/57928980 learned_addr learned my addr 172.21.15.29:0/57928980 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe5953b9700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe590066bd0 0x7fe5900aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe594bb8700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5900c36c0 0x7fe590134520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe595bba700 1 -- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5900c36c0 msgr2=0x7fe590134520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe595bba700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5900c36c0 0x7fe590134520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe595bba700 1 -- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe590066bd0 msgr2=0x7fe5900aac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe595bba700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe590066bd0 0x7fe5900aac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe595bba700 1 -- 172.21.15.29:0/57928980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe588009a20 con 0x7fe590134ea0 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe595bba700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe590134ea0 0x7fe5900abb80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fe59006c260 tx=0x7fe588009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe5817fa700 1 -- 172.21.15.29:0/57928980 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58801d630 con 0x7fe590134ea0 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe5817fa700 1 -- 172.21.15.29:0/57928980 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58801d790 con 0x7fe590134ea0 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe59761d700 1 -- 172.21.15.29:0/57928980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5900ab370 con 0x7fe590134ea0 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe5817fa700 1 -- 172.21.15.29:0/57928980 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58801db30 con 0x7fe590134ea0 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.931+0000 7fe59761d700 1 -- 172.21.15.29:0/57928980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe590065880 con 0x7fe590134ea0 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.932+0000 7fe59761d700 1 -- 172.21.15.29:0/57928980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5900475e0 con 0x7fe590134ea0 2024-01-24T13:58:55.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.933+0000 7fe5817fa700 1 -- 172.21.15.29:0/57928980 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe588003040 con 0x7fe590134ea0 2024-01-24T13:58:55.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.933+0000 7fe5817fa700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe57405f150 0x7fe574061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:55.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.933+0000 7fe5817fa700 1 -- 172.21.15.29:0/57928980 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe58807c330 con 0x7fe590134ea0 2024-01-24T13:58:55.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.933+0000 7fe5953b9700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe57405f150 0x7fe574061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:55.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.934+0000 7fe5953b9700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe57405f150 0x7fe574061610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fe58400bb10 tx=0x7fe584008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:55.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:55.938+0000 7fe5817fa700 1 -- 172.21.15.29:0/57928980 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe58804d2f0 con 0x7fe590134ea0 2024-01-24T13:58:56.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.116+0000 7fe59761d700 1 -- 172.21.15.29:0/57928980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe590068050 con 0x7fe590134ea0 2024-01-24T13:58:56.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.117+0000 7fe5817fa700 1 -- 172.21.15.29:0/57928980 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe588050910 con 0x7fe590134ea0 2024-01-24T13:58:56.117 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T13:58:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.119+0000 7fe59761d700 1 -- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe57405f150 msgr2=0x7fe574061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.119+0000 7fe59761d700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe57405f150 0x7fe574061610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fe58400bb10 tx=0x7fe584008040).stop 2024-01-24T13:58:56.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.119+0000 7fe59761d700 1 -- 172.21.15.29:0/57928980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe590134ea0 msgr2=0x7fe5900abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.119+0000 7fe59761d700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe590134ea0 0x7fe5900abb80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fe59006c260 tx=0x7fe588009780).stop 2024-01-24T13:58:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.119+0000 7fe59761d700 1 -- 172.21.15.29:0/57928980 shutdown_connections 2024-01-24T13:58:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.119+0000 7fe59761d700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe57405f150 0x7fe574061610 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.119+0000 7fe59761d700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe590134ea0 0x7fe5900abb80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.119+0000 7fe59761d700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe590066bd0 0x7fe5900aac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.119+0000 7fe59761d700 1 --2- 172.21.15.29:0/57928980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5900c36c0 0x7fe590134520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.119+0000 7fe59761d700 1 -- 172.21.15.29:0/57928980 >> 172.21.15.29:0/57928980 conn(0x7fe5900c8bd0 msgr2=0x7fe5900c76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.119+0000 7fe59761d700 1 -- 172.21.15.29:0/57928980 shutdown_connections 2024-01-24T13:58:56.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.119+0000 7fe59761d700 1 -- 172.21.15.29:0/57928980 wait complete. 2024-01-24T13:58:56.133 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T13:58:56.243 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T13:58:56.246 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T13:58:56.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.265+0000 7efed5313700 1 Processor -- start 2024-01-24T13:58:56.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.265+0000 7efed5313700 1 -- start start 2024-01-24T13:58:56.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.266+0000 7efed5313700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efed00c8490 0x7efed00a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.266+0000 7efed5313700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed00bfd40 0x7efed00c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.266+0000 7efed5313700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed0152240 con 0x7efed0064760 2024-01-24T13:58:56.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.266+0000 7efed5313700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed01586c0 con 0x7efed00c8490 2024-01-24T13:58:56.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.266+0000 7efed5313700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed00aa670 con 0x7efed00bfd40 2024-01-24T13:58:56.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.266+0000 7efece7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efed00c8490 0x7efed00a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.266+0000 7efece7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efed00c8490 0x7efed00a9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33656/0 (socket says 172.21.15.29:33656) 2024-01-24T13:58:56.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.266+0000 7efece7fc700 1 -- 172.21.15.29:0/425151419 learned_addr learned my addr 172.21.15.29:0/425151419 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:56.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.266+0000 7efecf7fe700 1 --2- 172.21.15.29:0/425151419 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed00bfd40 0x7efed00c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.267+0000 7efece7fc700 1 -- 172.21.15.29:0/425151419 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed00bfd40 msgr2=0x7efed00c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:56.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.267+0000 7efece7fc700 1 --2- 172.21.15.29:0/425151419 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed00bfd40 0x7efed00c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.267+0000 7efece7fc700 1 -- 172.21.15.29:0/425151419 >> v1:172.21.15.163:6789/0 conn(0x7efed0064760 legacy=0x7efed0064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.267+0000 7efece7fc700 1 -- 172.21.15.29:0/425151419 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed0157d90 con 0x7efed00c8490 2024-01-24T13:58:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.267+0000 7efecdffb700 1 -- 172.21.15.29:0/425151419 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925102693 0 0) 0x7efed0152240 con 0x7efed0064760 2024-01-24T13:58:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.267+0000 7efece7fc700 1 --2- 172.21.15.29:0/425151419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efed00c8490 0x7efed00a9d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7efec4006fd0 tx=0x7efec40090f0).ready entity=mon.0 client_cookie=12a5f4b2503067e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.267+0000 7efecdffb700 1 -- 172.21.15.29:0/425151419 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec400b610 con 0x7efed00c8490 2024-01-24T13:58:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.267+0000 7efecdffb700 1 -- 172.21.15.29:0/425151419 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec400b770 con 0x7efed00c8490 2024-01-24T13:58:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.267+0000 7efecdffb700 1 -- 172.21.15.29:0/425151419 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec400baf0 con 0x7efed00c8490 2024-01-24T13:58:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.268+0000 7efed5313700 1 -- 172.21.15.29:0/425151419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efed00c8490 msgr2=0x7efed00a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.268+0000 7efed5313700 1 --2- 172.21.15.29:0/425151419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efed00c8490 0x7efed00a9d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7efec4006fd0 tx=0x7efec40090f0).stop 2024-01-24T13:58:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.268+0000 7efed5313700 1 -- 172.21.15.29:0/425151419 shutdown_connections 2024-01-24T13:58:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.268+0000 7efed5313700 1 --2- 172.21.15.29:0/425151419 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed00bfd40 0x7efed00c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.268+0000 7efed5313700 1 --2- 172.21.15.29:0/425151419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efed00c8490 0x7efed00a9d90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.268+0000 7efed5313700 1 -- 172.21.15.29:0/425151419 >> 172.21.15.29:0/425151419 conn(0x7efed00cb8b0 msgr2=0x7efed00c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:56.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.268+0000 7efed5313700 1 -- 172.21.15.29:0/425151419 shutdown_connections 2024-01-24T13:58:56.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.268+0000 7efed5313700 1 -- 172.21.15.29:0/425151419 wait complete. 2024-01-24T13:58:56.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.269+0000 7efed5313700 1 Processor -- start 2024-01-24T13:58:56.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.270+0000 7efed5313700 1 -- start start 2024-01-24T13:58:56.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efed5313700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efed0064760 0x7efed01658d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efed5313700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed00bfd40 0x7efed0167510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efed5313700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efed00c8490 0x7efed0167d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efed5313700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed00c3d50 con 0x7efed0064760 2024-01-24T13:58:56.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efed5313700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed0154b80 con 0x7efed00c8490 2024-01-24T13:58:56.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efed5313700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed00c3420 con 0x7efed00bfd40 2024-01-24T13:58:56.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efeceffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efed0064760 0x7efed01658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efeceffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efed0064760 0x7efed01658d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33668/0 (socket says 172.21.15.29:33668) 2024-01-24T13:58:56.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efeceffd700 1 -- 172.21.15.29:0/814148322 learned_addr learned my addr 172.21.15.29:0/814148322 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:56.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efecf7fe700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efed00c8490 0x7efed0167d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efece7fc700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed00bfd40 0x7efed0167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efeceffd700 1 -- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed00bfd40 msgr2=0x7efed0167510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efeceffd700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed00bfd40 0x7efed0167510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efeceffd700 1 -- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efed00c8490 msgr2=0x7efed0167d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efeceffd700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efed00c8490 0x7efed0167d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efeceffd700 1 -- 172.21.15.29:0/814148322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec4009370 con 0x7efed0064760 2024-01-24T13:58:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.271+0000 7efeceffd700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efed0064760 0x7efed01658d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7efec0001eb0 tx=0x7efec0007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.272+0000 7efeb6ffd700 1 -- 172.21.15.29:0/814148322 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000ce60 con 0x7efed0064760 2024-01-24T13:58:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.272+0000 7efeb6ffd700 1 -- 172.21.15.29:0/814148322 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec000f470 con 0x7efed0064760 2024-01-24T13:58:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.272+0000 7efed5313700 1 -- 172.21.15.29:0/814148322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed00c3700 con 0x7efed0064760 2024-01-24T13:58:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.272+0000 7efeb6ffd700 1 -- 172.21.15.29:0/814148322 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000f810 con 0x7efed0064760 2024-01-24T13:58:56.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.272+0000 7efed5313700 1 -- 172.21.15.29:0/814148322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed0167a00 con 0x7efed0064760 2024-01-24T13:58:56.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.273+0000 7efed5313700 1 -- 172.21.15.29:0/814148322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed00c4030 con 0x7efed0064760 2024-01-24T13:58:56.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.273+0000 7efeb6ffd700 1 -- 172.21.15.29:0/814148322 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7efec000fb80 con 0x7efed0064760 2024-01-24T13:58:56.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.274+0000 7efeb6ffd700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efebc05f150 0x7efebc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.274+0000 7efeb6ffd700 1 -- 172.21.15.29:0/814148322 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7efec0079a60 con 0x7efed0064760 2024-01-24T13:58:56.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.274+0000 7efece7fc700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efebc05f150 0x7efebc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.275+0000 7efece7fc700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efebc05f150 0x7efebc061610 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7efec4009130 tx=0x7efec4017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:56.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.278+0000 7efeb6ffd700 1 -- 172.21.15.29:0/814148322 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7efec004aa20 con 0x7efed0064760 2024-01-24T13:58:56.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.459+0000 7efed5313700 1 -- 172.21.15.29:0/814148322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efed00475e0 con 0x7efed0064760 2024-01-24T13:58:56.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.460+0000 7efeb6ffd700 1 -- 172.21.15.29:0/814148322 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7efec004e040 con 0x7efed0064760 2024-01-24T13:58:56.460 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T13:58:56.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.462+0000 7efed5313700 1 -- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efebc05f150 msgr2=0x7efebc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.462+0000 7efed5313700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efebc05f150 0x7efebc061610 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7efec4009130 tx=0x7efec4017000).stop 2024-01-24T13:58:56.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.462+0000 7efed5313700 1 -- 172.21.15.29:0/814148322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efed0064760 msgr2=0x7efed01658d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.462+0000 7efed5313700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efed0064760 0x7efed01658d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7efec0001eb0 tx=0x7efec0007c70).stop 2024-01-24T13:58:56.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.462+0000 7efed5313700 1 -- 172.21.15.29:0/814148322 shutdown_connections 2024-01-24T13:58:56.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.462+0000 7efed5313700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efed00c8490 0x7efed0167d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.462+0000 7efed5313700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efebc05f150 0x7efebc061610 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.462+0000 7efed5313700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efed0064760 0x7efed01658d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.462+0000 7efed5313700 1 --2- 172.21.15.29:0/814148322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed00bfd40 0x7efed0167510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.462+0000 7efed5313700 1 -- 172.21.15.29:0/814148322 >> 172.21.15.29:0/814148322 conn(0x7efed00cb8b0 msgr2=0x7efed00c8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:56.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.462+0000 7efed5313700 1 -- 172.21.15.29:0/814148322 shutdown_connections 2024-01-24T13:58:56.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.462+0000 7efed5313700 1 -- 172.21.15.29:0/814148322 wait complete. 2024-01-24T13:58:56.476 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T13:58:56.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.608+0000 7fd3d9015700 1 Processor -- start 2024-01-24T13:58:56.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.608+0000 7fd3d9015700 1 -- start start 2024-01-24T13:58:56.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.609+0000 7fd3d9015700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd3d4134ea0 0x7fd3d40c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.609+0000 7fd3d9015700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd3d40c36c0 0x7fd3d40664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.609+0000 7fd3d9015700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3d40669b0 0x7fd3d41559f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.609+0000 7fd3d9015700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d4066e60 con 0x7fd3d40c36c0 2024-01-24T13:58:56.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.609+0000 7fd3d9015700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d40686a0 con 0x7fd3d4134ea0 2024-01-24T13:58:56.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.609+0000 7fd3d9015700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d4068b30 con 0x7fd3d40669b0 2024-01-24T13:58:56.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.609+0000 7fd3d259c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd3d40c36c0 0x7fd3d40664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.609+0000 7fd3d259c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd3d40c36c0 0x7fd3d40664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33674/0 (socket says 172.21.15.29:33674) 2024-01-24T13:58:56.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.609+0000 7fd3d259c700 1 -- 172.21.15.29:0/3690185776 learned_addr learned my addr 172.21.15.29:0/3690185776 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:56.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.610+0000 7fd3d2d9d700 1 --2- 172.21.15.29:0/3690185776 >> v2:172.21.15.163:3300/0 conn(0x7fd3d4134ea0 0x7fd3d40c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.610+0000 7fd3d359e700 1 --2- 172.21.15.29:0/3690185776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3d40669b0 0x7fd3d41559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.610+0000 7fd3d259c700 1 -- 172.21.15.29:0/3690185776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3d40669b0 msgr2=0x7fd3d41559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:56.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.610+0000 7fd3d259c700 1 --2- 172.21.15.29:0/3690185776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3d40669b0 0x7fd3d41559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.610+0000 7fd3d259c700 1 -- 172.21.15.29:0/3690185776 >> v2:172.21.15.163:3300/0 conn(0x7fd3d4134ea0 msgr2=0x7fd3d40c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.610+0000 7fd3d259c700 1 --2- 172.21.15.29:0/3690185776 >> v2:172.21.15.163:3300/0 conn(0x7fd3d4134ea0 0x7fd3d40c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.610+0000 7fd3d259c700 1 -- 172.21.15.29:0/3690185776 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4067a90 con 0x7fd3d40c36c0 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.610+0000 7fd3d259c700 1 --2- 172.21.15.29:0/3690185776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40c36c0 0x7fd3d40664c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fd3d40c66e0 tx=0x7fd3bc0090f0).ready entity=mon.0 client_cookie=fbcb69d4dff86afc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.610+0000 7fd3d1d9b700 1 -- 172.21.15.29:0/3690185776 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00b600 con 0x7fd3d40c36c0 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.610+0000 7fd3d1d9b700 1 -- 172.21.15.29:0/3690185776 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc00b760 con 0x7fd3d40c36c0 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.610+0000 7fd3d1d9b700 1 -- 172.21.15.29:0/3690185776 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00bae0 con 0x7fd3d40c36c0 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.611+0000 7fd3d9015700 1 -- 172.21.15.29:0/3690185776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40c36c0 msgr2=0x7fd3d40664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.611+0000 7fd3d9015700 1 --2- 172.21.15.29:0/3690185776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40c36c0 0x7fd3d40664c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fd3d40c66e0 tx=0x7fd3bc0090f0).stop 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.611+0000 7fd3d9015700 1 -- 172.21.15.29:0/3690185776 shutdown_connections 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.611+0000 7fd3d9015700 1 --2- 172.21.15.29:0/3690185776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3d40669b0 0x7fd3d41559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.611+0000 7fd3d9015700 1 --2- 172.21.15.29:0/3690185776 >> v2:172.21.15.163:3300/0 conn(0x7fd3d4134ea0 0x7fd3d40c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.611+0000 7fd3d9015700 1 --2- 172.21.15.29:0/3690185776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40c36c0 0x7fd3d40664c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.611+0000 7fd3d9015700 1 -- 172.21.15.29:0/3690185776 >> 172.21.15.29:0/3690185776 conn(0x7fd3d40c8bd0 msgr2=0x7fd3d40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:56.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.611+0000 7fd3d9015700 1 -- 172.21.15.29:0/3690185776 shutdown_connections 2024-01-24T13:58:56.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.611+0000 7fd3d9015700 1 -- 172.21.15.29:0/3690185776 wait complete. 2024-01-24T13:58:56.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.612+0000 7fd3d9015700 1 Processor -- start 2024-01-24T13:58:56.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.613+0000 7fd3d9015700 1 -- start start 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d9015700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3d40669b0 0x7fd3d40a86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d9015700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40c36c0 0x7fd3d40aa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d9015700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3d4134ea0 0x7fd3d40aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d9015700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40ab310 con 0x7fd3d40c36c0 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d9015700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40c1850 con 0x7fd3d40669b0 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d9015700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40c0c40 con 0x7fd3d4134ea0 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40c36c0 0x7fd3d40aa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40c36c0 0x7fd3d40aa930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33680/0 (socket says 172.21.15.29:33680) 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d259c700 1 -- 172.21.15.29:0/3699680693 learned_addr learned my addr 172.21.15.29:0/3699680693 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d2d9d700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3d40669b0 0x7fd3d40a86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d359e700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3d4134ea0 0x7fd3d40aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d259c700 1 -- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3d4134ea0 msgr2=0x7fd3d40aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d259c700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3d4134ea0 0x7fd3d40aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d259c700 1 -- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3d40669b0 msgr2=0x7fd3d40a86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d259c700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3d40669b0 0x7fd3d40a86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d259c700 1 -- 172.21.15.29:0/3699680693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc009370 con 0x7fd3d40c36c0 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d359e700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3d4134ea0 0x7fd3d40aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d2d9d700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3d40669b0 0x7fd3d40a86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d259c700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40c36c0 0x7fd3d40aa930 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fd3bc009130 tx=0x7fd3bc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3caffd700 1 -- 172.21.15.29:0/3699680693 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00b8d0 con 0x7fd3d40c36c0 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3caffd700 1 -- 172.21.15.29:0/3699680693 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc00ba30 con 0x7fd3d40c36c0 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.614+0000 7fd3d9015700 1 -- 172.21.15.29:0/3699680693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d40c0f20 con 0x7fd3d40c36c0 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.615+0000 7fd3caffd700 1 -- 172.21.15.29:0/3699680693 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc01c670 con 0x7fd3d40c36c0 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.615+0000 7fd3d9015700 1 -- 172.21.15.29:0/3699680693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d40c14e0 con 0x7fd3d40c36c0 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.616+0000 7fd3d9015700 1 -- 172.21.15.29:0/3699680693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d40c19c0 con 0x7fd3d40c36c0 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.616+0000 7fd3caffd700 1 -- 172.21.15.29:0/3699680693 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd3bc01ca20 con 0x7fd3d40c36c0 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.616+0000 7fd3caffd700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd3b405f150 0x7fd3b4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.616+0000 7fd3caffd700 1 -- 172.21.15.29:0/3699680693 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fd3bc07c290 con 0x7fd3d40c36c0 2024-01-24T13:58:56.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.617+0000 7fd3d2d9d700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd3b405f150 0x7fd3b4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.617+0000 7fd3d2d9d700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd3b405f150 0x7fd3b4061610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fd3c40098c0 tx=0x7fd3c4008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:56.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.621+0000 7fd3caffd700 1 -- 172.21.15.29:0/3699680693 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd3bc04d250 con 0x7fd3d40c36c0 2024-01-24T13:58:56.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.800+0000 7fd3d9015700 1 -- 172.21.15.29:0/3699680693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3d40475e0 con 0x7fd3d40c36c0 2024-01-24T13:58:56.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.801+0000 7fd3caffd700 1 -- 172.21.15.29:0/3699680693 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd3bc02a0a0 con 0x7fd3d40c36c0 2024-01-24T13:58:56.801 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:58:56.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.803+0000 7fd3d9015700 1 -- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd3b405f150 msgr2=0x7fd3b4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.803+0000 7fd3d9015700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd3b405f150 0x7fd3b4061610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fd3c40098c0 tx=0x7fd3c4008040).stop 2024-01-24T13:58:56.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.803+0000 7fd3d9015700 1 -- 172.21.15.29:0/3699680693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40c36c0 msgr2=0x7fd3d40aa930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.803+0000 7fd3d9015700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40c36c0 0x7fd3d40aa930 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fd3bc009130 tx=0x7fd3bc017000).stop 2024-01-24T13:58:56.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.803+0000 7fd3d9015700 1 -- 172.21.15.29:0/3699680693 shutdown_connections 2024-01-24T13:58:56.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.803+0000 7fd3d9015700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3d4134ea0 0x7fd3d40aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.803+0000 7fd3d9015700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3d40669b0 0x7fd3d40a86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.803+0000 7fd3d9015700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd3b405f150 0x7fd3b4061610 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.803+0000 7fd3d9015700 1 --2- 172.21.15.29:0/3699680693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3d40c36c0 0x7fd3d40aa930 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.803+0000 7fd3d9015700 1 -- 172.21.15.29:0/3699680693 >> 172.21.15.29:0/3699680693 conn(0x7fd3d40c8bd0 msgr2=0x7fd3d4151360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:56.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.803+0000 7fd3d9015700 1 -- 172.21.15.29:0/3699680693 shutdown_connections 2024-01-24T13:58:56.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.803+0000 7fd3d9015700 1 -- 172.21.15.29:0/3699680693 wait complete. 2024-01-24T13:58:56.817 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T13:58:56.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f0983b54700 1 Processor -- start 2024-01-24T13:58:56.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f0983b54700 1 -- start start 2024-01-24T13:58:56.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f0983b54700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c062ec0 0x7f097c15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f0983b54700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097c06a360 con 0x7f097c0c8290 2024-01-24T13:58:56.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f0983b54700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097c0bcef0 con 0x7f097c153f40 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f0983b54700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0a74c0 con 0x7f097c062ec0 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f09810ef700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f097c0c8290 0x7f097c0627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46302/0 (socket says 172.21.15.29:46302) 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f09810ef700 1 -- 172.21.15.29:0/648035711 learned_addr learned my addr 172.21.15.29:0/648035711 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f09820f1700 1 --2- 172.21.15.29:0/648035711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c062ec0 0x7f097c15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327493342 0 0) 0x7f097c06a360 con 0x7f097c0c8290 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09680033f0 con 0x7f097c0c8290 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2722372055 0 0) 0x7f09680033f0 con 0x7f097c0c8290 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c062ec0 msgr2=0x7f097c15f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f09808ee700 1 --2- 172.21.15.29:0/648035711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c062ec0 0x7f097c15f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.950+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 >> v1:172.21.15.163:6789/0 conn(0x7f097c153f40 legacy=0x7f097c0a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c069de0 con 0x7f097c0c8290 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f096c001470 con 0x7f097c0c8290 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 >> v1:172.21.15.29:6789/0 conn(0x7f097c0c8290 legacy=0x7f097c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09808ee700 1 --2- 172.21.15.29:0/648035711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0968003910 0x7f0968003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09808ee700 1 --2- 172.21.15.29:0/648035711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c062ec0 0x7f0968004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09808ee700 1 --2- 172.21.15.29:0/648035711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0968004b80 0x7f0968005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0a74c0 con 0x7f0968003910 2024-01-24T13:58:56.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096800b9f0 con 0x7f0968004b80 2024-01-24T13:58:56.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096800bbb0 con 0x7f097c062ec0 2024-01-24T13:58:56.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09818f0700 1 --2- 172.21.15.29:0/648035711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0968003910 0x7f0968003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09810ef700 1 --2- 172.21.15.29:0/648035711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c062ec0 0x7f0968004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09820f1700 1 --2- 172.21.15.29:0/648035711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0968004b80 0x7f0968005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09818f0700 1 -- 172.21.15.29:0/648035711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c062ec0 msgr2=0x7f0968004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:56.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09818f0700 1 --2- 172.21.15.29:0/648035711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c062ec0 0x7f0968004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09818f0700 1 -- 172.21.15.29:0/648035711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0968004b80 msgr2=0x7f0968005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:56.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09818f0700 1 --2- 172.21.15.29:0/648035711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0968004b80 0x7f0968005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09818f0700 1 -- 172.21.15.29:0/648035711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09680033f0 con 0x7f0968003910 2024-01-24T13:58:56.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09818f0700 1 --2- 172.21.15.29:0/648035711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0968003910 0x7f0968003d30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f0978001700 tx=0x7f0978007bb0).ready entity=mon.0 client_cookie=24f6861d55e44498 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:56.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978007020 con 0x7f0968003910 2024-01-24T13:58:56.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0978007180 con 0x7f0968003910 2024-01-24T13:58:56.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.951+0000 7f09808ee700 1 -- 172.21.15.29:0/648035711 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978007500 con 0x7f0968003910 2024-01-24T13:58:56.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.952+0000 7f0983b54700 1 -- 172.21.15.29:0/648035711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0968003910 msgr2=0x7f0968003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.952+0000 7f0983b54700 1 --2- 172.21.15.29:0/648035711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0968003910 0x7f0968003d30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f0978001700 tx=0x7f0978007bb0).stop 2024-01-24T13:58:56.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.952+0000 7f09820f1700 1 -- 172.21.15.29:0/648035711 reap_dead start 2024-01-24T13:58:56.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.952+0000 7f0983b54700 1 -- 172.21.15.29:0/648035711 shutdown_connections 2024-01-24T13:58:56.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.952+0000 7f0983b54700 1 -- 172.21.15.29:0/648035711 >> 172.21.15.29:0/648035711 conn(0x7f097c064390 msgr2=0x7f097c0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:56.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.952+0000 7f0983b54700 1 -- 172.21.15.29:0/648035711 shutdown_connections 2024-01-24T13:58:56.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.952+0000 7f0983b54700 1 -- 172.21.15.29:0/648035711 wait complete. 2024-01-24T13:58:56.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.954+0000 7f0983b54700 1 Processor -- start 2024-01-24T13:58:56.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.954+0000 7f0983b54700 1 -- start start 2024-01-24T13:58:56.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f0983b54700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0968003910 0x7f097c152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f0983b54700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c0c36a0 0x7f097c0a8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f0983b54700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f097c0a8530 0x7f097c1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f0983b54700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c153580 con 0x7f097c0a8530 2024-01-24T13:58:56.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f0983b54700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0abc10 con 0x7f0968003910 2024-01-24T13:58:56.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f0983b54700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0abef0 con 0x7f097c0c36a0 2024-01-24T13:58:56.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f09820f1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f097c0a8530 0x7f097c1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f09820f1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f097c0a8530 0x7f097c1502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33686/0 (socket says 172.21.15.29:33686) 2024-01-24T13:58:56.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f09820f1700 1 -- 172.21.15.29:0/1487844948 learned_addr learned my addr 172.21.15.29:0/1487844948 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:56.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f09818f0700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0968003910 0x7f097c152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f09810ef700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c0c36a0 0x7f097c0a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f09820f1700 1 -- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c0c36a0 msgr2=0x7f097c0a8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f09820f1700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c0c36a0 0x7f097c0a8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f09820f1700 1 -- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0968003910 msgr2=0x7f097c152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:56.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f09820f1700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0968003910 0x7f097c152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:56.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f09820f1700 1 -- 172.21.15.29:0/1487844948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c0ac1d0 con 0x7f097c0a8530 2024-01-24T13:58:56.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f09820f1700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f097c0a8530 0x7f097c1502d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f097c0648e0 tx=0x7f097400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:56.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f0971ffb700 1 -- 172.21.15.29:0/1487844948 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097400e040 con 0x7f097c0a8530 2024-01-24T13:58:56.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f0971ffb700 1 -- 172.21.15.29:0/1487844948 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097400f040 con 0x7f097c0a8530 2024-01-24T13:58:56.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f0983b54700 1 -- 172.21.15.29:0/1487844948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097c0ac4b0 con 0x7f097c0a8530 2024-01-24T13:58:56.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f0971ffb700 1 -- 172.21.15.29:0/1487844948 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097400fd70 con 0x7f097c0a8530 2024-01-24T13:58:56.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.956+0000 7f0983b54700 1 -- 172.21.15.29:0/1487844948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097c1507c0 con 0x7f097c0a8530 2024-01-24T13:58:56.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.957+0000 7f0983b54700 1 -- 172.21.15.29:0/1487844948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097c0ac6a0 con 0x7f097c0a8530 2024-01-24T13:58:56.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.958+0000 7f0971ffb700 1 -- 172.21.15.29:0/1487844948 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0974015660 con 0x7f097c0a8530 2024-01-24T13:58:56.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.958+0000 7f0971ffb700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f096005f150 0x7f0960061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:56.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.958+0000 7f0971ffb700 1 -- 172.21.15.29:0/1487844948 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f097407af20 con 0x7f097c0a8530 2024-01-24T13:58:56.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.958+0000 7f09818f0700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f096005f150 0x7f0960061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:56.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.959+0000 7f09818f0700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f096005f150 0x7f0960061610 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f0978001010 tx=0x7f0978016000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:56.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:56.963+0000 7f0971ffb700 1 -- 172.21.15.29:0/1487844948 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f097404bee0 con 0x7f097c0a8530 2024-01-24T13:58:57.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.145+0000 7f0983b54700 1 -- 172.21.15.29:0/1487844948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f097c150a40 con 0x7f097c0a8530 2024-01-24T13:58:57.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.145+0000 7f0971ffb700 1 -- 172.21.15.29:0/1487844948 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f097404f500 con 0x7f097c0a8530 2024-01-24T13:58:57.145 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:58:57.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.148+0000 7f0983b54700 1 -- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f096005f150 msgr2=0x7f0960061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.148+0000 7f0983b54700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f096005f150 0x7f0960061610 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f0978001010 tx=0x7f0978016000).stop 2024-01-24T13:58:57.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.148+0000 7f0983b54700 1 -- 172.21.15.29:0/1487844948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f097c0a8530 msgr2=0x7f097c1502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.148+0000 7f0983b54700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f097c0a8530 0x7f097c1502d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f097c0648e0 tx=0x7f097400b040).stop 2024-01-24T13:58:57.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.148+0000 7f0983b54700 1 -- 172.21.15.29:0/1487844948 shutdown_connections 2024-01-24T13:58:57.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.148+0000 7f0983b54700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f096005f150 0x7f0960061610 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.148+0000 7f0983b54700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f097c0a8530 0x7f097c1502d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.148+0000 7f0983b54700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0968003910 0x7f097c152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.148+0000 7f0983b54700 1 --2- 172.21.15.29:0/1487844948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f097c0c36a0 0x7f097c0a8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.148+0000 7f0983b54700 1 -- 172.21.15.29:0/1487844948 >> 172.21.15.29:0/1487844948 conn(0x7f097c064390 msgr2=0x7f097c0a6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:57.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.148+0000 7f0983b54700 1 -- 172.21.15.29:0/1487844948 shutdown_connections 2024-01-24T13:58:57.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.148+0000 7f0983b54700 1 -- 172.21.15.29:0/1487844948 wait complete. 2024-01-24T13:58:57.160 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-24T13:58:57.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.293+0000 7f2fea998700 1 Processor -- start 2024-01-24T13:58:57.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.293+0000 7f2fea998700 1 -- start start 2024-01-24T13:58:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.294+0000 7f2fea998700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe413d580 0x7f2fe413d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.294+0000 7f2fea998700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2fe4066520 0x7f2fe40669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.294+0000 7f2fea998700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2fe4157de0 0x7f2fe4158250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.294+0000 7f2fea998700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe4158b50 con 0x7f2fe4066520 2024-01-24T13:58:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.294+0000 7f2fea998700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe4068470 con 0x7f2fe4157de0 2024-01-24T13:58:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.294+0000 7f2fea998700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4066e90 con 0x7f2fe413d580 2024-01-24T13:58:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.294+0000 7f2fe37fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2fe4066520 0x7f2fe40669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.294+0000 7f2fe37fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2fe4066520 0x7f2fe40669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33698/0 (socket says 172.21.15.29:33698) 2024-01-24T13:58:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.294+0000 7f2fe37fe700 1 -- 172.21.15.29:0/4283070155 learned_addr learned my addr 172.21.15.29:0/4283070155 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:57.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.294+0000 7f2fe8f35700 1 --2- 172.21.15.29:0/4283070155 >> v2:172.21.15.163:3300/0 conn(0x7f2fe4157de0 0x7f2fe4158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.294+0000 7f2fe3fff700 1 --2- 172.21.15.29:0/4283070155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe413d580 0x7f2fe413d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fe37fe700 1 -- 172.21.15.29:0/4283070155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe413d580 msgr2=0x7f2fe413d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fe37fe700 1 --2- 172.21.15.29:0/4283070155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe413d580 0x7f2fe413d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fe37fe700 1 -- 172.21.15.29:0/4283070155 >> v2:172.21.15.163:3300/0 conn(0x7f2fe4157de0 msgr2=0x7f2fe4158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fe37fe700 1 --2- 172.21.15.29:0/4283070155 >> v2:172.21.15.163:3300/0 conn(0x7f2fe4157de0 0x7f2fe4158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fe37fe700 1 -- 172.21.15.29:0/4283070155 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe4067b40 con 0x7f2fe4066520 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fe37fe700 1 --2- 172.21.15.29:0/4283070155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fe4066520 0x7f2fe40669a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f2fe4069040 tx=0x7f2fcc0090f0).ready entity=mon.0 client_cookie=35536b851a22dd71 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fe2ffd700 1 -- 172.21.15.29:0/4283070155 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc00f510 con 0x7f2fe4066520 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fe2ffd700 1 -- 172.21.15.29:0/4283070155 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc00f670 con 0x7f2fe4066520 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fe2ffd700 1 -- 172.21.15.29:0/4283070155 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc00f9f0 con 0x7f2fe4066520 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fea998700 1 -- 172.21.15.29:0/4283070155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fe4066520 msgr2=0x7f2fe40669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fea998700 1 --2- 172.21.15.29:0/4283070155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fe4066520 0x7f2fe40669a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f2fe4069040 tx=0x7f2fcc0090f0).stop 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fea998700 1 -- 172.21.15.29:0/4283070155 shutdown_connections 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fea998700 1 --2- 172.21.15.29:0/4283070155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe413d580 0x7f2fe413d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fea998700 1 --2- 172.21.15.29:0/4283070155 >> v2:172.21.15.163:3300/0 conn(0x7f2fe4157de0 0x7f2fe4158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fea998700 1 --2- 172.21.15.29:0/4283070155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fe4066520 0x7f2fe40669a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.295+0000 7f2fea998700 1 -- 172.21.15.29:0/4283070155 >> 172.21.15.29:0/4283070155 conn(0x7f2fe40c8bc0 msgr2=0x7f2fe40c8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.296+0000 7f2fea998700 1 -- 172.21.15.29:0/4283070155 shutdown_connections 2024-01-24T13:58:57.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.296+0000 7f2fea998700 1 -- 172.21.15.29:0/4283070155 wait complete. 2024-01-24T13:58:57.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.297+0000 7f2fea998700 1 Processor -- start 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.297+0000 7f2fea998700 1 -- start start 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fea998700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fe4066520 0x7f2fe4164ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fea998700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe413d580 0x7f2fe413be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fea998700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe4157de0 0x7f2fe40a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fea998700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe41651d0 con 0x7f2fe4066520 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fea998700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4164850 con 0x7f2fe413d580 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fea998700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4164220 con 0x7f2fe4157de0 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fe4066520 0x7f2fe4164ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fe4066520 0x7f2fe4164ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33700/0 (socket says 172.21.15.29:33700) 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe3fff700 1 -- 172.21.15.29:0/351046261 learned_addr learned my addr 172.21.15.29:0/351046261 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe37fe700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe413d580 0x7f2fe413be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe8f35700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe4157de0 0x7f2fe40a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe3fff700 1 -- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe4157de0 msgr2=0x7f2fe40a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe3fff700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe4157de0 0x7f2fe40a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe3fff700 1 -- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe413d580 msgr2=0x7f2fe413be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe3fff700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe413d580 0x7f2fe413be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe3fff700 1 -- 172.21.15.29:0/351046261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc009370 con 0x7f2fe4066520 2024-01-24T13:58:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe8f35700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe4157de0 0x7f2fe40a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe37fe700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe413d580 0x7f2fe413be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.298+0000 7f2fe3fff700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fe4066520 0x7f2fe4164ce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f2fd400e930 tx=0x7f2fd4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.299+0000 7f2fdbfff700 1 -- 172.21.15.29:0/351046261 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400eed0 con 0x7f2fe4066520 2024-01-24T13:58:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.299+0000 7f2fdbfff700 1 -- 172.21.15.29:0/351046261 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd4009510 con 0x7f2fe4066520 2024-01-24T13:58:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.299+0000 7f2fea998700 1 -- 172.21.15.29:0/351046261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe40a71e0 con 0x7f2fe4066520 2024-01-24T13:58:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.299+0000 7f2fdbfff700 1 -- 172.21.15.29:0/351046261 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400ca60 con 0x7f2fe4066520 2024-01-24T13:58:57.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.299+0000 7f2fea998700 1 -- 172.21.15.29:0/351046261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe41644e0 con 0x7f2fe4066520 2024-01-24T13:58:57.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.300+0000 7f2fea998700 1 -- 172.21.15.29:0/351046261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe4164b30 con 0x7f2fe4066520 2024-01-24T13:58:57.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.300+0000 7f2fdbfff700 1 -- 172.21.15.29:0/351046261 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2fd401b400 con 0x7f2fe4066520 2024-01-24T13:58:57.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.301+0000 7f2fdbfff700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2fc405f150 0x7f2fc4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.301+0000 7f2fdbfff700 1 -- 172.21.15.29:0/351046261 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f2fd407a9c0 con 0x7f2fe4066520 2024-01-24T13:58:57.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.301+0000 7f2fe37fe700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2fc405f150 0x7f2fc4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.301+0000 7f2fe37fe700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2fc405f150 0x7f2fc4061610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f2fcc009130 tx=0x7f2fcc017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:57.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.305+0000 7f2fdbfff700 1 -- 172.21.15.29:0/351046261 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2fd404b920 con 0x7f2fe4066520 2024-01-24T13:58:57.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.484+0000 7f2fea998700 1 -- 172.21.15.29:0/351046261 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2fe40475e0 con 0x7f2fe4066520 2024-01-24T13:58:57.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.485+0000 7f2fdbfff700 1 -- 172.21.15.29:0/351046261 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2fd404ef40 con 0x7f2fe4066520 2024-01-24T13:58:57.484 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-01-24T13:58:57.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.486+0000 7f2fea998700 1 -- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2fc405f150 msgr2=0x7f2fc4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.486+0000 7f2fea998700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2fc405f150 0x7f2fc4061610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f2fcc009130 tx=0x7f2fcc017000).stop 2024-01-24T13:58:57.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.487+0000 7f2fea998700 1 -- 172.21.15.29:0/351046261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fe4066520 msgr2=0x7f2fe4164ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.487+0000 7f2fea998700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fe4066520 0x7f2fe4164ce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f2fd400e930 tx=0x7f2fd4009060).stop 2024-01-24T13:58:57.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.487+0000 7f2fea998700 1 -- 172.21.15.29:0/351046261 shutdown_connections 2024-01-24T13:58:57.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.487+0000 7f2fea998700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe4157de0 0x7f2fe40a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.487+0000 7f2fea998700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2fc405f150 0x7f2fc4061610 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.487+0000 7f2fea998700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fe4066520 0x7f2fe4164ce0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.487+0000 7f2fea998700 1 --2- 172.21.15.29:0/351046261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe413d580 0x7f2fe413be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.487+0000 7f2fea998700 1 -- 172.21.15.29:0/351046261 >> 172.21.15.29:0/351046261 conn(0x7f2fe40c8bc0 msgr2=0x7f2fe40c7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:57.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.487+0000 7f2fea998700 1 -- 172.21.15.29:0/351046261 shutdown_connections 2024-01-24T13:58:57.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.487+0000 7f2fea998700 1 -- 172.21.15.29:0/351046261 wait complete. 2024-01-24T13:58:57.499 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:57.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.630+0000 7fb6213b4700 1 Processor -- start 2024-01-24T13:58:57.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.630+0000 7fb6213b4700 1 -- start start 2024-01-24T13:58:57.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb6213b4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb61c1540a0 0x7fb61c0c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb6213b4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb61c1550f0 0x7fb61c0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb6213b4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb61c062ca0 0x7fb61c157c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb6213b4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61c0a74f0 con 0x7fb61c1550f0 2024-01-24T13:58:57.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb6213b4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61c065360 con 0x7fb61c1540a0 2024-01-24T13:58:57.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb6213b4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c0a6ce0 con 0x7fb61c062ca0 2024-01-24T13:58:57.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb61a7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb61c1550f0 0x7fb61c0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb61a7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb61c1550f0 0x7fb61c0627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33716/0 (socket says 172.21.15.29:33716) 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb61a7fc700 1 -- 172.21.15.29:0/1035911040 learned_addr learned my addr 172.21.15.29:0/1035911040 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb61affd700 1 --2- 172.21.15.29:0/1035911040 >> v2:172.21.15.163:3300/0 conn(0x7fb61c1540a0 0x7fb61c0c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb61b7fe700 1 --2- 172.21.15.29:0/1035911040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb61c062ca0 0x7fb61c157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb61a7fc700 1 -- 172.21.15.29:0/1035911040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb61c062ca0 msgr2=0x7fb61c157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb61a7fc700 1 --2- 172.21.15.29:0/1035911040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb61c062ca0 0x7fb61c157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb61a7fc700 1 -- 172.21.15.29:0/1035911040 >> v2:172.21.15.163:3300/0 conn(0x7fb61c1540a0 msgr2=0x7fb61c0c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb61a7fc700 1 --2- 172.21.15.29:0/1035911040 >> v2:172.21.15.163:3300/0 conn(0x7fb61c1540a0 0x7fb61c0c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.631+0000 7fb61a7fc700 1 -- 172.21.15.29:0/1035911040 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c0636f0 con 0x7fb61c1550f0 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb61b7fe700 1 --2- 172.21.15.29:0/1035911040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb61c062ca0 0x7fb61c157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb61affd700 1 --2- 172.21.15.29:0/1035911040 >> v2:172.21.15.163:3300/0 conn(0x7fb61c1540a0 0x7fb61c0c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb61a7fc700 1 --2- 172.21.15.29:0/1035911040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c1550f0 0x7fb61c0627b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fb61c06af90 tx=0x7fb60c0090f0).ready entity=mon.0 client_cookie=893b1fa3ec4a69b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:57.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb619ffb700 1 -- 172.21.15.29:0/1035911040 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c00b600 con 0x7fb61c1550f0 2024-01-24T13:58:57.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb619ffb700 1 -- 172.21.15.29:0/1035911040 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c00b760 con 0x7fb61c1550f0 2024-01-24T13:58:57.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb619ffb700 1 -- 172.21.15.29:0/1035911040 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c00bae0 con 0x7fb61c1550f0 2024-01-24T13:58:57.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb6213b4700 1 -- 172.21.15.29:0/1035911040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c1550f0 msgr2=0x7fb61c0627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb6213b4700 1 --2- 172.21.15.29:0/1035911040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c1550f0 0x7fb61c0627b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fb61c06af90 tx=0x7fb60c0090f0).stop 2024-01-24T13:58:57.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb6213b4700 1 -- 172.21.15.29:0/1035911040 shutdown_connections 2024-01-24T13:58:57.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb6213b4700 1 --2- 172.21.15.29:0/1035911040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb61c062ca0 0x7fb61c157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb6213b4700 1 --2- 172.21.15.29:0/1035911040 >> v2:172.21.15.163:3300/0 conn(0x7fb61c1540a0 0x7fb61c0c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb6213b4700 1 --2- 172.21.15.29:0/1035911040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c1550f0 0x7fb61c0627b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb6213b4700 1 -- 172.21.15.29:0/1035911040 >> 172.21.15.29:0/1035911040 conn(0x7fb61c0c85c0 msgr2=0x7fb61c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:57.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb6213b4700 1 -- 172.21.15.29:0/1035911040 shutdown_connections 2024-01-24T13:58:57.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.632+0000 7fb6213b4700 1 -- 172.21.15.29:0/1035911040 wait complete. 2024-01-24T13:58:57.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.634+0000 7fb6213b4700 1 Processor -- start 2024-01-24T13:58:57.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.634+0000 7fb6213b4700 1 -- start start 2024-01-24T13:58:57.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb6213b4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb61c062ca0 0x7fb61c1510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb6213b4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c1540a0 0x7fb61c13ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb6213b4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb61c1550f0 0x7fb61c0a8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb6213b4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c0a9b10 con 0x7fb61c1540a0 2024-01-24T13:58:57.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb6213b4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c0a8f00 con 0x7fb61c062ca0 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb6213b4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c0a91e0 con 0x7fb61c1550f0 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb61a7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c1540a0 0x7fb61c13ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb61a7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c1540a0 0x7fb61c13ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33730/0 (socket says 172.21.15.29:33730) 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb61a7fc700 1 -- 172.21.15.29:0/1360176014 learned_addr learned my addr 172.21.15.29:0/1360176014 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb61affd700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb61c062ca0 0x7fb61c1510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb61b7fe700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb61c1550f0 0x7fb61c0a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb61a7fc700 1 -- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb61c1550f0 msgr2=0x7fb61c0a8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb61a7fc700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb61c1550f0 0x7fb61c0a8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb61a7fc700 1 -- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb61c062ca0 msgr2=0x7fb61c1510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb61a7fc700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb61c062ca0 0x7fb61c1510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb61a7fc700 1 -- 172.21.15.29:0/1360176014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c009370 con 0x7fb61c1540a0 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.635+0000 7fb61a7fc700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c1540a0 0x7fb61c13ecb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fb61c069030 tx=0x7fb60c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.636+0000 7fb602ffd700 1 -- 172.21.15.29:0/1360176014 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c00bea0 con 0x7fb61c1540a0 2024-01-24T13:58:57.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.636+0000 7fb602ffd700 1 -- 172.21.15.29:0/1360176014 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c00b8d0 con 0x7fb61c1540a0 2024-01-24T13:58:57.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.636+0000 7fb6213b4700 1 -- 172.21.15.29:0/1360176014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c0a94c0 con 0x7fb61c1540a0 2024-01-24T13:58:57.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.636+0000 7fb602ffd700 1 -- 172.21.15.29:0/1360176014 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c01b660 con 0x7fb61c1540a0 2024-01-24T13:58:57.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.636+0000 7fb6213b4700 1 -- 172.21.15.29:0/1360176014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c13f1a0 con 0x7fb61c1540a0 2024-01-24T13:58:57.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.637+0000 7fb6213b4700 1 -- 172.21.15.29:0/1360176014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c0a9c80 con 0x7fb61c1540a0 2024-01-24T13:58:57.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.637+0000 7fb602ffd700 1 -- 172.21.15.29:0/1360176014 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb60c01b9d0 con 0x7fb61c1540a0 2024-01-24T13:58:57.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.637+0000 7fb602ffd700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb60805f150 0x7fb608061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.637+0000 7fb602ffd700 1 -- 172.21.15.29:0/1360176014 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fb60c07b8d0 con 0x7fb61c1540a0 2024-01-24T13:58:57.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.638+0000 7fb61affd700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb60805f150 0x7fb608061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.638+0000 7fb61affd700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb60805f150 0x7fb608061610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fb610006fd0 tx=0x7fb610008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:57.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.642+0000 7fb602ffd700 1 -- 172.21.15.29:0/1360176014 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb60c04c890 con 0x7fb61c1540a0 2024-01-24T13:58:57.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.825+0000 7fb6213b4700 1 -- 172.21.15.29:0/1360176014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb61c06a5f0 con 0x7fb61c1540a0 2024-01-24T13:58:57.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.826+0000 7fb602ffd700 1 -- 172.21.15.29:0/1360176014 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7fb60c01f020 con 0x7fb61c1540a0 2024-01-24T13:58:57.825 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:57.825 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:57.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.827+0000 7fb6213b4700 1 -- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb60805f150 msgr2=0x7fb608061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.827+0000 7fb6213b4700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb60805f150 0x7fb608061610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fb610006fd0 tx=0x7fb610008040).stop 2024-01-24T13:58:57.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.827+0000 7fb6213b4700 1 -- 172.21.15.29:0/1360176014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c1540a0 msgr2=0x7fb61c13ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.827+0000 7fb6213b4700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c1540a0 0x7fb61c13ecb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fb61c069030 tx=0x7fb60c017000).stop 2024-01-24T13:58:57.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.827+0000 7fb6213b4700 1 -- 172.21.15.29:0/1360176014 shutdown_connections 2024-01-24T13:58:57.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.828+0000 7fb6213b4700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb61c1550f0 0x7fb61c0a8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.828+0000 7fb6213b4700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb61c062ca0 0x7fb61c1510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.828+0000 7fb6213b4700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb60805f150 0x7fb608061610 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.828+0000 7fb6213b4700 1 --2- 172.21.15.29:0/1360176014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb61c1540a0 0x7fb61c13ecb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.828+0000 7fb6213b4700 1 -- 172.21.15.29:0/1360176014 >> 172.21.15.29:0/1360176014 conn(0x7fb61c0c85c0 msgr2=0x7fb61c0c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:57.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.828+0000 7fb6213b4700 1 -- 172.21.15.29:0/1360176014 shutdown_connections 2024-01-24T13:58:57.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.828+0000 7fb6213b4700 1 -- 172.21.15.29:0/1360176014 wait complete. 2024-01-24T13:58:57.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T13:58:16.949817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T13:58:18.174460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-24T13:58:21.232882+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T13:58:57.841 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T13:58:57.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.976+0000 7f30ac3bb700 1 Processor -- start 2024-01-24T13:58:57.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.976+0000 7f30ac3bb700 1 -- start start 2024-01-24T13:58:57.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.977+0000 7f30ac3bb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f30a413a5e0 0x7f30a40639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.977+0000 7f30ac3bb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30a4063eb0 0x7f30a40664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.977+0000 7f30ac3bb700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a40679f0 con 0x7f30a40669b0 2024-01-24T13:58:57.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.977+0000 7f30ac3bb700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a4067cd0 con 0x7f30a4063eb0 2024-01-24T13:58:57.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.977+0000 7f30ac3bb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a4067fb0 con 0x7f30a413a5e0 2024-01-24T13:58:57.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.977+0000 7f30a9956700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30a4063eb0 0x7f30a40664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.977+0000 7f30a9956700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30a4063eb0 0x7f30a40664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33732/0 (socket says 172.21.15.29:33732) 2024-01-24T13:58:57.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.977+0000 7f30a9956700 1 -- 172.21.15.29:0/2144543327 learned_addr learned my addr 172.21.15.29:0/2144543327 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:57.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.977+0000 7f30aa958700 1 --1- 172.21.15.29:0/2144543327 >> v1:172.21.15.29:6789/0 conn(0x7f30a40669b0 0x7f30a4157bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46306/0 (socket says 172.21.15.29:46306) 2024-01-24T13:58:57.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.977+0000 7f30aa157700 1 --2- 172.21.15.29:0/2144543327 >> v2:172.21.15.163:3300/0 conn(0x7f30a413a5e0 0x7f30a40639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.978+0000 7f30a9956700 1 -- 172.21.15.29:0/2144543327 >> v2:172.21.15.163:3300/0 conn(0x7f30a413a5e0 msgr2=0x7f30a40639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:57.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.978+0000 7f30a9956700 1 --2- 172.21.15.29:0/2144543327 >> v2:172.21.15.163:3300/0 conn(0x7f30a413a5e0 0x7f30a40639c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.978+0000 7f30a9956700 1 -- 172.21.15.29:0/2144543327 >> v1:172.21.15.29:6789/0 conn(0x7f30a40669b0 legacy=0x7f30a4157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.978+0000 7f30a9155700 1 -- 172.21.15.29:0/2144543327 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680101567 0 0) 0x7f30a40679f0 con 0x7f30a40669b0 2024-01-24T13:58:57.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.978+0000 7f30a9956700 1 -- 172.21.15.29:0/2144543327 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a4068290 con 0x7f30a4063eb0 2024-01-24T13:58:57.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.978+0000 7f30a9956700 1 --2- 172.21.15.29:0/2144543327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4063eb0 0x7f30a40664c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f30a4069ba0 tx=0x7f30940090f0).ready entity=mon.0 client_cookie=7e02caefdaac151 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:57.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.978+0000 7f30a9155700 1 -- 172.21.15.29:0/2144543327 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309400b520 con 0x7f30a4063eb0 2024-01-24T13:58:57.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.978+0000 7f30a9155700 1 -- 172.21.15.29:0/2144543327 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309400b680 con 0x7f30a4063eb0 2024-01-24T13:58:57.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.978+0000 7f30a9155700 1 -- 172.21.15.29:0/2144543327 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309400ba00 con 0x7f30a4063eb0 2024-01-24T13:58:57.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.978+0000 7f30ac3bb700 1 -- 172.21.15.29:0/2144543327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4063eb0 msgr2=0x7f30a40664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.978+0000 7f30ac3bb700 1 --2- 172.21.15.29:0/2144543327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4063eb0 0x7f30a40664c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f30a4069ba0 tx=0x7f30940090f0).stop 2024-01-24T13:58:57.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.979+0000 7f30ac3bb700 1 -- 172.21.15.29:0/2144543327 shutdown_connections 2024-01-24T13:58:57.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.979+0000 7f30ac3bb700 1 --2- 172.21.15.29:0/2144543327 >> v2:172.21.15.163:3300/0 conn(0x7f30a413a5e0 0x7f30a40639c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.979+0000 7f30ac3bb700 1 --2- 172.21.15.29:0/2144543327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a4063eb0 0x7f30a40664c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.979+0000 7f30ac3bb700 1 -- 172.21.15.29:0/2144543327 >> 172.21.15.29:0/2144543327 conn(0x7f30a40c8bd0 msgr2=0x7f30a40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:57.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.979+0000 7f30ac3bb700 1 -- 172.21.15.29:0/2144543327 shutdown_connections 2024-01-24T13:58:57.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.979+0000 7f30ac3bb700 1 -- 172.21.15.29:0/2144543327 wait complete. 2024-01-24T13:58:57.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.980+0000 7f30ac3bb700 1 Processor -- start 2024-01-24T13:58:57.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.980+0000 7f30ac3bb700 1 -- start start 2024-01-24T13:58:57.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30ac3bb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30a4063eb0 0x7f30a40acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30ac3bb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a40669b0 0x7f30a40a8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30ac3bb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30a413a5e0 0x7f30a40a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30ac3bb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40a61a0 con 0x7f30a40669b0 2024-01-24T13:58:57.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30ac3bb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40a6480 con 0x7f30a4063eb0 2024-01-24T13:58:57.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30ac3bb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40aa070 con 0x7f30a413a5e0 2024-01-24T13:58:57.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30a9956700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a40669b0 0x7f30a40a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30a9956700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a40669b0 0x7f30a40a8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33746/0 (socket says 172.21.15.29:33746) 2024-01-24T13:58:57.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30a9956700 1 -- 172.21.15.29:0/483105809 learned_addr learned my addr 172.21.15.29:0/483105809 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:57.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30aa157700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30a4063eb0 0x7f30a40acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30aa958700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30a413a5e0 0x7f30a40a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30a9956700 1 -- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30a413a5e0 msgr2=0x7f30a40a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30a9956700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30a413a5e0 0x7f30a40a5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30a9956700 1 -- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30a4063eb0 msgr2=0x7f30a40acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30a9956700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30a4063eb0 0x7f30a40acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30a9956700 1 -- 172.21.15.29:0/483105809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3094009370 con 0x7f30a40669b0 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30aa157700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30a4063eb0 0x7f30a40acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f30a9956700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a40669b0 0x7f30a40a8510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f3094009130 tx=0x7f3094017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f309a7fc700 1 -- 172.21.15.29:0/483105809 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309400be90 con 0x7f30a40669b0 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.981+0000 7f309a7fc700 1 -- 172.21.15.29:0/483105809 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309400b7f0 con 0x7f30a40669b0 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.982+0000 7f30ac3bb700 1 -- 172.21.15.29:0/483105809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a40aa350 con 0x7f30a40669b0 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.982+0000 7f309a7fc700 1 -- 172.21.15.29:0/483105809 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309401c670 con 0x7f30a40669b0 2024-01-24T13:58:57.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.982+0000 7f30ac3bb700 1 -- 172.21.15.29:0/483105809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a40a5e40 con 0x7f30a40669b0 2024-01-24T13:58:57.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.983+0000 7f30ac3bb700 1 -- 172.21.15.29:0/483105809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a40475e0 con 0x7f30a40669b0 2024-01-24T13:58:57.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.983+0000 7f309a7fc700 1 -- 172.21.15.29:0/483105809 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f309401c9e0 con 0x7f30a40669b0 2024-01-24T13:58:57.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.983+0000 7f309a7fc700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f308805f150 0x7f3088061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:57.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.983+0000 7f309a7fc700 1 -- 172.21.15.29:0/483105809 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f309407c1c0 con 0x7f30a40669b0 2024-01-24T13:58:57.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.984+0000 7f30aa157700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f308805f150 0x7f3088061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:57.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.984+0000 7f30aa157700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f308805f150 0x7f3088061610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f30a0006d80 tx=0x7f30a0008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:57.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:57.988+0000 7f309a7fc700 1 -- 172.21.15.29:0/483105809 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f309404d0c0 con 0x7f30a40669b0 2024-01-24T13:58:58.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.167+0000 7f30ac3bb700 1 -- 172.21.15.29:0/483105809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f30a4068570 con 0x7f30a40669b0 2024-01-24T13:58:58.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.168+0000 7f309a7fc700 1 -- 172.21.15.29:0/483105809 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f309402a070 con 0x7f30a40669b0 2024-01-24T13:58:58.167 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T13:58:58.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.170+0000 7f30ac3bb700 1 -- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f308805f150 msgr2=0x7f3088061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.170+0000 7f30ac3bb700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f308805f150 0x7f3088061610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f30a0006d80 tx=0x7f30a0008040).stop 2024-01-24T13:58:58.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.170+0000 7f30ac3bb700 1 -- 172.21.15.29:0/483105809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a40669b0 msgr2=0x7f30a40a8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.170+0000 7f30ac3bb700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a40669b0 0x7f30a40a8510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f3094009130 tx=0x7f3094017000).stop 2024-01-24T13:58:58.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.170+0000 7f30ac3bb700 1 -- 172.21.15.29:0/483105809 shutdown_connections 2024-01-24T13:58:58.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.170+0000 7f30ac3bb700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30a413a5e0 0x7f30a40a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.170+0000 7f30ac3bb700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30a4063eb0 0x7f30a40acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.170+0000 7f30ac3bb700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f308805f150 0x7f3088061610 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.170+0000 7f30ac3bb700 1 --2- 172.21.15.29:0/483105809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30a40669b0 0x7f30a40a8510 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.170+0000 7f30ac3bb700 1 -- 172.21.15.29:0/483105809 >> 172.21.15.29:0/483105809 conn(0x7f30a40c8bd0 msgr2=0x7f30a413ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:58.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.170+0000 7f30ac3bb700 1 -- 172.21.15.29:0/483105809 shutdown_connections 2024-01-24T13:58:58.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.170+0000 7f30ac3bb700 1 -- 172.21.15.29:0/483105809 wait complete. 2024-01-24T13:58:58.182 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T13:58:58.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.316+0000 7fdf64c14700 1 Processor -- start 2024-01-24T13:58:58.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.316+0000 7fdf64c14700 1 -- start start 2024-01-24T13:58:58.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.317+0000 7fdf64c14700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf60069680 0x7fdf60069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:58.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.317+0000 7fdf64c14700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdf601589e0 0x7fdf6015add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.317+0000 7fdf64c14700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf600c3e70 con 0x7fdf600692a0 2024-01-24T13:58:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.317+0000 7fdf64c14700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf600c4300 con 0x7fdf601589e0 2024-01-24T13:58:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.317+0000 7fdf64c14700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c3260 con 0x7fdf60069680 2024-01-24T13:58:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.317+0000 7fdf5ed9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdf601589e0 0x7fdf6015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.317+0000 7fdf5dd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf60069680 0x7fdf60069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.317+0000 7fdf5e59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fdf600692a0 0x7fdf60065e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:46540/0 (socket says 172.21.15.29:46540) 2024-01-24T13:58:58.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.317+0000 7fdf5e59c700 1 -- 172.21.15.29:0/668869294 learned_addr learned my addr 172.21.15.29:0/668869294 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.318+0000 7fdf5ed9d700 1 -- 172.21.15.29:0/668869294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf60069680 msgr2=0x7fdf60069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.318+0000 7fdf5ed9d700 1 --2- 172.21.15.29:0/668869294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf60069680 0x7fdf60069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.318+0000 7fdf5ed9d700 1 -- 172.21.15.29:0/668869294 >> v1:172.21.15.163:6789/0 conn(0x7fdf600692a0 legacy=0x7fdf60065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.318+0000 7fdf5ed9d700 1 -- 172.21.15.29:0/668869294 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf600c3540 con 0x7fdf601589e0 2024-01-24T13:58:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.318+0000 7fdf5d59a700 1 -- 172.21.15.29:0/668869294 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712324402 0 0) 0x7fdf600c3e70 con 0x7fdf600692a0 2024-01-24T13:58:58.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.318+0000 7fdf5dd9b700 1 --2- 172.21.15.29:0/668869294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf60069680 0x7fdf60069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.318+0000 7fdf5ed9d700 1 --2- 172.21.15.29:0/668869294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf601589e0 0x7fdf6015add0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fdf58009e60 tx=0x7fdf58009780).ready entity=mon.1 client_cookie=a9f15d67a29600d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.318+0000 7fdf5d59a700 1 -- 172.21.15.29:0/668869294 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5800e070 con 0x7fdf601589e0 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.318+0000 7fdf5d59a700 1 -- 172.21.15.29:0/668869294 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5800bb60 con 0x7fdf601589e0 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.318+0000 7fdf5d59a700 1 -- 172.21.15.29:0/668869294 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58012410 con 0x7fdf601589e0 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.319+0000 7fdf64c14700 1 -- 172.21.15.29:0/668869294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf601589e0 msgr2=0x7fdf6015add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.319+0000 7fdf64c14700 1 --2- 172.21.15.29:0/668869294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf601589e0 0x7fdf6015add0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fdf58009e60 tx=0x7fdf58009780).stop 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.319+0000 7fdf64c14700 1 -- 172.21.15.29:0/668869294 shutdown_connections 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.319+0000 7fdf64c14700 1 --2- 172.21.15.29:0/668869294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf601589e0 0x7fdf6015add0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.319+0000 7fdf64c14700 1 --2- 172.21.15.29:0/668869294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf60069680 0x7fdf60069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.319+0000 7fdf64c14700 1 -- 172.21.15.29:0/668869294 >> 172.21.15.29:0/668869294 conn(0x7fdf60064390 msgr2=0x7fdf600c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.319+0000 7fdf64c14700 1 -- 172.21.15.29:0/668869294 shutdown_connections 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.319+0000 7fdf64c14700 1 -- 172.21.15.29:0/668869294 wait complete. 2024-01-24T13:58:58.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.320+0000 7fdf64c14700 1 Processor -- start 2024-01-24T13:58:58.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.320+0000 7fdf64c14700 1 -- start start 2024-01-24T13:58:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf64c14700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf600692a0 0x7fdf6014b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf64c14700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf60069680 0x7fdf60065350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:58.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf64c14700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf601589e0 0x7fdf60065840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf64c14700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf60065d30 con 0x7fdf60069680 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf64c14700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c1be0 con 0x7fdf600692a0 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf64c14700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c1620 con 0x7fdf601589e0 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf5dd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf60069680 0x7fdf60065350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf5dd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf60069680 0x7fdf60065350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33752/0 (socket says 172.21.15.29:33752) 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf5dd9b700 1 -- 172.21.15.29:0/2383700730 learned_addr learned my addr 172.21.15.29:0/2383700730 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf5e59c700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf600692a0 0x7fdf6014b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf5ed9d700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf601589e0 0x7fdf60065840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf5dd9b700 1 -- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf601589e0 msgr2=0x7fdf60065840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf5dd9b700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf601589e0 0x7fdf60065840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf5dd9b700 1 -- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf600692a0 msgr2=0x7fdf6014b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf5dd9b700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf600692a0 0x7fdf6014b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.321+0000 7fdf5dd9b700 1 -- 172.21.15.29:0/2383700730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540092a0 con 0x7fdf60069680 2024-01-24T13:58:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.322+0000 7fdf5dd9b700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf60069680 0x7fdf60065350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fdf5400ea20 tx=0x7fdf54009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.322+0000 7fdf4e7fc700 1 -- 172.21.15.29:0/2383700730 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400ee60 con 0x7fdf60069680 2024-01-24T13:58:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.322+0000 7fdf4e7fc700 1 -- 172.21.15.29:0/2383700730 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5400c850 con 0x7fdf60069680 2024-01-24T13:58:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.322+0000 7fdf64c14700 1 -- 172.21.15.29:0/2383700730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf58009a20 con 0x7fdf60069680 2024-01-24T13:58:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.322+0000 7fdf4e7fc700 1 -- 172.21.15.29:0/2383700730 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400cbd0 con 0x7fdf60069680 2024-01-24T13:58:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.322+0000 7fdf64c14700 1 -- 172.21.15.29:0/2383700730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf600bf640 con 0x7fdf60069680 2024-01-24T13:58:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.323+0000 7fdf64c14700 1 -- 172.21.15.29:0/2383700730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf600bfc90 con 0x7fdf60069680 2024-01-24T13:58:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.323+0000 7fdf4e7fc700 1 -- 172.21.15.29:0/2383700730 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fdf5401b400 con 0x7fdf60069680 2024-01-24T13:58:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.324+0000 7fdf4e7fc700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdf4005f150 0x7fdf40061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.324+0000 7fdf4e7fc700 1 -- 172.21.15.29:0/2383700730 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fdf5407a9b0 con 0x7fdf60069680 2024-01-24T13:58:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.324+0000 7fdf5e59c700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdf4005f150 0x7fdf40061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.324+0000 7fdf5e59c700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdf4005f150 0x7fdf40061610 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fdf500011e0 tx=0x7fdf50007bb0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:58.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.328+0000 7fdf4e7fc700 1 -- 172.21.15.29:0/2383700730 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdf5404b9c0 con 0x7fdf60069680 2024-01-24T13:58:58.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.512+0000 7fdf64c14700 1 -- 172.21.15.29:0/2383700730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdf600475e0 con 0x7fdf60069680 2024-01-24T13:58:58.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.512+0000 7fdf4e7fc700 1 -- 172.21.15.29:0/2383700730 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdf5404efe0 con 0x7fdf60069680 2024-01-24T13:58:58.512 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T13:58:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.514+0000 7fdf64c14700 1 -- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdf4005f150 msgr2=0x7fdf40061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.514+0000 7fdf64c14700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdf4005f150 0x7fdf40061610 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fdf500011e0 tx=0x7fdf50007bb0).stop 2024-01-24T13:58:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.514+0000 7fdf64c14700 1 -- 172.21.15.29:0/2383700730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf60069680 msgr2=0x7fdf60065350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.514+0000 7fdf64c14700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf60069680 0x7fdf60065350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fdf5400ea20 tx=0x7fdf54009060).stop 2024-01-24T13:58:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.514+0000 7fdf64c14700 1 -- 172.21.15.29:0/2383700730 shutdown_connections 2024-01-24T13:58:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.514+0000 7fdf64c14700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf601589e0 0x7fdf60065840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.514+0000 7fdf64c14700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf600692a0 0x7fdf6014b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.514+0000 7fdf64c14700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdf4005f150 0x7fdf40061610 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.514+0000 7fdf64c14700 1 --2- 172.21.15.29:0/2383700730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf60069680 0x7fdf60065350 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.514+0000 7fdf64c14700 1 -- 172.21.15.29:0/2383700730 >> 172.21.15.29:0/2383700730 conn(0x7fdf60064390 msgr2=0x7fdf6014be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.514+0000 7fdf64c14700 1 -- 172.21.15.29:0/2383700730 shutdown_connections 2024-01-24T13:58:58.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.514+0000 7fdf64c14700 1 -- 172.21.15.29:0/2383700730 wait complete. 2024-01-24T13:58:58.527 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T13:58:58.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.660+0000 7fe3b7848700 1 Processor -- start 2024-01-24T13:58:58.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.660+0000 7fe3b7848700 1 -- start start 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.661+0000 7fe3b7848700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe3b01544e0 0x7fe3b015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.661+0000 7fe3b7848700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b006a110 con 0x7fe3b0069820 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.661+0000 7fe3b7848700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b00c3d40 con 0x7fe3b0069440 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.661+0000 7fe3b7848700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b0066540 con 0x7fe3b01544e0 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.662+0000 7fe3b4de3700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe3b0069820 0x7fe3b0153dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46314/0 (socket says 172.21.15.29:46314) 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.662+0000 7fe3b4de3700 1 -- 172.21.15.29:0/2418241732 learned_addr learned my addr 172.21.15.29:0/2418241732 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.662+0000 7fe3b5de5700 1 --2- 172.21.15.29:0/2418241732 >> v2:172.21.15.163:3300/0 conn(0x7fe3b01544e0 0x7fe3b015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.662+0000 7fe3a3fff700 1 -- 172.21.15.29:0/2418241732 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505627751 0 0) 0x7fe3b006a110 con 0x7fe3b0069820 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.662+0000 7fe3a3fff700 1 -- 172.21.15.29:0/2418241732 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39c0033f0 con 0x7fe3b0069820 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.662+0000 7fe3b5de5700 1 -- 172.21.15.29:0/2418241732 >> v1:172.21.15.163:6789/0 conn(0x7fe3b0069440 legacy=0x7fe3b0065e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.662+0000 7fe3b5de5700 1 -- 172.21.15.29:0/2418241732 >> v1:172.21.15.29:6789/0 conn(0x7fe3b0069820 legacy=0x7fe3b0153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.662+0000 7fe3b5de5700 1 -- 172.21.15.29:0/2418241732 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b00c3130 con 0x7fe3b01544e0 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.662+0000 7fe3b5de5700 1 --2- 172.21.15.29:0/2418241732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3b01544e0 0x7fe3b015bd70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fe3a8009e60 tx=0x7fe3a8009780).ready entity=mon.1 client_cookie=beeb2c9107cc2796 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:58.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.663+0000 7fe3a3fff700 1 -- 172.21.15.29:0/2418241732 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a800e070 con 0x7fe3b01544e0 2024-01-24T13:58:58.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.663+0000 7fe3a3fff700 1 -- 172.21.15.29:0/2418241732 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a800bb60 con 0x7fe3b01544e0 2024-01-24T13:58:58.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.663+0000 7fe3a3fff700 1 -- 172.21.15.29:0/2418241732 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8012410 con 0x7fe3b01544e0 2024-01-24T13:58:58.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.663+0000 7fe3b7848700 1 -- 172.21.15.29:0/2418241732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3b01544e0 msgr2=0x7fe3b015bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.663+0000 7fe3b7848700 1 --2- 172.21.15.29:0/2418241732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3b01544e0 0x7fe3b015bd70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fe3a8009e60 tx=0x7fe3a8009780).stop 2024-01-24T13:58:58.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.663+0000 7fe3b7848700 1 -- 172.21.15.29:0/2418241732 shutdown_connections 2024-01-24T13:58:58.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.663+0000 7fe3b7848700 1 --2- 172.21.15.29:0/2418241732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3b01544e0 0x7fe3b015bd70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.663+0000 7fe3b7848700 1 -- 172.21.15.29:0/2418241732 >> 172.21.15.29:0/2418241732 conn(0x7fe3b0064390 msgr2=0x7fe3b00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:58.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.663+0000 7fe3b7848700 1 -- 172.21.15.29:0/2418241732 shutdown_connections 2024-01-24T13:58:58.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.663+0000 7fe3b7848700 1 -- 172.21.15.29:0/2418241732 wait complete. 2024-01-24T13:58:58.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.665+0000 7fe3b7848700 1 Processor -- start 2024-01-24T13:58:58.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.665+0000 7fe3b7848700 1 -- start start 2024-01-24T13:58:58.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.666+0000 7fe3b7848700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3b0069440 0x7fe3b00a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:58.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.666+0000 7fe3b7848700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3b0069820 0x7fe3b0144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:58.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.666+0000 7fe3b7848700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3b01544e0 0x7fe3b00c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:58.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.666+0000 7fe3b7848700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b0150b60 con 0x7fe3b0069440 2024-01-24T13:58:58.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.666+0000 7fe3b7848700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00a13d0 con 0x7fe3b01544e0 2024-01-24T13:58:58.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.666+0000 7fe3b7848700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b014ff50 con 0x7fe3b0069820 2024-01-24T13:58:58.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.666+0000 7fe3b55e4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3b0069440 0x7fe3b00a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:58.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.666+0000 7fe3b55e4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3b0069440 0x7fe3b00a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33764/0 (socket says 172.21.15.29:33764) 2024-01-24T13:58:58.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.666+0000 7fe3b55e4700 1 -- 172.21.15.29:0/2063625998 learned_addr learned my addr 172.21.15.29:0/2063625998 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:58.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3b5de5700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3b01544e0 0x7fe3b00c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:58.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3b4de3700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3b0069820 0x7fe3b0144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3b55e4700 1 -- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3b0069820 msgr2=0x7fe3b0144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3b55e4700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3b0069820 0x7fe3b0144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3b55e4700 1 -- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3b01544e0 msgr2=0x7fe3b00c0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3b55e4700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3b01544e0 0x7fe3b00c0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3b55e4700 1 -- 172.21.15.29:0/2063625998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a8009a20 con 0x7fe3b0069440 2024-01-24T13:58:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3b55e4700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3b0069440 0x7fe3b00a0ee0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fe3a400c490 tx=0x7fe3a4007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3a17fa700 1 -- 172.21.15.29:0/2063625998 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a400f470 con 0x7fe3b0069440 2024-01-24T13:58:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3a17fa700 1 -- 172.21.15.29:0/2063625998 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a400f5d0 con 0x7fe3b0069440 2024-01-24T13:58:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3b7848700 1 -- 172.21.15.29:0/2063625998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b0150230 con 0x7fe3b0069440 2024-01-24T13:58:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3a17fa700 1 -- 172.21.15.29:0/2063625998 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a400f950 con 0x7fe3b0069440 2024-01-24T13:58:58.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.667+0000 7fe3b7848700 1 -- 172.21.15.29:0/2063625998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b01507f0 con 0x7fe3b0069440 2024-01-24T13:58:58.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.668+0000 7fe3b7848700 1 -- 172.21.15.29:0/2063625998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b00475e0 con 0x7fe3b0069440 2024-01-24T13:58:58.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.669+0000 7fe3a17fa700 1 -- 172.21.15.29:0/2063625998 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe3a400fd40 con 0x7fe3b0069440 2024-01-24T13:58:58.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.669+0000 7fe3a17fa700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe39405f150 0x7fe394061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:58.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.669+0000 7fe3a17fa700 1 -- 172.21.15.29:0/2063625998 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe3a4079a20 con 0x7fe3b0069440 2024-01-24T13:58:58.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.669+0000 7fe3b4de3700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe39405f150 0x7fe394061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:58.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.670+0000 7fe3b4de3700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe39405f150 0x7fe394061610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fe3b00c1c10 tx=0x7fe3ac009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:58.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.673+0000 7fe3a17fa700 1 -- 172.21.15.29:0/2063625998 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe3a4048b00 con 0x7fe3b0069440 2024-01-24T13:58:58.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.859+0000 7fe3b7848700 1 -- 172.21.15.29:0/2063625998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3b00c7fe0 con 0x7fe3b0069440 2024-01-24T13:58:58.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.860+0000 7fe3a17fa700 1 -- 172.21.15.29:0/2063625998 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3a404ab30 con 0x7fe3b0069440 2024-01-24T13:58:58.859 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:58:58.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.862+0000 7fe3b7848700 1 -- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe39405f150 msgr2=0x7fe394061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.862+0000 7fe3b7848700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe39405f150 0x7fe394061610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fe3b00c1c10 tx=0x7fe3ac009040).stop 2024-01-24T13:58:58.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.862+0000 7fe3b7848700 1 -- 172.21.15.29:0/2063625998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3b0069440 msgr2=0x7fe3b00a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:58.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.862+0000 7fe3b7848700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3b0069440 0x7fe3b00a0ee0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fe3a400c490 tx=0x7fe3a4007de0).stop 2024-01-24T13:58:58.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.862+0000 7fe3b7848700 1 -- 172.21.15.29:0/2063625998 shutdown_connections 2024-01-24T13:58:58.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.862+0000 7fe3b7848700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3b01544e0 0x7fe3b00c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.862+0000 7fe3b7848700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe39405f150 0x7fe394061610 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.862+0000 7fe3b7848700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3b0069440 0x7fe3b00a0ee0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.862+0000 7fe3b7848700 1 --2- 172.21.15.29:0/2063625998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3b0069820 0x7fe3b0144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:58.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.862+0000 7fe3b7848700 1 -- 172.21.15.29:0/2063625998 >> 172.21.15.29:0/2063625998 conn(0x7fe3b0064390 msgr2=0x7fe3b00c6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:58.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.862+0000 7fe3b7848700 1 -- 172.21.15.29:0/2063625998 shutdown_connections 2024-01-24T13:58:58.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:58.862+0000 7fe3b7848700 1 -- 172.21.15.29:0/2063625998 wait complete. 2024-01-24T13:58:58.875 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T13:58:59.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.008+0000 7f0dc668e700 1 Processor -- start 2024-01-24T13:58:59.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.008+0000 7f0dc668e700 1 -- start start 2024-01-24T13:58:59.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.009+0000 7f0dc668e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0dc0066950 0x7f0dc00be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.009+0000 7f0dc668e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc00640d0 con 0x7f0dc013a5e0 2024-01-24T13:58:59.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.009+0000 7f0dc668e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc00679f0 con 0x7f0dc00664c0 2024-01-24T13:58:59.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.009+0000 7f0dc668e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc0067cd0 con 0x7f0dc0066950 2024-01-24T13:58:59.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.009+0000 7f0dbffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0dc013a5e0 0x7f0dc00639c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46316/0 (socket says 172.21.15.29:46316) 2024-01-24T13:58:59.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbffff700 1 -- 172.21.15.29:0/3242983342 learned_addr learned my addr 172.21.15.29:0/3242983342 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:59.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dc4c2b700 1 --2- 172.21.15.29:0/3242983342 >> v2:172.21.15.163:3300/0 conn(0x7f0dc0066950 0x7f0dc00be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621760185 0 0) 0x7f0dc00640d0 con 0x7f0dc013a5e0 2024-01-24T13:58:59.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0da80033f0 con 0x7f0dc013a5e0 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201250015 0 0) 0x7f0dc00679f0 con 0x7f0dc00664c0 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dc00640d0 con 0x7f0dc00664c0 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 621241761 0 0) 0x7f0da80033f0 con 0x7f0dc013a5e0 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 >> v2:172.21.15.163:3300/0 conn(0x7f0dc0066950 msgr2=0x7f0dc00be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 --2- 172.21.15.29:0/3242983342 >> v2:172.21.15.163:3300/0 conn(0x7f0dc0066950 0x7f0dc00be240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 >> v1:172.21.15.163:6789/0 conn(0x7f0dc00664c0 legacy=0x7f0dc00bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc0067fb0 con 0x7f0dc013a5e0 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f0db0001470 con 0x7f0dc013a5e0 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 >> v1:172.21.15.29:6789/0 conn(0x7f0dc013a5e0 legacy=0x7f0dc00639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 --2- 172.21.15.29:0/3242983342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da8003a70 0x7f0da8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 --2- 172.21.15.29:0/3242983342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0da8004690 0x7f0da8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 --2- 172.21.15.29:0/3242983342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0da8005000 0x7f0da800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc0067cd0 con 0x7f0da8004690 2024-01-24T13:58:59.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da800be80 con 0x7f0da8003a70 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da800c040 con 0x7f0da8005000 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dc4c2b700 1 --2- 172.21.15.29:0/3242983342 >> v2:172.21.15.163:3300/0 conn(0x7f0dc0066950 0x7f0dc00be240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbf7fe700 1 --2- 172.21.15.29:0/3242983342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0da8004690 0x7f0da8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.010+0000 7f0dbffff700 1 --2- 172.21.15.29:0/3242983342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da8003a70 0x7f0da8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.011+0000 7f0dc4c2b700 1 --2- 172.21.15.29:0/3242983342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0da8005000 0x7f0da800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.011+0000 7f0dbf7fe700 1 -- 172.21.15.29:0/3242983342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0da8005000 msgr2=0x7f0da800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.011+0000 7f0dbf7fe700 1 --2- 172.21.15.29:0/3242983342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0da8005000 0x7f0da800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.011+0000 7f0dbf7fe700 1 -- 172.21.15.29:0/3242983342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da8003a70 msgr2=0x7f0da8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.011+0000 7f0dbf7fe700 1 --2- 172.21.15.29:0/3242983342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da8003a70 0x7f0da8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.011+0000 7f0dbf7fe700 1 -- 172.21.15.29:0/3242983342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da80033f0 con 0x7f0da8004690 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.011+0000 7f0dc4c2b700 1 --2- 172.21.15.29:0/3242983342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0da8005000 0x7f0da800b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.011+0000 7f0dc4c2b700 1 -- 172.21.15.29:0/3242983342 reap_dead start 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.011+0000 7f0dbf7fe700 1 --2- 172.21.15.29:0/3242983342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0da8004690 0x7f0da8004b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f0db4001050 tx=0x7f0db4007b90).ready entity=mon.0 client_cookie=a599415c4ac1afa5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.011+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4007ea0 con 0x7f0da8004690 2024-01-24T13:58:59.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.011+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db4012400 con 0x7f0da8004690 2024-01-24T13:58:59.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.011+0000 7f0dbeffd700 1 -- 172.21.15.29:0/3242983342 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db40127a0 con 0x7f0da8004690 2024-01-24T13:58:59.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.012+0000 7f0dc668e700 1 -- 172.21.15.29:0/3242983342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0da8004690 msgr2=0x7f0da8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.012+0000 7f0dc668e700 1 --2- 172.21.15.29:0/3242983342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0da8004690 0x7f0da8004b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f0db4001050 tx=0x7f0db4007b90).stop 2024-01-24T13:58:59.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.012+0000 7f0dc668e700 1 -- 172.21.15.29:0/3242983342 shutdown_connections 2024-01-24T13:58:59.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.012+0000 7f0dc668e700 1 --2- 172.21.15.29:0/3242983342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0da8004690 0x7f0da8004b10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.012+0000 7f0dc668e700 1 -- 172.21.15.29:0/3242983342 >> 172.21.15.29:0/3242983342 conn(0x7f0dc00c8bd0 msgr2=0x7f0dc00c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:59.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.012+0000 7f0dc668e700 1 -- 172.21.15.29:0/3242983342 shutdown_connections 2024-01-24T13:58:59.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.012+0000 7f0dc668e700 1 -- 172.21.15.29:0/3242983342 wait complete. 2024-01-24T13:58:59.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.013+0000 7f0dc668e700 1 Processor -- start 2024-01-24T13:58:59.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.013+0000 7f0dc668e700 1 -- start start 2024-01-24T13:58:59.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dc668e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da8004690 0x7f0dc00c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dc668e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dc00accc0 0x7f0dc00ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dc668e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0dc0163c80 0x7f0dc01640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dc668e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc00ac830 con 0x7f0dc00accc0 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dc668e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc01645d0 con 0x7f0da8004690 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dc668e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc00abc20 con 0x7f0dc0163c80 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dbf7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dc00accc0 0x7f0dc00ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dbf7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dc00accc0 0x7f0dc00ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33796/0 (socket says 172.21.15.29:33796) 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dbf7fe700 1 -- 172.21.15.29:0/3684648965 learned_addr learned my addr 172.21.15.29:0/3684648965 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dbffff700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da8004690 0x7f0dc00c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dc4c2b700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0dc0163c80 0x7f0dc01640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dbf7fe700 1 -- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0dc0163c80 msgr2=0x7f0dc01640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dbf7fe700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0dc0163c80 0x7f0dc01640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dbf7fe700 1 -- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da8004690 msgr2=0x7f0dc00c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.014+0000 7f0dbf7fe700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da8004690 0x7f0dc00c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.015+0000 7f0dbf7fe700 1 -- 172.21.15.29:0/3684648965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc00abf00 con 0x7f0dc00accc0 2024-01-24T13:58:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.015+0000 7f0dbf7fe700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dc00accc0 0x7f0dc00ad140 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f0db4001e90 tx=0x7f0db4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.015+0000 7f0da7fff700 1 -- 172.21.15.29:0/3684648965 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4011570 con 0x7f0dc00accc0 2024-01-24T13:58:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.015+0000 7f0da7fff700 1 -- 172.21.15.29:0/3684648965 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db40116f0 con 0x7f0dc00accc0 2024-01-24T13:58:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.015+0000 7f0dc668e700 1 -- 172.21.15.29:0/3684648965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc00ac1e0 con 0x7f0dc00accc0 2024-01-24T13:58:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.015+0000 7f0da7fff700 1 -- 172.21.15.29:0/3684648965 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4011ab0 con 0x7f0dc00accc0 2024-01-24T13:58:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.015+0000 7f0dc668e700 1 -- 172.21.15.29:0/3684648965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc00c52e0 con 0x7f0dc00accc0 2024-01-24T13:58:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.016+0000 7f0dc668e700 1 -- 172.21.15.29:0/3684648965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc00475e0 con 0x7f0dc00accc0 2024-01-24T13:58:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.016+0000 7f0da7fff700 1 -- 172.21.15.29:0/3684648965 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0db401e440 con 0x7f0dc00accc0 2024-01-24T13:58:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.017+0000 7f0da7fff700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0dac05f150 0x7f0dac061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.017+0000 7f0da7fff700 1 -- 172.21.15.29:0/3684648965 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f0db407b2d0 con 0x7f0dc00accc0 2024-01-24T13:58:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.017+0000 7f0dbffff700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0dac05f150 0x7f0dac061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.017+0000 7f0dbffff700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0dac05f150 0x7f0dac061610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f0db0001200 tx=0x7f0db0001df0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:59.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.021+0000 7f0da7fff700 1 -- 172.21.15.29:0/3684648965 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0db404c230 con 0x7f0dc00accc0 2024-01-24T13:58:59.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.200+0000 7f0dc668e700 1 -- 172.21.15.29:0/3684648965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0dc0068290 con 0x7f0dc00accc0 2024-01-24T13:58:59.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.201+0000 7f0da7fff700 1 -- 172.21.15.29:0/3684648965 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0db4011e60 con 0x7f0dc00accc0 2024-01-24T13:58:59.200 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:58:59.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.203+0000 7f0dc668e700 1 -- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0dac05f150 msgr2=0x7f0dac061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.203+0000 7f0dc668e700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0dac05f150 0x7f0dac061610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f0db0001200 tx=0x7f0db0001df0).stop 2024-01-24T13:58:59.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.203+0000 7f0dc668e700 1 -- 172.21.15.29:0/3684648965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dc00accc0 msgr2=0x7f0dc00ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.203+0000 7f0dc668e700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dc00accc0 0x7f0dc00ad140 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f0db4001e90 tx=0x7f0db4016000).stop 2024-01-24T13:58:59.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.203+0000 7f0dc668e700 1 -- 172.21.15.29:0/3684648965 shutdown_connections 2024-01-24T13:58:59.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.203+0000 7f0dc668e700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0dc0163c80 0x7f0dc01640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.203+0000 7f0dc668e700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da8004690 0x7f0dc00c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.203+0000 7f0dc668e700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0dac05f150 0x7f0dac061610 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.203+0000 7f0dc668e700 1 --2- 172.21.15.29:0/3684648965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dc00accc0 0x7f0dc00ad140 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.203+0000 7f0dc668e700 1 -- 172.21.15.29:0/3684648965 >> 172.21.15.29:0/3684648965 conn(0x7f0dc00c8bd0 msgr2=0x7f0dc0156fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:59.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.203+0000 7f0dc668e700 1 -- 172.21.15.29:0/3684648965 shutdown_connections 2024-01-24T13:58:59.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.203+0000 7f0dc668e700 1 -- 172.21.15.29:0/3684648965 wait complete. 2024-01-24T13:58:59.217 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-24T13:58:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.350+0000 7f179283b700 1 Processor -- start 2024-01-24T13:58:59.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.350+0000 7f179283b700 1 -- start start 2024-01-24T13:58:59.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.351+0000 7f179283b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f178c0664c0 0x7f178c066950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.351+0000 7f179283b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c066e40 con 0x7f178c13a5e0 2024-01-24T13:58:59.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.351+0000 7f179283b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c0640d0 con 0x7f178c0bdb30 2024-01-24T13:58:59.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.351+0000 7f179283b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178c0679f0 con 0x7f178c0664c0 2024-01-24T13:58:59.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.351+0000 7f178b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f178c0664c0 0x7f178c066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.351+0000 7f178bfff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f178c13a5e0 0x7f178c0639c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46318/0 (socket says 172.21.15.29:46318) 2024-01-24T13:58:59.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.351+0000 7f178b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f178c0664c0 0x7f178c066950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33804/0 (socket says 172.21.15.29:33804) 2024-01-24T13:58:59.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.351+0000 7f178bfff700 1 -- 172.21.15.29:0/142619259 learned_addr learned my addr 172.21.15.29:0/142619259 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:59.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.352+0000 7f178b7fe700 1 -- 172.21.15.29:0/142619259 >> v1:172.21.15.163:6789/0 conn(0x7f178c0bdb30 legacy=0x7f178c0bdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.352+0000 7f178b7fe700 1 -- 172.21.15.29:0/142619259 >> v1:172.21.15.29:6789/0 conn(0x7f178c13a5e0 legacy=0x7f178c0639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.352+0000 7f178b7fe700 1 -- 172.21.15.29:0/142619259 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c067cd0 con 0x7f178c0664c0 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.352+0000 7f178affd700 1 -- 172.21.15.29:0/142619259 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698105311 0 0) 0x7f178c066e40 con 0x7f178c13a5e0 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.352+0000 7f178b7fe700 1 --2- 172.21.15.29:0/142619259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f178c0664c0 0x7f178c066950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f1774009760 tx=0x7f1774009060).ready entity=mon.0 client_cookie=ce5b8d96ffee96cf server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.352+0000 7f178affd700 1 -- 172.21.15.29:0/142619259 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177400b580 con 0x7f178c0664c0 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.352+0000 7f178affd700 1 -- 172.21.15.29:0/142619259 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177400b6e0 con 0x7f178c0664c0 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.352+0000 7f178affd700 1 -- 172.21.15.29:0/142619259 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177400ba60 con 0x7f178c0664c0 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.353+0000 7f179283b700 1 -- 172.21.15.29:0/142619259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f178c0664c0 msgr2=0x7f178c066950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.353+0000 7f179283b700 1 --2- 172.21.15.29:0/142619259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f178c0664c0 0x7f178c066950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f1774009760 tx=0x7f1774009060).stop 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.353+0000 7f179283b700 1 -- 172.21.15.29:0/142619259 shutdown_connections 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.353+0000 7f179283b700 1 --2- 172.21.15.29:0/142619259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f178c0664c0 0x7f178c066950 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.353+0000 7f179283b700 1 -- 172.21.15.29:0/142619259 >> 172.21.15.29:0/142619259 conn(0x7f178c0c8bd0 msgr2=0x7f178c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.353+0000 7f179283b700 1 -- 172.21.15.29:0/142619259 shutdown_connections 2024-01-24T13:58:59.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.353+0000 7f179283b700 1 -- 172.21.15.29:0/142619259 wait complete. 2024-01-24T13:58:59.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.354+0000 7f179283b700 1 Processor -- start 2024-01-24T13:58:59.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.354+0000 7f179283b700 1 -- start start 2024-01-24T13:58:59.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f179283b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f178c0664c0 0x7f178c13c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f179283b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f178c0bdb30 0x7f178c153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f179283b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f178c13a5e0 0x7f178c156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f179283b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0aa6c0 con 0x7f178c13a5e0 2024-01-24T13:58:59.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f179283b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0a9ab0 con 0x7f178c0bdb30 2024-01-24T13:58:59.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f179283b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0a9d90 con 0x7f178c0664c0 2024-01-24T13:58:59.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f1790dd8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f178c13a5e0 0x7f178c156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f1790dd8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f178c13a5e0 0x7f178c156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33806/0 (socket says 172.21.15.29:33806) 2024-01-24T13:58:59.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f1790dd8700 1 -- 172.21.15.29:0/842471954 learned_addr learned my addr 172.21.15.29:0/842471954 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:59.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f178b7fe700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f178c0bdb30 0x7f178c153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f178bfff700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f178c0664c0 0x7f178c13c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f1790dd8700 1 -- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f178c0664c0 msgr2=0x7f178c13c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.355+0000 7f1790dd8700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f178c0664c0 0x7f178c13c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.356+0000 7f1790dd8700 1 -- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f178c0bdb30 msgr2=0x7f178c153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.356+0000 7f1790dd8700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f178c0bdb30 0x7f178c153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.356+0000 7f1790dd8700 1 -- 172.21.15.29:0/842471954 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17740092a0 con 0x7f178c13a5e0 2024-01-24T13:58:59.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.356+0000 7f178bfff700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f178c0664c0 0x7f178c13c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:58:59.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.356+0000 7f1790dd8700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f178c13a5e0 0x7f178c156890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f178400c7c0 tx=0x7f17840015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:59.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.356+0000 7f1783fff700 1 -- 172.21.15.29:0/842471954 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784010510 con 0x7f178c13a5e0 2024-01-24T13:58:59.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.356+0000 7f1783fff700 1 -- 172.21.15.29:0/842471954 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1784010670 con 0x7f178c13a5e0 2024-01-24T13:58:59.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.356+0000 7f179283b700 1 -- 172.21.15.29:0/842471954 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c0aa070 con 0x7f178c13a5e0 2024-01-24T13:58:59.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.356+0000 7f1783fff700 1 -- 172.21.15.29:0/842471954 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17840109f0 con 0x7f178c13a5e0 2024-01-24T13:58:59.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.356+0000 7f179283b700 1 -- 172.21.15.29:0/842471954 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c13d420 con 0x7f178c13a5e0 2024-01-24T13:58:59.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.357+0000 7f179283b700 1 -- 172.21.15.29:0/842471954 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c158110 con 0x7f178c13a5e0 2024-01-24T13:58:59.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.357+0000 7f1783fff700 1 -- 172.21.15.29:0/842471954 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f178401b400 con 0x7f178c13a5e0 2024-01-24T13:58:59.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.358+0000 7f1783fff700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f176c05f150 0x7f176c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.358+0000 7f1783fff700 1 -- 172.21.15.29:0/842471954 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f178407a320 con 0x7f178c13a5e0 2024-01-24T13:58:59.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.358+0000 7f178bfff700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f176c05f150 0x7f176c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.359+0000 7f178bfff700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f176c05f150 0x7f176c061610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f177c0011e0 tx=0x7f177c009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:59.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.362+0000 7f1783fff700 1 -- 172.21.15.29:0/842471954 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f178404b280 con 0x7f178c13a5e0 2024-01-24T13:58:59.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.540+0000 7f179283b700 1 -- 172.21.15.29:0/842471954 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f178c0475e0 con 0x7f178c13a5e0 2024-01-24T13:58:59.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.541+0000 7f1783fff700 1 -- 172.21.15.29:0/842471954 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f178404e8a0 con 0x7f178c13a5e0 2024-01-24T13:58:59.540 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-01-24T13:58:59.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.543+0000 7f179283b700 1 -- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f176c05f150 msgr2=0x7f176c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.543+0000 7f179283b700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f176c05f150 0x7f176c061610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f177c0011e0 tx=0x7f177c009040).stop 2024-01-24T13:58:59.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.543+0000 7f179283b700 1 -- 172.21.15.29:0/842471954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f178c13a5e0 msgr2=0x7f178c156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.543+0000 7f179283b700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f178c13a5e0 0x7f178c156890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f178400c7c0 tx=0x7f17840015e0).stop 2024-01-24T13:58:59.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.543+0000 7f179283b700 1 -- 172.21.15.29:0/842471954 shutdown_connections 2024-01-24T13:58:59.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.543+0000 7f179283b700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f176c05f150 0x7f176c061610 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.543+0000 7f179283b700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f178c13a5e0 0x7f178c156890 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.543+0000 7f179283b700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f178c0664c0 0x7f178c13c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.543+0000 7f179283b700 1 --2- 172.21.15.29:0/842471954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f178c0bdb30 0x7f178c153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.543+0000 7f179283b700 1 -- 172.21.15.29:0/842471954 >> 172.21.15.29:0/842471954 conn(0x7f178c0c8bd0 msgr2=0x7f178c155750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:59.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.543+0000 7f179283b700 1 -- 172.21.15.29:0/842471954 shutdown_connections 2024-01-24T13:58:59.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.543+0000 7f179283b700 1 -- 172.21.15.29:0/842471954 wait complete. 2024-01-24T13:58:59.556 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:58:59.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.686+0000 7ffb49036700 1 Processor -- start 2024-01-24T13:58:59.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.686+0000 7ffb49036700 1 -- start start 2024-01-24T13:58:59.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb49036700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb4414e840 0x7ffb440c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb49036700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffb44062ec0 0x7ffb440bc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb49036700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb440c21e0 con 0x7ffb440c1d60 2024-01-24T13:58:59.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb49036700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb440640f0 con 0x7ffb44062ec0 2024-01-24T13:58:59.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb49036700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb440634e0 con 0x7ffb4414e840 2024-01-24T13:58:59.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb4259c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ffb440c1d60 0x7ffb440627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46320/0 (socket says 172.21.15.29:46320) 2024-01-24T13:58:59.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb4359e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffb44062ec0 0x7ffb440bc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb4259c700 1 -- 172.21.15.29:0/2998332708 learned_addr learned my addr 172.21.15.29:0/2998332708 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:58:59.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb42d9d700 1 --2- 172.21.15.29:0/2998332708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb4414e840 0x7ffb440c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb4359e700 1 -- 172.21.15.29:0/2998332708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb4414e840 msgr2=0x7ffb440c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:58:59.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb4359e700 1 --2- 172.21.15.29:0/2998332708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb4414e840 0x7ffb440c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb4359e700 1 -- 172.21.15.29:0/2998332708 >> v1:172.21.15.29:6789/0 conn(0x7ffb440c1d60 legacy=0x7ffb440627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb41d9b700 1 -- 172.21.15.29:0/2998332708 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112237407 0 0) 0x7ffb440c21e0 con 0x7ffb440c1d60 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.687+0000 7ffb4359e700 1 -- 172.21.15.29:0/2998332708 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb440637c0 con 0x7ffb44062ec0 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.688+0000 7ffb4359e700 1 --2- 172.21.15.29:0/2998332708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb44062ec0 0x7ffb440bc4b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7ffb3c009e60 tx=0x7ffb3c009780).ready entity=mon.0 client_cookie=618134414d1a58f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.688+0000 7ffb41d9b700 1 -- 172.21.15.29:0/2998332708 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00e070 con 0x7ffb44062ec0 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.688+0000 7ffb41d9b700 1 -- 172.21.15.29:0/2998332708 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c00bc30 con 0x7ffb44062ec0 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.688+0000 7ffb41d9b700 1 -- 172.21.15.29:0/2998332708 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c012410 con 0x7ffb44062ec0 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.688+0000 7ffb49036700 1 -- 172.21.15.29:0/2998332708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb44062ec0 msgr2=0x7ffb440bc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.688+0000 7ffb49036700 1 --2- 172.21.15.29:0/2998332708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb44062ec0 0x7ffb440bc4b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7ffb3c009e60 tx=0x7ffb3c009780).stop 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.688+0000 7ffb49036700 1 -- 172.21.15.29:0/2998332708 shutdown_connections 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.688+0000 7ffb49036700 1 --2- 172.21.15.29:0/2998332708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb44062ec0 0x7ffb440bc4b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.688+0000 7ffb49036700 1 --2- 172.21.15.29:0/2998332708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb4414e840 0x7ffb440c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.688+0000 7ffb49036700 1 -- 172.21.15.29:0/2998332708 >> 172.21.15.29:0/2998332708 conn(0x7ffb44064aa0 msgr2=0x7ffb44064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.688+0000 7ffb49036700 1 -- 172.21.15.29:0/2998332708 shutdown_connections 2024-01-24T13:58:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.688+0000 7ffb49036700 1 -- 172.21.15.29:0/2998332708 wait complete. 2024-01-24T13:58:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.690+0000 7ffb49036700 1 Processor -- start 2024-01-24T13:58:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.690+0000 7ffb49036700 1 -- start start 2024-01-24T13:58:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb49036700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb44062ec0 0x7ffb441379c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb49036700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb440c1d60 0x7ffb44136c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb49036700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4414e840 0x7ffb4413a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb49036700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb44137180 con 0x7ffb4414e840 2024-01-24T13:58:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb49036700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb440c5120 con 0x7ffb440c1d60 2024-01-24T13:58:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb49036700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb440c5400 con 0x7ffb44062ec0 2024-01-24T13:58:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb4359e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4414e840 0x7ffb4413a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb4359e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4414e840 0x7ffb4413a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33826/0 (socket says 172.21.15.29:33826) 2024-01-24T13:58:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb4359e700 1 -- 172.21.15.29:0/3408076372 learned_addr learned my addr 172.21.15.29:0/3408076372 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:58:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb42d9d700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb44062ec0 0x7ffb441379c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb4259c700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb440c1d60 0x7ffb44136c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb4359e700 1 -- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb44062ec0 msgr2=0x7ffb441379c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb4359e700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb44062ec0 0x7ffb441379c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb4359e700 1 -- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb440c1d60 msgr2=0x7ffb44136c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb4359e700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb440c1d60 0x7ffb44136c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb4359e700 1 -- 172.21.15.29:0/3408076372 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c009a20 con 0x7ffb4414e840 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb4259c700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb440c1d60 0x7ffb44136c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb4359e700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4414e840 0x7ffb4413a150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7ffb3c00b6c0 tx=0x7ffb3c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb3affd700 1 -- 172.21.15.29:0/3408076372 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c011750 con 0x7ffb4414e840 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb3affd700 1 -- 172.21.15.29:0/3408076372 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c0118b0 con 0x7ffb4414e840 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.691+0000 7ffb49036700 1 -- 172.21.15.29:0/3408076372 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb440c56e0 con 0x7ffb4414e840 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.692+0000 7ffb3affd700 1 -- 172.21.15.29:0/3408076372 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c011c70 con 0x7ffb4414e840 2024-01-24T13:58:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.692+0000 7ffb49036700 1 -- 172.21.15.29:0/3408076372 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb440c5ca0 con 0x7ffb4414e840 2024-01-24T13:58:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.693+0000 7ffb49036700 1 -- 172.21.15.29:0/3408076372 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb440475e0 con 0x7ffb4414e840 2024-01-24T13:58:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.693+0000 7ffb3affd700 1 -- 172.21.15.29:0/3408076372 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ffb3c003040 con 0x7ffb4414e840 2024-01-24T13:58:59.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.693+0000 7ffb3affd700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffb2405f150 0x7ffb24061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:58:59.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.693+0000 7ffb3affd700 1 -- 172.21.15.29:0/3408076372 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7ffb3c07c4e0 con 0x7ffb4414e840 2024-01-24T13:58:59.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.694+0000 7ffb42d9d700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffb2405f150 0x7ffb24061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:58:59.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.694+0000 7ffb42d9d700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffb2405f150 0x7ffb24061610 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7ffb3400bb00 tx=0x7ffb34008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:58:59.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.698+0000 7ffb3affd700 1 -- 172.21.15.29:0/3408076372 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ffb3c04d4a0 con 0x7ffb4414e840 2024-01-24T13:58:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.880+0000 7ffb49036700 1 -- 172.21.15.29:0/3408076372 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb44063aa0 con 0x7ffb4414e840 2024-01-24T13:58:59.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.881+0000 7ffb3affd700 1 -- 172.21.15.29:0/3408076372 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7ffb3c050ac0 con 0x7ffb4414e840 2024-01-24T13:58:59.881 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:58:59.881 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:58:59.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.883+0000 7ffb49036700 1 -- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffb2405f150 msgr2=0x7ffb24061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.883+0000 7ffb49036700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffb2405f150 0x7ffb24061610 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7ffb3400bb00 tx=0x7ffb34008040).stop 2024-01-24T13:58:59.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.883+0000 7ffb49036700 1 -- 172.21.15.29:0/3408076372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4414e840 msgr2=0x7ffb4413a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:58:59.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.883+0000 7ffb49036700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4414e840 0x7ffb4413a150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7ffb3c00b6c0 tx=0x7ffb3c009780).stop 2024-01-24T13:58:59.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.884+0000 7ffb49036700 1 -- 172.21.15.29:0/3408076372 shutdown_connections 2024-01-24T13:58:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.884+0000 7ffb49036700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffb2405f150 0x7ffb24061610 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.884+0000 7ffb49036700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4414e840 0x7ffb4413a150 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.884+0000 7ffb49036700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb44062ec0 0x7ffb441379c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.884+0000 7ffb49036700 1 --2- 172.21.15.29:0/3408076372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb440c1d60 0x7ffb44136c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:58:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.884+0000 7ffb49036700 1 -- 172.21.15.29:0/3408076372 >> 172.21.15.29:0/3408076372 conn(0x7ffb44064aa0 msgr2=0x7ffb44153b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:58:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.884+0000 7ffb49036700 1 -- 172.21.15.29:0/3408076372 shutdown_connections 2024-01-24T13:58:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:58:59.884+0000 7ffb49036700 1 -- 172.21.15.29:0/3408076372 wait complete. 2024-01-24T13:58:59.897 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T13:58:16.949817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T13:58:18.174460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-24T13:58:21.232882+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T13:58:59.897 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T13:59:00.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.031+0000 7f263fc57700 1 Processor -- start 2024-01-24T13:59:00.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.031+0000 7f263fc57700 1 -- start start 2024-01-24T13:59:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.032+0000 7f263fc57700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26380c8490 0x7f26380a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.032+0000 7f263fc57700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2638152240 con 0x7f2638064760 2024-01-24T13:59:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.032+0000 7f263fc57700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26381586c0 con 0x7f26380bfd40 2024-01-24T13:59:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.032+0000 7f263fc57700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380aa670 con 0x7f26380c8490 2024-01-24T13:59:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.032+0000 7f263d9f3700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2638064760 0x7f2638064b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46328/0 (socket says 172.21.15.29:46328) 2024-01-24T13:59:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.032+0000 7f263d9f3700 1 -- 172.21.15.29:0/3285710205 learned_addr learned my addr 172.21.15.29:0/3285710205 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.032+0000 7f263d1f2700 1 --2- 172.21.15.29:0/3285710205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26380c8490 0x7f26380a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.033+0000 7f263c9f1700 1 -- 172.21.15.29:0/3285710205 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777921624 0 0) 0x7f2638152240 con 0x7f2638064760 2024-01-24T13:59:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.033+0000 7f263c9f1700 1 -- 172.21.15.29:0/3285710205 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26240033f0 con 0x7f2638064760 2024-01-24T13:59:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.033+0000 7f263c9f1700 1 -- 172.21.15.29:0/3285710205 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627157704 0 0) 0x7f26381586c0 con 0x7f26380bfd40 2024-01-24T13:59:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.033+0000 7f263c9f1700 1 -- 172.21.15.29:0/3285710205 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2638152240 con 0x7f26380bfd40 2024-01-24T13:59:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.033+0000 7f263d1f2700 1 -- 172.21.15.29:0/3285710205 >> v1:172.21.15.163:6789/0 conn(0x7f26380bfd40 legacy=0x7f26380c01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.033+0000 7f263d1f2700 1 -- 172.21.15.29:0/3285710205 >> v1:172.21.15.29:6789/0 conn(0x7f2638064760 legacy=0x7f2638064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.033+0000 7f263d1f2700 1 -- 172.21.15.29:0/3285710205 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2638157d90 con 0x7f26380c8490 2024-01-24T13:59:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.033+0000 7f263d1f2700 1 --2- 172.21.15.29:0/3285710205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26380c8490 0x7f26380a9d90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2628009790 tx=0x7f2628009060).ready entity=mon.2 client_cookie=10d4ac1c318c6702 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.033+0000 7f263c9f1700 1 -- 172.21.15.29:0/3285710205 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262800f510 con 0x7f26380c8490 2024-01-24T13:59:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.033+0000 7f263c9f1700 1 -- 172.21.15.29:0/3285710205 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262800f670 con 0x7f26380c8490 2024-01-24T13:59:00.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.033+0000 7f263c9f1700 1 -- 172.21.15.29:0/3285710205 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262800f9f0 con 0x7f26380c8490 2024-01-24T13:59:00.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.034+0000 7f263fc57700 1 -- 172.21.15.29:0/3285710205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26380c8490 msgr2=0x7f26380a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.034+0000 7f263fc57700 1 --2- 172.21.15.29:0/3285710205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26380c8490 0x7f26380a9d90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2628009790 tx=0x7f2628009060).stop 2024-01-24T13:59:00.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.034+0000 7f263fc57700 1 -- 172.21.15.29:0/3285710205 shutdown_connections 2024-01-24T13:59:00.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.034+0000 7f263fc57700 1 --2- 172.21.15.29:0/3285710205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26380c8490 0x7f26380a9d90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.034+0000 7f263fc57700 1 -- 172.21.15.29:0/3285710205 >> 172.21.15.29:0/3285710205 conn(0x7f26380cb8b0 msgr2=0x7f26380c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:00.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.034+0000 7f263fc57700 1 -- 172.21.15.29:0/3285710205 shutdown_connections 2024-01-24T13:59:00.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.034+0000 7f263fc57700 1 -- 172.21.15.29:0/3285710205 wait complete. 2024-01-24T13:59:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.035+0000 7f263fc57700 1 Processor -- start 2024-01-24T13:59:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.035+0000 7f263fc57700 1 -- start start 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263fc57700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2638064760 0x7f2638165530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263fc57700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26380bfd40 0x7f26380b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263fc57700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26380c8490 0x7f26380c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263fc57700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380ca580 con 0x7f26380c8490 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263fc57700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2638165a20 con 0x7f2638064760 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263fc57700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380c2310 con 0x7f26380bfd40 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263e1f4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26380c8490 0x7f26380c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263e1f4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26380c8490 0x7f26380c3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33830/0 (socket says 172.21.15.29:33830) 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263e1f4700 1 -- 172.21.15.29:0/45676575 learned_addr learned my addr 172.21.15.29:0/45676575 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263d9f3700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2638064760 0x7f2638165530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263d1f2700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26380bfd40 0x7f26380b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263e1f4700 1 -- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26380bfd40 msgr2=0x7f26380b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263e1f4700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26380bfd40 0x7f26380b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263e1f4700 1 -- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2638064760 msgr2=0x7f2638165530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263e1f4700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2638064760 0x7f2638165530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.036+0000 7f263e1f4700 1 -- 172.21.15.29:0/45676575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26280092a0 con 0x7f26380c8490 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.037+0000 7f263e1f4700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26380c8490 0x7f26380c3cb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f263000c790 tx=0x7f2630007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.037+0000 7f262dffb700 1 -- 172.21.15.29:0/45676575 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630010510 con 0x7f26380c8490 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.037+0000 7f262dffb700 1 -- 172.21.15.29:0/45676575 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2630010670 con 0x7f26380c8490 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.037+0000 7f263fc57700 1 -- 172.21.15.29:0/45676575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26380c25f0 con 0x7f26380c8490 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.037+0000 7f262dffb700 1 -- 172.21.15.29:0/45676575 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630010a30 con 0x7f26380c8490 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.037+0000 7f263fc57700 1 -- 172.21.15.29:0/45676575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26380c2bb0 con 0x7f26380c8490 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.038+0000 7f263fc57700 1 -- 172.21.15.29:0/45676575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26380c2f00 con 0x7f26380c8490 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.038+0000 7f262dffb700 1 -- 172.21.15.29:0/45676575 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f263001b400 con 0x7f26380c8490 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.039+0000 7f262dffb700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f261c05f150 0x7f261c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.039+0000 7f262dffb700 1 -- 172.21.15.29:0/45676575 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f263007a1f0 con 0x7f26380c8490 2024-01-24T13:59:00.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.039+0000 7f263d9f3700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f261c05f150 0x7f261c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.039+0000 7f263d9f3700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f261c05f150 0x7f261c061610 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f2634007fd0 tx=0x7f2634009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.043+0000 7f262dffb700 1 -- 172.21.15.29:0/45676575 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f263004b150 con 0x7f26380c8490 2024-01-24T13:59:00.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.228+0000 7f263fc57700 1 -- 172.21.15.29:0/45676575 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f26380475e0 con 0x7f26380c8490 2024-01-24T13:59:00.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.228+0000 7f262dffb700 1 -- 172.21.15.29:0/45676575 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f263004e770 con 0x7f26380c8490 2024-01-24T13:59:00.228 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T13:59:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.230+0000 7f263fc57700 1 -- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f261c05f150 msgr2=0x7f261c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.230+0000 7f263fc57700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f261c05f150 0x7f261c061610 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f2634007fd0 tx=0x7f2634009040).stop 2024-01-24T13:59:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.230+0000 7f263fc57700 1 -- 172.21.15.29:0/45676575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26380c8490 msgr2=0x7f26380c3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.230+0000 7f263fc57700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26380c8490 0x7f26380c3cb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f263000c790 tx=0x7f2630007f50).stop 2024-01-24T13:59:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.230+0000 7f263fc57700 1 -- 172.21.15.29:0/45676575 shutdown_connections 2024-01-24T13:59:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.230+0000 7f263fc57700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f261c05f150 0x7f261c061610 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.230+0000 7f263fc57700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26380c8490 0x7f26380c3cb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.230+0000 7f263fc57700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2638064760 0x7f2638165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.230+0000 7f263fc57700 1 --2- 172.21.15.29:0/45676575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26380bfd40 0x7f26380b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.230+0000 7f263fc57700 1 -- 172.21.15.29:0/45676575 >> 172.21.15.29:0/45676575 conn(0x7f26380cb8b0 msgr2=0x7f2638150530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.230+0000 7f263fc57700 1 -- 172.21.15.29:0/45676575 shutdown_connections 2024-01-24T13:59:00.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.230+0000 7f263fc57700 1 -- 172.21.15.29:0/45676575 wait complete. 2024-01-24T13:59:00.243 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T13:59:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.377+0000 7fd8f5e62700 1 Processor -- start 2024-01-24T13:59:00.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.377+0000 7fd8f5e62700 1 -- start start 2024-01-24T13:59:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.378+0000 7fd8f5e62700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8f00692a0 0x7fd8f0065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.378+0000 7fd8f5e62700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd8f0066370 0x7fd8f0069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.378+0000 7fd8f5e62700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f00c3e70 con 0x7fd8f0069b70 2024-01-24T13:59:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.378+0000 7fd8f5e62700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f00c4300 con 0x7fd8f0066370 2024-01-24T13:59:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.378+0000 7fd8f5e62700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00c3260 con 0x7fd8f00692a0 2024-01-24T13:59:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.378+0000 7fd8effff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd8f0069b70 0x7fd8f00bcd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46344/0 (socket says 172.21.15.29:46344) 2024-01-24T13:59:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.378+0000 7fd8effff700 1 -- 172.21.15.29:0/4060668602 learned_addr learned my addr 172.21.15.29:0/4060668602 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.378+0000 7fd8ef7fe700 1 --2- 172.21.15.29:0/4060668602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8f00692a0 0x7fd8f0065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.378+0000 7fd8eeffd700 1 --2- 172.21.15.29:0/4060668602 >> v2:172.21.15.163:3300/0 conn(0x7fd8f0066370 0x7fd8f0069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.379+0000 7fd8ee7fc700 1 -- 172.21.15.29:0/4060668602 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379015064 0 0) 0x7fd8f00c3e70 con 0x7fd8f0069b70 2024-01-24T13:59:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.379+0000 7fd8ee7fc700 1 -- 172.21.15.29:0/4060668602 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8d80033f0 con 0x7fd8f0069b70 2024-01-24T13:59:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.379+0000 7fd8ef7fe700 1 -- 172.21.15.29:0/4060668602 >> v2:172.21.15.163:3300/0 conn(0x7fd8f0066370 msgr2=0x7fd8f0069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.379+0000 7fd8ef7fe700 1 --2- 172.21.15.29:0/4060668602 >> v2:172.21.15.163:3300/0 conn(0x7fd8f0066370 0x7fd8f0069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.379+0000 7fd8ef7fe700 1 -- 172.21.15.29:0/4060668602 >> v1:172.21.15.29:6789/0 conn(0x7fd8f0069b70 legacy=0x7fd8f00bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.379+0000 7fd8ef7fe700 1 -- 172.21.15.29:0/4060668602 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f00c3540 con 0x7fd8f00692a0 2024-01-24T13:59:00.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.379+0000 7fd8ef7fe700 1 --2- 172.21.15.29:0/4060668602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8f00692a0 0x7fd8f0065e80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fd8e0009760 tx=0x7fd8e0009060).ready entity=mon.2 client_cookie=ad1199e688da2ed8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:00.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.379+0000 7fd8ee7fc700 1 -- 172.21.15.29:0/4060668602 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000d4a0 con 0x7fd8f00692a0 2024-01-24T13:59:00.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.379+0000 7fd8ee7fc700 1 -- 172.21.15.29:0/4060668602 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e000d600 con 0x7fd8f00692a0 2024-01-24T13:59:00.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.379+0000 7fd8ee7fc700 1 -- 172.21.15.29:0/4060668602 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000d980 con 0x7fd8f00692a0 2024-01-24T13:59:00.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.380+0000 7fd8f5e62700 1 -- 172.21.15.29:0/4060668602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8f00692a0 msgr2=0x7fd8f0065e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.380+0000 7fd8f5e62700 1 --2- 172.21.15.29:0/4060668602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8f00692a0 0x7fd8f0065e80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fd8e0009760 tx=0x7fd8e0009060).stop 2024-01-24T13:59:00.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.380+0000 7fd8f5e62700 1 -- 172.21.15.29:0/4060668602 shutdown_connections 2024-01-24T13:59:00.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.380+0000 7fd8f5e62700 1 --2- 172.21.15.29:0/4060668602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8f00692a0 0x7fd8f0065e80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.380+0000 7fd8f5e62700 1 --2- 172.21.15.29:0/4060668602 >> v2:172.21.15.163:3300/0 conn(0x7fd8f0066370 0x7fd8f0069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.380+0000 7fd8f5e62700 1 -- 172.21.15.29:0/4060668602 >> 172.21.15.29:0/4060668602 conn(0x7fd8f0064390 msgr2=0x7fd8f00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:00.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.380+0000 7fd8f5e62700 1 -- 172.21.15.29:0/4060668602 shutdown_connections 2024-01-24T13:59:00.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.380+0000 7fd8f5e62700 1 -- 172.21.15.29:0/4060668602 wait complete. 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.381+0000 7fd8f5e62700 1 Processor -- start 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.381+0000 7fd8f5e62700 1 -- start start 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8f5e62700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f0066370 0x7fd8f0144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8f5e62700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8f00692a0 0x7fd8f0165e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8f5e62700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8f0069b70 0x7fd8f0166340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8f5e62700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f01460e0 con 0x7fd8f0066370 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8f5e62700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f01454d0 con 0x7fd8f00692a0 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8f5e62700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f01457b0 con 0x7fd8f0069b70 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8ef7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f0066370 0x7fd8f0144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8ef7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f0066370 0x7fd8f0144ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33844/0 (socket says 172.21.15.29:33844) 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8ef7fe700 1 -- 172.21.15.29:0/2307791553 learned_addr learned my addr 172.21.15.29:0/2307791553 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8eeffd700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8f00692a0 0x7fd8f0165e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8effff700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8f0069b70 0x7fd8f0166340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8ef7fe700 1 -- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8f0069b70 msgr2=0x7fd8f0166340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8ef7fe700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8f0069b70 0x7fd8f0166340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8ef7fe700 1 -- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8f00692a0 msgr2=0x7fd8f0165e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8ef7fe700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8f00692a0 0x7fd8f0165e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.382+0000 7fd8ef7fe700 1 -- 172.21.15.29:0/2307791553 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e00092a0 con 0x7fd8f0066370 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.383+0000 7fd8eeffd700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8f00692a0 0x7fd8f0165e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.383+0000 7fd8ef7fe700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f0066370 0x7fd8f0144ca0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fd8e0009d50 tx=0x7fd8e0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.383+0000 7fd8d77fe700 1 -- 172.21.15.29:0/2307791553 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0000c90 con 0x7fd8f0066370 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.383+0000 7fd8f5e62700 1 -- 172.21.15.29:0/2307791553 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f0145a90 con 0x7fd8f0066370 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.383+0000 7fd8d77fe700 1 -- 172.21.15.29:0/2307791553 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e000d770 con 0x7fd8f0066370 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.383+0000 7fd8d77fe700 1 -- 172.21.15.29:0/2307791553 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e001d430 con 0x7fd8f0066370 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.383+0000 7fd8f5e62700 1 -- 172.21.15.29:0/2307791553 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f0145190 con 0x7fd8f0066370 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.384+0000 7fd8d77fe700 1 -- 172.21.15.29:0/2307791553 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd8e001d860 con 0x7fd8f0066370 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.384+0000 7fd8f5e62700 1 -- 172.21.15.29:0/2307791553 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f00475e0 con 0x7fd8f0066370 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.385+0000 7fd8d77fe700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd8dc05f120 0x7fd8dc0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.385+0000 7fd8d77fe700 1 -- 172.21.15.29:0/2307791553 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fd8e007bcb0 con 0x7fd8f0066370 2024-01-24T13:59:00.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.385+0000 7fd8eeffd700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd8dc05f120 0x7fd8dc0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.385+0000 7fd8eeffd700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd8dc05f120 0x7fd8dc0615e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fd8f013d290 tx=0x7fd8e4008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:00.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.389+0000 7fd8d77fe700 1 -- 172.21.15.29:0/2307791553 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd8e004cc70 con 0x7fd8f0066370 2024-01-24T13:59:00.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.571+0000 7fd8f5e62700 1 -- 172.21.15.29:0/2307791553 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd8f00c2190 con 0x7fd8f0066370 2024-01-24T13:59:00.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.571+0000 7fd8d77fe700 1 -- 172.21.15.29:0/2307791553 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd8e0050290 con 0x7fd8f0066370 2024-01-24T13:59:00.570 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T13:59:00.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.573+0000 7fd8f5e62700 1 -- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd8dc05f120 msgr2=0x7fd8dc0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.573+0000 7fd8f5e62700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd8dc05f120 0x7fd8dc0615e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fd8f013d290 tx=0x7fd8e4008040).stop 2024-01-24T13:59:00.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.573+0000 7fd8f5e62700 1 -- 172.21.15.29:0/2307791553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f0066370 msgr2=0x7fd8f0144ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.573+0000 7fd8f5e62700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f0066370 0x7fd8f0144ca0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fd8e0009d50 tx=0x7fd8e0016000).stop 2024-01-24T13:59:00.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.573+0000 7fd8f5e62700 1 -- 172.21.15.29:0/2307791553 shutdown_connections 2024-01-24T13:59:00.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.573+0000 7fd8f5e62700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8f0069b70 0x7fd8f0166340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.573+0000 7fd8f5e62700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd8dc05f120 0x7fd8dc0615e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.573+0000 7fd8f5e62700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8f0066370 0x7fd8f0144ca0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.573+0000 7fd8f5e62700 1 --2- 172.21.15.29:0/2307791553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8f00692a0 0x7fd8f0165e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.573+0000 7fd8f5e62700 1 -- 172.21.15.29:0/2307791553 >> 172.21.15.29:0/2307791553 conn(0x7fd8f0064390 msgr2=0x7fd8f00c44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:00.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.573+0000 7fd8f5e62700 1 -- 172.21.15.29:0/2307791553 shutdown_connections 2024-01-24T13:59:00.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.573+0000 7fd8f5e62700 1 -- 172.21.15.29:0/2307791553 wait complete. 2024-01-24T13:59:00.586 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T13:59:00.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.720+0000 7f5357651700 1 Processor -- start 2024-01-24T13:59:00.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.720+0000 7f5357651700 1 -- start start 2024-01-24T13:59:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.721+0000 7f5357651700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f53500664c0 0x7f5350066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.721+0000 7f5357651700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53500686a0 con 0x7f5350134ea0 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.721+0000 7f5357651700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5350066e30 con 0x7f53500bdb30 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.721+0000 7f5357651700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53500c38e0 con 0x7f53500664c0 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.721+0000 7f5354bec700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f53500664c0 0x7f5350066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.721+0000 7f5354bec700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f53500664c0 0x7f5350066940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33860/0 (socket says 172.21.15.29:33860) 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.721+0000 7f5354bec700 1 -- 172.21.15.29:0/1150395272 learned_addr learned my addr 172.21.15.29:0/1150395272 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.721+0000 7f53553ed700 1 --1- 172.21.15.29:0/1150395272 >> v1:172.21.15.29:6789/0 conn(0x7f5350134ea0 0x7f53500c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46352/0 (socket says 172.21.15.29:46352) 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.722+0000 7f5354bec700 1 -- 172.21.15.29:0/1150395272 >> v1:172.21.15.163:6789/0 conn(0x7f53500bdb30 legacy=0x7f53500bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.722+0000 7f5354bec700 1 -- 172.21.15.29:0/1150395272 >> v1:172.21.15.29:6789/0 conn(0x7f5350134ea0 legacy=0x7f53500c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.722+0000 7f5354bec700 1 -- 172.21.15.29:0/1150395272 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5350068b30 con 0x7f53500664c0 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.722+0000 7f5343fff700 1 -- 172.21.15.29:0/1150395272 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395635640 0 0) 0x7f53500686a0 con 0x7f5350134ea0 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.722+0000 7f5354bec700 1 --2- 172.21.15.29:0/1150395272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53500664c0 0x7f5350066940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f534c006fd0 tx=0x7f534c0090f0).ready entity=mon.0 client_cookie=90f0b67312f3f675 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.722+0000 7f5343fff700 1 -- 172.21.15.29:0/1150395272 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c00b600 con 0x7f53500664c0 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.722+0000 7f5343fff700 1 -- 172.21.15.29:0/1150395272 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c00b760 con 0x7f53500664c0 2024-01-24T13:59:00.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.722+0000 7f5343fff700 1 -- 172.21.15.29:0/1150395272 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c00bae0 con 0x7f53500664c0 2024-01-24T13:59:00.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.723+0000 7f5357651700 1 -- 172.21.15.29:0/1150395272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53500664c0 msgr2=0x7f5350066940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.723+0000 7f5357651700 1 --2- 172.21.15.29:0/1150395272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53500664c0 0x7f5350066940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f534c006fd0 tx=0x7f534c0090f0).stop 2024-01-24T13:59:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.723+0000 7f5357651700 1 -- 172.21.15.29:0/1150395272 shutdown_connections 2024-01-24T13:59:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.723+0000 7f5357651700 1 --2- 172.21.15.29:0/1150395272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53500664c0 0x7f5350066940 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.723+0000 7f5357651700 1 -- 172.21.15.29:0/1150395272 >> 172.21.15.29:0/1150395272 conn(0x7f53500c8bd0 msgr2=0x7f53500c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.723+0000 7f5357651700 1 -- 172.21.15.29:0/1150395272 shutdown_connections 2024-01-24T13:59:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.723+0000 7f5357651700 1 -- 172.21.15.29:0/1150395272 wait complete. 2024-01-24T13:59:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.724+0000 7f5357651700 1 Processor -- start 2024-01-24T13:59:00.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.724+0000 7f5357651700 1 -- start start 2024-01-24T13:59:00.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.725+0000 7f5357651700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53500664c0 0x7f5350133bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.725+0000 7f5357651700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53500bdb30 0x7f53501631f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.725+0000 7f5357651700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5350134ea0 0x7f53500c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.725+0000 7f5357651700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53501340a0 con 0x7f5350134ea0 2024-01-24T13:59:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.725+0000 7f5357651700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5350163030 con 0x7f53500664c0 2024-01-24T13:59:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.725+0000 7f5357651700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53500c0740 con 0x7f53500bdb30 2024-01-24T13:59:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.725+0000 7f5355bee700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5350134ea0 0x7f53500c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.725+0000 7f5355bee700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5350134ea0 0x7f53500c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33876/0 (socket says 172.21.15.29:33876) 2024-01-24T13:59:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.725+0000 7f5355bee700 1 -- 172.21.15.29:0/3863017202 learned_addr learned my addr 172.21.15.29:0/3863017202 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.725+0000 7f53553ed700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53500664c0 0x7f5350133bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.725+0000 7f5354bec700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53500bdb30 0x7f53501631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.726+0000 7f5355bee700 1 -- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53500bdb30 msgr2=0x7f53501631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.726+0000 7f5355bee700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53500bdb30 0x7f53501631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.726+0000 7f5355bee700 1 -- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53500664c0 msgr2=0x7f5350133bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.726+0000 7f5355bee700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53500664c0 0x7f5350133bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.726+0000 7f5355bee700 1 -- 172.21.15.29:0/3863017202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c009370 con 0x7f5350134ea0 2024-01-24T13:59:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.726+0000 7f5355bee700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5350134ea0 0x7f53500c1fe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f534800c790 tx=0x7f53480015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.726+0000 7f53417fa700 1 -- 172.21.15.29:0/3863017202 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534800e4a0 con 0x7f5350134ea0 2024-01-24T13:59:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.726+0000 7f53417fa700 1 -- 172.21.15.29:0/3863017202 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534800e600 con 0x7f5350134ea0 2024-01-24T13:59:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.726+0000 7f5357651700 1 -- 172.21.15.29:0/3863017202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53500bfe10 con 0x7f5350134ea0 2024-01-24T13:59:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.726+0000 7f53417fa700 1 -- 172.21.15.29:0/3863017202 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534800e980 con 0x7f5350134ea0 2024-01-24T13:59:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.726+0000 7f5357651700 1 -- 172.21.15.29:0/3863017202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53500c03d0 con 0x7f5350134ea0 2024-01-24T13:59:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.727+0000 7f5357651700 1 -- 172.21.15.29:0/3863017202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53500475e0 con 0x7f5350134ea0 2024-01-24T13:59:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.727+0000 7f53417fa700 1 -- 172.21.15.29:0/3863017202 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f534801b3b0 con 0x7f5350134ea0 2024-01-24T13:59:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.728+0000 7f53417fa700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f533405f150 0x7f5334061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.728+0000 7f53417fa700 1 -- 172.21.15.29:0/3863017202 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f5348079ac0 con 0x7f5350134ea0 2024-01-24T13:59:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.728+0000 7f53553ed700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f533405f150 0x7f5334061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.729+0000 7f53553ed700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f533405f150 0x7f5334061610 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f5350134790 tx=0x7f5344007bb0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:00.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.732+0000 7f53417fa700 1 -- 172.21.15.29:0/3863017202 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5348048b00 con 0x7f5350134ea0 2024-01-24T13:59:00.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.913+0000 7f5357651700 1 -- 172.21.15.29:0/3863017202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5350067a90 con 0x7f5350134ea0 2024-01-24T13:59:00.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.914+0000 7f53417fa700 1 -- 172.21.15.29:0/3863017202 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5348079e60 con 0x7f5350134ea0 2024-01-24T13:59:00.913 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:59:00.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.915+0000 7f5357651700 1 -- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f533405f150 msgr2=0x7f5334061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.915+0000 7f5357651700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f533405f150 0x7f5334061610 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f5350134790 tx=0x7f5344007bb0).stop 2024-01-24T13:59:00.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.915+0000 7f5357651700 1 -- 172.21.15.29:0/3863017202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5350134ea0 msgr2=0x7f53500c1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:00.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.915+0000 7f5357651700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5350134ea0 0x7f53500c1fe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f534800c790 tx=0x7f53480015e0).stop 2024-01-24T13:59:00.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.916+0000 7f5357651700 1 -- 172.21.15.29:0/3863017202 shutdown_connections 2024-01-24T13:59:00.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.916+0000 7f5357651700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f533405f150 0x7f5334061610 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.916+0000 7f5357651700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5350134ea0 0x7f53500c1fe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.916+0000 7f5357651700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53500664c0 0x7f5350133bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.916+0000 7f5357651700 1 --2- 172.21.15.29:0/3863017202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53500bdb30 0x7f53501631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:00.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.916+0000 7f5357651700 1 -- 172.21.15.29:0/3863017202 >> 172.21.15.29:0/3863017202 conn(0x7f53500c8bd0 msgr2=0x7f53500c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:00.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.916+0000 7f5357651700 1 -- 172.21.15.29:0/3863017202 shutdown_connections 2024-01-24T13:59:00.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:00.916+0000 7f5357651700 1 -- 172.21.15.29:0/3863017202 wait complete. 2024-01-24T13:59:00.928 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T13:59:01.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.062+0000 7ff837f5a700 1 Processor -- start 2024-01-24T13:59:01.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.062+0000 7ff837f5a700 1 -- start start 2024-01-24T13:59:01.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.063+0000 7ff837f5a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff8300664c0 0x7ff830066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.063+0000 7ff837f5a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8300bdb30 0x7ff8300bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.063+0000 7ff837f5a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff830066e30 con 0x7ff83013b4d0 2024-01-24T13:59:01.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.063+0000 7ff837f5a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff830064130 con 0x7ff8300664c0 2024-01-24T13:59:01.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.063+0000 7ff837f5a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff830067d20 con 0x7ff8300bdb30 2024-01-24T13:59:01.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.063+0000 7ff8354f5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff8300664c0 0x7ff830066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.063+0000 7ff8354f5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff8300664c0 0x7ff830066940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33882/0 (socket says 172.21.15.29:33882) 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.063+0000 7ff8354f5700 1 -- 172.21.15.29:0/2761259598 learned_addr learned my addr 172.21.15.29:0/2761259598 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.063+0000 7ff835cf6700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff83013b4d0 0x7ff830063a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46354/0 (socket says 172.21.15.29:46354) 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.063+0000 7ff8364f7700 1 --2- 172.21.15.29:0/2761259598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8300bdb30 0x7ff8300bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.064+0000 7ff8354f5700 1 -- 172.21.15.29:0/2761259598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8300bdb30 msgr2=0x7ff8300bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.064+0000 7ff8354f5700 1 --2- 172.21.15.29:0/2761259598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8300bdb30 0x7ff8300bdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.064+0000 7ff8354f5700 1 -- 172.21.15.29:0/2761259598 >> v1:172.21.15.29:6789/0 conn(0x7ff83013b4d0 legacy=0x7ff830063a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.064+0000 7ff8354f5700 1 -- 172.21.15.29:0/2761259598 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff830068000 con 0x7ff8300664c0 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.064+0000 7ff834cf4700 1 -- 172.21.15.29:0/2761259598 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165647728 0 0) 0x7ff830066e30 con 0x7ff83013b4d0 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.064+0000 7ff8354f5700 1 --2- 172.21.15.29:0/2761259598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8300664c0 0x7ff830066940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7ff82c006fd0 tx=0x7ff82c0090f0).ready entity=mon.0 client_cookie=77313e1227012b5b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.064+0000 7ff834cf4700 1 -- 172.21.15.29:0/2761259598 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c00b600 con 0x7ff8300664c0 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.064+0000 7ff834cf4700 1 -- 172.21.15.29:0/2761259598 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c00b760 con 0x7ff8300664c0 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.064+0000 7ff834cf4700 1 -- 172.21.15.29:0/2761259598 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c00bae0 con 0x7ff8300664c0 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.064+0000 7ff837f5a700 1 -- 172.21.15.29:0/2761259598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8300664c0 msgr2=0x7ff830066940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.064+0000 7ff837f5a700 1 --2- 172.21.15.29:0/2761259598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8300664c0 0x7ff830066940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7ff82c006fd0 tx=0x7ff82c0090f0).stop 2024-01-24T13:59:01.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.065+0000 7ff837f5a700 1 -- 172.21.15.29:0/2761259598 shutdown_connections 2024-01-24T13:59:01.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.065+0000 7ff837f5a700 1 --2- 172.21.15.29:0/2761259598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8300bdb30 0x7ff8300bdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.065+0000 7ff837f5a700 1 --2- 172.21.15.29:0/2761259598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8300664c0 0x7ff830066940 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.065+0000 7ff837f5a700 1 -- 172.21.15.29:0/2761259598 >> 172.21.15.29:0/2761259598 conn(0x7ff8300c8bd0 msgr2=0x7ff8300c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:01.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.065+0000 7ff837f5a700 1 -- 172.21.15.29:0/2761259598 shutdown_connections 2024-01-24T13:59:01.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.065+0000 7ff837f5a700 1 -- 172.21.15.29:0/2761259598 wait complete. 2024-01-24T13:59:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.066+0000 7ff837f5a700 1 Processor -- start 2024-01-24T13:59:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.066+0000 7ff837f5a700 1 -- start start 2024-01-24T13:59:01.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff837f5a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8300664c0 0x7ff8300a86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff837f5a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8300bdb30 0x7ff8300bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff837f5a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83013b4d0 0x7ff8300be490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff837f5a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8300be980 con 0x7ff8300664c0 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff837f5a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8300a8fb0 con 0x7ff83013b4d0 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff837f5a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8300a8cd0 con 0x7ff8300bdb30 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff835cf6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8300664c0 0x7ff8300a86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff835cf6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8300664c0 0x7ff8300a86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33888/0 (socket says 172.21.15.29:33888) 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff835cf6700 1 -- 172.21.15.29:0/260345407 learned_addr learned my addr 172.21.15.29:0/260345407 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff8364f7700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83013b4d0 0x7ff8300be490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff8354f5700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8300bdb30 0x7ff8300bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff835cf6700 1 -- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8300bdb30 msgr2=0x7ff8300bdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff835cf6700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8300bdb30 0x7ff8300bdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff835cf6700 1 -- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83013b4d0 msgr2=0x7ff8300be490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff835cf6700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83013b4d0 0x7ff8300be490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff835cf6700 1 -- 172.21.15.29:0/260345407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c009370 con 0x7ff8300664c0 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff8364f7700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83013b4d0 0x7ff8300be490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff835cf6700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8300664c0 0x7ff8300a86b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7ff82400c440 tx=0x7ff824007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff821ffb700 1 -- 172.21.15.29:0/260345407 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8240072e0 con 0x7ff8300664c0 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff821ffb700 1 -- 172.21.15.29:0/260345407 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff824007440 con 0x7ff8300664c0 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff821ffb700 1 -- 172.21.15.29:0/260345407 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824005910 con 0x7ff8300664c0 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.067+0000 7ff837f5a700 1 -- 172.21.15.29:0/260345407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83012a460 con 0x7ff8300664c0 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.068+0000 7ff837f5a700 1 -- 172.21.15.29:0/260345407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83012aa20 con 0x7ff8300664c0 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.069+0000 7ff837f5a700 1 -- 172.21.15.29:0/260345407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8300475e0 con 0x7ff8300664c0 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.069+0000 7ff821ffb700 1 -- 172.21.15.29:0/260345407 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff824005d00 con 0x7ff8300664c0 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.069+0000 7ff821ffb700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff81405f150 0x7ff814061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.069+0000 7ff821ffb700 1 -- 172.21.15.29:0/260345407 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7ff824079c30 con 0x7ff8300664c0 2024-01-24T13:59:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.070+0000 7ff8354f5700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff81405f150 0x7ff814061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.070+0000 7ff8354f5700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff81405f150 0x7ff814061610 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7ff82c009130 tx=0x7ff82c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:01.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.074+0000 7ff821ffb700 1 -- 172.21.15.29:0/260345407 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff82404abf0 con 0x7ff8300664c0 2024-01-24T13:59:01.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.256+0000 7ff837f5a700 1 -- 172.21.15.29:0/260345407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff830068db0 con 0x7ff8300664c0 2024-01-24T13:59:01.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.256+0000 7ff821ffb700 1 -- 172.21.15.29:0/260345407 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff824018030 con 0x7ff8300664c0 2024-01-24T13:59:01.255 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:59:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.258+0000 7ff837f5a700 1 -- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff81405f150 msgr2=0x7ff814061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.258+0000 7ff837f5a700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff81405f150 0x7ff814061610 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7ff82c009130 tx=0x7ff82c017000).stop 2024-01-24T13:59:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.258+0000 7ff837f5a700 1 -- 172.21.15.29:0/260345407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8300664c0 msgr2=0x7ff8300a86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.258+0000 7ff837f5a700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8300664c0 0x7ff8300a86b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7ff82400c440 tx=0x7ff824007e30).stop 2024-01-24T13:59:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.258+0000 7ff837f5a700 1 -- 172.21.15.29:0/260345407 shutdown_connections 2024-01-24T13:59:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.258+0000 7ff837f5a700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83013b4d0 0x7ff8300be490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.258+0000 7ff837f5a700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff81405f150 0x7ff814061610 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.258+0000 7ff837f5a700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8300664c0 0x7ff8300a86b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.258+0000 7ff837f5a700 1 --2- 172.21.15.29:0/260345407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8300bdb30 0x7ff8300bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.258+0000 7ff837f5a700 1 -- 172.21.15.29:0/260345407 >> 172.21.15.29:0/260345407 conn(0x7ff8300c8bd0 msgr2=0x7ff8301527d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.258+0000 7ff837f5a700 1 -- 172.21.15.29:0/260345407 shutdown_connections 2024-01-24T13:59:01.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.258+0000 7ff837f5a700 1 -- 172.21.15.29:0/260345407 wait complete. 2024-01-24T13:59:01.271 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-24T13:59:01.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.404+0000 7f5cee0ad700 1 Processor -- start 2024-01-24T13:59:01.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.404+0000 7f5cee0ad700 1 -- start start 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5cee0ad700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ce8153c20 0x7f5ce80648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5cee0ad700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce8069fd0 con 0x7f5ce80644d0 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5cee0ad700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce80bc970 con 0x7f5ce8154280 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5cee0ad700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce80693c0 con 0x7f5ce8153c20 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5ce7fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ce8153c20 0x7f5ce80648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5ce77fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5ce80644d0 0x7f5ce80689c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46362/0 (socket says 172.21.15.29:46362) 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5ce7fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ce8153c20 0x7f5ce80648b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33892/0 (socket says 172.21.15.29:33892) 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5ce77fe700 1 -- 172.21.15.29:0/3342700525 learned_addr learned my addr 172.21.15.29:0/3342700525 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5ce7fff700 1 -- 172.21.15.29:0/3342700525 >> v1:172.21.15.163:6789/0 conn(0x7f5ce8154280 legacy=0x7f5ce8153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5ce7fff700 1 -- 172.21.15.29:0/3342700525 >> v1:172.21.15.29:6789/0 conn(0x7f5ce80644d0 legacy=0x7f5ce80689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5ce7fff700 1 -- 172.21.15.29:0/3342700525 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce80696a0 con 0x7f5ce8153c20 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5ce67fc700 1 -- 172.21.15.29:0/3342700525 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172528179 0 0) 0x7f5ce8069fd0 con 0x7f5ce80644d0 2024-01-24T13:59:01.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.405+0000 7f5ce7fff700 1 --2- 172.21.15.29:0/3342700525 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ce8153c20 0x7f5ce80648b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f5ce000af70 tx=0x7f5ce000a890).ready entity=mon.0 client_cookie=4f01234beb656f9b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.406+0000 7f5ce67fc700 1 -- 172.21.15.29:0/3342700525 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce000f070 con 0x7f5ce8153c20 2024-01-24T13:59:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.406+0000 7f5ce67fc700 1 -- 172.21.15.29:0/3342700525 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce000a2e0 con 0x7f5ce8153c20 2024-01-24T13:59:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.406+0000 7f5ce67fc700 1 -- 172.21.15.29:0/3342700525 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce000cce0 con 0x7f5ce8153c20 2024-01-24T13:59:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.406+0000 7f5cee0ad700 1 -- 172.21.15.29:0/3342700525 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ce8153c20 msgr2=0x7f5ce80648b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.406+0000 7f5cee0ad700 1 --2- 172.21.15.29:0/3342700525 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ce8153c20 0x7f5ce80648b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f5ce000af70 tx=0x7f5ce000a890).stop 2024-01-24T13:59:01.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.406+0000 7f5cee0ad700 1 -- 172.21.15.29:0/3342700525 shutdown_connections 2024-01-24T13:59:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.406+0000 7f5cee0ad700 1 --2- 172.21.15.29:0/3342700525 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ce8153c20 0x7f5ce80648b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.406+0000 7f5cee0ad700 1 -- 172.21.15.29:0/3342700525 >> 172.21.15.29:0/3342700525 conn(0x7f5ce80627b0 msgr2=0x7f5ce8062bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.406+0000 7f5cee0ad700 1 -- 172.21.15.29:0/3342700525 shutdown_connections 2024-01-24T13:59:01.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.406+0000 7f5cee0ad700 1 -- 172.21.15.29:0/3342700525 wait complete. 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.408+0000 7f5cee0ad700 1 Processor -- start 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.408+0000 7f5cee0ad700 1 -- start start 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.408+0000 7f5cee0ad700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ce80644d0 0x7f5ce80a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.408+0000 7f5cee0ad700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce8153c20 0x7f5ce80ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.408+0000 7f5cee0ad700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ce8154280 0x7f5ce81649f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.408+0000 7f5cee0ad700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce81508d0 con 0x7f5ce8154280 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.408+0000 7f5cee0ad700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce8164ee0 con 0x7f5ce8153c20 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.408+0000 7f5cee0ad700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce80aac50 con 0x7f5ce80644d0 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ce8154280 0x7f5ce81649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ce8154280 0x7f5ce81649f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33900/0 (socket says 172.21.15.29:33900) 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce7fff700 1 -- 172.21.15.29:0/1466561846 learned_addr learned my addr 172.21.15.29:0/1466561846 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce6ffd700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce8153c20 0x7f5ce80ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce77fe700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ce80644d0 0x7f5ce80a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce7fff700 1 -- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ce80644d0 msgr2=0x7f5ce80a9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce7fff700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ce80644d0 0x7f5ce80a9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce7fff700 1 -- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce8153c20 msgr2=0x7f5ce80ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce7fff700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce8153c20 0x7f5ce80ab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce7fff700 1 -- 172.21.15.29:0/1466561846 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce000ab30 con 0x7f5ce8154280 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce77fe700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ce80644d0 0x7f5ce80a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce6ffd700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce8153c20 0x7f5ce80ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ce7fff700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ce8154280 0x7f5ce81649f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f5ce000c770 tx=0x7f5ce80bcb00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ccf7fe700 1 -- 172.21.15.29:0/1466561846 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce000ce90 con 0x7f5ce8154280 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ccf7fe700 1 -- 172.21.15.29:0/1466561846 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce001d430 con 0x7f5ce8154280 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5ccf7fe700 1 -- 172.21.15.29:0/1466561846 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce001d810 con 0x7f5ce8154280 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5cee0ad700 1 -- 172.21.15.29:0/1466561846 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce80aa040 con 0x7f5ce8154280 2024-01-24T13:59:01.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.409+0000 7f5cee0ad700 1 -- 172.21.15.29:0/1466561846 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce80aa600 con 0x7f5ce8154280 2024-01-24T13:59:01.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.410+0000 7f5cee0ad700 1 -- 172.21.15.29:0/1466561846 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce80aadc0 con 0x7f5ce8154280 2024-01-24T13:59:01.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.411+0000 7f5ccf7fe700 1 -- 172.21.15.29:0/1466561846 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5ce001db80 con 0x7f5ce8154280 2024-01-24T13:59:01.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.411+0000 7f5ccf7fe700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5cd405f180 0x7f5cd4061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.411+0000 7f5ccf7fe700 1 -- 172.21.15.29:0/1466561846 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f5ce007c1d0 con 0x7f5ce8154280 2024-01-24T13:59:01.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.411+0000 7f5ce77fe700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5cd405f180 0x7f5cd4061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.412+0000 7f5ce77fe700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5cd405f180 0x7f5cd4061640 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f5cd8000c00 tx=0x7f5cd8009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:01.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.416+0000 7f5ccf7fe700 1 -- 172.21.15.29:0/1466561846 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5ce004d190 con 0x7f5ce8154280 2024-01-24T13:59:01.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.597+0000 7f5cee0ad700 1 -- 172.21.15.29:0/1466561846 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5ce80475e0 con 0x7f5ce8154280 2024-01-24T13:59:01.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.597+0000 7f5ccf7fe700 1 -- 172.21.15.29:0/1466561846 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5ce00507b0 con 0x7f5ce8154280 2024-01-24T13:59:01.597 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-01-24T13:59:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.599+0000 7f5cee0ad700 1 -- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5cd405f180 msgr2=0x7f5cd4061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.599+0000 7f5cee0ad700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5cd405f180 0x7f5cd4061640 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f5cd8000c00 tx=0x7f5cd8009040).stop 2024-01-24T13:59:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.599+0000 7f5cee0ad700 1 -- 172.21.15.29:0/1466561846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ce8154280 msgr2=0x7f5ce81649f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.599+0000 7f5cee0ad700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ce8154280 0x7f5ce81649f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f5ce000c770 tx=0x7f5ce80bcb00).stop 2024-01-24T13:59:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.599+0000 7f5cee0ad700 1 -- 172.21.15.29:0/1466561846 shutdown_connections 2024-01-24T13:59:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.599+0000 7f5cee0ad700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5cd405f180 0x7f5cd4061640 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.599+0000 7f5cee0ad700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ce8154280 0x7f5ce81649f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.599+0000 7f5cee0ad700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ce80644d0 0x7f5ce80a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.599+0000 7f5cee0ad700 1 --2- 172.21.15.29:0/1466561846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce8153c20 0x7f5ce80ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.599+0000 7f5cee0ad700 1 -- 172.21.15.29:0/1466561846 >> 172.21.15.29:0/1466561846 conn(0x7f5ce80627b0 msgr2=0x7f5ce8152700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.599+0000 7f5cee0ad700 1 -- 172.21.15.29:0/1466561846 shutdown_connections 2024-01-24T13:59:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.599+0000 7f5cee0ad700 1 -- 172.21.15.29:0/1466561846 wait complete. 2024-01-24T13:59:01.647 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2024-01-24T13:59:01.647 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-01-24T13:59:01.647 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2024-01-24T13:59:01.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.781+0000 7f904cb0c700 1 Processor -- start 2024-01-24T13:59:01.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.781+0000 7f904cb0c700 1 -- start start 2024-01-24T13:59:01.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.782+0000 7f904cb0c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f904814e5a0 0x7f90480a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.782+0000 7f904cb0c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9048068440 0x7f90480a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f904cb0c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f90480638c0 0x7f90480664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f904cb0c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9048068920 con 0x7f90480638c0 2024-01-24T13:59:01.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f904cb0c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90481551c0 con 0x7f9048068440 2024-01-24T13:59:01.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f904cb0c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90480a84a0 con 0x7f904814e5a0 2024-01-24T13:59:01.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f9046d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f90480638c0 0x7f90480664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f9046d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f90480638c0 0x7f90480664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33902/0 (socket says 172.21.15.29:33902) 2024-01-24T13:59:01.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f9046d9d700 1 -- 172.21.15.29:0/3801819732 learned_addr learned my addr 172.21.15.29:0/3801819732 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:01.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f9045d9b700 1 --2- 172.21.15.29:0/3801819732 >> v2:172.21.15.163:3300/0 conn(0x7f9048068440 0x7f90480a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f904659c700 1 --2- 172.21.15.29:0/3801819732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f904814e5a0 0x7f90480a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f9046d9d700 1 -- 172.21.15.29:0/3801819732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f904814e5a0 msgr2=0x7f90480a7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:01.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f9046d9d700 1 --2- 172.21.15.29:0/3801819732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f904814e5a0 0x7f90480a7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f9046d9d700 1 -- 172.21.15.29:0/3801819732 >> v2:172.21.15.163:3300/0 conn(0x7f9048068440 msgr2=0x7f90480a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f9046d9d700 1 --2- 172.21.15.29:0/3801819732 >> v2:172.21.15.163:3300/0 conn(0x7f9048068440 0x7f90480a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f9046d9d700 1 -- 172.21.15.29:0/3801819732 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90481545c0 con 0x7f90480638c0 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f9046d9d700 1 --2- 172.21.15.29:0/3801819732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90480638c0 0x7f90480664f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f90480c66c0 tx=0x7f9040009840).ready entity=mon.0 client_cookie=efbbea3e7533799c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f904559a700 1 -- 172.21.15.29:0/3801819732 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904000e070 con 0x7f90480638c0 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.783+0000 7f904559a700 1 -- 172.21.15.29:0/3801819732 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904000b930 con 0x7f90480638c0 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.784+0000 7f904559a700 1 -- 172.21.15.29:0/3801819732 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904000bcb0 con 0x7f90480638c0 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.784+0000 7f904cb0c700 1 -- 172.21.15.29:0/3801819732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90480638c0 msgr2=0x7f90480664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.784+0000 7f904cb0c700 1 --2- 172.21.15.29:0/3801819732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90480638c0 0x7f90480664f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f90480c66c0 tx=0x7f9040009840).stop 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.784+0000 7f904cb0c700 1 -- 172.21.15.29:0/3801819732 shutdown_connections 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.784+0000 7f904cb0c700 1 --2- 172.21.15.29:0/3801819732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f904814e5a0 0x7f90480a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.784+0000 7f904cb0c700 1 --2- 172.21.15.29:0/3801819732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90480638c0 0x7f90480664f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.784+0000 7f904cb0c700 1 --2- 172.21.15.29:0/3801819732 >> v2:172.21.15.163:3300/0 conn(0x7f9048068440 0x7f90480a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.784+0000 7f904cb0c700 1 -- 172.21.15.29:0/3801819732 >> 172.21.15.29:0/3801819732 conn(0x7f90480c8bb0 msgr2=0x7f90480c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:01.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.784+0000 7f904cb0c700 1 -- 172.21.15.29:0/3801819732 shutdown_connections 2024-01-24T13:59:01.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.784+0000 7f904cb0c700 1 -- 172.21.15.29:0/3801819732 wait complete. 2024-01-24T13:59:01.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.785+0000 7f904cb0c700 1 Processor -- start 2024-01-24T13:59:01.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.785+0000 7f904cb0c700 1 -- start start 2024-01-24T13:59:01.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.786+0000 7f904cb0c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90480638c0 0x7f90480a8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.786+0000 7f904cb0c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9048068440 0x7f9048150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.786+0000 7f904cb0c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f904814e5a0 0x7f904814d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.786+0000 7f904cb0c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90480a97b0 con 0x7f90480638c0 2024-01-24T13:59:01.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.786+0000 7f904cb0c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9048151280 con 0x7f904814e5a0 2024-01-24T13:59:01.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.786+0000 7f904cb0c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90480a8ba0 con 0x7f9048068440 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.786+0000 7f904659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90480638c0 0x7f90480a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.786+0000 7f904659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90480638c0 0x7f90480a8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33910/0 (socket says 172.21.15.29:33910) 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.786+0000 7f904659c700 1 -- 172.21.15.29:0/983614113 learned_addr learned my addr 172.21.15.29:0/983614113 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f9046d9d700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f904814e5a0 0x7f904814d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f9045d9b700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9048068440 0x7f9048150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f904659c700 1 -- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9048068440 msgr2=0x7f9048150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f904659c700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9048068440 0x7f9048150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f904659c700 1 -- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f904814e5a0 msgr2=0x7f904814d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f904659c700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f904814e5a0 0x7f904814d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f904659c700 1 -- 172.21.15.29:0/983614113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040009ae0 con 0x7f90480638c0 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f904659c700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90480638c0 0x7f90480a8600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f903800e900 tx=0x7f9038009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f90367fc700 1 -- 172.21.15.29:0/983614113 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903800c640 con 0x7f90480638c0 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f90367fc700 1 -- 172.21.15.29:0/983614113 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903800c7a0 con 0x7f90480638c0 2024-01-24T13:59:01.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f90367fc700 1 -- 172.21.15.29:0/983614113 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903800cb20 con 0x7f90480638c0 2024-01-24T13:59:01.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f904cb0c700 1 -- 172.21.15.29:0/983614113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90480a8e80 con 0x7f90480638c0 2024-01-24T13:59:01.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.787+0000 7f904cb0c700 1 -- 172.21.15.29:0/983614113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90480a9440 con 0x7f90480638c0 2024-01-24T13:59:01.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.788+0000 7f90367fc700 1 -- 172.21.15.29:0/983614113 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f903801b400 con 0x7f90480638c0 2024-01-24T13:59:01.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.788+0000 7f904cb0c700 1 -- 172.21.15.29:0/983614113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90480475e0 con 0x7f90480638c0 2024-01-24T13:59:01.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.789+0000 7f90367fc700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f902805f120 0x7f90280615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:01.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.789+0000 7f90367fc700 1 -- 172.21.15.29:0/983614113 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f903807a910 con 0x7f90480638c0 2024-01-24T13:59:01.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.789+0000 7f9045d9b700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f902805f120 0x7f90280615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:01.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.790+0000 7f9045d9b700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f902805f120 0x7f90280615e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f90480c81a0 tx=0x7f903c008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:01.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:01.793+0000 7f90367fc700 1 -- 172.21.15.29:0/983614113 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f903804b870 con 0x7f90480638c0 2024-01-24T13:59:02.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.045+0000 7f904cb0c700 1 -- 172.21.15.29:0/983614113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f9048068d50 con 0x7f90480638c0 2024-01-24T13:59:02.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.816+0000 7f90367fc700 1 -- 172.21.15.29:0/983614113 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v61) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9038016070 con 0x7f90480638c0 2024-01-24T13:59:02.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.819+0000 7f904cb0c700 1 -- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f902805f120 msgr2=0x7f90280615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:02.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.819+0000 7f904cb0c700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f902805f120 0x7f90280615e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f90480c81a0 tx=0x7f903c008040).stop 2024-01-24T13:59:02.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.819+0000 7f904cb0c700 1 -- 172.21.15.29:0/983614113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90480638c0 msgr2=0x7f90480a8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:02.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.819+0000 7f904cb0c700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90480638c0 0x7f90480a8600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f903800e900 tx=0x7f9038009060).stop 2024-01-24T13:59:02.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.819+0000 7f904cb0c700 1 -- 172.21.15.29:0/983614113 shutdown_connections 2024-01-24T13:59:02.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.819+0000 7f904cb0c700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f904814e5a0 0x7f904814d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:02.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.819+0000 7f904cb0c700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f902805f120 0x7f90280615e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:02.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.819+0000 7f904cb0c700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90480638c0 0x7f90480a8600 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:02.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.819+0000 7f904cb0c700 1 --2- 172.21.15.29:0/983614113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9048068440 0x7f9048150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:02.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.819+0000 7f904cb0c700 1 -- 172.21.15.29:0/983614113 >> 172.21.15.29:0/983614113 conn(0x7f90480c8bb0 msgr2=0x7f90481522f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:02.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.819+0000 7f904cb0c700 1 -- 172.21.15.29:0/983614113 shutdown_connections 2024-01-24T13:59:02.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.819+0000 7f904cb0c700 1 -- 172.21.15.29:0/983614113 wait complete. 2024-01-24T13:59:02.833 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-24T13:59:02.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.970+0000 7eff6e26e700 1 Processor -- start 2024-01-24T13:59:02.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.970+0000 7eff6e26e700 1 -- start start 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff6e26e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff680692a0 0x7eff68065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff6e26e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7eff68069d90 0x7eff6815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff6e26e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff680c3e70 con 0x7eff68066370 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff6e26e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff680c4300 con 0x7eff68069d90 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff6e26e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff680c3260 con 0x7eff680692a0 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff6c80b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7eff68069d90 0x7eff6815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff6c80b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7eff68069d90 0x7eff6815a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33926/0 (socket says 172.21.15.29:33926) 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff6c80b700 1 -- 172.21.15.29:0/2789792712 learned_addr learned my addr 172.21.15.29:0/2789792712 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff67fff700 1 --2- 172.21.15.29:0/2789792712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff680692a0 0x7eff68065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff6c80b700 1 -- 172.21.15.29:0/2789792712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff680692a0 msgr2=0x7eff68065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff6c80b700 1 --2- 172.21.15.29:0/2789792712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff680692a0 0x7eff68065e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff6c80b700 1 -- 172.21.15.29:0/2789792712 >> v1:172.21.15.163:6789/0 conn(0x7eff68066370 legacy=0x7eff68069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:02.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.971+0000 7eff6c80b700 1 -- 172.21.15.29:0/2789792712 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680c3540 con 0x7eff68069d90 2024-01-24T13:59:02.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.972+0000 7eff6c80b700 1 --2- 172.21.15.29:0/2789792712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff68069d90 0x7eff6815a9f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7eff60009f20 tx=0x7eff60009840).ready entity=mon.0 client_cookie=9005c6dd967133cd server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:02.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.972+0000 7eff66ffd700 1 -- 172.21.15.29:0/2789792712 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6000e070 con 0x7eff68069d90 2024-01-24T13:59:02.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.972+0000 7eff66ffd700 1 -- 172.21.15.29:0/2789792712 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6000ba80 con 0x7eff68069d90 2024-01-24T13:59:02.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.972+0000 7eff66ffd700 1 -- 172.21.15.29:0/2789792712 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6000be00 con 0x7eff68069d90 2024-01-24T13:59:02.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.972+0000 7eff6e26e700 1 -- 172.21.15.29:0/2789792712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff68069d90 msgr2=0x7eff6815a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:02.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.972+0000 7eff6e26e700 1 --2- 172.21.15.29:0/2789792712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff68069d90 0x7eff6815a9f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7eff60009f20 tx=0x7eff60009840).stop 2024-01-24T13:59:02.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.972+0000 7eff6e26e700 1 -- 172.21.15.29:0/2789792712 shutdown_connections 2024-01-24T13:59:02.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.972+0000 7eff6e26e700 1 --2- 172.21.15.29:0/2789792712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff680692a0 0x7eff68065e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:02.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.972+0000 7eff6e26e700 1 --2- 172.21.15.29:0/2789792712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff68069d90 0x7eff6815a9f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:02.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.972+0000 7eff6e26e700 1 -- 172.21.15.29:0/2789792712 >> 172.21.15.29:0/2789792712 conn(0x7eff68064390 msgr2=0x7eff680c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:02.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.972+0000 7eff6e26e700 1 -- 172.21.15.29:0/2789792712 shutdown_connections 2024-01-24T13:59:02.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.972+0000 7eff6e26e700 1 -- 172.21.15.29:0/2789792712 wait complete. 2024-01-24T13:59:02.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.974+0000 7eff6e26e700 1 Processor -- start 2024-01-24T13:59:02.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.974+0000 7eff6e26e700 1 -- start start 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6e26e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff68066370 0x7eff68146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6e26e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff680692a0 0x7eff681478f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6e26e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff68069d90 0x7eff680bfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6e26e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff68132fc0 con 0x7eff68069d90 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6e26e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff68147de0 con 0x7eff68066370 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6e26e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff680c1000 con 0x7eff680692a0 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6c80b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff68069d90 0x7eff680bfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6c80b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff68069d90 0x7eff680bfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33942/0 (socket says 172.21.15.29:33942) 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6c80b700 1 -- 172.21.15.29:0/1495630011 learned_addr learned my addr 172.21.15.29:0/1495630011 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff67fff700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff68066370 0x7eff68146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff677fe700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff680692a0 0x7eff681478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6c80b700 1 -- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff680692a0 msgr2=0x7eff681478f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6c80b700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff680692a0 0x7eff681478f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6c80b700 1 -- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff68066370 msgr2=0x7eff68146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6c80b700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff68066370 0x7eff68146b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:02.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6c80b700 1 -- 172.21.15.29:0/1495630011 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff60009ae0 con 0x7eff68069d90 2024-01-24T13:59:02.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff677fe700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff680692a0 0x7eff681478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:02.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.975+0000 7eff6c80b700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff68069d90 0x7eff680bfb60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7eff60013d30 tx=0x7eff60009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:02.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.976+0000 7eff5ffff700 1 -- 172.21.15.29:0/1495630011 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600092e0 con 0x7eff68069d90 2024-01-24T13:59:02.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.976+0000 7eff5ffff700 1 -- 172.21.15.29:0/1495630011 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff60007610 con 0x7eff68069d90 2024-01-24T13:59:02.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.976+0000 7eff6e26e700 1 -- 172.21.15.29:0/1495630011 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff680c1490 con 0x7eff68069d90 2024-01-24T13:59:02.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.976+0000 7eff5ffff700 1 -- 172.21.15.29:0/1495630011 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600079d0 con 0x7eff68069d90 2024-01-24T13:59:02.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.976+0000 7eff6e26e700 1 -- 172.21.15.29:0/1495630011 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff680c06d0 con 0x7eff68069d90 2024-01-24T13:59:02.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.977+0000 7eff6e26e700 1 -- 172.21.15.29:0/1495630011 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff680475e0 con 0x7eff68069d90 2024-01-24T13:59:02.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.977+0000 7eff5ffff700 1 -- 172.21.15.29:0/1495630011 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7eff60003040 con 0x7eff68069d90 2024-01-24T13:59:02.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.977+0000 7eff5ffff700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff4805f120 0x7eff480615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:02.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.977+0000 7eff5ffff700 1 -- 172.21.15.29:0/1495630011 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7eff6007bfe0 con 0x7eff68069d90 2024-01-24T13:59:02.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.978+0000 7eff67fff700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff4805f120 0x7eff480615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:02.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.978+0000 7eff67fff700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff4805f120 0x7eff480615e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7eff5800bbc0 tx=0x7eff58008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:02.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:02.982+0000 7eff5ffff700 1 -- 172.21.15.29:0/1495630011 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7eff6004d050 con 0x7eff68069d90 2024-01-24T13:59:03.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.163+0000 7eff6e26e700 1 -- 172.21.15.29:0/1495630011 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7eff680c1140 con 0x7eff68069d90 2024-01-24T13:59:03.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.167+0000 7eff5ffff700 1 -- 172.21.15.29:0/1495630011 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7eff60024030 con 0x7eff68069d90 2024-01-24T13:59:03.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.169+0000 7eff6e26e700 1 -- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff4805f120 msgr2=0x7eff480615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:03.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.169+0000 7eff6e26e700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff4805f120 0x7eff480615e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7eff5800bbc0 tx=0x7eff58008040).stop 2024-01-24T13:59:03.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.169+0000 7eff6e26e700 1 -- 172.21.15.29:0/1495630011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff68069d90 msgr2=0x7eff680bfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:03.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.169+0000 7eff6e26e700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff68069d90 0x7eff680bfb60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7eff60013d30 tx=0x7eff60009840).stop 2024-01-24T13:59:03.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.169+0000 7eff6e26e700 1 -- 172.21.15.29:0/1495630011 shutdown_connections 2024-01-24T13:59:03.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.169+0000 7eff6e26e700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff4805f120 0x7eff480615e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:03.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.169+0000 7eff6e26e700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff68069d90 0x7eff680bfb60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:03.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.169+0000 7eff6e26e700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff68066370 0x7eff68146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:03.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.169+0000 7eff6e26e700 1 --2- 172.21.15.29:0/1495630011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff680692a0 0x7eff681478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:03.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.169+0000 7eff6e26e700 1 -- 172.21.15.29:0/1495630011 >> 172.21.15.29:0/1495630011 conn(0x7eff68064390 msgr2=0x7eff680c44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:03.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.169+0000 7eff6e26e700 1 -- 172.21.15.29:0/1495630011 shutdown_connections 2024-01-24T13:59:03.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.169+0000 7eff6e26e700 1 -- 172.21.15.29:0/1495630011 wait complete. 2024-01-24T13:59:03.183 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T13:59:03.183 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T13:59:03.225 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T13:59:03.283 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T13:59:03.283 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-24T13:59:03.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:03.284 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-01-24T13:59:03.325 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T13:59:03.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:59:03.325 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-24T13:59:03.387 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-24T13:59:03.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:03.387 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-01-24T13:59:03.434 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T13:59:03.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:59:03.434 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-24T13:59:03.460 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-01-24T13:59:03.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:03.460 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-01-24T13:59:03.502 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T13:59:03.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:59:03.502 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-24T13:59:03.564 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-01-24T13:59:03.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:03.564 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-01-24T13:59:03.602 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T13:59:03.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:59:03.602 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-24T13:59:03.661 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-24T13:59:03.662 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:59:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.800+0000 7f47c3215700 1 Processor -- start 2024-01-24T13:59:03.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.800+0000 7f47c3215700 1 -- start start 2024-01-24T13:59:03.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c3215700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f47bc0c4a10 0x7f47bc0bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c3215700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47bc066c00 0x7f47bc158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c3215700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47bc1582b0 con 0x7f47bc0670a0 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c3215700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47bc157ca0 con 0x7f47bc0c4a10 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c3215700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc063c60 con 0x7f47bc066c00 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c0fb1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f47bc0c4a10 0x7f47bc0bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47bbfff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f47bc0670a0 0x7f47bc0664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59542/0 (socket says 172.21.15.29:59542) 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c0fb1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f47bc0c4a10 0x7f47bc0bf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44902/0 (socket says 172.21.15.29:44902) 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47bbfff700 1 -- 172.21.15.29:0/660617003 learned_addr learned my addr 172.21.15.29:0/660617003 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c17b2700 1 --2- 172.21.15.29:0/660617003 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47bc066c00 0x7f47bc158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c0fb1700 1 -- 172.21.15.29:0/660617003 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47bc066c00 msgr2=0x7f47bc158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c0fb1700 1 --2- 172.21.15.29:0/660617003 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47bc066c00 0x7f47bc158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c0fb1700 1 -- 172.21.15.29:0/660617003 >> v1:172.21.15.29:6789/0 conn(0x7f47bc0670a0 legacy=0x7f47bc0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:03.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47bb7fe700 1 -- 172.21.15.29:0/660617003 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4206064285 0 0) 0x7f47bc1582b0 con 0x7f47bc0670a0 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c0fb1700 1 -- 172.21.15.29:0/660617003 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc157f40 con 0x7f47bc0c4a10 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.801+0000 7f47c0fb1700 1 --2- 172.21.15.29:0/660617003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47bc0c4a10 0x7f47bc0bf6d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f47b0009760 tx=0x7f47b0009060).ready entity=mon.0 client_cookie=fb07ca458cc86cc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.802+0000 7f47bb7fe700 1 -- 172.21.15.29:0/660617003 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000f510 con 0x7f47bc0c4a10 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.802+0000 7f47bb7fe700 1 -- 172.21.15.29:0/660617003 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b000f670 con 0x7f47bc0c4a10 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.802+0000 7f47bb7fe700 1 -- 172.21.15.29:0/660617003 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000f9f0 con 0x7f47bc0c4a10 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.802+0000 7f47c3215700 1 -- 172.21.15.29:0/660617003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47bc0c4a10 msgr2=0x7f47bc0bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.802+0000 7f47c3215700 1 --2- 172.21.15.29:0/660617003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47bc0c4a10 0x7f47bc0bf6d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f47b0009760 tx=0x7f47b0009060).stop 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.802+0000 7f47c3215700 1 -- 172.21.15.29:0/660617003 shutdown_connections 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.802+0000 7f47c3215700 1 --2- 172.21.15.29:0/660617003 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47bc066c00 0x7f47bc158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.802+0000 7f47c3215700 1 --2- 172.21.15.29:0/660617003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47bc0c4a10 0x7f47bc0bf6d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.802+0000 7f47c3215700 1 -- 172.21.15.29:0/660617003 >> 172.21.15.29:0/660617003 conn(0x7f47bc0c8bb0 msgr2=0x7f47bc0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.802+0000 7f47c3215700 1 -- 172.21.15.29:0/660617003 shutdown_connections 2024-01-24T13:59:03.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.802+0000 7f47c3215700 1 -- 172.21.15.29:0/660617003 wait complete. 2024-01-24T13:59:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.803+0000 7f47c3215700 1 Processor -- start 2024-01-24T13:59:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.804+0000 7f47c3215700 1 -- start start 2024-01-24T13:59:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.804+0000 7f47c3215700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47bc066c00 0x7f47bc1525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.804+0000 7f47c3215700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc0670a0 0x7f47bc164b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.804+0000 7f47c3215700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47bc0c4a10 0x7f47bc165040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.804+0000 7f47c3215700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc0c14e0 con 0x7f47bc066c00 2024-01-24T13:59:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.804+0000 7f47c3215700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc0c3a30 con 0x7f47bc0670a0 2024-01-24T13:59:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.804+0000 7f47c3215700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc0c0e90 con 0x7f47bc0c4a10 2024-01-24T13:59:03.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47c0fb1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47bc066c00 0x7f47bc1525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47c0fb1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47bc066c00 0x7f47bc1525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44912/0 (socket says 172.21.15.29:44912) 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47c0fb1700 1 -- 172.21.15.29:0/1907226588 learned_addr learned my addr 172.21.15.29:0/1907226588 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47c17b2700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47bc0c4a10 0x7f47bc165040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47bbfff700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc0670a0 0x7f47bc164b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47c0fb1700 1 -- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47bc0c4a10 msgr2=0x7f47bc165040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47c0fb1700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47bc0c4a10 0x7f47bc165040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47c0fb1700 1 -- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc0670a0 msgr2=0x7f47bc164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47c0fb1700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc0670a0 0x7f47bc164b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47c0fb1700 1 -- 172.21.15.29:0/1907226588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b00092a0 con 0x7f47bc066c00 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47bbfff700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc0670a0 0x7f47bc164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47c0fb1700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47bc066c00 0x7f47bc1525e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f47b0009e00 tx=0x7f47b0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47b8ff9700 1 -- 172.21.15.29:0/1907226588 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000f7e0 con 0x7f47bc066c00 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47b8ff9700 1 -- 172.21.15.29:0/1907226588 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b000f940 con 0x7f47bc066c00 2024-01-24T13:59:03.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47c3215700 1 -- 172.21.15.29:0/1907226588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47bc0c1170 con 0x7f47bc066c00 2024-01-24T13:59:03.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47b8ff9700 1 -- 172.21.15.29:0/1907226588 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b001b660 con 0x7f47bc066c00 2024-01-24T13:59:03.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.805+0000 7f47c3215700 1 -- 172.21.15.29:0/1907226588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47bc152d10 con 0x7f47bc066c00 2024-01-24T13:59:03.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.806+0000 7f47c3215700 1 -- 172.21.15.29:0/1907226588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47bc0c0cf0 con 0x7f47bc066c00 2024-01-24T13:59:03.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.807+0000 7f47b8ff9700 1 -- 172.21.15.29:0/1907226588 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f47b001ba90 con 0x7f47bc066c00 2024-01-24T13:59:03.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.807+0000 7f47b8ff9700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f47a005f150 0x7f47a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:03.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.807+0000 7f47b8ff9700 1 -- 172.21.15.29:0/1907226588 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f47b007be10 con 0x7f47bc066c00 2024-01-24T13:59:03.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.808+0000 7f47bbfff700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f47a005f150 0x7f47a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:03.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.808+0000 7f47bbfff700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f47a005f150 0x7f47a0061610 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f47a8007fd0 tx=0x7f47a8009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:03.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:03.812+0000 7f47b8ff9700 1 -- 172.21.15.29:0/1907226588 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f47bc0c0cf0 con 0x7f47bc066c00 2024-01-24T13:59:04.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.038+0000 7f47c3215700 1 -- 172.21.15.29:0/1907226588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47bc0c0cf0 con 0x7f47bc066c00 2024-01-24T13:59:04.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.039+0000 7f47b8ff9700 1 -- 172.21.15.29:0/1907226588 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4491 (secure 0 0 0) 0x7f47bc0c0cf0 con 0x7f47bc066c00 2024-01-24T13:59:04.039 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:04.040 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2038321939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2038321939},{"type":"v1","addr":"172.21.15.163:6836","nonce":2038321939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3675672874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3675672874},{"type":"v1","addr":"172.21.15.163:6837","nonce":3675672874}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1532435957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532435957},{"type":"v1","addr":"172.21.15.29:6834","nonce":1532435957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:58:19.190134+0000","modified":"2024-01-24T13:58:31.231527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1328130656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1328130656},{"type":"v1","addr":"172.21.15.29:6835","nonce":1328130656}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-24T13:59:04.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.041+0000 7f47c3215700 1 -- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f47a005f150 msgr2=0x7f47a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.041+0000 7f47c3215700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f47a005f150 0x7f47a0061610 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f47a8007fd0 tx=0x7f47a8009040).stop 2024-01-24T13:59:04.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.041+0000 7f47c3215700 1 -- 172.21.15.29:0/1907226588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47bc066c00 msgr2=0x7f47bc1525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.041+0000 7f47c3215700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47bc066c00 0x7f47bc1525e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f47b0009e00 tx=0x7f47b0017000).stop 2024-01-24T13:59:04.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.041+0000 7f47c3215700 1 -- 172.21.15.29:0/1907226588 shutdown_connections 2024-01-24T13:59:04.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.041+0000 7f47c3215700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47bc0c4a10 0x7f47bc165040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.041+0000 7f47c3215700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f47a005f150 0x7f47a0061610 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.041+0000 7f47c3215700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47bc066c00 0x7f47bc1525e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.041+0000 7f47c3215700 1 --2- 172.21.15.29:0/1907226588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc0670a0 0x7f47bc164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.041+0000 7f47c3215700 1 -- 172.21.15.29:0/1907226588 >> 172.21.15.29:0/1907226588 conn(0x7f47bc0c8bb0 msgr2=0x7f47bc153570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:04.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.041+0000 7f47c3215700 1 -- 172.21.15.29:0/1907226588 shutdown_connections 2024-01-24T13:59:04.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.041+0000 7f47c3215700 1 -- 172.21.15.29:0/1907226588 wait complete. 2024-01-24T13:59:04.043 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 9 2024-01-24T13:59:04.055 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:59:04.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.186+0000 7fd0c9198700 1 Processor -- start 2024-01-24T13:59:04.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.186+0000 7fd0c9198700 1 -- start start 2024-01-24T13:59:04.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.187+0000 7fd0c9198700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c4155040 0x7fd0c4069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.187+0000 7fd0c9198700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c40c2ce0 con 0x7fd0c4065ec0 2024-01-24T13:59:04.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.187+0000 7fd0c9198700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c4066370 con 0x7fd0c4150d40 2024-01-24T13:59:04.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.187+0000 7fd0c9198700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c4069890 con 0x7fd0c4155040 2024-01-24T13:59:04.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.187+0000 7fd0c359e700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd0c4065ec0 0x7fd0c40c3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59546/0 (socket says 172.21.15.29:59546) 2024-01-24T13:59:04.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.187+0000 7fd0c359e700 1 -- 172.21.15.29:0/873426264 learned_addr learned my addr 172.21.15.29:0/873426264 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.187+0000 7fd0c259c700 1 --2- 172.21.15.29:0/873426264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c4155040 0x7fd0c4069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.187+0000 7fd0c1d9b700 1 -- 172.21.15.29:0/873426264 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646339356 0 0) 0x7fd0c40c2ce0 con 0x7fd0c4065ec0 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.187+0000 7fd0c1d9b700 1 -- 172.21.15.29:0/873426264 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b00033f0 con 0x7fd0c4065ec0 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.188+0000 7fd0c1d9b700 1 -- 172.21.15.29:0/873426264 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711821435 0 0) 0x7fd0c4066370 con 0x7fd0c4150d40 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.188+0000 7fd0c1d9b700 1 -- 172.21.15.29:0/873426264 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0c40c2ce0 con 0x7fd0c4150d40 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.188+0000 7fd0c1d9b700 1 -- 172.21.15.29:0/873426264 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3366827208 0 0) 0x7fd0b00033f0 con 0x7fd0c4065ec0 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.188+0000 7fd0c259c700 1 -- 172.21.15.29:0/873426264 >> v1:172.21.15.163:6789/0 conn(0x7fd0c4150d40 legacy=0x7fd0c4154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.188+0000 7fd0c259c700 1 -- 172.21.15.29:0/873426264 >> v1:172.21.15.29:6789/0 conn(0x7fd0c4065ec0 legacy=0x7fd0c40c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.188+0000 7fd0c259c700 1 -- 172.21.15.29:0/873426264 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c4069b70 con 0x7fd0c4155040 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.188+0000 7fd0c259c700 1 --2- 172.21.15.29:0/873426264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c4155040 0x7fd0c4069240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fd0b4009790 tx=0x7fd0b4009060).ready entity=mon.2 client_cookie=d56e67e73118ead6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.188+0000 7fd0c1d9b700 1 -- 172.21.15.29:0/873426264 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0017f0 con 0x7fd0c4155040 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.188+0000 7fd0c1d9b700 1 -- 172.21.15.29:0/873426264 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4011400 con 0x7fd0c4155040 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.188+0000 7fd0c1d9b700 1 -- 172.21.15.29:0/873426264 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4011780 con 0x7fd0c4155040 2024-01-24T13:59:04.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.189+0000 7fd0c9198700 1 -- 172.21.15.29:0/873426264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c4155040 msgr2=0x7fd0c4069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.189+0000 7fd0c9198700 1 --2- 172.21.15.29:0/873426264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c4155040 0x7fd0c4069240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fd0b4009790 tx=0x7fd0b4009060).stop 2024-01-24T13:59:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.189+0000 7fd0c9198700 1 -- 172.21.15.29:0/873426264 shutdown_connections 2024-01-24T13:59:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.189+0000 7fd0c9198700 1 --2- 172.21.15.29:0/873426264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c4155040 0x7fd0c4069240 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.189+0000 7fd0c9198700 1 -- 172.21.15.29:0/873426264 >> 172.21.15.29:0/873426264 conn(0x7fd0c40c8580 msgr2=0x7fd0c40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.189+0000 7fd0c9198700 1 -- 172.21.15.29:0/873426264 shutdown_connections 2024-01-24T13:59:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.189+0000 7fd0c9198700 1 -- 172.21.15.29:0/873426264 wait complete. 2024-01-24T13:59:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.190+0000 7fd0c9198700 1 Processor -- start 2024-01-24T13:59:04.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.191+0000 7fd0c9198700 1 -- start start 2024-01-24T13:59:04.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c9198700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0c4065ec0 0x7fd0c40c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c9198700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0c4150d40 0x7fd0c40c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c9198700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c4155040 0x7fd0c4165a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c9198700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c40c7490 con 0x7fd0c4150d40 2024-01-24T13:59:04.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c9198700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c414d220 con 0x7fd0c4065ec0 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c9198700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c4162de0 con 0x7fd0c4155040 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0c4150d40 0x7fd0c40c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0c4150d40 0x7fd0c40c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44922/0 (socket says 172.21.15.29:44922) 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c259c700 1 -- 172.21.15.29:0/3345583173 learned_addr learned my addr 172.21.15.29:0/3345583173 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c359e700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c4155040 0x7fd0c4165a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c2d9d700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0c4065ec0 0x7fd0c40c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c259c700 1 -- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c4155040 msgr2=0x7fd0c4165a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c259c700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c4155040 0x7fd0c4165a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c259c700 1 -- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0c4065ec0 msgr2=0x7fd0c40c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c259c700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0c4065ec0 0x7fd0c40c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c259c700 1 -- 172.21.15.29:0/3345583173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b40092a0 con 0x7fd0c4150d40 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0c259c700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0c4150d40 0x7fd0c40c6fa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fd0b4009d80 tx=0x7fd0b4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.192+0000 7fd0baffd700 1 -- 172.21.15.29:0/3345583173 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b40107f0 con 0x7fd0c4150d40 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.193+0000 7fd0baffd700 1 -- 172.21.15.29:0/3345583173 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4010950 con 0x7fd0c4150d40 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.193+0000 7fd0c9198700 1 -- 172.21.15.29:0/3345583173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c41621d0 con 0x7fd0c4150d40 2024-01-24T13:59:04.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.193+0000 7fd0baffd700 1 -- 172.21.15.29:0/3345583173 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4010cf0 con 0x7fd0c4150d40 2024-01-24T13:59:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.193+0000 7fd0c9198700 1 -- 172.21.15.29:0/3345583173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c4162790 con 0x7fd0c4150d40 2024-01-24T13:59:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.194+0000 7fd0c9198700 1 -- 172.21.15.29:0/3345583173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c40475e0 con 0x7fd0c4150d40 2024-01-24T13:59:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.194+0000 7fd0baffd700 1 -- 172.21.15.29:0/3345583173 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd0b40263b0 con 0x7fd0c4150d40 2024-01-24T13:59:04.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.195+0000 7fd0baffd700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd0a405f120 0x7fd0a40615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.195+0000 7fd0baffd700 1 -- 172.21.15.29:0/3345583173 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fd0b407b200 con 0x7fd0c4150d40 2024-01-24T13:59:04.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.195+0000 7fd0c2d9d700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd0a405f120 0x7fd0a40615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.195+0000 7fd0c2d9d700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd0a405f120 0x7fd0a40615e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fd0ac007fd0 tx=0x7fd0ac007bd0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.199+0000 7fd0baffd700 1 -- 172.21.15.29:0/3345583173 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd0b404c160 con 0x7fd0c4150d40 2024-01-24T13:59:04.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.385+0000 7fd0c9198700 1 -- 172.21.15.29:0/3345583173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0c4162f20 con 0x7fd0c4150d40 2024-01-24T13:59:04.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.386+0000 7fd0baffd700 1 -- 172.21.15.29:0/3345583173 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18800 (secure 0 0 0) 0x7fd0b4022020 con 0x7fd0c4150d40 2024-01-24T13:59:04.386 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:04.386 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:58:23.623570+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:59:04.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.388+0000 7fd0c9198700 1 -- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd0a405f120 msgr2=0x7fd0a40615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.388+0000 7fd0c9198700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd0a405f120 0x7fd0a40615e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fd0ac007fd0 tx=0x7fd0ac007bd0).stop 2024-01-24T13:59:04.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.388+0000 7fd0c9198700 1 -- 172.21.15.29:0/3345583173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0c4150d40 msgr2=0x7fd0c40c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.388+0000 7fd0c9198700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0c4150d40 0x7fd0c40c6fa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fd0b4009d80 tx=0x7fd0b4016000).stop 2024-01-24T13:59:04.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.388+0000 7fd0c9198700 1 -- 172.21.15.29:0/3345583173 shutdown_connections 2024-01-24T13:59:04.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.388+0000 7fd0c9198700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c4155040 0x7fd0c4165a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.388+0000 7fd0c9198700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0c4065ec0 0x7fd0c40c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.388+0000 7fd0c9198700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd0a405f120 0x7fd0a40615e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.388+0000 7fd0c9198700 1 --2- 172.21.15.29:0/3345583173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0c4150d40 0x7fd0c40c6fa0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.388+0000 7fd0c9198700 1 -- 172.21.15.29:0/3345583173 >> 172.21.15.29:0/3345583173 conn(0x7fd0c40c8580 msgr2=0x7fd0c414ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:04.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.388+0000 7fd0c9198700 1 -- 172.21.15.29:0/3345583173 shutdown_connections 2024-01-24T13:59:04.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.388+0000 7fd0c9198700 1 -- 172.21.15.29:0/3345583173 wait complete. 2024-01-24T13:59:04.402 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T13:58:16.949817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T13:58:18.174460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-24T13:58:21.232882+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T13:59:04.402 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T13:59:04.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.534+0000 7f4747795700 1 Processor -- start 2024-01-24T13:59:04.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.535+0000 7f4747795700 1 -- start start 2024-01-24T13:59:04.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.535+0000 7f4747795700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f47400664c0 0x7f4740066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4747795700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400bdb30 0x7f47400bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4747795700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47400686a0 con 0x7f4740134ea0 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4747795700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4740066e30 con 0x7f47400664c0 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4747795700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47400c38e0 con 0x7f47400bdb30 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4745531700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4740134ea0 0x7f47400c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59560/0 (socket says 172.21.15.29:59560) 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4745531700 1 -- 172.21.15.29:0/1209664943 learned_addr learned my addr 172.21.15.29:0/1209664943 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4744d30700 1 --2- 172.21.15.29:0/1209664943 >> v2:172.21.15.163:3300/0 conn(0x7f47400664c0 0x7f4740066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4745d32700 1 --2- 172.21.15.29:0/1209664943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400bdb30 0x7f47400bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163888555 0 0) 0x7f47400686a0 con 0x7f4740134ea0 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f472c0033f0 con 0x7f4740134ea0 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1160428683 0 0) 0x7f472c0033f0 con 0x7f4740134ea0 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400bdb30 msgr2=0x7f47400bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 --2- 172.21.15.29:0/1209664943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400bdb30 0x7f47400bdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 >> v2:172.21.15.163:3300/0 conn(0x7f47400664c0 msgr2=0x7f4740066940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 --2- 172.21.15.29:0/1209664943 >> v2:172.21.15.163:3300/0 conn(0x7f47400664c0 0x7f4740066940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4740068b30 con 0x7f4740134ea0 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f4730001470 con 0x7f4740134ea0 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 >> v1:172.21.15.29:6789/0 conn(0x7f4740134ea0 legacy=0x7f47400c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 --2- 172.21.15.29:0/1209664943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400bdb30 0x7f472c0038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 --2- 172.21.15.29:0/1209664943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f472c003e10 0x7f472c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 --2- 172.21.15.29:0/1209664943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f472c004b80 0x7f472c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4740066e30 con 0x7f472c003e10 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.536+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47400c38e0 con 0x7f472c004b80 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472c00b8e0 con 0x7f47400bdb30 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4745531700 1 --2- 172.21.15.29:0/1209664943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400bdb30 0x7f472c0038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4744d30700 1 --2- 172.21.15.29:0/1209664943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f472c003e10 0x7f472c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4745d32700 1 --2- 172.21.15.29:0/1209664943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f472c004b80 0x7f472c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4744d30700 1 -- 172.21.15.29:0/1209664943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400bdb30 msgr2=0x7f472c0038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4744d30700 1 --2- 172.21.15.29:0/1209664943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400bdb30 0x7f472c0038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4744d30700 1 -- 172.21.15.29:0/1209664943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f472c004b80 msgr2=0x7f472c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4744d30700 1 --2- 172.21.15.29:0/1209664943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f472c004b80 0x7f472c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4744d30700 1 -- 172.21.15.29:0/1209664943 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c0033f0 con 0x7f472c003e10 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4744d30700 1 --2- 172.21.15.29:0/1209664943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f472c003e10 0x7f472c004690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f473c009ab0 tx=0x7f473c009060).ready entity=mon.0 client_cookie=af5663d3e4111853 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c009430 con 0x7f472c003e10 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c00e640 con 0x7f472c003e10 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.537+0000 7f4737fff700 1 -- 172.21.15.29:0/1209664943 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c00e9c0 con 0x7f472c003e10 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.538+0000 7f4747795700 1 -- 172.21.15.29:0/1209664943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f472c003e10 msgr2=0x7f472c004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.538+0000 7f4747795700 1 --2- 172.21.15.29:0/1209664943 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f472c003e10 0x7f472c004690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f473c009ab0 tx=0x7f473c009060).stop 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.538+0000 7f4745d32700 1 -- 172.21.15.29:0/1209664943 reap_dead start 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.538+0000 7f4747795700 1 -- 172.21.15.29:0/1209664943 shutdown_connections 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.538+0000 7f4747795700 1 -- 172.21.15.29:0/1209664943 >> 172.21.15.29:0/1209664943 conn(0x7f47400c8bd0 msgr2=0x7f47400c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.538+0000 7f4747795700 1 -- 172.21.15.29:0/1209664943 shutdown_connections 2024-01-24T13:59:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.538+0000 7f4747795700 1 -- 172.21.15.29:0/1209664943 wait complete. 2024-01-24T13:59:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.539+0000 7f4747795700 1 Processor -- start 2024-01-24T13:59:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.539+0000 7f4747795700 1 -- start start 2024-01-24T13:59:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.540+0000 7f4747795700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f472c003e10 0x7f47400ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.540+0000 7f4747795700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400ac140 0x7f4740164d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.540+0000 7f4747795700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4740163190 0x7f4740165650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.540+0000 7f4747795700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47400a9a00 con 0x7f472c003e10 2024-01-24T13:59:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.540+0000 7f4747795700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47400a9060 con 0x7f4740163190 2024-01-24T13:59:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.540+0000 7f4747795700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47400a9340 con 0x7f47400ac140 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.540+0000 7f4745531700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f472c003e10 0x7f47400ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.540+0000 7f4745531700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f472c003e10 0x7f47400ae400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44928/0 (socket says 172.21.15.29:44928) 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.540+0000 7f4745531700 1 -- 172.21.15.29:0/1582127523 learned_addr learned my addr 172.21.15.29:0/1582127523 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.540+0000 7f4744d30700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400ac140 0x7f4740164d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.541+0000 7f4745d32700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4740163190 0x7f4740165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.541+0000 7f4745531700 1 -- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400ac140 msgr2=0x7f4740164d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.541+0000 7f4745531700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400ac140 0x7f4740164d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.541+0000 7f4745531700 1 -- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4740163190 msgr2=0x7f4740165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.541+0000 7f4745531700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4740163190 0x7f4740165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.541+0000 7f4745531700 1 -- 172.21.15.29:0/1582127523 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47400a9620 con 0x7f472c003e10 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.541+0000 7f4745531700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f472c003e10 0x7f47400ae400 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f4730004960 tx=0x7f4730001e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.541+0000 7f47357fa700 1 -- 172.21.15.29:0/1582127523 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730011590 con 0x7f472c003e10 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.541+0000 7f47357fa700 1 -- 172.21.15.29:0/1582127523 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47300116f0 con 0x7f472c003e10 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.541+0000 7f4747795700 1 -- 172.21.15.29:0/1582127523 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47400a8d50 con 0x7f472c003e10 2024-01-24T13:59:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.541+0000 7f47357fa700 1 -- 172.21.15.29:0/1582127523 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730011a70 con 0x7f472c003e10 2024-01-24T13:59:04.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.541+0000 7f4747795700 1 -- 172.21.15.29:0/1582127523 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47400ae8f0 con 0x7f472c003e10 2024-01-24T13:59:04.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.542+0000 7f4747795700 1 -- 172.21.15.29:0/1582127523 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47400475e0 con 0x7f472c003e10 2024-01-24T13:59:04.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.543+0000 7f47357fa700 1 -- 172.21.15.29:0/1582127523 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f473001d3b0 con 0x7f472c003e10 2024-01-24T13:59:04.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.543+0000 7f47357fa700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f472405f150 0x7f4724061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.543+0000 7f47357fa700 1 -- 172.21.15.29:0/1582127523 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f4730079cd0 con 0x7f472c003e10 2024-01-24T13:59:04.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.543+0000 7f4744d30700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f472405f150 0x7f4724061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.544+0000 7f4744d30700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f472405f150 0x7f4724061610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f47400aaa50 tx=0x7f473c01a000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:04.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.547+0000 7f47357fa700 1 -- 172.21.15.29:0/1582127523 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f473004ace0 con 0x7f472c003e10 2024-01-24T13:59:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.732+0000 7f4747795700 1 -- 172.21.15.29:0/1582127523 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4740067a90 con 0x7f472c003e10 2024-01-24T13:59:04.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.733+0000 7f47357fa700 1 -- 172.21.15.29:0/1582127523 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4730019020 con 0x7f472c003e10 2024-01-24T13:59:04.733 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T13:59:04.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.735+0000 7f4747795700 1 -- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f472405f150 msgr2=0x7f4724061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.735+0000 7f4747795700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f472405f150 0x7f4724061610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f47400aaa50 tx=0x7f473c01a000).stop 2024-01-24T13:59:04.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.735+0000 7f4747795700 1 -- 172.21.15.29:0/1582127523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f472c003e10 msgr2=0x7f47400ae400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.735+0000 7f4747795700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f472c003e10 0x7f47400ae400 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f4730004960 tx=0x7f4730001e90).stop 2024-01-24T13:59:04.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.735+0000 7f4747795700 1 -- 172.21.15.29:0/1582127523 shutdown_connections 2024-01-24T13:59:04.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.735+0000 7f4747795700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4740163190 0x7f4740165650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.735+0000 7f4747795700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f472405f150 0x7f4724061610 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.735+0000 7f4747795700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f472c003e10 0x7f47400ae400 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.735+0000 7f4747795700 1 --2- 172.21.15.29:0/1582127523 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47400ac140 0x7f4740164d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.735+0000 7f4747795700 1 -- 172.21.15.29:0/1582127523 >> 172.21.15.29:0/1582127523 conn(0x7f47400c8bd0 msgr2=0x7f4740151860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:04.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.735+0000 7f4747795700 1 -- 172.21.15.29:0/1582127523 shutdown_connections 2024-01-24T13:59:04.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.735+0000 7f4747795700 1 -- 172.21.15.29:0/1582127523 wait complete. 2024-01-24T13:59:04.748 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T13:59:04.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.881+0000 7fea10876700 1 Processor -- start 2024-01-24T13:59:04.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.881+0000 7fea10876700 1 -- start start 2024-01-24T13:59:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.882+0000 7fea10876700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fea0c0692a0 0x7fea0c065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.882+0000 7fea10876700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fea0c066370 0x7fea0c069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.882+0000 7fea10876700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0c0c3e70 con 0x7fea0c069b70 2024-01-24T13:59:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.882+0000 7fea10876700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea0c0c4300 con 0x7fea0c0692a0 2024-01-24T13:59:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.882+0000 7fea10876700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea0c0c3260 con 0x7fea0c066370 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.882+0000 7fea0a59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fea0c0692a0 0x7fea0c065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.883+0000 7fea0a59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fea0c0692a0 0x7fea0c065e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44938/0 (socket says 172.21.15.29:44938) 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.883+0000 7fea0a59c700 1 -- 172.21.15.29:0/1635751956 learned_addr learned my addr 172.21.15.29:0/1635751956 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.883+0000 7fea03fff700 1 --2- 172.21.15.29:0/1635751956 >> v2:172.21.15.163:3300/0 conn(0x7fea0c066370 0x7fea0c069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.883+0000 7fea0a59c700 1 -- 172.21.15.29:0/1635751956 >> v2:172.21.15.163:3300/0 conn(0x7fea0c066370 msgr2=0x7fea0c069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.883+0000 7fea0a59c700 1 --2- 172.21.15.29:0/1635751956 >> v2:172.21.15.163:3300/0 conn(0x7fea0c066370 0x7fea0c069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.883+0000 7fea0a59c700 1 -- 172.21.15.29:0/1635751956 >> v1:172.21.15.163:6789/0 conn(0x7fea0c069b70 legacy=0x7fea0c0bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.883+0000 7fea0a59c700 1 -- 172.21.15.29:0/1635751956 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c0c3540 con 0x7fea0c0692a0 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.883+0000 7fea0a59c700 1 --2- 172.21.15.29:0/1635751956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea0c0692a0 0x7fea0c065e80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fea0c0c8b30 tx=0x7fe9fc0090f0).ready entity=mon.0 client_cookie=7593988283b69e12 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.883+0000 7fea09d9b700 1 -- 172.21.15.29:0/1635751956 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00b600 con 0x7fea0c0692a0 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.883+0000 7fea09d9b700 1 -- 172.21.15.29:0/1635751956 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc00b760 con 0x7fea0c0692a0 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.883+0000 7fea09d9b700 1 -- 172.21.15.29:0/1635751956 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00bae0 con 0x7fea0c0692a0 2024-01-24T13:59:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.884+0000 7fea10876700 1 -- 172.21.15.29:0/1635751956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea0c0692a0 msgr2=0x7fea0c065e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.884+0000 7fea10876700 1 --2- 172.21.15.29:0/1635751956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea0c0692a0 0x7fea0c065e80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fea0c0c8b30 tx=0x7fe9fc0090f0).stop 2024-01-24T13:59:04.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.884+0000 7fea10876700 1 -- 172.21.15.29:0/1635751956 shutdown_connections 2024-01-24T13:59:04.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.884+0000 7fea10876700 1 --2- 172.21.15.29:0/1635751956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea0c0692a0 0x7fea0c065e80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.884+0000 7fea10876700 1 --2- 172.21.15.29:0/1635751956 >> v2:172.21.15.163:3300/0 conn(0x7fea0c066370 0x7fea0c069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.884+0000 7fea10876700 1 -- 172.21.15.29:0/1635751956 >> 172.21.15.29:0/1635751956 conn(0x7fea0c064390 msgr2=0x7fea0c0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:04.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.884+0000 7fea10876700 1 -- 172.21.15.29:0/1635751956 shutdown_connections 2024-01-24T13:59:04.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.884+0000 7fea10876700 1 -- 172.21.15.29:0/1635751956 wait complete. 2024-01-24T13:59:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.886+0000 7fea10876700 1 Processor -- start 2024-01-24T13:59:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.886+0000 7fea10876700 1 -- start start 2024-01-24T13:59:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.886+0000 7fea10876700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea0c066370 0x7fea0c13cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.886+0000 7fea10876700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea0c0692a0 0x7fea0c145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.886+0000 7fea10876700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea0c069b70 0x7fea0c145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.886+0000 7fea10876700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c0c1940 con 0x7fea0c066370 2024-01-24T13:59:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.886+0000 7fea10876700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c146270 con 0x7fea0c0692a0 2024-01-24T13:59:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.886+0000 7fea10876700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c0beee0 con 0x7fea0c069b70 2024-01-24T13:59:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea0a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea0c066370 0x7fea0c13cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea0a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea0c066370 0x7fea0c13cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44954/0 (socket says 172.21.15.29:44954) 2024-01-24T13:59:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea0a59c700 1 -- 172.21.15.29:0/356613940 learned_addr learned my addr 172.21.15.29:0/356613940 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea03fff700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea0c0692a0 0x7fea0c145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea0ad9d700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea0c069b70 0x7fea0c145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea0a59c700 1 -- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea0c069b70 msgr2=0x7fea0c145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea0a59c700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea0c069b70 0x7fea0c145d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea0a59c700 1 -- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea0c0692a0 msgr2=0x7fea0c145890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea0a59c700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea0c0692a0 0x7fea0c145890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea0a59c700 1 -- 172.21.15.29:0/356613940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc009370 con 0x7fea0c066370 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea0a59c700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea0c066370 0x7fea0c13cc90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fea0c064910 tx=0x7fe9fc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea027fc700 1 -- 172.21.15.29:0/356613940 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00b8d0 con 0x7fea0c066370 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea027fc700 1 -- 172.21.15.29:0/356613940 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc00ba30 con 0x7fea0c066370 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea10876700 1 -- 172.21.15.29:0/356613940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0c0bf1c0 con 0x7fea0c066370 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea027fc700 1 -- 172.21.15.29:0/356613940 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc01c670 con 0x7fea0c066370 2024-01-24T13:59:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.887+0000 7fea10876700 1 -- 172.21.15.29:0/356613940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0c0bf780 con 0x7fea0c066370 2024-01-24T13:59:04.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.888+0000 7fea10876700 1 -- 172.21.15.29:0/356613940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0c0475e0 con 0x7fea0c066370 2024-01-24T13:59:04.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.889+0000 7fea027fc700 1 -- 172.21.15.29:0/356613940 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe9fc01ca20 con 0x7fea0c066370 2024-01-24T13:59:04.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.889+0000 7fea027fc700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe9ec05f150 0x7fe9ec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:04.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.889+0000 7fea027fc700 1 -- 172.21.15.29:0/356613940 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe9fc07bd00 con 0x7fea0c066370 2024-01-24T13:59:04.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.889+0000 7fea03fff700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe9ec05f150 0x7fe9ec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:04.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.890+0000 7fea03fff700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe9ec05f150 0x7fe9ec061610 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fe9f4006fd0 tx=0x7fe9f4008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:04.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:04.893+0000 7fea027fc700 1 -- 172.21.15.29:0/356613940 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe9fc04ccc0 con 0x7fea0c066370 2024-01-24T13:59:05.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.072+0000 7fea10876700 1 -- 172.21.15.29:0/356613940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fea0c0c6840 con 0x7fea0c066370 2024-01-24T13:59:05.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.072+0000 7fea027fc700 1 -- 172.21.15.29:0/356613940 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe9fc020020 con 0x7fea0c066370 2024-01-24T13:59:05.073 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T13:59:05.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.074+0000 7fea10876700 1 -- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe9ec05f150 msgr2=0x7fe9ec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.074+0000 7fea10876700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe9ec05f150 0x7fe9ec061610 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fe9f4006fd0 tx=0x7fe9f4008040).stop 2024-01-24T13:59:05.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.074+0000 7fea10876700 1 -- 172.21.15.29:0/356613940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea0c066370 msgr2=0x7fea0c13cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.074+0000 7fea10876700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea0c066370 0x7fea0c13cc90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fea0c064910 tx=0x7fe9fc017000).stop 2024-01-24T13:59:05.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.074+0000 7fea10876700 1 -- 172.21.15.29:0/356613940 shutdown_connections 2024-01-24T13:59:05.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.074+0000 7fea10876700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea0c069b70 0x7fea0c145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.074+0000 7fea10876700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe9ec05f150 0x7fe9ec061610 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.074+0000 7fea10876700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea0c066370 0x7fea0c13cc90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.074+0000 7fea10876700 1 --2- 172.21.15.29:0/356613940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea0c0692a0 0x7fea0c145890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.074+0000 7fea10876700 1 -- 172.21.15.29:0/356613940 >> 172.21.15.29:0/356613940 conn(0x7fea0c064390 msgr2=0x7fea0c0c6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:05.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.074+0000 7fea10876700 1 -- 172.21.15.29:0/356613940 shutdown_connections 2024-01-24T13:59:05.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.074+0000 7fea10876700 1 -- 172.21.15.29:0/356613940 wait complete. 2024-01-24T13:59:05.088 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T13:59:05.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.220+0000 7f43f6447700 1 Processor -- start 2024-01-24T13:59:05.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.220+0000 7f43f6447700 1 -- start start 2024-01-24T13:59:05.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.221+0000 7f43f6447700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f01599f0 0x7f43f015bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.221+0000 7f43f6447700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f00c3e70 con 0x7f43f00692a0 2024-01-24T13:59:05.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.221+0000 7f43f6447700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f00c4300 con 0x7f43f0069680 2024-01-24T13:59:05.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.221+0000 7f43f6447700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f00c3260 con 0x7f43f01599f0 2024-01-24T13:59:05.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.221+0000 7f43effff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f43f00692a0 0x7f43f0065e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59570/0 (socket says 172.21.15.29:59570) 2024-01-24T13:59:05.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.221+0000 7f43effff700 1 -- 172.21.15.29:0/1130357676 learned_addr learned my addr 172.21.15.29:0/1130357676 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:05.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.221+0000 7f43f49e4700 1 --2- 172.21.15.29:0/1130357676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f01599f0 0x7f43f015bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.222+0000 7f43eeffd700 1 -- 172.21.15.29:0/1130357676 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076508062 0 0) 0x7f43f00c3e70 con 0x7f43f00692a0 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.222+0000 7f43eeffd700 1 -- 172.21.15.29:0/1130357676 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43dc0033f0 con 0x7f43f00692a0 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.222+0000 7f43eeffd700 1 -- 172.21.15.29:0/1130357676 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027822713 0 0) 0x7f43f00c4300 con 0x7f43f0069680 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.222+0000 7f43eeffd700 1 -- 172.21.15.29:0/1130357676 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f00c3e70 con 0x7f43f0069680 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.222+0000 7f43f49e4700 1 -- 172.21.15.29:0/1130357676 >> v1:172.21.15.163:6789/0 conn(0x7f43f0069680 legacy=0x7f43f0069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.222+0000 7f43eeffd700 1 -- 172.21.15.29:0/1130357676 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4148103119 0 0) 0x7f43dc0033f0 con 0x7f43f00692a0 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.222+0000 7f43f49e4700 1 -- 172.21.15.29:0/1130357676 >> v1:172.21.15.29:6789/0 conn(0x7f43f00692a0 legacy=0x7f43f0065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.222+0000 7f43f49e4700 1 -- 172.21.15.29:0/1130357676 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f00c3540 con 0x7f43f01599f0 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.222+0000 7f43f49e4700 1 --2- 172.21.15.29:0/1130357676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f01599f0 0x7f43f015bde0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f43e8009e90 tx=0x7f43e8009780).ready entity=mon.2 client_cookie=97e53cd136653069 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.222+0000 7f43eeffd700 1 -- 172.21.15.29:0/1130357676 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e800e070 con 0x7f43f01599f0 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.222+0000 7f43eeffd700 1 -- 172.21.15.29:0/1130357676 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e800b7a0 con 0x7f43f01599f0 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.222+0000 7f43eeffd700 1 -- 172.21.15.29:0/1130357676 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0001470 con 0x7f43f01599f0 2024-01-24T13:59:05.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.223+0000 7f43f6447700 1 -- 172.21.15.29:0/1130357676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f01599f0 msgr2=0x7f43f015bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.223+0000 7f43f6447700 1 --2- 172.21.15.29:0/1130357676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f01599f0 0x7f43f015bde0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f43e8009e90 tx=0x7f43e8009780).stop 2024-01-24T13:59:05.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.223+0000 7f43f6447700 1 -- 172.21.15.29:0/1130357676 shutdown_connections 2024-01-24T13:59:05.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.223+0000 7f43f6447700 1 --2- 172.21.15.29:0/1130357676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f01599f0 0x7f43f015bde0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.223+0000 7f43f6447700 1 -- 172.21.15.29:0/1130357676 >> 172.21.15.29:0/1130357676 conn(0x7f43f0064390 msgr2=0x7f43f00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:05.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.223+0000 7f43f6447700 1 -- 172.21.15.29:0/1130357676 shutdown_connections 2024-01-24T13:59:05.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.223+0000 7f43f6447700 1 -- 172.21.15.29:0/1130357676 wait complete. 2024-01-24T13:59:05.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.224+0000 7f43f6447700 1 Processor -- start 2024-01-24T13:59:05.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.224+0000 7f43f6447700 1 -- start start 2024-01-24T13:59:05.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.225+0000 7f43f6447700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00692a0 0x7f43f0164b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.225+0000 7f43f6447700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f0069680 0x7f43f013fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.225+0000 7f43f6447700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f01599f0 0x7f43f00c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.225+0000 7f43f6447700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f0140ff0 con 0x7f43f01599f0 2024-01-24T13:59:05.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.225+0000 7f43f6447700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f0165040 con 0x7f43f0069680 2024-01-24T13:59:05.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.225+0000 7f43f6447700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f0151560 con 0x7f43f00692a0 2024-01-24T13:59:05.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.225+0000 7f43f49e4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f01599f0 0x7f43f00c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.225+0000 7f43f49e4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f01599f0 0x7f43f00c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44956/0 (socket says 172.21.15.29:44956) 2024-01-24T13:59:05.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.225+0000 7f43f49e4700 1 -- 172.21.15.29:0/3560421445 learned_addr learned my addr 172.21.15.29:0/3560421445 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.225+0000 7f43effff700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00692a0 0x7f43f0164b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.225+0000 7f43ef7fe700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f0069680 0x7f43f013fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.226+0000 7f43f49e4700 1 -- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00692a0 msgr2=0x7f43f0164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.226+0000 7f43f49e4700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00692a0 0x7f43f0164b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.226+0000 7f43f49e4700 1 -- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f0069680 msgr2=0x7f43f013fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.226+0000 7f43f49e4700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f0069680 0x7f43f013fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.226+0000 7f43f49e4700 1 -- 172.21.15.29:0/3560421445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e8009a20 con 0x7f43f01599f0 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.226+0000 7f43f49e4700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f01599f0 0x7f43f00c5cb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f43f0064910 tx=0x7f43e8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.226+0000 7f43e7fff700 1 -- 172.21.15.29:0/3560421445 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e801d440 con 0x7f43f01599f0 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.226+0000 7f43e7fff700 1 -- 172.21.15.29:0/3560421445 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e801d5a0 con 0x7f43f01599f0 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.226+0000 7f43e7fff700 1 -- 172.21.15.29:0/3560421445 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e801d940 con 0x7f43f01599f0 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.226+0000 7f43f6447700 1 -- 172.21.15.29:0/3560421445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f0167820 con 0x7f43f01599f0 2024-01-24T13:59:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.226+0000 7f43f6447700 1 -- 172.21.15.29:0/3560421445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f0167de0 con 0x7f43f01599f0 2024-01-24T13:59:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.227+0000 7f43f6447700 1 -- 172.21.15.29:0/3560421445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f0167680 con 0x7f43f01599f0 2024-01-24T13:59:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.227+0000 7f43e7fff700 1 -- 172.21.15.29:0/3560421445 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f43e801dd30 con 0x7f43f01599f0 2024-01-24T13:59:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.228+0000 7f43e7fff700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43d005f150 0x7f43d0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.228+0000 7f43e7fff700 1 -- 172.21.15.29:0/3560421445 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f43e807c600 con 0x7f43f01599f0 2024-01-24T13:59:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.228+0000 7f43effff700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43d005f150 0x7f43d0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.229+0000 7f43effff700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43d005f150 0x7f43d0061610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f43e00015a0 tx=0x7f43e0000bc0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:05.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.232+0000 7f43e7fff700 1 -- 172.21.15.29:0/3560421445 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f43f0167680 con 0x7f43f01599f0 2024-01-24T13:59:05.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.412+0000 7f43f6447700 1 -- 172.21.15.29:0/3560421445 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43f0167680 con 0x7f43f01599f0 2024-01-24T13:59:05.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.412+0000 7f43e7fff700 1 -- 172.21.15.29:0/3560421445 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43f0167680 con 0x7f43f01599f0 2024-01-24T13:59:05.412 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:59:05.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.414+0000 7f43f6447700 1 -- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43d005f150 msgr2=0x7f43d0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.414+0000 7f43f6447700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43d005f150 0x7f43d0061610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f43e00015a0 tx=0x7f43e0000bc0).stop 2024-01-24T13:59:05.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.414+0000 7f43f6447700 1 -- 172.21.15.29:0/3560421445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f01599f0 msgr2=0x7f43f00c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.414+0000 7f43f6447700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f01599f0 0x7f43f00c5cb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f43f0064910 tx=0x7f43e8009780).stop 2024-01-24T13:59:05.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.414+0000 7f43f6447700 1 -- 172.21.15.29:0/3560421445 shutdown_connections 2024-01-24T13:59:05.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.414+0000 7f43f6447700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43d005f150 0x7f43d0061610 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.414+0000 7f43f6447700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f01599f0 0x7f43f00c5cb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.414+0000 7f43f6447700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00692a0 0x7f43f0164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.414+0000 7f43f6447700 1 --2- 172.21.15.29:0/3560421445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f0069680 0x7f43f013fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.414+0000 7f43f6447700 1 -- 172.21.15.29:0/3560421445 >> 172.21.15.29:0/3560421445 conn(0x7f43f0064390 msgr2=0x7f43f00c7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:05.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.414+0000 7f43f6447700 1 -- 172.21.15.29:0/3560421445 shutdown_connections 2024-01-24T13:59:05.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.414+0000 7f43f6447700 1 -- 172.21.15.29:0/3560421445 wait complete. 2024-01-24T13:59:05.428 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T13:59:05.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.561+0000 7f0198a19700 1 Processor -- start 2024-01-24T13:59:05.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.561+0000 7f0198a19700 1 -- start start 2024-01-24T13:59:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f0198a19700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01940692a0 0x7f0194065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f0198a19700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0194066370 0x7f0194069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f0198a19700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01940c3e70 con 0x7f0194069b70 2024-01-24T13:59:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f0198a19700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01940c4300 con 0x7f0194066370 2024-01-24T13:59:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f0198a19700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01940c3260 con 0x7f01940692a0 2024-01-24T13:59:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f0191d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0194066370 0x7f0194069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f0191d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0194066370 0x7f0194069680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44964/0 (socket says 172.21.15.29:44964) 2024-01-24T13:59:05.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f0191d9b700 1 -- 172.21.15.29:0/4174540225 learned_addr learned my addr 172.21.15.29:0/4174540225 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f019259c700 1 --2- 172.21.15.29:0/4174540225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01940692a0 0x7f0194065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f0191d9b700 1 -- 172.21.15.29:0/4174540225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01940692a0 msgr2=0x7f0194065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f0191d9b700 1 --2- 172.21.15.29:0/4174540225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01940692a0 0x7f0194065e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f0191d9b700 1 -- 172.21.15.29:0/4174540225 >> v1:172.21.15.163:6789/0 conn(0x7f0194069b70 legacy=0x7f01940bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.562+0000 7f0191d9b700 1 -- 172.21.15.29:0/4174540225 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01940c3540 con 0x7f0194066370 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.563+0000 7f0191d9b700 1 --2- 172.21.15.29:0/4174540225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0194066370 0x7f0194069680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f0188006fd0 tx=0x7f018800b3b0).ready entity=mon.0 client_cookie=254220f21bfa0578 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.563+0000 7f019159a700 1 -- 172.21.15.29:0/4174540225 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018800da70 con 0x7f0194066370 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.563+0000 7f019159a700 1 -- 172.21.15.29:0/4174540225 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018800dbd0 con 0x7f0194066370 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.563+0000 7f019159a700 1 -- 172.21.15.29:0/4174540225 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188010470 con 0x7f0194066370 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.563+0000 7f0198a19700 1 -- 172.21.15.29:0/4174540225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0194066370 msgr2=0x7f0194069680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.563+0000 7f0198a19700 1 --2- 172.21.15.29:0/4174540225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0194066370 0x7f0194069680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f0188006fd0 tx=0x7f018800b3b0).stop 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.563+0000 7f0198a19700 1 -- 172.21.15.29:0/4174540225 shutdown_connections 2024-01-24T13:59:05.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.563+0000 7f0198a19700 1 --2- 172.21.15.29:0/4174540225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01940692a0 0x7f0194065e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.563+0000 7f0198a19700 1 --2- 172.21.15.29:0/4174540225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0194066370 0x7f0194069680 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.563+0000 7f0198a19700 1 -- 172.21.15.29:0/4174540225 >> 172.21.15.29:0/4174540225 conn(0x7f0194064390 msgr2=0x7f01940c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.563+0000 7f0198a19700 1 -- 172.21.15.29:0/4174540225 shutdown_connections 2024-01-24T13:59:05.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.564+0000 7f0198a19700 1 -- 172.21.15.29:0/4174540225 wait complete. 2024-01-24T13:59:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.565+0000 7f0198a19700 1 Processor -- start 2024-01-24T13:59:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.565+0000 7f0198a19700 1 -- start start 2024-01-24T13:59:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.566+0000 7f0198a19700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0194066370 0x7f0194165020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.566+0000 7f0198a19700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01940692a0 0x7f01940ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.566+0000 7f0198a19700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0194069b70 0x7f01940a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.566+0000 7f0198a19700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01940ab550 con 0x7f0194069b70 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.566+0000 7f0198a19700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01940aac20 con 0x7f01940692a0 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.566+0000 7f0198a19700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01940aaf00 con 0x7f0194066370 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.566+0000 7f0192d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0194069b70 0x7f01940a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.566+0000 7f0192d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0194069b70 0x7f01940a9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44972/0 (socket says 172.21.15.29:44972) 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.566+0000 7f0192d9d700 1 -- 172.21.15.29:0/3516646965 learned_addr learned my addr 172.21.15.29:0/3516646965 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f019259c700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0194066370 0x7f0194165020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f0191d9b700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01940692a0 0x7f01940ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f0192d9d700 1 -- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0194066370 msgr2=0x7f0194165020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f0192d9d700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0194066370 0x7f0194165020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f0192d9d700 1 -- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01940692a0 msgr2=0x7f01940ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f0192d9d700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01940692a0 0x7f01940ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f0192d9d700 1 -- 172.21.15.29:0/3516646965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018800b610 con 0x7f0194069b70 2024-01-24T13:59:05.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f0192d9d700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0194069b70 0x7f01940a9ef0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f018c002230 tx=0x7f018c0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:05.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f01827fc700 1 -- 172.21.15.29:0/3516646965 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c00e4a0 con 0x7f0194069b70 2024-01-24T13:59:05.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f01827fc700 1 -- 172.21.15.29:0/3516646965 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018c00e600 con 0x7f0194069b70 2024-01-24T13:59:05.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f0198a19700 1 -- 172.21.15.29:0/3516646965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01940ab1e0 con 0x7f0194069b70 2024-01-24T13:59:05.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f01827fc700 1 -- 172.21.15.29:0/3516646965 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c00e980 con 0x7f0194069b70 2024-01-24T13:59:05.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.567+0000 7f0198a19700 1 -- 172.21.15.29:0/3516646965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01940aa3e0 con 0x7f0194069b70 2024-01-24T13:59:05.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.568+0000 7f0198a19700 1 -- 172.21.15.29:0/3516646965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01940ab6c0 con 0x7f0194069b70 2024-01-24T13:59:05.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.569+0000 7f01827fc700 1 -- 172.21.15.29:0/3516646965 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f018c01b3b0 con 0x7f0194069b70 2024-01-24T13:59:05.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.569+0000 7f01827fc700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f017405f150 0x7f0174061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.569+0000 7f01827fc700 1 -- 172.21.15.29:0/3516646965 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f018c079b00 con 0x7f0194069b70 2024-01-24T13:59:05.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.569+0000 7f019259c700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f017405f150 0x7f0174061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.570+0000 7f019259c700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f017405f150 0x7f0174061610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f01940c3ad0 tx=0x7f0184009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:05.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.573+0000 7f01827fc700 1 -- 172.21.15.29:0/3516646965 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f018c048b00 con 0x7f0194069b70 2024-01-24T13:59:05.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.756+0000 7f0198a19700 1 -- 172.21.15.29:0/3516646965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01940475e0 con 0x7f0194069b70 2024-01-24T13:59:05.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.757+0000 7f01827fc700 1 -- 172.21.15.29:0/3516646965 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f018c04ab10 con 0x7f0194069b70 2024-01-24T13:59:05.757 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T13:59:05.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.759+0000 7f0198a19700 1 -- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f017405f150 msgr2=0x7f0174061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.759+0000 7f0198a19700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f017405f150 0x7f0174061610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f01940c3ad0 tx=0x7f0184009060).stop 2024-01-24T13:59:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.759+0000 7f0198a19700 1 -- 172.21.15.29:0/3516646965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0194069b70 msgr2=0x7f01940a9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.759+0000 7f0198a19700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0194069b70 0x7f01940a9ef0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f018c002230 tx=0x7f018c0015e0).stop 2024-01-24T13:59:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.759+0000 7f0198a19700 1 -- 172.21.15.29:0/3516646965 shutdown_connections 2024-01-24T13:59:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.759+0000 7f0198a19700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f017405f150 0x7f0174061610 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.759+0000 7f0198a19700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0194069b70 0x7f01940a9ef0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.759+0000 7f0198a19700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0194066370 0x7f0194165020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.759+0000 7f0198a19700 1 --2- 172.21.15.29:0/3516646965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01940692a0 0x7f01940ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.759+0000 7f0198a19700 1 -- 172.21.15.29:0/3516646965 >> 172.21.15.29:0/3516646965 conn(0x7f0194064390 msgr2=0x7f019414aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.759+0000 7f0198a19700 1 -- 172.21.15.29:0/3516646965 shutdown_connections 2024-01-24T13:59:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.759+0000 7f0198a19700 1 -- 172.21.15.29:0/3516646965 wait complete. 2024-01-24T13:59:05.772 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-24T13:59:05.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.906+0000 7f4b2befa700 1 Processor -- start 2024-01-24T13:59:05.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.907+0000 7f4b2befa700 1 -- start start 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.907+0000 7f4b2befa700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4b240664c0 0x7f4b24066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.907+0000 7f4b2befa700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b240686a0 con 0x7f4b240bdb30 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.907+0000 7f4b2befa700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b24066e30 con 0x7f4b24134ea0 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.907+0000 7f4b2befa700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b240c38e0 con 0x7f4b240664c0 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.908+0000 7f4b29495700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4b240664c0 0x7f4b24066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.908+0000 7f4b29495700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4b240664c0 0x7f4b24066940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44974/0 (socket says 172.21.15.29:44974) 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.908+0000 7f4b29495700 1 -- 172.21.15.29:0/3833966554 learned_addr learned my addr 172.21.15.29:0/3833966554 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.908+0000 7f4b2a497700 1 --1- 172.21.15.29:0/3833966554 >> v1:172.21.15.29:6789/0 conn(0x7f4b240bdb30 0x7f4b240bdfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59582/0 (socket says 172.21.15.29:59582) 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.908+0000 7f4b29495700 1 -- 172.21.15.29:0/3833966554 >> v1:172.21.15.163:6789/0 conn(0x7f4b24134ea0 legacy=0x7f4b240c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.908+0000 7f4b29495700 1 -- 172.21.15.29:0/3833966554 >> v1:172.21.15.29:6789/0 conn(0x7f4b240bdb30 legacy=0x7f4b240bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.908+0000 7f4b28c94700 1 -- 172.21.15.29:0/3833966554 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528024536 0 0) 0x7f4b240686a0 con 0x7f4b240bdb30 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.908+0000 7f4b29495700 1 -- 172.21.15.29:0/3833966554 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b24068b30 con 0x7f4b240664c0 2024-01-24T13:59:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.908+0000 7f4b29495700 1 --2- 172.21.15.29:0/3833966554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240664c0 0x7f4b24066940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f4b20006fd0 tx=0x7f4b200090f0).ready entity=mon.0 client_cookie=2b70fa0fde9f287c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:05.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.908+0000 7f4b28c94700 1 -- 172.21.15.29:0/3833966554 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000b600 con 0x7f4b240664c0 2024-01-24T13:59:05.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.908+0000 7f4b28c94700 1 -- 172.21.15.29:0/3833966554 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2000b760 con 0x7f4b240664c0 2024-01-24T13:59:05.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.908+0000 7f4b28c94700 1 -- 172.21.15.29:0/3833966554 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000bae0 con 0x7f4b240664c0 2024-01-24T13:59:05.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.909+0000 7f4b2befa700 1 -- 172.21.15.29:0/3833966554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240664c0 msgr2=0x7f4b24066940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.909+0000 7f4b2befa700 1 --2- 172.21.15.29:0/3833966554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240664c0 0x7f4b24066940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f4b20006fd0 tx=0x7f4b200090f0).stop 2024-01-24T13:59:05.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.909+0000 7f4b2befa700 1 -- 172.21.15.29:0/3833966554 shutdown_connections 2024-01-24T13:59:05.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.909+0000 7f4b2befa700 1 --2- 172.21.15.29:0/3833966554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240664c0 0x7f4b24066940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.909+0000 7f4b2befa700 1 -- 172.21.15.29:0/3833966554 >> 172.21.15.29:0/3833966554 conn(0x7f4b240c8bd0 msgr2=0x7f4b240c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:05.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.909+0000 7f4b2befa700 1 -- 172.21.15.29:0/3833966554 shutdown_connections 2024-01-24T13:59:05.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.909+0000 7f4b2befa700 1 -- 172.21.15.29:0/3833966554 wait complete. 2024-01-24T13:59:05.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.910+0000 7f4b2befa700 1 Processor -- start 2024-01-24T13:59:05.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.910+0000 7f4b2befa700 1 -- start start 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.911+0000 7f4b2befa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240664c0 0x7f4b240adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.911+0000 7f4b2befa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b240bdb30 0x7f4b240ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.911+0000 7f4b2befa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b24134ea0 0x7f4b240abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.911+0000 7f4b2befa700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240aead0 con 0x7f4b240664c0 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.911+0000 7f4b2befa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2414cb30 con 0x7f4b24134ea0 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.911+0000 7f4b2befa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2414ce10 con 0x7f4b240bdb30 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.911+0000 7f4b29c96700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240664c0 0x7f4b240adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.911+0000 7f4b29c96700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240664c0 0x7f4b240adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44986/0 (socket says 172.21.15.29:44986) 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.911+0000 7f4b29c96700 1 -- 172.21.15.29:0/2690555196 learned_addr learned my addr 172.21.15.29:0/2690555196 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.911+0000 7f4b29495700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b240bdb30 0x7f4b240ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.911+0000 7f4b2a497700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b24134ea0 0x7f4b240abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.912+0000 7f4b29c96700 1 -- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b240bdb30 msgr2=0x7f4b240ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.912+0000 7f4b29c96700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b240bdb30 0x7f4b240ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.912+0000 7f4b29c96700 1 -- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b24134ea0 msgr2=0x7f4b240abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.912+0000 7f4b29c96700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b24134ea0 0x7f4b240abb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.912+0000 7f4b29c96700 1 -- 172.21.15.29:0/2690555196 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b20009370 con 0x7f4b240664c0 2024-01-24T13:59:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.912+0000 7f4b29c96700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240664c0 0x7f4b240adc80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f4b1400c4c0 tx=0x7f4b14007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.912+0000 7f4b19ffb700 1 -- 172.21.15.29:0/2690555196 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400ce70 con 0x7f4b240664c0 2024-01-24T13:59:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.912+0000 7f4b19ffb700 1 -- 172.21.15.29:0/2690555196 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b14001e90 con 0x7f4b240664c0 2024-01-24T13:59:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.912+0000 7f4b2befa700 1 -- 172.21.15.29:0/2690555196 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b240ae230 con 0x7f4b240664c0 2024-01-24T13:59:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.912+0000 7f4b19ffb700 1 -- 172.21.15.29:0/2690555196 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14007250 con 0x7f4b240664c0 2024-01-24T13:59:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.912+0000 7f4b2befa700 1 -- 172.21.15.29:0/2690555196 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b240ae7f0 con 0x7f4b240664c0 2024-01-24T13:59:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.913+0000 7f4b2befa700 1 -- 172.21.15.29:0/2690555196 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b240475e0 con 0x7f4b240664c0 2024-01-24T13:59:05.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.913+0000 7f4b19ffb700 1 -- 172.21.15.29:0/2690555196 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4b140066a0 con 0x7f4b240664c0 2024-01-24T13:59:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.914+0000 7f4b19ffb700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4b0805f180 0x7f4b08061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.914+0000 7f4b19ffb700 1 -- 172.21.15.29:0/2690555196 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f4b14079d60 con 0x7f4b240664c0 2024-01-24T13:59:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.914+0000 7f4b29495700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4b0805f180 0x7f4b08061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.915+0000 7f4b29495700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4b0805f180 0x7f4b08061640 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f4b20009130 tx=0x7f4b20017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:05.918+0000 7f4b19ffb700 1 -- 172.21.15.29:0/2690555196 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4b1404ad20 con 0x7f4b240664c0 2024-01-24T13:59:06.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.098+0000 7f4b2befa700 1 -- 172.21.15.29:0/2690555196 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4b24067a90 con 0x7f4b240664c0 2024-01-24T13:59:06.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.098+0000 7f4b19ffb700 1 -- 172.21.15.29:0/2690555196 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4b1404e340 con 0x7f4b240664c0 2024-01-24T13:59:06.099 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 32 2024-01-24T13:59:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.100+0000 7f4b2befa700 1 -- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4b0805f180 msgr2=0x7f4b08061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.100+0000 7f4b2befa700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4b0805f180 0x7f4b08061640 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f4b20009130 tx=0x7f4b20017000).stop 2024-01-24T13:59:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.100+0000 7f4b2befa700 1 -- 172.21.15.29:0/2690555196 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240664c0 msgr2=0x7f4b240adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.100+0000 7f4b2befa700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240664c0 0x7f4b240adc80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f4b1400c4c0 tx=0x7f4b14007e00).stop 2024-01-24T13:59:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.100+0000 7f4b2befa700 1 -- 172.21.15.29:0/2690555196 shutdown_connections 2024-01-24T13:59:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.100+0000 7f4b2befa700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b24134ea0 0x7f4b240abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.100+0000 7f4b2befa700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4b0805f180 0x7f4b08061640 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.100+0000 7f4b2befa700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b240664c0 0x7f4b240adc80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.100+0000 7f4b2befa700 1 --2- 172.21.15.29:0/2690555196 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b240bdb30 0x7f4b240ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.100+0000 7f4b2befa700 1 -- 172.21.15.29:0/2690555196 >> 172.21.15.29:0/2690555196 conn(0x7f4b240c8bd0 msgr2=0x7f4b240c7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.100+0000 7f4b2befa700 1 -- 172.21.15.29:0/2690555196 shutdown_connections 2024-01-24T13:59:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.100+0000 7f4b2befa700 1 -- 172.21.15.29:0/2690555196 wait complete. 2024-01-24T13:59:06.114 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:59:06.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.243+0000 7fcf02c8c700 1 Processor -- start 2024-01-24T13:59:06.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.243+0000 7fcf02c8c700 1 -- start start 2024-01-24T13:59:06.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.244+0000 7fcf02c8c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc069240 0x7fcefc154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.244+0000 7fcf02c8c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcefc154e40 0x7fcefc0bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.244+0000 7fcf02c8c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcefc06a3a0 con 0x7fcefc150cb0 2024-01-24T13:59:06.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.244+0000 7fcf02c8c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcefc0c2c70 con 0x7fcefc154e40 2024-01-24T13:59:06.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.244+0000 7fcf02c8c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc1552d0 con 0x7fcefc069240 2024-01-24T13:59:06.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.244+0000 7fcf01229700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcefc154e40 0x7fcefc0bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.244+0000 7fcf01229700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcefc154e40 0x7fcefc0bcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45000/0 (socket says 172.21.15.29:45000) 2024-01-24T13:59:06.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.244+0000 7fcf01229700 1 -- 172.21.15.29:0/3730069936 learned_addr learned my addr 172.21.15.29:0/3730069936 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:06.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.244+0000 7fcefbfff700 1 --2- 172.21.15.29:0/3730069936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc069240 0x7fcefc154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.245+0000 7fcf01229700 1 -- 172.21.15.29:0/3730069936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc069240 msgr2=0x7fcefc154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:06.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.245+0000 7fcf01229700 1 --2- 172.21.15.29:0/3730069936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc069240 0x7fcefc154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.245+0000 7fcf01229700 1 -- 172.21.15.29:0/3730069936 >> v1:172.21.15.163:6789/0 conn(0x7fcefc150cb0 legacy=0x7fcefc065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.245+0000 7fcf01229700 1 -- 172.21.15.29:0/3730069936 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc069820 con 0x7fcefc154e40 2024-01-24T13:59:06.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.245+0000 7fcf01229700 1 --2- 172.21.15.29:0/3730069936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcefc154e40 0x7fcefc0bcf40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fcef4009f20 tx=0x7fcef4009840).ready entity=mon.0 client_cookie=46ac2d2988e29996 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:06.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.245+0000 7fcefb7fe700 1 -- 172.21.15.29:0/3730069936 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef400e070 con 0x7fcefc154e40 2024-01-24T13:59:06.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.245+0000 7fcefb7fe700 1 -- 172.21.15.29:0/3730069936 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef400ba80 con 0x7fcefc154e40 2024-01-24T13:59:06.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.245+0000 7fcefb7fe700 1 -- 172.21.15.29:0/3730069936 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef400be00 con 0x7fcefc154e40 2024-01-24T13:59:06.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.246+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3730069936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcefc154e40 msgr2=0x7fcefc0bcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.246+0000 7fcf02c8c700 1 --2- 172.21.15.29:0/3730069936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcefc154e40 0x7fcefc0bcf40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fcef4009f20 tx=0x7fcef4009840).stop 2024-01-24T13:59:06.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.246+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3730069936 shutdown_connections 2024-01-24T13:59:06.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.246+0000 7fcf02c8c700 1 --2- 172.21.15.29:0/3730069936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcefc154e40 0x7fcefc0bcf40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.246+0000 7fcf02c8c700 1 --2- 172.21.15.29:0/3730069936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc069240 0x7fcefc154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.246+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3730069936 >> 172.21.15.29:0/3730069936 conn(0x7fcefc0c85c0 msgr2=0x7fcefc0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:06.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.246+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3730069936 shutdown_connections 2024-01-24T13:59:06.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.246+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3730069936 wait complete. 2024-01-24T13:59:06.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.247+0000 7fcf02c8c700 1 Processor -- start 2024-01-24T13:59:06.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.247+0000 7fcf02c8c700 1 -- start start 2024-01-24T13:59:06.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf02c8c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc069240 0x7fcefc14f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf02c8c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc150cb0 0x7fcefc142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf02c8c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcefc154e40 0x7fcefc0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf02c8c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc14cf30 con 0x7fcefc154e40 2024-01-24T13:59:06.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf02c8c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc14d210 con 0x7fcefc069240 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf02c8c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc14d4f0 con 0x7fcefc150cb0 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf01229700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcefc154e40 0x7fcefc0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf01229700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcefc154e40 0x7fcefc0ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45002/0 (socket says 172.21.15.29:45002) 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf01229700 1 -- 172.21.15.29:0/3247102041 learned_addr learned my addr 172.21.15.29:0/3247102041 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf00a28700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc069240 0x7fcefc14f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcefbfff700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc150cb0 0x7fcefc142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf01229700 1 -- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc150cb0 msgr2=0x7fcefc142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf01229700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc150cb0 0x7fcefc142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf01229700 1 -- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc069240 msgr2=0x7fcefc14f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf01229700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc069240 0x7fcefc14f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcf01229700 1 -- 172.21.15.29:0/3247102041 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4009ae0 con 0x7fcefc154e40 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.248+0000 7fcefbfff700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc150cb0 0x7fcefc142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.249+0000 7fcf01229700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcefc154e40 0x7fcefc0ad130 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fcef400b6c0 tx=0x7fcef4009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:06.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.249+0000 7fcef8ff9700 1 -- 172.21.15.29:0/3247102041 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef401c400 con 0x7fcefc154e40 2024-01-24T13:59:06.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.249+0000 7fcef8ff9700 1 -- 172.21.15.29:0/3247102041 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef401c560 con 0x7fcefc154e40 2024-01-24T13:59:06.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.249+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3247102041 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc14d7d0 con 0x7fcefc154e40 2024-01-24T13:59:06.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.249+0000 7fcef8ff9700 1 -- 172.21.15.29:0/3247102041 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef401c920 con 0x7fcefc154e40 2024-01-24T13:59:06.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.249+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3247102041 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc14dd90 con 0x7fcefc154e40 2024-01-24T13:59:06.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.250+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3247102041 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcefc14df80 con 0x7fcefc154e40 2024-01-24T13:59:06.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.250+0000 7fcef8ff9700 1 -- 172.21.15.29:0/3247102041 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fcef401ccd0 con 0x7fcefc154e40 2024-01-24T13:59:06.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.251+0000 7fcef8ff9700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcee005f120 0x7fcee00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.251+0000 7fcef8ff9700 1 -- 172.21.15.29:0/3247102041 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fcef407c000 con 0x7fcefc154e40 2024-01-24T13:59:06.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.251+0000 7fcf00a28700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcee005f120 0x7fcee00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.252+0000 7fcf00a28700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcee005f120 0x7fcee00615e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fcef00036d0 tx=0x7fcef0009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:06.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.255+0000 7fcef8ff9700 1 -- 172.21.15.29:0/3247102041 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcef404cfc0 con 0x7fcefc154e40 2024-01-24T13:59:06.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.481+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3247102041 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcefc0475e0 con 0x7fcefc154e40 2024-01-24T13:59:06.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.482+0000 7fcef8ff9700 1 -- 172.21.15.29:0/3247102041 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4491 (secure 0 0 0) 0x7fcef40505e0 con 0x7fcefc154e40 2024-01-24T13:59:06.482 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:06.482 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2038321939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2038321939},{"type":"v1","addr":"172.21.15.163:6836","nonce":2038321939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3675672874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3675672874},{"type":"v1","addr":"172.21.15.163:6837","nonce":3675672874}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1532435957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532435957},{"type":"v1","addr":"172.21.15.29:6834","nonce":1532435957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:58:19.190134+0000","modified":"2024-01-24T13:58:31.231527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1328130656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1328130656},{"type":"v1","addr":"172.21.15.29:6835","nonce":1328130656}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-24T13:59:06.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.483+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcee005f120 msgr2=0x7fcee00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.483+0000 7fcf02c8c700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcee005f120 0x7fcee00615e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fcef00036d0 tx=0x7fcef0009040).stop 2024-01-24T13:59:06.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.483+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3247102041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcefc154e40 msgr2=0x7fcefc0ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.483+0000 7fcf02c8c700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcefc154e40 0x7fcefc0ad130 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fcef400b6c0 tx=0x7fcef4009840).stop 2024-01-24T13:59:06.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.484+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3247102041 shutdown_connections 2024-01-24T13:59:06.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.484+0000 7fcf02c8c700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcee005f120 0x7fcee00615e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.484+0000 7fcf02c8c700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcefc154e40 0x7fcefc0ad130 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.484+0000 7fcf02c8c700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc069240 0x7fcefc14f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.484+0000 7fcf02c8c700 1 --2- 172.21.15.29:0/3247102041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc150cb0 0x7fcefc142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.484+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3247102041 >> 172.21.15.29:0/3247102041 conn(0x7fcefc0c85c0 msgr2=0x7fcefc0627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:06.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.484+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3247102041 shutdown_connections 2024-01-24T13:59:06.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.484+0000 7fcf02c8c700 1 -- 172.21.15.29:0/3247102041 wait complete. 2024-01-24T13:59:06.485 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 9 2024-01-24T13:59:06.497 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:59:06.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.627+0000 7f019df61700 1 Processor -- start 2024-01-24T13:59:06.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.628+0000 7f019df61700 1 -- start start 2024-01-24T13:59:06.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.628+0000 7f019df61700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0198155040 0x7f0198069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.628+0000 7f019df61700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01980c2ce0 con 0x7f0198150d40 2024-01-24T13:59:06.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.628+0000 7f019df61700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0198069890 con 0x7f0198065ec0 2024-01-24T13:59:06.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.628+0000 7f019df61700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0198069b70 con 0x7f0198155040 2024-01-24T13:59:06.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01977fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0198150d40 0x7f0198154930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59596/0 (socket says 172.21.15.29:59596) 2024-01-24T13:59:06.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01977fe700 1 -- 172.21.15.29:0/359131354 learned_addr learned my addr 172.21.15.29:0/359131354 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:06.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f0196ffd700 1 --2- 172.21.15.29:0/359131354 >> v2:172.21.15.163:3300/0 conn(0x7f0198155040 0x7f0198069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897682637 0 0) 0x7f01980c2ce0 con 0x7f0198150d40 2024-01-24T13:59:06.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01800033f0 con 0x7f0198150d40 2024-01-24T13:59:06.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334625374 0 0) 0x7f0198069890 con 0x7f0198065ec0 2024-01-24T13:59:06.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01980c2ce0 con 0x7f0198065ec0 2024-01-24T13:59:06.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2165137938 0 0) 0x7f01800033f0 con 0x7f0198150d40 2024-01-24T13:59:06.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 >> v2:172.21.15.163:3300/0 conn(0x7f0198155040 msgr2=0x7f0198069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 --2- 172.21.15.29:0/359131354 >> v2:172.21.15.163:3300/0 conn(0x7f0198155040 0x7f0198069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 >> v1:172.21.15.163:6789/0 conn(0x7f0198065ec0 legacy=0x7f01980c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0198069e50 con 0x7f0198150d40 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f0188001470 con 0x7f0198150d40 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f0196ffd700 1 --2- 172.21.15.29:0/359131354 >> v2:172.21.15.163:3300/0 conn(0x7f0198155040 0x7f0198069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 >> v1:172.21.15.29:6789/0 conn(0x7f0198150d40 legacy=0x7f0198154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 --2- 172.21.15.29:0/359131354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0180003a70 0x7f0180003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 --2- 172.21.15.29:0/359131354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0180004690 0x7f0180004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 --2- 172.21.15.29:0/359131354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 0x7f018000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0198069b70 con 0x7f0180005000 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018000be80 con 0x7f0180004690 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.629+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018000c040 con 0x7f0180003a70 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f0197fff700 1 --2- 172.21.15.29:0/359131354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 0x7f018000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f0196ffd700 1 --2- 172.21.15.29:0/359131354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0180004690 0x7f0180004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f01977fe700 1 --2- 172.21.15.29:0/359131354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0180003a70 0x7f0180003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f0197fff700 1 -- 172.21.15.29:0/359131354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0180003a70 msgr2=0x7f0180003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f0197fff700 1 --2- 172.21.15.29:0/359131354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0180003a70 0x7f0180003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f0197fff700 1 -- 172.21.15.29:0/359131354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0180004690 msgr2=0x7f0180004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f0197fff700 1 --2- 172.21.15.29:0/359131354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0180004690 0x7f0180004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f0197fff700 1 -- 172.21.15.29:0/359131354 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01800033f0 con 0x7f0180005000 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f0197fff700 1 -- 172.21.15.29:0/359131354 reap_dead start 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f0197fff700 1 --2- 172.21.15.29:0/359131354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 0x7f018000b8e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f0190001ff0 tx=0x7f01900016e0).ready entity=mon.0 client_cookie=4aa3003a7c3d5391 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019000eac0 con 0x7f0180005000 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019000ec20 con 0x7f0180005000 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.630+0000 7f01967fc700 1 -- 172.21.15.29:0/359131354 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190006a20 con 0x7f0180005000 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.631+0000 7f019df61700 1 -- 172.21.15.29:0/359131354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 msgr2=0x7f018000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.631+0000 7f019df61700 1 --2- 172.21.15.29:0/359131354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 0x7f018000b8e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f0190001ff0 tx=0x7f01900016e0).stop 2024-01-24T13:59:06.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.631+0000 7f019df61700 1 -- 172.21.15.29:0/359131354 shutdown_connections 2024-01-24T13:59:06.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.631+0000 7f019df61700 1 --2- 172.21.15.29:0/359131354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 0x7f018000b8e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.631+0000 7f019df61700 1 -- 172.21.15.29:0/359131354 >> 172.21.15.29:0/359131354 conn(0x7f01980c8580 msgr2=0x7f01980c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:06.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.631+0000 7f019df61700 1 -- 172.21.15.29:0/359131354 shutdown_connections 2024-01-24T13:59:06.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.631+0000 7f019df61700 1 -- 172.21.15.29:0/359131354 wait complete. 2024-01-24T13:59:06.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.632+0000 7f019df61700 1 Processor -- start 2024-01-24T13:59:06.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.632+0000 7f019df61700 1 -- start start 2024-01-24T13:59:06.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f019df61700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 0x7f019814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f019df61700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01980c6450 0x7f01980c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f019df61700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01980acd60 0x7f01980ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f019df61700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01980adc30 con 0x7f0180005000 2024-01-24T13:59:06.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f019df61700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019813f6b0 con 0x7f01980acd60 2024-01-24T13:59:06.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f019df61700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01980c6dc0 con 0x7f01980c6450 2024-01-24T13:59:06.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f01977fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 0x7f019814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f01977fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 0x7f019814bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45016/0 (socket says 172.21.15.29:45016) 2024-01-24T13:59:06.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f01977fe700 1 -- 172.21.15.29:0/4066164328 learned_addr learned my addr 172.21.15.29:0/4066164328 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:06.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f0196ffd700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01980c6450 0x7f01980c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f0197fff700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01980acd60 0x7f01980ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f01977fe700 1 -- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01980c6450 msgr2=0x7f01980c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f01977fe700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01980c6450 0x7f01980c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f01977fe700 1 -- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01980acd60 msgr2=0x7f01980ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f01977fe700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01980acd60 0x7f01980ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.633+0000 7f01977fe700 1 -- 172.21.15.29:0/4066164328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01980ad6d0 con 0x7f0180005000 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.634+0000 7f01977fe700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 0x7f019814bc40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f0188007810 tx=0x7f0188001df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.634+0000 7f017f7fe700 1 -- 172.21.15.29:0/4066164328 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880114c0 con 0x7f0180005000 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.634+0000 7f017f7fe700 1 -- 172.21.15.29:0/4066164328 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0188011620 con 0x7f0180005000 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.634+0000 7f019df61700 1 -- 172.21.15.29:0/4066164328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019814c130 con 0x7f0180005000 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.634+0000 7f017f7fe700 1 -- 172.21.15.29:0/4066164328 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880119a0 con 0x7f0180005000 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.634+0000 7f019df61700 1 -- 172.21.15.29:0/4066164328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019813f060 con 0x7f0180005000 2024-01-24T13:59:06.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.635+0000 7f019df61700 1 -- 172.21.15.29:0/4066164328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019813eec0 con 0x7f0180005000 2024-01-24T13:59:06.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.635+0000 7f017f7fe700 1 -- 172.21.15.29:0/4066164328 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f018801d3b0 con 0x7f0180005000 2024-01-24T13:59:06.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.636+0000 7f017f7fe700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f018405f120 0x7f01840615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.636+0000 7f017f7fe700 1 -- 172.21.15.29:0/4066164328 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f0188079c40 con 0x7f0180005000 2024-01-24T13:59:06.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.636+0000 7f0196ffd700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f018405f120 0x7f01840615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.636+0000 7f0196ffd700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f018405f120 0x7f01840615e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f01980ac340 tx=0x7f018c009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:06.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.640+0000 7f017f7fe700 1 -- 172.21.15.29:0/4066164328 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f019813eec0 con 0x7f0180005000 2024-01-24T13:59:06.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.829+0000 7f019df61700 1 -- 172.21.15.29:0/4066164328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f019813eec0 con 0x7f0180005000 2024-01-24T13:59:06.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.830+0000 7f017f7fe700 1 -- 172.21.15.29:0/4066164328 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19083 (secure 0 0 0) 0x7f019813eec0 con 0x7f0180005000 2024-01-24T13:59:06.830 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:06.831 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":27,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:59:06.646077+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.3f","mappings":[{"from":3,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:59:06.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.832+0000 7f019df61700 1 -- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f018405f120 msgr2=0x7f01840615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.832+0000 7f019df61700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f018405f120 0x7f01840615e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f01980ac340 tx=0x7f018c009060).stop 2024-01-24T13:59:06.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.832+0000 7f019df61700 1 -- 172.21.15.29:0/4066164328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 msgr2=0x7f019814bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.832+0000 7f019df61700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 0x7f019814bc40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f0188007810 tx=0x7f0188001df0).stop 2024-01-24T13:59:06.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.832+0000 7f019df61700 1 -- 172.21.15.29:0/4066164328 shutdown_connections 2024-01-24T13:59:06.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.832+0000 7f019df61700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01980acd60 0x7f01980ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.832+0000 7f019df61700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f018405f120 0x7f01840615e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.832+0000 7f019df61700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0180005000 0x7f019814bc40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.832+0000 7f019df61700 1 --2- 172.21.15.29:0/4066164328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01980c6450 0x7f01980c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.832+0000 7f019df61700 1 -- 172.21.15.29:0/4066164328 >> 172.21.15.29:0/4066164328 conn(0x7f01980c8580 msgr2=0x7f0198154510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:06.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.832+0000 7f019df61700 1 -- 172.21.15.29:0/4066164328 shutdown_connections 2024-01-24T13:59:06.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.832+0000 7f019df61700 1 -- 172.21.15.29:0/4066164328 wait complete. 2024-01-24T13:59:06.846 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T13:59:06.846 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:59:06.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.977+0000 7feff788b700 1 Processor -- start 2024-01-24T13:59:06.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.977+0000 7feff788b700 1 -- start start 2024-01-24T13:59:06.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.978+0000 7feff788b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feff0155b00 0x7feff00664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.978+0000 7feff788b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff0066e90 con 0x7feff00669e0 2024-01-24T13:59:06.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.978+0000 7feff788b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff00c3910 con 0x7feff01565f0 2024-01-24T13:59:06.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.978+0000 7feff788b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff00c5500 con 0x7feff0155b00 2024-01-24T13:59:06.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.978+0000 7feff4e26700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7feff00669e0 0x7feff0155ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59598/0 (socket says 172.21.15.29:59598) 2024-01-24T13:59:06.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.978+0000 7feff4e26700 1 -- 172.21.15.29:0/325184662 learned_addr learned my addr 172.21.15.29:0/325184662 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:06.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.978+0000 7feff5627700 1 --2- 172.21.15.29:0/325184662 >> v2:172.21.15.163:3300/0 conn(0x7feff0155b00 0x7feff00664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.978+0000 7fefe3fff700 1 -- 172.21.15.29:0/325184662 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036164632 0 0) 0x7fefec001110 con 0x7feff00669e0 2024-01-24T13:59:06.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.978+0000 7fefe3fff700 1 -- 172.21.15.29:0/325184662 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefdc0033f0 con 0x7feff00669e0 2024-01-24T13:59:06.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.979+0000 7fefe3fff700 1 -- 172.21.15.29:0/325184662 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104886132 0 0) 0x7feff00c3910 con 0x7feff01565f0 2024-01-24T13:59:06.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.979+0000 7fefe3fff700 1 -- 172.21.15.29:0/325184662 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefec001110 con 0x7feff01565f0 2024-01-24T13:59:06.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.979+0000 7fefe3fff700 1 -- 172.21.15.29:0/325184662 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1434403358 0 0) 0x7fefdc0033f0 con 0x7feff00669e0 2024-01-24T13:59:06.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.979+0000 7feff5627700 1 -- 172.21.15.29:0/325184662 >> v1:172.21.15.163:6789/0 conn(0x7feff01565f0 legacy=0x7feff00bdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.979+0000 7feff5627700 1 -- 172.21.15.29:0/325184662 >> v1:172.21.15.29:6789/0 conn(0x7feff00669e0 legacy=0x7feff0155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.979+0000 7feff5627700 1 -- 172.21.15.29:0/325184662 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff00c5220 con 0x7feff0155b00 2024-01-24T13:59:06.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.979+0000 7feff5627700 1 --2- 172.21.15.29:0/325184662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff0155b00 0x7feff00664f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fefe4009760 tx=0x7fefe4009060).ready entity=mon.1 client_cookie=aa0499f1801f183 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:06.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.979+0000 7fefe3fff700 1 -- 172.21.15.29:0/325184662 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec001600 con 0x7feff0155b00 2024-01-24T13:59:06.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.979+0000 7fefe3fff700 1 -- 172.21.15.29:0/325184662 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe4011400 con 0x7feff0155b00 2024-01-24T13:59:06.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.979+0000 7fefe3fff700 1 -- 172.21.15.29:0/325184662 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4011780 con 0x7feff0155b00 2024-01-24T13:59:06.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.979+0000 7feff788b700 1 -- 172.21.15.29:0/325184662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff0155b00 msgr2=0x7feff00664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.979+0000 7feff788b700 1 --2- 172.21.15.29:0/325184662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff0155b00 0x7feff00664f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fefe4009760 tx=0x7fefe4009060).stop 2024-01-24T13:59:06.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.980+0000 7feff788b700 1 -- 172.21.15.29:0/325184662 shutdown_connections 2024-01-24T13:59:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.980+0000 7feff788b700 1 --2- 172.21.15.29:0/325184662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff0155b00 0x7feff00664f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.980+0000 7feff788b700 1 -- 172.21.15.29:0/325184662 >> 172.21.15.29:0/325184662 conn(0x7feff00c8bb0 msgr2=0x7feff00c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.980+0000 7feff788b700 1 -- 172.21.15.29:0/325184662 shutdown_connections 2024-01-24T13:59:06.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.980+0000 7feff788b700 1 -- 172.21.15.29:0/325184662 wait complete. 2024-01-24T13:59:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.981+0000 7feff788b700 1 Processor -- start 2024-01-24T13:59:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.981+0000 7feff788b700 1 -- start start 2024-01-24T13:59:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff788b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff00669e0 0x7feff00c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff788b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feff0155b00 0x7feff013a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff788b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff01565f0 0x7feff013a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff788b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff01621b0 con 0x7feff0155b00 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff788b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00ad210 con 0x7feff01565f0 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff788b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff00a1620 con 0x7feff00669e0 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff4e26700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feff0155b00 0x7feff013a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff4e26700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feff0155b00 0x7feff013a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45024/0 (socket says 172.21.15.29:45024) 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff4e26700 1 -- 172.21.15.29:0/1556750561 learned_addr learned my addr 172.21.15.29:0/1556750561 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff5627700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff00669e0 0x7feff00c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff5e28700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff01565f0 0x7feff013a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff4e26700 1 -- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff00669e0 msgr2=0x7feff00c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff4e26700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff00669e0 0x7feff00c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff4e26700 1 -- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff01565f0 msgr2=0x7feff013a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff4e26700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff01565f0 0x7feff013a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff4e26700 1 -- 172.21.15.29:0/1556750561 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe40092a0 con 0x7feff0155b00 2024-01-24T13:59:06.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.982+0000 7feff4e26700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feff0155b00 0x7feff013a060 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fefec00c880 tx=0x7fefec001500).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:06.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.983+0000 7fefe17fa700 1 -- 172.21.15.29:0/1556750561 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec007170 con 0x7feff0155b00 2024-01-24T13:59:06.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.983+0000 7fefe17fa700 1 -- 172.21.15.29:0/1556750561 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec0072f0 con 0x7feff0155b00 2024-01-24T13:59:06.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.983+0000 7feff788b700 1 -- 172.21.15.29:0/1556750561 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff00a1900 con 0x7feff0155b00 2024-01-24T13:59:06.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.983+0000 7fefe17fa700 1 -- 172.21.15.29:0/1556750561 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec0048c0 con 0x7feff0155b00 2024-01-24T13:59:06.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.983+0000 7feff788b700 1 -- 172.21.15.29:0/1556750561 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff00ad460 con 0x7feff0155b00 2024-01-24T13:59:06.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.984+0000 7feff788b700 1 -- 172.21.15.29:0/1556750561 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff00ad680 con 0x7feff0155b00 2024-01-24T13:59:06.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.984+0000 7fefe17fa700 1 -- 172.21.15.29:0/1556750561 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fefec004cb0 con 0x7feff0155b00 2024-01-24T13:59:06.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.985+0000 7fefe17fa700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fefd405f150 0x7fefd4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:06.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.985+0000 7fefe17fa700 1 -- 172.21.15.29:0/1556750561 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 7079+0+0 (secure 0 0 0) 0x7fefec07a0a0 con 0x7feff0155b00 2024-01-24T13:59:06.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.985+0000 7feff5627700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fefd405f150 0x7fefd4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:06.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.985+0000 7feff5627700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fefd405f150 0x7fefd4061610 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fefe4009760 tx=0x7fefe4016000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:06.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:06.989+0000 7fefe17fa700 1 -- 172.21.15.29:0/1556750561 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fefec04b060 con 0x7feff0155b00 2024-01-24T13:59:07.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.216+0000 7feff788b700 1 -- 172.21.15.29:0/1556750561 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feff0066e90 con 0x7feff0155b00 2024-01-24T13:59:07.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.217+0000 7fefe17fa700 1 -- 172.21.15.29:0/1556750561 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4491 (secure 0 0 0) 0x7feff0066e90 con 0x7feff0155b00 2024-01-24T13:59:07.217 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:07.217 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2038321939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2038321939},{"type":"v1","addr":"172.21.15.163:6836","nonce":2038321939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3675672874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3675672874},{"type":"v1","addr":"172.21.15.163:6837","nonce":3675672874}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1532435957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532435957},{"type":"v1","addr":"172.21.15.29:6834","nonce":1532435957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:58:19.190134+0000","modified":"2024-01-24T13:58:31.231527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1328130656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1328130656},{"type":"v1","addr":"172.21.15.29:6835","nonce":1328130656}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-24T13:59:07.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.219+0000 7feff788b700 1 -- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fefd405f150 msgr2=0x7fefd4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.219+0000 7feff788b700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fefd405f150 0x7fefd4061610 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fefe4009760 tx=0x7fefe4016000).stop 2024-01-24T13:59:07.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.219+0000 7feff788b700 1 -- 172.21.15.29:0/1556750561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feff0155b00 msgr2=0x7feff013a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.219+0000 7feff788b700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feff0155b00 0x7feff013a060 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fefec00c880 tx=0x7fefec001500).stop 2024-01-24T13:59:07.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.219+0000 7feff788b700 1 -- 172.21.15.29:0/1556750561 shutdown_connections 2024-01-24T13:59:07.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.219+0000 7feff788b700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff01565f0 0x7feff013a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.219+0000 7feff788b700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff00669e0 0x7feff00c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.219+0000 7feff788b700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fefd405f150 0x7fefd4061610 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.219+0000 7feff788b700 1 --2- 172.21.15.29:0/1556750561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feff0155b00 0x7feff013a060 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.219+0000 7feff788b700 1 -- 172.21.15.29:0/1556750561 >> 172.21.15.29:0/1556750561 conn(0x7feff00c8bb0 msgr2=0x7feff00a8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:07.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.219+0000 7feff788b700 1 -- 172.21.15.29:0/1556750561 shutdown_connections 2024-01-24T13:59:07.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.219+0000 7feff788b700 1 -- 172.21.15.29:0/1556750561 wait complete. 2024-01-24T13:59:07.221 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 9 2024-01-24T13:59:07.234 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:59:07.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.362+0000 7fd464cb2700 1 Processor -- start 2024-01-24T13:59:07.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.362+0000 7fd464cb2700 1 -- start start 2024-01-24T13:59:07.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.363+0000 7fd464cb2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4601540a0 0x7fd4600c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.363+0000 7fd464cb2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4601550f0 0x7fd4600627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.363+0000 7fd464cb2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd460062ca0 0x7fd460157c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.363+0000 7fd464cb2700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4600a74f0 con 0x7fd4601540a0 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.363+0000 7fd464cb2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd460065360 con 0x7fd460062ca0 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.363+0000 7fd464cb2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600a6ce0 con 0x7fd4601550f0 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45e59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4601540a0 0x7fd4600c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45e59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4601540a0 0x7fd4600c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45030/0 (socket says 172.21.15.29:45030) 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45e59c700 1 -- 172.21.15.29:0/974654797 learned_addr learned my addr 172.21.15.29:0/974654797 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45dd9b700 1 --2- 172.21.15.29:0/974654797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4601550f0 0x7fd4600627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45ed9d700 1 --2- 172.21.15.29:0/974654797 >> v2:172.21.15.163:3300/0 conn(0x7fd460062ca0 0x7fd460157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45e59c700 1 -- 172.21.15.29:0/974654797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4601550f0 msgr2=0x7fd4600627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45e59c700 1 --2- 172.21.15.29:0/974654797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4601550f0 0x7fd4600627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45e59c700 1 -- 172.21.15.29:0/974654797 >> v2:172.21.15.163:3300/0 conn(0x7fd460062ca0 msgr2=0x7fd460157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45e59c700 1 --2- 172.21.15.29:0/974654797 >> v2:172.21.15.163:3300/0 conn(0x7fd460062ca0 0x7fd460157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45e59c700 1 -- 172.21.15.29:0/974654797 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4600636f0 con 0x7fd4601540a0 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45dd9b700 1 --2- 172.21.15.29:0/974654797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4601550f0 0x7fd4600627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:07.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45e59c700 1 --2- 172.21.15.29:0/974654797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4601540a0 0x7fd4600c4ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fd460066b80 tx=0x7fd4480090f0).ready entity=mon.0 client_cookie=ae0bddc6e1edb03e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45d59a700 1 -- 172.21.15.29:0/974654797 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44800b600 con 0x7fd4601540a0 2024-01-24T13:59:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45d59a700 1 -- 172.21.15.29:0/974654797 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44800b760 con 0x7fd4601540a0 2024-01-24T13:59:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.364+0000 7fd45d59a700 1 -- 172.21.15.29:0/974654797 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44800bae0 con 0x7fd4601540a0 2024-01-24T13:59:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.365+0000 7fd464cb2700 1 -- 172.21.15.29:0/974654797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4601540a0 msgr2=0x7fd4600c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.365+0000 7fd464cb2700 1 --2- 172.21.15.29:0/974654797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4601540a0 0x7fd4600c4ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fd460066b80 tx=0x7fd4480090f0).stop 2024-01-24T13:59:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.365+0000 7fd464cb2700 1 -- 172.21.15.29:0/974654797 shutdown_connections 2024-01-24T13:59:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.365+0000 7fd464cb2700 1 --2- 172.21.15.29:0/974654797 >> v2:172.21.15.163:3300/0 conn(0x7fd460062ca0 0x7fd460157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.365+0000 7fd464cb2700 1 --2- 172.21.15.29:0/974654797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4601540a0 0x7fd4600c4ca0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.365+0000 7fd464cb2700 1 --2- 172.21.15.29:0/974654797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4601550f0 0x7fd4600627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.365+0000 7fd464cb2700 1 -- 172.21.15.29:0/974654797 >> 172.21.15.29:0/974654797 conn(0x7fd4600c85c0 msgr2=0x7fd4600c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.365+0000 7fd464cb2700 1 -- 172.21.15.29:0/974654797 shutdown_connections 2024-01-24T13:59:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.365+0000 7fd464cb2700 1 -- 172.21.15.29:0/974654797 wait complete. 2024-01-24T13:59:07.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.367+0000 7fd464cb2700 1 Processor -- start 2024-01-24T13:59:07.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.367+0000 7fd464cb2700 1 -- start start 2024-01-24T13:59:07.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd464cb2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd460062ca0 0x7fd46014f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd464cb2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4601540a0 0x7fd4600accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd464cb2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4601550f0 0x7fd4600abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd464cb2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600ab7e0 con 0x7fd4601550f0 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd464cb2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46014fa60 con 0x7fd4601540a0 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd464cb2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600aabd0 con 0x7fd460062ca0 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd45ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4601550f0 0x7fd4600abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd45ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4601550f0 0x7fd4600abc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45038/0 (socket says 172.21.15.29:45038) 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd45ed9d700 1 -- 172.21.15.29:0/1249921121 learned_addr learned my addr 172.21.15.29:0/1249921121 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd45e59c700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd460062ca0 0x7fd46014f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd45dd9b700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4601540a0 0x7fd4600accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd45ed9d700 1 -- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd460062ca0 msgr2=0x7fd46014f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd45ed9d700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd460062ca0 0x7fd46014f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd45ed9d700 1 -- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4601540a0 msgr2=0x7fd4600accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd45ed9d700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4601540a0 0x7fd4600accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.368+0000 7fd45ed9d700 1 -- 172.21.15.29:0/1249921121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd448009370 con 0x7fd4601550f0 2024-01-24T13:59:07.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.369+0000 7fd45ed9d700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4601550f0 0x7fd4600abc70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fd458007f20 tx=0x7fd458009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:07.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.369+0000 7fd4567fc700 1 -- 172.21.15.29:0/1249921121 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45800eb90 con 0x7fd4601550f0 2024-01-24T13:59:07.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.369+0000 7fd4567fc700 1 -- 172.21.15.29:0/1249921121 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45800ecf0 con 0x7fd4601550f0 2024-01-24T13:59:07.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.369+0000 7fd464cb2700 1 -- 172.21.15.29:0/1249921121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4600aaeb0 con 0x7fd4601550f0 2024-01-24T13:59:07.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.369+0000 7fd4567fc700 1 -- 172.21.15.29:0/1249921121 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd458014630 con 0x7fd4601550f0 2024-01-24T13:59:07.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.369+0000 7fd464cb2700 1 -- 172.21.15.29:0/1249921121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4600ab470 con 0x7fd4601550f0 2024-01-24T13:59:07.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.370+0000 7fd464cb2700 1 -- 172.21.15.29:0/1249921121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4600475e0 con 0x7fd4601550f0 2024-01-24T13:59:07.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.370+0000 7fd4567fc700 1 -- 172.21.15.29:0/1249921121 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd45801d3b0 con 0x7fd4601550f0 2024-01-24T13:59:07.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.371+0000 7fd4567fc700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd44005f150 0x7fd440061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.371+0000 7fd4567fc700 1 -- 172.21.15.29:0/1249921121 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 7079+0+0 (secure 0 0 0) 0x7fd45807a770 con 0x7fd4601550f0 2024-01-24T13:59:07.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.371+0000 7fd45e59c700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd44005f150 0x7fd440061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.371+0000 7fd45e59c700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd44005f150 0x7fd440061610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fd448009130 tx=0x7fd448017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:07.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.375+0000 7fd4567fc700 1 -- 172.21.15.29:0/1249921121 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd45804b5e0 con 0x7fd4601550f0 2024-01-24T13:59:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.599+0000 7fd464cb2700 1 -- 172.21.15.29:0/1249921121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4600ab920 con 0x7fd4601550f0 2024-01-24T13:59:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.599+0000 7fd4567fc700 1 -- 172.21.15.29:0/1249921121 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4491 (secure 0 0 0) 0x7fd45800ee60 con 0x7fd4601550f0 2024-01-24T13:59:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:07.600 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2038321939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2038321939},{"type":"v1","addr":"172.21.15.163:6836","nonce":2038321939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3675672874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3675672874},{"type":"v1","addr":"172.21.15.163:6837","nonce":3675672874}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1532435957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1532435957},{"type":"v1","addr":"172.21.15.29:6834","nonce":1532435957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:58:19.190134+0000","modified":"2024-01-24T13:58:31.231527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1328130656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1328130656},{"type":"v1","addr":"172.21.15.29:6835","nonce":1328130656}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-24T13:59:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.601+0000 7fd464cb2700 1 -- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd44005f150 msgr2=0x7fd440061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.601+0000 7fd464cb2700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd44005f150 0x7fd440061610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fd448009130 tx=0x7fd448017000).stop 2024-01-24T13:59:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.601+0000 7fd464cb2700 1 -- 172.21.15.29:0/1249921121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4601550f0 msgr2=0x7fd4600abc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.601+0000 7fd464cb2700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4601550f0 0x7fd4600abc70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fd458007f20 tx=0x7fd458009860).stop 2024-01-24T13:59:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.601+0000 7fd464cb2700 1 -- 172.21.15.29:0/1249921121 shutdown_connections 2024-01-24T13:59:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.601+0000 7fd464cb2700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd44005f150 0x7fd440061610 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.601+0000 7fd464cb2700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4601550f0 0x7fd4600abc70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.601+0000 7fd464cb2700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd460062ca0 0x7fd46014f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.601+0000 7fd464cb2700 1 --2- 172.21.15.29:0/1249921121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4601540a0 0x7fd4600accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.601+0000 7fd464cb2700 1 -- 172.21.15.29:0/1249921121 >> 172.21.15.29:0/1249921121 conn(0x7fd4600c85c0 msgr2=0x7fd460069240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.601+0000 7fd464cb2700 1 -- 172.21.15.29:0/1249921121 shutdown_connections 2024-01-24T13:59:07.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.601+0000 7fd464cb2700 1 -- 172.21.15.29:0/1249921121 wait complete. 2024-01-24T13:59:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 9 2024-01-24T13:59:07.615 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:59:07.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.749+0000 7f9164485700 1 Processor -- start 2024-01-24T13:59:07.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.749+0000 7f9164485700 1 -- start start 2024-01-24T13:59:07.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.750+0000 7f9164485700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f915c1540a0 0x7f915c0c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.750+0000 7f9164485700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f915c1550f0 0x7f915c0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.750+0000 7f9164485700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c065360 con 0x7f915c062ca0 2024-01-24T13:59:07.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.750+0000 7f9164485700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f915c0a74f0 con 0x7f915c1540a0 2024-01-24T13:59:07.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.750+0000 7f9164485700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915c0a6ce0 con 0x7f915c1550f0 2024-01-24T13:59:07.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f9162221700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f915c1540a0 0x7f915c0c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f9162221700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f915c1540a0 0x7f915c0c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45044/0 (socket says 172.21.15.29:45044) 2024-01-24T13:59:07.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f9162a22700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f915c062ca0 0x7f915c157c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59602/0 (socket says 172.21.15.29:59602) 2024-01-24T13:59:07.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f9162221700 1 -- 172.21.15.29:0/23667426 learned_addr learned my addr 172.21.15.29:0/23667426 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:07.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f9161a20700 1 --2- 172.21.15.29:0/23667426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f915c1550f0 0x7f915c0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f9162221700 1 -- 172.21.15.29:0/23667426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f915c1550f0 msgr2=0x7f915c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:07.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f9162221700 1 --2- 172.21.15.29:0/23667426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f915c1550f0 0x7f915c0627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f9162221700 1 -- 172.21.15.29:0/23667426 >> v1:172.21.15.29:6789/0 conn(0x7f915c062ca0 legacy=0x7f915c157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f916121f700 1 -- 172.21.15.29:0/23667426 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645481310 0 0) 0x7f915c065360 con 0x7f915c062ca0 2024-01-24T13:59:07.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f9162221700 1 -- 172.21.15.29:0/23667426 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c0636f0 con 0x7f915c1540a0 2024-01-24T13:59:07.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f9162221700 1 --2- 172.21.15.29:0/23667426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f915c1540a0 0x7f915c0c4ca0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f914c006fd0 tx=0x7f914c0090f0).ready entity=mon.0 client_cookie=f428d1d7f731fd05 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:07.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f916121f700 1 -- 172.21.15.29:0/23667426 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c00b580 con 0x7f915c1540a0 2024-01-24T13:59:07.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f916121f700 1 -- 172.21.15.29:0/23667426 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c00b6e0 con 0x7f915c1540a0 2024-01-24T13:59:07.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.751+0000 7f916121f700 1 -- 172.21.15.29:0/23667426 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c00ba60 con 0x7f915c1540a0 2024-01-24T13:59:07.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.752+0000 7f9164485700 1 -- 172.21.15.29:0/23667426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f915c1540a0 msgr2=0x7f915c0c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.752+0000 7f9164485700 1 --2- 172.21.15.29:0/23667426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f915c1540a0 0x7f915c0c4ca0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f914c006fd0 tx=0x7f914c0090f0).stop 2024-01-24T13:59:07.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.752+0000 7f9164485700 1 -- 172.21.15.29:0/23667426 shutdown_connections 2024-01-24T13:59:07.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.752+0000 7f9164485700 1 --2- 172.21.15.29:0/23667426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f915c1540a0 0x7f915c0c4ca0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.752+0000 7f9164485700 1 --2- 172.21.15.29:0/23667426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f915c1550f0 0x7f915c0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.752+0000 7f9164485700 1 -- 172.21.15.29:0/23667426 >> 172.21.15.29:0/23667426 conn(0x7f915c0c85c0 msgr2=0x7f915c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:07.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.752+0000 7f9164485700 1 -- 172.21.15.29:0/23667426 shutdown_connections 2024-01-24T13:59:07.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.752+0000 7f9164485700 1 -- 172.21.15.29:0/23667426 wait complete. 2024-01-24T13:59:07.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.753+0000 7f9164485700 1 Processor -- start 2024-01-24T13:59:07.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.753+0000 7f9164485700 1 -- start start 2024-01-24T13:59:07.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.754+0000 7f9164485700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f915c062ca0 0x7f915c0c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.754+0000 7f9164485700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f915c1540a0 0x7f915c0ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.754+0000 7f9164485700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f915c1550f0 0x7f915c0ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.754+0000 7f9164485700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c0c1dd0 con 0x7f915c1550f0 2024-01-24T13:59:07.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.754+0000 7f9164485700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c0ada50 con 0x7f915c1540a0 2024-01-24T13:59:07.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.754+0000 7f9164485700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915c0c1250 con 0x7f915c062ca0 2024-01-24T13:59:07.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.754+0000 7f9162a22700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f915c1550f0 0x7f915c0ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.754+0000 7f9162a22700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f915c1550f0 0x7f915c0ac940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45052/0 (socket says 172.21.15.29:45052) 2024-01-24T13:59:07.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.754+0000 7f9162a22700 1 -- 172.21.15.29:0/1105149153 learned_addr learned my addr 172.21.15.29:0/1105149153 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:07.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.754+0000 7f9162221700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f915c062ca0 0x7f915c0c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.754+0000 7f9161a20700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f915c1540a0 0x7f915c0ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.755+0000 7f9162a22700 1 -- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f915c062ca0 msgr2=0x7f915c0c0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.755+0000 7f9162a22700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f915c062ca0 0x7f915c0c0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.755+0000 7f9162a22700 1 -- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f915c1540a0 msgr2=0x7f915c0ad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.755+0000 7f9162a22700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f915c1540a0 0x7f915c0ad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.755+0000 7f9162a22700 1 -- 172.21.15.29:0/1105149153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914c009370 con 0x7f915c1550f0 2024-01-24T13:59:07.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.755+0000 7f9161a20700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f915c1540a0 0x7f915c0ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:07.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.755+0000 7f9162a22700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f915c1550f0 0x7f915c0ac940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f915400c7e0 tx=0x7f9154007f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:07.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.755+0000 7f91527fc700 1 -- 172.21.15.29:0/1105149153 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154005c70 con 0x7f915c1550f0 2024-01-24T13:59:07.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.755+0000 7f91527fc700 1 -- 172.21.15.29:0/1105149153 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9154005dd0 con 0x7f915c1550f0 2024-01-24T13:59:07.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.755+0000 7f91527fc700 1 -- 172.21.15.29:0/1105149153 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154007320 con 0x7f915c1550f0 2024-01-24T13:59:07.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.755+0000 7f9164485700 1 -- 172.21.15.29:0/1105149153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915c0c1530 con 0x7f915c1550f0 2024-01-24T13:59:07.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.755+0000 7f9164485700 1 -- 172.21.15.29:0/1105149153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915c0c1af0 con 0x7f915c1550f0 2024-01-24T13:59:07.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.756+0000 7f9164485700 1 -- 172.21.15.29:0/1105149153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915c0475e0 con 0x7f915c1550f0 2024-01-24T13:59:07.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.756+0000 7f91527fc700 1 -- 172.21.15.29:0/1105149153 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9154007710 con 0x7f915c1550f0 2024-01-24T13:59:07.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.757+0000 7f91527fc700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f914005f150 0x7f9140061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:07.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.757+0000 7f91527fc700 1 -- 172.21.15.29:0/1105149153 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 7079+0+0 (secure 0 0 0) 0x7f915407a590 con 0x7f915c1550f0 2024-01-24T13:59:07.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.757+0000 7f9162221700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f914005f150 0x7f9140061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:07.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.758+0000 7f9162221700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f914005f150 0x7f9140061610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f914c009fd0 tx=0x7f914c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:07.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.761+0000 7f91527fc700 1 -- 172.21.15.29:0/1105149153 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f915404b440 con 0x7f915c1550f0 2024-01-24T13:59:07.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.948+0000 7f9164485700 1 -- 172.21.15.29:0/1105149153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f915c0c5700 con 0x7f915c1550f0 2024-01-24T13:59:07.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.949+0000 7f91527fc700 1 -- 172.21.15.29:0/1105149153 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19083 (secure 0 0 0) 0x7f915404ea60 con 0x7f915c1550f0 2024-01-24T13:59:07.950 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:07.950 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":28,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:59:07.650717+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:58:16.949817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-01-24T13:58:18.174460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2024-01-24T13:58:21.232882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.3f","mappings":[{"from":3,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:59:07.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.951+0000 7f9164485700 1 -- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f914005f150 msgr2=0x7f9140061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.951+0000 7f9164485700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f914005f150 0x7f9140061610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f914c009fd0 tx=0x7f914c017000).stop 2024-01-24T13:59:07.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.951+0000 7f9164485700 1 -- 172.21.15.29:0/1105149153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f915c1550f0 msgr2=0x7f915c0ac940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:07.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.951+0000 7f9164485700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f915c1550f0 0x7f915c0ac940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f915400c7e0 tx=0x7f9154007f60).stop 2024-01-24T13:59:07.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.952+0000 7f9164485700 1 -- 172.21.15.29:0/1105149153 shutdown_connections 2024-01-24T13:59:07.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.952+0000 7f9164485700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f914005f150 0x7f9140061610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.952+0000 7f9164485700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f915c1550f0 0x7f915c0ac940 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.952+0000 7f9164485700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f915c062ca0 0x7f915c0c0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.952+0000 7f9164485700 1 --2- 172.21.15.29:0/1105149153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f915c1540a0 0x7f915c0ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:07.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.952+0000 7f9164485700 1 -- 172.21.15.29:0/1105149153 >> 172.21.15.29:0/1105149153 conn(0x7f915c0c85c0 msgr2=0x7f915c0658f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:07.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.952+0000 7f9164485700 1 -- 172.21.15.29:0/1105149153 shutdown_connections 2024-01-24T13:59:07.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:07.952+0000 7f9164485700 1 -- 172.21.15.29:0/1105149153 wait complete. 2024-01-24T13:59:07.966 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-24T13:59:08.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.096+0000 7f8851ba0700 1 Processor -- start 2024-01-24T13:59:08.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.096+0000 7f8851ba0700 1 -- start start 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f8851ba0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c151aa0 0x7f884c0a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f8851ba0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f884c151e80 0x7f884c0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f8851ba0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c065360 con 0x7f884c0a71d0 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f8851ba0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884c0bcab0 con 0x7f884c151e80 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f8851ba0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0bcd90 con 0x7f884c151aa0 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f884affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f884c151e80 0x7f884c0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f884affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f884c151e80 0x7f884c0627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45064/0 (socket says 172.21.15.29:45064) 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f884affd700 1 -- 172.21.15.29:0/2828495634 learned_addr learned my addr 172.21.15.29:0/2828495634 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f884b7fe700 1 --2- 172.21.15.29:0/2828495634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c151aa0 0x7f884c0a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f884affd700 1 -- 172.21.15.29:0/2828495634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c151aa0 msgr2=0x7f884c0a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f884affd700 1 --2- 172.21.15.29:0/2828495634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c151aa0 0x7f884c0a6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f884affd700 1 -- 172.21.15.29:0/2828495634 >> v1:172.21.15.163:6789/0 conn(0x7f884c0a71d0 legacy=0x7f884c157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:08.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.097+0000 7f884affd700 1 -- 172.21.15.29:0/2828495634 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c0bd070 con 0x7f884c151e80 2024-01-24T13:59:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.098+0000 7f884affd700 1 --2- 172.21.15.29:0/2828495634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c151e80 0x7f884c0627b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f8840006fd0 tx=0x7f88400090f0).ready entity=mon.0 client_cookie=1702551c583a88fe server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.098+0000 7f884a7fc700 1 -- 172.21.15.29:0/2828495634 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884000b600 con 0x7f884c151e80 2024-01-24T13:59:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.098+0000 7f884a7fc700 1 -- 172.21.15.29:0/2828495634 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884000b760 con 0x7f884c151e80 2024-01-24T13:59:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.098+0000 7f884a7fc700 1 -- 172.21.15.29:0/2828495634 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884000bae0 con 0x7f884c151e80 2024-01-24T13:59:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.098+0000 7f8851ba0700 1 -- 172.21.15.29:0/2828495634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c151e80 msgr2=0x7f884c0627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.098+0000 7f8851ba0700 1 --2- 172.21.15.29:0/2828495634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c151e80 0x7f884c0627b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f8840006fd0 tx=0x7f88400090f0).stop 2024-01-24T13:59:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.098+0000 7f8851ba0700 1 -- 172.21.15.29:0/2828495634 shutdown_connections 2024-01-24T13:59:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.098+0000 7f8851ba0700 1 --2- 172.21.15.29:0/2828495634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c151aa0 0x7f884c0a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.098+0000 7f8851ba0700 1 --2- 172.21.15.29:0/2828495634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c151e80 0x7f884c0627b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.098+0000 7f8851ba0700 1 -- 172.21.15.29:0/2828495634 >> 172.21.15.29:0/2828495634 conn(0x7f884c0c8580 msgr2=0x7f884c0c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.098+0000 7f8851ba0700 1 -- 172.21.15.29:0/2828495634 shutdown_connections 2024-01-24T13:59:08.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.098+0000 7f8851ba0700 1 -- 172.21.15.29:0/2828495634 wait complete. 2024-01-24T13:59:08.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.100+0000 7f8851ba0700 1 Processor -- start 2024-01-24T13:59:08.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.100+0000 7f8851ba0700 1 -- start start 2024-01-24T13:59:08.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f8851ba0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c0a71d0 0x7f884c167e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:08.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f8851ba0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c151aa0 0x7f884c14d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:08.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f8851ba0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c151e80 0x7f884c150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:08.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f8851ba0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c168720 con 0x7f884c0a71d0 2024-01-24T13:59:08.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f8851ba0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c168a00 con 0x7f884c151aa0 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f8851ba0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c168ce0 con 0x7f884c151e80 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f884b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c0a71d0 0x7f884c167e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f884b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c0a71d0 0x7f884c167e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45078/0 (socket says 172.21.15.29:45078) 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f884b7fe700 1 -- 172.21.15.29:0/3412348981 learned_addr learned my addr 172.21.15.29:0/3412348981 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f884affd700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c151aa0 0x7f884c14d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f884bfff700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c151e80 0x7f884c150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f884b7fe700 1 -- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c151e80 msgr2=0x7f884c150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f884b7fe700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c151e80 0x7f884c150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f884b7fe700 1 -- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c151aa0 msgr2=0x7f884c14d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f884b7fe700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c151aa0 0x7f884c14d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f884b7fe700 1 -- 172.21.15.29:0/3412348981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8840009370 con 0x7f884c0a71d0 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f884b7fe700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c0a71d0 0x7f884c167e60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f883c00d890 tx=0x7f883c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f88337fe700 1 -- 172.21.15.29:0/3412348981 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0125f0 con 0x7f884c0a71d0 2024-01-24T13:59:08.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f88337fe700 1 -- 172.21.15.29:0/3412348981 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c012750 con 0x7f884c0a71d0 2024-01-24T13:59:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f8851ba0700 1 -- 172.21.15.29:0/3412348981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c150430 con 0x7f884c0a71d0 2024-01-24T13:59:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.101+0000 7f88337fe700 1 -- 172.21.15.29:0/3412348981 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c012ad0 con 0x7f884c0a71d0 2024-01-24T13:59:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.102+0000 7f8851ba0700 1 -- 172.21.15.29:0/3412348981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c168350 con 0x7f884c0a71d0 2024-01-24T13:59:08.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.103+0000 7f8851ba0700 1 -- 172.21.15.29:0/3412348981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884c1508d0 con 0x7f884c0a71d0 2024-01-24T13:59:08.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.103+0000 7f88337fe700 1 -- 172.21.15.29:0/3412348981 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f883c013480 con 0x7f884c0a71d0 2024-01-24T13:59:08.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.103+0000 7f88337fe700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f883805f150 0x7f8838061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:08.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.103+0000 7f88337fe700 1 -- 172.21.15.29:0/3412348981 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 7079+0+0 (secure 0 0 0) 0x7f883c07a3b0 con 0x7f884c0a71d0 2024-01-24T13:59:08.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.104+0000 7f884affd700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f883805f150 0x7f8838061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:08.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.104+0000 7f884affd700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f883805f150 0x7f8838061610 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f884c06c700 tx=0x7f8840017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:08.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.108+0000 7f88337fe700 1 -- 172.21.15.29:0/3412348981 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f883c04b260 con 0x7f884c0a71d0 2024-01-24T13:59:08.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.333+0000 7f8851ba0700 1 -- 172.21.15.29:0/3412348981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f884c150a80 con 0x7f884c0a71d0 2024-01-24T13:59:08.822 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-01-24T13:59:08.821+0000 7fe09bfc3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-24T13:59:08.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.827+0000 7f88337fe700 1 -- 172.21.15.29:0/3412348981 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f883c04e880 con 0x7f884c0a71d0 2024-01-24T13:59:08.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.830+0000 7f8851ba0700 1 -- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f883805f150 msgr2=0x7f8838061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:08.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.830+0000 7f8851ba0700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f883805f150 0x7f8838061610 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f884c06c700 tx=0x7f8840017000).stop 2024-01-24T13:59:08.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.830+0000 7f8851ba0700 1 -- 172.21.15.29:0/3412348981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c0a71d0 msgr2=0x7f884c167e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:08.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.830+0000 7f8851ba0700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c0a71d0 0x7f884c167e60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f883c00d890 tx=0x7f883c008040).stop 2024-01-24T13:59:08.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.830+0000 7f8851ba0700 1 -- 172.21.15.29:0/3412348981 shutdown_connections 2024-01-24T13:59:08.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.830+0000 7f8851ba0700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c151e80 0x7f884c150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.830+0000 7f8851ba0700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f883805f150 0x7f8838061610 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.830+0000 7f8851ba0700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c0a71d0 0x7f884c167e60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.830+0000 7f8851ba0700 1 --2- 172.21.15.29:0/3412348981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c151aa0 0x7f884c14d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.830+0000 7f8851ba0700 1 -- 172.21.15.29:0/3412348981 >> 172.21.15.29:0/3412348981 conn(0x7f884c0c8580 msgr2=0x7f884c0c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.830+0000 7f8851ba0700 1 -- 172.21.15.29:0/3412348981 shutdown_connections 2024-01-24T13:59:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.830+0000 7f8851ba0700 1 -- 172.21.15.29:0/3412348981 wait complete. 2024-01-24T13:59:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-24T13:59:08.833 INFO:tasks.ceph.mds.c.smithi029.stderr: -3886> 2024-01-24T13:58:24.878+0000 7fe714eb4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T13:59:08.850 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-24T13:59:08.867 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T13:59:08.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.985+0000 7ff8251d8700 1 Processor -- start 2024-01-24T13:59:08.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.986+0000 7ff8251d8700 1 -- start start 2024-01-24T13:59:08.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.986+0000 7ff8251d8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff820151c30 0x7ff820155320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:08.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.986+0000 7ff8251d8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8200a6d20 0x7ff8200627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:08.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.986+0000 7ff8251d8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8200bca30 con 0x7ff820062ca0 2024-01-24T13:59:08.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff8251d8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8200c8410 con 0x7ff820151c30 2024-01-24T13:59:08.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff8251d8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200bd240 con 0x7ff8200a6d20 2024-01-24T13:59:08.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff81ed9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff820151c30 0x7ff820155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:08.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff81ed9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff820151c30 0x7ff820155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45092/0 (socket says 172.21.15.29:45092) 2024-01-24T13:59:08.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff81ed9d700 1 -- 172.21.15.29:0/1032559938 learned_addr learned my addr 172.21.15.29:0/1032559938 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:08.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff81e59c700 1 --2- 172.21.15.29:0/1032559938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8200a6d20 0x7ff8200627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:08.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff81ed9d700 1 -- 172.21.15.29:0/1032559938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8200a6d20 msgr2=0x7ff8200627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:08.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff81ed9d700 1 --2- 172.21.15.29:0/1032559938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8200a6d20 0x7ff8200627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff81ed9d700 1 -- 172.21.15.29:0/1032559938 >> v1:172.21.15.163:6789/0 conn(0x7ff820062ca0 legacy=0x7ff820157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:08.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff81ed9d700 1 -- 172.21.15.29:0/1032559938 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff820063110 con 0x7ff820151c30 2024-01-24T13:59:08.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff81ed9d700 1 --2- 172.21.15.29:0/1032559938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff820151c30 0x7ff820155320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7ff810006fd0 tx=0x7ff8100090f0).ready entity=mon.0 client_cookie=4c7b34c0279cd48c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:08.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff81dd9b700 1 -- 172.21.15.29:0/1032559938 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81000b600 con 0x7ff820151c30 2024-01-24T13:59:08.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff81dd9b700 1 -- 172.21.15.29:0/1032559938 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81000b760 con 0x7ff820151c30 2024-01-24T13:59:08.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.987+0000 7ff81dd9b700 1 -- 172.21.15.29:0/1032559938 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81000bb00 con 0x7ff820151c30 2024-01-24T13:59:08.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.988+0000 7ff8251d8700 1 -- 172.21.15.29:0/1032559938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff820151c30 msgr2=0x7ff820155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:08.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.988+0000 7ff8251d8700 1 --2- 172.21.15.29:0/1032559938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff820151c30 0x7ff820155320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7ff810006fd0 tx=0x7ff8100090f0).stop 2024-01-24T13:59:08.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.988+0000 7ff8251d8700 1 -- 172.21.15.29:0/1032559938 shutdown_connections 2024-01-24T13:59:08.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.988+0000 7ff8251d8700 1 --2- 172.21.15.29:0/1032559938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff820151c30 0x7ff820155320 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.988+0000 7ff8251d8700 1 --2- 172.21.15.29:0/1032559938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8200a6d20 0x7ff8200627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.988+0000 7ff8251d8700 1 -- 172.21.15.29:0/1032559938 >> 172.21.15.29:0/1032559938 conn(0x7ff820064390 msgr2=0x7ff8200c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:08.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.988+0000 7ff8251d8700 1 -- 172.21.15.29:0/1032559938 shutdown_connections 2024-01-24T13:59:08.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.988+0000 7ff8251d8700 1 -- 172.21.15.29:0/1032559938 wait complete. 2024-01-24T13:59:08.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.989+0000 7ff8251d8700 1 Processor -- start 2024-01-24T13:59:08.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.989+0000 7ff8251d8700 1 -- start start 2024-01-24T13:59:08.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.990+0000 7ff8251d8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff820062ca0 0x7ff820167f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:08.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.990+0000 7ff8251d8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8200a6d20 0x7ff8200a7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:08.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.990+0000 7ff8251d8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff820151c30 0x7ff82014f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:08.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.990+0000 7ff8251d8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201687b0 con 0x7ff8200a6d20 2024-01-24T13:59:08.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.990+0000 7ff8251d8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff820168a90 con 0x7ff820151c30 2024-01-24T13:59:08.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.990+0000 7ff8251d8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200a76d0 con 0x7ff820062ca0 2024-01-24T13:59:08.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.990+0000 7ff81e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8200a6d20 0x7ff8200a7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:08.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.990+0000 7ff81e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8200a6d20 0x7ff8200a7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45100/0 (socket says 172.21.15.29:45100) 2024-01-24T13:59:08.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.990+0000 7ff81e59c700 1 -- 172.21.15.29:0/2896818611 learned_addr learned my addr 172.21.15.29:0/2896818611 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:08.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff81ed9d700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff820062ca0 0x7ff820167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff81f59e700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff820151c30 0x7ff82014f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff81e59c700 1 -- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff820062ca0 msgr2=0x7ff820167f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff81e59c700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff820062ca0 0x7ff820167f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff81e59c700 1 -- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff820151c30 msgr2=0x7ff82014f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff81e59c700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff820151c30 0x7ff82014f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff81e59c700 1 -- 172.21.15.29:0/2896818611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff810009370 con 0x7ff8200a6d20 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff81f59e700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff820151c30 0x7ff82014f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff81e59c700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8200a6d20 0x7ff8200a7ed0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7ff81400d890 tx=0x7ff814008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff806ffd700 1 -- 172.21.15.29:0/2896818611 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8140125f0 con 0x7ff8200a6d20 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff806ffd700 1 -- 172.21.15.29:0/2896818611 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff814012750 con 0x7ff8200a6d20 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff8251d8700 1 -- 172.21.15.29:0/2896818611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8200a79b0 con 0x7ff8200a6d20 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff806ffd700 1 -- 172.21.15.29:0/2896818611 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814012ad0 con 0x7ff8200a6d20 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.991+0000 7ff8251d8700 1 -- 172.21.15.29:0/2896818611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82014fe30 con 0x7ff8200a6d20 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.992+0000 7ff8251d8700 1 -- 172.21.15.29:0/2896818611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8200a7ba0 con 0x7ff8200a6d20 2024-01-24T13:59:08.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.993+0000 7ff806ffd700 1 -- 172.21.15.29:0/2896818611 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff814013480 con 0x7ff8200a6d20 2024-01-24T13:59:08.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.993+0000 7ff806ffd700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff80c05f120 0x7ff80c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:08.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.993+0000 7ff806ffd700 1 -- 172.21.15.29:0/2896818611 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 7165+0+0 (secure 0 0 0) 0x7ff81407a400 con 0x7ff8200a6d20 2024-01-24T13:59:08.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.993+0000 7ff81ed9d700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff80c05f120 0x7ff80c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:08.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.994+0000 7ff81ed9d700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff80c05f120 0x7ff80c0615e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7ff81000b3c0 tx=0x7ff810017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:08.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:08.997+0000 7ff806ffd700 1 -- 172.21.15.29:0/2896818611 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff81404b260 con 0x7ff8200a6d20 2024-01-24T13:59:09.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.224+0000 7ff8251d8700 1 -- 172.21.15.29:0/2896818611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff8200475e0 con 0x7ff8200a6d20 2024-01-24T13:59:09.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.831+0000 7ff806ffd700 1 -- 172.21.15.29:0/2896818611 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff81404e880 con 0x7ff8200a6d20 2024-01-24T13:59:09.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.834+0000 7ff8251d8700 1 -- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff80c05f120 msgr2=0x7ff80c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:09.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.834+0000 7ff8251d8700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff80c05f120 0x7ff80c0615e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7ff81000b3c0 tx=0x7ff810017000).stop 2024-01-24T13:59:09.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.834+0000 7ff8251d8700 1 -- 172.21.15.29:0/2896818611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8200a6d20 msgr2=0x7ff8200a7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:09.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.834+0000 7ff8251d8700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8200a6d20 0x7ff8200a7ed0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7ff81400d890 tx=0x7ff814008040).stop 2024-01-24T13:59:09.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.834+0000 7ff8251d8700 1 -- 172.21.15.29:0/2896818611 shutdown_connections 2024-01-24T13:59:09.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.834+0000 7ff8251d8700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff820151c30 0x7ff82014f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:09.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.835+0000 7ff8251d8700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff820062ca0 0x7ff820167f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:09.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.835+0000 7ff8251d8700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff80c05f120 0x7ff80c0615e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:09.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.835+0000 7ff8251d8700 1 --2- 172.21.15.29:0/2896818611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8200a6d20 0x7ff8200a7ed0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:09.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.835+0000 7ff8251d8700 1 -- 172.21.15.29:0/2896818611 >> 172.21.15.29:0/2896818611 conn(0x7ff820064390 msgr2=0x7ff8200c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:09.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.835+0000 7ff8251d8700 1 -- 172.21.15.29:0/2896818611 shutdown_connections 2024-01-24T13:59:09.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:09.835+0000 7ff8251d8700 1 -- 172.21.15.29:0/2896818611 wait complete. 2024-01-24T13:59:09.850 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-24T13:59:10.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.031+0000 7f5f969a7700 1 Processor -- start 2024-01-24T13:59:10.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.031+0000 7f5f969a7700 1 -- start start 2024-01-24T13:59:10.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.031+0000 7f5f969a7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5f90152540 0x7f5f901529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:10.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.031+0000 7f5f969a7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5f90159da0 0x7f5f9015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:10.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.031+0000 7f5f969a7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f90153300 con 0x7f5f90066450 2024-01-24T13:59:10.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f969a7700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f90067b10 con 0x7f5f90159da0 2024-01-24T13:59:10.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f969a7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f90067df0 con 0x7f5f90152540 2024-01-24T13:59:10.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f94f44700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5f90159da0 0x7f5f9015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:10.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f94f44700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5f90159da0 0x7f5f9015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45108/0 (socket says 172.21.15.29:45108) 2024-01-24T13:59:10.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f94f44700 1 -- 172.21.15.29:0/4223542367 learned_addr learned my addr 172.21.15.29:0/4223542367 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:10.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f8f7fe700 1 --2- 172.21.15.29:0/4223542367 >> v2:172.21.15.163:3300/0 conn(0x7f5f90152540 0x7f5f901529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:10.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f94f44700 1 -- 172.21.15.29:0/4223542367 >> v2:172.21.15.163:3300/0 conn(0x7f5f90152540 msgr2=0x7f5f901529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:10.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f94f44700 1 --2- 172.21.15.29:0/4223542367 >> v2:172.21.15.163:3300/0 conn(0x7f5f90152540 0x7f5f901529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:10.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f94f44700 1 -- 172.21.15.29:0/4223542367 >> v1:172.21.15.163:6789/0 conn(0x7f5f90066450 legacy=0x7f5f90066830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:10.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f94f44700 1 -- 172.21.15.29:0/4223542367 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900680d0 con 0x7f5f90159da0 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f8f7fe700 1 --2- 172.21.15.29:0/4223542367 >> v2:172.21.15.163:3300/0 conn(0x7f5f90152540 0x7f5f901529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f94f44700 1 --2- 172.21.15.29:0/4223542367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f90159da0 0x7f5f9015c190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f5f90069ef0 tx=0x7f5f88009840).ready entity=mon.0 client_cookie=89502f26b950ad05 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f8effd700 1 -- 172.21.15.29:0/4223542367 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8800e070 con 0x7f5f90159da0 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f8effd700 1 -- 172.21.15.29:0/4223542367 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8800ba50 con 0x7f5f90159da0 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.032+0000 7f5f8effd700 1 -- 172.21.15.29:0/4223542367 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8800bdd0 con 0x7f5f90159da0 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.033+0000 7f5f969a7700 1 -- 172.21.15.29:0/4223542367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f90159da0 msgr2=0x7f5f9015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.033+0000 7f5f969a7700 1 --2- 172.21.15.29:0/4223542367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f90159da0 0x7f5f9015c190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f5f90069ef0 tx=0x7f5f88009840).stop 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.033+0000 7f5f969a7700 1 -- 172.21.15.29:0/4223542367 shutdown_connections 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.033+0000 7f5f969a7700 1 --2- 172.21.15.29:0/4223542367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f90159da0 0x7f5f9015c190 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.033+0000 7f5f969a7700 1 --2- 172.21.15.29:0/4223542367 >> v2:172.21.15.163:3300/0 conn(0x7f5f90152540 0x7f5f901529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.033+0000 7f5f969a7700 1 -- 172.21.15.29:0/4223542367 >> 172.21.15.29:0/4223542367 conn(0x7f5f900c8bd0 msgr2=0x7f5f900c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.033+0000 7f5f969a7700 1 -- 172.21.15.29:0/4223542367 shutdown_connections 2024-01-24T13:59:10.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.033+0000 7f5f969a7700 1 -- 172.21.15.29:0/4223542367 wait complete. 2024-01-24T13:59:10.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.034+0000 7f5f969a7700 1 Processor -- start 2024-01-24T13:59:10.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.034+0000 7f5f969a7700 1 -- start start 2024-01-24T13:59:10.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.035+0000 7f5f969a7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f90066450 0x7f5f901625f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:10.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.035+0000 7f5f969a7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f90152540 0x7f5f90155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:10.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.035+0000 7f5f969a7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f90159da0 0x7f5f90138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:10.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.035+0000 7f5f969a7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90154020 con 0x7f5f90152540 2024-01-24T13:59:10.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.035+0000 7f5f969a7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90154300 con 0x7f5f90159da0 2024-01-24T13:59:10.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.035+0000 7f5f969a7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f901545e0 con 0x7f5f90066450 2024-01-24T13:59:10.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.035+0000 7f5f8f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f90152540 0x7f5f90155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:10.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.035+0000 7f5f8f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f90152540 0x7f5f90155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45114/0 (socket says 172.21.15.29:45114) 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.035+0000 7f5f8f7fe700 1 -- 172.21.15.29:0/4196493162 learned_addr learned my addr 172.21.15.29:0/4196493162 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.035+0000 7f5f8ffff700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f90066450 0x7f5f901625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.036+0000 7f5f94f44700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f90159da0 0x7f5f90138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.036+0000 7f5f8f7fe700 1 -- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f90066450 msgr2=0x7f5f901625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.036+0000 7f5f8f7fe700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f90066450 0x7f5f901625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.036+0000 7f5f8f7fe700 1 -- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f90159da0 msgr2=0x7f5f90138a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.036+0000 7f5f8f7fe700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f90159da0 0x7f5f90138a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.036+0000 7f5f8f7fe700 1 -- 172.21.15.29:0/4196493162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f88009ae0 con 0x7f5f90152540 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.036+0000 7f5f8f7fe700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f90152540 0x7f5f90155c50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f5f8400d860 tx=0x7f5f84008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.036+0000 7f5f77fff700 1 -- 172.21.15.29:0/4196493162 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8400f520 con 0x7f5f90152540 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.036+0000 7f5f77fff700 1 -- 172.21.15.29:0/4196493162 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8400f680 con 0x7f5f90152540 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.036+0000 7f5f77fff700 1 -- 172.21.15.29:0/4196493162 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8400fa00 con 0x7f5f90152540 2024-01-24T13:59:10.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.036+0000 7f5f969a7700 1 -- 172.21.15.29:0/4196493162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f901548c0 con 0x7f5f90152540 2024-01-24T13:59:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.036+0000 7f5f969a7700 1 -- 172.21.15.29:0/4196493162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f90156140 con 0x7f5f90152540 2024-01-24T13:59:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.037+0000 7f5f969a7700 1 -- 172.21.15.29:0/4196493162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f900660a0 con 0x7f5f90152540 2024-01-24T13:59:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.038+0000 7f5f77fff700 1 -- 172.21.15.29:0/4196493162 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5f840123e0 con 0x7f5f90152540 2024-01-24T13:59:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.038+0000 7f5f77fff700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f7c05f150 0x7f5f7c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.038+0000 7f5f77fff700 1 -- 172.21.15.29:0/4196493162 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 7165+0+0 (secure 0 0 0) 0x7f5f8407a1c0 con 0x7f5f90152540 2024-01-24T13:59:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.038+0000 7f5f8ffff700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f7c05f150 0x7f5f7c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.039+0000 7f5f8ffff700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f7c05f150 0x7f5f7c061610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f5f900bf7b0 tx=0x7f5f80009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.043+0000 7f5f77fff700 1 -- 172.21.15.29:0/4196493162 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5f900660a0 con 0x7f5f90152540 2024-01-24T13:59:10.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.224+0000 7f5f969a7700 1 -- 172.21.15.29:0/4196493162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5f90154de0 con 0x7f5f90152540 2024-01-24T13:59:10.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.837+0000 7f5f77fff700 1 -- 172.21.15.29:0/4196493162 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5f90154de0 con 0x7f5f90152540 2024-01-24T13:59:10.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.841+0000 7f5f969a7700 1 -- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f7c05f150 msgr2=0x7f5f7c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:10.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.841+0000 7f5f969a7700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f7c05f150 0x7f5f7c061610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f5f900bf7b0 tx=0x7f5f80009040).stop 2024-01-24T13:59:10.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.841+0000 7f5f969a7700 1 -- 172.21.15.29:0/4196493162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f90152540 msgr2=0x7f5f90155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:10.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.841+0000 7f5f969a7700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f90152540 0x7f5f90155c50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f5f8400d860 tx=0x7f5f84008040).stop 2024-01-24T13:59:10.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.842+0000 7f5f969a7700 1 -- 172.21.15.29:0/4196493162 shutdown_connections 2024-01-24T13:59:10.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.842+0000 7f5f969a7700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f90159da0 0x7f5f90138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:10.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.842+0000 7f5f969a7700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f90066450 0x7f5f901625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:10.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.842+0000 7f5f969a7700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f7c05f150 0x7f5f7c061610 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:10.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.842+0000 7f5f969a7700 1 --2- 172.21.15.29:0/4196493162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f90152540 0x7f5f90155c50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:10.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.842+0000 7f5f969a7700 1 -- 172.21.15.29:0/4196493162 >> 172.21.15.29:0/4196493162 conn(0x7f5f900c8bd0 msgr2=0x7f5f90134130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:10.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.842+0000 7f5f969a7700 1 -- 172.21.15.29:0/4196493162 shutdown_connections 2024-01-24T13:59:10.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:10.842+0000 7f5f969a7700 1 -- 172.21.15.29:0/4196493162 wait complete. 2024-01-24T13:59:10.843 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-01-24T13:59:10.858 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-24T13:59:11.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.000+0000 7fe77d7b3700 1 Processor -- start 2024-01-24T13:59:11.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.000+0000 7fe77d7b3700 1 -- start start 2024-01-24T13:59:11.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.001+0000 7fe77d7b3700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe778066b60 0x7fe778068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.001+0000 7fe77d7b3700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7780643f0 con 0x7fe7780bf870 2024-01-24T13:59:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.001+0000 7fe77d7b3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7780bdb30 con 0x7fe77813a6f0 2024-01-24T13:59:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.001+0000 7fe77d7b3700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7780be3d0 con 0x7fe778066b60 2024-01-24T13:59:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.002+0000 7fe7777fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe778066b60 0x7fe778068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.002+0000 7fe7767fc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe7780bf870 0x7fe778066450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59610/0 (socket says 172.21.15.29:59610) 2024-01-24T13:59:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.002+0000 7fe7767fc700 1 -- 172.21.15.29:0/1569391473 learned_addr learned my addr 172.21.15.29:0/1569391473 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.002+0000 7fe7777fe700 1 -- 172.21.15.29:0/1569391473 >> v1:172.21.15.163:6789/0 conn(0x7fe77813a6f0 legacy=0x7fe7780638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.002+0000 7fe775ffb700 1 -- 172.21.15.29:0/1569391473 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547895836 0 0) 0x7fe7780643f0 con 0x7fe7780bf870 2024-01-24T13:59:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.002+0000 7fe7777fe700 1 -- 172.21.15.29:0/1569391473 >> v1:172.21.15.29:6789/0 conn(0x7fe7780bf870 legacy=0x7fe778066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.002+0000 7fe7777fe700 1 -- 172.21.15.29:0/1569391473 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe778064110 con 0x7fe778066b60 2024-01-24T13:59:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.002+0000 7fe7777fe700 1 --2- 172.21.15.29:0/1569391473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe778066b60 0x7fe778068660 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fe770009e60 tx=0x7fe770009780).ready entity=mon.0 client_cookie=bc52ceb1cfb7a3d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:11.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.002+0000 7fe775ffb700 1 -- 172.21.15.29:0/1569391473 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77000e070 con 0x7fe778066b60 2024-01-24T13:59:11.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.002+0000 7fe775ffb700 1 -- 172.21.15.29:0/1569391473 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77000bc30 con 0x7fe778066b60 2024-01-24T13:59:11.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.002+0000 7fe775ffb700 1 -- 172.21.15.29:0/1569391473 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770012410 con 0x7fe778066b60 2024-01-24T13:59:11.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.003+0000 7fe77d7b3700 1 -- 172.21.15.29:0/1569391473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe778066b60 msgr2=0x7fe778068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:11.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.003+0000 7fe77d7b3700 1 --2- 172.21.15.29:0/1569391473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe778066b60 0x7fe778068660 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fe770009e60 tx=0x7fe770009780).stop 2024-01-24T13:59:11.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.003+0000 7fe77d7b3700 1 -- 172.21.15.29:0/1569391473 shutdown_connections 2024-01-24T13:59:11.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.003+0000 7fe77d7b3700 1 --2- 172.21.15.29:0/1569391473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe778066b60 0x7fe778068660 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:11.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.003+0000 7fe77d7b3700 1 -- 172.21.15.29:0/1569391473 >> 172.21.15.29:0/1569391473 conn(0x7fe7780c8bd0 msgr2=0x7fe7780c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:11.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.003+0000 7fe77d7b3700 1 -- 172.21.15.29:0/1569391473 shutdown_connections 2024-01-24T13:59:11.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.003+0000 7fe77d7b3700 1 -- 172.21.15.29:0/1569391473 wait complete. 2024-01-24T13:59:11.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.004+0000 7fe77d7b3700 1 Processor -- start 2024-01-24T13:59:11.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.004+0000 7fe77d7b3700 1 -- start start 2024-01-24T13:59:11.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.005+0000 7fe77d7b3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe778066b60 0x7fe77813c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:11.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.005+0000 7fe77d7b3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7780bf870 0x7fe778139500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:11.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.005+0000 7fe77d7b3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77813a6f0 0x7fe7780adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:11.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.005+0000 7fe77d7b3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77813ca00 con 0x7fe77813a6f0 2024-01-24T13:59:11.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.005+0000 7fe77d7b3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77813cce0 con 0x7fe778066b60 2024-01-24T13:59:11.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.005+0000 7fe77d7b3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77813cfc0 con 0x7fe7780bf870 2024-01-24T13:59:11.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe7777fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77813a6f0 0x7fe7780adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:11.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe7777fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77813a6f0 0x7fe7780adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45130/0 (socket says 172.21.15.29:45130) 2024-01-24T13:59:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe7777fe700 1 -- 172.21.15.29:0/584594349 learned_addr learned my addr 172.21.15.29:0/584594349 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe7767fc700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7780bf870 0x7fe778139500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe776ffd700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe778066b60 0x7fe77813c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe7777fe700 1 -- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7780bf870 msgr2=0x7fe778139500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe7777fe700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7780bf870 0x7fe778139500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe7777fe700 1 -- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe778066b60 msgr2=0x7fe77813c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe7777fe700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe778066b60 0x7fe77813c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe7777fe700 1 -- 172.21.15.29:0/584594349 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe770009a20 con 0x7fe77813a6f0 2024-01-24T13:59:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe776ffd700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe778066b60 0x7fe77813c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe7777fe700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77813a6f0 0x7fe7780adf40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fe7780ab900 tx=0x7fe770009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe75effd700 1 -- 172.21.15.29:0/584594349 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700064a0 con 0x7fe77813a6f0 2024-01-24T13:59:11.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe75effd700 1 -- 172.21.15.29:0/584594349 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe770006600 con 0x7fe77813a6f0 2024-01-24T13:59:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe77d7b3700 1 -- 172.21.15.29:0/584594349 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77813d2a0 con 0x7fe77813a6f0 2024-01-24T13:59:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe75effd700 1 -- 172.21.15.29:0/584594349 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700069a0 con 0x7fe77813a6f0 2024-01-24T13:59:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.006+0000 7fe77d7b3700 1 -- 172.21.15.29:0/584594349 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77813c710 con 0x7fe77813a6f0 2024-01-24T13:59:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.008+0000 7fe77d7b3700 1 -- 172.21.15.29:0/584594349 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77806ae00 con 0x7fe77813a6f0 2024-01-24T13:59:11.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.008+0000 7fe75effd700 1 -- 172.21.15.29:0/584594349 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe77001d630 con 0x7fe77813a6f0 2024-01-24T13:59:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.008+0000 7fe75effd700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe76405f150 0x7fe764061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.009+0000 7fe75effd700 1 -- 172.21.15.29:0/584594349 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6611+0+0 (secure 0 0 0) 0x7fe77007c810 con 0x7fe77813a6f0 2024-01-24T13:59:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.009+0000 7fe776ffd700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe76405f150 0x7fe764061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:11.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.009+0000 7fe776ffd700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe76405f150 0x7fe764061610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fe76c001690 tx=0x7fe76c009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:11.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.013+0000 7fe75effd700 1 -- 172.21.15.29:0/584594349 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe77004d8f0 con 0x7fe77813a6f0 2024-01-24T13:59:11.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.192+0000 7fe77d7b3700 1 -- 172.21.15.29:0/584594349 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe77813d7c0 con 0x7fe77813a6f0 2024-01-24T13:59:11.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.842+0000 7fe75effd700 1 -- 172.21.15.29:0/584594349 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe770050f10 con 0x7fe77813a6f0 2024-01-24T13:59:11.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.846+0000 7fe77d7b3700 1 -- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe76405f150 msgr2=0x7fe764061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:11.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.846+0000 7fe77d7b3700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe76405f150 0x7fe764061610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fe76c001690 tx=0x7fe76c009040).stop 2024-01-24T13:59:11.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.846+0000 7fe77d7b3700 1 -- 172.21.15.29:0/584594349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77813a6f0 msgr2=0x7fe7780adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:11.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.846+0000 7fe77d7b3700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77813a6f0 0x7fe7780adf40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fe7780ab900 tx=0x7fe770009780).stop 2024-01-24T13:59:11.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.846+0000 7fe77d7b3700 1 -- 172.21.15.29:0/584594349 shutdown_connections 2024-01-24T13:59:11.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.846+0000 7fe77d7b3700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe76405f150 0x7fe764061610 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:11.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.846+0000 7fe77d7b3700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe77813a6f0 0x7fe7780adf40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:11.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.846+0000 7fe77d7b3700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe778066b60 0x7fe77813c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:11.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.846+0000 7fe77d7b3700 1 --2- 172.21.15.29:0/584594349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7780bf870 0x7fe778139500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:11.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.846+0000 7fe77d7b3700 1 -- 172.21.15.29:0/584594349 >> 172.21.15.29:0/584594349 conn(0x7fe7780c8bd0 msgr2=0x7fe7780c6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:11.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.846+0000 7fe77d7b3700 1 -- 172.21.15.29:0/584594349 shutdown_connections 2024-01-24T13:59:11.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:11.847+0000 7fe77d7b3700 1 -- 172.21.15.29:0/584594349 wait complete. 2024-01-24T13:59:11.848 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-01-24T13:59:11.863 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-01-24T13:59:12.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.002+0000 7fc8ba4c5700 1 Processor -- start 2024-01-24T13:59:12.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.002+0000 7fc8ba4c5700 1 -- start start 2024-01-24T13:59:12.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.003+0000 7fc8ba4c5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8b40bde60 0x7fc8b40661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:12.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.003+0000 7fc8ba4c5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc8b4150e70 0x7fc8b4154a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:12.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.003+0000 7fc8ba4c5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b41557e0 con 0x7fc8b4150a90 2024-01-24T13:59:12.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.003+0000 7fc8ba4c5700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b41512e0 con 0x7fc8b4150e70 2024-01-24T13:59:12.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.003+0000 7fc8ba4c5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40653e0 con 0x7fc8b40bde60 2024-01-24T13:59:12.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b37fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8b40bde60 0x7fc8b40661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:12.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b8a62700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc8b4150e70 0x7fc8b4154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:12.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b8a62700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc8b4150e70 0x7fc8b4154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:55952/0 (socket says 172.21.15.29:55952) 2024-01-24T13:59:12.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b37fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8b40bde60 0x7fc8b40661a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:44028/0 (socket says 172.21.15.29:44028) 2024-01-24T13:59:12.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b8a62700 1 -- 172.21.15.29:0/1438367902 learned_addr learned my addr 172.21.15.29:0/1438367902 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:12.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b8a62700 1 -- 172.21.15.29:0/1438367902 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8b40bde60 msgr2=0x7fc8b40661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:12.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b8a62700 1 --2- 172.21.15.29:0/1438367902 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8b40bde60 0x7fc8b40661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:12.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b8a62700 1 -- 172.21.15.29:0/1438367902 >> v1:172.21.15.163:6789/0 conn(0x7fc8b4150a90 legacy=0x7fc8b40bd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:12.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b8a62700 1 -- 172.21.15.29:0/1438367902 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b40656c0 con 0x7fc8b4150e70 2024-01-24T13:59:12.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b8a62700 1 --2- 172.21.15.29:0/1438367902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8b4150e70 0x7fc8b4154a40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fc8ac009f20 tx=0x7fc8ac009840).ready entity=mon.1 client_cookie=fb77ce36ff2d609 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:12.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b2ffd700 1 -- 172.21.15.29:0/1438367902 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac00e070 con 0x7fc8b4150e70 2024-01-24T13:59:12.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b2ffd700 1 -- 172.21.15.29:0/1438367902 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ac00b9d0 con 0x7fc8b4150e70 2024-01-24T13:59:12.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.004+0000 7fc8b2ffd700 1 -- 172.21.15.29:0/1438367902 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac00bd50 con 0x7fc8b4150e70 2024-01-24T13:59:12.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.005+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1438367902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8b4150e70 msgr2=0x7fc8b4154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:12.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.005+0000 7fc8ba4c5700 1 --2- 172.21.15.29:0/1438367902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8b4150e70 0x7fc8b4154a40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fc8ac009f20 tx=0x7fc8ac009840).stop 2024-01-24T13:59:12.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.005+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1438367902 shutdown_connections 2024-01-24T13:59:12.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.005+0000 7fc8ba4c5700 1 --2- 172.21.15.29:0/1438367902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8b4150e70 0x7fc8b4154a40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:12.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.005+0000 7fc8ba4c5700 1 --2- 172.21.15.29:0/1438367902 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8b40bde60 0x7fc8b40661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:12.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.005+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1438367902 >> 172.21.15.29:0/1438367902 conn(0x7fc8b4064390 msgr2=0x7fc8b40c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:12.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.005+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1438367902 shutdown_connections 2024-01-24T13:59:12.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.005+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1438367902 wait complete. 2024-01-24T13:59:12.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.006+0000 7fc8ba4c5700 1 Processor -- start 2024-01-24T13:59:12.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.007+0000 7fc8ba4c5700 1 -- start start 2024-01-24T13:59:12.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.007+0000 7fc8ba4c5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8b40bde60 0x7fc8b413c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.007+0000 7fc8ba4c5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8b4150a90 0x7fc8b40aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.007+0000 7fc8ba4c5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b4150e70 0x7fc8b4069240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.007+0000 7fc8ba4c5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40abf40 con 0x7fc8b4150e70 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.007+0000 7fc8ba4c5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40ab330 con 0x7fc8b4150a90 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.007+0000 7fc8ba4c5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40ab610 con 0x7fc8b40bde60 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8b8a62700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b4150e70 0x7fc8b4069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8b8a62700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b4150e70 0x7fc8b4069240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:45134/0 (socket says 172.21.15.29:45134) 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8b8a62700 1 -- 172.21.15.29:0/1008567325 learned_addr learned my addr 172.21.15.29:0/1008567325 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8b3fff700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8b40bde60 0x7fc8b413c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8b37fe700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8b4150a90 0x7fc8b40aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8b8a62700 1 -- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8b40bde60 msgr2=0x7fc8b413c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8b8a62700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8b40bde60 0x7fc8b413c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:12.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8b8a62700 1 -- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8b4150a90 msgr2=0x7fc8b40aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:12.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8b8a62700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8b4150a90 0x7fc8b40aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:12.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8b8a62700 1 -- 172.21.15.29:0/1008567325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac009ae0 con 0x7fc8b4150e70 2024-01-24T13:59:12.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8b37fe700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8b4150a90 0x7fc8b40aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:12.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8b8a62700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b4150e70 0x7fc8b4069240 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fc8ac00b610 tx=0x7fc8ac009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:12.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc89bfff700 1 -- 172.21.15.29:0/1008567325 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac0092e0 con 0x7fc8b4150e70 2024-01-24T13:59:12.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc89bfff700 1 -- 172.21.15.29:0/1008567325 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ac007610 con 0x7fc8b4150e70 2024-01-24T13:59:12.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1008567325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b40ab8f0 con 0x7fc8b4150e70 2024-01-24T13:59:12.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc89bfff700 1 -- 172.21.15.29:0/1008567325 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac0079d0 con 0x7fc8b4150e70 2024-01-24T13:59:12.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.008+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1008567325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b413cba0 con 0x7fc8b4150e70 2024-01-24T13:59:12.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.010+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1008567325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b40475e0 con 0x7fc8b4150e70 2024-01-24T13:59:12.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.010+0000 7fc89bfff700 1 -- 172.21.15.29:0/1008567325 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc8ac003040 con 0x7fc8b4150e70 2024-01-24T13:59:12.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.010+0000 7fc89bfff700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc8a005f150 0x7fc8a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:12.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.011+0000 7fc89bfff700 1 -- 172.21.15.29:0/1008567325 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6139+0+0 (secure 0 0 0) 0x7fc8ac07bb60 con 0x7fc8b4150e70 2024-01-24T13:59:12.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.011+0000 7fc8b3fff700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc8a005f150 0x7fc8a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:12.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.011+0000 7fc8b3fff700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc8a005f150 0x7fc8a0061610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fc8a4003900 tx=0x7fc8a4007d40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:12.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.015+0000 7fc89bfff700 1 -- 172.21.15.29:0/1008567325 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc8ac04ce20 con 0x7fc8b4150e70 2024-01-24T13:59:12.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.199+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1008567325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc8b40659a0 con 0x7fc8b4150e70 2024-01-24T13:59:12.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.844+0000 7fc89bfff700 1 -- 172.21.15.29:0/1008567325 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) v1 ==== 178+0+0 (secure 0 0 0) 0x7fc8ac023030 con 0x7fc8b4150e70 2024-01-24T13:59:12.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.848+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc8a005f150 msgr2=0x7fc8a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:12.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.848+0000 7fc8ba4c5700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc8a005f150 0x7fc8a0061610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fc8a4003900 tx=0x7fc8a4007d40).stop 2024-01-24T13:59:12.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.848+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1008567325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b4150e70 msgr2=0x7fc8b4069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:12.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.848+0000 7fc8ba4c5700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b4150e70 0x7fc8b4069240 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fc8ac00b610 tx=0x7fc8ac009840).stop 2024-01-24T13:59:12.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.848+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1008567325 shutdown_connections 2024-01-24T13:59:12.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.848+0000 7fc8ba4c5700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc8a005f150 0x7fc8a0061610 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:12.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.848+0000 7fc8ba4c5700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b4150e70 0x7fc8b4069240 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:12.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.848+0000 7fc8ba4c5700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8b40bde60 0x7fc8b413c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:12.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.848+0000 7fc8ba4c5700 1 --2- 172.21.15.29:0/1008567325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8b4150a90 0x7fc8b40aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:12.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.848+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1008567325 >> 172.21.15.29:0/1008567325 conn(0x7fc8b4064390 msgr2=0x7fc8b41491f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:12.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.848+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1008567325 shutdown_connections 2024-01-24T13:59:12.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:12.848+0000 7fc8ba4c5700 1 -- 172.21.15.29:0/1008567325 wait complete. 2024-01-24T13:59:12.851 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data_ec' removed 2024-01-24T13:59:12.868 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-24T13:59:12.868 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-24T13:59:12.869 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-24T13:59:12.869 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T13:59:12.869 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-24T13:59:12.869 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-24T13:59:12.869 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-24T13:59:12.869 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T13:59:12.869 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-24T13:59:12.870 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-24T13:59:12.870 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-24T13:59:12.870 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T13:59:12.870 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-24T13:59:12.870 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-24T13:59:12.870 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-24T13:59:12.870 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T13:59:12.880 INFO:tasks.ceph.mds.c:Stopped 2024-01-24T13:59:12.880 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-24T13:59:12.881 INFO:tasks.ceph.mds.b:Stopped 2024-01-24T13:59:12.881 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-24T13:59:12.882 INFO:tasks.ceph.mds.d:Stopped 2024-01-24T13:59:12.882 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-24T13:59:12.922 INFO:tasks.ceph.mds.c:Started 2024-01-24T13:59:12.923 DEBUG:teuthology.parallel:result is None 2024-01-24T13:59:12.925 INFO:tasks.ceph.mds.b:Started 2024-01-24T13:59:12.925 DEBUG:teuthology.parallel:result is None 2024-01-24T13:59:12.925 INFO:tasks.ceph.mds.d:Started 2024-01-24T13:59:12.926 DEBUG:teuthology.parallel:result is None 2024-01-24T13:59:13.011 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-01-24T13:59:13.011 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T13:59:13.012 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T13:59:14.631 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-01-24T13:59:18.880 INFO:tasks.ceph.mds.a:Stopped 2024-01-24T13:59:18.880 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-24T13:59:18.884 INFO:tasks.ceph.mds.a:Started 2024-01-24T13:59:18.884 DEBUG:teuthology.parallel:result is None 2024-01-24T13:59:18.884 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-24T13:59:18.976 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T13:59:19.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.028+0000 7f40a7071700 1 Processor -- start 2024-01-24T13:59:19.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.028+0000 7f40a7071700 1 -- start start 2024-01-24T13:59:19.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a7071700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f40a0154240 0x7f40a0064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:19.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a7071700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40a00a9d20 0x7f40a00aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:19.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a7071700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a01550e0 con 0x7f40a0157cf0 2024-01-24T13:59:19.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a7071700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a00c0d10 con 0x7f40a0154240 2024-01-24T13:59:19.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a7071700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a00aa670 con 0x7f40a00a9d20 2024-01-24T13:59:19.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a4e0d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f40a0154240 0x7f40a0064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:19.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a4e0d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f40a0154240 0x7f40a0064780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41374/0 (socket says 172.21.15.29:41374) 2024-01-24T13:59:19.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a4e0d700 1 -- 172.21.15.29:0/3785965106 learned_addr learned my addr 172.21.15.29:0/3785965106 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:19.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a560e700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f40a0157cf0 0x7f40a0158100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:45850/0 (socket says 172.21.15.29:45850) 2024-01-24T13:59:19.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f409ffff700 1 --2- 172.21.15.29:0/3785965106 >> v2:172.21.15.163:3300/0 conn(0x7f40a00a9d20 0x7f40a00aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:19.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a4e0d700 1 -- 172.21.15.29:0/3785965106 >> v2:172.21.15.163:3300/0 conn(0x7f40a00a9d20 msgr2=0x7f40a00aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a4e0d700 1 --2- 172.21.15.29:0/3785965106 >> v2:172.21.15.163:3300/0 conn(0x7f40a00a9d20 0x7f40a00aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a4e0d700 1 -- 172.21.15.29:0/3785965106 >> v1:172.21.15.29:6789/0 conn(0x7f40a0157cf0 legacy=0x7f40a0158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f409f7fe700 1 -- 172.21.15.29:0/3785965106 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720783251 0 0) 0x7f40a01550e0 con 0x7f40a0157cf0 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.029+0000 7f40a4e0d700 1 -- 172.21.15.29:0/3785965106 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a00650d0 con 0x7f40a0154240 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.030+0000 7f40a4e0d700 1 --2- 172.21.15.29:0/3785965106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40a0154240 0x7f40a0064780 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f40a0084090 tx=0x7f40940090f0).ready entity=mon.0 client_cookie=a04a2d264a8e3097 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.030+0000 7f409f7fe700 1 -- 172.21.15.29:0/3785965106 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400b520 con 0x7f40a0154240 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.030+0000 7f409f7fe700 1 -- 172.21.15.29:0/3785965106 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409400b680 con 0x7f40a0154240 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.030+0000 7f409f7fe700 1 -- 172.21.15.29:0/3785965106 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400ba00 con 0x7f40a0154240 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.030+0000 7f40a7071700 1 -- 172.21.15.29:0/3785965106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40a0154240 msgr2=0x7f40a0064780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.030+0000 7f40a7071700 1 --2- 172.21.15.29:0/3785965106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40a0154240 0x7f40a0064780 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f40a0084090 tx=0x7f40940090f0).stop 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.030+0000 7f40a7071700 1 -- 172.21.15.29:0/3785965106 shutdown_connections 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.030+0000 7f40a7071700 1 --2- 172.21.15.29:0/3785965106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40a0154240 0x7f40a0064780 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.030+0000 7f40a7071700 1 --2- 172.21.15.29:0/3785965106 >> v2:172.21.15.163:3300/0 conn(0x7f40a00a9d20 0x7f40a00aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:19.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.030+0000 7f40a7071700 1 -- 172.21.15.29:0/3785965106 >> 172.21.15.29:0/3785965106 conn(0x7f40a00c8310 msgr2=0x7f40a005e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:19.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.031+0000 7f40a7071700 1 -- 172.21.15.29:0/3785965106 shutdown_connections 2024-01-24T13:59:19.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.031+0000 7f40a7071700 1 -- 172.21.15.29:0/3785965106 wait complete. 2024-01-24T13:59:19.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.032+0000 7f40a7071700 1 Processor -- start 2024-01-24T13:59:19.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.032+0000 7f40a7071700 1 -- start start 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a7071700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a00a9d20 0x7f40a01689d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a7071700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40a0154240 0x7f40a0152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a7071700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40a0157cf0 0x7f40a00c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a7071700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a016bd30 con 0x7f40a0157cf0 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a7071700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00afa20 con 0x7f40a00a9d20 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a7071700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00c4cf0 con 0x7f40a0154240 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a560e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40a0157cf0 0x7f40a00c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a560e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40a0157cf0 0x7f40a00c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41378/0 (socket says 172.21.15.29:41378) 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a560e700 1 -- 172.21.15.29:0/1012468014 learned_addr learned my addr 172.21.15.29:0/1012468014 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a4e0d700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a00a9d20 0x7f40a01689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f409ffff700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40a0154240 0x7f40a0152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a560e700 1 -- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40a0154240 msgr2=0x7f40a0152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a560e700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40a0154240 0x7f40a0152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:19.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a560e700 1 -- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a00a9d20 msgr2=0x7f40a01689d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a560e700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a00a9d20 0x7f40a01689d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a560e700 1 -- 172.21.15.29:0/1012468014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094009370 con 0x7f40a0157cf0 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f409ffff700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40a0154240 0x7f40a0152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a4e0d700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a00a9d20 0x7f40a01689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.033+0000 7f40a560e700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40a0157cf0 0x7f40a00c4630 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f409800c730 tx=0x7f4098001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.034+0000 7f409cff9700 1 -- 172.21.15.29:0/1012468014 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098005c70 con 0x7f40a0157cf0 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.034+0000 7f409cff9700 1 -- 172.21.15.29:0/1012468014 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098005dd0 con 0x7f40a0157cf0 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.034+0000 7f40a7071700 1 -- 172.21.15.29:0/1012468014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a00c4fd0 con 0x7f40a0157cf0 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.034+0000 7f409cff9700 1 -- 172.21.15.29:0/1012468014 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098007340 con 0x7f40a0157cf0 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.034+0000 7f40a7071700 1 -- 172.21.15.29:0/1012468014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a00af6b0 con 0x7f40a0157cf0 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.035+0000 7f40a7071700 1 -- 172.21.15.29:0/1012468014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a00afbd0 con 0x7f40a0157cf0 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.035+0000 7f409cff9700 1 -- 172.21.15.29:0/1012468014 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f40980076f0 con 0x7f40a0157cf0 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.035+0000 7f409cff9700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f408405f150 0x7f4084061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:19.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.035+0000 7f409cff9700 1 -- 172.21.15.29:0/1012468014 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5708+0+0 (secure 0 0 0) 0x7f409807a090 con 0x7f40a0157cf0 2024-01-24T13:59:19.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.036+0000 7f40a4e0d700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f408405f150 0x7f4084061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:19.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.036+0000 7f40a4e0d700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f408405f150 0x7f4084061610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f4094009130 tx=0x7f4094017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:19.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.040+0000 7f409cff9700 1 -- 172.21.15.29:0/1012468014 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f409804b500 con 0x7f40a0157cf0 2024-01-24T13:59:19.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:19.240+0000 7f40a7071700 1 -- 172.21.15.29:0/1012468014 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f40a00475e0 con 0x7f40a0157cf0 2024-01-24T13:59:20.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.004+0000 7f409cff9700 1 -- 172.21.15.29:0/1012468014 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v37) v1 ==== 97+0+0 (secure 0 0 0) 0x7f409804eb20 con 0x7f40a0157cf0 2024-01-24T13:59:20.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.007+0000 7f40a7071700 1 -- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f408405f150 msgr2=0x7f4084061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.007+0000 7f40a7071700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f408405f150 0x7f4084061610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f4094009130 tx=0x7f4094017000).stop 2024-01-24T13:59:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.007+0000 7f40a7071700 1 -- 172.21.15.29:0/1012468014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40a0157cf0 msgr2=0x7f40a00c4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.007+0000 7f40a7071700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40a0157cf0 0x7f40a00c4630 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f409800c730 tx=0x7f4098001520).stop 2024-01-24T13:59:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.007+0000 7f40a7071700 1 -- 172.21.15.29:0/1012468014 shutdown_connections 2024-01-24T13:59:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.007+0000 7f40a7071700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f408405f150 0x7f4084061610 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.007+0000 7f40a7071700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40a0157cf0 0x7f40a00c4630 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.007+0000 7f40a7071700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a00a9d20 0x7f40a01689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.007+0000 7f40a7071700 1 --2- 172.21.15.29:0/1012468014 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40a0154240 0x7f40a0152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.007+0000 7f40a7071700 1 -- 172.21.15.29:0/1012468014 >> 172.21.15.29:0/1012468014 conn(0x7f40a00c8310 msgr2=0x7f40a00c1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.007+0000 7f40a7071700 1 -- 172.21.15.29:0/1012468014 shutdown_connections 2024-01-24T13:59:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.007+0000 7f40a7071700 1 -- 172.21.15.29:0/1012468014 wait complete. 2024-01-24T13:59:20.009 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-01-24T13:59:20.023 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-24T13:59:20.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.156+0000 7ff4fe693700 1 Processor -- start 2024-01-24T13:59:20.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.156+0000 7ff4fe693700 1 -- start start 2024-01-24T13:59:20.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.157+0000 7ff4fe693700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff4f80670a0 0x7ff4f80bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:20.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.157+0000 7ff4fe693700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f81582b0 con 0x7ff4f80c4a10 2024-01-24T13:59:20.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.157+0000 7ff4fe693700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f8157ca0 con 0x7ff4f8066c00 2024-01-24T13:59:20.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.157+0000 7ff4fe693700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f8063c60 con 0x7ff4f80670a0 2024-01-24T13:59:20.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.157+0000 7ff4fcc30700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff4f80670a0 0x7ff4f80bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:20.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.157+0000 7ff4f7fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff4f80c4a10 0x7ff4f80664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:45866/0 (socket says 172.21.15.29:45866) 2024-01-24T13:59:20.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.157+0000 7ff4f7fff700 1 -- 172.21.15.29:0/3892253093 learned_addr learned my addr 172.21.15.29:0/3892253093 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:20.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.157+0000 7ff4fcc30700 1 -- 172.21.15.29:0/3892253093 >> v1:172.21.15.163:6789/0 conn(0x7ff4f8066c00 legacy=0x7ff4f80bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:20.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4fcc30700 1 -- 172.21.15.29:0/3892253093 >> v1:172.21.15.29:6789/0 conn(0x7ff4f80c4a10 legacy=0x7ff4f80664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4fcc30700 1 -- 172.21.15.29:0/3892253093 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f8157f40 con 0x7ff4f80670a0 2024-01-24T13:59:20.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4f6ffd700 1 -- 172.21.15.29:0/3892253093 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808517168 0 0) 0x7ff4f81582b0 con 0x7ff4f80c4a10 2024-01-24T13:59:20.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4fcc30700 1 --2- 172.21.15.29:0/3892253093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f80670a0 0x7ff4f80bf6d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7ff4f0009e60 tx=0x7ff4f0009780).ready entity=mon.0 client_cookie=ea6d0d8c917892b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:20.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4f6ffd700 1 -- 172.21.15.29:0/3892253093 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f000e070 con 0x7ff4f80670a0 2024-01-24T13:59:20.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4f6ffd700 1 -- 172.21.15.29:0/3892253093 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f000bc30 con 0x7ff4f80670a0 2024-01-24T13:59:20.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4f6ffd700 1 -- 172.21.15.29:0/3892253093 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0012410 con 0x7ff4f80670a0 2024-01-24T13:59:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4fe693700 1 -- 172.21.15.29:0/3892253093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f80670a0 msgr2=0x7ff4f80bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4fe693700 1 --2- 172.21.15.29:0/3892253093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f80670a0 0x7ff4f80bf6d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7ff4f0009e60 tx=0x7ff4f0009780).stop 2024-01-24T13:59:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4fe693700 1 -- 172.21.15.29:0/3892253093 shutdown_connections 2024-01-24T13:59:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4fe693700 1 --2- 172.21.15.29:0/3892253093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f80670a0 0x7ff4f80bf6d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4fe693700 1 -- 172.21.15.29:0/3892253093 >> 172.21.15.29:0/3892253093 conn(0x7ff4f80c8bb0 msgr2=0x7ff4f80c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4fe693700 1 -- 172.21.15.29:0/3892253093 shutdown_connections 2024-01-24T13:59:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.158+0000 7ff4fe693700 1 -- 172.21.15.29:0/3892253093 wait complete. 2024-01-24T13:59:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.160+0000 7ff4fe693700 1 Processor -- start 2024-01-24T13:59:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.160+0000 7ff4fe693700 1 -- start start 2024-01-24T13:59:20.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4fe693700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f8066c00 0x7ff4f80a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:20.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4fe693700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f80670a0 0x7ff4f80aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:20.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4fe693700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4f80c4a10 0x7ff4f80ab170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:20.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4fe693700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80ab660 con 0x7ff4f8066c00 2024-01-24T13:59:20.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4fe693700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80a65f0 con 0x7ff4f80670a0 2024-01-24T13:59:20.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4fe693700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80a68d0 con 0x7ff4f80c4a10 2024-01-24T13:59:20.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4f7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f8066c00 0x7ff4f80a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:20.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4f7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f8066c00 0x7ff4f80a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41392/0 (socket says 172.21.15.29:41392) 2024-01-24T13:59:20.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4f7fff700 1 -- 172.21.15.29:0/2863918054 learned_addr learned my addr 172.21.15.29:0/2863918054 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:20.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4fcc30700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4f80c4a10 0x7ff4f80ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4f77fe700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f80670a0 0x7ff4f80aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4f7fff700 1 -- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4f80c4a10 msgr2=0x7ff4f80ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4f7fff700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4f80c4a10 0x7ff4f80ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4f7fff700 1 -- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f80670a0 msgr2=0x7ff4f80aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4f7fff700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f80670a0 0x7ff4f80aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4f7fff700 1 -- 172.21.15.29:0/2863918054 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f0009a20 con 0x7ff4f8066c00 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.161+0000 7ff4f7fff700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f8066c00 0x7ff4f80a4880 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7ff4ec00c490 tx=0x7ff4ec007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.162+0000 7ff4dffff700 1 -- 172.21.15.29:0/2863918054 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec0072e0 con 0x7ff4f8066c00 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.162+0000 7ff4dffff700 1 -- 172.21.15.29:0/2863918054 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ec007440 con 0x7ff4f8066c00 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.162+0000 7ff4fe693700 1 -- 172.21.15.29:0/2863918054 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f80a6bb0 con 0x7ff4f8066c00 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.162+0000 7ff4dffff700 1 -- 172.21.15.29:0/2863918054 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec0057a0 con 0x7ff4f8066c00 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.162+0000 7ff4fe693700 1 -- 172.21.15.29:0/2863918054 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f80a6f40 con 0x7ff4f8066c00 2024-01-24T13:59:20.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.163+0000 7ff4fe693700 1 -- 172.21.15.29:0/2863918054 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f80a73b0 con 0x7ff4f8066c00 2024-01-24T13:59:20.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.163+0000 7ff4dffff700 1 -- 172.21.15.29:0/2863918054 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff4ec005b90 con 0x7ff4f8066c00 2024-01-24T13:59:20.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.163+0000 7ff4dffff700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff4e405f150 0x7ff4e4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:20.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.163+0000 7ff4dffff700 1 -- 172.21.15.29:0/2863918054 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5622+0+0 (secure 0 0 0) 0x7ff4ec079860 con 0x7ff4f8066c00 2024-01-24T13:59:20.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.164+0000 7ff4f77fe700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff4e405f150 0x7ff4e4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:20.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.164+0000 7ff4f77fe700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff4e405f150 0x7ff4e4061610 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ff4f81577e0 tx=0x7ff4e8009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:20.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.168+0000 7ff4dffff700 1 -- 172.21.15.29:0/2863918054 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff4f80a73b0 con 0x7ff4f8066c00 2024-01-24T13:59:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.417+0000 7ff4fe693700 1 -- 172.21.15.29:0/2863918054 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff4f80c5020 con 0x7ff4f8066c00 2024-01-24T13:59:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.418+0000 7ff4dffff700 1 -- 172.21.15.29:0/2863918054 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7ff4ec04ac60 con 0x7ff4f8066c00 2024-01-24T13:59:20.419 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:20.419 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T13:59:20.419 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-01-24T13:59:20.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.420 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-01-24T13:59:20.420 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew==", 2024-01-24T13:59:20.420 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.420 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T13:59:20.421 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T13:59:20.421 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T13:59:20.421 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.421 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.422 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-01-24T13:59:20.422 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ==", 2024-01-24T13:59:20.422 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.422 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T13:59:20.422 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T13:59:20.423 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T13:59:20.423 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.423 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.423 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-01-24T13:59:20.424 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw==", 2024-01-24T13:59:20.424 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.424 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T13:59:20.424 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T13:59:20.425 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T13:59:20.425 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.425 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.426 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-01-24T13:59:20.426 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ==", 2024-01-24T13:59:20.426 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.426 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T13:59:20.427 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T13:59:20.427 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T13:59:20.427 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.427 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.428 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-01-24T13:59:20.428 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ==", 2024-01-24T13:59:20.428 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.428 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T13:59:20.428 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T13:59:20.429 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.429 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.429 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-01-24T13:59:20.429 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA==", 2024-01-24T13:59:20.430 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.430 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T13:59:20.430 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T13:59:20.430 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.430 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.431 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-01-24T13:59:20.431 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw==", 2024-01-24T13:59:20.431 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.431 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T13:59:20.431 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T13:59:20.432 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.432 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.432 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-01-24T13:59:20.433 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ==", 2024-01-24T13:59:20.433 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.433 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T13:59:20.433 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T13:59:20.433 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.433 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.434 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-01-24T13:59:20.434 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw==", 2024-01-24T13:59:20.434 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.434 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T13:59:20.434 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T13:59:20.435 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.435 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.435 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-01-24T13:59:20.436 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ==", 2024-01-24T13:59:20.436 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.436 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T13:59:20.436 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T13:59:20.436 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.436 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.437 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-01-24T13:59:20.437 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A==", 2024-01-24T13:59:20.437 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.438 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T13:59:20.438 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T13:59:20.438 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.438 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.439 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-01-24T13:59:20.439 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ==", 2024-01-24T13:59:20.439 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.439 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T13:59:20.439 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T13:59:20.440 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.440 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-01-24T13:59:20.440 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw==", 2024-01-24T13:59:20.440 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.440 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T13:59:20.440 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T13:59:20.440 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T13:59:20.440 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T13:59:20.440 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA==", 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-01-24T13:59:20.441 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g==", 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ==", 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T13:59:20.442 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T13:59:20.443 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T13:59:20.443 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T13:59:20.443 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.443 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-01-24T13:59:20.443 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlxK1bFxAAZpfJn+Tcpmq7R9Yq29FvVg==", 2024-01-24T13:59:20.443 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.443 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T13:59:20.443 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T13:59:20.443 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T13:59:20.443 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.444 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.444 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.bootstrap-mds", 2024-01-24T13:59:20.444 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBoF7FlfJV+NBAA35ZkgKEE0mOWLxzNRaO12A==", 2024-01-24T13:59:20.444 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.444 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile bootstrap-mds" 2024-01-24T13:59:20.444 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.444 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.bootstrap-mgr", 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBoF7FllqN+NBAA13rvDaJdisfMMjoICxgtxg==", 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile bootstrap-mgr" 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.bootstrap-osd", 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBoF7FlF61+NBAAJiWblQl6NxkcWvazNSyUBQ==", 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile bootstrap-osd" 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.446 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.bootstrap-rbd", 2024-01-24T13:59:20.446 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBoF7Flt7V+NBAAV0z0U2YYrcInY0/n2pF0uA==", 2024-01-24T13:59:20.446 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.446 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile bootstrap-rbd" 2024-01-24T13:59:20.446 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.446 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-01-24T13:59:20.446 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBoF7Fls75+NBAAPbO4jC8hnrXgtTvvtpcuvA==", 2024-01-24T13:59:20.446 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.446 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-01-24T13:59:20.446 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.447 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.bootstrap-rgw", 2024-01-24T13:59:20.447 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBoF7Fl98h+NBAAHqOsgZ3te/ZdgV3HeCUJMg==", 2024-01-24T13:59:20.447 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.447 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile bootstrap-rgw" 2024-01-24T13:59:20.447 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.447 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-01-24T13:59:20.447 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw==", 2024-01-24T13:59:20.447 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.448 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T13:59:20.448 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T13:59:20.448 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.448 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:20.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:20.448 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-01-24T13:59:20.448 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA==", 2024-01-24T13:59:20.448 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T13:59:20.448 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T13:59:20.448 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T13:59:20.448 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T13:59:20.449 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.449 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:20.449 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-01-24T13:59:20.449 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T13:59:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.420+0000 7ff4fe693700 1 -- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff4e405f150 msgr2=0x7ff4e4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.420+0000 7ff4fe693700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff4e405f150 0x7ff4e4061610 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ff4f81577e0 tx=0x7ff4e8009040).stop 2024-01-24T13:59:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.420+0000 7ff4fe693700 1 -- 172.21.15.29:0/2863918054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f8066c00 msgr2=0x7ff4f80a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.420+0000 7ff4fe693700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f8066c00 0x7ff4f80a4880 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7ff4ec00c490 tx=0x7ff4ec007e40).stop 2024-01-24T13:59:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.420+0000 7ff4fe693700 1 -- 172.21.15.29:0/2863918054 shutdown_connections 2024-01-24T13:59:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.420+0000 7ff4fe693700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4f80c4a10 0x7ff4f80ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.420+0000 7ff4fe693700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff4e405f150 0x7ff4e4061610 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.420+0000 7ff4fe693700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f8066c00 0x7ff4f80a4880 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.420+0000 7ff4fe693700 1 --2- 172.21.15.29:0/2863918054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f80670a0 0x7ff4f80aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.420+0000 7ff4fe693700 1 -- 172.21.15.29:0/2863918054 >> 172.21.15.29:0/2863918054 conn(0x7ff4f80c8bb0 msgr2=0x7ff4f80664f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.420+0000 7ff4fe693700 1 -- 172.21.15.29:0/2863918054 shutdown_connections 2024-01-24T13:59:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.420+0000 7ff4fe693700 1 -- 172.21.15.29:0/2863918054 wait complete. 2024-01-24T13:59:20.452 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-01-24T13:59:20.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.584+0000 7fc2e311d700 1 Processor -- start 2024-01-24T13:59:20.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.584+0000 7fc2e311d700 1 -- start start 2024-01-24T13:59:20.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2e311d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc2dc13e220 0x7fc2dc0639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:20.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2e311d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2dc0bd770 con 0x7fc2dc063ec0 2024-01-24T13:59:20.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2e311d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2dc063540 con 0x7fc2dc13de40 2024-01-24T13:59:20.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2e311d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2dc062930 con 0x7fc2dc13e220 2024-01-24T13:59:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2dbfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc2dc13e220 0x7fc2dc0639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2e16ba700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc2dc063ec0 0x7fc2dc0c1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:45878/0 (socket says 172.21.15.29:45878) 2024-01-24T13:59:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2dbfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc2dc13e220 0x7fc2dc0639d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41394/0 (socket says 172.21.15.29:41394) 2024-01-24T13:59:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2e16ba700 1 -- 172.21.15.29:0/943951542 learned_addr learned my addr 172.21.15.29:0/943951542 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2db7fe700 1 -- 172.21.15.29:0/943951542 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480397335 0 0) 0x7fc2dc0bd770 con 0x7fc2dc063ec0 2024-01-24T13:59:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2db7fe700 1 -- 172.21.15.29:0/943951542 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2cc0033f0 con 0x7fc2dc063ec0 2024-01-24T13:59:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2dbfff700 1 -- 172.21.15.29:0/943951542 >> v1:172.21.15.163:6789/0 conn(0x7fc2dc13de40 legacy=0x7fc2dc0bd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2db7fe700 1 -- 172.21.15.29:0/943951542 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949179879 0 0) 0x7fc2dc063540 con 0x7fc2dc13de40 2024-01-24T13:59:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.585+0000 7fc2dbfff700 1 -- 172.21.15.29:0/943951542 >> v1:172.21.15.29:6789/0 conn(0x7fc2dc063ec0 legacy=0x7fc2dc0c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.586+0000 7fc2dbfff700 1 -- 172.21.15.29:0/943951542 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc062c10 con 0x7fc2dc13e220 2024-01-24T13:59:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.586+0000 7fc2dbfff700 1 --2- 172.21.15.29:0/943951542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2dc13e220 0x7fc2dc0639d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fc2c8009760 tx=0x7fc2c8009060).ready entity=mon.0 client_cookie=cdf9618427001fec server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.586+0000 7fc2db7fe700 1 -- 172.21.15.29:0/943951542 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c800f510 con 0x7fc2dc13e220 2024-01-24T13:59:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.586+0000 7fc2db7fe700 1 -- 172.21.15.29:0/943951542 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c800f670 con 0x7fc2dc13e220 2024-01-24T13:59:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.586+0000 7fc2db7fe700 1 -- 172.21.15.29:0/943951542 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c800f9f0 con 0x7fc2dc13e220 2024-01-24T13:59:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.586+0000 7fc2e311d700 1 -- 172.21.15.29:0/943951542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2dc13e220 msgr2=0x7fc2dc0639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.586+0000 7fc2e311d700 1 --2- 172.21.15.29:0/943951542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2dc13e220 0x7fc2dc0639d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fc2c8009760 tx=0x7fc2c8009060).stop 2024-01-24T13:59:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.586+0000 7fc2e311d700 1 -- 172.21.15.29:0/943951542 shutdown_connections 2024-01-24T13:59:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.586+0000 7fc2e311d700 1 --2- 172.21.15.29:0/943951542 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2dc13e220 0x7fc2dc0639d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.586+0000 7fc2e311d700 1 -- 172.21.15.29:0/943951542 >> 172.21.15.29:0/943951542 conn(0x7fc2dc0c3cd0 msgr2=0x7fc2dc0c40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.587+0000 7fc2e311d700 1 -- 172.21.15.29:0/943951542 shutdown_connections 2024-01-24T13:59:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.587+0000 7fc2e311d700 1 -- 172.21.15.29:0/943951542 wait complete. 2024-01-24T13:59:20.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.588+0000 7fc2e311d700 1 Processor -- start 2024-01-24T13:59:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.588+0000 7fc2e311d700 1 -- start start 2024-01-24T13:59:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e311d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2dc063ec0 0x7fc2dc158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e311d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2dc13de40 0x7fc2dc0a8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e311d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2dc13e220 0x7fc2dc0aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e311d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0ab590 con 0x7fc2dc063ec0 2024-01-24T13:59:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e311d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0aa6d0 con 0x7fc2dc13e220 2024-01-24T13:59:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e311d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0aa980 con 0x7fc2dc13de40 2024-01-24T13:59:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e0eb9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2dc063ec0 0x7fc2dc158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e0eb9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2dc063ec0 0x7fc2dc158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41396/0 (socket says 172.21.15.29:41396) 2024-01-24T13:59:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e0eb9700 1 -- 172.21.15.29:0/184026041 learned_addr learned my addr 172.21.15.29:0/184026041 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2dbfff700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2dc13de40 0x7fc2dc0a8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e16ba700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2dc13e220 0x7fc2dc0aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e0eb9700 1 -- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2dc13de40 msgr2=0x7fc2dc0a8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e0eb9700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2dc13de40 0x7fc2dc0a8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e0eb9700 1 -- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2dc13e220 msgr2=0x7fc2dc0aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e0eb9700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2dc13e220 0x7fc2dc0aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e0eb9700 1 -- 172.21.15.29:0/184026041 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c80092a0 con 0x7fc2dc063ec0 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.589+0000 7fc2e0eb9700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2dc063ec0 0x7fc2dc158580 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fc2d0001eb0 tx=0x7fc2d0007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.590+0000 7fc2d8ff9700 1 -- 172.21.15.29:0/184026041 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000f470 con 0x7fc2dc063ec0 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.590+0000 7fc2d8ff9700 1 -- 172.21.15.29:0/184026041 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d000f5f0 con 0x7fc2dc063ec0 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.590+0000 7fc2e311d700 1 -- 172.21.15.29:0/184026041 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2dc0aac60 con 0x7fc2dc063ec0 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.590+0000 7fc2d8ff9700 1 -- 172.21.15.29:0/184026041 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000f9b0 con 0x7fc2dc063ec0 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.590+0000 7fc2e311d700 1 -- 172.21.15.29:0/184026041 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc0ab220 con 0x7fc2dc063ec0 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.591+0000 7fc2e311d700 1 -- 172.21.15.29:0/184026041 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2dc0475e0 con 0x7fc2dc063ec0 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.591+0000 7fc2d8ff9700 1 -- 172.21.15.29:0/184026041 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc2d000fd20 con 0x7fc2dc063ec0 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.592+0000 7fc2d8ff9700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2c005f150 0x7fc2c0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.592+0000 7fc2d8ff9700 1 -- 172.21.15.29:0/184026041 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5622+0+0 (secure 0 0 0) 0x7fc2d0079640 con 0x7fc2dc063ec0 2024-01-24T13:59:20.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.592+0000 7fc2dbfff700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2c005f150 0x7fc2c0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:20.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.592+0000 7fc2dbfff700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2c005f150 0x7fc2c0061610 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fc2dc138d50 tx=0x7fc2c8017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:20.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.596+0000 7fc2d8ff9700 1 -- 172.21.15.29:0/184026041 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc2d004ab00 con 0x7fc2dc063ec0 2024-01-24T13:59:20.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.849+0000 7fc2e311d700 1 -- 172.21.15.29:0/184026041 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fc2dc062ef0 con 0x7fc2dc063ec0 2024-01-24T13:59:20.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.853+0000 7fc2d8ff9700 1 -- 172.21.15.29:0/184026041 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc2d004e120 con 0x7fc2dc063ec0 2024-01-24T13:59:20.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.855+0000 7fc2e311d700 1 -- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2c005f150 msgr2=0x7fc2c0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.855+0000 7fc2e311d700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2c005f150 0x7fc2c0061610 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fc2dc138d50 tx=0x7fc2c8017000).stop 2024-01-24T13:59:20.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.855+0000 7fc2e311d700 1 -- 172.21.15.29:0/184026041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2dc063ec0 msgr2=0x7fc2dc158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:20.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.855+0000 7fc2e311d700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2dc063ec0 0x7fc2dc158580 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fc2d0001eb0 tx=0x7fc2d0007c70).stop 2024-01-24T13:59:20.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.855+0000 7fc2e311d700 1 -- 172.21.15.29:0/184026041 shutdown_connections 2024-01-24T13:59:20.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.855+0000 7fc2e311d700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2dc13e220 0x7fc2dc0aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.855+0000 7fc2e311d700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2c005f150 0x7fc2c0061610 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.855+0000 7fc2e311d700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2dc063ec0 0x7fc2dc158580 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.855+0000 7fc2e311d700 1 --2- 172.21.15.29:0/184026041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2dc13de40 0x7fc2dc0a8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:20.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.855+0000 7fc2e311d700 1 -- 172.21.15.29:0/184026041 >> 172.21.15.29:0/184026041 conn(0x7fc2dc0c3cd0 msgr2=0x7fc2dc0a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:20.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.855+0000 7fc2e311d700 1 -- 172.21.15.29:0/184026041 shutdown_connections 2024-01-24T13:59:20.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:20.855+0000 7fc2e311d700 1 -- 172.21.15.29:0/184026041 wait complete. 2024-01-24T13:59:20.859 INFO:teuthology.orchestra.run.smithi029.stderr:updated 2024-01-24T13:59:20.869 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-01-24T13:59:21.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.000+0000 7f8023191700 1 Processor -- start 2024-01-24T13:59:21.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.000+0000 7f8023191700 1 -- start start 2024-01-24T13:59:21.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8023191700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f801c151aa0 0x7f801c0a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8023191700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f801c151e80 0x7f801c0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8023191700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801c065360 con 0x7f801c0a71d0 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8023191700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801c0bcab0 con 0x7f801c151aa0 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8023191700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801c0bcd90 con 0x7f801c151e80 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8020f2d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f801c151aa0 0x7f801c0a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8020f2d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f801c151aa0 0x7f801c0a6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41398/0 (socket says 172.21.15.29:41398) 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8020f2d700 1 -- 172.21.15.29:0/3902553283 learned_addr learned my addr 172.21.15.29:0/3902553283 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f801bfff700 1 --2- 172.21.15.29:0/3902553283 >> v2:172.21.15.163:3300/0 conn(0x7f801c151e80 0x7f801c0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8020f2d700 1 -- 172.21.15.29:0/3902553283 >> v2:172.21.15.163:3300/0 conn(0x7f801c151e80 msgr2=0x7f801c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8020f2d700 1 --2- 172.21.15.29:0/3902553283 >> v2:172.21.15.163:3300/0 conn(0x7f801c151e80 0x7f801c0627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8020f2d700 1 -- 172.21.15.29:0/3902553283 >> v1:172.21.15.163:6789/0 conn(0x7f801c0a71d0 legacy=0x7f801c157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8020f2d700 1 -- 172.21.15.29:0/3902553283 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c0bd070 con 0x7f801c151aa0 2024-01-24T13:59:21.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f801bfff700 1 --2- 172.21.15.29:0/3902553283 >> v2:172.21.15.163:3300/0 conn(0x7f801c151e80 0x7f801c0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:21.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.001+0000 7f8020f2d700 1 --2- 172.21.15.29:0/3902553283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801c151aa0 0x7f801c0a6ce0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f801c06a540 tx=0x7f80100090f0).ready entity=mon.0 client_cookie=97db0c48fc9f1b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:21.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.002+0000 7f801b7fe700 1 -- 172.21.15.29:0/3902553283 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801000b5a0 con 0x7f801c151aa0 2024-01-24T13:59:21.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.002+0000 7f801b7fe700 1 -- 172.21.15.29:0/3902553283 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801000b700 con 0x7f801c151aa0 2024-01-24T13:59:21.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.002+0000 7f801b7fe700 1 -- 172.21.15.29:0/3902553283 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801000ba80 con 0x7f801c151aa0 2024-01-24T13:59:21.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.002+0000 7f8023191700 1 -- 172.21.15.29:0/3902553283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801c151aa0 msgr2=0x7f801c0a6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.002+0000 7f8023191700 1 --2- 172.21.15.29:0/3902553283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801c151aa0 0x7f801c0a6ce0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f801c06a540 tx=0x7f80100090f0).stop 2024-01-24T13:59:21.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.002+0000 7f8023191700 1 -- 172.21.15.29:0/3902553283 shutdown_connections 2024-01-24T13:59:21.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.002+0000 7f8023191700 1 --2- 172.21.15.29:0/3902553283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801c151aa0 0x7f801c0a6ce0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.002+0000 7f8023191700 1 --2- 172.21.15.29:0/3902553283 >> v2:172.21.15.163:3300/0 conn(0x7f801c151e80 0x7f801c0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.002+0000 7f8023191700 1 -- 172.21.15.29:0/3902553283 >> 172.21.15.29:0/3902553283 conn(0x7f801c0c8580 msgr2=0x7f801c0c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:21.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.002+0000 7f8023191700 1 -- 172.21.15.29:0/3902553283 shutdown_connections 2024-01-24T13:59:21.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.002+0000 7f8023191700 1 -- 172.21.15.29:0/3902553283 wait complete. 2024-01-24T13:59:21.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.003+0000 7f8023191700 1 Processor -- start 2024-01-24T13:59:21.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.004+0000 7f8023191700 1 -- start start 2024-01-24T13:59:21.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.004+0000 7f8023191700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f801c0a71d0 0x7f801c13c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.004+0000 7f8023191700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801c151aa0 0x7f801c0aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.004+0000 7f8023191700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f801c151e80 0x7f801c0aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.004+0000 7f8023191700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c13c520 con 0x7f801c151aa0 2024-01-24T13:59:21.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.004+0000 7f8023191700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0ab480 con 0x7f801c151e80 2024-01-24T13:59:21.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.004+0000 7f8023191700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0a6470 con 0x7f801c0a71d0 2024-01-24T13:59:21.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f801bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801c151aa0 0x7f801c0aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f801bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801c151aa0 0x7f801c0aaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41406/0 (socket says 172.21.15.29:41406) 2024-01-24T13:59:21.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f801bfff700 1 -- 172.21.15.29:0/1063567982 learned_addr learned my addr 172.21.15.29:0/1063567982 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f8020f2d700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f801c0a71d0 0x7f801c13c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f802172e700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f801c151e80 0x7f801c0aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f801bfff700 1 -- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f801c0a71d0 msgr2=0x7f801c13c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f801bfff700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f801c0a71d0 0x7f801c13c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f801bfff700 1 -- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f801c151e80 msgr2=0x7f801c0aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f801bfff700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f801c151e80 0x7f801c0aaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f801bfff700 1 -- 172.21.15.29:0/1063567982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8010009370 con 0x7f801c151aa0 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f801bfff700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801c151aa0 0x7f801c0aaaa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f800800c880 tx=0x7f8008008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f8018ff9700 1 -- 172.21.15.29:0/1063567982 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008010580 con 0x7f801c151aa0 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f8018ff9700 1 -- 172.21.15.29:0/1063567982 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80080106e0 con 0x7f801c151aa0 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f8023191700 1 -- 172.21.15.29:0/1063567982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801c0a7c80 con 0x7f801c151aa0 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f8018ff9700 1 -- 172.21.15.29:0/1063567982 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008010a60 con 0x7f801c151aa0 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.005+0000 7f8023191700 1 -- 172.21.15.29:0/1063567982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c0ada50 con 0x7f801c151aa0 2024-01-24T13:59:21.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.006+0000 7f8023191700 1 -- 172.21.15.29:0/1063567982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801c0475e0 con 0x7f801c151aa0 2024-01-24T13:59:21.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.007+0000 7f8018ff9700 1 -- 172.21.15.29:0/1063567982 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f800801b460 con 0x7f801c151aa0 2024-01-24T13:59:21.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.007+0000 7f8018ff9700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f800005f150 0x7f8000061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.007+0000 7f8018ff9700 1 -- 172.21.15.29:0/1063567982 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5622+0+0 (secure 0 0 0) 0x7f8008079f00 con 0x7f801c151aa0 2024-01-24T13:59:21.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.008+0000 7f8020f2d700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f800005f150 0x7f8000061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.008+0000 7f8020f2d700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f800005f150 0x7f8000061610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8010009130 tx=0x7f8010015000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:21.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.012+0000 7f8018ff9700 1 -- 172.21.15.29:0/1063567982 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f800804b360 con 0x7f801c151aa0 2024-01-24T13:59:21.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.269+0000 7f8023191700 1 -- 172.21.15.29:0/1063567982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f801c0a7a50 con 0x7f801c151aa0 2024-01-24T13:59:21.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.272+0000 7f8018ff9700 1 -- 172.21.15.29:0/1063567982 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f801c0a7a50 con 0x7f801c151aa0 2024-01-24T13:59:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.274+0000 7f8023191700 1 -- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f800005f150 msgr2=0x7f8000061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.274+0000 7f8023191700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f800005f150 0x7f8000061610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8010009130 tx=0x7f8010015000).stop 2024-01-24T13:59:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.274+0000 7f8023191700 1 -- 172.21.15.29:0/1063567982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801c151aa0 msgr2=0x7f801c0aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.274+0000 7f8023191700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801c151aa0 0x7f801c0aaaa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f800800c880 tx=0x7f8008008040).stop 2024-01-24T13:59:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.274+0000 7f8023191700 1 -- 172.21.15.29:0/1063567982 shutdown_connections 2024-01-24T13:59:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.274+0000 7f8023191700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f801c151e80 0x7f801c0aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.274+0000 7f8023191700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f801c0a71d0 0x7f801c13c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.274+0000 7f8023191700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f800005f150 0x7f8000061610 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.274+0000 7f8023191700 1 --2- 172.21.15.29:0/1063567982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f801c151aa0 0x7f801c0aaaa0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.274+0000 7f8023191700 1 -- 172.21.15.29:0/1063567982 >> 172.21.15.29:0/1063567982 conn(0x7f801c0c8580 msgr2=0x7f801c0c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:21.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.275+0000 7f8023191700 1 -- 172.21.15.29:0/1063567982 shutdown_connections 2024-01-24T13:59:21.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.275+0000 7f8023191700 1 -- 172.21.15.29:0/1063567982 wait complete. 2024-01-24T13:59:21.276 INFO:teuthology.orchestra.run.smithi029.stderr:updated 2024-01-24T13:59:21.288 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-01-24T13:59:21.355 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-01-24T13:59:21.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.420+0000 7f8a38d54700 1 Processor -- start 2024-01-24T13:59:21.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.420+0000 7f8a38d54700 1 -- start start 2024-01-24T13:59:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.421+0000 7f8a38d54700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a34066bb0 0x7f8a341525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.421+0000 7f8a38d54700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8a34152ae0 0x7f8a340bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.421+0000 7f8a38d54700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a34156460 con 0x7f8a34135350 2024-01-24T13:59:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.421+0000 7f8a38d54700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a34155c00 con 0x7f8a34152ae0 2024-01-24T13:59:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.421+0000 7f8a38d54700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a34153390 con 0x7f8a34066bb0 2024-01-24T13:59:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.421+0000 7f8a3259c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8a34135350 0x7f8a340664a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:45888/0 (socket says 172.21.15.29:45888) 2024-01-24T13:59:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.421+0000 7f8a3259c700 1 -- 172.21.15.29:0/799467312 learned_addr learned my addr 172.21.15.29:0/799467312 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.421+0000 7f8a32d9d700 1 --2- 172.21.15.29:0/799467312 >> v2:172.21.15.163:3300/0 conn(0x7f8a34152ae0 0x7f8a340bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.422+0000 7f8a31d9b700 1 --2- 172.21.15.29:0/799467312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a34066bb0 0x7f8a341525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.422+0000 7f8a3159a700 1 -- 172.21.15.29:0/799467312 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1676115164 0 0) 0x7f8a34156460 con 0x7f8a34135350 2024-01-24T13:59:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.422+0000 7f8a3159a700 1 -- 172.21.15.29:0/799467312 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a200033f0 con 0x7f8a34135350 2024-01-24T13:59:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.422+0000 7f8a32d9d700 1 -- 172.21.15.29:0/799467312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a34066bb0 msgr2=0x7f8a341525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.422+0000 7f8a32d9d700 1 --2- 172.21.15.29:0/799467312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a34066bb0 0x7f8a341525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.422+0000 7f8a32d9d700 1 -- 172.21.15.29:0/799467312 >> v1:172.21.15.29:6789/0 conn(0x7f8a34135350 legacy=0x7f8a340664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.422+0000 7f8a32d9d700 1 -- 172.21.15.29:0/799467312 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34155e10 con 0x7f8a34152ae0 2024-01-24T13:59:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.422+0000 7f8a31d9b700 1 --2- 172.21.15.29:0/799467312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a34066bb0 0x7f8a341525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:59:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.422+0000 7f8a32d9d700 1 --2- 172.21.15.29:0/799467312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a34152ae0 0x7f8a340bdb00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f8a2c009e60 tx=0x7f8a2c009780).ready entity=mon.1 client_cookie=447f87990f41ca16 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.422+0000 7f8a3159a700 1 -- 172.21.15.29:0/799467312 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c00e070 con 0x7f8a34152ae0 2024-01-24T13:59:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.422+0000 7f8a3159a700 1 -- 172.21.15.29:0/799467312 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c00bab0 con 0x7f8a34152ae0 2024-01-24T13:59:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.422+0000 7f8a3159a700 1 -- 172.21.15.29:0/799467312 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c00be30 con 0x7f8a34152ae0 2024-01-24T13:59:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.423+0000 7f8a38d54700 1 -- 172.21.15.29:0/799467312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a34152ae0 msgr2=0x7f8a340bdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.423+0000 7f8a38d54700 1 --2- 172.21.15.29:0/799467312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a34152ae0 0x7f8a340bdb00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f8a2c009e60 tx=0x7f8a2c009780).stop 2024-01-24T13:59:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.423+0000 7f8a38d54700 1 -- 172.21.15.29:0/799467312 shutdown_connections 2024-01-24T13:59:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.423+0000 7f8a38d54700 1 --2- 172.21.15.29:0/799467312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a34152ae0 0x7f8a340bdb00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.423+0000 7f8a38d54700 1 --2- 172.21.15.29:0/799467312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a34066bb0 0x7f8a341525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.423+0000 7f8a38d54700 1 -- 172.21.15.29:0/799467312 >> 172.21.15.29:0/799467312 conn(0x7f8a340c5640 msgr2=0x7f8a340c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.423+0000 7f8a38d54700 1 -- 172.21.15.29:0/799467312 shutdown_connections 2024-01-24T13:59:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.423+0000 7f8a38d54700 1 -- 172.21.15.29:0/799467312 wait complete. 2024-01-24T13:59:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.424+0000 7f8a38d54700 1 Processor -- start 2024-01-24T13:59:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.424+0000 7f8a38d54700 1 -- start start 2024-01-24T13:59:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.425+0000 7f8a38d54700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a34066bb0 0x7f8a340c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.425+0000 7f8a38d54700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a34135350 0x7f8a340a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.425+0000 7f8a38d54700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a34152ae0 0x7f8a34165920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.425+0000 7f8a38d54700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a34137f30 con 0x7f8a34152ae0 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.425+0000 7f8a38d54700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a340c3680 con 0x7f8a34135350 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.425+0000 7f8a38d54700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a34137320 con 0x7f8a34066bb0 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.425+0000 7f8a32d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a34152ae0 0x7f8a34165920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.425+0000 7f8a31d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a34135350 0x7f8a340a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.425+0000 7f8a32d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a34152ae0 0x7f8a34165920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41416/0 (socket says 172.21.15.29:41416) 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.425+0000 7f8a32d9d700 1 -- 172.21.15.29:0/145404526 learned_addr learned my addr 172.21.15.29:0/145404526 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.425+0000 7f8a3259c700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a34066bb0 0x7f8a340c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.426+0000 7f8a32d9d700 1 -- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a34066bb0 msgr2=0x7f8a340c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.426+0000 7f8a32d9d700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a34066bb0 0x7f8a340c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.426+0000 7f8a32d9d700 1 -- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a34135350 msgr2=0x7f8a340a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.426+0000 7f8a32d9d700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a34135350 0x7f8a340a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.426+0000 7f8a32d9d700 1 -- 172.21.15.29:0/145404526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a240092a0 con 0x7f8a34152ae0 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.426+0000 7f8a32d9d700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a34152ae0 0x7f8a34165920 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f8a2c00b610 tx=0x7f8a2c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.426+0000 7f8a2a7fc700 1 -- 172.21.15.29:0/145404526 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c012ea0 con 0x7f8a34152ae0 2024-01-24T13:59:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.426+0000 7f8a2a7fc700 1 -- 172.21.15.29:0/145404526 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c01d630 con 0x7f8a34152ae0 2024-01-24T13:59:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.426+0000 7f8a38d54700 1 -- 172.21.15.29:0/145404526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2c009a20 con 0x7f8a34152ae0 2024-01-24T13:59:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.426+0000 7f8a2a7fc700 1 -- 172.21.15.29:0/145404526 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c01d9f0 con 0x7f8a34152ae0 2024-01-24T13:59:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.426+0000 7f8a38d54700 1 -- 172.21.15.29:0/145404526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a341378e0 con 0x7f8a34152ae0 2024-01-24T13:59:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.427+0000 7f8a38d54700 1 -- 172.21.15.29:0/145404526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a341380a0 con 0x7f8a34152ae0 2024-01-24T13:59:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.427+0000 7f8a2a7fc700 1 -- 172.21.15.29:0/145404526 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8a2c026410 con 0x7f8a34152ae0 2024-01-24T13:59:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.428+0000 7f8a2a7fc700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8a1405f150 0x7f8a14061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.428+0000 7f8a2a7fc700 1 -- 172.21.15.29:0/145404526 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5622+0+0 (secure 0 0 0) 0x7f8a2c07bd20 con 0x7f8a34152ae0 2024-01-24T13:59:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.428+0000 7f8a3259c700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8a1405f150 0x7f8a14061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.429+0000 7f8a3259c700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8a1405f150 0x7f8a14061610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f8a1c005750 tx=0x7f8a1c009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:21.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.432+0000 7f8a2a7fc700 1 -- 172.21.15.29:0/145404526 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8a2c04d180 con 0x7f8a34152ae0 2024-01-24T13:59:21.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.685+0000 7f8a38d54700 1 -- 172.21.15.29:0/145404526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f8a340475e0 con 0x7f8a34152ae0 2024-01-24T13:59:21.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.688+0000 7f8a2a7fc700 1 -- 172.21.15.29:0/145404526 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8a2c0507a0 con 0x7f8a34152ae0 2024-01-24T13:59:21.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.690+0000 7f8a38d54700 1 -- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8a1405f150 msgr2=0x7f8a14061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.690+0000 7f8a38d54700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8a1405f150 0x7f8a14061610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f8a1c005750 tx=0x7f8a1c009040).stop 2024-01-24T13:59:21.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.690+0000 7f8a38d54700 1 -- 172.21.15.29:0/145404526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a34152ae0 msgr2=0x7f8a34165920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.690+0000 7f8a38d54700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a34152ae0 0x7f8a34165920 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f8a2c00b610 tx=0x7f8a2c009780).stop 2024-01-24T13:59:21.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.690+0000 7f8a38d54700 1 -- 172.21.15.29:0/145404526 shutdown_connections 2024-01-24T13:59:21.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.690+0000 7f8a38d54700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8a1405f150 0x7f8a14061610 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.690+0000 7f8a38d54700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a34152ae0 0x7f8a34165920 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.690+0000 7f8a38d54700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a34066bb0 0x7f8a340c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.690+0000 7f8a38d54700 1 --2- 172.21.15.29:0/145404526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a34135350 0x7f8a340a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.691+0000 7f8a38d54700 1 -- 172.21.15.29:0/145404526 >> 172.21.15.29:0/145404526 conn(0x7f8a340c5640 msgr2=0x7f8a340679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:21.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.691+0000 7f8a38d54700 1 -- 172.21.15.29:0/145404526 shutdown_connections 2024-01-24T13:59:21.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.691+0000 7f8a38d54700 1 -- 172.21.15.29:0/145404526 wait complete. 2024-01-24T13:59:21.693 INFO:teuthology.orchestra.run.smithi029.stderr:updated 2024-01-24T13:59:21.704 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-01-24T13:59:21.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.836+0000 7f6a1ae3d700 1 Processor -- start 2024-01-24T13:59:21.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.837+0000 7f6a1ae3d700 1 -- start start 2024-01-24T13:59:21.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.837+0000 7f6a1ae3d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6a14150bd0 0x7f6a14065e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.837+0000 7f6a1ae3d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a141516c0 0x7f6a140bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a1ae3d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1406a350 con 0x7f6a14069240 2024-01-24T13:59:21.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a1ae3d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a14069740 con 0x7f6a14150bd0 2024-01-24T13:59:21.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a1ae3d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14069a20 con 0x7f6a141516c0 2024-01-24T13:59:21.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a18bd9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6a14150bd0 0x7f6a14065e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a18bd9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6a14150bd0 0x7f6a14065e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41418/0 (socket says 172.21.15.29:41418) 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a18bd9700 1 -- 172.21.15.29:0/2553774981 learned_addr learned my addr 172.21.15.29:0/2553774981 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a193da700 1 --2- 172.21.15.29:0/2553774981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a141516c0 0x7f6a140bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a18bd9700 1 -- 172.21.15.29:0/2553774981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a141516c0 msgr2=0x7f6a140bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a18bd9700 1 --2- 172.21.15.29:0/2553774981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a141516c0 0x7f6a140bcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a18bd9700 1 -- 172.21.15.29:0/2553774981 >> v1:172.21.15.163:6789/0 conn(0x7f6a14069240 legacy=0x7f6a14150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a18bd9700 1 -- 172.21.15.29:0/2553774981 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a14069d00 con 0x7f6a14150bd0 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a18bd9700 1 --2- 172.21.15.29:0/2553774981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a14150bd0 0x7f6a14065e50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f6a08006fd0 tx=0x7f6a080090f0).ready entity=mon.0 client_cookie=b411b8071e4d8ceb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a137fe700 1 -- 172.21.15.29:0/2553774981 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0800b600 con 0x7f6a14150bd0 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a137fe700 1 -- 172.21.15.29:0/2553774981 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0800b760 con 0x7f6a14150bd0 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.838+0000 7f6a137fe700 1 -- 172.21.15.29:0/2553774981 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0800bae0 con 0x7f6a14150bd0 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.839+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2553774981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a14150bd0 msgr2=0x7f6a14065e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.839+0000 7f6a1ae3d700 1 --2- 172.21.15.29:0/2553774981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a14150bd0 0x7f6a14065e50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f6a08006fd0 tx=0x7f6a080090f0).stop 2024-01-24T13:59:21.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.839+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2553774981 shutdown_connections 2024-01-24T13:59:21.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.839+0000 7f6a1ae3d700 1 --2- 172.21.15.29:0/2553774981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a141516c0 0x7f6a140bcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.839+0000 7f6a1ae3d700 1 --2- 172.21.15.29:0/2553774981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a14150bd0 0x7f6a14065e50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.839+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2553774981 >> 172.21.15.29:0/2553774981 conn(0x7f6a140c8580 msgr2=0x7f6a140c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:21.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.839+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2553774981 shutdown_connections 2024-01-24T13:59:21.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.839+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2553774981 wait complete. 2024-01-24T13:59:21.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.841+0000 7f6a1ae3d700 1 Processor -- start 2024-01-24T13:59:21.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.841+0000 7f6a1ae3d700 1 -- start start 2024-01-24T13:59:21.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a1ae3d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a14069240 0x7f6a140bf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a1ae3d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a14150bd0 0x7f6a140bed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a1ae3d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a141516c0 0x7f6a140c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a1ae3d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14162030 con 0x7f6a14069240 2024-01-24T13:59:21.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a1ae3d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140bf250 con 0x7f6a141516c0 2024-01-24T13:59:21.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a1ae3d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14161440 con 0x7f6a14150bd0 2024-01-24T13:59:21.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a18bd9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a14069240 0x7f6a140bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a18bd9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a14069240 0x7f6a140bf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41430/0 (socket says 172.21.15.29:41430) 2024-01-24T13:59:21.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a18bd9700 1 -- 172.21.15.29:0/2287722688 learned_addr learned my addr 172.21.15.29:0/2287722688 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:21.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a13fff700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a14150bd0 0x7f6a140bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a193da700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a141516c0 0x7f6a140c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a18bd9700 1 -- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a14150bd0 msgr2=0x7f6a140bed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a18bd9700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a14150bd0 0x7f6a140bed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a18bd9700 1 -- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a141516c0 msgr2=0x7f6a140c16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:21.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a18bd9700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a141516c0 0x7f6a140c16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:21.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.842+0000 7f6a18bd9700 1 -- 172.21.15.29:0/2287722688 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a08009370 con 0x7f6a14069240 2024-01-24T13:59:21.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.843+0000 7f6a18bd9700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a14069240 0x7f6a140bf560 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f6a140649f0 tx=0x7f6a08017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:21.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.843+0000 7f6a10ff9700 1 -- 172.21.15.29:0/2287722688 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0801f070 con 0x7f6a14069240 2024-01-24T13:59:21.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.843+0000 7f6a10ff9700 1 -- 172.21.15.29:0/2287722688 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a08006a40 con 0x7f6a14069240 2024-01-24T13:59:21.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.843+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2287722688 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a14161700 con 0x7f6a14069240 2024-01-24T13:59:21.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.843+0000 7f6a10ff9700 1 -- 172.21.15.29:0/2287722688 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0800bae0 con 0x7f6a14069240 2024-01-24T13:59:21.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.843+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2287722688 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a14161cc0 con 0x7f6a14069240 2024-01-24T13:59:21.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.844+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2287722688 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a14162310 con 0x7f6a14069240 2024-01-24T13:59:21.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.844+0000 7f6a10ff9700 1 -- 172.21.15.29:0/2287722688 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6a08024900 con 0x7f6a14069240 2024-01-24T13:59:21.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.845+0000 7f6a10ff9700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69f805f150 0x7f69f8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:21.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.845+0000 7f6a10ff9700 1 -- 172.21.15.29:0/2287722688 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5622+0+0 (secure 0 0 0) 0x7f6a0807d200 con 0x7f6a14069240 2024-01-24T13:59:21.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.845+0000 7f6a13fff700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69f805f150 0x7f69f8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:21.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.845+0000 7f6a13fff700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69f805f150 0x7f69f8061610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f6a14164970 tx=0x7f6a00008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:21.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:21.849+0000 7f6a10ff9700 1 -- 172.21.15.29:0/2287722688 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f6a0804e6c0 con 0x7f6a14069240 2024-01-24T13:59:22.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.105+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2287722688 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f6a140475e0 con 0x7f6a14069240 2024-01-24T13:59:22.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.109+0000 7f6a10ff9700 1 -- 172.21.15.29:0/2287722688 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6a08029070 con 0x7f6a14069240 2024-01-24T13:59:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.111+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69f805f150 msgr2=0x7f69f8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.111+0000 7f6a1ae3d700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69f805f150 0x7f69f8061610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f6a14164970 tx=0x7f6a00008040).stop 2024-01-24T13:59:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.111+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2287722688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a14069240 msgr2=0x7f6a140bf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.111+0000 7f6a1ae3d700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a14069240 0x7f6a140bf560 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f6a140649f0 tx=0x7f6a08017000).stop 2024-01-24T13:59:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.111+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2287722688 shutdown_connections 2024-01-24T13:59:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.111+0000 7f6a1ae3d700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a141516c0 0x7f6a140c16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.111+0000 7f6a1ae3d700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69f805f150 0x7f69f8061610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.111+0000 7f6a1ae3d700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a14069240 0x7f6a140bf560 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.111+0000 7f6a1ae3d700 1 --2- 172.21.15.29:0/2287722688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a14150bd0 0x7f6a140bed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.111+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2287722688 >> 172.21.15.29:0/2287722688 conn(0x7f6a140c8580 msgr2=0x7f6a140c42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.111+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2287722688 shutdown_connections 2024-01-24T13:59:22.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.111+0000 7f6a1ae3d700 1 -- 172.21.15.29:0/2287722688 wait complete. 2024-01-24T13:59:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:updated 2024-01-24T13:59:22.125 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-01-24T13:59:22.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.254+0000 7f232cb67700 1 Processor -- start 2024-01-24T13:59:22.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.254+0000 7f232cb67700 1 -- start start 2024-01-24T13:59:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f232cb67700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2328150bd0 0x7f2328065e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f232cb67700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f232806a350 con 0x7f23281516c0 2024-01-24T13:59:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f232cb67700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2328069740 con 0x7f2328069240 2024-01-24T13:59:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f232cb67700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2328069a20 con 0x7f2328150bd0 2024-01-24T13:59:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f232659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2328150bd0 0x7f2328065e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f2326d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f23281516c0 0x7f23280bcd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:45890/0 (socket says 172.21.15.29:45890) 2024-01-24T13:59:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f2326d9d700 1 -- 172.21.15.29:0/1844866647 learned_addr learned my addr 172.21.15.29:0/1844866647 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:22.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f232659c700 1 -- 172.21.15.29:0/1844866647 >> v1:172.21.15.163:6789/0 conn(0x7f2328069240 legacy=0x7f2328150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f232659c700 1 -- 172.21.15.29:0/1844866647 >> v1:172.21.15.29:6789/0 conn(0x7f23281516c0 legacy=0x7f23280bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f232559a700 1 -- 172.21.15.29:0/1844866647 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775527763 0 0) 0x7f232806a350 con 0x7f23281516c0 2024-01-24T13:59:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f232659c700 1 -- 172.21.15.29:0/1844866647 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2328069d00 con 0x7f2328150bd0 2024-01-24T13:59:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f232659c700 1 --2- 172.21.15.29:0/1844866647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2328150bd0 0x7f2328065e50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f2318009760 tx=0x7f2318009060).ready entity=mon.0 client_cookie=4202de838cb2e6d2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f232559a700 1 -- 172.21.15.29:0/1844866647 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231800b580 con 0x7f2328150bd0 2024-01-24T13:59:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.256+0000 7f232559a700 1 -- 172.21.15.29:0/1844866647 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231800b6e0 con 0x7f2328150bd0 2024-01-24T13:59:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.257+0000 7f232559a700 1 -- 172.21.15.29:0/1844866647 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231800ba60 con 0x7f2328150bd0 2024-01-24T13:59:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.257+0000 7f232cb67700 1 -- 172.21.15.29:0/1844866647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2328150bd0 msgr2=0x7f2328065e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.257+0000 7f232cb67700 1 --2- 172.21.15.29:0/1844866647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2328150bd0 0x7f2328065e50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f2318009760 tx=0x7f2318009060).stop 2024-01-24T13:59:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.257+0000 7f232cb67700 1 -- 172.21.15.29:0/1844866647 shutdown_connections 2024-01-24T13:59:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.257+0000 7f232cb67700 1 --2- 172.21.15.29:0/1844866647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2328150bd0 0x7f2328065e50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.257+0000 7f232cb67700 1 -- 172.21.15.29:0/1844866647 >> 172.21.15.29:0/1844866647 conn(0x7f23280c8580 msgr2=0x7f23280c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.257+0000 7f232cb67700 1 -- 172.21.15.29:0/1844866647 shutdown_connections 2024-01-24T13:59:22.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.257+0000 7f232cb67700 1 -- 172.21.15.29:0/1844866647 wait complete. 2024-01-24T13:59:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.258+0000 7f232cb67700 1 Processor -- start 2024-01-24T13:59:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.258+0000 7f232cb67700 1 -- start start 2024-01-24T13:59:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.259+0000 7f232cb67700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2328069240 0x7f23280a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.259+0000 7f232cb67700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2328150bd0 0x7f232814d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.259+0000 7f232cb67700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23281516c0 0x7f23281672a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.259+0000 7f232cb67700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232814f5c0 con 0x7f2328069240 2024-01-24T13:59:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.259+0000 7f232cb67700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23280beb50 con 0x7f2328150bd0 2024-01-24T13:59:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.259+0000 7f232cb67700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232814de50 con 0x7f23281516c0 2024-01-24T13:59:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.259+0000 7f232659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2328069240 0x7f23280a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.259+0000 7f232659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2328069240 0x7f23280a9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41448/0 (socket says 172.21.15.29:41448) 2024-01-24T13:59:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.259+0000 7f232659c700 1 -- 172.21.15.29:0/3607071800 learned_addr learned my addr 172.21.15.29:0/3607071800 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:22.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f2325d9b700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2328150bd0 0x7f232814d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f2326d9d700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23281516c0 0x7f23281672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f232659c700 1 -- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23281516c0 msgr2=0x7f23281672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f232659c700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23281516c0 0x7f23281672a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f232659c700 1 -- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2328150bd0 msgr2=0x7f232814d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f232659c700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2328150bd0 0x7f232814d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f232659c700 1 -- 172.21.15.29:0/3607071800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23180092a0 con 0x7f2328069240 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f2326d9d700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23281516c0 0x7f23281672a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f232659c700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2328069240 0x7f23280a9ff0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f2318009eb0 tx=0x7f2318017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f23167fc700 1 -- 172.21.15.29:0/3607071800 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231800ba60 con 0x7f2328069240 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f23167fc700 1 -- 172.21.15.29:0/3607071800 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2318000c90 con 0x7f2328069240 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f232cb67700 1 -- 172.21.15.29:0/3607071800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232814e940 con 0x7f2328069240 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f23167fc700 1 -- 172.21.15.29:0/3607071800 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231801b640 con 0x7f2328069240 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.260+0000 7f232cb67700 1 -- 172.21.15.29:0/3607071800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232814ef00 con 0x7f2328069240 2024-01-24T13:59:22.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.261+0000 7f232cb67700 1 -- 172.21.15.29:0/3607071800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23280aa4e0 con 0x7f2328069240 2024-01-24T13:59:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.261+0000 7f23167fc700 1 -- 172.21.15.29:0/3607071800 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f231801b9f0 con 0x7f2328069240 2024-01-24T13:59:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.262+0000 7f23167fc700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f230805f150 0x7f2308061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.262+0000 7f23167fc700 1 -- 172.21.15.29:0/3607071800 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5622+0+0 (secure 0 0 0) 0x7f231807bb10 con 0x7f2328069240 2024-01-24T13:59:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.262+0000 7f2325d9b700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f230805f150 0x7f2308061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.263+0000 7f2325d9b700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f230805f150 0x7f2308061610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f232814e8d0 tx=0x7f231c007d40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:22.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.266+0000 7f23167fc700 1 -- 172.21.15.29:0/3607071800 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f231804cf70 con 0x7f2328069240 2024-01-24T13:59:22.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.521+0000 7f232cb67700 1 -- 172.21.15.29:0/3607071800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f2328069fe0 con 0x7f2328069240 2024-01-24T13:59:22.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.525+0000 7f23167fc700 1 -- 172.21.15.29:0/3607071800 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2318028070 con 0x7f2328069240 2024-01-24T13:59:22.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.527+0000 7f232cb67700 1 -- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f230805f150 msgr2=0x7f2308061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.527+0000 7f232cb67700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f230805f150 0x7f2308061610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f232814e8d0 tx=0x7f231c007d40).stop 2024-01-24T13:59:22.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.527+0000 7f232cb67700 1 -- 172.21.15.29:0/3607071800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2328069240 msgr2=0x7f23280a9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.527+0000 7f232cb67700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2328069240 0x7f23280a9ff0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f2318009eb0 tx=0x7f2318017000).stop 2024-01-24T13:59:22.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.527+0000 7f232cb67700 1 -- 172.21.15.29:0/3607071800 shutdown_connections 2024-01-24T13:59:22.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.527+0000 7f232cb67700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23281516c0 0x7f23281672a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.527+0000 7f232cb67700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f230805f150 0x7f2308061610 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.527+0000 7f232cb67700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2328069240 0x7f23280a9ff0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.527+0000 7f232cb67700 1 --2- 172.21.15.29:0/3607071800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2328150bd0 0x7f232814d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.527+0000 7f232cb67700 1 -- 172.21.15.29:0/3607071800 >> 172.21.15.29:0/3607071800 conn(0x7f23280c8580 msgr2=0x7f2328063ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:22.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.527+0000 7f232cb67700 1 -- 172.21.15.29:0/3607071800 shutdown_connections 2024-01-24T13:59:22.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.527+0000 7f232cb67700 1 -- 172.21.15.29:0/3607071800 wait complete. 2024-01-24T13:59:22.529 INFO:teuthology.orchestra.run.smithi029.stderr:updated 2024-01-24T13:59:22.541 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-01-24T13:59:22.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.671+0000 7fcd80f15700 1 Processor -- start 2024-01-24T13:59:22.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.671+0000 7fcd80f15700 1 -- start start 2024-01-24T13:59:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.672+0000 7fcd80f15700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c135350 0x7fcd7c0664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.672+0000 7fcd80f15700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcd7c152d00 0x7fcd7c0bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.672+0000 7fcd80f15700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c156460 con 0x7fcd7c066990 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.672+0000 7fcd80f15700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c1531b0 con 0x7fcd7c152d00 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.672+0000 7fcd80f15700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c155c00 con 0x7fcd7c135350 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.672+0000 7fcd79d9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcd7c066990 0x7fcd7c1525f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:45902/0 (socket says 172.21.15.29:45902) 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.672+0000 7fcd79d9b700 1 -- 172.21.15.29:0/3996412486 learned_addr learned my addr 172.21.15.29:0/3996412486 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.672+0000 7fcd7ad9d700 1 --2- 172.21.15.29:0/3996412486 >> v2:172.21.15.163:3300/0 conn(0x7fcd7c152d00 0x7fcd7c0bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.672+0000 7fcd7a59c700 1 --2- 172.21.15.29:0/3996412486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c135350 0x7fcd7c0664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.673+0000 7fcd7959a700 1 -- 172.21.15.29:0/3996412486 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622236769 0 0) 0x7fcd7c156460 con 0x7fcd7c066990 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.673+0000 7fcd7959a700 1 -- 172.21.15.29:0/3996412486 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd680033f0 con 0x7fcd7c066990 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.673+0000 7fcd7ad9d700 1 -- 172.21.15.29:0/3996412486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c135350 msgr2=0x7fcd7c0664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.673+0000 7fcd7ad9d700 1 --2- 172.21.15.29:0/3996412486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c135350 0x7fcd7c0664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.673+0000 7fcd7ad9d700 1 -- 172.21.15.29:0/3996412486 >> v1:172.21.15.29:6789/0 conn(0x7fcd7c066990 legacy=0x7fcd7c1525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.673+0000 7fcd7959a700 1 -- 172.21.15.29:0/3996412486 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3898297609 0 0) 0x7fcd680033f0 con 0x7fcd7c066990 2024-01-24T13:59:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.673+0000 7fcd7ad9d700 1 -- 172.21.15.29:0/3996412486 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c066e10 con 0x7fcd7c152d00 2024-01-24T13:59:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.673+0000 7fcd7ad9d700 1 --2- 172.21.15.29:0/3996412486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c152d00 0x7fcd7c0bdb00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fcd74009e60 tx=0x7fcd74009780).ready entity=mon.1 client_cookie=c88cc93583abe82b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.673+0000 7fcd7959a700 1 -- 172.21.15.29:0/3996412486 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7400e070 con 0x7fcd7c152d00 2024-01-24T13:59:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.673+0000 7fcd7959a700 1 -- 172.21.15.29:0/3996412486 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7400b7a0 con 0x7fcd7c152d00 2024-01-24T13:59:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.673+0000 7fcd7959a700 1 -- 172.21.15.29:0/3996412486 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c001470 con 0x7fcd7c152d00 2024-01-24T13:59:22.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.674+0000 7fcd80f15700 1 -- 172.21.15.29:0/3996412486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c152d00 msgr2=0x7fcd7c0bdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.674+0000 7fcd80f15700 1 --2- 172.21.15.29:0/3996412486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c152d00 0x7fcd7c0bdb00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fcd74009e60 tx=0x7fcd74009780).stop 2024-01-24T13:59:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.674+0000 7fcd80f15700 1 -- 172.21.15.29:0/3996412486 shutdown_connections 2024-01-24T13:59:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.674+0000 7fcd80f15700 1 --2- 172.21.15.29:0/3996412486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c152d00 0x7fcd7c0bdb00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.674+0000 7fcd80f15700 1 --2- 172.21.15.29:0/3996412486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c135350 0x7fcd7c0664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.674+0000 7fcd80f15700 1 -- 172.21.15.29:0/3996412486 >> 172.21.15.29:0/3996412486 conn(0x7fcd7c0c5640 msgr2=0x7fcd7c0c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.674+0000 7fcd80f15700 1 -- 172.21.15.29:0/3996412486 shutdown_connections 2024-01-24T13:59:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.674+0000 7fcd80f15700 1 -- 172.21.15.29:0/3996412486 wait complete. 2024-01-24T13:59:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.675+0000 7fcd80f15700 1 Processor -- start 2024-01-24T13:59:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.675+0000 7fcd80f15700 1 -- start start 2024-01-24T13:59:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.676+0000 7fcd80f15700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c066990 0x7fcd7c167680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.676+0000 7fcd80f15700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c135350 0x7fcd7c164830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:22.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.676+0000 7fcd80f15700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd7c152d00 0x7fcd7c164d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:22.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.676+0000 7fcd80f15700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c167b70 con 0x7fcd7c152d00 2024-01-24T13:59:22.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.676+0000 7fcd80f15700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c165360 con 0x7fcd7c066990 2024-01-24T13:59:22.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.676+0000 7fcd80f15700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c165640 con 0x7fcd7c135350 2024-01-24T13:59:22.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.676+0000 7fcd7ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd7c152d00 0x7fcd7c164d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:22.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.676+0000 7fcd7ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd7c152d00 0x7fcd7c164d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41460/0 (socket says 172.21.15.29:41460) 2024-01-24T13:59:22.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.676+0000 7fcd7ad9d700 1 -- 172.21.15.29:0/184388164 learned_addr learned my addr 172.21.15.29:0/184388164 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:22.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.676+0000 7fcd79d9b700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c135350 0x7fcd7c164830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:22.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.676+0000 7fcd7a59c700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c066990 0x7fcd7c167680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:22.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.677+0000 7fcd7ad9d700 1 -- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c135350 msgr2=0x7fcd7c164830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.677+0000 7fcd7ad9d700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c135350 0x7fcd7c164830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.677+0000 7fcd7ad9d700 1 -- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c066990 msgr2=0x7fcd7c167680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.677+0000 7fcd7ad9d700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c066990 0x7fcd7c167680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.677+0000 7fcd7ad9d700 1 -- 172.21.15.29:0/184388164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd74009a20 con 0x7fcd7c152d00 2024-01-24T13:59:22.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.677+0000 7fcd7ad9d700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd7c152d00 0x7fcd7c164d20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fcd7400b610 tx=0x7fcd74009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:22.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.677+0000 7fcd727fc700 1 -- 172.21.15.29:0/184388164 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7401d420 con 0x7fcd7c152d00 2024-01-24T13:59:22.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.677+0000 7fcd727fc700 1 -- 172.21.15.29:0/184388164 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7401d580 con 0x7fcd7c152d00 2024-01-24T13:59:22.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.677+0000 7fcd80f15700 1 -- 172.21.15.29:0/184388164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c0ad480 con 0x7fcd7c152d00 2024-01-24T13:59:22.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.677+0000 7fcd727fc700 1 -- 172.21.15.29:0/184388164 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7401d940 con 0x7fcd7c152d00 2024-01-24T13:59:22.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.677+0000 7fcd80f15700 1 -- 172.21.15.29:0/184388164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c168030 con 0x7fcd7c152d00 2024-01-24T13:59:22.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.678+0000 7fcd80f15700 1 -- 172.21.15.29:0/184388164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c0a8480 con 0x7fcd7c152d00 2024-01-24T13:59:22.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.678+0000 7fcd727fc700 1 -- 172.21.15.29:0/184388164 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fcd7401dd30 con 0x7fcd7c152d00 2024-01-24T13:59:22.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.679+0000 7fcd727fc700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd5c05f150 0x7fcd5c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:22.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.679+0000 7fcd727fc700 1 -- 172.21.15.29:0/184388164 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5622+0+0 (secure 0 0 0) 0x7fcd7407ba60 con 0x7fcd7c152d00 2024-01-24T13:59:22.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.679+0000 7fcd7a59c700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd5c05f150 0x7fcd5c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:22.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.680+0000 7fcd7a59c700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd5c05f150 0x7fcd5c061610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fcd7c14dfa0 tx=0x7fcd64009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:22.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.683+0000 7fcd727fc700 1 -- 172.21.15.29:0/184388164 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcd7404cf20 con 0x7fcd7c152d00 2024-01-24T13:59:22.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.937+0000 7fcd80f15700 1 -- 172.21.15.29:0/184388164 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fcd7c0475e0 con 0x7fcd7c152d00 2024-01-24T13:59:22.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.940+0000 7fcd727fc700 1 -- 172.21.15.29:0/184388164 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v14) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcd74050540 con 0x7fcd7c152d00 2024-01-24T13:59:22.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.942+0000 7fcd80f15700 1 -- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd5c05f150 msgr2=0x7fcd5c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.942+0000 7fcd80f15700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd5c05f150 0x7fcd5c061610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fcd7c14dfa0 tx=0x7fcd64009060).stop 2024-01-24T13:59:22.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.942+0000 7fcd80f15700 1 -- 172.21.15.29:0/184388164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd7c152d00 msgr2=0x7fcd7c164d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:22.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.942+0000 7fcd80f15700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd7c152d00 0x7fcd7c164d20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fcd7400b610 tx=0x7fcd74009780).stop 2024-01-24T13:59:22.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.942+0000 7fcd80f15700 1 -- 172.21.15.29:0/184388164 shutdown_connections 2024-01-24T13:59:22.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.942+0000 7fcd80f15700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd5c05f150 0x7fcd5c061610 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.942+0000 7fcd80f15700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd7c152d00 0x7fcd7c164d20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.942+0000 7fcd80f15700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c066990 0x7fcd7c167680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.942+0000 7fcd80f15700 1 --2- 172.21.15.29:0/184388164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c135350 0x7fcd7c164830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:22.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.942+0000 7fcd80f15700 1 -- 172.21.15.29:0/184388164 >> 172.21.15.29:0/184388164 conn(0x7fcd7c0c5640 msgr2=0x7fcd7c0bde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:22.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.942+0000 7fcd80f15700 1 -- 172.21.15.29:0/184388164 shutdown_connections 2024-01-24T13:59:22.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:22.942+0000 7fcd80f15700 1 -- 172.21.15.29:0/184388164 wait complete. 2024-01-24T13:59:22.944 INFO:teuthology.orchestra.run.smithi029.stderr:updated 2024-01-24T13:59:22.956 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:59:23.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.085+0000 7f1b84065700 1 Processor -- start 2024-01-24T13:59:23.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.085+0000 7f1b84065700 1 -- start start 2024-01-24T13:59:23.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.086+0000 7f1b84065700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b7c1350a0 0x7f1b7c0c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.086+0000 7f1b84065700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7c068360 con 0x7f1b7c134cc0 2024-01-24T13:59:23.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.086+0000 7f1b84065700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7c0c3820 con 0x7f1b7c066c00 2024-01-24T13:59:23.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.086+0000 7f1b84065700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b7c067a30 con 0x7f1b7c1350a0 2024-01-24T13:59:23.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.087+0000 7f1b81e01700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1b7c134cc0 0x7f1b7c0664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:45912/0 (socket says 172.21.15.29:45912) 2024-01-24T13:59:23.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.087+0000 7f1b81e01700 1 -- 172.21.15.29:0/2899854559 learned_addr learned my addr 172.21.15.29:0/2899854559 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:23.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.087+0000 7f1b82602700 1 --2- 172.21.15.29:0/2899854559 >> v2:172.21.15.163:3300/0 conn(0x7f1b7c1350a0 0x7f1b7c0c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.087+0000 7f1b80dff700 1 -- 172.21.15.29:0/2899854559 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054564757 0 0) 0x7f1b7c068360 con 0x7f1b7c134cc0 2024-01-24T13:59:23.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.087+0000 7f1b80dff700 1 -- 172.21.15.29:0/2899854559 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b680033f0 con 0x7f1b7c134cc0 2024-01-24T13:59:23.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.087+0000 7f1b80dff700 1 -- 172.21.15.29:0/2899854559 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313311608 0 0) 0x7f1b7c0c3820 con 0x7f1b7c066c00 2024-01-24T13:59:23.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.087+0000 7f1b80dff700 1 -- 172.21.15.29:0/2899854559 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b7c068360 con 0x7f1b7c066c00 2024-01-24T13:59:23.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.087+0000 7f1b82602700 1 -- 172.21.15.29:0/2899854559 >> v1:172.21.15.163:6789/0 conn(0x7f1b7c066c00 legacy=0x7f1b7c159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.087+0000 7f1b82602700 1 -- 172.21.15.29:0/2899854559 >> v1:172.21.15.29:6789/0 conn(0x7f1b7c134cc0 legacy=0x7f1b7c0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.087+0000 7f1b82602700 1 -- 172.21.15.29:0/2899854559 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c067d10 con 0x7f1b7c1350a0 2024-01-24T13:59:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.087+0000 7f1b82602700 1 --2- 172.21.15.29:0/2899854559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b7c1350a0 0x7f1b7c0c31b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f1b74009e60 tx=0x7f1b74009780).ready entity=mon.1 client_cookie=2e3650d7271215d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.088+0000 7f1b80dff700 1 -- 172.21.15.29:0/2899854559 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7400e070 con 0x7f1b7c1350a0 2024-01-24T13:59:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.088+0000 7f1b80dff700 1 -- 172.21.15.29:0/2899854559 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7400bab0 con 0x7f1b7c1350a0 2024-01-24T13:59:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.088+0000 7f1b80dff700 1 -- 172.21.15.29:0/2899854559 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7400be30 con 0x7f1b7c1350a0 2024-01-24T13:59:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.088+0000 7f1b84065700 1 -- 172.21.15.29:0/2899854559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b7c1350a0 msgr2=0x7f1b7c0c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.088+0000 7f1b84065700 1 --2- 172.21.15.29:0/2899854559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b7c1350a0 0x7f1b7c0c31b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f1b74009e60 tx=0x7f1b74009780).stop 2024-01-24T13:59:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.088+0000 7f1b84065700 1 -- 172.21.15.29:0/2899854559 shutdown_connections 2024-01-24T13:59:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.088+0000 7f1b84065700 1 --2- 172.21.15.29:0/2899854559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b7c1350a0 0x7f1b7c0c31b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.088+0000 7f1b84065700 1 -- 172.21.15.29:0/2899854559 >> 172.21.15.29:0/2899854559 conn(0x7f1b7c0c8bb0 msgr2=0x7f1b7c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.089+0000 7f1b84065700 1 -- 172.21.15.29:0/2899854559 shutdown_connections 2024-01-24T13:59:23.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.089+0000 7f1b84065700 1 -- 172.21.15.29:0/2899854559 wait complete. 2024-01-24T13:59:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.090+0000 7f1b84065700 1 Processor -- start 2024-01-24T13:59:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.090+0000 7f1b84065700 1 -- start start 2024-01-24T13:59:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b84065700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c066c00 0x7f1b7c133780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b84065700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b7c134cc0 0x7f1b7c0c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b84065700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b7c1350a0 0x7f1b7c0ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b84065700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c0abd90 con 0x7f1b7c066c00 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b84065700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c133c70 con 0x7f1b7c134cc0 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b84065700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c0ae620 con 0x7f1b7c1350a0 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b81e01700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c066c00 0x7f1b7c133780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b81e01700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c066c00 0x7f1b7c133780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41468/0 (socket says 172.21.15.29:41468) 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b81e01700 1 -- 172.21.15.29:0/2849900681 learned_addr learned my addr 172.21.15.29:0/2849900681 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b82602700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b7c1350a0 0x7f1b7c0ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b81600700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b7c134cc0 0x7f1b7c0c0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b81e01700 1 -- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b7c1350a0 msgr2=0x7f1b7c0ac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b81e01700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b7c1350a0 0x7f1b7c0ac220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b81e01700 1 -- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b7c134cc0 msgr2=0x7f1b7c0c0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b81e01700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b7c134cc0 0x7f1b7c0c0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b81e01700 1 -- 172.21.15.29:0/2849900681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b74009a20 con 0x7f1b7c066c00 2024-01-24T13:59:23.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b81600700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b7c134cc0 0x7f1b7c0c0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:23.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b81e01700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c066c00 0x7f1b7c133780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f1b7800c560 tx=0x7f1b78007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:23.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.091+0000 7f1b71ffb700 1 -- 172.21.15.29:0/2849900681 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b780072e0 con 0x7f1b7c066c00 2024-01-24T13:59:23.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.092+0000 7f1b71ffb700 1 -- 172.21.15.29:0/2849900681 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b78007440 con 0x7f1b7c066c00 2024-01-24T13:59:23.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.092+0000 7f1b84065700 1 -- 172.21.15.29:0/2849900681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7c0bf270 con 0x7f1b7c066c00 2024-01-24T13:59:23.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.092+0000 7f1b71ffb700 1 -- 172.21.15.29:0/2849900681 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b780057a0 con 0x7f1b7c066c00 2024-01-24T13:59:23.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.092+0000 7f1b84065700 1 -- 172.21.15.29:0/2849900681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7c0ae340 con 0x7f1b7c066c00 2024-01-24T13:59:23.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.093+0000 7f1b84065700 1 -- 172.21.15.29:0/2849900681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7c0abb60 con 0x7f1b7c066c00 2024-01-24T13:59:23.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.093+0000 7f1b71ffb700 1 -- 172.21.15.29:0/2849900681 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1b78005bd0 con 0x7f1b7c066c00 2024-01-24T13:59:23.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.093+0000 7f1b71ffb700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b6005f150 0x7f1b60061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.094+0000 7f1b71ffb700 1 -- 172.21.15.29:0/2849900681 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5622+0+0 (secure 0 0 0) 0x7f1b78079870 con 0x7f1b7c066c00 2024-01-24T13:59:23.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.094+0000 7f1b81600700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b6005f150 0x7f1b60061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.094+0000 7f1b81600700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b6005f150 0x7f1b60061610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f1b7c0bedf0 tx=0x7f1b6c009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.098+0000 7f1b71ffb700 1 -- 172.21.15.29:0/2849900681 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1b7c0abb60 con 0x7f1b7c066c00 2024-01-24T13:59:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.278+0000 7f1b84065700 1 -- 172.21.15.29:0/2849900681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b7c0abb60 con 0x7f1b7c066c00 2024-01-24T13:59:23.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.279+0000 7f1b71ffb700 1 -- 172.21.15.29:0/2849900681 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+13948 (secure 0 0 0) 0x7f1b7c0abb60 con 0x7f1b7c066c00 2024-01-24T13:59:23.279 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:23.280 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":37,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:59:19.999430+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:59:23.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.281+0000 7f1b84065700 1 -- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b6005f150 msgr2=0x7f1b60061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.281+0000 7f1b84065700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b6005f150 0x7f1b60061610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f1b7c0bedf0 tx=0x7f1b6c009040).stop 2024-01-24T13:59:23.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.281+0000 7f1b84065700 1 -- 172.21.15.29:0/2849900681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c066c00 msgr2=0x7f1b7c133780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.281+0000 7f1b84065700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c066c00 0x7f1b7c133780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f1b7800c560 tx=0x7f1b78007d20).stop 2024-01-24T13:59:23.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.281+0000 7f1b84065700 1 -- 172.21.15.29:0/2849900681 shutdown_connections 2024-01-24T13:59:23.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.281+0000 7f1b84065700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b7c1350a0 0x7f1b7c0ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.281+0000 7f1b84065700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b6005f150 0x7f1b60061610 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.281+0000 7f1b84065700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c066c00 0x7f1b7c133780 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.281+0000 7f1b84065700 1 --2- 172.21.15.29:0/2849900681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b7c134cc0 0x7f1b7c0c0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.281+0000 7f1b84065700 1 -- 172.21.15.29:0/2849900681 >> 172.21.15.29:0/2849900681 conn(0x7f1b7c0c8bb0 msgr2=0x7f1b7c13aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:23.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.281+0000 7f1b84065700 1 -- 172.21.15.29:0/2849900681 shutdown_connections 2024-01-24T13:59:23.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.281+0000 7f1b84065700 1 -- 172.21.15.29:0/2849900681 wait complete. 2024-01-24T13:59:23.295 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T13:59:23.295 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T13:59:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.428+0000 7f60c45e3700 1 Processor -- start 2024-01-24T13:59:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.428+0000 7f60c45e3700 1 -- start start 2024-01-24T13:59:23.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.429+0000 7f60c45e3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60bc0664c0 0x7f60bc066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.429+0000 7f60c45e3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f60bc0bdb30 0x7f60bc0bdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.429+0000 7f60c45e3700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60bc0686a0 con 0x7f60bc134ea0 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.429+0000 7f60c45e3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60bc066e30 con 0x7f60bc0bdb30 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.429+0000 7f60c45e3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc0c38e0 con 0x7f60bc0664c0 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.429+0000 7f60c237f700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f60bc134ea0 0x7f60bc0c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54006/0 (socket says 172.21.15.29:54006) 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.429+0000 7f60c237f700 1 -- 172.21.15.29:0/3976280028 learned_addr learned my addr 172.21.15.29:0/3976280028 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.429+0000 7f60c1b7e700 1 --2- 172.21.15.29:0/3976280028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60bc0664c0 0x7f60bc066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.429+0000 7f60c2b80700 1 --2- 172.21.15.29:0/3976280028 >> v2:172.21.15.163:3300/0 conn(0x7f60bc0bdb30 0x7f60bc0bdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.430+0000 7f60c137d700 1 -- 172.21.15.29:0/3976280028 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976591835 0 0) 0x7f60bc0686a0 con 0x7f60bc134ea0 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.430+0000 7f60c137d700 1 -- 172.21.15.29:0/3976280028 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60a80033f0 con 0x7f60bc134ea0 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.430+0000 7f60c1b7e700 1 -- 172.21.15.29:0/3976280028 >> v2:172.21.15.163:3300/0 conn(0x7f60bc0bdb30 msgr2=0x7f60bc0bdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.430+0000 7f60c1b7e700 1 --2- 172.21.15.29:0/3976280028 >> v2:172.21.15.163:3300/0 conn(0x7f60bc0bdb30 0x7f60bc0bdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.430+0000 7f60c1b7e700 1 -- 172.21.15.29:0/3976280028 >> v1:172.21.15.29:6789/0 conn(0x7f60bc134ea0 legacy=0x7f60bc0c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.430+0000 7f60c1b7e700 1 -- 172.21.15.29:0/3976280028 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60bc068b30 con 0x7f60bc0664c0 2024-01-24T13:59:23.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.430+0000 7f60c2b80700 1 --2- 172.21.15.29:0/3976280028 >> v2:172.21.15.163:3300/0 conn(0x7f60bc0bdb30 0x7f60bc0bdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:59:23.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.430+0000 7f60c1b7e700 1 --2- 172.21.15.29:0/3976280028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60bc0664c0 0x7f60bc066940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f60b8009760 tx=0x7f60b8009060).ready entity=mon.2 client_cookie=b2c3987c43948e0c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:23.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.430+0000 7f60c137d700 1 -- 172.21.15.29:0/3976280028 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b800d4a0 con 0x7f60bc0664c0 2024-01-24T13:59:23.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.430+0000 7f60c137d700 1 -- 172.21.15.29:0/3976280028 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b800d600 con 0x7f60bc0664c0 2024-01-24T13:59:23.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.430+0000 7f60c137d700 1 -- 172.21.15.29:0/3976280028 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b800d980 con 0x7f60bc0664c0 2024-01-24T13:59:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.431+0000 7f60c45e3700 1 -- 172.21.15.29:0/3976280028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60bc0664c0 msgr2=0x7f60bc066940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.431+0000 7f60c45e3700 1 --2- 172.21.15.29:0/3976280028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60bc0664c0 0x7f60bc066940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f60b8009760 tx=0x7f60b8009060).stop 2024-01-24T13:59:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.431+0000 7f60c45e3700 1 -- 172.21.15.29:0/3976280028 shutdown_connections 2024-01-24T13:59:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.431+0000 7f60c45e3700 1 --2- 172.21.15.29:0/3976280028 >> v2:172.21.15.163:3300/0 conn(0x7f60bc0bdb30 0x7f60bc0bdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.431+0000 7f60c45e3700 1 --2- 172.21.15.29:0/3976280028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60bc0664c0 0x7f60bc066940 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.431+0000 7f60c45e3700 1 -- 172.21.15.29:0/3976280028 >> 172.21.15.29:0/3976280028 conn(0x7f60bc0c8bd0 msgr2=0x7f60bc0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.431+0000 7f60c45e3700 1 -- 172.21.15.29:0/3976280028 shutdown_connections 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.431+0000 7f60c45e3700 1 -- 172.21.15.29:0/3976280028 wait complete. 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.432+0000 7f60c45e3700 1 Processor -- start 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.432+0000 7f60c45e3700 1 -- start start 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c45e3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60bc0664c0 0x7f60bc14b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c45e3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60bc0bdb30 0x7f60bc0ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c45e3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60bc134ea0 0x7f60bc1512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c45e3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc1657f0 con 0x7f60bc134ea0 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c45e3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc166650 con 0x7f60bc0664c0 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c45e3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc14bac0 con 0x7f60bc0bdb30 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c2b80700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60bc134ea0 0x7f60bc1512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c2b80700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60bc134ea0 0x7f60bc1512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39424/0 (socket says 172.21.15.29:39424) 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c2b80700 1 -- 172.21.15.29:0/1443526599 learned_addr learned my addr 172.21.15.29:0/1443526599 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:23.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c1b7e700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60bc0bdb30 0x7f60bc0ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c237f700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60bc0664c0 0x7f60bc14b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c2b80700 1 -- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60bc0bdb30 msgr2=0x7f60bc0ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c2b80700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60bc0bdb30 0x7f60bc0ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c2b80700 1 -- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60bc0664c0 msgr2=0x7f60bc14b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c2b80700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60bc0664c0 0x7f60bc14b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.433+0000 7f60c2b80700 1 -- 172.21.15.29:0/1443526599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b4009a20 con 0x7f60bc134ea0 2024-01-24T13:59:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.434+0000 7f60c2b80700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60bc134ea0 0x7f60bc1512b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f60b4009f50 tx=0x7f60b400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.434+0000 7f60b27fc700 1 -- 172.21.15.29:0/1443526599 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b400cb50 con 0x7f60bc134ea0 2024-01-24T13:59:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.434+0000 7f60b27fc700 1 -- 172.21.15.29:0/1443526599 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b400f040 con 0x7f60bc134ea0 2024-01-24T13:59:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.434+0000 7f60c45e3700 1 -- 172.21.15.29:0/1443526599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b80092a0 con 0x7f60bc134ea0 2024-01-24T13:59:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.434+0000 7f60b27fc700 1 -- 172.21.15.29:0/1443526599 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b400cec0 con 0x7f60bc134ea0 2024-01-24T13:59:23.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.434+0000 7f60c45e3700 1 -- 172.21.15.29:0/1443526599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60bc165d20 con 0x7f60bc134ea0 2024-01-24T13:59:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.435+0000 7f60c45e3700 1 -- 172.21.15.29:0/1443526599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60bc0475e0 con 0x7f60bc134ea0 2024-01-24T13:59:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.435+0000 7f60b27fc700 1 -- 172.21.15.29:0/1443526599 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f60b4013610 con 0x7f60bc134ea0 2024-01-24T13:59:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.436+0000 7f60b27fc700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f60a005f150 0x7f60a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.436+0000 7f60b27fc700 1 -- 172.21.15.29:0/1443526599 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5622+0+0 (secure 0 0 0) 0x7f60b407a860 con 0x7f60bc134ea0 2024-01-24T13:59:23.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.436+0000 7f60c237f700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f60a005f150 0x7f60a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.436+0000 7f60c237f700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f60a005f150 0x7f60a0061610 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f60ac005750 tx=0x7f60ac009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:23.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.440+0000 7f60b27fc700 1 -- 172.21.15.29:0/1443526599 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f60b404bd20 con 0x7f60bc134ea0 2024-01-24T13:59:23.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.618+0000 7f60c45e3700 1 -- 172.21.15.29:0/1443526599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f60bc067a90 con 0x7f60bc134ea0 2024-01-24T13:59:23.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.619+0000 7f60b27fc700 1 -- 172.21.15.29:0/1443526599 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v37) v1 ==== 110+0+10 (secure 0 0 0) 0x7f60b404f340 con 0x7f60bc134ea0 2024-01-24T13:59:23.619 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T13:59:23.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.621+0000 7f60c45e3700 1 -- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f60a005f150 msgr2=0x7f60a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.621+0000 7f60c45e3700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f60a005f150 0x7f60a0061610 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f60ac005750 tx=0x7f60ac009040).stop 2024-01-24T13:59:23.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.621+0000 7f60c45e3700 1 -- 172.21.15.29:0/1443526599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60bc134ea0 msgr2=0x7f60bc1512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.621+0000 7f60c45e3700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60bc134ea0 0x7f60bc1512b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f60b4009f50 tx=0x7f60b400b040).stop 2024-01-24T13:59:23.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.621+0000 7f60c45e3700 1 -- 172.21.15.29:0/1443526599 shutdown_connections 2024-01-24T13:59:23.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.621+0000 7f60c45e3700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f60a005f150 0x7f60a0061610 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.621+0000 7f60c45e3700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60bc134ea0 0x7f60bc1512b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.621+0000 7f60c45e3700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60bc0664c0 0x7f60bc14b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.621+0000 7f60c45e3700 1 --2- 172.21.15.29:0/1443526599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60bc0bdb30 0x7f60bc0ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.621+0000 7f60c45e3700 1 -- 172.21.15.29:0/1443526599 >> 172.21.15.29:0/1443526599 conn(0x7f60bc0c8bd0 msgr2=0x7f60bc154410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:23.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.621+0000 7f60c45e3700 1 -- 172.21.15.29:0/1443526599 shutdown_connections 2024-01-24T13:59:23.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.621+0000 7f60c45e3700 1 -- 172.21.15.29:0/1443526599 wait complete. 2024-01-24T13:59:23.635 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T13:59:23.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.767+0000 7f4ebb989700 1 Processor -- start 2024-01-24T13:59:23.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.768+0000 7f4ebb989700 1 -- start start 2024-01-24T13:59:23.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4ebb989700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4eb413a5e0 0x7f4eb40639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4ebb989700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4eb4066bd0 0x7f4eb40bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4ebb989700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb40679f0 con 0x7f4eb4063eb0 2024-01-24T13:59:23.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4ebb989700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb4067cd0 con 0x7f4eb4066bd0 2024-01-24T13:59:23.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4ebb989700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb4067fb0 con 0x7f4eb413a5e0 2024-01-24T13:59:23.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4eb8f24700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4eb4063eb0 0x7f4eb40664c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54022/0 (socket says 172.21.15.29:54022) 2024-01-24T13:59:23.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4eb8f24700 1 -- 172.21.15.29:0/241501466 learned_addr learned my addr 172.21.15.29:0/241501466 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:23.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4eb9725700 1 --2- 172.21.15.29:0/241501466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4eb413a5e0 0x7f4eb40639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4eb9f26700 1 --2- 172.21.15.29:0/241501466 >> v2:172.21.15.163:3300/0 conn(0x7f4eb4066bd0 0x7f4eb40bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4eabfff700 1 -- 172.21.15.29:0/241501466 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831320179 0 0) 0x7f4eb40679f0 con 0x7f4eb4063eb0 2024-01-24T13:59:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4eabfff700 1 -- 172.21.15.29:0/241501466 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea00033f0 con 0x7f4eb4063eb0 2024-01-24T13:59:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4eb9725700 1 -- 172.21.15.29:0/241501466 >> v2:172.21.15.163:3300/0 conn(0x7f4eb4066bd0 msgr2=0x7f4eb40bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4eb9725700 1 --2- 172.21.15.29:0/241501466 >> v2:172.21.15.163:3300/0 conn(0x7f4eb4066bd0 0x7f4eb40bdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4eb9725700 1 -- 172.21.15.29:0/241501466 >> v1:172.21.15.29:6789/0 conn(0x7f4eb4063eb0 legacy=0x7f4eb40664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4eabfff700 1 -- 172.21.15.29:0/241501466 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1755830986 0 0) 0x7f4ea00033f0 con 0x7f4eb4063eb0 2024-01-24T13:59:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.769+0000 7f4eb9725700 1 -- 172.21.15.29:0/241501466 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb4068290 con 0x7f4eb413a5e0 2024-01-24T13:59:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.770+0000 7f4eb9725700 1 --2- 172.21.15.29:0/241501466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4eb413a5e0 0x7f4eb40639c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f4eb0009760 tx=0x7f4eb0009060).ready entity=mon.2 client_cookie=19b2b298df0c473d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.770+0000 7f4eabfff700 1 -- 172.21.15.29:0/241501466 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4001470 con 0x7f4eb413a5e0 2024-01-24T13:59:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.770+0000 7f4eabfff700 1 -- 172.21.15.29:0/241501466 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb0011400 con 0x7f4eb413a5e0 2024-01-24T13:59:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.770+0000 7f4eabfff700 1 -- 172.21.15.29:0/241501466 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0011780 con 0x7f4eb413a5e0 2024-01-24T13:59:23.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.770+0000 7f4ebb989700 1 -- 172.21.15.29:0/241501466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4eb413a5e0 msgr2=0x7f4eb40639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.770+0000 7f4ebb989700 1 --2- 172.21.15.29:0/241501466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4eb413a5e0 0x7f4eb40639c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f4eb0009760 tx=0x7f4eb0009060).stop 2024-01-24T13:59:23.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.770+0000 7f4ebb989700 1 -- 172.21.15.29:0/241501466 shutdown_connections 2024-01-24T13:59:23.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.770+0000 7f4ebb989700 1 --2- 172.21.15.29:0/241501466 >> v2:172.21.15.163:3300/0 conn(0x7f4eb4066bd0 0x7f4eb40bdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.770+0000 7f4ebb989700 1 --2- 172.21.15.29:0/241501466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4eb413a5e0 0x7f4eb40639c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.770+0000 7f4ebb989700 1 -- 172.21.15.29:0/241501466 >> 172.21.15.29:0/241501466 conn(0x7f4eb40c8bd0 msgr2=0x7f4eb40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:23.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.770+0000 7f4ebb989700 1 -- 172.21.15.29:0/241501466 shutdown_connections 2024-01-24T13:59:23.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.771+0000 7f4ebb989700 1 -- 172.21.15.29:0/241501466 wait complete. 2024-01-24T13:59:23.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.772+0000 7f4ebb989700 1 Processor -- start 2024-01-24T13:59:23.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.772+0000 7f4ebb989700 1 -- start start 2024-01-24T13:59:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4ebb989700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4eb4063eb0 0x7f4eb4137d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4ebb989700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4eb4066bd0 0x7f4eb41561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4ebb989700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eb413a5e0 0x7f4eb40a8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4ebb989700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb40ad540 con 0x7f4eb413a5e0 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4ebb989700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb4138270 con 0x7f4eb4063eb0 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4ebb989700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb40ac930 con 0x7f4eb4066bd0 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4eb9f26700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eb413a5e0 0x7f4eb40a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4eb9f26700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eb413a5e0 0x7f4eb40a8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39436/0 (socket says 172.21.15.29:39436) 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4eb9f26700 1 -- 172.21.15.29:0/1012430400 learned_addr learned my addr 172.21.15.29:0/1012430400 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4eb8f24700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4eb4066bd0 0x7f4eb41561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4eb9725700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4eb4063eb0 0x7f4eb4137d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4eb9f26700 1 -- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4eb4066bd0 msgr2=0x7f4eb41561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4eb9f26700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4eb4066bd0 0x7f4eb41561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4eb9f26700 1 -- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4eb4063eb0 msgr2=0x7f4eb4137d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4eb9f26700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4eb4063eb0 0x7f4eb4137d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4eb9f26700 1 -- 172.21.15.29:0/1012430400 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb00092a0 con 0x7f4eb413a5e0 2024-01-24T13:59:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4eb9f26700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eb413a5e0 0x7f4eb40a8400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f4eac009f20 tx=0x7f4eac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.773+0000 7f4ea97fa700 1 -- 172.21.15.29:0/1012430400 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00cc00 con 0x7f4eb413a5e0 2024-01-24T13:59:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.774+0000 7f4ea97fa700 1 -- 172.21.15.29:0/1012430400 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eac00f040 con 0x7f4eb413a5e0 2024-01-24T13:59:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.774+0000 7f4ebb989700 1 -- 172.21.15.29:0/1012430400 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb40acc10 con 0x7f4eb413a5e0 2024-01-24T13:59:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.774+0000 7f4ea97fa700 1 -- 172.21.15.29:0/1012430400 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac013400 con 0x7f4eb413a5e0 2024-01-24T13:59:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.774+0000 7f4ebb989700 1 -- 172.21.15.29:0/1012430400 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb40ad1d0 con 0x7f4eb413a5e0 2024-01-24T13:59:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.775+0000 7f4ebb989700 1 -- 172.21.15.29:0/1012430400 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb40ad6f0 con 0x7f4eb413a5e0 2024-01-24T13:59:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.775+0000 7f4ea97fa700 1 -- 172.21.15.29:0/1012430400 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4eac0137b0 con 0x7f4eb413a5e0 2024-01-24T13:59:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.775+0000 7f4ea97fa700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4e9805f120 0x7f4e980615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:23.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.776+0000 7f4ea97fa700 1 -- 172.21.15.29:0/1012430400 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5622+0+0 (secure 0 0 0) 0x7f4eac07a880 con 0x7f4eb413a5e0 2024-01-24T13:59:23.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.776+0000 7f4eb9725700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4e9805f120 0x7f4e980615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:23.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.776+0000 7f4eb9725700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4e9805f120 0x7f4e980615e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f4eb0006be0 tx=0x7f4eb0016000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:23.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.780+0000 7f4ea97fa700 1 -- 172.21.15.29:0/1012430400 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4eb40ad6f0 con 0x7f4eb413a5e0 2024-01-24T13:59:23.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.965+0000 7f4ebb989700 1 -- 172.21.15.29:0/1012430400 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4eb40475e0 con 0x7f4eb413a5e0 2024-01-24T13:59:23.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.965+0000 7f4ea97fa700 1 -- 172.21.15.29:0/1012430400 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4eac04bd40 con 0x7f4eb413a5e0 2024-01-24T13:59:23.966 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T13:59:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.968+0000 7f4ebb989700 1 -- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4e9805f120 msgr2=0x7f4e980615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.968+0000 7f4ebb989700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4e9805f120 0x7f4e980615e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f4eb0006be0 tx=0x7f4eb0016000).stop 2024-01-24T13:59:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.968+0000 7f4ebb989700 1 -- 172.21.15.29:0/1012430400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eb413a5e0 msgr2=0x7f4eb40a8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.968+0000 7f4ebb989700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eb413a5e0 0x7f4eb40a8400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f4eac009f20 tx=0x7f4eac00b040).stop 2024-01-24T13:59:23.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.968+0000 7f4ebb989700 1 -- 172.21.15.29:0/1012430400 shutdown_connections 2024-01-24T13:59:23.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.968+0000 7f4ebb989700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4e9805f120 0x7f4e980615e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.968+0000 7f4ebb989700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4eb413a5e0 0x7f4eb40a8400 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.968+0000 7f4ebb989700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4eb4063eb0 0x7f4eb4137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.968+0000 7f4ebb989700 1 --2- 172.21.15.29:0/1012430400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4eb4066bd0 0x7f4eb41561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:23.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.968+0000 7f4ebb989700 1 -- 172.21.15.29:0/1012430400 >> 172.21.15.29:0/1012430400 conn(0x7f4eb40c8bd0 msgr2=0x7f4eb40c7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:23.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.968+0000 7f4ebb989700 1 -- 172.21.15.29:0/1012430400 shutdown_connections 2024-01-24T13:59:23.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:23.968+0000 7f4ebb989700 1 -- 172.21.15.29:0/1012430400 wait complete. 2024-01-24T13:59:23.981 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-24T13:59:24.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.110+0000 7ff13a8d1700 1 Processor -- start 2024-01-24T13:59:24.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.111+0000 7ff13a8d1700 1 -- start start 2024-01-24T13:59:24.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.111+0000 7ff13a8d1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff13414eb80 0x7ff134066210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:24.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.111+0000 7ff13a8d1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff134151b10 0x7ff1340c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:24.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.111+0000 7ff13a8d1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13406a5a0 con 0x7ff134069240 2024-01-24T13:59:24.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.111+0000 7ff13a8d1700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1340696c0 con 0x7ff134151b10 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.111+0000 7ff13a8d1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff134069990 con 0x7ff13414eb80 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.112+0000 7ff138e6e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff134151b10 0x7ff1340c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.112+0000 7ff138e6e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff134151b10 0x7ff1340c30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39450/0 (socket says 172.21.15.29:39450) 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.112+0000 7ff138e6e700 1 -- 172.21.15.29:0/1376436348 learned_addr learned my addr 172.21.15.29:0/1376436348 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.112+0000 7ff133fff700 1 --2- 172.21.15.29:0/1376436348 >> v2:172.21.15.163:3300/0 conn(0x7ff13414eb80 0x7ff134066210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.112+0000 7ff138e6e700 1 -- 172.21.15.29:0/1376436348 >> v2:172.21.15.163:3300/0 conn(0x7ff13414eb80 msgr2=0x7ff134066210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.112+0000 7ff138e6e700 1 --2- 172.21.15.29:0/1376436348 >> v2:172.21.15.163:3300/0 conn(0x7ff13414eb80 0x7ff134066210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.112+0000 7ff138e6e700 1 -- 172.21.15.29:0/1376436348 >> v1:172.21.15.163:6789/0 conn(0x7ff134069240 legacy=0x7ff134151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.112+0000 7ff138e6e700 1 -- 172.21.15.29:0/1376436348 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff134069c70 con 0x7ff134151b10 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.112+0000 7ff138e6e700 1 --2- 172.21.15.29:0/1376436348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff134151b10 0x7ff1340c30d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7ff13406af90 tx=0x7ff12c00ba50).ready entity=mon.0 client_cookie=b79434f967f5c61e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.112+0000 7ff132ffd700 1 -- 172.21.15.29:0/1376436348 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c010070 con 0x7ff134151b10 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.112+0000 7ff132ffd700 1 -- 172.21.15.29:0/1376436348 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c00dde0 con 0x7ff134151b10 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.112+0000 7ff132ffd700 1 -- 172.21.15.29:0/1376436348 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00b2e0 con 0x7ff134151b10 2024-01-24T13:59:24.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.113+0000 7ff13a8d1700 1 -- 172.21.15.29:0/1376436348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff134151b10 msgr2=0x7ff1340c30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.113+0000 7ff13a8d1700 1 --2- 172.21.15.29:0/1376436348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff134151b10 0x7ff1340c30d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7ff13406af90 tx=0x7ff12c00ba50).stop 2024-01-24T13:59:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.113+0000 7ff13a8d1700 1 -- 172.21.15.29:0/1376436348 shutdown_connections 2024-01-24T13:59:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.113+0000 7ff13a8d1700 1 --2- 172.21.15.29:0/1376436348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff134151b10 0x7ff1340c30d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.113+0000 7ff13a8d1700 1 --2- 172.21.15.29:0/1376436348 >> v2:172.21.15.163:3300/0 conn(0x7ff13414eb80 0x7ff134066210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.113+0000 7ff13a8d1700 1 -- 172.21.15.29:0/1376436348 >> 172.21.15.29:0/1376436348 conn(0x7ff1340c85c0 msgr2=0x7ff1340c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.113+0000 7ff13a8d1700 1 -- 172.21.15.29:0/1376436348 shutdown_connections 2024-01-24T13:59:24.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.113+0000 7ff13a8d1700 1 -- 172.21.15.29:0/1376436348 wait complete. 2024-01-24T13:59:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.114+0000 7ff13a8d1700 1 Processor -- start 2024-01-24T13:59:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.115+0000 7ff13a8d1700 1 -- start start 2024-01-24T13:59:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.115+0000 7ff13a8d1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff134069240 0x7ff134147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.115+0000 7ff13a8d1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff13414eb80 0x7ff1340a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.115+0000 7ff13a8d1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff134151b10 0x7ff1340a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.115+0000 7ff13a8d1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340a7230 con 0x7ff134151b10 2024-01-24T13:59:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.115+0000 7ff13a8d1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340a4f70 con 0x7ff134069240 2024-01-24T13:59:24.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.115+0000 7ff13a8d1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340a8f10 con 0x7ff13414eb80 2024-01-24T13:59:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.115+0000 7ff138e6e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff134151b10 0x7ff1340a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff138e6e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff134151b10 0x7ff1340a6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39460/0 (socket says 172.21.15.29:39460) 2024-01-24T13:59:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff138e6e700 1 -- 172.21.15.29:0/930866018 learned_addr learned my addr 172.21.15.29:0/930866018 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff1337fe700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff13414eb80 0x7ff1340a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff133fff700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff134069240 0x7ff134147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff138e6e700 1 -- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff13414eb80 msgr2=0x7ff1340a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff138e6e700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff13414eb80 0x7ff1340a6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff138e6e700 1 -- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff134069240 msgr2=0x7ff134147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff138e6e700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff134069240 0x7ff134147a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff138e6e700 1 -- 172.21.15.29:0/930866018 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c00bcf0 con 0x7ff134151b10 2024-01-24T13:59:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff138e6e700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff134151b10 0x7ff1340a6d40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7ff12c00ba90 tx=0x7ff134064bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:24.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff11bfff700 1 -- 172.21.15.29:0/930866018 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00b2e0 con 0x7ff134151b10 2024-01-24T13:59:24.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff11bfff700 1 -- 172.21.15.29:0/930866018 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c015420 con 0x7ff134151b10 2024-01-24T13:59:24.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff13a8d1700 1 -- 172.21.15.29:0/930866018 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff134165260 con 0x7ff134151b10 2024-01-24T13:59:24.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff11bfff700 1 -- 172.21.15.29:0/930866018 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0157c0 con 0x7ff134151b10 2024-01-24T13:59:24.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.116+0000 7ff13a8d1700 1 -- 172.21.15.29:0/930866018 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1340a4c00 con 0x7ff134151b10 2024-01-24T13:59:24.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.117+0000 7ff13a8d1700 1 -- 172.21.15.29:0/930866018 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1340475e0 con 0x7ff134151b10 2024-01-24T13:59:24.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.118+0000 7ff11bfff700 1 -- 172.21.15.29:0/930866018 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff12c003040 con 0x7ff134151b10 2024-01-24T13:59:24.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.118+0000 7ff11bfff700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff12005f150 0x7ff120061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:24.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.118+0000 7ff11bfff700 1 -- 172.21.15.29:0/930866018 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5622+0+0 (secure 0 0 0) 0x7ff12c07bdc0 con 0x7ff134151b10 2024-01-24T13:59:24.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.118+0000 7ff133fff700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff12005f150 0x7ff120061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:24.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.119+0000 7ff133fff700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff12005f150 0x7ff120061610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7ff124009270 tx=0x7ff124008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:24.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.123+0000 7ff11bfff700 1 -- 172.21.15.29:0/930866018 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff12c04d280 con 0x7ff134151b10 2024-01-24T13:59:24.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.303+0000 7ff13a8d1700 1 -- 172.21.15.29:0/930866018 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff1340c53e0 con 0x7ff134151b10 2024-01-24T13:59:24.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.303+0000 7ff11bfff700 1 -- 172.21.15.29:0/930866018 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) v1 ==== 84+0+143 (secure 0 0 0) 0x7ff12c0508a0 con 0x7ff134151b10 2024-01-24T13:59:24.304 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:24.304 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T13:59:24.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:24.304 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-01-24T13:59:24.304 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "device_health_metrics" 2024-01-24T13:59:24.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T13:59:24.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:24.305 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 2, 2024-01-24T13:59:24.305 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "rbd" 2024-01-24T13:59:24.305 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:24.305 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T13:59:24.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.305+0000 7ff13a8d1700 1 -- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff12005f150 msgr2=0x7ff120061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:24.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.305+0000 7ff13a8d1700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff12005f150 0x7ff120061610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7ff124009270 tx=0x7ff124008040).stop 2024-01-24T13:59:24.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.305+0000 7ff13a8d1700 1 -- 172.21.15.29:0/930866018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff134151b10 msgr2=0x7ff1340a6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:24.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.305+0000 7ff13a8d1700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff134151b10 0x7ff1340a6d40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7ff12c00ba90 tx=0x7ff134064bf0).stop 2024-01-24T13:59:24.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.305+0000 7ff13a8d1700 1 -- 172.21.15.29:0/930866018 shutdown_connections 2024-01-24T13:59:24.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.305+0000 7ff13a8d1700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff12005f150 0x7ff120061610 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.305+0000 7ff13a8d1700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff134151b10 0x7ff1340a6d40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.305+0000 7ff13a8d1700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff134069240 0x7ff134147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.305+0000 7ff13a8d1700 1 --2- 172.21.15.29:0/930866018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff13414eb80 0x7ff1340a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.305+0000 7ff13a8d1700 1 -- 172.21.15.29:0/930866018 >> 172.21.15.29:0/930866018 conn(0x7ff1340c85c0 msgr2=0x7ff1340627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.306+0000 7ff13a8d1700 1 -- 172.21.15.29:0/930866018 shutdown_connections 2024-01-24T13:59:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.306+0000 7ff13a8d1700 1 -- 172.21.15.29:0/930866018 wait complete. 2024-01-24T13:59:24.319 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-24T13:59:24.319 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2024-01-24T13:59:24.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.454+0000 7f3f30dda700 1 Processor -- start 2024-01-24T13:59:24.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.454+0000 7f3f30dda700 1 -- start start 2024-01-24T13:59:24.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f30dda700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3f2c0c3660 0x7f3f2c14b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:24.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f30dda700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3f2c0c4b00 0x7f3f2c154a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:24.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f30dda700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c1557e0 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f30dda700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2c0c28a0 con 0x7f3f2c0c3660 2024-01-24T13:59:24.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f30dda700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2c0c2b80 con 0x7f3f2c0c4b00 2024-01-24T13:59:24.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f29d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3f2c0c3660 0x7f3f2c14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:24.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f29d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3f2c0c3660 0x7f3f2c14b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39472/0 (socket says 172.21.15.29:39472) 2024-01-24T13:59:24.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f29d9b700 1 -- 172.21.15.29:0/3658560 learned_addr learned my addr 172.21.15.29:0/3658560 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:24.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f2ad9d700 1 --2- 172.21.15.29:0/3658560 >> v2:172.21.15.163:3300/0 conn(0x7f3f2c0c4b00 0x7f3f2c154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f29d9b700 1 -- 172.21.15.29:0/3658560 >> v2:172.21.15.163:3300/0 conn(0x7f3f2c0c4b00 msgr2=0x7f3f2c154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f29d9b700 1 --2- 172.21.15.29:0/3658560 >> v2:172.21.15.163:3300/0 conn(0x7f3f2c0c4b00 0x7f3f2c154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f29d9b700 1 -- 172.21.15.29:0/3658560 >> v1:172.21.15.163:6789/0 conn(0x7f3f2c14b0a0 legacy=0x7f3f2c0c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.455+0000 7f3f29d9b700 1 -- 172.21.15.29:0/3658560 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c0c2e60 con 0x7f3f2c0c3660 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.456+0000 7f3f29d9b700 1 --2- 172.21.15.29:0/3658560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c0c3660 0x7f3f2c14b480 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f3f2c06a370 tx=0x7f3f200090f0).ready entity=mon.0 client_cookie=f80473d2789202e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.456+0000 7f3f2959a700 1 -- 172.21.15.29:0/3658560 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2000b600 con 0x7f3f2c0c3660 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.456+0000 7f3f2959a700 1 -- 172.21.15.29:0/3658560 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2000b760 con 0x7f3f2c0c3660 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.456+0000 7f3f2959a700 1 -- 172.21.15.29:0/3658560 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2000bae0 con 0x7f3f2c0c3660 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.456+0000 7f3f30dda700 1 -- 172.21.15.29:0/3658560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c0c3660 msgr2=0x7f3f2c14b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.456+0000 7f3f30dda700 1 --2- 172.21.15.29:0/3658560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c0c3660 0x7f3f2c14b480 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f3f2c06a370 tx=0x7f3f200090f0).stop 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.456+0000 7f3f30dda700 1 -- 172.21.15.29:0/3658560 shutdown_connections 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.456+0000 7f3f30dda700 1 --2- 172.21.15.29:0/3658560 >> v2:172.21.15.163:3300/0 conn(0x7f3f2c0c4b00 0x7f3f2c154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.456+0000 7f3f30dda700 1 --2- 172.21.15.29:0/3658560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c0c3660 0x7f3f2c14b480 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.456+0000 7f3f30dda700 1 -- 172.21.15.29:0/3658560 >> 172.21.15.29:0/3658560 conn(0x7f3f2c067b20 msgr2=0x7f3f2c0638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:24.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.457+0000 7f3f30dda700 1 -- 172.21.15.29:0/3658560 shutdown_connections 2024-01-24T13:59:24.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.457+0000 7f3f30dda700 1 -- 172.21.15.29:0/3658560 wait complete. 2024-01-24T13:59:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.458+0000 7f3f30dda700 1 Processor -- start 2024-01-24T13:59:24.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.458+0000 7f3f30dda700 1 -- start start 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f30dda700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f2c0c3660 0x7f3f2c153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f30dda700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f2c0c4b00 0x7f3f2c1651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f30dda700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c14b0a0 0x7f3f2c0a71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f30dda700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c1541c0 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f30dda700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c165690 con 0x7f3f2c0c3660 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f30dda700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c0a76c0 con 0x7f3f2c0c4b00 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f2ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c14b0a0 0x7f3f2c0a71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f2ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c14b0a0 0x7f3f2c0a71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39488/0 (socket says 172.21.15.29:39488) 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f2ad9d700 1 -- 172.21.15.29:0/2320681449 learned_addr learned my addr 172.21.15.29:0/2320681449 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f29d9b700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f2c0c4b00 0x7f3f2c1651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f2a59c700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f2c0c3660 0x7f3f2c153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f2ad9d700 1 -- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f2c0c4b00 msgr2=0x7f3f2c1651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f2ad9d700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f2c0c4b00 0x7f3f2c1651a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f2ad9d700 1 -- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f2c0c3660 msgr2=0x7f3f2c153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f2ad9d700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f2c0c3660 0x7f3f2c153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f2ad9d700 1 -- 172.21.15.29:0/2320681449 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f20009370 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f29d9b700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f2c0c4b00 0x7f3f2c1651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f2ad9d700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c14b0a0 0x7f3f2c0a71d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f3f24009fd0 tx=0x7f3f24009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f1a7fc700 1 -- 172.21.15.29:0/2320681449 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400eca0 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f1a7fc700 1 -- 172.21.15.29:0/2320681449 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2400ee00 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.459+0000 7f3f30dda700 1 -- 172.21.15.29:0/2320681449 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2c1407e0 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.460+0000 7f3f1a7fc700 1 -- 172.21.15.29:0/2320681449 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24014610 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.460+0000 7f3f30dda700 1 -- 172.21.15.29:0/2320681449 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2c0a85c0 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.461+0000 7f3f30dda700 1 -- 172.21.15.29:0/2320681449 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2c0475e0 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.461+0000 7f3f1a7fc700 1 -- 172.21.15.29:0/2320681449 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3f2401d3b0 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.461+0000 7f3f1a7fc700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f0c05f150 0x7f3f0c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.461+0000 7f3f1a7fc700 1 -- 172.21.15.29:0/2320681449 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5622+0+0 (secure 0 0 0) 0x7f3f2407a320 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.461+0000 7f3f2a59c700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f0c05f150 0x7f3f0c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:24.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.462+0000 7f3f2a59c700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f0c05f150 0x7f3f0c061610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f3f1c003860 tx=0x7f3f1c007d40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:24.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.466+0000 7f3f1a7fc700 1 -- 172.21.15.29:0/2320681449 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3f2404b780 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.647+0000 7f3f30dda700 1 -- 172.21.15.29:0/2320681449 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f3f2c1407e0 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.944+0000 7f3f1a7fc700 1 -- 172.21.15.29:0/2320681449 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v38) v1 ==== 152+0+0 (secure 0 0 0) 0x7f3f2c1407e0 con 0x7f3f2c14b0a0 2024-01-24T13:59:24.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.949+0000 7f3f30dda700 1 -- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f0c05f150 msgr2=0x7f3f0c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:24.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.949+0000 7f3f30dda700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f0c05f150 0x7f3f0c061610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f3f1c003860 tx=0x7f3f1c007d40).stop 2024-01-24T13:59:24.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.949+0000 7f3f30dda700 1 -- 172.21.15.29:0/2320681449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c14b0a0 msgr2=0x7f3f2c0a71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:24.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.949+0000 7f3f30dda700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c14b0a0 0x7f3f2c0a71d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f3f24009fd0 tx=0x7f3f24009900).stop 2024-01-24T13:59:24.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.949+0000 7f3f30dda700 1 -- 172.21.15.29:0/2320681449 shutdown_connections 2024-01-24T13:59:24.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.949+0000 7f3f30dda700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f0c05f150 0x7f3f0c061610 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.949+0000 7f3f30dda700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c14b0a0 0x7f3f2c0a71d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.949+0000 7f3f30dda700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f2c0c3660 0x7f3f2c153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.949+0000 7f3f30dda700 1 --2- 172.21.15.29:0/2320681449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f2c0c4b00 0x7f3f2c1651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:24.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.949+0000 7f3f30dda700 1 -- 172.21.15.29:0/2320681449 >> 172.21.15.29:0/2320681449 conn(0x7f3f2c067b20 msgr2=0x7f3f2c065e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:24.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.950+0000 7f3f30dda700 1 -- 172.21.15.29:0/2320681449 shutdown_connections 2024-01-24T13:59:24.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:24.951+0000 7f3f30dda700 1 -- 172.21.15.29:0/2320681449 wait complete. 2024-01-24T13:59:24.954 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-01-24T13:59:24.971 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-24T13:59:25.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.113+0000 7f118cb19700 1 Processor -- start 2024-01-24T13:59:25.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.114+0000 7f118cb19700 1 -- start start 2024-01-24T13:59:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f118cb19700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1188154590 0x7f11880a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f118cb19700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f11880627b0 0x7f1188062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f118cb19700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1188063150 con 0x7f11881541b0 2024-01-24T13:59:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f118cb19700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11880bd270 con 0x7f11880627b0 2024-01-24T13:59:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f118cb19700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1188069bd0 con 0x7f1188154590 2024-01-24T13:59:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f1186d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f11880627b0 0x7f1188062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f1186d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f11880627b0 0x7f1188062c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39500/0 (socket says 172.21.15.29:39500) 2024-01-24T13:59:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f1186d9d700 1 -- 172.21.15.29:0/4028216375 learned_addr learned my addr 172.21.15.29:0/4028216375 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f1185d9b700 1 --2- 172.21.15.29:0/4028216375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1188154590 0x7f11880a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f1186d9d700 1 -- 172.21.15.29:0/4028216375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1188154590 msgr2=0x7f11880a71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:25.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f1186d9d700 1 --2- 172.21.15.29:0/4028216375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1188154590 0x7f11880a71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f1186d9d700 1 -- 172.21.15.29:0/4028216375 >> v1:172.21.15.163:6789/0 conn(0x7f11881541b0 legacy=0x7f1188069410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f1186d9d700 1 -- 172.21.15.29:0/4028216375 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1188069eb0 con 0x7f11880627b0 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.115+0000 7f1186d9d700 1 --2- 172.21.15.29:0/4028216375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11880627b0 0x7f1188062c60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f1180009f20 tx=0x7f1180009840).ready entity=mon.0 client_cookie=46172a011d31f5c8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.116+0000 7f118559a700 1 -- 172.21.15.29:0/4028216375 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000e070 con 0x7f11880627b0 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.116+0000 7f118559a700 1 -- 172.21.15.29:0/4028216375 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118000b9d0 con 0x7f11880627b0 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.116+0000 7f118559a700 1 -- 172.21.15.29:0/4028216375 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000bd50 con 0x7f11880627b0 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.116+0000 7f118cb19700 1 -- 172.21.15.29:0/4028216375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11880627b0 msgr2=0x7f1188062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.116+0000 7f118cb19700 1 --2- 172.21.15.29:0/4028216375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11880627b0 0x7f1188062c60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f1180009f20 tx=0x7f1180009840).stop 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.116+0000 7f118cb19700 1 -- 172.21.15.29:0/4028216375 shutdown_connections 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.116+0000 7f118cb19700 1 --2- 172.21.15.29:0/4028216375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11880627b0 0x7f1188062c60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.116+0000 7f118cb19700 1 --2- 172.21.15.29:0/4028216375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1188154590 0x7f11880a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.116+0000 7f118cb19700 1 -- 172.21.15.29:0/4028216375 >> 172.21.15.29:0/4028216375 conn(0x7f11880c7bc0 msgr2=0x7f11880c7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.116+0000 7f118cb19700 1 -- 172.21.15.29:0/4028216375 shutdown_connections 2024-01-24T13:59:25.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.116+0000 7f118cb19700 1 -- 172.21.15.29:0/4028216375 wait complete. 2024-01-24T13:59:25.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.118+0000 7f118cb19700 1 Processor -- start 2024-01-24T13:59:25.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.118+0000 7f118cb19700 1 -- start start 2024-01-24T13:59:25.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.118+0000 7f118cb19700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11880627b0 0x7f118814cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:25.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.118+0000 7f118cb19700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11881541b0 0x7f1188157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:25.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.118+0000 7f118cb19700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1188154590 0x7f11880bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:25.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.118+0000 7f118cb19700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1188156c00 con 0x7f11881541b0 2024-01-24T13:59:25.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.118+0000 7f118cb19700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1188155ff0 con 0x7f1188154590 2024-01-24T13:59:25.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.118+0000 7f118cb19700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11881562d0 con 0x7f11880627b0 2024-01-24T13:59:25.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f1185d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11881541b0 0x7f1188157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f1185d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11881541b0 0x7f1188157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39504/0 (socket says 172.21.15.29:39504) 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f1185d9b700 1 -- 172.21.15.29:0/1746975021 learned_addr learned my addr 172.21.15.29:0/1746975021 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f1186d9d700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1188154590 0x7f11880bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f118659c700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11880627b0 0x7f118814cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f1185d9b700 1 -- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11880627b0 msgr2=0x7f118814cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f1185d9b700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11880627b0 0x7f118814cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f1185d9b700 1 -- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1188154590 msgr2=0x7f11880bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f1185d9b700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1188154590 0x7f11880bfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f1185d9b700 1 -- 172.21.15.29:0/1746975021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1180009ae0 con 0x7f11881541b0 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f118659c700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11880627b0 0x7f118814cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f1185d9b700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11881541b0 0x7f1188157090 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f117800e900 tx=0x7f1178009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f117e7fc700 1 -- 172.21.15.29:0/1746975021 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117800c640 con 0x7f11881541b0 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f117e7fc700 1 -- 172.21.15.29:0/1746975021 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117800c7a0 con 0x7f11881541b0 2024-01-24T13:59:25.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f117e7fc700 1 -- 172.21.15.29:0/1746975021 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117800cb20 con 0x7f11881541b0 2024-01-24T13:59:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f118cb19700 1 -- 172.21.15.29:0/1746975021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11881565b0 con 0x7f11881541b0 2024-01-24T13:59:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.119+0000 7f118cb19700 1 -- 172.21.15.29:0/1746975021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1188155c30 con 0x7f11881541b0 2024-01-24T13:59:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.121+0000 7f118cb19700 1 -- 172.21.15.29:0/1746975021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1188156d70 con 0x7f11881541b0 2024-01-24T13:59:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.121+0000 7f117e7fc700 1 -- 172.21.15.29:0/1746975021 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f117801b400 con 0x7f11881541b0 2024-01-24T13:59:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.121+0000 7f117e7fc700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f116805f150 0x7f1168061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.121+0000 7f117e7fc700 1 -- 172.21.15.29:0/1746975021 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f117807a560 con 0x7f11881541b0 2024-01-24T13:59:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.121+0000 7f118659c700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f116805f150 0x7f1168061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:25.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.122+0000 7f118659c700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f116805f150 0x7f1168061610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f11700016d0 tx=0x7f1170009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:25.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.126+0000 7f117e7fc700 1 -- 172.21.15.29:0/1746975021 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f117804b850 con 0x7f11881541b0 2024-01-24T13:59:25.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.307+0000 7f118cb19700 1 -- 172.21.15.29:0/1746975021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f11880c2620 con 0x7f11881541b0 2024-01-24T13:59:25.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.948+0000 7f117e7fc700 1 -- 172.21.15.29:0/1746975021 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v39) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1178016020 con 0x7f11881541b0 2024-01-24T13:59:25.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.955+0000 7f118cb19700 1 -- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f116805f150 msgr2=0x7f1168061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:25.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.955+0000 7f118cb19700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f116805f150 0x7f1168061610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f11700016d0 tx=0x7f1170009040).stop 2024-01-24T13:59:25.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.955+0000 7f118cb19700 1 -- 172.21.15.29:0/1746975021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11881541b0 msgr2=0x7f1188157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:25.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.955+0000 7f118cb19700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11881541b0 0x7f1188157090 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f117800e900 tx=0x7f1178009060).stop 2024-01-24T13:59:25.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.957+0000 7f118cb19700 1 -- 172.21.15.29:0/1746975021 shutdown_connections 2024-01-24T13:59:25.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.957+0000 7f118cb19700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1188154590 0x7f11880bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:25.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.957+0000 7f118cb19700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11880627b0 0x7f118814cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:25.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.957+0000 7f118cb19700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f116805f150 0x7f1168061610 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:25.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.957+0000 7f118cb19700 1 --2- 172.21.15.29:0/1746975021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11881541b0 0x7f1188157090 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:25.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.957+0000 7f118cb19700 1 -- 172.21.15.29:0/1746975021 >> 172.21.15.29:0/1746975021 conn(0x7f11880c7bc0 msgr2=0x7f118814dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:25.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.958+0000 7f118cb19700 1 -- 172.21.15.29:0/1746975021 shutdown_connections 2024-01-24T13:59:25.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:25.958+0000 7f118cb19700 1 -- 172.21.15.29:0/1746975021 wait complete. 2024-01-24T13:59:25.968 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-01-24T13:59:25.990 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-24T13:59:26.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.125+0000 7f5c5736e700 1 Processor -- start 2024-01-24T13:59:26.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.125+0000 7f5c5736e700 1 -- start start 2024-01-24T13:59:26.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.126+0000 7f5c5736e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5c50134ea0 0x7f5c500c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.126+0000 7f5c5736e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c500c36c0 0x7f5c500664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.126+0000 7f5c5736e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c500669b0 0x7f5c501559f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.126+0000 7f5c5736e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c50066e60 con 0x7f5c50134ea0 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.126+0000 7f5c5736e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c500686a0 con 0x7f5c500c36c0 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.126+0000 7f5c5736e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c50068b30 con 0x7f5c500669b0 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.126+0000 7f5c5510a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5c50134ea0 0x7f5c500c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c5510a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5c50134ea0 0x7f5c500c31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39508/0 (socket says 172.21.15.29:39508) 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c5510a700 1 -- 172.21.15.29:0/709123910 learned_addr learned my addr 172.21.15.29:0/709123910 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c5590b700 1 --2- 172.21.15.29:0/709123910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c500669b0 0x7f5c501559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c54909700 1 --2- 172.21.15.29:0/709123910 >> v2:172.21.15.163:3300/0 conn(0x7f5c500c36c0 0x7f5c500664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c5510a700 1 -- 172.21.15.29:0/709123910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c500669b0 msgr2=0x7f5c501559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c5510a700 1 --2- 172.21.15.29:0/709123910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c500669b0 0x7f5c501559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c5510a700 1 -- 172.21.15.29:0/709123910 >> v2:172.21.15.163:3300/0 conn(0x7f5c500c36c0 msgr2=0x7f5c500664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c5510a700 1 --2- 172.21.15.29:0/709123910 >> v2:172.21.15.163:3300/0 conn(0x7f5c500c36c0 0x7f5c500664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c5510a700 1 -- 172.21.15.29:0/709123910 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c50067a90 con 0x7f5c50134ea0 2024-01-24T13:59:26.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c5510a700 1 --2- 172.21.15.29:0/709123910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c50134ea0 0x7f5c500c31d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f5c5006a270 tx=0x7f5c440090f0).ready entity=mon.0 client_cookie=aa76bb12880f46e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:26.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c43fff700 1 -- 172.21.15.29:0/709123910 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4400b600 con 0x7f5c50134ea0 2024-01-24T13:59:26.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c43fff700 1 -- 172.21.15.29:0/709123910 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4400b760 con 0x7f5c50134ea0 2024-01-24T13:59:26.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.127+0000 7f5c43fff700 1 -- 172.21.15.29:0/709123910 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4400bae0 con 0x7f5c50134ea0 2024-01-24T13:59:26.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.128+0000 7f5c5736e700 1 -- 172.21.15.29:0/709123910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c50134ea0 msgr2=0x7f5c500c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.128+0000 7f5c5736e700 1 --2- 172.21.15.29:0/709123910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c50134ea0 0x7f5c500c31d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f5c5006a270 tx=0x7f5c440090f0).stop 2024-01-24T13:59:26.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.128+0000 7f5c5736e700 1 -- 172.21.15.29:0/709123910 shutdown_connections 2024-01-24T13:59:26.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.128+0000 7f5c5736e700 1 --2- 172.21.15.29:0/709123910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c500669b0 0x7f5c501559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.128+0000 7f5c5736e700 1 --2- 172.21.15.29:0/709123910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c50134ea0 0x7f5c500c31d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.128+0000 7f5c5736e700 1 --2- 172.21.15.29:0/709123910 >> v2:172.21.15.163:3300/0 conn(0x7f5c500c36c0 0x7f5c500664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.128+0000 7f5c5736e700 1 -- 172.21.15.29:0/709123910 >> 172.21.15.29:0/709123910 conn(0x7f5c500c8bd0 msgr2=0x7f5c500c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:26.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.128+0000 7f5c5736e700 1 -- 172.21.15.29:0/709123910 shutdown_connections 2024-01-24T13:59:26.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.128+0000 7f5c5736e700 1 -- 172.21.15.29:0/709123910 wait complete. 2024-01-24T13:59:26.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.129+0000 7f5c5736e700 1 Processor -- start 2024-01-24T13:59:26.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.129+0000 7f5c5736e700 1 -- start start 2024-01-24T13:59:26.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.130+0000 7f5c5736e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c500669b0 0x7f5c50134400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.130+0000 7f5c5736e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c500c36c0 0x7f5c501531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.130+0000 7f5c5736e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c50134ea0 0x7f5c501536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.130+0000 7f5c5736e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c501380a0 con 0x7f5c50134ea0 2024-01-24T13:59:26.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.130+0000 7f5c5736e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c50135550 con 0x7f5c500c36c0 2024-01-24T13:59:26.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.130+0000 7f5c5736e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c50136090 con 0x7f5c500669b0 2024-01-24T13:59:26.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.130+0000 7f5c5590b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c50134ea0 0x7f5c501536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.130+0000 7f5c5590b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c50134ea0 0x7f5c501536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39510/0 (socket says 172.21.15.29:39510) 2024-01-24T13:59:26.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.130+0000 7f5c5590b700 1 -- 172.21.15.29:0/399310299 learned_addr learned my addr 172.21.15.29:0/399310299 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:26.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.130+0000 7f5c54909700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c500c36c0 0x7f5c501531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.130+0000 7f5c5510a700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c500669b0 0x7f5c50134400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.131+0000 7f5c5590b700 1 -- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c500669b0 msgr2=0x7f5c50134400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.131+0000 7f5c5590b700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c500669b0 0x7f5c50134400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.131+0000 7f5c5590b700 1 -- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c500c36c0 msgr2=0x7f5c501531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.131+0000 7f5c5590b700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c500c36c0 0x7f5c501531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.131+0000 7f5c5590b700 1 -- 172.21.15.29:0/399310299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44009370 con 0x7f5c50134ea0 2024-01-24T13:59:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.131+0000 7f5c5590b700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c50134ea0 0x7f5c501536d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f5c48007f20 tx=0x7f5c48009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.131+0000 7f5c417fa700 1 -- 172.21.15.29:0/399310299 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4800eb90 con 0x7f5c50134ea0 2024-01-24T13:59:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.131+0000 7f5c417fa700 1 -- 172.21.15.29:0/399310299 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4800ecf0 con 0x7f5c50134ea0 2024-01-24T13:59:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.131+0000 7f5c5736e700 1 -- 172.21.15.29:0/399310299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c50135760 con 0x7f5c50134ea0 2024-01-24T13:59:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.131+0000 7f5c417fa700 1 -- 172.21.15.29:0/399310299 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48014610 con 0x7f5c50134ea0 2024-01-24T13:59:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.131+0000 7f5c5736e700 1 -- 172.21.15.29:0/399310299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c50135af0 con 0x7f5c50134ea0 2024-01-24T13:59:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.132+0000 7f5c5736e700 1 -- 172.21.15.29:0/399310299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c500475e0 con 0x7f5c50134ea0 2024-01-24T13:59:26.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.133+0000 7f5c417fa700 1 -- 172.21.15.29:0/399310299 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5c48014980 con 0x7f5c50134ea0 2024-01-24T13:59:26.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.133+0000 7f5c417fa700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5c3405f150 0x7f5c34061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.133+0000 7f5c417fa700 1 -- 172.21.15.29:0/399310299 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6374+0+0 (secure 0 0 0) 0x7f5c4807a6e0 con 0x7f5c50134ea0 2024-01-24T13:59:26.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.133+0000 7f5c5510a700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5c3405f150 0x7f5c34061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.134+0000 7f5c5510a700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5c3405f150 0x7f5c34061610 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f5c44009130 tx=0x7f5c44017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:26.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.138+0000 7f5c417fa700 1 -- 172.21.15.29:0/399310299 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5c4804b8b0 con 0x7f5c50134ea0 2024-01-24T13:59:26.314 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T13:59:26.317 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T13:59:26.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.368+0000 7f5c5736e700 1 -- 172.21.15.29:0/399310299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5c501361d0 con 0x7f5c50134ea0 2024-01-24T13:59:26.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.374+0000 7f5c417fa700 1 -- 172.21.15.29:0/399310299 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5c4800ee60 con 0x7f5c50134ea0 2024-01-24T13:59:26.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.378+0000 7f5c5736e700 1 -- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5c3405f150 msgr2=0x7f5c34061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.378+0000 7f5c5736e700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5c3405f150 0x7f5c34061610 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f5c44009130 tx=0x7f5c44017000).stop 2024-01-24T13:59:26.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.378+0000 7f5c5736e700 1 -- 172.21.15.29:0/399310299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c50134ea0 msgr2=0x7f5c501536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.378+0000 7f5c5736e700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c50134ea0 0x7f5c501536d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f5c48007f20 tx=0x7f5c48009860).stop 2024-01-24T13:59:26.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.383+0000 7f5c5736e700 1 -- 172.21.15.29:0/399310299 shutdown_connections 2024-01-24T13:59:26.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.383+0000 7f5c5736e700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5c3405f150 0x7f5c34061610 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.383+0000 7f5c5736e700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c50134ea0 0x7f5c501536d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.383+0000 7f5c5736e700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c500669b0 0x7f5c50134400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.383+0000 7f5c5736e700 1 --2- 172.21.15.29:0/399310299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c500c36c0 0x7f5c501531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.383+0000 7f5c5736e700 1 -- 172.21.15.29:0/399310299 >> 172.21.15.29:0/399310299 conn(0x7f5c500c8bd0 msgr2=0x7f5c500c4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:26.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.385+0000 7f5c5736e700 1 -- 172.21.15.29:0/399310299 shutdown_connections 2024-01-24T13:59:26.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.385+0000 7f5c5736e700 1 -- 172.21.15.29:0/399310299 wait complete. 2024-01-24T13:59:26.404 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 6 and data pool 7 2024-01-24T13:59:26.417 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:59:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.589+0000 7fab80c74700 1 Processor -- start 2024-01-24T13:59:26.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.589+0000 7fab80c74700 1 -- start start 2024-01-24T13:59:26.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.590+0000 7fab80c74700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fab7c066c00 0x7fab7c0c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.590+0000 7fab80c74700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7c068360 con 0x7fab7c134cc0 2024-01-24T13:59:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.590+0000 7fab80c74700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7c0c3820 con 0x7fab7c1350a0 2024-01-24T13:59:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.590+0000 7fab80c74700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7c067a30 con 0x7fab7c066c00 2024-01-24T13:59:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.590+0000 7fab7a59c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fab7c134cc0 0x7fab7c0664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54038/0 (socket says 172.21.15.29:54038) 2024-01-24T13:59:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.590+0000 7fab7a59c700 1 -- 172.21.15.29:0/1819361924 learned_addr learned my addr 172.21.15.29:0/1819361924 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.590+0000 7fab79d9b700 1 --2- 172.21.15.29:0/1819361924 >> v2:172.21.15.163:3300/0 conn(0x7fab7c066c00 0x7fab7c0c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.590+0000 7fab7959a700 1 -- 172.21.15.29:0/1819361924 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238274375 0 0) 0x7fab7c068360 con 0x7fab7c134cc0 2024-01-24T13:59:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.590+0000 7fab7959a700 1 -- 172.21.15.29:0/1819361924 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab640033f0 con 0x7fab7c134cc0 2024-01-24T13:59:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.591+0000 7fab7959a700 1 -- 172.21.15.29:0/1819361924 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21010400 0 0) 0x7fab7c0c3820 con 0x7fab7c1350a0 2024-01-24T13:59:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.591+0000 7fab7959a700 1 -- 172.21.15.29:0/1819361924 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab7c068360 con 0x7fab7c1350a0 2024-01-24T13:59:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.591+0000 7fab79d9b700 1 -- 172.21.15.29:0/1819361924 >> v1:172.21.15.163:6789/0 conn(0x7fab7c1350a0 legacy=0x7fab7c15bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.591+0000 7fab79d9b700 1 -- 172.21.15.29:0/1819361924 >> v1:172.21.15.29:6789/0 conn(0x7fab7c134cc0 legacy=0x7fab7c0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.591+0000 7fab7959a700 1 -- 172.21.15.29:0/1819361924 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3526134872 0 0) 0x7fab640033f0 con 0x7fab7c134cc0 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.591+0000 7fab79d9b700 1 -- 172.21.15.29:0/1819361924 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c067d10 con 0x7fab7c066c00 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.591+0000 7fab79d9b700 1 --2- 172.21.15.29:0/1819361924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7c066c00 0x7fab7c0c31b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fab70009760 tx=0x7fab70009060).ready entity=mon.1 client_cookie=1db6a3198c41d3a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.591+0000 7fab7959a700 1 -- 172.21.15.29:0/1819361924 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c001470 con 0x7fab7c066c00 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.591+0000 7fab7959a700 1 -- 172.21.15.29:0/1819361924 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab70011400 con 0x7fab7c066c00 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.591+0000 7fab7959a700 1 -- 172.21.15.29:0/1819361924 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70011780 con 0x7fab7c066c00 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.592+0000 7fab80c74700 1 -- 172.21.15.29:0/1819361924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7c066c00 msgr2=0x7fab7c0c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.592+0000 7fab80c74700 1 --2- 172.21.15.29:0/1819361924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7c066c00 0x7fab7c0c31b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fab70009760 tx=0x7fab70009060).stop 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.592+0000 7fab80c74700 1 -- 172.21.15.29:0/1819361924 shutdown_connections 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.592+0000 7fab80c74700 1 --2- 172.21.15.29:0/1819361924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7c066c00 0x7fab7c0c31b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.592+0000 7fab80c74700 1 -- 172.21.15.29:0/1819361924 >> 172.21.15.29:0/1819361924 conn(0x7fab7c0c8bb0 msgr2=0x7fab7c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.592+0000 7fab80c74700 1 -- 172.21.15.29:0/1819361924 shutdown_connections 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.592+0000 7fab80c74700 1 -- 172.21.15.29:0/1819361924 wait complete. 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.593+0000 7fab80c74700 1 Processor -- start 2024-01-24T13:59:26.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.594+0000 7fab80c74700 1 -- start start 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab80c74700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7c066c00 0x7fab7c134670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab80c74700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab7c134cc0 0x7fab7c135b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab80c74700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7c1350a0 0x7fab7c165f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab80c74700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c134b60 con 0x7fab7c134cc0 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab80c74700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c136060 con 0x7fab7c066c00 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab80c74700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c0c1100 con 0x7fab7c1350a0 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab79d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab7c134cc0 0x7fab7c135b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab79d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab7c134cc0 0x7fab7c135b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39512/0 (socket says 172.21.15.29:39512) 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab79d9b700 1 -- 172.21.15.29:0/3889531945 learned_addr learned my addr 172.21.15.29:0/3889531945 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab7a59c700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7c066c00 0x7fab7c134670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab7ad9d700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7c1350a0 0x7fab7c165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab79d9b700 1 -- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7c1350a0 msgr2=0x7fab7c165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab79d9b700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7c1350a0 0x7fab7c165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab79d9b700 1 -- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7c066c00 msgr2=0x7fab7c134670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab79d9b700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7c066c00 0x7fab7c134670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab79d9b700 1 -- 172.21.15.29:0/3889531945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab700092a0 con 0x7fab7c134cc0 2024-01-24T13:59:26.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab79d9b700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab7c134cc0 0x7fab7c135b70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fab7c06a130 tx=0x7fab70016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:26.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab6a7fc700 1 -- 172.21.15.29:0/3889531945 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70010920 con 0x7fab7c134cc0 2024-01-24T13:59:26.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab6a7fc700 1 -- 172.21.15.29:0/3889531945 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab70010a80 con 0x7fab7c134cc0 2024-01-24T13:59:26.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab6a7fc700 1 -- 172.21.15.29:0/3889531945 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70010e20 con 0x7fab7c134cc0 2024-01-24T13:59:26.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.595+0000 7fab80c74700 1 -- 172.21.15.29:0/3889531945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7c0c16a0 con 0x7fab7c134cc0 2024-01-24T13:59:26.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.596+0000 7fab80c74700 1 -- 172.21.15.29:0/3889531945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7c166470 con 0x7fab7c134cc0 2024-01-24T13:59:26.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.596+0000 7fab80c74700 1 -- 172.21.15.29:0/3889531945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7c0c1830 con 0x7fab7c134cc0 2024-01-24T13:59:26.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.597+0000 7fab6a7fc700 1 -- 172.21.15.29:0/3889531945 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fab7001d640 con 0x7fab7c134cc0 2024-01-24T13:59:26.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.597+0000 7fab6a7fc700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab5c05f150 0x7fab5c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.597+0000 7fab6a7fc700 1 -- 172.21.15.29:0/3889531945 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6474+0+0 (secure 0 0 0) 0x7fab7007ba10 con 0x7fab7c134cc0 2024-01-24T13:59:26.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.598+0000 7fab7a59c700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab5c05f150 0x7fab5c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.598+0000 7fab7a59c700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab5c05f150 0x7fab5c061610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fab6c001660 tx=0x7fab6c001520).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:26.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.602+0000 7fab6a7fc700 1 -- 172.21.15.29:0/3889531945 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fab7004cb80 con 0x7fab7c134cc0 2024-01-24T13:59:26.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.785+0000 7fab80c74700 1 -- 172.21.15.29:0/3889531945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab7c0475e0 con 0x7fab7c134cc0 2024-01-24T13:59:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.786+0000 7fab6a7fc700 1 -- 172.21.15.29:0/3889531945 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17177 (secure 0 0 0) 0x7fab700501a0 con 0x7fab7c134cc0 2024-01-24T13:59:26.786 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:26.787 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":40,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:59:26.369828+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:59:24.684931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-24T13:59:25.345273+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:59:26.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.788+0000 7fab80c74700 1 -- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab5c05f150 msgr2=0x7fab5c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.788+0000 7fab80c74700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab5c05f150 0x7fab5c061610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fab6c001660 tx=0x7fab6c001520).stop 2024-01-24T13:59:26.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.788+0000 7fab80c74700 1 -- 172.21.15.29:0/3889531945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab7c134cc0 msgr2=0x7fab7c135b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.788+0000 7fab80c74700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab7c134cc0 0x7fab7c135b70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fab7c06a130 tx=0x7fab70016000).stop 2024-01-24T13:59:26.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.788+0000 7fab80c74700 1 -- 172.21.15.29:0/3889531945 shutdown_connections 2024-01-24T13:59:26.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.788+0000 7fab80c74700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7c1350a0 0x7fab7c165f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.788+0000 7fab80c74700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7c066c00 0x7fab7c134670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.788+0000 7fab80c74700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab5c05f150 0x7fab5c061610 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.788+0000 7fab80c74700 1 --2- 172.21.15.29:0/3889531945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab7c134cc0 0x7fab7c135b70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.788+0000 7fab80c74700 1 -- 172.21.15.29:0/3889531945 >> 172.21.15.29:0/3889531945 conn(0x7fab7c0c8bb0 msgr2=0x7fab7c0c6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:26.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.788+0000 7fab80c74700 1 -- 172.21.15.29:0/3889531945 shutdown_connections 2024-01-24T13:59:26.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.788+0000 7fab80c74700 1 -- 172.21.15.29:0/3889531945 wait complete. 2024-01-24T13:59:26.802 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:59:26.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.931+0000 7f7352c27700 1 Processor -- start 2024-01-24T13:59:26.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.931+0000 7f7352c27700 1 -- start start 2024-01-24T13:59:26.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f7352c27700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f734c0bd850 0x7f734c0bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f7352c27700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f734c0c1870 0x7f734c0c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f7352c27700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734c064040 con 0x7f734c0627b0 2024-01-24T13:59:26.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f7352c27700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734c0be120 con 0x7f734c0bd850 2024-01-24T13:59:26.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f7352c27700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734c0c21e0 con 0x7f734c0c1870 2024-01-24T13:59:26.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f73509c3700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f734c0bd850 0x7f734c0bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f73509c3700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f734c0bd850 0x7f734c0bdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39514/0 (socket says 172.21.15.29:39514) 2024-01-24T13:59:26.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f73509c3700 1 -- 172.21.15.29:0/62034455 learned_addr learned my addr 172.21.15.29:0/62034455 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:26.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f734bfff700 1 --2- 172.21.15.29:0/62034455 >> v2:172.21.15.163:3300/0 conn(0x7f734c0c1870 0x7f734c0c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f73509c3700 1 -- 172.21.15.29:0/62034455 >> v2:172.21.15.163:3300/0 conn(0x7f734c0c1870 msgr2=0x7f734c0c1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f73509c3700 1 --2- 172.21.15.29:0/62034455 >> v2:172.21.15.163:3300/0 conn(0x7f734c0c1870 0x7f734c0c1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f73509c3700 1 -- 172.21.15.29:0/62034455 >> v1:172.21.15.163:6789/0 conn(0x7f734c0627b0 legacy=0x7f734c062be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.932+0000 7f73509c3700 1 -- 172.21.15.29:0/62034455 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c063430 con 0x7f734c0bd850 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.933+0000 7f73509c3700 1 --2- 172.21.15.29:0/62034455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0bd850 0x7f734c0bdc30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f734c0c3530 tx=0x7f73400090f0).ready entity=mon.0 client_cookie=d8e4b34039d9411d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.933+0000 7f734b7fe700 1 -- 172.21.15.29:0/62034455 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734000b5a0 con 0x7f734c0bd850 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.933+0000 7f734b7fe700 1 -- 172.21.15.29:0/62034455 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734000b700 con 0x7f734c0bd850 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.933+0000 7f734b7fe700 1 -- 172.21.15.29:0/62034455 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734000ba80 con 0x7f734c0bd850 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.933+0000 7f7352c27700 1 -- 172.21.15.29:0/62034455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0bd850 msgr2=0x7f734c0bdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.933+0000 7f7352c27700 1 --2- 172.21.15.29:0/62034455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0bd850 0x7f734c0bdc30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f734c0c3530 tx=0x7f73400090f0).stop 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.933+0000 7f7352c27700 1 -- 172.21.15.29:0/62034455 shutdown_connections 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.933+0000 7f7352c27700 1 --2- 172.21.15.29:0/62034455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0bd850 0x7f734c0bdc30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.933+0000 7f7352c27700 1 --2- 172.21.15.29:0/62034455 >> v2:172.21.15.163:3300/0 conn(0x7f734c0c1870 0x7f734c0c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.933+0000 7f7352c27700 1 -- 172.21.15.29:0/62034455 >> 172.21.15.29:0/62034455 conn(0x7f734c064aa0 msgr2=0x7f734c064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:26.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.933+0000 7f7352c27700 1 -- 172.21.15.29:0/62034455 shutdown_connections 2024-01-24T13:59:26.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.933+0000 7f7352c27700 1 -- 172.21.15.29:0/62034455 wait complete. 2024-01-24T13:59:26.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.935+0000 7f7352c27700 1 Processor -- start 2024-01-24T13:59:26.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.935+0000 7f7352c27700 1 -- start start 2024-01-24T13:59:26.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f7352c27700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c0627b0 0x7f734c0aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f7352c27700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0bd850 0x7f734c0c1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f7352c27700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c0c1870 0x7f734c164210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f7352c27700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c157820 con 0x7f734c0bd850 2024-01-24T13:59:26.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f7352c27700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c138a30 con 0x7f734c0627b0 2024-01-24T13:59:26.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f7352c27700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c157260 con 0x7f734c0c1870 2024-01-24T13:59:26.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f734bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0bd850 0x7f734c0c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f734bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0bd850 0x7f734c0c1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39522/0 (socket says 172.21.15.29:39522) 2024-01-24T13:59:26.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f734bfff700 1 -- 172.21.15.29:0/1292490882 learned_addr learned my addr 172.21.15.29:0/1292490882 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f73511c4700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c0c1870 0x7f734c164210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f734bfff700 1 -- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c0c1870 msgr2=0x7f734c164210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f734bfff700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c0c1870 0x7f734c164210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f734bfff700 1 -- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c0627b0 msgr2=0x7f734c0aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f73509c3700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c0627b0 0x7f734c0aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f734bfff700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c0627b0 0x7f734c0aada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f734bfff700 1 -- 172.21.15.29:0/1292490882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7340009370 con 0x7f734c0bd850 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f73511c4700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c0c1870 0x7f734c164210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f734bfff700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0bd850 0x7f734c0c1040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f733800d890 tx=0x7f7338008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.936+0000 7f7348ff9700 1 -- 172.21.15.29:0/1292490882 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73380125f0 con 0x7f734c0bd850 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.937+0000 7f7348ff9700 1 -- 172.21.15.29:0/1292490882 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7338012750 con 0x7f734c0bd850 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.937+0000 7f7348ff9700 1 -- 172.21.15.29:0/1292490882 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338012ad0 con 0x7f734c0bd850 2024-01-24T13:59:26.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.937+0000 7f7352c27700 1 -- 172.21.15.29:0/1292490882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734c157540 con 0x7f734c0bd850 2024-01-24T13:59:26.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.937+0000 7f7352c27700 1 -- 172.21.15.29:0/1292490882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734c0ab680 con 0x7f734c0bd850 2024-01-24T13:59:26.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.938+0000 7f7352c27700 1 -- 172.21.15.29:0/1292490882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734c0475e0 con 0x7f734c0bd850 2024-01-24T13:59:26.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.939+0000 7f7348ff9700 1 -- 172.21.15.29:0/1292490882 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7338013480 con 0x7f734c0bd850 2024-01-24T13:59:26.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.939+0000 7f7348ff9700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f733005f150 0x7f7330061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:26.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.939+0000 7f7348ff9700 1 -- 172.21.15.29:0/1292490882 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f733807aaa0 con 0x7f734c0bd850 2024-01-24T13:59:26.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.939+0000 7f73509c3700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f733005f150 0x7f7330061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:26.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.940+0000 7f73509c3700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f733005f150 0x7f7330061610 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f734c0a8d70 tx=0x7f7340015000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:26.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:26.943+0000 7f7348ff9700 1 -- 172.21.15.29:0/1292490882 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f733804bc50 con 0x7f734c0bd850 2024-01-24T13:59:27.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.125+0000 7f7352c27700 1 -- 172.21.15.29:0/1292490882 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f734c05e2c0 con 0x7f734c0bd850 2024-01-24T13:59:27.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.126+0000 7f7348ff9700 1 -- 172.21.15.29:0/1292490882 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17177 (secure 0 0 0) 0x7f733804f270 con 0x7f734c0bd850 2024-01-24T13:59:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:27.127 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":40,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:59:26.369828+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:59:24.684931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-24T13:59:25.345273+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:59:27.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.127+0000 7f7352c27700 1 -- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f733005f150 msgr2=0x7f7330061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:27.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.127+0000 7f7352c27700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f733005f150 0x7f7330061610 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f734c0a8d70 tx=0x7f7340015000).stop 2024-01-24T13:59:27.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.127+0000 7f7352c27700 1 -- 172.21.15.29:0/1292490882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0bd850 msgr2=0x7f734c0c1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:27.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.127+0000 7f7352c27700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0bd850 0x7f734c0c1040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f733800d890 tx=0x7f7338008040).stop 2024-01-24T13:59:27.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.128+0000 7f7352c27700 1 -- 172.21.15.29:0/1292490882 shutdown_connections 2024-01-24T13:59:27.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.128+0000 7f7352c27700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c0c1870 0x7f734c164210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:27.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.128+0000 7f7352c27700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c0627b0 0x7f734c0aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:27.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.128+0000 7f7352c27700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f733005f150 0x7f7330061610 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:27.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.128+0000 7f7352c27700 1 --2- 172.21.15.29:0/1292490882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f734c0bd850 0x7f734c0c1040 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:27.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.128+0000 7f7352c27700 1 -- 172.21.15.29:0/1292490882 >> 172.21.15.29:0/1292490882 conn(0x7f734c064aa0 msgr2=0x7f734c0bd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:27.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.128+0000 7f7352c27700 1 -- 172.21.15.29:0/1292490882 shutdown_connections 2024-01-24T13:59:27.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.128+0000 7f7352c27700 1 -- 172.21.15.29:0/1292490882 wait complete. 2024-01-24T13:59:27.142 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-24T13:59:27.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.274+0000 7f88541d6700 1 Processor -- start 2024-01-24T13:59:27.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.274+0000 7f88541d6700 1 -- start start 2024-01-24T13:59:27.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.275+0000 7f88541d6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f884c157760 0x7f884c157c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:27.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.275+0000 7f88541d6700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c0627b0 con 0x7f884c157380 2024-01-24T13:59:27.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.275+0000 7f88541d6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c062d70 con 0x7f884c158110 2024-01-24T13:59:27.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.275+0000 7f88541d6700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884c1586e0 con 0x7f884c157760 2024-01-24T13:59:27.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.275+0000 7f8851771700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f884c157760 0x7f884c157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:27.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.275+0000 7f8851f72700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f884c157380 0x7f884c063200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54042/0 (socket says 172.21.15.29:54042) 2024-01-24T13:59:27.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.275+0000 7f8851f72700 1 -- 172.21.15.29:0/377735752 learned_addr learned my addr 172.21.15.29:0/377735752 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:27.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f8851771700 1 -- 172.21.15.29:0/377735752 >> v1:172.21.15.163:6789/0 conn(0x7f884c158110 legacy=0x7f884c064b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:27.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f8851771700 1 -- 172.21.15.29:0/377735752 >> v1:172.21.15.29:6789/0 conn(0x7f884c157380 legacy=0x7f884c063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:27.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f8851771700 1 -- 172.21.15.29:0/377735752 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c1589c0 con 0x7f884c157760 2024-01-24T13:59:27.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f8850f70700 1 -- 172.21.15.29:0/377735752 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233264842 0 0) 0x7f884c0627b0 con 0x7f884c157380 2024-01-24T13:59:27.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f8851771700 1 --2- 172.21.15.29:0/377735752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c157760 0x7f884c157c20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f8848009760 tx=0x7f8848009060).ready entity=mon.0 client_cookie=a299277e5f7d9865 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:27.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f8850f70700 1 -- 172.21.15.29:0/377735752 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848009120 con 0x7f884c157760 2024-01-24T13:59:27.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f8850f70700 1 -- 172.21.15.29:0/377735752 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884800b580 con 0x7f884c157760 2024-01-24T13:59:27.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f8850f70700 1 -- 172.21.15.29:0/377735752 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884800b900 con 0x7f884c157760 2024-01-24T13:59:27.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f88541d6700 1 -- 172.21.15.29:0/377735752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c157760 msgr2=0x7f884c157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:27.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f88541d6700 1 --2- 172.21.15.29:0/377735752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c157760 0x7f884c157c20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f8848009760 tx=0x7f8848009060).stop 2024-01-24T13:59:27.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f88541d6700 1 -- 172.21.15.29:0/377735752 shutdown_connections 2024-01-24T13:59:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f88541d6700 1 --2- 172.21.15.29:0/377735752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c157760 0x7f884c157c20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.276+0000 7f88541d6700 1 -- 172.21.15.29:0/377735752 >> 172.21.15.29:0/377735752 conn(0x7f884c0c7260 msgr2=0x7f884c0c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.277+0000 7f88541d6700 1 -- 172.21.15.29:0/377735752 shutdown_connections 2024-01-24T13:59:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.277+0000 7f88541d6700 1 -- 172.21.15.29:0/377735752 wait complete. 2024-01-24T13:59:27.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.278+0000 7f88541d6700 1 Processor -- start 2024-01-24T13:59:27.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.278+0000 7f88541d6700 1 -- start start 2024-01-24T13:59:27.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f88541d6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c157380 0x7f884c13f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:27.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f88541d6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c157760 0x7f884c1427b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:27.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f88541d6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c158110 0x7f884c0c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:27.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f88541d6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c141710 con 0x7f884c157380 2024-01-24T13:59:27.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f88541d6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c1419f0 con 0x7f884c158110 2024-01-24T13:59:27.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f88541d6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c141cd0 con 0x7f884c157760 2024-01-24T13:59:27.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f8851f72700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c157380 0x7f884c13f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:27.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f8851f72700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c157380 0x7f884c13f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39540/0 (socket says 172.21.15.29:39540) 2024-01-24T13:59:27.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f8851f72700 1 -- 172.21.15.29:0/3900939600 learned_addr learned my addr 172.21.15.29:0/3900939600 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:27.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f8851771700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c157760 0x7f884c1427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f8852773700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c158110 0x7f884c0c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f8851f72700 1 -- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c157760 msgr2=0x7f884c1427b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f8851f72700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c157760 0x7f884c1427b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f8851f72700 1 -- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c158110 msgr2=0x7f884c0c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f8851f72700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c158110 0x7f884c0c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f8851f72700 1 -- 172.21.15.29:0/3900939600 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88480092a0 con 0x7f884c157380 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.279+0000 7f8851f72700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c157380 0x7f884c13f5f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f883c00c4c0 tx=0x7f883c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.280+0000 7f8841ffb700 1 -- 172.21.15.29:0/3900939600 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0072e0 con 0x7f884c157380 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.280+0000 7f8841ffb700 1 -- 172.21.15.29:0/3900939600 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c007440 con 0x7f884c157380 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.280+0000 7f88541d6700 1 -- 172.21.15.29:0/3900939600 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c141fb0 con 0x7f884c157380 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.280+0000 7f8841ffb700 1 -- 172.21.15.29:0/3900939600 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0057a0 con 0x7f884c157380 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.280+0000 7f88541d6700 1 -- 172.21.15.29:0/3900939600 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c13fae0 con 0x7f884c157380 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.281+0000 7f88541d6700 1 -- 172.21.15.29:0/3900939600 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884c13fcd0 con 0x7f884c157380 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.281+0000 7f8841ffb700 1 -- 172.21.15.29:0/3900939600 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f883c005b90 con 0x7f884c157380 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.281+0000 7f8841ffb700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f883005f150 0x7f8830061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:27.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.282+0000 7f8841ffb700 1 -- 172.21.15.29:0/3900939600 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f883c079bb0 con 0x7f884c157380 2024-01-24T13:59:27.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.282+0000 7f8851771700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f883005f150 0x7f8830061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:27.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.282+0000 7f8851771700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f883005f150 0x7f8830061610 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f884c157b80 tx=0x7f8848017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:27.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.286+0000 7f8841ffb700 1 -- 172.21.15.29:0/3900939600 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f884c13fcd0 con 0x7f884c157380 2024-01-24T13:59:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:27.518+0000 7f88541d6700 1 -- 172.21.15.29:0/3900939600 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f884c13fcd0 con 0x7f884c157380 2024-01-24T13:59:28.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.380+0000 7f8841ffb700 1 -- 172.21.15.29:0/3900939600 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f884c13fcd0 con 0x7f884c157380 2024-01-24T13:59:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.383+0000 7f88541d6700 1 -- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f883005f150 msgr2=0x7f8830061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.383+0000 7f88541d6700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f883005f150 0x7f8830061610 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f884c157b80 tx=0x7f8848017000).stop 2024-01-24T13:59:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.383+0000 7f88541d6700 1 -- 172.21.15.29:0/3900939600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c157380 msgr2=0x7f884c13f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.383+0000 7f88541d6700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c157380 0x7f884c13f5f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f883c00c4c0 tx=0x7f883c007c10).stop 2024-01-24T13:59:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.383+0000 7f88541d6700 1 -- 172.21.15.29:0/3900939600 shutdown_connections 2024-01-24T13:59:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.383+0000 7f88541d6700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c158110 0x7f884c0c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.383+0000 7f88541d6700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f883005f150 0x7f8830061610 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.383+0000 7f88541d6700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f884c157380 0x7f884c13f5f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.383+0000 7f88541d6700 1 --2- 172.21.15.29:0/3900939600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c157760 0x7f884c1427b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.383+0000 7f88541d6700 1 -- 172.21.15.29:0/3900939600 >> 172.21.15.29:0/3900939600 conn(0x7f884c0c7260 msgr2=0x7f884c151ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.383+0000 7f88541d6700 1 -- 172.21.15.29:0/3900939600 shutdown_connections 2024-01-24T13:59:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.383+0000 7f88541d6700 1 -- 172.21.15.29:0/3900939600 wait complete. 2024-01-24T13:59:28.398 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-24T13:59:28.399 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-24T13:59:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.536+0000 7f3742230700 1 Processor -- start 2024-01-24T13:59:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.536+0000 7f3742230700 1 -- start start 2024-01-24T13:59:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f3742230700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f373c134ea0 0x7f373c0c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f3742230700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f373c0c36c0 0x7f373c0664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f3742230700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373c066e60 con 0x7f373c0669b0 2024-01-24T13:59:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f3742230700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c0686a0 con 0x7f373c134ea0 2024-01-24T13:59:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f3742230700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c068b30 con 0x7f373c0c36c0 2024-01-24T13:59:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f373b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f373c134ea0 0x7f373c0c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:28.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f373b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f373c134ea0 0x7f373c0c31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39550/0 (socket says 172.21.15.29:39550) 2024-01-24T13:59:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f373b7fe700 1 -- 172.21.15.29:0/1023060067 learned_addr learned my addr 172.21.15.29:0/1023060067 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f373affd700 1 --2- 172.21.15.29:0/1023060067 >> v2:172.21.15.163:3300/0 conn(0x7f373c0c36c0 0x7f373c0664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f373b7fe700 1 -- 172.21.15.29:0/1023060067 >> v2:172.21.15.163:3300/0 conn(0x7f373c0c36c0 msgr2=0x7f373c0664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f373b7fe700 1 --2- 172.21.15.29:0/1023060067 >> v2:172.21.15.163:3300/0 conn(0x7f373c0c36c0 0x7f373c0664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f373b7fe700 1 -- 172.21.15.29:0/1023060067 >> v1:172.21.15.163:6789/0 conn(0x7f373c0669b0 legacy=0x7f373c1559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f373b7fe700 1 -- 172.21.15.29:0/1023060067 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c067a90 con 0x7f373c134ea0 2024-01-24T13:59:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f373affd700 1 --2- 172.21.15.29:0/1023060067 >> v2:172.21.15.163:3300/0 conn(0x7f373c0c36c0 0x7f373c0664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f373b7fe700 1 --2- 172.21.15.29:0/1023060067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f373c134ea0 0x7f373c0c31d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f373c06a270 tx=0x7f372c0090f0).ready entity=mon.0 client_cookie=8ee8ca6ceee2a8a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.537+0000 7f373a7fc700 1 -- 172.21.15.29:0/1023060067 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c00b600 con 0x7f373c134ea0 2024-01-24T13:59:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.538+0000 7f373a7fc700 1 -- 172.21.15.29:0/1023060067 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c00b760 con 0x7f373c134ea0 2024-01-24T13:59:28.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.538+0000 7f373a7fc700 1 -- 172.21.15.29:0/1023060067 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c00bae0 con 0x7f373c134ea0 2024-01-24T13:59:28.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.538+0000 7f3742230700 1 -- 172.21.15.29:0/1023060067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f373c134ea0 msgr2=0x7f373c0c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:28.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.538+0000 7f3742230700 1 --2- 172.21.15.29:0/1023060067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f373c134ea0 0x7f373c0c31d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f373c06a270 tx=0x7f372c0090f0).stop 2024-01-24T13:59:28.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.538+0000 7f3742230700 1 -- 172.21.15.29:0/1023060067 shutdown_connections 2024-01-24T13:59:28.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.538+0000 7f3742230700 1 --2- 172.21.15.29:0/1023060067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f373c134ea0 0x7f373c0c31d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:28.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.538+0000 7f3742230700 1 --2- 172.21.15.29:0/1023060067 >> v2:172.21.15.163:3300/0 conn(0x7f373c0c36c0 0x7f373c0664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:28.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.538+0000 7f3742230700 1 -- 172.21.15.29:0/1023060067 >> 172.21.15.29:0/1023060067 conn(0x7f373c0c8bd0 msgr2=0x7f373c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:28.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.538+0000 7f3742230700 1 -- 172.21.15.29:0/1023060067 shutdown_connections 2024-01-24T13:59:28.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.538+0000 7f3742230700 1 -- 172.21.15.29:0/1023060067 wait complete. 2024-01-24T13:59:28.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.539+0000 7f3742230700 1 Processor -- start 2024-01-24T13:59:28.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.540+0000 7f3742230700 1 -- start start 2024-01-24T13:59:28.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.540+0000 7f3742230700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f373c0669b0 0x7f373c164480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:28.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.540+0000 7f3742230700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f373c0c36c0 0x7f373c152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:28.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f3742230700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f373c134ea0 0x7f373c0c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:28.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f3742230700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c14bc00 con 0x7f373c0669b0 2024-01-24T13:59:28.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f3742230700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0ab430 con 0x7f373c0c36c0 2024-01-24T13:59:28.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f3742230700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c164b40 con 0x7f373c134ea0 2024-01-24T13:59:28.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f373b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f373c0669b0 0x7f373c164480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:28.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f373b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f373c0669b0 0x7f373c164480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39564/0 (socket says 172.21.15.29:39564) 2024-01-24T13:59:28.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f373b7fe700 1 -- 172.21.15.29:0/2087209736 learned_addr learned my addr 172.21.15.29:0/2087209736 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:28.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f373affd700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f373c0c36c0 0x7f373c152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:28.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f373bfff700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f373c134ea0 0x7f373c0c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:28.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f373b7fe700 1 -- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f373c134ea0 msgr2=0x7f373c0c1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:28.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f373b7fe700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f373c134ea0 0x7f373c0c1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:28.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f373b7fe700 1 -- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f373c0c36c0 msgr2=0x7f373c152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:28.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f373b7fe700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f373c0c36c0 0x7f373c152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:28.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f373b7fe700 1 -- 172.21.15.29:0/2087209736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372c009370 con 0x7f373c0669b0 2024-01-24T13:59:28.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f373bfff700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f373c134ea0 0x7f373c0c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:28.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f373b7fe700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f373c0669b0 0x7f373c164480 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f373c06c260 tx=0x7f372c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:28.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f37237fe700 1 -- 172.21.15.29:0/2087209736 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c00b8d0 con 0x7f373c0669b0 2024-01-24T13:59:28.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f37237fe700 1 -- 172.21.15.29:0/2087209736 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c00ba50 con 0x7f373c0669b0 2024-01-24T13:59:28.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f3742230700 1 -- 172.21.15.29:0/2087209736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373c164e20 con 0x7f373c0669b0 2024-01-24T13:59:28.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f37237fe700 1 -- 172.21.15.29:0/2087209736 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c01b690 con 0x7f373c0669b0 2024-01-24T13:59:28.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.541+0000 7f3742230700 1 -- 172.21.15.29:0/2087209736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373c0a8750 con 0x7f373c0669b0 2024-01-24T13:59:28.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.543+0000 7f3742230700 1 -- 172.21.15.29:0/2087209736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373c0a8910 con 0x7f373c0669b0 2024-01-24T13:59:28.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.543+0000 7f37237fe700 1 -- 172.21.15.29:0/2087209736 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f372c01ba00 con 0x7f373c0669b0 2024-01-24T13:59:28.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.543+0000 7f37237fe700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f372805f150 0x7f3728061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:28.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.543+0000 7f37237fe700 1 -- 172.21.15.29:0/2087209736 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f372c07bb50 con 0x7f373c0669b0 2024-01-24T13:59:28.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.544+0000 7f373affd700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f372805f150 0x7f3728061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:28.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.544+0000 7f373affd700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f372805f150 0x7f3728061610 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f3730006d80 tx=0x7f3730008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:28.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.548+0000 7f37237fe700 1 -- 172.21.15.29:0/2087209736 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f372c04ccc0 con 0x7f373c0669b0 2024-01-24T13:59:28.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:28.775+0000 7f3742230700 1 -- 172.21.15.29:0/2087209736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f373c0a8ac0 con 0x7f373c0669b0 2024-01-24T13:59:29.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.386+0000 7f37237fe700 1 -- 172.21.15.29:0/2087209736 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f372c023360 con 0x7f373c0669b0 2024-01-24T13:59:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.389+0000 7f3742230700 1 -- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f372805f150 msgr2=0x7f3728061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.389+0000 7f3742230700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f372805f150 0x7f3728061610 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f3730006d80 tx=0x7f3730008040).stop 2024-01-24T13:59:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.389+0000 7f3742230700 1 -- 172.21.15.29:0/2087209736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f373c0669b0 msgr2=0x7f373c164480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.389+0000 7f3742230700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f373c0669b0 0x7f373c164480 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f373c06c260 tx=0x7f372c017000).stop 2024-01-24T13:59:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.389+0000 7f3742230700 1 -- 172.21.15.29:0/2087209736 shutdown_connections 2024-01-24T13:59:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.389+0000 7f3742230700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f373c134ea0 0x7f373c0c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.389+0000 7f3742230700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f372805f150 0x7f3728061610 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.389+0000 7f3742230700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f373c0669b0 0x7f373c164480 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.389+0000 7f3742230700 1 --2- 172.21.15.29:0/2087209736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f373c0c36c0 0x7f373c152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.389+0000 7f3742230700 1 -- 172.21.15.29:0/2087209736 >> 172.21.15.29:0/2087209736 conn(0x7f373c0c8bd0 msgr2=0x7f373c0c7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.389+0000 7f3742230700 1 -- 172.21.15.29:0/2087209736 shutdown_connections 2024-01-24T13:59:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.389+0000 7f3742230700 1 -- 172.21.15.29:0/2087209736 wait complete. 2024-01-24T13:59:29.404 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-24T13:59:29.405 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:59:29.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.538+0000 7f973f5c9700 1 Processor -- start 2024-01-24T13:59:29.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.538+0000 7f973f5c9700 1 -- start start 2024-01-24T13:59:29.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.539+0000 7f973f5c9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9738151b10 0x7f9738066210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:29.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.539+0000 7f973f5c9700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973806a5a0 con 0x7f9738069240 2024-01-24T13:59:29.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.539+0000 7f973f5c9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9738069990 con 0x7f973814eb80 2024-01-24T13:59:29.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.539+0000 7f973f5c9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9738069c70 con 0x7f9738151b10 2024-01-24T13:59:29.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.539+0000 7f973db66700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9738069240 0x7f97380c30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54044/0 (socket says 172.21.15.29:54044) 2024-01-24T13:59:29.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.539+0000 7f973db66700 1 -- 172.21.15.29:0/3810149768 learned_addr learned my addr 172.21.15.29:0/3810149768 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:29.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.539+0000 7f973cb64700 1 --2- 172.21.15.29:0/3810149768 >> v2:172.21.15.163:3300/0 conn(0x7f9738151b10 0x7f9738066210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.540+0000 7f972ffff700 1 -- 172.21.15.29:0/3810149768 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869731020 0 0) 0x7f973806a5a0 con 0x7f9738069240 2024-01-24T13:59:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.540+0000 7f972ffff700 1 -- 172.21.15.29:0/3810149768 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97240033f0 con 0x7f9738069240 2024-01-24T13:59:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.540+0000 7f973cb64700 1 -- 172.21.15.29:0/3810149768 >> v1:172.21.15.163:6789/0 conn(0x7f973814eb80 legacy=0x7f9738151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.540+0000 7f972ffff700 1 -- 172.21.15.29:0/3810149768 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989633540 0 0) 0x7f9738069990 con 0x7f973814eb80 2024-01-24T13:59:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.540+0000 7f973cb64700 1 -- 172.21.15.29:0/3810149768 >> v1:172.21.15.29:6789/0 conn(0x7f9738069240 legacy=0x7f97380c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.540+0000 7f973cb64700 1 -- 172.21.15.29:0/3810149768 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9738069f50 con 0x7f9738151b10 2024-01-24T13:59:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.540+0000 7f973cb64700 1 --2- 172.21.15.29:0/3810149768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9738151b10 0x7f9738066210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f9734009760 tx=0x7f9734009060).ready entity=mon.1 client_cookie=73a145cbf3574fd2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.540+0000 7f972ffff700 1 -- 172.21.15.29:0/3810149768 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400f510 con 0x7f9738151b10 2024-01-24T13:59:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.540+0000 7f972ffff700 1 -- 172.21.15.29:0/3810149768 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973400f670 con 0x7f9738151b10 2024-01-24T13:59:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.540+0000 7f972ffff700 1 -- 172.21.15.29:0/3810149768 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400f9f0 con 0x7f9738151b10 2024-01-24T13:59:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.541+0000 7f973f5c9700 1 -- 172.21.15.29:0/3810149768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9738151b10 msgr2=0x7f9738066210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.541+0000 7f973f5c9700 1 --2- 172.21.15.29:0/3810149768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9738151b10 0x7f9738066210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f9734009760 tx=0x7f9734009060).stop 2024-01-24T13:59:29.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.541+0000 7f973f5c9700 1 -- 172.21.15.29:0/3810149768 shutdown_connections 2024-01-24T13:59:29.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.541+0000 7f973f5c9700 1 --2- 172.21.15.29:0/3810149768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9738151b10 0x7f9738066210 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.541+0000 7f973f5c9700 1 -- 172.21.15.29:0/3810149768 >> 172.21.15.29:0/3810149768 conn(0x7f97380c85c0 msgr2=0x7f97380c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:29.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.541+0000 7f973f5c9700 1 -- 172.21.15.29:0/3810149768 shutdown_connections 2024-01-24T13:59:29.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.541+0000 7f973f5c9700 1 -- 172.21.15.29:0/3810149768 wait complete. 2024-01-24T13:59:29.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.542+0000 7f973f5c9700 1 Processor -- start 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.542+0000 7f973f5c9700 1 -- start start 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.543+0000 7f973f5c9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9738069240 0x7f9738141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.543+0000 7f973f5c9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f973814eb80 0x7f9738152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.543+0000 7f973f5c9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9738151b10 0x7f9738152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.543+0000 7f973f5c9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380a7600 con 0x7f9738151b10 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.543+0000 7f973f5c9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380bfb60 con 0x7f973814eb80 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.543+0000 7f973f5c9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97380c0120 con 0x7f9738069240 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.543+0000 7f973db66700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9738151b10 0x7f9738152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.543+0000 7f973db66700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9738151b10 0x7f9738152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39578/0 (socket says 172.21.15.29:39578) 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.543+0000 7f973db66700 1 -- 172.21.15.29:0/3183637195 learned_addr learned my addr 172.21.15.29:0/3183637195 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.543+0000 7f973cb64700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f973814eb80 0x7f9738152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.543+0000 7f973d365700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9738069240 0x7f9738141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.544+0000 7f973db66700 1 -- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9738069240 msgr2=0x7f9738141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.544+0000 7f973db66700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9738069240 0x7f9738141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.544+0000 7f973db66700 1 -- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f973814eb80 msgr2=0x7f9738152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.544+0000 7f973db66700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f973814eb80 0x7f9738152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.544+0000 7f973db66700 1 -- 172.21.15.29:0/3183637195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97340092a0 con 0x7f9738151b10 2024-01-24T13:59:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.544+0000 7f973db66700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9738151b10 0x7f9738152e10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f9730001460 tx=0x7f9730001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.544+0000 7f972d7fa700 1 -- 172.21.15.29:0/3183637195 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730005d30 con 0x7f9738151b10 2024-01-24T13:59:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.544+0000 7f972d7fa700 1 -- 172.21.15.29:0/3183637195 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9730005e90 con 0x7f9738151b10 2024-01-24T13:59:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.544+0000 7f973f5c9700 1 -- 172.21.15.29:0/3183637195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9738141c10 con 0x7f9738151b10 2024-01-24T13:59:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.544+0000 7f972d7fa700 1 -- 172.21.15.29:0/3183637195 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730007320 con 0x7f9738151b10 2024-01-24T13:59:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.544+0000 7f973f5c9700 1 -- 172.21.15.29:0/3183637195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97380a5a00 con 0x7f9738151b10 2024-01-24T13:59:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.545+0000 7f973f5c9700 1 -- 172.21.15.29:0/3183637195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97380a5c20 con 0x7f9738151b10 2024-01-24T13:59:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.546+0000 7f972d7fa700 1 -- 172.21.15.29:0/3183637195 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f97300076d0 con 0x7f9738151b10 2024-01-24T13:59:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.546+0000 7f972d7fa700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f971c05f150 0x7f971c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.546+0000 7f972d7fa700 1 -- 172.21.15.29:0/3183637195 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f973007a2b0 con 0x7f9738151b10 2024-01-24T13:59:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.546+0000 7f973d365700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f971c05f150 0x7f971c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.547+0000 7f973d365700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f971c05f150 0x7f971c061610 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f9728007fd0 tx=0x7f9728007bb0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:29.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.550+0000 7f972d7fa700 1 -- 172.21.15.29:0/3183637195 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f973004b420 con 0x7f9738151b10 2024-01-24T13:59:29.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.776+0000 7f973f5c9700 1 -- 172.21.15.29:0/3183637195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97380475e0 con 0x7f9738151b10 2024-01-24T13:59:29.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.777+0000 7f972d7fa700 1 -- 172.21.15.29:0/3183637195 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f973004ea40 con 0x7f9738151b10 2024-01-24T13:59:29.777 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:29.777 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1291112242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1291112242},{"type":"v1","addr":"172.21.15.163:6837","nonce":1291112242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4793,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/241296864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":241296864},{"type":"v1","addr":"172.21.15.163:6836","nonce":241296864}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5127,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2565530631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2565530631},{"type":"v1","addr":"172.21.15.29:6833","nonce":2565530631}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:59:26.370202+0000","modified":"2024-01-24T13:59:29.380658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1245725396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1245725396},{"type":"v1","addr":"172.21.15.29:6835","nonce":1245725396}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-24T13:59:29.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.778+0000 7f973f5c9700 1 -- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f971c05f150 msgr2=0x7f971c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.778+0000 7f973f5c9700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f971c05f150 0x7f971c061610 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f9728007fd0 tx=0x7f9728007bb0).stop 2024-01-24T13:59:29.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.778+0000 7f973f5c9700 1 -- 172.21.15.29:0/3183637195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9738151b10 msgr2=0x7f9738152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.778+0000 7f973f5c9700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9738151b10 0x7f9738152e10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f9730001460 tx=0x7f9730001650).stop 2024-01-24T13:59:29.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.779+0000 7f973f5c9700 1 -- 172.21.15.29:0/3183637195 shutdown_connections 2024-01-24T13:59:29.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.779+0000 7f973f5c9700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f971c05f150 0x7f971c061610 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.779+0000 7f973f5c9700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9738151b10 0x7f9738152e10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.779+0000 7f973f5c9700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9738069240 0x7f9738141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.779+0000 7f973f5c9700 1 --2- 172.21.15.29:0/3183637195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f973814eb80 0x7f9738152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.779+0000 7f973f5c9700 1 -- 172.21.15.29:0/3183637195 >> 172.21.15.29:0/3183637195 conn(0x7f97380c85c0 msgr2=0x7f97380c5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:29.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.779+0000 7f973f5c9700 1 -- 172.21.15.29:0/3183637195 shutdown_connections 2024-01-24T13:59:29.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.779+0000 7f973f5c9700 1 -- 172.21.15.29:0/3183637195 wait complete. 2024-01-24T13:59:29.780 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 18 2024-01-24T13:59:29.792 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T13:59:29.804 INFO:tasks.ceph.mds.a.smithi029.stderr:2024-01-24T13:59:29.803+0000 7f7c982a0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T13:59:29.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.922+0000 7f55b960d700 1 Processor -- start 2024-01-24T13:59:29.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.922+0000 7f55b960d700 1 -- start start 2024-01-24T13:59:29.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.923+0000 7f55b960d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f55b41540a0 0x7f55b40c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.923+0000 7f55b960d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b4062ec0 0x7f55b4069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.923+0000 7f55b960d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b4065360 con 0x7f55b41550f0 2024-01-24T13:59:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.923+0000 7f55b960d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b40a74f0 con 0x7f55b41540a0 2024-01-24T13:59:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.923+0000 7f55b960d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40a6ce0 con 0x7f55b4062ec0 2024-01-24T13:59:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.923+0000 7f55b2ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f55b41540a0 0x7f55b40c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.924+0000 7f55b2ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f55b41540a0 0x7f55b40c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39590/0 (socket says 172.21.15.29:39590) 2024-01-24T13:59:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.924+0000 7f55b2ffd700 1 -- 172.21.15.29:0/3959874838 learned_addr learned my addr 172.21.15.29:0/3959874838 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.924+0000 7f55b37fe700 1 --2- 172.21.15.29:0/3959874838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b4062ec0 0x7f55b4069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.924+0000 7f55b2ffd700 1 -- 172.21.15.29:0/3959874838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b4062ec0 msgr2=0x7f55b4069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.924+0000 7f55b2ffd700 1 --2- 172.21.15.29:0/3959874838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b4062ec0 0x7f55b4069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.924+0000 7f55b2ffd700 1 -- 172.21.15.29:0/3959874838 >> v1:172.21.15.163:6789/0 conn(0x7f55b41550f0 legacy=0x7f55b40627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.924+0000 7f55b2ffd700 1 -- 172.21.15.29:0/3959874838 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b40636f0 con 0x7f55b41540a0 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.924+0000 7f55b2ffd700 1 --2- 172.21.15.29:0/3959874838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b41540a0 0x7f55b40c4ca0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f55a4006fd0 tx=0x7f55a40090f0).ready entity=mon.0 client_cookie=7ea5965e57d19cf8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.924+0000 7f55b1ffb700 1 -- 172.21.15.29:0/3959874838 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400b600 con 0x7f55b41540a0 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.924+0000 7f55b1ffb700 1 -- 172.21.15.29:0/3959874838 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a400b760 con 0x7f55b41540a0 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.924+0000 7f55b1ffb700 1 -- 172.21.15.29:0/3959874838 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400bae0 con 0x7f55b41540a0 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.925+0000 7f55b960d700 1 -- 172.21.15.29:0/3959874838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b41540a0 msgr2=0x7f55b40c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.925+0000 7f55b960d700 1 --2- 172.21.15.29:0/3959874838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b41540a0 0x7f55b40c4ca0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f55a4006fd0 tx=0x7f55a40090f0).stop 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.925+0000 7f55b960d700 1 -- 172.21.15.29:0/3959874838 shutdown_connections 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.925+0000 7f55b960d700 1 --2- 172.21.15.29:0/3959874838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b4062ec0 0x7f55b4069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.925+0000 7f55b960d700 1 --2- 172.21.15.29:0/3959874838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b41540a0 0x7f55b40c4ca0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.925+0000 7f55b960d700 1 -- 172.21.15.29:0/3959874838 >> 172.21.15.29:0/3959874838 conn(0x7f55b40c85c0 msgr2=0x7f55b40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.925+0000 7f55b960d700 1 -- 172.21.15.29:0/3959874838 shutdown_connections 2024-01-24T13:59:29.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.925+0000 7f55b960d700 1 -- 172.21.15.29:0/3959874838 wait complete. 2024-01-24T13:59:29.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.926+0000 7f55b960d700 1 Processor -- start 2024-01-24T13:59:29.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.926+0000 7f55b960d700 1 -- start start 2024-01-24T13:59:29.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.927+0000 7f55b960d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b4062ec0 0x7f55b41524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:29.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.927+0000 7f55b960d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b41550f0 0x7f55b4150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:29.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.927+0000 7f55b960d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b41529e0 0x7f55b4151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.927+0000 7f55b960d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40ab390 con 0x7f55b41529e0 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.927+0000 7f55b960d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40aaaa0 con 0x7f55b4062ec0 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.927+0000 7f55b960d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b4151c50 con 0x7f55b41550f0 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.927+0000 7f55b37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b41529e0 0x7f55b4151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.927+0000 7f55b37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b41529e0 0x7f55b4151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39596/0 (socket says 172.21.15.29:39596) 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.927+0000 7f55b37fe700 1 -- 172.21.15.29:0/516513834 learned_addr learned my addr 172.21.15.29:0/516513834 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.927+0000 7f55b27fc700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b41550f0 0x7f55b4150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.927+0000 7f55b2ffd700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b4062ec0 0x7f55b41524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.928+0000 7f55b37fe700 1 -- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b41550f0 msgr2=0x7f55b4150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.928+0000 7f55b37fe700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b41550f0 0x7f55b4150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.928+0000 7f55b37fe700 1 -- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b4062ec0 msgr2=0x7f55b41524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.928+0000 7f55b37fe700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b4062ec0 0x7f55b41524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.928+0000 7f55b37fe700 1 -- 172.21.15.29:0/516513834 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4009370 con 0x7f55b41529e0 2024-01-24T13:59:29.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.928+0000 7f55b37fe700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b41529e0 0x7f55b4151410 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f55ac007f30 tx=0x7f55ac0098e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:29.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.928+0000 7f559affd700 1 -- 172.21.15.29:0/516513834 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00eb90 con 0x7f55b41529e0 2024-01-24T13:59:29.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.928+0000 7f559affd700 1 -- 172.21.15.29:0/516513834 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac00ecf0 con 0x7f55b41529e0 2024-01-24T13:59:29.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.928+0000 7f55b960d700 1 -- 172.21.15.29:0/516513834 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b40aad40 con 0x7f55b41529e0 2024-01-24T13:59:29.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.928+0000 7f559affd700 1 -- 172.21.15.29:0/516513834 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac014610 con 0x7f55b41529e0 2024-01-24T13:59:29.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.928+0000 7f55b960d700 1 -- 172.21.15.29:0/516513834 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b4151900 con 0x7f55b41529e0 2024-01-24T13:59:29.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.929+0000 7f55b960d700 1 -- 172.21.15.29:0/516513834 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b40ab540 con 0x7f55b41529e0 2024-01-24T13:59:29.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.929+0000 7f559affd700 1 -- 172.21.15.29:0/516513834 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f55ac014a00 con 0x7f55b41529e0 2024-01-24T13:59:29.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.930+0000 7f559affd700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55a005f150 0x7f55a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:29.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.930+0000 7f559affd700 1 -- 172.21.15.29:0/516513834 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f55ac07a750 con 0x7f55b41529e0 2024-01-24T13:59:29.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.930+0000 7f55b2ffd700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55a005f150 0x7f55a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:29.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.930+0000 7f55b2ffd700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55a005f150 0x7f55a0061610 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f55b4064960 tx=0x7f55a4017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:29.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:29.934+0000 7f559affd700 1 -- 172.21.15.29:0/516513834 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f55b40ab540 con 0x7f55b41529e0 2024-01-24T13:59:30.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.115+0000 7f55b960d700 1 -- 172.21.15.29:0/516513834 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55b40475e0 con 0x7f55b41529e0 2024-01-24T13:59:30.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.116+0000 7f559affd700 1 -- 172.21.15.29:0/516513834 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+17164 (secure 0 0 0) 0x7f55ac04b8c0 con 0x7f55b41529e0 2024-01-24T13:59:30.117 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:30.117 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":41,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T13:59:27.371939+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:59:24.684931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-24T13:59:25.345273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T13:59:30.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.118+0000 7f55b960d700 1 -- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55a005f150 msgr2=0x7f55a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.118+0000 7f55b960d700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55a005f150 0x7f55a0061610 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f55b4064960 tx=0x7f55a4017000).stop 2024-01-24T13:59:30.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.118+0000 7f55b960d700 1 -- 172.21.15.29:0/516513834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b41529e0 msgr2=0x7f55b4151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.118+0000 7f55b960d700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b41529e0 0x7f55b4151410 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f55ac007f30 tx=0x7f55ac0098e0).stop 2024-01-24T13:59:30.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.118+0000 7f55b960d700 1 -- 172.21.15.29:0/516513834 shutdown_connections 2024-01-24T13:59:30.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.118+0000 7f55b960d700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55a005f150 0x7f55a0061610 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.118+0000 7f55b960d700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b41529e0 0x7f55b4151410 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.118+0000 7f55b960d700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b4062ec0 0x7f55b41524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.118+0000 7f55b960d700 1 --2- 172.21.15.29:0/516513834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b41550f0 0x7f55b4150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.118+0000 7f55b960d700 1 -- 172.21.15.29:0/516513834 >> 172.21.15.29:0/516513834 conn(0x7f55b40c85c0 msgr2=0x7f55b40659e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:30.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.118+0000 7f55b960d700 1 -- 172.21.15.29:0/516513834 shutdown_connections 2024-01-24T13:59:30.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.118+0000 7f55b960d700 1 -- 172.21.15.29:0/516513834 wait complete. 2024-01-24T13:59:30.132 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-24T13:59:30.133 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T13:59:30.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.303+0000 7f5fb3a4e700 1 Processor -- start 2024-01-24T13:59:30.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.304+0000 7f5fb3a4e700 1 -- start start 2024-01-24T13:59:30.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.304+0000 7f5fb3a4e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fac062ec0 0x7f5fac0c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.304+0000 7f5fb3a4e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5fac1550f0 0x7f5fac069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.304+0000 7f5fb3a4e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fac065360 con 0x7f5fac1540a0 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.304+0000 7f5fb3a4e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac0a74f0 con 0x7f5fac1550f0 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.304+0000 7f5fb3a4e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0a6ce0 con 0x7f5fac062ec0 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fb1feb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5fac1550f0 0x7f5fac069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fb17ea700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5fac1540a0 0x7f5fac0627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54058/0 (socket says 172.21.15.29:54058) 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fb1feb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5fac1550f0 0x7f5fac069240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39606/0 (socket says 172.21.15.29:39606) 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fb17ea700 1 -- 172.21.15.29:0/1359143081 learned_addr learned my addr 172.21.15.29:0/1359143081 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fb0fe9700 1 --2- 172.21.15.29:0/1359143081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fac062ec0 0x7f5fac0c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fb1feb700 1 -- 172.21.15.29:0/1359143081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fac062ec0 msgr2=0x7f5fac0c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fb1feb700 1 --2- 172.21.15.29:0/1359143081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fac062ec0 0x7f5fac0c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fb1feb700 1 -- 172.21.15.29:0/1359143081 >> v1:172.21.15.29:6789/0 conn(0x7f5fac1540a0 legacy=0x7f5fac0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fa3fff700 1 -- 172.21.15.29:0/1359143081 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470260044 0 0) 0x7f5fac065360 con 0x7f5fac1540a0 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fb1feb700 1 -- 172.21.15.29:0/1359143081 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0636f0 con 0x7f5fac1550f0 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fb0fe9700 1 --2- 172.21.15.29:0/1359143081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fac062ec0 0x7f5fac0c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fb1feb700 1 --2- 172.21.15.29:0/1359143081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fac1550f0 0x7f5fac069240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f5fa4009e60 tx=0x7f5fa4009780).ready entity=mon.0 client_cookie=70466e83feeb3991 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fa3fff700 1 -- 172.21.15.29:0/1359143081 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa400e070 con 0x7f5fac1550f0 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fa3fff700 1 -- 172.21.15.29:0/1359143081 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa400bc30 con 0x7f5fac1550f0 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.305+0000 7f5fa3fff700 1 -- 172.21.15.29:0/1359143081 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4012410 con 0x7f5fac1550f0 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.306+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/1359143081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fac1550f0 msgr2=0x7f5fac069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.306+0000 7f5fb3a4e700 1 --2- 172.21.15.29:0/1359143081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fac1550f0 0x7f5fac069240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f5fa4009e60 tx=0x7f5fa4009780).stop 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.306+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/1359143081 shutdown_connections 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.306+0000 7f5fb3a4e700 1 --2- 172.21.15.29:0/1359143081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fac1550f0 0x7f5fac069240 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.306+0000 7f5fb3a4e700 1 --2- 172.21.15.29:0/1359143081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fac062ec0 0x7f5fac0c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.306+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/1359143081 >> 172.21.15.29:0/1359143081 conn(0x7f5fac0c85c0 msgr2=0x7f5fac0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.306+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/1359143081 shutdown_connections 2024-01-24T13:59:30.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.306+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/1359143081 wait complete. 2024-01-24T13:59:30.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.307+0000 7f5fb3a4e700 1 Processor -- start 2024-01-24T13:59:30.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.307+0000 7f5fb3a4e700 1 -- start start 2024-01-24T13:59:30.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.308+0000 7f5fb3a4e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fac062ec0 0x7f5fac0aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.308+0000 7f5fb3a4e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fac1540a0 0x7f5fac14fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.308+0000 7f5fb3a4e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fac1550f0 0x7f5fac155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.308+0000 7f5fb3a4e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac150080 con 0x7f5fac1550f0 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.308+0000 7f5fb3a4e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac151890 con 0x7f5fac062ec0 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.308+0000 7f5fb3a4e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac1560a0 con 0x7f5fac1540a0 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb1feb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fac1550f0 0x7f5fac155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb1feb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fac1550f0 0x7f5fac155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39608/0 (socket says 172.21.15.29:39608) 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb1feb700 1 -- 172.21.15.29:0/775969978 learned_addr learned my addr 172.21.15.29:0/775969978 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb0fe9700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fac1540a0 0x7f5fac14fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb17ea700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fac062ec0 0x7f5fac0aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb1feb700 1 -- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fac1540a0 msgr2=0x7f5fac14fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb1feb700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fac1540a0 0x7f5fac14fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb1feb700 1 -- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fac062ec0 msgr2=0x7f5fac0aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb1feb700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fac062ec0 0x7f5fac0aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb1feb700 1 -- 172.21.15.29:0/775969978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa4009a20 con 0x7f5fac1550f0 2024-01-24T13:59:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb17ea700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fac062ec0 0x7f5fac0aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb1feb700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fac1550f0 0x7f5fac155850 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f5fac069030 tx=0x7f5fa4009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fa17fa700 1 -- 172.21.15.29:0/775969978 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4009440 con 0x7f5fac1550f0 2024-01-24T13:59:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fa17fa700 1 -- 172.21.15.29:0/775969978 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa401c660 con 0x7f5fac1550f0 2024-01-24T13:59:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/775969978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac156380 con 0x7f5fac1550f0 2024-01-24T13:59:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fa17fa700 1 -- 172.21.15.29:0/775969978 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa401ca20 con 0x7f5fac1550f0 2024-01-24T13:59:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.309+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/775969978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac156940 con 0x7f5fac1550f0 2024-01-24T13:59:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.310+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/775969978 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac0475e0 con 0x7f5fac1550f0 2024-01-24T13:59:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.311+0000 7f5fa17fa700 1 -- 172.21.15.29:0/775969978 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5fa4003040 con 0x7f5fac1550f0 2024-01-24T13:59:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.311+0000 7f5fa17fa700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f9005f150 0x7f5f90061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.311+0000 7f5fa17fa700 1 -- 172.21.15.29:0/775969978 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f5fa407c290 con 0x7f5fac1550f0 2024-01-24T13:59:30.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.311+0000 7f5fb17ea700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f9005f150 0x7f5f90061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.312+0000 7f5fb17ea700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f9005f150 0x7f5f90061610 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f5fa8001750 tx=0x7f5fa8009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:30.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.316+0000 7f5fa17fa700 1 -- 172.21.15.29:0/775969978 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5fa404d400 con 0x7f5fac1550f0 2024-01-24T13:59:30.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.495+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/775969978 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5fac0c35a0 con 0x7f5f9005f150 2024-01-24T13:59:30.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.506+0000 7f5fa17fa700 1 -- 172.21.15.29:0/775969978 <== mgr.4103 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+354174 (secure 0 0 0) 0x7f5fac0c35a0 con 0x7f5f9005f150 2024-01-24T13:59:30.511 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:30.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.510+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f9005f150 msgr2=0x7f5f90061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.510+0000 7f5fb3a4e700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f9005f150 0x7f5f90061610 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f5fa8001750 tx=0x7f5fa8009040).stop 2024-01-24T13:59:30.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.510+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/775969978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fac1550f0 msgr2=0x7f5fac155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.510+0000 7f5fb3a4e700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fac1550f0 0x7f5fac155850 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f5fac069030 tx=0x7f5fa4009780).stop 2024-01-24T13:59:30.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.510+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/775969978 shutdown_connections 2024-01-24T13:59:30.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.510+0000 7f5fb3a4e700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f9005f150 0x7f5f90061610 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.510+0000 7f5fb3a4e700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fac1550f0 0x7f5fac155850 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.510+0000 7f5fb3a4e700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fac062ec0 0x7f5fac0aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.510+0000 7f5fb3a4e700 1 --2- 172.21.15.29:0/775969978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fac1540a0 0x7f5fac14fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.510+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/775969978 >> 172.21.15.29:0/775969978 conn(0x7f5fac0c85c0 msgr2=0x7f5fac0c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:30.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.511+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/775969978 shutdown_connections 2024-01-24T13:59:30.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.511+0000 7f5fb3a4e700 1 -- 172.21.15.29:0/775969978 wait complete. 2024-01-24T13:59:30.515 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T13:59:30.524 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-01-24T13:59:29.742125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":892,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380176,"kb_used_data":3408,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752594544,"statfs":{"total":773094113280,"available":770656813056,"internally_reserved":0,"allocated":3489792,"data_stored":1556865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6097,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-57,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.550072"},"pg_stats":[{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950914+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.952115+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394881+0000","last_change":"2024-01-24T13:59:26.394881+0000","last_active":"2024-01-24T13:59:26.394881+0000","last_peered":"2024-01-24T13:59:26.394881+0000","last_clean":"2024-01-24T13:59:26.394881+0000","last_became_active":"2024-01-24T13:59:26.394600+0000","last_became_peered":"2024-01-24T13:59:26.394600+0000","last_unstale":"2024-01-24T13:59:26.394881+0000","last_undegraded":"2024-01-24T13:59:26.394881+0000","last_fullsized":"2024-01-24T13:59:26.394881+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.982279+0000","last_change":"2024-01-24T13:59:25.980665+0000","last_active":"2024-01-24T13:59:25.982279+0000","last_peered":"2024-01-24T13:59:25.982279+0000","last_clean":"2024-01-24T13:59:25.982279+0000","last_became_active":"2024-01-24T13:59:25.980288+0000","last_became_peered":"2024-01-24T13:59:25.980288+0000","last_unstale":"2024-01-24T13:59:25.982279+0000","last_undegraded":"2024-01-24T13:59:25.982279+0000","last_fullsized":"2024-01-24T13:59:25.982279+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.947378+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.947325+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949460+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.397603+0000","last_change":"2024-01-24T13:59:26.397603+0000","last_active":"2024-01-24T13:59:26.397603+0000","last_peered":"2024-01-24T13:59:26.397603+0000","last_clean":"2024-01-24T13:59:26.397603+0000","last_became_active":"2024-01-24T13:59:26.397269+0000","last_became_peered":"2024-01-24T13:59:26.397269+0000","last_unstale":"2024-01-24T13:59:26.397603+0000","last_undegraded":"2024-01-24T13:59:26.397603+0000","last_fullsized":"2024-01-24T13:59:26.397603+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959680+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950356+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391986+0000","last_change":"2024-01-24T13:59:26.391986+0000","last_active":"2024-01-24T13:59:26.391986+0000","last_peered":"2024-01-24T13:59:26.391986+0000","last_clean":"2024-01-24T13:59:26.391986+0000","last_became_active":"2024-01-24T13:59:26.391698+0000","last_became_peered":"2024-01-24T13:59:26.391698+0000","last_unstale":"2024-01-24T13:59:26.391986+0000","last_undegraded":"2024-01-24T13:59:26.391986+0000","last_fullsized":"2024-01-24T13:59:26.391986+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.980747+0000","last_change":"2024-01-24T13:59:25.973798+0000","last_active":"2024-01-24T13:59:25.980747+0000","last_peered":"2024-01-24T13:59:25.980747+0000","last_clean":"2024-01-24T13:59:25.980747+0000","last_became_active":"2024-01-24T13:59:25.973506+0000","last_became_peered":"2024-01-24T13:59:25.973506+0000","last_unstale":"2024-01-24T13:59:25.980747+0000","last_undegraded":"2024-01-24T13:59:25.980747+0000","last_fullsized":"2024-01-24T13:59:25.980747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392777+0000","last_change":"2024-01-24T13:59:26.392777+0000","last_active":"2024-01-24T13:59:26.392777+0000","last_peered":"2024-01-24T13:59:26.392777+0000","last_clean":"2024-01-24T13:59:26.392777+0000","last_became_active":"2024-01-24T13:59:26.392496+0000","last_became_peered":"2024-01-24T13:59:26.392496+0000","last_unstale":"2024-01-24T13:59:26.392777+0000","last_undegraded":"2024-01-24T13:59:26.392777+0000","last_fullsized":"2024-01-24T13:59:26.392777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951917+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959171+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394489+0000","last_change":"2024-01-24T13:59:26.394489+0000","last_active":"2024-01-24T13:59:26.394489+0000","last_peered":"2024-01-24T13:59:26.394489+0000","last_clean":"2024-01-24T13:59:26.394489+0000","last_became_active":"2024-01-24T13:59:26.392243+0000","last_became_peered":"2024-01-24T13:59:26.392243+0000","last_unstale":"2024-01-24T13:59:26.394489+0000","last_undegraded":"2024-01-24T13:59:26.394489+0000","last_fullsized":"2024-01-24T13:59:26.394489+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"40'4","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404373+0000","last_change":"2024-01-24T13:59:25.973968+0000","last_active":"2024-01-24T13:59:26.404373+0000","last_peered":"2024-01-24T13:59:26.404373+0000","last_clean":"2024-01-24T13:59:26.404373+0000","last_became_active":"2024-01-24T13:59:25.973675+0000","last_became_peered":"2024-01-24T13:59:25.973675+0000","last_unstale":"2024-01-24T13:59:26.404373+0000","last_undegraded":"2024-01-24T13:59:26.404373+0000","last_fullsized":"2024-01-24T13:59:26.404373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.395811+0000","last_change":"2024-01-24T13:59:26.395811+0000","last_active":"2024-01-24T13:59:26.395811+0000","last_peered":"2024-01-24T13:59:26.395811+0000","last_clean":"2024-01-24T13:59:26.395811+0000","last_became_active":"2024-01-24T13:59:26.395486+0000","last_became_peered":"2024-01-24T13:59:26.395486+0000","last_unstale":"2024-01-24T13:59:26.395811+0000","last_undegraded":"2024-01-24T13:59:26.395811+0000","last_fullsized":"2024-01-24T13:59:26.395811+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959208+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949817+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959318+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959759+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959476+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949550+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946938+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949148+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393452+0000","last_change":"2024-01-24T13:59:26.393452+0000","last_active":"2024-01-24T13:59:26.393452+0000","last_peered":"2024-01-24T13:59:26.393452+0000","last_clean":"2024-01-24T13:59:26.393452+0000","last_became_active":"2024-01-24T13:59:26.393184+0000","last_became_peered":"2024-01-24T13:59:26.393184+0000","last_unstale":"2024-01-24T13:59:26.393452+0000","last_undegraded":"2024-01-24T13:59:26.393452+0000","last_fullsized":"2024-01-24T13:59:26.393452+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.947286+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949431+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958647+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946523+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949817+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949549+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949548+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946704+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.948979+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950028+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958063+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949470+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950354+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950439+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.952133+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959016+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951596+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399449+0000","last_change":"2024-01-24T13:59:26.399449+0000","last_active":"2024-01-24T13:59:26.399449+0000","last_peered":"2024-01-24T13:59:26.399449+0000","last_clean":"2024-01-24T13:59:26.399449+0000","last_became_active":"2024-01-24T13:59:26.399156+0000","last_became_peered":"2024-01-24T13:59:26.399156+0000","last_unstale":"2024-01-24T13:59:26.399449+0000","last_undegraded":"2024-01-24T13:59:26.399449+0000","last_fullsized":"2024-01-24T13:59:26.399449+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949041+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.397561+0000","last_change":"2024-01-24T13:59:26.397561+0000","last_active":"2024-01-24T13:59:26.397561+0000","last_peered":"2024-01-24T13:59:26.397561+0000","last_clean":"2024-01-24T13:59:26.397561+0000","last_became_active":"2024-01-24T13:59:26.397284+0000","last_became_peered":"2024-01-24T13:59:26.397284+0000","last_unstale":"2024-01-24T13:59:26.397561+0000","last_undegraded":"2024-01-24T13:59:26.397561+0000","last_fullsized":"2024-01-24T13:59:26.397561+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946640+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.982939+0000","last_change":"2024-01-24T13:59:25.981443+0000","last_active":"2024-01-24T13:59:25.982939+0000","last_peered":"2024-01-24T13:59:25.982939+0000","last_clean":"2024-01-24T13:59:25.982939+0000","last_became_active":"2024-01-24T13:59:25.980669+0000","last_became_peered":"2024-01-24T13:59:25.980669+0000","last_unstale":"2024-01-24T13:59:25.982939+0000","last_undegraded":"2024-01-24T13:59:25.982939+0000","last_fullsized":"2024-01-24T13:59:25.982939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949102+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.981798+0000","last_change":"2024-01-24T13:59:25.980155+0000","last_active":"2024-01-24T13:59:25.981798+0000","last_peered":"2024-01-24T13:59:25.981798+0000","last_clean":"2024-01-24T13:59:25.981798+0000","last_became_active":"2024-01-24T13:59:25.979649+0000","last_became_peered":"2024-01-24T13:59:25.979649+0000","last_unstale":"2024-01-24T13:59:25.981798+0000","last_undegraded":"2024-01-24T13:59:25.981798+0000","last_fullsized":"2024-01-24T13:59:25.981798+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958506+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946408+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951054+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.395796+0000","last_change":"2024-01-24T13:59:26.395796+0000","last_active":"2024-01-24T13:59:26.395796+0000","last_peered":"2024-01-24T13:59:26.395796+0000","last_clean":"2024-01-24T13:59:26.395796+0000","last_became_active":"2024-01-24T13:59:26.395521+0000","last_became_peered":"2024-01-24T13:59:26.395521+0000","last_unstale":"2024-01-24T13:59:26.395796+0000","last_undegraded":"2024-01-24T13:59:26.395796+0000","last_fullsized":"2024-01-24T13:59:26.395796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958118+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950823+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"40'4","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401238+0000","last_change":"2024-01-24T13:59:25.981003+0000","last_active":"2024-01-24T13:59:26.401238+0000","last_peered":"2024-01-24T13:59:26.401238+0000","last_clean":"2024-01-24T13:59:26.401238+0000","last_became_active":"2024-01-24T13:59:25.980309+0000","last_became_peered":"2024-01-24T13:59:25.980309+0000","last_unstale":"2024-01-24T13:59:26.401238+0000","last_undegraded":"2024-01-24T13:59:26.401238+0000","last_fullsized":"2024-01-24T13:59:26.401238+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.947411+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392588+0000","last_change":"2024-01-24T13:59:26.392588+0000","last_active":"2024-01-24T13:59:26.392588+0000","last_peered":"2024-01-24T13:59:26.392588+0000","last_clean":"2024-01-24T13:59:26.392588+0000","last_became_active":"2024-01-24T13:59:26.392285+0000","last_became_peered":"2024-01-24T13:59:26.392285+0000","last_unstale":"2024-01-24T13:59:26.392588+0000","last_undegraded":"2024-01-24T13:59:26.392588+0000","last_fullsized":"2024-01-24T13:59:26.392588+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.948612+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946600+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951306+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950508+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946829+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.981204+0000","last_change":"2024-01-24T13:59:25.980225+0000","last_active":"2024-01-24T13:59:25.981204+0000","last_peered":"2024-01-24T13:59:25.981204+0000","last_clean":"2024-01-24T13:59:25.981204+0000","last_became_active":"2024-01-24T13:59:25.979876+0000","last_became_peered":"2024-01-24T13:59:25.979876+0000","last_unstale":"2024-01-24T13:59:25.981204+0000","last_undegraded":"2024-01-24T13:59:25.981204+0000","last_fullsized":"2024-01-24T13:59:25.981204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392651+0000","last_change":"2024-01-24T13:59:26.392651+0000","last_active":"2024-01-24T13:59:26.392651+0000","last_peered":"2024-01-24T13:59:26.392651+0000","last_clean":"2024-01-24T13:59:26.392651+0000","last_became_active":"2024-01-24T13:59:26.392338+0000","last_became_peered":"2024-01-24T13:59:26.392338+0000","last_unstale":"2024-01-24T13:59:26.392651+0000","last_undegraded":"2024-01-24T13:59:26.392651+0000","last_fullsized":"2024-01-24T13:59:26.392651+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958226+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958985+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397561+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T13:59:21.397561+0000","last_peered":"2024-01-24T13:59:21.397561+0000","last_clean":"2024-01-24T13:59:21.397561+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T13:59:21.397561+0000","last_undegraded":"2024-01-24T13:59:21.397561+0000","last_fullsized":"2024-01-24T13:59:21.397561+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.948578+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397492+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T13:59:21.397492+0000","last_peered":"2024-01-24T13:59:21.397492+0000","last_clean":"2024-01-24T13:59:21.397492+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T13:59:21.397492+0000","last_undegraded":"2024-01-24T13:59:21.397492+0000","last_fullsized":"2024-01-24T13:59:21.397492+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949583+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397705+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T13:59:21.397705+0000","last_peered":"2024-01-24T13:59:21.397705+0000","last_clean":"2024-01-24T13:59:21.397705+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T13:59:21.397705+0000","last_undegraded":"2024-01-24T13:59:21.397705+0000","last_fullsized":"2024-01-24T13:59:21.397705+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950081+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397709+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T13:59:21.397709+0000","last_peered":"2024-01-24T13:59:21.397709+0000","last_clean":"2024-01-24T13:59:21.397709+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T13:59:21.397709+0000","last_undegraded":"2024-01-24T13:59:21.397709+0000","last_fullsized":"2024-01-24T13:59:21.397709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394231+0000","last_change":"2024-01-24T13:59:26.394231+0000","last_active":"2024-01-24T13:59:26.394231+0000","last_peered":"2024-01-24T13:59:26.394231+0000","last_clean":"2024-01-24T13:59:26.394231+0000","last_became_active":"2024-01-24T13:59:26.393918+0000","last_became_peered":"2024-01-24T13:59:26.393918+0000","last_unstale":"2024-01-24T13:59:26.394231+0000","last_undegraded":"2024-01-24T13:59:26.394231+0000","last_fullsized":"2024-01-24T13:59:26.394231+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.952375+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397711+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T13:59:21.397711+0000","last_peered":"2024-01-24T13:59:21.397711+0000","last_clean":"2024-01-24T13:59:21.397711+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T13:59:21.397711+0000","last_undegraded":"2024-01-24T13:59:21.397711+0000","last_fullsized":"2024-01-24T13:59:21.397711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:20.006269+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T13:59:20.006269+0000","last_peered":"2024-01-24T13:59:20.006269+0000","last_clean":"2024-01-24T13:59:20.006269+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T13:59:20.006269+0000","last_undegraded":"2024-01-24T13:59:20.006269+0000","last_fullsized":"2024-01-24T13:59:20.006269+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.388213+0000","last_change":"2024-01-24T13:59:26.388213+0000","last_active":"2024-01-24T13:59:26.388213+0000","last_peered":"2024-01-24T13:59:26.388213+0000","last_clean":"2024-01-24T13:59:26.388213+0000","last_became_active":"2024-01-24T13:59:26.387917+0000","last_became_peered":"2024-01-24T13:59:26.387917+0000","last_unstale":"2024-01-24T13:59:26.388213+0000","last_undegraded":"2024-01-24T13:59:26.388213+0000","last_fullsized":"2024-01-24T13:59:26.388213+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958091+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:20.005875+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T13:59:20.005875+0000","last_peered":"2024-01-24T13:59:20.005875+0000","last_clean":"2024-01-24T13:59:20.005875+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T13:59:20.005875+0000","last_undegraded":"2024-01-24T13:59:20.005875+0000","last_fullsized":"2024-01-24T13:59:20.005875+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951679+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:20.006269+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T13:59:20.006269+0000","last_peered":"2024-01-24T13:59:20.006269+0000","last_clean":"2024-01-24T13:59:20.006269+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T13:59:20.006269+0000","last_undegraded":"2024-01-24T13:59:20.006269+0000","last_fullsized":"2024-01-24T13:59:20.006269+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.983216+0000","last_change":"2024-01-24T13:59:25.982352+0000","last_active":"2024-01-24T13:59:25.983216+0000","last_peered":"2024-01-24T13:59:25.983216+0000","last_clean":"2024-01-24T13:59:25.983216+0000","last_became_active":"2024-01-24T13:59:25.981960+0000","last_became_peered":"2024-01-24T13:59:25.981960+0000","last_unstale":"2024-01-24T13:59:25.983216+0000","last_undegraded":"2024-01-24T13:59:25.983216+0000","last_fullsized":"2024-01-24T13:59:25.983216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397468+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T13:59:21.397468+0000","last_peered":"2024-01-24T13:59:21.397468+0000","last_clean":"2024-01-24T13:59:21.397468+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T13:59:21.397468+0000","last_undegraded":"2024-01-24T13:59:21.397468+0000","last_fullsized":"2024-01-24T13:59:21.397468+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":1},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":892,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":434176,"data_stored":194589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.31900000000000001}]},{"osd":2,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.377}]},{"osd":3,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.63300000000000001}]},{"osd":5,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.747}]},{"osd":7,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.77200000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52200000000000002}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.622}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42999999999999999}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46500000000000002}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.499}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.43099999999999999}]},{"osd":1,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.501},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55600000000000005}]},{"osd":5,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":425984,"data_stored":188976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.311},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.374}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.41499999999999998}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.623},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55300000000000005}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54300000000000004}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.59899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.498}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.38},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.51800000000000002}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.435}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.63500000000000001}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65800000000000003}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59099999999999997}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.623}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.56999999999999995}]},{"osd":1,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.63},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53400000000000003}]},{"osd":3,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.58499999999999996}]},{"osd":5,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.41399999999999998}]},{"osd":6,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.48199999999999998}]},{"osd":7,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.503},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.51800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.47599999999999998}]},{"osd":3,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.44}]},{"osd":6,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.502}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":1.3859999999999999}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":1.353},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.81200000000000006}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":6.8129999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":1.4079999999999999}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":7.016},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.85099999999999998}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":6.8719999999999999}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":6.976},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":7.8789999999999996}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":6.9080000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T13:59:30.530 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T13:59:30.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.699+0000 7f17fa8eb700 1 Processor -- start 2024-01-24T13:59:30.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.699+0000 7f17fa8eb700 1 -- start start 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17fa8eb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17f41540a0 0x7f17f40c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17fa8eb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17f41550f0 0x7f17f40627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17fa8eb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f17f4062ca0 0x7f17f4157c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17fa8eb700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f40a74f0 con 0x7f17f41540a0 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17fa8eb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f4065360 con 0x7f17f4062ca0 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17fa8eb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f40a6ce0 con 0x7f17f41550f0 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17f3fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17f41540a0 0x7f17f40c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17f3fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17f41540a0 0x7f17f40c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39624/0 (socket says 172.21.15.29:39624) 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17f3fff700 1 -- 172.21.15.29:0/3428335136 learned_addr learned my addr 172.21.15.29:0/3428335136 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17f37fe700 1 --2- 172.21.15.29:0/3428335136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17f41550f0 0x7f17f40627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17f8e88700 1 --2- 172.21.15.29:0/3428335136 >> v2:172.21.15.163:3300/0 conn(0x7f17f4062ca0 0x7f17f4157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17f3fff700 1 -- 172.21.15.29:0/3428335136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17f41550f0 msgr2=0x7f17f40627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17f3fff700 1 --2- 172.21.15.29:0/3428335136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17f41550f0 0x7f17f40627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17f3fff700 1 -- 172.21.15.29:0/3428335136 >> v2:172.21.15.163:3300/0 conn(0x7f17f4062ca0 msgr2=0x7f17f4157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17f3fff700 1 --2- 172.21.15.29:0/3428335136 >> v2:172.21.15.163:3300/0 conn(0x7f17f4062ca0 0x7f17f4157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17f3fff700 1 -- 172.21.15.29:0/3428335136 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f40636f0 con 0x7f17f41540a0 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.700+0000 7f17f3fff700 1 --2- 172.21.15.29:0/3428335136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f41540a0 0x7f17f40c4ca0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f17f4066b80 tx=0x7f17e40090f0).ready entity=mon.0 client_cookie=c6e2948bef85f0f7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.701+0000 7f17f2ffd700 1 -- 172.21.15.29:0/3428335136 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e400f510 con 0x7f17f41540a0 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.701+0000 7f17f2ffd700 1 -- 172.21.15.29:0/3428335136 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e400f670 con 0x7f17f41540a0 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.701+0000 7f17f2ffd700 1 -- 172.21.15.29:0/3428335136 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e400fa10 con 0x7f17f41540a0 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.701+0000 7f17fa8eb700 1 -- 172.21.15.29:0/3428335136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f41540a0 msgr2=0x7f17f40c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.701+0000 7f17fa8eb700 1 --2- 172.21.15.29:0/3428335136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f41540a0 0x7f17f40c4ca0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f17f4066b80 tx=0x7f17e40090f0).stop 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.701+0000 7f17fa8eb700 1 -- 172.21.15.29:0/3428335136 shutdown_connections 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.701+0000 7f17fa8eb700 1 --2- 172.21.15.29:0/3428335136 >> v2:172.21.15.163:3300/0 conn(0x7f17f4062ca0 0x7f17f4157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.701+0000 7f17fa8eb700 1 --2- 172.21.15.29:0/3428335136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f41540a0 0x7f17f40c4ca0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.701+0000 7f17fa8eb700 1 --2- 172.21.15.29:0/3428335136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17f41550f0 0x7f17f40627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.701+0000 7f17fa8eb700 1 -- 172.21.15.29:0/3428335136 >> 172.21.15.29:0/3428335136 conn(0x7f17f40c85c0 msgr2=0x7f17f40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.701+0000 7f17fa8eb700 1 -- 172.21.15.29:0/3428335136 shutdown_connections 2024-01-24T13:59:30.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.701+0000 7f17fa8eb700 1 -- 172.21.15.29:0/3428335136 wait complete. 2024-01-24T13:59:30.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.702+0000 7f17fa8eb700 1 Processor -- start 2024-01-24T13:59:30.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.703+0000 7f17fa8eb700 1 -- start start 2024-01-24T13:59:30.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.703+0000 7f17fa8eb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f4062ca0 0x7f17f40a8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.703+0000 7f17fa8eb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17f41540a0 0x7f17f41464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.703+0000 7f17fa8eb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17f41550f0 0x7f17f40a9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.703+0000 7f17fa8eb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f414ec50 con 0x7f17f4062ca0 2024-01-24T13:59:30.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.703+0000 7f17fa8eb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f414ff70 con 0x7f17f41540a0 2024-01-24T13:59:30.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.703+0000 7f17fa8eb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f40a89f0 con 0x7f17f41550f0 2024-01-24T13:59:30.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f4062ca0 0x7f17f40a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f4062ca0 0x7f17f40a8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39636/0 (socket says 172.21.15.29:39636) 2024-01-24T13:59:30.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f3fff700 1 -- 172.21.15.29:0/15916658 learned_addr learned my addr 172.21.15.29:0/15916658 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:30.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f8e88700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17f41550f0 0x7f17f40a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f37fe700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17f41540a0 0x7f17f41464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f3fff700 1 -- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17f41550f0 msgr2=0x7f17f40a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f3fff700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17f41550f0 0x7f17f40a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f3fff700 1 -- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17f41540a0 msgr2=0x7f17f41464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f3fff700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17f41540a0 0x7f17f41464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f3fff700 1 -- 172.21.15.29:0/15916658 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e4009370 con 0x7f17f4062ca0 2024-01-24T13:59:30.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f37fe700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17f41540a0 0x7f17f41464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:30.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f8e88700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17f41550f0 0x7f17f40a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:30.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17f3fff700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f4062ca0 0x7f17f40a8500 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f17e4006fd0 tx=0x7f17e4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:30.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17dbfff700 1 -- 172.21.15.29:0/15916658 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e400fea0 con 0x7f17f4062ca0 2024-01-24T13:59:30.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17dbfff700 1 -- 172.21.15.29:0/15916658 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e4000bc0 con 0x7f17f4062ca0 2024-01-24T13:59:30.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17dbfff700 1 -- 172.21.15.29:0/15916658 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e4024470 con 0x7f17f4062ca0 2024-01-24T13:59:30.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17fa8eb700 1 -- 172.21.15.29:0/15916658 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f40a8c50 con 0x7f17f4062ca0 2024-01-24T13:59:30.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.704+0000 7f17fa8eb700 1 -- 172.21.15.29:0/15916658 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f41501c0 con 0x7f17f4062ca0 2024-01-24T13:59:30.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.705+0000 7f17fa8eb700 1 -- 172.21.15.29:0/15916658 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f40475e0 con 0x7f17f4062ca0 2024-01-24T13:59:30.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.706+0000 7f17dbfff700 1 -- 172.21.15.29:0/15916658 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f17e4021070 con 0x7f17f4062ca0 2024-01-24T13:59:30.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.706+0000 7f17dbfff700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f17e005f150 0x7f17e0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:30.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.706+0000 7f17dbfff700 1 -- 172.21.15.29:0/15916658 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f17e407bd00 con 0x7f17f4062ca0 2024-01-24T13:59:30.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.706+0000 7f17f37fe700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f17e005f150 0x7f17e0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:30.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.707+0000 7f17f37fe700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f17e005f150 0x7f17e0061610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f17f4167360 tx=0x7f17e8008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:30.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.710+0000 7f17dbfff700 1 -- 172.21.15.29:0/15916658 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f17e404ce70 con 0x7f17f4062ca0 2024-01-24T13:59:30.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.888+0000 7f17fa8eb700 1 -- 172.21.15.29:0/15916658 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f17f4150810 con 0x7f17e005f150 2024-01-24T13:59:30.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.899+0000 7f17dbfff700 1 -- 172.21.15.29:0/15916658 <== mgr.4103 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+354174 (secure 0 0 0) 0x7f17f4150810 con 0x7f17e005f150 2024-01-24T13:59:30.904 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:30.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.903+0000 7f17fa8eb700 1 -- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f17e005f150 msgr2=0x7f17e0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.903+0000 7f17fa8eb700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f17e005f150 0x7f17e0061610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f17f4167360 tx=0x7f17e8008040).stop 2024-01-24T13:59:30.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.903+0000 7f17fa8eb700 1 -- 172.21.15.29:0/15916658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f4062ca0 msgr2=0x7f17f40a8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:30.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.903+0000 7f17fa8eb700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f4062ca0 0x7f17f40a8500 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f17e4006fd0 tx=0x7f17e4017000).stop 2024-01-24T13:59:30.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.903+0000 7f17fa8eb700 1 -- 172.21.15.29:0/15916658 shutdown_connections 2024-01-24T13:59:30.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.903+0000 7f17fa8eb700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17f41550f0 0x7f17f40a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.903+0000 7f17fa8eb700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f17e005f150 0x7f17e0061610 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.903+0000 7f17fa8eb700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17f4062ca0 0x7f17f40a8500 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.903+0000 7f17fa8eb700 1 --2- 172.21.15.29:0/15916658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17f41540a0 0x7f17f41464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:30.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.903+0000 7f17fa8eb700 1 -- 172.21.15.29:0/15916658 >> 172.21.15.29:0/15916658 conn(0x7f17f40c85c0 msgr2=0x7f17f40c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:30.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.903+0000 7f17fa8eb700 1 -- 172.21.15.29:0/15916658 shutdown_connections 2024-01-24T13:59:30.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:30.903+0000 7f17fa8eb700 1 -- 172.21.15.29:0/15916658 wait complete. 2024-01-24T13:59:30.907 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T13:59:30.916 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-01-24T13:59:29.742125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":892,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380176,"kb_used_data":3408,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752594544,"statfs":{"total":773094113280,"available":770656813056,"internally_reserved":0,"allocated":3489792,"data_stored":1556865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6097,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-57,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.550072"},"pg_stats":[{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950914+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.952115+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394881+0000","last_change":"2024-01-24T13:59:26.394881+0000","last_active":"2024-01-24T13:59:26.394881+0000","last_peered":"2024-01-24T13:59:26.394881+0000","last_clean":"2024-01-24T13:59:26.394881+0000","last_became_active":"2024-01-24T13:59:26.394600+0000","last_became_peered":"2024-01-24T13:59:26.394600+0000","last_unstale":"2024-01-24T13:59:26.394881+0000","last_undegraded":"2024-01-24T13:59:26.394881+0000","last_fullsized":"2024-01-24T13:59:26.394881+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.982279+0000","last_change":"2024-01-24T13:59:25.980665+0000","last_active":"2024-01-24T13:59:25.982279+0000","last_peered":"2024-01-24T13:59:25.982279+0000","last_clean":"2024-01-24T13:59:25.982279+0000","last_became_active":"2024-01-24T13:59:25.980288+0000","last_became_peered":"2024-01-24T13:59:25.980288+0000","last_unstale":"2024-01-24T13:59:25.982279+0000","last_undegraded":"2024-01-24T13:59:25.982279+0000","last_fullsized":"2024-01-24T13:59:25.982279+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.947378+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.947325+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949460+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.397603+0000","last_change":"2024-01-24T13:59:26.397603+0000","last_active":"2024-01-24T13:59:26.397603+0000","last_peered":"2024-01-24T13:59:26.397603+0000","last_clean":"2024-01-24T13:59:26.397603+0000","last_became_active":"2024-01-24T13:59:26.397269+0000","last_became_peered":"2024-01-24T13:59:26.397269+0000","last_unstale":"2024-01-24T13:59:26.397603+0000","last_undegraded":"2024-01-24T13:59:26.397603+0000","last_fullsized":"2024-01-24T13:59:26.397603+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959680+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950356+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391986+0000","last_change":"2024-01-24T13:59:26.391986+0000","last_active":"2024-01-24T13:59:26.391986+0000","last_peered":"2024-01-24T13:59:26.391986+0000","last_clean":"2024-01-24T13:59:26.391986+0000","last_became_active":"2024-01-24T13:59:26.391698+0000","last_became_peered":"2024-01-24T13:59:26.391698+0000","last_unstale":"2024-01-24T13:59:26.391986+0000","last_undegraded":"2024-01-24T13:59:26.391986+0000","last_fullsized":"2024-01-24T13:59:26.391986+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.980747+0000","last_change":"2024-01-24T13:59:25.973798+0000","last_active":"2024-01-24T13:59:25.980747+0000","last_peered":"2024-01-24T13:59:25.980747+0000","last_clean":"2024-01-24T13:59:25.980747+0000","last_became_active":"2024-01-24T13:59:25.973506+0000","last_became_peered":"2024-01-24T13:59:25.973506+0000","last_unstale":"2024-01-24T13:59:25.980747+0000","last_undegraded":"2024-01-24T13:59:25.980747+0000","last_fullsized":"2024-01-24T13:59:25.980747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392777+0000","last_change":"2024-01-24T13:59:26.392777+0000","last_active":"2024-01-24T13:59:26.392777+0000","last_peered":"2024-01-24T13:59:26.392777+0000","last_clean":"2024-01-24T13:59:26.392777+0000","last_became_active":"2024-01-24T13:59:26.392496+0000","last_became_peered":"2024-01-24T13:59:26.392496+0000","last_unstale":"2024-01-24T13:59:26.392777+0000","last_undegraded":"2024-01-24T13:59:26.392777+0000","last_fullsized":"2024-01-24T13:59:26.392777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951917+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959171+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394489+0000","last_change":"2024-01-24T13:59:26.394489+0000","last_active":"2024-01-24T13:59:26.394489+0000","last_peered":"2024-01-24T13:59:26.394489+0000","last_clean":"2024-01-24T13:59:26.394489+0000","last_became_active":"2024-01-24T13:59:26.392243+0000","last_became_peered":"2024-01-24T13:59:26.392243+0000","last_unstale":"2024-01-24T13:59:26.394489+0000","last_undegraded":"2024-01-24T13:59:26.394489+0000","last_fullsized":"2024-01-24T13:59:26.394489+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"40'4","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404373+0000","last_change":"2024-01-24T13:59:25.973968+0000","last_active":"2024-01-24T13:59:26.404373+0000","last_peered":"2024-01-24T13:59:26.404373+0000","last_clean":"2024-01-24T13:59:26.404373+0000","last_became_active":"2024-01-24T13:59:25.973675+0000","last_became_peered":"2024-01-24T13:59:25.973675+0000","last_unstale":"2024-01-24T13:59:26.404373+0000","last_undegraded":"2024-01-24T13:59:26.404373+0000","last_fullsized":"2024-01-24T13:59:26.404373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.395811+0000","last_change":"2024-01-24T13:59:26.395811+0000","last_active":"2024-01-24T13:59:26.395811+0000","last_peered":"2024-01-24T13:59:26.395811+0000","last_clean":"2024-01-24T13:59:26.395811+0000","last_became_active":"2024-01-24T13:59:26.395486+0000","last_became_peered":"2024-01-24T13:59:26.395486+0000","last_unstale":"2024-01-24T13:59:26.395811+0000","last_undegraded":"2024-01-24T13:59:26.395811+0000","last_fullsized":"2024-01-24T13:59:26.395811+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959208+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949817+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959318+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959759+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959476+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949550+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946938+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949148+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393452+0000","last_change":"2024-01-24T13:59:26.393452+0000","last_active":"2024-01-24T13:59:26.393452+0000","last_peered":"2024-01-24T13:59:26.393452+0000","last_clean":"2024-01-24T13:59:26.393452+0000","last_became_active":"2024-01-24T13:59:26.393184+0000","last_became_peered":"2024-01-24T13:59:26.393184+0000","last_unstale":"2024-01-24T13:59:26.393452+0000","last_undegraded":"2024-01-24T13:59:26.393452+0000","last_fullsized":"2024-01-24T13:59:26.393452+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.947286+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949431+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958647+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946523+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949817+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949549+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949548+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946704+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.948979+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950028+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958063+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949470+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950354+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950439+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.952133+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959016+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951596+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399449+0000","last_change":"2024-01-24T13:59:26.399449+0000","last_active":"2024-01-24T13:59:26.399449+0000","last_peered":"2024-01-24T13:59:26.399449+0000","last_clean":"2024-01-24T13:59:26.399449+0000","last_became_active":"2024-01-24T13:59:26.399156+0000","last_became_peered":"2024-01-24T13:59:26.399156+0000","last_unstale":"2024-01-24T13:59:26.399449+0000","last_undegraded":"2024-01-24T13:59:26.399449+0000","last_fullsized":"2024-01-24T13:59:26.399449+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949041+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.397561+0000","last_change":"2024-01-24T13:59:26.397561+0000","last_active":"2024-01-24T13:59:26.397561+0000","last_peered":"2024-01-24T13:59:26.397561+0000","last_clean":"2024-01-24T13:59:26.397561+0000","last_became_active":"2024-01-24T13:59:26.397284+0000","last_became_peered":"2024-01-24T13:59:26.397284+0000","last_unstale":"2024-01-24T13:59:26.397561+0000","last_undegraded":"2024-01-24T13:59:26.397561+0000","last_fullsized":"2024-01-24T13:59:26.397561+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946640+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.982939+0000","last_change":"2024-01-24T13:59:25.981443+0000","last_active":"2024-01-24T13:59:25.982939+0000","last_peered":"2024-01-24T13:59:25.982939+0000","last_clean":"2024-01-24T13:59:25.982939+0000","last_became_active":"2024-01-24T13:59:25.980669+0000","last_became_peered":"2024-01-24T13:59:25.980669+0000","last_unstale":"2024-01-24T13:59:25.982939+0000","last_undegraded":"2024-01-24T13:59:25.982939+0000","last_fullsized":"2024-01-24T13:59:25.982939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949102+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.981798+0000","last_change":"2024-01-24T13:59:25.980155+0000","last_active":"2024-01-24T13:59:25.981798+0000","last_peered":"2024-01-24T13:59:25.981798+0000","last_clean":"2024-01-24T13:59:25.981798+0000","last_became_active":"2024-01-24T13:59:25.979649+0000","last_became_peered":"2024-01-24T13:59:25.979649+0000","last_unstale":"2024-01-24T13:59:25.981798+0000","last_undegraded":"2024-01-24T13:59:25.981798+0000","last_fullsized":"2024-01-24T13:59:25.981798+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958506+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946408+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951054+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.395796+0000","last_change":"2024-01-24T13:59:26.395796+0000","last_active":"2024-01-24T13:59:26.395796+0000","last_peered":"2024-01-24T13:59:26.395796+0000","last_clean":"2024-01-24T13:59:26.395796+0000","last_became_active":"2024-01-24T13:59:26.395521+0000","last_became_peered":"2024-01-24T13:59:26.395521+0000","last_unstale":"2024-01-24T13:59:26.395796+0000","last_undegraded":"2024-01-24T13:59:26.395796+0000","last_fullsized":"2024-01-24T13:59:26.395796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958118+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950823+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"40'4","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401238+0000","last_change":"2024-01-24T13:59:25.981003+0000","last_active":"2024-01-24T13:59:26.401238+0000","last_peered":"2024-01-24T13:59:26.401238+0000","last_clean":"2024-01-24T13:59:26.401238+0000","last_became_active":"2024-01-24T13:59:25.980309+0000","last_became_peered":"2024-01-24T13:59:25.980309+0000","last_unstale":"2024-01-24T13:59:26.401238+0000","last_undegraded":"2024-01-24T13:59:26.401238+0000","last_fullsized":"2024-01-24T13:59:26.401238+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.947411+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392588+0000","last_change":"2024-01-24T13:59:26.392588+0000","last_active":"2024-01-24T13:59:26.392588+0000","last_peered":"2024-01-24T13:59:26.392588+0000","last_clean":"2024-01-24T13:59:26.392588+0000","last_became_active":"2024-01-24T13:59:26.392285+0000","last_became_peered":"2024-01-24T13:59:26.392285+0000","last_unstale":"2024-01-24T13:59:26.392588+0000","last_undegraded":"2024-01-24T13:59:26.392588+0000","last_fullsized":"2024-01-24T13:59:26.392588+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.948612+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946600+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951306+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950508+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946829+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.981204+0000","last_change":"2024-01-24T13:59:25.980225+0000","last_active":"2024-01-24T13:59:25.981204+0000","last_peered":"2024-01-24T13:59:25.981204+0000","last_clean":"2024-01-24T13:59:25.981204+0000","last_became_active":"2024-01-24T13:59:25.979876+0000","last_became_peered":"2024-01-24T13:59:25.979876+0000","last_unstale":"2024-01-24T13:59:25.981204+0000","last_undegraded":"2024-01-24T13:59:25.981204+0000","last_fullsized":"2024-01-24T13:59:25.981204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392651+0000","last_change":"2024-01-24T13:59:26.392651+0000","last_active":"2024-01-24T13:59:26.392651+0000","last_peered":"2024-01-24T13:59:26.392651+0000","last_clean":"2024-01-24T13:59:26.392651+0000","last_became_active":"2024-01-24T13:59:26.392338+0000","last_became_peered":"2024-01-24T13:59:26.392338+0000","last_unstale":"2024-01-24T13:59:26.392651+0000","last_undegraded":"2024-01-24T13:59:26.392651+0000","last_fullsized":"2024-01-24T13:59:26.392651+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958226+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958985+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397561+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T13:59:21.397561+0000","last_peered":"2024-01-24T13:59:21.397561+0000","last_clean":"2024-01-24T13:59:21.397561+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T13:59:21.397561+0000","last_undegraded":"2024-01-24T13:59:21.397561+0000","last_fullsized":"2024-01-24T13:59:21.397561+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.948578+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397492+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T13:59:21.397492+0000","last_peered":"2024-01-24T13:59:21.397492+0000","last_clean":"2024-01-24T13:59:21.397492+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T13:59:21.397492+0000","last_undegraded":"2024-01-24T13:59:21.397492+0000","last_fullsized":"2024-01-24T13:59:21.397492+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949583+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397705+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T13:59:21.397705+0000","last_peered":"2024-01-24T13:59:21.397705+0000","last_clean":"2024-01-24T13:59:21.397705+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T13:59:21.397705+0000","last_undegraded":"2024-01-24T13:59:21.397705+0000","last_fullsized":"2024-01-24T13:59:21.397705+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950081+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397709+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T13:59:21.397709+0000","last_peered":"2024-01-24T13:59:21.397709+0000","last_clean":"2024-01-24T13:59:21.397709+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T13:59:21.397709+0000","last_undegraded":"2024-01-24T13:59:21.397709+0000","last_fullsized":"2024-01-24T13:59:21.397709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394231+0000","last_change":"2024-01-24T13:59:26.394231+0000","last_active":"2024-01-24T13:59:26.394231+0000","last_peered":"2024-01-24T13:59:26.394231+0000","last_clean":"2024-01-24T13:59:26.394231+0000","last_became_active":"2024-01-24T13:59:26.393918+0000","last_became_peered":"2024-01-24T13:59:26.393918+0000","last_unstale":"2024-01-24T13:59:26.394231+0000","last_undegraded":"2024-01-24T13:59:26.394231+0000","last_fullsized":"2024-01-24T13:59:26.394231+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.952375+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397711+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T13:59:21.397711+0000","last_peered":"2024-01-24T13:59:21.397711+0000","last_clean":"2024-01-24T13:59:21.397711+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T13:59:21.397711+0000","last_undegraded":"2024-01-24T13:59:21.397711+0000","last_fullsized":"2024-01-24T13:59:21.397711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:20.006269+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T13:59:20.006269+0000","last_peered":"2024-01-24T13:59:20.006269+0000","last_clean":"2024-01-24T13:59:20.006269+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T13:59:20.006269+0000","last_undegraded":"2024-01-24T13:59:20.006269+0000","last_fullsized":"2024-01-24T13:59:20.006269+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.388213+0000","last_change":"2024-01-24T13:59:26.388213+0000","last_active":"2024-01-24T13:59:26.388213+0000","last_peered":"2024-01-24T13:59:26.388213+0000","last_clean":"2024-01-24T13:59:26.388213+0000","last_became_active":"2024-01-24T13:59:26.387917+0000","last_became_peered":"2024-01-24T13:59:26.387917+0000","last_unstale":"2024-01-24T13:59:26.388213+0000","last_undegraded":"2024-01-24T13:59:26.388213+0000","last_fullsized":"2024-01-24T13:59:26.388213+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958091+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:20.005875+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T13:59:20.005875+0000","last_peered":"2024-01-24T13:59:20.005875+0000","last_clean":"2024-01-24T13:59:20.005875+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T13:59:20.005875+0000","last_undegraded":"2024-01-24T13:59:20.005875+0000","last_fullsized":"2024-01-24T13:59:20.005875+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951679+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:20.006269+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T13:59:20.006269+0000","last_peered":"2024-01-24T13:59:20.006269+0000","last_clean":"2024-01-24T13:59:20.006269+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T13:59:20.006269+0000","last_undegraded":"2024-01-24T13:59:20.006269+0000","last_fullsized":"2024-01-24T13:59:20.006269+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.983216+0000","last_change":"2024-01-24T13:59:25.982352+0000","last_active":"2024-01-24T13:59:25.983216+0000","last_peered":"2024-01-24T13:59:25.983216+0000","last_clean":"2024-01-24T13:59:25.983216+0000","last_became_active":"2024-01-24T13:59:25.981960+0000","last_became_peered":"2024-01-24T13:59:25.981960+0000","last_unstale":"2024-01-24T13:59:25.983216+0000","last_undegraded":"2024-01-24T13:59:25.983216+0000","last_fullsized":"2024-01-24T13:59:25.983216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397468+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T13:59:21.397468+0000","last_peered":"2024-01-24T13:59:21.397468+0000","last_clean":"2024-01-24T13:59:21.397468+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T13:59:21.397468+0000","last_undegraded":"2024-01-24T13:59:21.397468+0000","last_fullsized":"2024-01-24T13:59:21.397468+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":1},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":892,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":434176,"data_stored":194589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.31900000000000001}]},{"osd":2,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.377}]},{"osd":3,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.63300000000000001}]},{"osd":5,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.747}]},{"osd":7,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.77200000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52200000000000002}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.622}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42999999999999999}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46500000000000002}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.499}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.43099999999999999}]},{"osd":1,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.501},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55600000000000005}]},{"osd":5,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":425984,"data_stored":188976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.311},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.374}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.41499999999999998}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.623},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55300000000000005}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54300000000000004}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.59899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.498}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.38},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.51800000000000002}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.435}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.63500000000000001}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65800000000000003}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59099999999999997}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.623}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.56999999999999995}]},{"osd":1,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.63},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53400000000000003}]},{"osd":3,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.58499999999999996}]},{"osd":5,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.41399999999999998}]},{"osd":6,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.48199999999999998}]},{"osd":7,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.503},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.51800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.47599999999999998}]},{"osd":3,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.44}]},{"osd":6,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.502}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":1.3859999999999999}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":1.353},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.81200000000000006}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":6.8129999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":1.4079999999999999}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":7.016},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.85099999999999998}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":6.8719999999999999}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":6.976},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":7.8789999999999996}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":6.9080000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T13:59:30.922 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T13:59:31.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.089+0000 7f354decd700 1 Processor -- start 2024-01-24T13:59:31.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.089+0000 7f354decd700 1 -- start start 2024-01-24T13:59:31.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.090+0000 7f354decd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3548138330 0x7f3548138710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.090+0000 7f354decd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35480627b0 0x7f3548062c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.090+0000 7f354decd700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3548065770 con 0x7f35480bca50 2024-01-24T13:59:31.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.090+0000 7f354decd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3548063130 con 0x7f3548138330 2024-01-24T13:59:31.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.090+0000 7f354decd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3548069cf0 con 0x7f35480627b0 2024-01-24T13:59:31.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.090+0000 7f3547fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35480627b0 0x7f3548062c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.090+0000 7f35477fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3548138330 0x7f3548138710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.090+0000 7f3547fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35480627b0 0x7f3548062c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:52382/0 (socket says 172.21.15.29:52382) 2024-01-24T13:59:31.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.090+0000 7f35477fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3548138330 0x7f3548138710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:57742/0 (socket says 172.21.15.29:57742) 2024-01-24T13:59:31.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.090+0000 7f3546ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f35480bca50 0x7f35480be3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:36646/0 (socket says 172.21.15.29:36646) 2024-01-24T13:59:31.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.090+0000 7f3547fff700 1 -- 172.21.15.29:0/843267135 learned_addr learned my addr 172.21.15.29:0/843267135 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.091+0000 7f35477fe700 1 -- 172.21.15.29:0/843267135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35480627b0 msgr2=0x7f3548062c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.091+0000 7f35477fe700 1 --2- 172.21.15.29:0/843267135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35480627b0 0x7f3548062c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.091+0000 7f35477fe700 1 -- 172.21.15.29:0/843267135 >> v1:172.21.15.163:6789/0 conn(0x7f35480bca50 legacy=0x7f35480be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.091+0000 7f35477fe700 1 -- 172.21.15.29:0/843267135 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3548069f60 con 0x7f3548138330 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.091+0000 7f35467fc700 1 -- 172.21.15.29:0/843267135 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781018104 0 0) 0x7f3548065770 con 0x7f35480bca50 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.091+0000 7f3547fff700 1 --2- 172.21.15.29:0/843267135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35480627b0 0x7f3548062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.091+0000 7f35477fe700 1 --2- 172.21.15.29:0/843267135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3548138330 0x7f3548138710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f3538009760 tx=0x7f3538009060).ready entity=mon.1 client_cookie=2392a63bd375277c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.091+0000 7f35467fc700 1 -- 172.21.15.29:0/843267135 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353800f510 con 0x7f3548138330 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.091+0000 7f35467fc700 1 -- 172.21.15.29:0/843267135 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353800f670 con 0x7f3548138330 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.091+0000 7f35467fc700 1 -- 172.21.15.29:0/843267135 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353800f9f0 con 0x7f3548138330 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.092+0000 7f354decd700 1 -- 172.21.15.29:0/843267135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3548138330 msgr2=0x7f3548138710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.092+0000 7f354decd700 1 --2- 172.21.15.29:0/843267135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3548138330 0x7f3548138710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f3538009760 tx=0x7f3538009060).stop 2024-01-24T13:59:31.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.092+0000 7f354decd700 1 -- 172.21.15.29:0/843267135 shutdown_connections 2024-01-24T13:59:31.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.092+0000 7f354decd700 1 --2- 172.21.15.29:0/843267135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35480627b0 0x7f3548062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.092+0000 7f354decd700 1 --2- 172.21.15.29:0/843267135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3548138330 0x7f3548138710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.092+0000 7f354decd700 1 -- 172.21.15.29:0/843267135 >> 172.21.15.29:0/843267135 conn(0x7f35480c85c0 msgr2=0x7f35480c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:31.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.092+0000 7f354decd700 1 -- 172.21.15.29:0/843267135 shutdown_connections 2024-01-24T13:59:31.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.092+0000 7f354decd700 1 -- 172.21.15.29:0/843267135 wait complete. 2024-01-24T13:59:31.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.093+0000 7f354decd700 1 Processor -- start 2024-01-24T13:59:31.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.093+0000 7f354decd700 1 -- start start 2024-01-24T13:59:31.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.094+0000 7f354decd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35480627b0 0x7f35480a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.094+0000 7f354decd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35480bca50 0x7f35480aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.094+0000 7f354decd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3548138330 0x7f35480aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.094+0000 7f354decd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35480ab480 con 0x7f35480627b0 2024-01-24T13:59:31.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.094+0000 7f354decd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35480c39c0 con 0x7f35480bca50 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.094+0000 7f354decd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35480c2db0 con 0x7f3548138330 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.094+0000 7f35477fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35480627b0 0x7f35480a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.094+0000 7f35477fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35480627b0 0x7f35480a68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39640/0 (socket says 172.21.15.29:39640) 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.094+0000 7f35477fe700 1 -- 172.21.15.29:0/3660440966 learned_addr learned my addr 172.21.15.29:0/3660440966 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.094+0000 7f3546ffd700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35480bca50 0x7f35480aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.094+0000 7f3547fff700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3548138330 0x7f35480aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.095+0000 7f35477fe700 1 -- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3548138330 msgr2=0x7f35480aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.095+0000 7f35477fe700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3548138330 0x7f35480aaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.095+0000 7f35477fe700 1 -- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35480bca50 msgr2=0x7f35480aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.095+0000 7f35477fe700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35480bca50 0x7f35480aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.095+0000 7f35477fe700 1 -- 172.21.15.29:0/3660440966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3540009ae0 con 0x7f35480627b0 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.095+0000 7f35477fe700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35480627b0 0x7f35480a68a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f3548068d50 tx=0x7f3538017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.095+0000 7f353f7fe700 1 -- 172.21.15.29:0/3660440966 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353800f7e0 con 0x7f35480627b0 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.095+0000 7f353f7fe700 1 -- 172.21.15.29:0/3660440966 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353800f940 con 0x7f35480627b0 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.095+0000 7f354decd700 1 -- 172.21.15.29:0/3660440966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35380092a0 con 0x7f35480627b0 2024-01-24T13:59:31.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.095+0000 7f353f7fe700 1 -- 172.21.15.29:0/3660440966 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353801b690 con 0x7f35480627b0 2024-01-24T13:59:31.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.095+0000 7f354decd700 1 -- 172.21.15.29:0/3660440966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35480c3370 con 0x7f35480627b0 2024-01-24T13:59:31.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.096+0000 7f354decd700 1 -- 172.21.15.29:0/3660440966 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35480475e0 con 0x7f35480627b0 2024-01-24T13:59:31.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.097+0000 7f353f7fe700 1 -- 172.21.15.29:0/3660440966 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f353801ba80 con 0x7f35480627b0 2024-01-24T13:59:31.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.097+0000 7f353f7fe700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f352805f150 0x7f3528061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.097+0000 7f353f7fe700 1 -- 172.21.15.29:0/3660440966 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f353807b910 con 0x7f35480627b0 2024-01-24T13:59:31.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.097+0000 7f3546ffd700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f352805f150 0x7f3528061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.098+0000 7f3546ffd700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f352805f150 0x7f3528061610 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f35480c76e0 tx=0x7f3530007bd0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:31.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.101+0000 7f353f7fe700 1 -- 172.21.15.29:0/3660440966 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f353804ca80 con 0x7f35480627b0 2024-01-24T13:59:31.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.277+0000 7f354decd700 1 -- 172.21.15.29:0/3660440966 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35480a74e0 con 0x7f352805f150 2024-01-24T13:59:31.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.287+0000 7f353f7fe700 1 -- 172.21.15.29:0/3660440966 <== mgr.4103 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+354174 (secure 0 0 0) 0x7f35480a74e0 con 0x7f352805f150 2024-01-24T13:59:31.294 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:31.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.291+0000 7f354decd700 1 -- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f352805f150 msgr2=0x7f3528061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.291+0000 7f354decd700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f352805f150 0x7f3528061610 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f35480c76e0 tx=0x7f3530007bd0).stop 2024-01-24T13:59:31.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.291+0000 7f354decd700 1 -- 172.21.15.29:0/3660440966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35480627b0 msgr2=0x7f35480a68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.291+0000 7f354decd700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35480627b0 0x7f35480a68a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f3548068d50 tx=0x7f3538017000).stop 2024-01-24T13:59:31.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.291+0000 7f354decd700 1 -- 172.21.15.29:0/3660440966 shutdown_connections 2024-01-24T13:59:31.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.291+0000 7f354decd700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3548138330 0x7f35480aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.291+0000 7f354decd700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f352805f150 0x7f3528061610 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.291+0000 7f354decd700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35480627b0 0x7f35480a68a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.291+0000 7f354decd700 1 --2- 172.21.15.29:0/3660440966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35480bca50 0x7f35480aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.291+0000 7f354decd700 1 -- 172.21.15.29:0/3660440966 >> 172.21.15.29:0/3660440966 conn(0x7f35480c85c0 msgr2=0x7f35480c4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:31.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.291+0000 7f354decd700 1 -- 172.21.15.29:0/3660440966 shutdown_connections 2024-01-24T13:59:31.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.291+0000 7f354decd700 1 -- 172.21.15.29:0/3660440966 wait complete. 2024-01-24T13:59:31.299 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T13:59:31.305 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-01-24T13:59:29.742125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":892,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380176,"kb_used_data":3408,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752594544,"statfs":{"total":773094113280,"available":770656813056,"internally_reserved":0,"allocated":3489792,"data_stored":1556865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6097,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-57,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.550072"},"pg_stats":[{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950914+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.952115+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394881+0000","last_change":"2024-01-24T13:59:26.394881+0000","last_active":"2024-01-24T13:59:26.394881+0000","last_peered":"2024-01-24T13:59:26.394881+0000","last_clean":"2024-01-24T13:59:26.394881+0000","last_became_active":"2024-01-24T13:59:26.394600+0000","last_became_peered":"2024-01-24T13:59:26.394600+0000","last_unstale":"2024-01-24T13:59:26.394881+0000","last_undegraded":"2024-01-24T13:59:26.394881+0000","last_fullsized":"2024-01-24T13:59:26.394881+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.982279+0000","last_change":"2024-01-24T13:59:25.980665+0000","last_active":"2024-01-24T13:59:25.982279+0000","last_peered":"2024-01-24T13:59:25.982279+0000","last_clean":"2024-01-24T13:59:25.982279+0000","last_became_active":"2024-01-24T13:59:25.980288+0000","last_became_peered":"2024-01-24T13:59:25.980288+0000","last_unstale":"2024-01-24T13:59:25.982279+0000","last_undegraded":"2024-01-24T13:59:25.982279+0000","last_fullsized":"2024-01-24T13:59:25.982279+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.947378+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.947325+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949460+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.397603+0000","last_change":"2024-01-24T13:59:26.397603+0000","last_active":"2024-01-24T13:59:26.397603+0000","last_peered":"2024-01-24T13:59:26.397603+0000","last_clean":"2024-01-24T13:59:26.397603+0000","last_became_active":"2024-01-24T13:59:26.397269+0000","last_became_peered":"2024-01-24T13:59:26.397269+0000","last_unstale":"2024-01-24T13:59:26.397603+0000","last_undegraded":"2024-01-24T13:59:26.397603+0000","last_fullsized":"2024-01-24T13:59:26.397603+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959680+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950356+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391986+0000","last_change":"2024-01-24T13:59:26.391986+0000","last_active":"2024-01-24T13:59:26.391986+0000","last_peered":"2024-01-24T13:59:26.391986+0000","last_clean":"2024-01-24T13:59:26.391986+0000","last_became_active":"2024-01-24T13:59:26.391698+0000","last_became_peered":"2024-01-24T13:59:26.391698+0000","last_unstale":"2024-01-24T13:59:26.391986+0000","last_undegraded":"2024-01-24T13:59:26.391986+0000","last_fullsized":"2024-01-24T13:59:26.391986+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.980747+0000","last_change":"2024-01-24T13:59:25.973798+0000","last_active":"2024-01-24T13:59:25.980747+0000","last_peered":"2024-01-24T13:59:25.980747+0000","last_clean":"2024-01-24T13:59:25.980747+0000","last_became_active":"2024-01-24T13:59:25.973506+0000","last_became_peered":"2024-01-24T13:59:25.973506+0000","last_unstale":"2024-01-24T13:59:25.980747+0000","last_undegraded":"2024-01-24T13:59:25.980747+0000","last_fullsized":"2024-01-24T13:59:25.980747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392777+0000","last_change":"2024-01-24T13:59:26.392777+0000","last_active":"2024-01-24T13:59:26.392777+0000","last_peered":"2024-01-24T13:59:26.392777+0000","last_clean":"2024-01-24T13:59:26.392777+0000","last_became_active":"2024-01-24T13:59:26.392496+0000","last_became_peered":"2024-01-24T13:59:26.392496+0000","last_unstale":"2024-01-24T13:59:26.392777+0000","last_undegraded":"2024-01-24T13:59:26.392777+0000","last_fullsized":"2024-01-24T13:59:26.392777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951917+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959171+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394489+0000","last_change":"2024-01-24T13:59:26.394489+0000","last_active":"2024-01-24T13:59:26.394489+0000","last_peered":"2024-01-24T13:59:26.394489+0000","last_clean":"2024-01-24T13:59:26.394489+0000","last_became_active":"2024-01-24T13:59:26.392243+0000","last_became_peered":"2024-01-24T13:59:26.392243+0000","last_unstale":"2024-01-24T13:59:26.394489+0000","last_undegraded":"2024-01-24T13:59:26.394489+0000","last_fullsized":"2024-01-24T13:59:26.394489+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"40'4","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404373+0000","last_change":"2024-01-24T13:59:25.973968+0000","last_active":"2024-01-24T13:59:26.404373+0000","last_peered":"2024-01-24T13:59:26.404373+0000","last_clean":"2024-01-24T13:59:26.404373+0000","last_became_active":"2024-01-24T13:59:25.973675+0000","last_became_peered":"2024-01-24T13:59:25.973675+0000","last_unstale":"2024-01-24T13:59:26.404373+0000","last_undegraded":"2024-01-24T13:59:26.404373+0000","last_fullsized":"2024-01-24T13:59:26.404373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.395811+0000","last_change":"2024-01-24T13:59:26.395811+0000","last_active":"2024-01-24T13:59:26.395811+0000","last_peered":"2024-01-24T13:59:26.395811+0000","last_clean":"2024-01-24T13:59:26.395811+0000","last_became_active":"2024-01-24T13:59:26.395486+0000","last_became_peered":"2024-01-24T13:59:26.395486+0000","last_unstale":"2024-01-24T13:59:26.395811+0000","last_undegraded":"2024-01-24T13:59:26.395811+0000","last_fullsized":"2024-01-24T13:59:26.395811+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959208+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949817+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959318+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959759+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959476+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949550+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946938+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949148+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393452+0000","last_change":"2024-01-24T13:59:26.393452+0000","last_active":"2024-01-24T13:59:26.393452+0000","last_peered":"2024-01-24T13:59:26.393452+0000","last_clean":"2024-01-24T13:59:26.393452+0000","last_became_active":"2024-01-24T13:59:26.393184+0000","last_became_peered":"2024-01-24T13:59:26.393184+0000","last_unstale":"2024-01-24T13:59:26.393452+0000","last_undegraded":"2024-01-24T13:59:26.393452+0000","last_fullsized":"2024-01-24T13:59:26.393452+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.947286+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949431+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958647+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946523+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949817+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949549+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949548+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946704+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.948979+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950028+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958063+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949470+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950354+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950439+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.952133+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.959016+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951596+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399449+0000","last_change":"2024-01-24T13:59:26.399449+0000","last_active":"2024-01-24T13:59:26.399449+0000","last_peered":"2024-01-24T13:59:26.399449+0000","last_clean":"2024-01-24T13:59:26.399449+0000","last_became_active":"2024-01-24T13:59:26.399156+0000","last_became_peered":"2024-01-24T13:59:26.399156+0000","last_unstale":"2024-01-24T13:59:26.399449+0000","last_undegraded":"2024-01-24T13:59:26.399449+0000","last_fullsized":"2024-01-24T13:59:26.399449+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949041+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.397561+0000","last_change":"2024-01-24T13:59:26.397561+0000","last_active":"2024-01-24T13:59:26.397561+0000","last_peered":"2024-01-24T13:59:26.397561+0000","last_clean":"2024-01-24T13:59:26.397561+0000","last_became_active":"2024-01-24T13:59:26.397284+0000","last_became_peered":"2024-01-24T13:59:26.397284+0000","last_unstale":"2024-01-24T13:59:26.397561+0000","last_undegraded":"2024-01-24T13:59:26.397561+0000","last_fullsized":"2024-01-24T13:59:26.397561+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946640+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.982939+0000","last_change":"2024-01-24T13:59:25.981443+0000","last_active":"2024-01-24T13:59:25.982939+0000","last_peered":"2024-01-24T13:59:25.982939+0000","last_clean":"2024-01-24T13:59:25.982939+0000","last_became_active":"2024-01-24T13:59:25.980669+0000","last_became_peered":"2024-01-24T13:59:25.980669+0000","last_unstale":"2024-01-24T13:59:25.982939+0000","last_undegraded":"2024-01-24T13:59:25.982939+0000","last_fullsized":"2024-01-24T13:59:25.982939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949102+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.981798+0000","last_change":"2024-01-24T13:59:25.980155+0000","last_active":"2024-01-24T13:59:25.981798+0000","last_peered":"2024-01-24T13:59:25.981798+0000","last_clean":"2024-01-24T13:59:25.981798+0000","last_became_active":"2024-01-24T13:59:25.979649+0000","last_became_peered":"2024-01-24T13:59:25.979649+0000","last_unstale":"2024-01-24T13:59:25.981798+0000","last_undegraded":"2024-01-24T13:59:25.981798+0000","last_fullsized":"2024-01-24T13:59:25.981798+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958506+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946408+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951054+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.395796+0000","last_change":"2024-01-24T13:59:26.395796+0000","last_active":"2024-01-24T13:59:26.395796+0000","last_peered":"2024-01-24T13:59:26.395796+0000","last_clean":"2024-01-24T13:59:26.395796+0000","last_became_active":"2024-01-24T13:59:26.395521+0000","last_became_peered":"2024-01-24T13:59:26.395521+0000","last_unstale":"2024-01-24T13:59:26.395796+0000","last_undegraded":"2024-01-24T13:59:26.395796+0000","last_fullsized":"2024-01-24T13:59:26.395796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958118+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950823+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"40'4","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401238+0000","last_change":"2024-01-24T13:59:25.981003+0000","last_active":"2024-01-24T13:59:26.401238+0000","last_peered":"2024-01-24T13:59:26.401238+0000","last_clean":"2024-01-24T13:59:26.401238+0000","last_became_active":"2024-01-24T13:59:25.980309+0000","last_became_peered":"2024-01-24T13:59:25.980309+0000","last_unstale":"2024-01-24T13:59:26.401238+0000","last_undegraded":"2024-01-24T13:59:26.401238+0000","last_fullsized":"2024-01-24T13:59:26.401238+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.947411+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392588+0000","last_change":"2024-01-24T13:59:26.392588+0000","last_active":"2024-01-24T13:59:26.392588+0000","last_peered":"2024-01-24T13:59:26.392588+0000","last_clean":"2024-01-24T13:59:26.392588+0000","last_became_active":"2024-01-24T13:59:26.392285+0000","last_became_peered":"2024-01-24T13:59:26.392285+0000","last_unstale":"2024-01-24T13:59:26.392588+0000","last_undegraded":"2024-01-24T13:59:26.392588+0000","last_fullsized":"2024-01-24T13:59:26.392588+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.948612+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946600+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951306+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950508+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.946829+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.981204+0000","last_change":"2024-01-24T13:59:25.980225+0000","last_active":"2024-01-24T13:59:25.981204+0000","last_peered":"2024-01-24T13:59:25.981204+0000","last_clean":"2024-01-24T13:59:25.981204+0000","last_became_active":"2024-01-24T13:59:25.979876+0000","last_became_peered":"2024-01-24T13:59:25.979876+0000","last_unstale":"2024-01-24T13:59:25.981204+0000","last_undegraded":"2024-01-24T13:59:25.981204+0000","last_fullsized":"2024-01-24T13:59:25.981204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392651+0000","last_change":"2024-01-24T13:59:26.392651+0000","last_active":"2024-01-24T13:59:26.392651+0000","last_peered":"2024-01-24T13:59:26.392651+0000","last_clean":"2024-01-24T13:59:26.392651+0000","last_became_active":"2024-01-24T13:59:26.392338+0000","last_became_peered":"2024-01-24T13:59:26.392338+0000","last_unstale":"2024-01-24T13:59:26.392651+0000","last_undegraded":"2024-01-24T13:59:26.392651+0000","last_fullsized":"2024-01-24T13:59:26.392651+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958226+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958985+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397561+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T13:59:21.397561+0000","last_peered":"2024-01-24T13:59:21.397561+0000","last_clean":"2024-01-24T13:59:21.397561+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T13:59:21.397561+0000","last_undegraded":"2024-01-24T13:59:21.397561+0000","last_fullsized":"2024-01-24T13:59:21.397561+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.948578+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397492+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T13:59:21.397492+0000","last_peered":"2024-01-24T13:59:21.397492+0000","last_clean":"2024-01-24T13:59:21.397492+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T13:59:21.397492+0000","last_undegraded":"2024-01-24T13:59:21.397492+0000","last_fullsized":"2024-01-24T13:59:21.397492+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.949583+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397705+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T13:59:21.397705+0000","last_peered":"2024-01-24T13:59:21.397705+0000","last_clean":"2024-01-24T13:59:21.397705+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T13:59:21.397705+0000","last_undegraded":"2024-01-24T13:59:21.397705+0000","last_fullsized":"2024-01-24T13:59:21.397705+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.950081+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397709+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T13:59:21.397709+0000","last_peered":"2024-01-24T13:59:21.397709+0000","last_clean":"2024-01-24T13:59:21.397709+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T13:59:21.397709+0000","last_undegraded":"2024-01-24T13:59:21.397709+0000","last_fullsized":"2024-01-24T13:59:21.397709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394231+0000","last_change":"2024-01-24T13:59:26.394231+0000","last_active":"2024-01-24T13:59:26.394231+0000","last_peered":"2024-01-24T13:59:26.394231+0000","last_clean":"2024-01-24T13:59:26.394231+0000","last_became_active":"2024-01-24T13:59:26.393918+0000","last_became_peered":"2024-01-24T13:59:26.393918+0000","last_unstale":"2024-01-24T13:59:26.394231+0000","last_undegraded":"2024-01-24T13:59:26.394231+0000","last_fullsized":"2024-01-24T13:59:26.394231+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.952375+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397711+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T13:59:21.397711+0000","last_peered":"2024-01-24T13:59:21.397711+0000","last_clean":"2024-01-24T13:59:21.397711+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T13:59:21.397711+0000","last_undegraded":"2024-01-24T13:59:21.397711+0000","last_fullsized":"2024-01-24T13:59:21.397711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:20.006269+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T13:59:20.006269+0000","last_peered":"2024-01-24T13:59:20.006269+0000","last_clean":"2024-01-24T13:59:20.006269+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T13:59:20.006269+0000","last_undegraded":"2024-01-24T13:59:20.006269+0000","last_fullsized":"2024-01-24T13:59:20.006269+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.388213+0000","last_change":"2024-01-24T13:59:26.388213+0000","last_active":"2024-01-24T13:59:26.388213+0000","last_peered":"2024-01-24T13:59:26.388213+0000","last_clean":"2024-01-24T13:59:26.388213+0000","last_became_active":"2024-01-24T13:59:26.387917+0000","last_became_peered":"2024-01-24T13:59:26.387917+0000","last_unstale":"2024-01-24T13:59:26.388213+0000","last_undegraded":"2024-01-24T13:59:26.388213+0000","last_fullsized":"2024-01-24T13:59:26.388213+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.958091+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:20.005875+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T13:59:20.005875+0000","last_peered":"2024-01-24T13:59:20.005875+0000","last_clean":"2024-01-24T13:59:20.005875+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T13:59:20.005875+0000","last_undegraded":"2024-01-24T13:59:20.005875+0000","last_fullsized":"2024-01-24T13:59:20.005875+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-24T13:59:24.941228+0000","last_change":"2024-01-24T13:59:24.951679+0000","last_active":"2024-01-24T13:59:24.941228+0000","last_peered":"2024-01-24T13:59:24.941228+0000","last_clean":"2024-01-24T13:59:24.941228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:24.941228+0000","last_undegraded":"2024-01-24T13:59:24.941228+0000","last_fullsized":"2024-01-24T13:59:24.941228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:20.006269+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T13:59:20.006269+0000","last_peered":"2024-01-24T13:59:20.006269+0000","last_clean":"2024-01-24T13:59:20.006269+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T13:59:20.006269+0000","last_undegraded":"2024-01-24T13:59:20.006269+0000","last_fullsized":"2024-01-24T13:59:20.006269+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T13:59:25.943243+0000","last_change":"2024-01-24T13:59:25.943243+0000","last_active":"2024-01-24T13:59:25.943243+0000","last_peered":"2024-01-24T13:59:25.943243+0000","last_clean":"2024-01-24T13:59:25.943243+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T13:59:25.943243+0000","last_undegraded":"2024-01-24T13:59:25.943243+0000","last_fullsized":"2024-01-24T13:59:25.943243+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.983216+0000","last_change":"2024-01-24T13:59:25.982352+0000","last_active":"2024-01-24T13:59:25.983216+0000","last_peered":"2024-01-24T13:59:25.983216+0000","last_clean":"2024-01-24T13:59:25.983216+0000","last_became_active":"2024-01-24T13:59:25.981960+0000","last_became_peered":"2024-01-24T13:59:25.981960+0000","last_unstale":"2024-01-24T13:59:25.983216+0000","last_undegraded":"2024-01-24T13:59:25.983216+0000","last_fullsized":"2024-01-24T13:59:25.983216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-01-24T13:59:21.397468+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T13:59:21.397468+0000","last_peered":"2024-01-24T13:59:21.397468+0000","last_clean":"2024-01-24T13:59:21.397468+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T13:59:21.397468+0000","last_undegraded":"2024-01-24T13:59:21.397468+0000","last_fullsized":"2024-01-24T13:59:21.397468+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":1},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":892,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":434176,"data_stored":194589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.31900000000000001}]},{"osd":2,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.377}]},{"osd":3,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.63300000000000001}]},{"osd":5,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.747}]},{"osd":7,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.77200000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52200000000000002}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.622}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42999999999999999}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46500000000000002}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.499}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.43099999999999999}]},{"osd":1,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.501},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55600000000000005}]},{"osd":5,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":425984,"data_stored":188976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.311},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.374}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.41499999999999998}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.623},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55300000000000005}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54300000000000004}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.59899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.498}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.38},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.51800000000000002}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.435}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.63500000000000001}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65800000000000003}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59099999999999997}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.623}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.56999999999999995}]},{"osd":1,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.63},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53400000000000003}]},{"osd":3,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.58499999999999996}]},{"osd":5,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.41399999999999998}]},{"osd":6,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.48199999999999998}]},{"osd":7,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.503},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.51800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.47599999999999998}]},{"osd":3,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.44}]},{"osd":6,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.502}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":438272,"data_stored":195550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":1.3859999999999999}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":1.353},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.81200000000000006}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":6.8129999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":1.4079999999999999}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":7.016},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.85099999999999998}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":6.8719999999999999}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":6.976},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":7.8789999999999996}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":6.9080000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T13:59:31.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2024-01-24T13:59:31.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-01-24T13:59:31.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.950914+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-24T13:59:31.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2024-01-24T13:59:31.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-01-24T13:59:31.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.952115+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-24T13:59:31.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2024-01-24T13:59:31.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-01-24T13:59:31.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.947378+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-24T13:59:31.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2024-01-24T13:59:31.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-01-24T13:59:31.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.947325+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-24T13:59:31.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2024-01-24T13:59:31.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-01-24T13:59:31.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.949460+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-24T13:59:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-01-24T13:59:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.959680+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-24T13:59:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2024-01-24T13:59:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-01-24T13:59:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.950356+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-24T13:59:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-01-24T13:59:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.951917+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-24T13:59:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2024-01-24T13:59:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-01-24T13:59:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.959171+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-24T13:59:31.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-01-24T13:59:31.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.959208+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-24T13:59:31.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2024-01-24T13:59:31.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-01-24T13:59:31.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.949817+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-24T13:59:31.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2024-01-24T13:59:31.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-01-24T13:59:31.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.959318+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-24T13:59:31.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-01-24T13:59:31.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-01-24T13:59:31.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.959759+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-24T13:59:31.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2024-01-24T13:59:31.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-01-24T13:59:31.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.959476+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-24T13:59:31.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2024-01-24T13:59:31.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-01-24T13:59:31.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.949550+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-24T13:59:31.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2024-01-24T13:59:31.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-01-24T13:59:31.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.946938+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-24T13:59:31.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2024-01-24T13:59:31.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-01-24T13:59:31.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.949148+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-24T13:59:31.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-01-24T13:59:31.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.947286+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-24T13:59:31.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-01-24T13:59:31.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-01-24T13:59:31.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.949431+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-24T13:59:31.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2024-01-24T13:59:31.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-01-24T13:59:31.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.958647+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-24T13:59:31.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2024-01-24T13:59:31.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.946523+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.949817+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.949549+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-01-24T13:59:31.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.949548+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-24T13:59:31.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2024-01-24T13:59:31.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-01-24T13:59:31.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.946704+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-24T13:59:31.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2024-01-24T13:59:31.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-01-24T13:59:31.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.948979+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-24T13:59:31.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2024-01-24T13:59:31.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-01-24T13:59:31.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.950028+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-24T13:59:31.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-01-24T13:59:31.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-01-24T13:59:31.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.958063+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-24T13:59:31.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2024-01-24T13:59:31.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-01-24T13:59:31.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.949470+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-24T13:59:31.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-01-24T13:59:31.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-01-24T13:59:31.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.950354+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-24T13:59:31.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-01-24T13:59:31.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-01-24T13:59:31.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.950439+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-24T13:59:31.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-01-24T13:59:31.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-01-24T13:59:31.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.952133+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-24T13:59:31.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-01-24T13:59:31.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-01-24T13:59:31.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.959016+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-24T13:59:31.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-01-24T13:59:31.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.951596+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.949041+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.946640+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.949102+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-01-24T13:59:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-01-24T13:59:31.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-01-24T13:59:31.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.958506+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-24T13:59:31.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-01-24T13:59:31.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-01-24T13:59:31.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.946408+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-24T13:59:31.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-01-24T13:59:31.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-01-24T13:59:31.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.951054+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-24T13:59:31.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-01-24T13:59:31.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.958118+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-24T13:59:31.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-01-24T13:59:31.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-01-24T13:59:31.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.950823+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-24T13:59:31.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-01-24T13:59:31.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-01-24T13:59:31.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-01-24T13:59:31.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.947411+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-24T13:59:31.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-01-24T13:59:31.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.948612+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-24T13:59:31.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-01-24T13:59:31.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-01-24T13:59:31.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.946600+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.951306+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.950508+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.946829+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-24T13:59:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-01-24T13:59:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-01-24T13:59:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.958226+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-24T13:59:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-01-24T13:59:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-01-24T13:59:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.958985+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-24T13:59:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-01-24T13:59:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-01-24T13:59:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.948578+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-24T13:59:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.949583+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.950081+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.952375+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.958091+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-01-24T13:59:31.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-01-24T13:59:31.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-24T13:59:24.941228+0000', 'last_change': '2024-01-24T13:59:24.951679+0000', 'last_active': '2024-01-24T13:59:24.941228+0000', 'last_peered': '2024-01-24T13:59:24.941228+0000', 'last_clean': '2024-01-24T13:59:24.941228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:24.941228+0000', 'last_undegraded': '2024-01-24T13:59:24.941228+0000', 'last_fullsized': '2024-01-24T13:59:24.941228+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:24.941228+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-24T13:59:31.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-01-24T13:59:31.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-24T13:59:25.943243+0000', 'last_change': '2024-01-24T13:59:25.943243+0000', 'last_active': '2024-01-24T13:59:25.943243+0000', 'last_peered': '2024-01-24T13:59:25.943243+0000', 'last_clean': '2024-01-24T13:59:25.943243+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-24T13:59:25.943243+0000', 'last_undegraded': '2024-01-24T13:59:25.943243+0000', 'last_fullsized': '2024-01-24T13:59:25.943243+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'last_clean_scrub_stamp': '2024-01-24T13:59:25.943243+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-24T13:59:31.330 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-24T13:59:31.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.476+0000 7f70be35e700 1 Processor -- start 2024-01-24T13:59:31.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.476+0000 7f70be35e700 1 -- start start 2024-01-24T13:59:31.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.477+0000 7f70be35e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70b8062ec0 0x7f70b80678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.477+0000 7f70be35e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b8065bc0 con 0x7f70b80a9e50 2024-01-24T13:59:31.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.477+0000 7f70be35e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b80645b0 con 0x7f70b8063fd0 2024-01-24T13:59:31.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.477+0000 7f70be35e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b8152430 con 0x7f70b8062ec0 2024-01-24T13:59:31.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.477+0000 7f70b7fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f70b80a9e50 0x7f70b8153600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54060/0 (socket says 172.21.15.29:54060) 2024-01-24T13:59:31.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.477+0000 7f70b7fff700 1 -- 172.21.15.29:0/3846464921 learned_addr learned my addr 172.21.15.29:0/3846464921 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:31.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.477+0000 7f70bc8fb700 1 --2- 172.21.15.29:0/3846464921 >> v2:172.21.15.163:3300/0 conn(0x7f70b8062ec0 0x7f70b80678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.477+0000 7f70b6ffd700 1 -- 172.21.15.29:0/3846464921 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309770234 0 0) 0x7f70b8065bc0 con 0x7f70b80a9e50 2024-01-24T13:59:31.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.478+0000 7f70b6ffd700 1 -- 172.21.15.29:0/3846464921 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70a00033f0 con 0x7f70b80a9e50 2024-01-24T13:59:31.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.478+0000 7f70b6ffd700 1 -- 172.21.15.29:0/3846464921 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322637246 0 0) 0x7f70b80645b0 con 0x7f70b8063fd0 2024-01-24T13:59:31.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.478+0000 7f70b6ffd700 1 -- 172.21.15.29:0/3846464921 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70b8065bc0 con 0x7f70b8063fd0 2024-01-24T13:59:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.478+0000 7f70bc8fb700 1 -- 172.21.15.29:0/3846464921 >> v1:172.21.15.163:6789/0 conn(0x7f70b8063fd0 legacy=0x7f70b80627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.478+0000 7f70b6ffd700 1 -- 172.21.15.29:0/3846464921 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3464642932 0 0) 0x7f70a00033f0 con 0x7f70b80a9e50 2024-01-24T13:59:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.478+0000 7f70bc8fb700 1 -- 172.21.15.29:0/3846464921 >> v1:172.21.15.29:6789/0 conn(0x7f70b80a9e50 legacy=0x7f70b8153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.478+0000 7f70bc8fb700 1 -- 172.21.15.29:0/3846464921 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b8065e80 con 0x7f70b8062ec0 2024-01-24T13:59:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.478+0000 7f70bc8fb700 1 --2- 172.21.15.29:0/3846464921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b8062ec0 0x7f70b80678c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f70b0009e60 tx=0x7f70b0009780).ready entity=mon.1 client_cookie=b9e33efd04ee77ff server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.478+0000 7f70b6ffd700 1 -- 172.21.15.29:0/3846464921 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000e070 con 0x7f70b8062ec0 2024-01-24T13:59:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.478+0000 7f70b6ffd700 1 -- 172.21.15.29:0/3846464921 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b000b7a0 con 0x7f70b8062ec0 2024-01-24T13:59:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.478+0000 7f70b6ffd700 1 -- 172.21.15.29:0/3846464921 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8001470 con 0x7f70b8062ec0 2024-01-24T13:59:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.479+0000 7f70be35e700 1 -- 172.21.15.29:0/3846464921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b8062ec0 msgr2=0x7f70b80678c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.479+0000 7f70be35e700 1 --2- 172.21.15.29:0/3846464921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b8062ec0 0x7f70b80678c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f70b0009e60 tx=0x7f70b0009780).stop 2024-01-24T13:59:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.479+0000 7f70be35e700 1 -- 172.21.15.29:0/3846464921 shutdown_connections 2024-01-24T13:59:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.479+0000 7f70be35e700 1 --2- 172.21.15.29:0/3846464921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b8062ec0 0x7f70b80678c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.479+0000 7f70be35e700 1 -- 172.21.15.29:0/3846464921 >> 172.21.15.29:0/3846464921 conn(0x7f70b80c5090 msgr2=0x7f70b80638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.479+0000 7f70be35e700 1 -- 172.21.15.29:0/3846464921 shutdown_connections 2024-01-24T13:59:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.479+0000 7f70be35e700 1 -- 172.21.15.29:0/3846464921 wait complete. 2024-01-24T13:59:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.480+0000 7f70be35e700 1 Processor -- start 2024-01-24T13:59:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.480+0000 7f70be35e700 1 -- start start 2024-01-24T13:59:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.481+0000 7f70be35e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70b8062ec0 0x7f70b80c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.481+0000 7f70be35e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b8063fd0 0x7f70b80c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.481+0000 7f70be35e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b80a9e50 0x7f70b8164010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.481+0000 7f70be35e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b80c41f0 con 0x7f70b8062ec0 2024-01-24T13:59:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.481+0000 7f70be35e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b80c25c0 con 0x7f70b8063fd0 2024-01-24T13:59:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.481+0000 7f70be35e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b80c2ef0 con 0x7f70b80a9e50 2024-01-24T13:59:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.481+0000 7f70b7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70b8062ec0 0x7f70b80c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.481+0000 7f70b7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70b8062ec0 0x7f70b80c20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39646/0 (socket says 172.21.15.29:39646) 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.481+0000 7f70b7fff700 1 -- 172.21.15.29:0/796245875 learned_addr learned my addr 172.21.15.29:0/796245875 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.481+0000 7f70b77fe700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b8063fd0 0x7f70b80c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.481+0000 7f70bc8fb700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b80a9e50 0x7f70b8164010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.482+0000 7f70b7fff700 1 -- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b80a9e50 msgr2=0x7f70b8164010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.482+0000 7f70b7fff700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b80a9e50 0x7f70b8164010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.482+0000 7f70b7fff700 1 -- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b8063fd0 msgr2=0x7f70b80c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.482+0000 7f70b7fff700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b8063fd0 0x7f70b80c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.482+0000 7f70b7fff700 1 -- 172.21.15.29:0/796245875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b0009a20 con 0x7f70b8062ec0 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.482+0000 7f70b77fe700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b8063fd0 0x7f70b80c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.482+0000 7f70b7fff700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70b8062ec0 0x7f70b80c20d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f70a800c800 tx=0x7f70a8001ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.482+0000 7f709ffff700 1 -- 172.21.15.29:0/796245875 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8007170 con 0x7f70b8062ec0 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.482+0000 7f709ffff700 1 -- 172.21.15.29:0/796245875 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a80072d0 con 0x7f70b8062ec0 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.482+0000 7f709ffff700 1 -- 172.21.15.29:0/796245875 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a80048c0 con 0x7f70b8062ec0 2024-01-24T13:59:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.482+0000 7f70be35e700 1 -- 172.21.15.29:0/796245875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b80c3150 con 0x7f70b8062ec0 2024-01-24T13:59:31.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.482+0000 7f70be35e700 1 -- 172.21.15.29:0/796245875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b80c39f0 con 0x7f70b8062ec0 2024-01-24T13:59:31.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.483+0000 7f709ffff700 1 -- 172.21.15.29:0/796245875 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f70a8004c30 con 0x7f70b8062ec0 2024-01-24T13:59:31.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.483+0000 7f70be35e700 1 -- 172.21.15.29:0/796245875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b80c3550 con 0x7f70b8062ec0 2024-01-24T13:59:31.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.484+0000 7f709ffff700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f70a405f150 0x7f70a4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.484+0000 7f709ffff700 1 -- 172.21.15.29:0/796245875 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f70a8079e60 con 0x7f70b8062ec0 2024-01-24T13:59:31.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.484+0000 7f70b77fe700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f70a405f150 0x7f70a4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.485+0000 7f70b77fe700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f70a405f150 0x7f70a4061610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f70ac007fd0 tx=0x7f70ac009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:31.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.488+0000 7f709ffff700 1 -- 172.21.15.29:0/796245875 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f70a804afa0 con 0x7f70b8062ec0 2024-01-24T13:59:31.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.742+0000 7f70be35e700 1 -- 172.21.15.29:0/796245875 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f70b80475e0 con 0x7f70b8062ec0 2024-01-24T13:59:31.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.742+0000 7f709ffff700 1 -- 172.21.15.29:0/796245875 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1529 (secure 0 0 0) 0x7f70a8019610 con 0x7f70b8062ec0 2024-01-24T13:59:31.743 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:31.743 INFO:teuthology.orchestra.run.smithi029.stdout:{"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":90,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":41,"num_osds":8,"num_up_osds":8,"osd_up_since":1706104690,"num_in_osds":8,"osd_in_since":1706104683,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":56},{"state_name":"unknown","count":50},{"state_name":"active+clean","count":31}],"num_pgs":137,"num_pools":4,"num_objects":4,"data_bytes":892,"bytes_used":2437300224,"bytes_avail":770656813056,"bytes_total":773094113280,"unknown_pgs_ratio":0.36496350169181824,"inactive_pgs_ratio":0.40875911712646484},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5133}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-01-24T13:59:15.739347+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-01-24T13:59:31.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.745+0000 7f70be35e700 1 -- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f70a405f150 msgr2=0x7f70a4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.745+0000 7f70be35e700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f70a405f150 0x7f70a4061610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f70ac007fd0 tx=0x7f70ac009040).stop 2024-01-24T13:59:31.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.745+0000 7f70be35e700 1 -- 172.21.15.29:0/796245875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70b8062ec0 msgr2=0x7f70b80c20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.745+0000 7f70be35e700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70b8062ec0 0x7f70b80c20d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f70a800c800 tx=0x7f70a8001ab0).stop 2024-01-24T13:59:31.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.745+0000 7f70be35e700 1 -- 172.21.15.29:0/796245875 shutdown_connections 2024-01-24T13:59:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.745+0000 7f70be35e700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b80a9e50 0x7f70b8164010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.745+0000 7f70be35e700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f70a405f150 0x7f70a4061610 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.745+0000 7f70be35e700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70b8062ec0 0x7f70b80c20d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.745+0000 7f70be35e700 1 --2- 172.21.15.29:0/796245875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b8063fd0 0x7f70b80c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.745+0000 7f70be35e700 1 -- 172.21.15.29:0/796245875 >> 172.21.15.29:0/796245875 conn(0x7f70b80c5090 msgr2=0x7f70b814a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.745+0000 7f70be35e700 1 -- 172.21.15.29:0/796245875 shutdown_connections 2024-01-24T13:59:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.745+0000 7f70be35e700 1 -- 172.21.15.29:0/796245875 wait complete. 2024-01-24T13:59:31.758 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-24T13:59:31.759 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T13:59:31.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.888+0000 7f8542687700 1 Processor -- start 2024-01-24T13:59:31.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.888+0000 7f8542687700 1 -- start start 2024-01-24T13:59:31.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.888+0000 7f8542687700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f853c154370 0x7f853c0c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.888+0000 7f8542687700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f853c0aa0e0 0x7f853c154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f8542687700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853c0aa6c0 con 0x7f853c0ab1f0 2024-01-24T13:59:31.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f8542687700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853c064240 con 0x7f853c0aa0e0 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f8542687700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853c0cb5c0 con 0x7f853c154370 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f853b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f853c0aa0e0 0x7f853c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f853b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f853c0aa0e0 0x7f853c154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39660/0 (socket says 172.21.15.29:39660) 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f853b7fe700 1 -- 172.21.15.29:0/3202814408 learned_addr learned my addr 172.21.15.29:0/3202814408 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f853bfff700 1 --2- 172.21.15.29:0/3202814408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f853c154370 0x7f853c0c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f853b7fe700 1 -- 172.21.15.29:0/3202814408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f853c154370 msgr2=0x7f853c0c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f853b7fe700 1 --2- 172.21.15.29:0/3202814408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f853c154370 0x7f853c0c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f853b7fe700 1 -- 172.21.15.29:0/3202814408 >> v1:172.21.15.163:6789/0 conn(0x7f853c0ab1f0 legacy=0x7f853c0631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f853b7fe700 1 -- 172.21.15.29:0/3202814408 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c0cb8a0 con 0x7f853c0aa0e0 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f853b7fe700 1 --2- 172.21.15.29:0/3202814408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f853c0aa0e0 0x7f853c154750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f8530006fd0 tx=0x7f85300090f0).ready entity=mon.0 client_cookie=bfd9b97bdedce3e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f853affd700 1 -- 172.21.15.29:0/3202814408 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853000b600 con 0x7f853c0aa0e0 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f853affd700 1 -- 172.21.15.29:0/3202814408 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853000b760 con 0x7f853c0aa0e0 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.889+0000 7f853affd700 1 -- 172.21.15.29:0/3202814408 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853000bae0 con 0x7f853c0aa0e0 2024-01-24T13:59:31.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.890+0000 7f8542687700 1 -- 172.21.15.29:0/3202814408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f853c0aa0e0 msgr2=0x7f853c154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.890+0000 7f8542687700 1 --2- 172.21.15.29:0/3202814408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f853c0aa0e0 0x7f853c154750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f8530006fd0 tx=0x7f85300090f0).stop 2024-01-24T13:59:31.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.890+0000 7f8542687700 1 -- 172.21.15.29:0/3202814408 shutdown_connections 2024-01-24T13:59:31.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.890+0000 7f8542687700 1 --2- 172.21.15.29:0/3202814408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f853c154370 0x7f853c0c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.890+0000 7f8542687700 1 --2- 172.21.15.29:0/3202814408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f853c0aa0e0 0x7f853c154750 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.890+0000 7f8542687700 1 -- 172.21.15.29:0/3202814408 >> 172.21.15.29:0/3202814408 conn(0x7f853c05ec80 msgr2=0x7f853c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:31.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.890+0000 7f8542687700 1 -- 172.21.15.29:0/3202814408 shutdown_connections 2024-01-24T13:59:31.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.890+0000 7f8542687700 1 -- 172.21.15.29:0/3202814408 wait complete. 2024-01-24T13:59:31.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.891+0000 7f8542687700 1 Processor -- start 2024-01-24T13:59:31.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.891+0000 7f8542687700 1 -- start start 2024-01-24T13:59:31.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.892+0000 7f8542687700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f853c0aa0e0 0x7f853c0c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.892+0000 7f8542687700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f853c0ab1f0 0x7f853c0c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.892+0000 7f8542687700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f853c154370 0x7f853c0c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.892+0000 7f8542687700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c14f850 con 0x7f853c0aa0e0 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.892+0000 7f8542687700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c0c4d50 con 0x7f853c0ab1f0 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.892+0000 7f8542687700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853c0c2980 con 0x7f853c154370 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.892+0000 7f853bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f853c0aa0e0 0x7f853c0c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.892+0000 7f853bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f853c0aa0e0 0x7f853c0c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39664/0 (socket says 172.21.15.29:39664) 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.892+0000 7f853bfff700 1 -- 172.21.15.29:0/1284301427 learned_addr learned my addr 172.21.15.29:0/1284301427 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.892+0000 7f853b7fe700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f853c0ab1f0 0x7f853c0c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.892+0000 7f8540c24700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f853c154370 0x7f853c0c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.893+0000 7f853bfff700 1 -- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f853c154370 msgr2=0x7f853c0c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.893+0000 7f853bfff700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f853c154370 0x7f853c0c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.893+0000 7f853bfff700 1 -- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f853c0ab1f0 msgr2=0x7f853c0c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.893+0000 7f853bfff700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f853c0ab1f0 0x7f853c0c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.893+0000 7f853bfff700 1 -- 172.21.15.29:0/1284301427 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8530009370 con 0x7f853c0aa0e0 2024-01-24T13:59:31.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.893+0000 7f853b7fe700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f853c0ab1f0 0x7f853c0c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:31.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.893+0000 7f853bfff700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f853c0aa0e0 0x7f853c0c2e10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f852c00c650 tx=0x7f852c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:31.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.893+0000 7f8523fff700 1 -- 172.21.15.29:0/1284301427 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c010580 con 0x7f853c0aa0e0 2024-01-24T13:59:31.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.893+0000 7f8523fff700 1 -- 172.21.15.29:0/1284301427 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c0106e0 con 0x7f853c0aa0e0 2024-01-24T13:59:31.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.893+0000 7f8523fff700 1 -- 172.21.15.29:0/1284301427 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c010a60 con 0x7f853c0aa0e0 2024-01-24T13:59:31.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.893+0000 7f8542687700 1 -- 172.21.15.29:0/1284301427 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853c0c2050 con 0x7f853c0aa0e0 2024-01-24T13:59:31.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.893+0000 7f8542687700 1 -- 172.21.15.29:0/1284301427 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c0c2610 con 0x7f853c0aa0e0 2024-01-24T13:59:31.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.894+0000 7f8542687700 1 -- 172.21.15.29:0/1284301427 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853c0c2c60 con 0x7f853c0aa0e0 2024-01-24T13:59:31.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.894+0000 7f8523fff700 1 -- 172.21.15.29:0/1284301427 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f852c01b460 con 0x7f853c0aa0e0 2024-01-24T13:59:31.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.895+0000 7f8523fff700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f852805f150 0x7f8528061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:31.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.895+0000 7f8523fff700 1 -- 172.21.15.29:0/1284301427 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f852c07a330 con 0x7f853c0aa0e0 2024-01-24T13:59:31.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.895+0000 7f853b7fe700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f852805f150 0x7f8528061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:31.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.895+0000 7f853b7fe700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f852805f150 0x7f8528061610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f8530006fd0 tx=0x7f8530017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:31.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:31.899+0000 7f8523fff700 1 -- 172.21.15.29:0/1284301427 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f852c04b4a0 con 0x7f853c0aa0e0 2024-01-24T13:59:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.077+0000 7f8542687700 1 -- 172.21.15.29:0/1284301427 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f853c0c6ae0 con 0x7f852805f150 2024-01-24T13:59:32.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.088+0000 7f8523fff700 1 -- 172.21.15.29:0/1284301427 <== mgr.4103 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361194 (secure 0 0 0) 0x7f853000bea0 con 0x7f852805f150 2024-01-24T13:59:32.093 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:32.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.092+0000 7f8542687700 1 -- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f852805f150 msgr2=0x7f8528061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:32.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.092+0000 7f8542687700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f852805f150 0x7f8528061610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f8530006fd0 tx=0x7f8530017000).stop 2024-01-24T13:59:32.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.092+0000 7f8542687700 1 -- 172.21.15.29:0/1284301427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f853c0aa0e0 msgr2=0x7f853c0c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:32.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.092+0000 7f8542687700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f853c0aa0e0 0x7f853c0c2e10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f852c00c650 tx=0x7f852c008040).stop 2024-01-24T13:59:32.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.092+0000 7f8542687700 1 -- 172.21.15.29:0/1284301427 shutdown_connections 2024-01-24T13:59:32.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.092+0000 7f8542687700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f853c154370 0x7f853c0c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:32.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.092+0000 7f8542687700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f852805f150 0x7f8528061610 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:32.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.092+0000 7f8542687700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f853c0aa0e0 0x7f853c0c2e10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:32.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.092+0000 7f8542687700 1 --2- 172.21.15.29:0/1284301427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f853c0ab1f0 0x7f853c0c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:32.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.092+0000 7f8542687700 1 -- 172.21.15.29:0/1284301427 >> 172.21.15.29:0/1284301427 conn(0x7f853c05ec80 msgr2=0x7f853c0636c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:32.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.093+0000 7f8542687700 1 -- 172.21.15.29:0/1284301427 shutdown_connections 2024-01-24T13:59:32.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:32.093+0000 7f8542687700 1 -- 172.21.15.29:0/1284301427 wait complete. 2024-01-24T13:59:32.097 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T13:59:32.106 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-01-24T13:59:31.742742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380492,"kb_used_data":3724,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752594228,"statfs":{"total":773094113280,"available":770656489472,"internally_reserved":0,"allocated":3813376,"data_stored":1717918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4683,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-22,"num_write_kb":-14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.466664"},"pg_stats":[{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392969+0000","last_change":"2024-01-24T13:59:26.387498+0000","last_active":"2024-01-24T13:59:26.392969+0000","last_peered":"2024-01-24T13:59:26.392969+0000","last_clean":"2024-01-24T13:59:26.392969+0000","last_became_active":"2024-01-24T13:59:26.387265+0000","last_became_peered":"2024-01-24T13:59:26.387265+0000","last_unstale":"2024-01-24T13:59:26.392969+0000","last_undegraded":"2024-01-24T13:59:26.392969+0000","last_fullsized":"2024-01-24T13:59:26.392969+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"40'3","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.403510+0000","last_change":"2024-01-24T13:59:25.981703+0000","last_active":"2024-01-24T13:59:26.403510+0000","last_peered":"2024-01-24T13:59:26.403510+0000","last_clean":"2024-01-24T13:59:26.403510+0000","last_became_active":"2024-01-24T13:59:25.981389+0000","last_became_peered":"2024-01-24T13:59:25.981389+0000","last_unstale":"2024-01-24T13:59:26.403510+0000","last_undegraded":"2024-01-24T13:59:26.403510+0000","last_fullsized":"2024-01-24T13:59:26.403510+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405281+0000","last_change":"2024-01-24T13:59:26.404051+0000","last_active":"2024-01-24T13:59:26.405281+0000","last_peered":"2024-01-24T13:59:26.405281+0000","last_clean":"2024-01-24T13:59:26.405281+0000","last_became_active":"2024-01-24T13:59:26.403766+0000","last_became_peered":"2024-01-24T13:59:26.403766+0000","last_unstale":"2024-01-24T13:59:26.405281+0000","last_undegraded":"2024-01-24T13:59:26.405281+0000","last_fullsized":"2024-01-24T13:59:26.405281+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.374771+0000","last_change":"2024-01-24T13:59:25.959944+0000","last_active":"2024-01-24T13:59:26.374771+0000","last_peered":"2024-01-24T13:59:26.374771+0000","last_clean":"2024-01-24T13:59:26.374771+0000","last_became_active":"2024-01-24T13:59:25.959636+0000","last_became_peered":"2024-01-24T13:59:25.959636+0000","last_unstale":"2024-01-24T13:59:26.374771+0000","last_undegraded":"2024-01-24T13:59:26.374771+0000","last_fullsized":"2024-01-24T13:59:26.374771+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394881+0000","last_change":"2024-01-24T13:59:26.394881+0000","last_active":"2024-01-24T13:59:26.394881+0000","last_peered":"2024-01-24T13:59:26.394881+0000","last_clean":"2024-01-24T13:59:26.394881+0000","last_became_active":"2024-01-24T13:59:26.394600+0000","last_became_peered":"2024-01-24T13:59:26.394600+0000","last_unstale":"2024-01-24T13:59:26.394881+0000","last_undegraded":"2024-01-24T13:59:26.394881+0000","last_fullsized":"2024-01-24T13:59:26.394881+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.982279+0000","last_change":"2024-01-24T13:59:25.980665+0000","last_active":"2024-01-24T13:59:25.982279+0000","last_peered":"2024-01-24T13:59:25.982279+0000","last_clean":"2024-01-24T13:59:25.982279+0000","last_became_active":"2024-01-24T13:59:25.980288+0000","last_became_peered":"2024-01-24T13:59:25.980288+0000","last_unstale":"2024-01-24T13:59:25.982279+0000","last_undegraded":"2024-01-24T13:59:25.982279+0000","last_fullsized":"2024-01-24T13:59:25.982279+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401782+0000","last_change":"2024-01-24T13:59:26.399272+0000","last_active":"2024-01-24T13:59:26.401782+0000","last_peered":"2024-01-24T13:59:26.401782+0000","last_clean":"2024-01-24T13:59:26.401782+0000","last_became_active":"2024-01-24T13:59:26.398992+0000","last_became_peered":"2024-01-24T13:59:26.398992+0000","last_unstale":"2024-01-24T13:59:26.401782+0000","last_undegraded":"2024-01-24T13:59:26.401782+0000","last_fullsized":"2024-01-24T13:59:26.401782+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"40'2","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404193+0000","last_change":"2024-01-24T13:59:25.970538+0000","last_active":"2024-01-24T13:59:26.404193+0000","last_peered":"2024-01-24T13:59:26.404193+0000","last_clean":"2024-01-24T13:59:26.404193+0000","last_became_active":"2024-01-24T13:59:25.969372+0000","last_became_peered":"2024-01-24T13:59:25.969372+0000","last_unstale":"2024-01-24T13:59:26.404193+0000","last_undegraded":"2024-01-24T13:59:26.404193+0000","last_fullsized":"2024-01-24T13:59:26.404193+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.402110+0000","last_change":"2024-01-24T13:59:26.396337+0000","last_active":"2024-01-24T13:59:26.402110+0000","last_peered":"2024-01-24T13:59:26.402110+0000","last_clean":"2024-01-24T13:59:26.402110+0000","last_became_active":"2024-01-24T13:59:26.396019+0000","last_became_peered":"2024-01-24T13:59:26.396019+0000","last_unstale":"2024-01-24T13:59:26.402110+0000","last_undegraded":"2024-01-24T13:59:26.402110+0000","last_fullsized":"2024-01-24T13:59:26.402110+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3b","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.400837+0000","last_change":"2024-01-24T13:59:25.969469+0000","last_active":"2024-01-24T13:59:26.400837+0000","last_peered":"2024-01-24T13:59:26.400837+0000","last_clean":"2024-01-24T13:59:26.400837+0000","last_became_active":"2024-01-24T13:59:25.969150+0000","last_became_peered":"2024-01-24T13:59:25.969150+0000","last_unstale":"2024-01-24T13:59:26.400837+0000","last_undegraded":"2024-01-24T13:59:26.400837+0000","last_fullsized":"2024-01-24T13:59:26.400837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.400961+0000","last_change":"2024-01-24T13:59:26.398010+0000","last_active":"2024-01-24T13:59:26.400961+0000","last_peered":"2024-01-24T13:59:26.400961+0000","last_clean":"2024-01-24T13:59:26.400961+0000","last_became_active":"2024-01-24T13:59:26.397688+0000","last_became_peered":"2024-01-24T13:59:26.397688+0000","last_unstale":"2024-01-24T13:59:26.400961+0000","last_undegraded":"2024-01-24T13:59:26.400961+0000","last_fullsized":"2024-01-24T13:59:26.400961+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3a","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394894+0000","last_change":"2024-01-24T13:59:25.963374+0000","last_active":"2024-01-24T13:59:26.394894+0000","last_peered":"2024-01-24T13:59:26.394894+0000","last_clean":"2024-01-24T13:59:26.394894+0000","last_became_active":"2024-01-24T13:59:25.963084+0000","last_became_peered":"2024-01-24T13:59:25.963084+0000","last_unstale":"2024-01-24T13:59:26.394894+0000","last_undegraded":"2024-01-24T13:59:26.394894+0000","last_fullsized":"2024-01-24T13:59:26.394894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.397603+0000","last_change":"2024-01-24T13:59:26.397603+0000","last_active":"2024-01-24T13:59:26.397603+0000","last_peered":"2024-01-24T13:59:26.397603+0000","last_clean":"2024-01-24T13:59:26.397603+0000","last_became_active":"2024-01-24T13:59:26.397269+0000","last_became_peered":"2024-01-24T13:59:26.397269+0000","last_unstale":"2024-01-24T13:59:26.397603+0000","last_undegraded":"2024-01-24T13:59:26.397603+0000","last_fullsized":"2024-01-24T13:59:26.397603+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377433+0000","last_change":"2024-01-24T13:59:25.976064+0000","last_active":"2024-01-24T13:59:26.377433+0000","last_peered":"2024-01-24T13:59:26.377433+0000","last_clean":"2024-01-24T13:59:26.377433+0000","last_became_active":"2024-01-24T13:59:25.975796+0000","last_became_peered":"2024-01-24T13:59:25.975796+0000","last_unstale":"2024-01-24T13:59:26.377433+0000","last_undegraded":"2024-01-24T13:59:26.377433+0000","last_fullsized":"2024-01-24T13:59:26.377433+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.400915+0000","last_change":"2024-01-24T13:59:26.400018+0000","last_active":"2024-01-24T13:59:26.400915+0000","last_peered":"2024-01-24T13:59:26.400915+0000","last_clean":"2024-01-24T13:59:26.400915+0000","last_became_active":"2024-01-24T13:59:26.399720+0000","last_became_peered":"2024-01-24T13:59:26.399720+0000","last_unstale":"2024-01-24T13:59:26.400915+0000","last_undegraded":"2024-01-24T13:59:26.400915+0000","last_fullsized":"2024-01-24T13:59:26.400915+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.374033+0000","last_change":"2024-01-24T13:59:25.976014+0000","last_active":"2024-01-24T13:59:26.374033+0000","last_peered":"2024-01-24T13:59:26.374033+0000","last_clean":"2024-01-24T13:59:26.374033+0000","last_became_active":"2024-01-24T13:59:25.975677+0000","last_became_peered":"2024-01-24T13:59:25.975677+0000","last_unstale":"2024-01-24T13:59:26.374033+0000","last_undegraded":"2024-01-24T13:59:26.374033+0000","last_fullsized":"2024-01-24T13:59:26.374033+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391986+0000","last_change":"2024-01-24T13:59:26.391986+0000","last_active":"2024-01-24T13:59:26.391986+0000","last_peered":"2024-01-24T13:59:26.391986+0000","last_clean":"2024-01-24T13:59:26.391986+0000","last_became_active":"2024-01-24T13:59:26.391698+0000","last_became_peered":"2024-01-24T13:59:26.391698+0000","last_unstale":"2024-01-24T13:59:26.391986+0000","last_undegraded":"2024-01-24T13:59:26.391986+0000","last_fullsized":"2024-01-24T13:59:26.391986+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.980747+0000","last_change":"2024-01-24T13:59:25.973798+0000","last_active":"2024-01-24T13:59:25.980747+0000","last_peered":"2024-01-24T13:59:25.980747+0000","last_clean":"2024-01-24T13:59:25.980747+0000","last_became_active":"2024-01-24T13:59:25.973506+0000","last_became_peered":"2024-01-24T13:59:25.973506+0000","last_unstale":"2024-01-24T13:59:25.980747+0000","last_undegraded":"2024-01-24T13:59:25.980747+0000","last_fullsized":"2024-01-24T13:59:25.980747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392777+0000","last_change":"2024-01-24T13:59:26.392777+0000","last_active":"2024-01-24T13:59:26.392777+0000","last_peered":"2024-01-24T13:59:26.392777+0000","last_clean":"2024-01-24T13:59:26.392777+0000","last_became_active":"2024-01-24T13:59:26.392496+0000","last_became_peered":"2024-01-24T13:59:26.392496+0000","last_unstale":"2024-01-24T13:59:26.392777+0000","last_undegraded":"2024-01-24T13:59:26.392777+0000","last_fullsized":"2024-01-24T13:59:26.392777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.380077+0000","last_change":"2024-01-24T13:59:25.974274+0000","last_active":"2024-01-24T13:59:26.380077+0000","last_peered":"2024-01-24T13:59:26.380077+0000","last_clean":"2024-01-24T13:59:26.380077+0000","last_became_active":"2024-01-24T13:59:25.973038+0000","last_became_peered":"2024-01-24T13:59:25.973038+0000","last_unstale":"2024-01-24T13:59:26.380077+0000","last_undegraded":"2024-01-24T13:59:26.380077+0000","last_fullsized":"2024-01-24T13:59:26.380077+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406291+0000","last_change":"2024-01-24T13:59:26.403631+0000","last_active":"2024-01-24T13:59:26.406291+0000","last_peered":"2024-01-24T13:59:26.406291+0000","last_clean":"2024-01-24T13:59:26.406291+0000","last_became_active":"2024-01-24T13:59:26.403208+0000","last_became_peered":"2024-01-24T13:59:26.403208+0000","last_unstale":"2024-01-24T13:59:26.406291+0000","last_undegraded":"2024-01-24T13:59:26.406291+0000","last_fullsized":"2024-01-24T13:59:26.406291+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377480+0000","last_change":"2024-01-24T13:59:25.975097+0000","last_active":"2024-01-24T13:59:26.377480+0000","last_peered":"2024-01-24T13:59:26.377480+0000","last_clean":"2024-01-24T13:59:26.377480+0000","last_became_active":"2024-01-24T13:59:25.973628+0000","last_became_peered":"2024-01-24T13:59:25.973628+0000","last_unstale":"2024-01-24T13:59:26.377480+0000","last_undegraded":"2024-01-24T13:59:26.377480+0000","last_fullsized":"2024-01-24T13:59:26.377480+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394489+0000","last_change":"2024-01-24T13:59:26.394489+0000","last_active":"2024-01-24T13:59:26.394489+0000","last_peered":"2024-01-24T13:59:26.394489+0000","last_clean":"2024-01-24T13:59:26.394489+0000","last_became_active":"2024-01-24T13:59:26.392243+0000","last_became_peered":"2024-01-24T13:59:26.392243+0000","last_unstale":"2024-01-24T13:59:26.394489+0000","last_undegraded":"2024-01-24T13:59:26.394489+0000","last_fullsized":"2024-01-24T13:59:26.394489+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"40'4","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404373+0000","last_change":"2024-01-24T13:59:25.973968+0000","last_active":"2024-01-24T13:59:26.404373+0000","last_peered":"2024-01-24T13:59:26.404373+0000","last_clean":"2024-01-24T13:59:26.404373+0000","last_became_active":"2024-01-24T13:59:25.973675+0000","last_became_peered":"2024-01-24T13:59:25.973675+0000","last_unstale":"2024-01-24T13:59:26.404373+0000","last_undegraded":"2024-01-24T13:59:26.404373+0000","last_fullsized":"2024-01-24T13:59:26.404373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.395811+0000","last_change":"2024-01-24T13:59:26.395811+0000","last_active":"2024-01-24T13:59:26.395811+0000","last_peered":"2024-01-24T13:59:26.395811+0000","last_clean":"2024-01-24T13:59:26.395811+0000","last_became_active":"2024-01-24T13:59:26.395486+0000","last_became_peered":"2024-01-24T13:59:26.395486+0000","last_unstale":"2024-01-24T13:59:26.395811+0000","last_undegraded":"2024-01-24T13:59:26.395811+0000","last_fullsized":"2024-01-24T13:59:26.395811+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.398344+0000","last_change":"2024-01-24T13:59:25.974319+0000","last_active":"2024-01-24T13:59:26.398344+0000","last_peered":"2024-01-24T13:59:26.398344+0000","last_clean":"2024-01-24T13:59:26.398344+0000","last_became_active":"2024-01-24T13:59:25.973872+0000","last_became_peered":"2024-01-24T13:59:25.973872+0000","last_unstale":"2024-01-24T13:59:26.398344+0000","last_undegraded":"2024-01-24T13:59:26.398344+0000","last_fullsized":"2024-01-24T13:59:26.398344+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405408+0000","last_change":"2024-01-24T13:59:26.397845+0000","last_active":"2024-01-24T13:59:26.405408+0000","last_peered":"2024-01-24T13:59:26.405408+0000","last_clean":"2024-01-24T13:59:26.405408+0000","last_became_active":"2024-01-24T13:59:26.397440+0000","last_became_peered":"2024-01-24T13:59:26.397440+0000","last_unstale":"2024-01-24T13:59:26.405408+0000","last_undegraded":"2024-01-24T13:59:26.405408+0000","last_fullsized":"2024-01-24T13:59:26.405408+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.378557+0000","last_change":"2024-01-24T13:59:25.975011+0000","last_active":"2024-01-24T13:59:26.378557+0000","last_peered":"2024-01-24T13:59:26.378557+0000","last_clean":"2024-01-24T13:59:26.378557+0000","last_became_active":"2024-01-24T13:59:25.974761+0000","last_became_peered":"2024-01-24T13:59:25.974761+0000","last_unstale":"2024-01-24T13:59:26.378557+0000","last_undegraded":"2024-01-24T13:59:26.378557+0000","last_fullsized":"2024-01-24T13:59:26.378557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406129+0000","last_change":"2024-01-24T13:59:26.399451+0000","last_active":"2024-01-24T13:59:26.406129+0000","last_peered":"2024-01-24T13:59:26.406129+0000","last_clean":"2024-01-24T13:59:26.406129+0000","last_became_active":"2024-01-24T13:59:26.398728+0000","last_became_peered":"2024-01-24T13:59:26.398728+0000","last_unstale":"2024-01-24T13:59:26.406129+0000","last_undegraded":"2024-01-24T13:59:26.406129+0000","last_fullsized":"2024-01-24T13:59:26.406129+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377543+0000","last_change":"2024-01-24T13:59:25.975767+0000","last_active":"2024-01-24T13:59:26.377543+0000","last_peered":"2024-01-24T13:59:26.377543+0000","last_clean":"2024-01-24T13:59:26.377543+0000","last_became_active":"2024-01-24T13:59:25.975467+0000","last_became_peered":"2024-01-24T13:59:25.975467+0000","last_unstale":"2024-01-24T13:59:26.377543+0000","last_undegraded":"2024-01-24T13:59:26.377543+0000","last_fullsized":"2024-01-24T13:59:26.377543+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391296+0000","last_change":"2024-01-24T13:59:26.383140+0000","last_active":"2024-01-24T13:59:26.391296+0000","last_peered":"2024-01-24T13:59:26.391296+0000","last_clean":"2024-01-24T13:59:26.391296+0000","last_became_active":"2024-01-24T13:59:26.382851+0000","last_became_peered":"2024-01-24T13:59:26.382851+0000","last_unstale":"2024-01-24T13:59:26.391296+0000","last_undegraded":"2024-01-24T13:59:26.391296+0000","last_fullsized":"2024-01-24T13:59:26.391296+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"40'4","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.398904+0000","last_change":"2024-01-24T13:59:25.977558+0000","last_active":"2024-01-24T13:59:26.398904+0000","last_peered":"2024-01-24T13:59:26.398904+0000","last_clean":"2024-01-24T13:59:26.398904+0000","last_became_active":"2024-01-24T13:59:25.977281+0000","last_became_peered":"2024-01-24T13:59:25.977281+0000","last_unstale":"2024-01-24T13:59:26.398904+0000","last_undegraded":"2024-01-24T13:59:26.398904+0000","last_fullsized":"2024-01-24T13:59:26.398904+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393550+0000","last_change":"2024-01-24T13:59:26.388561+0000","last_active":"2024-01-24T13:59:26.393550+0000","last_peered":"2024-01-24T13:59:26.393550+0000","last_clean":"2024-01-24T13:59:26.393550+0000","last_became_active":"2024-01-24T13:59:26.388275+0000","last_became_peered":"2024-01-24T13:59:26.388275+0000","last_unstale":"2024-01-24T13:59:26.393550+0000","last_undegraded":"2024-01-24T13:59:26.393550+0000","last_fullsized":"2024-01-24T13:59:26.393550+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394705+0000","last_change":"2024-01-24T13:59:25.973398+0000","last_active":"2024-01-24T13:59:26.394705+0000","last_peered":"2024-01-24T13:59:26.394705+0000","last_clean":"2024-01-24T13:59:26.394705+0000","last_became_active":"2024-01-24T13:59:25.973098+0000","last_became_peered":"2024-01-24T13:59:25.973098+0000","last_unstale":"2024-01-24T13:59:26.394705+0000","last_undegraded":"2024-01-24T13:59:26.394705+0000","last_fullsized":"2024-01-24T13:59:26.394705+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393371+0000","last_change":"2024-01-24T13:59:26.389534+0000","last_active":"2024-01-24T13:59:26.393371+0000","last_peered":"2024-01-24T13:59:26.393371+0000","last_clean":"2024-01-24T13:59:26.393371+0000","last_became_active":"2024-01-24T13:59:26.388756+0000","last_became_peered":"2024-01-24T13:59:26.388756+0000","last_unstale":"2024-01-24T13:59:26.393371+0000","last_undegraded":"2024-01-24T13:59:26.393371+0000","last_fullsized":"2024-01-24T13:59:26.393371+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.387025+0000","last_change":"2024-01-24T13:59:25.968204+0000","last_active":"2024-01-24T13:59:26.387025+0000","last_peered":"2024-01-24T13:59:26.387025+0000","last_clean":"2024-01-24T13:59:26.387025+0000","last_became_active":"2024-01-24T13:59:25.967654+0000","last_became_peered":"2024-01-24T13:59:25.967654+0000","last_unstale":"2024-01-24T13:59:26.387025+0000","last_undegraded":"2024-01-24T13:59:26.387025+0000","last_fullsized":"2024-01-24T13:59:26.387025+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.408225+0000","last_change":"2024-01-24T13:59:26.402355+0000","last_active":"2024-01-24T13:59:26.408225+0000","last_peered":"2024-01-24T13:59:26.408225+0000","last_clean":"2024-01-24T13:59:26.408225+0000","last_became_active":"2024-01-24T13:59:26.401991+0000","last_became_peered":"2024-01-24T13:59:26.401991+0000","last_unstale":"2024-01-24T13:59:26.408225+0000","last_undegraded":"2024-01-24T13:59:26.408225+0000","last_fullsized":"2024-01-24T13:59:26.408225+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2d","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399805+0000","last_change":"2024-01-24T13:59:25.960337+0000","last_active":"2024-01-24T13:59:26.399805+0000","last_peered":"2024-01-24T13:59:26.399805+0000","last_clean":"2024-01-24T13:59:26.399805+0000","last_became_active":"2024-01-24T13:59:25.960030+0000","last_became_peered":"2024-01-24T13:59:25.960030+0000","last_unstale":"2024-01-24T13:59:26.399805+0000","last_undegraded":"2024-01-24T13:59:26.399805+0000","last_fullsized":"2024-01-24T13:59:26.399805+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406353+0000","last_change":"2024-01-24T13:59:26.403095+0000","last_active":"2024-01-24T13:59:26.406353+0000","last_peered":"2024-01-24T13:59:26.406353+0000","last_clean":"2024-01-24T13:59:26.406353+0000","last_became_active":"2024-01-24T13:59:26.402795+0000","last_became_peered":"2024-01-24T13:59:26.402795+0000","last_unstale":"2024-01-24T13:59:26.406353+0000","last_undegraded":"2024-01-24T13:59:26.406353+0000","last_fullsized":"2024-01-24T13:59:26.406353+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.386554+0000","last_change":"2024-01-24T13:59:25.968410+0000","last_active":"2024-01-24T13:59:26.386554+0000","last_peered":"2024-01-24T13:59:26.386554+0000","last_clean":"2024-01-24T13:59:26.386554+0000","last_became_active":"2024-01-24T13:59:25.967771+0000","last_became_peered":"2024-01-24T13:59:25.967771+0000","last_unstale":"2024-01-24T13:59:26.386554+0000","last_undegraded":"2024-01-24T13:59:26.386554+0000","last_fullsized":"2024-01-24T13:59:26.386554+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393452+0000","last_change":"2024-01-24T13:59:26.393452+0000","last_active":"2024-01-24T13:59:26.393452+0000","last_peered":"2024-01-24T13:59:26.393452+0000","last_clean":"2024-01-24T13:59:26.393452+0000","last_became_active":"2024-01-24T13:59:26.393184+0000","last_became_peered":"2024-01-24T13:59:26.393184+0000","last_unstale":"2024-01-24T13:59:26.393452+0000","last_undegraded":"2024-01-24T13:59:26.393452+0000","last_fullsized":"2024-01-24T13:59:26.393452+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.398429+0000","last_change":"2024-01-24T13:59:25.974128+0000","last_active":"2024-01-24T13:59:26.398429+0000","last_peered":"2024-01-24T13:59:26.398429+0000","last_clean":"2024-01-24T13:59:26.398429+0000","last_became_active":"2024-01-24T13:59:25.973551+0000","last_became_peered":"2024-01-24T13:59:25.973551+0000","last_unstale":"2024-01-24T13:59:26.398429+0000","last_undegraded":"2024-01-24T13:59:26.398429+0000","last_fullsized":"2024-01-24T13:59:26.398429+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.397788+0000","last_change":"2024-01-24T13:59:26.393112+0000","last_active":"2024-01-24T13:59:26.397788+0000","last_peered":"2024-01-24T13:59:26.397788+0000","last_clean":"2024-01-24T13:59:26.397788+0000","last_became_active":"2024-01-24T13:59:26.392817+0000","last_became_peered":"2024-01-24T13:59:26.392817+0000","last_unstale":"2024-01-24T13:59:26.397788+0000","last_undegraded":"2024-01-24T13:59:26.397788+0000","last_fullsized":"2024-01-24T13:59:26.397788+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.378512+0000","last_change":"2024-01-24T13:59:25.963330+0000","last_active":"2024-01-24T13:59:26.378512+0000","last_peered":"2024-01-24T13:59:26.378512+0000","last_clean":"2024-01-24T13:59:26.378512+0000","last_became_active":"2024-01-24T13:59:25.963008+0000","last_became_peered":"2024-01-24T13:59:25.963008+0000","last_unstale":"2024-01-24T13:59:26.378512+0000","last_undegraded":"2024-01-24T13:59:26.378512+0000","last_fullsized":"2024-01-24T13:59:26.378512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.397984+0000","last_change":"2024-01-24T13:59:26.395109+0000","last_active":"2024-01-24T13:59:26.397984+0000","last_peered":"2024-01-24T13:59:26.397984+0000","last_clean":"2024-01-24T13:59:26.397984+0000","last_became_active":"2024-01-24T13:59:26.394738+0000","last_became_peered":"2024-01-24T13:59:26.394738+0000","last_unstale":"2024-01-24T13:59:26.397984+0000","last_undegraded":"2024-01-24T13:59:26.397984+0000","last_fullsized":"2024-01-24T13:59:26.397984+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392496+0000","last_change":"2024-01-24T13:59:25.971058+0000","last_active":"2024-01-24T13:59:26.392496+0000","last_peered":"2024-01-24T13:59:26.392496+0000","last_clean":"2024-01-24T13:59:26.392496+0000","last_became_active":"2024-01-24T13:59:25.970479+0000","last_became_peered":"2024-01-24T13:59:25.970479+0000","last_unstale":"2024-01-24T13:59:26.392496+0000","last_undegraded":"2024-01-24T13:59:26.392496+0000","last_fullsized":"2024-01-24T13:59:26.392496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401679+0000","last_change":"2024-01-24T13:59:26.396728+0000","last_active":"2024-01-24T13:59:26.401679+0000","last_peered":"2024-01-24T13:59:26.401679+0000","last_clean":"2024-01-24T13:59:26.401679+0000","last_became_active":"2024-01-24T13:59:26.396409+0000","last_became_peered":"2024-01-24T13:59:26.396409+0000","last_unstale":"2024-01-24T13:59:26.401679+0000","last_undegraded":"2024-01-24T13:59:26.401679+0000","last_fullsized":"2024-01-24T13:59:26.401679+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.390490+0000","last_change":"2024-01-24T13:59:25.971237+0000","last_active":"2024-01-24T13:59:26.390490+0000","last_peered":"2024-01-24T13:59:26.390490+0000","last_clean":"2024-01-24T13:59:26.390490+0000","last_became_active":"2024-01-24T13:59:25.970754+0000","last_became_peered":"2024-01-24T13:59:25.970754+0000","last_unstale":"2024-01-24T13:59:26.390490+0000","last_undegraded":"2024-01-24T13:59:26.390490+0000","last_fullsized":"2024-01-24T13:59:26.390490+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405458+0000","last_change":"2024-01-24T13:59:26.400578+0000","last_active":"2024-01-24T13:59:26.405458+0000","last_peered":"2024-01-24T13:59:26.405458+0000","last_clean":"2024-01-24T13:59:26.405458+0000","last_became_active":"2024-01-24T13:59:26.400281+0000","last_became_peered":"2024-01-24T13:59:26.400281+0000","last_unstale":"2024-01-24T13:59:26.405458+0000","last_undegraded":"2024-01-24T13:59:26.405458+0000","last_fullsized":"2024-01-24T13:59:26.405458+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.386714+0000","last_change":"2024-01-24T13:59:25.981337+0000","last_active":"2024-01-24T13:59:26.386714+0000","last_peered":"2024-01-24T13:59:26.386714+0000","last_clean":"2024-01-24T13:59:26.386714+0000","last_became_active":"2024-01-24T13:59:25.981032+0000","last_became_peered":"2024-01-24T13:59:25.981032+0000","last_unstale":"2024-01-24T13:59:26.386714+0000","last_undegraded":"2024-01-24T13:59:26.386714+0000","last_fullsized":"2024-01-24T13:59:26.386714+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406207+0000","last_change":"2024-01-24T13:59:26.402079+0000","last_active":"2024-01-24T13:59:26.406207+0000","last_peered":"2024-01-24T13:59:26.406207+0000","last_clean":"2024-01-24T13:59:26.406207+0000","last_became_active":"2024-01-24T13:59:26.401766+0000","last_became_peered":"2024-01-24T13:59:26.401766+0000","last_unstale":"2024-01-24T13:59:26.406207+0000","last_undegraded":"2024-01-24T13:59:26.406207+0000","last_fullsized":"2024-01-24T13:59:26.406207+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"41'1","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-01-24T13:59:27.378370+0000","last_change":"2024-01-24T13:59:25.976117+0000","last_active":"2024-01-24T13:59:27.378370+0000","last_peered":"2024-01-24T13:59:27.378370+0000","last_clean":"2024-01-24T13:59:27.378370+0000","last_became_active":"2024-01-24T13:59:25.975814+0000","last_became_peered":"2024-01-24T13:59:25.975814+0000","last_unstale":"2024-01-24T13:59:27.378370+0000","last_undegraded":"2024-01-24T13:59:27.378370+0000","last_fullsized":"2024-01-24T13:59:27.378370+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406271+0000","last_change":"2024-01-24T13:59:26.403460+0000","last_active":"2024-01-24T13:59:26.406271+0000","last_peered":"2024-01-24T13:59:26.406271+0000","last_clean":"2024-01-24T13:59:26.406271+0000","last_became_active":"2024-01-24T13:59:26.402947+0000","last_became_peered":"2024-01-24T13:59:26.402947+0000","last_unstale":"2024-01-24T13:59:26.406271+0000","last_undegraded":"2024-01-24T13:59:26.406271+0000","last_fullsized":"2024-01-24T13:59:26.406271+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.386692+0000","last_change":"2024-01-24T13:59:25.972833+0000","last_active":"2024-01-24T13:59:26.386692+0000","last_peered":"2024-01-24T13:59:26.386692+0000","last_clean":"2024-01-24T13:59:26.386692+0000","last_became_active":"2024-01-24T13:59:25.972518+0000","last_became_peered":"2024-01-24T13:59:25.972518+0000","last_unstale":"2024-01-24T13:59:26.386692+0000","last_undegraded":"2024-01-24T13:59:26.386692+0000","last_fullsized":"2024-01-24T13:59:26.386692+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393401+0000","last_change":"2024-01-24T13:59:26.389080+0000","last_active":"2024-01-24T13:59:26.393401+0000","last_peered":"2024-01-24T13:59:26.393401+0000","last_clean":"2024-01-24T13:59:26.393401+0000","last_became_active":"2024-01-24T13:59:26.388810+0000","last_became_peered":"2024-01-24T13:59:26.388810+0000","last_unstale":"2024-01-24T13:59:26.393401+0000","last_undegraded":"2024-01-24T13:59:26.393401+0000","last_fullsized":"2024-01-24T13:59:26.393401+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.379785+0000","last_change":"2024-01-24T13:59:25.975851+0000","last_active":"2024-01-24T13:59:26.379785+0000","last_peered":"2024-01-24T13:59:26.379785+0000","last_clean":"2024-01-24T13:59:26.379785+0000","last_became_active":"2024-01-24T13:59:25.975545+0000","last_became_peered":"2024-01-24T13:59:25.975545+0000","last_unstale":"2024-01-24T13:59:26.379785+0000","last_undegraded":"2024-01-24T13:59:26.379785+0000","last_fullsized":"2024-01-24T13:59:26.379785+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401748+0000","last_change":"2024-01-24T13:59:26.396104+0000","last_active":"2024-01-24T13:59:26.401748+0000","last_peered":"2024-01-24T13:59:26.401748+0000","last_clean":"2024-01-24T13:59:26.401748+0000","last_became_active":"2024-01-24T13:59:26.395835+0000","last_became_peered":"2024-01-24T13:59:26.395835+0000","last_unstale":"2024-01-24T13:59:26.401748+0000","last_undegraded":"2024-01-24T13:59:26.401748+0000","last_fullsized":"2024-01-24T13:59:26.401748+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.386450+0000","last_change":"2024-01-24T13:59:25.970355+0000","last_active":"2024-01-24T13:59:26.386450+0000","last_peered":"2024-01-24T13:59:26.386450+0000","last_clean":"2024-01-24T13:59:26.386450+0000","last_became_active":"2024-01-24T13:59:25.970149+0000","last_became_peered":"2024-01-24T13:59:25.970149+0000","last_unstale":"2024-01-24T13:59:26.386450+0000","last_undegraded":"2024-01-24T13:59:26.386450+0000","last_fullsized":"2024-01-24T13:59:26.386450+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404636+0000","last_change":"2024-01-24T13:59:26.399244+0000","last_active":"2024-01-24T13:59:26.404636+0000","last_peered":"2024-01-24T13:59:26.404636+0000","last_clean":"2024-01-24T13:59:26.404636+0000","last_became_active":"2024-01-24T13:59:26.398962+0000","last_became_peered":"2024-01-24T13:59:26.398962+0000","last_unstale":"2024-01-24T13:59:26.404636+0000","last_undegraded":"2024-01-24T13:59:26.404636+0000","last_fullsized":"2024-01-24T13:59:26.404636+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.387268+0000","last_change":"2024-01-24T13:59:25.977273+0000","last_active":"2024-01-24T13:59:26.387268+0000","last_peered":"2024-01-24T13:59:26.387268+0000","last_clean":"2024-01-24T13:59:26.387268+0000","last_became_active":"2024-01-24T13:59:25.977009+0000","last_became_peered":"2024-01-24T13:59:25.977009+0000","last_unstale":"2024-01-24T13:59:26.387268+0000","last_undegraded":"2024-01-24T13:59:26.387268+0000","last_fullsized":"2024-01-24T13:59:26.387268+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405831+0000","last_change":"2024-01-24T13:59:26.400727+0000","last_active":"2024-01-24T13:59:26.405831+0000","last_peered":"2024-01-24T13:59:26.405831+0000","last_clean":"2024-01-24T13:59:26.405831+0000","last_became_active":"2024-01-24T13:59:26.400259+0000","last_became_peered":"2024-01-24T13:59:26.400259+0000","last_unstale":"2024-01-24T13:59:26.405831+0000","last_undegraded":"2024-01-24T13:59:26.405831+0000","last_fullsized":"2024-01-24T13:59:26.405831+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392726+0000","last_change":"2024-01-24T13:59:25.971227+0000","last_active":"2024-01-24T13:59:26.392726+0000","last_peered":"2024-01-24T13:59:26.392726+0000","last_clean":"2024-01-24T13:59:26.392726+0000","last_became_active":"2024-01-24T13:59:25.970788+0000","last_became_peered":"2024-01-24T13:59:25.970788+0000","last_unstale":"2024-01-24T13:59:26.392726+0000","last_undegraded":"2024-01-24T13:59:26.392726+0000","last_fullsized":"2024-01-24T13:59:26.392726+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.395976+0000","last_change":"2024-01-24T13:59:26.386354+0000","last_active":"2024-01-24T13:59:26.395976+0000","last_peered":"2024-01-24T13:59:26.395976+0000","last_clean":"2024-01-24T13:59:26.395976+0000","last_became_active":"2024-01-24T13:59:26.386059+0000","last_became_peered":"2024-01-24T13:59:26.386059+0000","last_unstale":"2024-01-24T13:59:26.395976+0000","last_undegraded":"2024-01-24T13:59:26.395976+0000","last_fullsized":"2024-01-24T13:59:26.395976+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377356+0000","last_change":"2024-01-24T13:59:25.962615+0000","last_active":"2024-01-24T13:59:26.377356+0000","last_peered":"2024-01-24T13:59:26.377356+0000","last_clean":"2024-01-24T13:59:26.377356+0000","last_became_active":"2024-01-24T13:59:25.962088+0000","last_became_peered":"2024-01-24T13:59:25.962088+0000","last_unstale":"2024-01-24T13:59:26.377356+0000","last_undegraded":"2024-01-24T13:59:26.377356+0000","last_fullsized":"2024-01-24T13:59:26.377356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391112+0000","last_change":"2024-01-24T13:59:26.381154+0000","last_active":"2024-01-24T13:59:26.391112+0000","last_peered":"2024-01-24T13:59:26.391112+0000","last_clean":"2024-01-24T13:59:26.391112+0000","last_became_active":"2024-01-24T13:59:26.380865+0000","last_became_peered":"2024-01-24T13:59:26.380865+0000","last_unstale":"2024-01-24T13:59:26.391112+0000","last_undegraded":"2024-01-24T13:59:26.391112+0000","last_fullsized":"2024-01-24T13:59:26.391112+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399808+0000","last_change":"2024-01-24T13:59:25.973394+0000","last_active":"2024-01-24T13:59:26.399808+0000","last_peered":"2024-01-24T13:59:26.399808+0000","last_clean":"2024-01-24T13:59:26.399808+0000","last_became_active":"2024-01-24T13:59:25.973125+0000","last_became_peered":"2024-01-24T13:59:25.973125+0000","last_unstale":"2024-01-24T13:59:26.399808+0000","last_undegraded":"2024-01-24T13:59:26.399808+0000","last_fullsized":"2024-01-24T13:59:26.399808+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406187+0000","last_change":"2024-01-24T13:59:26.401718+0000","last_active":"2024-01-24T13:59:26.406187+0000","last_peered":"2024-01-24T13:59:26.406187+0000","last_clean":"2024-01-24T13:59:26.406187+0000","last_became_active":"2024-01-24T13:59:26.398491+0000","last_became_peered":"2024-01-24T13:59:26.398491+0000","last_unstale":"2024-01-24T13:59:26.406187+0000","last_undegraded":"2024-01-24T13:59:26.406187+0000","last_fullsized":"2024-01-24T13:59:26.406187+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393110+0000","last_change":"2024-01-24T13:59:25.969038+0000","last_active":"2024-01-24T13:59:26.393110+0000","last_peered":"2024-01-24T13:59:26.393110+0000","last_clean":"2024-01-24T13:59:26.393110+0000","last_became_active":"2024-01-24T13:59:25.968181+0000","last_became_peered":"2024-01-24T13:59:25.968181+0000","last_unstale":"2024-01-24T13:59:26.393110+0000","last_undegraded":"2024-01-24T13:59:26.393110+0000","last_fullsized":"2024-01-24T13:59:26.393110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399805+0000","last_change":"2024-01-24T13:59:26.393972+0000","last_active":"2024-01-24T13:59:26.399805+0000","last_peered":"2024-01-24T13:59:26.399805+0000","last_clean":"2024-01-24T13:59:26.399805+0000","last_became_active":"2024-01-24T13:59:26.393647+0000","last_became_peered":"2024-01-24T13:59:26.393647+0000","last_unstale":"2024-01-24T13:59:26.399805+0000","last_undegraded":"2024-01-24T13:59:26.399805+0000","last_fullsized":"2024-01-24T13:59:26.399805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"40'2","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391442+0000","last_change":"2024-01-24T13:59:25.976311+0000","last_active":"2024-01-24T13:59:26.391442+0000","last_peered":"2024-01-24T13:59:26.391442+0000","last_clean":"2024-01-24T13:59:26.391442+0000","last_became_active":"2024-01-24T13:59:25.975906+0000","last_became_peered":"2024-01-24T13:59:25.975906+0000","last_unstale":"2024-01-24T13:59:26.391442+0000","last_undegraded":"2024-01-24T13:59:26.391442+0000","last_fullsized":"2024-01-24T13:59:26.391442+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399147+0000","last_change":"2024-01-24T13:59:26.394479+0000","last_active":"2024-01-24T13:59:26.399147+0000","last_peered":"2024-01-24T13:59:26.399147+0000","last_clean":"2024-01-24T13:59:26.399147+0000","last_became_active":"2024-01-24T13:59:26.393691+0000","last_became_peered":"2024-01-24T13:59:26.393691+0000","last_unstale":"2024-01-24T13:59:26.399147+0000","last_undegraded":"2024-01-24T13:59:26.399147+0000","last_fullsized":"2024-01-24T13:59:26.399147+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391670+0000","last_change":"2024-01-24T13:59:25.977532+0000","last_active":"2024-01-24T13:59:26.391670+0000","last_peered":"2024-01-24T13:59:26.391670+0000","last_clean":"2024-01-24T13:59:26.391670+0000","last_became_active":"2024-01-24T13:59:25.977226+0000","last_became_peered":"2024-01-24T13:59:25.977226+0000","last_unstale":"2024-01-24T13:59:26.391670+0000","last_undegraded":"2024-01-24T13:59:26.391670+0000","last_fullsized":"2024-01-24T13:59:26.391670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394539+0000","last_change":"2024-01-24T13:59:26.391562+0000","last_active":"2024-01-24T13:59:26.394539+0000","last_peered":"2024-01-24T13:59:26.394539+0000","last_clean":"2024-01-24T13:59:26.394539+0000","last_became_active":"2024-01-24T13:59:26.391262+0000","last_became_peered":"2024-01-24T13:59:26.391262+0000","last_unstale":"2024-01-24T13:59:26.394539+0000","last_undegraded":"2024-01-24T13:59:26.394539+0000","last_fullsized":"2024-01-24T13:59:26.394539+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.374913+0000","last_change":"2024-01-24T13:59:25.972598+0000","last_active":"2024-01-24T13:59:26.374913+0000","last_peered":"2024-01-24T13:59:26.374913+0000","last_clean":"2024-01-24T13:59:26.374913+0000","last_became_active":"2024-01-24T13:59:25.971998+0000","last_became_peered":"2024-01-24T13:59:25.971998+0000","last_unstale":"2024-01-24T13:59:26.374913+0000","last_undegraded":"2024-01-24T13:59:26.374913+0000","last_fullsized":"2024-01-24T13:59:26.374913+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399449+0000","last_change":"2024-01-24T13:59:26.399449+0000","last_active":"2024-01-24T13:59:26.399449+0000","last_peered":"2024-01-24T13:59:26.399449+0000","last_clean":"2024-01-24T13:59:26.399449+0000","last_became_active":"2024-01-24T13:59:26.399156+0000","last_became_peered":"2024-01-24T13:59:26.399156+0000","last_unstale":"2024-01-24T13:59:26.399449+0000","last_undegraded":"2024-01-24T13:59:26.399449+0000","last_fullsized":"2024-01-24T13:59:26.399449+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.378664+0000","last_change":"2024-01-24T13:59:25.976748+0000","last_active":"2024-01-24T13:59:26.378664+0000","last_peered":"2024-01-24T13:59:26.378664+0000","last_clean":"2024-01-24T13:59:26.378664+0000","last_became_active":"2024-01-24T13:59:25.976464+0000","last_became_peered":"2024-01-24T13:59:25.976464+0000","last_unstale":"2024-01-24T13:59:26.378664+0000","last_undegraded":"2024-01-24T13:59:26.378664+0000","last_fullsized":"2024-01-24T13:59:26.378664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.397561+0000","last_change":"2024-01-24T13:59:26.397561+0000","last_active":"2024-01-24T13:59:26.397561+0000","last_peered":"2024-01-24T13:59:26.397561+0000","last_clean":"2024-01-24T13:59:26.397561+0000","last_became_active":"2024-01-24T13:59:26.397284+0000","last_became_peered":"2024-01-24T13:59:26.397284+0000","last_unstale":"2024-01-24T13:59:26.397561+0000","last_undegraded":"2024-01-24T13:59:26.397561+0000","last_fullsized":"2024-01-24T13:59:26.397561+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377654+0000","last_change":"2024-01-24T13:59:25.968531+0000","last_active":"2024-01-24T13:59:26.377654+0000","last_peered":"2024-01-24T13:59:26.377654+0000","last_clean":"2024-01-24T13:59:26.377654+0000","last_became_active":"2024-01-24T13:59:25.968224+0000","last_became_peered":"2024-01-24T13:59:25.968224+0000","last_unstale":"2024-01-24T13:59:26.377654+0000","last_undegraded":"2024-01-24T13:59:26.377654+0000","last_fullsized":"2024-01-24T13:59:26.377654+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401658+0000","last_change":"2024-01-24T13:59:26.396396+0000","last_active":"2024-01-24T13:59:26.401658+0000","last_peered":"2024-01-24T13:59:26.401658+0000","last_clean":"2024-01-24T13:59:26.401658+0000","last_became_active":"2024-01-24T13:59:26.396120+0000","last_became_peered":"2024-01-24T13:59:26.396120+0000","last_unstale":"2024-01-24T13:59:26.401658+0000","last_undegraded":"2024-01-24T13:59:26.401658+0000","last_fullsized":"2024-01-24T13:59:26.401658+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.982939+0000","last_change":"2024-01-24T13:59:25.981443+0000","last_active":"2024-01-24T13:59:25.982939+0000","last_peered":"2024-01-24T13:59:25.982939+0000","last_clean":"2024-01-24T13:59:25.982939+0000","last_became_active":"2024-01-24T13:59:25.980669+0000","last_became_peered":"2024-01-24T13:59:25.980669+0000","last_unstale":"2024-01-24T13:59:25.982939+0000","last_undegraded":"2024-01-24T13:59:25.982939+0000","last_fullsized":"2024-01-24T13:59:25.982939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393530+0000","last_change":"2024-01-24T13:59:26.388542+0000","last_active":"2024-01-24T13:59:26.393530+0000","last_peered":"2024-01-24T13:59:26.393530+0000","last_clean":"2024-01-24T13:59:26.393530+0000","last_became_active":"2024-01-24T13:59:26.388191+0000","last_became_peered":"2024-01-24T13:59:26.388191+0000","last_unstale":"2024-01-24T13:59:26.393530+0000","last_undegraded":"2024-01-24T13:59:26.393530+0000","last_fullsized":"2024-01-24T13:59:26.393530+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.378863+0000","last_change":"2024-01-24T13:59:25.966156+0000","last_active":"2024-01-24T13:59:26.378863+0000","last_peered":"2024-01-24T13:59:26.378863+0000","last_clean":"2024-01-24T13:59:26.378863+0000","last_became_active":"2024-01-24T13:59:25.965913+0000","last_became_peered":"2024-01-24T13:59:25.965913+0000","last_unstale":"2024-01-24T13:59:26.378863+0000","last_undegraded":"2024-01-24T13:59:26.378863+0000","last_fullsized":"2024-01-24T13:59:26.378863+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404451+0000","last_change":"2024-01-24T13:59:26.400543+0000","last_active":"2024-01-24T13:59:26.404451+0000","last_peered":"2024-01-24T13:59:26.404451+0000","last_clean":"2024-01-24T13:59:26.404451+0000","last_became_active":"2024-01-24T13:59:26.400275+0000","last_became_peered":"2024-01-24T13:59:26.400275+0000","last_unstale":"2024-01-24T13:59:26.404451+0000","last_undegraded":"2024-01-24T13:59:26.404451+0000","last_fullsized":"2024-01-24T13:59:26.404451+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.981798+0000","last_change":"2024-01-24T13:59:25.980155+0000","last_active":"2024-01-24T13:59:25.981798+0000","last_peered":"2024-01-24T13:59:25.981798+0000","last_clean":"2024-01-24T13:59:25.981798+0000","last_became_active":"2024-01-24T13:59:25.979649+0000","last_became_peered":"2024-01-24T13:59:25.979649+0000","last_unstale":"2024-01-24T13:59:25.981798+0000","last_undegraded":"2024-01-24T13:59:25.981798+0000","last_fullsized":"2024-01-24T13:59:25.981798+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404515+0000","last_change":"2024-01-24T13:59:26.403157+0000","last_active":"2024-01-24T13:59:26.404515+0000","last_peered":"2024-01-24T13:59:26.404515+0000","last_clean":"2024-01-24T13:59:26.404515+0000","last_became_active":"2024-01-24T13:59:26.402754+0000","last_became_peered":"2024-01-24T13:59:26.402754+0000","last_unstale":"2024-01-24T13:59:26.404515+0000","last_undegraded":"2024-01-24T13:59:26.404515+0000","last_fullsized":"2024-01-24T13:59:26.404515+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"40'3","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406346+0000","last_change":"2024-01-24T13:59:25.976636+0000","last_active":"2024-01-24T13:59:26.406346+0000","last_peered":"2024-01-24T13:59:26.406346+0000","last_clean":"2024-01-24T13:59:26.406346+0000","last_became_active":"2024-01-24T13:59:25.976307+0000","last_became_peered":"2024-01-24T13:59:25.976307+0000","last_unstale":"2024-01-24T13:59:26.406346+0000","last_undegraded":"2024-01-24T13:59:26.406346+0000","last_fullsized":"2024-01-24T13:59:26.406346+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401022+0000","last_change":"2024-01-24T13:59:26.399966+0000","last_active":"2024-01-24T13:59:26.401022+0000","last_peered":"2024-01-24T13:59:26.401022+0000","last_clean":"2024-01-24T13:59:26.401022+0000","last_became_active":"2024-01-24T13:59:26.399686+0000","last_became_peered":"2024-01-24T13:59:26.399686+0000","last_unstale":"2024-01-24T13:59:26.401022+0000","last_undegraded":"2024-01-24T13:59:26.401022+0000","last_fullsized":"2024-01-24T13:59:26.401022+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"40'2","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404086+0000","last_change":"2024-01-24T13:59:25.966740+0000","last_active":"2024-01-24T13:59:26.404086+0000","last_peered":"2024-01-24T13:59:26.404086+0000","last_clean":"2024-01-24T13:59:26.404086+0000","last_became_active":"2024-01-24T13:59:25.966467+0000","last_became_peered":"2024-01-24T13:59:25.966467+0000","last_unstale":"2024-01-24T13:59:26.404086+0000","last_undegraded":"2024-01-24T13:59:26.404086+0000","last_fullsized":"2024-01-24T13:59:26.404086+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406145+0000","last_change":"2024-01-24T13:59:26.399470+0000","last_active":"2024-01-24T13:59:26.406145+0000","last_peered":"2024-01-24T13:59:26.406145+0000","last_clean":"2024-01-24T13:59:26.406145+0000","last_became_active":"2024-01-24T13:59:26.399162+0000","last_became_peered":"2024-01-24T13:59:26.399162+0000","last_unstale":"2024-01-24T13:59:26.406145+0000","last_undegraded":"2024-01-24T13:59:26.406145+0000","last_fullsized":"2024-01-24T13:59:26.406145+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.402986+0000","last_change":"2024-01-24T13:59:25.970390+0000","last_active":"2024-01-24T13:59:26.402986+0000","last_peered":"2024-01-24T13:59:26.402986+0000","last_clean":"2024-01-24T13:59:26.402986+0000","last_became_active":"2024-01-24T13:59:25.970070+0000","last_became_peered":"2024-01-24T13:59:25.970070+0000","last_unstale":"2024-01-24T13:59:26.402986+0000","last_undegraded":"2024-01-24T13:59:26.402986+0000","last_fullsized":"2024-01-24T13:59:26.402986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.395796+0000","last_change":"2024-01-24T13:59:26.395796+0000","last_active":"2024-01-24T13:59:26.395796+0000","last_peered":"2024-01-24T13:59:26.395796+0000","last_clean":"2024-01-24T13:59:26.395796+0000","last_became_active":"2024-01-24T13:59:26.395521+0000","last_became_peered":"2024-01-24T13:59:26.395521+0000","last_unstale":"2024-01-24T13:59:26.395796+0000","last_undegraded":"2024-01-24T13:59:26.395796+0000","last_fullsized":"2024-01-24T13:59:26.395796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.389216+0000","last_change":"2024-01-24T13:59:25.969417+0000","last_active":"2024-01-24T13:59:26.389216+0000","last_peered":"2024-01-24T13:59:26.389216+0000","last_clean":"2024-01-24T13:59:26.389216+0000","last_became_active":"2024-01-24T13:59:25.969077+0000","last_became_peered":"2024-01-24T13:59:25.969077+0000","last_unstale":"2024-01-24T13:59:26.389216+0000","last_undegraded":"2024-01-24T13:59:26.389216+0000","last_fullsized":"2024-01-24T13:59:26.389216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405168+0000","last_change":"2024-01-24T13:59:26.399161+0000","last_active":"2024-01-24T13:59:26.405168+0000","last_peered":"2024-01-24T13:59:26.405168+0000","last_clean":"2024-01-24T13:59:26.405168+0000","last_became_active":"2024-01-24T13:59:26.398825+0000","last_became_peered":"2024-01-24T13:59:26.398825+0000","last_unstale":"2024-01-24T13:59:26.405168+0000","last_undegraded":"2024-01-24T13:59:26.405168+0000","last_fullsized":"2024-01-24T13:59:26.405168+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.375222+0000","last_change":"2024-01-24T13:59:25.970509+0000","last_active":"2024-01-24T13:59:26.375222+0000","last_peered":"2024-01-24T13:59:26.375222+0000","last_clean":"2024-01-24T13:59:26.375222+0000","last_became_active":"2024-01-24T13:59:25.970193+0000","last_became_peered":"2024-01-24T13:59:25.970193+0000","last_unstale":"2024-01-24T13:59:26.375222+0000","last_undegraded":"2024-01-24T13:59:26.375222+0000","last_fullsized":"2024-01-24T13:59:26.375222+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405247+0000","last_change":"2024-01-24T13:59:26.402327+0000","last_active":"2024-01-24T13:59:26.405247+0000","last_peered":"2024-01-24T13:59:26.405247+0000","last_clean":"2024-01-24T13:59:26.405247+0000","last_became_active":"2024-01-24T13:59:26.402037+0000","last_became_peered":"2024-01-24T13:59:26.402037+0000","last_unstale":"2024-01-24T13:59:26.405247+0000","last_undegraded":"2024-01-24T13:59:26.405247+0000","last_fullsized":"2024-01-24T13:59:26.405247+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"40'4","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401238+0000","last_change":"2024-01-24T13:59:25.981003+0000","last_active":"2024-01-24T13:59:26.401238+0000","last_peered":"2024-01-24T13:59:26.401238+0000","last_clean":"2024-01-24T13:59:26.401238+0000","last_became_active":"2024-01-24T13:59:25.980309+0000","last_became_peered":"2024-01-24T13:59:25.980309+0000","last_unstale":"2024-01-24T13:59:26.401238+0000","last_undegraded":"2024-01-24T13:59:26.401238+0000","last_fullsized":"2024-01-24T13:59:26.401238+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405439+0000","last_change":"2024-01-24T13:59:26.400511+0000","last_active":"2024-01-24T13:59:26.405439+0000","last_peered":"2024-01-24T13:59:26.405439+0000","last_clean":"2024-01-24T13:59:26.405439+0000","last_became_active":"2024-01-24T13:59:26.399924+0000","last_became_peered":"2024-01-24T13:59:26.399924+0000","last_unstale":"2024-01-24T13:59:26.405439+0000","last_undegraded":"2024-01-24T13:59:26.405439+0000","last_fullsized":"2024-01-24T13:59:26.405439+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401946+0000","last_change":"2024-01-24T13:59:25.961023+0000","last_active":"2024-01-24T13:59:26.401946+0000","last_peered":"2024-01-24T13:59:26.401946+0000","last_clean":"2024-01-24T13:59:26.401946+0000","last_became_active":"2024-01-24T13:59:25.960727+0000","last_became_peered":"2024-01-24T13:59:25.960727+0000","last_unstale":"2024-01-24T13:59:26.401946+0000","last_undegraded":"2024-01-24T13:59:26.401946+0000","last_fullsized":"2024-01-24T13:59:26.401946+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392588+0000","last_change":"2024-01-24T13:59:26.392588+0000","last_active":"2024-01-24T13:59:26.392588+0000","last_peered":"2024-01-24T13:59:26.392588+0000","last_clean":"2024-01-24T13:59:26.392588+0000","last_became_active":"2024-01-24T13:59:26.392285+0000","last_became_peered":"2024-01-24T13:59:26.392285+0000","last_unstale":"2024-01-24T13:59:26.392588+0000","last_undegraded":"2024-01-24T13:59:26.392588+0000","last_fullsized":"2024-01-24T13:59:26.392588+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"40'2","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.398125+0000","last_change":"2024-01-24T13:59:25.960160+0000","last_active":"2024-01-24T13:59:26.398125+0000","last_peered":"2024-01-24T13:59:26.398125+0000","last_clean":"2024-01-24T13:59:26.398125+0000","last_became_active":"2024-01-24T13:59:25.959877+0000","last_became_peered":"2024-01-24T13:59:25.959877+0000","last_unstale":"2024-01-24T13:59:26.398125+0000","last_undegraded":"2024-01-24T13:59:26.398125+0000","last_fullsized":"2024-01-24T13:59:26.398125+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401697+0000","last_change":"2024-01-24T13:59:26.397070+0000","last_active":"2024-01-24T13:59:26.401697+0000","last_peered":"2024-01-24T13:59:26.401697+0000","last_clean":"2024-01-24T13:59:26.401697+0000","last_became_active":"2024-01-24T13:59:26.396674+0000","last_became_peered":"2024-01-24T13:59:26.396674+0000","last_unstale":"2024-01-24T13:59:26.401697+0000","last_undegraded":"2024-01-24T13:59:26.401697+0000","last_fullsized":"2024-01-24T13:59:26.401697+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377584+0000","last_change":"2024-01-24T13:59:25.973752+0000","last_active":"2024-01-24T13:59:26.377584+0000","last_peered":"2024-01-24T13:59:26.377584+0000","last_clean":"2024-01-24T13:59:26.377584+0000","last_became_active":"2024-01-24T13:59:25.973363+0000","last_became_peered":"2024-01-24T13:59:25.973363+0000","last_unstale":"2024-01-24T13:59:26.377584+0000","last_undegraded":"2024-01-24T13:59:26.377584+0000","last_fullsized":"2024-01-24T13:59:26.377584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406199+0000","last_change":"2024-01-24T13:59:26.401916+0000","last_active":"2024-01-24T13:59:26.406199+0000","last_peered":"2024-01-24T13:59:26.406199+0000","last_clean":"2024-01-24T13:59:26.406199+0000","last_became_active":"2024-01-24T13:59:26.399223+0000","last_became_peered":"2024-01-24T13:59:26.399223+0000","last_unstale":"2024-01-24T13:59:26.406199+0000","last_undegraded":"2024-01-24T13:59:26.406199+0000","last_fullsized":"2024-01-24T13:59:26.406199+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.374644+0000","last_change":"2024-01-24T13:59:25.973319+0000","last_active":"2024-01-24T13:59:26.374644+0000","last_peered":"2024-01-24T13:59:26.374644+0000","last_clean":"2024-01-24T13:59:26.374644+0000","last_became_active":"2024-01-24T13:59:25.973003+0000","last_became_peered":"2024-01-24T13:59:25.973003+0000","last_unstale":"2024-01-24T13:59:26.374644+0000","last_undegraded":"2024-01-24T13:59:26.374644+0000","last_fullsized":"2024-01-24T13:59:26.374644+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404309+0000","last_change":"2024-01-24T13:59:26.399449+0000","last_active":"2024-01-24T13:59:26.404309+0000","last_peered":"2024-01-24T13:59:26.404309+0000","last_clean":"2024-01-24T13:59:26.404309+0000","last_became_active":"2024-01-24T13:59:26.399170+0000","last_became_peered":"2024-01-24T13:59:26.399170+0000","last_unstale":"2024-01-24T13:59:26.404309+0000","last_undegraded":"2024-01-24T13:59:26.404309+0000","last_fullsized":"2024-01-24T13:59:26.404309+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393277+0000","last_change":"2024-01-24T13:59:25.972411+0000","last_active":"2024-01-24T13:59:26.393277+0000","last_peered":"2024-01-24T13:59:26.393277+0000","last_clean":"2024-01-24T13:59:26.393277+0000","last_became_active":"2024-01-24T13:59:25.971978+0000","last_became_peered":"2024-01-24T13:59:25.971978+0000","last_unstale":"2024-01-24T13:59:26.393277+0000","last_undegraded":"2024-01-24T13:59:26.393277+0000","last_fullsized":"2024-01-24T13:59:26.393277+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.398188+0000","last_change":"2024-01-24T13:59:26.391558+0000","last_active":"2024-01-24T13:59:26.398188+0000","last_peered":"2024-01-24T13:59:26.398188+0000","last_clean":"2024-01-24T13:59:26.398188+0000","last_became_active":"2024-01-24T13:59:26.391273+0000","last_became_peered":"2024-01-24T13:59:26.391273+0000","last_unstale":"2024-01-24T13:59:26.398188+0000","last_undegraded":"2024-01-24T13:59:26.398188+0000","last_fullsized":"2024-01-24T13:59:26.398188+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377125+0000","last_change":"2024-01-24T13:59:25.973110+0000","last_active":"2024-01-24T13:59:26.377125+0000","last_peered":"2024-01-24T13:59:26.377125+0000","last_clean":"2024-01-24T13:59:26.377125+0000","last_became_active":"2024-01-24T13:59:25.972708+0000","last_became_peered":"2024-01-24T13:59:25.972708+0000","last_unstale":"2024-01-24T13:59:26.377125+0000","last_undegraded":"2024-01-24T13:59:26.377125+0000","last_fullsized":"2024-01-24T13:59:26.377125+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.403487+0000","last_change":"2024-01-24T13:59:26.394039+0000","last_active":"2024-01-24T13:59:26.403487+0000","last_peered":"2024-01-24T13:59:26.403487+0000","last_clean":"2024-01-24T13:59:26.403487+0000","last_became_active":"2024-01-24T13:59:26.393742+0000","last_became_peered":"2024-01-24T13:59:26.393742+0000","last_unstale":"2024-01-24T13:59:26.403487+0000","last_undegraded":"2024-01-24T13:59:26.403487+0000","last_fullsized":"2024-01-24T13:59:26.403487+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.981204+0000","last_change":"2024-01-24T13:59:25.980225+0000","last_active":"2024-01-24T13:59:25.981204+0000","last_peered":"2024-01-24T13:59:25.981204+0000","last_clean":"2024-01-24T13:59:25.981204+0000","last_became_active":"2024-01-24T13:59:25.979876+0000","last_became_peered":"2024-01-24T13:59:25.979876+0000","last_unstale":"2024-01-24T13:59:25.981204+0000","last_undegraded":"2024-01-24T13:59:25.981204+0000","last_fullsized":"2024-01-24T13:59:25.981204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392651+0000","last_change":"2024-01-24T13:59:26.392651+0000","last_active":"2024-01-24T13:59:26.392651+0000","last_peered":"2024-01-24T13:59:26.392651+0000","last_clean":"2024-01-24T13:59:26.392651+0000","last_became_active":"2024-01-24T13:59:26.392338+0000","last_became_peered":"2024-01-24T13:59:26.392338+0000","last_unstale":"2024-01-24T13:59:26.392651+0000","last_undegraded":"2024-01-24T13:59:26.392651+0000","last_fullsized":"2024-01-24T13:59:26.392651+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.389155+0000","last_change":"2024-01-24T13:59:25.970207+0000","last_active":"2024-01-24T13:59:26.389155+0000","last_peered":"2024-01-24T13:59:26.389155+0000","last_clean":"2024-01-24T13:59:26.389155+0000","last_became_active":"2024-01-24T13:59:25.969920+0000","last_became_peered":"2024-01-24T13:59:25.969920+0000","last_unstale":"2024-01-24T13:59:26.389155+0000","last_undegraded":"2024-01-24T13:59:26.389155+0000","last_fullsized":"2024-01-24T13:59:26.389155+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391267+0000","last_change":"2024-01-24T13:59:26.382366+0000","last_active":"2024-01-24T13:59:26.391267+0000","last_peered":"2024-01-24T13:59:26.391267+0000","last_clean":"2024-01-24T13:59:26.391267+0000","last_became_active":"2024-01-24T13:59:26.380486+0000","last_became_peered":"2024-01-24T13:59:26.380486+0000","last_unstale":"2024-01-24T13:59:26.391267+0000","last_undegraded":"2024-01-24T13:59:26.391267+0000","last_fullsized":"2024-01-24T13:59:26.391267+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391904+0000","last_change":"2024-01-24T13:59:25.973150+0000","last_active":"2024-01-24T13:59:26.391904+0000","last_peered":"2024-01-24T13:59:26.391904+0000","last_clean":"2024-01-24T13:59:26.391904+0000","last_became_active":"2024-01-24T13:59:25.972845+0000","last_became_peered":"2024-01-24T13:59:25.972845+0000","last_unstale":"2024-01-24T13:59:26.391904+0000","last_undegraded":"2024-01-24T13:59:26.391904+0000","last_fullsized":"2024-01-24T13:59:26.391904+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.386798+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T13:59:26.386798+0000","last_peered":"2024-01-24T13:59:26.386798+0000","last_clean":"2024-01-24T13:59:26.386798+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T13:59:26.386798+0000","last_undegraded":"2024-01-24T13:59:26.386798+0000","last_fullsized":"2024-01-24T13:59:26.386798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404595+0000","last_change":"2024-01-24T13:59:26.399114+0000","last_active":"2024-01-24T13:59:26.404595+0000","last_peered":"2024-01-24T13:59:26.404595+0000","last_clean":"2024-01-24T13:59:26.404595+0000","last_became_active":"2024-01-24T13:59:26.398864+0000","last_became_peered":"2024-01-24T13:59:26.398864+0000","last_unstale":"2024-01-24T13:59:26.404595+0000","last_undegraded":"2024-01-24T13:59:26.404595+0000","last_fullsized":"2024-01-24T13:59:26.404595+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.378720+0000","last_change":"2024-01-24T13:59:25.963171+0000","last_active":"2024-01-24T13:59:26.378720+0000","last_peered":"2024-01-24T13:59:26.378720+0000","last_clean":"2024-01-24T13:59:26.378720+0000","last_became_active":"2024-01-24T13:59:25.962563+0000","last_became_peered":"2024-01-24T13:59:25.962563+0000","last_unstale":"2024-01-24T13:59:26.378720+0000","last_undegraded":"2024-01-24T13:59:26.378720+0000","last_fullsized":"2024-01-24T13:59:26.378720+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.378161+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T13:59:26.378161+0000","last_peered":"2024-01-24T13:59:26.378161+0000","last_clean":"2024-01-24T13:59:26.378161+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T13:59:26.378161+0000","last_undegraded":"2024-01-24T13:59:26.378161+0000","last_fullsized":"2024-01-24T13:59:26.378161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392132+0000","last_change":"2024-01-24T13:59:26.387707+0000","last_active":"2024-01-24T13:59:26.392132+0000","last_peered":"2024-01-24T13:59:26.392132+0000","last_clean":"2024-01-24T13:59:26.392132+0000","last_became_active":"2024-01-24T13:59:26.387316+0000","last_became_peered":"2024-01-24T13:59:26.387316+0000","last_unstale":"2024-01-24T13:59:26.392132+0000","last_undegraded":"2024-01-24T13:59:26.392132+0000","last_fullsized":"2024-01-24T13:59:26.392132+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.376686+0000","last_change":"2024-01-24T13:59:25.975250+0000","last_active":"2024-01-24T13:59:26.376686+0000","last_peered":"2024-01-24T13:59:26.376686+0000","last_clean":"2024-01-24T13:59:26.376686+0000","last_became_active":"2024-01-24T13:59:25.974992+0000","last_became_peered":"2024-01-24T13:59:25.974992+0000","last_unstale":"2024-01-24T13:59:26.376686+0000","last_undegraded":"2024-01-24T13:59:26.376686+0000","last_fullsized":"2024-01-24T13:59:26.376686+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392514+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T13:59:26.392514+0000","last_peered":"2024-01-24T13:59:26.392514+0000","last_clean":"2024-01-24T13:59:26.392514+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T13:59:26.392514+0000","last_undegraded":"2024-01-24T13:59:26.392514+0000","last_fullsized":"2024-01-24T13:59:26.392514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405297+0000","last_change":"2024-01-24T13:59:26.404095+0000","last_active":"2024-01-24T13:59:26.405297+0000","last_peered":"2024-01-24T13:59:26.405297+0000","last_clean":"2024-01-24T13:59:26.405297+0000","last_became_active":"2024-01-24T13:59:26.403807+0000","last_became_peered":"2024-01-24T13:59:26.403807+0000","last_unstale":"2024-01-24T13:59:26.405297+0000","last_undegraded":"2024-01-24T13:59:26.405297+0000","last_fullsized":"2024-01-24T13:59:26.405297+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.376556+0000","last_change":"2024-01-24T13:59:25.977217+0000","last_active":"2024-01-24T13:59:26.376556+0000","last_peered":"2024-01-24T13:59:26.376556+0000","last_clean":"2024-01-24T13:59:26.376556+0000","last_became_active":"2024-01-24T13:59:25.976893+0000","last_became_peered":"2024-01-24T13:59:25.976893+0000","last_unstale":"2024-01-24T13:59:26.376556+0000","last_undegraded":"2024-01-24T13:59:26.376556+0000","last_fullsized":"2024-01-24T13:59:26.376556+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.388960+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T13:59:26.388960+0000","last_peered":"2024-01-24T13:59:26.388960+0000","last_clean":"2024-01-24T13:59:26.388960+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T13:59:26.388960+0000","last_undegraded":"2024-01-24T13:59:26.388960+0000","last_fullsized":"2024-01-24T13:59:26.388960+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394231+0000","last_change":"2024-01-24T13:59:26.394231+0000","last_active":"2024-01-24T13:59:26.394231+0000","last_peered":"2024-01-24T13:59:26.394231+0000","last_clean":"2024-01-24T13:59:26.394231+0000","last_became_active":"2024-01-24T13:59:26.393918+0000","last_became_peered":"2024-01-24T13:59:26.393918+0000","last_unstale":"2024-01-24T13:59:26.394231+0000","last_undegraded":"2024-01-24T13:59:26.394231+0000","last_fullsized":"2024-01-24T13:59:26.394231+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'2","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399061+0000","last_change":"2024-01-24T13:59:25.972566+0000","last_active":"2024-01-24T13:59:26.399061+0000","last_peered":"2024-01-24T13:59:26.399061+0000","last_clean":"2024-01-24T13:59:26.399061+0000","last_became_active":"2024-01-24T13:59:25.972296+0000","last_became_peered":"2024-01-24T13:59:25.972296+0000","last_unstale":"2024-01-24T13:59:26.399061+0000","last_undegraded":"2024-01-24T13:59:26.399061+0000","last_fullsized":"2024-01-24T13:59:26.399061+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.389040+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T13:59:26.389040+0000","last_peered":"2024-01-24T13:59:26.389040+0000","last_clean":"2024-01-24T13:59:26.389040+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T13:59:26.389040+0000","last_undegraded":"2024-01-24T13:59:26.389040+0000","last_fullsized":"2024-01-24T13:59:26.389040+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.380534+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T13:59:26.380534+0000","last_peered":"2024-01-24T13:59:26.380534+0000","last_clean":"2024-01-24T13:59:26.380534+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T13:59:26.380534+0000","last_undegraded":"2024-01-24T13:59:26.380534+0000","last_fullsized":"2024-01-24T13:59:26.380534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.388213+0000","last_change":"2024-01-24T13:59:26.388213+0000","last_active":"2024-01-24T13:59:26.388213+0000","last_peered":"2024-01-24T13:59:26.388213+0000","last_clean":"2024-01-24T13:59:26.388213+0000","last_became_active":"2024-01-24T13:59:26.387917+0000","last_became_peered":"2024-01-24T13:59:26.387917+0000","last_unstale":"2024-01-24T13:59:26.388213+0000","last_undegraded":"2024-01-24T13:59:26.388213+0000","last_fullsized":"2024-01-24T13:59:26.388213+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392428+0000","last_change":"2024-01-24T13:59:25.976888+0000","last_active":"2024-01-24T13:59:26.392428+0000","last_peered":"2024-01-24T13:59:26.392428+0000","last_clean":"2024-01-24T13:59:26.392428+0000","last_became_active":"2024-01-24T13:59:25.976551+0000","last_became_peered":"2024-01-24T13:59:25.976551+0000","last_unstale":"2024-01-24T13:59:26.392428+0000","last_undegraded":"2024-01-24T13:59:26.392428+0000","last_fullsized":"2024-01-24T13:59:26.392428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.389544+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T13:59:26.389544+0000","last_peered":"2024-01-24T13:59:26.389544+0000","last_clean":"2024-01-24T13:59:26.389544+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T13:59:26.389544+0000","last_undegraded":"2024-01-24T13:59:26.389544+0000","last_fullsized":"2024-01-24T13:59:26.389544+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406297+0000","last_change":"2024-01-24T13:59:26.402958+0000","last_active":"2024-01-24T13:59:26.406297+0000","last_peered":"2024-01-24T13:59:26.406297+0000","last_clean":"2024-01-24T13:59:26.406297+0000","last_became_active":"2024-01-24T13:59:26.402719+0000","last_became_peered":"2024-01-24T13:59:26.402719+0000","last_unstale":"2024-01-24T13:59:26.406297+0000","last_undegraded":"2024-01-24T13:59:26.406297+0000","last_fullsized":"2024-01-24T13:59:26.406297+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.374884+0000","last_change":"2024-01-24T13:59:25.966875+0000","last_active":"2024-01-24T13:59:26.374884+0000","last_peered":"2024-01-24T13:59:26.374884+0000","last_clean":"2024-01-24T13:59:26.374884+0000","last_became_active":"2024-01-24T13:59:25.966580+0000","last_became_peered":"2024-01-24T13:59:25.966580+0000","last_unstale":"2024-01-24T13:59:26.374884+0000","last_undegraded":"2024-01-24T13:59:26.374884+0000","last_fullsized":"2024-01-24T13:59:26.374884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377211+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T13:59:26.377211+0000","last_peered":"2024-01-24T13:59:26.377211+0000","last_clean":"2024-01-24T13:59:26.377211+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T13:59:26.377211+0000","last_undegraded":"2024-01-24T13:59:26.377211+0000","last_fullsized":"2024-01-24T13:59:26.377211+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394754+0000","last_change":"2024-01-24T13:59:26.392089+0000","last_active":"2024-01-24T13:59:26.394754+0000","last_peered":"2024-01-24T13:59:26.394754+0000","last_clean":"2024-01-24T13:59:26.394754+0000","last_became_active":"2024-01-24T13:59:26.391832+0000","last_became_peered":"2024-01-24T13:59:26.391832+0000","last_unstale":"2024-01-24T13:59:26.394754+0000","last_undegraded":"2024-01-24T13:59:26.394754+0000","last_fullsized":"2024-01-24T13:59:26.394754+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-01-24T13:59:25.983216+0000","last_change":"2024-01-24T13:59:25.982352+0000","last_active":"2024-01-24T13:59:25.983216+0000","last_peered":"2024-01-24T13:59:25.983216+0000","last_clean":"2024-01-24T13:59:25.983216+0000","last_became_active":"2024-01-24T13:59:25.981960+0000","last_became_peered":"2024-01-24T13:59:25.981960+0000","last_unstale":"2024-01-24T13:59:25.983216+0000","last_undegraded":"2024-01-24T13:59:25.983216+0000","last_fullsized":"2024-01-24T13:59:25.983216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.376644+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T13:59:26.376644+0000","last_peered":"2024-01-24T13:59:26.376644+0000","last_clean":"2024-01-24T13:59:26.376644+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T13:59:26.376644+0000","last_undegraded":"2024-01-24T13:59:26.376644+0000","last_fullsized":"2024-01-24T13:59:26.376644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":434176,"data_stored":194589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.31900000000000001}]},{"osd":2,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.377}]},{"osd":3,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.63300000000000001}]},{"osd":5,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.747}]},{"osd":7,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.77200000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":479232,"data_stored":217261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.378},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.497},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.37},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.309}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.41999999999999998}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.34999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":495616,"data_stored":218799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.498},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.54800000000000004}]},{"osd":3,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.625}]},{"osd":4,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.873}]},{"osd":5,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":1.044},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.94099999999999995}]},{"osd":6,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.96199999999999997}]},{"osd":7,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":1.0149999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":479232,"data_stored":217241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.45700000000000002}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.35899999999999999}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.68500000000000005}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.5}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":487424,"data_stored":217781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.42099999999999999}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.38300000000000001}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.436},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.46500000000000002}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.622}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.629}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60699999999999998}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68799999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":483328,"data_stored":217815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.502},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.67500000000000004}]},{"osd":1,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.63200000000000001}]},{"osd":2,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.71099999999999997}]},{"osd":3,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.57799999999999996}]},{"osd":6,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.755}]},{"osd":7,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.64500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":475136,"data_stored":217171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.44},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.52800000000000002}]},{"osd":2,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.54400000000000004}]},{"osd":3,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56100000000000005}]},{"osd":4,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":479232,"data_stored":217261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":1.3859999999999999}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":1.353},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.81200000000000006}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":6.8129999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":1.4079999999999999}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":7.016},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.85099999999999998}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":6.8719999999999999}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":6.976},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":7.8789999999999996}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":6.9080000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T13:59:35.112 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T13:59:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.252+0000 7f64824dd700 1 Processor -- start 2024-01-24T13:59:35.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.252+0000 7f64824dd700 1 -- start start 2024-01-24T13:59:35.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.253+0000 7f64824dd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f647c155060 0x7f647c15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.253+0000 7f64824dd700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c06a3a0 con 0x7f647c069240 2024-01-24T13:59:35.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.253+0000 7f64824dd700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c069820 con 0x7f647c150cb0 2024-01-24T13:59:35.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.253+0000 7f64824dd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647c0c2c70 con 0x7f647c155060 2024-01-24T13:59:35.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647b7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f647c069240 0x7f647c154950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42374/0 (socket says 172.21.15.29:42374) 2024-01-24T13:59:35.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647b7fe700 1 -- 172.21.15.29:0/3492983413 learned_addr learned my addr 172.21.15.29:0/3492983413 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:35.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f6480a7a700 1 --2- 172.21.15.29:0/3492983413 >> v2:172.21.15.163:3300/0 conn(0x7f647c155060 0x7f647c15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792507677 0 0) 0x7f647c06a3a0 con 0x7f647c069240 2024-01-24T13:59:35.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64680033f0 con 0x7f647c069240 2024-01-24T13:59:35.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2119597626 0 0) 0x7f64680033f0 con 0x7f647c069240 2024-01-24T13:59:35.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 >> v2:172.21.15.163:3300/0 conn(0x7f647c155060 msgr2=0x7f647c15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:35.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 --2- 172.21.15.29:0/3492983413 >> v2:172.21.15.163:3300/0 conn(0x7f647c155060 0x7f647c15bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 >> v1:172.21.15.163:6789/0 conn(0x7f647c150cb0 legacy=0x7f647c065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c069b00 con 0x7f647c069240 2024-01-24T13:59:35.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f6464001470 con 0x7f647c069240 2024-01-24T13:59:35.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 >> v1:172.21.15.29:6789/0 conn(0x7f647c069240 legacy=0x7f647c154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 --2- 172.21.15.29:0/3492983413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64680038e0 0x7f6468003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 --2- 172.21.15.29:0/3492983413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6468004690 0x7f6468004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 --2- 172.21.15.29:0/3492983413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6468005000 0x7f646800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0c2c70 con 0x7f64680038e0 2024-01-24T13:59:35.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646800beb0 con 0x7f6468004690 2024-01-24T13:59:35.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646800c070 con 0x7f6468005000 2024-01-24T13:59:35.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.254+0000 7f647bfff700 1 --2- 172.21.15.29:0/3492983413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64680038e0 0x7f6468003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f6480a7a700 1 --2- 172.21.15.29:0/3492983413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6468005000 0x7f646800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f647b7fe700 1 --2- 172.21.15.29:0/3492983413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6468004690 0x7f6468004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f647bfff700 1 -- 172.21.15.29:0/3492983413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6468005000 msgr2=0x7f646800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:35.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f647bfff700 1 --2- 172.21.15.29:0/3492983413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6468005000 0x7f646800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f647bfff700 1 -- 172.21.15.29:0/3492983413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6468004690 msgr2=0x7f6468004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:35.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f647bfff700 1 --2- 172.21.15.29:0/3492983413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6468004690 0x7f6468004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f647bfff700 1 -- 172.21.15.29:0/3492983413 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64680033f0 con 0x7f64680038e0 2024-01-24T13:59:35.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f6480a7a700 1 -- 172.21.15.29:0/3492983413 reap_dead start 2024-01-24T13:59:35.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f647b7fe700 1 --2- 172.21.15.29:0/3492983413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6468004690 0x7f6468004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f647bfff700 1 --2- 172.21.15.29:0/3492983413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64680038e0 0x7f6468003d00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f646c001760 tx=0x7f646c007bb0).ready entity=mon.0 client_cookie=9c623a60f846eefe server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c007020 con 0x7f64680038e0 2024-01-24T13:59:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646c007180 con 0x7f64680038e0 2024-01-24T13:59:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.255+0000 7f647affd700 1 -- 172.21.15.29:0/3492983413 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c007520 con 0x7f64680038e0 2024-01-24T13:59:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.256+0000 7f64824dd700 1 -- 172.21.15.29:0/3492983413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64680038e0 msgr2=0x7f6468003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.256+0000 7f64824dd700 1 --2- 172.21.15.29:0/3492983413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64680038e0 0x7f6468003d00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f646c001760 tx=0x7f646c007bb0).stop 2024-01-24T13:59:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.256+0000 7f64824dd700 1 -- 172.21.15.29:0/3492983413 shutdown_connections 2024-01-24T13:59:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.256+0000 7f64824dd700 1 --2- 172.21.15.29:0/3492983413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64680038e0 0x7f6468003d00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.256+0000 7f64824dd700 1 -- 172.21.15.29:0/3492983413 >> 172.21.15.29:0/3492983413 conn(0x7f647c0c85c0 msgr2=0x7f647c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.256+0000 7f64824dd700 1 -- 172.21.15.29:0/3492983413 shutdown_connections 2024-01-24T13:59:35.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.256+0000 7f64824dd700 1 -- 172.21.15.29:0/3492983413 wait complete. 2024-01-24T13:59:35.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.257+0000 7f64824dd700 1 Processor -- start 2024-01-24T13:59:35.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.257+0000 7f64824dd700 1 -- start start 2024-01-24T13:59:35.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f64824dd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64680038e0 0x7f647c0c0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f64824dd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f647c0c12e0 0x7f647c1655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f64824dd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f647c165ad0 0x7f647c0c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f64824dd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0c1830 con 0x7f647c165ad0 2024-01-24T13:59:35.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f64824dd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c168560 con 0x7f647c0c12e0 2024-01-24T13:59:35.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f64824dd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c168840 con 0x7f64680038e0 2024-01-24T13:59:35.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f6480a7a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f647c165ad0 0x7f647c0c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f647bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64680038e0 0x7f647c0c0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f6480a7a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f647c165ad0 0x7f647c0c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36236/0 (socket says 172.21.15.29:36236) 2024-01-24T13:59:35.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f6480a7a700 1 -- 172.21.15.29:0/1482968832 learned_addr learned my addr 172.21.15.29:0/1482968832 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:35.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f647b7fe700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f647c0c12e0 0x7f647c1655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f6480a7a700 1 -- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64680038e0 msgr2=0x7f647c0c0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f6480a7a700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64680038e0 0x7f647c0c0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f6480a7a700 1 -- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f647c0c12e0 msgr2=0x7f647c1655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f6480a7a700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f647c0c12e0 0x7f647c1655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.258+0000 7f6480a7a700 1 -- 172.21.15.29:0/1482968832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c168b20 con 0x7f647c165ad0 2024-01-24T13:59:35.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.259+0000 7f6480a7a700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f647c165ad0 0x7f647c0c6ef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f64740094d0 tx=0x7f647400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:35.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.259+0000 7f6473fff700 1 -- 172.21.15.29:0/1482968832 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647400e040 con 0x7f647c165ad0 2024-01-24T13:59:35.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.259+0000 7f6473fff700 1 -- 172.21.15.29:0/1482968832 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647400f040 con 0x7f647c165ad0 2024-01-24T13:59:35.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.259+0000 7f64824dd700 1 -- 172.21.15.29:0/1482968832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647c168e00 con 0x7f647c165ad0 2024-01-24T13:59:35.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.259+0000 7f6473fff700 1 -- 172.21.15.29:0/1482968832 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647400fd70 con 0x7f647c165ad0 2024-01-24T13:59:35.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.259+0000 7f64824dd700 1 -- 172.21.15.29:0/1482968832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c0c75a0 con 0x7f647c165ad0 2024-01-24T13:59:35.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.260+0000 7f64824dd700 1 -- 172.21.15.29:0/1482968832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647c0c7820 con 0x7f647c165ad0 2024-01-24T13:59:35.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.260+0000 7f6473fff700 1 -- 172.21.15.29:0/1482968832 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6474015660 con 0x7f647c165ad0 2024-01-24T13:59:35.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.261+0000 7f6473fff700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f645c05f150 0x7f645c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.261+0000 7f6473fff700 1 -- 172.21.15.29:0/1482968832 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f647407ae20 con 0x7f647c165ad0 2024-01-24T13:59:35.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.261+0000 7f647bfff700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f645c05f150 0x7f645c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.262+0000 7f647bfff700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f645c05f150 0x7f645c061610 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f646c001050 tx=0x7f646c016000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:35.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.265+0000 7f6473fff700 1 -- 172.21.15.29:0/1482968832 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f647404bed0 con 0x7f647c165ad0 2024-01-24T13:59:35.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.441+0000 7f64824dd700 1 -- 172.21.15.29:0/1482968832 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f647c0475e0 con 0x7f645c05f150 2024-01-24T13:59:35.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.452+0000 7f6473fff700 1 -- 172.21.15.29:0/1482968832 <== mgr.4103 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361189 (secure 0 0 0) 0x7f647c0475e0 con 0x7f645c05f150 2024-01-24T13:59:35.457 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:35.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.456+0000 7f64824dd700 1 -- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f645c05f150 msgr2=0x7f645c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.456+0000 7f64824dd700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f645c05f150 0x7f645c061610 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f646c001050 tx=0x7f646c016000).stop 2024-01-24T13:59:35.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.456+0000 7f64824dd700 1 -- 172.21.15.29:0/1482968832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f647c165ad0 msgr2=0x7f647c0c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.456+0000 7f64824dd700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f647c165ad0 0x7f647c0c6ef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f64740094d0 tx=0x7f647400b040).stop 2024-01-24T13:59:35.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.456+0000 7f64824dd700 1 -- 172.21.15.29:0/1482968832 shutdown_connections 2024-01-24T13:59:35.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.456+0000 7f64824dd700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f645c05f150 0x7f645c061610 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.456+0000 7f64824dd700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f647c165ad0 0x7f647c0c6ef0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.456+0000 7f64824dd700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64680038e0 0x7f647c0c0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.456+0000 7f64824dd700 1 --2- 172.21.15.29:0/1482968832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f647c0c12e0 0x7f647c1655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.456+0000 7f64824dd700 1 -- 172.21.15.29:0/1482968832 >> 172.21.15.29:0/1482968832 conn(0x7f647c0c85c0 msgr2=0x7f647c151090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:35.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.456+0000 7f64824dd700 1 -- 172.21.15.29:0/1482968832 shutdown_connections 2024-01-24T13:59:35.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.456+0000 7f64824dd700 1 -- 172.21.15.29:0/1482968832 wait complete. 2024-01-24T13:59:35.461 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T13:59:35.469 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-01-24T13:59:33.743436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":238,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380540,"kb_used_data":3772,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752594180,"statfs":{"total":773094113280,"available":770656440320,"internally_reserved":0,"allocated":3862528,"data_stored":1741019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4683,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-22,"num_write_kb":-14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.864660"},"pg_stats":[{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392969+0000","last_change":"2024-01-24T13:59:26.387498+0000","last_active":"2024-01-24T13:59:26.392969+0000","last_peered":"2024-01-24T13:59:26.392969+0000","last_clean":"2024-01-24T13:59:26.392969+0000","last_became_active":"2024-01-24T13:59:26.387265+0000","last_became_peered":"2024-01-24T13:59:26.387265+0000","last_unstale":"2024-01-24T13:59:26.392969+0000","last_undegraded":"2024-01-24T13:59:26.392969+0000","last_fullsized":"2024-01-24T13:59:26.392969+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"40'3","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.403510+0000","last_change":"2024-01-24T13:59:25.981703+0000","last_active":"2024-01-24T13:59:26.403510+0000","last_peered":"2024-01-24T13:59:26.403510+0000","last_clean":"2024-01-24T13:59:26.403510+0000","last_became_active":"2024-01-24T13:59:25.981389+0000","last_became_peered":"2024-01-24T13:59:25.981389+0000","last_unstale":"2024-01-24T13:59:26.403510+0000","last_undegraded":"2024-01-24T13:59:26.403510+0000","last_fullsized":"2024-01-24T13:59:26.403510+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405281+0000","last_change":"2024-01-24T13:59:26.404051+0000","last_active":"2024-01-24T13:59:26.405281+0000","last_peered":"2024-01-24T13:59:26.405281+0000","last_clean":"2024-01-24T13:59:26.405281+0000","last_became_active":"2024-01-24T13:59:26.403766+0000","last_became_peered":"2024-01-24T13:59:26.403766+0000","last_unstale":"2024-01-24T13:59:26.405281+0000","last_undegraded":"2024-01-24T13:59:26.405281+0000","last_fullsized":"2024-01-24T13:59:26.405281+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.374771+0000","last_change":"2024-01-24T13:59:25.959944+0000","last_active":"2024-01-24T13:59:26.374771+0000","last_peered":"2024-01-24T13:59:26.374771+0000","last_clean":"2024-01-24T13:59:26.374771+0000","last_became_active":"2024-01-24T13:59:25.959636+0000","last_became_peered":"2024-01-24T13:59:25.959636+0000","last_unstale":"2024-01-24T13:59:26.374771+0000","last_undegraded":"2024-01-24T13:59:26.374771+0000","last_fullsized":"2024-01-24T13:59:26.374771+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401833+0000","last_change":"2024-01-24T13:59:26.394881+0000","last_active":"2024-01-24T13:59:26.401833+0000","last_peered":"2024-01-24T13:59:26.401833+0000","last_clean":"2024-01-24T13:59:26.401833+0000","last_became_active":"2024-01-24T13:59:26.394600+0000","last_became_peered":"2024-01-24T13:59:26.394600+0000","last_unstale":"2024-01-24T13:59:26.401833+0000","last_undegraded":"2024-01-24T13:59:26.401833+0000","last_fullsized":"2024-01-24T13:59:26.401833+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.381875+0000","last_change":"2024-01-24T13:59:25.980665+0000","last_active":"2024-01-24T13:59:26.381875+0000","last_peered":"2024-01-24T13:59:26.381875+0000","last_clean":"2024-01-24T13:59:26.381875+0000","last_became_active":"2024-01-24T13:59:25.980288+0000","last_became_peered":"2024-01-24T13:59:25.980288+0000","last_unstale":"2024-01-24T13:59:26.381875+0000","last_undegraded":"2024-01-24T13:59:26.381875+0000","last_fullsized":"2024-01-24T13:59:26.381875+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401782+0000","last_change":"2024-01-24T13:59:26.399272+0000","last_active":"2024-01-24T13:59:26.401782+0000","last_peered":"2024-01-24T13:59:26.401782+0000","last_clean":"2024-01-24T13:59:26.401782+0000","last_became_active":"2024-01-24T13:59:26.398992+0000","last_became_peered":"2024-01-24T13:59:26.398992+0000","last_unstale":"2024-01-24T13:59:26.401782+0000","last_undegraded":"2024-01-24T13:59:26.401782+0000","last_fullsized":"2024-01-24T13:59:26.401782+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"40'2","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404193+0000","last_change":"2024-01-24T13:59:25.970538+0000","last_active":"2024-01-24T13:59:26.404193+0000","last_peered":"2024-01-24T13:59:26.404193+0000","last_clean":"2024-01-24T13:59:26.404193+0000","last_became_active":"2024-01-24T13:59:25.969372+0000","last_became_peered":"2024-01-24T13:59:25.969372+0000","last_unstale":"2024-01-24T13:59:26.404193+0000","last_undegraded":"2024-01-24T13:59:26.404193+0000","last_fullsized":"2024-01-24T13:59:26.404193+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.402110+0000","last_change":"2024-01-24T13:59:26.396337+0000","last_active":"2024-01-24T13:59:26.402110+0000","last_peered":"2024-01-24T13:59:26.402110+0000","last_clean":"2024-01-24T13:59:26.402110+0000","last_became_active":"2024-01-24T13:59:26.396019+0000","last_became_peered":"2024-01-24T13:59:26.396019+0000","last_unstale":"2024-01-24T13:59:26.402110+0000","last_undegraded":"2024-01-24T13:59:26.402110+0000","last_fullsized":"2024-01-24T13:59:26.402110+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3b","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.400837+0000","last_change":"2024-01-24T13:59:25.969469+0000","last_active":"2024-01-24T13:59:26.400837+0000","last_peered":"2024-01-24T13:59:26.400837+0000","last_clean":"2024-01-24T13:59:26.400837+0000","last_became_active":"2024-01-24T13:59:25.969150+0000","last_became_peered":"2024-01-24T13:59:25.969150+0000","last_unstale":"2024-01-24T13:59:26.400837+0000","last_undegraded":"2024-01-24T13:59:26.400837+0000","last_fullsized":"2024-01-24T13:59:26.400837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.400961+0000","last_change":"2024-01-24T13:59:26.398010+0000","last_active":"2024-01-24T13:59:26.400961+0000","last_peered":"2024-01-24T13:59:26.400961+0000","last_clean":"2024-01-24T13:59:26.400961+0000","last_became_active":"2024-01-24T13:59:26.397688+0000","last_became_peered":"2024-01-24T13:59:26.397688+0000","last_unstale":"2024-01-24T13:59:26.400961+0000","last_undegraded":"2024-01-24T13:59:26.400961+0000","last_fullsized":"2024-01-24T13:59:26.400961+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3a","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394894+0000","last_change":"2024-01-24T13:59:25.963374+0000","last_active":"2024-01-24T13:59:26.394894+0000","last_peered":"2024-01-24T13:59:26.394894+0000","last_clean":"2024-01-24T13:59:26.394894+0000","last_became_active":"2024-01-24T13:59:25.963084+0000","last_became_peered":"2024-01-24T13:59:25.963084+0000","last_unstale":"2024-01-24T13:59:26.394894+0000","last_undegraded":"2024-01-24T13:59:26.394894+0000","last_fullsized":"2024-01-24T13:59:26.394894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.402542+0000","last_change":"2024-01-24T13:59:26.397603+0000","last_active":"2024-01-24T13:59:26.402542+0000","last_peered":"2024-01-24T13:59:26.402542+0000","last_clean":"2024-01-24T13:59:26.402542+0000","last_became_active":"2024-01-24T13:59:26.397269+0000","last_became_peered":"2024-01-24T13:59:26.397269+0000","last_unstale":"2024-01-24T13:59:26.402542+0000","last_undegraded":"2024-01-24T13:59:26.402542+0000","last_fullsized":"2024-01-24T13:59:26.402542+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377433+0000","last_change":"2024-01-24T13:59:25.976064+0000","last_active":"2024-01-24T13:59:26.377433+0000","last_peered":"2024-01-24T13:59:26.377433+0000","last_clean":"2024-01-24T13:59:26.377433+0000","last_became_active":"2024-01-24T13:59:25.975796+0000","last_became_peered":"2024-01-24T13:59:25.975796+0000","last_unstale":"2024-01-24T13:59:26.377433+0000","last_undegraded":"2024-01-24T13:59:26.377433+0000","last_fullsized":"2024-01-24T13:59:26.377433+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.400915+0000","last_change":"2024-01-24T13:59:26.400018+0000","last_active":"2024-01-24T13:59:26.400915+0000","last_peered":"2024-01-24T13:59:26.400915+0000","last_clean":"2024-01-24T13:59:26.400915+0000","last_became_active":"2024-01-24T13:59:26.399720+0000","last_became_peered":"2024-01-24T13:59:26.399720+0000","last_unstale":"2024-01-24T13:59:26.400915+0000","last_undegraded":"2024-01-24T13:59:26.400915+0000","last_fullsized":"2024-01-24T13:59:26.400915+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.374033+0000","last_change":"2024-01-24T13:59:25.976014+0000","last_active":"2024-01-24T13:59:26.374033+0000","last_peered":"2024-01-24T13:59:26.374033+0000","last_clean":"2024-01-24T13:59:26.374033+0000","last_became_active":"2024-01-24T13:59:25.975677+0000","last_became_peered":"2024-01-24T13:59:25.975677+0000","last_unstale":"2024-01-24T13:59:26.374033+0000","last_undegraded":"2024-01-24T13:59:26.374033+0000","last_fullsized":"2024-01-24T13:59:26.374033+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401104+0000","last_change":"2024-01-24T13:59:26.391986+0000","last_active":"2024-01-24T13:59:26.401104+0000","last_peered":"2024-01-24T13:59:26.401104+0000","last_clean":"2024-01-24T13:59:26.401104+0000","last_became_active":"2024-01-24T13:59:26.391698+0000","last_became_peered":"2024-01-24T13:59:26.391698+0000","last_unstale":"2024-01-24T13:59:26.401104+0000","last_undegraded":"2024-01-24T13:59:26.401104+0000","last_fullsized":"2024-01-24T13:59:26.401104+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.381970+0000","last_change":"2024-01-24T13:59:25.973798+0000","last_active":"2024-01-24T13:59:26.381970+0000","last_peered":"2024-01-24T13:59:26.381970+0000","last_clean":"2024-01-24T13:59:26.381970+0000","last_became_active":"2024-01-24T13:59:25.973506+0000","last_became_peered":"2024-01-24T13:59:25.973506+0000","last_unstale":"2024-01-24T13:59:26.381970+0000","last_undegraded":"2024-01-24T13:59:26.381970+0000","last_fullsized":"2024-01-24T13:59:26.381970+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401824+0000","last_change":"2024-01-24T13:59:26.392777+0000","last_active":"2024-01-24T13:59:26.401824+0000","last_peered":"2024-01-24T13:59:26.401824+0000","last_clean":"2024-01-24T13:59:26.401824+0000","last_became_active":"2024-01-24T13:59:26.392496+0000","last_became_peered":"2024-01-24T13:59:26.392496+0000","last_unstale":"2024-01-24T13:59:26.401824+0000","last_undegraded":"2024-01-24T13:59:26.401824+0000","last_fullsized":"2024-01-24T13:59:26.401824+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.380077+0000","last_change":"2024-01-24T13:59:25.974274+0000","last_active":"2024-01-24T13:59:26.380077+0000","last_peered":"2024-01-24T13:59:26.380077+0000","last_clean":"2024-01-24T13:59:26.380077+0000","last_became_active":"2024-01-24T13:59:25.973038+0000","last_became_peered":"2024-01-24T13:59:25.973038+0000","last_unstale":"2024-01-24T13:59:26.380077+0000","last_undegraded":"2024-01-24T13:59:26.380077+0000","last_fullsized":"2024-01-24T13:59:26.380077+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406291+0000","last_change":"2024-01-24T13:59:26.403631+0000","last_active":"2024-01-24T13:59:26.406291+0000","last_peered":"2024-01-24T13:59:26.406291+0000","last_clean":"2024-01-24T13:59:26.406291+0000","last_became_active":"2024-01-24T13:59:26.403208+0000","last_became_peered":"2024-01-24T13:59:26.403208+0000","last_unstale":"2024-01-24T13:59:26.406291+0000","last_undegraded":"2024-01-24T13:59:26.406291+0000","last_fullsized":"2024-01-24T13:59:26.406291+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377480+0000","last_change":"2024-01-24T13:59:25.975097+0000","last_active":"2024-01-24T13:59:26.377480+0000","last_peered":"2024-01-24T13:59:26.377480+0000","last_clean":"2024-01-24T13:59:26.377480+0000","last_became_active":"2024-01-24T13:59:25.973628+0000","last_became_peered":"2024-01-24T13:59:25.973628+0000","last_unstale":"2024-01-24T13:59:26.377480+0000","last_undegraded":"2024-01-24T13:59:26.377480+0000","last_fullsized":"2024-01-24T13:59:26.377480+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401802+0000","last_change":"2024-01-24T13:59:26.394489+0000","last_active":"2024-01-24T13:59:26.401802+0000","last_peered":"2024-01-24T13:59:26.401802+0000","last_clean":"2024-01-24T13:59:26.401802+0000","last_became_active":"2024-01-24T13:59:26.392243+0000","last_became_peered":"2024-01-24T13:59:26.392243+0000","last_unstale":"2024-01-24T13:59:26.401802+0000","last_undegraded":"2024-01-24T13:59:26.401802+0000","last_fullsized":"2024-01-24T13:59:26.401802+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"40'4","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.408968+0000","last_change":"2024-01-24T13:59:25.973968+0000","last_active":"2024-01-24T13:59:26.408968+0000","last_peered":"2024-01-24T13:59:26.408968+0000","last_clean":"2024-01-24T13:59:26.408968+0000","last_became_active":"2024-01-24T13:59:25.973675+0000","last_became_peered":"2024-01-24T13:59:25.973675+0000","last_unstale":"2024-01-24T13:59:26.408968+0000","last_undegraded":"2024-01-24T13:59:26.408968+0000","last_fullsized":"2024-01-24T13:59:26.408968+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401878+0000","last_change":"2024-01-24T13:59:26.395811+0000","last_active":"2024-01-24T13:59:26.401878+0000","last_peered":"2024-01-24T13:59:26.401878+0000","last_clean":"2024-01-24T13:59:26.401878+0000","last_became_active":"2024-01-24T13:59:26.395486+0000","last_became_peered":"2024-01-24T13:59:26.395486+0000","last_unstale":"2024-01-24T13:59:26.401878+0000","last_undegraded":"2024-01-24T13:59:26.401878+0000","last_fullsized":"2024-01-24T13:59:26.401878+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.398344+0000","last_change":"2024-01-24T13:59:25.974319+0000","last_active":"2024-01-24T13:59:26.398344+0000","last_peered":"2024-01-24T13:59:26.398344+0000","last_clean":"2024-01-24T13:59:26.398344+0000","last_became_active":"2024-01-24T13:59:25.973872+0000","last_became_peered":"2024-01-24T13:59:25.973872+0000","last_unstale":"2024-01-24T13:59:26.398344+0000","last_undegraded":"2024-01-24T13:59:26.398344+0000","last_fullsized":"2024-01-24T13:59:26.398344+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405408+0000","last_change":"2024-01-24T13:59:26.397845+0000","last_active":"2024-01-24T13:59:26.405408+0000","last_peered":"2024-01-24T13:59:26.405408+0000","last_clean":"2024-01-24T13:59:26.405408+0000","last_became_active":"2024-01-24T13:59:26.397440+0000","last_became_peered":"2024-01-24T13:59:26.397440+0000","last_unstale":"2024-01-24T13:59:26.405408+0000","last_undegraded":"2024-01-24T13:59:26.405408+0000","last_fullsized":"2024-01-24T13:59:26.405408+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.378557+0000","last_change":"2024-01-24T13:59:25.975011+0000","last_active":"2024-01-24T13:59:26.378557+0000","last_peered":"2024-01-24T13:59:26.378557+0000","last_clean":"2024-01-24T13:59:26.378557+0000","last_became_active":"2024-01-24T13:59:25.974761+0000","last_became_peered":"2024-01-24T13:59:25.974761+0000","last_unstale":"2024-01-24T13:59:26.378557+0000","last_undegraded":"2024-01-24T13:59:26.378557+0000","last_fullsized":"2024-01-24T13:59:26.378557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406129+0000","last_change":"2024-01-24T13:59:26.399451+0000","last_active":"2024-01-24T13:59:26.406129+0000","last_peered":"2024-01-24T13:59:26.406129+0000","last_clean":"2024-01-24T13:59:26.406129+0000","last_became_active":"2024-01-24T13:59:26.398728+0000","last_became_peered":"2024-01-24T13:59:26.398728+0000","last_unstale":"2024-01-24T13:59:26.406129+0000","last_undegraded":"2024-01-24T13:59:26.406129+0000","last_fullsized":"2024-01-24T13:59:26.406129+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377543+0000","last_change":"2024-01-24T13:59:25.975767+0000","last_active":"2024-01-24T13:59:26.377543+0000","last_peered":"2024-01-24T13:59:26.377543+0000","last_clean":"2024-01-24T13:59:26.377543+0000","last_became_active":"2024-01-24T13:59:25.975467+0000","last_became_peered":"2024-01-24T13:59:25.975467+0000","last_unstale":"2024-01-24T13:59:26.377543+0000","last_undegraded":"2024-01-24T13:59:26.377543+0000","last_fullsized":"2024-01-24T13:59:26.377543+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391296+0000","last_change":"2024-01-24T13:59:26.383140+0000","last_active":"2024-01-24T13:59:26.391296+0000","last_peered":"2024-01-24T13:59:26.391296+0000","last_clean":"2024-01-24T13:59:26.391296+0000","last_became_active":"2024-01-24T13:59:26.382851+0000","last_became_peered":"2024-01-24T13:59:26.382851+0000","last_unstale":"2024-01-24T13:59:26.391296+0000","last_undegraded":"2024-01-24T13:59:26.391296+0000","last_fullsized":"2024-01-24T13:59:26.391296+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"40'4","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.398904+0000","last_change":"2024-01-24T13:59:25.977558+0000","last_active":"2024-01-24T13:59:26.398904+0000","last_peered":"2024-01-24T13:59:26.398904+0000","last_clean":"2024-01-24T13:59:26.398904+0000","last_became_active":"2024-01-24T13:59:25.977281+0000","last_became_peered":"2024-01-24T13:59:25.977281+0000","last_unstale":"2024-01-24T13:59:26.398904+0000","last_undegraded":"2024-01-24T13:59:26.398904+0000","last_fullsized":"2024-01-24T13:59:26.398904+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393550+0000","last_change":"2024-01-24T13:59:26.388561+0000","last_active":"2024-01-24T13:59:26.393550+0000","last_peered":"2024-01-24T13:59:26.393550+0000","last_clean":"2024-01-24T13:59:26.393550+0000","last_became_active":"2024-01-24T13:59:26.388275+0000","last_became_peered":"2024-01-24T13:59:26.388275+0000","last_unstale":"2024-01-24T13:59:26.393550+0000","last_undegraded":"2024-01-24T13:59:26.393550+0000","last_fullsized":"2024-01-24T13:59:26.393550+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394705+0000","last_change":"2024-01-24T13:59:25.973398+0000","last_active":"2024-01-24T13:59:26.394705+0000","last_peered":"2024-01-24T13:59:26.394705+0000","last_clean":"2024-01-24T13:59:26.394705+0000","last_became_active":"2024-01-24T13:59:25.973098+0000","last_became_peered":"2024-01-24T13:59:25.973098+0000","last_unstale":"2024-01-24T13:59:26.394705+0000","last_undegraded":"2024-01-24T13:59:26.394705+0000","last_fullsized":"2024-01-24T13:59:26.394705+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393371+0000","last_change":"2024-01-24T13:59:26.389534+0000","last_active":"2024-01-24T13:59:26.393371+0000","last_peered":"2024-01-24T13:59:26.393371+0000","last_clean":"2024-01-24T13:59:26.393371+0000","last_became_active":"2024-01-24T13:59:26.388756+0000","last_became_peered":"2024-01-24T13:59:26.388756+0000","last_unstale":"2024-01-24T13:59:26.393371+0000","last_undegraded":"2024-01-24T13:59:26.393371+0000","last_fullsized":"2024-01-24T13:59:26.393371+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.387025+0000","last_change":"2024-01-24T13:59:25.968204+0000","last_active":"2024-01-24T13:59:26.387025+0000","last_peered":"2024-01-24T13:59:26.387025+0000","last_clean":"2024-01-24T13:59:26.387025+0000","last_became_active":"2024-01-24T13:59:25.967654+0000","last_became_peered":"2024-01-24T13:59:25.967654+0000","last_unstale":"2024-01-24T13:59:26.387025+0000","last_undegraded":"2024-01-24T13:59:26.387025+0000","last_fullsized":"2024-01-24T13:59:26.387025+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.408225+0000","last_change":"2024-01-24T13:59:26.402355+0000","last_active":"2024-01-24T13:59:26.408225+0000","last_peered":"2024-01-24T13:59:26.408225+0000","last_clean":"2024-01-24T13:59:26.408225+0000","last_became_active":"2024-01-24T13:59:26.401991+0000","last_became_peered":"2024-01-24T13:59:26.401991+0000","last_unstale":"2024-01-24T13:59:26.408225+0000","last_undegraded":"2024-01-24T13:59:26.408225+0000","last_fullsized":"2024-01-24T13:59:26.408225+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2d","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399805+0000","last_change":"2024-01-24T13:59:25.960337+0000","last_active":"2024-01-24T13:59:26.399805+0000","last_peered":"2024-01-24T13:59:26.399805+0000","last_clean":"2024-01-24T13:59:26.399805+0000","last_became_active":"2024-01-24T13:59:25.960030+0000","last_became_peered":"2024-01-24T13:59:25.960030+0000","last_unstale":"2024-01-24T13:59:26.399805+0000","last_undegraded":"2024-01-24T13:59:26.399805+0000","last_fullsized":"2024-01-24T13:59:26.399805+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406353+0000","last_change":"2024-01-24T13:59:26.403095+0000","last_active":"2024-01-24T13:59:26.406353+0000","last_peered":"2024-01-24T13:59:26.406353+0000","last_clean":"2024-01-24T13:59:26.406353+0000","last_became_active":"2024-01-24T13:59:26.402795+0000","last_became_peered":"2024-01-24T13:59:26.402795+0000","last_unstale":"2024-01-24T13:59:26.406353+0000","last_undegraded":"2024-01-24T13:59:26.406353+0000","last_fullsized":"2024-01-24T13:59:26.406353+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.386554+0000","last_change":"2024-01-24T13:59:25.968410+0000","last_active":"2024-01-24T13:59:26.386554+0000","last_peered":"2024-01-24T13:59:26.386554+0000","last_clean":"2024-01-24T13:59:26.386554+0000","last_became_active":"2024-01-24T13:59:25.967771+0000","last_became_peered":"2024-01-24T13:59:25.967771+0000","last_unstale":"2024-01-24T13:59:26.386554+0000","last_undegraded":"2024-01-24T13:59:26.386554+0000","last_fullsized":"2024-01-24T13:59:26.386554+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401763+0000","last_change":"2024-01-24T13:59:26.393452+0000","last_active":"2024-01-24T13:59:26.401763+0000","last_peered":"2024-01-24T13:59:26.401763+0000","last_clean":"2024-01-24T13:59:26.401763+0000","last_became_active":"2024-01-24T13:59:26.393184+0000","last_became_peered":"2024-01-24T13:59:26.393184+0000","last_unstale":"2024-01-24T13:59:26.401763+0000","last_undegraded":"2024-01-24T13:59:26.401763+0000","last_fullsized":"2024-01-24T13:59:26.401763+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.398429+0000","last_change":"2024-01-24T13:59:25.974128+0000","last_active":"2024-01-24T13:59:26.398429+0000","last_peered":"2024-01-24T13:59:26.398429+0000","last_clean":"2024-01-24T13:59:26.398429+0000","last_became_active":"2024-01-24T13:59:25.973551+0000","last_became_peered":"2024-01-24T13:59:25.973551+0000","last_unstale":"2024-01-24T13:59:26.398429+0000","last_undegraded":"2024-01-24T13:59:26.398429+0000","last_fullsized":"2024-01-24T13:59:26.398429+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.397788+0000","last_change":"2024-01-24T13:59:26.393112+0000","last_active":"2024-01-24T13:59:26.397788+0000","last_peered":"2024-01-24T13:59:26.397788+0000","last_clean":"2024-01-24T13:59:26.397788+0000","last_became_active":"2024-01-24T13:59:26.392817+0000","last_became_peered":"2024-01-24T13:59:26.392817+0000","last_unstale":"2024-01-24T13:59:26.397788+0000","last_undegraded":"2024-01-24T13:59:26.397788+0000","last_fullsized":"2024-01-24T13:59:26.397788+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.378512+0000","last_change":"2024-01-24T13:59:25.963330+0000","last_active":"2024-01-24T13:59:26.378512+0000","last_peered":"2024-01-24T13:59:26.378512+0000","last_clean":"2024-01-24T13:59:26.378512+0000","last_became_active":"2024-01-24T13:59:25.963008+0000","last_became_peered":"2024-01-24T13:59:25.963008+0000","last_unstale":"2024-01-24T13:59:26.378512+0000","last_undegraded":"2024-01-24T13:59:26.378512+0000","last_fullsized":"2024-01-24T13:59:26.378512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.397984+0000","last_change":"2024-01-24T13:59:26.395109+0000","last_active":"2024-01-24T13:59:26.397984+0000","last_peered":"2024-01-24T13:59:26.397984+0000","last_clean":"2024-01-24T13:59:26.397984+0000","last_became_active":"2024-01-24T13:59:26.394738+0000","last_became_peered":"2024-01-24T13:59:26.394738+0000","last_unstale":"2024-01-24T13:59:26.397984+0000","last_undegraded":"2024-01-24T13:59:26.397984+0000","last_fullsized":"2024-01-24T13:59:26.397984+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392496+0000","last_change":"2024-01-24T13:59:25.971058+0000","last_active":"2024-01-24T13:59:26.392496+0000","last_peered":"2024-01-24T13:59:26.392496+0000","last_clean":"2024-01-24T13:59:26.392496+0000","last_became_active":"2024-01-24T13:59:25.970479+0000","last_became_peered":"2024-01-24T13:59:25.970479+0000","last_unstale":"2024-01-24T13:59:26.392496+0000","last_undegraded":"2024-01-24T13:59:26.392496+0000","last_fullsized":"2024-01-24T13:59:26.392496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401679+0000","last_change":"2024-01-24T13:59:26.396728+0000","last_active":"2024-01-24T13:59:26.401679+0000","last_peered":"2024-01-24T13:59:26.401679+0000","last_clean":"2024-01-24T13:59:26.401679+0000","last_became_active":"2024-01-24T13:59:26.396409+0000","last_became_peered":"2024-01-24T13:59:26.396409+0000","last_unstale":"2024-01-24T13:59:26.401679+0000","last_undegraded":"2024-01-24T13:59:26.401679+0000","last_fullsized":"2024-01-24T13:59:26.401679+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.390490+0000","last_change":"2024-01-24T13:59:25.971237+0000","last_active":"2024-01-24T13:59:26.390490+0000","last_peered":"2024-01-24T13:59:26.390490+0000","last_clean":"2024-01-24T13:59:26.390490+0000","last_became_active":"2024-01-24T13:59:25.970754+0000","last_became_peered":"2024-01-24T13:59:25.970754+0000","last_unstale":"2024-01-24T13:59:26.390490+0000","last_undegraded":"2024-01-24T13:59:26.390490+0000","last_fullsized":"2024-01-24T13:59:26.390490+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405458+0000","last_change":"2024-01-24T13:59:26.400578+0000","last_active":"2024-01-24T13:59:26.405458+0000","last_peered":"2024-01-24T13:59:26.405458+0000","last_clean":"2024-01-24T13:59:26.405458+0000","last_became_active":"2024-01-24T13:59:26.400281+0000","last_became_peered":"2024-01-24T13:59:26.400281+0000","last_unstale":"2024-01-24T13:59:26.405458+0000","last_undegraded":"2024-01-24T13:59:26.405458+0000","last_fullsized":"2024-01-24T13:59:26.405458+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.386714+0000","last_change":"2024-01-24T13:59:25.981337+0000","last_active":"2024-01-24T13:59:26.386714+0000","last_peered":"2024-01-24T13:59:26.386714+0000","last_clean":"2024-01-24T13:59:26.386714+0000","last_became_active":"2024-01-24T13:59:25.981032+0000","last_became_peered":"2024-01-24T13:59:25.981032+0000","last_unstale":"2024-01-24T13:59:26.386714+0000","last_undegraded":"2024-01-24T13:59:26.386714+0000","last_fullsized":"2024-01-24T13:59:26.386714+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406207+0000","last_change":"2024-01-24T13:59:26.402079+0000","last_active":"2024-01-24T13:59:26.406207+0000","last_peered":"2024-01-24T13:59:26.406207+0000","last_clean":"2024-01-24T13:59:26.406207+0000","last_became_active":"2024-01-24T13:59:26.401766+0000","last_became_peered":"2024-01-24T13:59:26.401766+0000","last_unstale":"2024-01-24T13:59:26.406207+0000","last_undegraded":"2024-01-24T13:59:26.406207+0000","last_fullsized":"2024-01-24T13:59:26.406207+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"41'1","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-01-24T13:59:27.378370+0000","last_change":"2024-01-24T13:59:25.976117+0000","last_active":"2024-01-24T13:59:27.378370+0000","last_peered":"2024-01-24T13:59:27.378370+0000","last_clean":"2024-01-24T13:59:27.378370+0000","last_became_active":"2024-01-24T13:59:25.975814+0000","last_became_peered":"2024-01-24T13:59:25.975814+0000","last_unstale":"2024-01-24T13:59:27.378370+0000","last_undegraded":"2024-01-24T13:59:27.378370+0000","last_fullsized":"2024-01-24T13:59:27.378370+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406271+0000","last_change":"2024-01-24T13:59:26.403460+0000","last_active":"2024-01-24T13:59:26.406271+0000","last_peered":"2024-01-24T13:59:26.406271+0000","last_clean":"2024-01-24T13:59:26.406271+0000","last_became_active":"2024-01-24T13:59:26.402947+0000","last_became_peered":"2024-01-24T13:59:26.402947+0000","last_unstale":"2024-01-24T13:59:26.406271+0000","last_undegraded":"2024-01-24T13:59:26.406271+0000","last_fullsized":"2024-01-24T13:59:26.406271+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.386692+0000","last_change":"2024-01-24T13:59:25.972833+0000","last_active":"2024-01-24T13:59:26.386692+0000","last_peered":"2024-01-24T13:59:26.386692+0000","last_clean":"2024-01-24T13:59:26.386692+0000","last_became_active":"2024-01-24T13:59:25.972518+0000","last_became_peered":"2024-01-24T13:59:25.972518+0000","last_unstale":"2024-01-24T13:59:26.386692+0000","last_undegraded":"2024-01-24T13:59:26.386692+0000","last_fullsized":"2024-01-24T13:59:26.386692+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393401+0000","last_change":"2024-01-24T13:59:26.389080+0000","last_active":"2024-01-24T13:59:26.393401+0000","last_peered":"2024-01-24T13:59:26.393401+0000","last_clean":"2024-01-24T13:59:26.393401+0000","last_became_active":"2024-01-24T13:59:26.388810+0000","last_became_peered":"2024-01-24T13:59:26.388810+0000","last_unstale":"2024-01-24T13:59:26.393401+0000","last_undegraded":"2024-01-24T13:59:26.393401+0000","last_fullsized":"2024-01-24T13:59:26.393401+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.379785+0000","last_change":"2024-01-24T13:59:25.975851+0000","last_active":"2024-01-24T13:59:26.379785+0000","last_peered":"2024-01-24T13:59:26.379785+0000","last_clean":"2024-01-24T13:59:26.379785+0000","last_became_active":"2024-01-24T13:59:25.975545+0000","last_became_peered":"2024-01-24T13:59:25.975545+0000","last_unstale":"2024-01-24T13:59:26.379785+0000","last_undegraded":"2024-01-24T13:59:26.379785+0000","last_fullsized":"2024-01-24T13:59:26.379785+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401748+0000","last_change":"2024-01-24T13:59:26.396104+0000","last_active":"2024-01-24T13:59:26.401748+0000","last_peered":"2024-01-24T13:59:26.401748+0000","last_clean":"2024-01-24T13:59:26.401748+0000","last_became_active":"2024-01-24T13:59:26.395835+0000","last_became_peered":"2024-01-24T13:59:26.395835+0000","last_unstale":"2024-01-24T13:59:26.401748+0000","last_undegraded":"2024-01-24T13:59:26.401748+0000","last_fullsized":"2024-01-24T13:59:26.401748+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.386450+0000","last_change":"2024-01-24T13:59:25.970355+0000","last_active":"2024-01-24T13:59:26.386450+0000","last_peered":"2024-01-24T13:59:26.386450+0000","last_clean":"2024-01-24T13:59:26.386450+0000","last_became_active":"2024-01-24T13:59:25.970149+0000","last_became_peered":"2024-01-24T13:59:25.970149+0000","last_unstale":"2024-01-24T13:59:26.386450+0000","last_undegraded":"2024-01-24T13:59:26.386450+0000","last_fullsized":"2024-01-24T13:59:26.386450+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404636+0000","last_change":"2024-01-24T13:59:26.399244+0000","last_active":"2024-01-24T13:59:26.404636+0000","last_peered":"2024-01-24T13:59:26.404636+0000","last_clean":"2024-01-24T13:59:26.404636+0000","last_became_active":"2024-01-24T13:59:26.398962+0000","last_became_peered":"2024-01-24T13:59:26.398962+0000","last_unstale":"2024-01-24T13:59:26.404636+0000","last_undegraded":"2024-01-24T13:59:26.404636+0000","last_fullsized":"2024-01-24T13:59:26.404636+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.387268+0000","last_change":"2024-01-24T13:59:25.977273+0000","last_active":"2024-01-24T13:59:26.387268+0000","last_peered":"2024-01-24T13:59:26.387268+0000","last_clean":"2024-01-24T13:59:26.387268+0000","last_became_active":"2024-01-24T13:59:25.977009+0000","last_became_peered":"2024-01-24T13:59:25.977009+0000","last_unstale":"2024-01-24T13:59:26.387268+0000","last_undegraded":"2024-01-24T13:59:26.387268+0000","last_fullsized":"2024-01-24T13:59:26.387268+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405831+0000","last_change":"2024-01-24T13:59:26.400727+0000","last_active":"2024-01-24T13:59:26.405831+0000","last_peered":"2024-01-24T13:59:26.405831+0000","last_clean":"2024-01-24T13:59:26.405831+0000","last_became_active":"2024-01-24T13:59:26.400259+0000","last_became_peered":"2024-01-24T13:59:26.400259+0000","last_unstale":"2024-01-24T13:59:26.405831+0000","last_undegraded":"2024-01-24T13:59:26.405831+0000","last_fullsized":"2024-01-24T13:59:26.405831+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392726+0000","last_change":"2024-01-24T13:59:25.971227+0000","last_active":"2024-01-24T13:59:26.392726+0000","last_peered":"2024-01-24T13:59:26.392726+0000","last_clean":"2024-01-24T13:59:26.392726+0000","last_became_active":"2024-01-24T13:59:25.970788+0000","last_became_peered":"2024-01-24T13:59:25.970788+0000","last_unstale":"2024-01-24T13:59:26.392726+0000","last_undegraded":"2024-01-24T13:59:26.392726+0000","last_fullsized":"2024-01-24T13:59:26.392726+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.395976+0000","last_change":"2024-01-24T13:59:26.386354+0000","last_active":"2024-01-24T13:59:26.395976+0000","last_peered":"2024-01-24T13:59:26.395976+0000","last_clean":"2024-01-24T13:59:26.395976+0000","last_became_active":"2024-01-24T13:59:26.386059+0000","last_became_peered":"2024-01-24T13:59:26.386059+0000","last_unstale":"2024-01-24T13:59:26.395976+0000","last_undegraded":"2024-01-24T13:59:26.395976+0000","last_fullsized":"2024-01-24T13:59:26.395976+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377356+0000","last_change":"2024-01-24T13:59:25.962615+0000","last_active":"2024-01-24T13:59:26.377356+0000","last_peered":"2024-01-24T13:59:26.377356+0000","last_clean":"2024-01-24T13:59:26.377356+0000","last_became_active":"2024-01-24T13:59:25.962088+0000","last_became_peered":"2024-01-24T13:59:25.962088+0000","last_unstale":"2024-01-24T13:59:26.377356+0000","last_undegraded":"2024-01-24T13:59:26.377356+0000","last_fullsized":"2024-01-24T13:59:26.377356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391112+0000","last_change":"2024-01-24T13:59:26.381154+0000","last_active":"2024-01-24T13:59:26.391112+0000","last_peered":"2024-01-24T13:59:26.391112+0000","last_clean":"2024-01-24T13:59:26.391112+0000","last_became_active":"2024-01-24T13:59:26.380865+0000","last_became_peered":"2024-01-24T13:59:26.380865+0000","last_unstale":"2024-01-24T13:59:26.391112+0000","last_undegraded":"2024-01-24T13:59:26.391112+0000","last_fullsized":"2024-01-24T13:59:26.391112+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399808+0000","last_change":"2024-01-24T13:59:25.973394+0000","last_active":"2024-01-24T13:59:26.399808+0000","last_peered":"2024-01-24T13:59:26.399808+0000","last_clean":"2024-01-24T13:59:26.399808+0000","last_became_active":"2024-01-24T13:59:25.973125+0000","last_became_peered":"2024-01-24T13:59:25.973125+0000","last_unstale":"2024-01-24T13:59:26.399808+0000","last_undegraded":"2024-01-24T13:59:26.399808+0000","last_fullsized":"2024-01-24T13:59:26.399808+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406187+0000","last_change":"2024-01-24T13:59:26.401718+0000","last_active":"2024-01-24T13:59:26.406187+0000","last_peered":"2024-01-24T13:59:26.406187+0000","last_clean":"2024-01-24T13:59:26.406187+0000","last_became_active":"2024-01-24T13:59:26.398491+0000","last_became_peered":"2024-01-24T13:59:26.398491+0000","last_unstale":"2024-01-24T13:59:26.406187+0000","last_undegraded":"2024-01-24T13:59:26.406187+0000","last_fullsized":"2024-01-24T13:59:26.406187+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393110+0000","last_change":"2024-01-24T13:59:25.969038+0000","last_active":"2024-01-24T13:59:26.393110+0000","last_peered":"2024-01-24T13:59:26.393110+0000","last_clean":"2024-01-24T13:59:26.393110+0000","last_became_active":"2024-01-24T13:59:25.968181+0000","last_became_peered":"2024-01-24T13:59:25.968181+0000","last_unstale":"2024-01-24T13:59:26.393110+0000","last_undegraded":"2024-01-24T13:59:26.393110+0000","last_fullsized":"2024-01-24T13:59:26.393110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399805+0000","last_change":"2024-01-24T13:59:26.393972+0000","last_active":"2024-01-24T13:59:26.399805+0000","last_peered":"2024-01-24T13:59:26.399805+0000","last_clean":"2024-01-24T13:59:26.399805+0000","last_became_active":"2024-01-24T13:59:26.393647+0000","last_became_peered":"2024-01-24T13:59:26.393647+0000","last_unstale":"2024-01-24T13:59:26.399805+0000","last_undegraded":"2024-01-24T13:59:26.399805+0000","last_fullsized":"2024-01-24T13:59:26.399805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"40'2","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391442+0000","last_change":"2024-01-24T13:59:25.976311+0000","last_active":"2024-01-24T13:59:26.391442+0000","last_peered":"2024-01-24T13:59:26.391442+0000","last_clean":"2024-01-24T13:59:26.391442+0000","last_became_active":"2024-01-24T13:59:25.975906+0000","last_became_peered":"2024-01-24T13:59:25.975906+0000","last_unstale":"2024-01-24T13:59:26.391442+0000","last_undegraded":"2024-01-24T13:59:26.391442+0000","last_fullsized":"2024-01-24T13:59:26.391442+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399147+0000","last_change":"2024-01-24T13:59:26.394479+0000","last_active":"2024-01-24T13:59:26.399147+0000","last_peered":"2024-01-24T13:59:26.399147+0000","last_clean":"2024-01-24T13:59:26.399147+0000","last_became_active":"2024-01-24T13:59:26.393691+0000","last_became_peered":"2024-01-24T13:59:26.393691+0000","last_unstale":"2024-01-24T13:59:26.399147+0000","last_undegraded":"2024-01-24T13:59:26.399147+0000","last_fullsized":"2024-01-24T13:59:26.399147+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391670+0000","last_change":"2024-01-24T13:59:25.977532+0000","last_active":"2024-01-24T13:59:26.391670+0000","last_peered":"2024-01-24T13:59:26.391670+0000","last_clean":"2024-01-24T13:59:26.391670+0000","last_became_active":"2024-01-24T13:59:25.977226+0000","last_became_peered":"2024-01-24T13:59:25.977226+0000","last_unstale":"2024-01-24T13:59:26.391670+0000","last_undegraded":"2024-01-24T13:59:26.391670+0000","last_fullsized":"2024-01-24T13:59:26.391670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394539+0000","last_change":"2024-01-24T13:59:26.391562+0000","last_active":"2024-01-24T13:59:26.394539+0000","last_peered":"2024-01-24T13:59:26.394539+0000","last_clean":"2024-01-24T13:59:26.394539+0000","last_became_active":"2024-01-24T13:59:26.391262+0000","last_became_peered":"2024-01-24T13:59:26.391262+0000","last_unstale":"2024-01-24T13:59:26.394539+0000","last_undegraded":"2024-01-24T13:59:26.394539+0000","last_fullsized":"2024-01-24T13:59:26.394539+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.374913+0000","last_change":"2024-01-24T13:59:25.972598+0000","last_active":"2024-01-24T13:59:26.374913+0000","last_peered":"2024-01-24T13:59:26.374913+0000","last_clean":"2024-01-24T13:59:26.374913+0000","last_became_active":"2024-01-24T13:59:25.971998+0000","last_became_peered":"2024-01-24T13:59:25.971998+0000","last_unstale":"2024-01-24T13:59:26.374913+0000","last_undegraded":"2024-01-24T13:59:26.374913+0000","last_fullsized":"2024-01-24T13:59:26.374913+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.403656+0000","last_change":"2024-01-24T13:59:26.399449+0000","last_active":"2024-01-24T13:59:26.403656+0000","last_peered":"2024-01-24T13:59:26.403656+0000","last_clean":"2024-01-24T13:59:26.403656+0000","last_became_active":"2024-01-24T13:59:26.399156+0000","last_became_peered":"2024-01-24T13:59:26.399156+0000","last_unstale":"2024-01-24T13:59:26.403656+0000","last_undegraded":"2024-01-24T13:59:26.403656+0000","last_fullsized":"2024-01-24T13:59:26.403656+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.378664+0000","last_change":"2024-01-24T13:59:25.976748+0000","last_active":"2024-01-24T13:59:26.378664+0000","last_peered":"2024-01-24T13:59:26.378664+0000","last_clean":"2024-01-24T13:59:26.378664+0000","last_became_active":"2024-01-24T13:59:25.976464+0000","last_became_peered":"2024-01-24T13:59:25.976464+0000","last_unstale":"2024-01-24T13:59:26.378664+0000","last_undegraded":"2024-01-24T13:59:26.378664+0000","last_fullsized":"2024-01-24T13:59:26.378664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.402522+0000","last_change":"2024-01-24T13:59:26.397561+0000","last_active":"2024-01-24T13:59:26.402522+0000","last_peered":"2024-01-24T13:59:26.402522+0000","last_clean":"2024-01-24T13:59:26.402522+0000","last_became_active":"2024-01-24T13:59:26.397284+0000","last_became_peered":"2024-01-24T13:59:26.397284+0000","last_unstale":"2024-01-24T13:59:26.402522+0000","last_undegraded":"2024-01-24T13:59:26.402522+0000","last_fullsized":"2024-01-24T13:59:26.402522+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377654+0000","last_change":"2024-01-24T13:59:25.968531+0000","last_active":"2024-01-24T13:59:26.377654+0000","last_peered":"2024-01-24T13:59:26.377654+0000","last_clean":"2024-01-24T13:59:26.377654+0000","last_became_active":"2024-01-24T13:59:25.968224+0000","last_became_peered":"2024-01-24T13:59:25.968224+0000","last_unstale":"2024-01-24T13:59:26.377654+0000","last_undegraded":"2024-01-24T13:59:26.377654+0000","last_fullsized":"2024-01-24T13:59:26.377654+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401658+0000","last_change":"2024-01-24T13:59:26.396396+0000","last_active":"2024-01-24T13:59:26.401658+0000","last_peered":"2024-01-24T13:59:26.401658+0000","last_clean":"2024-01-24T13:59:26.401658+0000","last_became_active":"2024-01-24T13:59:26.396120+0000","last_became_peered":"2024-01-24T13:59:26.396120+0000","last_unstale":"2024-01-24T13:59:26.401658+0000","last_undegraded":"2024-01-24T13:59:26.401658+0000","last_fullsized":"2024-01-24T13:59:26.401658+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.383941+0000","last_change":"2024-01-24T13:59:25.981443+0000","last_active":"2024-01-24T13:59:26.383941+0000","last_peered":"2024-01-24T13:59:26.383941+0000","last_clean":"2024-01-24T13:59:26.383941+0000","last_became_active":"2024-01-24T13:59:25.980669+0000","last_became_peered":"2024-01-24T13:59:25.980669+0000","last_unstale":"2024-01-24T13:59:26.383941+0000","last_undegraded":"2024-01-24T13:59:26.383941+0000","last_fullsized":"2024-01-24T13:59:26.383941+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393530+0000","last_change":"2024-01-24T13:59:26.388542+0000","last_active":"2024-01-24T13:59:26.393530+0000","last_peered":"2024-01-24T13:59:26.393530+0000","last_clean":"2024-01-24T13:59:26.393530+0000","last_became_active":"2024-01-24T13:59:26.388191+0000","last_became_peered":"2024-01-24T13:59:26.388191+0000","last_unstale":"2024-01-24T13:59:26.393530+0000","last_undegraded":"2024-01-24T13:59:26.393530+0000","last_fullsized":"2024-01-24T13:59:26.393530+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.378863+0000","last_change":"2024-01-24T13:59:25.966156+0000","last_active":"2024-01-24T13:59:26.378863+0000","last_peered":"2024-01-24T13:59:26.378863+0000","last_clean":"2024-01-24T13:59:26.378863+0000","last_became_active":"2024-01-24T13:59:25.965913+0000","last_became_peered":"2024-01-24T13:59:25.965913+0000","last_unstale":"2024-01-24T13:59:26.378863+0000","last_undegraded":"2024-01-24T13:59:26.378863+0000","last_fullsized":"2024-01-24T13:59:26.378863+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404451+0000","last_change":"2024-01-24T13:59:26.400543+0000","last_active":"2024-01-24T13:59:26.404451+0000","last_peered":"2024-01-24T13:59:26.404451+0000","last_clean":"2024-01-24T13:59:26.404451+0000","last_became_active":"2024-01-24T13:59:26.400275+0000","last_became_peered":"2024-01-24T13:59:26.400275+0000","last_unstale":"2024-01-24T13:59:26.404451+0000","last_undegraded":"2024-01-24T13:59:26.404451+0000","last_fullsized":"2024-01-24T13:59:26.404451+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.384236+0000","last_change":"2024-01-24T13:59:25.980155+0000","last_active":"2024-01-24T13:59:26.384236+0000","last_peered":"2024-01-24T13:59:26.384236+0000","last_clean":"2024-01-24T13:59:26.384236+0000","last_became_active":"2024-01-24T13:59:25.979649+0000","last_became_peered":"2024-01-24T13:59:25.979649+0000","last_unstale":"2024-01-24T13:59:26.384236+0000","last_undegraded":"2024-01-24T13:59:26.384236+0000","last_fullsized":"2024-01-24T13:59:26.384236+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404515+0000","last_change":"2024-01-24T13:59:26.403157+0000","last_active":"2024-01-24T13:59:26.404515+0000","last_peered":"2024-01-24T13:59:26.404515+0000","last_clean":"2024-01-24T13:59:26.404515+0000","last_became_active":"2024-01-24T13:59:26.402754+0000","last_became_peered":"2024-01-24T13:59:26.402754+0000","last_unstale":"2024-01-24T13:59:26.404515+0000","last_undegraded":"2024-01-24T13:59:26.404515+0000","last_fullsized":"2024-01-24T13:59:26.404515+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"40'3","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406346+0000","last_change":"2024-01-24T13:59:25.976636+0000","last_active":"2024-01-24T13:59:26.406346+0000","last_peered":"2024-01-24T13:59:26.406346+0000","last_clean":"2024-01-24T13:59:26.406346+0000","last_became_active":"2024-01-24T13:59:25.976307+0000","last_became_peered":"2024-01-24T13:59:25.976307+0000","last_unstale":"2024-01-24T13:59:26.406346+0000","last_undegraded":"2024-01-24T13:59:26.406346+0000","last_fullsized":"2024-01-24T13:59:26.406346+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401022+0000","last_change":"2024-01-24T13:59:26.399966+0000","last_active":"2024-01-24T13:59:26.401022+0000","last_peered":"2024-01-24T13:59:26.401022+0000","last_clean":"2024-01-24T13:59:26.401022+0000","last_became_active":"2024-01-24T13:59:26.399686+0000","last_became_peered":"2024-01-24T13:59:26.399686+0000","last_unstale":"2024-01-24T13:59:26.401022+0000","last_undegraded":"2024-01-24T13:59:26.401022+0000","last_fullsized":"2024-01-24T13:59:26.401022+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"40'2","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404086+0000","last_change":"2024-01-24T13:59:25.966740+0000","last_active":"2024-01-24T13:59:26.404086+0000","last_peered":"2024-01-24T13:59:26.404086+0000","last_clean":"2024-01-24T13:59:26.404086+0000","last_became_active":"2024-01-24T13:59:25.966467+0000","last_became_peered":"2024-01-24T13:59:25.966467+0000","last_unstale":"2024-01-24T13:59:26.404086+0000","last_undegraded":"2024-01-24T13:59:26.404086+0000","last_fullsized":"2024-01-24T13:59:26.404086+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406145+0000","last_change":"2024-01-24T13:59:26.399470+0000","last_active":"2024-01-24T13:59:26.406145+0000","last_peered":"2024-01-24T13:59:26.406145+0000","last_clean":"2024-01-24T13:59:26.406145+0000","last_became_active":"2024-01-24T13:59:26.399162+0000","last_became_peered":"2024-01-24T13:59:26.399162+0000","last_unstale":"2024-01-24T13:59:26.406145+0000","last_undegraded":"2024-01-24T13:59:26.406145+0000","last_fullsized":"2024-01-24T13:59:26.406145+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.402986+0000","last_change":"2024-01-24T13:59:25.970390+0000","last_active":"2024-01-24T13:59:26.402986+0000","last_peered":"2024-01-24T13:59:26.402986+0000","last_clean":"2024-01-24T13:59:26.402986+0000","last_became_active":"2024-01-24T13:59:25.970070+0000","last_became_peered":"2024-01-24T13:59:25.970070+0000","last_unstale":"2024-01-24T13:59:26.402986+0000","last_undegraded":"2024-01-24T13:59:26.402986+0000","last_fullsized":"2024-01-24T13:59:26.402986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401899+0000","last_change":"2024-01-24T13:59:26.395796+0000","last_active":"2024-01-24T13:59:26.401899+0000","last_peered":"2024-01-24T13:59:26.401899+0000","last_clean":"2024-01-24T13:59:26.401899+0000","last_became_active":"2024-01-24T13:59:26.395521+0000","last_became_peered":"2024-01-24T13:59:26.395521+0000","last_unstale":"2024-01-24T13:59:26.401899+0000","last_undegraded":"2024-01-24T13:59:26.401899+0000","last_fullsized":"2024-01-24T13:59:26.401899+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.389216+0000","last_change":"2024-01-24T13:59:25.969417+0000","last_active":"2024-01-24T13:59:26.389216+0000","last_peered":"2024-01-24T13:59:26.389216+0000","last_clean":"2024-01-24T13:59:26.389216+0000","last_became_active":"2024-01-24T13:59:25.969077+0000","last_became_peered":"2024-01-24T13:59:25.969077+0000","last_unstale":"2024-01-24T13:59:26.389216+0000","last_undegraded":"2024-01-24T13:59:26.389216+0000","last_fullsized":"2024-01-24T13:59:26.389216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405168+0000","last_change":"2024-01-24T13:59:26.399161+0000","last_active":"2024-01-24T13:59:26.405168+0000","last_peered":"2024-01-24T13:59:26.405168+0000","last_clean":"2024-01-24T13:59:26.405168+0000","last_became_active":"2024-01-24T13:59:26.398825+0000","last_became_peered":"2024-01-24T13:59:26.398825+0000","last_unstale":"2024-01-24T13:59:26.405168+0000","last_undegraded":"2024-01-24T13:59:26.405168+0000","last_fullsized":"2024-01-24T13:59:26.405168+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.375222+0000","last_change":"2024-01-24T13:59:25.970509+0000","last_active":"2024-01-24T13:59:26.375222+0000","last_peered":"2024-01-24T13:59:26.375222+0000","last_clean":"2024-01-24T13:59:26.375222+0000","last_became_active":"2024-01-24T13:59:25.970193+0000","last_became_peered":"2024-01-24T13:59:25.970193+0000","last_unstale":"2024-01-24T13:59:26.375222+0000","last_undegraded":"2024-01-24T13:59:26.375222+0000","last_fullsized":"2024-01-24T13:59:26.375222+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405247+0000","last_change":"2024-01-24T13:59:26.402327+0000","last_active":"2024-01-24T13:59:26.405247+0000","last_peered":"2024-01-24T13:59:26.405247+0000","last_clean":"2024-01-24T13:59:26.405247+0000","last_became_active":"2024-01-24T13:59:26.402037+0000","last_became_peered":"2024-01-24T13:59:26.402037+0000","last_unstale":"2024-01-24T13:59:26.405247+0000","last_undegraded":"2024-01-24T13:59:26.405247+0000","last_fullsized":"2024-01-24T13:59:26.405247+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"40'4","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401319+0000","last_change":"2024-01-24T13:59:25.981003+0000","last_active":"2024-01-24T13:59:26.401319+0000","last_peered":"2024-01-24T13:59:26.401319+0000","last_clean":"2024-01-24T13:59:26.401319+0000","last_became_active":"2024-01-24T13:59:25.980309+0000","last_became_peered":"2024-01-24T13:59:25.980309+0000","last_unstale":"2024-01-24T13:59:26.401319+0000","last_undegraded":"2024-01-24T13:59:26.401319+0000","last_fullsized":"2024-01-24T13:59:26.401319+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405439+0000","last_change":"2024-01-24T13:59:26.400511+0000","last_active":"2024-01-24T13:59:26.405439+0000","last_peered":"2024-01-24T13:59:26.405439+0000","last_clean":"2024-01-24T13:59:26.405439+0000","last_became_active":"2024-01-24T13:59:26.399924+0000","last_became_peered":"2024-01-24T13:59:26.399924+0000","last_unstale":"2024-01-24T13:59:26.405439+0000","last_undegraded":"2024-01-24T13:59:26.405439+0000","last_fullsized":"2024-01-24T13:59:26.405439+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401946+0000","last_change":"2024-01-24T13:59:25.961023+0000","last_active":"2024-01-24T13:59:26.401946+0000","last_peered":"2024-01-24T13:59:26.401946+0000","last_clean":"2024-01-24T13:59:26.401946+0000","last_became_active":"2024-01-24T13:59:25.960727+0000","last_became_peered":"2024-01-24T13:59:25.960727+0000","last_unstale":"2024-01-24T13:59:26.401946+0000","last_undegraded":"2024-01-24T13:59:26.401946+0000","last_fullsized":"2024-01-24T13:59:26.401946+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401351+0000","last_change":"2024-01-24T13:59:26.392588+0000","last_active":"2024-01-24T13:59:26.401351+0000","last_peered":"2024-01-24T13:59:26.401351+0000","last_clean":"2024-01-24T13:59:26.401351+0000","last_became_active":"2024-01-24T13:59:26.392285+0000","last_became_peered":"2024-01-24T13:59:26.392285+0000","last_unstale":"2024-01-24T13:59:26.401351+0000","last_undegraded":"2024-01-24T13:59:26.401351+0000","last_fullsized":"2024-01-24T13:59:26.401351+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"40'2","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.398125+0000","last_change":"2024-01-24T13:59:25.960160+0000","last_active":"2024-01-24T13:59:26.398125+0000","last_peered":"2024-01-24T13:59:26.398125+0000","last_clean":"2024-01-24T13:59:26.398125+0000","last_became_active":"2024-01-24T13:59:25.959877+0000","last_became_peered":"2024-01-24T13:59:25.959877+0000","last_unstale":"2024-01-24T13:59:26.398125+0000","last_undegraded":"2024-01-24T13:59:26.398125+0000","last_fullsized":"2024-01-24T13:59:26.398125+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401697+0000","last_change":"2024-01-24T13:59:26.397070+0000","last_active":"2024-01-24T13:59:26.401697+0000","last_peered":"2024-01-24T13:59:26.401697+0000","last_clean":"2024-01-24T13:59:26.401697+0000","last_became_active":"2024-01-24T13:59:26.396674+0000","last_became_peered":"2024-01-24T13:59:26.396674+0000","last_unstale":"2024-01-24T13:59:26.401697+0000","last_undegraded":"2024-01-24T13:59:26.401697+0000","last_fullsized":"2024-01-24T13:59:26.401697+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377584+0000","last_change":"2024-01-24T13:59:25.973752+0000","last_active":"2024-01-24T13:59:26.377584+0000","last_peered":"2024-01-24T13:59:26.377584+0000","last_clean":"2024-01-24T13:59:26.377584+0000","last_became_active":"2024-01-24T13:59:25.973363+0000","last_became_peered":"2024-01-24T13:59:25.973363+0000","last_unstale":"2024-01-24T13:59:26.377584+0000","last_undegraded":"2024-01-24T13:59:26.377584+0000","last_fullsized":"2024-01-24T13:59:26.377584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406199+0000","last_change":"2024-01-24T13:59:26.401916+0000","last_active":"2024-01-24T13:59:26.406199+0000","last_peered":"2024-01-24T13:59:26.406199+0000","last_clean":"2024-01-24T13:59:26.406199+0000","last_became_active":"2024-01-24T13:59:26.399223+0000","last_became_peered":"2024-01-24T13:59:26.399223+0000","last_unstale":"2024-01-24T13:59:26.406199+0000","last_undegraded":"2024-01-24T13:59:26.406199+0000","last_fullsized":"2024-01-24T13:59:26.406199+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.374644+0000","last_change":"2024-01-24T13:59:25.973319+0000","last_active":"2024-01-24T13:59:26.374644+0000","last_peered":"2024-01-24T13:59:26.374644+0000","last_clean":"2024-01-24T13:59:26.374644+0000","last_became_active":"2024-01-24T13:59:25.973003+0000","last_became_peered":"2024-01-24T13:59:25.973003+0000","last_unstale":"2024-01-24T13:59:26.374644+0000","last_undegraded":"2024-01-24T13:59:26.374644+0000","last_fullsized":"2024-01-24T13:59:26.374644+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404309+0000","last_change":"2024-01-24T13:59:26.399449+0000","last_active":"2024-01-24T13:59:26.404309+0000","last_peered":"2024-01-24T13:59:26.404309+0000","last_clean":"2024-01-24T13:59:26.404309+0000","last_became_active":"2024-01-24T13:59:26.399170+0000","last_became_peered":"2024-01-24T13:59:26.399170+0000","last_unstale":"2024-01-24T13:59:26.404309+0000","last_undegraded":"2024-01-24T13:59:26.404309+0000","last_fullsized":"2024-01-24T13:59:26.404309+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.393277+0000","last_change":"2024-01-24T13:59:25.972411+0000","last_active":"2024-01-24T13:59:26.393277+0000","last_peered":"2024-01-24T13:59:26.393277+0000","last_clean":"2024-01-24T13:59:26.393277+0000","last_became_active":"2024-01-24T13:59:25.971978+0000","last_became_peered":"2024-01-24T13:59:25.971978+0000","last_unstale":"2024-01-24T13:59:26.393277+0000","last_undegraded":"2024-01-24T13:59:26.393277+0000","last_fullsized":"2024-01-24T13:59:26.393277+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.398188+0000","last_change":"2024-01-24T13:59:26.391558+0000","last_active":"2024-01-24T13:59:26.398188+0000","last_peered":"2024-01-24T13:59:26.398188+0000","last_clean":"2024-01-24T13:59:26.398188+0000","last_became_active":"2024-01-24T13:59:26.391273+0000","last_became_peered":"2024-01-24T13:59:26.391273+0000","last_unstale":"2024-01-24T13:59:26.398188+0000","last_undegraded":"2024-01-24T13:59:26.398188+0000","last_fullsized":"2024-01-24T13:59:26.398188+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377125+0000","last_change":"2024-01-24T13:59:25.973110+0000","last_active":"2024-01-24T13:59:26.377125+0000","last_peered":"2024-01-24T13:59:26.377125+0000","last_clean":"2024-01-24T13:59:26.377125+0000","last_became_active":"2024-01-24T13:59:25.972708+0000","last_became_peered":"2024-01-24T13:59:25.972708+0000","last_unstale":"2024-01-24T13:59:26.377125+0000","last_undegraded":"2024-01-24T13:59:26.377125+0000","last_fullsized":"2024-01-24T13:59:26.377125+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.403487+0000","last_change":"2024-01-24T13:59:26.394039+0000","last_active":"2024-01-24T13:59:26.403487+0000","last_peered":"2024-01-24T13:59:26.403487+0000","last_clean":"2024-01-24T13:59:26.403487+0000","last_became_active":"2024-01-24T13:59:26.393742+0000","last_became_peered":"2024-01-24T13:59:26.393742+0000","last_unstale":"2024-01-24T13:59:26.403487+0000","last_undegraded":"2024-01-24T13:59:26.403487+0000","last_fullsized":"2024-01-24T13:59:26.403487+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.381249+0000","last_change":"2024-01-24T13:59:25.980225+0000","last_active":"2024-01-24T13:59:26.381249+0000","last_peered":"2024-01-24T13:59:26.381249+0000","last_clean":"2024-01-24T13:59:26.381249+0000","last_became_active":"2024-01-24T13:59:25.979876+0000","last_became_peered":"2024-01-24T13:59:25.979876+0000","last_unstale":"2024-01-24T13:59:26.381249+0000","last_undegraded":"2024-01-24T13:59:26.381249+0000","last_fullsized":"2024-01-24T13:59:26.381249+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401699+0000","last_change":"2024-01-24T13:59:26.392651+0000","last_active":"2024-01-24T13:59:26.401699+0000","last_peered":"2024-01-24T13:59:26.401699+0000","last_clean":"2024-01-24T13:59:26.401699+0000","last_became_active":"2024-01-24T13:59:26.392338+0000","last_became_peered":"2024-01-24T13:59:26.392338+0000","last_unstale":"2024-01-24T13:59:26.401699+0000","last_undegraded":"2024-01-24T13:59:26.401699+0000","last_fullsized":"2024-01-24T13:59:26.401699+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.389155+0000","last_change":"2024-01-24T13:59:25.970207+0000","last_active":"2024-01-24T13:59:26.389155+0000","last_peered":"2024-01-24T13:59:26.389155+0000","last_clean":"2024-01-24T13:59:26.389155+0000","last_became_active":"2024-01-24T13:59:25.969920+0000","last_became_peered":"2024-01-24T13:59:25.969920+0000","last_unstale":"2024-01-24T13:59:26.389155+0000","last_undegraded":"2024-01-24T13:59:26.389155+0000","last_fullsized":"2024-01-24T13:59:26.389155+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391267+0000","last_change":"2024-01-24T13:59:26.382366+0000","last_active":"2024-01-24T13:59:26.391267+0000","last_peered":"2024-01-24T13:59:26.391267+0000","last_clean":"2024-01-24T13:59:26.391267+0000","last_became_active":"2024-01-24T13:59:26.380486+0000","last_became_peered":"2024-01-24T13:59:26.380486+0000","last_unstale":"2024-01-24T13:59:26.391267+0000","last_undegraded":"2024-01-24T13:59:26.391267+0000","last_fullsized":"2024-01-24T13:59:26.391267+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.391904+0000","last_change":"2024-01-24T13:59:25.973150+0000","last_active":"2024-01-24T13:59:26.391904+0000","last_peered":"2024-01-24T13:59:26.391904+0000","last_clean":"2024-01-24T13:59:26.391904+0000","last_became_active":"2024-01-24T13:59:25.972845+0000","last_became_peered":"2024-01-24T13:59:25.972845+0000","last_unstale":"2024-01-24T13:59:26.391904+0000","last_undegraded":"2024-01-24T13:59:26.391904+0000","last_fullsized":"2024-01-24T13:59:26.391904+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.386798+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T13:59:26.386798+0000","last_peered":"2024-01-24T13:59:26.386798+0000","last_clean":"2024-01-24T13:59:26.386798+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T13:59:26.386798+0000","last_undegraded":"2024-01-24T13:59:26.386798+0000","last_fullsized":"2024-01-24T13:59:26.386798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.404595+0000","last_change":"2024-01-24T13:59:26.399114+0000","last_active":"2024-01-24T13:59:26.404595+0000","last_peered":"2024-01-24T13:59:26.404595+0000","last_clean":"2024-01-24T13:59:26.404595+0000","last_became_active":"2024-01-24T13:59:26.398864+0000","last_became_peered":"2024-01-24T13:59:26.398864+0000","last_unstale":"2024-01-24T13:59:26.404595+0000","last_undegraded":"2024-01-24T13:59:26.404595+0000","last_fullsized":"2024-01-24T13:59:26.404595+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.378720+0000","last_change":"2024-01-24T13:59:25.963171+0000","last_active":"2024-01-24T13:59:26.378720+0000","last_peered":"2024-01-24T13:59:26.378720+0000","last_clean":"2024-01-24T13:59:26.378720+0000","last_became_active":"2024-01-24T13:59:25.962563+0000","last_became_peered":"2024-01-24T13:59:25.962563+0000","last_unstale":"2024-01-24T13:59:26.378720+0000","last_undegraded":"2024-01-24T13:59:26.378720+0000","last_fullsized":"2024-01-24T13:59:26.378720+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.378161+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T13:59:26.378161+0000","last_peered":"2024-01-24T13:59:26.378161+0000","last_clean":"2024-01-24T13:59:26.378161+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T13:59:26.378161+0000","last_undegraded":"2024-01-24T13:59:26.378161+0000","last_fullsized":"2024-01-24T13:59:26.378161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392132+0000","last_change":"2024-01-24T13:59:26.387707+0000","last_active":"2024-01-24T13:59:26.392132+0000","last_peered":"2024-01-24T13:59:26.392132+0000","last_clean":"2024-01-24T13:59:26.392132+0000","last_became_active":"2024-01-24T13:59:26.387316+0000","last_became_peered":"2024-01-24T13:59:26.387316+0000","last_unstale":"2024-01-24T13:59:26.392132+0000","last_undegraded":"2024-01-24T13:59:26.392132+0000","last_fullsized":"2024-01-24T13:59:26.392132+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.376686+0000","last_change":"2024-01-24T13:59:25.975250+0000","last_active":"2024-01-24T13:59:26.376686+0000","last_peered":"2024-01-24T13:59:26.376686+0000","last_clean":"2024-01-24T13:59:26.376686+0000","last_became_active":"2024-01-24T13:59:25.974992+0000","last_became_peered":"2024-01-24T13:59:25.974992+0000","last_unstale":"2024-01-24T13:59:26.376686+0000","last_undegraded":"2024-01-24T13:59:26.376686+0000","last_fullsized":"2024-01-24T13:59:26.376686+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392514+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T13:59:26.392514+0000","last_peered":"2024-01-24T13:59:26.392514+0000","last_clean":"2024-01-24T13:59:26.392514+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T13:59:26.392514+0000","last_undegraded":"2024-01-24T13:59:26.392514+0000","last_fullsized":"2024-01-24T13:59:26.392514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.405297+0000","last_change":"2024-01-24T13:59:26.404095+0000","last_active":"2024-01-24T13:59:26.405297+0000","last_peered":"2024-01-24T13:59:26.405297+0000","last_clean":"2024-01-24T13:59:26.405297+0000","last_became_active":"2024-01-24T13:59:26.403807+0000","last_became_peered":"2024-01-24T13:59:26.403807+0000","last_unstale":"2024-01-24T13:59:26.405297+0000","last_undegraded":"2024-01-24T13:59:26.405297+0000","last_fullsized":"2024-01-24T13:59:26.405297+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.376556+0000","last_change":"2024-01-24T13:59:25.977217+0000","last_active":"2024-01-24T13:59:26.376556+0000","last_peered":"2024-01-24T13:59:26.376556+0000","last_clean":"2024-01-24T13:59:26.376556+0000","last_became_active":"2024-01-24T13:59:25.976893+0000","last_became_peered":"2024-01-24T13:59:25.976893+0000","last_unstale":"2024-01-24T13:59:26.376556+0000","last_undegraded":"2024-01-24T13:59:26.376556+0000","last_fullsized":"2024-01-24T13:59:26.376556+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.388960+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T13:59:26.388960+0000","last_peered":"2024-01-24T13:59:26.388960+0000","last_clean":"2024-01-24T13:59:26.388960+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T13:59:26.388960+0000","last_undegraded":"2024-01-24T13:59:26.388960+0000","last_fullsized":"2024-01-24T13:59:26.388960+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.401805+0000","last_change":"2024-01-24T13:59:26.394231+0000","last_active":"2024-01-24T13:59:26.401805+0000","last_peered":"2024-01-24T13:59:26.401805+0000","last_clean":"2024-01-24T13:59:26.401805+0000","last_became_active":"2024-01-24T13:59:26.393918+0000","last_became_peered":"2024-01-24T13:59:26.393918+0000","last_unstale":"2024-01-24T13:59:26.401805+0000","last_undegraded":"2024-01-24T13:59:26.401805+0000","last_fullsized":"2024-01-24T13:59:26.401805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'2","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.399061+0000","last_change":"2024-01-24T13:59:25.972566+0000","last_active":"2024-01-24T13:59:26.399061+0000","last_peered":"2024-01-24T13:59:26.399061+0000","last_clean":"2024-01-24T13:59:26.399061+0000","last_became_active":"2024-01-24T13:59:25.972296+0000","last_became_peered":"2024-01-24T13:59:25.972296+0000","last_unstale":"2024-01-24T13:59:26.399061+0000","last_undegraded":"2024-01-24T13:59:26.399061+0000","last_fullsized":"2024-01-24T13:59:26.399061+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.389040+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T13:59:26.389040+0000","last_peered":"2024-01-24T13:59:26.389040+0000","last_clean":"2024-01-24T13:59:26.389040+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T13:59:26.389040+0000","last_undegraded":"2024-01-24T13:59:26.389040+0000","last_fullsized":"2024-01-24T13:59:26.389040+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.380534+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T13:59:26.380534+0000","last_peered":"2024-01-24T13:59:26.380534+0000","last_clean":"2024-01-24T13:59:26.380534+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T13:59:26.380534+0000","last_undegraded":"2024-01-24T13:59:26.380534+0000","last_fullsized":"2024-01-24T13:59:26.380534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.400939+0000","last_change":"2024-01-24T13:59:26.388213+0000","last_active":"2024-01-24T13:59:26.400939+0000","last_peered":"2024-01-24T13:59:26.400939+0000","last_clean":"2024-01-24T13:59:26.400939+0000","last_became_active":"2024-01-24T13:59:26.387917+0000","last_became_peered":"2024-01-24T13:59:26.387917+0000","last_unstale":"2024-01-24T13:59:26.400939+0000","last_undegraded":"2024-01-24T13:59:26.400939+0000","last_fullsized":"2024-01-24T13:59:26.400939+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.392428+0000","last_change":"2024-01-24T13:59:25.976888+0000","last_active":"2024-01-24T13:59:26.392428+0000","last_peered":"2024-01-24T13:59:26.392428+0000","last_clean":"2024-01-24T13:59:26.392428+0000","last_became_active":"2024-01-24T13:59:25.976551+0000","last_became_peered":"2024-01-24T13:59:25.976551+0000","last_unstale":"2024-01-24T13:59:26.392428+0000","last_undegraded":"2024-01-24T13:59:26.392428+0000","last_fullsized":"2024-01-24T13:59:26.392428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.389544+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T13:59:26.389544+0000","last_peered":"2024-01-24T13:59:26.389544+0000","last_clean":"2024-01-24T13:59:26.389544+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T13:59:26.389544+0000","last_undegraded":"2024-01-24T13:59:26.389544+0000","last_fullsized":"2024-01-24T13:59:26.389544+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.406297+0000","last_change":"2024-01-24T13:59:26.402958+0000","last_active":"2024-01-24T13:59:26.406297+0000","last_peered":"2024-01-24T13:59:26.406297+0000","last_clean":"2024-01-24T13:59:26.406297+0000","last_became_active":"2024-01-24T13:59:26.402719+0000","last_became_peered":"2024-01-24T13:59:26.402719+0000","last_unstale":"2024-01-24T13:59:26.406297+0000","last_undegraded":"2024-01-24T13:59:26.406297+0000","last_fullsized":"2024-01-24T13:59:26.406297+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.374884+0000","last_change":"2024-01-24T13:59:25.966875+0000","last_active":"2024-01-24T13:59:26.374884+0000","last_peered":"2024-01-24T13:59:26.374884+0000","last_clean":"2024-01-24T13:59:26.374884+0000","last_became_active":"2024-01-24T13:59:25.966580+0000","last_became_peered":"2024-01-24T13:59:25.966580+0000","last_unstale":"2024-01-24T13:59:26.374884+0000","last_undegraded":"2024-01-24T13:59:26.374884+0000","last_fullsized":"2024-01-24T13:59:26.374884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.377211+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T13:59:26.377211+0000","last_peered":"2024-01-24T13:59:26.377211+0000","last_clean":"2024-01-24T13:59:26.377211+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T13:59:26.377211+0000","last_undegraded":"2024-01-24T13:59:26.377211+0000","last_fullsized":"2024-01-24T13:59:26.377211+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.394754+0000","last_change":"2024-01-24T13:59:26.392089+0000","last_active":"2024-01-24T13:59:26.394754+0000","last_peered":"2024-01-24T13:59:26.394754+0000","last_clean":"2024-01-24T13:59:26.394754+0000","last_became_active":"2024-01-24T13:59:26.391832+0000","last_became_peered":"2024-01-24T13:59:26.391832+0000","last_unstale":"2024-01-24T13:59:26.394754+0000","last_undegraded":"2024-01-24T13:59:26.394754+0000","last_fullsized":"2024-01-24T13:59:26.394754+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:25.943243+0000","last_clean_scrub_stamp":"2024-01-24T13:59:25.943243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.383189+0000","last_change":"2024-01-24T13:59:25.982352+0000","last_active":"2024-01-24T13:59:26.383189+0000","last_peered":"2024-01-24T13:59:26.383189+0000","last_clean":"2024-01-24T13:59:26.383189+0000","last_became_active":"2024-01-24T13:59:25.981960+0000","last_became_peered":"2024-01-24T13:59:25.981960+0000","last_unstale":"2024-01-24T13:59:26.383189+0000","last_undegraded":"2024-01-24T13:59:26.383189+0000","last_fullsized":"2024-01-24T13:59:26.383189+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:59:24.941228+0000","last_clean_scrub_stamp":"2024-01-24T13:59:24.941228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-24T13:59:26.376644+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T13:59:26.376644+0000","last_peered":"2024-01-24T13:59:26.376644+0000","last_clean":"2024-01-24T13:59:26.376644+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T13:59:26.376644+0000","last_undegraded":"2024-01-24T13:59:26.376644+0000","last_fullsized":"2024-01-24T13:59:26.376644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":483328,"data_stored":217690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.372}]},{"osd":2,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.33600000000000002}]},{"osd":3,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48799999999999999}]},{"osd":4,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.499},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59999999999999998}]}]},{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":479232,"data_stored":217261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.378},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.497},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.37},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.309}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.41999999999999998}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.34999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":495616,"data_stored":218799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.498},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.54800000000000004}]},{"osd":3,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.625}]},{"osd":4,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.873}]},{"osd":5,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":1.044},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.94099999999999995}]},{"osd":6,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.96199999999999997}]},{"osd":7,"last update":"Wed Jan 24 13:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":1.0149999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":479232,"data_stored":217241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.45700000000000002}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.35899999999999999}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.68500000000000005}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.5}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":487424,"data_stored":217781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.42099999999999999}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.38300000000000001}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.436},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.46500000000000002}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.622}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.629}]},{"osd":6,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60699999999999998}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68799999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":483328,"data_stored":217815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.502},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.67500000000000004}]},{"osd":1,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.63200000000000001}]},{"osd":2,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.71099999999999997}]},{"osd":3,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.57799999999999996}]},{"osd":6,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.755}]},{"osd":7,"last update":"Wed Jan 24 13:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.64500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":475136,"data_stored":217171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.44},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.52800000000000002}]},{"osd":2,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.54400000000000004}]},{"osd":3,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56100000000000005}]},{"osd":4,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Jan 24 13:59:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":479232,"data_stored":217261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 13:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":1.3859999999999999}]},{"osd":1,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":1.353},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.81200000000000006}]},{"osd":2,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":6.8129999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":1.4079999999999999}]},{"osd":3,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":7.016},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.85099999999999998}]},{"osd":4,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":6.8719999999999999}]},{"osd":5,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":6.976},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":7.8789999999999996}]},{"osd":7,"last update":"Wed Jan 24 13:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":6.9080000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T13:59:35.476 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-24T13:59:35.476 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-24T13:59:35.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.651+0000 7f56cc94b700 1 Processor -- start 2024-01-24T13:59:35.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.651+0000 7f56cc94b700 1 -- start start 2024-01-24T13:59:35.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56cc94b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f56c80627b0 0x7f56c8062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56cc94b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c8064830 0x7f56c8064ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56cc94b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c8065190 con 0x7f56c8157d50 2024-01-24T13:59:35.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56cc94b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c8063320 con 0x7f56c80627b0 2024-01-24T13:59:35.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56cc94b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c8063600 con 0x7f56c8064830 2024-01-24T13:59:35.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56c659c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f56c8157d50 0x7f56c8158130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42380/0 (socket says 172.21.15.29:42380) 2024-01-24T13:59:35.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56c659c700 1 -- 172.21.15.29:0/1662890560 learned_addr learned my addr 172.21.15.29:0/1662890560 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:35.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56c6d9d700 1 --2- 172.21.15.29:0/1662890560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c8064830 0x7f56c8064ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56c5d9b700 1 --2- 172.21.15.29:0/1662890560 >> v2:172.21.15.163:3300/0 conn(0x7f56c80627b0 0x7f56c8062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56c559a700 1 -- 172.21.15.29:0/1662890560 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473717997 0 0) 0x7f56c8065190 con 0x7f56c8157d50 2024-01-24T13:59:35.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56c559a700 1 -- 172.21.15.29:0/1662890560 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56b00033f0 con 0x7f56c8157d50 2024-01-24T13:59:35.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56c6d9d700 1 -- 172.21.15.29:0/1662890560 >> v2:172.21.15.163:3300/0 conn(0x7f56c80627b0 msgr2=0x7f56c8062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:35.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56c6d9d700 1 --2- 172.21.15.29:0/1662890560 >> v2:172.21.15.163:3300/0 conn(0x7f56c80627b0 0x7f56c8062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56c6d9d700 1 -- 172.21.15.29:0/1662890560 >> v1:172.21.15.29:6789/0 conn(0x7f56c8157d50 legacy=0x7f56c8158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.652+0000 7f56c6d9d700 1 -- 172.21.15.29:0/1662890560 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c80638e0 con 0x7f56c8064830 2024-01-24T13:59:35.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.653+0000 7f56c6d9d700 1 --2- 172.21.15.29:0/1662890560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c8064830 0x7f56c8064ca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f56c0009e60 tx=0x7f56c0009780).ready entity=mon.2 client_cookie=e83c88629b4d9453 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:35.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.653+0000 7f56c559a700 1 -- 172.21.15.29:0/1662890560 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c000e070 con 0x7f56c8064830 2024-01-24T13:59:35.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.653+0000 7f56c559a700 1 -- 172.21.15.29:0/1662890560 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c000bab0 con 0x7f56c8064830 2024-01-24T13:59:35.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.653+0000 7f56c559a700 1 -- 172.21.15.29:0/1662890560 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c000be30 con 0x7f56c8064830 2024-01-24T13:59:35.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.654+0000 7f56cc94b700 1 -- 172.21.15.29:0/1662890560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c8064830 msgr2=0x7f56c8064ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.654+0000 7f56cc94b700 1 --2- 172.21.15.29:0/1662890560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c8064830 0x7f56c8064ca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f56c0009e60 tx=0x7f56c0009780).stop 2024-01-24T13:59:35.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.654+0000 7f56cc94b700 1 -- 172.21.15.29:0/1662890560 shutdown_connections 2024-01-24T13:59:35.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.654+0000 7f56cc94b700 1 --2- 172.21.15.29:0/1662890560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c8064830 0x7f56c8064ca0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.654+0000 7f56cc94b700 1 --2- 172.21.15.29:0/1662890560 >> v2:172.21.15.163:3300/0 conn(0x7f56c80627b0 0x7f56c8062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.654+0000 7f56cc94b700 1 -- 172.21.15.29:0/1662890560 >> 172.21.15.29:0/1662890560 conn(0x7f56c80c7df0 msgr2=0x7f56c80c3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:35.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.654+0000 7f56cc94b700 1 -- 172.21.15.29:0/1662890560 shutdown_connections 2024-01-24T13:59:35.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.654+0000 7f56cc94b700 1 -- 172.21.15.29:0/1662890560 wait complete. 2024-01-24T13:59:35.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.655+0000 7f56cc94b700 1 Processor -- start 2024-01-24T13:59:35.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.655+0000 7f56cc94b700 1 -- start start 2024-01-24T13:59:35.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.656+0000 7f56cc94b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c80627b0 0x7f56c8136c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.656+0000 7f56cc94b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c8064830 0x7f56c80ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.656+0000 7f56cc94b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c8157d50 0x7f56c80abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.656+0000 7f56cc94b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80ac370 con 0x7f56c8064830 2024-01-24T13:59:35.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.656+0000 7f56cc94b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80be3c0 con 0x7f56c80627b0 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.656+0000 7f56cc94b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80be980 con 0x7f56c8157d50 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.656+0000 7f56c5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c8064830 0x7f56c80ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.656+0000 7f56c5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c8064830 0x7f56c80ab990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36246/0 (socket says 172.21.15.29:36246) 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.656+0000 7f56c5d9b700 1 -- 172.21.15.29:0/949837721 learned_addr learned my addr 172.21.15.29:0/949837721 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.656+0000 7f56c6d9d700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c8157d50 0x7f56c80abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.656+0000 7f56c659c700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c80627b0 0x7f56c8136c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.657+0000 7f56c5d9b700 1 -- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c8157d50 msgr2=0x7f56c80abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.657+0000 7f56c5d9b700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c8157d50 0x7f56c80abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.657+0000 7f56c5d9b700 1 -- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c80627b0 msgr2=0x7f56c8136c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.657+0000 7f56c5d9b700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c80627b0 0x7f56c8136c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.657+0000 7f56c5d9b700 1 -- 172.21.15.29:0/949837721 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c0009a20 con 0x7f56c8064830 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.657+0000 7f56c5d9b700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c8064830 0x7f56c80ab990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f56bc00e900 tx=0x7f56bc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.657+0000 7f56b67fc700 1 -- 172.21.15.29:0/949837721 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc012600 con 0x7f56c8064830 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.657+0000 7f56b67fc700 1 -- 172.21.15.29:0/949837721 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc012760 con 0x7f56c8064830 2024-01-24T13:59:35.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.657+0000 7f56cc94b700 1 -- 172.21.15.29:0/949837721 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c80ab130 con 0x7f56c8064830 2024-01-24T13:59:35.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.657+0000 7f56b67fc700 1 -- 172.21.15.29:0/949837721 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc012ae0 con 0x7f56c8064830 2024-01-24T13:59:35.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.657+0000 7f56cc94b700 1 -- 172.21.15.29:0/949837721 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c80be690 con 0x7f56c8064830 2024-01-24T13:59:35.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.658+0000 7f56cc94b700 1 -- 172.21.15.29:0/949837721 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c80beb30 con 0x7f56c8064830 2024-01-24T13:59:35.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.659+0000 7f56b67fc700 1 -- 172.21.15.29:0/949837721 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f56bc01b400 con 0x7f56c8064830 2024-01-24T13:59:35.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.659+0000 7f56b67fc700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56a805f150 0x7f56a8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:35.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.659+0000 7f56b67fc700 1 -- 172.21.15.29:0/949837721 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f56bc07a590 con 0x7f56c8064830 2024-01-24T13:59:35.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.659+0000 7f56c659c700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56a805f150 0x7f56a8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:35.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.660+0000 7f56c659c700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56a805f150 0x7f56a8061610 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f56b8005750 tx=0x7f56b8009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:35.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.664+0000 7f56b67fc700 1 -- 172.21.15.29:0/949837721 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f56c80beb30 con 0x7f56c8064830 2024-01-24T13:59:35.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.920+0000 7f56cc94b700 1 -- 172.21.15.29:0/949837721 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f56c80475e0 con 0x7f56c8064830 2024-01-24T13:59:35.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.923+0000 7f56b67fc700 1 -- 172.21.15.29:0/949837721 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f56bc04b6a0 con 0x7f56c8064830 2024-01-24T13:59:35.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.926+0000 7f56cc94b700 1 -- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56a805f150 msgr2=0x7f56a8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.926+0000 7f56cc94b700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56a805f150 0x7f56a8061610 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f56b8005750 tx=0x7f56b8009040).stop 2024-01-24T13:59:35.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.926+0000 7f56cc94b700 1 -- 172.21.15.29:0/949837721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c8064830 msgr2=0x7f56c80ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:35.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.926+0000 7f56cc94b700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c8064830 0x7f56c80ab990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f56bc00e900 tx=0x7f56bc009060).stop 2024-01-24T13:59:35.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.926+0000 7f56cc94b700 1 -- 172.21.15.29:0/949837721 shutdown_connections 2024-01-24T13:59:35.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.926+0000 7f56cc94b700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56c8157d50 0x7f56c80abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.926+0000 7f56cc94b700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c80627b0 0x7f56c8136c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.926+0000 7f56cc94b700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56a805f150 0x7f56a8061610 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.926+0000 7f56cc94b700 1 --2- 172.21.15.29:0/949837721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56c8064830 0x7f56c80ab990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:35.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.926+0000 7f56cc94b700 1 -- 172.21.15.29:0/949837721 >> 172.21.15.29:0/949837721 conn(0x7f56c80c7df0 msgr2=0x7f56c8069290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:35.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.926+0000 7f56cc94b700 1 -- 172.21.15.29:0/949837721 shutdown_connections 2024-01-24T13:59:35.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:35.926+0000 7f56cc94b700 1 -- 172.21.15.29:0/949837721 wait complete. 2024-01-24T13:59:35.928 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-01-24T13:59:35.940 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T13:59:36.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.069+0000 7f3367236700 1 Processor -- start 2024-01-24T13:59:36.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.069+0000 7f3367236700 1 -- start start 2024-01-24T13:59:36.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.070+0000 7f3367236700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f336013a5c0 0x7f3360063a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.070+0000 7f3367236700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f33600be520 0x7f33600664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.070+0000 7f3367236700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33600bf6a0 con 0x7f33600669e0 2024-01-24T13:59:36.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.070+0000 7f3367236700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3360066e00 con 0x7f336013a5c0 2024-01-24T13:59:36.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.070+0000 7f3367236700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33600bea90 con 0x7f33600be520 2024-01-24T13:59:36.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.070+0000 7f3364fd2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f336013a5c0 0x7f3360063a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.070+0000 7f3364fd2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f336013a5c0 0x7f3360063a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36258/0 (socket says 172.21.15.29:36258) 2024-01-24T13:59:36.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.070+0000 7f33657d3700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f33600669e0 0x7f3360067a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42396/0 (socket says 172.21.15.29:42396) 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.070+0000 7f3364fd2700 1 -- 172.21.15.29:0/870705541 learned_addr learned my addr 172.21.15.29:0/870705541 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.070+0000 7f335ffff700 1 --2- 172.21.15.29:0/870705541 >> v2:172.21.15.163:3300/0 conn(0x7f33600be520 0x7f33600664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.071+0000 7f3364fd2700 1 -- 172.21.15.29:0/870705541 >> v2:172.21.15.163:3300/0 conn(0x7f33600be520 msgr2=0x7f33600664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.071+0000 7f3364fd2700 1 --2- 172.21.15.29:0/870705541 >> v2:172.21.15.163:3300/0 conn(0x7f33600be520 0x7f33600664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.071+0000 7f335f7fe700 1 -- 172.21.15.29:0/870705541 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848849453 0 0) 0x7f33600bf6a0 con 0x7f33600669e0 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.071+0000 7f3364fd2700 1 -- 172.21.15.29:0/870705541 >> v1:172.21.15.29:6789/0 conn(0x7f33600669e0 legacy=0x7f3360067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.071+0000 7f3364fd2700 1 -- 172.21.15.29:0/870705541 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33600bed70 con 0x7f336013a5c0 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.071+0000 7f3364fd2700 1 --2- 172.21.15.29:0/870705541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336013a5c0 0x7f3360063a10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f3360065a20 tx=0x7f33540090f0).ready entity=mon.0 client_cookie=89b9f9a5fa8474cd server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.071+0000 7f335f7fe700 1 -- 172.21.15.29:0/870705541 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335400b520 con 0x7f336013a5c0 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.071+0000 7f335f7fe700 1 -- 172.21.15.29:0/870705541 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335400b680 con 0x7f336013a5c0 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.071+0000 7f335f7fe700 1 -- 172.21.15.29:0/870705541 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335400ba00 con 0x7f336013a5c0 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.072+0000 7f3367236700 1 -- 172.21.15.29:0/870705541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336013a5c0 msgr2=0x7f3360063a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.072+0000 7f3367236700 1 --2- 172.21.15.29:0/870705541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336013a5c0 0x7f3360063a10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f3360065a20 tx=0x7f33540090f0).stop 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.072+0000 7f3367236700 1 -- 172.21.15.29:0/870705541 shutdown_connections 2024-01-24T13:59:36.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.072+0000 7f3367236700 1 --2- 172.21.15.29:0/870705541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336013a5c0 0x7f3360063a10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.072+0000 7f3367236700 1 --2- 172.21.15.29:0/870705541 >> v2:172.21.15.163:3300/0 conn(0x7f33600be520 0x7f33600664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.072+0000 7f3367236700 1 -- 172.21.15.29:0/870705541 >> 172.21.15.29:0/870705541 conn(0x7f33600c8bb0 msgr2=0x7f33600c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:36.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.072+0000 7f3367236700 1 -- 172.21.15.29:0/870705541 shutdown_connections 2024-01-24T13:59:36.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.072+0000 7f3367236700 1 -- 172.21.15.29:0/870705541 wait complete. 2024-01-24T13:59:36.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.073+0000 7f3367236700 1 Processor -- start 2024-01-24T13:59:36.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.073+0000 7f3367236700 1 -- start start 2024-01-24T13:59:36.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.074+0000 7f3367236700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33600669e0 0x7f3360137890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.074+0000 7f3367236700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33600be520 0x7f3360138050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.074+0000 7f3367236700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336013a5c0 0x7f3360138540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.074+0000 7f3367236700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3360138a30 con 0x7f336013a5c0 2024-01-24T13:59:36.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.074+0000 7f3367236700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33600ac470 con 0x7f33600669e0 2024-01-24T13:59:36.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.074+0000 7f3367236700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336013c580 con 0x7f33600be520 2024-01-24T13:59:36.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.074+0000 7f33657d3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336013a5c0 0x7f3360138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.074+0000 7f33657d3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336013a5c0 0x7f3360138540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36274/0 (socket says 172.21.15.29:36274) 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.074+0000 7f33657d3700 1 -- 172.21.15.29:0/667166864 learned_addr learned my addr 172.21.15.29:0/667166864 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.074+0000 7f335ffff700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33600be520 0x7f3360138050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.074+0000 7f3364fd2700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33600669e0 0x7f3360137890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.075+0000 7f33657d3700 1 -- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33600be520 msgr2=0x7f3360138050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.075+0000 7f33657d3700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33600be520 0x7f3360138050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.075+0000 7f33657d3700 1 -- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33600669e0 msgr2=0x7f3360137890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.075+0000 7f33657d3700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33600669e0 0x7f3360137890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.075+0000 7f33657d3700 1 -- 172.21.15.29:0/667166864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3354009370 con 0x7f336013a5c0 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.075+0000 7f335ffff700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33600be520 0x7f3360138050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.075+0000 7f33657d3700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336013a5c0 0x7f3360138540 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f335800c810 tx=0x7f3358007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.075+0000 7f335cff9700 1 -- 172.21.15.29:0/667166864 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358005c70 con 0x7f336013a5c0 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.075+0000 7f335cff9700 1 -- 172.21.15.29:0/667166864 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3358005dd0 con 0x7f336013a5c0 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.075+0000 7f3367236700 1 -- 172.21.15.29:0/667166864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336013c860 con 0x7f336013a5c0 2024-01-24T13:59:36.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.075+0000 7f335cff9700 1 -- 172.21.15.29:0/667166864 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358007340 con 0x7f336013a5c0 2024-01-24T13:59:36.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.075+0000 7f3367236700 1 -- 172.21.15.29:0/667166864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336013ce20 con 0x7f336013a5c0 2024-01-24T13:59:36.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.076+0000 7f3367236700 1 -- 172.21.15.29:0/667166864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33600475e0 con 0x7f336013a5c0 2024-01-24T13:59:36.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.076+0000 7f335cff9700 1 -- 172.21.15.29:0/667166864 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f33580076f0 con 0x7f336013a5c0 2024-01-24T13:59:36.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.077+0000 7f335cff9700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f334405f120 0x7f33440615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.077+0000 7f335cff9700 1 -- 172.21.15.29:0/667166864 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f335807a390 con 0x7f336013a5c0 2024-01-24T13:59:36.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.077+0000 7f3364fd2700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f334405f120 0x7f33440615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.078+0000 7f3364fd2700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f334405f120 0x7f33440615e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f3354009130 tx=0x7f3354017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:36.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.081+0000 7f335cff9700 1 -- 172.21.15.29:0/667166864 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f335804b500 con 0x7f336013a5c0 2024-01-24T13:59:36.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.309+0000 7f3367236700 1 -- 172.21.15.29:0/667166864 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33600bf050 con 0x7f336013a5c0 2024-01-24T13:59:36.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.310+0000 7f335cff9700 1 -- 172.21.15.29:0/667166864 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f335804eb20 con 0x7f336013a5c0 2024-01-24T13:59:36.310 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T13:59:36.310 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1291112242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1291112242},{"type":"v1","addr":"172.21.15.163:6837","nonce":1291112242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4793,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/241296864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":241296864},{"type":"v1","addr":"172.21.15.163:6836","nonce":241296864}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5127,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2565530631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2565530631},{"type":"v1","addr":"172.21.15.29:6833","nonce":2565530631}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:59:26.370202+0000","modified":"2024-01-24T13:59:29.380658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1245725396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1245725396},{"type":"v1","addr":"172.21.15.29:6835","nonce":1245725396}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-24T13:59:36.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.312+0000 7f3367236700 1 -- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f334405f120 msgr2=0x7f33440615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.312+0000 7f3367236700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f334405f120 0x7f33440615e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f3354009130 tx=0x7f3354017000).stop 2024-01-24T13:59:36.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.312+0000 7f3367236700 1 -- 172.21.15.29:0/667166864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336013a5c0 msgr2=0x7f3360138540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.312+0000 7f3367236700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336013a5c0 0x7f3360138540 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f335800c810 tx=0x7f3358007f10).stop 2024-01-24T13:59:36.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.312+0000 7f3367236700 1 -- 172.21.15.29:0/667166864 shutdown_connections 2024-01-24T13:59:36.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.312+0000 7f3367236700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f334405f120 0x7f33440615e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.312+0000 7f3367236700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336013a5c0 0x7f3360138540 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.312+0000 7f3367236700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33600669e0 0x7f3360137890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.312+0000 7f3367236700 1 --2- 172.21.15.29:0/667166864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33600be520 0x7f3360138050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.312+0000 7f3367236700 1 -- 172.21.15.29:0/667166864 >> 172.21.15.29:0/667166864 conn(0x7f33600c8bb0 msgr2=0x7f3360138ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:36.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.312+0000 7f3367236700 1 -- 172.21.15.29:0/667166864 shutdown_connections 2024-01-24T13:59:36.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.312+0000 7f3367236700 1 -- 172.21.15.29:0/667166864 wait complete. 2024-01-24T13:59:36.313 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 18 2024-01-24T13:59:36.728 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T13:59:26.370202+0000', 'modified': '2024-01-24T13:59:29.380658+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5133': {'gid': 5133, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/1245725396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 1245725396}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 1245725396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-24T13:59:36.729 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T13:59:36.729 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-24T13:59:36.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.869+0000 7f388edcf700 1 Processor -- start 2024-01-24T13:59:36.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.869+0000 7f388edcf700 1 -- start start 2024-01-24T13:59:36.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.870+0000 7f388edcf700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3888151c80 0x7f38880a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.870+0000 7f388edcf700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38880bca10 0x7f3888155060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.870+0000 7f388edcf700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3888063620 con 0x7f38880627b0 2024-01-24T13:59:36.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.870+0000 7f388edcf700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38880bd1f0 con 0x7f38880bca10 2024-01-24T13:59:36.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.870+0000 7f388edcf700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38880a7420 con 0x7f3888151c80 2024-01-24T13:59:36.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.870+0000 7f3887fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38880bca10 0x7f3888155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.870+0000 7f388d36c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f38880627b0 0x7f3888062be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42404/0 (socket says 172.21.15.29:42404) 2024-01-24T13:59:36.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.870+0000 7f388d36c700 1 -- 172.21.15.29:0/3639199008 learned_addr learned my addr 172.21.15.29:0/3639199008 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:36.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.870+0000 7f388cb6b700 1 --2- 172.21.15.29:0/3639199008 >> v2:172.21.15.163:3300/0 conn(0x7f3888151c80 0x7f38880a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.870+0000 7f3887fff700 1 -- 172.21.15.29:0/3639199008 >> v2:172.21.15.163:3300/0 conn(0x7f3888151c80 msgr2=0x7f38880a6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:36.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.870+0000 7f3887fff700 1 --2- 172.21.15.29:0/3639199008 >> v2:172.21.15.163:3300/0 conn(0x7f3888151c80 0x7f38880a6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.870+0000 7f3887fff700 1 -- 172.21.15.29:0/3639199008 >> v1:172.21.15.29:6789/0 conn(0x7f38880627b0 legacy=0x7f3888062be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f3887fff700 1 -- 172.21.15.29:0/3639199008 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3888152060 con 0x7f38880bca10 2024-01-24T13:59:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f3887fff700 1 --2- 172.21.15.29:0/3639199008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38880bca10 0x7f3888155060 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f3888068d10 tx=0x7f3874009060).ready entity=mon.0 client_cookie=b2fb6704e2661b3a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f38877fe700 1 -- 172.21.15.29:0/3639199008 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387400b580 con 0x7f38880bca10 2024-01-24T13:59:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f38877fe700 1 -- 172.21.15.29:0/3639199008 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387400b6e0 con 0x7f38880bca10 2024-01-24T13:59:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f38877fe700 1 -- 172.21.15.29:0/3639199008 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387400ba60 con 0x7f38880bca10 2024-01-24T13:59:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f388edcf700 1 -- 172.21.15.29:0/3639199008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38880bca10 msgr2=0x7f3888155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f388edcf700 1 --2- 172.21.15.29:0/3639199008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38880bca10 0x7f3888155060 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f3888068d10 tx=0x7f3874009060).stop 2024-01-24T13:59:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f388edcf700 1 -- 172.21.15.29:0/3639199008 shutdown_connections 2024-01-24T13:59:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f388edcf700 1 --2- 172.21.15.29:0/3639199008 >> v2:172.21.15.163:3300/0 conn(0x7f3888151c80 0x7f38880a6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f388edcf700 1 --2- 172.21.15.29:0/3639199008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38880bca10 0x7f3888155060 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f388edcf700 1 -- 172.21.15.29:0/3639199008 >> 172.21.15.29:0/3639199008 conn(0x7f38880c8580 msgr2=0x7f38880c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f388edcf700 1 -- 172.21.15.29:0/3639199008 shutdown_connections 2024-01-24T13:59:36.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.871+0000 7f388edcf700 1 -- 172.21.15.29:0/3639199008 wait complete. 2024-01-24T13:59:36.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.873+0000 7f388edcf700 1 Processor -- start 2024-01-24T13:59:36.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.873+0000 7f388edcf700 1 -- start start 2024-01-24T13:59:36.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f388edcf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38880627b0 0x7f38880a7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f388edcf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38880bca10 0x7f3888150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f388edcf700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3888151c80 0x7f38880aa030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f388edcf700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3888151330 con 0x7f38880bca10 2024-01-24T13:59:36.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f388edcf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38880a7720 con 0x7f38880627b0 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f388edcf700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38880a7a00 con 0x7f3888151c80 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f3887fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38880bca10 0x7f3888150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f3887fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38880bca10 0x7f3888150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36294/0 (socket says 172.21.15.29:36294) 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f3887fff700 1 -- 172.21.15.29:0/2991897408 learned_addr learned my addr 172.21.15.29:0/2991897408 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f388cb6b700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38880627b0 0x7f38880a7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f3887fff700 1 -- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3888151c80 msgr2=0x7f38880aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f388d36c700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3888151c80 0x7f38880aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f3887fff700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3888151c80 0x7f38880aa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f3887fff700 1 -- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38880627b0 msgr2=0x7f38880a7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f3887fff700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38880627b0 0x7f38880a7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f3887fff700 1 -- 172.21.15.29:0/2991897408 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38740092a0 con 0x7f38880bca10 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f388cb6b700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38880627b0 0x7f38880a7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f3887fff700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38880bca10 0x7f3888150e40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f3874009fd0 tx=0x7f3874017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.874+0000 7f3884ff9700 1 -- 172.21.15.29:0/2991897408 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387400b850 con 0x7f38880bca10 2024-01-24T13:59:36.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.875+0000 7f3884ff9700 1 -- 172.21.15.29:0/2991897408 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387400b9b0 con 0x7f38880bca10 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.875+0000 7f3884ff9700 1 -- 172.21.15.29:0/2991897408 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387401b690 con 0x7f38880bca10 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.875+0000 7f388edcf700 1 -- 172.21.15.29:0/2991897408 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38880a7ce0 con 0x7f38880bca10 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.875+0000 7f388edcf700 1 -- 172.21.15.29:0/2991897408 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38881644d0 con 0x7f38880bca10 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.876+0000 7f3884ff9700 1 -- 172.21.15.29:0/2991897408 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f387401bac0 con 0x7f38880bca10 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.876+0000 7f3884ff9700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f386c05f120 0x7f386c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.876+0000 7f3884ff9700 1 -- 172.21.15.29:0/2991897408 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f387407bb30 con 0x7f38880bca10 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.877+0000 7f388cb6b700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f386c05f120 0x7f386c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.877+0000 7f388cb6b700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f386c05f120 0x7f386c0615e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f3888069df0 tx=0x7f387c008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.879+0000 7f38a079ab80 1 Processor -- start 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.879+0000 7f38a079ab80 1 -- start start 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f38a079ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f38a079ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008fe2c0 0x5638008b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f38a079ab80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008b0df0 0x5638008b1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f38a079ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380082c3d0 con 0x5638008fe2c0 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f38a079ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638007f1be0 con 0x5638008b5620 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f38a079ab80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x563800807260 con 0x5638008b0df0 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f3887fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008fe2c0 0x5638008b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f3887fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008fe2c0 0x5638008b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36298/0 (socket says 172.21.15.29:36298) 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f3887fff700 1 -- 172.21.15.29:0/1605169079 learned_addr learned my addr 172.21.15.29:0/1605169079 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f388cb6b700 1 --2- 172.21.15.29:0/1605169079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f388d36c700 1 --2- 172.21.15.29:0/1605169079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008b0df0 0x5638008b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f3887fff700 1 -- 172.21.15.29:0/1605169079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008b0df0 msgr2=0x5638008b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f3887fff700 1 --2- 172.21.15.29:0/1605169079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008b0df0 0x5638008b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f3887fff700 1 -- 172.21.15.29:0/1605169079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 msgr2=0x5638008b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f3887fff700 1 --2- 172.21.15.29:0/1605169079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f3887fff700 1 -- 172.21.15.29:0/1605169079 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638008002f0 con 0x5638008fe2c0 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f388d36c700 1 --2- 172.21.15.29:0/1605169079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008b0df0 0x5638008b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:36.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f3887fff700 1 --2- 172.21.15.29:0/1605169079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008fe2c0 0x5638008b0900 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f3874050890 tx=0x7f387400e000).ready entity=mon.0 client_cookie=cbba50d23b3a342a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f3871ffb700 1 -- 172.21.15.29:0/1605169079 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874055070 con 0x5638008fe2c0 2024-01-24T13:59:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f3871ffb700 1 -- 172.21.15.29:0/1605169079 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387401bd70 con 0x5638008fe2c0 2024-01-24T13:59:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.880+0000 7f3871ffb700 1 -- 172.21.15.29:0/1605169079 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874050900 con 0x5638008fe2c0 2024-01-24T13:59:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.881+0000 7f38a079ab80 1 -- 172.21.15.29:0/1605169079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008fe2c0 msgr2=0x5638008b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.881+0000 7f38a079ab80 1 --2- 172.21.15.29:0/1605169079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008fe2c0 0x5638008b0900 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f3874050890 tx=0x7f387400e000).stop 2024-01-24T13:59:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.881+0000 7f38a079ab80 1 -- 172.21.15.29:0/1605169079 shutdown_connections 2024-01-24T13:59:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.881+0000 7f38a079ab80 1 --2- 172.21.15.29:0/1605169079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008b0df0 0x5638008b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.881+0000 7f38a079ab80 1 --2- 172.21.15.29:0/1605169079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.881+0000 7f38a079ab80 1 --2- 172.21.15.29:0/1605169079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008fe2c0 0x5638008b0900 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.881+0000 7f38a079ab80 1 -- 172.21.15.29:0/1605169079 >> 172.21.15.29:0/1605169079 conn(0x56380071e2e0 msgr2=0x5638008fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:36.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.881+0000 7f38a079ab80 1 -- 172.21.15.29:0/1605169079 shutdown_connections 2024-01-24T13:59:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.881+0000 7f38a079ab80 1 -- 172.21.15.29:0/1605169079 wait complete. 2024-01-24T13:59:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.882+0000 7f38a079ab80 1 -- start start 2024-01-24T13:59:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.882+0000 7f38a079ab80 1 Processor -- start 2024-01-24T13:59:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f38a079ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 0x5638008df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f38a079ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008e3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f38a079ab80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008e3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f38a079ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638008073a0 con 0x5638008b0df0 2024-01-24T13:59:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f38a079ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563800807620 con 0x5638008b5620 2024-01-24T13:59:36.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f38a079ab80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638007d2990 con 0x5638008fe2c0 2024-01-24T13:59:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f38a079ab80 10 client.0 ll_register_callbacks cb 0x563800875610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T13:59:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f388cb6b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 0x5638008df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f388cb6b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 0x5638008df410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36312/0 (socket says 172.21.15.29:36312) 2024-01-24T13:59:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f388cb6b700 1 -- 172.21.15.29:0/3969588698 learned_addr learned my addr 172.21.15.29:0/3969588698 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f3887fff700 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008e3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f388d36c700 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008e3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f388cb6b700 1 -- 172.21.15.29:0/3969588698 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 msgr2=0x5638008e3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f388cb6b700 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008e3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f388cb6b700 1 -- 172.21.15.29:0/3969588698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 msgr2=0x5638008e3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f388cb6b700 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008e3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f388cb6b700 1 -- 172.21.15.29:0/3969588698 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38881644d0 con 0x5638008b0df0 2024-01-24T13:59:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f388cb6b700 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 0x5638008df410 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f387c013ec0 tx=0x7f387c015040).ready entity=mon.0 client_cookie=c66f916ff3afb669 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.883+0000 7f3866ffd700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T13:59:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f3866ffd700 1 -- 172.21.15.29:0/3969588698 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c013450 con 0x5638008b0df0 2024-01-24T13:59:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f3866ffd700 1 -- 172.21.15.29:0/3969588698 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f387c00edb0 con 0x5638008b0df0 2024-01-24T13:59:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f38a079ab80 1 -- 172.21.15.29:0/3969588698 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638008afdf0 con 0x5638008b0df0 2024-01-24T13:59:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f3866ffd700 1 -- 172.21.15.29:0/3969588698 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f387c020640 con 0x5638008b0df0 2024-01-24T13:59:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f38a079ab80 10 client.5340 fetch_fsmap learned FSMap version 18 2024-01-24T13:59:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f38a079ab80 1 -- 172.21.15.29:0/3969588698 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5638007fb520 con 0x5638008b0df0 2024-01-24T13:59:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f3866ffd700 1 -- 172.21.15.29:0/3969588698 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5638008afdf0 con 0x5638008b0df0 2024-01-24T13:59:36.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f3866ffd700 1 -- 172.21.15.29:0/3969588698 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 18) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f387c0207a0 con 0x5638008b0df0 2024-01-24T13:59:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f38a079ab80 10 client.5340 fetch_fsmap finished waiting for FSMap version 18 2024-01-24T13:59:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f38a079ab80 10 client.5340 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-24T13:59:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f38a079ab80 20 client.5340 populate_metadata read hostname 'smithi029' 2024-01-24T13:59:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f38a079ab80 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.29:6834/1245725396,v1:172.21.15.29:6835/1245725396] conn(0x5638008f9da0 0x5638008fa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f38a079ab80 4 client.5340 mds_command: new command op to 5133 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T13:59:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f38a079ab80 1 -- 172.21.15.29:0/3969588698 --> [v2:172.21.15.29:6834/1245725396,v1:172.21.15.29:6835/1245725396] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638007fcd60 con 0x5638008f9da0 2024-01-24T13:59:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.884+0000 7f3887fff700 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.29:6834/1245725396,v1:172.21.15.29:6835/1245725396] conn(0x5638008f9da0 0x5638008fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.885+0000 7f3866ffd700 10 client.5340 ms_handle_connect on v2:172.21.15.29:6834/1245725396 2024-01-24T13:59:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.885+0000 7f3887fff700 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.29:6834/1245725396,v1:172.21.15.29:6835/1245725396] conn(0x5638008f9da0 0x5638008fa180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a80be895ab787547 server_cookie=a763c2b4ef7ba036 in_seq=0 out_seq=0 2024-01-24T13:59:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.885+0000 7f3866ffd700 1 -- 172.21.15.29:0/3969588698 <== mds.0 v2:172.21.15.29:6834/1245725396 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x5638007fcd60 con 0x5638008f9da0 2024-01-24T13:59:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.885+0000 7f3866ffd700 10 client.5340 handle_command_reply: tid=0 2024-01-24T13:59:36.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.885+0000 7f38a079ab80 1 client.5340 shutdown 2024-01-24T13:59:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.885+0000 7f38a079ab80 1 -- 172.21.15.29:0/3969588698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 msgr2=0x5638008df410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.885+0000 7f38a079ab80 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 0x5638008df410 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f387c013ec0 tx=0x7f387c015040).stop 2024-01-24T13:59:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.885+0000 7f38a079ab80 1 -- 172.21.15.29:0/3969588698 shutdown_connections 2024-01-24T13:59:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.885+0000 7f38a079ab80 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008e3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.885+0000 7f38a079ab80 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.29:6834/1245725396,v1:172.21.15.29:6835/1245725396] conn(0x5638008f9da0 0x5638008fa180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.885+0000 7f38a079ab80 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 0x5638008df410 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.886+0000 7f38a079ab80 1 --2- 172.21.15.29:0/3969588698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.886+0000 7f3866ffd700 0 client.5340 ms_handle_reset on v2:172.21.15.29:6834/1245725396 2024-01-24T13:59:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.886+0000 7f38a079ab80 1 -- 172.21.15.29:0/3969588698 >> 172.21.15.29:0/3969588698 conn(0x56380071e2e0 msgr2=0x5638008b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.886+0000 7f38a079ab80 1 -- 172.21.15.29:0/3969588698 shutdown_connections 2024-01-24T13:59:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.886+0000 7f38a079ab80 1 -- 172.21.15.29:0/3969588698 wait complete. 2024-01-24T13:59:36.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.886+0000 7f38a079ab80 20 client.5340 trim_cache size 0 max 16384 2024-01-24T13:59:36.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.903+0000 7f38a079ab80 1 Processor -- start 2024-01-24T13:59:36.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.903+0000 7f38a079ab80 1 -- start start 2024-01-24T13:59:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.904+0000 7f38a079ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 0x5638008b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.904+0000 7f38a079ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.904+0000 7f38a079ab80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.904+0000 7f38a079ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638008b7a80 con 0x5638008b0df0 2024-01-24T13:59:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.904+0000 7f38a079ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638008b4440 con 0x5638008b5620 2024-01-24T13:59:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.904+0000 7f38a079ab80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638008b2de0 con 0x5638008fe2c0 2024-01-24T13:59:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.904+0000 7f388cb6b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 0x5638008b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.904+0000 7f388cb6b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 0x5638008b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36320/0 (socket says 172.21.15.29:36320) 2024-01-24T13:59:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.904+0000 7f388cb6b700 1 -- 172.21.15.29:0/1717928116 learned_addr learned my addr 172.21.15.29:0/1717928116 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f388d36c700 1 --2- 172.21.15.29:0/1717928116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f3887fff700 1 --2- 172.21.15.29:0/1717928116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f388cb6b700 1 -- 172.21.15.29:0/1717928116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 msgr2=0x5638008f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:36.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f388cb6b700 1 --2- 172.21.15.29:0/1717928116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f388cb6b700 1 -- 172.21.15.29:0/1717928116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 msgr2=0x5638008f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:36.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f388cb6b700 1 --2- 172.21.15.29:0/1717928116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f388cb6b700 1 -- 172.21.15.29:0/1717928116 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638007fb270 con 0x5638008b0df0 2024-01-24T13:59:36.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f388d36c700 1 --2- 172.21.15.29:0/1717928116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:36.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f3887fff700 1 --2- 172.21.15.29:0/1717928116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:36.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f388cb6b700 1 --2- 172.21.15.29:0/1717928116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 0x5638008b38b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f387c013e80 tx=0x7f387c01f000).ready entity=mon.0 client_cookie=a9384d9bb9ffbc37 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:36.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f38727fc700 1 -- 172.21.15.29:0/1717928116 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c032030 con 0x5638008b0df0 2024-01-24T13:59:36.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f38727fc700 1 -- 172.21.15.29:0/1717928116 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c00e9f0 con 0x5638008b0df0 2024-01-24T13:59:36.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.905+0000 7f38727fc700 1 -- 172.21.15.29:0/1717928116 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c00edb0 con 0x5638008b0df0 2024-01-24T13:59:36.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.906+0000 7f38a079ab80 1 -- 172.21.15.29:0/1717928116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 msgr2=0x5638008b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.906+0000 7f38a079ab80 1 --2- 172.21.15.29:0/1717928116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 0x5638008b38b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f387c013e80 tx=0x7f387c01f000).stop 2024-01-24T13:59:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.906+0000 7f38a079ab80 1 -- 172.21.15.29:0/1717928116 shutdown_connections 2024-01-24T13:59:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.906+0000 7f38a079ab80 1 --2- 172.21.15.29:0/1717928116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.906+0000 7f38a079ab80 1 --2- 172.21.15.29:0/1717928116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b0df0 0x5638008b38b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.906+0000 7f38a079ab80 1 --2- 172.21.15.29:0/1717928116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b5620 0x5638008f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.906+0000 7f38a079ab80 1 -- 172.21.15.29:0/1717928116 >> 172.21.15.29:0/1717928116 conn(0x56380071e2e0 msgr2=0x5638008de230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.906+0000 7f38a079ab80 1 -- 172.21.15.29:0/1717928116 shutdown_connections 2024-01-24T13:59:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.906+0000 7f38a079ab80 1 -- 172.21.15.29:0/1717928116 wait complete. 2024-01-24T13:59:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.906+0000 7f38a079ab80 1 -- start start 2024-01-24T13:59:36.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.907+0000 7f38a079ab80 1 Processor -- start 2024-01-24T13:59:36.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f38a079ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b0df0 0x56380095f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f38a079ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b5620 0x5638008e26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f38a079ab80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f38a079ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638008fedb0 con 0x5638008b5620 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f38a079ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563800914560 con 0x5638008b0df0 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f38a079ab80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x563800962d50 con 0x5638008fe2c0 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f38a079ab80 10 client.0 ll_register_callbacks cb 0x563800875610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3887fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b5620 0x5638008e26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3887fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b5620 0x5638008e26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36336/0 (socket says 172.21.15.29:36336) 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3887fff700 1 -- 172.21.15.29:0/2334047962 learned_addr learned my addr 172.21.15.29:0/2334047962 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f388d36c700 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f388cb6b700 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b0df0 0x56380095f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3887fff700 1 -- 172.21.15.29:0/2334047962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 msgr2=0x5638008f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3887fff700 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3887fff700 1 -- 172.21.15.29:0/2334047962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b0df0 msgr2=0x56380095f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3887fff700 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b0df0 0x56380095f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3887fff700 1 -- 172.21.15.29:0/2334047962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638007fb520 con 0x5638008b5620 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3887fff700 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b5620 0x5638008e26f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f387400bbd0 tx=0x7f3874031040).ready entity=mon.0 client_cookie=7b62607e01d2253a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3866ffd700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3866ffd700 1 -- 172.21.15.29:0/2334047962 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387403f030 con 0x5638008b5620 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3866ffd700 1 -- 172.21.15.29:0/2334047962 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f3874040070 con 0x5638008b5620 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f38a079ab80 1 -- 172.21.15.29:0/2334047962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f387401bd70 con 0x5638008b5620 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f3866ffd700 1 -- 172.21.15.29:0/2334047962 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3874034040 con 0x5638008b5620 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f38a079ab80 10 client.5346 fetch_fsmap learned FSMap version 18 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.908+0000 7f38a079ab80 1 -- 172.21.15.29:0/2334047962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x563800674f80 con 0x5638008b5620 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.909+0000 7f3866ffd700 1 -- 172.21.15.29:0/2334047962 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387401bd70 con 0x5638008b5620 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.909+0000 7f3866ffd700 1 -- 172.21.15.29:0/2334047962 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 18) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3874018070 con 0x5638008b5620 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.909+0000 7f38a079ab80 10 client.5346 fetch_fsmap finished waiting for FSMap version 18 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.909+0000 7f38a079ab80 10 client.5346 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-24T13:59:36.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.909+0000 7f38a079ab80 20 client.5346 populate_metadata read hostname 'smithi029' 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.909+0000 7f38a079ab80 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.29:6834/1245725396,v1:172.21.15.29:6835/1245725396] conn(0x563800913660 0x563800913a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.909+0000 7f38a079ab80 4 client.5346 mds_command: new command op to 5133 tid=0[{"prefix": "status"}] 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.909+0000 7f38a079ab80 1 -- 172.21.15.29:0/2334047962 --> [v2:172.21.15.29:6834/1245725396,v1:172.21.15.29:6835/1245725396] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5638007f37d0 con 0x563800913660 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.909+0000 7f388cb6b700 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.29:6834/1245725396,v1:172.21.15.29:6835/1245725396] conn(0x563800913660 0x563800913a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.909+0000 7f3866ffd700 10 client.5346 ms_handle_connect on v2:172.21.15.29:6834/1245725396 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.909+0000 7f388cb6b700 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.29:6834/1245725396,v1:172.21.15.29:6835/1245725396] conn(0x563800913660 0x563800913a40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=11c2836d2b7b1860 server_cookie=bb1e5f8121a9ad10 in_seq=0 out_seq=0 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f3866ffd700 1 -- 172.21.15.29:0/2334047962 <== mds.0 v2:172.21.15.29:6834/1245725396 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5638007f37d0 con 0x563800913660 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f3866ffd700 10 client.5346 handle_command_reply: tid=0 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f38a079ab80 1 client.5346 shutdown 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f38a079ab80 1 -- 172.21.15.29:0/2334047962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b5620 msgr2=0x5638008e26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f38a079ab80 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b5620 0x5638008e26f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f387400bbd0 tx=0x7f3874031040).stop 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f38a079ab80 1 -- 172.21.15.29:0/2334047962 shutdown_connections 2024-01-24T13:59:36.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f38a079ab80 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5638008fe2c0 0x5638008f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f38a079ab80 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5638008b0df0 0x56380095f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f38a079ab80 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.29:6834/1245725396,v1:172.21.15.29:6835/1245725396] conn(0x563800913660 0x563800913a40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f38a079ab80 1 --2- 172.21.15.29:0/2334047962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5638008b5620 0x5638008e26f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f3866ffd700 0 client.5346 ms_handle_reset on v2:172.21.15.29:6834/1245725396 2024-01-24T13:59:36.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f38a079ab80 1 -- 172.21.15.29:0/2334047962 >> 172.21.15.29:0/2334047962 conn(0x56380071e2e0 msgr2=0x5638008de230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:36.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f38a079ab80 1 -- 172.21.15.29:0/2334047962 shutdown_connections 2024-01-24T13:59:36.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f38a079ab80 1 -- 172.21.15.29:0/2334047962 wait complete. 2024-01-24T13:59:36.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f38a079ab80 20 client.5346 trim_cache size 0 max 16384 2024-01-24T13:59:36.920 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T13:59:36.920 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T13:59:36.921 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T13:59:36.921 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 5133, 2024-01-24T13:59:36.921 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T13:59:36.921 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T13:59:36.921 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T13:59:36.921 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 10.539187099999999, 2024-01-24T13:59:36.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 18, 2024-01-24T13:59:36.921 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 41, 2024-01-24T13:59:36.921 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 41, 2024-01-24T13:59:36.921 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 17.93190001 2024-01-24T13:59:36.921 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T13:59:36.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f388edcf700 1 -- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f386c05f120 msgr2=0x7f386c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f388edcf700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f386c05f120 0x7f386c0615e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f3888069df0 tx=0x7f387c008040).stop 2024-01-24T13:59:36.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f388edcf700 1 -- 172.21.15.29:0/2991897408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38880bca10 msgr2=0x7f3888150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:36.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.910+0000 7f388edcf700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38880bca10 0x7f3888150e40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f3874009fd0 tx=0x7f3874017000).stop 2024-01-24T13:59:36.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.911+0000 7f388edcf700 1 -- 172.21.15.29:0/2991897408 shutdown_connections 2024-01-24T13:59:36.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.911+0000 7f388edcf700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3888151c80 0x7f38880aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.911+0000 7f388edcf700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38880627b0 0x7f38880a7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.911+0000 7f388edcf700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f386c05f120 0x7f386c0615e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.911+0000 7f388edcf700 1 --2- 172.21.15.29:0/2991897408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38880bca10 0x7f3888150e40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:36.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.911+0000 7f388edcf700 1 -- 172.21.15.29:0/2991897408 >> 172.21.15.29:0/2991897408 conn(0x7f38880c8580 msgr2=0x7f3888069240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:36.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.911+0000 7f388edcf700 1 -- 172.21.15.29:0/2991897408 shutdown_connections 2024-01-24T13:59:36.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:36.911+0000 7f388edcf700 1 -- 172.21.15.29:0/2991897408 wait complete. 2024-01-24T13:59:36.924 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T13:59:37.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.039+0000 7f20d8860700 1 Processor -- start 2024-01-24T13:59:37.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.039+0000 7f20d8860700 1 -- start start 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.040+0000 7f20d8860700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f20d4069bb0 0x7f20d415b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.040+0000 7f20d8860700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d4067ea0 con 0x7f20d413a890 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d8860700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d4067290 con 0x7f20d413a4b0 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d8860700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d4067570 con 0x7f20d4069bb0 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d2d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f20d4069bb0 0x7f20d415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d2d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f20d4069bb0 0x7f20d415b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36348/0 (socket says 172.21.15.29:36348) 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d2d9d700 1 -- 172.21.15.29:0/3640836131 learned_addr learned my addr 172.21.15.29:0/3640836131 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d1d9b700 1 --1- 172.21.15.29:0/3640836131 >> v1:172.21.15.29:6789/0 conn(0x7f20d413a890 0x7f20d40694a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42416/0 (socket says 172.21.15.29:42416) 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d2d9d700 1 -- 172.21.15.29:0/3640836131 >> v1:172.21.15.163:6789/0 conn(0x7f20d413a4b0 legacy=0x7f20d4066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d2d9d700 1 -- 172.21.15.29:0/3640836131 >> v1:172.21.15.29:6789/0 conn(0x7f20d413a890 legacy=0x7f20d40694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d2d9d700 1 -- 172.21.15.29:0/3640836131 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d4067850 con 0x7f20d4069bb0 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d2d9d700 1 --2- 172.21.15.29:0/3640836131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d4069bb0 0x7f20d415b910 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f20cc009f20 tx=0x7f20cc009840).ready entity=mon.0 client_cookie=db470b66bd627230 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d159a700 1 -- 172.21.15.29:0/3640836131 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00e070 con 0x7f20d4069bb0 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d159a700 1 -- 172.21.15.29:0/3640836131 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc00ba40 con 0x7f20d4069bb0 2024-01-24T13:59:37.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.041+0000 7f20d159a700 1 -- 172.21.15.29:0/3640836131 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00bdc0 con 0x7f20d4069bb0 2024-01-24T13:59:37.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.042+0000 7f20d8860700 1 -- 172.21.15.29:0/3640836131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d4069bb0 msgr2=0x7f20d415b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:37.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.042+0000 7f20d8860700 1 --2- 172.21.15.29:0/3640836131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d4069bb0 0x7f20d415b910 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f20cc009f20 tx=0x7f20cc009840).stop 2024-01-24T13:59:37.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.042+0000 7f20d8860700 1 -- 172.21.15.29:0/3640836131 shutdown_connections 2024-01-24T13:59:37.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.042+0000 7f20d8860700 1 --2- 172.21.15.29:0/3640836131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d4069bb0 0x7f20d415b910 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:37.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.042+0000 7f20d8860700 1 -- 172.21.15.29:0/3640836131 >> 172.21.15.29:0/3640836131 conn(0x7f20d40c7b10 msgr2=0x7f20d40c7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:37.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.042+0000 7f20d8860700 1 -- 172.21.15.29:0/3640836131 shutdown_connections 2024-01-24T13:59:37.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.042+0000 7f20d8860700 1 -- 172.21.15.29:0/3640836131 wait complete. 2024-01-24T13:59:37.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.044+0000 7f20d8860700 1 Processor -- start 2024-01-24T13:59:37.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.044+0000 7f20d8860700 1 -- start start 2024-01-24T13:59:37.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d8860700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d4069bb0 0x7f20d40ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:37.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d8860700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d413a4b0 0x7f20d40be9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d8860700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d413a890 0x7f20d40627b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d8860700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40acf60 con 0x7f20d413a890 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d8860700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40ad240 con 0x7f20d413a4b0 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d8860700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40ad520 con 0x7f20d4069bb0 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d2d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d413a890 0x7f20d40627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d2d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d413a890 0x7f20d40627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36360/0 (socket says 172.21.15.29:36360) 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d2d9d700 1 -- 172.21.15.29:0/3343110716 learned_addr learned my addr 172.21.15.29:0/3343110716 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d259c700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d4069bb0 0x7f20d40ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d1d9b700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d413a4b0 0x7f20d40be9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d2d9d700 1 -- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d4069bb0 msgr2=0x7f20d40ac6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d2d9d700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d4069bb0 0x7f20d40ac6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d2d9d700 1 -- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d413a4b0 msgr2=0x7f20d40be9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d2d9d700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d413a4b0 0x7f20d40be9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:37.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d2d9d700 1 -- 172.21.15.29:0/3343110716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc009ae0 con 0x7f20d413a890 2024-01-24T13:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.045+0000 7f20d2d9d700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d413a890 0x7f20d40627b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f20cc00c010 tx=0x7f20cc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.046+0000 7f20c27fc700 1 -- 172.21.15.29:0/3343110716 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc01d410 con 0x7f20d413a890 2024-01-24T13:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.046+0000 7f20c27fc700 1 -- 172.21.15.29:0/3343110716 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc01d570 con 0x7f20d413a890 2024-01-24T13:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.046+0000 7f20d8860700 1 -- 172.21.15.29:0/3343110716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d40ad800 con 0x7f20d413a890 2024-01-24T13:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.046+0000 7f20c27fc700 1 -- 172.21.15.29:0/3343110716 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc01d910 con 0x7f20d413a890 2024-01-24T13:59:37.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.046+0000 7f20d8860700 1 -- 172.21.15.29:0/3343110716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d40aac10 con 0x7f20d413a890 2024-01-24T13:59:37.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.047+0000 7f20d8860700 1 -- 172.21.15.29:0/3343110716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d40689c0 con 0x7f20d413a890 2024-01-24T13:59:37.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.047+0000 7f20c27fc700 1 -- 172.21.15.29:0/3343110716 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f20cc01dd40 con 0x7f20d413a890 2024-01-24T13:59:37.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.047+0000 7f20c27fc700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f20b405f150 0x7f20b4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:37.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.048+0000 7f20c27fc700 1 -- 172.21.15.29:0/3343110716 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f20cc07c220 con 0x7f20d413a890 2024-01-24T13:59:37.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.048+0000 7f20d259c700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f20b405f150 0x7f20b4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:37.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.048+0000 7f20d259c700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f20b405f150 0x7f20b4061610 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f20c40036e0 tx=0x7f20c4009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:37.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.052+0000 7f20c27fc700 1 -- 172.21.15.29:0/3343110716 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f20cc04d2d0 con 0x7f20d413a890 2024-01-24T13:59:37.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.277+0000 7f20d8860700 1 -- 172.21.15.29:0/3343110716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f20d40689c0 con 0x7f20d413a890 2024-01-24T13:59:37.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.277+0000 7f20c27fc700 1 -- 172.21.15.29:0/3343110716 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f20cc0508f0 con 0x7f20d413a890 2024-01-24T13:59:37.278 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-24T13:59:37.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.279+0000 7f20d8860700 1 -- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f20b405f150 msgr2=0x7f20b4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:37.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.279+0000 7f20d8860700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f20b405f150 0x7f20b4061610 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f20c40036e0 tx=0x7f20c4009040).stop 2024-01-24T13:59:37.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.279+0000 7f20d8860700 1 -- 172.21.15.29:0/3343110716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d413a890 msgr2=0x7f20d40627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:37.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.279+0000 7f20d8860700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d413a890 0x7f20d40627b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f20cc00c010 tx=0x7f20cc009840).stop 2024-01-24T13:59:37.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.279+0000 7f20d8860700 1 -- 172.21.15.29:0/3343110716 shutdown_connections 2024-01-24T13:59:37.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.279+0000 7f20d8860700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f20b405f150 0x7f20b4061610 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:37.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.279+0000 7f20d8860700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20d413a890 0x7f20d40627b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:37.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.279+0000 7f20d8860700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d4069bb0 0x7f20d40ac6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:37.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.279+0000 7f20d8860700 1 --2- 172.21.15.29:0/3343110716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d413a4b0 0x7f20d40be9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:37.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.280+0000 7f20d8860700 1 -- 172.21.15.29:0/3343110716 >> 172.21.15.29:0/3343110716 conn(0x7f20d40c7b10 msgr2=0x7f20d4150500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:37.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.280+0000 7f20d8860700 1 -- 172.21.15.29:0/3343110716 shutdown_connections 2024-01-24T13:59:37.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:37.280+0000 7f20d8860700 1 -- 172.21.15.29:0/3343110716 wait complete. 2024-01-24T13:59:37.292 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-24T13:59:37.292 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T13:59:37.292 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T13:59:37.292 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-24T13:59:37.292 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-24T13:59:37.292 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-24T13:59:37.292 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-24T13:59:37.292 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T13:59:37.292 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T13:59:37.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:59:37.293 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T13:59:37.312 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T13:59:37.312 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T13:59:37.312 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T13:59:37.312 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:59:37.312 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T13:59:37.313 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:59:37.313 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:59:37.313 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T13:59:37.313 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T13:59:37.313 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T13:59:37.313 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T13:59:37.313 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T13:59:37.313 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42273sec preferred_lft 42273sec 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:59:37.314 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::24bc:1cff:fe57:e704/64 scope link 2024-01-24T13:59:37.315 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:59:37.315 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T13:59:37.315 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T13:59:37.315 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::2034:aaff:fef3:f12b/64 scope link 2024-01-24T13:59:37.315 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:59:37.315 INFO:teuthology.orchestra.run.smithi029.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T13:59:37.315 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether ba:ca:42:4a:9e:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T13:59:37.315 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::b8ca:42ff:fe4a:9e52/64 scope link 2024-01-24T13:59:37.315 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T13:59:37.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:59:37.315 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T13:59:37.384 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T13:59:37.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:59:37.386 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T13:59:37.451 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T13:59:37.451 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T13:59:37.451 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T13:59:37.451 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T13:59:37.452 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T13:59:37.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:59:37.452 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T13:59:37.452 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T13:59:37.452 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T13:59:37.453 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T13:59:37.453 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T13:59:37.453 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T13:59:37.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:59:37.577 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T13:59:37.597 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-24T13:59:37.597 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T13:59:37.597 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T13:59:37.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:59:37.598 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T13:59:37.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:59:37.674 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-01-24T13:59:37.739 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T13:59:37.739 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:59:37.832 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:59:37.833 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T13:59:37.855 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T13:59:37.855 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T13:59:37.859 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T13:59:37.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T13:59:37.859 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-24T13:59:37.892 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-24T13:59:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.070+0000 7facc0875700 1 Processor -- start 2024-01-24T13:59:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.070+0000 7facc0875700 1 -- start start 2024-01-24T13:59:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.071+0000 7facc0875700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7facbc0bd130 0x7facbc15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:38.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.071+0000 7facc0875700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facbc0691b0 con 0x7facbc133c90 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.071+0000 7facc0875700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facbc0685a0 con 0x7facbc134070 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.071+0000 7facc0875700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facbc068880 con 0x7facbc0bd130 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.071+0000 7facbad9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7facbc0bd130 0x7facbc15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.071+0000 7facba59c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7facbc133c90 0x7facbc0699f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42418/0 (socket says 172.21.15.29:42418) 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.071+0000 7facba59c700 1 -- 172.21.15.29:0/4190637340 learned_addr learned my addr 172.21.15.29:0/4190637340 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.072+0000 7facbad9d700 1 -- 172.21.15.29:0/4190637340 >> v1:172.21.15.163:6789/0 conn(0x7facbc134070 legacy=0x7facbc0bca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.072+0000 7facbad9d700 1 -- 172.21.15.29:0/4190637340 >> v1:172.21.15.29:6789/0 conn(0x7facbc133c90 legacy=0x7facbc0699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.072+0000 7facbad9d700 1 -- 172.21.15.29:0/4190637340 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc068b60 con 0x7facbc0bd130 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.072+0000 7facb959a700 1 -- 172.21.15.29:0/4190637340 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286775170 0 0) 0x7facbc0691b0 con 0x7facbc133c90 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.072+0000 7facbad9d700 1 --2- 172.21.15.29:0/4190637340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facbc0bd130 0x7facbc15bd40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7facb4009e60 tx=0x7facb4009780).ready entity=mon.0 client_cookie=2bc638f59672c7aa server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.072+0000 7facb959a700 1 -- 172.21.15.29:0/4190637340 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb400e070 con 0x7facbc0bd130 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.072+0000 7facb959a700 1 -- 172.21.15.29:0/4190637340 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb400bc30 con 0x7facbc0bd130 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.072+0000 7facb959a700 1 -- 172.21.15.29:0/4190637340 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4012410 con 0x7facbc0bd130 2024-01-24T13:59:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.073+0000 7facc0875700 1 -- 172.21.15.29:0/4190637340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facbc0bd130 msgr2=0x7facbc15bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:38.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.073+0000 7facc0875700 1 --2- 172.21.15.29:0/4190637340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facbc0bd130 0x7facbc15bd40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7facb4009e60 tx=0x7facb4009780).stop 2024-01-24T13:59:38.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.073+0000 7facc0875700 1 -- 172.21.15.29:0/4190637340 shutdown_connections 2024-01-24T13:59:38.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.073+0000 7facc0875700 1 --2- 172.21.15.29:0/4190637340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facbc0bd130 0x7facbc15bd40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:38.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.073+0000 7facc0875700 1 -- 172.21.15.29:0/4190637340 >> 172.21.15.29:0/4190637340 conn(0x7facbc0c7480 msgr2=0x7facbc0c3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:38.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.073+0000 7facc0875700 1 -- 172.21.15.29:0/4190637340 shutdown_connections 2024-01-24T13:59:38.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.073+0000 7facc0875700 1 -- 172.21.15.29:0/4190637340 wait complete. 2024-01-24T13:59:38.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.074+0000 7facc0875700 1 Processor -- start 2024-01-24T13:59:38.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.074+0000 7facc0875700 1 -- start start 2024-01-24T13:59:38.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facc0875700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facbc0bd130 0x7facbc1333e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:38.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facc0875700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facbc133c90 0x7facbc0c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:38.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facc0875700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facbc134070 0x7facbc0c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:38.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facc0875700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc0bedd0 con 0x7facbc0bd130 2024-01-24T13:59:38.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facc0875700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc1338d0 con 0x7facbc133c90 2024-01-24T13:59:38.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facc0875700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facbc0be1c0 con 0x7facbc134070 2024-01-24T13:59:38.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facba59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facbc0bd130 0x7facbc1333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:38.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facba59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facbc0bd130 0x7facbc1333e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36384/0 (socket says 172.21.15.29:36384) 2024-01-24T13:59:38.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facba59c700 1 -- 172.21.15.29:0/866621012 learned_addr learned my addr 172.21.15.29:0/866621012 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:38.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facbad9d700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facbc134070 0x7facbc0c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:38.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facb9d9b700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facbc133c90 0x7facbc0c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:38.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facba59c700 1 -- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facbc134070 msgr2=0x7facbc0c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:38.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facba59c700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facbc134070 0x7facbc0c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:38.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facba59c700 1 -- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facbc133c90 msgr2=0x7facbc0c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:38.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facba59c700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facbc133c90 0x7facbc0c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:38.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facba59c700 1 -- 172.21.15.29:0/866621012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb4009a20 con 0x7facbc0bd130 2024-01-24T13:59:38.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facbad9d700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facbc134070 0x7facbc0c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:38.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facb9d9b700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facbc133c90 0x7facbc0c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:38.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.075+0000 7facba59c700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facbc0bd130 0x7facbc1333e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7facac00c4c0 tx=0x7facac007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:38.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.076+0000 7facb27fc700 1 -- 172.21.15.29:0/866621012 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac0072e0 con 0x7facbc0bd130 2024-01-24T13:59:38.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.076+0000 7facb27fc700 1 -- 172.21.15.29:0/866621012 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facac007440 con 0x7facbc0bd130 2024-01-24T13:59:38.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.076+0000 7facc0875700 1 -- 172.21.15.29:0/866621012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facbc0be4a0 con 0x7facbc0bd130 2024-01-24T13:59:38.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.076+0000 7facb27fc700 1 -- 172.21.15.29:0/866621012 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac0057a0 con 0x7facbc0bd130 2024-01-24T13:59:38.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.076+0000 7facc0875700 1 -- 172.21.15.29:0/866621012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facbc0bea60 con 0x7facbc0bd130 2024-01-24T13:59:38.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.077+0000 7facc0875700 1 -- 172.21.15.29:0/866621012 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facbc0475e0 con 0x7facbc0bd130 2024-01-24T13:59:38.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.077+0000 7facb27fc700 1 -- 172.21.15.29:0/866621012 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7facac005b90 con 0x7facbc0bd130 2024-01-24T13:59:38.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.077+0000 7facb27fc700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fac9c05f150 0x7fac9c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:38.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.078+0000 7facb27fc700 1 -- 172.21.15.29:0/866621012 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7facac079c80 con 0x7facbc0bd130 2024-01-24T13:59:38.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.078+0000 7facb9d9b700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fac9c05f150 0x7fac9c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:38.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.078+0000 7facb9d9b700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fac9c05f150 0x7fac9c061610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7facbc14edc0 tx=0x7faca4009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:38.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.082+0000 7facb27fc700 1 -- 172.21.15.29:0/866621012 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7facac04adf0 con 0x7facbc0bd130 2024-01-24T13:59:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.291+0000 7facc0875700 1 -- 172.21.15.29:0/866621012 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7facbc0bef10 con 0x7fac9c05f150 2024-01-24T13:59:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.292+0000 7facb27fc700 1 -- 172.21.15.29:0/866621012 <== mgr.4103 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7faca4012410 con 0x7fac9c05f150 2024-01-24T13:59:38.293 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T13:59:38.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T13:59:38.293 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-01-24T13:59:38.293 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T13:59:38.293 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T13:59:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.294+0000 7facc0875700 1 -- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fac9c05f150 msgr2=0x7fac9c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.294+0000 7facc0875700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fac9c05f150 0x7fac9c061610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7facbc14edc0 tx=0x7faca4009040).stop 2024-01-24T13:59:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.295+0000 7facc0875700 1 -- 172.21.15.29:0/866621012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facbc0bd130 msgr2=0x7facbc1333e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.295+0000 7facc0875700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facbc0bd130 0x7facbc1333e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7facac00c4c0 tx=0x7facac007c10).stop 2024-01-24T13:59:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.295+0000 7facc0875700 1 -- 172.21.15.29:0/866621012 shutdown_connections 2024-01-24T13:59:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.295+0000 7facc0875700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facbc134070 0x7facbc0c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.295+0000 7facc0875700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fac9c05f150 0x7fac9c061610 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:38.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.295+0000 7facc0875700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facbc0bd130 0x7facbc1333e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:38.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.295+0000 7facc0875700 1 --2- 172.21.15.29:0/866621012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facbc133c90 0x7facbc0c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:38.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.295+0000 7facc0875700 1 -- 172.21.15.29:0/866621012 >> 172.21.15.29:0/866621012 conn(0x7facbc0c7480 msgr2=0x7facbc065370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:38.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.295+0000 7facc0875700 1 -- 172.21.15.29:0/866621012 shutdown_connections 2024-01-24T13:59:38.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.295+0000 7facc0875700 1 -- 172.21.15.29:0/866621012 wait complete. 2024-01-24T13:59:38.308 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-01-24T13:59:38.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.440+0000 7f9d8528c700 1 Processor -- start 2024-01-24T13:59:38.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.440+0000 7f9d8528c700 1 -- start start 2024-01-24T13:59:38.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d8528c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9d800c5af0 0x7f9d800c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:38.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d8528c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d800bfd20 0x7f9d800c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:38.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d8528c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d800c6820 con 0x7f9d80064740 2024-01-24T13:59:38.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d8528c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d801556b0 con 0x7f9d800c5af0 2024-01-24T13:59:38.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d8528c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d80155220 con 0x7f9d800bfd20 2024-01-24T13:59:38.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d7effd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9d80064740 0x7f9d80064b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42434/0 (socket says 172.21.15.29:42434) 2024-01-24T13:59:38.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d7effd700 1 -- 172.21.15.29:0/2332995947 learned_addr learned my addr 172.21.15.29:0/2332995947 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:38.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d7f7fe700 1 --2- 172.21.15.29:0/2332995947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d800bfd20 0x7f9d800c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:38.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d7e7fc700 1 --2- 172.21.15.29:0/2332995947 >> v2:172.21.15.163:3300/0 conn(0x7f9d800c5af0 0x7f9d800c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:38.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d7dffb700 1 -- 172.21.15.29:0/2332995947 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566163483 0 0) 0x7f9d70001110 con 0x7f9d80064740 2024-01-24T13:59:38.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d7dffb700 1 -- 172.21.15.29:0/2332995947 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d680033f0 con 0x7f9d80064740 2024-01-24T13:59:38.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d7f7fe700 1 -- 172.21.15.29:0/2332995947 >> v2:172.21.15.163:3300/0 conn(0x7f9d800c5af0 msgr2=0x7f9d800c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:38.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d7f7fe700 1 --2- 172.21.15.29:0/2332995947 >> v2:172.21.15.163:3300/0 conn(0x7f9d800c5af0 0x7f9d800c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:38.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d7f7fe700 1 -- 172.21.15.29:0/2332995947 >> v1:172.21.15.29:6789/0 conn(0x7f9d80064740 legacy=0x7f9d80064b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:38.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d7f7fe700 1 -- 172.21.15.29:0/2332995947 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d80154680 con 0x7f9d800bfd20 2024-01-24T13:59:38.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.441+0000 7f9d7e7fc700 1 --2- 172.21.15.29:0/2332995947 >> v2:172.21.15.163:3300/0 conn(0x7f9d800c5af0 0x7f9d800c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:59:38.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.442+0000 7f9d7f7fe700 1 --2- 172.21.15.29:0/2332995947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d800bfd20 0x7f9d800c01b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f9d78009e60 tx=0x7f9d78009780).ready entity=mon.2 client_cookie=8b6299f8bf4d05d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:38.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.442+0000 7f9d7dffb700 1 -- 172.21.15.29:0/2332995947 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7800e070 con 0x7f9d800bfd20 2024-01-24T13:59:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.442+0000 7f9d7dffb700 1 -- 172.21.15.29:0/2332995947 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7800bab0 con 0x7f9d800bfd20 2024-01-24T13:59:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.442+0000 7f9d7dffb700 1 -- 172.21.15.29:0/2332995947 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7800be30 con 0x7f9d800bfd20 2024-01-24T13:59:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.442+0000 7f9d8528c700 1 -- 172.21.15.29:0/2332995947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d800bfd20 msgr2=0x7f9d800c01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.442+0000 7f9d8528c700 1 --2- 172.21.15.29:0/2332995947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d800bfd20 0x7f9d800c01b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f9d78009e60 tx=0x7f9d78009780).stop 2024-01-24T13:59:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.442+0000 7f9d8528c700 1 -- 172.21.15.29:0/2332995947 shutdown_connections 2024-01-24T13:59:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.442+0000 7f9d8528c700 1 --2- 172.21.15.29:0/2332995947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d800bfd20 0x7f9d800c01b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.442+0000 7f9d8528c700 1 --2- 172.21.15.29:0/2332995947 >> v2:172.21.15.163:3300/0 conn(0x7f9d800c5af0 0x7f9d800c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.442+0000 7f9d8528c700 1 -- 172.21.15.29:0/2332995947 >> 172.21.15.29:0/2332995947 conn(0x7f9d800cb890 msgr2=0x7f9d800c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.442+0000 7f9d8528c700 1 -- 172.21.15.29:0/2332995947 shutdown_connections 2024-01-24T13:59:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.442+0000 7f9d8528c700 1 -- 172.21.15.29:0/2332995947 wait complete. 2024-01-24T13:59:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.444+0000 7f9d8528c700 1 Processor -- start 2024-01-24T13:59:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.444+0000 7f9d8528c700 1 -- start start 2024-01-24T13:59:38.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d8528c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d80064740 0x7f9d800c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:38.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d8528c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d800bfd20 0x7f9d800c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:38.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d8528c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d800c5af0 0x7f9d8014e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d8528c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800c4050 con 0x7f9d800c5af0 2024-01-24T13:59:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d8528c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800c4b50 con 0x7f9d80064740 2024-01-24T13:59:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d8528c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800c31d0 con 0x7f9d800bfd20 2024-01-24T13:59:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d7f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d800c5af0 0x7f9d8014e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d7f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d800c5af0 0x7f9d8014e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36396/0 (socket says 172.21.15.29:36396) 2024-01-24T13:59:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d7f7fe700 1 -- 172.21.15.29:0/3090391827 learned_addr learned my addr 172.21.15.29:0/3090391827 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d7e7fc700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d800bfd20 0x7f9d800c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d7effd700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d80064740 0x7f9d800c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d7f7fe700 1 -- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d800bfd20 msgr2=0x7f9d800c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d7f7fe700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d800bfd20 0x7f9d800c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d7f7fe700 1 -- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d80064740 msgr2=0x7f9d800c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d7f7fe700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d80064740 0x7f9d800c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d7f7fe700 1 -- 172.21.15.29:0/3090391827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d740092a0 con 0x7f9d800c5af0 2024-01-24T13:59:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d7f7fe700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d800c5af0 0x7f9d8014e960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f9d78013cc0 tx=0x7f9d78009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.445+0000 7f9d66ffd700 1 -- 172.21.15.29:0/3090391827 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7801d630 con 0x7f9d800c5af0 2024-01-24T13:59:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.446+0000 7f9d66ffd700 1 -- 172.21.15.29:0/3090391827 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7801d790 con 0x7f9d800c5af0 2024-01-24T13:59:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.446+0000 7f9d66ffd700 1 -- 172.21.15.29:0/3090391827 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7801db50 con 0x7f9d800c5af0 2024-01-24T13:59:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.446+0000 7f9d8528c700 1 -- 172.21.15.29:0/3090391827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d78009a20 con 0x7f9d800c5af0 2024-01-24T13:59:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.446+0000 7f9d8528c700 1 -- 172.21.15.29:0/3090391827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d800c2b80 con 0x7f9d800c5af0 2024-01-24T13:59:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.447+0000 7f9d8528c700 1 -- 172.21.15.29:0/3090391827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d80063da0 con 0x7f9d800c5af0 2024-01-24T13:59:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.447+0000 7f9d66ffd700 1 -- 172.21.15.29:0/3090391827 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9d78003040 con 0x7f9d800c5af0 2024-01-24T13:59:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.447+0000 7f9d66ffd700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9d6c05f150 0x7f9d6c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.447+0000 7f9d66ffd700 1 -- 172.21.15.29:0/3090391827 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f9d7807c140 con 0x7f9d800c5af0 2024-01-24T13:59:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.448+0000 7f9d7effd700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9d6c05f150 0x7f9d6c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.448+0000 7f9d7effd700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9d6c05f150 0x7f9d6c061610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f9d70001270 tx=0x7f9d70009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:38.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.452+0000 7f9d66ffd700 1 -- 172.21.15.29:0/3090391827 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9d7804d2b0 con 0x7f9d800c5af0 2024-01-24T13:59:38.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:38.684+0000 7f9d8528c700 1 -- 172.21.15.29:0/3090391827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f9d800c3340 con 0x7f9d800c5af0 2024-01-24T13:59:39.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.673+0000 7f9d66ffd700 1 -- 172.21.15.29:0/3090391827 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 5) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f9d78079db0 con 0x7f9d800c5af0 2024-01-24T13:59:39.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.674+0000 7f9d66ffd700 1 -- 172.21.15.29:0/3090391827 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v5) v1 ==== 92+0+0 (secure 0 0 0) 0x7f9d780433e0 con 0x7f9d800c5af0 2024-01-24T13:59:39.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.677+0000 7f9d8528c700 1 -- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9d6c05f150 msgr2=0x7f9d6c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:39.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.677+0000 7f9d8528c700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9d6c05f150 0x7f9d6c061610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f9d70001270 tx=0x7f9d70009040).stop 2024-01-24T13:59:39.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.677+0000 7f9d8528c700 1 -- 172.21.15.29:0/3090391827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d800c5af0 msgr2=0x7f9d8014e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:39.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.677+0000 7f9d8528c700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d800c5af0 0x7f9d8014e960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f9d78013cc0 tx=0x7f9d78009780).stop 2024-01-24T13:59:39.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.677+0000 7f9d8528c700 1 -- 172.21.15.29:0/3090391827 shutdown_connections 2024-01-24T13:59:39.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.677+0000 7f9d8528c700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9d6c05f150 0x7f9d6c061610 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:39.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.677+0000 7f9d8528c700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d800c5af0 0x7f9d8014e960 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:39.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.677+0000 7f9d8528c700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d80064740 0x7f9d800c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:39.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.677+0000 7f9d8528c700 1 --2- 172.21.15.29:0/3090391827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d800bfd20 0x7f9d800c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:39.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.677+0000 7f9d8528c700 1 -- 172.21.15.29:0/3090391827 >> 172.21.15.29:0/3090391827 conn(0x7f9d800cb890 msgr2=0x7f9d80063140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:39.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.677+0000 7f9d8528c700 1 -- 172.21.15.29:0/3090391827 shutdown_connections 2024-01-24T13:59:39.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.677+0000 7f9d8528c700 1 -- 172.21.15.29:0/3090391827 wait complete. 2024-01-24T13:59:39.693 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-01-24T13:59:39.811 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:39.811+0000 7fba89496680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T13:59:39.828 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:39.828+0000 7fee7843d680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T13:59:39.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.830+0000 7f14e063b700 1 Processor -- start 2024-01-24T13:59:39.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.830+0000 7f14e063b700 1 -- start start 2024-01-24T13:59:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.831+0000 7f14e063b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f14d813a5e0 0x7f14d80639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.831+0000 7f14e063b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14d8066bd0 0x7f14d80bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.831+0000 7f14e063b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d80679f0 con 0x7f14d8063eb0 2024-01-24T13:59:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.831+0000 7f14e063b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d8067cd0 con 0x7f14d813a5e0 2024-01-24T13:59:39.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.831+0000 7f14e063b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d8067fb0 con 0x7f14d8066bd0 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.831+0000 7f14de3d7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f14d813a5e0 0x7f14d80639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.831+0000 7f14de3d7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f14d813a5e0 0x7f14d80639c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36414/0 (socket says 172.21.15.29:36414) 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.831+0000 7f14de3d7700 1 -- 172.21.15.29:0/1853570678 learned_addr learned my addr 172.21.15.29:0/1853570678 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.831+0000 7f14debd8700 1 --2- 172.21.15.29:0/1853570678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14d8066bd0 0x7f14d80bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.832+0000 7f14de3d7700 1 -- 172.21.15.29:0/1853570678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14d8066bd0 msgr2=0x7f14d80bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.832+0000 7f14de3d7700 1 --2- 172.21.15.29:0/1853570678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14d8066bd0 0x7f14d80bdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.832+0000 7f14de3d7700 1 -- 172.21.15.29:0/1853570678 >> v1:172.21.15.163:6789/0 conn(0x7f14d8063eb0 legacy=0x7f14d80664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.832+0000 7f14dd3d5700 1 -- 172.21.15.29:0/1853570678 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470055038 0 0) 0x7f14d80679f0 con 0x7f14d8063eb0 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.832+0000 7f14de3d7700 1 -- 172.21.15.29:0/1853570678 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d8068290 con 0x7f14d813a5e0 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.832+0000 7f14de3d7700 1 --2- 172.21.15.29:0/1853570678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14d813a5e0 0x7f14d80639c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f14d4006fd0 tx=0x7f14d40090f0).ready entity=mon.0 client_cookie=752bdbaed019ce05 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.832+0000 7f14dd3d5700 1 -- 172.21.15.29:0/1853570678 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400b580 con 0x7f14d813a5e0 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.832+0000 7f14dd3d5700 1 -- 172.21.15.29:0/1853570678 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d400b6e0 con 0x7f14d813a5e0 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.832+0000 7f14dd3d5700 1 -- 172.21.15.29:0/1853570678 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400ba60 con 0x7f14d813a5e0 2024-01-24T13:59:39.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.833+0000 7f14e063b700 1 -- 172.21.15.29:0/1853570678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14d813a5e0 msgr2=0x7f14d80639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:39.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.833+0000 7f14e063b700 1 --2- 172.21.15.29:0/1853570678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14d813a5e0 0x7f14d80639c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f14d4006fd0 tx=0x7f14d40090f0).stop 2024-01-24T13:59:39.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.833+0000 7f14e063b700 1 -- 172.21.15.29:0/1853570678 shutdown_connections 2024-01-24T13:59:39.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.833+0000 7f14e063b700 1 --2- 172.21.15.29:0/1853570678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14d8066bd0 0x7f14d80bdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:39.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.833+0000 7f14e063b700 1 --2- 172.21.15.29:0/1853570678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14d813a5e0 0x7f14d80639c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:39.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.833+0000 7f14e063b700 1 -- 172.21.15.29:0/1853570678 >> 172.21.15.29:0/1853570678 conn(0x7f14d80c8bd0 msgr2=0x7f14d80c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:39.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.833+0000 7f14e063b700 1 -- 172.21.15.29:0/1853570678 shutdown_connections 2024-01-24T13:59:39.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.833+0000 7f14e063b700 1 -- 172.21.15.29:0/1853570678 wait complete. 2024-01-24T13:59:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.835+0000 7f14e063b700 1 Processor -- start 2024-01-24T13:59:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.835+0000 7f14e063b700 1 -- start start 2024-01-24T13:59:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14e063b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14d8063eb0 0x7f14d80a9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14e063b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14d8066bd0 0x7f14d80accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14e063b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14d813a5e0 0x7f14d8138a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14e063b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80a9b20 con 0x7f14d8066bd0 2024-01-24T13:59:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14e063b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80ad1a0 con 0x7f14d813a5e0 2024-01-24T13:59:39.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14e063b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80aa920 con 0x7f14d8063eb0 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14ddbd6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14d8066bd0 0x7f14d80accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14de3d7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14d8063eb0 0x7f14d80a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14ddbd6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14d8066bd0 0x7f14d80accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36418/0 (socket says 172.21.15.29:36418) 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14ddbd6700 1 -- 172.21.15.29:0/2302572625 learned_addr learned my addr 172.21.15.29:0/2302572625 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14debd8700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14d813a5e0 0x7f14d8138a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14ddbd6700 1 -- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14d8063eb0 msgr2=0x7f14d80a9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14ddbd6700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14d8063eb0 0x7f14d80a9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14ddbd6700 1 -- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14d813a5e0 msgr2=0x7f14d8138a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14ddbd6700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14d813a5e0 0x7f14d8138a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14ddbd6700 1 -- 172.21.15.29:0/2302572625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d4009370 con 0x7f14d8066bd0 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14de3d7700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14d8063eb0 0x7f14d80a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.836+0000 7f14ddbd6700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14d8066bd0 0x7f14d80accb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f14c8001eb0 tx=0x7f14c8007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.837+0000 7f14ce7fc700 1 -- 172.21.15.29:0/2302572625 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c800ce60 con 0x7f14d8066bd0 2024-01-24T13:59:39.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.837+0000 7f14ce7fc700 1 -- 172.21.15.29:0/2302572625 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c800f470 con 0x7f14d8066bd0 2024-01-24T13:59:39.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.837+0000 7f14e063b700 1 -- 172.21.15.29:0/2302572625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d8138ef0 con 0x7f14d8066bd0 2024-01-24T13:59:39.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.837+0000 7f14ce7fc700 1 -- 172.21.15.29:0/2302572625 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c800f810 con 0x7f14d8066bd0 2024-01-24T13:59:39.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.837+0000 7f14e063b700 1 -- 172.21.15.29:0/2302572625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d80a9ff0 con 0x7f14d8066bd0 2024-01-24T13:59:39.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.838+0000 7f14e063b700 1 -- 172.21.15.29:0/2302572625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d80475e0 con 0x7f14d8066bd0 2024-01-24T13:59:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.838+0000 7f14ce7fc700 1 -- 172.21.15.29:0/2302572625 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f14c800fb80 con 0x7f14d8066bd0 2024-01-24T13:59:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.839+0000 7f14ce7fc700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f14bc05f1a0 0x7f14bc061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.839+0000 7f14ce7fc700 1 -- 172.21.15.29:0/2302572625 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f14c8079820 con 0x7f14d8066bd0 2024-01-24T13:59:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.839+0000 7f14de3d7700 1 -- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f14bc05f1a0 msgr2=0x7f14bc061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T13:59:39.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.839+0000 7f14de3d7700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f14bc05f1a0 0x7f14bc061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T13:59:39.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:39.843+0000 7f14ce7fc700 1 -- 172.21.15.29:0/2302572625 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f14c804a990 con 0x7f14d8066bd0 2024-01-24T13:59:39.874 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:39.875+0000 7fba89496680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T13:59:39.902 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:39.902+0000 7fee7843d680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T13:59:40.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.028+0000 7f14e063b700 1 -- 172.21.15.29:0/2302572625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f14d8068570 con 0x7f14d8066bd0 2024-01-24T13:59:40.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.032+0000 7f14ce7fc700 1 -- 172.21.15.29:0/2302572625 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v3)=0 v3) v1 ==== 138+0+0 (secure 0 0 0) 0x7f14c8077e60 con 0x7f14d8066bd0 2024-01-24T13:59:40.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.034+0000 7f14e063b700 1 -- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f14bc05f1a0 msgr2=0x7f14bc061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T13:59:40.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.034+0000 7f14e063b700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f14bc05f1a0 0x7f14bc061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T13:59:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.034+0000 7f14e063b700 1 -- 172.21.15.29:0/2302572625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14d8066bd0 msgr2=0x7f14d80accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.034+0000 7f14e063b700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14d8066bd0 0x7f14d80accb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f14c8001eb0 tx=0x7f14c8007c70).stop 2024-01-24T13:59:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.034+0000 7f14e063b700 1 -- 172.21.15.29:0/2302572625 shutdown_connections 2024-01-24T13:59:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.034+0000 7f14e063b700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14d813a5e0 0x7f14d8138a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.034+0000 7f14e063b700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14d8063eb0 0x7f14d80a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.034+0000 7f14e063b700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f14bc05f1a0 0x7f14bc061660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T13:59:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.034+0000 7f14e063b700 1 --2- 172.21.15.29:0/2302572625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14d8066bd0 0x7f14d80accb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.034+0000 7f14e063b700 1 -- 172.21.15.29:0/2302572625 >> 172.21.15.29:0/2302572625 conn(0x7f14d80c8bd0 msgr2=0x7f14d8139990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.034+0000 7f14e063b700 1 -- 172.21.15.29:0/2302572625 shutdown_connections 2024-01-24T13:59:40.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.034+0000 7f14e063b700 1 -- 172.21.15.29:0/2302572625 wait complete. 2024-01-24T13:59:40.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:40.048 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-01-24T13:59:40.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:40.071 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2024-01-24T13:59:40.131 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir0 1M 2024-01-24T13:59:40.150 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:40.151+0000 7fba89496680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T13:59:40.219 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:40.219+0000 7fee7843d680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T13:59:40.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.307+0000 7f52d6cf0700 1 Processor -- start 2024-01-24T13:59:40.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.307+0000 7f52d6cf0700 1 -- start start 2024-01-24T13:59:40.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.308+0000 7f52d6cf0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f52d0151e50 0x7f52d00c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:40.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.308+0000 7f52d6cf0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f52d00c7f70 0x7f52d0155420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:40.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.308+0000 7f52d6cf0700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d0154f90 con 0x7f52d0064780 2024-01-24T13:59:40.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.308+0000 7f52d6cf0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d0155910 con 0x7f52d0151e50 2024-01-24T13:59:40.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.308+0000 7f52d6cf0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d00a9f10 con 0x7f52d00c7f70 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.308+0000 7f52d4a8c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f52d0151e50 0x7f52d00c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.308+0000 7f52d4a8c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f52d0151e50 0x7f52d00c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36420/0 (socket says 172.21.15.29:36420) 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.308+0000 7f52d528d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f52d0064780 0x7f52d00bfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42448/0 (socket says 172.21.15.29:42448) 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.308+0000 7f52d4a8c700 1 -- 172.21.15.29:0/2051945836 learned_addr learned my addr 172.21.15.29:0/2051945836 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.308+0000 7f52cffff700 1 --2- 172.21.15.29:0/2051945836 >> v2:172.21.15.163:3300/0 conn(0x7f52d00c7f70 0x7f52d0155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.309+0000 7f52d4a8c700 1 -- 172.21.15.29:0/2051945836 >> v2:172.21.15.163:3300/0 conn(0x7f52d00c7f70 msgr2=0x7f52d0155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.309+0000 7f52d4a8c700 1 --2- 172.21.15.29:0/2051945836 >> v2:172.21.15.163:3300/0 conn(0x7f52d00c7f70 0x7f52d0155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.309+0000 7f52d4a8c700 1 -- 172.21.15.29:0/2051945836 >> v1:172.21.15.29:6789/0 conn(0x7f52d0064780 legacy=0x7f52d00bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.309+0000 7f52d4a8c700 1 -- 172.21.15.29:0/2051945836 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d00aa1f0 con 0x7f52d0151e50 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.309+0000 7f52d4a8c700 1 --2- 172.21.15.29:0/2051945836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d0151e50 0x7f52d00c53c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f52d006a960 tx=0x7f52c00090f0).ready entity=mon.0 client_cookie=8b6959a0355322cb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.309+0000 7f52cf7fe700 1 -- 172.21.15.29:0/2051945836 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c000b520 con 0x7f52d0151e50 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.309+0000 7f52cf7fe700 1 -- 172.21.15.29:0/2051945836 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c000b680 con 0x7f52d0151e50 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.309+0000 7f52cf7fe700 1 -- 172.21.15.29:0/2051945836 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c000ba00 con 0x7f52d0151e50 2024-01-24T13:59:40.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.310+0000 7f52d6cf0700 1 -- 172.21.15.29:0/2051945836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d0151e50 msgr2=0x7f52d00c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.310+0000 7f52d6cf0700 1 --2- 172.21.15.29:0/2051945836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d0151e50 0x7f52d00c53c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f52d006a960 tx=0x7f52c00090f0).stop 2024-01-24T13:59:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.310+0000 7f52d6cf0700 1 -- 172.21.15.29:0/2051945836 shutdown_connections 2024-01-24T13:59:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.310+0000 7f52d6cf0700 1 --2- 172.21.15.29:0/2051945836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d0151e50 0x7f52d00c53c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.310+0000 7f52d6cf0700 1 --2- 172.21.15.29:0/2051945836 >> v2:172.21.15.163:3300/0 conn(0x7f52d00c7f70 0x7f52d0155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.310+0000 7f52d6cf0700 1 -- 172.21.15.29:0/2051945836 >> 172.21.15.29:0/2051945836 conn(0x7f52d00cb880 msgr2=0x7f52d00c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.310+0000 7f52d6cf0700 1 -- 172.21.15.29:0/2051945836 shutdown_connections 2024-01-24T13:59:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.310+0000 7f52d6cf0700 1 -- 172.21.15.29:0/2051945836 wait complete. 2024-01-24T13:59:40.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.312+0000 7f52d6cf0700 1 Processor -- start 2024-01-24T13:59:40.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.312+0000 7f52d6cf0700 1 -- start start 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.313+0000 7f52d6cf0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52d0064780 0x7f52d0149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.313+0000 7f52d6cf0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52d00c7f70 0x7f52d00a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.313+0000 7f52d6cf0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d0151e50 0x7f52d00af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.313+0000 7f52d6cf0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d01494f0 con 0x7f52d0151e50 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.313+0000 7f52d6cf0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d00c2d50 con 0x7f52d00c7f70 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.313+0000 7f52d6cf0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d00c1e60 con 0x7f52d0064780 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.313+0000 7f52d528d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d0151e50 0x7f52d00af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.313+0000 7f52d528d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d0151e50 0x7f52d00af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:36428/0 (socket says 172.21.15.29:36428) 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.313+0000 7f52cffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52d00c7f70 0x7f52d00a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.313+0000 7f52d528d700 1 -- 172.21.15.29:0/8385673 learned_addr learned my addr 172.21.15.29:0/8385673 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.313+0000 7f52d4a8c700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52d0064780 0x7f52d0149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.314+0000 7f52d528d700 1 -- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52d0064780 msgr2=0x7f52d0149000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:40.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.314+0000 7f52d528d700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52d0064780 0x7f52d0149000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:40.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.314+0000 7f52d528d700 1 -- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52d00c7f70 msgr2=0x7f52d00a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:40.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.314+0000 7f52d528d700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52d00c7f70 0x7f52d00a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:40.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.314+0000 7f52d528d700 1 -- 172.21.15.29:0/8385673 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c0009370 con 0x7f52d0151e50 2024-01-24T13:59:40.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.314+0000 7f52cffff700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52d00c7f70 0x7f52d00a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:59:40.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.314+0000 7f52d528d700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d0151e50 0x7f52d00af490 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f52c800c790 tx=0x7f52c80015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:40.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.314+0000 7f52ccff9700 1 -- 172.21.15.29:0/8385673 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8005c70 con 0x7f52d0151e50 2024-01-24T13:59:40.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.314+0000 7f52ccff9700 1 -- 172.21.15.29:0/8385673 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c8005dd0 con 0x7f52d0151e50 2024-01-24T13:59:40.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.314+0000 7f52d6cf0700 1 -- 172.21.15.29:0/8385673 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d00c2140 con 0x7f52d0151e50 2024-01-24T13:59:40.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.314+0000 7f52ccff9700 1 -- 172.21.15.29:0/8385673 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8007320 con 0x7f52d0151e50 2024-01-24T13:59:40.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.314+0000 7f52d6cf0700 1 -- 172.21.15.29:0/8385673 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d00c2700 con 0x7f52d0151e50 2024-01-24T13:59:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.315+0000 7f52d6cf0700 1 -- 172.21.15.29:0/8385673 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d00aa710 con 0x7f52d0151e50 2024-01-24T13:59:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.316+0000 7f52ccff9700 1 -- 172.21.15.29:0/8385673 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f52c80076d0 con 0x7f52d0151e50 2024-01-24T13:59:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.316+0000 7f52ccff9700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 0x7f52bc061630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.316+0000 7f52d4a8c700 1 -- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 msgr2=0x7f52bc061630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T13:59:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.316+0000 7f52ccff9700 1 -- 172.21.15.29:0/8385673 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f52c807a2c0 con 0x7f52d0151e50 2024-01-24T13:59:40.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.316+0000 7f52d4a8c700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 0x7f52bc061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T13:59:40.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.321+0000 7f52ccff9700 1 -- 172.21.15.29:0/8385673 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f52c804b430 con 0x7f52d0151e50 2024-01-24T13:59:40.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.517+0000 7f52d4a8c700 1 -- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 msgr2=0x7f52bc061630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T13:59:40.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.517+0000 7f52d4a8c700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 0x7f52bc061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T13:59:40.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.532+0000 7f52d6cf0700 1 -- 172.21.15.29:0/8385673 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f52d0083310 con 0x7f52bc05f170 2024-01-24T13:59:40.767 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:40.767+0000 7fba89496680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T13:59:40.839 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:40.839+0000 7fba89496680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T13:59:40.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.918+0000 7f52d4a8c700 1 -- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 msgr2=0x7f52bc061630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T13:59:40.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:40.918+0000 7f52d4a8c700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 0x7f52bc061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-01-24T13:59:40.945 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:40.945+0000 7fee7843d680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T13:59:41.020 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:41.019+0000 7fee7843d680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T13:59:41.024 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:41.025+0000 7fba89496680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T13:59:41.232 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:41.232+0000 7fee7843d680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T13:59:41.442 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:41.442+0000 7fba89496680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T13:59:41.545 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:41.545+0000 7fba89496680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T13:59:41.696 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:41.697+0000 7fba89496680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T13:59:41.709 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:41.709+0000 7fee7843d680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T13:59:41.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:41.719+0000 7f52d4a8c700 1 -- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 msgr2=0x7f52bc061630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T13:59:41.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:41.719+0000 7f52d4a8c700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 0x7f52bc061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-01-24T13:59:41.771 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:41.771+0000 7fba89496680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T13:59:41.826 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:41.826+0000 7fee7843d680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T13:59:41.838 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:41.838+0000 7fba89496680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T13:59:42.004 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:42.003+0000 7fee7843d680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T13:59:42.088 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:42.088+0000 7fee7843d680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T13:59:42.096 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:42.096+0000 7fba89496680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T13:59:42.164 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:42.164+0000 7fee7843d680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T13:59:42.281 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:42.281+0000 7fba89496680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T13:59:42.458 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:42.458+0000 7fee7843d680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T13:59:42.668 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:42.668+0000 7fee7843d680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T13:59:42.933 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:42.934+0000 7fba89496680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T13:59:43.001 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.002+0000 7fba89496680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T13:59:43.076 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.076+0000 7fba89496680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T13:59:43.221 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.221+0000 7fba89496680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T13:59:43.290 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.291+0000 7fba89496680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T13:59:43.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:43.321+0000 7f52d4a8c700 1 -- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 msgr2=0x7f52bc061630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T13:59:43.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:43.321+0000 7f52d4a8c700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 0x7f52bc061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-01-24T13:59:43.403 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.403+0000 7fba89496680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T13:59:43.418 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:43.417+0000 7fee7843d680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T13:59:43.495 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:43.494+0000 7fee7843d680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T13:59:43.510 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.510+0000 7fba89496680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T13:59:43.579 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:43.578+0000 7fee7843d680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T13:59:43.743 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:43.743+0000 7fee7843d680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T13:59:43.800 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.801+0000 7fba89496680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T13:59:43.822 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:43.822+0000 7fee7843d680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T13:59:43.870 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.870+0000 7fba89496680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T13:59:43.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:43.890+0000 7f52ccff9700 1 -- 172.21.15.29:0/8385673 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 6) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f52c801adb0 con 0x7f52d0151e50 2024-01-24T13:59:43.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:43.891+0000 7f52ccff9700 1 -- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 msgr2=0x7f52bc061630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T13:59:43.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:43.891+0000 7f52ccff9700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 0x7f52bc061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T13:59:43.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.958+0000 7fb9fba09700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T13:59:43.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.958+0000 7fb9fba09700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T13:59:43.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.958+0000 7fb9fba09700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T13:59:43.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.958+0000 7fb9fba09700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T13:59:43.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.958+0000 7fb9fba09700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T13:59:43.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.958+0000 7fba00212700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T13:59:43.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.958+0000 7fba00212700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T13:59:43.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.958+0000 7fba00212700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T13:59:43.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.958+0000 7fba00212700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T13:59:43.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T13:59:43.958+0000 7fba00212700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T13:59:43.959 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:43.958+0000 7fee7843d680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T13:59:44.082 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:44.082+0000 7fee7843d680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T13:59:44.416 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:44.415+0000 7fee7843d680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T13:59:44.495 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T13:59:44.495+0000 7fee7843d680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T13:59:44.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.893+0000 7f52ccff9700 1 -- 172.21.15.29:0/8385673 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f52c80782b0 con 0x7f52d0151e50 2024-01-24T13:59:44.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.893+0000 7f52ccff9700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 0x7f52bc0615b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:44.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.893+0000 7f52ccff9700 1 -- 172.21.15.29:0/8385673 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f52d0083310 con 0x7f52bc05f170 2024-01-24T13:59:44.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.895+0000 7f52d4a8c700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 0x7f52bc0615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:44.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.896+0000 7f52d4a8c700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 0x7f52bc0615b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f52c0009130 tx=0x7f52c0017000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:44.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.916+0000 7f52ccff9700 1 -- 172.21.15.29:0/8385673 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f52d0083310 con 0x7f52bc05f170 2024-01-24T13:59:44.917 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /snap_test_dir1/dir0 2024-01-24T13:59:44.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.919+0000 7f52d6cf0700 1 -- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 msgr2=0x7f52bc0615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:44.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.919+0000 7f52d6cf0700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 0x7f52bc0615b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f52c0009130 tx=0x7f52c0017000).stop 2024-01-24T13:59:44.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.919+0000 7f52d6cf0700 1 -- 172.21.15.29:0/8385673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d0151e50 msgr2=0x7f52d00af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:44.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.919+0000 7f52d6cf0700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d0151e50 0x7f52d00af490 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f52c800c790 tx=0x7f52c80015e0).stop 2024-01-24T13:59:44.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.919+0000 7f52d6cf0700 1 -- 172.21.15.29:0/8385673 shutdown_connections 2024-01-24T13:59:44.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.919+0000 7f52d6cf0700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f52bc05f170 0x7f52bc0615b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:44.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.919+0000 7f52d6cf0700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f52d0151e50 0x7f52d00af490 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:44.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.919+0000 7f52d6cf0700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52d0064780 0x7f52d0149000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:44.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.919+0000 7f52d6cf0700 1 --2- 172.21.15.29:0/8385673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52d00c7f70 0x7f52d00a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:44.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.919+0000 7f52d6cf0700 1 -- 172.21.15.29:0/8385673 >> 172.21.15.29:0/8385673 conn(0x7f52d00cb880 msgr2=0x7f52d0152620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:44.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.919+0000 7f52d6cf0700 1 -- 172.21.15.29:0/8385673 shutdown_connections 2024-01-24T13:59:44.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:44.919+0000 7f52d6cf0700 1 -- 172.21.15.29:0/8385673 wait complete. 2024-01-24T13:59:44.935 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2024-01-24T13:59:44.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:44.936 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2024-01-24T13:59:44.957 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir1 1M 2024-01-24T13:59:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.142+0000 7f1073844700 1 Processor -- start 2024-01-24T13:59:45.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.142+0000 7f1073844700 1 -- start start 2024-01-24T13:59:45.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.143+0000 7f1073844700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f106c0c3660 0x7f106c14b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:45.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.143+0000 7f1073844700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f106c0c4b00 0x7f106c154a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:45.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.143+0000 7f1073844700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106c1557e0 con 0x7f106c14b0a0 2024-01-24T13:59:45.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.143+0000 7f1073844700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106c0c28a0 con 0x7f106c0c4b00 2024-01-24T13:59:45.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.143+0000 7f1073844700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0c2b80 con 0x7f106c0c3660 2024-01-24T13:59:45.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.143+0000 7f1071de1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f106c0c4b00 0x7f106c154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:45.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.143+0000 7f1071de1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f106c0c4b00 0x7f106c154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51004/0 (socket says 172.21.15.29:51004) 2024-01-24T13:59:45.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.143+0000 7f1071de1700 1 -- 172.21.15.29:0/2751116551 learned_addr learned my addr 172.21.15.29:0/2751116551 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:45.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.143+0000 7f1070ddf700 1 --2- 172.21.15.29:0/2751116551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f106c0c3660 0x7f106c14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:45.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.143+0000 7f1071de1700 1 -- 172.21.15.29:0/2751116551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f106c0c3660 msgr2=0x7f106c14b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:45.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.143+0000 7f1071de1700 1 --2- 172.21.15.29:0/2751116551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f106c0c3660 0x7f106c14b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.143+0000 7f1071de1700 1 -- 172.21.15.29:0/2751116551 >> v1:172.21.15.163:6789/0 conn(0x7f106c14b0a0 legacy=0x7f106c0c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:45.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.144+0000 7f1071de1700 1 -- 172.21.15.29:0/2751116551 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c0c2e60 con 0x7f106c0c4b00 2024-01-24T13:59:45.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.144+0000 7f1071de1700 1 --2- 172.21.15.29:0/2751116551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f106c0c4b00 0x7f106c154a70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f106400d1a0 tx=0x7f106400ba50).ready entity=mon.0 client_cookie=eae0557f309c76c5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:45.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.144+0000 7f1063fff700 1 -- 172.21.15.29:0/2751116551 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064010070 con 0x7f106c0c4b00 2024-01-24T13:59:45.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.144+0000 7f1063fff700 1 -- 172.21.15.29:0/2751116551 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106400b0d0 con 0x7f106c0c4b00 2024-01-24T13:59:45.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.144+0000 7f1063fff700 1 -- 172.21.15.29:0/2751116551 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106400b450 con 0x7f106c0c4b00 2024-01-24T13:59:45.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.144+0000 7f1073844700 1 -- 172.21.15.29:0/2751116551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f106c0c4b00 msgr2=0x7f106c154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:45.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.144+0000 7f1073844700 1 --2- 172.21.15.29:0/2751116551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f106c0c4b00 0x7f106c154a70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f106400d1a0 tx=0x7f106400ba50).stop 2024-01-24T13:59:45.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.145+0000 7f1073844700 1 -- 172.21.15.29:0/2751116551 shutdown_connections 2024-01-24T13:59:45.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.145+0000 7f1073844700 1 --2- 172.21.15.29:0/2751116551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f106c0c4b00 0x7f106c154a70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.145+0000 7f1073844700 1 --2- 172.21.15.29:0/2751116551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f106c0c3660 0x7f106c14b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.145+0000 7f1073844700 1 -- 172.21.15.29:0/2751116551 >> 172.21.15.29:0/2751116551 conn(0x7f106c067b20 msgr2=0x7f106c0638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:45.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.145+0000 7f1073844700 1 -- 172.21.15.29:0/2751116551 shutdown_connections 2024-01-24T13:59:45.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.145+0000 7f1073844700 1 -- 172.21.15.29:0/2751116551 wait complete. 2024-01-24T13:59:45.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.146+0000 7f1073844700 1 Processor -- start 2024-01-24T13:59:45.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.146+0000 7f1073844700 1 -- start start 2024-01-24T13:59:45.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f1073844700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f106c0c3660 0x7f106c0a9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:45.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f1073844700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f106c0c4b00 0x7f106c149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f1073844700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f106c14b0a0 0x7f106c14a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f1073844700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c147360 con 0x7f106c0c3660 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f1073844700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c146790 con 0x7f106c0c4b00 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f1073844700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c146a30 con 0x7f106c14b0a0 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f10715e0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f106c0c3660 0x7f106c0a9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f10715e0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f106c0c3660 0x7f106c0a9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51014/0 (socket says 172.21.15.29:51014) 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f10715e0700 1 -- 172.21.15.29:0/1819304551 learned_addr learned my addr 172.21.15.29:0/1819304551 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f1071de1700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f106c14b0a0 0x7f106c14a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f1070ddf700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f106c0c4b00 0x7f106c149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f10715e0700 1 -- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f106c14b0a0 msgr2=0x7f106c14a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f10715e0700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f106c14b0a0 0x7f106c14a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f10715e0700 1 -- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f106c0c4b00 msgr2=0x7f106c149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f10715e0700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f106c0c4b00 0x7f106c149fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f10715e0700 1 -- 172.21.15.29:0/1819304551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106400bcf0 con 0x7f106c0c3660 2024-01-24T13:59:45.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f1070ddf700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f106c0c4b00 0x7f106c149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:45.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f10715e0700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f106c0c3660 0x7f106c0a9340 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f106800c410 tx=0x7f1068007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:45.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f10617fa700 1 -- 172.21.15.29:0/1819304551 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106800f470 con 0x7f106c0c3660 2024-01-24T13:59:45.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f10617fa700 1 -- 172.21.15.29:0/1819304551 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106800f5d0 con 0x7f106c0c3660 2024-01-24T13:59:45.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f1073844700 1 -- 172.21.15.29:0/1819304551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c146d10 con 0x7f106c0c3660 2024-01-24T13:59:45.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.148+0000 7f10617fa700 1 -- 172.21.15.29:0/1819304551 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106800f970 con 0x7f106c0c3660 2024-01-24T13:59:45.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.149+0000 7f1073844700 1 -- 172.21.15.29:0/1819304551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c149570 con 0x7f106c0c3660 2024-01-24T13:59:45.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.150+0000 7f1073844700 1 -- 172.21.15.29:0/1819304551 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106c0475e0 con 0x7f106c0c3660 2024-01-24T13:59:45.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.150+0000 7f10617fa700 1 -- 172.21.15.29:0/1819304551 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f106800fd20 con 0x7f106c0c3660 2024-01-24T13:59:45.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.150+0000 7f10617fa700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f105005f080 0x7f1050061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:45.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.151+0000 7f10617fa700 1 -- 172.21.15.29:0/1819304551 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f10680797a0 con 0x7f106c0c3660 2024-01-24T13:59:45.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.151+0000 7f1070ddf700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f105005f080 0x7f1050061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:45.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.151+0000 7f1070ddf700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f105005f080 0x7f1050061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f106c065b40 tx=0x7f105c008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:45.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.155+0000 7f10617fa700 1 -- 172.21.15.29:0/1819304551 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f106804aa30 con 0x7f106c0c3660 2024-01-24T13:59:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.361+0000 7f1073844700 1 -- 172.21.15.29:0/1819304551 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f106c0c3140 con 0x7f105005f080 2024-01-24T13:59:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.367+0000 7f10617fa700 1 -- 172.21.15.29:0/1819304551 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f106c0c3140 con 0x7f105005f080 2024-01-24T13:59:45.367 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /snap_test_dir1/dir1 2024-01-24T13:59:45.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.369+0000 7f1073844700 1 -- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f105005f080 msgr2=0x7f1050061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:45.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.369+0000 7f1073844700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f105005f080 0x7f1050061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f106c065b40 tx=0x7f105c008040).stop 2024-01-24T13:59:45.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.369+0000 7f1073844700 1 -- 172.21.15.29:0/1819304551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f106c0c3660 msgr2=0x7f106c0a9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:45.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.369+0000 7f1073844700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f106c0c3660 0x7f106c0a9340 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f106800c410 tx=0x7f1068007db0).stop 2024-01-24T13:59:45.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.369+0000 7f1073844700 1 -- 172.21.15.29:0/1819304551 shutdown_connections 2024-01-24T13:59:45.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.369+0000 7f1073844700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f106c14b0a0 0x7f106c14a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.369+0000 7f1073844700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f105005f080 0x7f1050061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.369+0000 7f1073844700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f106c0c3660 0x7f106c0a9340 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.369+0000 7f1073844700 1 --2- 172.21.15.29:0/1819304551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f106c0c4b00 0x7f106c149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.369+0000 7f1073844700 1 -- 172.21.15.29:0/1819304551 >> 172.21.15.29:0/1819304551 conn(0x7f106c067b20 msgr2=0x7f106c144bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:45.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.369+0000 7f1073844700 1 -- 172.21.15.29:0/1819304551 shutdown_connections 2024-01-24T13:59:45.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.369+0000 7f1073844700 1 -- 172.21.15.29:0/1819304551 wait complete. 2024-01-24T13:59:45.382 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2024-01-24T13:59:45.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:45.383 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2024-01-24T13:59:45.403 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir2 1M 2024-01-24T13:59:45.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.582+0000 7f2853252700 1 Processor -- start 2024-01-24T13:59:45.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.582+0000 7f2853252700 1 -- start start 2024-01-24T13:59:45.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.583+0000 7f2853252700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f284c0c3d80 0x7f284c151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:45.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.583+0000 7f2853252700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f284c069260 0x7f284c0696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:45.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.583+0000 7f2853252700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284c0bd2b0 con 0x7f284c067140 2024-01-24T13:59:45.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.583+0000 7f2853252700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284c0bca50 con 0x7f284c069260 2024-01-24T13:59:45.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.583+0000 7f2853252700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284c1523a0 con 0x7f284c0c3d80 2024-01-24T13:59:45.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.583+0000 7f28517ef700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f284c069260 0x7f284c0696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:45.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.583+0000 7f28517ef700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f284c069260 0x7f284c0696d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51022/0 (socket says 172.21.15.29:51022) 2024-01-24T13:59:45.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.583+0000 7f28517ef700 1 -- 172.21.15.29:0/1532176689 learned_addr learned my addr 172.21.15.29:0/1532176689 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:45.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.583+0000 7f2850fee700 1 --2- 172.21.15.29:0/1532176689 >> v2:172.21.15.163:3300/0 conn(0x7f284c0c3d80 0x7f284c151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:45.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.584+0000 7f28517ef700 1 -- 172.21.15.29:0/1532176689 >> v2:172.21.15.163:3300/0 conn(0x7f284c0c3d80 msgr2=0x7f284c151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:45.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.584+0000 7f28517ef700 1 --2- 172.21.15.29:0/1532176689 >> v2:172.21.15.163:3300/0 conn(0x7f284c0c3d80 0x7f284c151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.584+0000 7f28517ef700 1 -- 172.21.15.29:0/1532176689 >> v1:172.21.15.163:6789/0 conn(0x7f284c067140 legacy=0x7f284c0c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:45.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.584+0000 7f28517ef700 1 -- 172.21.15.29:0/1532176689 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c069bc0 con 0x7f284c069260 2024-01-24T13:59:45.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.584+0000 7f28517ef700 1 --2- 172.21.15.29:0/1532176689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f284c069260 0x7f284c0696d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f284c069ec0 tx=0x7f2844009840).ready entity=mon.0 client_cookie=2be5d6344d0d375b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:45.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.584+0000 7f284b7fe700 1 -- 172.21.15.29:0/1532176689 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400e070 con 0x7f284c069260 2024-01-24T13:59:45.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.584+0000 7f284b7fe700 1 -- 172.21.15.29:0/1532176689 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284400ba20 con 0x7f284c069260 2024-01-24T13:59:45.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.584+0000 7f284b7fe700 1 -- 172.21.15.29:0/1532176689 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400bda0 con 0x7f284c069260 2024-01-24T13:59:45.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.584+0000 7f2853252700 1 -- 172.21.15.29:0/1532176689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f284c069260 msgr2=0x7f284c0696d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:45.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.585+0000 7f2853252700 1 --2- 172.21.15.29:0/1532176689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f284c069260 0x7f284c0696d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f284c069ec0 tx=0x7f2844009840).stop 2024-01-24T13:59:45.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.585+0000 7f2853252700 1 -- 172.21.15.29:0/1532176689 shutdown_connections 2024-01-24T13:59:45.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.585+0000 7f2853252700 1 --2- 172.21.15.29:0/1532176689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f284c069260 0x7f284c0696d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.585+0000 7f2853252700 1 --2- 172.21.15.29:0/1532176689 >> v2:172.21.15.163:3300/0 conn(0x7f284c0c3d80 0x7f284c151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.585+0000 7f2853252700 1 -- 172.21.15.29:0/1532176689 >> 172.21.15.29:0/1532176689 conn(0x7f284c0c7af0 msgr2=0x7f284c0c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:45.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.585+0000 7f2853252700 1 -- 172.21.15.29:0/1532176689 shutdown_connections 2024-01-24T13:59:45.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.585+0000 7f2853252700 1 -- 172.21.15.29:0/1532176689 wait complete. 2024-01-24T13:59:45.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.586+0000 7f2853252700 1 Processor -- start 2024-01-24T13:59:45.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.587+0000 7f2853252700 1 -- start start 2024-01-24T13:59:45.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f2853252700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f284c067140 0x7f284c0c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:45.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f2853252700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f284c069260 0x7f284c0bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:45.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f2853252700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f284c0c3d80 0x7f284c1497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:45.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f2853252700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0c0a20 con 0x7f284c069260 2024-01-24T13:59:45.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f2853252700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0c0d00 con 0x7f284c0c3d80 2024-01-24T13:59:45.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f2853252700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0c0fe0 con 0x7f284c067140 2024-01-24T13:59:45.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f284bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f284c069260 0x7f284c0bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f284bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f284c069260 0x7f284c0bfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51034/0 (socket says 172.21.15.29:51034) 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f284bfff700 1 -- 172.21.15.29:0/4141841048 learned_addr learned my addr 172.21.15.29:0/4141841048 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f28517ef700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f284c0c3d80 0x7f284c1497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f2850fee700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f284c067140 0x7f284c0c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f284bfff700 1 -- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f284c067140 msgr2=0x7f284c0c0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f284bfff700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f284c067140 0x7f284c0c0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f284bfff700 1 -- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f284c0c3d80 msgr2=0x7f284c1497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f284bfff700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f284c0c3d80 0x7f284c1497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f284bfff700 1 -- 172.21.15.29:0/4141841048 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2844009ae0 con 0x7f284c069260 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f284bfff700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f284c069260 0x7f284c0bfa10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f283400c4c0 tx=0x7f2834007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f2848ff9700 1 -- 172.21.15.29:0/4141841048 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283400ce70 con 0x7f284c069260 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f2848ff9700 1 -- 172.21.15.29:0/4141841048 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2834001e90 con 0x7f284c069260 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f2853252700 1 -- 172.21.15.29:0/4141841048 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284c0c12c0 con 0x7f284c069260 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.588+0000 7f2848ff9700 1 -- 172.21.15.29:0/4141841048 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834007250 con 0x7f284c069260 2024-01-24T13:59:45.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.589+0000 7f2853252700 1 -- 172.21.15.29:0/4141841048 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284c0bff00 con 0x7f284c069260 2024-01-24T13:59:45.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.590+0000 7f2853252700 1 -- 172.21.15.29:0/4141841048 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284c0c17e0 con 0x7f284c069260 2024-01-24T13:59:45.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.590+0000 7f2848ff9700 1 -- 172.21.15.29:0/4141841048 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f28340066a0 con 0x7f284c069260 2024-01-24T13:59:45.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.590+0000 7f2848ff9700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f282c05f080 0x7f282c061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:45.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.590+0000 7f2848ff9700 1 -- 172.21.15.29:0/4141841048 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f2834079a90 con 0x7f284c069260 2024-01-24T13:59:45.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.591+0000 7f2850fee700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f282c05f080 0x7f282c061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:45.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.591+0000 7f2850fee700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f282c05f080 0x7f282c061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f283c009b90 tx=0x7f283c009060).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:45.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.595+0000 7f2848ff9700 1 -- 172.21.15.29:0/4141841048 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f283404ad20 con 0x7f284c069260 2024-01-24T13:59:45.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.804+0000 7f2853252700 1 -- 172.21.15.29:0/4141841048 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f284c0475e0 con 0x7f282c05f080 2024-01-24T13:59:45.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.809+0000 7f2848ff9700 1 -- 172.21.15.29:0/4141841048 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f284c0475e0 con 0x7f282c05f080 2024-01-24T13:59:45.809 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /snap_test_dir1/dir2 2024-01-24T13:59:45.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.811+0000 7f2853252700 1 -- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f282c05f080 msgr2=0x7f282c061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:45.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.811+0000 7f2853252700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f282c05f080 0x7f282c061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f283c009b90 tx=0x7f283c009060).stop 2024-01-24T13:59:45.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.811+0000 7f2853252700 1 -- 172.21.15.29:0/4141841048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f284c069260 msgr2=0x7f284c0bfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:45.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.811+0000 7f2853252700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f284c069260 0x7f284c0bfa10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f283400c4c0 tx=0x7f2834007e00).stop 2024-01-24T13:59:45.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.811+0000 7f2853252700 1 -- 172.21.15.29:0/4141841048 shutdown_connections 2024-01-24T13:59:45.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.811+0000 7f2853252700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f284c0c3d80 0x7f284c1497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.811+0000 7f2853252700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f284c067140 0x7f284c0c0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.811+0000 7f2853252700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f282c05f080 0x7f282c061540 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.811+0000 7f2853252700 1 --2- 172.21.15.29:0/4141841048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f284c069260 0x7f284c0bfa10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:45.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.811+0000 7f2853252700 1 -- 172.21.15.29:0/4141841048 >> 172.21.15.29:0/4141841048 conn(0x7f284c0c7af0 msgr2=0x7f284c0bdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:45.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.811+0000 7f2853252700 1 -- 172.21.15.29:0/4141841048 shutdown_connections 2024-01-24T13:59:45.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:45.811+0000 7f2853252700 1 -- 172.21.15.29:0/4141841048 wait complete. 2024-01-24T13:59:45.825 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2024-01-24T13:59:45.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:45.825 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2024-01-24T13:59:45.845 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir3 1M 2024-01-24T13:59:46.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.023+0000 7f4592782700 1 Processor -- start 2024-01-24T13:59:46.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.023+0000 7f4592782700 1 -- start start 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f4592782700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c0c3d80 0x7f458c151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f4592782700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f458c067140 0x7f458c0c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f4592782700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c0bd2b0 con 0x7f458c069260 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f4592782700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c0bca50 con 0x7f458c067140 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f4592782700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c1523a0 con 0x7f458c0c3d80 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f458b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f458c067140 0x7f458c0c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f4590d1f700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f458c069260 0x7f458c0bd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33246/0 (socket says 172.21.15.29:33246) 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f4590d1f700 1 -- 172.21.15.29:0/3792195574 learned_addr learned my addr 172.21.15.29:0/3792195574 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f458bfff700 1 --2- 172.21.15.29:0/3792195574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c0c3d80 0x7f458c151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f458b7fe700 1 -- 172.21.15.29:0/3792195574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c0c3d80 msgr2=0x7f458c151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f458b7fe700 1 --2- 172.21.15.29:0/3792195574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c0c3d80 0x7f458c151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f458b7fe700 1 -- 172.21.15.29:0/3792195574 >> v1:172.21.15.29:6789/0 conn(0x7f458c069260 legacy=0x7f458c0bd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f458affd700 1 -- 172.21.15.29:0/3792195574 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491715384 0 0) 0x7f458c0bd2b0 con 0x7f458c069260 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f458b7fe700 1 -- 172.21.15.29:0/3792195574 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c0678c0 con 0x7f458c067140 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f458bfff700 1 --2- 172.21.15.29:0/3792195574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c0c3d80 0x7f458c151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.024+0000 7f458b7fe700 1 --2- 172.21.15.29:0/3792195574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f458c067140 0x7f458c0c20f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f4580009760 tx=0x7f4580009060).ready entity=mon.0 client_cookie=49d53afb74d31188 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.025+0000 7f458affd700 1 -- 172.21.15.29:0/3792195574 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458000b580 con 0x7f458c067140 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.025+0000 7f458affd700 1 -- 172.21.15.29:0/3792195574 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458000b6e0 con 0x7f458c067140 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.025+0000 7f458affd700 1 -- 172.21.15.29:0/3792195574 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458000ba60 con 0x7f458c067140 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.025+0000 7f4592782700 1 -- 172.21.15.29:0/3792195574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f458c067140 msgr2=0x7f458c0c20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.025+0000 7f4592782700 1 --2- 172.21.15.29:0/3792195574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f458c067140 0x7f458c0c20f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f4580009760 tx=0x7f4580009060).stop 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.025+0000 7f4592782700 1 -- 172.21.15.29:0/3792195574 shutdown_connections 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.025+0000 7f4592782700 1 --2- 172.21.15.29:0/3792195574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c0c3d80 0x7f458c151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.025+0000 7f4592782700 1 --2- 172.21.15.29:0/3792195574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f458c067140 0x7f458c0c20f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.025+0000 7f4592782700 1 -- 172.21.15.29:0/3792195574 >> 172.21.15.29:0/3792195574 conn(0x7f458c0c7af0 msgr2=0x7f458c0c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:46.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.025+0000 7f4592782700 1 -- 172.21.15.29:0/3792195574 shutdown_connections 2024-01-24T13:59:46.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.025+0000 7f4592782700 1 -- 172.21.15.29:0/3792195574 wait complete. 2024-01-24T13:59:46.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.026+0000 7f4592782700 1 Processor -- start 2024-01-24T13:59:46.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.027+0000 7f4592782700 1 -- start start 2024-01-24T13:59:46.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4592782700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458c067140 0x7f458c0c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4592782700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f458c069260 0x7f458c14cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4592782700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c0c3d80 0x7f458c149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4592782700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c14e580 con 0x7f458c069260 2024-01-24T13:59:46.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4592782700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c14e0f0 con 0x7f458c067140 2024-01-24T13:59:46.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4592782700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c14d4e0 con 0x7f458c0c3d80 2024-01-24T13:59:46.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f458b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f458c069260 0x7f458c14cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f458b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f458c069260 0x7f458c14cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51048/0 (socket says 172.21.15.29:51048) 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f458b7fe700 1 -- 172.21.15.29:0/3117255281 learned_addr learned my addr 172.21.15.29:0/3117255281 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4590d1f700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c0c3d80 0x7f458c149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f458bfff700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458c067140 0x7f458c0c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f458b7fe700 1 -- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c0c3d80 msgr2=0x7f458c149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f458b7fe700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c0c3d80 0x7f458c149cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f458b7fe700 1 -- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458c067140 msgr2=0x7f458c0c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f458b7fe700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458c067140 0x7f458c0c6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f458b7fe700 1 -- 172.21.15.29:0/3117255281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45800092a0 con 0x7f458c069260 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4590d1f700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c0c3d80 0x7f458c149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f458b7fe700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f458c069260 0x7f458c14cc90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f4580009eb0 tx=0x7f4580017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4573fff700 1 -- 172.21.15.29:0/3117255281 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458000b850 con 0x7f458c069260 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4573fff700 1 -- 172.21.15.29:0/3117255281 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458000b9b0 con 0x7f458c069260 2024-01-24T13:59:46.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4592782700 1 -- 172.21.15.29:0/3117255281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c14d7c0 con 0x7f458c069260 2024-01-24T13:59:46.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4573fff700 1 -- 172.21.15.29:0/3117255281 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458001b690 con 0x7f458c069260 2024-01-24T13:59:46.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.028+0000 7f4592782700 1 -- 172.21.15.29:0/3117255281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c14dd80 con 0x7f458c069260 2024-01-24T13:59:46.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.029+0000 7f4592782700 1 -- 172.21.15.29:0/3117255281 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c0475e0 con 0x7f458c069260 2024-01-24T13:59:46.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.030+0000 7f4573fff700 1 -- 172.21.15.29:0/3117255281 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f458001ba40 con 0x7f458c069260 2024-01-24T13:59:46.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.030+0000 7f4573fff700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f457805f0b0 0x7f4578061570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.030+0000 7f4573fff700 1 -- 172.21.15.29:0/3117255281 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f458007b9e0 con 0x7f458c069260 2024-01-24T13:59:46.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.030+0000 7f458bfff700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f457805f0b0 0x7f4578061570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.031+0000 7f458bfff700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f457805f0b0 0x7f4578061570 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f457c006fd0 tx=0x7f457c008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:46.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.034+0000 7f4573fff700 1 -- 172.21.15.29:0/3117255281 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f458004cc70 con 0x7f458c069260 2024-01-24T13:59:46.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.243+0000 7f4592782700 1 -- 172.21.15.29:0/3117255281 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f458c0bcc60 con 0x7f457805f0b0 2024-01-24T13:59:46.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.247+0000 7f4573fff700 1 -- 172.21.15.29:0/3117255281 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f458c0bcc60 con 0x7f457805f0b0 2024-01-24T13:59:46.248 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /snap_test_dir1/dir3 2024-01-24T13:59:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.250+0000 7f4592782700 1 -- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f457805f0b0 msgr2=0x7f4578061570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.250+0000 7f4592782700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f457805f0b0 0x7f4578061570 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f457c006fd0 tx=0x7f457c008040).stop 2024-01-24T13:59:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.250+0000 7f4592782700 1 -- 172.21.15.29:0/3117255281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f458c069260 msgr2=0x7f458c14cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.250+0000 7f4592782700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f458c069260 0x7f458c14cc90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f4580009eb0 tx=0x7f4580017000).stop 2024-01-24T13:59:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.250+0000 7f4592782700 1 -- 172.21.15.29:0/3117255281 shutdown_connections 2024-01-24T13:59:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.250+0000 7f4592782700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c0c3d80 0x7f458c149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.250+0000 7f4592782700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458c067140 0x7f458c0c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.250+0000 7f4592782700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f457805f0b0 0x7f4578061570 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.250+0000 7f4592782700 1 --2- 172.21.15.29:0/3117255281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f458c069260 0x7f458c14cc90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.250+0000 7f4592782700 1 -- 172.21.15.29:0/3117255281 >> 172.21.15.29:0/3117255281 conn(0x7f458c0c7af0 msgr2=0x7f458c0c4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.250+0000 7f4592782700 1 -- 172.21.15.29:0/3117255281 shutdown_connections 2024-01-24T13:59:46.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.250+0000 7f4592782700 1 -- 172.21.15.29:0/3117255281 wait complete. 2024-01-24T13:59:46.264 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2024-01-24T13:59:46.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:46.264 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2024-01-24T13:59:46.284 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir4 1M 2024-01-24T13:59:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.463+0000 7fcadc6ce700 1 Processor -- start 2024-01-24T13:59:46.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.463+0000 7fcadc6ce700 1 -- start start 2024-01-24T13:59:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.464+0000 7fcadc6ce700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad4062970 0x7fcad4062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.464+0000 7fcadc6ce700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad40c3e00 con 0x7fcad40c2190 2024-01-24T13:59:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.464+0000 7fcadc6ce700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad40c2900 con 0x7fcad4158720 2024-01-24T13:59:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.464+0000 7fcadc6ce700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad4065720 con 0x7fcad4062970 2024-01-24T13:59:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.464+0000 7fcad9c69700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcad40c2190 0x7fcad40c4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33248/0 (socket says 172.21.15.29:33248) 2024-01-24T13:59:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.464+0000 7fcad9c69700 1 -- 172.21.15.29:0/269728027 learned_addr learned my addr 172.21.15.29:0/269728027 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.464+0000 7fcada46a700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad4062970 0x7fcad4062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.464+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630275284 0 0) 0x7fcad40c3e00 con 0x7fcad40c2190 2024-01-24T13:59:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.464+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcac00033f0 con 0x7fcad40c2190 2024-01-24T13:59:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3480145515 0 0) 0x7fcac00033f0 con 0x7fcad40c2190 2024-01-24T13:59:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad4062970 msgr2=0x7fcad4062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:46.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad4062970 0x7fcad4062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 >> v1:172.21.15.163:6789/0 conn(0x7fcad4158720 legacy=0x7fcad40a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad4158bb0 con 0x7fcad40c2190 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fcad0001470 con 0x7fcad40c2190 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 >> v1:172.21.15.29:6789/0 conn(0x7fcad40c2190 legacy=0x7fcad40c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad4062970 0x7fcac0003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac0003e00 0x7fcac0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcac0004b80 0x7fcac0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad4065720 con 0x7fcac0004b80 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac000b8e0 con 0x7fcac0003e00 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac000baa0 con 0x7fcad4062970 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcada46a700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad4062970 0x7fcac0003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcad9c69700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac0003e00 0x7fcac0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcadac6b700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcac0004b80 0x7fcac0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcada46a700 1 -- 172.21.15.29:0/269728027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac0003e00 msgr2=0x7fcac0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:46.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcada46a700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac0003e00 0x7fcac0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcada46a700 1 -- 172.21.15.29:0/269728027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcac0004b80 msgr2=0x7fcac0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:46.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcada46a700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcac0004b80 0x7fcac0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcada46a700 1 -- 172.21.15.29:0/269728027 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac00033f0 con 0x7fcad4062970 2024-01-24T13:59:46.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.465+0000 7fcadac6b700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcac0004b80 0x7fcac0005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:59:46.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.466+0000 7fcada46a700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad4062970 0x7fcac0003910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fcac4009a80 tx=0x7fcac4009060).ready entity=mon.2 client_cookie=1b5fc302ea794599 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:46.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.466+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4009430 con 0x7fcad4062970 2024-01-24T13:59:46.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.466+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac400e640 con 0x7fcad4062970 2024-01-24T13:59:46.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.466+0000 7fcad9468700 1 -- 172.21.15.29:0/269728027 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac400e9c0 con 0x7fcad4062970 2024-01-24T13:59:46.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.466+0000 7fcadc6ce700 1 -- 172.21.15.29:0/269728027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad4062970 msgr2=0x7fcac0003910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.466+0000 7fcadc6ce700 1 --2- 172.21.15.29:0/269728027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad4062970 0x7fcac0003910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fcac4009a80 tx=0x7fcac4009060).stop 2024-01-24T13:59:46.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.466+0000 7fcadac6b700 1 -- 172.21.15.29:0/269728027 reap_dead start 2024-01-24T13:59:46.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.466+0000 7fcadc6ce700 1 -- 172.21.15.29:0/269728027 shutdown_connections 2024-01-24T13:59:46.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.466+0000 7fcadc6ce700 1 -- 172.21.15.29:0/269728027 >> 172.21.15.29:0/269728027 conn(0x7fcad4067b20 msgr2=0x7fcad40638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:46.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.467+0000 7fcadc6ce700 1 -- 172.21.15.29:0/269728027 shutdown_connections 2024-01-24T13:59:46.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.467+0000 7fcadc6ce700 1 -- 172.21.15.29:0/269728027 wait complete. 2024-01-24T13:59:46.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.468+0000 7fcadc6ce700 1 Processor -- start 2024-01-24T13:59:46.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.468+0000 7fcadc6ce700 1 -- start start 2024-01-24T13:59:46.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.469+0000 7fcadc6ce700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcad4062970 0x7fcad4168580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.469+0000 7fcadc6ce700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad414e740 0x7fcad40c1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.469+0000 7fcadc6ce700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcad414ebc0 0x7fcad40c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.469+0000 7fcadc6ce700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad414b2b0 con 0x7fcad4062970 2024-01-24T13:59:46.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.469+0000 7fcadc6ce700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad41680f0 con 0x7fcad414ebc0 2024-01-24T13:59:46.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.469+0000 7fcadc6ce700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad414b500 con 0x7fcad414e740 2024-01-24T13:59:46.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.469+0000 7fcada46a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcad4062970 0x7fcad4168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.469+0000 7fcada46a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcad4062970 0x7fcad4168580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51056/0 (socket says 172.21.15.29:51056) 2024-01-24T13:59:46.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.469+0000 7fcada46a700 1 -- 172.21.15.29:0/4130677146 learned_addr learned my addr 172.21.15.29:0/4130677146 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:46.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.469+0000 7fcad9c69700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad414e740 0x7fcad40c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.470+0000 7fcadac6b700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcad414ebc0 0x7fcad40c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.470+0000 7fcada46a700 1 -- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad414e740 msgr2=0x7fcad40c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.470+0000 7fcada46a700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad414e740 0x7fcad40c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.470+0000 7fcada46a700 1 -- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcad414ebc0 msgr2=0x7fcad40c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.470+0000 7fcada46a700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcad414ebc0 0x7fcad40c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.470+0000 7fcada46a700 1 -- 172.21.15.29:0/4130677146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad4167aa0 con 0x7fcad4062970 2024-01-24T13:59:46.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.470+0000 7fcada46a700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcad4062970 0x7fcad4168580 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fcac000bc30 tx=0x7fcac401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:46.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.470+0000 7fcaca7fc700 1 -- 172.21.15.29:0/4130677146 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac400ed30 con 0x7fcad4062970 2024-01-24T13:59:46.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.470+0000 7fcaca7fc700 1 -- 172.21.15.29:0/4130677146 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac400ee90 con 0x7fcad4062970 2024-01-24T13:59:46.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.470+0000 7fcadc6ce700 1 -- 172.21.15.29:0/4130677146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad4167d80 con 0x7fcad4062970 2024-01-24T13:59:46.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.470+0000 7fcaca7fc700 1 -- 172.21.15.29:0/4130677146 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac400e9c0 con 0x7fcad4062970 2024-01-24T13:59:46.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.470+0000 7fcadc6ce700 1 -- 172.21.15.29:0/4130677146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad4147620 con 0x7fcad4062970 2024-01-24T13:59:46.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.471+0000 7fcadc6ce700 1 -- 172.21.15.29:0/4130677146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad40475e0 con 0x7fcad4062970 2024-01-24T13:59:46.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.471+0000 7fcaca7fc700 1 -- 172.21.15.29:0/4130677146 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fcac401b660 con 0x7fcad4062970 2024-01-24T13:59:46.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.472+0000 7fcaca7fc700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcab805f080 0x7fcab8061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.472+0000 7fcaca7fc700 1 -- 172.21.15.29:0/4130677146 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7fcac407bbd0 con 0x7fcad4062970 2024-01-24T13:59:46.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.472+0000 7fcad9c69700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcab805f080 0x7fcab8061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.472+0000 7fcad9c69700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcab805f080 0x7fcab8061540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fcad0001050 tx=0x7fcad4063280).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:46.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.476+0000 7fcaca7fc700 1 -- 172.21.15.29:0/4130677146 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcac404ce60 con 0x7fcad4062970 2024-01-24T13:59:46.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.685+0000 7fcadc6ce700 1 -- 172.21.15.29:0/4130677146 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcad40c2620 con 0x7fcab805f080 2024-01-24T13:59:46.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.690+0000 7fcaca7fc700 1 -- 172.21.15.29:0/4130677146 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fcad40c2620 con 0x7fcab805f080 2024-01-24T13:59:46.691 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /snap_test_dir1/dir4 2024-01-24T13:59:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.692+0000 7fcadc6ce700 1 -- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcab805f080 msgr2=0x7fcab8061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.692+0000 7fcadc6ce700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcab805f080 0x7fcab8061540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fcad0001050 tx=0x7fcad4063280).stop 2024-01-24T13:59:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.692+0000 7fcadc6ce700 1 -- 172.21.15.29:0/4130677146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcad4062970 msgr2=0x7fcad4168580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.692+0000 7fcadc6ce700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcad4062970 0x7fcad4168580 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fcac000bc30 tx=0x7fcac401a000).stop 2024-01-24T13:59:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.692+0000 7fcadc6ce700 1 -- 172.21.15.29:0/4130677146 shutdown_connections 2024-01-24T13:59:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.692+0000 7fcadc6ce700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcad414ebc0 0x7fcad40c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.692+0000 7fcadc6ce700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcab805f080 0x7fcab8061540 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.692+0000 7fcadc6ce700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcad4062970 0x7fcad4168580 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.692+0000 7fcadc6ce700 1 --2- 172.21.15.29:0/4130677146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcad414e740 0x7fcad40c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.692+0000 7fcadc6ce700 1 -- 172.21.15.29:0/4130677146 >> 172.21.15.29:0/4130677146 conn(0x7fcad4067b20 msgr2=0x7fcad40c7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:46.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.692+0000 7fcadc6ce700 1 -- 172.21.15.29:0/4130677146 shutdown_connections 2024-01-24T13:59:46.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.692+0000 7fcadc6ce700 1 -- 172.21.15.29:0/4130677146 wait complete. 2024-01-24T13:59:46.706 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2024-01-24T13:59:46.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:46.706 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2024-01-24T13:59:46.726 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir5 1M 2024-01-24T13:59:46.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.907+0000 7f6f74924700 1 Processor -- start 2024-01-24T13:59:46.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.907+0000 7f6f74924700 1 -- start start 2024-01-24T13:59:46.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.908+0000 7f6f74924700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6f70064e90 0x7f6f700c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.908+0000 7f6f74924700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f70154f90 con 0x7f6f700c7f70 2024-01-24T13:59:46.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.908+0000 7f6f74924700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f700a9f10 con 0x7f6f70151e50 2024-01-24T13:59:46.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.908+0000 7f6f74924700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f700aa1f0 con 0x7f6f70064e90 2024-01-24T13:59:46.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.908+0000 7f6f6dd9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6f70064e90 0x7f6f700c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.908+0000 7f6f6ed9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6f700c7f70 0x7f6f700bfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33262/0 (socket says 172.21.15.29:33262) 2024-01-24T13:59:46.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.908+0000 7f6f6ed9d700 1 -- 172.21.15.29:0/3674675504 learned_addr learned my addr 172.21.15.29:0/3674675504 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:46.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.908+0000 7f6f6dd9b700 1 -- 172.21.15.29:0/3674675504 >> v1:172.21.15.163:6789/0 conn(0x7f6f70151e50 legacy=0x7f6f70064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:46.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.908+0000 7f6f6dd9b700 1 -- 172.21.15.29:0/3674675504 >> v1:172.21.15.29:6789/0 conn(0x7f6f700c7f70 legacy=0x7f6f700bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.908+0000 7f6f6dd9b700 1 -- 172.21.15.29:0/3674675504 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f700aa4d0 con 0x7f6f70064e90 2024-01-24T13:59:46.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.908+0000 7f6f6dd9b700 1 --2- 172.21.15.29:0/3674675504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f70064e90 0x7f6f700c53c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f6f64009760 tx=0x7f6f64009060).ready entity=mon.0 client_cookie=e40fd3f60610321 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:46.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.909+0000 7f6f6d59a700 1 -- 172.21.15.29:0/3674675504 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400b520 con 0x7f6f70064e90 2024-01-24T13:59:46.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.909+0000 7f6f6d59a700 1 -- 172.21.15.29:0/3674675504 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6400b680 con 0x7f6f70064e90 2024-01-24T13:59:46.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.909+0000 7f6f6d59a700 1 -- 172.21.15.29:0/3674675504 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400ba00 con 0x7f6f70064e90 2024-01-24T13:59:46.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.909+0000 7f6f74924700 1 -- 172.21.15.29:0/3674675504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f70064e90 msgr2=0x7f6f700c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.909+0000 7f6f74924700 1 --2- 172.21.15.29:0/3674675504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f70064e90 0x7f6f700c53c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f6f64009760 tx=0x7f6f64009060).stop 2024-01-24T13:59:46.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.909+0000 7f6f74924700 1 -- 172.21.15.29:0/3674675504 shutdown_connections 2024-01-24T13:59:46.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.909+0000 7f6f74924700 1 --2- 172.21.15.29:0/3674675504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f70064e90 0x7f6f700c53c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.909+0000 7f6f74924700 1 -- 172.21.15.29:0/3674675504 >> 172.21.15.29:0/3674675504 conn(0x7f6f700cb880 msgr2=0x7f6f700c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:46.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.909+0000 7f6f74924700 1 -- 172.21.15.29:0/3674675504 shutdown_connections 2024-01-24T13:59:46.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.909+0000 7f6f74924700 1 -- 172.21.15.29:0/3674675504 wait complete. 2024-01-24T13:59:46.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.910+0000 7f6f74924700 1 Processor -- start 2024-01-24T13:59:46.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.911+0000 7f6f74924700 1 -- start start 2024-01-24T13:59:46.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.911+0000 7f6f74924700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f70064e90 0x7f6f701537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.911+0000 7f6f74924700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f700c7f70 0x7f6f70153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.911+0000 7f6f74924700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f70151e50 0x7f6f70156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.911+0000 7f6f74924700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f700c49c0 con 0x7f6f70151e50 2024-01-24T13:59:46.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.911+0000 7f6f74924700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f701541a0 con 0x7f6f700c7f70 2024-01-24T13:59:46.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.911+0000 7f6f74924700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f700ca660 con 0x7f6f70064e90 2024-01-24T13:59:46.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.911+0000 7f6f6ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f70151e50 0x7f6f70156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f6ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f70151e50 0x7f6f70156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51076/0 (socket says 172.21.15.29:51076) 2024-01-24T13:59:46.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f6ed9d700 1 -- 172.21.15.29:0/2447475995 learned_addr learned my addr 172.21.15.29:0/2447475995 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:46.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f6e59c700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f70064e90 0x7f6f701537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f6dd9b700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f700c7f70 0x7f6f70153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f6ed9d700 1 -- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f70064e90 msgr2=0x7f6f701537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f6ed9d700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f70064e90 0x7f6f701537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f6ed9d700 1 -- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f700c7f70 msgr2=0x7f6f70153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f6ed9d700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f700c7f70 0x7f6f70153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f6ed9d700 1 -- 172.21.15.29:0/2447475995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f640092a0 con 0x7f6f70151e50 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f6dd9b700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f700c7f70 0x7f6f70153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f6ed9d700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f70151e50 0x7f6f70156c40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f6f6800c790 tx=0x7f6f680015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f5e7fc700 1 -- 172.21.15.29:0/2447475995 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68005d30 con 0x7f6f70151e50 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f5e7fc700 1 -- 172.21.15.29:0/2447475995 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f68005e90 con 0x7f6f70151e50 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f74924700 1 -- 172.21.15.29:0/2447475995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7014bcd0 con 0x7f6f70151e50 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f5e7fc700 1 -- 172.21.15.29:0/2447475995 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68007320 con 0x7f6f70151e50 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.912+0000 7f6f74924700 1 -- 172.21.15.29:0/2447475995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f700c4300 con 0x7f6f70151e50 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.913+0000 7f6f74924700 1 -- 172.21.15.29:0/2447475995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f700aa710 con 0x7f6f70151e50 2024-01-24T13:59:46.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.914+0000 7f6f5e7fc700 1 -- 172.21.15.29:0/2447475995 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f6f68007690 con 0x7f6f70151e50 2024-01-24T13:59:46.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.914+0000 7f6f5e7fc700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6f5005f350 0x7f6f50061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:46.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.914+0000 7f6f5e7fc700 1 -- 172.21.15.29:0/2447475995 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f6f6807a6e0 con 0x7f6f70151e50 2024-01-24T13:59:46.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.914+0000 7f6f6e59c700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6f5005f350 0x7f6f50061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:46.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.915+0000 7f6f6e59c700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6f5005f350 0x7f6f50061810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f6f6000b7f0 tx=0x7f6f60008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:46.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:46.919+0000 7f6f5e7fc700 1 -- 172.21.15.29:0/2447475995 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f6f6804b5a0 con 0x7f6f70151e50 2024-01-24T13:59:47.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.125+0000 7f6f74924700 1 -- 172.21.15.29:0/2447475995 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f700c4ce0 con 0x7f6f5005f350 2024-01-24T13:59:47.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.130+0000 7f6f5e7fc700 1 -- 172.21.15.29:0/2447475995 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6f700c4ce0 con 0x7f6f5005f350 2024-01-24T13:59:47.130 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /snap_test_dir1/dir5 2024-01-24T13:59:47.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.132+0000 7f6f74924700 1 -- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6f5005f350 msgr2=0x7f6f50061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:47.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.132+0000 7f6f74924700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6f5005f350 0x7f6f50061810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f6f6000b7f0 tx=0x7f6f60008040).stop 2024-01-24T13:59:47.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.132+0000 7f6f74924700 1 -- 172.21.15.29:0/2447475995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f70151e50 msgr2=0x7f6f70156c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:47.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.132+0000 7f6f74924700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f70151e50 0x7f6f70156c40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f6f6800c790 tx=0x7f6f680015e0).stop 2024-01-24T13:59:47.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.132+0000 7f6f74924700 1 -- 172.21.15.29:0/2447475995 shutdown_connections 2024-01-24T13:59:47.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.132+0000 7f6f74924700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6f5005f350 0x7f6f50061810 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.132+0000 7f6f74924700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f70151e50 0x7f6f70156c40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.132+0000 7f6f74924700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f70064e90 0x7f6f701537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.132+0000 7f6f74924700 1 --2- 172.21.15.29:0/2447475995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f700c7f70 0x7f6f70153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.132+0000 7f6f74924700 1 -- 172.21.15.29:0/2447475995 >> 172.21.15.29:0/2447475995 conn(0x7f6f700cb880 msgr2=0x7f6f700ab1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:47.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.132+0000 7f6f74924700 1 -- 172.21.15.29:0/2447475995 shutdown_connections 2024-01-24T13:59:47.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.132+0000 7f6f74924700 1 -- 172.21.15.29:0/2447475995 wait complete. 2024-01-24T13:59:47.146 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2024-01-24T13:59:47.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:47.146 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2024-01-24T13:59:47.167 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir6 1M 2024-01-24T13:59:47.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.346+0000 7f816638b700 1 Processor -- start 2024-01-24T13:59:47.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.346+0000 7f816638b700 1 -- start start 2024-01-24T13:59:47.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.347+0000 7f816638b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f81601552a0 0x7f8160066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:47.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.347+0000 7f816638b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8160068540 con 0x7f8160154ec0 2024-01-24T13:59:47.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.347+0000 7f816638b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8160066dd0 con 0x7f8160066940 2024-01-24T13:59:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.347+0000 7f816638b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81600641c0 con 0x7f81601552a0 2024-01-24T13:59:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.347+0000 7f815f7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f81601552a0 0x7f8160066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.347+0000 7f815ffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8160154ec0 0x7f81600638c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33272/0 (socket says 172.21.15.29:33272) 2024-01-24T13:59:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.347+0000 7f815ffff700 1 -- 172.21.15.29:0/2220757297 learned_addr learned my addr 172.21.15.29:0/2220757297 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.348+0000 7f815f7fe700 1 -- 172.21.15.29:0/2220757297 >> v1:172.21.15.163:6789/0 conn(0x7f8160066940 legacy=0x7f8160151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.348+0000 7f815f7fe700 1 -- 172.21.15.29:0/2220757297 >> v1:172.21.15.29:6789/0 conn(0x7f8160154ec0 legacy=0x7f81600638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.348+0000 7f815effd700 1 -- 172.21.15.29:0/2220757297 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606974287 0 0) 0x7f8160068540 con 0x7f8160154ec0 2024-01-24T13:59:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.348+0000 7f815f7fe700 1 -- 172.21.15.29:0/2220757297 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81600689d0 con 0x7f81601552a0 2024-01-24T13:59:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.348+0000 7f815f7fe700 1 --2- 172.21.15.29:0/2220757297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81601552a0 0x7f8160066450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f8154009760 tx=0x7f8154009060).ready entity=mon.0 client_cookie=718468101188014f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.348+0000 7f815effd700 1 -- 172.21.15.29:0/2220757297 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815400b580 con 0x7f81601552a0 2024-01-24T13:59:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.348+0000 7f815effd700 1 -- 172.21.15.29:0/2220757297 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815400b6e0 con 0x7f81601552a0 2024-01-24T13:59:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.348+0000 7f815effd700 1 -- 172.21.15.29:0/2220757297 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815400ba60 con 0x7f81601552a0 2024-01-24T13:59:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.349+0000 7f816638b700 1 -- 172.21.15.29:0/2220757297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81601552a0 msgr2=0x7f8160066450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.349+0000 7f816638b700 1 --2- 172.21.15.29:0/2220757297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81601552a0 0x7f8160066450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f8154009760 tx=0x7f8154009060).stop 2024-01-24T13:59:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.349+0000 7f816638b700 1 -- 172.21.15.29:0/2220757297 shutdown_connections 2024-01-24T13:59:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.349+0000 7f816638b700 1 --2- 172.21.15.29:0/2220757297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81601552a0 0x7f8160066450 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.349+0000 7f816638b700 1 -- 172.21.15.29:0/2220757297 >> 172.21.15.29:0/2220757297 conn(0x7f81600c8bd0 msgr2=0x7f81600c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.349+0000 7f816638b700 1 -- 172.21.15.29:0/2220757297 shutdown_connections 2024-01-24T13:59:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.349+0000 7f816638b700 1 -- 172.21.15.29:0/2220757297 wait complete. 2024-01-24T13:59:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.350+0000 7f816638b700 1 Processor -- start 2024-01-24T13:59:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.350+0000 7f816638b700 1 -- start start 2024-01-24T13:59:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.351+0000 7f816638b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8160066940 0x7f8160139850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.351+0000 7f816638b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8160154ec0 0x7f81600c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.351+0000 7f816638b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81601552a0 0x7f8160156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.351+0000 7f816638b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81600c46c0 con 0x7f8160066940 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.351+0000 7f816638b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81600c3ab0 con 0x7f81601552a0 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.351+0000 7f816638b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81600c3d90 con 0x7f8160154ec0 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.351+0000 7f815ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8160066940 0x7f8160139850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.351+0000 7f815ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8160066940 0x7f8160139850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51082/0 (socket says 172.21.15.29:51082) 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.351+0000 7f815ffff700 1 -- 172.21.15.29:0/3446706013 learned_addr learned my addr 172.21.15.29:0/3446706013 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.351+0000 7f815f7fe700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8160154ec0 0x7f81600c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.351+0000 7f8164928700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81601552a0 0x7f8160156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.352+0000 7f815ffff700 1 -- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8160154ec0 msgr2=0x7f81600c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.352+0000 7f815ffff700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8160154ec0 0x7f81600c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.352+0000 7f815ffff700 1 -- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81601552a0 msgr2=0x7f8160156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.352+0000 7f815ffff700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81601552a0 0x7f8160156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.352+0000 7f815ffff700 1 -- 172.21.15.29:0/3446706013 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81540092a0 con 0x7f8160066940 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.352+0000 7f8164928700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81601552a0 0x7f8160156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.352+0000 7f815ffff700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8160066940 0x7f8160139850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f815000c4c0 tx=0x7f8150007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.352+0000 7f8147fff700 1 -- 172.21.15.29:0/3446706013 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81500072e0 con 0x7f8160066940 2024-01-24T13:59:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.352+0000 7f8147fff700 1 -- 172.21.15.29:0/3446706013 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8150007440 con 0x7f8160066940 2024-01-24T13:59:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.352+0000 7f816638b700 1 -- 172.21.15.29:0/3446706013 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81600c4070 con 0x7f8160066940 2024-01-24T13:59:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.352+0000 7f8147fff700 1 -- 172.21.15.29:0/3446706013 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81500057a0 con 0x7f8160066940 2024-01-24T13:59:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.352+0000 7f816638b700 1 -- 172.21.15.29:0/3446706013 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8160139d40 con 0x7f8160066940 2024-01-24T13:59:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.353+0000 7f816638b700 1 -- 172.21.15.29:0/3446706013 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81600475e0 con 0x7f8160066940 2024-01-24T13:59:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.354+0000 7f8147fff700 1 -- 172.21.15.29:0/3446706013 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f8150005b90 con 0x7f8160066940 2024-01-24T13:59:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.354+0000 7f8147fff700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f814c05f350 0x7f814c061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.354+0000 7f8147fff700 1 -- 172.21.15.29:0/3446706013 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f8150079f70 con 0x7f8160066940 2024-01-24T13:59:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.354+0000 7f815f7fe700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f814c05f350 0x7f814c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.355+0000 7f815f7fe700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f814c05f350 0x7f814c061810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f815400b340 tx=0x7f8154017000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:47.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.358+0000 7f8147fff700 1 -- 172.21.15.29:0/3446706013 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f815004ae30 con 0x7f8160066940 2024-01-24T13:59:47.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.567+0000 7f816638b700 1 -- 172.21.15.29:0/3446706013 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f81600c4800 con 0x7f814c05f350 2024-01-24T13:59:47.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.572+0000 7f8147fff700 1 -- 172.21.15.29:0/3446706013 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f8154020070 con 0x7f814c05f350 2024-01-24T13:59:47.573 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /snap_test_dir1/dir6 2024-01-24T13:59:47.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.574+0000 7f816638b700 1 -- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f814c05f350 msgr2=0x7f814c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:47.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.574+0000 7f816638b700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f814c05f350 0x7f814c061810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f815400b340 tx=0x7f8154017000).stop 2024-01-24T13:59:47.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.574+0000 7f816638b700 1 -- 172.21.15.29:0/3446706013 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8160066940 msgr2=0x7f8160139850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:47.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.574+0000 7f816638b700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8160066940 0x7f8160139850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f815000c4c0 tx=0x7f8150007c10).stop 2024-01-24T13:59:47.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.574+0000 7f816638b700 1 -- 172.21.15.29:0/3446706013 shutdown_connections 2024-01-24T13:59:47.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.574+0000 7f816638b700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81601552a0 0x7f8160156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.574+0000 7f816638b700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f814c05f350 0x7f814c061810 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.574+0000 7f816638b700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8160066940 0x7f8160139850 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.574+0000 7f816638b700 1 --2- 172.21.15.29:0/3446706013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8160154ec0 0x7f81600c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.574+0000 7f816638b700 1 -- 172.21.15.29:0/3446706013 >> 172.21.15.29:0/3446706013 conn(0x7f81600c8bd0 msgr2=0x7f81600bede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:47.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.574+0000 7f816638b700 1 -- 172.21.15.29:0/3446706013 shutdown_connections 2024-01-24T13:59:47.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.574+0000 7f816638b700 1 -- 172.21.15.29:0/3446706013 wait complete. 2024-01-24T13:59:47.588 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2024-01-24T13:59:47.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:47.589 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2024-01-24T13:59:47.609 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir7 1M 2024-01-24T13:59:47.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.789+0000 7f6882a29700 1 Processor -- start 2024-01-24T13:59:47.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.789+0000 7f6882a29700 1 -- start start 2024-01-24T13:59:47.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f6882a29700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f687c14fd10 0x7f687c0c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:47.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f6882a29700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0684c0 0x7f687c153090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f6882a29700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f687c066450 0x7f687c0668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f6882a29700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687c0c3d50 con 0x7f687c14fd10 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f6882a29700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687c1500f0 con 0x7f687c066450 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f6882a29700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c067a70 con 0x7f687c0684c0 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f687bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f687c14fd10 0x7f687c0c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f687bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f687c14fd10 0x7f687c0c31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51090/0 (socket says 172.21.15.29:51090) 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f687bfff700 1 -- 172.21.15.29:0/3867591406 learned_addr learned my addr 172.21.15.29:0/3867591406 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f687b7fe700 1 --2- 172.21.15.29:0/3867591406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0684c0 0x7f687c153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f6880fc6700 1 --2- 172.21.15.29:0/3867591406 >> v2:172.21.15.163:3300/0 conn(0x7f687c066450 0x7f687c0668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f687bfff700 1 -- 172.21.15.29:0/3867591406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0684c0 msgr2=0x7f687c153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f687bfff700 1 --2- 172.21.15.29:0/3867591406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0684c0 0x7f687c153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f687bfff700 1 -- 172.21.15.29:0/3867591406 >> v2:172.21.15.163:3300/0 conn(0x7f687c066450 msgr2=0x7f687c0668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:47.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f687bfff700 1 --2- 172.21.15.29:0/3867591406 >> v2:172.21.15.163:3300/0 conn(0x7f687c066450 0x7f687c0668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f687bfff700 1 -- 172.21.15.29:0/3867591406 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c066dd0 con 0x7f687c14fd10 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f6880fc6700 1 --2- 172.21.15.29:0/3867591406 >> v2:172.21.15.163:3300/0 conn(0x7f687c066450 0x7f687c0668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.790+0000 7f687bfff700 1 --2- 172.21.15.29:0/3867591406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c14fd10 0x7f687c0c31d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f687c068ba0 tx=0x7f686c0090f0).ready entity=mon.0 client_cookie=d708677af565479d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.791+0000 7f687affd700 1 -- 172.21.15.29:0/3867591406 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c00b520 con 0x7f687c14fd10 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.791+0000 7f687affd700 1 -- 172.21.15.29:0/3867591406 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686c00b680 con 0x7f687c14fd10 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.791+0000 7f687affd700 1 -- 172.21.15.29:0/3867591406 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c00ba00 con 0x7f687c14fd10 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.791+0000 7f6882a29700 1 -- 172.21.15.29:0/3867591406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c14fd10 msgr2=0x7f687c0c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.791+0000 7f6882a29700 1 --2- 172.21.15.29:0/3867591406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c14fd10 0x7f687c0c31d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f687c068ba0 tx=0x7f686c0090f0).stop 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.791+0000 7f6882a29700 1 -- 172.21.15.29:0/3867591406 shutdown_connections 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.791+0000 7f6882a29700 1 --2- 172.21.15.29:0/3867591406 >> v2:172.21.15.163:3300/0 conn(0x7f687c066450 0x7f687c0668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.791+0000 7f6882a29700 1 --2- 172.21.15.29:0/3867591406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c14fd10 0x7f687c0c31d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.791+0000 7f6882a29700 1 --2- 172.21.15.29:0/3867591406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0684c0 0x7f687c153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.791+0000 7f6882a29700 1 -- 172.21.15.29:0/3867591406 >> 172.21.15.29:0/3867591406 conn(0x7f687c0c8bd0 msgr2=0x7f687c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.791+0000 7f6882a29700 1 -- 172.21.15.29:0/3867591406 shutdown_connections 2024-01-24T13:59:47.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.791+0000 7f6882a29700 1 -- 172.21.15.29:0/3867591406 wait complete. 2024-01-24T13:59:47.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.792+0000 7f6882a29700 1 Processor -- start 2024-01-24T13:59:47.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.793+0000 7f6882a29700 1 -- start start 2024-01-24T13:59:47.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.793+0000 7f6882a29700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f687c066450 0x7f687c0aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:47.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.793+0000 7f6882a29700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0684c0 0x7f687c13b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:47.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.793+0000 7f6882a29700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c14fd10 0x7f687c153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.793+0000 7f6882a29700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c063a50 con 0x7f687c0684c0 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.793+0000 7f6882a29700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c168150 con 0x7f687c066450 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.793+0000 7f6882a29700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c168430 con 0x7f687c14fd10 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f687b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0684c0 0x7f687c13b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f687b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0684c0 0x7f687c13b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51104/0 (socket says 172.21.15.29:51104) 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f687b7fe700 1 -- 172.21.15.29:0/392035173 learned_addr learned my addr 172.21.15.29:0/392035173 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f6880fc6700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c14fd10 0x7f687c153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f687bfff700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f687c066450 0x7f687c0aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f687b7fe700 1 -- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c14fd10 msgr2=0x7f687c153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f687b7fe700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c14fd10 0x7f687c153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f687b7fe700 1 -- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f687c066450 msgr2=0x7f687c0aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f687b7fe700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f687c066450 0x7f687c0aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f687b7fe700 1 -- 172.21.15.29:0/392035173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c009370 con 0x7f687c0684c0 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f687bfff700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f687c066450 0x7f687c0aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:47.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f6880fc6700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c14fd10 0x7f687c153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:47.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f687b7fe700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0684c0 0x7f687c13b590 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f686400e950 tx=0x7f6864009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:47.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f6873fff700 1 -- 172.21.15.29:0/392035173 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864009530 con 0x7f687c0684c0 2024-01-24T13:59:47.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f6873fff700 1 -- 172.21.15.29:0/392035173 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686400c850 con 0x7f687c0684c0 2024-01-24T13:59:47.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f6882a29700 1 -- 172.21.15.29:0/392035173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687c168710 con 0x7f687c0684c0 2024-01-24T13:59:47.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.794+0000 7f6873fff700 1 -- 172.21.15.29:0/392035173 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686400cbd0 con 0x7f687c0684c0 2024-01-24T13:59:47.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.795+0000 7f6882a29700 1 -- 172.21.15.29:0/392035173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c0ab300 con 0x7f687c0684c0 2024-01-24T13:59:47.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.796+0000 7f6882a29700 1 -- 172.21.15.29:0/392035173 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687c06ae00 con 0x7f687c0684c0 2024-01-24T13:59:47.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.796+0000 7f6873fff700 1 -- 172.21.15.29:0/392035173 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f686401b400 con 0x7f687c0684c0 2024-01-24T13:59:47.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.796+0000 7f6873fff700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f685c05f350 0x7f685c061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:47.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.796+0000 7f6873fff700 1 -- 172.21.15.29:0/392035173 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f686407ac10 con 0x7f687c0684c0 2024-01-24T13:59:47.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.796+0000 7f687bfff700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f685c05f350 0x7f685c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:47.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.797+0000 7f687bfff700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f685c05f350 0x7f685c061810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f686c009130 tx=0x7f686c017000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:47.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:47.801+0000 7f6873fff700 1 -- 172.21.15.29:0/392035173 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f686404ba70 con 0x7f687c0684c0 2024-01-24T13:59:48.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.008+0000 7f6882a29700 1 -- 172.21.15.29:0/392035173 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f687c0475e0 con 0x7f685c05f350 2024-01-24T13:59:48.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.013+0000 7f6873fff700 1 -- 172.21.15.29:0/392035173 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f687c0475e0 con 0x7f685c05f350 2024-01-24T13:59:48.014 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /snap_test_dir1/dir7 2024-01-24T13:59:48.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.016+0000 7f6882a29700 1 -- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f685c05f350 msgr2=0x7f685c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.016+0000 7f6882a29700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f685c05f350 0x7f685c061810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f686c009130 tx=0x7f686c017000).stop 2024-01-24T13:59:48.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.016+0000 7f6882a29700 1 -- 172.21.15.29:0/392035173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0684c0 msgr2=0x7f687c13b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.016+0000 7f6882a29700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0684c0 0x7f687c13b590 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f686400e950 tx=0x7f6864009060).stop 2024-01-24T13:59:48.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.016+0000 7f6882a29700 1 -- 172.21.15.29:0/392035173 shutdown_connections 2024-01-24T13:59:48.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.016+0000 7f6882a29700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c14fd10 0x7f687c153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.016+0000 7f6882a29700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f687c066450 0x7f687c0aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.016+0000 7f6882a29700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f685c05f350 0x7f685c061810 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.016+0000 7f6882a29700 1 --2- 172.21.15.29:0/392035173 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0684c0 0x7f687c13b590 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.016+0000 7f6882a29700 1 -- 172.21.15.29:0/392035173 >> 172.21.15.29:0/392035173 conn(0x7f687c0c8bd0 msgr2=0x7f687c154720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:48.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.016+0000 7f6882a29700 1 -- 172.21.15.29:0/392035173 shutdown_connections 2024-01-24T13:59:48.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.016+0000 7f6882a29700 1 -- 172.21.15.29:0/392035173 wait complete. 2024-01-24T13:59:48.029 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2024-01-24T13:59:48.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:48.030 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2024-01-24T13:59:48.050 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir8 1M 2024-01-24T13:59:48.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.230+0000 7f37c4d66700 1 Processor -- start 2024-01-24T13:59:48.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.230+0000 7f37c4d66700 1 -- start start 2024-01-24T13:59:48.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.231+0000 7f37c4d66700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37c0062970 0x7f37c0062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:48.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.231+0000 7f37c4d66700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f37c0158720 0x7f37c00a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:48.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.231+0000 7f37c4d66700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c00c3e00 con 0x7f37c00c2190 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.231+0000 7f37c4d66700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c00c2900 con 0x7f37c0158720 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.231+0000 7f37c4d66700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c0065720 con 0x7f37c0062970 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37bed9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f37c0158720 0x7f37c00a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37bdd9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f37c00c2190 0x7f37c00c4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33282/0 (socket says 172.21.15.29:33282) 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37bdd9b700 1 -- 172.21.15.29:0/2784545395 learned_addr learned my addr 172.21.15.29:0/2784545395 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37be59c700 1 --2- 172.21.15.29:0/2784545395 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37c0062970 0x7f37c0062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37bed9d700 1 -- 172.21.15.29:0/2784545395 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37c0062970 msgr2=0x7f37c0062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37bed9d700 1 --2- 172.21.15.29:0/2784545395 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37c0062970 0x7f37c0062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37bed9d700 1 -- 172.21.15.29:0/2784545395 >> v1:172.21.15.29:6789/0 conn(0x7f37c00c2190 legacy=0x7f37c00c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37bd59a700 1 -- 172.21.15.29:0/2784545395 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297013797 0 0) 0x7f37c00c3e00 con 0x7f37c00c2190 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37bed9d700 1 -- 172.21.15.29:0/2784545395 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c0158bb0 con 0x7f37c0158720 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37be59c700 1 --2- 172.21.15.29:0/2784545395 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37c0062970 0x7f37c0062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37bed9d700 1 --2- 172.21.15.29:0/2784545395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37c0158720 0x7f37c00a4890 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f37b8009e60 tx=0x7f37b8009780).ready entity=mon.0 client_cookie=9920ea6e1b7cb80f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:48.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37bd59a700 1 -- 172.21.15.29:0/2784545395 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800e070 con 0x7f37c0158720 2024-01-24T13:59:48.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37bd59a700 1 -- 172.21.15.29:0/2784545395 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b800bcc0 con 0x7f37c0158720 2024-01-24T13:59:48.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.232+0000 7f37bd59a700 1 -- 172.21.15.29:0/2784545395 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8012620 con 0x7f37c0158720 2024-01-24T13:59:48.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.233+0000 7f37c4d66700 1 -- 172.21.15.29:0/2784545395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37c0158720 msgr2=0x7f37c00a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.233+0000 7f37c4d66700 1 --2- 172.21.15.29:0/2784545395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37c0158720 0x7f37c00a4890 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f37b8009e60 tx=0x7f37b8009780).stop 2024-01-24T13:59:48.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.233+0000 7f37c4d66700 1 -- 172.21.15.29:0/2784545395 shutdown_connections 2024-01-24T13:59:48.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.233+0000 7f37c4d66700 1 --2- 172.21.15.29:0/2784545395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37c0158720 0x7f37c00a4890 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.233+0000 7f37c4d66700 1 --2- 172.21.15.29:0/2784545395 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37c0062970 0x7f37c0062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.233+0000 7f37c4d66700 1 -- 172.21.15.29:0/2784545395 >> 172.21.15.29:0/2784545395 conn(0x7f37c0067b20 msgr2=0x7f37c00638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:48.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.233+0000 7f37c4d66700 1 -- 172.21.15.29:0/2784545395 shutdown_connections 2024-01-24T13:59:48.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.233+0000 7f37c4d66700 1 -- 172.21.15.29:0/2784545395 wait complete. 2024-01-24T13:59:48.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.234+0000 7f37c4d66700 1 Processor -- start 2024-01-24T13:59:48.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.234+0000 7f37c4d66700 1 -- start start 2024-01-24T13:59:48.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.235+0000 7f37c4d66700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37c0062970 0x7f37c013e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:48.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.235+0000 7f37c4d66700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37c0158720 0x7f37c00ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:48.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.235+0000 7f37c4d66700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37c00ad690 0x7f37c014b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:48.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.235+0000 7f37c4d66700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c00adaf0 con 0x7f37c00ad690 2024-01-24T13:59:48.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.235+0000 7f37c4d66700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c013eed0 con 0x7f37c0158720 2024-01-24T13:59:48.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.235+0000 7f37c4d66700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c014dde0 con 0x7f37c0062970 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.235+0000 7f37bed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37c00ad690 0x7f37c014b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.235+0000 7f37bed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37c00ad690 0x7f37c014b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51124/0 (socket says 172.21.15.29:51124) 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.235+0000 7f37bed9d700 1 -- 172.21.15.29:0/619349745 learned_addr learned my addr 172.21.15.29:0/619349745 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.235+0000 7f37be59c700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37c0062970 0x7f37c013e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.235+0000 7f37bdd9b700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37c0158720 0x7f37c00ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.236+0000 7f37bed9d700 1 -- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37c0062970 msgr2=0x7f37c013e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.236+0000 7f37bed9d700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37c0062970 0x7f37c013e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.236+0000 7f37bed9d700 1 -- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37c0158720 msgr2=0x7f37c00ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.236+0000 7f37bed9d700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37c0158720 0x7f37c00ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.236+0000 7f37bed9d700 1 -- 172.21.15.29:0/619349745 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b8009a20 con 0x7f37c00ad690 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.236+0000 7f37bed9d700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37c00ad690 0x7f37c014b9f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f37c0068d70 tx=0x7f37c00c2a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.236+0000 7f37b67fc700 1 -- 172.21.15.29:0/619349745 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8009440 con 0x7f37c00ad690 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.236+0000 7f37b67fc700 1 -- 172.21.15.29:0/619349745 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b801d630 con 0x7f37c00ad690 2024-01-24T13:59:48.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.236+0000 7f37c4d66700 1 -- 172.21.15.29:0/619349745 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c014e0c0 con 0x7f37c00ad690 2024-01-24T13:59:48.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.236+0000 7f37b67fc700 1 -- 172.21.15.29:0/619349745 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b801d9f0 con 0x7f37c00ad690 2024-01-24T13:59:48.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.236+0000 7f37c4d66700 1 -- 172.21.15.29:0/619349745 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c014e680 con 0x7f37c00ad690 2024-01-24T13:59:48.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.237+0000 7f37c4d66700 1 -- 172.21.15.29:0/619349745 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c00475e0 con 0x7f37c00ad690 2024-01-24T13:59:48.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.237+0000 7f37b67fc700 1 -- 172.21.15.29:0/619349745 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f37b8003040 con 0x7f37c00ad690 2024-01-24T13:59:48.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.238+0000 7f37b67fc700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f37a005f350 0x7f37a0061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:48.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.238+0000 7f37b67fc700 1 -- 172.21.15.29:0/619349745 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f37b807c600 con 0x7f37c00ad690 2024-01-24T13:59:48.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.238+0000 7f37be59c700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f37a005f350 0x7f37a0061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:48.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.238+0000 7f37be59c700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f37a005f350 0x7f37a0061810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f37b0006d80 tx=0x7f37b0008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:48.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.242+0000 7f37b67fc700 1 -- 172.21.15.29:0/619349745 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f37b804d3b0 con 0x7f37c00ad690 2024-01-24T13:59:48.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.450+0000 7f37c4d66700 1 -- 172.21.15.29:0/619349745 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f37c00c2620 con 0x7f37a005f350 2024-01-24T13:59:48.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.455+0000 7f37b67fc700 1 -- 172.21.15.29:0/619349745 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f37c00c2620 con 0x7f37a005f350 2024-01-24T13:59:48.455 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /snap_test_dir1/dir8 2024-01-24T13:59:48.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.457+0000 7f37c4d66700 1 -- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f37a005f350 msgr2=0x7f37a0061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.457+0000 7f37c4d66700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f37a005f350 0x7f37a0061810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f37b0006d80 tx=0x7f37b0008040).stop 2024-01-24T13:59:48.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.457+0000 7f37c4d66700 1 -- 172.21.15.29:0/619349745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37c00ad690 msgr2=0x7f37c014b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.457+0000 7f37c4d66700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37c00ad690 0x7f37c014b9f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f37c0068d70 tx=0x7f37c00c2a40).stop 2024-01-24T13:59:48.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.457+0000 7f37c4d66700 1 -- 172.21.15.29:0/619349745 shutdown_connections 2024-01-24T13:59:48.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.457+0000 7f37c4d66700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f37a005f350 0x7f37a0061810 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.457+0000 7f37c4d66700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37c00ad690 0x7f37c014b9f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.457+0000 7f37c4d66700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37c0062970 0x7f37c013e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.457+0000 7f37c4d66700 1 --2- 172.21.15.29:0/619349745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37c0158720 0x7f37c00ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.457+0000 7f37c4d66700 1 -- 172.21.15.29:0/619349745 >> 172.21.15.29:0/619349745 conn(0x7f37c0067b20 msgr2=0x7f37c0151cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:48.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.457+0000 7f37c4d66700 1 -- 172.21.15.29:0/619349745 shutdown_connections 2024-01-24T13:59:48.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.457+0000 7f37c4d66700 1 -- 172.21.15.29:0/619349745 wait complete. 2024-01-24T13:59:48.471 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2024-01-24T13:59:48.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T13:59:48.471 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2024-01-24T13:59:48.491 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir9 1M 2024-01-24T13:59:48.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.671+0000 7fbf65c8c700 1 Processor -- start 2024-01-24T13:59:48.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.671+0000 7fbf65c8c700 1 -- start start 2024-01-24T13:59:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.672+0000 7fbf65c8c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbf600bf870 0x7fbf60068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.672+0000 7fbf65c8c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf600643f0 con 0x7fbf60066450 2024-01-24T13:59:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.672+0000 7fbf65c8c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf600bdb30 con 0x7fbf6013a6f0 2024-01-24T13:59:48.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.672+0000 7fbf65c8c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf600be3d0 con 0x7fbf600bf870 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.672+0000 7fbf5ffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbf60066450 0x7fbf6015de30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33292/0 (socket says 172.21.15.29:33292) 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.672+0000 7fbf5ffff700 1 -- 172.21.15.29:0/1367128934 learned_addr learned my addr 172.21.15.29:0/1367128934 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.673+0000 7fbf5effd700 1 --2- 172.21.15.29:0/1367128934 >> v2:172.21.15.163:3300/0 conn(0x7fbf600bf870 0x7fbf60068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.673+0000 7fbf5e7fc700 1 -- 172.21.15.29:0/1367128934 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922892783 0 0) 0x7fbf600643f0 con 0x7fbf60066450 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.673+0000 7fbf5e7fc700 1 -- 172.21.15.29:0/1367128934 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf480033f0 con 0x7fbf60066450 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.673+0000 7fbf5effd700 1 -- 172.21.15.29:0/1367128934 >> v1:172.21.15.163:6789/0 conn(0x7fbf6013a6f0 legacy=0x7fbf600638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.673+0000 7fbf5e7fc700 1 -- 172.21.15.29:0/1367128934 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437204608 0 0) 0x7fbf600bdb30 con 0x7fbf6013a6f0 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.673+0000 7fbf5effd700 1 -- 172.21.15.29:0/1367128934 >> v1:172.21.15.29:6789/0 conn(0x7fbf60066450 legacy=0x7fbf6015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.673+0000 7fbf5effd700 1 -- 172.21.15.29:0/1367128934 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf60064110 con 0x7fbf600bf870 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.673+0000 7fbf5effd700 1 --2- 172.21.15.29:0/1367128934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf600bf870 0x7fbf60068660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fbf54009760 tx=0x7fbf54009060).ready entity=mon.1 client_cookie=397119bd7adeec8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.673+0000 7fbf5e7fc700 1 -- 172.21.15.29:0/1367128934 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400f510 con 0x7fbf600bf870 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.673+0000 7fbf5e7fc700 1 -- 172.21.15.29:0/1367128934 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5400f670 con 0x7fbf600bf870 2024-01-24T13:59:48.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.673+0000 7fbf5e7fc700 1 -- 172.21.15.29:0/1367128934 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400f9f0 con 0x7fbf600bf870 2024-01-24T13:59:48.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.674+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1367128934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf600bf870 msgr2=0x7fbf60068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.674+0000 7fbf65c8c700 1 --2- 172.21.15.29:0/1367128934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf600bf870 0x7fbf60068660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fbf54009760 tx=0x7fbf54009060).stop 2024-01-24T13:59:48.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.674+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1367128934 shutdown_connections 2024-01-24T13:59:48.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.674+0000 7fbf65c8c700 1 --2- 172.21.15.29:0/1367128934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf600bf870 0x7fbf60068660 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.674+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1367128934 >> 172.21.15.29:0/1367128934 conn(0x7fbf600c8bd0 msgr2=0x7fbf600c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:48.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.674+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1367128934 shutdown_connections 2024-01-24T13:59:48.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.674+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1367128934 wait complete. 2024-01-24T13:59:48.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.675+0000 7fbf65c8c700 1 Processor -- start 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.676+0000 7fbf65c8c700 1 -- start start 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.676+0000 7fbf65c8c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf60066450 0x7fbf601572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.676+0000 7fbf65c8c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf600bf870 0x7fbf6013c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.676+0000 7fbf65c8c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6013a6f0 0x7fbf60165b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.676+0000 7fbf65c8c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf600add30 con 0x7fbf600bf870 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.676+0000 7fbf65c8c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf601577d0 con 0x7fbf6013a6f0 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.676+0000 7fbf65c8c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf600abd00 con 0x7fbf60066450 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.676+0000 7fbf5effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf600bf870 0x7fbf6013c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.676+0000 7fbf5effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf600bf870 0x7fbf6013c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51132/0 (socket says 172.21.15.29:51132) 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.676+0000 7fbf5effd700 1 -- 172.21.15.29:0/1936207761 learned_addr learned my addr 172.21.15.29:0/1936207761 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf5ffff700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6013a6f0 0x7fbf60165b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf5f7fe700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf60066450 0x7fbf601572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:48.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf5effd700 1 -- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf60066450 msgr2=0x7fbf601572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf5effd700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf60066450 0x7fbf601572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf5effd700 1 -- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6013a6f0 msgr2=0x7fbf60165b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf5effd700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6013a6f0 0x7fbf60165b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf5effd700 1 -- 172.21.15.29:0/1936207761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf540092a0 con 0x7fbf600bf870 2024-01-24T13:59:48.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf5ffff700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6013a6f0 0x7fbf60165b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:48.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf5effd700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf600bf870 0x7fbf6013c5c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fbf54006d70 tx=0x7fbf54017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:48.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf477fe700 1 -- 172.21.15.29:0/1936207761 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400f7e0 con 0x7fbf600bf870 2024-01-24T13:59:48.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf477fe700 1 -- 172.21.15.29:0/1936207761 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5400f940 con 0x7fbf600bf870 2024-01-24T13:59:48.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1936207761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf600abfe0 con 0x7fbf600bf870 2024-01-24T13:59:48.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf477fe700 1 -- 172.21.15.29:0/1936207761 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5401b660 con 0x7fbf600bf870 2024-01-24T13:59:48.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.677+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1936207761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf601672d0 con 0x7fbf600bf870 2024-01-24T13:59:48.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.679+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1936207761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf600ad820 con 0x7fbf600bf870 2024-01-24T13:59:48.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.679+0000 7fbf477fe700 1 -- 172.21.15.29:0/1936207761 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7fbf5401b9d0 con 0x7fbf600bf870 2024-01-24T13:59:48.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.679+0000 7fbf477fe700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf4c05f350 0x7fbf4c061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:48.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.679+0000 7fbf477fe700 1 -- 172.21.15.29:0/1936207761 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7fbf5407c240 con 0x7fbf600bf870 2024-01-24T13:59:48.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.680+0000 7fbf5f7fe700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf4c05f350 0x7fbf4c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:48.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.680+0000 7fbf5f7fe700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf4c05f350 0x7fbf4c061810 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fbf500038c0 tx=0x7fbf50007d40).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:48.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.684+0000 7fbf477fe700 1 -- 172.21.15.29:0/1936207761 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbf600ad820 con 0x7fbf600bf870 2024-01-24T13:59:48.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.890+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1936207761 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf600ad820 con 0x7fbf4c05f350 2024-01-24T13:59:48.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.895+0000 7fbf477fe700 1 -- 172.21.15.29:0/1936207761 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fbf600ad820 con 0x7fbf4c05f350 2024-01-24T13:59:48.895 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /snap_test_dir1/dir9 2024-01-24T13:59:48.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.897+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf4c05f350 msgr2=0x7fbf4c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.897+0000 7fbf65c8c700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf4c05f350 0x7fbf4c061810 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fbf500038c0 tx=0x7fbf50007d40).stop 2024-01-24T13:59:48.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.897+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1936207761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf600bf870 msgr2=0x7fbf6013c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:48.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.897+0000 7fbf65c8c700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf600bf870 0x7fbf6013c5c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fbf54006d70 tx=0x7fbf54017000).stop 2024-01-24T13:59:48.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.897+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1936207761 shutdown_connections 2024-01-24T13:59:48.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.897+0000 7fbf65c8c700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6013a6f0 0x7fbf60165b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.897+0000 7fbf65c8c700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf60066450 0x7fbf601572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.897+0000 7fbf65c8c700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf4c05f350 0x7fbf4c061810 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.897+0000 7fbf65c8c700 1 --2- 172.21.15.29:0/1936207761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf600bf870 0x7fbf6013c5c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:48.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.897+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1936207761 >> 172.21.15.29:0/1936207761 conn(0x7fbf600c8bd0 msgr2=0x7fbf600c3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:48.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.897+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1936207761 shutdown_connections 2024-01-24T13:59:48.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:48.897+0000 7fbf65c8c700 1 -- 172.21.15.29:0/1936207761 wait complete. 2024-01-24T13:59:48.910 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2024-01-24T13:59:48.911 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir0 1M 2024-01-24T13:59:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.048+0000 7f95e4bb2700 1 Processor -- start 2024-01-24T13:59:49.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.048+0000 7f95e4bb2700 1 -- start start 2024-01-24T13:59:49.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.049+0000 7f95e4bb2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95e0062970 0x7f95e0062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.049+0000 7f95e4bb2700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e00c3e00 con 0x7f95e00c2190 2024-01-24T13:59:49.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.049+0000 7f95e4bb2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e00c2900 con 0x7f95e0158720 2024-01-24T13:59:49.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.049+0000 7f95e4bb2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e0065720 con 0x7f95e0062970 2024-01-24T13:59:49.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.049+0000 7f95ddd9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f95e00c2190 0x7f95e00c4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33296/0 (socket says 172.21.15.29:33296) 2024-01-24T13:59:49.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.049+0000 7f95ddd9b700 1 -- 172.21.15.29:0/3306318694 learned_addr learned my addr 172.21.15.29:0/3306318694 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:49.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.049+0000 7f95de59c700 1 --2- 172.21.15.29:0/3306318694 >> v2:172.21.15.163:3300/0 conn(0x7f95e0062970 0x7f95e0062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.049+0000 7f95dd59a700 1 -- 172.21.15.29:0/3306318694 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025285446 0 0) 0x7f95e00c3e00 con 0x7f95e00c2190 2024-01-24T13:59:49.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.049+0000 7f95dd59a700 1 -- 172.21.15.29:0/3306318694 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c80033f0 con 0x7f95e00c2190 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.050+0000 7f95dd59a700 1 -- 172.21.15.29:0/3306318694 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777731775 0 0) 0x7f95e00c2900 con 0x7f95e0158720 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.050+0000 7f95dd59a700 1 -- 172.21.15.29:0/3306318694 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95e00c3e00 con 0x7f95e0158720 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.050+0000 7f95de59c700 1 -- 172.21.15.29:0/3306318694 >> v1:172.21.15.163:6789/0 conn(0x7f95e0158720 legacy=0x7f95e00a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.050+0000 7f95de59c700 1 -- 172.21.15.29:0/3306318694 >> v1:172.21.15.29:6789/0 conn(0x7f95e00c2190 legacy=0x7f95e00c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.050+0000 7f95de59c700 1 -- 172.21.15.29:0/3306318694 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e0158bd0 con 0x7f95e0062970 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.050+0000 7f95de59c700 1 --2- 172.21.15.29:0/3306318694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95e0062970 0x7f95e0062d50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f95d0009760 tx=0x7f95d0009060).ready entity=mon.1 client_cookie=9613448e3da9222d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.050+0000 7f95dd59a700 1 -- 172.21.15.29:0/3306318694 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000d4a0 con 0x7f95e0062970 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.050+0000 7f95dd59a700 1 -- 172.21.15.29:0/3306318694 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d000d600 con 0x7f95e0062970 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.050+0000 7f95dd59a700 1 -- 172.21.15.29:0/3306318694 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000d980 con 0x7f95e0062970 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.050+0000 7f95e4bb2700 1 -- 172.21.15.29:0/3306318694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95e0062970 msgr2=0x7f95e0062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.050+0000 7f95e4bb2700 1 --2- 172.21.15.29:0/3306318694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95e0062970 0x7f95e0062d50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f95d0009760 tx=0x7f95d0009060).stop 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.051+0000 7f95e4bb2700 1 -- 172.21.15.29:0/3306318694 shutdown_connections 2024-01-24T13:59:49.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.051+0000 7f95e4bb2700 1 --2- 172.21.15.29:0/3306318694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95e0062970 0x7f95e0062d50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.051+0000 7f95e4bb2700 1 -- 172.21.15.29:0/3306318694 >> 172.21.15.29:0/3306318694 conn(0x7f95e0067b20 msgr2=0x7f95e00638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:49.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.051+0000 7f95e4bb2700 1 -- 172.21.15.29:0/3306318694 shutdown_connections 2024-01-24T13:59:49.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.051+0000 7f95e4bb2700 1 -- 172.21.15.29:0/3306318694 wait complete. 2024-01-24T13:59:49.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.052+0000 7f95e4bb2700 1 Processor -- start 2024-01-24T13:59:49.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.052+0000 7f95e4bb2700 1 -- start start 2024-01-24T13:59:49.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95e4bb2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95e0062970 0x7f95e014f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95e4bb2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95e0158720 0x7f95e00c1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95e4bb2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e0159810 0x7f95e01562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95e4bb2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00c2030 con 0x7f95e0159810 2024-01-24T13:59:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95e4bb2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0153a50 con 0x7f95e0158720 2024-01-24T13:59:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95e4bb2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0153d30 con 0x7f95e0062970 2024-01-24T13:59:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95ded9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e0159810 0x7f95e01562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95ded9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e0159810 0x7f95e01562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51136/0 (socket says 172.21.15.29:51136) 2024-01-24T13:59:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95ded9d700 1 -- 172.21.15.29:0/2082299617 learned_addr learned my addr 172.21.15.29:0/2082299617 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95ddd9b700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95e0158720 0x7f95e00c1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95de59c700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95e0062970 0x7f95e014f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95ded9d700 1 -- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95e0062970 msgr2=0x7f95e014f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95ded9d700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95e0062970 0x7f95e014f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95ded9d700 1 -- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95e0158720 msgr2=0x7f95e00c1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95ded9d700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95e0158720 0x7f95e00c1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95ded9d700 1 -- 172.21.15.29:0/2082299617 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00092a0 con 0x7f95e0159810 2024-01-24T13:59:49.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95ddd9b700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95e0158720 0x7f95e00c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:49.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.053+0000 7f95ded9d700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e0159810 0x7f95e01562a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f95d800c7c0 tx=0x7f95d800cb80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:49.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.054+0000 7f95ce7fc700 1 -- 172.21.15.29:0/2082299617 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8002240 con 0x7f95e0159810 2024-01-24T13:59:49.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.054+0000 7f95ce7fc700 1 -- 172.21.15.29:0/2082299617 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d80023a0 con 0x7f95e0159810 2024-01-24T13:59:49.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.054+0000 7f95e4bb2700 1 -- 172.21.15.29:0/2082299617 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e015a040 con 0x7f95e0159810 2024-01-24T13:59:49.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.054+0000 7f95ce7fc700 1 -- 172.21.15.29:0/2082299617 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d80017e0 con 0x7f95e0159810 2024-01-24T13:59:49.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.054+0000 7f95e4bb2700 1 -- 172.21.15.29:0/2082299617 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e015a600 con 0x7f95e0159810 2024-01-24T13:59:49.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.055+0000 7f95e4bb2700 1 -- 172.21.15.29:0/2082299617 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e0153ee0 con 0x7f95e0159810 2024-01-24T13:59:49.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.055+0000 7f95ce7fc700 1 -- 172.21.15.29:0/2082299617 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f95d8002510 con 0x7f95e0159810 2024-01-24T13:59:49.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.056+0000 7f95ce7fc700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f95c005f350 0x7f95c0061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.056+0000 7f95ce7fc700 1 -- 172.21.15.29:0/2082299617 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f95d8079ed0 con 0x7f95e0159810 2024-01-24T13:59:49.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.056+0000 7f95de59c700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f95c005f350 0x7f95c0061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.056+0000 7f95de59c700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f95c005f350 0x7f95c0061810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f95d000d260 tx=0x7f95d0016000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:49.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.060+0000 7f95ce7fc700 1 -- 172.21.15.29:0/2082299617 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f95d8048b00 con 0x7f95e0159810 2024-01-24T13:59:49.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.271+0000 7f95e4bb2700 1 -- 172.21.15.29:0/2082299617 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f95e00475e0 con 0x7f95c005f350 2024-01-24T13:59:49.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.277+0000 7f95ce7fc700 1 -- 172.21.15.29:0/2082299617 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f95e00475e0 con 0x7f95c005f350 2024-01-24T13:59:49.277 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule activated for path /snap_test_dir1/dir0 2024-01-24T13:59:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.279+0000 7f95e4bb2700 1 -- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f95c005f350 msgr2=0x7f95c0061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.279+0000 7f95e4bb2700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f95c005f350 0x7f95c0061810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f95d000d260 tx=0x7f95d0016000).stop 2024-01-24T13:59:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.279+0000 7f95e4bb2700 1 -- 172.21.15.29:0/2082299617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e0159810 msgr2=0x7f95e01562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.279+0000 7f95e4bb2700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e0159810 0x7f95e01562a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f95d800c7c0 tx=0x7f95d800cb80).stop 2024-01-24T13:59:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.279+0000 7f95e4bb2700 1 -- 172.21.15.29:0/2082299617 shutdown_connections 2024-01-24T13:59:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.279+0000 7f95e4bb2700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f95c005f350 0x7f95c0061810 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.279+0000 7f95e4bb2700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95e0159810 0x7f95e01562a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.279+0000 7f95e4bb2700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95e0062970 0x7f95e014f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.279+0000 7f95e4bb2700 1 --2- 172.21.15.29:0/2082299617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95e0158720 0x7f95e00c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.279+0000 7f95e4bb2700 1 -- 172.21.15.29:0/2082299617 >> 172.21.15.29:0/2082299617 conn(0x7f95e0067b20 msgr2=0x7f95e00abb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.279+0000 7f95e4bb2700 1 -- 172.21.15.29:0/2082299617 shutdown_connections 2024-01-24T13:59:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.279+0000 7f95e4bb2700 1 -- 172.21.15.29:0/2082299617 wait complete. 2024-01-24T13:59:49.293 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2024-01-24T13:59:49.293 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir1 1M 2024-01-24T13:59:49.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.428+0000 7ff23bbbc700 1 Processor -- start 2024-01-24T13:59:49.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.428+0000 7ff23bbbc700 1 -- start start 2024-01-24T13:59:49.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.429+0000 7ff23bbbc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff23414fcd0 0x7ff234156190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.429+0000 7ff23bbbc700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23406a690 con 0x7ff234156680 2024-01-24T13:59:49.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.429+0000 7ff23bbbc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff234155d00 con 0x7ff23414f8f0 2024-01-24T13:59:49.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.429+0000 7ff23bbbc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff234155a60 con 0x7ff23414fcd0 2024-01-24T13:59:49.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.429+0000 7ff23a159700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff234156680 0x7ff234065410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33310/0 (socket says 172.21.15.29:33310) 2024-01-24T13:59:49.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.429+0000 7ff23a159700 1 -- 172.21.15.29:0/2460154796 learned_addr learned my addr 172.21.15.29:0/2460154796 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:49.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.429+0000 7ff239157700 1 --2- 172.21.15.29:0/2460154796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff23414fcd0 0x7ff234156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.430+0000 7ff238956700 1 -- 172.21.15.29:0/2460154796 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428032493 0 0) 0x7ff22c001490 con 0x7ff234156680 2024-01-24T13:59:49.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.430+0000 7ff238956700 1 -- 172.21.15.29:0/2460154796 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2200033f0 con 0x7ff234156680 2024-01-24T13:59:49.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.430+0000 7ff238956700 1 -- 172.21.15.29:0/2460154796 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244684977 0 0) 0x7ff234155d00 con 0x7ff23414f8f0 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.430+0000 7ff238956700 1 -- 172.21.15.29:0/2460154796 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff22c001490 con 0x7ff23414f8f0 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.430+0000 7ff239157700 1 -- 172.21.15.29:0/2460154796 >> v1:172.21.15.163:6789/0 conn(0x7ff23414f8f0 legacy=0x7ff234152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.430+0000 7ff239157700 1 -- 172.21.15.29:0/2460154796 >> v1:172.21.15.29:6789/0 conn(0x7ff234156680 legacy=0x7ff234065410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.430+0000 7ff239157700 1 -- 172.21.15.29:0/2460154796 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff234152cf0 con 0x7ff23414fcd0 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.430+0000 7ff239157700 1 --2- 172.21.15.29:0/2460154796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff23414fcd0 0x7ff234156190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7ff224009790 tx=0x7ff224009060).ready entity=mon.2 client_cookie=2c949a68edf90e61 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.430+0000 7ff238956700 1 -- 172.21.15.29:0/2460154796 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22400f510 con 0x7ff23414fcd0 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.430+0000 7ff238956700 1 -- 172.21.15.29:0/2460154796 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22400f670 con 0x7ff23414fcd0 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.430+0000 7ff238956700 1 -- 172.21.15.29:0/2460154796 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22400f9f0 con 0x7ff23414fcd0 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.431+0000 7ff23bbbc700 1 -- 172.21.15.29:0/2460154796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff23414fcd0 msgr2=0x7ff234156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.431+0000 7ff23bbbc700 1 --2- 172.21.15.29:0/2460154796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff23414fcd0 0x7ff234156190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7ff224009790 tx=0x7ff224009060).stop 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.431+0000 7ff23bbbc700 1 -- 172.21.15.29:0/2460154796 shutdown_connections 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.431+0000 7ff23bbbc700 1 --2- 172.21.15.29:0/2460154796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff23414fcd0 0x7ff234156190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.431+0000 7ff23bbbc700 1 -- 172.21.15.29:0/2460154796 >> 172.21.15.29:0/2460154796 conn(0x7ff234067b20 msgr2=0x7ff2340638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.431+0000 7ff23bbbc700 1 -- 172.21.15.29:0/2460154796 shutdown_connections 2024-01-24T13:59:49.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.431+0000 7ff23bbbc700 1 -- 172.21.15.29:0/2460154796 wait complete. 2024-01-24T13:59:49.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.432+0000 7ff23bbbc700 1 Processor -- start 2024-01-24T13:59:49.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.432+0000 7ff23bbbc700 1 -- start start 2024-01-24T13:59:49.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff23bbbc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff23414f8f0 0x7ff23414c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff23bbbc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff23414fcd0 0x7ff2340c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff23bbbc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff234156680 0x7ff234161f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff23bbbc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff234162800 con 0x7ff23414fcd0 2024-01-24T13:59:49.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff23bbbc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff234162ae0 con 0x7ff234156680 2024-01-24T13:59:49.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff23bbbc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2340a9380 con 0x7ff23414f8f0 2024-01-24T13:59:49.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff239157700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff23414fcd0 0x7ff2340c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff239157700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff23414fcd0 0x7ff2340c12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51142/0 (socket says 172.21.15.29:51142) 2024-01-24T13:59:49.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff239157700 1 -- 172.21.15.29:0/3656652975 learned_addr learned my addr 172.21.15.29:0/3656652975 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:49.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff23a159700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff234156680 0x7ff234161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff239958700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff23414f8f0 0x7ff23414c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff239157700 1 -- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff23414f8f0 msgr2=0x7ff23414c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff239157700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff23414f8f0 0x7ff23414c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff239157700 1 -- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff234156680 msgr2=0x7ff234161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff239157700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff234156680 0x7ff234161f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff239157700 1 -- 172.21.15.29:0/3656652975 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2240092a0 con 0x7ff23414fcd0 2024-01-24T13:59:49.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff23a159700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff234156680 0x7ff234161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:49.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.433+0000 7ff239157700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff23414fcd0 0x7ff2340c12e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7ff234068a90 tx=0x7ff224017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:49.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.434+0000 7ff229ffb700 1 -- 172.21.15.29:0/3656652975 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22400f7e0 con 0x7ff23414fcd0 2024-01-24T13:59:49.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.434+0000 7ff229ffb700 1 -- 172.21.15.29:0/3656652975 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22400f940 con 0x7ff23414fcd0 2024-01-24T13:59:49.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.434+0000 7ff23bbbc700 1 -- 172.21.15.29:0/3656652975 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2340a9660 con 0x7ff23414fcd0 2024-01-24T13:59:49.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.434+0000 7ff229ffb700 1 -- 172.21.15.29:0/3656652975 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22401b620 con 0x7ff23414fcd0 2024-01-24T13:59:49.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.434+0000 7ff23bbbc700 1 -- 172.21.15.29:0/3656652975 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff234162dc0 con 0x7ff23414fcd0 2024-01-24T13:59:49.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.435+0000 7ff23bbbc700 1 -- 172.21.15.29:0/3656652975 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2340a9880 con 0x7ff23414fcd0 2024-01-24T13:59:49.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.435+0000 7ff229ffb700 1 -- 172.21.15.29:0/3656652975 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7ff22401ba50 con 0x7ff23414fcd0 2024-01-24T13:59:49.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.436+0000 7ff229ffb700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff21805f320 0x7ff2180617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.436+0000 7ff229ffb700 1 -- 172.21.15.29:0/3656652975 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7ff22407c370 con 0x7ff23414fcd0 2024-01-24T13:59:49.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.436+0000 7ff239958700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff21805f320 0x7ff2180617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.437+0000 7ff239958700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff21805f320 0x7ff2180617e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff2340adcb0 tx=0x7ff230009040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:49.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.440+0000 7ff229ffb700 1 -- 172.21.15.29:0/3656652975 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff22404d130 con 0x7ff23414fcd0 2024-01-24T13:59:49.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.650+0000 7ff23bbbc700 1 -- 172.21.15.29:0/3656652975 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2340475e0 con 0x7ff21805f320 2024-01-24T13:59:49.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.655+0000 7ff229ffb700 1 -- 172.21.15.29:0/3656652975 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff2340475e0 con 0x7ff21805f320 2024-01-24T13:59:49.655 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule activated for path /snap_test_dir1/dir1 2024-01-24T13:59:49.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.657+0000 7ff23bbbc700 1 -- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff21805f320 msgr2=0x7ff2180617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.657+0000 7ff23bbbc700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff21805f320 0x7ff2180617e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff2340adcb0 tx=0x7ff230009040).stop 2024-01-24T13:59:49.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.657+0000 7ff23bbbc700 1 -- 172.21.15.29:0/3656652975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff23414fcd0 msgr2=0x7ff2340c12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.657+0000 7ff23bbbc700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff23414fcd0 0x7ff2340c12e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7ff234068a90 tx=0x7ff224017000).stop 2024-01-24T13:59:49.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.657+0000 7ff23bbbc700 1 -- 172.21.15.29:0/3656652975 shutdown_connections 2024-01-24T13:59:49.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.657+0000 7ff23bbbc700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff234156680 0x7ff234161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.657+0000 7ff23bbbc700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff23414f8f0 0x7ff23414c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.657+0000 7ff23bbbc700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff21805f320 0x7ff2180617e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.657+0000 7ff23bbbc700 1 --2- 172.21.15.29:0/3656652975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff23414fcd0 0x7ff2340c12e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.657+0000 7ff23bbbc700 1 -- 172.21.15.29:0/3656652975 >> 172.21.15.29:0/3656652975 conn(0x7ff234067b20 msgr2=0x7ff234151080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:49.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.657+0000 7ff23bbbc700 1 -- 172.21.15.29:0/3656652975 shutdown_connections 2024-01-24T13:59:49.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.657+0000 7ff23bbbc700 1 -- 172.21.15.29:0/3656652975 wait complete. 2024-01-24T13:59:49.671 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2024-01-24T13:59:49.671 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir2 1M 2024-01-24T13:59:49.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.806+0000 7ff5cda4a700 1 Processor -- start 2024-01-24T13:59:49.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.806+0000 7ff5cda4a700 1 -- start start 2024-01-24T13:59:49.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.807+0000 7ff5cda4a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5c8154170 0x7ff5c80627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.807+0000 7ff5cda4a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5c8062ca0 0x7ff5c80c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.807+0000 7ff5cda4a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c8069360 con 0x7ff5c8153d90 2024-01-24T13:59:49.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.807+0000 7ff5cda4a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c81545f0 con 0x7ff5c8062ca0 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.807+0000 7ff5cda4a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c806a1c0 con 0x7ff5c8154170 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.807+0000 7ff5c77fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5c8062ca0 0x7ff5c80c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.807+0000 7ff5c67fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5c8154170 0x7ff5c80627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.807+0000 7ff5c77fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5c8062ca0 0x7ff5c80c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51158/0 (socket says 172.21.15.29:51158) 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.807+0000 7ff5c77fe700 1 -- 172.21.15.29:0/246489348 learned_addr learned my addr 172.21.15.29:0/246489348 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.808+0000 7ff5c5ffb700 1 -- 172.21.15.29:0/246489348 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688979265 0 0) 0x7ff5c8069360 con 0x7ff5c8153d90 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.808+0000 7ff5c77fe700 1 -- 172.21.15.29:0/246489348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5c8154170 msgr2=0x7ff5c80627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.808+0000 7ff5c77fe700 1 --2- 172.21.15.29:0/246489348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5c8154170 0x7ff5c80627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.808+0000 7ff5c77fe700 1 -- 172.21.15.29:0/246489348 >> v1:172.21.15.163:6789/0 conn(0x7ff5c8153d90 legacy=0x7ff5c80c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.808+0000 7ff5c77fe700 1 -- 172.21.15.29:0/246489348 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c806a650 con 0x7ff5c8062ca0 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.808+0000 7ff5c77fe700 1 --2- 172.21.15.29:0/246489348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c8062ca0 0x7ff5c80c3e00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7ff5c0009f20 tx=0x7ff5c0009840).ready entity=mon.0 client_cookie=a0aa6f1c28222416 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.808+0000 7ff5c5ffb700 1 -- 172.21.15.29:0/246489348 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c000e070 con 0x7ff5c8062ca0 2024-01-24T13:59:49.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.808+0000 7ff5c5ffb700 1 -- 172.21.15.29:0/246489348 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c000b9d0 con 0x7ff5c8062ca0 2024-01-24T13:59:49.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.808+0000 7ff5c5ffb700 1 -- 172.21.15.29:0/246489348 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c000bd50 con 0x7ff5c8062ca0 2024-01-24T13:59:49.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.809+0000 7ff5cda4a700 1 -- 172.21.15.29:0/246489348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c8062ca0 msgr2=0x7ff5c80c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.809+0000 7ff5cda4a700 1 --2- 172.21.15.29:0/246489348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c8062ca0 0x7ff5c80c3e00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7ff5c0009f20 tx=0x7ff5c0009840).stop 2024-01-24T13:59:49.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.809+0000 7ff5cda4a700 1 -- 172.21.15.29:0/246489348 shutdown_connections 2024-01-24T13:59:49.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.809+0000 7ff5cda4a700 1 --2- 172.21.15.29:0/246489348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c8062ca0 0x7ff5c80c3e00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.809+0000 7ff5cda4a700 1 --2- 172.21.15.29:0/246489348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5c8154170 0x7ff5c80627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.809+0000 7ff5cda4a700 1 -- 172.21.15.29:0/246489348 >> 172.21.15.29:0/246489348 conn(0x7ff5c8067b20 msgr2=0x7ff5c80638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:49.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.809+0000 7ff5cda4a700 1 -- 172.21.15.29:0/246489348 shutdown_connections 2024-01-24T13:59:49.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.809+0000 7ff5cda4a700 1 -- 172.21.15.29:0/246489348 wait complete. 2024-01-24T13:59:49.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.810+0000 7ff5cda4a700 1 Processor -- start 2024-01-24T13:59:49.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.810+0000 7ff5cda4a700 1 -- start start 2024-01-24T13:59:49.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.811+0000 7ff5cda4a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c8062ca0 0x7ff5c80ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.811+0000 7ff5cda4a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5c8153d90 0x7ff5c8156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.811+0000 7ff5cda4a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c8154170 0x7ff5c8157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.811+0000 7ff5cda4a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80ab5b0 con 0x7ff5c8062ca0 2024-01-24T13:59:49.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.811+0000 7ff5cda4a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80a85e0 con 0x7ff5c8154170 2024-01-24T13:59:49.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.811+0000 7ff5cda4a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80a79d0 con 0x7ff5c8153d90 2024-01-24T13:59:49.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.811+0000 7ff5c6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c8062ca0 0x7ff5c80ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.811+0000 7ff5c6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c8062ca0 0x7ff5c80ab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51174/0 (socket says 172.21.15.29:51174) 2024-01-24T13:59:49.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.811+0000 7ff5c6ffd700 1 -- 172.21.15.29:0/3558660540 learned_addr learned my addr 172.21.15.29:0/3558660540 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:49.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.811+0000 7ff5c67fc700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5c8153d90 0x7ff5c8156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.811+0000 7ff5c77fe700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c8154170 0x7ff5c8157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.812+0000 7ff5c6ffd700 1 -- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5c8153d90 msgr2=0x7ff5c8156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.812+0000 7ff5c6ffd700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5c8153d90 0x7ff5c8156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.812+0000 7ff5c6ffd700 1 -- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c8154170 msgr2=0x7ff5c8157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.812+0000 7ff5c6ffd700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c8154170 0x7ff5c8157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.812+0000 7ff5c6ffd700 1 -- 172.21.15.29:0/3558660540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0009ae0 con 0x7ff5c8062ca0 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.812+0000 7ff5c6ffd700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c8062ca0 0x7ff5c80ab0c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7ff5b800c4e0 tx=0x7ff5b8007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.812+0000 7ff5beffd700 1 -- 172.21.15.29:0/3558660540 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b800ce50 con 0x7ff5c8062ca0 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.812+0000 7ff5beffd700 1 -- 172.21.15.29:0/3558660540 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b800f470 con 0x7ff5c8062ca0 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.812+0000 7ff5beffd700 1 -- 172.21.15.29:0/3558660540 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b800f830 con 0x7ff5c8062ca0 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.812+0000 7ff5cda4a700 1 -- 172.21.15.29:0/3558660540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c80a7cb0 con 0x7ff5c8062ca0 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.812+0000 7ff5cda4a700 1 -- 172.21.15.29:0/3558660540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c80a8270 con 0x7ff5c8062ca0 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.813+0000 7ff5cda4a700 1 -- 172.21.15.29:0/3558660540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c80475e0 con 0x7ff5c8062ca0 2024-01-24T13:59:49.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.813+0000 7ff5beffd700 1 -- 172.21.15.29:0/3558660540 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7ff5b800fba0 con 0x7ff5c8062ca0 2024-01-24T13:59:49.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.814+0000 7ff5beffd700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5a805f350 0x7ff5a8061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:49.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.814+0000 7ff5beffd700 1 -- 172.21.15.29:0/3558660540 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7ff5b8079a90 con 0x7ff5c8062ca0 2024-01-24T13:59:49.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.814+0000 7ff5c67fc700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5a805f350 0x7ff5a8061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:49.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.815+0000 7ff5c67fc700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5a805f350 0x7ff5a8061810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ff5c80ade10 tx=0x7ff5b0009060).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:49.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:49.818+0000 7ff5beffd700 1 -- 172.21.15.29:0/3558660540 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff5b8048b00 con 0x7ff5c8062ca0 2024-01-24T13:59:50.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.032+0000 7ff5cda4a700 1 -- 172.21.15.29:0/3558660540 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5c8153550 con 0x7ff5a805f350 2024-01-24T13:59:50.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.037+0000 7ff5beffd700 1 -- 172.21.15.29:0/3558660540 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff5c8153550 con 0x7ff5a805f350 2024-01-24T13:59:50.037 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule activated for path /snap_test_dir1/dir2 2024-01-24T13:59:50.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.039+0000 7ff5cda4a700 1 -- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5a805f350 msgr2=0x7ff5a8061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.039+0000 7ff5cda4a700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5a805f350 0x7ff5a8061810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ff5c80ade10 tx=0x7ff5b0009060).stop 2024-01-24T13:59:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.039+0000 7ff5cda4a700 1 -- 172.21.15.29:0/3558660540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c8062ca0 msgr2=0x7ff5c80ab0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.039+0000 7ff5cda4a700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c8062ca0 0x7ff5c80ab0c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7ff5b800c4e0 tx=0x7ff5b8007c70).stop 2024-01-24T13:59:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.039+0000 7ff5cda4a700 1 -- 172.21.15.29:0/3558660540 shutdown_connections 2024-01-24T13:59:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.039+0000 7ff5cda4a700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c8154170 0x7ff5c8157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.039+0000 7ff5cda4a700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5a805f350 0x7ff5a8061810 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.039+0000 7ff5cda4a700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5c8062ca0 0x7ff5c80ab0c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.039+0000 7ff5cda4a700 1 --2- 172.21.15.29:0/3558660540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5c8153d90 0x7ff5c8156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.039+0000 7ff5cda4a700 1 -- 172.21.15.29:0/3558660540 >> 172.21.15.29:0/3558660540 conn(0x7ff5c8067b20 msgr2=0x7ff5c80c6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.039+0000 7ff5cda4a700 1 -- 172.21.15.29:0/3558660540 shutdown_connections 2024-01-24T13:59:50.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.039+0000 7ff5cda4a700 1 -- 172.21.15.29:0/3558660540 wait complete. 2024-01-24T13:59:50.053 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2024-01-24T13:59:50.053 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir3 1M 2024-01-24T13:59:50.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.188+0000 7fac7c33b700 1 Processor -- start 2024-01-24T13:59:50.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.188+0000 7fac7c33b700 1 -- start start 2024-01-24T13:59:50.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.189+0000 7fac7c33b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fac7414fb30 0x7fac740bca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.189+0000 7fac7c33b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac74153130 0x7fac740653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.189+0000 7fac7c33b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac740658d0 0x7fac740698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.189+0000 7fac7c33b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac740694c0 con 0x7fac74153130 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.189+0000 7fac7c33b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac740c89b0 con 0x7fac7414fb30 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.189+0000 7fac7c33b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac740c81e0 con 0x7fac740658d0 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.189+0000 7fac798d6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac74153130 0x7fac740653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.189+0000 7fac798d6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac74153130 0x7fac740653e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51176/0 (socket says 172.21.15.29:51176) 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.189+0000 7fac798d6700 1 -- 172.21.15.29:0/559597216 learned_addr learned my addr 172.21.15.29:0/559597216 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.189+0000 7fac7a0d7700 1 --2- 172.21.15.29:0/559597216 >> v2:172.21.15.163:3300/0 conn(0x7fac7414fb30 0x7fac740bca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac7a8d8700 1 --2- 172.21.15.29:0/559597216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac740658d0 0x7fac740698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac798d6700 1 -- 172.21.15.29:0/559597216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac740658d0 msgr2=0x7fac740698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac798d6700 1 --2- 172.21.15.29:0/559597216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac740658d0 0x7fac740698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac798d6700 1 -- 172.21.15.29:0/559597216 >> v2:172.21.15.163:3300/0 conn(0x7fac7414fb30 msgr2=0x7fac740bca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:50.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac798d6700 1 --2- 172.21.15.29:0/559597216 >> v2:172.21.15.163:3300/0 conn(0x7fac7414fb30 0x7fac740bca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac798d6700 1 -- 172.21.15.29:0/559597216 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac740c83f0 con 0x7fac74153130 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac7a8d8700 1 --2- 172.21.15.29:0/559597216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac740658d0 0x7fac740698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac798d6700 1 --2- 172.21.15.29:0/559597216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac74153130 0x7fac740653e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fac74068d50 tx=0x7fac640090f0).ready entity=mon.0 client_cookie=5cfacc9db73565a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac790d5700 1 -- 172.21.15.29:0/559597216 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6400b600 con 0x7fac74153130 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac790d5700 1 -- 172.21.15.29:0/559597216 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6400b760 con 0x7fac74153130 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac790d5700 1 -- 172.21.15.29:0/559597216 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6400bae0 con 0x7fac74153130 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac7c33b700 1 -- 172.21.15.29:0/559597216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac74153130 msgr2=0x7fac740653e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.190+0000 7fac7c33b700 1 --2- 172.21.15.29:0/559597216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac74153130 0x7fac740653e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fac74068d50 tx=0x7fac640090f0).stop 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.191+0000 7fac7c33b700 1 -- 172.21.15.29:0/559597216 shutdown_connections 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.191+0000 7fac7c33b700 1 --2- 172.21.15.29:0/559597216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac740658d0 0x7fac740698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.191+0000 7fac7c33b700 1 --2- 172.21.15.29:0/559597216 >> v2:172.21.15.163:3300/0 conn(0x7fac7414fb30 0x7fac740bca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.191+0000 7fac7c33b700 1 --2- 172.21.15.29:0/559597216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac74153130 0x7fac740653e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.191+0000 7fac7c33b700 1 -- 172.21.15.29:0/559597216 >> 172.21.15.29:0/559597216 conn(0x7fac74064390 msgr2=0x7fac740c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:50.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.191+0000 7fac7c33b700 1 -- 172.21.15.29:0/559597216 shutdown_connections 2024-01-24T13:59:50.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.191+0000 7fac7c33b700 1 -- 172.21.15.29:0/559597216 wait complete. 2024-01-24T13:59:50.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.192+0000 7fac7c33b700 1 Processor -- start 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.192+0000 7fac7c33b700 1 -- start start 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac7c33b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac740658d0 0x7fac740a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac7c33b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac7414fb30 0x7fac741374b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac7c33b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac74153130 0x7fac740c0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac7c33b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac740beb70 con 0x7fac7414fb30 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac7c33b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac740bf130 con 0x7fac74153130 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac7c33b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac740a9330 con 0x7fac740658d0 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac798d6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac7414fb30 0x7fac741374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac798d6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac7414fb30 0x7fac741374b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51178/0 (socket says 172.21.15.29:51178) 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac798d6700 1 -- 172.21.15.29:0/346800734 learned_addr learned my addr 172.21.15.29:0/346800734 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac7a8d8700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac74153130 0x7fac740c0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac7a0d7700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac740658d0 0x7fac740a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac798d6700 1 -- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac740658d0 msgr2=0x7fac740a9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac798d6700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac740658d0 0x7fac740a9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac798d6700 1 -- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac74153130 msgr2=0x7fac740c0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac798d6700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac74153130 0x7fac740c0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac798d6700 1 -- 172.21.15.29:0/346800734 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac64009370 con 0x7fac7414fb30 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac7a8d8700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac74153130 0x7fac740c0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.193+0000 7fac798d6700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac7414fb30 0x7fac741374b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fac64009130 tx=0x7fac64017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.194+0000 7fac6a7fc700 1 -- 172.21.15.29:0/346800734 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6400b8d0 con 0x7fac7414fb30 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.194+0000 7fac6a7fc700 1 -- 172.21.15.29:0/346800734 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6400ba30 con 0x7fac7414fb30 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.194+0000 7fac7c33b700 1 -- 172.21.15.29:0/346800734 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac740a9610 con 0x7fac7414fb30 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.194+0000 7fac6a7fc700 1 -- 172.21.15.29:0/346800734 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6401d680 con 0x7fac7414fb30 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.194+0000 7fac7c33b700 1 -- 172.21.15.29:0/346800734 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac740aa4c0 con 0x7fac7414fb30 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.195+0000 7fac7c33b700 1 -- 172.21.15.29:0/346800734 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac740bf410 con 0x7fac7414fb30 2024-01-24T13:59:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.195+0000 7fac6a7fc700 1 -- 172.21.15.29:0/346800734 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7fac6401da30 con 0x7fac7414fb30 2024-01-24T13:59:50.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.195+0000 7fac6a7fc700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fac5805f350 0x7fac58061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.195+0000 7fac6a7fc700 1 -- 172.21.15.29:0/346800734 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7fac6407c3a0 con 0x7fac7414fb30 2024-01-24T13:59:50.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.196+0000 7fac7a0d7700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fac5805f350 0x7fac58061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.196+0000 7fac7a0d7700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fac5805f350 0x7fac58061810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fac70006fd0 tx=0x7fac70008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:50.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.200+0000 7fac6a7fc700 1 -- 172.21.15.29:0/346800734 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fac64050c90 con 0x7fac7414fb30 2024-01-24T13:59:50.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.415+0000 7fac7c33b700 1 -- 172.21.15.29:0/346800734 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fac740475e0 con 0x7fac5805f350 2024-01-24T13:59:50.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.420+0000 7fac6a7fc700 1 -- 172.21.15.29:0/346800734 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fac740475e0 con 0x7fac5805f350 2024-01-24T13:59:50.420 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule activated for path /snap_test_dir1/dir3 2024-01-24T13:59:50.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.422+0000 7fac7c33b700 1 -- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fac5805f350 msgr2=0x7fac58061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.422+0000 7fac7c33b700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fac5805f350 0x7fac58061810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fac70006fd0 tx=0x7fac70008040).stop 2024-01-24T13:59:50.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.422+0000 7fac7c33b700 1 -- 172.21.15.29:0/346800734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac7414fb30 msgr2=0x7fac741374b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.422+0000 7fac7c33b700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac7414fb30 0x7fac741374b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fac64009130 tx=0x7fac64017000).stop 2024-01-24T13:59:50.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.422+0000 7fac7c33b700 1 -- 172.21.15.29:0/346800734 shutdown_connections 2024-01-24T13:59:50.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.422+0000 7fac7c33b700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac74153130 0x7fac740c0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.422+0000 7fac7c33b700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac740658d0 0x7fac740a9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.422+0000 7fac7c33b700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fac5805f350 0x7fac58061810 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.422+0000 7fac7c33b700 1 --2- 172.21.15.29:0/346800734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac7414fb30 0x7fac741374b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.422+0000 7fac7c33b700 1 -- 172.21.15.29:0/346800734 >> 172.21.15.29:0/346800734 conn(0x7fac74064390 msgr2=0x7fac740627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:50.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.422+0000 7fac7c33b700 1 -- 172.21.15.29:0/346800734 shutdown_connections 2024-01-24T13:59:50.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.422+0000 7fac7c33b700 1 -- 172.21.15.29:0/346800734 wait complete. 2024-01-24T13:59:50.435 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2024-01-24T13:59:50.436 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir4 1M 2024-01-24T13:59:50.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.570+0000 7f2b7058f700 1 Processor -- start 2024-01-24T13:59:50.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.570+0000 7f2b7058f700 1 -- start start 2024-01-24T13:59:50.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.571+0000 7f2b7058f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b680ac7f0 0x7f2b680acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.571+0000 7f2b7058f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2b680627b0 0x7f2b681559f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.571+0000 7f2b7058f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b68069b00 con 0x7f2b68155ee0 2024-01-24T13:59:50.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.571+0000 7f2b7058f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b680bc9e0 con 0x7f2b680627b0 2024-01-24T13:59:50.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.571+0000 7f2b7058f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b680a9fe0 con 0x7f2b680ac7f0 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6db2a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2b680627b0 0x7f2b681559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6eb2c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2b68155ee0 0x7f2b68151950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33312/0 (socket says 172.21.15.29:33312) 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6eb2c700 1 -- 172.21.15.29:0/3987223800 learned_addr learned my addr 172.21.15.29:0/3987223800 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6e32b700 1 --2- 172.21.15.29:0/3987223800 >> v2:172.21.15.163:3300/0 conn(0x7f2b680ac7f0 0x7f2b680acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6db2a700 1 -- 172.21.15.29:0/3987223800 >> v2:172.21.15.163:3300/0 conn(0x7f2b680ac7f0 msgr2=0x7f2b680acbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6db2a700 1 --2- 172.21.15.29:0/3987223800 >> v2:172.21.15.163:3300/0 conn(0x7f2b680ac7f0 0x7f2b680acbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6db2a700 1 -- 172.21.15.29:0/3987223800 >> v1:172.21.15.29:6789/0 conn(0x7f2b68155ee0 legacy=0x7f2b68151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6db2a700 1 -- 172.21.15.29:0/3987223800 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b68068ef0 con 0x7f2b680627b0 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6d329700 1 -- 172.21.15.29:0/3987223800 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136532309 0 0) 0x7f2b68069b00 con 0x7f2b68155ee0 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6db2a700 1 --2- 172.21.15.29:0/3987223800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b680627b0 0x7f2b681559f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f2b68065c30 tx=0x7f2b5c009060).ready entity=mon.0 client_cookie=250f78e1a157d13 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6d329700 1 -- 172.21.15.29:0/3987223800 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00b580 con 0x7f2b680627b0 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6d329700 1 -- 172.21.15.29:0/3987223800 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c00b6e0 con 0x7f2b680627b0 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.572+0000 7f2b6d329700 1 -- 172.21.15.29:0/3987223800 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00ba60 con 0x7f2b680627b0 2024-01-24T13:59:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.573+0000 7f2b7058f700 1 -- 172.21.15.29:0/3987223800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b680627b0 msgr2=0x7f2b681559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.573+0000 7f2b7058f700 1 --2- 172.21.15.29:0/3987223800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b680627b0 0x7f2b681559f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f2b68065c30 tx=0x7f2b5c009060).stop 2024-01-24T13:59:50.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.573+0000 7f2b7058f700 1 -- 172.21.15.29:0/3987223800 shutdown_connections 2024-01-24T13:59:50.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.573+0000 7f2b7058f700 1 --2- 172.21.15.29:0/3987223800 >> v2:172.21.15.163:3300/0 conn(0x7f2b680ac7f0 0x7f2b680acbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.573+0000 7f2b7058f700 1 --2- 172.21.15.29:0/3987223800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b680627b0 0x7f2b681559f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.573+0000 7f2b7058f700 1 -- 172.21.15.29:0/3987223800 >> 172.21.15.29:0/3987223800 conn(0x7f2b680c7280 msgr2=0x7f2b680c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:50.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.573+0000 7f2b7058f700 1 -- 172.21.15.29:0/3987223800 shutdown_connections 2024-01-24T13:59:50.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.573+0000 7f2b7058f700 1 -- 172.21.15.29:0/3987223800 wait complete. 2024-01-24T13:59:50.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.574+0000 7f2b7058f700 1 Processor -- start 2024-01-24T13:59:50.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.574+0000 7f2b7058f700 1 -- start start 2024-01-24T13:59:50.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.575+0000 7f2b7058f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b680627b0 0x7f2b68164a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.575+0000 7f2b7058f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b680ac7f0 0x7f2b681540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.575+0000 7f2b7058f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b68155ee0 0x7f2b681545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.575+0000 7f2b7058f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68154ac0 con 0x7f2b68155ee0 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.575+0000 7f2b7058f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680a4d50 con 0x7f2b680ac7f0 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.575+0000 7f2b7058f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680a4a00 con 0x7f2b680627b0 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.575+0000 7f2b6eb2c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b68155ee0 0x7f2b681545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.575+0000 7f2b6eb2c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b68155ee0 0x7f2b681545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51196/0 (socket says 172.21.15.29:51196) 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.575+0000 7f2b6eb2c700 1 -- 172.21.15.29:0/1709158103 learned_addr learned my addr 172.21.15.29:0/1709158103 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.575+0000 7f2b6e32b700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b680627b0 0x7f2b68164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.575+0000 7f2b6db2a700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b680ac7f0 0x7f2b681540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.576+0000 7f2b6eb2c700 1 -- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b680627b0 msgr2=0x7f2b68164a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.576+0000 7f2b6eb2c700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b680627b0 0x7f2b68164a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.576+0000 7f2b6eb2c700 1 -- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b680ac7f0 msgr2=0x7f2b681540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.576+0000 7f2b6eb2c700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b680ac7f0 0x7f2b681540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.576+0000 7f2b6eb2c700 1 -- 172.21.15.29:0/1709158103 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c0092a0 con 0x7f2b68155ee0 2024-01-24T13:59:50.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.576+0000 7f2b6eb2c700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b68155ee0 0x7f2b681545d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f2b6000c730 tx=0x7f2b60001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:50.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.576+0000 7f2b5a7fc700 1 -- 172.21.15.29:0/1709158103 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b60005d30 con 0x7f2b68155ee0 2024-01-24T13:59:50.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.576+0000 7f2b5a7fc700 1 -- 172.21.15.29:0/1709158103 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b60005e90 con 0x7f2b68155ee0 2024-01-24T13:59:50.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.576+0000 7f2b7058f700 1 -- 172.21.15.29:0/1709158103 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b680a8df0 con 0x7f2b68155ee0 2024-01-24T13:59:50.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.576+0000 7f2b5a7fc700 1 -- 172.21.15.29:0/1709158103 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b60007300 con 0x7f2b68155ee0 2024-01-24T13:59:50.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.576+0000 7f2b7058f700 1 -- 172.21.15.29:0/1709158103 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b68164f50 con 0x7f2b68155ee0 2024-01-24T13:59:50.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.577+0000 7f2b7058f700 1 -- 172.21.15.29:0/1709158103 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b68165140 con 0x7f2b68155ee0 2024-01-24T13:59:50.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.577+0000 7f2b5a7fc700 1 -- 172.21.15.29:0/1709158103 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f2b600076b0 con 0x7f2b68155ee0 2024-01-24T13:59:50.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.578+0000 7f2b5a7fc700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b4c05f350 0x7f2b4c061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.578+0000 7f2b5a7fc700 1 -- 172.21.15.29:0/1709158103 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f2b6007a060 con 0x7f2b68155ee0 2024-01-24T13:59:50.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.578+0000 7f2b6e32b700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b4c05f350 0x7f2b4c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.579+0000 7f2b6e32b700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b4c05f350 0x7f2b4c061810 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f2b6400bbc0 tx=0x7f2b64008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:50.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.582+0000 7f2b5a7fc700 1 -- 172.21.15.29:0/1709158103 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2b6004af20 con 0x7f2b68155ee0 2024-01-24T13:59:50.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.792+0000 7f2b7058f700 1 -- 172.21.15.29:0/1709158103 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b680a4ee0 con 0x7f2b4c05f350 2024-01-24T13:59:50.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.798+0000 7f2b5a7fc700 1 -- 172.21.15.29:0/1709158103 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f2b680a4ee0 con 0x7f2b4c05f350 2024-01-24T13:59:50.798 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule activated for path /snap_test_dir1/dir4 2024-01-24T13:59:50.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.799+0000 7f2b7058f700 1 -- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b4c05f350 msgr2=0x7f2b4c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.799+0000 7f2b7058f700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b4c05f350 0x7f2b4c061810 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f2b6400bbc0 tx=0x7f2b64008040).stop 2024-01-24T13:59:50.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.799+0000 7f2b7058f700 1 -- 172.21.15.29:0/1709158103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b68155ee0 msgr2=0x7f2b681545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.799+0000 7f2b7058f700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b68155ee0 0x7f2b681545d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f2b6000c730 tx=0x7f2b60001520).stop 2024-01-24T13:59:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.800+0000 7f2b7058f700 1 -- 172.21.15.29:0/1709158103 shutdown_connections 2024-01-24T13:59:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.800+0000 7f2b7058f700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b4c05f350 0x7f2b4c061810 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.800+0000 7f2b7058f700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b68155ee0 0x7f2b681545d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.800+0000 7f2b7058f700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b680627b0 0x7f2b68164a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.800+0000 7f2b7058f700 1 --2- 172.21.15.29:0/1709158103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b680ac7f0 0x7f2b681540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.800+0000 7f2b7058f700 1 -- 172.21.15.29:0/1709158103 >> 172.21.15.29:0/1709158103 conn(0x7f2b680c7280 msgr2=0x7f2b6814c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.800+0000 7f2b7058f700 1 -- 172.21.15.29:0/1709158103 shutdown_connections 2024-01-24T13:59:50.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.800+0000 7f2b7058f700 1 -- 172.21.15.29:0/1709158103 wait complete. 2024-01-24T13:59:50.813 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2024-01-24T13:59:50.813 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir5 1M 2024-01-24T13:59:50.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.948+0000 7f902a266700 1 Processor -- start 2024-01-24T13:59:50.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.948+0000 7f902a266700 1 -- start start 2024-01-24T13:59:50.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.949+0000 7f902a266700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9024150970 0x7f9024150d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.949+0000 7f902a266700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9024069a50 con 0x7f9024155060 2024-01-24T13:59:50.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.949+0000 7f902a266700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90240c2b20 con 0x7f9024069ee0 2024-01-24T13:59:50.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.949+0000 7f902a266700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9024151240 con 0x7f9024150970 2024-01-24T13:59:50.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.950+0000 7f9028803700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9024155060 0x7f90240c4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33324/0 (socket says 172.21.15.29:33324) 2024-01-24T13:59:50.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.950+0000 7f9028803700 1 -- 172.21.15.29:0/1477976739 learned_addr learned my addr 172.21.15.29:0/1477976739 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:50.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.950+0000 7f9023fff700 1 --2- 172.21.15.29:0/1477976739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9024150970 0x7f9024150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.950+0000 7f9022ffd700 1 -- 172.21.15.29:0/1477976739 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241536430 0 0) 0x7f9024069a50 con 0x7f9024155060 2024-01-24T13:59:50.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.950+0000 7f9022ffd700 1 -- 172.21.15.29:0/1477976739 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f900c0033f0 con 0x7f9024155060 2024-01-24T13:59:50.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.950+0000 7f9022ffd700 1 -- 172.21.15.29:0/1477976739 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498468008 0 0) 0x7f90240c2b20 con 0x7f9024069ee0 2024-01-24T13:59:50.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.950+0000 7f9022ffd700 1 -- 172.21.15.29:0/1477976739 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9024069a50 con 0x7f9024069ee0 2024-01-24T13:59:50.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.950+0000 7f9023fff700 1 -- 172.21.15.29:0/1477976739 >> v1:172.21.15.163:6789/0 conn(0x7f9024069ee0 legacy=0x7f9024154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.950+0000 7f9023fff700 1 -- 172.21.15.29:0/1477976739 >> v1:172.21.15.29:6789/0 conn(0x7f9024155060 legacy=0x7f90240c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.950+0000 7f9023fff700 1 -- 172.21.15.29:0/1477976739 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9024155510 con 0x7f9024150970 2024-01-24T13:59:50.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.950+0000 7f9023fff700 1 --2- 172.21.15.29:0/1477976739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9024150970 0x7f9024150d50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f9014009790 tx=0x7f9014009060).ready entity=mon.2 client_cookie=b24d045c73815c52 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:50.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.951+0000 7f9022ffd700 1 -- 172.21.15.29:0/1477976739 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901400f510 con 0x7f9024150970 2024-01-24T13:59:50.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.951+0000 7f9022ffd700 1 -- 172.21.15.29:0/1477976739 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901400f670 con 0x7f9024150970 2024-01-24T13:59:50.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.951+0000 7f9022ffd700 1 -- 172.21.15.29:0/1477976739 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901400f9f0 con 0x7f9024150970 2024-01-24T13:59:50.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.951+0000 7f902a266700 1 -- 172.21.15.29:0/1477976739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9024150970 msgr2=0x7f9024150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.951+0000 7f902a266700 1 --2- 172.21.15.29:0/1477976739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9024150970 0x7f9024150d50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f9014009790 tx=0x7f9014009060).stop 2024-01-24T13:59:50.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.951+0000 7f902a266700 1 -- 172.21.15.29:0/1477976739 shutdown_connections 2024-01-24T13:59:50.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.951+0000 7f902a266700 1 --2- 172.21.15.29:0/1477976739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9024150970 0x7f9024150d50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.951+0000 7f902a266700 1 -- 172.21.15.29:0/1477976739 >> 172.21.15.29:0/1477976739 conn(0x7f9024067b20 msgr2=0x7f90240638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:50.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.951+0000 7f902a266700 1 -- 172.21.15.29:0/1477976739 shutdown_connections 2024-01-24T13:59:50.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.951+0000 7f902a266700 1 -- 172.21.15.29:0/1477976739 wait complete. 2024-01-24T13:59:50.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.953+0000 7f902a266700 1 Processor -- start 2024-01-24T13:59:50.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.953+0000 7f902a266700 1 -- start start 2024-01-24T13:59:50.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f902a266700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9024069ee0 0x7f9024146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f902a266700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9024150970 0x7f90240ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f902a266700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9024155060 0x7f90241647d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f902a266700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90240a5a10 con 0x7f9024150970 2024-01-24T13:59:50.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f902a266700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90240a4e00 con 0x7f9024069ee0 2024-01-24T13:59:50.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f902a266700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90240a50e0 con 0x7f9024155060 2024-01-24T13:59:50.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f90237fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9024150970 0x7f90240ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f90237fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9024150970 0x7f90240ad3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51212/0 (socket says 172.21.15.29:51212) 2024-01-24T13:59:50.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f90237fe700 1 -- 172.21.15.29:0/3821709746 learned_addr learned my addr 172.21.15.29:0/3821709746 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f9023fff700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9024069ee0 0x7f9024146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f9028803700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9024155060 0x7f90241647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f90237fe700 1 -- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9024155060 msgr2=0x7f90241647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f90237fe700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9024155060 0x7f90241647d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f90237fe700 1 -- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9024069ee0 msgr2=0x7f9024146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f90237fe700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9024069ee0 0x7f9024146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f90237fe700 1 -- 172.21.15.29:0/3821709746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90140092a0 con 0x7f9024150970 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f90237fe700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9024150970 0x7f90240ad3d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f901800c420 tx=0x7f9018007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.954+0000 7f900bfff700 1 -- 172.21.15.29:0/3821709746 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018012400 con 0x7f9024150970 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.955+0000 7f900bfff700 1 -- 172.21.15.29:0/3821709746 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9018012560 con 0x7f9024150970 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.955+0000 7f902a266700 1 -- 172.21.15.29:0/3821709746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90240a53c0 con 0x7f9024150970 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.955+0000 7f900bfff700 1 -- 172.21.15.29:0/3821709746 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018012920 con 0x7f9024150970 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.955+0000 7f902a266700 1 -- 172.21.15.29:0/3821709746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90241473e0 con 0x7f9024150970 2024-01-24T13:59:50.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.956+0000 7f902a266700 1 -- 172.21.15.29:0/3821709746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90240a5bc0 con 0x7f9024150970 2024-01-24T13:59:50.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.956+0000 7f900bfff700 1 -- 172.21.15.29:0/3821709746 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f901801b3b0 con 0x7f9024150970 2024-01-24T13:59:50.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.956+0000 7f900bfff700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f901005f350 0x7f9010061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:50.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.956+0000 7f900bfff700 1 -- 172.21.15.29:0/3821709746 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f9018079e50 con 0x7f9024150970 2024-01-24T13:59:50.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.957+0000 7f9023fff700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f901005f350 0x7f9010061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:50.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.957+0000 7f9023fff700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f901005f350 0x7f9010061810 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9014009fd0 tx=0x7f9014017000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:50.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:50.961+0000 7f900bfff700 1 -- 172.21.15.29:0/3821709746 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f90240a5bc0 con 0x7f9024150970 2024-01-24T13:59:51.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.169+0000 7f902a266700 1 -- 172.21.15.29:0/3821709746 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f90240475e0 con 0x7f901005f350 2024-01-24T13:59:51.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.174+0000 7f900bfff700 1 -- 172.21.15.29:0/3821709746 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f90240475e0 con 0x7f901005f350 2024-01-24T13:59:51.174 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule activated for path /snap_test_dir1/dir5 2024-01-24T13:59:51.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.176+0000 7f902a266700 1 -- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f901005f350 msgr2=0x7f9010061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.176+0000 7f902a266700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f901005f350 0x7f9010061810 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9014009fd0 tx=0x7f9014017000).stop 2024-01-24T13:59:51.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.176+0000 7f902a266700 1 -- 172.21.15.29:0/3821709746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9024150970 msgr2=0x7f90240ad3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.176+0000 7f902a266700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9024150970 0x7f90240ad3d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f901800c420 tx=0x7f9018007c60).stop 2024-01-24T13:59:51.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.176+0000 7f902a266700 1 -- 172.21.15.29:0/3821709746 shutdown_connections 2024-01-24T13:59:51.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.176+0000 7f902a266700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9024155060 0x7f90241647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.176+0000 7f902a266700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9024069ee0 0x7f9024146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.176+0000 7f902a266700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f901005f350 0x7f9010061810 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.176+0000 7f902a266700 1 --2- 172.21.15.29:0/3821709746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9024150970 0x7f90240ad3d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.176+0000 7f902a266700 1 -- 172.21.15.29:0/3821709746 >> 172.21.15.29:0/3821709746 conn(0x7f9024067b20 msgr2=0x7f90240627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:51.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.176+0000 7f902a266700 1 -- 172.21.15.29:0/3821709746 shutdown_connections 2024-01-24T13:59:51.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.176+0000 7f902a266700 1 -- 172.21.15.29:0/3821709746 wait complete. 2024-01-24T13:59:51.190 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2024-01-24T13:59:51.190 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir6 1M 2024-01-24T13:59:51.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.324+0000 7f4c9eb41700 1 Processor -- start 2024-01-24T13:59:51.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.324+0000 7f4c9eb41700 1 -- start start 2024-01-24T13:59:51.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.325+0000 7f4c9eb41700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4c9813a6f0 0x7f4c98068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.325+0000 7f4c9eb41700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c980638c0 0x7f4c98066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.325+0000 7f4c9eb41700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c980643f0 con 0x7f4c98066940 2024-01-24T13:59:51.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.325+0000 7f4c9eb41700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c980bdb30 con 0x7f4c9813a6f0 2024-01-24T13:59:51.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.325+0000 7f4c9eb41700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980be3d0 con 0x7f4c980638c0 2024-01-24T13:59:51.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.325+0000 7f4c9c8dd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4c9813a6f0 0x7f4c98068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.325+0000 7f4c9d0de700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4c98066940 0x7f4c980be860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33332/0 (socket says 172.21.15.29:33332) 2024-01-24T13:59:51.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.325+0000 7f4c9d0de700 1 -- 172.21.15.29:0/1919096581 learned_addr learned my addr 172.21.15.29:0/1919096581 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:51.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c97fff700 1 --2- 172.21.15.29:0/1919096581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c980638c0 0x7f4c98066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c9c8dd700 1 -- 172.21.15.29:0/1919096581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c980638c0 msgr2=0x7f4c98066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:51.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c9c8dd700 1 --2- 172.21.15.29:0/1919096581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c980638c0 0x7f4c98066450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c9c8dd700 1 -- 172.21.15.29:0/1919096581 >> v1:172.21.15.29:6789/0 conn(0x7f4c98066940 legacy=0x7f4c980be860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c977fe700 1 -- 172.21.15.29:0/1919096581 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319790407 0 0) 0x7f4c980643f0 con 0x7f4c98066940 2024-01-24T13:59:51.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c9c8dd700 1 -- 172.21.15.29:0/1919096581 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c98066dd0 con 0x7f4c9813a6f0 2024-01-24T13:59:51.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c97fff700 1 --2- 172.21.15.29:0/1919096581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c980638c0 0x7f4c98066450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:51.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c9c8dd700 1 --2- 172.21.15.29:0/1919096581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c9813a6f0 0x7f4c98068660 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f4c8c009760 tx=0x7f4c8c009060).ready entity=mon.0 client_cookie=c0b4deb5c242971f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:51.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c977fe700 1 -- 172.21.15.29:0/1919096581 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00b580 con 0x7f4c9813a6f0 2024-01-24T13:59:51.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c977fe700 1 -- 172.21.15.29:0/1919096581 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00b6e0 con 0x7f4c9813a6f0 2024-01-24T13:59:51.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c977fe700 1 -- 172.21.15.29:0/1919096581 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00ba60 con 0x7f4c9813a6f0 2024-01-24T13:59:51.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1919096581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c9813a6f0 msgr2=0x7f4c98068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.326+0000 7f4c9eb41700 1 --2- 172.21.15.29:0/1919096581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c9813a6f0 0x7f4c98068660 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f4c8c009760 tx=0x7f4c8c009060).stop 2024-01-24T13:59:51.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.327+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1919096581 shutdown_connections 2024-01-24T13:59:51.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.327+0000 7f4c9eb41700 1 --2- 172.21.15.29:0/1919096581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c9813a6f0 0x7f4c98068660 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.327+0000 7f4c9eb41700 1 --2- 172.21.15.29:0/1919096581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c980638c0 0x7f4c98066450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.327+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1919096581 >> 172.21.15.29:0/1919096581 conn(0x7f4c980c8bd0 msgr2=0x7f4c980c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:51.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.327+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1919096581 shutdown_connections 2024-01-24T13:59:51.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.327+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1919096581 wait complete. 2024-01-24T13:59:51.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.328+0000 7f4c9eb41700 1 Processor -- start 2024-01-24T13:59:51.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.328+0000 7f4c9eb41700 1 -- start start 2024-01-24T13:59:51.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9eb41700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c980638c0 0x7f4c981530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9eb41700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98066940 0x7f4c980aac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9eb41700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c9813a6f0 0x7f4c980c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9eb41700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c98165f10 con 0x7f4c980638c0 2024-01-24T13:59:51.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9eb41700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c98165300 con 0x7f4c9813a6f0 2024-01-24T13:59:51.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9eb41700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c981655e0 con 0x7f4c98066940 2024-01-24T13:59:51.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9c8dd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c980638c0 0x7f4c981530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9c8dd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c980638c0 0x7f4c981530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51222/0 (socket says 172.21.15.29:51222) 2024-01-24T13:59:51.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9c8dd700 1 -- 172.21.15.29:0/1785454793 learned_addr learned my addr 172.21.15.29:0/1785454793 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:51.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c97fff700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98066940 0x7f4c980aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9d0de700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c9813a6f0 0x7f4c980c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9c8dd700 1 -- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98066940 msgr2=0x7f4c980aac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9c8dd700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98066940 0x7f4c980aac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9c8dd700 1 -- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c9813a6f0 msgr2=0x7f4c980c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9c8dd700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c9813a6f0 0x7f4c980c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9c8dd700 1 -- 172.21.15.29:0/1785454793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c0092a0 con 0x7f4c980638c0 2024-01-24T13:59:51.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c97fff700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98066940 0x7f4c980aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:51.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.329+0000 7f4c9d0de700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c9813a6f0 0x7f4c980c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:51.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.330+0000 7f4c9c8dd700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c980638c0 0x7f4c981530c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f4c8c009eb0 tx=0x7f4c8c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:51.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.330+0000 7f4c94ff9700 1 -- 172.21.15.29:0/1785454793 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00b850 con 0x7f4c980638c0 2024-01-24T13:59:51.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.330+0000 7f4c94ff9700 1 -- 172.21.15.29:0/1785454793 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00b9b0 con 0x7f4c980638c0 2024-01-24T13:59:51.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.330+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1785454793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c981658c0 con 0x7f4c980638c0 2024-01-24T13:59:51.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.330+0000 7f4c94ff9700 1 -- 172.21.15.29:0/1785454793 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c01b690 con 0x7f4c980638c0 2024-01-24T13:59:51.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.330+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1785454793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c980abb60 con 0x7f4c980638c0 2024-01-24T13:59:51.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.331+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1785454793 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c981661f0 con 0x7f4c980638c0 2024-01-24T13:59:51.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.331+0000 7f4c94ff9700 1 -- 172.21.15.29:0/1785454793 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f4c8c01ba80 con 0x7f4c980638c0 2024-01-24T13:59:51.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.332+0000 7f4c94ff9700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c7c05f350 0x7f4c7c061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.332+0000 7f4c94ff9700 1 -- 172.21.15.29:0/1785454793 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f4c8c07bdb0 con 0x7f4c980638c0 2024-01-24T13:59:51.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.332+0000 7f4c97fff700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c7c05f350 0x7f4c7c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.332+0000 7f4c97fff700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c7c05f350 0x7f4c7c061810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4c980c2830 tx=0x7f4c84008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:51.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.336+0000 7f4c94ff9700 1 -- 172.21.15.29:0/1785454793 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4c8c04cc70 con 0x7f4c980638c0 2024-01-24T13:59:51.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.543+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1785454793 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c98166050 con 0x7f4c7c05f350 2024-01-24T13:59:51.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.548+0000 7f4c94ff9700 1 -- 172.21.15.29:0/1785454793 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4c8400b7e0 con 0x7f4c7c05f350 2024-01-24T13:59:51.548 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule activated for path /snap_test_dir1/dir6 2024-01-24T13:59:51.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.549+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c7c05f350 msgr2=0x7f4c7c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.549+0000 7f4c9eb41700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c7c05f350 0x7f4c7c061810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4c980c2830 tx=0x7f4c84008040).stop 2024-01-24T13:59:51.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.549+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1785454793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c980638c0 msgr2=0x7f4c981530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.549+0000 7f4c9eb41700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c980638c0 0x7f4c981530c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f4c8c009eb0 tx=0x7f4c8c017000).stop 2024-01-24T13:59:51.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.550+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1785454793 shutdown_connections 2024-01-24T13:59:51.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.550+0000 7f4c9eb41700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c9813a6f0 0x7f4c980c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.550+0000 7f4c9eb41700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c7c05f350 0x7f4c7c061810 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.550+0000 7f4c9eb41700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c980638c0 0x7f4c981530c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.550+0000 7f4c9eb41700 1 --2- 172.21.15.29:0/1785454793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98066940 0x7f4c980aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.550+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1785454793 >> 172.21.15.29:0/1785454793 conn(0x7f4c980c8bd0 msgr2=0x7f4c980c47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:51.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.550+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1785454793 shutdown_connections 2024-01-24T13:59:51.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.550+0000 7f4c9eb41700 1 -- 172.21.15.29:0/1785454793 wait complete. 2024-01-24T13:59:51.563 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2024-01-24T13:59:51.564 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir7 1M 2024-01-24T13:59:51.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.699+0000 7fc4b898e700 1 Processor -- start 2024-01-24T13:59:51.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.699+0000 7fc4b898e700 1 -- start start 2024-01-24T13:59:51.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.700+0000 7fc4b898e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4b4150d50 0x7fc4b4154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.700+0000 7fc4b898e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b41552d0 con 0x7fc4b4154e40 2024-01-24T13:59:51.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.700+0000 7fc4b898e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b4069a50 con 0x7fc4b4150970 2024-01-24T13:59:51.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b898e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b406a5f0 con 0x7fc4b4150d50 2024-01-24T13:59:51.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b2d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc4b4154e40 0x7fc4b40c4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33338/0 (socket says 172.21.15.29:33338) 2024-01-24T13:59:51.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b2d9d700 1 -- 172.21.15.29:0/3934956066 learned_addr learned my addr 172.21.15.29:0/3934956066 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b1d9b700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4b4150d50 0x7fc4b4154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219734728 0 0) 0x7fc4ac001490 con 0x7fc4b4154e40 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a00033f0 con 0x7fc4b4154e40 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3088603325 0 0) 0x7fc4a00033f0 con 0x7fc4b4154e40 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4b4150d50 msgr2=0x7fc4b4154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4b4150d50 0x7fc4b4154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 >> v1:172.21.15.163:6789/0 conn(0x7fc4b4150970 legacy=0x7fc4b4069ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b40c2b20 con 0x7fc4b4154e40 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fc4ac001980 con 0x7fc4b4154e40 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 >> v1:172.21.15.29:6789/0 conn(0x7fc4b4154e40 legacy=0x7fc4b40c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4b4150d50 0x7fc4a0003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4a0003e00 0x7fc4a0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a0004b80 0x7fc4a0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b406a5f0 con 0x7fc4a0004b80 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a000b8e0 con 0x7fc4a0003e00 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.701+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a000baa0 con 0x7fc4b4150d50 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b1d9b700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4a0003e00 0x7fc4a0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b2d9d700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a0004b80 0x7fc4a0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b259c700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4b4150d50 0x7fc4a0003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b2d9d700 1 -- 172.21.15.29:0/3934956066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4b4150d50 msgr2=0x7fc4a0003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b2d9d700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4b4150d50 0x7fc4a0003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b2d9d700 1 -- 172.21.15.29:0/3934956066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4a0003e00 msgr2=0x7fc4a0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b2d9d700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4a0003e00 0x7fc4a0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b2d9d700 1 -- 172.21.15.29:0/3934956066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a00033f0 con 0x7fc4a0004b80 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b1d9b700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4a0003e00 0x7fc4a0004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b2d9d700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a0004b80 0x7fc4a0005000 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fc4ac00c690 tx=0x7fc4b406a730).ready entity=mon.0 client_cookie=2daabba59160a07b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac005b20 con 0x7fc4a0004b80 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac005c80 con 0x7fc4a0004b80 2024-01-24T13:59:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.702+0000 7fc4b159a700 1 -- 172.21.15.29:0/3934956066 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac007110 con 0x7fc4a0004b80 2024-01-24T13:59:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.703+0000 7fc4b898e700 1 -- 172.21.15.29:0/3934956066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a0004b80 msgr2=0x7fc4a0005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.703+0000 7fc4b898e700 1 --2- 172.21.15.29:0/3934956066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4a0004b80 0x7fc4a0005000 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fc4ac00c690 tx=0x7fc4b406a730).stop 2024-01-24T13:59:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.703+0000 7fc4b2d9d700 1 -- 172.21.15.29:0/3934956066 reap_dead start 2024-01-24T13:59:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.703+0000 7fc4b898e700 1 -- 172.21.15.29:0/3934956066 shutdown_connections 2024-01-24T13:59:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.703+0000 7fc4b898e700 1 -- 172.21.15.29:0/3934956066 >> 172.21.15.29:0/3934956066 conn(0x7fc4b4067b20 msgr2=0x7fc4b40638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.703+0000 7fc4b898e700 1 -- 172.21.15.29:0/3934956066 shutdown_connections 2024-01-24T13:59:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.703+0000 7fc4b898e700 1 -- 172.21.15.29:0/3934956066 wait complete. 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.704+0000 7fc4b898e700 1 Processor -- start 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.704+0000 7fc4b898e700 1 -- start start 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b898e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4a0004b80 0x7fc4b40a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b898e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4143020 0x7fc4b4153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b898e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4b4153e00 0x7fc4b4162640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b898e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b41434a0 con 0x7fc4b4143020 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b898e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b4154280 con 0x7fc4b4153e00 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b898e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b40a0520 con 0x7fc4a0004b80 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b1d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4143020 0x7fc4b4153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b1d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4143020 0x7fc4b4153910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51244/0 (socket says 172.21.15.29:51244) 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b1d9b700 1 -- 172.21.15.29:0/1062480257 learned_addr learned my addr 172.21.15.29:0/1062480257 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b259c700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4a0004b80 0x7fc4b40a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b2d9d700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4b4153e00 0x7fc4b4162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b1d9b700 1 -- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4a0004b80 msgr2=0x7fc4b40a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b1d9b700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4a0004b80 0x7fc4b40a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b1d9b700 1 -- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4b4153e00 msgr2=0x7fc4b4162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b1d9b700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4b4153e00 0x7fc4b4162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.705+0000 7fc4b1d9b700 1 -- 172.21.15.29:0/1062480257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b40a07c0 con 0x7fc4b4143020 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.706+0000 7fc4b1d9b700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4143020 0x7fc4b4153910 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fc49c011b40 tx=0x7fc49c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.706+0000 7fc4aa7fc700 1 -- 172.21.15.29:0/1062480257 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00f940 con 0x7fc4b4143020 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.706+0000 7fc4aa7fc700 1 -- 172.21.15.29:0/1062480257 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c00faa0 con 0x7fc4b4143020 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.706+0000 7fc4aa7fc700 1 -- 172.21.15.29:0/1062480257 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00fe20 con 0x7fc4b4143020 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.706+0000 7fc4b898e700 1 -- 172.21.15.29:0/1062480257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b40a0aa0 con 0x7fc4b4143020 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.706+0000 7fc4b898e700 1 -- 172.21.15.29:0/1062480257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b41633b0 con 0x7fc4b4143020 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.707+0000 7fc4b898e700 1 -- 172.21.15.29:0/1062480257 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b40a10f0 con 0x7fc4b4143020 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.707+0000 7fc4aa7fc700 1 -- 172.21.15.29:0/1062480257 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7fc49c014650 con 0x7fc4b4143020 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.708+0000 7fc4aa7fc700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc49405f350 0x7fc494061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:51.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.708+0000 7fc4aa7fc700 1 -- 172.21.15.29:0/1062480257 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7fc49c07b0c0 con 0x7fc4b4143020 2024-01-24T13:59:51.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.708+0000 7fc4b259c700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc49405f350 0x7fc494061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:51.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.708+0000 7fc4b259c700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc49405f350 0x7fc494061810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fc4b40a74c0 tx=0x7fc4a4007b90).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:51.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.712+0000 7fc4aa7fc700 1 -- 172.21.15.29:0/1062480257 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc49c04bf80 con 0x7fc4b4143020 2024-01-24T13:59:51.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.921+0000 7fc4b898e700 1 -- 172.21.15.29:0/1062480257 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4b40475e0 con 0x7fc49405f350 2024-01-24T13:59:51.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.926+0000 7fc4aa7fc700 1 -- 172.21.15.29:0/1062480257 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc4b40475e0 con 0x7fc49405f350 2024-01-24T13:59:51.926 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule activated for path /snap_test_dir1/dir7 2024-01-24T13:59:51.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.928+0000 7fc4b898e700 1 -- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc49405f350 msgr2=0x7fc494061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.928+0000 7fc4b898e700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc49405f350 0x7fc494061810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fc4b40a74c0 tx=0x7fc4a4007b90).stop 2024-01-24T13:59:51.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.928+0000 7fc4b898e700 1 -- 172.21.15.29:0/1062480257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4143020 msgr2=0x7fc4b4153910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:51.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.928+0000 7fc4b898e700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4143020 0x7fc4b4153910 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fc49c011b40 tx=0x7fc49c009060).stop 2024-01-24T13:59:51.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.928+0000 7fc4b898e700 1 -- 172.21.15.29:0/1062480257 shutdown_connections 2024-01-24T13:59:51.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.928+0000 7fc4b898e700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4b4153e00 0x7fc4b4162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.928+0000 7fc4b898e700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4a0004b80 0x7fc4b40a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.928+0000 7fc4b898e700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc49405f350 0x7fc494061810 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.928+0000 7fc4b898e700 1 --2- 172.21.15.29:0/1062480257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4143020 0x7fc4b4153910 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:51.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.928+0000 7fc4b898e700 1 -- 172.21.15.29:0/1062480257 >> 172.21.15.29:0/1062480257 conn(0x7fc4b4067b20 msgr2=0x7fc4b40c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:51.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.928+0000 7fc4b898e700 1 -- 172.21.15.29:0/1062480257 shutdown_connections 2024-01-24T13:59:51.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:51.928+0000 7fc4b898e700 1 -- 172.21.15.29:0/1062480257 wait complete. 2024-01-24T13:59:51.942 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2024-01-24T13:59:51.942 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir8 1M 2024-01-24T13:59:52.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.076+0000 7fe5f94ce700 1 Processor -- start 2024-01-24T13:59:52.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.077+0000 7fe5f94ce700 1 -- start start 2024-01-24T13:59:52.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.077+0000 7fe5f94ce700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe5f4062ec0 0x7fe5f4155030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.077+0000 7fe5f94ce700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f40c81e0 con 0x7fe5f414e540 2024-01-24T13:59:52.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.077+0000 7fe5f94ce700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f4154620 con 0x7fe5f406a620 2024-01-24T13:59:52.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.077+0000 7fe5f94ce700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f40c89f0 con 0x7fe5f4062ec0 2024-01-24T13:59:52.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f2ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe5f414e540 0x7fe5f40627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33344/0 (socket says 172.21.15.29:33344) 2024-01-24T13:59:52.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f2ffd700 1 -- 172.21.15.29:0/354708417 learned_addr learned my addr 172.21.15.29:0/354708417 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:52.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f27fc700 1 --2- 172.21.15.29:0/354708417 >> v2:172.21.15.163:3300/0 conn(0x7fe5f4062ec0 0x7fe5f4155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015272891 0 0) 0x7fe5f40c81e0 con 0x7fe5f414e540 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5dc0033f0 con 0x7fe5f414e540 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750250112 0 0) 0x7fe5f4154620 con 0x7fe5f406a620 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5f40c81e0 con 0x7fe5f406a620 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2960060788 0 0) 0x7fe5dc0033f0 con 0x7fe5f414e540 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 >> v2:172.21.15.163:3300/0 conn(0x7fe5f4062ec0 msgr2=0x7fe5f4155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 --2- 172.21.15.29:0/354708417 >> v2:172.21.15.163:3300/0 conn(0x7fe5f4062ec0 0x7fe5f4155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 >> v1:172.21.15.163:6789/0 conn(0x7fe5f406a620 legacy=0x7fe5f4069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f4154ba0 con 0x7fe5f414e540 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fe5e8001470 con 0x7fe5f414e540 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 >> v1:172.21.15.29:6789/0 conn(0x7fe5f414e540 legacy=0x7fe5f40627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 --2- 172.21.15.29:0/354708417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5dc003a70 0x7fe5dc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 --2- 172.21.15.29:0/354708417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5dc004690 0x7fe5dc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 --2- 172.21.15.29:0/354708417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5dc005000 0x7fe5dc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40c89f0 con 0x7fe5dc004690 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc00bf60 con 0x7fe5dc005000 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.078+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc00c0d0 con 0x7fe5dc003a70 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f27fc700 1 --2- 172.21.15.29:0/354708417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5dc004690 0x7fe5dc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f2ffd700 1 --2- 172.21.15.29:0/354708417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5dc003a70 0x7fe5dc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f37fe700 1 --2- 172.21.15.29:0/354708417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5dc005000 0x7fe5dc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f27fc700 1 -- 172.21.15.29:0/354708417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5dc003a70 msgr2=0x7fe5dc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f27fc700 1 --2- 172.21.15.29:0/354708417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5dc003a70 0x7fe5dc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f27fc700 1 -- 172.21.15.29:0/354708417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5dc005000 msgr2=0x7fe5dc00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f27fc700 1 --2- 172.21.15.29:0/354708417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5dc005000 0x7fe5dc00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f27fc700 1 -- 172.21.15.29:0/354708417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc0033f0 con 0x7fe5dc004690 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f37fe700 1 -- 172.21.15.29:0/354708417 reap_dead start 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f27fc700 1 --2- 172.21.15.29:0/354708417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5dc004690 0x7fe5dc004b10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fe5f40632c0 tx=0x7fe5e4009060).ready entity=mon.0 client_cookie=652c0f16ca769006 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:52.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4009e90 con 0x7fe5dc004690 2024-01-24T13:59:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e4009430 con 0x7fe5dc004690 2024-01-24T13:59:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.079+0000 7fe5f1ffb700 1 -- 172.21.15.29:0/354708417 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e400e850 con 0x7fe5dc004690 2024-01-24T13:59:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.080+0000 7fe5f94ce700 1 -- 172.21.15.29:0/354708417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5dc004690 msgr2=0x7fe5dc004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.080+0000 7fe5f94ce700 1 --2- 172.21.15.29:0/354708417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5dc004690 0x7fe5dc004b10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fe5f40632c0 tx=0x7fe5e4009060).stop 2024-01-24T13:59:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.080+0000 7fe5f94ce700 1 -- 172.21.15.29:0/354708417 shutdown_connections 2024-01-24T13:59:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.080+0000 7fe5f94ce700 1 --2- 172.21.15.29:0/354708417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5dc004690 0x7fe5dc004b10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.080+0000 7fe5f94ce700 1 -- 172.21.15.29:0/354708417 >> 172.21.15.29:0/354708417 conn(0x7fe5f4064390 msgr2=0x7fe5f40c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.080+0000 7fe5f94ce700 1 -- 172.21.15.29:0/354708417 shutdown_connections 2024-01-24T13:59:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.080+0000 7fe5f94ce700 1 -- 172.21.15.29:0/354708417 wait complete. 2024-01-24T13:59:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.081+0000 7fe5f94ce700 1 Processor -- start 2024-01-24T13:59:52.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.081+0000 7fe5f94ce700 1 -- start start 2024-01-24T13:59:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f94ce700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5dc004690 0x7fe5f414eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f94ce700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5f414f330 0x7fe5f4150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f94ce700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5f4151310 0x7fe5f4151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f94ce700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40c0a60 con 0x7fe5f414f330 2024-01-24T13:59:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f94ce700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40c0d20 con 0x7fe5f4151310 2024-01-24T13:59:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f94ce700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40c1000 con 0x7fe5dc004690 2024-01-24T13:59:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f27fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5f414f330 0x7fe5f4150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f27fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5f414f330 0x7fe5f4150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51268/0 (socket says 172.21.15.29:51268) 2024-01-24T13:59:52.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f27fc700 1 -- 172.21.15.29:0/2891370379 learned_addr learned my addr 172.21.15.29:0/2891370379 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f37fe700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5f4151310 0x7fe5f4151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f2ffd700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5dc004690 0x7fe5f414eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f27fc700 1 -- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5dc004690 msgr2=0x7fe5f414eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f27fc700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5dc004690 0x7fe5f414eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f27fc700 1 -- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5f4151310 msgr2=0x7fe5f4151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f27fc700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5f4151310 0x7fe5f4151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f27fc700 1 -- 172.21.15.29:0/2891370379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f40c12e0 con 0x7fe5f414f330 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.082+0000 7fe5f2ffd700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5dc004690 0x7fe5f414eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.083+0000 7fe5f27fc700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5f414f330 0x7fe5f4150e20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fe5e400e4b0 tx=0x7fe5e401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.083+0000 7fe5daffd700 1 -- 172.21.15.29:0/2891370379 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e401c9f0 con 0x7fe5f414f330 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.083+0000 7fe5daffd700 1 -- 172.21.15.29:0/2891370379 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e401cb50 con 0x7fe5f414f330 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.083+0000 7fe5f94ce700 1 -- 172.21.15.29:0/2891370379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f40c15c0 con 0x7fe5f414f330 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.083+0000 7fe5daffd700 1 -- 172.21.15.29:0/2891370379 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40243f0 con 0x7fe5f414f330 2024-01-24T13:59:52.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.083+0000 7fe5f94ce700 1 -- 172.21.15.29:0/2891370379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f4151c60 con 0x7fe5f414f330 2024-01-24T13:59:52.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.084+0000 7fe5f94ce700 1 -- 172.21.15.29:0/2891370379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f4152fc0 con 0x7fe5f414f330 2024-01-24T13:59:52.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.084+0000 7fe5daffd700 1 -- 172.21.15.29:0/2891370379 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7fe5e40123f0 con 0x7fe5f414f330 2024-01-24T13:59:52.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.085+0000 7fe5daffd700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe5e005f350 0x7fe5e0061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.085+0000 7fe5daffd700 1 -- 172.21.15.29:0/2891370379 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7fe5e407bb70 con 0x7fe5f414f330 2024-01-24T13:59:52.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.085+0000 7fe5f2ffd700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe5e005f350 0x7fe5e0061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.085+0000 7fe5f2ffd700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe5e005f350 0x7fe5e0061810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fe5e8001120 tx=0x7fe5e8009040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:52.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.089+0000 7fe5daffd700 1 -- 172.21.15.29:0/2891370379 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe5e404c9d0 con 0x7fe5f414f330 2024-01-24T13:59:52.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.296+0000 7fe5f94ce700 1 -- 172.21.15.29:0/2891370379 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5f40475e0 con 0x7fe5e005f350 2024-01-24T13:59:52.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.301+0000 7fe5daffd700 1 -- 172.21.15.29:0/2891370379 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe5f40475e0 con 0x7fe5e005f350 2024-01-24T13:59:52.302 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule activated for path /snap_test_dir1/dir8 2024-01-24T13:59:52.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.304+0000 7fe5f94ce700 1 -- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe5e005f350 msgr2=0x7fe5e0061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.304+0000 7fe5f94ce700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe5e005f350 0x7fe5e0061810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fe5e8001120 tx=0x7fe5e8009040).stop 2024-01-24T13:59:52.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.304+0000 7fe5f94ce700 1 -- 172.21.15.29:0/2891370379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5f414f330 msgr2=0x7fe5f4150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.304+0000 7fe5f94ce700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5f414f330 0x7fe5f4150e20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fe5e400e4b0 tx=0x7fe5e401a000).stop 2024-01-24T13:59:52.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.304+0000 7fe5f94ce700 1 -- 172.21.15.29:0/2891370379 shutdown_connections 2024-01-24T13:59:52.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.304+0000 7fe5f94ce700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5f4151310 0x7fe5f4151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.304+0000 7fe5f94ce700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5dc004690 0x7fe5f414eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.304+0000 7fe5f94ce700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe5e005f350 0x7fe5e0061810 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.304+0000 7fe5f94ce700 1 --2- 172.21.15.29:0/2891370379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5f414f330 0x7fe5f4150e20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.304+0000 7fe5f94ce700 1 -- 172.21.15.29:0/2891370379 >> 172.21.15.29:0/2891370379 conn(0x7fe5f4064390 msgr2=0x7fe5f4153630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:52.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.304+0000 7fe5f94ce700 1 -- 172.21.15.29:0/2891370379 shutdown_connections 2024-01-24T13:59:52.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.304+0000 7fe5f94ce700 1 -- 172.21.15.29:0/2891370379 wait complete. 2024-01-24T13:59:52.317 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2024-01-24T13:59:52.318 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir9 1M 2024-01-24T13:59:52.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.452+0000 7f1b2f580700 1 Processor -- start 2024-01-24T13:59:52.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.452+0000 7f1b2f580700 1 -- start start 2024-01-24T13:59:52.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.453+0000 7f1b2f580700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b28150d50 0x7f1b28154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.453+0000 7f1b2f580700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b28154e40 0x7f1b280c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.453+0000 7f1b2f580700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b281552d0 con 0x7f1b28150970 2024-01-24T13:59:52.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.453+0000 7f1b2f580700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b28069a50 con 0x7f1b28150d50 2024-01-24T13:59:52.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.453+0000 7f1b2f580700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2806a5f0 con 0x7f1b28154e40 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.453+0000 7f1b2d31c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1b28150970 0x7f1b28069ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:33352/0 (socket says 172.21.15.29:33352) 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.453+0000 7f1b2d31c700 1 -- 172.21.15.29:0/1253262216 learned_addr learned my addr 172.21.15.29:0/1253262216 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b2db1d700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b28154e40 0x7f1b280c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b2cb1b700 1 --2- 172.21.15.29:0/1253262216 >> v2:172.21.15.163:3300/0 conn(0x7f1b28150d50 0x7f1b28154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192699246 0 0) 0x7f1b24001110 con 0x7f1b28150970 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b140033f0 con 0x7f1b28150970 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 792654254 0 0) 0x7f1b140033f0 con 0x7f1b28150970 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b28154e40 msgr2=0x7f1b280c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b28154e40 0x7f1b280c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 >> v2:172.21.15.163:3300/0 conn(0x7f1b28150d50 msgr2=0x7f1b28154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 --2- 172.21.15.29:0/1253262216 >> v2:172.21.15.163:3300/0 conn(0x7f1b28150d50 0x7f1b28154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b280c2b20 con 0x7f1b28150970 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f1b24001600 con 0x7f1b28150970 2024-01-24T13:59:52.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b2cb1b700 1 --2- 172.21.15.29:0/1253262216 >> v2:172.21.15.163:3300/0 conn(0x7f1b28150d50 0x7f1b28154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 >> v1:172.21.15.29:6789/0 conn(0x7f1b28150970 legacy=0x7f1b28069ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b140038e0 0x7f1b14003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b14004690 0x7f1b14004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b14005000 0x7f1b1400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28069a50 con 0x7f1b14004690 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2806a5f0 con 0x7f1b140038e0 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1400dfe0 con 0x7f1b14005000 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b2db1d700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b28154e40 0x7f1b280c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b2cb1b700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b14004690 0x7f1b14004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b2d31c700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b140038e0 0x7f1b14003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b2db1d700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b14005000 0x7f1b1400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b2cb1b700 1 -- 172.21.15.29:0/1253262216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b14005000 msgr2=0x7f1b1400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:52.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.454+0000 7f1b2cb1b700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b14005000 0x7f1b1400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.455+0000 7f1b2cb1b700 1 -- 172.21.15.29:0/1253262216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b140038e0 msgr2=0x7f1b14003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.455+0000 7f1b2cb1b700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b140038e0 0x7f1b14003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.455+0000 7f1b2cb1b700 1 -- 172.21.15.29:0/1253262216 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b140033f0 con 0x7f1b14004690 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.455+0000 7f1b2cb1b700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b14004690 0x7f1b14004b10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f1b18009c70 tx=0x7f1b18009060).ready entity=mon.0 client_cookie=7dc926624accd33 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.455+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b180095c0 con 0x7f1b14004690 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.455+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1800e6f0 con 0x7f1b14004690 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.455+0000 7f1b1ffff700 1 -- 172.21.15.29:0/1253262216 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1800ea70 con 0x7f1b14004690 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.455+0000 7f1b2f580700 1 -- 172.21.15.29:0/1253262216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b14004690 msgr2=0x7f1b14004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.455+0000 7f1b2f580700 1 --2- 172.21.15.29:0/1253262216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b14004690 0x7f1b14004b10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f1b18009c70 tx=0x7f1b18009060).stop 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.455+0000 7f1b2db1d700 1 -- 172.21.15.29:0/1253262216 reap_dead start 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.455+0000 7f1b2f580700 1 -- 172.21.15.29:0/1253262216 shutdown_connections 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.455+0000 7f1b2f580700 1 -- 172.21.15.29:0/1253262216 >> 172.21.15.29:0/1253262216 conn(0x7f1b28067b20 msgr2=0x7f1b280638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:52.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.456+0000 7f1b2f580700 1 -- 172.21.15.29:0/1253262216 shutdown_connections 2024-01-24T13:59:52.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.456+0000 7f1b2f580700 1 -- 172.21.15.29:0/1253262216 wait complete. 2024-01-24T13:59:52.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.457+0000 7f1b2f580700 1 Processor -- start 2024-01-24T13:59:52.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.457+0000 7f1b2f580700 1 -- start start 2024-01-24T13:59:52.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2f580700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b14004690 0x7f1b281643e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2f580700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b28143ca0 0x7f1b280c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2f580700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b280c7fc0 0x7f1b280c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2f580700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280a7110 con 0x7f1b14004690 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2f580700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280a6290 con 0x7f1b280c7fc0 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2f580700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280c8440 con 0x7f1b28143ca0 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2d31c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b14004690 0x7f1b281643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2d31c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b14004690 0x7f1b281643e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51292/0 (socket says 172.21.15.29:51292) 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2d31c700 1 -- 172.21.15.29:0/562999724 learned_addr learned my addr 172.21.15.29:0/562999724 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2db1d700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b280c7fc0 0x7f1b280c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2cb1b700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b28143ca0 0x7f1b280c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2d31c700 1 -- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b28143ca0 msgr2=0x7f1b280c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2d31c700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b28143ca0 0x7f1b280c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2d31c700 1 -- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b280c7fc0 msgr2=0x7f1b280c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2d31c700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b280c7fc0 0x7f1b280c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2d31c700 1 -- 172.21.15.29:0/562999724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b281648d0 con 0x7f1b14004690 2024-01-24T13:59:52.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2db1d700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b280c7fc0 0x7f1b280c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T13:59:52.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.458+0000 7f1b2d31c700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b14004690 0x7f1b281643e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f1b24004aa0 tx=0x7f1b28069b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:52.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.459+0000 7f1b1d7fa700 1 -- 172.21.15.29:0/562999724 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400f8d0 con 0x7f1b14004690 2024-01-24T13:59:52.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.459+0000 7f1b1d7fa700 1 -- 172.21.15.29:0/562999724 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2400fa30 con 0x7f1b14004690 2024-01-24T13:59:52.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.459+0000 7f1b1d7fa700 1 -- 172.21.15.29:0/562999724 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400fdd0 con 0x7f1b14004690 2024-01-24T13:59:52.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.459+0000 7f1b2f580700 1 -- 172.21.15.29:0/562999724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b280a5680 con 0x7f1b14004690 2024-01-24T13:59:52.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.459+0000 7f1b2f580700 1 -- 172.21.15.29:0/562999724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b280a5c40 con 0x7f1b14004690 2024-01-24T13:59:52.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.460+0000 7f1b2f580700 1 -- 172.21.15.29:0/562999724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b280c59e0 con 0x7f1b14004690 2024-01-24T13:59:52.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.460+0000 7f1b1d7fa700 1 -- 172.21.15.29:0/562999724 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f1b24015620 con 0x7f1b14004690 2024-01-24T13:59:52.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.460+0000 7f1b1d7fa700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b0c05f350 0x7f1b0c061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T13:59:52.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.461+0000 7f1b1d7fa700 1 -- 172.21.15.29:0/562999724 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f1b2407a970 con 0x7f1b14004690 2024-01-24T13:59:52.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.461+0000 7f1b2cb1b700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b0c05f350 0x7f1b0c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T13:59:52.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.461+0000 7f1b2cb1b700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b0c05f350 0x7f1b0c061810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f1b2814bf00 tx=0x7f1b1801a000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T13:59:52.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.465+0000 7f1b1d7fa700 1 -- 172.21.15.29:0/562999724 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1b2404b830 con 0x7f1b14004690 2024-01-24T13:59:52.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.675+0000 7f1b2f580700 1 -- 172.21.15.29:0/562999724 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b281552d0 con 0x7f1b0c05f350 2024-01-24T13:59:52.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.679+0000 7f1b1d7fa700 1 -- 172.21.15.29:0/562999724 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f1b281552d0 con 0x7f1b0c05f350 2024-01-24T13:59:52.680 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule activated for path /snap_test_dir1/dir9 2024-01-24T13:59:52.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.682+0000 7f1b2f580700 1 -- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b0c05f350 msgr2=0x7f1b0c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.682+0000 7f1b2f580700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b0c05f350 0x7f1b0c061810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f1b2814bf00 tx=0x7f1b1801a000).stop 2024-01-24T13:59:52.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.682+0000 7f1b2f580700 1 -- 172.21.15.29:0/562999724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b14004690 msgr2=0x7f1b281643e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T13:59:52.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.682+0000 7f1b2f580700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b14004690 0x7f1b281643e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f1b24004aa0 tx=0x7f1b28069b90).stop 2024-01-24T13:59:52.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.682+0000 7f1b2f580700 1 -- 172.21.15.29:0/562999724 shutdown_connections 2024-01-24T13:59:52.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.682+0000 7f1b2f580700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b280c7fc0 0x7f1b280c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.682+0000 7f1b2f580700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b0c05f350 0x7f1b0c061810 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.682+0000 7f1b2f580700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b14004690 0x7f1b281643e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.682+0000 7f1b2f580700 1 --2- 172.21.15.29:0/562999724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b28143ca0 0x7f1b280c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T13:59:52.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.682+0000 7f1b2f580700 1 -- 172.21.15.29:0/562999724 >> 172.21.15.29:0/562999724 conn(0x7f1b28067b20 msgr2=0x7f1b280ab410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T13:59:52.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.682+0000 7f1b2f580700 1 -- 172.21.15.29:0/562999724 shutdown_connections 2024-01-24T13:59:52.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T13:59:52.682+0000 7f1b2f580700 1 -- 172.21.15.29:0/562999724 wait complete. 2024-01-24T13:59:52.695 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2024-01-24T13:59:56.384 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T13:59:56.387 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:00:00.041 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:00:00.040+0000 7fb9ef1f0700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:00:00.041 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:00:00.041+0000 7fb9ef1f0700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:00:00.041 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:00:00.041+0000 7fb9ef1f0700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:00:00.042 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:00:00.041+0000 7fb9ef1f0700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:00:00.042 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:00:00.041+0000 7fb9ef1f0700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:00:26.455 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:00:26.458 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:00:56.530 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:00:56.534 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:01:26.597 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:01:26.602 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:01:56.669 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:01:56.672 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:02:26.806 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:02:26.809 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:02:56.907 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:02:56.911 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:03:26.972 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:03:26.976 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:03:57.054 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:03:57.057 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:04:27.195 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:04:27.199 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:04:57.289 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:04:57.292 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:05:27.370 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:05:27.373 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:05:57.444 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:05:57.447 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:06:27.580 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:06:27.584 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:06:57.643 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:06:57.647 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:07:27.734 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:07:27.739 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:07:57.834 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:07:57.838 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:08:27.940 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:08:27.949 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:08:58.010 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:08:58.014 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:09:28.089 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:09:28.093 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:09:58.167 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:09:58.170 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:10:19.697 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir0 1M 2024-01-24T14:10:19.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.842+0000 7fabc479d700 1 Processor -- start 2024-01-24T14:10:19.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.842+0000 7fabc479d700 1 -- start start 2024-01-24T14:10:19.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.843+0000 7fabc479d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fabbc14b0a0 0x7fabbc14b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:19.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.843+0000 7fabc479d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fabbc0c3660 0x7fabbc0c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:19.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.843+0000 7fabc479d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabbc0c48e0 0x7fabbc154a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:19.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.843+0000 7fabc479d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabbc1557e0 con 0x7fabbc0c3660 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.843+0000 7fabc479d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabbc0c28a0 con 0x7fabbc14b0a0 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.843+0000 7fabc479d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0c2b80 con 0x7fabbc0c48e0 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.843+0000 7fabc1d38700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fabbc0c3660 0x7fabbc0c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.843+0000 7fabc1d38700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fabbc0c3660 0x7fabbc0c43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46084/0 (socket says 172.21.15.29:46084) 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.843+0000 7fabc1d38700 1 -- 172.21.15.29:0/1942116389 learned_addr learned my addr 172.21.15.29:0/1942116389 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.844+0000 7fabc2539700 1 --2- 172.21.15.29:0/1942116389 >> v2:172.21.15.163:3300/0 conn(0x7fabbc14b0a0 0x7fabbc14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.844+0000 7fabc2d3a700 1 --2- 172.21.15.29:0/1942116389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabbc0c48e0 0x7fabbc154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.844+0000 7fabc1d38700 1 -- 172.21.15.29:0/1942116389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabbc0c48e0 msgr2=0x7fabbc154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.844+0000 7fabc1d38700 1 --2- 172.21.15.29:0/1942116389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabbc0c48e0 0x7fabbc154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.844+0000 7fabc1d38700 1 -- 172.21.15.29:0/1942116389 >> v2:172.21.15.163:3300/0 conn(0x7fabbc14b0a0 msgr2=0x7fabbc14b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.844+0000 7fabc1d38700 1 --2- 172.21.15.29:0/1942116389 >> v2:172.21.15.163:3300/0 conn(0x7fabbc14b0a0 0x7fabbc14b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.844+0000 7fabc1d38700 1 -- 172.21.15.29:0/1942116389 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc0c2e60 con 0x7fabbc0c3660 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.844+0000 7fabc2d3a700 1 --2- 172.21.15.29:0/1942116389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabbc0c48e0 0x7fabbc154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:19.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.844+0000 7fabc1d38700 1 --2- 172.21.15.29:0/1942116389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabbc0c3660 0x7fabbc0c43f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7fabbc0685a0 tx=0x7fabb00090f0).ready entity=mon.0 client_cookie=ecd6c19bdb2a902b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:19.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.844+0000 7fabc1537700 1 -- 172.21.15.29:0/1942116389 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb000b520 con 0x7fabbc0c3660 2024-01-24T14:10:19.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.844+0000 7fabc1537700 1 -- 172.21.15.29:0/1942116389 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb000b680 con 0x7fabbc0c3660 2024-01-24T14:10:19.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.844+0000 7fabc1537700 1 -- 172.21.15.29:0/1942116389 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb000ba00 con 0x7fabbc0c3660 2024-01-24T14:10:19.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.845+0000 7fabc479d700 1 -- 172.21.15.29:0/1942116389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabbc0c3660 msgr2=0x7fabbc0c43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:19.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.845+0000 7fabc479d700 1 --2- 172.21.15.29:0/1942116389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabbc0c3660 0x7fabbc0c43f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7fabbc0685a0 tx=0x7fabb00090f0).stop 2024-01-24T14:10:19.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.845+0000 7fabc479d700 1 -- 172.21.15.29:0/1942116389 shutdown_connections 2024-01-24T14:10:19.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.845+0000 7fabc479d700 1 --2- 172.21.15.29:0/1942116389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabbc0c48e0 0x7fabbc154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:19.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.845+0000 7fabc479d700 1 --2- 172.21.15.29:0/1942116389 >> v2:172.21.15.163:3300/0 conn(0x7fabbc14b0a0 0x7fabbc14b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:19.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.845+0000 7fabc479d700 1 --2- 172.21.15.29:0/1942116389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabbc0c3660 0x7fabbc0c43f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:19.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.845+0000 7fabc479d700 1 -- 172.21.15.29:0/1942116389 >> 172.21.15.29:0/1942116389 conn(0x7fabbc067b20 msgr2=0x7fabbc0638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:19.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.845+0000 7fabc479d700 1 -- 172.21.15.29:0/1942116389 shutdown_connections 2024-01-24T14:10:19.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.845+0000 7fabc479d700 1 -- 172.21.15.29:0/1942116389 wait complete. 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.846+0000 7fabc479d700 1 Processor -- start 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.846+0000 7fabc479d700 1 -- start start 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc479d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabbc0c3660 0x7fabbc0a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc479d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabbc0c48e0 0x7fabbc0c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc479d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabbc14b0a0 0x7fabbc147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc479d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0ac610 con 0x7fabbc0c48e0 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc479d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0bfbd0 con 0x7fabbc0c3660 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc479d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0a7390 con 0x7fabbc14b0a0 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc1d38700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabbc0c48e0 0x7fabbc0c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc1d38700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabbc0c48e0 0x7fabbc0c5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46098/0 (socket says 172.21.15.29:46098) 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc1d38700 1 -- 172.21.15.29:0/2780016933 learned_addr learned my addr 172.21.15.29:0/2780016933 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc2539700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabbc0c3660 0x7fabbc0a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc2d3a700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabbc14b0a0 0x7fabbc147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:19.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc1d38700 1 -- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabbc14b0a0 msgr2=0x7fabbc147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc1d38700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabbc14b0a0 0x7fabbc147e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc1d38700 1 -- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabbc0c3660 msgr2=0x7fabbc0a6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc1d38700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabbc0c3660 0x7fabbc0a6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc1d38700 1 -- 172.21.15.29:0/2780016933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb0009370 con 0x7fabbc0c48e0 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.847+0000 7fabc2539700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabbc0c3660 0x7fabbc0a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.848+0000 7fabc1d38700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabbc0c48e0 0x7fabbc0c5ab0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fabb0009130 tx=0x7fabb0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.848+0000 7fabae7fc700 1 -- 172.21.15.29:0/2780016933 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb000be90 con 0x7fabbc0c48e0 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.848+0000 7fabae7fc700 1 -- 172.21.15.29:0/2780016933 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb000b7f0 con 0x7fabbc0c48e0 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.848+0000 7fabc479d700 1 -- 172.21.15.29:0/2780016933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabbc0bf610 con 0x7fabbc0c48e0 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.848+0000 7fabae7fc700 1 -- 172.21.15.29:0/2780016933 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb001c670 con 0x7fabbc0c48e0 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.848+0000 7fabc479d700 1 -- 172.21.15.29:0/2780016933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabbc0ac2a0 con 0x7fabbc0c48e0 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.849+0000 7fabc479d700 1 -- 172.21.15.29:0/2780016933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabbc0ac7c0 con 0x7fabbc0c48e0 2024-01-24T14:10:19.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.849+0000 7fabae7fc700 1 -- 172.21.15.29:0/2780016933 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fabb001ca60 con 0x7fabbc0c48e0 2024-01-24T14:10:19.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.850+0000 7fabae7fc700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faba005f2c0 0x7faba0061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:19.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.850+0000 7fabae7fc700 1 -- 172.21.15.29:0/2780016933 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fabb007c2f0 con 0x7fabbc0c48e0 2024-01-24T14:10:19.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.850+0000 7fabc2539700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faba005f2c0 0x7faba0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:19.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.851+0000 7fabc2539700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faba005f2c0 0x7faba0061780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fabb8006d80 tx=0x7fabb8008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:19.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:19.855+0000 7fabae7fc700 1 -- 172.21.15.29:0/2780016933 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fabbc0ac7c0 con 0x7fabbc0c48e0 2024-01-24T14:10:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.063+0000 7fabc479d700 1 -- 172.21.15.29:0/2780016933 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fabbc0475e0 con 0x7faba005f2c0 2024-01-24T14:10:20.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.068+0000 7fabae7fc700 1 -- 172.21.15.29:0/2780016933 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fabbc0475e0 con 0x7faba005f2c0 2024-01-24T14:10:20.069 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2024-01-24T14:10:20.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.070+0000 7fabc479d700 1 -- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faba005f2c0 msgr2=0x7faba0061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.070+0000 7fabc479d700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faba005f2c0 0x7faba0061780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fabb8006d80 tx=0x7fabb8008040).stop 2024-01-24T14:10:20.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.071+0000 7fabc479d700 1 -- 172.21.15.29:0/2780016933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabbc0c48e0 msgr2=0x7fabbc0c5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.071+0000 7fabc479d700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabbc0c48e0 0x7fabbc0c5ab0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fabb0009130 tx=0x7fabb0017000).stop 2024-01-24T14:10:20.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.071+0000 7fabc479d700 1 -- 172.21.15.29:0/2780016933 shutdown_connections 2024-01-24T14:10:20.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.071+0000 7fabc479d700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabbc14b0a0 0x7fabbc147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.071+0000 7fabc479d700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabbc0c3660 0x7fabbc0a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.071+0000 7fabc479d700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faba005f2c0 0x7faba0061780 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.071+0000 7fabc479d700 1 --2- 172.21.15.29:0/2780016933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabbc0c48e0 0x7fabbc0c5ab0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.071+0000 7fabc479d700 1 -- 172.21.15.29:0/2780016933 >> 172.21.15.29:0/2780016933 conn(0x7fabbc067b20 msgr2=0x7fabbc1490b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:20.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.071+0000 7fabc479d700 1 -- 172.21.15.29:0/2780016933 shutdown_connections 2024-01-24T14:10:20.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.071+0000 7fabc479d700 1 -- 172.21.15.29:0/2780016933 wait complete. 2024-01-24T14:10:20.084 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2024-01-24T14:10:20.084 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir1 1M 2024-01-24T14:10:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.218+0000 7f92d0794700 1 Processor -- start 2024-01-24T14:10:20.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.218+0000 7f92d0794700 1 -- start start 2024-01-24T14:10:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.219+0000 7f92d0794700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f92c814f8f0 0x7f92c8152490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.219+0000 7f92d0794700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92c814fcd0 0x7f92c8156190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.219+0000 7f92d0794700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c806a690 con 0x7f92c8156680 2024-01-24T14:10:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.219+0000 7f92d0794700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c8155a60 con 0x7f92c814f8f0 2024-01-24T14:10:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.219+0000 7f92d0794700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c8155d00 con 0x7f92c814fcd0 2024-01-24T14:10:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92cdd2f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92c814fcd0 0x7f92c8156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92ce530700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f92c814f8f0 0x7f92c8152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92cdd2f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92c814fcd0 0x7f92c8156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:39372/0 (socket says 172.21.15.29:39372) 2024-01-24T14:10:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92ce530700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f92c814f8f0 0x7f92c8152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:49724/0 (socket says 172.21.15.29:49724) 2024-01-24T14:10:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92cdd2f700 1 -- 172.21.15.29:0/2100939408 learned_addr learned my addr 172.21.15.29:0/2100939408 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:20.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92ced31700 1 --1- 172.21.15.29:0/2100939408 >> v1:172.21.15.163:6789/0 conn(0x7f92c8156680 0x7f92c8065410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:34352/0 (socket says 172.21.15.29:34352) 2024-01-24T14:10:20.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92ce530700 1 -- 172.21.15.29:0/2100939408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92c814fcd0 msgr2=0x7f92c8156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:20.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92ce530700 1 --2- 172.21.15.29:0/2100939408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92c814fcd0 0x7f92c8156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92ce530700 1 -- 172.21.15.29:0/2100939408 >> v1:172.21.15.163:6789/0 conn(0x7f92c8156680 legacy=0x7f92c8065410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92ce530700 1 -- 172.21.15.29:0/2100939408 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c8152cf0 con 0x7f92c814f8f0 2024-01-24T14:10:20.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92cd52e700 1 -- 172.21.15.29:0/2100939408 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820661362 0 0) 0x7f92c0001490 con 0x7f92c8156680 2024-01-24T14:10:20.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92cdd2f700 1 --2- 172.21.15.29:0/2100939408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92c814fcd0 0x7f92c8156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:20.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.220+0000 7f92ce530700 1 --2- 172.21.15.29:0/2100939408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92c814f8f0 0x7f92c8152490 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f92c4009760 tx=0x7f92c4009060).ready entity=mon.1 client_cookie=e2b69dd97b93a80f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:20.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.221+0000 7f92cd52e700 1 -- 172.21.15.29:0/2100939408 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c400f510 con 0x7f92c814f8f0 2024-01-24T14:10:20.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.221+0000 7f92cd52e700 1 -- 172.21.15.29:0/2100939408 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c400f670 con 0x7f92c814f8f0 2024-01-24T14:10:20.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.221+0000 7f92cd52e700 1 -- 172.21.15.29:0/2100939408 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c400f9f0 con 0x7f92c814f8f0 2024-01-24T14:10:20.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.221+0000 7f92d0794700 1 -- 172.21.15.29:0/2100939408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92c814f8f0 msgr2=0x7f92c8152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.221+0000 7f92d0794700 1 --2- 172.21.15.29:0/2100939408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92c814f8f0 0x7f92c8152490 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f92c4009760 tx=0x7f92c4009060).stop 2024-01-24T14:10:20.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.221+0000 7f92d0794700 1 -- 172.21.15.29:0/2100939408 shutdown_connections 2024-01-24T14:10:20.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.221+0000 7f92d0794700 1 --2- 172.21.15.29:0/2100939408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92c814f8f0 0x7f92c8152490 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.221+0000 7f92d0794700 1 --2- 172.21.15.29:0/2100939408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92c814fcd0 0x7f92c8156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.221+0000 7f92d0794700 1 -- 172.21.15.29:0/2100939408 >> 172.21.15.29:0/2100939408 conn(0x7f92c8067b20 msgr2=0x7f92c80638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:20.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.221+0000 7f92d0794700 1 -- 172.21.15.29:0/2100939408 shutdown_connections 2024-01-24T14:10:20.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.221+0000 7f92d0794700 1 -- 172.21.15.29:0/2100939408 wait complete. 2024-01-24T14:10:20.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.222+0000 7f92d0794700 1 Processor -- start 2024-01-24T14:10:20.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.223+0000 7f92d0794700 1 -- start start 2024-01-24T14:10:20.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.223+0000 7f92d0794700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92c814f8f0 0x7f92c80bb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.223+0000 7f92d0794700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92c814fcd0 0x7f92c80bbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.223+0000 7f92d0794700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92c8156680 0x7f92c8162a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.223+0000 7f92d0794700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c80c7270 con 0x7f92c814fcd0 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.223+0000 7f92d0794700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c80c7550 con 0x7f92c8156680 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.223+0000 7f92d0794700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c80c7830 con 0x7f92c814f8f0 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.223+0000 7f92cdd2f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92c814fcd0 0x7f92c80bbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.223+0000 7f92cdd2f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92c814fcd0 0x7f92c80bbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46102/0 (socket says 172.21.15.29:46102) 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.223+0000 7f92cdd2f700 1 -- 172.21.15.29:0/3780538534 learned_addr learned my addr 172.21.15.29:0/3780538534 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92ced31700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92c8156680 0x7f92c8162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92ce530700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92c814f8f0 0x7f92c80bb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92cdd2f700 1 -- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92c814f8f0 msgr2=0x7f92c80bb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92cdd2f700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92c814f8f0 0x7f92c80bb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92cdd2f700 1 -- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92c8156680 msgr2=0x7f92c8162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92cdd2f700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92c8156680 0x7f92c8162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92cdd2f700 1 -- 172.21.15.29:0/3780538534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b8009370 con 0x7f92c814fcd0 2024-01-24T14:10:20.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92cdd2f700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92c814fcd0 0x7f92c80bbbc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f92b800ec50 tx=0x7f92b80090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92be7fc700 1 -- 172.21.15.29:0/3780538534 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b800c850 con 0x7f92c814fcd0 2024-01-24T14:10:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92be7fc700 1 -- 172.21.15.29:0/3780538534 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b800c9b0 con 0x7f92c814fcd0 2024-01-24T14:10:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92d0794700 1 -- 172.21.15.29:0/3780538534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c40092a0 con 0x7f92c814fcd0 2024-01-24T14:10:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92be7fc700 1 -- 172.21.15.29:0/3780538534 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b800cd30 con 0x7f92c814fcd0 2024-01-24T14:10:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.224+0000 7f92d0794700 1 -- 172.21.15.29:0/3780538534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c80c7df0 con 0x7f92c814fcd0 2024-01-24T14:10:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.225+0000 7f92d0794700 1 -- 172.21.15.29:0/3780538534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c80475e0 con 0x7f92c814fcd0 2024-01-24T14:10:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.226+0000 7f92be7fc700 1 -- 172.21.15.29:0/3780538534 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f92b8012600 con 0x7f92c814fcd0 2024-01-24T14:10:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.226+0000 7f92be7fc700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f92ac05f290 0x7f92ac061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.226+0000 7f92be7fc700 1 -- 172.21.15.29:0/3780538534 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f92b807ac70 con 0x7f92c814fcd0 2024-01-24T14:10:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.227+0000 7f92ce530700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f92ac05f290 0x7f92ac061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.227+0000 7f92ce530700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f92ac05f290 0x7f92ac061750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f92c4009fd0 tx=0x7f92c4017000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:20.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.231+0000 7f92be7fc700 1 -- 172.21.15.29:0/3780538534 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f92b804baa0 con 0x7f92c814fcd0 2024-01-24T14:10:20.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.442+0000 7f92d0794700 1 -- 172.21.15.29:0/3780538534 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f92c80c76c0 con 0x7f92ac05f290 2024-01-24T14:10:20.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.446+0000 7f92be7fc700 1 -- 172.21.15.29:0/3780538534 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f92c80c76c0 con 0x7f92ac05f290 2024-01-24T14:10:20.447 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2024-01-24T14:10:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.448+0000 7f92d0794700 1 -- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f92ac05f290 msgr2=0x7f92ac061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.448+0000 7f92d0794700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f92ac05f290 0x7f92ac061750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f92c4009fd0 tx=0x7f92c4017000).stop 2024-01-24T14:10:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.448+0000 7f92d0794700 1 -- 172.21.15.29:0/3780538534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92c814fcd0 msgr2=0x7f92c80bbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.448+0000 7f92d0794700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92c814fcd0 0x7f92c80bbbc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f92b800ec50 tx=0x7f92b80090f0).stop 2024-01-24T14:10:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.448+0000 7f92d0794700 1 -- 172.21.15.29:0/3780538534 shutdown_connections 2024-01-24T14:10:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.448+0000 7f92d0794700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92c8156680 0x7f92c8162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.448+0000 7f92d0794700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92c814f8f0 0x7f92c80bb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.448+0000 7f92d0794700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f92ac05f290 0x7f92ac061750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.448+0000 7f92d0794700 1 --2- 172.21.15.29:0/3780538534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92c814fcd0 0x7f92c80bbbc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.448+0000 7f92d0794700 1 -- 172.21.15.29:0/3780538534 >> 172.21.15.29:0/3780538534 conn(0x7f92c8067b20 msgr2=0x7f92c814ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.449+0000 7f92d0794700 1 -- 172.21.15.29:0/3780538534 shutdown_connections 2024-01-24T14:10:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.449+0000 7f92d0794700 1 -- 172.21.15.29:0/3780538534 wait complete. 2024-01-24T14:10:20.462 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2024-01-24T14:10:20.462 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir2 1M 2024-01-24T14:10:20.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.598+0000 7fd57ed37700 1 Processor -- start 2024-01-24T14:10:20.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.598+0000 7fd57ed37700 1 -- start start 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57ed37700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd578151e50 0x7fd5780c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57ed37700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd578064e90 0x7fd578155420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57ed37700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd578154f90 con 0x7fd5780c7f70 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57ed37700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd578155910 con 0x7fd578151e50 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57ed37700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780a9f10 con 0x7fd578064e90 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57cad3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd578151e50 0x7fd5780c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57d2d4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd578064e90 0x7fd578155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57cad3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd578151e50 0x7fd5780c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:49750/0 (socket says 172.21.15.29:49750) 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57cad3700 1 -- 172.21.15.29:0/1678972658 learned_addr learned my addr 172.21.15.29:0/1678972658 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd577fff700 1 --1- 172.21.15.29:0/1678972658 >> v1:172.21.15.163:6789/0 conn(0x7fd5780c7f70 0x7fd578064780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:34362/0 (socket says 172.21.15.29:34362) 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57cad3700 1 -- 172.21.15.29:0/1678972658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd578064e90 msgr2=0x7fd578155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57cad3700 1 --2- 172.21.15.29:0/1678972658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd578064e90 0x7fd578155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57cad3700 1 -- 172.21.15.29:0/1678972658 >> v1:172.21.15.163:6789/0 conn(0x7fd5780c7f70 legacy=0x7fd578064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57cad3700 1 -- 172.21.15.29:0/1678972658 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5780aa1f0 con 0x7fd578151e50 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57d2d4700 1 --2- 172.21.15.29:0/1678972658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd578064e90 0x7fd578155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.599+0000 7fd57cad3700 1 --2- 172.21.15.29:0/1678972658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578151e50 0x7fd5780c53c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fd56c006fd0 tx=0x7fd56c0090f0).ready entity=mon.1 client_cookie=747c14e4aa90b747 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.600+0000 7fd5777fe700 1 -- 172.21.15.29:0/1678972658 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c00b580 con 0x7fd578151e50 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.600+0000 7fd5777fe700 1 -- 172.21.15.29:0/1678972658 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c00b6e0 con 0x7fd578151e50 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.600+0000 7fd5777fe700 1 -- 172.21.15.29:0/1678972658 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c00ba60 con 0x7fd578151e50 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.600+0000 7fd57ed37700 1 -- 172.21.15.29:0/1678972658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578151e50 msgr2=0x7fd5780c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.600+0000 7fd57ed37700 1 --2- 172.21.15.29:0/1678972658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578151e50 0x7fd5780c53c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fd56c006fd0 tx=0x7fd56c0090f0).stop 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.600+0000 7fd57ed37700 1 -- 172.21.15.29:0/1678972658 shutdown_connections 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.600+0000 7fd57ed37700 1 --2- 172.21.15.29:0/1678972658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd578064e90 0x7fd578155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.600+0000 7fd57ed37700 1 --2- 172.21.15.29:0/1678972658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578151e50 0x7fd5780c53c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.600+0000 7fd57ed37700 1 -- 172.21.15.29:0/1678972658 >> 172.21.15.29:0/1678972658 conn(0x7fd5780cb880 msgr2=0x7fd5780c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.600+0000 7fd57ed37700 1 -- 172.21.15.29:0/1678972658 shutdown_connections 2024-01-24T14:10:20.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.600+0000 7fd57ed37700 1 -- 172.21.15.29:0/1678972658 wait complete. 2024-01-24T14:10:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.601+0000 7fd57ed37700 1 Processor -- start 2024-01-24T14:10:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.602+0000 7fd57ed37700 1 -- start start 2024-01-24T14:10:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.602+0000 7fd57ed37700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578064e90 0x7fd5780c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.602+0000 7fd57ed37700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5780c7f70 0x7fd57814c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.602+0000 7fd57ed37700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd578151e50 0x7fd57814f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.602+0000 7fd57ed37700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57814e150 con 0x7fd5780c7f70 2024-01-24T14:10:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.602+0000 7fd57ed37700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780c3d10 con 0x7fd578064e90 2024-01-24T14:10:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.602+0000 7fd57ed37700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780aede0 con 0x7fd578151e50 2024-01-24T14:10:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd577fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5780c7f70 0x7fd57814c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd577fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5780c7f70 0x7fd57814c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46110/0 (socket says 172.21.15.29:46110) 2024-01-24T14:10:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd577fff700 1 -- 172.21.15.29:0/4065364345 learned_addr learned my addr 172.21.15.29:0/4065364345 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:20.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd57cad3700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578064e90 0x7fd5780c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd57d2d4700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd578151e50 0x7fd57814f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd577fff700 1 -- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd578151e50 msgr2=0x7fd57814f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd577fff700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd578151e50 0x7fd57814f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd577fff700 1 -- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578064e90 msgr2=0x7fd5780c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd577fff700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578064e90 0x7fd5780c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd577fff700 1 -- 172.21.15.29:0/4065364345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd570009a20 con 0x7fd5780c7f70 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd577fff700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5780c7f70 0x7fd57814c9c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fd56400c490 tx=0x7fd564007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd574ff9700 1 -- 172.21.15.29:0/4065364345 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640070d0 con 0x7fd5780c7f70 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd574ff9700 1 -- 172.21.15.29:0/4065364345 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd564007230 con 0x7fd5780c7f70 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd57ed37700 1 -- 172.21.15.29:0/4065364345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c009370 con 0x7fd5780c7f70 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd574ff9700 1 -- 172.21.15.29:0/4065364345 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640075b0 con 0x7fd5780c7f70 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.603+0000 7fd57ed37700 1 -- 172.21.15.29:0/4065364345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5780af680 con 0x7fd5780c7f70 2024-01-24T14:10:20.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.605+0000 7fd57ed37700 1 -- 172.21.15.29:0/4065364345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5780aa710 con 0x7fd5780c7f70 2024-01-24T14:10:20.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.605+0000 7fd574ff9700 1 -- 172.21.15.29:0/4065364345 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fd5640058f0 con 0x7fd5780c7f70 2024-01-24T14:10:20.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.605+0000 7fd574ff9700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd55c05f2c0 0x7fd55c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.605+0000 7fd574ff9700 1 -- 172.21.15.29:0/4065364345 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fd564079d50 con 0x7fd5780c7f70 2024-01-24T14:10:20.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.606+0000 7fd57cad3700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd55c05f2c0 0x7fd55c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.606+0000 7fd57cad3700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd55c05f2c0 0x7fd55c061780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fd56c009130 tx=0x7fd56c017000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:20.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.610+0000 7fd574ff9700 1 -- 172.21.15.29:0/4065364345 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd56404ac40 con 0x7fd5780c7f70 2024-01-24T14:10:20.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.820+0000 7fd57ed37700 1 -- 172.21.15.29:0/4065364345 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd578083310 con 0x7fd55c05f2c0 2024-01-24T14:10:20.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.825+0000 7fd574ff9700 1 -- 172.21.15.29:0/4065364345 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd578083310 con 0x7fd55c05f2c0 2024-01-24T14:10:20.826 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2024-01-24T14:10:20.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.827+0000 7fd57ed37700 1 -- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd55c05f2c0 msgr2=0x7fd55c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.827+0000 7fd57ed37700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd55c05f2c0 0x7fd55c061780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fd56c009130 tx=0x7fd56c017000).stop 2024-01-24T14:10:20.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.827+0000 7fd57ed37700 1 -- 172.21.15.29:0/4065364345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5780c7f70 msgr2=0x7fd57814c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.827+0000 7fd57ed37700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5780c7f70 0x7fd57814c9c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fd56400c490 tx=0x7fd564007df0).stop 2024-01-24T14:10:20.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.827+0000 7fd57ed37700 1 -- 172.21.15.29:0/4065364345 shutdown_connections 2024-01-24T14:10:20.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.827+0000 7fd57ed37700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd578151e50 0x7fd57814f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.827+0000 7fd57ed37700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578064e90 0x7fd5780c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.827+0000 7fd57ed37700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd55c05f2c0 0x7fd55c061780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.827+0000 7fd57ed37700 1 --2- 172.21.15.29:0/4065364345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5780c7f70 0x7fd57814c9c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.827+0000 7fd57ed37700 1 -- 172.21.15.29:0/4065364345 >> 172.21.15.29:0/4065364345 conn(0x7fd5780cb880 msgr2=0x7fd578152600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:20.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.827+0000 7fd57ed37700 1 -- 172.21.15.29:0/4065364345 shutdown_connections 2024-01-24T14:10:20.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.827+0000 7fd57ed37700 1 -- 172.21.15.29:0/4065364345 wait complete. 2024-01-24T14:10:20.841 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2024-01-24T14:10:20.842 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir3 1M 2024-01-24T14:10:20.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.977+0000 7f57af603700 1 Processor -- start 2024-01-24T14:10:20.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.977+0000 7f57af603700 1 -- start start 2024-01-24T14:10:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57af603700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8066610 0x7f57a80686b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57af603700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57a81597f0 0x7f57a8158080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57af603700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a8067c10 con 0x7f57a80669f0 2024-01-24T14:10:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57af603700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a8159c50 con 0x7f57a81597f0 2024-01-24T14:10:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57af603700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8064360 con 0x7f57a8066610 2024-01-24T14:10:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57adba0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57a81597f0 0x7f57a8158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57adba0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57a81597f0 0x7f57a8158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46120/0 (socket says 172.21.15.29:46120) 2024-01-24T14:10:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57adba0700 1 -- 172.21.15.29:0/2284787592 learned_addr learned my addr 172.21.15.29:0/2284787592 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57ad39f700 1 --2- 172.21.15.29:0/2284787592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8066610 0x7f57a80686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57adba0700 1 -- 172.21.15.29:0/2284787592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8066610 msgr2=0x7f57a80686b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57adba0700 1 --2- 172.21.15.29:0/2284787592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8066610 0x7f57a80686b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57adba0700 1 -- 172.21.15.29:0/2284787592 >> v1:172.21.15.163:6789/0 conn(0x7f57a80669f0 legacy=0x7f57a80638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.978+0000 7f57adba0700 1 -- 172.21.15.29:0/2284787592 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a8064080 con 0x7f57a81597f0 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.979+0000 7f57adba0700 1 --2- 172.21.15.29:0/2284787592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57a81597f0 0x7f57a8158080 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f57a0009f20 tx=0x7f57a0009840).ready entity=mon.0 client_cookie=6f5e31b564abf972 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.979+0000 7f579ffff700 1 -- 172.21.15.29:0/2284787592 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a000e070 con 0x7f57a81597f0 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.979+0000 7f579ffff700 1 -- 172.21.15.29:0/2284787592 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a000ba80 con 0x7f57a81597f0 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.979+0000 7f579ffff700 1 -- 172.21.15.29:0/2284787592 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a000be00 con 0x7f57a81597f0 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.979+0000 7f57af603700 1 -- 172.21.15.29:0/2284787592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57a81597f0 msgr2=0x7f57a8158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.979+0000 7f57af603700 1 --2- 172.21.15.29:0/2284787592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57a81597f0 0x7f57a8158080 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f57a0009f20 tx=0x7f57a0009840).stop 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.979+0000 7f57af603700 1 -- 172.21.15.29:0/2284787592 shutdown_connections 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.979+0000 7f57af603700 1 --2- 172.21.15.29:0/2284787592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8066610 0x7f57a80686b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.979+0000 7f57af603700 1 --2- 172.21.15.29:0/2284787592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57a81597f0 0x7f57a8158080 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.979+0000 7f57af603700 1 -- 172.21.15.29:0/2284787592 >> 172.21.15.29:0/2284787592 conn(0x7f57a80c8bd0 msgr2=0x7f57a80c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.979+0000 7f57af603700 1 -- 172.21.15.29:0/2284787592 shutdown_connections 2024-01-24T14:10:20.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.979+0000 7f57af603700 1 -- 172.21.15.29:0/2284787592 wait complete. 2024-01-24T14:10:20.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.981+0000 7f57af603700 1 Processor -- start 2024-01-24T14:10:20.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.981+0000 7f57af603700 1 -- start start 2024-01-24T14:10:20.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.982+0000 7f57af603700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57a8066610 0x7f57a8150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.982+0000 7f57af603700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a80669f0 0x7f57a815b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.982+0000 7f57af603700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57a81597f0 0x7f57a80aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.982+0000 7f57af603700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80a8690 con 0x7f57a8066610 2024-01-24T14:10:20.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.982+0000 7f57af603700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8150500 con 0x7f57a81597f0 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.982+0000 7f57af603700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a815bc60 con 0x7f57a80669f0 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57ad39f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57a8066610 0x7f57a8150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57ad39f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57a8066610 0x7f57a8150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46134/0 (socket says 172.21.15.29:46134) 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57ad39f700 1 -- 172.21.15.29:0/47029065 learned_addr learned my addr 172.21.15.29:0/47029065 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57acb9e700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a80669f0 0x7f57a815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57adba0700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57a81597f0 0x7f57a80aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57ad39f700 1 -- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a80669f0 msgr2=0x7f57a815b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57ad39f700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a80669f0 0x7f57a815b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57ad39f700 1 -- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57a81597f0 msgr2=0x7f57a80aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57ad39f700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57a81597f0 0x7f57a80aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57ad39f700 1 -- 172.21.15.29:0/47029065 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a0009ae0 con 0x7f57a8066610 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57ad39f700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57a8066610 0x7f57a8150010 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f579800d890 tx=0x7f5798008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f579d7fa700 1 -- 172.21.15.29:0/47029065 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798011590 con 0x7f57a8066610 2024-01-24T14:10:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f579d7fa700 1 -- 172.21.15.29:0/47029065 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57980116f0 con 0x7f57a8066610 2024-01-24T14:10:20.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57af603700 1 -- 172.21.15.29:0/47029065 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a815bf40 con 0x7f57a8066610 2024-01-24T14:10:20.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f579d7fa700 1 -- 172.21.15.29:0/47029065 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798011a70 con 0x7f57a8066610 2024-01-24T14:10:20.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.983+0000 7f57af603700 1 -- 172.21.15.29:0/47029065 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a815c500 con 0x7f57a8066610 2024-01-24T14:10:20.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.984+0000 7f57af603700 1 -- 172.21.15.29:0/47029065 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a806ae00 con 0x7f57a8066610 2024-01-24T14:10:20.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.985+0000 7f579d7fa700 1 -- 172.21.15.29:0/47029065 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f579801c460 con 0x7f57a8066610 2024-01-24T14:10:20.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.985+0000 7f579d7fa700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f578c05f2f0 0x7f578c0617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:20.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.985+0000 7f579d7fa700 1 -- 172.21.15.29:0/47029065 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f579807a270 con 0x7f57a8066610 2024-01-24T14:10:20.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.985+0000 7f57acb9e700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f578c05f2f0 0x7f578c0617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:20.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.986+0000 7f57acb9e700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f578c05f2f0 0x7f578c0617b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f57a813ba30 tx=0x7f57a4000bc0).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:20.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:20.989+0000 7f579d7fa700 1 -- 172.21.15.29:0/47029065 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f579804b100 con 0x7f57a8066610 2024-01-24T14:10:21.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.197+0000 7f57af603700 1 -- 172.21.15.29:0/47029065 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f57a80a87d0 con 0x7f578c05f2f0 2024-01-24T14:10:21.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.202+0000 7f579d7fa700 1 -- 172.21.15.29:0/47029065 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f57a4003550 con 0x7f578c05f2f0 2024-01-24T14:10:21.202 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2024-01-24T14:10:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.204+0000 7f57af603700 1 -- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f578c05f2f0 msgr2=0x7f578c0617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.204+0000 7f57af603700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f578c05f2f0 0x7f578c0617b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f57a813ba30 tx=0x7f57a4000bc0).stop 2024-01-24T14:10:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.204+0000 7f57af603700 1 -- 172.21.15.29:0/47029065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57a8066610 msgr2=0x7f57a8150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.204+0000 7f57af603700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57a8066610 0x7f57a8150010 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f579800d890 tx=0x7f5798008040).stop 2024-01-24T14:10:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.204+0000 7f57af603700 1 -- 172.21.15.29:0/47029065 shutdown_connections 2024-01-24T14:10:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.204+0000 7f57af603700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57a81597f0 0x7f57a80aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.204+0000 7f57af603700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f578c05f2f0 0x7f578c0617b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.204+0000 7f57af603700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57a8066610 0x7f57a8150010 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.204+0000 7f57af603700 1 --2- 172.21.15.29:0/47029065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a80669f0 0x7f57a815b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.204+0000 7f57af603700 1 -- 172.21.15.29:0/47029065 >> 172.21.15.29:0/47029065 conn(0x7f57a80c8bd0 msgr2=0x7f57a80bfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.204+0000 7f57af603700 1 -- 172.21.15.29:0/47029065 shutdown_connections 2024-01-24T14:10:21.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.204+0000 7f57af603700 1 -- 172.21.15.29:0/47029065 wait complete. 2024-01-24T14:10:21.218 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2024-01-24T14:10:21.218 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir4 1M 2024-01-24T14:10:21.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.354+0000 7fe0af148700 1 Processor -- start 2024-01-24T14:10:21.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.355+0000 7fe0af148700 1 -- start start 2024-01-24T14:10:21.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0af148700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0a8150970 0x7fe0a8150d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0af148700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe0a8069ee0 0x7fe0a8154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0af148700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0a806a340 0x7fe0a8154e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0af148700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a8155330 con 0x7fe0a806a340 2024-01-24T14:10:21.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0af148700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a8069a50 con 0x7fe0a8069ee0 2024-01-24T14:10:21.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0af148700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a80c2b20 con 0x7fe0a8150970 2024-01-24T14:10:21.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0ad6e5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0a806a340 0x7fe0a8154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0ad6e5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0a806a340 0x7fe0a8154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46136/0 (socket says 172.21.15.29:46136) 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0ad6e5700 1 -- 172.21.15.29:0/1840235305 learned_addr learned my addr 172.21.15.29:0/1840235305 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0acee4700 1 --2- 172.21.15.29:0/1840235305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0a8150970 0x7fe0a8150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0a7fff700 1 --2- 172.21.15.29:0/1840235305 >> v2:172.21.15.163:3300/0 conn(0x7fe0a8069ee0 0x7fe0a8154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0ad6e5700 1 -- 172.21.15.29:0/1840235305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0a8150970 msgr2=0x7fe0a8150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0ad6e5700 1 --2- 172.21.15.29:0/1840235305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0a8150970 0x7fe0a8150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0ad6e5700 1 -- 172.21.15.29:0/1840235305 >> v2:172.21.15.163:3300/0 conn(0x7fe0a8069ee0 msgr2=0x7fe0a8154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0ad6e5700 1 --2- 172.21.15.29:0/1840235305 >> v2:172.21.15.163:3300/0 conn(0x7fe0a8069ee0 0x7fe0a8154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0ad6e5700 1 -- 172.21.15.29:0/1840235305 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a8151240 con 0x7fe0a806a340 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.356+0000 7fe0a7fff700 1 --2- 172.21.15.29:0/1840235305 >> v2:172.21.15.163:3300/0 conn(0x7fe0a8069ee0 0x7fe0a8154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0ad6e5700 1 --2- 172.21.15.29:0/1840235305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0a806a340 0x7fe0a8154e40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fe0a80685d0 tx=0x7fe0a0009840).ready entity=mon.0 client_cookie=c5677bbb843e11ad server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0a77fe700 1 -- 172.21.15.29:0/1840235305 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a000e070 con 0x7fe0a806a340 2024-01-24T14:10:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0a77fe700 1 -- 172.21.15.29:0/1840235305 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a000ba20 con 0x7fe0a806a340 2024-01-24T14:10:21.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0a77fe700 1 -- 172.21.15.29:0/1840235305 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a000bda0 con 0x7fe0a806a340 2024-01-24T14:10:21.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0af148700 1 -- 172.21.15.29:0/1840235305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0a806a340 msgr2=0x7fe0a8154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0af148700 1 --2- 172.21.15.29:0/1840235305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0a806a340 0x7fe0a8154e40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fe0a80685d0 tx=0x7fe0a0009840).stop 2024-01-24T14:10:21.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0af148700 1 -- 172.21.15.29:0/1840235305 shutdown_connections 2024-01-24T14:10:21.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0af148700 1 --2- 172.21.15.29:0/1840235305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0a8150970 0x7fe0a8150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0af148700 1 --2- 172.21.15.29:0/1840235305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0a806a340 0x7fe0a8154e40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0af148700 1 --2- 172.21.15.29:0/1840235305 >> v2:172.21.15.163:3300/0 conn(0x7fe0a8069ee0 0x7fe0a8154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0af148700 1 -- 172.21.15.29:0/1840235305 >> 172.21.15.29:0/1840235305 conn(0x7fe0a8067b20 msgr2=0x7fe0a80638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:21.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0af148700 1 -- 172.21.15.29:0/1840235305 shutdown_connections 2024-01-24T14:10:21.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.357+0000 7fe0af148700 1 -- 172.21.15.29:0/1840235305 wait complete. 2024-01-24T14:10:21.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.359+0000 7fe0af148700 1 Processor -- start 2024-01-24T14:10:21.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.359+0000 7fe0af148700 1 -- start start 2024-01-24T14:10:21.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0af148700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0a8069ee0 0x7fe0a8151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0af148700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0a806a340 0x7fe0a8153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0af148700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0a8150970 0x7fe0a8153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0af148700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a8154270 con 0x7fe0a8069ee0 2024-01-24T14:10:21.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0af148700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a80c6d00 con 0x7fe0a806a340 2024-01-24T14:10:21.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0af148700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a8152d10 con 0x7fe0a8150970 2024-01-24T14:10:21.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0acee4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0a8069ee0 0x7fe0a8151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0acee4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0a8069ee0 0x7fe0a8151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46148/0 (socket says 172.21.15.29:46148) 2024-01-24T14:10:21.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0acee4700 1 -- 172.21.15.29:0/2753899260 learned_addr learned my addr 172.21.15.29:0/2753899260 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0a7fff700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0a806a340 0x7fe0a8153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0ad6e5700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0a8150970 0x7fe0a8153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0acee4700 1 -- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0a8150970 msgr2=0x7fe0a8153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0acee4700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0a8150970 0x7fe0a8153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0acee4700 1 -- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0a806a340 msgr2=0x7fe0a8153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0acee4700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0a806a340 0x7fe0a8153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0acee4700 1 -- 172.21.15.29:0/2753899260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a0009ae0 con 0x7fe0a8069ee0 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0ad6e5700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0a8150970 0x7fe0a8153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.360+0000 7fe0acee4700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0a8069ee0 0x7fe0a8151da0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fe09800e950 tx=0x7fe098009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.361+0000 7fe0a4ff9700 1 -- 172.21.15.29:0/2753899260 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09800c6a0 con 0x7fe0a8069ee0 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.361+0000 7fe0a4ff9700 1 -- 172.21.15.29:0/2753899260 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09800c800 con 0x7fe0a8069ee0 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.361+0000 7fe0af148700 1 -- 172.21.15.29:0/2753899260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a8152750 con 0x7fe0a8069ee0 2024-01-24T14:10:21.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.361+0000 7fe0a4ff9700 1 -- 172.21.15.29:0/2753899260 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09800cb80 con 0x7fe0a8069ee0 2024-01-24T14:10:21.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.361+0000 7fe0af148700 1 -- 172.21.15.29:0/2753899260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a80c6990 con 0x7fe0a8069ee0 2024-01-24T14:10:21.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.362+0000 7fe0af148700 1 -- 172.21.15.29:0/2753899260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a80475e0 con 0x7fe0a8069ee0 2024-01-24T14:10:21.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.362+0000 7fe0a4ff9700 1 -- 172.21.15.29:0/2753899260 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fe09801b400 con 0x7fe0a8069ee0 2024-01-24T14:10:21.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.362+0000 7fe0a4ff9700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe09405f2c0 0x7fe094061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.363+0000 7fe0a4ff9700 1 -- 172.21.15.29:0/2753899260 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fe09807aba0 con 0x7fe0a8069ee0 2024-01-24T14:10:21.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.363+0000 7fe0a7fff700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe09405f2c0 0x7fe094061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.363+0000 7fe0a7fff700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe09405f2c0 0x7fe094061780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe0a8154870 tx=0x7fe09c008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:21.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.367+0000 7fe0a4ff9700 1 -- 172.21.15.29:0/2753899260 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe09804ba30 con 0x7fe0a8069ee0 2024-01-24T14:10:21.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.578+0000 7fe0af148700 1 -- 172.21.15.29:0/2753899260 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0a8064720 con 0x7fe09405f2c0 2024-01-24T14:10:21.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.582+0000 7fe0a4ff9700 1 -- 172.21.15.29:0/2753899260 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe0a8064720 con 0x7fe09405f2c0 2024-01-24T14:10:21.583 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2024-01-24T14:10:21.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.584+0000 7fe0af148700 1 -- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe09405f2c0 msgr2=0x7fe094061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.584+0000 7fe0af148700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe09405f2c0 0x7fe094061780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe0a8154870 tx=0x7fe09c008040).stop 2024-01-24T14:10:21.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.584+0000 7fe0af148700 1 -- 172.21.15.29:0/2753899260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0a8069ee0 msgr2=0x7fe0a8151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.584+0000 7fe0af148700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0a8069ee0 0x7fe0a8151da0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fe09800e950 tx=0x7fe098009060).stop 2024-01-24T14:10:21.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.584+0000 7fe0af148700 1 -- 172.21.15.29:0/2753899260 shutdown_connections 2024-01-24T14:10:21.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.584+0000 7fe0af148700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0a8150970 0x7fe0a8153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.584+0000 7fe0af148700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe09405f2c0 0x7fe094061780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.584+0000 7fe0af148700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0a8069ee0 0x7fe0a8151da0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.584+0000 7fe0af148700 1 --2- 172.21.15.29:0/2753899260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0a806a340 0x7fe0a8153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.584+0000 7fe0af148700 1 -- 172.21.15.29:0/2753899260 >> 172.21.15.29:0/2753899260 conn(0x7fe0a8067b20 msgr2=0x7fe0a80a8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:21.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.584+0000 7fe0af148700 1 -- 172.21.15.29:0/2753899260 shutdown_connections 2024-01-24T14:10:21.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.585+0000 7fe0af148700 1 -- 172.21.15.29:0/2753899260 wait complete. 2024-01-24T14:10:21.598 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2024-01-24T14:10:21.598 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir5 1M 2024-01-24T14:10:21.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.733+0000 7f56f0a0f700 1 Processor -- start 2024-01-24T14:10:21.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.733+0000 7f56f0a0f700 1 -- start start 2024-01-24T14:10:21.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.734+0000 7f56f0a0f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f56ec14fd10 0x7f56ec0c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.734+0000 7f56f0a0f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f56ec0684c0 0x7f56ec153090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.734+0000 7f56f0a0f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec067a70 con 0x7f56ec066450 2024-01-24T14:10:21.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.734+0000 7f56f0a0f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ec0c3d50 con 0x7f56ec0684c0 2024-01-24T14:10:21.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.734+0000 7f56f0a0f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ec1500f0 con 0x7f56ec14fd10 2024-01-24T14:10:21.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.734+0000 7f56e9d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f56ec0684c0 0x7f56ec153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.734+0000 7f56e9d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f56ec0684c0 0x7f56ec153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46156/0 (socket says 172.21.15.29:46156) 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.734+0000 7f56e9d9b700 1 -- 172.21.15.29:0/478814735 learned_addr learned my addr 172.21.15.29:0/478814735 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.735+0000 7f56ea59c700 1 --2- 172.21.15.29:0/478814735 >> v2:172.21.15.163:3300/0 conn(0x7f56ec14fd10 0x7f56ec0c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.735+0000 7f56e9d9b700 1 -- 172.21.15.29:0/478814735 >> v2:172.21.15.163:3300/0 conn(0x7f56ec14fd10 msgr2=0x7f56ec0c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.735+0000 7f56e9d9b700 1 --2- 172.21.15.29:0/478814735 >> v2:172.21.15.163:3300/0 conn(0x7f56ec14fd10 0x7f56ec0c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.735+0000 7f56e9d9b700 1 -- 172.21.15.29:0/478814735 >> v1:172.21.15.29:6789/0 conn(0x7f56ec066450 legacy=0x7f56ec0bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.735+0000 7f56e9d9b700 1 -- 172.21.15.29:0/478814735 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec153580 con 0x7f56ec0684c0 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.735+0000 7f56e9d9b700 1 --2- 172.21.15.29:0/478814735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56ec0684c0 0x7f56ec153090 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f56ec06c140 tx=0x7f56dc0090f0).ready entity=mon.0 client_cookie=3195996462bd7ab6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.735+0000 7f56e959a700 1 -- 172.21.15.29:0/478814735 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00b600 con 0x7f56ec0684c0 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.735+0000 7f56e959a700 1 -- 172.21.15.29:0/478814735 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc00b760 con 0x7f56ec0684c0 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.735+0000 7f56e959a700 1 -- 172.21.15.29:0/478814735 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00bae0 con 0x7f56ec0684c0 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.736+0000 7f56f0a0f700 1 -- 172.21.15.29:0/478814735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56ec0684c0 msgr2=0x7f56ec153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.736+0000 7f56f0a0f700 1 --2- 172.21.15.29:0/478814735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56ec0684c0 0x7f56ec153090 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f56ec06c140 tx=0x7f56dc0090f0).stop 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.736+0000 7f56f0a0f700 1 -- 172.21.15.29:0/478814735 shutdown_connections 2024-01-24T14:10:21.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.736+0000 7f56f0a0f700 1 --2- 172.21.15.29:0/478814735 >> v2:172.21.15.163:3300/0 conn(0x7f56ec14fd10 0x7f56ec0c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.736+0000 7f56f0a0f700 1 --2- 172.21.15.29:0/478814735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56ec0684c0 0x7f56ec153090 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.736+0000 7f56f0a0f700 1 -- 172.21.15.29:0/478814735 >> 172.21.15.29:0/478814735 conn(0x7f56ec0c8bd0 msgr2=0x7f56ec0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:21.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.736+0000 7f56f0a0f700 1 -- 172.21.15.29:0/478814735 shutdown_connections 2024-01-24T14:10:21.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.736+0000 7f56f0a0f700 1 -- 172.21.15.29:0/478814735 wait complete. 2024-01-24T14:10:21.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.737+0000 7f56f0a0f700 1 Processor -- start 2024-01-24T14:10:21.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.737+0000 7f56f0a0f700 1 -- start start 2024-01-24T14:10:21.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56f0a0f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56ec066450 0x7f56ec13b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56f0a0f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec0684c0 0x7f56ec14d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56f0a0f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56ec14fd10 0x7f56ec14d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56f0a0f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec14cce0 con 0x7f56ec066450 2024-01-24T14:10:21.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56f0a0f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec14ddb0 con 0x7f56ec0684c0 2024-01-24T14:10:21.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56f0a0f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec139930 con 0x7f56ec14fd10 2024-01-24T14:10:21.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56ea59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56ec066450 0x7f56ec13b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56ea59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56ec066450 0x7f56ec13b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46168/0 (socket says 172.21.15.29:46168) 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56ea59c700 1 -- 172.21.15.29:0/1144458338 learned_addr learned my addr 172.21.15.29:0/1144458338 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56ead9d700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56ec14fd10 0x7f56ec14d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56e9d9b700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec0684c0 0x7f56ec14d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56ea59c700 1 -- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56ec14fd10 msgr2=0x7f56ec14d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56ea59c700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56ec14fd10 0x7f56ec14d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56ea59c700 1 -- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec0684c0 msgr2=0x7f56ec14d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56ea59c700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec0684c0 0x7f56ec14d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56ea59c700 1 -- 172.21.15.29:0/1144458338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc009370 con 0x7f56ec066450 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56ead9d700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56ec14fd10 0x7f56ec14d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.738+0000 7f56ea59c700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56ec066450 0x7f56ec13b5d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f56d400d860 tx=0x7f56d4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.739+0000 7f56e27fc700 1 -- 172.21.15.29:0/1144458338 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4011590 con 0x7f56ec066450 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.739+0000 7f56e27fc700 1 -- 172.21.15.29:0/1144458338 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d40116f0 con 0x7f56ec066450 2024-01-24T14:10:21.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.739+0000 7f56f0a0f700 1 -- 172.21.15.29:0/1144458338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec165940 con 0x7f56ec066450 2024-01-24T14:10:21.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.739+0000 7f56e27fc700 1 -- 172.21.15.29:0/1144458338 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4011a70 con 0x7f56ec066450 2024-01-24T14:10:21.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.739+0000 7f56f0a0f700 1 -- 172.21.15.29:0/1144458338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec165f00 con 0x7f56ec066450 2024-01-24T14:10:21.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.740+0000 7f56f0a0f700 1 -- 172.21.15.29:0/1144458338 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ec06ae00 con 0x7f56ec066450 2024-01-24T14:10:21.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.740+0000 7f56e27fc700 1 -- 172.21.15.29:0/1144458338 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f56d4012420 con 0x7f56ec066450 2024-01-24T14:10:21.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.741+0000 7f56e27fc700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56cc05f2c0 0x7f56cc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:21.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.741+0000 7f56e27fc700 1 -- 172.21.15.29:0/1144458338 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f56d407a130 con 0x7f56ec066450 2024-01-24T14:10:21.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.741+0000 7f56e9d9b700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56cc05f2c0 0x7f56cc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:21.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.741+0000 7f56e9d9b700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56cc05f2c0 0x7f56cc061780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f56dc009700 tx=0x7f56dc017000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:21.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.745+0000 7f56e27fc700 1 -- 172.21.15.29:0/1144458338 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f56d404afc0 con 0x7f56ec066450 2024-01-24T14:10:21.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.956+0000 7f56f0a0f700 1 -- 172.21.15.29:0/1144458338 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f56ec166540 con 0x7f56cc05f2c0 2024-01-24T14:10:21.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.960+0000 7f56e27fc700 1 -- 172.21.15.29:0/1144458338 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f56ec166540 con 0x7f56cc05f2c0 2024-01-24T14:10:21.961 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2024-01-24T14:10:21.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.962+0000 7f56f0a0f700 1 -- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56cc05f2c0 msgr2=0x7f56cc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.962+0000 7f56f0a0f700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56cc05f2c0 0x7f56cc061780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f56dc009700 tx=0x7f56dc017000).stop 2024-01-24T14:10:21.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.962+0000 7f56f0a0f700 1 -- 172.21.15.29:0/1144458338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56ec066450 msgr2=0x7f56ec13b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:21.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.962+0000 7f56f0a0f700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56ec066450 0x7f56ec13b5d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f56d400d860 tx=0x7f56d4008040).stop 2024-01-24T14:10:21.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.963+0000 7f56f0a0f700 1 -- 172.21.15.29:0/1144458338 shutdown_connections 2024-01-24T14:10:21.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.963+0000 7f56f0a0f700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56ec14fd10 0x7f56ec14d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.963+0000 7f56f0a0f700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f56cc05f2c0 0x7f56cc061780 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.963+0000 7f56f0a0f700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56ec066450 0x7f56ec13b5d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.963+0000 7f56f0a0f700 1 --2- 172.21.15.29:0/1144458338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec0684c0 0x7f56ec14d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:21.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.963+0000 7f56f0a0f700 1 -- 172.21.15.29:0/1144458338 >> 172.21.15.29:0/1144458338 conn(0x7f56ec0c8bd0 msgr2=0x7f56ec0688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:21.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.963+0000 7f56f0a0f700 1 -- 172.21.15.29:0/1144458338 shutdown_connections 2024-01-24T14:10:21.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:21.963+0000 7f56f0a0f700 1 -- 172.21.15.29:0/1144458338 wait complete. 2024-01-24T14:10:21.976 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2024-01-24T14:10:21.976 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir6 1M 2024-01-24T14:10:22.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.113+0000 7f55fa5a7700 1 Processor -- start 2024-01-24T14:10:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.113+0000 7f55fa5a7700 1 -- start start 2024-01-24T14:10:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.114+0000 7f55fa5a7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55f4062ec0 0x7f55f40a4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.114+0000 7f55fa5a7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f40a4e50 0x7f55f40bca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.114+0000 7f55fa5a7700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f40a62a0 con 0x7f55f41541f0 2024-01-24T14:10:22.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.114+0000 7f55fa5a7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f40bea30 con 0x7f55f4062ec0 2024-01-24T14:10:22.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.114+0000 7f55fa5a7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40a6730 con 0x7f55f40a4e50 2024-01-24T14:10:22.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.114+0000 7f55f3fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f55f41541f0 0x7f55f40627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:36012/0 (socket says 172.21.15.29:36012) 2024-01-24T14:10:22.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.114+0000 7f55f3fff700 1 -- 172.21.15.29:0/3250257430 learned_addr learned my addr 172.21.15.29:0/3250257430 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:22.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.114+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564991539 0 0) 0x7f55f40a62a0 con 0x7f55f41541f0 2024-01-24T14:10:22.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.114+0000 7f55f8b44700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f40a4e50 0x7f55f40bca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.114+0000 7f55f37fe700 1 --2- 172.21.15.29:0/3250257430 >> v2:172.21.15.163:3300/0 conn(0x7f55f4062ec0 0x7f55f40a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.114+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55dc0033f0 con 0x7f55f41541f0 2024-01-24T14:10:22.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4182528687 0 0) 0x7f55dc0033f0 con 0x7f55f41541f0 2024-01-24T14:10:22.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f40a4e50 msgr2=0x7f55f40bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:22.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f40a4e50 0x7f55f40bca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 >> v2:172.21.15.163:3300/0 conn(0x7f55f4062ec0 msgr2=0x7f55f40a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 --2- 172.21.15.29:0/3250257430 >> v2:172.21.15.163:3300/0 conn(0x7f55f4062ec0 0x7f55f40a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40a5690 con 0x7f55f41541f0 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f55e4001470 con 0x7f55f41541f0 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 >> v1:172.21.15.29:6789/0 conn(0x7f55f41541f0 legacy=0x7f55f40627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55dc0038e0 0x7f55dc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f40a4e50 0x7f55dc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55dc004b80 0x7f55dc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40bea30 con 0x7f55dc0038e0 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40a6730 con 0x7f55dc004b80 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc00b8e0 con 0x7f55f40a4e50 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f3fff700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55dc0038e0 0x7f55dc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f37fe700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f40a4e50 0x7f55dc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f8b44700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55dc004b80 0x7f55dc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f3fff700 1 -- 172.21.15.29:0/3250257430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f40a4e50 msgr2=0x7f55dc004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:22.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f3fff700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f40a4e50 0x7f55dc004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f3fff700 1 -- 172.21.15.29:0/3250257430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55dc004b80 msgr2=0x7f55dc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f3fff700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55dc004b80 0x7f55dc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f3fff700 1 -- 172.21.15.29:0/3250257430 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc0033f0 con 0x7f55dc0038e0 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.115+0000 7f55f8b44700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55dc004b80 0x7f55dc005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.116+0000 7f55f3fff700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55dc0038e0 0x7f55dc003d00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f55e400c210 tx=0x7f55e4001e90).ready entity=mon.0 client_cookie=b6106e5f893d8e0c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.116+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4007a60 con 0x7f55dc0038e0 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.116+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e400cae0 con 0x7f55dc0038e0 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.116+0000 7f55f2ffd700 1 -- 172.21.15.29:0/3250257430 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e400ce60 con 0x7f55dc0038e0 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.116+0000 7f55fa5a7700 1 -- 172.21.15.29:0/3250257430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55dc0038e0 msgr2=0x7f55dc003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.116+0000 7f55fa5a7700 1 --2- 172.21.15.29:0/3250257430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55dc0038e0 0x7f55dc003d00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f55e400c210 tx=0x7f55e4001e90).stop 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.116+0000 7f55f8b44700 1 -- 172.21.15.29:0/3250257430 reap_dead start 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.116+0000 7f55fa5a7700 1 -- 172.21.15.29:0/3250257430 shutdown_connections 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.116+0000 7f55fa5a7700 1 -- 172.21.15.29:0/3250257430 >> 172.21.15.29:0/3250257430 conn(0x7f55f40c7af0 msgr2=0x7f55f40c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:22.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.116+0000 7f55fa5a7700 1 -- 172.21.15.29:0/3250257430 shutdown_connections 2024-01-24T14:10:22.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.116+0000 7f55fa5a7700 1 -- 172.21.15.29:0/3250257430 wait complete. 2024-01-24T14:10:22.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.118+0000 7f55fa5a7700 1 Processor -- start 2024-01-24T14:10:22.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.118+0000 7f55fa5a7700 1 -- start start 2024-01-24T14:10:22.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55fa5a7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55dc0038e0 0x7f55f4148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55fa5a7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f4159910 0x7f55f414d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55fa5a7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f4159d90 0x7f55f414d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55fa5a7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4158870 con 0x7f55f4159910 2024-01-24T14:10:22.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55fa5a7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4158b50 con 0x7f55dc0038e0 2024-01-24T14:10:22.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55fa5a7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4158e30 con 0x7f55f4159d90 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f4159910 0x7f55f414d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f4159910 0x7f55f414d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46178/0 (socket says 172.21.15.29:46178) 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f37fe700 1 -- 172.21.15.29:0/741171107 learned_addr learned my addr 172.21.15.29:0/741171107 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f8b44700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f4159d90 0x7f55f414d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f3fff700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55dc0038e0 0x7f55f4148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f37fe700 1 -- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f4159d90 msgr2=0x7f55f414d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f37fe700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f4159d90 0x7f55f414d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f37fe700 1 -- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55dc0038e0 msgr2=0x7f55f4148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f37fe700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55dc0038e0 0x7f55f4148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f37fe700 1 -- 172.21.15.29:0/741171107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f4159110 con 0x7f55f4159910 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f3fff700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55dc0038e0 0x7f55f4148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f8b44700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f4159d90 0x7f55f414d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55f37fe700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f4159910 0x7f55f414d2e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f55e80119f0 tx=0x7f55e8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55dbfff700 1 -- 172.21.15.29:0/741171107 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e800f9d0 con 0x7f55f4159910 2024-01-24T14:10:22.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55dbfff700 1 -- 172.21.15.29:0/741171107 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e800fb30 con 0x7f55f4159910 2024-01-24T14:10:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55dbfff700 1 -- 172.21.15.29:0/741171107 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e800feb0 con 0x7f55f4159910 2024-01-24T14:10:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55fa5a7700 1 -- 172.21.15.29:0/741171107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f41593f0 con 0x7f55f4159910 2024-01-24T14:10:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.119+0000 7f55fa5a7700 1 -- 172.21.15.29:0/741171107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f4148cc0 con 0x7f55f4159910 2024-01-24T14:10:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.121+0000 7f55fa5a7700 1 -- 172.21.15.29:0/741171107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f4159630 con 0x7f55f4159910 2024-01-24T14:10:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.121+0000 7f55dbfff700 1 -- 172.21.15.29:0/741171107 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f55e8014650 con 0x7f55f4159910 2024-01-24T14:10:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.121+0000 7f55dbfff700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55e005f2f0 0x7f55e00617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.121+0000 7f55dbfff700 1 -- 172.21.15.29:0/741171107 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f55e807b230 con 0x7f55f4159910 2024-01-24T14:10:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.121+0000 7f55f3fff700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55e005f2f0 0x7f55e00617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.122+0000 7f55f3fff700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55e005f2f0 0x7f55e00617b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f55e4001ed0 tx=0x7f55e4015000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:22.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.126+0000 7f55dbfff700 1 -- 172.21.15.29:0/741171107 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f55f4159630 con 0x7f55f4159910 2024-01-24T14:10:22.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.337+0000 7f55fa5a7700 1 -- 172.21.15.29:0/741171107 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f55f4159630 con 0x7f55e005f2f0 2024-01-24T14:10:22.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.341+0000 7f55dbfff700 1 -- 172.21.15.29:0/741171107 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f55f4159630 con 0x7f55e005f2f0 2024-01-24T14:10:22.341 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2024-01-24T14:10:22.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.343+0000 7f55fa5a7700 1 -- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55e005f2f0 msgr2=0x7f55e00617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.343+0000 7f55fa5a7700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55e005f2f0 0x7f55e00617b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f55e4001ed0 tx=0x7f55e4015000).stop 2024-01-24T14:10:22.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.343+0000 7f55fa5a7700 1 -- 172.21.15.29:0/741171107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f4159910 msgr2=0x7f55f414d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.343+0000 7f55fa5a7700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f4159910 0x7f55f414d2e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f55e80119f0 tx=0x7f55e8009060).stop 2024-01-24T14:10:22.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.343+0000 7f55fa5a7700 1 -- 172.21.15.29:0/741171107 shutdown_connections 2024-01-24T14:10:22.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.343+0000 7f55fa5a7700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55f4159d90 0x7f55f414d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.343+0000 7f55fa5a7700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55dc0038e0 0x7f55f4148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.343+0000 7f55fa5a7700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55e005f2f0 0x7f55e00617b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.343+0000 7f55fa5a7700 1 --2- 172.21.15.29:0/741171107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f4159910 0x7f55f414d2e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.343+0000 7f55fa5a7700 1 -- 172.21.15.29:0/741171107 >> 172.21.15.29:0/741171107 conn(0x7f55f40c7af0 msgr2=0x7f55f40c66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:22.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.343+0000 7f55fa5a7700 1 -- 172.21.15.29:0/741171107 shutdown_connections 2024-01-24T14:10:22.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.343+0000 7f55fa5a7700 1 -- 172.21.15.29:0/741171107 wait complete. 2024-01-24T14:10:22.356 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2024-01-24T14:10:22.357 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir7 1M 2024-01-24T14:10:22.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.493+0000 7f1bcbd5a700 1 Processor -- start 2024-01-24T14:10:22.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.494+0000 7f1bcbd5a700 1 -- start start 2024-01-24T14:10:22.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.494+0000 7f1bcbd5a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1bc4066bc0 0x7f1bc413a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.494+0000 7f1bcbd5a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc40638c0 0x7f1bc4063d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.494+0000 7f1bcbd5a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc40bdb10 con 0x7f1bc413a330 2024-01-24T14:10:22.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.494+0000 7f1bcbd5a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc4068990 con 0x7f1bc4066bc0 2024-01-24T14:10:22.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.494+0000 7f1bcbd5a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc40bdd60 con 0x7f1bc40638c0 2024-01-24T14:10:22.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bc92f5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1bc4066bc0 0x7f1bc413a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bc92f5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1bc4066bc0 0x7f1bc413a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46186/0 (socket says 172.21.15.29:46186) 2024-01-24T14:10:22.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bc92f5700 1 -- 172.21.15.29:0/1193121500 learned_addr learned my addr 172.21.15.29:0/1193121500 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:22.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bca2f7700 1 --2- 172.21.15.29:0/1193121500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc40638c0 0x7f1bc4063d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bc92f5700 1 -- 172.21.15.29:0/1193121500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc40638c0 msgr2=0x7f1bc4063d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:22.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bc92f5700 1 --2- 172.21.15.29:0/1193121500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc40638c0 0x7f1bc4063d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bc92f5700 1 -- 172.21.15.29:0/1193121500 >> v1:172.21.15.29:6789/0 conn(0x7f1bc413a330 legacy=0x7f1bc40664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bc92f5700 1 -- 172.21.15.29:0/1193121500 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc40be040 con 0x7f1bc4066bc0 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bca2f7700 1 --2- 172.21.15.29:0/1193121500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc40638c0 0x7f1bc4063d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bc92f5700 1 --2- 172.21.15.29:0/1193121500 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc4066bc0 0x7f1bc413a710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f1bc0006fd0 tx=0x7f1bc00090f0).ready entity=mon.0 client_cookie=5184d3418880efa6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bc8af4700 1 -- 172.21.15.29:0/1193121500 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc000b600 con 0x7f1bc4066bc0 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bc8af4700 1 -- 172.21.15.29:0/1193121500 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc000b760 con 0x7f1bc4066bc0 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.495+0000 7f1bc8af4700 1 -- 172.21.15.29:0/1193121500 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc000bae0 con 0x7f1bc4066bc0 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.496+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/1193121500 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc4066bc0 msgr2=0x7f1bc413a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.496+0000 7f1bcbd5a700 1 --2- 172.21.15.29:0/1193121500 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc4066bc0 0x7f1bc413a710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f1bc0006fd0 tx=0x7f1bc00090f0).stop 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.496+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/1193121500 shutdown_connections 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.496+0000 7f1bcbd5a700 1 --2- 172.21.15.29:0/1193121500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc40638c0 0x7f1bc4063d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.496+0000 7f1bcbd5a700 1 --2- 172.21.15.29:0/1193121500 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc4066bc0 0x7f1bc413a710 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.496+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/1193121500 >> 172.21.15.29:0/1193121500 conn(0x7f1bc40c8bb0 msgr2=0x7f1bc40c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.496+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/1193121500 shutdown_connections 2024-01-24T14:10:22.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.496+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/1193121500 wait complete. 2024-01-24T14:10:22.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.497+0000 7f1bcbd5a700 1 Processor -- start 2024-01-24T14:10:22.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.497+0000 7f1bcbd5a700 1 -- start start 2024-01-24T14:10:22.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.498+0000 7f1bcbd5a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc40638c0 0x7f1bc40c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.498+0000 7f1bcbd5a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc4066bc0 0x7f1bc4132280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.498+0000 7f1bcbd5a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc413a330 0x7f1bc40c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.498+0000 7f1bcbd5a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc40bfc50 con 0x7f1bc413a330 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.498+0000 7f1bcbd5a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc40c01d0 con 0x7f1bc4066bc0 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.498+0000 7f1bcbd5a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc40c5e20 con 0x7f1bc40638c0 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.498+0000 7f1bca2f7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc413a330 0x7f1bc40c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.498+0000 7f1bca2f7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc413a330 0x7f1bc40c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46200/0 (socket says 172.21.15.29:46200) 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.498+0000 7f1bca2f7700 1 -- 172.21.15.29:0/4110521417 learned_addr learned my addr 172.21.15.29:0/4110521417 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.498+0000 7f1bc9af6700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc40638c0 0x7f1bc40c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.498+0000 7f1bc92f5700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc4066bc0 0x7f1bc4132280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.499+0000 7f1bca2f7700 1 -- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc40638c0 msgr2=0x7f1bc40c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.499+0000 7f1bca2f7700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc40638c0 0x7f1bc40c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.499+0000 7f1bca2f7700 1 -- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc4066bc0 msgr2=0x7f1bc4132280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.499+0000 7f1bca2f7700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc4066bc0 0x7f1bc4132280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.499+0000 7f1bca2f7700 1 -- 172.21.15.29:0/4110521417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc0009370 con 0x7f1bc413a330 2024-01-24T14:10:22.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.499+0000 7f1bca2f7700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc413a330 0x7f1bc40c7eb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f1bbc00cef0 tx=0x7f1bbc007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.499+0000 7f1bb5ffb700 1 -- 172.21.15.29:0/4110521417 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00ad20 con 0x7f1bc413a330 2024-01-24T14:10:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.499+0000 7f1bb5ffb700 1 -- 172.21.15.29:0/4110521417 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc00ae80 con 0x7f1bc413a330 2024-01-24T14:10:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.499+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/4110521417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc4138e40 con 0x7f1bc413a330 2024-01-24T14:10:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.499+0000 7f1bb5ffb700 1 -- 172.21.15.29:0/4110521417 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc016610 con 0x7f1bc413a330 2024-01-24T14:10:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.499+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/4110521417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc40c63a0 con 0x7f1bc413a330 2024-01-24T14:10:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.500+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/4110521417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc406ae00 con 0x7f1bc413a330 2024-01-24T14:10:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.501+0000 7f1bb5ffb700 1 -- 172.21.15.29:0/4110521417 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f1bbc01f3f0 con 0x7f1bc413a330 2024-01-24T14:10:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.501+0000 7f1bb5ffb700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ba805f2c0 0x7f1ba8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.501+0000 7f1bb5ffb700 1 -- 172.21.15.29:0/4110521417 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f1bbc07ace0 con 0x7f1bc413a330 2024-01-24T14:10:22.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.501+0000 7f1bc9af6700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ba805f2c0 0x7f1ba8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.502+0000 7f1bc9af6700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ba805f2c0 0x7f1ba8061780 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1bb8003900 tx=0x7f1bb8007d40).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:22.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.505+0000 7f1bb5ffb700 1 -- 172.21.15.29:0/4110521417 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1bbc07f070 con 0x7f1bc413a330 2024-01-24T14:10:22.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.715+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/4110521417 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bc40c0360 con 0x7f1ba805f2c0 2024-01-24T14:10:22.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.720+0000 7f1bb5ffb700 1 -- 172.21.15.29:0/4110521417 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f1bc40c0360 con 0x7f1ba805f2c0 2024-01-24T14:10:22.720 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2024-01-24T14:10:22.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.722+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ba805f2c0 msgr2=0x7f1ba8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.722+0000 7f1bcbd5a700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ba805f2c0 0x7f1ba8061780 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1bb8003900 tx=0x7f1bb8007d40).stop 2024-01-24T14:10:22.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.722+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/4110521417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc413a330 msgr2=0x7f1bc40c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.722+0000 7f1bcbd5a700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc413a330 0x7f1bc40c7eb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f1bbc00cef0 tx=0x7f1bbc007860).stop 2024-01-24T14:10:22.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.722+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/4110521417 shutdown_connections 2024-01-24T14:10:22.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.722+0000 7f1bcbd5a700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ba805f2c0 0x7f1ba8061780 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.722+0000 7f1bcbd5a700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc413a330 0x7f1bc40c7eb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.722+0000 7f1bcbd5a700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc40638c0 0x7f1bc40c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.722+0000 7f1bcbd5a700 1 --2- 172.21.15.29:0/4110521417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc4066bc0 0x7f1bc4132280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.722+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/4110521417 >> 172.21.15.29:0/4110521417 conn(0x7f1bc40c8bb0 msgr2=0x7f1bc413ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.722+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/4110521417 shutdown_connections 2024-01-24T14:10:22.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.722+0000 7f1bcbd5a700 1 -- 172.21.15.29:0/4110521417 wait complete. 2024-01-24T14:10:22.735 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2024-01-24T14:10:22.736 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir8 1M 2024-01-24T14:10:22.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.871+0000 7f16660d0700 1 Processor -- start 2024-01-24T14:10:22.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.871+0000 7f16660d0700 1 -- start start 2024-01-24T14:10:22.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.871+0000 7f16660d0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1660151c70 0x7f1660152050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.871+0000 7f16660d0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1660156410 0x7f1660069ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f16660d0700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1660069a50 con 0x7f166006a3d0 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f16660d0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16600a77d0 con 0x7f1660156410 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f16660d0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16600a7ab0 con 0x7f1660151c70 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f165effd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1660156410 0x7f1660069ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f165effd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1660156410 0x7f1660069ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46208/0 (socket says 172.21.15.29:46208) 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f165effd700 1 -- 172.21.15.29:0/257118997 learned_addr learned my addr 172.21.15.29:0/257118997 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f165f7fe700 1 --2- 172.21.15.29:0/257118997 >> v2:172.21.15.163:3300/0 conn(0x7f1660151c70 0x7f1660152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f165effd700 1 -- 172.21.15.29:0/257118997 >> v2:172.21.15.163:3300/0 conn(0x7f1660151c70 msgr2=0x7f1660152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f165effd700 1 --2- 172.21.15.29:0/257118997 >> v2:172.21.15.163:3300/0 conn(0x7f1660151c70 0x7f1660152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f165effd700 1 -- 172.21.15.29:0/257118997 >> v1:172.21.15.29:6789/0 conn(0x7f166006a3d0 legacy=0x7f16600627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f165effd700 1 -- 172.21.15.29:0/257118997 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1660069400 con 0x7f1660156410 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f165effd700 1 --2- 172.21.15.29:0/257118997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1660156410 0x7f1660069ee0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f16600c55a0 tx=0x7f16540090f0).ready entity=mon.0 client_cookie=8a3381f34e65428b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f165e7fc700 1 -- 172.21.15.29:0/257118997 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165400b600 con 0x7f1660156410 2024-01-24T14:10:22.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f165e7fc700 1 -- 172.21.15.29:0/257118997 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165400b760 con 0x7f1660156410 2024-01-24T14:10:22.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.872+0000 7f165e7fc700 1 -- 172.21.15.29:0/257118997 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165400bae0 con 0x7f1660156410 2024-01-24T14:10:22.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.873+0000 7f16660d0700 1 -- 172.21.15.29:0/257118997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1660156410 msgr2=0x7f1660069ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.873+0000 7f16660d0700 1 --2- 172.21.15.29:0/257118997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1660156410 0x7f1660069ee0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f16600c55a0 tx=0x7f16540090f0).stop 2024-01-24T14:10:22.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.873+0000 7f16660d0700 1 -- 172.21.15.29:0/257118997 shutdown_connections 2024-01-24T14:10:22.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.873+0000 7f16660d0700 1 --2- 172.21.15.29:0/257118997 >> v2:172.21.15.163:3300/0 conn(0x7f1660151c70 0x7f1660152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.873+0000 7f16660d0700 1 --2- 172.21.15.29:0/257118997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1660156410 0x7f1660069ee0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.873+0000 7f16660d0700 1 -- 172.21.15.29:0/257118997 >> 172.21.15.29:0/257118997 conn(0x7f1660067b20 msgr2=0x7f16600638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:22.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.873+0000 7f16660d0700 1 -- 172.21.15.29:0/257118997 shutdown_connections 2024-01-24T14:10:22.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.873+0000 7f16660d0700 1 -- 172.21.15.29:0/257118997 wait complete. 2024-01-24T14:10:22.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.874+0000 7f16660d0700 1 Processor -- start 2024-01-24T14:10:22.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.875+0000 7f16660d0700 1 -- start start 2024-01-24T14:10:22.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.875+0000 7f16660d0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f166006a3d0 0x7f16600c0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.875+0000 7f16660d0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1660151c70 0x7f166014ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.875+0000 7f16660d0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1660156410 0x7f16600c2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.875+0000 7f16660d0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16600c04d0 con 0x7f1660156410 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.875+0000 7f16660d0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16600bf8c0 con 0x7f166006a3d0 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.875+0000 7f16660d0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16600bfba0 con 0x7f1660151c70 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.875+0000 7f165ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1660156410 0x7f16600c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f165ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1660156410 0x7f16600c2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46214/0 (socket says 172.21.15.29:46214) 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f165ffff700 1 -- 172.21.15.29:0/4278161672 learned_addr learned my addr 172.21.15.29:0/4278161672 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f165f7fe700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f166006a3d0 0x7f16600c0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f165effd700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1660151c70 0x7f166014ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f165ffff700 1 -- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1660151c70 msgr2=0x7f166014ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f165ffff700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1660151c70 0x7f166014ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f165ffff700 1 -- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f166006a3d0 msgr2=0x7f16600c0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:22.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f165ffff700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f166006a3d0 0x7f16600c0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:22.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f165ffff700 1 -- 172.21.15.29:0/4278161672 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1654009370 con 0x7f1660156410 2024-01-24T14:10:22.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f165effd700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1660151c70 0x7f166014ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:22.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f165ffff700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1660156410 0x7f16600c2160 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f165800c7c0 tx=0x7f1658001560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:22.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f16477fe700 1 -- 172.21.15.29:0/4278161672 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658010510 con 0x7f1660156410 2024-01-24T14:10:22.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f16477fe700 1 -- 172.21.15.29:0/4278161672 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1658010670 con 0x7f1660156410 2024-01-24T14:10:22.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f16660d0700 1 -- 172.21.15.29:0/4278161672 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16600bfe80 con 0x7f1660156410 2024-01-24T14:10:22.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f16477fe700 1 -- 172.21.15.29:0/4278161672 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16580109f0 con 0x7f1660156410 2024-01-24T14:10:22.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.876+0000 7f16660d0700 1 -- 172.21.15.29:0/4278161672 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166014f230 con 0x7f1660156410 2024-01-24T14:10:22.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.878+0000 7f16660d0700 1 -- 172.21.15.29:0/4278161672 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16600c0640 con 0x7f1660156410 2024-01-24T14:10:22.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.878+0000 7f16477fe700 1 -- 172.21.15.29:0/4278161672 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f165801b400 con 0x7f1660156410 2024-01-24T14:10:22.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.878+0000 7f16477fe700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f164c05f2f0 0x7f164c0617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:22.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.878+0000 7f16477fe700 1 -- 172.21.15.29:0/4278161672 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f165807a400 con 0x7f1660156410 2024-01-24T14:10:22.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.879+0000 7f165f7fe700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f164c05f2f0 0x7f164c0617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:22.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.879+0000 7f165f7fe700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f164c05f2f0 0x7f164c0617b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f165000bbc0 tx=0x7f1650008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:22.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:22.883+0000 7f16477fe700 1 -- 172.21.15.29:0/4278161672 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f165804b1d0 con 0x7f1660156410 2024-01-24T14:10:23.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.091+0000 7f16660d0700 1 -- 172.21.15.29:0/4278161672 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f16600475e0 con 0x7f164c05f2f0 2024-01-24T14:10:23.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.095+0000 7f16477fe700 1 -- 172.21.15.29:0/4278161672 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f16600475e0 con 0x7f164c05f2f0 2024-01-24T14:10:23.096 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2024-01-24T14:10:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.097+0000 7f16660d0700 1 -- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f164c05f2f0 msgr2=0x7f164c0617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.097+0000 7f16660d0700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f164c05f2f0 0x7f164c0617b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f165000bbc0 tx=0x7f1650008040).stop 2024-01-24T14:10:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.097+0000 7f16660d0700 1 -- 172.21.15.29:0/4278161672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1660156410 msgr2=0x7f16600c2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.097+0000 7f16660d0700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1660156410 0x7f16600c2160 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f165800c7c0 tx=0x7f1658001560).stop 2024-01-24T14:10:23.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.098+0000 7f16660d0700 1 -- 172.21.15.29:0/4278161672 shutdown_connections 2024-01-24T14:10:23.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.098+0000 7f16660d0700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f164c05f2f0 0x7f164c0617b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.098+0000 7f16660d0700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1660156410 0x7f16600c2160 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.098+0000 7f16660d0700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f166006a3d0 0x7f16600c0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.098+0000 7f16660d0700 1 --2- 172.21.15.29:0/4278161672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1660151c70 0x7f166014ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.098+0000 7f16660d0700 1 -- 172.21.15.29:0/4278161672 >> 172.21.15.29:0/4278161672 conn(0x7f1660067b20 msgr2=0x7f16600a92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:23.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.098+0000 7f16660d0700 1 -- 172.21.15.29:0/4278161672 shutdown_connections 2024-01-24T14:10:23.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.098+0000 7f16660d0700 1 -- 172.21.15.29:0/4278161672 wait complete. 2024-01-24T14:10:23.111 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2024-01-24T14:10:23.111 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir9 1M 2024-01-24T14:10:23.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.246+0000 7f26ad618700 1 Processor -- start 2024-01-24T14:10:23.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.246+0000 7f26ad618700 1 -- start start 2024-01-24T14:10:23.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26ad618700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f26a8064e90 0x7f26a80c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:23.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26ad618700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a80c7f70 0x7f26a8155420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:23.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26ad618700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a8154f90 con 0x7f26a8151e50 2024-01-24T14:10:23.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26ad618700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a8155910 con 0x7f26a8064e90 2024-01-24T14:10:23.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26ad618700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80a9f10 con 0x7f26a80c7f70 2024-01-24T14:10:23.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26a67fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f26a8064e90 0x7f26a80c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26a67fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f26a8064e90 0x7f26a80c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46230/0 (socket says 172.21.15.29:46230) 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26a67fc700 1 -- 172.21.15.29:0/2043893659 learned_addr learned my addr 172.21.15.29:0/2043893659 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26a77fe700 1 --2- 172.21.15.29:0/2043893659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a80c7f70 0x7f26a8155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26a67fc700 1 -- 172.21.15.29:0/2043893659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a80c7f70 msgr2=0x7f26a8155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26a67fc700 1 --2- 172.21.15.29:0/2043893659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a80c7f70 0x7f26a8155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26a67fc700 1 -- 172.21.15.29:0/2043893659 >> v1:172.21.15.29:6789/0 conn(0x7f26a8151e50 legacy=0x7f26a8064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26a67fc700 1 -- 172.21.15.29:0/2043893659 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a80aa1f0 con 0x7f26a8064e90 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.247+0000 7f26a77fe700 1 --2- 172.21.15.29:0/2043893659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a80c7f70 0x7f26a8155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.248+0000 7f26a67fc700 1 --2- 172.21.15.29:0/2043893659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a8064e90 0x7f26a80c53c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f2690006fd0 tx=0x7f26900090f0).ready entity=mon.0 client_cookie=dd10cf5100f5d3a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.248+0000 7f26a5ffb700 1 -- 172.21.15.29:0/2043893659 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269000b600 con 0x7f26a8064e90 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.248+0000 7f26a5ffb700 1 -- 172.21.15.29:0/2043893659 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269000b760 con 0x7f26a8064e90 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.248+0000 7f26a5ffb700 1 -- 172.21.15.29:0/2043893659 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269000bae0 con 0x7f26a8064e90 2024-01-24T14:10:23.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.248+0000 7f26ad618700 1 -- 172.21.15.29:0/2043893659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a8064e90 msgr2=0x7f26a80c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.248+0000 7f26ad618700 1 --2- 172.21.15.29:0/2043893659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a8064e90 0x7f26a80c53c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f2690006fd0 tx=0x7f26900090f0).stop 2024-01-24T14:10:23.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.248+0000 7f26ad618700 1 -- 172.21.15.29:0/2043893659 shutdown_connections 2024-01-24T14:10:23.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.248+0000 7f26ad618700 1 --2- 172.21.15.29:0/2043893659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a80c7f70 0x7f26a8155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.248+0000 7f26ad618700 1 --2- 172.21.15.29:0/2043893659 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a8064e90 0x7f26a80c53c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.248+0000 7f26ad618700 1 -- 172.21.15.29:0/2043893659 >> 172.21.15.29:0/2043893659 conn(0x7f26a80cb880 msgr2=0x7f26a80c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:23.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.248+0000 7f26ad618700 1 -- 172.21.15.29:0/2043893659 shutdown_connections 2024-01-24T14:10:23.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.248+0000 7f26ad618700 1 -- 172.21.15.29:0/2043893659 wait complete. 2024-01-24T14:10:23.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.250+0000 7f26ad618700 1 Processor -- start 2024-01-24T14:10:23.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.250+0000 7f26ad618700 1 -- start start 2024-01-24T14:10:23.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26ad618700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a8064e90 0x7f26a81546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:23.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26ad618700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a80c7f70 0x7f26a80aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:23.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26ad618700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a8151e50 0x7f26a814d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:23.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26ad618700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80aef60 con 0x7f26a80c7f70 2024-01-24T14:10:23.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26ad618700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a8154c80 con 0x7f26a8151e50 2024-01-24T14:10:23.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26ad618700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80afac0 con 0x7f26a8064e90 2024-01-24T14:10:23.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a67fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a80c7f70 0x7f26a80aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:23.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a67fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a80c7f70 0x7f26a80aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:46242/0 (socket says 172.21.15.29:46242) 2024-01-24T14:10:23.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a67fc700 1 -- 172.21.15.29:0/397342785 learned_addr learned my addr 172.21.15.29:0/397342785 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:23.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a77fe700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a8151e50 0x7f26a814d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:23.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a6ffd700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a8064e90 0x7f26a81546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:23.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a67fc700 1 -- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a8064e90 msgr2=0x7f26a81546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a67fc700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a8064e90 0x7f26a81546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a67fc700 1 -- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a8151e50 msgr2=0x7f26a814d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a67fc700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a8151e50 0x7f26a814d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a67fc700 1 -- 172.21.15.29:0/397342785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2690009370 con 0x7f26a80c7f70 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a77fe700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a8151e50 0x7f26a814d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a6ffd700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a8064e90 0x7f26a81546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.251+0000 7f26a67fc700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a80c7f70 0x7f26a80aff50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f26a8062f40 tx=0x7f2690017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.252+0000 7f269effd700 1 -- 172.21.15.29:0/397342785 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269000b8d0 con 0x7f26a80c7f70 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.252+0000 7f269effd700 1 -- 172.21.15.29:0/397342785 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269000ba30 con 0x7f26a80c7f70 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.252+0000 7f269effd700 1 -- 172.21.15.29:0/397342785 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269001b660 con 0x7f26a80c7f70 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.252+0000 7f26ad618700 1 -- 172.21.15.29:0/397342785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a80af190 con 0x7f26a80c7f70 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.252+0000 7f26ad618700 1 -- 172.21.15.29:0/397342785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a80af750 con 0x7f26a80c7f70 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.253+0000 7f26ad618700 1 -- 172.21.15.29:0/397342785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a80aa710 con 0x7f26a80c7f70 2024-01-24T14:10:23.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.253+0000 7f269effd700 1 -- 172.21.15.29:0/397342785 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f269001ba50 con 0x7f26a80c7f70 2024-01-24T14:10:23.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.254+0000 7f269effd700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f268805f2c0 0x7f2688061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:23.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.254+0000 7f269effd700 1 -- 172.21.15.29:0/397342785 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f269007c070 con 0x7f26a80c7f70 2024-01-24T14:10:23.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.254+0000 7f26a6ffd700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f268805f2c0 0x7f2688061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:23.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.255+0000 7f26a6ffd700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f268805f2c0 0x7f2688061780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f26a81551f0 tx=0x7f2698009040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:23.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.258+0000 7f269effd700 1 -- 172.21.15.29:0/397342785 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f269004cf60 con 0x7f26a80c7f70 2024-01-24T14:10:23.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.470+0000 7f26ad618700 1 -- 172.21.15.29:0/397342785 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f26a8083310 con 0x7f268805f2c0 2024-01-24T14:10:23.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.474+0000 7f269effd700 1 -- 172.21.15.29:0/397342785 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f26a8083310 con 0x7f268805f2c0 2024-01-24T14:10:23.475 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2024-01-24T14:10:23.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.476+0000 7f26ad618700 1 -- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f268805f2c0 msgr2=0x7f2688061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.476+0000 7f26ad618700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f268805f2c0 0x7f2688061780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f26a81551f0 tx=0x7f2698009040).stop 2024-01-24T14:10:23.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.477+0000 7f26ad618700 1 -- 172.21.15.29:0/397342785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a80c7f70 msgr2=0x7f26a80aff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.477+0000 7f26ad618700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a80c7f70 0x7f26a80aff50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f26a8062f40 tx=0x7f2690017000).stop 2024-01-24T14:10:23.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.477+0000 7f26ad618700 1 -- 172.21.15.29:0/397342785 shutdown_connections 2024-01-24T14:10:23.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.477+0000 7f26ad618700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a8151e50 0x7f26a814d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.477+0000 7f26ad618700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a8064e90 0x7f26a81546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.477+0000 7f26ad618700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f268805f2c0 0x7f2688061780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.477+0000 7f26ad618700 1 --2- 172.21.15.29:0/397342785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a80c7f70 0x7f26a80aff50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.477+0000 7f26ad618700 1 -- 172.21.15.29:0/397342785 >> 172.21.15.29:0/397342785 conn(0x7f26a80cb880 msgr2=0x7f26a8155a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:23.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.477+0000 7f26ad618700 1 -- 172.21.15.29:0/397342785 shutdown_connections 2024-01-24T14:10:23.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.477+0000 7f26ad618700 1 -- 172.21.15.29:0/397342785 wait complete. 2024-01-24T14:10:23.490 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2024-01-24T14:10:23.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:23.491 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2024-01-24T14:10:23.511 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:23.511 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:23.511 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:23.511 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:23.512 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:23.512 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:23.512 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:23.512 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:23.512 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:23.512 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:23.512 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:23.512 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-14_00_00_UTC', 'scheduled-2024-01-24-14_01_00_UTC', 'scheduled-2024-01-24-14_02_00_UTC', 'scheduled-2024-01-24-14_03_00_UTC', 'scheduled-2024-01-24-14_04_00_UTC', 'scheduled-2024-01-24-14_05_00_UTC', 'scheduled-2024-01-24-14_06_00_UTC', 'scheduled-2024-01-24-14_07_00_UTC', 'scheduled-2024-01-24-14_08_00_UTC', 'scheduled-2024-01-24-14_09_00_UTC', 'scheduled-2024-01-24-14_10_00_UTC'] 2024-01-24T14:10:23.512 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir0 2024-01-24T14:10:23.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.688+0000 7f5cabfd4700 1 Processor -- start 2024-01-24T14:10:23.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.688+0000 7f5cabfd4700 1 -- start start 2024-01-24T14:10:23.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.689+0000 7f5cabfd4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ca4066450 0x7f5ca4066830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:23.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.689+0000 7f5cabfd4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152540 0x7f5ca41529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:23.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.689+0000 7f5cabfd4700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca4153300 con 0x7f5ca4155a10 2024-01-24T14:10:23.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.689+0000 7f5cabfd4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca4067b10 con 0x7f5ca4066450 2024-01-24T14:10:23.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.689+0000 7f5cabfd4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca4067df0 con 0x7f5ca4152540 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.689+0000 7f5ca9d70700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ca4066450 0x7f5ca4066830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.690+0000 7f5ca9d70700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ca4066450 0x7f5ca4066830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54100/0 (socket says 172.21.15.29:54100) 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.690+0000 7f5ca9d70700 1 -- 172.21.15.29:0/815878996 learned_addr learned my addr 172.21.15.29:0/815878996 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.690+0000 7f5ca956f700 1 --2- 172.21.15.29:0/815878996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152540 0x7f5ca41529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.690+0000 7f5ca9d70700 1 -- 172.21.15.29:0/815878996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152540 msgr2=0x7f5ca41529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.690+0000 7f5ca9d70700 1 --2- 172.21.15.29:0/815878996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152540 0x7f5ca41529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.690+0000 7f5ca9d70700 1 -- 172.21.15.29:0/815878996 >> v1:172.21.15.29:6789/0 conn(0x7f5ca4155a10 legacy=0x7f5ca4155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.690+0000 7f5ca9d70700 1 -- 172.21.15.29:0/815878996 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca40680d0 con 0x7f5ca4066450 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.690+0000 7f5ca9d70700 1 --2- 172.21.15.29:0/815878996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ca4066450 0x7f5ca4066830 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f5c98006fd0 tx=0x7f5c980090f0).ready entity=mon.0 client_cookie=bf7bdbab3b3913ae server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.690+0000 7f5ca8d6e700 1 -- 172.21.15.29:0/815878996 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800b600 con 0x7f5ca4066450 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.690+0000 7f5ca8d6e700 1 -- 172.21.15.29:0/815878996 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9800b760 con 0x7f5ca4066450 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.690+0000 7f5ca8d6e700 1 -- 172.21.15.29:0/815878996 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800bae0 con 0x7f5ca4066450 2024-01-24T14:10:23.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.691+0000 7f5cabfd4700 1 -- 172.21.15.29:0/815878996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ca4066450 msgr2=0x7f5ca4066830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.691+0000 7f5cabfd4700 1 --2- 172.21.15.29:0/815878996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ca4066450 0x7f5ca4066830 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f5c98006fd0 tx=0x7f5c980090f0).stop 2024-01-24T14:10:23.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.691+0000 7f5cabfd4700 1 -- 172.21.15.29:0/815878996 shutdown_connections 2024-01-24T14:10:23.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.691+0000 7f5cabfd4700 1 --2- 172.21.15.29:0/815878996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ca4066450 0x7f5ca4066830 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.691+0000 7f5cabfd4700 1 --2- 172.21.15.29:0/815878996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152540 0x7f5ca41529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.691+0000 7f5cabfd4700 1 -- 172.21.15.29:0/815878996 >> 172.21.15.29:0/815878996 conn(0x7f5ca40c8bd0 msgr2=0x7f5ca40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:23.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.691+0000 7f5cabfd4700 1 -- 172.21.15.29:0/815878996 shutdown_connections 2024-01-24T14:10:23.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.691+0000 7f5cabfd4700 1 -- 172.21.15.29:0/815878996 wait complete. 2024-01-24T14:10:23.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.692+0000 7f5cabfd4700 1 Processor -- start 2024-01-24T14:10:23.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.692+0000 7f5cabfd4700 1 -- start start 2024-01-24T14:10:23.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.693+0000 7f5cabfd4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ca4066450 0x7f5ca41381d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:23.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.693+0000 7f5cabfd4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152540 0x7f5ca41370d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:23.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.693+0000 7f5cabfd4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ca4155a10 0x7f5ca41375c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:23.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.693+0000 7f5cabfd4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca4137ab0 con 0x7f5ca4155a10 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.693+0000 7f5cabfd4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca413b9f0 con 0x7f5ca4066450 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.693+0000 7f5cabfd4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca4138b70 con 0x7f5ca4152540 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.693+0000 7f5caa571700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ca4155a10 0x7f5ca41375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.693+0000 7f5caa571700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ca4155a10 0x7f5ca41375c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54112/0 (socket says 172.21.15.29:54112) 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.693+0000 7f5caa571700 1 -- 172.21.15.29:0/2862771113 learned_addr learned my addr 172.21.15.29:0/2862771113 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.693+0000 7f5ca956f700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152540 0x7f5ca41370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.693+0000 7f5ca9d70700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ca4066450 0x7f5ca41381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.694+0000 7f5caa571700 1 -- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152540 msgr2=0x7f5ca41370d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.694+0000 7f5caa571700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152540 0x7f5ca41370d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.694+0000 7f5caa571700 1 -- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ca4066450 msgr2=0x7f5ca41381d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.694+0000 7f5caa571700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ca4066450 0x7f5ca41381d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.694+0000 7f5caa571700 1 -- 172.21.15.29:0/2862771113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c98009370 con 0x7f5ca4155a10 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.694+0000 7f5caa571700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ca4155a10 0x7f5ca41375c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f5c9c002230 tx=0x7f5c9c0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:23.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.694+0000 7f5c95ffb700 1 -- 172.21.15.29:0/2862771113 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00e4a0 con 0x7f5ca4155a10 2024-01-24T14:10:23.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.694+0000 7f5c95ffb700 1 -- 172.21.15.29:0/2862771113 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c00e600 con 0x7f5ca4155a10 2024-01-24T14:10:23.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.694+0000 7f5c95ffb700 1 -- 172.21.15.29:0/2862771113 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00e980 con 0x7f5ca4155a10 2024-01-24T14:10:23.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.694+0000 7f5cabfd4700 1 -- 172.21.15.29:0/2862771113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca4138e50 con 0x7f5ca4155a10 2024-01-24T14:10:23.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.694+0000 7f5cabfd4700 1 -- 172.21.15.29:0/2862771113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca40abc00 con 0x7f5ca4155a10 2024-01-24T14:10:23.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.695+0000 7f5cabfd4700 1 -- 172.21.15.29:0/2862771113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca40abe20 con 0x7f5ca4155a10 2024-01-24T14:10:23.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.696+0000 7f5c95ffb700 1 -- 172.21.15.29:0/2862771113 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f5c9c01b3b0 con 0x7f5ca4155a10 2024-01-24T14:10:23.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.696+0000 7f5c95ffb700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5c8805f2c0 0x7f5c88061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:23.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.696+0000 7f5c95ffb700 1 -- 172.21.15.29:0/2862771113 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f5c9c079c80 con 0x7f5ca4155a10 2024-01-24T14:10:23.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.696+0000 7f5ca9d70700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5c8805f2c0 0x7f5c88061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:23.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.697+0000 7f5ca9d70700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5c8805f2c0 0x7f5c88061780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f5c98009130 tx=0x7f5c98017000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:23.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.700+0000 7f5c95ffb700 1 -- 172.21.15.29:0/2862771113 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5c9c048b00 con 0x7f5ca4155a10 2024-01-24T14:10:23.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.907+0000 7f5cabfd4700 1 -- 172.21.15.29:0/2862771113 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ca413b7e0 con 0x7f5c8805f2c0 2024-01-24T14:10:23.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.910+0000 7f5c95ffb700 1 -- 172.21.15.29:0/2862771113 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f5ca413b7e0 con 0x7f5c8805f2c0 2024-01-24T14:10:23.910 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:23.910 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:44", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:23.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.911+0000 7f5cabfd4700 1 -- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5c8805f2c0 msgr2=0x7f5c88061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.911+0000 7f5cabfd4700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5c8805f2c0 0x7f5c88061780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f5c98009130 tx=0x7f5c98017000).stop 2024-01-24T14:10:23.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.911+0000 7f5cabfd4700 1 -- 172.21.15.29:0/2862771113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ca4155a10 msgr2=0x7f5ca41375c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:23.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.911+0000 7f5cabfd4700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ca4155a10 0x7f5ca41375c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f5c9c002230 tx=0x7f5c9c0015e0).stop 2024-01-24T14:10:23.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.911+0000 7f5cabfd4700 1 -- 172.21.15.29:0/2862771113 shutdown_connections 2024-01-24T14:10:23.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.911+0000 7f5cabfd4700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5c8805f2c0 0x7f5c88061780 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.911+0000 7f5cabfd4700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ca4155a10 0x7f5ca41375c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.911+0000 7f5cabfd4700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ca4066450 0x7f5ca41381d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.911+0000 7f5cabfd4700 1 --2- 172.21.15.29:0/2862771113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152540 0x7f5ca41370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:23.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.911+0000 7f5cabfd4700 1 -- 172.21.15.29:0/2862771113 >> 172.21.15.29:0/2862771113 conn(0x7f5ca40c8bd0 msgr2=0x7f5ca4155e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:23.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.912+0000 7f5cabfd4700 1 -- 172.21.15.29:0/2862771113 shutdown_connections 2024-01-24T14:10:23.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:23.912+0000 7f5cabfd4700 1 -- 172.21.15.29:0/2862771113 wait complete. 2024-01-24T14:10:23.925 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:44", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:23.925 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T13:59:44', 'first': '2024-01-24T14:00:00', 'last': '2024-01-24T14:10:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-01-24T14:10:23.925 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-01-24T14:10:23.926 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-01-24T14:10:23.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:23.926 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2024-01-24T14:10:23.946 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:23.946 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:23.946 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:23.946 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:23.946 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:23.946 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:23.947 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:23.947 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:23.947 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:23.947 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:23.947 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:23.947 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-14_00_00_UTC', 'scheduled-2024-01-24-14_01_00_UTC', 'scheduled-2024-01-24-14_02_00_UTC', 'scheduled-2024-01-24-14_03_00_UTC', 'scheduled-2024-01-24-14_04_00_UTC', 'scheduled-2024-01-24-14_05_00_UTC', 'scheduled-2024-01-24-14_06_00_UTC', 'scheduled-2024-01-24-14_07_00_UTC', 'scheduled-2024-01-24-14_08_00_UTC', 'scheduled-2024-01-24-14_09_00_UTC', 'scheduled-2024-01-24-14_10_00_UTC'] 2024-01-24T14:10:23.947 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir1 2024-01-24T14:10:24.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.125+0000 7fb0331b1700 1 Processor -- start 2024-01-24T14:10:24.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.125+0000 7fb0331b1700 1 -- start start 2024-01-24T14:10:24.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.126+0000 7fb0331b1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb02c13a6f0 0x7fb02c068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.126+0000 7fb0331b1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb02c0638c0 0x7fb02c066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.126+0000 7fb0331b1700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb02c0643f0 con 0x7fb02c066940 2024-01-24T14:10:24.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.126+0000 7fb0331b1700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb02c0bdb30 con 0x7fb02c0638c0 2024-01-24T14:10:24.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.126+0000 7fb0331b1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb02c0be3d0 con 0x7fb02c13a6f0 2024-01-24T14:10:24.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.126+0000 7fb02bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb02c0638c0 0x7fb02c066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.126+0000 7fb02bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb02c0638c0 0x7fb02c066450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54124/0 (socket says 172.21.15.29:54124) 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.126+0000 7fb02bfff700 1 -- 172.21.15.29:0/2786321570 learned_addr learned my addr 172.21.15.29:0/2786321570 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.126+0000 7fb03174e700 1 --1- 172.21.15.29:0/2786321570 >> v1:172.21.15.29:6789/0 conn(0x7fb02c066940 0x7fb02c0be860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44204/0 (socket says 172.21.15.29:44204) 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.127+0000 7fb030f4d700 1 --2- 172.21.15.29:0/2786321570 >> v2:172.21.15.163:3300/0 conn(0x7fb02c13a6f0 0x7fb02c068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.127+0000 7fb02bfff700 1 -- 172.21.15.29:0/2786321570 >> v2:172.21.15.163:3300/0 conn(0x7fb02c13a6f0 msgr2=0x7fb02c068660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.127+0000 7fb02bfff700 1 --2- 172.21.15.29:0/2786321570 >> v2:172.21.15.163:3300/0 conn(0x7fb02c13a6f0 0x7fb02c068660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.127+0000 7fb02bfff700 1 -- 172.21.15.29:0/2786321570 >> v1:172.21.15.29:6789/0 conn(0x7fb02c066940 legacy=0x7fb02c0be860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.127+0000 7fb02bfff700 1 -- 172.21.15.29:0/2786321570 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c066dd0 con 0x7fb02c0638c0 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.127+0000 7fb02bfff700 1 --2- 172.21.15.29:0/2786321570 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02c0638c0 0x7fb02c066450 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fb02c0695b0 tx=0x7fb0180090f0).ready entity=mon.0 client_cookie=7a2ede134d31f1b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.127+0000 7fb02b7fe700 1 -- 172.21.15.29:0/2786321570 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01800b600 con 0x7fb02c0638c0 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.127+0000 7fb02b7fe700 1 -- 172.21.15.29:0/2786321570 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01800b760 con 0x7fb02c0638c0 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.127+0000 7fb02b7fe700 1 -- 172.21.15.29:0/2786321570 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01800bae0 con 0x7fb02c0638c0 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.128+0000 7fb0331b1700 1 -- 172.21.15.29:0/2786321570 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02c0638c0 msgr2=0x7fb02c066450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.128+0000 7fb0331b1700 1 --2- 172.21.15.29:0/2786321570 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02c0638c0 0x7fb02c066450 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fb02c0695b0 tx=0x7fb0180090f0).stop 2024-01-24T14:10:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.128+0000 7fb0331b1700 1 -- 172.21.15.29:0/2786321570 shutdown_connections 2024-01-24T14:10:24.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.128+0000 7fb0331b1700 1 --2- 172.21.15.29:0/2786321570 >> v2:172.21.15.163:3300/0 conn(0x7fb02c13a6f0 0x7fb02c068660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.128+0000 7fb0331b1700 1 --2- 172.21.15.29:0/2786321570 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02c0638c0 0x7fb02c066450 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.128+0000 7fb0331b1700 1 -- 172.21.15.29:0/2786321570 >> 172.21.15.29:0/2786321570 conn(0x7fb02c0c8bd0 msgr2=0x7fb02c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:24.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.128+0000 7fb0331b1700 1 -- 172.21.15.29:0/2786321570 shutdown_connections 2024-01-24T14:10:24.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.128+0000 7fb0331b1700 1 -- 172.21.15.29:0/2786321570 wait complete. 2024-01-24T14:10:24.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.129+0000 7fb0331b1700 1 Processor -- start 2024-01-24T14:10:24.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.129+0000 7fb0331b1700 1 -- start start 2024-01-24T14:10:24.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb0331b1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb02c0638c0 0x7fb02c138c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb0331b1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02c066940 0x7fb02c156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb0331b1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb02c13a6f0 0x7fb02c0c3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb0331b1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c0abec0 con 0x7fb02c066940 2024-01-24T14:10:24.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb0331b1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c13d110 con 0x7fb02c0638c0 2024-01-24T14:10:24.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb0331b1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c13c7e0 con 0x7fb02c13a6f0 2024-01-24T14:10:24.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb02bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02c066940 0x7fb02c156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb02bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02c066940 0x7fb02c156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54126/0 (socket says 172.21.15.29:54126) 2024-01-24T14:10:24.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb02bfff700 1 -- 172.21.15.29:0/2277076387 learned_addr learned my addr 172.21.15.29:0/2277076387 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb030f4d700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb02c0638c0 0x7fb02c138c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb03174e700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb02c13a6f0 0x7fb02c0c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb02bfff700 1 -- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb02c13a6f0 msgr2=0x7fb02c0c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb02bfff700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb02c13a6f0 0x7fb02c0c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb02bfff700 1 -- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb02c0638c0 msgr2=0x7fb02c138c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb02bfff700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb02c0638c0 0x7fb02c138c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb02bfff700 1 -- 172.21.15.29:0/2277076387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb018009370 con 0x7fb02c066940 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.130+0000 7fb03174e700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb02c13a6f0 0x7fb02c0c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.131+0000 7fb02bfff700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02c066940 0x7fb02c156bf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fb018009130 tx=0x7fb018017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.131+0000 7fb028ff9700 1 -- 172.21.15.29:0/2277076387 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01800b8d0 con 0x7fb02c066940 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.131+0000 7fb028ff9700 1 -- 172.21.15.29:0/2277076387 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01800ba30 con 0x7fb02c066940 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.131+0000 7fb0331b1700 1 -- 172.21.15.29:0/2277076387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c13cac0 con 0x7fb02c066940 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.131+0000 7fb028ff9700 1 -- 172.21.15.29:0/2277076387 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01801c670 con 0x7fb02c066940 2024-01-24T14:10:24.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.131+0000 7fb0331b1700 1 -- 172.21.15.29:0/2277076387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c139180 con 0x7fb02c066940 2024-01-24T14:10:24.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.132+0000 7fb0331b1700 1 -- 172.21.15.29:0/2277076387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02c06ae00 con 0x7fb02c066940 2024-01-24T14:10:24.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.132+0000 7fb028ff9700 1 -- 172.21.15.29:0/2277076387 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fb01801c9e0 con 0x7fb02c066940 2024-01-24T14:10:24.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.133+0000 7fb028ff9700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb01005f2c0 0x7fb010061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.133+0000 7fb028ff9700 1 -- 172.21.15.29:0/2277076387 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fb01807c470 con 0x7fb02c066940 2024-01-24T14:10:24.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.133+0000 7fb030f4d700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb01005f2c0 0x7fb010061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.133+0000 7fb030f4d700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb01005f2c0 0x7fb010061780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb020006fd0 tx=0x7fb020008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:24.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.137+0000 7fb028ff9700 1 -- 172.21.15.29:0/2277076387 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb01804d360 con 0x7fb02c066940 2024-01-24T14:10:24.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.344+0000 7fb0331b1700 1 -- 172.21.15.29:0/2277076387 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb02c0475e0 con 0x7fb01005f2c0 2024-01-24T14:10:24.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.346+0000 7fb028ff9700 1 -- 172.21.15.29:0/2277076387 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fb02c0475e0 con 0x7fb01005f2c0 2024-01-24T14:10:24.347 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:24.347 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:45", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:24.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.349+0000 7fb0331b1700 1 -- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb01005f2c0 msgr2=0x7fb010061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:24.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.349+0000 7fb0331b1700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb01005f2c0 0x7fb010061780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb020006fd0 tx=0x7fb020008040).stop 2024-01-24T14:10:24.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.349+0000 7fb0331b1700 1 -- 172.21.15.29:0/2277076387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02c066940 msgr2=0x7fb02c156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:24.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.349+0000 7fb0331b1700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02c066940 0x7fb02c156bf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fb018009130 tx=0x7fb018017000).stop 2024-01-24T14:10:24.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.349+0000 7fb0331b1700 1 -- 172.21.15.29:0/2277076387 shutdown_connections 2024-01-24T14:10:24.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.349+0000 7fb0331b1700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb02c13a6f0 0x7fb02c0c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.349+0000 7fb0331b1700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb02c0638c0 0x7fb02c138c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.349+0000 7fb0331b1700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb01005f2c0 0x7fb010061780 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.349+0000 7fb0331b1700 1 --2- 172.21.15.29:0/2277076387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02c066940 0x7fb02c156bf0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.349+0000 7fb0331b1700 1 -- 172.21.15.29:0/2277076387 >> 172.21.15.29:0/2277076387 conn(0x7fb02c0c8bd0 msgr2=0x7fb02c0c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:24.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.349+0000 7fb0331b1700 1 -- 172.21.15.29:0/2277076387 shutdown_connections 2024-01-24T14:10:24.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.349+0000 7fb0331b1700 1 -- 172.21.15.29:0/2277076387 wait complete. 2024-01-24T14:10:24.362 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:45", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:24.363 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T13:59:45', 'first': '2024-01-24T14:00:00', 'last': '2024-01-24T14:10:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-01-24T14:10:24.363 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-01-24T14:10:24.363 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-01-24T14:10:24.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:24.363 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2024-01-24T14:10:24.383 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:24.383 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:24.383 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:24.383 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:24.383 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:24.383 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:24.383 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:24.383 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:24.384 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:24.384 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:24.384 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:24.384 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-14_00_00_UTC', 'scheduled-2024-01-24-14_01_00_UTC', 'scheduled-2024-01-24-14_02_00_UTC', 'scheduled-2024-01-24-14_03_00_UTC', 'scheduled-2024-01-24-14_04_00_UTC', 'scheduled-2024-01-24-14_05_00_UTC', 'scheduled-2024-01-24-14_06_00_UTC', 'scheduled-2024-01-24-14_07_00_UTC', 'scheduled-2024-01-24-14_08_00_UTC', 'scheduled-2024-01-24-14_09_00_UTC', 'scheduled-2024-01-24-14_10_00_UTC'] 2024-01-24T14:10:24.384 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir2 2024-01-24T14:10:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.561+0000 7fcd9aa49700 1 Processor -- start 2024-01-24T14:10:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.561+0000 7fcd9aa49700 1 -- start start 2024-01-24T14:10:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd9aa49700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcd9413a6f0 0x7fcd94068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd9aa49700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd940638c0 0x7fcd94066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd9aa49700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcd94066940 0x7fcd940be860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd9aa49700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd940643f0 con 0x7fcd94066940 2024-01-24T14:10:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd9aa49700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd940bdb30 con 0x7fcd9413a6f0 2024-01-24T14:10:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd9aa49700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940be3d0 con 0x7fcd940638c0 2024-01-24T14:10:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd98fe6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcd94066940 0x7fcd940be860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd98fe6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcd94066940 0x7fcd940be860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54138/0 (socket says 172.21.15.29:54138) 2024-01-24T14:10:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd98fe6700 1 -- 172.21.15.29:0/2671480884 learned_addr learned my addr 172.21.15.29:0/2671480884 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd8bfff700 1 --2- 172.21.15.29:0/2671480884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd940638c0 0x7fcd94066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd93fff700 1 --2- 172.21.15.29:0/2671480884 >> v2:172.21.15.163:3300/0 conn(0x7fcd9413a6f0 0x7fcd94068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd98fe6700 1 -- 172.21.15.29:0/2671480884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd940638c0 msgr2=0x7fcd94066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd98fe6700 1 --2- 172.21.15.29:0/2671480884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd940638c0 0x7fcd94066450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd98fe6700 1 -- 172.21.15.29:0/2671480884 >> v2:172.21.15.163:3300/0 conn(0x7fcd9413a6f0 msgr2=0x7fcd94068660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd98fe6700 1 --2- 172.21.15.29:0/2671480884 >> v2:172.21.15.163:3300/0 conn(0x7fcd9413a6f0 0x7fcd94068660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.562+0000 7fcd98fe6700 1 -- 172.21.15.29:0/2671480884 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd94066dd0 con 0x7fcd94066940 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.563+0000 7fcd98fe6700 1 --2- 172.21.15.29:0/2671480884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd94066940 0x7fcd940be860 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fcd940695b0 tx=0x7fcd8c009840).ready entity=mon.0 client_cookie=9525e4b680ffe4c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.563+0000 7fcd937fe700 1 -- 172.21.15.29:0/2671480884 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00e070 con 0x7fcd94066940 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.563+0000 7fcd937fe700 1 -- 172.21.15.29:0/2671480884 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c00b970 con 0x7fcd94066940 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.563+0000 7fcd937fe700 1 -- 172.21.15.29:0/2671480884 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00bcf0 con 0x7fcd94066940 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.563+0000 7fcd9aa49700 1 -- 172.21.15.29:0/2671480884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd94066940 msgr2=0x7fcd940be860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.563+0000 7fcd9aa49700 1 --2- 172.21.15.29:0/2671480884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd94066940 0x7fcd940be860 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fcd940695b0 tx=0x7fcd8c009840).stop 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.563+0000 7fcd9aa49700 1 -- 172.21.15.29:0/2671480884 shutdown_connections 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.563+0000 7fcd9aa49700 1 --2- 172.21.15.29:0/2671480884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd94066940 0x7fcd940be860 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.563+0000 7fcd9aa49700 1 --2- 172.21.15.29:0/2671480884 >> v2:172.21.15.163:3300/0 conn(0x7fcd9413a6f0 0x7fcd94068660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.563+0000 7fcd9aa49700 1 --2- 172.21.15.29:0/2671480884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd940638c0 0x7fcd94066450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.563+0000 7fcd9aa49700 1 -- 172.21.15.29:0/2671480884 >> 172.21.15.29:0/2671480884 conn(0x7fcd940c8bd0 msgr2=0x7fcd940c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:24.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.564+0000 7fcd9aa49700 1 -- 172.21.15.29:0/2671480884 shutdown_connections 2024-01-24T14:10:24.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.564+0000 7fcd9aa49700 1 -- 172.21.15.29:0/2671480884 wait complete. 2024-01-24T14:10:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.565+0000 7fcd9aa49700 1 Processor -- start 2024-01-24T14:10:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.565+0000 7fcd9aa49700 1 -- start start 2024-01-24T14:10:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.565+0000 7fcd9aa49700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd940638c0 0x7fcd94139570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd9aa49700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd94066940 0x7fcd940ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd9aa49700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd9413a6f0 0x7fcd9413d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd9aa49700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd941592f0 con 0x7fcd940638c0 2024-01-24T14:10:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd9aa49700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd941595d0 con 0x7fcd94066940 2024-01-24T14:10:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd9aa49700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd941598b0 con 0x7fcd9413a6f0 2024-01-24T14:10:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd93fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd940638c0 0x7fcd94139570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd93fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd940638c0 0x7fcd94139570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54146/0 (socket says 172.21.15.29:54146) 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd93fff700 1 -- 172.21.15.29:0/1725406336 learned_addr learned my addr 172.21.15.29:0/1725406336 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd8bfff700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd94066940 0x7fcd940ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd98fe6700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd9413a6f0 0x7fcd9413d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd93fff700 1 -- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd9413a6f0 msgr2=0x7fcd9413d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd93fff700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd9413a6f0 0x7fcd9413d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd93fff700 1 -- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd94066940 msgr2=0x7fcd940ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd93fff700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd94066940 0x7fcd940ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd93fff700 1 -- 172.21.15.29:0/1725406336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c009ae0 con 0x7fcd940638c0 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd93fff700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd940638c0 0x7fcd94139570 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fcd8400e930 tx=0x7fcd84009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd90ff9700 1 -- 172.21.15.29:0/1725406336 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84012600 con 0x7fcd940638c0 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd90ff9700 1 -- 172.21.15.29:0/1725406336 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd84012760 con 0x7fcd940638c0 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd9aa49700 1 -- 172.21.15.29:0/1725406336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd94159b90 con 0x7fcd940638c0 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.566+0000 7fcd90ff9700 1 -- 172.21.15.29:0/1725406336 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84012ae0 con 0x7fcd940638c0 2024-01-24T14:10:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.567+0000 7fcd9aa49700 1 -- 172.21.15.29:0/1725406336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd94139a60 con 0x7fcd940638c0 2024-01-24T14:10:24.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.568+0000 7fcd9aa49700 1 -- 172.21.15.29:0/1725406336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9415a1e0 con 0x7fcd940638c0 2024-01-24T14:10:24.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.568+0000 7fcd90ff9700 1 -- 172.21.15.29:0/1725406336 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fcd8401b400 con 0x7fcd940638c0 2024-01-24T14:10:24.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.569+0000 7fcd90ff9700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd7405f2c0 0x7fcd74061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:24.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.569+0000 7fcd90ff9700 1 -- 172.21.15.29:0/1725406336 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fcd8407aa80 con 0x7fcd940638c0 2024-01-24T14:10:24.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.569+0000 7fcd8bfff700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd7405f2c0 0x7fcd74061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:24.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.569+0000 7fcd8bfff700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd7405f2c0 0x7fcd74061780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fcd940c2560 tx=0x7fcd7c009060).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.573+0000 7fcd90ff9700 1 -- 172.21.15.29:0/1725406336 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcd8404b910 con 0x7fcd940638c0 2024-01-24T14:10:24.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.782+0000 7fcd9aa49700 1 -- 172.21.15.29:0/1725406336 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd94159740 con 0x7fcd7405f2c0 2024-01-24T14:10:24.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.784+0000 7fcd90ff9700 1 -- 172.21.15.29:0/1725406336 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fcd94159740 con 0x7fcd7405f2c0 2024-01-24T14:10:24.784 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:24.785 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:45", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:24.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.786+0000 7fcd9aa49700 1 -- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd7405f2c0 msgr2=0x7fcd74061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:24.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.786+0000 7fcd9aa49700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd7405f2c0 0x7fcd74061780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fcd940c2560 tx=0x7fcd7c009060).stop 2024-01-24T14:10:24.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.786+0000 7fcd9aa49700 1 -- 172.21.15.29:0/1725406336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd940638c0 msgr2=0x7fcd94139570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:24.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.786+0000 7fcd9aa49700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd940638c0 0x7fcd94139570 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fcd8400e930 tx=0x7fcd84009060).stop 2024-01-24T14:10:24.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.786+0000 7fcd9aa49700 1 -- 172.21.15.29:0/1725406336 shutdown_connections 2024-01-24T14:10:24.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.786+0000 7fcd9aa49700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd9413a6f0 0x7fcd9413d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.786+0000 7fcd9aa49700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd7405f2c0 0x7fcd74061780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.786+0000 7fcd9aa49700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd940638c0 0x7fcd94139570 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.786+0000 7fcd9aa49700 1 --2- 172.21.15.29:0/1725406336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd94066940 0x7fcd940ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:24.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.786+0000 7fcd9aa49700 1 -- 172.21.15.29:0/1725406336 >> 172.21.15.29:0/1725406336 conn(0x7fcd940c8bd0 msgr2=0x7fcd940c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:24.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.786+0000 7fcd9aa49700 1 -- 172.21.15.29:0/1725406336 shutdown_connections 2024-01-24T14:10:24.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:24.786+0000 7fcd9aa49700 1 -- 172.21.15.29:0/1725406336 wait complete. 2024-01-24T14:10:24.800 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:45", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:24.800 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T13:59:45', 'first': '2024-01-24T14:00:00', 'last': '2024-01-24T14:10:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-01-24T14:10:24.800 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-01-24T14:10:24.800 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-01-24T14:10:24.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:24.801 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2024-01-24T14:10:24.820 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:24.820 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:24.820 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:24.820 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:24.820 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:24.820 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:24.821 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:24.821 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:24.821 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:24.821 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:24.821 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:24.822 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-14_00_00_UTC', 'scheduled-2024-01-24-14_01_00_UTC', 'scheduled-2024-01-24-14_02_00_UTC', 'scheduled-2024-01-24-14_03_00_UTC', 'scheduled-2024-01-24-14_04_00_UTC', 'scheduled-2024-01-24-14_05_00_UTC', 'scheduled-2024-01-24-14_06_00_UTC', 'scheduled-2024-01-24-14_07_00_UTC', 'scheduled-2024-01-24-14_08_00_UTC', 'scheduled-2024-01-24-14_09_00_UTC', 'scheduled-2024-01-24-14_10_00_UTC'] 2024-01-24T14:10:24.822 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir3 2024-01-24T14:10:25.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.000+0000 7f442281f700 1 Processor -- start 2024-01-24T14:10:25.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.000+0000 7f442281f700 1 -- start start 2024-01-24T14:10:25.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.001+0000 7f442281f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f441c0bf870 0x7f441c068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.001+0000 7f442281f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f441c066450 0x7f441c0668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.001+0000 7f442281f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441c0643f0 con 0x7f441c13a6f0 2024-01-24T14:10:25.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.001+0000 7f442281f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441c0bdb30 con 0x7f441c066450 2024-01-24T14:10:25.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.001+0000 7f442281f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441c0be3d0 con 0x7f441c0bf870 2024-01-24T14:10:25.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.001+0000 7f4420dbc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f441c066450 0x7f441c0668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.002+0000 7f4420dbc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f441c066450 0x7f441c0668a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54156/0 (socket says 172.21.15.29:54156) 2024-01-24T14:10:25.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.002+0000 7f4420dbc700 1 -- 172.21.15.29:0/2210290715 learned_addr learned my addr 172.21.15.29:0/2210290715 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:25.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.002+0000 7f4413fff700 1 --2- 172.21.15.29:0/2210290715 >> v2:172.21.15.163:3300/0 conn(0x7f441c0bf870 0x7f441c068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.002+0000 7f4420dbc700 1 -- 172.21.15.29:0/2210290715 >> v2:172.21.15.163:3300/0 conn(0x7f441c0bf870 msgr2=0x7f441c068660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:25.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.002+0000 7f4420dbc700 1 --2- 172.21.15.29:0/2210290715 >> v2:172.21.15.163:3300/0 conn(0x7f441c0bf870 0x7f441c068660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.002+0000 7f4420dbc700 1 -- 172.21.15.29:0/2210290715 >> v1:172.21.15.29:6789/0 conn(0x7f441c13a6f0 legacy=0x7f441c0638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.002+0000 7f4420dbc700 1 -- 172.21.15.29:0/2210290715 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c066d90 con 0x7f441c066450 2024-01-24T14:10:25.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.002+0000 7f4420dbc700 1 --2- 172.21.15.29:0/2210290715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f441c066450 0x7f441c0668a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f441c0aeda0 tx=0x7f4414009840).ready entity=mon.0 client_cookie=ec6a6ed7cd80156e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:25.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.002+0000 7f441b7fe700 1 -- 172.21.15.29:0/2210290715 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441400e070 con 0x7f441c066450 2024-01-24T14:10:25.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.002+0000 7f441b7fe700 1 -- 172.21.15.29:0/2210290715 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441400ba50 con 0x7f441c066450 2024-01-24T14:10:25.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.002+0000 7f441b7fe700 1 -- 172.21.15.29:0/2210290715 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441400bdd0 con 0x7f441c066450 2024-01-24T14:10:25.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.003+0000 7f442281f700 1 -- 172.21.15.29:0/2210290715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f441c066450 msgr2=0x7f441c0668a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.003+0000 7f442281f700 1 --2- 172.21.15.29:0/2210290715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f441c066450 0x7f441c0668a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f441c0aeda0 tx=0x7f4414009840).stop 2024-01-24T14:10:25.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.003+0000 7f442281f700 1 -- 172.21.15.29:0/2210290715 shutdown_connections 2024-01-24T14:10:25.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.003+0000 7f442281f700 1 --2- 172.21.15.29:0/2210290715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f441c066450 0x7f441c0668a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.003+0000 7f442281f700 1 --2- 172.21.15.29:0/2210290715 >> v2:172.21.15.163:3300/0 conn(0x7f441c0bf870 0x7f441c068660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.003+0000 7f442281f700 1 -- 172.21.15.29:0/2210290715 >> 172.21.15.29:0/2210290715 conn(0x7f441c0c8bd0 msgr2=0x7f441c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:25.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.003+0000 7f442281f700 1 -- 172.21.15.29:0/2210290715 shutdown_connections 2024-01-24T14:10:25.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.003+0000 7f442281f700 1 -- 172.21.15.29:0/2210290715 wait complete. 2024-01-24T14:10:25.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.005+0000 7f442281f700 1 Processor -- start 2024-01-24T14:10:25.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.005+0000 7f442281f700 1 -- start start 2024-01-24T14:10:25.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f442281f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f441c066450 0x7f441c166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f442281f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f441c0bf870 0x7f441c1260d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f442281f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f441c13a6f0 0x7f441c1265c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f442281f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c139770 con 0x7f441c066450 2024-01-24T14:10:25.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f442281f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c156a70 con 0x7f441c0bf870 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f442281f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c155e60 con 0x7f441c13a6f0 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f441bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f441c066450 0x7f441c166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f441bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f441c066450 0x7f441c166aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54160/0 (socket says 172.21.15.29:54160) 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f441bfff700 1 -- 172.21.15.29:0/1112794951 learned_addr learned my addr 172.21.15.29:0/1112794951 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f4413fff700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f441c0bf870 0x7f441c1260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f4420dbc700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f441c13a6f0 0x7f441c1265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f441bfff700 1 -- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f441c13a6f0 msgr2=0x7f441c1265c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f441bfff700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f441c13a6f0 0x7f441c1265c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f441bfff700 1 -- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f441c0bf870 msgr2=0x7f441c1260d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f441bfff700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f441c0bf870 0x7f441c1260d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f441bfff700 1 -- 172.21.15.29:0/1112794951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414009ae0 con 0x7f441c066450 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f4420dbc700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f441c13a6f0 0x7f441c1265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:10:25.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f4413fff700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f441c0bf870 0x7f441c1260d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:25.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f441bfff700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f441c066450 0x7f441c166aa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f440c00c440 tx=0x7f440c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:25.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.006+0000 7f4418ff9700 1 -- 172.21.15.29:0/1112794951 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c007020 con 0x7f441c066450 2024-01-24T14:10:25.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.007+0000 7f4418ff9700 1 -- 172.21.15.29:0/1112794951 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440c007180 con 0x7f441c066450 2024-01-24T14:10:25.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.007+0000 7f442281f700 1 -- 172.21.15.29:0/1112794951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c156140 con 0x7f441c066450 2024-01-24T14:10:25.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.007+0000 7f4418ff9700 1 -- 172.21.15.29:0/1112794951 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c007520 con 0x7f441c066450 2024-01-24T14:10:25.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.007+0000 7f442281f700 1 -- 172.21.15.29:0/1112794951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c156700 con 0x7f441c066450 2024-01-24T14:10:25.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.008+0000 7f442281f700 1 -- 172.21.15.29:0/1112794951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c06ae00 con 0x7f441c066450 2024-01-24T14:10:25.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.008+0000 7f4418ff9700 1 -- 172.21.15.29:0/1112794951 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f440c0068b0 con 0x7f441c066450 2024-01-24T14:10:25.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.009+0000 7f4418ff9700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43fc05f2c0 0x7f43fc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.009+0000 7f4418ff9700 1 -- 172.21.15.29:0/1112794951 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f440c079b40 con 0x7f441c066450 2024-01-24T14:10:25.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.009+0000 7f4413fff700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43fc05f2c0 0x7f43fc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.009+0000 7f4413fff700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43fc05f2c0 0x7f43fc061780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f441c13e110 tx=0x7f4404008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:25.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.013+0000 7f4418ff9700 1 -- 172.21.15.29:0/1112794951 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f440c047b00 con 0x7f441c066450 2024-01-24T14:10:25.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.222+0000 7f442281f700 1 -- 172.21.15.29:0/1112794951 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f441c0475e0 con 0x7f43fc05f2c0 2024-01-24T14:10:25.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.223+0000 7f4418ff9700 1 -- 172.21.15.29:0/1112794951 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f441c0475e0 con 0x7f43fc05f2c0 2024-01-24T14:10:25.224 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:25.224 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:46", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:25.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.225+0000 7f442281f700 1 -- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43fc05f2c0 msgr2=0x7f43fc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.225+0000 7f442281f700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43fc05f2c0 0x7f43fc061780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f441c13e110 tx=0x7f4404008040).stop 2024-01-24T14:10:25.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.225+0000 7f442281f700 1 -- 172.21.15.29:0/1112794951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f441c066450 msgr2=0x7f441c166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.225+0000 7f442281f700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f441c066450 0x7f441c166aa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f440c00c440 tx=0x7f440c007e00).stop 2024-01-24T14:10:25.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.225+0000 7f442281f700 1 -- 172.21.15.29:0/1112794951 shutdown_connections 2024-01-24T14:10:25.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.225+0000 7f442281f700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f441c13a6f0 0x7f441c1265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.225+0000 7f442281f700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43fc05f2c0 0x7f43fc061780 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.225+0000 7f442281f700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f441c066450 0x7f441c166aa0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.225+0000 7f442281f700 1 --2- 172.21.15.29:0/1112794951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f441c0bf870 0x7f441c1260d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.225+0000 7f442281f700 1 -- 172.21.15.29:0/1112794951 >> 172.21.15.29:0/1112794951 conn(0x7f441c0c8bd0 msgr2=0x7f441c0c3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:25.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.226+0000 7f442281f700 1 -- 172.21.15.29:0/1112794951 shutdown_connections 2024-01-24T14:10:25.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.226+0000 7f442281f700 1 -- 172.21.15.29:0/1112794951 wait complete. 2024-01-24T14:10:25.239 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:46", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:25.239 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T13:59:46', 'first': '2024-01-24T14:00:00', 'last': '2024-01-24T14:10:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-01-24T14:10:25.239 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-01-24T14:10:25.239 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-01-24T14:10:25.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:25.240 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2024-01-24T14:10:25.260 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:25.260 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:25.261 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:25.261 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:25.261 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:25.261 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:25.261 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:25.261 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:25.261 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:25.261 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:25.261 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:25.261 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-14_00_00_UTC', 'scheduled-2024-01-24-14_01_00_UTC', 'scheduled-2024-01-24-14_02_00_UTC', 'scheduled-2024-01-24-14_03_00_UTC', 'scheduled-2024-01-24-14_04_00_UTC', 'scheduled-2024-01-24-14_05_00_UTC', 'scheduled-2024-01-24-14_06_00_UTC', 'scheduled-2024-01-24-14_07_00_UTC', 'scheduled-2024-01-24-14_08_00_UTC', 'scheduled-2024-01-24-14_09_00_UTC', 'scheduled-2024-01-24-14_10_00_UTC'] 2024-01-24T14:10:25.261 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir4 2024-01-24T14:10:25.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.439+0000 7f014863c700 1 Processor -- start 2024-01-24T14:10:25.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.439+0000 7f014863c700 1 -- start start 2024-01-24T14:10:25.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.440+0000 7f014863c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0140150970 0x7f0140150d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.440+0000 7f014863c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0140069ee0 0x7f0140154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.440+0000 7f014863c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0140069a50 con 0x7f014006a340 2024-01-24T14:10:25.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.440+0000 7f014863c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01400c2b20 con 0x7f0140150970 2024-01-24T14:10:25.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.440+0000 7f014863c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0140151240 con 0x7f0140069ee0 2024-01-24T14:10:25.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.440+0000 7f01463d8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0140150970 0x7f0140150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.440+0000 7f01463d8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0140150970 0x7f0140150d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54162/0 (socket says 172.21.15.29:54162) 2024-01-24T14:10:25.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.440+0000 7f01463d8700 1 -- 172.21.15.29:0/1315736396 learned_addr learned my addr 172.21.15.29:0/1315736396 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.440+0000 7f0145bd7700 1 --2- 172.21.15.29:0/1315736396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0140069ee0 0x7f0140154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.441+0000 7f01463d8700 1 -- 172.21.15.29:0/1315736396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0140069ee0 msgr2=0x7f0140154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.441+0000 7f01463d8700 1 --2- 172.21.15.29:0/1315736396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0140069ee0 0x7f0140154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.441+0000 7f01463d8700 1 -- 172.21.15.29:0/1315736396 >> v1:172.21.15.163:6789/0 conn(0x7f014006a340 legacy=0x7f01400c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.441+0000 7f01463d8700 1 -- 172.21.15.29:0/1315736396 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140155510 con 0x7f0140150970 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.441+0000 7f01463d8700 1 --2- 172.21.15.29:0/1315736396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0140150970 0x7f0140150d50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f013c009830 tx=0x7f013c0090f0).ready entity=mon.0 client_cookie=814082acdf865b7c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.441+0000 7f01453d6700 1 -- 172.21.15.29:0/1315736396 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00b580 con 0x7f0140150970 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.441+0000 7f01453d6700 1 -- 172.21.15.29:0/1315736396 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c00b6e0 con 0x7f0140150970 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.441+0000 7f01453d6700 1 -- 172.21.15.29:0/1315736396 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00ba60 con 0x7f0140150970 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.441+0000 7f014863c700 1 -- 172.21.15.29:0/1315736396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0140150970 msgr2=0x7f0140150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.441+0000 7f014863c700 1 --2- 172.21.15.29:0/1315736396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0140150970 0x7f0140150d50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f013c009830 tx=0x7f013c0090f0).stop 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.442+0000 7f014863c700 1 -- 172.21.15.29:0/1315736396 shutdown_connections 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.442+0000 7f014863c700 1 --2- 172.21.15.29:0/1315736396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0140150970 0x7f0140150d50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.442+0000 7f014863c700 1 --2- 172.21.15.29:0/1315736396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0140069ee0 0x7f0140154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.442+0000 7f014863c700 1 -- 172.21.15.29:0/1315736396 >> 172.21.15.29:0/1315736396 conn(0x7f0140067b20 msgr2=0x7f01400638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:25.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.442+0000 7f014863c700 1 -- 172.21.15.29:0/1315736396 shutdown_connections 2024-01-24T14:10:25.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.442+0000 7f014863c700 1 -- 172.21.15.29:0/1315736396 wait complete. 2024-01-24T14:10:25.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.443+0000 7f014863c700 1 Processor -- start 2024-01-24T14:10:25.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.443+0000 7f014863c700 1 -- start start 2024-01-24T14:10:25.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f014863c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0140069ee0 0x7f014013cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f014863c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014006a340 0x7f01400c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f014863c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0140150970 0x7f01400a9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f014863c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0140143360 con 0x7f0140150970 2024-01-24T14:10:25.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f014863c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0140143640 con 0x7f0140069ee0 2024-01-24T14:10:25.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f014863c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0140143920 con 0x7f014006a340 2024-01-24T14:10:25.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f0146bd9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0140150970 0x7f01400a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f0146bd9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0140150970 0x7f01400a9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54168/0 (socket says 172.21.15.29:54168) 2024-01-24T14:10:25.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f0146bd9700 1 -- 172.21.15.29:0/3961757981 learned_addr learned my addr 172.21.15.29:0/3961757981 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f0145bd7700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014006a340 0x7f01400c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f01463d8700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0140069ee0 0x7f014013cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f0146bd9700 1 -- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014006a340 msgr2=0x7f01400c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f0146bd9700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014006a340 0x7f01400c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f0146bd9700 1 -- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0140069ee0 msgr2=0x7f014013cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f0146bd9700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0140069ee0 0x7f014013cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.444+0000 7f0146bd9700 1 -- 172.21.15.29:0/3961757981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c009370 con 0x7f0140150970 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.445+0000 7f0146bd9700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0140150970 0x7f01400a9220 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f013800c7c0 tx=0x7f0138001560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.445+0000 7f01367fc700 1 -- 172.21.15.29:0/3961757981 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013800e4a0 con 0x7f0140150970 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.445+0000 7f01367fc700 1 -- 172.21.15.29:0/3961757981 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013800e600 con 0x7f0140150970 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.445+0000 7f014863c700 1 -- 172.21.15.29:0/3961757981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0140143c00 con 0x7f0140150970 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.445+0000 7f01367fc700 1 -- 172.21.15.29:0/3961757981 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013800e980 con 0x7f0140150970 2024-01-24T14:10:25.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.445+0000 7f014863c700 1 -- 172.21.15.29:0/3961757981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01400ac3c0 con 0x7f0140150970 2024-01-24T14:10:25.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.446+0000 7f014863c700 1 -- 172.21.15.29:0/3961757981 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01400475e0 con 0x7f0140150970 2024-01-24T14:10:25.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.446+0000 7f01367fc700 1 -- 172.21.15.29:0/3961757981 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f0138012400 con 0x7f0140150970 2024-01-24T14:10:25.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.447+0000 7f01367fc700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f012405f2c0 0x7f0124061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.447+0000 7f01367fc700 1 -- 172.21.15.29:0/3961757981 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f0138079cc0 con 0x7f0140150970 2024-01-24T14:10:25.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.447+0000 7f01463d8700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f012405f2c0 0x7f0124061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.448+0000 7f01463d8700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f012405f2c0 0x7f0124061780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f013c009130 tx=0x7f013c017000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:25.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.451+0000 7f01367fc700 1 -- 172.21.15.29:0/3961757981 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0138048b00 con 0x7f0140150970 2024-01-24T14:10:25.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.663+0000 7f014863c700 1 -- 172.21.15.29:0/3961757981 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01401557f0 con 0x7f012405f2c0 2024-01-24T14:10:25.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.665+0000 7f01367fc700 1 -- 172.21.15.29:0/3961757981 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f01401557f0 con 0x7f012405f2c0 2024-01-24T14:10:25.666 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:25.666 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:46", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:25.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.667+0000 7f014863c700 1 -- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f012405f2c0 msgr2=0x7f0124061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.667+0000 7f014863c700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f012405f2c0 0x7f0124061780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f013c009130 tx=0x7f013c017000).stop 2024-01-24T14:10:25.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.667+0000 7f014863c700 1 -- 172.21.15.29:0/3961757981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0140150970 msgr2=0x7f01400a9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.667+0000 7f014863c700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0140150970 0x7f01400a9220 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f013800c7c0 tx=0x7f0138001560).stop 2024-01-24T14:10:25.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.667+0000 7f014863c700 1 -- 172.21.15.29:0/3961757981 shutdown_connections 2024-01-24T14:10:25.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.667+0000 7f014863c700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f012405f2c0 0x7f0124061780 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.667+0000 7f014863c700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0140150970 0x7f01400a9220 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.667+0000 7f014863c700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0140069ee0 0x7f014013cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.667+0000 7f014863c700 1 --2- 172.21.15.29:0/3961757981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014006a340 0x7f01400c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.667+0000 7f014863c700 1 -- 172.21.15.29:0/3961757981 >> 172.21.15.29:0/3961757981 conn(0x7f0140067b20 msgr2=0x7f0140150230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:25.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.667+0000 7f014863c700 1 -- 172.21.15.29:0/3961757981 shutdown_connections 2024-01-24T14:10:25.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.667+0000 7f014863c700 1 -- 172.21.15.29:0/3961757981 wait complete. 2024-01-24T14:10:25.681 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:46", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:25.681 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T13:59:46', 'first': '2024-01-24T14:00:00', 'last': '2024-01-24T14:10:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-01-24T14:10:25.681 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-01-24T14:10:25.681 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-01-24T14:10:25.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:25.681 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2024-01-24T14:10:25.701 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:25.701 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:25.701 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:25.701 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:25.701 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:25.702 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:25.702 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:25.702 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:25.702 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:25.702 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:25.702 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:25.702 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-14_00_00_UTC', 'scheduled-2024-01-24-14_01_00_UTC', 'scheduled-2024-01-24-14_02_00_UTC', 'scheduled-2024-01-24-14_03_00_UTC', 'scheduled-2024-01-24-14_04_00_UTC', 'scheduled-2024-01-24-14_05_00_UTC', 'scheduled-2024-01-24-14_06_00_UTC', 'scheduled-2024-01-24-14_07_00_UTC', 'scheduled-2024-01-24-14_08_00_UTC', 'scheduled-2024-01-24-14_09_00_UTC', 'scheduled-2024-01-24-14_10_00_UTC'] 2024-01-24T14:10:25.702 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir5 2024-01-24T14:10:25.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.882+0000 7f9fb0b28700 1 Processor -- start 2024-01-24T14:10:25.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.882+0000 7f9fb0b28700 1 -- start start 2024-01-24T14:10:25.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.883+0000 7f9fb0b28700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fac0bf870 0x7f9fac068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.883+0000 7f9fb0b28700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9fac066450 0x7f9fac066900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.883+0000 7f9fb0b28700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fac0643f0 con 0x7f9fac13a6f0 2024-01-24T14:10:25.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.883+0000 7f9fb0b28700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fac066df0 con 0x7f9fac066450 2024-01-24T14:10:25.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.883+0000 7f9fb0b28700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac0bdb30 con 0x7f9fac0bf870 2024-01-24T14:10:25.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.883+0000 7f9faa59c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9fac13a6f0 0x7f9fac0638c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44218/0 (socket says 172.21.15.29:44218) 2024-01-24T14:10:25.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.883+0000 7f9faa59c700 1 -- 172.21.15.29:0/2255552000 learned_addr learned my addr 172.21.15.29:0/2255552000 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:25.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.883+0000 7f9faad9d700 1 --2- 172.21.15.29:0/2255552000 >> v2:172.21.15.163:3300/0 conn(0x7f9fac066450 0x7f9fac066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.883+0000 7f9fa9d9b700 1 --2- 172.21.15.29:0/2255552000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fac0bf870 0x7f9fac068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.884+0000 7f9fa959a700 1 -- 172.21.15.29:0/2255552000 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420430606 0 0) 0x7f9fac0643f0 con 0x7f9fac13a6f0 2024-01-24T14:10:25.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.884+0000 7f9fa959a700 1 -- 172.21.15.29:0/2255552000 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f940033f0 con 0x7f9fac13a6f0 2024-01-24T14:10:25.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.884+0000 7f9fa959a700 1 -- 172.21.15.29:0/2255552000 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3213963007 0 0) 0x7f9f940033f0 con 0x7f9fac13a6f0 2024-01-24T14:10:25.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.884+0000 7f9fa9d9b700 1 -- 172.21.15.29:0/2255552000 >> v2:172.21.15.163:3300/0 conn(0x7f9fac066450 msgr2=0x7f9fac066900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:25.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.884+0000 7f9fa9d9b700 1 --2- 172.21.15.29:0/2255552000 >> v2:172.21.15.163:3300/0 conn(0x7f9fac066450 0x7f9fac066900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.884+0000 7f9fa9d9b700 1 -- 172.21.15.29:0/2255552000 >> v1:172.21.15.29:6789/0 conn(0x7f9fac13a6f0 legacy=0x7f9fac0638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.884+0000 7f9fa9d9b700 1 -- 172.21.15.29:0/2255552000 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac0be3d0 con 0x7f9fac0bf870 2024-01-24T14:10:25.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.884+0000 7f9faad9d700 1 --2- 172.21.15.29:0/2255552000 >> v2:172.21.15.163:3300/0 conn(0x7f9fac066450 0x7f9fac066900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:25.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.884+0000 7f9fa9d9b700 1 --2- 172.21.15.29:0/2255552000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fac0bf870 0x7f9fac068660 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f9fa0009760 tx=0x7f9fa0009060).ready entity=mon.2 client_cookie=c2922e6d0c068a94 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:25.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.884+0000 7f9fa959a700 1 -- 172.21.15.29:0/2255552000 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c001470 con 0x7f9fac0bf870 2024-01-24T14:10:25.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.884+0000 7f9fa959a700 1 -- 172.21.15.29:0/2255552000 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa0011400 con 0x7f9fac0bf870 2024-01-24T14:10:25.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.884+0000 7f9fa959a700 1 -- 172.21.15.29:0/2255552000 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0011780 con 0x7f9fac0bf870 2024-01-24T14:10:25.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.885+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2255552000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fac0bf870 msgr2=0x7f9fac068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.885+0000 7f9fb0b28700 1 --2- 172.21.15.29:0/2255552000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fac0bf870 0x7f9fac068660 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f9fa0009760 tx=0x7f9fa0009060).stop 2024-01-24T14:10:25.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.885+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2255552000 shutdown_connections 2024-01-24T14:10:25.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.885+0000 7f9fb0b28700 1 --2- 172.21.15.29:0/2255552000 >> v2:172.21.15.163:3300/0 conn(0x7f9fac066450 0x7f9fac066900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.885+0000 7f9fb0b28700 1 --2- 172.21.15.29:0/2255552000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fac0bf870 0x7f9fac068660 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.885+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2255552000 >> 172.21.15.29:0/2255552000 conn(0x7f9fac0c8bd0 msgr2=0x7f9fac0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:25.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.885+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2255552000 shutdown_connections 2024-01-24T14:10:25.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.885+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2255552000 wait complete. 2024-01-24T14:10:25.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.886+0000 7f9fb0b28700 1 Processor -- start 2024-01-24T14:10:25.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.886+0000 7f9fb0b28700 1 -- start start 2024-01-24T14:10:25.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.887+0000 7f9fb0b28700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fac066450 0x7f9fac153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.887+0000 7f9fb0b28700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fac0bf870 0x7f9fac067c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.887+0000 7f9fb0b28700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fac13a6f0 0x7f9fac0680f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.887+0000 7f9fb0b28700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac13ca40 con 0x7f9fac0bf870 2024-01-24T14:10:25.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.887+0000 7f9fb0b28700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac13c110 con 0x7f9fac13a6f0 2024-01-24T14:10:25.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.887+0000 7f9fb0b28700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac13c3f0 con 0x7f9fac066450 2024-01-24T14:10:25.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.887+0000 7f9fa9d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fac0bf870 0x7f9fac067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.887+0000 7f9fa9d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fac0bf870 0x7f9fac067c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54182/0 (socket says 172.21.15.29:54182) 2024-01-24T14:10:25.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.887+0000 7f9fa9d9b700 1 -- 172.21.15.29:0/2469869163 learned_addr learned my addr 172.21.15.29:0/2469869163 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:25.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.887+0000 7f9faa59c700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fac066450 0x7f9fac153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.887+0000 7f9faad9d700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fac13a6f0 0x7f9fac0680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.888+0000 7f9fa9d9b700 1 -- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fac066450 msgr2=0x7f9fac153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.888+0000 7f9fa9d9b700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fac066450 0x7f9fac153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.888+0000 7f9fa9d9b700 1 -- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fac13a6f0 msgr2=0x7f9fac0680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:25.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.888+0000 7f9fa9d9b700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fac13a6f0 0x7f9fac0680f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:25.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.888+0000 7f9fa9d9b700 1 -- 172.21.15.29:0/2469869163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa4009a20 con 0x7f9fac0bf870 2024-01-24T14:10:25.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.888+0000 7f9fa9d9b700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fac0bf870 0x7f9fac067c00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f9fac0bdc70 tx=0x7f9fa0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:25.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.888+0000 7f9f9a7fc700 1 -- 172.21.15.29:0/2469869163 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0010870 con 0x7f9fac0bf870 2024-01-24T14:10:25.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.888+0000 7f9f9a7fc700 1 -- 172.21.15.29:0/2469869163 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa00109d0 con 0x7f9fac0bf870 2024-01-24T14:10:25.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.888+0000 7f9f9a7fc700 1 -- 172.21.15.29:0/2469869163 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0010d90 con 0x7f9fac0bf870 2024-01-24T14:10:25.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.888+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2469869163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa00092a0 con 0x7f9fac0bf870 2024-01-24T14:10:25.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.888+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2469869163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fac154b60 con 0x7f9fac0bf870 2024-01-24T14:10:25.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.889+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2469869163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fac13cbf0 con 0x7f9fac0bf870 2024-01-24T14:10:25.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.889+0000 7f9f9a7fc700 1 -- 172.21.15.29:0/2469869163 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f9fa001d640 con 0x7f9fac0bf870 2024-01-24T14:10:25.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.890+0000 7f9f9a7fc700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f8c05f2c0 0x7f9f8c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:25.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.890+0000 7f9f9a7fc700 1 -- 172.21.15.29:0/2469869163 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f9fa007bbd0 con 0x7f9fac0bf870 2024-01-24T14:10:25.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.890+0000 7f9faa59c700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f8c05f2c0 0x7f9f8c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:25.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.891+0000 7f9faa59c700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f8c05f2c0 0x7f9f8c061780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9f9c001ed0 tx=0x7f9f9c001520).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:25.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:25.894+0000 7f9f9a7fc700 1 -- 172.21.15.29:0/2469869163 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9fac13cbf0 con 0x7f9fac0bf870 2024-01-24T14:10:26.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.101+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2469869163 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9fac06ae00 con 0x7f9f8c05f2c0 2024-01-24T14:10:26.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.103+0000 7f9f9a7fc700 1 -- 172.21.15.29:0/2469869163 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f9fac06ae00 con 0x7f9f8c05f2c0 2024-01-24T14:10:26.104 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:26.104 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:47", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:26.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.105+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f8c05f2c0 msgr2=0x7f9f8c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.105+0000 7f9fb0b28700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f8c05f2c0 0x7f9f8c061780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9f9c001ed0 tx=0x7f9f9c001520).stop 2024-01-24T14:10:26.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.105+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2469869163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fac0bf870 msgr2=0x7f9fac067c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.105+0000 7f9fb0b28700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fac0bf870 0x7f9fac067c00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f9fac0bdc70 tx=0x7f9fa0016000).stop 2024-01-24T14:10:26.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.105+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2469869163 shutdown_connections 2024-01-24T14:10:26.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.105+0000 7f9fb0b28700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fac13a6f0 0x7f9fac0680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.105+0000 7f9fb0b28700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fac066450 0x7f9fac153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.105+0000 7f9fb0b28700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f8c05f2c0 0x7f9f8c061780 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.105+0000 7f9fb0b28700 1 --2- 172.21.15.29:0/2469869163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fac0bf870 0x7f9fac067c00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.105+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2469869163 >> 172.21.15.29:0/2469869163 conn(0x7f9fac0c8bd0 msgr2=0x7f9fac0c2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:26.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.105+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2469869163 shutdown_connections 2024-01-24T14:10:26.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.105+0000 7f9fb0b28700 1 -- 172.21.15.29:0/2469869163 wait complete. 2024-01-24T14:10:26.119 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:47", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:26.119 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T13:59:47', 'first': '2024-01-24T14:00:00', 'last': '2024-01-24T14:10:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-01-24T14:10:26.119 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-01-24T14:10:26.119 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-01-24T14:10:26.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:26.120 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2024-01-24T14:10:26.139 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:26.139 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:26.139 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:26.139 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:26.139 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:26.140 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:26.140 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:26.140 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:26.140 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:26.140 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:26.140 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:26.141 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-14_00_00_UTC', 'scheduled-2024-01-24-14_01_00_UTC', 'scheduled-2024-01-24-14_02_00_UTC', 'scheduled-2024-01-24-14_03_00_UTC', 'scheduled-2024-01-24-14_04_00_UTC', 'scheduled-2024-01-24-14_05_00_UTC', 'scheduled-2024-01-24-14_06_00_UTC', 'scheduled-2024-01-24-14_07_00_UTC', 'scheduled-2024-01-24-14_08_00_UTC', 'scheduled-2024-01-24-14_09_00_UTC', 'scheduled-2024-01-24-14_10_00_UTC'] 2024-01-24T14:10:26.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir6 2024-01-24T14:10:26.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.318+0000 7f0b86431700 1 Processor -- start 2024-01-24T14:10:26.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.318+0000 7f0b86431700 1 -- start start 2024-01-24T14:10:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.319+0000 7f0b86431700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0b8014fd10 0x7f0b800c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.319+0000 7f0b86431700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b80066b60 0x7f0b80153090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.319+0000 7f0b86431700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b80067a70 con 0x7f0b800684c0 2024-01-24T14:10:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.319+0000 7f0b86431700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b800c3d50 con 0x7f0b8014fd10 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.319+0000 7f0b86431700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b801500f0 con 0x7f0b80066b60 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.319+0000 7f0b7ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0b8014fd10 0x7f0b800c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.319+0000 7f0b7ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0b8014fd10 0x7f0b800c31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54188/0 (socket says 172.21.15.29:54188) 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.319+0000 7f0b7ffff700 1 -- 172.21.15.29:0/3609513855 learned_addr learned my addr 172.21.15.29:0/3609513855 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.319+0000 7f0b849ce700 1 --2- 172.21.15.29:0/3609513855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b80066b60 0x7f0b80153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.320+0000 7f0b7ffff700 1 -- 172.21.15.29:0/3609513855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b80066b60 msgr2=0x7f0b80153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.320+0000 7f0b7ffff700 1 --2- 172.21.15.29:0/3609513855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b80066b60 0x7f0b80153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.320+0000 7f0b7ffff700 1 -- 172.21.15.29:0/3609513855 >> v1:172.21.15.163:6789/0 conn(0x7f0b800684c0 legacy=0x7f0b80066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.320+0000 7f0b7ffff700 1 -- 172.21.15.29:0/3609513855 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b80153580 con 0x7f0b8014fd10 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.320+0000 7f0b7ffff700 1 --2- 172.21.15.29:0/3609513855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b8014fd10 0x7f0b800c31d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f0b68006fd0 tx=0x7f0b680090f0).ready entity=mon.0 client_cookie=de5a53c37efcb594 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.320+0000 7f0b7effd700 1 -- 172.21.15.29:0/3609513855 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800b600 con 0x7f0b8014fd10 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.320+0000 7f0b7effd700 1 -- 172.21.15.29:0/3609513855 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6800b760 con 0x7f0b8014fd10 2024-01-24T14:10:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.320+0000 7f0b7effd700 1 -- 172.21.15.29:0/3609513855 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800bae0 con 0x7f0b8014fd10 2024-01-24T14:10:26.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.320+0000 7f0b86431700 1 -- 172.21.15.29:0/3609513855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b8014fd10 msgr2=0x7f0b800c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.320+0000 7f0b86431700 1 --2- 172.21.15.29:0/3609513855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b8014fd10 0x7f0b800c31d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f0b68006fd0 tx=0x7f0b680090f0).stop 2024-01-24T14:10:26.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.321+0000 7f0b86431700 1 -- 172.21.15.29:0/3609513855 shutdown_connections 2024-01-24T14:10:26.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.321+0000 7f0b86431700 1 --2- 172.21.15.29:0/3609513855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b80066b60 0x7f0b80153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.321+0000 7f0b86431700 1 --2- 172.21.15.29:0/3609513855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b8014fd10 0x7f0b800c31d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.321+0000 7f0b86431700 1 -- 172.21.15.29:0/3609513855 >> 172.21.15.29:0/3609513855 conn(0x7f0b800c8bd0 msgr2=0x7f0b800c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:26.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.321+0000 7f0b86431700 1 -- 172.21.15.29:0/3609513855 shutdown_connections 2024-01-24T14:10:26.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.321+0000 7f0b86431700 1 -- 172.21.15.29:0/3609513855 wait complete. 2024-01-24T14:10:26.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.322+0000 7f0b86431700 1 Processor -- start 2024-01-24T14:10:26.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.322+0000 7f0b86431700 1 -- start start 2024-01-24T14:10:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.323+0000 7f0b86431700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b80066b60 0x7f0b80133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.323+0000 7f0b86431700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b800684c0 0x7f0b80168b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.323+0000 7f0b86431700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b8014fd10 0x7f0b8014e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.323+0000 7f0b86431700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80169d60 con 0x7f0b80066b60 2024-01-24T14:10:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.323+0000 7f0b86431700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80169b30 con 0x7f0b800684c0 2024-01-24T14:10:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.323+0000 7f0b86431700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80063da0 con 0x7f0b8014fd10 2024-01-24T14:10:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.323+0000 7f0b7ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b80066b60 0x7f0b80133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.323+0000 7f0b7ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b80066b60 0x7f0b80133c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54202/0 (socket says 172.21.15.29:54202) 2024-01-24T14:10:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.323+0000 7f0b7ffff700 1 -- 172.21.15.29:0/3857389459 learned_addr learned my addr 172.21.15.29:0/3857389459 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.323+0000 7f0b849ce700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b8014fd10 0x7f0b8014e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.323+0000 7f0b7f7fe700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b800684c0 0x7f0b80168b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.324+0000 7f0b7ffff700 1 -- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b8014fd10 msgr2=0x7f0b8014e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.324+0000 7f0b7ffff700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b8014fd10 0x7f0b8014e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.324+0000 7f0b7ffff700 1 -- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b800684c0 msgr2=0x7f0b80168b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.324+0000 7f0b7ffff700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b800684c0 0x7f0b80168b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.324+0000 7f0b7ffff700 1 -- 172.21.15.29:0/3857389459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b68009370 con 0x7f0b80066b60 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.324+0000 7f0b7ffff700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b80066b60 0x7f0b80133c40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f0b68009f50 tx=0x7f0b68017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.324+0000 7f0b77fff700 1 -- 172.21.15.29:0/3857389459 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800bea0 con 0x7f0b80066b60 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.324+0000 7f0b77fff700 1 -- 172.21.15.29:0/3857389459 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6800b8d0 con 0x7f0b80066b60 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.324+0000 7f0b86431700 1 -- 172.21.15.29:0/3857389459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b80169570 con 0x7f0b80066b60 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.324+0000 7f0b77fff700 1 -- 172.21.15.29:0/3857389459 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6801c670 con 0x7f0b80066b60 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.324+0000 7f0b86431700 1 -- 172.21.15.29:0/3857389459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b80063a30 con 0x7f0b80066b60 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.325+0000 7f0b86431700 1 -- 172.21.15.29:0/3857389459 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b800c6600 con 0x7f0b80066b60 2024-01-24T14:10:26.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.325+0000 7f0b77fff700 1 -- 172.21.15.29:0/3857389459 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f0b6801c9e0 con 0x7f0b80066b60 2024-01-24T14:10:26.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.326+0000 7f0b77fff700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0b6005f2c0 0x7f0b60061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:26.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.326+0000 7f0b77fff700 1 -- 172.21.15.29:0/3857389459 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f0b6807c320 con 0x7f0b80066b60 2024-01-24T14:10:26.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.326+0000 7f0b7f7fe700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0b6005f2c0 0x7f0b60061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:26.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.327+0000 7f0b7f7fe700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0b6005f2c0 0x7f0b60061780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f0b800aee50 tx=0x7f0b70007d20).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:26.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.330+0000 7f0b77fff700 1 -- 172.21.15.29:0/3857389459 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0b6804d210 con 0x7f0b80066b60 2024-01-24T14:10:26.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.539+0000 7f0b86431700 1 -- 172.21.15.29:0/3857389459 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b80063ee0 con 0x7f0b6005f2c0 2024-01-24T14:10:26.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.541+0000 7f0b77fff700 1 -- 172.21.15.29:0/3857389459 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f0b700034f0 con 0x7f0b6005f2c0 2024-01-24T14:10:26.541 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:26.541 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:47", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.543+0000 7f0b86431700 1 -- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0b6005f2c0 msgr2=0x7f0b60061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.543+0000 7f0b86431700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0b6005f2c0 0x7f0b60061780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f0b800aee50 tx=0x7f0b70007d20).stop 2024-01-24T14:10:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.543+0000 7f0b86431700 1 -- 172.21.15.29:0/3857389459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b80066b60 msgr2=0x7f0b80133c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.543+0000 7f0b86431700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b80066b60 0x7f0b80133c40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f0b68009f50 tx=0x7f0b68017000).stop 2024-01-24T14:10:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.543+0000 7f0b86431700 1 -- 172.21.15.29:0/3857389459 shutdown_connections 2024-01-24T14:10:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.543+0000 7f0b86431700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b8014fd10 0x7f0b8014e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.543+0000 7f0b86431700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0b6005f2c0 0x7f0b60061780 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.543+0000 7f0b86431700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b80066b60 0x7f0b80133c40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.543+0000 7f0b86431700 1 --2- 172.21.15.29:0/3857389459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b800684c0 0x7f0b80168b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.543+0000 7f0b86431700 1 -- 172.21.15.29:0/3857389459 >> 172.21.15.29:0/3857389459 conn(0x7f0b800c8bd0 msgr2=0x7f0b801342a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.543+0000 7f0b86431700 1 -- 172.21.15.29:0/3857389459 shutdown_connections 2024-01-24T14:10:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.543+0000 7f0b86431700 1 -- 172.21.15.29:0/3857389459 wait complete. 2024-01-24T14:10:26.557 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:47", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:26.557 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T13:59:47', 'first': '2024-01-24T14:00:00', 'last': '2024-01-24T14:10:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-01-24T14:10:26.557 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-01-24T14:10:26.557 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-01-24T14:10:26.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:26.557 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2024-01-24T14:10:26.576 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:26.576 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:26.576 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:26.577 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:26.577 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:26.577 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:26.577 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:26.577 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:26.577 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:26.577 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:26.577 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:26.578 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-14_00_00_UTC', 'scheduled-2024-01-24-14_01_00_UTC', 'scheduled-2024-01-24-14_02_00_UTC', 'scheduled-2024-01-24-14_03_00_UTC', 'scheduled-2024-01-24-14_04_00_UTC', 'scheduled-2024-01-24-14_05_00_UTC', 'scheduled-2024-01-24-14_06_00_UTC', 'scheduled-2024-01-24-14_07_00_UTC', 'scheduled-2024-01-24-14_08_00_UTC', 'scheduled-2024-01-24-14_09_00_UTC', 'scheduled-2024-01-24-14_10_00_UTC'] 2024-01-24T14:10:26.578 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir7 2024-01-24T14:10:26.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.756+0000 7f3f6e712700 1 Processor -- start 2024-01-24T14:10:26.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.756+0000 7f3f6e712700 1 -- start start 2024-01-24T14:10:26.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.757+0000 7f3f6e712700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3f6813b4f0 0x7f3f68065d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:26.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.757+0000 7f3f6e712700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f68066280 con 0x7f3f6813b8d0 2024-01-24T14:10:26.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.757+0000 7f3f6e712700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f68152840 con 0x7f3f68062ec0 2024-01-24T14:10:26.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.757+0000 7f3f6e712700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f68153450 con 0x7f3f6813b4f0 2024-01-24T14:10:26.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.757+0000 7f3f677fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3f6813b8d0 0x7f3f680627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44226/0 (socket says 172.21.15.29:44226) 2024-01-24T14:10:26.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.757+0000 7f3f677fe700 1 -- 172.21.15.29:0/1309834797 learned_addr learned my addr 172.21.15.29:0/1309834797 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:26.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.757+0000 7f3f67fff700 1 --2- 172.21.15.29:0/1309834797 >> v2:172.21.15.163:3300/0 conn(0x7f3f6813b4f0 0x7f3f68065d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:26.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.758+0000 7f3f66ffd700 1 -- 172.21.15.29:0/1309834797 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044820546 0 0) 0x7f3f68066280 con 0x7f3f6813b8d0 2024-01-24T14:10:26.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.758+0000 7f3f66ffd700 1 -- 172.21.15.29:0/1309834797 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f500033f0 con 0x7f3f6813b8d0 2024-01-24T14:10:26.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.758+0000 7f3f67fff700 1 -- 172.21.15.29:0/1309834797 >> v1:172.21.15.163:6789/0 conn(0x7f3f68062ec0 legacy=0x7f3f680bce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.758+0000 7f3f67fff700 1 -- 172.21.15.29:0/1309834797 >> v1:172.21.15.29:6789/0 conn(0x7f3f6813b8d0 legacy=0x7f3f680627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.758+0000 7f3f67fff700 1 -- 172.21.15.29:0/1309834797 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f68152b20 con 0x7f3f6813b4f0 2024-01-24T14:10:26.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.758+0000 7f3f67fff700 1 --2- 172.21.15.29:0/1309834797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f6813b4f0 0x7f3f68065d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f3f58009760 tx=0x7f3f58009060).ready entity=mon.1 client_cookie=a99d5235088d9d7e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:26.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.758+0000 7f3f66ffd700 1 -- 172.21.15.29:0/1309834797 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800f510 con 0x7f3f6813b4f0 2024-01-24T14:10:26.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.758+0000 7f3f66ffd700 1 -- 172.21.15.29:0/1309834797 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5800f670 con 0x7f3f6813b4f0 2024-01-24T14:10:26.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.758+0000 7f3f66ffd700 1 -- 172.21.15.29:0/1309834797 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800f9f0 con 0x7f3f6813b4f0 2024-01-24T14:10:26.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.759+0000 7f3f6e712700 1 -- 172.21.15.29:0/1309834797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f6813b4f0 msgr2=0x7f3f68065d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.759+0000 7f3f6e712700 1 --2- 172.21.15.29:0/1309834797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f6813b4f0 0x7f3f68065d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f3f58009760 tx=0x7f3f58009060).stop 2024-01-24T14:10:26.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.759+0000 7f3f6e712700 1 -- 172.21.15.29:0/1309834797 shutdown_connections 2024-01-24T14:10:26.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.759+0000 7f3f6e712700 1 --2- 172.21.15.29:0/1309834797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f6813b4f0 0x7f3f68065d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.759+0000 7f3f6e712700 1 -- 172.21.15.29:0/1309834797 >> 172.21.15.29:0/1309834797 conn(0x7f3f680c7b20 msgr2=0x7f3f680c7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:26.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.759+0000 7f3f6e712700 1 -- 172.21.15.29:0/1309834797 shutdown_connections 2024-01-24T14:10:26.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.759+0000 7f3f6e712700 1 -- 172.21.15.29:0/1309834797 wait complete. 2024-01-24T14:10:26.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.760+0000 7f3f6e712700 1 Processor -- start 2024-01-24T14:10:26.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.760+0000 7f3f6e712700 1 -- start start 2024-01-24T14:10:26.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.761+0000 7f3f6e712700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f68062ec0 0x7f3f680c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:26.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.761+0000 7f3f6e712700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f6813b4f0 0x7f3f68151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:26.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.761+0000 7f3f6e712700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f6813b8d0 0x7f3f681399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:26.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.761+0000 7f3f6e712700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f681651b0 con 0x7f3f6813b8d0 2024-01-24T14:10:26.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.761+0000 7f3f6e712700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68165450 con 0x7f3f68062ec0 2024-01-24T14:10:26.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.761+0000 7f3f6e712700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68165730 con 0x7f3f6813b4f0 2024-01-24T14:10:26.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.761+0000 7f3f6ccaf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f6813b8d0 0x7f3f681399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:26.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.761+0000 7f3f6ccaf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f6813b8d0 0x7f3f681399b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54218/0 (socket says 172.21.15.29:54218) 2024-01-24T14:10:26.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.761+0000 7f3f6ccaf700 1 -- 172.21.15.29:0/640466163 learned_addr learned my addr 172.21.15.29:0/640466163 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.761+0000 7f3f67fff700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f68062ec0 0x7f3f680c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.761+0000 7f3f677fe700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f6813b4f0 0x7f3f68151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.762+0000 7f3f6ccaf700 1 -- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f6813b4f0 msgr2=0x7f3f68151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.762+0000 7f3f6ccaf700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f6813b4f0 0x7f3f68151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.762+0000 7f3f6ccaf700 1 -- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f68062ec0 msgr2=0x7f3f680c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.762+0000 7f3f6ccaf700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f68062ec0 0x7f3f680c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.762+0000 7f3f6ccaf700 1 -- 172.21.15.29:0/640466163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580092a0 con 0x7f3f6813b8d0 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.762+0000 7f3f6ccaf700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f6813b8d0 0x7f3f681399b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f3f60002230 tx=0x7f3f600015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.762+0000 7f3f4ffff700 1 -- 172.21.15.29:0/640466163 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000e4a0 con 0x7f3f6813b8d0 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.762+0000 7f3f4ffff700 1 -- 172.21.15.29:0/640466163 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6000e600 con 0x7f3f6813b8d0 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.762+0000 7f3f6e712700 1 -- 172.21.15.29:0/640466163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f68165a10 con 0x7f3f6813b8d0 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.762+0000 7f3f4ffff700 1 -- 172.21.15.29:0/640466163 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000e980 con 0x7f3f6813b8d0 2024-01-24T14:10:26.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.762+0000 7f3f6e712700 1 -- 172.21.15.29:0/640466163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f680c1690 con 0x7f3f6813b8d0 2024-01-24T14:10:26.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.763+0000 7f3f6e712700 1 -- 172.21.15.29:0/640466163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f68165f30 con 0x7f3f6813b8d0 2024-01-24T14:10:26.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.764+0000 7f3f4ffff700 1 -- 172.21.15.29:0/640466163 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f3f6001b3b0 con 0x7f3f6813b8d0 2024-01-24T14:10:26.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.764+0000 7f3f4ffff700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f5405f2c0 0x7f3f54061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:26.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.764+0000 7f3f4ffff700 1 -- 172.21.15.29:0/640466163 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f3f6007a0d0 con 0x7f3f6813b8d0 2024-01-24T14:10:26.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.764+0000 7f3f67fff700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f5405f2c0 0x7f3f54061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:26.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.765+0000 7f3f67fff700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f5405f2c0 0x7f3f54061780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f3f58009760 tx=0x7f3f58017000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:26.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.768+0000 7f3f4ffff700 1 -- 172.21.15.29:0/640466163 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3f6004afd0 con 0x7f3f6813b8d0 2024-01-24T14:10:26.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.977+0000 7f3f6e712700 1 -- 172.21.15.29:0/640466163 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f680475e0 con 0x7f3f5405f2c0 2024-01-24T14:10:26.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.979+0000 7f3f4ffff700 1 -- 172.21.15.29:0/640466163 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f3f680475e0 con 0x7f3f5405f2c0 2024-01-24T14:10:26.979 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:26.979 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:48", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:26.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.981+0000 7f3f6e712700 1 -- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f5405f2c0 msgr2=0x7f3f54061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.981+0000 7f3f6e712700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f5405f2c0 0x7f3f54061780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f3f58009760 tx=0x7f3f58017000).stop 2024-01-24T14:10:26.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.981+0000 7f3f6e712700 1 -- 172.21.15.29:0/640466163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f6813b8d0 msgr2=0x7f3f681399b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:26.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.981+0000 7f3f6e712700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f6813b8d0 0x7f3f681399b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f3f60002230 tx=0x7f3f600015e0).stop 2024-01-24T14:10:26.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.981+0000 7f3f6e712700 1 -- 172.21.15.29:0/640466163 shutdown_connections 2024-01-24T14:10:26.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.981+0000 7f3f6e712700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f5405f2c0 0x7f3f54061780 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.981+0000 7f3f6e712700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f6813b8d0 0x7f3f681399b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.981+0000 7f3f6e712700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f68062ec0 0x7f3f680c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.981+0000 7f3f6e712700 1 --2- 172.21.15.29:0/640466163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f6813b4f0 0x7f3f68151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:26.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.981+0000 7f3f6e712700 1 -- 172.21.15.29:0/640466163 >> 172.21.15.29:0/640466163 conn(0x7f3f680c7b20 msgr2=0x7f3f680c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:26.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.981+0000 7f3f6e712700 1 -- 172.21.15.29:0/640466163 shutdown_connections 2024-01-24T14:10:26.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:26.981+0000 7f3f6e712700 1 -- 172.21.15.29:0/640466163 wait complete. 2024-01-24T14:10:26.995 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:48", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:26.995 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T13:59:48', 'first': '2024-01-24T14:00:00', 'last': '2024-01-24T14:10:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-01-24T14:10:26.995 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-01-24T14:10:26.995 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-01-24T14:10:26.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:26.995 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2024-01-24T14:10:27.015 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:27.090 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:27.090 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:27.090 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:27.090 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:27.090 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:27.090 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:27.090 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:27.090 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:27.090 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:27.090 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:27.091 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-14_00_00_UTC', 'scheduled-2024-01-24-14_01_00_UTC', 'scheduled-2024-01-24-14_02_00_UTC', 'scheduled-2024-01-24-14_03_00_UTC', 'scheduled-2024-01-24-14_04_00_UTC', 'scheduled-2024-01-24-14_05_00_UTC', 'scheduled-2024-01-24-14_06_00_UTC', 'scheduled-2024-01-24-14_07_00_UTC', 'scheduled-2024-01-24-14_08_00_UTC', 'scheduled-2024-01-24-14_09_00_UTC', 'scheduled-2024-01-24-14_10_00_UTC'] 2024-01-24T14:10:27.091 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir8 2024-01-24T14:10:27.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.230+0000 7ff01fc4e700 1 Processor -- start 2024-01-24T14:10:27.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.230+0000 7ff01fc4e700 1 -- start start 2024-01-24T14:10:27.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01fc4e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff018134ce0 0x7ff0181350c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:27.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01fc4e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff018066450 0x7ff0180668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:27.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01fc4e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff018068720 con 0x7ff018152870 2024-01-24T14:10:27.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01fc4e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01814c9e0 con 0x7ff018134ce0 2024-01-24T14:10:27.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01fc4e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff018066da0 con 0x7ff018066450 2024-01-24T14:10:27.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01d9ea700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff018134ce0 0x7ff0181350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:27.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01d9ea700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff018134ce0 0x7ff0181350c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54228/0 (socket says 172.21.15.29:54228) 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01d9ea700 1 -- 172.21.15.29:0/841860318 learned_addr learned my addr 172.21.15.29:0/841860318 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01d1e9700 1 --2- 172.21.15.29:0/841860318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff018066450 0x7ff0180668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01d9ea700 1 -- 172.21.15.29:0/841860318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff018066450 msgr2=0x7ff0180668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01d9ea700 1 --2- 172.21.15.29:0/841860318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff018066450 0x7ff0180668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01d9ea700 1 -- 172.21.15.29:0/841860318 >> v1:172.21.15.163:6789/0 conn(0x7ff018152870 legacy=0x7ff018152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.231+0000 7ff01d9ea700 1 -- 172.21.15.29:0/841860318 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff018068bb0 con 0x7ff018134ce0 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.232+0000 7ff01d9ea700 1 --2- 172.21.15.29:0/841860318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff018134ce0 0x7ff0181350c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7ff008006fd0 tx=0x7ff0080090f0).ready entity=mon.0 client_cookie=9b7f71dbaa1ce8a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.232+0000 7ff01c9e8700 1 -- 172.21.15.29:0/841860318 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800b600 con 0x7ff018134ce0 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.232+0000 7ff01c9e8700 1 -- 172.21.15.29:0/841860318 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00800b760 con 0x7ff018134ce0 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.232+0000 7ff01c9e8700 1 -- 172.21.15.29:0/841860318 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800bae0 con 0x7ff018134ce0 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.232+0000 7ff01fc4e700 1 -- 172.21.15.29:0/841860318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff018134ce0 msgr2=0x7ff0181350c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.232+0000 7ff01fc4e700 1 --2- 172.21.15.29:0/841860318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff018134ce0 0x7ff0181350c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7ff008006fd0 tx=0x7ff0080090f0).stop 2024-01-24T14:10:27.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.232+0000 7ff01fc4e700 1 -- 172.21.15.29:0/841860318 shutdown_connections 2024-01-24T14:10:27.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.232+0000 7ff01fc4e700 1 --2- 172.21.15.29:0/841860318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff018134ce0 0x7ff0181350c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.232+0000 7ff01fc4e700 1 --2- 172.21.15.29:0/841860318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff018066450 0x7ff0180668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.232+0000 7ff01fc4e700 1 -- 172.21.15.29:0/841860318 >> 172.21.15.29:0/841860318 conn(0x7ff0180c8bd0 msgr2=0x7ff0180c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:27.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.233+0000 7ff01fc4e700 1 -- 172.21.15.29:0/841860318 shutdown_connections 2024-01-24T14:10:27.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.233+0000 7ff01fc4e700 1 -- 172.21.15.29:0/841860318 wait complete. 2024-01-24T14:10:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.234+0000 7ff01fc4e700 1 Processor -- start 2024-01-24T14:10:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.234+0000 7ff01fc4e700 1 -- start start 2024-01-24T14:10:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01fc4e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff018066450 0x7ff018162000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01fc4e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff018134ce0 0x7ff0180c0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01fc4e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff018152870 0x7ff018134690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01fc4e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff018134b80 con 0x7ff018152870 2024-01-24T14:10:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01fc4e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180ac1d0 con 0x7ff018134ce0 2024-01-24T14:10:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01fc4e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180c1f90 con 0x7ff018066450 2024-01-24T14:10:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01e1eb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff018152870 0x7ff018134690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01e1eb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff018152870 0x7ff018134690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54236/0 (socket says 172.21.15.29:54236) 2024-01-24T14:10:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01e1eb700 1 -- 172.21.15.29:0/2678188332 learned_addr learned my addr 172.21.15.29:0/2678188332 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01d9ea700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff018066450 0x7ff018162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01d1e9700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff018134ce0 0x7ff0180c0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01e1eb700 1 -- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff018066450 msgr2=0x7ff018162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01e1eb700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff018066450 0x7ff018162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01e1eb700 1 -- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff018134ce0 msgr2=0x7ff0180c0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01e1eb700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff018134ce0 0x7ff0180c0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01e1eb700 1 -- 172.21.15.29:0/2678188332 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff008009370 con 0x7ff018152870 2024-01-24T14:10:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01e1eb700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff018152870 0x7ff018134690 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7ff010002230 tx=0x7ff0100015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff00dffb700 1 -- 172.21.15.29:0/2678188332 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff010010510 con 0x7ff018152870 2024-01-24T14:10:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff00dffb700 1 -- 172.21.15.29:0/2678188332 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff010010670 con 0x7ff018152870 2024-01-24T14:10:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff00dffb700 1 -- 172.21.15.29:0/2678188332 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0100109f0 con 0x7ff018152870 2024-01-24T14:10:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.235+0000 7ff01fc4e700 1 -- 172.21.15.29:0/2678188332 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0180c2270 con 0x7ff018152870 2024-01-24T14:10:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.236+0000 7ff01fc4e700 1 -- 172.21.15.29:0/2678188332 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0180abe60 con 0x7ff018152870 2024-01-24T14:10:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.237+0000 7ff01fc4e700 1 -- 172.21.15.29:0/2678188332 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0180ac380 con 0x7ff018152870 2024-01-24T14:10:27.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.237+0000 7ff00dffb700 1 -- 172.21.15.29:0/2678188332 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7ff01001b400 con 0x7ff018152870 2024-01-24T14:10:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.238+0000 7ff00dffb700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feffc05f2c0 0x7feffc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.238+0000 7ff00dffb700 1 -- 172.21.15.29:0/2678188332 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7ff01007a490 con 0x7ff018152870 2024-01-24T14:10:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.238+0000 7ff01d9ea700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feffc05f2c0 0x7feffc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.238+0000 7ff01d9ea700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feffc05f2c0 0x7feffc061780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff008009130 tx=0x7ff008017000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:27.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.242+0000 7ff00dffb700 1 -- 172.21.15.29:0/2678188332 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff0180ac380 con 0x7ff018152870 2024-01-24T14:10:27.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.453+0000 7ff01fc4e700 1 -- 172.21.15.29:0/2678188332 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff0180ac340 con 0x7feffc05f2c0 2024-01-24T14:10:27.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.455+0000 7ff00dffb700 1 -- 172.21.15.29:0/2678188332 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7ff0180ac340 con 0x7feffc05f2c0 2024-01-24T14:10:27.455 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:27.455 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:48", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:27.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.457+0000 7ff01fc4e700 1 -- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feffc05f2c0 msgr2=0x7feffc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.457+0000 7ff01fc4e700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feffc05f2c0 0x7feffc061780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff008009130 tx=0x7ff008017000).stop 2024-01-24T14:10:27.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.457+0000 7ff01fc4e700 1 -- 172.21.15.29:0/2678188332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff018152870 msgr2=0x7ff018134690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.457+0000 7ff01fc4e700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff018152870 0x7ff018134690 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7ff010002230 tx=0x7ff0100015e0).stop 2024-01-24T14:10:27.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.457+0000 7ff01fc4e700 1 -- 172.21.15.29:0/2678188332 shutdown_connections 2024-01-24T14:10:27.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.457+0000 7ff01fc4e700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feffc05f2c0 0x7feffc061780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.457+0000 7ff01fc4e700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff018152870 0x7ff018134690 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.457+0000 7ff01fc4e700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff018066450 0x7ff018162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.457+0000 7ff01fc4e700 1 --2- 172.21.15.29:0/2678188332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff018134ce0 0x7ff0180c0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.457+0000 7ff01fc4e700 1 -- 172.21.15.29:0/2678188332 >> 172.21.15.29:0/2678188332 conn(0x7ff0180c8bd0 msgr2=0x7ff01814bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:27.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.457+0000 7ff01fc4e700 1 -- 172.21.15.29:0/2678188332 shutdown_connections 2024-01-24T14:10:27.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.457+0000 7ff01fc4e700 1 -- 172.21.15.29:0/2678188332 wait complete. 2024-01-24T14:10:27.470 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:48", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:27.470 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T13:59:48', 'first': '2024-01-24T14:00:00', 'last': '2024-01-24T14:10:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-01-24T14:10:27.471 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-01-24T14:10:27.471 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-01-24T14:10:27.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:27.471 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2024-01-24T14:10:27.491 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:27.491 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:27.491 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:27.491 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:27.491 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:27.491 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:27.491 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:27.491 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:27.491 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:27.491 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:27.491 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:27.492 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-14_00_00_UTC', 'scheduled-2024-01-24-14_01_00_UTC', 'scheduled-2024-01-24-14_02_00_UTC', 'scheduled-2024-01-24-14_03_00_UTC', 'scheduled-2024-01-24-14_04_00_UTC', 'scheduled-2024-01-24-14_05_00_UTC', 'scheduled-2024-01-24-14_06_00_UTC', 'scheduled-2024-01-24-14_07_00_UTC', 'scheduled-2024-01-24-14_08_00_UTC', 'scheduled-2024-01-24-14_09_00_UTC', 'scheduled-2024-01-24-14_10_00_UTC'] 2024-01-24T14:10:27.492 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir9 2024-01-24T14:10:27.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.670+0000 7f37388a1700 1 Processor -- start 2024-01-24T14:10:27.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.670+0000 7f37388a1700 1 -- start start 2024-01-24T14:10:27.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.671+0000 7f37388a1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37340acaa0 0x7f3734150640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:27.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.671+0000 7f37388a1700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3734062e60 con 0x7f37340c3010 2024-01-24T14:10:27.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.671+0000 7f37388a1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3734062850 con 0x7f3734150260 2024-01-24T14:10:27.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.671+0000 7f37388a1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37340ad030 con 0x7f37340acaa0 2024-01-24T14:10:27.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.671+0000 7f3732d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f37340c3010 0x7f37340632f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44232/0 (socket says 172.21.15.29:44232) 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.671+0000 7f3732d9d700 1 -- 172.21.15.29:0/639681831 learned_addr learned my addr 172.21.15.29:0/639681831 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.671+0000 7f3731d9b700 1 --2- 172.21.15.29:0/639681831 >> v2:172.21.15.163:3300/0 conn(0x7f37340acaa0 0x7f3734150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.671+0000 7f373159a700 1 -- 172.21.15.29:0/639681831 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565260155 0 0) 0x7f3734062e60 con 0x7f37340c3010 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.671+0000 7f373159a700 1 -- 172.21.15.29:0/639681831 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f371c0033f0 con 0x7f37340c3010 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.672+0000 7f373159a700 1 -- 172.21.15.29:0/639681831 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307775075 0 0) 0x7f3734062850 con 0x7f3734150260 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.672+0000 7f373159a700 1 -- 172.21.15.29:0/639681831 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3734062e60 con 0x7f3734150260 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.672+0000 7f3731d9b700 1 -- 172.21.15.29:0/639681831 >> v1:172.21.15.163:6789/0 conn(0x7f3734150260 legacy=0x7f37340aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.672+0000 7f3731d9b700 1 -- 172.21.15.29:0/639681831 >> v1:172.21.15.29:6789/0 conn(0x7f37340c3010 legacy=0x7f37340632f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.672+0000 7f373159a700 1 -- 172.21.15.29:0/639681831 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1081411937 0 0) 0x7f371c0033f0 con 0x7f37340c3010 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.672+0000 7f3731d9b700 1 -- 172.21.15.29:0/639681831 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3734062af0 con 0x7f37340acaa0 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.672+0000 7f3731d9b700 1 --2- 172.21.15.29:0/639681831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340acaa0 0x7f3734150640 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f372800a900 tx=0x7f372800a200).ready entity=mon.1 client_cookie=6ddf9f55b12e8829 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.672+0000 7f373159a700 1 -- 172.21.15.29:0/639681831 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0017f0 con 0x7f37340acaa0 2024-01-24T14:10:27.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.672+0000 7f373159a700 1 -- 172.21.15.29:0/639681831 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3728010410 con 0x7f37340acaa0 2024-01-24T14:10:27.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.672+0000 7f373159a700 1 -- 172.21.15.29:0/639681831 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728010790 con 0x7f37340acaa0 2024-01-24T14:10:27.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.673+0000 7f37388a1700 1 -- 172.21.15.29:0/639681831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340acaa0 msgr2=0x7f3734150640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.673+0000 7f37388a1700 1 --2- 172.21.15.29:0/639681831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340acaa0 0x7f3734150640 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f372800a900 tx=0x7f372800a200).stop 2024-01-24T14:10:27.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.673+0000 7f37388a1700 1 -- 172.21.15.29:0/639681831 shutdown_connections 2024-01-24T14:10:27.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.673+0000 7f37388a1700 1 --2- 172.21.15.29:0/639681831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340acaa0 0x7f3734150640 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.673+0000 7f37388a1700 1 -- 172.21.15.29:0/639681831 >> 172.21.15.29:0/639681831 conn(0x7f37340c7280 msgr2=0x7f37340c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:27.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.673+0000 7f37388a1700 1 -- 172.21.15.29:0/639681831 shutdown_connections 2024-01-24T14:10:27.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.673+0000 7f37388a1700 1 -- 172.21.15.29:0/639681831 wait complete. 2024-01-24T14:10:27.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.674+0000 7f37388a1700 1 Processor -- start 2024-01-24T14:10:27.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.674+0000 7f37388a1700 1 -- start start 2024-01-24T14:10:27.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f37388a1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340acaa0 0x7f37340a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:27.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f37388a1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c3010 0x7f373414d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:27.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f37388a1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3734150260 0x7f37341647c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:27.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f37388a1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37340a1710 con 0x7f3734150260 2024-01-24T14:10:27.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f37388a1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373414eca0 con 0x7f37340acaa0 2024-01-24T14:10:27.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f37388a1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373414e090 con 0x7f37340c3010 2024-01-24T14:10:27.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f3732d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3734150260 0x7f37341647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:27.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f3732d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3734150260 0x7f37341647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54248/0 (socket says 172.21.15.29:54248) 2024-01-24T14:10:27.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f3732d9d700 1 -- 172.21.15.29:0/509033479 learned_addr learned my addr 172.21.15.29:0/509033479 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:27.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f3731d9b700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c3010 0x7f373414d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:27.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f373259c700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340acaa0 0x7f37340a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:27.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f3732d9d700 1 -- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c3010 msgr2=0x7f373414d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f3732d9d700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c3010 0x7f373414d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f3732d9d700 1 -- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340acaa0 msgr2=0x7f37340a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f3732d9d700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340acaa0 0x7f37340a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f3732d9d700 1 -- 172.21.15.29:0/509033479 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372800a440 con 0x7f3734150260 2024-01-24T14:10:27.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f373259c700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340acaa0 0x7f37340a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:10:27.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f3732d9d700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3734150260 0x7f37341647c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f372c00cb90 tx=0x7f372c007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:27.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.675+0000 7f37227fc700 1 -- 172.21.15.29:0/509033479 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c004c40 con 0x7f3734150260 2024-01-24T14:10:27.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.676+0000 7f37227fc700 1 -- 172.21.15.29:0/509033479 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c004da0 con 0x7f3734150260 2024-01-24T14:10:27.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.676+0000 7f37388a1700 1 -- 172.21.15.29:0/509033479 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373414e370 con 0x7f3734150260 2024-01-24T14:10:27.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.676+0000 7f37227fc700 1 -- 172.21.15.29:0/509033479 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0073a0 con 0x7f3734150260 2024-01-24T14:10:27.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.676+0000 7f37388a1700 1 -- 172.21.15.29:0/509033479 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373414e930 con 0x7f3734150260 2024-01-24T14:10:27.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.677+0000 7f37227fc700 1 -- 172.21.15.29:0/509033479 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f372c007790 con 0x7f3734150260 2024-01-24T14:10:27.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.677+0000 7f37388a1700 1 -- 172.21.15.29:0/509033479 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373414ee30 con 0x7f3734150260 2024-01-24T14:10:27.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.677+0000 7f37227fc700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f371405f2c0 0x7f3714061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:27.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.677+0000 7f37227fc700 1 -- 172.21.15.29:0/509033479 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f372c07a640 con 0x7f3734150260 2024-01-24T14:10:27.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.678+0000 7f373259c700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f371405f2c0 0x7f3714061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:27.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.678+0000 7f373259c700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f371405f2c0 0x7f3714061780 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f37240026b0 tx=0x7f3724009040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:27.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.682+0000 7f37227fc700 1 -- 172.21.15.29:0/509033479 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f372c04b530 con 0x7f3734150260 2024-01-24T14:10:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.890+0000 7f37388a1700 1 -- 172.21.15.29:0/509033479 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f37340c4aa0 con 0x7f371405f2c0 2024-01-24T14:10:27.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.892+0000 7f37227fc700 1 -- 172.21.15.29:0/509033479 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f37340c4aa0 con 0x7f371405f2c0 2024-01-24T14:10:27.892 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:27.892 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:48", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.894+0000 7f37388a1700 1 -- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f371405f2c0 msgr2=0x7f3714061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.894+0000 7f37388a1700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f371405f2c0 0x7f3714061780 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f37240026b0 tx=0x7f3724009040).stop 2024-01-24T14:10:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.894+0000 7f37388a1700 1 -- 172.21.15.29:0/509033479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3734150260 msgr2=0x7f37341647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.894+0000 7f37388a1700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3734150260 0x7f37341647c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f372c00cb90 tx=0x7f372c007ba0).stop 2024-01-24T14:10:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.894+0000 7f37388a1700 1 -- 172.21.15.29:0/509033479 shutdown_connections 2024-01-24T14:10:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.894+0000 7f37388a1700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f371405f2c0 0x7f3714061780 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.894+0000 7f37388a1700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3734150260 0x7f37341647c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.894+0000 7f37388a1700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340acaa0 0x7f37340a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.894+0000 7f37388a1700 1 --2- 172.21.15.29:0/509033479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c3010 0x7f373414d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.894+0000 7f37388a1700 1 -- 172.21.15.29:0/509033479 >> 172.21.15.29:0/509033479 conn(0x7f37340c7280 msgr2=0x7f37340c5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.894+0000 7f37388a1700 1 -- 172.21.15.29:0/509033479 shutdown_connections 2024-01-24T14:10:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:27.894+0000 7f37388a1700 1 -- 172.21.15.29:0/509033479 wait complete. 2024-01-24T14:10:27.908 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T13:59:48", "first": "2024-01-24T14:00:00", "last": "2024-01-24T14:10:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-01-24T14:10:27.908 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T13:59:48', 'first': '2024-01-24T14:00:00', 'last': '2024-01-24T14:10:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-01-24T14:10:27.908 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-01-24T14:10:27.908 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-01-24T14:10:27.908 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir0 1M 2024-01-24T14:10:28.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.044+0000 7f9401ee6700 1 Processor -- start 2024-01-24T14:10:28.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.044+0000 7f9401ee6700 1 -- start start 2024-01-24T14:10:28.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.045+0000 7f9401ee6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f93fc150970 0x7f93fc150d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:28.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.045+0000 7f9401ee6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc155060 0x7f93fc0c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:28.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.045+0000 7f9401ee6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc069a50 con 0x7f93fc069ee0 2024-01-24T14:10:28.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.045+0000 7f9401ee6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93fc0c2b20 con 0x7f93fc150970 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.045+0000 7f9401ee6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc151240 con 0x7f93fc155060 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.045+0000 7f93fb7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f93fc150970 0x7f93fc150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.045+0000 7f93fbfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc155060 0x7f93fc0c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.045+0000 7f93faffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f93fc069ee0 0x7f93fc154950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:33062/0 (socket says 172.21.15.29:33062) 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.046+0000 7f93faffd700 1 -- 172.21.15.29:0/3523040380 learned_addr learned my addr 172.21.15.29:0/3523040380 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.046+0000 7f93fa7fc700 1 -- 172.21.15.29:0/3523040380 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113132091 0 0) 0x7f93fc069a50 con 0x7f93fc069ee0 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.046+0000 7f93fa7fc700 1 -- 172.21.15.29:0/3523040380 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93e40033f0 con 0x7f93fc069ee0 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.046+0000 7f93fb7fe700 1 -- 172.21.15.29:0/3523040380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc155060 msgr2=0x7f93fc0c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.046+0000 7f93fb7fe700 1 --2- 172.21.15.29:0/3523040380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc155060 0x7f93fc0c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.046+0000 7f93fb7fe700 1 -- 172.21.15.29:0/3523040380 >> v1:172.21.15.163:6789/0 conn(0x7f93fc069ee0 legacy=0x7f93fc154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.046+0000 7f93fb7fe700 1 -- 172.21.15.29:0/3523040380 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc155510 con 0x7f93fc150970 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.046+0000 7f93fbfff700 1 --2- 172.21.15.29:0/3523040380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc155060 0x7f93fc0c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.046+0000 7f93fb7fe700 1 --2- 172.21.15.29:0/3523040380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc150970 0x7f93fc150d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f93ec009760 tx=0x7f93ec009060).ready entity=mon.1 client_cookie=d00e1b765c24f9a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:28.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.046+0000 7f93fa7fc700 1 -- 172.21.15.29:0/3523040380 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec00d4a0 con 0x7f93fc150970 2024-01-24T14:10:28.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.046+0000 7f93fa7fc700 1 -- 172.21.15.29:0/3523040380 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ec00d600 con 0x7f93fc150970 2024-01-24T14:10:28.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.046+0000 7f93fa7fc700 1 -- 172.21.15.29:0/3523040380 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec00d980 con 0x7f93fc150970 2024-01-24T14:10:28.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.047+0000 7f9401ee6700 1 -- 172.21.15.29:0/3523040380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc150970 msgr2=0x7f93fc150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:28.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.047+0000 7f9401ee6700 1 --2- 172.21.15.29:0/3523040380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc150970 0x7f93fc150d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f93ec009760 tx=0x7f93ec009060).stop 2024-01-24T14:10:28.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.047+0000 7f9401ee6700 1 -- 172.21.15.29:0/3523040380 shutdown_connections 2024-01-24T14:10:28.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.047+0000 7f9401ee6700 1 --2- 172.21.15.29:0/3523040380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc155060 0x7f93fc0c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:28.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.047+0000 7f9401ee6700 1 --2- 172.21.15.29:0/3523040380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc150970 0x7f93fc150d50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:28.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.047+0000 7f9401ee6700 1 -- 172.21.15.29:0/3523040380 >> 172.21.15.29:0/3523040380 conn(0x7f93fc067b20 msgr2=0x7f93fc0638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:28.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.047+0000 7f9401ee6700 1 -- 172.21.15.29:0/3523040380 shutdown_connections 2024-01-24T14:10:28.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.047+0000 7f9401ee6700 1 -- 172.21.15.29:0/3523040380 wait complete. 2024-01-24T14:10:28.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.048+0000 7f9401ee6700 1 Processor -- start 2024-01-24T14:10:28.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.048+0000 7f9401ee6700 1 -- start start 2024-01-24T14:10:28.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.049+0000 7f9401ee6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc069ee0 0x7f93fc1665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.049+0000 7f9401ee6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc150970 0x7f93fc0c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.049+0000 7f9401ee6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93fc155060 0x7f93fc0c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.049+0000 7f9401ee6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0c4f00 con 0x7f93fc155060 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.049+0000 7f9401ee6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc1514b0 con 0x7f93fc069ee0 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.049+0000 7f9401ee6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc151790 con 0x7f93fc150970 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.049+0000 7f93fbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93fc155060 0x7f93fc0c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.049+0000 7f93fbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93fc155060 0x7f93fc0c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54254/0 (socket says 172.21.15.29:54254) 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.049+0000 7f93fbfff700 1 -- 172.21.15.29:0/267047651 learned_addr learned my addr 172.21.15.29:0/267047651 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.049+0000 7f93fb7fe700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc069ee0 0x7f93fc1665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.049+0000 7f93faffd700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc150970 0x7f93fc0c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.050+0000 7f93fbfff700 1 -- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc150970 msgr2=0x7f93fc0c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.050+0000 7f93fbfff700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc150970 0x7f93fc0c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.050+0000 7f93fbfff700 1 -- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc069ee0 msgr2=0x7f93fc1665c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:28.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.050+0000 7f93fbfff700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc069ee0 0x7f93fc1665c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:28.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.050+0000 7f93fbfff700 1 -- 172.21.15.29:0/267047651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f4009a20 con 0x7f93fc155060 2024-01-24T14:10:28.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.050+0000 7f93fbfff700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93fc155060 0x7f93fc0c4a10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f93f4009f50 tx=0x7f93f400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:28.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.050+0000 7f93e37fe700 1 -- 172.21.15.29:0/267047651 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f400caf0 con 0x7f93fc155060 2024-01-24T14:10:28.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.050+0000 7f93e37fe700 1 -- 172.21.15.29:0/267047651 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f400f040 con 0x7f93fc155060 2024-01-24T14:10:28.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.050+0000 7f9401ee6700 1 -- 172.21.15.29:0/267047651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec0092a0 con 0x7f93fc155060 2024-01-24T14:10:28.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.050+0000 7f93e37fe700 1 -- 172.21.15.29:0/267047651 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f400ce60 con 0x7f93fc155060 2024-01-24T14:10:28.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.050+0000 7f9401ee6700 1 -- 172.21.15.29:0/267047651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc151d50 con 0x7f93fc155060 2024-01-24T14:10:28.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.051+0000 7f9401ee6700 1 -- 172.21.15.29:0/267047651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc152270 con 0x7f93fc155060 2024-01-24T14:10:28.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.052+0000 7f93e37fe700 1 -- 172.21.15.29:0/267047651 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f93f4013610 con 0x7f93fc155060 2024-01-24T14:10:28.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.053+0000 7f93e37fe700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f93e805f2c0 0x7f93e8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:28.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.053+0000 7f93e37fe700 1 -- 172.21.15.29:0/267047651 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7f93f407b750 con 0x7f93fc155060 2024-01-24T14:10:28.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.053+0000 7f93fb7fe700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f93e805f2c0 0x7f93e8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:28.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.053+0000 7f93fb7fe700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f93e805f2c0 0x7f93e8061780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f93fc1498e0 tx=0x7f93ec016000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:28.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.057+0000 7f93e37fe700 1 -- 172.21.15.29:0/267047651 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f93f404c700 con 0x7f93fc155060 2024-01-24T14:10:28.249 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:10:28.251 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:10:28.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.263+0000 7f9401ee6700 1 -- 172.21.15.29:0/267047651 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f93fc152270 con 0x7f93e805f2c0 2024-01-24T14:10:28.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.268+0000 7f93e37fe700 1 -- 172.21.15.29:0/267047651 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f93fc152270 con 0x7f93e805f2c0 2024-01-24T14:10:28.268 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /snap_test_dir1/dir0 2024-01-24T14:10:28.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.270+0000 7f9401ee6700 1 -- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f93e805f2c0 msgr2=0x7f93e8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:28.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.270+0000 7f9401ee6700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f93e805f2c0 0x7f93e8061780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f93fc1498e0 tx=0x7f93ec016000).stop 2024-01-24T14:10:28.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.270+0000 7f9401ee6700 1 -- 172.21.15.29:0/267047651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93fc155060 msgr2=0x7f93fc0c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:28.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.270+0000 7f9401ee6700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93fc155060 0x7f93fc0c4a10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f93f4009f50 tx=0x7f93f400b040).stop 2024-01-24T14:10:28.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.270+0000 7f9401ee6700 1 -- 172.21.15.29:0/267047651 shutdown_connections 2024-01-24T14:10:28.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.270+0000 7f9401ee6700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f93e805f2c0 0x7f93e8061780 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:28.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.270+0000 7f9401ee6700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93fc155060 0x7f93fc0c4a10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:28.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.270+0000 7f9401ee6700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc069ee0 0x7f93fc1665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:28.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.270+0000 7f9401ee6700 1 --2- 172.21.15.29:0/267047651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc150970 0x7f93fc0c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:28.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.270+0000 7f9401ee6700 1 -- 172.21.15.29:0/267047651 >> 172.21.15.29:0/267047651 conn(0x7f93fc067b20 msgr2=0x7f93fc0ad530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:28.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.270+0000 7f9401ee6700 1 -- 172.21.15.29:0/267047651 shutdown_connections 2024-01-24T14:10:28.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:28.270+0000 7f9401ee6700 1 -- 172.21.15.29:0/267047651 wait complete. 2024-01-24T14:10:28.284 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2024-01-24T14:10:28.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:28.285 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2024-01-24T14:10:28.363 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:28.364 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:28.364 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:28.364 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:28.364 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:28.364 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:28.364 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:28.364 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:28.364 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:28.364 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:28.364 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:28.364 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:28.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:28.365 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_00_00_UTC) 2024-01-24T14:10:28.433 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:28.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:28.433 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_01_00_UTC) 2024-01-24T14:10:28.502 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:28.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:28.503 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_02_00_UTC) 2024-01-24T14:10:28.572 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:28.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:28.573 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_03_00_UTC) 2024-01-24T14:10:28.642 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:28.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:28.642 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_04_00_UTC) 2024-01-24T14:10:28.712 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:28.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:28.712 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_05_00_UTC) 2024-01-24T14:10:28.783 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:28.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:28.783 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_06_00_UTC) 2024-01-24T14:10:28.853 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:28.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:28.853 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_07_00_UTC) 2024-01-24T14:10:28.923 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:28.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:28.923 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_08_00_UTC) 2024-01-24T14:10:28.992 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:28.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:28.993 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_09_00_UTC) 2024-01-24T14:10:29.062 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:29.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:29.062 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-01-24-14_10_00_UTC) 2024-01-24T14:10:29.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:29.131 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2024-01-24T14:10:29.198 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir1 1M 2024-01-24T14:10:29.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.382+0000 7fb7d11cb700 1 Processor -- start 2024-01-24T14:10:29.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.382+0000 7fb7d11cb700 1 -- start start 2024-01-24T14:10:29.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.383+0000 7fb7d11cb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0c4b00 0x7fb7cc155180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:29.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.383+0000 7fb7d11cb700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7cc1557e0 con 0x7fb7cc14b0a0 2024-01-24T14:10:29.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.383+0000 7fb7d11cb700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7cc0c28a0 con 0x7fb7cc0c3660 2024-01-24T14:10:29.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.383+0000 7fb7d11cb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0c2b80 con 0x7fb7cc0c4b00 2024-01-24T14:10:29.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.383+0000 7fb7cad9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb7cc14b0a0 0x7fb7cc0c43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44236/0 (socket says 172.21.15.29:44236) 2024-01-24T14:10:29.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.383+0000 7fb7cad9d700 1 -- 172.21.15.29:0/2974015675 learned_addr learned my addr 172.21.15.29:0/2974015675 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:29.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7cb59e700 1 --2- 172.21.15.29:0/2974015675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0c4b00 0x7fb7cc155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:29.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167613805 0 0) 0x7fb7cc1557e0 con 0x7fb7cc14b0a0 2024-01-24T14:10:29.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7b80033f0 con 0x7fb7cc14b0a0 2024-01-24T14:10:29.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319207831 0 0) 0x7fb7cc0c28a0 con 0x7fb7cc0c3660 2024-01-24T14:10:29.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7cc1557e0 con 0x7fb7cc0c3660 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3570363760 0 0) 0x7fb7b80033f0 con 0x7fb7cc14b0a0 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0c4b00 msgr2=0x7fb7cc155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 --2- 172.21.15.29:0/2974015675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0c4b00 0x7fb7cc155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 >> v1:172.21.15.163:6789/0 conn(0x7fb7cc0c3660 legacy=0x7fb7cc154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0c2e60 con 0x7fb7cc14b0a0 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fb7bc001470 con 0x7fb7cc14b0a0 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 >> v1:172.21.15.29:6789/0 conn(0x7fb7cc14b0a0 legacy=0x7fb7cc0c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 --2- 172.21.15.29:0/2974015675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b8003aa0 0x7fb7b8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 --2- 172.21.15.29:0/2974015675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b8004690 0x7fb7b8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 --2- 172.21.15.29:0/2974015675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0c4b00 0x7fb7b8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0c2b80 con 0x7fb7b8004690 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b800db20 con 0x7fb7b8003aa0 2024-01-24T14:10:29.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b800dce0 con 0x7fb7cc0c4b00 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7ca59c700 1 --2- 172.21.15.29:0/2974015675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b8004690 0x7fb7b8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7cad9d700 1 --2- 172.21.15.29:0/2974015675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b8003aa0 0x7fb7b8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.384+0000 7fb7cb59e700 1 --2- 172.21.15.29:0/2974015675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0c4b00 0x7fb7b8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.385+0000 7fb7ca59c700 1 -- 172.21.15.29:0/2974015675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0c4b00 msgr2=0x7fb7b8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.385+0000 7fb7ca59c700 1 --2- 172.21.15.29:0/2974015675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0c4b00 0x7fb7b8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.385+0000 7fb7ca59c700 1 -- 172.21.15.29:0/2974015675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b8003aa0 msgr2=0x7fb7b8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.385+0000 7fb7ca59c700 1 --2- 172.21.15.29:0/2974015675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b8003aa0 0x7fb7b8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.385+0000 7fb7ca59c700 1 -- 172.21.15.29:0/2974015675 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b80033f0 con 0x7fb7b8004690 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.385+0000 7fb7ca59c700 1 --2- 172.21.15.29:0/2974015675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b8004690 0x7fb7b8004b10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fb7b4001890 tx=0x7fb7b4007bb0).ready entity=mon.0 client_cookie=79cef84f9ee67c51 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.385+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4007020 con 0x7fb7b8004690 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.385+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b4007180 con 0x7fb7b8004690 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.385+0000 7fb7c9d9b700 1 -- 172.21.15.29:0/2974015675 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4007500 con 0x7fb7b8004690 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.385+0000 7fb7d11cb700 1 -- 172.21.15.29:0/2974015675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b8004690 msgr2=0x7fb7b8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.385+0000 7fb7d11cb700 1 --2- 172.21.15.29:0/2974015675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b8004690 0x7fb7b8004b10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fb7b4001890 tx=0x7fb7b4007bb0).stop 2024-01-24T14:10:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.385+0000 7fb7cb59e700 1 -- 172.21.15.29:0/2974015675 reap_dead start 2024-01-24T14:10:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.386+0000 7fb7d11cb700 1 -- 172.21.15.29:0/2974015675 shutdown_connections 2024-01-24T14:10:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.386+0000 7fb7d11cb700 1 -- 172.21.15.29:0/2974015675 >> 172.21.15.29:0/2974015675 conn(0x7fb7cc067b20 msgr2=0x7fb7cc0638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.386+0000 7fb7d11cb700 1 -- 172.21.15.29:0/2974015675 shutdown_connections 2024-01-24T14:10:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.386+0000 7fb7d11cb700 1 -- 172.21.15.29:0/2974015675 wait complete. 2024-01-24T14:10:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.387+0000 7fb7d11cb700 1 Processor -- start 2024-01-24T14:10:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.387+0000 7fb7d11cb700 1 -- start start 2024-01-24T14:10:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7d11cb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b8004690 0x7fb7cc0b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7d11cb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0ab0c0 0x7fb7cc0befe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7d11cb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7cc0bf4d0 0x7fb7cc0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7d11cb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc1604f0 con 0x7fb7b8004690 2024-01-24T14:10:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7d11cb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0adb90 con 0x7fb7cc0bf4d0 2024-01-24T14:10:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7d11cb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0ad5d0 con 0x7fb7cc0ab0c0 2024-01-24T14:10:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7cad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b8004690 0x7fb7cc0b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7cad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b8004690 0x7fb7cc0b9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54278/0 (socket says 172.21.15.29:54278) 2024-01-24T14:10:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7cad9d700 1 -- 172.21.15.29:0/1711610716 learned_addr learned my addr 172.21.15.29:0/1711610716 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:29.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7ca59c700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0ab0c0 0x7fb7cc0befe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7cb59e700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7cc0bf4d0 0x7fb7cc0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7cad9d700 1 -- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0ab0c0 msgr2=0x7fb7cc0befe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7cad9d700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0ab0c0 0x7fb7cc0befe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7cad9d700 1 -- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7cc0bf4d0 msgr2=0x7fb7cc0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7cad9d700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7cc0bf4d0 0x7fb7cc0627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.388+0000 7fb7cad9d700 1 -- 172.21.15.29:0/1711610716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0ad8b0 con 0x7fb7b8004690 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.389+0000 7fb7cad9d700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b8004690 0x7fb7cc0b9f00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fb7bc004760 tx=0x7fb7bc001df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.389+0000 7fb7c2ffd700 1 -- 172.21.15.29:0/1711610716 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc011590 con 0x7fb7b8004690 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.389+0000 7fb7c2ffd700 1 -- 172.21.15.29:0/1711610716 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc0116f0 con 0x7fb7b8004690 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.389+0000 7fb7d11cb700 1 -- 172.21.15.29:0/1711610716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7cc0bfaa0 con 0x7fb7b8004690 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.389+0000 7fb7c2ffd700 1 -- 172.21.15.29:0/1711610716 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc011a70 con 0x7fb7b8004690 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.389+0000 7fb7d11cb700 1 -- 172.21.15.29:0/1711610716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc0ba3f0 con 0x7fb7b8004690 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.390+0000 7fb7d11cb700 1 -- 172.21.15.29:0/1711610716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7cc0bfce0 con 0x7fb7b8004690 2024-01-24T14:10:29.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.390+0000 7fb7c2ffd700 1 -- 172.21.15.29:0/1711610716 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fb7bc01d3b0 con 0x7fb7b8004690 2024-01-24T14:10:29.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.391+0000 7fb7c2ffd700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb7ac05f2c0 0x7fb7ac061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:29.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.391+0000 7fb7c2ffd700 1 -- 172.21.15.29:0/1711610716 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6733+0+0 (secure 0 0 0) 0x7fb7bc07a010 con 0x7fb7b8004690 2024-01-24T14:10:29.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.391+0000 7fb7ca59c700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb7ac05f2c0 0x7fb7ac061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:29.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.391+0000 7fb7ca59c700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb7ac05f2c0 0x7fb7ac061780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb7cc0baee0 tx=0x7fb7b4016000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:29.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.395+0000 7fb7c2ffd700 1 -- 172.21.15.29:0/1711610716 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb7cc0bfce0 con 0x7fb7b8004690 2024-01-24T14:10:29.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.603+0000 7fb7d11cb700 1 -- 172.21.15.29:0/1711610716 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7cc0ad740 con 0x7fb7ac05f2c0 2024-01-24T14:10:29.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.607+0000 7fb7c2ffd700 1 -- 172.21.15.29:0/1711610716 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fb7cc0ad740 con 0x7fb7ac05f2c0 2024-01-24T14:10:29.608 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /snap_test_dir1/dir1 2024-01-24T14:10:29.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.609+0000 7fb7d11cb700 1 -- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb7ac05f2c0 msgr2=0x7fb7ac061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:29.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.609+0000 7fb7d11cb700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb7ac05f2c0 0x7fb7ac061780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb7cc0baee0 tx=0x7fb7b4016000).stop 2024-01-24T14:10:29.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.609+0000 7fb7d11cb700 1 -- 172.21.15.29:0/1711610716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b8004690 msgr2=0x7fb7cc0b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:29.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.609+0000 7fb7d11cb700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b8004690 0x7fb7cc0b9f00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fb7bc004760 tx=0x7fb7bc001df0).stop 2024-01-24T14:10:29.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.609+0000 7fb7d11cb700 1 -- 172.21.15.29:0/1711610716 shutdown_connections 2024-01-24T14:10:29.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.609+0000 7fb7d11cb700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7cc0bf4d0 0x7fb7cc0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:29.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.609+0000 7fb7d11cb700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb7ac05f2c0 0x7fb7ac061780 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:29.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.609+0000 7fb7d11cb700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7b8004690 0x7fb7cc0b9f00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:29.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.609+0000 7fb7d11cb700 1 --2- 172.21.15.29:0/1711610716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7cc0ab0c0 0x7fb7cc0befe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:29.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.609+0000 7fb7d11cb700 1 -- 172.21.15.29:0/1711610716 >> 172.21.15.29:0/1711610716 conn(0x7fb7cc067b20 msgr2=0x7fb7cc063c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:29.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.609+0000 7fb7d11cb700 1 -- 172.21.15.29:0/1711610716 shutdown_connections 2024-01-24T14:10:29.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:29.609+0000 7fb7d11cb700 1 -- 172.21.15.29:0/1711610716 wait complete. 2024-01-24T14:10:29.623 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2024-01-24T14:10:29.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:29.623 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2024-01-24T14:10:29.643 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:29.643 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:29.643 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:29.644 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:29.644 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:29.644 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:29.644 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:29.644 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:29.644 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:29.644 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:29.644 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:29.644 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:29.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:29.644 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_00_00_UTC) 2024-01-24T14:10:29.713 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:29.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:29.714 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_01_00_UTC) 2024-01-24T14:10:29.783 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:29.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:29.783 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_02_00_UTC) 2024-01-24T14:10:29.877 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:29.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:29.877 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_03_00_UTC) 2024-01-24T14:10:29.918 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:29.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:29.918 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_04_00_UTC) 2024-01-24T14:10:29.985 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:29.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:29.986 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_05_00_UTC) 2024-01-24T14:10:30.054 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:30.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:30.055 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_06_00_UTC) 2024-01-24T14:10:30.124 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:30.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:30.124 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_07_00_UTC) 2024-01-24T14:10:30.193 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:30.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:30.193 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_08_00_UTC) 2024-01-24T14:10:30.269 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:30.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:30.269 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_09_00_UTC) 2024-01-24T14:10:30.338 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:30.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:30.338 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-01-24-14_10_00_UTC) 2024-01-24T14:10:30.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:30.407 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2024-01-24T14:10:30.473 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir2 1M 2024-01-24T14:10:30.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.655+0000 7f72b3f45700 1 Processor -- start 2024-01-24T14:10:30.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.655+0000 7f72b3f45700 1 -- start start 2024-01-24T14:10:30.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.656+0000 7f72b3f45700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72ac154ff0 0x7f72ac064ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:30.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.656+0000 7f72b3f45700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f72ac1553d0 0x7f72ac0c3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:30.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.656+0000 7f72b3f45700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ac0c3d10 con 0x7f72ac0c3860 2024-01-24T14:10:30.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.656+0000 7f72b3f45700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ac158c30 con 0x7f72ac1553d0 2024-01-24T14:10:30.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.656+0000 7f72b3f45700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac064a50 con 0x7f72ac154ff0 2024-01-24T14:10:30.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.656+0000 7f72b14e0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f72ac1553d0 0x7f72ac0c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:30.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.656+0000 7f72b14e0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f72ac1553d0 0x7f72ac0c3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54290/0 (socket says 172.21.15.29:54290) 2024-01-24T14:10:30.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.656+0000 7f72b14e0700 1 -- 172.21.15.29:0/1623503947 learned_addr learned my addr 172.21.15.29:0/1623503947 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:30.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.656+0000 7f72b1ce1700 1 --2- 172.21.15.29:0/1623503947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72ac154ff0 0x7f72ac064ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:30.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.657+0000 7f72b14e0700 1 -- 172.21.15.29:0/1623503947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72ac154ff0 msgr2=0x7f72ac064ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:30.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.657+0000 7f72b14e0700 1 --2- 172.21.15.29:0/1623503947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72ac154ff0 0x7f72ac064ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:30.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.657+0000 7f72b14e0700 1 -- 172.21.15.29:0/1623503947 >> v1:172.21.15.29:6789/0 conn(0x7f72ac0c3860 legacy=0x7f72ac157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:30.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.657+0000 7f72b14e0700 1 -- 172.21.15.29:0/1623503947 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac158950 con 0x7f72ac1553d0 2024-01-24T14:10:30.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.657+0000 7f72b0cdf700 1 -- 172.21.15.29:0/1623503947 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128962793 0 0) 0x7f72a4001490 con 0x7f72ac0c3860 2024-01-24T14:10:30.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.657+0000 7f72b14e0700 1 --2- 172.21.15.29:0/1623503947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72ac1553d0 0x7f72ac0c3370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f729c006fd0 tx=0x7f729c0090f0).ready entity=mon.0 client_cookie=340c9f0f810ba1ab server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:30.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.657+0000 7f72b0cdf700 1 -- 172.21.15.29:0/1623503947 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c00b580 con 0x7f72ac1553d0 2024-01-24T14:10:30.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.657+0000 7f72b0cdf700 1 -- 172.21.15.29:0/1623503947 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c00b6e0 con 0x7f72ac1553d0 2024-01-24T14:10:30.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.657+0000 7f72b0cdf700 1 -- 172.21.15.29:0/1623503947 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c00ba80 con 0x7f72ac1553d0 2024-01-24T14:10:30.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.658+0000 7f72b3f45700 1 -- 172.21.15.29:0/1623503947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72ac1553d0 msgr2=0x7f72ac0c3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:30.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.658+0000 7f72b3f45700 1 --2- 172.21.15.29:0/1623503947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72ac1553d0 0x7f72ac0c3370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f729c006fd0 tx=0x7f729c0090f0).stop 2024-01-24T14:10:30.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.658+0000 7f72b3f45700 1 -- 172.21.15.29:0/1623503947 shutdown_connections 2024-01-24T14:10:30.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.658+0000 7f72b3f45700 1 --2- 172.21.15.29:0/1623503947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72ac154ff0 0x7f72ac064ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:30.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.658+0000 7f72b3f45700 1 --2- 172.21.15.29:0/1623503947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72ac1553d0 0x7f72ac0c3370 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:30.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.658+0000 7f72b3f45700 1 -- 172.21.15.29:0/1623503947 >> 172.21.15.29:0/1623503947 conn(0x7f72ac0627b0 msgr2=0x7f72ac062bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:30.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.658+0000 7f72b3f45700 1 -- 172.21.15.29:0/1623503947 shutdown_connections 2024-01-24T14:10:30.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.658+0000 7f72b3f45700 1 -- 172.21.15.29:0/1623503947 wait complete. 2024-01-24T14:10:30.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.659+0000 7f72b3f45700 1 Processor -- start 2024-01-24T14:10:30.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.659+0000 7f72b3f45700 1 -- start start 2024-01-24T14:10:30.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b3f45700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72ac0c3860 0x7f72ac0ac500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:30.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b3f45700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72ac154ff0 0x7f72ac0a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:30.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b3f45700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72ac1553d0 0x7f72ac164750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:30.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b3f45700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac157690 con 0x7f72ac0c3860 2024-01-24T14:10:30.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b3f45700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0a64e0 con 0x7f72ac1553d0 2024-01-24T14:10:30.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b3f45700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0a6780 con 0x7f72ac154ff0 2024-01-24T14:10:30.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b1ce1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72ac0c3860 0x7f72ac0ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:30.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b1ce1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72ac0c3860 0x7f72ac0ac500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54306/0 (socket says 172.21.15.29:54306) 2024-01-24T14:10:30.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b1ce1700 1 -- 172.21.15.29:0/1339521534 learned_addr learned my addr 172.21.15.29:0/1339521534 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:30.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b24e2700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72ac1553d0 0x7f72ac164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:30.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b14e0700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72ac154ff0 0x7f72ac0a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b1ce1700 1 -- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72ac154ff0 msgr2=0x7f72ac0a8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b1ce1700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72ac154ff0 0x7f72ac0a8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b1ce1700 1 -- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72ac1553d0 msgr2=0x7f72ac164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b1ce1700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72ac1553d0 0x7f72ac164750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b1ce1700 1 -- 172.21.15.29:0/1339521534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c009370 con 0x7f72ac0c3860 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.660+0000 7f72b1ce1700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72ac0c3860 0x7f72ac0ac500 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f72a800e950 tx=0x7f72a8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.661+0000 7f72a1ffb700 1 -- 172.21.15.29:0/1339521534 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8012600 con 0x7f72ac0c3860 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.661+0000 7f72a1ffb700 1 -- 172.21.15.29:0/1339521534 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a8012760 con 0x7f72ac0c3860 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.661+0000 7f72b3f45700 1 -- 172.21.15.29:0/1339521534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ac0a6a60 con 0x7f72ac0c3860 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.661+0000 7f72a1ffb700 1 -- 172.21.15.29:0/1339521534 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8012ae0 con 0x7f72ac0c3860 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.661+0000 7f72b3f45700 1 -- 172.21.15.29:0/1339521534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ac0a7020 con 0x7f72ac0c3860 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.662+0000 7f72b3f45700 1 -- 172.21.15.29:0/1339521534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ac0475e0 con 0x7f72ac0c3860 2024-01-24T14:10:30.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.662+0000 7f72a1ffb700 1 -- 172.21.15.29:0/1339521534 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f72a801b400 con 0x7f72ac0c3860 2024-01-24T14:10:30.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.663+0000 7f72a1ffb700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f729005f2c0 0x7f7290061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:30.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.663+0000 7f72a1ffb700 1 -- 172.21.15.29:0/1339521534 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 7045+0+0 (secure 0 0 0) 0x7f72a807ad00 con 0x7f72ac0c3860 2024-01-24T14:10:30.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.663+0000 7f72b14e0700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f729005f2c0 0x7f7290061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:30.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.664+0000 7f72b14e0700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f729005f2c0 0x7f7290061780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f729c006fd0 tx=0x7f729c017000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:30.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.667+0000 7f72a1ffb700 1 -- 172.21.15.29:0/1339521534 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f72a804bab0 con 0x7f72ac0c3860 2024-01-24T14:10:30.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.884+0000 7f72b3f45700 1 -- 172.21.15.29:0/1339521534 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f72ac064400 con 0x7f729005f2c0 2024-01-24T14:10:30.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.888+0000 7f72a1ffb700 1 -- 172.21.15.29:0/1339521534 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f72ac064400 con 0x7f729005f2c0 2024-01-24T14:10:30.890 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /snap_test_dir1/dir2 2024-01-24T14:10:30.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.892+0000 7f72b3f45700 1 -- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f729005f2c0 msgr2=0x7f7290061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:30.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.892+0000 7f72b3f45700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f729005f2c0 0x7f7290061780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f729c006fd0 tx=0x7f729c017000).stop 2024-01-24T14:10:30.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.892+0000 7f72b3f45700 1 -- 172.21.15.29:0/1339521534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72ac0c3860 msgr2=0x7f72ac0ac500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:30.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.892+0000 7f72b3f45700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72ac0c3860 0x7f72ac0ac500 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f72a800e950 tx=0x7f72a8009060).stop 2024-01-24T14:10:30.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.892+0000 7f72b3f45700 1 -- 172.21.15.29:0/1339521534 shutdown_connections 2024-01-24T14:10:30.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.892+0000 7f72b3f45700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72ac1553d0 0x7f72ac164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:30.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.892+0000 7f72b3f45700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f729005f2c0 0x7f7290061780 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0x7f729c006fd0 tx=0x7f729c017000).stop 2024-01-24T14:10:30.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.892+0000 7f72b3f45700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72ac0c3860 0x7f72ac0ac500 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:30.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.892+0000 7f72b3f45700 1 --2- 172.21.15.29:0/1339521534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72ac154ff0 0x7f72ac0a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:30.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.892+0000 7f72b3f45700 1 -- 172.21.15.29:0/1339521534 >> 172.21.15.29:0/1339521534 conn(0x7f72ac0627b0 msgr2=0x7f72ac157ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:30.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.892+0000 7f72b3f45700 1 -- 172.21.15.29:0/1339521534 shutdown_connections 2024-01-24T14:10:30.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:30.893+0000 7f72b3f45700 1 -- 172.21.15.29:0/1339521534 wait complete. 2024-01-24T14:10:30.910 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2024-01-24T14:10:30.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:30.911 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2024-01-24T14:10:30.931 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:30.931 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:30.931 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:30.931 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:30.931 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:30.931 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:30.931 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:30.931 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:30.931 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:30.931 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:30.931 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:30.932 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:30.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:30.932 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_00_00_UTC) 2024-01-24T14:10:30.999 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:30.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:30.999 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_01_00_UTC) 2024-01-24T14:10:31.068 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:31.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:31.069 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_02_00_UTC) 2024-01-24T14:10:31.138 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:31.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:31.138 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_03_00_UTC) 2024-01-24T14:10:31.207 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:31.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:31.207 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_04_00_UTC) 2024-01-24T14:10:31.276 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:31.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:31.276 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_05_00_UTC) 2024-01-24T14:10:31.344 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:31.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:31.345 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_06_00_UTC) 2024-01-24T14:10:31.416 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:31.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:31.417 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_07_00_UTC) 2024-01-24T14:10:31.486 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:31.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:31.486 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_08_00_UTC) 2024-01-24T14:10:31.555 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:31.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:31.555 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_09_00_UTC) 2024-01-24T14:10:31.624 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:31.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:31.625 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-01-24-14_10_00_UTC) 2024-01-24T14:10:31.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:31.694 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2024-01-24T14:10:31.759 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir3 1M 2024-01-24T14:10:31.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.956+0000 7fcd771fd700 1 Processor -- start 2024-01-24T14:10:31.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.957+0000 7fcd771fd700 1 -- start start 2024-01-24T14:10:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.957+0000 7fcd771fd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7014e540 0x7fcd70155030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.957+0000 7fcd771fd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcd70062ec0 0x7fcd70069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.957+0000 7fcd771fd700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd700c81e0 con 0x7fcd7006a620 2024-01-24T14:10:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.957+0000 7fcd771fd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd700c89f0 con 0x7fcd70062ec0 2024-01-24T14:10:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.957+0000 7fcd771fd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70154620 con 0x7fcd7014e540 2024-01-24T14:10:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd6ffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcd7006a620 0x7fcd700627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44242/0 (socket says 172.21.15.29:44242) 2024-01-24T14:10:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd6ffff700 1 -- 172.21.15.29:0/889870928 learned_addr learned my addr 172.21.15.29:0/889870928 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd74f99700 1 --2- 172.21.15.29:0/889870928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7014e540 0x7fcd70155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd7579a700 1 --2- 172.21.15.29:0/889870928 >> v2:172.21.15.163:3300/0 conn(0x7fcd70062ec0 0x7fcd70069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd6f7fe700 1 -- 172.21.15.29:0/889870928 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120003599 0 0) 0x7fcd700c81e0 con 0x7fcd7006a620 2024-01-24T14:10:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd6f7fe700 1 -- 172.21.15.29:0/889870928 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd600033f0 con 0x7fcd7006a620 2024-01-24T14:10:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd74f99700 1 -- 172.21.15.29:0/889870928 >> v2:172.21.15.163:3300/0 conn(0x7fcd70062ec0 msgr2=0x7fcd70069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd74f99700 1 --2- 172.21.15.29:0/889870928 >> v2:172.21.15.163:3300/0 conn(0x7fcd70062ec0 0x7fcd70069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd74f99700 1 -- 172.21.15.29:0/889870928 >> v1:172.21.15.29:6789/0 conn(0x7fcd7006a620 legacy=0x7fcd700627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd74f99700 1 -- 172.21.15.29:0/889870928 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd70154ba0 con 0x7fcd7014e540 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd7579a700 1 --2- 172.21.15.29:0/889870928 >> v2:172.21.15.163:3300/0 conn(0x7fcd70062ec0 0x7fcd70069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd74f99700 1 --2- 172.21.15.29:0/889870928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7014e540 0x7fcd70155030 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fcd64009760 tx=0x7fcd64009060).ready entity=mon.2 client_cookie=2135e702aecbb396 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.958+0000 7fcd6f7fe700 1 -- 172.21.15.29:0/889870928 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400d4a0 con 0x7fcd7014e540 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.959+0000 7fcd6f7fe700 1 -- 172.21.15.29:0/889870928 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6400d600 con 0x7fcd7014e540 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.959+0000 7fcd6f7fe700 1 -- 172.21.15.29:0/889870928 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400d980 con 0x7fcd7014e540 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.959+0000 7fcd771fd700 1 -- 172.21.15.29:0/889870928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7014e540 msgr2=0x7fcd70155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.959+0000 7fcd771fd700 1 --2- 172.21.15.29:0/889870928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7014e540 0x7fcd70155030 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fcd64009760 tx=0x7fcd64009060).stop 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.959+0000 7fcd771fd700 1 -- 172.21.15.29:0/889870928 shutdown_connections 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.959+0000 7fcd771fd700 1 --2- 172.21.15.29:0/889870928 >> v2:172.21.15.163:3300/0 conn(0x7fcd70062ec0 0x7fcd70069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.959+0000 7fcd771fd700 1 --2- 172.21.15.29:0/889870928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7014e540 0x7fcd70155030 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.959+0000 7fcd771fd700 1 -- 172.21.15.29:0/889870928 >> 172.21.15.29:0/889870928 conn(0x7fcd70064390 msgr2=0x7fcd700c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.959+0000 7fcd771fd700 1 -- 172.21.15.29:0/889870928 shutdown_connections 2024-01-24T14:10:31.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.959+0000 7fcd771fd700 1 -- 172.21.15.29:0/889870928 wait complete. 2024-01-24T14:10:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.960+0000 7fcd771fd700 1 Processor -- start 2024-01-24T14:10:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.961+0000 7fcd771fd700 1 -- start start 2024-01-24T14:10:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.961+0000 7fcd771fd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd70062ec0 0x7fcd70164e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.961+0000 7fcd771fd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7006a620 0x7fcd700ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.961+0000 7fcd771fd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7014e540 0x7fcd7014f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.961+0000 7fcd771fd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd701652f0 con 0x7fcd70062ec0 2024-01-24T14:10:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.961+0000 7fcd771fd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700adae0 con 0x7fcd7014e540 2024-01-24T14:10:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.961+0000 7fcd771fd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7014fb40 con 0x7fcd7006a620 2024-01-24T14:10:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.961+0000 7fcd74f99700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd70062ec0 0x7fcd70164e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd74f99700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd70062ec0 0x7fcd70164e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54320/0 (socket says 172.21.15.29:54320) 2024-01-24T14:10:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd74f99700 1 -- 172.21.15.29:0/1470937629 learned_addr learned my addr 172.21.15.29:0/1470937629 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd6ffff700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7006a620 0x7fcd700ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd7579a700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7014e540 0x7fcd7014f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd74f99700 1 -- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7006a620 msgr2=0x7fcd700ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd74f99700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7006a620 0x7fcd700ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd74f99700 1 -- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7014e540 msgr2=0x7fcd7014f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd74f99700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7014e540 0x7fcd7014f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd74f99700 1 -- 172.21.15.29:0/1470937629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68009a20 con 0x7fcd70062ec0 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd6ffff700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7006a620 0x7fcd700ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd7579a700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7014e540 0x7fcd7014f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd74f99700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd70062ec0 0x7fcd70164e00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fcd64009d50 tx=0x7fcd64016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd6cff9700 1 -- 172.21.15.29:0/1470937629 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64000c90 con 0x7fcd70062ec0 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd6cff9700 1 -- 172.21.15.29:0/1470937629 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6400d770 con 0x7fcd70062ec0 2024-01-24T14:10:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd771fd700 1 -- 172.21.15.29:0/1470937629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd640092a0 con 0x7fcd70062ec0 2024-01-24T14:10:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd6cff9700 1 -- 172.21.15.29:0/1470937629 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6401d640 con 0x7fcd70062ec0 2024-01-24T14:10:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.962+0000 7fcd771fd700 1 -- 172.21.15.29:0/1470937629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd70164600 con 0x7fcd70062ec0 2024-01-24T14:10:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.963+0000 7fcd771fd700 1 -- 172.21.15.29:0/1470937629 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd700475e0 con 0x7fcd70062ec0 2024-01-24T14:10:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.964+0000 7fcd6cff9700 1 -- 172.21.15.29:0/1470937629 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fcd6401da70 con 0x7fcd70062ec0 2024-01-24T14:10:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.964+0000 7fcd6cff9700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd5405f2c0 0x7fcd54061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.964+0000 7fcd6cff9700 1 -- 172.21.15.29:0/1470937629 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7269+0+0 (secure 0 0 0) 0x7fcd6407c0f0 con 0x7fcd70062ec0 2024-01-24T14:10:31.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.964+0000 7fcd6ffff700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd5405f2c0 0x7fcd54061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.965+0000 7fcd6ffff700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd5405f2c0 0x7fcd54061780 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fcd5c007fd0 tx=0x7fcd5c009040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:31.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:31.969+0000 7fcd6cff9700 1 -- 172.21.15.29:0/1470937629 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcd6404cdc0 con 0x7fcd70062ec0 2024-01-24T14:10:32.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.216+0000 7fcd771fd700 1 -- 172.21.15.29:0/1470937629 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd70066520 con 0x7fcd5405f2c0 2024-01-24T14:10:32.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.227+0000 7fcd6cff9700 1 -- 172.21.15.29:0/1470937629 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fcd70066520 con 0x7fcd5405f2c0 2024-01-24T14:10:32.227 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /snap_test_dir1/dir3 2024-01-24T14:10:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.229+0000 7fcd771fd700 1 -- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd5405f2c0 msgr2=0x7fcd54061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.229+0000 7fcd771fd700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd5405f2c0 0x7fcd54061780 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fcd5c007fd0 tx=0x7fcd5c009040).stop 2024-01-24T14:10:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.229+0000 7fcd771fd700 1 -- 172.21.15.29:0/1470937629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd70062ec0 msgr2=0x7fcd70164e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.229+0000 7fcd771fd700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd70062ec0 0x7fcd70164e00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fcd64009d50 tx=0x7fcd64016000).stop 2024-01-24T14:10:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.229+0000 7fcd771fd700 1 -- 172.21.15.29:0/1470937629 shutdown_connections 2024-01-24T14:10:32.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.229+0000 7fcd771fd700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7014e540 0x7fcd7014f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.229+0000 7fcd771fd700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcd5405f2c0 0x7fcd54061780 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.229+0000 7fcd771fd700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd70062ec0 0x7fcd70164e00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.229+0000 7fcd771fd700 1 --2- 172.21.15.29:0/1470937629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7006a620 0x7fcd700ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.229+0000 7fcd771fd700 1 -- 172.21.15.29:0/1470937629 >> 172.21.15.29:0/1470937629 conn(0x7fcd70064390 msgr2=0x7fcd700c7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.229+0000 7fcd771fd700 1 -- 172.21.15.29:0/1470937629 shutdown_connections 2024-01-24T14:10:32.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:32.229+0000 7fcd771fd700 1 -- 172.21.15.29:0/1470937629 wait complete. 2024-01-24T14:10:32.244 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2024-01-24T14:10:32.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:32.244 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2024-01-24T14:10:32.266 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:32.266 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:32.266 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:32.266 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:32.267 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:32.267 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:32.267 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:32.267 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:32.267 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:32.267 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:32.267 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:32.267 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:32.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:32.268 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_00_00_UTC) 2024-01-24T14:10:32.334 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:32.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:32.335 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_01_00_UTC) 2024-01-24T14:10:32.413 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:32.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:32.413 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_02_00_UTC) 2024-01-24T14:10:32.484 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:32.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:32.485 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_03_00_UTC) 2024-01-24T14:10:32.557 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:32.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:32.557 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_04_00_UTC) 2024-01-24T14:10:32.628 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:32.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:32.629 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_05_00_UTC) 2024-01-24T14:10:32.698 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:32.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:32.698 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_06_00_UTC) 2024-01-24T14:10:32.767 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:32.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:32.767 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_07_00_UTC) 2024-01-24T14:10:32.837 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:32.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:32.838 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_08_00_UTC) 2024-01-24T14:10:32.906 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:32.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:32.906 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_09_00_UTC) 2024-01-24T14:10:32.994 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:32.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:32.995 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-01-24-14_10_00_UTC) 2024-01-24T14:10:33.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:33.024 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2024-01-24T14:10:33.105 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir4 1M 2024-01-24T14:10:33.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.281+0000 7f06509e3700 1 Processor -- start 2024-01-24T14:10:33.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.281+0000 7f06509e3700 1 -- start start 2024-01-24T14:10:33.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.282+0000 7f06509e3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f064c154ec0 0x7f064c1552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.282+0000 7f06509e3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f064c0638c0 0x7f064c066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.282+0000 7f06509e3700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064c068540 con 0x7f064c063d40 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.282+0000 7f06509e3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064c0641c0 con 0x7f064c0638c0 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.282+0000 7f06509e3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0689d0 con 0x7f064c154ec0 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.282+0000 7f064ad9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f064c063d40 0x7f064c151350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44252/0 (socket says 172.21.15.29:44252) 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.282+0000 7f064ad9d700 1 -- 172.21.15.29:0/3757803676 learned_addr learned my addr 172.21.15.29:0/3757803676 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.282+0000 7f0641d9b700 1 --2- 172.21.15.29:0/3757803676 >> v2:172.21.15.163:3300/0 conn(0x7f064c0638c0 0x7f064c066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.282+0000 7f064a59c700 1 --2- 172.21.15.29:0/3757803676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f064c154ec0 0x7f064c1552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.282+0000 7f0649d9b700 1 -- 172.21.15.29:0/3757803676 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521545126 0 0) 0x7f064c068540 con 0x7f064c063d40 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.282+0000 7f0649d9b700 1 -- 172.21.15.29:0/3757803676 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06380033f0 con 0x7f064c063d40 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.283+0000 7f0641d9b700 1 -- 172.21.15.29:0/3757803676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f064c154ec0 msgr2=0x7f064c1552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.283+0000 7f0641d9b700 1 --2- 172.21.15.29:0/3757803676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f064c154ec0 0x7f064c1552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.283+0000 7f0641d9b700 1 -- 172.21.15.29:0/3757803676 >> v1:172.21.15.29:6789/0 conn(0x7f064c063d40 legacy=0x7f064c151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:33.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.283+0000 7f0641d9b700 1 -- 172.21.15.29:0/3757803676 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c067c10 con 0x7f064c0638c0 2024-01-24T14:10:33.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.283+0000 7f0641d9b700 1 --2- 172.21.15.29:0/3757803676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c0638c0 0x7f064c066450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f0634009760 tx=0x7f0634009060).ready entity=mon.1 client_cookie=3866fcde812bb0f8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:33.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.283+0000 7f0649d9b700 1 -- 172.21.15.29:0/3757803676 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063400d4a0 con 0x7f064c0638c0 2024-01-24T14:10:33.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.283+0000 7f0649d9b700 1 -- 172.21.15.29:0/3757803676 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063400d600 con 0x7f064c0638c0 2024-01-24T14:10:33.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.283+0000 7f0649d9b700 1 -- 172.21.15.29:0/3757803676 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063400d980 con 0x7f064c0638c0 2024-01-24T14:10:33.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.283+0000 7f06509e3700 1 -- 172.21.15.29:0/3757803676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c0638c0 msgr2=0x7f064c066450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:33.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.283+0000 7f06509e3700 1 --2- 172.21.15.29:0/3757803676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c0638c0 0x7f064c066450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f0634009760 tx=0x7f0634009060).stop 2024-01-24T14:10:33.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.284+0000 7f06509e3700 1 -- 172.21.15.29:0/3757803676 shutdown_connections 2024-01-24T14:10:33.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.284+0000 7f06509e3700 1 --2- 172.21.15.29:0/3757803676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f064c154ec0 0x7f064c1552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:33.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.284+0000 7f06509e3700 1 --2- 172.21.15.29:0/3757803676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c0638c0 0x7f064c066450 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:33.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.284+0000 7f06509e3700 1 -- 172.21.15.29:0/3757803676 >> 172.21.15.29:0/3757803676 conn(0x7f064c0c8bd0 msgr2=0x7f064c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:33.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.284+0000 7f06509e3700 1 -- 172.21.15.29:0/3757803676 shutdown_connections 2024-01-24T14:10:33.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.284+0000 7f06509e3700 1 -- 172.21.15.29:0/3757803676 wait complete. 2024-01-24T14:10:33.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.285+0000 7f06509e3700 1 Processor -- start 2024-01-24T14:10:33.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.285+0000 7f06509e3700 1 -- start start 2024-01-24T14:10:33.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f06509e3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c0638c0 0x7f064c152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:33.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f06509e3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f064c063d40 0x7f064c0aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:33.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f06509e3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c154ec0 0x7f064c0aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:33.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f06509e3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0aa4e0 con 0x7f064c0638c0 2024-01-24T14:10:33.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f06509e3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0a98d0 con 0x7f064c154ec0 2024-01-24T14:10:33.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f06509e3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0a9bb0 con 0x7f064c063d40 2024-01-24T14:10:33.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f064a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c0638c0 0x7f064c152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:33.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f064a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c0638c0 0x7f064c152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54334/0 (socket says 172.21.15.29:54334) 2024-01-24T14:10:33.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f064a59c700 1 -- 172.21.15.29:0/177906746 learned_addr learned my addr 172.21.15.29:0/177906746 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:33.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f0641d9b700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f064c063d40 0x7f064c0aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:33.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f064ad9d700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c154ec0 0x7f064c0aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:33.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f064a59c700 1 -- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f064c063d40 msgr2=0x7f064c0aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:33.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f064a59c700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f064c063d40 0x7f064c0aa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:33.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f064a59c700 1 -- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c154ec0 msgr2=0x7f064c0aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:33.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f064a59c700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c154ec0 0x7f064c0aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:33.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f064a59c700 1 -- 172.21.15.29:0/177906746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06340092a0 con 0x7f064c0638c0 2024-01-24T14:10:33.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.286+0000 7f064a59c700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c0638c0 0x7f064c152ad0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f063c00e900 tx=0x7f063c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:33.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.287+0000 7f0642ffd700 1 -- 172.21.15.29:0/177906746 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c012600 con 0x7f064c0638c0 2024-01-24T14:10:33.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.287+0000 7f0642ffd700 1 -- 172.21.15.29:0/177906746 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c012760 con 0x7f064c0638c0 2024-01-24T14:10:33.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.287+0000 7f06509e3700 1 -- 172.21.15.29:0/177906746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064c0a9e90 con 0x7f064c0638c0 2024-01-24T14:10:33.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.287+0000 7f0642ffd700 1 -- 172.21.15.29:0/177906746 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c012ae0 con 0x7f064c0638c0 2024-01-24T14:10:33.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.287+0000 7f06509e3700 1 -- 172.21.15.29:0/177906746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064c152fc0 con 0x7f064c0638c0 2024-01-24T14:10:33.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.288+0000 7f06509e3700 1 -- 172.21.15.29:0/177906746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064c0aa7c0 con 0x7f064c0638c0 2024-01-24T14:10:33.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.288+0000 7f0642ffd700 1 -- 172.21.15.29:0/177906746 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f063c01b400 con 0x7f064c0638c0 2024-01-24T14:10:33.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.289+0000 7f0642ffd700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f062c05f2c0 0x7f062c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:33.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.289+0000 7f0642ffd700 1 -- 172.21.15.29:0/177906746 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7477+0+0 (secure 0 0 0) 0x7f063c07ac60 con 0x7f064c0638c0 2024-01-24T14:10:33.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.289+0000 7f0641d9b700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f062c05f2c0 0x7f062c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:33.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.289+0000 7f0641d9b700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f062c05f2c0 0x7f062c061780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f0634009760 tx=0x7f0634016000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:33.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.293+0000 7f0642ffd700 1 -- 172.21.15.29:0/177906746 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f063c04b800 con 0x7f064c0638c0 2024-01-24T14:10:33.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.502+0000 7f06509e3700 1 -- 172.21.15.29:0/177906746 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f064c0aa620 con 0x7f062c05f2c0 2024-01-24T14:10:33.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.506+0000 7f0642ffd700 1 -- 172.21.15.29:0/177906746 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f063400dd30 con 0x7f062c05f2c0 2024-01-24T14:10:33.506 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /snap_test_dir1/dir4 2024-01-24T14:10:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.508+0000 7f06509e3700 1 -- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f062c05f2c0 msgr2=0x7f062c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.508+0000 7f06509e3700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f062c05f2c0 0x7f062c061780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f0634009760 tx=0x7f0634016000).stop 2024-01-24T14:10:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.508+0000 7f06509e3700 1 -- 172.21.15.29:0/177906746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c0638c0 msgr2=0x7f064c152ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.508+0000 7f06509e3700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c0638c0 0x7f064c152ad0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f063c00e900 tx=0x7f063c009060).stop 2024-01-24T14:10:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.508+0000 7f06509e3700 1 -- 172.21.15.29:0/177906746 shutdown_connections 2024-01-24T14:10:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.508+0000 7f06509e3700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c154ec0 0x7f064c0aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.508+0000 7f06509e3700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f062c05f2c0 0x7f062c061780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.508+0000 7f06509e3700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f064c0638c0 0x7f064c152ad0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.508+0000 7f06509e3700 1 --2- 172.21.15.29:0/177906746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f064c063d40 0x7f064c0aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:33.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.508+0000 7f06509e3700 1 -- 172.21.15.29:0/177906746 >> 172.21.15.29:0/177906746 conn(0x7f064c0c8bd0 msgr2=0x7f064c0c3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:33.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.508+0000 7f06509e3700 1 -- 172.21.15.29:0/177906746 shutdown_connections 2024-01-24T14:10:33.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:33.508+0000 7f06509e3700 1 -- 172.21.15.29:0/177906746 wait complete. 2024-01-24T14:10:33.522 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2024-01-24T14:10:33.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:33.522 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2024-01-24T14:10:33.542 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:33.542 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:33.542 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:33.542 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:33.542 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:33.542 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:33.542 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:33.542 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:33.542 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:33.542 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:33.542 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:33.542 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:33.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:33.543 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_00_00_UTC) 2024-01-24T14:10:33.610 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:33.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:33.610 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_01_00_UTC) 2024-01-24T14:10:33.681 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:33.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:33.681 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_02_00_UTC) 2024-01-24T14:10:33.750 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:33.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:33.751 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_03_00_UTC) 2024-01-24T14:10:33.821 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:33.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:33.821 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_04_00_UTC) 2024-01-24T14:10:33.946 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:33.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:33.946 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_05_00_UTC) 2024-01-24T14:10:33.972 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:33.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:33.972 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_06_00_UTC) 2024-01-24T14:10:34.039 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:34.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:34.039 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_07_00_UTC) 2024-01-24T14:10:34.108 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:34.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:34.109 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_08_00_UTC) 2024-01-24T14:10:34.178 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:34.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:34.178 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_09_00_UTC) 2024-01-24T14:10:34.248 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:34.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:34.249 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-01-24-14_10_00_UTC) 2024-01-24T14:10:34.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:34.319 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2024-01-24T14:10:34.385 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir5 1M 2024-01-24T14:10:34.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.568+0000 7fa958304700 1 Processor -- start 2024-01-24T14:10:34.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.568+0000 7fa958304700 1 -- start start 2024-01-24T14:10:34.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.569+0000 7fa958304700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa9500c31d0 0x7fa9500684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:34.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.569+0000 7fa958304700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa950067a70 con 0x7fa950066b60 2024-01-24T14:10:34.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.569+0000 7fa958304700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9500689b0 con 0x7fa95014fd10 2024-01-24T14:10:34.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.569+0000 7fa958304700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9500c3d50 con 0x7fa9500c31d0 2024-01-24T14:10:34.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.569+0000 7fa9568a1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa9500c31d0 0x7fa9500684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:34.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.569+0000 7fa9568a1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa9500c31d0 0x7fa9500684c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52672/0 (socket says 172.21.15.29:52672) 2024-01-24T14:10:34.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.569+0000 7fa9568a1700 1 -- 172.21.15.29:0/854456441 learned_addr learned my addr 172.21.15.29:0/854456441 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:34.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.569+0000 7fa9568a1700 1 -- 172.21.15.29:0/854456441 >> v1:172.21.15.163:6789/0 conn(0x7fa95014fd10 legacy=0x7fa950066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:34.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.569+0000 7fa9568a1700 1 -- 172.21.15.29:0/854456441 >> v1:172.21.15.29:6789/0 conn(0x7fa950066b60 legacy=0x7fa9500bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:34.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.569+0000 7fa9568a1700 1 -- 172.21.15.29:0/854456441 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9500c3790 con 0x7fa9500c31d0 2024-01-24T14:10:34.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.570+0000 7fa9568a1700 1 --2- 172.21.15.29:0/854456441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9500c31d0 0x7fa9500684c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fa948009f20 tx=0x7fa948009840).ready entity=mon.0 client_cookie=dc9b272629e57768 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.570+0000 7fa95509e700 1 -- 172.21.15.29:0/854456441 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94800e070 con 0x7fa9500c31d0 2024-01-24T14:10:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.570+0000 7fa95509e700 1 -- 172.21.15.29:0/854456441 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94800ba40 con 0x7fa9500c31d0 2024-01-24T14:10:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.570+0000 7fa95509e700 1 -- 172.21.15.29:0/854456441 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94800bdc0 con 0x7fa9500c31d0 2024-01-24T14:10:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.570+0000 7fa958304700 1 -- 172.21.15.29:0/854456441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9500c31d0 msgr2=0x7fa9500684c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.570+0000 7fa958304700 1 --2- 172.21.15.29:0/854456441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9500c31d0 0x7fa9500684c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fa948009f20 tx=0x7fa948009840).stop 2024-01-24T14:10:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.570+0000 7fa958304700 1 -- 172.21.15.29:0/854456441 shutdown_connections 2024-01-24T14:10:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.570+0000 7fa958304700 1 --2- 172.21.15.29:0/854456441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9500c31d0 0x7fa9500684c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.570+0000 7fa958304700 1 -- 172.21.15.29:0/854456441 >> 172.21.15.29:0/854456441 conn(0x7fa9500c8bd0 msgr2=0x7fa9500c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.571+0000 7fa958304700 1 -- 172.21.15.29:0/854456441 shutdown_connections 2024-01-24T14:10:34.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.571+0000 7fa958304700 1 -- 172.21.15.29:0/854456441 wait complete. 2024-01-24T14:10:34.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.572+0000 7fa958304700 1 Processor -- start 2024-01-24T14:10:34.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.572+0000 7fa958304700 1 -- start start 2024-01-24T14:10:34.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.572+0000 7fa958304700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa950066b60 0x7fa950064540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:34.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa958304700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9500c31d0 0x7fa950164d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa958304700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa95014fd10 0x7fa950064f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa958304700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9500640b0 con 0x7fa950066b60 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa958304700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9500656f0 con 0x7fa9500c31d0 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa958304700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa950063a60 con 0x7fa95014fd10 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9560a0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa950066b60 0x7fa950064540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9560a0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa950066b60 0x7fa950064540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52676/0 (socket says 172.21.15.29:52676) 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9560a0700 1 -- 172.21.15.29:0/4015353050 learned_addr learned my addr 172.21.15.29:0/4015353050 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa95589f700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9500c31d0 0x7fa950164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9568a1700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa95014fd10 0x7fa950064f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9560a0700 1 -- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa95014fd10 msgr2=0x7fa950064f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9560a0700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa95014fd10 0x7fa950064f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9560a0700 1 -- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9500c31d0 msgr2=0x7fa950164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9560a0700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9500c31d0 0x7fa950164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:34.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9560a0700 1 -- 172.21.15.29:0/4015353050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948009ae0 con 0x7fa950066b60 2024-01-24T14:10:34.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa95589f700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9500c31d0 0x7fa950164d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:10:34.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9560a0700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa950066b60 0x7fa950064540 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fa94400b460 tx=0x7fa944006f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:34.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9427fc700 1 -- 172.21.15.29:0/4015353050 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944012410 con 0x7fa950066b60 2024-01-24T14:10:34.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9427fc700 1 -- 172.21.15.29:0/4015353050 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa944012590 con 0x7fa950066b60 2024-01-24T14:10:34.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa958304700 1 -- 172.21.15.29:0/4015353050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa950063d40 con 0x7fa950066b60 2024-01-24T14:10:34.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa9427fc700 1 -- 172.21.15.29:0/4015353050 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944012950 con 0x7fa950066b60 2024-01-24T14:10:34.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.573+0000 7fa958304700 1 -- 172.21.15.29:0/4015353050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa950154c60 con 0x7fa950066b60 2024-01-24T14:10:34.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.575+0000 7fa958304700 1 -- 172.21.15.29:0/4015353050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9500638c0 con 0x7fa950066b60 2024-01-24T14:10:34.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.575+0000 7fa9427fc700 1 -- 172.21.15.29:0/4015353050 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fa944012d00 con 0x7fa950066b60 2024-01-24T14:10:34.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.575+0000 7fa9427fc700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa93405f2c0 0x7fa934061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:34.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.575+0000 7fa9427fc700 1 -- 172.21.15.29:0/4015353050 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7557+0+0 (secure 0 0 0) 0x7fa94407a7c0 con 0x7fa950066b60 2024-01-24T14:10:34.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.576+0000 7fa95589f700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa93405f2c0 0x7fa934061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:34.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.576+0000 7fa95589f700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa93405f2c0 0x7fa934061780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fa950133000 tx=0x7fa94c009040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:34.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.580+0000 7fa9427fc700 1 -- 172.21.15.29:0/4015353050 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa9500638c0 con 0x7fa950066b60 2024-01-24T14:10:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.787+0000 7fa958304700 1 -- 172.21.15.29:0/4015353050 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9500641f0 con 0x7fa93405f2c0 2024-01-24T14:10:34.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.791+0000 7fa9427fc700 1 -- 172.21.15.29:0/4015353050 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fa94c001360 con 0x7fa93405f2c0 2024-01-24T14:10:34.791 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /snap_test_dir1/dir5 2024-01-24T14:10:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.793+0000 7fa958304700 1 -- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa93405f2c0 msgr2=0x7fa934061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.793+0000 7fa958304700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa93405f2c0 0x7fa934061780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fa950133000 tx=0x7fa94c009040).stop 2024-01-24T14:10:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.793+0000 7fa958304700 1 -- 172.21.15.29:0/4015353050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa950066b60 msgr2=0x7fa950064540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.793+0000 7fa958304700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa950066b60 0x7fa950064540 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fa94400b460 tx=0x7fa944006f20).stop 2024-01-24T14:10:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.793+0000 7fa958304700 1 -- 172.21.15.29:0/4015353050 shutdown_connections 2024-01-24T14:10:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.793+0000 7fa958304700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa95014fd10 0x7fa950064f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.793+0000 7fa958304700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa93405f2c0 0x7fa934061780 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.793+0000 7fa958304700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa950066b60 0x7fa950064540 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.793+0000 7fa958304700 1 --2- 172.21.15.29:0/4015353050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9500c31d0 0x7fa950164d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.793+0000 7fa958304700 1 -- 172.21.15.29:0/4015353050 >> 172.21.15.29:0/4015353050 conn(0x7fa9500c8bd0 msgr2=0x7fa950066f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:34.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.793+0000 7fa958304700 1 -- 172.21.15.29:0/4015353050 shutdown_connections 2024-01-24T14:10:34.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:34.793+0000 7fa958304700 1 -- 172.21.15.29:0/4015353050 wait complete. 2024-01-24T14:10:34.807 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2024-01-24T14:10:34.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:34.807 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2024-01-24T14:10:34.827 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:34.828 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:34.828 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:34.828 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:34.828 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:34.828 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:34.828 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:34.828 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:34.828 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:34.828 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:34.828 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:34.828 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:34.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:34.829 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_00_00_UTC) 2024-01-24T14:10:34.912 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:34.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:34.913 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_01_00_UTC) 2024-01-24T14:10:34.980 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:34.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:34.980 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_02_00_UTC) 2024-01-24T14:10:35.050 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:35.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:35.050 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_03_00_UTC) 2024-01-24T14:10:35.143 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:35.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:35.143 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_04_00_UTC) 2024-01-24T14:10:35.169 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:35.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:35.170 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_05_00_UTC) 2024-01-24T14:10:35.237 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:35.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:35.237 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_06_00_UTC) 2024-01-24T14:10:35.306 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:35.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:35.306 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_07_00_UTC) 2024-01-24T14:10:35.376 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:35.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:35.376 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_08_00_UTC) 2024-01-24T14:10:35.450 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:35.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:35.450 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_09_00_UTC) 2024-01-24T14:10:35.519 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:35.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:35.520 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-01-24-14_10_00_UTC) 2024-01-24T14:10:35.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:35.590 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2024-01-24T14:10:35.657 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir6 1M 2024-01-24T14:10:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.840+0000 7f1ceea85700 1 Processor -- start 2024-01-24T14:10:35.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.841+0000 7f1ceea85700 1 -- start start 2024-01-24T14:10:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.841+0000 7f1ceea85700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ce8151e50 0x7f1ce80c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.841+0000 7f1ceea85700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1ce80c7f70 0x7f1ce8155420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.841+0000 7f1ceea85700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce8154f90 con 0x7f1ce8064780 2024-01-24T14:10:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.841+0000 7f1ceea85700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce8155910 con 0x7f1ce80c7f70 2024-01-24T14:10:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.841+0000 7f1ceea85700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80a9f10 con 0x7f1ce8151e50 2024-01-24T14:10:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.842+0000 7f1ced022700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1ce8064780 0x7f1ce80bfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54676/0 (socket says 172.21.15.29:54676) 2024-01-24T14:10:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.842+0000 7f1ced022700 1 -- 172.21.15.29:0/3535461673 learned_addr learned my addr 172.21.15.29:0/3535461673 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.842+0000 7f1ce7fff700 1 --2- 172.21.15.29:0/3535461673 >> v2:172.21.15.163:3300/0 conn(0x7f1ce80c7f70 0x7f1ce8155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.842+0000 7f1cec821700 1 --2- 172.21.15.29:0/3535461673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ce8151e50 0x7f1ce80c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:35.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.842+0000 7f1ce77fe700 1 -- 172.21.15.29:0/3535461673 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140354428 0 0) 0x7f1ce8154f90 con 0x7f1ce8064780 2024-01-24T14:10:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.842+0000 7f1ce77fe700 1 -- 172.21.15.29:0/3535461673 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cd80033f0 con 0x7f1ce8064780 2024-01-24T14:10:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.842+0000 7f1ce7fff700 1 -- 172.21.15.29:0/3535461673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ce8151e50 msgr2=0x7f1ce80c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.842+0000 7f1ce7fff700 1 --2- 172.21.15.29:0/3535461673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ce8151e50 0x7f1ce80c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.842+0000 7f1ce7fff700 1 -- 172.21.15.29:0/3535461673 >> v1:172.21.15.29:6789/0 conn(0x7f1ce8064780 legacy=0x7f1ce80bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.842+0000 7f1ce7fff700 1 -- 172.21.15.29:0/3535461673 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce80aa1f0 con 0x7f1ce80c7f70 2024-01-24T14:10:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.842+0000 7f1cec821700 1 --2- 172.21.15.29:0/3535461673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ce8151e50 0x7f1ce80c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.842+0000 7f1ce7fff700 1 --2- 172.21.15.29:0/3535461673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ce80c7f70 0x7f1ce8155420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f1cd4009760 tx=0x7f1cd4009060).ready entity=mon.1 client_cookie=6d2acff176ca6888 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.843+0000 7f1ce77fe700 1 -- 172.21.15.29:0/3535461673 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400d4a0 con 0x7f1ce80c7f70 2024-01-24T14:10:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.843+0000 7f1ce77fe700 1 -- 172.21.15.29:0/3535461673 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd400d600 con 0x7f1ce80c7f70 2024-01-24T14:10:35.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.843+0000 7f1ce77fe700 1 -- 172.21.15.29:0/3535461673 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400d980 con 0x7f1ce80c7f70 2024-01-24T14:10:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.843+0000 7f1ceea85700 1 -- 172.21.15.29:0/3535461673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ce80c7f70 msgr2=0x7f1ce8155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.843+0000 7f1ceea85700 1 --2- 172.21.15.29:0/3535461673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ce80c7f70 0x7f1ce8155420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f1cd4009760 tx=0x7f1cd4009060).stop 2024-01-24T14:10:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.843+0000 7f1ceea85700 1 -- 172.21.15.29:0/3535461673 shutdown_connections 2024-01-24T14:10:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.843+0000 7f1ceea85700 1 --2- 172.21.15.29:0/3535461673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ce8151e50 0x7f1ce80c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:35.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.843+0000 7f1ceea85700 1 --2- 172.21.15.29:0/3535461673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ce80c7f70 0x7f1ce8155420 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.843+0000 7f1ceea85700 1 -- 172.21.15.29:0/3535461673 >> 172.21.15.29:0/3535461673 conn(0x7f1ce80cb880 msgr2=0x7f1ce80c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.843+0000 7f1ceea85700 1 -- 172.21.15.29:0/3535461673 shutdown_connections 2024-01-24T14:10:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.843+0000 7f1ceea85700 1 -- 172.21.15.29:0/3535461673 wait complete. 2024-01-24T14:10:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.845+0000 7f1ceea85700 1 Processor -- start 2024-01-24T14:10:35.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.845+0000 7f1ceea85700 1 -- start start 2024-01-24T14:10:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.846+0000 7f1ceea85700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ce8064780 0x7f1ce81425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:35.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.846+0000 7f1ceea85700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ce80c7f70 0x7f1ce8167010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.846+0000 7f1ceea85700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ce8151e50 0x7f1ce81653b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.846+0000 7f1ceea85700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8164590 con 0x7f1ce8064780 2024-01-24T14:10:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.846+0000 7f1ceea85700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8165f70 con 0x7f1ce8151e50 2024-01-24T14:10:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.846+0000 7f1ceea85700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8166250 con 0x7f1ce80c7f70 2024-01-24T14:10:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.846+0000 7f1cec821700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ce8064780 0x7f1ce81425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.846+0000 7f1cec821700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ce8064780 0x7f1ce81425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52684/0 (socket says 172.21.15.29:52684) 2024-01-24T14:10:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.846+0000 7f1cec821700 1 -- 172.21.15.29:0/432237724 learned_addr learned my addr 172.21.15.29:0/432237724 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.846+0000 7f1ce7fff700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ce80c7f70 0x7f1ce8167010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.846+0000 7f1ced022700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ce8151e50 0x7f1ce81653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.847+0000 7f1cec821700 1 -- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ce80c7f70 msgr2=0x7f1ce8167010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.847+0000 7f1cec821700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ce80c7f70 0x7f1ce8167010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:35.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.847+0000 7f1cec821700 1 -- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ce8151e50 msgr2=0x7f1ce81653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.847+0000 7f1cec821700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ce8151e50 0x7f1ce81653b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.847+0000 7f1cec821700 1 -- 172.21.15.29:0/432237724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc0092a0 con 0x7f1ce8064780 2024-01-24T14:10:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.847+0000 7f1cec821700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ce8064780 0x7f1ce81425a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f1cdc00ea20 tx=0x7f1cdc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.847+0000 7f1ce4ff9700 1 -- 172.21.15.29:0/432237724 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc00ee60 con 0x7f1ce8064780 2024-01-24T14:10:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.847+0000 7f1ce4ff9700 1 -- 172.21.15.29:0/432237724 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cdc00c850 con 0x7f1ce8064780 2024-01-24T14:10:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.847+0000 7f1ce4ff9700 1 -- 172.21.15.29:0/432237724 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc00cbd0 con 0x7f1ce8064780 2024-01-24T14:10:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.847+0000 7f1ceea85700 1 -- 172.21.15.29:0/432237724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd40092a0 con 0x7f1ce8064780 2024-01-24T14:10:35.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.847+0000 7f1ceea85700 1 -- 172.21.15.29:0/432237724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce8166810 con 0x7f1ce8064780 2024-01-24T14:10:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.848+0000 7f1ceea85700 1 -- 172.21.15.29:0/432237724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce80aa710 con 0x7f1ce8064780 2024-01-24T14:10:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.849+0000 7f1ce4ff9700 1 -- 172.21.15.29:0/432237724 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f1cdc01b400 con 0x7f1ce8064780 2024-01-24T14:10:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.849+0000 7f1ce4ff9700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ccc05f2c0 0x7f1ccc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.849+0000 7f1ce4ff9700 1 -- 172.21.15.29:0/432237724 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f1cdc07b020 con 0x7f1ce8064780 2024-01-24T14:10:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.849+0000 7f1ce7fff700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ccc05f2c0 0x7f1ccc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:35.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.850+0000 7f1ce7fff700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ccc05f2c0 0x7f1ccc061780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f1cd4009760 tx=0x7f1cd4016000).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:35.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:35.860+0000 7f1ce4ff9700 1 -- 172.21.15.29:0/432237724 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1cdc04bb80 con 0x7f1ce8064780 2024-01-24T14:10:36.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.095+0000 7f1ceea85700 1 -- 172.21.15.29:0/432237724 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ce8083310 con 0x7f1ccc05f2c0 2024-01-24T14:10:36.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.100+0000 7f1ce4ff9700 1 -- 172.21.15.29:0/432237724 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f1ce8083310 con 0x7f1ccc05f2c0 2024-01-24T14:10:36.100 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /snap_test_dir1/dir6 2024-01-24T14:10:36.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.102+0000 7f1ceea85700 1 -- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ccc05f2c0 msgr2=0x7f1ccc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:36.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.102+0000 7f1ceea85700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ccc05f2c0 0x7f1ccc061780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f1cd4009760 tx=0x7f1cd4016000).stop 2024-01-24T14:10:36.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.102+0000 7f1ceea85700 1 -- 172.21.15.29:0/432237724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ce8064780 msgr2=0x7f1ce81425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:36.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.102+0000 7f1ceea85700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ce8064780 0x7f1ce81425a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f1cdc00ea20 tx=0x7f1cdc009060).stop 2024-01-24T14:10:36.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.102+0000 7f1ceea85700 1 -- 172.21.15.29:0/432237724 shutdown_connections 2024-01-24T14:10:36.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.102+0000 7f1ceea85700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ce8151e50 0x7f1ce81653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:36.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.102+0000 7f1ceea85700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ccc05f2c0 0x7f1ccc061780 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:36.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.102+0000 7f1ceea85700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ce8064780 0x7f1ce81425a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:36.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.102+0000 7f1ceea85700 1 --2- 172.21.15.29:0/432237724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ce80c7f70 0x7f1ce8167010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:36.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.102+0000 7f1ceea85700 1 -- 172.21.15.29:0/432237724 >> 172.21.15.29:0/432237724 conn(0x7f1ce80cb880 msgr2=0x7f1ce8152230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:36.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.102+0000 7f1ceea85700 1 -- 172.21.15.29:0/432237724 shutdown_connections 2024-01-24T14:10:36.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:36.102+0000 7f1ceea85700 1 -- 172.21.15.29:0/432237724 wait complete. 2024-01-24T14:10:36.115 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2024-01-24T14:10:36.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.115 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2024-01-24T14:10:36.135 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:36.136 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:36.136 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:36.136 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:36.136 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:36.136 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:36.136 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:36.136 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:36.136 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:36.136 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:36.136 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:36.136 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:36.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.137 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_00_00_UTC) 2024-01-24T14:10:36.211 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:36.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.211 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_01_00_UTC) 2024-01-24T14:10:36.281 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:36.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.281 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_02_00_UTC) 2024-01-24T14:10:36.352 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:36.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.352 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_03_00_UTC) 2024-01-24T14:10:36.481 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:36.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.481 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_04_00_UTC) 2024-01-24T14:10:36.510 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:36.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.511 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_05_00_UTC) 2024-01-24T14:10:36.581 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:36.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.582 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_06_00_UTC) 2024-01-24T14:10:36.652 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:36.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.652 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_07_00_UTC) 2024-01-24T14:10:36.722 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:36.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.722 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_08_00_UTC) 2024-01-24T14:10:36.792 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:36.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.792 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_09_00_UTC) 2024-01-24T14:10:36.880 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:36.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.881 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-01-24-14_10_00_UTC) 2024-01-24T14:10:36.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:36.919 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2024-01-24T14:10:36.996 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir7 1M 2024-01-24T14:10:37.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.184+0000 7f31cde46700 1 Processor -- start 2024-01-24T14:10:37.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.184+0000 7f31cde46700 1 -- start start 2024-01-24T14:10:37.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31cde46700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f31c8150970 0x7f31c8150d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:37.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31cde46700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31c8155060 0x7f31c80c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:37.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31cde46700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c8069a50 con 0x7f31c8069ee0 2024-01-24T14:10:37.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31cde46700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31c80c2b20 con 0x7f31c8150970 2024-01-24T14:10:37.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31cde46700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c8151240 con 0x7f31c8155060 2024-01-24T14:10:37.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31c6ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f31c8069ee0 0x7f31c8154950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54682/0 (socket says 172.21.15.29:54682) 2024-01-24T14:10:37.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31c6ffd700 1 -- 172.21.15.29:0/4126196579 learned_addr learned my addr 172.21.15.29:0/4126196579 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:37.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31c7fff700 1 --2- 172.21.15.29:0/4126196579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31c8155060 0x7f31c80c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31c77fe700 1 --2- 172.21.15.29:0/4126196579 >> v2:172.21.15.163:3300/0 conn(0x7f31c8150970 0x7f31c8150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31c67fc700 1 -- 172.21.15.29:0/4126196579 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371019433 0 0) 0x7f31c8069a50 con 0x7f31c8069ee0 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31c67fc700 1 -- 172.21.15.29:0/4126196579 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31b00033f0 con 0x7f31c8069ee0 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31c7fff700 1 -- 172.21.15.29:0/4126196579 >> v2:172.21.15.163:3300/0 conn(0x7f31c8150970 msgr2=0x7f31c8150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31c7fff700 1 --2- 172.21.15.29:0/4126196579 >> v2:172.21.15.163:3300/0 conn(0x7f31c8150970 0x7f31c8150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.185+0000 7f31c7fff700 1 -- 172.21.15.29:0/4126196579 >> v1:172.21.15.29:6789/0 conn(0x7f31c8069ee0 legacy=0x7f31c8154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.186+0000 7f31c7fff700 1 -- 172.21.15.29:0/4126196579 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c8155510 con 0x7f31c8155060 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.186+0000 7f31c77fe700 1 --2- 172.21.15.29:0/4126196579 >> v2:172.21.15.163:3300/0 conn(0x7f31c8150970 0x7f31c8150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.186+0000 7f31c7fff700 1 --2- 172.21.15.29:0/4126196579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31c8155060 0x7f31c80c4520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f31c0009e60 tx=0x7f31c0009780).ready entity=mon.2 client_cookie=162112e7523c45c5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.186+0000 7f31c67fc700 1 -- 172.21.15.29:0/4126196579 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c000e070 con 0x7f31c8155060 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.186+0000 7f31c67fc700 1 -- 172.21.15.29:0/4126196579 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c000bab0 con 0x7f31c8155060 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.186+0000 7f31c67fc700 1 -- 172.21.15.29:0/4126196579 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c000be30 con 0x7f31c8155060 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.186+0000 7f31cde46700 1 -- 172.21.15.29:0/4126196579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31c8155060 msgr2=0x7f31c80c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:37.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.186+0000 7f31cde46700 1 --2- 172.21.15.29:0/4126196579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31c8155060 0x7f31c80c4520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f31c0009e60 tx=0x7f31c0009780).stop 2024-01-24T14:10:37.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.186+0000 7f31cde46700 1 -- 172.21.15.29:0/4126196579 shutdown_connections 2024-01-24T14:10:37.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.186+0000 7f31cde46700 1 --2- 172.21.15.29:0/4126196579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31c8155060 0x7f31c80c4520 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:37.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.187+0000 7f31cde46700 1 --2- 172.21.15.29:0/4126196579 >> v2:172.21.15.163:3300/0 conn(0x7f31c8150970 0x7f31c8150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:37.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.187+0000 7f31cde46700 1 -- 172.21.15.29:0/4126196579 >> 172.21.15.29:0/4126196579 conn(0x7f31c8067b20 msgr2=0x7f31c80638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:37.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.187+0000 7f31cde46700 1 -- 172.21.15.29:0/4126196579 shutdown_connections 2024-01-24T14:10:37.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.187+0000 7f31cde46700 1 -- 172.21.15.29:0/4126196579 wait complete. 2024-01-24T14:10:37.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.188+0000 7f31cde46700 1 Processor -- start 2024-01-24T14:10:37.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.188+0000 7f31cde46700 1 -- start start 2024-01-24T14:10:37.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31cde46700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31c8069ee0 0x7f31c81452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:37.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31cde46700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31c8150970 0x7f31c814c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:37.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31cde46700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31c8155060 0x7f31c814ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:37.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31cde46700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c814d1e0 con 0x7f31c8150970 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31cde46700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c80c7310 con 0x7f31c8155060 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31cde46700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c80c6700 con 0x7f31c8069ee0 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31c6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31c8150970 0x7f31c814c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31c6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31c8150970 0x7f31c814c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52698/0 (socket says 172.21.15.29:52698) 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31c6ffd700 1 -- 172.21.15.29:0/1058532457 learned_addr learned my addr 172.21.15.29:0/1058532457 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31c7fff700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31c8155060 0x7f31c814ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31c77fe700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31c8069ee0 0x7f31c81452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31c6ffd700 1 -- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31c8069ee0 msgr2=0x7f31c81452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31c6ffd700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31c8069ee0 0x7f31c81452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31c6ffd700 1 -- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31c8155060 msgr2=0x7f31c814ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31c6ffd700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31c8155060 0x7f31c814ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.189+0000 7f31c6ffd700 1 -- 172.21.15.29:0/1058532457 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b80092a0 con 0x7f31c8150970 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.190+0000 7f31c6ffd700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31c8150970 0x7f31c814c800 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f31bc00c590 tx=0x7f31bc007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:37.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.190+0000 7f31af7fe700 1 -- 172.21.15.29:0/1058532457 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc0072e0 con 0x7f31c8150970 2024-01-24T14:10:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.190+0000 7f31af7fe700 1 -- 172.21.15.29:0/1058532457 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31bc007440 con 0x7f31c8150970 2024-01-24T14:10:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.190+0000 7f31af7fe700 1 -- 172.21.15.29:0/1058532457 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc0057a0 con 0x7f31c8150970 2024-01-24T14:10:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.190+0000 7f31cde46700 1 -- 172.21.15.29:0/1058532457 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c0009a20 con 0x7f31c8150970 2024-01-24T14:10:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.190+0000 7f31cde46700 1 -- 172.21.15.29:0/1058532457 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c80c6cc0 con 0x7f31c8150970 2024-01-24T14:10:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.191+0000 7f31cde46700 1 -- 172.21.15.29:0/1058532457 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c80475e0 con 0x7f31c8150970 2024-01-24T14:10:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.191+0000 7f31af7fe700 1 -- 172.21.15.29:0/1058532457 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f31bc005bd0 con 0x7f31c8150970 2024-01-24T14:10:37.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.191+0000 7f31af7fe700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f31b405f2c0 0x7f31b4061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:37.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.192+0000 7f31af7fe700 1 -- 172.21.15.29:0/1058532457 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7669+0+0 (secure 0 0 0) 0x7f31bc07a120 con 0x7f31c8150970 2024-01-24T14:10:37.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.192+0000 7f31c77fe700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f31b405f2c0 0x7f31b4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:37.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.192+0000 7f31c77fe700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f31b405f2c0 0x7f31b4061780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f31c80664f0 tx=0x7f31b8009060).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:37.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.196+0000 7f31af7fe700 1 -- 172.21.15.29:0/1058532457 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f31bc04ac60 con 0x7f31c8150970 2024-01-24T14:10:37.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.403+0000 7f31cde46700 1 -- 172.21.15.29:0/1058532457 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f31c81557f0 con 0x7f31b405f2c0 2024-01-24T14:10:37.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.407+0000 7f31af7fe700 1 -- 172.21.15.29:0/1058532457 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f31c81557f0 con 0x7f31b405f2c0 2024-01-24T14:10:37.407 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /snap_test_dir1/dir7 2024-01-24T14:10:37.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.409+0000 7f31cde46700 1 -- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f31b405f2c0 msgr2=0x7f31b4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:37.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.409+0000 7f31cde46700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f31b405f2c0 0x7f31b4061780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f31c80664f0 tx=0x7f31b8009060).stop 2024-01-24T14:10:37.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.409+0000 7f31cde46700 1 -- 172.21.15.29:0/1058532457 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31c8150970 msgr2=0x7f31c814c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:37.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.409+0000 7f31cde46700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31c8150970 0x7f31c814c800 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f31bc00c590 tx=0x7f31bc007c80).stop 2024-01-24T14:10:37.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.409+0000 7f31cde46700 1 -- 172.21.15.29:0/1058532457 shutdown_connections 2024-01-24T14:10:37.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.409+0000 7f31cde46700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31c8155060 0x7f31c814ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:37.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.409+0000 7f31cde46700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31c8069ee0 0x7f31c81452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:37.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.409+0000 7f31cde46700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f31b405f2c0 0x7f31b4061780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:37.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.409+0000 7f31cde46700 1 --2- 172.21.15.29:0/1058532457 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31c8150970 0x7f31c814c800 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:37.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.409+0000 7f31cde46700 1 -- 172.21.15.29:0/1058532457 >> 172.21.15.29:0/1058532457 conn(0x7f31c8067b20 msgr2=0x7f31c8147090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:37.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.409+0000 7f31cde46700 1 -- 172.21.15.29:0/1058532457 shutdown_connections 2024-01-24T14:10:37.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:37.409+0000 7f31cde46700 1 -- 172.21.15.29:0/1058532457 wait complete. 2024-01-24T14:10:37.423 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2024-01-24T14:10:37.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:37.423 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2024-01-24T14:10:37.442 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:37.442 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:37.442 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:37.443 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:37.443 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:37.443 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:37.443 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:37.443 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:37.443 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:37.443 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:37.443 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:37.444 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:37.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:37.444 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_00_00_UTC) 2024-01-24T14:10:37.513 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:37.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:37.514 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_01_00_UTC) 2024-01-24T14:10:37.583 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:37.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:37.584 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_02_00_UTC) 2024-01-24T14:10:37.654 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:37.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:37.655 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_03_00_UTC) 2024-01-24T14:10:37.726 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:37.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:37.726 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_04_00_UTC) 2024-01-24T14:10:37.797 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:37.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:37.797 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_05_00_UTC) 2024-01-24T14:10:37.882 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:37.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:37.883 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_06_00_UTC) 2024-01-24T14:10:37.916 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:37.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:37.916 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_07_00_UTC) 2024-01-24T14:10:38.025 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:38.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:38.026 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_08_00_UTC) 2024-01-24T14:10:38.056 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:38.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:38.056 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_09_00_UTC) 2024-01-24T14:10:38.123 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:38.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:38.123 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-01-24-14_10_00_UTC) 2024-01-24T14:10:38.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:38.194 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2024-01-24T14:10:38.260 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir8 1M 2024-01-24T14:10:38.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.444+0000 7f197782b700 1 Processor -- start 2024-01-24T14:10:38.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.444+0000 7f197782b700 1 -- start start 2024-01-24T14:10:38.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.445+0000 7f197782b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f197014b0a0 0x7f197014b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:38.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.445+0000 7f197782b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19700c3660 0x7f19700c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:38.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.445+0000 7f197782b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19701557e0 con 0x7f19700c48e0 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.445+0000 7f197782b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19700c28a0 con 0x7f19700c3660 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.445+0000 7f197782b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19700c2b80 con 0x7f197014b0a0 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.445+0000 7f1974dc6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19700c3660 0x7f19700c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.445+0000 7f1974dc6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19700c3660 0x7f19700c43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52704/0 (socket says 172.21.15.29:52704) 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.445+0000 7f1974dc6700 1 -- 172.21.15.29:0/3946094199 learned_addr learned my addr 172.21.15.29:0/3946094199 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.446+0000 7f19755c7700 1 --2- 172.21.15.29:0/3946094199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f197014b0a0 0x7f197014b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.446+0000 7f1974dc6700 1 -- 172.21.15.29:0/3946094199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f197014b0a0 msgr2=0x7f197014b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.446+0000 7f1974dc6700 1 --2- 172.21.15.29:0/3946094199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f197014b0a0 0x7f197014b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.446+0000 7f1974dc6700 1 -- 172.21.15.29:0/3946094199 >> v1:172.21.15.163:6789/0 conn(0x7f19700c48e0 legacy=0x7f1970154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.446+0000 7f1974dc6700 1 -- 172.21.15.29:0/3946094199 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19700c2e60 con 0x7f19700c3660 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.446+0000 7f1974dc6700 1 --2- 172.21.15.29:0/3946094199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19700c3660 0x7f19700c43f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f1960006fd0 tx=0x7f19600090f0).ready entity=mon.0 client_cookie=a933ba1a613215e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.446+0000 7f1967fff700 1 -- 172.21.15.29:0/3946094199 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000b600 con 0x7f19700c3660 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.446+0000 7f1967fff700 1 -- 172.21.15.29:0/3946094199 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196000b760 con 0x7f19700c3660 2024-01-24T14:10:38.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.446+0000 7f1967fff700 1 -- 172.21.15.29:0/3946094199 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000bae0 con 0x7f19700c3660 2024-01-24T14:10:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.447+0000 7f197782b700 1 -- 172.21.15.29:0/3946094199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19700c3660 msgr2=0x7f19700c43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.447+0000 7f197782b700 1 --2- 172.21.15.29:0/3946094199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19700c3660 0x7f19700c43f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f1960006fd0 tx=0x7f19600090f0).stop 2024-01-24T14:10:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.447+0000 7f197782b700 1 -- 172.21.15.29:0/3946094199 shutdown_connections 2024-01-24T14:10:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.447+0000 7f197782b700 1 --2- 172.21.15.29:0/3946094199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f197014b0a0 0x7f197014b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.447+0000 7f197782b700 1 --2- 172.21.15.29:0/3946094199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19700c3660 0x7f19700c43f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.447+0000 7f197782b700 1 -- 172.21.15.29:0/3946094199 >> 172.21.15.29:0/3946094199 conn(0x7f1970067b20 msgr2=0x7f19700638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.447+0000 7f197782b700 1 -- 172.21.15.29:0/3946094199 shutdown_connections 2024-01-24T14:10:38.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.447+0000 7f197782b700 1 -- 172.21.15.29:0/3946094199 wait complete. 2024-01-24T14:10:38.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.448+0000 7f197782b700 1 Processor -- start 2024-01-24T14:10:38.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.448+0000 7f197782b700 1 -- start start 2024-01-24T14:10:38.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.449+0000 7f197782b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19700c3660 0x7f197014a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:38.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.449+0000 7f197782b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19700c48e0 0x7f19700bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.449+0000 7f197782b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f197014b0a0 0x7f19700c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.449+0000 7f197782b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19700bf100 con 0x7f19700c48e0 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.449+0000 7f197782b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19700aab70 con 0x7f197014b0a0 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.449+0000 7f197782b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19700aae50 con 0x7f19700c3660 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.449+0000 7f1974dc6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19700c48e0 0x7f19700bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f1974dc6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19700c48e0 0x7f19700bfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52708/0 (socket says 172.21.15.29:52708) 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f1974dc6700 1 -- 172.21.15.29:0/3393201766 learned_addr learned my addr 172.21.15.29:0/3393201766 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f19755c7700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19700c3660 0x7f197014a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f1975dc8700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f197014b0a0 0x7f19700c1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f1974dc6700 1 -- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19700c3660 msgr2=0x7f197014a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f1974dc6700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19700c3660 0x7f197014a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f1974dc6700 1 -- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f197014b0a0 msgr2=0x7f19700c1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:38.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f1974dc6700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f197014b0a0 0x7f19700c1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f1974dc6700 1 -- 172.21.15.29:0/3393201766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1960009370 con 0x7f19700c48e0 2024-01-24T14:10:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f1974dc6700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19700c48e0 0x7f19700bfd20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f1960011000 tx=0x7f1960017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f19657fa700 1 -- 172.21.15.29:0/3393201766 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000bea0 con 0x7f19700c48e0 2024-01-24T14:10:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f19657fa700 1 -- 172.21.15.29:0/3393201766 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196000b8d0 con 0x7f19700c48e0 2024-01-24T14:10:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f197782b700 1 -- 172.21.15.29:0/3393201766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19700ab130 con 0x7f19700c48e0 2024-01-24T14:10:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f19657fa700 1 -- 172.21.15.29:0/3393201766 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196001c670 con 0x7f19700c48e0 2024-01-24T14:10:38.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.450+0000 7f197782b700 1 -- 172.21.15.29:0/3393201766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19700ab6f0 con 0x7f19700c48e0 2024-01-24T14:10:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.452+0000 7f197782b700 1 -- 172.21.15.29:0/3393201766 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19700475e0 con 0x7f19700c48e0 2024-01-24T14:10:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.452+0000 7f19657fa700 1 -- 172.21.15.29:0/3393201766 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f196001ca20 con 0x7f19700c48e0 2024-01-24T14:10:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.452+0000 7f19657fa700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f195405f2c0 0x7f1954061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.452+0000 7f19657fa700 1 -- 172.21.15.29:0/3393201766 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7493+0+0 (secure 0 0 0) 0x7f196007c630 con 0x7f19700c48e0 2024-01-24T14:10:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.452+0000 7f19755c7700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f195405f2c0 0x7f1954061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.453+0000 7f19755c7700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f195405f2c0 0x7f1954061780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f196c00bb10 tx=0x7f196c008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:38.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.457+0000 7f19657fa700 1 -- 172.21.15.29:0/3393201766 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1960050c50 con 0x7f19700c48e0 2024-01-24T14:10:38.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.665+0000 7f197782b700 1 -- 172.21.15.29:0/3393201766 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f19700abd30 con 0x7f195405f2c0 2024-01-24T14:10:38.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.669+0000 7f19657fa700 1 -- 172.21.15.29:0/3393201766 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f19700abd30 con 0x7f195405f2c0 2024-01-24T14:10:38.670 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /snap_test_dir1/dir8 2024-01-24T14:10:38.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.671+0000 7f197782b700 1 -- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f195405f2c0 msgr2=0x7f1954061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:38.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.671+0000 7f197782b700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f195405f2c0 0x7f1954061780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f196c00bb10 tx=0x7f196c008040).stop 2024-01-24T14:10:38.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.671+0000 7f197782b700 1 -- 172.21.15.29:0/3393201766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19700c48e0 msgr2=0x7f19700bfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:38.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.671+0000 7f197782b700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19700c48e0 0x7f19700bfd20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f1960011000 tx=0x7f1960017000).stop 2024-01-24T14:10:38.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.671+0000 7f197782b700 1 -- 172.21.15.29:0/3393201766 shutdown_connections 2024-01-24T14:10:38.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.671+0000 7f197782b700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f197014b0a0 0x7f19700c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:38.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.671+0000 7f197782b700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19700c3660 0x7f197014a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:38.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.671+0000 7f197782b700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f195405f2c0 0x7f1954061780 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:38.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.671+0000 7f197782b700 1 --2- 172.21.15.29:0/3393201766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19700c48e0 0x7f19700bfd20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:38.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.671+0000 7f197782b700 1 -- 172.21.15.29:0/3393201766 >> 172.21.15.29:0/3393201766 conn(0x7f1970067b20 msgr2=0x7f19700a9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:38.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.671+0000 7f197782b700 1 -- 172.21.15.29:0/3393201766 shutdown_connections 2024-01-24T14:10:38.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:38.671+0000 7f197782b700 1 -- 172.21.15.29:0/3393201766 wait complete. 2024-01-24T14:10:38.685 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2024-01-24T14:10:38.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:38.685 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2024-01-24T14:10:38.705 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:38.705 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:38.705 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:38.705 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:38.705 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:38.705 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:38.705 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:38.705 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:38.706 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:38.706 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:38.706 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:38.706 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:38.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:38.706 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_00_00_UTC) 2024-01-24T14:10:38.775 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:38.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:38.775 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_01_00_UTC) 2024-01-24T14:10:38.846 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:38.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:38.847 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_02_00_UTC) 2024-01-24T14:10:38.919 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:38.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:38.919 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_03_00_UTC) 2024-01-24T14:10:38.998 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:38.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:38.998 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_04_00_UTC) 2024-01-24T14:10:39.065 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:39.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:39.066 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_05_00_UTC) 2024-01-24T14:10:39.135 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:39.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:39.135 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_06_00_UTC) 2024-01-24T14:10:39.205 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:39.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:39.205 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_07_00_UTC) 2024-01-24T14:10:39.275 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:39.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:39.275 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_08_00_UTC) 2024-01-24T14:10:39.344 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:39.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:39.345 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_09_00_UTC) 2024-01-24T14:10:39.415 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:39.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:39.415 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-01-24-14_10_00_UTC) 2024-01-24T14:10:39.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:39.486 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2024-01-24T14:10:39.552 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir9 1M 2024-01-24T14:10:39.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.735+0000 7fa0cab10700 1 Processor -- start 2024-01-24T14:10:39.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.735+0000 7fa0cab10700 1 -- start start 2024-01-24T14:10:39.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.736+0000 7fa0cab10700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa0c4154ec0 0x7fa0c41552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:39.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.736+0000 7fa0cab10700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0c4063d40 0x7fa0c4151350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.736+0000 7fa0cab10700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c4068540 con 0x7fa0c40638c0 2024-01-24T14:10:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.736+0000 7fa0cab10700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40641c0 con 0x7fa0c4154ec0 2024-01-24T14:10:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.736+0000 7fa0cab10700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40689d0 con 0x7fa0c4063d40 2024-01-24T14:10:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c88ac700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa0c4154ec0 0x7fa0c41552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c88ac700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa0c4154ec0 0x7fa0c41552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52710/0 (socket says 172.21.15.29:52710) 2024-01-24T14:10:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c88ac700 1 -- 172.21.15.29:0/437751436 learned_addr learned my addr 172.21.15.29:0/437751436 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c90ad700 1 --2- 172.21.15.29:0/437751436 >> v2:172.21.15.163:3300/0 conn(0x7fa0c4063d40 0x7fa0c4151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c88ac700 1 -- 172.21.15.29:0/437751436 >> v2:172.21.15.163:3300/0 conn(0x7fa0c4063d40 msgr2=0x7fa0c4151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c88ac700 1 --2- 172.21.15.29:0/437751436 >> v2:172.21.15.163:3300/0 conn(0x7fa0c4063d40 0x7fa0c4151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c88ac700 1 -- 172.21.15.29:0/437751436 >> v1:172.21.15.29:6789/0 conn(0x7fa0c40638c0 legacy=0x7fa0c4066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c88ac700 1 -- 172.21.15.29:0/437751436 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4067c10 con 0x7fa0c4154ec0 2024-01-24T14:10:39.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c90ad700 1 --2- 172.21.15.29:0/437751436 >> v2:172.21.15.163:3300/0 conn(0x7fa0c4063d40 0x7fa0c4151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:10:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c88ac700 1 --2- 172.21.15.29:0/437751436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c4154ec0 0x7fa0c41552a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fa0c406a240 tx=0x7fa0b80090f0).ready entity=mon.0 client_cookie=d6cbc36d5beaf2f8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c37fe700 1 -- 172.21.15.29:0/437751436 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800b600 con 0x7fa0c4154ec0 2024-01-24T14:10:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c37fe700 1 -- 172.21.15.29:0/437751436 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b800b760 con 0x7fa0c4154ec0 2024-01-24T14:10:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.737+0000 7fa0c37fe700 1 -- 172.21.15.29:0/437751436 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800bae0 con 0x7fa0c4154ec0 2024-01-24T14:10:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.738+0000 7fa0cab10700 1 -- 172.21.15.29:0/437751436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c4154ec0 msgr2=0x7fa0c41552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.738+0000 7fa0cab10700 1 --2- 172.21.15.29:0/437751436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c4154ec0 0x7fa0c41552a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fa0c406a240 tx=0x7fa0b80090f0).stop 2024-01-24T14:10:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.738+0000 7fa0cab10700 1 -- 172.21.15.29:0/437751436 shutdown_connections 2024-01-24T14:10:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.738+0000 7fa0cab10700 1 --2- 172.21.15.29:0/437751436 >> v2:172.21.15.163:3300/0 conn(0x7fa0c4063d40 0x7fa0c4151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.738+0000 7fa0cab10700 1 --2- 172.21.15.29:0/437751436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c4154ec0 0x7fa0c41552a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.738+0000 7fa0cab10700 1 -- 172.21.15.29:0/437751436 >> 172.21.15.29:0/437751436 conn(0x7fa0c40c8bd0 msgr2=0x7fa0c40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.738+0000 7fa0cab10700 1 -- 172.21.15.29:0/437751436 shutdown_connections 2024-01-24T14:10:39.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.738+0000 7fa0cab10700 1 -- 172.21.15.29:0/437751436 wait complete. 2024-01-24T14:10:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.739+0000 7fa0cab10700 1 Processor -- start 2024-01-24T14:10:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.739+0000 7fa0cab10700 1 -- start start 2024-01-24T14:10:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.740+0000 7fa0cab10700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c40638c0 0x7fa0c40c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.740+0000 7fa0cab10700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0c4063d40 0x7fa0c4141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.740+0000 7fa0cab10700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0c4154ec0 0x7fa0c4141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.740+0000 7fa0cab10700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c41647b0 con 0x7fa0c40638c0 2024-01-24T14:10:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.740+0000 7fa0cab10700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4141d20 con 0x7fa0c4063d40 2024-01-24T14:10:39.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.740+0000 7fa0cab10700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40aa0a0 con 0x7fa0c4154ec0 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.740+0000 7fa0c88ac700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c40638c0 0x7fa0c40c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.740+0000 7fa0c88ac700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c40638c0 0x7fa0c40c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52722/0 (socket says 172.21.15.29:52722) 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.740+0000 7fa0c88ac700 1 -- 172.21.15.29:0/133341252 learned_addr learned my addr 172.21.15.29:0/133341252 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.740+0000 7fa0c3fff700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0c4063d40 0x7fa0c4141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.740+0000 7fa0c90ad700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0c4154ec0 0x7fa0c4141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.741+0000 7fa0c88ac700 1 -- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0c4154ec0 msgr2=0x7fa0c4141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.741+0000 7fa0c88ac700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0c4154ec0 0x7fa0c4141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.741+0000 7fa0c88ac700 1 -- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0c4063d40 msgr2=0x7fa0c4141340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.741+0000 7fa0c88ac700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0c4063d40 0x7fa0c4141340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.741+0000 7fa0c88ac700 1 -- 172.21.15.29:0/133341252 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b8009370 con 0x7fa0c40638c0 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.741+0000 7fa0c90ad700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0c4154ec0 0x7fa0c4141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.741+0000 7fa0c88ac700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c40638c0 0x7fa0c40c7b60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7fa0b8009130 tx=0x7fa0b8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.741+0000 7fa0c0ff9700 1 -- 172.21.15.29:0/133341252 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800b8d0 con 0x7fa0c40638c0 2024-01-24T14:10:39.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.741+0000 7fa0c0ff9700 1 -- 172.21.15.29:0/133341252 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b800ba70 con 0x7fa0c40638c0 2024-01-24T14:10:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.741+0000 7fa0cab10700 1 -- 172.21.15.29:0/133341252 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c40a9d70 con 0x7fa0c40638c0 2024-01-24T14:10:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.741+0000 7fa0c0ff9700 1 -- 172.21.15.29:0/133341252 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b801c670 con 0x7fa0c40638c0 2024-01-24T14:10:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.741+0000 7fa0cab10700 1 -- 172.21.15.29:0/133341252 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40ad090 con 0x7fa0c40638c0 2024-01-24T14:10:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.742+0000 7fa0cab10700 1 -- 172.21.15.29:0/133341252 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c40aa380 con 0x7fa0c40638c0 2024-01-24T14:10:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.743+0000 7fa0c0ff9700 1 -- 172.21.15.29:0/133341252 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fa0b801c9e0 con 0x7fa0c40638c0 2024-01-24T14:10:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.743+0000 7fa0c0ff9700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa0a805f2c0 0x7fa0a8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.743+0000 7fa0c0ff9700 1 -- 172.21.15.29:0/133341252 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7349+0+0 (secure 0 0 0) 0x7fa0b807c640 con 0x7fa0c40638c0 2024-01-24T14:10:39.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.743+0000 7fa0c3fff700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa0a805f2c0 0x7fa0a8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:39.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.744+0000 7fa0c3fff700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa0a805f2c0 0x7fa0a8061780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa0c4168e90 tx=0x7fa0b0007d40).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:39.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.747+0000 7fa0c0ff9700 1 -- 172.21.15.29:0/133341252 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa0b804d2c0 con 0x7fa0c40638c0 2024-01-24T14:10:39.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.974+0000 7fa0cab10700 1 -- 172.21.15.29:0/133341252 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0c40656e0 con 0x7fa0a805f2c0 2024-01-24T14:10:39.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.979+0000 7fa0c0ff9700 1 -- 172.21.15.29:0/133341252 <== mgr.4937 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fa0c40656e0 con 0x7fa0a805f2c0 2024-01-24T14:10:39.979 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /snap_test_dir1/dir9 2024-01-24T14:10:39.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.981+0000 7fa0cab10700 1 -- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa0a805f2c0 msgr2=0x7fa0a8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:39.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.981+0000 7fa0cab10700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa0a805f2c0 0x7fa0a8061780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa0c4168e90 tx=0x7fa0b0007d40).stop 2024-01-24T14:10:39.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.981+0000 7fa0cab10700 1 -- 172.21.15.29:0/133341252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c40638c0 msgr2=0x7fa0c40c7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:39.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.981+0000 7fa0cab10700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c40638c0 0x7fa0c40c7b60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7fa0b8009130 tx=0x7fa0b8017000).stop 2024-01-24T14:10:39.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.981+0000 7fa0cab10700 1 -- 172.21.15.29:0/133341252 shutdown_connections 2024-01-24T14:10:39.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.981+0000 7fa0cab10700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0c4154ec0 0x7fa0c4141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:39.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.981+0000 7fa0cab10700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa0a805f2c0 0x7fa0a8061780 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:39.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.981+0000 7fa0cab10700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c40638c0 0x7fa0c40c7b60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:39.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.981+0000 7fa0cab10700 1 --2- 172.21.15.29:0/133341252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0c4063d40 0x7fa0c4141340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:39.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.981+0000 7fa0cab10700 1 -- 172.21.15.29:0/133341252 >> 172.21.15.29:0/133341252 conn(0x7fa0c40c8bd0 msgr2=0x7fa0c4066450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:39.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.981+0000 7fa0cab10700 1 -- 172.21.15.29:0/133341252 shutdown_connections 2024-01-24T14:10:39.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:39.981+0000 7fa0cab10700 1 -- 172.21.15.29:0/133341252 wait complete. 2024-01-24T14:10:39.995 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2024-01-24T14:10:39.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:39.995 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2024-01-24T14:10:40.015 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:40.015 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:40.015 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:40.016 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:40.016 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:40.016 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:40.016 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:40.016 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:40.016 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:40.016 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:40.016 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:40.016 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_00_00_UTC 2024-01-24T14:10:40.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:40.017 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_00_00_UTC) 2024-01-24T14:10:40.125 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_01_00_UTC 2024-01-24T14:10:40.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:40.125 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_01_00_UTC) 2024-01-24T14:10:40.153 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_02_00_UTC 2024-01-24T14:10:40.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:40.153 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_02_00_UTC) 2024-01-24T14:10:40.220 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_03_00_UTC 2024-01-24T14:10:40.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:40.220 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_03_00_UTC) 2024-01-24T14:10:40.289 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_04_00_UTC 2024-01-24T14:10:40.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:40.291 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_04_00_UTC) 2024-01-24T14:10:40.360 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_05_00_UTC 2024-01-24T14:10:40.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:40.360 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_05_00_UTC) 2024-01-24T14:10:40.433 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_06_00_UTC 2024-01-24T14:10:40.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:40.433 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_06_00_UTC) 2024-01-24T14:10:40.503 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_07_00_UTC 2024-01-24T14:10:40.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:40.503 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_07_00_UTC) 2024-01-24T14:10:40.574 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_08_00_UTC 2024-01-24T14:10:40.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:40.574 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_08_00_UTC) 2024-01-24T14:10:40.647 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_09_00_UTC 2024-01-24T14:10:40.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:40.647 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_09_00_UTC) 2024-01-24T14:10:40.718 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_10_00_UTC 2024-01-24T14:10:40.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:40.718 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-01-24-14_10_00_UTC) 2024-01-24T14:10:40.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:40.789 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2024-01-24T14:10:40.857 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-01-24T14:10:41.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.037+0000 7f87e0b4c700 1 Processor -- start 2024-01-24T14:10:41.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.037+0000 7f87e0b4c700 1 -- start start 2024-01-24T14:10:41.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.038+0000 7f87e0b4c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87dc13b660 0x7f87dc063a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:41.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.039+0000 7f87e0b4c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc067aa0 con 0x7f87dc066520 2024-01-24T14:10:41.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.039+0000 7f87e0b4c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc068900 con 0x7f87dc0beb10 2024-01-24T14:10:41.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.039+0000 7f87e0b4c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc15ce20 con 0x7f87dc13b660 2024-01-24T14:10:41.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.039+0000 7f87dad9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f87dc0beb10 0x7f87dc15c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:54968/0 (socket says 172.21.15.29:54968) 2024-01-24T14:10:41.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.039+0000 7f87dad9d700 1 -- 172.21.15.29:0/1757538559 learned_addr learned my addr 172.21.15.29:0/1757538559 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:41.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.039+0000 7f87da59c700 1 --2- 172.21.15.29:0/1757538559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87dc13b660 0x7f87dc063a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:41.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254862624 0 0) 0x7f87dc067aa0 con 0x7f87dc066520 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87c40033f0 con 0x7f87dc066520 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2101586307 0 0) 0x7f87c40033f0 con 0x7f87dc066520 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87dc13b660 msgr2=0x7f87dc063a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 --2- 172.21.15.29:0/1757538559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87dc13b660 0x7f87dc063a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 >> v1:172.21.15.163:6789/0 conn(0x7f87dc0beb10 legacy=0x7f87dc15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc13ba40 con 0x7f87dc066520 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f87cc001470 con 0x7f87dc066520 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 >> v1:172.21.15.29:6789/0 conn(0x7f87dc066520 legacy=0x7f87dc15c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 --2- 172.21.15.29:0/1757538559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87c4003910 0x7f87c4003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 --2- 172.21.15.29:0/1757538559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87c4004690 0x7f87c4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 --2- 172.21.15.29:0/1757538559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87c4005000 0x7f87c400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc15ce20 con 0x7f87c4005000 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c400be00 con 0x7f87c4003910 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c400bfc0 con 0x7f87c4004690 2024-01-24T14:10:41.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.040+0000 7f87d9d9b700 1 --2- 172.21.15.29:0/1757538559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87c4004690 0x7f87c4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:41.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.041+0000 7f87dad9d700 1 --2- 172.21.15.29:0/1757538559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87c4005000 0x7f87c400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:41.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.041+0000 7f87d9d9b700 1 -- 172.21.15.29:0/1757538559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87c4003910 msgr2=0x7f87c4003d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-24T14:10:41.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.041+0000 7f87d9d9b700 1 --2- 172.21.15.29:0/1757538559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87c4003910 0x7f87c4003d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:41.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.041+0000 7f87d9d9b700 1 -- 172.21.15.29:0/1757538559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87c4005000 msgr2=0x7f87c400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:41.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.041+0000 7f87d9d9b700 1 --2- 172.21.15.29:0/1757538559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87c4005000 0x7f87c400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:41.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.041+0000 7f87d9d9b700 1 -- 172.21.15.29:0/1757538559 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c40033f0 con 0x7f87c4004690 2024-01-24T14:10:41.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.041+0000 7f87d9d9b700 1 --2- 172.21.15.29:0/1757538559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87c4004690 0x7f87c4004b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f87dc065320 tx=0x7f87cc0076b0).ready entity=mon.2 client_cookie=1bce0f92fb8917aa server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:41.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.042+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00c980 con 0x7f87c4004690 2024-01-24T14:10:41.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.042+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc00cae0 con 0x7f87c4004690 2024-01-24T14:10:41.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.042+0000 7f87d959a700 1 -- 172.21.15.29:0/1757538559 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00ce60 con 0x7f87c4004690 2024-01-24T14:10:41.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.043+0000 7f87e0b4c700 1 -- 172.21.15.29:0/1757538559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87c4004690 msgr2=0x7f87c4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:41.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.043+0000 7f87e0b4c700 1 --2- 172.21.15.29:0/1757538559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87c4004690 0x7f87c4004b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f87dc065320 tx=0x7f87cc0076b0).stop 2024-01-24T14:10:41.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.043+0000 7f87dad9d700 1 -- 172.21.15.29:0/1757538559 reap_dead start 2024-01-24T14:10:41.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.043+0000 7f87e0b4c700 1 -- 172.21.15.29:0/1757538559 shutdown_connections 2024-01-24T14:10:41.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.043+0000 7f87e0b4c700 1 -- 172.21.15.29:0/1757538559 >> 172.21.15.29:0/1757538559 conn(0x7f87dc0c8ba0 msgr2=0x7f87dc0c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:41.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.043+0000 7f87e0b4c700 1 -- 172.21.15.29:0/1757538559 shutdown_connections 2024-01-24T14:10:41.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.043+0000 7f87e0b4c700 1 -- 172.21.15.29:0/1757538559 wait complete. 2024-01-24T14:10:41.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.045+0000 7f87e0b4c700 1 Processor -- start 2024-01-24T14:10:41.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.045+0000 7f87e0b4c700 1 -- start start 2024-01-24T14:10:41.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.046+0000 7f87e0b4c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87c4004690 0x7f87dc1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:41.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.046+0000 7f87e0b4c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87dc151a50 0x7f87dc13aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87e0b4c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87dc0ad9c0 0x7f87dc0ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87e0b4c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0a93a0 con 0x7f87c4004690 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87e0b4c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc140490 con 0x7f87dc151a50 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87e0b4c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0ac530 con 0x7f87dc0ad9c0 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87d9d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87dc151a50 0x7f87dc13aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87d9d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87dc151a50 0x7f87dc13aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:53024/0 (socket says 172.21.15.29:53024) 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87d9d9b700 1 -- 172.21.15.29:0/2883944946 learned_addr learned my addr 172.21.15.29:0/2883944946 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87dad9d700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87dc0ad9c0 0x7f87dc0ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87da59c700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87c4004690 0x7f87dc1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87d9d9b700 1 -- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87dc0ad9c0 msgr2=0x7f87dc0ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87d9d9b700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87dc0ad9c0 0x7f87dc0ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87d9d9b700 1 -- 172.21.15.29:0/2883944946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87c4004690 msgr2=0x7f87dc1512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:41.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87d9d9b700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87c4004690 0x7f87dc1512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:41.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.047+0000 7f87d9d9b700 1 -- 172.21.15.29:0/2883944946 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc1517c0 con 0x7f87dc151a50 2024-01-24T14:10:41.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.048+0000 7f87da59c700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87c4004690 0x7f87dc1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:10:41.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.048+0000 7f87d9d9b700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87dc151a50 0x7f87dc13aca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f87cc001940 tx=0x7f87cc017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:41.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.048+0000 7f87ca7fc700 1 -- 172.21.15.29:0/2883944946 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc007350 con 0x7f87dc151a50 2024-01-24T14:10:41.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.048+0000 7f87ca7fc700 1 -- 172.21.15.29:0/2883944946 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc0074b0 con 0x7f87dc151a50 2024-01-24T14:10:41.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.048+0000 7f87ca7fc700 1 -- 172.21.15.29:0/2883944946 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0068d0 con 0x7f87dc151a50 2024-01-24T14:10:41.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.048+0000 7f87e0b4c700 1 -- 172.21.15.29:0/2883944946 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc0ae310 con 0x7f87dc151a50 2024-01-24T14:10:41.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.048+0000 7f87e0b4c700 1 -- 172.21.15.29:0/2883944946 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc0ac7b0 con 0x7f87dc151a50 2024-01-24T14:10:41.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.050+0000 7f87ca7fc700 1 -- 172.21.15.29:0/2883944946 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f87cc006c40 con 0x7f87dc151a50 2024-01-24T14:10:41.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.050+0000 7f87e0b4c700 1 -- 172.21.15.29:0/2883944946 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc06c230 con 0x7f87dc151a50 2024-01-24T14:10:41.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.051+0000 7f87ca7fc700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f87bc05f2c0 0x7f87bc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:41.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.051+0000 7f87ca7fc700 1 -- 172.21.15.29:0/2883944946 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 8209+0+0 (secure 0 0 0) 0x7f87cc07c280 con 0x7f87dc151a50 2024-01-24T14:10:41.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.051+0000 7f87da59c700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f87bc05f2c0 0x7f87bc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:41.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.052+0000 7f87da59c700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f87bc05f2c0 0x7f87bc061780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f87d0006d80 tx=0x7f87d0008040).ready entity=mgr.4937 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:41.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.056+0000 7f87ca7fc700 1 -- 172.21.15.29:0/2883944946 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f87cc04cae0 con 0x7f87dc151a50 2024-01-24T14:10:41.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.295+0000 7f87e0b4c700 1 -- 172.21.15.29:0/2883944946 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f87dc067cf0 con 0x7f87dc151a50 2024-01-24T14:10:41.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.879+0000 7f87ca7fc700 1 -- 172.21.15.29:0/2883944946 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f87cc0795e0 con 0x7f87dc151a50 2024-01-24T14:10:41.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.882+0000 7f87ca7fc700 1 -- 172.21.15.29:0/2883944946 <== mon.1 v2:172.21.15.163:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v11) v1 ==== 93+0+0 (secure 0 0 0) 0x7f87cc041f40 con 0x7f87dc151a50 2024-01-24T14:10:41.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.884+0000 7f87e0b4c700 1 -- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f87bc05f2c0 msgr2=0x7f87bc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:41.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.884+0000 7f87e0b4c700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f87bc05f2c0 0x7f87bc061780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f87d0006d80 tx=0x7f87d0008040).stop 2024-01-24T14:10:41.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.884+0000 7f87e0b4c700 1 -- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87dc151a50 msgr2=0x7f87dc13aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:41.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.884+0000 7f87e0b4c700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87dc151a50 0x7f87dc13aca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f87cc001940 tx=0x7f87cc017000).stop 2024-01-24T14:10:41.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.884+0000 7f87e0b4c700 1 -- 172.21.15.29:0/2883944946 shutdown_connections 2024-01-24T14:10:41.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.884+0000 7f87e0b4c700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87dc0ad9c0 0x7f87dc0ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:41.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.884+0000 7f87e0b4c700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f87bc05f2c0 0x7f87bc061780 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:41.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.884+0000 7f87e0b4c700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87c4004690 0x7f87dc1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:41.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.884+0000 7f87e0b4c700 1 --2- 172.21.15.29:0/2883944946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87dc151a50 0x7f87dc13aca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:41.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.884+0000 7f87e0b4c700 1 -- 172.21.15.29:0/2883944946 >> 172.21.15.29:0/2883944946 conn(0x7f87dc0c8ba0 msgr2=0x7f87dc139c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:41.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.885+0000 7f87e0b4c700 1 -- 172.21.15.29:0/2883944946 shutdown_connections 2024-01-24T14:10:41.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:41.885+0000 7f87e0b4c700 1 -- 172.21.15.29:0/2883944946 wait complete. 2024-01-24T14:10:41.901 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T14:10:41.901 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:10:41.947 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:10:41.993 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T14:10:41.994 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-24T14:10:41.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:10:41.994 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-24T14:10:42.028 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:42.028+0000 7f3897302680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:10:42.045 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:42.045+0000 7fdadf88b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:10:42.060 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T14:10:42.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:10:42.061 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-24T14:10:42.095 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:10:42.109 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:42.110+0000 7f3897302680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:10:42.139 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:42.138+0000 7fdadf88b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:10:42.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.284+0000 7fbf1d61b700 1 Processor -- start 2024-01-24T14:10:42.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.284+0000 7fbf1d61b700 1 -- start start 2024-01-24T14:10:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.285+0000 7fbf1d61b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbf1814df80 0x7fbf1814e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.286+0000 7fbf1d61b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180bcfd0 0x7fbf180c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.286+0000 7fbf1d61b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf180c29e0 con 0x7fbf180c2040 2024-01-24T14:10:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.286+0000 7fbf1d61b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf180bd410 con 0x7fbf1814df80 2024-01-24T14:10:42.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.286+0000 7fbf1d61b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf18062840 con 0x7fbf180bcfd0 2024-01-24T14:10:42.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.286+0000 7fbf167fc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbf180c2040 0x7fbf180bc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54720/0 (socket says 172.21.15.29:54720) 2024-01-24T14:10:42.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.286+0000 7fbf167fc700 1 -- 172.21.15.29:0/2186694096 learned_addr learned my addr 172.21.15.29:0/2186694096 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:42.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.286+0000 7fbf177fe700 1 --2- 172.21.15.29:0/2186694096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180bcfd0 0x7fbf180c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:42.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.287+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362158800 0 0) 0x7fbf08001110 con 0x7fbf180c2040 2024-01-24T14:10:42.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.287+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf000033f0 con 0x7fbf180c2040 2024-01-24T14:10:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.287+0000 7fbf16ffd700 1 --2- 172.21.15.29:0/2186694096 >> v2:172.21.15.163:3300/0 conn(0x7fbf1814df80 0x7fbf1814e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.287+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 214942709 0 0) 0x7fbf000033f0 con 0x7fbf180c2040 2024-01-24T14:10:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.287+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180bcfd0 msgr2=0x7fbf180c24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.287+0000 7fbf15ffb700 1 --2- 172.21.15.29:0/2186694096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180bcfd0 0x7fbf180c24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.287+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 >> v2:172.21.15.163:3300/0 conn(0x7fbf1814df80 msgr2=0x7fbf1814e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.287+0000 7fbf15ffb700 1 --2- 172.21.15.29:0/2186694096 >> v2:172.21.15.163:3300/0 conn(0x7fbf1814df80 0x7fbf1814e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.287+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180bd630 con 0x7fbf180c2040 2024-01-24T14:10:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.287+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fbf08001600 con 0x7fbf180c2040 2024-01-24T14:10:42.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.288+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 >> v1:172.21.15.29:6789/0 conn(0x7fbf180c2040 legacy=0x7fbf180bc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:42.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.288+0000 7fbf15ffb700 1 --2- 172.21.15.29:0/2186694096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf00003960 0x7fbf00003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.288+0000 7fbf15ffb700 1 --2- 172.21.15.29:0/2186694096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf00004690 0x7fbf00004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.288+0000 7fbf15ffb700 1 --2- 172.21.15.29:0/2186694096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf00005000 0x7fbf0000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.288+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180bd410 con 0x7fbf00004690 2024-01-24T14:10:42.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.288+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf18062840 con 0x7fbf00005000 2024-01-24T14:10:42.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.288+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0000be00 con 0x7fbf00003960 2024-01-24T14:10:42.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.288+0000 7fbf167fc700 1 --2- 172.21.15.29:0/2186694096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf00004690 0x7fbf00004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:42.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.288+0000 7fbf16ffd700 1 --2- 172.21.15.29:0/2186694096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf00003960 0x7fbf00003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:42.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.289+0000 7fbf167fc700 1 -- 172.21.15.29:0/2186694096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf00003960 msgr2=0x7fbf00003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:42.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.289+0000 7fbf167fc700 1 --2- 172.21.15.29:0/2186694096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf00003960 0x7fbf00003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:42.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.289+0000 7fbf167fc700 1 -- 172.21.15.29:0/2186694096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf00005000 msgr2=0x7fbf0000b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T14:10:42.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.289+0000 7fbf167fc700 1 --2- 172.21.15.29:0/2186694096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf00005000 0x7fbf0000b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:42.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.289+0000 7fbf167fc700 1 -- 172.21.15.29:0/2186694096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180bd630 con 0x7fbf00004690 2024-01-24T14:10:42.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.289+0000 7fbf16ffd700 1 --2- 172.21.15.29:0/2186694096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf00003960 0x7fbf00003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:42.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.289+0000 7fbf167fc700 1 --2- 172.21.15.29:0/2186694096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf00004690 0x7fbf00004b10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fbf0800c490 tx=0x7fbf08007960).ready entity=mon.0 client_cookie=88cefae7763b554b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:42.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.289+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf08007c90 con 0x7fbf00004690 2024-01-24T14:10:42.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.289+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0800cce0 con 0x7fbf00004690 2024-01-24T14:10:42.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.289+0000 7fbf15ffb700 1 -- 172.21.15.29:0/2186694096 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf080072e0 con 0x7fbf00004690 2024-01-24T14:10:42.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.291+0000 7fbf1d61b700 1 -- 172.21.15.29:0/2186694096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf00004690 msgr2=0x7fbf00004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:42.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.291+0000 7fbf1d61b700 1 --2- 172.21.15.29:0/2186694096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf00004690 0x7fbf00004b10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fbf0800c490 tx=0x7fbf08007960).stop 2024-01-24T14:10:42.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.293+0000 7fbf177fe700 1 -- 172.21.15.29:0/2186694096 reap_dead start 2024-01-24T14:10:42.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.293+0000 7fbf1d61b700 1 -- 172.21.15.29:0/2186694096 shutdown_connections 2024-01-24T14:10:42.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.293+0000 7fbf1d61b700 1 -- 172.21.15.29:0/2186694096 >> 172.21.15.29:0/2186694096 conn(0x7fbf18064aa0 msgr2=0x7fbf18064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:42.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.294+0000 7fbf1d61b700 1 -- 172.21.15.29:0/2186694096 shutdown_connections 2024-01-24T14:10:42.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.294+0000 7fbf1d61b700 1 -- 172.21.15.29:0/2186694096 wait complete. 2024-01-24T14:10:42.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.296+0000 7fbf1d61b700 1 Processor -- start 2024-01-24T14:10:42.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.297+0000 7fbf1d61b700 1 -- start start 2024-01-24T14:10:42.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.298+0000 7fbf1d61b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf181649b0 0x7fbf180be310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.298+0000 7fbf1d61b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf180be800 0x7fbf180a8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.298+0000 7fbf1d61b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180ac4f0 0x7fbf180a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.298+0000 7fbf1d61b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180a9570 con 0x7fbf180be800 2024-01-24T14:10:42.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.298+0000 7fbf1d61b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf18165400 con 0x7fbf181649b0 2024-01-24T14:10:42.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.298+0000 7fbf1d61b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180bec80 con 0x7fbf180ac4f0 2024-01-24T14:10:42.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.299+0000 7fbf16ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf181649b0 0x7fbf180be310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:42.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.299+0000 7fbf16ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf181649b0 0x7fbf180be310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:53064/0 (socket says 172.21.15.29:53064) 2024-01-24T14:10:42.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.299+0000 7fbf16ffd700 1 -- 172.21.15.29:0/1322800724 learned_addr learned my addr 172.21.15.29:0/1322800724 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:42.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.299+0000 7fbf167fc700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf180be800 0x7fbf180a8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:42.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.299+0000 7fbf16ffd700 1 -- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180ac4f0 msgr2=0x7fbf180a9080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:10:42.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.299+0000 7fbf16ffd700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180ac4f0 0x7fbf180a9080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:42.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.299+0000 7fbf16ffd700 1 -- 172.21.15.29:0/1322800724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf180be800 msgr2=0x7fbf180a8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:42.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.299+0000 7fbf16ffd700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf180be800 0x7fbf180a8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:42.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.299+0000 7fbf16ffd700 1 -- 172.21.15.29:0/1322800724 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180bef60 con 0x7fbf181649b0 2024-01-24T14:10:42.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.299+0000 7fbf16ffd700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf181649b0 0x7fbf180be310 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fbf10010570 tx=0x7fbf1000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:42.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.300+0000 7fbefeffd700 1 -- 172.21.15.29:0/1322800724 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1000e030 con 0x7fbf181649b0 2024-01-24T14:10:42.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.300+0000 7fbefeffd700 1 -- 172.21.15.29:0/1322800724 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1000f040 con 0x7fbf181649b0 2024-01-24T14:10:42.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.300+0000 7fbefeffd700 1 -- 172.21.15.29:0/1322800724 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10015690 con 0x7fbf181649b0 2024-01-24T14:10:42.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.300+0000 7fbf1d61b700 1 -- 172.21.15.29:0/1322800724 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf180bf240 con 0x7fbf181649b0 2024-01-24T14:10:42.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.300+0000 7fbf1d61b700 1 -- 172.21.15.29:0/1322800724 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf18164d90 con 0x7fbf181649b0 2024-01-24T14:10:42.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.303+0000 7fbf1d61b700 1 -- 172.21.15.29:0/1322800724 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf180bf760 con 0x7fbf181649b0 2024-01-24T14:10:42.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.303+0000 7fbefeffd700 1 -- 172.21.15.29:0/1322800724 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fbf10003020 con 0x7fbf181649b0 2024-01-24T14:10:42.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.304+0000 7fbefeffd700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0405f2c0 0x7fbf04061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.304+0000 7fbefeffd700 1 -- 172.21.15.29:0/1322800724 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7605+0+0 (secure 0 0 0) 0x7fbf1007b300 con 0x7fbf181649b0 2024-01-24T14:10:42.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.304+0000 7fbf167fc700 1 -- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0405f2c0 msgr2=0x7fbf04061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:42.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.304+0000 7fbf167fc700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0405f2c0 0x7fbf04061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:10:42.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.309+0000 7fbefeffd700 1 -- 172.21.15.29:0/1322800724 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbf180bf760 con 0x7fbf181649b0 2024-01-24T14:10:42.411 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:42.412+0000 7f3897302680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:10:42.490 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:42.489+0000 7fdadf88b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:10:42.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.505+0000 7fbf167fc700 1 -- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0405f2c0 msgr2=0x7fbf04061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:42.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.505+0000 7fbf167fc700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0405f2c0 0x7fbf04061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:10:42.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.553+0000 7fbf1d61b700 1 -- 172.21.15.29:0/1322800724 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf180bf760 con 0x7fbf181649b0 2024-01-24T14:10:42.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.553+0000 7fbefeffd700 1 -- 172.21.15.29:0/1322800724 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4492 (secure 0 0 0) 0x7fbf180bf760 con 0x7fbf181649b0 2024-01-24T14:10:42.554 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:42.554 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1291112242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1291112242},{"type":"v1","addr":"172.21.15.163:6837","nonce":1291112242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4793,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/241296864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":241296864},{"type":"v1","addr":"172.21.15.163:6836","nonce":241296864}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5127,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2565530631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2565530631},{"type":"v1","addr":"172.21.15.29:6833","nonce":2565530631}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:59:26.370202+0000","modified":"2024-01-24T13:59:29.380658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1245725396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1245725396},{"type":"v1","addr":"172.21.15.29:6835","nonce":1245725396}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-24T14:10:42.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.555+0000 7fbf1d61b700 1 -- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0405f2c0 msgr2=0x7fbf04061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:10:42.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.555+0000 7fbf1d61b700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0405f2c0 0x7fbf04061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:42.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.556+0000 7fbf1d61b700 1 -- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf181649b0 msgr2=0x7fbf180be310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:42.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.556+0000 7fbf1d61b700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf181649b0 0x7fbf180be310 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fbf10010570 tx=0x7fbf1000b040).stop 2024-01-24T14:10:42.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.556+0000 7fbf1d61b700 1 -- 172.21.15.29:0/1322800724 shutdown_connections 2024-01-24T14:10:42.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.556+0000 7fbf1d61b700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180ac4f0 0x7fbf180a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:42.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.556+0000 7fbf1d61b700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf181649b0 0x7fbf180be310 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:42.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.556+0000 7fbf1d61b700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf0405f2c0 0x7fbf04061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:42.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.556+0000 7fbf1d61b700 1 --2- 172.21.15.29:0/1322800724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf180be800 0x7fbf180a8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:42.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.556+0000 7fbf1d61b700 1 -- 172.21.15.29:0/1322800724 >> 172.21.15.29:0/1322800724 conn(0x7fbf18064aa0 msgr2=0x7fbf18150510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:42.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.556+0000 7fbf1d61b700 1 -- 172.21.15.29:0/1322800724 shutdown_connections 2024-01-24T14:10:42.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.556+0000 7fbf1d61b700 1 -- 172.21.15.29:0/1322800724 wait complete. 2024-01-24T14:10:42.557 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 18 2024-01-24T14:10:42.575 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:10:42.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.706+0000 7f814d448700 1 Processor -- start 2024-01-24T14:10:42.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.706+0000 7f814d448700 1 -- start start 2024-01-24T14:10:42.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.707+0000 7f814d448700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81480c5410 0x7f81480aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.707+0000 7f814d448700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81480aa6c0 con 0x7f8148154750 2024-01-24T14:10:42.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.707+0000 7f814d448700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8148064240 con 0x7f8148154370 2024-01-24T14:10:42.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.707+0000 7f814d448700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81480cb5c0 con 0x7f81480c5410 2024-01-24T14:10:42.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.707+0000 7f81477fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8148154750 0x7f81480631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54732/0 (socket says 172.21.15.29:54732) 2024-01-24T14:10:42.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.707+0000 7f81477fe700 1 -- 172.21.15.29:0/4089217030 learned_addr learned my addr 172.21.15.29:0/4089217030 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:42.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.707+0000 7f81467fc700 1 --2- 172.21.15.29:0/4089217030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81480c5410 0x7f81480aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:42.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.708+0000 7f8145ffb700 1 -- 172.21.15.29:0/4089217030 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698804997 0 0) 0x7f81480aa6c0 con 0x7f8148154750 2024-01-24T14:10:42.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.708+0000 7f8145ffb700 1 -- 172.21.15.29:0/4089217030 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81300033f0 con 0x7f8148154750 2024-01-24T14:10:42.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.708+0000 7f81467fc700 1 -- 172.21.15.29:0/4089217030 >> v1:172.21.15.163:6789/0 conn(0x7f8148154370 legacy=0x7f81480ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:42.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.708+0000 7f8145ffb700 1 -- 172.21.15.29:0/4089217030 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146076962 0 0) 0x7f8148064240 con 0x7f8148154370 2024-01-24T14:10:42.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.708+0000 7f81467fc700 1 -- 172.21.15.29:0/4089217030 >> v1:172.21.15.29:6789/0 conn(0x7f8148154750 legacy=0x7f81480631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:42.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.708+0000 7f81467fc700 1 -- 172.21.15.29:0/4089217030 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81480cb8a0 con 0x7f81480c5410 2024-01-24T14:10:42.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.708+0000 7f81467fc700 1 --2- 172.21.15.29:0/4089217030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81480c5410 0x7f81480aa0e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f813c009790 tx=0x7f813c009060).ready entity=mon.2 client_cookie=1c4ee53c8e1c97ad server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:42.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.708+0000 7f8145ffb700 1 -- 172.21.15.29:0/4089217030 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c00f510 con 0x7f81480c5410 2024-01-24T14:10:42.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.708+0000 7f8145ffb700 1 -- 172.21.15.29:0/4089217030 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c00f670 con 0x7f81480c5410 2024-01-24T14:10:42.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.708+0000 7f8145ffb700 1 -- 172.21.15.29:0/4089217030 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c00f9f0 con 0x7f81480c5410 2024-01-24T14:10:42.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.709+0000 7f814d448700 1 -- 172.21.15.29:0/4089217030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81480c5410 msgr2=0x7f81480aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:42.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.709+0000 7f814d448700 1 --2- 172.21.15.29:0/4089217030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81480c5410 0x7f81480aa0e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f813c009790 tx=0x7f813c009060).stop 2024-01-24T14:10:42.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.709+0000 7f814d448700 1 -- 172.21.15.29:0/4089217030 shutdown_connections 2024-01-24T14:10:42.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.709+0000 7f814d448700 1 --2- 172.21.15.29:0/4089217030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81480c5410 0x7f81480aa0e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:42.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.709+0000 7f814d448700 1 -- 172.21.15.29:0/4089217030 >> 172.21.15.29:0/4089217030 conn(0x7f814805ec80 msgr2=0x7f81480cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:42.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.709+0000 7f814d448700 1 -- 172.21.15.29:0/4089217030 shutdown_connections 2024-01-24T14:10:42.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.709+0000 7f814d448700 1 -- 172.21.15.29:0/4089217030 wait complete. 2024-01-24T14:10:42.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.711+0000 7f814d448700 1 Processor -- start 2024-01-24T14:10:42.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.711+0000 7f814d448700 1 -- start start 2024-01-24T14:10:42.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f814d448700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81480c5410 0x7f8148165a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f814d448700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8148154370 0x7f8148155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f814d448700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8148154750 0x7f81481569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f814d448700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8148165f80 con 0x7f81480c5410 2024-01-24T14:10:42.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f814d448700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81480ab8a0 con 0x7f8148154370 2024-01-24T14:10:42.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f814d448700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8148155b30 con 0x7f8148154750 2024-01-24T14:10:42.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f8146ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81480c5410 0x7f8148165a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:42.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f81467fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8148154370 0x7f8148155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:42.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f81477fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8148154750 0x7f81481569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f8146ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81480c5410 0x7f8148165a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52774/0 (socket says 172.21.15.29:52774) 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f81467fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8148154370 0x7f8148155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:53080/0 (socket says 172.21.15.29:53080) 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f81477fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8148154750 0x7f81481569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:52024/0 (socket says 172.21.15.29:52024) 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.712+0000 7f8146ffd700 1 -- 172.21.15.29:0/979565143 learned_addr learned my addr 172.21.15.29:0/979565143 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.713+0000 7f81467fc700 1 -- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8148154750 msgr2=0x7f81481569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.713+0000 7f81467fc700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8148154750 0x7f81481569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.713+0000 7f81467fc700 1 -- 172.21.15.29:0/979565143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81480c5410 msgr2=0x7f8148165a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.713+0000 7f81467fc700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81480c5410 0x7f8148165a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.713+0000 7f81467fc700 1 -- 172.21.15.29:0/979565143 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c0092a0 con 0x7f8148154370 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.713+0000 7f8146ffd700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81480c5410 0x7f8148165a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.713+0000 7f81467fc700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8148154370 0x7f8148155640 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f813c014000 tx=0x7f813c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.713+0000 7f812effd700 1 -- 172.21.15.29:0/979565143 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c01a4d0 con 0x7f8148154370 2024-01-24T14:10:42.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.713+0000 7f812effd700 1 -- 172.21.15.29:0/979565143 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c01e070 con 0x7f8148154370 2024-01-24T14:10:42.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.713+0000 7f814d448700 1 -- 172.21.15.29:0/979565143 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8148156ee0 con 0x7f8148154370 2024-01-24T14:10:42.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.713+0000 7f812effd700 1 -- 172.21.15.29:0/979565143 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c01a8a0 con 0x7f8148154370 2024-01-24T14:10:42.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.713+0000 7f814d448700 1 -- 172.21.15.29:0/979565143 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8148156090 con 0x7f8148154370 2024-01-24T14:10:42.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.714+0000 7f814d448700 1 -- 172.21.15.29:0/979565143 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81480475e0 con 0x7f8148154370 2024-01-24T14:10:42.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.715+0000 7f812effd700 1 -- 172.21.15.29:0/979565143 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f813c020030 con 0x7f8148154370 2024-01-24T14:10:42.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.715+0000 7f812effd700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f813405f270 0x7f8134061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:42.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.715+0000 7f812effd700 1 -- 172.21.15.29:0/979565143 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7605+0+0 (secure 0 0 0) 0x7f813c07b8a0 con 0x7f8148154370 2024-01-24T14:10:42.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.715+0000 7f8146ffd700 1 -- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f813405f270 msgr2=0x7f8134061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:42.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.715+0000 7f8146ffd700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f813405f270 0x7f8134061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:10:42.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.719+0000 7f812effd700 1 -- 172.21.15.29:0/979565143 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f813c04c420 con 0x7f8148154370 2024-01-24T14:10:42.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.916+0000 7f8146ffd700 1 -- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f813405f270 msgr2=0x7f8134061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:42.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.916+0000 7f8146ffd700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f813405f270 0x7f8134061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:10:42.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.917+0000 7f814d448700 1 -- 172.21.15.29:0/979565143 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81480cbc10 con 0x7f8148154370 2024-01-24T14:10:42.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.918+0000 7f812effd700 1 -- 172.21.15.29:0/979565143 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+18892 (secure 0 0 0) 0x7f813c04fa40 con 0x7f8148154370 2024-01-24T14:10:42.919 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:42.919 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":57,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:10:41.787946+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:59:24.684931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-24T13:59:25.345273+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]},{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6832/43241":"2024-01-25T13:59:43.884293+0000","172.21.15.163:0/46643247":"2024-01-25T13:59:43.884293+0000","172.21.15.163:6833/43241":"2024-01-25T13:59:43.884293+0000","172.21.15.163:0/233457199":"2024-01-25T13:59:43.884293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":111,"length":1},{"begin":152,"length":69}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":9,"length":1},{"begin":19,"length":1},{"begin":29,"length":1},{"begin":39,"length":1},{"begin":49,"length":1},{"begin":59,"length":1},{"begin":69,"length":1},{"begin":79,"length":1},{"begin":90,"length":1},{"begin":100,"length":1},{"begin":106,"length":1},{"begin":210,"length":11}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:10:42.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.920+0000 7f814d448700 1 -- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f813405f270 msgr2=0x7f8134061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:10:42.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.920+0000 7f814d448700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f813405f270 0x7f8134061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:42.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.920+0000 7f814d448700 1 -- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8148154370 msgr2=0x7f8148155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:42.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.920+0000 7f814d448700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8148154370 0x7f8148155640 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f813c014000 tx=0x7f813c017000).stop 2024-01-24T14:10:42.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.920+0000 7f814d448700 1 -- 172.21.15.29:0/979565143 shutdown_connections 2024-01-24T14:10:42.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.920+0000 7f814d448700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8148154750 0x7f81481569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:42.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.920+0000 7f814d448700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f813405f270 0x7f8134061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:42.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.920+0000 7f814d448700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81480c5410 0x7f8148165a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:42.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.920+0000 7f814d448700 1 --2- 172.21.15.29:0/979565143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8148154370 0x7f8148155640 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:42.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.920+0000 7f814d448700 1 -- 172.21.15.29:0/979565143 >> 172.21.15.29:0/979565143 conn(0x7f814805ec80 msgr2=0x7f8148153310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:42.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.920+0000 7f814d448700 1 -- 172.21.15.29:0/979565143 shutdown_connections 2024-01-24T14:10:42.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:42.920+0000 7f814d448700 1 -- 172.21.15.29:0/979565143 wait complete. 2024-01-24T14:10:42.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T13:59:24.684931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T13:59:25.345273+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 57, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T14:10:42.935 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T14:10:43.027 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:43.028+0000 7f3897302680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:10:43.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.070+0000 7fd61b0bd700 1 Processor -- start 2024-01-24T14:10:43.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.070+0000 7fd61b0bd700 1 -- start start 2024-01-24T14:10:43.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.071+0000 7fd61b0bd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd614062ec0 0x7fd61406a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.071+0000 7fd61b0bd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6140c8270 0x7fd6141391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.071+0000 7fd61b0bd700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6140bcda0 con 0x7fd6141384f0 2024-01-24T14:10:43.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.071+0000 7fd61b0bd700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd614065f80 con 0x7fd614062ec0 2024-01-24T14:10:43.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.071+0000 7fd61b0bd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd614066260 con 0x7fd6140c8270 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.071+0000 7fd613fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd614062ec0 0x7fd61406a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.071+0000 7fd618e59700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd6141384f0 0x7fd6140627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54744/0 (socket says 172.21.15.29:54744) 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.071+0000 7fd613fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd614062ec0 0x7fd61406a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52780/0 (socket says 172.21.15.29:52780) 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.071+0000 7fd618e59700 1 -- 172.21.15.29:0/3564364337 learned_addr learned my addr 172.21.15.29:0/3564364337 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.072+0000 7fd61965a700 1 --2- 172.21.15.29:0/3564364337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6140c8270 0x7fd6141391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.072+0000 7fd613fff700 1 -- 172.21.15.29:0/3564364337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6140c8270 msgr2=0x7fd6141391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.072+0000 7fd613fff700 1 --2- 172.21.15.29:0/3564364337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6140c8270 0x7fd6141391d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.072+0000 7fd613fff700 1 -- 172.21.15.29:0/3564364337 >> v1:172.21.15.29:6789/0 conn(0x7fd6141384f0 legacy=0x7fd6140627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.072+0000 7fd613fff700 1 -- 172.21.15.29:0/3564364337 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6140bca50 con 0x7fd614062ec0 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.072+0000 7fd6137fe700 1 -- 172.21.15.29:0/3564364337 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041636793 0 0) 0x7fd6140bcda0 con 0x7fd6141384f0 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.072+0000 7fd613fff700 1 --2- 172.21.15.29:0/3564364337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd614062ec0 0x7fd61406a460 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fd600009760 tx=0x7fd600009060).ready entity=mon.0 client_cookie=1eba94685be0c0e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.072+0000 7fd6137fe700 1 -- 172.21.15.29:0/3564364337 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000090d0 con 0x7fd614062ec0 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.072+0000 7fd6137fe700 1 -- 172.21.15.29:0/3564364337 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60000f510 con 0x7fd614062ec0 2024-01-24T14:10:43.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.072+0000 7fd6137fe700 1 -- 172.21.15.29:0/3564364337 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60000f890 con 0x7fd614062ec0 2024-01-24T14:10:43.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.073+0000 7fd61b0bd700 1 -- 172.21.15.29:0/3564364337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd614062ec0 msgr2=0x7fd61406a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.073+0000 7fd61b0bd700 1 --2- 172.21.15.29:0/3564364337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd614062ec0 0x7fd61406a460 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fd600009760 tx=0x7fd600009060).stop 2024-01-24T14:10:43.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.073+0000 7fd61b0bd700 1 -- 172.21.15.29:0/3564364337 shutdown_connections 2024-01-24T14:10:43.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.073+0000 7fd61b0bd700 1 --2- 172.21.15.29:0/3564364337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6140c8270 0x7fd6141391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.073+0000 7fd61b0bd700 1 --2- 172.21.15.29:0/3564364337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd614062ec0 0x7fd61406a460 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.073+0000 7fd61b0bd700 1 -- 172.21.15.29:0/3564364337 >> 172.21.15.29:0/3564364337 conn(0x7fd614064390 msgr2=0x7fd6140c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:43.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.073+0000 7fd61b0bd700 1 -- 172.21.15.29:0/3564364337 shutdown_connections 2024-01-24T14:10:43.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.073+0000 7fd61b0bd700 1 -- 172.21.15.29:0/3564364337 wait complete. 2024-01-24T14:10:43.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.075+0000 7fd61b0bd700 1 Processor -- start 2024-01-24T14:10:43.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.075+0000 7fd61b0bd700 1 -- start start 2024-01-24T14:10:43.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd61b0bd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd614062ec0 0x7fd6140ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd61b0bd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6140c8270 0x7fd6140bf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd61b0bd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6141384f0 0x7fd6140bf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd61b0bd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140bfdb0 con 0x7fd614062ec0 2024-01-24T14:10:43.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd61b0bd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140c28e0 con 0x7fd6141384f0 2024-01-24T14:10:43.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd61b0bd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140ad7e0 con 0x7fd6140c8270 2024-01-24T14:10:43.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd618e59700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd614062ec0 0x7fd6140ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd618e59700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd614062ec0 0x7fd6140ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52796/0 (socket says 172.21.15.29:52796) 2024-01-24T14:10:43.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd618e59700 1 -- 172.21.15.29:0/2078146670 learned_addr learned my addr 172.21.15.29:0/2078146670 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:43.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd613fff700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6140c8270 0x7fd6140bf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd61965a700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6141384f0 0x7fd6140bf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd613fff700 1 -- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6141384f0 msgr2=0x7fd6140bf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd613fff700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6141384f0 0x7fd6140bf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd613fff700 1 -- 172.21.15.29:0/2078146670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd614062ec0 msgr2=0x7fd6140ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd613fff700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd614062ec0 0x7fd6140ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd613fff700 1 -- 172.21.15.29:0/2078146670 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6000092a0 con 0x7fd6140c8270 2024-01-24T14:10:43.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.076+0000 7fd618e59700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd614062ec0 0x7fd6140ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:43.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.077+0000 7fd613fff700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6140c8270 0x7fd6140bf3d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fd600014000 tx=0x7fd600017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:43.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.077+0000 7fd610ff9700 1 -- 172.21.15.29:0/2078146670 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60000f890 con 0x7fd6140c8270 2024-01-24T14:10:43.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.077+0000 7fd610ff9700 1 -- 172.21.15.29:0/2078146670 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd600000d10 con 0x7fd6140c8270 2024-01-24T14:10:43.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.077+0000 7fd61b0bd700 1 -- 172.21.15.29:0/2078146670 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61413b920 con 0x7fd6140c8270 2024-01-24T14:10:43.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.077+0000 7fd610ff9700 1 -- 172.21.15.29:0/2078146670 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60000fea0 con 0x7fd6140c8270 2024-01-24T14:10:43.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.077+0000 7fd61b0bd700 1 -- 172.21.15.29:0/2078146670 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6140c2570 con 0x7fd6140c8270 2024-01-24T14:10:43.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.078+0000 7fd61b0bd700 1 -- 172.21.15.29:0/2078146670 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6140c20f0 con 0x7fd6140c8270 2024-01-24T14:10:43.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.078+0000 7fd610ff9700 1 -- 172.21.15.29:0/2078146670 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fd60001b620 con 0x7fd6140c8270 2024-01-24T14:10:43.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.079+0000 7fd610ff9700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5f805f270 0x7fd5f8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.079+0000 7fd618e59700 1 -- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5f805f270 msgr2=0x7fd5f8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:43.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.079+0000 7fd610ff9700 1 -- 172.21.15.29:0/2078146670 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7605+0+0 (secure 0 0 0) 0x7fd60007c2d0 con 0x7fd6140c8270 2024-01-24T14:10:43.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.079+0000 7fd618e59700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5f805f270 0x7fd5f8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:10:43.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.084+0000 7fd610ff9700 1 -- 172.21.15.29:0/2078146670 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd6140c20f0 con 0x7fd6140c8270 2024-01-24T14:10:43.100 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:43.100+0000 7f3897302680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:10:43.240 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:43.239+0000 7fdadf88b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:10:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.280+0000 7fd618e59700 1 -- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5f805f270 msgr2=0x7fd5f8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:43.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.280+0000 7fd618e59700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5f805f270 0x7fd5f8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:10:43.284 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:43.285+0000 7f3897302680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:10:43.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.312+0000 7fd61b0bd700 1 -- 172.21.15.29:0/2078146670 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd6140c20f0 con 0x7fd6140c8270 2024-01-24T14:10:43.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.312+0000 7fd610ff9700 1 -- 172.21.15.29:0/2078146670 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v57) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd6140c20f0 con 0x7fd6140c8270 2024-01-24T14:10:43.313 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T14:10:43.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.314+0000 7fd61b0bd700 1 -- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5f805f270 msgr2=0x7fd5f8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:10:43.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.314+0000 7fd61b0bd700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5f805f270 0x7fd5f8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:43.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.314+0000 7fd61b0bd700 1 -- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6140c8270 msgr2=0x7fd6140bf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.314+0000 7fd61b0bd700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6140c8270 0x7fd6140bf3d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fd600014000 tx=0x7fd600017000).stop 2024-01-24T14:10:43.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.314+0000 7fd61b0bd700 1 -- 172.21.15.29:0/2078146670 shutdown_connections 2024-01-24T14:10:43.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.314+0000 7fd61b0bd700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6141384f0 0x7fd6140bf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.314+0000 7fd61b0bd700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5f805f270 0x7fd5f8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:43.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.314+0000 7fd61b0bd700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd614062ec0 0x7fd6140ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.314+0000 7fd61b0bd700 1 --2- 172.21.15.29:0/2078146670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6140c8270 0x7fd6140bf3d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.314+0000 7fd61b0bd700 1 -- 172.21.15.29:0/2078146670 >> 172.21.15.29:0/2078146670 conn(0x7fd614064390 msgr2=0x7fd6140c7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:43.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.314+0000 7fd61b0bd700 1 -- 172.21.15.29:0/2078146670 shutdown_connections 2024-01-24T14:10:43.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.314+0000 7fd61b0bd700 1 -- 172.21.15.29:0/2078146670 wait complete. 2024-01-24T14:10:43.316 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:43.315+0000 7fdadf88b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:10:43.328 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T14:10:43.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.464+0000 7fad7ce31700 1 Processor -- start 2024-01-24T14:10:43.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.464+0000 7fad7ce31700 1 -- start start 2024-01-24T14:10:43.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.465+0000 7fad7ce31700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fad7813a5e0 0x7fad780639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.465+0000 7fad7ce31700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad78063eb0 0x7fad780664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.465+0000 7fad7ce31700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad78066e20 con 0x7fad780669b0 2024-01-24T14:10:43.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.465+0000 7fad7ce31700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad780679f0 con 0x7fad7813a5e0 2024-01-24T14:10:43.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.465+0000 7fad7ce31700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad78067cd0 con 0x7fad78063eb0 2024-01-24T14:10:43.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.466+0000 7fad7659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fad7813a5e0 0x7fad780639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.466+0000 7fad7659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fad7813a5e0 0x7fad780639c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55836/0 (socket says 172.21.15.29:55836) 2024-01-24T14:10:43.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.466+0000 7fad7659c700 1 -- 172.21.15.29:0/2131947413 learned_addr learned my addr 172.21.15.29:0/2131947413 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:43.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.466+0000 7fad75d9b700 1 --2- 172.21.15.29:0/2131947413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad78063eb0 0x7fad780664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.466+0000 7fad7659c700 1 -- 172.21.15.29:0/2131947413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad78063eb0 msgr2=0x7fad780664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.466+0000 7fad7659c700 1 --2- 172.21.15.29:0/2131947413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad78063eb0 0x7fad780664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.466+0000 7fad7659c700 1 -- 172.21.15.29:0/2131947413 >> v1:172.21.15.163:6789/0 conn(0x7fad780669b0 legacy=0x7fad78157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.466+0000 7fad7659c700 1 -- 172.21.15.29:0/2131947413 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad78067fb0 con 0x7fad7813a5e0 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.466+0000 7fad7559a700 1 -- 172.21.15.29:0/2131947413 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828849667 0 0) 0x7fad78066e20 con 0x7fad780669b0 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.466+0000 7fad7659c700 1 --2- 172.21.15.29:0/2131947413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad7813a5e0 0x7fad780639c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fad68006fd0 tx=0x7fad680090f0).ready entity=mon.0 client_cookie=90429b1b48a247f8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.466+0000 7fad7559a700 1 -- 172.21.15.29:0/2131947413 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6800b580 con 0x7fad7813a5e0 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.467+0000 7fad7559a700 1 -- 172.21.15.29:0/2131947413 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6800b6e0 con 0x7fad7813a5e0 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.467+0000 7fad7559a700 1 -- 172.21.15.29:0/2131947413 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6800ba60 con 0x7fad7813a5e0 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.467+0000 7fad7ce31700 1 -- 172.21.15.29:0/2131947413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad7813a5e0 msgr2=0x7fad780639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.467+0000 7fad7ce31700 1 --2- 172.21.15.29:0/2131947413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad7813a5e0 0x7fad780639c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fad68006fd0 tx=0x7fad680090f0).stop 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.467+0000 7fad7ce31700 1 -- 172.21.15.29:0/2131947413 shutdown_connections 2024-01-24T14:10:43.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.467+0000 7fad7ce31700 1 --2- 172.21.15.29:0/2131947413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad7813a5e0 0x7fad780639c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.467+0000 7fad7ce31700 1 --2- 172.21.15.29:0/2131947413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad78063eb0 0x7fad780664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.467+0000 7fad7ce31700 1 -- 172.21.15.29:0/2131947413 >> 172.21.15.29:0/2131947413 conn(0x7fad780c8bd0 msgr2=0x7fad780c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:43.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.467+0000 7fad7ce31700 1 -- 172.21.15.29:0/2131947413 shutdown_connections 2024-01-24T14:10:43.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.467+0000 7fad7ce31700 1 -- 172.21.15.29:0/2131947413 wait complete. 2024-01-24T14:10:43.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.469+0000 7fad7ce31700 1 Processor -- start 2024-01-24T14:10:43.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.469+0000 7fad7ce31700 1 -- start start 2024-01-24T14:10:43.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.470+0000 7fad7ce31700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad78063eb0 0x7fad780ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.470+0000 7fad7ce31700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad780669b0 0x7fad7815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.470+0000 7fad7ce31700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad7813a5e0 0x7fad7815a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.470+0000 7fad7ce31700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780aa520 con 0x7fad7813a5e0 2024-01-24T14:10:43.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.470+0000 7fad7ce31700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7815abf0 con 0x7fad780669b0 2024-01-24T14:10:43.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.470+0000 7fad7ce31700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7813cfe0 con 0x7fad78063eb0 2024-01-24T14:10:43.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.470+0000 7fad76d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad7813a5e0 0x7fad7815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.470+0000 7fad7659c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad78063eb0 0x7fad780ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.470+0000 7fad75d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad780669b0 0x7fad7815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.470+0000 7fad76d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad7813a5e0 0x7fad7815a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55852/0 (socket says 172.21.15.29:55852) 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.470+0000 7fad76d9d700 1 -- 172.21.15.29:0/1160763079 learned_addr learned my addr 172.21.15.29:0/1160763079 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad7659c700 1 -- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad780669b0 msgr2=0x7fad7815a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad7659c700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad780669b0 0x7fad7815a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad7659c700 1 -- 172.21.15.29:0/1160763079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad7813a5e0 msgr2=0x7fad7815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad7659c700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad7813a5e0 0x7fad7815a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad7659c700 1 -- 172.21.15.29:0/1160763079 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad68009370 con 0x7fad78063eb0 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad75d9b700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad780669b0 0x7fad7815a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad76d9d700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad7813a5e0 0x7fad7815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad7659c700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad78063eb0 0x7fad780ae200 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fad78158d00 tx=0x7fad68017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:43.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad667fc700 1 -- 172.21.15.29:0/1160763079 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68006d90 con 0x7fad78063eb0 2024-01-24T14:10:43.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad667fc700 1 -- 172.21.15.29:0/1160763079 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad68000bc0 con 0x7fad78063eb0 2024-01-24T14:10:43.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad667fc700 1 -- 172.21.15.29:0/1160763079 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6801c420 con 0x7fad78063eb0 2024-01-24T14:10:43.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad7ce31700 1 -- 172.21.15.29:0/1160763079 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7813c3d0 con 0x7fad78063eb0 2024-01-24T14:10:43.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.471+0000 7fad7ce31700 1 -- 172.21.15.29:0/1160763079 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7813c990 con 0x7fad78063eb0 2024-01-24T14:10:43.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.473+0000 7fad667fc700 1 -- 172.21.15.29:0/1160763079 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fad6801ca20 con 0x7fad78063eb0 2024-01-24T14:10:43.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.473+0000 7fad7ce31700 1 -- 172.21.15.29:0/1160763079 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad780c52f0 con 0x7fad78063eb0 2024-01-24T14:10:43.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.473+0000 7fad667fc700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad5805f270 0x7fad58061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.473+0000 7fad75d9b700 1 -- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad5805f270 msgr2=0x7fad58061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:43.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.473+0000 7fad667fc700 1 -- 172.21.15.29:0/1160763079 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7605+0+0 (secure 0 0 0) 0x7fad6807c930 con 0x7fad78063eb0 2024-01-24T14:10:43.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.473+0000 7fad75d9b700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad5805f270 0x7fad58061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:10:43.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.478+0000 7fad667fc700 1 -- 172.21.15.29:0/1160763079 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fad6804d3f0 con 0x7fad78063eb0 2024-01-24T14:10:43.538 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:43.537+0000 7fdadf88b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:10:43.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.657+0000 7fad7ce31700 1 -- 172.21.15.29:0/1160763079 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fad780ac140 con 0x7fad78063eb0 2024-01-24T14:10:43.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.658+0000 7fad667fc700 1 -- 172.21.15.29:0/1160763079 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v57) v1 ==== 92+0+10 (secure 0 0 0) 0x7fad6802a070 con 0x7fad78063eb0 2024-01-24T14:10:43.658 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T14:10:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.660+0000 7fad7ce31700 1 -- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad5805f270 msgr2=0x7fad58061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:10:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.660+0000 7fad7ce31700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad5805f270 0x7fad58061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.660+0000 7fad7ce31700 1 -- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad78063eb0 msgr2=0x7fad780ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.660+0000 7fad7ce31700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad78063eb0 0x7fad780ae200 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fad78158d00 tx=0x7fad68017000).stop 2024-01-24T14:10:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.660+0000 7fad7ce31700 1 -- 172.21.15.29:0/1160763079 shutdown_connections 2024-01-24T14:10:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.660+0000 7fad7ce31700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad5805f270 0x7fad58061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.660+0000 7fad7ce31700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad7813a5e0 0x7fad7815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.660+0000 7fad7ce31700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad78063eb0 0x7fad780ae200 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0x7fad78158d00 tx=0x7fad68017000).stop 2024-01-24T14:10:43.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.660+0000 7fad7ce31700 1 --2- 172.21.15.29:0/1160763079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad780669b0 0x7fad7815a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.660+0000 7fad7ce31700 1 -- 172.21.15.29:0/1160763079 >> 172.21.15.29:0/1160763079 conn(0x7fad780c8bd0 msgr2=0x7fad780be900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:43.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.660+0000 7fad7ce31700 1 -- 172.21.15.29:0/1160763079 shutdown_connections 2024-01-24T14:10:43.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.660+0000 7fad7ce31700 1 -- 172.21.15.29:0/1160763079 wait complete. 2024-01-24T14:10:43.675 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T14:10:43.704 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:43.704+0000 7f3897302680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:10:43.807 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:43.807+0000 7f3897302680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:10:43.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.811+0000 7fc03aa76700 1 Processor -- start 2024-01-24T14:10:43.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.811+0000 7fc03aa76700 1 -- start start 2024-01-24T14:10:43.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc03aa76700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc034064760 0x7fc034064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc03aa76700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0340bfd40 0x7fc0340c01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc03aa76700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc034152240 con 0x7fc0340c8490 2024-01-24T14:10:43.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc03aa76700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0341586c0 con 0x7fc0340bfd40 2024-01-24T14:10:43.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc03aa76700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340aa670 con 0x7fc034064760 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc033fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc0340c8490 0x7fc0340a9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48174/0 (socket says 172.21.15.29:48174) 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc033fff700 1 -- 172.21.15.29:0/134805107 learned_addr learned my addr 172.21.15.29:0/134805107 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc038812700 1 --2- 172.21.15.29:0/134805107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc034064760 0x7fc034064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc039013700 1 --2- 172.21.15.29:0/134805107 >> v2:172.21.15.163:3300/0 conn(0x7fc0340bfd40 0x7fc0340c01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc0337fe700 1 -- 172.21.15.29:0/134805107 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664829065 0 0) 0x7fc034152240 con 0x7fc0340c8490 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc0337fe700 1 -- 172.21.15.29:0/134805107 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0240033f0 con 0x7fc0340c8490 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc038812700 1 -- 172.21.15.29:0/134805107 >> v2:172.21.15.163:3300/0 conn(0x7fc0340bfd40 msgr2=0x7fc0340c01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc038812700 1 --2- 172.21.15.29:0/134805107 >> v2:172.21.15.163:3300/0 conn(0x7fc0340bfd40 0x7fc0340c01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc038812700 1 -- 172.21.15.29:0/134805107 >> v1:172.21.15.29:6789/0 conn(0x7fc0340c8490 legacy=0x7fc0340a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc038812700 1 -- 172.21.15.29:0/134805107 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc034157d90 con 0x7fc034064760 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.813+0000 7fc039013700 1 --2- 172.21.15.29:0/134805107 >> v2:172.21.15.163:3300/0 conn(0x7fc0340bfd40 0x7fc0340c01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.814+0000 7fc038812700 1 --2- 172.21.15.29:0/134805107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc034064760 0x7fc034064b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fc028009760 tx=0x7fc028009060).ready entity=mon.2 client_cookie=49343047ddcc1e80 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.814+0000 7fc0337fe700 1 -- 172.21.15.29:0/134805107 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02800f510 con 0x7fc034064760 2024-01-24T14:10:43.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.814+0000 7fc0337fe700 1 -- 172.21.15.29:0/134805107 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02800f670 con 0x7fc034064760 2024-01-24T14:10:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.814+0000 7fc0337fe700 1 -- 172.21.15.29:0/134805107 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02800f9f0 con 0x7fc034064760 2024-01-24T14:10:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.814+0000 7fc03aa76700 1 -- 172.21.15.29:0/134805107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc034064760 msgr2=0x7fc034064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.814+0000 7fc03aa76700 1 --2- 172.21.15.29:0/134805107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc034064760 0x7fc034064b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fc028009760 tx=0x7fc028009060).stop 2024-01-24T14:10:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.814+0000 7fc03aa76700 1 -- 172.21.15.29:0/134805107 shutdown_connections 2024-01-24T14:10:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.814+0000 7fc03aa76700 1 --2- 172.21.15.29:0/134805107 >> v2:172.21.15.163:3300/0 conn(0x7fc0340bfd40 0x7fc0340c01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.815+0000 7fc03aa76700 1 --2- 172.21.15.29:0/134805107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc034064760 0x7fc034064b40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.815+0000 7fc03aa76700 1 -- 172.21.15.29:0/134805107 >> 172.21.15.29:0/134805107 conn(0x7fc0340cb8b0 msgr2=0x7fc0340c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.815+0000 7fc03aa76700 1 -- 172.21.15.29:0/134805107 shutdown_connections 2024-01-24T14:10:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.815+0000 7fc03aa76700 1 -- 172.21.15.29:0/134805107 wait complete. 2024-01-24T14:10:43.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.817+0000 7fc03aa76700 1 Processor -- start 2024-01-24T14:10:43.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.817+0000 7fc03aa76700 1 -- start start 2024-01-24T14:10:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc03aa76700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc034064760 0x7fc03414f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc03aa76700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0340bfd40 0x7fc0340afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc03aa76700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0340c8490 0x7fc0340b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc03aa76700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340b0660 con 0x7fc034064760 2024-01-24T14:10:43.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc03aa76700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340c2850 con 0x7fc0340bfd40 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc03aa76700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340af700 con 0x7fc0340c8490 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc038812700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc034064760 0x7fc03414f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc038812700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc034064760 0x7fc03414f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55864/0 (socket says 172.21.15.29:55864) 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc038812700 1 -- 172.21.15.29:0/1186840363 learned_addr learned my addr 172.21.15.29:0/1186840363 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc039013700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0340c8490 0x7fc0340b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc033fff700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0340bfd40 0x7fc0340afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc038812700 1 -- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0340c8490 msgr2=0x7fc0340b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc038812700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0340c8490 0x7fc0340b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc038812700 1 -- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0340bfd40 msgr2=0x7fc0340afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc038812700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0340bfd40 0x7fc0340afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc038812700 1 -- 172.21.15.29:0/1186840363 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c009a20 con 0x7fc034064760 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc038812700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc034064760 0x7fc03414f280 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fc028009d50 tx=0x7fc028017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:43.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.819+0000 7fc030ff9700 1 -- 172.21.15.29:0/1186840363 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02800f7e0 con 0x7fc034064760 2024-01-24T14:10:43.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.820+0000 7fc030ff9700 1 -- 172.21.15.29:0/1186840363 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02800f940 con 0x7fc034064760 2024-01-24T14:10:43.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.820+0000 7fc03aa76700 1 -- 172.21.15.29:0/1186840363 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0280092a0 con 0x7fc034064760 2024-01-24T14:10:43.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.820+0000 7fc030ff9700 1 -- 172.21.15.29:0/1186840363 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02801b690 con 0x7fc034064760 2024-01-24T14:10:43.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.820+0000 7fc03aa76700 1 -- 172.21.15.29:0/1186840363 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0340c2200 con 0x7fc034064760 2024-01-24T14:10:43.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.821+0000 7fc03aa76700 1 -- 172.21.15.29:0/1186840363 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0340c2b30 con 0x7fc034064760 2024-01-24T14:10:43.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.821+0000 7fc030ff9700 1 -- 172.21.15.29:0/1186840363 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fc02801ba80 con 0x7fc034064760 2024-01-24T14:10:43.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.822+0000 7fc030ff9700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc01805f270 0x7fc018061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:43.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.822+0000 7fc033fff700 1 -- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc01805f270 msgr2=0x7fc018061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:43.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.822+0000 7fc030ff9700 1 -- 172.21.15.29:0/1186840363 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7605+0+0 (secure 0 0 0) 0x7fc02807c230 con 0x7fc034064760 2024-01-24T14:10:43.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.822+0000 7fc033fff700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc01805f270 0x7fc018061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:10:43.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:43.826+0000 7fc030ff9700 1 -- 172.21.15.29:0/1186840363 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc02804cdb0 con 0x7fc034064760 2024-01-24T14:10:43.962 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:43.962+0000 7f3897302680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:10:44.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.012+0000 7fc03aa76700 1 -- 172.21.15.29:0/1186840363 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc0340475e0 con 0x7fc034064760 2024-01-24T14:10:44.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.012+0000 7fc030ff9700 1 -- 172.21.15.29:0/1186840363 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc0280503d0 con 0x7fc034064760 2024-01-24T14:10:44.013 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T14:10:44.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.014+0000 7fc03aa76700 1 -- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc01805f270 msgr2=0x7fc018061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:10:44.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.014+0000 7fc03aa76700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc01805f270 0x7fc018061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:44.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.014+0000 7fc03aa76700 1 -- 172.21.15.29:0/1186840363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc034064760 msgr2=0x7fc03414f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.014+0000 7fc03aa76700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc034064760 0x7fc03414f280 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fc028009d50 tx=0x7fc028017000).stop 2024-01-24T14:10:44.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.014+0000 7fc03aa76700 1 -- 172.21.15.29:0/1186840363 shutdown_connections 2024-01-24T14:10:44.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.014+0000 7fc03aa76700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0340c8490 0x7fc0340b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.014+0000 7fc03aa76700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc01805f270 0x7fc018061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:44.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.014+0000 7fc03aa76700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc034064760 0x7fc03414f280 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.014+0000 7fc03aa76700 1 --2- 172.21.15.29:0/1186840363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0340bfd40 0x7fc0340afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.014+0000 7fc03aa76700 1 -- 172.21.15.29:0/1186840363 >> 172.21.15.29:0/1186840363 conn(0x7fc0340cb8b0 msgr2=0x7fc034150400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:44.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.014+0000 7fc03aa76700 1 -- 172.21.15.29:0/1186840363 shutdown_connections 2024-01-24T14:10:44.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.014+0000 7fc03aa76700 1 -- 172.21.15.29:0/1186840363 wait complete. 2024-01-24T14:10:44.028 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T14:10:44.029 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:44.029+0000 7fdadf88b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:10:44.037 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:44.037+0000 7f3897302680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:10:44.104 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:44.104+0000 7f3897302680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:10:44.151 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:44.151+0000 7fdadf88b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:10:44.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.162+0000 7f549baf0700 1 Processor -- start 2024-01-24T14:10:44.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.163+0000 7f549baf0700 1 -- start start 2024-01-24T14:10:44.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.163+0000 7f549baf0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5494064760 0x7f5494064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549baf0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f54940c8490 0x7f54940a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549baf0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5494154700 0x7f5494154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549baf0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5494152240 con 0x7f5494154700 2024-01-24T14:10:44.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549baf0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54941586c0 con 0x7f54940c8490 2024-01-24T14:10:44.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549baf0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54940aa670 con 0x7f5494064760 2024-01-24T14:10:44.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549a08d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5494154700 0x7f5494154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549a08d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5494154700 0x7f5494154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55878/0 (socket says 172.21.15.29:55878) 2024-01-24T14:10:44.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549a08d700 1 -- 172.21.15.29:0/2450811757 learned_addr learned my addr 172.21.15.29:0/2450811757 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549908b700 1 --2- 172.21.15.29:0/2450811757 >> v2:172.21.15.163:3300/0 conn(0x7f54940c8490 0x7f54940a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549988c700 1 --2- 172.21.15.29:0/2450811757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5494064760 0x7f5494064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549a08d700 1 -- 172.21.15.29:0/2450811757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5494064760 msgr2=0x7f5494064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549a08d700 1 --2- 172.21.15.29:0/2450811757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5494064760 0x7f5494064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549a08d700 1 -- 172.21.15.29:0/2450811757 >> v2:172.21.15.163:3300/0 conn(0x7f54940c8490 msgr2=0x7f54940a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549a08d700 1 --2- 172.21.15.29:0/2450811757 >> v2:172.21.15.163:3300/0 conn(0x7f54940c8490 0x7f54940a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549a08d700 1 -- 172.21.15.29:0/2450811757 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5494157d90 con 0x7f5494154700 2024-01-24T14:10:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.164+0000 7f549a08d700 1 --2- 172.21.15.29:0/2450811757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5494154700 0x7f5494154bb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f5494062b50 tx=0x7f548c009840).ready entity=mon.0 client_cookie=bf80a296ae651646 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.165+0000 7f549888a700 1 -- 172.21.15.29:0/2450811757 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c00e070 con 0x7f5494154700 2024-01-24T14:10:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.165+0000 7f549888a700 1 -- 172.21.15.29:0/2450811757 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c00ba00 con 0x7f5494154700 2024-01-24T14:10:44.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.165+0000 7f549888a700 1 -- 172.21.15.29:0/2450811757 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c00bd80 con 0x7f5494154700 2024-01-24T14:10:44.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.165+0000 7f549baf0700 1 -- 172.21.15.29:0/2450811757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5494154700 msgr2=0x7f5494154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.165+0000 7f549baf0700 1 --2- 172.21.15.29:0/2450811757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5494154700 0x7f5494154bb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f5494062b50 tx=0x7f548c009840).stop 2024-01-24T14:10:44.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.165+0000 7f549baf0700 1 -- 172.21.15.29:0/2450811757 shutdown_connections 2024-01-24T14:10:44.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.165+0000 7f549baf0700 1 --2- 172.21.15.29:0/2450811757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5494064760 0x7f5494064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.165+0000 7f549baf0700 1 --2- 172.21.15.29:0/2450811757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5494154700 0x7f5494154bb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.165+0000 7f549baf0700 1 --2- 172.21.15.29:0/2450811757 >> v2:172.21.15.163:3300/0 conn(0x7f54940c8490 0x7f54940a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.165+0000 7f549baf0700 1 -- 172.21.15.29:0/2450811757 >> 172.21.15.29:0/2450811757 conn(0x7f54940cb8b0 msgr2=0x7f54940c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:44.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.165+0000 7f549baf0700 1 -- 172.21.15.29:0/2450811757 shutdown_connections 2024-01-24T14:10:44.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.165+0000 7f549baf0700 1 -- 172.21.15.29:0/2450811757 wait complete. 2024-01-24T14:10:44.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.167+0000 7f549baf0700 1 Processor -- start 2024-01-24T14:10:44.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.167+0000 7f549baf0700 1 -- start start 2024-01-24T14:10:44.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549baf0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5494064760 0x7f54940c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549baf0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54940c8490 0x7f549414f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549baf0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5494154700 0x7f54940b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549baf0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5494164720 con 0x7f5494154700 2024-01-24T14:10:44.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549baf0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5494164a00 con 0x7f54940c8490 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549baf0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5494164ce0 con 0x7f5494064760 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549a08d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5494154700 0x7f54940b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549a08d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5494154700 0x7f54940b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55892/0 (socket says 172.21.15.29:55892) 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549a08d700 1 -- 172.21.15.29:0/44119021 learned_addr learned my addr 172.21.15.29:0/44119021 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549908b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54940c8490 0x7f549414f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549988c700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5494064760 0x7f54940c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549a08d700 1 -- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5494064760 msgr2=0x7f54940c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549a08d700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5494064760 0x7f54940c2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549a08d700 1 -- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54940c8490 msgr2=0x7f549414f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549a08d700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54940c8490 0x7f549414f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549a08d700 1 -- 172.21.15.29:0/44119021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c009ae0 con 0x7f5494154700 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549988c700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5494064760 0x7f54940c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549a08d700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5494154700 0x7f54940b0450 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f548c00b610 tx=0x7f548c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:44.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f5489ffb700 1 -- 172.21.15.29:0/44119021 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c013e60 con 0x7f5494154700 2024-01-24T14:10:44.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f5489ffb700 1 -- 172.21.15.29:0/44119021 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c0092e0 con 0x7f5494154700 2024-01-24T14:10:44.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.168+0000 7f549baf0700 1 -- 172.21.15.29:0/44119021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5494164fc0 con 0x7f5494154700 2024-01-24T14:10:44.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.169+0000 7f5489ffb700 1 -- 172.21.15.29:0/44119021 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c007860 con 0x7f5494154700 2024-01-24T14:10:44.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.169+0000 7f549baf0700 1 -- 172.21.15.29:0/44119021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549414f950 con 0x7f5494154700 2024-01-24T14:10:44.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.170+0000 7f549baf0700 1 -- 172.21.15.29:0/44119021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54940475e0 con 0x7f5494154700 2024-01-24T14:10:44.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.170+0000 7f5489ffb700 1 -- 172.21.15.29:0/44119021 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f548c003040 con 0x7f5494154700 2024-01-24T14:10:44.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.171+0000 7f5489ffb700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f547805f270 0x7f5478061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.171+0000 7f5489ffb700 1 -- 172.21.15.29:0/44119021 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7605+0+0 (secure 0 0 0) 0x7f548c07c130 con 0x7f5494154700 2024-01-24T14:10:44.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.171+0000 7f549988c700 1 -- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f547805f270 msgr2=0x7f5478061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:44.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.171+0000 7f549988c700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f547805f270 0x7f5478061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:10:44.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.175+0000 7f5489ffb700 1 -- 172.21.15.29:0/44119021 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f548c04cd60 con 0x7f5494154700 2024-01-24T14:10:44.331 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:44.331+0000 7fdadf88b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:10:44.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.359+0000 7f549baf0700 1 -- 172.21.15.29:0/44119021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5494158070 con 0x7f5494154700 2024-01-24T14:10:44.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.359+0000 7f5489ffb700 1 -- 172.21.15.29:0/44119021 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7f548c023070 con 0x7f5494154700 2024-01-24T14:10:44.360 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T14:10:44.361 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:44.361+0000 7f3897302680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:10:44.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.362+0000 7f549baf0700 1 -- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f547805f270 msgr2=0x7f5478061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:10:44.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.362+0000 7f549baf0700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f547805f270 0x7f5478061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:44.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.362+0000 7f549baf0700 1 -- 172.21.15.29:0/44119021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5494154700 msgr2=0x7f54940b0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.362+0000 7f549baf0700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5494154700 0x7f54940b0450 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f548c00b610 tx=0x7f548c009840).stop 2024-01-24T14:10:44.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.362+0000 7f549baf0700 1 -- 172.21.15.29:0/44119021 shutdown_connections 2024-01-24T14:10:44.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.362+0000 7f549baf0700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f547805f270 0x7f5478061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:44.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.362+0000 7f549baf0700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5494154700 0x7f54940b0450 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.362+0000 7f549baf0700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5494064760 0x7f54940c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.362+0000 7f549baf0700 1 --2- 172.21.15.29:0/44119021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54940c8490 0x7f549414f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.362+0000 7f549baf0700 1 -- 172.21.15.29:0/44119021 >> 172.21.15.29:0/44119021 conn(0x7f54940cb8b0 msgr2=0x7f54940ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:44.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.362+0000 7f549baf0700 1 -- 172.21.15.29:0/44119021 shutdown_connections 2024-01-24T14:10:44.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.362+0000 7f549baf0700 1 -- 172.21.15.29:0/44119021 wait complete. 2024-01-24T14:10:44.376 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:10:44.423 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:44.422+0000 7fdadf88b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:10:44.501 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:44.501+0000 7fdadf88b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:10:44.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.509+0000 7fda3c627700 1 Processor -- start 2024-01-24T14:10:44.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.509+0000 7fda3c627700 1 -- start start 2024-01-24T14:10:44.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.510+0000 7fda3c627700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda34150cb0 0x7fda34069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.510+0000 7fda3c627700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 0x7fda340bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.510+0000 7fda3c627700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3406a3a0 con 0x7fda34065e50 2024-01-24T14:10:44.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.510+0000 7fda3c627700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda340c2c70 con 0x7fda34150cb0 2024-01-24T14:10:44.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.510+0000 7fda3c627700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda34069820 con 0x7fda34155060 2024-01-24T14:10:44.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.511+0000 7fda3a3c3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda34150cb0 0x7fda34069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.511+0000 7fda3abc4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 0x7fda340bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.511+0000 7fda39bc2700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fda34065e50 0x7fda34154950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:53132/0 (socket says 172.21.15.29:53132) 2024-01-24T14:10:44.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.511+0000 7fda39bc2700 1 -- 172.21.15.29:0/665100407 learned_addr learned my addr 172.21.15.29:0/665100407 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:44.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.511+0000 7fda3a3c3700 1 -- 172.21.15.29:0/665100407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 msgr2=0x7fda340bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:44.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.511+0000 7fda3a3c3700 1 --2- 172.21.15.29:0/665100407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 0x7fda340bcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.511+0000 7fda3a3c3700 1 -- 172.21.15.29:0/665100407 >> v1:172.21.15.163:6789/0 conn(0x7fda34065e50 legacy=0x7fda34154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.511+0000 7fda3a3c3700 1 -- 172.21.15.29:0/665100407 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda34069b00 con 0x7fda34150cb0 2024-01-24T14:10:44.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.511+0000 7fda393c1700 1 -- 172.21.15.29:0/665100407 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849219958 0 0) 0x7fda3406a3a0 con 0x7fda34065e50 2024-01-24T14:10:44.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.511+0000 7fda3abc4700 1 --2- 172.21.15.29:0/665100407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 0x7fda340bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:44.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.511+0000 7fda3a3c3700 1 --2- 172.21.15.29:0/665100407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda34150cb0 0x7fda34069240 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fda28009760 tx=0x7fda28009060).ready entity=mon.1 client_cookie=5323bc29d19f3e05 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:44.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.511+0000 7fda393c1700 1 -- 172.21.15.29:0/665100407 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2800f510 con 0x7fda34150cb0 2024-01-24T14:10:44.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.512+0000 7fda393c1700 1 -- 172.21.15.29:0/665100407 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2800f670 con 0x7fda34150cb0 2024-01-24T14:10:44.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.512+0000 7fda393c1700 1 -- 172.21.15.29:0/665100407 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2800f9f0 con 0x7fda34150cb0 2024-01-24T14:10:44.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.512+0000 7fda3c627700 1 -- 172.21.15.29:0/665100407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda34150cb0 msgr2=0x7fda34069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.512+0000 7fda3c627700 1 --2- 172.21.15.29:0/665100407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda34150cb0 0x7fda34069240 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fda28009760 tx=0x7fda28009060).stop 2024-01-24T14:10:44.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.512+0000 7fda3c627700 1 -- 172.21.15.29:0/665100407 shutdown_connections 2024-01-24T14:10:44.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.512+0000 7fda3c627700 1 --2- 172.21.15.29:0/665100407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 0x7fda340bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.512+0000 7fda3c627700 1 --2- 172.21.15.29:0/665100407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda34150cb0 0x7fda34069240 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.512+0000 7fda3c627700 1 -- 172.21.15.29:0/665100407 >> 172.21.15.29:0/665100407 conn(0x7fda340c85c0 msgr2=0x7fda340c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:44.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.512+0000 7fda3c627700 1 -- 172.21.15.29:0/665100407 shutdown_connections 2024-01-24T14:10:44.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.512+0000 7fda3c627700 1 -- 172.21.15.29:0/665100407 wait complete. 2024-01-24T14:10:44.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.514+0000 7fda3c627700 1 Processor -- start 2024-01-24T14:10:44.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.514+0000 7fda3c627700 1 -- start start 2024-01-24T14:10:44.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.514+0000 7fda3c627700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda34065e50 0x7fda340bf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.514+0000 7fda3c627700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda34150cb0 0x7fda3413e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.514+0000 7fda3c627700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 0x7fda340631b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.514+0000 7fda3c627700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3414cfc0 con 0x7fda34150cb0 2024-01-24T14:10:44.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.514+0000 7fda3c627700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda340c1500 con 0x7fda34065e50 2024-01-24T14:10:44.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.514+0000 7fda3c627700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda340c08f0 con 0x7fda34155060 2024-01-24T14:10:44.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda3abc4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 0x7fda340631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda39bc2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda34150cb0 0x7fda3413e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda3abc4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 0x7fda340631b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:56870/0 (socket says 172.21.15.29:56870) 2024-01-24T14:10:44.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda3a3c3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda34065e50 0x7fda340bf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda3abc4700 1 -- 172.21.15.29:0/3843245830 learned_addr learned my addr 172.21.15.29:0/3843245830 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:44.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda3abc4700 1 -- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda34065e50 msgr2=0x7fda340bf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda3abc4700 1 --2- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda34065e50 0x7fda340bf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda3abc4700 1 -- 172.21.15.29:0/3843245830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda34150cb0 msgr2=0x7fda3413e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda3abc4700 1 --2- 172.21.15.29:0/3843245830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda34150cb0 0x7fda3413e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda3abc4700 1 -- 172.21.15.29:0/3843245830 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c009a20 con 0x7fda34155060 2024-01-24T14:10:44.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda3abc4700 1 --2- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 0x7fda340631b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fda2c009ec0 tx=0x7fda2c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:44.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda267fc700 1 -- 172.21.15.29:0/3843245830 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00cc00 con 0x7fda34155060 2024-01-24T14:10:44.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.515+0000 7fda267fc700 1 -- 172.21.15.29:0/3843245830 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c00f040 con 0x7fda34155060 2024-01-24T14:10:44.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.516+0000 7fda3c627700 1 -- 172.21.15.29:0/3843245830 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda280092a0 con 0x7fda34155060 2024-01-24T14:10:44.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.516+0000 7fda267fc700 1 -- 172.21.15.29:0/3843245830 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c013400 con 0x7fda34155060 2024-01-24T14:10:44.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.516+0000 7fda3c627700 1 -- 172.21.15.29:0/3843245830 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda340c0eb0 con 0x7fda34155060 2024-01-24T14:10:44.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.517+0000 7fda3c627700 1 -- 172.21.15.29:0/3843245830 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda340c2030 con 0x7fda34155060 2024-01-24T14:10:44.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.517+0000 7fda267fc700 1 -- 172.21.15.29:0/3843245830 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fda2c0137b0 con 0x7fda34155060 2024-01-24T14:10:44.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.517+0000 7fda267fc700 1 --2- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fda1805f270 0x7fda18061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.517+0000 7fda3a3c3700 1 -- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fda1805f270 msgr2=0x7fda18061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:44.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.517+0000 7fda267fc700 1 -- 172.21.15.29:0/3843245830 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7605+0+0 (secure 0 0 0) 0x7fda2c07b270 con 0x7fda34155060 2024-01-24T14:10:44.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.517+0000 7fda3a3c3700 1 --2- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fda1805f270 0x7fda18061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:10:44.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.522+0000 7fda267fc700 1 -- 172.21.15.29:0/3843245830 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fda2c04bdf0 con 0x7fda34155060 2024-01-24T14:10:44.545 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:44.546+0000 7f3897302680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:10:44.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.718+0000 7fda3a3c3700 1 -- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fda1805f270 msgr2=0x7fda18061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:44.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.718+0000 7fda3a3c3700 1 --2- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fda1805f270 0x7fda18061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:10:44.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.748+0000 7fda3c627700 1 -- 172.21.15.29:0/3843245830 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda340c1640 con 0x7fda34155060 2024-01-24T14:10:44.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.748+0000 7fda267fc700 1 -- 172.21.15.29:0/3843245830 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4492 (secure 0 0 0) 0x7fda2c04f410 con 0x7fda34155060 2024-01-24T14:10:44.749 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:44.749 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1291112242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1291112242},{"type":"v1","addr":"172.21.15.163:6837","nonce":1291112242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4793,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/241296864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":241296864},{"type":"v1","addr":"172.21.15.163:6836","nonce":241296864}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5127,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2565530631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2565530631},{"type":"v1","addr":"172.21.15.29:6833","nonce":2565530631}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:59:26.370202+0000","modified":"2024-01-24T13:59:29.380658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1245725396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1245725396},{"type":"v1","addr":"172.21.15.29:6835","nonce":1245725396}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-24T14:10:44.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.751+0000 7fda3c627700 1 -- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fda1805f270 msgr2=0x7fda18061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:10:44.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.751+0000 7fda3c627700 1 --2- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fda1805f270 0x7fda18061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:44.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.751+0000 7fda3c627700 1 -- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 msgr2=0x7fda340631b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.751+0000 7fda3c627700 1 --2- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 0x7fda340631b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fda2c009ec0 tx=0x7fda2c00b040).stop 2024-01-24T14:10:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.751+0000 7fda3c627700 1 -- 172.21.15.29:0/3843245830 shutdown_connections 2024-01-24T14:10:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.751+0000 7fda3c627700 1 --2- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda34155060 0x7fda340631b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.751+0000 7fda3c627700 1 --2- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda34065e50 0x7fda340bf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.751+0000 7fda3c627700 1 --2- 172.21.15.29:0/3843245830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fda1805f270 0x7fda18061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.751+0000 7fda3c627700 1 --2- 172.21.15.29:0/3843245830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda34150cb0 0x7fda3413e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.751+0000 7fda3c627700 1 -- 172.21.15.29:0/3843245830 >> 172.21.15.29:0/3843245830 conn(0x7fda340c85c0 msgr2=0x7fda340bce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.751+0000 7fda3c627700 1 -- 172.21.15.29:0/3843245830 shutdown_connections 2024-01-24T14:10:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.751+0000 7fda3c627700 1 -- 172.21.15.29:0/3843245830 wait complete. 2024-01-24T14:10:44.752 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 18 2024-01-24T14:10:44.765 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:10:44.802 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:44.801+0000 7fdadf88b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:10:44.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.898+0000 7f5ad72ab700 1 Processor -- start 2024-01-24T14:10:44.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.899+0000 7f5ad72ab700 1 -- start start 2024-01-24T14:10:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.899+0000 7f5ad72ab700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ad0153fe0 0x7f5ad01543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad72ab700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ad00bd340 0x7f5ad00c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad72ab700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad005e850 con 0x7f5ad00a4880 2024-01-24T14:10:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad72ab700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad00631f0 con 0x7f5ad0153fe0 2024-01-24T14:10:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad72ab700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad00bd7e0 con 0x7f5ad00bd340 2024-01-24T14:10:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad5047700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ad0153fe0 0x7f5ad01543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad5848700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ad00bd340 0x7f5ad00c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad4846700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5ad00a4880 0x7f5ad00bcc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:53142/0 (socket says 172.21.15.29:53142) 2024-01-24T14:10:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad4846700 1 -- 172.21.15.29:0/4062269268 learned_addr learned my addr 172.21.15.29:0/4062269268 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad5047700 1 -- 172.21.15.29:0/4062269268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ad00bd340 msgr2=0x7f5ad00c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad5047700 1 --2- 172.21.15.29:0/4062269268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ad00bd340 0x7f5ad00c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad5047700 1 -- 172.21.15.29:0/4062269268 >> v1:172.21.15.163:6789/0 conn(0x7f5ad00a4880 legacy=0x7f5ad00bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ac7fff700 1 -- 172.21.15.29:0/4062269268 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254936250 0 0) 0x7f5ad005e850 con 0x7f5ad00a4880 2024-01-24T14:10:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad5047700 1 -- 172.21.15.29:0/4062269268 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad00628c0 con 0x7f5ad0153fe0 2024-01-24T14:10:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad5848700 1 --2- 172.21.15.29:0/4062269268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ad00bd340 0x7f5ad00c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.900+0000 7f5ad5047700 1 --2- 172.21.15.29:0/4062269268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ad0153fe0 0x7f5ad01543c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f5acc009760 tx=0x7f5acc009060).ready entity=mon.1 client_cookie=54a40c29e9a65f8f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.901+0000 7f5ac7fff700 1 -- 172.21.15.29:0/4062269268 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc00f510 con 0x7f5ad0153fe0 2024-01-24T14:10:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.901+0000 7f5ac7fff700 1 -- 172.21.15.29:0/4062269268 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5acc00f670 con 0x7f5ad0153fe0 2024-01-24T14:10:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.901+0000 7f5ac7fff700 1 -- 172.21.15.29:0/4062269268 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc00f9f0 con 0x7f5ad0153fe0 2024-01-24T14:10:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.901+0000 7f5ad72ab700 1 -- 172.21.15.29:0/4062269268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ad0153fe0 msgr2=0x7f5ad01543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.901+0000 7f5ad72ab700 1 --2- 172.21.15.29:0/4062269268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ad0153fe0 0x7f5ad01543c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f5acc009760 tx=0x7f5acc009060).stop 2024-01-24T14:10:44.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.901+0000 7f5ad72ab700 1 -- 172.21.15.29:0/4062269268 shutdown_connections 2024-01-24T14:10:44.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.901+0000 7f5ad72ab700 1 --2- 172.21.15.29:0/4062269268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ad00bd340 0x7f5ad00c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.901+0000 7f5ad72ab700 1 --2- 172.21.15.29:0/4062269268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ad0153fe0 0x7f5ad01543c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.901+0000 7f5ad72ab700 1 -- 172.21.15.29:0/4062269268 >> 172.21.15.29:0/4062269268 conn(0x7f5ad0064aa0 msgr2=0x7f5ad0064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:44.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.901+0000 7f5ad72ab700 1 -- 172.21.15.29:0/4062269268 shutdown_connections 2024-01-24T14:10:44.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.902+0000 7f5ad72ab700 1 -- 172.21.15.29:0/4062269268 wait complete. 2024-01-24T14:10:44.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.903+0000 7f5ad72ab700 1 Processor -- start 2024-01-24T14:10:44.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.903+0000 7f5ad72ab700 1 -- start start 2024-01-24T14:10:44.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.904+0000 7f5ad72ab700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ad00a4880 0x7f5ad0150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.904+0000 7f5ad72ab700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ad00bd340 0x7f5ad0154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad72ab700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ad0153fe0 0x7f5ad0154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad72ab700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad0156030 con 0x7f5ad00a4880 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad72ab700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad00bf630 con 0x7f5ad0153fe0 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad72ab700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad0155420 con 0x7f5ad00bd340 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad5047700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ad00a4880 0x7f5ad0150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad5047700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ad00a4880 0x7f5ad0150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55906/0 (socket says 172.21.15.29:55906) 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad5047700 1 -- 172.21.15.29:0/3915519423 learned_addr learned my addr 172.21.15.29:0/3915519423 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad4846700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ad00bd340 0x7f5ad0154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad5848700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ad0153fe0 0x7f5ad0154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad5047700 1 -- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ad00bd340 msgr2=0x7f5ad0154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad5047700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ad00bd340 0x7f5ad0154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad5047700 1 -- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ad0153fe0 msgr2=0x7f5ad0154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:44.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad5047700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ad0153fe0 0x7f5ad0154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad5047700 1 -- 172.21.15.29:0/3915519423 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac8009a20 con 0x7f5ad00a4880 2024-01-24T14:10:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad4846700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ad00bd340 0x7f5ad0154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad5848700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ad0153fe0 0x7f5ad0154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ad5047700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ad00a4880 0x7f5ad0150830 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f5acc009e00 tx=0x7f5acc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.905+0000 7f5ac57fa700 1 -- 172.21.15.29:0/3915519423 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc00f7e0 con 0x7f5ad00a4880 2024-01-24T14:10:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.906+0000 7f5ac57fa700 1 -- 172.21.15.29:0/3915519423 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5acc00f940 con 0x7f5ad00a4880 2024-01-24T14:10:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.906+0000 7f5ad72ab700 1 -- 172.21.15.29:0/3915519423 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc0092a0 con 0x7f5ad00a4880 2024-01-24T14:10:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.906+0000 7f5ac57fa700 1 -- 172.21.15.29:0/3915519423 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc01b660 con 0x7f5ad00a4880 2024-01-24T14:10:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.906+0000 7f5ad72ab700 1 -- 172.21.15.29:0/3915519423 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad01559e0 con 0x7f5ad00a4880 2024-01-24T14:10:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.907+0000 7f5ad72ab700 1 -- 172.21.15.29:0/3915519423 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad00475e0 con 0x7f5ad00a4880 2024-01-24T14:10:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.907+0000 7f5ac57fa700 1 -- 172.21.15.29:0/3915519423 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f5acc01ba10 con 0x7f5ad00a4880 2024-01-24T14:10:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.908+0000 7f5ac57fa700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5ab405f270 0x7f5ab4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.908+0000 7f5ac57fa700 1 -- 172.21.15.29:0/3915519423 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7445+0+0 (secure 0 0 0) 0x7f5acc07c090 con 0x7f5ad00a4880 2024-01-24T14:10:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.908+0000 7f5ad4846700 1 -- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5ab405f270 msgr2=0x7f5ab4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.908+0000 7f5ad4846700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5ab405f270 0x7f5ab4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:10:44.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:44.912+0000 7f5ac57fa700 1 -- 172.21.15.29:0/3915519423 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5acc04ccb0 con 0x7f5ad00a4880 2024-01-24T14:10:45.019 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:45.018+0000 7fdadf88b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:10:45.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.097+0000 7f5ad72ab700 1 -- 172.21.15.29:0/3915519423 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ad005e330 con 0x7f5ad00a4880 2024-01-24T14:10:45.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.099+0000 7f5ac57fa700 1 -- 172.21.15.29:0/3915519423 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+18651 (secure 0 0 0) 0x7f5acc0502d0 con 0x7f5ad00a4880 2024-01-24T14:10:45.100 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:45.100 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":58,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:10:44.887644+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:59:24.684931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-24T13:59:25.345273+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":58,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]},{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6832/43241":"2024-01-25T13:59:43.884293+0000","172.21.15.163:0/46643247":"2024-01-25T13:59:43.884293+0000","172.21.15.163:6833/43241":"2024-01-25T13:59:43.884293+0000","172.21.15.163:0/233457199":"2024-01-25T13:59:43.884293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":110,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:10:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.101+0000 7f5ad72ab700 1 -- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5ab405f270 msgr2=0x7f5ab4061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:10:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.101+0000 7f5ad72ab700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5ab405f270 0x7f5ab4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.101+0000 7f5ad72ab700 1 -- 172.21.15.29:0/3915519423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ad00a4880 msgr2=0x7f5ad0150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.101+0000 7f5ad72ab700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ad00a4880 0x7f5ad0150830 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f5acc009e00 tx=0x7f5acc017000).stop 2024-01-24T14:10:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.101+0000 7f5ad72ab700 1 -- 172.21.15.29:0/3915519423 shutdown_connections 2024-01-24T14:10:45.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.101+0000 7f5ad72ab700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ad0153fe0 0x7f5ad0154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.101+0000 7f5ad72ab700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5ab405f270 0x7f5ab4061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.101+0000 7f5ad72ab700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ad00a4880 0x7f5ad0150830 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.101+0000 7f5ad72ab700 1 --2- 172.21.15.29:0/3915519423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ad00bd340 0x7f5ad0154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.101+0000 7f5ad72ab700 1 -- 172.21.15.29:0/3915519423 >> 172.21.15.29:0/3915519423 conn(0x7f5ad0064aa0 msgr2=0x7f5ad00c1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.101+0000 7f5ad72ab700 1 -- 172.21.15.29:0/3915519423 shutdown_connections 2024-01-24T14:10:45.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.102+0000 7f5ad72ab700 1 -- 172.21.15.29:0/3915519423 wait complete. 2024-01-24T14:10:45.116 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T14:10:45.117 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:10:45.201 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:45.201+0000 7f3897302680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:10:45.268 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:45.269+0000 7f3897302680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:10:45.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.290+0000 7ff89babd700 1 Processor -- start 2024-01-24T14:10:45.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.290+0000 7ff89babd700 1 -- start start 2024-01-24T14:10:45.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff89babd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff894154370 0x7ff8940c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:45.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff89babd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff894154750 0x7ff8940631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:45.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff89babd700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8940aa6c0 con 0x7ff8940aa0e0 2024-01-24T14:10:45.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff89babd700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff894154ba0 con 0x7ff894154370 2024-01-24T14:10:45.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff89babd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff894064240 con 0x7ff894154750 2024-01-24T14:10:45.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff899859700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff894154370 0x7ff8940c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:45.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff899859700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff894154370 0x7ff8940c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55908/0 (socket says 172.21.15.29:55908) 2024-01-24T14:10:45.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff899058700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff8940aa0e0 0x7ff8940ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48182/0 (socket says 172.21.15.29:48182) 2024-01-24T14:10:45.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff899859700 1 -- 172.21.15.29:0/1827982516 learned_addr learned my addr 172.21.15.29:0/1827982516 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:45.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff89a05a700 1 --2- 172.21.15.29:0/1827982516 >> v2:172.21.15.163:3300/0 conn(0x7ff894154750 0x7ff8940631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff899859700 1 -- 172.21.15.29:0/1827982516 >> v2:172.21.15.163:3300/0 conn(0x7ff894154750 msgr2=0x7ff8940631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff899859700 1 --2- 172.21.15.29:0/1827982516 >> v2:172.21.15.163:3300/0 conn(0x7ff894154750 0x7ff8940631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff899859700 1 -- 172.21.15.29:0/1827982516 >> v1:172.21.15.29:6789/0 conn(0x7ff8940aa0e0 legacy=0x7ff8940ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff899859700 1 -- 172.21.15.29:0/1827982516 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8940cb5c0 con 0x7ff894154370 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.291+0000 7ff898857700 1 -- 172.21.15.29:0/1827982516 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757563714 0 0) 0x7ff8940aa6c0 con 0x7ff8940aa0e0 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.292+0000 7ff899859700 1 --2- 172.21.15.29:0/1827982516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff894154370 0x7ff8940c5410 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7ff89405f190 tx=0x7ff890009060).ready entity=mon.0 client_cookie=da499a4c433f3003 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.292+0000 7ff898857700 1 -- 172.21.15.29:0/1827982516 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890005880 con 0x7ff894154370 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.292+0000 7ff898857700 1 -- 172.21.15.29:0/1827982516 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8900059e0 con 0x7ff894154370 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.292+0000 7ff898857700 1 -- 172.21.15.29:0/1827982516 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890005d60 con 0x7ff894154370 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.292+0000 7ff89babd700 1 -- 172.21.15.29:0/1827982516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff894154370 msgr2=0x7ff8940c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.292+0000 7ff89babd700 1 --2- 172.21.15.29:0/1827982516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff894154370 0x7ff8940c5410 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7ff89405f190 tx=0x7ff890009060).stop 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.292+0000 7ff89babd700 1 -- 172.21.15.29:0/1827982516 shutdown_connections 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.292+0000 7ff89babd700 1 --2- 172.21.15.29:0/1827982516 >> v2:172.21.15.163:3300/0 conn(0x7ff894154750 0x7ff8940631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.292+0000 7ff89babd700 1 --2- 172.21.15.29:0/1827982516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff894154370 0x7ff8940c5410 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.292+0000 7ff89babd700 1 -- 172.21.15.29:0/1827982516 >> 172.21.15.29:0/1827982516 conn(0x7ff89405ec80 msgr2=0x7ff8940cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:45.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.293+0000 7ff89babd700 1 -- 172.21.15.29:0/1827982516 shutdown_connections 2024-01-24T14:10:45.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.293+0000 7ff89babd700 1 -- 172.21.15.29:0/1827982516 wait complete. 2024-01-24T14:10:45.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.294+0000 7ff89babd700 1 Processor -- start 2024-01-24T14:10:45.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.294+0000 7ff89babd700 1 -- start start 2024-01-24T14:10:45.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.295+0000 7ff89babd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8940aa0e0 0x7ff8940ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:45.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.295+0000 7ff89babd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff894154370 0x7ff89414dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:45.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.295+0000 7ff89babd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff894154750 0x7ff894158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:45.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.295+0000 7ff89babd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89414e4d0 con 0x7ff894154750 2024-01-24T14:10:45.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.295+0000 7ff89babd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940afce0 con 0x7ff8940aa0e0 2024-01-24T14:10:45.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.295+0000 7ff89babd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff894158a50 con 0x7ff894154370 2024-01-24T14:10:45.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff89a05a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff894154750 0x7ff894158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff89a05a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff894154750 0x7ff894158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55920/0 (socket says 172.21.15.29:55920) 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff899058700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff894154370 0x7ff89414dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff89a05a700 1 -- 172.21.15.29:0/3082574378 learned_addr learned my addr 172.21.15.29:0/3082574378 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff899859700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8940aa0e0 0x7ff8940ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff89a05a700 1 -- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff894154370 msgr2=0x7ff89414dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff89a05a700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff894154370 0x7ff89414dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff89a05a700 1 -- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8940aa0e0 msgr2=0x7ff8940ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff89a05a700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8940aa0e0 0x7ff8940ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff89a05a700 1 -- 172.21.15.29:0/3082574378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff890009300 con 0x7ff894154750 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff89a05a700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff894154750 0x7ff894158150 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7ff88c009df0 tx=0x7ff88c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff889ffb700 1 -- 172.21.15.29:0/3082574378 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00caa0 con 0x7ff894154750 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff889ffb700 1 -- 172.21.15.29:0/3082574378 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c00f040 con 0x7ff894154750 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff89babd700 1 -- 172.21.15.29:0/3082574378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff894158d30 con 0x7ff894154750 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.296+0000 7ff889ffb700 1 -- 172.21.15.29:0/3082574378 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00ce10 con 0x7ff894154750 2024-01-24T14:10:45.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.297+0000 7ff89babd700 1 -- 172.21.15.29:0/3082574378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8940af980 con 0x7ff894154750 2024-01-24T14:10:45.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.298+0000 7ff89babd700 1 -- 172.21.15.29:0/3082574378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8940afe50 con 0x7ff894154750 2024-01-24T14:10:45.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.298+0000 7ff889ffb700 1 -- 172.21.15.29:0/3082574378 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7ff88c013610 con 0x7ff894154750 2024-01-24T14:10:45.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.298+0000 7ff889ffb700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff87805f240 0x7ff878061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:45.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.299+0000 7ff899859700 1 -- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff87805f240 msgr2=0x7ff878061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:45.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.299+0000 7ff889ffb700 1 -- 172.21.15.29:0/3082574378 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7445+0+0 (secure 0 0 0) 0x7ff88c07aff0 con 0x7ff894154750 2024-01-24T14:10:45.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.299+0000 7ff899859700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff87805f240 0x7ff878061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:10:45.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.303+0000 7ff889ffb700 1 -- 172.21.15.29:0/3082574378 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff88c04bc10 con 0x7ff894154750 2024-01-24T14:10:45.344 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:45.344+0000 7f3897302680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:10:45.490 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:45.490+0000 7f3897302680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:10:45.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.500+0000 7ff899859700 1 -- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff87805f240 msgr2=0x7ff878061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:45.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.500+0000 7ff899859700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff87805f240 0x7ff878061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:10:45.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.532+0000 7ff89babd700 1 -- 172.21.15.29:0/3082574378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8940475e0 con 0x7ff894154750 2024-01-24T14:10:45.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.533+0000 7ff889ffb700 1 -- 172.21.15.29:0/3082574378 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4492 (secure 0 0 0) 0x7ff88c04f230 con 0x7ff894154750 2024-01-24T14:10:45.533 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:45.533 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1291112242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1291112242},{"type":"v1","addr":"172.21.15.163:6837","nonce":1291112242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4793,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/241296864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":241296864},{"type":"v1","addr":"172.21.15.163:6836","nonce":241296864}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5127,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2565530631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2565530631},{"type":"v1","addr":"172.21.15.29:6833","nonce":2565530631}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:59:26.370202+0000","modified":"2024-01-24T13:59:29.380658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1245725396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1245725396},{"type":"v1","addr":"172.21.15.29:6835","nonce":1245725396}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-24T14:10:45.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.535+0000 7ff89babd700 1 -- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff87805f240 msgr2=0x7ff878061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:10:45.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.535+0000 7ff89babd700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff87805f240 0x7ff878061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.535+0000 7ff89babd700 1 -- 172.21.15.29:0/3082574378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff894154750 msgr2=0x7ff894158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.535+0000 7ff89babd700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff894154750 0x7ff894158150 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7ff88c009df0 tx=0x7ff88c00b040).stop 2024-01-24T14:10:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.535+0000 7ff89babd700 1 -- 172.21.15.29:0/3082574378 shutdown_connections 2024-01-24T14:10:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.535+0000 7ff89babd700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff87805f240 0x7ff878061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.535+0000 7ff89babd700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff894154750 0x7ff894158150 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.535+0000 7ff89babd700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8940aa0e0 0x7ff8940ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.535+0000 7ff89babd700 1 --2- 172.21.15.29:0/3082574378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff894154370 0x7ff89414dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.535+0000 7ff89babd700 1 -- 172.21.15.29:0/3082574378 >> 172.21.15.29:0/3082574378 conn(0x7ff89405ec80 msgr2=0x7ff8940c7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.535+0000 7ff89babd700 1 -- 172.21.15.29:0/3082574378 shutdown_connections 2024-01-24T14:10:45.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.535+0000 7ff89babd700 1 -- 172.21.15.29:0/3082574378 wait complete. 2024-01-24T14:10:45.537 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 18 2024-01-24T14:10:45.549 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:10:45.559 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:45.560+0000 7f3897302680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:10:45.672 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:45.673+0000 7f3897302680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:10:45.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.683+0000 7f7ee865d700 1 Processor -- start 2024-01-24T14:10:45.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.683+0000 7f7ee865d700 1 -- start start 2024-01-24T14:10:45.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.684+0000 7f7ee865d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee0154370 0x7f7ee00c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:45.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.684+0000 7f7ee865d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ee00aa0e0 0x7f7ee0154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:45.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.684+0000 7f7ee865d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee00aa6c0 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.684+0000 7f7ee865d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee0064240 con 0x7f7ee00aa0e0 2024-01-24T14:10:45.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.684+0000 7f7ee865d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00cb5c0 con 0x7f7ee0154370 2024-01-24T14:10:45.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.684+0000 7f7ee5bf8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ee00aa0e0 0x7f7ee0154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:45.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.684+0000 7f7ee5bf8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ee00aa0e0 0x7f7ee0154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55934/0 (socket says 172.21.15.29:55934) 2024-01-24T14:10:45.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.684+0000 7f7ee5bf8700 1 -- 172.21.15.29:0/3733883607 learned_addr learned my addr 172.21.15.29:0/3733883607 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:45.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.684+0000 7f7ee63f9700 1 --2- 172.21.15.29:0/3733883607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee0154370 0x7f7ee00c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:45.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.685+0000 7f7ee5bf8700 1 -- 172.21.15.29:0/3733883607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee0154370 msgr2=0x7f7ee00c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:45.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.685+0000 7f7ee5bf8700 1 --2- 172.21.15.29:0/3733883607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee0154370 0x7f7ee00c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.685+0000 7f7ee5bf8700 1 -- 172.21.15.29:0/3733883607 >> v1:172.21.15.163:6789/0 conn(0x7f7ee00ab1f0 legacy=0x7f7ee00631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:45.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.685+0000 7f7ee5bf8700 1 -- 172.21.15.29:0/3733883607 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee00cb8a0 con 0x7f7ee00aa0e0 2024-01-24T14:10:45.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.685+0000 7f7ee53f7700 1 -- 172.21.15.29:0/3733883607 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473500985 0 0) 0x7f7ee00aa6c0 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.685+0000 7f7ee5bf8700 1 --2- 172.21.15.29:0/3733883607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ee00aa0e0 0x7f7ee0154750 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f7ed0006fd0 tx=0x7f7ed00090f0).ready entity=mon.0 client_cookie=726d568586e310b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:45.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.685+0000 7f7ee53f7700 1 -- 172.21.15.29:0/3733883607 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed000b580 con 0x7f7ee00aa0e0 2024-01-24T14:10:45.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.685+0000 7f7ee53f7700 1 -- 172.21.15.29:0/3733883607 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed000b6e0 con 0x7f7ee00aa0e0 2024-01-24T14:10:45.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.685+0000 7f7ee53f7700 1 -- 172.21.15.29:0/3733883607 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed000ba60 con 0x7f7ee00aa0e0 2024-01-24T14:10:45.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.686+0000 7f7ee865d700 1 -- 172.21.15.29:0/3733883607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ee00aa0e0 msgr2=0x7f7ee0154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:45.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.686+0000 7f7ee865d700 1 --2- 172.21.15.29:0/3733883607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ee00aa0e0 0x7f7ee0154750 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f7ed0006fd0 tx=0x7f7ed00090f0).stop 2024-01-24T14:10:45.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.686+0000 7f7ee865d700 1 -- 172.21.15.29:0/3733883607 shutdown_connections 2024-01-24T14:10:45.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.686+0000 7f7ee865d700 1 --2- 172.21.15.29:0/3733883607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee0154370 0x7f7ee00c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.686+0000 7f7ee865d700 1 --2- 172.21.15.29:0/3733883607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ee00aa0e0 0x7f7ee0154750 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.686+0000 7f7ee865d700 1 -- 172.21.15.29:0/3733883607 >> 172.21.15.29:0/3733883607 conn(0x7f7ee005ec80 msgr2=0x7f7ee00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:45.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.686+0000 7f7ee865d700 1 -- 172.21.15.29:0/3733883607 shutdown_connections 2024-01-24T14:10:45.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.686+0000 7f7ee865d700 1 -- 172.21.15.29:0/3733883607 wait complete. 2024-01-24T14:10:45.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.687+0000 7f7ee865d700 1 Processor -- start 2024-01-24T14:10:45.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.687+0000 7f7ee865d700 1 -- start start 2024-01-24T14:10:45.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.688+0000 7f7ee865d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ee00aa0e0 0x7f7ee0165ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:45.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.688+0000 7f7ee865d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ee00ab1f0 0x7f7ee00c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:45.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.688+0000 7f7ee865d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee0154370 0x7f7ee00c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:45.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.688+0000 7f7ee865d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00ab7a0 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.688+0000 7f7ee865d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00aba80 con 0x7f7ee00aa0e0 2024-01-24T14:10:45.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.688+0000 7f7ee865d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00abd60 con 0x7f7ee0154370 2024-01-24T14:10:45.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.688+0000 7f7ee5bf8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ee00ab1f0 0x7f7ee00c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:45.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.688+0000 7f7ee63f9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ee00aa0e0 0x7f7ee0165ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:45.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.688+0000 7f7ee5bf8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ee00ab1f0 0x7f7ee00c23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55944/0 (socket says 172.21.15.29:55944) 2024-01-24T14:10:45.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.688+0000 7f7ee5bf8700 1 -- 172.21.15.29:0/3996778988 learned_addr learned my addr 172.21.15.29:0/3996778988 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:45.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.688+0000 7f7ee6bfa700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee0154370 0x7f7ee00c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:45.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.689+0000 7f7ee5bf8700 1 -- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee0154370 msgr2=0x7f7ee00c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:45.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.689+0000 7f7ee5bf8700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee0154370 0x7f7ee00c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.689+0000 7f7ee5bf8700 1 -- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ee00aa0e0 msgr2=0x7f7ee0165ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:45.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.689+0000 7f7ee5bf8700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ee00aa0e0 0x7f7ee0165ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.689+0000 7f7ee5bf8700 1 -- 172.21.15.29:0/3996778988 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed0009370 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.689+0000 7f7ee63f9700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ee00aa0e0 0x7f7ee0165ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:45.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.689+0000 7f7ee5bf8700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ee00ab1f0 0x7f7ee00c23c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f7ed0011000 tx=0x7f7ed0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:45.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.689+0000 7f7ed67fc700 1 -- 172.21.15.29:0/3996778988 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0000bc0 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.689+0000 7f7ed67fc700 1 -- 172.21.15.29:0/3996778988 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed000b850 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.689+0000 7f7ed67fc700 1 -- 172.21.15.29:0/3996778988 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0000d20 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.689+0000 7f7ee865d700 1 -- 172.21.15.29:0/3996778988 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee00ac040 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.689+0000 7f7ee865d700 1 -- 172.21.15.29:0/3996778988 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee013c3b0 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.691+0000 7f7ed67fc700 1 -- 172.21.15.29:0/3996778988 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f7ed001cad0 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.691+0000 7f7ee865d700 1 -- 172.21.15.29:0/3996778988 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee00842b0 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.691+0000 7f7ed67fc700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ec405f240 0x7f7ec4061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:45.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.691+0000 7f7ed67fc700 1 -- 172.21.15.29:0/3996778988 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7445+0+0 (secure 0 0 0) 0x7f7ed007c6f0 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.691+0000 7f7ee63f9700 1 -- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ec405f240 msgr2=0x7f7ec4061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:45.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.691+0000 7f7ee63f9700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ec405f240 0x7f7ec4061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:10:45.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.696+0000 7f7ed67fc700 1 -- 172.21.15.29:0/3996778988 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7ed004d2b0 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.780 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:45.781+0000 7f3897302680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:10:45.797 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:45.797+0000 7fdadf88b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:10:45.876 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:45.875+0000 7fdadf88b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:10:45.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.892+0000 7f7ee63f9700 1 -- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ec405f240 msgr2=0x7f7ec4061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:45.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.892+0000 7f7ee63f9700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ec405f240 0x7f7ec4061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:10:45.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.926+0000 7f7ee865d700 1 -- 172.21.15.29:0/3996778988 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ee00ab8e0 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.927+0000 7f7ed67fc700 1 -- 172.21.15.29:0/3996778988 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f7ee00ab8e0 con 0x7f7ee00ab1f0 2024-01-24T14:10:45.927 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:45.927 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1291112242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1291112242},{"type":"v1","addr":"172.21.15.163:6837","nonce":1291112242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4793,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/241296864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":241296864},{"type":"v1","addr":"172.21.15.163:6836","nonce":241296864}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5127,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2565530631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2565530631},{"type":"v1","addr":"172.21.15.29:6833","nonce":2565530631}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T13:59:26.370202+0000","modified":"2024-01-24T13:59:29.380658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1245725396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1245725396},{"type":"v1","addr":"172.21.15.29:6835","nonce":1245725396}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-24T14:10:45.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.929+0000 7f7ee865d700 1 -- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ec405f240 msgr2=0x7f7ec4061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:10:45.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.929+0000 7f7ee865d700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ec405f240 0x7f7ec4061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:45.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.929+0000 7f7ee865d700 1 -- 172.21.15.29:0/3996778988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ee00ab1f0 msgr2=0x7f7ee00c23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:45.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.929+0000 7f7ee865d700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ee00ab1f0 0x7f7ee00c23c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f7ed0011000 tx=0x7f7ed0017000).stop 2024-01-24T14:10:45.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.929+0000 7f7ee865d700 1 -- 172.21.15.29:0/3996778988 shutdown_connections 2024-01-24T14:10:45.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.929+0000 7f7ee865d700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee0154370 0x7f7ee00c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.929+0000 7f7ee865d700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ee00aa0e0 0x7f7ee0165ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.929+0000 7f7ee865d700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ec405f240 0x7f7ec4061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:45.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.929+0000 7f7ee865d700 1 --2- 172.21.15.29:0/3996778988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ee00ab1f0 0x7f7ee00c23c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:45.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.929+0000 7f7ee865d700 1 -- 172.21.15.29:0/3996778988 >> 172.21.15.29:0/3996778988 conn(0x7f7ee005ec80 msgr2=0x7f7ee00c9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:45.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.929+0000 7f7ee865d700 1 -- 172.21.15.29:0/3996778988 shutdown_connections 2024-01-24T14:10:45.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:45.929+0000 7f7ee865d700 1 -- 172.21.15.29:0/3996778988 wait complete. 2024-01-24T14:10:45.931 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 18 2024-01-24T14:10:45.944 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:10:45.964 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:45.964+0000 7fdadf88b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:10:46.076 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:46.076+0000 7f3897302680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:10:46.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.078+0000 7fdbd7aef700 1 Processor -- start 2024-01-24T14:10:46.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.078+0000 7fdbd7aef700 1 -- start start 2024-01-24T14:10:46.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.079+0000 7fdbd7aef700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd014b300 0x7fdbd0068dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:46.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.079+0000 7fdbd7aef700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdbd014b6e0 0x7fdbd00692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:46.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.079+0000 7fdbd7aef700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd006a1c0 con 0x7fdbd00627b0 2024-01-24T14:10:46.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.079+0000 7fdbd7aef700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd00c2a90 con 0x7fdbd014b6e0 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.079+0000 7fdbd7aef700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd0069920 con 0x7fdbd014b300 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.079+0000 7fdbd508a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdbd014b6e0 0x7fdbd00692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.079+0000 7fdbd608c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdbd00627b0 0x7fdbd0062be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48192/0 (socket says 172.21.15.29:48192) 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.079+0000 7fdbd608c700 1 -- 172.21.15.29:0/1407541032 learned_addr learned my addr 172.21.15.29:0/1407541032 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.079+0000 7fdbd588b700 1 --2- 172.21.15.29:0/1407541032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd014b300 0x7fdbd0068dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.079+0000 7fdbd508a700 1 -- 172.21.15.29:0/1407541032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd014b300 msgr2=0x7fdbd0068dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.079+0000 7fdbd508a700 1 --2- 172.21.15.29:0/1407541032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd014b300 0x7fdbd0068dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.080+0000 7fdbd508a700 1 -- 172.21.15.29:0/1407541032 >> v1:172.21.15.29:6789/0 conn(0x7fdbd00627b0 legacy=0x7fdbd0062be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.080+0000 7fdbd508a700 1 -- 172.21.15.29:0/1407541032 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd0069c00 con 0x7fdbd014b6e0 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.080+0000 7fdbd4889700 1 -- 172.21.15.29:0/1407541032 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2440539623 0 0) 0x7fdbd006a1c0 con 0x7fdbd00627b0 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.080+0000 7fdbd588b700 1 --2- 172.21.15.29:0/1407541032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd014b300 0x7fdbd0068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.080+0000 7fdbd508a700 1 --2- 172.21.15.29:0/1407541032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd014b6e0 0x7fdbd00692c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7fdbc0009760 tx=0x7fdbc0009060).ready entity=mon.0 client_cookie=fc90399acf2ecd02 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.080+0000 7fdbd4889700 1 -- 172.21.15.29:0/1407541032 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00090d0 con 0x7fdbd014b6e0 2024-01-24T14:10:46.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.080+0000 7fdbd4889700 1 -- 172.21.15.29:0/1407541032 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc000f510 con 0x7fdbd014b6e0 2024-01-24T14:10:46.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.080+0000 7fdbd4889700 1 -- 172.21.15.29:0/1407541032 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000f890 con 0x7fdbd014b6e0 2024-01-24T14:10:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.080+0000 7fdbd7aef700 1 -- 172.21.15.29:0/1407541032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd014b6e0 msgr2=0x7fdbd00692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.080+0000 7fdbd7aef700 1 --2- 172.21.15.29:0/1407541032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd014b6e0 0x7fdbd00692c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7fdbc0009760 tx=0x7fdbc0009060).stop 2024-01-24T14:10:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.081+0000 7fdbd7aef700 1 -- 172.21.15.29:0/1407541032 shutdown_connections 2024-01-24T14:10:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.081+0000 7fdbd7aef700 1 --2- 172.21.15.29:0/1407541032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd014b300 0x7fdbd0068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.081+0000 7fdbd7aef700 1 --2- 172.21.15.29:0/1407541032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd014b6e0 0x7fdbd00692c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.081+0000 7fdbd7aef700 1 -- 172.21.15.29:0/1407541032 >> 172.21.15.29:0/1407541032 conn(0x7fdbd00c69f0 msgr2=0x7fdbd00c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.081+0000 7fdbd7aef700 1 -- 172.21.15.29:0/1407541032 shutdown_connections 2024-01-24T14:10:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.081+0000 7fdbd7aef700 1 -- 172.21.15.29:0/1407541032 wait complete. 2024-01-24T14:10:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.082+0000 7fdbd7aef700 1 Processor -- start 2024-01-24T14:10:46.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.082+0000 7fdbd7aef700 1 -- start start 2024-01-24T14:10:46.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd7aef700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd00627b0 0x7fdbd00ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:46.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd7aef700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd014b300 0x7fdbd0154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd7aef700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbd0155170 0x7fdbd0167fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd7aef700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00ab5c0 con 0x7fdbd00627b0 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd7aef700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01555b0 con 0x7fdbd0155170 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd7aef700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00aac90 con 0x7fdbd014b300 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd588b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd00627b0 0x7fdbd00ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd588b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd00627b0 0x7fdbd00ac480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55950/0 (socket says 172.21.15.29:55950) 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd588b700 1 -- 172.21.15.29:0/803833281 learned_addr learned my addr 172.21.15.29:0/803833281 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd508a700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd014b300 0x7fdbd0154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd608c700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbd0155170 0x7fdbd0167fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd508a700 1 -- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbd0155170 msgr2=0x7fdbd0167fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd508a700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbd0155170 0x7fdbd0167fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd508a700 1 -- 172.21.15.29:0/803833281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd00627b0 msgr2=0x7fdbd00ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:46.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd508a700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd00627b0 0x7fdbd00ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd508a700 1 -- 172.21.15.29:0/803833281 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc0092a0 con 0x7fdbd014b300 2024-01-24T14:10:46.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.083+0000 7fdbd588b700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd00627b0 0x7fdbd00ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:46.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.084+0000 7fdbd508a700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd014b300 0x7fdbd0154c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fdbc0014000 tx=0x7fdbc0017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:46.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.084+0000 7fdbc5ffb700 1 -- 172.21.15.29:0/803833281 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000f890 con 0x7fdbd014b300 2024-01-24T14:10:46.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.084+0000 7fdbc5ffb700 1 -- 172.21.15.29:0/803833281 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc0009e30 con 0x7fdbd014b300 2024-01-24T14:10:46.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.084+0000 7fdbd7aef700 1 -- 172.21.15.29:0/803833281 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc00092a0 con 0x7fdbd014b300 2024-01-24T14:10:46.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.084+0000 7fdbc5ffb700 1 -- 172.21.15.29:0/803833281 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000fea0 con 0x7fdbd014b300 2024-01-24T14:10:46.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.084+0000 7fdbd7aef700 1 -- 172.21.15.29:0/803833281 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd00ab250 con 0x7fdbd014b300 2024-01-24T14:10:46.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.085+0000 7fdbd7aef700 1 -- 172.21.15.29:0/803833281 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd00475e0 con 0x7fdbd014b300 2024-01-24T14:10:46.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.085+0000 7fdbc5ffb700 1 -- 172.21.15.29:0/803833281 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fdbc001b620 con 0x7fdbd014b300 2024-01-24T14:10:46.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.086+0000 7fdbc5ffb700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdbb405f270 0x7fdbb4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:46.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.086+0000 7fdbd588b700 1 -- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdbb405f270 msgr2=0x7fdbb4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:10:46.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.086+0000 7fdbc5ffb700 1 -- 172.21.15.29:0/803833281 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7445+0+0 (secure 0 0 0) 0x7fdbc007c0d0 con 0x7fdbd014b300 2024-01-24T14:10:46.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.086+0000 7fdbd588b700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdbb405f270 0x7fdbb4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:10:46.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.090+0000 7fdbc5ffb700 1 -- 172.21.15.29:0/803833281 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdbc004cc30 con 0x7fdbd014b300 2024-01-24T14:10:46.140 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:46.140+0000 7fdadf88b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:10:46.147 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:46.147+0000 7f3897302680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:10:46.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.171+0000 7fdbc5ffb700 1 -- 172.21.15.29:0/803833281 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mgrmap(e 12) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fdbc0079730 con 0x7fdbd014b300 2024-01-24T14:10:46.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.172+0000 7fdbc5ffb700 1 -- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdbb405f270 msgr2=0x7fdbb4061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:10:46.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.172+0000 7fdbc5ffb700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdbb405f270 0x7fdbb4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:46.227 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:46.227+0000 7fdadf88b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:10:46.231 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:46.231+0000 7f381408a700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:10:46.231 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:46.231+0000 7f381408a700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:10:46.231 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:46.231+0000 7f3810082700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:10:46.231 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:46.231+0000 7f381408a700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:10:46.231 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:46.231+0000 7f381408a700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:10:46.231 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:46.231+0000 7f381408a700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:10:46.231 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:46.231+0000 7f3810082700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:10:46.231 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:46.231+0000 7f3810082700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:10:46.231 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:46.231+0000 7f3810082700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:10:46.231 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:10:46.231+0000 7f3810082700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:10:46.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.284+0000 7fdbd7aef700 1 -- 172.21.15.29:0/803833281 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbd00ab700 con 0x7fdbd014b300 2024-01-24T14:10:46.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.285+0000 7fdbc5ffb700 1 -- 172.21.15.29:0/803833281 <== mon.2 v2:172.21.15.163:3301/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18832 (secure 0 0 0) 0x7fdbc001f090 con 0x7fdbd014b300 2024-01-24T14:10:46.286 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:46.286 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":59,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:10:46.161329+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-24T13:59:24.684931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-24T13:59:25.345273+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":58,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]},{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3803950841":"2024-01-25T14:10:46.161295+0000","172.21.15.163:0/907048490":"2024-01-25T14:10:46.161295+0000","172.21.15.163:0/2450130962":"2024-01-25T14:10:46.161295+0000","172.21.15.163:0/1674777101":"2024-01-25T14:10:46.161295+0000","172.21.15.163:0/233457199":"2024-01-25T13:59:43.884293+0000","172.21.15.163:0/46643247":"2024-01-25T13:59:43.884293+0000","172.21.15.163:6833/43241":"2024-01-25T13:59:43.884293+0000","172.21.15.163:6832/43241":"2024-01-25T13:59:43.884293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:10:46.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.287+0000 7fdbd7aef700 1 -- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd014b300 msgr2=0x7fdbd0154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:46.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.287+0000 7fdbd7aef700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd014b300 0x7fdbd0154c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fdbc0014000 tx=0x7fdbc0017000).stop 2024-01-24T14:10:46.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.287+0000 7fdbd7aef700 1 -- 172.21.15.29:0/803833281 shutdown_connections 2024-01-24T14:10:46.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.287+0000 7fdbd7aef700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbd0155170 0x7fdbd0167fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.287+0000 7fdbd7aef700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdbb405f270 0x7fdbb4061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:10:46.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.287+0000 7fdbd7aef700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd00627b0 0x7fdbd00ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.287+0000 7fdbd7aef700 1 --2- 172.21.15.29:0/803833281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd014b300 0x7fdbd0154c80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.287+0000 7fdbd7aef700 1 -- 172.21.15.29:0/803833281 >> 172.21.15.29:0/803833281 conn(0x7fdbd00c69f0 msgr2=0x7fdbd0149490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:46.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.287+0000 7fdbd7aef700 1 -- 172.21.15.29:0/803833281 shutdown_connections 2024-01-24T14:10:46.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.287+0000 7fdbd7aef700 1 -- 172.21.15.29:0/803833281 wait complete. 2024-01-24T14:10:46.302 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-24T14:10:46.366 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:46.365+0000 7fdadf88b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:10:46.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.479+0000 7f21a6397700 1 Processor -- start 2024-01-24T14:10:46.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.479+0000 7f21a6397700 1 -- start start 2024-01-24T14:10:46.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.480+0000 7f21a6397700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f21a0150bd0 0x7f21a0065e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:46.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.480+0000 7f21a6397700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21a01516c0 0x7f21a00bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:46.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.480+0000 7f21a6397700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a006a350 con 0x7f21a0069240 2024-01-24T14:10:46.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.480+0000 7f21a6397700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a0069740 con 0x7f21a0150bd0 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.480+0000 7f21a6397700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a0069a20 con 0x7f21a01516c0 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.480+0000 7f219ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f21a0150bd0 0x7f21a0065e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.480+0000 7f219ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f21a0150bd0 0x7f21a0065e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55952/0 (socket says 172.21.15.29:55952) 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.480+0000 7f219ffff700 1 -- 172.21.15.29:0/1214725200 learned_addr learned my addr 172.21.15.29:0/1214725200 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.480+0000 7f21a4934700 1 --2- 172.21.15.29:0/1214725200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21a01516c0 0x7f21a00bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.480+0000 7f219ffff700 1 -- 172.21.15.29:0/1214725200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21a01516c0 msgr2=0x7f21a00bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.480+0000 7f219ffff700 1 --2- 172.21.15.29:0/1214725200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21a01516c0 0x7f21a00bcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.480+0000 7f219ffff700 1 -- 172.21.15.29:0/1214725200 >> v1:172.21.15.163:6789/0 conn(0x7f21a0069240 legacy=0x7f21a0150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.481+0000 7f219ffff700 1 -- 172.21.15.29:0/1214725200 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a0069d00 con 0x7f21a0150bd0 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.481+0000 7f21a4934700 1 --2- 172.21.15.29:0/1214725200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21a01516c0 0x7f21a00bcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.481+0000 7f219ffff700 1 --2- 172.21.15.29:0/1214725200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a0150bd0 0x7f21a0065e50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f2190006fd0 tx=0x7f219000b3b0).ready entity=mon.0 client_cookie=1cefedc6a407efdc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.481+0000 7f219effd700 1 -- 172.21.15.29:0/1214725200 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219000db00 con 0x7f21a0150bd0 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.481+0000 7f219effd700 1 -- 172.21.15.29:0/1214725200 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219000dc60 con 0x7f21a0150bd0 2024-01-24T14:10:46.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.481+0000 7f219effd700 1 -- 172.21.15.29:0/1214725200 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2190010470 con 0x7f21a0150bd0 2024-01-24T14:10:46.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.481+0000 7f21a6397700 1 -- 172.21.15.29:0/1214725200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a0150bd0 msgr2=0x7f21a0065e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:46.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.481+0000 7f21a6397700 1 --2- 172.21.15.29:0/1214725200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a0150bd0 0x7f21a0065e50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f2190006fd0 tx=0x7f219000b3b0).stop 2024-01-24T14:10:46.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.482+0000 7f21a6397700 1 -- 172.21.15.29:0/1214725200 shutdown_connections 2024-01-24T14:10:46.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.482+0000 7f21a6397700 1 --2- 172.21.15.29:0/1214725200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21a01516c0 0x7f21a00bcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.482+0000 7f21a6397700 1 --2- 172.21.15.29:0/1214725200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a0150bd0 0x7f21a0065e50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.482+0000 7f21a6397700 1 -- 172.21.15.29:0/1214725200 >> 172.21.15.29:0/1214725200 conn(0x7f21a00c8580 msgr2=0x7f21a00c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:46.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.482+0000 7f21a6397700 1 -- 172.21.15.29:0/1214725200 shutdown_connections 2024-01-24T14:10:46.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.482+0000 7f21a6397700 1 -- 172.21.15.29:0/1214725200 wait complete. 2024-01-24T14:10:46.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.483+0000 7f21a6397700 1 Processor -- start 2024-01-24T14:10:46.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.484+0000 7f21a6397700 1 -- start start 2024-01-24T14:10:46.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.484+0000 7f21a6397700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a0069240 0x7f21a00bf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:46.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f21a6397700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21a0150bd0 0x7f21a014ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:46.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f21a6397700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21a01516c0 0x7f21a00c0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:46.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f21a6397700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a014b200 con 0x7f21a0069240 2024-01-24T14:10:46.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f21a6397700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00aaf20 con 0x7f21a01516c0 2024-01-24T14:10:46.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f21a6397700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00c1760 con 0x7f21a0150bd0 2024-01-24T14:10:46.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f219ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a0069240 0x7f21a00bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:46.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f219ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a0069240 0x7f21a00bf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55966/0 (socket says 172.21.15.29:55966) 2024-01-24T14:10:46.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f219ffff700 1 -- 172.21.15.29:0/3416204625 learned_addr learned my addr 172.21.15.29:0/3416204625 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f21a4934700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21a01516c0 0x7f21a00c0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f219f7fe700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21a0150bd0 0x7f21a014ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f219ffff700 1 -- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21a0150bd0 msgr2=0x7f21a014ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f219ffff700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21a0150bd0 0x7f21a014ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f219ffff700 1 -- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21a01516c0 msgr2=0x7f21a00c0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f219ffff700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21a01516c0 0x7f21a00c0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f219ffff700 1 -- 172.21.15.29:0/3416204625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219000b610 con 0x7f21a0069240 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f21a4934700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21a01516c0 0x7f21a00c0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f219ffff700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a0069240 0x7f21a00bf750 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f219000db80 tx=0x7f219000b3b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f2187fff700 1 -- 172.21.15.29:0/3416204625 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2190020070 con 0x7f21a0069240 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f2187fff700 1 -- 172.21.15.29:0/3416204625 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219001b540 con 0x7f21a0069240 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f21a6397700 1 -- 172.21.15.29:0/3416204625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a00c0df0 con 0x7f21a0069240 2024-01-24T14:10:46.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.485+0000 7f2187fff700 1 -- 172.21.15.29:0/3416204625 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219001b900 con 0x7f21a0069240 2024-01-24T14:10:46.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.486+0000 7f21a6397700 1 -- 172.21.15.29:0/3416204625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a00c1480 con 0x7f21a0069240 2024-01-24T14:10:46.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.487+0000 7f21a6397700 1 -- 172.21.15.29:0/3416204625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a00475e0 con 0x7f21a0069240 2024-01-24T14:10:46.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.487+0000 7f2187fff700 1 -- 172.21.15.29:0/3416204625 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 12) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f219001bc70 con 0x7f21a0069240 2024-01-24T14:10:46.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.488+0000 7f2187fff700 1 -- 172.21.15.29:0/3416204625 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7573+0+0 (secure 0 0 0) 0x7f219007e0f0 con 0x7f21a0069240 2024-01-24T14:10:46.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.492+0000 7f2187fff700 1 -- 172.21.15.29:0/3416204625 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f219004efe0 con 0x7f21a0069240 2024-01-24T14:10:46.496 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:46.496+0000 7fdadf88b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:10:46.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:46.719+0000 7f21a6397700 1 -- 172.21.15.29:0/3416204625 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f21a00c5140 con 0x7f21a0069240 2024-01-24T14:10:46.833 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:46.833+0000 7fdadf88b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:10:46.914 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:10:46.914+0000 7fdadf88b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:10:47.164 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-01-24T14:10:47.164+0000 7fe09bfc3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-24T14:10:47.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.170+0000 7f2187fff700 1 -- 172.21.15.29:0/3416204625 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 13) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f219007bae0 con 0x7f21a0069240 2024-01-24T14:10:47.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.170+0000 7f2187fff700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f218c0302f0 0x7f218c068a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:47.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.172+0000 7f219f7fe700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f218c0302f0 0x7f218c068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:47.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.173+0000 7f219f7fe700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f218c0302f0 0x7f218c068a20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f21a00c1ab0 tx=0x7f2194008040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:47.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.176+0000 7f2187fff700 1 -- 172.21.15.29:0/3416204625 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2190040f40 con 0x7f21a0069240 2024-01-24T14:10:47.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.179+0000 7f21a6397700 1 -- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f218c0302f0 msgr2=0x7f218c068a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:47.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.179+0000 7f21a6397700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f218c0302f0 0x7f218c068a20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f21a00c1ab0 tx=0x7f2194008040).stop 2024-01-24T14:10:47.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.179+0000 7f21a6397700 1 -- 172.21.15.29:0/3416204625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a0069240 msgr2=0x7f21a00bf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:47.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.179+0000 7f21a6397700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a0069240 0x7f21a00bf750 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f219000db80 tx=0x7f219000b3b0).stop 2024-01-24T14:10:47.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.179+0000 7f21a6397700 1 -- 172.21.15.29:0/3416204625 shutdown_connections 2024-01-24T14:10:47.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.179+0000 7f21a6397700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21a01516c0 0x7f21a00c0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:47.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.179+0000 7f21a6397700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f218c0302f0 0x7f218c068a20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:47.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.179+0000 7f21a6397700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a0069240 0x7f21a00bf750 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:47.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.179+0000 7f21a6397700 1 --2- 172.21.15.29:0/3416204625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21a0150bd0 0x7f21a014ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:47.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.179+0000 7f21a6397700 1 -- 172.21.15.29:0/3416204625 >> 172.21.15.29:0/3416204625 conn(0x7f21a00c8580 msgr2=0x7f21a00638e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:47.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.179+0000 7f21a6397700 1 -- 172.21.15.29:0/3416204625 shutdown_connections 2024-01-24T14:10:47.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.179+0000 7f21a6397700 1 -- 172.21.15.29:0/3416204625 wait complete. 2024-01-24T14:10:47.181 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-24T14:10:47.195 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-24T14:10:47.214 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T14:10:47.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.346+0000 7f095b1f9700 1 Processor -- start 2024-01-24T14:10:47.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.347+0000 7f095b1f9700 1 -- start start 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.347+0000 7f095b1f9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0954066520 0x7f0954066960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.347+0000 7f095b1f9700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09540be080 con 0x7f09540be510 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.347+0000 7f095b1f9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0954067380 con 0x7f0954157530 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.347+0000 7f095b1f9700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0954065040 con 0x7f0954066520 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.348+0000 7f0953fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0954066520 0x7f0954066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.348+0000 7f0959796700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f09540be510 0x7f09540be960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48212/0 (socket says 172.21.15.29:48212) 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.348+0000 7f0959796700 1 -- 172.21.15.29:0/1928832592 learned_addr learned my addr 172.21.15.29:0/1928832592 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.348+0000 7f0953fff700 1 -- 172.21.15.29:0/1928832592 >> v1:172.21.15.163:6789/0 conn(0x7f0954157530 legacy=0x7f0954068470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.348+0000 7f0953fff700 1 -- 172.21.15.29:0/1928832592 >> v1:172.21.15.29:6789/0 conn(0x7f09540be510 legacy=0x7f09540be960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.348+0000 7f0953fff700 1 -- 172.21.15.29:0/1928832592 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0954068b80 con 0x7f0954066520 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.348+0000 7f09537fe700 1 -- 172.21.15.29:0/1928832592 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709237901 0 0) 0x7f09540be080 con 0x7f09540be510 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.348+0000 7f0953fff700 1 --2- 172.21.15.29:0/1928832592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0954066520 0x7f0954066960 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f0940009760 tx=0x7f0940009060).ready entity=mon.0 client_cookie=bbb9f8716de6b30f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:47.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.348+0000 7f09537fe700 1 -- 172.21.15.29:0/1928832592 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094000b580 con 0x7f0954066520 2024-01-24T14:10:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.348+0000 7f09537fe700 1 -- 172.21.15.29:0/1928832592 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094000b6e0 con 0x7f0954066520 2024-01-24T14:10:47.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.348+0000 7f09537fe700 1 -- 172.21.15.29:0/1928832592 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094000ba60 con 0x7f0954066520 2024-01-24T14:10:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.349+0000 7f095b1f9700 1 -- 172.21.15.29:0/1928832592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0954066520 msgr2=0x7f0954066960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.349+0000 7f095b1f9700 1 --2- 172.21.15.29:0/1928832592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0954066520 0x7f0954066960 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f0940009760 tx=0x7f0940009060).stop 2024-01-24T14:10:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.349+0000 7f095b1f9700 1 -- 172.21.15.29:0/1928832592 shutdown_connections 2024-01-24T14:10:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.349+0000 7f095b1f9700 1 --2- 172.21.15.29:0/1928832592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0954066520 0x7f0954066960 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.349+0000 7f095b1f9700 1 -- 172.21.15.29:0/1928832592 >> 172.21.15.29:0/1928832592 conn(0x7f09540c8ba0 msgr2=0x7f09540c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.349+0000 7f095b1f9700 1 -- 172.21.15.29:0/1928832592 shutdown_connections 2024-01-24T14:10:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.349+0000 7f095b1f9700 1 -- 172.21.15.29:0/1928832592 wait complete. 2024-01-24T14:10:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.350+0000 7f095b1f9700 1 Processor -- start 2024-01-24T14:10:47.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.351+0000 7f095b1f9700 1 -- start start 2024-01-24T14:10:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.351+0000 7f095b1f9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0954066520 0x7f095414e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.351+0000 7f095b1f9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09540be510 0x7f09540a8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:47.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.351+0000 7f095b1f9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0954157530 0x7f09540a9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.351+0000 7f095b1f9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540aa2a0 con 0x7f0954066520 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.351+0000 7f095b1f9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540a9690 con 0x7f09540be510 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.351+0000 7f095b1f9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09540a9970 con 0x7f0954157530 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.351+0000 7f0958f95700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0954066520 0x7f095414e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0958f95700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0954066520 0x7f095414e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55996/0 (socket says 172.21.15.29:55996) 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0958f95700 1 -- 172.21.15.29:0/3922985607 learned_addr learned my addr 172.21.15.29:0/3922985607 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0953fff700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09540be510 0x7f09540a8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0959796700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0954157530 0x7f09540a9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0958f95700 1 -- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0954157530 msgr2=0x7f09540a9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0958f95700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0954157530 0x7f09540a9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0958f95700 1 -- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09540be510 msgr2=0x7f09540a8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0958f95700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09540be510 0x7f09540a8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:47.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0958f95700 1 -- 172.21.15.29:0/3922985607 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09400092a0 con 0x7f0954066520 2024-01-24T14:10:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0958f95700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0954066520 0x7f095414e670 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f094800c440 tx=0x7f0948007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0950ff9700 1 -- 172.21.15.29:0/3922985607 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094800ce60 con 0x7f0954066520 2024-01-24T14:10:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0950ff9700 1 -- 172.21.15.29:0/3922985607 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0948007020 con 0x7f0954066520 2024-01-24T14:10:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f0950ff9700 1 -- 172.21.15.29:0/3922985607 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09480073c0 con 0x7f0954066520 2024-01-24T14:10:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f095b1f9700 1 -- 172.21.15.29:0/3922985607 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09540a9c50 con 0x7f0954066520 2024-01-24T14:10:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.352+0000 7f095b1f9700 1 -- 172.21.15.29:0/3922985607 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0954168740 con 0x7f0954066520 2024-01-24T14:10:47.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.353+0000 7f095b1f9700 1 -- 172.21.15.29:0/3922985607 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09540475e0 con 0x7f0954066520 2024-01-24T14:10:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.354+0000 7f0950ff9700 1 -- 172.21.15.29:0/3922985607 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 13) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f09480068b0 con 0x7f0954066520 2024-01-24T14:10:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.354+0000 7f0950ff9700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f093805f030 0x7f09380614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.354+0000 7f0950ff9700 1 -- 172.21.15.29:0/3922985607 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7659+0+0 (secure 0 0 0) 0x7f0948079a50 con 0x7f0954066520 2024-01-24T14:10:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.354+0000 7f0953fff700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f093805f030 0x7f09380614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:47.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.355+0000 7f0953fff700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f093805f030 0x7f09380614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f094000b340 tx=0x7f0940017000).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:47.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.358+0000 7f0950ff9700 1 -- 172.21.15.29:0/3922985607 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0948047b00 con 0x7f0954066520 2024-01-24T14:10:47.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:47.586+0000 7f095b1f9700 1 -- 172.21.15.29:0/3922985607 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0954156d80 con 0x7f0954066520 2024-01-24T14:10:48.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.182+0000 7f0950ff9700 1 -- 172.21.15.29:0/3922985607 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f094804a7e0 con 0x7f0954066520 2024-01-24T14:10:48.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.185+0000 7f095b1f9700 1 -- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f093805f030 msgr2=0x7f09380614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:48.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.185+0000 7f095b1f9700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f093805f030 0x7f09380614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f094000b340 tx=0x7f0940017000).stop 2024-01-24T14:10:48.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.185+0000 7f095b1f9700 1 -- 172.21.15.29:0/3922985607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0954066520 msgr2=0x7f095414e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:48.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.185+0000 7f095b1f9700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0954066520 0x7f095414e670 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f094800c440 tx=0x7f0948007e00).stop 2024-01-24T14:10:48.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.185+0000 7f095b1f9700 1 -- 172.21.15.29:0/3922985607 shutdown_connections 2024-01-24T14:10:48.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.185+0000 7f095b1f9700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0954157530 0x7f09540a9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:48.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.185+0000 7f095b1f9700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f093805f030 0x7f09380614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:48.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.185+0000 7f095b1f9700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0954066520 0x7f095414e670 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:48.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.185+0000 7f095b1f9700 1 --2- 172.21.15.29:0/3922985607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09540be510 0x7f09540a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:48.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.185+0000 7f095b1f9700 1 -- 172.21.15.29:0/3922985607 >> 172.21.15.29:0/3922985607 conn(0x7f09540c8ba0 msgr2=0x7f09540c7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:48.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.186+0000 7f095b1f9700 1 -- 172.21.15.29:0/3922985607 shutdown_connections 2024-01-24T14:10:48.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.186+0000 7f095b1f9700 1 -- 172.21.15.29:0/3922985607 wait complete. 2024-01-24T14:10:48.201 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-24T14:10:48.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.339+0000 7fa7abb06700 1 Processor -- start 2024-01-24T14:10:48.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.339+0000 7fa7abb06700 1 -- start start 2024-01-24T14:10:48.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.340+0000 7fa7abb06700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa7a4150970 0x7fa7a4150d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:48.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.340+0000 7fa7abb06700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7a4069ee0 0x7fa7a4154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:48.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.340+0000 7fa7abb06700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa7a406a340 0x7fa7a4154e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:48.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.340+0000 7fa7abb06700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a4155330 con 0x7fa7a4150970 2024-01-24T14:10:48.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.340+0000 7fa7abb06700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a4069a50 con 0x7fa7a406a340 2024-01-24T14:10:48.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.340+0000 7fa7abb06700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a40c2b20 con 0x7fa7a4069ee0 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.340+0000 7fa7a98a2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa7a4150970 0x7fa7a4150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.340+0000 7fa7a98a2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa7a4150970 0x7fa7a4150d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56002/0 (socket says 172.21.15.29:56002) 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.340+0000 7fa7a98a2700 1 -- 172.21.15.29:0/80644840 learned_addr learned my addr 172.21.15.29:0/80644840 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.341+0000 7fa7aa0a3700 1 --2- 172.21.15.29:0/80644840 >> v2:172.21.15.163:3300/0 conn(0x7fa7a406a340 0x7fa7a4154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.341+0000 7fa7a90a1700 1 --2- 172.21.15.29:0/80644840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7a4069ee0 0x7fa7a4154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.341+0000 7fa7a98a2700 1 -- 172.21.15.29:0/80644840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7a4069ee0 msgr2=0x7fa7a4154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.341+0000 7fa7a98a2700 1 --2- 172.21.15.29:0/80644840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7a4069ee0 0x7fa7a4154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.341+0000 7fa7a98a2700 1 -- 172.21.15.29:0/80644840 >> v2:172.21.15.163:3300/0 conn(0x7fa7a406a340 msgr2=0x7fa7a4154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.341+0000 7fa7a98a2700 1 --2- 172.21.15.29:0/80644840 >> v2:172.21.15.163:3300/0 conn(0x7fa7a406a340 0x7fa7a4154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.341+0000 7fa7a98a2700 1 -- 172.21.15.29:0/80644840 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a4151240 con 0x7fa7a4150970 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.341+0000 7fa7aa0a3700 1 --2- 172.21.15.29:0/80644840 >> v2:172.21.15.163:3300/0 conn(0x7fa7a406a340 0x7fa7a4154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.341+0000 7fa7a98a2700 1 --2- 172.21.15.29:0/80644840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a4150970 0x7fa7a4150d50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fa7a0006fd0 tx=0x7fa7a000b3b0).ready entity=mon.0 client_cookie=a8a7d5afa2df4d6b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.341+0000 7fa7a88a0700 1 -- 172.21.15.29:0/80644840 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a000db00 con 0x7fa7a4150970 2024-01-24T14:10:48.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.341+0000 7fa7a88a0700 1 -- 172.21.15.29:0/80644840 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a000dc60 con 0x7fa7a4150970 2024-01-24T14:10:48.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.341+0000 7fa7a88a0700 1 -- 172.21.15.29:0/80644840 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0010470 con 0x7fa7a4150970 2024-01-24T14:10:48.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.342+0000 7fa7abb06700 1 -- 172.21.15.29:0/80644840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a4150970 msgr2=0x7fa7a4150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:48.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.342+0000 7fa7abb06700 1 --2- 172.21.15.29:0/80644840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a4150970 0x7fa7a4150d50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fa7a0006fd0 tx=0x7fa7a000b3b0).stop 2024-01-24T14:10:48.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.342+0000 7fa7abb06700 1 -- 172.21.15.29:0/80644840 shutdown_connections 2024-01-24T14:10:48.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.342+0000 7fa7abb06700 1 --2- 172.21.15.29:0/80644840 >> v2:172.21.15.163:3300/0 conn(0x7fa7a406a340 0x7fa7a4154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:48.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.342+0000 7fa7abb06700 1 --2- 172.21.15.29:0/80644840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a4150970 0x7fa7a4150d50 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:48.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.342+0000 7fa7abb06700 1 --2- 172.21.15.29:0/80644840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7a4069ee0 0x7fa7a4154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:48.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.342+0000 7fa7abb06700 1 -- 172.21.15.29:0/80644840 >> 172.21.15.29:0/80644840 conn(0x7fa7a4067b20 msgr2=0x7fa7a40638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:48.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.342+0000 7fa7abb06700 1 -- 172.21.15.29:0/80644840 shutdown_connections 2024-01-24T14:10:48.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.342+0000 7fa7abb06700 1 -- 172.21.15.29:0/80644840 wait complete. 2024-01-24T14:10:48.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.343+0000 7fa7abb06700 1 Processor -- start 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.343+0000 7fa7abb06700 1 -- start start 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7abb06700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a4069ee0 0x7fa7a4151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7abb06700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7a406a340 0x7fa7a40a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7abb06700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7a4150970 0x7fa7a40a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7abb06700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4151e00 con 0x7fa7a4069ee0 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7abb06700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4152c20 con 0x7fa7a4150970 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7abb06700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a40aa210 con 0x7fa7a406a340 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7a98a2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a4069ee0 0x7fa7a4151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7a98a2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a4069ee0 0x7fa7a4151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56006/0 (socket says 172.21.15.29:56006) 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7a98a2700 1 -- 172.21.15.29:0/1718751988 learned_addr learned my addr 172.21.15.29:0/1718751988 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7a90a1700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7a406a340 0x7fa7a40a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7aa0a3700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7a4150970 0x7fa7a40a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7a98a2700 1 -- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7a406a340 msgr2=0x7fa7a40a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.344+0000 7fa7a98a2700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7a406a340 0x7fa7a40a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:48.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.345+0000 7fa7a98a2700 1 -- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7a4150970 msgr2=0x7fa7a40a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:48.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.345+0000 7fa7a98a2700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7a4150970 0x7fa7a40a6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:48.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.345+0000 7fa7a98a2700 1 -- 172.21.15.29:0/1718751988 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a000b610 con 0x7fa7a4069ee0 2024-01-24T14:10:48.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.345+0000 7fa7a98a2700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a4069ee0 0x7fa7a4151910 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fa7a000d5d0 tx=0x7fa7a40a7dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:48.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.345+0000 7fa795ffb700 1 -- 172.21.15.29:0/1718751988 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a001f070 con 0x7fa7a4069ee0 2024-01-24T14:10:48.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.345+0000 7fa795ffb700 1 -- 172.21.15.29:0/1718751988 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a0000c90 con 0x7fa7a4069ee0 2024-01-24T14:10:48.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.345+0000 7fa7abb06700 1 -- 172.21.15.29:0/1718751988 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a4152010 con 0x7fa7a4069ee0 2024-01-24T14:10:48.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.345+0000 7fa795ffb700 1 -- 172.21.15.29:0/1718751988 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a00198b0 con 0x7fa7a4069ee0 2024-01-24T14:10:48.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.345+0000 7fa7abb06700 1 -- 172.21.15.29:0/1718751988 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a41525d0 con 0x7fa7a4069ee0 2024-01-24T14:10:48.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.346+0000 7fa7abb06700 1 -- 172.21.15.29:0/1718751988 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a40475e0 con 0x7fa7a4069ee0 2024-01-24T14:10:48.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.347+0000 7fa795ffb700 1 -- 172.21.15.29:0/1718751988 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 13) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fa7a0019c20 con 0x7fa7a4069ee0 2024-01-24T14:10:48.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.347+0000 7fa795ffb700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa78805f030 0x7fa7880614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:48.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.347+0000 7fa795ffb700 1 -- 172.21.15.29:0/1718751988 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7659+0+0 (secure 0 0 0) 0x7fa7a007dff0 con 0x7fa7a4069ee0 2024-01-24T14:10:48.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.347+0000 7fa7a90a1700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa78805f030 0x7fa7880614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:48.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.348+0000 7fa7a90a1700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa78805f030 0x7fa7880614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa7a40c2af0 tx=0x7fa798008040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:48.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.351+0000 7fa795ffb700 1 -- 172.21.15.29:0/1718751988 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa7a004ee90 con 0x7fa7a4069ee0 2024-01-24T14:10:48.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.532+0000 7fa7abb06700 1 -- 172.21.15.29:0/1718751988 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa7a4155510 con 0x7fa7a4069ee0 2024-01-24T14:10:48.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:48.906+0000 7fa795ffb700 1 -- 172.21.15.29:0/1718751988 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa7a007b990 con 0x7fa7a4069ee0 2024-01-24T14:10:49.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.189+0000 7fa795ffb700 1 -- 172.21.15.29:0/1718751988 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v62) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa7a00436d0 con 0x7fa7a4069ee0 2024-01-24T14:10:49.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.193+0000 7fa7abb06700 1 -- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa78805f030 msgr2=0x7fa7880614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:49.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.193+0000 7fa7abb06700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa78805f030 0x7fa7880614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa7a40c2af0 tx=0x7fa798008040).stop 2024-01-24T14:10:49.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.193+0000 7fa7abb06700 1 -- 172.21.15.29:0/1718751988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a4069ee0 msgr2=0x7fa7a4151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:49.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.193+0000 7fa7abb06700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a4069ee0 0x7fa7a4151910 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fa7a000d5d0 tx=0x7fa7a40a7dc0).stop 2024-01-24T14:10:49.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.194+0000 7fa7abb06700 1 -- 172.21.15.29:0/1718751988 shutdown_connections 2024-01-24T14:10:49.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.194+0000 7fa7abb06700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7a4150970 0x7fa7a40a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:49.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.194+0000 7fa7abb06700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa78805f030 0x7fa7880614f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:49.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.194+0000 7fa7abb06700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a4069ee0 0x7fa7a4151910 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:49.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.194+0000 7fa7abb06700 1 --2- 172.21.15.29:0/1718751988 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7a406a340 0x7fa7a40a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:49.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.194+0000 7fa7abb06700 1 -- 172.21.15.29:0/1718751988 >> 172.21.15.29:0/1718751988 conn(0x7fa7a4067b20 msgr2=0x7fa7a40ad4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:49.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.194+0000 7fa7abb06700 1 -- 172.21.15.29:0/1718751988 shutdown_connections 2024-01-24T14:10:49.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.194+0000 7fa7abb06700 1 -- 172.21.15.29:0/1718751988 wait complete. 2024-01-24T14:10:49.196 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-01-24T14:10:49.212 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-24T14:10:49.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.350+0000 7fbabf144700 1 Processor -- start 2024-01-24T14:10:49.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.350+0000 7fbabf144700 1 -- start start 2024-01-24T14:10:49.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.351+0000 7fbabf144700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbab81552a0 0x7fbab8066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:49.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.351+0000 7fbabf144700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab8068540 con 0x7fbab8066940 2024-01-24T14:10:49.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.351+0000 7fbabf144700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab80689d0 con 0x7fbab8154ec0 2024-01-24T14:10:49.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.351+0000 7fbabf144700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab80641c0 con 0x7fbab81552a0 2024-01-24T14:10:49.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.351+0000 7fbabd6e1700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbab8066940 0x7fbab8151350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48228/0 (socket says 172.21.15.29:48228) 2024-01-24T14:10:49.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.351+0000 7fbabd6e1700 1 -- 172.21.15.29:0/2430187340 learned_addr learned my addr 172.21.15.29:0/2430187340 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:49.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.351+0000 7fbab7fff700 1 --2- 172.21.15.29:0/2430187340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbab81552a0 0x7fbab8066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.351+0000 7fbab77fe700 1 -- 172.21.15.29:0/2430187340 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276216390 0 0) 0x7fbab8068540 con 0x7fbab8066940 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.351+0000 7fbab77fe700 1 -- 172.21.15.29:0/2430187340 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaa80033f0 con 0x7fbab8066940 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.351+0000 7fbab77fe700 1 -- 172.21.15.29:0/2430187340 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667754681 0 0) 0x7fbab80689d0 con 0x7fbab8154ec0 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.352+0000 7fbab77fe700 1 -- 172.21.15.29:0/2430187340 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbab8068540 con 0x7fbab8154ec0 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.352+0000 7fbab7fff700 1 -- 172.21.15.29:0/2430187340 >> v1:172.21.15.163:6789/0 conn(0x7fbab8154ec0 legacy=0x7fbab80638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.352+0000 7fbab7fff700 1 -- 172.21.15.29:0/2430187340 >> v1:172.21.15.29:6789/0 conn(0x7fbab8066940 legacy=0x7fbab8151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.352+0000 7fbab77fe700 1 -- 172.21.15.29:0/2430187340 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3231694256 0 0) 0x7fbaa80033f0 con 0x7fbab8066940 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.352+0000 7fbab7fff700 1 -- 172.21.15.29:0/2430187340 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab8066da0 con 0x7fbab81552a0 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.352+0000 7fbab7fff700 1 --2- 172.21.15.29:0/2430187340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbab81552a0 0x7fbab8066450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fbaa4009790 tx=0x7fbaa4009060).ready entity=mon.2 client_cookie=ed550a181ec2f884 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.352+0000 7fbab77fe700 1 -- 172.21.15.29:0/2430187340 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab00017f0 con 0x7fbab81552a0 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.352+0000 7fbab77fe700 1 -- 172.21.15.29:0/2430187340 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa4011400 con 0x7fbab81552a0 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.352+0000 7fbab77fe700 1 -- 172.21.15.29:0/2430187340 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4011780 con 0x7fbab81552a0 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.352+0000 7fbabf144700 1 -- 172.21.15.29:0/2430187340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbab81552a0 msgr2=0x7fbab8066450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:49.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.353+0000 7fbabf144700 1 --2- 172.21.15.29:0/2430187340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbab81552a0 0x7fbab8066450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fbaa4009790 tx=0x7fbaa4009060).stop 2024-01-24T14:10:49.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.353+0000 7fbabf144700 1 -- 172.21.15.29:0/2430187340 shutdown_connections 2024-01-24T14:10:49.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.353+0000 7fbabf144700 1 --2- 172.21.15.29:0/2430187340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbab81552a0 0x7fbab8066450 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:49.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.353+0000 7fbabf144700 1 -- 172.21.15.29:0/2430187340 >> 172.21.15.29:0/2430187340 conn(0x7fbab80c8bd0 msgr2=0x7fbab80c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:49.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.353+0000 7fbabf144700 1 -- 172.21.15.29:0/2430187340 shutdown_connections 2024-01-24T14:10:49.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.353+0000 7fbabf144700 1 -- 172.21.15.29:0/2430187340 wait complete. 2024-01-24T14:10:49.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.354+0000 7fbabf144700 1 Processor -- start 2024-01-24T14:10:49.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.355+0000 7fbabf144700 1 -- start start 2024-01-24T14:10:49.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.355+0000 7fbabf144700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbab8066940 0x7fbab81650f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:49.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbabf144700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbab8154ec0 0x7fbab813a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbabf144700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbab81552a0 0x7fbab80a86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbabf144700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab81655e0 con 0x7fbab8154ec0 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbabf144700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab80a9c70 con 0x7fbab8066940 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbabf144700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab80a9f50 con 0x7fbab81552a0 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbab7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbab8154ec0 0x7fbab813a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbab7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbab8154ec0 0x7fbab813a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56020/0 (socket says 172.21.15.29:56020) 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbab7fff700 1 -- 172.21.15.29:0/3652678411 learned_addr learned my addr 172.21.15.29:0/3652678411 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbabd6e1700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbab81552a0 0x7fbab80a86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbabcee0700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbab8066940 0x7fbab81650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbab7fff700 1 -- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbab81552a0 msgr2=0x7fbab80a86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbab7fff700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbab81552a0 0x7fbab80a86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbab7fff700 1 -- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbab8066940 msgr2=0x7fbab81650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbab7fff700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbab8066940 0x7fbab81650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbab7fff700 1 -- 172.21.15.29:0/3652678411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa40092a0 con 0x7fbab8154ec0 2024-01-24T14:10:49.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbab7fff700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbab8154ec0 0x7fbab813a420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fbab806a130 tx=0x7fbaa4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:49.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbab4ff9700 1 -- 172.21.15.29:0/3652678411 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4010920 con 0x7fbab8154ec0 2024-01-24T14:10:49.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbab4ff9700 1 -- 172.21.15.29:0/3652678411 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa4010a80 con 0x7fbab8154ec0 2024-01-24T14:10:49.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbabf144700 1 -- 172.21.15.29:0/3652678411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab80aa230 con 0x7fbab8154ec0 2024-01-24T14:10:49.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.356+0000 7fbab4ff9700 1 -- 172.21.15.29:0/3652678411 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4010e20 con 0x7fbab8154ec0 2024-01-24T14:10:49.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.357+0000 7fbabf144700 1 -- 172.21.15.29:0/3652678411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab80aa7f0 con 0x7fbab8154ec0 2024-01-24T14:10:49.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.358+0000 7fbabf144700 1 -- 172.21.15.29:0/3652678411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab80aa9e0 con 0x7fbab8154ec0 2024-01-24T14:10:49.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.358+0000 7fbab4ff9700 1 -- 172.21.15.29:0/3652678411 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbaa401d640 con 0x7fbab8154ec0 2024-01-24T14:10:49.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.359+0000 7fbab4ff9700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fba9c05f270 0x7fba9c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:49.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.359+0000 7fbab4ff9700 1 -- 172.21.15.29:0/3652678411 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7192+0+0 (secure 0 0 0) 0x7fbaa407bfc0 con 0x7fbab8154ec0 2024-01-24T14:10:49.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.359+0000 7fbabcee0700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fba9c05f270 0x7fba9c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:49.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.359+0000 7fbabcee0700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fba9c05f270 0x7fba9c061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fbab80c3070 tx=0x7fbaac009040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:49.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.363+0000 7fbab4ff9700 1 -- 172.21.15.29:0/3652678411 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbaa404cce0 con 0x7fbab8154ec0 2024-01-24T14:10:49.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:49.546+0000 7fbabf144700 1 -- 172.21.15.29:0/3652678411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbab80475e0 con 0x7fbab8154ec0 2024-01-24T14:10:50.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.193+0000 7fbab4ff9700 1 -- 172.21.15.29:0/3652678411 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v63) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbaa4050300 con 0x7fbab8154ec0 2024-01-24T14:10:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.196+0000 7fbabf144700 1 -- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fba9c05f270 msgr2=0x7fba9c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.196+0000 7fbabf144700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fba9c05f270 0x7fba9c061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fbab80c3070 tx=0x7fbaac009040).stop 2024-01-24T14:10:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.196+0000 7fbabf144700 1 -- 172.21.15.29:0/3652678411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbab8154ec0 msgr2=0x7fbab813a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.196+0000 7fbabf144700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbab8154ec0 0x7fbab813a420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fbab806a130 tx=0x7fbaa4016000).stop 2024-01-24T14:10:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.196+0000 7fbabf144700 1 -- 172.21.15.29:0/3652678411 shutdown_connections 2024-01-24T14:10:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.196+0000 7fbabf144700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbab81552a0 0x7fbab80a86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.196+0000 7fbabf144700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbab8066940 0x7fbab81650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.196+0000 7fbabf144700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fba9c05f270 0x7fba9c061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.196+0000 7fbabf144700 1 --2- 172.21.15.29:0/3652678411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbab8154ec0 0x7fbab813a420 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.196+0000 7fbabf144700 1 -- 172.21.15.29:0/3652678411 >> 172.21.15.29:0/3652678411 conn(0x7fbab80c8bd0 msgr2=0x7fbab8156670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.197+0000 7fbabf144700 1 -- 172.21.15.29:0/3652678411 shutdown_connections 2024-01-24T14:10:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.197+0000 7fbabf144700 1 -- 172.21.15.29:0/3652678411 wait complete. 2024-01-24T14:10:50.198 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-01-24T14:10:50.214 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-24T14:10:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.348+0000 7fe20445e700 1 Processor -- start 2024-01-24T14:10:50.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.348+0000 7fe20445e700 1 -- start start 2024-01-24T14:10:50.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe20445e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe1fc064740 0x7fe1fc064b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe20445e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1fc0bfd20 0x7fe1fc0c0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe20445e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1fc0c6820 con 0x7fe1fc0c5af0 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe20445e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1fc1556b0 con 0x7fe1fc064740 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe20445e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1fc155220 con 0x7fe1fc0bfd20 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe2021fa700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe1fc064740 0x7fe1fc064b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe2021fa700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe1fc064740 0x7fe1fc064b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56032/0 (socket says 172.21.15.29:56032) 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe2021fa700 1 -- 172.21.15.29:0/3487955144 learned_addr learned my addr 172.21.15.29:0/3487955144 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe2029fb700 1 --2- 172.21.15.29:0/3487955144 >> v2:172.21.15.163:3300/0 conn(0x7fe1fc0bfd20 0x7fe1fc0c0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe2021fa700 1 -- 172.21.15.29:0/3487955144 >> v2:172.21.15.163:3300/0 conn(0x7fe1fc0bfd20 msgr2=0x7fe1fc0c0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe2021fa700 1 --2- 172.21.15.29:0/3487955144 >> v2:172.21.15.163:3300/0 conn(0x7fe1fc0bfd20 0x7fe1fc0c0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe2021fa700 1 -- 172.21.15.29:0/3487955144 >> v1:172.21.15.163:6789/0 conn(0x7fe1fc0c5af0 legacy=0x7fe1fc0c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe2021fa700 1 -- 172.21.15.29:0/3487955144 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc154680 con 0x7fe1fc064740 2024-01-24T14:10:50.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe2021fa700 1 --2- 172.21.15.29:0/3487955144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc064740 0x7fe1fc064b20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fe1fc065450 tx=0x7fe1f80090f0).ready entity=mon.0 client_cookie=f30c8e1aae5c6e9a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe2011f8700 1 -- 172.21.15.29:0/3487955144 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800b5a0 con 0x7fe1fc064740 2024-01-24T14:10:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe2011f8700 1 -- 172.21.15.29:0/3487955144 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f800b700 con 0x7fe1fc064740 2024-01-24T14:10:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.350+0000 7fe2011f8700 1 -- 172.21.15.29:0/3487955144 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800ba80 con 0x7fe1fc064740 2024-01-24T14:10:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.351+0000 7fe20445e700 1 -- 172.21.15.29:0/3487955144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc064740 msgr2=0x7fe1fc064b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.351+0000 7fe20445e700 1 --2- 172.21.15.29:0/3487955144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc064740 0x7fe1fc064b20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fe1fc065450 tx=0x7fe1f80090f0).stop 2024-01-24T14:10:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.351+0000 7fe20445e700 1 -- 172.21.15.29:0/3487955144 shutdown_connections 2024-01-24T14:10:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.351+0000 7fe20445e700 1 --2- 172.21.15.29:0/3487955144 >> v2:172.21.15.163:3300/0 conn(0x7fe1fc0bfd20 0x7fe1fc0c0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.351+0000 7fe20445e700 1 --2- 172.21.15.29:0/3487955144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc064740 0x7fe1fc064b20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.351+0000 7fe20445e700 1 -- 172.21.15.29:0/3487955144 >> 172.21.15.29:0/3487955144 conn(0x7fe1fc0cb890 msgr2=0x7fe1fc0c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.351+0000 7fe20445e700 1 -- 172.21.15.29:0/3487955144 shutdown_connections 2024-01-24T14:10:50.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.351+0000 7fe20445e700 1 -- 172.21.15.29:0/3487955144 wait complete. 2024-01-24T14:10:50.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.352+0000 7fe20445e700 1 Processor -- start 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.353+0000 7fe20445e700 1 -- start start 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.353+0000 7fe20445e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc064740 0x7fe1fc0c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.353+0000 7fe20445e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1fc0bfd20 0x7fe1fc151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.353+0000 7fe20445e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1fc0c5af0 0x7fe1fc0c0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.353+0000 7fe20445e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0c4460 con 0x7fe1fc064740 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.353+0000 7fe20445e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0c3850 con 0x7fe1fc0c5af0 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.353+0000 7fe20445e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0c3b30 con 0x7fe1fc0bfd20 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.353+0000 7fe2021fa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc064740 0x7fe1fc0c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.353+0000 7fe2021fa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc064740 0x7fe1fc0c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56044/0 (socket says 172.21.15.29:56044) 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.353+0000 7fe2021fa700 1 -- 172.21.15.29:0/203380157 learned_addr learned my addr 172.21.15.29:0/203380157 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe2029fb700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1fc0c5af0 0x7fe1fc0c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe2019f9700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1fc0bfd20 0x7fe1fc151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:50.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe2021fa700 1 -- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1fc0bfd20 msgr2=0x7fe1fc151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe2021fa700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1fc0bfd20 0x7fe1fc151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe2021fa700 1 -- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1fc0c5af0 msgr2=0x7fe1fc0c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe2021fa700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1fc0c5af0 0x7fe1fc0c0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe2021fa700 1 -- 172.21.15.29:0/203380157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f8009370 con 0x7fe1fc064740 2024-01-24T14:10:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe2019f9700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1fc0bfd20 0x7fe1fc151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe2021fa700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc064740 0x7fe1fc0c48f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fe1fc062f00 tx=0x7fe1f8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe1f27fc700 1 -- 172.21.15.29:0/203380157 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800b870 con 0x7fe1fc064740 2024-01-24T14:10:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe1f27fc700 1 -- 172.21.15.29:0/203380157 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f800b9f0 con 0x7fe1fc064740 2024-01-24T14:10:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe20445e700 1 -- 172.21.15.29:0/203380157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1fc0c3e10 con 0x7fe1fc064740 2024-01-24T14:10:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe1f27fc700 1 -- 172.21.15.29:0/203380157 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f801a430 con 0x7fe1fc064740 2024-01-24T14:10:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.354+0000 7fe20445e700 1 -- 172.21.15.29:0/203380157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1fc1516f0 con 0x7fe1fc064740 2024-01-24T14:10:50.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.356+0000 7fe20445e700 1 -- 172.21.15.29:0/203380157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1fc0c4740 con 0x7fe1fc064740 2024-01-24T14:10:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.356+0000 7fe1f27fc700 1 -- 172.21.15.29:0/203380157 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe1f8021030 con 0x7fe1fc064740 2024-01-24T14:10:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.356+0000 7fe1f27fc700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe1e005f270 0x7fe1e0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.356+0000 7fe1f27fc700 1 -- 172.21.15.29:0/203380157 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6720+0+0 (secure 0 0 0) 0x7fe1f807bb70 con 0x7fe1fc064740 2024-01-24T14:10:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.356+0000 7fe2019f9700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe1e005f270 0x7fe1e0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:50.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.357+0000 7fe2019f9700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe1e005f270 0x7fe1e0061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fe1ec006fd0 tx=0x7fe1ec008040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:50.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.361+0000 7fe1f27fc700 1 -- 172.21.15.29:0/203380157 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe1f804ca70 con 0x7fe1fc064740 2024-01-24T14:10:50.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.544+0000 7fe20445e700 1 -- 172.21.15.29:0/203380157 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe1fc0c6820 con 0x7fe1fc064740 2024-01-24T14:10:50.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.548+0000 7fe1f27fc700 1 -- 172.21.15.29:0/203380157 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v4) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe1fc0c6820 con 0x7fe1fc064740 2024-01-24T14:10:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.550+0000 7fe20445e700 1 -- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe1e005f270 msgr2=0x7fe1e0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.550+0000 7fe20445e700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe1e005f270 0x7fe1e0061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fe1ec006fd0 tx=0x7fe1ec008040).stop 2024-01-24T14:10:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.550+0000 7fe20445e700 1 -- 172.21.15.29:0/203380157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc064740 msgr2=0x7fe1fc0c48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.550+0000 7fe20445e700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc064740 0x7fe1fc0c48f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fe1fc062f00 tx=0x7fe1f8015000).stop 2024-01-24T14:10:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.550+0000 7fe20445e700 1 -- 172.21.15.29:0/203380157 shutdown_connections 2024-01-24T14:10:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.550+0000 7fe20445e700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1fc0c5af0 0x7fe1fc0c0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.550+0000 7fe20445e700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe1e005f270 0x7fe1e0061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.550+0000 7fe20445e700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc064740 0x7fe1fc0c48f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.550+0000 7fe20445e700 1 --2- 172.21.15.29:0/203380157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1fc0bfd20 0x7fe1fc151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.550+0000 7fe20445e700 1 -- 172.21.15.29:0/203380157 >> 172.21.15.29:0/203380157 conn(0x7fe1fc0cb890 msgr2=0x7fe1fc0c8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:50.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.550+0000 7fe20445e700 1 -- 172.21.15.29:0/203380157 shutdown_connections 2024-01-24T14:10:50.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.550+0000 7fe20445e700 1 -- 172.21.15.29:0/203380157 wait complete. 2024-01-24T14:10:50.564 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-01-24T14:10:50.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.696+0000 7f9a2c696700 1 Processor -- start 2024-01-24T14:10:50.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.696+0000 7f9a2c696700 1 -- start start 2024-01-24T14:10:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.697+0000 7f9a2c696700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a24068900 0x7f9a240c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.697+0000 7f9a2c696700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a24068470 con 0x7f9a24066c30 2024-01-24T14:10:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.697+0000 7f9a2c696700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a240c3860 con 0x7f9a24134e70 2024-01-24T14:10:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.697+0000 7f9a2c696700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2414c910 con 0x7f9a24068900 2024-01-24T14:10:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.698+0000 7f9a29c31700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9a24066c30 0x7f9a240bdb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48230/0 (socket says 172.21.15.29:48230) 2024-01-24T14:10:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.698+0000 7f9a29c31700 1 -- 172.21.15.29:0/967847429 learned_addr learned my addr 172.21.15.29:0/967847429 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.698+0000 7f9a2ac33700 1 --2- 172.21.15.29:0/967847429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a24068900 0x7f9a240c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.698+0000 7f9a29430700 1 -- 172.21.15.29:0/967847429 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764345659 0 0) 0x7f9a24068470 con 0x7f9a24066c30 2024-01-24T14:10:50.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.698+0000 7f9a29430700 1 -- 172.21.15.29:0/967847429 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a100033f0 con 0x7f9a24066c30 2024-01-24T14:10:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.698+0000 7f9a29430700 1 -- 172.21.15.29:0/967847429 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594132964 0 0) 0x7f9a240c3860 con 0x7f9a24134e70 2024-01-24T14:10:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.698+0000 7f9a29430700 1 -- 172.21.15.29:0/967847429 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a24068470 con 0x7f9a24134e70 2024-01-24T14:10:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.698+0000 7f9a29430700 1 -- 172.21.15.29:0/967847429 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3531485073 0 0) 0x7f9a100033f0 con 0x7f9a24066c30 2024-01-24T14:10:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.698+0000 7f9a2ac33700 1 -- 172.21.15.29:0/967847429 >> v1:172.21.15.163:6789/0 conn(0x7f9a24134e70 legacy=0x7f9a24066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.698+0000 7f9a2ac33700 1 -- 172.21.15.29:0/967847429 >> v1:172.21.15.29:6789/0 conn(0x7f9a24066c30 legacy=0x7f9a240bdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.698+0000 7f9a2ac33700 1 -- 172.21.15.29:0/967847429 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a24067b40 con 0x7f9a24068900 2024-01-24T14:10:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.698+0000 7f9a2ac33700 1 --2- 172.21.15.29:0/967847429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a24068900 0x7f9a240c31a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f9a1c009790 tx=0x7f9a1c009060).ready entity=mon.2 client_cookie=8081f9b028598c3c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.699+0000 7f9a29430700 1 -- 172.21.15.29:0/967847429 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14001470 con 0x7f9a24068900 2024-01-24T14:10:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.699+0000 7f9a29430700 1 -- 172.21.15.29:0/967847429 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c011400 con 0x7f9a24068900 2024-01-24T14:10:50.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.699+0000 7f9a29430700 1 -- 172.21.15.29:0/967847429 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c011780 con 0x7f9a24068900 2024-01-24T14:10:50.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.699+0000 7f9a2c696700 1 -- 172.21.15.29:0/967847429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a24068900 msgr2=0x7f9a240c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.699+0000 7f9a2c696700 1 --2- 172.21.15.29:0/967847429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a24068900 0x7f9a240c31a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f9a1c009790 tx=0x7f9a1c009060).stop 2024-01-24T14:10:50.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.699+0000 7f9a2c696700 1 -- 172.21.15.29:0/967847429 shutdown_connections 2024-01-24T14:10:50.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.699+0000 7f9a2c696700 1 --2- 172.21.15.29:0/967847429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a24068900 0x7f9a240c31a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.699+0000 7f9a2c696700 1 -- 172.21.15.29:0/967847429 >> 172.21.15.29:0/967847429 conn(0x7f9a240c8ba0 msgr2=0x7f9a240c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:50.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.699+0000 7f9a2c696700 1 -- 172.21.15.29:0/967847429 shutdown_connections 2024-01-24T14:10:50.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.699+0000 7f9a2c696700 1 -- 172.21.15.29:0/967847429 wait complete. 2024-01-24T14:10:50.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.701+0000 7f9a2c696700 1 Processor -- start 2024-01-24T14:10:50.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.701+0000 7f9a2c696700 1 -- start start 2024-01-24T14:10:50.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.701+0000 7f9a2c696700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a24066c30 0x7f9a240a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:50.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2c696700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a24068900 0x7f9a240c0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:50.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2c696700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a24134e70 0x7f9a24149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:50.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2c696700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a240c6d20 con 0x7f9a24134e70 2024-01-24T14:10:50.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2c696700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a24165e60 con 0x7f9a24068900 2024-01-24T14:10:50.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2c696700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a24166140 con 0x7f9a24066c30 2024-01-24T14:10:50.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2ac33700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a24134e70 0x7f9a24149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:50.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2ac33700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a24134e70 0x7f9a24149650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56058/0 (socket says 172.21.15.29:56058) 2024-01-24T14:10:50.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2ac33700 1 -- 172.21.15.29:0/1389750354 learned_addr learned my addr 172.21.15.29:0/1389750354 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2a432700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a24066c30 0x7f9a240a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a29c31700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a24068900 0x7f9a240c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2ac33700 1 -- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a24066c30 msgr2=0x7f9a240a8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2ac33700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a24066c30 0x7f9a240a8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2ac33700 1 -- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a24068900 msgr2=0x7f9a240c0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2ac33700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a24068900 0x7f9a240c0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2ac33700 1 -- 172.21.15.29:0/1389750354 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c0092a0 con 0x7f9a24134e70 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2a432700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a24066c30 0x7f9a240a8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2ac33700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a24134e70 0x7f9a24149650 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f9a2406c200 tx=0x7f9a1c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a1a7fc700 1 -- 172.21.15.29:0/1389750354 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c021070 con 0x7f9a24134e70 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a1a7fc700 1 -- 172.21.15.29:0/1389750354 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c010b80 con 0x7f9a24134e70 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a2c696700 1 -- 172.21.15.29:0/1389750354 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a24166420 con 0x7f9a24134e70 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.702+0000 7f9a1a7fc700 1 -- 172.21.15.29:0/1389750354 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c01d430 con 0x7f9a24134e70 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.703+0000 7f9a2c696700 1 -- 172.21.15.29:0/1389750354 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a240aaa40 con 0x7f9a24134e70 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.704+0000 7f9a2c696700 1 -- 172.21.15.29:0/1389750354 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a240c6e90 con 0x7f9a24134e70 2024-01-24T14:10:50.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.704+0000 7f9a1a7fc700 1 -- 172.21.15.29:0/1389750354 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9a1c01d7e0 con 0x7f9a24134e70 2024-01-24T14:10:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.704+0000 7f9a1a7fc700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a0805f270 0x7f9a08061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.704+0000 7f9a1a7fc700 1 -- 172.21.15.29:0/1389750354 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f9a1c07ba10 con 0x7f9a24134e70 2024-01-24T14:10:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.705+0000 7f9a2a432700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a0805f270 0x7f9a08061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:50.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.705+0000 7f9a2a432700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a0805f270 0x7f9a08061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f9a20003e10 tx=0x7f9a20009040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:50.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.709+0000 7f9a1a7fc700 1 -- 172.21.15.29:0/1389750354 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9a1c04c910 con 0x7f9a24134e70 2024-01-24T14:10:50.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.889+0000 7f9a2c696700 1 -- 172.21.15.29:0/1389750354 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f9a240475e0 con 0x7f9a24134e70 2024-01-24T14:10:50.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.893+0000 7f9a1a7fc700 1 -- 172.21.15.29:0/1389750354 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v5) v1 ==== 120+0+0 (secure 0 0 0) 0x7f9a1c04ff30 con 0x7f9a24134e70 2024-01-24T14:10:50.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.895+0000 7f9a2c696700 1 -- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a0805f270 msgr2=0x7f9a08061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.895+0000 7f9a2c696700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a0805f270 0x7f9a08061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f9a20003e10 tx=0x7f9a20009040).stop 2024-01-24T14:10:50.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.895+0000 7f9a2c696700 1 -- 172.21.15.29:0/1389750354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a24134e70 msgr2=0x7f9a24149650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:50.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.895+0000 7f9a2c696700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a24134e70 0x7f9a24149650 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f9a2406c200 tx=0x7f9a1c016000).stop 2024-01-24T14:10:50.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.895+0000 7f9a2c696700 1 -- 172.21.15.29:0/1389750354 shutdown_connections 2024-01-24T14:10:50.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.895+0000 7f9a2c696700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a0805f270 0x7f9a08061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.895+0000 7f9a2c696700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a24134e70 0x7f9a24149650 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.895+0000 7f9a2c696700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a24066c30 0x7f9a240a8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.895+0000 7f9a2c696700 1 --2- 172.21.15.29:0/1389750354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a24068900 0x7f9a240c0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:50.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.895+0000 7f9a2c696700 1 -- 172.21.15.29:0/1389750354 >> 172.21.15.29:0/1389750354 conn(0x7f9a240c8ba0 msgr2=0x7f9a24154320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:50.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.895+0000 7f9a2c696700 1 -- 172.21.15.29:0/1389750354 shutdown_connections 2024-01-24T14:10:50.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:50.895+0000 7f9a2c696700 1 -- 172.21.15.29:0/1389750354 wait complete. 2024-01-24T14:10:50.909 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2024-01-24T14:10:51.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.039+0000 7f338874e700 1 Processor -- start 2024-01-24T14:10:51.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.039+0000 7f338874e700 1 -- start start 2024-01-24T14:10:51.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.040+0000 7f338874e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f338013ab00 0x7f33800686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:51.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.040+0000 7f338874e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33800be4e0 con 0x7f338013a720 2024-01-24T14:10:51.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.040+0000 7f338874e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33800bdbf0 con 0x7f3380066bb0 2024-01-24T14:10:51.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.040+0000 7f338874e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33800bde90 con 0x7f338013ab00 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.040+0000 7f33864ea700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f338013a720 0x7f33800664a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48232/0 (socket says 172.21.15.29:48232) 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.040+0000 7f33864ea700 1 -- 172.21.15.29:0/2641155979 learned_addr learned my addr 172.21.15.29:0/2641155979 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.040+0000 7f3385ce9700 1 --2- 172.21.15.29:0/2641155979 >> v2:172.21.15.163:3300/0 conn(0x7f338013ab00 0x7f33800686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.040+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096322152 0 0) 0x7f33800be4e0 con 0x7f338013a720 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.040+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f336c0033f0 con 0x7f338013a720 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2256988396 0 0) 0x7f336c0033f0 con 0x7f338013a720 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 >> v2:172.21.15.163:3300/0 conn(0x7f338013ab00 msgr2=0x7f33800686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 --2- 172.21.15.29:0/2641155979 >> v2:172.21.15.163:3300/0 conn(0x7f338013ab00 0x7f33800686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 >> v1:172.21.15.163:6789/0 conn(0x7f3380066bb0 legacy=0x7f33800be970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33800be170 con 0x7f338013a720 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f337c001470 con 0x7f338013a720 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 >> v1:172.21.15.29:6789/0 conn(0x7f338013a720 legacy=0x7f33800664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 --2- 172.21.15.29:0/2641155979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0038e0 0x7f336c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 --2- 172.21.15.29:0/2641155979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f336c004690 0x7f336c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 --2- 172.21.15.29:0/2641155979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f336c005000 0x7f336c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33800bde90 con 0x7f336c0038e0 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c00bf60 con 0x7f336c004690 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336c00c0f0 con 0x7f336c005000 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33864ea700 1 --2- 172.21.15.29:0/2641155979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0038e0 0x7f336c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f3385ce9700 1 --2- 172.21.15.29:0/2641155979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f336c004690 0x7f336c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f3386ceb700 1 --2- 172.21.15.29:0/2641155979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f336c005000 0x7f336c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33864ea700 1 -- 172.21.15.29:0/2641155979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f336c005000 msgr2=0x7f336c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33864ea700 1 --2- 172.21.15.29:0/2641155979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f336c005000 0x7f336c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33864ea700 1 -- 172.21.15.29:0/2641155979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f336c004690 msgr2=0x7f336c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33864ea700 1 --2- 172.21.15.29:0/2641155979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f336c004690 0x7f336c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33864ea700 1 -- 172.21.15.29:0/2641155979 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c0033f0 con 0x7f336c0038e0 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f3386ceb700 1 --2- 172.21.15.29:0/2641155979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f336c005000 0x7f336c00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f3386ceb700 1 -- 172.21.15.29:0/2641155979 reap_dead start 2024-01-24T14:10:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.041+0000 7f33864ea700 1 --2- 172.21.15.29:0/2641155979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0038e0 0x7f336c003d00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f337c00c160 tx=0x7f3380154820).ready entity=mon.0 client_cookie=d12fc7b66b058551 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:51.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.042+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00ca90 con 0x7f336c0038e0 2024-01-24T14:10:51.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.042+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c00cbf0 con 0x7f336c0038e0 2024-01-24T14:10:51.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.042+0000 7f33854e8700 1 -- 172.21.15.29:0/2641155979 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c0070d0 con 0x7f336c0038e0 2024-01-24T14:10:51.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.042+0000 7f338874e700 1 -- 172.21.15.29:0/2641155979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0038e0 msgr2=0x7f336c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:51.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.042+0000 7f338874e700 1 --2- 172.21.15.29:0/2641155979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0038e0 0x7f336c003d00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f337c00c160 tx=0x7f3380154820).stop 2024-01-24T14:10:51.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.042+0000 7f338874e700 1 -- 172.21.15.29:0/2641155979 shutdown_connections 2024-01-24T14:10:51.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.042+0000 7f338874e700 1 --2- 172.21.15.29:0/2641155979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f336c0038e0 0x7f336c003d00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:51.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.042+0000 7f338874e700 1 -- 172.21.15.29:0/2641155979 >> 172.21.15.29:0/2641155979 conn(0x7f33800c8b60 msgr2=0x7f33800c8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:51.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.042+0000 7f338874e700 1 -- 172.21.15.29:0/2641155979 shutdown_connections 2024-01-24T14:10:51.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.042+0000 7f338874e700 1 -- 172.21.15.29:0/2641155979 wait complete. 2024-01-24T14:10:51.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.043+0000 7f338874e700 1 Processor -- start 2024-01-24T14:10:51.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.044+0000 7f338874e700 1 -- start start 2024-01-24T14:10:51.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.044+0000 7f338874e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f336c0038e0 0x7f33800a7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:51.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.044+0000 7f338874e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3380152da0 0x7f3380139080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:51.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.044+0000 7f338874e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33800bedb0 0x7f33800bf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:51.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.044+0000 7f338874e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33800bf700 con 0x7f3380152da0 2024-01-24T14:10:51.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.044+0000 7f338874e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3380153220 con 0x7f33800bedb0 2024-01-24T14:10:51.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.044+0000 7f338874e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33801534f0 con 0x7f336c0038e0 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.044+0000 7f3385ce9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3380152da0 0x7f3380139080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.044+0000 7f3385ce9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3380152da0 0x7f3380139080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56066/0 (socket says 172.21.15.29:56066) 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.044+0000 7f3385ce9700 1 -- 172.21.15.29:0/2604505638 learned_addr learned my addr 172.21.15.29:0/2604505638 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f3386ceb700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33800bedb0 0x7f33800bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f33864ea700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f336c0038e0 0x7f33800a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f3385ce9700 1 -- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f336c0038e0 msgr2=0x7f33800a7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f3385ce9700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f336c0038e0 0x7f33800a7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f3385ce9700 1 -- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33800bedb0 msgr2=0x7f33800bf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f3385ce9700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33800bedb0 0x7f33800bf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f3385ce9700 1 -- 172.21.15.29:0/2604505638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33800c2110 con 0x7f3380152da0 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f3385ce9700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3380152da0 0x7f3380139080 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f33700119f0 tx=0x7f3370009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f33767fc700 1 -- 172.21.15.29:0/2604505638 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370011ee0 con 0x7f3380152da0 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f33767fc700 1 -- 172.21.15.29:0/2604505638 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337000f940 con 0x7f3380152da0 2024-01-24T14:10:51.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f338874e700 1 -- 172.21.15.29:0/2604505638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33800c23f0 con 0x7f3380152da0 2024-01-24T14:10:51.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f33767fc700 1 -- 172.21.15.29:0/2604505638 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337000fcc0 con 0x7f3380152da0 2024-01-24T14:10:51.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.045+0000 7f338874e700 1 -- 172.21.15.29:0/2604505638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33800c29b0 con 0x7f3380152da0 2024-01-24T14:10:51.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.046+0000 7f338874e700 1 -- 172.21.15.29:0/2604505638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33800475e0 con 0x7f3380152da0 2024-01-24T14:10:51.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.047+0000 7f33767fc700 1 -- 172.21.15.29:0/2604505638 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3370014650 con 0x7f3380152da0 2024-01-24T14:10:51.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.047+0000 7f33767fc700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f336405f270 0x7f3364061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:51.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.047+0000 7f33767fc700 1 -- 172.21.15.29:0/2604505638 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f337007b0f0 con 0x7f3380152da0 2024-01-24T14:10:51.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.047+0000 7f33864ea700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f336405f270 0x7f3364061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:51.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.048+0000 7f33864ea700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f336405f270 0x7f3364061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f337c00c160 tx=0x7f337c015000).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:51.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.052+0000 7f33767fc700 1 -- 172.21.15.29:0/2604505638 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f337004bff0 con 0x7f3380152da0 2024-01-24T14:10:51.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:51.312+0000 7f338874e700 1 -- 172.21.15.29:0/2604505638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f3380064020 con 0x7f3380152da0 2024-01-24T14:10:52.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.200+0000 7f33767fc700 1 -- 172.21.15.29:0/2604505638 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v452) v1 ==== 158+0+0 (secure 0 0 0) 0x7f337004f610 con 0x7f3380152da0 2024-01-24T14:10:52.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.203+0000 7f338874e700 1 -- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f336405f270 msgr2=0x7f3364061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.203+0000 7f338874e700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f336405f270 0x7f3364061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f337c00c160 tx=0x7f337c015000).stop 2024-01-24T14:10:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.203+0000 7f338874e700 1 -- 172.21.15.29:0/2604505638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3380152da0 msgr2=0x7f3380139080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.203+0000 7f338874e700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3380152da0 0x7f3380139080 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f33700119f0 tx=0x7f3370009060).stop 2024-01-24T14:10:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.203+0000 7f338874e700 1 -- 172.21.15.29:0/2604505638 shutdown_connections 2024-01-24T14:10:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.203+0000 7f338874e700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33800bedb0 0x7f33800bf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.203+0000 7f338874e700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f336c0038e0 0x7f33800a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.203+0000 7f338874e700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f336405f270 0x7f3364061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.203+0000 7f338874e700 1 --2- 172.21.15.29:0/2604505638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3380152da0 0x7f3380139080 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.203+0000 7f338874e700 1 -- 172.21.15.29:0/2604505638 >> 172.21.15.29:0/2604505638 conn(0x7f33800c8b60 msgr2=0x7f33800c36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.203+0000 7f338874e700 1 -- 172.21.15.29:0/2604505638 shutdown_connections 2024-01-24T14:10:52.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.203+0000 7f338874e700 1 -- 172.21.15.29:0/2604505638 wait complete. 2024-01-24T14:10:52.218 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2024-01-24T14:10:52.219 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2024-01-24T14:10:52.219 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-01-24T14:10:52.219 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2024-01-24T14:10:52.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.353+0000 7fb38a117700 1 Processor -- start 2024-01-24T14:10:52.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.353+0000 7fb38a117700 1 -- start start 2024-01-24T14:10:52.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb38a117700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3840c4a10 0x7fb3840bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb38a117700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb3840670a0 0x7fb384158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb38a117700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3841582b0 con 0x7fb3840664f0 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb38a117700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb384157ca0 con 0x7fb3840670a0 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb38a117700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb384063c60 con 0x7fb3840c4a10 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb383fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb3840664f0 0x7fb3840bdb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48242/0 (socket says 172.21.15.29:48242) 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb383fff700 1 -- 172.21.15.29:0/607053081 learned_addr learned my addr 172.21.15.29:0/607053081 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb382ffd700 1 --2- 172.21.15.29:0/607053081 >> v2:172.21.15.163:3300/0 conn(0x7fb3840670a0 0x7fb384158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb3837fe700 1 --2- 172.21.15.29:0/607053081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3840c4a10 0x7fb3840bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894633141 0 0) 0x7fb3841582b0 con 0x7fb3840664f0 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb36c0033f0 con 0x7fb3840664f0 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 589275649 0 0) 0x7fb36c0033f0 con 0x7fb3840664f0 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3840c4a10 msgr2=0x7fb3840bf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:52.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb3827fc700 1 --2- 172.21.15.29:0/607053081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3840c4a10 0x7fb3840bf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 >> v2:172.21.15.163:3300/0 conn(0x7fb3840670a0 msgr2=0x7fb384158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb3827fc700 1 --2- 172.21.15.29:0/607053081 >> v2:172.21.15.163:3300/0 conn(0x7fb3840670a0 0x7fb384158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb384157f40 con 0x7fb3840664f0 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fb37c0017f0 con 0x7fb3840664f0 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.354+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 >> v1:172.21.15.29:6789/0 conn(0x7fb3840664f0 legacy=0x7fb3840bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb3827fc700 1 --2- 172.21.15.29:0/607053081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb36c0038e0 0x7fb36c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb3827fc700 1 --2- 172.21.15.29:0/607053081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3840c4a10 0x7fb36c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb3827fc700 1 --2- 172.21.15.29:0/607053081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb36c004b80 0x7fb36c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb384157ca0 con 0x7fb36c004b80 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb384063c60 con 0x7fb36c0038e0 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c00b8e0 con 0x7fb3840c4a10 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb383fff700 1 --2- 172.21.15.29:0/607053081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb36c004b80 0x7fb36c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb3837fe700 1 --2- 172.21.15.29:0/607053081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb36c0038e0 0x7fb36c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb382ffd700 1 --2- 172.21.15.29:0/607053081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3840c4a10 0x7fb36c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb383fff700 1 -- 172.21.15.29:0/607053081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3840c4a10 msgr2=0x7fb36c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:52.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb383fff700 1 --2- 172.21.15.29:0/607053081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3840c4a10 0x7fb36c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb383fff700 1 -- 172.21.15.29:0/607053081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb36c0038e0 msgr2=0x7fb36c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb383fff700 1 --2- 172.21.15.29:0/607053081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb36c0038e0 0x7fb36c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb383fff700 1 -- 172.21.15.29:0/607053081 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c0033f0 con 0x7fb36c004b80 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb383fff700 1 --2- 172.21.15.29:0/607053081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb36c004b80 0x7fb36c005000 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fb37c00c530 tx=0x7fb37c002210).ready entity=mon.0 client_cookie=f2c3d61be330033f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c007de0 con 0x7fb36c004b80 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c00ce60 con 0x7fb36c004b80 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.355+0000 7fb3827fc700 1 -- 172.21.15.29:0/607053081 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c005d50 con 0x7fb36c004b80 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.356+0000 7fb38a117700 1 -- 172.21.15.29:0/607053081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb36c004b80 msgr2=0x7fb36c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.356+0000 7fb38a117700 1 --2- 172.21.15.29:0/607053081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb36c004b80 0x7fb36c005000 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fb37c00c530 tx=0x7fb37c002210).stop 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.356+0000 7fb383fff700 1 -- 172.21.15.29:0/607053081 reap_dead start 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.356+0000 7fb38a117700 1 -- 172.21.15.29:0/607053081 shutdown_connections 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.356+0000 7fb38a117700 1 -- 172.21.15.29:0/607053081 >> 172.21.15.29:0/607053081 conn(0x7fb3840c8bb0 msgr2=0x7fb3840c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.356+0000 7fb38a117700 1 -- 172.21.15.29:0/607053081 shutdown_connections 2024-01-24T14:10:52.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.356+0000 7fb38a117700 1 -- 172.21.15.29:0/607053081 wait complete. 2024-01-24T14:10:52.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.357+0000 7fb38a117700 1 Processor -- start 2024-01-24T14:10:52.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.357+0000 7fb38a117700 1 -- start start 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.358+0000 7fb38a117700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb36c004b80 0x7fb3840c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.358+0000 7fb38a117700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3840c2230 0x7fb384153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.358+0000 7fb38a117700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb384156d10 0x7fb384157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.358+0000 7fb38a117700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840c33b0 con 0x7fb36c004b80 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.358+0000 7fb38a117700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb384157680 con 0x7fb3840c2230 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.358+0000 7fb38a117700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840c1fc0 con 0x7fb384156d10 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.358+0000 7fb3837fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb36c004b80 0x7fb3840c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.358+0000 7fb3837fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb36c004b80 0x7fb3840c1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56080/0 (socket says 172.21.15.29:56080) 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.358+0000 7fb3837fe700 1 -- 172.21.15.29:0/2893265983 learned_addr learned my addr 172.21.15.29:0/2893265983 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.358+0000 7fb383fff700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb384156d10 0x7fb384157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.358+0000 7fb382ffd700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3840c2230 0x7fb384153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.359+0000 7fb3837fe700 1 -- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb384156d10 msgr2=0x7fb384157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.359+0000 7fb3837fe700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb384156d10 0x7fb384157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:52.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.359+0000 7fb3837fe700 1 -- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3840c2230 msgr2=0x7fb384153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.359+0000 7fb3837fe700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3840c2230 0x7fb384153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.359+0000 7fb3837fe700 1 -- 172.21.15.29:0/2893265983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3840c27a0 con 0x7fb36c004b80 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.359+0000 7fb3837fe700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb36c004b80 0x7fb3840c1ad0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fb3740119f0 tx=0x7fb374009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.359+0000 7fb36b7fe700 1 -- 172.21.15.29:0/2893265983 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37400f760 con 0x7fb36c004b80 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.359+0000 7fb36b7fe700 1 -- 172.21.15.29:0/2893265983 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37400f8c0 con 0x7fb36c004b80 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.359+0000 7fb38a117700 1 -- 172.21.15.29:0/2893265983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3840c2a80 con 0x7fb36c004b80 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.359+0000 7fb36b7fe700 1 -- 172.21.15.29:0/2893265983 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37400fc40 con 0x7fb36c004b80 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.359+0000 7fb38a117700 1 -- 172.21.15.29:0/2893265983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3840c3040 con 0x7fb36c004b80 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.360+0000 7fb38a117700 1 -- 172.21.15.29:0/2893265983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3840475e0 con 0x7fb36c004b80 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.360+0000 7fb36b7fe700 1 -- 172.21.15.29:0/2893265983 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb374014650 con 0x7fb36c004b80 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.361+0000 7fb36b7fe700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb37005f270 0x7fb370061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.361+0000 7fb36b7fe700 1 -- 172.21.15.29:0/2893265983 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6720+0+0 (secure 0 0 0) 0x7fb37407aad0 con 0x7fb36c004b80 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.361+0000 7fb382ffd700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb37005f270 0x7fb370061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:52.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.362+0000 7fb382ffd700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb37005f270 0x7fb370061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fb3840ab900 tx=0x7fb378009060).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:52.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.365+0000 7fb36b7fe700 1 -- 172.21.15.29:0/2893265983 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb37404b9d0 con 0x7fb36c004b80 2024-01-24T14:10:52.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:52.625+0000 7fb38a117700 1 -- 172.21.15.29:0/2893265983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fb3840640f0 con 0x7fb36c004b80 2024-01-24T14:10:53.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.203+0000 7fb36b7fe700 1 -- 172.21.15.29:0/2893265983 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v453) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb37404eff0 con 0x7fb36c004b80 2024-01-24T14:10:53.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.206+0000 7fb38a117700 1 -- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb37005f270 msgr2=0x7fb370061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:53.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.206+0000 7fb38a117700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb37005f270 0x7fb370061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fb3840ab900 tx=0x7fb378009060).stop 2024-01-24T14:10:53.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.206+0000 7fb38a117700 1 -- 172.21.15.29:0/2893265983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb36c004b80 msgr2=0x7fb3840c1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:53.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.206+0000 7fb38a117700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb36c004b80 0x7fb3840c1ad0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fb3740119f0 tx=0x7fb374009060).stop 2024-01-24T14:10:53.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.206+0000 7fb38a117700 1 -- 172.21.15.29:0/2893265983 shutdown_connections 2024-01-24T14:10:53.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.206+0000 7fb38a117700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb384156d10 0x7fb384157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.206+0000 7fb38a117700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb37005f270 0x7fb370061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.206+0000 7fb38a117700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb36c004b80 0x7fb3840c1ad0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.206+0000 7fb38a117700 1 --2- 172.21.15.29:0/2893265983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3840c2230 0x7fb384153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.206+0000 7fb38a117700 1 -- 172.21.15.29:0/2893265983 >> 172.21.15.29:0/2893265983 conn(0x7fb3840c8bb0 msgr2=0x7fb3840c72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:53.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.206+0000 7fb38a117700 1 -- 172.21.15.29:0/2893265983 shutdown_connections 2024-01-24T14:10:53.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.206+0000 7fb38a117700 1 -- 172.21.15.29:0/2893265983 wait complete. 2024-01-24T14:10:53.221 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-24T14:10:53.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.359+0000 7f504df8e700 1 Processor -- start 2024-01-24T14:10:53.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.359+0000 7f504df8e700 1 -- start start 2024-01-24T14:10:53.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.360+0000 7f504df8e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5048066940 0x7f50480be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:53.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.360+0000 7f504df8e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50480686a0 con 0x7f50480664c0 2024-01-24T14:10:53.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.360+0000 7f504df8e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50480c38e0 con 0x7f5048134ea0 2024-01-24T14:10:53.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.360+0000 7f504df8e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5048068b30 con 0x7f5048066940 2024-01-24T14:10:53.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.360+0000 7f5047fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5048066940 0x7f50480be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:53.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.360+0000 7f5047fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5048066940 0x7f50480be240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56084/0 (socket says 172.21.15.29:56084) 2024-01-24T14:10:53.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.360+0000 7f5047fff700 1 -- 172.21.15.29:0/1352452303 learned_addr learned my addr 172.21.15.29:0/1352452303 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:53.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.360+0000 7f5046ffd700 1 --1- 172.21.15.29:0/1352452303 >> v1:172.21.15.29:6789/0 conn(0x7f50480664c0 0x7f50480bdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48246/0 (socket says 172.21.15.29:48246) 2024-01-24T14:10:53.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.360+0000 7f5047fff700 1 -- 172.21.15.29:0/1352452303 >> v1:172.21.15.163:6789/0 conn(0x7f5048134ea0 legacy=0x7f50480c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:53.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.360+0000 7f5047fff700 1 -- 172.21.15.29:0/1352452303 >> v1:172.21.15.29:6789/0 conn(0x7f50480664c0 legacy=0x7f50480bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:53.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.360+0000 7f50467fc700 1 -- 172.21.15.29:0/1352452303 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18957428 0 0) 0x7f50480686a0 con 0x7f50480664c0 2024-01-24T14:10:53.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.360+0000 7f5047fff700 1 -- 172.21.15.29:0/1352452303 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5048066dd0 con 0x7f5048066940 2024-01-24T14:10:53.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.361+0000 7f5047fff700 1 --2- 172.21.15.29:0/1352452303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5048066940 0x7f50480be240 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f5040009f20 tx=0x7f5040009840).ready entity=mon.0 client_cookie=100b1ef4d40ea3ee server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:53.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.361+0000 7f50467fc700 1 -- 172.21.15.29:0/1352452303 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504000e070 con 0x7f5048066940 2024-01-24T14:10:53.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.361+0000 7f50467fc700 1 -- 172.21.15.29:0/1352452303 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504000ba80 con 0x7f5048066940 2024-01-24T14:10:53.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.361+0000 7f50467fc700 1 -- 172.21.15.29:0/1352452303 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504000be00 con 0x7f5048066940 2024-01-24T14:10:53.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.361+0000 7f504df8e700 1 -- 172.21.15.29:0/1352452303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5048066940 msgr2=0x7f50480be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:53.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.361+0000 7f504df8e700 1 --2- 172.21.15.29:0/1352452303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5048066940 0x7f50480be240 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f5040009f20 tx=0x7f5040009840).stop 2024-01-24T14:10:53.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.361+0000 7f504df8e700 1 -- 172.21.15.29:0/1352452303 shutdown_connections 2024-01-24T14:10:53.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.361+0000 7f504df8e700 1 --2- 172.21.15.29:0/1352452303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5048066940 0x7f50480be240 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.361+0000 7f504df8e700 1 -- 172.21.15.29:0/1352452303 >> 172.21.15.29:0/1352452303 conn(0x7f50480c8bd0 msgr2=0x7f50480c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:53.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.361+0000 7f504df8e700 1 -- 172.21.15.29:0/1352452303 shutdown_connections 2024-01-24T14:10:53.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.361+0000 7f504df8e700 1 -- 172.21.15.29:0/1352452303 wait complete. 2024-01-24T14:10:53.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.363+0000 7f504df8e700 1 Processor -- start 2024-01-24T14:10:53.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.363+0000 7f504df8e700 1 -- start start 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f504df8e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50480664c0 0x7f5048154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f504df8e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5048066940 0x7f50481523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f504df8e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5048134ea0 0x7f5048134520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f504df8e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5048155340 con 0x7f5048066940 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f504df8e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50481528e0 con 0x7f5048134ea0 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f504df8e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5048152b60 con 0x7f50480664c0 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f5046ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5048066940 0x7f50481523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f5046ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5048066940 0x7f50481523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56100/0 (socket says 172.21.15.29:56100) 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f5046ffd700 1 -- 172.21.15.29:0/1825513927 learned_addr learned my addr 172.21.15.29:0/1825513927 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f5047fff700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5048134ea0 0x7f5048134520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f50477fe700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50480664c0 0x7f5048154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f5046ffd700 1 -- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50480664c0 msgr2=0x7f5048154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f5046ffd700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50480664c0 0x7f5048154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f5046ffd700 1 -- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5048134ea0 msgr2=0x7f5048134520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:53.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f5046ffd700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5048134ea0 0x7f5048134520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f5046ffd700 1 -- 172.21.15.29:0/1825513927 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5040009ae0 con 0x7f5048066940 2024-01-24T14:10:53.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.364+0000 7f5046ffd700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5048066940 0x7f50481523f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f503c00c490 tx=0x7f503c007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:53.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.365+0000 7f502f7fe700 1 -- 172.21.15.29:0/1825513927 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c0070d0 con 0x7f5048066940 2024-01-24T14:10:53.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.365+0000 7f502f7fe700 1 -- 172.21.15.29:0/1825513927 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c007250 con 0x7f5048066940 2024-01-24T14:10:53.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.365+0000 7f504df8e700 1 -- 172.21.15.29:0/1825513927 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5048152e40 con 0x7f5048066940 2024-01-24T14:10:53.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.365+0000 7f502f7fe700 1 -- 172.21.15.29:0/1825513927 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c0056e0 con 0x7f5048066940 2024-01-24T14:10:53.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.365+0000 7f504df8e700 1 -- 172.21.15.29:0/1825513927 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50480c07f0 con 0x7f5048066940 2024-01-24T14:10:53.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.366+0000 7f504df8e700 1 -- 172.21.15.29:0/1825513927 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50480c0a10 con 0x7f5048066940 2024-01-24T14:10:53.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.366+0000 7f502f7fe700 1 -- 172.21.15.29:0/1825513927 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f503c005ce0 con 0x7f5048066940 2024-01-24T14:10:53.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.366+0000 7f502f7fe700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f503405f270 0x7f5034061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:53.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.367+0000 7f502f7fe700 1 -- 172.21.15.29:0/1825513927 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f503c079ef0 con 0x7f5048066940 2024-01-24T14:10:53.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.367+0000 7f50477fe700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f503405f270 0x7f5034061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:53.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.367+0000 7f50477fe700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f503405f270 0x7f5034061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f5038003900 tx=0x7f5038007d40).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:53.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.371+0000 7f502f7fe700 1 -- 172.21.15.29:0/1825513927 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f503c04ad50 con 0x7f5048066940 2024-01-24T14:10:53.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.552+0000 7f504df8e700 1 -- 172.21.15.29:0/1825513927 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f50480475e0 con 0x7f5048066940 2024-01-24T14:10:53.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.556+0000 7f502f7fe700 1 -- 172.21.15.29:0/1825513927 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f503c04e370 con 0x7f5048066940 2024-01-24T14:10:53.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.558+0000 7f504df8e700 1 -- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f503405f270 msgr2=0x7f5034061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:53.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.558+0000 7f504df8e700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f503405f270 0x7f5034061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f5038003900 tx=0x7f5038007d40).stop 2024-01-24T14:10:53.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.558+0000 7f504df8e700 1 -- 172.21.15.29:0/1825513927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5048066940 msgr2=0x7f50481523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:53.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.558+0000 7f504df8e700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5048066940 0x7f50481523f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f503c00c490 tx=0x7f503c007de0).stop 2024-01-24T14:10:53.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.558+0000 7f504df8e700 1 -- 172.21.15.29:0/1825513927 shutdown_connections 2024-01-24T14:10:53.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.558+0000 7f504df8e700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5048134ea0 0x7f5048134520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.558+0000 7f504df8e700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50480664c0 0x7f5048154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.558+0000 7f504df8e700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f503405f270 0x7f5034061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.558+0000 7f504df8e700 1 --2- 172.21.15.29:0/1825513927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5048066940 0x7f50481523f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.558+0000 7f504df8e700 1 -- 172.21.15.29:0/1825513927 >> 172.21.15.29:0/1825513927 conn(0x7f50480c8bd0 msgr2=0x7f504814b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:53.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.558+0000 7f504df8e700 1 -- 172.21.15.29:0/1825513927 shutdown_connections 2024-01-24T14:10:53.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.558+0000 7f504df8e700 1 -- 172.21.15.29:0/1825513927 wait complete. 2024-01-24T14:10:53.571 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T14:10:53.571 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:10:53.616 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:10:53.670 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T14:10:53.671 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-24T14:10:53.671 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:10:53.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.807+0000 7f5ffb06d700 1 Processor -- start 2024-01-24T14:10:53.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.807+0000 7f5ffb06d700 1 -- start start 2024-01-24T14:10:53.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ffb06d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ff4155b00 0x7f5ff40664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:53.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ffb06d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff40669e0 0x7f5ff4155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:53.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ffb06d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ff41563d0 0x7f5ff4067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:53.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ffb06d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff40c5500 con 0x7f5ff41563d0 2024-01-24T14:10:53.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ffb06d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff4066e60 con 0x7f5ff4155b00 2024-01-24T14:10:53.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ffb06d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40c3910 con 0x7f5ff40669e0 2024-01-24T14:10:53.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ff960a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ff41563d0 0x7f5ff4067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:53.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ff960a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ff41563d0 0x7f5ff4067a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37830/0 (socket says 172.21.15.29:37830) 2024-01-24T14:10:53.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ff960a700 1 -- 172.21.15.29:0/1465234699 learned_addr learned my addr 172.21.15.29:0/1465234699 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:53.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ff3fff700 1 --2- 172.21.15.29:0/1465234699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff40669e0 0x7f5ff4155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:53.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ff8e09700 1 --2- 172.21.15.29:0/1465234699 >> v2:172.21.15.163:3300/0 conn(0x7f5ff4155b00 0x7f5ff40664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ff960a700 1 -- 172.21.15.29:0/1465234699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff40669e0 msgr2=0x7f5ff4155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ff960a700 1 --2- 172.21.15.29:0/1465234699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff40669e0 0x7f5ff4155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ff960a700 1 -- 172.21.15.29:0/1465234699 >> v2:172.21.15.163:3300/0 conn(0x7f5ff4155b00 msgr2=0x7f5ff40664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ff960a700 1 --2- 172.21.15.29:0/1465234699 >> v2:172.21.15.163:3300/0 conn(0x7f5ff4155b00 0x7f5ff40664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ff960a700 1 -- 172.21.15.29:0/1465234699 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff40c5220 con 0x7f5ff41563d0 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ff8e09700 1 --2- 172.21.15.29:0/1465234699 >> v2:172.21.15.163:3300/0 conn(0x7f5ff4155b00 0x7f5ff40664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.808+0000 7f5ff960a700 1 --2- 172.21.15.29:0/1465234699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff41563d0 0x7f5ff4067a30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f5ff4069df0 tx=0x7f5fec009840).ready entity=mon.0 client_cookie=23687b223336a572 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.809+0000 7f5ff37fe700 1 -- 172.21.15.29:0/1465234699 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00e070 con 0x7f5ff41563d0 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.809+0000 7f5ff37fe700 1 -- 172.21.15.29:0/1465234699 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec00ba20 con 0x7f5ff41563d0 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.809+0000 7f5ff37fe700 1 -- 172.21.15.29:0/1465234699 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00bda0 con 0x7f5ff41563d0 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.809+0000 7f5ffb06d700 1 -- 172.21.15.29:0/1465234699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff41563d0 msgr2=0x7f5ff4067a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.809+0000 7f5ffb06d700 1 --2- 172.21.15.29:0/1465234699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff41563d0 0x7f5ff4067a30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f5ff4069df0 tx=0x7f5fec009840).stop 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.809+0000 7f5ffb06d700 1 -- 172.21.15.29:0/1465234699 shutdown_connections 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.809+0000 7f5ffb06d700 1 --2- 172.21.15.29:0/1465234699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff41563d0 0x7f5ff4067a30 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.809+0000 7f5ffb06d700 1 --2- 172.21.15.29:0/1465234699 >> v2:172.21.15.163:3300/0 conn(0x7f5ff4155b00 0x7f5ff40664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.809+0000 7f5ffb06d700 1 --2- 172.21.15.29:0/1465234699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff40669e0 0x7f5ff4155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.809+0000 7f5ffb06d700 1 -- 172.21.15.29:0/1465234699 >> 172.21.15.29:0/1465234699 conn(0x7f5ff40c8bb0 msgr2=0x7f5ff40c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:53.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.809+0000 7f5ffb06d700 1 -- 172.21.15.29:0/1465234699 shutdown_connections 2024-01-24T14:10:53.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.809+0000 7f5ffb06d700 1 -- 172.21.15.29:0/1465234699 wait complete. 2024-01-24T14:10:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.811+0000 7f5ffb06d700 1 Processor -- start 2024-01-24T14:10:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.811+0000 7f5ffb06d700 1 -- start start 2024-01-24T14:10:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.812+0000 7f5ffb06d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff40669e0 0x7f5ff4149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.812+0000 7f5ffb06d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4155b00 0x7f5ff40ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.812+0000 7f5ffb06d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff41563d0 0x7f5ff40abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.812+0000 7f5ffb06d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40aeb60 con 0x7f5ff4155b00 2024-01-24T14:10:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.812+0000 7f5ffb06d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff4149ef0 con 0x7f5ff40669e0 2024-01-24T14:10:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.812+0000 7f5ffb06d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff414a300 con 0x7f5ff41563d0 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.812+0000 7f5ff3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4155b00 0x7f5ff40ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.812+0000 7f5ff3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4155b00 0x7f5ff40ada50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37840/0 (socket says 172.21.15.29:37840) 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.812+0000 7f5ff3fff700 1 -- 172.21.15.29:0/2257458265 learned_addr learned my addr 172.21.15.29:0/2257458265 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.812+0000 7f5ff8e09700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff40669e0 0x7f5ff4149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.812+0000 7f5ff960a700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff41563d0 0x7f5ff40abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ff3fff700 1 -- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff41563d0 msgr2=0x7f5ff40abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ff3fff700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff41563d0 0x7f5ff40abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ff3fff700 1 -- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff40669e0 msgr2=0x7f5ff4149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ff3fff700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff40669e0 0x7f5ff4149a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ff3fff700 1 -- 172.21.15.29:0/2257458265 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec009ae0 con 0x7f5ff4155b00 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ff8e09700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff40669e0 0x7f5ff4149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ff960a700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff41563d0 0x7f5ff40abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ff3fff700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4155b00 0x7f5ff40ada50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f5fe000e930 tx=0x7f5fe0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:53.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ff0ff9700 1 -- 172.21.15.29:0/2257458265 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000eed0 con 0x7f5ff4155b00 2024-01-24T14:10:53.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ff0ff9700 1 -- 172.21.15.29:0/2257458265 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe000c850 con 0x7f5ff4155b00 2024-01-24T14:10:53.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ff0ff9700 1 -- 172.21.15.29:0/2257458265 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000cbd0 con 0x7f5ff4155b00 2024-01-24T14:10:53.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ffb06d700 1 -- 172.21.15.29:0/2257458265 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff40ae2c0 con 0x7f5ff4155b00 2024-01-24T14:10:53.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.813+0000 7f5ffb06d700 1 -- 172.21.15.29:0/2257458265 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff40ae880 con 0x7f5ff4155b00 2024-01-24T14:10:53.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.814+0000 7f5ffb06d700 1 -- 172.21.15.29:0/2257458265 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff414c450 con 0x7f5ff4155b00 2024-01-24T14:10:53.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.815+0000 7f5ff0ff9700 1 -- 172.21.15.29:0/2257458265 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5fe001b400 con 0x7f5ff4155b00 2024-01-24T14:10:53.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.815+0000 7f5ff0ff9700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5fd805f240 0x7f5fd8061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:53.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.815+0000 7f5ff0ff9700 1 -- 172.21.15.29:0/2257458265 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f5fe007abd0 con 0x7f5ff4155b00 2024-01-24T14:10:53.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.815+0000 7f5ff8e09700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5fd805f240 0x7f5fd8061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:53.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.816+0000 7f5ff8e09700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5fd805f240 0x7f5fd8061700 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f5fe8006fd0 tx=0x7f5fe8008040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:53.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:53.819+0000 7f5ff0ff9700 1 -- 172.21.15.29:0/2257458265 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5fe004ba70 con 0x7f5ff4155b00 2024-01-24T14:10:54.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.045+0000 7f5ffb06d700 1 -- 172.21.15.29:0/2257458265 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ff40475e0 con 0x7f5ff4155b00 2024-01-24T14:10:54.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.045+0000 7f5ff0ff9700 1 -- 172.21.15.29:0/2257458265 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f5fe004f090 con 0x7f5ff4155b00 2024-01-24T14:10:54.045 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:54.045 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1291112242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1291112242},{"type":"v1","addr":"172.21.15.163:6837","nonce":1291112242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4793,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/241296864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":241296864},{"type":"v1","addr":"172.21.15.163:6836","nonce":241296864}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5127,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2565530631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2565530631},{"type":"v1","addr":"172.21.15.29:6833","nonce":2565530631}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3978805460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3978805460},{"type":"v1","addr":"172.21.15.29:6835","nonce":3978805460}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[]} 2024-01-24T14:10:54.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.047+0000 7f5ffb06d700 1 -- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5fd805f240 msgr2=0x7f5fd8061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:54.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.047+0000 7f5ffb06d700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5fd805f240 0x7f5fd8061700 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f5fe8006fd0 tx=0x7f5fe8008040).stop 2024-01-24T14:10:54.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.047+0000 7f5ffb06d700 1 -- 172.21.15.29:0/2257458265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4155b00 msgr2=0x7f5ff40ada50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:54.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.047+0000 7f5ffb06d700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4155b00 0x7f5ff40ada50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f5fe000e930 tx=0x7f5fe0009060).stop 2024-01-24T14:10:54.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.047+0000 7f5ffb06d700 1 -- 172.21.15.29:0/2257458265 shutdown_connections 2024-01-24T14:10:54.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.047+0000 7f5ffb06d700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff41563d0 0x7f5ff40abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:54.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.047+0000 7f5ffb06d700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff40669e0 0x7f5ff4149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:54.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.047+0000 7f5ffb06d700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5fd805f240 0x7f5fd8061700 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:54.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.047+0000 7f5ffb06d700 1 --2- 172.21.15.29:0/2257458265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4155b00 0x7f5ff40ada50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:54.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.047+0000 7f5ffb06d700 1 -- 172.21.15.29:0/2257458265 >> 172.21.15.29:0/2257458265 conn(0x7f5ff40c8bb0 msgr2=0x7f5ff41517c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:54.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.048+0000 7f5ffb06d700 1 -- 172.21.15.29:0/2257458265 shutdown_connections 2024-01-24T14:10:54.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.048+0000 7f5ffb06d700 1 -- 172.21.15.29:0/2257458265 wait complete. 2024-01-24T14:10:54.048 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 20 2024-01-24T14:10:54.061 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-24T14:10:54.061 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-24T14:10:54.061 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-24T14:10:54.061 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:10:54.061 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-24T14:10:54.061 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-24T14:10:54.062 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-24T14:10:54.062 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:10:54.062 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-24T14:10:54.062 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-24T14:10:54.062 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-24T14:10:54.062 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:10:54.062 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-24T14:10:54.062 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-24T14:10:54.063 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-24T14:10:54.063 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:10:54.071 INFO:tasks.ceph.mds.a:Stopped 2024-01-24T14:10:54.071 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-24T14:10:54.071 INFO:tasks.ceph.mds.c:Stopped 2024-01-24T14:10:54.072 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-24T14:10:54.073 INFO:tasks.ceph.mds.d:Stopped 2024-01-24T14:10:54.074 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-24T14:10:54.074 INFO:tasks.ceph.mds.b:Stopped 2024-01-24T14:10:54.074 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-24T14:10:54.114 INFO:tasks.ceph.mds.a:Started 2024-01-24T14:10:54.114 DEBUG:teuthology.parallel:result is None 2024-01-24T14:10:54.115 INFO:tasks.ceph.mds.c:Started 2024-01-24T14:10:54.115 DEBUG:teuthology.parallel:result is None 2024-01-24T14:10:54.116 INFO:tasks.ceph.mds.d:Started 2024-01-24T14:10:54.116 DEBUG:teuthology.parallel:result is None 2024-01-24T14:10:54.117 INFO:tasks.ceph.mds.b:Started 2024-01-24T14:10:54.117 DEBUG:teuthology.parallel:result is None 2024-01-24T14:10:54.117 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-24T14:10:54.197 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-01-24T14:10:54.198 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T14:10:54.200 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T14:10:54.200 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T14:10:54.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.260+0000 7f834ee6a700 1 Processor -- start 2024-01-24T14:10:54.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.260+0000 7f834ee6a700 1 -- start start 2024-01-24T14:10:54.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.261+0000 7f834ee6a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8348154240 0x7f8348064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:54.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.261+0000 7f834ee6a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480a9d20 0x7f83480aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:54.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.261+0000 7f834ee6a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83481550e0 con 0x7f8348157cf0 2024-01-24T14:10:54.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.261+0000 7f834ee6a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83480c0d10 con 0x7f8348154240 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.261+0000 7f834ee6a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83480aa670 con 0x7f83480a9d20 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.261+0000 7f834cc06700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8348154240 0x7f8348064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.261+0000 7f8347fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480a9d20 0x7f83480aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.261+0000 7f834d407700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8348157cf0 0x7f8348158160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:52230/0 (socket says 172.21.15.29:52230) 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.261+0000 7f834cc06700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8348154240 0x7f8348064780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:43184/0 (socket says 172.21.15.29:43184) 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.261+0000 7f8347fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480a9d20 0x7f83480aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:49578/0 (socket says 172.21.15.29:49578) 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.261+0000 7f834d407700 1 -- 172.21.15.29:0/464999338 learned_addr learned my addr 172.21.15.29:0/464999338 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f8347fff700 1 -- 172.21.15.29:0/464999338 >> v2:172.21.15.163:3300/0 conn(0x7f8348154240 msgr2=0x7f8348064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f8347fff700 1 --2- 172.21.15.29:0/464999338 >> v2:172.21.15.163:3300/0 conn(0x7f8348154240 0x7f8348064780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f8347fff700 1 -- 172.21.15.29:0/464999338 >> v1:172.21.15.163:6789/0 conn(0x7f8348157cf0 legacy=0x7f8348158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f8347fff700 1 -- 172.21.15.29:0/464999338 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83480650d0 con 0x7f83480a9d20 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f83477fe700 1 -- 172.21.15.29:0/464999338 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622418476 0 0) 0x7f83481550e0 con 0x7f8348157cf0 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f834cc06700 1 --2- 172.21.15.29:0/464999338 >> v2:172.21.15.163:3300/0 conn(0x7f8348154240 0x7f8348064780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f8347fff700 1 --2- 172.21.15.29:0/464999338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480a9d20 0x7f83480aa180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f8334009760 tx=0x7f8334009060).ready entity=mon.2 client_cookie=905314603b736355 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:54.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f83477fe700 1 -- 172.21.15.29:0/464999338 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400f510 con 0x7f83480a9d20 2024-01-24T14:10:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f83477fe700 1 -- 172.21.15.29:0/464999338 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833400f670 con 0x7f83480a9d20 2024-01-24T14:10:54.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f83477fe700 1 -- 172.21.15.29:0/464999338 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400f9f0 con 0x7f83480a9d20 2024-01-24T14:10:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f834ee6a700 1 -- 172.21.15.29:0/464999338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480a9d20 msgr2=0x7f83480aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f834ee6a700 1 --2- 172.21.15.29:0/464999338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480a9d20 0x7f83480aa180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f8334009760 tx=0x7f8334009060).stop 2024-01-24T14:10:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f834ee6a700 1 -- 172.21.15.29:0/464999338 shutdown_connections 2024-01-24T14:10:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f834ee6a700 1 --2- 172.21.15.29:0/464999338 >> v2:172.21.15.163:3300/0 conn(0x7f8348154240 0x7f8348064780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f834ee6a700 1 --2- 172.21.15.29:0/464999338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480a9d20 0x7f83480aa180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:54.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f834ee6a700 1 -- 172.21.15.29:0/464999338 >> 172.21.15.29:0/464999338 conn(0x7f83480c8310 msgr2=0x7f834805e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.262+0000 7f834ee6a700 1 -- 172.21.15.29:0/464999338 shutdown_connections 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.263+0000 7f834ee6a700 1 -- 172.21.15.29:0/464999338 wait complete. 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.264+0000 7f834ee6a700 1 Processor -- start 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.264+0000 7f834ee6a700 1 -- start start 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f834ee6a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480a9d20 0x7f8348158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f834ee6a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8348154240 0x7f83480b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f834ee6a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8348157cf0 0x7f83481504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f834ee6a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8348159330 con 0x7f8348154240 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f834ee6a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8348158720 con 0x7f8348157cf0 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f834ee6a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8348158a00 con 0x7f83480a9d20 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f8347fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8348154240 0x7f83480b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f8347fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8348154240 0x7f83480b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37890/0 (socket says 172.21.15.29:37890) 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f8347fff700 1 -- 172.21.15.29:0/163928656 learned_addr learned my addr 172.21.15.29:0/163928656 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f834cc06700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480a9d20 0x7f8348158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f834d407700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8348157cf0 0x7f83481504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:54.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f8347fff700 1 -- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480a9d20 msgr2=0x7f8348158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f8347fff700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480a9d20 0x7f8348158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f8347fff700 1 -- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8348157cf0 msgr2=0x7f83481504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f8347fff700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8348157cf0 0x7f83481504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f8347fff700 1 -- 172.21.15.29:0/163928656 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c0092a0 con 0x7f8348154240 2024-01-24T14:10:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f8347fff700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8348154240 0x7f83480b05d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f83480aa7b0 tx=0x7f8334017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.265+0000 7f8344ff9700 1 -- 172.21.15.29:0/163928656 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400f7e0 con 0x7f8348154240 2024-01-24T14:10:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.266+0000 7f8344ff9700 1 -- 172.21.15.29:0/163928656 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833400f940 con 0x7f8348154240 2024-01-24T14:10:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.266+0000 7f834ee6a700 1 -- 172.21.15.29:0/163928656 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83340092a0 con 0x7f8348154240 2024-01-24T14:10:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.266+0000 7f8344ff9700 1 -- 172.21.15.29:0/163928656 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833401b660 con 0x7f8348154240 2024-01-24T14:10:54.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.266+0000 7f834ee6a700 1 -- 172.21.15.29:0/163928656 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8348158fc0 con 0x7f8348154240 2024-01-24T14:10:54.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.267+0000 7f834ee6a700 1 -- 172.21.15.29:0/163928656 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83480475e0 con 0x7f8348154240 2024-01-24T14:10:54.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.267+0000 7f8344ff9700 1 -- 172.21.15.29:0/163928656 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f833401ba50 con 0x7f8348154240 2024-01-24T14:10:54.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.267+0000 7f8344ff9700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f832c05f240 0x7f832c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:54.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.267+0000 7f8344ff9700 1 -- 172.21.15.29:0/163928656 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f833407bf10 con 0x7f8348154240 2024-01-24T14:10:54.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.268+0000 7f834cc06700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f832c05f240 0x7f832c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:54.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.268+0000 7f834cc06700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f832c05f240 0x7f832c061700 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f833c009270 tx=0x7f833c008040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:54.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.272+0000 7f8344ff9700 1 -- 172.21.15.29:0/163928656 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f833404ce10 con 0x7f8348154240 2024-01-24T14:10:54.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:54.466+0000 7f834ee6a700 1 -- 172.21.15.29:0/163928656 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8348159470 con 0x7f8348154240 2024-01-24T14:10:55.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.250+0000 7f8344ff9700 1 -- 172.21.15.29:0/163928656 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8334029020 con 0x7f8348154240 2024-01-24T14:10:55.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.253+0000 7f834ee6a700 1 -- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f832c05f240 msgr2=0x7f832c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:55.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.253+0000 7f834ee6a700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f832c05f240 0x7f832c061700 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f833c009270 tx=0x7f833c008040).stop 2024-01-24T14:10:55.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.253+0000 7f834ee6a700 1 -- 172.21.15.29:0/163928656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8348154240 msgr2=0x7f83480b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:55.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.253+0000 7f834ee6a700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8348154240 0x7f83480b05d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f83480aa7b0 tx=0x7f8334017000).stop 2024-01-24T14:10:55.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.253+0000 7f834ee6a700 1 -- 172.21.15.29:0/163928656 shutdown_connections 2024-01-24T14:10:55.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.253+0000 7f834ee6a700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8348157cf0 0x7f83481504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.253+0000 7f834ee6a700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480a9d20 0x7f8348158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.253+0000 7f834ee6a700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f832c05f240 0x7f832c061700 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.253+0000 7f834ee6a700 1 --2- 172.21.15.29:0/163928656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8348154240 0x7f83480b05d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.253+0000 7f834ee6a700 1 -- 172.21.15.29:0/163928656 >> 172.21.15.29:0/163928656 conn(0x7f83480c8310 msgr2=0x7f8348155570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:55.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.253+0000 7f834ee6a700 1 -- 172.21.15.29:0/163928656 shutdown_connections 2024-01-24T14:10:55.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.253+0000 7f834ee6a700 1 -- 172.21.15.29:0/163928656 wait complete. 2024-01-24T14:10:55.254 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-01-24T14:10:55.268 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-24T14:10:55.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.403+0000 7f723e25f700 1 Processor -- start 2024-01-24T14:10:55.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.403+0000 7f723e25f700 1 -- start start 2024-01-24T14:10:55.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.404+0000 7f723e25f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7238154370 0x7f72380c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:55.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.404+0000 7f723e25f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7238154750 0x7f72380631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:55.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.404+0000 7f723e25f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72380aa6c0 con 0x7f72380aa0e0 2024-01-24T14:10:55.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.404+0000 7f723e25f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7238064240 con 0x7f7238154370 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.404+0000 7f723e25f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72380cb5c0 con 0x7f7238154750 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.404+0000 7f7237fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7238154750 0x7f72380631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.404+0000 7f72377fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7238154370 0x7f72380c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.404+0000 7f7236ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f72380aa0e0 0x7f72380ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:52238/0 (socket says 172.21.15.29:52238) 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.404+0000 7f7236ffd700 1 -- 172.21.15.29:0/1822939754 learned_addr learned my addr 172.21.15.29:0/1822939754 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.405+0000 7f72367fc700 1 -- 172.21.15.29:0/1822939754 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747302992 0 0) 0x7f72380aa6c0 con 0x7f72380aa0e0 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.405+0000 7f72367fc700 1 -- 172.21.15.29:0/1822939754 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72200033f0 con 0x7f72380aa0e0 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.405+0000 7f72377fe700 1 -- 172.21.15.29:0/1822939754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7238154750 msgr2=0x7f72380631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.405+0000 7f72377fe700 1 --2- 172.21.15.29:0/1822939754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7238154750 0x7f72380631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.405+0000 7f72377fe700 1 -- 172.21.15.29:0/1822939754 >> v1:172.21.15.163:6789/0 conn(0x7f72380aa0e0 legacy=0x7f72380ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.405+0000 7f72377fe700 1 -- 172.21.15.29:0/1822939754 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72380cb8a0 con 0x7f7238154370 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.405+0000 7f7237fff700 1 --2- 172.21.15.29:0/1822939754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7238154750 0x7f72380631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.405+0000 7f72377fe700 1 --2- 172.21.15.29:0/1822939754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7238154370 0x7f72380c5410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f7228006fd0 tx=0x7f7228006f40).ready entity=mon.1 client_cookie=f0815470339b8135 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.405+0000 7f72367fc700 1 -- 172.21.15.29:0/1822939754 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228005880 con 0x7f7238154370 2024-01-24T14:10:55.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.405+0000 7f72367fc700 1 -- 172.21.15.29:0/1822939754 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72280059e0 con 0x7f7238154370 2024-01-24T14:10:55.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.405+0000 7f72367fc700 1 -- 172.21.15.29:0/1822939754 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228005d60 con 0x7f7238154370 2024-01-24T14:10:55.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.406+0000 7f723e25f700 1 -- 172.21.15.29:0/1822939754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7238154370 msgr2=0x7f72380c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:55.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.406+0000 7f723e25f700 1 --2- 172.21.15.29:0/1822939754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7238154370 0x7f72380c5410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f7228006fd0 tx=0x7f7228006f40).stop 2024-01-24T14:10:55.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.406+0000 7f723e25f700 1 -- 172.21.15.29:0/1822939754 shutdown_connections 2024-01-24T14:10:55.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.406+0000 7f723e25f700 1 --2- 172.21.15.29:0/1822939754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7238154750 0x7f72380631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.406+0000 7f723e25f700 1 --2- 172.21.15.29:0/1822939754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7238154370 0x7f72380c5410 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.406+0000 7f723e25f700 1 -- 172.21.15.29:0/1822939754 >> 172.21.15.29:0/1822939754 conn(0x7f723805ec80 msgr2=0x7f72380cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:55.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.406+0000 7f723e25f700 1 -- 172.21.15.29:0/1822939754 shutdown_connections 2024-01-24T14:10:55.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.406+0000 7f723e25f700 1 -- 172.21.15.29:0/1822939754 wait complete. 2024-01-24T14:10:55.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.407+0000 7f723e25f700 1 Processor -- start 2024-01-24T14:10:55.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.407+0000 7f723e25f700 1 -- start start 2024-01-24T14:10:55.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.408+0000 7f723e25f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72380aa0e0 0x7f7238135ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:55.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.408+0000 7f723e25f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7238154370 0x7f7238165470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:55.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.408+0000 7f723e25f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7238154750 0x7f72380c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:55.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.408+0000 7f723e25f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7238135fc0 con 0x7f7238154750 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.408+0000 7f723e25f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7238159ab0 con 0x7f72380aa0e0 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.408+0000 7f723e25f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723814f5d0 con 0x7f7238154370 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.408+0000 7f7237fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7238154750 0x7f72380c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.408+0000 7f7237fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7238154750 0x7f72380c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37906/0 (socket says 172.21.15.29:37906) 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.408+0000 7f7237fff700 1 -- 172.21.15.29:0/1780746653 learned_addr learned my addr 172.21.15.29:0/1780746653 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.408+0000 7f7236ffd700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7238154370 0x7f7238165470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.408+0000 7f72377fe700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72380aa0e0 0x7f7238135ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.409+0000 7f7237fff700 1 -- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7238154370 msgr2=0x7f7238165470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.409+0000 7f7237fff700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7238154370 0x7f7238165470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.409+0000 7f7237fff700 1 -- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72380aa0e0 msgr2=0x7f7238135ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.409+0000 7f7237fff700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72380aa0e0 0x7f7238135ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.409+0000 7f7237fff700 1 -- 172.21.15.29:0/1780746653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7230009a20 con 0x7f7238154750 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.409+0000 7f7237fff700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7238154750 0x7f72380c9da0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f7230009f20 tx=0x7f723000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:55.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.409+0000 7f721f7fe700 1 -- 172.21.15.29:0/1780746653 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723000cb70 con 0x7f7238154750 2024-01-24T14:10:55.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.409+0000 7f721f7fe700 1 -- 172.21.15.29:0/1780746653 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723000f040 con 0x7f7238154750 2024-01-24T14:10:55.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.409+0000 7f723e25f700 1 -- 172.21.15.29:0/1780746653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7228009250 con 0x7f7238154750 2024-01-24T14:10:55.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.409+0000 7f721f7fe700 1 -- 172.21.15.29:0/1780746653 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7230013400 con 0x7f7238154750 2024-01-24T14:10:55.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.409+0000 7f723e25f700 1 -- 172.21.15.29:0/1780746653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723814fb90 con 0x7f7238154750 2024-01-24T14:10:55.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.410+0000 7f723e25f700 1 -- 172.21.15.29:0/1780746653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7238159c70 con 0x7f7238154750 2024-01-24T14:10:55.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.410+0000 7f721f7fe700 1 -- 172.21.15.29:0/1780746653 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f72300137b0 con 0x7f7238154750 2024-01-24T14:10:55.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.411+0000 7f721f7fe700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f722405f270 0x7f7224061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:55.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.411+0000 7f721f7fe700 1 -- 172.21.15.29:0/1780746653 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f723007ac90 con 0x7f7238154750 2024-01-24T14:10:55.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.411+0000 7f72377fe700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f722405f270 0x7f7224061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:55.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.411+0000 7f72377fe700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f722405f270 0x7f7224061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f7228006f80 tx=0x7f7228015000).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:55.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.415+0000 7f721f7fe700 1 -- 172.21.15.29:0/1780746653 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f723004bd30 con 0x7f7238154750 2024-01-24T14:10:55.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.669+0000 7f723e25f700 1 -- 172.21.15.29:0/1780746653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f72380475e0 con 0x7f7238154750 2024-01-24T14:10:55.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.670+0000 7f721f7fe700 1 -- 172.21.15.29:0/1780746653 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f723004f350 con 0x7f7238154750 2024-01-24T14:10:55.670 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:55.670 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T14:10:55.670 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-01-24T14:10:55.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.670 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-01-24T14:10:55.671 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew==", 2024-01-24T14:10:55.671 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.671 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:10:55.671 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:10:55.671 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:10:55.671 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.671 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-01-24T14:10:55.671 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ==", 2024-01-24T14:10:55.671 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:10:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:10:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:10:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-01-24T14:10:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw==", 2024-01-24T14:10:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:10:55.672 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ==", 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.674 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-01-24T14:10:55.674 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ==", 2024-01-24T14:10:55.674 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.674 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:10:55.674 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:10:55.674 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.674 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA==", 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw==", 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:10:55.675 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ==", 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.676 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw==", 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ==", 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.677 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A==", 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ==", 2024-01-24T14:10:55.678 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw==", 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA==", 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g==", 2024-01-24T14:10:55.680 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ==", 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:10:55.681 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlxK1bFxAAZpfJn+Tcpmq7R9Yq29FvVg==", 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw==", 2024-01-24T14:10:55.682 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA==", 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.683 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:55.684 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-01-24T14:10:55.684 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T14:10:55.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.672+0000 7f723e25f700 1 -- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f722405f270 msgr2=0x7f7224061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:55.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.672+0000 7f723e25f700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f722405f270 0x7f7224061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f7228006f80 tx=0x7f7228015000).stop 2024-01-24T14:10:55.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.672+0000 7f723e25f700 1 -- 172.21.15.29:0/1780746653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7238154750 msgr2=0x7f72380c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:55.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.672+0000 7f723e25f700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7238154750 0x7f72380c9da0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f7230009f20 tx=0x7f723000b040).stop 2024-01-24T14:10:55.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.672+0000 7f723e25f700 1 -- 172.21.15.29:0/1780746653 shutdown_connections 2024-01-24T14:10:55.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.672+0000 7f723e25f700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f722405f270 0x7f7224061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.672+0000 7f723e25f700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7238154750 0x7f72380c9da0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.672+0000 7f723e25f700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72380aa0e0 0x7f7238135ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.672+0000 7f723e25f700 1 --2- 172.21.15.29:0/1780746653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7238154370 0x7f7238165470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.672+0000 7f723e25f700 1 -- 172.21.15.29:0/1780746653 >> 172.21.15.29:0/1780746653 conn(0x7f723805ec80 msgr2=0x7f7238153500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:55.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.672+0000 7f723e25f700 1 -- 172.21.15.29:0/1780746653 shutdown_connections 2024-01-24T14:10:55.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.672+0000 7f723e25f700 1 -- 172.21.15.29:0/1780746653 wait complete. 2024-01-24T14:10:55.687 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:10:55.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.819+0000 7f84b82de700 1 Processor -- start 2024-01-24T14:10:55.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.819+0000 7f84b82de700 1 -- start start 2024-01-24T14:10:55.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b82de700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84b01543c0 0x7f84b00bcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:55.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b82de700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f84b00bd120 0x7f84b00c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:55.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b82de700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b005e850 con 0x7f84b0153fe0 2024-01-24T14:10:55.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b82de700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b00a4f90 con 0x7f84b01543c0 2024-01-24T14:10:55.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b82de700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b00631f0 con 0x7f84b00bd120 2024-01-24T14:10:55.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b5879700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84b01543c0 0x7f84b00bcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:55.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b5879700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84b01543c0 0x7f84b00bcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37914/0 (socket says 172.21.15.29:37914) 2024-01-24T14:10:55.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b5879700 1 -- 172.21.15.29:0/3761720356 learned_addr learned my addr 172.21.15.29:0/3761720356 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:55.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b687b700 1 --2- 172.21.15.29:0/3761720356 >> v2:172.21.15.163:3300/0 conn(0x7f84b00bd120 0x7f84b00c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:55.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b5879700 1 -- 172.21.15.29:0/3761720356 >> v2:172.21.15.163:3300/0 conn(0x7f84b00bd120 msgr2=0x7f84b00c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:55.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b5879700 1 --2- 172.21.15.29:0/3761720356 >> v2:172.21.15.163:3300/0 conn(0x7f84b00bd120 0x7f84b00c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b5879700 1 -- 172.21.15.29:0/3761720356 >> v1:172.21.15.163:6789/0 conn(0x7f84b0153fe0 legacy=0x7f84b00a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b5879700 1 -- 172.21.15.29:0/3761720356 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b00bd7e0 con 0x7f84b01543c0 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.820+0000 7f84b5879700 1 --2- 172.21.15.29:0/3761720356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b01543c0 0x7f84b00bcc30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f84b0065ac0 tx=0x7f84a00090f0).ready entity=mon.0 client_cookie=f2a3b3fa686f00ef server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.821+0000 7f84b5078700 1 -- 172.21.15.29:0/3761720356 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a000b520 con 0x7f84b01543c0 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.821+0000 7f84b5078700 1 -- 172.21.15.29:0/3761720356 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a000b680 con 0x7f84b01543c0 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.821+0000 7f84b5078700 1 -- 172.21.15.29:0/3761720356 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a000ba00 con 0x7f84b01543c0 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.821+0000 7f84b82de700 1 -- 172.21.15.29:0/3761720356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b01543c0 msgr2=0x7f84b00bcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.821+0000 7f84b82de700 1 --2- 172.21.15.29:0/3761720356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b01543c0 0x7f84b00bcc30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f84b0065ac0 tx=0x7f84a00090f0).stop 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.821+0000 7f84b82de700 1 -- 172.21.15.29:0/3761720356 shutdown_connections 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.821+0000 7f84b82de700 1 --2- 172.21.15.29:0/3761720356 >> v2:172.21.15.163:3300/0 conn(0x7f84b00bd120 0x7f84b00c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.821+0000 7f84b82de700 1 --2- 172.21.15.29:0/3761720356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b01543c0 0x7f84b00bcc30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.821+0000 7f84b82de700 1 -- 172.21.15.29:0/3761720356 >> 172.21.15.29:0/3761720356 conn(0x7f84b0064aa0 msgr2=0x7f84b0064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.821+0000 7f84b82de700 1 -- 172.21.15.29:0/3761720356 shutdown_connections 2024-01-24T14:10:55.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.821+0000 7f84b82de700 1 -- 172.21.15.29:0/3761720356 wait complete. 2024-01-24T14:10:55.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.823+0000 7f84b82de700 1 Processor -- start 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.823+0000 7f84b82de700 1 -- start start 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.823+0000 7f84b82de700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b00bd120 0x7f84b0145160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.823+0000 7f84b82de700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84b0153fe0 0x7f84b0154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.823+0000 7f84b82de700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b01543c0 0x7f84b0154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.823+0000 7f84b82de700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b01559a0 con 0x7f84b00bd120 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.823+0000 7f84b82de700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0156230 con 0x7f84b0153fe0 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.823+0000 7f84b82de700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0145650 con 0x7f84b01543c0 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b607a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b00bd120 0x7f84b0145160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b607a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b00bd120 0x7f84b0145160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37924/0 (socket says 172.21.15.29:37924) 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b607a700 1 -- 172.21.15.29:0/2773124135 learned_addr learned my addr 172.21.15.29:0/2773124135 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b687b700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b01543c0 0x7f84b0154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b5879700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84b0153fe0 0x7f84b0154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:55.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b607a700 1 -- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b01543c0 msgr2=0x7f84b0154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b607a700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b01543c0 0x7f84b0154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b607a700 1 -- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84b0153fe0 msgr2=0x7f84b0154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b607a700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84b0153fe0 0x7f84b0154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b607a700 1 -- 172.21.15.29:0/2773124135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a0009370 con 0x7f84b00bd120 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b687b700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b01543c0 0x7f84b0154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b5879700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84b0153fe0 0x7f84b0154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b607a700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b00bd120 0x7f84b0145160 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f84ac00c4f0 tx=0x7f84ac007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84a67fc700 1 -- 172.21.15.29:0/2773124135 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac007020 con 0x7f84b00bd120 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84a67fc700 1 -- 172.21.15.29:0/2773124135 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ac001e90 con 0x7f84b00bd120 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84a67fc700 1 -- 172.21.15.29:0/2773124135 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac0073d0 con 0x7f84b00bd120 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b82de700 1 -- 172.21.15.29:0/2773124135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b0165880 con 0x7f84b00bd120 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.824+0000 7f84b82de700 1 -- 172.21.15.29:0/2773124135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b01556c0 con 0x7f84b00bd120 2024-01-24T14:10:55.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.825+0000 7f84b82de700 1 -- 172.21.15.29:0/2773124135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b00475e0 con 0x7f84b00bd120 2024-01-24T14:10:55.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.826+0000 7f84a67fc700 1 -- 172.21.15.29:0/2773124135 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f84ac0068b0 con 0x7f84b00bd120 2024-01-24T14:10:55.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.826+0000 7f84a67fc700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f849405f240 0x7f8494061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:55.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.826+0000 7f84a67fc700 1 -- 172.21.15.29:0/2773124135 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f84ac079d80 con 0x7f84b00bd120 2024-01-24T14:10:55.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.827+0000 7f84b5879700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f849405f240 0x7f8494061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:55.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.827+0000 7f84b5879700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f849405f240 0x7f8494061700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f84a0009130 tx=0x7f84a0017000).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:55.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:55.831+0000 7f84a67fc700 1 -- 172.21.15.29:0/2773124135 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f84ac04ae20 con 0x7f84b00bd120 2024-01-24T14:10:56.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.014+0000 7f84b82de700 1 -- 172.21.15.29:0/2773124135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84b0156510 con 0x7f84b00bd120 2024-01-24T14:10:56.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.015+0000 7f84a67fc700 1 -- 172.21.15.29:0/2773124135 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+14954 (secure 0 0 0) 0x7f84ac04e440 con 0x7f84b00bd120 2024-01-24T14:10:56.014 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:56.015 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":68,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:10:55.246338+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:10:56.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.017+0000 7f84b82de700 1 -- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f849405f240 msgr2=0x7f8494061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.017+0000 7f84b82de700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f849405f240 0x7f8494061700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f84a0009130 tx=0x7f84a0017000).stop 2024-01-24T14:10:56.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.017+0000 7f84b82de700 1 -- 172.21.15.29:0/2773124135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b00bd120 msgr2=0x7f84b0145160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.017+0000 7f84b82de700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b00bd120 0x7f84b0145160 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f84ac00c4f0 tx=0x7f84ac007e00).stop 2024-01-24T14:10:56.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.017+0000 7f84b82de700 1 -- 172.21.15.29:0/2773124135 shutdown_connections 2024-01-24T14:10:56.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.017+0000 7f84b82de700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b01543c0 0x7f84b0154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.017+0000 7f84b82de700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f849405f240 0x7f8494061700 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.017+0000 7f84b82de700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b00bd120 0x7f84b0145160 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.017+0000 7f84b82de700 1 --2- 172.21.15.29:0/2773124135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84b0153fe0 0x7f84b0154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.017+0000 7f84b82de700 1 -- 172.21.15.29:0/2773124135 >> 172.21.15.29:0/2773124135 conn(0x7f84b0064aa0 msgr2=0x7f84b00a5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:56.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.017+0000 7f84b82de700 1 -- 172.21.15.29:0/2773124135 shutdown_connections 2024-01-24T14:10:56.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.017+0000 7f84b82de700 1 -- 172.21.15.29:0/2773124135 wait complete. 2024-01-24T14:10:56.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T14:10:56.031 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T14:10:56.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.164+0000 7f75e35c0700 1 Processor -- start 2024-01-24T14:10:56.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.164+0000 7f75e35c0700 1 -- start start 2024-01-24T14:10:56.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e35c0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f75dc064760 0x7f75dc064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e35c0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75dc0bfd40 0x7f75dc0c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e35c0700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75dc152240 con 0x7f75dc0c8490 2024-01-24T14:10:56.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e35c0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75dc1586c0 con 0x7f75dc064760 2024-01-24T14:10:56.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e35c0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0aa670 con 0x7f75dc0bfd40 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e0b5b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f75dc0c8490 0x7f75dc0a9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59636/0 (socket says 172.21.15.29:59636) 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e0b5b700 1 -- 172.21.15.29:0/942854314 learned_addr learned my addr 172.21.15.29:0/942854314 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e1b5d700 1 --2- 172.21.15.29:0/942854314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75dc0bfd40 0x7f75dc0c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e135c700 1 --2- 172.21.15.29:0/942854314 >> v2:172.21.15.163:3300/0 conn(0x7f75dc064760 0x7f75dc064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75cffff700 1 -- 172.21.15.29:0/942854314 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209193259 0 0) 0x7f75dc152240 con 0x7f75dc0c8490 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75cffff700 1 -- 172.21.15.29:0/942854314 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c80033f0 con 0x7f75dc0c8490 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75cffff700 1 -- 172.21.15.29:0/942854314 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4154569425 0 0) 0x7f75c80033f0 con 0x7f75dc0c8490 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e1b5d700 1 -- 172.21.15.29:0/942854314 >> v2:172.21.15.163:3300/0 conn(0x7f75dc064760 msgr2=0x7f75dc064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e1b5d700 1 --2- 172.21.15.29:0/942854314 >> v2:172.21.15.163:3300/0 conn(0x7f75dc064760 0x7f75dc064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e1b5d700 1 -- 172.21.15.29:0/942854314 >> v1:172.21.15.29:6789/0 conn(0x7f75dc0c8490 legacy=0x7f75dc0a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e1b5d700 1 -- 172.21.15.29:0/942854314 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc157d90 con 0x7f75dc0bfd40 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.166+0000 7f75e135c700 1 --2- 172.21.15.29:0/942854314 >> v2:172.21.15.163:3300/0 conn(0x7f75dc064760 0x7f75dc064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.167+0000 7f75e1b5d700 1 --2- 172.21.15.29:0/942854314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75dc0bfd40 0x7f75dc0c01d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f75d4009e60 tx=0x7f75d4009780).ready entity=mon.2 client_cookie=15bd008dffeb2aff server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:56.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.167+0000 7f75cffff700 1 -- 172.21.15.29:0/942854314 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d400e070 con 0x7f75dc0bfd40 2024-01-24T14:10:56.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.167+0000 7f75cffff700 1 -- 172.21.15.29:0/942854314 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d400b830 con 0x7f75dc0bfd40 2024-01-24T14:10:56.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.167+0000 7f75cffff700 1 -- 172.21.15.29:0/942854314 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8001470 con 0x7f75dc0bfd40 2024-01-24T14:10:56.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.167+0000 7f75e35c0700 1 -- 172.21.15.29:0/942854314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75dc0bfd40 msgr2=0x7f75dc0c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.167+0000 7f75e35c0700 1 --2- 172.21.15.29:0/942854314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75dc0bfd40 0x7f75dc0c01d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f75d4009e60 tx=0x7f75d4009780).stop 2024-01-24T14:10:56.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.167+0000 7f75e35c0700 1 -- 172.21.15.29:0/942854314 shutdown_connections 2024-01-24T14:10:56.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.167+0000 7f75e35c0700 1 --2- 172.21.15.29:0/942854314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75dc0bfd40 0x7f75dc0c01d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.167+0000 7f75e35c0700 1 --2- 172.21.15.29:0/942854314 >> v2:172.21.15.163:3300/0 conn(0x7f75dc064760 0x7f75dc064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.167+0000 7f75e35c0700 1 -- 172.21.15.29:0/942854314 >> 172.21.15.29:0/942854314 conn(0x7f75dc0cb8b0 msgr2=0x7f75dc0c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:56.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.168+0000 7f75e35c0700 1 -- 172.21.15.29:0/942854314 shutdown_connections 2024-01-24T14:10:56.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.168+0000 7f75e35c0700 1 -- 172.21.15.29:0/942854314 wait complete. 2024-01-24T14:10:56.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.169+0000 7f75e35c0700 1 Processor -- start 2024-01-24T14:10:56.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.169+0000 7f75e35c0700 1 -- start start 2024-01-24T14:10:56.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e35c0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75dc064760 0x7f75dc14ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e35c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75dc0bfd40 0x7f75dc0ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e35c0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75dc0c8490 0x7f75dc0accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e35c0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0ad1b0 con 0x7f75dc064760 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e35c0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0c4480 con 0x7f75dc0bfd40 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e35c0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0c3b50 con 0x7f75dc0c8490 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e135c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75dc064760 0x7f75dc14ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e135c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75dc064760 0x7f75dc14ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37932/0 (socket says 172.21.15.29:37932) 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e135c700 1 -- 172.21.15.29:0/2914180003 learned_addr learned my addr 172.21.15.29:0/2914180003 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e0b5b700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75dc0bfd40 0x7f75dc0ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e1b5d700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75dc0c8490 0x7f75dc0accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e135c700 1 -- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75dc0c8490 msgr2=0x7f75dc0accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e135c700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75dc0c8490 0x7f75dc0accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e135c700 1 -- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75dc0bfd40 msgr2=0x7f75dc0ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e135c700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75dc0bfd40 0x7f75dc0ac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e135c700 1 -- 172.21.15.29:0/2914180003 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d00092a0 con 0x7f75dc064760 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.170+0000 7f75e135c700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75dc064760 0x7f75dc14ee40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f75d000e980 tx=0x7f75d0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.171+0000 7f75cd7fa700 1 -- 172.21.15.29:0/2914180003 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000c850 con 0x7f75dc064760 2024-01-24T14:10:56.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.171+0000 7f75cd7fa700 1 -- 172.21.15.29:0/2914180003 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d000c9b0 con 0x7f75dc064760 2024-01-24T14:10:56.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.171+0000 7f75e35c0700 1 -- 172.21.15.29:0/2914180003 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d4009a20 con 0x7f75dc064760 2024-01-24T14:10:56.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.171+0000 7f75cd7fa700 1 -- 172.21.15.29:0/2914180003 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000cd30 con 0x7f75dc064760 2024-01-24T14:10:56.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.171+0000 7f75e35c0700 1 -- 172.21.15.29:0/2914180003 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc0c4110 con 0x7f75dc064760 2024-01-24T14:10:56.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.172+0000 7f75e35c0700 1 -- 172.21.15.29:0/2914180003 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75dc0c4760 con 0x7f75dc064760 2024-01-24T14:10:56.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.172+0000 7f75cd7fa700 1 -- 172.21.15.29:0/2914180003 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f75d0012600 con 0x7f75dc064760 2024-01-24T14:10:56.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.172+0000 7f75cd7fa700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f75c005f270 0x7f75c0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.173+0000 7f75cd7fa700 1 -- 172.21.15.29:0/2914180003 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f75d007aad0 con 0x7f75dc064760 2024-01-24T14:10:56.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.173+0000 7f75e0b5b700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f75c005f270 0x7f75c0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.173+0000 7f75e0b5b700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f75c005f270 0x7f75c0061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f75d80016e0 tx=0x7f75d8001500).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:56.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.177+0000 7f75cd7fa700 1 -- 172.21.15.29:0/2914180003 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f75d004bb70 con 0x7f75dc064760 2024-01-24T14:10:56.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.360+0000 7f75e35c0700 1 -- 172.21.15.29:0/2914180003 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f75dc0475e0 con 0x7f75dc064760 2024-01-24T14:10:56.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.360+0000 7f75cd7fa700 1 -- 172.21.15.29:0/2914180003 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v68) v1 ==== 110+0+10 (secure 0 0 0) 0x7f75d004f190 con 0x7f75dc064760 2024-01-24T14:10:56.360 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T14:10:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.362+0000 7f75e35c0700 1 -- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f75c005f270 msgr2=0x7f75c0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.362+0000 7f75e35c0700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f75c005f270 0x7f75c0061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f75d80016e0 tx=0x7f75d8001500).stop 2024-01-24T14:10:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.362+0000 7f75e35c0700 1 -- 172.21.15.29:0/2914180003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75dc064760 msgr2=0x7f75dc14ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.362+0000 7f75e35c0700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75dc064760 0x7f75dc14ee40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f75d000e980 tx=0x7f75d0009060).stop 2024-01-24T14:10:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.363+0000 7f75e35c0700 1 -- 172.21.15.29:0/2914180003 shutdown_connections 2024-01-24T14:10:56.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.363+0000 7f75e35c0700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75dc0c8490 0x7f75dc0accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.363+0000 7f75e35c0700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f75c005f270 0x7f75c0061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.363+0000 7f75e35c0700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75dc064760 0x7f75dc14ee40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.363+0000 7f75e35c0700 1 --2- 172.21.15.29:0/2914180003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75dc0bfd40 0x7f75dc0ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.363+0000 7f75e35c0700 1 -- 172.21.15.29:0/2914180003 >> 172.21.15.29:0/2914180003 conn(0x7f75dc0cb8b0 msgr2=0x7f75dc1502a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.363+0000 7f75e35c0700 1 -- 172.21.15.29:0/2914180003 shutdown_connections 2024-01-24T14:10:56.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.363+0000 7f75e35c0700 1 -- 172.21.15.29:0/2914180003 wait complete. 2024-01-24T14:10:56.376 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T14:10:56.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.510+0000 7f33c5f24700 1 Processor -- start 2024-01-24T14:10:56.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.510+0000 7f33c5f24700 1 -- start start 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.510+0000 7f33c5f24700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33c00c8490 0x7f33c00a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.510+0000 7f33c5f24700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c0152240 con 0x7f33c0064760 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.511+0000 7f33c5f24700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c01586c0 con 0x7f33c00bfd40 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.511+0000 7f33c5f24700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c00aa670 con 0x7f33c00c8490 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.511+0000 7f33beffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33c00c8490 0x7f33c00a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.511+0000 7f33bf7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f33c0064760 0x7f33c0064b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59642/0 (socket says 172.21.15.29:59642) 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.511+0000 7f33bf7fe700 1 -- 172.21.15.29:0/434085455 learned_addr learned my addr 172.21.15.29:0/434085455 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.511+0000 7f33beffd700 1 -- 172.21.15.29:0/434085455 >> v1:172.21.15.163:6789/0 conn(0x7f33c00bfd40 legacy=0x7f33c00c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.511+0000 7f33beffd700 1 -- 172.21.15.29:0/434085455 >> v1:172.21.15.29:6789/0 conn(0x7f33c0064760 legacy=0x7f33c0064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.511+0000 7f33beffd700 1 -- 172.21.15.29:0/434085455 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c0157d90 con 0x7f33c00c8490 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.511+0000 7f33beffd700 1 --2- 172.21.15.29:0/434085455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c00c8490 0x7f33c00a9d90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f33b4009760 tx=0x7f33b4009060).ready entity=mon.0 client_cookie=ba080d07e503b59 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.511+0000 7f33be7fc700 1 -- 172.21.15.29:0/434085455 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400b610 con 0x7f33c00c8490 2024-01-24T14:10:56.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.511+0000 7f33be7fc700 1 -- 172.21.15.29:0/434085455 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b400b770 con 0x7f33c00c8490 2024-01-24T14:10:56.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.511+0000 7f33be7fc700 1 -- 172.21.15.29:0/434085455 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400baf0 con 0x7f33c00c8490 2024-01-24T14:10:56.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.512+0000 7f33c5f24700 1 -- 172.21.15.29:0/434085455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c00c8490 msgr2=0x7f33c00a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.512+0000 7f33c5f24700 1 --2- 172.21.15.29:0/434085455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c00c8490 0x7f33c00a9d90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f33b4009760 tx=0x7f33b4009060).stop 2024-01-24T14:10:56.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.512+0000 7f33c5f24700 1 -- 172.21.15.29:0/434085455 shutdown_connections 2024-01-24T14:10:56.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.512+0000 7f33c5f24700 1 --2- 172.21.15.29:0/434085455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c00c8490 0x7f33c00a9d90 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.512+0000 7f33c5f24700 1 -- 172.21.15.29:0/434085455 >> 172.21.15.29:0/434085455 conn(0x7f33c00cb8b0 msgr2=0x7f33c00c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:56.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.512+0000 7f33c5f24700 1 -- 172.21.15.29:0/434085455 shutdown_connections 2024-01-24T14:10:56.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.512+0000 7f33c5f24700 1 -- 172.21.15.29:0/434085455 wait complete. 2024-01-24T14:10:56.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.513+0000 7f33c5f24700 1 Processor -- start 2024-01-24T14:10:56.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.514+0000 7f33c5f24700 1 -- start start 2024-01-24T14:10:56.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.514+0000 7f33c5f24700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c0064760 0x7f33c014fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.514+0000 7f33c5f24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33c00bfd40 0x7f33c00c0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.514+0000 7f33c5f24700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33c00c8490 0x7f33c015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.514+0000 7f33c5f24700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c00afae0 con 0x7f33c0064760 2024-01-24T14:10:56.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.514+0000 7f33c5f24700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c015b700 con 0x7f33c00bfd40 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.514+0000 7f33c5f24700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c00af1b0 con 0x7f33c00c8490 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.514+0000 7f33bf7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c0064760 0x7f33c014fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33bf7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c0064760 0x7f33c014fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37940/0 (socket says 172.21.15.29:37940) 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33bf7fe700 1 -- 172.21.15.29:0/4243730404 learned_addr learned my addr 172.21.15.29:0/4243730404 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33beffd700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33c00bfd40 0x7f33c00c0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33bffff700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33c00c8490 0x7f33c015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33bf7fe700 1 -- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33c00c8490 msgr2=0x7f33c015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33bf7fe700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33c00c8490 0x7f33c015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33bf7fe700 1 -- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33c00bfd40 msgr2=0x7f33c00c0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33bf7fe700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33c00bfd40 0x7f33c00c0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33bf7fe700 1 -- 172.21.15.29:0/4243730404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b40092a0 con 0x7f33c0064760 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33bf7fe700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c0064760 0x7f33c014fa30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f33b0007fd0 tx=0x7f33b0007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33a77fe700 1 -- 172.21.15.29:0/4243730404 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b00072e0 con 0x7f33c0064760 2024-01-24T14:10:56.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33a77fe700 1 -- 172.21.15.29:0/4243730404 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b0007440 con 0x7f33c0064760 2024-01-24T14:10:56.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33c5f24700 1 -- 172.21.15.29:0/4243730404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c00af490 con 0x7f33c0064760 2024-01-24T14:10:56.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33a77fe700 1 -- 172.21.15.29:0/4243730404 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b00057a0 con 0x7f33c0064760 2024-01-24T14:10:56.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.515+0000 7f33c5f24700 1 -- 172.21.15.29:0/4243730404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c015b3d0 con 0x7f33c0064760 2024-01-24T14:10:56.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.516+0000 7f33c5f24700 1 -- 172.21.15.29:0/4243730404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c00475e0 con 0x7f33c0064760 2024-01-24T14:10:56.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.517+0000 7f33a77fe700 1 -- 172.21.15.29:0/4243730404 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f33b0005b90 con 0x7f33c0064760 2024-01-24T14:10:56.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.517+0000 7f33a77fe700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f33ac05f270 0x7f33ac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.517+0000 7f33a77fe700 1 -- 172.21.15.29:0/4243730404 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f33b0079cf0 con 0x7f33c0064760 2024-01-24T14:10:56.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.517+0000 7f33beffd700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f33ac05f270 0x7f33ac061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.518+0000 7f33beffd700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f33ac05f270 0x7f33ac061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f33b400b3d0 tx=0x7f33b4017000).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:56.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.522+0000 7f33a77fe700 1 -- 172.21.15.29:0/4243730404 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f33b004ad90 con 0x7f33c0064760 2024-01-24T14:10:56.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.699+0000 7f33c5f24700 1 -- 172.21.15.29:0/4243730404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f33c0158070 con 0x7f33c0064760 2024-01-24T14:10:56.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.700+0000 7f33a77fe700 1 -- 172.21.15.29:0/4243730404 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v68) v1 ==== 92+0+10 (secure 0 0 0) 0x7f33b004e3b0 con 0x7f33c0064760 2024-01-24T14:10:56.699 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T14:10:56.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.702+0000 7f33c5f24700 1 -- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f33ac05f270 msgr2=0x7f33ac061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.702+0000 7f33c5f24700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f33ac05f270 0x7f33ac061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f33b400b3d0 tx=0x7f33b4017000).stop 2024-01-24T14:10:56.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.702+0000 7f33c5f24700 1 -- 172.21.15.29:0/4243730404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c0064760 msgr2=0x7f33c014fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.702+0000 7f33c5f24700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c0064760 0x7f33c014fa30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f33b0007fd0 tx=0x7f33b0007df0).stop 2024-01-24T14:10:56.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.702+0000 7f33c5f24700 1 -- 172.21.15.29:0/4243730404 shutdown_connections 2024-01-24T14:10:56.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.702+0000 7f33c5f24700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33c00c8490 0x7f33c015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.702+0000 7f33c5f24700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f33ac05f270 0x7f33ac061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.702+0000 7f33c5f24700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33c0064760 0x7f33c014fa30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.702+0000 7f33c5f24700 1 --2- 172.21.15.29:0/4243730404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33c00bfd40 0x7f33c00c0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.702+0000 7f33c5f24700 1 -- 172.21.15.29:0/4243730404 >> 172.21.15.29:0/4243730404 conn(0x7f33c00cb8b0 msgr2=0x7f33c01560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:56.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.702+0000 7f33c5f24700 1 -- 172.21.15.29:0/4243730404 shutdown_connections 2024-01-24T14:10:56.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.702+0000 7f33c5f24700 1 -- 172.21.15.29:0/4243730404 wait complete. 2024-01-24T14:10:56.715 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-24T14:10:56.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.846+0000 7fef31710700 1 Processor -- start 2024-01-24T14:10:56.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.846+0000 7fef31710700 1 -- start start 2024-01-24T14:10:56.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef31710700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef2c062970 0x7fef2c062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef31710700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fef2c069dd0 0x7fef2c0a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef31710700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fef2c153a80 0x7fef2c153f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef31710700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2c0bd190 con 0x7fef2c153a80 2024-01-24T14:10:56.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef31710700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2c0c4e60 con 0x7fef2c069dd0 2024-01-24T14:10:56.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef31710700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c0a72a0 con 0x7fef2c062970 2024-01-24T14:10:56.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef2b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fef2c153a80 0x7fef2c153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef2b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fef2c153a80 0x7fef2c153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37950/0 (socket says 172.21.15.29:37950) 2024-01-24T14:10:56.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef2b7fe700 1 -- 172.21.15.29:0/1588828035 learned_addr learned my addr 172.21.15.29:0/1588828035 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:56.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef2a7fc700 1 --2- 172.21.15.29:0/1588828035 >> v2:172.21.15.163:3300/0 conn(0x7fef2c069dd0 0x7fef2c0a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef2affd700 1 --2- 172.21.15.29:0/1588828035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef2c062970 0x7fef2c062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef2b7fe700 1 -- 172.21.15.29:0/1588828035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef2c062970 msgr2=0x7fef2c062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:56.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef2b7fe700 1 --2- 172.21.15.29:0/1588828035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef2c062970 0x7fef2c062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef2b7fe700 1 -- 172.21.15.29:0/1588828035 >> v2:172.21.15.163:3300/0 conn(0x7fef2c069dd0 msgr2=0x7fef2c0a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef2b7fe700 1 --2- 172.21.15.29:0/1588828035 >> v2:172.21.15.163:3300/0 conn(0x7fef2c069dd0 0x7fef2c0a6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef2b7fe700 1 -- 172.21.15.29:0/1588828035 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c06a250 con 0x7fef2c153a80 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.847+0000 7fef2b7fe700 1 --2- 172.21.15.29:0/1588828035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef2c153a80 0x7fef2c153f00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fef2c06a940 tx=0x7fef24009840).ready entity=mon.0 client_cookie=d80dae74f43dbaed server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.848+0000 7fef29ffb700 1 -- 172.21.15.29:0/1588828035 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2400e070 con 0x7fef2c153a80 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.848+0000 7fef29ffb700 1 -- 172.21.15.29:0/1588828035 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2400ba50 con 0x7fef2c153a80 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.848+0000 7fef29ffb700 1 -- 172.21.15.29:0/1588828035 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2400bdd0 con 0x7fef2c153a80 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.848+0000 7fef31710700 1 -- 172.21.15.29:0/1588828035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef2c153a80 msgr2=0x7fef2c153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.848+0000 7fef31710700 1 --2- 172.21.15.29:0/1588828035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef2c153a80 0x7fef2c153f00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fef2c06a940 tx=0x7fef24009840).stop 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.848+0000 7fef31710700 1 -- 172.21.15.29:0/1588828035 shutdown_connections 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.848+0000 7fef31710700 1 --2- 172.21.15.29:0/1588828035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef2c062970 0x7fef2c062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.848+0000 7fef31710700 1 --2- 172.21.15.29:0/1588828035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef2c153a80 0x7fef2c153f00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.848+0000 7fef31710700 1 --2- 172.21.15.29:0/1588828035 >> v2:172.21.15.163:3300/0 conn(0x7fef2c069dd0 0x7fef2c0a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.848+0000 7fef31710700 1 -- 172.21.15.29:0/1588828035 >> 172.21.15.29:0/1588828035 conn(0x7fef2c0c85c0 msgr2=0x7fef2c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:56.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.848+0000 7fef31710700 1 -- 172.21.15.29:0/1588828035 shutdown_connections 2024-01-24T14:10:56.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.848+0000 7fef31710700 1 -- 172.21.15.29:0/1588828035 wait complete. 2024-01-24T14:10:56.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.850+0000 7fef31710700 1 Processor -- start 2024-01-24T14:10:56.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.850+0000 7fef31710700 1 -- start start 2024-01-24T14:10:56.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.850+0000 7fef31710700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef2c062970 0x7fef2c14c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.850+0000 7fef31710700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef2c069dd0 0x7fef2c0ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.850+0000 7fef31710700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef2c153a80 0x7fef2c0abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.850+0000 7fef31710700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c14cd50 con 0x7fef2c153a80 2024-01-24T14:10:56.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.850+0000 7fef31710700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c0bed30 con 0x7fef2c062970 2024-01-24T14:10:56.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.850+0000 7fef31710700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c0bf010 con 0x7fef2c069dd0 2024-01-24T14:10:56.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef2b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef2c153a80 0x7fef2c0abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef2b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef2c153a80 0x7fef2c0abea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37964/0 (socket says 172.21.15.29:37964) 2024-01-24T14:10:56.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef2b7fe700 1 -- 172.21.15.29:0/3790404418 learned_addr learned my addr 172.21.15.29:0/3790404418 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef2affd700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef2c062970 0x7fef2c14c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef2a7fc700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef2c069dd0 0x7fef2c0ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef2b7fe700 1 -- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef2c069dd0 msgr2=0x7fef2c0ab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef2b7fe700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef2c069dd0 0x7fef2c0ab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef2b7fe700 1 -- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef2c062970 msgr2=0x7fef2c14c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef2b7fe700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef2c062970 0x7fef2c14c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef2b7fe700 1 -- 172.21.15.29:0/3790404418 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef24009ae0 con 0x7fef2c153a80 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef2b7fe700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef2c153a80 0x7fef2c0abea0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fef24009fd0 tx=0x7fef2c0be230).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef12ffd700 1 -- 172.21.15.29:0/3790404418 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef240092e0 con 0x7fef2c153a80 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef12ffd700 1 -- 172.21.15.29:0/3790404418 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2401e470 con 0x7fef2c153a80 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef31710700 1 -- 172.21.15.29:0/3790404418 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2c0bf2f0 con 0x7fef2c153a80 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef12ffd700 1 -- 172.21.15.29:0/3790404418 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2401e810 con 0x7fef2c153a80 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.851+0000 7fef31710700 1 -- 172.21.15.29:0/3790404418 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2c0bf8b0 con 0x7fef2c153a80 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.853+0000 7fef31710700 1 -- 172.21.15.29:0/3790404418 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2c0475e0 con 0x7fef2c153a80 2024-01-24T14:10:56.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.853+0000 7fef12ffd700 1 -- 172.21.15.29:0/3790404418 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fef24003040 con 0x7fef2c153a80 2024-01-24T14:10:56.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.853+0000 7fef12ffd700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fef1805f270 0x7fef18061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:56.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.853+0000 7fef12ffd700 1 -- 172.21.15.29:0/3790404418 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6290+0+0 (secure 0 0 0) 0x7fef2407b840 con 0x7fef2c153a80 2024-01-24T14:10:56.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.854+0000 7fef2affd700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fef1805f270 0x7fef18061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:56.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.854+0000 7fef2affd700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fef1805f270 0x7fef18061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fef2000bbc0 tx=0x7fef20008040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:56.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:56.858+0000 7fef12ffd700 1 -- 172.21.15.29:0/3790404418 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fef2404c990 con 0x7fef2c153a80 2024-01-24T14:10:57.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.041+0000 7fef31710700 1 -- 172.21.15.29:0/3790404418 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fef2c063380 con 0x7fef2c153a80 2024-01-24T14:10:57.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.042+0000 7fef12ffd700 1 -- 172.21.15.29:0/3790404418 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) v1 ==== 84+0+143 (secure 0 0 0) 0x7fef24023070 con 0x7fef2c153a80 2024-01-24T14:10:57.041 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:10:57.041 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T14:10:57.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:57.041 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-01-24T14:10:57.041 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "device_health_metrics" 2024-01-24T14:10:57.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:10:57.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:10:57.042 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 2, 2024-01-24T14:10:57.042 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "rbd" 2024-01-24T14:10:57.042 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:10:57.042 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T14:10:57.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.044+0000 7fef31710700 1 -- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fef1805f270 msgr2=0x7fef18061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:57.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.044+0000 7fef31710700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fef1805f270 0x7fef18061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fef2000bbc0 tx=0x7fef20008040).stop 2024-01-24T14:10:57.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.044+0000 7fef31710700 1 -- 172.21.15.29:0/3790404418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef2c153a80 msgr2=0x7fef2c0abea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:57.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.044+0000 7fef31710700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef2c153a80 0x7fef2c0abea0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fef24009fd0 tx=0x7fef2c0be230).stop 2024-01-24T14:10:57.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.044+0000 7fef31710700 1 -- 172.21.15.29:0/3790404418 shutdown_connections 2024-01-24T14:10:57.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.044+0000 7fef31710700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fef1805f270 0x7fef18061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:57.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.044+0000 7fef31710700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef2c153a80 0x7fef2c0abea0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:57.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.044+0000 7fef31710700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef2c062970 0x7fef2c14c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:57.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.044+0000 7fef31710700 1 --2- 172.21.15.29:0/3790404418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef2c069dd0 0x7fef2c0ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:57.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.044+0000 7fef31710700 1 -- 172.21.15.29:0/3790404418 >> 172.21.15.29:0/3790404418 conn(0x7fef2c0c85c0 msgr2=0x7fef2c066100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:57.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.044+0000 7fef31710700 1 -- 172.21.15.29:0/3790404418 shutdown_connections 2024-01-24T14:10:57.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.044+0000 7fef31710700 1 -- 172.21.15.29:0/3790404418 wait complete. 2024-01-24T14:10:57.056 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-24T14:10:57.057 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2024-01-24T14:10:57.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.193+0000 7f7aebd24700 1 Processor -- start 2024-01-24T14:10:57.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.193+0000 7f7aebd24700 1 -- start start 2024-01-24T14:10:57.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.194+0000 7f7aebd24700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ae4154ec0 0x7f7ae41552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:57.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.194+0000 7f7aebd24700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ae4063d40 0x7f7ae4151350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:57.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.194+0000 7f7aebd24700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae4068540 con 0x7f7ae40638c0 2024-01-24T14:10:57.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.194+0000 7f7aebd24700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae40641c0 con 0x7f7ae4154ec0 2024-01-24T14:10:57.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.194+0000 7f7aebd24700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40689d0 con 0x7f7ae4063d40 2024-01-24T14:10:57.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.194+0000 7f7ae9ac0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ae4154ec0 0x7f7ae41552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:57.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.194+0000 7f7ae9ac0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ae4154ec0 0x7f7ae41552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37978/0 (socket says 172.21.15.29:37978) 2024-01-24T14:10:57.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.194+0000 7f7ae9ac0700 1 -- 172.21.15.29:0/2960917476 learned_addr learned my addr 172.21.15.29:0/2960917476 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:57.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.194+0000 7f7aea2c1700 1 --2- 172.21.15.29:0/2960917476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ae4063d40 0x7f7ae4151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:57.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7ae9ac0700 1 -- 172.21.15.29:0/2960917476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ae4063d40 msgr2=0x7f7ae4151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:57.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7ae9ac0700 1 --2- 172.21.15.29:0/2960917476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ae4063d40 0x7f7ae4151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7ae9ac0700 1 -- 172.21.15.29:0/2960917476 >> v1:172.21.15.163:6789/0 conn(0x7f7ae40638c0 legacy=0x7f7ae4066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7ae9ac0700 1 -- 172.21.15.29:0/2960917476 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae4067c10 con 0x7f7ae4154ec0 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7aea2c1700 1 --2- 172.21.15.29:0/2960917476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ae4063d40 0x7f7ae4151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7ae9ac0700 1 --2- 172.21.15.29:0/2960917476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ae4154ec0 0x7f7ae41552a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f7ae0006fd0 tx=0x7f7ae00090f0).ready entity=mon.0 client_cookie=9bb02b1526fadff5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7ae8abe700 1 -- 172.21.15.29:0/2960917476 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae000b600 con 0x7f7ae4154ec0 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7ae8abe700 1 -- 172.21.15.29:0/2960917476 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae000b760 con 0x7f7ae4154ec0 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7ae8abe700 1 -- 172.21.15.29:0/2960917476 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae000bae0 con 0x7f7ae4154ec0 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7aebd24700 1 -- 172.21.15.29:0/2960917476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ae4154ec0 msgr2=0x7f7ae41552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7aebd24700 1 --2- 172.21.15.29:0/2960917476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ae4154ec0 0x7f7ae41552a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f7ae0006fd0 tx=0x7f7ae00090f0).stop 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7aebd24700 1 -- 172.21.15.29:0/2960917476 shutdown_connections 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7aebd24700 1 --2- 172.21.15.29:0/2960917476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ae4063d40 0x7f7ae4151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.195+0000 7f7aebd24700 1 --2- 172.21.15.29:0/2960917476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ae4154ec0 0x7f7ae41552a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.196+0000 7f7aebd24700 1 -- 172.21.15.29:0/2960917476 >> 172.21.15.29:0/2960917476 conn(0x7f7ae40c8bd0 msgr2=0x7f7ae40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:57.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.196+0000 7f7aebd24700 1 -- 172.21.15.29:0/2960917476 shutdown_connections 2024-01-24T14:10:57.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.196+0000 7f7aebd24700 1 -- 172.21.15.29:0/2960917476 wait complete. 2024-01-24T14:10:57.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.197+0000 7f7aebd24700 1 Processor -- start 2024-01-24T14:10:57.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.197+0000 7f7aebd24700 1 -- start start 2024-01-24T14:10:57.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7aebd24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae40638c0 0x7f7ae40aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:57.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7aebd24700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ae4063d40 0x7f7ae4153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:57.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7aebd24700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ae4154ec0 0x7f7ae40a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:57.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7aebd24700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40a55c0 con 0x7f7ae4063d40 2024-01-24T14:10:57.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7aebd24700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40a49b0 con 0x7f7ae40638c0 2024-01-24T14:10:57.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7aebd24700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40a4c90 con 0x7f7ae4154ec0 2024-01-24T14:10:57.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ae92bf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ae4063d40 0x7f7ae4153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:57.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ae92bf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ae4063d40 0x7f7ae4153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37992/0 (socket says 172.21.15.29:37992) 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ae92bf700 1 -- 172.21.15.29:0/384546884 learned_addr learned my addr 172.21.15.29:0/384546884 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ae9ac0700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae40638c0 0x7f7ae40aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7aea2c1700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ae4154ec0 0x7f7ae40a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ae92bf700 1 -- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ae4154ec0 msgr2=0x7f7ae40a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ae92bf700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ae4154ec0 0x7f7ae40a9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ae92bf700 1 -- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae40638c0 msgr2=0x7f7ae40aaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ae92bf700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae40638c0 0x7f7ae40aaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ae92bf700 1 -- 172.21.15.29:0/384546884 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae0009370 con 0x7f7ae4063d40 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ae9ac0700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae40638c0 0x7f7ae40aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ae92bf700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ae4063d40 0x7f7ae4153350 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f7ad400c4c0 tx=0x7f7ad4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ad9ffb700 1 -- 172.21.15.29:0/384546884 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4007020 con 0x7f7ae4063d40 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.198+0000 7f7ad9ffb700 1 -- 172.21.15.29:0/384546884 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad4001e90 con 0x7f7ae4063d40 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.199+0000 7f7aebd24700 1 -- 172.21.15.29:0/384546884 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae40a4f70 con 0x7f7ae4063d40 2024-01-24T14:10:57.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.199+0000 7f7ad9ffb700 1 -- 172.21.15.29:0/384546884 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40073b0 con 0x7f7ae4063d40 2024-01-24T14:10:57.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.199+0000 7f7aebd24700 1 -- 172.21.15.29:0/384546884 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae40a9570 con 0x7f7ae4063d40 2024-01-24T14:10:57.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.200+0000 7f7aebd24700 1 -- 172.21.15.29:0/384546884 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae40475e0 con 0x7f7ae4063d40 2024-01-24T14:10:57.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.200+0000 7f7ad9ffb700 1 -- 172.21.15.29:0/384546884 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7ad40068b0 con 0x7f7ae4063d40 2024-01-24T14:10:57.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.201+0000 7f7ad9ffb700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ac805f240 0x7f7ac8061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:57.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.201+0000 7f7ad9ffb700 1 -- 172.21.15.29:0/384546884 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f7ad4079e70 con 0x7f7ae4063d40 2024-01-24T14:10:57.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.201+0000 7f7ae9ac0700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ac805f240 0x7f7ac8061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:57.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.201+0000 7f7ae9ac0700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ac805f240 0x7f7ac8061700 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f7ae40c3110 tx=0x7f7ae0017000).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:57.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.205+0000 7f7ad9ffb700 1 -- 172.21.15.29:0/384546884 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7ad404af10 con 0x7f7ae4063d40 2024-01-24T14:10:57.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:57.387+0000 7f7aebd24700 1 -- 172.21.15.29:0/384546884 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f7ae40a5700 con 0x7f7ae4063d40 2024-01-24T14:10:58.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.219+0000 7f7ad9ffb700 1 -- 172.21.15.29:0/384546884 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) v1 ==== 152+0+0 (secure 0 0 0) 0x7f7ad404e530 con 0x7f7ae4063d40 2024-01-24T14:10:58.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.223+0000 7f7aebd24700 1 -- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ac805f240 msgr2=0x7f7ac8061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:58.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.223+0000 7f7aebd24700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ac805f240 0x7f7ac8061700 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f7ae40c3110 tx=0x7f7ae0017000).stop 2024-01-24T14:10:58.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.223+0000 7f7aebd24700 1 -- 172.21.15.29:0/384546884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ae4063d40 msgr2=0x7f7ae4153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:58.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.223+0000 7f7aebd24700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ae4063d40 0x7f7ae4153350 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f7ad400c4c0 tx=0x7f7ad4007e00).stop 2024-01-24T14:10:58.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.223+0000 7f7aebd24700 1 -- 172.21.15.29:0/384546884 shutdown_connections 2024-01-24T14:10:58.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.223+0000 7f7aebd24700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ae4154ec0 0x7f7ae40a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:58.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.223+0000 7f7aebd24700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae40638c0 0x7f7ae40aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:58.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.223+0000 7f7aebd24700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7ac805f240 0x7f7ac8061700 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:58.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.223+0000 7f7aebd24700 1 --2- 172.21.15.29:0/384546884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ae4063d40 0x7f7ae4153350 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:58.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.223+0000 7f7aebd24700 1 -- 172.21.15.29:0/384546884 >> 172.21.15.29:0/384546884 conn(0x7f7ae40c8bd0 msgr2=0x7f7ae41557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:58.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.229+0000 7f7aebd24700 1 -- 172.21.15.29:0/384546884 shutdown_connections 2024-01-24T14:10:58.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.232+0000 7f7aebd24700 1 -- 172.21.15.29:0/384546884 wait complete. 2024-01-24T14:10:58.233 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-01-24T14:10:58.248 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-24T14:10:58.326 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:10:58.328 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:10:58.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.389+0000 7f7177551700 1 Processor -- start 2024-01-24T14:10:58.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.389+0000 7f7177551700 1 -- start start 2024-01-24T14:10:58.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.390+0000 7f7177551700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7170155030 0x7f71700c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:58.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.390+0000 7f7177551700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71700c3b60 con 0x7f7170150fe0 2024-01-24T14:10:58.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.390+0000 7f7177551700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71701558a0 con 0x7f7170150c00 2024-01-24T14:10:58.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.390+0000 7f7177551700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7170066960 con 0x7f7170155030 2024-01-24T14:10:58.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.390+0000 7f7174aec700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7170150fe0 0x7f7170154920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59652/0 (socket says 172.21.15.29:59652) 2024-01-24T14:10:58.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.390+0000 7f7174aec700 1 -- 172.21.15.29:0/3929109131 learned_addr learned my addr 172.21.15.29:0/3929109131 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:58.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.391+0000 7f7175aee700 1 --2- 172.21.15.29:0/3929109131 >> v2:172.21.15.163:3300/0 conn(0x7f7170155030 0x7f71700c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:58.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.391+0000 7f7167fff700 1 -- 172.21.15.29:0/3929109131 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837675289 0 0) 0x7f71700c3b60 con 0x7f7170150fe0 2024-01-24T14:10:58.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.391+0000 7f7167fff700 1 -- 172.21.15.29:0/3929109131 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f715c0033f0 con 0x7f7170150fe0 2024-01-24T14:10:58.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.391+0000 7f7175aee700 1 -- 172.21.15.29:0/3929109131 >> v1:172.21.15.163:6789/0 conn(0x7f7170150c00 legacy=0x7f71700c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:58.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.391+0000 7f7167fff700 1 -- 172.21.15.29:0/3929109131 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642829701 0 0) 0x7f71701558a0 con 0x7f7170150c00 2024-01-24T14:10:58.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.391+0000 7f7175aee700 1 -- 172.21.15.29:0/3929109131 >> v1:172.21.15.29:6789/0 conn(0x7f7170150fe0 legacy=0x7f7170154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:58.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.391+0000 7f7175aee700 1 -- 172.21.15.29:0/3929109131 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71700c2fc0 con 0x7f7170155030 2024-01-24T14:10:58.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.391+0000 7f7175aee700 1 --2- 172.21.15.29:0/3929109131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7170155030 0x7f71700c44c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f7168009e60 tx=0x7f7168009780).ready entity=mon.1 client_cookie=fe5aa78069ef03af server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:58.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.391+0000 7f7167fff700 1 -- 172.21.15.29:0/3929109131 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716800e070 con 0x7f7170155030 2024-01-24T14:10:58.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.391+0000 7f7167fff700 1 -- 172.21.15.29:0/3929109131 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716800b850 con 0x7f7170155030 2024-01-24T14:10:58.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.391+0000 7f7167fff700 1 -- 172.21.15.29:0/3929109131 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160001470 con 0x7f7170155030 2024-01-24T14:10:58.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.392+0000 7f7177551700 1 -- 172.21.15.29:0/3929109131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7170155030 msgr2=0x7f71700c44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:58.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.392+0000 7f7177551700 1 --2- 172.21.15.29:0/3929109131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7170155030 0x7f71700c44c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f7168009e60 tx=0x7f7168009780).stop 2024-01-24T14:10:58.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.392+0000 7f7177551700 1 -- 172.21.15.29:0/3929109131 shutdown_connections 2024-01-24T14:10:58.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.392+0000 7f7177551700 1 --2- 172.21.15.29:0/3929109131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7170155030 0x7f71700c44c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:58.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.392+0000 7f7177551700 1 -- 172.21.15.29:0/3929109131 >> 172.21.15.29:0/3929109131 conn(0x7f71700638c0 msgr2=0x7f7170063cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:58.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.392+0000 7f7177551700 1 -- 172.21.15.29:0/3929109131 shutdown_connections 2024-01-24T14:10:58.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.392+0000 7f7177551700 1 -- 172.21.15.29:0/3929109131 wait complete. 2024-01-24T14:10:58.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.393+0000 7f7177551700 1 Processor -- start 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.393+0000 7f7177551700 1 -- start start 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.394+0000 7f7177551700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7170150c00 0x7f7170152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.394+0000 7f7177551700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7170150fe0 0x7f7170161c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.394+0000 7f7177551700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7170155030 0x7f7170141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.394+0000 7f7177551700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717014e2a0 con 0x7f7170155030 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.394+0000 7f7177551700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71701524b0 con 0x7f7170150fe0 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.394+0000 7f7177551700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7170162140 con 0x7f7170150c00 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.394+0000 7f7175aee700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7170155030 0x7f7170141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.394+0000 7f7175aee700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7170155030 0x7f7170141060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38006/0 (socket says 172.21.15.29:38006) 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.394+0000 7f7175aee700 1 -- 172.21.15.29:0/3295576050 learned_addr learned my addr 172.21.15.29:0/3295576050 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f7174aec700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7170150fe0 0x7f7170161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f71752ed700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7170150c00 0x7f7170152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f7175aee700 1 -- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7170150c00 msgr2=0x7f7170152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:58.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f7175aee700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7170150c00 0x7f7170152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:58.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f7175aee700 1 -- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7170150fe0 msgr2=0x7f7170161c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:58.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f7175aee700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7170150fe0 0x7f7170161c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:58.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f7175aee700 1 -- 172.21.15.29:0/3295576050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7168009a20 con 0x7f7170155030 2024-01-24T14:10:58.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f7174aec700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7170150fe0 0x7f7170161c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:10:58.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f7175aee700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7170155030 0x7f7170141060 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f71700c8c60 tx=0x7f7168009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:58.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f71657fa700 1 -- 172.21.15.29:0/3295576050 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716801e480 con 0x7f7170155030 2024-01-24T14:10:58.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f71657fa700 1 -- 172.21.15.29:0/3295576050 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716801e5e0 con 0x7f7170155030 2024-01-24T14:10:58.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f7177551700 1 -- 172.21.15.29:0/3295576050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7170141550 con 0x7f7170155030 2024-01-24T14:10:58.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f71657fa700 1 -- 172.21.15.29:0/3295576050 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716801e980 con 0x7f7170155030 2024-01-24T14:10:58.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.395+0000 7f7177551700 1 -- 172.21.15.29:0/3295576050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7170151b80 con 0x7f7170155030 2024-01-24T14:10:58.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.396+0000 7f7177551700 1 -- 172.21.15.29:0/3295576050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71700475e0 con 0x7f7170155030 2024-01-24T14:10:58.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.397+0000 7f71657fa700 1 -- 172.21.15.29:0/3295576050 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f716801ecf0 con 0x7f7170155030 2024-01-24T14:10:58.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.397+0000 7f71657fa700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f715405f270 0x7f7154061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:58.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.397+0000 7f71657fa700 1 -- 172.21.15.29:0/3295576050 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6660+0+0 (secure 0 0 0) 0x7f716807c810 con 0x7f7170155030 2024-01-24T14:10:58.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.398+0000 7f71752ed700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f715405f270 0x7f7154061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:58.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.398+0000 7f71752ed700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f715405f270 0x7f7154061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f716c0036f0 tx=0x7f716c009040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:58.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.402+0000 7f71657fa700 1 -- 172.21.15.29:0/3295576050 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f716804d740 con 0x7f7170155030 2024-01-24T14:10:58.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:58.584+0000 7f7177551700 1 -- 172.21.15.29:0/3295576050 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f71700475e0 con 0x7f7170155030 2024-01-24T14:10:59.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.224+0000 7f71657fa700 1 -- 172.21.15.29:0/3295576050 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7168050d60 con 0x7f7170155030 2024-01-24T14:10:59.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.236+0000 7f7177551700 1 -- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f715405f270 msgr2=0x7f7154061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:59.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.236+0000 7f7177551700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f715405f270 0x7f7154061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f716c0036f0 tx=0x7f716c009040).stop 2024-01-24T14:10:59.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.236+0000 7f7177551700 1 -- 172.21.15.29:0/3295576050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7170155030 msgr2=0x7f7170141060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:59.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.236+0000 7f7177551700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7170155030 0x7f7170141060 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f71700c8c60 tx=0x7f7168009780).stop 2024-01-24T14:10:59.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.241+0000 7f7177551700 1 -- 172.21.15.29:0/3295576050 shutdown_connections 2024-01-24T14:10:59.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.241+0000 7f7177551700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f715405f270 0x7f7154061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.241+0000 7f7177551700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7170155030 0x7f7170141060 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.241+0000 7f7177551700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7170150c00 0x7f7170152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.241+0000 7f7177551700 1 --2- 172.21.15.29:0/3295576050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7170150fe0 0x7f7170161c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.241+0000 7f7177551700 1 -- 172.21.15.29:0/3295576050 >> 172.21.15.29:0/3295576050 conn(0x7f71700638c0 msgr2=0x7f7170154600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:59.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.245+0000 7f7177551700 1 -- 172.21.15.29:0/3295576050 shutdown_connections 2024-01-24T14:10:59.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.246+0000 7f7177551700 1 -- 172.21.15.29:0/3295576050 wait complete. 2024-01-24T14:10:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-01-24T14:10:59.272 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-24T14:10:59.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.453+0000 7fdf6c61d700 1 Processor -- start 2024-01-24T14:10:59.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.453+0000 7fdf6c61d700 1 -- start start 2024-01-24T14:10:59.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6c61d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdf64154100 0x7fdf640c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:59.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6c61d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdf64062ec0 0x7fdf64065420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:59.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6c61d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf640a6d80 con 0x7fdf640a7210 2024-01-24T14:10:59.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6c61d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640bcaf0 con 0x7fdf64154100 2024-01-24T14:10:59.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6c61d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640bcd90 con 0x7fdf64062ec0 2024-01-24T14:10:59.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6a3b9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdf64154100 0x7fdf640c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:59.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6a3b9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdf64154100 0x7fdf640c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38018/0 (socket says 172.21.15.29:38018) 2024-01-24T14:10:59.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6a3b9700 1 -- 172.21.15.29:0/3606490246 learned_addr learned my addr 172.21.15.29:0/3606490246 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:59.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6abba700 1 --2- 172.21.15.29:0/3606490246 >> v2:172.21.15.163:3300/0 conn(0x7fdf64062ec0 0x7fdf64065420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:59.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6a3b9700 1 -- 172.21.15.29:0/3606490246 >> v2:172.21.15.163:3300/0 conn(0x7fdf64062ec0 msgr2=0x7fdf64065420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:59.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6a3b9700 1 --2- 172.21.15.29:0/3606490246 >> v2:172.21.15.163:3300/0 conn(0x7fdf64062ec0 0x7fdf64065420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6a3b9700 1 -- 172.21.15.29:0/3606490246 >> v1:172.21.15.163:6789/0 conn(0x7fdf640a7210 legacy=0x7fdf640627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.454+0000 7fdf6a3b9700 1 -- 172.21.15.29:0/3606490246 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640bd070 con 0x7fdf64154100 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.455+0000 7fdf6a3b9700 1 --2- 172.21.15.29:0/3606490246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf64154100 0x7fdf640c8200 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fdf640c2080 tx=0x7fdf600090f0).ready entity=mon.0 client_cookie=61012ceb2454e8ad server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.455+0000 7fdf693b7700 1 -- 172.21.15.29:0/3606490246 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000b5a0 con 0x7fdf64154100 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.455+0000 7fdf693b7700 1 -- 172.21.15.29:0/3606490246 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6000b700 con 0x7fdf64154100 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.455+0000 7fdf693b7700 1 -- 172.21.15.29:0/3606490246 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000ba80 con 0x7fdf64154100 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.455+0000 7fdf6c61d700 1 -- 172.21.15.29:0/3606490246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf64154100 msgr2=0x7fdf640c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.455+0000 7fdf6c61d700 1 --2- 172.21.15.29:0/3606490246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf64154100 0x7fdf640c8200 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fdf640c2080 tx=0x7fdf600090f0).stop 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.455+0000 7fdf6c61d700 1 -- 172.21.15.29:0/3606490246 shutdown_connections 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.455+0000 7fdf6c61d700 1 --2- 172.21.15.29:0/3606490246 >> v2:172.21.15.163:3300/0 conn(0x7fdf64062ec0 0x7fdf64065420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.455+0000 7fdf6c61d700 1 --2- 172.21.15.29:0/3606490246 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf64154100 0x7fdf640c8200 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.455+0000 7fdf6c61d700 1 -- 172.21.15.29:0/3606490246 >> 172.21.15.29:0/3606490246 conn(0x7fdf64064390 msgr2=0x7fdf640c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.455+0000 7fdf6c61d700 1 -- 172.21.15.29:0/3606490246 shutdown_connections 2024-01-24T14:10:59.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.455+0000 7fdf6c61d700 1 -- 172.21.15.29:0/3606490246 wait complete. 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.457+0000 7fdf6c61d700 1 Processor -- start 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.457+0000 7fdf6c61d700 1 -- start start 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.457+0000 7fdf6c61d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf64062ec0 0x7fdf640a8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.457+0000 7fdf6c61d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf640a7210 0x7fdf6414e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.457+0000 7fdf6c61d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf64154100 0x7fdf640c0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.457+0000 7fdf6c61d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640a93a0 con 0x7fdf64062ec0 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.457+0000 7fdf6c61d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640a9680 con 0x7fdf640a7210 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.457+0000 7fdf6c61d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640a9960 con 0x7fdf64154100 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf6a3b9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf64062ec0 0x7fdf640a8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf6a3b9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf64062ec0 0x7fdf640a8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38032/0 (socket says 172.21.15.29:38032) 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf6a3b9700 1 -- 172.21.15.29:0/2121702713 learned_addr learned my addr 172.21.15.29:0/2121702713 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf69bb8700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf640a7210 0x7fdf6414e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:59.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf6abba700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf64154100 0x7fdf640c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:59.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf6a3b9700 1 -- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf64154100 msgr2=0x7fdf640c0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:59.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf6a3b9700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf64154100 0x7fdf640c0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf6a3b9700 1 -- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf640a7210 msgr2=0x7fdf6414e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:59.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf6a3b9700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf640a7210 0x7fdf6414e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf6a3b9700 1 -- 172.21.15.29:0/2121702713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf60009370 con 0x7fdf64062ec0 2024-01-24T14:10:59.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf6a3b9700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf64062ec0 0x7fdf640a8aa0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fdf60009130 tx=0x7fdf60015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:59.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf5a7fc700 1 -- 172.21.15.29:0/2121702713 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000b870 con 0x7fdf64062ec0 2024-01-24T14:10:59.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf5a7fc700 1 -- 172.21.15.29:0/2121702713 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6000b9d0 con 0x7fdf64062ec0 2024-01-24T14:10:59.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf6c61d700 1 -- 172.21.15.29:0/2121702713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf640a9c40 con 0x7fdf64062ec0 2024-01-24T14:10:59.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf5a7fc700 1 -- 172.21.15.29:0/2121702713 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6001b660 con 0x7fdf64062ec0 2024-01-24T14:10:59.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.458+0000 7fdf6c61d700 1 -- 172.21.15.29:0/2121702713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf640aa440 con 0x7fdf64062ec0 2024-01-24T14:10:59.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.459+0000 7fdf6c61d700 1 -- 172.21.15.29:0/2121702713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf640aa290 con 0x7fdf64062ec0 2024-01-24T14:10:59.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.460+0000 7fdf5a7fc700 1 -- 172.21.15.29:0/2121702713 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdf6001b9d0 con 0x7fdf64062ec0 2024-01-24T14:10:59.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.460+0000 7fdf5a7fc700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdf4805f270 0x7fdf48061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:59.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.460+0000 7fdf5a7fc700 1 -- 172.21.15.29:0/2121702713 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7042+0+0 (secure 0 0 0) 0x7fdf6007c010 con 0x7fdf64062ec0 2024-01-24T14:10:59.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.461+0000 7fdf69bb8700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdf4805f270 0x7fdf48061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:59.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.461+0000 7fdf69bb8700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdf4805f270 0x7fdf48061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fdf64153b00 tx=0x7fdf54007d20).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:59.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.465+0000 7fdf5a7fc700 1 -- 172.21.15.29:0/2121702713 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdf6004cdc0 con 0x7fdf64062ec0 2024-01-24T14:10:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.692+0000 7fdf6c61d700 1 -- 172.21.15.29:0/2121702713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdf640475e0 con 0x7fdf64062ec0 2024-01-24T14:10:59.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.699+0000 7fdf5a7fc700 1 -- 172.21.15.29:0/2121702713 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdf600290c0 con 0x7fdf64062ec0 2024-01-24T14:10:59.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.703+0000 7fdf6c61d700 1 -- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdf4805f270 msgr2=0x7fdf48061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:59.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.704+0000 7fdf6c61d700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdf4805f270 0x7fdf48061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fdf64153b00 tx=0x7fdf54007d20).stop 2024-01-24T14:10:59.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.704+0000 7fdf6c61d700 1 -- 172.21.15.29:0/2121702713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf64062ec0 msgr2=0x7fdf640a8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:59.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.704+0000 7fdf6c61d700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf64062ec0 0x7fdf640a8aa0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fdf60009130 tx=0x7fdf60015000).stop 2024-01-24T14:10:59.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.708+0000 7fdf6c61d700 1 -- 172.21.15.29:0/2121702713 shutdown_connections 2024-01-24T14:10:59.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.708+0000 7fdf6c61d700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf64154100 0x7fdf640c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.708+0000 7fdf6c61d700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdf4805f270 0x7fdf48061730 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0x7fdf64153b00 tx=0x7fdf54007d20).stop 2024-01-24T14:10:59.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.708+0000 7fdf6c61d700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf64062ec0 0x7fdf640a8aa0 secure :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0x7fdf60009130 tx=0x7fdf60015000).stop 2024-01-24T14:10:59.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.708+0000 7fdf6c61d700 1 --2- 172.21.15.29:0/2121702713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf640a7210 0x7fdf6414e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.708+0000 7fdf6c61d700 1 -- 172.21.15.29:0/2121702713 >> 172.21.15.29:0/2121702713 conn(0x7fdf64064390 msgr2=0x7fdf640c3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:59.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.713+0000 7fdf6c61d700 1 -- 172.21.15.29:0/2121702713 shutdown_connections 2024-01-24T14:10:59.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.714+0000 7fdf6c61d700 1 -- 172.21.15.29:0/2121702713 wait complete. 2024-01-24T14:10:59.728 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 8 and data pool 9 2024-01-24T14:10:59.737 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:10:59.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.866+0000 7f32778b9700 1 Processor -- start 2024-01-24T14:10:59.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.866+0000 7f32778b9700 1 -- start start 2024-01-24T14:10:59.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.867+0000 7f32778b9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f327014b300 0x7f3270068dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:59.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.867+0000 7f32778b9700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f327006a1c0 con 0x7f3270062ec0 2024-01-24T14:10:59.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.867+0000 7f32778b9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3270069920 con 0x7f327014b6e0 2024-01-24T14:10:59.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.867+0000 7f32778b9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3270069c00 con 0x7f327014b300 2024-01-24T14:10:59.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.867+0000 7f3275e56700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3270062ec0 0x7f32700c1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59664/0 (socket says 172.21.15.29:59664) 2024-01-24T14:10:59.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.867+0000 7f3275e56700 1 -- 172.21.15.29:0/4241645708 learned_addr learned my addr 172.21.15.29:0/4241645708 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:10:59.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.867+0000 7f3275655700 1 --2- 172.21.15.29:0/4241645708 >> v2:172.21.15.163:3300/0 conn(0x7f327014b300 0x7f3270068dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:59.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.868+0000 7f3267fff700 1 -- 172.21.15.29:0/4241645708 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223096326 0 0) 0x7f327006a1c0 con 0x7f3270062ec0 2024-01-24T14:10:59.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.868+0000 7f3267fff700 1 -- 172.21.15.29:0/4241645708 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f325c0033f0 con 0x7f3270062ec0 2024-01-24T14:10:59.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.868+0000 7f3275655700 1 -- 172.21.15.29:0/4241645708 >> v1:172.21.15.163:6789/0 conn(0x7f327014b6e0 legacy=0x7f32700627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:59.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.868+0000 7f3275655700 1 -- 172.21.15.29:0/4241645708 >> v1:172.21.15.29:6789/0 conn(0x7f3270062ec0 legacy=0x7f32700c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:59.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.868+0000 7f3275655700 1 -- 172.21.15.29:0/4241645708 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3270069ee0 con 0x7f327014b300 2024-01-24T14:10:59.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.868+0000 7f3275655700 1 --2- 172.21.15.29:0/4241645708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f327014b300 0x7f3270068dd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f3260009760 tx=0x7f3260009060).ready entity=mon.1 client_cookie=4a7ce85af3030e4f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:59.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.868+0000 7f3267fff700 1 -- 172.21.15.29:0/4241645708 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600090d0 con 0x7f327014b300 2024-01-24T14:10:59.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.868+0000 7f3267fff700 1 -- 172.21.15.29:0/4241645708 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326000f510 con 0x7f327014b300 2024-01-24T14:10:59.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.868+0000 7f3267fff700 1 -- 172.21.15.29:0/4241645708 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326000f890 con 0x7f327014b300 2024-01-24T14:10:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.869+0000 7f32778b9700 1 -- 172.21.15.29:0/4241645708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f327014b300 msgr2=0x7f3270068dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.869+0000 7f32778b9700 1 --2- 172.21.15.29:0/4241645708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f327014b300 0x7f3270068dd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f3260009760 tx=0x7f3260009060).stop 2024-01-24T14:10:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.869+0000 7f32778b9700 1 -- 172.21.15.29:0/4241645708 shutdown_connections 2024-01-24T14:10:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.869+0000 7f32778b9700 1 --2- 172.21.15.29:0/4241645708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f327014b300 0x7f3270068dd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.869+0000 7f32778b9700 1 -- 172.21.15.29:0/4241645708 >> 172.21.15.29:0/4241645708 conn(0x7f32700c69f0 msgr2=0x7f32700c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:10:59.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.869+0000 7f32778b9700 1 -- 172.21.15.29:0/4241645708 shutdown_connections 2024-01-24T14:10:59.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.869+0000 7f32778b9700 1 -- 172.21.15.29:0/4241645708 wait complete. 2024-01-24T14:10:59.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.870+0000 7f32778b9700 1 Processor -- start 2024-01-24T14:10:59.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.870+0000 7f32778b9700 1 -- start start 2024-01-24T14:10:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.871+0000 7f32778b9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3270062ec0 0x7f32701577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.871+0000 7f32778b9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f327014b300 0x7f32700c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.871+0000 7f32778b9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32701410f0 0x7f32700c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.871+0000 7f32778b9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32700c68a0 con 0x7f3270062ec0 2024-01-24T14:10:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.871+0000 7f32778b9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32700c5760 con 0x7f327014b300 2024-01-24T14:10:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.871+0000 7f32778b9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32700c49a0 con 0x7f32701410f0 2024-01-24T14:10:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.871+0000 7f3275655700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3270062ec0 0x7f32701577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.871+0000 7f3275655700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3270062ec0 0x7f32701577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38044/0 (socket says 172.21.15.29:38044) 2024-01-24T14:10:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.871+0000 7f3275655700 1 -- 172.21.15.29:0/1066096779 learned_addr learned my addr 172.21.15.29:0/1066096779 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:10:59.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.871+0000 7f3274e54700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f327014b300 0x7f32700c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.871+0000 7f3275e56700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32701410f0 0x7f32700c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.872+0000 7f3275655700 1 -- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32701410f0 msgr2=0x7f32700c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.872+0000 7f3275655700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32701410f0 0x7f32700c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.872+0000 7f3275655700 1 -- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f327014b300 msgr2=0x7f32700c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:10:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.872+0000 7f3275655700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f327014b300 0x7f32700c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:10:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.872+0000 7f3275655700 1 -- 172.21.15.29:0/1066096779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600092a0 con 0x7f3270062ec0 2024-01-24T14:10:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.872+0000 7f3275655700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3270062ec0 0x7f32701577f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f32600091d0 tx=0x7f3260017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.872+0000 7f32657fa700 1 -- 172.21.15.29:0/1066096779 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326000f890 con 0x7f3270062ec0 2024-01-24T14:10:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.872+0000 7f32657fa700 1 -- 172.21.15.29:0/1066096779 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326000fc50 con 0x7f3270062ec0 2024-01-24T14:10:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.872+0000 7f32657fa700 1 -- 172.21.15.29:0/1066096779 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326001b410 con 0x7f3270062ec0 2024-01-24T14:10:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.872+0000 7f32778b9700 1 -- 172.21.15.29:0/1066096779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32700c4c80 con 0x7f3270062ec0 2024-01-24T14:10:59.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.872+0000 7f32778b9700 1 -- 172.21.15.29:0/1066096779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32700c5240 con 0x7f3270062ec0 2024-01-24T14:10:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.873+0000 7f32778b9700 1 -- 172.21.15.29:0/1066096779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32700475e0 con 0x7f3270062ec0 2024-01-24T14:10:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.874+0000 7f32657fa700 1 -- 172.21.15.29:0/1066096779 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f326001b800 con 0x7f3270062ec0 2024-01-24T14:10:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.874+0000 7f32657fa700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f325405f270 0x7f3254061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:10:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.874+0000 7f32657fa700 1 -- 172.21.15.29:0/1066096779 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f326007c0a0 con 0x7f3270062ec0 2024-01-24T14:10:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.875+0000 7f3274e54700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f325405f270 0x7f3254061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:10:59.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.875+0000 7f3274e54700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f325405f270 0x7f3254061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f326c003860 tx=0x7f326c007d40).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:10:59.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:10:59.879+0000 7f32657fa700 1 -- 172.21.15.29:0/1066096779 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f326004cdf0 con 0x7f3270062ec0 2024-01-24T14:11:00.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.061+0000 7f32778b9700 1 -- 172.21.15.29:0/1066096779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32700a9500 con 0x7f3270062ec0 2024-01-24T14:11:00.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.062+0000 7f32657fa700 1 -- 172.21.15.29:0/1066096779 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+18183 (secure 0 0 0) 0x7f326001f090 con 0x7f3270062ec0 2024-01-24T14:11:00.062 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:11:00.063 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":71,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:10:59.693584+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:10:57.425223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-24T14:10:58.622652+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:11:00.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.064+0000 7f32778b9700 1 -- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f325405f270 msgr2=0x7f3254061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:00.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.064+0000 7f32778b9700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f325405f270 0x7f3254061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f326c003860 tx=0x7f326c007d40).stop 2024-01-24T14:11:00.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.064+0000 7f32778b9700 1 -- 172.21.15.29:0/1066096779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3270062ec0 msgr2=0x7f32701577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:00.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.064+0000 7f32778b9700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3270062ec0 0x7f32701577f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f32600091d0 tx=0x7f3260017000).stop 2024-01-24T14:11:00.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.064+0000 7f32778b9700 1 -- 172.21.15.29:0/1066096779 shutdown_connections 2024-01-24T14:11:00.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.064+0000 7f32778b9700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32701410f0 0x7f32700c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.064+0000 7f32778b9700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f325405f270 0x7f3254061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.064+0000 7f32778b9700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3270062ec0 0x7f32701577f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.064+0000 7f32778b9700 1 --2- 172.21.15.29:0/1066096779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f327014b300 0x7f32700c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.064+0000 7f32778b9700 1 -- 172.21.15.29:0/1066096779 >> 172.21.15.29:0/1066096779 conn(0x7f32700c69f0 msgr2=0x7f327014a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:00.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.064+0000 7f32778b9700 1 -- 172.21.15.29:0/1066096779 shutdown_connections 2024-01-24T14:11:00.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.064+0000 7f32778b9700 1 -- 172.21.15.29:0/1066096779 wait complete. 2024-01-24T14:11:00.079 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:11:00.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.209+0000 7f07db54f700 1 Processor -- start 2024-01-24T14:11:00.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.209+0000 7f07db54f700 1 -- start start 2024-01-24T14:11:00.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.210+0000 7f07db54f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07d4066b40 0x7f07d4062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:00.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.210+0000 7f07db54f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d40bcf60 con 0x7f07d4158600 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.210+0000 7f07db54f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d40a6ec0 con 0x7f07d4062970 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.210+0000 7f07db54f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40bdba0 con 0x7f07d4066b40 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.210+0000 7f07d9aec700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f07d4158600 0x7f07d4154d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59668/0 (socket says 172.21.15.29:59668) 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.210+0000 7f07d9aec700 1 -- 172.21.15.29:0/3554183548 learned_addr learned my addr 172.21.15.29:0/3554183548 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.210+0000 7f07d8aea700 1 --2- 172.21.15.29:0/3554183548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07d4066b40 0x7f07d4062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.211+0000 7f07cbfff700 1 -- 172.21.15.29:0/3554183548 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013024304 0 0) 0x7f07d40bcf60 con 0x7f07d4158600 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.211+0000 7f07cbfff700 1 -- 172.21.15.29:0/3554183548 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07c00033f0 con 0x7f07d4158600 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.211+0000 7f07cbfff700 1 -- 172.21.15.29:0/3554183548 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373194813 0 0) 0x7f07d40a6ec0 con 0x7f07d4062970 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.211+0000 7f07cbfff700 1 -- 172.21.15.29:0/3554183548 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07d40bcf60 con 0x7f07d4062970 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.211+0000 7f07d8aea700 1 -- 172.21.15.29:0/3554183548 >> v1:172.21.15.163:6789/0 conn(0x7f07d4062970 legacy=0x7f07d40c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.211+0000 7f07d8aea700 1 -- 172.21.15.29:0/3554183548 >> v1:172.21.15.29:6789/0 conn(0x7f07d4158600 legacy=0x7f07d4154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.211+0000 7f07d8aea700 1 -- 172.21.15.29:0/3554183548 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d40a71a0 con 0x7f07d4066b40 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.211+0000 7f07d8aea700 1 --2- 172.21.15.29:0/3554183548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07d4066b40 0x7f07d4062d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f07d0009790 tx=0x7f07d0009060).ready entity=mon.2 client_cookie=e0d0b3adc8497164 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:00.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.211+0000 7f07cbfff700 1 -- 172.21.15.29:0/3554183548 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d000f510 con 0x7f07d4066b40 2024-01-24T14:11:00.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.211+0000 7f07cbfff700 1 -- 172.21.15.29:0/3554183548 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d000f670 con 0x7f07d4066b40 2024-01-24T14:11:00.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.211+0000 7f07cbfff700 1 -- 172.21.15.29:0/3554183548 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d000f9f0 con 0x7f07d4066b40 2024-01-24T14:11:00.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.212+0000 7f07db54f700 1 -- 172.21.15.29:0/3554183548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07d4066b40 msgr2=0x7f07d4062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:00.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.212+0000 7f07db54f700 1 --2- 172.21.15.29:0/3554183548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07d4066b40 0x7f07d4062d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f07d0009790 tx=0x7f07d0009060).stop 2024-01-24T14:11:00.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.212+0000 7f07db54f700 1 -- 172.21.15.29:0/3554183548 shutdown_connections 2024-01-24T14:11:00.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.212+0000 7f07db54f700 1 --2- 172.21.15.29:0/3554183548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07d4066b40 0x7f07d4062d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.212+0000 7f07db54f700 1 -- 172.21.15.29:0/3554183548 >> 172.21.15.29:0/3554183548 conn(0x7f07d40c8580 msgr2=0x7f07d40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:00.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.212+0000 7f07db54f700 1 -- 172.21.15.29:0/3554183548 shutdown_connections 2024-01-24T14:11:00.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.212+0000 7f07db54f700 1 -- 172.21.15.29:0/3554183548 wait complete. 2024-01-24T14:11:00.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.213+0000 7f07db54f700 1 Processor -- start 2024-01-24T14:11:00.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.213+0000 7f07db54f700 1 -- start start 2024-01-24T14:11:00.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.214+0000 7f07db54f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07d4062970 0x7f07d40c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:00.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.214+0000 7f07db54f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07d4066b40 0x7f07d40aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:00.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.214+0000 7f07db54f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07d4158600 0x7f07d40aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:00.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.214+0000 7f07db54f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40a7480 con 0x7f07d4158600 2024-01-24T14:11:00.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.214+0000 7f07db54f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40ab480 con 0x7f07d4066b40 2024-01-24T14:11:00.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.214+0000 7f07db54f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40c66d0 con 0x7f07d4062970 2024-01-24T14:11:00.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.214+0000 7f07d9aec700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07d4158600 0x7f07d40aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:00.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.214+0000 7f07d9aec700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07d4158600 0x7f07d40aaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38058/0 (socket says 172.21.15.29:38058) 2024-01-24T14:11:00.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.214+0000 7f07d9aec700 1 -- 172.21.15.29:0/3468546444 learned_addr learned my addr 172.21.15.29:0/3468546444 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:00.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.214+0000 7f07d8aea700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07d4066b40 0x7f07d40aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:00.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.214+0000 7f07d92eb700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07d4062970 0x7f07d40c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:00.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.215+0000 7f07d9aec700 1 -- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07d4062970 msgr2=0x7f07d40c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:00.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.215+0000 7f07d9aec700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07d4062970 0x7f07d40c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.215+0000 7f07d9aec700 1 -- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07d4066b40 msgr2=0x7f07d40aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:00.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.215+0000 7f07d9aec700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07d4066b40 0x7f07d40aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.215+0000 7f07d9aec700 1 -- 172.21.15.29:0/3468546444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d00092a0 con 0x7f07d4158600 2024-01-24T14:11:00.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.215+0000 7f07d9aec700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07d4158600 0x7f07d40aaf90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f07cc001560 tx=0x7f07cc0015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:00.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.215+0000 7f07c97fa700 1 -- 172.21.15.29:0/3468546444 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc005d30 con 0x7f07d4158600 2024-01-24T14:11:00.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.215+0000 7f07c97fa700 1 -- 172.21.15.29:0/3468546444 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc005e90 con 0x7f07d4158600 2024-01-24T14:11:00.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.215+0000 7f07db54f700 1 -- 172.21.15.29:0/3468546444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d413c230 con 0x7f07d4158600 2024-01-24T14:11:00.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.215+0000 7f07c97fa700 1 -- 172.21.15.29:0/3468546444 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0072e0 con 0x7f07d4158600 2024-01-24T14:11:00.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.215+0000 7f07db54f700 1 -- 172.21.15.29:0/3468546444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d40c5da0 con 0x7f07d4158600 2024-01-24T14:11:00.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.216+0000 7f07db54f700 1 -- 172.21.15.29:0/3468546444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d40475e0 con 0x7f07d4158600 2024-01-24T14:11:00.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.216+0000 7f07c97fa700 1 -- 172.21.15.29:0/3468546444 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f07cc007650 con 0x7f07d4158600 2024-01-24T14:11:00.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.217+0000 7f07c97fa700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f07b805f2a0 0x7f07b8061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:00.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.217+0000 7f07c97fa700 1 -- 172.21.15.29:0/3468546444 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f07cc07a0a0 con 0x7f07d4158600 2024-01-24T14:11:00.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.217+0000 7f07d92eb700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f07b805f2a0 0x7f07b8061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:00.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.218+0000 7f07d92eb700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f07b805f2a0 0x7f07b8061760 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f07c4003930 tx=0x7f07c4007bd0).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:00.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.221+0000 7f07c97fa700 1 -- 172.21.15.29:0/3468546444 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f07cc04adf0 con 0x7f07d4158600 2024-01-24T14:11:00.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.403+0000 7f07db54f700 1 -- 172.21.15.29:0/3468546444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07d40c6810 con 0x7f07d4158600 2024-01-24T14:11:00.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.404+0000 7f07c97fa700 1 -- 172.21.15.29:0/3468546444 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+18183 (secure 0 0 0) 0x7f07cc04e410 con 0x7f07d4158600 2024-01-24T14:11:00.405 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:11:00.405 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":71,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:10:59.693584+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:10:57.425223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-24T14:10:58.622652+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:11:00.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.406+0000 7f07db54f700 1 -- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f07b805f2a0 msgr2=0x7f07b8061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:00.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.406+0000 7f07db54f700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f07b805f2a0 0x7f07b8061760 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f07c4003930 tx=0x7f07c4007bd0).stop 2024-01-24T14:11:00.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.406+0000 7f07db54f700 1 -- 172.21.15.29:0/3468546444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07d4158600 msgr2=0x7f07d40aaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:00.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.406+0000 7f07db54f700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07d4158600 0x7f07d40aaf90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f07cc001560 tx=0x7f07cc0015b0).stop 2024-01-24T14:11:00.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.406+0000 7f07db54f700 1 -- 172.21.15.29:0/3468546444 shutdown_connections 2024-01-24T14:11:00.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.406+0000 7f07db54f700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f07b805f2a0 0x7f07b8061760 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.406+0000 7f07db54f700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07d4158600 0x7f07d40aaf90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.406+0000 7f07db54f700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07d4062970 0x7f07d40c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.406+0000 7f07db54f700 1 --2- 172.21.15.29:0/3468546444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07d4066b40 0x7f07d40aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.406+0000 7f07db54f700 1 -- 172.21.15.29:0/3468546444 >> 172.21.15.29:0/3468546444 conn(0x7f07d40c8580 msgr2=0x7f07d40c4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:00.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.406+0000 7f07db54f700 1 -- 172.21.15.29:0/3468546444 shutdown_connections 2024-01-24T14:11:00.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.406+0000 7f07db54f700 1 -- 172.21.15.29:0/3468546444 wait complete. 2024-01-24T14:11:00.421 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-24T14:11:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.553+0000 7f4ad46c0700 1 Processor -- start 2024-01-24T14:11:00.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.553+0000 7f4ad46c0700 1 -- start start 2024-01-24T14:11:00.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad46c0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4acc153f40 0x7f4acc0a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:00.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad46c0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4acc0c8290 0x7f4acc0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:00.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad46c0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4acc06a360 con 0x7f4acc154320 2024-01-24T14:11:00.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad46c0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4acc0bcef0 con 0x7f4acc0c8290 2024-01-24T14:11:00.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad46c0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4acc069de0 con 0x7f4acc153f40 2024-01-24T14:11:00.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad1c5b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4acc0c8290 0x7f4acc0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:00.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad1c5b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4acc0c8290 0x7f4acc0627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38062/0 (socket says 172.21.15.29:38062) 2024-01-24T14:11:00.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad1c5b700 1 -- 172.21.15.29:0/3518007033 learned_addr learned my addr 172.21.15.29:0/3518007033 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad245c700 1 --2- 172.21.15.29:0/3518007033 >> v2:172.21.15.163:3300/0 conn(0x7f4acc153f40 0x7f4acc0a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad1c5b700 1 -- 172.21.15.29:0/3518007033 >> v2:172.21.15.163:3300/0 conn(0x7f4acc153f40 msgr2=0x7f4acc0a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad1c5b700 1 --2- 172.21.15.29:0/3518007033 >> v2:172.21.15.163:3300/0 conn(0x7f4acc153f40 0x7f4acc0a6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad1c5b700 1 -- 172.21.15.29:0/3518007033 >> v1:172.21.15.163:6789/0 conn(0x7f4acc154320 legacy=0x7f4acc157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.554+0000 7f4ad1c5b700 1 -- 172.21.15.29:0/3518007033 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc06a080 con 0x7f4acc0c8290 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.555+0000 7f4ad1c5b700 1 --2- 172.21.15.29:0/3518007033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4acc0c8290 0x7f4acc0627b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f4acc0691a0 tx=0x7f4abc0090f0).ready entity=mon.0 client_cookie=54428eb66a6aae server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.555+0000 7f4ad145a700 1 -- 172.21.15.29:0/3518007033 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc00b5a0 con 0x7f4acc0c8290 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.555+0000 7f4ad145a700 1 -- 172.21.15.29:0/3518007033 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc00b700 con 0x7f4acc0c8290 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.555+0000 7f4ad145a700 1 -- 172.21.15.29:0/3518007033 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc00baa0 con 0x7f4acc0c8290 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.555+0000 7f4ad46c0700 1 -- 172.21.15.29:0/3518007033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4acc0c8290 msgr2=0x7f4acc0627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.555+0000 7f4ad46c0700 1 --2- 172.21.15.29:0/3518007033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4acc0c8290 0x7f4acc0627b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f4acc0691a0 tx=0x7f4abc0090f0).stop 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.555+0000 7f4ad46c0700 1 -- 172.21.15.29:0/3518007033 shutdown_connections 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.555+0000 7f4ad46c0700 1 --2- 172.21.15.29:0/3518007033 >> v2:172.21.15.163:3300/0 conn(0x7f4acc153f40 0x7f4acc0a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.555+0000 7f4ad46c0700 1 --2- 172.21.15.29:0/3518007033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4acc0c8290 0x7f4acc0627b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.555+0000 7f4ad46c0700 1 -- 172.21.15.29:0/3518007033 >> 172.21.15.29:0/3518007033 conn(0x7f4acc064390 msgr2=0x7f4acc0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.555+0000 7f4ad46c0700 1 -- 172.21.15.29:0/3518007033 shutdown_connections 2024-01-24T14:11:00.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.555+0000 7f4ad46c0700 1 -- 172.21.15.29:0/3518007033 wait complete. 2024-01-24T14:11:00.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.557+0000 7f4ad46c0700 1 Processor -- start 2024-01-24T14:11:00.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.557+0000 7f4ad46c0700 1 -- start start 2024-01-24T14:11:00.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.557+0000 7f4ad46c0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4acc0c8290 0x7f4acc0a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:00.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.557+0000 7f4ad46c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4acc153f40 0x7f4acc0ac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:00.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.557+0000 7f4ad46c0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4acc154320 0x7f4acc148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:00.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.557+0000 7f4ad46c0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc148f80 con 0x7f4acc0c8290 2024-01-24T14:11:00.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.557+0000 7f4ad46c0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc1497e0 con 0x7f4acc153f40 2024-01-24T14:11:00.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.557+0000 7f4ad46c0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc0ace20 con 0x7f4acc154320 2024-01-24T14:11:00.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad245c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4acc0c8290 0x7f4acc0a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad245c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4acc0c8290 0x7f4acc0a6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38070/0 (socket says 172.21.15.29:38070) 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad245c700 1 -- 172.21.15.29:0/79291738 learned_addr learned my addr 172.21.15.29:0/79291738 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad1c5b700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4acc153f40 0x7f4acc0ac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad2c5d700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4acc154320 0x7f4acc148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad245c700 1 -- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4acc154320 msgr2=0x7f4acc148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad245c700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4acc154320 0x7f4acc148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad245c700 1 -- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4acc153f40 msgr2=0x7f4acc0ac5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad245c700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4acc153f40 0x7f4acc0ac5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad245c700 1 -- 172.21.15.29:0/79291738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc009370 con 0x7f4acc0c8290 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad1c5b700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4acc153f40 0x7f4acc0ac5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad245c700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4acc0c8290 0x7f4acc0a6ce0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f4ac800d890 tx=0x7f4ac8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ac27fc700 1 -- 172.21.15.29:0/79291738 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80125f0 con 0x7f4acc0c8290 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ac27fc700 1 -- 172.21.15.29:0/79291738 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac8012750 con 0x7f4acc0c8290 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ac27fc700 1 -- 172.21.15.29:0/79291738 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8012ad0 con 0x7f4acc0c8290 2024-01-24T14:11:00.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad46c0700 1 -- 172.21.15.29:0/79291738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4acc149190 con 0x7f4acc0c8290 2024-01-24T14:11:00.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.558+0000 7f4ad46c0700 1 -- 172.21.15.29:0/79291738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4acc0a71d0 con 0x7f4acc0c8290 2024-01-24T14:11:00.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.560+0000 7f4ad46c0700 1 -- 172.21.15.29:0/79291738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4acc0475e0 con 0x7f4acc0c8290 2024-01-24T14:11:00.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.560+0000 7f4ac27fc700 1 -- 172.21.15.29:0/79291738 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4ac8013480 con 0x7f4acc0c8290 2024-01-24T14:11:00.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.560+0000 7f4ac27fc700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ab005f240 0x7f4ab0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:00.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.560+0000 7f4ac27fc700 1 -- 172.21.15.29:0/79291738 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f4ac807a5f0 con 0x7f4acc0c8290 2024-01-24T14:11:00.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.561+0000 7f4ad1c5b700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ab005f240 0x7f4ab0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:00.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.561+0000 7f4ad1c5b700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ab005f240 0x7f4ab0061700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f4acc0c86b0 tx=0x7f4abc015000).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:00.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.565+0000 7f4ac27fc700 1 -- 172.21.15.29:0/79291738 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4ac804b2e0 con 0x7f4acc0c8290 2024-01-24T14:11:00.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:00.796+0000 7f4ad46c0700 1 -- 172.21.15.29:0/79291738 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4acc06a540 con 0x7f4acc0c8290 2024-01-24T14:11:01.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.794+0000 7f4ac27fc700 1 -- 172.21.15.29:0/79291738 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4ac804e900 con 0x7f4acc0c8290 2024-01-24T14:11:01.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.796+0000 7f4ad46c0700 1 -- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ab005f240 msgr2=0x7f4ab0061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:01.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.796+0000 7f4ad46c0700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ab005f240 0x7f4ab0061700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f4acc0c86b0 tx=0x7f4abc015000).stop 2024-01-24T14:11:01.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.796+0000 7f4ad46c0700 1 -- 172.21.15.29:0/79291738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4acc0c8290 msgr2=0x7f4acc0a6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:01.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.796+0000 7f4ad46c0700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4acc0c8290 0x7f4acc0a6ce0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f4ac800d890 tx=0x7f4ac8008040).stop 2024-01-24T14:11:01.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.796+0000 7f4ad46c0700 1 -- 172.21.15.29:0/79291738 shutdown_connections 2024-01-24T14:11:01.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.796+0000 7f4ad46c0700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4acc154320 0x7f4acc148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:01.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.796+0000 7f4ad46c0700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ab005f240 0x7f4ab0061700 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:01.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.796+0000 7f4ad46c0700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4acc0c8290 0x7f4acc0a6ce0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:01.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.796+0000 7f4ad46c0700 1 --2- 172.21.15.29:0/79291738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4acc153f40 0x7f4acc0ac5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:01.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.796+0000 7f4ad46c0700 1 -- 172.21.15.29:0/79291738 >> 172.21.15.29:0/79291738 conn(0x7f4acc064390 msgr2=0x7f4acc0c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:01.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.796+0000 7f4ad46c0700 1 -- 172.21.15.29:0/79291738 shutdown_connections 2024-01-24T14:11:01.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.796+0000 7f4ad46c0700 1 -- 172.21.15.29:0/79291738 wait complete. 2024-01-24T14:11:01.812 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-24T14:11:01.812 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-24T14:11:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.948+0000 7f1c898db700 1 Processor -- start 2024-01-24T14:11:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.949+0000 7f1c898db700 1 -- start start 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.949+0000 7f1c898db700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1c840c8490 0x7f1c840a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.949+0000 7f1c898db700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c84152240 con 0x7f1c840bfd40 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.949+0000 7f1c898db700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c841586c0 con 0x7f1c84064760 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.949+0000 7f1c898db700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c840aa670 con 0x7f1c840c8490 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.950+0000 7f1c837fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1c840bfd40 0x7f1c840c01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59672/0 (socket says 172.21.15.29:59672) 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.950+0000 7f1c837fe700 1 -- 172.21.15.29:0/1217794781 learned_addr learned my addr 172.21.15.29:0/1217794781 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.950+0000 7f1c827fc700 1 --2- 172.21.15.29:0/1217794781 >> v2:172.21.15.163:3300/0 conn(0x7f1c840c8490 0x7f1c840a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.950+0000 7f1c81ffb700 1 -- 172.21.15.29:0/1217794781 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751192914 0 0) 0x7f1c84152240 con 0x7f1c840bfd40 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.950+0000 7f1c81ffb700 1 -- 172.21.15.29:0/1217794781 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c700033f0 con 0x7f1c840bfd40 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.950+0000 7f1c81ffb700 1 -- 172.21.15.29:0/1217794781 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585508317 0 0) 0x7f1c841586c0 con 0x7f1c84064760 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.950+0000 7f1c81ffb700 1 -- 172.21.15.29:0/1217794781 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c84152240 con 0x7f1c84064760 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.950+0000 7f1c827fc700 1 -- 172.21.15.29:0/1217794781 >> v1:172.21.15.163:6789/0 conn(0x7f1c84064760 legacy=0x7f1c84064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:01.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.950+0000 7f1c81ffb700 1 -- 172.21.15.29:0/1217794781 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1833665166 0 0) 0x7f1c700033f0 con 0x7f1c840bfd40 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.950+0000 7f1c827fc700 1 -- 172.21.15.29:0/1217794781 >> v1:172.21.15.29:6789/0 conn(0x7f1c840bfd40 legacy=0x7f1c840c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.950+0000 7f1c827fc700 1 -- 172.21.15.29:0/1217794781 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c84157d90 con 0x7f1c840c8490 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.950+0000 7f1c827fc700 1 --2- 172.21.15.29:0/1217794781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c840c8490 0x7f1c840a9d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f1c6c009760 tx=0x7f1c6c009060).ready entity=mon.1 client_cookie=a36f1b55191d1a67 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.951+0000 7f1c81ffb700 1 -- 172.21.15.29:0/1217794781 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0017f0 con 0x7f1c840c8490 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.951+0000 7f1c81ffb700 1 -- 172.21.15.29:0/1217794781 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c011400 con 0x7f1c840c8490 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.951+0000 7f1c81ffb700 1 -- 172.21.15.29:0/1217794781 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c011780 con 0x7f1c840c8490 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.951+0000 7f1c898db700 1 -- 172.21.15.29:0/1217794781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c840c8490 msgr2=0x7f1c840a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.951+0000 7f1c898db700 1 --2- 172.21.15.29:0/1217794781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c840c8490 0x7f1c840a9d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f1c6c009760 tx=0x7f1c6c009060).stop 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.951+0000 7f1c898db700 1 -- 172.21.15.29:0/1217794781 shutdown_connections 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.951+0000 7f1c898db700 1 --2- 172.21.15.29:0/1217794781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c840c8490 0x7f1c840a9d90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.951+0000 7f1c898db700 1 -- 172.21.15.29:0/1217794781 >> 172.21.15.29:0/1217794781 conn(0x7f1c840cb8b0 msgr2=0x7f1c840c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.951+0000 7f1c898db700 1 -- 172.21.15.29:0/1217794781 shutdown_connections 2024-01-24T14:11:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.951+0000 7f1c898db700 1 -- 172.21.15.29:0/1217794781 wait complete. 2024-01-24T14:11:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.953+0000 7f1c898db700 1 Processor -- start 2024-01-24T14:11:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.953+0000 7f1c898db700 1 -- start start 2024-01-24T14:11:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.953+0000 7f1c898db700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c84064760 0x7f1c840b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c898db700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c840bfd40 0x7f1c84153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c898db700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c840c8490 0x7f1c840c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c898db700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c840af9b0 con 0x7f1c840bfd40 2024-01-24T14:11:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c898db700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c840afc90 con 0x7f1c840c8490 2024-01-24T14:11:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c898db700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c840aff70 con 0x7f1c84064760 2024-01-24T14:11:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c827fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c840bfd40 0x7f1c84153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c827fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c840bfd40 0x7f1c84153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38072/0 (socket says 172.21.15.29:38072) 2024-01-24T14:11:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c827fc700 1 -- 172.21.15.29:0/1188152002 learned_addr learned my addr 172.21.15.29:0/1188152002 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c837fe700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c840c8490 0x7f1c840c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c82ffd700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c84064760 0x7f1c840b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c827fc700 1 -- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c84064760 msgr2=0x7f1c840b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c827fc700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c84064760 0x7f1c840b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c827fc700 1 -- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c840c8490 msgr2=0x7f1c840c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c827fc700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c840c8490 0x7f1c840c2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c827fc700 1 -- 172.21.15.29:0/1188152002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c0092a0 con 0x7f1c840bfd40 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c837fe700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c840c8490 0x7f1c840c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c827fc700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c840bfd40 0x7f1c84153b80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f1c6c009d50 tx=0x7f1c6c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c7affd700 1 -- 172.21.15.29:0/1188152002 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c010950 con 0x7f1c840bfd40 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c7affd700 1 -- 172.21.15.29:0/1188152002 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c010ab0 con 0x7f1c840bfd40 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c898db700 1 -- 172.21.15.29:0/1188152002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c840b0250 con 0x7f1c840bfd40 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c7affd700 1 -- 172.21.15.29:0/1188152002 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c010e30 con 0x7f1c840bfd40 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.954+0000 7f1c898db700 1 -- 172.21.15.29:0/1188152002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c840af660 con 0x7f1c840bfd40 2024-01-24T14:11:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.955+0000 7f1c898db700 1 -- 172.21.15.29:0/1188152002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c840475e0 con 0x7f1c840bfd40 2024-01-24T14:11:01.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.956+0000 7f1c7affd700 1 -- 172.21.15.29:0/1188152002 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1c6c01d640 con 0x7f1c840bfd40 2024-01-24T14:11:01.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.956+0000 7f1c7affd700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c6405f270 0x7f1c64061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:01.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.956+0000 7f1c7affd700 1 -- 172.21.15.29:0/1188152002 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f1c6c07be60 con 0x7f1c840bfd40 2024-01-24T14:11:01.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.956+0000 7f1c82ffd700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c6405f270 0x7f1c64061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:01.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.957+0000 7f1c82ffd700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c6405f270 0x7f1c64061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1c74003700 tx=0x7f1c74009040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:01.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:01.961+0000 7f1c7affd700 1 -- 172.21.15.29:0/1188152002 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1c6c04cb50 con 0x7f1c840bfd40 2024-01-24T14:11:02.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.188+0000 7f1c898db700 1 -- 172.21.15.29:0/1188152002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1c84158070 con 0x7f1c840bfd40 2024-01-24T14:11:02.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.799+0000 7f1c7affd700 1 -- 172.21.15.29:0/1188152002 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1c6c050170 con 0x7f1c840bfd40 2024-01-24T14:11:02.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.801+0000 7f1c898db700 1 -- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c6405f270 msgr2=0x7f1c64061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:02.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.801+0000 7f1c898db700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c6405f270 0x7f1c64061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1c74003700 tx=0x7f1c74009040).stop 2024-01-24T14:11:02.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.801+0000 7f1c898db700 1 -- 172.21.15.29:0/1188152002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c840bfd40 msgr2=0x7f1c84153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:02.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.801+0000 7f1c898db700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c840bfd40 0x7f1c84153b80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f1c6c009d50 tx=0x7f1c6c016000).stop 2024-01-24T14:11:02.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.801+0000 7f1c898db700 1 -- 172.21.15.29:0/1188152002 shutdown_connections 2024-01-24T14:11:02.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.801+0000 7f1c898db700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c840c8490 0x7f1c840c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:02.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.801+0000 7f1c898db700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c84064760 0x7f1c840b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:02.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.801+0000 7f1c898db700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c6405f270 0x7f1c64061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:02.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.801+0000 7f1c898db700 1 --2- 172.21.15.29:0/1188152002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c840bfd40 0x7f1c84153b80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:02.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.801+0000 7f1c898db700 1 -- 172.21.15.29:0/1188152002 >> 172.21.15.29:0/1188152002 conn(0x7f1c840cb8b0 msgr2=0x7f1c840c1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:02.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.802+0000 7f1c898db700 1 -- 172.21.15.29:0/1188152002 shutdown_connections 2024-01-24T14:11:02.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.802+0000 7f1c898db700 1 -- 172.21.15.29:0/1188152002 wait complete. 2024-01-24T14:11:02.817 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-24T14:11:02.818 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:11:02.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.951+0000 7f909a4f3700 1 Processor -- start 2024-01-24T14:11:02.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.952+0000 7f909a4f3700 1 -- start start 2024-01-24T14:11:02.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.952+0000 7f909a4f3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9094062970 0x7f9094062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:02.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.952+0000 7f909a4f3700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9094154190 0x7f90940a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:02.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.952+0000 7f909a4f3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90940bd190 con 0x7f9094069dd0 2024-01-24T14:11:02.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.952+0000 7f909a4f3700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9094154610 con 0x7f9094154190 2024-01-24T14:11:02.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.952+0000 7f909a4f3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90940c4e60 con 0x7f9094062970 2024-01-24T14:11:02.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.953+0000 7f9098a90700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9094154190 0x7f90940a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:02.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.953+0000 7f9098a90700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9094154190 0x7f90940a6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38082/0 (socket says 172.21.15.29:38082) 2024-01-24T14:11:02.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.953+0000 7f9098a90700 1 -- 172.21.15.29:0/2066064208 learned_addr learned my addr 172.21.15.29:0/2066064208 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:02.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.953+0000 7f9093fff700 1 --2- 172.21.15.29:0/2066064208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9094062970 0x7f9094062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:02.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.953+0000 7f9098a90700 1 -- 172.21.15.29:0/2066064208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9094062970 msgr2=0x7f9094062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.953+0000 7f9098a90700 1 --2- 172.21.15.29:0/2066064208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9094062970 0x7f9094062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.953+0000 7f9098a90700 1 -- 172.21.15.29:0/2066064208 >> v1:172.21.15.163:6789/0 conn(0x7f9094069dd0 legacy=0x7f9094153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.953+0000 7f9098a90700 1 -- 172.21.15.29:0/2066064208 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094066370 con 0x7f9094154190 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.953+0000 7f9098a90700 1 --2- 172.21.15.29:0/2066064208 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094154190 0x7f90940a6db0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f908c009f20 tx=0x7f908c009840).ready entity=mon.0 client_cookie=dd703d3d71cb9fe8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.953+0000 7f9092ffd700 1 -- 172.21.15.29:0/2066064208 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c00e070 con 0x7f9094154190 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.953+0000 7f9092ffd700 1 -- 172.21.15.29:0/2066064208 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c00ba40 con 0x7f9094154190 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.953+0000 7f9092ffd700 1 -- 172.21.15.29:0/2066064208 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c00bdc0 con 0x7f9094154190 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.954+0000 7f909a4f3700 1 -- 172.21.15.29:0/2066064208 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094154190 msgr2=0x7f90940a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.954+0000 7f909a4f3700 1 --2- 172.21.15.29:0/2066064208 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094154190 0x7f90940a6db0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f908c009f20 tx=0x7f908c009840).stop 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.954+0000 7f909a4f3700 1 -- 172.21.15.29:0/2066064208 shutdown_connections 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.954+0000 7f909a4f3700 1 --2- 172.21.15.29:0/2066064208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9094062970 0x7f9094062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.954+0000 7f909a4f3700 1 --2- 172.21.15.29:0/2066064208 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094154190 0x7f90940a6db0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:02.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.954+0000 7f909a4f3700 1 -- 172.21.15.29:0/2066064208 >> 172.21.15.29:0/2066064208 conn(0x7f90940c85c0 msgr2=0x7f90940c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:02.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.954+0000 7f909a4f3700 1 -- 172.21.15.29:0/2066064208 shutdown_connections 2024-01-24T14:11:02.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.954+0000 7f909a4f3700 1 -- 172.21.15.29:0/2066064208 wait complete. 2024-01-24T14:11:02.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.955+0000 7f909a4f3700 1 Processor -- start 2024-01-24T14:11:02.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.955+0000 7f909a4f3700 1 -- start start 2024-01-24T14:11:02.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.956+0000 7f909a4f3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9094062970 0x7f9094150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.956+0000 7f909a4f3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9094069dd0 0x7f909414f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.956+0000 7f909a4f3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094154190 0x7f90940c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.956+0000 7f909a4f3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90940a8370 con 0x7f9094154190 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.956+0000 7f909a4f3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9094150980 con 0x7f9094062970 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.956+0000 7f909a4f3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9094150c60 con 0x7f9094069dd0 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.956+0000 7f9098a90700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094154190 0x7f90940c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.956+0000 7f9098a90700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094154190 0x7f90940c0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38098/0 (socket says 172.21.15.29:38098) 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.956+0000 7f9098a90700 1 -- 172.21.15.29:0/2329265989 learned_addr learned my addr 172.21.15.29:0/2329265989 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.956+0000 7f90937fe700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9094069dd0 0x7f909414f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.956+0000 7f9093fff700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9094062970 0x7f9094150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.957+0000 7f9098a90700 1 -- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9094069dd0 msgr2=0x7f909414f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.957+0000 7f9098a90700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9094069dd0 0x7f909414f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:02.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.957+0000 7f9098a90700 1 -- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9094062970 msgr2=0x7f9094150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:02.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.957+0000 7f9098a90700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9094062970 0x7f9094150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:02.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.957+0000 7f9098a90700 1 -- 172.21.15.29:0/2329265989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c009ae0 con 0x7f9094154190 2024-01-24T14:11:02.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.957+0000 7f9098a90700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094154190 0x7f90940c0e00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f908c00b6c0 tx=0x7f90940be230).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:02.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.957+0000 7f907bfff700 1 -- 172.21.15.29:0/2329265989 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c007080 con 0x7f9094154190 2024-01-24T14:11:02.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.957+0000 7f907bfff700 1 -- 172.21.15.29:0/2329265989 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c0092e0 con 0x7f9094154190 2024-01-24T14:11:02.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.957+0000 7f907bfff700 1 -- 172.21.15.29:0/2329265989 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c01e6a0 con 0x7f9094154190 2024-01-24T14:11:02.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.957+0000 7f909a4f3700 1 -- 172.21.15.29:0/2329265989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9094150f40 con 0x7f9094154190 2024-01-24T14:11:02.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.957+0000 7f909a4f3700 1 -- 172.21.15.29:0/2329265989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9094150640 con 0x7f9094154190 2024-01-24T14:11:02.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.958+0000 7f909a4f3700 1 -- 172.21.15.29:0/2329265989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90940a8140 con 0x7f9094154190 2024-01-24T14:11:02.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.959+0000 7f907bfff700 1 -- 172.21.15.29:0/2329265989 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f908c003040 con 0x7f9094154190 2024-01-24T14:11:02.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.959+0000 7f907bfff700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f908005f2a0 0x7f9080061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:02.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.959+0000 7f907bfff700 1 -- 172.21.15.29:0/2329265989 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f908c07bb90 con 0x7f9094154190 2024-01-24T14:11:02.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.959+0000 7f9093fff700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f908005f2a0 0x7f9080061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:02.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.960+0000 7f9093fff700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f908005f2a0 0x7f9080061760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f908400bbc0 tx=0x7f9084008040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:02.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:02.964+0000 7f907bfff700 1 -- 172.21.15.29:0/2329265989 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f90940a8140 con 0x7f9094154190 2024-01-24T14:11:03.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.190+0000 7f909a4f3700 1 -- 172.21.15.29:0/2329265989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90940a8140 con 0x7f9094154190 2024-01-24T14:11:03.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.191+0000 7f907bfff700 1 -- 172.21.15.29:0/2329265989 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f90940a8140 con 0x7f9094154190 2024-01-24T14:11:03.191 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:11:03.191 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2239835986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2239835986},{"type":"v1","addr":"172.21.15.163:6837","nonce":2239835986}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2208246875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2208246875},{"type":"v1","addr":"172.21.15.29:6835","nonce":2208246875}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/547152226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":547152226},{"type":"v1","addr":"172.21.15.163:6836","nonce":547152226}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:10:59.693964+0000","modified":"2024-01-24T14:11:02.793474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2804469957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2804469957},{"type":"v1","addr":"172.21.15.29:6834","nonce":2804469957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-24T14:11:03.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.193+0000 7f909a4f3700 1 -- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f908005f2a0 msgr2=0x7f9080061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.193+0000 7f909a4f3700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f908005f2a0 0x7f9080061760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f908400bbc0 tx=0x7f9084008040).stop 2024-01-24T14:11:03.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.193+0000 7f909a4f3700 1 -- 172.21.15.29:0/2329265989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094154190 msgr2=0x7f90940c0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.193+0000 7f909a4f3700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094154190 0x7f90940c0e00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f908c00b6c0 tx=0x7f90940be230).stop 2024-01-24T14:11:03.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.193+0000 7f909a4f3700 1 -- 172.21.15.29:0/2329265989 shutdown_connections 2024-01-24T14:11:03.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.193+0000 7f909a4f3700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f908005f2a0 0x7f9080061760 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.193+0000 7f909a4f3700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9094154190 0x7f90940c0e00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.193+0000 7f909a4f3700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9094062970 0x7f9094150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.193+0000 7f909a4f3700 1 --2- 172.21.15.29:0/2329265989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9094069dd0 0x7f909414f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.193+0000 7f909a4f3700 1 -- 172.21.15.29:0/2329265989 >> 172.21.15.29:0/2329265989 conn(0x7f90940c85c0 msgr2=0x7f90940c3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:03.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.193+0000 7f909a4f3700 1 -- 172.21.15.29:0/2329265989 shutdown_connections 2024-01-24T14:11:03.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.193+0000 7f909a4f3700 1 -- 172.21.15.29:0/2329265989 wait complete. 2024-01-24T14:11:03.194 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 26 2024-01-24T14:11:03.207 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:11:03.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.336+0000 7f1199793700 1 Processor -- start 2024-01-24T14:11:03.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.336+0000 7f1199793700 1 -- start start 2024-01-24T14:11:03.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.337+0000 7f1199793700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1194134cc0 0x7f11940c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:03.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.337+0000 7f1199793700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1194066c00 0x7f1194152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:03.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.337+0000 7f1199793700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1194068360 con 0x7f11941350a0 2024-01-24T14:11:03.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.337+0000 7f1199793700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1194153360 con 0x7f1194134cc0 2024-01-24T14:11:03.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.337+0000 7f1199793700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11940c3820 con 0x7f1194066c00 2024-01-24T14:11:03.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f1192ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1194134cc0 0x7f11940c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f1192ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1194134cc0 0x7f11940c31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38106/0 (socket says 172.21.15.29:38106) 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f1192ffd700 1 -- 172.21.15.29:0/1280853855 learned_addr learned my addr 172.21.15.29:0/1280853855 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f11937fe700 1 --2- 172.21.15.29:0/1280853855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1194066c00 0x7f1194152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f1192ffd700 1 -- 172.21.15.29:0/1280853855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1194066c00 msgr2=0x7f1194152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f1191ffb700 1 -- 172.21.15.29:0/1280853855 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631856988 0 0) 0x7f1194068360 con 0x7f11941350a0 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f1192ffd700 1 --2- 172.21.15.29:0/1280853855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1194066c00 0x7f1194152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f1192ffd700 1 -- 172.21.15.29:0/1280853855 >> v1:172.21.15.163:6789/0 conn(0x7f11941350a0 legacy=0x7f11940664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f1192ffd700 1 -- 172.21.15.29:0/1280853855 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1194067a30 con 0x7f1194134cc0 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f11937fe700 1 --2- 172.21.15.29:0/1280853855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1194066c00 0x7f1194152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f1192ffd700 1 --2- 172.21.15.29:0/1280853855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1194134cc0 0x7f11940c31b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f117c006fd0 tx=0x7f117c0090f0).ready entity=mon.0 client_cookie=a432d183f7e9d1c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f1191ffb700 1 -- 172.21.15.29:0/1280853855 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00b580 con 0x7f1194134cc0 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f1191ffb700 1 -- 172.21.15.29:0/1280853855 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00b6e0 con 0x7f1194134cc0 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.338+0000 7f1191ffb700 1 -- 172.21.15.29:0/1280853855 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00ba60 con 0x7f1194134cc0 2024-01-24T14:11:03.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.339+0000 7f1199793700 1 -- 172.21.15.29:0/1280853855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1194134cc0 msgr2=0x7f11940c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.339+0000 7f1199793700 1 --2- 172.21.15.29:0/1280853855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1194134cc0 0x7f11940c31b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f117c006fd0 tx=0x7f117c0090f0).stop 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.339+0000 7f1199793700 1 -- 172.21.15.29:0/1280853855 shutdown_connections 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.339+0000 7f1199793700 1 --2- 172.21.15.29:0/1280853855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1194066c00 0x7f1194152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.339+0000 7f1199793700 1 --2- 172.21.15.29:0/1280853855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1194134cc0 0x7f11940c31b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.339+0000 7f1199793700 1 -- 172.21.15.29:0/1280853855 >> 172.21.15.29:0/1280853855 conn(0x7f11940c8bb0 msgr2=0x7f11940c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.339+0000 7f1199793700 1 -- 172.21.15.29:0/1280853855 shutdown_connections 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.339+0000 7f1199793700 1 -- 172.21.15.29:0/1280853855 wait complete. 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.341+0000 7f1199793700 1 Processor -- start 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.341+0000 7f1199793700 1 -- start start 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f1199793700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1194066c00 0x7f11941345f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f1199793700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1194134cc0 0x7f11940abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f1199793700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11941350a0 0x7f1194136ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f1199793700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11940a9990 con 0x7f1194134cc0 2024-01-24T14:11:03.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f1199793700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1194134ae0 con 0x7f1194066c00 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f1199793700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11940ac7a0 con 0x7f11941350a0 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f11927fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1194134cc0 0x7f11940abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f11927fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1194134cc0 0x7f11940abd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38116/0 (socket says 172.21.15.29:38116) 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f11927fc700 1 -- 172.21.15.29:0/4228730703 learned_addr learned my addr 172.21.15.29:0/4228730703 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f1192ffd700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1194066c00 0x7f11941345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f11937fe700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11941350a0 0x7f1194136ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f11927fc700 1 -- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11941350a0 msgr2=0x7f1194136ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f11927fc700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11941350a0 0x7f1194136ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f11927fc700 1 -- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1194066c00 msgr2=0x7f11941345f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f11927fc700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1194066c00 0x7f11941345f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.342+0000 7f11927fc700 1 -- 172.21.15.29:0/4228730703 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c009a20 con 0x7f1194134cc0 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.343+0000 7f11927fc700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1194134cc0 0x7f11940abd70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f118400c440 tx=0x7f1184007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.343+0000 7f118affd700 1 -- 172.21.15.29:0/4228730703 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184007020 con 0x7f1194134cc0 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.343+0000 7f118affd700 1 -- 172.21.15.29:0/4228730703 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11840071a0 con 0x7f1194134cc0 2024-01-24T14:11:03.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.343+0000 7f1199793700 1 -- 172.21.15.29:0/4228730703 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c009370 con 0x7f1194134cc0 2024-01-24T14:11:03.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.343+0000 7f118affd700 1 -- 172.21.15.29:0/4228730703 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184007560 con 0x7f1194134cc0 2024-01-24T14:11:03.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.343+0000 7f1199793700 1 -- 172.21.15.29:0/4228730703 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11940acd60 con 0x7f1194134cc0 2024-01-24T14:11:03.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.344+0000 7f1199793700 1 -- 172.21.15.29:0/4228730703 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11940475e0 con 0x7f1194134cc0 2024-01-24T14:11:03.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.344+0000 7f118affd700 1 -- 172.21.15.29:0/4228730703 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f11840068b0 con 0x7f1194134cc0 2024-01-24T14:11:03.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.345+0000 7f118affd700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f117405f270 0x7f1174061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:03.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.345+0000 7f118affd700 1 -- 172.21.15.29:0/4228730703 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f1184079c80 con 0x7f1194134cc0 2024-01-24T14:11:03.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.345+0000 7f1192ffd700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f117405f270 0x7f1174061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:03.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.346+0000 7f1192ffd700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f117405f270 0x7f1174061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f117c009130 tx=0x7f117c017000).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:03.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.349+0000 7f118affd700 1 -- 172.21.15.29:0/4228730703 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1184047b00 con 0x7f1194134cc0 2024-01-24T14:11:03.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.532+0000 7f1199793700 1 -- 172.21.15.29:0/4228730703 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f119406bca0 con 0x7f1194134cc0 2024-01-24T14:11:03.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.533+0000 7f118affd700 1 -- 172.21.15.29:0/4228730703 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+18170 (secure 0 0 0) 0x7f118404a9d0 con 0x7f1194134cc0 2024-01-24T14:11:03.534 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:11:03.534 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":72,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:11:00.695786+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:10:57.425223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-24T14:10:58.622652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:11:03.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.535+0000 7f1199793700 1 -- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f117405f270 msgr2=0x7f1174061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.535+0000 7f1199793700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f117405f270 0x7f1174061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f117c009130 tx=0x7f117c017000).stop 2024-01-24T14:11:03.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.535+0000 7f1199793700 1 -- 172.21.15.29:0/4228730703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1194134cc0 msgr2=0x7f11940abd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.535+0000 7f1199793700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1194134cc0 0x7f11940abd70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f118400c440 tx=0x7f1184007e00).stop 2024-01-24T14:11:03.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.535+0000 7f1199793700 1 -- 172.21.15.29:0/4228730703 shutdown_connections 2024-01-24T14:11:03.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.535+0000 7f1199793700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11941350a0 0x7f1194136ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.535+0000 7f1199793700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1194066c00 0x7f11941345f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.535+0000 7f1199793700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f117405f270 0x7f1174061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.535+0000 7f1199793700 1 --2- 172.21.15.29:0/4228730703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1194134cc0 0x7f11940abd70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.535+0000 7f1199793700 1 -- 172.21.15.29:0/4228730703 >> 172.21.15.29:0/4228730703 conn(0x7f11940c8bb0 msgr2=0x7f1194152230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:03.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.536+0000 7f1199793700 1 -- 172.21.15.29:0/4228730703 shutdown_connections 2024-01-24T14:11:03.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.536+0000 7f1199793700 1 -- 172.21.15.29:0/4228730703 wait complete. 2024-01-24T14:11:03.550 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-24T14:11:03.550 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:11:03.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.679+0000 7fae28618700 1 Processor -- start 2024-01-24T14:11:03.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.680+0000 7fae28618700 1 -- start start 2024-01-24T14:11:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae28618700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae2013a5c0 0x7fae20063a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae28618700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fae200be520 0x7fae200664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae28618700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae20066e90 con 0x7fae200669e0 2024-01-24T14:11:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae28618700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae200bf6a0 con 0x7fae200be520 2024-01-24T14:11:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae28618700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae200bea90 con 0x7fae2013a5c0 2024-01-24T14:11:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae25bb3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fae200be520 0x7fae200664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae263b4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae2013a5c0 0x7fae20063a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae25bb3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fae200be520 0x7fae200664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:43114/0 (socket says 172.21.15.29:43114) 2024-01-24T14:11:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae25bb3700 1 -- 172.21.15.29:0/198413029 learned_addr learned my addr 172.21.15.29:0/198413029 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae26bb5700 1 --1- 172.21.15.29:0/198413029 >> v1:172.21.15.163:6789/0 conn(0x7fae200669e0 0x7fae20067a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:55782/0 (socket says 172.21.15.29:55782) 2024-01-24T14:11:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae25bb3700 1 -- 172.21.15.29:0/198413029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae2013a5c0 msgr2=0x7fae20063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae25bb3700 1 --2- 172.21.15.29:0/198413029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae2013a5c0 0x7fae20063a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae25bb3700 1 -- 172.21.15.29:0/198413029 >> v1:172.21.15.163:6789/0 conn(0x7fae200669e0 legacy=0x7fae20067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae25bb3700 1 -- 172.21.15.29:0/198413029 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae200bed70 con 0x7fae200be520 2024-01-24T14:11:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae253b2700 1 -- 172.21.15.29:0/198413029 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491948833 0 0) 0x7fae18001490 con 0x7fae200669e0 2024-01-24T14:11:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.681+0000 7fae25bb3700 1 --2- 172.21.15.29:0/198413029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200be520 0x7fae200664f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fae10006fd0 tx=0x7fae100090f0).ready entity=mon.1 client_cookie=6e1f8c4379e6cf37 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.682+0000 7fae253b2700 1 -- 172.21.15.29:0/198413029 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1000b580 con 0x7fae200be520 2024-01-24T14:11:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.682+0000 7fae253b2700 1 -- 172.21.15.29:0/198413029 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1000b6e0 con 0x7fae200be520 2024-01-24T14:11:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.682+0000 7fae253b2700 1 -- 172.21.15.29:0/198413029 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1000ba60 con 0x7fae200be520 2024-01-24T14:11:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.682+0000 7fae28618700 1 -- 172.21.15.29:0/198413029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200be520 msgr2=0x7fae200664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.682+0000 7fae28618700 1 --2- 172.21.15.29:0/198413029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200be520 0x7fae200664f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fae10006fd0 tx=0x7fae100090f0).stop 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.682+0000 7fae28618700 1 -- 172.21.15.29:0/198413029 shutdown_connections 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.682+0000 7fae28618700 1 --2- 172.21.15.29:0/198413029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae2013a5c0 0x7fae20063a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.682+0000 7fae28618700 1 --2- 172.21.15.29:0/198413029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200be520 0x7fae200664f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.682+0000 7fae28618700 1 -- 172.21.15.29:0/198413029 >> 172.21.15.29:0/198413029 conn(0x7fae200c8bb0 msgr2=0x7fae200c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.682+0000 7fae28618700 1 -- 172.21.15.29:0/198413029 shutdown_connections 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.682+0000 7fae28618700 1 -- 172.21.15.29:0/198413029 wait complete. 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.684+0000 7fae28618700 1 Processor -- start 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.684+0000 7fae28618700 1 -- start start 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.684+0000 7fae28618700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200669e0 0x7fae200aab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae28618700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae200be520 0x7fae2013d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae28618700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae2013a5c0 0x7fae20155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae28618700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae20157120 con 0x7fae200be520 2024-01-24T14:11:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae28618700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae20157400 con 0x7fae200669e0 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae28618700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2013ccf0 con 0x7fae2013a5c0 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae25bb3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae200be520 0x7fae2013d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae25bb3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae200be520 0x7fae2013d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42672/0 (socket says 172.21.15.29:42672) 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae25bb3700 1 -- 172.21.15.29:0/2516003741 learned_addr learned my addr 172.21.15.29:0/2516003741 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae263b4700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200669e0 0x7fae200aab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae26bb5700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae2013a5c0 0x7fae20155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae25bb3700 1 -- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae2013a5c0 msgr2=0x7fae20155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae25bb3700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae2013a5c0 0x7fae20155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae25bb3700 1 -- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200669e0 msgr2=0x7fae200aab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae25bb3700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200669e0 0x7fae200aab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae25bb3700 1 -- 172.21.15.29:0/2516003741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae10009370 con 0x7fae200be520 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae25bb3700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae200be520 0x7fae2013d7d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fae10009130 tx=0x7fae10017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae167fc700 1 -- 172.21.15.29:0/2516003741 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10000bc0 con 0x7fae200be520 2024-01-24T14:11:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae167fc700 1 -- 172.21.15.29:0/2516003741 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1000b850 con 0x7fae200be520 2024-01-24T14:11:03.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae28618700 1 -- 172.21.15.29:0/2516003741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2013cfd0 con 0x7fae200be520 2024-01-24T14:11:03.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.685+0000 7fae167fc700 1 -- 172.21.15.29:0/2516003741 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10000d20 con 0x7fae200be520 2024-01-24T14:11:03.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.686+0000 7fae28618700 1 -- 172.21.15.29:0/2516003741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2013dcc0 con 0x7fae200be520 2024-01-24T14:11:03.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.687+0000 7fae28618700 1 -- 172.21.15.29:0/2516003741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2013cb50 con 0x7fae200be520 2024-01-24T14:11:03.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.687+0000 7fae167fc700 1 -- 172.21.15.29:0/2516003741 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fae1001c900 con 0x7fae200be520 2024-01-24T14:11:03.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.688+0000 7fae167fc700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fae0405f240 0x7fae04061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:03.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.688+0000 7fae167fc700 1 -- 172.21.15.29:0/2516003741 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fae1007c4c0 con 0x7fae200be520 2024-01-24T14:11:03.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.688+0000 7fae263b4700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fae0405f240 0x7fae04061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:03.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.688+0000 7fae263b4700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fae0405f240 0x7fae04061700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fae1c009c30 tx=0x7fae1c009060).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:03.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.692+0000 7fae167fc700 1 -- 172.21.15.29:0/2516003741 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fae2013cb50 con 0x7fae200be520 2024-01-24T14:11:03.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.868+0000 7fae28618700 1 -- 172.21.15.29:0/2516003741 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae2013cb50 con 0x7fae0405f240 2024-01-24T14:11:03.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.879+0000 7fae167fc700 1 -- 172.21.15.29:0/2516003741 <== mgr.5524 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361249 (secure 0 0 0) 0x7fae2013cb50 con 0x7fae0405f240 2024-01-24T14:11:03.885 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:11:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.883+0000 7fae28618700 1 -- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fae0405f240 msgr2=0x7fae04061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.883+0000 7fae28618700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fae0405f240 0x7fae04061700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fae1c009c30 tx=0x7fae1c009060).stop 2024-01-24T14:11:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.883+0000 7fae28618700 1 -- 172.21.15.29:0/2516003741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae200be520 msgr2=0x7fae2013d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.883+0000 7fae28618700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae200be520 0x7fae2013d7d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fae10009130 tx=0x7fae10017000).stop 2024-01-24T14:11:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.883+0000 7fae28618700 1 -- 172.21.15.29:0/2516003741 shutdown_connections 2024-01-24T14:11:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.883+0000 7fae28618700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae2013a5c0 0x7fae20155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.883+0000 7fae28618700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200669e0 0x7fae200aab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.883+0000 7fae28618700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fae0405f240 0x7fae04061700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.883+0000 7fae28618700 1 --2- 172.21.15.29:0/2516003741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae200be520 0x7fae2013d7d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.883+0000 7fae28618700 1 -- 172.21.15.29:0/2516003741 >> 172.21.15.29:0/2516003741 conn(0x7fae200c8bb0 msgr2=0x7fae200abb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.883+0000 7fae28618700 1 -- 172.21.15.29:0/2516003741 shutdown_connections 2024-01-24T14:11:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:03.883+0000 7fae28618700 1 -- 172.21.15.29:0/2516003741 wait complete. 2024-01-24T14:11:03.889 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:11:03.897 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T14:11:02.184153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":240,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2416348,"kb_used_data":6812,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752558372,"statfs":{"total":773094113280,"available":770619772928,"internally_reserved":0,"allocated":6975488,"data_stored":3676363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15000139,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-719,"num_write_kb":-15551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.013232"},"pg_stats":[{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.729964+0000","last_change":"2024-01-24T14:10:59.715939+0000","last_active":"2024-01-24T14:10:59.729964+0000","last_peered":"2024-01-24T14:10:59.729964+0000","last_clean":"2024-01-24T14:10:59.729964+0000","last_became_active":"2024-01-24T14:10:59.715657+0000","last_became_peered":"2024-01-24T14:10:59.715657+0000","last_unstale":"2024-01-24T14:10:59.729964+0000","last_undegraded":"2024-01-24T14:10:59.729964+0000","last_fullsized":"2024-01-24T14:10:59.729964+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3f","version":"71'3","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.733438+0000","last_change":"2024-01-24T14:10:59.247665+0000","last_active":"2024-01-24T14:10:59.733438+0000","last_peered":"2024-01-24T14:10:59.733438+0000","last_clean":"2024-01-24T14:10:59.733438+0000","last_became_active":"2024-01-24T14:10:59.246919+0000","last_became_peered":"2024-01-24T14:10:59.246919+0000","last_unstale":"2024-01-24T14:10:59.733438+0000","last_undegraded":"2024-01-24T14:10:59.733438+0000","last_fullsized":"2024-01-24T14:10:59.733438+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730005+0000","last_change":"2024-01-24T14:10:59.712157+0000","last_active":"2024-01-24T14:10:59.730005+0000","last_peered":"2024-01-24T14:10:59.730005+0000","last_clean":"2024-01-24T14:10:59.730005+0000","last_became_active":"2024-01-24T14:10:59.711793+0000","last_became_peered":"2024-01-24T14:10:59.711793+0000","last_unstale":"2024-01-24T14:10:59.730005+0000","last_undegraded":"2024-01-24T14:10:59.730005+0000","last_fullsized":"2024-01-24T14:10:59.730005+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.708141+0000","last_change":"2024-01-24T14:10:59.248350+0000","last_active":"2024-01-24T14:10:59.708141+0000","last_peered":"2024-01-24T14:10:59.708141+0000","last_clean":"2024-01-24T14:10:59.708141+0000","last_became_active":"2024-01-24T14:10:59.247993+0000","last_became_peered":"2024-01-24T14:10:59.247993+0000","last_unstale":"2024-01-24T14:10:59.708141+0000","last_undegraded":"2024-01-24T14:10:59.708141+0000","last_fullsized":"2024-01-24T14:10:59.708141+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.723143+0000","last_change":"2024-01-24T14:10:59.719672+0000","last_active":"2024-01-24T14:10:59.723143+0000","last_peered":"2024-01-24T14:10:59.723143+0000","last_clean":"2024-01-24T14:10:59.723143+0000","last_became_active":"2024-01-24T14:10:59.719350+0000","last_became_peered":"2024-01-24T14:10:59.719350+0000","last_unstale":"2024-01-24T14:10:59.723143+0000","last_undegraded":"2024-01-24T14:10:59.723143+0000","last_fullsized":"2024-01-24T14:10:59.723143+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.708311+0000","last_change":"2024-01-24T14:10:59.254110+0000","last_active":"2024-01-24T14:10:59.708311+0000","last_peered":"2024-01-24T14:10:59.708311+0000","last_clean":"2024-01-24T14:10:59.708311+0000","last_became_active":"2024-01-24T14:10:59.253768+0000","last_became_peered":"2024-01-24T14:10:59.253768+0000","last_unstale":"2024-01-24T14:10:59.708311+0000","last_undegraded":"2024-01-24T14:10:59.708311+0000","last_fullsized":"2024-01-24T14:10:59.708311+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724000+0000","last_change":"2024-01-24T14:10:59.718793+0000","last_active":"2024-01-24T14:10:59.724000+0000","last_peered":"2024-01-24T14:10:59.724000+0000","last_clean":"2024-01-24T14:10:59.724000+0000","last_became_active":"2024-01-24T14:10:59.718510+0000","last_became_peered":"2024-01-24T14:10:59.718510+0000","last_unstale":"2024-01-24T14:10:59.724000+0000","last_undegraded":"2024-01-24T14:10:59.724000+0000","last_fullsized":"2024-01-24T14:10:59.724000+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3c","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731575+0000","last_change":"2024-01-24T14:10:59.253305+0000","last_active":"2024-01-24T14:10:59.731575+0000","last_peered":"2024-01-24T14:10:59.731575+0000","last_clean":"2024-01-24T14:10:59.731575+0000","last_became_active":"2024-01-24T14:10:59.253015+0000","last_became_peered":"2024-01-24T14:10:59.253015+0000","last_unstale":"2024-01-24T14:10:59.731575+0000","last_undegraded":"2024-01-24T14:10:59.731575+0000","last_fullsized":"2024-01-24T14:10:59.731575+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725365+0000","last_change":"2024-01-24T14:10:59.718476+0000","last_active":"2024-01-24T14:10:59.725365+0000","last_peered":"2024-01-24T14:10:59.725365+0000","last_clean":"2024-01-24T14:10:59.725365+0000","last_became_active":"2024-01-24T14:10:59.718185+0000","last_became_peered":"2024-01-24T14:10:59.718185+0000","last_unstale":"2024-01-24T14:10:59.725365+0000","last_undegraded":"2024-01-24T14:10:59.725365+0000","last_fullsized":"2024-01-24T14:10:59.725365+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.718463+0000","last_change":"2024-01-24T14:10:59.250903+0000","last_active":"2024-01-24T14:10:59.718463+0000","last_peered":"2024-01-24T14:10:59.718463+0000","last_clean":"2024-01-24T14:10:59.718463+0000","last_became_active":"2024-01-24T14:10:59.250583+0000","last_became_peered":"2024-01-24T14:10:59.250583+0000","last_unstale":"2024-01-24T14:10:59.718463+0000","last_undegraded":"2024-01-24T14:10:59.718463+0000","last_fullsized":"2024-01-24T14:10:59.718463+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.729816+0000","last_change":"2024-01-24T14:10:59.712738+0000","last_active":"2024-01-24T14:10:59.729816+0000","last_peered":"2024-01-24T14:10:59.729816+0000","last_clean":"2024-01-24T14:10:59.729816+0000","last_became_active":"2024-01-24T14:10:59.712443+0000","last_became_peered":"2024-01-24T14:10:59.712443+0000","last_unstale":"2024-01-24T14:10:59.729816+0000","last_undegraded":"2024-01-24T14:10:59.729816+0000","last_fullsized":"2024-01-24T14:10:59.729816+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732686+0000","last_change":"2024-01-24T14:10:59.249275+0000","last_active":"2024-01-24T14:10:59.732686+0000","last_peered":"2024-01-24T14:10:59.732686+0000","last_clean":"2024-01-24T14:10:59.732686+0000","last_became_active":"2024-01-24T14:10:59.248982+0000","last_became_peered":"2024-01-24T14:10:59.248982+0000","last_unstale":"2024-01-24T14:10:59.732686+0000","last_undegraded":"2024-01-24T14:10:59.732686+0000","last_fullsized":"2024-01-24T14:10:59.732686+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731298+0000","last_change":"2024-01-24T14:10:59.717196+0000","last_active":"2024-01-24T14:10:59.731298+0000","last_peered":"2024-01-24T14:10:59.731298+0000","last_clean":"2024-01-24T14:10:59.731298+0000","last_became_active":"2024-01-24T14:10:59.716919+0000","last_became_peered":"2024-01-24T14:10:59.716919+0000","last_unstale":"2024-01-24T14:10:59.731298+0000","last_undegraded":"2024-01-24T14:10:59.731298+0000","last_fullsized":"2024-01-24T14:10:59.731298+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.707722+0000","last_change":"2024-01-24T14:10:59.254462+0000","last_active":"2024-01-24T14:10:59.707722+0000","last_peered":"2024-01-24T14:10:59.707722+0000","last_clean":"2024-01-24T14:10:59.707722+0000","last_became_active":"2024-01-24T14:10:59.253839+0000","last_became_peered":"2024-01-24T14:10:59.253839+0000","last_unstale":"2024-01-24T14:10:59.707722+0000","last_undegraded":"2024-01-24T14:10:59.707722+0000","last_fullsized":"2024-01-24T14:10:59.707722+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.720589+0000","last_change":"2024-01-24T14:10:59.715610+0000","last_active":"2024-01-24T14:10:59.720589+0000","last_peered":"2024-01-24T14:10:59.720589+0000","last_clean":"2024-01-24T14:10:59.720589+0000","last_became_active":"2024-01-24T14:10:59.715227+0000","last_became_peered":"2024-01-24T14:10:59.715227+0000","last_unstale":"2024-01-24T14:10:59.720589+0000","last_undegraded":"2024-01-24T14:10:59.720589+0000","last_fullsized":"2024-01-24T14:10:59.720589+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.706874+0000","last_change":"2024-01-24T14:10:59.252933+0000","last_active":"2024-01-24T14:10:59.706874+0000","last_peered":"2024-01-24T14:10:59.706874+0000","last_clean":"2024-01-24T14:10:59.706874+0000","last_became_active":"2024-01-24T14:10:59.252461+0000","last_became_peered":"2024-01-24T14:10:59.252461+0000","last_unstale":"2024-01-24T14:10:59.706874+0000","last_undegraded":"2024-01-24T14:10:59.706874+0000","last_fullsized":"2024-01-24T14:10:59.706874+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.733154+0000","last_change":"2024-01-24T14:10:59.727871+0000","last_active":"2024-01-24T14:10:59.733154+0000","last_peered":"2024-01-24T14:10:59.733154+0000","last_clean":"2024-01-24T14:10:59.733154+0000","last_became_active":"2024-01-24T14:10:59.727602+0000","last_became_peered":"2024-01-24T14:10:59.727602+0000","last_unstale":"2024-01-24T14:10:59.733154+0000","last_undegraded":"2024-01-24T14:10:59.733154+0000","last_fullsized":"2024-01-24T14:10:59.733154+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711282+0000","last_change":"2024-01-24T14:10:59.253063+0000","last_active":"2024-01-24T14:10:59.711282+0000","last_peered":"2024-01-24T14:10:59.711282+0000","last_clean":"2024-01-24T14:10:59.711282+0000","last_became_active":"2024-01-24T14:10:59.252748+0000","last_became_peered":"2024-01-24T14:10:59.252748+0000","last_unstale":"2024-01-24T14:10:59.711282+0000","last_undegraded":"2024-01-24T14:10:59.711282+0000","last_fullsized":"2024-01-24T14:10:59.711282+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.726716+0000","last_change":"2024-01-24T14:10:59.723498+0000","last_active":"2024-01-24T14:10:59.726716+0000","last_peered":"2024-01-24T14:10:59.726716+0000","last_clean":"2024-01-24T14:10:59.726716+0000","last_became_active":"2024-01-24T14:10:59.723252+0000","last_became_peered":"2024-01-24T14:10:59.723252+0000","last_unstale":"2024-01-24T14:10:59.726716+0000","last_undegraded":"2024-01-24T14:10:59.726716+0000","last_fullsized":"2024-01-24T14:10:59.726716+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.704981+0000","last_change":"2024-01-24T14:10:59.255826+0000","last_active":"2024-01-24T14:10:59.704981+0000","last_peered":"2024-01-24T14:10:59.704981+0000","last_clean":"2024-01-24T14:10:59.704981+0000","last_became_active":"2024-01-24T14:10:59.255489+0000","last_became_peered":"2024-01-24T14:10:59.255489+0000","last_unstale":"2024-01-24T14:10:59.704981+0000","last_undegraded":"2024-01-24T14:10:59.704981+0000","last_fullsized":"2024-01-24T14:10:59.704981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.722974+0000","last_change":"2024-01-24T14:10:59.718057+0000","last_active":"2024-01-24T14:10:59.722974+0000","last_peered":"2024-01-24T14:10:59.722974+0000","last_clean":"2024-01-24T14:10:59.722974+0000","last_became_active":"2024-01-24T14:10:59.717755+0000","last_became_peered":"2024-01-24T14:10:59.717755+0000","last_unstale":"2024-01-24T14:10:59.722974+0000","last_undegraded":"2024-01-24T14:10:59.722974+0000","last_fullsized":"2024-01-24T14:10:59.722974+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.698049+0000","last_change":"2024-01-24T14:10:59.245756+0000","last_active":"2024-01-24T14:10:59.698049+0000","last_peered":"2024-01-24T14:10:59.698049+0000","last_clean":"2024-01-24T14:10:59.698049+0000","last_became_active":"2024-01-24T14:10:59.245448+0000","last_became_peered":"2024-01-24T14:10:59.245448+0000","last_unstale":"2024-01-24T14:10:59.698049+0000","last_undegraded":"2024-01-24T14:10:59.698049+0000","last_fullsized":"2024-01-24T14:10:59.698049+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732752+0000","last_change":"2024-01-24T14:10:59.716176+0000","last_active":"2024-01-24T14:10:59.732752+0000","last_peered":"2024-01-24T14:10:59.732752+0000","last_clean":"2024-01-24T14:10:59.732752+0000","last_became_active":"2024-01-24T14:10:59.715901+0000","last_became_peered":"2024-01-24T14:10:59.715901+0000","last_unstale":"2024-01-24T14:10:59.732752+0000","last_undegraded":"2024-01-24T14:10:59.732752+0000","last_fullsized":"2024-01-24T14:10:59.732752+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.34","version":"71'4","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.735298+0000","last_change":"2024-01-24T14:10:59.245051+0000","last_active":"2024-01-24T14:10:59.735298+0000","last_peered":"2024-01-24T14:10:59.735298+0000","last_clean":"2024-01-24T14:10:59.735298+0000","last_became_active":"2024-01-24T14:10:59.244749+0000","last_became_peered":"2024-01-24T14:10:59.244749+0000","last_unstale":"2024-01-24T14:10:59.735298+0000","last_undegraded":"2024-01-24T14:10:59.735298+0000","last_fullsized":"2024-01-24T14:10:59.735298+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.717746+0000","last_change":"2024-01-24T14:10:59.713714+0000","last_active":"2024-01-24T14:10:59.717746+0000","last_peered":"2024-01-24T14:10:59.717746+0000","last_clean":"2024-01-24T14:10:59.717746+0000","last_became_active":"2024-01-24T14:10:59.713418+0000","last_became_peered":"2024-01-24T14:10:59.713418+0000","last_unstale":"2024-01-24T14:10:59.717746+0000","last_undegraded":"2024-01-24T14:10:59.717746+0000","last_fullsized":"2024-01-24T14:10:59.717746+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.33","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732550+0000","last_change":"2024-01-24T14:10:59.255680+0000","last_active":"2024-01-24T14:10:59.732550+0000","last_peered":"2024-01-24T14:10:59.732550+0000","last_clean":"2024-01-24T14:10:59.732550+0000","last_became_active":"2024-01-24T14:10:59.255269+0000","last_became_peered":"2024-01-24T14:10:59.255269+0000","last_unstale":"2024-01-24T14:10:59.732550+0000","last_undegraded":"2024-01-24T14:10:59.732550+0000","last_fullsized":"2024-01-24T14:10:59.732550+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.723840+0000","last_change":"2024-01-24T14:10:59.718254+0000","last_active":"2024-01-24T14:10:59.723840+0000","last_peered":"2024-01-24T14:10:59.723840+0000","last_clean":"2024-01-24T14:10:59.723840+0000","last_became_active":"2024-01-24T14:10:59.717982+0000","last_became_peered":"2024-01-24T14:10:59.717982+0000","last_unstale":"2024-01-24T14:10:59.723840+0000","last_undegraded":"2024-01-24T14:10:59.723840+0000","last_fullsized":"2024-01-24T14:10:59.723840+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705919+0000","last_change":"2024-01-24T14:10:59.249757+0000","last_active":"2024-01-24T14:10:59.705919+0000","last_peered":"2024-01-24T14:10:59.705919+0000","last_clean":"2024-01-24T14:10:59.705919+0000","last_became_active":"2024-01-24T14:10:59.249347+0000","last_became_peered":"2024-01-24T14:10:59.249347+0000","last_unstale":"2024-01-24T14:10:59.705919+0000","last_undegraded":"2024-01-24T14:10:59.705919+0000","last_fullsized":"2024-01-24T14:10:59.705919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725160+0000","last_change":"2024-01-24T14:10:59.718124+0000","last_active":"2024-01-24T14:10:59.725160+0000","last_peered":"2024-01-24T14:10:59.725160+0000","last_clean":"2024-01-24T14:10:59.725160+0000","last_became_active":"2024-01-24T14:10:59.717797+0000","last_became_peered":"2024-01-24T14:10:59.717797+0000","last_unstale":"2024-01-24T14:10:59.725160+0000","last_undegraded":"2024-01-24T14:10:59.725160+0000","last_fullsized":"2024-01-24T14:10:59.725160+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705961+0000","last_change":"2024-01-24T14:10:59.249006+0000","last_active":"2024-01-24T14:10:59.705961+0000","last_peered":"2024-01-24T14:10:59.705961+0000","last_clean":"2024-01-24T14:10:59.705961+0000","last_became_active":"2024-01-24T14:10:59.248706+0000","last_became_peered":"2024-01-24T14:10:59.248706+0000","last_unstale":"2024-01-24T14:10:59.705961+0000","last_undegraded":"2024-01-24T14:10:59.705961+0000","last_fullsized":"2024-01-24T14:10:59.705961+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730075+0000","last_change":"2024-01-24T14:10:59.729586+0000","last_active":"2024-01-24T14:10:59.730075+0000","last_peered":"2024-01-24T14:10:59.730075+0000","last_clean":"2024-01-24T14:10:59.730075+0000","last_became_active":"2024-01-24T14:10:59.729257+0000","last_became_peered":"2024-01-24T14:10:59.729257+0000","last_unstale":"2024-01-24T14:10:59.730075+0000","last_undegraded":"2024-01-24T14:10:59.730075+0000","last_fullsized":"2024-01-24T14:10:59.730075+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.30","version":"71'4","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732867+0000","last_change":"2024-01-24T14:10:59.247241+0000","last_active":"2024-01-24T14:10:59.732867+0000","last_peered":"2024-01-24T14:10:59.732867+0000","last_clean":"2024-01-24T14:10:59.732867+0000","last_became_active":"2024-01-24T14:10:59.246844+0000","last_became_peered":"2024-01-24T14:10:59.246844+0000","last_unstale":"2024-01-24T14:10:59.732867+0000","last_undegraded":"2024-01-24T14:10:59.732867+0000","last_fullsized":"2024-01-24T14:10:59.732867+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724531+0000","last_change":"2024-01-24T14:10:59.721721+0000","last_active":"2024-01-24T14:10:59.724531+0000","last_peered":"2024-01-24T14:10:59.724531+0000","last_clean":"2024-01-24T14:10:59.724531+0000","last_became_active":"2024-01-24T14:10:59.721345+0000","last_became_peered":"2024-01-24T14:10:59.721345+0000","last_unstale":"2024-01-24T14:10:59.724531+0000","last_undegraded":"2024-01-24T14:10:59.724531+0000","last_fullsized":"2024-01-24T14:10:59.724531+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.709686+0000","last_change":"2024-01-24T14:10:59.250419+0000","last_active":"2024-01-24T14:10:59.709686+0000","last_peered":"2024-01-24T14:10:59.709686+0000","last_clean":"2024-01-24T14:10:59.709686+0000","last_became_active":"2024-01-24T14:10:59.250069+0000","last_became_peered":"2024-01-24T14:10:59.250069+0000","last_unstale":"2024-01-24T14:10:59.709686+0000","last_undegraded":"2024-01-24T14:10:59.709686+0000","last_fullsized":"2024-01-24T14:10:59.709686+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731091+0000","last_change":"2024-01-24T14:10:59.722739+0000","last_active":"2024-01-24T14:10:59.731091+0000","last_peered":"2024-01-24T14:10:59.731091+0000","last_clean":"2024-01-24T14:10:59.731091+0000","last_became_active":"2024-01-24T14:10:59.722517+0000","last_became_peered":"2024-01-24T14:10:59.722517+0000","last_unstale":"2024-01-24T14:10:59.731091+0000","last_undegraded":"2024-01-24T14:10:59.731091+0000","last_fullsized":"2024-01-24T14:10:59.731091+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.708251+0000","last_change":"2024-01-24T14:10:59.248434+0000","last_active":"2024-01-24T14:10:59.708251+0000","last_peered":"2024-01-24T14:10:59.708251+0000","last_clean":"2024-01-24T14:10:59.708251+0000","last_became_active":"2024-01-24T14:10:59.248143+0000","last_became_peered":"2024-01-24T14:10:59.248143+0000","last_unstale":"2024-01-24T14:10:59.708251+0000","last_undegraded":"2024-01-24T14:10:59.708251+0000","last_fullsized":"2024-01-24T14:10:59.708251+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731043+0000","last_change":"2024-01-24T14:10:59.719177+0000","last_active":"2024-01-24T14:10:59.731043+0000","last_peered":"2024-01-24T14:10:59.731043+0000","last_clean":"2024-01-24T14:10:59.731043+0000","last_became_active":"2024-01-24T14:10:59.718882+0000","last_became_peered":"2024-01-24T14:10:59.718882+0000","last_unstale":"2024-01-24T14:10:59.731043+0000","last_undegraded":"2024-01-24T14:10:59.731043+0000","last_fullsized":"2024-01-24T14:10:59.731043+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2d","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.722197+0000","last_change":"2024-01-24T14:10:59.257334+0000","last_active":"2024-01-24T14:10:59.722197+0000","last_peered":"2024-01-24T14:10:59.722197+0000","last_clean":"2024-01-24T14:10:59.722197+0000","last_became_active":"2024-01-24T14:10:59.257013+0000","last_became_peered":"2024-01-24T14:10:59.257013+0000","last_unstale":"2024-01-24T14:10:59.722197+0000","last_undegraded":"2024-01-24T14:10:59.722197+0000","last_fullsized":"2024-01-24T14:10:59.722197+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732325+0000","last_change":"2024-01-24T14:10:59.718817+0000","last_active":"2024-01-24T14:10:59.732325+0000","last_peered":"2024-01-24T14:10:59.732325+0000","last_clean":"2024-01-24T14:10:59.732325+0000","last_became_active":"2024-01-24T14:10:59.713272+0000","last_became_peered":"2024-01-24T14:10:59.713272+0000","last_unstale":"2024-01-24T14:10:59.732325+0000","last_undegraded":"2024-01-24T14:10:59.732325+0000","last_fullsized":"2024-01-24T14:10:59.732325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.701011+0000","last_change":"2024-01-24T14:10:59.253760+0000","last_active":"2024-01-24T14:10:59.701011+0000","last_peered":"2024-01-24T14:10:59.701011+0000","last_clean":"2024-01-24T14:10:59.701011+0000","last_became_active":"2024-01-24T14:10:59.253458+0000","last_became_peered":"2024-01-24T14:10:59.253458+0000","last_unstale":"2024-01-24T14:10:59.701011+0000","last_undegraded":"2024-01-24T14:10:59.701011+0000","last_fullsized":"2024-01-24T14:10:59.701011+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.729967+0000","last_change":"2024-01-24T14:10:59.715051+0000","last_active":"2024-01-24T14:10:59.729967+0000","last_peered":"2024-01-24T14:10:59.729967+0000","last_clean":"2024-01-24T14:10:59.729967+0000","last_became_active":"2024-01-24T14:10:59.714515+0000","last_became_peered":"2024-01-24T14:10:59.714515+0000","last_unstale":"2024-01-24T14:10:59.729967+0000","last_undegraded":"2024-01-24T14:10:59.729967+0000","last_fullsized":"2024-01-24T14:10:59.729967+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724324+0000","last_change":"2024-01-24T14:10:59.248692+0000","last_active":"2024-01-24T14:10:59.724324+0000","last_peered":"2024-01-24T14:10:59.724324+0000","last_clean":"2024-01-24T14:10:59.724324+0000","last_became_active":"2024-01-24T14:10:59.248421+0000","last_became_peered":"2024-01-24T14:10:59.248421+0000","last_unstale":"2024-01-24T14:10:59.724324+0000","last_undegraded":"2024-01-24T14:10:59.724324+0000","last_fullsized":"2024-01-24T14:10:59.724324+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.717701+0000","last_change":"2024-01-24T14:10:59.714524+0000","last_active":"2024-01-24T14:10:59.717701+0000","last_peered":"2024-01-24T14:10:59.717701+0000","last_clean":"2024-01-24T14:10:59.717701+0000","last_became_active":"2024-01-24T14:10:59.714248+0000","last_became_peered":"2024-01-24T14:10:59.714248+0000","last_unstale":"2024-01-24T14:10:59.717701+0000","last_undegraded":"2024-01-24T14:10:59.717701+0000","last_fullsized":"2024-01-24T14:10:59.717701+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.707964+0000","last_change":"2024-01-24T14:10:59.260935+0000","last_active":"2024-01-24T14:10:59.707964+0000","last_peered":"2024-01-24T14:10:59.707964+0000","last_clean":"2024-01-24T14:10:59.707964+0000","last_became_active":"2024-01-24T14:10:59.260632+0000","last_became_peered":"2024-01-24T14:10:59.260632+0000","last_unstale":"2024-01-24T14:10:59.707964+0000","last_undegraded":"2024-01-24T14:10:59.707964+0000","last_fullsized":"2024-01-24T14:10:59.707964+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.723919+0000","last_change":"2024-01-24T14:10:59.719221+0000","last_active":"2024-01-24T14:10:59.723919+0000","last_peered":"2024-01-24T14:10:59.723919+0000","last_clean":"2024-01-24T14:10:59.723919+0000","last_became_active":"2024-01-24T14:10:59.718831+0000","last_became_peered":"2024-01-24T14:10:59.718831+0000","last_unstale":"2024-01-24T14:10:59.723919+0000","last_undegraded":"2024-01-24T14:10:59.723919+0000","last_fullsized":"2024-01-24T14:10:59.723919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.698243+0000","last_change":"2024-01-24T14:10:59.247208+0000","last_active":"2024-01-24T14:10:59.698243+0000","last_peered":"2024-01-24T14:10:59.698243+0000","last_clean":"2024-01-24T14:10:59.698243+0000","last_became_active":"2024-01-24T14:10:59.245845+0000","last_became_peered":"2024-01-24T14:10:59.245845+0000","last_unstale":"2024-01-24T14:10:59.698243+0000","last_undegraded":"2024-01-24T14:10:59.698243+0000","last_fullsized":"2024-01-24T14:10:59.698243+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.719732+0000","last_change":"2024-01-24T14:10:59.711935+0000","last_active":"2024-01-24T14:10:59.719732+0000","last_peered":"2024-01-24T14:10:59.719732+0000","last_clean":"2024-01-24T14:10:59.719732+0000","last_became_active":"2024-01-24T14:10:59.711637+0000","last_became_peered":"2024-01-24T14:10:59.711637+0000","last_unstale":"2024-01-24T14:10:59.719732+0000","last_undegraded":"2024-01-24T14:10:59.719732+0000","last_fullsized":"2024-01-24T14:10:59.719732+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.707081+0000","last_change":"2024-01-24T14:10:59.254043+0000","last_active":"2024-01-24T14:10:59.707081+0000","last_peered":"2024-01-24T14:10:59.707081+0000","last_clean":"2024-01-24T14:10:59.707081+0000","last_became_active":"2024-01-24T14:10:59.253710+0000","last_became_peered":"2024-01-24T14:10:59.253710+0000","last_unstale":"2024-01-24T14:10:59.707081+0000","last_undegraded":"2024-01-24T14:10:59.707081+0000","last_fullsized":"2024-01-24T14:10:59.707081+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.723172+0000","last_change":"2024-01-24T14:10:59.720897+0000","last_active":"2024-01-24T14:10:59.723172+0000","last_peered":"2024-01-24T14:10:59.723172+0000","last_clean":"2024-01-24T14:10:59.723172+0000","last_became_active":"2024-01-24T14:10:59.720599+0000","last_became_peered":"2024-01-24T14:10:59.720599+0000","last_unstale":"2024-01-24T14:10:59.723172+0000","last_undegraded":"2024-01-24T14:10:59.723172+0000","last_fullsized":"2024-01-24T14:10:59.723172+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.709784+0000","last_change":"2024-01-24T14:10:59.262835+0000","last_active":"2024-01-24T14:10:59.709784+0000","last_peered":"2024-01-24T14:10:59.709784+0000","last_clean":"2024-01-24T14:10:59.709784+0000","last_became_active":"2024-01-24T14:10:59.262585+0000","last_became_peered":"2024-01-24T14:10:59.262585+0000","last_unstale":"2024-01-24T14:10:59.709784+0000","last_undegraded":"2024-01-24T14:10:59.709784+0000","last_fullsized":"2024-01-24T14:10:59.709784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730027+0000","last_change":"2024-01-24T14:10:59.712491+0000","last_active":"2024-01-24T14:10:59.730027+0000","last_peered":"2024-01-24T14:10:59.730027+0000","last_clean":"2024-01-24T14:10:59.730027+0000","last_became_active":"2024-01-24T14:10:59.712206+0000","last_became_peered":"2024-01-24T14:10:59.712206+0000","last_unstale":"2024-01-24T14:10:59.730027+0000","last_undegraded":"2024-01-24T14:10:59.730027+0000","last_fullsized":"2024-01-24T14:10:59.730027+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711149+0000","last_change":"2024-01-24T14:10:59.246549+0000","last_active":"2024-01-24T14:10:59.711149+0000","last_peered":"2024-01-24T14:10:59.711149+0000","last_clean":"2024-01-24T14:10:59.711149+0000","last_became_active":"2024-01-24T14:10:59.246268+0000","last_became_peered":"2024-01-24T14:10:59.246268+0000","last_unstale":"2024-01-24T14:10:59.711149+0000","last_undegraded":"2024-01-24T14:10:59.711149+0000","last_fullsized":"2024-01-24T14:10:59.711149+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732158+0000","last_change":"2024-01-24T14:10:59.718560+0000","last_active":"2024-01-24T14:10:59.732158+0000","last_peered":"2024-01-24T14:10:59.732158+0000","last_clean":"2024-01-24T14:10:59.732158+0000","last_became_active":"2024-01-24T14:10:59.718277+0000","last_became_peered":"2024-01-24T14:10:59.718277+0000","last_unstale":"2024-01-24T14:10:59.732158+0000","last_undegraded":"2024-01-24T14:10:59.732158+0000","last_fullsized":"2024-01-24T14:10:59.732158+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705155+0000","last_change":"2024-01-24T14:10:59.255711+0000","last_active":"2024-01-24T14:10:59.705155+0000","last_peered":"2024-01-24T14:10:59.705155+0000","last_clean":"2024-01-24T14:10:59.705155+0000","last_became_active":"2024-01-24T14:10:59.255294+0000","last_became_peered":"2024-01-24T14:10:59.255294+0000","last_unstale":"2024-01-24T14:10:59.705155+0000","last_undegraded":"2024-01-24T14:10:59.705155+0000","last_fullsized":"2024-01-24T14:10:59.705155+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725444+0000","last_change":"2024-01-24T14:10:59.719130+0000","last_active":"2024-01-24T14:10:59.725444+0000","last_peered":"2024-01-24T14:10:59.725444+0000","last_clean":"2024-01-24T14:10:59.725444+0000","last_became_active":"2024-01-24T14:10:59.718656+0000","last_became_peered":"2024-01-24T14:10:59.718656+0000","last_unstale":"2024-01-24T14:10:59.725444+0000","last_undegraded":"2024-01-24T14:10:59.725444+0000","last_fullsized":"2024-01-24T14:10:59.725444+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.701102+0000","last_change":"2024-01-24T14:10:59.243916+0000","last_active":"2024-01-24T14:10:59.701102+0000","last_peered":"2024-01-24T14:10:59.701102+0000","last_clean":"2024-01-24T14:10:59.701102+0000","last_became_active":"2024-01-24T14:10:59.243607+0000","last_became_peered":"2024-01-24T14:10:59.243607+0000","last_unstale":"2024-01-24T14:10:59.701102+0000","last_undegraded":"2024-01-24T14:10:59.701102+0000","last_fullsized":"2024-01-24T14:10:59.701102+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724039+0000","last_change":"2024-01-24T14:10:59.719200+0000","last_active":"2024-01-24T14:10:59.724039+0000","last_peered":"2024-01-24T14:10:59.724039+0000","last_clean":"2024-01-24T14:10:59.724039+0000","last_became_active":"2024-01-24T14:10:59.718895+0000","last_became_peered":"2024-01-24T14:10:59.718895+0000","last_unstale":"2024-01-24T14:10:59.724039+0000","last_undegraded":"2024-01-24T14:10:59.724039+0000","last_fullsized":"2024-01-24T14:10:59.724039+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.706758+0000","last_change":"2024-01-24T14:10:59.259168+0000","last_active":"2024-01-24T14:10:59.706758+0000","last_peered":"2024-01-24T14:10:59.706758+0000","last_clean":"2024-01-24T14:10:59.706758+0000","last_became_active":"2024-01-24T14:10:59.258902+0000","last_became_peered":"2024-01-24T14:10:59.258902+0000","last_unstale":"2024-01-24T14:10:59.706758+0000","last_undegraded":"2024-01-24T14:10:59.706758+0000","last_fullsized":"2024-01-24T14:10:59.706758+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730814+0000","last_change":"2024-01-24T14:10:59.714078+0000","last_active":"2024-01-24T14:10:59.730814+0000","last_peered":"2024-01-24T14:10:59.730814+0000","last_clean":"2024-01-24T14:10:59.730814+0000","last_became_active":"2024-01-24T14:10:59.712655+0000","last_became_peered":"2024-01-24T14:10:59.712655+0000","last_unstale":"2024-01-24T14:10:59.730814+0000","last_undegraded":"2024-01-24T14:10:59.730814+0000","last_fullsized":"2024-01-24T14:10:59.730814+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.706003+0000","last_change":"2024-01-24T14:10:59.248954+0000","last_active":"2024-01-24T14:10:59.706003+0000","last_peered":"2024-01-24T14:10:59.706003+0000","last_clean":"2024-01-24T14:10:59.706003+0000","last_became_active":"2024-01-24T14:10:59.248653+0000","last_became_peered":"2024-01-24T14:10:59.248653+0000","last_unstale":"2024-01-24T14:10:59.706003+0000","last_undegraded":"2024-01-24T14:10:59.706003+0000","last_fullsized":"2024-01-24T14:10:59.706003+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724988+0000","last_change":"2024-01-24T14:10:59.717867+0000","last_active":"2024-01-24T14:10:59.724988+0000","last_peered":"2024-01-24T14:10:59.724988+0000","last_clean":"2024-01-24T14:10:59.724988+0000","last_became_active":"2024-01-24T14:10:59.717564+0000","last_became_peered":"2024-01-24T14:10:59.717564+0000","last_unstale":"2024-01-24T14:10:59.724988+0000","last_undegraded":"2024-01-24T14:10:59.724988+0000","last_fullsized":"2024-01-24T14:10:59.724988+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.707825+0000","last_change":"2024-01-24T14:10:59.254286+0000","last_active":"2024-01-24T14:10:59.707825+0000","last_peered":"2024-01-24T14:10:59.707825+0000","last_clean":"2024-01-24T14:10:59.707825+0000","last_became_active":"2024-01-24T14:10:59.253619+0000","last_became_peered":"2024-01-24T14:10:59.253619+0000","last_unstale":"2024-01-24T14:10:59.707825+0000","last_undegraded":"2024-01-24T14:10:59.707825+0000","last_fullsized":"2024-01-24T14:10:59.707825+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732399+0000","last_change":"2024-01-24T14:10:59.729798+0000","last_active":"2024-01-24T14:10:59.732399+0000","last_peered":"2024-01-24T14:10:59.732399+0000","last_clean":"2024-01-24T14:10:59.732399+0000","last_became_active":"2024-01-24T14:10:59.729492+0000","last_became_peered":"2024-01-24T14:10:59.729492+0000","last_unstale":"2024-01-24T14:10:59.732399+0000","last_undegraded":"2024-01-24T14:10:59.732399+0000","last_fullsized":"2024-01-24T14:10:59.732399+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.710093+0000","last_change":"2024-01-24T14:10:59.251175+0000","last_active":"2024-01-24T14:10:59.710093+0000","last_peered":"2024-01-24T14:10:59.710093+0000","last_clean":"2024-01-24T14:10:59.710093+0000","last_became_active":"2024-01-24T14:10:59.250923+0000","last_became_peered":"2024-01-24T14:10:59.250923+0000","last_unstale":"2024-01-24T14:10:59.710093+0000","last_undegraded":"2024-01-24T14:10:59.710093+0000","last_fullsized":"2024-01-24T14:10:59.710093+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725933+0000","last_change":"2024-01-24T14:10:59.719887+0000","last_active":"2024-01-24T14:10:59.725933+0000","last_peered":"2024-01-24T14:10:59.725933+0000","last_clean":"2024-01-24T14:10:59.725933+0000","last_became_active":"2024-01-24T14:10:59.719522+0000","last_became_peered":"2024-01-24T14:10:59.719522+0000","last_unstale":"2024-01-24T14:10:59.725933+0000","last_undegraded":"2024-01-24T14:10:59.725933+0000","last_fullsized":"2024-01-24T14:10:59.725933+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.721066+0000","last_change":"2024-01-24T14:10:59.246566+0000","last_active":"2024-01-24T14:10:59.721066+0000","last_peered":"2024-01-24T14:10:59.721066+0000","last_clean":"2024-01-24T14:10:59.721066+0000","last_became_active":"2024-01-24T14:10:59.246263+0000","last_became_peered":"2024-01-24T14:10:59.246263+0000","last_unstale":"2024-01-24T14:10:59.721066+0000","last_undegraded":"2024-01-24T14:10:59.721066+0000","last_fullsized":"2024-01-24T14:10:59.721066+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732439+0000","last_change":"2024-01-24T14:10:59.715649+0000","last_active":"2024-01-24T14:10:59.732439+0000","last_peered":"2024-01-24T14:10:59.732439+0000","last_clean":"2024-01-24T14:10:59.732439+0000","last_became_active":"2024-01-24T14:10:59.715376+0000","last_became_peered":"2024-01-24T14:10:59.715376+0000","last_unstale":"2024-01-24T14:10:59.732439+0000","last_undegraded":"2024-01-24T14:10:59.732439+0000","last_fullsized":"2024-01-24T14:10:59.732439+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.721107+0000","last_change":"2024-01-24T14:10:59.248963+0000","last_active":"2024-01-24T14:10:59.721107+0000","last_peered":"2024-01-24T14:10:59.721107+0000","last_clean":"2024-01-24T14:10:59.721107+0000","last_became_active":"2024-01-24T14:10:59.248500+0000","last_became_peered":"2024-01-24T14:10:59.248500+0000","last_unstale":"2024-01-24T14:10:59.721107+0000","last_undegraded":"2024-01-24T14:10:59.721107+0000","last_fullsized":"2024-01-24T14:10:59.721107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.735377+0000","last_change":"2024-01-24T14:10:59.734815+0000","last_active":"2024-01-24T14:10:59.735377+0000","last_peered":"2024-01-24T14:10:59.735377+0000","last_clean":"2024-01-24T14:10:59.735377+0000","last_became_active":"2024-01-24T14:10:59.734541+0000","last_became_peered":"2024-01-24T14:10:59.734541+0000","last_unstale":"2024-01-24T14:10:59.735377+0000","last_undegraded":"2024-01-24T14:10:59.735377+0000","last_fullsized":"2024-01-24T14:10:59.735377+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.733025+0000","last_change":"2024-01-24T14:10:59.251973+0000","last_active":"2024-01-24T14:10:59.733025+0000","last_peered":"2024-01-24T14:10:59.733025+0000","last_clean":"2024-01-24T14:10:59.733025+0000","last_became_active":"2024-01-24T14:10:59.251655+0000","last_became_peered":"2024-01-24T14:10:59.251655+0000","last_unstale":"2024-01-24T14:10:59.733025+0000","last_undegraded":"2024-01-24T14:10:59.733025+0000","last_fullsized":"2024-01-24T14:10:59.733025+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725455+0000","last_change":"2024-01-24T14:10:59.719693+0000","last_active":"2024-01-24T14:10:59.725455+0000","last_peered":"2024-01-24T14:10:59.725455+0000","last_clean":"2024-01-24T14:10:59.725455+0000","last_became_active":"2024-01-24T14:10:59.719401+0000","last_became_peered":"2024-01-24T14:10:59.719401+0000","last_unstale":"2024-01-24T14:10:59.725455+0000","last_undegraded":"2024-01-24T14:10:59.725455+0000","last_fullsized":"2024-01-24T14:10:59.725455+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.710964+0000","last_change":"2024-01-24T14:10:59.248372+0000","last_active":"2024-01-24T14:10:59.710964+0000","last_peered":"2024-01-24T14:10:59.710964+0000","last_clean":"2024-01-24T14:10:59.710964+0000","last_became_active":"2024-01-24T14:10:59.248063+0000","last_became_peered":"2024-01-24T14:10:59.248063+0000","last_unstale":"2024-01-24T14:10:59.710964+0000","last_undegraded":"2024-01-24T14:10:59.710964+0000","last_fullsized":"2024-01-24T14:10:59.710964+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.729943+0000","last_change":"2024-01-24T14:10:59.714670+0000","last_active":"2024-01-24T14:10:59.729943+0000","last_peered":"2024-01-24T14:10:59.729943+0000","last_clean":"2024-01-24T14:10:59.729943+0000","last_became_active":"2024-01-24T14:10:59.714126+0000","last_became_peered":"2024-01-24T14:10:59.714126+0000","last_unstale":"2024-01-24T14:10:59.729943+0000","last_undegraded":"2024-01-24T14:10:59.729943+0000","last_fullsized":"2024-01-24T14:10:59.729943+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.702002+0000","last_change":"2024-01-24T14:10:59.254046+0000","last_active":"2024-01-24T14:10:59.702002+0000","last_peered":"2024-01-24T14:10:59.702002+0000","last_clean":"2024-01-24T14:10:59.702002+0000","last_became_active":"2024-01-24T14:10:59.253731+0000","last_became_peered":"2024-01-24T14:10:59.253731+0000","last_unstale":"2024-01-24T14:10:59.702002+0000","last_undegraded":"2024-01-24T14:10:59.702002+0000","last_fullsized":"2024-01-24T14:10:59.702002+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730941+0000","last_change":"2024-01-24T14:10:59.720630+0000","last_active":"2024-01-24T14:10:59.730941+0000","last_peered":"2024-01-24T14:10:59.730941+0000","last_clean":"2024-01-24T14:10:59.730941+0000","last_became_active":"2024-01-24T14:10:59.720357+0000","last_became_peered":"2024-01-24T14:10:59.720357+0000","last_unstale":"2024-01-24T14:10:59.730941+0000","last_undegraded":"2024-01-24T14:10:59.730941+0000","last_fullsized":"2024-01-24T14:10:59.730941+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.706025+0000","last_change":"2024-01-24T14:10:59.251278+0000","last_active":"2024-01-24T14:10:59.706025+0000","last_peered":"2024-01-24T14:10:59.706025+0000","last_clean":"2024-01-24T14:10:59.706025+0000","last_became_active":"2024-01-24T14:10:59.250925+0000","last_became_peered":"2024-01-24T14:10:59.250925+0000","last_unstale":"2024-01-24T14:10:59.706025+0000","last_undegraded":"2024-01-24T14:10:59.706025+0000","last_fullsized":"2024-01-24T14:10:59.706025+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.717321+0000","last_change":"2024-01-24T14:10:59.713943+0000","last_active":"2024-01-24T14:10:59.717321+0000","last_peered":"2024-01-24T14:10:59.717321+0000","last_clean":"2024-01-24T14:10:59.717321+0000","last_became_active":"2024-01-24T14:10:59.713270+0000","last_became_peered":"2024-01-24T14:10:59.713270+0000","last_unstale":"2024-01-24T14:10:59.717321+0000","last_undegraded":"2024-01-24T14:10:59.717321+0000","last_fullsized":"2024-01-24T14:10:59.717321+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711000+0000","last_change":"2024-01-24T14:10:59.254271+0000","last_active":"2024-01-24T14:10:59.711000+0000","last_peered":"2024-01-24T14:10:59.711000+0000","last_clean":"2024-01-24T14:10:59.711000+0000","last_became_active":"2024-01-24T14:10:59.253957+0000","last_became_peered":"2024-01-24T14:10:59.253957+0000","last_unstale":"2024-01-24T14:10:59.711000+0000","last_undegraded":"2024-01-24T14:10:59.711000+0000","last_fullsized":"2024-01-24T14:10:59.711000+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730214+0000","last_change":"2024-01-24T14:10:59.714769+0000","last_active":"2024-01-24T14:10:59.730214+0000","last_peered":"2024-01-24T14:10:59.730214+0000","last_clean":"2024-01-24T14:10:59.730214+0000","last_became_active":"2024-01-24T14:10:59.714307+0000","last_became_peered":"2024-01-24T14:10:59.714307+0000","last_unstale":"2024-01-24T14:10:59.730214+0000","last_undegraded":"2024-01-24T14:10:59.730214+0000","last_fullsized":"2024-01-24T14:10:59.730214+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.702156+0000","last_change":"2024-01-24T14:10:59.252651+0000","last_active":"2024-01-24T14:10:59.702156+0000","last_peered":"2024-01-24T14:10:59.702156+0000","last_clean":"2024-01-24T14:10:59.702156+0000","last_became_active":"2024-01-24T14:10:59.252357+0000","last_became_peered":"2024-01-24T14:10:59.252357+0000","last_unstale":"2024-01-24T14:10:59.702156+0000","last_undegraded":"2024-01-24T14:10:59.702156+0000","last_fullsized":"2024-01-24T14:10:59.702156+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.717662+0000","last_change":"2024-01-24T14:10:59.708895+0000","last_active":"2024-01-24T14:10:59.717662+0000","last_peered":"2024-01-24T14:10:59.717662+0000","last_clean":"2024-01-24T14:10:59.717662+0000","last_became_active":"2024-01-24T14:10:59.708600+0000","last_became_peered":"2024-01-24T14:10:59.708600+0000","last_unstale":"2024-01-24T14:10:59.717662+0000","last_undegraded":"2024-01-24T14:10:59.717662+0000","last_fullsized":"2024-01-24T14:10:59.717662+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.707908+0000","last_change":"2024-01-24T14:10:59.252594+0000","last_active":"2024-01-24T14:10:59.707908+0000","last_peered":"2024-01-24T14:10:59.707908+0000","last_clean":"2024-01-24T14:10:59.707908+0000","last_became_active":"2024-01-24T14:10:59.252323+0000","last_became_peered":"2024-01-24T14:10:59.252323+0000","last_unstale":"2024-01-24T14:10:59.707908+0000","last_undegraded":"2024-01-24T14:10:59.707908+0000","last_fullsized":"2024-01-24T14:10:59.707908+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.735476+0000","last_change":"2024-01-24T14:10:59.733106+0000","last_active":"2024-01-24T14:10:59.735476+0000","last_peered":"2024-01-24T14:10:59.735476+0000","last_clean":"2024-01-24T14:10:59.735476+0000","last_became_active":"2024-01-24T14:10:59.732820+0000","last_became_peered":"2024-01-24T14:10:59.732820+0000","last_unstale":"2024-01-24T14:10:59.735476+0000","last_undegraded":"2024-01-24T14:10:59.735476+0000","last_fullsized":"2024-01-24T14:10:59.735476+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711288+0000","last_change":"2024-01-24T14:10:59.245753+0000","last_active":"2024-01-24T14:10:59.711288+0000","last_peered":"2024-01-24T14:10:59.711288+0000","last_clean":"2024-01-24T14:10:59.711288+0000","last_became_active":"2024-01-24T14:10:59.245398+0000","last_became_peered":"2024-01-24T14:10:59.245398+0000","last_unstale":"2024-01-24T14:10:59.711288+0000","last_undegraded":"2024-01-24T14:10:59.711288+0000","last_fullsized":"2024-01-24T14:10:59.711288+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.734165+0000","last_change":"2024-01-24T14:10:59.723010+0000","last_active":"2024-01-24T14:10:59.734165+0000","last_peered":"2024-01-24T14:10:59.734165+0000","last_clean":"2024-01-24T14:10:59.734165+0000","last_became_active":"2024-01-24T14:10:59.722578+0000","last_became_peered":"2024-01-24T14:10:59.722578+0000","last_unstale":"2024-01-24T14:10:59.734165+0000","last_undegraded":"2024-01-24T14:10:59.734165+0000","last_fullsized":"2024-01-24T14:10:59.734165+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"71'3","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.733932+0000","last_change":"2024-01-24T14:10:59.248002+0000","last_active":"2024-01-24T14:10:59.733932+0000","last_peered":"2024-01-24T14:10:59.733932+0000","last_clean":"2024-01-24T14:10:59.733932+0000","last_became_active":"2024-01-24T14:10:59.247737+0000","last_became_peered":"2024-01-24T14:10:59.247737+0000","last_unstale":"2024-01-24T14:10:59.733932+0000","last_undegraded":"2024-01-24T14:10:59.733932+0000","last_fullsized":"2024-01-24T14:10:59.733932+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.735460+0000","last_change":"2024-01-24T14:10:59.733030+0000","last_active":"2024-01-24T14:10:59.735460+0000","last_peered":"2024-01-24T14:10:59.735460+0000","last_clean":"2024-01-24T14:10:59.735460+0000","last_became_active":"2024-01-24T14:10:59.732752+0000","last_became_peered":"2024-01-24T14:10:59.732752+0000","last_unstale":"2024-01-24T14:10:59.735460+0000","last_undegraded":"2024-01-24T14:10:59.735460+0000","last_fullsized":"2024-01-24T14:10:59.735460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725066+0000","last_change":"2024-01-24T14:10:59.254883+0000","last_active":"2024-01-24T14:10:59.725066+0000","last_peered":"2024-01-24T14:10:59.725066+0000","last_clean":"2024-01-24T14:10:59.725066+0000","last_became_active":"2024-01-24T14:10:59.254539+0000","last_became_peered":"2024-01-24T14:10:59.254539+0000","last_unstale":"2024-01-24T14:10:59.725066+0000","last_undegraded":"2024-01-24T14:10:59.725066+0000","last_fullsized":"2024-01-24T14:10:59.725066+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730891+0000","last_change":"2024-01-24T14:10:59.729755+0000","last_active":"2024-01-24T14:10:59.730891+0000","last_peered":"2024-01-24T14:10:59.730891+0000","last_clean":"2024-01-24T14:10:59.730891+0000","last_became_active":"2024-01-24T14:10:59.729448+0000","last_became_peered":"2024-01-24T14:10:59.729448+0000","last_unstale":"2024-01-24T14:10:59.730891+0000","last_undegraded":"2024-01-24T14:10:59.730891+0000","last_fullsized":"2024-01-24T14:10:59.730891+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731303+0000","last_change":"2024-01-24T14:10:59.245411+0000","last_active":"2024-01-24T14:10:59.731303+0000","last_peered":"2024-01-24T14:10:59.731303+0000","last_clean":"2024-01-24T14:10:59.731303+0000","last_became_active":"2024-01-24T14:10:59.245066+0000","last_became_peered":"2024-01-24T14:10:59.245066+0000","last_unstale":"2024-01-24T14:10:59.731303+0000","last_undegraded":"2024-01-24T14:10:59.731303+0000","last_fullsized":"2024-01-24T14:10:59.731303+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724037+0000","last_change":"2024-01-24T14:10:59.719149+0000","last_active":"2024-01-24T14:10:59.724037+0000","last_peered":"2024-01-24T14:10:59.724037+0000","last_clean":"2024-01-24T14:10:59.724037+0000","last_became_active":"2024-01-24T14:10:59.718909+0000","last_became_peered":"2024-01-24T14:10:59.718909+0000","last_unstale":"2024-01-24T14:10:59.724037+0000","last_undegraded":"2024-01-24T14:10:59.724037+0000","last_fullsized":"2024-01-24T14:10:59.724037+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.701098+0000","last_change":"2024-01-24T14:10:59.245311+0000","last_active":"2024-01-24T14:10:59.701098+0000","last_peered":"2024-01-24T14:10:59.701098+0000","last_clean":"2024-01-24T14:10:59.701098+0000","last_became_active":"2024-01-24T14:10:59.245042+0000","last_became_peered":"2024-01-24T14:10:59.245042+0000","last_unstale":"2024-01-24T14:10:59.701098+0000","last_undegraded":"2024-01-24T14:10:59.701098+0000","last_fullsized":"2024-01-24T14:10:59.701098+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.717900+0000","last_change":"2024-01-24T14:10:59.709940+0000","last_active":"2024-01-24T14:10:59.717900+0000","last_peered":"2024-01-24T14:10:59.717900+0000","last_clean":"2024-01-24T14:10:59.717900+0000","last_became_active":"2024-01-24T14:10:59.709663+0000","last_became_peered":"2024-01-24T14:10:59.709663+0000","last_unstale":"2024-01-24T14:10:59.717900+0000","last_undegraded":"2024-01-24T14:10:59.717900+0000","last_fullsized":"2024-01-24T14:10:59.717900+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711104+0000","last_change":"2024-01-24T14:10:59.250674+0000","last_active":"2024-01-24T14:10:59.711104+0000","last_peered":"2024-01-24T14:10:59.711104+0000","last_clean":"2024-01-24T14:10:59.711104+0000","last_became_active":"2024-01-24T14:10:59.250346+0000","last_became_peered":"2024-01-24T14:10:59.250346+0000","last_unstale":"2024-01-24T14:10:59.711104+0000","last_undegraded":"2024-01-24T14:10:59.711104+0000","last_fullsized":"2024-01-24T14:10:59.711104+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.719753+0000","last_change":"2024-01-24T14:10:59.712829+0000","last_active":"2024-01-24T14:10:59.719753+0000","last_peered":"2024-01-24T14:10:59.719753+0000","last_clean":"2024-01-24T14:10:59.719753+0000","last_became_active":"2024-01-24T14:10:59.712474+0000","last_became_peered":"2024-01-24T14:10:59.712474+0000","last_unstale":"2024-01-24T14:10:59.719753+0000","last_undegraded":"2024-01-24T14:10:59.719753+0000","last_fullsized":"2024-01-24T14:10:59.719753+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"71'4","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.722355+0000","last_change":"2024-01-24T14:10:59.243121+0000","last_active":"2024-01-24T14:10:59.722355+0000","last_peered":"2024-01-24T14:10:59.722355+0000","last_clean":"2024-01-24T14:10:59.722355+0000","last_became_active":"2024-01-24T14:10:59.242850+0000","last_became_peered":"2024-01-24T14:10:59.242850+0000","last_unstale":"2024-01-24T14:10:59.722355+0000","last_undegraded":"2024-01-24T14:10:59.722355+0000","last_fullsized":"2024-01-24T14:10:59.722355+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.721632+0000","last_change":"2024-01-24T14:10:59.714287+0000","last_active":"2024-01-24T14:10:59.721632+0000","last_peered":"2024-01-24T14:10:59.721632+0000","last_clean":"2024-01-24T14:10:59.721632+0000","last_became_active":"2024-01-24T14:10:59.713927+0000","last_became_peered":"2024-01-24T14:10:59.713927+0000","last_unstale":"2024-01-24T14:10:59.721632+0000","last_undegraded":"2024-01-24T14:10:59.721632+0000","last_fullsized":"2024-01-24T14:10:59.721632+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.733286+0000","last_change":"2024-01-24T14:10:59.256094+0000","last_active":"2024-01-24T14:10:59.733286+0000","last_peered":"2024-01-24T14:10:59.733286+0000","last_clean":"2024-01-24T14:10:59.733286+0000","last_became_active":"2024-01-24T14:10:59.254906+0000","last_became_peered":"2024-01-24T14:10:59.254906+0000","last_unstale":"2024-01-24T14:10:59.733286+0000","last_undegraded":"2024-01-24T14:10:59.733286+0000","last_fullsized":"2024-01-24T14:10:59.733286+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.723005+0000","last_change":"2024-01-24T14:10:59.718313+0000","last_active":"2024-01-24T14:10:59.723005+0000","last_peered":"2024-01-24T14:10:59.723005+0000","last_clean":"2024-01-24T14:10:59.723005+0000","last_became_active":"2024-01-24T14:10:59.718034+0000","last_became_peered":"2024-01-24T14:10:59.718034+0000","last_unstale":"2024-01-24T14:10:59.723005+0000","last_undegraded":"2024-01-24T14:10:59.723005+0000","last_fullsized":"2024-01-24T14:10:59.723005+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.706702+0000","last_change":"2024-01-24T14:10:59.249010+0000","last_active":"2024-01-24T14:10:59.706702+0000","last_peered":"2024-01-24T14:10:59.706702+0000","last_clean":"2024-01-24T14:10:59.706702+0000","last_became_active":"2024-01-24T14:10:59.248751+0000","last_became_peered":"2024-01-24T14:10:59.248751+0000","last_unstale":"2024-01-24T14:10:59.706702+0000","last_undegraded":"2024-01-24T14:10:59.706702+0000","last_fullsized":"2024-01-24T14:10:59.706702+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.734135+0000","last_change":"2024-01-24T14:10:59.722799+0000","last_active":"2024-01-24T14:10:59.734135+0000","last_peered":"2024-01-24T14:10:59.734135+0000","last_clean":"2024-01-24T14:10:59.734135+0000","last_became_active":"2024-01-24T14:10:59.722182+0000","last_became_peered":"2024-01-24T14:10:59.722182+0000","last_unstale":"2024-01-24T14:10:59.734135+0000","last_undegraded":"2024-01-24T14:10:59.734135+0000","last_fullsized":"2024-01-24T14:10:59.734135+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.701416+0000","last_change":"2024-01-24T14:10:59.254892+0000","last_active":"2024-01-24T14:10:59.701416+0000","last_peered":"2024-01-24T14:10:59.701416+0000","last_clean":"2024-01-24T14:10:59.701416+0000","last_became_active":"2024-01-24T14:10:59.254606+0000","last_became_peered":"2024-01-24T14:10:59.254606+0000","last_unstale":"2024-01-24T14:10:59.701416+0000","last_undegraded":"2024-01-24T14:10:59.701416+0000","last_fullsized":"2024-01-24T14:10:59.701416+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725429+0000","last_change":"2024-01-24T14:10:59.718937+0000","last_active":"2024-01-24T14:10:59.725429+0000","last_peered":"2024-01-24T14:10:59.725429+0000","last_clean":"2024-01-24T14:10:59.725429+0000","last_became_active":"2024-01-24T14:10:59.718337+0000","last_became_peered":"2024-01-24T14:10:59.718337+0000","last_unstale":"2024-01-24T14:10:59.725429+0000","last_undegraded":"2024-01-24T14:10:59.725429+0000","last_fullsized":"2024-01-24T14:10:59.725429+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711233+0000","last_change":"2024-01-24T14:10:59.256851+0000","last_active":"2024-01-24T14:10:59.711233+0000","last_peered":"2024-01-24T14:10:59.711233+0000","last_clean":"2024-01-24T14:10:59.711233+0000","last_became_active":"2024-01-24T14:10:59.256551+0000","last_became_peered":"2024-01-24T14:10:59.256551+0000","last_unstale":"2024-01-24T14:10:59.711233+0000","last_undegraded":"2024-01-24T14:10:59.711233+0000","last_fullsized":"2024-01-24T14:10:59.711233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731290+0000","last_change":"2024-01-24T14:10:59.726296+0000","last_active":"2024-01-24T14:10:59.731290+0000","last_peered":"2024-01-24T14:10:59.731290+0000","last_clean":"2024-01-24T14:10:59.731290+0000","last_became_active":"2024-01-24T14:10:59.725859+0000","last_became_peered":"2024-01-24T14:10:59.725859+0000","last_unstale":"2024-01-24T14:10:59.731290+0000","last_undegraded":"2024-01-24T14:10:59.731290+0000","last_fullsized":"2024-01-24T14:10:59.731290+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.708589+0000","last_change":"2024-01-24T14:10:59.247220+0000","last_active":"2024-01-24T14:10:59.708589+0000","last_peered":"2024-01-24T14:10:59.708589+0000","last_clean":"2024-01-24T14:10:59.708589+0000","last_became_active":"2024-01-24T14:10:59.246553+0000","last_became_peered":"2024-01-24T14:10:59.246553+0000","last_unstale":"2024-01-24T14:10:59.708589+0000","last_undegraded":"2024-01-24T14:10:59.708589+0000","last_fullsized":"2024-01-24T14:10:59.708589+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.717332+0000","last_change":"2024-01-24T14:10:59.711443+0000","last_active":"2024-01-24T14:10:59.717332+0000","last_peered":"2024-01-24T14:10:59.717332+0000","last_clean":"2024-01-24T14:10:59.717332+0000","last_became_active":"2024-01-24T14:10:59.711120+0000","last_became_peered":"2024-01-24T14:10:59.711120+0000","last_unstale":"2024-01-24T14:10:59.717332+0000","last_undegraded":"2024-01-24T14:10:59.717332+0000","last_fullsized":"2024-01-24T14:10:59.717332+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.701868+0000","last_change":"2024-01-24T14:10:59.254697+0000","last_active":"2024-01-24T14:10:59.701868+0000","last_peered":"2024-01-24T14:10:59.701868+0000","last_clean":"2024-01-24T14:10:59.701868+0000","last_became_active":"2024-01-24T14:10:59.254415+0000","last_became_peered":"2024-01-24T14:10:59.254415+0000","last_unstale":"2024-01-24T14:10:59.701868+0000","last_undegraded":"2024-01-24T14:10:59.701868+0000","last_fullsized":"2024-01-24T14:10:59.701868+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.733864+0000","last_change":"2024-01-24T14:10:59.728552+0000","last_active":"2024-01-24T14:10:59.733864+0000","last_peered":"2024-01-24T14:10:59.733864+0000","last_clean":"2024-01-24T14:10:59.733864+0000","last_became_active":"2024-01-24T14:10:59.728066+0000","last_became_peered":"2024-01-24T14:10:59.728066+0000","last_unstale":"2024-01-24T14:10:59.733864+0000","last_undegraded":"2024-01-24T14:10:59.733864+0000","last_fullsized":"2024-01-24T14:10:59.733864+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.709279+0000","last_change":"2024-01-24T14:10:59.256071+0000","last_active":"2024-01-24T14:10:59.709279+0000","last_peered":"2024-01-24T14:10:59.709279+0000","last_clean":"2024-01-24T14:10:59.709279+0000","last_became_active":"2024-01-24T14:10:59.255745+0000","last_became_peered":"2024-01-24T14:10:59.255745+0000","last_unstale":"2024-01-24T14:10:59.709279+0000","last_undegraded":"2024-01-24T14:10:59.709279+0000","last_fullsized":"2024-01-24T14:10:59.709279+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730193+0000","last_change":"2024-01-24T14:10:59.714593+0000","last_active":"2024-01-24T14:10:59.730193+0000","last_peered":"2024-01-24T14:10:59.730193+0000","last_clean":"2024-01-24T14:10:59.730193+0000","last_became_active":"2024-01-24T14:10:59.713932+0000","last_became_peered":"2024-01-24T14:10:59.713932+0000","last_unstale":"2024-01-24T14:10:59.730193+0000","last_undegraded":"2024-01-24T14:10:59.730193+0000","last_fullsized":"2024-01-24T14:10:59.730193+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.706976+0000","last_change":"2024-01-24T14:10:59.241277+0000","last_active":"2024-01-24T14:10:59.706976+0000","last_peered":"2024-01-24T14:10:59.706976+0000","last_clean":"2024-01-24T14:10:59.706976+0000","last_became_active":"2024-01-24T14:10:59.240964+0000","last_became_peered":"2024-01-24T14:10:59.240964+0000","last_unstale":"2024-01-24T14:10:59.706976+0000","last_undegraded":"2024-01-24T14:10:59.706976+0000","last_fullsized":"2024-01-24T14:10:59.706976+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732472+0000","last_change":"2024-01-24T14:10:59.731171+0000","last_active":"2024-01-24T14:10:59.732472+0000","last_peered":"2024-01-24T14:10:59.732472+0000","last_clean":"2024-01-24T14:10:59.732472+0000","last_became_active":"2024-01-24T14:10:59.730879+0000","last_became_peered":"2024-01-24T14:10:59.730879+0000","last_unstale":"2024-01-24T14:10:59.732472+0000","last_undegraded":"2024-01-24T14:10:59.732472+0000","last_fullsized":"2024-01-24T14:10:59.732472+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.719333+0000","last_change":"2024-01-24T14:10:59.254672+0000","last_active":"2024-01-24T14:10:59.719333+0000","last_peered":"2024-01-24T14:10:59.719333+0000","last_clean":"2024-01-24T14:10:59.719333+0000","last_became_active":"2024-01-24T14:10:59.254321+0000","last_became_peered":"2024-01-24T14:10:59.254321+0000","last_unstale":"2024-01-24T14:10:59.719333+0000","last_undegraded":"2024-01-24T14:10:59.719333+0000","last_fullsized":"2024-01-24T14:10:59.719333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.704485+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:10:59.704485+0000","last_peered":"2024-01-24T14:10:59.704485+0000","last_clean":"2024-01-24T14:10:59.704485+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:10:59.704485+0000","last_undegraded":"2024-01-24T14:10:59.704485+0000","last_fullsized":"2024-01-24T14:10:59.704485+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725066+0000","last_change":"2024-01-24T14:10:59.717924+0000","last_active":"2024-01-24T14:10:59.725066+0000","last_peered":"2024-01-24T14:10:59.725066+0000","last_clean":"2024-01-24T14:10:59.725066+0000","last_became_active":"2024-01-24T14:10:59.717728+0000","last_became_peered":"2024-01-24T14:10:59.717728+0000","last_unstale":"2024-01-24T14:10:59.725066+0000","last_undegraded":"2024-01-24T14:10:59.725066+0000","last_fullsized":"2024-01-24T14:10:59.725066+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.710214+0000","last_change":"2024-01-24T14:10:59.251159+0000","last_active":"2024-01-24T14:10:59.710214+0000","last_peered":"2024-01-24T14:10:59.710214+0000","last_clean":"2024-01-24T14:10:59.710214+0000","last_became_active":"2024-01-24T14:10:59.250861+0000","last_became_peered":"2024-01-24T14:10:59.250861+0000","last_unstale":"2024-01-24T14:10:59.710214+0000","last_undegraded":"2024-01-24T14:10:59.710214+0000","last_fullsized":"2024-01-24T14:10:59.710214+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":62,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.710658+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:10:59.710658+0000","last_peered":"2024-01-24T14:10:59.710658+0000","last_clean":"2024-01-24T14:10:59.710658+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:10:59.710658+0000","last_undegraded":"2024-01-24T14:10:59.710658+0000","last_fullsized":"2024-01-24T14:10:59.710658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.734716+0000","last_change":"2024-01-24T14:10:59.733631+0000","last_active":"2024-01-24T14:10:59.734716+0000","last_peered":"2024-01-24T14:10:59.734716+0000","last_clean":"2024-01-24T14:10:59.734716+0000","last_became_active":"2024-01-24T14:10:59.733408+0000","last_became_peered":"2024-01-24T14:10:59.733408+0000","last_unstale":"2024-01-24T14:10:59.734716+0000","last_undegraded":"2024-01-24T14:10:59.734716+0000","last_fullsized":"2024-01-24T14:10:59.734716+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730885+0000","last_change":"2024-01-24T14:10:59.258912+0000","last_active":"2024-01-24T14:10:59.730885+0000","last_peered":"2024-01-24T14:10:59.730885+0000","last_clean":"2024-01-24T14:10:59.730885+0000","last_became_active":"2024-01-24T14:10:59.258572+0000","last_became_peered":"2024-01-24T14:10:59.258572+0000","last_unstale":"2024-01-24T14:10:59.730885+0000","last_undegraded":"2024-01-24T14:10:59.730885+0000","last_fullsized":"2024-01-24T14:10:59.730885+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.698330+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:10:59.698330+0000","last_peered":"2024-01-24T14:10:59.698330+0000","last_clean":"2024-01-24T14:10:59.698330+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:10:59.698330+0000","last_undegraded":"2024-01-24T14:10:59.698330+0000","last_fullsized":"2024-01-24T14:10:59.698330+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731320+0000","last_change":"2024-01-24T14:10:59.720032+0000","last_active":"2024-01-24T14:10:59.731320+0000","last_peered":"2024-01-24T14:10:59.731320+0000","last_clean":"2024-01-24T14:10:59.731320+0000","last_became_active":"2024-01-24T14:10:59.719535+0000","last_became_peered":"2024-01-24T14:10:59.719535+0000","last_unstale":"2024-01-24T14:10:59.731320+0000","last_undegraded":"2024-01-24T14:10:59.731320+0000","last_fullsized":"2024-01-24T14:10:59.731320+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.700816+0000","last_change":"2024-01-24T14:10:59.243012+0000","last_active":"2024-01-24T14:10:59.700816+0000","last_peered":"2024-01-24T14:10:59.700816+0000","last_clean":"2024-01-24T14:10:59.700816+0000","last_became_active":"2024-01-24T14:10:59.242722+0000","last_became_peered":"2024-01-24T14:10:59.242722+0000","last_unstale":"2024-01-24T14:10:59.700816+0000","last_undegraded":"2024-01-24T14:10:59.700816+0000","last_fullsized":"2024-01-24T14:10:59.700816+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":62,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711759+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:10:59.711759+0000","last_peered":"2024-01-24T14:10:59.711759+0000","last_clean":"2024-01-24T14:10:59.711759+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:10:59.711759+0000","last_undegraded":"2024-01-24T14:10:59.711759+0000","last_fullsized":"2024-01-24T14:10:59.711759+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":61,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705343+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:10:59.705343+0000","last_peered":"2024-01-24T14:10:59.705343+0000","last_clean":"2024-01-24T14:10:59.705343+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:10:59.705343+0000","last_undegraded":"2024-01-24T14:10:59.705343+0000","last_fullsized":"2024-01-24T14:10:59.705343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.727396+0000","last_change":"2024-01-24T14:10:59.721991+0000","last_active":"2024-01-24T14:10:59.727396+0000","last_peered":"2024-01-24T14:10:59.727396+0000","last_clean":"2024-01-24T14:10:59.727396+0000","last_became_active":"2024-01-24T14:10:59.721672+0000","last_became_peered":"2024-01-24T14:10:59.721672+0000","last_unstale":"2024-01-24T14:10:59.727396+0000","last_undegraded":"2024-01-24T14:10:59.727396+0000","last_fullsized":"2024-01-24T14:10:59.727396+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705188+0000","last_change":"2024-01-24T14:10:59.241490+0000","last_active":"2024-01-24T14:10:59.705188+0000","last_peered":"2024-01-24T14:10:59.705188+0000","last_clean":"2024-01-24T14:10:59.705188+0000","last_became_active":"2024-01-24T14:10:59.241195+0000","last_became_peered":"2024-01-24T14:10:59.241195+0000","last_unstale":"2024-01-24T14:10:59.705188+0000","last_undegraded":"2024-01-24T14:10:59.705188+0000","last_fullsized":"2024-01-24T14:10:59.705188+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705253+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:10:59.705253+0000","last_peered":"2024-01-24T14:10:59.705253+0000","last_clean":"2024-01-24T14:10:59.705253+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:10:59.705253+0000","last_undegraded":"2024-01-24T14:10:59.705253+0000","last_fullsized":"2024-01-24T14:10:59.705253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724127+0000","last_change":"2024-01-24T14:10:59.720486+0000","last_active":"2024-01-24T14:10:59.724127+0000","last_peered":"2024-01-24T14:10:59.724127+0000","last_clean":"2024-01-24T14:10:59.724127+0000","last_became_active":"2024-01-24T14:10:59.720206+0000","last_became_peered":"2024-01-24T14:10:59.720206+0000","last_unstale":"2024-01-24T14:10:59.724127+0000","last_undegraded":"2024-01-24T14:10:59.724127+0000","last_fullsized":"2024-01-24T14:10:59.724127+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731287+0000","last_change":"2024-01-24T14:10:59.238503+0000","last_active":"2024-01-24T14:10:59.731287+0000","last_peered":"2024-01-24T14:10:59.731287+0000","last_clean":"2024-01-24T14:10:59.731287+0000","last_became_active":"2024-01-24T14:10:59.238177+0000","last_became_peered":"2024-01-24T14:10:59.238177+0000","last_unstale":"2024-01-24T14:10:59.731287+0000","last_undegraded":"2024-01-24T14:10:59.731287+0000","last_fullsized":"2024-01-24T14:10:59.731287+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.707334+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:10:59.707334+0000","last_peered":"2024-01-24T14:10:59.707334+0000","last_clean":"2024-01-24T14:10:59.707334+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:10:59.707334+0000","last_undegraded":"2024-01-24T14:10:59.707334+0000","last_fullsized":"2024-01-24T14:10:59.707334+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.735478+0000","last_change":"2024-01-24T14:10:59.733001+0000","last_active":"2024-01-24T14:10:59.735478+0000","last_peered":"2024-01-24T14:10:59.735478+0000","last_clean":"2024-01-24T14:10:59.735478+0000","last_became_active":"2024-01-24T14:10:59.732735+0000","last_became_peered":"2024-01-24T14:10:59.732735+0000","last_unstale":"2024-01-24T14:10:59.735478+0000","last_undegraded":"2024-01-24T14:10:59.735478+0000","last_fullsized":"2024-01-24T14:10:59.735478+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.708408+0000","last_change":"2024-01-24T14:10:59.261830+0000","last_active":"2024-01-24T14:10:59.708408+0000","last_peered":"2024-01-24T14:10:59.708408+0000","last_clean":"2024-01-24T14:10:59.708408+0000","last_became_active":"2024-01-24T14:10:59.261502+0000","last_became_peered":"2024-01-24T14:10:59.261502+0000","last_unstale":"2024-01-24T14:10:59.708408+0000","last_undegraded":"2024-01-24T14:10:59.708408+0000","last_fullsized":"2024-01-24T14:10:59.708408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705065+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:10:59.705065+0000","last_peered":"2024-01-24T14:10:59.705065+0000","last_clean":"2024-01-24T14:10:59.705065+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:10:59.705065+0000","last_undegraded":"2024-01-24T14:10:59.705065+0000","last_fullsized":"2024-01-24T14:10:59.705065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730971+0000","last_change":"2024-01-24T14:10:59.712655+0000","last_active":"2024-01-24T14:10:59.730971+0000","last_peered":"2024-01-24T14:10:59.730971+0000","last_clean":"2024-01-24T14:10:59.730971+0000","last_became_active":"2024-01-24T14:10:59.712302+0000","last_became_peered":"2024-01-24T14:10:59.712302+0000","last_unstale":"2024-01-24T14:10:59.730971+0000","last_undegraded":"2024-01-24T14:10:59.730971+0000","last_fullsized":"2024-01-24T14:10:59.730971+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.710822+0000","last_change":"2024-01-24T14:10:59.248629+0000","last_active":"2024-01-24T14:10:59.710822+0000","last_peered":"2024-01-24T14:10:59.710822+0000","last_clean":"2024-01-24T14:10:59.710822+0000","last_became_active":"2024-01-24T14:10:59.248302+0000","last_became_peered":"2024-01-24T14:10:59.248302+0000","last_unstale":"2024-01-24T14:10:59.710822+0000","last_undegraded":"2024-01-24T14:10:59.710822+0000","last_fullsized":"2024-01-24T14:10:59.710822+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.701001+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:10:59.701001+0000","last_peered":"2024-01-24T14:10:59.701001+0000","last_clean":"2024-01-24T14:10:59.701001+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:10:59.701001+0000","last_undegraded":"2024-01-24T14:10:59.701001+0000","last_fullsized":"2024-01-24T14:10:59.701001+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297944,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073896,"statfs":{"total":96636764160,"available":96331669504,"internally_reserved":0,"allocated":868352,"data_stored":459179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57899999999999996,"15min":0.54100000000000004},"min":{"1min":0.42899999999999999,"5min":0.34899999999999998,"15min":0.26000000000000001},"max":{"1min":0.63400000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.41499999999999998,"15min":0.32700000000000001},"max":{"1min":0.75700000000000001,"5min":0.82599999999999996,"15min":0.83499999999999996},"last":0.76100000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61099999999999999,"15min":0.57599999999999996},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.45800000000000002,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.79100000000000004,"15min":3.2570000000000001},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.58799999999999997},"min":{"1min":0.54700000000000004,"5min":0.42499999999999999,"15min":0.32000000000000001},"max":{"1min":0.69399999999999995,"5min":0.82499999999999996,"15min":0.876},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.58999999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.36799999999999999},"max":{"1min":0.79600000000000004,"5min":0.80900000000000005,"15min":0.83099999999999996},"last":0.77100000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61099999999999999,"15min":0.58799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.36499999999999999},"max":{"1min":0.73699999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.35499999999999998},"max":{"1min":0.71799999999999997,"5min":0.77500000000000002,"15min":3.0049999999999999},"last":0.751}]},{"osd":4,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.48399999999999999},"min":{"1min":0.35399999999999998,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":0.68799999999999994,"5min":0.755,"15min":0.84699999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.496},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":0.71499999999999997,"15min":0.83599999999999997},"last":0.55600000000000005}]},{"osd":5,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.497,"15min":0.496},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.29999999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.84899999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41299999999999998,"5min":0.376,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":0.78600000000000003,"15min":0.80900000000000005},"last":0.64400000000000002}]},{"osd":6,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.34899999999999998,"15min":0.33000000000000002},"max":{"1min":0.67700000000000005,"5min":0.72299999999999998,"15min":0.81999999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.58699999999999997},"min":{"1min":0.42399999999999999,"5min":0.377,"15min":0.34999999999999998},"max":{"1min":0.73599999999999999,"5min":0.83399999999999996,"15min":0.87},"last":0.70699999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302040,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069800,"statfs":{"total":96636764160,"available":96327475200,"internally_reserved":0,"allocated":868352,"data_stored":459981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82599999999999996,"5min":0.85699999999999998,"15min":0.86899999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.29999999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":1.3859999999999999},"last":0.60599999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.433,"5min":0.39300000000000002,"15min":0.35399999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":1.353},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58999999999999997,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.497}]},{"osd":2,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61199999999999999,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.34300000000000003},"max":{"1min":0.78300000000000003,"5min":0.83499999999999996,"15min":6.8129999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.44600000000000001,"5min":0.442,"15min":0.40000000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":1.4079999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.629,"15min":0.66200000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":7.016},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.38900000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.496,"15min":0.502},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.34399999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":6.8719999999999999},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54000000000000004,"15min":0.57299999999999995},"min":{"1min":0.40699999999999997,"5min":0.34699999999999998,"15min":0.32400000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":6.976},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52800000000000002,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.35899999999999999,"15min":0.33600000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":7.8789999999999996},"last":0.64800000000000002}]},{"osd":7,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.63200000000000001},"min":{"1min":0.44500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":6.9080000000000004},"last":0.70299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302044,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069796,"statfs":{"total":96636764160,"available":96327471104,"internally_reserved":0,"allocated":872448,"data_stored":459733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.27500000000000002,"15min":0.23400000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.311},{"interface":"front","average":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.315,"5min":0.254,"15min":0.254},"max":{"1min":0.57599999999999996,"5min":0.62,"15min":0.65600000000000003},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45500000000000002,"15min":0.443},"min":{"1min":0.34999999999999998,"5min":0.28000000000000003,"15min":0.26000000000000001},"max":{"1min":0.60499999999999998,"5min":0.64900000000000002,"15min":0.70799999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45100000000000001,"15min":0.437},"min":{"1min":0.35799999999999998,"5min":0.28199999999999997,"15min":0.25800000000000001},"max":{"1min":0.67900000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34000000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45900000000000002,"15min":0.44600000000000001},"min":{"1min":0.37,"5min":0.32900000000000001,"15min":0.309},"max":{"1min":0.64800000000000002,"5min":0.67700000000000005,"15min":0.71899999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.46500000000000002},"min":{"1min":0.40500000000000003,"5min":0.34799999999999998,"15min":0.33400000000000002},"max":{"1min":0.68200000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.50600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.91600000000000004,"5min":1.073,"15min":1.137},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.51200000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":1.218,"15min":1.218},"last":0.60799999999999998}]},{"osd":5,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":2.698,"5min":1.101,"15min":0.82199999999999995},"min":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.443},"max":{"1min":38.808,"5min":38.808,"15min":38.808},"last":0.79600000000000004},{"interface":"front","average":{"1min":2.6840000000000002,"5min":1.0760000000000001,"15min":0.80500000000000005},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":38.823,"5min":38.823,"15min":38.823},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.67300000000000004},"min":{"1min":0.52500000000000002,"5min":0.47199999999999998,"15min":0.43099999999999999},"max":{"1min":0.90700000000000003,"5min":1.0449999999999999,"15min":1.0509999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.47999999999999998,"15min":0.40000000000000002},"max":{"1min":0.95499999999999996,"5min":1.1559999999999999,"15min":1.1879999999999999},"last":0.76800000000000002}]},{"osd":7,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.45300000000000001},"max":{"1min":0.86199999999999999,"5min":1.105,"15min":1.155},"last":0.873},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.753,"15min":0.747},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":0.92400000000000004,"5min":1.123,"15min":1.131},"last":0.85099999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542301,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302044,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069796,"statfs":{"total":96636764160,"available":96327471104,"internally_reserved":0,"allocated":872448,"data_stored":458842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40300000000000002,"15min":0.40100000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.43099999999999999,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.66700000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.318,"15min":0.28799999999999998},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.79400000000000004},"last":0.32500000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.432,"15min":0.42299999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59299999999999997,"5min":0.63700000000000001,"15min":0.70099999999999996},"last":0.372},{"interface":"front","average":{"1min":0.503,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.34799999999999998,"15min":0.32100000000000001},"max":{"1min":0.71399999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82599999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64500000000000002,"15min":0.65000000000000002},"min":{"1min":0.439,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":1.0149999999999999,"15min":1.0429999999999999},"last":0.46999999999999997}]},{"osd":5,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.44900000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.99099999999999999},"last":0.439},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68799999999999994,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85899999999999999,"5min":0.98599999999999999,"15min":1.0840000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.498,"5min":0.48199999999999998,"15min":0.47499999999999998},"max":{"1min":0.90700000000000003,"5min":1.0309999999999999,"15min":1.137},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.51200000000000001,"5min":0.48799999999999999,"15min":0.48199999999999998},"max":{"1min":0.93100000000000005,"5min":0.95899999999999996,"15min":1.117},"last":0.52700000000000002}]},{"osd":7,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.47099999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.98199999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69299999999999995,"15min":0.68500000000000005},"min":{"1min":0.52000000000000002,"5min":0.503,"15min":0.48299999999999998},"max":{"1min":0.88300000000000001,"5min":0.96699999999999997,"15min":1.1799999999999999},"last":0.50600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575003,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306140,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94065700,"statfs":{"total":96636764160,"available":96323276800,"internally_reserved":0,"allocated":872448,"data_stored":459145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39400000000000002,"15min":0.38400000000000001},"min":{"1min":0.29399999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46700000000000003,"5min":0.64400000000000002,"15min":0.65800000000000003},"last":0.495},{"interface":"front","average":{"1min":0.436,"5min":0.43099999999999999,"15min":0.42199999999999999},"min":{"1min":0.32500000000000001,"5min":0.26300000000000001,"15min":0.24399999999999999},"max":{"1min":0.63600000000000001,"5min":0.65900000000000003,"15min":0.67100000000000004},"last":0.32000000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.39200000000000002},"min":{"1min":0.311,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.55600000000000005,"5min":0.61499999999999999,"15min":0.67200000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46000000000000002,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.29699999999999999,"15min":0.255},"max":{"1min":0.55800000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"last":0.41599999999999998}]},{"osd":3,"last update":"Wed Jan 24 14:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.438},"min":{"1min":0.309,"5min":0.309,"15min":0.30099999999999999},"max":{"1min":0.52700000000000002,"5min":0.68500000000000005,"15min":0.71699999999999997},"last":0.436},{"interface":"front","average":{"1min":0.44,"5min":0.46800000000000003,"15min":0.45900000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.57799999999999996,"5min":0.66200000000000003,"15min":0.72799999999999998},"last":0.45200000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65600000000000003,"15min":0.63300000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.41299999999999998},"max":{"1min":0.79500000000000004,"5min":0.96499999999999997,"15min":0.96699999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.46200000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.98599999999999999,"5min":1.008,"15min":1.042},"last":0.54100000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69099999999999995,"15min":0.66600000000000004},"min":{"1min":0.501,"5min":0.44800000000000001,"15min":0.443},"max":{"1min":0.94899999999999995,"5min":0.98699999999999999,"15min":1.0469999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.67300000000000004,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.877,"5min":0.97099999999999997,"15min":1.026},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Jan 24 14:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70699999999999996,"15min":0.68000000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92000000000000004,"5min":1.016,"15min":1.1100000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71999999999999997,"15min":0.70399999999999996},"min":{"1min":0.504,"5min":0.497,"15min":0.47799999999999998},"max":{"1min":0.94099999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57399999999999995}]},{"osd":7,"last update":"Wed Jan 24 14:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72899999999999998,"15min":0.70099999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91200000000000003,"5min":0.97299999999999998,"15min":1.054},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.70899999999999996},"min":{"1min":0.52200000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.89900000000000002,"5min":1.036,"15min":1.036},"last":0.621}]}]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302048,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069792,"statfs":{"total":96636764160,"available":96327467008,"internally_reserved":0,"allocated":876544,"data_stored":460037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39800000000000002,"15min":0.39100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41099999999999998,"15min":0.40999999999999998},"min":{"1min":0.312,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.57499999999999996,"5min":0.70299999999999996,"15min":0.71299999999999997},"last":0.38100000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.69099999999999995},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.45400000000000001,"15min":0.45200000000000001},"min":{"1min":0.312,"5min":0.311,"15min":0.28299999999999997},"max":{"1min":0.61499999999999999,"5min":0.66300000000000003,"15min":0.68799999999999994},"last":0.42599999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.498,"15min":0.495},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.60599999999999998,"5min":0.68799999999999994,"15min":0.77200000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.318},"max":{"1min":0.625,"5min":0.72999999999999998,"15min":0.78800000000000003},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64500000000000002,"15min":0.63300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.41799999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.624},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67400000000000004,"15min":0.65500000000000003},"min":{"1min":0.55200000000000005,"5min":0.44400000000000001,"15min":0.40200000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.93500000000000005,"5min":0.97599999999999998,"15min":0.98999999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.70199999999999996},"min":{"1min":0.53400000000000003,"5min":0.51200000000000001,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.98099999999999998,"15min":1.0369999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.45400000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.018},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.53100000000000003,"5min":0.48799999999999999,"15min":0.437},"max":{"1min":0.874,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63500000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72999999999999998,"15min":0.71299999999999997},"min":{"1min":0.52700000000000002,"5min":0.5,"15min":0.45200000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0740000000000001},"last":0.755},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.57899999999999996,"5min":0.502,"15min":0.45800000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302048,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069792,"statfs":{"total":96636764160,"available":96327467008,"internally_reserved":0,"allocated":876544,"data_stored":459803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.88600000000000001},"last":0.498},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59799999999999998,"15min":0.57899999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.38200000000000001},"max":{"1min":0.755,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48699999999999999}]},{"osd":1,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.79000000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79400000000000004,"5min":0.82599999999999996,"15min":1.506},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58899999999999997,"15min":0.56999999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":0.81999999999999995,"15min":0.83999999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61399999999999999,"15min":0.60099999999999998},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54100000000000004}]},{"osd":3,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.77800000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.36499999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.86299999999999999},"last":0.42999999999999999}]},{"osd":5,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46700000000000003,"15min":0.497},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68600000000000005,"5min":0.75800000000000001,"15min":0.92000000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48499999999999999,"15min":0.52400000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65200000000000002,"5min":0.72599999999999998,"15min":1.556},"last":0.60799999999999998}]},{"osd":6,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46999999999999997,"15min":0.497},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.86099999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53000000000000003,"15min":0.55000000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":1.0409999999999999},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.54200000000000004},"min":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.33500000000000002},"max":{"1min":0.73199999999999998,"5min":0.76900000000000002,"15min":0.88100000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.57199999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":1.26},"last":0.60199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302040,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069800,"statfs":{"total":96636764160,"available":96327475200,"internally_reserved":0,"allocated":868352,"data_stored":459643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35699999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":1.101},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":1.1439999999999999},"last":0.65200000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.58099999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.093},"last":0.751},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":1.1639999999999999},"last":0.56200000000000006}]},{"osd":2,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.33500000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":1.1100000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":1.1220000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38500000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":1.139},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.62,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":0.82599999999999996,"15min":1.0609999999999999},"last":0.83099999999999996}]},{"osd":4,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48699999999999999,"15min":0.48999999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72199999999999998,"5min":0.72599999999999998,"15min":0.82499999999999996},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.49299999999999999,"15min":0.501},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72799999999999998,"5min":0.81100000000000005,"15min":0.90000000000000002},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.49199999999999999,"15min":0.50800000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.81100000000000005,"15min":1.1839999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.746,"5min":0.78900000000000003,"15min":0.872},"last":0.92800000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71999999999999997,"5min":0.79000000000000004,"15min":0.93700000000000006},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.58199999999999996},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81499999999999995,"5min":0.84099999999999997,"15min":1.2030000000000001},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:11:03.904 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:11:04.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.069+0000 7f2a972e4700 1 Processor -- start 2024-01-24T14:11:04.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.070+0000 7f2a972e4700 1 -- start start 2024-01-24T14:11:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.070+0000 7f2a972e4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2a90134cc0 0x7f2a900c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.070+0000 7f2a972e4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a901350a0 0x7f2a90152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.070+0000 7f2a972e4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a90068360 con 0x7f2a900664f0 2024-01-24T14:11:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a972e4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a90153360 con 0x7f2a90134cc0 2024-01-24T14:11:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a972e4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a900c3820 con 0x7f2a901350a0 2024-01-24T14:11:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a95080700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2a90134cc0 0x7f2a900c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a95080700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2a90134cc0 0x7f2a900c31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42676/0 (socket says 172.21.15.29:42676) 2024-01-24T14:11:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a95080700 1 -- 172.21.15.29:0/1479744729 learned_addr learned my addr 172.21.15.29:0/1479744729 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a9487f700 1 --2- 172.21.15.29:0/1479744729 >> v2:172.21.15.163:3300/0 conn(0x7f2a901350a0 0x7f2a90152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a95080700 1 -- 172.21.15.29:0/1479744729 >> v2:172.21.15.163:3300/0 conn(0x7f2a901350a0 msgr2=0x7f2a90152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a95080700 1 --2- 172.21.15.29:0/1479744729 >> v2:172.21.15.163:3300/0 conn(0x7f2a901350a0 0x7f2a90152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a95080700 1 -- 172.21.15.29:0/1479744729 >> v1:172.21.15.163:6789/0 conn(0x7f2a900664f0 legacy=0x7f2a90155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a95080700 1 -- 172.21.15.29:0/1479744729 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a90067a30 con 0x7f2a90134cc0 2024-01-24T14:11:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a95080700 1 --2- 172.21.15.29:0/1479744729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a90134cc0 0x7f2a900c31b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f2a900aed60 tx=0x7f2a8c0090f0).ready entity=mon.0 client_cookie=1c18fec1e997488d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a87fff700 1 -- 172.21.15.29:0/1479744729 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00b600 con 0x7f2a90134cc0 2024-01-24T14:11:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a87fff700 1 -- 172.21.15.29:0/1479744729 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c00b760 con 0x7f2a90134cc0 2024-01-24T14:11:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.071+0000 7f2a87fff700 1 -- 172.21.15.29:0/1479744729 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00bae0 con 0x7f2a90134cc0 2024-01-24T14:11:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.072+0000 7f2a972e4700 1 -- 172.21.15.29:0/1479744729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a90134cc0 msgr2=0x7f2a900c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.072+0000 7f2a972e4700 1 --2- 172.21.15.29:0/1479744729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a90134cc0 0x7f2a900c31b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f2a900aed60 tx=0x7f2a8c0090f0).stop 2024-01-24T14:11:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.072+0000 7f2a972e4700 1 -- 172.21.15.29:0/1479744729 shutdown_connections 2024-01-24T14:11:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.072+0000 7f2a972e4700 1 --2- 172.21.15.29:0/1479744729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a90134cc0 0x7f2a900c31b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.072+0000 7f2a972e4700 1 --2- 172.21.15.29:0/1479744729 >> v2:172.21.15.163:3300/0 conn(0x7f2a901350a0 0x7f2a90152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.072+0000 7f2a972e4700 1 -- 172.21.15.29:0/1479744729 >> 172.21.15.29:0/1479744729 conn(0x7f2a900c8bb0 msgr2=0x7f2a900c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:04.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.072+0000 7f2a972e4700 1 -- 172.21.15.29:0/1479744729 shutdown_connections 2024-01-24T14:11:04.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.072+0000 7f2a972e4700 1 -- 172.21.15.29:0/1479744729 wait complete. 2024-01-24T14:11:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.073+0000 7f2a972e4700 1 Processor -- start 2024-01-24T14:11:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.073+0000 7f2a972e4700 1 -- start start 2024-01-24T14:11:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.074+0000 7f2a972e4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a900664f0 0x7f2a90133e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.074+0000 7f2a972e4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a90134cc0 0x7f2a90155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.074+0000 7f2a972e4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a901350a0 0x7f2a901655c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.074+0000 7f2a972e4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90134b80 con 0x7f2a90134cc0 2024-01-24T14:11:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.074+0000 7f2a972e4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90155df0 con 0x7f2a901350a0 2024-01-24T14:11:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.074+0000 7f2a972e4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90165ab0 con 0x7f2a900664f0 2024-01-24T14:11:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a9487f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a90134cc0 0x7f2a90155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a9487f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a90134cc0 0x7f2a90155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42686/0 (socket says 172.21.15.29:42686) 2024-01-24T14:11:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a9487f700 1 -- 172.21.15.29:0/3046366442 learned_addr learned my addr 172.21.15.29:0/3046366442 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a95080700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a900664f0 0x7f2a90133e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a95881700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a901350a0 0x7f2a901655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a9487f700 1 -- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a900664f0 msgr2=0x7f2a90133e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a9487f700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a900664f0 0x7f2a90133e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a9487f700 1 -- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a901350a0 msgr2=0x7f2a901655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a9487f700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a901350a0 0x7f2a901655c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a9487f700 1 -- 172.21.15.29:0/3046366442 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c009370 con 0x7f2a90134cc0 2024-01-24T14:11:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a95080700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a900664f0 0x7f2a90133e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:11:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a9487f700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a90134cc0 0x7f2a90155900 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f2a8000d8c0 tx=0x7f2a80008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a857fa700 1 -- 172.21.15.29:0/3046366442 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80011590 con 0x7f2a90134cc0 2024-01-24T14:11:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a857fa700 1 -- 172.21.15.29:0/3046366442 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a800116f0 con 0x7f2a90134cc0 2024-01-24T14:11:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a857fa700 1 -- 172.21.15.29:0/3046366442 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80011a70 con 0x7f2a90134cc0 2024-01-24T14:11:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a972e4700 1 -- 172.21.15.29:0/3046366442 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a90165d80 con 0x7f2a90134cc0 2024-01-24T14:11:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.075+0000 7f2a972e4700 1 -- 172.21.15.29:0/3046366442 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a90166340 con 0x7f2a90134cc0 2024-01-24T14:11:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.076+0000 7f2a972e4700 1 -- 172.21.15.29:0/3046366442 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a900c3190 con 0x7f2a90134cc0 2024-01-24T14:11:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.077+0000 7f2a857fa700 1 -- 172.21.15.29:0/3046366442 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2a8001c460 con 0x7f2a90134cc0 2024-01-24T14:11:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.077+0000 7f2a857fa700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2a7405f240 0x7f2a74061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.077+0000 7f2a857fa700 1 -- 172.21.15.29:0/3046366442 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f2a8007a4e0 con 0x7f2a90134cc0 2024-01-24T14:11:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.078+0000 7f2a95080700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2a7405f240 0x7f2a74061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.078+0000 7f2a95080700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2a7405f240 0x7f2a74061700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f2a8c009130 tx=0x7f2a8c017000).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:04.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.082+0000 7f2a857fa700 1 -- 172.21.15.29:0/3046366442 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2a8004b1d0 con 0x7f2a90134cc0 2024-01-24T14:11:04.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.261+0000 7f2a972e4700 1 -- 172.21.15.29:0/3046366442 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a900475e0 con 0x7f2a7405f240 2024-01-24T14:11:04.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.272+0000 7f2a857fa700 1 -- 172.21.15.29:0/3046366442 <== mgr.5524 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361249 (secure 0 0 0) 0x7f2a900475e0 con 0x7f2a7405f240 2024-01-24T14:11:04.277 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:11:04.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.275+0000 7f2a972e4700 1 -- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2a7405f240 msgr2=0x7f2a74061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.275+0000 7f2a972e4700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2a7405f240 0x7f2a74061700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f2a8c009130 tx=0x7f2a8c017000).stop 2024-01-24T14:11:04.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.275+0000 7f2a972e4700 1 -- 172.21.15.29:0/3046366442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a90134cc0 msgr2=0x7f2a90155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.275+0000 7f2a972e4700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a90134cc0 0x7f2a90155900 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f2a8000d8c0 tx=0x7f2a80008040).stop 2024-01-24T14:11:04.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.275+0000 7f2a972e4700 1 -- 172.21.15.29:0/3046366442 shutdown_connections 2024-01-24T14:11:04.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.275+0000 7f2a972e4700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a901350a0 0x7f2a901655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.275+0000 7f2a972e4700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a900664f0 0x7f2a90133e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.275+0000 7f2a972e4700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2a7405f240 0x7f2a74061700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.275+0000 7f2a972e4700 1 --2- 172.21.15.29:0/3046366442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a90134cc0 0x7f2a90155900 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.275+0000 7f2a972e4700 1 -- 172.21.15.29:0/3046366442 >> 172.21.15.29:0/3046366442 conn(0x7f2a900c8bb0 msgr2=0x7f2a900c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:04.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.276+0000 7f2a972e4700 1 -- 172.21.15.29:0/3046366442 shutdown_connections 2024-01-24T14:11:04.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.276+0000 7f2a972e4700 1 -- 172.21.15.29:0/3046366442 wait complete. 2024-01-24T14:11:04.280 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:11:04.289 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-01-24T14:11:04.184735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":240,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2416348,"kb_used_data":6812,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752558372,"statfs":{"total":773094113280,"available":770619772928,"internally_reserved":0,"allocated":6975488,"data_stored":3676363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15000139,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-719,"num_write_kb":-15551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013814"},"pg_stats":[{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.729964+0000","last_change":"2024-01-24T14:10:59.715939+0000","last_active":"2024-01-24T14:10:59.729964+0000","last_peered":"2024-01-24T14:10:59.729964+0000","last_clean":"2024-01-24T14:10:59.729964+0000","last_became_active":"2024-01-24T14:10:59.715657+0000","last_became_peered":"2024-01-24T14:10:59.715657+0000","last_unstale":"2024-01-24T14:10:59.729964+0000","last_undegraded":"2024-01-24T14:10:59.729964+0000","last_fullsized":"2024-01-24T14:10:59.729964+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3f","version":"71'3","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.733438+0000","last_change":"2024-01-24T14:10:59.247665+0000","last_active":"2024-01-24T14:10:59.733438+0000","last_peered":"2024-01-24T14:10:59.733438+0000","last_clean":"2024-01-24T14:10:59.733438+0000","last_became_active":"2024-01-24T14:10:59.246919+0000","last_became_peered":"2024-01-24T14:10:59.246919+0000","last_unstale":"2024-01-24T14:10:59.733438+0000","last_undegraded":"2024-01-24T14:10:59.733438+0000","last_fullsized":"2024-01-24T14:10:59.733438+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730005+0000","last_change":"2024-01-24T14:10:59.712157+0000","last_active":"2024-01-24T14:10:59.730005+0000","last_peered":"2024-01-24T14:10:59.730005+0000","last_clean":"2024-01-24T14:10:59.730005+0000","last_became_active":"2024-01-24T14:10:59.711793+0000","last_became_peered":"2024-01-24T14:10:59.711793+0000","last_unstale":"2024-01-24T14:10:59.730005+0000","last_undegraded":"2024-01-24T14:10:59.730005+0000","last_fullsized":"2024-01-24T14:10:59.730005+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.708141+0000","last_change":"2024-01-24T14:10:59.248350+0000","last_active":"2024-01-24T14:10:59.708141+0000","last_peered":"2024-01-24T14:10:59.708141+0000","last_clean":"2024-01-24T14:10:59.708141+0000","last_became_active":"2024-01-24T14:10:59.247993+0000","last_became_peered":"2024-01-24T14:10:59.247993+0000","last_unstale":"2024-01-24T14:10:59.708141+0000","last_undegraded":"2024-01-24T14:10:59.708141+0000","last_fullsized":"2024-01-24T14:10:59.708141+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.723143+0000","last_change":"2024-01-24T14:10:59.719672+0000","last_active":"2024-01-24T14:10:59.723143+0000","last_peered":"2024-01-24T14:10:59.723143+0000","last_clean":"2024-01-24T14:10:59.723143+0000","last_became_active":"2024-01-24T14:10:59.719350+0000","last_became_peered":"2024-01-24T14:10:59.719350+0000","last_unstale":"2024-01-24T14:10:59.723143+0000","last_undegraded":"2024-01-24T14:10:59.723143+0000","last_fullsized":"2024-01-24T14:10:59.723143+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.708311+0000","last_change":"2024-01-24T14:10:59.254110+0000","last_active":"2024-01-24T14:10:59.708311+0000","last_peered":"2024-01-24T14:10:59.708311+0000","last_clean":"2024-01-24T14:10:59.708311+0000","last_became_active":"2024-01-24T14:10:59.253768+0000","last_became_peered":"2024-01-24T14:10:59.253768+0000","last_unstale":"2024-01-24T14:10:59.708311+0000","last_undegraded":"2024-01-24T14:10:59.708311+0000","last_fullsized":"2024-01-24T14:10:59.708311+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724000+0000","last_change":"2024-01-24T14:10:59.718793+0000","last_active":"2024-01-24T14:10:59.724000+0000","last_peered":"2024-01-24T14:10:59.724000+0000","last_clean":"2024-01-24T14:10:59.724000+0000","last_became_active":"2024-01-24T14:10:59.718510+0000","last_became_peered":"2024-01-24T14:10:59.718510+0000","last_unstale":"2024-01-24T14:10:59.724000+0000","last_undegraded":"2024-01-24T14:10:59.724000+0000","last_fullsized":"2024-01-24T14:10:59.724000+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3c","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731575+0000","last_change":"2024-01-24T14:10:59.253305+0000","last_active":"2024-01-24T14:10:59.731575+0000","last_peered":"2024-01-24T14:10:59.731575+0000","last_clean":"2024-01-24T14:10:59.731575+0000","last_became_active":"2024-01-24T14:10:59.253015+0000","last_became_peered":"2024-01-24T14:10:59.253015+0000","last_unstale":"2024-01-24T14:10:59.731575+0000","last_undegraded":"2024-01-24T14:10:59.731575+0000","last_fullsized":"2024-01-24T14:10:59.731575+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725365+0000","last_change":"2024-01-24T14:10:59.718476+0000","last_active":"2024-01-24T14:10:59.725365+0000","last_peered":"2024-01-24T14:10:59.725365+0000","last_clean":"2024-01-24T14:10:59.725365+0000","last_became_active":"2024-01-24T14:10:59.718185+0000","last_became_peered":"2024-01-24T14:10:59.718185+0000","last_unstale":"2024-01-24T14:10:59.725365+0000","last_undegraded":"2024-01-24T14:10:59.725365+0000","last_fullsized":"2024-01-24T14:10:59.725365+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.718463+0000","last_change":"2024-01-24T14:10:59.250903+0000","last_active":"2024-01-24T14:10:59.718463+0000","last_peered":"2024-01-24T14:10:59.718463+0000","last_clean":"2024-01-24T14:10:59.718463+0000","last_became_active":"2024-01-24T14:10:59.250583+0000","last_became_peered":"2024-01-24T14:10:59.250583+0000","last_unstale":"2024-01-24T14:10:59.718463+0000","last_undegraded":"2024-01-24T14:10:59.718463+0000","last_fullsized":"2024-01-24T14:10:59.718463+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.729816+0000","last_change":"2024-01-24T14:10:59.712738+0000","last_active":"2024-01-24T14:10:59.729816+0000","last_peered":"2024-01-24T14:10:59.729816+0000","last_clean":"2024-01-24T14:10:59.729816+0000","last_became_active":"2024-01-24T14:10:59.712443+0000","last_became_peered":"2024-01-24T14:10:59.712443+0000","last_unstale":"2024-01-24T14:10:59.729816+0000","last_undegraded":"2024-01-24T14:10:59.729816+0000","last_fullsized":"2024-01-24T14:10:59.729816+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732686+0000","last_change":"2024-01-24T14:10:59.249275+0000","last_active":"2024-01-24T14:10:59.732686+0000","last_peered":"2024-01-24T14:10:59.732686+0000","last_clean":"2024-01-24T14:10:59.732686+0000","last_became_active":"2024-01-24T14:10:59.248982+0000","last_became_peered":"2024-01-24T14:10:59.248982+0000","last_unstale":"2024-01-24T14:10:59.732686+0000","last_undegraded":"2024-01-24T14:10:59.732686+0000","last_fullsized":"2024-01-24T14:10:59.732686+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731298+0000","last_change":"2024-01-24T14:10:59.717196+0000","last_active":"2024-01-24T14:10:59.731298+0000","last_peered":"2024-01-24T14:10:59.731298+0000","last_clean":"2024-01-24T14:10:59.731298+0000","last_became_active":"2024-01-24T14:10:59.716919+0000","last_became_peered":"2024-01-24T14:10:59.716919+0000","last_unstale":"2024-01-24T14:10:59.731298+0000","last_undegraded":"2024-01-24T14:10:59.731298+0000","last_fullsized":"2024-01-24T14:10:59.731298+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.707722+0000","last_change":"2024-01-24T14:10:59.254462+0000","last_active":"2024-01-24T14:10:59.707722+0000","last_peered":"2024-01-24T14:10:59.707722+0000","last_clean":"2024-01-24T14:10:59.707722+0000","last_became_active":"2024-01-24T14:10:59.253839+0000","last_became_peered":"2024-01-24T14:10:59.253839+0000","last_unstale":"2024-01-24T14:10:59.707722+0000","last_undegraded":"2024-01-24T14:10:59.707722+0000","last_fullsized":"2024-01-24T14:10:59.707722+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.720589+0000","last_change":"2024-01-24T14:10:59.715610+0000","last_active":"2024-01-24T14:10:59.720589+0000","last_peered":"2024-01-24T14:10:59.720589+0000","last_clean":"2024-01-24T14:10:59.720589+0000","last_became_active":"2024-01-24T14:10:59.715227+0000","last_became_peered":"2024-01-24T14:10:59.715227+0000","last_unstale":"2024-01-24T14:10:59.720589+0000","last_undegraded":"2024-01-24T14:10:59.720589+0000","last_fullsized":"2024-01-24T14:10:59.720589+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.706874+0000","last_change":"2024-01-24T14:10:59.252933+0000","last_active":"2024-01-24T14:10:59.706874+0000","last_peered":"2024-01-24T14:10:59.706874+0000","last_clean":"2024-01-24T14:10:59.706874+0000","last_became_active":"2024-01-24T14:10:59.252461+0000","last_became_peered":"2024-01-24T14:10:59.252461+0000","last_unstale":"2024-01-24T14:10:59.706874+0000","last_undegraded":"2024-01-24T14:10:59.706874+0000","last_fullsized":"2024-01-24T14:10:59.706874+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.733154+0000","last_change":"2024-01-24T14:10:59.727871+0000","last_active":"2024-01-24T14:10:59.733154+0000","last_peered":"2024-01-24T14:10:59.733154+0000","last_clean":"2024-01-24T14:10:59.733154+0000","last_became_active":"2024-01-24T14:10:59.727602+0000","last_became_peered":"2024-01-24T14:10:59.727602+0000","last_unstale":"2024-01-24T14:10:59.733154+0000","last_undegraded":"2024-01-24T14:10:59.733154+0000","last_fullsized":"2024-01-24T14:10:59.733154+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711282+0000","last_change":"2024-01-24T14:10:59.253063+0000","last_active":"2024-01-24T14:10:59.711282+0000","last_peered":"2024-01-24T14:10:59.711282+0000","last_clean":"2024-01-24T14:10:59.711282+0000","last_became_active":"2024-01-24T14:10:59.252748+0000","last_became_peered":"2024-01-24T14:10:59.252748+0000","last_unstale":"2024-01-24T14:10:59.711282+0000","last_undegraded":"2024-01-24T14:10:59.711282+0000","last_fullsized":"2024-01-24T14:10:59.711282+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.726716+0000","last_change":"2024-01-24T14:10:59.723498+0000","last_active":"2024-01-24T14:10:59.726716+0000","last_peered":"2024-01-24T14:10:59.726716+0000","last_clean":"2024-01-24T14:10:59.726716+0000","last_became_active":"2024-01-24T14:10:59.723252+0000","last_became_peered":"2024-01-24T14:10:59.723252+0000","last_unstale":"2024-01-24T14:10:59.726716+0000","last_undegraded":"2024-01-24T14:10:59.726716+0000","last_fullsized":"2024-01-24T14:10:59.726716+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.704981+0000","last_change":"2024-01-24T14:10:59.255826+0000","last_active":"2024-01-24T14:10:59.704981+0000","last_peered":"2024-01-24T14:10:59.704981+0000","last_clean":"2024-01-24T14:10:59.704981+0000","last_became_active":"2024-01-24T14:10:59.255489+0000","last_became_peered":"2024-01-24T14:10:59.255489+0000","last_unstale":"2024-01-24T14:10:59.704981+0000","last_undegraded":"2024-01-24T14:10:59.704981+0000","last_fullsized":"2024-01-24T14:10:59.704981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.722974+0000","last_change":"2024-01-24T14:10:59.718057+0000","last_active":"2024-01-24T14:10:59.722974+0000","last_peered":"2024-01-24T14:10:59.722974+0000","last_clean":"2024-01-24T14:10:59.722974+0000","last_became_active":"2024-01-24T14:10:59.717755+0000","last_became_peered":"2024-01-24T14:10:59.717755+0000","last_unstale":"2024-01-24T14:10:59.722974+0000","last_undegraded":"2024-01-24T14:10:59.722974+0000","last_fullsized":"2024-01-24T14:10:59.722974+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.698049+0000","last_change":"2024-01-24T14:10:59.245756+0000","last_active":"2024-01-24T14:10:59.698049+0000","last_peered":"2024-01-24T14:10:59.698049+0000","last_clean":"2024-01-24T14:10:59.698049+0000","last_became_active":"2024-01-24T14:10:59.245448+0000","last_became_peered":"2024-01-24T14:10:59.245448+0000","last_unstale":"2024-01-24T14:10:59.698049+0000","last_undegraded":"2024-01-24T14:10:59.698049+0000","last_fullsized":"2024-01-24T14:10:59.698049+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732752+0000","last_change":"2024-01-24T14:10:59.716176+0000","last_active":"2024-01-24T14:10:59.732752+0000","last_peered":"2024-01-24T14:10:59.732752+0000","last_clean":"2024-01-24T14:10:59.732752+0000","last_became_active":"2024-01-24T14:10:59.715901+0000","last_became_peered":"2024-01-24T14:10:59.715901+0000","last_unstale":"2024-01-24T14:10:59.732752+0000","last_undegraded":"2024-01-24T14:10:59.732752+0000","last_fullsized":"2024-01-24T14:10:59.732752+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.34","version":"71'4","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.735298+0000","last_change":"2024-01-24T14:10:59.245051+0000","last_active":"2024-01-24T14:10:59.735298+0000","last_peered":"2024-01-24T14:10:59.735298+0000","last_clean":"2024-01-24T14:10:59.735298+0000","last_became_active":"2024-01-24T14:10:59.244749+0000","last_became_peered":"2024-01-24T14:10:59.244749+0000","last_unstale":"2024-01-24T14:10:59.735298+0000","last_undegraded":"2024-01-24T14:10:59.735298+0000","last_fullsized":"2024-01-24T14:10:59.735298+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.717746+0000","last_change":"2024-01-24T14:10:59.713714+0000","last_active":"2024-01-24T14:10:59.717746+0000","last_peered":"2024-01-24T14:10:59.717746+0000","last_clean":"2024-01-24T14:10:59.717746+0000","last_became_active":"2024-01-24T14:10:59.713418+0000","last_became_peered":"2024-01-24T14:10:59.713418+0000","last_unstale":"2024-01-24T14:10:59.717746+0000","last_undegraded":"2024-01-24T14:10:59.717746+0000","last_fullsized":"2024-01-24T14:10:59.717746+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.33","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732550+0000","last_change":"2024-01-24T14:10:59.255680+0000","last_active":"2024-01-24T14:10:59.732550+0000","last_peered":"2024-01-24T14:10:59.732550+0000","last_clean":"2024-01-24T14:10:59.732550+0000","last_became_active":"2024-01-24T14:10:59.255269+0000","last_became_peered":"2024-01-24T14:10:59.255269+0000","last_unstale":"2024-01-24T14:10:59.732550+0000","last_undegraded":"2024-01-24T14:10:59.732550+0000","last_fullsized":"2024-01-24T14:10:59.732550+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.723840+0000","last_change":"2024-01-24T14:10:59.718254+0000","last_active":"2024-01-24T14:10:59.723840+0000","last_peered":"2024-01-24T14:10:59.723840+0000","last_clean":"2024-01-24T14:10:59.723840+0000","last_became_active":"2024-01-24T14:10:59.717982+0000","last_became_peered":"2024-01-24T14:10:59.717982+0000","last_unstale":"2024-01-24T14:10:59.723840+0000","last_undegraded":"2024-01-24T14:10:59.723840+0000","last_fullsized":"2024-01-24T14:10:59.723840+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705919+0000","last_change":"2024-01-24T14:10:59.249757+0000","last_active":"2024-01-24T14:10:59.705919+0000","last_peered":"2024-01-24T14:10:59.705919+0000","last_clean":"2024-01-24T14:10:59.705919+0000","last_became_active":"2024-01-24T14:10:59.249347+0000","last_became_peered":"2024-01-24T14:10:59.249347+0000","last_unstale":"2024-01-24T14:10:59.705919+0000","last_undegraded":"2024-01-24T14:10:59.705919+0000","last_fullsized":"2024-01-24T14:10:59.705919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725160+0000","last_change":"2024-01-24T14:10:59.718124+0000","last_active":"2024-01-24T14:10:59.725160+0000","last_peered":"2024-01-24T14:10:59.725160+0000","last_clean":"2024-01-24T14:10:59.725160+0000","last_became_active":"2024-01-24T14:10:59.717797+0000","last_became_peered":"2024-01-24T14:10:59.717797+0000","last_unstale":"2024-01-24T14:10:59.725160+0000","last_undegraded":"2024-01-24T14:10:59.725160+0000","last_fullsized":"2024-01-24T14:10:59.725160+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705961+0000","last_change":"2024-01-24T14:10:59.249006+0000","last_active":"2024-01-24T14:10:59.705961+0000","last_peered":"2024-01-24T14:10:59.705961+0000","last_clean":"2024-01-24T14:10:59.705961+0000","last_became_active":"2024-01-24T14:10:59.248706+0000","last_became_peered":"2024-01-24T14:10:59.248706+0000","last_unstale":"2024-01-24T14:10:59.705961+0000","last_undegraded":"2024-01-24T14:10:59.705961+0000","last_fullsized":"2024-01-24T14:10:59.705961+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730075+0000","last_change":"2024-01-24T14:10:59.729586+0000","last_active":"2024-01-24T14:10:59.730075+0000","last_peered":"2024-01-24T14:10:59.730075+0000","last_clean":"2024-01-24T14:10:59.730075+0000","last_became_active":"2024-01-24T14:10:59.729257+0000","last_became_peered":"2024-01-24T14:10:59.729257+0000","last_unstale":"2024-01-24T14:10:59.730075+0000","last_undegraded":"2024-01-24T14:10:59.730075+0000","last_fullsized":"2024-01-24T14:10:59.730075+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.30","version":"71'4","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732867+0000","last_change":"2024-01-24T14:10:59.247241+0000","last_active":"2024-01-24T14:10:59.732867+0000","last_peered":"2024-01-24T14:10:59.732867+0000","last_clean":"2024-01-24T14:10:59.732867+0000","last_became_active":"2024-01-24T14:10:59.246844+0000","last_became_peered":"2024-01-24T14:10:59.246844+0000","last_unstale":"2024-01-24T14:10:59.732867+0000","last_undegraded":"2024-01-24T14:10:59.732867+0000","last_fullsized":"2024-01-24T14:10:59.732867+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724531+0000","last_change":"2024-01-24T14:10:59.721721+0000","last_active":"2024-01-24T14:10:59.724531+0000","last_peered":"2024-01-24T14:10:59.724531+0000","last_clean":"2024-01-24T14:10:59.724531+0000","last_became_active":"2024-01-24T14:10:59.721345+0000","last_became_peered":"2024-01-24T14:10:59.721345+0000","last_unstale":"2024-01-24T14:10:59.724531+0000","last_undegraded":"2024-01-24T14:10:59.724531+0000","last_fullsized":"2024-01-24T14:10:59.724531+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.709686+0000","last_change":"2024-01-24T14:10:59.250419+0000","last_active":"2024-01-24T14:10:59.709686+0000","last_peered":"2024-01-24T14:10:59.709686+0000","last_clean":"2024-01-24T14:10:59.709686+0000","last_became_active":"2024-01-24T14:10:59.250069+0000","last_became_peered":"2024-01-24T14:10:59.250069+0000","last_unstale":"2024-01-24T14:10:59.709686+0000","last_undegraded":"2024-01-24T14:10:59.709686+0000","last_fullsized":"2024-01-24T14:10:59.709686+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731091+0000","last_change":"2024-01-24T14:10:59.722739+0000","last_active":"2024-01-24T14:10:59.731091+0000","last_peered":"2024-01-24T14:10:59.731091+0000","last_clean":"2024-01-24T14:10:59.731091+0000","last_became_active":"2024-01-24T14:10:59.722517+0000","last_became_peered":"2024-01-24T14:10:59.722517+0000","last_unstale":"2024-01-24T14:10:59.731091+0000","last_undegraded":"2024-01-24T14:10:59.731091+0000","last_fullsized":"2024-01-24T14:10:59.731091+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.708251+0000","last_change":"2024-01-24T14:10:59.248434+0000","last_active":"2024-01-24T14:10:59.708251+0000","last_peered":"2024-01-24T14:10:59.708251+0000","last_clean":"2024-01-24T14:10:59.708251+0000","last_became_active":"2024-01-24T14:10:59.248143+0000","last_became_peered":"2024-01-24T14:10:59.248143+0000","last_unstale":"2024-01-24T14:10:59.708251+0000","last_undegraded":"2024-01-24T14:10:59.708251+0000","last_fullsized":"2024-01-24T14:10:59.708251+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731043+0000","last_change":"2024-01-24T14:10:59.719177+0000","last_active":"2024-01-24T14:10:59.731043+0000","last_peered":"2024-01-24T14:10:59.731043+0000","last_clean":"2024-01-24T14:10:59.731043+0000","last_became_active":"2024-01-24T14:10:59.718882+0000","last_became_peered":"2024-01-24T14:10:59.718882+0000","last_unstale":"2024-01-24T14:10:59.731043+0000","last_undegraded":"2024-01-24T14:10:59.731043+0000","last_fullsized":"2024-01-24T14:10:59.731043+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2d","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.722197+0000","last_change":"2024-01-24T14:10:59.257334+0000","last_active":"2024-01-24T14:10:59.722197+0000","last_peered":"2024-01-24T14:10:59.722197+0000","last_clean":"2024-01-24T14:10:59.722197+0000","last_became_active":"2024-01-24T14:10:59.257013+0000","last_became_peered":"2024-01-24T14:10:59.257013+0000","last_unstale":"2024-01-24T14:10:59.722197+0000","last_undegraded":"2024-01-24T14:10:59.722197+0000","last_fullsized":"2024-01-24T14:10:59.722197+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732325+0000","last_change":"2024-01-24T14:10:59.718817+0000","last_active":"2024-01-24T14:10:59.732325+0000","last_peered":"2024-01-24T14:10:59.732325+0000","last_clean":"2024-01-24T14:10:59.732325+0000","last_became_active":"2024-01-24T14:10:59.713272+0000","last_became_peered":"2024-01-24T14:10:59.713272+0000","last_unstale":"2024-01-24T14:10:59.732325+0000","last_undegraded":"2024-01-24T14:10:59.732325+0000","last_fullsized":"2024-01-24T14:10:59.732325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.701011+0000","last_change":"2024-01-24T14:10:59.253760+0000","last_active":"2024-01-24T14:10:59.701011+0000","last_peered":"2024-01-24T14:10:59.701011+0000","last_clean":"2024-01-24T14:10:59.701011+0000","last_became_active":"2024-01-24T14:10:59.253458+0000","last_became_peered":"2024-01-24T14:10:59.253458+0000","last_unstale":"2024-01-24T14:10:59.701011+0000","last_undegraded":"2024-01-24T14:10:59.701011+0000","last_fullsized":"2024-01-24T14:10:59.701011+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.729967+0000","last_change":"2024-01-24T14:10:59.715051+0000","last_active":"2024-01-24T14:10:59.729967+0000","last_peered":"2024-01-24T14:10:59.729967+0000","last_clean":"2024-01-24T14:10:59.729967+0000","last_became_active":"2024-01-24T14:10:59.714515+0000","last_became_peered":"2024-01-24T14:10:59.714515+0000","last_unstale":"2024-01-24T14:10:59.729967+0000","last_undegraded":"2024-01-24T14:10:59.729967+0000","last_fullsized":"2024-01-24T14:10:59.729967+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724324+0000","last_change":"2024-01-24T14:10:59.248692+0000","last_active":"2024-01-24T14:10:59.724324+0000","last_peered":"2024-01-24T14:10:59.724324+0000","last_clean":"2024-01-24T14:10:59.724324+0000","last_became_active":"2024-01-24T14:10:59.248421+0000","last_became_peered":"2024-01-24T14:10:59.248421+0000","last_unstale":"2024-01-24T14:10:59.724324+0000","last_undegraded":"2024-01-24T14:10:59.724324+0000","last_fullsized":"2024-01-24T14:10:59.724324+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.717701+0000","last_change":"2024-01-24T14:10:59.714524+0000","last_active":"2024-01-24T14:10:59.717701+0000","last_peered":"2024-01-24T14:10:59.717701+0000","last_clean":"2024-01-24T14:10:59.717701+0000","last_became_active":"2024-01-24T14:10:59.714248+0000","last_became_peered":"2024-01-24T14:10:59.714248+0000","last_unstale":"2024-01-24T14:10:59.717701+0000","last_undegraded":"2024-01-24T14:10:59.717701+0000","last_fullsized":"2024-01-24T14:10:59.717701+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.707964+0000","last_change":"2024-01-24T14:10:59.260935+0000","last_active":"2024-01-24T14:10:59.707964+0000","last_peered":"2024-01-24T14:10:59.707964+0000","last_clean":"2024-01-24T14:10:59.707964+0000","last_became_active":"2024-01-24T14:10:59.260632+0000","last_became_peered":"2024-01-24T14:10:59.260632+0000","last_unstale":"2024-01-24T14:10:59.707964+0000","last_undegraded":"2024-01-24T14:10:59.707964+0000","last_fullsized":"2024-01-24T14:10:59.707964+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.723919+0000","last_change":"2024-01-24T14:10:59.719221+0000","last_active":"2024-01-24T14:10:59.723919+0000","last_peered":"2024-01-24T14:10:59.723919+0000","last_clean":"2024-01-24T14:10:59.723919+0000","last_became_active":"2024-01-24T14:10:59.718831+0000","last_became_peered":"2024-01-24T14:10:59.718831+0000","last_unstale":"2024-01-24T14:10:59.723919+0000","last_undegraded":"2024-01-24T14:10:59.723919+0000","last_fullsized":"2024-01-24T14:10:59.723919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.698243+0000","last_change":"2024-01-24T14:10:59.247208+0000","last_active":"2024-01-24T14:10:59.698243+0000","last_peered":"2024-01-24T14:10:59.698243+0000","last_clean":"2024-01-24T14:10:59.698243+0000","last_became_active":"2024-01-24T14:10:59.245845+0000","last_became_peered":"2024-01-24T14:10:59.245845+0000","last_unstale":"2024-01-24T14:10:59.698243+0000","last_undegraded":"2024-01-24T14:10:59.698243+0000","last_fullsized":"2024-01-24T14:10:59.698243+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.719732+0000","last_change":"2024-01-24T14:10:59.711935+0000","last_active":"2024-01-24T14:10:59.719732+0000","last_peered":"2024-01-24T14:10:59.719732+0000","last_clean":"2024-01-24T14:10:59.719732+0000","last_became_active":"2024-01-24T14:10:59.711637+0000","last_became_peered":"2024-01-24T14:10:59.711637+0000","last_unstale":"2024-01-24T14:10:59.719732+0000","last_undegraded":"2024-01-24T14:10:59.719732+0000","last_fullsized":"2024-01-24T14:10:59.719732+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.707081+0000","last_change":"2024-01-24T14:10:59.254043+0000","last_active":"2024-01-24T14:10:59.707081+0000","last_peered":"2024-01-24T14:10:59.707081+0000","last_clean":"2024-01-24T14:10:59.707081+0000","last_became_active":"2024-01-24T14:10:59.253710+0000","last_became_peered":"2024-01-24T14:10:59.253710+0000","last_unstale":"2024-01-24T14:10:59.707081+0000","last_undegraded":"2024-01-24T14:10:59.707081+0000","last_fullsized":"2024-01-24T14:10:59.707081+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.723172+0000","last_change":"2024-01-24T14:10:59.720897+0000","last_active":"2024-01-24T14:10:59.723172+0000","last_peered":"2024-01-24T14:10:59.723172+0000","last_clean":"2024-01-24T14:10:59.723172+0000","last_became_active":"2024-01-24T14:10:59.720599+0000","last_became_peered":"2024-01-24T14:10:59.720599+0000","last_unstale":"2024-01-24T14:10:59.723172+0000","last_undegraded":"2024-01-24T14:10:59.723172+0000","last_fullsized":"2024-01-24T14:10:59.723172+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.709784+0000","last_change":"2024-01-24T14:10:59.262835+0000","last_active":"2024-01-24T14:10:59.709784+0000","last_peered":"2024-01-24T14:10:59.709784+0000","last_clean":"2024-01-24T14:10:59.709784+0000","last_became_active":"2024-01-24T14:10:59.262585+0000","last_became_peered":"2024-01-24T14:10:59.262585+0000","last_unstale":"2024-01-24T14:10:59.709784+0000","last_undegraded":"2024-01-24T14:10:59.709784+0000","last_fullsized":"2024-01-24T14:10:59.709784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730027+0000","last_change":"2024-01-24T14:10:59.712491+0000","last_active":"2024-01-24T14:10:59.730027+0000","last_peered":"2024-01-24T14:10:59.730027+0000","last_clean":"2024-01-24T14:10:59.730027+0000","last_became_active":"2024-01-24T14:10:59.712206+0000","last_became_peered":"2024-01-24T14:10:59.712206+0000","last_unstale":"2024-01-24T14:10:59.730027+0000","last_undegraded":"2024-01-24T14:10:59.730027+0000","last_fullsized":"2024-01-24T14:10:59.730027+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711149+0000","last_change":"2024-01-24T14:10:59.246549+0000","last_active":"2024-01-24T14:10:59.711149+0000","last_peered":"2024-01-24T14:10:59.711149+0000","last_clean":"2024-01-24T14:10:59.711149+0000","last_became_active":"2024-01-24T14:10:59.246268+0000","last_became_peered":"2024-01-24T14:10:59.246268+0000","last_unstale":"2024-01-24T14:10:59.711149+0000","last_undegraded":"2024-01-24T14:10:59.711149+0000","last_fullsized":"2024-01-24T14:10:59.711149+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732158+0000","last_change":"2024-01-24T14:10:59.718560+0000","last_active":"2024-01-24T14:10:59.732158+0000","last_peered":"2024-01-24T14:10:59.732158+0000","last_clean":"2024-01-24T14:10:59.732158+0000","last_became_active":"2024-01-24T14:10:59.718277+0000","last_became_peered":"2024-01-24T14:10:59.718277+0000","last_unstale":"2024-01-24T14:10:59.732158+0000","last_undegraded":"2024-01-24T14:10:59.732158+0000","last_fullsized":"2024-01-24T14:10:59.732158+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705155+0000","last_change":"2024-01-24T14:10:59.255711+0000","last_active":"2024-01-24T14:10:59.705155+0000","last_peered":"2024-01-24T14:10:59.705155+0000","last_clean":"2024-01-24T14:10:59.705155+0000","last_became_active":"2024-01-24T14:10:59.255294+0000","last_became_peered":"2024-01-24T14:10:59.255294+0000","last_unstale":"2024-01-24T14:10:59.705155+0000","last_undegraded":"2024-01-24T14:10:59.705155+0000","last_fullsized":"2024-01-24T14:10:59.705155+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725444+0000","last_change":"2024-01-24T14:10:59.719130+0000","last_active":"2024-01-24T14:10:59.725444+0000","last_peered":"2024-01-24T14:10:59.725444+0000","last_clean":"2024-01-24T14:10:59.725444+0000","last_became_active":"2024-01-24T14:10:59.718656+0000","last_became_peered":"2024-01-24T14:10:59.718656+0000","last_unstale":"2024-01-24T14:10:59.725444+0000","last_undegraded":"2024-01-24T14:10:59.725444+0000","last_fullsized":"2024-01-24T14:10:59.725444+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.701102+0000","last_change":"2024-01-24T14:10:59.243916+0000","last_active":"2024-01-24T14:10:59.701102+0000","last_peered":"2024-01-24T14:10:59.701102+0000","last_clean":"2024-01-24T14:10:59.701102+0000","last_became_active":"2024-01-24T14:10:59.243607+0000","last_became_peered":"2024-01-24T14:10:59.243607+0000","last_unstale":"2024-01-24T14:10:59.701102+0000","last_undegraded":"2024-01-24T14:10:59.701102+0000","last_fullsized":"2024-01-24T14:10:59.701102+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724039+0000","last_change":"2024-01-24T14:10:59.719200+0000","last_active":"2024-01-24T14:10:59.724039+0000","last_peered":"2024-01-24T14:10:59.724039+0000","last_clean":"2024-01-24T14:10:59.724039+0000","last_became_active":"2024-01-24T14:10:59.718895+0000","last_became_peered":"2024-01-24T14:10:59.718895+0000","last_unstale":"2024-01-24T14:10:59.724039+0000","last_undegraded":"2024-01-24T14:10:59.724039+0000","last_fullsized":"2024-01-24T14:10:59.724039+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.706758+0000","last_change":"2024-01-24T14:10:59.259168+0000","last_active":"2024-01-24T14:10:59.706758+0000","last_peered":"2024-01-24T14:10:59.706758+0000","last_clean":"2024-01-24T14:10:59.706758+0000","last_became_active":"2024-01-24T14:10:59.258902+0000","last_became_peered":"2024-01-24T14:10:59.258902+0000","last_unstale":"2024-01-24T14:10:59.706758+0000","last_undegraded":"2024-01-24T14:10:59.706758+0000","last_fullsized":"2024-01-24T14:10:59.706758+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730814+0000","last_change":"2024-01-24T14:10:59.714078+0000","last_active":"2024-01-24T14:10:59.730814+0000","last_peered":"2024-01-24T14:10:59.730814+0000","last_clean":"2024-01-24T14:10:59.730814+0000","last_became_active":"2024-01-24T14:10:59.712655+0000","last_became_peered":"2024-01-24T14:10:59.712655+0000","last_unstale":"2024-01-24T14:10:59.730814+0000","last_undegraded":"2024-01-24T14:10:59.730814+0000","last_fullsized":"2024-01-24T14:10:59.730814+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.706003+0000","last_change":"2024-01-24T14:10:59.248954+0000","last_active":"2024-01-24T14:10:59.706003+0000","last_peered":"2024-01-24T14:10:59.706003+0000","last_clean":"2024-01-24T14:10:59.706003+0000","last_became_active":"2024-01-24T14:10:59.248653+0000","last_became_peered":"2024-01-24T14:10:59.248653+0000","last_unstale":"2024-01-24T14:10:59.706003+0000","last_undegraded":"2024-01-24T14:10:59.706003+0000","last_fullsized":"2024-01-24T14:10:59.706003+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724988+0000","last_change":"2024-01-24T14:10:59.717867+0000","last_active":"2024-01-24T14:10:59.724988+0000","last_peered":"2024-01-24T14:10:59.724988+0000","last_clean":"2024-01-24T14:10:59.724988+0000","last_became_active":"2024-01-24T14:10:59.717564+0000","last_became_peered":"2024-01-24T14:10:59.717564+0000","last_unstale":"2024-01-24T14:10:59.724988+0000","last_undegraded":"2024-01-24T14:10:59.724988+0000","last_fullsized":"2024-01-24T14:10:59.724988+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.707825+0000","last_change":"2024-01-24T14:10:59.254286+0000","last_active":"2024-01-24T14:10:59.707825+0000","last_peered":"2024-01-24T14:10:59.707825+0000","last_clean":"2024-01-24T14:10:59.707825+0000","last_became_active":"2024-01-24T14:10:59.253619+0000","last_became_peered":"2024-01-24T14:10:59.253619+0000","last_unstale":"2024-01-24T14:10:59.707825+0000","last_undegraded":"2024-01-24T14:10:59.707825+0000","last_fullsized":"2024-01-24T14:10:59.707825+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732399+0000","last_change":"2024-01-24T14:10:59.729798+0000","last_active":"2024-01-24T14:10:59.732399+0000","last_peered":"2024-01-24T14:10:59.732399+0000","last_clean":"2024-01-24T14:10:59.732399+0000","last_became_active":"2024-01-24T14:10:59.729492+0000","last_became_peered":"2024-01-24T14:10:59.729492+0000","last_unstale":"2024-01-24T14:10:59.732399+0000","last_undegraded":"2024-01-24T14:10:59.732399+0000","last_fullsized":"2024-01-24T14:10:59.732399+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.710093+0000","last_change":"2024-01-24T14:10:59.251175+0000","last_active":"2024-01-24T14:10:59.710093+0000","last_peered":"2024-01-24T14:10:59.710093+0000","last_clean":"2024-01-24T14:10:59.710093+0000","last_became_active":"2024-01-24T14:10:59.250923+0000","last_became_peered":"2024-01-24T14:10:59.250923+0000","last_unstale":"2024-01-24T14:10:59.710093+0000","last_undegraded":"2024-01-24T14:10:59.710093+0000","last_fullsized":"2024-01-24T14:10:59.710093+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725933+0000","last_change":"2024-01-24T14:10:59.719887+0000","last_active":"2024-01-24T14:10:59.725933+0000","last_peered":"2024-01-24T14:10:59.725933+0000","last_clean":"2024-01-24T14:10:59.725933+0000","last_became_active":"2024-01-24T14:10:59.719522+0000","last_became_peered":"2024-01-24T14:10:59.719522+0000","last_unstale":"2024-01-24T14:10:59.725933+0000","last_undegraded":"2024-01-24T14:10:59.725933+0000","last_fullsized":"2024-01-24T14:10:59.725933+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.721066+0000","last_change":"2024-01-24T14:10:59.246566+0000","last_active":"2024-01-24T14:10:59.721066+0000","last_peered":"2024-01-24T14:10:59.721066+0000","last_clean":"2024-01-24T14:10:59.721066+0000","last_became_active":"2024-01-24T14:10:59.246263+0000","last_became_peered":"2024-01-24T14:10:59.246263+0000","last_unstale":"2024-01-24T14:10:59.721066+0000","last_undegraded":"2024-01-24T14:10:59.721066+0000","last_fullsized":"2024-01-24T14:10:59.721066+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732439+0000","last_change":"2024-01-24T14:10:59.715649+0000","last_active":"2024-01-24T14:10:59.732439+0000","last_peered":"2024-01-24T14:10:59.732439+0000","last_clean":"2024-01-24T14:10:59.732439+0000","last_became_active":"2024-01-24T14:10:59.715376+0000","last_became_peered":"2024-01-24T14:10:59.715376+0000","last_unstale":"2024-01-24T14:10:59.732439+0000","last_undegraded":"2024-01-24T14:10:59.732439+0000","last_fullsized":"2024-01-24T14:10:59.732439+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.721107+0000","last_change":"2024-01-24T14:10:59.248963+0000","last_active":"2024-01-24T14:10:59.721107+0000","last_peered":"2024-01-24T14:10:59.721107+0000","last_clean":"2024-01-24T14:10:59.721107+0000","last_became_active":"2024-01-24T14:10:59.248500+0000","last_became_peered":"2024-01-24T14:10:59.248500+0000","last_unstale":"2024-01-24T14:10:59.721107+0000","last_undegraded":"2024-01-24T14:10:59.721107+0000","last_fullsized":"2024-01-24T14:10:59.721107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.735377+0000","last_change":"2024-01-24T14:10:59.734815+0000","last_active":"2024-01-24T14:10:59.735377+0000","last_peered":"2024-01-24T14:10:59.735377+0000","last_clean":"2024-01-24T14:10:59.735377+0000","last_became_active":"2024-01-24T14:10:59.734541+0000","last_became_peered":"2024-01-24T14:10:59.734541+0000","last_unstale":"2024-01-24T14:10:59.735377+0000","last_undegraded":"2024-01-24T14:10:59.735377+0000","last_fullsized":"2024-01-24T14:10:59.735377+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.733025+0000","last_change":"2024-01-24T14:10:59.251973+0000","last_active":"2024-01-24T14:10:59.733025+0000","last_peered":"2024-01-24T14:10:59.733025+0000","last_clean":"2024-01-24T14:10:59.733025+0000","last_became_active":"2024-01-24T14:10:59.251655+0000","last_became_peered":"2024-01-24T14:10:59.251655+0000","last_unstale":"2024-01-24T14:10:59.733025+0000","last_undegraded":"2024-01-24T14:10:59.733025+0000","last_fullsized":"2024-01-24T14:10:59.733025+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725455+0000","last_change":"2024-01-24T14:10:59.719693+0000","last_active":"2024-01-24T14:10:59.725455+0000","last_peered":"2024-01-24T14:10:59.725455+0000","last_clean":"2024-01-24T14:10:59.725455+0000","last_became_active":"2024-01-24T14:10:59.719401+0000","last_became_peered":"2024-01-24T14:10:59.719401+0000","last_unstale":"2024-01-24T14:10:59.725455+0000","last_undegraded":"2024-01-24T14:10:59.725455+0000","last_fullsized":"2024-01-24T14:10:59.725455+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.710964+0000","last_change":"2024-01-24T14:10:59.248372+0000","last_active":"2024-01-24T14:10:59.710964+0000","last_peered":"2024-01-24T14:10:59.710964+0000","last_clean":"2024-01-24T14:10:59.710964+0000","last_became_active":"2024-01-24T14:10:59.248063+0000","last_became_peered":"2024-01-24T14:10:59.248063+0000","last_unstale":"2024-01-24T14:10:59.710964+0000","last_undegraded":"2024-01-24T14:10:59.710964+0000","last_fullsized":"2024-01-24T14:10:59.710964+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.729943+0000","last_change":"2024-01-24T14:10:59.714670+0000","last_active":"2024-01-24T14:10:59.729943+0000","last_peered":"2024-01-24T14:10:59.729943+0000","last_clean":"2024-01-24T14:10:59.729943+0000","last_became_active":"2024-01-24T14:10:59.714126+0000","last_became_peered":"2024-01-24T14:10:59.714126+0000","last_unstale":"2024-01-24T14:10:59.729943+0000","last_undegraded":"2024-01-24T14:10:59.729943+0000","last_fullsized":"2024-01-24T14:10:59.729943+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.702002+0000","last_change":"2024-01-24T14:10:59.254046+0000","last_active":"2024-01-24T14:10:59.702002+0000","last_peered":"2024-01-24T14:10:59.702002+0000","last_clean":"2024-01-24T14:10:59.702002+0000","last_became_active":"2024-01-24T14:10:59.253731+0000","last_became_peered":"2024-01-24T14:10:59.253731+0000","last_unstale":"2024-01-24T14:10:59.702002+0000","last_undegraded":"2024-01-24T14:10:59.702002+0000","last_fullsized":"2024-01-24T14:10:59.702002+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730941+0000","last_change":"2024-01-24T14:10:59.720630+0000","last_active":"2024-01-24T14:10:59.730941+0000","last_peered":"2024-01-24T14:10:59.730941+0000","last_clean":"2024-01-24T14:10:59.730941+0000","last_became_active":"2024-01-24T14:10:59.720357+0000","last_became_peered":"2024-01-24T14:10:59.720357+0000","last_unstale":"2024-01-24T14:10:59.730941+0000","last_undegraded":"2024-01-24T14:10:59.730941+0000","last_fullsized":"2024-01-24T14:10:59.730941+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.706025+0000","last_change":"2024-01-24T14:10:59.251278+0000","last_active":"2024-01-24T14:10:59.706025+0000","last_peered":"2024-01-24T14:10:59.706025+0000","last_clean":"2024-01-24T14:10:59.706025+0000","last_became_active":"2024-01-24T14:10:59.250925+0000","last_became_peered":"2024-01-24T14:10:59.250925+0000","last_unstale":"2024-01-24T14:10:59.706025+0000","last_undegraded":"2024-01-24T14:10:59.706025+0000","last_fullsized":"2024-01-24T14:10:59.706025+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.717321+0000","last_change":"2024-01-24T14:10:59.713943+0000","last_active":"2024-01-24T14:10:59.717321+0000","last_peered":"2024-01-24T14:10:59.717321+0000","last_clean":"2024-01-24T14:10:59.717321+0000","last_became_active":"2024-01-24T14:10:59.713270+0000","last_became_peered":"2024-01-24T14:10:59.713270+0000","last_unstale":"2024-01-24T14:10:59.717321+0000","last_undegraded":"2024-01-24T14:10:59.717321+0000","last_fullsized":"2024-01-24T14:10:59.717321+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711000+0000","last_change":"2024-01-24T14:10:59.254271+0000","last_active":"2024-01-24T14:10:59.711000+0000","last_peered":"2024-01-24T14:10:59.711000+0000","last_clean":"2024-01-24T14:10:59.711000+0000","last_became_active":"2024-01-24T14:10:59.253957+0000","last_became_peered":"2024-01-24T14:10:59.253957+0000","last_unstale":"2024-01-24T14:10:59.711000+0000","last_undegraded":"2024-01-24T14:10:59.711000+0000","last_fullsized":"2024-01-24T14:10:59.711000+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730214+0000","last_change":"2024-01-24T14:10:59.714769+0000","last_active":"2024-01-24T14:10:59.730214+0000","last_peered":"2024-01-24T14:10:59.730214+0000","last_clean":"2024-01-24T14:10:59.730214+0000","last_became_active":"2024-01-24T14:10:59.714307+0000","last_became_peered":"2024-01-24T14:10:59.714307+0000","last_unstale":"2024-01-24T14:10:59.730214+0000","last_undegraded":"2024-01-24T14:10:59.730214+0000","last_fullsized":"2024-01-24T14:10:59.730214+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.702156+0000","last_change":"2024-01-24T14:10:59.252651+0000","last_active":"2024-01-24T14:10:59.702156+0000","last_peered":"2024-01-24T14:10:59.702156+0000","last_clean":"2024-01-24T14:10:59.702156+0000","last_became_active":"2024-01-24T14:10:59.252357+0000","last_became_peered":"2024-01-24T14:10:59.252357+0000","last_unstale":"2024-01-24T14:10:59.702156+0000","last_undegraded":"2024-01-24T14:10:59.702156+0000","last_fullsized":"2024-01-24T14:10:59.702156+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.717662+0000","last_change":"2024-01-24T14:10:59.708895+0000","last_active":"2024-01-24T14:10:59.717662+0000","last_peered":"2024-01-24T14:10:59.717662+0000","last_clean":"2024-01-24T14:10:59.717662+0000","last_became_active":"2024-01-24T14:10:59.708600+0000","last_became_peered":"2024-01-24T14:10:59.708600+0000","last_unstale":"2024-01-24T14:10:59.717662+0000","last_undegraded":"2024-01-24T14:10:59.717662+0000","last_fullsized":"2024-01-24T14:10:59.717662+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.707908+0000","last_change":"2024-01-24T14:10:59.252594+0000","last_active":"2024-01-24T14:10:59.707908+0000","last_peered":"2024-01-24T14:10:59.707908+0000","last_clean":"2024-01-24T14:10:59.707908+0000","last_became_active":"2024-01-24T14:10:59.252323+0000","last_became_peered":"2024-01-24T14:10:59.252323+0000","last_unstale":"2024-01-24T14:10:59.707908+0000","last_undegraded":"2024-01-24T14:10:59.707908+0000","last_fullsized":"2024-01-24T14:10:59.707908+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.735476+0000","last_change":"2024-01-24T14:10:59.733106+0000","last_active":"2024-01-24T14:10:59.735476+0000","last_peered":"2024-01-24T14:10:59.735476+0000","last_clean":"2024-01-24T14:10:59.735476+0000","last_became_active":"2024-01-24T14:10:59.732820+0000","last_became_peered":"2024-01-24T14:10:59.732820+0000","last_unstale":"2024-01-24T14:10:59.735476+0000","last_undegraded":"2024-01-24T14:10:59.735476+0000","last_fullsized":"2024-01-24T14:10:59.735476+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711288+0000","last_change":"2024-01-24T14:10:59.245753+0000","last_active":"2024-01-24T14:10:59.711288+0000","last_peered":"2024-01-24T14:10:59.711288+0000","last_clean":"2024-01-24T14:10:59.711288+0000","last_became_active":"2024-01-24T14:10:59.245398+0000","last_became_peered":"2024-01-24T14:10:59.245398+0000","last_unstale":"2024-01-24T14:10:59.711288+0000","last_undegraded":"2024-01-24T14:10:59.711288+0000","last_fullsized":"2024-01-24T14:10:59.711288+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.734165+0000","last_change":"2024-01-24T14:10:59.723010+0000","last_active":"2024-01-24T14:10:59.734165+0000","last_peered":"2024-01-24T14:10:59.734165+0000","last_clean":"2024-01-24T14:10:59.734165+0000","last_became_active":"2024-01-24T14:10:59.722578+0000","last_became_peered":"2024-01-24T14:10:59.722578+0000","last_unstale":"2024-01-24T14:10:59.734165+0000","last_undegraded":"2024-01-24T14:10:59.734165+0000","last_fullsized":"2024-01-24T14:10:59.734165+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"71'3","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.733932+0000","last_change":"2024-01-24T14:10:59.248002+0000","last_active":"2024-01-24T14:10:59.733932+0000","last_peered":"2024-01-24T14:10:59.733932+0000","last_clean":"2024-01-24T14:10:59.733932+0000","last_became_active":"2024-01-24T14:10:59.247737+0000","last_became_peered":"2024-01-24T14:10:59.247737+0000","last_unstale":"2024-01-24T14:10:59.733932+0000","last_undegraded":"2024-01-24T14:10:59.733932+0000","last_fullsized":"2024-01-24T14:10:59.733932+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.735460+0000","last_change":"2024-01-24T14:10:59.733030+0000","last_active":"2024-01-24T14:10:59.735460+0000","last_peered":"2024-01-24T14:10:59.735460+0000","last_clean":"2024-01-24T14:10:59.735460+0000","last_became_active":"2024-01-24T14:10:59.732752+0000","last_became_peered":"2024-01-24T14:10:59.732752+0000","last_unstale":"2024-01-24T14:10:59.735460+0000","last_undegraded":"2024-01-24T14:10:59.735460+0000","last_fullsized":"2024-01-24T14:10:59.735460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725066+0000","last_change":"2024-01-24T14:10:59.254883+0000","last_active":"2024-01-24T14:10:59.725066+0000","last_peered":"2024-01-24T14:10:59.725066+0000","last_clean":"2024-01-24T14:10:59.725066+0000","last_became_active":"2024-01-24T14:10:59.254539+0000","last_became_peered":"2024-01-24T14:10:59.254539+0000","last_unstale":"2024-01-24T14:10:59.725066+0000","last_undegraded":"2024-01-24T14:10:59.725066+0000","last_fullsized":"2024-01-24T14:10:59.725066+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730891+0000","last_change":"2024-01-24T14:10:59.729755+0000","last_active":"2024-01-24T14:10:59.730891+0000","last_peered":"2024-01-24T14:10:59.730891+0000","last_clean":"2024-01-24T14:10:59.730891+0000","last_became_active":"2024-01-24T14:10:59.729448+0000","last_became_peered":"2024-01-24T14:10:59.729448+0000","last_unstale":"2024-01-24T14:10:59.730891+0000","last_undegraded":"2024-01-24T14:10:59.730891+0000","last_fullsized":"2024-01-24T14:10:59.730891+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731303+0000","last_change":"2024-01-24T14:10:59.245411+0000","last_active":"2024-01-24T14:10:59.731303+0000","last_peered":"2024-01-24T14:10:59.731303+0000","last_clean":"2024-01-24T14:10:59.731303+0000","last_became_active":"2024-01-24T14:10:59.245066+0000","last_became_peered":"2024-01-24T14:10:59.245066+0000","last_unstale":"2024-01-24T14:10:59.731303+0000","last_undegraded":"2024-01-24T14:10:59.731303+0000","last_fullsized":"2024-01-24T14:10:59.731303+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724037+0000","last_change":"2024-01-24T14:10:59.719149+0000","last_active":"2024-01-24T14:10:59.724037+0000","last_peered":"2024-01-24T14:10:59.724037+0000","last_clean":"2024-01-24T14:10:59.724037+0000","last_became_active":"2024-01-24T14:10:59.718909+0000","last_became_peered":"2024-01-24T14:10:59.718909+0000","last_unstale":"2024-01-24T14:10:59.724037+0000","last_undegraded":"2024-01-24T14:10:59.724037+0000","last_fullsized":"2024-01-24T14:10:59.724037+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.701098+0000","last_change":"2024-01-24T14:10:59.245311+0000","last_active":"2024-01-24T14:10:59.701098+0000","last_peered":"2024-01-24T14:10:59.701098+0000","last_clean":"2024-01-24T14:10:59.701098+0000","last_became_active":"2024-01-24T14:10:59.245042+0000","last_became_peered":"2024-01-24T14:10:59.245042+0000","last_unstale":"2024-01-24T14:10:59.701098+0000","last_undegraded":"2024-01-24T14:10:59.701098+0000","last_fullsized":"2024-01-24T14:10:59.701098+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.717900+0000","last_change":"2024-01-24T14:10:59.709940+0000","last_active":"2024-01-24T14:10:59.717900+0000","last_peered":"2024-01-24T14:10:59.717900+0000","last_clean":"2024-01-24T14:10:59.717900+0000","last_became_active":"2024-01-24T14:10:59.709663+0000","last_became_peered":"2024-01-24T14:10:59.709663+0000","last_unstale":"2024-01-24T14:10:59.717900+0000","last_undegraded":"2024-01-24T14:10:59.717900+0000","last_fullsized":"2024-01-24T14:10:59.717900+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711104+0000","last_change":"2024-01-24T14:10:59.250674+0000","last_active":"2024-01-24T14:10:59.711104+0000","last_peered":"2024-01-24T14:10:59.711104+0000","last_clean":"2024-01-24T14:10:59.711104+0000","last_became_active":"2024-01-24T14:10:59.250346+0000","last_became_peered":"2024-01-24T14:10:59.250346+0000","last_unstale":"2024-01-24T14:10:59.711104+0000","last_undegraded":"2024-01-24T14:10:59.711104+0000","last_fullsized":"2024-01-24T14:10:59.711104+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.719753+0000","last_change":"2024-01-24T14:10:59.712829+0000","last_active":"2024-01-24T14:10:59.719753+0000","last_peered":"2024-01-24T14:10:59.719753+0000","last_clean":"2024-01-24T14:10:59.719753+0000","last_became_active":"2024-01-24T14:10:59.712474+0000","last_became_peered":"2024-01-24T14:10:59.712474+0000","last_unstale":"2024-01-24T14:10:59.719753+0000","last_undegraded":"2024-01-24T14:10:59.719753+0000","last_fullsized":"2024-01-24T14:10:59.719753+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"71'4","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.722355+0000","last_change":"2024-01-24T14:10:59.243121+0000","last_active":"2024-01-24T14:10:59.722355+0000","last_peered":"2024-01-24T14:10:59.722355+0000","last_clean":"2024-01-24T14:10:59.722355+0000","last_became_active":"2024-01-24T14:10:59.242850+0000","last_became_peered":"2024-01-24T14:10:59.242850+0000","last_unstale":"2024-01-24T14:10:59.722355+0000","last_undegraded":"2024-01-24T14:10:59.722355+0000","last_fullsized":"2024-01-24T14:10:59.722355+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.721632+0000","last_change":"2024-01-24T14:10:59.714287+0000","last_active":"2024-01-24T14:10:59.721632+0000","last_peered":"2024-01-24T14:10:59.721632+0000","last_clean":"2024-01-24T14:10:59.721632+0000","last_became_active":"2024-01-24T14:10:59.713927+0000","last_became_peered":"2024-01-24T14:10:59.713927+0000","last_unstale":"2024-01-24T14:10:59.721632+0000","last_undegraded":"2024-01-24T14:10:59.721632+0000","last_fullsized":"2024-01-24T14:10:59.721632+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.733286+0000","last_change":"2024-01-24T14:10:59.256094+0000","last_active":"2024-01-24T14:10:59.733286+0000","last_peered":"2024-01-24T14:10:59.733286+0000","last_clean":"2024-01-24T14:10:59.733286+0000","last_became_active":"2024-01-24T14:10:59.254906+0000","last_became_peered":"2024-01-24T14:10:59.254906+0000","last_unstale":"2024-01-24T14:10:59.733286+0000","last_undegraded":"2024-01-24T14:10:59.733286+0000","last_fullsized":"2024-01-24T14:10:59.733286+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.723005+0000","last_change":"2024-01-24T14:10:59.718313+0000","last_active":"2024-01-24T14:10:59.723005+0000","last_peered":"2024-01-24T14:10:59.723005+0000","last_clean":"2024-01-24T14:10:59.723005+0000","last_became_active":"2024-01-24T14:10:59.718034+0000","last_became_peered":"2024-01-24T14:10:59.718034+0000","last_unstale":"2024-01-24T14:10:59.723005+0000","last_undegraded":"2024-01-24T14:10:59.723005+0000","last_fullsized":"2024-01-24T14:10:59.723005+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.706702+0000","last_change":"2024-01-24T14:10:59.249010+0000","last_active":"2024-01-24T14:10:59.706702+0000","last_peered":"2024-01-24T14:10:59.706702+0000","last_clean":"2024-01-24T14:10:59.706702+0000","last_became_active":"2024-01-24T14:10:59.248751+0000","last_became_peered":"2024-01-24T14:10:59.248751+0000","last_unstale":"2024-01-24T14:10:59.706702+0000","last_undegraded":"2024-01-24T14:10:59.706702+0000","last_fullsized":"2024-01-24T14:10:59.706702+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.734135+0000","last_change":"2024-01-24T14:10:59.722799+0000","last_active":"2024-01-24T14:10:59.734135+0000","last_peered":"2024-01-24T14:10:59.734135+0000","last_clean":"2024-01-24T14:10:59.734135+0000","last_became_active":"2024-01-24T14:10:59.722182+0000","last_became_peered":"2024-01-24T14:10:59.722182+0000","last_unstale":"2024-01-24T14:10:59.734135+0000","last_undegraded":"2024-01-24T14:10:59.734135+0000","last_fullsized":"2024-01-24T14:10:59.734135+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.701416+0000","last_change":"2024-01-24T14:10:59.254892+0000","last_active":"2024-01-24T14:10:59.701416+0000","last_peered":"2024-01-24T14:10:59.701416+0000","last_clean":"2024-01-24T14:10:59.701416+0000","last_became_active":"2024-01-24T14:10:59.254606+0000","last_became_peered":"2024-01-24T14:10:59.254606+0000","last_unstale":"2024-01-24T14:10:59.701416+0000","last_undegraded":"2024-01-24T14:10:59.701416+0000","last_fullsized":"2024-01-24T14:10:59.701416+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725429+0000","last_change":"2024-01-24T14:10:59.718937+0000","last_active":"2024-01-24T14:10:59.725429+0000","last_peered":"2024-01-24T14:10:59.725429+0000","last_clean":"2024-01-24T14:10:59.725429+0000","last_became_active":"2024-01-24T14:10:59.718337+0000","last_became_peered":"2024-01-24T14:10:59.718337+0000","last_unstale":"2024-01-24T14:10:59.725429+0000","last_undegraded":"2024-01-24T14:10:59.725429+0000","last_fullsized":"2024-01-24T14:10:59.725429+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711233+0000","last_change":"2024-01-24T14:10:59.256851+0000","last_active":"2024-01-24T14:10:59.711233+0000","last_peered":"2024-01-24T14:10:59.711233+0000","last_clean":"2024-01-24T14:10:59.711233+0000","last_became_active":"2024-01-24T14:10:59.256551+0000","last_became_peered":"2024-01-24T14:10:59.256551+0000","last_unstale":"2024-01-24T14:10:59.711233+0000","last_undegraded":"2024-01-24T14:10:59.711233+0000","last_fullsized":"2024-01-24T14:10:59.711233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731290+0000","last_change":"2024-01-24T14:10:59.726296+0000","last_active":"2024-01-24T14:10:59.731290+0000","last_peered":"2024-01-24T14:10:59.731290+0000","last_clean":"2024-01-24T14:10:59.731290+0000","last_became_active":"2024-01-24T14:10:59.725859+0000","last_became_peered":"2024-01-24T14:10:59.725859+0000","last_unstale":"2024-01-24T14:10:59.731290+0000","last_undegraded":"2024-01-24T14:10:59.731290+0000","last_fullsized":"2024-01-24T14:10:59.731290+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.708589+0000","last_change":"2024-01-24T14:10:59.247220+0000","last_active":"2024-01-24T14:10:59.708589+0000","last_peered":"2024-01-24T14:10:59.708589+0000","last_clean":"2024-01-24T14:10:59.708589+0000","last_became_active":"2024-01-24T14:10:59.246553+0000","last_became_peered":"2024-01-24T14:10:59.246553+0000","last_unstale":"2024-01-24T14:10:59.708589+0000","last_undegraded":"2024-01-24T14:10:59.708589+0000","last_fullsized":"2024-01-24T14:10:59.708589+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.717332+0000","last_change":"2024-01-24T14:10:59.711443+0000","last_active":"2024-01-24T14:10:59.717332+0000","last_peered":"2024-01-24T14:10:59.717332+0000","last_clean":"2024-01-24T14:10:59.717332+0000","last_became_active":"2024-01-24T14:10:59.711120+0000","last_became_peered":"2024-01-24T14:10:59.711120+0000","last_unstale":"2024-01-24T14:10:59.717332+0000","last_undegraded":"2024-01-24T14:10:59.717332+0000","last_fullsized":"2024-01-24T14:10:59.717332+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.701868+0000","last_change":"2024-01-24T14:10:59.254697+0000","last_active":"2024-01-24T14:10:59.701868+0000","last_peered":"2024-01-24T14:10:59.701868+0000","last_clean":"2024-01-24T14:10:59.701868+0000","last_became_active":"2024-01-24T14:10:59.254415+0000","last_became_peered":"2024-01-24T14:10:59.254415+0000","last_unstale":"2024-01-24T14:10:59.701868+0000","last_undegraded":"2024-01-24T14:10:59.701868+0000","last_fullsized":"2024-01-24T14:10:59.701868+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.733864+0000","last_change":"2024-01-24T14:10:59.728552+0000","last_active":"2024-01-24T14:10:59.733864+0000","last_peered":"2024-01-24T14:10:59.733864+0000","last_clean":"2024-01-24T14:10:59.733864+0000","last_became_active":"2024-01-24T14:10:59.728066+0000","last_became_peered":"2024-01-24T14:10:59.728066+0000","last_unstale":"2024-01-24T14:10:59.733864+0000","last_undegraded":"2024-01-24T14:10:59.733864+0000","last_fullsized":"2024-01-24T14:10:59.733864+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.709279+0000","last_change":"2024-01-24T14:10:59.256071+0000","last_active":"2024-01-24T14:10:59.709279+0000","last_peered":"2024-01-24T14:10:59.709279+0000","last_clean":"2024-01-24T14:10:59.709279+0000","last_became_active":"2024-01-24T14:10:59.255745+0000","last_became_peered":"2024-01-24T14:10:59.255745+0000","last_unstale":"2024-01-24T14:10:59.709279+0000","last_undegraded":"2024-01-24T14:10:59.709279+0000","last_fullsized":"2024-01-24T14:10:59.709279+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730193+0000","last_change":"2024-01-24T14:10:59.714593+0000","last_active":"2024-01-24T14:10:59.730193+0000","last_peered":"2024-01-24T14:10:59.730193+0000","last_clean":"2024-01-24T14:10:59.730193+0000","last_became_active":"2024-01-24T14:10:59.713932+0000","last_became_peered":"2024-01-24T14:10:59.713932+0000","last_unstale":"2024-01-24T14:10:59.730193+0000","last_undegraded":"2024-01-24T14:10:59.730193+0000","last_fullsized":"2024-01-24T14:10:59.730193+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.706976+0000","last_change":"2024-01-24T14:10:59.241277+0000","last_active":"2024-01-24T14:10:59.706976+0000","last_peered":"2024-01-24T14:10:59.706976+0000","last_clean":"2024-01-24T14:10:59.706976+0000","last_became_active":"2024-01-24T14:10:59.240964+0000","last_became_peered":"2024-01-24T14:10:59.240964+0000","last_unstale":"2024-01-24T14:10:59.706976+0000","last_undegraded":"2024-01-24T14:10:59.706976+0000","last_fullsized":"2024-01-24T14:10:59.706976+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.732472+0000","last_change":"2024-01-24T14:10:59.731171+0000","last_active":"2024-01-24T14:10:59.732472+0000","last_peered":"2024-01-24T14:10:59.732472+0000","last_clean":"2024-01-24T14:10:59.732472+0000","last_became_active":"2024-01-24T14:10:59.730879+0000","last_became_peered":"2024-01-24T14:10:59.730879+0000","last_unstale":"2024-01-24T14:10:59.732472+0000","last_undegraded":"2024-01-24T14:10:59.732472+0000","last_fullsized":"2024-01-24T14:10:59.732472+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.719333+0000","last_change":"2024-01-24T14:10:59.254672+0000","last_active":"2024-01-24T14:10:59.719333+0000","last_peered":"2024-01-24T14:10:59.719333+0000","last_clean":"2024-01-24T14:10:59.719333+0000","last_became_active":"2024-01-24T14:10:59.254321+0000","last_became_peered":"2024-01-24T14:10:59.254321+0000","last_unstale":"2024-01-24T14:10:59.719333+0000","last_undegraded":"2024-01-24T14:10:59.719333+0000","last_fullsized":"2024-01-24T14:10:59.719333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.704485+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:10:59.704485+0000","last_peered":"2024-01-24T14:10:59.704485+0000","last_clean":"2024-01-24T14:10:59.704485+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:10:59.704485+0000","last_undegraded":"2024-01-24T14:10:59.704485+0000","last_fullsized":"2024-01-24T14:10:59.704485+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.725066+0000","last_change":"2024-01-24T14:10:59.717924+0000","last_active":"2024-01-24T14:10:59.725066+0000","last_peered":"2024-01-24T14:10:59.725066+0000","last_clean":"2024-01-24T14:10:59.725066+0000","last_became_active":"2024-01-24T14:10:59.717728+0000","last_became_peered":"2024-01-24T14:10:59.717728+0000","last_unstale":"2024-01-24T14:10:59.725066+0000","last_undegraded":"2024-01-24T14:10:59.725066+0000","last_fullsized":"2024-01-24T14:10:59.725066+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.710214+0000","last_change":"2024-01-24T14:10:59.251159+0000","last_active":"2024-01-24T14:10:59.710214+0000","last_peered":"2024-01-24T14:10:59.710214+0000","last_clean":"2024-01-24T14:10:59.710214+0000","last_became_active":"2024-01-24T14:10:59.250861+0000","last_became_peered":"2024-01-24T14:10:59.250861+0000","last_unstale":"2024-01-24T14:10:59.710214+0000","last_undegraded":"2024-01-24T14:10:59.710214+0000","last_fullsized":"2024-01-24T14:10:59.710214+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":62,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.710658+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:10:59.710658+0000","last_peered":"2024-01-24T14:10:59.710658+0000","last_clean":"2024-01-24T14:10:59.710658+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:10:59.710658+0000","last_undegraded":"2024-01-24T14:10:59.710658+0000","last_fullsized":"2024-01-24T14:10:59.710658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.734716+0000","last_change":"2024-01-24T14:10:59.733631+0000","last_active":"2024-01-24T14:10:59.734716+0000","last_peered":"2024-01-24T14:10:59.734716+0000","last_clean":"2024-01-24T14:10:59.734716+0000","last_became_active":"2024-01-24T14:10:59.733408+0000","last_became_peered":"2024-01-24T14:10:59.733408+0000","last_unstale":"2024-01-24T14:10:59.734716+0000","last_undegraded":"2024-01-24T14:10:59.734716+0000","last_fullsized":"2024-01-24T14:10:59.734716+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730885+0000","last_change":"2024-01-24T14:10:59.258912+0000","last_active":"2024-01-24T14:10:59.730885+0000","last_peered":"2024-01-24T14:10:59.730885+0000","last_clean":"2024-01-24T14:10:59.730885+0000","last_became_active":"2024-01-24T14:10:59.258572+0000","last_became_peered":"2024-01-24T14:10:59.258572+0000","last_unstale":"2024-01-24T14:10:59.730885+0000","last_undegraded":"2024-01-24T14:10:59.730885+0000","last_fullsized":"2024-01-24T14:10:59.730885+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.698330+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:10:59.698330+0000","last_peered":"2024-01-24T14:10:59.698330+0000","last_clean":"2024-01-24T14:10:59.698330+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:10:59.698330+0000","last_undegraded":"2024-01-24T14:10:59.698330+0000","last_fullsized":"2024-01-24T14:10:59.698330+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731320+0000","last_change":"2024-01-24T14:10:59.720032+0000","last_active":"2024-01-24T14:10:59.731320+0000","last_peered":"2024-01-24T14:10:59.731320+0000","last_clean":"2024-01-24T14:10:59.731320+0000","last_became_active":"2024-01-24T14:10:59.719535+0000","last_became_peered":"2024-01-24T14:10:59.719535+0000","last_unstale":"2024-01-24T14:10:59.731320+0000","last_undegraded":"2024-01-24T14:10:59.731320+0000","last_fullsized":"2024-01-24T14:10:59.731320+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.700816+0000","last_change":"2024-01-24T14:10:59.243012+0000","last_active":"2024-01-24T14:10:59.700816+0000","last_peered":"2024-01-24T14:10:59.700816+0000","last_clean":"2024-01-24T14:10:59.700816+0000","last_became_active":"2024-01-24T14:10:59.242722+0000","last_became_peered":"2024-01-24T14:10:59.242722+0000","last_unstale":"2024-01-24T14:10:59.700816+0000","last_undegraded":"2024-01-24T14:10:59.700816+0000","last_fullsized":"2024-01-24T14:10:59.700816+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":62,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.711759+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:10:59.711759+0000","last_peered":"2024-01-24T14:10:59.711759+0000","last_clean":"2024-01-24T14:10:59.711759+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:10:59.711759+0000","last_undegraded":"2024-01-24T14:10:59.711759+0000","last_fullsized":"2024-01-24T14:10:59.711759+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":61,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705343+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:10:59.705343+0000","last_peered":"2024-01-24T14:10:59.705343+0000","last_clean":"2024-01-24T14:10:59.705343+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:10:59.705343+0000","last_undegraded":"2024-01-24T14:10:59.705343+0000","last_fullsized":"2024-01-24T14:10:59.705343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.727396+0000","last_change":"2024-01-24T14:10:59.721991+0000","last_active":"2024-01-24T14:10:59.727396+0000","last_peered":"2024-01-24T14:10:59.727396+0000","last_clean":"2024-01-24T14:10:59.727396+0000","last_became_active":"2024-01-24T14:10:59.721672+0000","last_became_peered":"2024-01-24T14:10:59.721672+0000","last_unstale":"2024-01-24T14:10:59.727396+0000","last_undegraded":"2024-01-24T14:10:59.727396+0000","last_fullsized":"2024-01-24T14:10:59.727396+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705188+0000","last_change":"2024-01-24T14:10:59.241490+0000","last_active":"2024-01-24T14:10:59.705188+0000","last_peered":"2024-01-24T14:10:59.705188+0000","last_clean":"2024-01-24T14:10:59.705188+0000","last_became_active":"2024-01-24T14:10:59.241195+0000","last_became_peered":"2024-01-24T14:10:59.241195+0000","last_unstale":"2024-01-24T14:10:59.705188+0000","last_undegraded":"2024-01-24T14:10:59.705188+0000","last_fullsized":"2024-01-24T14:10:59.705188+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705253+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:10:59.705253+0000","last_peered":"2024-01-24T14:10:59.705253+0000","last_clean":"2024-01-24T14:10:59.705253+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:10:59.705253+0000","last_undegraded":"2024-01-24T14:10:59.705253+0000","last_fullsized":"2024-01-24T14:10:59.705253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.724127+0000","last_change":"2024-01-24T14:10:59.720486+0000","last_active":"2024-01-24T14:10:59.724127+0000","last_peered":"2024-01-24T14:10:59.724127+0000","last_clean":"2024-01-24T14:10:59.724127+0000","last_became_active":"2024-01-24T14:10:59.720206+0000","last_became_peered":"2024-01-24T14:10:59.720206+0000","last_unstale":"2024-01-24T14:10:59.724127+0000","last_undegraded":"2024-01-24T14:10:59.724127+0000","last_fullsized":"2024-01-24T14:10:59.724127+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.731287+0000","last_change":"2024-01-24T14:10:59.238503+0000","last_active":"2024-01-24T14:10:59.731287+0000","last_peered":"2024-01-24T14:10:59.731287+0000","last_clean":"2024-01-24T14:10:59.731287+0000","last_became_active":"2024-01-24T14:10:59.238177+0000","last_became_peered":"2024-01-24T14:10:59.238177+0000","last_unstale":"2024-01-24T14:10:59.731287+0000","last_undegraded":"2024-01-24T14:10:59.731287+0000","last_fullsized":"2024-01-24T14:10:59.731287+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.707334+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:10:59.707334+0000","last_peered":"2024-01-24T14:10:59.707334+0000","last_clean":"2024-01-24T14:10:59.707334+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:10:59.707334+0000","last_undegraded":"2024-01-24T14:10:59.707334+0000","last_fullsized":"2024-01-24T14:10:59.707334+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.735478+0000","last_change":"2024-01-24T14:10:59.733001+0000","last_active":"2024-01-24T14:10:59.735478+0000","last_peered":"2024-01-24T14:10:59.735478+0000","last_clean":"2024-01-24T14:10:59.735478+0000","last_became_active":"2024-01-24T14:10:59.732735+0000","last_became_peered":"2024-01-24T14:10:59.732735+0000","last_unstale":"2024-01-24T14:10:59.735478+0000","last_undegraded":"2024-01-24T14:10:59.735478+0000","last_fullsized":"2024-01-24T14:10:59.735478+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.708408+0000","last_change":"2024-01-24T14:10:59.261830+0000","last_active":"2024-01-24T14:10:59.708408+0000","last_peered":"2024-01-24T14:10:59.708408+0000","last_clean":"2024-01-24T14:10:59.708408+0000","last_became_active":"2024-01-24T14:10:59.261502+0000","last_became_peered":"2024-01-24T14:10:59.261502+0000","last_unstale":"2024-01-24T14:10:59.708408+0000","last_undegraded":"2024-01-24T14:10:59.708408+0000","last_fullsized":"2024-01-24T14:10:59.708408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.705065+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:10:59.705065+0000","last_peered":"2024-01-24T14:10:59.705065+0000","last_clean":"2024-01-24T14:10:59.705065+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:10:59.705065+0000","last_undegraded":"2024-01-24T14:10:59.705065+0000","last_fullsized":"2024-01-24T14:10:59.705065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.730971+0000","last_change":"2024-01-24T14:10:59.712655+0000","last_active":"2024-01-24T14:10:59.730971+0000","last_peered":"2024-01-24T14:10:59.730971+0000","last_clean":"2024-01-24T14:10:59.730971+0000","last_became_active":"2024-01-24T14:10:59.712302+0000","last_became_peered":"2024-01-24T14:10:59.712302+0000","last_unstale":"2024-01-24T14:10:59.730971+0000","last_undegraded":"2024-01-24T14:10:59.730971+0000","last_fullsized":"2024-01-24T14:10:59.730971+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:59.218022+0000","last_clean_scrub_stamp":"2024-01-24T14:10:59.218022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.710822+0000","last_change":"2024-01-24T14:10:59.248629+0000","last_active":"2024-01-24T14:10:59.710822+0000","last_peered":"2024-01-24T14:10:59.710822+0000","last_clean":"2024-01-24T14:10:59.710822+0000","last_became_active":"2024-01-24T14:10:59.248302+0000","last_became_peered":"2024-01-24T14:10:59.248302+0000","last_unstale":"2024-01-24T14:10:59.710822+0000","last_undegraded":"2024-01-24T14:10:59.710822+0000","last_fullsized":"2024-01-24T14:10:59.710822+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:10:58.211911+0000","last_clean_scrub_stamp":"2024-01-24T14:10:58.211911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-24T14:10:59.701001+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:10:59.701001+0000","last_peered":"2024-01-24T14:10:59.701001+0000","last_clean":"2024-01-24T14:10:59.701001+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:10:59.701001+0000","last_undegraded":"2024-01-24T14:10:59.701001+0000","last_fullsized":"2024-01-24T14:10:59.701001+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297944,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073896,"statfs":{"total":96636764160,"available":96331669504,"internally_reserved":0,"allocated":868352,"data_stored":459179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57899999999999996,"15min":0.54100000000000004},"min":{"1min":0.42899999999999999,"5min":0.34899999999999998,"15min":0.26000000000000001},"max":{"1min":0.63400000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.41499999999999998,"15min":0.32700000000000001},"max":{"1min":0.75700000000000001,"5min":0.82599999999999996,"15min":0.83499999999999996},"last":0.76100000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61099999999999999,"15min":0.57599999999999996},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.45800000000000002,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.79100000000000004,"15min":3.2570000000000001},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.58799999999999997},"min":{"1min":0.54700000000000004,"5min":0.42499999999999999,"15min":0.32000000000000001},"max":{"1min":0.69399999999999995,"5min":0.82499999999999996,"15min":0.876},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.58999999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.36799999999999999},"max":{"1min":0.79600000000000004,"5min":0.80900000000000005,"15min":0.83099999999999996},"last":0.77100000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61099999999999999,"15min":0.58799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.36499999999999999},"max":{"1min":0.73699999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.35499999999999998},"max":{"1min":0.71799999999999997,"5min":0.77500000000000002,"15min":3.0049999999999999},"last":0.751}]},{"osd":4,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.48399999999999999},"min":{"1min":0.35399999999999998,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":0.68799999999999994,"5min":0.755,"15min":0.84699999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.496},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":0.71499999999999997,"15min":0.83599999999999997},"last":0.55600000000000005}]},{"osd":5,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.497,"15min":0.496},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.29999999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.84899999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41299999999999998,"5min":0.376,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":0.78600000000000003,"15min":0.80900000000000005},"last":0.64400000000000002}]},{"osd":6,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.34899999999999998,"15min":0.33000000000000002},"max":{"1min":0.67700000000000005,"5min":0.72299999999999998,"15min":0.81999999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.58699999999999997},"min":{"1min":0.42399999999999999,"5min":0.377,"15min":0.34999999999999998},"max":{"1min":0.73599999999999999,"5min":0.83399999999999996,"15min":0.87},"last":0.70699999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302040,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069800,"statfs":{"total":96636764160,"available":96327475200,"internally_reserved":0,"allocated":868352,"data_stored":459981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82599999999999996,"5min":0.85699999999999998,"15min":0.86899999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.29999999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":1.3859999999999999},"last":0.60599999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.433,"5min":0.39300000000000002,"15min":0.35399999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":1.353},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58999999999999997,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.497}]},{"osd":2,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61199999999999999,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.34300000000000003},"max":{"1min":0.78300000000000003,"5min":0.83499999999999996,"15min":6.8129999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.44600000000000001,"5min":0.442,"15min":0.40000000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":1.4079999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.629,"15min":0.66200000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":7.016},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.38900000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.496,"15min":0.502},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.34399999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":6.8719999999999999},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54000000000000004,"15min":0.57299999999999995},"min":{"1min":0.40699999999999997,"5min":0.34699999999999998,"15min":0.32400000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":6.976},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52800000000000002,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.35899999999999999,"15min":0.33600000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":7.8789999999999996},"last":0.64800000000000002}]},{"osd":7,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.63200000000000001},"min":{"1min":0.44500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":6.9080000000000004},"last":0.70299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302044,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069796,"statfs":{"total":96636764160,"available":96327471104,"internally_reserved":0,"allocated":872448,"data_stored":459733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.27500000000000002,"15min":0.23400000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.311},{"interface":"front","average":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.315,"5min":0.254,"15min":0.254},"max":{"1min":0.57599999999999996,"5min":0.62,"15min":0.65600000000000003},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45500000000000002,"15min":0.443},"min":{"1min":0.34999999999999998,"5min":0.28000000000000003,"15min":0.26000000000000001},"max":{"1min":0.60499999999999998,"5min":0.64900000000000002,"15min":0.70799999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45100000000000001,"15min":0.437},"min":{"1min":0.35799999999999998,"5min":0.28199999999999997,"15min":0.25800000000000001},"max":{"1min":0.67900000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34000000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45900000000000002,"15min":0.44600000000000001},"min":{"1min":0.37,"5min":0.32900000000000001,"15min":0.309},"max":{"1min":0.64800000000000002,"5min":0.67700000000000005,"15min":0.71899999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.46500000000000002},"min":{"1min":0.40500000000000003,"5min":0.34799999999999998,"15min":0.33400000000000002},"max":{"1min":0.68200000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.50600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.91600000000000004,"5min":1.073,"15min":1.137},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.51200000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":1.218,"15min":1.218},"last":0.60799999999999998}]},{"osd":5,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":2.698,"5min":1.101,"15min":0.82199999999999995},"min":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.443},"max":{"1min":38.808,"5min":38.808,"15min":38.808},"last":0.79600000000000004},{"interface":"front","average":{"1min":2.6840000000000002,"5min":1.0760000000000001,"15min":0.80500000000000005},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":38.823,"5min":38.823,"15min":38.823},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.67300000000000004},"min":{"1min":0.52500000000000002,"5min":0.47199999999999998,"15min":0.43099999999999999},"max":{"1min":0.90700000000000003,"5min":1.0449999999999999,"15min":1.0509999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.47999999999999998,"15min":0.40000000000000002},"max":{"1min":0.95499999999999996,"5min":1.1559999999999999,"15min":1.1879999999999999},"last":0.76800000000000002}]},{"osd":7,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.45300000000000001},"max":{"1min":0.86199999999999999,"5min":1.105,"15min":1.155},"last":0.873},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.753,"15min":0.747},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":0.92400000000000004,"5min":1.123,"15min":1.131},"last":0.85099999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542301,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302044,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069796,"statfs":{"total":96636764160,"available":96327471104,"internally_reserved":0,"allocated":872448,"data_stored":458842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40300000000000002,"15min":0.40100000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.43099999999999999,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.66700000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.318,"15min":0.28799999999999998},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.79400000000000004},"last":0.32500000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.432,"15min":0.42299999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59299999999999997,"5min":0.63700000000000001,"15min":0.70099999999999996},"last":0.372},{"interface":"front","average":{"1min":0.503,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.34799999999999998,"15min":0.32100000000000001},"max":{"1min":0.71399999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82599999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64500000000000002,"15min":0.65000000000000002},"min":{"1min":0.439,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":1.0149999999999999,"15min":1.0429999999999999},"last":0.46999999999999997}]},{"osd":5,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.44900000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.99099999999999999},"last":0.439},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68799999999999994,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85899999999999999,"5min":0.98599999999999999,"15min":1.0840000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.498,"5min":0.48199999999999998,"15min":0.47499999999999998},"max":{"1min":0.90700000000000003,"5min":1.0309999999999999,"15min":1.137},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.51200000000000001,"5min":0.48799999999999999,"15min":0.48199999999999998},"max":{"1min":0.93100000000000005,"5min":0.95899999999999996,"15min":1.117},"last":0.52700000000000002}]},{"osd":7,"last update":"Wed Jan 24 14:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.47099999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.98199999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69299999999999995,"15min":0.68500000000000005},"min":{"1min":0.52000000000000002,"5min":0.503,"15min":0.48299999999999998},"max":{"1min":0.88300000000000001,"5min":0.96699999999999997,"15min":1.1799999999999999},"last":0.50600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575003,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306140,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94065700,"statfs":{"total":96636764160,"available":96323276800,"internally_reserved":0,"allocated":872448,"data_stored":459145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39400000000000002,"15min":0.38400000000000001},"min":{"1min":0.29399999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46700000000000003,"5min":0.64400000000000002,"15min":0.65800000000000003},"last":0.495},{"interface":"front","average":{"1min":0.436,"5min":0.43099999999999999,"15min":0.42199999999999999},"min":{"1min":0.32500000000000001,"5min":0.26300000000000001,"15min":0.24399999999999999},"max":{"1min":0.63600000000000001,"5min":0.65900000000000003,"15min":0.67100000000000004},"last":0.32000000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.39200000000000002},"min":{"1min":0.311,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.55600000000000005,"5min":0.61499999999999999,"15min":0.67200000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46000000000000002,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.29699999999999999,"15min":0.255},"max":{"1min":0.55800000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"last":0.41599999999999998}]},{"osd":3,"last update":"Wed Jan 24 14:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.438},"min":{"1min":0.309,"5min":0.309,"15min":0.30099999999999999},"max":{"1min":0.52700000000000002,"5min":0.68500000000000005,"15min":0.71699999999999997},"last":0.436},{"interface":"front","average":{"1min":0.44,"5min":0.46800000000000003,"15min":0.45900000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.57799999999999996,"5min":0.66200000000000003,"15min":0.72799999999999998},"last":0.45200000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65600000000000003,"15min":0.63300000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.41299999999999998},"max":{"1min":0.79500000000000004,"5min":0.96499999999999997,"15min":0.96699999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.46200000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.98599999999999999,"5min":1.008,"15min":1.042},"last":0.54100000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69099999999999995,"15min":0.66600000000000004},"min":{"1min":0.501,"5min":0.44800000000000001,"15min":0.443},"max":{"1min":0.94899999999999995,"5min":0.98699999999999999,"15min":1.0469999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.67300000000000004,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.877,"5min":0.97099999999999997,"15min":1.026},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Jan 24 14:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70699999999999996,"15min":0.68000000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92000000000000004,"5min":1.016,"15min":1.1100000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71999999999999997,"15min":0.70399999999999996},"min":{"1min":0.504,"5min":0.497,"15min":0.47799999999999998},"max":{"1min":0.94099999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57399999999999995}]},{"osd":7,"last update":"Wed Jan 24 14:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72899999999999998,"15min":0.70099999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91200000000000003,"5min":0.97299999999999998,"15min":1.054},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.70899999999999996},"min":{"1min":0.52200000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.89900000000000002,"5min":1.036,"15min":1.036},"last":0.621}]}]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302048,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069792,"statfs":{"total":96636764160,"available":96327467008,"internally_reserved":0,"allocated":876544,"data_stored":460037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39800000000000002,"15min":0.39100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41099999999999998,"15min":0.40999999999999998},"min":{"1min":0.312,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.57499999999999996,"5min":0.70299999999999996,"15min":0.71299999999999997},"last":0.38100000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.69099999999999995},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.45400000000000001,"15min":0.45200000000000001},"min":{"1min":0.312,"5min":0.311,"15min":0.28299999999999997},"max":{"1min":0.61499999999999999,"5min":0.66300000000000003,"15min":0.68799999999999994},"last":0.42599999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.498,"15min":0.495},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.60599999999999998,"5min":0.68799999999999994,"15min":0.77200000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.318},"max":{"1min":0.625,"5min":0.72999999999999998,"15min":0.78800000000000003},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64500000000000002,"15min":0.63300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.41799999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.624},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67400000000000004,"15min":0.65500000000000003},"min":{"1min":0.55200000000000005,"5min":0.44400000000000001,"15min":0.40200000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.93500000000000005,"5min":0.97599999999999998,"15min":0.98999999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.70199999999999996},"min":{"1min":0.53400000000000003,"5min":0.51200000000000001,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.98099999999999998,"15min":1.0369999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.45400000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.018},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.53100000000000003,"5min":0.48799999999999999,"15min":0.437},"max":{"1min":0.874,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63500000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72999999999999998,"15min":0.71299999999999997},"min":{"1min":0.52700000000000002,"5min":0.5,"15min":0.45200000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0740000000000001},"last":0.755},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.57899999999999996,"5min":0.502,"15min":0.45800000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302048,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069792,"statfs":{"total":96636764160,"available":96327467008,"internally_reserved":0,"allocated":876544,"data_stored":459803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.88600000000000001},"last":0.498},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59799999999999998,"15min":0.57899999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.38200000000000001},"max":{"1min":0.755,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48699999999999999}]},{"osd":1,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.79000000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79400000000000004,"5min":0.82599999999999996,"15min":1.506},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58899999999999997,"15min":0.56999999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.36899999999999999},"max":{"1min":0.78000000000000003,"5min":0.81999999999999995,"15min":0.83999999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61399999999999999,"15min":0.60099999999999998},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54100000000000004}]},{"osd":3,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.77800000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.36499999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.86299999999999999},"last":0.42999999999999999}]},{"osd":5,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46700000000000003,"15min":0.497},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68600000000000005,"5min":0.75800000000000001,"15min":0.92000000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48499999999999999,"15min":0.52400000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65200000000000002,"5min":0.72599999999999998,"15min":1.556},"last":0.60799999999999998}]},{"osd":6,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46999999999999997,"15min":0.497},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.86099999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53000000000000003,"15min":0.55000000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":1.0409999999999999},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Jan 24 14:10:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.54200000000000004},"min":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.33500000000000002},"max":{"1min":0.73199999999999998,"5min":0.76900000000000002,"15min":0.88100000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.57199999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":1.26},"last":0.60199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302040,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069800,"statfs":{"total":96636764160,"available":96327475200,"internally_reserved":0,"allocated":868352,"data_stored":459643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35699999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":1.101},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":1.1439999999999999},"last":0.65200000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.58099999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.093},"last":0.751},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":1.1639999999999999},"last":0.56200000000000006}]},{"osd":2,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.33500000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":1.1100000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":1.1220000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38500000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":1.139},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.62,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":0.82599999999999996,"15min":1.0609999999999999},"last":0.83099999999999996}]},{"osd":4,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48699999999999999,"15min":0.48999999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72199999999999998,"5min":0.72599999999999998,"15min":0.82499999999999996},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.49299999999999999,"15min":0.501},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72799999999999998,"5min":0.81100000000000005,"15min":0.90000000000000002},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.49199999999999999,"15min":0.50800000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.81100000000000005,"15min":1.1839999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.746,"5min":0.78900000000000003,"15min":0.872},"last":0.92800000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71999999999999997,"5min":0.79000000000000004,"15min":0.93700000000000006},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.58199999999999996},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81499999999999995,"5min":0.84099999999999997,"15min":1.2030000000000001},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:11:04.295 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-24T14:11:04.295 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-24T14:11:04.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.469+0000 7f8531a90700 1 Processor -- start 2024-01-24T14:11:04.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.469+0000 7f8531a90700 1 -- start start 2024-01-24T14:11:04.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f8531a90700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f852c066720 0x7f852c066b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f8531a90700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f852c13ba60 0x7f852c156500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f8531a90700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c0c6be0 con 0x7f852c0c4ee0 2024-01-24T14:11:04.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f8531a90700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852c0c5360 con 0x7f852c13ba60 2024-01-24T14:11:04.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f8531a90700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852c0c33e0 con 0x7f852c066720 2024-01-24T14:11:04.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f852bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f852c13ba60 0x7f852c156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f8522ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f852c0c4ee0 0x7f852c13b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42038/0 (socket says 172.21.15.29:42038) 2024-01-24T14:11:04.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f8522ffd700 1 -- 172.21.15.29:0/3307143410 learned_addr learned my addr 172.21.15.29:0/3307143410 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f852b7fe700 1 --2- 172.21.15.29:0/3307143410 >> v2:172.21.15.163:3300/0 conn(0x7f852c066720 0x7f852c066b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f852bfff700 1 -- 172.21.15.29:0/3307143410 >> v2:172.21.15.163:3300/0 conn(0x7f852c066720 msgr2=0x7f852c066b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f852bfff700 1 --2- 172.21.15.29:0/3307143410 >> v2:172.21.15.163:3300/0 conn(0x7f852c066720 0x7f852c066b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f852bfff700 1 -- 172.21.15.29:0/3307143410 >> v1:172.21.15.29:6789/0 conn(0x7f852c0c4ee0 legacy=0x7f852c13b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f852affd700 1 -- 172.21.15.29:0/3307143410 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048141290 0 0) 0x7f852c0c6be0 con 0x7f852c0c4ee0 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f852bfff700 1 -- 172.21.15.29:0/3307143410 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0c68b0 con 0x7f852c13ba60 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f852b7fe700 1 --2- 172.21.15.29:0/3307143410 >> v2:172.21.15.163:3300/0 conn(0x7f852c066720 0x7f852c066b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f852bfff700 1 --2- 172.21.15.29:0/3307143410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f852c13ba60 0x7f852c156500 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f852c067730 tx=0x7f8524009780).ready entity=mon.0 client_cookie=1e7ecc4d333317d6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f852affd700 1 -- 172.21.15.29:0/3307143410 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852400e070 con 0x7f852c13ba60 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.470+0000 7f852affd700 1 -- 172.21.15.29:0/3307143410 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852400bc30 con 0x7f852c13ba60 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.471+0000 7f852affd700 1 -- 172.21.15.29:0/3307143410 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524012410 con 0x7f852c13ba60 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.471+0000 7f8531a90700 1 -- 172.21.15.29:0/3307143410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f852c13ba60 msgr2=0x7f852c156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.471+0000 7f8531a90700 1 --2- 172.21.15.29:0/3307143410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f852c13ba60 0x7f852c156500 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f852c067730 tx=0x7f8524009780).stop 2024-01-24T14:11:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.471+0000 7f8531a90700 1 -- 172.21.15.29:0/3307143410 shutdown_connections 2024-01-24T14:11:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.471+0000 7f8531a90700 1 --2- 172.21.15.29:0/3307143410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f852c13ba60 0x7f852c156500 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.471+0000 7f8531a90700 1 --2- 172.21.15.29:0/3307143410 >> v2:172.21.15.163:3300/0 conn(0x7f852c066720 0x7f852c066b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.471+0000 7f8531a90700 1 -- 172.21.15.29:0/3307143410 >> 172.21.15.29:0/3307143410 conn(0x7f852c05ec80 msgr2=0x7f852c0c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.471+0000 7f8531a90700 1 -- 172.21.15.29:0/3307143410 shutdown_connections 2024-01-24T14:11:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.471+0000 7f8531a90700 1 -- 172.21.15.29:0/3307143410 wait complete. 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.472+0000 7f8531a90700 1 Processor -- start 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.473+0000 7f8531a90700 1 -- start start 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.473+0000 7f8531a90700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f852c066720 0x7f852c151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.473+0000 7f8531a90700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f852c0c4ee0 0x7f852c0a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.473+0000 7f8531a90700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f852c13ba60 0x7f852c0a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.473+0000 7f8531a90700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0a9dd0 con 0x7f852c0c4ee0 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.473+0000 7f8531a90700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0ac070 con 0x7f852c13ba60 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.473+0000 7f8531a90700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c064560 con 0x7f852c066720 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.473+0000 7f8522ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f852c0c4ee0 0x7f852c0a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8522ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f852c0c4ee0 0x7f852c0a93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42698/0 (socket says 172.21.15.29:42698) 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8522ffd700 1 -- 172.21.15.29:0/3797923022 learned_addr learned my addr 172.21.15.29:0/3797923022 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f852bfff700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f852c13ba60 0x7f852c0a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f852b7fe700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f852c066720 0x7f852c151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8522ffd700 1 -- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f852c066720 msgr2=0x7f852c151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8522ffd700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f852c066720 0x7f852c151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8522ffd700 1 -- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f852c13ba60 msgr2=0x7f852c0a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8522ffd700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f852c13ba60 0x7f852c0a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8522ffd700 1 -- 172.21.15.29:0/3797923022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8524009a20 con 0x7f852c0c4ee0 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f852b7fe700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f852c066720 0x7f852c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8522ffd700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f852c0c4ee0 0x7f852c0a93f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f851400c4e0 tx=0x7f8514007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8523fff700 1 -- 172.21.15.29:0/3797923022 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85140072e0 con 0x7f852c0c4ee0 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8523fff700 1 -- 172.21.15.29:0/3797923022 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8514007440 con 0x7f852c0c4ee0 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8531a90700 1 -- 172.21.15.29:0/3797923022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c064840 con 0x7f852c0c4ee0 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8523fff700 1 -- 172.21.15.29:0/3797923022 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85140057a0 con 0x7f852c0c4ee0 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.474+0000 7f8531a90700 1 -- 172.21.15.29:0/3797923022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c0abd00 con 0x7f852c0c4ee0 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.475+0000 7f8531a90700 1 -- 172.21.15.29:0/3797923022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c0ac200 con 0x7f852c0c4ee0 2024-01-24T14:11:04.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.476+0000 7f8523fff700 1 -- 172.21.15.29:0/3797923022 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8514005bd0 con 0x7f852c0c4ee0 2024-01-24T14:11:04.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.476+0000 7f8523fff700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f850c05f270 0x7f850c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.476+0000 7f8523fff700 1 -- 172.21.15.29:0/3797923022 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f851407a010 con 0x7f852c0c4ee0 2024-01-24T14:11:04.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.476+0000 7f852b7fe700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f850c05f270 0x7f850c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.477+0000 7f852b7fe700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f850c05f270 0x7f850c061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f851c00aa90 tx=0x7f851c008040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:04.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.481+0000 7f8523fff700 1 -- 172.21.15.29:0/3797923022 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f851404ad60 con 0x7f852c0c4ee0 2024-01-24T14:11:04.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.730+0000 7f8531a90700 1 -- 172.21.15.29:0/3797923022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f852c0475e0 con 0x7f852c0c4ee0 2024-01-24T14:11:04.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.734+0000 7f8523fff700 1 -- 172.21.15.29:0/3797923022 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f851404e380 con 0x7f852c0c4ee0 2024-01-24T14:11:04.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.736+0000 7f8531a90700 1 -- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f850c05f270 msgr2=0x7f850c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.736+0000 7f8531a90700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f850c05f270 0x7f850c061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f851c00aa90 tx=0x7f851c008040).stop 2024-01-24T14:11:04.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.736+0000 7f8531a90700 1 -- 172.21.15.29:0/3797923022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f852c0c4ee0 msgr2=0x7f852c0a93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.736+0000 7f8531a90700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f852c0c4ee0 0x7f852c0a93f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f851400c4e0 tx=0x7f8514007c70).stop 2024-01-24T14:11:04.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.736+0000 7f8531a90700 1 -- 172.21.15.29:0/3797923022 shutdown_connections 2024-01-24T14:11:04.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.736+0000 7f8531a90700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f852c13ba60 0x7f852c0a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.736+0000 7f8531a90700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f852c066720 0x7f852c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.736+0000 7f8531a90700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f850c05f270 0x7f850c061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.736+0000 7f8531a90700 1 --2- 172.21.15.29:0/3797923022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f852c0c4ee0 0x7f852c0a93f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.736+0000 7f8531a90700 1 -- 172.21.15.29:0/3797923022 >> 172.21.15.29:0/3797923022 conn(0x7f852c05ec80 msgr2=0x7f852c138250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:04.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.736+0000 7f8531a90700 1 -- 172.21.15.29:0/3797923022 shutdown_connections 2024-01-24T14:11:04.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.736+0000 7f8531a90700 1 -- 172.21.15.29:0/3797923022 wait complete. 2024-01-24T14:11:04.738 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-01-24T14:11:04.750 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:11:04.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.880+0000 7fd3a509a700 1 Processor -- start 2024-01-24T14:11:04.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.880+0000 7fd3a509a700 1 -- start start 2024-01-24T14:11:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd3a509a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd3a0066c00 0x7fd3a00c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd3a509a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3a01350a0 0x7fd3a0152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd3a509a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a0068360 con 0x7fd3a0134cc0 2024-01-24T14:11:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd3a509a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a0153360 con 0x7fd3a0066c00 2024-01-24T14:11:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd3a509a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00c3820 con 0x7fd3a01350a0 2024-01-24T14:11:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39e59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd3a0066c00 0x7fd3a00c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3a01350a0 0x7fd3a0152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3a01350a0 0x7fd3a0152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:48712/0 (socket says 172.21.15.29:48712) 2024-01-24T14:11:04.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39e59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd3a0066c00 0x7fd3a00c31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:43154/0 (socket says 172.21.15.29:43154) 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39ed9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd3a0134cc0 0x7fd3a00664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:55802/0 (socket says 172.21.15.29:55802) 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39f59e700 1 -- 172.21.15.29:0/1357469277 learned_addr learned my addr 172.21.15.29:0/1357469277 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39dd9b700 1 -- 172.21.15.29:0/1357469277 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644951615 0 0) 0x7fd3a0068360 con 0x7fd3a0134cc0 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39dd9b700 1 -- 172.21.15.29:0/1357469277 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3880033f0 con 0x7fd3a0134cc0 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39f59e700 1 -- 172.21.15.29:0/1357469277 >> v2:172.21.15.163:3300/0 conn(0x7fd3a0066c00 msgr2=0x7fd3a00c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39f59e700 1 --2- 172.21.15.29:0/1357469277 >> v2:172.21.15.163:3300/0 conn(0x7fd3a0066c00 0x7fd3a00c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39f59e700 1 -- 172.21.15.29:0/1357469277 >> v1:172.21.15.163:6789/0 conn(0x7fd3a0134cc0 legacy=0x7fd3a00664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39f59e700 1 -- 172.21.15.29:0/1357469277 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a0067a30 con 0x7fd3a01350a0 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.881+0000 7fd39e59c700 1 --2- 172.21.15.29:0/1357469277 >> v2:172.21.15.163:3300/0 conn(0x7fd3a0066c00 0x7fd3a00c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.882+0000 7fd39f59e700 1 --2- 172.21.15.29:0/1357469277 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3a01350a0 0x7fd3a0152e70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fd398009f20 tx=0x7fd398009840).ready entity=mon.2 client_cookie=683d69750eb90dfd server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.882+0000 7fd39dd9b700 1 -- 172.21.15.29:0/1357469277 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800e070 con 0x7fd3a01350a0 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.882+0000 7fd39dd9b700 1 -- 172.21.15.29:0/1357469277 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39800b970 con 0x7fd3a01350a0 2024-01-24T14:11:04.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.882+0000 7fd39dd9b700 1 -- 172.21.15.29:0/1357469277 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800bcf0 con 0x7fd3a01350a0 2024-01-24T14:11:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.882+0000 7fd3a509a700 1 -- 172.21.15.29:0/1357469277 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3a01350a0 msgr2=0x7fd3a0152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.882+0000 7fd3a509a700 1 --2- 172.21.15.29:0/1357469277 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3a01350a0 0x7fd3a0152e70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fd398009f20 tx=0x7fd398009840).stop 2024-01-24T14:11:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.882+0000 7fd3a509a700 1 -- 172.21.15.29:0/1357469277 shutdown_connections 2024-01-24T14:11:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.882+0000 7fd3a509a700 1 --2- 172.21.15.29:0/1357469277 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3a01350a0 0x7fd3a0152e70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.882+0000 7fd3a509a700 1 --2- 172.21.15.29:0/1357469277 >> v2:172.21.15.163:3300/0 conn(0x7fd3a0066c00 0x7fd3a00c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.882+0000 7fd3a509a700 1 -- 172.21.15.29:0/1357469277 >> 172.21.15.29:0/1357469277 conn(0x7fd3a00c8bb0 msgr2=0x7fd3a00c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.882+0000 7fd3a509a700 1 -- 172.21.15.29:0/1357469277 shutdown_connections 2024-01-24T14:11:04.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.882+0000 7fd3a509a700 1 -- 172.21.15.29:0/1357469277 wait complete. 2024-01-24T14:11:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.884+0000 7fd3a509a700 1 Processor -- start 2024-01-24T14:11:04.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.884+0000 7fd3a509a700 1 -- start start 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.885+0000 7fd3a509a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3a0066c00 0x7fd3a0133780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.885+0000 7fd3a509a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3a0134cc0 0x7fd3a00aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.885+0000 7fd3a509a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3a01350a0 0x7fd3a00aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.885+0000 7fd3a509a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00ab410 con 0x7fd3a01350a0 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.885+0000 7fd3a509a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00a8610 con 0x7fd3a0066c00 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.885+0000 7fd3a509a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a01657c0 con 0x7fd3a0134cc0 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd39f59e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3a01350a0 0x7fd3a00aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd39f59e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3a01350a0 0x7fd3a00aaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42708/0 (socket says 172.21.15.29:42708) 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd39f59e700 1 -- 172.21.15.29:0/3520621637 learned_addr learned my addr 172.21.15.29:0/3520621637 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd39ed9d700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3a0066c00 0x7fd3a0133780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd39e59c700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3a0134cc0 0x7fd3a00aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd39f59e700 1 -- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3a0134cc0 msgr2=0x7fd3a00aaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd39f59e700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3a0134cc0 0x7fd3a00aaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd39f59e700 1 -- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3a0066c00 msgr2=0x7fd3a0133780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd39f59e700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3a0066c00 0x7fd3a0133780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd39f59e700 1 -- 172.21.15.29:0/3520621637 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3940092a0 con 0x7fd3a01350a0 2024-01-24T14:11:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd39f59e700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3a01350a0 0x7fd3a00aaf20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fd3a00c3960 tx=0x7fd398009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd386ffd700 1 -- 172.21.15.29:0/3520621637 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800bee0 con 0x7fd3a01350a0 2024-01-24T14:11:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd386ffd700 1 -- 172.21.15.29:0/3520621637 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3980092e0 con 0x7fd3a01350a0 2024-01-24T14:11:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd3a509a700 1 -- 172.21.15.29:0/3520621637 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd398009ae0 con 0x7fd3a01350a0 2024-01-24T14:11:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.886+0000 7fd386ffd700 1 -- 172.21.15.29:0/3520621637 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398007860 con 0x7fd3a01350a0 2024-01-24T14:11:04.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.887+0000 7fd3a509a700 1 -- 172.21.15.29:0/3520621637 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a0165d80 con 0x7fd3a01350a0 2024-01-24T14:11:04.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.888+0000 7fd3a509a700 1 -- 172.21.15.29:0/3520621637 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a00475e0 con 0x7fd3a01350a0 2024-01-24T14:11:04.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.888+0000 7fd386ffd700 1 -- 172.21.15.29:0/3520621637 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd398003040 con 0x7fd3a01350a0 2024-01-24T14:11:04.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.888+0000 7fd386ffd700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd38c05f270 0x7fd38c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:04.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.888+0000 7fd386ffd700 1 -- 172.21.15.29:0/3520621637 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fd39807c020 con 0x7fd3a01350a0 2024-01-24T14:11:04.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.888+0000 7fd39ed9d700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd38c05f270 0x7fd38c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:04.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.889+0000 7fd39ed9d700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd38c05f270 0x7fd38c061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd390007640 tx=0x7fd390007bc0).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:04.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:04.893+0000 7fd386ffd700 1 -- 172.21.15.29:0/3520621637 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd39804cee0 con 0x7fd3a01350a0 2024-01-24T14:11:05.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.117+0000 7fd3a509a700 1 -- 172.21.15.29:0/3520621637 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3a00c33f0 con 0x7fd3a01350a0 2024-01-24T14:11:05.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.118+0000 7fd386ffd700 1 -- 172.21.15.29:0/3520621637 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4492 (secure 0 0 0) 0x7fd398023070 con 0x7fd3a01350a0 2024-01-24T14:11:05.119 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:11:05.119 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2239835986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2239835986},{"type":"v1","addr":"172.21.15.163:6837","nonce":2239835986}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2208246875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2208246875},{"type":"v1","addr":"172.21.15.29:6835","nonce":2208246875}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/547152226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":547152226},{"type":"v1","addr":"172.21.15.163:6836","nonce":547152226}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:10:59.693964+0000","modified":"2024-01-24T14:11:02.793474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2804469957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2804469957},{"type":"v1","addr":"172.21.15.29:6834","nonce":2804469957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-24T14:11:05.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.120+0000 7fd3a509a700 1 -- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd38c05f270 msgr2=0x7fd38c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.120+0000 7fd3a509a700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd38c05f270 0x7fd38c061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd390007640 tx=0x7fd390007bc0).stop 2024-01-24T14:11:05.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.120+0000 7fd3a509a700 1 -- 172.21.15.29:0/3520621637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3a01350a0 msgr2=0x7fd3a00aaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.120+0000 7fd3a509a700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3a01350a0 0x7fd3a00aaf20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fd3a00c3960 tx=0x7fd398009840).stop 2024-01-24T14:11:05.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.120+0000 7fd3a509a700 1 -- 172.21.15.29:0/3520621637 shutdown_connections 2024-01-24T14:11:05.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.120+0000 7fd3a509a700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd38c05f270 0x7fd38c061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.120+0000 7fd3a509a700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3a01350a0 0x7fd3a00aaf20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.120+0000 7fd3a509a700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3a0066c00 0x7fd3a0133780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.120+0000 7fd3a509a700 1 --2- 172.21.15.29:0/3520621637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3a0134cc0 0x7fd3a00aaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.120+0000 7fd3a509a700 1 -- 172.21.15.29:0/3520621637 >> 172.21.15.29:0/3520621637 conn(0x7fd3a00c8bb0 msgr2=0x7fd3a00c6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:05.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.120+0000 7fd3a509a700 1 -- 172.21.15.29:0/3520621637 shutdown_connections 2024-01-24T14:11:05.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.120+0000 7fd3a509a700 1 -- 172.21.15.29:0/3520621637 wait complete. 2024-01-24T14:11:05.122 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 26 2024-01-24T14:11:05.212 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-01-24T14:11:05.212+0000 7fa23d84c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T14:11:05.538 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T14:10:59.693964+0000', 'modified': '2024-01-24T14:11:02.793474+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5955}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5955': {'gid': 5955, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6834/2804469957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 2804469957}, {'type': 'v1', 'addr': '172.21.15.29:6834', 'nonce': 2804469957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-24T14:11:05.538 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T14:11:05.538 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-24T14:11:05.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.679+0000 7fc30f845700 1 Processor -- start 2024-01-24T14:11:05.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.679+0000 7fc30f845700 1 -- start start 2024-01-24T14:11:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.680+0000 7fc30f845700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3080bdd10 0x7fc3081516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.680+0000 7fc30f845700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc30806a350 con 0x7fc308069240 2024-01-24T14:11:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.680+0000 7fc30f845700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc308069740 con 0x7fc308150bd0 2024-01-24T14:11:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.680+0000 7fc30f845700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc308069a20 con 0x7fc3080bdd10 2024-01-24T14:11:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.680+0000 7fc30cde0700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc308069240 0x7fc308150fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42050/0 (socket says 172.21.15.29:42050) 2024-01-24T14:11:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.680+0000 7fc30cde0700 1 -- 172.21.15.29:0/3128248835 learned_addr learned my addr 172.21.15.29:0/3128248835 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:11:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.681+0000 7fc30dde2700 1 --2- 172.21.15.29:0/3128248835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3080bdd10 0x7fc3081516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.681+0000 7fc2fbfff700 1 -- 172.21.15.29:0/3128248835 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72103656 0 0) 0x7fc30806a350 con 0x7fc308069240 2024-01-24T14:11:05.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.681+0000 7fc2fbfff700 1 -- 172.21.15.29:0/3128248835 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2f40033f0 con 0x7fc308069240 2024-01-24T14:11:05.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.681+0000 7fc2fbfff700 1 -- 172.21.15.29:0/3128248835 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467239944 0 0) 0x7fc308069740 con 0x7fc308150bd0 2024-01-24T14:11:05.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.681+0000 7fc2fbfff700 1 -- 172.21.15.29:0/3128248835 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc30806a350 con 0x7fc308150bd0 2024-01-24T14:11:05.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.681+0000 7fc30dde2700 1 -- 172.21.15.29:0/3128248835 >> v1:172.21.15.163:6789/0 conn(0x7fc308150bd0 legacy=0x7fc308065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.681+0000 7fc2fbfff700 1 -- 172.21.15.29:0/3128248835 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 572029124 0 0) 0x7fc2f40033f0 con 0x7fc308069240 2024-01-24T14:11:05.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.681+0000 7fc30dde2700 1 -- 172.21.15.29:0/3128248835 >> v1:172.21.15.29:6789/0 conn(0x7fc308069240 legacy=0x7fc308150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.681+0000 7fc30dde2700 1 -- 172.21.15.29:0/3128248835 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc308069d00 con 0x7fc3080bdd10 2024-01-24T14:11:05.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.681+0000 7fc30dde2700 1 --2- 172.21.15.29:0/3128248835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3080bdd10 0x7fc3081516c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fc300009e90 tx=0x7fc300009780).ready entity=mon.2 client_cookie=22357dadc2c6c30a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:05.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.681+0000 7fc2fbfff700 1 -- 172.21.15.29:0/3128248835 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc001470 con 0x7fc3080bdd10 2024-01-24T14:11:05.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.681+0000 7fc2fbfff700 1 -- 172.21.15.29:0/3128248835 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30000b7a0 con 0x7fc3080bdd10 2024-01-24T14:11:05.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.682+0000 7fc2fbfff700 1 -- 172.21.15.29:0/3128248835 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000bb20 con 0x7fc3080bdd10 2024-01-24T14:11:05.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.682+0000 7fc30f845700 1 -- 172.21.15.29:0/3128248835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3080bdd10 msgr2=0x7fc3081516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.682+0000 7fc30f845700 1 --2- 172.21.15.29:0/3128248835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3080bdd10 0x7fc3081516c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fc300009e90 tx=0x7fc300009780).stop 2024-01-24T14:11:05.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.682+0000 7fc30f845700 1 -- 172.21.15.29:0/3128248835 shutdown_connections 2024-01-24T14:11:05.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.682+0000 7fc30f845700 1 --2- 172.21.15.29:0/3128248835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3080bdd10 0x7fc3081516c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.682+0000 7fc30f845700 1 -- 172.21.15.29:0/3128248835 >> 172.21.15.29:0/3128248835 conn(0x7fc3080c8580 msgr2=0x7fc3080c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:05.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.682+0000 7fc30f845700 1 -- 172.21.15.29:0/3128248835 shutdown_connections 2024-01-24T14:11:05.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.682+0000 7fc30f845700 1 -- 172.21.15.29:0/3128248835 wait complete. 2024-01-24T14:11:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.683+0000 7fc30f845700 1 Processor -- start 2024-01-24T14:11:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.684+0000 7fc30f845700 1 -- start start 2024-01-24T14:11:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.684+0000 7fc30f845700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc308069240 0x7fc308163c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.684+0000 7fc30f845700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3080bdd10 0x7fc308164120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30f845700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc308150bd0 0x7fc3080abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30f845700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3080c16a0 con 0x7fc308069240 2024-01-24T14:11:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30f845700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3080c0690 con 0x7fc3080bdd10 2024-01-24T14:11:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30f845700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3080c1210 con 0x7fc308150bd0 2024-01-24T14:11:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30d5e1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc308069240 0x7fc308163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30d5e1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc308069240 0x7fc308163c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42714/0 (socket says 172.21.15.29:42714) 2024-01-24T14:11:05.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30d5e1700 1 -- 172.21.15.29:0/2655387113 learned_addr learned my addr 172.21.15.29:0/2655387113 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30cde0700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3080bdd10 0x7fc308164120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30dde2700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc308150bd0 0x7fc3080abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30d5e1700 1 -- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc308150bd0 msgr2=0x7fc3080abd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc308150bd0 0x7fc3080abd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30d5e1700 1 -- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3080bdd10 msgr2=0x7fc308164120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3080bdd10 0x7fc308164120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30d5e1700 1 -- 172.21.15.29:0/2655387113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc300009a20 con 0x7fc308069240 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc308069240 0x7fc308163c30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fc30400c4c0 tx=0x7fc304007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc2f97fa700 1 -- 172.21.15.29:0/2655387113 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30400ce30 con 0x7fc308069240 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc2f97fa700 1 -- 172.21.15.29:0/2655387113 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30400f470 con 0x7fc308069240 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc2f97fa700 1 -- 172.21.15.29:0/2655387113 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30400f810 con 0x7fc308069240 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.685+0000 7fc30f845700 1 -- 172.21.15.29:0/2655387113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3080c08e0 con 0x7fc308069240 2024-01-24T14:11:05.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.686+0000 7fc30f845700 1 -- 172.21.15.29:0/2655387113 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3080c0ea0 con 0x7fc308069240 2024-01-24T14:11:05.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.687+0000 7fc2f97fa700 1 -- 172.21.15.29:0/2655387113 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc30400fb80 con 0x7fc308069240 2024-01-24T14:11:05.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.688+0000 7fc2f97fa700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2ec05f270 0x7fc2ec061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.688+0000 7fc2f97fa700 1 -- 172.21.15.29:0/2655387113 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fc304079cd0 con 0x7fc308069240 2024-01-24T14:11:05.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.688+0000 7fc30cde0700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2ec05f270 0x7fc2ec061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.688+0000 7fc30cde0700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2ec05f270 0x7fc2ec061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc2fc001ed0 tx=0x7fc2fc001980).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:05.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.690+0000 7fc321210b80 1 Processor -- start 2024-01-24T14:11:05.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.690+0000 7fc321210b80 1 -- start start 2024-01-24T14:11:05.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.690+0000 7fc321210b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a494620 0x56182a494a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.690+0000 7fc321210b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a4dd2c0 0x56182a48f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.690+0000 7fc321210b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 0x56182a490270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.690+0000 7fc321210b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a40b3d0 con 0x56182a48fdf0 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.690+0000 7fc321210b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a3d0be0 con 0x56182a4dd2c0 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.690+0000 7fc321210b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56182a3e6260 con 0x56182a494620 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc30dde2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 0x56182a490270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc30dde2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 0x56182a490270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42718/0 (socket says 172.21.15.29:42718) 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc30dde2700 1 -- 172.21.15.29:0/1087517967 learned_addr learned my addr 172.21.15.29:0/1087517967 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc30cde0700 1 --2- 172.21.15.29:0/1087517967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a4dd2c0 0x56182a48f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/1087517967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a494620 0x56182a494a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc30dde2700 1 -- 172.21.15.29:0/1087517967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a494620 msgr2=0x56182a494a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc30dde2700 1 --2- 172.21.15.29:0/1087517967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a494620 0x56182a494a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc30dde2700 1 -- 172.21.15.29:0/1087517967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a4dd2c0 msgr2=0x56182a48f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc30dde2700 1 --2- 172.21.15.29:0/1087517967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a4dd2c0 0x56182a48f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc30dde2700 1 -- 172.21.15.29:0/1087517967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56182a3df2f0 con 0x56182a48fdf0 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc30dde2700 1 --2- 172.21.15.29:0/1087517967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 0x56182a490270 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fc3080649f0 tx=0x7fc30001a000).ready entity=mon.0 client_cookie=92ffbfaddd05d4a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc2f27fc700 1 -- 172.21.15.29:0/1087517967 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000b910 con 0x56182a48fdf0 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc2f27fc700 1 -- 172.21.15.29:0/1087517967 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30000ba70 con 0x56182a48fdf0 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.691+0000 7fc2f27fc700 1 -- 172.21.15.29:0/1087517967 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3000106e0 con 0x56182a48fdf0 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.692+0000 7fc321210b80 1 -- 172.21.15.29:0/1087517967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 msgr2=0x56182a490270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.692+0000 7fc321210b80 1 --2- 172.21.15.29:0/1087517967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 0x56182a490270 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fc3080649f0 tx=0x7fc30001a000).stop 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.692+0000 7fc321210b80 1 -- 172.21.15.29:0/1087517967 shutdown_connections 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.692+0000 7fc321210b80 1 --2- 172.21.15.29:0/1087517967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 0x56182a490270 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.692+0000 7fc321210b80 1 --2- 172.21.15.29:0/1087517967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a494620 0x56182a494a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.692+0000 7fc321210b80 1 --2- 172.21.15.29:0/1087517967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a4dd2c0 0x56182a48f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.692+0000 7fc321210b80 1 -- 172.21.15.29:0/1087517967 >> 172.21.15.29:0/1087517967 conn(0x56182a2fd2e0 msgr2=0x56182a4dd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.692+0000 7fc321210b80 1 -- 172.21.15.29:0/1087517967 shutdown_connections 2024-01-24T14:11:05.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.692+0000 7fc321210b80 1 -- 172.21.15.29:0/1087517967 wait complete. 2024-01-24T14:11:05.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.693+0000 7fc321210b80 1 -- start start 2024-01-24T14:11:05.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.693+0000 7fc321210b80 1 Processor -- start 2024-01-24T14:11:05.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc321210b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a48fdf0 0x56182a4bf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc321210b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a494620 0x56182a4c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc321210b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 0x56182a4c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc321210b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a3e63a0 con 0x56182a4dd2c0 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc321210b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a3e6620 con 0x56182a494620 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc321210b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56182a3b1990 con 0x56182a48fdf0 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc321210b80 10 client.0 ll_register_callbacks cb 0x56182a454610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc30dde2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 0x56182a4c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc30dde2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 0x56182a4c30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42734/0 (socket says 172.21.15.29:42734) 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc30dde2700 1 -- 172.21.15.29:0/1247916818 learned_addr learned my addr 172.21.15.29:0/1247916818 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc30cde0700 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a494620 0x56182a4c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a48fdf0 0x56182a4bf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc30dde2700 1 -- 172.21.15.29:0/1247916818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a48fdf0 msgr2=0x56182a4bf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc30dde2700 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a48fdf0 0x56182a4bf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc30dde2700 1 -- 172.21.15.29:0/1247916818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a494620 msgr2=0x56182a4c2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc30dde2700 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a494620 0x56182a4c2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.694+0000 7fc30dde2700 1 -- 172.21.15.29:0/1247916818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc308069d00 con 0x56182a4dd2c0 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc30dde2700 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 0x56182a4c30d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x56182a4dd720 tx=0x7fc300022000).ready entity=mon.0 client_cookie=560fcd6cf6b18762 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:05.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc2e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T14:11:05.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc2e77fe700 1 -- 172.21.15.29:0/1247916818 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300010d70 con 0x56182a4dd2c0 2024-01-24T14:11:05.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc2e77fe700 1 -- 172.21.15.29:0/1247916818 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fc30002d460 con 0x56182a4dd2c0 2024-01-24T14:11:05.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc321210b80 1 -- 172.21.15.29:0/1247916818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56182a48edf0 con 0x56182a4dd2c0 2024-01-24T14:11:05.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc2e77fe700 1 -- 172.21.15.29:0/1247916818 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc30002d830 con 0x56182a4dd2c0 2024-01-24T14:11:05.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc321210b80 10 client.6078 fetch_fsmap learned FSMap version 26 2024-01-24T14:11:05.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc321210b80 1 -- 172.21.15.29:0/1247916818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x56182a3da520 con 0x56182a4dd2c0 2024-01-24T14:11:05.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc2e77fe700 1 -- 172.21.15.29:0/1247916818 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56182a48edf0 con 0x56182a4dd2c0 2024-01-24T14:11:05.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc2e77fe700 1 -- 172.21.15.29:0/1247916818 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 26) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc30002d990 con 0x56182a4dd2c0 2024-01-24T14:11:05.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc321210b80 10 client.6078 fetch_fsmap finished waiting for FSMap version 26 2024-01-24T14:11:05.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc321210b80 10 client.6078 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T14:11:05.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc321210b80 20 client.6078 populate_metadata read hostname 'smithi029' 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc321210b80 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.29:6833/2804469957,v1:172.21.15.29:6834/2804469957] conn(0x56182a4be960 0x56182a4e9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc321210b80 4 client.6078 mds_command: new command op to 5955 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.695+0000 7fc321210b80 1 -- 172.21.15.29:0/1247916818 --> [v2:172.21.15.29:6833/2804469957,v1:172.21.15.29:6834/2804469957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56182a3dbd60 con 0x56182a4be960 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.696+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.29:6833/2804469957,v1:172.21.15.29:6834/2804469957] conn(0x56182a4be960 0x56182a4e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.696+0000 7fc2e77fe700 10 client.6078 ms_handle_connect on v2:172.21.15.29:6833/2804469957 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.696+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.29:6833/2804469957,v1:172.21.15.29:6834/2804469957] conn(0x56182a4be960 0x56182a4e9290 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=51845e264c72034 server_cookie=5166b1a09495d3fd in_seq=0 out_seq=0 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.696+0000 7fc2e77fe700 1 -- 172.21.15.29:0/1247916818 <== mds.0 v2:172.21.15.29:6833/2804469957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x56182a3dbd60 con 0x56182a4be960 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.696+0000 7fc2e77fe700 10 client.6078 handle_command_reply: tid=0 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc321210b80 1 client.6078 shutdown 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc321210b80 1 -- 172.21.15.29:0/1247916818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 msgr2=0x56182a4c30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc321210b80 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 0x56182a4c30d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x56182a4dd720 tx=0x7fc300022000).stop 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc321210b80 1 -- 172.21.15.29:0/1247916818 shutdown_connections 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc321210b80 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.29:6833/2804469957,v1:172.21.15.29:6834/2804469957] conn(0x56182a4be960 0x56182a4e9290 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc321210b80 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 0x56182a4c30d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc321210b80 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a48fdf0 0x56182a4bf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc321210b80 1 --2- 172.21.15.29:0/1247916818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a494620 0x56182a4c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc2e77fe700 0 client.6078 ms_handle_reset on v2:172.21.15.29:6833/2804469957 2024-01-24T14:11:05.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc321210b80 1 -- 172.21.15.29:0/1247916818 >> 172.21.15.29:0/1247916818 conn(0x56182a2fd2e0 msgr2=0x56182a494bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:05.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc321210b80 1 -- 172.21.15.29:0/1247916818 shutdown_connections 2024-01-24T14:11:05.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc321210b80 1 -- 172.21.15.29:0/1247916818 wait complete. 2024-01-24T14:11:05.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.697+0000 7fc321210b80 20 client.6078 trim_cache size 0 max 16384 2024-01-24T14:11:05.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.715+0000 7fc321210b80 1 Processor -- start 2024-01-24T14:11:05.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.715+0000 7fc321210b80 1 -- start start 2024-01-24T14:11:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc321210b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 0x56182a4e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc321210b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a494620 0x56182a48b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc321210b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a4dd2c0 0x56182a4bf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc321210b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a4dd180 con 0x56182a48fdf0 2024-01-24T14:11:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc321210b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a48e380 con 0x56182a4dd2c0 2024-01-24T14:11:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc321210b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56182a4c2aa0 con 0x56182a494620 2024-01-24T14:11:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30d5e1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 0x56182a4e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30d5e1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 0x56182a4e8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42748/0 (socket says 172.21.15.29:42748) 2024-01-24T14:11:05.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30d5e1700 1 -- 172.21.15.29:0/1105141159 learned_addr learned my addr 172.21.15.29:0/1105141159 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30dde2700 1 --2- 172.21.15.29:0/1105141159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a4dd2c0 0x56182a4bf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30cde0700 1 --2- 172.21.15.29:0/1105141159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a494620 0x56182a48b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30d5e1700 1 -- 172.21.15.29:0/1105141159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a494620 msgr2=0x56182a48b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/1105141159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a494620 0x56182a48b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30d5e1700 1 -- 172.21.15.29:0/1105141159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a4dd2c0 msgr2=0x56182a4bf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/1105141159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a4dd2c0 0x56182a4bf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30d5e1700 1 -- 172.21.15.29:0/1105141159 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56182a3da270 con 0x56182a48fdf0 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30dde2700 1 --2- 172.21.15.29:0/1105141159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a4dd2c0 0x56182a4bf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30cde0700 1 --2- 172.21.15.29:0/1105141159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a494620 0x56182a48b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.716+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/1105141159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 0x56182a4e8610 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fc30404e3e0 tx=0x7fc304014040).ready entity=mon.0 client_cookie=c69c39fbdad7d169 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.717+0000 7fc2f2ffd700 1 -- 172.21.15.29:0/1105141159 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304053070 con 0x56182a48fdf0 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.717+0000 7fc2f2ffd700 1 -- 172.21.15.29:0/1105141159 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30404c8e0 con 0x56182a48fdf0 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.717+0000 7fc2f2ffd700 1 -- 172.21.15.29:0/1105141159 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30404adf0 con 0x56182a48fdf0 2024-01-24T14:11:05.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.717+0000 7fc321210b80 1 -- 172.21.15.29:0/1105141159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 msgr2=0x56182a4e8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.717+0000 7fc321210b80 1 --2- 172.21.15.29:0/1105141159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 0x56182a4e8610 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fc30404e3e0 tx=0x7fc304014040).stop 2024-01-24T14:11:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.717+0000 7fc321210b80 1 -- 172.21.15.29:0/1105141159 shutdown_connections 2024-01-24T14:11:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.717+0000 7fc321210b80 1 --2- 172.21.15.29:0/1105141159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a4dd2c0 0x56182a4bf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.717+0000 7fc321210b80 1 --2- 172.21.15.29:0/1105141159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a48fdf0 0x56182a4e8610 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.717+0000 7fc321210b80 1 --2- 172.21.15.29:0/1105141159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a494620 0x56182a48b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.717+0000 7fc321210b80 1 -- 172.21.15.29:0/1105141159 >> 172.21.15.29:0/1105141159 conn(0x56182a2fd2e0 msgr2=0x56182a4919a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.717+0000 7fc321210b80 1 -- 172.21.15.29:0/1105141159 shutdown_connections 2024-01-24T14:11:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.717+0000 7fc321210b80 1 -- 172.21.15.29:0/1105141159 wait complete. 2024-01-24T14:11:05.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.718+0000 7fc321210b80 1 -- start start 2024-01-24T14:11:05.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.718+0000 7fc321210b80 1 Processor -- start 2024-01-24T14:11:05.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc321210b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a48fdf0 0x56182a48b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc321210b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a494620 0x56182a48b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc321210b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 0x56182a53ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc321210b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a4b8f00 con 0x56182a4dd2c0 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc321210b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a48ba00 con 0x56182a494620 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc321210b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56182a4eb720 con 0x56182a48fdf0 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc321210b80 10 client.0 ll_register_callbacks cb 0x56182a454610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc30dde2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 0x56182a53ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc30dde2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 0x56182a53ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42750/0 (socket says 172.21.15.29:42750) 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc30dde2700 1 -- 172.21.15.29:0/2637002965 learned_addr learned my addr 172.21.15.29:0/2637002965 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc30cde0700 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a494620 0x56182a48b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a48fdf0 0x56182a48b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc30dde2700 1 -- 172.21.15.29:0/2637002965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a48fdf0 msgr2=0x56182a48b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc30dde2700 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a48fdf0 0x56182a48b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc30dde2700 1 -- 172.21.15.29:0/2637002965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a494620 msgr2=0x56182a48b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc30dde2700 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a494620 0x56182a48b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.719+0000 7fc30dde2700 1 -- 172.21.15.29:0/2637002965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3080c0ea0 con 0x56182a4dd2c0 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc30cde0700 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a494620 0x56182a48b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a48fdf0 0x56182a48b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc30dde2700 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 0x56182a53ed30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fc300010a90 tx=0x7fc30002b000).ready entity=mon.0 client_cookie=3a9e5824f8e1291f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc2e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc2e77fe700 1 -- 172.21.15.29:0/2637002965 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30002d460 con 0x56182a4dd2c0 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc2e77fe700 1 -- 172.21.15.29:0/2637002965 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fc30000bbe0 con 0x56182a4dd2c0 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc321210b80 1 -- 172.21.15.29:0/2637002965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56182a3d27d0 con 0x56182a4dd2c0 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc2e77fe700 1 -- 172.21.15.29:0/2637002965 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc300018630 con 0x56182a4dd2c0 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc321210b80 10 client.6084 fetch_fsmap learned FSMap version 26 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc321210b80 1 -- 172.21.15.29:0/2637002965 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x56182a253f80 con 0x56182a4dd2c0 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc2e77fe700 1 -- 172.21.15.29:0/2637002965 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56182a3d27d0 con 0x56182a4dd2c0 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc2e77fe700 1 -- 172.21.15.29:0/2637002965 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 26) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc300018790 con 0x56182a4dd2c0 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc321210b80 10 client.6084 fetch_fsmap finished waiting for FSMap version 26 2024-01-24T14:11:05.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc321210b80 10 client.6084 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc321210b80 20 client.6084 populate_metadata read hostname 'smithi029' 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc321210b80 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.29:6833/2804469957,v1:172.21.15.29:6834/2804469957] conn(0x56182a4eaff0 0x56182a53f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc321210b80 4 client.6084 mds_command: new command op to 5955 tid=0[{"prefix": "status"}] 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc321210b80 1 -- 172.21.15.29:0/2637002965 --> [v2:172.21.15.29:6833/2804469957,v1:172.21.15.29:6834/2804469957] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56182a24eca0 con 0x56182a4eaff0 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.720+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.29:6833/2804469957,v1:172.21.15.29:6834/2804469957] conn(0x56182a4eaff0 0x56182a53f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc2e77fe700 10 client.6084 ms_handle_connect on v2:172.21.15.29:6833/2804469957 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc30d5e1700 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.29:6833/2804469957,v1:172.21.15.29:6834/2804469957] conn(0x56182a4eaff0 0x56182a53f220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=aeb47317d7114d55 server_cookie=681114b3d2898960 in_seq=0 out_seq=0 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc2e77fe700 1 -- 172.21.15.29:0/2637002965 <== mds.0 v2:172.21.15.29:6833/2804469957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56182a24eca0 con 0x56182a4eaff0 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc2e77fe700 10 client.6084 handle_command_reply: tid=0 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc321210b80 1 client.6084 shutdown 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc321210b80 1 -- 172.21.15.29:0/2637002965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 msgr2=0x56182a53ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc321210b80 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 0x56182a53ed30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fc300010a90 tx=0x7fc30002b000).stop 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc321210b80 1 -- 172.21.15.29:0/2637002965 shutdown_connections 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc321210b80 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.29:6833/2804469957,v1:172.21.15.29:6834/2804469957] conn(0x56182a4eaff0 0x56182a53f220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc321210b80 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182a4dd2c0 0x56182a53ed30 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc321210b80 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56182a48fdf0 0x56182a48b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc2e77fe700 0 client.6084 ms_handle_reset on v2:172.21.15.29:6833/2804469957 2024-01-24T14:11:05.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc321210b80 1 --2- 172.21.15.29:0/2637002965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56182a494620 0x56182a48b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc321210b80 1 -- 172.21.15.29:0/2637002965 >> 172.21.15.29:0/2637002965 conn(0x56182a2fd2e0 msgr2=0x56182a4c1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:05.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc321210b80 1 -- 172.21.15.29:0/2637002965 shutdown_connections 2024-01-24T14:11:05.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.721+0000 7fc321210b80 1 -- 172.21.15.29:0/2637002965 wait complete. 2024-01-24T14:11:05.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc321210b80 20 client.6084 trim_cache size 0 max 16384 2024-01-24T14:11:05.728 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T14:11:05.728 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T14:11:05.729 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T14:11:05.729 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 5955, 2024-01-24T14:11:05.729 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T14:11:05.729 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T14:11:05.729 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T14:11:05.729 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 6.0266652389999997, 2024-01-24T14:11:05.729 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 26, 2024-01-24T14:11:05.729 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 72, 2024-01-24T14:11:05.729 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 72, 2024-01-24T14:11:05.729 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 11.519125172000001 2024-01-24T14:11:05.729 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T14:11:05.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc30f845700 1 -- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2ec05f270 msgr2=0x7fc2ec061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc30f845700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2ec05f270 0x7fc2ec061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc2fc001ed0 tx=0x7fc2fc001980).stop 2024-01-24T14:11:05.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc30f845700 1 -- 172.21.15.29:0/2655387113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc308069240 msgr2=0x7fc308163c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc30f845700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc308069240 0x7fc308163c30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fc30400c4c0 tx=0x7fc304007c90).stop 2024-01-24T14:11:05.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc30f845700 1 -- 172.21.15.29:0/2655387113 shutdown_connections 2024-01-24T14:11:05.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc30f845700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc308150bd0 0x7fc3080abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc30f845700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc2ec05f270 0x7fc2ec061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc30f845700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc308069240 0x7fc308163c30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc30f845700 1 --2- 172.21.15.29:0/2655387113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3080bdd10 0x7fc308164120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc30f845700 1 -- 172.21.15.29:0/2655387113 >> 172.21.15.29:0/2655387113 conn(0x7fc3080c8580 msgr2=0x7fc30814a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:05.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc30f845700 1 -- 172.21.15.29:0/2655387113 shutdown_connections 2024-01-24T14:11:05.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.722+0000 7fc30f845700 1 -- 172.21.15.29:0/2655387113 wait complete. 2024-01-24T14:11:05.736 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T14:11:05.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.851+0000 7f03b05c2700 1 Processor -- start 2024-01-24T14:11:05.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.851+0000 7f03b05c2700 1 -- start start 2024-01-24T14:11:05.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.852+0000 7f03b05c2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f03a814b180 0x7f03a80a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.852+0000 7f03b05c2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03a81587e0 0x7f03a80a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.852+0000 7f03b05c2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f03a81545c0 0x7f03a8154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.852+0000 7f03b05c2700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a80a7480 con 0x7f03a81545c0 2024-01-24T14:11:05.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.852+0000 7f03b05c2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a814b560 con 0x7f03a814b180 2024-01-24T14:11:05.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.852+0000 7f03b05c2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8155530 con 0x7f03a81587e0 2024-01-24T14:11:05.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03aeb5f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f03a81545c0 0x7f03a8154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03aeb5f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f03a81545c0 0x7f03a8154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42766/0 (socket says 172.21.15.29:42766) 2024-01-24T14:11:05.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03aeb5f700 1 -- 172.21.15.29:0/2215517004 learned_addr learned my addr 172.21.15.29:0/2215517004 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:05.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03adb5d700 1 --2- 172.21.15.29:0/2215517004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03a81587e0 0x7f03a80a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03ae35e700 1 --2- 172.21.15.29:0/2215517004 >> v2:172.21.15.163:3300/0 conn(0x7f03a814b180 0x7f03a80a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03aeb5f700 1 -- 172.21.15.29:0/2215517004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03a81587e0 msgr2=0x7f03a80a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:05.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03aeb5f700 1 --2- 172.21.15.29:0/2215517004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03a81587e0 0x7f03a80a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03aeb5f700 1 -- 172.21.15.29:0/2215517004 >> v2:172.21.15.163:3300/0 conn(0x7f03a814b180 msgr2=0x7f03a80a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:05.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03aeb5f700 1 --2- 172.21.15.29:0/2215517004 >> v2:172.21.15.163:3300/0 conn(0x7f03a814b180 0x7f03a80a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03aeb5f700 1 -- 172.21.15.29:0/2215517004 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a8158350 con 0x7f03a81545c0 2024-01-24T14:11:05.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03adb5d700 1 --2- 172.21.15.29:0/2215517004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03a81587e0 0x7f03a80a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:11:05.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03ae35e700 1 --2- 172.21.15.29:0/2215517004 >> v2:172.21.15.163:3300/0 conn(0x7f03a814b180 0x7f03a80a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:11:05.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03aeb5f700 1 --2- 172.21.15.29:0/2215517004 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03a81545c0 0x7f03a8154a50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f03a80c58d0 tx=0x7f03a0009840).ready entity=mon.0 client_cookie=7468b269b991904b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:05.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03ad35c700 1 -- 172.21.15.29:0/2215517004 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a000e070 con 0x7f03a81545c0 2024-01-24T14:11:05.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03ad35c700 1 -- 172.21.15.29:0/2215517004 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a000ba50 con 0x7f03a81545c0 2024-01-24T14:11:05.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.853+0000 7f03ad35c700 1 -- 172.21.15.29:0/2215517004 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a000bdd0 con 0x7f03a81545c0 2024-01-24T14:11:05.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.854+0000 7f03b05c2700 1 -- 172.21.15.29:0/2215517004 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03a81545c0 msgr2=0x7f03a8154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.854+0000 7f03b05c2700 1 --2- 172.21.15.29:0/2215517004 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03a81545c0 0x7f03a8154a50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f03a80c58d0 tx=0x7f03a0009840).stop 2024-01-24T14:11:05.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.854+0000 7f03b05c2700 1 -- 172.21.15.29:0/2215517004 shutdown_connections 2024-01-24T14:11:05.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.854+0000 7f03b05c2700 1 --2- 172.21.15.29:0/2215517004 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03a81545c0 0x7f03a8154a50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.854+0000 7f03b05c2700 1 --2- 172.21.15.29:0/2215517004 >> v2:172.21.15.163:3300/0 conn(0x7f03a814b180 0x7f03a80a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.854+0000 7f03b05c2700 1 --2- 172.21.15.29:0/2215517004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03a81587e0 0x7f03a80a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.854+0000 7f03b05c2700 1 -- 172.21.15.29:0/2215517004 >> 172.21.15.29:0/2215517004 conn(0x7f03a80c7b10 msgr2=0x7f03a80c7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:05.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.854+0000 7f03b05c2700 1 -- 172.21.15.29:0/2215517004 shutdown_connections 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.854+0000 7f03b05c2700 1 -- 172.21.15.29:0/2215517004 wait complete. 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.855+0000 7f03b05c2700 1 Processor -- start 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.855+0000 7f03b05c2700 1 -- start start 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.856+0000 7f03b05c2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03a814b180 0x7f03a81493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.856+0000 7f03b05c2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03a81545c0 0x7f03a80a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.856+0000 7f03b05c2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a81587e0 0x7f03a8153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.856+0000 7f03b05c2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a80a66d0 con 0x7f03a81545c0 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.856+0000 7f03b05c2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a80a4ec0 con 0x7f03a81587e0 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.856+0000 7f03b05c2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a81498c0 con 0x7f03a814b180 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.856+0000 7f03adb5d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03a81545c0 0x7f03a80a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.856+0000 7f03adb5d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03a81545c0 0x7f03a80a5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42768/0 (socket says 172.21.15.29:42768) 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.856+0000 7f03adb5d700 1 -- 172.21.15.29:0/171673059 learned_addr learned my addr 172.21.15.29:0/171673059 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.856+0000 7f03ae35e700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03a814b180 0x7f03a81493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.856+0000 7f03aeb5f700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a81587e0 0x7f03a8153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.857+0000 7f03adb5d700 1 -- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03a814b180 msgr2=0x7f03a81493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.857+0000 7f03adb5d700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03a814b180 0x7f03a81493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.857+0000 7f03adb5d700 1 -- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a81587e0 msgr2=0x7f03a8153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:05.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.857+0000 7f03adb5d700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a81587e0 0x7f03a8153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:05.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.857+0000 7f03adb5d700 1 -- 172.21.15.29:0/171673059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a0009ae0 con 0x7f03a81545c0 2024-01-24T14:11:05.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.857+0000 7f03adb5d700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03a81545c0 0x7f03a80a5350 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f03a400d860 tx=0x7f03a4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:05.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.857+0000 7f039a7fc700 1 -- 172.21.15.29:0/171673059 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4011590 con 0x7f03a81545c0 2024-01-24T14:11:05.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.857+0000 7f039a7fc700 1 -- 172.21.15.29:0/171673059 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a40116f0 con 0x7f03a81545c0 2024-01-24T14:11:05.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.857+0000 7f03b05c2700 1 -- 172.21.15.29:0/171673059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a8164ec0 con 0x7f03a81545c0 2024-01-24T14:11:05.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.857+0000 7f039a7fc700 1 -- 172.21.15.29:0/171673059 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4011a70 con 0x7f03a81545c0 2024-01-24T14:11:05.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.857+0000 7f03b05c2700 1 -- 172.21.15.29:0/171673059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a80a4870 con 0x7f03a81545c0 2024-01-24T14:11:05.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.858+0000 7f03b05c2700 1 -- 172.21.15.29:0/171673059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a80a5070 con 0x7f03a81545c0 2024-01-24T14:11:05.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.858+0000 7f039a7fc700 1 -- 172.21.15.29:0/171673059 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f03a401c460 con 0x7f03a81545c0 2024-01-24T14:11:05.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.859+0000 7f039a7fc700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f038c05f270 0x7f038c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:05.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.859+0000 7f039a7fc700 1 -- 172.21.15.29:0/171673059 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f03a407a450 con 0x7f03a81545c0 2024-01-24T14:11:05.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.859+0000 7f03ae35e700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f038c05f270 0x7f038c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:05.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.859+0000 7f03ae35e700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f038c05f270 0x7f038c061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f03a8154340 tx=0x7f039c008040).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:05.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:05.863+0000 7f039a7fc700 1 -- 172.21.15.29:0/171673059 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f03a80a5070 con 0x7f03a81545c0 2024-01-24T14:11:06.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.090+0000 7f03b05c2700 1 -- 172.21.15.29:0/171673059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f03a80a5070 con 0x7f03a81545c0 2024-01-24T14:11:06.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.090+0000 7f039a7fc700 1 -- 172.21.15.29:0/171673059 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f03a80a5070 con 0x7f03a81545c0 2024-01-24T14:11:06.091 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-24T14:11:06.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.092+0000 7f03b05c2700 1 -- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f038c05f270 msgr2=0x7f038c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:06.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.092+0000 7f03b05c2700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f038c05f270 0x7f038c061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f03a8154340 tx=0x7f039c008040).stop 2024-01-24T14:11:06.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.092+0000 7f03b05c2700 1 -- 172.21.15.29:0/171673059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03a81545c0 msgr2=0x7f03a80a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:06.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.092+0000 7f03b05c2700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03a81545c0 0x7f03a80a5350 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f03a400d860 tx=0x7f03a4008040).stop 2024-01-24T14:11:06.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.092+0000 7f03b05c2700 1 -- 172.21.15.29:0/171673059 shutdown_connections 2024-01-24T14:11:06.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.092+0000 7f03b05c2700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a81587e0 0x7f03a8153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:06.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.092+0000 7f03b05c2700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03a814b180 0x7f03a81493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:06.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.092+0000 7f03b05c2700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f038c05f270 0x7f038c061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:06.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.092+0000 7f03b05c2700 1 --2- 172.21.15.29:0/171673059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03a81545c0 0x7f03a80a5350 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:06.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.092+0000 7f03b05c2700 1 -- 172.21.15.29:0/171673059 >> 172.21.15.29:0/171673059 conn(0x7f03a80c7b10 msgr2=0x7f03a80c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:06.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.092+0000 7f03b05c2700 1 -- 172.21.15.29:0/171673059 shutdown_connections 2024-01-24T14:11:06.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.092+0000 7f03b05c2700 1 -- 172.21.15.29:0/171673059 wait complete. 2024-01-24T14:11:06.105 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-24T14:11:06.105 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T14:11:06.105 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T14:11:06.105 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-24T14:11:06.105 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-24T14:11:06.105 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-24T14:11:06.105 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-24T14:11:06.105 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T14:11:06.105 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T14:11:06.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:11:06.106 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T14:11:06.126 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 41584sec preferred_lft 41584sec 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:11:06.127 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::24bc:1cff:fe57:e704/64 scope link 2024-01-24T14:11:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:11:06.128 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T14:11:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T14:11:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::2034:aaff:fef3:f12b/64 scope link 2024-01-24T14:11:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:11:06.128 INFO:teuthology.orchestra.run.smithi029.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T14:11:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether ba:ca:42:4a:9e:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T14:11:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::b8ca:42ff:fe4a:9e52/64 scope link 2024-01-24T14:11:06.128 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:11:06.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:11:06.129 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T14:11:06.197 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T14:11:06.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:11:06.198 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T14:11:06.262 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T14:11:06.262 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T14:11:06.263 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T14:11:06.263 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T14:11:06.263 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T14:11:06.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:11:06.264 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T14:11:06.264 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T14:11:06.264 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T14:11:06.264 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T14:11:06.264 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T14:11:06.264 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T14:11:06.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:11:06.390 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T14:11:06.410 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-24T14:11:06.410 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T14:11:06.411 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T14:11:06.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:11:06.411 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T14:11:06.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:11:06.485 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-01-24T14:11:06.547 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T14:11:06.548 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:11:06.640 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:11:06.641 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T14:11:06.663 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:11:06.663 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T14:11:06.666 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T14:11:06.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:11:06.667 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-24T14:11:06.700 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-24T14:11:06.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.881+0000 7f23200c1700 1 Processor -- start 2024-01-24T14:11:06.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.882+0000 7f23200c1700 1 -- start start 2024-01-24T14:11:06.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.882+0000 7f23200c1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2318151de0 0x7f2318155470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.882+0000 7f23200c1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f23180638c0 0x7f23180664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f23200c1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2318066990 0x7f231813d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f23200c1700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2318158800 con 0x7f23180638c0 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f23200c1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2318066e20 con 0x7f2318151de0 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f23200c1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2318157ed0 con 0x7f2318066990 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231d65c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f23180638c0 0x7f23180664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231d65c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f23180638c0 0x7f23180664a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42776/0 (socket says 172.21.15.29:42776) 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231d65c700 1 -- 172.21.15.29:0/2342072447 learned_addr learned my addr 172.21.15.29:0/2342072447 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231e65e700 1 --2- 172.21.15.29:0/2342072447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2318066990 0x7f231813d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231de5d700 1 --2- 172.21.15.29:0/2342072447 >> v2:172.21.15.163:3300/0 conn(0x7f2318151de0 0x7f2318155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231d65c700 1 -- 172.21.15.29:0/2342072447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2318066990 msgr2=0x7f231813d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231d65c700 1 --2- 172.21.15.29:0/2342072447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2318066990 0x7f231813d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:06.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231d65c700 1 -- 172.21.15.29:0/2342072447 >> v2:172.21.15.163:3300/0 conn(0x7f2318151de0 msgr2=0x7f2318155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231d65c700 1 --2- 172.21.15.29:0/2342072447 >> v2:172.21.15.163:3300/0 conn(0x7f2318151de0 0x7f2318155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231d65c700 1 -- 172.21.15.29:0/2342072447 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23181581b0 con 0x7f23180638c0 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231de5d700 1 --2- 172.21.15.29:0/2342072447 >> v2:172.21.15.163:3300/0 conn(0x7f2318151de0 0x7f2318155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231d65c700 1 --2- 172.21.15.29:0/2342072447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23180638c0 0x7f23180664a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f231805f270 tx=0x7f23080090f0).ready entity=mon.0 client_cookie=a69c7a0bbbb1e1ea server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231ce5b700 1 -- 172.21.15.29:0/2342072447 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230800b600 con 0x7f23180638c0 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231ce5b700 1 -- 172.21.15.29:0/2342072447 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230800b760 con 0x7f23180638c0 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.883+0000 7f231ce5b700 1 -- 172.21.15.29:0/2342072447 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230800bae0 con 0x7f23180638c0 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.884+0000 7f23200c1700 1 -- 172.21.15.29:0/2342072447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23180638c0 msgr2=0x7f23180664a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.884+0000 7f23200c1700 1 --2- 172.21.15.29:0/2342072447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23180638c0 0x7f23180664a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f231805f270 tx=0x7f23080090f0).stop 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.884+0000 7f23200c1700 1 -- 172.21.15.29:0/2342072447 shutdown_connections 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.884+0000 7f23200c1700 1 --2- 172.21.15.29:0/2342072447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2318066990 0x7f231813d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.884+0000 7f23200c1700 1 --2- 172.21.15.29:0/2342072447 >> v2:172.21.15.163:3300/0 conn(0x7f2318151de0 0x7f2318155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.884+0000 7f23200c1700 1 --2- 172.21.15.29:0/2342072447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23180638c0 0x7f23180664a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.884+0000 7f23200c1700 1 -- 172.21.15.29:0/2342072447 >> 172.21.15.29:0/2342072447 conn(0x7f23180c5640 msgr2=0x7f23180c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:06.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.884+0000 7f23200c1700 1 -- 172.21.15.29:0/2342072447 shutdown_connections 2024-01-24T14:11:06.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.884+0000 7f23200c1700 1 -- 172.21.15.29:0/2342072447 wait complete. 2024-01-24T14:11:06.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.885+0000 7f23200c1700 1 Processor -- start 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.886+0000 7f23200c1700 1 -- start start 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.886+0000 7f23200c1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23180638c0 0x7f2318156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.886+0000 7f23200c1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2318066990 0x7f2318156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.886+0000 7f23200c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2318151de0 0x7f23180a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.886+0000 7f23200c1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23180a5430 con 0x7f23180638c0 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.886+0000 7f23200c1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2318157370 con 0x7f2318151de0 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.886+0000 7f23200c1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23180abec0 con 0x7f2318066990 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.886+0000 7f231de5d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23180638c0 0x7f2318156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.886+0000 7f231de5d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23180638c0 0x7f2318156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42782/0 (socket says 172.21.15.29:42782) 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f231de5d700 1 -- 172.21.15.29:0/3884899715 learned_addr learned my addr 172.21.15.29:0/3884899715 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f231d65c700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2318066990 0x7f2318156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f231e65e700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2318151de0 0x7f23180a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f231de5d700 1 -- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2318066990 msgr2=0x7f2318156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:06.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f231de5d700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2318066990 0x7f2318156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f231de5d700 1 -- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2318151de0 msgr2=0x7f23180a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f231de5d700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2318151de0 0x7f23180a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f231de5d700 1 -- 172.21.15.29:0/3884899715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2308009370 con 0x7f23180638c0 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f231d65c700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2318066990 0x7f2318156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f231e65e700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2318151de0 0x7f23180a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f231de5d700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23180638c0 0x7f2318156990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f231400c880 tx=0x7f2314008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f230dffb700 1 -- 172.21.15.29:0/3884899715 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314010580 con 0x7f23180638c0 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f230dffb700 1 -- 172.21.15.29:0/3884899715 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23140106e0 con 0x7f23180638c0 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f230dffb700 1 -- 172.21.15.29:0/3884899715 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314010a60 con 0x7f23180638c0 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f23200c1700 1 -- 172.21.15.29:0/3884899715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23180a7020 con 0x7f23180638c0 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.887+0000 7f23200c1700 1 -- 172.21.15.29:0/3884899715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23180a75e0 con 0x7f23180638c0 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.889+0000 7f23200c1700 1 -- 172.21.15.29:0/3884899715 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23180a56a0 con 0x7f23180638c0 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.889+0000 7f230dffb700 1 -- 172.21.15.29:0/3884899715 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f231401b460 con 0x7f23180638c0 2024-01-24T14:11:06.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.889+0000 7f230dffb700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f22fc05f240 0x7f22fc061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:06.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.889+0000 7f230dffb700 1 -- 172.21.15.29:0/3884899715 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f231407a840 con 0x7f23180638c0 2024-01-24T14:11:06.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.889+0000 7f231d65c700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f22fc05f240 0x7f22fc061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:06.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.890+0000 7f231d65c700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f22fc05f240 0x7f22fc061700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f2308009130 tx=0x7f2308017000).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:06.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:06.893+0000 7f230dffb700 1 -- 172.21.15.29:0/3884899715 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f231404b530 con 0x7f23180638c0 2024-01-24T14:11:07.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.103+0000 7f23200c1700 1 -- 172.21.15.29:0/3884899715 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f23180475e0 con 0x7f22fc05f240 2024-01-24T14:11:07.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.105+0000 7f230dffb700 1 -- 172.21.15.29:0/3884899715 <== mgr.5524 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f23180475e0 con 0x7f22fc05f240 2024-01-24T14:11:07.106 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T14:11:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:11:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-01-24T14:11:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:11:07.106 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T14:11:07.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.107+0000 7f23200c1700 1 -- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f22fc05f240 msgr2=0x7f22fc061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:07.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.107+0000 7f23200c1700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f22fc05f240 0x7f22fc061700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f2308009130 tx=0x7f2308017000).stop 2024-01-24T14:11:07.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.107+0000 7f23200c1700 1 -- 172.21.15.29:0/3884899715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23180638c0 msgr2=0x7f2318156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:07.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.107+0000 7f23200c1700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23180638c0 0x7f2318156990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f231400c880 tx=0x7f2314008040).stop 2024-01-24T14:11:07.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.107+0000 7f23200c1700 1 -- 172.21.15.29:0/3884899715 shutdown_connections 2024-01-24T14:11:07.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.107+0000 7f23200c1700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2318151de0 0x7f23180a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.107+0000 7f23200c1700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f22fc05f240 0x7f22fc061700 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.107+0000 7f23200c1700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23180638c0 0x7f2318156990 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.107+0000 7f23200c1700 1 --2- 172.21.15.29:0/3884899715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2318066990 0x7f2318156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.107+0000 7f23200c1700 1 -- 172.21.15.29:0/3884899715 >> 172.21.15.29:0/3884899715 conn(0x7f23180c5640 msgr2=0x7f23180bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:07.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.107+0000 7f23200c1700 1 -- 172.21.15.29:0/3884899715 shutdown_connections 2024-01-24T14:11:07.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.107+0000 7f23200c1700 1 -- 172.21.15.29:0/3884899715 wait complete. 2024-01-24T14:11:07.121 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-01-24T14:11:07.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.255+0000 7f707dbd2700 1 Processor -- start 2024-01-24T14:11:07.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.255+0000 7f707dbd2700 1 -- start start 2024-01-24T14:11:07.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.256+0000 7f707dbd2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7078066c30 0x7f70780c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:07.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.256+0000 7f707dbd2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7078068900 0x7f7078152e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:07.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.256+0000 7f707dbd2700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7078068470 con 0x7f7078134e70 2024-01-24T14:11:07.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.256+0000 7f707dbd2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707814c910 con 0x7f7078066c30 2024-01-24T14:11:07.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.256+0000 7f707dbd2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7078153340 con 0x7f7078068900 2024-01-24T14:11:07.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.256+0000 7f70777fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7078134e70 0x7f7078066520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42064/0 (socket says 172.21.15.29:42064) 2024-01-24T14:11:07.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.256+0000 7f70777fe700 1 -- 172.21.15.29:0/4237554239 learned_addr learned my addr 172.21.15.29:0/4237554239 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:11:07.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f7076ffd700 1 --2- 172.21.15.29:0/4237554239 >> v2:172.21.15.163:3300/0 conn(0x7f7078066c30 0x7f70780c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:07.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f7077fff700 1 --2- 172.21.15.29:0/4237554239 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7078068900 0x7f7078152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:07.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f70767fc700 1 -- 172.21.15.29:0/4237554239 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104625518 0 0) 0x7f7078068470 con 0x7f7078134e70 2024-01-24T14:11:07.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f70767fc700 1 -- 172.21.15.29:0/4237554239 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70600033f0 con 0x7f7078134e70 2024-01-24T14:11:07.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f7076ffd700 1 -- 172.21.15.29:0/4237554239 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7078068900 msgr2=0x7f7078152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:07.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f7076ffd700 1 --2- 172.21.15.29:0/4237554239 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7078068900 0x7f7078152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f7076ffd700 1 -- 172.21.15.29:0/4237554239 >> v1:172.21.15.29:6789/0 conn(0x7f7078134e70 legacy=0x7f7078066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:07.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f7076ffd700 1 -- 172.21.15.29:0/4237554239 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70780c3860 con 0x7f7078066c30 2024-01-24T14:11:07.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f7077fff700 1 --2- 172.21.15.29:0/4237554239 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7078068900 0x7f7078152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:11:07.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f7076ffd700 1 --2- 172.21.15.29:0/4237554239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7078066c30 0x7f70780c31a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f706c009760 tx=0x7f706c009060).ready entity=mon.1 client_cookie=3c356e0f3ec4800a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:07.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f70767fc700 1 -- 172.21.15.29:0/4237554239 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c00d4a0 con 0x7f7078066c30 2024-01-24T14:11:07.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f70767fc700 1 -- 172.21.15.29:0/4237554239 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c00d600 con 0x7f7078066c30 2024-01-24T14:11:07.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.257+0000 7f70767fc700 1 -- 172.21.15.29:0/4237554239 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c00d980 con 0x7f7078066c30 2024-01-24T14:11:07.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.258+0000 7f707dbd2700 1 -- 172.21.15.29:0/4237554239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7078066c30 msgr2=0x7f70780c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:07.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.258+0000 7f707dbd2700 1 --2- 172.21.15.29:0/4237554239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7078066c30 0x7f70780c31a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f706c009760 tx=0x7f706c009060).stop 2024-01-24T14:11:07.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.258+0000 7f707dbd2700 1 -- 172.21.15.29:0/4237554239 shutdown_connections 2024-01-24T14:11:07.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.258+0000 7f707dbd2700 1 --2- 172.21.15.29:0/4237554239 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7078068900 0x7f7078152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.258+0000 7f707dbd2700 1 --2- 172.21.15.29:0/4237554239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7078066c30 0x7f70780c31a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.258+0000 7f707dbd2700 1 -- 172.21.15.29:0/4237554239 >> 172.21.15.29:0/4237554239 conn(0x7f70780c8ba0 msgr2=0x7f70780c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:07.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.258+0000 7f707dbd2700 1 -- 172.21.15.29:0/4237554239 shutdown_connections 2024-01-24T14:11:07.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.258+0000 7f707dbd2700 1 -- 172.21.15.29:0/4237554239 wait complete. 2024-01-24T14:11:07.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.260+0000 7f707dbd2700 1 Processor -- start 2024-01-24T14:11:07.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.260+0000 7f707dbd2700 1 -- start start 2024-01-24T14:11:07.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.260+0000 7f707dbd2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7078066c30 0x7f7078161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.260+0000 7f707dbd2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7078068900 0x7f707814d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.260+0000 7f707dbd2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7078134e70 0x7f7078148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.260+0000 7f707dbd2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707814f8a0 con 0x7f7078066c30 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.260+0000 7f707dbd2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707814f410 con 0x7f7078068900 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.260+0000 7f707dbd2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70781622b0 con 0x7f7078134e70 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f70777fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7078066c30 0x7f7078161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f70777fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7078066c30 0x7f7078161dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42798/0 (socket says 172.21.15.29:42798) 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f70777fe700 1 -- 172.21.15.29:0/3842463680 learned_addr learned my addr 172.21.15.29:0/3842463680 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f7076ffd700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7078068900 0x7f707814d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f7077fff700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7078134e70 0x7f7078148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f70777fe700 1 -- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7078134e70 msgr2=0x7f7078148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f70777fe700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7078134e70 0x7f7078148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f70777fe700 1 -- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7078068900 msgr2=0x7f707814d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f70777fe700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7078068900 0x7f707814d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f70777fe700 1 -- 172.21.15.29:0/3842463680 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070009a20 con 0x7f7078066c30 2024-01-24T14:11:07.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f70777fe700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7078066c30 0x7f7078161dc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f706800c5c0 tx=0x7f7068007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:07.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f705f7fe700 1 -- 172.21.15.29:0/3842463680 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70680072e0 con 0x7f7078066c30 2024-01-24T14:11:07.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f707dbd2700 1 -- 172.21.15.29:0/3842463680 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706c0092a0 con 0x7f7078066c30 2024-01-24T14:11:07.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f705f7fe700 1 -- 172.21.15.29:0/3842463680 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7068007650 con 0x7f7078066c30 2024-01-24T14:11:07.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f707dbd2700 1 -- 172.21.15.29:0/3842463680 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707814eae0 con 0x7f7078066c30 2024-01-24T14:11:07.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.261+0000 7f705f7fe700 1 -- 172.21.15.29:0/3842463680 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068005780 con 0x7f7078066c30 2024-01-24T14:11:07.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.262+0000 7f707dbd2700 1 -- 172.21.15.29:0/3842463680 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7058005120 con 0x7f7078066c30 2024-01-24T14:11:07.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.263+0000 7f705f7fe700 1 -- 172.21.15.29:0/3842463680 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7068005bb0 con 0x7f7078066c30 2024-01-24T14:11:07.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.263+0000 7f705f7fe700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f706405f270 0x7f7064061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:07.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.263+0000 7f705f7fe700 1 -- 172.21.15.29:0/3842463680 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f706807a050 con 0x7f7078066c30 2024-01-24T14:11:07.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.264+0000 7f7076ffd700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f706405f270 0x7f7064061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:07.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.264+0000 7f7076ffd700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f706405f270 0x7f7064061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f706c009760 tx=0x7f706c016000).ready entity=mgr.5524 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:07.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.268+0000 7f705f7fe700 1 -- 172.21.15.29:0/3842463680 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f706804ace0 con 0x7f7078066c30 2024-01-24T14:11:07.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.495+0000 7f707dbd2700 1 -- 172.21.15.29:0/3842463680 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f7058005d70 con 0x7f7078066c30 2024-01-24T14:11:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.824+0000 7f705f7fe700 1 -- 172.21.15.29:0/3842463680 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 15) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f70680777e0 con 0x7f7078066c30 2024-01-24T14:11:07.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.826+0000 7f705f7fe700 1 -- 172.21.15.29:0/3842463680 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v15) v1 ==== 92+0+0 (secure 0 0 0) 0x7f7068076ec0 con 0x7f7078066c30 2024-01-24T14:11:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.828+0000 7f707dbd2700 1 -- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f706405f270 msgr2=0x7f7064061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.828+0000 7f707dbd2700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f706405f270 0x7f7064061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f706c009760 tx=0x7f706c016000).stop 2024-01-24T14:11:07.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.828+0000 7f707dbd2700 1 -- 172.21.15.29:0/3842463680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7078066c30 msgr2=0x7f7078161dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.828+0000 7f707dbd2700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7078066c30 0x7f7078161dc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f706800c5c0 tx=0x7f7068007c80).stop 2024-01-24T14:11:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.828+0000 7f707dbd2700 1 -- 172.21.15.29:0/3842463680 shutdown_connections 2024-01-24T14:11:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.828+0000 7f707dbd2700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7078134e70 0x7f7078148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.828+0000 7f707dbd2700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f706405f270 0x7f7064061730 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.828+0000 7f707dbd2700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7078066c30 0x7f7078161dc0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.828+0000 7f707dbd2700 1 --2- 172.21.15.29:0/3842463680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7078068900 0x7f707814d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.829+0000 7f707dbd2700 1 -- 172.21.15.29:0/3842463680 >> 172.21.15.29:0/3842463680 conn(0x7f70780c8ba0 msgr2=0x7f70781543f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.829+0000 7f707dbd2700 1 -- 172.21.15.29:0/3842463680 shutdown_connections 2024-01-24T14:11:07.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.829+0000 7f707dbd2700 1 -- 172.21.15.29:0/3842463680 wait complete. 2024-01-24T14:11:07.846 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-01-24T14:11:07.963 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:07.962+0000 7f1bfc234680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:11:07.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.983+0000 7fc896001700 1 Processor -- start 2024-01-24T14:11:07.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.983+0000 7fc896001700 1 -- start start 2024-01-24T14:11:07.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc896001700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc890134ea0 0x7fc8900c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:07.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc896001700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc890066bd0 0x7fc8900bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:07.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc896001700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8900686a0 con 0x7fc8900c36c0 2024-01-24T14:11:07.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc896001700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc890068b30 con 0x7fc890066bd0 2024-01-24T14:11:07.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc896001700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc890067a90 con 0x7fc890134ea0 2024-01-24T14:11:07.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc88effd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc8900c36c0 0x7fc8900664c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42066/0 (socket says 172.21.15.29:42066) 2024-01-24T14:11:07.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc88effd700 1 -- 172.21.15.29:0/3247172066 learned_addr learned my addr 172.21.15.29:0/3247172066 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:11:07.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc88f7fe700 1 --2- 172.21.15.29:0/3247172066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc890134ea0 0x7fc8900c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:07.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc88ffff700 1 --2- 172.21.15.29:0/3247172066 >> v2:172.21.15.163:3300/0 conn(0x7fc890066bd0 0x7fc8900bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:07.986 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc7d03db680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:11:07.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc88e7fc700 1 -- 172.21.15.29:0/3247172066 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224610925 0 0) 0x7fc8900686a0 con 0x7fc8900c36c0 2024-01-24T14:11:07.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc88e7fc700 1 -- 172.21.15.29:0/3247172066 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8780033f0 con 0x7fc8900c36c0 2024-01-24T14:11:07.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc88ffff700 1 -- 172.21.15.29:0/3247172066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc890134ea0 msgr2=0x7fc8900c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:07.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc88ffff700 1 --2- 172.21.15.29:0/3247172066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc890134ea0 0x7fc8900c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc88ffff700 1 -- 172.21.15.29:0/3247172066 >> v1:172.21.15.29:6789/0 conn(0x7fc8900c36c0 legacy=0x7fc8900664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:07.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc88ffff700 1 -- 172.21.15.29:0/3247172066 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc890067d70 con 0x7fc890066bd0 2024-01-24T14:11:07.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.984+0000 7fc88f7fe700 1 --2- 172.21.15.29:0/3247172066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc890134ea0 0x7fc8900c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:11:07.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.985+0000 7fc88ffff700 1 --2- 172.21.15.29:0/3247172066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc890066bd0 0x7fc8900bdb30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fc888009e60 tx=0x7fc888009780).ready entity=mon.1 client_cookie=4138b0355c8f57d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:07.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.985+0000 7fc88e7fc700 1 -- 172.21.15.29:0/3247172066 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88800e070 con 0x7fc890066bd0 2024-01-24T14:11:07.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.985+0000 7fc88e7fc700 1 -- 172.21.15.29:0/3247172066 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88800bab0 con 0x7fc890066bd0 2024-01-24T14:11:07.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.985+0000 7fc88e7fc700 1 -- 172.21.15.29:0/3247172066 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88800be30 con 0x7fc890066bd0 2024-01-24T14:11:07.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.985+0000 7fc896001700 1 -- 172.21.15.29:0/3247172066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc890066bd0 msgr2=0x7fc8900bdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.985+0000 7fc896001700 1 --2- 172.21.15.29:0/3247172066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc890066bd0 0x7fc8900bdb30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fc888009e60 tx=0x7fc888009780).stop 2024-01-24T14:11:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.985+0000 7fc896001700 1 -- 172.21.15.29:0/3247172066 shutdown_connections 2024-01-24T14:11:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.985+0000 7fc896001700 1 --2- 172.21.15.29:0/3247172066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc890066bd0 0x7fc8900bdb30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.985+0000 7fc896001700 1 --2- 172.21.15.29:0/3247172066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc890134ea0 0x7fc8900c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.985+0000 7fc896001700 1 -- 172.21.15.29:0/3247172066 >> 172.21.15.29:0/3247172066 conn(0x7fc8900c8bd0 msgr2=0x7fc8900c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.986+0000 7fc896001700 1 -- 172.21.15.29:0/3247172066 shutdown_connections 2024-01-24T14:11:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.986+0000 7fc896001700 1 -- 172.21.15.29:0/3247172066 wait complete. 2024-01-24T14:11:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.987+0000 7fc896001700 1 Processor -- start 2024-01-24T14:11:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.987+0000 7fc896001700 1 -- start start 2024-01-24T14:11:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc896001700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc890066bd0 0x7fc890134650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc896001700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8900c36c0 0x7fc890137f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc896001700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc890134ea0 0x7fc8900abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc896001700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc890134b40 con 0x7fc8900c36c0 2024-01-24T14:11:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc896001700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8900acae0 con 0x7fc890134ea0 2024-01-24T14:11:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc896001700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8900ac520 con 0x7fc890066bd0 2024-01-24T14:11:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc88effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8900c36c0 0x7fc890137f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc88effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8900c36c0 0x7fc890137f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42820/0 (socket says 172.21.15.29:42820) 2024-01-24T14:11:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc88effd700 1 -- 172.21.15.29:0/4049779937 learned_addr learned my addr 172.21.15.29:0/4049779937 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc88f7fe700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc890066bd0 0x7fc890134650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc88ffff700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc890134ea0 0x7fc8900abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc88f7fe700 1 -- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc890134ea0 msgr2=0x7fc8900abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc88f7fe700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc890134ea0 0x7fc8900abb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc88f7fe700 1 -- 172.21.15.29:0/4049779937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8900c36c0 msgr2=0x7fc890137f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc88f7fe700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8900c36c0 0x7fc890137f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc88f7fe700 1 -- 172.21.15.29:0/4049779937 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8800092a0 con 0x7fc890066bd0 2024-01-24T14:11:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc88effd700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8900c36c0 0x7fc890137f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:11:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.988+0000 7fc88ffff700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc890134ea0 0x7fc8900abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:11:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.989+0000 7fc88f7fe700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc890066bd0 0x7fc890134650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fc88000e880 tx=0x7fc880009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.989+0000 7fc8777fe700 1 -- 172.21.15.29:0/4049779937 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8800094e0 con 0x7fc890066bd0 2024-01-24T14:11:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.989+0000 7fc8777fe700 1 -- 172.21.15.29:0/4049779937 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc880012600 con 0x7fc890066bd0 2024-01-24T14:11:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.989+0000 7fc8777fe700 1 -- 172.21.15.29:0/4049779937 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc880012980 con 0x7fc890066bd0 2024-01-24T14:11:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.989+0000 7fc896001700 1 -- 172.21.15.29:0/4049779937 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc888009a20 con 0x7fc890066bd0 2024-01-24T14:11:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.989+0000 7fc896001700 1 -- 172.21.15.29:0/4049779937 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc890137760 con 0x7fc890066bd0 2024-01-24T14:11:07.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.990+0000 7fc896001700 1 -- 172.21.15.29:0/4049779937 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8900475e0 con 0x7fc890066bd0 2024-01-24T14:11:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.990+0000 7fc8777fe700 1 -- 172.21.15.29:0/4049779937 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 15) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fc880012d30 con 0x7fc890066bd0 2024-01-24T14:11:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.991+0000 7fc8777fe700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc87c05f2c0 0x7fc87c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.991+0000 7fc8777fe700 1 -- 172.21.15.29:0/4049779937 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fc88007ac10 con 0x7fc890066bd0 2024-01-24T14:11:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.991+0000 7fc88effd700 1 -- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc87c05f2c0 msgr2=0x7fc87c061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:11:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.991+0000 7fc88effd700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc87c05f2c0 0x7fc87c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:11:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:07.995+0000 7fc8777fe700 1 -- 172.21.15.29:0/4049779937 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc88004b8a0 con 0x7fc890066bd0 2024-01-24T14:11:08.028 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:08.027+0000 7f1bfc234680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:11:08.059 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:08.059+0000 7fc7d03db680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:11:08.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.178+0000 7fc896001700 1 -- 172.21.15.29:0/4049779937 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fc890068050 con 0x7fc890066bd0 2024-01-24T14:11:08.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.184+0000 7fc8777fe700 1 -- 172.21.15.29:0/4049779937 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v7)=0 v7) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc880016020 con 0x7fc890066bd0 2024-01-24T14:11:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.186+0000 7fc896001700 1 -- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc87c05f2c0 msgr2=0x7fc87c061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:11:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.186+0000 7fc896001700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc87c05f2c0 0x7fc87c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:11:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.186+0000 7fc896001700 1 -- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc890066bd0 msgr2=0x7fc890134650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.186+0000 7fc896001700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc890066bd0 0x7fc890134650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fc88000e880 tx=0x7fc880009060).stop 2024-01-24T14:11:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.186+0000 7fc896001700 1 -- 172.21.15.29:0/4049779937 shutdown_connections 2024-01-24T14:11:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.186+0000 7fc896001700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc890134ea0 0x7fc8900abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:08.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.186+0000 7fc896001700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc890066bd0 0x7fc890134650 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:08.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.186+0000 7fc896001700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc87c05f2c0 0x7fc87c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:11:08.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.186+0000 7fc896001700 1 --2- 172.21.15.29:0/4049779937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8900c36c0 0x7fc890137f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:08.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.186+0000 7fc896001700 1 -- 172.21.15.29:0/4049779937 >> 172.21.15.29:0/4049779937 conn(0x7fc8900c8bd0 msgr2=0x7fc8901535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:08.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.186+0000 7fc896001700 1 -- 172.21.15.29:0/4049779937 shutdown_connections 2024-01-24T14:11:08.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.186+0000 7fc896001700 1 -- 172.21.15.29:0/4049779937 wait complete. 2024-01-24T14:11:08.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:08.201 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-01-24T14:11:08.227 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2024-01-24T14:11:08.300 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:08.299+0000 7f1bfc234680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:11:08.384 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:08.383+0000 7fc7d03db680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:11:08.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.408+0000 7ff5d1da8700 1 Processor -- start 2024-01-24T14:11:08.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.408+0000 7ff5d1da8700 1 -- start start 2024-01-24T14:11:08.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.409+0000 7ff5d1da8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5cc151e50 0x7ff5cc0c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:08.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.409+0000 7ff5d1da8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5cc0c7f70 0x7ff5cc155420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:08.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.409+0000 7ff5d1da8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5cc064780 0x7ff5cc064c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:08.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.409+0000 7ff5d1da8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc155910 con 0x7ff5cc151e50 2024-01-24T14:11:08.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.409+0000 7ff5d1da8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc154f90 con 0x7ff5cc0c7f70 2024-01-24T14:11:08.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.409+0000 7ff5d1da8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc065100 con 0x7ff5cc064780 2024-01-24T14:11:08.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.409+0000 7ff5cb7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5cc151e50 0x7ff5cc0c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:08.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5cb7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5cc151e50 0x7ff5cc0c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42836/0 (socket says 172.21.15.29:42836) 2024-01-24T14:11:08.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5cb7fe700 1 -- 172.21.15.29:0/2390254102 learned_addr learned my addr 172.21.15.29:0/2390254102 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:08.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5cbfff700 1 --2- 172.21.15.29:0/2390254102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5cc064780 0x7ff5cc064c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:08.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5caffd700 1 --2- 172.21.15.29:0/2390254102 >> v2:172.21.15.163:3300/0 conn(0x7ff5cc0c7f70 0x7ff5cc155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:08.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5cb7fe700 1 -- 172.21.15.29:0/2390254102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5cc064780 msgr2=0x7ff5cc064c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:08.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5cb7fe700 1 --2- 172.21.15.29:0/2390254102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5cc064780 0x7ff5cc064c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:08.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5cb7fe700 1 -- 172.21.15.29:0/2390254102 >> v2:172.21.15.163:3300/0 conn(0x7ff5cc0c7f70 msgr2=0x7ff5cc155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:08.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5cb7fe700 1 --2- 172.21.15.29:0/2390254102 >> v2:172.21.15.163:3300/0 conn(0x7ff5cc0c7f70 0x7ff5cc155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:08.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5cb7fe700 1 -- 172.21.15.29:0/2390254102 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc0a9f10 con 0x7ff5cc151e50 2024-01-24T14:11:08.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5cb7fe700 1 --2- 172.21.15.29:0/2390254102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc151e50 0x7ff5cc0c53c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7ff5cc06a960 tx=0x7ff5bc009840).ready entity=mon.0 client_cookie=94de8cc96a6232fe server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:08.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5ca7fc700 1 -- 172.21.15.29:0/2390254102 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc00e070 con 0x7ff5cc151e50 2024-01-24T14:11:08.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5ca7fc700 1 -- 172.21.15.29:0/2390254102 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc00b970 con 0x7ff5cc151e50 2024-01-24T14:11:08.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.410+0000 7ff5ca7fc700 1 -- 172.21.15.29:0/2390254102 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc00bcf0 con 0x7ff5cc151e50 2024-01-24T14:11:08.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.411+0000 7ff5d1da8700 1 -- 172.21.15.29:0/2390254102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc151e50 msgr2=0x7ff5cc0c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:08.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.411+0000 7ff5d1da8700 1 --2- 172.21.15.29:0/2390254102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc151e50 0x7ff5cc0c53c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7ff5cc06a960 tx=0x7ff5bc009840).stop 2024-01-24T14:11:08.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.411+0000 7ff5d1da8700 1 -- 172.21.15.29:0/2390254102 shutdown_connections 2024-01-24T14:11:08.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.411+0000 7ff5d1da8700 1 --2- 172.21.15.29:0/2390254102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5cc064780 0x7ff5cc064c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:08.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.411+0000 7ff5d1da8700 1 --2- 172.21.15.29:0/2390254102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc151e50 0x7ff5cc0c53c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:08.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.411+0000 7ff5d1da8700 1 --2- 172.21.15.29:0/2390254102 >> v2:172.21.15.163:3300/0 conn(0x7ff5cc0c7f70 0x7ff5cc155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:08.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.411+0000 7ff5d1da8700 1 -- 172.21.15.29:0/2390254102 >> 172.21.15.29:0/2390254102 conn(0x7ff5cc0cb880 msgr2=0x7ff5cc0c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:08.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.411+0000 7ff5d1da8700 1 -- 172.21.15.29:0/2390254102 shutdown_connections 2024-01-24T14:11:08.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.411+0000 7ff5d1da8700 1 -- 172.21.15.29:0/2390254102 wait complete. 2024-01-24T14:11:08.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.412+0000 7ff5d1da8700 1 Processor -- start 2024-01-24T14:11:08.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.412+0000 7ff5d1da8700 1 -- start start 2024-01-24T14:11:08.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.413+0000 7ff5d1da8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5cc064780 0x7ff5cc0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:08.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.413+0000 7ff5d1da8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc0c7f70 0x7ff5cc142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:08.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.413+0000 7ff5d1da8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5cc151e50 0x7ff5cc0c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:08.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.413+0000 7ff5d1da8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc0af090 con 0x7ff5cc0c7f70 2024-01-24T14:11:08.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.413+0000 7ff5d1da8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc0ae780 con 0x7ff5cc151e50 2024-01-24T14:11:08.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.413+0000 7ff5d1da8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1511b0 con 0x7ff5cc064780 2024-01-24T14:11:08.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.413+0000 7ff5caffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc0c7f70 0x7ff5cc142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:08.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.413+0000 7ff5caffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc0c7f70 0x7ff5cc142670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42844/0 (socket says 172.21.15.29:42844) 2024-01-24T14:11:08.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.413+0000 7ff5caffd700 1 -- 172.21.15.29:0/769871121 learned_addr learned my addr 172.21.15.29:0/769871121 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:08.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.413+0000 7ff5cbfff700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5cc151e50 0x7ff5cc0c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:08.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.413+0000 7ff5cb7fe700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5cc064780 0x7ff5cc0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:08.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.414+0000 7ff5caffd700 1 -- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5cc064780 msgr2=0x7ff5cc0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:08.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.414+0000 7ff5caffd700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5cc064780 0x7ff5cc0b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:08.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.414+0000 7ff5caffd700 1 -- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5cc151e50 msgr2=0x7ff5cc0c3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:08.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.414+0000 7ff5caffd700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5cc151e50 0x7ff5cc0c3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:08.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.414+0000 7ff5caffd700 1 -- 172.21.15.29:0/769871121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc009ae0 con 0x7ff5cc0c7f70 2024-01-24T14:11:08.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.414+0000 7ff5caffd700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc0c7f70 0x7ff5cc142670 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7ff5c000e930 tx=0x7ff5c0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:08.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.414+0000 7ff5b37fe700 1 -- 172.21.15.29:0/769871121 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0012600 con 0x7ff5cc0c7f70 2024-01-24T14:11:08.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.414+0000 7ff5b37fe700 1 -- 172.21.15.29:0/769871121 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c0012760 con 0x7ff5cc0c7f70 2024-01-24T14:11:08.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.414+0000 7ff5d1da8700 1 -- 172.21.15.29:0/769871121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5cc0aea40 con 0x7ff5cc0c7f70 2024-01-24T14:11:08.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.414+0000 7ff5b37fe700 1 -- 172.21.15.29:0/769871121 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0012ae0 con 0x7ff5cc0c7f70 2024-01-24T14:11:08.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.414+0000 7ff5d1da8700 1 -- 172.21.15.29:0/769871121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5cc0b0cb0 con 0x7ff5cc0c7f70 2024-01-24T14:11:08.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.415+0000 7ff5d1da8700 1 -- 172.21.15.29:0/769871121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5cc0aa710 con 0x7ff5cc0c7f70 2024-01-24T14:11:08.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.416+0000 7ff5b37fe700 1 -- 172.21.15.29:0/769871121 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) v1 ==== 75565+0+0 (secure 0 0 0) 0x7ff5c001b400 con 0x7ff5cc0c7f70 2024-01-24T14:11:08.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.416+0000 7ff5b37fe700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 0x7ff5b8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:08.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.416+0000 7ff5cb7fe700 1 -- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 msgr2=0x7ff5b8061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:11:08.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.416+0000 7ff5b37fe700 1 -- 172.21.15.29:0/769871121 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7142+0+0 (secure 0 0 0) 0x7ff5c007ac10 con 0x7ff5cc0c7f70 2024-01-24T14:11:08.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.416+0000 7ff5cb7fe700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 0x7ff5b8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:11:08.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.425+0000 7ff5b37fe700 1 -- 172.21.15.29:0/769871121 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff5c004b900 con 0x7ff5cc0c7f70 2024-01-24T14:11:08.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.617+0000 7ff5cb7fe700 1 -- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 msgr2=0x7ff5b8061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:11:08.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.617+0000 7ff5cb7fe700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 0x7ff5b8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:11:08.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:08.633+0000 7ff5d1da8700 1 -- 172.21.15.29:0/769871121 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5cc083310 con 0x7ff5b805f2c0 2024-01-24T14:11:08.914 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:08.913+0000 7f1bfc234680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:11:08.986 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:08.985+0000 7f1bfc234680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:11:09.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:09.018+0000 7ff5cb7fe700 1 -- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 msgr2=0x7ff5b8061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:11:09.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:09.018+0000 7ff5cb7fe700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 0x7ff5b8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-01-24T14:11:09.111 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:09.111+0000 7fc7d03db680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:11:09.173 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:09.172+0000 7f1bfc234680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:11:09.186 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:09.186+0000 7fc7d03db680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:11:09.400 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:09.400+0000 7fc7d03db680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:11:09.590 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:09.589+0000 7f1bfc234680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:11:09.692 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:09.691+0000 7f1bfc234680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:11:09.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:09.819+0000 7ff5cb7fe700 1 -- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 msgr2=0x7ff5b8061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:11:09.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:09.819+0000 7ff5cb7fe700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 0x7ff5b8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-01-24T14:11:09.844 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:09.843+0000 7f1bfc234680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:11:09.882 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:09.881+0000 7fc7d03db680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:11:09.919 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:09.918+0000 7f1bfc234680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:11:09.985 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:09.984+0000 7f1bfc234680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:11:09.999 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:09.999+0000 7fc7d03db680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:11:10.175 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:10.175+0000 7fc7d03db680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:11:10.240 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:10.240+0000 7f1bfc234680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:11:10.261 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:10.260+0000 7fc7d03db680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:11:10.337 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:10.337+0000 7fc7d03db680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:11:10.423 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:10.423+0000 7f1bfc234680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:11:10.632 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:10.631+0000 7fc7d03db680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:11:10.842 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:10.842+0000 7fc7d03db680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:11:11.071 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:11.071+0000 7f1bfc234680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:11:11.138 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:11.139+0000 7f1bfc234680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:11:11.213 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:11.213+0000 7f1bfc234680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:11:11.357 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:11.357+0000 7f1bfc234680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:11:11.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:11.421+0000 7ff5cb7fe700 1 -- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 msgr2=0x7ff5b8061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:11:11.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:11.421+0000 7ff5cb7fe700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 0x7ff5b8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-01-24T14:11:11.425 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:11.426+0000 7f1bfc234680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:11:11.538 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:11.538+0000 7f1bfc234680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:11:11.594 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:11.594+0000 7fc7d03db680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:11:11.645 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:11.645+0000 7f1bfc234680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:11:11.672 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:11.671+0000 7fc7d03db680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:11:11.757 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:11.757+0000 7fc7d03db680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:11:11.924 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:11.923+0000 7fc7d03db680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:11:11.936 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:11.936+0000 7f1bfc234680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:11:12.004 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:12.004+0000 7fc7d03db680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:11:12.006 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:12.006+0000 7f1bfc234680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:11:12.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:12.025+0000 7ff5b37fe700 1 -- 172.21.15.29:0/769871121 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 16) v1 ==== 75287+0+0 (secure 0 0 0) 0x7ff5c0078460 con 0x7ff5cc0c7f70 2024-01-24T14:11:12.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:12.025+0000 7ff5b37fe700 1 -- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 msgr2=0x7ff5b8061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:11:12.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:12.025+0000 7ff5b37fe700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 0x7ff5b8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:11:12.080 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:12.080+0000 7f1b6ffaa700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:11:12.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:12.080+0000 7f1b6ffaa700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:11:12.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:12.080+0000 7f1b6ffaa700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:11:12.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:12.080+0000 7f1b6ffaa700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:11:12.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:12.080+0000 7f1b6ffaa700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:11:12.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:12.081+0000 7f1b72fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:11:12.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:12.081+0000 7f1b72fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:11:12.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:12.081+0000 7f1b72fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:11:12.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:12.081+0000 7f1b72fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:11:12.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:11:12.081+0000 7f1b72fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:11:12.145 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:12.144+0000 7fc7d03db680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:11:12.268 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:12.268+0000 7fc7d03db680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:11:12.604 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:12.603+0000 7fc7d03db680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:11:12.684 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:11:12.684+0000 7fc7d03db680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:11:13.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.028+0000 7ff5b37fe700 1 -- 172.21.15.29:0/769871121 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mgrmap(e 17) v1 ==== 75357+0+0 (secure 0 0 0) 0x7ff5c00787e0 con 0x7ff5cc0c7f70 2024-01-24T14:11:13.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.029+0000 7ff5b37fe700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 0x7ff5b8061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.029+0000 7ff5b37fe700 1 -- 172.21.15.29:0/769871121 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5cc083310 con 0x7ff5b805f2c0 2024-01-24T14:11:13.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.030+0000 7ff5cb7fe700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 0x7ff5b8061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.031+0000 7ff5cb7fe700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 0x7ff5b8061700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7ff5cc06a960 tx=0x7ff5bc009840).ready entity=mgr.5866 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:13.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.052+0000 7ff5b37fe700 1 -- 172.21.15.29:0/769871121 <== mgr.5866 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff5cc083310 con 0x7ff5b805f2c0 2024-01-24T14:11:13.052 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path snap_test_dir1 2024-01-24T14:11:13.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.054+0000 7ff5d1da8700 1 -- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 msgr2=0x7ff5b8061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.054+0000 7ff5d1da8700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 0x7ff5b8061700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7ff5cc06a960 tx=0x7ff5bc009840).stop 2024-01-24T14:11:13.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.055+0000 7ff5d1da8700 1 -- 172.21.15.29:0/769871121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc0c7f70 msgr2=0x7ff5cc142670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.055+0000 7ff5d1da8700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc0c7f70 0x7ff5cc142670 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7ff5c000e930 tx=0x7ff5c0009060).stop 2024-01-24T14:11:13.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.055+0000 7ff5d1da8700 1 -- 172.21.15.29:0/769871121 shutdown_connections 2024-01-24T14:11:13.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.055+0000 7ff5d1da8700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5cc151e50 0x7ff5cc0c3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.055+0000 7ff5d1da8700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5cc064780 0x7ff5cc0b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.055+0000 7ff5d1da8700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5b805f2c0 0x7ff5b8061700 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.055+0000 7ff5d1da8700 1 --2- 172.21.15.29:0/769871121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc0c7f70 0x7ff5cc142670 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.055+0000 7ff5d1da8700 1 -- 172.21.15.29:0/769871121 >> 172.21.15.29:0/769871121 conn(0x7ff5cc0cb880 msgr2=0x7ff5cc0c9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:13.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.055+0000 7ff5d1da8700 1 -- 172.21.15.29:0/769871121 shutdown_connections 2024-01-24T14:11:13.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.055+0000 7ff5d1da8700 1 -- 172.21.15.29:0/769871121 wait complete. 2024-01-24T14:11:13.071 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-01-24T14:11:13.071 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 2M 2024-01-24T14:11:13.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.209+0000 7f6257b1f700 1 Processor -- start 2024-01-24T14:11:13.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.209+0000 7f6257b1f700 1 -- start start 2024-01-24T14:11:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f6257b1f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6250150d50 0x7f6250154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f6257b1f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6250154e40 0x7f62500c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f6257b1f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62501552d0 con 0x7f6250150970 2024-01-24T14:11:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f6257b1f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6250069a50 con 0x7f6250154e40 2024-01-24T14:11:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f6257b1f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625006a5f0 con 0x7f6250150d50 2024-01-24T14:11:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f62560bc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6250154e40 0x7f62500c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f62558bb700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6250150970 0x7f6250069ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42070/0 (socket says 172.21.15.29:42070) 2024-01-24T14:11:13.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f62558bb700 1 -- 172.21.15.29:0/2553873990 learned_addr learned my addr 172.21.15.29:0/2553873990 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f62550ba700 1 --2- 172.21.15.29:0/2553873990 >> v2:172.21.15.163:3300/0 conn(0x7f6250150d50 0x7f6250154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f62560bc700 1 -- 172.21.15.29:0/2553873990 >> v2:172.21.15.163:3300/0 conn(0x7f6250150d50 msgr2=0x7f6250154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f62560bc700 1 --2- 172.21.15.29:0/2553873990 >> v2:172.21.15.163:3300/0 conn(0x7f6250150d50 0x7f6250154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f62560bc700 1 -- 172.21.15.29:0/2553873990 >> v1:172.21.15.29:6789/0 conn(0x7f6250150970 legacy=0x7f6250069ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f62560bc700 1 -- 172.21.15.29:0/2553873990 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62500c2b20 con 0x7f6250154e40 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.211+0000 7f62550ba700 1 --2- 172.21.15.29:0/2553873990 >> v2:172.21.15.163:3300/0 conn(0x7f6250150d50 0x7f6250154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.210+0000 7f62548b9700 1 -- 172.21.15.29:0/2553873990 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776369820 0 0) 0x7f624c001110 con 0x7f6250150970 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.211+0000 7f62560bc700 1 --2- 172.21.15.29:0/2553873990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6250154e40 0x7f62500c4520 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f62500a7ab0 tx=0x7f6248009780).ready entity=mon.0 client_cookie=e0fd840043d7e0eb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.211+0000 7f62548b9700 1 -- 172.21.15.29:0/2553873990 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624800e070 con 0x7f6250154e40 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.211+0000 7f62548b9700 1 -- 172.21.15.29:0/2553873990 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624800bc30 con 0x7f6250154e40 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.211+0000 7f62548b9700 1 -- 172.21.15.29:0/2553873990 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248012410 con 0x7f6250154e40 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.211+0000 7f6257b1f700 1 -- 172.21.15.29:0/2553873990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6250154e40 msgr2=0x7f62500c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.211+0000 7f6257b1f700 1 --2- 172.21.15.29:0/2553873990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6250154e40 0x7f62500c4520 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f62500a7ab0 tx=0x7f6248009780).stop 2024-01-24T14:11:13.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.211+0000 7f6257b1f700 1 -- 172.21.15.29:0/2553873990 shutdown_connections 2024-01-24T14:11:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.211+0000 7f6257b1f700 1 --2- 172.21.15.29:0/2553873990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6250154e40 0x7f62500c4520 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.211+0000 7f6257b1f700 1 --2- 172.21.15.29:0/2553873990 >> v2:172.21.15.163:3300/0 conn(0x7f6250150d50 0x7f6250154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.211+0000 7f6257b1f700 1 -- 172.21.15.29:0/2553873990 >> 172.21.15.29:0/2553873990 conn(0x7f6250067b20 msgr2=0x7f62500638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.212+0000 7f6257b1f700 1 -- 172.21.15.29:0/2553873990 shutdown_connections 2024-01-24T14:11:13.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.212+0000 7f6257b1f700 1 -- 172.21.15.29:0/2553873990 wait complete. 2024-01-24T14:11:13.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.213+0000 7f6257b1f700 1 Processor -- start 2024-01-24T14:11:13.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.213+0000 7f6257b1f700 1 -- start start 2024-01-24T14:11:13.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.213+0000 7f6257b1f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6250150970 0x7f6250143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f6257b1f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6250150d50 0x7f62500ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f6257b1f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6250154e40 0x7f62500aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f6257b1f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62500ad3c0 con 0x7f6250154e40 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f6257b1f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62500c06d0 con 0x7f6250150d50 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f6257b1f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62500bfac0 con 0x7f6250150970 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f62560bc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6250154e40 0x7f62500aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f62560bc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6250154e40 0x7f62500aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42868/0 (socket says 172.21.15.29:42868) 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f62560bc700 1 -- 172.21.15.29:0/772203609 learned_addr learned my addr 172.21.15.29:0/772203609 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f62550ba700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6250150d50 0x7f62500ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f62558bb700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6250150970 0x7f6250143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f62560bc700 1 -- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6250150970 msgr2=0x7f6250143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f62560bc700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6250150970 0x7f6250143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f62560bc700 1 -- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6250150d50 msgr2=0x7f62500ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f62560bc700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6250150d50 0x7f62500ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f62560bc700 1 -- 172.21.15.29:0/772203609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6248009a20 con 0x7f6250154e40 2024-01-24T14:11:13.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f62560bc700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6250154e40 0x7f62500aced0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f6250069050 tx=0x7f6250069b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f6245ffb700 1 -- 172.21.15.29:0/772203609 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248012410 con 0x7f6250154e40 2024-01-24T14:11:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f6245ffb700 1 -- 172.21.15.29:0/772203609 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624800b6f0 con 0x7f6250154e40 2024-01-24T14:11:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f6257b1f700 1 -- 172.21.15.29:0/772203609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62500bfda0 con 0x7f6250154e40 2024-01-24T14:11:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.214+0000 7f6245ffb700 1 -- 172.21.15.29:0/772203609 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248011de0 con 0x7f6250154e40 2024-01-24T14:11:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.215+0000 7f6257b1f700 1 -- 172.21.15.29:0/772203609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62500c0360 con 0x7f6250154e40 2024-01-24T14:11:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.216+0000 7f6257b1f700 1 -- 172.21.15.29:0/772203609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62500c0860 con 0x7f6250154e40 2024-01-24T14:11:13.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.216+0000 7f6245ffb700 1 -- 172.21.15.29:0/772203609 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f6248003040 con 0x7f6250154e40 2024-01-24T14:11:13.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.216+0000 7f6245ffb700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f623405f080 0x7f6234061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.217+0000 7f6245ffb700 1 -- 172.21.15.29:0/772203609 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f624807c550 con 0x7f6250154e40 2024-01-24T14:11:13.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.217+0000 7f62558bb700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f623405f080 0x7f6234061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.217+0000 7f62558bb700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f623405f080 0x7f6234061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f624c00c7d0 tx=0x7f624c009040).ready entity=mgr.5866 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:13.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.221+0000 7f6245ffb700 1 -- 172.21.15.29:0/772203609 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f624804d490 con 0x7f6250154e40 2024-01-24T14:11:13.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.428+0000 7f6257b1f700 1 -- 172.21.15.29:0/772203609 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7f62500475e0 con 0x7f623405f080 2024-01-24T14:11:13.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.433+0000 7f6245ffb700 1 -- 172.21.15.29:0/772203609 <== mgr.5866 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f62500475e0 con 0x7f623405f080 2024-01-24T14:11:13.433 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path snap_test_dir1 2024-01-24T14:11:13.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.435+0000 7f6257b1f700 1 -- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f623405f080 msgr2=0x7f6234061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.435+0000 7f6257b1f700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f623405f080 0x7f6234061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f624c00c7d0 tx=0x7f624c009040).stop 2024-01-24T14:11:13.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.435+0000 7f6257b1f700 1 -- 172.21.15.29:0/772203609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6250154e40 msgr2=0x7f62500aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.435+0000 7f6257b1f700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6250154e40 0x7f62500aced0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f6250069050 tx=0x7f6250069b90).stop 2024-01-24T14:11:13.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.435+0000 7f6257b1f700 1 -- 172.21.15.29:0/772203609 shutdown_connections 2024-01-24T14:11:13.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.435+0000 7f6257b1f700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f623405f080 0x7f6234061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.435+0000 7f6257b1f700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6250154e40 0x7f62500aced0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.435+0000 7f6257b1f700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6250150970 0x7f6250143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.435+0000 7f6257b1f700 1 --2- 172.21.15.29:0/772203609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6250150d50 0x7f62500ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.435+0000 7f6257b1f700 1 -- 172.21.15.29:0/772203609 >> 172.21.15.29:0/772203609 conn(0x7f6250067b20 msgr2=0x7f625014dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:13.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.435+0000 7f6257b1f700 1 -- 172.21.15.29:0/772203609 shutdown_connections 2024-01-24T14:11:13.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.435+0000 7f6257b1f700 1 -- 172.21.15.29:0/772203609 wait complete. 2024-01-24T14:11:13.449 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-01-24T14:11:13.449 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2024-01-24-14_12_00 in ~47s... 2024-01-24T14:11:13.449 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2024-01-24-14_13_00 in ~107s... 2024-01-24T14:11:13.449 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2024-01-24T14:11:13.449 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2024-01-24T14:11:13.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.583+0000 7fc6fb920700 1 Processor -- start 2024-01-24T14:11:13.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.583+0000 7fc6fb920700 1 -- start start 2024-01-24T14:11:13.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6fb920700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc6f40c3660 0x7fc6f414b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6fb920700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc6f40c4b00 0x7fc6f4154a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6fb920700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f41557e0 con 0x7fc6f414b0a0 2024-01-24T14:11:13.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6fb920700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f40c28a0 con 0x7fc6f40c4b00 2024-01-24T14:11:13.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6fb920700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f40c2b80 con 0x7fc6f40c3660 2024-01-24T14:11:13.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6f9ebd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc6f40c4b00 0x7fc6f4154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6f96bc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc6f414b0a0 0x7fc6f40c43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56514/0 (socket says 172.21.15.29:56514) 2024-01-24T14:11:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6f96bc700 1 -- 172.21.15.29:0/145952501 learned_addr learned my addr 172.21.15.29:0/145952501 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:11:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6f8ebb700 1 --2- 172.21.15.29:0/145952501 >> v2:172.21.15.163:3300/0 conn(0x7fc6f40c3660 0x7fc6f414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6f9ebd700 1 -- 172.21.15.29:0/145952501 >> v2:172.21.15.163:3300/0 conn(0x7fc6f40c3660 msgr2=0x7fc6f414b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:11:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6f9ebd700 1 --2- 172.21.15.29:0/145952501 >> v2:172.21.15.163:3300/0 conn(0x7fc6f40c3660 0x7fc6f414b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6f9ebd700 1 -- 172.21.15.29:0/145952501 >> v1:172.21.15.29:6789/0 conn(0x7fc6f414b0a0 legacy=0x7fc6f40c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.584+0000 7fc6f9ebd700 1 -- 172.21.15.29:0/145952501 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f40c2e60 con 0x7fc6f40c4b00 2024-01-24T14:11:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.585+0000 7fc6f9ebd700 1 --2- 172.21.15.29:0/145952501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f40c4b00 0x7fc6f4154a70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fc6f406a370 tx=0x7fc6ec009780).ready entity=mon.0 client_cookie=c5c43a8314ad4e37 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.585+0000 7fc6e7fff700 1 -- 172.21.15.29:0/145952501 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec00e070 con 0x7fc6f40c4b00 2024-01-24T14:11:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.585+0000 7fc6e7fff700 1 -- 172.21.15.29:0/145952501 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ec00bc00 con 0x7fc6f40c4b00 2024-01-24T14:11:13.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.585+0000 7fc6e7fff700 1 -- 172.21.15.29:0/145952501 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec012410 con 0x7fc6f40c4b00 2024-01-24T14:11:13.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.585+0000 7fc6fb920700 1 -- 172.21.15.29:0/145952501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f40c4b00 msgr2=0x7fc6f4154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.585+0000 7fc6fb920700 1 --2- 172.21.15.29:0/145952501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f40c4b00 0x7fc6f4154a70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fc6f406a370 tx=0x7fc6ec009780).stop 2024-01-24T14:11:13.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.585+0000 7fc6fb920700 1 -- 172.21.15.29:0/145952501 shutdown_connections 2024-01-24T14:11:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.585+0000 7fc6fb920700 1 --2- 172.21.15.29:0/145952501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f40c4b00 0x7fc6f4154a70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.585+0000 7fc6fb920700 1 --2- 172.21.15.29:0/145952501 >> v2:172.21.15.163:3300/0 conn(0x7fc6f40c3660 0x7fc6f414b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.585+0000 7fc6fb920700 1 -- 172.21.15.29:0/145952501 >> 172.21.15.29:0/145952501 conn(0x7fc6f4067b20 msgr2=0x7fc6f40638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.585+0000 7fc6fb920700 1 -- 172.21.15.29:0/145952501 shutdown_connections 2024-01-24T14:11:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.586+0000 7fc6fb920700 1 -- 172.21.15.29:0/145952501 wait complete. 2024-01-24T14:11:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.587+0000 7fc6fb920700 1 Processor -- start 2024-01-24T14:11:13.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.587+0000 7fc6fb920700 1 -- start start 2024-01-24T14:11:13.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6fb920700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6f40c3660 0x7fc6f41578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6fb920700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f40c4b00 0x7fc6f413da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6fb920700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6f414b0a0 0x7fc6f4165d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6fb920700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f413def0 con 0x7fc6f40c4b00 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6fb920700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f41562a0 con 0x7fc6f40c3660 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6fb920700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f413e2c0 con 0x7fc6f414b0a0 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6f8ebb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f40c4b00 0x7fc6f413da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6f8ebb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f40c4b00 0x7fc6f413da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57820/0 (socket says 172.21.15.29:57820) 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6f8ebb700 1 -- 172.21.15.29:0/3694545315 learned_addr learned my addr 172.21.15.29:0/3694545315 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6f96bc700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6f40c3660 0x7fc6f41578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6f9ebd700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6f414b0a0 0x7fc6f4165d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6f8ebb700 1 -- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6f414b0a0 msgr2=0x7fc6f4165d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6f8ebb700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6f414b0a0 0x7fc6f4165d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6f8ebb700 1 -- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6f40c3660 msgr2=0x7fc6f41578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6f8ebb700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6f40c3660 0x7fc6f41578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6f8ebb700 1 -- 172.21.15.29:0/3694545315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec009a20 con 0x7fc6f40c4b00 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.588+0000 7fc6f8ebb700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f40c4b00 0x7fc6f413da00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fc6f000d890 tx=0x7fc6f0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.589+0000 7fc6e57fa700 1 -- 172.21.15.29:0/3694545315 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0011590 con 0x7fc6f40c4b00 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.589+0000 7fc6e57fa700 1 -- 172.21.15.29:0/3694545315 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f00116f0 con 0x7fc6f40c4b00 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.589+0000 7fc6fb920700 1 -- 172.21.15.29:0/3694545315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f41564d0 con 0x7fc6f40c4b00 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.589+0000 7fc6e57fa700 1 -- 172.21.15.29:0/3694545315 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0011a70 con 0x7fc6f40c4b00 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.589+0000 7fc6fb920700 1 -- 172.21.15.29:0/3694545315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f4156a90 con 0x7fc6f40c4b00 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.590+0000 7fc6fb920700 1 -- 172.21.15.29:0/3694545315 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f41570e0 con 0x7fc6f40c4b00 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.590+0000 7fc6e57fa700 1 -- 172.21.15.29:0/3694545315 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fc6f001c460 con 0x7fc6f40c4b00 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.590+0000 7fc6e57fa700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc6d805f080 0x7fc6d8061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.590+0000 7fc6e57fa700 1 -- 172.21.15.29:0/3694545315 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fc6f007a240 con 0x7fc6f40c4b00 2024-01-24T14:11:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.591+0000 7fc6f96bc700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc6d805f080 0x7fc6d8061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:11:13.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.591+0000 7fc6f96bc700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc6d805f080 0x7fc6d8061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc6e8007fd0 tx=0x7fc6e8009040).ready entity=mgr.5866 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:11:13.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.595+0000 7fc6e57fa700 1 -- 172.21.15.29:0/3694545315 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc6f004b180 con 0x7fc6f40c4b00 2024-01-24T14:11:13.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.802+0000 7fc6fb920700 1 -- 172.21.15.29:0/3694545315 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6f4157290 con 0x7fc6d805f080 2024-01-24T14:11:13.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.805+0000 7fc6e57fa700 1 -- 172.21.15.29:0/3694545315 <== mgr.5866 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7fc6f4157290 con 0x7fc6d805f080 2024-01-24T14:11:13.805 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:11:13.806 INFO:teuthology.orchestra.run.smithi029.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2024-01-24T14:11:13.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.807+0000 7fc6fb920700 1 -- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc6d805f080 msgr2=0x7fc6d8061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.807+0000 7fc6fb920700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc6d805f080 0x7fc6d8061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc6e8007fd0 tx=0x7fc6e8009040).stop 2024-01-24T14:11:13.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.807+0000 7fc6fb920700 1 -- 172.21.15.29:0/3694545315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f40c4b00 msgr2=0x7fc6f413da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:11:13.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.807+0000 7fc6fb920700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f40c4b00 0x7fc6f413da00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fc6f000d890 tx=0x7fc6f0008040).stop 2024-01-24T14:11:13.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.807+0000 7fc6fb920700 1 -- 172.21.15.29:0/3694545315 shutdown_connections 2024-01-24T14:11:13.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.807+0000 7fc6fb920700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6f414b0a0 0x7fc6f4165d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.807+0000 7fc6fb920700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6f40c3660 0x7fc6f41578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.807+0000 7fc6fb920700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc6d805f080 0x7fc6d8061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.807+0000 7fc6fb920700 1 --2- 172.21.15.29:0/3694545315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6f40c4b00 0x7fc6f413da00 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:11:13.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.807+0000 7fc6fb920700 1 -- 172.21.15.29:0/3694545315 >> 172.21.15.29:0/3694545315 conn(0x7fc6f4067b20 msgr2=0x7fc6f41476a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:11:13.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.807+0000 7fc6fb920700 1 -- 172.21.15.29:0/3694545315 shutdown_connections 2024-01-24T14:11:13.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:11:13.808+0000 7fc6fb920700 1 -- 172.21.15.29:0/3694545315 wait complete. 2024-01-24T14:11:13.821 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2024-01-24T14:11:13.821 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2024-01-24T14:11:13.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:13.822 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:14.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:14.842 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:15.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:15.866 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:16.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:16.890 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:17.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:17.915 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:18.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:18.938 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:19.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:19.963 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:20.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:20.987 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:22.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:22.011 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:23.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:23.035 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:24.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:24.059 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:25.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:25.084 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:26.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:26.108 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:27.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:27.132 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:28.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:28.157 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:28.401 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:11:28.403 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:11:29.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:29.181 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:30.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:30.205 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:31.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:31.229 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:32.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:32.254 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:33.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:33.278 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:34.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:34.302 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:35.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:35.326 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:36.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:36.349 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:37.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:37.373 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:38.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:38.397 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:39.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:39.421 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:40.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:40.445 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:41.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:41.469 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:42.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:42.494 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:43.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:43.518 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:44.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:44.542 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:45.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:45.568 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:46.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:46.592 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:47.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:47.616 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:48.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:48.641 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:49.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:49.666 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:50.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:50.694 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:51.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:51.718 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:52.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:52.742 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:53.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:53.766 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:54.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:54.790 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:55.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:55.815 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:56.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:56.840 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:57.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:57.864 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:58.478 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:11:58.481 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:11:58.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:58.888 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:11:59.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:11:59.913 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:00.438 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:12:00.438+0000 7f1b61f8e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:12:00.438 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:12:00.438+0000 7f1b61f8e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:12:00.438 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:12:00.438+0000 7f1b61f8e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:12:00.439 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:12:00.438+0000 7f1b61f8e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:12:00.439 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:12:00.438+0000 7f1b61f8e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:12:00.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:00.938 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:00.960 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:00.961 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2024-01-24-14_12_00_UTC'] 2024-01-24T14:12:00.962 DEBUG:tasks.cephfs.test_snap_schedules:exec=1706105473.4491062, now = 1706105520.9621, timo = 47 2024-01-24T14:12:01.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:01.963 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:01.986 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:02.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:02.988 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:03.010 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:04.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:04.012 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:04.034 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:05.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:05.036 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:05.059 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:06.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:06.061 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:06.083 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:07.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:07.085 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:07.108 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:08.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:08.110 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:08.133 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:09.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:09.135 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:09.158 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:10.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:10.159 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:10.182 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:11.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:11.184 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:11.207 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:12.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:12.209 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:12.232 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:13.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:13.234 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:13.257 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:14.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:14.259 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:14.281 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:15.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:15.283 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:15.306 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:16.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:16.308 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:16.331 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:17.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:17.332 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:17.354 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:18.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:18.356 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:18.380 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:19.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:19.381 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:19.403 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:20.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:20.406 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:20.429 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:21.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:21.430 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:21.452 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:22.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:22.454 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:22.477 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:23.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:23.478 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:23.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:24.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:24.503 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:24.526 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:25.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:25.527 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:25.550 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:26.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:26.552 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:26.576 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:27.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:27.578 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:27.601 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:28.549 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:12:28.552 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:12:28.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:28.602 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:28.624 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:29.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:29.626 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:29.648 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:30.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:30.650 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:30.673 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:31.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:31.675 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:31.699 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:32.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:32.700 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:32.722 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:33.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:33.724 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:33.746 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:34.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:34.748 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:34.771 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:35.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:35.773 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:35.795 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:36.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:36.797 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:36.820 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:37.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:37.822 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:37.844 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:38.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:38.846 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:38.868 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:39.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:39.870 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:39.892 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:40.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:40.894 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:40.918 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:41.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:41.919 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:41.941 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:42.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:42.943 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:42.970 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:43.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:43.973 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:44.003 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:45.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:45.005 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:45.035 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:46.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:46.036 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:46.064 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:47.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:47.066 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:47.089 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:48.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:48.090 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:48.112 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:49.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:49.114 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:49.137 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:50.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:50.140 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:50.163 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:51.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:51.165 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:51.187 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:52.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:52.190 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:52.213 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:53.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:53.215 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:53.237 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:54.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:54.240 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:54.263 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:55.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:55.264 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:55.286 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:56.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:56.288 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:56.311 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:57.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:57.313 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:57.335 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:58.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:58.337 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:58.361 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:12:58.646 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:12:58.649 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:12:59.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:12:59.362 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:12:59.384 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:13:00.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:13:00.387 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:13:00.410 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:13:01.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:13:01.411 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:13:01.434 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:13:01.434 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_13_00_UTC 2024-01-24T14:13:01.435 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2024-01-24-14_13_00_UTC'] 2024-01-24T14:13:01.435 DEBUG:tasks.cephfs.test_snap_schedules:exec=1706105473.4491062, now = 1706105581.435236, timo = 107 2024-01-24T14:13:02.436 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2024-01-24T14:13:02.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.578+0000 7f2bc8b3c700 1 Processor -- start 2024-01-24T14:13:02.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.579+0000 7f2bc8b3c700 1 -- start start 2024-01-24T14:13:02.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.579+0000 7f2bc8b3c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2bc4153d90 0x7f2bc4154170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:02.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.579+0000 7f2bc8b3c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c2190 0x7f2bc40627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:02.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.579+0000 7f2bc8b3c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc4069360 con 0x7f2bc40c2610 2024-01-24T14:13:02.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.579+0000 7f2bc8b3c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc406a1c0 con 0x7f2bc4153d90 2024-01-24T14:13:02.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.579+0000 7f2bc8b3c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc406a650 con 0x7f2bc40c2190 2024-01-24T14:13:02.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.580+0000 7f2bc259c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2bc4153d90 0x7f2bc4154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:02.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.580+0000 7f2bbbfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c2190 0x7f2bc40627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:02.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.580+0000 7f2bc2d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2bc40c2610 0x7f2bc4065410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:37370/0 (socket says 172.21.15.29:37370) 2024-01-24T14:13:02.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.580+0000 7f2bc2d9d700 1 -- 172.21.15.29:0/190049291 learned_addr learned my addr 172.21.15.29:0/190049291 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:02.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.580+0000 7f2bc259c700 1 -- 172.21.15.29:0/190049291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c2190 msgr2=0x7f2bc40627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:02.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.580+0000 7f2bc259c700 1 --2- 172.21.15.29:0/190049291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c2190 0x7f2bc40627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:02.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.580+0000 7f2bc259c700 1 -- 172.21.15.29:0/190049291 >> v1:172.21.15.163:6789/0 conn(0x7f2bc40c2610 legacy=0x7f2bc4065410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:02.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.580+0000 7f2bc259c700 1 -- 172.21.15.29:0/190049291 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc4158b90 con 0x7f2bc4153d90 2024-01-24T14:13:02.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.580+0000 7f2bc1d9b700 1 -- 172.21.15.29:0/190049291 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057596146 0 0) 0x7f2bc4069360 con 0x7f2bc40c2610 2024-01-24T14:13:02.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.580+0000 7f2bc259c700 1 --2- 172.21.15.29:0/190049291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc4153d90 0x7f2bc4154170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f2bb4009760 tx=0x7f2bb4009060).ready entity=mon.1 client_cookie=e196e649ed79e747 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:02.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.581+0000 7f2bc1d9b700 1 -- 172.21.15.29:0/190049291 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb400f510 con 0x7f2bc4153d90 2024-01-24T14:13:02.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.581+0000 7f2bc1d9b700 1 -- 172.21.15.29:0/190049291 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb400f670 con 0x7f2bc4153d90 2024-01-24T14:13:02.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.581+0000 7f2bc1d9b700 1 -- 172.21.15.29:0/190049291 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb400f9f0 con 0x7f2bc4153d90 2024-01-24T14:13:02.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.581+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/190049291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc4153d90 msgr2=0x7f2bc4154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:02.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.581+0000 7f2bc8b3c700 1 --2- 172.21.15.29:0/190049291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc4153d90 0x7f2bc4154170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f2bb4009760 tx=0x7f2bb4009060).stop 2024-01-24T14:13:02.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.581+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/190049291 shutdown_connections 2024-01-24T14:13:02.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.581+0000 7f2bc8b3c700 1 --2- 172.21.15.29:0/190049291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc4153d90 0x7f2bc4154170 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:02.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.581+0000 7f2bc8b3c700 1 --2- 172.21.15.29:0/190049291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c2190 0x7f2bc40627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:02.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.581+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/190049291 >> 172.21.15.29:0/190049291 conn(0x7f2bc4067b20 msgr2=0x7f2bc40638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:02.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.581+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/190049291 shutdown_connections 2024-01-24T14:13:02.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.581+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/190049291 wait complete. 2024-01-24T14:13:02.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.582+0000 7f2bc8b3c700 1 Processor -- start 2024-01-24T14:13:02.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.582+0000 7f2bc8b3c700 1 -- start start 2024-01-24T14:13:02.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.583+0000 7f2bc8b3c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c2190 0x7f2bc40ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:02.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.583+0000 7f2bc8b3c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc40c2610 0x7f2bc4156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:02.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.583+0000 7f2bc8b3c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc4153d90 0x7f2bc41573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:02.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.583+0000 7f2bc8b3c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc40ab720 con 0x7f2bc40c2610 2024-01-24T14:13:02.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.583+0000 7f2bc8b3c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4157970 con 0x7f2bc4153d90 2024-01-24T14:13:02.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.583+0000 7f2bc8b3c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4157c50 con 0x7f2bc40c2190 2024-01-24T14:13:02.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.583+0000 7f2bbbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc40c2610 0x7f2bc4156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:02.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.583+0000 7f2bbbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc40c2610 0x7f2bc4156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33788/0 (socket says 172.21.15.29:33788) 2024-01-24T14:13:02.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.583+0000 7f2bbbfff700 1 -- 172.21.15.29:0/4254092135 learned_addr learned my addr 172.21.15.29:0/4254092135 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:02.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.583+0000 7f2bc259c700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c2190 0x7f2bc40ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:02.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.583+0000 7f2bc2d9d700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc4153d90 0x7f2bc41573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:02.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.584+0000 7f2bbbfff700 1 -- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c2190 msgr2=0x7f2bc40ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:02.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.584+0000 7f2bbbfff700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c2190 0x7f2bc40ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:02.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.584+0000 7f2bbbfff700 1 -- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc4153d90 msgr2=0x7f2bc41573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:02.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.584+0000 7f2bbbfff700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc4153d90 0x7f2bc41573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:02.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.584+0000 7f2bbbfff700 1 -- 172.21.15.29:0/4254092135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40092a0 con 0x7f2bc40c2610 2024-01-24T14:13:02.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.584+0000 7f2bbbfff700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc40c2610 0x7f2bc4156ee0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f2bac00e930 tx=0x7f2bac009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:02.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.584+0000 7f2bba7fc700 1 -- 172.21.15.29:0/4254092135 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac012600 con 0x7f2bc40c2610 2024-01-24T14:13:02.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.584+0000 7f2bba7fc700 1 -- 172.21.15.29:0/4254092135 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac012760 con 0x7f2bc40c2610 2024-01-24T14:13:02.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.584+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/4254092135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc4157f30 con 0x7f2bc40c2610 2024-01-24T14:13:02.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.584+0000 7f2bba7fc700 1 -- 172.21.15.29:0/4254092135 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac012ae0 con 0x7f2bc40c2610 2024-01-24T14:13:02.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.584+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/4254092135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc4152340 con 0x7f2bc40c2610 2024-01-24T14:13:02.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.585+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/4254092135 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc40ab8d0 con 0x7f2bc40c2610 2024-01-24T14:13:02.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.585+0000 7f2bba7fc700 1 -- 172.21.15.29:0/4254092135 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 20) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f2bac01b400 con 0x7f2bc40c2610 2024-01-24T14:13:02.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.586+0000 7f2bba7fc700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2ba405f2f0 0x7f2ba40617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:02.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.586+0000 7f2bba7fc700 1 -- 172.21.15.29:0/4254092135 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7458+0+0 (secure 0 0 0) 0x7f2bac07ade0 con 0x7f2bc40c2610 2024-01-24T14:13:02.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.586+0000 7f2bc259c700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2ba405f2f0 0x7f2ba40617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:02.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.587+0000 7f2bc259c700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2ba405f2f0 0x7f2ba40617b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2bb4009fd0 tx=0x7f2bb4017000).ready entity=mgr.5866 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:02.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.590+0000 7f2bba7fc700 1 -- 172.21.15.29:0/4254092135 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2bc40ab8d0 con 0x7f2bc40c2610 2024-01-24T14:13:02.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.800+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/4254092135 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bc40475e0 con 0x7f2ba405f2f0 2024-01-24T14:13:02.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.804+0000 7f2bba7fc700 1 -- 172.21.15.29:0/4254092135 <== mgr.5866 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f2bc40475e0 con 0x7f2ba405f2f0 2024-01-24T14:13:02.805 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path snap_test_dir1 2024-01-24T14:13:02.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.806+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2ba405f2f0 msgr2=0x7f2ba40617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:02.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.806+0000 7f2bc8b3c700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2ba405f2f0 0x7f2ba40617b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2bb4009fd0 tx=0x7f2bb4017000).stop 2024-01-24T14:13:02.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.806+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/4254092135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc40c2610 msgr2=0x7f2bc4156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:02.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.806+0000 7f2bc8b3c700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc40c2610 0x7f2bc4156ee0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f2bac00e930 tx=0x7f2bac009060).stop 2024-01-24T14:13:02.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.806+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/4254092135 shutdown_connections 2024-01-24T14:13:02.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.806+0000 7f2bc8b3c700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc4153d90 0x7f2bc41573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:02.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.806+0000 7f2bc8b3c700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c2190 0x7f2bc40ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:02.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.806+0000 7f2bc8b3c700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2ba405f2f0 0x7f2ba40617b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:02.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.806+0000 7f2bc8b3c700 1 --2- 172.21.15.29:0/4254092135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc40c2610 0x7f2bc4156ee0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:02.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.806+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/4254092135 >> 172.21.15.29:0/4254092135 conn(0x7f2bc4067b20 msgr2=0x7f2bc40c3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:02.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.806+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/4254092135 shutdown_connections 2024-01-24T14:13:02.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:02.806+0000 7f2bc8b3c700 1 -- 172.21.15.29:0/4254092135 wait complete. 2024-01-24T14:13:02.820 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-01-24T14:13:02.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:13:02.820 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:13:02.840 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:13:02.840 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_13_00_UTC 2024-01-24T14:13:02.840 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_12_00_UTC 2024-01-24T14:13:02.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:13:02.840 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_12_00_UTC) 2024-01-24T14:13:02.905 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_13_00_UTC 2024-01-24T14:13:02.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:13:02.906 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_13_00_UTC) 2024-01-24T14:13:02.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:13:02.974 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-01-24T14:13:03.039 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-01-24T14:13:03.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.221+0000 7fc820775700 1 Processor -- start 2024-01-24T14:13:03.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.221+0000 7fc820775700 1 -- start start 2024-01-24T14:13:03.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.222+0000 7fc820775700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc818066520 0x7fc818066960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:03.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.222+0000 7fc820775700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8180be080 con 0x7fc818157530 2024-01-24T14:13:03.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.222+0000 7fc820775700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc818067380 con 0x7fc8180be510 2024-01-24T14:13:03.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.222+0000 7fc820775700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc818065040 con 0x7fc818066520 2024-01-24T14:13:03.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.222+0000 7fc81e511700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc818157530 0x7fc818068470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57740/0 (socket says 172.21.15.29:57740) 2024-01-24T14:13:03.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.222+0000 7fc81e511700 1 -- 172.21.15.29:0/1374175063 learned_addr learned my addr 172.21.15.29:0/1374175063 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:03.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.222+0000 7fc81dd10700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc818066520 0x7fc818066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:03.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.222+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720528614 0 0) 0x7fc8180be080 con 0x7fc818157530 2024-01-24T14:13:03.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.222+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8040033f0 con 0x7fc818157530 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479030312 0 0) 0x7fc818067380 con 0x7fc8180be510 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8180be080 con 0x7fc8180be510 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3585281776 0 0) 0x7fc8040033f0 con 0x7fc818157530 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc818066520 msgr2=0x7fc818066960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc818066520 0x7fc818066960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 >> v1:172.21.15.163:6789/0 conn(0x7fc8180be510 legacy=0x7fc8180be9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818068b80 con 0x7fc818157530 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fc814001470 con 0x7fc818157530 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81dd10700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc818066520 0x7fc818066960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 >> v1:172.21.15.29:6789/0 conn(0x7fc818157530 legacy=0x7fc818068470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc804003aa0 0x7fc804003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc804004690 0x7fc804004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc818066520 0x7fc804005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818065040 con 0x7fc804004690 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80400b8e0 con 0x7fc804003aa0 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80400ba70 con 0x7fc818066520 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81dd10700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc804004690 0x7fc804004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81e511700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc804003aa0 0x7fc804003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81ed12700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc818066520 0x7fc804005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81dd10700 1 -- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc818066520 msgr2=0x7fc804005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81dd10700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc818066520 0x7fc804005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81dd10700 1 -- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc804003aa0 msgr2=0x7fc804003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81dd10700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc804003aa0 0x7fc804003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81dd10700 1 -- 172.21.15.29:0/1374175063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8080092a0 con 0x7fc804004690 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81e511700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc804003aa0 0x7fc804003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81dd10700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc804004690 0x7fc804004b10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fc808009c70 tx=0x7fc808009060).ready entity=mon.0 client_cookie=ca9bf7cce0e6e893 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:03.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8080095c0 con 0x7fc804004690 2024-01-24T14:13:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.223+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80800e690 con 0x7fc804004690 2024-01-24T14:13:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.224+0000 7fc81d50f700 1 -- 172.21.15.29:0/1374175063 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800ea10 con 0x7fc804004690 2024-01-24T14:13:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.224+0000 7fc820775700 1 -- 172.21.15.29:0/1374175063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc804004690 msgr2=0x7fc804004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.224+0000 7fc820775700 1 --2- 172.21.15.29:0/1374175063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc804004690 0x7fc804004b10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fc808009c70 tx=0x7fc808009060).stop 2024-01-24T14:13:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.224+0000 7fc81ed12700 1 -- 172.21.15.29:0/1374175063 reap_dead start 2024-01-24T14:13:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.224+0000 7fc820775700 1 -- 172.21.15.29:0/1374175063 shutdown_connections 2024-01-24T14:13:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.224+0000 7fc820775700 1 -- 172.21.15.29:0/1374175063 >> 172.21.15.29:0/1374175063 conn(0x7fc8180c8ba0 msgr2=0x7fc8180c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.224+0000 7fc820775700 1 -- 172.21.15.29:0/1374175063 shutdown_connections 2024-01-24T14:13:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.224+0000 7fc820775700 1 -- 172.21.15.29:0/1374175063 wait complete. 2024-01-24T14:13:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.226+0000 7fc820775700 1 Processor -- start 2024-01-24T14:13:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.226+0000 7fc820775700 1 -- start start 2024-01-24T14:13:03.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc820775700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc804004690 0x7fc81814aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:03.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc820775700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8180c1250 0x7fc818149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:03.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc820775700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc81814a560 0x7fc818158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:03.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc820775700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180c0840 con 0x7fc804004690 2024-01-24T14:13:03.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc820775700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180c0dc0 con 0x7fc8180c1250 2024-01-24T14:13:03.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc820775700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180a5bf0 con 0x7fc81814a560 2024-01-24T14:13:03.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc81e511700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc804004690 0x7fc81814aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:03.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc81e511700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc804004690 0x7fc81814aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33804/0 (socket says 172.21.15.29:33804) 2024-01-24T14:13:03.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc81e511700 1 -- 172.21.15.29:0/3022853601 learned_addr learned my addr 172.21.15.29:0/3022853601 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:03.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc81dd10700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8180c1250 0x7fc818149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:03.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc81ed12700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc81814a560 0x7fc818158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:03.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc81e511700 1 -- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc81814a560 msgr2=0x7fc818158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:03.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc81e511700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc81814a560 0x7fc818158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:03.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc81e511700 1 -- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8180c1250 msgr2=0x7fc818149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:03.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc81e511700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8180c1250 0x7fc818149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:03.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc81e511700 1 -- 172.21.15.29:0/3022853601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8180a5ed0 con 0x7fc804004690 2024-01-24T14:13:03.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc81dd10700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8180c1250 0x7fc818149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:03.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.227+0000 7fc81ed12700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc81814a560 0x7fc818158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:03.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.228+0000 7fc81e511700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc804004690 0x7fc81814aa70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fc81400e810 tx=0x7fc814001e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:03.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.228+0000 7fc80e7fc700 1 -- 172.21.15.29:0/3022853601 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814010520 con 0x7fc804004690 2024-01-24T14:13:03.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.228+0000 7fc80e7fc700 1 -- 172.21.15.29:0/3022853601 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc814010680 con 0x7fc804004690 2024-01-24T14:13:03.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.228+0000 7fc820775700 1 -- 172.21.15.29:0/3022853601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8180a61b0 con 0x7fc804004690 2024-01-24T14:13:03.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.228+0000 7fc80e7fc700 1 -- 172.21.15.29:0/3022853601 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814010a20 con 0x7fc804004690 2024-01-24T14:13:03.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.228+0000 7fc820775700 1 -- 172.21.15.29:0/3022853601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81814a2f0 con 0x7fc804004690 2024-01-24T14:13:03.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.229+0000 7fc820775700 1 -- 172.21.15.29:0/3022853601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8180475e0 con 0x7fc804004690 2024-01-24T14:13:03.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.229+0000 7fc80e7fc700 1 -- 172.21.15.29:0/3022853601 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 20) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fc814014430 con 0x7fc804004690 2024-01-24T14:13:03.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.230+0000 7fc80e7fc700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc7fc05f2c0 0x7fc7fc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:03.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.230+0000 7fc80e7fc700 1 -- 172.21.15.29:0/3022853601 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7458+0+0 (secure 0 0 0) 0x7fc81407a3e0 con 0x7fc804004690 2024-01-24T14:13:03.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.230+0000 7fc81dd10700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc7fc05f2c0 0x7fc7fc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:03.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.231+0000 7fc81dd10700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc7fc05f2c0 0x7fc7fc061780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc808006d50 tx=0x7fc80801a000).ready entity=mgr.5866 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:03.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.234+0000 7fc80e7fc700 1 -- 172.21.15.29:0/3022853601 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc814049b00 con 0x7fc804004690 2024-01-24T14:13:03.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.464+0000 7fc820775700 1 -- 172.21.15.29:0/3022853601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fc818068df0 con 0x7fc804004690 2024-01-24T14:13:03.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.839+0000 7fc80e7fc700 1 -- 172.21.15.29:0/3022853601 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fc814077af0 con 0x7fc804004690 2024-01-24T14:13:03.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.841+0000 7fc80e7fc700 1 -- 172.21.15.29:0/3022853601 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v21) v1 ==== 93+0+0 (secure 0 0 0) 0x7fc8140731b0 con 0x7fc804004690 2024-01-24T14:13:03.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.843+0000 7fc820775700 1 -- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc7fc05f2c0 msgr2=0x7fc7fc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:03.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.843+0000 7fc820775700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc7fc05f2c0 0x7fc7fc061780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc808006d50 tx=0x7fc80801a000).stop 2024-01-24T14:13:03.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.843+0000 7fc820775700 1 -- 172.21.15.29:0/3022853601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc804004690 msgr2=0x7fc81814aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:03.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.843+0000 7fc820775700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc804004690 0x7fc81814aa70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fc81400e810 tx=0x7fc814001e90).stop 2024-01-24T14:13:03.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.843+0000 7fc820775700 1 -- 172.21.15.29:0/3022853601 shutdown_connections 2024-01-24T14:13:03.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.843+0000 7fc820775700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc81814a560 0x7fc818158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:03.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.843+0000 7fc820775700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc7fc05f2c0 0x7fc7fc061780 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:03.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.843+0000 7fc820775700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc804004690 0x7fc81814aa70 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:03.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.843+0000 7fc820775700 1 --2- 172.21.15.29:0/3022853601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8180c1250 0x7fc818149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:03.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.843+0000 7fc820775700 1 -- 172.21.15.29:0/3022853601 >> 172.21.15.29:0/3022853601 conn(0x7fc8180c8ba0 msgr2=0x7fc81814b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:03.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.843+0000 7fc820775700 1 -- 172.21.15.29:0/3022853601 shutdown_connections 2024-01-24T14:13:03.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:03.843+0000 7fc820775700 1 -- 172.21.15.29:0/3022853601 wait complete. 2024-01-24T14:13:03.859 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T14:13:03.859 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:13:03.910 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:13:03.950 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T14:13:03.951 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-24T14:13:03.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:13:03.951 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-24T14:13:03.988 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:03.987+0000 7f40bde04680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:13:04.000 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:04.000+0000 7f0742117680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:13:04.005 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T14:13:04.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:13:04.006 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-24T14:13:04.025 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:13:04.052 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:04.051+0000 7f40bde04680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:13:04.074 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:04.073+0000 7f0742117680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:13:04.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.196+0000 7f2ea290c700 1 Processor -- start 2024-01-24T14:13:04.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.196+0000 7f2ea290c700 1 -- start start 2024-01-24T14:13:04.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.197+0000 7f2ea290c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2e9c0c5410 0x7f2e9c0aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.197+0000 7f2ea290c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c154750 0x7f2e9c0631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.197+0000 7f2ea290c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c0aa6c0 con 0x7f2e9c154370 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.197+0000 7f2ea290c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c064240 con 0x7f2e9c0c5410 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.197+0000 7f2ea290c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0cb5c0 con 0x7f2e9c154750 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.197+0000 7f2ea0ea9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c154750 0x7f2e9c0631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.197+0000 7f2e9b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2e9c0c5410 0x7f2e9c0aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.197+0000 7f2ea0ea9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c154750 0x7f2e9c0631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:45006/0 (socket says 172.21.15.29:45006) 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.197+0000 7f2ea0ea9700 1 -- 172.21.15.29:0/3999282461 learned_addr learned my addr 172.21.15.29:0/3999282461 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.198+0000 7f2ea0ea9700 1 -- 172.21.15.29:0/3999282461 >> v2:172.21.15.163:3300/0 conn(0x7f2e9c0c5410 msgr2=0x7f2e9c0aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.198+0000 7f2ea0ea9700 1 --2- 172.21.15.29:0/3999282461 >> v2:172.21.15.163:3300/0 conn(0x7f2e9c0c5410 0x7f2e9c0aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.198+0000 7f2ea0ea9700 1 -- 172.21.15.29:0/3999282461 >> v1:172.21.15.163:6789/0 conn(0x7f2e9c154370 legacy=0x7f2e9c0ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.198+0000 7f2ea0ea9700 1 -- 172.21.15.29:0/3999282461 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0cb8a0 con 0x7f2e9c154750 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.198+0000 7f2ea0ea9700 1 --2- 172.21.15.29:0/3999282461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c154750 0x7f2e9c0631d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f2e94009f20 tx=0x7f2e94009840).ready entity=mon.2 client_cookie=e361e911d4950eee server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:04.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.198+0000 7f2e9affd700 1 -- 172.21.15.29:0/3999282461 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400e070 con 0x7f2e9c154750 2024-01-24T14:13:04.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.198+0000 7f2e9affd700 1 -- 172.21.15.29:0/3999282461 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9400b9d0 con 0x7f2e9c154750 2024-01-24T14:13:04.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.198+0000 7f2e9affd700 1 -- 172.21.15.29:0/3999282461 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400bd50 con 0x7f2e9c154750 2024-01-24T14:13:04.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.199+0000 7f2ea290c700 1 -- 172.21.15.29:0/3999282461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c154750 msgr2=0x7f2e9c0631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.199+0000 7f2ea290c700 1 --2- 172.21.15.29:0/3999282461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c154750 0x7f2e9c0631d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f2e94009f20 tx=0x7f2e94009840).stop 2024-01-24T14:13:04.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.199+0000 7f2ea290c700 1 -- 172.21.15.29:0/3999282461 shutdown_connections 2024-01-24T14:13:04.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.199+0000 7f2ea290c700 1 --2- 172.21.15.29:0/3999282461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c154750 0x7f2e9c0631d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.199+0000 7f2ea290c700 1 --2- 172.21.15.29:0/3999282461 >> v2:172.21.15.163:3300/0 conn(0x7f2e9c0c5410 0x7f2e9c0aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.199+0000 7f2ea290c700 1 -- 172.21.15.29:0/3999282461 >> 172.21.15.29:0/3999282461 conn(0x7f2e9c05ec80 msgr2=0x7f2e9c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:04.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.199+0000 7f2ea290c700 1 -- 172.21.15.29:0/3999282461 shutdown_connections 2024-01-24T14:13:04.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.199+0000 7f2ea290c700 1 -- 172.21.15.29:0/3999282461 wait complete. 2024-01-24T14:13:04.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.200+0000 7f2ea290c700 1 Processor -- start 2024-01-24T14:13:04.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.201+0000 7f2ea290c700 1 -- start start 2024-01-24T14:13:04.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2ea290c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c0c5410 0x7f2e9c14b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2ea290c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e9c154370 0x7f2e9c0af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2ea290c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c154750 0x7f2e9c14d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2ea290c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0ad170 con 0x7f2e9c154370 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2ea290c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c155e50 con 0x7f2e9c154750 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2ea290c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c155890 con 0x7f2e9c0c5410 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2e9b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e9c154370 0x7f2e9c0af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2e9b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e9c154370 0x7f2e9c0af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59148/0 (socket says 172.21.15.29:59148) 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2e9bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c0c5410 0x7f2e9c14b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2e9b7fe700 1 -- 172.21.15.29:0/3013274444 learned_addr learned my addr 172.21.15.29:0/3013274444 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2ea0ea9700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c154750 0x7f2e9c14d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2e9b7fe700 1 -- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c0c5410 msgr2=0x7f2e9c14b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2e9b7fe700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c0c5410 0x7f2e9c14b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2e9b7fe700 1 -- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c154750 msgr2=0x7f2e9c14d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2e9b7fe700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c154750 0x7f2e9c14d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2e9b7fe700 1 -- 172.21.15.29:0/3013274444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e94009ae0 con 0x7f2e9c154370 2024-01-24T14:13:04.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2e9bfff700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c0c5410 0x7f2e9c14b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:04.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.202+0000 7f2e9b7fe700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e9c154370 0x7f2e9c0af0c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f2e9000e930 tx=0x7f2e90009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:04.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.203+0000 7f2e83fff700 1 -- 172.21.15.29:0/3013274444 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90012600 con 0x7f2e9c154370 2024-01-24T14:13:04.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.203+0000 7f2e83fff700 1 -- 172.21.15.29:0/3013274444 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e90012760 con 0x7f2e9c154370 2024-01-24T14:13:04.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.203+0000 7f2ea290c700 1 -- 172.21.15.29:0/3013274444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c155b70 con 0x7f2e9c154370 2024-01-24T14:13:04.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.203+0000 7f2e83fff700 1 -- 172.21.15.29:0/3013274444 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90012ae0 con 0x7f2e9c154370 2024-01-24T14:13:04.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.203+0000 7f2ea290c700 1 -- 172.21.15.29:0/3013274444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c156910 con 0x7f2e9c154370 2024-01-24T14:13:04.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.204+0000 7f2ea290c700 1 -- 172.21.15.29:0/3013274444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c156c80 con 0x7f2e9c154370 2024-01-24T14:13:04.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.205+0000 7f2e83fff700 1 -- 172.21.15.29:0/3013274444 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f2e9001b400 con 0x7f2e9c154370 2024-01-24T14:13:04.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.205+0000 7f2e83fff700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2e8805f270 0x7f2e88061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.206+0000 7f2e83fff700 1 -- 172.21.15.29:0/3013274444 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7458+0+0 (secure 0 0 0) 0x7f2e9007b280 con 0x7f2e9c154370 2024-01-24T14:13:04.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.206+0000 7f2e9bfff700 1 -- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2e8805f270 msgr2=0x7f2e88061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:04.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.206+0000 7f2e9bfff700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2e8805f270 0x7f2e88061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:04.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.210+0000 7f2e83fff700 1 -- 172.21.15.29:0/3013274444 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2e9004bef0 con 0x7f2e9c154370 2024-01-24T14:13:04.327 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:04.326+0000 7f40bde04680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:13:04.393 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:04.392+0000 7f0742117680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:13:04.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.407+0000 7f2e9bfff700 1 -- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2e8805f270 msgr2=0x7f2e88061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:04.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.407+0000 7f2e9bfff700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2e8805f270 0x7f2e88061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:13:04.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.438+0000 7f2ea290c700 1 -- 172.21.15.29:0/3013274444 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e9c14bb60 con 0x7f2e9c154370 2024-01-24T14:13:04.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.439+0000 7f2e83fff700 1 -- 172.21.15.29:0/3013274444 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f2e9004f510 con 0x7f2e9c154370 2024-01-24T14:13:04.440 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:04.440 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2239835986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2239835986},{"type":"v1","addr":"172.21.15.163:6837","nonce":2239835986}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2208246875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2208246875},{"type":"v1","addr":"172.21.15.29:6835","nonce":2208246875}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/547152226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":547152226},{"type":"v1","addr":"172.21.15.163:6836","nonce":547152226}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:10:59.693964+0000","modified":"2024-01-24T14:11:02.793474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2804469957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2804469957},{"type":"v1","addr":"172.21.15.29:6834","nonce":2804469957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-24T14:13:04.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.441+0000 7f2ea290c700 1 -- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2e8805f270 msgr2=0x7f2e88061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:04.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.441+0000 7f2ea290c700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2e8805f270 0x7f2e88061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:04.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.441+0000 7f2ea290c700 1 -- 172.21.15.29:0/3013274444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e9c154370 msgr2=0x7f2e9c0af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.441+0000 7f2ea290c700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e9c154370 0x7f2e9c0af0c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f2e9000e930 tx=0x7f2e90009060).stop 2024-01-24T14:13:04.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.441+0000 7f2ea290c700 1 -- 172.21.15.29:0/3013274444 shutdown_connections 2024-01-24T14:13:04.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.441+0000 7f2ea290c700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c154750 0x7f2e9c14d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.441+0000 7f2ea290c700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c0c5410 0x7f2e9c14b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.441+0000 7f2ea290c700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2e8805f270 0x7f2e88061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:04.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.441+0000 7f2ea290c700 1 --2- 172.21.15.29:0/3013274444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e9c154370 0x7f2e9c0af0c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.441+0000 7f2ea290c700 1 -- 172.21.15.29:0/3013274444 >> 172.21.15.29:0/3013274444 conn(0x7f2e9c05ec80 msgr2=0x7f2e9c0c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:04.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.441+0000 7f2ea290c700 1 -- 172.21.15.29:0/3013274444 shutdown_connections 2024-01-24T14:13:04.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.441+0000 7f2ea290c700 1 -- 172.21.15.29:0/3013274444 wait complete. 2024-01-24T14:13:04.443 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 26 2024-01-24T14:13:04.456 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:13:04.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.626+0000 7f31662b2700 1 Processor -- start 2024-01-24T14:13:04.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.627+0000 7f31662b2700 1 -- start start 2024-01-24T14:13:04.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.627+0000 7f31662b2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f31600c5410 0x7f31600aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.627+0000 7f31662b2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3160154750 0x7f31600631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.627+0000 7f31662b2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31600aa6c0 con 0x7f3160154370 2024-01-24T14:13:04.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.627+0000 7f31662b2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3160064240 con 0x7f31600c5410 2024-01-24T14:13:04.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.627+0000 7f31662b2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31600cb5c0 con 0x7f3160154750 2024-01-24T14:13:04.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.628+0000 7f315f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f31600c5410 0x7f31600aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.628+0000 7f316484f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3160154750 0x7f31600631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.628+0000 7f315f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f31600c5410 0x7f31600aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:38852/0 (socket says 172.21.15.29:38852) 2024-01-24T14:13:04.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.628+0000 7f315f7fe700 1 -- 172.21.15.29:0/2036599118 learned_addr learned my addr 172.21.15.29:0/2036599118 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:04.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.628+0000 7f315f7fe700 1 -- 172.21.15.29:0/2036599118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3160154750 msgr2=0x7f31600631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:04.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.628+0000 7f315f7fe700 1 --2- 172.21.15.29:0/2036599118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3160154750 0x7f31600631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.628+0000 7f315f7fe700 1 -- 172.21.15.29:0/2036599118 >> v1:172.21.15.163:6789/0 conn(0x7f3160154370 legacy=0x7f31600ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.628+0000 7f315f7fe700 1 -- 172.21.15.29:0/2036599118 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31600cb8a0 con 0x7f31600c5410 2024-01-24T14:13:04.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.628+0000 7f315effd700 1 -- 172.21.15.29:0/2036599118 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487313023 0 0) 0x7f31600aa6c0 con 0x7f3160154370 2024-01-24T14:13:04.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.628+0000 7f316484f700 1 --2- 172.21.15.29:0/2036599118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3160154750 0x7f31600631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:04.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.628+0000 7f315f7fe700 1 --2- 172.21.15.29:0/2036599118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31600c5410 0x7f31600aa0e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f3154006fd0 tx=0x7f31540090f0).ready entity=mon.1 client_cookie=b3ef68bf85d35a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:04.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.628+0000 7f315effd700 1 -- 172.21.15.29:0/2036599118 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315400b580 con 0x7f31600c5410 2024-01-24T14:13:04.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.629+0000 7f315effd700 1 -- 172.21.15.29:0/2036599118 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315400b6e0 con 0x7f31600c5410 2024-01-24T14:13:04.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.629+0000 7f315effd700 1 -- 172.21.15.29:0/2036599118 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315400ba60 con 0x7f31600c5410 2024-01-24T14:13:04.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.629+0000 7f31662b2700 1 -- 172.21.15.29:0/2036599118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31600c5410 msgr2=0x7f31600aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.629+0000 7f31662b2700 1 --2- 172.21.15.29:0/2036599118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31600c5410 0x7f31600aa0e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f3154006fd0 tx=0x7f31540090f0).stop 2024-01-24T14:13:04.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.629+0000 7f31662b2700 1 -- 172.21.15.29:0/2036599118 shutdown_connections 2024-01-24T14:13:04.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.629+0000 7f31662b2700 1 --2- 172.21.15.29:0/2036599118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3160154750 0x7f31600631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.629+0000 7f31662b2700 1 --2- 172.21.15.29:0/2036599118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31600c5410 0x7f31600aa0e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.629+0000 7f31662b2700 1 -- 172.21.15.29:0/2036599118 >> 172.21.15.29:0/2036599118 conn(0x7f316005ec80 msgr2=0x7f31600cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:04.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.629+0000 7f31662b2700 1 -- 172.21.15.29:0/2036599118 shutdown_connections 2024-01-24T14:13:04.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.629+0000 7f31662b2700 1 -- 172.21.15.29:0/2036599118 wait complete. 2024-01-24T14:13:04.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.630+0000 7f31662b2700 1 Processor -- start 2024-01-24T14:13:04.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.631+0000 7f31662b2700 1 -- start start 2024-01-24T14:13:04.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.631+0000 7f31662b2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31600c5410 0x7f31601576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.631+0000 7f31662b2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3160154370 0x7f31600c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f31662b2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3160154750 0x7f31600ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f31662b2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31600c3aa0 con 0x7f31600c5410 2024-01-24T14:13:04.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f31662b2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31601588b0 con 0x7f3160154750 2024-01-24T14:13:04.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f31662b2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3160158b90 con 0x7f3160154370 2024-01-24T14:13:04.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f315ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31600c5410 0x7f31601576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f315ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31600c5410 0x7f31601576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59152/0 (socket says 172.21.15.29:59152) 2024-01-24T14:13:04.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f315ffff700 1 -- 172.21.15.29:0/2635606369 learned_addr learned my addr 172.21.15.29:0/2635606369 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:04.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f315f7fe700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3160154370 0x7f31600c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f316484f700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3160154750 0x7f31600ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f315f7fe700 1 -- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3160154750 msgr2=0x7f31600ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f315f7fe700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3160154750 0x7f31600ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f315f7fe700 1 -- 172.21.15.29:0/2635606369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31600c5410 msgr2=0x7f31601576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f315f7fe700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31600c5410 0x7f31601576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f315f7fe700 1 -- 172.21.15.29:0/2635606369 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3158009a20 con 0x7f3160154370 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f315f7fe700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3160154370 0x7f31600c48c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f3154014000 tx=0x7f3154017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f3147fff700 1 -- 172.21.15.29:0/2635606369 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154000bc0 con 0x7f3160154370 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f3147fff700 1 -- 172.21.15.29:0/2635606369 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315400b850 con 0x7f3160154370 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.632+0000 7f31662b2700 1 -- 172.21.15.29:0/2635606369 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3154009370 con 0x7f3160154370 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.633+0000 7f3147fff700 1 -- 172.21.15.29:0/2635606369 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154000d20 con 0x7f3160154370 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.633+0000 7f31662b2700 1 -- 172.21.15.29:0/2635606369 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31600c3f60 con 0x7f3160154370 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.634+0000 7f31662b2700 1 -- 172.21.15.29:0/2635606369 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31600475e0 con 0x7f3160154370 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.634+0000 7f3147fff700 1 -- 172.21.15.29:0/2635606369 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f315401cad0 con 0x7f3160154370 2024-01-24T14:13:04.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.634+0000 7f3147fff700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f314c05f270 0x7f314c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.634+0000 7f315ffff700 1 -- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f314c05f270 msgr2=0x7f314c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:04.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.634+0000 7f3147fff700 1 -- 172.21.15.29:0/2635606369 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7458+0+0 (secure 0 0 0) 0x7f315407c9c0 con 0x7f3160154370 2024-01-24T14:13:04.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.634+0000 7f315ffff700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f314c05f270 0x7f314c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:04.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.639+0000 7f3147fff700 1 -- 172.21.15.29:0/2635606369 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f315404d5d0 con 0x7f3160154370 2024-01-24T14:13:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.824+0000 7f31662b2700 1 -- 172.21.15.29:0/2635606369 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31600cbb80 con 0x7f3160154370 2024-01-24T14:13:04.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.825+0000 7f3147fff700 1 -- 172.21.15.29:0/2635606369 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18637 (secure 0 0 0) 0x7f315402a020 con 0x7f3160154370 2024-01-24T14:13:04.826 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:04.826 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":75,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:12:13.813494+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:10:57.425223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-24T14:10:58.622652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.17","mappings":[{"from":0,"to":6}]},{"pgid":"8.21","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6833/43241":"2024-01-25T14:11:12.020309+0000","172.21.15.163:6832/43241":"2024-01-25T14:11:12.020309+0000","172.21.15.163:0/3166674027":"2024-01-25T14:11:12.020309+0000","172.21.15.163:0/2273027649":"2024-01-25T14:11:12.020309+0000","172.21.15.163:0/4130041143":"2024-01-25T14:11:12.020309+0000","172.21.15.163:0/97990424":"2024-01-25T14:11:12.020309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:13:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.827+0000 7f31662b2700 1 -- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f314c05f270 msgr2=0x7f314c061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.827+0000 7f31662b2700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f314c05f270 0x7f314c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.827+0000 7f31662b2700 1 -- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3160154370 msgr2=0x7f31600c48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.827+0000 7f31662b2700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3160154370 0x7f31600c48c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f3154014000 tx=0x7f3154017000).stop 2024-01-24T14:13:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.827+0000 7f31662b2700 1 -- 172.21.15.29:0/2635606369 shutdown_connections 2024-01-24T14:13:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.827+0000 7f31662b2700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3160154750 0x7f31600ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.827+0000 7f31662b2700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f314c05f270 0x7f314c061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.827+0000 7f31662b2700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31600c5410 0x7f31601576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.827+0000 7f31662b2700 1 --2- 172.21.15.29:0/2635606369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3160154370 0x7f31600c48c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.827+0000 7f31662b2700 1 -- 172.21.15.29:0/2635606369 >> 172.21.15.29:0/2635606369 conn(0x7f316005ec80 msgr2=0x7f31600ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.828+0000 7f31662b2700 1 -- 172.21.15.29:0/2635606369 shutdown_connections 2024-01-24T14:13:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.828+0000 7f31662b2700 1 -- 172.21.15.29:0/2635606369 wait complete. 2024-01-24T14:13:04.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T14:10:57.425223+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T14:10:58.622652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T14:13:04.843 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T14:13:04.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:04.957+0000 7f40bde04680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:13:04.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.982+0000 7f8f4ba37700 1 Processor -- start 2024-01-24T14:13:04.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.983+0000 7f8f4ba37700 1 -- start start 2024-01-24T14:13:04.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f4ba37700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8f44064760 0x7f8f44064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f4ba37700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8f440c8490 0x7f8f440a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f4ba37700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f44152240 con 0x7f8f44154700 2024-01-24T14:13:04.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f4ba37700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f441586c0 con 0x7f8f440c8490 2024-01-24T14:13:04.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f4ba37700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f440c0d50 con 0x7f8f44064760 2024-01-24T14:13:04.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f48fd2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8f440c8490 0x7f8f440a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f48fd2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8f440c8490 0x7f8f440a9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59160/0 (socket says 172.21.15.29:59160) 2024-01-24T14:13:04.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f48fd2700 1 -- 172.21.15.29:0/2373410166 learned_addr learned my addr 172.21.15.29:0/2373410166 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:04.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f497d3700 1 --2- 172.21.15.29:0/2373410166 >> v2:172.21.15.163:3300/0 conn(0x7f8f44064760 0x7f8f44064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f48fd2700 1 -- 172.21.15.29:0/2373410166 >> v2:172.21.15.163:3300/0 conn(0x7f8f44064760 msgr2=0x7f8f44064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:04.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f48fd2700 1 --2- 172.21.15.29:0/2373410166 >> v2:172.21.15.163:3300/0 conn(0x7f8f44064760 0x7f8f44064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f48fd2700 1 -- 172.21.15.29:0/2373410166 >> v1:172.21.15.163:6789/0 conn(0x7f8f44154700 legacy=0x7f8f44154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f48fd2700 1 -- 172.21.15.29:0/2373410166 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440aa670 con 0x7f8f440c8490 2024-01-24T14:13:04.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.984+0000 7f8f48fd2700 1 --2- 172.21.15.29:0/2373410166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f440c8490 0x7f8f440a9d90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8f44084450 tx=0x7f8f340090f0).ready entity=mon.0 client_cookie=7a104dbba07f71cc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:04.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.985+0000 7f8f3bfff700 1 -- 172.21.15.29:0/2373410166 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400b5b0 con 0x7f8f440c8490 2024-01-24T14:13:04.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.985+0000 7f8f3bfff700 1 -- 172.21.15.29:0/2373410166 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3400b710 con 0x7f8f440c8490 2024-01-24T14:13:04.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.985+0000 7f8f3bfff700 1 -- 172.21.15.29:0/2373410166 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400ba90 con 0x7f8f440c8490 2024-01-24T14:13:04.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.985+0000 7f8f4ba37700 1 -- 172.21.15.29:0/2373410166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f440c8490 msgr2=0x7f8f440a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.985+0000 7f8f4ba37700 1 --2- 172.21.15.29:0/2373410166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f440c8490 0x7f8f440a9d90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8f44084450 tx=0x7f8f340090f0).stop 2024-01-24T14:13:04.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.985+0000 7f8f4ba37700 1 -- 172.21.15.29:0/2373410166 shutdown_connections 2024-01-24T14:13:04.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.985+0000 7f8f4ba37700 1 --2- 172.21.15.29:0/2373410166 >> v2:172.21.15.163:3300/0 conn(0x7f8f44064760 0x7f8f44064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.985+0000 7f8f4ba37700 1 --2- 172.21.15.29:0/2373410166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f440c8490 0x7f8f440a9d90 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.985+0000 7f8f4ba37700 1 -- 172.21.15.29:0/2373410166 >> 172.21.15.29:0/2373410166 conn(0x7f8f440cb8b0 msgr2=0x7f8f440c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:04.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.985+0000 7f8f4ba37700 1 -- 172.21.15.29:0/2373410166 shutdown_connections 2024-01-24T14:13:04.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.986+0000 7f8f4ba37700 1 -- 172.21.15.29:0/2373410166 wait complete. 2024-01-24T14:13:04.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.987+0000 7f8f4ba37700 1 Processor -- start 2024-01-24T14:13:04.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.987+0000 7f8f4ba37700 1 -- start start 2024-01-24T14:13:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.988+0000 7f8f4ba37700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f44064760 0x7f8f4414f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.988+0000 7f8f4ba37700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f440c8490 0x7f8f440c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.988+0000 7f8f4ba37700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f44154700 0x7f8f441505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.988+0000 7f8f4ba37700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440afbe0 con 0x7f8f44154700 2024-01-24T14:13:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.988+0000 7f8f4ba37700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440aefd0 con 0x7f8f440c8490 2024-01-24T14:13:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.988+0000 7f8f4ba37700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440af2b0 con 0x7f8f44064760 2024-01-24T14:13:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.988+0000 7f8f49fd4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f44154700 0x7f8f441505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.988+0000 7f8f48fd2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f440c8490 0x7f8f440c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.988+0000 7f8f49fd4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f44154700 0x7f8f441505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59166/0 (socket says 172.21.15.29:59166) 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.988+0000 7f8f49fd4700 1 -- 172.21.15.29:0/3557107597 learned_addr learned my addr 172.21.15.29:0/3557107597 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.988+0000 7f8f497d3700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f44064760 0x7f8f4414f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f48fd2700 1 -- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f44064760 msgr2=0x7f8f4414f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f48fd2700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f44064760 0x7f8f4414f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f48fd2700 1 -- 172.21.15.29:0/3557107597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f44154700 msgr2=0x7f8f441505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f48fd2700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f44154700 0x7f8f441505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f48fd2700 1 -- 172.21.15.29:0/3557107597 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34009370 con 0x7f8f440c8490 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f497d3700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f44064760 0x7f8f4414f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f49fd4700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f44154700 0x7f8f441505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f48fd2700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f440c8490 0x7f8f440c45a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f8f34014000 tx=0x7f8f34017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f397fa700 1 -- 172.21.15.29:0/3557107597 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400be50 con 0x7f8f440c8490 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f397fa700 1 -- 172.21.15.29:0/3557107597 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3400b880 con 0x7f8f440c8490 2024-01-24T14:13:04.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f4ba37700 1 -- 172.21.15.29:0/3557107597 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f440af590 con 0x7f8f440c8490 2024-01-24T14:13:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f397fa700 1 -- 172.21.15.29:0/3557107597 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3401c670 con 0x7f8f440c8490 2024-01-24T14:13:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.989+0000 7f8f4ba37700 1 -- 172.21.15.29:0/3557107597 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f440b0070 con 0x7f8f440c8490 2024-01-24T14:13:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.990+0000 7f8f4ba37700 1 -- 172.21.15.29:0/3557107597 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f440475e0 con 0x7f8f440c8490 2024-01-24T14:13:04.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.990+0000 7f8f397fa700 1 -- 172.21.15.29:0/3557107597 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f8f3401ca20 con 0x7f8f440c8490 2024-01-24T14:13:04.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.991+0000 7f8f397fa700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f2805f270 0x7f8f28061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:04.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.991+0000 7f8f497d3700 1 -- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f2805f270 msgr2=0x7f8f28061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:04.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.991+0000 7f8f397fa700 1 -- 172.21.15.29:0/3557107597 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7514+0+0 (secure 0 0 0) 0x7f8f3407c430 con 0x7f8f440c8490 2024-01-24T14:13:04.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.991+0000 7f8f497d3700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f2805f270 0x7f8f28061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:04.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:04.995+0000 7f8f397fa700 1 -- 172.21.15.29:0/3557107597 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8f3404cf50 con 0x7f8f440c8490 2024-01-24T14:13:05.027 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:05.026+0000 7f40bde04680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:13:05.132 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:05.132+0000 7f0742117680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:13:05.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.192+0000 7f8f497d3700 1 -- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f2805f270 msgr2=0x7f8f28061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:05.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.192+0000 7f8f497d3700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f2805f270 0x7f8f28061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:13:05.207 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:05.207+0000 7f0742117680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:13:05.215 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:05.214+0000 7f40bde04680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:13:05.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.228+0000 7f8f4ba37700 1 -- 172.21.15.29:0/3557107597 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8f440af3f0 con 0x7f8f440c8490 2024-01-24T14:13:05.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.229+0000 7f8f397fa700 1 -- 172.21.15.29:0/3557107597 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v76) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8f440af3f0 con 0x7f8f440c8490 2024-01-24T14:13:05.229 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T14:13:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.231+0000 7f8f4ba37700 1 -- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f2805f270 msgr2=0x7f8f28061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.231+0000 7f8f4ba37700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f2805f270 0x7f8f28061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.231+0000 7f8f4ba37700 1 -- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f440c8490 msgr2=0x7f8f440c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.231+0000 7f8f4ba37700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f440c8490 0x7f8f440c45a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f8f34014000 tx=0x7f8f34017000).stop 2024-01-24T14:13:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.231+0000 7f8f4ba37700 1 -- 172.21.15.29:0/3557107597 shutdown_connections 2024-01-24T14:13:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.231+0000 7f8f4ba37700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f2805f270 0x7f8f28061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.231+0000 7f8f4ba37700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f44154700 0x7f8f441505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.231+0000 7f8f4ba37700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f44064760 0x7f8f4414f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.231+0000 7f8f4ba37700 1 --2- 172.21.15.29:0/3557107597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f440c8490 0x7f8f440c45a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.231+0000 7f8f4ba37700 1 -- 172.21.15.29:0/3557107597 >> 172.21.15.29:0/3557107597 conn(0x7f8f440cb8b0 msgr2=0x7f8f44152f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.231+0000 7f8f4ba37700 1 -- 172.21.15.29:0/3557107597 shutdown_connections 2024-01-24T14:13:05.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.231+0000 7f8f4ba37700 1 -- 172.21.15.29:0/3557107597 wait complete. 2024-01-24T14:13:05.245 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T14:13:05.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.386+0000 7f079c275700 1 Processor -- start 2024-01-24T14:13:05.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.386+0000 7f079c275700 1 -- start start 2024-01-24T14:13:05.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.387+0000 7f079c275700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0794134b30 0x7f0794134f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:05.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f079c275700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0794156740 0x7f07940c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:05.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f079c275700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07940c3920 0x7f0794152560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:05.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f079c275700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07940c3dd0 con 0x7f0794156740 2024-01-24T14:13:05.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f079c275700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0794153490 con 0x7f07940c3920 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f079c275700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07941562b0 con 0x7f0794134b30 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f0799810700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0794156740 0x7f07940c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f0799810700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0794156740 0x7f07940c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59172/0 (socket says 172.21.15.29:59172) 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f0799810700 1 -- 172.21.15.29:0/3663378324 learned_addr learned my addr 172.21.15.29:0/3663378324 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f079a011700 1 --2- 172.21.15.29:0/3663378324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0794134b30 0x7f0794134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f079a812700 1 --2- 172.21.15.29:0/3663378324 >> v2:172.21.15.163:3300/0 conn(0x7f07940c3920 0x7f0794152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f0799810700 1 -- 172.21.15.29:0/3663378324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0794134b30 msgr2=0x7f0794134f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f0799810700 1 --2- 172.21.15.29:0/3663378324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0794134b30 0x7f0794134f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f0799810700 1 -- 172.21.15.29:0/3663378324 >> v2:172.21.15.163:3300/0 conn(0x7f07940c3920 msgr2=0x7f0794152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f0799810700 1 --2- 172.21.15.29:0/3663378324 >> v2:172.21.15.163:3300/0 conn(0x7f07940c3920 0x7f0794152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f0799810700 1 -- 172.21.15.29:0/3663378324 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0794135400 con 0x7f0794156740 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f079a812700 1 --2- 172.21.15.29:0/3663378324 >> v2:172.21.15.163:3300/0 conn(0x7f07940c3920 0x7f0794152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f079a011700 1 --2- 172.21.15.29:0/3663378324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0794134b30 0x7f0794134f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.388+0000 7f0799810700 1 --2- 172.21.15.29:0/3663378324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0794156740 0x7f07940c3430 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f0794063800 tx=0x7f07900090f0).ready entity=mon.0 client_cookie=7327819380bbad80 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:05.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.389+0000 7f079900f700 1 -- 172.21.15.29:0/3663378324 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000f510 con 0x7f0794156740 2024-01-24T14:13:05.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.389+0000 7f079900f700 1 -- 172.21.15.29:0/3663378324 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079000f670 con 0x7f0794156740 2024-01-24T14:13:05.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.389+0000 7f079900f700 1 -- 172.21.15.29:0/3663378324 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000fa10 con 0x7f0794156740 2024-01-24T14:13:05.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.389+0000 7f079c275700 1 -- 172.21.15.29:0/3663378324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0794156740 msgr2=0x7f07940c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:05.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.389+0000 7f079c275700 1 --2- 172.21.15.29:0/3663378324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0794156740 0x7f07940c3430 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f0794063800 tx=0x7f07900090f0).stop 2024-01-24T14:13:05.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.389+0000 7f079c275700 1 -- 172.21.15.29:0/3663378324 shutdown_connections 2024-01-24T14:13:05.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.389+0000 7f079c275700 1 --2- 172.21.15.29:0/3663378324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0794134b30 0x7f0794134f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.389+0000 7f079c275700 1 --2- 172.21.15.29:0/3663378324 >> v2:172.21.15.163:3300/0 conn(0x7f07940c3920 0x7f0794152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.389+0000 7f079c275700 1 --2- 172.21.15.29:0/3663378324 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0794156740 0x7f07940c3430 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.389+0000 7f079c275700 1 -- 172.21.15.29:0/3663378324 >> 172.21.15.29:0/3663378324 conn(0x7f07940627b0 msgr2=0x7f0794062bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.389+0000 7f079c275700 1 -- 172.21.15.29:0/3663378324 shutdown_connections 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.389+0000 7f079c275700 1 -- 172.21.15.29:0/3663378324 wait complete. 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.391+0000 7f079c275700 1 Processor -- start 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.391+0000 7f079c275700 1 -- start start 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079c275700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07940c3920 0x7f07940aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079c275700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0794134b30 0x7f0794133010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079c275700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0794156740 0x7f079413a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079c275700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07940bdbb0 con 0x7f07940c3920 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079c275700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07940bde90 con 0x7f0794156740 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079c275700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07940be170 con 0x7f0794134b30 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079a011700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07940c3920 0x7f07940aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079a011700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07940c3920 0x7f07940aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59180/0 (socket says 172.21.15.29:59180) 2024-01-24T14:13:05.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079a011700 1 -- 172.21.15.29:0/3733691681 learned_addr learned my addr 172.21.15.29:0/3733691681 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f0799810700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0794134b30 0x7f0794133010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079a812700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0794156740 0x7f079413a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079a011700 1 -- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0794134b30 msgr2=0x7f0794133010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079a011700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0794134b30 0x7f0794133010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079a011700 1 -- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0794156740 msgr2=0x7f079413a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079a011700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0794156740 0x7f079413a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079a011700 1 -- 172.21.15.29:0/3733691681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c009a20 con 0x7f07940c3920 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079a011700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07940c3920 0x7f07940aa4a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f078400ea20 tx=0x7f0784009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f078a7fc700 1 -- 172.21.15.29:0/3733691681 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400c850 con 0x7f07940c3920 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f078a7fc700 1 -- 172.21.15.29:0/3733691681 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078400c9b0 con 0x7f07940c3920 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f079c275700 1 -- 172.21.15.29:0/3733691681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07840092a0 con 0x7f07940c3920 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.392+0000 7f078a7fc700 1 -- 172.21.15.29:0/3733691681 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400cd30 con 0x7f07940c3920 2024-01-24T14:13:05.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.393+0000 7f079c275700 1 -- 172.21.15.29:0/3733691681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0790009370 con 0x7f07940c3920 2024-01-24T14:13:05.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.394+0000 7f078a7fc700 1 -- 172.21.15.29:0/3733691681 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f0784012600 con 0x7f07940c3920 2024-01-24T14:13:05.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.394+0000 7f079c275700 1 -- 172.21.15.29:0/3733691681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07940be5c0 con 0x7f07940c3920 2024-01-24T14:13:05.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.394+0000 7f078a7fc700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f077805f270 0x7f0778061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:05.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.394+0000 7f078a7fc700 1 -- 172.21.15.29:0/3733691681 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7514+0+0 (secure 0 0 0) 0x7f078407aef0 con 0x7f07940c3920 2024-01-24T14:13:05.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.394+0000 7f0799810700 1 -- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f077805f270 msgr2=0x7f0778061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:05.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.395+0000 7f0799810700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f077805f270 0x7f0778061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:05.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.399+0000 7f078a7fc700 1 -- 172.21.15.29:0/3733691681 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f078404bad0 con 0x7f07940c3920 2024-01-24T14:13:05.435 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:05.435+0000 7f0742117680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:13:05.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.585+0000 7f079c275700 1 -- 172.21.15.29:0/3733691681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f07940475e0 con 0x7f07940c3920 2024-01-24T14:13:05.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.586+0000 7f078a7fc700 1 -- 172.21.15.29:0/3733691681 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7f078404f0f0 con 0x7f07940c3920 2024-01-24T14:13:05.586 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T14:13:05.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.587+0000 7f079c275700 1 -- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f077805f270 msgr2=0x7f0778061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:05.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.587+0000 7f079c275700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f077805f270 0x7f0778061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:05.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.587+0000 7f079c275700 1 -- 172.21.15.29:0/3733691681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07940c3920 msgr2=0x7f07940aa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:05.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.587+0000 7f079c275700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07940c3920 0x7f07940aa4a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f078400ea20 tx=0x7f0784009060).stop 2024-01-24T14:13:05.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.588+0000 7f079c275700 1 -- 172.21.15.29:0/3733691681 shutdown_connections 2024-01-24T14:13:05.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.588+0000 7f079c275700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0794156740 0x7f079413a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.588+0000 7f079c275700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f077805f270 0x7f0778061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:05.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.588+0000 7f079c275700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07940c3920 0x7f07940aa4a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.588+0000 7f079c275700 1 --2- 172.21.15.29:0/3733691681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0794134b30 0x7f0794133010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.588+0000 7f079c275700 1 -- 172.21.15.29:0/3733691681 >> 172.21.15.29:0/3733691681 conn(0x7f07940627b0 msgr2=0x7f07940c7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:05.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.588+0000 7f079c275700 1 -- 172.21.15.29:0/3733691681 shutdown_connections 2024-01-24T14:13:05.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.588+0000 7f079c275700 1 -- 172.21.15.29:0/3733691681 wait complete. 2024-01-24T14:13:05.602 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T14:13:05.640 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:05.640+0000 7f40bde04680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:13:05.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.738+0000 7fa5610df700 1 Processor -- start 2024-01-24T14:13:05.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.738+0000 7fa5610df700 1 -- start start 2024-01-24T14:13:05.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.739+0000 7fa5610df700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa55c066940 0x7fa55c0be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:05.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.739+0000 7fa5610df700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55c0686a0 con 0x7fa55c134ea0 2024-01-24T14:13:05.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.739+0000 7fa5610df700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55c0c38e0 con 0x7fa55c0664c0 2024-01-24T14:13:05.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.739+0000 7fa5610df700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55c068b30 con 0x7fa55c066940 2024-01-24T14:13:05.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.739+0000 7fa55ad9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa55c134ea0 0x7fa55c0c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57006/0 (socket says 172.21.15.29:57006) 2024-01-24T14:13:05.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.739+0000 7fa55ad9d700 1 -- 172.21.15.29:0/2527454470 learned_addr learned my addr 172.21.15.29:0/2527454470 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:05.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.740+0000 7fa55b59e700 1 --2- 172.21.15.29:0/2527454470 >> v2:172.21.15.163:3300/0 conn(0x7fa55c066940 0x7fa55c0be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:05.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.740+0000 7fa559d9b700 1 -- 172.21.15.29:0/2527454470 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855011620 0 0) 0x7fa55c0686a0 con 0x7fa55c134ea0 2024-01-24T14:13:05.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.740+0000 7fa559d9b700 1 -- 172.21.15.29:0/2527454470 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5440033f0 con 0x7fa55c134ea0 2024-01-24T14:13:05.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.740+0000 7fa55b59e700 1 -- 172.21.15.29:0/2527454470 >> v1:172.21.15.163:6789/0 conn(0x7fa55c0664c0 legacy=0x7fa55c0bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:05.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.740+0000 7fa55b59e700 1 -- 172.21.15.29:0/2527454470 >> v1:172.21.15.29:6789/0 conn(0x7fa55c134ea0 legacy=0x7fa55c0c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:05.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.740+0000 7fa55b59e700 1 -- 172.21.15.29:0/2527454470 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c066dd0 con 0x7fa55c066940 2024-01-24T14:13:05.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.740+0000 7fa55b59e700 1 --2- 172.21.15.29:0/2527454470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa55c066940 0x7fa55c0be240 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fa554009e60 tx=0x7fa554009780).ready entity=mon.1 client_cookie=a5b985ed572bb4b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:05.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.740+0000 7fa559d9b700 1 -- 172.21.15.29:0/2527454470 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55400e070 con 0x7fa55c066940 2024-01-24T14:13:05.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.740+0000 7fa559d9b700 1 -- 172.21.15.29:0/2527454470 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55400bb60 con 0x7fa55c066940 2024-01-24T14:13:05.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.740+0000 7fa559d9b700 1 -- 172.21.15.29:0/2527454470 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa554012410 con 0x7fa55c066940 2024-01-24T14:13:05.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.741+0000 7fa5610df700 1 -- 172.21.15.29:0/2527454470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa55c066940 msgr2=0x7fa55c0be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:05.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.741+0000 7fa5610df700 1 --2- 172.21.15.29:0/2527454470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa55c066940 0x7fa55c0be240 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fa554009e60 tx=0x7fa554009780).stop 2024-01-24T14:13:05.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.741+0000 7fa5610df700 1 -- 172.21.15.29:0/2527454470 shutdown_connections 2024-01-24T14:13:05.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.741+0000 7fa5610df700 1 --2- 172.21.15.29:0/2527454470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa55c066940 0x7fa55c0be240 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.741+0000 7fa5610df700 1 -- 172.21.15.29:0/2527454470 >> 172.21.15.29:0/2527454470 conn(0x7fa55c0c8bd0 msgr2=0x7fa55c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:05.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.741+0000 7fa5610df700 1 -- 172.21.15.29:0/2527454470 shutdown_connections 2024-01-24T14:13:05.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.741+0000 7fa5610df700 1 -- 172.21.15.29:0/2527454470 wait complete. 2024-01-24T14:13:05.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.742+0000 7fa5610df700 1 Processor -- start 2024-01-24T14:13:05.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.742+0000 7fa5610df700 1 -- start start 2024-01-24T14:13:05.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.743+0000 7fa5610df700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa55c0664c0 0x7fa55c134520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:05.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.743+0000 7fa5610df700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa55c066940 0x7fa55c0c0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:05.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.743+0000 7fa5610df700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa55c134ea0 0x7fa55c0aa670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:05.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.743+0000 7fa5610df700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c153060 con 0x7fa55c134ea0 2024-01-24T14:13:05.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.743+0000 7fa5610df700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c152490 con 0x7fa55c0664c0 2024-01-24T14:13:05.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.743+0000 7fa5610df700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c152730 con 0x7fa55c066940 2024-01-24T14:13:05.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.743+0000 7fa55b59e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa55c134ea0 0x7fa55c0aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:05.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.743+0000 7fa55b59e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa55c134ea0 0x7fa55c0aa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59194/0 (socket says 172.21.15.29:59194) 2024-01-24T14:13:05.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.743+0000 7fa55b59e700 1 -- 172.21.15.29:0/1725604289 learned_addr learned my addr 172.21.15.29:0/1725604289 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:05.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.743+0000 7fa55ad9d700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa55c0664c0 0x7fa55c134520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:05.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.743+0000 7fa55a59c700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa55c066940 0x7fa55c0c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:05.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa55b59e700 1 -- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa55c066940 msgr2=0x7fa55c0c0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:05.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa55b59e700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa55c066940 0x7fa55c0c0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa55b59e700 1 -- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa55c0664c0 msgr2=0x7fa55c134520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:05.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa55b59e700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa55c0664c0 0x7fa55c134520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa55b59e700 1 -- 172.21.15.29:0/1725604289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa554009a20 con 0x7fa55c134ea0 2024-01-24T14:13:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa55a59c700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa55c066940 0x7fa55c0c0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa55ad9d700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa55c0664c0 0x7fa55c134520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa55b59e700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa55c134ea0 0x7fa55c0aa670 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fa55c06c260 tx=0x7fa554009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa542ffd700 1 -- 172.21.15.29:0/1725604289 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa554012410 con 0x7fa55c134ea0 2024-01-24T14:13:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa542ffd700 1 -- 172.21.15.29:0/1725604289 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55401d630 con 0x7fa55c134ea0 2024-01-24T14:13:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa5610df700 1 -- 172.21.15.29:0/1725604289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c152a10 con 0x7fa55c134ea0 2024-01-24T14:13:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa542ffd700 1 -- 172.21.15.29:0/1725604289 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55401d9d0 con 0x7fa55c134ea0 2024-01-24T14:13:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.744+0000 7fa5610df700 1 -- 172.21.15.29:0/1725604289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c134a10 con 0x7fa55c134ea0 2024-01-24T14:13:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.745+0000 7fa5610df700 1 -- 172.21.15.29:0/1725604289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c1531d0 con 0x7fa55c134ea0 2024-01-24T14:13:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.746+0000 7fa542ffd700 1 -- 172.21.15.29:0/1725604289 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fa55401dd40 con 0x7fa55c134ea0 2024-01-24T14:13:05.749 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:05.744+0000 7f40bde04680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:13:05.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.746+0000 7fa542ffd700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa54805f240 0x7fa548061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:05.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.746+0000 7fa55ad9d700 1 -- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa54805f240 msgr2=0x7fa548061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:05.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.746+0000 7fa542ffd700 1 -- 172.21.15.29:0/1725604289 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7514+0+0 (secure 0 0 0) 0x7fa55407c740 con 0x7fa55c134ea0 2024-01-24T14:13:05.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.746+0000 7fa55ad9d700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa54805f240 0x7fa548061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:05.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.754+0000 7fa542ffd700 1 -- 172.21.15.29:0/1725604289 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa554050e10 con 0x7fa55c134ea0 2024-01-24T14:13:05.900 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:05.899+0000 7f40bde04680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:13:05.930 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:05.930+0000 7f0742117680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:13:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.939+0000 7fa5610df700 1 -- 172.21.15.29:0/1725604289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa55c0475e0 con 0x7fa55c134ea0 2024-01-24T14:13:05.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.940+0000 7fa542ffd700 1 -- 172.21.15.29:0/1725604289 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa554050a00 con 0x7fa55c134ea0 2024-01-24T14:13:05.940 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T14:13:05.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.942+0000 7fa5610df700 1 -- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa54805f240 msgr2=0x7fa548061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:05.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.942+0000 7fa5610df700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa54805f240 0x7fa548061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:05.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.942+0000 7fa5610df700 1 -- 172.21.15.29:0/1725604289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa55c134ea0 msgr2=0x7fa55c0aa670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:05.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.942+0000 7fa5610df700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa55c134ea0 0x7fa55c0aa670 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fa55c06c260 tx=0x7fa554009780).stop 2024-01-24T14:13:05.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.942+0000 7fa5610df700 1 -- 172.21.15.29:0/1725604289 shutdown_connections 2024-01-24T14:13:05.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.942+0000 7fa5610df700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa54805f240 0x7fa548061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:05.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.942+0000 7fa5610df700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa55c134ea0 0x7fa55c0aa670 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.942+0000 7fa5610df700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa55c0664c0 0x7fa55c134520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.942+0000 7fa5610df700 1 --2- 172.21.15.29:0/1725604289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa55c066940 0x7fa55c0c0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:05.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.942+0000 7fa5610df700 1 -- 172.21.15.29:0/1725604289 >> 172.21.15.29:0/1725604289 conn(0x7fa55c0c8bd0 msgr2=0x7fa55c135f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.942+0000 7fa5610df700 1 -- 172.21.15.29:0/1725604289 shutdown_connections 2024-01-24T14:13:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:05.942+0000 7fa5610df700 1 -- 172.21.15.29:0/1725604289 wait complete. 2024-01-24T14:13:05.957 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T14:13:05.975 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:05.974+0000 7f40bde04680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:13:06.043 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:06.042+0000 7f40bde04680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:13:06.063 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:06.063+0000 7f0742117680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:13:06.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.102+0000 7f7b49b69700 1 Processor -- start 2024-01-24T14:13:06.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.102+0000 7f7b49b69700 1 -- start start 2024-01-24T14:13:06.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.103+0000 7f7b49b69700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440664c0 0x7f7b44066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.103+0000 7f7b49b69700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7b440bdb30 0x7f7b440bdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.103+0000 7f7b49b69700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b440686a0 con 0x7f7b44134ea0 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.103+0000 7f7b49b69700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b44066e30 con 0x7f7b440bdb30 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.103+0000 7f7b49b69700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440c38e0 con 0x7f7b440664c0 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.103+0000 7f7b3bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440664c0 0x7f7b44066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.103+0000 7f7b43fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7b440bdb30 0x7f7b440bdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.103+0000 7f7b43fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7b440bdb30 0x7f7b440bdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:38926/0 (socket says 172.21.15.29:38926) 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.103+0000 7f7b3bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440664c0 0x7f7b44066940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:45066/0 (socket says 172.21.15.29:45066) 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.103+0000 7f7b43fff700 1 -- 172.21.15.29:0/3346658779 learned_addr learned my addr 172.21.15.29:0/3346658779 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.103+0000 7f7b437fe700 1 --1- 172.21.15.29:0/3346658779 >> v1:172.21.15.163:6789/0 conn(0x7f7b44134ea0 0x7f7b440c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:52356/0 (socket says 172.21.15.29:52356) 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.104+0000 7f7b3bfff700 1 -- 172.21.15.29:0/3346658779 >> v2:172.21.15.163:3300/0 conn(0x7f7b440bdb30 msgr2=0x7f7b440bdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.104+0000 7f7b3bfff700 1 --2- 172.21.15.29:0/3346658779 >> v2:172.21.15.163:3300/0 conn(0x7f7b440bdb30 0x7f7b440bdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.104+0000 7f7b3bfff700 1 -- 172.21.15.29:0/3346658779 >> v1:172.21.15.163:6789/0 conn(0x7f7b44134ea0 legacy=0x7f7b440c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.104+0000 7f7b42ffd700 1 -- 172.21.15.29:0/3346658779 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105025143 0 0) 0x7f7b440686a0 con 0x7f7b44134ea0 2024-01-24T14:13:06.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.104+0000 7f7b3bfff700 1 -- 172.21.15.29:0/3346658779 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b44068b30 con 0x7f7b440664c0 2024-01-24T14:13:06.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.104+0000 7f7b43fff700 1 --2- 172.21.15.29:0/3346658779 >> v2:172.21.15.163:3300/0 conn(0x7f7b440bdb30 0x7f7b440bdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:06.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.104+0000 7f7b3bfff700 1 --2- 172.21.15.29:0/3346658779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440664c0 0x7f7b44066940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7b2c009760 tx=0x7f7b2c009060).ready entity=mon.2 client_cookie=54f1bbf84111dd10 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:06.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.104+0000 7f7b42ffd700 1 -- 172.21.15.29:0/3346658779 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c00f510 con 0x7f7b440664c0 2024-01-24T14:13:06.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.104+0000 7f7b42ffd700 1 -- 172.21.15.29:0/3346658779 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c00f670 con 0x7f7b440664c0 2024-01-24T14:13:06.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.104+0000 7f7b42ffd700 1 -- 172.21.15.29:0/3346658779 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c00f9f0 con 0x7f7b440664c0 2024-01-24T14:13:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.105+0000 7f7b49b69700 1 -- 172.21.15.29:0/3346658779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440664c0 msgr2=0x7f7b44066940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.105+0000 7f7b49b69700 1 --2- 172.21.15.29:0/3346658779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440664c0 0x7f7b44066940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7b2c009760 tx=0x7f7b2c009060).stop 2024-01-24T14:13:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.105+0000 7f7b49b69700 1 -- 172.21.15.29:0/3346658779 shutdown_connections 2024-01-24T14:13:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.105+0000 7f7b49b69700 1 --2- 172.21.15.29:0/3346658779 >> v2:172.21.15.163:3300/0 conn(0x7f7b440bdb30 0x7f7b440bdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.105+0000 7f7b49b69700 1 --2- 172.21.15.29:0/3346658779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440664c0 0x7f7b44066940 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.105+0000 7f7b49b69700 1 -- 172.21.15.29:0/3346658779 >> 172.21.15.29:0/3346658779 conn(0x7f7b440c8bd0 msgr2=0x7f7b440c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.105+0000 7f7b49b69700 1 -- 172.21.15.29:0/3346658779 shutdown_connections 2024-01-24T14:13:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.105+0000 7f7b49b69700 1 -- 172.21.15.29:0/3346658779 wait complete. 2024-01-24T14:13:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.106+0000 7f7b49b69700 1 Processor -- start 2024-01-24T14:13:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.106+0000 7f7b49b69700 1 -- start start 2024-01-24T14:13:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b49b69700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b440664c0 0x7f7b440bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b49b69700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440bdb30 0x7f7b44137e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b49b69700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b44134ea0 0x7f7b44134490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b49b69700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440c0160 con 0x7f7b440664c0 2024-01-24T14:13:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b49b69700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4414bba0 con 0x7f7b44134ea0 2024-01-24T14:13:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b49b69700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b441373b0 con 0x7f7b440bdb30 2024-01-24T14:13:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b437fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b440664c0 0x7f7b440bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b437fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b440664c0 0x7f7b440bfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59202/0 (socket says 172.21.15.29:59202) 2024-01-24T14:13:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b437fe700 1 -- 172.21.15.29:0/1115924211 learned_addr learned my addr 172.21.15.29:0/1115924211 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b43fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b44134ea0 0x7f7b44134490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b3bfff700 1 --2- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440bdb30 0x7f7b44137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b3bfff700 1 -- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b44134ea0 msgr2=0x7f7b44134490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b3bfff700 1 --2- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b44134ea0 0x7f7b44134490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b3bfff700 1 -- 172.21.15.29:0/1115924211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b440664c0 msgr2=0x7f7b440bfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b3bfff700 1 --2- 172.21.15.29:0/1115924211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b440664c0 0x7f7b440bfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.107+0000 7f7b3bfff700 1 -- 172.21.15.29:0/1115924211 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c009ae0 con 0x7f7b440bdb30 2024-01-24T14:13:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.108+0000 7f7b3bfff700 1 --2- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440bdb30 0x7f7b44137e90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7b2c014000 tx=0x7f7b2c017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.108+0000 7f7b3b7fe700 1 -- 172.21.15.29:0/1115924211 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c00f9f0 con 0x7f7b440bdb30 2024-01-24T14:13:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.108+0000 7f7b3b7fe700 1 -- 172.21.15.29:0/1115924211 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c000d10 con 0x7f7b440bdb30 2024-01-24T14:13:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.108+0000 7f7b3b7fe700 1 -- 172.21.15.29:0/1115924211 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c011630 con 0x7f7b440bdb30 2024-01-24T14:13:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.108+0000 7f7b49b69700 1 -- 172.21.15.29:0/1115924211 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c0092a0 con 0x7f7b440bdb30 2024-01-24T14:13:06.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.108+0000 7f7b49b69700 1 -- 172.21.15.29:0/1115924211 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b44137970 con 0x7f7b440bdb30 2024-01-24T14:13:06.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.109+0000 7f7b49b69700 1 -- 172.21.15.29:0/1115924211 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b44137210 con 0x7f7b440bdb30 2024-01-24T14:13:06.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.109+0000 7f7b3b7fe700 1 -- 172.21.15.29:0/1115924211 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f7b2c011a20 con 0x7f7b440bdb30 2024-01-24T14:13:06.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.110+0000 7f7b3b7fe700 1 --2- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7b2405f270 0x7f7b24061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.110+0000 7f7b3b7fe700 1 -- 172.21.15.29:0/1115924211 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7514+0+0 (secure 0 0 0) 0x7f7b2c07f680 con 0x7f7b440bdb30 2024-01-24T14:13:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.110+0000 7f7b437fe700 1 -- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7b2405f270 msgr2=0x7f7b24061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.110+0000 7f7b437fe700 1 --2- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7b2405f270 0x7f7b24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:06.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.114+0000 7f7b3b7fe700 1 -- 172.21.15.29:0/1115924211 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7b44137210 con 0x7f7b440bdb30 2024-01-24T14:13:06.243 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:06.242+0000 7f0742117680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:13:06.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.296+0000 7f7b49b69700 1 -- 172.21.15.29:0/1115924211 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b44137210 con 0x7f7b440bdb30 2024-01-24T14:13:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.296+0000 7f7b3b7fe700 1 -- 172.21.15.29:0/1115924211 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b44137210 con 0x7f7b440bdb30 2024-01-24T14:13:06.297 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T14:13:06.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.298+0000 7f7b49b69700 1 -- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7b2405f270 msgr2=0x7f7b24061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:06.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.298+0000 7f7b49b69700 1 --2- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7b2405f270 0x7f7b24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:06.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.299+0000 7f7b49b69700 1 -- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440bdb30 msgr2=0x7f7b44137e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.299+0000 7f7b49b69700 1 --2- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440bdb30 0x7f7b44137e90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7b2c014000 tx=0x7f7b2c017000).stop 2024-01-24T14:13:06.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.299+0000 7f7b49b69700 1 -- 172.21.15.29:0/1115924211 shutdown_connections 2024-01-24T14:13:06.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.299+0000 7f7b49b69700 1 --2- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b44134ea0 0x7f7b44134490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.299+0000 7f7b49b69700 1 --2- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7b2405f270 0x7f7b24061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:06.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.299+0000 7f7b49b69700 1 --2- 172.21.15.29:0/1115924211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7b440664c0 0x7f7b440bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.299+0000 7f7b49b69700 1 --2- 172.21.15.29:0/1115924211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b440bdb30 0x7f7b44137e90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.299+0000 7f7b49b69700 1 -- 172.21.15.29:0/1115924211 >> 172.21.15.29:0/1115924211 conn(0x7f7b440c8bd0 msgr2=0x7f7b440c3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:06.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.299+0000 7f7b49b69700 1 -- 172.21.15.29:0/1115924211 shutdown_connections 2024-01-24T14:13:06.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.299+0000 7f7b49b69700 1 -- 172.21.15.29:0/1115924211 wait complete. 2024-01-24T14:13:06.301 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:06.300+0000 7f40bde04680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:13:06.313 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:13:06.333 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:06.332+0000 7f0742117680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:13:06.412 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:06.411+0000 7f0742117680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:13:06.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.444+0000 7fb377d06700 1 Processor -- start 2024-01-24T14:13:06.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.444+0000 7fb377d06700 1 -- start start 2024-01-24T14:13:06.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.445+0000 7fb377d06700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb370150ba0 0x7fb370150f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.445+0000 7fb377d06700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3700c3c40 con 0x7fb370066480 2024-01-24T14:13:06.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.445+0000 7fb377d06700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3700c2d90 con 0x7fb3700c40d0 2024-01-24T14:13:06.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.445+0000 7fb377d06700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3700c3030 con 0x7fb370150ba0 2024-01-24T14:13:06.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.445+0000 7fb375aa2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb370150ba0 0x7fb370150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.445+0000 7fb3762a3700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb370066480 0x7fb370069240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57010/0 (socket says 172.21.15.29:57010) 2024-01-24T14:13:06.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.445+0000 7fb3762a3700 1 -- 172.21.15.29:0/3304219189 learned_addr learned my addr 172.21.15.29:0/3304219189 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:06.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.446+0000 7fb375aa2700 1 -- 172.21.15.29:0/3304219189 >> v1:172.21.15.163:6789/0 conn(0x7fb3700c40d0 legacy=0x7fb370065d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.446+0000 7fb375aa2700 1 -- 172.21.15.29:0/3304219189 >> v1:172.21.15.29:6789/0 conn(0x7fb370066480 legacy=0x7fb370069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.446+0000 7fb374aa0700 1 -- 172.21.15.29:0/3304219189 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217962432 0 0) 0x7fb3700c3c40 con 0x7fb370066480 2024-01-24T14:13:06.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.446+0000 7fb375aa2700 1 -- 172.21.15.29:0/3304219189 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3700c3310 con 0x7fb370150ba0 2024-01-24T14:13:06.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.446+0000 7fb375aa2700 1 --2- 172.21.15.29:0/3304219189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb370150ba0 0x7fb370150f80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fb36c009760 tx=0x7fb36c009060).ready entity=mon.0 client_cookie=8e03fa951662c9b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:06.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.446+0000 7fb374aa0700 1 -- 172.21.15.29:0/3304219189 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00b580 con 0x7fb370150ba0 2024-01-24T14:13:06.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.446+0000 7fb374aa0700 1 -- 172.21.15.29:0/3304219189 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36c00b6e0 con 0x7fb370150ba0 2024-01-24T14:13:06.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.446+0000 7fb374aa0700 1 -- 172.21.15.29:0/3304219189 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00ba60 con 0x7fb370150ba0 2024-01-24T14:13:06.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.447+0000 7fb377d06700 1 -- 172.21.15.29:0/3304219189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb370150ba0 msgr2=0x7fb370150f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.447+0000 7fb377d06700 1 --2- 172.21.15.29:0/3304219189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb370150ba0 0x7fb370150f80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fb36c009760 tx=0x7fb36c009060).stop 2024-01-24T14:13:06.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.447+0000 7fb377d06700 1 -- 172.21.15.29:0/3304219189 shutdown_connections 2024-01-24T14:13:06.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.447+0000 7fb377d06700 1 --2- 172.21.15.29:0/3304219189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb370150ba0 0x7fb370150f80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.447+0000 7fb377d06700 1 -- 172.21.15.29:0/3304219189 >> 172.21.15.29:0/3304219189 conn(0x7fb3700c85c0 msgr2=0x7fb3700c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:06.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.447+0000 7fb377d06700 1 -- 172.21.15.29:0/3304219189 shutdown_connections 2024-01-24T14:13:06.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.447+0000 7fb377d06700 1 -- 172.21.15.29:0/3304219189 wait complete. 2024-01-24T14:13:06.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.449+0000 7fb377d06700 1 Processor -- start 2024-01-24T14:13:06.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.449+0000 7fb377d06700 1 -- start start 2024-01-24T14:13:06.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.450+0000 7fb377d06700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb370066480 0x7fb37014b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.450+0000 7fb377d06700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3700c40d0 0x7fb370147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.450+0000 7fb377d06700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb370150ba0 0x7fb370148020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.450+0000 7fb377d06700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3700ad520 con 0x7fb370066480 2024-01-24T14:13:06.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.450+0000 7fb377d06700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3700ac910 con 0x7fb370150ba0 2024-01-24T14:13:06.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.450+0000 7fb377d06700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3700acbf0 con 0x7fb3700c40d0 2024-01-24T14:13:06.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.450+0000 7fb375aa2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb370066480 0x7fb37014b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.450+0000 7fb3762a3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb370150ba0 0x7fb370148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.450+0000 7fb375aa2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb370066480 0x7fb37014b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59214/0 (socket says 172.21.15.29:59214) 2024-01-24T14:13:06.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.450+0000 7fb375aa2700 1 -- 172.21.15.29:0/3474884387 learned_addr learned my addr 172.21.15.29:0/3474884387 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:06.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.450+0000 7fb3752a1700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3700c40d0 0x7fb370147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.451+0000 7fb375aa2700 1 -- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3700c40d0 msgr2=0x7fb370147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.451+0000 7fb375aa2700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3700c40d0 0x7fb370147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.451+0000 7fb375aa2700 1 -- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb370150ba0 msgr2=0x7fb370148020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.451+0000 7fb375aa2700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb370150ba0 0x7fb370148020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.451+0000 7fb375aa2700 1 -- 172.21.15.29:0/3474884387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c0092a0 con 0x7fb370066480 2024-01-24T14:13:06.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.451+0000 7fb375aa2700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb370066480 0x7fb37014b400 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fb36c009eb0 tx=0x7fb36c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:06.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.451+0000 7fb365ffb700 1 -- 172.21.15.29:0/3474884387 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00b850 con 0x7fb370066480 2024-01-24T14:13:06.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.451+0000 7fb365ffb700 1 -- 172.21.15.29:0/3474884387 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36c00b9b0 con 0x7fb370066480 2024-01-24T14:13:06.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.451+0000 7fb365ffb700 1 -- 172.21.15.29:0/3474884387 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c01b690 con 0x7fb370066480 2024-01-24T14:13:06.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.451+0000 7fb377d06700 1 -- 172.21.15.29:0/3474884387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3700aced0 con 0x7fb370066480 2024-01-24T14:13:06.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.451+0000 7fb377d06700 1 -- 172.21.15.29:0/3474884387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb370148510 con 0x7fb370066480 2024-01-24T14:13:06.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.452+0000 7fb377d06700 1 -- 172.21.15.29:0/3474884387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3700475e0 con 0x7fb370066480 2024-01-24T14:13:06.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.453+0000 7fb365ffb700 1 -- 172.21.15.29:0/3474884387 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fb36c01ba40 con 0x7fb370066480 2024-01-24T14:13:06.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.454+0000 7fb365ffb700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb35405f2a0 0x7fb354061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.454+0000 7fb365ffb700 1 -- 172.21.15.29:0/3474884387 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7514+0+0 (secure 0 0 0) 0x7fb36c07ccc0 con 0x7fb370066480 2024-01-24T14:13:06.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.454+0000 7fb3752a1700 1 -- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb35405f2a0 msgr2=0x7fb354061760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:06.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.454+0000 7fb3752a1700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb35405f2a0 0x7fb354061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:06.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.458+0000 7fb365ffb700 1 -- 172.21.15.29:0/3474884387 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb36c04d860 con 0x7fb370066480 2024-01-24T14:13:06.484 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:06.483+0000 7f40bde04680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:13:06.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.655+0000 7fb3752a1700 1 -- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb35405f2a0 msgr2=0x7fb354061760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:06.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.655+0000 7fb3752a1700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb35405f2a0 0x7fb354061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:13:06.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.688+0000 7fb377d06700 1 -- 172.21.15.29:0/3474884387 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3700c20f0 con 0x7fb370066480 2024-01-24T14:13:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.689+0000 7fb365ffb700 1 -- 172.21.15.29:0/3474884387 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4492 (secure 0 0 0) 0x7fb36c050e80 con 0x7fb370066480 2024-01-24T14:13:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:06.692 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2239835986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2239835986},{"type":"v1","addr":"172.21.15.163:6837","nonce":2239835986}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2208246875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2208246875},{"type":"v1","addr":"172.21.15.29:6835","nonce":2208246875}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/547152226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":547152226},{"type":"v1","addr":"172.21.15.163:6836","nonce":547152226}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:10:59.693964+0000","modified":"2024-01-24T14:11:02.793474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2804469957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2804469957},{"type":"v1","addr":"172.21.15.29:6834","nonce":2804469957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-24T14:13:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.691+0000 7fb377d06700 1 -- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb35405f2a0 msgr2=0x7fb354061760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.691+0000 7fb377d06700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb35405f2a0 0x7fb354061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.691+0000 7fb377d06700 1 -- 172.21.15.29:0/3474884387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb370066480 msgr2=0x7fb37014b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.691+0000 7fb377d06700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb370066480 0x7fb37014b400 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fb36c009eb0 tx=0x7fb36c017000).stop 2024-01-24T14:13:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.691+0000 7fb377d06700 1 -- 172.21.15.29:0/3474884387 shutdown_connections 2024-01-24T14:13:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.691+0000 7fb377d06700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb370150ba0 0x7fb370148020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.691+0000 7fb377d06700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb35405f2a0 0x7fb354061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.691+0000 7fb377d06700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb370066480 0x7fb37014b400 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.691+0000 7fb377d06700 1 --2- 172.21.15.29:0/3474884387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3700c40d0 0x7fb370147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.691+0000 7fb377d06700 1 -- 172.21.15.29:0/3474884387 >> 172.21.15.29:0/3474884387 conn(0x7fb3700c85c0 msgr2=0x7fb3700c6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.691+0000 7fb377d06700 1 -- 172.21.15.29:0/3474884387 shutdown_connections 2024-01-24T14:13:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.691+0000 7fb377d06700 1 -- 172.21.15.29:0/3474884387 wait complete. 2024-01-24T14:13:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 26 2024-01-24T14:13:06.706 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:13:06.713 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:06.712+0000 7f0742117680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:13:06.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.845+0000 7f8b35f08700 1 Processor -- start 2024-01-24T14:13:06.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.846+0000 7f8b35f08700 1 -- start start 2024-01-24T14:13:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.846+0000 7f8b35f08700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8b300be520 0x7f8b300664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.846+0000 7f8b35f08700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b300669e0 0x7f8b300c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b35f08700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b30064120 con 0x7f8b3013a5c0 2024-01-24T14:13:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b35f08700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b300bf6a0 con 0x7f8b300669e0 2024-01-24T14:13:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b35f08700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b30066e00 con 0x7f8b300be520 2024-01-24T14:13:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b2ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b300669e0 0x7f8b300c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b2ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b300669e0 0x7f8b300c31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59216/0 (socket says 172.21.15.29:59216) 2024-01-24T14:13:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b2ffff700 1 -- 172.21.15.29:0/1407196682 learned_addr learned my addr 172.21.15.29:0/1407196682 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b2effd700 1 --2- 172.21.15.29:0/1407196682 >> v2:172.21.15.163:3300/0 conn(0x7f8b300be520 0x7f8b300664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b2ffff700 1 -- 172.21.15.29:0/1407196682 >> v2:172.21.15.163:3300/0 conn(0x7f8b300be520 msgr2=0x7f8b300664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b2ffff700 1 --2- 172.21.15.29:0/1407196682 >> v2:172.21.15.163:3300/0 conn(0x7f8b300be520 0x7f8b300664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b2ffff700 1 -- 172.21.15.29:0/1407196682 >> v1:172.21.15.163:6789/0 conn(0x7f8b3013a5c0 legacy=0x7f8b30063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b2ffff700 1 -- 172.21.15.29:0/1407196682 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b300bea90 con 0x7f8b300669e0 2024-01-24T14:13:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b2ffff700 1 --2- 172.21.15.29:0/1407196682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b300669e0 0x7f8b300c31b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f8b30065a20 tx=0x7f8b28009840).ready entity=mon.0 client_cookie=aa8bfd9c6f4120de server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.847+0000 7f8b2e7fc700 1 -- 172.21.15.29:0/1407196682 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800e070 con 0x7f8b300669e0 2024-01-24T14:13:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.848+0000 7f8b2e7fc700 1 -- 172.21.15.29:0/1407196682 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2800ba20 con 0x7f8b300669e0 2024-01-24T14:13:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.848+0000 7f8b2e7fc700 1 -- 172.21.15.29:0/1407196682 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800bda0 con 0x7f8b300669e0 2024-01-24T14:13:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.848+0000 7f8b35f08700 1 -- 172.21.15.29:0/1407196682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b300669e0 msgr2=0x7f8b300c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.848+0000 7f8b35f08700 1 --2- 172.21.15.29:0/1407196682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b300669e0 0x7f8b300c31b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f8b30065a20 tx=0x7f8b28009840).stop 2024-01-24T14:13:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.848+0000 7f8b35f08700 1 -- 172.21.15.29:0/1407196682 shutdown_connections 2024-01-24T14:13:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.848+0000 7f8b35f08700 1 --2- 172.21.15.29:0/1407196682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b300669e0 0x7f8b300c31b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.848+0000 7f8b35f08700 1 --2- 172.21.15.29:0/1407196682 >> v2:172.21.15.163:3300/0 conn(0x7f8b300be520 0x7f8b300664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.848+0000 7f8b35f08700 1 -- 172.21.15.29:0/1407196682 >> 172.21.15.29:0/1407196682 conn(0x7f8b300c8bb0 msgr2=0x7f8b300c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:06.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.848+0000 7f8b35f08700 1 -- 172.21.15.29:0/1407196682 shutdown_connections 2024-01-24T14:13:06.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.848+0000 7f8b35f08700 1 -- 172.21.15.29:0/1407196682 wait complete. 2024-01-24T14:13:06.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.850+0000 7f8b35f08700 1 Processor -- start 2024-01-24T14:13:06.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.850+0000 7f8b35f08700 1 -- start start 2024-01-24T14:13:06.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b35f08700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b300669e0 0x7f8b3014fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b35f08700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b300be520 0x7f8b30151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b35f08700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b3013a5c0 0x7f8b300a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b35f08700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b300a7970 con 0x7f8b300669e0 2024-01-24T14:13:06.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b35f08700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b300a8610 con 0x7f8b300be520 2024-01-24T14:13:06.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b35f08700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b301525e0 con 0x7f8b3013a5c0 2024-01-24T14:13:06.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b2f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b300669e0 0x7f8b3014fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b2f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b300669e0 0x7f8b3014fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59226/0 (socket says 172.21.15.29:59226) 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b2f7fe700 1 -- 172.21.15.29:0/2158552770 learned_addr learned my addr 172.21.15.29:0/2158552770 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b2effd700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b300be520 0x7f8b30151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b2ffff700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b3013a5c0 0x7f8b300a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b2f7fe700 1 -- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b3013a5c0 msgr2=0x7f8b300a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b2f7fe700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b3013a5c0 0x7f8b300a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b2f7fe700 1 -- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b300be520 msgr2=0x7f8b30151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b2f7fe700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b300be520 0x7f8b30151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b2f7fe700 1 -- 172.21.15.29:0/2158552770 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b28009ae0 con 0x7f8b300669e0 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b2effd700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b300be520 0x7f8b30151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.851+0000 7f8b2f7fe700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b300669e0 0x7f8b3014fee0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f8b2000c440 tx=0x7f8b20007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.852+0000 7f8b177fe700 1 -- 172.21.15.29:0/2158552770 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b20007020 con 0x7f8b300669e0 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.852+0000 7f8b177fe700 1 -- 172.21.15.29:0/2158552770 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b20007180 con 0x7f8b300669e0 2024-01-24T14:13:06.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.852+0000 7f8b35f08700 1 -- 172.21.15.29:0/2158552770 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b301528c0 con 0x7f8b300669e0 2024-01-24T14:13:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.852+0000 7f8b177fe700 1 -- 172.21.15.29:0/2158552770 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b20007520 con 0x7f8b300669e0 2024-01-24T14:13:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.852+0000 7f8b35f08700 1 -- 172.21.15.29:0/2158552770 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b30152e80 con 0x7f8b300669e0 2024-01-24T14:13:06.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.853+0000 7f8b35f08700 1 -- 172.21.15.29:0/2158552770 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b300acda0 con 0x7f8b300669e0 2024-01-24T14:13:06.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.853+0000 7f8b177fe700 1 -- 172.21.15.29:0/2158552770 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f8b200068b0 con 0x7f8b300669e0 2024-01-24T14:13:06.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.854+0000 7f8b177fe700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8b1c05f240 0x7f8b1c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:06.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.854+0000 7f8b2effd700 1 -- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8b1c05f240 msgr2=0x7f8b1c061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:06.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.854+0000 7f8b2effd700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8b1c05f240 0x7f8b1c061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:06.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.855+0000 7f8b177fe700 1 -- 172.21.15.29:0/2158552770 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7514+0+0 (secure 0 0 0) 0x7f8b2004e410 con 0x7f8b300669e0 2024-01-24T14:13:06.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:06.859+0000 7f8b177fe700 1 -- 172.21.15.29:0/2158552770 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8b20047b00 con 0x7f8b300669e0 2024-01-24T14:13:06.932 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:06.932+0000 7f0742117680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:13:07.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.045+0000 7f8b35f08700 1 -- 172.21.15.29:0/2158552770 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b300475e0 con 0x7f8b300669e0 2024-01-24T14:13:07.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.046+0000 7f8b177fe700 1 -- 172.21.15.29:0/2158552770 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18746 (secure 0 0 0) 0x7f8b2004a7b0 con 0x7f8b300669e0 2024-01-24T14:13:07.047 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:07.047 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:13:04.844307+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:10:57.425223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-24T14:10:58.622652+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":76,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.17","mappings":[{"from":0,"to":6}]},{"pgid":"8.21","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6833/43241":"2024-01-25T14:11:12.020309+0000","172.21.15.163:6832/43241":"2024-01-25T14:11:12.020309+0000","172.21.15.163:0/3166674027":"2024-01-25T14:11:12.020309+0000","172.21.15.163:0/2273027649":"2024-01-25T14:11:12.020309+0000","172.21.15.163:0/4130041143":"2024-01-25T14:11:12.020309+0000","172.21.15.163:0/97990424":"2024-01-25T14:11:12.020309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":9,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:13:07.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.048+0000 7f8b35f08700 1 -- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8b1c05f240 msgr2=0x7f8b1c061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:07.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.048+0000 7f8b35f08700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8b1c05f240 0x7f8b1c061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:07.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.048+0000 7f8b35f08700 1 -- 172.21.15.29:0/2158552770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b300669e0 msgr2=0x7f8b3014fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.048+0000 7f8b35f08700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b300669e0 0x7f8b3014fee0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f8b2000c440 tx=0x7f8b20007e00).stop 2024-01-24T14:13:07.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.048+0000 7f8b35f08700 1 -- 172.21.15.29:0/2158552770 shutdown_connections 2024-01-24T14:13:07.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.048+0000 7f8b35f08700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b3013a5c0 0x7f8b300a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.048+0000 7f8b35f08700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8b1c05f240 0x7f8b1c061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:07.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.048+0000 7f8b35f08700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b300669e0 0x7f8b3014fee0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.048+0000 7f8b35f08700 1 --2- 172.21.15.29:0/2158552770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b300be520 0x7f8b30151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.048+0000 7f8b35f08700 1 -- 172.21.15.29:0/2158552770 >> 172.21.15.29:0/2158552770 conn(0x7f8b300c8bb0 msgr2=0x7f8b30063880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:07.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.048+0000 7f8b35f08700 1 -- 172.21.15.29:0/2158552770 shutdown_connections 2024-01-24T14:13:07.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.048+0000 7f8b35f08700 1 -- 172.21.15.29:0/2158552770 wait complete. 2024-01-24T14:13:07.063 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T14:13:07.063 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:13:07.133 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:07.132+0000 7f40bde04680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:13:07.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.194+0000 7fd4f17cc700 1 Processor -- start 2024-01-24T14:13:07.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.194+0000 7fd4f17cc700 1 -- start start 2024-01-24T14:13:07.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4f17cc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4ec066c00 0x7fd4ec0638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4f17cc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd4ec154060 0x7fd4ec0bf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4f17cc700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ec068400 con 0x7fd4ec153c80 2024-01-24T14:13:07.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4f17cc700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec1544f0 con 0x7fd4ec066c00 2024-01-24T14:13:07.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4f17cc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec0bfad0 con 0x7fd4ec154060 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4ea7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4ec066c00 0x7fd4ec0638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4eaffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd4ec153c80 0x7fd4ec0664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57016/0 (socket says 172.21.15.29:57016) 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4eaffd700 1 -- 172.21.15.29:0/2766670304 learned_addr learned my addr 172.21.15.29:0/2766670304 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4eb7fe700 1 --2- 172.21.15.29:0/2766670304 >> v2:172.21.15.163:3300/0 conn(0x7fd4ec154060 0x7fd4ec0bf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4ea7fc700 1 -- 172.21.15.29:0/2766670304 >> v2:172.21.15.163:3300/0 conn(0x7fd4ec154060 msgr2=0x7fd4ec0bf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4ea7fc700 1 --2- 172.21.15.29:0/2766670304 >> v2:172.21.15.163:3300/0 conn(0x7fd4ec154060 0x7fd4ec0bf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4ea7fc700 1 -- 172.21.15.29:0/2766670304 >> v1:172.21.15.29:6789/0 conn(0x7fd4ec153c80 legacy=0x7fd4ec0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4ea7fc700 1 -- 172.21.15.29:0/2766670304 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec067ad0 con 0x7fd4ec066c00 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4e9ffb700 1 -- 172.21.15.29:0/2766670304 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117926366 0 0) 0x7fd4ec068400 con 0x7fd4ec153c80 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.195+0000 7fd4ea7fc700 1 --2- 172.21.15.29:0/2766670304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ec066c00 0x7fd4ec0638c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fd4e0009760 tx=0x7fd4e0009060).ready entity=mon.0 client_cookie=c5cbc9baf5265c6f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.196+0000 7fd4e9ffb700 1 -- 172.21.15.29:0/2766670304 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000f510 con 0x7fd4ec066c00 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.196+0000 7fd4e9ffb700 1 -- 172.21.15.29:0/2766670304 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e000f670 con 0x7fd4ec066c00 2024-01-24T14:13:07.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.196+0000 7fd4e9ffb700 1 -- 172.21.15.29:0/2766670304 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000f9f0 con 0x7fd4ec066c00 2024-01-24T14:13:07.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.196+0000 7fd4f17cc700 1 -- 172.21.15.29:0/2766670304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ec066c00 msgr2=0x7fd4ec0638c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.196+0000 7fd4f17cc700 1 --2- 172.21.15.29:0/2766670304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ec066c00 0x7fd4ec0638c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fd4e0009760 tx=0x7fd4e0009060).stop 2024-01-24T14:13:07.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.196+0000 7fd4f17cc700 1 -- 172.21.15.29:0/2766670304 shutdown_connections 2024-01-24T14:13:07.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.196+0000 7fd4f17cc700 1 --2- 172.21.15.29:0/2766670304 >> v2:172.21.15.163:3300/0 conn(0x7fd4ec154060 0x7fd4ec0bf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.196+0000 7fd4f17cc700 1 --2- 172.21.15.29:0/2766670304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ec066c00 0x7fd4ec0638c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.196+0000 7fd4f17cc700 1 -- 172.21.15.29:0/2766670304 >> 172.21.15.29:0/2766670304 conn(0x7fd4ec0c8bb0 msgr2=0x7fd4ec0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:07.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.196+0000 7fd4f17cc700 1 -- 172.21.15.29:0/2766670304 shutdown_connections 2024-01-24T14:13:07.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.196+0000 7fd4f17cc700 1 -- 172.21.15.29:0/2766670304 wait complete. 2024-01-24T14:13:07.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.198+0000 7fd4f17cc700 1 Processor -- start 2024-01-24T14:13:07.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.198+0000 7fd4f17cc700 1 -- start start 2024-01-24T14:13:07.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4f17cc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ec066c00 0x7fd4ec155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4f17cc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ec153c80 0x7fd4ec0add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4f17cc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ec154060 0x7fd4ec163c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4f17cc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec1510c0 con 0x7fd4ec153c80 2024-01-24T14:13:07.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4f17cc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec155f10 con 0x7fd4ec154060 2024-01-24T14:13:07.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4f17cc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec1561f0 con 0x7fd4ec066c00 2024-01-24T14:13:07.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4ea7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ec153c80 0x7fd4ec0add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4ea7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ec153c80 0x7fd4ec0add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59244/0 (socket says 172.21.15.29:59244) 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4ea7fc700 1 -- 172.21.15.29:0/942593362 learned_addr learned my addr 172.21.15.29:0/942593362 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4eb7fe700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ec154060 0x7fd4ec163c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4eaffd700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ec066c00 0x7fd4ec155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4ea7fc700 1 -- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ec066c00 msgr2=0x7fd4ec155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4ea7fc700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ec066c00 0x7fd4ec155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4ea7fc700 1 -- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ec154060 msgr2=0x7fd4ec163c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4ea7fc700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ec154060 0x7fd4ec163c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4ea7fc700 1 -- 172.21.15.29:0/942593362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e00092a0 con 0x7fd4ec153c80 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4eb7fe700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ec154060 0x7fd4ec163c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.199+0000 7fd4ea7fc700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ec153c80 0x7fd4ec0add20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fd4ec0c69b0 tx=0x7fd4e0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.200+0000 7fd4d2ffd700 1 -- 172.21.15.29:0/942593362 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0011b50 con 0x7fd4ec153c80 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.200+0000 7fd4d2ffd700 1 -- 172.21.15.29:0/942593362 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e0011cd0 con 0x7fd4ec153c80 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.200+0000 7fd4f17cc700 1 -- 172.21.15.29:0/942593362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec1564d0 con 0x7fd4ec153c80 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.200+0000 7fd4d2ffd700 1 -- 172.21.15.29:0/942593362 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e001b660 con 0x7fd4ec153c80 2024-01-24T14:13:07.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.200+0000 7fd4f17cc700 1 -- 172.21.15.29:0/942593362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec155b00 con 0x7fd4ec153c80 2024-01-24T14:13:07.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.201+0000 7fd4d2ffd700 1 -- 172.21.15.29:0/942593362 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fd4e001b9d0 con 0x7fd4ec153c80 2024-01-24T14:13:07.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.201+0000 7fd4f17cc700 1 -- 172.21.15.29:0/942593362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ec0475e0 con 0x7fd4ec153c80 2024-01-24T14:13:07.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.202+0000 7fd4d2ffd700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4d805f270 0x7fd4d8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.202+0000 7fd4d2ffd700 1 -- 172.21.15.29:0/942593362 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7514+0+0 (secure 0 0 0) 0x7fd4e007c330 con 0x7fd4ec153c80 2024-01-24T14:13:07.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.202+0000 7fd4eaffd700 1 -- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4d805f270 msgr2=0x7fd4d8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:07.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.202+0000 7fd4eaffd700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4d805f270 0x7fd4d8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:07.204 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:07.200+0000 7f40bde04680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:13:07.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.209+0000 7fd4d2ffd700 1 -- 172.21.15.29:0/942593362 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd4e004cf10 con 0x7fd4ec153c80 2024-01-24T14:13:07.277 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:07.276+0000 7f40bde04680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:13:07.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.403+0000 7fd4eaffd700 1 -- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4d805f270 msgr2=0x7fd4d8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:07.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.403+0000 7fd4eaffd700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4d805f270 0x7fd4d8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:13:07.423 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:07.422+0000 7f40bde04680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:13:07.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.437+0000 7fd4f17cc700 1 -- 172.21.15.29:0/942593362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4ec067db0 con 0x7fd4ec153c80 2024-01-24T14:13:07.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.438+0000 7fd4d2ffd700 1 -- 172.21.15.29:0/942593362 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4492 (secure 0 0 0) 0x7fd4e0050530 con 0x7fd4ec153c80 2024-01-24T14:13:07.438 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:07.439 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2239835986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2239835986},{"type":"v1","addr":"172.21.15.163:6837","nonce":2239835986}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2208246875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2208246875},{"type":"v1","addr":"172.21.15.29:6835","nonce":2208246875}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/547152226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":547152226},{"type":"v1","addr":"172.21.15.163:6836","nonce":547152226}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:10:59.693964+0000","modified":"2024-01-24T14:11:02.793474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2804469957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2804469957},{"type":"v1","addr":"172.21.15.29:6834","nonce":2804469957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-24T14:13:07.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.440+0000 7fd4f17cc700 1 -- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4d805f270 msgr2=0x7fd4d8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:07.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.440+0000 7fd4f17cc700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4d805f270 0x7fd4d8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:07.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.440+0000 7fd4f17cc700 1 -- 172.21.15.29:0/942593362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ec153c80 msgr2=0x7fd4ec0add20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.440+0000 7fd4f17cc700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ec153c80 0x7fd4ec0add20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fd4ec0c69b0 tx=0x7fd4e0017000).stop 2024-01-24T14:13:07.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.440+0000 7fd4f17cc700 1 -- 172.21.15.29:0/942593362 shutdown_connections 2024-01-24T14:13:07.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.440+0000 7fd4f17cc700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ec154060 0x7fd4ec163c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.440+0000 7fd4f17cc700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ec066c00 0x7fd4ec155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.440+0000 7fd4f17cc700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4d805f270 0x7fd4d8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:07.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.440+0000 7fd4f17cc700 1 --2- 172.21.15.29:0/942593362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ec153c80 0x7fd4ec0add20 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.440+0000 7fd4f17cc700 1 -- 172.21.15.29:0/942593362 >> 172.21.15.29:0/942593362 conn(0x7fd4ec0c8bb0 msgr2=0x7fd4ec0664f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:07.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.440+0000 7fd4f17cc700 1 -- 172.21.15.29:0/942593362 shutdown_connections 2024-01-24T14:13:07.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.440+0000 7fd4f17cc700 1 -- 172.21.15.29:0/942593362 wait complete. 2024-01-24T14:13:07.444 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 26 2024-01-24T14:13:07.455 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:13:07.493 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:07.492+0000 7f40bde04680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:13:07.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.596+0000 7ff6f20d6700 1 Processor -- start 2024-01-24T14:13:07.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.596+0000 7ff6f20d6700 1 -- start start 2024-01-24T14:13:07.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.597+0000 7ff6f20d6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ec0c5410 0x7ff6ec0aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.597+0000 7ff6f20d6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6ec154750 0x7ff6ec0631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.597+0000 7ff6f20d6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ec0aa6c0 con 0x7ff6ec154370 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.597+0000 7ff6f20d6700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec064240 con 0x7ff6ec154750 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.597+0000 7ff6f20d6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec0cb5c0 con 0x7ff6ec0c5410 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6ebfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6ec154750 0x7ff6ec0631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6ebfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6ec154750 0x7ff6ec0631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59248/0 (socket says 172.21.15.29:59248) 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6ebfff700 1 -- 172.21.15.29:0/2608782619 learned_addr learned my addr 172.21.15.29:0/2608782619 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6eaffd700 1 --2- 172.21.15.29:0/2608782619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ec0c5410 0x7ff6ec0aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6ebfff700 1 -- 172.21.15.29:0/2608782619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ec0c5410 msgr2=0x7ff6ec0aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6ebfff700 1 --2- 172.21.15.29:0/2608782619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ec0c5410 0x7ff6ec0aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6ebfff700 1 -- 172.21.15.29:0/2608782619 >> v1:172.21.15.163:6789/0 conn(0x7ff6ec154370 legacy=0x7ff6ec0ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6ebfff700 1 -- 172.21.15.29:0/2608782619 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0cb8a0 con 0x7ff6ec154750 2024-01-24T14:13:07.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6ea7fc700 1 -- 172.21.15.29:0/2608782619 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120895578 0 0) 0x7ff6ec0aa6c0 con 0x7ff6ec154370 2024-01-24T14:13:07.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6ebfff700 1 --2- 172.21.15.29:0/2608782619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ec154750 0x7ff6ec0631d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7ff6e4009f20 tx=0x7ff6e4009840).ready entity=mon.0 client_cookie=921ea4d5eb378949 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:07.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6ea7fc700 1 -- 172.21.15.29:0/2608782619 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400e070 con 0x7ff6ec154750 2024-01-24T14:13:07.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6ea7fc700 1 -- 172.21.15.29:0/2608782619 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e400b9d0 con 0x7ff6ec154750 2024-01-24T14:13:07.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.598+0000 7ff6ea7fc700 1 -- 172.21.15.29:0/2608782619 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400bd50 con 0x7ff6ec154750 2024-01-24T14:13:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.599+0000 7ff6f20d6700 1 -- 172.21.15.29:0/2608782619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ec154750 msgr2=0x7ff6ec0631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.599+0000 7ff6f20d6700 1 --2- 172.21.15.29:0/2608782619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ec154750 0x7ff6ec0631d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7ff6e4009f20 tx=0x7ff6e4009840).stop 2024-01-24T14:13:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.599+0000 7ff6f20d6700 1 -- 172.21.15.29:0/2608782619 shutdown_connections 2024-01-24T14:13:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.599+0000 7ff6f20d6700 1 --2- 172.21.15.29:0/2608782619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ec154750 0x7ff6ec0631d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.599+0000 7ff6f20d6700 1 --2- 172.21.15.29:0/2608782619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ec0c5410 0x7ff6ec0aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.599+0000 7ff6f20d6700 1 -- 172.21.15.29:0/2608782619 >> 172.21.15.29:0/2608782619 conn(0x7ff6ec05ec80 msgr2=0x7ff6ec0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.599+0000 7ff6f20d6700 1 -- 172.21.15.29:0/2608782619 shutdown_connections 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.599+0000 7ff6f20d6700 1 -- 172.21.15.29:0/2608782619 wait complete. 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.601+0000 7ff6f20d6700 1 Processor -- start 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.601+0000 7ff6f20d6700 1 -- start start 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.601+0000 7ff6f20d6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6ec0c5410 0x7ff6ec146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.601+0000 7ff6f20d6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ec154370 0x7ff6ec14f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.601+0000 7ff6f20d6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ec154750 0x7ff6ec154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.601+0000 7ff6f20d6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec0c4d00 con 0x7ff6ec154370 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.601+0000 7ff6f20d6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec14ed30 con 0x7ff6ec0c5410 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.601+0000 7ff6f20d6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec14e120 con 0x7ff6ec154750 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6eaffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ec154370 0x7ff6ec14f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6ebfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ec154750 0x7ff6ec154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6eb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6ec0c5410 0x7ff6ec146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6eaffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ec154370 0x7ff6ec14f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59258/0 (socket says 172.21.15.29:59258) 2024-01-24T14:13:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6ebfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ec154750 0x7ff6ec154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:45128/0 (socket says 172.21.15.29:45128) 2024-01-24T14:13:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6eaffd700 1 -- 172.21.15.29:0/1691837230 learned_addr learned my addr 172.21.15.29:0/1691837230 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6ebfff700 1 -- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6ec0c5410 msgr2=0x7ff6ec146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6ebfff700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6ec0c5410 0x7ff6ec146db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6ebfff700 1 -- 172.21.15.29:0/1691837230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ec154370 msgr2=0x7ff6ec14f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6ebfff700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ec154370 0x7ff6ec14f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6ebfff700 1 -- 172.21.15.29:0/1691837230 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e4009ae0 con 0x7ff6ec154750 2024-01-24T14:13:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6eaffd700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ec154370 0x7ff6ec14f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:07.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6ebfff700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ec154750 0x7ff6ec154c70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7ff6e4013000 tx=0x7ff6e4009840).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:07.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.602+0000 7ff6d37fe700 1 -- 172.21.15.29:0/1691837230 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400bd50 con 0x7ff6ec154750 2024-01-24T14:13:07.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.603+0000 7ff6d37fe700 1 -- 172.21.15.29:0/1691837230 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e40092e0 con 0x7ff6ec154750 2024-01-24T14:13:07.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.603+0000 7ff6f20d6700 1 -- 172.21.15.29:0/1691837230 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec14e400 con 0x7ff6ec154750 2024-01-24T14:13:07.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.603+0000 7ff6d37fe700 1 -- 172.21.15.29:0/1691837230 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e401d6b0 con 0x7ff6ec154750 2024-01-24T14:13:07.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.603+0000 7ff6f20d6700 1 -- 172.21.15.29:0/1691837230 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ec14e9c0 con 0x7ff6ec154750 2024-01-24T14:13:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.604+0000 7ff6f20d6700 1 -- 172.21.15.29:0/1691837230 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ec0475e0 con 0x7ff6ec154750 2024-01-24T14:13:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.604+0000 7ff6d37fe700 1 -- 172.21.15.29:0/1691837230 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7ff6e4003040 con 0x7ff6ec154750 2024-01-24T14:13:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.604+0000 7ff6d37fe700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff6d805f2c0 0x7ff6d8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.604+0000 7ff6d37fe700 1 -- 172.21.15.29:0/1691837230 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7514+0+0 (secure 0 0 0) 0x7ff6e407b9b0 con 0x7ff6ec154750 2024-01-24T14:13:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.604+0000 7ff6eb7fe700 1 -- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff6d805f2c0 msgr2=0x7ff6d8061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:07.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.604+0000 7ff6eb7fe700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff6d805f2c0 0x7ff6d8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:07.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.609+0000 7ff6d37fe700 1 -- 172.21.15.29:0/1691837230 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff6e404c700 con 0x7ff6ec154750 2024-01-24T14:13:07.609 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:07.608+0000 7f40bde04680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:13:07.706 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:07.706+0000 7f0742117680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:13:07.718 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:07.718+0000 7f40bde04680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:13:07.788 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:07.788+0000 7f0742117680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:13:07.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.805+0000 7ff6eb7fe700 1 -- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff6d805f2c0 msgr2=0x7ff6d8061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:07.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.805+0000 7ff6eb7fe700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff6d805f2c0 0x7ff6d8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:13:07.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.840+0000 7ff6f20d6700 1 -- 172.21.15.29:0/1691837230 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6ec0cbb80 con 0x7ff6ec154750 2024-01-24T14:13:07.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.841+0000 7ff6d37fe700 1 -- 172.21.15.29:0/1691837230 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4492 (secure 0 0 0) 0x7ff6e4022020 con 0x7ff6ec154750 2024-01-24T14:13:07.841 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:07.841 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2239835986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2239835986},{"type":"v1","addr":"172.21.15.163:6837","nonce":2239835986}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2208246875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2208246875},{"type":"v1","addr":"172.21.15.29:6835","nonce":2208246875}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/547152226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":547152226},{"type":"v1","addr":"172.21.15.163:6836","nonce":547152226}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:10:59.693964+0000","modified":"2024-01-24T14:11:02.793474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2804469957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2804469957},{"type":"v1","addr":"172.21.15.29:6834","nonce":2804469957}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-24T14:13:07.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.843+0000 7ff6f20d6700 1 -- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff6d805f2c0 msgr2=0x7ff6d8061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:07.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.843+0000 7ff6f20d6700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff6d805f2c0 0x7ff6d8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:07.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.843+0000 7ff6f20d6700 1 -- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ec154750 msgr2=0x7ff6ec154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.843+0000 7ff6f20d6700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ec154750 0x7ff6ec154c70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7ff6e4013000 tx=0x7ff6e4009840).stop 2024-01-24T14:13:07.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.843+0000 7ff6f20d6700 1 -- 172.21.15.29:0/1691837230 shutdown_connections 2024-01-24T14:13:07.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.843+0000 7ff6f20d6700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ec154750 0x7ff6ec154c70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.843+0000 7ff6f20d6700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6ec0c5410 0x7ff6ec146db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.843+0000 7ff6f20d6700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff6d805f2c0 0x7ff6d8061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:07.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.843+0000 7ff6f20d6700 1 --2- 172.21.15.29:0/1691837230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ec154370 0x7ff6ec14f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.843+0000 7ff6f20d6700 1 -- 172.21.15.29:0/1691837230 >> 172.21.15.29:0/1691837230 conn(0x7ff6ec05ec80 msgr2=0x7ff6ec152140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:07.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.843+0000 7ff6f20d6700 1 -- 172.21.15.29:0/1691837230 shutdown_connections 2024-01-24T14:13:07.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.843+0000 7ff6f20d6700 1 -- 172.21.15.29:0/1691837230 wait complete. 2024-01-24T14:13:07.845 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 26 2024-01-24T14:13:07.859 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:13:07.876 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:07.875+0000 7f0742117680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:13:07.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.989+0000 7fc36be1d700 1 Processor -- start 2024-01-24T14:13:07.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.990+0000 7fc36be1d700 1 -- start start 2024-01-24T14:13:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc36be1d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc364154370 0x7fc3640c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc36be1d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3640aa0e0 0x7fc364154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc36be1d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3640ab1f0 0x7fc3640ab680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc36be1d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3640aa6c0 con 0x7fc364154370 2024-01-24T14:13:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc36be1d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc364064240 con 0x7fc3640aa0e0 2024-01-24T14:13:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc36be1d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640cb5c0 con 0x7fc3640ab1f0 2024-01-24T14:13:07.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc369bb9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc364154370 0x7fc3640c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc369bb9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc364154370 0x7fc3640c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59262/0 (socket says 172.21.15.29:59262) 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc369bb9700 1 -- 172.21.15.29:0/583190617 learned_addr learned my addr 172.21.15.29:0/583190617 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc3693b8700 1 --2- 172.21.15.29:0/583190617 >> v2:172.21.15.163:3300/0 conn(0x7fc3640aa0e0 0x7fc364154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc36a3ba700 1 --2- 172.21.15.29:0/583190617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3640ab1f0 0x7fc3640ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc369bb9700 1 -- 172.21.15.29:0/583190617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3640ab1f0 msgr2=0x7fc3640ab680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc369bb9700 1 --2- 172.21.15.29:0/583190617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3640ab1f0 0x7fc3640ab680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc369bb9700 1 -- 172.21.15.29:0/583190617 >> v2:172.21.15.163:3300/0 conn(0x7fc3640aa0e0 msgr2=0x7fc364154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc369bb9700 1 --2- 172.21.15.29:0/583190617 >> v2:172.21.15.163:3300/0 conn(0x7fc3640aa0e0 0x7fc364154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc369bb9700 1 -- 172.21.15.29:0/583190617 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640cb8a0 con 0x7fc364154370 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.991+0000 7fc369bb9700 1 --2- 172.21.15.29:0/583190617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc364154370 0x7fc3640c5410 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fc3640aafe0 tx=0x7fc360009060).ready entity=mon.0 client_cookie=ef4646a12078951c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.992+0000 7fc368bb7700 1 -- 172.21.15.29:0/583190617 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360005880 con 0x7fc364154370 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.992+0000 7fc368bb7700 1 -- 172.21.15.29:0/583190617 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3600059e0 con 0x7fc364154370 2024-01-24T14:13:07.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.992+0000 7fc368bb7700 1 -- 172.21.15.29:0/583190617 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360005d60 con 0x7fc364154370 2024-01-24T14:13:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.992+0000 7fc36be1d700 1 -- 172.21.15.29:0/583190617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc364154370 msgr2=0x7fc3640c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.992+0000 7fc36be1d700 1 --2- 172.21.15.29:0/583190617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc364154370 0x7fc3640c5410 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fc3640aafe0 tx=0x7fc360009060).stop 2024-01-24T14:13:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.992+0000 7fc36be1d700 1 -- 172.21.15.29:0/583190617 shutdown_connections 2024-01-24T14:13:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.992+0000 7fc36be1d700 1 --2- 172.21.15.29:0/583190617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3640ab1f0 0x7fc3640ab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.992+0000 7fc36be1d700 1 --2- 172.21.15.29:0/583190617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc364154370 0x7fc3640c5410 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.992+0000 7fc36be1d700 1 --2- 172.21.15.29:0/583190617 >> v2:172.21.15.163:3300/0 conn(0x7fc3640aa0e0 0x7fc364154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.992+0000 7fc36be1d700 1 -- 172.21.15.29:0/583190617 >> 172.21.15.29:0/583190617 conn(0x7fc36405ec80 msgr2=0x7fc3640cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.992+0000 7fc36be1d700 1 -- 172.21.15.29:0/583190617 shutdown_connections 2024-01-24T14:13:07.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.992+0000 7fc36be1d700 1 -- 172.21.15.29:0/583190617 wait complete. 2024-01-24T14:13:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.994+0000 7fc36be1d700 1 Processor -- start 2024-01-24T14:13:07.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.994+0000 7fc36be1d700 1 -- start start 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.994+0000 7fc36be1d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3640aa0e0 0x7fc3640aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.994+0000 7fc36be1d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3640ab1f0 0x7fc3640af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.994+0000 7fc36be1d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc364154370 0x7fc36414f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.994+0000 7fc36be1d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640b0580 con 0x7fc3640aa0e0 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.994+0000 7fc36be1d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640af970 con 0x7fc364154370 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.994+0000 7fc36be1d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640afc50 con 0x7fc3640ab1f0 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc369bb9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3640aa0e0 0x7fc3640aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc369bb9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3640aa0e0 0x7fc3640aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59266/0 (socket says 172.21.15.29:59266) 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc369bb9700 1 -- 172.21.15.29:0/1032790852 learned_addr learned my addr 172.21.15.29:0/1032790852 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc36a3ba700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc364154370 0x7fc36414f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc3693b8700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3640ab1f0 0x7fc3640af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc369bb9700 1 -- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3640ab1f0 msgr2=0x7fc3640af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc369bb9700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3640ab1f0 0x7fc3640af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc369bb9700 1 -- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc364154370 msgr2=0x7fc36414f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:07.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc369bb9700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc364154370 0x7fc36414f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:07.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc369bb9700 1 -- 172.21.15.29:0/1032790852 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc360009300 con 0x7fc3640aa0e0 2024-01-24T14:13:07.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc369bb9700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3640aa0e0 0x7fc3640aee70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fc360009dd0 tx=0x7fc360016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:07.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc359ffb700 1 -- 172.21.15.29:0/1032790852 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36000f720 con 0x7fc3640aa0e0 2024-01-24T14:13:07.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc359ffb700 1 -- 172.21.15.29:0/1032790852 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36000f880 con 0x7fc3640aa0e0 2024-01-24T14:13:07.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc36be1d700 1 -- 172.21.15.29:0/1032790852 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3640aff30 con 0x7fc3640aa0e0 2024-01-24T14:13:07.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.995+0000 7fc359ffb700 1 -- 172.21.15.29:0/1032790852 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36000fc40 con 0x7fc3640aa0e0 2024-01-24T14:13:07.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.996+0000 7fc36be1d700 1 -- 172.21.15.29:0/1032790852 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc364153a10 con 0x7fc3640aa0e0 2024-01-24T14:13:07.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.997+0000 7fc36be1d700 1 -- 172.21.15.29:0/1032790852 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3640475e0 con 0x7fc3640aa0e0 2024-01-24T14:13:08.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.997+0000 7fc359ffb700 1 -- 172.21.15.29:0/1032790852 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fc360011650 con 0x7fc3640aa0e0 2024-01-24T14:13:08.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.998+0000 7fc359ffb700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc34805f270 0x7fc348061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:08.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.998+0000 7fc359ffb700 1 -- 172.21.15.29:0/1032790852 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7514+0+0 (secure 0 0 0) 0x7fc36007c6a0 con 0x7fc3640aa0e0 2024-01-24T14:13:08.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.998+0000 7fc3693b8700 1 -- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc34805f270 msgr2=0x7fc348061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:08.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:07.998+0000 7fc3693b8700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc34805f270 0x7fc348061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:08.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.002+0000 7fc359ffb700 1 -- 172.21.15.29:0/1032790852 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc36004d340 con 0x7fc3640aa0e0 2024-01-24T14:13:08.014 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:08.013+0000 7f40bde04680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:13:08.050 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:08.049+0000 7f0742117680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:13:08.085 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:08.084+0000 7f40bde04680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:13:08.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.106+0000 7fc359ffb700 1 -- 172.21.15.29:0/1032790852 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 22) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fc360079e40 con 0x7fc3640aa0e0 2024-01-24T14:13:08.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.106+0000 7fc359ffb700 1 -- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc34805f270 msgr2=0x7fc348061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:08.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.106+0000 7fc359ffb700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc34805f270 0x7fc348061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:08.204 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:08.166+0000 7f4039389700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:08.205 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:08.166+0000 7f4039389700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:08.205 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:08.166+0000 7f4039389700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:08.205 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:08.166+0000 7f4039389700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:08.205 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:08.166+0000 7f4039389700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:08.205 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:08.166+0000 7f4037b86700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:08.205 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:08.166+0000 7f4037b86700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:08.205 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:08.166+0000 7f4037b86700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:08.205 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:08.166+0000 7f4037b86700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:08.205 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:08.166+0000 7f4037b86700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:08.205 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:08.134+0000 7f0742117680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:13:08.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.196+0000 7fc36be1d700 1 -- 172.21.15.29:0/1032790852 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc36406c2d0 con 0x7fc3640aa0e0 2024-01-24T14:13:08.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.197+0000 7fc359ffb700 1 -- 172.21.15.29:0/1032790852 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18955 (secure 0 0 0) 0x7fc36406c2d0 con 0x7fc3640aa0e0 2024-01-24T14:13:08.206 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:08.207 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":77,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:13:08.100557+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:10:57.425223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-24T14:10:58.622652+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":76,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.17","mappings":[{"from":0,"to":6}]},{"pgid":"8.21","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/2816209883":"2024-01-25T14:13:08.100494+0000","172.21.15.163:0/1752842923":"2024-01-25T14:13:08.100494+0000","172.21.15.163:0/2255554200":"2024-01-25T14:13:08.100494+0000","172.21.15.163:0/3273095716":"2024-01-25T14:13:08.100494+0000","172.21.15.163:6833/43241":"2024-01-25T14:11:12.020309+0000","172.21.15.163:6832/43241":"2024-01-25T14:11:12.020309+0000","172.21.15.163:0/3166674027":"2024-01-25T14:11:12.020309+0000","172.21.15.163:0/2273027649":"2024-01-25T14:11:12.020309+0000","172.21.15.163:0/4130041143":"2024-01-25T14:11:12.020309+0000","172.21.15.163:0/97990424":"2024-01-25T14:11:12.020309+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":9,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:13:08.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.199+0000 7fc36be1d700 1 -- 172.21.15.29:0/1032790852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3640aa0e0 msgr2=0x7fc3640aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:08.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.199+0000 7fc36be1d700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3640aa0e0 0x7fc3640aee70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fc360009dd0 tx=0x7fc360016000).stop 2024-01-24T14:13:08.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.199+0000 7fc36be1d700 1 -- 172.21.15.29:0/1032790852 shutdown_connections 2024-01-24T14:13:08.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.199+0000 7fc36be1d700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc364154370 0x7fc36414f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:08.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.199+0000 7fc36be1d700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc34805f270 0x7fc348061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:08.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.199+0000 7fc36be1d700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3640aa0e0 0x7fc3640aee70 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:08.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.199+0000 7fc36be1d700 1 --2- 172.21.15.29:0/1032790852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3640ab1f0 0x7fc3640af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:08.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.199+0000 7fc36be1d700 1 -- 172.21.15.29:0/1032790852 >> 172.21.15.29:0/1032790852 conn(0x7fc36405ec80 msgr2=0x7fc3640ace00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:08.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.200+0000 7fc36be1d700 1 -- 172.21.15.29:0/1032790852 shutdown_connections 2024-01-24T14:13:08.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.200+0000 7fc36be1d700 1 -- 172.21.15.29:0/1032790852 wait complete. 2024-01-24T14:13:08.215 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-24T14:13:08.273 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:08.272+0000 7f0742117680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:13:08.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.388+0000 7fe2363af700 1 Processor -- start 2024-01-24T14:13:08.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.388+0000 7fe2363af700 1 -- start start 2024-01-24T14:13:08.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.389+0000 7fe2363af700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe230135350 0x7fe2300664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:08.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.389+0000 7fe2363af700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe230066990 0x7fe2301525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:08.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.389+0000 7fe2363af700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe230156460 con 0x7fe230152ae0 2024-01-24T14:13:08.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.389+0000 7fe2363af700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe230155c00 con 0x7fe230066990 2024-01-24T14:13:08.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.389+0000 7fe2363af700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe230066e10 con 0x7fe230135350 2024-01-24T14:13:08.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.389+0000 7fe22ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe230135350 0x7fe2300664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:08.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.389+0000 7fe22f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe230066990 0x7fe2301525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:08.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.389+0000 7fe22ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe230135350 0x7fe2300664a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:45160/0 (socket says 172.21.15.29:45160) 2024-01-24T14:13:08.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.389+0000 7fe22ffff700 1 -- 172.21.15.29:0/2077336995 learned_addr learned my addr 172.21.15.29:0/2077336995 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.390+0000 7fe22ffff700 1 -- 172.21.15.29:0/2077336995 >> v2:172.21.15.163:3300/0 conn(0x7fe230066990 msgr2=0x7fe2301525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.390+0000 7fe22ffff700 1 --2- 172.21.15.29:0/2077336995 >> v2:172.21.15.163:3300/0 conn(0x7fe230066990 0x7fe2301525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.390+0000 7fe22ffff700 1 -- 172.21.15.29:0/2077336995 >> v1:172.21.15.163:6789/0 conn(0x7fe230152ae0 legacy=0x7fe2300679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.390+0000 7fe22ffff700 1 -- 172.21.15.29:0/2077336995 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230153390 con 0x7fe230135350 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.390+0000 7fe22ffff700 1 --2- 172.21.15.29:0/2077336995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe230135350 0x7fe2300664a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fe220006fd0 tx=0x7fe2200090f0).ready entity=mon.2 client_cookie=2de7c787601ab60e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.390+0000 7fe22effd700 1 -- 172.21.15.29:0/2077336995 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22000b580 con 0x7fe230135350 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.390+0000 7fe22effd700 1 -- 172.21.15.29:0/2077336995 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22000b6e0 con 0x7fe230135350 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.390+0000 7fe22effd700 1 -- 172.21.15.29:0/2077336995 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22000ba60 con 0x7fe230135350 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.390+0000 7fe2363af700 1 -- 172.21.15.29:0/2077336995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe230135350 msgr2=0x7fe2300664a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.390+0000 7fe2363af700 1 --2- 172.21.15.29:0/2077336995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe230135350 0x7fe2300664a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fe220006fd0 tx=0x7fe2200090f0).stop 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.391+0000 7fe2363af700 1 -- 172.21.15.29:0/2077336995 shutdown_connections 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.391+0000 7fe2363af700 1 --2- 172.21.15.29:0/2077336995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe230135350 0x7fe2300664a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.391+0000 7fe2363af700 1 --2- 172.21.15.29:0/2077336995 >> v2:172.21.15.163:3300/0 conn(0x7fe230066990 0x7fe2301525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:08.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.391+0000 7fe2363af700 1 -- 172.21.15.29:0/2077336995 >> 172.21.15.29:0/2077336995 conn(0x7fe2300c5640 msgr2=0x7fe2300c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:08.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.391+0000 7fe2363af700 1 -- 172.21.15.29:0/2077336995 shutdown_connections 2024-01-24T14:13:08.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.391+0000 7fe2363af700 1 -- 172.21.15.29:0/2077336995 wait complete. 2024-01-24T14:13:08.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.392+0000 7fe2363af700 1 Processor -- start 2024-01-24T14:13:08.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.392+0000 7fe2363af700 1 -- start start 2024-01-24T14:13:08.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe2363af700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe230066990 0x7fe230168870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:08.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe2363af700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe230135350 0x7fe230064a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:08.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe2363af700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe230152ae0 0x7fe230163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:08.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe2363af700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe230064670 con 0x7fe230066990 2024-01-24T14:13:08.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe2363af700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe230168d60 con 0x7fe230135350 2024-01-24T14:13:08.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe2363af700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300639f0 con 0x7fe230152ae0 2024-01-24T14:13:08.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe22ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe230066990 0x7fe230168870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe22ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe230066990 0x7fe230168870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59280/0 (socket says 172.21.15.29:59280) 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe22ffff700 1 -- 172.21.15.29:0/2464927184 learned_addr learned my addr 172.21.15.29:0/2464927184 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe22f7fe700 1 --2- 172.21.15.29:0/2464927184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe230135350 0x7fe230064a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe23494c700 1 --2- 172.21.15.29:0/2464927184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe230152ae0 0x7fe230163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe22ffff700 1 -- 172.21.15.29:0/2464927184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe230152ae0 msgr2=0x7fe230163fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe22ffff700 1 --2- 172.21.15.29:0/2464927184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe230152ae0 0x7fe230163fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe22ffff700 1 -- 172.21.15.29:0/2464927184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe230135350 msgr2=0x7fe230064a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe22ffff700 1 --2- 172.21.15.29:0/2464927184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe230135350 0x7fe230064a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe22ffff700 1 -- 172.21.15.29:0/2464927184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe220009370 con 0x7fe230066990 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.393+0000 7fe22f7fe700 1 --2- 172.21.15.29:0/2464927184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe230135350 0x7fe230064a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.394+0000 7fe22ffff700 1 --2- 172.21.15.29:0/2464927184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe230066990 0x7fe230168870 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fe220009130 tx=0x7fe220017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.394+0000 7fe217fff700 1 -- 172.21.15.29:0/2464927184 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220000bc0 con 0x7fe230066990 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.394+0000 7fe217fff700 1 -- 172.21.15.29:0/2464927184 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22000b850 con 0x7fe230066990 2024-01-24T14:13:08.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.394+0000 7fe2363af700 1 -- 172.21.15.29:0/2464927184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe230063cd0 con 0x7fe230066990 2024-01-24T14:13:08.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.394+0000 7fe217fff700 1 -- 172.21.15.29:0/2464927184 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220000d20 con 0x7fe230066990 2024-01-24T14:13:08.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.394+0000 7fe2363af700 1 -- 172.21.15.29:0/2464927184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe230064290 con 0x7fe230066990 2024-01-24T14:13:08.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.395+0000 7fe2363af700 1 -- 172.21.15.29:0/2464927184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2300475e0 con 0x7fe230066990 2024-01-24T14:13:08.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.395+0000 7fe217fff700 1 -- 172.21.15.29:0/2464927184 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 22) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fe22001c880 con 0x7fe230066990 2024-01-24T14:13:08.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.396+0000 7fe217fff700 1 -- 172.21.15.29:0/2464927184 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7658+0+0 (secure 0 0 0) 0x7fe22007c160 con 0x7fe230066990 2024-01-24T14:13:08.400 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:08.399+0000 7f0742117680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:13:08.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.400+0000 7fe217fff700 1 -- 172.21.15.29:0/2464927184 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe22004d000 con 0x7fe230066990 2024-01-24T14:13:08.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:08.629+0000 7fe2363af700 1 -- 172.21.15.29:0/2464927184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe230155e10 con 0x7fe230066990 2024-01-24T14:13:08.736 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:08.736+0000 7f0742117680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:13:08.818 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:08.817+0000 7f0742117680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:13:09.102 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-01-24T14:13:09.101+0000 7fe09bfc3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-24T14:13:09.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.105+0000 7fe217fff700 1 -- 172.21.15.29:0/2464927184 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe22002a070 con 0x7fe230066990 2024-01-24T14:13:09.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.108+0000 7fe2363af700 1 -- 172.21.15.29:0/2464927184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe230066990 msgr2=0x7fe230168870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:09.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.108+0000 7fe2363af700 1 --2- 172.21.15.29:0/2464927184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe230066990 0x7fe230168870 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fe220009130 tx=0x7fe220017000).stop 2024-01-24T14:13:09.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.108+0000 7fe2363af700 1 -- 172.21.15.29:0/2464927184 shutdown_connections 2024-01-24T14:13:09.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.108+0000 7fe2363af700 1 --2- 172.21.15.29:0/2464927184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe230152ae0 0x7fe230163fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:09.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.108+0000 7fe2363af700 1 --2- 172.21.15.29:0/2464927184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe230066990 0x7fe230168870 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:09.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.108+0000 7fe2363af700 1 --2- 172.21.15.29:0/2464927184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe230135350 0x7fe230064a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:09.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.108+0000 7fe2363af700 1 -- 172.21.15.29:0/2464927184 >> 172.21.15.29:0/2464927184 conn(0x7fe2300c5640 msgr2=0x7fe2301535b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:09.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.108+0000 7fe2363af700 1 -- 172.21.15.29:0/2464927184 shutdown_connections 2024-01-24T14:13:09.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.108+0000 7fe2363af700 1 -- 172.21.15.29:0/2464927184 wait complete. 2024-01-24T14:13:09.110 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-24T14:13:09.128 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-24T14:13:09.148 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T14:13:09.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.283+0000 7f5077555700 1 Processor -- start 2024-01-24T14:13:09.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.283+0000 7f5077555700 1 -- start start 2024-01-24T14:13:09.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.284+0000 7f5077555700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5070068900 0x7f50700c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:09.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.284+0000 7f5077555700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5070068470 con 0x7f5070134e70 2024-01-24T14:13:09.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.284+0000 7f5077555700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50700c3860 con 0x7f5070066c30 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.284+0000 7f5077555700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507014c910 con 0x7f5070068900 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.284+0000 7f5075af2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5070068900 0x7f50700c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.284+0000 7f5075af2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5070068900 0x7f50700c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59306/0 (socket says 172.21.15.29:59306) 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.284+0000 7f50752f1700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5070134e70 0x7f5070066520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57024/0 (socket says 172.21.15.29:57024) 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.284+0000 7f5075af2700 1 -- 172.21.15.29:0/2492140370 learned_addr learned my addr 172.21.15.29:0/2492140370 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5075af2700 1 -- 172.21.15.29:0/2492140370 >> v1:172.21.15.163:6789/0 conn(0x7f5070066c30 legacy=0x7f50700bdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5075af2700 1 -- 172.21.15.29:0/2492140370 >> v1:172.21.15.29:6789/0 conn(0x7f5070134e70 legacy=0x7f5070066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5075af2700 1 -- 172.21.15.29:0/2492140370 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5070067b40 con 0x7f5070068900 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5063fff700 1 -- 172.21.15.29:0/2492140370 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018589286 0 0) 0x7f5070068470 con 0x7f5070134e70 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5075af2700 1 --2- 172.21.15.29:0/2492140370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5070068900 0x7f50700c31a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f5068009f20 tx=0x7f5068009840).ready entity=mon.0 client_cookie=c43965921794ef5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5063fff700 1 -- 172.21.15.29:0/2492140370 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506800e070 con 0x7f5070068900 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5063fff700 1 -- 172.21.15.29:0/2492140370 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506800ba80 con 0x7f5070068900 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5063fff700 1 -- 172.21.15.29:0/2492140370 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506800be00 con 0x7f5070068900 2024-01-24T14:13:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5077555700 1 -- 172.21.15.29:0/2492140370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5070068900 msgr2=0x7f50700c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:09.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5077555700 1 --2- 172.21.15.29:0/2492140370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5070068900 0x7f50700c31a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f5068009f20 tx=0x7f5068009840).stop 2024-01-24T14:13:09.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5077555700 1 -- 172.21.15.29:0/2492140370 shutdown_connections 2024-01-24T14:13:09.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5077555700 1 --2- 172.21.15.29:0/2492140370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5070068900 0x7f50700c31a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:09.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.285+0000 7f5077555700 1 -- 172.21.15.29:0/2492140370 >> 172.21.15.29:0/2492140370 conn(0x7f50700c8ba0 msgr2=0x7f50700c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:09.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.286+0000 7f5077555700 1 -- 172.21.15.29:0/2492140370 shutdown_connections 2024-01-24T14:13:09.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.286+0000 7f5077555700 1 -- 172.21.15.29:0/2492140370 wait complete. 2024-01-24T14:13:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.287+0000 7f5077555700 1 Processor -- start 2024-01-24T14:13:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.287+0000 7f5077555700 1 -- start start 2024-01-24T14:13:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f5077555700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5070066c30 0x7f50701494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f5077555700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5070068900 0x7f5070121b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f5077555700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5070134e70 0x7f50700c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f5077555700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50700adcd0 con 0x7f5070066c30 2024-01-24T14:13:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f5077555700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50700ad3a0 con 0x7f5070068900 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f5077555700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50700ad680 con 0x7f5070134e70 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f50752f1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5070066c30 0x7f50701494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f50752f1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5070066c30 0x7f50701494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59308/0 (socket says 172.21.15.29:59308) 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f50752f1700 1 -- 172.21.15.29:0/4165822067 learned_addr learned my addr 172.21.15.29:0/4165822067 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f5075af2700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5070134e70 0x7f50700c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f5074af0700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5070068900 0x7f5070121b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f50752f1700 1 -- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5070134e70 msgr2=0x7f50700c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f50752f1700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5070134e70 0x7f50700c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f50752f1700 1 -- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5070068900 msgr2=0x7f5070121b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f50752f1700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5070068900 0x7f5070121b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f50752f1700 1 -- 172.21.15.29:0/4165822067 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5068009ae0 con 0x7f5070066c30 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f5075af2700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5070134e70 0x7f50700c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.288+0000 7f5074af0700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5070068900 0x7f5070121b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.289+0000 7f50752f1700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5070066c30 0x7f50701494d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f506400c490 tx=0x7f5064007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:09.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.289+0000 7f50617fa700 1 -- 172.21.15.29:0/4165822067 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50640072e0 con 0x7f5070066c30 2024-01-24T14:13:09.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.289+0000 7f50617fa700 1 -- 172.21.15.29:0/4165822067 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5064007440 con 0x7f5070066c30 2024-01-24T14:13:09.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.289+0000 7f5077555700 1 -- 172.21.15.29:0/4165822067 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50700ad960 con 0x7f5070066c30 2024-01-24T14:13:09.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.289+0000 7f50617fa700 1 -- 172.21.15.29:0/4165822067 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064005930 con 0x7f5070066c30 2024-01-24T14:13:09.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.289+0000 7f5077555700 1 -- 172.21.15.29:0/4165822067 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50700ad050 con 0x7f5070066c30 2024-01-24T14:13:09.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.290+0000 7f5077555700 1 -- 172.21.15.29:0/4165822067 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50700475e0 con 0x7f5070066c30 2024-01-24T14:13:09.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.290+0000 7f50617fa700 1 -- 172.21.15.29:0/4165822067 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 23) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f5064005d20 con 0x7f5070066c30 2024-01-24T14:13:09.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.291+0000 7f50617fa700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f505405f030 0x7f50540614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:09.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.291+0000 7f50617fa700 1 -- 172.21.15.29:0/4165822067 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7744+0+0 (secure 0 0 0) 0x7f5064079d60 con 0x7f5070066c30 2024-01-24T14:13:09.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.291+0000 7f5074af0700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f505405f030 0x7f50540614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:09.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.291+0000 7f5074af0700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f505405f030 0x7f50540614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f5070149ef0 tx=0x7f506c009040).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:09.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.295+0000 7f50617fa700 1 -- 172.21.15.29:0/4165822067 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f506404ac60 con 0x7f5070066c30 2024-01-24T14:13:09.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:09.520+0000 7f5077555700 1 -- 172.21.15.29:0/4165822067 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5070151f70 con 0x7f5070066c30 2024-01-24T14:13:10.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.114+0000 7f50617fa700 1 -- 172.21.15.29:0/4165822067 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f506404e280 con 0x7f5070066c30 2024-01-24T14:13:10.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.117+0000 7f5077555700 1 -- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f505405f030 msgr2=0x7f50540614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:10.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.117+0000 7f5077555700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f505405f030 0x7f50540614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f5070149ef0 tx=0x7f506c009040).stop 2024-01-24T14:13:10.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.117+0000 7f5077555700 1 -- 172.21.15.29:0/4165822067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5070066c30 msgr2=0x7f50701494d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:10.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.117+0000 7f5077555700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5070066c30 0x7f50701494d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f506400c490 tx=0x7f5064007de0).stop 2024-01-24T14:13:10.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.117+0000 7f5077555700 1 -- 172.21.15.29:0/4165822067 shutdown_connections 2024-01-24T14:13:10.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.117+0000 7f5077555700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5070134e70 0x7f50700c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:10.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.117+0000 7f5077555700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f505405f030 0x7f50540614f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:10.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.117+0000 7f5077555700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5070066c30 0x7f50701494d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:10.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.117+0000 7f5077555700 1 --2- 172.21.15.29:0/4165822067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5070068900 0x7f5070121b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:10.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.117+0000 7f5077555700 1 -- 172.21.15.29:0/4165822067 >> 172.21.15.29:0/4165822067 conn(0x7f50700c8ba0 msgr2=0x7f50700c6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:10.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.117+0000 7f5077555700 1 -- 172.21.15.29:0/4165822067 shutdown_connections 2024-01-24T14:13:10.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.118+0000 7f5077555700 1 -- 172.21.15.29:0/4165822067 wait complete. 2024-01-24T14:13:10.133 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-24T14:13:10.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.279+0000 7fa52a17b700 1 Processor -- start 2024-01-24T14:13:10.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.279+0000 7fa52a17b700 1 -- start start 2024-01-24T14:13:10.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.280+0000 7fa52a17b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5241525c0 0x7fa5240c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:10.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.280+0000 7fa52a17b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa524067f90 con 0x7fa5241530b0 2024-01-24T14:13:10.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.280+0000 7fa52a17b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa524067c60 con 0x7fa524068420 2024-01-24T14:13:10.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.280+0000 7fa52a17b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5240c33a0 con 0x7fa5241525c0 2024-01-24T14:13:10.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.280+0000 7fa523fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa5241530b0 0x7fa5240664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57034/0 (socket says 172.21.15.29:57034) 2024-01-24T14:13:10.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.280+0000 7fa523fff700 1 -- 172.21.15.29:0/383046560 learned_addr learned my addr 172.21.15.29:0/383046560 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:10.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.280+0000 7fa5237fe700 1 --2- 172.21.15.29:0/383046560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5241525c0 0x7fa5240c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:10.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.281+0000 7fa5227fc700 1 -- 172.21.15.29:0/383046560 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452532062 0 0) 0x7fa524067f90 con 0x7fa5241530b0 2024-01-24T14:13:10.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.281+0000 7fa5227fc700 1 -- 172.21.15.29:0/383046560 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa50c0033f0 con 0x7fa5241530b0 2024-01-24T14:13:10.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.281+0000 7fa5227fc700 1 -- 172.21.15.29:0/383046560 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059773915 0 0) 0x7fa524067c60 con 0x7fa524068420 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.281+0000 7fa5227fc700 1 -- 172.21.15.29:0/383046560 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa524067f90 con 0x7fa524068420 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.281+0000 7fa5237fe700 1 -- 172.21.15.29:0/383046560 >> v1:172.21.15.163:6789/0 conn(0x7fa524068420 legacy=0x7fa5241529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.281+0000 7fa5237fe700 1 -- 172.21.15.29:0/383046560 >> v1:172.21.15.29:6789/0 conn(0x7fa5241530b0 legacy=0x7fa5240664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.281+0000 7fa5237fe700 1 -- 172.21.15.29:0/383046560 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5240c3680 con 0x7fa5241525c0 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.281+0000 7fa5237fe700 1 --2- 172.21.15.29:0/383046560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5241525c0 0x7fa5240c4e70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fa514009790 tx=0x7fa514009060).ready entity=mon.2 client_cookie=2b2448fd89d78154 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.281+0000 7fa5227fc700 1 -- 172.21.15.29:0/383046560 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51400f510 con 0x7fa5241525c0 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.281+0000 7fa5227fc700 1 -- 172.21.15.29:0/383046560 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51400f670 con 0x7fa5241525c0 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.281+0000 7fa5227fc700 1 -- 172.21.15.29:0/383046560 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51400f9f0 con 0x7fa5241525c0 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.282+0000 7fa52a17b700 1 -- 172.21.15.29:0/383046560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5241525c0 msgr2=0x7fa5240c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.282+0000 7fa52a17b700 1 --2- 172.21.15.29:0/383046560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5241525c0 0x7fa5240c4e70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fa514009790 tx=0x7fa514009060).stop 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.282+0000 7fa52a17b700 1 -- 172.21.15.29:0/383046560 shutdown_connections 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.282+0000 7fa52a17b700 1 --2- 172.21.15.29:0/383046560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5241525c0 0x7fa5240c4e70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.282+0000 7fa52a17b700 1 -- 172.21.15.29:0/383046560 >> 172.21.15.29:0/383046560 conn(0x7fa52405ecc0 msgr2=0x7fa5240c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:10.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.282+0000 7fa52a17b700 1 -- 172.21.15.29:0/383046560 shutdown_connections 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.282+0000 7fa52a17b700 1 -- 172.21.15.29:0/383046560 wait complete. 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.283+0000 7fa52a17b700 1 Processor -- start 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.283+0000 7fa52a17b700 1 -- start start 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.284+0000 7fa52a17b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa524068420 0x7fa5240c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.284+0000 7fa52a17b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5241525c0 0x7fa5240abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.284+0000 7fa52a17b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5241530b0 0x7fa5241659c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.284+0000 7fa52a17b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa524166800 con 0x7fa524068420 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.284+0000 7fa52a17b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa524135880 con 0x7fa5241525c0 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.284+0000 7fa52a17b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa524135b60 con 0x7fa5241530b0 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.284+0000 7fa5237fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa524068420 0x7fa5240c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.284+0000 7fa5237fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa524068420 0x7fa5240c7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59312/0 (socket says 172.21.15.29:59312) 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.284+0000 7fa5237fe700 1 -- 172.21.15.29:0/3233896163 learned_addr learned my addr 172.21.15.29:0/3233896163 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.284+0000 7fa523fff700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5241530b0 0x7fa5241659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.284+0000 7fa522ffd700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5241525c0 0x7fa5240abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:10.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.285+0000 7fa5237fe700 1 -- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5241530b0 msgr2=0x7fa5241659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.285+0000 7fa5237fe700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5241530b0 0x7fa5241659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.285+0000 7fa5237fe700 1 -- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5241525c0 msgr2=0x7fa5240abbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.285+0000 7fa5237fe700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5241525c0 0x7fa5240abbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.285+0000 7fa5237fe700 1 -- 172.21.15.29:0/3233896163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5140092a0 con 0x7fa524068420 2024-01-24T14:13:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.285+0000 7fa5237fe700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa524068420 0x7fa5240c7200 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fa5240bfba0 tx=0x7fa514017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.285+0000 7fa50b7fe700 1 -- 172.21.15.29:0/3233896163 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51400fe80 con 0x7fa524068420 2024-01-24T14:13:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.285+0000 7fa50b7fe700 1 -- 172.21.15.29:0/3233896163 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51400f7e0 con 0x7fa524068420 2024-01-24T14:13:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.285+0000 7fa52a17b700 1 -- 172.21.15.29:0/3233896163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa524135e40 con 0x7fa524068420 2024-01-24T14:13:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.285+0000 7fa50b7fe700 1 -- 172.21.15.29:0/3233896163 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51401b640 con 0x7fa524068420 2024-01-24T14:13:10.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.285+0000 7fa52a17b700 1 -- 172.21.15.29:0/3233896163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5240c6d00 con 0x7fa524068420 2024-01-24T14:13:10.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.286+0000 7fa52a17b700 1 -- 172.21.15.29:0/3233896163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5240c77b0 con 0x7fa524068420 2024-01-24T14:13:10.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.286+0000 7fa50b7fe700 1 -- 172.21.15.29:0/3233896163 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 23) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fa51401b9b0 con 0x7fa524068420 2024-01-24T14:13:10.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.287+0000 7fa50b7fe700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa51005f030 0x7fa5100614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:10.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.287+0000 7fa50b7fe700 1 -- 172.21.15.29:0/3233896163 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7744+0+0 (secure 0 0 0) 0x7fa51407bd50 con 0x7fa524068420 2024-01-24T14:13:10.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.287+0000 7fa522ffd700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa51005f030 0x7fa5100614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:10.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.288+0000 7fa522ffd700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa51005f030 0x7fa5100614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa5240be370 tx=0x7fa518007bd0).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:10.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.291+0000 7fa50b7fe700 1 -- 172.21.15.29:0/3233896163 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa5240c77b0 con 0x7fa524068420 2024-01-24T14:13:10.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.475+0000 7fa52a17b700 1 -- 172.21.15.29:0/3233896163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa5240c77b0 con 0x7fa524068420 2024-01-24T14:13:10.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.822+0000 7fa50b7fe700 1 -- 172.21.15.29:0/3233896163 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa5240c77b0 con 0x7fa524068420 2024-01-24T14:13:10.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.829+0000 7fa52a17b700 1 -- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa51005f030 msgr2=0x7fa5100614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:10.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.830+0000 7fa52a17b700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa51005f030 0x7fa5100614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa5240be370 tx=0x7fa518007bd0).stop 2024-01-24T14:13:10.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.830+0000 7fa52a17b700 1 -- 172.21.15.29:0/3233896163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa524068420 msgr2=0x7fa5240c7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:10.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.830+0000 7fa52a17b700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa524068420 0x7fa5240c7200 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fa5240bfba0 tx=0x7fa514017000).stop 2024-01-24T14:13:10.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.831+0000 7fa52a17b700 1 -- 172.21.15.29:0/3233896163 shutdown_connections 2024-01-24T14:13:10.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.831+0000 7fa52a17b700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5241530b0 0x7fa5241659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:10.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.831+0000 7fa52a17b700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa51005f030 0x7fa5100614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:10.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.831+0000 7fa52a17b700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa524068420 0x7fa5240c7200 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:10.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.831+0000 7fa52a17b700 1 --2- 172.21.15.29:0/3233896163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5241525c0 0x7fa5240abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:10.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.831+0000 7fa52a17b700 1 -- 172.21.15.29:0/3233896163 >> 172.21.15.29:0/3233896163 conn(0x7fa52405ecc0 msgr2=0x7fa52413b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:10.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.832+0000 7fa52a17b700 1 -- 172.21.15.29:0/3233896163 shutdown_connections 2024-01-24T14:13:10.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:10.833+0000 7fa52a17b700 1 -- 172.21.15.29:0/3233896163 wait complete. 2024-01-24T14:13:10.841 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-01-24T14:13:10.870 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-24T14:13:11.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.019+0000 7ff93a79d700 1 Processor -- start 2024-01-24T14:13:11.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.019+0000 7ff93a79d700 1 -- start start 2024-01-24T14:13:11.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.020+0000 7ff93a79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff934154070 0x7ff9340aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:11.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.020+0000 7ff93a79d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9340bfe90 con 0x7ff934153c90 2024-01-24T14:13:11.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.020+0000 7ff93a79d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9340c0170 con 0x7ff9340c5b30 2024-01-24T14:13:11.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.020+0000 7ff93a79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9341582b0 con 0x7ff934154070 2024-01-24T14:13:11.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.020+0000 7ff93979b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff934153c90 0x7ff934157a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57040/0 (socket says 172.21.15.29:57040) 2024-01-24T14:13:11.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.020+0000 7ff93979b700 1 -- 172.21.15.29:0/3752324595 learned_addr learned my addr 172.21.15.29:0/3752324595 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:11.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.020+0000 7ff938f9a700 1 --2- 172.21.15.29:0/3752324595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff934154070 0x7ff9340aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:11.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.021+0000 7ff92bfff700 1 -- 172.21.15.29:0/3752324595 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196817863 0 0) 0x7ff9340bfe90 con 0x7ff934153c90 2024-01-24T14:13:11.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.021+0000 7ff92bfff700 1 -- 172.21.15.29:0/3752324595 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9200033f0 con 0x7ff934153c90 2024-01-24T14:13:11.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.021+0000 7ff92bfff700 1 -- 172.21.15.29:0/3752324595 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149078781 0 0) 0x7ff9340c0170 con 0x7ff9340c5b30 2024-01-24T14:13:11.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.021+0000 7ff92bfff700 1 -- 172.21.15.29:0/3752324595 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9340bfe90 con 0x7ff9340c5b30 2024-01-24T14:13:11.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.021+0000 7ff938f9a700 1 -- 172.21.15.29:0/3752324595 >> v1:172.21.15.163:6789/0 conn(0x7ff9340c5b30 legacy=0x7ff9340c0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:11.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.021+0000 7ff92bfff700 1 -- 172.21.15.29:0/3752324595 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2766160446 0 0) 0x7ff9200033f0 con 0x7ff934153c90 2024-01-24T14:13:11.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.021+0000 7ff938f9a700 1 -- 172.21.15.29:0/3752324595 >> v1:172.21.15.29:6789/0 conn(0x7ff934153c90 legacy=0x7ff934157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:11.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.021+0000 7ff938f9a700 1 -- 172.21.15.29:0/3752324595 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff934158590 con 0x7ff934154070 2024-01-24T14:13:11.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.021+0000 7ff938f9a700 1 --2- 172.21.15.29:0/3752324595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff934154070 0x7ff9340aa0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7ff924009790 tx=0x7ff924009060).ready entity=mon.2 client_cookie=e7361fc187e8f1bc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:11.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.021+0000 7ff92bfff700 1 -- 172.21.15.29:0/3752324595 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930001470 con 0x7ff934154070 2024-01-24T14:13:11.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.021+0000 7ff92bfff700 1 -- 172.21.15.29:0/3752324595 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff924011400 con 0x7ff934154070 2024-01-24T14:13:11.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.021+0000 7ff92bfff700 1 -- 172.21.15.29:0/3752324595 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924011780 con 0x7ff934154070 2024-01-24T14:13:11.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.022+0000 7ff93a79d700 1 -- 172.21.15.29:0/3752324595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff934154070 msgr2=0x7ff9340aa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:11.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.022+0000 7ff93a79d700 1 --2- 172.21.15.29:0/3752324595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff934154070 0x7ff9340aa0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7ff924009790 tx=0x7ff924009060).stop 2024-01-24T14:13:11.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.022+0000 7ff93a79d700 1 -- 172.21.15.29:0/3752324595 shutdown_connections 2024-01-24T14:13:11.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.022+0000 7ff93a79d700 1 --2- 172.21.15.29:0/3752324595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff934154070 0x7ff9340aa0a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:11.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.022+0000 7ff93a79d700 1 -- 172.21.15.29:0/3752324595 >> 172.21.15.29:0/3752324595 conn(0x7ff93405ecc0 msgr2=0x7ff9340cac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:11.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.022+0000 7ff93a79d700 1 -- 172.21.15.29:0/3752324595 shutdown_connections 2024-01-24T14:13:11.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.022+0000 7ff93a79d700 1 -- 172.21.15.29:0/3752324595 wait complete. 2024-01-24T14:13:11.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.023+0000 7ff93a79d700 1 Processor -- start 2024-01-24T14:13:11.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.023+0000 7ff93a79d700 1 -- start start 2024-01-24T14:13:11.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.024+0000 7ff93a79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340c5b30 0x7ff93414aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:11.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.024+0000 7ff93a79d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff934153c90 0x7ff934165a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:11.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.024+0000 7ff93a79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff934154070 0x7ff934165f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:11.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.024+0000 7ff93a79d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9341560e0 con 0x7ff934153c90 2024-01-24T14:13:11.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.024+0000 7ff93a79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9341554d0 con 0x7ff934154070 2024-01-24T14:13:11.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.024+0000 7ff93a79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9341557b0 con 0x7ff9340c5b30 2024-01-24T14:13:11.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.024+0000 7ff938f9a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff934153c90 0x7ff934165a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:11.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.024+0000 7ff938f9a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff934153c90 0x7ff934165a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59326/0 (socket says 172.21.15.29:59326) 2024-01-24T14:13:11.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.024+0000 7ff938f9a700 1 -- 172.21.15.29:0/539539669 learned_addr learned my addr 172.21.15.29:0/539539669 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:11.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff93979b700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340c5b30 0x7ff93414aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:11.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff939f9c700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff934154070 0x7ff934165f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:11.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff938f9a700 1 -- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340c5b30 msgr2=0x7ff93414aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:11.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff938f9a700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340c5b30 0x7ff93414aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:11.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff938f9a700 1 -- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff934154070 msgr2=0x7ff934165f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:11.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff938f9a700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff934154070 0x7ff934165f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:11.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff938f9a700 1 -- 172.21.15.29:0/539539669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9240092a0 con 0x7ff934153c90 2024-01-24T14:13:11.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff939f9c700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff934154070 0x7ff934165f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:11.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff938f9a700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff934153c90 0x7ff934165a30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7ff924010000 tx=0x7ff924016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:11.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff9297fa700 1 -- 172.21.15.29:0/539539669 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924021070 con 0x7ff934153c90 2024-01-24T14:13:11.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff9297fa700 1 -- 172.21.15.29:0/539539669 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9240107c0 con 0x7ff934153c90 2024-01-24T14:13:11.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff93a79d700 1 -- 172.21.15.29:0/539539669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff934155a90 con 0x7ff934153c90 2024-01-24T14:13:11.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff9297fa700 1 -- 172.21.15.29:0/539539669 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924010b80 con 0x7ff934153c90 2024-01-24T14:13:11.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.025+0000 7ff93a79d700 1 -- 172.21.15.29:0/539539669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93414af20 con 0x7ff934153c90 2024-01-24T14:13:11.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.026+0000 7ff93a79d700 1 -- 172.21.15.29:0/539539669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff934083c80 con 0x7ff934153c90 2024-01-24T14:13:11.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.027+0000 7ff9297fa700 1 -- 172.21.15.29:0/539539669 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 23) v1 ==== 75340+0+0 (secure 0 0 0) 0x7ff92401d430 con 0x7ff934153c90 2024-01-24T14:13:11.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.027+0000 7ff9297fa700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff91805f030 0x7ff9180614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:11.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.027+0000 7ff9297fa700 1 -- 172.21.15.29:0/539539669 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7248+0+0 (secure 0 0 0) 0x7ff92407b630 con 0x7ff934153c90 2024-01-24T14:13:11.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.027+0000 7ff93979b700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff91805f030 0x7ff9180614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:11.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.028+0000 7ff93979b700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff91805f030 0x7ff9180614f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7ff930001ed0 tx=0x7ff930000bc0).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:11.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.032+0000 7ff9297fa700 1 -- 172.21.15.29:0/539539669 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff92404c610 con 0x7ff934153c90 2024-01-24T14:13:11.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.121+0000 7ff9297fa700 1 -- 172.21.15.29:0/539539669 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff924079170 con 0x7ff934153c90 2024-01-24T14:13:11.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:11.220+0000 7ff93a79d700 1 -- 172.21.15.29:0/539539669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff934156220 con 0x7ff934153c90 2024-01-24T14:13:12.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.122+0000 7ff9297fa700 1 -- 172.21.15.29:0/539539669 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff924041580 con 0x7ff934153c90 2024-01-24T14:13:12.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.126+0000 7ff93a79d700 1 -- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff91805f030 msgr2=0x7ff9180614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.126+0000 7ff93a79d700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff91805f030 0x7ff9180614f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7ff930001ed0 tx=0x7ff930000bc0).stop 2024-01-24T14:13:12.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.126+0000 7ff93a79d700 1 -- 172.21.15.29:0/539539669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff934153c90 msgr2=0x7ff934165a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.126+0000 7ff93a79d700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff934153c90 0x7ff934165a30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7ff924010000 tx=0x7ff924016000).stop 2024-01-24T14:13:12.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.126+0000 7ff93a79d700 1 -- 172.21.15.29:0/539539669 shutdown_connections 2024-01-24T14:13:12.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.126+0000 7ff93a79d700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff934154070 0x7ff934165f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.126+0000 7ff93a79d700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340c5b30 0x7ff93414aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.126+0000 7ff93a79d700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff91805f030 0x7ff9180614f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.126+0000 7ff93a79d700 1 --2- 172.21.15.29:0/539539669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff934153c90 0x7ff934165a30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.126+0000 7ff93a79d700 1 -- 172.21.15.29:0/539539669 >> 172.21.15.29:0/539539669 conn(0x7ff93405ecc0 msgr2=0x7ff93414eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:12.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.127+0000 7ff93a79d700 1 -- 172.21.15.29:0/539539669 shutdown_connections 2024-01-24T14:13:12.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.127+0000 7ff93a79d700 1 -- 172.21.15.29:0/539539669 wait complete. 2024-01-24T14:13:12.129 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-01-24T14:13:12.143 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-24T14:13:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.279+0000 7fb4b805f700 1 Processor -- start 2024-01-24T14:13:12.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.279+0000 7fb4b805f700 1 -- start start 2024-01-24T14:13:12.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.280+0000 7fb4b805f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb4b015d040 0x7fb4b015f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.280+0000 7fb4b805f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b00c6820 con 0x7fb4b0064740 2024-01-24T14:13:12.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.280+0000 7fb4b805f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b0155220 con 0x7fb4b00c5af0 2024-01-24T14:13:12.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.280+0000 7fb4b805f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b01556b0 con 0x7fb4b015d040 2024-01-24T14:13:12.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.280+0000 7fb4b5dfb700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb4b0064740 0x7fb4b0064b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57048/0 (socket says 172.21.15.29:57048) 2024-01-24T14:13:12.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.280+0000 7fb4b5dfb700 1 -- 172.21.15.29:0/667273534 learned_addr learned my addr 172.21.15.29:0/667273534 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:12.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.280+0000 7fb4b65fc700 1 --2- 172.21.15.29:0/667273534 >> v2:172.21.15.163:3300/0 conn(0x7fb4b015d040 0x7fb4b015f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549732547 0 0) 0x7fb4a4001110 con 0x7fb4b0064740 2024-01-24T14:13:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb49c0033f0 con 0x7fb4b0064740 2024-01-24T14:13:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4145723591 0 0) 0x7fb49c0033f0 con 0x7fb4b0064740 2024-01-24T14:13:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 >> v2:172.21.15.163:3300/0 conn(0x7fb4b015d040 msgr2=0x7fb4b015f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 --2- 172.21.15.29:0/667273534 >> v2:172.21.15.163:3300/0 conn(0x7fb4b015d040 0x7fb4b015f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 >> v1:172.21.15.163:6789/0 conn(0x7fb4b00c5af0 legacy=0x7fb4b00c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b0154680 con 0x7fb4b0064740 2024-01-24T14:13:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fb4a4001620 con 0x7fb4b0064740 2024-01-24T14:13:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 >> v1:172.21.15.29:6789/0 conn(0x7fb4b0064740 legacy=0x7fb4b0064b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 --2- 172.21.15.29:0/667273534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49c003930 0x7fb49c003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 --2- 172.21.15.29:0/667273534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb49c004690 0x7fb49c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 --2- 172.21.15.29:0/667273534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb49c005000 0x7fb49c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b01556b0 con 0x7fb49c004690 2024-01-24T14:13:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c00bfe0 con 0x7fb49c005000 2024-01-24T14:13:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c00c1a0 con 0x7fb49c003930 2024-01-24T14:13:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b55fa700 1 --2- 172.21.15.29:0/667273534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb49c004690 0x7fb49c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b5dfb700 1 --2- 172.21.15.29:0/667273534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49c003930 0x7fb49c003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b65fc700 1 --2- 172.21.15.29:0/667273534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb49c005000 0x7fb49c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b55fa700 1 -- 172.21.15.29:0/667273534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49c003930 msgr2=0x7fb49c003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:12.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b55fa700 1 --2- 172.21.15.29:0/667273534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49c003930 0x7fb49c003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b55fa700 1 -- 172.21.15.29:0/667273534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb49c005000 msgr2=0x7fb49c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b55fa700 1 --2- 172.21.15.29:0/667273534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb49c005000 0x7fb49c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b55fa700 1 -- 172.21.15.29:0/667273534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c0033f0 con 0x7fb49c004690 2024-01-24T14:13:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.281+0000 7fb4b65fc700 1 -- 172.21.15.29:0/667273534 reap_dead start 2024-01-24T14:13:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.282+0000 7fb4b55fa700 1 --2- 172.21.15.29:0/667273534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb49c004690 0x7fb49c004b10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fb4ac001730 tx=0x7fb4ac007bb0).ready entity=mon.0 client_cookie=f12f224cd6311aca server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.282+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac007020 con 0x7fb49c004690 2024-01-24T14:13:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.282+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac007180 con 0x7fb49c004690 2024-01-24T14:13:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.282+0000 7fb4b4df9700 1 -- 172.21.15.29:0/667273534 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac007520 con 0x7fb49c004690 2024-01-24T14:13:12.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.282+0000 7fb4b805f700 1 -- 172.21.15.29:0/667273534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb49c004690 msgr2=0x7fb49c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.282+0000 7fb4b805f700 1 --2- 172.21.15.29:0/667273534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb49c004690 0x7fb49c004b10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fb4ac001730 tx=0x7fb4ac007bb0).stop 2024-01-24T14:13:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.282+0000 7fb4b805f700 1 -- 172.21.15.29:0/667273534 shutdown_connections 2024-01-24T14:13:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.282+0000 7fb4b805f700 1 --2- 172.21.15.29:0/667273534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb49c004690 0x7fb49c004b10 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.282+0000 7fb4b805f700 1 -- 172.21.15.29:0/667273534 >> 172.21.15.29:0/667273534 conn(0x7fb4b00cb890 msgr2=0x7fb4b00c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.283+0000 7fb4b805f700 1 -- 172.21.15.29:0/667273534 shutdown_connections 2024-01-24T14:13:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.283+0000 7fb4b805f700 1 -- 172.21.15.29:0/667273534 wait complete. 2024-01-24T14:13:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.284+0000 7fb4b805f700 1 Processor -- start 2024-01-24T14:13:12.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.284+0000 7fb4b805f700 1 -- start start 2024-01-24T14:13:12.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b805f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49c004690 0x7fb4b00aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b805f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4b01504c0 0x7fb4b014e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b805f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4b014ea60 0x7fb4b014eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b805f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00abff0 con 0x7fb4b014ea60 2024-01-24T14:13:12.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b805f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b014f3d0 con 0x7fb4b01504c0 2024-01-24T14:13:12.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b805f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00c3280 con 0x7fb49c004690 2024-01-24T14:13:12.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b65fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4b014ea60 0x7fb4b014eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b65fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4b014ea60 0x7fb4b014eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59352/0 (socket says 172.21.15.29:59352) 2024-01-24T14:13:12.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b65fc700 1 -- 172.21.15.29:0/298667334 learned_addr learned my addr 172.21.15.29:0/298667334 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:12.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b5dfb700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49c004690 0x7fb4b00aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b55fa700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4b01504c0 0x7fb4b014e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b65fc700 1 -- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49c004690 msgr2=0x7fb4b00aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b65fc700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49c004690 0x7fb4b00aae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b65fc700 1 -- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4b01504c0 msgr2=0x7fb4b014e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b65fc700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4b01504c0 0x7fb4b014e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b65fc700 1 -- 172.21.15.29:0/298667334 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b00ab750 con 0x7fb4b014ea60 2024-01-24T14:13:12.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b5dfb700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49c004690 0x7fb4b00aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:12.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b55fa700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4b01504c0 0x7fb4b014e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:12.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b65fc700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4b014ea60 0x7fb4b014eee0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fb4b0065250 tx=0x7fb4a800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:12.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4a1ffb700 1 -- 172.21.15.29:0/298667334 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a800e040 con 0x7fb4b014ea60 2024-01-24T14:13:12.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4a1ffb700 1 -- 172.21.15.29:0/298667334 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a800f040 con 0x7fb4b014ea60 2024-01-24T14:13:12.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4a1ffb700 1 -- 172.21.15.29:0/298667334 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a800fdf0 con 0x7fb4b014ea60 2024-01-24T14:13:12.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.285+0000 7fb4b805f700 1 -- 172.21.15.29:0/298667334 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b00aba30 con 0x7fb4b014ea60 2024-01-24T14:13:12.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.286+0000 7fb4b805f700 1 -- 172.21.15.29:0/298667334 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b00c3520 con 0x7fb4b014ea60 2024-01-24T14:13:12.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.287+0000 7fb4b805f700 1 -- 172.21.15.29:0/298667334 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b00ad860 con 0x7fb4b014ea60 2024-01-24T14:13:12.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.287+0000 7fb4a1ffb700 1 -- 172.21.15.29:0/298667334 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb4a8015660 con 0x7fb4b014ea60 2024-01-24T14:13:12.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.287+0000 7fb4a1ffb700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb49405f270 0x7fb494061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.288+0000 7fb4a1ffb700 1 -- 172.21.15.29:0/298667334 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6806+0+0 (secure 0 0 0) 0x7fb4a807b210 con 0x7fb4b014ea60 2024-01-24T14:13:12.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.288+0000 7fb4b5dfb700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb49405f270 0x7fb494061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.288+0000 7fb4b5dfb700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb49405f270 0x7fb494061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fb4a4001dc0 tx=0x7fb4a4009040).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:12.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.292+0000 7fb4a1ffb700 1 -- 172.21.15.29:0/298667334 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb4a804c0b0 con 0x7fb4b014ea60 2024-01-24T14:13:12.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.475+0000 7fb4b805f700 1 -- 172.21.15.29:0/298667334 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb4b00c3740 con 0x7fb4b014ea60 2024-01-24T14:13:12.479 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.478+0000 7fb4a1ffb700 1 -- 172.21.15.29:0/298667334 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb4a804f6d0 con 0x7fb4b014ea60 2024-01-24T14:13:12.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.481+0000 7fb4b805f700 1 -- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb49405f270 msgr2=0x7fb494061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.481+0000 7fb4b805f700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb49405f270 0x7fb494061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fb4a4001dc0 tx=0x7fb4a4009040).stop 2024-01-24T14:13:12.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.481+0000 7fb4b805f700 1 -- 172.21.15.29:0/298667334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4b014ea60 msgr2=0x7fb4b014eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.481+0000 7fb4b805f700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4b014ea60 0x7fb4b014eee0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fb4b0065250 tx=0x7fb4a800b040).stop 2024-01-24T14:13:12.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.481+0000 7fb4b805f700 1 -- 172.21.15.29:0/298667334 shutdown_connections 2024-01-24T14:13:12.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.481+0000 7fb4b805f700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb49405f270 0x7fb494061730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.481+0000 7fb4b805f700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4b014ea60 0x7fb4b014eee0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.481+0000 7fb4b805f700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49c004690 0x7fb4b00aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.481+0000 7fb4b805f700 1 --2- 172.21.15.29:0/298667334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4b01504c0 0x7fb4b014e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.481+0000 7fb4b805f700 1 -- 172.21.15.29:0/298667334 >> 172.21.15.29:0/298667334 conn(0x7fb4b00cb890 msgr2=0x7fb4b0063140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:12.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.481+0000 7fb4b805f700 1 -- 172.21.15.29:0/298667334 shutdown_connections 2024-01-24T14:13:12.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.481+0000 7fb4b805f700 1 -- 172.21.15.29:0/298667334 wait complete. 2024-01-24T14:13:12.496 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-01-24T14:13:12.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.629+0000 7f4c4b533700 1 Processor -- start 2024-01-24T14:13:12.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.629+0000 7f4c4b533700 1 -- start start 2024-01-24T14:13:12.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.630+0000 7f4c4b533700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4c44141c00 0x7f4c44065420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.630+0000 7f4c4b533700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c44062ec0 0x7f4c44069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.630+0000 7f4c4b533700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4415cf20 con 0x7f4c44141fe0 2024-01-24T14:13:12.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.630+0000 7f4c4b533700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4415ca60 con 0x7f4c44141c00 2024-01-24T14:13:12.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.630+0000 7f4c4b533700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440c8440 con 0x7f4c44062ec0 2024-01-24T14:13:12.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.630+0000 7f4c48ace700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4c44141fe0 0x7f4c440627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57054/0 (socket says 172.21.15.29:57054) 2024-01-24T14:13:12.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.630+0000 7f4c48ace700 1 -- 172.21.15.29:0/1498517611 learned_addr learned my addr 172.21.15.29:0/1498517611 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:12.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.630+0000 7f4c49ad0700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c44062ec0 0x7f4c44069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.630+0000 7f4c492cf700 1 --2- 172.21.15.29:0/1498517611 >> v2:172.21.15.163:3300/0 conn(0x7f4c44141c00 0x7f4c44065420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282569889 0 0) 0x7f4c4415cf20 con 0x7f4c44141fe0 2024-01-24T14:13:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c300033f0 con 0x7f4c44141fe0 2024-01-24T14:13:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 18999558 0 0) 0x7f4c300033f0 con 0x7f4c44141fe0 2024-01-24T14:13:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c44062ec0 msgr2=0x7f4c44069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:12.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c44062ec0 0x7f4c44069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 >> v2:172.21.15.163:3300/0 conn(0x7f4c44141c00 msgr2=0x7f4c44065420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:12.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 --2- 172.21.15.29:0/1498517611 >> v2:172.21.15.163:3300/0 conn(0x7f4c44141c00 0x7f4c44065420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4415cc40 con 0x7f4c44141fe0 2024-01-24T14:13:12.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f4c40001470 con 0x7f4c44141fe0 2024-01-24T14:13:12.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 >> v1:172.21.15.29:6789/0 conn(0x7f4c44141fe0 legacy=0x7f4c440627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c300038e0 0x7f4c30003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c30004690 0x7f4c30004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c30005000 0x7f4c3000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4415ca60 con 0x7f4c300038e0 2024-01-24T14:13:12.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440c8440 con 0x7f4c30004690 2024-01-24T14:13:12.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3000bdd0 con 0x7f4c30005000 2024-01-24T14:13:12.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c492cf700 1 --2- 172.21.15.29:0/1498517611 >> v2:172.21.15.163:3300/0 conn(0x7f4c44141c00 0x7f4c44065420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:12.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c49ad0700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c44062ec0 0x7f4c44069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:12.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c492cf700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c300038e0 0x7f4c30003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c48ace700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c30004690 0x7f4c30004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c49ad0700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c30005000 0x7f4c3000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c492cf700 1 -- 172.21.15.29:0/1498517611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c30005000 msgr2=0x7f4c3000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:12.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.631+0000 7f4c492cf700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c30005000 0x7f4c3000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.632+0000 7f4c492cf700 1 -- 172.21.15.29:0/1498517611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c30004690 msgr2=0x7f4c30004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:12.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.632+0000 7f4c492cf700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c30004690 0x7f4c30004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.632+0000 7f4c492cf700 1 -- 172.21.15.29:0/1498517611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c300033f0 con 0x7f4c300038e0 2024-01-24T14:13:12.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.632+0000 7f4c492cf700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c300038e0 0x7f4c30003d00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f4c38009c60 tx=0x7f4c38009060).ready entity=mon.0 client_cookie=b50f79534191938d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:12.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.632+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380095c0 con 0x7f4c300038e0 2024-01-24T14:13:12.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.632+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3800e710 con 0x7f4c300038e0 2024-01-24T14:13:12.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.632+0000 7f4c37fff700 1 -- 172.21.15.29:0/1498517611 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800ea90 con 0x7f4c300038e0 2024-01-24T14:13:12.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.632+0000 7f4c4b533700 1 -- 172.21.15.29:0/1498517611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c300038e0 msgr2=0x7f4c30003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.632+0000 7f4c4b533700 1 --2- 172.21.15.29:0/1498517611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c300038e0 0x7f4c30003d00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f4c38009c60 tx=0x7f4c38009060).stop 2024-01-24T14:13:12.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.632+0000 7f4c49ad0700 1 -- 172.21.15.29:0/1498517611 reap_dead start 2024-01-24T14:13:12.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.632+0000 7f4c4b533700 1 -- 172.21.15.29:0/1498517611 shutdown_connections 2024-01-24T14:13:12.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.632+0000 7f4c4b533700 1 -- 172.21.15.29:0/1498517611 >> 172.21.15.29:0/1498517611 conn(0x7f4c44064390 msgr2=0x7f4c440c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:12.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.633+0000 7f4c4b533700 1 -- 172.21.15.29:0/1498517611 shutdown_connections 2024-01-24T14:13:12.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.633+0000 7f4c4b533700 1 -- 172.21.15.29:0/1498517611 wait complete. 2024-01-24T14:13:12.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.634+0000 7f4c4b533700 1 Processor -- start 2024-01-24T14:13:12.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.634+0000 7f4c4b533700 1 -- start start 2024-01-24T14:13:12.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c4b533700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c300038e0 0x7f4c44157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c4b533700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c440beb70 0x7f4c440c1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c4b533700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c441587c0 0x7f4c44158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c4b533700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440aca00 con 0x7f4c440beb70 2024-01-24T14:13:12.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c4b533700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4413f330 con 0x7f4c441587c0 2024-01-24T14:13:12.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c4b533700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c441583d0 con 0x7f4c300038e0 2024-01-24T14:13:12.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c48ace700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c440beb70 0x7f4c440c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c48ace700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c440beb70 0x7f4c440c1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59370/0 (socket says 172.21.15.29:59370) 2024-01-24T14:13:12.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c48ace700 1 -- 172.21.15.29:0/2164170674 learned_addr learned my addr 172.21.15.29:0/2164170674 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:12.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c492cf700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c300038e0 0x7f4c44157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c49ad0700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c441587c0 0x7f4c44158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c48ace700 1 -- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c300038e0 msgr2=0x7f4c44157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c48ace700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c300038e0 0x7f4c44157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c48ace700 1 -- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c441587c0 msgr2=0x7f4c44158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c48ace700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c441587c0 0x7f4c44158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c48ace700 1 -- 172.21.15.29:0/2164170674 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c44159110 con 0x7f4c440beb70 2024-01-24T14:13:12.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.635+0000 7f4c48ace700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c440beb70 0x7f4c440c1390 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f4c400016e0 tx=0x7f4c40001e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:12.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.636+0000 7f4c357fa700 1 -- 172.21.15.29:0/2164170674 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40011590 con 0x7f4c440beb70 2024-01-24T14:13:12.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.636+0000 7f4c357fa700 1 -- 172.21.15.29:0/2164170674 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c400116f0 con 0x7f4c440beb70 2024-01-24T14:13:12.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.636+0000 7f4c4b533700 1 -- 172.21.15.29:0/2164170674 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c440c66b0 con 0x7f4c440beb70 2024-01-24T14:13:12.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.636+0000 7f4c357fa700 1 -- 172.21.15.29:0/2164170674 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40011a70 con 0x7f4c440beb70 2024-01-24T14:13:12.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.636+0000 7f4c4b533700 1 -- 172.21.15.29:0/2164170674 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c440ac690 con 0x7f4c440beb70 2024-01-24T14:13:12.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.637+0000 7f4c4b533700 1 -- 172.21.15.29:0/2164170674 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c440475e0 con 0x7f4c440beb70 2024-01-24T14:13:12.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.637+0000 7f4c357fa700 1 -- 172.21.15.29:0/2164170674 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4c4001d3b0 con 0x7f4c440beb70 2024-01-24T14:13:12.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.638+0000 7f4c357fa700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c2805f270 0x7f4c28061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.638+0000 7f4c357fa700 1 -- 172.21.15.29:0/2164170674 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6806+0+0 (secure 0 0 0) 0x7f4c40079f40 con 0x7f4c440beb70 2024-01-24T14:13:12.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.638+0000 7f4c492cf700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c2805f270 0x7f4c28061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.638+0000 7f4c492cf700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c2805f270 0x7f4c28061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4c3800a010 tx=0x7f4c38019000).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:12.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.642+0000 7f4c357fa700 1 -- 172.21.15.29:0/2164170674 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4c4004ad70 con 0x7f4c440beb70 2024-01-24T14:13:12.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.835+0000 7f4c4b533700 1 -- 172.21.15.29:0/2164170674 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f4c440acb40 con 0x7f4c440beb70 2024-01-24T14:13:12.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.838+0000 7f4c357fa700 1 -- 172.21.15.29:0/2164170674 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v9) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4c40019070 con 0x7f4c440beb70 2024-01-24T14:13:12.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.841+0000 7f4c4b533700 1 -- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c2805f270 msgr2=0x7f4c28061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.841+0000 7f4c4b533700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c2805f270 0x7f4c28061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4c3800a010 tx=0x7f4c38019000).stop 2024-01-24T14:13:12.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.841+0000 7f4c4b533700 1 -- 172.21.15.29:0/2164170674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c440beb70 msgr2=0x7f4c440c1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.841+0000 7f4c4b533700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c440beb70 0x7f4c440c1390 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f4c400016e0 tx=0x7f4c40001e90).stop 2024-01-24T14:13:12.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.841+0000 7f4c4b533700 1 -- 172.21.15.29:0/2164170674 shutdown_connections 2024-01-24T14:13:12.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.841+0000 7f4c4b533700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c441587c0 0x7f4c44158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.841+0000 7f4c4b533700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c300038e0 0x7f4c44157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.841+0000 7f4c4b533700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c2805f270 0x7f4c28061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.841+0000 7f4c4b533700 1 --2- 172.21.15.29:0/2164170674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c440beb70 0x7f4c440c1390 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.841+0000 7f4c4b533700 1 -- 172.21.15.29:0/2164170674 >> 172.21.15.29:0/2164170674 conn(0x7f4c44064390 msgr2=0x7f4c44140e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:12.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.841+0000 7f4c4b533700 1 -- 172.21.15.29:0/2164170674 shutdown_connections 2024-01-24T14:13:12.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.841+0000 7f4c4b533700 1 -- 172.21.15.29:0/2164170674 wait complete. 2024-01-24T14:13:12.855 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2024-01-24T14:13:12.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.985+0000 7fa051da0700 1 Processor -- start 2024-01-24T14:13:12.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.985+0000 7fa051da0700 1 -- start start 2024-01-24T14:13:12.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa051da0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa04c155b00 0x7fa04c0664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa051da0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa04c1565f0 0x7fa04c0bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa051da0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa04c066e90 con 0x7fa04c0669e0 2024-01-24T14:13:12.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa051da0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04c0c5500 con 0x7fa04c155b00 2024-01-24T14:13:12.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa051da0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c0c3910 con 0x7fa04c1565f0 2024-01-24T14:13:12.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa04b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa04c155b00 0x7fa04c0664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa04b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa04c155b00 0x7fa04c0664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59378/0 (socket says 172.21.15.29:59378) 2024-01-24T14:13:12.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa04b7fe700 1 -- 172.21.15.29:0/3231829025 learned_addr learned my addr 172.21.15.29:0/3231829025 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:12.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa04bfff700 1 --2- 172.21.15.29:0/3231829025 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa04c1565f0 0x7fa04c0bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa04b7fe700 1 -- 172.21.15.29:0/3231829025 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa04c1565f0 msgr2=0x7fa04c0bdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa04b7fe700 1 --2- 172.21.15.29:0/3231829025 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa04c1565f0 0x7fa04c0bdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa04b7fe700 1 -- 172.21.15.29:0/3231829025 >> v1:172.21.15.163:6789/0 conn(0x7fa04c0669e0 legacy=0x7fa04c155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa04b7fe700 1 -- 172.21.15.29:0/3231829025 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c0c5220 con 0x7fa04c155b00 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa04bfff700 1 --2- 172.21.15.29:0/3231829025 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa04c1565f0 0x7fa04c0bdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa04b7fe700 1 --2- 172.21.15.29:0/3231829025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa04c155b00 0x7fa04c0664f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fa03c006fd0 tx=0x7fa03c0090f0).ready entity=mon.0 client_cookie=85a199e9d27b4e9c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.986+0000 7fa04a7fc700 1 -- 172.21.15.29:0/3231829025 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c00b600 con 0x7fa04c155b00 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.987+0000 7fa04a7fc700 1 -- 172.21.15.29:0/3231829025 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c00b760 con 0x7fa04c155b00 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.987+0000 7fa04a7fc700 1 -- 172.21.15.29:0/3231829025 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c00bae0 con 0x7fa04c155b00 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.987+0000 7fa051da0700 1 -- 172.21.15.29:0/3231829025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa04c155b00 msgr2=0x7fa04c0664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.987+0000 7fa051da0700 1 --2- 172.21.15.29:0/3231829025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa04c155b00 0x7fa04c0664f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fa03c006fd0 tx=0x7fa03c0090f0).stop 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.987+0000 7fa051da0700 1 -- 172.21.15.29:0/3231829025 shutdown_connections 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.987+0000 7fa051da0700 1 --2- 172.21.15.29:0/3231829025 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa04c1565f0 0x7fa04c0bdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.987+0000 7fa051da0700 1 --2- 172.21.15.29:0/3231829025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa04c155b00 0x7fa04c0664f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.987+0000 7fa051da0700 1 -- 172.21.15.29:0/3231829025 >> 172.21.15.29:0/3231829025 conn(0x7fa04c0c8bb0 msgr2=0x7fa04c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:12.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.987+0000 7fa051da0700 1 -- 172.21.15.29:0/3231829025 shutdown_connections 2024-01-24T14:13:12.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.987+0000 7fa051da0700 1 -- 172.21.15.29:0/3231829025 wait complete. 2024-01-24T14:13:12.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.989+0000 7fa051da0700 1 Processor -- start 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.989+0000 7fa051da0700 1 -- start start 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.989+0000 7fa051da0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa04c0669e0 0x7fa04c0c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.989+0000 7fa051da0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa04c155b00 0x7fa04c0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.989+0000 7fa051da0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa04c1565f0 0x7fa04c154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.989+0000 7fa051da0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c0c8780 con 0x7fa04c1565f0 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.989+0000 7fa051da0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c154d80 con 0x7fa04c0669e0 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.989+0000 7fa051da0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c155060 con 0x7fa04c155b00 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa04bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa04c1565f0 0x7fa04c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa04bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa04c1565f0 0x7fa04c154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59386/0 (socket says 172.21.15.29:59386) 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa04bfff700 1 -- 172.21.15.29:0/436313814 learned_addr learned my addr 172.21.15.29:0/436313814 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa04b7fe700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa04c0669e0 0x7fa04c0c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa04affd700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa04c155b00 0x7fa04c0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa04bfff700 1 -- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa04c155b00 msgr2=0x7fa04c0c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa04bfff700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa04c155b00 0x7fa04c0c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa04bfff700 1 -- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa04c0669e0 msgr2=0x7fa04c0c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa04bfff700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa04c0669e0 0x7fa04c0c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa04bfff700 1 -- 172.21.15.29:0/436313814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c009370 con 0x7fa04c1565f0 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa04bfff700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa04c1565f0 0x7fa04c154050 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fa044009fc0 tx=0x7fa0440099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa0337fe700 1 -- 172.21.15.29:0/436313814 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04400dd40 con 0x7fa04c1565f0 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa0337fe700 1 -- 172.21.15.29:0/436313814 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04400f040 con 0x7fa04c1565f0 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa051da0700 1 -- 172.21.15.29:0/436313814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04c12b320 con 0x7fa04c1565f0 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa0337fe700 1 -- 172.21.15.29:0/436313814 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04400dea0 con 0x7fa04c1565f0 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.990+0000 7fa051da0700 1 -- 172.21.15.29:0/436313814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04c155340 con 0x7fa04c1565f0 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.991+0000 7fa051da0700 1 -- 172.21.15.29:0/436313814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04c12b840 con 0x7fa04c1565f0 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.992+0000 7fa0337fe700 1 -- 172.21.15.29:0/436313814 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa044013890 con 0x7fa04c1565f0 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.992+0000 7fa0337fe700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa03805f240 0x7fa038061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:12.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.992+0000 7fa0337fe700 1 -- 172.21.15.29:0/436313814 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6806+0+0 (secure 0 0 0) 0x7fa04407ae20 con 0x7fa04c1565f0 2024-01-24T14:13:12.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.992+0000 7fa04b7fe700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa03805f240 0x7fa038061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:12.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.993+0000 7fa04b7fe700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa03805f240 0x7fa038061700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa03c009130 tx=0x7fa03c017000).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:12.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:12.996+0000 7fa0337fe700 1 -- 172.21.15.29:0/436313814 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa04c12b840 con 0x7fa04c1565f0 2024-01-24T14:13:13.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:13.249+0000 7fa051da0700 1 -- 172.21.15.29:0/436313814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fa04c12b840 con 0x7fa04c1565f0 2024-01-24T14:13:14.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.127+0000 7fa0337fe700 1 -- 172.21.15.29:0/436313814 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v538) v1 ==== 154+0+0 (secure 0 0 0) 0x7fa04c12b840 con 0x7fa04c1565f0 2024-01-24T14:13:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.130+0000 7fa051da0700 1 -- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa03805f240 msgr2=0x7fa038061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.130+0000 7fa051da0700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa03805f240 0x7fa038061700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa03c009130 tx=0x7fa03c017000).stop 2024-01-24T14:13:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.130+0000 7fa051da0700 1 -- 172.21.15.29:0/436313814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa04c1565f0 msgr2=0x7fa04c154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.130+0000 7fa051da0700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa04c1565f0 0x7fa04c154050 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fa044009fc0 tx=0x7fa0440099b0).stop 2024-01-24T14:13:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.130+0000 7fa051da0700 1 -- 172.21.15.29:0/436313814 shutdown_connections 2024-01-24T14:13:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.130+0000 7fa051da0700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa03805f240 0x7fa038061700 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.130+0000 7fa051da0700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa04c1565f0 0x7fa04c154050 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.130+0000 7fa051da0700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa04c0669e0 0x7fa04c0c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.130+0000 7fa051da0700 1 --2- 172.21.15.29:0/436313814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa04c155b00 0x7fa04c0c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.130+0000 7fa051da0700 1 -- 172.21.15.29:0/436313814 >> 172.21.15.29:0/436313814 conn(0x7fa04c0c8bb0 msgr2=0x7fa04c0a9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.130+0000 7fa051da0700 1 -- 172.21.15.29:0/436313814 shutdown_connections 2024-01-24T14:13:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.130+0000 7fa051da0700 1 -- 172.21.15.29:0/436313814 wait complete. 2024-01-24T14:13:14.146 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2024-01-24T14:13:14.146 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2024-01-24T14:13:14.146 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-01-24T14:13:14.147 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2024-01-24T14:13:14.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.281+0000 7fa8186a7700 1 Processor -- start 2024-01-24T14:13:14.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.281+0000 7fa8186a7700 1 -- start start 2024-01-24T14:13:14.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.282+0000 7fa8186a7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa8100c4a10 0x7fa8100bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:14.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.282+0000 7fa8186a7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8100670a0 0x7fa810158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:14.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.282+0000 7fa8186a7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8101582b0 con 0x7fa8100664f0 2024-01-24T14:13:14.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.282+0000 7fa8186a7700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa810157ca0 con 0x7fa8100c4a10 2024-01-24T14:13:14.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.282+0000 7fa8186a7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa810063c60 con 0x7fa8100670a0 2024-01-24T14:13:14.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.282+0000 7fa816443700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa8100c4a10 0x7fa8100bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.282+0000 7fa816443700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa8100c4a10 0x7fa8100bf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41282/0 (socket says 172.21.15.29:41282) 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.282+0000 7fa816443700 1 -- 172.21.15.29:0/2929530381 learned_addr learned my addr 172.21.15.29:0/2929530381 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.282+0000 7fa815c42700 1 --2- 172.21.15.29:0/2929530381 >> v2:172.21.15.163:3300/0 conn(0x7fa8100670a0 0x7fa810158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.282+0000 7fa816443700 1 -- 172.21.15.29:0/2929530381 >> v2:172.21.15.163:3300/0 conn(0x7fa8100670a0 msgr2=0x7fa810158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.282+0000 7fa816443700 1 --2- 172.21.15.29:0/2929530381 >> v2:172.21.15.163:3300/0 conn(0x7fa8100670a0 0x7fa810158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.283+0000 7fa816443700 1 -- 172.21.15.29:0/2929530381 >> v1:172.21.15.163:6789/0 conn(0x7fa8100664f0 legacy=0x7fa8100bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.283+0000 7fa816443700 1 -- 172.21.15.29:0/2929530381 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa810157f40 con 0x7fa8100c4a10 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.283+0000 7fa816443700 1 --2- 172.21.15.29:0/2929530381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8100c4a10 0x7fa8100bf6d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7fa810068cb0 tx=0x7fa80c0090f0).ready entity=mon.0 client_cookie=3c8dbaa5077b241b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.283+0000 7fa815441700 1 -- 172.21.15.29:0/2929530381 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00b600 con 0x7fa8100c4a10 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.283+0000 7fa815441700 1 -- 172.21.15.29:0/2929530381 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c00b760 con 0x7fa8100c4a10 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.283+0000 7fa815441700 1 -- 172.21.15.29:0/2929530381 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00bae0 con 0x7fa8100c4a10 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.283+0000 7fa8186a7700 1 -- 172.21.15.29:0/2929530381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8100c4a10 msgr2=0x7fa8100bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.283+0000 7fa8186a7700 1 --2- 172.21.15.29:0/2929530381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8100c4a10 0x7fa8100bf6d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7fa810068cb0 tx=0x7fa80c0090f0).stop 2024-01-24T14:13:14.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.284+0000 7fa8186a7700 1 -- 172.21.15.29:0/2929530381 shutdown_connections 2024-01-24T14:13:14.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.284+0000 7fa8186a7700 1 --2- 172.21.15.29:0/2929530381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8100c4a10 0x7fa8100bf6d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:14.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.284+0000 7fa8186a7700 1 --2- 172.21.15.29:0/2929530381 >> v2:172.21.15.163:3300/0 conn(0x7fa8100670a0 0x7fa810158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:14.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.284+0000 7fa8186a7700 1 -- 172.21.15.29:0/2929530381 >> 172.21.15.29:0/2929530381 conn(0x7fa8100c8bb0 msgr2=0x7fa8100c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:14.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.284+0000 7fa8186a7700 1 -- 172.21.15.29:0/2929530381 shutdown_connections 2024-01-24T14:13:14.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.284+0000 7fa8186a7700 1 -- 172.21.15.29:0/2929530381 wait complete. 2024-01-24T14:13:14.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.285+0000 7fa8186a7700 1 Processor -- start 2024-01-24T14:13:14.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.285+0000 7fa8186a7700 1 -- start start 2024-01-24T14:13:14.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.286+0000 7fa8186a7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8100664f0 0x7fa810151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:14.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.286+0000 7fa8186a7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8100670a0 0x7fa8100bffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:14.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.286+0000 7fa8186a7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8100c4a10 0x7fa8100c2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:14.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.286+0000 7fa8186a7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa810156300 con 0x7fa8100c4a10 2024-01-24T14:13:14.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.286+0000 7fa8186a7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8101565e0 con 0x7fa8100664f0 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.286+0000 7fa8186a7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8101568c0 con 0x7fa8100670a0 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa816c44700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8100c4a10 0x7fa8100c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa816c44700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8100c4a10 0x7fa8100c2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41292/0 (socket says 172.21.15.29:41292) 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa816c44700 1 -- 172.21.15.29:0/1942254662 learned_addr learned my addr 172.21.15.29:0/1942254662 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa815c42700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8100670a0 0x7fa8100bffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa816443700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8100664f0 0x7fa810151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa816c44700 1 -- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8100670a0 msgr2=0x7fa8100bffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa816c44700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8100670a0 0x7fa8100bffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa816c44700 1 -- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8100664f0 msgr2=0x7fa810151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa816c44700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8100664f0 0x7fa810151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa816c44700 1 -- 172.21.15.29:0/1942254662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c009370 con 0x7fa8100c4a10 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa816c44700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8100c4a10 0x7fa8100c2920 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fa808002230 tx=0x7fa8080015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:14.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa8067fc700 1 -- 172.21.15.29:0/1942254662 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808010510 con 0x7fa8100c4a10 2024-01-24T14:13:14.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa8067fc700 1 -- 172.21.15.29:0/1942254662 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa808010670 con 0x7fa8100c4a10 2024-01-24T14:13:14.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa8186a7700 1 -- 172.21.15.29:0/1942254662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa810156ba0 con 0x7fa8100c4a10 2024-01-24T14:13:14.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa8067fc700 1 -- 172.21.15.29:0/1942254662 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8080109f0 con 0x7fa8100c4a10 2024-01-24T14:13:14.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.287+0000 7fa8186a7700 1 -- 172.21.15.29:0/1942254662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8101524c0 con 0x7fa8100c4a10 2024-01-24T14:13:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.288+0000 7fa8186a7700 1 -- 172.21.15.29:0/1942254662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa810156440 con 0x7fa8100c4a10 2024-01-24T14:13:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.289+0000 7fa8067fc700 1 -- 172.21.15.29:0/1942254662 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa80801b400 con 0x7fa8100c4a10 2024-01-24T14:13:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.289+0000 7fa8067fc700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa7f405f270 0x7fa7f4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.289+0000 7fa8067fc700 1 -- 172.21.15.29:0/1942254662 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6806+0+0 (secure 0 0 0) 0x7fa80807a4e0 con 0x7fa8100c4a10 2024-01-24T14:13:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.290+0000 7fa816443700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa7f405f270 0x7fa7f4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:14.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.290+0000 7fa816443700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa7f405f270 0x7fa7f4061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa80c009130 tx=0x7fa80c017000).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:14.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.294+0000 7fa8067fc700 1 -- 172.21.15.29:0/1942254662 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa810156440 con 0x7fa8100c4a10 2024-01-24T14:13:14.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:14.546+0000 7fa8186a7700 1 -- 172.21.15.29:0/1942254662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fa810156440 con 0x7fa8100c4a10 2024-01-24T14:13:15.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.130+0000 7fa8067fc700 1 -- 172.21.15.29:0/1942254662 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v539) v1 ==== 159+0+0 (secure 0 0 0) 0x7fa810156440 con 0x7fa8100c4a10 2024-01-24T14:13:15.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.133+0000 7fa8186a7700 1 -- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa7f405f270 msgr2=0x7fa7f4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.133+0000 7fa8186a7700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa7f405f270 0x7fa7f4061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa80c009130 tx=0x7fa80c017000).stop 2024-01-24T14:13:15.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.133+0000 7fa8186a7700 1 -- 172.21.15.29:0/1942254662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8100c4a10 msgr2=0x7fa8100c2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.133+0000 7fa8186a7700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8100c4a10 0x7fa8100c2920 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fa808002230 tx=0x7fa8080015e0).stop 2024-01-24T14:13:15.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.133+0000 7fa8186a7700 1 -- 172.21.15.29:0/1942254662 shutdown_connections 2024-01-24T14:13:15.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.133+0000 7fa8186a7700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa7f405f270 0x7fa7f4061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.133+0000 7fa8186a7700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8100c4a10 0x7fa8100c2920 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.133+0000 7fa8186a7700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8100664f0 0x7fa810151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.133+0000 7fa8186a7700 1 --2- 172.21.15.29:0/1942254662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8100670a0 0x7fa8100bffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.133+0000 7fa8186a7700 1 -- 172.21.15.29:0/1942254662 >> 172.21.15.29:0/1942254662 conn(0x7fa8100c8bb0 msgr2=0x7fa8100c76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:15.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.133+0000 7fa8186a7700 1 -- 172.21.15.29:0/1942254662 shutdown_connections 2024-01-24T14:13:15.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.133+0000 7fa8186a7700 1 -- 172.21.15.29:0/1942254662 wait complete. 2024-01-24T14:13:15.149 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-24T14:13:15.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.287+0000 7fb3dafd1700 1 Processor -- start 2024-01-24T14:13:15.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.287+0000 7fb3dafd1700 1 -- start start 2024-01-24T14:13:15.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3dafd1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb3d40664c0 0x7fb3d4066950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:15.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3dafd1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb3d40bdb30 0x7fb3d40bdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:15.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3dafd1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d4066e40 con 0x7fb3d413a5e0 2024-01-24T14:13:15.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3dafd1700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d40640d0 con 0x7fb3d40bdb30 2024-01-24T14:13:15.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3dafd1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d40679f0 con 0x7fb3d40664c0 2024-01-24T14:13:15.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3d956e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb3d40bdb30 0x7fb3d40bdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:15.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3d956e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb3d40bdb30 0x7fb3d40bdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41304/0 (socket says 172.21.15.29:41304) 2024-01-24T14:13:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3d956e700 1 -- 172.21.15.29:0/302831559 learned_addr learned my addr 172.21.15.29:0/302831559 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3d3fff700 1 --2- 172.21.15.29:0/302831559 >> v2:172.21.15.163:3300/0 conn(0x7fb3d40664c0 0x7fb3d4066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3d956e700 1 -- 172.21.15.29:0/302831559 >> v2:172.21.15.163:3300/0 conn(0x7fb3d40664c0 msgr2=0x7fb3d4066950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3d956e700 1 --2- 172.21.15.29:0/302831559 >> v2:172.21.15.163:3300/0 conn(0x7fb3d40664c0 0x7fb3d4066950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3d956e700 1 -- 172.21.15.29:0/302831559 >> v1:172.21.15.163:6789/0 conn(0x7fb3d413a5e0 legacy=0x7fb3d40639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3d956e700 1 -- 172.21.15.29:0/302831559 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d4067cd0 con 0x7fb3d40bdb30 2024-01-24T14:13:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.288+0000 7fb3d956e700 1 --2- 172.21.15.29:0/302831559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d40bdb30 0x7fb3d40bdf60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fb3d40632c0 tx=0x7fb3cc009840).ready entity=mon.0 client_cookie=6d919cf82f6c0052 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.289+0000 7fb3d37fe700 1 -- 172.21.15.29:0/302831559 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc00e070 con 0x7fb3d40bdb30 2024-01-24T14:13:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.289+0000 7fb3d37fe700 1 -- 172.21.15.29:0/302831559 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc00ba50 con 0x7fb3d40bdb30 2024-01-24T14:13:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.289+0000 7fb3d37fe700 1 -- 172.21.15.29:0/302831559 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc00bdd0 con 0x7fb3d40bdb30 2024-01-24T14:13:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.289+0000 7fb3dafd1700 1 -- 172.21.15.29:0/302831559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d40bdb30 msgr2=0x7fb3d40bdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.289+0000 7fb3dafd1700 1 --2- 172.21.15.29:0/302831559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d40bdb30 0x7fb3d40bdf60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fb3d40632c0 tx=0x7fb3cc009840).stop 2024-01-24T14:13:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.289+0000 7fb3dafd1700 1 -- 172.21.15.29:0/302831559 shutdown_connections 2024-01-24T14:13:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.289+0000 7fb3dafd1700 1 --2- 172.21.15.29:0/302831559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d40bdb30 0x7fb3d40bdf60 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.289+0000 7fb3dafd1700 1 --2- 172.21.15.29:0/302831559 >> v2:172.21.15.163:3300/0 conn(0x7fb3d40664c0 0x7fb3d4066950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.289+0000 7fb3dafd1700 1 -- 172.21.15.29:0/302831559 >> 172.21.15.29:0/302831559 conn(0x7fb3d40c8bd0 msgr2=0x7fb3d40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.289+0000 7fb3dafd1700 1 -- 172.21.15.29:0/302831559 shutdown_connections 2024-01-24T14:13:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.289+0000 7fb3dafd1700 1 -- 172.21.15.29:0/302831559 wait complete. 2024-01-24T14:13:15.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.290+0000 7fb3dafd1700 1 Processor -- start 2024-01-24T14:13:15.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.291+0000 7fb3dafd1700 1 -- start start 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.291+0000 7fb3dafd1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3d40664c0 0x7fb3d4138560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.291+0000 7fb3dafd1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3d40bdb30 0x7fb3d40ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.291+0000 7fb3dafd1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d413a5e0 0x7fb3d4154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.291+0000 7fb3dafd1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d40ae150 con 0x7fb3d413a5e0 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.291+0000 7fb3dafd1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d40ae430 con 0x7fb3d40664c0 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.291+0000 7fb3dafd1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d40ae710 con 0x7fb3d40bdb30 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d956e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d413a5e0 0x7fb3d4154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d956e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d413a5e0 0x7fb3d4154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41314/0 (socket says 172.21.15.29:41314) 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d956e700 1 -- 172.21.15.29:0/328491968 learned_addr learned my addr 172.21.15.29:0/328491968 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d3fff700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3d40bdb30 0x7fb3d40ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d8d6d700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3d40664c0 0x7fb3d4138560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d956e700 1 -- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3d40bdb30 msgr2=0x7fb3d40ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d956e700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3d40bdb30 0x7fb3d40ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d956e700 1 -- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3d40664c0 msgr2=0x7fb3d4138560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d956e700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3d40664c0 0x7fb3d4138560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d956e700 1 -- 172.21.15.29:0/328491968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc009ae0 con 0x7fb3d413a5e0 2024-01-24T14:13:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d956e700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d413a5e0 0x7fb3d4154690 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fb3cc009fd0 tx=0x7fb3cc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d0ff9700 1 -- 172.21.15.29:0/328491968 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc0092e0 con 0x7fb3d413a5e0 2024-01-24T14:13:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d0ff9700 1 -- 172.21.15.29:0/328491968 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc01d410 con 0x7fb3d413a5e0 2024-01-24T14:13:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3dafd1700 1 -- 172.21.15.29:0/328491968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d40ae9f0 con 0x7fb3d413a5e0 2024-01-24T14:13:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3d0ff9700 1 -- 172.21.15.29:0/328491968 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc01d790 con 0x7fb3d413a5e0 2024-01-24T14:13:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.292+0000 7fb3dafd1700 1 -- 172.21.15.29:0/328491968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d4138a50 con 0x7fb3d413a5e0 2024-01-24T14:13:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.294+0000 7fb3dafd1700 1 -- 172.21.15.29:0/328491968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d40475e0 con 0x7fb3d413a5e0 2024-01-24T14:13:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.294+0000 7fb3d0ff9700 1 -- 172.21.15.29:0/328491968 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb3cc01db80 con 0x7fb3d413a5e0 2024-01-24T14:13:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.294+0000 7fb3d0ff9700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb3b805f2a0 0x7fb3b8061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.294+0000 7fb3d0ff9700 1 -- 172.21.15.29:0/328491968 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6806+0+0 (secure 0 0 0) 0x7fb3cc07c310 con 0x7fb3d413a5e0 2024-01-24T14:13:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.294+0000 7fb3d8d6d700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb3b805f2a0 0x7fb3b8061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:15.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.295+0000 7fb3d8d6d700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb3b805f2a0 0x7fb3b8061760 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb3c80036d0 tx=0x7fb3c8009040).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:15.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.298+0000 7fb3d0ff9700 1 -- 172.21.15.29:0/328491968 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb3cc04d1b0 con 0x7fb3d413a5e0 2024-01-24T14:13:15.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.477+0000 7fb3dafd1700 1 -- 172.21.15.29:0/328491968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb3d4067fb0 con 0x7fb3d413a5e0 2024-01-24T14:13:15.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.481+0000 7fb3d0ff9700 1 -- 172.21.15.29:0/328491968 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb3cc0507d0 con 0x7fb3d413a5e0 2024-01-24T14:13:15.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.483+0000 7fb3dafd1700 1 -- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb3b805f2a0 msgr2=0x7fb3b8061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.483+0000 7fb3dafd1700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb3b805f2a0 0x7fb3b8061760 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb3c80036d0 tx=0x7fb3c8009040).stop 2024-01-24T14:13:15.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.483+0000 7fb3dafd1700 1 -- 172.21.15.29:0/328491968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d413a5e0 msgr2=0x7fb3d4154690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.483+0000 7fb3dafd1700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d413a5e0 0x7fb3d4154690 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fb3cc009fd0 tx=0x7fb3cc009840).stop 2024-01-24T14:13:15.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.483+0000 7fb3dafd1700 1 -- 172.21.15.29:0/328491968 shutdown_connections 2024-01-24T14:13:15.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.483+0000 7fb3dafd1700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb3b805f2a0 0x7fb3b8061760 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.483+0000 7fb3dafd1700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3d413a5e0 0x7fb3d4154690 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.483+0000 7fb3dafd1700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3d40664c0 0x7fb3d4138560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.483+0000 7fb3dafd1700 1 --2- 172.21.15.29:0/328491968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3d40bdb30 0x7fb3d40ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.483+0000 7fb3dafd1700 1 -- 172.21.15.29:0/328491968 >> 172.21.15.29:0/328491968 conn(0x7fb3d40c8bd0 msgr2=0x7fb3d41393f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:15.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.483+0000 7fb3dafd1700 1 -- 172.21.15.29:0/328491968 shutdown_connections 2024-01-24T14:13:15.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.483+0000 7fb3dafd1700 1 -- 172.21.15.29:0/328491968 wait complete. 2024-01-24T14:13:15.497 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T14:13:15.497 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:13:15.539 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:13:15.590 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T14:13:15.590 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-24T14:13:15.591 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:13:15.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.726+0000 7f5ffba1b700 1 Processor -- start 2024-01-24T14:13:15.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.727+0000 7f5ffba1b700 1 -- start start 2024-01-24T14:13:15.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.727+0000 7f5ffba1b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff40be540 0x7f5ff40633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:15.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.727+0000 7f5ffba1b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ff4066ab0 0x7f5ff4062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:15.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.727+0000 7f5ffba1b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff40bce60 con 0x7f5ff4062970 2024-01-24T14:13:15.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.727+0000 7f5ffba1b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff4069c10 con 0x7f5ff4066ab0 2024-01-24T14:13:15.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.727+0000 7f5ffba1b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40be9c0 con 0x7f5ff40be540 2024-01-24T14:13:15.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.728+0000 7f5ff9fb8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ff4066ab0 0x7f5ff4062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:15.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.728+0000 7f5ff8fb6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff40be540 0x7f5ff40633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:15.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.728+0000 7f5ff9fb8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ff4066ab0 0x7f5ff4062d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:60384/0 (socket says 172.21.15.29:60384) 2024-01-24T14:13:15.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.728+0000 7f5ff9fb8700 1 -- 172.21.15.29:0/3429193128 learned_addr learned my addr 172.21.15.29:0/3429193128 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:15.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.728+0000 7f5ff97b7700 1 --1- 172.21.15.29:0/3429193128 >> v1:172.21.15.163:6789/0 conn(0x7f5ff4062970 0x7f5ff4065360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:49788/0 (socket says 172.21.15.29:49788) 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.728+0000 7f5ff9fb8700 1 -- 172.21.15.29:0/3429193128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff40be540 msgr2=0x7f5ff40633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.728+0000 7f5ff9fb8700 1 --2- 172.21.15.29:0/3429193128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff40be540 0x7f5ff40633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.728+0000 7f5ff9fb8700 1 -- 172.21.15.29:0/3429193128 >> v1:172.21.15.163:6789/0 conn(0x7f5ff4062970 legacy=0x7f5ff4065360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.728+0000 7f5ff9fb8700 1 -- 172.21.15.29:0/3429193128 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff4069df0 con 0x7f5ff4066ab0 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.728+0000 7f5fe7fff700 1 -- 172.21.15.29:0/3429193128 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754548222 0 0) 0x7f5ff40bce60 con 0x7f5ff4062970 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.728+0000 7f5ff9fb8700 1 --2- 172.21.15.29:0/3429193128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff4066ab0 0x7f5ff4062d50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f5fec009f50 tx=0x7f5fec009840).ready entity=mon.1 client_cookie=6fbb8d454583218 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.729+0000 7f5fe7fff700 1 -- 172.21.15.29:0/3429193128 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00e070 con 0x7f5ff4066ab0 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.729+0000 7f5fe7fff700 1 -- 172.21.15.29:0/3429193128 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec00b9c0 con 0x7f5ff4066ab0 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.729+0000 7f5fe7fff700 1 -- 172.21.15.29:0/3429193128 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00bd40 con 0x7f5ff4066ab0 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.729+0000 7f5ffba1b700 1 -- 172.21.15.29:0/3429193128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff4066ab0 msgr2=0x7f5ff4062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.729+0000 7f5ffba1b700 1 --2- 172.21.15.29:0/3429193128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff4066ab0 0x7f5ff4062d50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f5fec009f50 tx=0x7f5fec009840).stop 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.729+0000 7f5ffba1b700 1 -- 172.21.15.29:0/3429193128 shutdown_connections 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.729+0000 7f5ffba1b700 1 --2- 172.21.15.29:0/3429193128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff4066ab0 0x7f5ff4062d50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.729+0000 7f5ffba1b700 1 --2- 172.21.15.29:0/3429193128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff40be540 0x7f5ff40633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.729+0000 7f5ffba1b700 1 -- 172.21.15.29:0/3429193128 >> 172.21.15.29:0/3429193128 conn(0x7f5ff40c85c0 msgr2=0x7f5ff40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:15.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.729+0000 7f5ffba1b700 1 -- 172.21.15.29:0/3429193128 shutdown_connections 2024-01-24T14:13:15.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.729+0000 7f5ffba1b700 1 -- 172.21.15.29:0/3429193128 wait complete. 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.731+0000 7f5ffba1b700 1 Processor -- start 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.731+0000 7f5ffba1b700 1 -- start start 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ffba1b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4062970 0x7f5ff4164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ffba1b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff4066ab0 0x7f5ff40c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ffba1b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff40be540 0x7f5ff4152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ffba1b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40abb00 con 0x7f5ff4062970 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ffba1b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40aaef0 con 0x7f5ff40be540 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ffba1b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40ab1d0 con 0x7f5ff4066ab0 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ff97b7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4062970 0x7f5ff4164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ff97b7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4062970 0x7f5ff4164fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41328/0 (socket says 172.21.15.29:41328) 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ff97b7700 1 -- 172.21.15.29:0/2184751062 learned_addr learned my addr 172.21.15.29:0/2184751062 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ff9fb8700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff40be540 0x7f5ff4152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:15.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ff8fb6700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff4066ab0 0x7f5ff40c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:15.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ff97b7700 1 -- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff4066ab0 msgr2=0x7f5ff40c0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ff97b7700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff4066ab0 0x7f5ff40c0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ff97b7700 1 -- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff40be540 msgr2=0x7f5ff4152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ff97b7700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff40be540 0x7f5ff4152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ff97b7700 1 -- 172.21.15.29:0/2184751062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec009ae0 con 0x7f5ff4062970 2024-01-24T14:13:15.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.732+0000 7f5ff97b7700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4062970 0x7f5ff4164fb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f5fe800c560 tx=0x7f5fe8007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:15.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.733+0000 7f5fe57fa700 1 -- 172.21.15.29:0/2184751062 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80072e0 con 0x7f5ff4062970 2024-01-24T14:13:15.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.733+0000 7f5ffba1b700 1 -- 172.21.15.29:0/2184751062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff40ab4b0 con 0x7f5ff4062970 2024-01-24T14:13:15.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.733+0000 7f5fe57fa700 1 -- 172.21.15.29:0/2184751062 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8007440 con 0x7f5ff4062970 2024-01-24T14:13:15.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.733+0000 7f5fe57fa700 1 -- 172.21.15.29:0/2184751062 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80058f0 con 0x7f5ff4062970 2024-01-24T14:13:15.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.733+0000 7f5ffba1b700 1 -- 172.21.15.29:0/2184751062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff41654a0 con 0x7f5ff4062970 2024-01-24T14:13:15.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.734+0000 7f5ffba1b700 1 -- 172.21.15.29:0/2184751062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff40475e0 con 0x7f5ff4062970 2024-01-24T14:13:15.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.734+0000 7f5fe57fa700 1 -- 172.21.15.29:0/2184751062 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5fe8005ce0 con 0x7f5ff4062970 2024-01-24T14:13:15.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.734+0000 7f5fe57fa700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5fd805f270 0x7f5fd8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:15.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.735+0000 7f5fe57fa700 1 -- 172.21.15.29:0/2184751062 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6806+0+0 (secure 0 0 0) 0x7f5fe8079db0 con 0x7f5ff4062970 2024-01-24T14:13:15.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.735+0000 7f5ff8fb6700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5fd805f270 0x7f5fd8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:15.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.735+0000 7f5ff8fb6700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5fd805f270 0x7f5fd8061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5ff40a9900 tx=0x7f5ff0009060).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:15.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.739+0000 7f5fe57fa700 1 -- 172.21.15.29:0/2184751062 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5fe804ad10 con 0x7f5ff4062970 2024-01-24T14:13:15.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.966+0000 7f5ffba1b700 1 -- 172.21.15.29:0/2184751062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ff406a0d0 con 0x7f5ff4062970 2024-01-24T14:13:15.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.966+0000 7f5fe57fa700 1 -- 172.21.15.29:0/2184751062 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f5fe804e330 con 0x7f5ff4062970 2024-01-24T14:13:15.967 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:15.967 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2239835986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2239835986},{"type":"v1","addr":"172.21.15.163:6837","nonce":2239835986}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2208246875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2208246875},{"type":"v1","addr":"172.21.15.29:6835","nonce":2208246875}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/547152226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":547152226},{"type":"v1","addr":"172.21.15.163:6836","nonce":547152226}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6297,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/293722483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":293722483},{"type":"v1","addr":"172.21.15.29:6834","nonce":293722483}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[]} 2024-01-24T14:13:15.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.968+0000 7f5ffba1b700 1 -- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5fd805f270 msgr2=0x7f5fd8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.968+0000 7f5ffba1b700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5fd805f270 0x7f5fd8061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5ff40a9900 tx=0x7f5ff0009060).stop 2024-01-24T14:13:15.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.968+0000 7f5ffba1b700 1 -- 172.21.15.29:0/2184751062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4062970 msgr2=0x7f5ff4164fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.968+0000 7f5ffba1b700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4062970 0x7f5ff4164fb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f5fe800c560 tx=0x7f5fe8007c90).stop 2024-01-24T14:13:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.969+0000 7f5ffba1b700 1 -- 172.21.15.29:0/2184751062 shutdown_connections 2024-01-24T14:13:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.969+0000 7f5ffba1b700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ff40be540 0x7f5ff4152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.969+0000 7f5ffba1b700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5fd805f270 0x7f5fd8061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.969+0000 7f5ffba1b700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ff4062970 0x7f5ff4164fb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.969+0000 7f5ffba1b700 1 --2- 172.21.15.29:0/2184751062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ff4066ab0 0x7f5ff40c0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.969+0000 7f5ffba1b700 1 -- 172.21.15.29:0/2184751062 >> 172.21.15.29:0/2184751062 conn(0x7f5ff40c85c0 msgr2=0x7f5ff4139100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.969+0000 7f5ffba1b700 1 -- 172.21.15.29:0/2184751062 shutdown_connections 2024-01-24T14:13:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:15.969+0000 7f5ffba1b700 1 -- 172.21.15.29:0/2184751062 wait complete. 2024-01-24T14:13:15.970 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 28 2024-01-24T14:13:15.983 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-24T14:13:15.983 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-24T14:13:15.983 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-24T14:13:15.983 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:13:15.984 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-24T14:13:15.984 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-24T14:13:15.984 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-24T14:13:15.984 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:13:15.984 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-24T14:13:15.984 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-24T14:13:15.984 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-24T14:13:15.985 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:13:15.985 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-24T14:13:15.985 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-24T14:13:15.985 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-24T14:13:15.985 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:13:15.993 INFO:tasks.ceph.mds.a:Stopped 2024-01-24T14:13:15.994 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-24T14:13:15.994 INFO:tasks.ceph.mds.c:Stopped 2024-01-24T14:13:15.994 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-24T14:13:15.996 INFO:tasks.ceph.mds.b:Stopped 2024-01-24T14:13:15.996 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-24T14:13:16.036 INFO:tasks.ceph.mds.a:Started 2024-01-24T14:13:16.036 DEBUG:teuthology.parallel:result is None 2024-01-24T14:13:16.037 INFO:tasks.ceph.mds.c:Started 2024-01-24T14:13:16.037 DEBUG:teuthology.parallel:result is None 2024-01-24T14:13:16.039 INFO:tasks.ceph.mds.b:Started 2024-01-24T14:13:16.039 DEBUG:teuthology.parallel:result is None 2024-01-24T14:13:16.119 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-01-24T14:13:16.120 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T14:13:16.120 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T14:13:20.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-01-24T14:13:21.996 INFO:tasks.ceph.mds.d:Stopped 2024-01-24T14:13:21.996 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-24T14:13:21.999 INFO:tasks.ceph.mds.d:Started 2024-01-24T14:13:21.999 DEBUG:teuthology.parallel:result is None 2024-01-24T14:13:21.999 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-24T14:13:22.083 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T14:13:22.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.142+0000 7f7635715700 1 Processor -- start 2024-01-24T14:13:22.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.142+0000 7f7635715700 1 -- start start 2024-01-24T14:13:22.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.143+0000 7f7635715700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7630064410 0x7f76301524d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:22.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.143+0000 7f7635715700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76300647f0 0x7f763014f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:22.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.143+0000 7f7635715700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76300695f0 con 0x7f7630066160 2024-01-24T14:13:22.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.143+0000 7f7635715700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763006a190 con 0x7f7630064410 2024-01-24T14:13:22.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.143+0000 7f7635715700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7630064c40 con 0x7f76300647f0 2024-01-24T14:13:22.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.143+0000 7f762e7fc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7630066160 0x7f7630069a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35298/0 (socket says 172.21.15.29:35298) 2024-01-24T14:13:22.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.143+0000 7f762e7fc700 1 -- 172.21.15.29:0/791649882 learned_addr learned my addr 172.21.15.29:0/791649882 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:22.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.143+0000 7f762effd700 1 --2- 172.21.15.29:0/791649882 >> v2:172.21.15.163:3300/0 conn(0x7f7630064410 0x7f76301524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:22.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.143+0000 7f762f7fe700 1 --2- 172.21.15.29:0/791649882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76300647f0 0x7f763014f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:22.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.144+0000 7f762dffb700 1 -- 172.21.15.29:0/791649882 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206832277 0 0) 0x7f76300695f0 con 0x7f7630066160 2024-01-24T14:13:22.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.144+0000 7f762dffb700 1 -- 172.21.15.29:0/791649882 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f761c0033f0 con 0x7f7630066160 2024-01-24T14:13:22.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.144+0000 7f762effd700 1 -- 172.21.15.29:0/791649882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76300647f0 msgr2=0x7f763014f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.144+0000 7f762effd700 1 --2- 172.21.15.29:0/791649882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76300647f0 0x7f763014f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.144+0000 7f762dffb700 1 -- 172.21.15.29:0/791649882 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3820057054 0 0) 0x7f761c0033f0 con 0x7f7630066160 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.144+0000 7f762effd700 1 -- 172.21.15.29:0/791649882 >> v1:172.21.15.29:6789/0 conn(0x7f7630066160 legacy=0x7f7630069a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.144+0000 7f762effd700 1 -- 172.21.15.29:0/791649882 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7630068740 con 0x7f7630064410 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.144+0000 7f762effd700 1 --2- 172.21.15.29:0/791649882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7630064410 0x7f76301524d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f7618009760 tx=0x7f7618009060).ready entity=mon.1 client_cookie=14452741bf147586 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.144+0000 7f762dffb700 1 -- 172.21.15.29:0/791649882 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620001470 con 0x7f7630064410 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.144+0000 7f762dffb700 1 -- 172.21.15.29:0/791649882 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618011400 con 0x7f7630064410 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.144+0000 7f762dffb700 1 -- 172.21.15.29:0/791649882 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618011780 con 0x7f7630064410 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.145+0000 7f7635715700 1 -- 172.21.15.29:0/791649882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7630064410 msgr2=0x7f76301524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.145+0000 7f7635715700 1 --2- 172.21.15.29:0/791649882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7630064410 0x7f76301524d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f7618009760 tx=0x7f7618009060).stop 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.145+0000 7f7635715700 1 -- 172.21.15.29:0/791649882 shutdown_connections 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.145+0000 7f7635715700 1 --2- 172.21.15.29:0/791649882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76300647f0 0x7f763014f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.145+0000 7f7635715700 1 --2- 172.21.15.29:0/791649882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7630064410 0x7f76301524d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:22.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.145+0000 7f7635715700 1 -- 172.21.15.29:0/791649882 >> 172.21.15.29:0/791649882 conn(0x7f76300c7480 msgr2=0x7f76300c3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:22.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.145+0000 7f7635715700 1 -- 172.21.15.29:0/791649882 shutdown_connections 2024-01-24T14:13:22.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.145+0000 7f7635715700 1 -- 172.21.15.29:0/791649882 wait complete. 2024-01-24T14:13:22.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.146+0000 7f7635715700 1 Processor -- start 2024-01-24T14:13:22.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.146+0000 7f7635715700 1 -- start start 2024-01-24T14:13:22.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.147+0000 7f7635715700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7630064410 0x7f7630138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:22.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.147+0000 7f7635715700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76300647f0 0x7f76301653d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:22.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.147+0000 7f7635715700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7630066160 0x7f7630156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:22.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.147+0000 7f7635715700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76300c1dc0 con 0x7f7630064410 2024-01-24T14:13:22.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.147+0000 7f7635715700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76300bf650 con 0x7f76300647f0 2024-01-24T14:13:22.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.147+0000 7f7635715700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76301391d0 con 0x7f7630066160 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.147+0000 7f762effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7630064410 0x7f7630138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.147+0000 7f762effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7630064410 0x7f7630138ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41376/0 (socket says 172.21.15.29:41376) 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.147+0000 7f762effd700 1 -- 172.21.15.29:0/3361403066 learned_addr learned my addr 172.21.15.29:0/3361403066 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.147+0000 7f762e7fc700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76300647f0 0x7f76301653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.147+0000 7f762f7fe700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7630066160 0x7f7630156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.148+0000 7f762effd700 1 -- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7630066160 msgr2=0x7f7630156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.148+0000 7f762effd700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7630066160 0x7f7630156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.148+0000 7f762effd700 1 -- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76300647f0 msgr2=0x7f76301653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.148+0000 7f762effd700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76300647f0 0x7f76301653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.148+0000 7f762effd700 1 -- 172.21.15.29:0/3361403066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76180092a0 con 0x7f7630064410 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.148+0000 7f762effd700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7630064410 0x7f7630138ce0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f7618009d50 tx=0x7f7618016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.148+0000 7f7626ffd700 1 -- 172.21.15.29:0/3361403066 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180106c0 con 0x7f7630064410 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.148+0000 7f7626ffd700 1 -- 172.21.15.29:0/3361403066 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618010820 con 0x7f7630064410 2024-01-24T14:13:22.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.148+0000 7f7635715700 1 -- 172.21.15.29:0/3361403066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76300bf090 con 0x7f7630064410 2024-01-24T14:13:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.148+0000 7f7626ffd700 1 -- 172.21.15.29:0/3361403066 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618010be0 con 0x7f7630064410 2024-01-24T14:13:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.148+0000 7f7635715700 1 -- 172.21.15.29:0/3361403066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76300c1a50 con 0x7f7630064410 2024-01-24T14:13:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.149+0000 7f7635715700 1 -- 172.21.15.29:0/3361403066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76300c18b0 con 0x7f7630064410 2024-01-24T14:13:22.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.150+0000 7f7626ffd700 1 -- 172.21.15.29:0/3361403066 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f76180263b0 con 0x7f7630064410 2024-01-24T14:13:22.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.150+0000 7f7626ffd700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f761005f270 0x7f7610061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:22.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.150+0000 7f7626ffd700 1 -- 172.21.15.29:0/3361403066 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6806+0+0 (secure 0 0 0) 0x7f761807b280 con 0x7f7630064410 2024-01-24T14:13:22.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.150+0000 7f762e7fc700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f761005f270 0x7f7610061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:22.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.151+0000 7f762e7fc700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f761005f270 0x7f7610061730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7630167730 tx=0x7f7620001a00).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:22.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.155+0000 7f7626ffd700 1 -- 172.21.15.29:0/3361403066 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f76300c18b0 con 0x7f7630064410 2024-01-24T14:13:22.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:22.350+0000 7f7635715700 1 -- 172.21.15.29:0/3361403066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f76300c18b0 con 0x7f7630064410 2024-01-24T14:13:23.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.113+0000 7f7626ffd700 1 -- 172.21.15.29:0/3361403066 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) v1 ==== 97+0+0 (secure 0 0 0) 0x7f76300c18b0 con 0x7f7630064410 2024-01-24T14:13:23.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.116+0000 7f7635715700 1 -- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f761005f270 msgr2=0x7f7610061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.116+0000 7f7635715700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f761005f270 0x7f7610061730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7630167730 tx=0x7f7620001a00).stop 2024-01-24T14:13:23.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.116+0000 7f7635715700 1 -- 172.21.15.29:0/3361403066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7630064410 msgr2=0x7f7630138ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.116+0000 7f7635715700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7630064410 0x7f7630138ce0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f7618009d50 tx=0x7f7618016000).stop 2024-01-24T14:13:23.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.116+0000 7f7635715700 1 -- 172.21.15.29:0/3361403066 shutdown_connections 2024-01-24T14:13:23.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.116+0000 7f7635715700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7630066160 0x7f7630156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.116+0000 7f7635715700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f761005f270 0x7f7610061730 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.116+0000 7f7635715700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7630064410 0x7f7630138ce0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.116+0000 7f7635715700 1 --2- 172.21.15.29:0/3361403066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76300647f0 0x7f76301653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.116+0000 7f7635715700 1 -- 172.21.15.29:0/3361403066 >> 172.21.15.29:0/3361403066 conn(0x7f76300c7480 msgr2=0x7f76300aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:23.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.116+0000 7f7635715700 1 -- 172.21.15.29:0/3361403066 shutdown_connections 2024-01-24T14:13:23.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.116+0000 7f7635715700 1 -- 172.21.15.29:0/3361403066 wait complete. 2024-01-24T14:13:23.118 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-01-24T14:13:23.132 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-24T14:13:23.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.267+0000 7f28aecf3700 1 Processor -- start 2024-01-24T14:13:23.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.267+0000 7f28aecf3700 1 -- start start 2024-01-24T14:13:23.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.268+0000 7f28aecf3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28a8154370 0x7f28a80c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.268+0000 7f28aecf3700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a80aa6c0 con 0x7f28a8154750 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.268+0000 7f28aecf3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a8064240 con 0x7f28a80aa0e0 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.268+0000 7f28aecf3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a8154ba0 con 0x7f28a8154370 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.268+0000 7f28ad290700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f28a8154750 0x7f28a80631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35300/0 (socket says 172.21.15.29:35300) 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.268+0000 7f28ad290700 1 -- 172.21.15.29:0/674502344 learned_addr learned my addr 172.21.15.29:0/674502344 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28aca8f700 1 --2- 172.21.15.29:0/674502344 >> v2:172.21.15.163:3300/0 conn(0x7f28a8154370 0x7f28a80c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764714359 0 0) 0x7f28a80aa6c0 con 0x7f28a8154750 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28900033f0 con 0x7f28a8154750 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2279806639 0 0) 0x7f28900033f0 con 0x7f28a8154750 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 >> v2:172.21.15.163:3300/0 conn(0x7f28a8154370 msgr2=0x7f28a80c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 --2- 172.21.15.29:0/674502344 >> v2:172.21.15.163:3300/0 conn(0x7f28a8154370 0x7f28a80c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 >> v1:172.21.15.163:6789/0 conn(0x7f28a80aa0e0 legacy=0x7f28a80ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a80cb5c0 con 0x7f28a8154750 2024-01-24T14:13:23.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f28a00017f0 con 0x7f28a8154750 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 >> v1:172.21.15.29:6789/0 conn(0x7f28a8154750 legacy=0x7f28a80631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 --2- 172.21.15.29:0/674502344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 0x7f2890003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 --2- 172.21.15.29:0/674502344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2890004690 0x7f2890004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 --2- 172.21.15.29:0/674502344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2890005000 0x7f289000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a8154ba0 con 0x7f28900038e0 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289000beb0 con 0x7f2890004690 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289000c070 con 0x7f2890005000 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28aca8f700 1 --2- 172.21.15.29:0/674502344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 0x7f2890003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28a7fff700 1 --2- 172.21.15.29:0/674502344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2890004690 0x7f2890004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28ad290700 1 --2- 172.21.15.29:0/674502344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2890005000 0x7f289000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28aca8f700 1 -- 172.21.15.29:0/674502344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2890005000 msgr2=0x7f289000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28aca8f700 1 --2- 172.21.15.29:0/674502344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2890005000 0x7f289000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28aca8f700 1 -- 172.21.15.29:0/674502344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2890004690 msgr2=0x7f2890004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:23.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28aca8f700 1 --2- 172.21.15.29:0/674502344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2890004690 0x7f2890004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.269+0000 7f28aca8f700 1 -- 172.21.15.29:0/674502344 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28900033f0 con 0x7f28900038e0 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.270+0000 7f28ad290700 1 -- 172.21.15.29:0/674502344 reap_dead start 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.270+0000 7f28aca8f700 1 --2- 172.21.15.29:0/674502344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 0x7f2890003d00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f2898006fd0 tx=0x7f2898006f40).ready entity=mon.0 client_cookie=89f9fa5c4d61cbaa server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.270+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898009430 con 0x7f28900038e0 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.270+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289800e640 con 0x7f28900038e0 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.270+0000 7f28a77fe700 1 -- 172.21.15.29:0/674502344 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289800e9c0 con 0x7f28900038e0 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.270+0000 7f28aecf3700 1 -- 172.21.15.29:0/674502344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 msgr2=0x7f2890003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.270+0000 7f28aecf3700 1 --2- 172.21.15.29:0/674502344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 0x7f2890003d00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f2898006fd0 tx=0x7f2898006f40).stop 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.270+0000 7f28aecf3700 1 -- 172.21.15.29:0/674502344 shutdown_connections 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.270+0000 7f28aecf3700 1 --2- 172.21.15.29:0/674502344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 0x7f2890003d00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.270+0000 7f28aecf3700 1 -- 172.21.15.29:0/674502344 >> 172.21.15.29:0/674502344 conn(0x7f28a805ec80 msgr2=0x7f28a80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.270+0000 7f28aecf3700 1 -- 172.21.15.29:0/674502344 shutdown_connections 2024-01-24T14:13:23.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.270+0000 7f28aecf3700 1 -- 172.21.15.29:0/674502344 wait complete. 2024-01-24T14:13:23.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.272+0000 7f28aecf3700 1 Processor -- start 2024-01-24T14:13:23.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.272+0000 7f28aecf3700 1 -- start start 2024-01-24T14:13:23.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aecf3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 0x7f28a81571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aecf3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28a8145b30 0x7f28a8146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aecf3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28a8146e70 0x7f28a8159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aecf3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a8156d10 con 0x7f28900038e0 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aecf3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a81472f0 con 0x7f28a8146e70 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aecf3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a81563e0 con 0x7f28a8145b30 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aca8f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 0x7f28a81571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aca8f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 0x7f28a81571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41394/0 (socket says 172.21.15.29:41394) 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aca8f700 1 -- 172.21.15.29:0/1391068171 learned_addr learned my addr 172.21.15.29:0/1391068171 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28a7fff700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28a8145b30 0x7f28a8146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28ad290700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28a8146e70 0x7f28a8159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aca8f700 1 -- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28a8145b30 msgr2=0x7f28a8146980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aca8f700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28a8145b30 0x7f28a8146980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aca8f700 1 -- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28a8146e70 msgr2=0x7f28a8159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aca8f700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28a8146e70 0x7f28a8159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aca8f700 1 -- 172.21.15.29:0/1391068171 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a81566c0 con 0x7f28900038e0 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28ad290700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28a8146e70 0x7f28a8159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:23.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aca8f700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 0x7f28a81571a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f289000be80 tx=0x7f289801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:23.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28a4ff9700 1 -- 172.21.15.29:0/1391068171 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289800ed70 con 0x7f28900038e0 2024-01-24T14:13:23.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28a4ff9700 1 -- 172.21.15.29:0/1391068171 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289800eed0 con 0x7f28900038e0 2024-01-24T14:13:23.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28a4ff9700 1 -- 172.21.15.29:0/1391068171 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289800ea00 con 0x7f28900038e0 2024-01-24T14:13:23.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.273+0000 7f28aecf3700 1 -- 172.21.15.29:0/1391068171 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a81569a0 con 0x7f28900038e0 2024-01-24T14:13:23.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.274+0000 7f28aecf3700 1 -- 172.21.15.29:0/1391068171 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a8145fb0 con 0x7f28900038e0 2024-01-24T14:13:23.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.275+0000 7f28aecf3700 1 -- 172.21.15.29:0/1391068171 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a8156ff0 con 0x7f28900038e0 2024-01-24T14:13:23.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.275+0000 7f28a4ff9700 1 -- 172.21.15.29:0/1391068171 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f289801b660 con 0x7f28900038e0 2024-01-24T14:13:23.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.275+0000 7f28a4ff9700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f289405f270 0x7f2894061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.275+0000 7f28a4ff9700 1 -- 172.21.15.29:0/1391068171 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f289807bec0 con 0x7f28900038e0 2024-01-24T14:13:23.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.276+0000 7f28a7fff700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f289405f270 0x7f2894061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.276+0000 7f28a7fff700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f289405f270 0x7f2894061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f28a80c96e0 tx=0x7f289c007bb0).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:23.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.280+0000 7f28a4ff9700 1 -- 172.21.15.29:0/1391068171 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f289804cf60 con 0x7f28900038e0 2024-01-24T14:13:23.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.531+0000 7f28aecf3700 1 -- 172.21.15.29:0/1391068171 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f28a81461d0 con 0x7f28900038e0 2024-01-24T14:13:23.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.532+0000 7f28a4ff9700 1 -- 172.21.15.29:0/1391068171 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2898016030 con 0x7f28900038e0 2024-01-24T14:13:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:23.532 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T14:13:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-01-24T14:13:23.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew==", 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ==", 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.533 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw==", 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.534 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ==", 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ==", 2024-01-24T14:13:23.535 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA==", 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.536 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw==", 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ==", 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.537 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw==", 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.538 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ==", 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A==", 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:23.539 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ==", 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw==", 2024-01-24T14:13:23.540 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-01-24T14:13:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-24T14:13:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-01-24T14:13:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA==", 2024-01-24T14:13:23.541 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g==", 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:13:23.542 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ==", 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlxK1bFxAAZpfJn+Tcpmq7R9Yq29FvVg==", 2024-01-24T14:13:23.543 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:13:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:13:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:13:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-01-24T14:13:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw==", 2024-01-24T14:13:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.544 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:13:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T14:13:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-01-24T14:13:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA==", 2024-01-24T14:13:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:13:23.545 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T14:13:23.546 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:23.546 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.546 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:23.546 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-01-24T14:13:23.546 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T14:13:23.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.533+0000 7f28aecf3700 1 -- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f289405f270 msgr2=0x7f2894061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.533+0000 7f28aecf3700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f289405f270 0x7f2894061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f28a80c96e0 tx=0x7f289c007bb0).stop 2024-01-24T14:13:23.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.533+0000 7f28aecf3700 1 -- 172.21.15.29:0/1391068171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 msgr2=0x7f28a81571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.533+0000 7f28aecf3700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 0x7f28a81571a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f289000be80 tx=0x7f289801a000).stop 2024-01-24T14:13:23.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.534+0000 7f28aecf3700 1 -- 172.21.15.29:0/1391068171 shutdown_connections 2024-01-24T14:13:23.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.534+0000 7f28aecf3700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28a8146e70 0x7f28a8159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.534+0000 7f28aecf3700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f289405f270 0x7f2894061730 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.534+0000 7f28aecf3700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28900038e0 0x7f28a81571a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.534+0000 7f28aecf3700 1 --2- 172.21.15.29:0/1391068171 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28a8145b30 0x7f28a8146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.534+0000 7f28aecf3700 1 -- 172.21.15.29:0/1391068171 >> 172.21.15.29:0/1391068171 conn(0x7f28a805ec80 msgr2=0x7f28a8151990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:23.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.534+0000 7f28aecf3700 1 -- 172.21.15.29:0/1391068171 shutdown_connections 2024-01-24T14:13:23.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.534+0000 7f28aecf3700 1 -- 172.21.15.29:0/1391068171 wait complete. 2024-01-24T14:13:23.549 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:13:23.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.678+0000 7f0cd7d7d700 1 Processor -- start 2024-01-24T14:13:23.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.678+0000 7f0cd7d7d700 1 -- start start 2024-01-24T14:13:23.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.679+0000 7f0cd7d7d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cd0063ce0 0x7f0cd014e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.679+0000 7f0cd7d7d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd0062840 con 0x7f0cd014df80 2024-01-24T14:13:23.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.679+0000 7f0cd7d7d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd00bd630 con 0x7f0cd00c2750 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.679+0000 7f0cd7d7d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd00bd910 con 0x7f0cd0063ce0 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.679+0000 7f0cd5b19700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0cd014df80 0x7f0cd00c2040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35840/0 (socket says 172.21.15.29:35840) 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.679+0000 7f0cd5b19700 1 -- 172.21.15.29:0/3968650987 learned_addr learned my addr 172.21.15.29:0/3968650987 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.679+0000 7f0cd631a700 1 --2- 172.21.15.29:0/3968650987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cd0063ce0 0x7f0cd014e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.679+0000 7f0cd4b17700 1 -- 172.21.15.29:0/3968650987 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141051901 0 0) 0x7f0cd0062840 con 0x7f0cd014df80 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.679+0000 7f0cd4b17700 1 -- 172.21.15.29:0/3968650987 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cbc0033f0 con 0x7f0cd014df80 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.680+0000 7f0cd4b17700 1 -- 172.21.15.29:0/3968650987 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568002112 0 0) 0x7f0cd00bd630 con 0x7f0cd00c2750 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.680+0000 7f0cd4b17700 1 -- 172.21.15.29:0/3968650987 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd0062840 con 0x7f0cd00c2750 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.680+0000 7f0cd4b17700 1 -- 172.21.15.29:0/3968650987 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1877742284 0 0) 0x7f0cbc0033f0 con 0x7f0cd014df80 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.680+0000 7f0cd631a700 1 -- 172.21.15.29:0/3968650987 >> v1:172.21.15.163:6789/0 conn(0x7f0cd00c2750 legacy=0x7f0cd00bc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.680+0000 7f0cd631a700 1 -- 172.21.15.29:0/3968650987 >> v1:172.21.15.29:6789/0 conn(0x7f0cd014df80 legacy=0x7f0cd00c2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.680+0000 7f0cd631a700 1 -- 172.21.15.29:0/3968650987 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd00bdbf0 con 0x7f0cd0063ce0 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.680+0000 7f0cd631a700 1 --2- 172.21.15.29:0/3968650987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cd0063ce0 0x7f0cd014e360 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f0cc8009e90 tx=0x7f0cc8009780).ready entity=mon.2 client_cookie=d942050a95ad43e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:23.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.680+0000 7f0cd4b17700 1 -- 172.21.15.29:0/3968650987 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800e070 con 0x7f0cd0063ce0 2024-01-24T14:13:23.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.680+0000 7f0cd4b17700 1 -- 172.21.15.29:0/3968650987 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc800b7a0 con 0x7f0cd0063ce0 2024-01-24T14:13:23.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.680+0000 7f0cd4b17700 1 -- 172.21.15.29:0/3968650987 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc40013f0 con 0x7f0cd0063ce0 2024-01-24T14:13:23.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.681+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/3968650987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cd0063ce0 msgr2=0x7f0cd014e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.681+0000 7f0cd7d7d700 1 --2- 172.21.15.29:0/3968650987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cd0063ce0 0x7f0cd014e360 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f0cc8009e90 tx=0x7f0cc8009780).stop 2024-01-24T14:13:23.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.681+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/3968650987 shutdown_connections 2024-01-24T14:13:23.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.681+0000 7f0cd7d7d700 1 --2- 172.21.15.29:0/3968650987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cd0063ce0 0x7f0cd014e360 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.681+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/3968650987 >> 172.21.15.29:0/3968650987 conn(0x7f0cd0064aa0 msgr2=0x7f0cd0064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:23.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.681+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/3968650987 shutdown_connections 2024-01-24T14:13:23.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.681+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/3968650987 wait complete. 2024-01-24T14:13:23.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.682+0000 7f0cd7d7d700 1 Processor -- start 2024-01-24T14:13:23.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.682+0000 7f0cd7d7d700 1 -- start start 2024-01-24T14:13:23.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd7d7d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cd0063ce0 0x7f0cd00acc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd7d7d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cd00c2750 0x7f0cd0163430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd7d7d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd014df80 0x7f0cd0164e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd7d7d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd0164990 con 0x7f0cd00c2750 2024-01-24T14:13:23.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd7d7d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd0163d80 con 0x7f0cd014df80 2024-01-24T14:13:23.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd7d7d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd0164060 con 0x7f0cd0063ce0 2024-01-24T14:13:23.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd5318700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cd00c2750 0x7f0cd0163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd5318700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cd00c2750 0x7f0cd0163430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41368/0 (socket says 172.21.15.29:41368) 2024-01-24T14:13:23.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd5318700 1 -- 172.21.15.29:0/859236302 learned_addr learned my addr 172.21.15.29:0/859236302 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:23.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd5b19700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cd0063ce0 0x7f0cd00acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd631a700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd014df80 0x7f0cd0164e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd5318700 1 -- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cd0063ce0 msgr2=0x7f0cd00acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd5318700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cd0063ce0 0x7f0cd00acc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd5318700 1 -- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd014df80 msgr2=0x7f0cd0164e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd5318700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd014df80 0x7f0cd0164e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd5318700 1 -- 172.21.15.29:0/859236302 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc8009a20 con 0x7f0cd00c2750 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd5b19700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cd0063ce0 0x7f0cd00acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.683+0000 7f0cd631a700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd014df80 0x7f0cd0164e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.684+0000 7f0cd5318700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cd00c2750 0x7f0cd0163430 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f0ccc00c440 tx=0x7f0ccc007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.684+0000 7f0cc1ffb700 1 -- 172.21.15.29:0/859236302 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00f470 con 0x7f0cd00c2750 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.684+0000 7f0cc1ffb700 1 -- 172.21.15.29:0/859236302 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc00f5d0 con 0x7f0cd00c2750 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.684+0000 7f0cc1ffb700 1 -- 172.21.15.29:0/859236302 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00f990 con 0x7f0cd00c2750 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.684+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/859236302 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd0164340 con 0x7f0cd00c2750 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.684+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/859236302 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd00ad130 con 0x7f0cd00c2750 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.685+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/859236302 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd0164b00 con 0x7f0cd00c2750 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.685+0000 7f0cc1ffb700 1 -- 172.21.15.29:0/859236302 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0ccc01b3b0 con 0x7f0cd00c2750 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.686+0000 7f0cc1ffb700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0cb405f270 0x7f0cb4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:23.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.686+0000 7f0cc1ffb700 1 -- 172.21.15.29:0/859236302 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f0ccc079940 con 0x7f0cd00c2750 2024-01-24T14:13:23.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.686+0000 7f0cd5b19700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0cb405f270 0x7f0cb4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:23.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.687+0000 7f0cd5b19700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0cb405f270 0x7f0cb4061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f0cc40011e0 tx=0x7f0cc40018b0).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:23.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.690+0000 7f0cc1ffb700 1 -- 172.21.15.29:0/859236302 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0ccc048b00 con 0x7f0cd00c2750 2024-01-24T14:13:23.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.873+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/859236302 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cd00475e0 con 0x7f0cd00c2750 2024-01-24T14:13:23.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.874+0000 7f0cc1ffb700 1 -- 172.21.15.29:0/859236302 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+14954 (secure 0 0 0) 0x7f0ccc04e460 con 0x7f0cd00c2750 2024-01-24T14:13:23.874 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:23.875 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":86,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:13:23.108660+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:13:23.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.876+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0cb405f270 msgr2=0x7f0cb4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.876+0000 7f0cd7d7d700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0cb405f270 0x7f0cb4061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f0cc40011e0 tx=0x7f0cc40018b0).stop 2024-01-24T14:13:23.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.876+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/859236302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cd00c2750 msgr2=0x7f0cd0163430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:23.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.876+0000 7f0cd7d7d700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cd00c2750 0x7f0cd0163430 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f0ccc00c440 tx=0x7f0ccc007be0).stop 2024-01-24T14:13:23.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.876+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/859236302 shutdown_connections 2024-01-24T14:13:23.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.876+0000 7f0cd7d7d700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd014df80 0x7f0cd0164e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.876+0000 7f0cd7d7d700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cd0063ce0 0x7f0cd00acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.876+0000 7f0cd7d7d700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0cb405f270 0x7f0cb4061730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.876+0000 7f0cd7d7d700 1 --2- 172.21.15.29:0/859236302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cd00c2750 0x7f0cd0163430 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:23.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.876+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/859236302 >> 172.21.15.29:0/859236302 conn(0x7f0cd0064aa0 msgr2=0x7f0cd014efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:23.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.876+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/859236302 shutdown_connections 2024-01-24T14:13:23.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:23.876+0000 7f0cd7d7d700 1 -- 172.21.15.29:0/859236302 wait complete. 2024-01-24T14:13:23.890 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T14:13:23.891 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T14:13:24.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.023+0000 7f28f63b6700 1 Processor -- start 2024-01-24T14:13:24.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.023+0000 7f28f63b6700 1 -- start start 2024-01-24T14:13:24.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28f63b6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f0154100 0x7f28f00c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28f63b6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28f00a7210 0x7f28f00627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28f63b6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28f0069c90 0x7f28f0062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28f63b6700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f0063190 con 0x7f28f00a7210 2024-01-24T14:13:24.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28f63b6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f00a6d80 con 0x7f28f0069c90 2024-01-24T14:13:24.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28f63b6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00bcaf0 con 0x7f28f0154100 2024-01-24T14:13:24.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28ef7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28f00a7210 0x7f28f00627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28ef7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28f00a7210 0x7f28f00627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41370/0 (socket says 172.21.15.29:41370) 2024-01-24T14:13:24.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28ef7fe700 1 -- 172.21.15.29:0/2783640074 learned_addr learned my addr 172.21.15.29:0/2783640074 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:24.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28effff700 1 --2- 172.21.15.29:0/2783640074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f0154100 0x7f28f00c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28f4953700 1 --2- 172.21.15.29:0/2783640074 >> v2:172.21.15.163:3300/0 conn(0x7f28f0069c90 0x7f28f0062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28ef7fe700 1 -- 172.21.15.29:0/2783640074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f0154100 msgr2=0x7f28f00c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28ef7fe700 1 --2- 172.21.15.29:0/2783640074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f0154100 0x7f28f00c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28ef7fe700 1 -- 172.21.15.29:0/2783640074 >> v2:172.21.15.163:3300/0 conn(0x7f28f0069c90 msgr2=0x7f28f0062ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28ef7fe700 1 --2- 172.21.15.29:0/2783640074 >> v2:172.21.15.163:3300/0 conn(0x7f28f0069c90 0x7f28f0062ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28ef7fe700 1 -- 172.21.15.29:0/2783640074 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f00bcd90 con 0x7f28f00a7210 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.024+0000 7f28effff700 1 --2- 172.21.15.29:0/2783640074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f0154100 0x7f28f00c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28ef7fe700 1 --2- 172.21.15.29:0/2783640074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00a7210 0x7f28f00627b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f28f00c58f0 tx=0x7f28e40090f0).ready entity=mon.0 client_cookie=940dbe205445fa4e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28eeffd700 1 -- 172.21.15.29:0/2783640074 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e400b600 con 0x7f28f00a7210 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28eeffd700 1 -- 172.21.15.29:0/2783640074 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e400b760 con 0x7f28f00a7210 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28eeffd700 1 -- 172.21.15.29:0/2783640074 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e400bae0 con 0x7f28f00a7210 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28f63b6700 1 -- 172.21.15.29:0/2783640074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00a7210 msgr2=0x7f28f00627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28f63b6700 1 --2- 172.21.15.29:0/2783640074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00a7210 0x7f28f00627b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f28f00c58f0 tx=0x7f28e40090f0).stop 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28f63b6700 1 -- 172.21.15.29:0/2783640074 shutdown_connections 2024-01-24T14:13:24.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28f63b6700 1 --2- 172.21.15.29:0/2783640074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f0154100 0x7f28f00c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28f63b6700 1 --2- 172.21.15.29:0/2783640074 >> v2:172.21.15.163:3300/0 conn(0x7f28f0069c90 0x7f28f0062ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28f63b6700 1 --2- 172.21.15.29:0/2783640074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00a7210 0x7f28f00627b0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28f63b6700 1 -- 172.21.15.29:0/2783640074 >> 172.21.15.29:0/2783640074 conn(0x7f28f0064390 msgr2=0x7f28f00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:24.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28f63b6700 1 -- 172.21.15.29:0/2783640074 shutdown_connections 2024-01-24T14:13:24.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.025+0000 7f28f63b6700 1 -- 172.21.15.29:0/2783640074 wait complete. 2024-01-24T14:13:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.027+0000 7f28f63b6700 1 Processor -- start 2024-01-24T14:13:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.027+0000 7f28f63b6700 1 -- start start 2024-01-24T14:13:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28f63b6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f0069c90 0x7f28f00beb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28f63b6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00a7210 0x7f28f013e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28f63b6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f0154100 0x7f28f00ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28f63b6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00ac970 con 0x7f28f00a7210 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28f63b6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f01532d0 con 0x7f28f0069c90 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28f63b6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f01535b0 con 0x7f28f0154100 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28ef7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00a7210 0x7f28f013e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28ef7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00a7210 0x7f28f013e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41372/0 (socket says 172.21.15.29:41372) 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28ef7fe700 1 -- 172.21.15.29:0/1027711574 learned_addr learned my addr 172.21.15.29:0/1027711574 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28f4953700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f0154100 0x7f28f00ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28ef7fe700 1 -- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f0154100 msgr2=0x7f28f00ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28ef7fe700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f0154100 0x7f28f00ace00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28effff700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f0069c90 0x7f28f00beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28ef7fe700 1 -- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f0069c90 msgr2=0x7f28f00beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28ef7fe700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f0069c90 0x7f28f00beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28ef7fe700 1 -- 172.21.15.29:0/1027711574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e4009370 con 0x7f28f00a7210 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28f4953700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f0154100 0x7f28f00ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28effff700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f0069c90 0x7f28f00beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:24.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28ef7fe700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00a7210 0x7f28f013e550 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f28f005ee30 tx=0x7f28e4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28d7fff700 1 -- 172.21.15.29:0/1027711574 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e400b8d0 con 0x7f28f00a7210 2024-01-24T14:13:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28d7fff700 1 -- 172.21.15.29:0/1027711574 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e400ba30 con 0x7f28f00a7210 2024-01-24T14:13:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28f63b6700 1 -- 172.21.15.29:0/1027711574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f00ac600 con 0x7f28f00a7210 2024-01-24T14:13:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.028+0000 7f28d7fff700 1 -- 172.21.15.29:0/1027711574 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e401b660 con 0x7f28f00a7210 2024-01-24T14:13:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.029+0000 7f28f63b6700 1 -- 172.21.15.29:0/1027711574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f00ad2f0 con 0x7f28f00a7210 2024-01-24T14:13:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.030+0000 7f28f63b6700 1 -- 172.21.15.29:0/1027711574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f00475e0 con 0x7f28f00a7210 2024-01-24T14:13:24.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.030+0000 7f28d7fff700 1 -- 172.21.15.29:0/1027711574 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f28e401ba10 con 0x7f28f00a7210 2024-01-24T14:13:24.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.030+0000 7f28d7fff700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28dc05f240 0x7f28dc061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.030+0000 7f28d7fff700 1 -- 172.21.15.29:0/1027711574 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f28e407b830 con 0x7f28f00a7210 2024-01-24T14:13:24.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.031+0000 7f28effff700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28dc05f240 0x7f28dc061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.031+0000 7f28effff700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28dc05f240 0x7f28dc061700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f28e0006d80 tx=0x7f28e0008040).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:24.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.035+0000 7f28d7fff700 1 -- 172.21.15.29:0/1027711574 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f28e404c990 con 0x7f28f00a7210 2024-01-24T14:13:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.219+0000 7f28f63b6700 1 -- 172.21.15.29:0/1027711574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f28f00bd070 con 0x7f28f00a7210 2024-01-24T14:13:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.220+0000 7f28d7fff700 1 -- 172.21.15.29:0/1027711574 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v86) v1 ==== 110+0+10 (secure 0 0 0) 0x7f28e401f0b0 con 0x7f28f00a7210 2024-01-24T14:13:24.221 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T14:13:24.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.222+0000 7f28f63b6700 1 -- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28dc05f240 msgr2=0x7f28dc061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.222+0000 7f28f63b6700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28dc05f240 0x7f28dc061700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f28e0006d80 tx=0x7f28e0008040).stop 2024-01-24T14:13:24.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.222+0000 7f28f63b6700 1 -- 172.21.15.29:0/1027711574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00a7210 msgr2=0x7f28f013e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.222+0000 7f28f63b6700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00a7210 0x7f28f013e550 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f28f005ee30 tx=0x7f28e4017000).stop 2024-01-24T14:13:24.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.222+0000 7f28f63b6700 1 -- 172.21.15.29:0/1027711574 shutdown_connections 2024-01-24T14:13:24.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.222+0000 7f28f63b6700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f0154100 0x7f28f00ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.222+0000 7f28f63b6700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f0069c90 0x7f28f00beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.222+0000 7f28f63b6700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28dc05f240 0x7f28dc061700 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.222+0000 7f28f63b6700 1 --2- 172.21.15.29:0/1027711574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28f00a7210 0x7f28f013e550 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.222+0000 7f28f63b6700 1 -- 172.21.15.29:0/1027711574 >> 172.21.15.29:0/1027711574 conn(0x7f28f0064390 msgr2=0x7f28f00a4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:24.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.222+0000 7f28f63b6700 1 -- 172.21.15.29:0/1027711574 shutdown_connections 2024-01-24T14:13:24.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.222+0000 7f28f63b6700 1 -- 172.21.15.29:0/1027711574 wait complete. 2024-01-24T14:13:24.236 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T14:13:24.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.369+0000 7f3d3a556700 1 Processor -- start 2024-01-24T14:13:24.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.369+0000 7f3d3a556700 1 -- start start 2024-01-24T14:13:24.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.370+0000 7f3d3a556700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d340692a0 0x7f3d34065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.370+0000 7f3d3a556700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d340c3e70 con 0x7f3d34066370 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.370+0000 7f3d3a556700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d340c4300 con 0x7f3d34069d90 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.370+0000 7f3d3a556700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d340c3260 con 0x7f3d340692a0 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.370+0000 7f3d337fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3d34066370 0x7f3d34069680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35844/0 (socket says 172.21.15.29:35844) 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.370+0000 7f3d337fe700 1 -- 172.21.15.29:0/3069523971 learned_addr learned my addr 172.21.15.29:0/3069523971 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.370+0000 7f3d33fff700 1 --2- 172.21.15.29:0/3069523971 >> v2:172.21.15.163:3300/0 conn(0x7f3d340692a0 0x7f3d34065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.370+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445447136 0 0) 0x7f3d340c3e70 con 0x7f3d34066370 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.370+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d200033f0 con 0x7f3d34066370 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868220980 0 0) 0x7f3d340c4300 con 0x7f3d34069d90 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d340c3e70 con 0x7f3d34069d90 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1766944801 0 0) 0x7f3d200033f0 con 0x7f3d34066370 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 >> v2:172.21.15.163:3300/0 conn(0x7f3d340692a0 msgr2=0x7f3d34065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 --2- 172.21.15.29:0/3069523971 >> v2:172.21.15.163:3300/0 conn(0x7f3d340692a0 0x7f3d34065e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 >> v1:172.21.15.163:6789/0 conn(0x7f3d34069d90 legacy=0x7f3d3415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d340c3540 con 0x7f3d34066370 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f3d1c001470 con 0x7f3d34066370 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 >> v1:172.21.15.29:6789/0 conn(0x7f3d34066370 legacy=0x7f3d34069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 --2- 172.21.15.29:0/3069523971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d20003ac0 0x7f3d20003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 --2- 172.21.15.29:0/3069523971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 0x7f3d20004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 --2- 172.21.15.29:0/3069523971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d20005000 0x7f3d2000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d340c3260 con 0x7f3d20004690 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2000bf60 con 0x7f3d20005000 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2000c120 con 0x7f3d20003ac0 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d337fe700 1 --2- 172.21.15.29:0/3069523971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 0x7f3d20004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d33fff700 1 --2- 172.21.15.29:0/3069523971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d20003ac0 0x7f3d20003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d38af3700 1 --2- 172.21.15.29:0/3069523971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d20005000 0x7f3d2000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d337fe700 1 -- 172.21.15.29:0/3069523971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d20003ac0 msgr2=0x7f3d20003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:24.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d337fe700 1 --2- 172.21.15.29:0/3069523971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d20003ac0 0x7f3d20003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d337fe700 1 -- 172.21.15.29:0/3069523971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d20005000 msgr2=0x7f3d2000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d337fe700 1 --2- 172.21.15.29:0/3069523971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d20005000 0x7f3d2000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d337fe700 1 -- 172.21.15.29:0/3069523971 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d200033f0 con 0x7f3d20004690 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d38af3700 1 --2- 172.21.15.29:0/3069523971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d20005000 0x7f3d2000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d38af3700 1 -- 172.21.15.29:0/3069523971 reap_dead start 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.371+0000 7f3d337fe700 1 --2- 172.21.15.29:0/3069523971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 0x7f3d20004b10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f3d1c007f80 tx=0x7f3d1c001d40).ready entity=mon.0 client_cookie=b6dad86dd805a28d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.372+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c00c9e0 con 0x7f3d20004690 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.372+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c00cb40 con 0x7f3d20004690 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.372+0000 7f3d32ffd700 1 -- 172.21.15.29:0/3069523971 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c0070d0 con 0x7f3d20004690 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.372+0000 7f3d3a556700 1 -- 172.21.15.29:0/3069523971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 msgr2=0x7f3d20004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.372+0000 7f3d3a556700 1 --2- 172.21.15.29:0/3069523971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 0x7f3d20004b10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f3d1c007f80 tx=0x7f3d1c001d40).stop 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.372+0000 7f3d3a556700 1 -- 172.21.15.29:0/3069523971 shutdown_connections 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.372+0000 7f3d3a556700 1 --2- 172.21.15.29:0/3069523971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 0x7f3d20004b10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.372+0000 7f3d3a556700 1 -- 172.21.15.29:0/3069523971 >> 172.21.15.29:0/3069523971 conn(0x7f3d34064390 msgr2=0x7f3d340c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:24.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.372+0000 7f3d3a556700 1 -- 172.21.15.29:0/3069523971 shutdown_connections 2024-01-24T14:13:24.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.372+0000 7f3d3a556700 1 -- 172.21.15.29:0/3069523971 wait complete. 2024-01-24T14:13:24.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.374+0000 7f3d3a556700 1 Processor -- start 2024-01-24T14:13:24.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.374+0000 7f3d3a556700 1 -- start start 2024-01-24T14:13:24.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.375+0000 7f3d3a556700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 0x7f3d34151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.375+0000 7f3d3a556700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d34151780 0x7f3d3412f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.375+0000 7f3d3a556700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d34165560 0x7f3d3412f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.375+0000 7f3d3a556700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3413fe90 con 0x7f3d20004690 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.375+0000 7f3d3a556700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3412fb60 con 0x7f3d34151780 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.375+0000 7f3d3a556700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d34153ea0 con 0x7f3d34165560 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.375+0000 7f3d33fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 0x7f3d34151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.375+0000 7f3d33fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 0x7f3d34151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41396/0 (socket says 172.21.15.29:41396) 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.375+0000 7f3d33fff700 1 -- 172.21.15.29:0/3396507786 learned_addr learned my addr 172.21.15.29:0/3396507786 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.375+0000 7f3d337fe700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d34151780 0x7f3d3412f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.375+0000 7f3d38af3700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d34165560 0x7f3d3412f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.376+0000 7f3d33fff700 1 -- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d34165560 msgr2=0x7f3d3412f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.376+0000 7f3d33fff700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d34165560 0x7f3d3412f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.376+0000 7f3d33fff700 1 -- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d34151780 msgr2=0x7f3d3412f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.376+0000 7f3d33fff700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d34151780 0x7f3d3412f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.376+0000 7f3d33fff700 1 -- 172.21.15.29:0/3396507786 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d34154180 con 0x7f3d20004690 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.376+0000 7f3d337fe700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d34151780 0x7f3d3412f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.376+0000 7f3d33fff700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 0x7f3d34151290 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f3d24011a20 tx=0x7f3d24009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.376+0000 7f3d2bfff700 1 -- 172.21.15.29:0/3396507786 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400f940 con 0x7f3d20004690 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.376+0000 7f3d2bfff700 1 -- 172.21.15.29:0/3396507786 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2400faa0 con 0x7f3d20004690 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.376+0000 7f3d2bfff700 1 -- 172.21.15.29:0/3396507786 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400fe20 con 0x7f3d20004690 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.376+0000 7f3d3a556700 1 -- 172.21.15.29:0/3396507786 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3413f840 con 0x7f3d20004690 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.376+0000 7f3d3a556700 1 -- 172.21.15.29:0/3396507786 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d34153100 con 0x7f3d20004690 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.377+0000 7f3d3a556700 1 -- 172.21.15.29:0/3396507786 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d3413fce0 con 0x7f3d20004690 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.378+0000 7f3d2bfff700 1 -- 172.21.15.29:0/3396507786 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3d24014650 con 0x7f3d20004690 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.378+0000 7f3d2bfff700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3d1405f270 0x7f3d14061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.378+0000 7f3d2bfff700 1 -- 172.21.15.29:0/3396507786 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f3d2407b0c0 con 0x7f3d20004690 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.378+0000 7f3d337fe700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3d1405f270 0x7f3d14061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.379+0000 7f3d337fe700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3d1405f270 0x7f3d14061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f3d1c001120 tx=0x7f3d1c015000).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:24.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.383+0000 7f3d2bfff700 1 -- 172.21.15.29:0/3396507786 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3d2404c0a0 con 0x7f3d20004690 2024-01-24T14:13:24.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.561+0000 7f3d3a556700 1 -- 172.21.15.29:0/3396507786 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3d340475e0 con 0x7f3d20004690 2024-01-24T14:13:24.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.562+0000 7f3d2bfff700 1 -- 172.21.15.29:0/3396507786 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3d2404f6c0 con 0x7f3d20004690 2024-01-24T14:13:24.562 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T14:13:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.563+0000 7f3d3a556700 1 -- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3d1405f270 msgr2=0x7f3d14061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.563+0000 7f3d3a556700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3d1405f270 0x7f3d14061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f3d1c001120 tx=0x7f3d1c015000).stop 2024-01-24T14:13:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.563+0000 7f3d3a556700 1 -- 172.21.15.29:0/3396507786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 msgr2=0x7f3d34151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.563+0000 7f3d3a556700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 0x7f3d34151290 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f3d24011a20 tx=0x7f3d24009060).stop 2024-01-24T14:13:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.563+0000 7f3d3a556700 1 -- 172.21.15.29:0/3396507786 shutdown_connections 2024-01-24T14:13:24.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.563+0000 7f3d3a556700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d34165560 0x7f3d3412f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.563+0000 7f3d3a556700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3d1405f270 0x7f3d14061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.563+0000 7f3d3a556700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d20004690 0x7f3d34151290 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.563+0000 7f3d3a556700 1 --2- 172.21.15.29:0/3396507786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d34151780 0x7f3d3412f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.563+0000 7f3d3a556700 1 -- 172.21.15.29:0/3396507786 >> 172.21.15.29:0/3396507786 conn(0x7f3d34064390 msgr2=0x7f3d340c4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.564+0000 7f3d3a556700 1 -- 172.21.15.29:0/3396507786 shutdown_connections 2024-01-24T14:13:24.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.564+0000 7f3d3a556700 1 -- 172.21.15.29:0/3396507786 wait complete. 2024-01-24T14:13:24.578 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-24T14:13:24.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.707+0000 7f4cbda30700 1 Processor -- start 2024-01-24T14:13:24.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.707+0000 7f4cbda30700 1 -- start start 2024-01-24T14:13:24.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.708+0000 7f4cbda30700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4cb81540a0 0x7f4cb80c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.708+0000 7f4cbda30700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4cb81550f0 0x7f4cb80627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.708+0000 7f4cbda30700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb8065360 con 0x7f4cb8062ca0 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.708+0000 7f4cbda30700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb80a74f0 con 0x7f4cb81550f0 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.708+0000 7f4cbda30700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb80a6ce0 con 0x7f4cb81540a0 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.708+0000 7f4cb67fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4cb81550f0 0x7f4cb80627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb77fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4cb8062ca0 0x7f4cb8157c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35848/0 (socket says 172.21.15.29:35848) 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb77fe700 1 -- 172.21.15.29:0/4162463791 learned_addr learned my addr 172.21.15.29:0/4162463791 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb67fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4cb81550f0 0x7f4cb80627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41408/0 (socket says 172.21.15.29:41408) 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb6ffd700 1 --2- 172.21.15.29:0/4162463791 >> v2:172.21.15.163:3300/0 conn(0x7f4cb81540a0 0x7f4cb80c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb67fc700 1 -- 172.21.15.29:0/4162463791 >> v2:172.21.15.163:3300/0 conn(0x7f4cb81540a0 msgr2=0x7f4cb80c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb67fc700 1 --2- 172.21.15.29:0/4162463791 >> v2:172.21.15.163:3300/0 conn(0x7f4cb81540a0 0x7f4cb80c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb67fc700 1 -- 172.21.15.29:0/4162463791 >> v1:172.21.15.29:6789/0 conn(0x7f4cb8062ca0 legacy=0x7f4cb8157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb5ffb700 1 -- 172.21.15.29:0/4162463791 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389786622 0 0) 0x7f4cb8065360 con 0x7f4cb8062ca0 2024-01-24T14:13:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb67fc700 1 -- 172.21.15.29:0/4162463791 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb80636f0 con 0x7f4cb81550f0 2024-01-24T14:13:24.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb67fc700 1 --2- 172.21.15.29:0/4162463791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cb81550f0 0x7f4cb80627b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f4cb806af90 tx=0x7f4cac009060).ready entity=mon.0 client_cookie=b61bc492d9a0d37b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:24.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb5ffb700 1 -- 172.21.15.29:0/4162463791 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac00b580 con 0x7f4cb81550f0 2024-01-24T14:13:24.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb5ffb700 1 -- 172.21.15.29:0/4162463791 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cac00b6e0 con 0x7f4cb81550f0 2024-01-24T14:13:24.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.709+0000 7f4cb5ffb700 1 -- 172.21.15.29:0/4162463791 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac00ba60 con 0x7f4cb81550f0 2024-01-24T14:13:24.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.710+0000 7f4cbda30700 1 -- 172.21.15.29:0/4162463791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cb81550f0 msgr2=0x7f4cb80627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.710+0000 7f4cbda30700 1 --2- 172.21.15.29:0/4162463791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cb81550f0 0x7f4cb80627b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f4cb806af90 tx=0x7f4cac009060).stop 2024-01-24T14:13:24.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.710+0000 7f4cbda30700 1 -- 172.21.15.29:0/4162463791 shutdown_connections 2024-01-24T14:13:24.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.710+0000 7f4cbda30700 1 --2- 172.21.15.29:0/4162463791 >> v2:172.21.15.163:3300/0 conn(0x7f4cb81540a0 0x7f4cb80c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.710+0000 7f4cbda30700 1 --2- 172.21.15.29:0/4162463791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cb81550f0 0x7f4cb80627b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.710+0000 7f4cbda30700 1 -- 172.21.15.29:0/4162463791 >> 172.21.15.29:0/4162463791 conn(0x7f4cb80c85c0 msgr2=0x7f4cb80c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:24.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.710+0000 7f4cbda30700 1 -- 172.21.15.29:0/4162463791 shutdown_connections 2024-01-24T14:13:24.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.710+0000 7f4cbda30700 1 -- 172.21.15.29:0/4162463791 wait complete. 2024-01-24T14:13:24.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.711+0000 7f4cbda30700 1 Processor -- start 2024-01-24T14:13:24.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.712+0000 7f4cbda30700 1 -- start start 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.712+0000 7f4cbda30700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb8062ca0 0x7f4cb814cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.712+0000 7f4cbda30700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cb81540a0 0x7f4cb80bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.712+0000 7f4cbda30700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb81550f0 0x7f4cb80aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.712+0000 7f4cbda30700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb814f0d0 con 0x7f4cb81540a0 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.712+0000 7f4cbda30700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb814f3b0 con 0x7f4cb8062ca0 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.712+0000 7f4cbda30700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb814f690 con 0x7f4cb81550f0 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.712+0000 7f4cb67fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cb81540a0 0x7f4cb80bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.712+0000 7f4cb67fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cb81540a0 0x7f4cb80bf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41420/0 (socket says 172.21.15.29:41420) 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4cb67fc700 1 -- 172.21.15.29:0/1362529406 learned_addr learned my addr 172.21.15.29:0/1362529406 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4cb6ffd700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb8062ca0 0x7f4cb814cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4cb77fe700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb81550f0 0x7f4cb80aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4cb67fc700 1 -- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb81550f0 msgr2=0x7f4cb80aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4cb67fc700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb81550f0 0x7f4cb80aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4cb67fc700 1 -- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb8062ca0 msgr2=0x7f4cb814cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4cb67fc700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb8062ca0 0x7f4cb814cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4cb67fc700 1 -- 172.21.15.29:0/1362529406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac0092a0 con 0x7f4cb81540a0 2024-01-24T14:13:24.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4cb67fc700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cb81540a0 0x7f4cb80bf5f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f4cac009e80 tx=0x7f4cac017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:24.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4c9effd700 1 -- 172.21.15.29:0/1362529406 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac000bc0 con 0x7f4cb81540a0 2024-01-24T14:13:24.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4c9effd700 1 -- 172.21.15.29:0/1362529406 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cac00b850 con 0x7f4cb81540a0 2024-01-24T14:13:24.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4cbda30700 1 -- 172.21.15.29:0/1362529406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb814f970 con 0x7f4cb81540a0 2024-01-24T14:13:24.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4c9effd700 1 -- 172.21.15.29:0/1362529406 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac000d20 con 0x7f4cb81540a0 2024-01-24T14:13:24.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.713+0000 7f4cbda30700 1 -- 172.21.15.29:0/1362529406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb814d440 con 0x7f4cb81540a0 2024-01-24T14:13:24.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.714+0000 7f4cbda30700 1 -- 172.21.15.29:0/1362529406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb814fe90 con 0x7f4cb81540a0 2024-01-24T14:13:24.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.715+0000 7f4c9effd700 1 -- 172.21.15.29:0/1362529406 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4cac01b8e0 con 0x7f4cb81540a0 2024-01-24T14:13:24.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.715+0000 7f4c9effd700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ca405f270 0x7f4ca4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:24.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.715+0000 7f4c9effd700 1 -- 172.21.15.29:0/1362529406 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f4cac07bbd0 con 0x7f4cb81540a0 2024-01-24T14:13:24.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.715+0000 7f4cb6ffd700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ca405f270 0x7f4ca4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:24.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.716+0000 7f4cb6ffd700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ca405f270 0x7f4ca4061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f4cb80c34d0 tx=0x7f4ca8008040).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.720+0000 7f4c9effd700 1 -- 172.21.15.29:0/1362529406 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4cb814fe90 con 0x7f4cb81540a0 2024-01-24T14:13:24.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.904+0000 7f4cbda30700 1 -- 172.21.15.29:0/1362529406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4cb814fe90 con 0x7f4cb81540a0 2024-01-24T14:13:24.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.904+0000 7f4c9effd700 1 -- 172.21.15.29:0/1362529406 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4cb814fe90 con 0x7f4cb81540a0 2024-01-24T14:13:24.905 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:24.905 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T14:13:24.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:24.905 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-01-24T14:13:24.906 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "device_health_metrics" 2024-01-24T14:13:24.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:24.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:24.906 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 2, 2024-01-24T14:13:24.906 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "rbd" 2024-01-24T14:13:24.906 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:24.906 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T14:13:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.906+0000 7f4cbda30700 1 -- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ca405f270 msgr2=0x7f4ca4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.906+0000 7f4cbda30700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ca405f270 0x7f4ca4061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f4cb80c34d0 tx=0x7f4ca8008040).stop 2024-01-24T14:13:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.906+0000 7f4cbda30700 1 -- 172.21.15.29:0/1362529406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cb81540a0 msgr2=0x7f4cb80bf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.906+0000 7f4cbda30700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cb81540a0 0x7f4cb80bf5f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f4cac009e80 tx=0x7f4cac017000).stop 2024-01-24T14:13:24.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.906+0000 7f4cbda30700 1 -- 172.21.15.29:0/1362529406 shutdown_connections 2024-01-24T14:13:24.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.906+0000 7f4cbda30700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb81550f0 0x7f4cb80aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.906+0000 7f4cbda30700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb8062ca0 0x7f4cb814cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.906+0000 7f4cbda30700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ca405f270 0x7f4ca4061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.906+0000 7f4cbda30700 1 --2- 172.21.15.29:0/1362529406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cb81540a0 0x7f4cb80bf5f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:24.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.906+0000 7f4cbda30700 1 -- 172.21.15.29:0/1362529406 >> 172.21.15.29:0/1362529406 conn(0x7f4cb80c85c0 msgr2=0x7f4cb8066040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:24.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.907+0000 7f4cbda30700 1 -- 172.21.15.29:0/1362529406 shutdown_connections 2024-01-24T14:13:24.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:24.907+0000 7f4cbda30700 1 -- 172.21.15.29:0/1362529406 wait complete. 2024-01-24T14:13:24.921 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-24T14:13:24.921 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2024-01-24T14:13:25.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.055+0000 7f70fea91700 1 Processor -- start 2024-01-24T14:13:25.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.055+0000 7f70fea91700 1 -- start start 2024-01-24T14:13:25.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.056+0000 7f70fea91700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70f81350b0 0x7f70f80664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:25.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.056+0000 7f70fea91700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f8153240 0x7f70f80669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:25.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.056+0000 7f70fea91700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f8068450 con 0x7f70f80c31b0 2024-01-24T14:13:25.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.056+0000 7f70fea91700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f8066f70 con 0x7f70f81350b0 2024-01-24T14:13:25.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.056+0000 7f70fea91700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f8067b20 con 0x7f70f8153240 2024-01-24T14:13:25.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.057+0000 7f70fc82d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70f81350b0 0x7f70f80664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.057+0000 7f70fd02e700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f70f80c31b0 0x7f70f80c3600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35854/0 (socket says 172.21.15.29:35854) 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.057+0000 7f70fd02e700 1 -- 172.21.15.29:0/2481924015 learned_addr learned my addr 172.21.15.29:0/2481924015 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.057+0000 7f70f7fff700 1 --2- 172.21.15.29:0/2481924015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f8153240 0x7f70f80669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.057+0000 7f70fc82d700 1 -- 172.21.15.29:0/2481924015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f8153240 msgr2=0x7f70f80669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.057+0000 7f70fc82d700 1 --2- 172.21.15.29:0/2481924015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f8153240 0x7f70f80669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.057+0000 7f70fc82d700 1 -- 172.21.15.29:0/2481924015 >> v1:172.21.15.29:6789/0 conn(0x7f70f80c31b0 legacy=0x7f70f80c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.057+0000 7f70fc82d700 1 -- 172.21.15.29:0/2481924015 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f8067e00 con 0x7f70f81350b0 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.057+0000 7f70fc82d700 1 --2- 172.21.15.29:0/2481924015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70f81350b0 0x7f70f80664b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f70ec009760 tx=0x7f70ec009060).ready entity=mon.0 client_cookie=6da436f3b9983598 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.057+0000 7f70f77fe700 1 -- 172.21.15.29:0/2481924015 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec0090d0 con 0x7f70f81350b0 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.057+0000 7f70f77fe700 1 -- 172.21.15.29:0/2481924015 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec00f510 con 0x7f70f81350b0 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.057+0000 7f70f77fe700 1 -- 172.21.15.29:0/2481924015 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00f890 con 0x7f70f81350b0 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.058+0000 7f70fea91700 1 -- 172.21.15.29:0/2481924015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70f81350b0 msgr2=0x7f70f80664b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:25.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.058+0000 7f70fea91700 1 --2- 172.21.15.29:0/2481924015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70f81350b0 0x7f70f80664b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f70ec009760 tx=0x7f70ec009060).stop 2024-01-24T14:13:25.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.058+0000 7f70fea91700 1 -- 172.21.15.29:0/2481924015 shutdown_connections 2024-01-24T14:13:25.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.058+0000 7f70fea91700 1 --2- 172.21.15.29:0/2481924015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70f81350b0 0x7f70f80664b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:25.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.058+0000 7f70fea91700 1 --2- 172.21.15.29:0/2481924015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f8153240 0x7f70f80669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:25.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.058+0000 7f70fea91700 1 -- 172.21.15.29:0/2481924015 >> 172.21.15.29:0/2481924015 conn(0x7f70f80c8bb0 msgr2=0x7f70f80c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:25.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.058+0000 7f70fea91700 1 -- 172.21.15.29:0/2481924015 shutdown_connections 2024-01-24T14:13:25.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.058+0000 7f70fea91700 1 -- 172.21.15.29:0/2481924015 wait complete. 2024-01-24T14:13:25.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.059+0000 7f70fea91700 1 Processor -- start 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.059+0000 7f70fea91700 1 -- start start 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.060+0000 7f70fea91700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f80c31b0 0x7f70f8133cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.060+0000 7f70fea91700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70f81350b0 0x7f70f80bfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.060+0000 7f70fea91700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f8153240 0x7f70f80c0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.060+0000 7f70fea91700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f80c2790 con 0x7f70f81350b0 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.060+0000 7f70fea91700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f80c0710 con 0x7f70f8153240 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.060+0000 7f70fea91700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f80c2140 con 0x7f70f80c31b0 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.060+0000 7f70f7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70f81350b0 0x7f70f80bfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.060+0000 7f70f7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70f81350b0 0x7f70f80bfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41434/0 (socket says 172.21.15.29:41434) 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.060+0000 7f70f7fff700 1 -- 172.21.15.29:0/2262421843 learned_addr learned my addr 172.21.15.29:0/2262421843 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.060+0000 7f70fc82d700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f80c31b0 0x7f70f8133cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.060+0000 7f70fd02e700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f8153240 0x7f70f80c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:25.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.061+0000 7f70f7fff700 1 -- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f80c31b0 msgr2=0x7f70f8133cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.061+0000 7f70f7fff700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f80c31b0 0x7f70f8133cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.061+0000 7f70f7fff700 1 -- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f8153240 msgr2=0x7f70f80c0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.061+0000 7f70f7fff700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f8153240 0x7f70f80c0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.061+0000 7f70f7fff700 1 -- 172.21.15.29:0/2262421843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec0092a0 con 0x7f70f81350b0 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.061+0000 7f70f7fff700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70f81350b0 0x7f70f80bfc50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f70e400d860 tx=0x7f70e4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.061+0000 7f70f4ff9700 1 -- 172.21.15.29:0/2262421843 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4011590 con 0x7f70f81350b0 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.061+0000 7f70f4ff9700 1 -- 172.21.15.29:0/2262421843 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e40116f0 con 0x7f70f81350b0 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.061+0000 7f70fea91700 1 -- 172.21.15.29:0/2262421843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f80c2420 con 0x7f70f81350b0 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.061+0000 7f70f4ff9700 1 -- 172.21.15.29:0/2262421843 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4011a70 con 0x7f70f81350b0 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.061+0000 7f70fea91700 1 -- 172.21.15.29:0/2262421843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f80c2c20 con 0x7f70f81350b0 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.062+0000 7f70fea91700 1 -- 172.21.15.29:0/2262421843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f80c2e40 con 0x7f70f81350b0 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.063+0000 7f70f4ff9700 1 -- 172.21.15.29:0/2262421843 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f70e4012420 con 0x7f70f81350b0 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.063+0000 7f70f4ff9700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f70dc05f240 0x7f70dc061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.063+0000 7f70f4ff9700 1 -- 172.21.15.29:0/2262421843 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f70e4079d90 con 0x7f70f81350b0 2024-01-24T14:13:25.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.063+0000 7f70fc82d700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f70dc05f240 0x7f70dc061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:25.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.064+0000 7f70fc82d700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f70dc05f240 0x7f70dc061700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f70ec006f70 tx=0x7f70ec017000).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:25.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.067+0000 7f70f4ff9700 1 -- 172.21.15.29:0/2262421843 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f70e404add0 con 0x7f70f81350b0 2024-01-24T14:13:25.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.250+0000 7f70fea91700 1 -- 172.21.15.29:0/2262421843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f70f80c1fa0 con 0x7f70f81350b0 2024-01-24T14:13:25.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.844+0000 7f70f4ff9700 1 -- 172.21.15.29:0/2262421843 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) v1 ==== 152+0+0 (secure 0 0 0) 0x7f70f80c1fa0 con 0x7f70f81350b0 2024-01-24T14:13:25.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.849+0000 7f70fea91700 1 -- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f70dc05f240 msgr2=0x7f70dc061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:25.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.849+0000 7f70fea91700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f70dc05f240 0x7f70dc061700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f70ec006f70 tx=0x7f70ec017000).stop 2024-01-24T14:13:25.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.849+0000 7f70fea91700 1 -- 172.21.15.29:0/2262421843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70f81350b0 msgr2=0x7f70f80bfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:25.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.849+0000 7f70fea91700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70f81350b0 0x7f70f80bfc50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f70e400d860 tx=0x7f70e4008040).stop 2024-01-24T14:13:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.850+0000 7f70fea91700 1 -- 172.21.15.29:0/2262421843 shutdown_connections 2024-01-24T14:13:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.850+0000 7f70fea91700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f8153240 0x7f70f80c0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.850+0000 7f70fea91700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f80c31b0 0x7f70f8133cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.850+0000 7f70fea91700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f70dc05f240 0x7f70dc061700 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.850+0000 7f70fea91700 1 --2- 172.21.15.29:0/2262421843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70f81350b0 0x7f70f80bfc50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.850+0000 7f70fea91700 1 -- 172.21.15.29:0/2262421843 >> 172.21.15.29:0/2262421843 conn(0x7f70f80c8bb0 msgr2=0x7f70f8154f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:25.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.851+0000 7f70fea91700 1 -- 172.21.15.29:0/2262421843 shutdown_connections 2024-01-24T14:13:25.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:25.851+0000 7f70fea91700 1 -- 172.21.15.29:0/2262421843 wait complete. 2024-01-24T14:13:25.854 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-01-24T14:13:25.869 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-24T14:13:26.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.009+0000 7f057dbfb700 1 Processor -- start 2024-01-24T14:13:26.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.009+0000 7f057dbfb700 1 -- start start 2024-01-24T14:13:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.009+0000 7f057dbfb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f057814e4f0 0x7f057814e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.009+0000 7f057dbfb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0578065a80 0x7f0578157c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.010+0000 7f057dbfb700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0578158220 con 0x7f05780627b0 2024-01-24T14:13:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.010+0000 7f057dbfb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0578154850 con 0x7f057814e4f0 2024-01-24T14:13:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.010+0000 7f057dbfb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0578158500 con 0x7f0578065a80 2024-01-24T14:13:26.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.010+0000 7f0576ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0578065a80 0x7f0578157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.010+0000 7f0576ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0578065a80 0x7f0578157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:52010/0 (socket says 172.21.15.29:52010) 2024-01-24T14:13:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.010+0000 7f0576ffd700 1 -- 172.21.15.29:0/1536329194 learned_addr learned my addr 172.21.15.29:0/1536329194 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.010+0000 7f05777fe700 1 --2- 172.21.15.29:0/1536329194 >> v2:172.21.15.163:3300/0 conn(0x7f057814e4f0 0x7f057814e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.010+0000 7f05777fe700 1 -- 172.21.15.29:0/1536329194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0578065a80 msgr2=0x7f0578157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.010+0000 7f05777fe700 1 --2- 172.21.15.29:0/1536329194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0578065a80 0x7f0578157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.010+0000 7f05777fe700 1 -- 172.21.15.29:0/1536329194 >> v1:172.21.15.163:6789/0 conn(0x7f05780627b0 legacy=0x7f0578154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.010+0000 7f05777fe700 1 -- 172.21.15.29:0/1536329194 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05781587e0 con 0x7f057814e4f0 2024-01-24T14:13:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.010+0000 7f0576ffd700 1 --2- 172.21.15.29:0/1536329194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0578065a80 0x7f0578157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.011+0000 7f05777fe700 1 --2- 172.21.15.29:0/1536329194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f057814e4f0 0x7f057814e8d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f0568009eb0 tx=0x7f0568009780).ready entity=mon.1 client_cookie=c5662f0fd9a1e3f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.011+0000 7f05767fc700 1 -- 172.21.15.29:0/1536329194 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056800e070 con 0x7f057814e4f0 2024-01-24T14:13:26.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.011+0000 7f05767fc700 1 -- 172.21.15.29:0/1536329194 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056800bc30 con 0x7f057814e4f0 2024-01-24T14:13:26.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.011+0000 7f05767fc700 1 -- 172.21.15.29:0/1536329194 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568012410 con 0x7f057814e4f0 2024-01-24T14:13:26.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.011+0000 7f057dbfb700 1 -- 172.21.15.29:0/1536329194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f057814e4f0 msgr2=0x7f057814e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:26.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.011+0000 7f057dbfb700 1 --2- 172.21.15.29:0/1536329194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f057814e4f0 0x7f057814e8d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f0568009eb0 tx=0x7f0568009780).stop 2024-01-24T14:13:26.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.011+0000 7f057dbfb700 1 -- 172.21.15.29:0/1536329194 shutdown_connections 2024-01-24T14:13:26.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.011+0000 7f057dbfb700 1 --2- 172.21.15.29:0/1536329194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f057814e4f0 0x7f057814e8d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:26.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.011+0000 7f057dbfb700 1 --2- 172.21.15.29:0/1536329194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0578065a80 0x7f0578157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:26.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.011+0000 7f057dbfb700 1 -- 172.21.15.29:0/1536329194 >> 172.21.15.29:0/1536329194 conn(0x7f05780c7330 msgr2=0x7f05780c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:26.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.011+0000 7f057dbfb700 1 -- 172.21.15.29:0/1536329194 shutdown_connections 2024-01-24T14:13:26.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.011+0000 7f057dbfb700 1 -- 172.21.15.29:0/1536329194 wait complete. 2024-01-24T14:13:26.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.013+0000 7f057dbfb700 1 Processor -- start 2024-01-24T14:13:26.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.013+0000 7f057dbfb700 1 -- start start 2024-01-24T14:13:26.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f057dbfb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05780627b0 0x7f057813d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:26.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f057dbfb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578065a80 0x7f0578166fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:26.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f057dbfb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f057814e4f0 0x7f0578156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:26.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f057dbfb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057813d2f0 con 0x7f057814e4f0 2024-01-24T14:13:26.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f057dbfb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05781674a0 con 0x7f0578065a80 2024-01-24T14:13:26.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f057dbfb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057813c6e0 con 0x7f05780627b0 2024-01-24T14:13:26.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f0577fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f057814e4f0 0x7f0578156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:26.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f0577fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f057814e4f0 0x7f0578156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41444/0 (socket says 172.21.15.29:41444) 2024-01-24T14:13:26.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f0577fff700 1 -- 172.21.15.29:0/110662414 learned_addr learned my addr 172.21.15.29:0/110662414 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:26.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f05777fe700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05780627b0 0x7f057813d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:26.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f0576ffd700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578065a80 0x7f0578166fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:26.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f0577fff700 1 -- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05780627b0 msgr2=0x7f057813d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:26.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f0577fff700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05780627b0 0x7f057813d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:26.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f0577fff700 1 -- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578065a80 msgr2=0x7f0578166fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:26.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f0577fff700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578065a80 0x7f0578166fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:26.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f0577fff700 1 -- 172.21.15.29:0/110662414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560009370 con 0x7f057814e4f0 2024-01-24T14:13:26.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f05777fe700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05780627b0 0x7f057813d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:26.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.014+0000 7f0576ffd700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578065a80 0x7f0578166fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:26.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.015+0000 7f0577fff700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f057814e4f0 0x7f0578156df0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f057000c440 tx=0x7f0570007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:26.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.015+0000 7f056f7fe700 1 -- 172.21.15.29:0/110662414 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05700123f0 con 0x7f057814e4f0 2024-01-24T14:13:26.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.015+0000 7f056f7fe700 1 -- 172.21.15.29:0/110662414 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0570001e90 con 0x7f057814e4f0 2024-01-24T14:13:26.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.015+0000 7f056f7fe700 1 -- 172.21.15.29:0/110662414 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570012760 con 0x7f057814e4f0 2024-01-24T14:13:26.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.015+0000 7f057dbfb700 1 -- 172.21.15.29:0/110662414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0568009a20 con 0x7f057814e4f0 2024-01-24T14:13:26.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.015+0000 7f057dbfb700 1 -- 172.21.15.29:0/110662414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057813cca0 con 0x7f057814e4f0 2024-01-24T14:13:26.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.016+0000 7f057dbfb700 1 -- 172.21.15.29:0/110662414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057813d460 con 0x7f057814e4f0 2024-01-24T14:13:26.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.016+0000 7f056f7fe700 1 -- 172.21.15.29:0/110662414 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0570012b50 con 0x7f057814e4f0 2024-01-24T14:13:26.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.017+0000 7f056f7fe700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f055805f240 0x7f0558061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:26.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.017+0000 7f056f7fe700 1 -- 172.21.15.29:0/110662414 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6660+0+0 (secure 0 0 0) 0x7f05700798d0 con 0x7f057814e4f0 2024-01-24T14:13:26.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.017+0000 7f05777fe700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f055805f240 0x7f0558061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:26.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.018+0000 7f05777fe700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f055805f240 0x7f0558061700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f0568009eb0 tx=0x7f0568009780).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:26.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.022+0000 7f056f7fe700 1 -- 172.21.15.29:0/110662414 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0570047b00 con 0x7f057814e4f0 2024-01-24T14:13:26.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.203+0000 7f057dbfb700 1 -- 172.21.15.29:0/110662414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0578158ac0 con 0x7f057814e4f0 2024-01-24T14:13:26.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.854+0000 7f056f7fe700 1 -- 172.21.15.29:0/110662414 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) v1 ==== 170+0+0 (secure 0 0 0) 0x7f057004a800 con 0x7f057814e4f0 2024-01-24T14:13:26.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.862+0000 7f057dbfb700 1 -- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f055805f240 msgr2=0x7f0558061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:26.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.862+0000 7f057dbfb700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f055805f240 0x7f0558061700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f0568009eb0 tx=0x7f0568009780).stop 2024-01-24T14:13:26.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.862+0000 7f057dbfb700 1 -- 172.21.15.29:0/110662414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f057814e4f0 msgr2=0x7f0578156df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:26.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.862+0000 7f057dbfb700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f057814e4f0 0x7f0578156df0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f057000c440 tx=0x7f0570007d70).stop 2024-01-24T14:13:26.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.863+0000 7f057dbfb700 1 -- 172.21.15.29:0/110662414 shutdown_connections 2024-01-24T14:13:26.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.863+0000 7f057dbfb700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f055805f240 0x7f0558061700 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:26.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.863+0000 7f057dbfb700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f057814e4f0 0x7f0578156df0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:26.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.863+0000 7f057dbfb700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05780627b0 0x7f057813d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:26.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.863+0000 7f057dbfb700 1 --2- 172.21.15.29:0/110662414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578065a80 0x7f0578166fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:26.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.863+0000 7f057dbfb700 1 -- 172.21.15.29:0/110662414 >> 172.21.15.29:0/110662414 conn(0x7f05780c7330 msgr2=0x7f05780abec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:26.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.863+0000 7f057dbfb700 1 -- 172.21.15.29:0/110662414 shutdown_connections 2024-01-24T14:13:26.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:26.863+0000 7f057dbfb700 1 -- 172.21.15.29:0/110662414 wait complete. 2024-01-24T14:13:26.873 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-01-24T14:13:26.892 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-24T14:13:27.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.028+0000 7f2f3196d700 1 Processor -- start 2024-01-24T14:13:27.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.028+0000 7f2f3196d700 1 -- start start 2024-01-24T14:13:27.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f3196d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2f2c064760 0x7f2f2c064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f3196d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2c152240 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f3196d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2c1586c0 con 0x7f2f2c0c8490 2024-01-24T14:13:27.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f3196d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2c0aa670 con 0x7f2f2c064760 2024-01-24T14:13:27.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f2affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2f2c064760 0x7f2f2c064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f2affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2f2c064760 0x7f2f2c064b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41458/0 (socket says 172.21.15.29:41458) 2024-01-24T14:13:27.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f2affd700 1 -- 172.21.15.29:0/3043188374 learned_addr learned my addr 172.21.15.29:0/3043188374 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:27.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f2b7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2f2c0bfd40 0x7f2f2c0c01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35866/0 (socket says 172.21.15.29:35866) 2024-01-24T14:13:27.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f2affd700 1 -- 172.21.15.29:0/3043188374 >> v1:172.21.15.163:6789/0 conn(0x7f2f2c0c8490 legacy=0x7f2f2c0a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f2affd700 1 -- 172.21.15.29:0/3043188374 >> v1:172.21.15.29:6789/0 conn(0x7f2f2c0bfd40 legacy=0x7f2f2c0c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f2affd700 1 -- 172.21.15.29:0/3043188374 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c157d90 con 0x7f2f2c064760 2024-01-24T14:13:27.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f29ffb700 1 -- 172.21.15.29:0/3043188374 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795993748 0 0) 0x7f2f2c152240 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.029+0000 7f2f2affd700 1 --2- 172.21.15.29:0/3043188374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c064760 0x7f2f2c064b40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f2f1c006fd0 tx=0x7f2f1c0090f0).ready entity=mon.0 client_cookie=97ff752828cdfeaa server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:27.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.030+0000 7f2f29ffb700 1 -- 172.21.15.29:0/3043188374 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00b690 con 0x7f2f2c064760 2024-01-24T14:13:27.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.030+0000 7f2f29ffb700 1 -- 172.21.15.29:0/3043188374 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c00b7f0 con 0x7f2f2c064760 2024-01-24T14:13:27.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.030+0000 7f2f29ffb700 1 -- 172.21.15.29:0/3043188374 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00bb70 con 0x7f2f2c064760 2024-01-24T14:13:27.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.030+0000 7f2f3196d700 1 -- 172.21.15.29:0/3043188374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c064760 msgr2=0x7f2f2c064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.030+0000 7f2f3196d700 1 --2- 172.21.15.29:0/3043188374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c064760 0x7f2f2c064b40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f2f1c006fd0 tx=0x7f2f1c0090f0).stop 2024-01-24T14:13:27.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.030+0000 7f2f3196d700 1 -- 172.21.15.29:0/3043188374 shutdown_connections 2024-01-24T14:13:27.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.030+0000 7f2f3196d700 1 --2- 172.21.15.29:0/3043188374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c064760 0x7f2f2c064b40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.030+0000 7f2f3196d700 1 -- 172.21.15.29:0/3043188374 >> 172.21.15.29:0/3043188374 conn(0x7f2f2c0cb8b0 msgr2=0x7f2f2c0c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:27.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.030+0000 7f2f3196d700 1 -- 172.21.15.29:0/3043188374 shutdown_connections 2024-01-24T14:13:27.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.030+0000 7f2f3196d700 1 -- 172.21.15.29:0/3043188374 wait complete. 2024-01-24T14:13:27.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.032+0000 7f2f3196d700 1 Processor -- start 2024-01-24T14:13:27.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.032+0000 7f2f3196d700 1 -- start start 2024-01-24T14:13:27.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.033+0000 7f2f3196d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c064760 0x7f2f2c164520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.033+0000 7f2f3196d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c0bfd40 0x7f2f2c1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.033+0000 7f2f3196d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2c0c8490 0x7f2f2c1568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.033+0000 7f2f3196d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c156da0 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.033+0000 7f2f3196d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c155040 con 0x7f2f2c0c8490 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.033+0000 7f2f3196d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c155320 con 0x7f2f2c064760 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.033+0000 7f2f2a7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c0bfd40 0x7f2f2c1563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.033+0000 7f2f2a7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c0bfd40 0x7f2f2c1563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41460/0 (socket says 172.21.15.29:41460) 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.033+0000 7f2f2a7fc700 1 -- 172.21.15.29:0/2845928296 learned_addr learned my addr 172.21.15.29:0/2845928296 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f2affd700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c064760 0x7f2f2c164520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f2b7fe700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2c0c8490 0x7f2f2c1568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f2a7fc700 1 -- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c064760 msgr2=0x7f2f2c164520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f2a7fc700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c064760 0x7f2f2c164520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f2a7fc700 1 -- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2c0c8490 msgr2=0x7f2f2c1568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f2a7fc700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2c0c8490 0x7f2f2c1568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f2a7fc700 1 -- 172.21.15.29:0/2845928296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c009370 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f2b7fe700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2c0c8490 0x7f2f2c1568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:27.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f2affd700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c064760 0x7f2f2c164520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:27.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f2a7fc700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c0bfd40 0x7f2f2c1563c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f2f2000c440 tx=0x7f2f20007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:27.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f12ffd700 1 -- 172.21.15.29:0/2845928296 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20007020 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f12ffd700 1 -- 172.21.15.29:0/2845928296 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f20007180 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f12ffd700 1 -- 172.21.15.29:0/2845928296 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20007540 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f3196d700 1 -- 172.21.15.29:0/2845928296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c0adf10 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.034+0000 7f2f3196d700 1 -- 172.21.15.29:0/2845928296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c164a10 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.035+0000 7f2f3196d700 1 -- 172.21.15.29:0/2845928296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c0475e0 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.036+0000 7f2f12ffd700 1 -- 172.21.15.29:0/2845928296 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2f200068b0 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.036+0000 7f2f12ffd700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2f1805f270 0x7f2f18061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.036+0000 7f2f12ffd700 1 -- 172.21.15.29:0/2845928296 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7042+0+0 (secure 0 0 0) 0x7f2f2007a0c0 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.036+0000 7f2f2affd700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2f1805f270 0x7f2f18061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.037+0000 7f2f2affd700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2f1805f270 0x7f2f18061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f2f1c009130 tx=0x7f2f1c017000).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:27.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.041+0000 7f2f12ffd700 1 -- 172.21.15.29:0/2845928296 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2f2004ae70 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.117 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-01-24T14:13:27.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.269+0000 7f2f3196d700 1 -- 172.21.15.29:0/2845928296 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2f2c0657a0 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.275+0000 7f2f12ffd700 1 -- 172.21.15.29:0/2845928296 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v31) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2f2004e490 con 0x7f2f2c0bfd40 2024-01-24T14:13:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.278+0000 7f2f3196d700 1 -- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2f1805f270 msgr2=0x7f2f18061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.278+0000 7f2f3196d700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2f1805f270 0x7f2f18061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f2f1c009130 tx=0x7f2f1c017000).stop 2024-01-24T14:13:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.278+0000 7f2f3196d700 1 -- 172.21.15.29:0/2845928296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c0bfd40 msgr2=0x7f2f2c1563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.278+0000 7f2f3196d700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c0bfd40 0x7f2f2c1563c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f2f2000c440 tx=0x7f2f20007e00).stop 2024-01-24T14:13:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.279+0000 7f2f3196d700 1 -- 172.21.15.29:0/2845928296 shutdown_connections 2024-01-24T14:13:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.279+0000 7f2f3196d700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2c0c8490 0x7f2f2c1568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.279+0000 7f2f3196d700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c064760 0x7f2f2c164520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.279+0000 7f2f3196d700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2f1805f270 0x7f2f18061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.279+0000 7f2f3196d700 1 --2- 172.21.15.29:0/2845928296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c0bfd40 0x7f2f2c1563c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.279+0000 7f2f3196d700 1 -- 172.21.15.29:0/2845928296 >> 172.21.15.29:0/2845928296 conn(0x7f2f2c0cb8b0 msgr2=0x7f2f2c150520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:27.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.280+0000 7f2f3196d700 1 -- 172.21.15.29:0/2845928296 shutdown_connections 2024-01-24T14:13:27.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.280+0000 7f2f3196d700 1 -- 172.21.15.29:0/2845928296 wait complete. 2024-01-24T14:13:27.292 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 10 and data pool 11 2024-01-24T14:13:27.312 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:13:27.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.441+0000 7f8764c32700 1 Processor -- start 2024-01-24T14:13:27.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.441+0000 7f8764c32700 1 -- start start 2024-01-24T14:13:27.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.442+0000 7f8764c32700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8760066c00 0x7f87600bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.442+0000 7f8764c32700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87601582b0 con 0x7f87600670a0 2024-01-24T14:13:27.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.442+0000 7f8764c32700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8760157ca0 con 0x7f87600c4a10 2024-01-24T14:13:27.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.442+0000 7f8764c32700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8760063c60 con 0x7f8760066c00 2024-01-24T14:13:27.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.442+0000 7f875dd9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8760066c00 0x7f87600bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.442+0000 7f875ed9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f87600670a0 0x7f87600bdb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35876/0 (socket says 172.21.15.29:35876) 2024-01-24T14:13:27.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.442+0000 7f875ed9d700 1 -- 172.21.15.29:0/667455176 learned_addr learned my addr 172.21.15.29:0/667455176 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:27.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.442+0000 7f875dd9b700 1 -- 172.21.15.29:0/667455176 >> v1:172.21.15.163:6789/0 conn(0x7f87600c4a10 legacy=0x7f87600664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.442+0000 7f875dd9b700 1 -- 172.21.15.29:0/667455176 >> v1:172.21.15.29:6789/0 conn(0x7f87600670a0 legacy=0x7f87600bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.442+0000 7f875d59a700 1 -- 172.21.15.29:0/667455176 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124705588 0 0) 0x7f87601582b0 con 0x7f87600670a0 2024-01-24T14:13:27.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.442+0000 7f875dd9b700 1 -- 172.21.15.29:0/667455176 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8760157f40 con 0x7f8760066c00 2024-01-24T14:13:27.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.443+0000 7f875dd9b700 1 --2- 172.21.15.29:0/667455176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8760066c00 0x7f87600bf6d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f8754009760 tx=0x7f8754009060).ready entity=mon.0 client_cookie=89ec7fce63c5f636 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:27.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.443+0000 7f875d59a700 1 -- 172.21.15.29:0/667455176 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875400b580 con 0x7f8760066c00 2024-01-24T14:13:27.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.443+0000 7f875d59a700 1 -- 172.21.15.29:0/667455176 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875400b6e0 con 0x7f8760066c00 2024-01-24T14:13:27.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.443+0000 7f875d59a700 1 -- 172.21.15.29:0/667455176 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875400ba60 con 0x7f8760066c00 2024-01-24T14:13:27.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.443+0000 7f8764c32700 1 -- 172.21.15.29:0/667455176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8760066c00 msgr2=0x7f87600bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.443+0000 7f8764c32700 1 --2- 172.21.15.29:0/667455176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8760066c00 0x7f87600bf6d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f8754009760 tx=0x7f8754009060).stop 2024-01-24T14:13:27.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.443+0000 7f8764c32700 1 -- 172.21.15.29:0/667455176 shutdown_connections 2024-01-24T14:13:27.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.443+0000 7f8764c32700 1 --2- 172.21.15.29:0/667455176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8760066c00 0x7f87600bf6d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.443+0000 7f8764c32700 1 -- 172.21.15.29:0/667455176 >> 172.21.15.29:0/667455176 conn(0x7f87600c8bb0 msgr2=0x7f87600c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:27.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.443+0000 7f8764c32700 1 -- 172.21.15.29:0/667455176 shutdown_connections 2024-01-24T14:13:27.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.443+0000 7f8764c32700 1 -- 172.21.15.29:0/667455176 wait complete. 2024-01-24T14:13:27.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.445+0000 7f8764c32700 1 Processor -- start 2024-01-24T14:13:27.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.445+0000 7f8764c32700 1 -- start start 2024-01-24T14:13:27.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f8764c32700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8760066c00 0x7f87600a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f8764c32700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87600670a0 0x7f8760153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f8764c32700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87600c4a10 0x7f87600a7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f8764c32700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8760154f90 con 0x7f87600c4a10 2024-01-24T14:13:27.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f8764c32700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8760165200 con 0x7f87600670a0 2024-01-24T14:13:27.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f8764c32700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8760154660 con 0x7f8760066c00 2024-01-24T14:13:27.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87600c4a10 0x7f87600a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87600c4a10 0x7f87600a7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41484/0 (socket says 172.21.15.29:41484) 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875ed9d700 1 -- 172.21.15.29:0/3198362126 learned_addr learned my addr 172.21.15.29:0/3198362126 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875e59c700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8760066c00 0x7f87600a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875ed9d700 1 -- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8760066c00 msgr2=0x7f87600a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875dd9b700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87600670a0 0x7f8760153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875ed9d700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8760066c00 0x7f87600a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875ed9d700 1 -- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87600670a0 msgr2=0x7f8760153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875ed9d700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87600670a0 0x7f8760153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875ed9d700 1 -- 172.21.15.29:0/3198362126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540092a0 con 0x7f87600c4a10 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875e59c700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8760066c00 0x7f87600a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875dd9b700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87600670a0 0x7f8760153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f875ed9d700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87600c4a10 0x7f87600a7040 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f875800c7e0 tx=0x7f8758007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f874e7fc700 1 -- 172.21.15.29:0/3198362126 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758005d30 con 0x7f87600c4a10 2024-01-24T14:13:27.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f874e7fc700 1 -- 172.21.15.29:0/3198362126 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8758005e90 con 0x7f87600c4a10 2024-01-24T14:13:27.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f8764c32700 1 -- 172.21.15.29:0/3198362126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8760154940 con 0x7f87600c4a10 2024-01-24T14:13:27.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f874e7fc700 1 -- 172.21.15.29:0/3198362126 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758007320 con 0x7f87600c4a10 2024-01-24T14:13:27.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.446+0000 7f8764c32700 1 -- 172.21.15.29:0/3198362126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87600a4d70 con 0x7f87600c4a10 2024-01-24T14:13:27.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.448+0000 7f8764c32700 1 -- 172.21.15.29:0/3198362126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87600475e0 con 0x7f87600c4a10 2024-01-24T14:13:27.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.448+0000 7f874e7fc700 1 -- 172.21.15.29:0/3198362126 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8758007710 con 0x7f87600c4a10 2024-01-24T14:13:27.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.448+0000 7f874e7fc700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f874005f270 0x7f8740061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.448+0000 7f874e7fc700 1 -- 172.21.15.29:0/3198362126 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f875807a800 con 0x7f87600c4a10 2024-01-24T14:13:27.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.448+0000 7f875e59c700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f874005f270 0x7f8740061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.449+0000 7f875e59c700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f874005f270 0x7f8740061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f87600c3d50 tx=0x7f8750009040).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:27.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.453+0000 7f874e7fc700 1 -- 172.21.15.29:0/3198362126 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f875804b550 con 0x7f87600c4a10 2024-01-24T14:13:27.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.633+0000 7f8764c32700 1 -- 172.21.15.29:0/3198362126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8760154440 con 0x7f87600c4a10 2024-01-24T14:13:27.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.634+0000 7f874e7fc700 1 -- 172.21.15.29:0/3198362126 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+18186 (secure 0 0 0) 0x7f875804eb70 con 0x7f87600c4a10 2024-01-24T14:13:27.635 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:27.636 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":89,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:13:27.270541+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:13:25.288458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-24T14:13:26.243223+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:13:27.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.637+0000 7f8764c32700 1 -- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f874005f270 msgr2=0x7f8740061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.637+0000 7f8764c32700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f874005f270 0x7f8740061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f87600c3d50 tx=0x7f8750009040).stop 2024-01-24T14:13:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.637+0000 7f8764c32700 1 -- 172.21.15.29:0/3198362126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87600c4a10 msgr2=0x7f87600a7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.637+0000 7f8764c32700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87600c4a10 0x7f87600a7040 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f875800c7e0 tx=0x7f8758007f90).stop 2024-01-24T14:13:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.637+0000 7f8764c32700 1 -- 172.21.15.29:0/3198362126 shutdown_connections 2024-01-24T14:13:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.637+0000 7f8764c32700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f874005f270 0x7f8740061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.637+0000 7f8764c32700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87600c4a10 0x7f87600a7040 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.637+0000 7f8764c32700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8760066c00 0x7f87600a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.637+0000 7f8764c32700 1 --2- 172.21.15.29:0/3198362126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87600670a0 0x7f8760153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.637+0000 7f8764c32700 1 -- 172.21.15.29:0/3198362126 >> 172.21.15.29:0/3198362126 conn(0x7f87600c8bb0 msgr2=0x7f87600ac010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.637+0000 7f8764c32700 1 -- 172.21.15.29:0/3198362126 shutdown_connections 2024-01-24T14:13:27.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.637+0000 7f8764c32700 1 -- 172.21.15.29:0/3198362126 wait complete. 2024-01-24T14:13:27.651 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:13:27.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.780+0000 7fcdc256d700 1 Processor -- start 2024-01-24T14:13:27.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.780+0000 7fcdc256d700 1 -- start start 2024-01-24T14:13:27.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.781+0000 7fcdc256d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcdbc155b00 0x7fcdbc0664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.781+0000 7fcdc256d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcdbc0669e0 0x7fcdbc155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.781+0000 7fcdc256d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdbc066e90 con 0x7fcdbc1563d0 2024-01-24T14:13:27.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.781+0000 7fcdc256d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdbc0c5500 con 0x7fcdbc155b00 2024-01-24T14:13:27.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.781+0000 7fcdc256d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdbc0c3910 con 0x7fcdbc0669e0 2024-01-24T14:13:27.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.781+0000 7fcdbbfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcdbc155b00 0x7fcdbc0664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.782+0000 7fcdc0b0a700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcdbc1563d0 0x7fcdbc067a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35888/0 (socket says 172.21.15.29:35888) 2024-01-24T14:13:27.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.782+0000 7fcdc0b0a700 1 -- 172.21.15.29:0/1910931982 learned_addr learned my addr 172.21.15.29:0/1910931982 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:27.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.782+0000 7fcdbb7fe700 1 --2- 172.21.15.29:0/1910931982 >> v2:172.21.15.163:3300/0 conn(0x7fcdbc0669e0 0x7fcdbc155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.782+0000 7fcdbbfff700 1 -- 172.21.15.29:0/1910931982 >> v2:172.21.15.163:3300/0 conn(0x7fcdbc0669e0 msgr2=0x7fcdbc155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:27.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.782+0000 7fcdbbfff700 1 --2- 172.21.15.29:0/1910931982 >> v2:172.21.15.163:3300/0 conn(0x7fcdbc0669e0 0x7fcdbc155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.782+0000 7fcdbbfff700 1 -- 172.21.15.29:0/1910931982 >> v1:172.21.15.29:6789/0 conn(0x7fcdbc1563d0 legacy=0x7fcdbc067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.782+0000 7fcdbaffd700 1 -- 172.21.15.29:0/1910931982 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692125274 0 0) 0x7fcdb4001490 con 0x7fcdbc1563d0 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.782+0000 7fcdbbfff700 1 -- 172.21.15.29:0/1910931982 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdbc0c5220 con 0x7fcdbc155b00 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.782+0000 7fcdbbfff700 1 --2- 172.21.15.29:0/1910931982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc155b00 0x7fcdbc0664f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fcdbc0c6960 tx=0x7fcdac009060).ready entity=mon.0 client_cookie=869cec71365e4ce server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.782+0000 7fcdbaffd700 1 -- 172.21.15.29:0/1910931982 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00b580 con 0x7fcdbc155b00 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.782+0000 7fcdbaffd700 1 -- 172.21.15.29:0/1910931982 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdac00b6e0 con 0x7fcdbc155b00 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.782+0000 7fcdbaffd700 1 -- 172.21.15.29:0/1910931982 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00ba60 con 0x7fcdbc155b00 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.783+0000 7fcdc256d700 1 -- 172.21.15.29:0/1910931982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc155b00 msgr2=0x7fcdbc0664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.783+0000 7fcdc256d700 1 --2- 172.21.15.29:0/1910931982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc155b00 0x7fcdbc0664f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fcdbc0c6960 tx=0x7fcdac009060).stop 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.783+0000 7fcdc256d700 1 -- 172.21.15.29:0/1910931982 shutdown_connections 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.783+0000 7fcdc256d700 1 --2- 172.21.15.29:0/1910931982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc155b00 0x7fcdbc0664f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.783+0000 7fcdc256d700 1 --2- 172.21.15.29:0/1910931982 >> v2:172.21.15.163:3300/0 conn(0x7fcdbc0669e0 0x7fcdbc155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.783+0000 7fcdc256d700 1 -- 172.21.15.29:0/1910931982 >> 172.21.15.29:0/1910931982 conn(0x7fcdbc0c8bb0 msgr2=0x7fcdbc0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.783+0000 7fcdc256d700 1 -- 172.21.15.29:0/1910931982 shutdown_connections 2024-01-24T14:13:27.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.783+0000 7fcdc256d700 1 -- 172.21.15.29:0/1910931982 wait complete. 2024-01-24T14:13:27.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.784+0000 7fcdc256d700 1 Processor -- start 2024-01-24T14:13:27.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.784+0000 7fcdc256d700 1 -- start start 2024-01-24T14:13:27.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.785+0000 7fcdc256d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdbc0669e0 0x7fcdbc0c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.785+0000 7fcdc256d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdbc155b00 0x7fcdbc0c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.785+0000 7fcdc256d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc1563d0 0x7fcdbc162130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.785+0000 7fcdc256d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc0aa280 con 0x7fcdbc1563d0 2024-01-24T14:13:27.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.785+0000 7fcdc256d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc0aab00 con 0x7fcdbc155b00 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.785+0000 7fcdc256d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc0c2f50 con 0x7fcdbc0669e0 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.785+0000 7fcdc0b0a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc1563d0 0x7fcdbc162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.785+0000 7fcdc0b0a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc1563d0 0x7fcdbc162130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41504/0 (socket says 172.21.15.29:41504) 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.785+0000 7fcdc0b0a700 1 -- 172.21.15.29:0/325248295 learned_addr learned my addr 172.21.15.29:0/325248295 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.785+0000 7fcdbbfff700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdbc0669e0 0x7fcdbc0c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.785+0000 7fcdbb7fe700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdbc155b00 0x7fcdbc0c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.786+0000 7fcdc0b0a700 1 -- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdbc0669e0 msgr2=0x7fcdbc0c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.786+0000 7fcdc0b0a700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdbc0669e0 0x7fcdbc0c02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.786+0000 7fcdc0b0a700 1 -- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdbc155b00 msgr2=0x7fcdbc0c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.786+0000 7fcdc0b0a700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdbc155b00 0x7fcdbc0c1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.786+0000 7fcdc0b0a700 1 -- 172.21.15.29:0/325248295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac0092a0 con 0x7fcdbc1563d0 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.786+0000 7fcdc0b0a700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc1563d0 0x7fcdbc162130 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fcdb400c730 tx=0x7fcdb4001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.786+0000 7fcda3fff700 1 -- 172.21.15.29:0/325248295 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4005d30 con 0x7fcdbc1563d0 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.786+0000 7fcda3fff700 1 -- 172.21.15.29:0/325248295 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb4005e90 con 0x7fcdbc1563d0 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.786+0000 7fcdc256d700 1 -- 172.21.15.29:0/325248295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdbc0a0850 con 0x7fcdbc1563d0 2024-01-24T14:13:27.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.786+0000 7fcda3fff700 1 -- 172.21.15.29:0/325248295 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4007300 con 0x7fcdbc1563d0 2024-01-24T14:13:27.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.786+0000 7fcdc256d700 1 -- 172.21.15.29:0/325248295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc0aa790 con 0x7fcdbc1563d0 2024-01-24T14:13:27.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.787+0000 7fcdc256d700 1 -- 172.21.15.29:0/325248295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdbc0aacb0 con 0x7fcdbc1563d0 2024-01-24T14:13:27.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.787+0000 7fcda3fff700 1 -- 172.21.15.29:0/325248295 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcdb40076b0 con 0x7fcdbc1563d0 2024-01-24T14:13:27.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.788+0000 7fcda3fff700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcda805f270 0x7fcda8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:27.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.788+0000 7fcda3fff700 1 -- 172.21.15.29:0/325248295 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fcdb4079f40 con 0x7fcdbc1563d0 2024-01-24T14:13:27.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.788+0000 7fcdbbfff700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcda805f270 0x7fcda8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:27.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.789+0000 7fcdbbfff700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcda805f270 0x7fcda8061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fcdac009fd0 tx=0x7fcdac017000).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:27.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.792+0000 7fcda3fff700 1 -- 172.21.15.29:0/325248295 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcdb404ac90 con 0x7fcdbc1563d0 2024-01-24T14:13:27.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.978+0000 7fcdc256d700 1 -- 172.21.15.29:0/325248295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdbc066e90 con 0x7fcdbc1563d0 2024-01-24T14:13:27.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.979+0000 7fcda3fff700 1 -- 172.21.15.29:0/325248295 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+18186 (secure 0 0 0) 0x7fcdbc066e90 con 0x7fcdbc1563d0 2024-01-24T14:13:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:27.980 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":89,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:13:27.270541+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:13:25.288458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-24T14:13:26.243223+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:13:27.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.981+0000 7fcdc256d700 1 -- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcda805f270 msgr2=0x7fcda8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.981+0000 7fcdc256d700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcda805f270 0x7fcda8061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fcdac009fd0 tx=0x7fcdac017000).stop 2024-01-24T14:13:27.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.981+0000 7fcdc256d700 1 -- 172.21.15.29:0/325248295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc1563d0 msgr2=0x7fcdbc162130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:27.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.981+0000 7fcdc256d700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc1563d0 0x7fcdbc162130 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fcdb400c730 tx=0x7fcdb4001520).stop 2024-01-24T14:13:27.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.981+0000 7fcdc256d700 1 -- 172.21.15.29:0/325248295 shutdown_connections 2024-01-24T14:13:27.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.981+0000 7fcdc256d700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcda805f270 0x7fcda8061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.981+0000 7fcdc256d700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdbc1563d0 0x7fcdbc162130 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.981+0000 7fcdc256d700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdbc0669e0 0x7fcdbc0c02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.981+0000 7fcdc256d700 1 --2- 172.21.15.29:0/325248295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdbc155b00 0x7fcdbc0c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:27.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.981+0000 7fcdc256d700 1 -- 172.21.15.29:0/325248295 >> 172.21.15.29:0/325248295 conn(0x7fcdbc0c8bb0 msgr2=0x7fcdbc14a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:27.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.982+0000 7fcdc256d700 1 -- 172.21.15.29:0/325248295 shutdown_connections 2024-01-24T14:13:27.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:27.982+0000 7fcdc256d700 1 -- 172.21.15.29:0/325248295 wait complete. 2024-01-24T14:13:27.996 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-24T14:13:28.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.129+0000 7f1aa6b10700 1 Processor -- start 2024-01-24T14:13:28.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.129+0000 7f1aa6b10700 1 -- start start 2024-01-24T14:13:28.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa6b10700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1aa0153f40 0x7f1aa00a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:28.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa6b10700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1aa0062ec0 0x7f1aa0065440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:28.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa6b10700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa006a360 con 0x7f1aa00c8290 2024-01-24T14:13:28.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa6b10700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa00bcef0 con 0x7f1aa0153f40 2024-01-24T14:13:28.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa6b10700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa0069de0 con 0x7f1aa0062ec0 2024-01-24T14:13:28.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa48ac700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1aa0153f40 0x7f1aa00a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:28.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1a9ffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1aa00c8290 0x7f1aa00627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35904/0 (socket says 172.21.15.29:35904) 2024-01-24T14:13:28.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa48ac700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1aa0153f40 0x7f1aa00a6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41512/0 (socket says 172.21.15.29:41512) 2024-01-24T14:13:28.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1a9ffff700 1 -- 172.21.15.29:0/840299757 learned_addr learned my addr 172.21.15.29:0/840299757 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:28.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa48ac700 1 -- 172.21.15.29:0/840299757 >> v2:172.21.15.163:3300/0 conn(0x7f1aa0062ec0 msgr2=0x7f1aa0065440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:28.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa50ad700 1 --2- 172.21.15.29:0/840299757 >> v2:172.21.15.163:3300/0 conn(0x7f1aa0062ec0 0x7f1aa0065440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:28.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa48ac700 1 --2- 172.21.15.29:0/840299757 >> v2:172.21.15.163:3300/0 conn(0x7f1aa0062ec0 0x7f1aa0065440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:28.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa48ac700 1 -- 172.21.15.29:0/840299757 >> v1:172.21.15.29:6789/0 conn(0x7f1aa00c8290 legacy=0x7f1aa00627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:28.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1a9f7fe700 1 -- 172.21.15.29:0/840299757 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496041905 0 0) 0x7f1aa006a360 con 0x7f1aa00c8290 2024-01-24T14:13:28.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa48ac700 1 -- 172.21.15.29:0/840299757 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa006a080 con 0x7f1aa0153f40 2024-01-24T14:13:28.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.130+0000 7f1aa50ad700 1 --2- 172.21.15.29:0/840299757 >> v2:172.21.15.163:3300/0 conn(0x7f1aa0062ec0 0x7f1aa0065440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:28.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.131+0000 7f1aa48ac700 1 --2- 172.21.15.29:0/840299757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa0153f40 0x7f1aa00a6db0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f1aa00bce60 tx=0x7f1a94009060).ready entity=mon.0 client_cookie=a1cb892f6c0b67bc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:28.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.131+0000 7f1a9f7fe700 1 -- 172.21.15.29:0/840299757 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a940090d0 con 0x7f1aa0153f40 2024-01-24T14:13:28.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.131+0000 7f1a9f7fe700 1 -- 172.21.15.29:0/840299757 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9400b580 con 0x7f1aa0153f40 2024-01-24T14:13:28.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.131+0000 7f1a9f7fe700 1 -- 172.21.15.29:0/840299757 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400b900 con 0x7f1aa0153f40 2024-01-24T14:13:28.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.131+0000 7f1aa6b10700 1 -- 172.21.15.29:0/840299757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa0153f40 msgr2=0x7f1aa00a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:28.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.131+0000 7f1aa6b10700 1 --2- 172.21.15.29:0/840299757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa0153f40 0x7f1aa00a6db0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f1aa00bce60 tx=0x7f1a94009060).stop 2024-01-24T14:13:28.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.131+0000 7f1aa6b10700 1 -- 172.21.15.29:0/840299757 shutdown_connections 2024-01-24T14:13:28.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.131+0000 7f1aa6b10700 1 --2- 172.21.15.29:0/840299757 >> v2:172.21.15.163:3300/0 conn(0x7f1aa0062ec0 0x7f1aa0065440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:28.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.131+0000 7f1aa6b10700 1 --2- 172.21.15.29:0/840299757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa0153f40 0x7f1aa00a6db0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:28.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.131+0000 7f1aa6b10700 1 -- 172.21.15.29:0/840299757 >> 172.21.15.29:0/840299757 conn(0x7f1aa0064390 msgr2=0x7f1aa00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:28.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.131+0000 7f1aa6b10700 1 -- 172.21.15.29:0/840299757 shutdown_connections 2024-01-24T14:13:28.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.131+0000 7f1aa6b10700 1 -- 172.21.15.29:0/840299757 wait complete. 2024-01-24T14:13:28.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.132+0000 7f1aa6b10700 1 Processor -- start 2024-01-24T14:13:28.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.133+0000 7f1aa6b10700 1 -- start start 2024-01-24T14:13:28.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.133+0000 7f1aa6b10700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1aa0062ec0 0x7f1aa00a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:28.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.133+0000 7f1aa6b10700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa00c8290 0x7f1aa014b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:28.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1aa6b10700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1aa0153f40 0x7f1aa00a7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:28.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1aa6b10700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00ab190 con 0x7f1aa00c8290 2024-01-24T14:13:28.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1aa6b10700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00ab470 con 0x7f1aa0062ec0 2024-01-24T14:13:28.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1aa6b10700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00ab750 con 0x7f1aa0153f40 2024-01-24T14:13:28.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1a9ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa00c8290 0x7f1aa014b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:28.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1a9ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa00c8290 0x7f1aa014b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41520/0 (socket says 172.21.15.29:41520) 2024-01-24T14:13:28.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1a9ffff700 1 -- 172.21.15.29:0/2350981768 learned_addr learned my addr 172.21.15.29:0/2350981768 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:28.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1aa50ad700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1aa0153f40 0x7f1aa00a7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1aa48ac700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1aa0062ec0 0x7f1aa00a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1a9ffff700 1 -- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1aa0153f40 msgr2=0x7f1aa00a7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1a9ffff700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1aa0153f40 0x7f1aa00a7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1a9ffff700 1 -- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1aa0062ec0 msgr2=0x7f1aa00a8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1a9ffff700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1aa0062ec0 0x7f1aa00a8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1a9ffff700 1 -- 172.21.15.29:0/2350981768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a940092a0 con 0x7f1aa00c8290 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1aa50ad700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1aa0153f40 0x7f1aa00a7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1aa48ac700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1aa0062ec0 0x7f1aa00a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1a9ffff700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa00c8290 0x7f1aa014b730 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f1a8c00c4a0 tx=0x7f1a8c007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1a9cff9700 1 -- 172.21.15.29:0/2350981768 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0072e0 con 0x7f1aa00c8290 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1a9cff9700 1 -- 172.21.15.29:0/2350981768 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c007440 con 0x7f1aa00c8290 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1a9cff9700 1 -- 172.21.15.29:0/2350981768 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0057a0 con 0x7f1aa00c8290 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1aa6b10700 1 -- 172.21.15.29:0/2350981768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa00aba30 con 0x7f1aa00c8290 2024-01-24T14:13:28.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.134+0000 7f1aa6b10700 1 -- 172.21.15.29:0/2350981768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa00a9130 con 0x7f1aa00c8290 2024-01-24T14:13:28.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.135+0000 7f1aa6b10700 1 -- 172.21.15.29:0/2350981768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa00475e0 con 0x7f1aa00c8290 2024-01-24T14:13:28.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.136+0000 7f1a9cff9700 1 -- 172.21.15.29:0/2350981768 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1a8c005bd0 con 0x7f1aa00c8290 2024-01-24T14:13:28.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.136+0000 7f1a9cff9700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a8405f240 0x7f1a84061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:28.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.136+0000 7f1a9cff9700 1 -- 172.21.15.29:0/2350981768 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f1a8c079fc0 con 0x7f1aa00c8290 2024-01-24T14:13:28.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.136+0000 7f1aa48ac700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a8405f240 0x7f1a84061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:28.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.137+0000 7f1aa48ac700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a8405f240 0x7f1a84061700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1a9400b340 tx=0x7f1a94017000).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:28.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.141+0000 7f1a9cff9700 1 -- 172.21.15.29:0/2350981768 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1a8c04ad10 con 0x7f1aa00c8290 2024-01-24T14:13:28.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:28.375+0000 7f1aa6b10700 1 -- 172.21.15.29:0/2350981768 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1aa006a540 con 0x7f1aa00c8290 2024-01-24T14:13:28.735 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:13:28.738 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:13:29.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.279+0000 7f1a9cff9700 1 -- 172.21.15.29:0/2350981768 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1a8c04e330 con 0x7f1aa00c8290 2024-01-24T14:13:29.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.282+0000 7f1aa6b10700 1 -- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a8405f240 msgr2=0x7f1a84061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:29.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.282+0000 7f1aa6b10700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a8405f240 0x7f1a84061700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1a9400b340 tx=0x7f1a94017000).stop 2024-01-24T14:13:29.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.282+0000 7f1aa6b10700 1 -- 172.21.15.29:0/2350981768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa00c8290 msgr2=0x7f1aa014b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:29.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.282+0000 7f1aa6b10700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa00c8290 0x7f1aa014b730 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f1a8c00c4a0 tx=0x7f1a8c007c30).stop 2024-01-24T14:13:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.282+0000 7f1aa6b10700 1 -- 172.21.15.29:0/2350981768 shutdown_connections 2024-01-24T14:13:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.282+0000 7f1aa6b10700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1aa0153f40 0x7f1aa00a7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.282+0000 7f1aa6b10700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1aa0062ec0 0x7f1aa00a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.282+0000 7f1aa6b10700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a8405f240 0x7f1a84061700 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.282+0000 7f1aa6b10700 1 --2- 172.21.15.29:0/2350981768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa00c8290 0x7f1aa014b730 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.282+0000 7f1aa6b10700 1 -- 172.21.15.29:0/2350981768 >> 172.21.15.29:0/2350981768 conn(0x7f1aa0064390 msgr2=0x7f1aa0065350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.283+0000 7f1aa6b10700 1 -- 172.21.15.29:0/2350981768 shutdown_connections 2024-01-24T14:13:29.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.283+0000 7f1aa6b10700 1 -- 172.21.15.29:0/2350981768 wait complete. 2024-01-24T14:13:29.298 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-24T14:13:29.299 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-24T14:13:29.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.435+0000 7f7cb8925700 1 Processor -- start 2024-01-24T14:13:29.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.435+0000 7f7cb8925700 1 -- start start 2024-01-24T14:13:29.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.436+0000 7f7cb8925700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb40692a0 0x7f7cb4065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:29.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.436+0000 7f7cb8925700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7cb4069d90 0x7f7cb415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:29.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.436+0000 7f7cb8925700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40c3e70 con 0x7f7cb4066370 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.436+0000 7f7cb8925700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40c4300 con 0x7f7cb4069d90 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.436+0000 7f7cb8925700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40c3260 con 0x7f7cb40692a0 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.436+0000 7f7cb2d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7cb4069d90 0x7f7cb415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.437+0000 7f7cb2d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7cb4069d90 0x7f7cb415a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41524/0 (socket says 172.21.15.29:41524) 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.437+0000 7f7cb2d9d700 1 -- 172.21.15.29:0/356140580 learned_addr learned my addr 172.21.15.29:0/356140580 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.437+0000 7f7cb259c700 1 --2- 172.21.15.29:0/356140580 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb40692a0 0x7f7cb4065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.437+0000 7f7cb2d9d700 1 -- 172.21.15.29:0/356140580 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb40692a0 msgr2=0x7f7cb4065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.437+0000 7f7cb2d9d700 1 --2- 172.21.15.29:0/356140580 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb40692a0 0x7f7cb4065e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.437+0000 7f7cb2d9d700 1 -- 172.21.15.29:0/356140580 >> v1:172.21.15.163:6789/0 conn(0x7f7cb4066370 legacy=0x7f7cb4069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.437+0000 7f7cb2d9d700 1 -- 172.21.15.29:0/356140580 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40c3540 con 0x7f7cb4069d90 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.437+0000 7f7cb2d9d700 1 --2- 172.21.15.29:0/356140580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb4069d90 0x7f7cb415a9f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f7cac009f20 tx=0x7f7cac009840).ready entity=mon.0 client_cookie=e28f65c6de8bb968 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.437+0000 7f7cb159a700 1 -- 172.21.15.29:0/356140580 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00e070 con 0x7f7cb4069d90 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.437+0000 7f7cb159a700 1 -- 172.21.15.29:0/356140580 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac00b9d0 con 0x7f7cb4069d90 2024-01-24T14:13:29.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.437+0000 7f7cb159a700 1 -- 172.21.15.29:0/356140580 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00bd50 con 0x7f7cb4069d90 2024-01-24T14:13:29.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.438+0000 7f7cb8925700 1 -- 172.21.15.29:0/356140580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb4069d90 msgr2=0x7f7cb415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:29.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.438+0000 7f7cb8925700 1 --2- 172.21.15.29:0/356140580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb4069d90 0x7f7cb415a9f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f7cac009f20 tx=0x7f7cac009840).stop 2024-01-24T14:13:29.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.438+0000 7f7cb8925700 1 -- 172.21.15.29:0/356140580 shutdown_connections 2024-01-24T14:13:29.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.438+0000 7f7cb8925700 1 --2- 172.21.15.29:0/356140580 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb40692a0 0x7f7cb4065e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:29.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.438+0000 7f7cb8925700 1 --2- 172.21.15.29:0/356140580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb4069d90 0x7f7cb415a9f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:29.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.438+0000 7f7cb8925700 1 -- 172.21.15.29:0/356140580 >> 172.21.15.29:0/356140580 conn(0x7f7cb4064390 msgr2=0x7f7cb40c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:29.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.438+0000 7f7cb8925700 1 -- 172.21.15.29:0/356140580 shutdown_connections 2024-01-24T14:13:29.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.438+0000 7f7cb8925700 1 -- 172.21.15.29:0/356140580 wait complete. 2024-01-24T14:13:29.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.439+0000 7f7cb8925700 1 Processor -- start 2024-01-24T14:13:29.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.439+0000 7f7cb8925700 1 -- start start 2024-01-24T14:13:29.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb8925700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb4066370 0x7f7cb414bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:29.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb8925700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb40692a0 0x7f7cb40c07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:29.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb8925700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb4069d90 0x7f7cb4152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:29.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb8925700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40c0350 con 0x7f7cb4066370 2024-01-24T14:13:29.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb8925700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb414c480 con 0x7f7cb40692a0 2024-01-24T14:13:29.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb8925700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40c0cd0 con 0x7f7cb4069d90 2024-01-24T14:13:29.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb4066370 0x7f7cb414bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb4066370 0x7f7cb414bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41534/0 (socket says 172.21.15.29:41534) 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb259c700 1 -- 172.21.15.29:0/877008174 learned_addr learned my addr 172.21.15.29:0/877008174 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb259c700 1 -- 172.21.15.29:0/877008174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb4069d90 msgr2=0x7f7cb4152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb259c700 1 --2- 172.21.15.29:0/877008174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb4069d90 0x7f7cb4152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb259c700 1 -- 172.21.15.29:0/877008174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb40692a0 msgr2=0x7f7cb40c07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb259c700 1 --2- 172.21.15.29:0/877008174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb40692a0 0x7f7cb40c07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.440+0000 7f7cb259c700 1 -- 172.21.15.29:0/877008174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac009ae0 con 0x7f7cb4066370 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.441+0000 7f7cb259c700 1 --2- 172.21.15.29:0/877008174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb4066370 0x7f7cb414bf90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f7ca400e900 tx=0x7f7ca4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.441+0000 7f7caa7fc700 1 -- 172.21.15.29:0/877008174 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400c640 con 0x7f7cb4066370 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.441+0000 7f7caa7fc700 1 -- 172.21.15.29:0/877008174 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca400c7a0 con 0x7f7cb4066370 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.441+0000 7f7cb8925700 1 -- 172.21.15.29:0/877008174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb40bf740 con 0x7f7cb4066370 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.441+0000 7f7caa7fc700 1 -- 172.21.15.29:0/877008174 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400cb20 con 0x7f7cb4066370 2024-01-24T14:13:29.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.441+0000 7f7cb8925700 1 -- 172.21.15.29:0/877008174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb40bfd00 con 0x7f7cb4066370 2024-01-24T14:13:29.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.442+0000 7f7cb8925700 1 -- 172.21.15.29:0/877008174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb41542f0 con 0x7f7cb4066370 2024-01-24T14:13:29.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.442+0000 7f7caa7fc700 1 -- 172.21.15.29:0/877008174 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7ca401b400 con 0x7f7cb4066370 2024-01-24T14:13:29.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.443+0000 7f7caa7fc700 1 --2- 172.21.15.29:0/877008174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c9405f240 0x7f7c94061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:29.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.443+0000 7f7caa7fc700 1 -- 172.21.15.29:0/877008174 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f7ca407ac20 con 0x7f7cb4066370 2024-01-24T14:13:29.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.443+0000 7f7cb1d9b700 1 --2- 172.21.15.29:0/877008174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c9405f240 0x7f7c94061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:29.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.443+0000 7f7cb1d9b700 1 --2- 172.21.15.29:0/877008174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c9405f240 0x7f7c94061700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f7cb405ed50 tx=0x7f7c9c0079c0).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:29.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.447+0000 7f7caa7fc700 1 -- 172.21.15.29:0/877008174 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7ca404b910 con 0x7f7cb4066370 2024-01-24T14:13:29.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:29.673+0000 7f7cb8925700 1 -- 172.21.15.29:0/877008174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7cb40475e0 con 0x7f7cb4066370 2024-01-24T14:13:30.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.281+0000 7f7caa7fc700 1 -- 172.21.15.29:0/877008174 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7ca4016070 con 0x7f7cb4066370 2024-01-24T14:13:30.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.283+0000 7f7cb8925700 1 -- 172.21.15.29:0/877008174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c9405f240 msgr2=0x7f7c94061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.283+0000 7f7cb8925700 1 --2- 172.21.15.29:0/877008174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c9405f240 0x7f7c94061700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f7cb405ed50 tx=0x7f7c9c0079c0).stop 2024-01-24T14:13:30.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.283+0000 7f7cb8925700 1 -- 172.21.15.29:0/877008174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb4066370 msgr2=0x7f7cb414bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.283+0000 7f7cb8925700 1 --2- 172.21.15.29:0/877008174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb4066370 0x7f7cb414bf90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f7ca400e900 tx=0x7f7ca4009060).stop 2024-01-24T14:13:30.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.284+0000 7f7cb8925700 1 -- 172.21.15.29:0/877008174 shutdown_connections 2024-01-24T14:13:30.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.284+0000 7f7cb8925700 1 --2- 172.21.15.29:0/877008174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb4069d90 0x7f7cb4152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:30.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.284+0000 7f7cb8925700 1 --2- 172.21.15.29:0/877008174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c9405f240 0x7f7c94061700 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.284+0000 7f7cb8925700 1 --2- 172.21.15.29:0/877008174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb4066370 0x7f7cb414bf90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.284+0000 7f7cb8925700 1 --2- 172.21.15.29:0/877008174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb40692a0 0x7f7cb40c07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:30.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.284+0000 7f7cb8925700 1 -- 172.21.15.29:0/877008174 >> 172.21.15.29:0/877008174 conn(0x7f7cb4064390 msgr2=0x7f7cb40627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:30.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.284+0000 7f7cb8925700 1 -- 172.21.15.29:0/877008174 shutdown_connections 2024-01-24T14:13:30.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.284+0000 7f7cb8925700 1 -- 172.21.15.29:0/877008174 wait complete. 2024-01-24T14:13:30.299 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-24T14:13:30.300 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:13:30.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.433+0000 7f337b907700 1 Processor -- start 2024-01-24T14:13:30.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.433+0000 7f337b907700 1 -- start start 2024-01-24T14:13:30.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.434+0000 7f337b907700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3374150d40 0x7f3374069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.434+0000 7f337b907700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3374155040 0x7f33740c3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.434+0000 7f337b907700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33740c2ce0 con 0x7f3374065ec0 2024-01-24T14:13:30.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.434+0000 7f337b907700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3374069890 con 0x7f3374155040 2024-01-24T14:13:30.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.434+0000 7f337b907700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3374069b70 con 0x7f3374150d40 2024-01-24T14:13:30.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.434+0000 7f3378ea2700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3374065ec0 0x7f3374154930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35916/0 (socket says 172.21.15.29:35916) 2024-01-24T14:13:30.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.434+0000 7f3378ea2700 1 -- 172.21.15.29:0/3368687412 learned_addr learned my addr 172.21.15.29:0/3368687412 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:30.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.434+0000 7f33796a3700 1 --2- 172.21.15.29:0/3368687412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3374150d40 0x7f3374069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.434+0000 7f3379ea4700 1 --2- 172.21.15.29:0/3368687412 >> v2:172.21.15.163:3300/0 conn(0x7f3374155040 0x7f33740c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.434+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909948321 0 0) 0x7f33740c2ce0 con 0x7f3374065ec0 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.434+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33600033f0 con 0x7f3374065ec0 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1481281091 0 0) 0x7f33600033f0 con 0x7f3374065ec0 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3374150d40 msgr2=0x7f3374069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 --2- 172.21.15.29:0/3368687412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3374150d40 0x7f3374069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 >> v2:172.21.15.163:3300/0 conn(0x7f3374155040 msgr2=0x7f33740c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 --2- 172.21.15.29:0/3368687412 >> v2:172.21.15.163:3300/0 conn(0x7f3374155040 0x7f33740c3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3374069e50 con 0x7f3374065ec0 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f3364001470 con 0x7f3374065ec0 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 >> v1:172.21.15.29:6789/0 conn(0x7f3374065ec0 legacy=0x7f3374154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 --2- 172.21.15.29:0/3368687412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3374150d40 0x7f33600038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 --2- 172.21.15.29:0/3368687412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3360003e10 0x7f3360004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 --2- 172.21.15.29:0/3368687412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3360004b80 0x7f3360005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3374069890 con 0x7f3360003e10 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3374069b70 con 0x7f3360004b80 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336000b8e0 con 0x7f3374150d40 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f3378ea2700 1 --2- 172.21.15.29:0/3368687412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3360003e10 0x7f3360004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f3379ea4700 1 --2- 172.21.15.29:0/3368687412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3360004b80 0x7f3360005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f33796a3700 1 --2- 172.21.15.29:0/3368687412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3374150d40 0x7f33600038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f3378ea2700 1 -- 172.21.15.29:0/3368687412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3374150d40 msgr2=0x7f33600038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f3378ea2700 1 --2- 172.21.15.29:0/3368687412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3374150d40 0x7f33600038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f3378ea2700 1 -- 172.21.15.29:0/3368687412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3360004b80 msgr2=0x7f3360005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f3378ea2700 1 --2- 172.21.15.29:0/3368687412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3360004b80 0x7f3360005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f3378ea2700 1 -- 172.21.15.29:0/3368687412 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33600033f0 con 0x7f3360003e10 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f3378ea2700 1 --2- 172.21.15.29:0/3368687412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3360003e10 0x7f3360004690 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f336400c210 tx=0x7f3364001e90).ready entity=mon.0 client_cookie=d0e705151a649dd7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.435+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364007a60 con 0x7f3360003e10 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.436+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336400caa0 con 0x7f3360003e10 2024-01-24T14:13:30.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.436+0000 7f336bfff700 1 -- 172.21.15.29:0/3368687412 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336400ce20 con 0x7f3360003e10 2024-01-24T14:13:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.436+0000 7f337b907700 1 -- 172.21.15.29:0/3368687412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3360003e10 msgr2=0x7f3360004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.436+0000 7f337b907700 1 --2- 172.21.15.29:0/3368687412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3360003e10 0x7f3360004690 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f336400c210 tx=0x7f3364001e90).stop 2024-01-24T14:13:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.436+0000 7f3379ea4700 1 -- 172.21.15.29:0/3368687412 reap_dead start 2024-01-24T14:13:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.436+0000 7f337b907700 1 -- 172.21.15.29:0/3368687412 shutdown_connections 2024-01-24T14:13:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.436+0000 7f337b907700 1 -- 172.21.15.29:0/3368687412 >> 172.21.15.29:0/3368687412 conn(0x7f33740c8580 msgr2=0x7f33740c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.436+0000 7f337b907700 1 -- 172.21.15.29:0/3368687412 shutdown_connections 2024-01-24T14:13:30.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.436+0000 7f337b907700 1 -- 172.21.15.29:0/3368687412 wait complete. 2024-01-24T14:13:30.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.437+0000 7f337b907700 1 Processor -- start 2024-01-24T14:13:30.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.437+0000 7f337b907700 1 -- start start 2024-01-24T14:13:30.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.438+0000 7f337b907700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3360003e10 0x7f337414b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.438+0000 7f337b907700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3374153110 0x7f33740abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.438+0000 7f337b907700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f337414bab0 0x7f3374154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.438+0000 7f337b907700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33741541c0 con 0x7f3374153110 2024-01-24T14:13:30.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.438+0000 7f337b907700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33741535b0 con 0x7f337414bab0 2024-01-24T14:13:30.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.438+0000 7f337b907700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3374153890 con 0x7f3360003e10 2024-01-24T14:13:30.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f3378ea2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3374153110 0x7f33740abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f3378ea2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3374153110 0x7f33740abde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41554/0 (socket says 172.21.15.29:41554) 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f3378ea2700 1 -- 172.21.15.29:0/587916741 learned_addr learned my addr 172.21.15.29:0/587916741 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f33796a3700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3360003e10 0x7f337414b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f3379ea4700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f337414bab0 0x7f3374154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f3378ea2700 1 -- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3360003e10 msgr2=0x7f337414b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f3378ea2700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3360003e10 0x7f337414b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f3378ea2700 1 -- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f337414bab0 msgr2=0x7f3374154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f3378ea2700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f337414bab0 0x7f3374154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f3378ea2700 1 -- 172.21.15.29:0/587916741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3374153b70 con 0x7f3374153110 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f3378ea2700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3374153110 0x7f33740abde0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f3364001ed0 tx=0x7f3364015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f33697fa700 1 -- 172.21.15.29:0/587916741 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364007640 con 0x7f3374153110 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f33697fa700 1 -- 172.21.15.29:0/587916741 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33640046b0 con 0x7f3374153110 2024-01-24T14:13:30.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f33697fa700 1 -- 172.21.15.29:0/587916741 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364004a70 con 0x7f3374153110 2024-01-24T14:13:30.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f337b907700 1 -- 172.21.15.29:0/587916741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3374153e50 con 0x7f3374153110 2024-01-24T14:13:30.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.439+0000 7f337b907700 1 -- 172.21.15.29:0/587916741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33740ac510 con 0x7f3374153110 2024-01-24T14:13:30.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.440+0000 7f337b907700 1 -- 172.21.15.29:0/587916741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33740475e0 con 0x7f3374153110 2024-01-24T14:13:30.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.441+0000 7f33697fa700 1 -- 172.21.15.29:0/587916741 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3364020450 con 0x7f3374153110 2024-01-24T14:13:30.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.441+0000 7f33697fa700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f335805f270 0x7f3358061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.441+0000 7f33697fa700 1 -- 172.21.15.29:0/587916741 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f336407b380 con 0x7f3374153110 2024-01-24T14:13:30.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.441+0000 7f33796a3700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f335805f270 0x7f3358061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.442+0000 7f33796a3700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f335805f270 0x7f3358061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f3370006d80 tx=0x7f3370009060).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:30.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.446+0000 7f33697fa700 1 -- 172.21.15.29:0/587916741 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f336404c070 con 0x7f3374153110 2024-01-24T14:13:30.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.673+0000 7f337b907700 1 -- 172.21.15.29:0/587916741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33740c51a0 con 0x7f3374153110 2024-01-24T14:13:30.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.674+0000 7f33697fa700 1 -- 172.21.15.29:0/587916741 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f336401a090 con 0x7f3374153110 2024-01-24T14:13:30.675 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:30.675 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1149323432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1149323432},{"type":"v1","addr":"172.21.15.163:6835","nonce":1149323432}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6157,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2941255566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":2941255566},{"type":"v1","addr":"172.21.15.29:6835","nonce":2941255566}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6163,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2235282008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2235282008},{"type":"v1","addr":"172.21.15.163:6837","nonce":2235282008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:13:27.270903+0000","modified":"2024-01-24T14:13:30.278839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6366":{"gid":6366,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":4,"addr":"172.21.15.29:6833/3957899438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3957899438},{"type":"v1","addr":"172.21.15.29:6833","nonce":3957899438}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-24T14:13:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.676+0000 7f337b907700 1 -- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f335805f270 msgr2=0x7f3358061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.676+0000 7f337b907700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f335805f270 0x7f3358061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f3370006d80 tx=0x7f3370009060).stop 2024-01-24T14:13:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.676+0000 7f337b907700 1 -- 172.21.15.29:0/587916741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3374153110 msgr2=0x7f33740abde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.676+0000 7f337b907700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3374153110 0x7f33740abde0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f3364001ed0 tx=0x7f3364015000).stop 2024-01-24T14:13:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.676+0000 7f337b907700 1 -- 172.21.15.29:0/587916741 shutdown_connections 2024-01-24T14:13:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.676+0000 7f337b907700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f337414bab0 0x7f3374154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.676+0000 7f337b907700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3360003e10 0x7f337414b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.676+0000 7f337b907700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f335805f270 0x7f3358061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.676+0000 7f337b907700 1 --2- 172.21.15.29:0/587916741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3374153110 0x7f33740abde0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.676+0000 7f337b907700 1 -- 172.21.15.29:0/587916741 >> 172.21.15.29:0/587916741 conn(0x7f33740c8580 msgr2=0x7f33740c4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:30.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.676+0000 7f337b907700 1 -- 172.21.15.29:0/587916741 shutdown_connections 2024-01-24T14:13:30.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.676+0000 7f337b907700 1 -- 172.21.15.29:0/587916741 wait complete. 2024-01-24T14:13:30.678 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 35 2024-01-24T14:13:30.690 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:13:30.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.820+0000 7f3760787700 1 Processor -- start 2024-01-24T14:13:30.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.820+0000 7f3760787700 1 -- start start 2024-01-24T14:13:30.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.821+0000 7f3760787700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37581565f0 0x7f375815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.821+0000 7f3760787700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37580c3910 con 0x7f3758066c00 2024-01-24T14:13:30.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.821+0000 7f3760787700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37580c5220 con 0x7f3758155b00 2024-01-24T14:13:30.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.821+0000 7f3760787700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37580c5500 con 0x7f37581565f0 2024-01-24T14:13:30.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.821+0000 7f375dd22700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3758066c00 0x7f3758155ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35918/0 (socket says 172.21.15.29:35918) 2024-01-24T14:13:30.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.821+0000 7f375dd22700 1 -- 172.21.15.29:0/2325199123 learned_addr learned my addr 172.21.15.29:0/2325199123 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:30.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.822+0000 7f375ed24700 1 --2- 172.21.15.29:0/2325199123 >> v2:172.21.15.163:3300/0 conn(0x7f37581565f0 0x7f375815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.822+0000 7f375d521700 1 -- 172.21.15.29:0/2325199123 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604914297 0 0) 0x7f37580c3910 con 0x7f3758066c00 2024-01-24T14:13:30.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.822+0000 7f375d521700 1 -- 172.21.15.29:0/2325199123 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37440033f0 con 0x7f3758066c00 2024-01-24T14:13:30.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.822+0000 7f375ed24700 1 -- 172.21.15.29:0/2325199123 >> v1:172.21.15.163:6789/0 conn(0x7f3758155b00 legacy=0x7f37580664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.822+0000 7f375ed24700 1 -- 172.21.15.29:0/2325199123 >> v1:172.21.15.29:6789/0 conn(0x7f3758066c00 legacy=0x7f3758155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.822+0000 7f375ed24700 1 -- 172.21.15.29:0/2325199123 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37580c3350 con 0x7f37581565f0 2024-01-24T14:13:30.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.822+0000 7f375ed24700 1 --2- 172.21.15.29:0/2325199123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37581565f0 0x7f375815cd70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f3750009e90 tx=0x7f3750009780).ready entity=mon.1 client_cookie=aac1e35793643a23 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:30.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.825+0000 7f375d521700 1 -- 172.21.15.29:0/2325199123 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375000e070 con 0x7f37581565f0 2024-01-24T14:13:30.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.825+0000 7f375d521700 1 -- 172.21.15.29:0/2325199123 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375000bb20 con 0x7f37581565f0 2024-01-24T14:13:30.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.825+0000 7f375d521700 1 -- 172.21.15.29:0/2325199123 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375000bea0 con 0x7f37581565f0 2024-01-24T14:13:30.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.825+0000 7f3760787700 1 -- 172.21.15.29:0/2325199123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37581565f0 msgr2=0x7f375815cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.825+0000 7f3760787700 1 --2- 172.21.15.29:0/2325199123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37581565f0 0x7f375815cd70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f3750009e90 tx=0x7f3750009780).stop 2024-01-24T14:13:30.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.825+0000 7f3760787700 1 -- 172.21.15.29:0/2325199123 shutdown_connections 2024-01-24T14:13:30.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.825+0000 7f3760787700 1 --2- 172.21.15.29:0/2325199123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37581565f0 0x7f375815cd70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.825+0000 7f3760787700 1 -- 172.21.15.29:0/2325199123 >> 172.21.15.29:0/2325199123 conn(0x7f37580c8bb0 msgr2=0x7f37580c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:30.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.825+0000 7f3760787700 1 -- 172.21.15.29:0/2325199123 shutdown_connections 2024-01-24T14:13:30.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.825+0000 7f3760787700 1 -- 172.21.15.29:0/2325199123 wait complete. 2024-01-24T14:13:30.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.827+0000 7f3760787700 1 Processor -- start 2024-01-24T14:13:30.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.827+0000 7f3760787700 1 -- start start 2024-01-24T14:13:30.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.828+0000 7f3760787700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3758066c00 0x7f37581531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.828+0000 7f3760787700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3758155b00 0x7f3758150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.828+0000 7f3760787700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37581565f0 0x7f375814d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.828+0000 7f3760787700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3758150c40 con 0x7f3758155b00 2024-01-24T14:13:30.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.828+0000 7f3760787700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375814d8b0 con 0x7f37581565f0 2024-01-24T14:13:30.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.828+0000 7f3760787700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3758151140 con 0x7f3758066c00 2024-01-24T14:13:30.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.828+0000 7f375dd22700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3758155b00 0x7f3758150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.828+0000 7f375dd22700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3758155b00 0x7f3758150750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41566/0 (socket says 172.21.15.29:41566) 2024-01-24T14:13:30.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.828+0000 7f375dd22700 1 -- 172.21.15.29:0/2508140360 learned_addr learned my addr 172.21.15.29:0/2508140360 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:30.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f375ed24700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37581565f0 0x7f375814d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f375e523700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3758066c00 0x7f37581531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f375dd22700 1 -- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3758066c00 msgr2=0x7f37581531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f375dd22700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3758066c00 0x7f37581531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f375dd22700 1 -- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37581565f0 msgr2=0x7f375814d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:30.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f375dd22700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37581565f0 0x7f375814d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:30.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f375dd22700 1 -- 172.21.15.29:0/2508140360 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3750009a20 con 0x7f3758155b00 2024-01-24T14:13:30.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f375dd22700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3758155b00 0x7f3758150750 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f374800c440 tx=0x7f3748007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:30.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f374e7fc700 1 -- 172.21.15.29:0/2508140360 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37480070d0 con 0x7f3758155b00 2024-01-24T14:13:30.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f3760787700 1 -- 172.21.15.29:0/2508140360 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3758153850 con 0x7f3758155b00 2024-01-24T14:13:30.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f374e7fc700 1 -- 172.21.15.29:0/2508140360 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3748007230 con 0x7f3758155b00 2024-01-24T14:13:30.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f374e7fc700 1 -- 172.21.15.29:0/2508140360 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37480075d0 con 0x7f3758155b00 2024-01-24T14:13:30.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.829+0000 7f3760787700 1 -- 172.21.15.29:0/2508140360 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3758153e10 con 0x7f3758155b00 2024-01-24T14:13:30.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.830+0000 7f3760787700 1 -- 172.21.15.29:0/2508140360 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37580475e0 con 0x7f3758155b00 2024-01-24T14:13:30.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.831+0000 7f374e7fc700 1 -- 172.21.15.29:0/2508140360 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f37480059b0 con 0x7f3758155b00 2024-01-24T14:13:30.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.831+0000 7f374e7fc700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f373c05f270 0x7f373c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:30.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.831+0000 7f374e7fc700 1 -- 172.21.15.29:0/2508140360 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f3748079da0 con 0x7f3758155b00 2024-01-24T14:13:30.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.831+0000 7f375e523700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f373c05f270 0x7f373c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:30.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.832+0000 7f375e523700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f373c05f270 0x7f373c061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f3754003900 tx=0x7f3754007d40).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:30.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:30.835+0000 7f374e7fc700 1 -- 172.21.15.29:0/2508140360 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f374804aa90 con 0x7f3758155b00 2024-01-24T14:13:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.018+0000 7f3760787700 1 -- 172.21.15.29:0/2508140360 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3758151280 con 0x7f3758155b00 2024-01-24T14:13:31.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.019+0000 7f374e7fc700 1 -- 172.21.15.29:0/2508140360 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+18173 (secure 0 0 0) 0x7f3748018030 con 0x7f3758155b00 2024-01-24T14:13:31.020 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:31.020 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":90,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:13:28.272658+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:13:25.288458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-24T14:13:26.243223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:13:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.021+0000 7f3760787700 1 -- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f373c05f270 msgr2=0x7f373c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.021+0000 7f3760787700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f373c05f270 0x7f373c061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f3754003900 tx=0x7f3754007d40).stop 2024-01-24T14:13:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.021+0000 7f3760787700 1 -- 172.21.15.29:0/2508140360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3758155b00 msgr2=0x7f3758150750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.021+0000 7f3760787700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3758155b00 0x7f3758150750 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f374800c440 tx=0x7f3748007c30).stop 2024-01-24T14:13:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.021+0000 7f3760787700 1 -- 172.21.15.29:0/2508140360 shutdown_connections 2024-01-24T14:13:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.021+0000 7f3760787700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37581565f0 0x7f375814d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.021+0000 7f3760787700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3758066c00 0x7f37581531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.021+0000 7f3760787700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f373c05f270 0x7f373c061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.021+0000 7f3760787700 1 --2- 172.21.15.29:0/2508140360 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3758155b00 0x7f3758150750 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.021+0000 7f3760787700 1 -- 172.21.15.29:0/2508140360 >> 172.21.15.29:0/2508140360 conn(0x7f37580c8bb0 msgr2=0x7f37580c6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.021+0000 7f3760787700 1 -- 172.21.15.29:0/2508140360 shutdown_connections 2024-01-24T14:13:31.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.021+0000 7f3760787700 1 -- 172.21.15.29:0/2508140360 wait complete. 2024-01-24T14:13:31.036 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-24T14:13:31.036 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:13:31.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.166+0000 7effa65d0700 1 Processor -- start 2024-01-24T14:13:31.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.166+0000 7effa65d0700 1 -- start start 2024-01-24T14:13:31.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.167+0000 7effa65d0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7effa01543c0 0x7effa00bcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.167+0000 7effa65d0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7effa00bd120 0x7effa00c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.167+0000 7effa65d0700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa005e850 con 0x7effa0153fe0 2024-01-24T14:13:31.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.167+0000 7effa65d0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa00a4f90 con 0x7effa00bd120 2024-01-24T14:13:31.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.167+0000 7effa65d0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa00631f0 con 0x7effa01543c0 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.167+0000 7effa4b6d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7effa00bd120 0x7effa00c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.167+0000 7effa4b6d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7effa00bd120 0x7effa00c1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41574/0 (socket says 172.21.15.29:41574) 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.167+0000 7effa4b6d700 1 -- 172.21.15.29:0/1512199602 learned_addr learned my addr 172.21.15.29:0/1512199602 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.167+0000 7eff9ffff700 1 --1- 172.21.15.29:0/1512199602 >> v1:172.21.15.29:6789/0 conn(0x7effa0153fe0 0x7effa00a4880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35932/0 (socket says 172.21.15.29:35932) 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.168+0000 7eff9f7fe700 1 --2- 172.21.15.29:0/1512199602 >> v2:172.21.15.163:3300/0 conn(0x7effa01543c0 0x7effa00bcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.168+0000 7effa4b6d700 1 -- 172.21.15.29:0/1512199602 >> v2:172.21.15.163:3300/0 conn(0x7effa01543c0 msgr2=0x7effa00bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.168+0000 7effa4b6d700 1 --2- 172.21.15.29:0/1512199602 >> v2:172.21.15.163:3300/0 conn(0x7effa01543c0 0x7effa00bcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.168+0000 7effa4b6d700 1 -- 172.21.15.29:0/1512199602 >> v1:172.21.15.29:6789/0 conn(0x7effa0153fe0 legacy=0x7effa00a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.168+0000 7effa4b6d700 1 -- 172.21.15.29:0/1512199602 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa00bd7e0 con 0x7effa00bd120 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.168+0000 7eff9effd700 1 -- 172.21.15.29:0/1512199602 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120194275 0 0) 0x7effa005e850 con 0x7effa0153fe0 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.168+0000 7eff9f7fe700 1 --2- 172.21.15.29:0/1512199602 >> v2:172.21.15.163:3300/0 conn(0x7effa01543c0 0x7effa00bcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.168+0000 7effa4b6d700 1 --2- 172.21.15.29:0/1512199602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa00bd120 0x7effa00c1870 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7effa00c3ce0 tx=0x7eff98009840).ready entity=mon.0 client_cookie=3abbae5304759c89 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:31.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.168+0000 7eff9effd700 1 -- 172.21.15.29:0/1512199602 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800e070 con 0x7effa00bd120 2024-01-24T14:13:31.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.168+0000 7eff9effd700 1 -- 172.21.15.29:0/1512199602 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9800ba50 con 0x7effa00bd120 2024-01-24T14:13:31.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.168+0000 7eff9effd700 1 -- 172.21.15.29:0/1512199602 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800bdd0 con 0x7effa00bd120 2024-01-24T14:13:31.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.169+0000 7effa65d0700 1 -- 172.21.15.29:0/1512199602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa00bd120 msgr2=0x7effa00c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.169+0000 7effa65d0700 1 --2- 172.21.15.29:0/1512199602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa00bd120 0x7effa00c1870 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7effa00c3ce0 tx=0x7eff98009840).stop 2024-01-24T14:13:31.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.169+0000 7effa65d0700 1 -- 172.21.15.29:0/1512199602 shutdown_connections 2024-01-24T14:13:31.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.169+0000 7effa65d0700 1 --2- 172.21.15.29:0/1512199602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa00bd120 0x7effa00c1870 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.169+0000 7effa65d0700 1 --2- 172.21.15.29:0/1512199602 >> v2:172.21.15.163:3300/0 conn(0x7effa01543c0 0x7effa00bcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.169+0000 7effa65d0700 1 -- 172.21.15.29:0/1512199602 >> 172.21.15.29:0/1512199602 conn(0x7effa0064aa0 msgr2=0x7effa0064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:31.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.169+0000 7effa65d0700 1 -- 172.21.15.29:0/1512199602 shutdown_connections 2024-01-24T14:13:31.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.169+0000 7effa65d0700 1 -- 172.21.15.29:0/1512199602 wait complete. 2024-01-24T14:13:31.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.170+0000 7effa65d0700 1 Processor -- start 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.170+0000 7effa65d0700 1 -- start start 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.171+0000 7effa65d0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa00bd120 0x7effa0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.171+0000 7effa65d0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa0153fe0 0x7effa0150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.171+0000 7effa65d0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01543c0 0x7effa014c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.171+0000 7effa65d0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa0069a40 con 0x7effa01543c0 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.171+0000 7effa65d0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa0068e30 con 0x7effa0153fe0 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.171+0000 7effa65d0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa0069110 con 0x7effa00bd120 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.171+0000 7effa4b6d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01543c0 0x7effa014c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.171+0000 7effa4b6d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01543c0 0x7effa014c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41586/0 (socket says 172.21.15.29:41586) 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.171+0000 7effa4b6d700 1 -- 172.21.15.29:0/2583795982 learned_addr learned my addr 172.21.15.29:0/2583795982 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.171+0000 7eff9f7fe700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa0153fe0 0x7effa0150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.171+0000 7eff9ffff700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa00bd120 0x7effa0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.171+0000 7effa4b6d700 1 -- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa00bd120 msgr2=0x7effa0069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.172+0000 7effa4b6d700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa00bd120 0x7effa0069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.172+0000 7effa4b6d700 1 -- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa0153fe0 msgr2=0x7effa0150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.172+0000 7effa4b6d700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa0153fe0 0x7effa0150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.172+0000 7effa4b6d700 1 -- 172.21.15.29:0/2583795982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff98009ae0 con 0x7effa01543c0 2024-01-24T14:13:31.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.172+0000 7effa4b6d700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01543c0 0x7effa014c0d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7effa0064330 tx=0x7effa00c3320).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:31.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.172+0000 7eff97fff700 1 -- 172.21.15.29:0/2583795982 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff980032d0 con 0x7effa01543c0 2024-01-24T14:13:31.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.172+0000 7eff97fff700 1 -- 172.21.15.29:0/2583795982 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff980092e0 con 0x7effa01543c0 2024-01-24T14:13:31.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.172+0000 7effa65d0700 1 -- 172.21.15.29:0/2583795982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa00693f0 con 0x7effa01543c0 2024-01-24T14:13:31.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.172+0000 7eff97fff700 1 -- 172.21.15.29:0/2583795982 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9801d640 con 0x7effa01543c0 2024-01-24T14:13:31.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.172+0000 7effa65d0700 1 -- 172.21.15.29:0/2583795982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa0150f40 con 0x7effa01543c0 2024-01-24T14:13:31.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.173+0000 7effa65d0700 1 -- 172.21.15.29:0/2583795982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa0151100 con 0x7effa01543c0 2024-01-24T14:13:31.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.173+0000 7eff97fff700 1 -- 172.21.15.29:0/2583795982 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7eff9801da30 con 0x7effa01543c0 2024-01-24T14:13:31.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.174+0000 7eff97fff700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff8005f270 0x7eff80061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.174+0000 7eff97fff700 1 -- 172.21.15.29:0/2583795982 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7eff9807c230 con 0x7effa01543c0 2024-01-24T14:13:31.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.174+0000 7eff9ffff700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff8005f270 0x7eff80061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.174+0000 7eff9ffff700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff8005f270 0x7eff80061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7eff900011e0 tx=0x7eff90007bd0).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:31.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.178+0000 7eff97fff700 1 -- 172.21.15.29:0/2583795982 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7eff9804d040 con 0x7effa01543c0 2024-01-24T14:13:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.355+0000 7effa65d0700 1 -- 172.21.15.29:0/2583795982 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7effa0064110 con 0x7eff8005f270 2024-01-24T14:13:31.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.368+0000 7eff97fff700 1 -- 172.21.15.29:0/2583795982 <== mgr.5558 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361387 (secure 0 0 0) 0x7effa0064110 con 0x7eff8005f270 2024-01-24T14:13:31.373 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:31.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.371+0000 7effa65d0700 1 -- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff8005f270 msgr2=0x7eff80061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.371+0000 7effa65d0700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff8005f270 0x7eff80061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7eff900011e0 tx=0x7eff90007bd0).stop 2024-01-24T14:13:31.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.371+0000 7effa65d0700 1 -- 172.21.15.29:0/2583795982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01543c0 msgr2=0x7effa014c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.371+0000 7effa65d0700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01543c0 0x7effa014c0d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7effa0064330 tx=0x7effa00c3320).stop 2024-01-24T14:13:31.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.371+0000 7effa65d0700 1 -- 172.21.15.29:0/2583795982 shutdown_connections 2024-01-24T14:13:31.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.371+0000 7effa65d0700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7eff8005f270 0x7eff80061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.371+0000 7effa65d0700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa01543c0 0x7effa014c0d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.371+0000 7effa65d0700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa00bd120 0x7effa0069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.371+0000 7effa65d0700 1 --2- 172.21.15.29:0/2583795982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa0153fe0 0x7effa0150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.371+0000 7effa65d0700 1 -- 172.21.15.29:0/2583795982 >> 172.21.15.29:0/2583795982 conn(0x7effa0064aa0 msgr2=0x7effa00a5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:31.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.372+0000 7effa65d0700 1 -- 172.21.15.29:0/2583795982 shutdown_connections 2024-01-24T14:13:31.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.372+0000 7effa65d0700 1 -- 172.21.15.29:0/2583795982 wait complete. 2024-01-24T14:13:31.377 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:13:31.386 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-01-24T14:13:30.120629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422212,"kb_used_data":8580,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752552508,"statfs":{"total":773094113280,"available":770613768192,"internally_reserved":0,"allocated":8785920,"data_stored":4759393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28929,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-81,"num_write_kb":-66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-1067,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.699997"},"pg_stats":[{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.312108+0000","last_change":"2024-01-24T14:13:27.308964+0000","last_active":"2024-01-24T14:13:27.312108+0000","last_peered":"2024-01-24T14:13:27.312108+0000","last_clean":"2024-01-24T14:13:27.312108+0000","last_became_active":"2024-01-24T14:13:27.306530+0000","last_became_peered":"2024-01-24T14:13:27.306530+0000","last_unstale":"2024-01-24T14:13:27.312108+0000","last_undegraded":"2024-01-24T14:13:27.312108+0000","last_fullsized":"2024-01-24T14:13:27.312108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3f","version":"89'3","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.307054+0000","last_change":"2024-01-24T14:13:26.871570+0000","last_active":"2024-01-24T14:13:27.307054+0000","last_peered":"2024-01-24T14:13:27.307054+0000","last_clean":"2024-01-24T14:13:27.307054+0000","last_became_active":"2024-01-24T14:13:26.870323+0000","last_became_peered":"2024-01-24T14:13:26.870323+0000","last_unstale":"2024-01-24T14:13:27.307054+0000","last_undegraded":"2024-01-24T14:13:27.307054+0000","last_fullsized":"2024-01-24T14:13:27.307054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.307904+0000","last_change":"2024-01-24T14:13:27.302233+0000","last_active":"2024-01-24T14:13:27.307904+0000","last_peered":"2024-01-24T14:13:27.307904+0000","last_clean":"2024-01-24T14:13:27.307904+0000","last_became_active":"2024-01-24T14:13:27.301920+0000","last_became_peered":"2024-01-24T14:13:27.301920+0000","last_unstale":"2024-01-24T14:13:27.307904+0000","last_undegraded":"2024-01-24T14:13:27.307904+0000","last_fullsized":"2024-01-24T14:13:27.307904+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291665+0000","last_change":"2024-01-24T14:13:26.882561+0000","last_active":"2024-01-24T14:13:27.291665+0000","last_peered":"2024-01-24T14:13:27.291665+0000","last_clean":"2024-01-24T14:13:27.291665+0000","last_became_active":"2024-01-24T14:13:26.881499+0000","last_became_peered":"2024-01-24T14:13:26.881499+0000","last_unstale":"2024-01-24T14:13:27.291665+0000","last_undegraded":"2024-01-24T14:13:27.291665+0000","last_fullsized":"2024-01-24T14:13:27.291665+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.306396+0000","last_change":"2024-01-24T14:13:27.304914+0000","last_active":"2024-01-24T14:13:27.306396+0000","last_peered":"2024-01-24T14:13:27.306396+0000","last_clean":"2024-01-24T14:13:27.306396+0000","last_became_active":"2024-01-24T14:13:27.297643+0000","last_became_peered":"2024-01-24T14:13:27.297643+0000","last_unstale":"2024-01-24T14:13:27.306396+0000","last_undegraded":"2024-01-24T14:13:27.306396+0000","last_fullsized":"2024-01-24T14:13:27.306396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296654+0000","last_change":"2024-01-24T14:13:26.881205+0000","last_active":"2024-01-24T14:13:27.296654+0000","last_peered":"2024-01-24T14:13:27.296654+0000","last_clean":"2024-01-24T14:13:27.296654+0000","last_became_active":"2024-01-24T14:13:26.880723+0000","last_became_peered":"2024-01-24T14:13:26.880723+0000","last_unstale":"2024-01-24T14:13:27.296654+0000","last_undegraded":"2024-01-24T14:13:27.296654+0000","last_fullsized":"2024-01-24T14:13:27.296654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.303518+0000","last_change":"2024-01-24T14:13:27.295901+0000","last_active":"2024-01-24T14:13:27.303518+0000","last_peered":"2024-01-24T14:13:27.303518+0000","last_clean":"2024-01-24T14:13:27.303518+0000","last_became_active":"2024-01-24T14:13:27.295620+0000","last_became_peered":"2024-01-24T14:13:27.295620+0000","last_unstale":"2024-01-24T14:13:27.303518+0000","last_undegraded":"2024-01-24T14:13:27.303518+0000","last_fullsized":"2024-01-24T14:13:27.303518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297309+0000","last_change":"2024-01-24T14:13:26.876152+0000","last_active":"2024-01-24T14:13:27.297309+0000","last_peered":"2024-01-24T14:13:27.297309+0000","last_clean":"2024-01-24T14:13:27.297309+0000","last_became_active":"2024-01-24T14:13:26.875854+0000","last_became_peered":"2024-01-24T14:13:26.875854+0000","last_unstale":"2024-01-24T14:13:27.297309+0000","last_undegraded":"2024-01-24T14:13:27.297309+0000","last_fullsized":"2024-01-24T14:13:27.297309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311941+0000","last_change":"2024-01-24T14:13:27.307224+0000","last_active":"2024-01-24T14:13:27.311941+0000","last_peered":"2024-01-24T14:13:27.311941+0000","last_clean":"2024-01-24T14:13:27.311941+0000","last_became_active":"2024-01-24T14:13:27.306551+0000","last_became_peered":"2024-01-24T14:13:27.306551+0000","last_unstale":"2024-01-24T14:13:27.311941+0000","last_undegraded":"2024-01-24T14:13:27.311941+0000","last_fullsized":"2024-01-24T14:13:27.311941+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3b","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309339+0000","last_change":"2024-01-24T14:13:26.886280+0000","last_active":"2024-01-24T14:13:27.309339+0000","last_peered":"2024-01-24T14:13:27.309339+0000","last_clean":"2024-01-24T14:13:27.309339+0000","last_became_active":"2024-01-24T14:13:26.885965+0000","last_became_peered":"2024-01-24T14:13:26.885965+0000","last_unstale":"2024-01-24T14:13:27.309339+0000","last_undegraded":"2024-01-24T14:13:27.309339+0000","last_fullsized":"2024-01-24T14:13:27.309339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.302150+0000","last_change":"2024-01-24T14:13:27.300613+0000","last_active":"2024-01-24T14:13:27.302150+0000","last_peered":"2024-01-24T14:13:27.302150+0000","last_clean":"2024-01-24T14:13:27.302150+0000","last_became_active":"2024-01-24T14:13:27.300337+0000","last_became_peered":"2024-01-24T14:13:27.300337+0000","last_unstale":"2024-01-24T14:13:27.302150+0000","last_undegraded":"2024-01-24T14:13:27.302150+0000","last_fullsized":"2024-01-24T14:13:27.302150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3a","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296996+0000","last_change":"2024-01-24T14:13:26.878801+0000","last_active":"2024-01-24T14:13:27.296996+0000","last_peered":"2024-01-24T14:13:27.296996+0000","last_clean":"2024-01-24T14:13:27.296996+0000","last_became_active":"2024-01-24T14:13:26.878495+0000","last_became_peered":"2024-01-24T14:13:26.878495+0000","last_unstale":"2024-01-24T14:13:27.296996+0000","last_undegraded":"2024-01-24T14:13:27.296996+0000","last_fullsized":"2024-01-24T14:13:27.296996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309479+0000","last_change":"2024-01-24T14:13:27.295823+0000","last_active":"2024-01-24T14:13:27.309479+0000","last_peered":"2024-01-24T14:13:27.309479+0000","last_clean":"2024-01-24T14:13:27.309479+0000","last_became_active":"2024-01-24T14:13:27.295159+0000","last_became_peered":"2024-01-24T14:13:27.295159+0000","last_unstale":"2024-01-24T14:13:27.309479+0000","last_undegraded":"2024-01-24T14:13:27.309479+0000","last_fullsized":"2024-01-24T14:13:27.309479+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.275061+0000","last_change":"2024-01-24T14:13:26.878394+0000","last_active":"2024-01-24T14:13:27.275061+0000","last_peered":"2024-01-24T14:13:27.275061+0000","last_clean":"2024-01-24T14:13:27.275061+0000","last_became_active":"2024-01-24T14:13:26.878090+0000","last_became_peered":"2024-01-24T14:13:26.878090+0000","last_unstale":"2024-01-24T14:13:27.275061+0000","last_undegraded":"2024-01-24T14:13:27.275061+0000","last_fullsized":"2024-01-24T14:13:27.275061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294497+0000","last_change":"2024-01-24T14:13:27.291413+0000","last_active":"2024-01-24T14:13:27.294497+0000","last_peered":"2024-01-24T14:13:27.294497+0000","last_clean":"2024-01-24T14:13:27.294497+0000","last_became_active":"2024-01-24T14:13:27.290960+0000","last_became_peered":"2024-01-24T14:13:27.290960+0000","last_unstale":"2024-01-24T14:13:27.294497+0000","last_undegraded":"2024-01-24T14:13:27.294497+0000","last_fullsized":"2024-01-24T14:13:27.294497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.280494+0000","last_change":"2024-01-24T14:13:26.875046+0000","last_active":"2024-01-24T14:13:27.280494+0000","last_peered":"2024-01-24T14:13:27.280494+0000","last_clean":"2024-01-24T14:13:27.280494+0000","last_became_active":"2024-01-24T14:13:26.874106+0000","last_became_peered":"2024-01-24T14:13:26.874106+0000","last_unstale":"2024-01-24T14:13:27.280494+0000","last_undegraded":"2024-01-24T14:13:27.280494+0000","last_fullsized":"2024-01-24T14:13:27.280494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.299182+0000","last_change":"2024-01-24T14:13:27.286360+0000","last_active":"2024-01-24T14:13:27.299182+0000","last_peered":"2024-01-24T14:13:27.299182+0000","last_clean":"2024-01-24T14:13:27.299182+0000","last_became_active":"2024-01-24T14:13:27.286072+0000","last_became_peered":"2024-01-24T14:13:27.286072+0000","last_unstale":"2024-01-24T14:13:27.299182+0000","last_undegraded":"2024-01-24T14:13:27.299182+0000","last_fullsized":"2024-01-24T14:13:27.299182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297712+0000","last_change":"2024-01-24T14:13:26.878664+0000","last_active":"2024-01-24T14:13:27.297712+0000","last_peered":"2024-01-24T14:13:27.297712+0000","last_clean":"2024-01-24T14:13:27.297712+0000","last_became_active":"2024-01-24T14:13:26.878345+0000","last_became_peered":"2024-01-24T14:13:26.878345+0000","last_unstale":"2024-01-24T14:13:27.297712+0000","last_undegraded":"2024-01-24T14:13:27.297712+0000","last_fullsized":"2024-01-24T14:13:27.297712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.292080+0000","last_change":"2024-01-24T14:13:27.290026+0000","last_active":"2024-01-24T14:13:27.292080+0000","last_peered":"2024-01-24T14:13:27.292080+0000","last_clean":"2024-01-24T14:13:27.292080+0000","last_became_active":"2024-01-24T14:13:27.289736+0000","last_became_peered":"2024-01-24T14:13:27.289736+0000","last_unstale":"2024-01-24T14:13:27.292080+0000","last_undegraded":"2024-01-24T14:13:27.292080+0000","last_fullsized":"2024-01-24T14:13:27.292080+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297602+0000","last_change":"2024-01-24T14:13:26.881814+0000","last_active":"2024-01-24T14:13:27.297602+0000","last_peered":"2024-01-24T14:13:27.297602+0000","last_clean":"2024-01-24T14:13:27.297602+0000","last_became_active":"2024-01-24T14:13:26.881517+0000","last_became_peered":"2024-01-24T14:13:26.881517+0000","last_unstale":"2024-01-24T14:13:27.297602+0000","last_undegraded":"2024-01-24T14:13:27.297602+0000","last_fullsized":"2024-01-24T14:13:27.297602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.305196+0000","last_change":"2024-01-24T14:13:27.294094+0000","last_active":"2024-01-24T14:13:27.305196+0000","last_peered":"2024-01-24T14:13:27.305196+0000","last_clean":"2024-01-24T14:13:27.305196+0000","last_became_active":"2024-01-24T14:13:27.293472+0000","last_became_peered":"2024-01-24T14:13:27.293472+0000","last_unstale":"2024-01-24T14:13:27.305196+0000","last_undegraded":"2024-01-24T14:13:27.305196+0000","last_fullsized":"2024-01-24T14:13:27.305196+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.282075+0000","last_change":"2024-01-24T14:13:26.876449+0000","last_active":"2024-01-24T14:13:27.282075+0000","last_peered":"2024-01-24T14:13:27.282075+0000","last_clean":"2024-01-24T14:13:27.282075+0000","last_became_active":"2024-01-24T14:13:26.875833+0000","last_became_peered":"2024-01-24T14:13:26.875833+0000","last_unstale":"2024-01-24T14:13:27.282075+0000","last_undegraded":"2024-01-24T14:13:27.282075+0000","last_fullsized":"2024-01-24T14:13:27.282075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.310448+0000","last_change":"2024-01-24T14:13:27.305044+0000","last_active":"2024-01-24T14:13:27.310448+0000","last_peered":"2024-01-24T14:13:27.310448+0000","last_clean":"2024-01-24T14:13:27.310448+0000","last_became_active":"2024-01-24T14:13:27.304734+0000","last_became_peered":"2024-01-24T14:13:27.304734+0000","last_unstale":"2024-01-24T14:13:27.310448+0000","last_undegraded":"2024-01-24T14:13:27.310448+0000","last_fullsized":"2024-01-24T14:13:27.310448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.34","version":"89'4","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.314002+0000","last_change":"2024-01-24T14:13:26.874452+0000","last_active":"2024-01-24T14:13:27.314002+0000","last_peered":"2024-01-24T14:13:27.314002+0000","last_clean":"2024-01-24T14:13:27.314002+0000","last_became_active":"2024-01-24T14:13:26.874170+0000","last_became_peered":"2024-01-24T14:13:26.874170+0000","last_unstale":"2024-01-24T14:13:27.314002+0000","last_undegraded":"2024-01-24T14:13:27.314002+0000","last_fullsized":"2024-01-24T14:13:27.314002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297752+0000","last_change":"2024-01-24T14:13:27.295298+0000","last_active":"2024-01-24T14:13:27.297752+0000","last_peered":"2024-01-24T14:13:27.297752+0000","last_clean":"2024-01-24T14:13:27.297752+0000","last_became_active":"2024-01-24T14:13:27.294990+0000","last_became_peered":"2024-01-24T14:13:27.294990+0000","last_unstale":"2024-01-24T14:13:27.297752+0000","last_undegraded":"2024-01-24T14:13:27.297752+0000","last_fullsized":"2024-01-24T14:13:27.297752+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294222+0000","last_change":"2024-01-24T14:13:26.870270+0000","last_active":"2024-01-24T14:13:27.294222+0000","last_peered":"2024-01-24T14:13:27.294222+0000","last_clean":"2024-01-24T14:13:27.294222+0000","last_became_active":"2024-01-24T14:13:26.869551+0000","last_became_peered":"2024-01-24T14:13:26.869551+0000","last_unstale":"2024-01-24T14:13:27.294222+0000","last_undegraded":"2024-01-24T14:13:27.294222+0000","last_fullsized":"2024-01-24T14:13:27.294222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.301614+0000","last_change":"2024-01-24T14:13:27.298389+0000","last_active":"2024-01-24T14:13:27.301614+0000","last_peered":"2024-01-24T14:13:27.301614+0000","last_clean":"2024-01-24T14:13:27.301614+0000","last_became_active":"2024-01-24T14:13:27.298084+0000","last_became_peered":"2024-01-24T14:13:27.298084+0000","last_unstale":"2024-01-24T14:13:27.301614+0000","last_undegraded":"2024-01-24T14:13:27.301614+0000","last_fullsized":"2024-01-24T14:13:27.301614+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.295498+0000","last_change":"2024-01-24T14:13:26.881359+0000","last_active":"2024-01-24T14:13:27.295498+0000","last_peered":"2024-01-24T14:13:27.295498+0000","last_clean":"2024-01-24T14:13:27.295498+0000","last_became_active":"2024-01-24T14:13:26.881068+0000","last_became_peered":"2024-01-24T14:13:26.881068+0000","last_unstale":"2024-01-24T14:13:27.295498+0000","last_undegraded":"2024-01-24T14:13:27.295498+0000","last_fullsized":"2024-01-24T14:13:27.295498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.303902+0000","last_change":"2024-01-24T14:13:27.296706+0000","last_active":"2024-01-24T14:13:27.303902+0000","last_peered":"2024-01-24T14:13:27.303902+0000","last_clean":"2024-01-24T14:13:27.303902+0000","last_became_active":"2024-01-24T14:13:27.296387+0000","last_became_peered":"2024-01-24T14:13:27.296387+0000","last_unstale":"2024-01-24T14:13:27.303902+0000","last_undegraded":"2024-01-24T14:13:27.303902+0000","last_fullsized":"2024-01-24T14:13:27.303902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.295172+0000","last_change":"2024-01-24T14:13:26.882076+0000","last_active":"2024-01-24T14:13:27.295172+0000","last_peered":"2024-01-24T14:13:27.295172+0000","last_clean":"2024-01-24T14:13:27.295172+0000","last_became_active":"2024-01-24T14:13:26.881479+0000","last_became_peered":"2024-01-24T14:13:26.881479+0000","last_unstale":"2024-01-24T14:13:27.295172+0000","last_undegraded":"2024-01-24T14:13:27.295172+0000","last_fullsized":"2024-01-24T14:13:27.295172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294831+0000","last_change":"2024-01-24T14:13:27.290668+0000","last_active":"2024-01-24T14:13:27.294831+0000","last_peered":"2024-01-24T14:13:27.294831+0000","last_clean":"2024-01-24T14:13:27.294831+0000","last_became_active":"2024-01-24T14:13:27.290389+0000","last_became_peered":"2024-01-24T14:13:27.290389+0000","last_unstale":"2024-01-24T14:13:27.294831+0000","last_undegraded":"2024-01-24T14:13:27.294831+0000","last_fullsized":"2024-01-24T14:13:27.294831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.30","version":"89'4","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294953+0000","last_change":"2024-01-24T14:13:26.863361+0000","last_active":"2024-01-24T14:13:27.294953+0000","last_peered":"2024-01-24T14:13:27.294953+0000","last_clean":"2024-01-24T14:13:27.294953+0000","last_became_active":"2024-01-24T14:13:26.863143+0000","last_became_peered":"2024-01-24T14:13:26.863143+0000","last_unstale":"2024-01-24T14:13:27.294953+0000","last_undegraded":"2024-01-24T14:13:27.294953+0000","last_fullsized":"2024-01-24T14:13:27.294953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.308398+0000","last_change":"2024-01-24T14:13:27.306909+0000","last_active":"2024-01-24T14:13:27.308398+0000","last_peered":"2024-01-24T14:13:27.308398+0000","last_clean":"2024-01-24T14:13:27.308398+0000","last_became_active":"2024-01-24T14:13:27.306663+0000","last_became_peered":"2024-01-24T14:13:27.306663+0000","last_unstale":"2024-01-24T14:13:27.308398+0000","last_undegraded":"2024-01-24T14:13:27.308398+0000","last_fullsized":"2024-01-24T14:13:27.308398+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291778+0000","last_change":"2024-01-24T14:13:26.871737+0000","last_active":"2024-01-24T14:13:27.291778+0000","last_peered":"2024-01-24T14:13:27.291778+0000","last_clean":"2024-01-24T14:13:27.291778+0000","last_became_active":"2024-01-24T14:13:26.869983+0000","last_became_peered":"2024-01-24T14:13:26.869983+0000","last_unstale":"2024-01-24T14:13:27.291778+0000","last_undegraded":"2024-01-24T14:13:27.291778+0000","last_fullsized":"2024-01-24T14:13:27.291778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.304821+0000","last_change":"2024-01-24T14:13:27.299811+0000","last_active":"2024-01-24T14:13:27.304821+0000","last_peered":"2024-01-24T14:13:27.304821+0000","last_clean":"2024-01-24T14:13:27.304821+0000","last_became_active":"2024-01-24T14:13:27.299130+0000","last_became_peered":"2024-01-24T14:13:27.299130+0000","last_unstale":"2024-01-24T14:13:27.304821+0000","last_undegraded":"2024-01-24T14:13:27.304821+0000","last_fullsized":"2024-01-24T14:13:27.304821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.284753+0000","last_change":"2024-01-24T14:13:26.882012+0000","last_active":"2024-01-24T14:13:27.284753+0000","last_peered":"2024-01-24T14:13:27.284753+0000","last_clean":"2024-01-24T14:13:27.284753+0000","last_became_active":"2024-01-24T14:13:26.881709+0000","last_became_peered":"2024-01-24T14:13:26.881709+0000","last_unstale":"2024-01-24T14:13:27.284753+0000","last_undegraded":"2024-01-24T14:13:27.284753+0000","last_fullsized":"2024-01-24T14:13:27.284753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297744+0000","last_change":"2024-01-24T14:13:27.295393+0000","last_active":"2024-01-24T14:13:27.297744+0000","last_peered":"2024-01-24T14:13:27.297744+0000","last_clean":"2024-01-24T14:13:27.297744+0000","last_became_active":"2024-01-24T14:13:27.295086+0000","last_became_peered":"2024-01-24T14:13:27.295086+0000","last_unstale":"2024-01-24T14:13:27.297744+0000","last_undegraded":"2024-01-24T14:13:27.297744+0000","last_fullsized":"2024-01-24T14:13:27.297744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2d","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309447+0000","last_change":"2024-01-24T14:13:26.879410+0000","last_active":"2024-01-24T14:13:27.309447+0000","last_peered":"2024-01-24T14:13:27.309447+0000","last_clean":"2024-01-24T14:13:27.309447+0000","last_became_active":"2024-01-24T14:13:26.879111+0000","last_became_peered":"2024-01-24T14:13:26.879111+0000","last_unstale":"2024-01-24T14:13:27.309447+0000","last_undegraded":"2024-01-24T14:13:27.309447+0000","last_fullsized":"2024-01-24T14:13:27.309447+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309379+0000","last_change":"2024-01-24T14:13:27.295752+0000","last_active":"2024-01-24T14:13:27.309379+0000","last_peered":"2024-01-24T14:13:27.309379+0000","last_clean":"2024-01-24T14:13:27.309379+0000","last_became_active":"2024-01-24T14:13:27.295386+0000","last_became_peered":"2024-01-24T14:13:27.295386+0000","last_unstale":"2024-01-24T14:13:27.309379+0000","last_undegraded":"2024-01-24T14:13:27.309379+0000","last_fullsized":"2024-01-24T14:13:27.309379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.282865+0000","last_change":"2024-01-24T14:13:26.886393+0000","last_active":"2024-01-24T14:13:27.282865+0000","last_peered":"2024-01-24T14:13:27.282865+0000","last_clean":"2024-01-24T14:13:27.282865+0000","last_became_active":"2024-01-24T14:13:26.886118+0000","last_became_peered":"2024-01-24T14:13:26.886118+0000","last_unstale":"2024-01-24T14:13:27.282865+0000","last_undegraded":"2024-01-24T14:13:27.282865+0000","last_fullsized":"2024-01-24T14:13:27.282865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296989+0000","last_change":"2024-01-24T14:13:27.286362+0000","last_active":"2024-01-24T14:13:27.296989+0000","last_peered":"2024-01-24T14:13:27.296989+0000","last_clean":"2024-01-24T14:13:27.296989+0000","last_became_active":"2024-01-24T14:13:27.286067+0000","last_became_peered":"2024-01-24T14:13:27.286067+0000","last_unstale":"2024-01-24T14:13:27.296989+0000","last_undegraded":"2024-01-24T14:13:27.296989+0000","last_fullsized":"2024-01-24T14:13:27.296989+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294286+0000","last_change":"2024-01-24T14:13:26.870592+0000","last_active":"2024-01-24T14:13:27.294286+0000","last_peered":"2024-01-24T14:13:27.294286+0000","last_clean":"2024-01-24T14:13:27.294286+0000","last_became_active":"2024-01-24T14:13:26.870307+0000","last_became_peered":"2024-01-24T14:13:26.870307+0000","last_unstale":"2024-01-24T14:13:27.294286+0000","last_undegraded":"2024-01-24T14:13:27.294286+0000","last_fullsized":"2024-01-24T14:13:27.294286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.299203+0000","last_change":"2024-01-24T14:13:27.293871+0000","last_active":"2024-01-24T14:13:27.299203+0000","last_peered":"2024-01-24T14:13:27.299203+0000","last_clean":"2024-01-24T14:13:27.299203+0000","last_became_active":"2024-01-24T14:13:27.291893+0000","last_became_peered":"2024-01-24T14:13:27.291893+0000","last_unstale":"2024-01-24T14:13:27.299203+0000","last_undegraded":"2024-01-24T14:13:27.299203+0000","last_fullsized":"2024-01-24T14:13:27.299203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.286980+0000","last_change":"2024-01-24T14:13:26.879131+0000","last_active":"2024-01-24T14:13:27.286980+0000","last_peered":"2024-01-24T14:13:27.286980+0000","last_clean":"2024-01-24T14:13:27.286980+0000","last_became_active":"2024-01-24T14:13:26.878760+0000","last_became_peered":"2024-01-24T14:13:26.878760+0000","last_unstale":"2024-01-24T14:13:27.286980+0000","last_undegraded":"2024-01-24T14:13:27.286980+0000","last_fullsized":"2024-01-24T14:13:27.286980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.313785+0000","last_change":"2024-01-24T14:13:27.310107+0000","last_active":"2024-01-24T14:13:27.313785+0000","last_peered":"2024-01-24T14:13:27.313785+0000","last_clean":"2024-01-24T14:13:27.313785+0000","last_became_active":"2024-01-24T14:13:27.309809+0000","last_became_peered":"2024-01-24T14:13:27.309809+0000","last_unstale":"2024-01-24T14:13:27.313785+0000","last_undegraded":"2024-01-24T14:13:27.313785+0000","last_fullsized":"2024-01-24T14:13:27.313785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.282909+0000","last_change":"2024-01-24T14:13:26.879546+0000","last_active":"2024-01-24T14:13:27.282909+0000","last_peered":"2024-01-24T14:13:27.282909+0000","last_clean":"2024-01-24T14:13:27.282909+0000","last_became_active":"2024-01-24T14:13:26.878706+0000","last_became_peered":"2024-01-24T14:13:26.878706+0000","last_unstale":"2024-01-24T14:13:27.282909+0000","last_undegraded":"2024-01-24T14:13:27.282909+0000","last_fullsized":"2024-01-24T14:13:27.282909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.307697+0000","last_change":"2024-01-24T14:13:27.304315+0000","last_active":"2024-01-24T14:13:27.307697+0000","last_peered":"2024-01-24T14:13:27.307697+0000","last_clean":"2024-01-24T14:13:27.307697+0000","last_became_active":"2024-01-24T14:13:27.303528+0000","last_became_peered":"2024-01-24T14:13:27.303528+0000","last_unstale":"2024-01-24T14:13:27.307697+0000","last_undegraded":"2024-01-24T14:13:27.307697+0000","last_fullsized":"2024-01-24T14:13:27.307697+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294498+0000","last_change":"2024-01-24T14:13:26.881176+0000","last_active":"2024-01-24T14:13:27.294498+0000","last_peered":"2024-01-24T14:13:27.294498+0000","last_clean":"2024-01-24T14:13:27.294498+0000","last_became_active":"2024-01-24T14:13:26.880825+0000","last_became_peered":"2024-01-24T14:13:26.880825+0000","last_unstale":"2024-01-24T14:13:27.294498+0000","last_undegraded":"2024-01-24T14:13:27.294498+0000","last_fullsized":"2024-01-24T14:13:27.294498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.310807+0000","last_change":"2024-01-24T14:13:27.307524+0000","last_active":"2024-01-24T14:13:27.310807+0000","last_peered":"2024-01-24T14:13:27.310807+0000","last_clean":"2024-01-24T14:13:27.310807+0000","last_became_active":"2024-01-24T14:13:27.307203+0000","last_became_peered":"2024-01-24T14:13:27.307203+0000","last_unstale":"2024-01-24T14:13:27.310807+0000","last_undegraded":"2024-01-24T14:13:27.310807+0000","last_fullsized":"2024-01-24T14:13:27.310807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.281163+0000","last_change":"2024-01-24T14:13:26.873351+0000","last_active":"2024-01-24T14:13:27.281163+0000","last_peered":"2024-01-24T14:13:27.281163+0000","last_clean":"2024-01-24T14:13:27.281163+0000","last_became_active":"2024-01-24T14:13:26.873043+0000","last_became_peered":"2024-01-24T14:13:26.873043+0000","last_unstale":"2024-01-24T14:13:27.281163+0000","last_undegraded":"2024-01-24T14:13:27.281163+0000","last_fullsized":"2024-01-24T14:13:27.281163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.306396+0000","last_change":"2024-01-24T14:13:27.302968+0000","last_active":"2024-01-24T14:13:27.306396+0000","last_peered":"2024-01-24T14:13:27.306396+0000","last_clean":"2024-01-24T14:13:27.306396+0000","last_became_active":"2024-01-24T14:13:27.302631+0000","last_became_peered":"2024-01-24T14:13:27.302631+0000","last_unstale":"2024-01-24T14:13:27.306396+0000","last_undegraded":"2024-01-24T14:13:27.306396+0000","last_fullsized":"2024-01-24T14:13:27.306396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.281815+0000","last_change":"2024-01-24T14:13:26.876101+0000","last_active":"2024-01-24T14:13:27.281815+0000","last_peered":"2024-01-24T14:13:27.281815+0000","last_clean":"2024-01-24T14:13:27.281815+0000","last_became_active":"2024-01-24T14:13:26.875225+0000","last_became_peered":"2024-01-24T14:13:26.875225+0000","last_unstale":"2024-01-24T14:13:27.281815+0000","last_undegraded":"2024-01-24T14:13:27.281815+0000","last_fullsized":"2024-01-24T14:13:27.281815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296952+0000","last_change":"2024-01-24T14:13:27.287754+0000","last_active":"2024-01-24T14:13:27.296952+0000","last_peered":"2024-01-24T14:13:27.296952+0000","last_clean":"2024-01-24T14:13:27.296952+0000","last_became_active":"2024-01-24T14:13:27.287464+0000","last_became_peered":"2024-01-24T14:13:27.287464+0000","last_unstale":"2024-01-24T14:13:27.296952+0000","last_undegraded":"2024-01-24T14:13:27.296952+0000","last_fullsized":"2024-01-24T14:13:27.296952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.282178+0000","last_change":"2024-01-24T14:13:26.876609+0000","last_active":"2024-01-24T14:13:27.282178+0000","last_peered":"2024-01-24T14:13:27.282178+0000","last_clean":"2024-01-24T14:13:27.282178+0000","last_became_active":"2024-01-24T14:13:26.876183+0000","last_became_peered":"2024-01-24T14:13:26.876183+0000","last_unstale":"2024-01-24T14:13:27.282178+0000","last_undegraded":"2024-01-24T14:13:27.282178+0000","last_fullsized":"2024-01-24T14:13:27.282178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.304540+0000","last_change":"2024-01-24T14:13:27.302578+0000","last_active":"2024-01-24T14:13:27.304540+0000","last_peered":"2024-01-24T14:13:27.304540+0000","last_clean":"2024-01-24T14:13:27.304540+0000","last_became_active":"2024-01-24T14:13:27.302240+0000","last_became_peered":"2024-01-24T14:13:27.302240+0000","last_unstale":"2024-01-24T14:13:27.304540+0000","last_undegraded":"2024-01-24T14:13:27.304540+0000","last_fullsized":"2024-01-24T14:13:27.304540+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.288042+0000","last_change":"2024-01-24T14:13:26.887792+0000","last_active":"2024-01-24T14:13:27.288042+0000","last_peered":"2024-01-24T14:13:27.288042+0000","last_clean":"2024-01-24T14:13:27.288042+0000","last_became_active":"2024-01-24T14:13:26.887467+0000","last_became_peered":"2024-01-24T14:13:26.887467+0000","last_unstale":"2024-01-24T14:13:27.288042+0000","last_undegraded":"2024-01-24T14:13:27.288042+0000","last_fullsized":"2024-01-24T14:13:27.288042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.310856+0000","last_change":"2024-01-24T14:13:27.307430+0000","last_active":"2024-01-24T14:13:27.310856+0000","last_peered":"2024-01-24T14:13:27.310856+0000","last_clean":"2024-01-24T14:13:27.310856+0000","last_became_active":"2024-01-24T14:13:27.307133+0000","last_became_peered":"2024-01-24T14:13:27.307133+0000","last_unstale":"2024-01-24T14:13:27.310856+0000","last_undegraded":"2024-01-24T14:13:27.310856+0000","last_fullsized":"2024-01-24T14:13:27.310856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297267+0000","last_change":"2024-01-24T14:13:26.878331+0000","last_active":"2024-01-24T14:13:27.297267+0000","last_peered":"2024-01-24T14:13:27.297267+0000","last_clean":"2024-01-24T14:13:27.297267+0000","last_became_active":"2024-01-24T14:13:26.878088+0000","last_became_peered":"2024-01-24T14:13:26.878088+0000","last_unstale":"2024-01-24T14:13:27.297267+0000","last_undegraded":"2024-01-24T14:13:27.297267+0000","last_fullsized":"2024-01-24T14:13:27.297267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.301882+0000","last_change":"2024-01-24T14:13:27.294844+0000","last_active":"2024-01-24T14:13:27.301882+0000","last_peered":"2024-01-24T14:13:27.301882+0000","last_clean":"2024-01-24T14:13:27.301882+0000","last_became_active":"2024-01-24T14:13:27.294558+0000","last_became_peered":"2024-01-24T14:13:27.294558+0000","last_unstale":"2024-01-24T14:13:27.301882+0000","last_undegraded":"2024-01-24T14:13:27.301882+0000","last_fullsized":"2024-01-24T14:13:27.301882+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.283980+0000","last_change":"2024-01-24T14:13:26.874664+0000","last_active":"2024-01-24T14:13:27.283980+0000","last_peered":"2024-01-24T14:13:27.283980+0000","last_clean":"2024-01-24T14:13:27.283980+0000","last_became_active":"2024-01-24T14:13:26.874341+0000","last_became_peered":"2024-01-24T14:13:26.874341+0000","last_unstale":"2024-01-24T14:13:27.283980+0000","last_undegraded":"2024-01-24T14:13:27.283980+0000","last_fullsized":"2024-01-24T14:13:27.283980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311838+0000","last_change":"2024-01-24T14:13:27.305933+0000","last_active":"2024-01-24T14:13:27.311838+0000","last_peered":"2024-01-24T14:13:27.311838+0000","last_clean":"2024-01-24T14:13:27.311838+0000","last_became_active":"2024-01-24T14:13:27.305404+0000","last_became_peered":"2024-01-24T14:13:27.305404+0000","last_unstale":"2024-01-24T14:13:27.311838+0000","last_undegraded":"2024-01-24T14:13:27.311838+0000","last_fullsized":"2024-01-24T14:13:27.311838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.280388+0000","last_change":"2024-01-24T14:13:26.875765+0000","last_active":"2024-01-24T14:13:27.280388+0000","last_peered":"2024-01-24T14:13:27.280388+0000","last_clean":"2024-01-24T14:13:27.280388+0000","last_became_active":"2024-01-24T14:13:26.875459+0000","last_became_peered":"2024-01-24T14:13:26.875459+0000","last_unstale":"2024-01-24T14:13:27.280388+0000","last_undegraded":"2024-01-24T14:13:27.280388+0000","last_fullsized":"2024-01-24T14:13:27.280388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294465+0000","last_change":"2024-01-24T14:13:27.291235+0000","last_active":"2024-01-24T14:13:27.294465+0000","last_peered":"2024-01-24T14:13:27.294465+0000","last_clean":"2024-01-24T14:13:27.294465+0000","last_became_active":"2024-01-24T14:13:27.290611+0000","last_became_peered":"2024-01-24T14:13:27.290611+0000","last_unstale":"2024-01-24T14:13:27.294465+0000","last_undegraded":"2024-01-24T14:13:27.294465+0000","last_fullsized":"2024-01-24T14:13:27.294465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.287812+0000","last_change":"2024-01-24T14:13:26.877427+0000","last_active":"2024-01-24T14:13:27.287812+0000","last_peered":"2024-01-24T14:13:27.287812+0000","last_clean":"2024-01-24T14:13:27.287812+0000","last_became_active":"2024-01-24T14:13:26.876994+0000","last_became_peered":"2024-01-24T14:13:26.876994+0000","last_unstale":"2024-01-24T14:13:27.287812+0000","last_undegraded":"2024-01-24T14:13:27.287812+0000","last_fullsized":"2024-01-24T14:13:27.287812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311381+0000","last_change":"2024-01-24T14:13:27.310488+0000","last_active":"2024-01-24T14:13:27.311381+0000","last_peered":"2024-01-24T14:13:27.311381+0000","last_clean":"2024-01-24T14:13:27.311381+0000","last_became_active":"2024-01-24T14:13:27.310104+0000","last_became_peered":"2024-01-24T14:13:27.310104+0000","last_unstale":"2024-01-24T14:13:27.311381+0000","last_undegraded":"2024-01-24T14:13:27.311381+0000","last_fullsized":"2024-01-24T14:13:27.311381+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.308731+0000","last_change":"2024-01-24T14:13:26.881790+0000","last_active":"2024-01-24T14:13:27.308731+0000","last_peered":"2024-01-24T14:13:27.308731+0000","last_clean":"2024-01-24T14:13:27.308731+0000","last_became_active":"2024-01-24T14:13:26.880948+0000","last_became_peered":"2024-01-24T14:13:26.880948+0000","last_unstale":"2024-01-24T14:13:27.308731+0000","last_undegraded":"2024-01-24T14:13:27.308731+0000","last_fullsized":"2024-01-24T14:13:27.308731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.304843+0000","last_change":"2024-01-24T14:13:27.299897+0000","last_active":"2024-01-24T14:13:27.304843+0000","last_peered":"2024-01-24T14:13:27.304843+0000","last_clean":"2024-01-24T14:13:27.304843+0000","last_became_active":"2024-01-24T14:13:27.299523+0000","last_became_peered":"2024-01-24T14:13:27.299523+0000","last_unstale":"2024-01-24T14:13:27.304843+0000","last_undegraded":"2024-01-24T14:13:27.304843+0000","last_fullsized":"2024-01-24T14:13:27.304843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294537+0000","last_change":"2024-01-24T14:13:26.875673+0000","last_active":"2024-01-24T14:13:27.294537+0000","last_peered":"2024-01-24T14:13:27.294537+0000","last_clean":"2024-01-24T14:13:27.294537+0000","last_became_active":"2024-01-24T14:13:26.875271+0000","last_became_peered":"2024-01-24T14:13:26.875271+0000","last_unstale":"2024-01-24T14:13:27.294537+0000","last_undegraded":"2024-01-24T14:13:27.294537+0000","last_fullsized":"2024-01-24T14:13:27.294537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309595+0000","last_change":"2024-01-24T14:13:27.301977+0000","last_active":"2024-01-24T14:13:27.309595+0000","last_peered":"2024-01-24T14:13:27.309595+0000","last_clean":"2024-01-24T14:13:27.309595+0000","last_became_active":"2024-01-24T14:13:27.301693+0000","last_became_peered":"2024-01-24T14:13:27.301693+0000","last_unstale":"2024-01-24T14:13:27.309595+0000","last_undegraded":"2024-01-24T14:13:27.309595+0000","last_fullsized":"2024-01-24T14:13:27.309595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297016+0000","last_change":"2024-01-24T14:13:26.886887+0000","last_active":"2024-01-24T14:13:27.297016+0000","last_peered":"2024-01-24T14:13:27.297016+0000","last_clean":"2024-01-24T14:13:27.297016+0000","last_became_active":"2024-01-24T14:13:26.886589+0000","last_became_peered":"2024-01-24T14:13:26.886589+0000","last_unstale":"2024-01-24T14:13:27.297016+0000","last_undegraded":"2024-01-24T14:13:27.297016+0000","last_fullsized":"2024-01-24T14:13:27.297016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.310490+0000","last_change":"2024-01-24T14:13:27.306113+0000","last_active":"2024-01-24T14:13:27.310490+0000","last_peered":"2024-01-24T14:13:27.310490+0000","last_clean":"2024-01-24T14:13:27.310490+0000","last_became_active":"2024-01-24T14:13:27.305688+0000","last_became_peered":"2024-01-24T14:13:27.305688+0000","last_unstale":"2024-01-24T14:13:27.310490+0000","last_undegraded":"2024-01-24T14:13:27.310490+0000","last_fullsized":"2024-01-24T14:13:27.310490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.290983+0000","last_change":"2024-01-24T14:13:26.887178+0000","last_active":"2024-01-24T14:13:27.290983+0000","last_peered":"2024-01-24T14:13:27.290983+0000","last_clean":"2024-01-24T14:13:27.290983+0000","last_became_active":"2024-01-24T14:13:26.886907+0000","last_became_peered":"2024-01-24T14:13:26.886907+0000","last_unstale":"2024-01-24T14:13:27.290983+0000","last_undegraded":"2024-01-24T14:13:27.290983+0000","last_fullsized":"2024-01-24T14:13:27.290983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.304506+0000","last_change":"2024-01-24T14:13:27.298383+0000","last_active":"2024-01-24T14:13:27.304506+0000","last_peered":"2024-01-24T14:13:27.304506+0000","last_clean":"2024-01-24T14:13:27.304506+0000","last_became_active":"2024-01-24T14:13:27.298054+0000","last_became_peered":"2024-01-24T14:13:27.298054+0000","last_unstale":"2024-01-24T14:13:27.304506+0000","last_undegraded":"2024-01-24T14:13:27.304506+0000","last_fullsized":"2024-01-24T14:13:27.304506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.282888+0000","last_change":"2024-01-24T14:13:26.874605+0000","last_active":"2024-01-24T14:13:27.282888+0000","last_peered":"2024-01-24T14:13:27.282888+0000","last_clean":"2024-01-24T14:13:27.282888+0000","last_became_active":"2024-01-24T14:13:26.874252+0000","last_became_peered":"2024-01-24T14:13:26.874252+0000","last_unstale":"2024-01-24T14:13:27.282888+0000","last_undegraded":"2024-01-24T14:13:27.282888+0000","last_fullsized":"2024-01-24T14:13:27.282888+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.306417+0000","last_change":"2024-01-24T14:13:27.300564+0000","last_active":"2024-01-24T14:13:27.306417+0000","last_peered":"2024-01-24T14:13:27.306417+0000","last_clean":"2024-01-24T14:13:27.306417+0000","last_became_active":"2024-01-24T14:13:27.300246+0000","last_became_peered":"2024-01-24T14:13:27.300246+0000","last_unstale":"2024-01-24T14:13:27.306417+0000","last_undegraded":"2024-01-24T14:13:27.306417+0000","last_fullsized":"2024-01-24T14:13:27.306417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291350+0000","last_change":"2024-01-24T14:13:26.886129+0000","last_active":"2024-01-24T14:13:27.291350+0000","last_peered":"2024-01-24T14:13:27.291350+0000","last_clean":"2024-01-24T14:13:27.291350+0000","last_became_active":"2024-01-24T14:13:26.885874+0000","last_became_peered":"2024-01-24T14:13:26.885874+0000","last_unstale":"2024-01-24T14:13:27.291350+0000","last_undegraded":"2024-01-24T14:13:27.291350+0000","last_fullsized":"2024-01-24T14:13:27.291350+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.286694+0000","last_change":"2024-01-24T14:13:27.282826+0000","last_active":"2024-01-24T14:13:27.286694+0000","last_peered":"2024-01-24T14:13:27.286694+0000","last_clean":"2024-01-24T14:13:27.286694+0000","last_became_active":"2024-01-24T14:13:27.282543+0000","last_became_peered":"2024-01-24T14:13:27.282543+0000","last_unstale":"2024-01-24T14:13:27.286694+0000","last_undegraded":"2024-01-24T14:13:27.286694+0000","last_fullsized":"2024-01-24T14:13:27.286694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.287597+0000","last_change":"2024-01-24T14:13:26.870366+0000","last_active":"2024-01-24T14:13:27.287597+0000","last_peered":"2024-01-24T14:13:27.287597+0000","last_clean":"2024-01-24T14:13:27.287597+0000","last_became_active":"2024-01-24T14:13:26.870053+0000","last_became_peered":"2024-01-24T14:13:26.870053+0000","last_unstale":"2024-01-24T14:13:27.287597+0000","last_undegraded":"2024-01-24T14:13:27.287597+0000","last_fullsized":"2024-01-24T14:13:27.287597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.302760+0000","last_change":"2024-01-24T14:13:27.300752+0000","last_active":"2024-01-24T14:13:27.302760+0000","last_peered":"2024-01-24T14:13:27.302760+0000","last_clean":"2024-01-24T14:13:27.302760+0000","last_became_active":"2024-01-24T14:13:27.300470+0000","last_became_peered":"2024-01-24T14:13:27.300470+0000","last_unstale":"2024-01-24T14:13:27.302760+0000","last_undegraded":"2024-01-24T14:13:27.302760+0000","last_fullsized":"2024-01-24T14:13:27.302760+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.275185+0000","last_change":"2024-01-24T14:13:26.880676+0000","last_active":"2024-01-24T14:13:27.275185+0000","last_peered":"2024-01-24T14:13:27.275185+0000","last_clean":"2024-01-24T14:13:27.275185+0000","last_became_active":"2024-01-24T14:13:26.880329+0000","last_became_peered":"2024-01-24T14:13:26.880329+0000","last_unstale":"2024-01-24T14:13:27.275185+0000","last_undegraded":"2024-01-24T14:13:27.275185+0000","last_fullsized":"2024-01-24T14:13:27.275185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.304335+0000","last_change":"2024-01-24T14:13:27.301690+0000","last_active":"2024-01-24T14:13:27.304335+0000","last_peered":"2024-01-24T14:13:27.304335+0000","last_clean":"2024-01-24T14:13:27.304335+0000","last_became_active":"2024-01-24T14:13:27.301406+0000","last_became_peered":"2024-01-24T14:13:27.301406+0000","last_unstale":"2024-01-24T14:13:27.304335+0000","last_undegraded":"2024-01-24T14:13:27.304335+0000","last_fullsized":"2024-01-24T14:13:27.304335+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.284863+0000","last_change":"2024-01-24T14:13:26.875798+0000","last_active":"2024-01-24T14:13:27.284863+0000","last_peered":"2024-01-24T14:13:27.284863+0000","last_clean":"2024-01-24T14:13:27.284863+0000","last_became_active":"2024-01-24T14:13:26.875521+0000","last_became_peered":"2024-01-24T14:13:26.875521+0000","last_unstale":"2024-01-24T14:13:27.284863+0000","last_undegraded":"2024-01-24T14:13:27.284863+0000","last_fullsized":"2024-01-24T14:13:27.284863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311564+0000","last_change":"2024-01-24T14:13:27.310241+0000","last_active":"2024-01-24T14:13:27.311564+0000","last_peered":"2024-01-24T14:13:27.311564+0000","last_clean":"2024-01-24T14:13:27.311564+0000","last_became_active":"2024-01-24T14:13:27.309931+0000","last_became_peered":"2024-01-24T14:13:27.309931+0000","last_unstale":"2024-01-24T14:13:27.311564+0000","last_undegraded":"2024-01-24T14:13:27.311564+0000","last_fullsized":"2024-01-24T14:13:27.311564+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294670+0000","last_change":"2024-01-24T14:13:26.886368+0000","last_active":"2024-01-24T14:13:27.294670+0000","last_peered":"2024-01-24T14:13:27.294670+0000","last_clean":"2024-01-24T14:13:27.294670+0000","last_became_active":"2024-01-24T14:13:26.886063+0000","last_became_peered":"2024-01-24T14:13:26.886063+0000","last_unstale":"2024-01-24T14:13:27.294670+0000","last_undegraded":"2024-01-24T14:13:27.294670+0000","last_fullsized":"2024-01-24T14:13:27.294670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311915+0000","last_change":"2024-01-24T14:13:27.307211+0000","last_active":"2024-01-24T14:13:27.311915+0000","last_peered":"2024-01-24T14:13:27.311915+0000","last_clean":"2024-01-24T14:13:27.311915+0000","last_became_active":"2024-01-24T14:13:27.306472+0000","last_became_peered":"2024-01-24T14:13:27.306472+0000","last_unstale":"2024-01-24T14:13:27.311915+0000","last_undegraded":"2024-01-24T14:13:27.311915+0000","last_fullsized":"2024-01-24T14:13:27.311915+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"89'3","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.299541+0000","last_change":"2024-01-24T14:13:26.874205+0000","last_active":"2024-01-24T14:13:27.299541+0000","last_peered":"2024-01-24T14:13:27.299541+0000","last_clean":"2024-01-24T14:13:27.299541+0000","last_became_active":"2024-01-24T14:13:26.873885+0000","last_became_peered":"2024-01-24T14:13:26.873885+0000","last_unstale":"2024-01-24T14:13:27.299541+0000","last_undegraded":"2024-01-24T14:13:27.299541+0000","last_fullsized":"2024-01-24T14:13:27.299541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.292952+0000","last_change":"2024-01-24T14:13:27.291406+0000","last_active":"2024-01-24T14:13:27.292952+0000","last_peered":"2024-01-24T14:13:27.292952+0000","last_clean":"2024-01-24T14:13:27.292952+0000","last_became_active":"2024-01-24T14:13:27.291134+0000","last_became_peered":"2024-01-24T14:13:27.291134+0000","last_unstale":"2024-01-24T14:13:27.292952+0000","last_undegraded":"2024-01-24T14:13:27.292952+0000","last_fullsized":"2024-01-24T14:13:27.292952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.306312+0000","last_change":"2024-01-24T14:13:26.876317+0000","last_active":"2024-01-24T14:13:27.306312+0000","last_peered":"2024-01-24T14:13:27.306312+0000","last_clean":"2024-01-24T14:13:27.306312+0000","last_became_active":"2024-01-24T14:13:26.876022+0000","last_became_peered":"2024-01-24T14:13:26.876022+0000","last_unstale":"2024-01-24T14:13:27.306312+0000","last_undegraded":"2024-01-24T14:13:27.306312+0000","last_fullsized":"2024-01-24T14:13:27.306312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.307861+0000","last_change":"2024-01-24T14:13:27.302963+0000","last_active":"2024-01-24T14:13:27.307861+0000","last_peered":"2024-01-24T14:13:27.307861+0000","last_clean":"2024-01-24T14:13:27.307861+0000","last_became_active":"2024-01-24T14:13:27.302657+0000","last_became_peered":"2024-01-24T14:13:27.302657+0000","last_unstale":"2024-01-24T14:13:27.307861+0000","last_undegraded":"2024-01-24T14:13:27.307861+0000","last_fullsized":"2024-01-24T14:13:27.307861+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.305091+0000","last_change":"2024-01-24T14:13:26.881920+0000","last_active":"2024-01-24T14:13:27.305091+0000","last_peered":"2024-01-24T14:13:27.305091+0000","last_clean":"2024-01-24T14:13:27.305091+0000","last_became_active":"2024-01-24T14:13:26.881589+0000","last_became_peered":"2024-01-24T14:13:26.881589+0000","last_unstale":"2024-01-24T14:13:27.305091+0000","last_undegraded":"2024-01-24T14:13:27.305091+0000","last_fullsized":"2024-01-24T14:13:27.305091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294376+0000","last_change":"2024-01-24T14:13:27.290720+0000","last_active":"2024-01-24T14:13:27.294376+0000","last_peered":"2024-01-24T14:13:27.294376+0000","last_clean":"2024-01-24T14:13:27.294376+0000","last_became_active":"2024-01-24T14:13:27.290431+0000","last_became_peered":"2024-01-24T14:13:27.290431+0000","last_unstale":"2024-01-24T14:13:27.294376+0000","last_undegraded":"2024-01-24T14:13:27.294376+0000","last_fullsized":"2024-01-24T14:13:27.294376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291466+0000","last_change":"2024-01-24T14:13:26.879811+0000","last_active":"2024-01-24T14:13:27.291466+0000","last_peered":"2024-01-24T14:13:27.291466+0000","last_clean":"2024-01-24T14:13:27.291466+0000","last_became_active":"2024-01-24T14:13:26.879552+0000","last_became_peered":"2024-01-24T14:13:26.879552+0000","last_unstale":"2024-01-24T14:13:27.291466+0000","last_undegraded":"2024-01-24T14:13:27.291466+0000","last_fullsized":"2024-01-24T14:13:27.291466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.301595+0000","last_change":"2024-01-24T14:13:27.297557+0000","last_active":"2024-01-24T14:13:27.301595+0000","last_peered":"2024-01-24T14:13:27.301595+0000","last_clean":"2024-01-24T14:13:27.301595+0000","last_became_active":"2024-01-24T14:13:27.297259+0000","last_became_peered":"2024-01-24T14:13:27.297259+0000","last_unstale":"2024-01-24T14:13:27.301595+0000","last_undegraded":"2024-01-24T14:13:27.301595+0000","last_fullsized":"2024-01-24T14:13:27.301595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.280052+0000","last_change":"2024-01-24T14:13:26.875603+0000","last_active":"2024-01-24T14:13:27.280052+0000","last_peered":"2024-01-24T14:13:27.280052+0000","last_clean":"2024-01-24T14:13:27.280052+0000","last_became_active":"2024-01-24T14:13:26.875275+0000","last_became_peered":"2024-01-24T14:13:26.875275+0000","last_unstale":"2024-01-24T14:13:27.280052+0000","last_undegraded":"2024-01-24T14:13:27.280052+0000","last_fullsized":"2024-01-24T14:13:27.280052+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297779+0000","last_change":"2024-01-24T14:13:27.296404+0000","last_active":"2024-01-24T14:13:27.297779+0000","last_peered":"2024-01-24T14:13:27.297779+0000","last_clean":"2024-01-24T14:13:27.297779+0000","last_became_active":"2024-01-24T14:13:27.296065+0000","last_became_peered":"2024-01-24T14:13:27.296065+0000","last_unstale":"2024-01-24T14:13:27.297779+0000","last_undegraded":"2024-01-24T14:13:27.297779+0000","last_fullsized":"2024-01-24T14:13:27.297779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"89'4","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297121+0000","last_change":"2024-01-24T14:13:26.879963+0000","last_active":"2024-01-24T14:13:27.297121+0000","last_peered":"2024-01-24T14:13:27.297121+0000","last_clean":"2024-01-24T14:13:27.297121+0000","last_became_active":"2024-01-24T14:13:26.877231+0000","last_became_peered":"2024-01-24T14:13:26.877231+0000","last_unstale":"2024-01-24T14:13:27.297121+0000","last_undegraded":"2024-01-24T14:13:27.297121+0000","last_fullsized":"2024-01-24T14:13:27.297121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296451+0000","last_change":"2024-01-24T14:13:27.291898+0000","last_active":"2024-01-24T14:13:27.296451+0000","last_peered":"2024-01-24T14:13:27.296451+0000","last_clean":"2024-01-24T14:13:27.296451+0000","last_became_active":"2024-01-24T14:13:27.291619+0000","last_became_peered":"2024-01-24T14:13:27.291619+0000","last_unstale":"2024-01-24T14:13:27.296451+0000","last_undegraded":"2024-01-24T14:13:27.296451+0000","last_fullsized":"2024-01-24T14:13:27.296451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309294+0000","last_change":"2024-01-24T14:13:26.885373+0000","last_active":"2024-01-24T14:13:27.309294+0000","last_peered":"2024-01-24T14:13:27.309294+0000","last_clean":"2024-01-24T14:13:27.309294+0000","last_became_active":"2024-01-24T14:13:26.884871+0000","last_became_peered":"2024-01-24T14:13:26.884871+0000","last_unstale":"2024-01-24T14:13:27.309294+0000","last_undegraded":"2024-01-24T14:13:27.309294+0000","last_fullsized":"2024-01-24T14:13:27.309294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294474+0000","last_change":"2024-01-24T14:13:27.291973+0000","last_active":"2024-01-24T14:13:27.294474+0000","last_peered":"2024-01-24T14:13:27.294474+0000","last_clean":"2024-01-24T14:13:27.294474+0000","last_became_active":"2024-01-24T14:13:27.291681+0000","last_became_peered":"2024-01-24T14:13:27.291681+0000","last_unstale":"2024-01-24T14:13:27.294474+0000","last_undegraded":"2024-01-24T14:13:27.294474+0000","last_fullsized":"2024-01-24T14:13:27.294474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.281950+0000","last_change":"2024-01-24T14:13:26.882217+0000","last_active":"2024-01-24T14:13:27.281950+0000","last_peered":"2024-01-24T14:13:27.281950+0000","last_clean":"2024-01-24T14:13:27.281950+0000","last_became_active":"2024-01-24T14:13:26.881903+0000","last_became_peered":"2024-01-24T14:13:26.881903+0000","last_unstale":"2024-01-24T14:13:27.281950+0000","last_undegraded":"2024-01-24T14:13:27.281950+0000","last_fullsized":"2024-01-24T14:13:27.281950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311402+0000","last_change":"2024-01-24T14:13:27.310356+0000","last_active":"2024-01-24T14:13:27.311402+0000","last_peered":"2024-01-24T14:13:27.311402+0000","last_clean":"2024-01-24T14:13:27.311402+0000","last_became_active":"2024-01-24T14:13:27.309988+0000","last_became_peered":"2024-01-24T14:13:27.309988+0000","last_unstale":"2024-01-24T14:13:27.311402+0000","last_undegraded":"2024-01-24T14:13:27.311402+0000","last_fullsized":"2024-01-24T14:13:27.311402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.277450+0000","last_change":"2024-01-24T14:13:26.877334+0000","last_active":"2024-01-24T14:13:27.277450+0000","last_peered":"2024-01-24T14:13:27.277450+0000","last_clean":"2024-01-24T14:13:27.277450+0000","last_became_active":"2024-01-24T14:13:26.876747+0000","last_became_peered":"2024-01-24T14:13:26.876747+0000","last_unstale":"2024-01-24T14:13:27.277450+0000","last_undegraded":"2024-01-24T14:13:27.277450+0000","last_fullsized":"2024-01-24T14:13:27.277450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.310472+0000","last_change":"2024-01-24T14:13:27.305953+0000","last_active":"2024-01-24T14:13:27.310472+0000","last_peered":"2024-01-24T14:13:27.310472+0000","last_clean":"2024-01-24T14:13:27.310472+0000","last_became_active":"2024-01-24T14:13:27.305324+0000","last_became_peered":"2024-01-24T14:13:27.305324+0000","last_unstale":"2024-01-24T14:13:27.310472+0000","last_undegraded":"2024-01-24T14:13:27.310472+0000","last_fullsized":"2024-01-24T14:13:27.310472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.288116+0000","last_change":"2024-01-24T14:13:26.876390+0000","last_active":"2024-01-24T14:13:27.288116+0000","last_peered":"2024-01-24T14:13:27.288116+0000","last_clean":"2024-01-24T14:13:27.288116+0000","last_became_active":"2024-01-24T14:13:26.876051+0000","last_became_peered":"2024-01-24T14:13:26.876051+0000","last_unstale":"2024-01-24T14:13:27.288116+0000","last_undegraded":"2024-01-24T14:13:27.288116+0000","last_fullsized":"2024-01-24T14:13:27.288116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.303823+0000","last_change":"2024-01-24T14:13:27.303021+0000","last_active":"2024-01-24T14:13:27.303823+0000","last_peered":"2024-01-24T14:13:27.303823+0000","last_clean":"2024-01-24T14:13:27.303823+0000","last_became_active":"2024-01-24T14:13:27.302612+0000","last_became_peered":"2024-01-24T14:13:27.302612+0000","last_unstale":"2024-01-24T14:13:27.303823+0000","last_undegraded":"2024-01-24T14:13:27.303823+0000","last_fullsized":"2024-01-24T14:13:27.303823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.276291+0000","last_change":"2024-01-24T14:13:26.862510+0000","last_active":"2024-01-24T14:13:27.276291+0000","last_peered":"2024-01-24T14:13:27.276291+0000","last_clean":"2024-01-24T14:13:27.276291+0000","last_became_active":"2024-01-24T14:13:26.862021+0000","last_became_peered":"2024-01-24T14:13:26.862021+0000","last_unstale":"2024-01-24T14:13:27.276291+0000","last_undegraded":"2024-01-24T14:13:27.276291+0000","last_fullsized":"2024-01-24T14:13:27.276291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309470+0000","last_change":"2024-01-24T14:13:27.301052+0000","last_active":"2024-01-24T14:13:27.309470+0000","last_peered":"2024-01-24T14:13:27.309470+0000","last_clean":"2024-01-24T14:13:27.309470+0000","last_became_active":"2024-01-24T14:13:27.300765+0000","last_became_peered":"2024-01-24T14:13:27.300765+0000","last_unstale":"2024-01-24T14:13:27.309470+0000","last_undegraded":"2024-01-24T14:13:27.309470+0000","last_fullsized":"2024-01-24T14:13:27.309470+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.287494+0000","last_change":"2024-01-24T14:13:26.869656+0000","last_active":"2024-01-24T14:13:27.287494+0000","last_peered":"2024-01-24T14:13:27.287494+0000","last_clean":"2024-01-24T14:13:27.287494+0000","last_became_active":"2024-01-24T14:13:26.869313+0000","last_became_peered":"2024-01-24T14:13:26.869313+0000","last_unstale":"2024-01-24T14:13:27.287494+0000","last_undegraded":"2024-01-24T14:13:27.287494+0000","last_fullsized":"2024-01-24T14:13:27.287494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.305568+0000","last_change":"2024-01-24T14:13:27.303376+0000","last_active":"2024-01-24T14:13:27.305568+0000","last_peered":"2024-01-24T14:13:27.305568+0000","last_clean":"2024-01-24T14:13:27.305568+0000","last_became_active":"2024-01-24T14:13:27.303116+0000","last_became_peered":"2024-01-24T14:13:27.303116+0000","last_unstale":"2024-01-24T14:13:27.305568+0000","last_undegraded":"2024-01-24T14:13:27.305568+0000","last_fullsized":"2024-01-24T14:13:27.305568+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.280518+0000","last_change":"2024-01-24T14:13:26.878593+0000","last_active":"2024-01-24T14:13:27.280518+0000","last_peered":"2024-01-24T14:13:27.280518+0000","last_clean":"2024-01-24T14:13:27.280518+0000","last_became_active":"2024-01-24T14:13:26.878317+0000","last_became_peered":"2024-01-24T14:13:26.878317+0000","last_unstale":"2024-01-24T14:13:27.280518+0000","last_undegraded":"2024-01-24T14:13:27.280518+0000","last_fullsized":"2024-01-24T14:13:27.280518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311871+0000","last_change":"2024-01-24T14:13:27.305601+0000","last_active":"2024-01-24T14:13:27.311871+0000","last_peered":"2024-01-24T14:13:27.311871+0000","last_clean":"2024-01-24T14:13:27.311871+0000","last_became_active":"2024-01-24T14:13:27.305296+0000","last_became_peered":"2024-01-24T14:13:27.305296+0000","last_unstale":"2024-01-24T14:13:27.311871+0000","last_undegraded":"2024-01-24T14:13:27.311871+0000","last_fullsized":"2024-01-24T14:13:27.311871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291589+0000","last_change":"2024-01-24T14:13:26.883714+0000","last_active":"2024-01-24T14:13:27.291589+0000","last_peered":"2024-01-24T14:13:27.291589+0000","last_clean":"2024-01-24T14:13:27.291589+0000","last_became_active":"2024-01-24T14:13:26.880281+0000","last_became_peered":"2024-01-24T14:13:26.880281+0000","last_unstale":"2024-01-24T14:13:27.291589+0000","last_undegraded":"2024-01-24T14:13:27.291589+0000","last_fullsized":"2024-01-24T14:13:27.291589+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.286956+0000","last_change":"2024-01-24T14:13:27.284503+0000","last_active":"2024-01-24T14:13:27.286956+0000","last_peered":"2024-01-24T14:13:27.286956+0000","last_clean":"2024-01-24T14:13:27.286956+0000","last_became_active":"2024-01-24T14:13:27.284214+0000","last_became_peered":"2024-01-24T14:13:27.284214+0000","last_unstale":"2024-01-24T14:13:27.286956+0000","last_undegraded":"2024-01-24T14:13:27.286956+0000","last_fullsized":"2024-01-24T14:13:27.286956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296289+0000","last_change":"2024-01-24T14:13:26.878667+0000","last_active":"2024-01-24T14:13:27.296289+0000","last_peered":"2024-01-24T14:13:27.296289+0000","last_clean":"2024-01-24T14:13:27.296289+0000","last_became_active":"2024-01-24T14:13:26.878027+0000","last_became_peered":"2024-01-24T14:13:26.878027+0000","last_unstale":"2024-01-24T14:13:27.296289+0000","last_undegraded":"2024-01-24T14:13:27.296289+0000","last_fullsized":"2024-01-24T14:13:27.296289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.288182+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:13:27.288182+0000","last_peered":"2024-01-24T14:13:27.288182+0000","last_clean":"2024-01-24T14:13:27.288182+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:13:27.288182+0000","last_undegraded":"2024-01-24T14:13:27.288182+0000","last_fullsized":"2024-01-24T14:13:27.288182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.301860+0000","last_change":"2024-01-24T14:13:27.293092+0000","last_active":"2024-01-24T14:13:27.301860+0000","last_peered":"2024-01-24T14:13:27.301860+0000","last_clean":"2024-01-24T14:13:27.301860+0000","last_became_active":"2024-01-24T14:13:27.292806+0000","last_became_peered":"2024-01-24T14:13:27.292806+0000","last_unstale":"2024-01-24T14:13:27.301860+0000","last_undegraded":"2024-01-24T14:13:27.301860+0000","last_fullsized":"2024-01-24T14:13:27.301860+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291588+0000","last_change":"2024-01-24T14:13:26.879405+0000","last_active":"2024-01-24T14:13:27.291588+0000","last_peered":"2024-01-24T14:13:27.291588+0000","last_clean":"2024-01-24T14:13:27.291588+0000","last_became_active":"2024-01-24T14:13:26.879151+0000","last_became_peered":"2024-01-24T14:13:26.879151+0000","last_unstale":"2024-01-24T14:13:27.291588+0000","last_undegraded":"2024-01-24T14:13:27.291588+0000","last_fullsized":"2024-01-24T14:13:27.291588+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.284012+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:13:27.284012+0000","last_peered":"2024-01-24T14:13:27.284012+0000","last_clean":"2024-01-24T14:13:27.284012+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:13:27.284012+0000","last_undegraded":"2024-01-24T14:13:27.284012+0000","last_fullsized":"2024-01-24T14:13:27.284012+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.304515+0000","last_change":"2024-01-24T14:13:27.302815+0000","last_active":"2024-01-24T14:13:27.304515+0000","last_peered":"2024-01-24T14:13:27.304515+0000","last_clean":"2024-01-24T14:13:27.304515+0000","last_became_active":"2024-01-24T14:13:27.302518+0000","last_became_peered":"2024-01-24T14:13:27.302518+0000","last_unstale":"2024-01-24T14:13:27.304515+0000","last_undegraded":"2024-01-24T14:13:27.304515+0000","last_fullsized":"2024-01-24T14:13:27.304515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.280170+0000","last_change":"2024-01-24T14:13:26.873558+0000","last_active":"2024-01-24T14:13:27.280170+0000","last_peered":"2024-01-24T14:13:27.280170+0000","last_clean":"2024-01-24T14:13:27.280170+0000","last_became_active":"2024-01-24T14:13:26.873317+0000","last_became_peered":"2024-01-24T14:13:26.873317+0000","last_unstale":"2024-01-24T14:13:27.280170+0000","last_undegraded":"2024-01-24T14:13:27.280170+0000","last_fullsized":"2024-01-24T14:13:27.280170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.274928+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:13:27.274928+0000","last_peered":"2024-01-24T14:13:27.274928+0000","last_clean":"2024-01-24T14:13:27.274928+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:13:27.274928+0000","last_undegraded":"2024-01-24T14:13:27.274928+0000","last_fullsized":"2024-01-24T14:13:27.274928+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.308693+0000","last_change":"2024-01-24T14:13:27.306012+0000","last_active":"2024-01-24T14:13:27.308693+0000","last_peered":"2024-01-24T14:13:27.308693+0000","last_clean":"2024-01-24T14:13:27.308693+0000","last_became_active":"2024-01-24T14:13:27.305708+0000","last_became_peered":"2024-01-24T14:13:27.305708+0000","last_unstale":"2024-01-24T14:13:27.308693+0000","last_undegraded":"2024-01-24T14:13:27.308693+0000","last_fullsized":"2024-01-24T14:13:27.308693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.307746+0000","last_change":"2024-01-24T14:13:26.884032+0000","last_active":"2024-01-24T14:13:27.307746+0000","last_peered":"2024-01-24T14:13:27.307746+0000","last_clean":"2024-01-24T14:13:27.307746+0000","last_became_active":"2024-01-24T14:13:26.880397+0000","last_became_peered":"2024-01-24T14:13:26.880397+0000","last_unstale":"2024-01-24T14:13:27.307746+0000","last_undegraded":"2024-01-24T14:13:27.307746+0000","last_fullsized":"2024-01-24T14:13:27.307746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.282208+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:13:27.282208+0000","last_peered":"2024-01-24T14:13:27.282208+0000","last_clean":"2024-01-24T14:13:27.282208+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:13:27.282208+0000","last_undegraded":"2024-01-24T14:13:27.282208+0000","last_fullsized":"2024-01-24T14:13:27.282208+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294768+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:13:27.294768+0000","last_peered":"2024-01-24T14:13:27.294768+0000","last_clean":"2024-01-24T14:13:27.294768+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:13:27.294768+0000","last_undegraded":"2024-01-24T14:13:27.294768+0000","last_fullsized":"2024-01-24T14:13:27.294768+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.301794+0000","last_change":"2024-01-24T14:13:27.295986+0000","last_active":"2024-01-24T14:13:27.301794+0000","last_peered":"2024-01-24T14:13:27.301794+0000","last_clean":"2024-01-24T14:13:27.301794+0000","last_became_active":"2024-01-24T14:13:27.295787+0000","last_became_peered":"2024-01-24T14:13:27.295787+0000","last_unstale":"2024-01-24T14:13:27.301794+0000","last_undegraded":"2024-01-24T14:13:27.301794+0000","last_fullsized":"2024-01-24T14:13:27.301794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.290979+0000","last_change":"2024-01-24T14:13:26.877280+0000","last_active":"2024-01-24T14:13:27.290979+0000","last_peered":"2024-01-24T14:13:27.290979+0000","last_clean":"2024-01-24T14:13:27.290979+0000","last_became_active":"2024-01-24T14:13:26.876588+0000","last_became_peered":"2024-01-24T14:13:26.876588+0000","last_unstale":"2024-01-24T14:13:27.290979+0000","last_undegraded":"2024-01-24T14:13:27.290979+0000","last_fullsized":"2024-01-24T14:13:27.290979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294605+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:13:27.294605+0000","last_peered":"2024-01-24T14:13:27.294605+0000","last_clean":"2024-01-24T14:13:27.294605+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:13:27.294605+0000","last_undegraded":"2024-01-24T14:13:27.294605+0000","last_fullsized":"2024-01-24T14:13:27.294605+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.305222+0000","last_change":"2024-01-24T14:13:27.294273+0000","last_active":"2024-01-24T14:13:27.305222+0000","last_peered":"2024-01-24T14:13:27.305222+0000","last_clean":"2024-01-24T14:13:27.305222+0000","last_became_active":"2024-01-24T14:13:27.293842+0000","last_became_peered":"2024-01-24T14:13:27.293842+0000","last_unstale":"2024-01-24T14:13:27.305222+0000","last_undegraded":"2024-01-24T14:13:27.305222+0000","last_fullsized":"2024-01-24T14:13:27.305222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.292440+0000","last_change":"2024-01-24T14:13:26.884832+0000","last_active":"2024-01-24T14:13:27.292440+0000","last_peered":"2024-01-24T14:13:27.292440+0000","last_clean":"2024-01-24T14:13:27.292440+0000","last_became_active":"2024-01-24T14:13:26.884544+0000","last_became_peered":"2024-01-24T14:13:26.884544+0000","last_unstale":"2024-01-24T14:13:27.292440+0000","last_undegraded":"2024-01-24T14:13:27.292440+0000","last_fullsized":"2024-01-24T14:13:27.292440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296740+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:13:27.296740+0000","last_peered":"2024-01-24T14:13:27.296740+0000","last_clean":"2024-01-24T14:13:27.296740+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:13:27.296740+0000","last_undegraded":"2024-01-24T14:13:27.296740+0000","last_fullsized":"2024-01-24T14:13:27.296740+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.310896+0000","last_change":"2024-01-24T14:13:27.308585+0000","last_active":"2024-01-24T14:13:27.310896+0000","last_peered":"2024-01-24T14:13:27.310896+0000","last_clean":"2024-01-24T14:13:27.310896+0000","last_became_active":"2024-01-24T14:13:27.308187+0000","last_became_peered":"2024-01-24T14:13:27.308187+0000","last_unstale":"2024-01-24T14:13:27.310896+0000","last_undegraded":"2024-01-24T14:13:27.310896+0000","last_fullsized":"2024-01-24T14:13:27.310896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.306557+0000","last_change":"2024-01-24T14:13:26.886922+0000","last_active":"2024-01-24T14:13:27.306557+0000","last_peered":"2024-01-24T14:13:27.306557+0000","last_clean":"2024-01-24T14:13:27.306557+0000","last_became_active":"2024-01-24T14:13:26.886649+0000","last_became_peered":"2024-01-24T14:13:26.886649+0000","last_unstale":"2024-01-24T14:13:27.306557+0000","last_undegraded":"2024-01-24T14:13:27.306557+0000","last_fullsized":"2024-01-24T14:13:27.306557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291729+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:13:27.291729+0000","last_peered":"2024-01-24T14:13:27.291729+0000","last_clean":"2024-01-24T14:13:27.291729+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:13:27.291729+0000","last_undegraded":"2024-01-24T14:13:27.291729+0000","last_fullsized":"2024-01-24T14:13:27.291729+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.308659+0000","last_change":"2024-01-24T14:13:27.304283+0000","last_active":"2024-01-24T14:13:27.308659+0000","last_peered":"2024-01-24T14:13:27.308659+0000","last_clean":"2024-01-24T14:13:27.308659+0000","last_became_active":"2024-01-24T14:13:27.303477+0000","last_became_peered":"2024-01-24T14:13:27.303477+0000","last_unstale":"2024-01-24T14:13:27.308659+0000","last_undegraded":"2024-01-24T14:13:27.308659+0000","last_fullsized":"2024-01-24T14:13:27.308659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.305213+0000","last_change":"2024-01-24T14:13:26.885513+0000","last_active":"2024-01-24T14:13:27.305213+0000","last_peered":"2024-01-24T14:13:27.305213+0000","last_clean":"2024-01-24T14:13:27.305213+0000","last_became_active":"2024-01-24T14:13:26.885133+0000","last_became_peered":"2024-01-24T14:13:26.885133+0000","last_unstale":"2024-01-24T14:13:27.305213+0000","last_undegraded":"2024-01-24T14:13:27.305213+0000","last_fullsized":"2024-01-24T14:13:27.305213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":72,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291018+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:13:27.291018+0000","last_peered":"2024-01-24T14:13:27.291018+0000","last_clean":"2024-01-24T14:13:27.291018+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:13:27.291018+0000","last_undegraded":"2024-01-24T14:13:27.291018+0000","last_fullsized":"2024-01-24T14:13:27.291018+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575032,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302268,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069572,"statfs":{"total":96636764160,"available":96327241728,"internally_reserved":0,"allocated":1101824,"data_stored":597325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.46300000000000002,"5min":0.34899999999999998,"15min":0.26000000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.47699999999999998,"5min":0.41499999999999998,"15min":0.32700000000000001},"max":{"1min":0.82499999999999996,"5min":0.82599999999999996,"15min":0.83499999999999996},"last":0.64200000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60899999999999999,"15min":0.59299999999999997},"min":{"1min":0.48999999999999999,"5min":0.42299999999999999,"15min":0.312},"max":{"1min":0.85099999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.5},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.33700000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":3.2570000000000001},"last":0.55600000000000005}]},{"osd":2,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.624,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.32000000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.876},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.50900000000000001,"5min":0.39300000000000002,"15min":0.36799999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.51600000000000001,"5min":0.41899999999999998,"15min":0.36499999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.48799999999999999,"5min":0.39000000000000001,"15min":0.35499999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":3.0049999999999999},"last":0.52800000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.38100000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72599999999999998,"5min":0.755,"15min":0.84699999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.83599999999999997},"last":0.47299999999999998}]},{"osd":5,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":0.746,"5min":0.75,"15min":0.84899999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54500000000000004,"15min":0.54600000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.80900000000000005},"last":0.57799999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.81999999999999995},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58099999999999996,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.83399999999999996,"15min":0.87},"last":0.54100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575033,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302268,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069572,"statfs":{"total":96636764160,"available":96327241728,"internally_reserved":0,"allocated":1101824,"data_stored":596491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.51300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75,"5min":0.85699999999999998,"15min":0.86899999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.438,"5min":0.371,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.84899999999999998,"15min":1.3859999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:12:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.48999999999999999,"5min":0.40100000000000002,"15min":0.35399999999999998},"max":{"1min":0.73099999999999998,"5min":0.77000000000000002,"15min":1.353},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.72599999999999998,"5min":1.032,"15min":1.032},"last":0.58599999999999997}]},{"osd":2,"last update":"Wed Jan 24 14:12:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.63100000000000001},"min":{"1min":0.39500000000000002,"5min":0.39100000000000001,"15min":0.34300000000000003},"max":{"1min":0.748,"5min":0.88500000000000001,"15min":6.8129999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.60199999999999998},"min":{"1min":0.48699999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77500000000000002,"5min":0.80800000000000005,"15min":1.4079999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:12:38 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.626,"15min":0.66700000000000004},"min":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":7.016},"last":0.62},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.47199999999999998,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.79600000000000004,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.52200000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:12:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50700000000000001,"15min":0.497},"min":{"1min":0.374,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.83899999999999997,"5min":1.165,"15min":1.165},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.53100000000000003,"15min":0.54700000000000004},"min":{"1min":0.35299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.74399999999999999,"5min":0.82299999999999995,"15min":6.8719999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Wed Jan 24 14:12:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55300000000000005,"15min":0.56899999999999995},"min":{"1min":0.374,"5min":0.35399999999999998,"15min":0.32400000000000001},"max":{"1min":0.81100000000000005,"5min":0.86699999999999999,"15min":6.976},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78400000000000003,"5min":0.85699999999999998,"15min":7.8789999999999996},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed Jan 24 14:12:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75900000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.63},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.62,"15min":0.63200000000000001},"min":{"1min":0.40699999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79000000000000004,"5min":0.89600000000000002,"15min":6.9080000000000004},"last":0.67300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302276,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069564,"statfs":{"total":96636764160,"available":96327233536,"internally_reserved":0,"allocated":1110016,"data_stored":597631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.52100000000000002,"15min":0.442},"min":{"1min":0.30299999999999999,"5min":0.245,"15min":0.23400000000000001},"max":{"1min":0.629,"5min":15.298999999999999,"15min":15.298999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.55700000000000005,"15min":0.46899999999999997},"min":{"1min":0.32700000000000001,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.628,"5min":14.654,"15min":14.654},"last":0.28299999999999997}]},{"osd":2,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45700000000000002,"15min":0.44900000000000001},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45600000000000002,"15min":0.443},"min":{"1min":0.32900000000000001,"5min":0.30499999999999999,"15min":0.25800000000000001},"max":{"1min":0.60799999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.86699999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46800000000000003,"15min":0.45300000000000001},"min":{"1min":0.38500000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61899999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49199999999999999,"15min":0.47299999999999998},"min":{"1min":0.376,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.628,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.91000000000000003}]},{"osd":4,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.5,"5min":0.439,"15min":0.439},"max":{"1min":0.88100000000000001,"5min":1.073,"15min":1.137},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.51600000000000001,"5min":0.40500000000000003,"15min":0.36499999999999999},"max":{"1min":0.90000000000000002,"5min":1.218,"15min":1.218},"last":0.63500000000000001}]},{"osd":5,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":1.109,"15min":0.83099999999999996},"min":{"1min":0.46600000000000003,"5min":0.45000000000000001,"15min":0.443},"max":{"1min":0.94999999999999996,"5min":38.808,"15min":38.808},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":1.085,"15min":0.81200000000000006},"min":{"1min":0.51100000000000001,"5min":0.44900000000000001,"15min":0.40500000000000003},"max":{"1min":0.93400000000000005,"5min":38.823,"15min":38.823},"last":0.47899999999999998}]},{"osd":6,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68200000000000005,"15min":0.68100000000000005},"min":{"1min":0.55400000000000005,"5min":0.46000000000000002,"15min":0.43099999999999999},"max":{"1min":0.94299999999999995,"5min":1.0449999999999999,"15min":1.0509999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72599999999999998,"15min":0.72399999999999998},"min":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.40000000000000002},"max":{"1min":0.98199999999999998,"5min":1.1559999999999999,"15min":1.1879999999999999},"last":0.624}]},{"osd":7,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.45300000000000001},"max":{"1min":0.995,"5min":1.105,"15min":1.155},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75},"min":{"1min":0.57299999999999995,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":0.98699999999999999,"5min":1.123,"15min":1.131},"last":0.68600000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542330,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":1089536,"data_stored":595438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.25700000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.60899999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.33800000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.44500000000000001,"15min":0.434},"min":{"1min":0.30199999999999999,"5min":0.29599999999999999,"15min":0.28199999999999997},"max":{"1min":0.56599999999999995,"5min":0.64600000000000002,"15min":0.66700000000000004},"last":0.505},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.45300000000000001},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.28799999999999998},"max":{"1min":0.58899999999999997,"5min":0.754,"15min":0.79400000000000004},"last":0.61399999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44600000000000001,"15min":0.432},"min":{"1min":0.34100000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56999999999999995,"5min":1.482,"15min":1.482},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.505,"15min":0.495},"min":{"1min":0.36399999999999999,"5min":0.35399999999999998,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":0.872,"15min":0.872},"last":0.58799999999999997}]},{"osd":4,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66500000000000004,"15min":0.65300000000000002},"min":{"1min":0.53200000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.65600000000000003},"min":{"1min":0.53300000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84399999999999997,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.78700000000000003}]},{"osd":5,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.56499999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.84899999999999998,"5min":1.655,"15min":1.655},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68799999999999994,"15min":0.68300000000000005},"min":{"1min":0.497,"5min":0.46200000000000002,"15min":0.42999999999999999},"max":{"1min":0.94799999999999995,"5min":0.98599999999999999,"15min":1.0840000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.55500000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90000000000000002,"5min":1.0309999999999999,"15min":1.137},"last":0.875},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69699999999999995,"15min":0.69399999999999995},"min":{"1min":0.58599999999999997,"5min":0.48799999999999999,"15min":0.48199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.117},"last":0.80400000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71299999999999997,"15min":0.70599999999999996},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.47099999999999997},"max":{"1min":0.93999999999999995,"5min":0.96099999999999997,"15min":0.98199999999999998},"last":0.751},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.69599999999999995},"min":{"1min":0.51000000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83599999999999997,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.90500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575032,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306356,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94065484,"statfs":{"total":96636764160,"available":96323055616,"internally_reserved":0,"allocated":1093632,"data_stored":589388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.39000000000000001},"min":{"1min":0.28299999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59299999999999997,"5min":0.621,"15min":0.65800000000000003},"last":0.433},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42799999999999999},"min":{"1min":0.28599999999999998,"5min":0.26300000000000001,"15min":0.24399999999999999},"max":{"1min":0.627,"5min":0.65900000000000003,"15min":0.67100000000000004},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27300000000000002},"max":{"1min":0.57699999999999996,"5min":0.61499999999999999,"15min":0.67200000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.44500000000000001,"15min":0.45200000000000001},"min":{"1min":0.30399999999999999,"5min":0.29699999999999999,"15min":0.255},"max":{"1min":0.625,"5min":0.66700000000000004,"15min":0.67100000000000004},"last":0.53200000000000003}]},{"osd":3,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.32000000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.60999999999999999,"5min":0.67700000000000005,"15min":0.71699999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.60499999999999998,"5min":0.68000000000000005,"15min":0.72799999999999998},"last":0.70799999999999996}]},{"osd":4,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.54900000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66300000000000003,"15min":0.66600000000000004},"min":{"1min":0.52600000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.96599999999999997,"5min":0.996,"15min":1.042},"last":0.68000000000000005}]},{"osd":5,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.54200000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.0469999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.54000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.754}]},{"osd":6,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69199999999999995,"15min":0.68899999999999995},"min":{"1min":0.53100000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.1100000000000001},"last":0.625},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.58399999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.56000000000000005,"5min":0.435,"15min":0.435},"max":{"1min":0.91800000000000004,"5min":0.97299999999999998,"15min":1.054},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93000000000000005,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.69699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575032,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302268,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069572,"statfs":{"total":96636764160,"available":96327241728,"internally_reserved":0,"allocated":1101824,"data_stored":596511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.45600000000000002,"15min":0.41099999999999998},"min":{"1min":0.28000000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58499999999999996,"5min":7.593,"15min":7.593},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.48399999999999999,"15min":0.434},"min":{"1min":0.29899999999999999,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.55500000000000005,"5min":8.4239999999999995,"15min":8.4239999999999995},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.40899999999999997},"min":{"1min":0.312,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56000000000000005,"5min":0.58799999999999997,"15min":0.69099999999999995},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.45200000000000001},"min":{"1min":0.34699999999999998,"5min":0.221,"15min":0.221},"max":{"1min":0.58399999999999996,"5min":0.66300000000000003,"15min":0.68799999999999994},"last":0.35899999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.504,"15min":0.496},"min":{"1min":0.39200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.61699999999999999,"5min":0.76300000000000001,"15min":0.77200000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.38900000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.58499999999999996,"5min":0.72999999999999998,"15min":0.78800000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64700000000000002,"15min":0.63600000000000001},"min":{"1min":0.502,"5min":0.45300000000000001,"15min":0.41799999999999998},"max":{"1min":0.88,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.47499999999999998,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":0.81899999999999995,"5min":1.976,"15min":1.976},"last":0.495}]},{"osd":5,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.497,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.54400000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91200000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.55600000000000005}]},{"osd":6,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68500000000000005,"15min":0.68400000000000005},"min":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.45400000000000001},"max":{"1min":0.85699999999999998,"5min":1.53,"15min":1.53},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70599999999999996,"15min":0.68500000000000005},"min":{"1min":0.52900000000000003,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.89900000000000002,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.48499999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.71999999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.45200000000000001},"max":{"1min":0.91500000000000004,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73899999999999999,"15min":0.73199999999999998},"min":{"1min":0.58899999999999997,"5min":0.49199999999999999,"15min":0.45800000000000002},"max":{"1min":0.88,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.60899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302248,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069592,"statfs":{"total":96636764160,"available":96327262208,"internally_reserved":0,"allocated":1081344,"data_stored":588674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.57599999999999996},"min":{"1min":0.48799999999999999,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.88600000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.44500000000000001,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.77300000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.65500000000000003}]},{"osd":1,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.53200000000000003,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.80700000000000005,"5min":0.83099999999999996,"15min":0.85299999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72399999999999998,"5min":0.82599999999999996,"15min":1.506},"last":0.68100000000000005}]},{"osd":2,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.46600000000000003,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.78400000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.60999999999999999,"15min":0.60499999999999998},"min":{"1min":0.46300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76200000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.75600000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.51200000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.73799999999999999,"5min":0.77800000000000002,"15min":0.80600000000000005},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.501,"5min":0.41499999999999998,"15min":0.36499999999999999},"max":{"1min":0.78100000000000003,"5min":0.81000000000000005,"15min":0.86299999999999999},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48599999999999999},"min":{"1min":0.34100000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65200000000000002,"5min":0.76500000000000001,"15min":0.92000000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.498,"5min":0.49299999999999999,"15min":0.50900000000000001},"min":{"1min":0.36699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":1.556},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71299999999999997,"5min":0.78500000000000003,"15min":0.86099999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72799999999999998,"5min":0.82099999999999995,"15min":1.0409999999999999},"last":0.61499999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70299999999999996,"5min":0.77400000000000002,"15min":0.88100000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.36699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.80000000000000004,"5min":0.81399999999999995,"15min":1.26},"last":0.626}]}]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302272,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069568,"statfs":{"total":96636764160,"available":96327237632,"internally_reserved":0,"allocated":1105920,"data_stored":597935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.435,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.85199999999999998,"15min":1.101},"last":0.253},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.48899999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76000000000000001,"5min":0.82699999999999996,"15min":1.1439999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.40799999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72799999999999998,"5min":0.84099999999999997,"15min":1.093},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.50900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79000000000000004,"5min":0.81299999999999994,"15min":1.1639999999999999},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.47499999999999998,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.77500000000000002,"5min":0.84299999999999997,"15min":1.1100000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.48599999999999999,"5min":0.435,"15min":0.35499999999999998},"max":{"1min":0.77700000000000002,"5min":0.83599999999999997,"15min":1.1220000000000001},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.51000000000000001,"5min":0.39900000000000002,"15min":0.38500000000000001},"max":{"1min":0.74199999999999999,"5min":0.80500000000000005,"15min":1.139},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.61899999999999999},"min":{"1min":0.47599999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.70399999999999996,"5min":0.83099999999999996,"15min":1.0609999999999999},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47899999999999998,"15min":0.48999999999999999},"min":{"1min":0.33900000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73099999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.501,"15min":0.503},"min":{"1min":0.36699999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71199999999999997,"5min":0.84999999999999998,"15min":0.90000000000000002},"last":0.56100000000000005}]},{"osd":6,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.36199999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.68700000000000006,"5min":0.86399999999999999,"15min":1.1839999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53000000000000003,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.53200000000000003}]},{"osd":7,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.5},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.435,"5min":0.34100000000000003,"15min":0.33200000000000002},"max":{"1min":0.83999999999999997,"5min":0.95999999999999996,"15min":1.2030000000000001},"last":0.52200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:13:31.392 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:13:31.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.558+0000 7f8f54f51700 1 Processor -- start 2024-01-24T14:13:31.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.558+0000 7f8f54f51700 1 -- start start 2024-01-24T14:13:31.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.559+0000 7f8f54f51700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8f50066c00 0x7f8f500bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.559+0000 7f8f54f51700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f500670a0 0x7f8f50158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.559+0000 7f8f54f51700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f501582b0 con 0x7f8f500c4a10 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.559+0000 7f8f54f51700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f50157ca0 con 0x7f8f50066c00 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.559+0000 7f8f54f51700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50063c60 con 0x7f8f500670a0 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.560+0000 7f8f4dd9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8f50066c00 0x7f8f500bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.560+0000 7f8f4dd9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8f50066c00 0x7f8f500bf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41602/0 (socket says 172.21.15.29:41602) 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.560+0000 7f8f4dd9b700 1 -- 172.21.15.29:0/2225882357 learned_addr learned my addr 172.21.15.29:0/2225882357 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.560+0000 7f8f4ed9d700 1 --2- 172.21.15.29:0/2225882357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f500670a0 0x7f8f50158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.560+0000 7f8f4dd9b700 1 -- 172.21.15.29:0/2225882357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f500670a0 msgr2=0x7f8f50158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.560+0000 7f8f4dd9b700 1 --2- 172.21.15.29:0/2225882357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f500670a0 0x7f8f50158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.560+0000 7f8f4dd9b700 1 -- 172.21.15.29:0/2225882357 >> v1:172.21.15.163:6789/0 conn(0x7f8f500c4a10 legacy=0x7f8f500664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.560+0000 7f8f4dd9b700 1 -- 172.21.15.29:0/2225882357 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f50157f40 con 0x7f8f50066c00 2024-01-24T14:13:31.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.560+0000 7f8f4dd9b700 1 --2- 172.21.15.29:0/2225882357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f50066c00 0x7f8f500bf6d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f8f44006fd0 tx=0x7f8f440090f0).ready entity=mon.0 client_cookie=41569800b2d17b43 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:31.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.560+0000 7f8f4d59a700 1 -- 172.21.15.29:0/2225882357 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4400b600 con 0x7f8f50066c00 2024-01-24T14:13:31.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.560+0000 7f8f4d59a700 1 -- 172.21.15.29:0/2225882357 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4400b760 con 0x7f8f50066c00 2024-01-24T14:13:31.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.560+0000 7f8f4d59a700 1 -- 172.21.15.29:0/2225882357 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4400bae0 con 0x7f8f50066c00 2024-01-24T14:13:31.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.561+0000 7f8f54f51700 1 -- 172.21.15.29:0/2225882357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f50066c00 msgr2=0x7f8f500bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.561+0000 7f8f54f51700 1 --2- 172.21.15.29:0/2225882357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f50066c00 0x7f8f500bf6d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f8f44006fd0 tx=0x7f8f440090f0).stop 2024-01-24T14:13:31.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.561+0000 7f8f54f51700 1 -- 172.21.15.29:0/2225882357 shutdown_connections 2024-01-24T14:13:31.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.561+0000 7f8f54f51700 1 --2- 172.21.15.29:0/2225882357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f500670a0 0x7f8f50158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.561+0000 7f8f54f51700 1 --2- 172.21.15.29:0/2225882357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f50066c00 0x7f8f500bf6d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.561+0000 7f8f54f51700 1 -- 172.21.15.29:0/2225882357 >> 172.21.15.29:0/2225882357 conn(0x7f8f500c8bb0 msgr2=0x7f8f500c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:31.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.561+0000 7f8f54f51700 1 -- 172.21.15.29:0/2225882357 shutdown_connections 2024-01-24T14:13:31.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.561+0000 7f8f54f51700 1 -- 172.21.15.29:0/2225882357 wait complete. 2024-01-24T14:13:31.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.562+0000 7f8f54f51700 1 Processor -- start 2024-01-24T14:13:31.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.562+0000 7f8f54f51700 1 -- start start 2024-01-24T14:13:31.565 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.563+0000 7f8f54f51700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f50066c00 0x7f8f500a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.563+0000 7f8f54f51700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f500670a0 0x7f8f500a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.563+0000 7f8f54f51700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f500c4a10 0x7f8f50148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.563+0000 7f8f54f51700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f500ab110 con 0x7f8f50066c00 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.563+0000 7f8f54f51700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f501494b0 con 0x7f8f500c4a10 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.563+0000 7f8f54f51700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f500a5830 con 0x7f8f500670a0 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.563+0000 7f8f4e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f50066c00 0x7f8f500a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.563+0000 7f8f4e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f50066c00 0x7f8f500a5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41612/0 (socket says 172.21.15.29:41612) 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.563+0000 7f8f4e59c700 1 -- 172.21.15.29:0/467220358 learned_addr learned my addr 172.21.15.29:0/467220358 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.563+0000 7f8f4dd9b700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f500670a0 0x7f8f500a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.563+0000 7f8f4ed9d700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f500c4a10 0x7f8f50148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.564+0000 7f8f4e59c700 1 -- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f500670a0 msgr2=0x7f8f500a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.564+0000 7f8f4e59c700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f500670a0 0x7f8f500a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.564+0000 7f8f4e59c700 1 -- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f500c4a10 msgr2=0x7f8f50148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.564+0000 7f8f4e59c700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f500c4a10 0x7f8f50148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.564+0000 7f8f4e59c700 1 -- 172.21.15.29:0/467220358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f44009370 con 0x7f8f50066c00 2024-01-24T14:13:31.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.564+0000 7f8f4e59c700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f50066c00 0x7f8f500a5270 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f8f4000b460 tx=0x7f8f40006f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:31.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.564+0000 7f8f3e7fc700 1 -- 172.21.15.29:0/467220358 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40012410 con 0x7f8f50066c00 2024-01-24T14:13:31.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.564+0000 7f8f3e7fc700 1 -- 172.21.15.29:0/467220358 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f40012590 con 0x7f8f50066c00 2024-01-24T14:13:31.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.564+0000 7f8f54f51700 1 -- 172.21.15.29:0/467220358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f500a7270 con 0x7f8f50066c00 2024-01-24T14:13:31.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.564+0000 7f8f3e7fc700 1 -- 172.21.15.29:0/467220358 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40012950 con 0x7f8f50066c00 2024-01-24T14:13:31.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.564+0000 7f8f54f51700 1 -- 172.21.15.29:0/467220358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f50149da0 con 0x7f8f50066c00 2024-01-24T14:13:31.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.565+0000 7f8f54f51700 1 -- 172.21.15.29:0/467220358 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f500a4a90 con 0x7f8f50066c00 2024-01-24T14:13:31.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.566+0000 7f8f3e7fc700 1 -- 172.21.15.29:0/467220358 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8f40012cc0 con 0x7f8f50066c00 2024-01-24T14:13:31.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.566+0000 7f8f3e7fc700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f3005f270 0x7f8f30061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.566+0000 7f8f3e7fc700 1 -- 172.21.15.29:0/467220358 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f8f4007a540 con 0x7f8f50066c00 2024-01-24T14:13:31.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.567+0000 7f8f4dd9b700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f3005f270 0x7f8f30061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.567+0000 7f8f4dd9b700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f3005f270 0x7f8f30061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8f44009130 tx=0x7f8f44017000).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:31.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.571+0000 7f8f3e7fc700 1 -- 172.21.15.29:0/467220358 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8f4004b1d0 con 0x7f8f50066c00 2024-01-24T14:13:31.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.746+0000 7f8f54f51700 1 -- 172.21.15.29:0/467220358 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f500a4880 con 0x7f8f3005f270 2024-01-24T14:13:31.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.757+0000 7f8f3e7fc700 1 -- 172.21.15.29:0/467220358 <== mgr.5558 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361387 (secure 0 0 0) 0x7f8f500a4880 con 0x7f8f3005f270 2024-01-24T14:13:31.762 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:31.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.761+0000 7f8f54f51700 1 -- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f3005f270 msgr2=0x7f8f30061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.761+0000 7f8f54f51700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f3005f270 0x7f8f30061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8f44009130 tx=0x7f8f44017000).stop 2024-01-24T14:13:31.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.761+0000 7f8f54f51700 1 -- 172.21.15.29:0/467220358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f50066c00 msgr2=0x7f8f500a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.761+0000 7f8f54f51700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f50066c00 0x7f8f500a5270 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f8f4000b460 tx=0x7f8f40006f20).stop 2024-01-24T14:13:31.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.761+0000 7f8f54f51700 1 -- 172.21.15.29:0/467220358 shutdown_connections 2024-01-24T14:13:31.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.761+0000 7f8f54f51700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f500c4a10 0x7f8f50148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.761+0000 7f8f54f51700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8f3005f270 0x7f8f30061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.761+0000 7f8f54f51700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f50066c00 0x7f8f500a5270 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.761+0000 7f8f54f51700 1 --2- 172.21.15.29:0/467220358 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f500670a0 0x7f8f500a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.761+0000 7f8f54f51700 1 -- 172.21.15.29:0/467220358 >> 172.21.15.29:0/467220358 conn(0x7f8f500c8bb0 msgr2=0x7f8f500664f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:31.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.761+0000 7f8f54f51700 1 -- 172.21.15.29:0/467220358 shutdown_connections 2024-01-24T14:13:31.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.761+0000 7f8f54f51700 1 -- 172.21.15.29:0/467220358 wait complete. 2024-01-24T14:13:31.766 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:13:31.775 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-01-24T14:13:30.120629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422212,"kb_used_data":8580,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752552508,"statfs":{"total":773094113280,"available":770613768192,"internally_reserved":0,"allocated":8785920,"data_stored":4759393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28929,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-81,"num_write_kb":-66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-1067,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.699997"},"pg_stats":[{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.312108+0000","last_change":"2024-01-24T14:13:27.308964+0000","last_active":"2024-01-24T14:13:27.312108+0000","last_peered":"2024-01-24T14:13:27.312108+0000","last_clean":"2024-01-24T14:13:27.312108+0000","last_became_active":"2024-01-24T14:13:27.306530+0000","last_became_peered":"2024-01-24T14:13:27.306530+0000","last_unstale":"2024-01-24T14:13:27.312108+0000","last_undegraded":"2024-01-24T14:13:27.312108+0000","last_fullsized":"2024-01-24T14:13:27.312108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3f","version":"89'3","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.307054+0000","last_change":"2024-01-24T14:13:26.871570+0000","last_active":"2024-01-24T14:13:27.307054+0000","last_peered":"2024-01-24T14:13:27.307054+0000","last_clean":"2024-01-24T14:13:27.307054+0000","last_became_active":"2024-01-24T14:13:26.870323+0000","last_became_peered":"2024-01-24T14:13:26.870323+0000","last_unstale":"2024-01-24T14:13:27.307054+0000","last_undegraded":"2024-01-24T14:13:27.307054+0000","last_fullsized":"2024-01-24T14:13:27.307054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.307904+0000","last_change":"2024-01-24T14:13:27.302233+0000","last_active":"2024-01-24T14:13:27.307904+0000","last_peered":"2024-01-24T14:13:27.307904+0000","last_clean":"2024-01-24T14:13:27.307904+0000","last_became_active":"2024-01-24T14:13:27.301920+0000","last_became_peered":"2024-01-24T14:13:27.301920+0000","last_unstale":"2024-01-24T14:13:27.307904+0000","last_undegraded":"2024-01-24T14:13:27.307904+0000","last_fullsized":"2024-01-24T14:13:27.307904+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291665+0000","last_change":"2024-01-24T14:13:26.882561+0000","last_active":"2024-01-24T14:13:27.291665+0000","last_peered":"2024-01-24T14:13:27.291665+0000","last_clean":"2024-01-24T14:13:27.291665+0000","last_became_active":"2024-01-24T14:13:26.881499+0000","last_became_peered":"2024-01-24T14:13:26.881499+0000","last_unstale":"2024-01-24T14:13:27.291665+0000","last_undegraded":"2024-01-24T14:13:27.291665+0000","last_fullsized":"2024-01-24T14:13:27.291665+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.306396+0000","last_change":"2024-01-24T14:13:27.304914+0000","last_active":"2024-01-24T14:13:27.306396+0000","last_peered":"2024-01-24T14:13:27.306396+0000","last_clean":"2024-01-24T14:13:27.306396+0000","last_became_active":"2024-01-24T14:13:27.297643+0000","last_became_peered":"2024-01-24T14:13:27.297643+0000","last_unstale":"2024-01-24T14:13:27.306396+0000","last_undegraded":"2024-01-24T14:13:27.306396+0000","last_fullsized":"2024-01-24T14:13:27.306396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296654+0000","last_change":"2024-01-24T14:13:26.881205+0000","last_active":"2024-01-24T14:13:27.296654+0000","last_peered":"2024-01-24T14:13:27.296654+0000","last_clean":"2024-01-24T14:13:27.296654+0000","last_became_active":"2024-01-24T14:13:26.880723+0000","last_became_peered":"2024-01-24T14:13:26.880723+0000","last_unstale":"2024-01-24T14:13:27.296654+0000","last_undegraded":"2024-01-24T14:13:27.296654+0000","last_fullsized":"2024-01-24T14:13:27.296654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.303518+0000","last_change":"2024-01-24T14:13:27.295901+0000","last_active":"2024-01-24T14:13:27.303518+0000","last_peered":"2024-01-24T14:13:27.303518+0000","last_clean":"2024-01-24T14:13:27.303518+0000","last_became_active":"2024-01-24T14:13:27.295620+0000","last_became_peered":"2024-01-24T14:13:27.295620+0000","last_unstale":"2024-01-24T14:13:27.303518+0000","last_undegraded":"2024-01-24T14:13:27.303518+0000","last_fullsized":"2024-01-24T14:13:27.303518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297309+0000","last_change":"2024-01-24T14:13:26.876152+0000","last_active":"2024-01-24T14:13:27.297309+0000","last_peered":"2024-01-24T14:13:27.297309+0000","last_clean":"2024-01-24T14:13:27.297309+0000","last_became_active":"2024-01-24T14:13:26.875854+0000","last_became_peered":"2024-01-24T14:13:26.875854+0000","last_unstale":"2024-01-24T14:13:27.297309+0000","last_undegraded":"2024-01-24T14:13:27.297309+0000","last_fullsized":"2024-01-24T14:13:27.297309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311941+0000","last_change":"2024-01-24T14:13:27.307224+0000","last_active":"2024-01-24T14:13:27.311941+0000","last_peered":"2024-01-24T14:13:27.311941+0000","last_clean":"2024-01-24T14:13:27.311941+0000","last_became_active":"2024-01-24T14:13:27.306551+0000","last_became_peered":"2024-01-24T14:13:27.306551+0000","last_unstale":"2024-01-24T14:13:27.311941+0000","last_undegraded":"2024-01-24T14:13:27.311941+0000","last_fullsized":"2024-01-24T14:13:27.311941+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3b","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309339+0000","last_change":"2024-01-24T14:13:26.886280+0000","last_active":"2024-01-24T14:13:27.309339+0000","last_peered":"2024-01-24T14:13:27.309339+0000","last_clean":"2024-01-24T14:13:27.309339+0000","last_became_active":"2024-01-24T14:13:26.885965+0000","last_became_peered":"2024-01-24T14:13:26.885965+0000","last_unstale":"2024-01-24T14:13:27.309339+0000","last_undegraded":"2024-01-24T14:13:27.309339+0000","last_fullsized":"2024-01-24T14:13:27.309339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.302150+0000","last_change":"2024-01-24T14:13:27.300613+0000","last_active":"2024-01-24T14:13:27.302150+0000","last_peered":"2024-01-24T14:13:27.302150+0000","last_clean":"2024-01-24T14:13:27.302150+0000","last_became_active":"2024-01-24T14:13:27.300337+0000","last_became_peered":"2024-01-24T14:13:27.300337+0000","last_unstale":"2024-01-24T14:13:27.302150+0000","last_undegraded":"2024-01-24T14:13:27.302150+0000","last_fullsized":"2024-01-24T14:13:27.302150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3a","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296996+0000","last_change":"2024-01-24T14:13:26.878801+0000","last_active":"2024-01-24T14:13:27.296996+0000","last_peered":"2024-01-24T14:13:27.296996+0000","last_clean":"2024-01-24T14:13:27.296996+0000","last_became_active":"2024-01-24T14:13:26.878495+0000","last_became_peered":"2024-01-24T14:13:26.878495+0000","last_unstale":"2024-01-24T14:13:27.296996+0000","last_undegraded":"2024-01-24T14:13:27.296996+0000","last_fullsized":"2024-01-24T14:13:27.296996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309479+0000","last_change":"2024-01-24T14:13:27.295823+0000","last_active":"2024-01-24T14:13:27.309479+0000","last_peered":"2024-01-24T14:13:27.309479+0000","last_clean":"2024-01-24T14:13:27.309479+0000","last_became_active":"2024-01-24T14:13:27.295159+0000","last_became_peered":"2024-01-24T14:13:27.295159+0000","last_unstale":"2024-01-24T14:13:27.309479+0000","last_undegraded":"2024-01-24T14:13:27.309479+0000","last_fullsized":"2024-01-24T14:13:27.309479+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.275061+0000","last_change":"2024-01-24T14:13:26.878394+0000","last_active":"2024-01-24T14:13:27.275061+0000","last_peered":"2024-01-24T14:13:27.275061+0000","last_clean":"2024-01-24T14:13:27.275061+0000","last_became_active":"2024-01-24T14:13:26.878090+0000","last_became_peered":"2024-01-24T14:13:26.878090+0000","last_unstale":"2024-01-24T14:13:27.275061+0000","last_undegraded":"2024-01-24T14:13:27.275061+0000","last_fullsized":"2024-01-24T14:13:27.275061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294497+0000","last_change":"2024-01-24T14:13:27.291413+0000","last_active":"2024-01-24T14:13:27.294497+0000","last_peered":"2024-01-24T14:13:27.294497+0000","last_clean":"2024-01-24T14:13:27.294497+0000","last_became_active":"2024-01-24T14:13:27.290960+0000","last_became_peered":"2024-01-24T14:13:27.290960+0000","last_unstale":"2024-01-24T14:13:27.294497+0000","last_undegraded":"2024-01-24T14:13:27.294497+0000","last_fullsized":"2024-01-24T14:13:27.294497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.280494+0000","last_change":"2024-01-24T14:13:26.875046+0000","last_active":"2024-01-24T14:13:27.280494+0000","last_peered":"2024-01-24T14:13:27.280494+0000","last_clean":"2024-01-24T14:13:27.280494+0000","last_became_active":"2024-01-24T14:13:26.874106+0000","last_became_peered":"2024-01-24T14:13:26.874106+0000","last_unstale":"2024-01-24T14:13:27.280494+0000","last_undegraded":"2024-01-24T14:13:27.280494+0000","last_fullsized":"2024-01-24T14:13:27.280494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.299182+0000","last_change":"2024-01-24T14:13:27.286360+0000","last_active":"2024-01-24T14:13:27.299182+0000","last_peered":"2024-01-24T14:13:27.299182+0000","last_clean":"2024-01-24T14:13:27.299182+0000","last_became_active":"2024-01-24T14:13:27.286072+0000","last_became_peered":"2024-01-24T14:13:27.286072+0000","last_unstale":"2024-01-24T14:13:27.299182+0000","last_undegraded":"2024-01-24T14:13:27.299182+0000","last_fullsized":"2024-01-24T14:13:27.299182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297712+0000","last_change":"2024-01-24T14:13:26.878664+0000","last_active":"2024-01-24T14:13:27.297712+0000","last_peered":"2024-01-24T14:13:27.297712+0000","last_clean":"2024-01-24T14:13:27.297712+0000","last_became_active":"2024-01-24T14:13:26.878345+0000","last_became_peered":"2024-01-24T14:13:26.878345+0000","last_unstale":"2024-01-24T14:13:27.297712+0000","last_undegraded":"2024-01-24T14:13:27.297712+0000","last_fullsized":"2024-01-24T14:13:27.297712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.292080+0000","last_change":"2024-01-24T14:13:27.290026+0000","last_active":"2024-01-24T14:13:27.292080+0000","last_peered":"2024-01-24T14:13:27.292080+0000","last_clean":"2024-01-24T14:13:27.292080+0000","last_became_active":"2024-01-24T14:13:27.289736+0000","last_became_peered":"2024-01-24T14:13:27.289736+0000","last_unstale":"2024-01-24T14:13:27.292080+0000","last_undegraded":"2024-01-24T14:13:27.292080+0000","last_fullsized":"2024-01-24T14:13:27.292080+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297602+0000","last_change":"2024-01-24T14:13:26.881814+0000","last_active":"2024-01-24T14:13:27.297602+0000","last_peered":"2024-01-24T14:13:27.297602+0000","last_clean":"2024-01-24T14:13:27.297602+0000","last_became_active":"2024-01-24T14:13:26.881517+0000","last_became_peered":"2024-01-24T14:13:26.881517+0000","last_unstale":"2024-01-24T14:13:27.297602+0000","last_undegraded":"2024-01-24T14:13:27.297602+0000","last_fullsized":"2024-01-24T14:13:27.297602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.305196+0000","last_change":"2024-01-24T14:13:27.294094+0000","last_active":"2024-01-24T14:13:27.305196+0000","last_peered":"2024-01-24T14:13:27.305196+0000","last_clean":"2024-01-24T14:13:27.305196+0000","last_became_active":"2024-01-24T14:13:27.293472+0000","last_became_peered":"2024-01-24T14:13:27.293472+0000","last_unstale":"2024-01-24T14:13:27.305196+0000","last_undegraded":"2024-01-24T14:13:27.305196+0000","last_fullsized":"2024-01-24T14:13:27.305196+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.282075+0000","last_change":"2024-01-24T14:13:26.876449+0000","last_active":"2024-01-24T14:13:27.282075+0000","last_peered":"2024-01-24T14:13:27.282075+0000","last_clean":"2024-01-24T14:13:27.282075+0000","last_became_active":"2024-01-24T14:13:26.875833+0000","last_became_peered":"2024-01-24T14:13:26.875833+0000","last_unstale":"2024-01-24T14:13:27.282075+0000","last_undegraded":"2024-01-24T14:13:27.282075+0000","last_fullsized":"2024-01-24T14:13:27.282075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.310448+0000","last_change":"2024-01-24T14:13:27.305044+0000","last_active":"2024-01-24T14:13:27.310448+0000","last_peered":"2024-01-24T14:13:27.310448+0000","last_clean":"2024-01-24T14:13:27.310448+0000","last_became_active":"2024-01-24T14:13:27.304734+0000","last_became_peered":"2024-01-24T14:13:27.304734+0000","last_unstale":"2024-01-24T14:13:27.310448+0000","last_undegraded":"2024-01-24T14:13:27.310448+0000","last_fullsized":"2024-01-24T14:13:27.310448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.34","version":"89'4","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.314002+0000","last_change":"2024-01-24T14:13:26.874452+0000","last_active":"2024-01-24T14:13:27.314002+0000","last_peered":"2024-01-24T14:13:27.314002+0000","last_clean":"2024-01-24T14:13:27.314002+0000","last_became_active":"2024-01-24T14:13:26.874170+0000","last_became_peered":"2024-01-24T14:13:26.874170+0000","last_unstale":"2024-01-24T14:13:27.314002+0000","last_undegraded":"2024-01-24T14:13:27.314002+0000","last_fullsized":"2024-01-24T14:13:27.314002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297752+0000","last_change":"2024-01-24T14:13:27.295298+0000","last_active":"2024-01-24T14:13:27.297752+0000","last_peered":"2024-01-24T14:13:27.297752+0000","last_clean":"2024-01-24T14:13:27.297752+0000","last_became_active":"2024-01-24T14:13:27.294990+0000","last_became_peered":"2024-01-24T14:13:27.294990+0000","last_unstale":"2024-01-24T14:13:27.297752+0000","last_undegraded":"2024-01-24T14:13:27.297752+0000","last_fullsized":"2024-01-24T14:13:27.297752+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294222+0000","last_change":"2024-01-24T14:13:26.870270+0000","last_active":"2024-01-24T14:13:27.294222+0000","last_peered":"2024-01-24T14:13:27.294222+0000","last_clean":"2024-01-24T14:13:27.294222+0000","last_became_active":"2024-01-24T14:13:26.869551+0000","last_became_peered":"2024-01-24T14:13:26.869551+0000","last_unstale":"2024-01-24T14:13:27.294222+0000","last_undegraded":"2024-01-24T14:13:27.294222+0000","last_fullsized":"2024-01-24T14:13:27.294222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.301614+0000","last_change":"2024-01-24T14:13:27.298389+0000","last_active":"2024-01-24T14:13:27.301614+0000","last_peered":"2024-01-24T14:13:27.301614+0000","last_clean":"2024-01-24T14:13:27.301614+0000","last_became_active":"2024-01-24T14:13:27.298084+0000","last_became_peered":"2024-01-24T14:13:27.298084+0000","last_unstale":"2024-01-24T14:13:27.301614+0000","last_undegraded":"2024-01-24T14:13:27.301614+0000","last_fullsized":"2024-01-24T14:13:27.301614+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.295498+0000","last_change":"2024-01-24T14:13:26.881359+0000","last_active":"2024-01-24T14:13:27.295498+0000","last_peered":"2024-01-24T14:13:27.295498+0000","last_clean":"2024-01-24T14:13:27.295498+0000","last_became_active":"2024-01-24T14:13:26.881068+0000","last_became_peered":"2024-01-24T14:13:26.881068+0000","last_unstale":"2024-01-24T14:13:27.295498+0000","last_undegraded":"2024-01-24T14:13:27.295498+0000","last_fullsized":"2024-01-24T14:13:27.295498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.303902+0000","last_change":"2024-01-24T14:13:27.296706+0000","last_active":"2024-01-24T14:13:27.303902+0000","last_peered":"2024-01-24T14:13:27.303902+0000","last_clean":"2024-01-24T14:13:27.303902+0000","last_became_active":"2024-01-24T14:13:27.296387+0000","last_became_peered":"2024-01-24T14:13:27.296387+0000","last_unstale":"2024-01-24T14:13:27.303902+0000","last_undegraded":"2024-01-24T14:13:27.303902+0000","last_fullsized":"2024-01-24T14:13:27.303902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.295172+0000","last_change":"2024-01-24T14:13:26.882076+0000","last_active":"2024-01-24T14:13:27.295172+0000","last_peered":"2024-01-24T14:13:27.295172+0000","last_clean":"2024-01-24T14:13:27.295172+0000","last_became_active":"2024-01-24T14:13:26.881479+0000","last_became_peered":"2024-01-24T14:13:26.881479+0000","last_unstale":"2024-01-24T14:13:27.295172+0000","last_undegraded":"2024-01-24T14:13:27.295172+0000","last_fullsized":"2024-01-24T14:13:27.295172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294831+0000","last_change":"2024-01-24T14:13:27.290668+0000","last_active":"2024-01-24T14:13:27.294831+0000","last_peered":"2024-01-24T14:13:27.294831+0000","last_clean":"2024-01-24T14:13:27.294831+0000","last_became_active":"2024-01-24T14:13:27.290389+0000","last_became_peered":"2024-01-24T14:13:27.290389+0000","last_unstale":"2024-01-24T14:13:27.294831+0000","last_undegraded":"2024-01-24T14:13:27.294831+0000","last_fullsized":"2024-01-24T14:13:27.294831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.30","version":"89'4","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294953+0000","last_change":"2024-01-24T14:13:26.863361+0000","last_active":"2024-01-24T14:13:27.294953+0000","last_peered":"2024-01-24T14:13:27.294953+0000","last_clean":"2024-01-24T14:13:27.294953+0000","last_became_active":"2024-01-24T14:13:26.863143+0000","last_became_peered":"2024-01-24T14:13:26.863143+0000","last_unstale":"2024-01-24T14:13:27.294953+0000","last_undegraded":"2024-01-24T14:13:27.294953+0000","last_fullsized":"2024-01-24T14:13:27.294953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.308398+0000","last_change":"2024-01-24T14:13:27.306909+0000","last_active":"2024-01-24T14:13:27.308398+0000","last_peered":"2024-01-24T14:13:27.308398+0000","last_clean":"2024-01-24T14:13:27.308398+0000","last_became_active":"2024-01-24T14:13:27.306663+0000","last_became_peered":"2024-01-24T14:13:27.306663+0000","last_unstale":"2024-01-24T14:13:27.308398+0000","last_undegraded":"2024-01-24T14:13:27.308398+0000","last_fullsized":"2024-01-24T14:13:27.308398+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291778+0000","last_change":"2024-01-24T14:13:26.871737+0000","last_active":"2024-01-24T14:13:27.291778+0000","last_peered":"2024-01-24T14:13:27.291778+0000","last_clean":"2024-01-24T14:13:27.291778+0000","last_became_active":"2024-01-24T14:13:26.869983+0000","last_became_peered":"2024-01-24T14:13:26.869983+0000","last_unstale":"2024-01-24T14:13:27.291778+0000","last_undegraded":"2024-01-24T14:13:27.291778+0000","last_fullsized":"2024-01-24T14:13:27.291778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.304821+0000","last_change":"2024-01-24T14:13:27.299811+0000","last_active":"2024-01-24T14:13:27.304821+0000","last_peered":"2024-01-24T14:13:27.304821+0000","last_clean":"2024-01-24T14:13:27.304821+0000","last_became_active":"2024-01-24T14:13:27.299130+0000","last_became_peered":"2024-01-24T14:13:27.299130+0000","last_unstale":"2024-01-24T14:13:27.304821+0000","last_undegraded":"2024-01-24T14:13:27.304821+0000","last_fullsized":"2024-01-24T14:13:27.304821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.284753+0000","last_change":"2024-01-24T14:13:26.882012+0000","last_active":"2024-01-24T14:13:27.284753+0000","last_peered":"2024-01-24T14:13:27.284753+0000","last_clean":"2024-01-24T14:13:27.284753+0000","last_became_active":"2024-01-24T14:13:26.881709+0000","last_became_peered":"2024-01-24T14:13:26.881709+0000","last_unstale":"2024-01-24T14:13:27.284753+0000","last_undegraded":"2024-01-24T14:13:27.284753+0000","last_fullsized":"2024-01-24T14:13:27.284753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297744+0000","last_change":"2024-01-24T14:13:27.295393+0000","last_active":"2024-01-24T14:13:27.297744+0000","last_peered":"2024-01-24T14:13:27.297744+0000","last_clean":"2024-01-24T14:13:27.297744+0000","last_became_active":"2024-01-24T14:13:27.295086+0000","last_became_peered":"2024-01-24T14:13:27.295086+0000","last_unstale":"2024-01-24T14:13:27.297744+0000","last_undegraded":"2024-01-24T14:13:27.297744+0000","last_fullsized":"2024-01-24T14:13:27.297744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2d","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309447+0000","last_change":"2024-01-24T14:13:26.879410+0000","last_active":"2024-01-24T14:13:27.309447+0000","last_peered":"2024-01-24T14:13:27.309447+0000","last_clean":"2024-01-24T14:13:27.309447+0000","last_became_active":"2024-01-24T14:13:26.879111+0000","last_became_peered":"2024-01-24T14:13:26.879111+0000","last_unstale":"2024-01-24T14:13:27.309447+0000","last_undegraded":"2024-01-24T14:13:27.309447+0000","last_fullsized":"2024-01-24T14:13:27.309447+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309379+0000","last_change":"2024-01-24T14:13:27.295752+0000","last_active":"2024-01-24T14:13:27.309379+0000","last_peered":"2024-01-24T14:13:27.309379+0000","last_clean":"2024-01-24T14:13:27.309379+0000","last_became_active":"2024-01-24T14:13:27.295386+0000","last_became_peered":"2024-01-24T14:13:27.295386+0000","last_unstale":"2024-01-24T14:13:27.309379+0000","last_undegraded":"2024-01-24T14:13:27.309379+0000","last_fullsized":"2024-01-24T14:13:27.309379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.282865+0000","last_change":"2024-01-24T14:13:26.886393+0000","last_active":"2024-01-24T14:13:27.282865+0000","last_peered":"2024-01-24T14:13:27.282865+0000","last_clean":"2024-01-24T14:13:27.282865+0000","last_became_active":"2024-01-24T14:13:26.886118+0000","last_became_peered":"2024-01-24T14:13:26.886118+0000","last_unstale":"2024-01-24T14:13:27.282865+0000","last_undegraded":"2024-01-24T14:13:27.282865+0000","last_fullsized":"2024-01-24T14:13:27.282865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296989+0000","last_change":"2024-01-24T14:13:27.286362+0000","last_active":"2024-01-24T14:13:27.296989+0000","last_peered":"2024-01-24T14:13:27.296989+0000","last_clean":"2024-01-24T14:13:27.296989+0000","last_became_active":"2024-01-24T14:13:27.286067+0000","last_became_peered":"2024-01-24T14:13:27.286067+0000","last_unstale":"2024-01-24T14:13:27.296989+0000","last_undegraded":"2024-01-24T14:13:27.296989+0000","last_fullsized":"2024-01-24T14:13:27.296989+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294286+0000","last_change":"2024-01-24T14:13:26.870592+0000","last_active":"2024-01-24T14:13:27.294286+0000","last_peered":"2024-01-24T14:13:27.294286+0000","last_clean":"2024-01-24T14:13:27.294286+0000","last_became_active":"2024-01-24T14:13:26.870307+0000","last_became_peered":"2024-01-24T14:13:26.870307+0000","last_unstale":"2024-01-24T14:13:27.294286+0000","last_undegraded":"2024-01-24T14:13:27.294286+0000","last_fullsized":"2024-01-24T14:13:27.294286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.299203+0000","last_change":"2024-01-24T14:13:27.293871+0000","last_active":"2024-01-24T14:13:27.299203+0000","last_peered":"2024-01-24T14:13:27.299203+0000","last_clean":"2024-01-24T14:13:27.299203+0000","last_became_active":"2024-01-24T14:13:27.291893+0000","last_became_peered":"2024-01-24T14:13:27.291893+0000","last_unstale":"2024-01-24T14:13:27.299203+0000","last_undegraded":"2024-01-24T14:13:27.299203+0000","last_fullsized":"2024-01-24T14:13:27.299203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.286980+0000","last_change":"2024-01-24T14:13:26.879131+0000","last_active":"2024-01-24T14:13:27.286980+0000","last_peered":"2024-01-24T14:13:27.286980+0000","last_clean":"2024-01-24T14:13:27.286980+0000","last_became_active":"2024-01-24T14:13:26.878760+0000","last_became_peered":"2024-01-24T14:13:26.878760+0000","last_unstale":"2024-01-24T14:13:27.286980+0000","last_undegraded":"2024-01-24T14:13:27.286980+0000","last_fullsized":"2024-01-24T14:13:27.286980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.313785+0000","last_change":"2024-01-24T14:13:27.310107+0000","last_active":"2024-01-24T14:13:27.313785+0000","last_peered":"2024-01-24T14:13:27.313785+0000","last_clean":"2024-01-24T14:13:27.313785+0000","last_became_active":"2024-01-24T14:13:27.309809+0000","last_became_peered":"2024-01-24T14:13:27.309809+0000","last_unstale":"2024-01-24T14:13:27.313785+0000","last_undegraded":"2024-01-24T14:13:27.313785+0000","last_fullsized":"2024-01-24T14:13:27.313785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.282909+0000","last_change":"2024-01-24T14:13:26.879546+0000","last_active":"2024-01-24T14:13:27.282909+0000","last_peered":"2024-01-24T14:13:27.282909+0000","last_clean":"2024-01-24T14:13:27.282909+0000","last_became_active":"2024-01-24T14:13:26.878706+0000","last_became_peered":"2024-01-24T14:13:26.878706+0000","last_unstale":"2024-01-24T14:13:27.282909+0000","last_undegraded":"2024-01-24T14:13:27.282909+0000","last_fullsized":"2024-01-24T14:13:27.282909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.307697+0000","last_change":"2024-01-24T14:13:27.304315+0000","last_active":"2024-01-24T14:13:27.307697+0000","last_peered":"2024-01-24T14:13:27.307697+0000","last_clean":"2024-01-24T14:13:27.307697+0000","last_became_active":"2024-01-24T14:13:27.303528+0000","last_became_peered":"2024-01-24T14:13:27.303528+0000","last_unstale":"2024-01-24T14:13:27.307697+0000","last_undegraded":"2024-01-24T14:13:27.307697+0000","last_fullsized":"2024-01-24T14:13:27.307697+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294498+0000","last_change":"2024-01-24T14:13:26.881176+0000","last_active":"2024-01-24T14:13:27.294498+0000","last_peered":"2024-01-24T14:13:27.294498+0000","last_clean":"2024-01-24T14:13:27.294498+0000","last_became_active":"2024-01-24T14:13:26.880825+0000","last_became_peered":"2024-01-24T14:13:26.880825+0000","last_unstale":"2024-01-24T14:13:27.294498+0000","last_undegraded":"2024-01-24T14:13:27.294498+0000","last_fullsized":"2024-01-24T14:13:27.294498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.310807+0000","last_change":"2024-01-24T14:13:27.307524+0000","last_active":"2024-01-24T14:13:27.310807+0000","last_peered":"2024-01-24T14:13:27.310807+0000","last_clean":"2024-01-24T14:13:27.310807+0000","last_became_active":"2024-01-24T14:13:27.307203+0000","last_became_peered":"2024-01-24T14:13:27.307203+0000","last_unstale":"2024-01-24T14:13:27.310807+0000","last_undegraded":"2024-01-24T14:13:27.310807+0000","last_fullsized":"2024-01-24T14:13:27.310807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.281163+0000","last_change":"2024-01-24T14:13:26.873351+0000","last_active":"2024-01-24T14:13:27.281163+0000","last_peered":"2024-01-24T14:13:27.281163+0000","last_clean":"2024-01-24T14:13:27.281163+0000","last_became_active":"2024-01-24T14:13:26.873043+0000","last_became_peered":"2024-01-24T14:13:26.873043+0000","last_unstale":"2024-01-24T14:13:27.281163+0000","last_undegraded":"2024-01-24T14:13:27.281163+0000","last_fullsized":"2024-01-24T14:13:27.281163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.306396+0000","last_change":"2024-01-24T14:13:27.302968+0000","last_active":"2024-01-24T14:13:27.306396+0000","last_peered":"2024-01-24T14:13:27.306396+0000","last_clean":"2024-01-24T14:13:27.306396+0000","last_became_active":"2024-01-24T14:13:27.302631+0000","last_became_peered":"2024-01-24T14:13:27.302631+0000","last_unstale":"2024-01-24T14:13:27.306396+0000","last_undegraded":"2024-01-24T14:13:27.306396+0000","last_fullsized":"2024-01-24T14:13:27.306396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.281815+0000","last_change":"2024-01-24T14:13:26.876101+0000","last_active":"2024-01-24T14:13:27.281815+0000","last_peered":"2024-01-24T14:13:27.281815+0000","last_clean":"2024-01-24T14:13:27.281815+0000","last_became_active":"2024-01-24T14:13:26.875225+0000","last_became_peered":"2024-01-24T14:13:26.875225+0000","last_unstale":"2024-01-24T14:13:27.281815+0000","last_undegraded":"2024-01-24T14:13:27.281815+0000","last_fullsized":"2024-01-24T14:13:27.281815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296952+0000","last_change":"2024-01-24T14:13:27.287754+0000","last_active":"2024-01-24T14:13:27.296952+0000","last_peered":"2024-01-24T14:13:27.296952+0000","last_clean":"2024-01-24T14:13:27.296952+0000","last_became_active":"2024-01-24T14:13:27.287464+0000","last_became_peered":"2024-01-24T14:13:27.287464+0000","last_unstale":"2024-01-24T14:13:27.296952+0000","last_undegraded":"2024-01-24T14:13:27.296952+0000","last_fullsized":"2024-01-24T14:13:27.296952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.282178+0000","last_change":"2024-01-24T14:13:26.876609+0000","last_active":"2024-01-24T14:13:27.282178+0000","last_peered":"2024-01-24T14:13:27.282178+0000","last_clean":"2024-01-24T14:13:27.282178+0000","last_became_active":"2024-01-24T14:13:26.876183+0000","last_became_peered":"2024-01-24T14:13:26.876183+0000","last_unstale":"2024-01-24T14:13:27.282178+0000","last_undegraded":"2024-01-24T14:13:27.282178+0000","last_fullsized":"2024-01-24T14:13:27.282178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.304540+0000","last_change":"2024-01-24T14:13:27.302578+0000","last_active":"2024-01-24T14:13:27.304540+0000","last_peered":"2024-01-24T14:13:27.304540+0000","last_clean":"2024-01-24T14:13:27.304540+0000","last_became_active":"2024-01-24T14:13:27.302240+0000","last_became_peered":"2024-01-24T14:13:27.302240+0000","last_unstale":"2024-01-24T14:13:27.304540+0000","last_undegraded":"2024-01-24T14:13:27.304540+0000","last_fullsized":"2024-01-24T14:13:27.304540+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.288042+0000","last_change":"2024-01-24T14:13:26.887792+0000","last_active":"2024-01-24T14:13:27.288042+0000","last_peered":"2024-01-24T14:13:27.288042+0000","last_clean":"2024-01-24T14:13:27.288042+0000","last_became_active":"2024-01-24T14:13:26.887467+0000","last_became_peered":"2024-01-24T14:13:26.887467+0000","last_unstale":"2024-01-24T14:13:27.288042+0000","last_undegraded":"2024-01-24T14:13:27.288042+0000","last_fullsized":"2024-01-24T14:13:27.288042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.310856+0000","last_change":"2024-01-24T14:13:27.307430+0000","last_active":"2024-01-24T14:13:27.310856+0000","last_peered":"2024-01-24T14:13:27.310856+0000","last_clean":"2024-01-24T14:13:27.310856+0000","last_became_active":"2024-01-24T14:13:27.307133+0000","last_became_peered":"2024-01-24T14:13:27.307133+0000","last_unstale":"2024-01-24T14:13:27.310856+0000","last_undegraded":"2024-01-24T14:13:27.310856+0000","last_fullsized":"2024-01-24T14:13:27.310856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297267+0000","last_change":"2024-01-24T14:13:26.878331+0000","last_active":"2024-01-24T14:13:27.297267+0000","last_peered":"2024-01-24T14:13:27.297267+0000","last_clean":"2024-01-24T14:13:27.297267+0000","last_became_active":"2024-01-24T14:13:26.878088+0000","last_became_peered":"2024-01-24T14:13:26.878088+0000","last_unstale":"2024-01-24T14:13:27.297267+0000","last_undegraded":"2024-01-24T14:13:27.297267+0000","last_fullsized":"2024-01-24T14:13:27.297267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.301882+0000","last_change":"2024-01-24T14:13:27.294844+0000","last_active":"2024-01-24T14:13:27.301882+0000","last_peered":"2024-01-24T14:13:27.301882+0000","last_clean":"2024-01-24T14:13:27.301882+0000","last_became_active":"2024-01-24T14:13:27.294558+0000","last_became_peered":"2024-01-24T14:13:27.294558+0000","last_unstale":"2024-01-24T14:13:27.301882+0000","last_undegraded":"2024-01-24T14:13:27.301882+0000","last_fullsized":"2024-01-24T14:13:27.301882+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.283980+0000","last_change":"2024-01-24T14:13:26.874664+0000","last_active":"2024-01-24T14:13:27.283980+0000","last_peered":"2024-01-24T14:13:27.283980+0000","last_clean":"2024-01-24T14:13:27.283980+0000","last_became_active":"2024-01-24T14:13:26.874341+0000","last_became_peered":"2024-01-24T14:13:26.874341+0000","last_unstale":"2024-01-24T14:13:27.283980+0000","last_undegraded":"2024-01-24T14:13:27.283980+0000","last_fullsized":"2024-01-24T14:13:27.283980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311838+0000","last_change":"2024-01-24T14:13:27.305933+0000","last_active":"2024-01-24T14:13:27.311838+0000","last_peered":"2024-01-24T14:13:27.311838+0000","last_clean":"2024-01-24T14:13:27.311838+0000","last_became_active":"2024-01-24T14:13:27.305404+0000","last_became_peered":"2024-01-24T14:13:27.305404+0000","last_unstale":"2024-01-24T14:13:27.311838+0000","last_undegraded":"2024-01-24T14:13:27.311838+0000","last_fullsized":"2024-01-24T14:13:27.311838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.280388+0000","last_change":"2024-01-24T14:13:26.875765+0000","last_active":"2024-01-24T14:13:27.280388+0000","last_peered":"2024-01-24T14:13:27.280388+0000","last_clean":"2024-01-24T14:13:27.280388+0000","last_became_active":"2024-01-24T14:13:26.875459+0000","last_became_peered":"2024-01-24T14:13:26.875459+0000","last_unstale":"2024-01-24T14:13:27.280388+0000","last_undegraded":"2024-01-24T14:13:27.280388+0000","last_fullsized":"2024-01-24T14:13:27.280388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294465+0000","last_change":"2024-01-24T14:13:27.291235+0000","last_active":"2024-01-24T14:13:27.294465+0000","last_peered":"2024-01-24T14:13:27.294465+0000","last_clean":"2024-01-24T14:13:27.294465+0000","last_became_active":"2024-01-24T14:13:27.290611+0000","last_became_peered":"2024-01-24T14:13:27.290611+0000","last_unstale":"2024-01-24T14:13:27.294465+0000","last_undegraded":"2024-01-24T14:13:27.294465+0000","last_fullsized":"2024-01-24T14:13:27.294465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.287812+0000","last_change":"2024-01-24T14:13:26.877427+0000","last_active":"2024-01-24T14:13:27.287812+0000","last_peered":"2024-01-24T14:13:27.287812+0000","last_clean":"2024-01-24T14:13:27.287812+0000","last_became_active":"2024-01-24T14:13:26.876994+0000","last_became_peered":"2024-01-24T14:13:26.876994+0000","last_unstale":"2024-01-24T14:13:27.287812+0000","last_undegraded":"2024-01-24T14:13:27.287812+0000","last_fullsized":"2024-01-24T14:13:27.287812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311381+0000","last_change":"2024-01-24T14:13:27.310488+0000","last_active":"2024-01-24T14:13:27.311381+0000","last_peered":"2024-01-24T14:13:27.311381+0000","last_clean":"2024-01-24T14:13:27.311381+0000","last_became_active":"2024-01-24T14:13:27.310104+0000","last_became_peered":"2024-01-24T14:13:27.310104+0000","last_unstale":"2024-01-24T14:13:27.311381+0000","last_undegraded":"2024-01-24T14:13:27.311381+0000","last_fullsized":"2024-01-24T14:13:27.311381+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.308731+0000","last_change":"2024-01-24T14:13:26.881790+0000","last_active":"2024-01-24T14:13:27.308731+0000","last_peered":"2024-01-24T14:13:27.308731+0000","last_clean":"2024-01-24T14:13:27.308731+0000","last_became_active":"2024-01-24T14:13:26.880948+0000","last_became_peered":"2024-01-24T14:13:26.880948+0000","last_unstale":"2024-01-24T14:13:27.308731+0000","last_undegraded":"2024-01-24T14:13:27.308731+0000","last_fullsized":"2024-01-24T14:13:27.308731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.304843+0000","last_change":"2024-01-24T14:13:27.299897+0000","last_active":"2024-01-24T14:13:27.304843+0000","last_peered":"2024-01-24T14:13:27.304843+0000","last_clean":"2024-01-24T14:13:27.304843+0000","last_became_active":"2024-01-24T14:13:27.299523+0000","last_became_peered":"2024-01-24T14:13:27.299523+0000","last_unstale":"2024-01-24T14:13:27.304843+0000","last_undegraded":"2024-01-24T14:13:27.304843+0000","last_fullsized":"2024-01-24T14:13:27.304843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294537+0000","last_change":"2024-01-24T14:13:26.875673+0000","last_active":"2024-01-24T14:13:27.294537+0000","last_peered":"2024-01-24T14:13:27.294537+0000","last_clean":"2024-01-24T14:13:27.294537+0000","last_became_active":"2024-01-24T14:13:26.875271+0000","last_became_peered":"2024-01-24T14:13:26.875271+0000","last_unstale":"2024-01-24T14:13:27.294537+0000","last_undegraded":"2024-01-24T14:13:27.294537+0000","last_fullsized":"2024-01-24T14:13:27.294537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309595+0000","last_change":"2024-01-24T14:13:27.301977+0000","last_active":"2024-01-24T14:13:27.309595+0000","last_peered":"2024-01-24T14:13:27.309595+0000","last_clean":"2024-01-24T14:13:27.309595+0000","last_became_active":"2024-01-24T14:13:27.301693+0000","last_became_peered":"2024-01-24T14:13:27.301693+0000","last_unstale":"2024-01-24T14:13:27.309595+0000","last_undegraded":"2024-01-24T14:13:27.309595+0000","last_fullsized":"2024-01-24T14:13:27.309595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297016+0000","last_change":"2024-01-24T14:13:26.886887+0000","last_active":"2024-01-24T14:13:27.297016+0000","last_peered":"2024-01-24T14:13:27.297016+0000","last_clean":"2024-01-24T14:13:27.297016+0000","last_became_active":"2024-01-24T14:13:26.886589+0000","last_became_peered":"2024-01-24T14:13:26.886589+0000","last_unstale":"2024-01-24T14:13:27.297016+0000","last_undegraded":"2024-01-24T14:13:27.297016+0000","last_fullsized":"2024-01-24T14:13:27.297016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.310490+0000","last_change":"2024-01-24T14:13:27.306113+0000","last_active":"2024-01-24T14:13:27.310490+0000","last_peered":"2024-01-24T14:13:27.310490+0000","last_clean":"2024-01-24T14:13:27.310490+0000","last_became_active":"2024-01-24T14:13:27.305688+0000","last_became_peered":"2024-01-24T14:13:27.305688+0000","last_unstale":"2024-01-24T14:13:27.310490+0000","last_undegraded":"2024-01-24T14:13:27.310490+0000","last_fullsized":"2024-01-24T14:13:27.310490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.290983+0000","last_change":"2024-01-24T14:13:26.887178+0000","last_active":"2024-01-24T14:13:27.290983+0000","last_peered":"2024-01-24T14:13:27.290983+0000","last_clean":"2024-01-24T14:13:27.290983+0000","last_became_active":"2024-01-24T14:13:26.886907+0000","last_became_peered":"2024-01-24T14:13:26.886907+0000","last_unstale":"2024-01-24T14:13:27.290983+0000","last_undegraded":"2024-01-24T14:13:27.290983+0000","last_fullsized":"2024-01-24T14:13:27.290983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.304506+0000","last_change":"2024-01-24T14:13:27.298383+0000","last_active":"2024-01-24T14:13:27.304506+0000","last_peered":"2024-01-24T14:13:27.304506+0000","last_clean":"2024-01-24T14:13:27.304506+0000","last_became_active":"2024-01-24T14:13:27.298054+0000","last_became_peered":"2024-01-24T14:13:27.298054+0000","last_unstale":"2024-01-24T14:13:27.304506+0000","last_undegraded":"2024-01-24T14:13:27.304506+0000","last_fullsized":"2024-01-24T14:13:27.304506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.282888+0000","last_change":"2024-01-24T14:13:26.874605+0000","last_active":"2024-01-24T14:13:27.282888+0000","last_peered":"2024-01-24T14:13:27.282888+0000","last_clean":"2024-01-24T14:13:27.282888+0000","last_became_active":"2024-01-24T14:13:26.874252+0000","last_became_peered":"2024-01-24T14:13:26.874252+0000","last_unstale":"2024-01-24T14:13:27.282888+0000","last_undegraded":"2024-01-24T14:13:27.282888+0000","last_fullsized":"2024-01-24T14:13:27.282888+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.306417+0000","last_change":"2024-01-24T14:13:27.300564+0000","last_active":"2024-01-24T14:13:27.306417+0000","last_peered":"2024-01-24T14:13:27.306417+0000","last_clean":"2024-01-24T14:13:27.306417+0000","last_became_active":"2024-01-24T14:13:27.300246+0000","last_became_peered":"2024-01-24T14:13:27.300246+0000","last_unstale":"2024-01-24T14:13:27.306417+0000","last_undegraded":"2024-01-24T14:13:27.306417+0000","last_fullsized":"2024-01-24T14:13:27.306417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291350+0000","last_change":"2024-01-24T14:13:26.886129+0000","last_active":"2024-01-24T14:13:27.291350+0000","last_peered":"2024-01-24T14:13:27.291350+0000","last_clean":"2024-01-24T14:13:27.291350+0000","last_became_active":"2024-01-24T14:13:26.885874+0000","last_became_peered":"2024-01-24T14:13:26.885874+0000","last_unstale":"2024-01-24T14:13:27.291350+0000","last_undegraded":"2024-01-24T14:13:27.291350+0000","last_fullsized":"2024-01-24T14:13:27.291350+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.286694+0000","last_change":"2024-01-24T14:13:27.282826+0000","last_active":"2024-01-24T14:13:27.286694+0000","last_peered":"2024-01-24T14:13:27.286694+0000","last_clean":"2024-01-24T14:13:27.286694+0000","last_became_active":"2024-01-24T14:13:27.282543+0000","last_became_peered":"2024-01-24T14:13:27.282543+0000","last_unstale":"2024-01-24T14:13:27.286694+0000","last_undegraded":"2024-01-24T14:13:27.286694+0000","last_fullsized":"2024-01-24T14:13:27.286694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.287597+0000","last_change":"2024-01-24T14:13:26.870366+0000","last_active":"2024-01-24T14:13:27.287597+0000","last_peered":"2024-01-24T14:13:27.287597+0000","last_clean":"2024-01-24T14:13:27.287597+0000","last_became_active":"2024-01-24T14:13:26.870053+0000","last_became_peered":"2024-01-24T14:13:26.870053+0000","last_unstale":"2024-01-24T14:13:27.287597+0000","last_undegraded":"2024-01-24T14:13:27.287597+0000","last_fullsized":"2024-01-24T14:13:27.287597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.302760+0000","last_change":"2024-01-24T14:13:27.300752+0000","last_active":"2024-01-24T14:13:27.302760+0000","last_peered":"2024-01-24T14:13:27.302760+0000","last_clean":"2024-01-24T14:13:27.302760+0000","last_became_active":"2024-01-24T14:13:27.300470+0000","last_became_peered":"2024-01-24T14:13:27.300470+0000","last_unstale":"2024-01-24T14:13:27.302760+0000","last_undegraded":"2024-01-24T14:13:27.302760+0000","last_fullsized":"2024-01-24T14:13:27.302760+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.275185+0000","last_change":"2024-01-24T14:13:26.880676+0000","last_active":"2024-01-24T14:13:27.275185+0000","last_peered":"2024-01-24T14:13:27.275185+0000","last_clean":"2024-01-24T14:13:27.275185+0000","last_became_active":"2024-01-24T14:13:26.880329+0000","last_became_peered":"2024-01-24T14:13:26.880329+0000","last_unstale":"2024-01-24T14:13:27.275185+0000","last_undegraded":"2024-01-24T14:13:27.275185+0000","last_fullsized":"2024-01-24T14:13:27.275185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.304335+0000","last_change":"2024-01-24T14:13:27.301690+0000","last_active":"2024-01-24T14:13:27.304335+0000","last_peered":"2024-01-24T14:13:27.304335+0000","last_clean":"2024-01-24T14:13:27.304335+0000","last_became_active":"2024-01-24T14:13:27.301406+0000","last_became_peered":"2024-01-24T14:13:27.301406+0000","last_unstale":"2024-01-24T14:13:27.304335+0000","last_undegraded":"2024-01-24T14:13:27.304335+0000","last_fullsized":"2024-01-24T14:13:27.304335+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.284863+0000","last_change":"2024-01-24T14:13:26.875798+0000","last_active":"2024-01-24T14:13:27.284863+0000","last_peered":"2024-01-24T14:13:27.284863+0000","last_clean":"2024-01-24T14:13:27.284863+0000","last_became_active":"2024-01-24T14:13:26.875521+0000","last_became_peered":"2024-01-24T14:13:26.875521+0000","last_unstale":"2024-01-24T14:13:27.284863+0000","last_undegraded":"2024-01-24T14:13:27.284863+0000","last_fullsized":"2024-01-24T14:13:27.284863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311564+0000","last_change":"2024-01-24T14:13:27.310241+0000","last_active":"2024-01-24T14:13:27.311564+0000","last_peered":"2024-01-24T14:13:27.311564+0000","last_clean":"2024-01-24T14:13:27.311564+0000","last_became_active":"2024-01-24T14:13:27.309931+0000","last_became_peered":"2024-01-24T14:13:27.309931+0000","last_unstale":"2024-01-24T14:13:27.311564+0000","last_undegraded":"2024-01-24T14:13:27.311564+0000","last_fullsized":"2024-01-24T14:13:27.311564+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294670+0000","last_change":"2024-01-24T14:13:26.886368+0000","last_active":"2024-01-24T14:13:27.294670+0000","last_peered":"2024-01-24T14:13:27.294670+0000","last_clean":"2024-01-24T14:13:27.294670+0000","last_became_active":"2024-01-24T14:13:26.886063+0000","last_became_peered":"2024-01-24T14:13:26.886063+0000","last_unstale":"2024-01-24T14:13:27.294670+0000","last_undegraded":"2024-01-24T14:13:27.294670+0000","last_fullsized":"2024-01-24T14:13:27.294670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311915+0000","last_change":"2024-01-24T14:13:27.307211+0000","last_active":"2024-01-24T14:13:27.311915+0000","last_peered":"2024-01-24T14:13:27.311915+0000","last_clean":"2024-01-24T14:13:27.311915+0000","last_became_active":"2024-01-24T14:13:27.306472+0000","last_became_peered":"2024-01-24T14:13:27.306472+0000","last_unstale":"2024-01-24T14:13:27.311915+0000","last_undegraded":"2024-01-24T14:13:27.311915+0000","last_fullsized":"2024-01-24T14:13:27.311915+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"89'3","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.299541+0000","last_change":"2024-01-24T14:13:26.874205+0000","last_active":"2024-01-24T14:13:27.299541+0000","last_peered":"2024-01-24T14:13:27.299541+0000","last_clean":"2024-01-24T14:13:27.299541+0000","last_became_active":"2024-01-24T14:13:26.873885+0000","last_became_peered":"2024-01-24T14:13:26.873885+0000","last_unstale":"2024-01-24T14:13:27.299541+0000","last_undegraded":"2024-01-24T14:13:27.299541+0000","last_fullsized":"2024-01-24T14:13:27.299541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.292952+0000","last_change":"2024-01-24T14:13:27.291406+0000","last_active":"2024-01-24T14:13:27.292952+0000","last_peered":"2024-01-24T14:13:27.292952+0000","last_clean":"2024-01-24T14:13:27.292952+0000","last_became_active":"2024-01-24T14:13:27.291134+0000","last_became_peered":"2024-01-24T14:13:27.291134+0000","last_unstale":"2024-01-24T14:13:27.292952+0000","last_undegraded":"2024-01-24T14:13:27.292952+0000","last_fullsized":"2024-01-24T14:13:27.292952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.306312+0000","last_change":"2024-01-24T14:13:26.876317+0000","last_active":"2024-01-24T14:13:27.306312+0000","last_peered":"2024-01-24T14:13:27.306312+0000","last_clean":"2024-01-24T14:13:27.306312+0000","last_became_active":"2024-01-24T14:13:26.876022+0000","last_became_peered":"2024-01-24T14:13:26.876022+0000","last_unstale":"2024-01-24T14:13:27.306312+0000","last_undegraded":"2024-01-24T14:13:27.306312+0000","last_fullsized":"2024-01-24T14:13:27.306312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.307861+0000","last_change":"2024-01-24T14:13:27.302963+0000","last_active":"2024-01-24T14:13:27.307861+0000","last_peered":"2024-01-24T14:13:27.307861+0000","last_clean":"2024-01-24T14:13:27.307861+0000","last_became_active":"2024-01-24T14:13:27.302657+0000","last_became_peered":"2024-01-24T14:13:27.302657+0000","last_unstale":"2024-01-24T14:13:27.307861+0000","last_undegraded":"2024-01-24T14:13:27.307861+0000","last_fullsized":"2024-01-24T14:13:27.307861+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.305091+0000","last_change":"2024-01-24T14:13:26.881920+0000","last_active":"2024-01-24T14:13:27.305091+0000","last_peered":"2024-01-24T14:13:27.305091+0000","last_clean":"2024-01-24T14:13:27.305091+0000","last_became_active":"2024-01-24T14:13:26.881589+0000","last_became_peered":"2024-01-24T14:13:26.881589+0000","last_unstale":"2024-01-24T14:13:27.305091+0000","last_undegraded":"2024-01-24T14:13:27.305091+0000","last_fullsized":"2024-01-24T14:13:27.305091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294376+0000","last_change":"2024-01-24T14:13:27.290720+0000","last_active":"2024-01-24T14:13:27.294376+0000","last_peered":"2024-01-24T14:13:27.294376+0000","last_clean":"2024-01-24T14:13:27.294376+0000","last_became_active":"2024-01-24T14:13:27.290431+0000","last_became_peered":"2024-01-24T14:13:27.290431+0000","last_unstale":"2024-01-24T14:13:27.294376+0000","last_undegraded":"2024-01-24T14:13:27.294376+0000","last_fullsized":"2024-01-24T14:13:27.294376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291466+0000","last_change":"2024-01-24T14:13:26.879811+0000","last_active":"2024-01-24T14:13:27.291466+0000","last_peered":"2024-01-24T14:13:27.291466+0000","last_clean":"2024-01-24T14:13:27.291466+0000","last_became_active":"2024-01-24T14:13:26.879552+0000","last_became_peered":"2024-01-24T14:13:26.879552+0000","last_unstale":"2024-01-24T14:13:27.291466+0000","last_undegraded":"2024-01-24T14:13:27.291466+0000","last_fullsized":"2024-01-24T14:13:27.291466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.301595+0000","last_change":"2024-01-24T14:13:27.297557+0000","last_active":"2024-01-24T14:13:27.301595+0000","last_peered":"2024-01-24T14:13:27.301595+0000","last_clean":"2024-01-24T14:13:27.301595+0000","last_became_active":"2024-01-24T14:13:27.297259+0000","last_became_peered":"2024-01-24T14:13:27.297259+0000","last_unstale":"2024-01-24T14:13:27.301595+0000","last_undegraded":"2024-01-24T14:13:27.301595+0000","last_fullsized":"2024-01-24T14:13:27.301595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.280052+0000","last_change":"2024-01-24T14:13:26.875603+0000","last_active":"2024-01-24T14:13:27.280052+0000","last_peered":"2024-01-24T14:13:27.280052+0000","last_clean":"2024-01-24T14:13:27.280052+0000","last_became_active":"2024-01-24T14:13:26.875275+0000","last_became_peered":"2024-01-24T14:13:26.875275+0000","last_unstale":"2024-01-24T14:13:27.280052+0000","last_undegraded":"2024-01-24T14:13:27.280052+0000","last_fullsized":"2024-01-24T14:13:27.280052+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297779+0000","last_change":"2024-01-24T14:13:27.296404+0000","last_active":"2024-01-24T14:13:27.297779+0000","last_peered":"2024-01-24T14:13:27.297779+0000","last_clean":"2024-01-24T14:13:27.297779+0000","last_became_active":"2024-01-24T14:13:27.296065+0000","last_became_peered":"2024-01-24T14:13:27.296065+0000","last_unstale":"2024-01-24T14:13:27.297779+0000","last_undegraded":"2024-01-24T14:13:27.297779+0000","last_fullsized":"2024-01-24T14:13:27.297779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"89'4","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.297121+0000","last_change":"2024-01-24T14:13:26.879963+0000","last_active":"2024-01-24T14:13:27.297121+0000","last_peered":"2024-01-24T14:13:27.297121+0000","last_clean":"2024-01-24T14:13:27.297121+0000","last_became_active":"2024-01-24T14:13:26.877231+0000","last_became_peered":"2024-01-24T14:13:26.877231+0000","last_unstale":"2024-01-24T14:13:27.297121+0000","last_undegraded":"2024-01-24T14:13:27.297121+0000","last_fullsized":"2024-01-24T14:13:27.297121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296451+0000","last_change":"2024-01-24T14:13:27.291898+0000","last_active":"2024-01-24T14:13:27.296451+0000","last_peered":"2024-01-24T14:13:27.296451+0000","last_clean":"2024-01-24T14:13:27.296451+0000","last_became_active":"2024-01-24T14:13:27.291619+0000","last_became_peered":"2024-01-24T14:13:27.291619+0000","last_unstale":"2024-01-24T14:13:27.296451+0000","last_undegraded":"2024-01-24T14:13:27.296451+0000","last_fullsized":"2024-01-24T14:13:27.296451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309294+0000","last_change":"2024-01-24T14:13:26.885373+0000","last_active":"2024-01-24T14:13:27.309294+0000","last_peered":"2024-01-24T14:13:27.309294+0000","last_clean":"2024-01-24T14:13:27.309294+0000","last_became_active":"2024-01-24T14:13:26.884871+0000","last_became_peered":"2024-01-24T14:13:26.884871+0000","last_unstale":"2024-01-24T14:13:27.309294+0000","last_undegraded":"2024-01-24T14:13:27.309294+0000","last_fullsized":"2024-01-24T14:13:27.309294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294474+0000","last_change":"2024-01-24T14:13:27.291973+0000","last_active":"2024-01-24T14:13:27.294474+0000","last_peered":"2024-01-24T14:13:27.294474+0000","last_clean":"2024-01-24T14:13:27.294474+0000","last_became_active":"2024-01-24T14:13:27.291681+0000","last_became_peered":"2024-01-24T14:13:27.291681+0000","last_unstale":"2024-01-24T14:13:27.294474+0000","last_undegraded":"2024-01-24T14:13:27.294474+0000","last_fullsized":"2024-01-24T14:13:27.294474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.281950+0000","last_change":"2024-01-24T14:13:26.882217+0000","last_active":"2024-01-24T14:13:27.281950+0000","last_peered":"2024-01-24T14:13:27.281950+0000","last_clean":"2024-01-24T14:13:27.281950+0000","last_became_active":"2024-01-24T14:13:26.881903+0000","last_became_peered":"2024-01-24T14:13:26.881903+0000","last_unstale":"2024-01-24T14:13:27.281950+0000","last_undegraded":"2024-01-24T14:13:27.281950+0000","last_fullsized":"2024-01-24T14:13:27.281950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311402+0000","last_change":"2024-01-24T14:13:27.310356+0000","last_active":"2024-01-24T14:13:27.311402+0000","last_peered":"2024-01-24T14:13:27.311402+0000","last_clean":"2024-01-24T14:13:27.311402+0000","last_became_active":"2024-01-24T14:13:27.309988+0000","last_became_peered":"2024-01-24T14:13:27.309988+0000","last_unstale":"2024-01-24T14:13:27.311402+0000","last_undegraded":"2024-01-24T14:13:27.311402+0000","last_fullsized":"2024-01-24T14:13:27.311402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.277450+0000","last_change":"2024-01-24T14:13:26.877334+0000","last_active":"2024-01-24T14:13:27.277450+0000","last_peered":"2024-01-24T14:13:27.277450+0000","last_clean":"2024-01-24T14:13:27.277450+0000","last_became_active":"2024-01-24T14:13:26.876747+0000","last_became_peered":"2024-01-24T14:13:26.876747+0000","last_unstale":"2024-01-24T14:13:27.277450+0000","last_undegraded":"2024-01-24T14:13:27.277450+0000","last_fullsized":"2024-01-24T14:13:27.277450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.310472+0000","last_change":"2024-01-24T14:13:27.305953+0000","last_active":"2024-01-24T14:13:27.310472+0000","last_peered":"2024-01-24T14:13:27.310472+0000","last_clean":"2024-01-24T14:13:27.310472+0000","last_became_active":"2024-01-24T14:13:27.305324+0000","last_became_peered":"2024-01-24T14:13:27.305324+0000","last_unstale":"2024-01-24T14:13:27.310472+0000","last_undegraded":"2024-01-24T14:13:27.310472+0000","last_fullsized":"2024-01-24T14:13:27.310472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.288116+0000","last_change":"2024-01-24T14:13:26.876390+0000","last_active":"2024-01-24T14:13:27.288116+0000","last_peered":"2024-01-24T14:13:27.288116+0000","last_clean":"2024-01-24T14:13:27.288116+0000","last_became_active":"2024-01-24T14:13:26.876051+0000","last_became_peered":"2024-01-24T14:13:26.876051+0000","last_unstale":"2024-01-24T14:13:27.288116+0000","last_undegraded":"2024-01-24T14:13:27.288116+0000","last_fullsized":"2024-01-24T14:13:27.288116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.303823+0000","last_change":"2024-01-24T14:13:27.303021+0000","last_active":"2024-01-24T14:13:27.303823+0000","last_peered":"2024-01-24T14:13:27.303823+0000","last_clean":"2024-01-24T14:13:27.303823+0000","last_became_active":"2024-01-24T14:13:27.302612+0000","last_became_peered":"2024-01-24T14:13:27.302612+0000","last_unstale":"2024-01-24T14:13:27.303823+0000","last_undegraded":"2024-01-24T14:13:27.303823+0000","last_fullsized":"2024-01-24T14:13:27.303823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.276291+0000","last_change":"2024-01-24T14:13:26.862510+0000","last_active":"2024-01-24T14:13:27.276291+0000","last_peered":"2024-01-24T14:13:27.276291+0000","last_clean":"2024-01-24T14:13:27.276291+0000","last_became_active":"2024-01-24T14:13:26.862021+0000","last_became_peered":"2024-01-24T14:13:26.862021+0000","last_unstale":"2024-01-24T14:13:27.276291+0000","last_undegraded":"2024-01-24T14:13:27.276291+0000","last_fullsized":"2024-01-24T14:13:27.276291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.309470+0000","last_change":"2024-01-24T14:13:27.301052+0000","last_active":"2024-01-24T14:13:27.309470+0000","last_peered":"2024-01-24T14:13:27.309470+0000","last_clean":"2024-01-24T14:13:27.309470+0000","last_became_active":"2024-01-24T14:13:27.300765+0000","last_became_peered":"2024-01-24T14:13:27.300765+0000","last_unstale":"2024-01-24T14:13:27.309470+0000","last_undegraded":"2024-01-24T14:13:27.309470+0000","last_fullsized":"2024-01-24T14:13:27.309470+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.287494+0000","last_change":"2024-01-24T14:13:26.869656+0000","last_active":"2024-01-24T14:13:27.287494+0000","last_peered":"2024-01-24T14:13:27.287494+0000","last_clean":"2024-01-24T14:13:27.287494+0000","last_became_active":"2024-01-24T14:13:26.869313+0000","last_became_peered":"2024-01-24T14:13:26.869313+0000","last_unstale":"2024-01-24T14:13:27.287494+0000","last_undegraded":"2024-01-24T14:13:27.287494+0000","last_fullsized":"2024-01-24T14:13:27.287494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.305568+0000","last_change":"2024-01-24T14:13:27.303376+0000","last_active":"2024-01-24T14:13:27.305568+0000","last_peered":"2024-01-24T14:13:27.305568+0000","last_clean":"2024-01-24T14:13:27.305568+0000","last_became_active":"2024-01-24T14:13:27.303116+0000","last_became_peered":"2024-01-24T14:13:27.303116+0000","last_unstale":"2024-01-24T14:13:27.305568+0000","last_undegraded":"2024-01-24T14:13:27.305568+0000","last_fullsized":"2024-01-24T14:13:27.305568+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.280518+0000","last_change":"2024-01-24T14:13:26.878593+0000","last_active":"2024-01-24T14:13:27.280518+0000","last_peered":"2024-01-24T14:13:27.280518+0000","last_clean":"2024-01-24T14:13:27.280518+0000","last_became_active":"2024-01-24T14:13:26.878317+0000","last_became_peered":"2024-01-24T14:13:26.878317+0000","last_unstale":"2024-01-24T14:13:27.280518+0000","last_undegraded":"2024-01-24T14:13:27.280518+0000","last_fullsized":"2024-01-24T14:13:27.280518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.311871+0000","last_change":"2024-01-24T14:13:27.305601+0000","last_active":"2024-01-24T14:13:27.311871+0000","last_peered":"2024-01-24T14:13:27.311871+0000","last_clean":"2024-01-24T14:13:27.311871+0000","last_became_active":"2024-01-24T14:13:27.305296+0000","last_became_peered":"2024-01-24T14:13:27.305296+0000","last_unstale":"2024-01-24T14:13:27.311871+0000","last_undegraded":"2024-01-24T14:13:27.311871+0000","last_fullsized":"2024-01-24T14:13:27.311871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291589+0000","last_change":"2024-01-24T14:13:26.883714+0000","last_active":"2024-01-24T14:13:27.291589+0000","last_peered":"2024-01-24T14:13:27.291589+0000","last_clean":"2024-01-24T14:13:27.291589+0000","last_became_active":"2024-01-24T14:13:26.880281+0000","last_became_peered":"2024-01-24T14:13:26.880281+0000","last_unstale":"2024-01-24T14:13:27.291589+0000","last_undegraded":"2024-01-24T14:13:27.291589+0000","last_fullsized":"2024-01-24T14:13:27.291589+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.286956+0000","last_change":"2024-01-24T14:13:27.284503+0000","last_active":"2024-01-24T14:13:27.286956+0000","last_peered":"2024-01-24T14:13:27.286956+0000","last_clean":"2024-01-24T14:13:27.286956+0000","last_became_active":"2024-01-24T14:13:27.284214+0000","last_became_peered":"2024-01-24T14:13:27.284214+0000","last_unstale":"2024-01-24T14:13:27.286956+0000","last_undegraded":"2024-01-24T14:13:27.286956+0000","last_fullsized":"2024-01-24T14:13:27.286956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296289+0000","last_change":"2024-01-24T14:13:26.878667+0000","last_active":"2024-01-24T14:13:27.296289+0000","last_peered":"2024-01-24T14:13:27.296289+0000","last_clean":"2024-01-24T14:13:27.296289+0000","last_became_active":"2024-01-24T14:13:26.878027+0000","last_became_peered":"2024-01-24T14:13:26.878027+0000","last_unstale":"2024-01-24T14:13:27.296289+0000","last_undegraded":"2024-01-24T14:13:27.296289+0000","last_fullsized":"2024-01-24T14:13:27.296289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.288182+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:13:27.288182+0000","last_peered":"2024-01-24T14:13:27.288182+0000","last_clean":"2024-01-24T14:13:27.288182+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:13:27.288182+0000","last_undegraded":"2024-01-24T14:13:27.288182+0000","last_fullsized":"2024-01-24T14:13:27.288182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.301860+0000","last_change":"2024-01-24T14:13:27.293092+0000","last_active":"2024-01-24T14:13:27.301860+0000","last_peered":"2024-01-24T14:13:27.301860+0000","last_clean":"2024-01-24T14:13:27.301860+0000","last_became_active":"2024-01-24T14:13:27.292806+0000","last_became_peered":"2024-01-24T14:13:27.292806+0000","last_unstale":"2024-01-24T14:13:27.301860+0000","last_undegraded":"2024-01-24T14:13:27.301860+0000","last_fullsized":"2024-01-24T14:13:27.301860+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291588+0000","last_change":"2024-01-24T14:13:26.879405+0000","last_active":"2024-01-24T14:13:27.291588+0000","last_peered":"2024-01-24T14:13:27.291588+0000","last_clean":"2024-01-24T14:13:27.291588+0000","last_became_active":"2024-01-24T14:13:26.879151+0000","last_became_peered":"2024-01-24T14:13:26.879151+0000","last_unstale":"2024-01-24T14:13:27.291588+0000","last_undegraded":"2024-01-24T14:13:27.291588+0000","last_fullsized":"2024-01-24T14:13:27.291588+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.284012+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:13:27.284012+0000","last_peered":"2024-01-24T14:13:27.284012+0000","last_clean":"2024-01-24T14:13:27.284012+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:13:27.284012+0000","last_undegraded":"2024-01-24T14:13:27.284012+0000","last_fullsized":"2024-01-24T14:13:27.284012+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.304515+0000","last_change":"2024-01-24T14:13:27.302815+0000","last_active":"2024-01-24T14:13:27.304515+0000","last_peered":"2024-01-24T14:13:27.304515+0000","last_clean":"2024-01-24T14:13:27.304515+0000","last_became_active":"2024-01-24T14:13:27.302518+0000","last_became_peered":"2024-01-24T14:13:27.302518+0000","last_unstale":"2024-01-24T14:13:27.304515+0000","last_undegraded":"2024-01-24T14:13:27.304515+0000","last_fullsized":"2024-01-24T14:13:27.304515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.280170+0000","last_change":"2024-01-24T14:13:26.873558+0000","last_active":"2024-01-24T14:13:27.280170+0000","last_peered":"2024-01-24T14:13:27.280170+0000","last_clean":"2024-01-24T14:13:27.280170+0000","last_became_active":"2024-01-24T14:13:26.873317+0000","last_became_peered":"2024-01-24T14:13:26.873317+0000","last_unstale":"2024-01-24T14:13:27.280170+0000","last_undegraded":"2024-01-24T14:13:27.280170+0000","last_fullsized":"2024-01-24T14:13:27.280170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.274928+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:13:27.274928+0000","last_peered":"2024-01-24T14:13:27.274928+0000","last_clean":"2024-01-24T14:13:27.274928+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:13:27.274928+0000","last_undegraded":"2024-01-24T14:13:27.274928+0000","last_fullsized":"2024-01-24T14:13:27.274928+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.308693+0000","last_change":"2024-01-24T14:13:27.306012+0000","last_active":"2024-01-24T14:13:27.308693+0000","last_peered":"2024-01-24T14:13:27.308693+0000","last_clean":"2024-01-24T14:13:27.308693+0000","last_became_active":"2024-01-24T14:13:27.305708+0000","last_became_peered":"2024-01-24T14:13:27.305708+0000","last_unstale":"2024-01-24T14:13:27.308693+0000","last_undegraded":"2024-01-24T14:13:27.308693+0000","last_fullsized":"2024-01-24T14:13:27.308693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.307746+0000","last_change":"2024-01-24T14:13:26.884032+0000","last_active":"2024-01-24T14:13:27.307746+0000","last_peered":"2024-01-24T14:13:27.307746+0000","last_clean":"2024-01-24T14:13:27.307746+0000","last_became_active":"2024-01-24T14:13:26.880397+0000","last_became_peered":"2024-01-24T14:13:26.880397+0000","last_unstale":"2024-01-24T14:13:27.307746+0000","last_undegraded":"2024-01-24T14:13:27.307746+0000","last_fullsized":"2024-01-24T14:13:27.307746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.282208+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:13:27.282208+0000","last_peered":"2024-01-24T14:13:27.282208+0000","last_clean":"2024-01-24T14:13:27.282208+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:13:27.282208+0000","last_undegraded":"2024-01-24T14:13:27.282208+0000","last_fullsized":"2024-01-24T14:13:27.282208+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294768+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:13:27.294768+0000","last_peered":"2024-01-24T14:13:27.294768+0000","last_clean":"2024-01-24T14:13:27.294768+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:13:27.294768+0000","last_undegraded":"2024-01-24T14:13:27.294768+0000","last_fullsized":"2024-01-24T14:13:27.294768+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.301794+0000","last_change":"2024-01-24T14:13:27.295986+0000","last_active":"2024-01-24T14:13:27.301794+0000","last_peered":"2024-01-24T14:13:27.301794+0000","last_clean":"2024-01-24T14:13:27.301794+0000","last_became_active":"2024-01-24T14:13:27.295787+0000","last_became_peered":"2024-01-24T14:13:27.295787+0000","last_unstale":"2024-01-24T14:13:27.301794+0000","last_undegraded":"2024-01-24T14:13:27.301794+0000","last_fullsized":"2024-01-24T14:13:27.301794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.290979+0000","last_change":"2024-01-24T14:13:26.877280+0000","last_active":"2024-01-24T14:13:27.290979+0000","last_peered":"2024-01-24T14:13:27.290979+0000","last_clean":"2024-01-24T14:13:27.290979+0000","last_became_active":"2024-01-24T14:13:26.876588+0000","last_became_peered":"2024-01-24T14:13:26.876588+0000","last_unstale":"2024-01-24T14:13:27.290979+0000","last_undegraded":"2024-01-24T14:13:27.290979+0000","last_fullsized":"2024-01-24T14:13:27.290979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.294605+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:13:27.294605+0000","last_peered":"2024-01-24T14:13:27.294605+0000","last_clean":"2024-01-24T14:13:27.294605+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:13:27.294605+0000","last_undegraded":"2024-01-24T14:13:27.294605+0000","last_fullsized":"2024-01-24T14:13:27.294605+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.305222+0000","last_change":"2024-01-24T14:13:27.294273+0000","last_active":"2024-01-24T14:13:27.305222+0000","last_peered":"2024-01-24T14:13:27.305222+0000","last_clean":"2024-01-24T14:13:27.305222+0000","last_became_active":"2024-01-24T14:13:27.293842+0000","last_became_peered":"2024-01-24T14:13:27.293842+0000","last_unstale":"2024-01-24T14:13:27.305222+0000","last_undegraded":"2024-01-24T14:13:27.305222+0000","last_fullsized":"2024-01-24T14:13:27.305222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.292440+0000","last_change":"2024-01-24T14:13:26.884832+0000","last_active":"2024-01-24T14:13:27.292440+0000","last_peered":"2024-01-24T14:13:27.292440+0000","last_clean":"2024-01-24T14:13:27.292440+0000","last_became_active":"2024-01-24T14:13:26.884544+0000","last_became_peered":"2024-01-24T14:13:26.884544+0000","last_unstale":"2024-01-24T14:13:27.292440+0000","last_undegraded":"2024-01-24T14:13:27.292440+0000","last_fullsized":"2024-01-24T14:13:27.292440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.296740+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:13:27.296740+0000","last_peered":"2024-01-24T14:13:27.296740+0000","last_clean":"2024-01-24T14:13:27.296740+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:13:27.296740+0000","last_undegraded":"2024-01-24T14:13:27.296740+0000","last_fullsized":"2024-01-24T14:13:27.296740+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.310896+0000","last_change":"2024-01-24T14:13:27.308585+0000","last_active":"2024-01-24T14:13:27.310896+0000","last_peered":"2024-01-24T14:13:27.310896+0000","last_clean":"2024-01-24T14:13:27.310896+0000","last_became_active":"2024-01-24T14:13:27.308187+0000","last_became_peered":"2024-01-24T14:13:27.308187+0000","last_unstale":"2024-01-24T14:13:27.310896+0000","last_undegraded":"2024-01-24T14:13:27.310896+0000","last_fullsized":"2024-01-24T14:13:27.310896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.306557+0000","last_change":"2024-01-24T14:13:26.886922+0000","last_active":"2024-01-24T14:13:27.306557+0000","last_peered":"2024-01-24T14:13:27.306557+0000","last_clean":"2024-01-24T14:13:27.306557+0000","last_became_active":"2024-01-24T14:13:26.886649+0000","last_became_peered":"2024-01-24T14:13:26.886649+0000","last_unstale":"2024-01-24T14:13:27.306557+0000","last_undegraded":"2024-01-24T14:13:27.306557+0000","last_fullsized":"2024-01-24T14:13:27.306557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291729+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:13:27.291729+0000","last_peered":"2024-01-24T14:13:27.291729+0000","last_clean":"2024-01-24T14:13:27.291729+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:13:27.291729+0000","last_undegraded":"2024-01-24T14:13:27.291729+0000","last_fullsized":"2024-01-24T14:13:27.291729+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.308659+0000","last_change":"2024-01-24T14:13:27.304283+0000","last_active":"2024-01-24T14:13:27.308659+0000","last_peered":"2024-01-24T14:13:27.308659+0000","last_clean":"2024-01-24T14:13:27.308659+0000","last_became_active":"2024-01-24T14:13:27.303477+0000","last_became_peered":"2024-01-24T14:13:27.303477+0000","last_unstale":"2024-01-24T14:13:27.308659+0000","last_undegraded":"2024-01-24T14:13:27.308659+0000","last_fullsized":"2024-01-24T14:13:27.308659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:26.843968+0000","last_clean_scrub_stamp":"2024-01-24T14:13:26.843968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.305213+0000","last_change":"2024-01-24T14:13:26.885513+0000","last_active":"2024-01-24T14:13:27.305213+0000","last_peered":"2024-01-24T14:13:27.305213+0000","last_clean":"2024-01-24T14:13:27.305213+0000","last_became_active":"2024-01-24T14:13:26.885133+0000","last_became_peered":"2024-01-24T14:13:26.885133+0000","last_unstale":"2024-01-24T14:13:27.305213+0000","last_undegraded":"2024-01-24T14:13:27.305213+0000","last_fullsized":"2024-01-24T14:13:27.305213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:25.838002+0000","last_clean_scrub_stamp":"2024-01-24T14:13:25.838002+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":72,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-01-24T14:13:27.291018+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:13:27.291018+0000","last_peered":"2024-01-24T14:13:27.291018+0000","last_clean":"2024-01-24T14:13:27.291018+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:13:27.291018+0000","last_undegraded":"2024-01-24T14:13:27.291018+0000","last_fullsized":"2024-01-24T14:13:27.291018+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575032,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302268,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069572,"statfs":{"total":96636764160,"available":96327241728,"internally_reserved":0,"allocated":1101824,"data_stored":597325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.46300000000000002,"5min":0.34899999999999998,"15min":0.26000000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.47699999999999998,"5min":0.41499999999999998,"15min":0.32700000000000001},"max":{"1min":0.82499999999999996,"5min":0.82599999999999996,"15min":0.83499999999999996},"last":0.64200000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60899999999999999,"15min":0.59299999999999997},"min":{"1min":0.48999999999999999,"5min":0.42299999999999999,"15min":0.312},"max":{"1min":0.85099999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.5},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.33700000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":3.2570000000000001},"last":0.55600000000000005}]},{"osd":2,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.624,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.32000000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.876},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.50900000000000001,"5min":0.39300000000000002,"15min":0.36799999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.51600000000000001,"5min":0.41899999999999998,"15min":0.36499999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.48799999999999999,"5min":0.39000000000000001,"15min":0.35499999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":3.0049999999999999},"last":0.52800000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.38100000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72599999999999998,"5min":0.755,"15min":0.84699999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.83599999999999997},"last":0.47299999999999998}]},{"osd":5,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":0.746,"5min":0.75,"15min":0.84899999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54500000000000004,"15min":0.54600000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.80900000000000005},"last":0.57799999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:12:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.81999999999999995},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58099999999999996,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.83399999999999996,"15min":0.87},"last":0.54100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575033,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302268,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069572,"statfs":{"total":96636764160,"available":96327241728,"internally_reserved":0,"allocated":1101824,"data_stored":596491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.51300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75,"5min":0.85699999999999998,"15min":0.86899999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.438,"5min":0.371,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.84899999999999998,"15min":1.3859999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:12:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.48999999999999999,"5min":0.40100000000000002,"15min":0.35399999999999998},"max":{"1min":0.73099999999999998,"5min":0.77000000000000002,"15min":1.353},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.72599999999999998,"5min":1.032,"15min":1.032},"last":0.58599999999999997}]},{"osd":2,"last update":"Wed Jan 24 14:12:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.63100000000000001},"min":{"1min":0.39500000000000002,"5min":0.39100000000000001,"15min":0.34300000000000003},"max":{"1min":0.748,"5min":0.88500000000000001,"15min":6.8129999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.60199999999999998},"min":{"1min":0.48699999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77500000000000002,"5min":0.80800000000000005,"15min":1.4079999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:12:38 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.626,"15min":0.66700000000000004},"min":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":7.016},"last":0.62},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.47199999999999998,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.79600000000000004,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.52200000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:12:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50700000000000001,"15min":0.497},"min":{"1min":0.374,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.83899999999999997,"5min":1.165,"15min":1.165},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.53100000000000003,"15min":0.54700000000000004},"min":{"1min":0.35299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.74399999999999999,"5min":0.82299999999999995,"15min":6.8719999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Wed Jan 24 14:12:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55300000000000005,"15min":0.56899999999999995},"min":{"1min":0.374,"5min":0.35399999999999998,"15min":0.32400000000000001},"max":{"1min":0.81100000000000005,"5min":0.86699999999999999,"15min":6.976},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78400000000000003,"5min":0.85699999999999998,"15min":7.8789999999999996},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed Jan 24 14:12:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75900000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.63},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.62,"15min":0.63200000000000001},"min":{"1min":0.40699999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79000000000000004,"5min":0.89600000000000002,"15min":6.9080000000000004},"last":0.67300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302276,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069564,"statfs":{"total":96636764160,"available":96327233536,"internally_reserved":0,"allocated":1110016,"data_stored":597631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.52100000000000002,"15min":0.442},"min":{"1min":0.30299999999999999,"5min":0.245,"15min":0.23400000000000001},"max":{"1min":0.629,"5min":15.298999999999999,"15min":15.298999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.55700000000000005,"15min":0.46899999999999997},"min":{"1min":0.32700000000000001,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.628,"5min":14.654,"15min":14.654},"last":0.28299999999999997}]},{"osd":2,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45700000000000002,"15min":0.44900000000000001},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45600000000000002,"15min":0.443},"min":{"1min":0.32900000000000001,"5min":0.30499999999999999,"15min":0.25800000000000001},"max":{"1min":0.60799999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.86699999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46800000000000003,"15min":0.45300000000000001},"min":{"1min":0.38500000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61899999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49199999999999999,"15min":0.47299999999999998},"min":{"1min":0.376,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.628,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.91000000000000003}]},{"osd":4,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.5,"5min":0.439,"15min":0.439},"max":{"1min":0.88100000000000001,"5min":1.073,"15min":1.137},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.51600000000000001,"5min":0.40500000000000003,"15min":0.36499999999999999},"max":{"1min":0.90000000000000002,"5min":1.218,"15min":1.218},"last":0.63500000000000001}]},{"osd":5,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":1.109,"15min":0.83099999999999996},"min":{"1min":0.46600000000000003,"5min":0.45000000000000001,"15min":0.443},"max":{"1min":0.94999999999999996,"5min":38.808,"15min":38.808},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":1.085,"15min":0.81200000000000006},"min":{"1min":0.51100000000000001,"5min":0.44900000000000001,"15min":0.40500000000000003},"max":{"1min":0.93400000000000005,"5min":38.823,"15min":38.823},"last":0.47899999999999998}]},{"osd":6,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68200000000000005,"15min":0.68100000000000005},"min":{"1min":0.55400000000000005,"5min":0.46000000000000002,"15min":0.43099999999999999},"max":{"1min":0.94299999999999995,"5min":1.0449999999999999,"15min":1.0509999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72599999999999998,"15min":0.72399999999999998},"min":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.40000000000000002},"max":{"1min":0.98199999999999998,"5min":1.1559999999999999,"15min":1.1879999999999999},"last":0.624}]},{"osd":7,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.45300000000000001},"max":{"1min":0.995,"5min":1.105,"15min":1.155},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75},"min":{"1min":0.57299999999999995,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":0.98699999999999999,"5min":1.123,"15min":1.131},"last":0.68600000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542330,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":1089536,"data_stored":595438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.25700000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.60899999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.33800000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.44500000000000001,"15min":0.434},"min":{"1min":0.30199999999999999,"5min":0.29599999999999999,"15min":0.28199999999999997},"max":{"1min":0.56599999999999995,"5min":0.64600000000000002,"15min":0.66700000000000004},"last":0.505},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.45300000000000001},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.28799999999999998},"max":{"1min":0.58899999999999997,"5min":0.754,"15min":0.79400000000000004},"last":0.61399999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44600000000000001,"15min":0.432},"min":{"1min":0.34100000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56999999999999995,"5min":1.482,"15min":1.482},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.505,"15min":0.495},"min":{"1min":0.36399999999999999,"5min":0.35399999999999998,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":0.872,"15min":0.872},"last":0.58799999999999997}]},{"osd":4,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66500000000000004,"15min":0.65300000000000002},"min":{"1min":0.53200000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.65600000000000003},"min":{"1min":0.53300000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84399999999999997,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.78700000000000003}]},{"osd":5,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.56499999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.84899999999999998,"5min":1.655,"15min":1.655},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68799999999999994,"15min":0.68300000000000005},"min":{"1min":0.497,"5min":0.46200000000000002,"15min":0.42999999999999999},"max":{"1min":0.94799999999999995,"5min":0.98599999999999999,"15min":1.0840000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.55500000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90000000000000002,"5min":1.0309999999999999,"15min":1.137},"last":0.875},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69699999999999995,"15min":0.69399999999999995},"min":{"1min":0.58599999999999997,"5min":0.48799999999999999,"15min":0.48199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.117},"last":0.80400000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71299999999999997,"15min":0.70599999999999996},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.47099999999999997},"max":{"1min":0.93999999999999995,"5min":0.96099999999999997,"15min":0.98199999999999998},"last":0.751},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.69599999999999995},"min":{"1min":0.51000000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83599999999999997,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.90500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575032,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306356,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94065484,"statfs":{"total":96636764160,"available":96323055616,"internally_reserved":0,"allocated":1093632,"data_stored":589388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.39000000000000001},"min":{"1min":0.28299999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59299999999999997,"5min":0.621,"15min":0.65800000000000003},"last":0.433},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42799999999999999},"min":{"1min":0.28599999999999998,"5min":0.26300000000000001,"15min":0.24399999999999999},"max":{"1min":0.627,"5min":0.65900000000000003,"15min":0.67100000000000004},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27300000000000002},"max":{"1min":0.57699999999999996,"5min":0.61499999999999999,"15min":0.67200000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.44500000000000001,"15min":0.45200000000000001},"min":{"1min":0.30399999999999999,"5min":0.29699999999999999,"15min":0.255},"max":{"1min":0.625,"5min":0.66700000000000004,"15min":0.67100000000000004},"last":0.53200000000000003}]},{"osd":3,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.32000000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.60999999999999999,"5min":0.67700000000000005,"15min":0.71699999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.60499999999999998,"5min":0.68000000000000005,"15min":0.72799999999999998},"last":0.70799999999999996}]},{"osd":4,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.54900000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66300000000000003,"15min":0.66600000000000004},"min":{"1min":0.52600000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.96599999999999997,"5min":0.996,"15min":1.042},"last":0.68000000000000005}]},{"osd":5,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.54200000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.0469999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.54000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.754}]},{"osd":6,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69199999999999995,"15min":0.68899999999999995},"min":{"1min":0.53100000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.1100000000000001},"last":0.625},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.58399999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.56000000000000005,"5min":0.435,"15min":0.435},"max":{"1min":0.91800000000000004,"5min":0.97299999999999998,"15min":1.054},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93000000000000005,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.69699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575032,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302268,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069572,"statfs":{"total":96636764160,"available":96327241728,"internally_reserved":0,"allocated":1101824,"data_stored":596511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.45600000000000002,"15min":0.41099999999999998},"min":{"1min":0.28000000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58499999999999996,"5min":7.593,"15min":7.593},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.48399999999999999,"15min":0.434},"min":{"1min":0.29899999999999999,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.55500000000000005,"5min":8.4239999999999995,"15min":8.4239999999999995},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.40899999999999997},"min":{"1min":0.312,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56000000000000005,"5min":0.58799999999999997,"15min":0.69099999999999995},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.45200000000000001},"min":{"1min":0.34699999999999998,"5min":0.221,"15min":0.221},"max":{"1min":0.58399999999999996,"5min":0.66300000000000003,"15min":0.68799999999999994},"last":0.35899999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.504,"15min":0.496},"min":{"1min":0.39200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.61699999999999999,"5min":0.76300000000000001,"15min":0.77200000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.38900000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.58499999999999996,"5min":0.72999999999999998,"15min":0.78800000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64700000000000002,"15min":0.63600000000000001},"min":{"1min":0.502,"5min":0.45300000000000001,"15min":0.41799999999999998},"max":{"1min":0.88,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.47499999999999998,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":0.81899999999999995,"5min":1.976,"15min":1.976},"last":0.495}]},{"osd":5,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.497,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.54400000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91200000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.55600000000000005}]},{"osd":6,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68500000000000005,"15min":0.68400000000000005},"min":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.45400000000000001},"max":{"1min":0.85699999999999998,"5min":1.53,"15min":1.53},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70599999999999996,"15min":0.68500000000000005},"min":{"1min":0.52900000000000003,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.89900000000000002,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.48499999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:12:32 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.71999999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.45200000000000001},"max":{"1min":0.91500000000000004,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73899999999999999,"15min":0.73199999999999998},"min":{"1min":0.58899999999999997,"5min":0.49199999999999999,"15min":0.45800000000000002},"max":{"1min":0.88,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.60899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302248,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069592,"statfs":{"total":96636764160,"available":96327262208,"internally_reserved":0,"allocated":1081344,"data_stored":588674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.57599999999999996},"min":{"1min":0.48799999999999999,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.88600000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.44500000000000001,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.77300000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.65500000000000003}]},{"osd":1,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.53200000000000003,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.80700000000000005,"5min":0.83099999999999996,"15min":0.85299999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72399999999999998,"5min":0.82599999999999996,"15min":1.506},"last":0.68100000000000005}]},{"osd":2,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.46600000000000003,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.78400000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.60999999999999999,"15min":0.60499999999999998},"min":{"1min":0.46300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76200000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.75600000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.51200000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.73799999999999999,"5min":0.77800000000000002,"15min":0.80600000000000005},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.501,"5min":0.41499999999999998,"15min":0.36499999999999999},"max":{"1min":0.78100000000000003,"5min":0.81000000000000005,"15min":0.86299999999999999},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48599999999999999},"min":{"1min":0.34100000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65200000000000002,"5min":0.76500000000000001,"15min":0.92000000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.498,"5min":0.49299999999999999,"15min":0.50900000000000001},"min":{"1min":0.36699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":1.556},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71299999999999997,"5min":0.78500000000000003,"15min":0.86099999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72799999999999998,"5min":0.82099999999999995,"15min":1.0409999999999999},"last":0.61499999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:12:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70299999999999996,"5min":0.77400000000000002,"15min":0.88100000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.36699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.80000000000000004,"5min":0.81399999999999995,"15min":1.26},"last":0.626}]}]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302272,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069568,"statfs":{"total":96636764160,"available":96327237632,"internally_reserved":0,"allocated":1105920,"data_stored":597935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.435,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.85199999999999998,"15min":1.101},"last":0.253},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.48899999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76000000000000001,"5min":0.82699999999999996,"15min":1.1439999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.40799999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72799999999999998,"5min":0.84099999999999997,"15min":1.093},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.50900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79000000000000004,"5min":0.81299999999999994,"15min":1.1639999999999999},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.47499999999999998,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.77500000000000002,"5min":0.84299999999999997,"15min":1.1100000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.48599999999999999,"5min":0.435,"15min":0.35499999999999998},"max":{"1min":0.77700000000000002,"5min":0.83599999999999997,"15min":1.1220000000000001},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.51000000000000001,"5min":0.39900000000000002,"15min":0.38500000000000001},"max":{"1min":0.74199999999999999,"5min":0.80500000000000005,"15min":1.139},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.61899999999999999},"min":{"1min":0.47599999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.70399999999999996,"5min":0.83099999999999996,"15min":1.0609999999999999},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47899999999999998,"15min":0.48999999999999999},"min":{"1min":0.33900000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73099999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.501,"15min":0.503},"min":{"1min":0.36699999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71199999999999997,"5min":0.84999999999999998,"15min":0.90000000000000002},"last":0.56100000000000005}]},{"osd":6,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.36199999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.68700000000000006,"5min":0.86399999999999999,"15min":1.1839999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53000000000000003,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.53200000000000003}]},{"osd":7,"last update":"Wed Jan 24 14:12:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.5},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.435,"5min":0.34100000000000003,"15min":0.33200000000000002},"max":{"1min":0.83999999999999997,"5min":0.95999999999999996,"15min":1.2030000000000001},"last":0.52200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:13:31.782 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-24T14:13:31.783 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-24T14:13:31.827 INFO:tasks.ceph.mds.a.smithi029.stderr:2024-01-24T14:13:31.827+0000 7fef47954700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T14:13:31.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.955+0000 7f0a9ebd3700 1 Processor -- start 2024-01-24T14:13:31.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.955+0000 7f0a9ebd3700 1 -- start start 2024-01-24T14:13:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.956+0000 7f0a9ebd3700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0a98150c00 0x7f0a980c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.956+0000 7f0a9ebd3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a98155030 0x7f0a980c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.956+0000 7f0a9ebd3700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a981558a0 con 0x7f0a98150fe0 2024-01-24T14:13:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.956+0000 7f0a9ebd3700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a98066960 con 0x7f0a98150c00 2024-01-24T14:13:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.956+0000 7f0a9ebd3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980c2fc0 con 0x7f0a98155030 2024-01-24T14:13:31.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.956+0000 7f0a9c96f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0a98150c00 0x7f0a980c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.956+0000 7f0a97fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0a98150fe0 0x7f0a98154920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35944/0 (socket says 172.21.15.29:35944) 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.956+0000 7f0a9c96f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0a98150c00 0x7f0a980c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41618/0 (socket says 172.21.15.29:41618) 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.956+0000 7f0a97fff700 1 -- 172.21.15.29:0/3775952272 learned_addr learned my addr 172.21.15.29:0/3775952272 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.956+0000 7f0a9d170700 1 --2- 172.21.15.29:0/3775952272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a98155030 0x7f0a980c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.957+0000 7f0a9c96f700 1 -- 172.21.15.29:0/3775952272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a98155030 msgr2=0x7f0a980c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.957+0000 7f0a9c96f700 1 --2- 172.21.15.29:0/3775952272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a98155030 0x7f0a980c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.957+0000 7f0a9c96f700 1 -- 172.21.15.29:0/3775952272 >> v1:172.21.15.29:6789/0 conn(0x7f0a98150fe0 legacy=0x7f0a98154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.957+0000 7f0a977fe700 1 -- 172.21.15.29:0/3775952272 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447996713 0 0) 0x7f0a981558a0 con 0x7f0a98150fe0 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.957+0000 7f0a9c96f700 1 -- 172.21.15.29:0/3775952272 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a981555c0 con 0x7f0a98150c00 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.957+0000 7f0a9c96f700 1 --2- 172.21.15.29:0/3775952272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a98150c00 0x7f0a980c3450 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f0a88009760 tx=0x7f0a88009060).ready entity=mon.0 client_cookie=ace74eaed8230a22 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.957+0000 7f0a977fe700 1 -- 172.21.15.29:0/3775952272 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8800b580 con 0x7f0a98150c00 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.957+0000 7f0a977fe700 1 -- 172.21.15.29:0/3775952272 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8800b6e0 con 0x7f0a98150c00 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.957+0000 7f0a977fe700 1 -- 172.21.15.29:0/3775952272 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8800ba60 con 0x7f0a98150c00 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.957+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3775952272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a98150c00 msgr2=0x7f0a980c3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.957+0000 7f0a9ebd3700 1 --2- 172.21.15.29:0/3775952272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a98150c00 0x7f0a980c3450 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f0a88009760 tx=0x7f0a88009060).stop 2024-01-24T14:13:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.958+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3775952272 shutdown_connections 2024-01-24T14:13:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.958+0000 7f0a9ebd3700 1 --2- 172.21.15.29:0/3775952272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a98155030 0x7f0a980c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.958+0000 7f0a9ebd3700 1 --2- 172.21.15.29:0/3775952272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a98150c00 0x7f0a980c3450 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.958+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3775952272 >> 172.21.15.29:0/3775952272 conn(0x7f0a980638c0 msgr2=0x7f0a98063cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.958+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3775952272 shutdown_connections 2024-01-24T14:13:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.958+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3775952272 wait complete. 2024-01-24T14:13:31.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.959+0000 7f0a9ebd3700 1 Processor -- start 2024-01-24T14:13:31.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.959+0000 7f0a9ebd3700 1 -- start start 2024-01-24T14:13:31.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9ebd3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a98150c00 0x7f0a981687b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9ebd3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a98150fe0 0x7f0a980c1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9ebd3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a98155030 0x7f0a9814e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9ebd3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980c73f0 con 0x7f0a98155030 2024-01-24T14:13:31.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9ebd3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980c76d0 con 0x7f0a98150c00 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9ebd3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a98167cd0 con 0x7f0a98150fe0 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9d170700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a98155030 0x7f0a9814e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9d170700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a98155030 0x7f0a9814e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41628/0 (socket says 172.21.15.29:41628) 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9d170700 1 -- 172.21.15.29:0/3744240624 learned_addr learned my addr 172.21.15.29:0/3744240624 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a97fff700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a98150fe0 0x7f0a980c1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9c96f700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a98150c00 0x7f0a981687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9d170700 1 -- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a98150fe0 msgr2=0x7f0a980c1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9d170700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a98150fe0 0x7f0a980c1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9d170700 1 -- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a98150c00 msgr2=0x7f0a981687b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9d170700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a98150c00 0x7f0a981687b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.960+0000 7f0a9d170700 1 -- 172.21.15.29:0/3744240624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a880092a0 con 0x7f0a98155030 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.961+0000 7f0a9d170700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a98155030 0x7f0a9814e9b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f0a90007f20 tx=0x7f0a90009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.961+0000 7f0a94ff9700 1 -- 172.21.15.29:0/3744240624 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9000eca0 con 0x7f0a98155030 2024-01-24T14:13:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.961+0000 7f0a94ff9700 1 -- 172.21.15.29:0/3744240624 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9000ee00 con 0x7f0a98155030 2024-01-24T14:13:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.961+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3744240624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a98167fb0 con 0x7f0a98155030 2024-01-24T14:13:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.961+0000 7f0a94ff9700 1 -- 172.21.15.29:0/3744240624 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90014610 con 0x7f0a98155030 2024-01-24T14:13:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.961+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3744240624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a980c1f90 con 0x7f0a98155030 2024-01-24T14:13:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.962+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3744240624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a981684b0 con 0x7f0a98155030 2024-01-24T14:13:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.962+0000 7f0a94ff9700 1 -- 172.21.15.29:0/3744240624 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0a90014980 con 0x7f0a98155030 2024-01-24T14:13:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.963+0000 7f0a94ff9700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0a7805f270 0x7f0a78061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.963+0000 7f0a94ff9700 1 -- 172.21.15.29:0/3744240624 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f0a9007ace0 con 0x7f0a98155030 2024-01-24T14:13:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.963+0000 7f0a9c96f700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0a7805f270 0x7f0a78061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.963+0000 7f0a9c96f700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0a7805f270 0x7f0a78061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f0a88009760 tx=0x7f0a88017000).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:31.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:31.967+0000 7f0a94ff9700 1 -- 172.21.15.29:0/3744240624 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0a9007f070 con 0x7f0a98155030 2024-01-24T14:13:32.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.223+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3744240624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0a980475e0 con 0x7f0a98155030 2024-01-24T14:13:32.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.227+0000 7f0a94ff9700 1 -- 172.21.15.29:0/3744240624 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0a9004f460 con 0x7f0a98155030 2024-01-24T14:13:32.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.229+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0a7805f270 msgr2=0x7f0a78061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:32.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.229+0000 7f0a9ebd3700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0a7805f270 0x7f0a78061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f0a88009760 tx=0x7f0a88017000).stop 2024-01-24T14:13:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.229+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3744240624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a98155030 msgr2=0x7f0a9814e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.229+0000 7f0a9ebd3700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a98155030 0x7f0a9814e9b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f0a90007f20 tx=0x7f0a90009860).stop 2024-01-24T14:13:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.229+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3744240624 shutdown_connections 2024-01-24T14:13:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.229+0000 7f0a9ebd3700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0a7805f270 0x7f0a78061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.229+0000 7f0a9ebd3700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a98155030 0x7f0a9814e9b0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.229+0000 7f0a9ebd3700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a98150c00 0x7f0a981687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.229+0000 7f0a9ebd3700 1 --2- 172.21.15.29:0/3744240624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a98150fe0 0x7f0a980c1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.229+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3744240624 >> 172.21.15.29:0/3744240624 conn(0x7f0a980638c0 msgr2=0x7f0a980a86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.229+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3744240624 shutdown_connections 2024-01-24T14:13:32.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.229+0000 7f0a9ebd3700 1 -- 172.21.15.29:0/3744240624 wait complete. 2024-01-24T14:13:32.231 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-01-24T14:13:32.243 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:13:32.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.416+0000 7f800c898700 1 Processor -- start 2024-01-24T14:13:32.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.416+0000 7f800c898700 1 -- start start 2024-01-24T14:13:32.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.417+0000 7f800c898700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8008155040 0x7f8008069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:32.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.417+0000 7f800c898700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8008065ec0 0x7f80080c3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:32.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.417+0000 7f800c898700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80080c2ce0 con 0x7f8008150d40 2024-01-24T14:13:32.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f800c898700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8008066370 con 0x7f8008065ec0 2024-01-24T14:13:32.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f800c898700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8008069890 con 0x7f8008155040 2024-01-24T14:13:32.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f8006d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8008065ec0 0x7f80080c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:32.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f8006d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8008065ec0 0x7f80080c3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41636/0 (socket says 172.21.15.29:41636) 2024-01-24T14:13:32.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f8006d9d700 1 -- 172.21.15.29:0/4071620814 learned_addr learned my addr 172.21.15.29:0/4071620814 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:32.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f8005d9b700 1 --2- 172.21.15.29:0/4071620814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8008155040 0x7f8008069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f8006d9d700 1 -- 172.21.15.29:0/4071620814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8008155040 msgr2=0x7f8008069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f8006d9d700 1 --2- 172.21.15.29:0/4071620814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8008155040 0x7f8008069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f8006d9d700 1 -- 172.21.15.29:0/4071620814 >> v1:172.21.15.163:6789/0 conn(0x7f8008150d40 legacy=0x7f8008154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f8006d9d700 1 -- 172.21.15.29:0/4071620814 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8008069b70 con 0x7f8008065ec0 2024-01-24T14:13:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f800559a700 1 -- 172.21.15.29:0/4071620814 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232305581 0 0) 0x7f80080c2ce0 con 0x7f8008150d40 2024-01-24T14:13:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f8006d9d700 1 --2- 172.21.15.29:0/4071620814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8008065ec0 0x7f80080c3150 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f8000009f20 tx=0x7f8000009840).ready entity=mon.0 client_cookie=69af0f1ba1572519 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.418+0000 7f800559a700 1 -- 172.21.15.29:0/4071620814 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800000e070 con 0x7f8008065ec0 2024-01-24T14:13:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.419+0000 7f800559a700 1 -- 172.21.15.29:0/4071620814 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800000b9d0 con 0x7f8008065ec0 2024-01-24T14:13:32.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.419+0000 7f800559a700 1 -- 172.21.15.29:0/4071620814 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800000bd50 con 0x7f8008065ec0 2024-01-24T14:13:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.419+0000 7f800c898700 1 -- 172.21.15.29:0/4071620814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8008065ec0 msgr2=0x7f80080c3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.419+0000 7f800c898700 1 --2- 172.21.15.29:0/4071620814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8008065ec0 0x7f80080c3150 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f8000009f20 tx=0x7f8000009840).stop 2024-01-24T14:13:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.419+0000 7f800c898700 1 -- 172.21.15.29:0/4071620814 shutdown_connections 2024-01-24T14:13:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.419+0000 7f800c898700 1 --2- 172.21.15.29:0/4071620814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8008065ec0 0x7f80080c3150 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.419+0000 7f800c898700 1 --2- 172.21.15.29:0/4071620814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8008155040 0x7f8008069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.419+0000 7f800c898700 1 -- 172.21.15.29:0/4071620814 >> 172.21.15.29:0/4071620814 conn(0x7f80080c8580 msgr2=0x7f80080c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.419+0000 7f800c898700 1 -- 172.21.15.29:0/4071620814 shutdown_connections 2024-01-24T14:13:32.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.419+0000 7f800c898700 1 -- 172.21.15.29:0/4071620814 wait complete. 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.420+0000 7f800c898700 1 Processor -- start 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.420+0000 7f800c898700 1 -- start start 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.421+0000 7f800c898700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8008065ec0 0x7f8008139e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.421+0000 7f800c898700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8008150d40 0x7f80080ab470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.421+0000 7f800c898700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8008155040 0x7f800814f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.421+0000 7f800c898700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80080c1ef0 con 0x7f8008065ec0 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.421+0000 7f800c898700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80080bf2b0 con 0x7f8008150d40 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.421+0000 7f800c898700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800814ed50 con 0x7f8008155040 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.421+0000 7f800659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8008065ec0 0x7f8008139e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.421+0000 7f800659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8008065ec0 0x7f8008139e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41640/0 (socket says 172.21.15.29:41640) 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.421+0000 7f800659c700 1 -- 172.21.15.29:0/2042707052 learned_addr learned my addr 172.21.15.29:0/2042707052 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f8006d9d700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8008155040 0x7f800814f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:32.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f8005d9b700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8008150d40 0x7f80080ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f800659c700 1 -- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8008155040 msgr2=0x7f800814f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f800659c700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8008155040 0x7f800814f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f800659c700 1 -- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8008150d40 msgr2=0x7f80080ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f800659c700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8008150d40 0x7f80080ab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f800659c700 1 -- 172.21.15.29:0/2042707052 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000009ae0 con 0x7f8008065ec0 2024-01-24T14:13:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f800659c700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8008065ec0 0x7f8008139e20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f7ff800c4e0 tx=0x7f7ff8007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f7ff67fc700 1 -- 172.21.15.29:0/2042707052 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800ce50 con 0x7f8008065ec0 2024-01-24T14:13:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f7ff67fc700 1 -- 172.21.15.29:0/2042707052 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff800f470 con 0x7f8008065ec0 2024-01-24T14:13:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f800c898700 1 -- 172.21.15.29:0/2042707052 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80080bed30 con 0x7f8008065ec0 2024-01-24T14:13:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f7ff67fc700 1 -- 172.21.15.29:0/2042707052 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800f830 con 0x7f8008065ec0 2024-01-24T14:13:32.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.422+0000 7f800c898700 1 -- 172.21.15.29:0/2042707052 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80080befd0 con 0x7f8008065ec0 2024-01-24T14:13:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.423+0000 7f800c898700 1 -- 172.21.15.29:0/2042707052 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800814f030 con 0x7f8008065ec0 2024-01-24T14:13:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.423+0000 7f7ff67fc700 1 -- 172.21.15.29:0/2042707052 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7ff800fba0 con 0x7f8008065ec0 2024-01-24T14:13:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.424+0000 7f7ff67fc700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7fe805f270 0x7f7fe8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.424+0000 7f7ff67fc700 1 -- 172.21.15.29:0/2042707052 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f7ff8079b00 con 0x7f8008065ec0 2024-01-24T14:13:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.424+0000 7f8005d9b700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7fe805f270 0x7f7fe8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:32.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.425+0000 7f8005d9b700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7fe805f270 0x7f7fe8061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f80081502c0 tx=0x7f7ffc009060).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:32.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.428+0000 7f7ff67fc700 1 -- 172.21.15.29:0/2042707052 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7ff8048b00 con 0x7f8008065ec0 2024-01-24T14:13:32.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.656+0000 7f800c898700 1 -- 172.21.15.29:0/2042707052 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80080475e0 con 0x7f8008065ec0 2024-01-24T14:13:32.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.657+0000 7f7ff67fc700 1 -- 172.21.15.29:0/2042707052 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f7ff804a900 con 0x7f8008065ec0 2024-01-24T14:13:32.657 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:32.657 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1149323432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1149323432},{"type":"v1","addr":"172.21.15.163:6835","nonce":1149323432}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6157,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2941255566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":2941255566},{"type":"v1","addr":"172.21.15.29:6835","nonce":2941255566}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6163,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2235282008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2235282008},{"type":"v1","addr":"172.21.15.163:6837","nonce":2235282008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:13:27.270903+0000","modified":"2024-01-24T14:13:30.278839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6366":{"gid":6366,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":4,"addr":"172.21.15.29:6833/3957899438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3957899438},{"type":"v1","addr":"172.21.15.29:6833","nonce":3957899438}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-24T14:13:32.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.658+0000 7f800c898700 1 -- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7fe805f270 msgr2=0x7f7fe8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:32.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.658+0000 7f800c898700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7fe805f270 0x7f7fe8061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f80081502c0 tx=0x7f7ffc009060).stop 2024-01-24T14:13:32.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.658+0000 7f800c898700 1 -- 172.21.15.29:0/2042707052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8008065ec0 msgr2=0x7f8008139e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.658+0000 7f800c898700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8008065ec0 0x7f8008139e20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f7ff800c4e0 tx=0x7f7ff8007c70).stop 2024-01-24T14:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.658+0000 7f800c898700 1 -- 172.21.15.29:0/2042707052 shutdown_connections 2024-01-24T14:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.658+0000 7f800c898700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8008155040 0x7f800814f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.658+0000 7f800c898700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7fe805f270 0x7f7fe8061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.659+0000 7f800c898700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8008065ec0 0x7f8008139e20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.659+0000 7f800c898700 1 --2- 172.21.15.29:0/2042707052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8008150d40 0x7f80080ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.659+0000 7f800c898700 1 -- 172.21.15.29:0/2042707052 >> 172.21.15.29:0/2042707052 conn(0x7f80080c8580 msgr2=0x7f800814b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.659+0000 7f800c898700 1 -- 172.21.15.29:0/2042707052 shutdown_connections 2024-01-24T14:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:32.659+0000 7f800c898700 1 -- 172.21.15.29:0/2042707052 wait complete. 2024-01-24T14:13:32.661 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 35 2024-01-24T14:13:33.076 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T14:13:27.270903+0000', 'modified': '2024-01-24T14:13:30.278839+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6366}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6366': {'gid': 6366, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.29:6833/3957899438', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6832', 'nonce': 3957899438}, {'type': 'v1', 'addr': '172.21.15.29:6833', 'nonce': 3957899438}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-24T14:13:33.076 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T14:13:33.076 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-24T14:13:33.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.215+0000 7f6733b2b700 1 Processor -- start 2024-01-24T14:13:33.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.215+0000 7f6733b2b700 1 -- start start 2024-01-24T14:13:33.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.216+0000 7f6733b2b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f672c150bd0 0x7f672c065e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.216+0000 7f6733b2b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f672c069240 0x7f672c150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.216+0000 7f6733b2b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f672c1514a0 0x7f672c151950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.216+0000 7f6733b2b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672c06a350 con 0x7f672c069240 2024-01-24T14:13:33.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.216+0000 7f6733b2b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672c069740 con 0x7f672c150bd0 2024-01-24T14:13:33.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.216+0000 7f6733b2b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c069a20 con 0x7f672c1514a0 2024-01-24T14:13:33.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67310c6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f672c069240 0x7f672c150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67310c6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f672c069240 0x7f672c150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41654/0 (socket says 172.21.15.29:41654) 2024-01-24T14:13:33.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67320c8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f672c1514a0 0x7f672c151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67318c7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f672c150bd0 0x7f672c065e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67310c6700 1 -- 172.21.15.29:0/3989880313 learned_addr learned my addr 172.21.15.29:0/3989880313 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:33.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67310c6700 1 -- 172.21.15.29:0/3989880313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f672c1514a0 msgr2=0x7f672c151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:33.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67310c6700 1 --2- 172.21.15.29:0/3989880313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f672c1514a0 0x7f672c151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67310c6700 1 -- 172.21.15.29:0/3989880313 >> v2:172.21.15.163:3300/0 conn(0x7f672c150bd0 msgr2=0x7f672c065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:33.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67310c6700 1 --2- 172.21.15.29:0/3989880313 >> v2:172.21.15.163:3300/0 conn(0x7f672c150bd0 0x7f672c065e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67310c6700 1 -- 172.21.15.29:0/3989880313 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c069d00 con 0x7f672c069240 2024-01-24T14:13:33.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67310c6700 1 --2- 172.21.15.29:0/3989880313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f672c069240 0x7f672c150fb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f672c06c700 tx=0x7f67280090f0).ready entity=mon.0 client_cookie=aae6d0cf03950554 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:33.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67308c5700 1 -- 172.21.15.29:0/3989880313 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672800f510 con 0x7f672c069240 2024-01-24T14:13:33.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67308c5700 1 -- 172.21.15.29:0/3989880313 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672800f670 con 0x7f672c069240 2024-01-24T14:13:33.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.217+0000 7f67308c5700 1 -- 172.21.15.29:0/3989880313 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672800f9f0 con 0x7f672c069240 2024-01-24T14:13:33.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.218+0000 7f6733b2b700 1 -- 172.21.15.29:0/3989880313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f672c069240 msgr2=0x7f672c150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.218+0000 7f6733b2b700 1 --2- 172.21.15.29:0/3989880313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f672c069240 0x7f672c150fb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f672c06c700 tx=0x7f67280090f0).stop 2024-01-24T14:13:33.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.218+0000 7f6733b2b700 1 -- 172.21.15.29:0/3989880313 shutdown_connections 2024-01-24T14:13:33.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.218+0000 7f6733b2b700 1 --2- 172.21.15.29:0/3989880313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f672c1514a0 0x7f672c151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.218+0000 7f6733b2b700 1 --2- 172.21.15.29:0/3989880313 >> v2:172.21.15.163:3300/0 conn(0x7f672c150bd0 0x7f672c065e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.218+0000 7f6733b2b700 1 --2- 172.21.15.29:0/3989880313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f672c069240 0x7f672c150fb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.218+0000 7f6733b2b700 1 -- 172.21.15.29:0/3989880313 >> 172.21.15.29:0/3989880313 conn(0x7f672c0c8580 msgr2=0x7f672c0c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:33.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.218+0000 7f6733b2b700 1 -- 172.21.15.29:0/3989880313 shutdown_connections 2024-01-24T14:13:33.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.218+0000 7f6733b2b700 1 -- 172.21.15.29:0/3989880313 wait complete. 2024-01-24T14:13:33.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.219+0000 7f6733b2b700 1 Processor -- start 2024-01-24T14:13:33.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.219+0000 7f6733b2b700 1 -- start start 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.220+0000 7f6733b2b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f672c069240 0x7f672c148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.220+0000 7f6733b2b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f672c150bd0 0x7f672c0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.220+0000 7f6733b2b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f672c1514a0 0x7f672c0c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.220+0000 7f6733b2b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c0acff0 con 0x7f672c150bd0 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.220+0000 7f6733b2b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c0ac420 con 0x7f672c1514a0 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.220+0000 7f6733b2b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c0ac6c0 con 0x7f672c069240 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.220+0000 7f67310c6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f672c150bd0 0x7f672c0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.220+0000 7f67310c6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f672c150bd0 0x7f672c0ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41660/0 (socket says 172.21.15.29:41660) 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.220+0000 7f67318c7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f672c069240 0x7f672c148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.220+0000 7f67310c6700 1 -- 172.21.15.29:0/1960168046 learned_addr learned my addr 172.21.15.29:0/1960168046 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.220+0000 7f67320c8700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f672c1514a0 0x7f672c0c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.221+0000 7f67310c6700 1 -- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f672c069240 msgr2=0x7f672c148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.221+0000 7f67310c6700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f672c069240 0x7f672c148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.221+0000 7f67310c6700 1 -- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f672c1514a0 msgr2=0x7f672c0c04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.221+0000 7f67310c6700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f672c1514a0 0x7f672c0c04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.221+0000 7f67310c6700 1 -- 172.21.15.29:0/1960168046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6728009370 con 0x7f672c150bd0 2024-01-24T14:13:33.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.221+0000 7f67310c6700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f672c150bd0 0x7f672c0ad480 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f6728011000 tx=0x7f6728017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:33.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.221+0000 7f671dffb700 1 -- 172.21.15.29:0/1960168046 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672800f7e0 con 0x7f672c150bd0 2024-01-24T14:13:33.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.221+0000 7f671dffb700 1 -- 172.21.15.29:0/1960168046 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672800f940 con 0x7f672c150bd0 2024-01-24T14:13:33.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.221+0000 7f6733b2b700 1 -- 172.21.15.29:0/1960168046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672c0ac9a0 con 0x7f672c150bd0 2024-01-24T14:13:33.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.221+0000 7f671dffb700 1 -- 172.21.15.29:0/1960168046 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672801c670 con 0x7f672c150bd0 2024-01-24T14:13:33.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.221+0000 7f6733b2b700 1 -- 172.21.15.29:0/1960168046 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c148e10 con 0x7f672c150bd0 2024-01-24T14:13:33.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.223+0000 7f671dffb700 1 -- 172.21.15.29:0/1960168046 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f672801ca60 con 0x7f672c150bd0 2024-01-24T14:13:33.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.223+0000 7f671dffb700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f671005f240 0x7f6710061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.223+0000 7f671dffb700 1 -- 172.21.15.29:0/1960168046 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f672807c2b0 con 0x7f672c150bd0 2024-01-24T14:13:33.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.224+0000 7f67318c7700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f671005f240 0x7f6710061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.224+0000 7f67318c7700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f671005f240 0x7f6710061700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f6720009c30 tx=0x7f6720009060).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:33.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.225+0000 7f67454f6b80 1 Processor -- start 2024-01-24T14:13:33.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.226+0000 7f67454f6b80 1 -- start start 2024-01-24T14:13:33.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.226+0000 7f67454f6b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 0x5609cdebfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.226+0000 7f67454f6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdeba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.226+0000 7f67454f6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdebb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.226+0000 7f67454f6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cde363d0 con 0x5609cdf082c0 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.226+0000 7f67454f6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cddfbbe0 con 0x5609cdebadf0 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.226+0000 7f67454f6b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609cde11260 con 0x5609cdebf620 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.226+0000 7f67310c6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdeba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.226+0000 7f67310c6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdeba900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41670/0 (socket says 172.21.15.29:41670) 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.227+0000 7f67310c6700 1 -- 172.21.15.29:0/596528848 learned_addr learned my addr 172.21.15.29:0/596528848 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.227+0000 7f67318c7700 1 --2- 172.21.15.29:0/596528848 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 0x5609cdebfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.227+0000 7f67320c8700 1 --2- 172.21.15.29:0/596528848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdebb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.227+0000 7f67310c6700 1 -- 172.21.15.29:0/596528848 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 msgr2=0x5609cdebfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.227+0000 7f67310c6700 1 --2- 172.21.15.29:0/596528848 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 0x5609cdebfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.227+0000 7f67310c6700 1 -- 172.21.15.29:0/596528848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 msgr2=0x5609cdebb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:33.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.227+0000 7f67310c6700 1 --2- 172.21.15.29:0/596528848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdebb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.227+0000 7f67310c6700 1 -- 172.21.15.29:0/596528848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609cde0a2f0 con 0x5609cdf082c0 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.227+0000 7f67310c6700 1 --2- 172.21.15.29:0/596528848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdeba900 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f672807a230 tx=0x7f672800e000).ready entity=mon.0 client_cookie=692029cf58b2b6e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.227+0000 7f6716ffd700 1 -- 172.21.15.29:0/596528848 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728056070 con 0x5609cdf082c0 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.227+0000 7f6716ffd700 1 -- 172.21.15.29:0/596528848 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6728050c60 con 0x5609cdf082c0 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.227+0000 7f6716ffd700 1 -- 172.21.15.29:0/596528848 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672804ee00 con 0x5609cdf082c0 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.228+0000 7f67454f6b80 1 -- 172.21.15.29:0/596528848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 msgr2=0x5609cdeba900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.228+0000 7f67454f6b80 1 --2- 172.21.15.29:0/596528848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdeba900 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f672807a230 tx=0x7f672800e000).stop 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.228+0000 7f67454f6b80 1 -- 172.21.15.29:0/596528848 shutdown_connections 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.228+0000 7f67454f6b80 1 --2- 172.21.15.29:0/596528848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdebb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.228+0000 7f67454f6b80 1 --2- 172.21.15.29:0/596528848 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 0x5609cdebfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.228+0000 7f67454f6b80 1 --2- 172.21.15.29:0/596528848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdeba900 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.228+0000 7f67454f6b80 1 -- 172.21.15.29:0/596528848 >> 172.21.15.29:0/596528848 conn(0x5609cdd282e0 msgr2=0x5609cdf08750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.228+0000 7f67454f6b80 1 -- 172.21.15.29:0/596528848 shutdown_connections 2024-01-24T14:13:33.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.228+0000 7f67454f6b80 1 -- 172.21.15.29:0/596528848 wait complete. 2024-01-24T14:13:33.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.228+0000 7f67454f6b80 1 -- start start 2024-01-24T14:13:33.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.229+0000 7f67454f6b80 1 Processor -- start 2024-01-24T14:13:33.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67454f6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdee1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67454f6b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 0x5609cdee1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67454f6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdeec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67454f6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cde113a0 con 0x5609cdf082c0 2024-01-24T14:13:33.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67454f6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cde11620 con 0x5609cdebadf0 2024-01-24T14:13:33.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67454f6b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609cdddc990 con 0x5609cdebf620 2024-01-24T14:13:33.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67454f6b80 10 client.0 ll_register_callbacks cb 0x5609cde7f610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T14:13:33.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67320c8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdeec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67320c8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdeec0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41672/0 (socket says 172.21.15.29:41672) 2024-01-24T14:13:33.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67320c8700 1 -- 172.21.15.29:0/3941961328 learned_addr learned my addr 172.21.15.29:0/3941961328 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:33.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67318c7700 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdee1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67310c6700 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 0x5609cdee1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67320c8700 1 -- 172.21.15.29:0/3941961328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 msgr2=0x5609cdee1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67320c8700 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 0x5609cdee1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67320c8700 1 -- 172.21.15.29:0/3941961328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 msgr2=0x5609cdee1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67320c8700 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdee1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67320c8700 1 -- 172.21.15.29:0/3941961328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c148e10 con 0x5609cdf082c0 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67318c7700 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdee1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f67320c8700 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdeec0a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f6724012ee0 tx=0x7f672400b040).ready entity=mon.0 client_cookie=6c9f374edf552180 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f670bfff700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.230+0000 7f670bfff700 1 -- 172.21.15.29:0/3941961328 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400e040 con 0x5609cdf082c0 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f670bfff700 1 -- 172.21.15.29:0/3941961328 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f6724014dd0 con 0x5609cdf082c0 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f67454f6b80 1 -- 172.21.15.29:0/3941961328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609cdeb9df0 con 0x5609cdf082c0 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f670bfff700 1 -- 172.21.15.29:0/3941961328 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67240122a0 con 0x5609cdf082c0 2024-01-24T14:13:33.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f67454f6b80 10 client.6522 fetch_fsmap learned FSMap version 35 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f67454f6b80 1 -- 172.21.15.29:0/3941961328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x5609cde05520 con 0x5609cdf082c0 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f670bfff700 1 -- 172.21.15.29:0/3941961328 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5609cdeb9df0 con 0x5609cdf082c0 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f670bfff700 1 -- 172.21.15.29:0/3941961328 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 35) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f672401c020 con 0x5609cdf082c0 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f67454f6b80 10 client.6522 fetch_fsmap finished waiting for FSMap version 35 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f67454f6b80 10 client.6522 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f67454f6b80 20 client.6522 populate_metadata read hostname 'smithi029' 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f67454f6b80 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.29:6832/3957899438,v1:172.21.15.29:6833/3957899438] conn(0x5609cdf04460 0x5609cdf219e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f67454f6b80 4 client.6522 mds_command: new command op to 6366 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f67454f6b80 1 -- 172.21.15.29:0/3941961328 --> [v2:172.21.15.29:6832/3957899438,v1:172.21.15.29:6833/3957899438] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609cde06d60 con 0x5609cdf04460 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.231+0000 7f67318c7700 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.29:6832/3957899438,v1:172.21.15.29:6833/3957899438] conn(0x5609cdf04460 0x5609cdf219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f670bfff700 10 client.6522 ms_handle_connect on v2:172.21.15.29:6832/3957899438 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f67318c7700 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.29:6832/3957899438,v1:172.21.15.29:6833/3957899438] conn(0x5609cdf04460 0x5609cdf219e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ab683fb2e4bdbfc1 server_cookie=7af5b2b04577bee4 in_seq=0 out_seq=0 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f670bfff700 1 -- 172.21.15.29:0/3941961328 <== mds.0 v2:172.21.15.29:6832/3957899438 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x5609cde06d60 con 0x5609cdf04460 2024-01-24T14:13:33.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f670bfff700 10 client.6522 handle_command_reply: tid=0 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f67454f6b80 1 client.6522 shutdown 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f67454f6b80 1 -- 172.21.15.29:0/3941961328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 msgr2=0x5609cdeec0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f67454f6b80 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdeec0a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f6724012ee0 tx=0x7f672400b040).stop 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f67454f6b80 1 -- 172.21.15.29:0/3941961328 shutdown_connections 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f67454f6b80 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.29:6832/3957899438,v1:172.21.15.29:6833/3957899438] conn(0x5609cdf04460 0x5609cdf219e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f67454f6b80 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdeec0a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f67454f6b80 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdee1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f67454f6b80 1 --2- 172.21.15.29:0/3941961328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 0x5609cdee1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f670bfff700 0 client.6522 ms_handle_reset on v2:172.21.15.29:6832/3957899438 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.232+0000 7f67454f6b80 1 -- 172.21.15.29:0/3941961328 >> 172.21.15.29:0/3941961328 conn(0x5609cdd282e0 msgr2=0x5609cdebfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.233+0000 7f67454f6b80 1 -- 172.21.15.29:0/3941961328 shutdown_connections 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.233+0000 7f67454f6b80 1 -- 172.21.15.29:0/3941961328 wait complete. 2024-01-24T14:13:33.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.233+0000 7f67454f6b80 20 client.6522 trim_cache size 0 max 16384 2024-01-24T14:13:33.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.250+0000 7f67454f6b80 1 Processor -- start 2024-01-24T14:13:33.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.250+0000 7f67454f6b80 1 -- start start 2024-01-24T14:13:33.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67454f6b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebadf0 0x5609cdebe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67454f6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdebf620 0x5609cdebb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67454f6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdf082c0 0x5609cdebb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67454f6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cdec15c0 con 0x5609cdebf620 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67454f6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cdeb7560 con 0x5609cdf082c0 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67454f6b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609cdebc480 con 0x5609cdebadf0 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67310c6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdebf620 0x5609cdebb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67310c6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdebf620 0x5609cdebb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41678/0 (socket says 172.21.15.29:41678) 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67310c6700 1 -- 172.21.15.29:0/4272893790 learned_addr learned my addr 172.21.15.29:0/4272893790 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67318c7700 1 --2- 172.21.15.29:0/4272893790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebadf0 0x5609cdebe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67320c8700 1 --2- 172.21.15.29:0/4272893790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdf082c0 0x5609cdebb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67310c6700 1 -- 172.21.15.29:0/4272893790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebadf0 msgr2=0x5609cdebe4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67310c6700 1 --2- 172.21.15.29:0/4272893790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebadf0 0x5609cdebe4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67310c6700 1 -- 172.21.15.29:0/4272893790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdf082c0 msgr2=0x5609cdebb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67310c6700 1 --2- 172.21.15.29:0/4272893790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdf082c0 0x5609cdebb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.251+0000 7f67310c6700 1 -- 172.21.15.29:0/4272893790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609cde05270 con 0x5609cdebf620 2024-01-24T14:13:33.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67310c6700 1 --2- 172.21.15.29:0/4272893790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdebf620 0x5609cdebb400 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f672804f2b0 tx=0x7f6728056000).ready entity=mon.0 client_cookie=ddbf1739e222d8b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67177fe700 1 -- 172.21.15.29:0/4272893790 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728035070 con 0x5609cdebf620 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67177fe700 1 -- 172.21.15.29:0/4272893790 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6728050620 con 0x5609cdebf620 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67177fe700 1 -- 172.21.15.29:0/4272893790 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672804ee00 con 0x5609cdebf620 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67454f6b80 1 -- 172.21.15.29:0/4272893790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdebf620 msgr2=0x5609cdebb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67454f6b80 1 --2- 172.21.15.29:0/4272893790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdebf620 0x5609cdebb400 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f672804f2b0 tx=0x7f6728056000).stop 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67454f6b80 1 -- 172.21.15.29:0/4272893790 shutdown_connections 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67454f6b80 1 --2- 172.21.15.29:0/4272893790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdf082c0 0x5609cdebb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67454f6b80 1 --2- 172.21.15.29:0/4272893790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebadf0 0x5609cdebe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67454f6b80 1 --2- 172.21.15.29:0/4272893790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdebf620 0x5609cdebb400 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67454f6b80 1 -- 172.21.15.29:0/4272893790 >> 172.21.15.29:0/4272893790 conn(0x5609cdd282e0 msgr2=0x5609cdeb6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67454f6b80 1 -- 172.21.15.29:0/4272893790 shutdown_connections 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.252+0000 7f67454f6b80 1 -- 172.21.15.29:0/4272893790 wait complete. 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.253+0000 7f67454f6b80 1 -- start start 2024-01-24T14:13:33.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.253+0000 7f67454f6b80 1 Processor -- start 2024-01-24T14:13:33.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67454f6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdf6c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67454f6b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 0x5609cdee37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67454f6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdee3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67454f6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cdee48d0 con 0x5609cdf082c0 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67454f6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cdee4190 con 0x5609cdebadf0 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67454f6b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609cdf13a30 con 0x5609cdebf620 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67454f6b80 10 client.0 ll_register_callbacks cb 0x5609cde7f610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67320c8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdee3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67320c8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdee3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41680/0 (socket says 172.21.15.29:41680) 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67320c8700 1 -- 172.21.15.29:0/1355686414 learned_addr learned my addr 172.21.15.29:0/1355686414 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67310c6700 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 0x5609cdee37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67318c7700 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdf6c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67320c8700 1 -- 172.21.15.29:0/1355686414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 msgr2=0x5609cdee37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67320c8700 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 0x5609cdee37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.254+0000 7f67320c8700 1 -- 172.21.15.29:0/1355686414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 msgr2=0x5609cdf6c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f67320c8700 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdf6c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f67320c8700 1 -- 172.21.15.29:0/1355686414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609cde0a2f0 con 0x5609cdf082c0 2024-01-24T14:13:33.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f67320c8700 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdee3ca0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f6724012eb0 tx=0x7f6724023000).ready entity=mon.0 client_cookie=60fe1e94131c3e84 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:33.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f670bfff700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T14:13:33.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f670bfff700 1 -- 172.21.15.29:0/1355686414 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400e040 con 0x5609cdf082c0 2024-01-24T14:13:33.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f670bfff700 1 -- 172.21.15.29:0/1355686414 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f6724014dd0 con 0x5609cdf082c0 2024-01-24T14:13:33.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f67454f6b80 1 -- 172.21.15.29:0/1355686414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6728050c60 con 0x5609cdf082c0 2024-01-24T14:13:33.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f670bfff700 1 -- 172.21.15.29:0/1355686414 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67240122a0 con 0x5609cdf082c0 2024-01-24T14:13:33.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f67454f6b80 10 client.6528 fetch_fsmap learned FSMap version 35 2024-01-24T14:13:33.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f67454f6b80 1 -- 172.21.15.29:0/1355686414 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x5609cdc7ef80 con 0x5609cdf082c0 2024-01-24T14:13:33.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f670bfff700 1 -- 172.21.15.29:0/1355686414 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6728050c60 con 0x5609cdf082c0 2024-01-24T14:13:33.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f670bfff700 1 -- 172.21.15.29:0/1355686414 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 35) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f672401c030 con 0x5609cdf082c0 2024-01-24T14:13:33.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f67454f6b80 10 client.6528 fetch_fsmap finished waiting for FSMap version 35 2024-01-24T14:13:33.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f67454f6b80 10 client.6528 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-24T14:13:33.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f67454f6b80 20 client.6528 populate_metadata read hostname 'smithi029' 2024-01-24T14:13:33.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f67454f6b80 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.29:6832/3957899438,v1:172.21.15.29:6833/3957899438] conn(0x5609cdeecf80 0x5609cdeed360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f67454f6b80 4 client.6528 mds_command: new command op to 6366 tid=0[{"prefix": "status"}] 2024-01-24T14:13:33.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.255+0000 7f67454f6b80 1 -- 172.21.15.29:0/1355686414 --> [v2:172.21.15.29:6832/3957899438,v1:172.21.15.29:6833/3957899438] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5609cddfd7d0 con 0x5609cdeecf80 2024-01-24T14:13:33.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67318c7700 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.29:6832/3957899438,v1:172.21.15.29:6833/3957899438] conn(0x5609cdeecf80 0x5609cdeed360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67318c7700 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.29:6832/3957899438,v1:172.21.15.29:6833/3957899438] conn(0x5609cdeecf80 0x5609cdeed360 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=28bc67442f4e52bd server_cookie=4932750f7a7e16a9 in_seq=0 out_seq=0 2024-01-24T14:13:33.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f670bfff700 10 client.6528 ms_handle_connect on v2:172.21.15.29:6832/3957899438 2024-01-24T14:13:33.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f670bfff700 1 -- 172.21.15.29:0/1355686414 <== mds.0 v2:172.21.15.29:6832/3957899438 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5609cddfd7d0 con 0x5609cdeecf80 2024-01-24T14:13:33.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f670bfff700 10 client.6528 handle_command_reply: tid=0 2024-01-24T14:13:33.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67454f6b80 1 client.6528 shutdown 2024-01-24T14:13:33.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67454f6b80 1 -- 172.21.15.29:0/1355686414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 msgr2=0x5609cdee3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67454f6b80 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdee3ca0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f6724012eb0 tx=0x7f6724023000).stop 2024-01-24T14:13:33.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67454f6b80 1 -- 172.21.15.29:0/1355686414 shutdown_connections 2024-01-24T14:13:33.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67454f6b80 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.29:6832/3957899438,v1:172.21.15.29:6833/3957899438] conn(0x5609cdeecf80 0x5609cdeed360 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67454f6b80 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609cdf082c0 0x5609cdee3ca0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67454f6b80 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5609cdebadf0 0x5609cdf6c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67454f6b80 1 --2- 172.21.15.29:0/1355686414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5609cdebf620 0x5609cdee37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67454f6b80 1 -- 172.21.15.29:0/1355686414 >> 172.21.15.29:0/1355686414 conn(0x5609cdd282e0 msgr2=0x5609cdeee1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:33.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f670bfff700 0 client.6528 ms_handle_reset on v2:172.21.15.29:6832/3957899438 2024-01-24T14:13:33.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67454f6b80 1 -- 172.21.15.29:0/1355686414 shutdown_connections 2024-01-24T14:13:33.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.256+0000 7f67454f6b80 1 -- 172.21.15.29:0/1355686414 wait complete. 2024-01-24T14:13:33.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f67454f6b80 20 client.6528 trim_cache size 0 max 16384 2024-01-24T14:13:33.264 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T14:13:33.264 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T14:13:33.264 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T14:13:33.264 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 6366, 2024-01-24T14:13:33.264 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T14:13:33.264 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T14:13:33.264 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T14:13:33.265 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 5.9848533020000003, 2024-01-24T14:13:33.265 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 35, 2024-01-24T14:13:33.265 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 90, 2024-01-24T14:13:33.265 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 90, 2024-01-24T14:13:33.265 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 17.134798049 2024-01-24T14:13:33.265 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T14:13:33.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f6733b2b700 1 -- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f671005f240 msgr2=0x7f6710061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f6733b2b700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f671005f240 0x7f6710061700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f6720009c30 tx=0x7f6720009060).stop 2024-01-24T14:13:33.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f6733b2b700 1 -- 172.21.15.29:0/1960168046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f672c150bd0 msgr2=0x7f672c0ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f6733b2b700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f672c150bd0 0x7f672c0ad480 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f6728011000 tx=0x7f6728017000).stop 2024-01-24T14:13:33.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f6733b2b700 1 -- 172.21.15.29:0/1960168046 shutdown_connections 2024-01-24T14:13:33.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f6733b2b700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f672c1514a0 0x7f672c0c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f6733b2b700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f672c069240 0x7f672c148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f6733b2b700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f671005f240 0x7f6710061700 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f6733b2b700 1 --2- 172.21.15.29:0/1960168046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f672c150bd0 0x7f672c0ad480 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f6733b2b700 1 -- 172.21.15.29:0/1960168046 >> 172.21.15.29:0/1960168046 conn(0x7f672c0c8580 msgr2=0x7f672c0bcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:33.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f6733b2b700 1 -- 172.21.15.29:0/1960168046 shutdown_connections 2024-01-24T14:13:33.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.257+0000 7f6733b2b700 1 -- 172.21.15.29:0/1960168046 wait complete. 2024-01-24T14:13:33.271 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T14:13:33.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.387+0000 7f1612c97700 1 Processor -- start 2024-01-24T14:13:33.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.387+0000 7f1612c97700 1 -- start start 2024-01-24T14:13:33.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.388+0000 7f1612c97700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f160c0c6b50 0x7f160c0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.388+0000 7f1612c97700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160c065940 con 0x7f160c157480 2024-01-24T14:13:33.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.388+0000 7f1612c97700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160c0c6fe0 con 0x7f160c157860 2024-01-24T14:13:33.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.388+0000 7f1612c97700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f160c0c72c0 con 0x7f160c0c6b50 2024-01-24T14:13:33.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.388+0000 7f160bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f160c0c6b50 0x7f160c0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.388+0000 7f1610a33700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f160c157480 0x7f160c065230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35954/0 (socket says 172.21.15.29:35954) 2024-01-24T14:13:33.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.388+0000 7f1610a33700 1 -- 172.21.15.29:0/1847392164 learned_addr learned my addr 172.21.15.29:0/1847392164 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:33.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.388+0000 7f160bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f160c0c6b50 0x7f160c0a5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41690/0 (socket says 172.21.15.29:41690) 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.388+0000 7f160bfff700 1 -- 172.21.15.29:0/1847392164 >> v1:172.21.15.163:6789/0 conn(0x7f160c157860 legacy=0x7f160c0be490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.388+0000 7f160bfff700 1 -- 172.21.15.29:0/1847392164 >> v1:172.21.15.29:6789/0 conn(0x7f160c157480 legacy=0x7f160c065230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.388+0000 7f160bfff700 1 -- 172.21.15.29:0/1847392164 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c0c75a0 con 0x7f160c0c6b50 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.388+0000 7f160b7fe700 1 -- 172.21.15.29:0/1847392164 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941465247 0 0) 0x7f160c065940 con 0x7f160c157480 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.389+0000 7f160bfff700 1 --2- 172.21.15.29:0/1847392164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f160c0c6b50 0x7f160c0a5a20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f15f8009760 tx=0x7f15f8009060).ready entity=mon.0 client_cookie=8c528a524ca64d2e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.389+0000 7f160b7fe700 1 -- 172.21.15.29:0/1847392164 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f800b580 con 0x7f160c0c6b50 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.389+0000 7f160b7fe700 1 -- 172.21.15.29:0/1847392164 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f800b6e0 con 0x7f160c0c6b50 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.389+0000 7f160b7fe700 1 -- 172.21.15.29:0/1847392164 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f800ba60 con 0x7f160c0c6b50 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.389+0000 7f1612c97700 1 -- 172.21.15.29:0/1847392164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f160c0c6b50 msgr2=0x7f160c0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.389+0000 7f1612c97700 1 --2- 172.21.15.29:0/1847392164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f160c0c6b50 0x7f160c0a5a20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f15f8009760 tx=0x7f15f8009060).stop 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.389+0000 7f1612c97700 1 -- 172.21.15.29:0/1847392164 shutdown_connections 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.389+0000 7f1612c97700 1 --2- 172.21.15.29:0/1847392164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f160c0c6b50 0x7f160c0a5a20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.389+0000 7f1612c97700 1 -- 172.21.15.29:0/1847392164 >> 172.21.15.29:0/1847392164 conn(0x7f160c0922f0 msgr2=0x7f160c062f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:33.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.389+0000 7f1612c97700 1 -- 172.21.15.29:0/1847392164 shutdown_connections 2024-01-24T14:13:33.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.389+0000 7f1612c97700 1 -- 172.21.15.29:0/1847392164 wait complete. 2024-01-24T14:13:33.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.391+0000 7f1612c97700 1 Processor -- start 2024-01-24T14:13:33.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.391+0000 7f1612c97700 1 -- start start 2024-01-24T14:13:33.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1612c97700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f160c157480 0x7f160c0a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1612c97700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f160c157860 0x7f160c0a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1612c97700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f160c066780 0x7f160c066be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1612c97700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c0670d0 con 0x7f160c157860 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1612c97700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c0aa1c0 con 0x7f160c066780 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1612c97700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160c0aa4a0 con 0x7f160c157480 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f160bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f160c157860 0x7f160c0a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f160bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f160c157860 0x7f160c0a9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37612/0 (socket says 172.21.15.29:37612) 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f160bfff700 1 -- 172.21.15.29:0/1645332677 learned_addr learned my addr 172.21.15.29:0/1645332677 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1610a33700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f160c157480 0x7f160c0a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1611234700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f160c066780 0x7f160c066be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f160bfff700 1 -- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f160c157480 msgr2=0x7f160c0a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f160bfff700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f160c157480 0x7f160c0a9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f160bfff700 1 -- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f160c066780 msgr2=0x7f160c066be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f160bfff700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f160c066780 0x7f160c066be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f160bfff700 1 -- 172.21.15.29:0/1645332677 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f80092a0 con 0x7f160c157860 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1611234700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f160c066780 0x7f160c066be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:33.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f160bfff700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f160c157860 0x7f160c0a9bf0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f160c069970 tx=0x7f15f8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:33.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1608ff9700 1 -- 172.21.15.29:0/1645332677 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f800ba60 con 0x7f160c157860 2024-01-24T14:13:33.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1608ff9700 1 -- 172.21.15.29:0/1645332677 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f8000c90 con 0x7f160c157860 2024-01-24T14:13:33.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1612c97700 1 -- 172.21.15.29:0/1645332677 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160c0aa780 con 0x7f160c157860 2024-01-24T14:13:33.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1608ff9700 1 -- 172.21.15.29:0/1645332677 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f801b670 con 0x7f160c157860 2024-01-24T14:13:33.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.392+0000 7f1612c97700 1 -- 172.21.15.29:0/1645332677 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160c1502d0 con 0x7f160c157860 2024-01-24T14:13:33.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.393+0000 7f1612c97700 1 -- 172.21.15.29:0/1645332677 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f160c162340 con 0x7f160c157860 2024-01-24T14:13:33.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.394+0000 7f1608ff9700 1 -- 172.21.15.29:0/1645332677 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f15f801ba60 con 0x7f160c157860 2024-01-24T14:13:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.394+0000 7f1608ff9700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f15f005f270 0x7f15f0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.395+0000 7f1608ff9700 1 -- 172.21.15.29:0/1645332677 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f15f807bfa0 con 0x7f160c157860 2024-01-24T14:13:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.395+0000 7f1610a33700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f15f005f270 0x7f15f0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:33.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.395+0000 7f1610a33700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f15f005f270 0x7f15f0061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f160c1532c0 tx=0x7f1600009040).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.399+0000 7f1608ff9700 1 -- 172.21.15.29:0/1645332677 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f15f80aab00 con 0x7f160c157860 2024-01-24T14:13:33.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.627+0000 7f1612c97700 1 -- 172.21.15.29:0/1645332677 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f160c05f400 con 0x7f160c157860 2024-01-24T14:13:33.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.628+0000 7f1608ff9700 1 -- 172.21.15.29:0/1645332677 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7f15f8028020 con 0x7f160c157860 2024-01-24T14:13:33.628 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-24T14:13:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.630+0000 7f1612c97700 1 -- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f15f005f270 msgr2=0x7f15f0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.630+0000 7f1612c97700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f15f005f270 0x7f15f0061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f160c1532c0 tx=0x7f1600009040).stop 2024-01-24T14:13:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.630+0000 7f1612c97700 1 -- 172.21.15.29:0/1645332677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f160c157860 msgr2=0x7f160c0a9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.630+0000 7f1612c97700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f160c157860 0x7f160c0a9bf0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f160c069970 tx=0x7f15f8017000).stop 2024-01-24T14:13:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.630+0000 7f1612c97700 1 -- 172.21.15.29:0/1645332677 shutdown_connections 2024-01-24T14:13:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.630+0000 7f1612c97700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f160c066780 0x7f160c066be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.630+0000 7f1612c97700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f160c157480 0x7f160c0a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.630+0000 7f1612c97700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f15f005f270 0x7f15f0061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.630+0000 7f1612c97700 1 --2- 172.21.15.29:0/1645332677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f160c157860 0x7f160c0a9bf0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:33.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.630+0000 7f1612c97700 1 -- 172.21.15.29:0/1645332677 >> 172.21.15.29:0/1645332677 conn(0x7f160c0922f0 msgr2=0x7f160c067da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:33.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.630+0000 7f1612c97700 1 -- 172.21.15.29:0/1645332677 shutdown_connections 2024-01-24T14:13:33.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:33.630+0000 7f1612c97700 1 -- 172.21.15.29:0/1645332677 wait complete. 2024-01-24T14:13:33.643 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-24T14:13:33.643 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T14:13:33.643 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T14:13:33.643 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-24T14:13:33.643 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-24T14:13:33.643 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-24T14:13:33.643 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-24T14:13:33.643 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T14:13:33.643 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T14:13:33.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:13:33.643 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T14:13:33.664 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T14:13:33.665 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T14:13:33.665 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T14:13:33.665 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:13:33.665 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T14:13:33.665 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:13:33.665 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:13:33.665 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T14:13:33.665 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T14:13:33.665 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 41437sec preferred_lft 41437sec 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T14:13:33.666 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff 2024-01-24T14:13:33.667 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T14:13:33.667 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:13:33.667 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::24bc:1cff:fe57:e704/64 scope link 2024-01-24T14:13:33.667 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:13:33.667 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T14:13:33.667 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T14:13:33.667 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::2034:aaff:fef3:f12b/64 scope link 2024-01-24T14:13:33.667 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:13:33.667 INFO:teuthology.orchestra.run.smithi029.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T14:13:33.667 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether ba:ca:42:4a:9e:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T14:13:33.667 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::b8ca:42ff:fe4a:9e52/64 scope link 2024-01-24T14:13:33.667 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:13:33.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:13:33.668 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T14:13:33.736 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T14:13:33.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:13:33.738 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T14:13:33.801 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T14:13:33.801 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T14:13:33.801 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T14:13:33.801 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T14:13:33.802 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T14:13:33.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:13:33.802 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T14:13:33.803 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T14:13:33.803 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T14:13:33.803 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T14:13:33.803 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T14:13:33.803 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T14:13:33.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:13:33.926 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T14:13:33.945 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-24T14:13:33.946 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T14:13:33.946 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T14:13:33.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:13:33.946 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T14:13:34.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:13:34.020 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-01-24T14:13:34.083 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T14:13:34.083 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:13:34.175 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:13:34.175 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T14:13:34.198 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:13:34.198 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T14:13:34.201 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T14:13:34.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:13:34.201 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-24T14:13:34.233 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-24T14:13:34.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.412+0000 7feccde77700 1 Processor -- start 2024-01-24T14:13:34.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.412+0000 7feccde77700 1 -- start start 2024-01-24T14:13:34.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.413+0000 7feccde77700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fecc8151de0 0x7fecc8155470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:34.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.413+0000 7feccde77700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fecc8066bb0 0x7fecc813d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:34.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.413+0000 7feccde77700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc8158800 con 0x7fecc80638c0 2024-01-24T14:13:34.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.413+0000 7feccde77700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc8157ed0 con 0x7fecc8151de0 2024-01-24T14:13:34.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.413+0000 7feccde77700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc81581b0 con 0x7fecc8066bb0 2024-01-24T14:13:34.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.413+0000 7fecc77fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fecc8151de0 0x7fecc8155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:34.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.413+0000 7fecc77fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fecc8151de0 0x7fecc8155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37620/0 (socket says 172.21.15.29:37620) 2024-01-24T14:13:34.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.413+0000 7fecc77fe700 1 -- 172.21.15.29:0/506893826 learned_addr learned my addr 172.21.15.29:0/506893826 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:34.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.413+0000 7fecc7fff700 1 --2- 172.21.15.29:0/506893826 >> v2:172.21.15.163:3300/0 conn(0x7fecc8066bb0 0x7fecc813d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:34.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.414+0000 7fecc77fe700 1 -- 172.21.15.29:0/506893826 >> v2:172.21.15.163:3300/0 conn(0x7fecc8066bb0 msgr2=0x7fecc813d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.414+0000 7fecc77fe700 1 --2- 172.21.15.29:0/506893826 >> v2:172.21.15.163:3300/0 conn(0x7fecc8066bb0 0x7fecc813d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.414+0000 7fecc77fe700 1 -- 172.21.15.29:0/506893826 >> v1:172.21.15.163:6789/0 conn(0x7fecc80638c0 legacy=0x7fecc80664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.414+0000 7fecc77fe700 1 -- 172.21.15.29:0/506893826 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc8158490 con 0x7fecc8151de0 2024-01-24T14:13:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.414+0000 7fecc77fe700 1 --2- 172.21.15.29:0/506893826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecc8151de0 0x7fecc8155470 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fecc80c6680 tx=0x7fecbc0090f0).ready entity=mon.0 client_cookie=7fc18530f51a5e74 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.414+0000 7fecc67fc700 1 -- 172.21.15.29:0/506893826 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00b600 con 0x7fecc8151de0 2024-01-24T14:13:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.414+0000 7fecc67fc700 1 -- 172.21.15.29:0/506893826 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc00b760 con 0x7fecc8151de0 2024-01-24T14:13:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.414+0000 7fecc67fc700 1 -- 172.21.15.29:0/506893826 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00bae0 con 0x7fecc8151de0 2024-01-24T14:13:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.415+0000 7feccde77700 1 -- 172.21.15.29:0/506893826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecc8151de0 msgr2=0x7fecc8155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.415+0000 7feccde77700 1 --2- 172.21.15.29:0/506893826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecc8151de0 0x7fecc8155470 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fecc80c6680 tx=0x7fecbc0090f0).stop 2024-01-24T14:13:34.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.415+0000 7feccde77700 1 -- 172.21.15.29:0/506893826 shutdown_connections 2024-01-24T14:13:34.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.415+0000 7feccde77700 1 --2- 172.21.15.29:0/506893826 >> v2:172.21.15.163:3300/0 conn(0x7fecc8066bb0 0x7fecc813d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.415+0000 7feccde77700 1 --2- 172.21.15.29:0/506893826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecc8151de0 0x7fecc8155470 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.415+0000 7feccde77700 1 -- 172.21.15.29:0/506893826 >> 172.21.15.29:0/506893826 conn(0x7fecc80c5640 msgr2=0x7fecc80c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:34.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.415+0000 7feccde77700 1 -- 172.21.15.29:0/506893826 shutdown_connections 2024-01-24T14:13:34.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.415+0000 7feccde77700 1 -- 172.21.15.29:0/506893826 wait complete. 2024-01-24T14:13:34.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.416+0000 7feccde77700 1 Processor -- start 2024-01-24T14:13:34.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.416+0000 7feccde77700 1 -- start start 2024-01-24T14:13:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7feccde77700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc80638c0 0x7fecc8150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7feccde77700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecc8066bb0 0x7fecc80acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7feccde77700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecc8151de0 0x7fecc80ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7feccde77700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc812fa50 con 0x7fecc8151de0 2024-01-24T14:13:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7feccde77700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc80ad5d0 con 0x7fecc80638c0 2024-01-24T14:13:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7feccde77700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc812ee40 con 0x7fecc8066bb0 2024-01-24T14:13:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecc8151de0 0x7fecc80ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecc8151de0 0x7fecc80ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37622/0 (socket says 172.21.15.29:37622) 2024-01-24T14:13:34.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7fecc7fff700 1 -- 172.21.15.29:0/959038596 learned_addr learned my addr 172.21.15.29:0/959038596 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7fecc77fe700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc80638c0 0x7fecc8150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7fecc6ffd700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecc8066bb0 0x7fecc80acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7fecc7fff700 1 -- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecc8066bb0 msgr2=0x7fecc80acbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7fecc7fff700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecc8066bb0 0x7fecc80acbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7fecc7fff700 1 -- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc80638c0 msgr2=0x7fecc8150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7fecc7fff700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc80638c0 0x7fecc8150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7fecc7fff700 1 -- 172.21.15.29:0/959038596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc009370 con 0x7fecc8151de0 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.417+0000 7fecc6ffd700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecc8066bb0 0x7fecc80acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.418+0000 7fecc7fff700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecc8151de0 0x7fecc80ad0e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7fecc0007f30 tx=0x7fecc00098e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.418+0000 7fecaf7fe700 1 -- 172.21.15.29:0/959038596 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc000ecf0 con 0x7fecc8151de0 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.418+0000 7fecaf7fe700 1 -- 172.21.15.29:0/959038596 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc000ee50 con 0x7fecc8151de0 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.418+0000 7feccde77700 1 -- 172.21.15.29:0/959038596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc812f120 con 0x7fecc8151de0 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.418+0000 7fecaf7fe700 1 -- 172.21.15.29:0/959038596 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0014610 con 0x7fecc8151de0 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.418+0000 7feccde77700 1 -- 172.21.15.29:0/959038596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc812f6e0 con 0x7fecc8151de0 2024-01-24T14:13:34.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.419+0000 7feccde77700 1 -- 172.21.15.29:0/959038596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc80475e0 con 0x7fecc8151de0 2024-01-24T14:13:34.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.419+0000 7fecaf7fe700 1 -- 172.21.15.29:0/959038596 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fecc00149c0 con 0x7fecc8151de0 2024-01-24T14:13:34.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.420+0000 7fecaf7fe700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fecb405f270 0x7fecb4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:34.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.420+0000 7fecaf7fe700 1 -- 172.21.15.29:0/959038596 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fecc007ae90 con 0x7fecc8151de0 2024-01-24T14:13:34.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.420+0000 7fecc77fe700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fecb405f270 0x7fecb4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:34.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.421+0000 7fecc77fe700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fecb405f270 0x7fecb4061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fecbc009130 tx=0x7fecbc017000).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:34.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.424+0000 7fecaf7fe700 1 -- 172.21.15.29:0/959038596 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fecc004bb20 con 0x7fecc8151de0 2024-01-24T14:13:34.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.633+0000 7feccde77700 1 -- 172.21.15.29:0/959038596 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fecc813e020 con 0x7fecb405f270 2024-01-24T14:13:34.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.636+0000 7fecaf7fe700 1 -- 172.21.15.29:0/959038596 <== mgr.5558 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fecc813e020 con 0x7fecb405f270 2024-01-24T14:13:34.636 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T14:13:34.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:34.636 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-01-24T14:13:34.636 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:34.636 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T14:13:34.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.637+0000 7feccde77700 1 -- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fecb405f270 msgr2=0x7fecb4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:34.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.637+0000 7feccde77700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fecb405f270 0x7fecb4061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fecbc009130 tx=0x7fecbc017000).stop 2024-01-24T14:13:34.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.637+0000 7feccde77700 1 -- 172.21.15.29:0/959038596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecc8151de0 msgr2=0x7fecc80ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:34.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.637+0000 7feccde77700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecc8151de0 0x7fecc80ad0e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7fecc0007f30 tx=0x7fecc00098e0).stop 2024-01-24T14:13:34.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.637+0000 7feccde77700 1 -- 172.21.15.29:0/959038596 shutdown_connections 2024-01-24T14:13:34.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.637+0000 7feccde77700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fecb405f270 0x7fecb4061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.637+0000 7feccde77700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fecc8151de0 0x7fecc80ad0e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.637+0000 7feccde77700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc80638c0 0x7fecc8150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.637+0000 7feccde77700 1 --2- 172.21.15.29:0/959038596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecc8066bb0 0x7fecc80acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.637+0000 7feccde77700 1 -- 172.21.15.29:0/959038596 >> 172.21.15.29:0/959038596 conn(0x7fecc80c5640 msgr2=0x7fecc80679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:34.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.637+0000 7feccde77700 1 -- 172.21.15.29:0/959038596 shutdown_connections 2024-01-24T14:13:34.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.637+0000 7feccde77700 1 -- 172.21.15.29:0/959038596 wait complete. 2024-01-24T14:13:34.651 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-01-24T14:13:34.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.782+0000 7f17132c8700 1 Processor -- start 2024-01-24T14:13:34.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.782+0000 7f17132c8700 1 -- start start 2024-01-24T14:13:34.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.783+0000 7f17132c8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f170c14b330 0x7f170c1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:34.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.783+0000 7f17132c8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f170c0c3360 0x7f170c154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:34.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.783+0000 7f17132c8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170c155840 con 0x7f170c14b710 2024-01-24T14:13:34.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.783+0000 7f17132c8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c155390 con 0x7f170c0c3360 2024-01-24T14:13:34.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.783+0000 7f17132c8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c069280 con 0x7f170c14b330 2024-01-24T14:13:34.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1711865700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f170c0c3360 0x7f170c154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:34.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1710863700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f170c14b710 0x7f170c0c2c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38684/0 (socket says 172.21.15.29:38684) 2024-01-24T14:13:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1710863700 1 -- 172.21.15.29:0/2052517488 learned_addr learned my addr 172.21.15.29:0/2052517488 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1711064700 1 --2- 172.21.15.29:0/2052517488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f170c14b330 0x7f170c1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1711865700 1 -- 172.21.15.29:0/2052517488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f170c14b330 msgr2=0x7f170c1549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1711865700 1 --2- 172.21.15.29:0/2052517488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f170c14b330 0x7f170c1549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1711865700 1 -- 172.21.15.29:0/2052517488 >> v1:172.21.15.29:6789/0 conn(0x7f170c14b710 legacy=0x7f170c0c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1711865700 1 -- 172.21.15.29:0/2052517488 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c0c3910 con 0x7f170c0c3360 2024-01-24T14:13:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1703fff700 1 -- 172.21.15.29:0/2052517488 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2607121340 0 0) 0x7f170c155840 con 0x7f170c14b710 2024-01-24T14:13:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1711865700 1 --2- 172.21.15.29:0/2052517488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0c3360 0x7f170c154ea0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f1704009e60 tx=0x7f1704009780).ready entity=mon.0 client_cookie=8925cd857d1b1150 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:34.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1703fff700 1 -- 172.21.15.29:0/2052517488 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170400e070 con 0x7f170c0c3360 2024-01-24T14:13:34.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1703fff700 1 -- 172.21.15.29:0/2052517488 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170400bc30 con 0x7f170c0c3360 2024-01-24T14:13:34.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.784+0000 7f1703fff700 1 -- 172.21.15.29:0/2052517488 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704012410 con 0x7f170c0c3360 2024-01-24T14:13:34.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.785+0000 7f17132c8700 1 -- 172.21.15.29:0/2052517488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0c3360 msgr2=0x7f170c154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:34.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.785+0000 7f17132c8700 1 --2- 172.21.15.29:0/2052517488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0c3360 0x7f170c154ea0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f1704009e60 tx=0x7f1704009780).stop 2024-01-24T14:13:34.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.785+0000 7f17132c8700 1 -- 172.21.15.29:0/2052517488 shutdown_connections 2024-01-24T14:13:34.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.785+0000 7f17132c8700 1 --2- 172.21.15.29:0/2052517488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0c3360 0x7f170c154ea0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.785+0000 7f17132c8700 1 --2- 172.21.15.29:0/2052517488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f170c14b330 0x7f170c1549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.785+0000 7f17132c8700 1 -- 172.21.15.29:0/2052517488 >> 172.21.15.29:0/2052517488 conn(0x7f170c064390 msgr2=0x7f170c0c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:34.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.785+0000 7f17132c8700 1 -- 172.21.15.29:0/2052517488 shutdown_connections 2024-01-24T14:13:34.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.785+0000 7f17132c8700 1 -- 172.21.15.29:0/2052517488 wait complete. 2024-01-24T14:13:34.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.786+0000 7f17132c8700 1 Processor -- start 2024-01-24T14:13:34.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.786+0000 7f17132c8700 1 -- start start 2024-01-24T14:13:34.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.787+0000 7f17132c8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0c3360 0x7f170c136c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:34.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.787+0000 7f17132c8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f170c14b330 0x7f170c147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:34.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.787+0000 7f17132c8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f170c14b710 0x7f170c0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:34.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.787+0000 7f17132c8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0aca60 con 0x7f170c0c3360 2024-01-24T14:13:34.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.787+0000 7f17132c8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c1654e0 con 0x7f170c14b330 2024-01-24T14:13:34.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.787+0000 7f17132c8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c137100 con 0x7f170c14b710 2024-01-24T14:13:34.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.787+0000 7f1711064700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0c3360 0x7f170c136c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:34.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.787+0000 7f1711064700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0c3360 0x7f170c136c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37630/0 (socket says 172.21.15.29:37630) 2024-01-24T14:13:34.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f1711064700 1 -- 172.21.15.29:0/3311297370 learned_addr learned my addr 172.21.15.29:0/3311297370 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:34.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f1711865700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f170c14b710 0x7f170c0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:34.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f1710863700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f170c14b330 0x7f170c147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:34.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f1711064700 1 -- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f170c14b710 msgr2=0x7f170c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:34.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f1711064700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f170c14b710 0x7f170c0627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f1711064700 1 -- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f170c14b330 msgr2=0x7f170c147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:34.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f1711064700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f170c14b330 0x7f170c147280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:34.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f1711064700 1 -- 172.21.15.29:0/3311297370 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1704009a20 con 0x7f170c0c3360 2024-01-24T14:13:34.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f1710863700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f170c14b330 0x7f170c147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:34.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f1711865700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f170c14b710 0x7f170c0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:34.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f1711064700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0c3360 0x7f170c136c10 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f16fc00d890 tx=0x7f16fc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:34.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f17017fa700 1 -- 172.21.15.29:0/3311297370 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0125f0 con 0x7f170c0c3360 2024-01-24T14:13:34.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f17017fa700 1 -- 172.21.15.29:0/3311297370 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc012750 con 0x7f170c0c3360 2024-01-24T14:13:34.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f17132c8700 1 -- 172.21.15.29:0/3311297370 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c147770 con 0x7f170c0c3360 2024-01-24T14:13:34.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f17017fa700 1 -- 172.21.15.29:0/3311297370 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc012ad0 con 0x7f170c0c3360 2024-01-24T14:13:34.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.788+0000 7f17132c8700 1 -- 172.21.15.29:0/3311297370 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c0ac0c0 con 0x7f170c0c3360 2024-01-24T14:13:34.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.789+0000 7f17132c8700 1 -- 172.21.15.29:0/3311297370 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c0acba0 con 0x7f170c0c3360 2024-01-24T14:13:34.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.790+0000 7f17017fa700 1 -- 172.21.15.29:0/3311297370 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f16fc013480 con 0x7f170c0c3360 2024-01-24T14:13:34.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.790+0000 7f17017fa700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f16f005f270 0x7f16f0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.790+0000 7f17017fa700 1 -- 172.21.15.29:0/3311297370 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f16fc07a5f0 con 0x7f170c0c3360 2024-01-24T14:13:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.790+0000 7f1710863700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f16f005f270 0x7f16f0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:34.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.791+0000 7f1710863700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f16f005f270 0x7f16f0061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f170c145620 tx=0x7f1708009040).ready entity=mgr.5558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:34.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:34.795+0000 7f17017fa700 1 -- 172.21.15.29:0/3311297370 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f16fc04b2e0 con 0x7f170c0c3360 2024-01-24T14:13:35.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.029+0000 7f17132c8700 1 -- 172.21.15.29:0/3311297370 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f170c0475e0 con 0x7f170c0c3360 2024-01-24T14:13:35.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.300+0000 7f17017fa700 1 -- 172.21.15.29:0/3311297370 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 25) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f16fc077e40 con 0x7f170c0c3360 2024-01-24T14:13:35.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.303+0000 7f17017fa700 1 -- 172.21.15.29:0/3311297370 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v25) v1 ==== 92+0+0 (secure 0 0 0) 0x7f16fc072bd0 con 0x7f170c0c3360 2024-01-24T14:13:35.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.306+0000 7f17132c8700 1 -- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f16f005f270 msgr2=0x7f16f0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:35.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.306+0000 7f17132c8700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f16f005f270 0x7f16f0061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f170c145620 tx=0x7f1708009040).stop 2024-01-24T14:13:35.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.306+0000 7f17132c8700 1 -- 172.21.15.29:0/3311297370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0c3360 msgr2=0x7f170c136c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:35.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.306+0000 7f17132c8700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0c3360 0x7f170c136c10 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f16fc00d890 tx=0x7f16fc008040).stop 2024-01-24T14:13:35.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.306+0000 7f17132c8700 1 -- 172.21.15.29:0/3311297370 shutdown_connections 2024-01-24T14:13:35.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.306+0000 7f17132c8700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f170c14b710 0x7f170c0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.306+0000 7f17132c8700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f16f005f270 0x7f16f0061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.306+0000 7f17132c8700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0c3360 0x7f170c136c10 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.306+0000 7f17132c8700 1 --2- 172.21.15.29:0/3311297370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f170c14b330 0x7f170c147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.306+0000 7f17132c8700 1 -- 172.21.15.29:0/3311297370 >> 172.21.15.29:0/3311297370 conn(0x7f170c064390 msgr2=0x7f170c153400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:35.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.306+0000 7f17132c8700 1 -- 172.21.15.29:0/3311297370 shutdown_connections 2024-01-24T14:13:35.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.306+0000 7f17132c8700 1 -- 172.21.15.29:0/3311297370 wait complete. 2024-01-24T14:13:35.327 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-01-24T14:13:35.449 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:35.449+0000 7fab33335680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:13:35.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.464+0000 7f0e26046700 1 Processor -- start 2024-01-24T14:13:35.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.464+0000 7f0e26046700 1 -- start start 2024-01-24T14:13:35.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e26046700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0e20064760 0x7f0e20064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:35.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e26046700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e200bfd40 0x7f0e200c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:35.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e26046700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e20152240 con 0x7f0e200c8490 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e26046700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e201586c0 con 0x7f0e200bfd40 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e26046700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e200aa670 con 0x7f0e20064760 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e1ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e200bfd40 0x7f0e200c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e1ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e200bfd40 0x7f0e200c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37638/0 (socket says 172.21.15.29:37638) 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e1f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0e20064760 0x7f0e20064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e1ffff700 1 -- 172.21.15.29:0/2596744477 learned_addr learned my addr 172.21.15.29:0/2596744477 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e1effd700 1 -- 172.21.15.29:0/2596744477 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584592736 0 0) 0x7f0e20152240 con 0x7f0e200c8490 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e1effd700 1 -- 172.21.15.29:0/2596744477 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e0c0033f0 con 0x7f0e200c8490 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e1f7fe700 1 -- 172.21.15.29:0/2596744477 >> v2:172.21.15.29:3300/0 conn(0x7f0e200bfd40 msgr2=0x7f0e200c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e1f7fe700 1 --2- 172.21.15.29:0/2596744477 >> v2:172.21.15.29:3300/0 conn(0x7f0e200bfd40 0x7f0e200c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e1f7fe700 1 -- 172.21.15.29:0/2596744477 >> v1:172.21.15.163:6789/0 conn(0x7f0e200c8490 legacy=0x7f0e200a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e1f7fe700 1 -- 172.21.15.29:0/2596744477 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20157d90 con 0x7f0e20064760 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.465+0000 7f0e1ffff700 1 --2- 172.21.15.29:0/2596744477 >> v2:172.21.15.29:3300/0 conn(0x7f0e200bfd40 0x7f0e200c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.466+0000 7f0e1f7fe700 1 --2- 172.21.15.29:0/2596744477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e20064760 0x7f0e20064b40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f0e20084450 tx=0x7f0e10009060).ready entity=mon.1 client_cookie=8a4d631c14185380 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:35.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.466+0000 7f0e1effd700 1 -- 172.21.15.29:0/2596744477 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000f510 con 0x7f0e20064760 2024-01-24T14:13:35.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.466+0000 7f0e1effd700 1 -- 172.21.15.29:0/2596744477 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1000f670 con 0x7f0e20064760 2024-01-24T14:13:35.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.466+0000 7f0e1effd700 1 -- 172.21.15.29:0/2596744477 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000f9f0 con 0x7f0e20064760 2024-01-24T14:13:35.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.466+0000 7f0e26046700 1 -- 172.21.15.29:0/2596744477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e20064760 msgr2=0x7f0e20064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:35.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.466+0000 7f0e26046700 1 --2- 172.21.15.29:0/2596744477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e20064760 0x7f0e20064b40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f0e20084450 tx=0x7f0e10009060).stop 2024-01-24T14:13:35.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.466+0000 7f0e26046700 1 -- 172.21.15.29:0/2596744477 shutdown_connections 2024-01-24T14:13:35.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.466+0000 7f0e26046700 1 --2- 172.21.15.29:0/2596744477 >> v2:172.21.15.29:3300/0 conn(0x7f0e200bfd40 0x7f0e200c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.466+0000 7f0e26046700 1 --2- 172.21.15.29:0/2596744477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e20064760 0x7f0e20064b40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.466+0000 7f0e26046700 1 -- 172.21.15.29:0/2596744477 >> 172.21.15.29:0/2596744477 conn(0x7f0e200cb8b0 msgr2=0x7f0e200c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:35.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.467+0000 7f0e26046700 1 -- 172.21.15.29:0/2596744477 shutdown_connections 2024-01-24T14:13:35.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.467+0000 7f0e26046700 1 -- 172.21.15.29:0/2596744477 wait complete. 2024-01-24T14:13:35.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.468+0000 7f0e26046700 1 Processor -- start 2024-01-24T14:13:35.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.468+0000 7f0e26046700 1 -- start start 2024-01-24T14:13:35.470 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:35.468+0000 7fa35a4eb680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:13:35.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e26046700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e20064760 0x7f0e200add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:35.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e26046700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e200bfd40 0x7f0e20154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:35.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e26046700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c8490 0x7f0e200b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e26046700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200ae280 con 0x7f0e20064760 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e26046700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20154f50 con 0x7f0e200c8490 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e26046700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200b0570 con 0x7f0e200bfd40 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e1f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e20064760 0x7f0e200add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e1f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e20064760 0x7f0e200add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37654/0 (socket says 172.21.15.29:37654) 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e1f7fe700 1 -- 172.21.15.29:0/4185286340 learned_addr learned my addr 172.21.15.29:0/4185286340 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e1ffff700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c8490 0x7f0e200b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e16ffd700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e200bfd40 0x7f0e20154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e1f7fe700 1 -- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e200bfd40 msgr2=0x7f0e20154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e1f7fe700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e200bfd40 0x7f0e20154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e1f7fe700 1 -- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c8490 msgr2=0x7f0e200b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e1f7fe700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c8490 0x7f0e200b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e1f7fe700 1 -- 172.21.15.29:0/4185286340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e18009ae0 con 0x7f0e20064760 2024-01-24T14:13:35.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.469+0000 7f0e1ffff700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c8490 0x7f0e200b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:35.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.470+0000 7f0e1f7fe700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e20064760 0x7f0e200add90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f0e10009d20 tx=0x7f0e10017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:35.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.470+0000 7f0e17fff700 1 -- 172.21.15.29:0/4185286340 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1001a4d0 con 0x7f0e20064760 2024-01-24T14:13:35.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.470+0000 7f0e17fff700 1 -- 172.21.15.29:0/4185286340 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1001e070 con 0x7f0e20064760 2024-01-24T14:13:35.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.470+0000 7f0e26046700 1 -- 172.21.15.29:0/4185286340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e100092a0 con 0x7f0e20064760 2024-01-24T14:13:35.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.470+0000 7f0e17fff700 1 -- 172.21.15.29:0/4185286340 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1001a860 con 0x7f0e20064760 2024-01-24T14:13:35.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.470+0000 7f0e26046700 1 -- 172.21.15.29:0/4185286340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e200ae770 con 0x7f0e20064760 2024-01-24T14:13:35.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.471+0000 7f0e26046700 1 -- 172.21.15.29:0/4185286340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e200af380 con 0x7f0e20064760 2024-01-24T14:13:35.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.471+0000 7f0e17fff700 1 -- 172.21.15.29:0/4185286340 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 25) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f0e10020030 con 0x7f0e20064760 2024-01-24T14:13:35.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.472+0000 7f0e17fff700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0e0005f2c0 0x7f0e00061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:35.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.472+0000 7f0e17fff700 1 -- 172.21.15.29:0/4185286340 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f0e1007b830 con 0x7f0e20064760 2024-01-24T14:13:35.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.472+0000 7f0e16ffd700 1 -- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0e0005f2c0 msgr2=0x7f0e00061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:35.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.472+0000 7f0e16ffd700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0e0005f2c0 0x7f0e00061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:35.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.477+0000 7f0e17fff700 1 -- 172.21.15.29:0/4185286340 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0e1004c640 con 0x7f0e20064760 2024-01-24T14:13:35.513 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:35.514+0000 7fab33335680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:13:35.543 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:35.542+0000 7fa35a4eb680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:13:35.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.661+0000 7f0e26046700 1 -- 172.21.15.29:0/4185286340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f0e200475e0 con 0x7f0e20064760 2024-01-24T14:13:35.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.665+0000 7f0e17fff700 1 -- 172.21.15.29:0/4185286340 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0e1004fc60 con 0x7f0e20064760 2024-01-24T14:13:35.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.667+0000 7f0e26046700 1 -- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0e0005f2c0 msgr2=0x7f0e00061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:35.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.667+0000 7f0e26046700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0e0005f2c0 0x7f0e00061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:35.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.667+0000 7f0e26046700 1 -- 172.21.15.29:0/4185286340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e20064760 msgr2=0x7f0e200add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:35.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.667+0000 7f0e26046700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e20064760 0x7f0e200add90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f0e10009d20 tx=0x7f0e10017000).stop 2024-01-24T14:13:35.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.667+0000 7f0e26046700 1 -- 172.21.15.29:0/4185286340 shutdown_connections 2024-01-24T14:13:35.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.667+0000 7f0e26046700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c8490 0x7f0e200b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.667+0000 7f0e26046700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0e0005f2c0 0x7f0e00061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:35.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.667+0000 7f0e26046700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e20064760 0x7f0e200add90 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.667+0000 7f0e26046700 1 --2- 172.21.15.29:0/4185286340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e200bfd40 0x7f0e20154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.667+0000 7f0e26046700 1 -- 172.21.15.29:0/4185286340 >> 172.21.15.29:0/4185286340 conn(0x7f0e200cb8b0 msgr2=0x7f0e20150370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:35.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.667+0000 7f0e26046700 1 -- 172.21.15.29:0/4185286340 shutdown_connections 2024-01-24T14:13:35.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.667+0000 7f0e26046700 1 -- 172.21.15.29:0/4185286340 wait complete. 2024-01-24T14:13:35.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:13:35.682 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-01-24T14:13:35.706 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2024-01-24T14:13:35.790 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:35.790+0000 7fab33335680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:13:35.865 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:35.864+0000 7fa35a4eb680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:13:35.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.883+0000 7fa7737a1700 1 Processor -- start 2024-01-24T14:13:35.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.883+0000 7fa7737a1700 1 -- start start 2024-01-24T14:13:35.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.884+0000 7fa7737a1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa76c066450 0x7fa76c066830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.884+0000 7fa7737a1700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76c1530b0 con 0x7fa76c159da0 2024-01-24T14:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.884+0000 7fa7737a1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76c153300 con 0x7fa76c152540 2024-01-24T14:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.884+0000 7fa7737a1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c067b10 con 0x7fa76c066450 2024-01-24T14:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.884+0000 7fa771d3e700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa76c159da0 0x7fa76c15c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38710/0 (socket says 172.21.15.29:38710) 2024-01-24T14:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.884+0000 7fa771d3e700 1 -- 172.21.15.29:0/3709882132 learned_addr learned my addr 172.21.15.29:0/3709882132 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.884+0000 7fa77153d700 1 --2- 172.21.15.29:0/3709882132 >> v2:172.21.15.163:3300/0 conn(0x7fa76c066450 0x7fa76c066830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:35.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.884+0000 7fa763fff700 1 -- 172.21.15.29:0/3709882132 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310549696 0 0) 0x7fa76c1530b0 con 0x7fa76c159da0 2024-01-24T14:13:35.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.885+0000 7fa763fff700 1 -- 172.21.15.29:0/3709882132 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7580033f0 con 0x7fa76c159da0 2024-01-24T14:13:35.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.885+0000 7fa763fff700 1 -- 172.21.15.29:0/3709882132 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317715410 0 0) 0x7fa76c153300 con 0x7fa76c152540 2024-01-24T14:13:35.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.885+0000 7fa763fff700 1 -- 172.21.15.29:0/3709882132 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa76c1530b0 con 0x7fa76c152540 2024-01-24T14:13:35.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.885+0000 7fa77153d700 1 -- 172.21.15.29:0/3709882132 >> v1:172.21.15.163:6789/0 conn(0x7fa76c152540 legacy=0x7fa76c1529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:35.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.885+0000 7fa77153d700 1 -- 172.21.15.29:0/3709882132 >> v1:172.21.15.29:6789/0 conn(0x7fa76c159da0 legacy=0x7fa76c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:35.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.885+0000 7fa77153d700 1 -- 172.21.15.29:0/3709882132 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c067df0 con 0x7fa76c066450 2024-01-24T14:13:35.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.885+0000 7fa77153d700 1 --2- 172.21.15.29:0/3709882132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76c066450 0x7fa76c066830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fa768009760 tx=0x7fa768009060).ready entity=mon.1 client_cookie=f040a2d88607a4a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:35.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.885+0000 7fa763fff700 1 -- 172.21.15.29:0/3709882132 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800d4a0 con 0x7fa76c066450 2024-01-24T14:13:35.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.885+0000 7fa763fff700 1 -- 172.21.15.29:0/3709882132 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76800d600 con 0x7fa76c066450 2024-01-24T14:13:35.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.885+0000 7fa763fff700 1 -- 172.21.15.29:0/3709882132 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800d980 con 0x7fa76c066450 2024-01-24T14:13:35.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.886+0000 7fa7737a1700 1 -- 172.21.15.29:0/3709882132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76c066450 msgr2=0x7fa76c066830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:35.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.886+0000 7fa7737a1700 1 --2- 172.21.15.29:0/3709882132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76c066450 0x7fa76c066830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fa768009760 tx=0x7fa768009060).stop 2024-01-24T14:13:35.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.886+0000 7fa7737a1700 1 -- 172.21.15.29:0/3709882132 shutdown_connections 2024-01-24T14:13:35.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.886+0000 7fa7737a1700 1 --2- 172.21.15.29:0/3709882132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76c066450 0x7fa76c066830 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.886+0000 7fa7737a1700 1 -- 172.21.15.29:0/3709882132 >> 172.21.15.29:0/3709882132 conn(0x7fa76c0c8bd0 msgr2=0x7fa76c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:35.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.886+0000 7fa7737a1700 1 -- 172.21.15.29:0/3709882132 shutdown_connections 2024-01-24T14:13:35.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.886+0000 7fa7737a1700 1 -- 172.21.15.29:0/3709882132 wait complete. 2024-01-24T14:13:35.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.888+0000 7fa7737a1700 1 Processor -- start 2024-01-24T14:13:35.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.888+0000 7fa7737a1700 1 -- start start 2024-01-24T14:13:35.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa7737a1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76c066450 0x7fa76c155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:35.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa7737a1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76c152540 0x7fa76c14cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:35.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa7737a1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa76c159da0 0x7fa76c162900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:35.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa7737a1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c153b60 con 0x7fa76c152540 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa7737a1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c14f250 con 0x7fa76c066450 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa7737a1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c14f530 con 0x7fa76c159da0 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa770d3c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76c152540 0x7fa76c14cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa77153d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76c066450 0x7fa76c155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa770d3c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76c152540 0x7fa76c14cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37664/0 (socket says 172.21.15.29:37664) 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa770d3c700 1 -- 172.21.15.29:0/2289502567 learned_addr learned my addr 172.21.15.29:0/2289502567 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa771d3e700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa76c159da0 0x7fa76c162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa77153d700 1 -- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa76c159da0 msgr2=0x7fa76c162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa77153d700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa76c159da0 0x7fa76c162900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa77153d700 1 -- 172.21.15.29:0/2289502567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76c152540 msgr2=0x7fa76c14cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa77153d700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76c152540 0x7fa76c14cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa77153d700 1 -- 172.21.15.29:0/2289502567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7680092a0 con 0x7fa76c066450 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.889+0000 7fa770d3c700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76c152540 0x7fa76c14cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:35.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.890+0000 7fa77153d700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76c066450 0x7fa76c155f60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fa768013000 tx=0x7fa768016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.890+0000 7fa7617fa700 1 -- 172.21.15.29:0/2289502567 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800dcf0 con 0x7fa76c066450 2024-01-24T14:13:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.890+0000 7fa7617fa700 1 -- 172.21.15.29:0/2289502567 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76800de50 con 0x7fa76c066450 2024-01-24T14:13:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.890+0000 7fa7737a1700 1 -- 172.21.15.29:0/2289502567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76c14f810 con 0x7fa76c066450 2024-01-24T14:13:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.890+0000 7fa7617fa700 1 -- 172.21.15.29:0/2289502567 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800d980 con 0x7fa76c066450 2024-01-24T14:13:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.890+0000 7fa7737a1700 1 -- 172.21.15.29:0/2289502567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76c14c140 con 0x7fa76c066450 2024-01-24T14:13:35.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.891+0000 7fa7737a1700 1 -- 172.21.15.29:0/2289502567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76c14c330 con 0x7fa76c066450 2024-01-24T14:13:35.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.892+0000 7fa7617fa700 1 -- 172.21.15.29:0/2289502567 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 25) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fa76801d640 con 0x7fa76c066450 2024-01-24T14:13:35.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.892+0000 7fa7617fa700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 0x7fa750061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:35.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.892+0000 7fa770d3c700 1 -- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 msgr2=0x7fa750061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:35.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.892+0000 7fa7617fa700 1 -- 172.21.15.29:0/2289502567 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fa76807beb0 con 0x7fa76c066450 2024-01-24T14:13:35.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.892+0000 7fa770d3c700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 0x7fa750061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:35.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:35.897+0000 7fa7617fa700 1 -- 172.21.15.29:0/2289502567 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa76804cc00 con 0x7fa76c066450 2024-01-24T14:13:36.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:36.093+0000 7fa770d3c700 1 -- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 msgr2=0x7fa750061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:36.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:36.093+0000 7fa770d3c700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 0x7fa750061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:13:36.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:36.107+0000 7fa7737a1700 1 -- 172.21.15.29:0/2289502567 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa76c153ca0 con 0x7fa75005f2c0 2024-01-24T14:13:36.415 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:36.415+0000 7fab33335680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:13:36.487 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:36.487+0000 7fab33335680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:13:36.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:36.494+0000 7fa770d3c700 1 -- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 msgr2=0x7fa750061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:36.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:36.494+0000 7fa770d3c700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 0x7fa750061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-01-24T14:13:36.593 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:36.592+0000 7fa35a4eb680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:13:36.667 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:36.667+0000 7fa35a4eb680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:13:36.674 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:36.675+0000 7fab33335680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:13:36.882 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:36.881+0000 7fa35a4eb680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:13:37.097 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:37.097+0000 7fab33335680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:13:37.202 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:37.202+0000 7fab33335680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:13:37.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:37.295+0000 7fa770d3c700 1 -- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 msgr2=0x7fa750061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:37.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:37.295+0000 7fa770d3c700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 0x7fa750061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-01-24T14:13:37.355 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:37.355+0000 7fab33335680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:13:37.361 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:37.361+0000 7fa35a4eb680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:13:37.430 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:37.430+0000 7fab33335680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:13:37.480 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:37.479+0000 7fa35a4eb680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:13:37.498 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:37.498+0000 7fab33335680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:13:37.656 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:37.656+0000 7fa35a4eb680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:13:37.741 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:37.741+0000 7fa35a4eb680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:13:37.754 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:37.754+0000 7fab33335680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:13:37.818 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:37.818+0000 7fa35a4eb680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:13:37.937 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:37.937+0000 7fab33335680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:13:38.113 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:38.113+0000 7fa35a4eb680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:13:38.324 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:38.323+0000 7fa35a4eb680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:13:38.591 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:38.591+0000 7fab33335680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:13:38.659 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:38.658+0000 7fab33335680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:13:38.735 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:38.734+0000 7fab33335680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:13:38.882 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:38.881+0000 7fab33335680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:13:38.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:38.897+0000 7fa770d3c700 1 -- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 msgr2=0x7fa750061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:38.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:38.897+0000 7fa770d3c700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 0x7fa750061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-01-24T14:13:38.953 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:38.952+0000 7fab33335680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:13:39.069 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.068+0000 7fab33335680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:13:39.081 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:39.080+0000 7fa35a4eb680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:13:39.160 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:39.160+0000 7fa35a4eb680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:13:39.177 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.176+0000 7fab33335680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:13:39.246 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:39.245+0000 7fa35a4eb680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:13:39.413 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:39.413+0000 7fa35a4eb680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:13:39.468 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.468+0000 7fab33335680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:13:39.494 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:39.494+0000 7fa35a4eb680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:13:39.539 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.539+0000 7fab33335680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:13:39.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:39.565+0000 7fa7617fa700 1 -- 172.21.15.29:0/2289502567 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mgrmap(e 26) v1 ==== 75287+0+0 (secure 0 0 0) 0x7fa768079700 con 0x7fa76c066450 2024-01-24T14:13:39.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:39.565+0000 7fa7617fa700 1 -- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 msgr2=0x7fa750061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:39.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:39.565+0000 7fa7617fa700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 0x7fa750061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:39.632 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.630+0000 7faaa90af700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:39.632 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.630+0000 7faaa90af700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:39.632 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.630+0000 7faaa90af700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:39.632 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.630+0000 7faaa90af700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:39.632 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.630+0000 7faaa90af700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:39.632 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.630+0000 7faaa70ab700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:39.632 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.630+0000 7faaa70ab700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:39.632 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.630+0000 7faaa70ab700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:39.632 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.630+0000 7faaa70ab700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:39.632 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:39.630+0000 7faaa70ab700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:39.634 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:39.633+0000 7fa35a4eb680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:13:39.758 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:39.758+0000 7fa35a4eb680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:13:40.093 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:40.092+0000 7fa35a4eb680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:13:40.173 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:40.173+0000 7fa35a4eb680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:13:40.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.565+0000 7fa7617fa700 1 -- 172.21.15.29:0/2289502567 <== mon.1 v2:172.21.15.163:3300/0 8 ==== mgrmap(e 27) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fa768079a80 con 0x7fa76c066450 2024-01-24T14:13:40.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.565+0000 7fa7617fa700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 0x7fa750061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:40.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.565+0000 7fa7617fa700 1 -- 172.21.15.29:0/2289502567 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa750065e50 con 0x7fa75005f2c0 2024-01-24T14:13:40.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.567+0000 7fa770d3c700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 0x7fa750061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:40.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.568+0000 7fa770d3c700 1 --2- 172.21.15.29:0/2289502567 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa75005f2c0 0x7fa750061700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fa76c150550 tx=0x7fa75c007c50).ready entity=mgr.6298 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:40.571 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:40.570+0000 7faab20c1700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2024-01-24T14:13:40.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.571+0000 7fa7617fa700 1 -- 172.21.15.29:0/2289502567 <== mgr.6298 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7fa750065e50 con 0x7fa75005f2c0 2024-01-24T14:13:40.573 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2024-01-24T14:13:40.577 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-01-24T14:13:40.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:13:40.578 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-01-24T14:13:40.600 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-01-24T14:13:40.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.781+0000 7f6764df7700 1 Processor -- start 2024-01-24T14:13:40.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.781+0000 7f6764df7700 1 -- start start 2024-01-24T14:13:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.782+0000 7f6764df7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f676014b710 0x7f67600c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.782+0000 7f6764df7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67601550c0 0x7f67600c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.782+0000 7f6764df7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6760155840 con 0x7f676014b330 2024-01-24T14:13:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.782+0000 7f6764df7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6760069280 con 0x7f676014b710 2024-01-24T14:13:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.782+0000 7f6764df7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67600c3910 con 0x7f67601550c0 2024-01-24T14:13:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675ed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67601550c0 0x7f67600c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675ed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67601550c0 0x7f67600c3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:35802/0 (socket says 172.21.15.29:35802) 2024-01-24T14:13:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675dd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f676014b710 0x7f67600c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675ed9d700 1 -- 172.21.15.29:0/1600667754 learned_addr learned my addr 172.21.15.29:0/1600667754 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675ed9d700 1 -- 172.21.15.29:0/1600667754 >> v2:172.21.15.163:3300/0 conn(0x7f676014b710 msgr2=0x7f67600c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675ed9d700 1 --2- 172.21.15.29:0/1600667754 >> v2:172.21.15.163:3300/0 conn(0x7f676014b710 0x7f67600c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:40.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675ed9d700 1 -- 172.21.15.29:0/1600667754 >> v1:172.21.15.163:6789/0 conn(0x7f676014b330 legacy=0x7f67601549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:40.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675ed9d700 1 -- 172.21.15.29:0/1600667754 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67600c3bf0 con 0x7f67601550c0 2024-01-24T14:13:40.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675dd9b700 1 --2- 172.21.15.29:0/1600667754 >> v2:172.21.15.163:3300/0 conn(0x7f676014b710 0x7f67600c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:40.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675ed9d700 1 --2- 172.21.15.29:0/1600667754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67601550c0 0x7f67600c3140 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f6758009f20 tx=0x7f6758009840).ready entity=mon.2 client_cookie=e345db5f7176c1b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:40.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675d59a700 1 -- 172.21.15.29:0/1600667754 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675800e070 con 0x7f67601550c0 2024-01-24T14:13:40.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675d59a700 1 -- 172.21.15.29:0/1600667754 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675800b9d0 con 0x7f67601550c0 2024-01-24T14:13:40.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.783+0000 7f675d59a700 1 -- 172.21.15.29:0/1600667754 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675800bd50 con 0x7f67601550c0 2024-01-24T14:13:40.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.784+0000 7f6764df7700 1 -- 172.21.15.29:0/1600667754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67601550c0 msgr2=0x7f67600c3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:40.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.784+0000 7f6764df7700 1 --2- 172.21.15.29:0/1600667754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67601550c0 0x7f67600c3140 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f6758009f20 tx=0x7f6758009840).stop 2024-01-24T14:13:40.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.784+0000 7f6764df7700 1 -- 172.21.15.29:0/1600667754 shutdown_connections 2024-01-24T14:13:40.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.784+0000 7f6764df7700 1 --2- 172.21.15.29:0/1600667754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67601550c0 0x7f67600c3140 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:40.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.784+0000 7f6764df7700 1 --2- 172.21.15.29:0/1600667754 >> v2:172.21.15.163:3300/0 conn(0x7f676014b710 0x7f67600c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:40.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.784+0000 7f6764df7700 1 -- 172.21.15.29:0/1600667754 >> 172.21.15.29:0/1600667754 conn(0x7f6760064390 msgr2=0x7f67600c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:40.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.784+0000 7f6764df7700 1 -- 172.21.15.29:0/1600667754 shutdown_connections 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.784+0000 7f6764df7700 1 -- 172.21.15.29:0/1600667754 wait complete. 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.785+0000 7f6764df7700 1 Processor -- start 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.785+0000 7f6764df7700 1 -- start start 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.786+0000 7f6764df7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f676014b330 0x7f67600aad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.786+0000 7f6764df7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f676014b710 0x7f67600ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.786+0000 7f6764df7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67601550c0 0x7f6760143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.786+0000 7f6764df7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67601512f0 con 0x7f67601550c0 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.786+0000 7f6764df7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67600abbb0 con 0x7f676014b330 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.786+0000 7f6764df7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6760150af0 con 0x7f676014b710 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.786+0000 7f675ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67601550c0 0x7f6760143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.786+0000 7f675ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67601550c0 0x7f6760143cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37680/0 (socket says 172.21.15.29:37680) 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.786+0000 7f675ed9d700 1 -- 172.21.15.29:0/1453227194 learned_addr learned my addr 172.21.15.29:0/1453227194 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.786+0000 7f675e59c700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f676014b330 0x7f67600aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.786+0000 7f675dd9b700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f676014b710 0x7f67600ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:40.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.787+0000 7f675ed9d700 1 -- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f676014b710 msgr2=0x7f67600ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.787+0000 7f675ed9d700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f676014b710 0x7f67600ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.787+0000 7f675ed9d700 1 -- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f676014b330 msgr2=0x7f67600aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.787+0000 7f675ed9d700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f676014b330 0x7f67600aad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.787+0000 7f675ed9d700 1 -- 172.21.15.29:0/1453227194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67500092a0 con 0x7f67601550c0 2024-01-24T14:13:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.787+0000 7f675e59c700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f676014b330 0x7f67600aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.787+0000 7f675ed9d700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67601550c0 0x7f6760143cc0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f675800b610 tx=0x7f6758009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.787+0000 7f67567fc700 1 -- 172.21.15.29:0/1453227194 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580092e0 con 0x7f67601550c0 2024-01-24T14:13:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.787+0000 7f67567fc700 1 -- 172.21.15.29:0/1453227194 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6758007610 con 0x7f67601550c0 2024-01-24T14:13:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.787+0000 7f6764df7700 1 -- 172.21.15.29:0/1453227194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6758009ae0 con 0x7f67601550c0 2024-01-24T14:13:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.787+0000 7f67567fc700 1 -- 172.21.15.29:0/1453227194 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580079d0 con 0x7f67601550c0 2024-01-24T14:13:40.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.787+0000 7f6764df7700 1 -- 172.21.15.29:0/1453227194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6760151010 con 0x7f67601550c0 2024-01-24T14:13:40.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.788+0000 7f6764df7700 1 -- 172.21.15.29:0/1453227194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6760152510 con 0x7f67601550c0 2024-01-24T14:13:40.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.789+0000 7f67567fc700 1 -- 172.21.15.29:0/1453227194 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 27) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f6758003040 con 0x7f67601550c0 2024-01-24T14:13:40.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.789+0000 7f67567fc700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f674005f080 0x7f6740061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:40.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.789+0000 7f67567fc700 1 -- 172.21.15.29:0/1453227194 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f675807bc60 con 0x7f67601550c0 2024-01-24T14:13:40.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.789+0000 7f675e59c700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f674005f080 0x7f6740061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:40.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.790+0000 7f675e59c700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f674005f080 0x7f6740061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f6748001c40 tx=0x7f6748009040).ready entity=mgr.6298 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:40.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:40.793+0000 7f67567fc700 1 -- 172.21.15.29:0/1453227194 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f6760152510 con 0x7f67601550c0 2024-01-24T14:13:41.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.022+0000 7f6764df7700 1 -- 172.21.15.29:0/1453227194 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f6760152510 con 0x7f67601550c0 2024-01-24T14:13:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.834+0000 7f67567fc700 1 -- 172.21.15.29:0/1453227194 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f6758079700 con 0x7f67601550c0 2024-01-24T14:13:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.835+0000 7f67567fc700 1 -- 172.21.15.29:0/1453227194 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v28) v1 ==== 93+0+0 (secure 0 0 0) 0x7f6758041b80 con 0x7f67601550c0 2024-01-24T14:13:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.838+0000 7f6764df7700 1 -- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f674005f080 msgr2=0x7f6740061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.838+0000 7f6764df7700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f674005f080 0x7f6740061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f6748001c40 tx=0x7f6748009040).stop 2024-01-24T14:13:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.838+0000 7f6764df7700 1 -- 172.21.15.29:0/1453227194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67601550c0 msgr2=0x7f6760143cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.838+0000 7f6764df7700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67601550c0 0x7f6760143cc0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f675800b610 tx=0x7f6758009840).stop 2024-01-24T14:13:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.838+0000 7f6764df7700 1 -- 172.21.15.29:0/1453227194 shutdown_connections 2024-01-24T14:13:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.838+0000 7f6764df7700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f674005f080 0x7f6740061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.838+0000 7f6764df7700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67601550c0 0x7f6760143cc0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.838+0000 7f6764df7700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f676014b330 0x7f67600aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.838+0000 7f6764df7700 1 --2- 172.21.15.29:0/1453227194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f676014b710 0x7f67600ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.838+0000 7f6764df7700 1 -- 172.21.15.29:0/1453227194 >> 172.21.15.29:0/1453227194 conn(0x7f6760064390 msgr2=0x7f6760145b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.838+0000 7f6764df7700 1 -- 172.21.15.29:0/1453227194 shutdown_connections 2024-01-24T14:13:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:41.838+0000 7f6764df7700 1 -- 172.21.15.29:0/1453227194 wait complete. 2024-01-24T14:13:41.853 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T14:13:41.853 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:13:41.896 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:13:41.944 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T14:13:41.944 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-24T14:13:41.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:13:41.945 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-24T14:13:41.984 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:41.983+0000 7f71eab8f680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:13:41.987 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T14:13:41.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:13:41.987 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-24T14:13:41.994 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:41.994+0000 7f4e80e9c680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:13:42.013 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:13:42.048 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:42.047+0000 7f71eab8f680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:13:42.067 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:42.067+0000 7f4e80e9c680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:13:42.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.184+0000 7fce80560700 1 Processor -- start 2024-01-24T14:13:42.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.184+0000 7fce80560700 1 -- start start 2024-01-24T14:13:42.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.185+0000 7fce80560700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fce78154370 0x7fce780c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce80560700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce780aa6c0 con 0x7fce78154750 2024-01-24T14:13:42.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce80560700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce78064240 con 0x7fce780aa0e0 2024-01-24T14:13:42.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce80560700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce78154ba0 con 0x7fce78154370 2024-01-24T14:13:42.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce7eafd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fce78154750 0x7fce780631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38716/0 (socket says 172.21.15.29:38716) 2024-01-24T14:13:42.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce7eafd700 1 -- 172.21.15.29:0/4171962390 learned_addr learned my addr 172.21.15.29:0/4171962390 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:42.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce7e2fc700 1 --2- 172.21.15.29:0/4171962390 >> v2:172.21.15.163:3300/0 conn(0x7fce78154370 0x7fce780c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce7d2fa700 1 -- 172.21.15.29:0/4171962390 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798688750 0 0) 0x7fce780aa6c0 con 0x7fce78154750 2024-01-24T14:13:42.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce7d2fa700 1 -- 172.21.15.29:0/4171962390 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce640033f0 con 0x7fce78154750 2024-01-24T14:13:42.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce7d2fa700 1 -- 172.21.15.29:0/4171962390 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645639423 0 0) 0x7fce78064240 con 0x7fce780aa0e0 2024-01-24T14:13:42.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce7d2fa700 1 -- 172.21.15.29:0/4171962390 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce780aa6c0 con 0x7fce780aa0e0 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce7e2fc700 1 -- 172.21.15.29:0/4171962390 >> v1:172.21.15.163:6789/0 conn(0x7fce780aa0e0 legacy=0x7fce780ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce7e2fc700 1 -- 172.21.15.29:0/4171962390 >> v1:172.21.15.29:6789/0 conn(0x7fce78154750 legacy=0x7fce780631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce7e2fc700 1 -- 172.21.15.29:0/4171962390 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce780cb5c0 con 0x7fce78154370 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.186+0000 7fce7e2fc700 1 --2- 172.21.15.29:0/4171962390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce78154370 0x7fce780c5410 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fce74006fd0 tx=0x7fce74006f40).ready entity=mon.1 client_cookie=9cbfc307f03de4a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.187+0000 7fce7d2fa700 1 -- 172.21.15.29:0/4171962390 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74005880 con 0x7fce78154370 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.187+0000 7fce7d2fa700 1 -- 172.21.15.29:0/4171962390 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce740059e0 con 0x7fce78154370 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.187+0000 7fce7d2fa700 1 -- 172.21.15.29:0/4171962390 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74005d60 con 0x7fce78154370 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.187+0000 7fce80560700 1 -- 172.21.15.29:0/4171962390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce78154370 msgr2=0x7fce780c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.187+0000 7fce80560700 1 --2- 172.21.15.29:0/4171962390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce78154370 0x7fce780c5410 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fce74006fd0 tx=0x7fce74006f40).stop 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.187+0000 7fce80560700 1 -- 172.21.15.29:0/4171962390 shutdown_connections 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.187+0000 7fce80560700 1 --2- 172.21.15.29:0/4171962390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce78154370 0x7fce780c5410 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.187+0000 7fce80560700 1 -- 172.21.15.29:0/4171962390 >> 172.21.15.29:0/4171962390 conn(0x7fce7805ec80 msgr2=0x7fce780cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.187+0000 7fce80560700 1 -- 172.21.15.29:0/4171962390 shutdown_connections 2024-01-24T14:13:42.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.187+0000 7fce80560700 1 -- 172.21.15.29:0/4171962390 wait complete. 2024-01-24T14:13:42.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.189+0000 7fce80560700 1 Processor -- start 2024-01-24T14:13:42.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.189+0000 7fce80560700 1 -- start start 2024-01-24T14:13:42.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.189+0000 7fce80560700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce780aa0e0 0x7fce78146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.189+0000 7fce80560700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce78154370 0x7fce78159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.189+0000 7fce80560700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce78154750 0x7fce780c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.189+0000 7fce80560700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce78159b30 con 0x7fce78154370 2024-01-24T14:13:42.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.189+0000 7fce80560700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce78159200 con 0x7fce780aa0e0 2024-01-24T14:13:42.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.189+0000 7fce80560700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce781594e0 con 0x7fce78154750 2024-01-24T14:13:42.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce7dafb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce78154370 0x7fce78159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce7dafb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce78154370 0x7fce78159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37708/0 (socket says 172.21.15.29:37708) 2024-01-24T14:13:42.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce7e2fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce780aa0e0 0x7fce78146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce7dafb700 1 -- 172.21.15.29:0/1320047253 learned_addr learned my addr 172.21.15.29:0/1320047253 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce7eafd700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce78154750 0x7fce780c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce7dafb700 1 -- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce78154750 msgr2=0x7fce780c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce7dafb700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce78154750 0x7fce780c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce7dafb700 1 -- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce780aa0e0 msgr2=0x7fce78146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce7dafb700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce780aa0e0 0x7fce78146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce7dafb700 1 -- 172.21.15.29:0/1320047253 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74009250 con 0x7fce78154370 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce7e2fc700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce780aa0e0 0x7fce78146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce7dafb700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce78154370 0x7fce78159fc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fce6800c490 tx=0x7fce68007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce6e7fc700 1 -- 172.21.15.29:0/1320047253 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800ce40 con 0x7fce78154370 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce6e7fc700 1 -- 172.21.15.29:0/1320047253 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6800f470 con 0x7fce78154370 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce80560700 1 -- 172.21.15.29:0/1320047253 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce781597c0 con 0x7fce78154370 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.190+0000 7fce6e7fc700 1 -- 172.21.15.29:0/1320047253 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800f810 con 0x7fce78154370 2024-01-24T14:13:42.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.191+0000 7fce80560700 1 -- 172.21.15.29:0/1320047253 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce780c2c10 con 0x7fce78154370 2024-01-24T14:13:42.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.192+0000 7fce80560700 1 -- 172.21.15.29:0/1320047253 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce780475e0 con 0x7fce78154370 2024-01-24T14:13:42.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.192+0000 7fce6e7fc700 1 -- 172.21.15.29:0/1320047253 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fce6800fbc0 con 0x7fce78154370 2024-01-24T14:13:42.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.192+0000 7fce6e7fc700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fce5c05f300 0x7fce5c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.193+0000 7fce6e7fc700 1 -- 172.21.15.29:0/1320047253 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fce68079e50 con 0x7fce78154370 2024-01-24T14:13:42.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.193+0000 7fce7e2fc700 1 -- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fce5c05f300 msgr2=0x7fce5c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:42.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.193+0000 7fce7e2fc700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fce5c05f300 0x7fce5c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:42.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.198+0000 7fce6e7fc700 1 -- 172.21.15.29:0/1320047253 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fce6804aa20 con 0x7fce78154370 2024-01-24T14:13:42.324 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:42.323+0000 7f71eab8f680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:13:42.391 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:42.390+0000 7f4e80e9c680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:13:42.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.394+0000 7fce7e2fc700 1 -- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fce5c05f300 msgr2=0x7fce5c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:42.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.394+0000 7fce7e2fc700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fce5c05f300 0x7fce5c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:13:42.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.429+0000 7fce80560700 1 -- 172.21.15.29:0/1320047253 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce780cb8a0 con 0x7fce78154370 2024-01-24T14:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.430+0000 7fce6e7fc700 1 -- 172.21.15.29:0/1320047253 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fce6804e040 con 0x7fce78154370 2024-01-24T14:13:42.430 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:42.430 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1149323432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1149323432},{"type":"v1","addr":"172.21.15.163:6835","nonce":1149323432}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6157,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2941255566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":2941255566},{"type":"v1","addr":"172.21.15.29:6835","nonce":2941255566}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6163,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2235282008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2235282008},{"type":"v1","addr":"172.21.15.163:6837","nonce":2235282008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:13:27.270903+0000","modified":"2024-01-24T14:13:30.278839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6366":{"gid":6366,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":4,"addr":"172.21.15.29:6833/3957899438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3957899438},{"type":"v1","addr":"172.21.15.29:6833","nonce":3957899438}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-24T14:13:42.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.432+0000 7fce80560700 1 -- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fce5c05f300 msgr2=0x7fce5c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:42.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.432+0000 7fce80560700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fce5c05f300 0x7fce5c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:42.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.432+0000 7fce80560700 1 -- 172.21.15.29:0/1320047253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce78154370 msgr2=0x7fce78159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.432+0000 7fce80560700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce78154370 0x7fce78159fc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fce6800c490 tx=0x7fce68007c90).stop 2024-01-24T14:13:42.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.432+0000 7fce80560700 1 -- 172.21.15.29:0/1320047253 shutdown_connections 2024-01-24T14:13:42.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.432+0000 7fce80560700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce78154750 0x7fce780c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.432+0000 7fce80560700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce780aa0e0 0x7fce78146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.432+0000 7fce80560700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fce5c05f300 0x7fce5c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:42.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.432+0000 7fce80560700 1 --2- 172.21.15.29:0/1320047253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce78154370 0x7fce78159fc0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.432+0000 7fce80560700 1 -- 172.21.15.29:0/1320047253 >> 172.21.15.29:0/1320047253 conn(0x7fce7805ec80 msgr2=0x7fce781535f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:42.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.432+0000 7fce80560700 1 -- 172.21.15.29:0/1320047253 shutdown_connections 2024-01-24T14:13:42.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.432+0000 7fce80560700 1 -- 172.21.15.29:0/1320047253 wait complete. 2024-01-24T14:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 35 2024-01-24T14:13:42.446 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:13:42.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.618+0000 7ff28c6fa700 1 Processor -- start 2024-01-24T14:13:42.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.618+0000 7ff28c6fa700 1 -- start start 2024-01-24T14:13:42.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.619+0000 7ff28c6fa700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2840c5410 0x7ff2840aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.619+0000 7ff28c6fa700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2840aa6c0 con 0x7ff284154370 2024-01-24T14:13:42.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.619+0000 7ff28c6fa700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff284064240 con 0x7ff284154750 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.619+0000 7ff28c6fa700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff284154ba0 con 0x7ff2840c5410 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.619+0000 7ff28a496700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff284154370 0x7ff2840ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38732/0 (socket says 172.21.15.29:38732) 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.619+0000 7ff28a496700 1 -- 172.21.15.29:0/252717957 learned_addr learned my addr 172.21.15.29:0/252717957 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.619+0000 7ff289c95700 1 --2- 172.21.15.29:0/252717957 >> v2:172.21.15.163:3300/0 conn(0x7ff2840c5410 0x7ff2840aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.620+0000 7ff289494700 1 -- 172.21.15.29:0/252717957 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447684411 0 0) 0x7ff2840aa6c0 con 0x7ff284154370 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.620+0000 7ff289494700 1 -- 172.21.15.29:0/252717957 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2700033f0 con 0x7ff284154370 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.620+0000 7ff289494700 1 -- 172.21.15.29:0/252717957 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194502174 0 0) 0x7ff284064240 con 0x7ff284154750 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.620+0000 7ff289494700 1 -- 172.21.15.29:0/252717957 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2840aa6c0 con 0x7ff284154750 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.620+0000 7ff289c95700 1 -- 172.21.15.29:0/252717957 >> v1:172.21.15.163:6789/0 conn(0x7ff284154750 legacy=0x7ff2840631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.620+0000 7ff289c95700 1 -- 172.21.15.29:0/252717957 >> v1:172.21.15.29:6789/0 conn(0x7ff284154370 legacy=0x7ff2840ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.620+0000 7ff289c95700 1 -- 172.21.15.29:0/252717957 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2840cb5c0 con 0x7ff2840c5410 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.620+0000 7ff289c95700 1 --2- 172.21.15.29:0/252717957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2840c5410 0x7ff2840aa0e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7ff274009760 tx=0x7ff274009060).ready entity=mon.1 client_cookie=b0fe3c32c24e427c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.620+0000 7ff289494700 1 -- 172.21.15.29:0/252717957 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400d4a0 con 0x7ff2840c5410 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.620+0000 7ff289494700 1 -- 172.21.15.29:0/252717957 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27400d600 con 0x7ff2840c5410 2024-01-24T14:13:42.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.620+0000 7ff289494700 1 -- 172.21.15.29:0/252717957 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400d980 con 0x7ff2840c5410 2024-01-24T14:13:42.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.621+0000 7ff28c6fa700 1 -- 172.21.15.29:0/252717957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2840c5410 msgr2=0x7ff2840aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.621+0000 7ff28c6fa700 1 --2- 172.21.15.29:0/252717957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2840c5410 0x7ff2840aa0e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7ff274009760 tx=0x7ff274009060).stop 2024-01-24T14:13:42.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.621+0000 7ff28c6fa700 1 -- 172.21.15.29:0/252717957 shutdown_connections 2024-01-24T14:13:42.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.621+0000 7ff28c6fa700 1 --2- 172.21.15.29:0/252717957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2840c5410 0x7ff2840aa0e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.621+0000 7ff28c6fa700 1 -- 172.21.15.29:0/252717957 >> 172.21.15.29:0/252717957 conn(0x7ff28405ec80 msgr2=0x7ff2840cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:42.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.621+0000 7ff28c6fa700 1 -- 172.21.15.29:0/252717957 shutdown_connections 2024-01-24T14:13:42.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.621+0000 7ff28c6fa700 1 -- 172.21.15.29:0/252717957 wait complete. 2024-01-24T14:13:42.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.622+0000 7ff28c6fa700 1 Processor -- start 2024-01-24T14:13:42.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.622+0000 7ff28c6fa700 1 -- start start 2024-01-24T14:13:42.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.623+0000 7ff28c6fa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2840c5410 0x7ff28414dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.623+0000 7ff28c6fa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff284154370 0x7ff2841508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.623+0000 7ff28c6fa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff284154750 0x7ff2840c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.623+0000 7ff28c6fa700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2841587a0 con 0x7ff2840c5410 2024-01-24T14:13:42.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.623+0000 7ff28c6fa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2841510c0 con 0x7ff284154370 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.623+0000 7ff28c6fa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2841513a0 con 0x7ff284154750 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.623+0000 7ff28a496700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2840c5410 0x7ff28414dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.623+0000 7ff28a496700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2840c5410 0x7ff28414dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37712/0 (socket says 172.21.15.29:37712) 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.623+0000 7ff28a496700 1 -- 172.21.15.29:0/1219449098 learned_addr learned my addr 172.21.15.29:0/1219449098 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.623+0000 7ff289c95700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff284154370 0x7ff2841508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.623+0000 7ff28ac97700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff284154750 0x7ff2840c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.624+0000 7ff28a496700 1 -- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff284154750 msgr2=0x7ff2840c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.624+0000 7ff28a496700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff284154750 0x7ff2840c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.624+0000 7ff28a496700 1 -- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff284154370 msgr2=0x7ff2841508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.624+0000 7ff28a496700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff284154370 0x7ff2841508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.624+0000 7ff28a496700 1 -- 172.21.15.29:0/1219449098 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2740092a0 con 0x7ff2840c5410 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.624+0000 7ff28a496700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2840c5410 0x7ff28414dbe0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7ff28000c790 tx=0x7ff280007b60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.624+0000 7ff27a7fc700 1 -- 172.21.15.29:0/1219449098 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2800072e0 con 0x7ff2840c5410 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.624+0000 7ff27a7fc700 1 -- 172.21.15.29:0/1219449098 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff280007440 con 0x7ff2840c5410 2024-01-24T14:13:42.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.624+0000 7ff28c6fa700 1 -- 172.21.15.29:0/1219449098 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff284158150 con 0x7ff2840c5410 2024-01-24T14:13:42.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.624+0000 7ff27a7fc700 1 -- 172.21.15.29:0/1219449098 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280005880 con 0x7ff2840c5410 2024-01-24T14:13:42.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.624+0000 7ff28c6fa700 1 -- 172.21.15.29:0/1219449098 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff284150dd0 con 0x7ff2840c5410 2024-01-24T14:13:42.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.625+0000 7ff28c6fa700 1 -- 172.21.15.29:0/1219449098 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff284157f90 con 0x7ff2840c5410 2024-01-24T14:13:42.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.625+0000 7ff27a7fc700 1 -- 172.21.15.29:0/1219449098 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7ff280005c70 con 0x7ff2840c5410 2024-01-24T14:13:42.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.626+0000 7ff27a7fc700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff26805f300 0x7ff2680617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.626+0000 7ff289c95700 1 -- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff26805f300 msgr2=0x7ff2680617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:42.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.626+0000 7ff27a7fc700 1 -- 172.21.15.29:0/1219449098 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7400+0+0 (secure 0 0 0) 0x7ff28007a1a0 con 0x7ff2840c5410 2024-01-24T14:13:42.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.626+0000 7ff289c95700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff26805f300 0x7ff2680617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:42.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.630+0000 7ff27a7fc700 1 -- 172.21.15.29:0/1219449098 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff28004ad70 con 0x7ff2840c5410 2024-01-24T14:13:42.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.816+0000 7ff28c6fa700 1 -- 172.21.15.29:0/1219449098 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2840475e0 con 0x7ff2840c5410 2024-01-24T14:13:42.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.817+0000 7ff27a7fc700 1 -- 172.21.15.29:0/1219449098 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+18546 (secure 0 0 0) 0x7ff280018070 con 0x7ff2840c5410 2024-01-24T14:13:42.817 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:42.818 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":91,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:13:39.554230+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:13:25.288458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-24T14:13:26.243223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6833/43241":"2024-01-25T14:13:39.554196+0000","172.21.15.163:6832/43241":"2024-01-25T14:13:39.554196+0000","172.21.15.163:0/2629162943":"2024-01-25T14:13:39.554196+0000","172.21.15.163:0/1057348791":"2024-01-25T14:13:39.554196+0000","172.21.15.163:0/1616535923":"2024-01-25T14:13:39.554196+0000","172.21.15.163:0/1701533468":"2024-01-25T14:13:39.554196+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:13:42.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.819+0000 7ff28c6fa700 1 -- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff26805f300 msgr2=0x7ff2680617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:42.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.819+0000 7ff28c6fa700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff26805f300 0x7ff2680617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:42.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.819+0000 7ff28c6fa700 1 -- 172.21.15.29:0/1219449098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2840c5410 msgr2=0x7ff28414dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.819+0000 7ff28c6fa700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2840c5410 0x7ff28414dbe0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7ff28000c790 tx=0x7ff280007b60).stop 2024-01-24T14:13:42.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.819+0000 7ff28c6fa700 1 -- 172.21.15.29:0/1219449098 shutdown_connections 2024-01-24T14:13:42.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.819+0000 7ff28c6fa700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff284154750 0x7ff2840c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.819+0000 7ff28c6fa700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff26805f300 0x7ff2680617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:42.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.819+0000 7ff28c6fa700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2840c5410 0x7ff28414dbe0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.819+0000 7ff28c6fa700 1 --2- 172.21.15.29:0/1219449098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff284154370 0x7ff2841508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.819+0000 7ff28c6fa700 1 -- 172.21.15.29:0/1219449098 >> 172.21.15.29:0/1219449098 conn(0x7ff28405ec80 msgr2=0x7ff2840ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:42.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.819+0000 7ff28c6fa700 1 -- 172.21.15.29:0/1219449098 shutdown_connections 2024-01-24T14:13:42.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.819+0000 7ff28c6fa700 1 -- 172.21.15.29:0/1219449098 wait complete. 2024-01-24T14:13:42.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T14:13:25.288458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T14:13:26.243223+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T14:13:42.834 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T14:13:42.943 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:42.942+0000 7f71eab8f680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:13:42.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.971+0000 7fb2eaba4700 1 Processor -- start 2024-01-24T14:13:42.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.971+0000 7fb2eaba4700 1 -- start start 2024-01-24T14:13:42.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.972+0000 7fb2eaba4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb2e413a5e0 0x7fb2e40639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.972+0000 7fb2eaba4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb2e4066bd0 0x7fb2e40bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.972+0000 7fb2eaba4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e40679f0 con 0x7fb2e4063eb0 2024-01-24T14:13:42.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.972+0000 7fb2eaba4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e4067cd0 con 0x7fb2e4066bd0 2024-01-24T14:13:42.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.972+0000 7fb2eaba4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e4067fb0 con 0x7fb2e413a5e0 2024-01-24T14:13:42.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e9141700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb2e4066bd0 0x7fb2e40bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e9141700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb2e4066bd0 0x7fb2e40bdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37716/0 (socket says 172.21.15.29:37716) 2024-01-24T14:13:42.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e8940700 1 --2- 172.21.15.29:0/761325781 >> v2:172.21.15.163:3300/0 conn(0x7fb2e413a5e0 0x7fb2e40639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e9141700 1 -- 172.21.15.29:0/761325781 learned_addr learned my addr 172.21.15.29:0/761325781 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:42.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e9141700 1 -- 172.21.15.29:0/761325781 >> v2:172.21.15.163:3300/0 conn(0x7fb2e413a5e0 msgr2=0x7fb2e40639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:42.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e9141700 1 --2- 172.21.15.29:0/761325781 >> v2:172.21.15.163:3300/0 conn(0x7fb2e413a5e0 0x7fb2e40639c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e37fe700 1 -- 172.21.15.29:0/761325781 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417261479 0 0) 0x7fb2e40679f0 con 0x7fb2e4063eb0 2024-01-24T14:13:42.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e9141700 1 -- 172.21.15.29:0/761325781 >> v1:172.21.15.163:6789/0 conn(0x7fb2e4063eb0 legacy=0x7fb2e40664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e9141700 1 -- 172.21.15.29:0/761325781 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e4068290 con 0x7fb2e4066bd0 2024-01-24T14:13:42.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e9141700 1 --2- 172.21.15.29:0/761325781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e4066bd0 0x7fb2e40bdb30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fb2e4069ba0 tx=0x7fb2dc009840).ready entity=mon.0 client_cookie=72b1af4a941188ed server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:42.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e37fe700 1 -- 172.21.15.29:0/761325781 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00e070 con 0x7fb2e4066bd0 2024-01-24T14:13:42.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e37fe700 1 -- 172.21.15.29:0/761325781 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc00ba20 con 0x7fb2e4066bd0 2024-01-24T14:13:42.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.973+0000 7fb2e37fe700 1 -- 172.21.15.29:0/761325781 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00bda0 con 0x7fb2e4066bd0 2024-01-24T14:13:42.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.974+0000 7fb2eaba4700 1 -- 172.21.15.29:0/761325781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e4066bd0 msgr2=0x7fb2e40bdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.974+0000 7fb2eaba4700 1 --2- 172.21.15.29:0/761325781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e4066bd0 0x7fb2e40bdb30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fb2e4069ba0 tx=0x7fb2dc009840).stop 2024-01-24T14:13:42.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.974+0000 7fb2eaba4700 1 -- 172.21.15.29:0/761325781 shutdown_connections 2024-01-24T14:13:42.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.974+0000 7fb2eaba4700 1 --2- 172.21.15.29:0/761325781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e4066bd0 0x7fb2e40bdb30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.974+0000 7fb2eaba4700 1 --2- 172.21.15.29:0/761325781 >> v2:172.21.15.163:3300/0 conn(0x7fb2e413a5e0 0x7fb2e40639c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.974+0000 7fb2eaba4700 1 -- 172.21.15.29:0/761325781 >> 172.21.15.29:0/761325781 conn(0x7fb2e40c8bd0 msgr2=0x7fb2e40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:42.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.974+0000 7fb2eaba4700 1 -- 172.21.15.29:0/761325781 shutdown_connections 2024-01-24T14:13:42.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.974+0000 7fb2eaba4700 1 -- 172.21.15.29:0/761325781 wait complete. 2024-01-24T14:13:42.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.975+0000 7fb2eaba4700 1 Processor -- start 2024-01-24T14:13:42.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.976+0000 7fb2eaba4700 1 -- start start 2024-01-24T14:13:42.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.976+0000 7fb2eaba4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2e4063eb0 0x7fb2e40ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.976+0000 7fb2eaba4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e4066bd0 0x7fb2e413b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.976+0000 7fb2eaba4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2e413a5e0 0x7fb2e4139470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.976+0000 7fb2eaba4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e413bc40 con 0x7fb2e4066bd0 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.976+0000 7fb2eaba4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e41586c0 con 0x7fb2e413a5e0 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.976+0000 7fb2eaba4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e413bed0 con 0x7fb2e4063eb0 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e4066bd0 0x7fb2e413b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e4066bd0 0x7fb2e413b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37726/0 (socket says 172.21.15.29:37726) 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e3fff700 1 -- 172.21.15.29:0/403851624 learned_addr learned my addr 172.21.15.29:0/403851624 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e8940700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2e4063eb0 0x7fb2e40ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e9141700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2e413a5e0 0x7fb2e4139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e3fff700 1 -- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2e4063eb0 msgr2=0x7fb2e40ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e3fff700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2e4063eb0 0x7fb2e40ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e3fff700 1 -- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2e413a5e0 msgr2=0x7fb2e4139470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e3fff700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2e413a5e0 0x7fb2e4139470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e3fff700 1 -- 172.21.15.29:0/403851624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc009ae0 con 0x7fb2e4066bd0 2024-01-24T14:13:42.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e3fff700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e4066bd0 0x7fb2e413b750 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fb2d0001eb0 tx=0x7fb2d0007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:42.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e0ff9700 1 -- 172.21.15.29:0/403851624 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000ce60 con 0x7fb2e4066bd0 2024-01-24T14:13:42.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e0ff9700 1 -- 172.21.15.29:0/403851624 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d000f470 con 0x7fb2e4066bd0 2024-01-24T14:13:42.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2eaba4700 1 -- 172.21.15.29:0/403851624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e413c1b0 con 0x7fb2e4066bd0 2024-01-24T14:13:42.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2e0ff9700 1 -- 172.21.15.29:0/403851624 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000f810 con 0x7fb2e4066bd0 2024-01-24T14:13:42.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.977+0000 7fb2eaba4700 1 -- 172.21.15.29:0/403851624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e413c770 con 0x7fb2e4066bd0 2024-01-24T14:13:42.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.979+0000 7fb2eaba4700 1 -- 172.21.15.29:0/403851624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e40475e0 con 0x7fb2e4066bd0 2024-01-24T14:13:42.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.979+0000 7fb2e0ff9700 1 -- 172.21.15.29:0/403851624 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fb2d000fb80 con 0x7fb2e4066bd0 2024-01-24T14:13:42.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.979+0000 7fb2e0ff9700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2c805f300 0x7fb2c80617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:42.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.979+0000 7fb2e0ff9700 1 -- 172.21.15.29:0/403851624 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fb2d0079ea0 con 0x7fb2e4066bd0 2024-01-24T14:13:42.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.979+0000 7fb2e8940700 1 -- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2c805f300 msgr2=0x7fb2c80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:42.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.979+0000 7fb2e8940700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2c805f300 0x7fb2c80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:42.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:42.984+0000 7fb2e0ff9700 1 -- 172.21.15.29:0/403851624 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb2d004aa70 con 0x7fb2e4066bd0 2024-01-24T14:13:43.016 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:43.015+0000 7f71eab8f680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:13:43.134 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:43.134+0000 7f4e80e9c680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:13:43.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.180+0000 7fb2e8940700 1 -- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2c805f300 msgr2=0x7fb2c80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:43.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.180+0000 7fb2e8940700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2c805f300 0x7fb2c80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:13:43.203 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:43.203+0000 7f71eab8f680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:13:43.210 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:43.209+0000 7f4e80e9c680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:13:43.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.210+0000 7fb2eaba4700 1 -- 172.21.15.29:0/403851624 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb2e4068570 con 0x7fb2e4066bd0 2024-01-24T14:13:43.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.211+0000 7fb2e0ff9700 1 -- 172.21.15.29:0/403851624 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v91) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb2d004e090 con 0x7fb2e4066bd0 2024-01-24T14:13:43.211 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T14:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.213+0000 7fb2eaba4700 1 -- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2c805f300 msgr2=0x7fb2c80617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.213+0000 7fb2eaba4700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2c805f300 0x7fb2c80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.213+0000 7fb2eaba4700 1 -- 172.21.15.29:0/403851624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e4066bd0 msgr2=0x7fb2e413b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.213+0000 7fb2eaba4700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e4066bd0 0x7fb2e413b750 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fb2d0001eb0 tx=0x7fb2d0007c70).stop 2024-01-24T14:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.213+0000 7fb2eaba4700 1 -- 172.21.15.29:0/403851624 shutdown_connections 2024-01-24T14:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.213+0000 7fb2eaba4700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2e413a5e0 0x7fb2e4139470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.213+0000 7fb2eaba4700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2e4063eb0 0x7fb2e40ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.213+0000 7fb2eaba4700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2c805f300 0x7fb2c80617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.213+0000 7fb2eaba4700 1 --2- 172.21.15.29:0/403851624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e4066bd0 0x7fb2e413b750 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.213+0000 7fb2eaba4700 1 -- 172.21.15.29:0/403851624 >> 172.21.15.29:0/403851624 conn(0x7fb2e40c8bd0 msgr2=0x7fb2e40c5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.213+0000 7fb2eaba4700 1 -- 172.21.15.29:0/403851624 shutdown_connections 2024-01-24T14:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.213+0000 7fb2eaba4700 1 -- 172.21.15.29:0/403851624 wait complete. 2024-01-24T14:13:43.228 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T14:13:43.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.367+0000 7f6db7ad3700 1 Processor -- start 2024-01-24T14:13:43.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.367+0000 7f6db7ad3700 1 -- start start 2024-01-24T14:13:43.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.368+0000 7f6db7ad3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6db00664c0 0x7f6db0066950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:43.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.368+0000 7f6db7ad3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db00bdb30 0x7f6db00bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:43.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.368+0000 7f6db7ad3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db0066e40 con 0x7f6db013a5e0 2024-01-24T14:13:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.368+0000 7f6db7ad3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db00640d0 con 0x7f6db00664c0 2024-01-24T14:13:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.368+0000 7f6db7ad3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db00679f0 con 0x7f6db00bdb30 2024-01-24T14:13:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.368+0000 7f6db506e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6db00664c0 0x7f6db0066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.368+0000 7f6db6070700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db00bdb30 0x7f6db00bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.368+0000 7f6db586f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6db013a5e0 0x7f6db00639c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:48338/0 (socket says 172.21.15.29:48338) 2024-01-24T14:13:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.368+0000 7f6db586f700 1 -- 172.21.15.29:0/2763336547 learned_addr learned my addr 172.21.15.29:0/2763336547 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.369+0000 7f6db506e700 1 -- 172.21.15.29:0/2763336547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db00bdb30 msgr2=0x7f6db00bdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.369+0000 7f6db506e700 1 --2- 172.21.15.29:0/2763336547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db00bdb30 0x7f6db00bdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.369+0000 7f6db506e700 1 -- 172.21.15.29:0/2763336547 >> v1:172.21.15.163:6789/0 conn(0x7f6db013a5e0 legacy=0x7f6db00639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.369+0000 7f6db506e700 1 -- 172.21.15.29:0/2763336547 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0067cd0 con 0x7f6db00664c0 2024-01-24T14:13:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.369+0000 7f6db486d700 1 -- 172.21.15.29:0/2763336547 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973354278 0 0) 0x7f6db0066e40 con 0x7f6db013a5e0 2024-01-24T14:13:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.369+0000 7f6db506e700 1 --2- 172.21.15.29:0/2763336547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db00664c0 0x7f6db0066950 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f6da0009760 tx=0x7f6da0009060).ready entity=mon.1 client_cookie=fe7b30429952138a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.369+0000 7f6db486d700 1 -- 172.21.15.29:0/2763336547 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da000f510 con 0x7f6db00664c0 2024-01-24T14:13:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.369+0000 7f6db486d700 1 -- 172.21.15.29:0/2763336547 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da000f670 con 0x7f6db00664c0 2024-01-24T14:13:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.369+0000 7f6db486d700 1 -- 172.21.15.29:0/2763336547 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da000f9f0 con 0x7f6db00664c0 2024-01-24T14:13:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.370+0000 7f6db7ad3700 1 -- 172.21.15.29:0/2763336547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db00664c0 msgr2=0x7f6db0066950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.370+0000 7f6db7ad3700 1 --2- 172.21.15.29:0/2763336547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db00664c0 0x7f6db0066950 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f6da0009760 tx=0x7f6da0009060).stop 2024-01-24T14:13:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.370+0000 7f6db7ad3700 1 -- 172.21.15.29:0/2763336547 shutdown_connections 2024-01-24T14:13:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.370+0000 7f6db7ad3700 1 --2- 172.21.15.29:0/2763336547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db00bdb30 0x7f6db00bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.370+0000 7f6db7ad3700 1 --2- 172.21.15.29:0/2763336547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db00664c0 0x7f6db0066950 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.370+0000 7f6db7ad3700 1 -- 172.21.15.29:0/2763336547 >> 172.21.15.29:0/2763336547 conn(0x7f6db00c8bd0 msgr2=0x7f6db00c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.370+0000 7f6db7ad3700 1 -- 172.21.15.29:0/2763336547 shutdown_connections 2024-01-24T14:13:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.370+0000 7f6db7ad3700 1 -- 172.21.15.29:0/2763336547 wait complete. 2024-01-24T14:13:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.371+0000 7f6db7ad3700 1 Processor -- start 2024-01-24T14:13:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.371+0000 7f6db7ad3700 1 -- start start 2024-01-24T14:13:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db7ad3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db00664c0 0x7f6db0138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db7ad3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db00bdb30 0x7f6db00a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db7ad3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db013a5e0 0x7f6db00a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db7ad3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db00a7fa0 con 0x7f6db013a5e0 2024-01-24T14:13:43.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db7ad3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db00a53c0 con 0x7f6db00664c0 2024-01-24T14:13:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db7ad3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db00a4dd0 con 0x7f6db00bdb30 2024-01-24T14:13:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db6070700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db013a5e0 0x7f6db00a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db586f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db00664c0 0x7f6db0138630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db506e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db00bdb30 0x7f6db00a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db6070700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db013a5e0 0x7f6db00a7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37736/0 (socket says 172.21.15.29:37736) 2024-01-24T14:13:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db6070700 1 -- 172.21.15.29:0/91935080 learned_addr learned my addr 172.21.15.29:0/91935080 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db506e700 1 -- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db00664c0 msgr2=0x7f6db0138630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db506e700 1 --2- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db00664c0 0x7f6db0138630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db506e700 1 -- 172.21.15.29:0/91935080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db013a5e0 msgr2=0x7f6db00a7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:43.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db506e700 1 --2- 172.21.15.29:0/91935080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db013a5e0 0x7f6db00a7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db506e700 1 -- 172.21.15.29:0/91935080 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da00092a0 con 0x7f6db00bdb30 2024-01-24T14:13:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.372+0000 7f6db6070700 1 --2- 172.21.15.29:0/91935080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db013a5e0 0x7f6db00a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.373+0000 7f6db506e700 1 --2- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db00bdb30 0x7f6db00a75c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f6db0069ba0 tx=0x7f6da0017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.373+0000 7f6da5ffb700 1 -- 172.21.15.29:0/91935080 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da000fe80 con 0x7f6db00bdb30 2024-01-24T14:13:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.373+0000 7f6da5ffb700 1 -- 172.21.15.29:0/91935080 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da000f7e0 con 0x7f6db00bdb30 2024-01-24T14:13:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.373+0000 7f6db7ad3700 1 -- 172.21.15.29:0/91935080 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db013c410 con 0x7f6db00bdb30 2024-01-24T14:13:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.373+0000 7f6da5ffb700 1 -- 172.21.15.29:0/91935080 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da001b670 con 0x7f6db00bdb30 2024-01-24T14:13:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.373+0000 7f6db7ad3700 1 -- 172.21.15.29:0/91935080 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db00a5050 con 0x7f6db00bdb30 2024-01-24T14:13:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.374+0000 7f6db7ad3700 1 -- 172.21.15.29:0/91935080 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db013c930 con 0x7f6db00bdb30 2024-01-24T14:13:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.375+0000 7f6da5ffb700 1 -- 172.21.15.29:0/91935080 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f6da001ba60 con 0x7f6db00bdb30 2024-01-24T14:13:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.375+0000 7f6da5ffb700 1 --2- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6d9405f2d0 0x7f6d94061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.375+0000 7f6da5ffb700 1 -- 172.21.15.29:0/91935080 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f6da007bf20 con 0x7f6db00bdb30 2024-01-24T14:13:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.375+0000 7f6db586f700 1 -- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6d9405f2d0 msgr2=0x7f6d94061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.375+0000 7f6db586f700 1 --2- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6d9405f2d0 0x7f6d94061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.379+0000 7f6da5ffb700 1 -- 172.21.15.29:0/91935080 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f6da004caf0 con 0x7f6db00bdb30 2024-01-24T14:13:43.437 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:43.437+0000 7f4e80e9c680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:13:43.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.563+0000 7f6db7ad3700 1 -- 172.21.15.29:0/91935080 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6db00475e0 con 0x7f6db00bdb30 2024-01-24T14:13:43.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.563+0000 7f6da5ffb700 1 -- 172.21.15.29:0/91935080 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v91) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6da0050110 con 0x7f6db00bdb30 2024-01-24T14:13:43.564 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T14:13:43.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.565+0000 7f6db7ad3700 1 -- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6d9405f2d0 msgr2=0x7f6d94061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:43.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.565+0000 7f6db7ad3700 1 --2- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6d9405f2d0 0x7f6d94061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:43.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.565+0000 7f6db7ad3700 1 -- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db00bdb30 msgr2=0x7f6db00a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:43.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.565+0000 7f6db7ad3700 1 --2- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db00bdb30 0x7f6db00a75c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f6db0069ba0 tx=0x7f6da0017000).stop 2024-01-24T14:13:43.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.565+0000 7f6db7ad3700 1 -- 172.21.15.29:0/91935080 shutdown_connections 2024-01-24T14:13:43.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.565+0000 7f6db7ad3700 1 --2- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6d9405f2d0 0x7f6d94061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:43.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.565+0000 7f6db7ad3700 1 --2- 172.21.15.29:0/91935080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db013a5e0 0x7f6db00a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.565+0000 7f6db7ad3700 1 --2- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db00664c0 0x7f6db0138630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.565+0000 7f6db7ad3700 1 --2- 172.21.15.29:0/91935080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db00bdb30 0x7f6db00a75c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.565+0000 7f6db7ad3700 1 -- 172.21.15.29:0/91935080 >> 172.21.15.29:0/91935080 conn(0x7f6db00c8bd0 msgr2=0x7f6db00be640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:43.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.565+0000 7f6db7ad3700 1 -- 172.21.15.29:0/91935080 shutdown_connections 2024-01-24T14:13:43.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.566+0000 7f6db7ad3700 1 -- 172.21.15.29:0/91935080 wait complete. 2024-01-24T14:13:43.580 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T14:13:43.629 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:43.628+0000 7f71eab8f680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:13:43.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.716+0000 7f1314c50700 1 Processor -- start 2024-01-24T14:13:43.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.717+0000 7f1314c50700 1 -- start start 2024-01-24T14:13:43.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f1314c50700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1310064760 0x7f1310064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:43.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f1314c50700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1310152240 con 0x7f13100bfd40 2024-01-24T14:13:43.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f1314c50700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13101586c0 con 0x7f13100c8490 2024-01-24T14:13:43.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f1314c50700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13100aa670 con 0x7f1310064760 2024-01-24T14:13:43.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f130ed9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f13100bfd40 0x7f13100c01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46612/0 (socket says 172.21.15.29:46612) 2024-01-24T14:13:43.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f130ed9d700 1 -- 172.21.15.29:0/992556038 learned_addr learned my addr 172.21.15.29:0/992556038 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:43.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f130e59c700 1 --2- 172.21.15.29:0/992556038 >> v2:172.21.15.163:3300/0 conn(0x7f1310064760 0x7f1310064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:43.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f130d59a700 1 -- 172.21.15.29:0/992556038 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090115860 0 0) 0x7f1310152240 con 0x7f13100bfd40 2024-01-24T14:13:43.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f130d59a700 1 -- 172.21.15.29:0/992556038 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12f80033f0 con 0x7f13100bfd40 2024-01-24T14:13:43.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f130e59c700 1 -- 172.21.15.29:0/992556038 >> v1:172.21.15.163:6789/0 conn(0x7f13100c8490 legacy=0x7f13100a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:43.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f130e59c700 1 -- 172.21.15.29:0/992556038 >> v1:172.21.15.29:6789/0 conn(0x7f13100bfd40 legacy=0x7f13100c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:43.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f130e59c700 1 -- 172.21.15.29:0/992556038 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1310157d90 con 0x7f1310064760 2024-01-24T14:13:43.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.718+0000 7f130e59c700 1 --2- 172.21.15.29:0/992556038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310064760 0x7f1310064b40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f1300009760 tx=0x7f1300009060).ready entity=mon.1 client_cookie=9f67ba62a85ef8ac server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:43.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.719+0000 7f130d59a700 1 -- 172.21.15.29:0/992556038 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130000b590 con 0x7f1310064760 2024-01-24T14:13:43.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.719+0000 7f130d59a700 1 -- 172.21.15.29:0/992556038 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130000b6f0 con 0x7f1310064760 2024-01-24T14:13:43.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.719+0000 7f130d59a700 1 -- 172.21.15.29:0/992556038 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130000ba70 con 0x7f1310064760 2024-01-24T14:13:43.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.719+0000 7f1314c50700 1 -- 172.21.15.29:0/992556038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310064760 msgr2=0x7f1310064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:43.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.719+0000 7f1314c50700 1 --2- 172.21.15.29:0/992556038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310064760 0x7f1310064b40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f1300009760 tx=0x7f1300009060).stop 2024-01-24T14:13:43.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.719+0000 7f1314c50700 1 -- 172.21.15.29:0/992556038 shutdown_connections 2024-01-24T14:13:43.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.719+0000 7f1314c50700 1 --2- 172.21.15.29:0/992556038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310064760 0x7f1310064b40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.719+0000 7f1314c50700 1 -- 172.21.15.29:0/992556038 >> 172.21.15.29:0/992556038 conn(0x7f13100cb8b0 msgr2=0x7f13100c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:43.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.719+0000 7f1314c50700 1 -- 172.21.15.29:0/992556038 shutdown_connections 2024-01-24T14:13:43.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.719+0000 7f1314c50700 1 -- 172.21.15.29:0/992556038 wait complete. 2024-01-24T14:13:43.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.721+0000 7f1314c50700 1 Processor -- start 2024-01-24T14:13:43.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.721+0000 7f1314c50700 1 -- start start 2024-01-24T14:13:43.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f1314c50700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1310064760 0x7f1310141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:43.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f1314c50700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13100bfd40 0x7f13100c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:43.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f1314c50700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13100c8490 0x7f13100c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:43.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f1314c50700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13100c66f0 con 0x7f1310064760 2024-01-24T14:13:43.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f1314c50700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1310153e90 con 0x7f13100bfd40 2024-01-24T14:13:43.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f1314c50700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13100c3f20 con 0x7f13100c8490 2024-01-24T14:13:43.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f130e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1310064760 0x7f1310141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f130e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1310064760 0x7f1310141940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35126/0 (socket says 172.21.15.29:35126) 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f130e59c700 1 -- 172.21.15.29:0/3181686967 learned_addr learned my addr 172.21.15.29:0/3181686967 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f130dd9b700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13100bfd40 0x7f13100c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f130ed9d700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13100c8490 0x7f13100c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f130e59c700 1 -- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13100c8490 msgr2=0x7f13100c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f130e59c700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13100c8490 0x7f13100c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f130e59c700 1 -- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13100bfd40 msgr2=0x7f13100c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f130e59c700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13100bfd40 0x7f13100c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f130e59c700 1 -- 172.21.15.29:0/3181686967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13000092a0 con 0x7f1310064760 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f130ed9d700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13100c8490 0x7f13100c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f130e59c700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1310064760 0x7f1310141940 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f1300009e00 tx=0x7f1300017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f12fe7fc700 1 -- 172.21.15.29:0/3181686967 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130001a4d0 con 0x7f1310064760 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f12fe7fc700 1 -- 172.21.15.29:0/3181686967 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130001e070 con 0x7f1310064760 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f12fe7fc700 1 -- 172.21.15.29:0/3181686967 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130001a880 con 0x7f1310064760 2024-01-24T14:13:43.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.722+0000 7f1314c50700 1 -- 172.21.15.29:0/3181686967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1310153560 con 0x7f1310064760 2024-01-24T14:13:43.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.723+0000 7f1314c50700 1 -- 172.21.15.29:0/3181686967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1310153b20 con 0x7f1310064760 2024-01-24T14:13:43.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.724+0000 7f1314c50700 1 -- 172.21.15.29:0/3181686967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13100475e0 con 0x7f1310064760 2024-01-24T14:13:43.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.724+0000 7f12fe7fc700 1 -- 172.21.15.29:0/3181686967 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f1300020030 con 0x7f1310064760 2024-01-24T14:13:43.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.725+0000 7f12fe7fc700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f12f005f300 0x7f12f00617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:43.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.725+0000 7f130dd9b700 1 -- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f12f005f300 msgr2=0x7f12f00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:43.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.725+0000 7f12fe7fc700 1 -- 172.21.15.29:0/3181686967 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f130007b630 con 0x7f1310064760 2024-01-24T14:13:43.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.725+0000 7f130dd9b700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f12f005f300 0x7f12f00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:43.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.729+0000 7f12fe7fc700 1 -- 172.21.15.29:0/3181686967 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f130004c200 con 0x7f1310064760 2024-01-24T14:13:43.733 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:43.732+0000 7f71eab8f680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:13:43.888 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:43.887+0000 7f71eab8f680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:13:43.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.912+0000 7f1314c50700 1 -- 172.21.15.29:0/3181686967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13100657a0 con 0x7f1310064760 2024-01-24T14:13:43.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.913+0000 7f12fe7fc700 1 -- 172.21.15.29:0/3181686967 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f13000290a0 con 0x7f1310064760 2024-01-24T14:13:43.913 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T14:13:43.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.915+0000 7f1314c50700 1 -- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f12f005f300 msgr2=0x7f12f00617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:43.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.915+0000 7f1314c50700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f12f005f300 0x7f12f00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:43.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.915+0000 7f1314c50700 1 -- 172.21.15.29:0/3181686967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1310064760 msgr2=0x7f1310141940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:43.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.915+0000 7f1314c50700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1310064760 0x7f1310141940 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f1300009e00 tx=0x7f1300017000).stop 2024-01-24T14:13:43.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.915+0000 7f1314c50700 1 -- 172.21.15.29:0/3181686967 shutdown_connections 2024-01-24T14:13:43.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.915+0000 7f1314c50700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13100c8490 0x7f13100c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.915+0000 7f1314c50700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f12f005f300 0x7f12f00617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:43.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.915+0000 7f1314c50700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1310064760 0x7f1310141940 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.915+0000 7f1314c50700 1 --2- 172.21.15.29:0/3181686967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13100bfd40 0x7f13100c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:43.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.915+0000 7f1314c50700 1 -- 172.21.15.29:0/3181686967 >> 172.21.15.29:0/3181686967 conn(0x7f13100cb8b0 msgr2=0x7f1310156720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:43.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.915+0000 7f1314c50700 1 -- 172.21.15.29:0/3181686967 shutdown_connections 2024-01-24T14:13:43.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:43.915+0000 7f1314c50700 1 -- 172.21.15.29:0/3181686967 wait complete. 2024-01-24T14:13:43.931 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T14:13:43.932 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:43.930+0000 7f4e80e9c680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:13:43.962 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:43.961+0000 7f71eab8f680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:13:44.029 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:44.028+0000 7f71eab8f680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:13:44.056 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:44.056+0000 7f4e80e9c680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:13:44.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.066+0000 7f0990c1c700 1 Processor -- start 2024-01-24T14:13:44.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.066+0000 7f0990c1c700 1 -- start start 2024-01-24T14:13:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.067+0000 7f0990c1c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f098c1384f0 0x7f098c06a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.067+0000 7f0990c1c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f098c0c8270 0x7f098c0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.067+0000 7f0990c1c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098c063150 con 0x7f098c062ca0 2024-01-24T14:13:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.067+0000 7f0990c1c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098c0bcda0 con 0x7f098c0c8270 2024-01-24T14:13:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.067+0000 7f0990c1c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098c065f80 con 0x7f098c1384f0 2024-01-24T14:13:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.067+0000 7f0989d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f098c0c8270 0x7f098c0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.067+0000 7f0989d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f098c0c8270 0x7f098c0627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35128/0 (socket says 172.21.15.29:35128) 2024-01-24T14:13:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.067+0000 7f0989d9b700 1 -- 172.21.15.29:0/2767108123 learned_addr learned my addr 172.21.15.29:0/2767108123 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:44.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.067+0000 7f098a59c700 1 --2- 172.21.15.29:0/2767108123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f098c1384f0 0x7f098c06a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.068+0000 7f0989d9b700 1 -- 172.21.15.29:0/2767108123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f098c1384f0 msgr2=0x7f098c06a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:44.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.068+0000 7f0989d9b700 1 --2- 172.21.15.29:0/2767108123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f098c1384f0 0x7f098c06a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.068+0000 7f0989d9b700 1 -- 172.21.15.29:0/2767108123 >> v1:172.21.15.163:6789/0 conn(0x7f098c062ca0 legacy=0x7f098c0692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.068+0000 7f0989d9b700 1 -- 172.21.15.29:0/2767108123 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c066260 con 0x7f098c0c8270 2024-01-24T14:13:44.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.068+0000 7f098959a700 1 -- 172.21.15.29:0/2767108123 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404764699 0 0) 0x7f0984001490 con 0x7f098c062ca0 2024-01-24T14:13:44.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.068+0000 7f0989d9b700 1 --2- 172.21.15.29:0/2767108123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f098c0c8270 0x7f098c0627b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f097c006fd0 tx=0x7f097c0090f0).ready entity=mon.0 client_cookie=2a692700915b3672 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:44.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.068+0000 7f098959a700 1 -- 172.21.15.29:0/2767108123 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c00b580 con 0x7f098c0c8270 2024-01-24T14:13:44.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.068+0000 7f098959a700 1 -- 172.21.15.29:0/2767108123 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097c00b6e0 con 0x7f098c0c8270 2024-01-24T14:13:44.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.068+0000 7f098959a700 1 -- 172.21.15.29:0/2767108123 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c00ba80 con 0x7f098c0c8270 2024-01-24T14:13:44.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.069+0000 7f0990c1c700 1 -- 172.21.15.29:0/2767108123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f098c0c8270 msgr2=0x7f098c0627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.069+0000 7f0990c1c700 1 --2- 172.21.15.29:0/2767108123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f098c0c8270 0x7f098c0627b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f097c006fd0 tx=0x7f097c0090f0).stop 2024-01-24T14:13:44.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.069+0000 7f0990c1c700 1 -- 172.21.15.29:0/2767108123 shutdown_connections 2024-01-24T14:13:44.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.069+0000 7f0990c1c700 1 --2- 172.21.15.29:0/2767108123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f098c1384f0 0x7f098c06a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.069+0000 7f0990c1c700 1 --2- 172.21.15.29:0/2767108123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f098c0c8270 0x7f098c0627b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.069+0000 7f0990c1c700 1 -- 172.21.15.29:0/2767108123 >> 172.21.15.29:0/2767108123 conn(0x7f098c064390 msgr2=0x7f098c0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:44.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.069+0000 7f0990c1c700 1 -- 172.21.15.29:0/2767108123 shutdown_connections 2024-01-24T14:13:44.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.069+0000 7f0990c1c700 1 -- 172.21.15.29:0/2767108123 wait complete. 2024-01-24T14:13:44.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.070+0000 7f0990c1c700 1 Processor -- start 2024-01-24T14:13:44.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.070+0000 7f0990c1c700 1 -- start start 2024-01-24T14:13:44.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.071+0000 7f0990c1c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c062ca0 0x7f098c0c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.071+0000 7f0990c1c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f098c0c8270 0x7f098c0aa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.071+0000 7f0990c1c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f098c1384f0 0x7f098c0c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.071+0000 7f0990c1c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c0aaaa0 con 0x7f098c1384f0 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.071+0000 7f0990c1c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c0ab620 con 0x7f098c062ca0 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.071+0000 7f0990c1c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098c0aacf0 con 0x7f098c0c8270 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.071+0000 7f098ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f098c1384f0 0x7f098c0c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.071+0000 7f098ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f098c1384f0 0x7f098c0c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35138/0 (socket says 172.21.15.29:35138) 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.071+0000 7f098ad9d700 1 -- 172.21.15.29:0/1108638635 learned_addr learned my addr 172.21.15.29:0/1108638635 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.071+0000 7f098a59c700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c062ca0 0x7f098c0c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.071+0000 7f0989d9b700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f098c0c8270 0x7f098c0aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.072+0000 7f098ad9d700 1 -- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f098c0c8270 msgr2=0x7f098c0aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.072+0000 7f098ad9d700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f098c0c8270 0x7f098c0aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.072+0000 7f098ad9d700 1 -- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c062ca0 msgr2=0x7f098c0c34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.072+0000 7f098ad9d700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c062ca0 0x7f098c0c34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.072+0000 7f098ad9d700 1 -- 172.21.15.29:0/1108638635 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c009370 con 0x7f098c1384f0 2024-01-24T14:13:44.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.072+0000 7f098ad9d700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f098c1384f0 0x7f098c0c3a30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f098400c7f0 tx=0x7f0984007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:44.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.072+0000 7f09827fc700 1 -- 172.21.15.29:0/1108638635 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984010510 con 0x7f098c1384f0 2024-01-24T14:13:44.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.072+0000 7f09827fc700 1 -- 172.21.15.29:0/1108638635 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0984010670 con 0x7f098c1384f0 2024-01-24T14:13:44.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.072+0000 7f0990c1c700 1 -- 172.21.15.29:0/1108638635 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098c0aafd0 con 0x7f098c1384f0 2024-01-24T14:13:44.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.072+0000 7f09827fc700 1 -- 172.21.15.29:0/1108638635 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09840109f0 con 0x7f098c1384f0 2024-01-24T14:13:44.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.072+0000 7f0990c1c700 1 -- 172.21.15.29:0/1108638635 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098c0c3f20 con 0x7f098c1384f0 2024-01-24T14:13:44.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.073+0000 7f0990c1c700 1 -- 172.21.15.29:0/1108638635 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098c0ab7b0 con 0x7f098c1384f0 2024-01-24T14:13:44.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.074+0000 7f09827fc700 1 -- 172.21.15.29:0/1108638635 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f098401b400 con 0x7f098c1384f0 2024-01-24T14:13:44.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.074+0000 7f09827fc700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f096c05f330 0x7f096c0617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.074+0000 7f098a59c700 1 -- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f096c05f330 msgr2=0x7f096c0617f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:44.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.074+0000 7f09827fc700 1 -- 172.21.15.29:0/1108638635 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f098407a580 con 0x7f098c1384f0 2024-01-24T14:13:44.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.074+0000 7f098a59c700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f096c05f330 0x7f096c0617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:44.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.079+0000 7f09827fc700 1 -- 172.21.15.29:0/1108638635 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f098404b0f0 con 0x7f098c1384f0 2024-01-24T14:13:44.238 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:44.238+0000 7f4e80e9c680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:13:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.261+0000 7f0990c1c700 1 -- 172.21.15.29:0/1108638635 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f098c063150 con 0x7f098c1384f0 2024-01-24T14:13:44.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.261+0000 7f09827fc700 1 -- 172.21.15.29:0/1108638635 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f098c063150 con 0x7f098c1384f0 2024-01-24T14:13:44.262 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T14:13:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.263+0000 7f0990c1c700 1 -- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f096c05f330 msgr2=0x7f096c0617f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.263+0000 7f0990c1c700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f096c05f330 0x7f096c0617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.263+0000 7f0990c1c700 1 -- 172.21.15.29:0/1108638635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f098c1384f0 msgr2=0x7f098c0c3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.263+0000 7f0990c1c700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f098c1384f0 0x7f098c0c3a30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f098400c7f0 tx=0x7f0984007de0).stop 2024-01-24T14:13:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.263+0000 7f0990c1c700 1 -- 172.21.15.29:0/1108638635 shutdown_connections 2024-01-24T14:13:44.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.263+0000 7f0990c1c700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f096c05f330 0x7f096c0617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:44.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.263+0000 7f0990c1c700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f098c1384f0 0x7f098c0c3a30 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.263+0000 7f0990c1c700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c062ca0 0x7f098c0c34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.263+0000 7f0990c1c700 1 --2- 172.21.15.29:0/1108638635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f098c0c8270 0x7f098c0aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.263+0000 7f0990c1c700 1 -- 172.21.15.29:0/1108638635 >> 172.21.15.29:0/1108638635 conn(0x7f098c064390 msgr2=0x7f098c153cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:44.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.263+0000 7f0990c1c700 1 -- 172.21.15.29:0/1108638635 shutdown_connections 2024-01-24T14:13:44.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.263+0000 7f0990c1c700 1 -- 172.21.15.29:0/1108638635 wait complete. 2024-01-24T14:13:44.278 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:13:44.285 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:44.285+0000 7f71eab8f680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:13:44.328 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:44.327+0000 7f4e80e9c680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:13:44.409 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:44.409+0000 7f4e80e9c680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:13:44.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.409+0000 7fab4927a700 1 Processor -- start 2024-01-24T14:13:44.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.409+0000 7fab4927a700 1 -- start start 2024-01-24T14:13:44.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.410+0000 7fab4927a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab440c5410 0x7fab440aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.410+0000 7fab4927a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab44154750 0x7fab440631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.410+0000 7fab4927a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab440aa6c0 con 0x7fab44154370 2024-01-24T14:13:44.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.410+0000 7fab4927a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab44064240 con 0x7fab440c5410 2024-01-24T14:13:44.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.410+0000 7fab4927a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab440cb5c0 con 0x7fab44154750 2024-01-24T14:13:44.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.410+0000 7fab427fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab440c5410 0x7fab440aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.410+0000 7fab42ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fab44154370 0x7fab440ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46624/0 (socket says 172.21.15.29:46624) 2024-01-24T14:13:44.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.410+0000 7fab427fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab440c5410 0x7fab440aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35144/0 (socket says 172.21.15.29:35144) 2024-01-24T14:13:44.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.410+0000 7fab42ffd700 1 -- 172.21.15.29:0/173372127 learned_addr learned my addr 172.21.15.29:0/173372127 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:44.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.410+0000 7fab437fe700 1 --2- 172.21.15.29:0/173372127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab44154750 0x7fab440631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.411+0000 7fab427fc700 1 -- 172.21.15.29:0/173372127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab44154750 msgr2=0x7fab440631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:44.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.411+0000 7fab427fc700 1 --2- 172.21.15.29:0/173372127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab44154750 0x7fab440631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.411+0000 7fab427fc700 1 -- 172.21.15.29:0/173372127 >> v1:172.21.15.29:6789/0 conn(0x7fab44154370 legacy=0x7fab440ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.411+0000 7fab427fc700 1 -- 172.21.15.29:0/173372127 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440cb8a0 con 0x7fab440c5410 2024-01-24T14:13:44.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.411+0000 7fab41ffb700 1 -- 172.21.15.29:0/173372127 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617785762 0 0) 0x7fab440aa6c0 con 0x7fab44154370 2024-01-24T14:13:44.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.411+0000 7fab427fc700 1 --2- 172.21.15.29:0/173372127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab440c5410 0x7fab440aa0e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fab38009760 tx=0x7fab38009060).ready entity=mon.0 client_cookie=acffe001f19a8539 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:44.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.411+0000 7fab41ffb700 1 -- 172.21.15.29:0/173372127 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800f510 con 0x7fab440c5410 2024-01-24T14:13:44.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.411+0000 7fab41ffb700 1 -- 172.21.15.29:0/173372127 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3800f670 con 0x7fab440c5410 2024-01-24T14:13:44.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.411+0000 7fab41ffb700 1 -- 172.21.15.29:0/173372127 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800f9f0 con 0x7fab440c5410 2024-01-24T14:13:44.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.412+0000 7fab4927a700 1 -- 172.21.15.29:0/173372127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab440c5410 msgr2=0x7fab440aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.412+0000 7fab4927a700 1 --2- 172.21.15.29:0/173372127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab440c5410 0x7fab440aa0e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fab38009760 tx=0x7fab38009060).stop 2024-01-24T14:13:44.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.412+0000 7fab4927a700 1 -- 172.21.15.29:0/173372127 shutdown_connections 2024-01-24T14:13:44.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.412+0000 7fab4927a700 1 --2- 172.21.15.29:0/173372127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab44154750 0x7fab440631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.412+0000 7fab4927a700 1 --2- 172.21.15.29:0/173372127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab440c5410 0x7fab440aa0e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.412+0000 7fab4927a700 1 -- 172.21.15.29:0/173372127 >> 172.21.15.29:0/173372127 conn(0x7fab4405ec80 msgr2=0x7fab440cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:44.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.412+0000 7fab4927a700 1 -- 172.21.15.29:0/173372127 shutdown_connections 2024-01-24T14:13:44.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.412+0000 7fab4927a700 1 -- 172.21.15.29:0/173372127 wait complete. 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.413+0000 7fab4927a700 1 Processor -- start 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.413+0000 7fab4927a700 1 -- start start 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.414+0000 7fab4927a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab440c5410 0x7fab44158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.414+0000 7fab4927a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab44154370 0x7fab44150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.414+0000 7fab4927a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab44154750 0x7fab4413f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.414+0000 7fab4927a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab440addc0 con 0x7fab44154370 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.414+0000 7fab4927a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab440c3570 con 0x7fab440c5410 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.414+0000 7fab4927a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab441590b0 con 0x7fab44154750 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.414+0000 7fab427fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab44154370 0x7fab44150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.414+0000 7fab427fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab44154370 0x7fab44150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35156/0 (socket says 172.21.15.29:35156) 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.414+0000 7fab427fc700 1 -- 172.21.15.29:0/1176305741 learned_addr learned my addr 172.21.15.29:0/1176305741 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.414+0000 7fab437fe700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab44154750 0x7fab4413f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.414+0000 7fab42ffd700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab440c5410 0x7fab44158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab427fc700 1 -- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab44154750 msgr2=0x7fab4413f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab427fc700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab44154750 0x7fab4413f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab427fc700 1 -- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab440c5410 msgr2=0x7fab44158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab427fc700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab440c5410 0x7fab44158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab427fc700 1 -- 172.21.15.29:0/1176305741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab380092a0 con 0x7fab44154370 2024-01-24T14:13:44.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab42ffd700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab440c5410 0x7fab44158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:44.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab437fe700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab44154750 0x7fab4413f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:44.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab427fc700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab44154370 0x7fab44150ef0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7fab440c8920 tx=0x7fab38017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:44.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab2affd700 1 -- 172.21.15.29:0/1176305741 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800f7e0 con 0x7fab44154370 2024-01-24T14:13:44.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab2affd700 1 -- 172.21.15.29:0/1176305741 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3800f940 con 0x7fab44154370 2024-01-24T14:13:44.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab4927a700 1 -- 172.21.15.29:0/1176305741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab440c3830 con 0x7fab44154370 2024-01-24T14:13:44.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab2affd700 1 -- 172.21.15.29:0/1176305741 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3801b660 con 0x7fab44154370 2024-01-24T14:13:44.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.415+0000 7fab4927a700 1 -- 172.21.15.29:0/1176305741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab44158810 con 0x7fab44154370 2024-01-24T14:13:44.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.417+0000 7fab4927a700 1 -- 172.21.15.29:0/1176305741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab440c39c0 con 0x7fab44154370 2024-01-24T14:13:44.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.417+0000 7fab2affd700 1 -- 172.21.15.29:0/1176305741 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fab3801ba50 con 0x7fab44154370 2024-01-24T14:13:44.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.417+0000 7fab2affd700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3005f300 0x7fab300617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.417+0000 7fab2affd700 1 -- 172.21.15.29:0/1176305741 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fab3807c220 con 0x7fab44154370 2024-01-24T14:13:44.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.417+0000 7fab42ffd700 1 -- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3005f300 msgr2=0x7fab300617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:44.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.417+0000 7fab42ffd700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3005f300 0x7fab300617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:44.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.422+0000 7fab2affd700 1 -- 172.21.15.29:0/1176305741 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fab3804ccf0 con 0x7fab44154370 2024-01-24T14:13:44.470 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:44.469+0000 7f71eab8f680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:13:44.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.618+0000 7fab42ffd700 1 -- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3005f300 msgr2=0x7fab300617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:44.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.618+0000 7fab42ffd700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3005f300 0x7fab300617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:13:44.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.649+0000 7fab4927a700 1 -- 172.21.15.29:0/1176305741 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab44158f10 con 0x7fab44154370 2024-01-24T14:13:44.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.650+0000 7fab2affd700 1 -- 172.21.15.29:0/1176305741 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fab44158f10 con 0x7fab44154370 2024-01-24T14:13:44.651 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:44.651 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1149323432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1149323432},{"type":"v1","addr":"172.21.15.163:6835","nonce":1149323432}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6157,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2941255566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":2941255566},{"type":"v1","addr":"172.21.15.29:6835","nonce":2941255566}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6163,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2235282008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2235282008},{"type":"v1","addr":"172.21.15.163:6837","nonce":2235282008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:13:27.270903+0000","modified":"2024-01-24T14:13:30.278839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6366":{"gid":6366,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":4,"addr":"172.21.15.29:6833/3957899438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3957899438},{"type":"v1","addr":"172.21.15.29:6833","nonce":3957899438}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-24T14:13:44.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.652+0000 7fab4927a700 1 -- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3005f300 msgr2=0x7fab300617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:44.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.652+0000 7fab4927a700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3005f300 0x7fab300617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:44.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.652+0000 7fab4927a700 1 -- 172.21.15.29:0/1176305741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab44154370 msgr2=0x7fab44150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.652+0000 7fab4927a700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab44154370 0x7fab44150ef0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7fab440c8920 tx=0x7fab38017000).stop 2024-01-24T14:13:44.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.652+0000 7fab4927a700 1 -- 172.21.15.29:0/1176305741 shutdown_connections 2024-01-24T14:13:44.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.652+0000 7fab4927a700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab44154750 0x7fab4413f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.652+0000 7fab4927a700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab440c5410 0x7fab44158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.652+0000 7fab4927a700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3005f300 0x7fab300617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.652+0000 7fab4927a700 1 --2- 172.21.15.29:0/1176305741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab44154370 0x7fab44150ef0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.652+0000 7fab4927a700 1 -- 172.21.15.29:0/1176305741 >> 172.21.15.29:0/1176305741 conn(0x7fab4405ec80 msgr2=0x7fab440c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.652+0000 7fab4927a700 1 -- 172.21.15.29:0/1176305741 shutdown_connections 2024-01-24T14:13:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.652+0000 7fab4927a700 1 -- 172.21.15.29:0/1176305741 wait complete. 2024-01-24T14:13:44.654 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 35 2024-01-24T14:13:44.666 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:13:44.711 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:44.711+0000 7f4e80e9c680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:13:44.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.797+0000 7fbbcf9e0700 1 Processor -- start 2024-01-24T14:13:44.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.797+0000 7fbbcf9e0700 1 -- start start 2024-01-24T14:13:44.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbcf9e0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbc8154370 0x7fbbc80c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbcf9e0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbbc80aa0e0 0x7fbbc8154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbcf9e0700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc80aa6c0 con 0x7fbbc80ab1f0 2024-01-24T14:13:44.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbcf9e0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc8064240 con 0x7fbbc80aa0e0 2024-01-24T14:13:44.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbcf9e0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80cb5c0 con 0x7fbbc8154370 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbccf7b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbbc80aa0e0 0x7fbbc8154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbccf7b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbbc80aa0e0 0x7fbbc8154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35164/0 (socket says 172.21.15.29:35164) 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbccf7b700 1 -- 172.21.15.29:0/1704895465 learned_addr learned my addr 172.21.15.29:0/1704895465 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbcd77c700 1 --2- 172.21.15.29:0/1704895465 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbc8154370 0x7fbbc80c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbccf7b700 1 -- 172.21.15.29:0/1704895465 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbc8154370 msgr2=0x7fbbc80c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbccf7b700 1 --2- 172.21.15.29:0/1704895465 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbc8154370 0x7fbbc80c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbccf7b700 1 -- 172.21.15.29:0/1704895465 >> v1:172.21.15.29:6789/0 conn(0x7fbbc80ab1f0 legacy=0x7fbbc80631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbccf7b700 1 -- 172.21.15.29:0/1704895465 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc80cb8a0 con 0x7fbbc80aa0e0 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.798+0000 7fbbbffff700 1 -- 172.21.15.29:0/1704895465 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161123553 0 0) 0x7fbbc80aa6c0 con 0x7fbbc80ab1f0 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.799+0000 7fbbccf7b700 1 --2- 172.21.15.29:0/1704895465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbc80aa0e0 0x7fbbc8154750 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7fbbb8006fd0 tx=0x7fbbb80090f0).ready entity=mon.0 client_cookie=7df206fd418b8736 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.799+0000 7fbbbffff700 1 -- 172.21.15.29:0/1704895465 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb800b580 con 0x7fbbc80aa0e0 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.799+0000 7fbbbffff700 1 -- 172.21.15.29:0/1704895465 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb800b6e0 con 0x7fbbc80aa0e0 2024-01-24T14:13:44.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.799+0000 7fbbbffff700 1 -- 172.21.15.29:0/1704895465 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb800ba60 con 0x7fbbc80aa0e0 2024-01-24T14:13:44.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.799+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/1704895465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbc80aa0e0 msgr2=0x7fbbc8154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.799+0000 7fbbcf9e0700 1 --2- 172.21.15.29:0/1704895465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbc80aa0e0 0x7fbbc8154750 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7fbbb8006fd0 tx=0x7fbbb80090f0).stop 2024-01-24T14:13:44.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.800+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/1704895465 shutdown_connections 2024-01-24T14:13:44.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.800+0000 7fbbcf9e0700 1 --2- 172.21.15.29:0/1704895465 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbc8154370 0x7fbbc80c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.800+0000 7fbbcf9e0700 1 --2- 172.21.15.29:0/1704895465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbc80aa0e0 0x7fbbc8154750 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.800+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/1704895465 >> 172.21.15.29:0/1704895465 conn(0x7fbbc805ec80 msgr2=0x7fbbc80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:44.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.800+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/1704895465 shutdown_connections 2024-01-24T14:13:44.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.800+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/1704895465 wait complete. 2024-01-24T14:13:44.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.801+0000 7fbbcf9e0700 1 Processor -- start 2024-01-24T14:13:44.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.801+0000 7fbbcf9e0700 1 -- start start 2024-01-24T14:13:44.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.802+0000 7fbbcf9e0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbc80aa0e0 0x7fbbc80adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.802+0000 7fbbcf9e0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbc80ab1f0 0x7fbbc814e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.802+0000 7fbbcf9e0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbc8154370 0x7fbbc815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.802+0000 7fbbcf9e0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc8158eb0 con 0x7fbbc8154370 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.802+0000 7fbbcf9e0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc8159150 con 0x7fbbc80aa0e0 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.802+0000 7fbbcf9e0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc814ea30 con 0x7fbbc80ab1f0 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.802+0000 7fbbcdf7d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbc8154370 0x7fbbc815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.802+0000 7fbbccf7b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbc80ab1f0 0x7fbbc814e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.802+0000 7fbbcdf7d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbc8154370 0x7fbbc815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35174/0 (socket says 172.21.15.29:35174) 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.802+0000 7fbbcdf7d700 1 -- 172.21.15.29:0/4023010033 learned_addr learned my addr 172.21.15.29:0/4023010033 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.802+0000 7fbbcd77c700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbc80aa0e0 0x7fbbc80adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbcdf7d700 1 -- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbc80ab1f0 msgr2=0x7fbbc814e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbcdf7d700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbc80ab1f0 0x7fbbc814e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbcdf7d700 1 -- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbc80aa0e0 msgr2=0x7fbbc80adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbcdf7d700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbc80aa0e0 0x7fbbc80adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbcdf7d700 1 -- 172.21.15.29:0/4023010033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb8009370 con 0x7fbbc8154370 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbccf7b700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbc80ab1f0 0x7fbbc814e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:44.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbcd77c700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbc80aa0e0 0x7fbbc80adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:44.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbcdf7d700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbc8154370 0x7fbbc815aaa0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7fbbc000c7e0 tx=0x7fbbc0007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:44.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbbd7fa700 1 -- 172.21.15.29:0/4023010033 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0010510 con 0x7fbbc8154370 2024-01-24T14:13:44.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbbd7fa700 1 -- 172.21.15.29:0/4023010033 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc0010670 con 0x7fbbc8154370 2024-01-24T14:13:44.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbbd7fa700 1 -- 172.21.15.29:0/4023010033 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0010a10 con 0x7fbbc8154370 2024-01-24T14:13:44.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/4023010033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc80aff90 con 0x7fbbc8154370 2024-01-24T14:13:44.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.803+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/4023010033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc80b0550 con 0x7fbbc8154370 2024-01-24T14:13:44.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.804+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/4023010033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc80475e0 con 0x7fbbc8154370 2024-01-24T14:13:44.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.805+0000 7fbbbd7fa700 1 -- 172.21.15.29:0/4023010033 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fbbc001b400 con 0x7fbbc8154370 2024-01-24T14:13:44.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.805+0000 7fbbbd7fa700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbbac05f2d0 0x7fbbac061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:44.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.805+0000 7fbbbd7fa700 1 -- 172.21.15.29:0/4023010033 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fbbc007a750 con 0x7fbbc8154370 2024-01-24T14:13:44.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.805+0000 7fbbcd77c700 1 -- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbbac05f2d0 msgr2=0x7fbbac061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:44.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.805+0000 7fbbcd77c700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbbac05f2d0 0x7fbbac061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:44.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:44.814+0000 7fbbbd7fa700 1 -- 172.21.15.29:0/4023010033 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbbc004b2c0 con 0x7fbbc8154370 2024-01-24T14:13:44.932 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:44.931+0000 7f4e80e9c680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:13:45.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.002+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/4023010033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbc80cbb80 con 0x7fbbc8154370 2024-01-24T14:13:45.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.003+0000 7fbbbd7fa700 1 -- 172.21.15.29:0/4023010033 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+18546 (secure 0 0 0) 0x7fbbc004e8e0 con 0x7fbbc8154370 2024-01-24T14:13:45.004 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:45.004 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":91,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:13:39.554230+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:13:25.288458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-24T14:13:26.243223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6833/43241":"2024-01-25T14:13:39.554196+0000","172.21.15.163:6832/43241":"2024-01-25T14:13:39.554196+0000","172.21.15.163:0/2629162943":"2024-01-25T14:13:39.554196+0000","172.21.15.163:0/1057348791":"2024-01-25T14:13:39.554196+0000","172.21.15.163:0/1616535923":"2024-01-25T14:13:39.554196+0000","172.21.15.163:0/1701533468":"2024-01-25T14:13:39.554196+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:13:45.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.005+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbbac05f2d0 msgr2=0x7fbbac061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:45.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.005+0000 7fbbcf9e0700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbbac05f2d0 0x7fbbac061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:45.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.005+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/4023010033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbc8154370 msgr2=0x7fbbc815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.005+0000 7fbbcf9e0700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbc8154370 0x7fbbc815aaa0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7fbbc000c7e0 tx=0x7fbbc0007f70).stop 2024-01-24T14:13:45.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.006+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/4023010033 shutdown_connections 2024-01-24T14:13:45.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.006+0000 7fbbcf9e0700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbbac05f2d0 0x7fbbac061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:45.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.006+0000 7fbbcf9e0700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbc8154370 0x7fbbc815aaa0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.006+0000 7fbbcf9e0700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbc80aa0e0 0x7fbbc80adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.006+0000 7fbbcf9e0700 1 --2- 172.21.15.29:0/4023010033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbc80ab1f0 0x7fbbc814e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.006+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/4023010033 >> 172.21.15.29:0/4023010033 conn(0x7fbbc805ec80 msgr2=0x7fbbc80ca6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:45.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.006+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/4023010033 shutdown_connections 2024-01-24T14:13:45.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.006+0000 7fbbcf9e0700 1 -- 172.21.15.29:0/4023010033 wait complete. 2024-01-24T14:13:45.020 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T14:13:45.020 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:13:45.124 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:45.123+0000 7f71eab8f680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:13:45.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.151+0000 7f2610533700 1 Processor -- start 2024-01-24T14:13:45.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.151+0000 7f2610533700 1 -- start start 2024-01-24T14:13:45.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.152+0000 7f2610533700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2608154370 0x7f26080c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.152+0000 7f2610533700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26080aa0e0 0x7f2608154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.152+0000 7f2610533700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26080aa6c0 con 0x7f26080ab1f0 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.152+0000 7f2610533700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2608064240 con 0x7f2608154370 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.152+0000 7f2610533700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26080cb5c0 con 0x7f26080aa0e0 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.152+0000 7f260e2cf700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2608154370 0x7f26080c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.152+0000 7f260ead0700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f26080ab1f0 0x7f26080631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46648/0 (socket says 172.21.15.29:46648) 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.152+0000 7f260e2cf700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2608154370 0x7f26080c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35184/0 (socket says 172.21.15.29:35184) 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.152+0000 7f260ead0700 1 -- 172.21.15.29:0/3118972414 learned_addr learned my addr 172.21.15.29:0/3118972414 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.152+0000 7f260dace700 1 --2- 172.21.15.29:0/3118972414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26080aa0e0 0x7f2608154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.153+0000 7f260e2cf700 1 -- 172.21.15.29:0/3118972414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26080aa0e0 msgr2=0x7f2608154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.153+0000 7f260e2cf700 1 --2- 172.21.15.29:0/3118972414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26080aa0e0 0x7f2608154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.153+0000 7f260d2cd700 1 -- 172.21.15.29:0/3118972414 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204229412 0 0) 0x7f26080aa6c0 con 0x7f26080ab1f0 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.153+0000 7f260e2cf700 1 -- 172.21.15.29:0/3118972414 >> v1:172.21.15.29:6789/0 conn(0x7f26080ab1f0 legacy=0x7f26080631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.153+0000 7f260e2cf700 1 -- 172.21.15.29:0/3118972414 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26080cb8a0 con 0x7f2608154370 2024-01-24T14:13:45.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.153+0000 7f260dace700 1 --2- 172.21.15.29:0/3118972414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26080aa0e0 0x7f2608154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:45.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.153+0000 7f260e2cf700 1 --2- 172.21.15.29:0/3118972414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2608154370 0x7f26080c5410 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f2604006fd0 tx=0x7f2604006f40).ready entity=mon.0 client_cookie=58fe1f824396451b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:45.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.153+0000 7f260d2cd700 1 -- 172.21.15.29:0/3118972414 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604005880 con 0x7f2608154370 2024-01-24T14:13:45.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.153+0000 7f260d2cd700 1 -- 172.21.15.29:0/3118972414 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26040059e0 con 0x7f2608154370 2024-01-24T14:13:45.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.153+0000 7f260d2cd700 1 -- 172.21.15.29:0/3118972414 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604005d60 con 0x7f2608154370 2024-01-24T14:13:45.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.154+0000 7f2610533700 1 -- 172.21.15.29:0/3118972414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2608154370 msgr2=0x7f26080c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.154+0000 7f2610533700 1 --2- 172.21.15.29:0/3118972414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2608154370 0x7f26080c5410 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f2604006fd0 tx=0x7f2604006f40).stop 2024-01-24T14:13:45.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.154+0000 7f2610533700 1 -- 172.21.15.29:0/3118972414 shutdown_connections 2024-01-24T14:13:45.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.154+0000 7f2610533700 1 --2- 172.21.15.29:0/3118972414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2608154370 0x7f26080c5410 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.154+0000 7f2610533700 1 --2- 172.21.15.29:0/3118972414 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26080aa0e0 0x7f2608154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.154+0000 7f2610533700 1 -- 172.21.15.29:0/3118972414 >> 172.21.15.29:0/3118972414 conn(0x7f260805ec80 msgr2=0x7f26080cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.154+0000 7f2610533700 1 -- 172.21.15.29:0/3118972414 shutdown_connections 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.154+0000 7f2610533700 1 -- 172.21.15.29:0/3118972414 wait complete. 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.155+0000 7f2610533700 1 Processor -- start 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.155+0000 7f2610533700 1 -- start start 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.156+0000 7f2610533700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26080aa0e0 0x7f26080c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.156+0000 7f2610533700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26080ab1f0 0x7f260814d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.156+0000 7f2610533700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2608154370 0x7f260814d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.156+0000 7f2610533700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260814dbf0 con 0x7f2608154370 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.156+0000 7f2610533700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608158c00 con 0x7f26080aa0e0 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.156+0000 7f2610533700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26081582d0 con 0x7f26080ab1f0 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.156+0000 7f260ead0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2608154370 0x7f260814d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.156+0000 7f260dace700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26080ab1f0 0x7f260814d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.156+0000 7f260ead0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2608154370 0x7f260814d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35198/0 (socket says 172.21.15.29:35198) 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.156+0000 7f260ead0700 1 -- 172.21.15.29:0/3191658968 learned_addr learned my addr 172.21.15.29:0/3191658968 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.156+0000 7f260e2cf700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26080aa0e0 0x7f26080c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f260ead0700 1 -- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26080ab1f0 msgr2=0x7f260814d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f260ead0700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26080ab1f0 0x7f260814d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f260ead0700 1 -- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26080aa0e0 msgr2=0x7f26080c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f260ead0700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26080aa0e0 0x7f26080c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f260ead0700 1 -- 172.21.15.29:0/3191658968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80092a0 con 0x7f2608154370 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f260e2cf700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26080aa0e0 0x7f26080c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f260dace700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26080ab1f0 0x7f260814d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f260ead0700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2608154370 0x7f260814d700 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f260000c7e0 tx=0x7f2600007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f25fe7fc700 1 -- 172.21.15.29:0/3191658968 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600005d30 con 0x7f2608154370 2024-01-24T14:13:45.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f25fe7fc700 1 -- 172.21.15.29:0/3191658968 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2600005e90 con 0x7f2608154370 2024-01-24T14:13:45.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f2610533700 1 -- 172.21.15.29:0/3191658968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2604009250 con 0x7f2608154370 2024-01-24T14:13:45.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f25fe7fc700 1 -- 172.21.15.29:0/3191658968 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600007320 con 0x7f2608154370 2024-01-24T14:13:45.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.157+0000 7f2610533700 1 -- 172.21.15.29:0/3191658968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2608158890 con 0x7f2608154370 2024-01-24T14:13:45.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.158+0000 7f25fe7fc700 1 -- 172.21.15.29:0/3191658968 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f2600007710 con 0x7f2608154370 2024-01-24T14:13:45.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.159+0000 7f2610533700 1 -- 172.21.15.29:0/3191658968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2608158070 con 0x7f2608154370 2024-01-24T14:13:45.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.159+0000 7f25fe7fc700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f25ec05f2d0 0x7f25ec061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.159+0000 7f25fe7fc700 1 -- 172.21.15.29:0/3191658968 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f260007a8e0 con 0x7f2608154370 2024-01-24T14:13:45.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.159+0000 7f260e2cf700 1 -- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f25ec05f2d0 msgr2=0x7f25ec061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:45.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.159+0000 7f260e2cf700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f25ec05f2d0 0x7f25ec061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:45.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.163+0000 7f25fe7fc700 1 -- 172.21.15.29:0/3191658968 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f260004b450 con 0x7f2608154370 2024-01-24T14:13:45.193 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:45.192+0000 7f71eab8f680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:13:45.268 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:45.267+0000 7f71eab8f680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:13:45.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.360+0000 7f260e2cf700 1 -- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f25ec05f2d0 msgr2=0x7f25ec061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:45.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.360+0000 7f260e2cf700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f25ec05f2d0 0x7f25ec061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:13:45.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.391+0000 7f2610533700 1 -- 172.21.15.29:0/3191658968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26080475e0 con 0x7f2608154370 2024-01-24T14:13:45.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.391+0000 7f25fe7fc700 1 -- 172.21.15.29:0/3191658968 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f260004ea70 con 0x7f2608154370 2024-01-24T14:13:45.392 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:45.392 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1149323432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1149323432},{"type":"v1","addr":"172.21.15.163:6835","nonce":1149323432}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6157,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2941255566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":2941255566},{"type":"v1","addr":"172.21.15.29:6835","nonce":2941255566}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6163,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2235282008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2235282008},{"type":"v1","addr":"172.21.15.163:6837","nonce":2235282008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:13:27.270903+0000","modified":"2024-01-24T14:13:30.278839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6366":{"gid":6366,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":4,"addr":"172.21.15.29:6833/3957899438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3957899438},{"type":"v1","addr":"172.21.15.29:6833","nonce":3957899438}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-24T14:13:45.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.395+0000 7f2610533700 1 -- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f25ec05f2d0 msgr2=0x7f25ec061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:45.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.395+0000 7f2610533700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f25ec05f2d0 0x7f25ec061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:45.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.395+0000 7f2610533700 1 -- 172.21.15.29:0/3191658968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2608154370 msgr2=0x7f260814d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.395+0000 7f2610533700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2608154370 0x7f260814d700 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f260000c7e0 tx=0x7f2600007f90).stop 2024-01-24T14:13:45.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.395+0000 7f2610533700 1 -- 172.21.15.29:0/3191658968 shutdown_connections 2024-01-24T14:13:45.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.395+0000 7f2610533700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f25ec05f2d0 0x7f25ec061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:45.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.395+0000 7f2610533700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2608154370 0x7f260814d700 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.395+0000 7f2610533700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26080aa0e0 0x7f26080c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.395+0000 7f2610533700 1 --2- 172.21.15.29:0/3191658968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26080ab1f0 0x7f260814d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.396+0000 7f2610533700 1 -- 172.21.15.29:0/3191658968 >> 172.21.15.29:0/3191658968 conn(0x7f260805ec80 msgr2=0x7f260814fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:45.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.396+0000 7f2610533700 1 -- 172.21.15.29:0/3191658968 shutdown_connections 2024-01-24T14:13:45.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.396+0000 7f2610533700 1 -- 172.21.15.29:0/3191658968 wait complete. 2024-01-24T14:13:45.397 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 35 2024-01-24T14:13:45.414 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:45.414+0000 7f71eab8f680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:13:45.416 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:13:45.485 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:45.484+0000 7f71eab8f680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:13:45.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.548+0000 7f6753fc9700 1 Processor -- start 2024-01-24T14:13:45.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.548+0000 7f6753fc9700 1 -- start start 2024-01-24T14:13:45.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.549+0000 7f6753fc9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f674c134cc0 0x7f674c0c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.549+0000 7f6753fc9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f674c1350a0 0x7f674c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.549+0000 7f6753fc9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f674c0664f0 0x7f674c0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.549+0000 7f6753fc9700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674c153360 con 0x7f674c1350a0 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.549+0000 7f6753fc9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674c066e90 con 0x7f674c134cc0 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.549+0000 7f6753fc9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c068360 con 0x7f674c0664f0 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6751564700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f674c1350a0 0x7f674c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6751564700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f674c1350a0 0x7f674c152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35202/0 (socket says 172.21.15.29:35202) 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6751564700 1 -- 172.21.15.29:0/1826890303 learned_addr learned my addr 172.21.15.29:0/1826890303 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6751d65700 1 --2- 172.21.15.29:0/1826890303 >> v2:172.21.15.163:3300/0 conn(0x7f674c134cc0 0x7f674c0c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6752566700 1 --2- 172.21.15.29:0/1826890303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f674c0664f0 0x7f674c0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6751564700 1 -- 172.21.15.29:0/1826890303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f674c0664f0 msgr2=0x7f674c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6751564700 1 --2- 172.21.15.29:0/1826890303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f674c0664f0 0x7f674c0669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6751564700 1 -- 172.21.15.29:0/1826890303 >> v2:172.21.15.163:3300/0 conn(0x7f674c134cc0 msgr2=0x7f674c0c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:45.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6751564700 1 --2- 172.21.15.29:0/1826890303 >> v2:172.21.15.163:3300/0 conn(0x7f674c134cc0 0x7f674c0c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6751564700 1 -- 172.21.15.29:0/1826890303 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c0c3820 con 0x7f674c1350a0 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6751d65700 1 --2- 172.21.15.29:0/1826890303 >> v2:172.21.15.163:3300/0 conn(0x7f674c134cc0 0x7f674c0c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6751564700 1 --2- 172.21.15.29:0/1826890303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f674c1350a0 0x7f674c152e70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f674c0aee60 tx=0x7f673c0090f0).ready entity=mon.0 client_cookie=b7c060348ed934c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6750d63700 1 -- 172.21.15.29:0/1826890303 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c00f510 con 0x7f674c1350a0 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6750d63700 1 -- 172.21.15.29:0/1826890303 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c00f670 con 0x7f674c1350a0 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.550+0000 7f6750d63700 1 -- 172.21.15.29:0/1826890303 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c00f9f0 con 0x7f674c1350a0 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.551+0000 7f6753fc9700 1 -- 172.21.15.29:0/1826890303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f674c1350a0 msgr2=0x7f674c152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.551+0000 7f6753fc9700 1 --2- 172.21.15.29:0/1826890303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f674c1350a0 0x7f674c152e70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f674c0aee60 tx=0x7f673c0090f0).stop 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.551+0000 7f6753fc9700 1 -- 172.21.15.29:0/1826890303 shutdown_connections 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.551+0000 7f6753fc9700 1 --2- 172.21.15.29:0/1826890303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f674c0664f0 0x7f674c0669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.551+0000 7f6753fc9700 1 --2- 172.21.15.29:0/1826890303 >> v2:172.21.15.163:3300/0 conn(0x7f674c134cc0 0x7f674c0c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.551+0000 7f6753fc9700 1 --2- 172.21.15.29:0/1826890303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f674c1350a0 0x7f674c152e70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.551+0000 7f6753fc9700 1 -- 172.21.15.29:0/1826890303 >> 172.21.15.29:0/1826890303 conn(0x7f674c0c8bb0 msgr2=0x7f674c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.551+0000 7f6753fc9700 1 -- 172.21.15.29:0/1826890303 shutdown_connections 2024-01-24T14:13:45.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.551+0000 7f6753fc9700 1 -- 172.21.15.29:0/1826890303 wait complete. 2024-01-24T14:13:45.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.553+0000 7f6753fc9700 1 Processor -- start 2024-01-24T14:13:45.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.553+0000 7f6753fc9700 1 -- start start 2024-01-24T14:13:45.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.553+0000 7f6753fc9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f674c0664f0 0x7f674c0c2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.553+0000 7f6753fc9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f674c134cc0 0x7f674c064280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.553+0000 7f6753fc9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f674c1350a0 0x7f674c150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.553+0000 7f6753fc9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c13b2d0 con 0x7f674c134cc0 2024-01-24T14:13:45.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.553+0000 7f6753fc9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c065210 con 0x7f674c0664f0 2024-01-24T14:13:45.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.553+0000 7f6753fc9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c13a230 con 0x7f674c1350a0 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6751564700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f674c134cc0 0x7f674c064280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6751564700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f674c134cc0 0x7f674c064280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35208/0 (socket says 172.21.15.29:35208) 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6751564700 1 -- 172.21.15.29:0/3177556273 learned_addr learned my addr 172.21.15.29:0/3177556273 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6751d65700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f674c0664f0 0x7f674c0c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6752566700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f674c1350a0 0x7f674c150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6751564700 1 -- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f674c1350a0 msgr2=0x7f674c150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6751564700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f674c1350a0 0x7f674c150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6751564700 1 -- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f674c0664f0 msgr2=0x7f674c0c2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6751564700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f674c0664f0 0x7f674c0c2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6751564700 1 -- 172.21.15.29:0/3177556273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67480092a0 con 0x7f674c134cc0 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6751d65700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f674c0664f0 0x7f674c0c2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6751564700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f674c134cc0 0x7f674c064280 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f674c06a130 tx=0x7f673c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6741ffb700 1 -- 172.21.15.29:0/3177556273 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c00f7e0 con 0x7f674c134cc0 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6741ffb700 1 -- 172.21.15.29:0/3177556273 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c00f940 con 0x7f674c134cc0 2024-01-24T14:13:45.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6753fc9700 1 -- 172.21.15.29:0/3177556273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c009370 con 0x7f674c134cc0 2024-01-24T14:13:45.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.554+0000 7f6741ffb700 1 -- 172.21.15.29:0/3177556273 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c01c670 con 0x7f674c134cc0 2024-01-24T14:13:45.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.555+0000 7f6753fc9700 1 -- 172.21.15.29:0/3177556273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674c13a7f0 con 0x7f674c134cc0 2024-01-24T14:13:45.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.556+0000 7f6741ffb700 1 -- 172.21.15.29:0/3177556273 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f673c01ca20 con 0x7f674c134cc0 2024-01-24T14:13:45.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.556+0000 7f6753fc9700 1 -- 172.21.15.29:0/3177556273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674c0475e0 con 0x7f674c134cc0 2024-01-24T14:13:45.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.556+0000 7f6741ffb700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f300 0x7f67300617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.557+0000 7f6741ffb700 1 -- 172.21.15.29:0/3177556273 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f673c07c240 con 0x7f674c134cc0 2024-01-24T14:13:45.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.557+0000 7f6751d65700 1 -- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f300 msgr2=0x7f67300617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:45.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.557+0000 7f6751d65700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f300 0x7f67300617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:45.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.561+0000 7f6741ffb700 1 -- 172.21.15.29:0/3177556273 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f673c04ce10 con 0x7f674c134cc0 2024-01-24T14:13:45.601 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:45.600+0000 7f71eab8f680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:13:45.707 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:45.707+0000 7f71eab8f680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:13:45.742 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:45.742+0000 7f4e80e9c680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:13:45.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.758+0000 7f6751d65700 1 -- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f300 msgr2=0x7f67300617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:45.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.758+0000 7f6751d65700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f300 0x7f67300617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:13:45.821 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:45.821+0000 7f4e80e9c680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:13:45.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.822+0000 7f6753fc9700 1 -- 172.21.15.29:0/3177556273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f674c06bda0 con 0x7f674c134cc0 2024-01-24T14:13:45.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.823+0000 7f6741ffb700 1 -- 172.21.15.29:0/3177556273 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f673c050430 con 0x7f674c134cc0 2024-01-24T14:13:45.824 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:45.824 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1149323432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1149323432},{"type":"v1","addr":"172.21.15.163:6835","nonce":1149323432}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6157,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2941255566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":2941255566},{"type":"v1","addr":"172.21.15.29:6835","nonce":2941255566}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6163,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2235282008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2235282008},{"type":"v1","addr":"172.21.15.163:6837","nonce":2235282008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:13:27.270903+0000","modified":"2024-01-24T14:13:30.278839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6366":{"gid":6366,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":4,"addr":"172.21.15.29:6833/3957899438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3957899438},{"type":"v1","addr":"172.21.15.29:6833","nonce":3957899438}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-24T14:13:45.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.827+0000 7f6753fc9700 1 -- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f300 msgr2=0x7f67300617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:45.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.827+0000 7f6753fc9700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f300 0x7f67300617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:45.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.827+0000 7f6753fc9700 1 -- 172.21.15.29:0/3177556273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f674c134cc0 msgr2=0x7f674c064280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.827+0000 7f6753fc9700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f674c134cc0 0x7f674c064280 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f674c06a130 tx=0x7f673c017000).stop 2024-01-24T14:13:45.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.827+0000 7f6753fc9700 1 -- 172.21.15.29:0/3177556273 shutdown_connections 2024-01-24T14:13:45.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.827+0000 7f6753fc9700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f674c1350a0 0x7f674c150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.827+0000 7f6753fc9700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f674c0664f0 0x7f674c0c2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.827+0000 7f6753fc9700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f300 0x7f67300617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:45.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.827+0000 7f6753fc9700 1 --2- 172.21.15.29:0/3177556273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f674c134cc0 0x7f674c064280 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.827+0000 7f6753fc9700 1 -- 172.21.15.29:0/3177556273 >> 172.21.15.29:0/3177556273 conn(0x7f674c0c8bb0 msgr2=0x7f674c0abb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:45.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.827+0000 7f6753fc9700 1 -- 172.21.15.29:0/3177556273 shutdown_connections 2024-01-24T14:13:45.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.827+0000 7f6753fc9700 1 -- 172.21.15.29:0/3177556273 wait complete. 2024-01-24T14:13:45.828 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 35 2024-01-24T14:13:45.846 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:13:45.919 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:45.918+0000 7f4e80e9c680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:13:45.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.978+0000 7ff6ad2cf700 1 Processor -- start 2024-01-24T14:13:45.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.978+0000 7ff6ad2cf700 1 -- start start 2024-01-24T14:13:45.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6ad2cf700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6a8150cb0 0x7ff6a8069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6ad2cf700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff6a8065e50 0x7ff6a8154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6ad2cf700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a806a3a0 con 0x7ff6a8154e40 2024-01-24T14:13:45.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6ad2cf700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a80c2c70 con 0x7ff6a8150cb0 2024-01-24T14:13:45.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6ad2cf700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a8155290 con 0x7ff6a8065e50 2024-01-24T14:13:45.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6a6ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6a8150cb0 0x7ff6a8069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6a6ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6a8150cb0 0x7ff6a8069240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35220/0 (socket says 172.21.15.29:35220) 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6a6ffd700 1 -- 172.21.15.29:0/709575437 learned_addr learned my addr 172.21.15.29:0/709575437 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6a77fe700 1 --1- 172.21.15.29:0/709575437 >> v1:172.21.15.29:6789/0 conn(0x7ff6a8154e40 0x7ff6a8151090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46658/0 (socket says 172.21.15.29:46658) 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6a67fc700 1 --2- 172.21.15.29:0/709575437 >> v2:172.21.15.163:3300/0 conn(0x7ff6a8065e50 0x7ff6a8154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6a6ffd700 1 -- 172.21.15.29:0/709575437 >> v2:172.21.15.163:3300/0 conn(0x7ff6a8065e50 msgr2=0x7ff6a8154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6a6ffd700 1 --2- 172.21.15.29:0/709575437 >> v2:172.21.15.163:3300/0 conn(0x7ff6a8065e50 0x7ff6a8154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6a6ffd700 1 -- 172.21.15.29:0/709575437 >> v1:172.21.15.29:6789/0 conn(0x7ff6a8154e40 legacy=0x7ff6a8151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6a6ffd700 1 -- 172.21.15.29:0/709575437 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a8069820 con 0x7ff6a8150cb0 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.979+0000 7ff6a5ffb700 1 -- 172.21.15.29:0/709575437 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757870246 0 0) 0x7ff6a806a3a0 con 0x7ff6a8154e40 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.980+0000 7ff6a6ffd700 1 --2- 172.21.15.29:0/709575437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6a8150cb0 0x7ff6a8069240 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7ff6a8155a50 tx=0x7ff6980090f0).ready entity=mon.0 client_cookie=60887b4cbe8d74e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.980+0000 7ff6a5ffb700 1 -- 172.21.15.29:0/709575437 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800b520 con 0x7ff6a8150cb0 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.980+0000 7ff6a5ffb700 1 -- 172.21.15.29:0/709575437 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69800b680 con 0x7ff6a8150cb0 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.980+0000 7ff6a5ffb700 1 -- 172.21.15.29:0/709575437 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800ba00 con 0x7ff6a8150cb0 2024-01-24T14:13:45.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.980+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/709575437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6a8150cb0 msgr2=0x7ff6a8069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.980+0000 7ff6ad2cf700 1 --2- 172.21.15.29:0/709575437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6a8150cb0 0x7ff6a8069240 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7ff6a8155a50 tx=0x7ff6980090f0).stop 2024-01-24T14:13:45.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.980+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/709575437 shutdown_connections 2024-01-24T14:13:45.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.980+0000 7ff6ad2cf700 1 --2- 172.21.15.29:0/709575437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6a8150cb0 0x7ff6a8069240 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.980+0000 7ff6ad2cf700 1 --2- 172.21.15.29:0/709575437 >> v2:172.21.15.163:3300/0 conn(0x7ff6a8065e50 0x7ff6a8154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.980+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/709575437 >> 172.21.15.29:0/709575437 conn(0x7ff6a80c85c0 msgr2=0x7ff6a80c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:45.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.980+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/709575437 shutdown_connections 2024-01-24T14:13:45.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.981+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/709575437 wait complete. 2024-01-24T14:13:45.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.982+0000 7ff6ad2cf700 1 Processor -- start 2024-01-24T14:13:45.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.982+0000 7ff6ad2cf700 1 -- start start 2024-01-24T14:13:45.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6ad2cf700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6a8065e50 0x7ff6a80aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6ad2cf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6a8150cb0 0x7ff6a80aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6ad2cf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6a8154e40 0x7ff6a80ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6ad2cf700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a80c1360 con 0x7ff6a8150cb0 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6ad2cf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a80ab9c0 con 0x7ff6a8154e40 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6ad2cf700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a81473d0 con 0x7ff6a8065e50 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6a67fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6a8150cb0 0x7ff6a80aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6a67fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6a8150cb0 0x7ff6a80aafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35234/0 (socket says 172.21.15.29:35234) 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6a67fc700 1 -- 172.21.15.29:0/252526879 learned_addr learned my addr 172.21.15.29:0/252526879 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6a6ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6a8065e50 0x7ff6a80aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6a77fe700 1 --2- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6a8154e40 0x7ff6a80ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6a67fc700 1 -- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6a8065e50 msgr2=0x7ff6a80aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6a67fc700 1 --2- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6a8065e50 0x7ff6a80aaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6a67fc700 1 -- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6a8154e40 msgr2=0x7ff6a80ab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6a67fc700 1 --2- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6a8154e40 0x7ff6a80ab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6a67fc700 1 -- 172.21.15.29:0/252526879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698009370 con 0x7ff6a8150cb0 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff6a67fc700 1 --2- 172.21.15.29:0/252526879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6a8150cb0 0x7ff6a80aafe0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7ff69c00e950 tx=0x7ff69c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:45.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.984+0000 7ff68effd700 1 -- 172.21.15.29:0/252526879 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c012600 con 0x7ff6a8150cb0 2024-01-24T14:13:45.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.985+0000 7ff68effd700 1 -- 172.21.15.29:0/252526879 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c012760 con 0x7ff6a8150cb0 2024-01-24T14:13:45.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.985+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/252526879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a81475c0 con 0x7ff6a8150cb0 2024-01-24T14:13:45.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.985+0000 7ff68effd700 1 -- 172.21.15.29:0/252526879 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c012ae0 con 0x7ff6a8150cb0 2024-01-24T14:13:45.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.985+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/252526879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a8147b80 con 0x7ff6a8150cb0 2024-01-24T14:13:45.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.986+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/252526879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a8066a20 con 0x7ff6a8150cb0 2024-01-24T14:13:45.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.986+0000 7ff68effd700 1 -- 172.21.15.29:0/252526879 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7ff69c01b400 con 0x7ff6a8150cb0 2024-01-24T14:13:45.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.987+0000 7ff68effd700 1 --2- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff69405f2d0 0x7ff694061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:45.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.987+0000 7ff6a6ffd700 1 -- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff69405f2d0 msgr2=0x7ff694061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:13:45.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.987+0000 7ff68effd700 1 -- 172.21.15.29:0/252526879 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7400+0+0 (secure 0 0 0) 0x7ff69c07ae60 con 0x7ff6a8150cb0 2024-01-24T14:13:45.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.987+0000 7ff6a6ffd700 1 --2- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff69405f2d0 0x7ff694061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:13:45.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:45.992+0000 7ff68effd700 1 -- 172.21.15.29:0/252526879 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff69c04ba30 con 0x7ff6a8150cb0 2024-01-24T14:13:46.001 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:46.000+0000 7f71eab8f680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:13:46.072 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:46.072+0000 7f71eab8f680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:13:46.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.094+0000 7ff68effd700 1 -- 172.21.15.29:0/252526879 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 29) v1 ==== 75270+0+0 (secure 0 0 0) 0x7ff69c078530 con 0x7ff6a8150cb0 2024-01-24T14:13:46.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.095+0000 7ff68effd700 1 -- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff69405f2d0 msgr2=0x7ff694061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:13:46.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.095+0000 7ff68effd700 1 --2- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff69405f2d0 0x7ff694061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:46.097 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:46.097+0000 7f4e80e9c680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:13:46.155 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:46.154+0000 7f7166915700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:46.155 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:46.154+0000 7f7166915700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:46.155 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:46.154+0000 7f7166915700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:46.155 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:46.154+0000 7f7166915700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:46.155 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:46.154+0000 7f7166915700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:46.155 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:46.154+0000 7f716210c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:46.156 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:46.154+0000 7f716210c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:46.156 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:46.154+0000 7f716210c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:46.156 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:46.154+0000 7f716210c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:46.156 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:13:46.154+0000 7f716210c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:13:46.182 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:46.182+0000 7f4e80e9c680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:13:46.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.184+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/252526879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6a80475e0 con 0x7ff6a8150cb0 2024-01-24T14:13:46.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.185+0000 7ff68effd700 1 -- 172.21.15.29:0/252526879 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+18860 (secure 0 0 0) 0x7ff69c016090 con 0x7ff6a8150cb0 2024-01-24T14:13:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:46.186 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":92,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:13:46.087184+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:13:25.288458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-24T14:13:26.243223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3770779612":"2024-01-25T14:13:46.087140+0000","172.21.15.163:0/1767643060":"2024-01-25T14:13:46.087140+0000","172.21.15.163:0/3828159917":"2024-01-25T14:13:46.087140+0000","172.21.15.163:0/1388374675":"2024-01-25T14:13:46.087140+0000","172.21.15.163:0/200616850":"2024-01-25T14:13:46.087140+0000","172.21.15.163:0/1701533468":"2024-01-25T14:13:39.554196+0000","172.21.15.163:0/1616535923":"2024-01-25T14:13:39.554196+0000","172.21.15.163:0/1057348791":"2024-01-25T14:13:39.554196+0000","172.21.15.163:0/2629162943":"2024-01-25T14:13:39.554196+0000","172.21.15.163:6832/43241":"2024-01-25T14:13:39.554196+0000","172.21.15.163:6833/43241":"2024-01-25T14:13:39.554196+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:13:46.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.187+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/252526879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6a8150cb0 msgr2=0x7ff6a80aafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:46.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.187+0000 7ff6ad2cf700 1 --2- 172.21.15.29:0/252526879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6a8150cb0 0x7ff6a80aafe0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7ff69c00e950 tx=0x7ff69c009060).stop 2024-01-24T14:13:46.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.187+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/252526879 shutdown_connections 2024-01-24T14:13:46.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.187+0000 7ff6ad2cf700 1 --2- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6a8154e40 0x7ff6a80ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:46.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.187+0000 7ff6ad2cf700 1 --2- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6a8065e50 0x7ff6a80aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:46.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.187+0000 7ff6ad2cf700 1 --2- 172.21.15.29:0/252526879 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff69405f2d0 0x7ff694061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:13:46.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.187+0000 7ff6ad2cf700 1 --2- 172.21.15.29:0/252526879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6a8150cb0 0x7ff6a80aafe0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:46.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.187+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/252526879 >> 172.21.15.29:0/252526879 conn(0x7ff6a80c85c0 msgr2=0x7ff6a80bcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:46.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.188+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/252526879 shutdown_connections 2024-01-24T14:13:46.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.188+0000 7ff6ad2cf700 1 -- 172.21.15.29:0/252526879 wait complete. 2024-01-24T14:13:46.202 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-24T14:13:46.325 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:46.324+0000 7f4e80e9c680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:13:46.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.334+0000 7f43f80fa700 1 Processor -- start 2024-01-24T14:13:46.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.334+0000 7f43f80fa700 1 -- start start 2024-01-24T14:13:46.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f80fa700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f43f00a9d20 0x7f43f00aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:46.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f80fa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00bfd00 0x7f43f00c0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:46.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f80fa700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f0064e90 con 0x7f43f0154240 2024-01-24T14:13:46.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f80fa700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f01550e0 con 0x7f43f00a9d20 2024-01-24T14:13:46.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f80fa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f00aa670 con 0x7f43f00bfd00 2024-01-24T14:13:46.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f5695700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f43f00a9d20 0x7f43f00aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:46.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f5695700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f43f00a9d20 0x7f43f00aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35250/0 (socket says 172.21.15.29:35250) 2024-01-24T14:13:46.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f5695700 1 -- 172.21.15.29:0/908443064 learned_addr learned my addr 172.21.15.29:0/908443064 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:46.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f6697700 1 --2- 172.21.15.29:0/908443064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00bfd00 0x7f43f00c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:46.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f5695700 1 -- 172.21.15.29:0/908443064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00bfd00 msgr2=0x7f43f00c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:46.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f5695700 1 --2- 172.21.15.29:0/908443064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00bfd00 0x7f43f00c0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:46.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f5695700 1 -- 172.21.15.29:0/908443064 >> v1:172.21.15.163:6789/0 conn(0x7f43f0154240 legacy=0x7f43f0064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:46.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f5695700 1 -- 172.21.15.29:0/908443064 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f00650d0 con 0x7f43f00a9d20 2024-01-24T14:13:46.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f5695700 1 --2- 172.21.15.29:0/908443064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f00a9d20 0x7f43f00aa180 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f43e0006fd0 tx=0x7f43e00090f0).ready entity=mon.0 client_cookie=a08a35b92173e457 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:46.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.336+0000 7f43f4e94700 1 -- 172.21.15.29:0/908443064 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000b580 con 0x7f43f00a9d20 2024-01-24T14:13:46.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.337+0000 7f43f4e94700 1 -- 172.21.15.29:0/908443064 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e000b6e0 con 0x7f43f00a9d20 2024-01-24T14:13:46.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.337+0000 7f43f4e94700 1 -- 172.21.15.29:0/908443064 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000ba60 con 0x7f43f00a9d20 2024-01-24T14:13:46.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.337+0000 7f43f80fa700 1 -- 172.21.15.29:0/908443064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f00a9d20 msgr2=0x7f43f00aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:46.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.337+0000 7f43f80fa700 1 --2- 172.21.15.29:0/908443064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f00a9d20 0x7f43f00aa180 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f43e0006fd0 tx=0x7f43e00090f0).stop 2024-01-24T14:13:46.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.337+0000 7f43f80fa700 1 -- 172.21.15.29:0/908443064 shutdown_connections 2024-01-24T14:13:46.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.337+0000 7f43f80fa700 1 --2- 172.21.15.29:0/908443064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00bfd00 0x7f43f00c0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:46.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.337+0000 7f43f80fa700 1 --2- 172.21.15.29:0/908443064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f00a9d20 0x7f43f00aa180 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:46.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.337+0000 7f43f80fa700 1 -- 172.21.15.29:0/908443064 >> 172.21.15.29:0/908443064 conn(0x7f43f00c8310 msgr2=0x7f43f005e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:46.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.337+0000 7f43f80fa700 1 -- 172.21.15.29:0/908443064 shutdown_connections 2024-01-24T14:13:46.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.337+0000 7f43f80fa700 1 -- 172.21.15.29:0/908443064 wait complete. 2024-01-24T14:13:46.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.339+0000 7f43f80fa700 1 Processor -- start 2024-01-24T14:13:46.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.339+0000 7f43f80fa700 1 -- start start 2024-01-24T14:13:46.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.339+0000 7f43f80fa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00a9d20 0x7f43f00ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:46.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.339+0000 7f43f80fa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f00bfd00 0x7f43f0145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:46.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.339+0000 7f43f80fa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f0154240 0x7f43f014ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:46.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.339+0000 7f43f80fa700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f0164950 con 0x7f43f0154240 2024-01-24T14:13:46.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.339+0000 7f43f80fa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f014d950 con 0x7f43f00bfd00 2024-01-24T14:13:46.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.339+0000 7f43f80fa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f00c3040 con 0x7f43f00a9d20 2024-01-24T14:13:46.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f6697700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f0154240 0x7f43f014ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:46.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f6697700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f0154240 0x7f43f014ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35260/0 (socket says 172.21.15.29:35260) 2024-01-24T14:13:46.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f6697700 1 -- 172.21.15.29:0/1675737195 learned_addr learned my addr 172.21.15.29:0/1675737195 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:46.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f5695700 1 --2- 172.21.15.29:0/1675737195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f00bfd00 0x7f43f0145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f5e96700 1 --2- 172.21.15.29:0/1675737195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00a9d20 0x7f43f00ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f6697700 1 -- 172.21.15.29:0/1675737195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00a9d20 msgr2=0x7f43f00ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f6697700 1 --2- 172.21.15.29:0/1675737195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00a9d20 0x7f43f00ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f6697700 1 -- 172.21.15.29:0/1675737195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f00bfd00 msgr2=0x7f43f0145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f6697700 1 --2- 172.21.15.29:0/1675737195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f00bfd00 0x7f43f0145850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f6697700 1 -- 172.21.15.29:0/1675737195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e0009370 con 0x7f43f0154240 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f5e96700 1 --2- 172.21.15.29:0/1675737195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00a9d20 0x7f43f00ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f6697700 1 --2- 172.21.15.29:0/1675737195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f0154240 0x7f43f014ed50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f43e8009e90 tx=0x7f43e800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43e5ffb700 1 -- 172.21.15.29:0/1675737195 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e800caa0 con 0x7f43f0154240 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43e5ffb700 1 -- 172.21.15.29:0/1675737195 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e800f040 con 0x7f43f0154240 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43f80fa700 1 -- 172.21.15.29:0/1675737195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f00afc20 con 0x7f43f0154240 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.340+0000 7f43e5ffb700 1 -- 172.21.15.29:0/1675737195 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e800ce10 con 0x7f43f0154240 2024-01-24T14:13:46.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.341+0000 7f43f80fa700 1 -- 172.21.15.29:0/1675737195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f00ae630 con 0x7f43f0154240 2024-01-24T14:13:46.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.342+0000 7f43f80fa700 1 -- 172.21.15.29:0/1675737195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f00b0140 con 0x7f43f0154240 2024-01-24T14:13:46.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.342+0000 7f43e5ffb700 1 -- 172.21.15.29:0/1675737195 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 29) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f43e8013610 con 0x7f43f0154240 2024-01-24T14:13:46.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.343+0000 7f43e5ffb700 1 -- 172.21.15.29:0/1675737195 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 7615+0+0 (secure 0 0 0) 0x7f43e807aed0 con 0x7f43f0154240 2024-01-24T14:13:46.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.348+0000 7f43e5ffb700 1 -- 172.21.15.29:0/1675737195 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f43f00b0140 con 0x7f43f0154240 2024-01-24T14:13:46.454 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:46.453+0000 7f4e80e9c680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:13:46.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:46.576+0000 7f43f80fa700 1 -- 172.21.15.29:0/1675737195 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f43f00c2e10 con 0x7f43f0154240 2024-01-24T14:13:46.792 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:46.792+0000 7f4e80e9c680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:13:46.874 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:13:46.874+0000 7f4e80e9c680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:13:47.089 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-01-24T14:13:47.090+0000 7fe09bfc3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-24T14:13:47.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.094+0000 7f43e5ffb700 1 -- 172.21.15.29:0/1675737195 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7f43f00c2e10 con 0x7f43f0154240 2024-01-24T14:13:47.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.097+0000 7f43f80fa700 1 -- 172.21.15.29:0/1675737195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f0154240 msgr2=0x7f43f014ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:47.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.097+0000 7f43f80fa700 1 --2- 172.21.15.29:0/1675737195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f0154240 0x7f43f014ed50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f43e8009e90 tx=0x7f43e800b040).stop 2024-01-24T14:13:47.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.097+0000 7f43f80fa700 1 -- 172.21.15.29:0/1675737195 shutdown_connections 2024-01-24T14:13:47.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.097+0000 7f43f80fa700 1 --2- 172.21.15.29:0/1675737195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43f0154240 0x7f43f014ed50 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:47.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.097+0000 7f43f80fa700 1 --2- 172.21.15.29:0/1675737195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43f00a9d20 0x7f43f00ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:47.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.097+0000 7f43f80fa700 1 --2- 172.21.15.29:0/1675737195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f00bfd00 0x7f43f0145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:47.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.097+0000 7f43f80fa700 1 -- 172.21.15.29:0/1675737195 >> 172.21.15.29:0/1675737195 conn(0x7f43f00c8310 msgr2=0x7f43f00c8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:47.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.097+0000 7f43f80fa700 1 -- 172.21.15.29:0/1675737195 shutdown_connections 2024-01-24T14:13:47.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.097+0000 7f43f80fa700 1 -- 172.21.15.29:0/1675737195 wait complete. 2024-01-24T14:13:47.097 INFO:tasks.ceph.mds.a.smithi029.stderr: -2646> 2024-01-24T14:13:31.827+0000 7fef47954700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T14:13:47.100 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-24T14:13:47.117 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-24T14:13:47.126 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T14:13:47.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.252+0000 7f5050b25700 1 Processor -- start 2024-01-24T14:13:47.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.252+0000 7f5050b25700 1 -- start start 2024-01-24T14:13:47.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f5050b25700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f504c152f20 0x7f504c0be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:47.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f5050b25700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f504c065420 0x7f504c065890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:47.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f5050b25700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c065d80 con 0x7f504c0663b0 2024-01-24T14:13:47.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f5050b25700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504c0bca30 con 0x7f504c152f20 2024-01-24T14:13:47.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f5050b25700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504c069690 con 0x7f504c065420 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f5049d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f504c152f20 0x7f504c0be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f504a59c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f504c0663b0 0x7f504c069b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46684/0 (socket says 172.21.15.29:46684) 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f504a59c700 1 -- 172.21.15.29:0/3786827330 learned_addr learned my addr 172.21.15.29:0/3786827330 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f504ad9d700 1 --2- 172.21.15.29:0/3786827330 >> v2:172.21.15.163:3300/0 conn(0x7f504c065420 0x7f504c065890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f5049d9b700 1 -- 172.21.15.29:0/3786827330 >> v2:172.21.15.163:3300/0 conn(0x7f504c065420 msgr2=0x7f504c065890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f5049d9b700 1 --2- 172.21.15.29:0/3786827330 >> v2:172.21.15.163:3300/0 conn(0x7f504c065420 0x7f504c065890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f5049d9b700 1 -- 172.21.15.29:0/3786827330 >> v1:172.21.15.29:6789/0 conn(0x7f504c0663b0 legacy=0x7f504c069b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f504959a700 1 -- 172.21.15.29:0/3786827330 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246420579 0 0) 0x7f503c001110 con 0x7f504c0663b0 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f5049d9b700 1 -- 172.21.15.29:0/3786827330 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c0bcc40 con 0x7f504c152f20 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f504ad9d700 1 --2- 172.21.15.29:0/3786827330 >> v2:172.21.15.163:3300/0 conn(0x7f504c065420 0x7f504c065890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.253+0000 7f5049d9b700 1 --2- 172.21.15.29:0/3786827330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f504c152f20 0x7f504c0be4b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f504c05ee30 tx=0x7f5040009060).ready entity=mon.0 client_cookie=781daa8de6dbba80 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.254+0000 7f504959a700 1 -- 172.21.15.29:0/3786827330 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504000b580 con 0x7f504c152f20 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.254+0000 7f504959a700 1 -- 172.21.15.29:0/3786827330 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504000b6e0 con 0x7f504c152f20 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.254+0000 7f504959a700 1 -- 172.21.15.29:0/3786827330 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504000ba60 con 0x7f504c152f20 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.254+0000 7f5050b25700 1 -- 172.21.15.29:0/3786827330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f504c152f20 msgr2=0x7f504c0be4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:47.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.254+0000 7f5050b25700 1 --2- 172.21.15.29:0/3786827330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f504c152f20 0x7f504c0be4b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f504c05ee30 tx=0x7f5040009060).stop 2024-01-24T14:13:47.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.254+0000 7f5050b25700 1 -- 172.21.15.29:0/3786827330 shutdown_connections 2024-01-24T14:13:47.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.254+0000 7f5050b25700 1 --2- 172.21.15.29:0/3786827330 >> v2:172.21.15.163:3300/0 conn(0x7f504c065420 0x7f504c065890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:47.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.254+0000 7f5050b25700 1 --2- 172.21.15.29:0/3786827330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f504c152f20 0x7f504c0be4b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:47.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.254+0000 7f5050b25700 1 -- 172.21.15.29:0/3786827330 >> 172.21.15.29:0/3786827330 conn(0x7f504c064390 msgr2=0x7f504c0c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:47.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.254+0000 7f5050b25700 1 -- 172.21.15.29:0/3786827330 shutdown_connections 2024-01-24T14:13:47.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.254+0000 7f5050b25700 1 -- 172.21.15.29:0/3786827330 wait complete. 2024-01-24T14:13:47.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.256+0000 7f5050b25700 1 Processor -- start 2024-01-24T14:13:47.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.256+0000 7f5050b25700 1 -- start start 2024-01-24T14:13:47.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.256+0000 7f5050b25700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f504c065420 0x7f504c135640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:47.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.256+0000 7f5050b25700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0663b0 0x7f504c136d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:47.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.256+0000 7f5050b25700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f504c152f20 0x7f504c1371f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:47.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.256+0000 7f5050b25700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c1376e0 con 0x7f504c152f20 2024-01-24T14:13:47.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.256+0000 7f5050b25700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c13b840 con 0x7f504c065420 2024-01-24T14:13:47.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.256+0000 7f5050b25700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0a81e0 con 0x7f504c0663b0 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.256+0000 7f504ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f504c152f20 0x7f504c1371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.256+0000 7f504ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f504c152f20 0x7f504c1371f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35302/0 (socket says 172.21.15.29:35302) 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f504ad9d700 1 -- 172.21.15.29:0/961058170 learned_addr learned my addr 172.21.15.29:0/961058170 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f5049d9b700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0663b0 0x7f504c136d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f504a59c700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f504c065420 0x7f504c135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f504ad9d700 1 -- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0663b0 msgr2=0x7f504c136d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f504ad9d700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0663b0 0x7f504c136d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f504ad9d700 1 -- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f504c065420 msgr2=0x7f504c135640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f504ad9d700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f504c065420 0x7f504c135640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f504ad9d700 1 -- 172.21.15.29:0/961058170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400092a0 con 0x7f504c152f20 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f504a59c700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f504c065420 0x7f504c135640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f504ad9d700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f504c152f20 0x7f504c1371f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f504400d180 tx=0x7f5044009a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f503a7fc700 1 -- 172.21.15.29:0/961058170 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504400de20 con 0x7f504c152f20 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f503a7fc700 1 -- 172.21.15.29:0/961058170 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504400f040 con 0x7f504c152f20 2024-01-24T14:13:47.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f5050b25700 1 -- 172.21.15.29:0/961058170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504c135b30 con 0x7f504c152f20 2024-01-24T14:13:47.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f503a7fc700 1 -- 172.21.15.29:0/961058170 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044013600 con 0x7f504c152f20 2024-01-24T14:13:47.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.257+0000 7f5050b25700 1 -- 172.21.15.29:0/961058170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504c0a7b90 con 0x7f504c152f20 2024-01-24T14:13:47.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.258+0000 7f5050b25700 1 -- 172.21.15.29:0/961058170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504c0a8350 con 0x7f504c152f20 2024-01-24T14:13:47.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.259+0000 7f503a7fc700 1 -- 172.21.15.29:0/961058170 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 30) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f50440139f0 con 0x7f504c152f20 2024-01-24T14:13:47.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.259+0000 7f503a7fc700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f502c05f000 0x7f502c0614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:47.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.259+0000 7f503a7fc700 1 -- 172.21.15.29:0/961058170 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 7701+0+0 (secure 0 0 0) 0x7f504407ad80 con 0x7f504c152f20 2024-01-24T14:13:47.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.259+0000 7f504a59c700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f502c05f000 0x7f502c0614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:47.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.260+0000 7f504a59c700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f502c05f000 0x7f502c0614c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f503c001740 tx=0x7f503c009040).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:47.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.264+0000 7f503a7fc700 1 -- 172.21.15.29:0/961058170 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f504404bb90 con 0x7f504c152f20 2024-01-24T14:13:47.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:47.491+0000 7f5050b25700 1 -- 172.21.15.29:0/961058170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f504c0475e0 con 0x7f504c152f20 2024-01-24T14:13:48.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.107+0000 7f503a7fc700 1 -- 172.21.15.29:0/961058170 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f504404f1b0 con 0x7f504c152f20 2024-01-24T14:13:48.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.109+0000 7f5050b25700 1 -- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f502c05f000 msgr2=0x7f502c0614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:48.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.109+0000 7f5050b25700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f502c05f000 0x7f502c0614c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f503c001740 tx=0x7f503c009040).stop 2024-01-24T14:13:48.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.109+0000 7f5050b25700 1 -- 172.21.15.29:0/961058170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f504c152f20 msgr2=0x7f504c1371f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:48.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.109+0000 7f5050b25700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f504c152f20 0x7f504c1371f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f504400d180 tx=0x7f5044009a50).stop 2024-01-24T14:13:48.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.110+0000 7f5050b25700 1 -- 172.21.15.29:0/961058170 shutdown_connections 2024-01-24T14:13:48.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.110+0000 7f5050b25700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f502c05f000 0x7f502c0614c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:48.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.110+0000 7f5050b25700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f504c152f20 0x7f504c1371f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:48.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.110+0000 7f5050b25700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f504c065420 0x7f504c135640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:48.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.110+0000 7f5050b25700 1 --2- 172.21.15.29:0/961058170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0663b0 0x7f504c136d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:48.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.110+0000 7f5050b25700 1 -- 172.21.15.29:0/961058170 >> 172.21.15.29:0/961058170 conn(0x7f504c064390 msgr2=0x7f504c0c6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:48.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.110+0000 7f5050b25700 1 -- 172.21.15.29:0/961058170 shutdown_connections 2024-01-24T14:13:48.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.110+0000 7f5050b25700 1 -- 172.21.15.29:0/961058170 wait complete. 2024-01-24T14:13:48.125 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-24T14:13:48.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.307+0000 7f846c19b700 1 Processor -- start 2024-01-24T14:13:48.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.308+0000 7f846c19b700 1 -- start start 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.308+0000 7f846c19b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84640c3d80 0x7f8464151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.308+0000 7f846c19b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8464067140 0x7f84640c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.308+0000 7f846c19b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8464069260 0x7f8464069710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.308+0000 7f846c19b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84640bca50 con 0x7f8464067140 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.308+0000 7f846c19b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84641523a0 con 0x7f8464069260 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.308+0000 7f846c19b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8464069c00 con 0x7f84640c3d80 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8469736700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8464067140 0x7f84640c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8469736700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8464067140 0x7f84640c20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35306/0 (socket says 172.21.15.29:35306) 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8469736700 1 -- 172.21.15.29:0/3171339895 learned_addr learned my addr 172.21.15.29:0/3171339895 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f846a738700 1 --2- 172.21.15.29:0/3171339895 >> v2:172.21.15.163:3300/0 conn(0x7f8464069260 0x7f8464069710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8469f37700 1 --2- 172.21.15.29:0/3171339895 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84640c3d80 0x7f8464151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8469736700 1 -- 172.21.15.29:0/3171339895 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84640c3d80 msgr2=0x7f8464151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:48.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8469736700 1 --2- 172.21.15.29:0/3171339895 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84640c3d80 0x7f8464151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8469736700 1 -- 172.21.15.29:0/3171339895 >> v2:172.21.15.163:3300/0 conn(0x7f8464069260 msgr2=0x7f8464069710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8469736700 1 --2- 172.21.15.29:0/3171339895 >> v2:172.21.15.163:3300/0 conn(0x7f8464069260 0x7f8464069710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8469736700 1 -- 172.21.15.29:0/3171339895 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84640bd2b0 con 0x7f8464067140 2024-01-24T14:13:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8469736700 1 --2- 172.21.15.29:0/3171339895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8464067140 0x7f84640c20f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f846406a3b0 tx=0x7f84580090f0).ready entity=mon.0 client_cookie=c595663f1085265f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8468f35700 1 -- 172.21.15.29:0/3171339895 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845800f510 con 0x7f8464067140 2024-01-24T14:13:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8468f35700 1 -- 172.21.15.29:0/3171339895 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845800f670 con 0x7f8464067140 2024-01-24T14:13:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.309+0000 7f8468f35700 1 -- 172.21.15.29:0/3171339895 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845800f9f0 con 0x7f8464067140 2024-01-24T14:13:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.310+0000 7f846c19b700 1 -- 172.21.15.29:0/3171339895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8464067140 msgr2=0x7f84640c20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.310+0000 7f846c19b700 1 --2- 172.21.15.29:0/3171339895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8464067140 0x7f84640c20f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f846406a3b0 tx=0x7f84580090f0).stop 2024-01-24T14:13:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.310+0000 7f846c19b700 1 -- 172.21.15.29:0/3171339895 shutdown_connections 2024-01-24T14:13:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.310+0000 7f846c19b700 1 --2- 172.21.15.29:0/3171339895 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84640c3d80 0x7f8464151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:48.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.310+0000 7f846c19b700 1 --2- 172.21.15.29:0/3171339895 >> v2:172.21.15.163:3300/0 conn(0x7f8464069260 0x7f8464069710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:48.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.310+0000 7f846c19b700 1 --2- 172.21.15.29:0/3171339895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8464067140 0x7f84640c20f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:48.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.310+0000 7f846c19b700 1 -- 172.21.15.29:0/3171339895 >> 172.21.15.29:0/3171339895 conn(0x7f84640c7af0 msgr2=0x7f84640c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:48.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.310+0000 7f846c19b700 1 -- 172.21.15.29:0/3171339895 shutdown_connections 2024-01-24T14:13:48.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.310+0000 7f846c19b700 1 -- 172.21.15.29:0/3171339895 wait complete. 2024-01-24T14:13:48.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.312+0000 7f846c19b700 1 Processor -- start 2024-01-24T14:13:48.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.312+0000 7f846c19b700 1 -- start start 2024-01-24T14:13:48.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f846c19b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8464067140 0x7f84641657e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:48.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f846c19b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8464069260 0x7f84640acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:48.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f846c19b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84640c3d80 0x7f8464162280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:48.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f846c19b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846413d380 con 0x7f8464067140 2024-01-24T14:13:48.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f846c19b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8464166ab0 con 0x7f84640c3d80 2024-01-24T14:13:48.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f846c19b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84640ad480 con 0x7f8464069260 2024-01-24T14:13:48.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f8469f37700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8464067140 0x7f84641657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:48.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f8469f37700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8464067140 0x7f84641657e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35318/0 (socket says 172.21.15.29:35318) 2024-01-24T14:13:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f8469f37700 1 -- 172.21.15.29:0/277060949 learned_addr learned my addr 172.21.15.29:0/277060949 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f8469736700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8464069260 0x7f84640acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f846a738700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84640c3d80 0x7f8464162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f8469f37700 1 -- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8464069260 msgr2=0x7f84640acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f8469f37700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8464069260 0x7f84640acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f8469f37700 1 -- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84640c3d80 msgr2=0x7f8464162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f8469f37700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84640c3d80 0x7f8464162280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f8469f37700 1 -- 172.21.15.29:0/277060949 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8458009370 con 0x7f8464067140 2024-01-24T14:13:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f846a738700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84640c3d80 0x7f8464162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.313+0000 7f8469f37700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8464067140 0x7f84641657e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f846000e930 tx=0x7f8460009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.314+0000 7f8455ffb700 1 -- 172.21.15.29:0/277060949 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846000c640 con 0x7f8464067140 2024-01-24T14:13:48.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.314+0000 7f8455ffb700 1 -- 172.21.15.29:0/277060949 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846000c7a0 con 0x7f8464067140 2024-01-24T14:13:48.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.314+0000 7f846c19b700 1 -- 172.21.15.29:0/277060949 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8464165ea0 con 0x7f8464067140 2024-01-24T14:13:48.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.314+0000 7f8455ffb700 1 -- 172.21.15.29:0/277060949 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846000cb20 con 0x7f8464067140 2024-01-24T14:13:48.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.314+0000 7f846c19b700 1 -- 172.21.15.29:0/277060949 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8464166460 con 0x7f8464067140 2024-01-24T14:13:48.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.315+0000 7f8455ffb700 1 -- 172.21.15.29:0/277060949 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 30) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f846001b400 con 0x7f8464067140 2024-01-24T14:13:48.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.315+0000 7f846c19b700 1 -- 172.21.15.29:0/277060949 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84640475e0 con 0x7f8464067140 2024-01-24T14:13:48.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.315+0000 7f8455ffb700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f844805f030 0x7f84480614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:48.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.316+0000 7f8455ffb700 1 -- 172.21.15.29:0/277060949 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 7701+0+0 (secure 0 0 0) 0x7f846007aa80 con 0x7f8464067140 2024-01-24T14:13:48.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.316+0000 7f8469736700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f844805f030 0x7f84480614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:48.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.316+0000 7f8469736700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f844805f030 0x7f84480614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f846406a3b0 tx=0x7f8458017000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:48.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.320+0000 7f8455ffb700 1 -- 172.21.15.29:0/277060949 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f846004b890 con 0x7f8464067140 2024-01-24T14:13:48.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:48.500+0000 7f846c19b700 1 -- 172.21.15.29:0/277060949 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84640678c0 con 0x7f8464067140 2024-01-24T14:13:49.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.111+0000 7f8455ffb700 1 -- 172.21.15.29:0/277060949 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8460016020 con 0x7f8464067140 2024-01-24T14:13:49.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.115+0000 7f846c19b700 1 -- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f844805f030 msgr2=0x7f84480614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:49.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.115+0000 7f846c19b700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f844805f030 0x7f84480614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f846406a3b0 tx=0x7f8458017000).stop 2024-01-24T14:13:49.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.115+0000 7f846c19b700 1 -- 172.21.15.29:0/277060949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8464067140 msgr2=0x7f84641657e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:49.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.115+0000 7f846c19b700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8464067140 0x7f84641657e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f846000e930 tx=0x7f8460009060).stop 2024-01-24T14:13:49.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.115+0000 7f846c19b700 1 -- 172.21.15.29:0/277060949 shutdown_connections 2024-01-24T14:13:49.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.115+0000 7f846c19b700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84640c3d80 0x7f8464162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:49.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.115+0000 7f846c19b700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f844805f030 0x7f84480614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:49.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.115+0000 7f846c19b700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8464067140 0x7f84641657e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:49.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.115+0000 7f846c19b700 1 --2- 172.21.15.29:0/277060949 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8464069260 0x7f84640acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.115+0000 7f846c19b700 1 -- 172.21.15.29:0/277060949 >> 172.21.15.29:0/277060949 conn(0x7f84640c7af0 msgr2=0x7f8464155130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.115+0000 7f846c19b700 1 -- 172.21.15.29:0/277060949 shutdown_connections 2024-01-24T14:13:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.115+0000 7f846c19b700 1 -- 172.21.15.29:0/277060949 wait complete. 2024-01-24T14:13:49.116 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-01-24T14:13:49.133 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-24T14:13:49.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.272+0000 7f6a0ee35700 1 Processor -- start 2024-01-24T14:13:49.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.272+0000 7f6a0ee35700 1 -- start start 2024-01-24T14:13:49.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0ee35700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a08154ec0 0x7f6a081552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:49.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0ee35700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6a08063d40 0x7f6a08151350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:49.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0ee35700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a08068540 con 0x7f6a080638c0 2024-01-24T14:13:49.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0ee35700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a080641c0 con 0x7f6a08063d40 2024-01-24T14:13:49.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0ee35700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a080689d0 con 0x7f6a08154ec0 2024-01-24T14:13:49.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0d3d2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6a08063d40 0x7f6a08151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:49.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0d3d2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6a08063d40 0x7f6a08151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35328/0 (socket says 172.21.15.29:35328) 2024-01-24T14:13:49.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0d3d2700 1 -- 172.21.15.29:0/2707373067 learned_addr learned my addr 172.21.15.29:0/2707373067 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:49.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0cbd1700 1 --2- 172.21.15.29:0/2707373067 >> v2:172.21.15.163:3300/0 conn(0x7f6a08154ec0 0x7f6a081552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0d3d2700 1 -- 172.21.15.29:0/2707373067 >> v2:172.21.15.163:3300/0 conn(0x7f6a08154ec0 msgr2=0x7f6a081552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0d3d2700 1 --2- 172.21.15.29:0/2707373067 >> v2:172.21.15.163:3300/0 conn(0x7f6a08154ec0 0x7f6a081552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0d3d2700 1 -- 172.21.15.29:0/2707373067 >> v1:172.21.15.163:6789/0 conn(0x7f6a080638c0 legacy=0x7f6a08066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0d3d2700 1 -- 172.21.15.29:0/2707373067 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a08067c10 con 0x7f6a08063d40 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.273+0000 7f6a0d3d2700 1 --2- 172.21.15.29:0/2707373067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a08063d40 0x7f6a08151350 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f6a0805fad0 tx=0x7f6a00009840).ready entity=mon.0 client_cookie=7a582c3dfc2249a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.274+0000 7f6a077fe700 1 -- 172.21.15.29:0/2707373067 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0000e070 con 0x7f6a08063d40 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.274+0000 7f6a077fe700 1 -- 172.21.15.29:0/2707373067 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0000ba50 con 0x7f6a08063d40 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.274+0000 7f6a077fe700 1 -- 172.21.15.29:0/2707373067 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0000bdd0 con 0x7f6a08063d40 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.274+0000 7f6a0ee35700 1 -- 172.21.15.29:0/2707373067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a08063d40 msgr2=0x7f6a08151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.274+0000 7f6a0ee35700 1 --2- 172.21.15.29:0/2707373067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a08063d40 0x7f6a08151350 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f6a0805fad0 tx=0x7f6a00009840).stop 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.274+0000 7f6a0ee35700 1 -- 172.21.15.29:0/2707373067 shutdown_connections 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.274+0000 7f6a0ee35700 1 --2- 172.21.15.29:0/2707373067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a08063d40 0x7f6a08151350 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.274+0000 7f6a0ee35700 1 --2- 172.21.15.29:0/2707373067 >> v2:172.21.15.163:3300/0 conn(0x7f6a08154ec0 0x7f6a081552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.274+0000 7f6a0ee35700 1 -- 172.21.15.29:0/2707373067 >> 172.21.15.29:0/2707373067 conn(0x7f6a080c8bd0 msgr2=0x7f6a080c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:49.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.274+0000 7f6a0ee35700 1 -- 172.21.15.29:0/2707373067 shutdown_connections 2024-01-24T14:13:49.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.274+0000 7f6a0ee35700 1 -- 172.21.15.29:0/2707373067 wait complete. 2024-01-24T14:13:49.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.276+0000 7f6a0ee35700 1 Processor -- start 2024-01-24T14:13:49.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.276+0000 7f6a0ee35700 1 -- start start 2024-01-24T14:13:49.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.277+0000 7f6a0ee35700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a080638c0 0x7f6a080ab130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:49.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.277+0000 7f6a0ee35700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a08063d40 0x7f6a080a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:49.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.277+0000 7f6a0ee35700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a08154ec0 0x7f6a080aa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:49.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.277+0000 7f6a0ee35700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a08168ef0 con 0x7f6a08154ec0 2024-01-24T14:13:49.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.277+0000 7f6a0ee35700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a080ab620 con 0x7f6a08063d40 2024-01-24T14:13:49.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.277+0000 7f6a0ee35700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a080aeb40 con 0x7f6a080638c0 2024-01-24T14:13:49.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.277+0000 7f6a0d3d2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a08154ec0 0x7f6a080aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:49.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.277+0000 7f6a0d3d2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a08154ec0 0x7f6a080aa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35330/0 (socket says 172.21.15.29:35330) 2024-01-24T14:13:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.277+0000 7f6a0d3d2700 1 -- 172.21.15.29:0/603511068 learned_addr learned my addr 172.21.15.29:0/603511068 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.277+0000 7f6a07fff700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a08063d40 0x7f6a080a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.277+0000 7f6a0cbd1700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a080638c0 0x7f6a080ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.278+0000 7f6a0d3d2700 1 -- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a080638c0 msgr2=0x7f6a080ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.278+0000 7f6a0d3d2700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a080638c0 0x7f6a080ab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.278+0000 7f6a0d3d2700 1 -- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a08063d40 msgr2=0x7f6a080a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.278+0000 7f6a0d3d2700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a08063d40 0x7f6a080a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.278+0000 7f6a0d3d2700 1 -- 172.21.15.29:0/603511068 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a00009ae0 con 0x7f6a08154ec0 2024-01-24T14:13:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.278+0000 7f6a0d3d2700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a08154ec0 0x7f6a080aa000 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f6a0806a130 tx=0x7f6a00009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.278+0000 7f6a04ff9700 1 -- 172.21.15.29:0/603511068 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0000bdd0 con 0x7f6a08154ec0 2024-01-24T14:13:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.278+0000 7f6a04ff9700 1 -- 172.21.15.29:0/603511068 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a000092e0 con 0x7f6a08154ec0 2024-01-24T14:13:49.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.278+0000 7f6a04ff9700 1 -- 172.21.15.29:0/603511068 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0001c630 con 0x7f6a08154ec0 2024-01-24T14:13:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.279+0000 7f6a0ee35700 1 -- 172.21.15.29:0/603511068 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a080a4d70 con 0x7f6a08154ec0 2024-01-24T14:13:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.279+0000 7f6a0ee35700 1 -- 172.21.15.29:0/603511068 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a081685c0 con 0x7f6a08154ec0 2024-01-24T14:13:49.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.280+0000 7f6a0ee35700 1 -- 172.21.15.29:0/603511068 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a080475e0 con 0x7f6a08154ec0 2024-01-24T14:13:49.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.281+0000 7f6a04ff9700 1 -- 172.21.15.29:0/603511068 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6a0001c9e0 con 0x7f6a08154ec0 2024-01-24T14:13:49.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.282+0000 7f6a04ff9700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69ec05f2f0 0x7f69ec0617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:49.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.282+0000 7f6a04ff9700 1 -- 172.21.15.29:0/603511068 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7263+0+0 (secure 0 0 0) 0x7f6a0007c050 con 0x7f6a08154ec0 2024-01-24T14:13:49.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.282+0000 7f6a0cbd1700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69ec05f2f0 0x7f69ec0617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:49.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.282+0000 7f6a0cbd1700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69ec05f2f0 0x7f69ec0617b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f69fc00bbc0 tx=0x7f69fc008040).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:49.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.287+0000 7f6a04ff9700 1 -- 172.21.15.29:0/603511068 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f6a0004cdf0 con 0x7f6a08154ec0 2024-01-24T14:13:49.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:49.472+0000 7f6a0ee35700 1 -- 172.21.15.29:0/603511068 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6a08139dd0 con 0x7f6a08154ec0 2024-01-24T14:13:50.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.121+0000 7f6a04ff9700 1 -- 172.21.15.29:0/603511068 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6a00050410 con 0x7f6a08154ec0 2024-01-24T14:13:50.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.126+0000 7f6a0ee35700 1 -- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69ec05f2f0 msgr2=0x7f69ec0617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.126+0000 7f6a0ee35700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69ec05f2f0 0x7f69ec0617b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f69fc00bbc0 tx=0x7f69fc008040).stop 2024-01-24T14:13:50.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.126+0000 7f6a0ee35700 1 -- 172.21.15.29:0/603511068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a08154ec0 msgr2=0x7f6a080aa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.126+0000 7f6a0ee35700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a08154ec0 0x7f6a080aa000 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f6a0806a130 tx=0x7f6a00009840).stop 2024-01-24T14:13:50.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.126+0000 7f6a0ee35700 1 -- 172.21.15.29:0/603511068 shutdown_connections 2024-01-24T14:13:50.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.126+0000 7f6a0ee35700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69ec05f2f0 0x7f69ec0617b0 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0x7f69fc00bbc0 tx=0x7f69fc008040).stop 2024-01-24T14:13:50.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.126+0000 7f6a0ee35700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a08154ec0 0x7f6a080aa000 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.126+0000 7f6a0ee35700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a080638c0 0x7f6a080ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.126+0000 7f6a0ee35700 1 --2- 172.21.15.29:0/603511068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a08063d40 0x7f6a080a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.126+0000 7f6a0ee35700 1 -- 172.21.15.29:0/603511068 >> 172.21.15.29:0/603511068 conn(0x7f6a080c8bd0 msgr2=0x7f6a08066450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:50.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.127+0000 7f6a0ee35700 1 -- 172.21.15.29:0/603511068 shutdown_connections 2024-01-24T14:13:50.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.127+0000 7f6a0ee35700 1 -- 172.21.15.29:0/603511068 wait complete. 2024-01-24T14:13:50.127 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-01-24T14:13:50.142 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-24T14:13:50.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.283+0000 7f9eb4995700 1 Processor -- start 2024-01-24T14:13:50.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.283+0000 7f9eb4995700 1 -- start start 2024-01-24T14:13:50.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.284+0000 7f9eb4995700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9eb014b140 0x7f9eb0155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.284+0000 7f9eb4995700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9eb00bca30 0x7f9eb00bce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.284+0000 7f9eb4995700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eb00627b0 0x7f9eb0062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.284+0000 7f9eb4995700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb0154cd0 con 0x7f9eb00bca30 2024-01-24T14:13:50.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.284+0000 7f9eb4995700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb0063150 con 0x7f9eb014b140 2024-01-24T14:13:50.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.284+0000 7f9eb4995700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb01546a0 con 0x7f9eb00627b0 2024-01-24T14:13:50.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9eadd9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9eb00bca30 0x7f9eb00bce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9eadd9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9eb00bca30 0x7f9eb00bce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35338/0 (socket says 172.21.15.29:35338) 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9eadd9b700 1 -- 172.21.15.29:0/129494599 learned_addr learned my addr 172.21.15.29:0/129494599 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9eaed9d700 1 --2- 172.21.15.29:0/129494599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eb00627b0 0x7f9eb0062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9eae59c700 1 --2- 172.21.15.29:0/129494599 >> v2:172.21.15.163:3300/0 conn(0x7f9eb014b140 0x7f9eb0155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9eadd9b700 1 -- 172.21.15.29:0/129494599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eb00627b0 msgr2=0x7f9eb0062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9eadd9b700 1 --2- 172.21.15.29:0/129494599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eb00627b0 0x7f9eb0062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9eadd9b700 1 -- 172.21.15.29:0/129494599 >> v2:172.21.15.163:3300/0 conn(0x7f9eb014b140 msgr2=0x7f9eb0155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9eadd9b700 1 --2- 172.21.15.29:0/129494599 >> v2:172.21.15.163:3300/0 conn(0x7f9eb014b140 0x7f9eb0155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9eadd9b700 1 -- 172.21.15.29:0/129494599 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb00c81e0 con 0x7f9eb00bca30 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9eae59c700 1 --2- 172.21.15.29:0/129494599 >> v2:172.21.15.163:3300/0 conn(0x7f9eb014b140 0x7f9eb0155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9eadd9b700 1 --2- 172.21.15.29:0/129494599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00bca30 0x7f9eb00bce90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f9eb006c730 tx=0x7f9ea00090f0).ready entity=mon.0 client_cookie=aac57f5f6d476c77 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9ead59a700 1 -- 172.21.15.29:0/129494599 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000b600 con 0x7f9eb00bca30 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9ead59a700 1 -- 172.21.15.29:0/129494599 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea000b760 con 0x7f9eb00bca30 2024-01-24T14:13:50.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.285+0000 7f9ead59a700 1 -- 172.21.15.29:0/129494599 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000bae0 con 0x7f9eb00bca30 2024-01-24T14:13:50.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.286+0000 7f9eb4995700 1 -- 172.21.15.29:0/129494599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00bca30 msgr2=0x7f9eb00bce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.286+0000 7f9eb4995700 1 --2- 172.21.15.29:0/129494599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00bca30 0x7f9eb00bce90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f9eb006c730 tx=0x7f9ea00090f0).stop 2024-01-24T14:13:50.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.286+0000 7f9eb4995700 1 -- 172.21.15.29:0/129494599 shutdown_connections 2024-01-24T14:13:50.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.286+0000 7f9eb4995700 1 --2- 172.21.15.29:0/129494599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eb00627b0 0x7f9eb0062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.286+0000 7f9eb4995700 1 --2- 172.21.15.29:0/129494599 >> v2:172.21.15.163:3300/0 conn(0x7f9eb014b140 0x7f9eb0155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.286+0000 7f9eb4995700 1 --2- 172.21.15.29:0/129494599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00bca30 0x7f9eb00bce90 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.286+0000 7f9eb4995700 1 -- 172.21.15.29:0/129494599 >> 172.21.15.29:0/129494599 conn(0x7f9eb0064390 msgr2=0x7f9eb00c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:50.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.286+0000 7f9eb4995700 1 -- 172.21.15.29:0/129494599 shutdown_connections 2024-01-24T14:13:50.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.286+0000 7f9eb4995700 1 -- 172.21.15.29:0/129494599 wait complete. 2024-01-24T14:13:50.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.288+0000 7f9eb4995700 1 Processor -- start 2024-01-24T14:13:50.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.288+0000 7f9eb4995700 1 -- start start 2024-01-24T14:13:50.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.288+0000 7f9eb4995700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00627b0 0x7f9eb0148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.288+0000 7f9eb4995700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eb00bca30 0x7f9eb00ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.288+0000 7f9eb4995700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eb014b140 0x7f9eb00ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.288+0000 7f9eb4995700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb00a7460 con 0x7f9eb00627b0 2024-01-24T14:13:50.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.288+0000 7f9eb4995700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb00a7a20 con 0x7f9eb00bca30 2024-01-24T14:13:50.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.288+0000 7f9eb4995700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb00aadd0 con 0x7f9eb014b140 2024-01-24T14:13:50.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eae59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00627b0 0x7f9eb0148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eae59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00627b0 0x7f9eb0148a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35344/0 (socket says 172.21.15.29:35344) 2024-01-24T14:13:50.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eae59c700 1 -- 172.21.15.29:0/3626457758 learned_addr learned my addr 172.21.15.29:0/3626457758 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:50.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eaed9d700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eb014b140 0x7f9eb00ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eadd9b700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eb00bca30 0x7f9eb00ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eae59c700 1 -- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eb014b140 msgr2=0x7f9eb00ab490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eae59c700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eb014b140 0x7f9eb00ab490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eae59c700 1 -- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eb00bca30 msgr2=0x7f9eb00ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eae59c700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eb00bca30 0x7f9eb00ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eae59c700 1 -- 172.21.15.29:0/3626457758 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea0009370 con 0x7f9eb00627b0 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eae59c700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00627b0 0x7f9eb0148a30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f9e9800c880 tx=0x7f9e98008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9ea67fc700 1 -- 172.21.15.29:0/3626457758 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98010580 con 0x7f9eb00627b0 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9ea67fc700 1 -- 172.21.15.29:0/3626457758 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e980106e0 con 0x7f9eb00627b0 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eb4995700 1 -- 172.21.15.29:0/3626457758 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb00ab0b0 con 0x7f9eb00627b0 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9ea67fc700 1 -- 172.21.15.29:0/3626457758 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98010a80 con 0x7f9eb00627b0 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.289+0000 7f9eb4995700 1 -- 172.21.15.29:0/3626457758 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb0148f20 con 0x7f9eb00627b0 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.290+0000 7f9eb4995700 1 -- 172.21.15.29:0/3626457758 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb00475e0 con 0x7f9eb00627b0 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.291+0000 7f9ea67fc700 1 -- 172.21.15.29:0/3626457758 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9e9801b460 con 0x7f9eb00627b0 2024-01-24T14:13:50.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.291+0000 7f9ea67fc700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9e9005f270 0x7f9e90061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.291+0000 7f9ea67fc700 1 -- 172.21.15.29:0/3626457758 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6849+0+0 (secure 0 0 0) 0x7f9e9807a5f0 con 0x7f9eb00627b0 2024-01-24T14:13:50.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.291+0000 7f9eadd9b700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9e9005f270 0x7f9e90061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.292+0000 7f9eadd9b700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9e9005f270 0x7f9e90061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9eb006c730 tx=0x7f9ea0017000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:50.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.295+0000 7f9ea67fc700 1 -- 172.21.15.29:0/3626457758 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9e9804b400 con 0x7f9eb00627b0 2024-01-24T14:13:50.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.477+0000 7f9eb4995700 1 -- 172.21.15.29:0/3626457758 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9eb00a7d00 con 0x7f9eb00627b0 2024-01-24T14:13:50.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.481+0000 7f9ea67fc700 1 -- 172.21.15.29:0/3626457758 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9e9804ea20 con 0x7f9eb00627b0 2024-01-24T14:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.483+0000 7f9eb4995700 1 -- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9e9005f270 msgr2=0x7f9e90061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.483+0000 7f9eb4995700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9e9005f270 0x7f9e90061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9eb006c730 tx=0x7f9ea0017000).stop 2024-01-24T14:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.483+0000 7f9eb4995700 1 -- 172.21.15.29:0/3626457758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00627b0 msgr2=0x7f9eb0148a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.483+0000 7f9eb4995700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00627b0 0x7f9eb0148a30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f9e9800c880 tx=0x7f9e98008040).stop 2024-01-24T14:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.483+0000 7f9eb4995700 1 -- 172.21.15.29:0/3626457758 shutdown_connections 2024-01-24T14:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.483+0000 7f9eb4995700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eb014b140 0x7f9eb00ab490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.483+0000 7f9eb4995700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9e9005f270 0x7f9e90061730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.483+0000 7f9eb4995700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9eb00627b0 0x7f9eb0148a30 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.483+0000 7f9eb4995700 1 --2- 172.21.15.29:0/3626457758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eb00bca30 0x7f9eb00ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.483+0000 7f9eb4995700 1 -- 172.21.15.29:0/3626457758 >> 172.21.15.29:0/3626457758 conn(0x7f9eb0064390 msgr2=0x7f9eb00699d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:50.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.483+0000 7f9eb4995700 1 -- 172.21.15.29:0/3626457758 shutdown_connections 2024-01-24T14:13:50.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.483+0000 7f9eb4995700 1 -- 172.21.15.29:0/3626457758 wait complete. 2024-01-24T14:13:50.496 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-01-24T14:13:50.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.628+0000 7f45ca15b700 1 Processor -- start 2024-01-24T14:13:50.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.628+0000 7f45ca15b700 1 -- start start 2024-01-24T14:13:50.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.629+0000 7f45ca15b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45c4134e70 0x7f45c40c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.629+0000 7f45ca15b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4068900 0x7f45c4152e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.629+0000 7f45ca15b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f45c4066520 0x7f45c4066990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.629+0000 7f45ca15b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c414c910 con 0x7f45c4134e70 2024-01-24T14:13:50.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.629+0000 7f45ca15b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c4068470 con 0x7f45c4066520 2024-01-24T14:13:50.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.629+0000 7f45ca15b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c4066e80 con 0x7f45c4068900 2024-01-24T14:13:50.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c37fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45c4134e70 0x7f45c40c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c37fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45c4134e70 0x7f45c40c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35346/0 (socket says 172.21.15.29:35346) 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c37fe700 1 -- 172.21.15.29:0/1159239391 learned_addr learned my addr 172.21.15.29:0/1159239391 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c2ffd700 1 --2- 172.21.15.29:0/1159239391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4068900 0x7f45c4152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c3fff700 1 --2- 172.21.15.29:0/1159239391 >> v2:172.21.15.163:3300/0 conn(0x7f45c4066520 0x7f45c4066990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c37fe700 1 -- 172.21.15.29:0/1159239391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4068900 msgr2=0x7f45c4152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c37fe700 1 --2- 172.21.15.29:0/1159239391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4068900 0x7f45c4152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c37fe700 1 -- 172.21.15.29:0/1159239391 >> v2:172.21.15.163:3300/0 conn(0x7f45c4066520 msgr2=0x7f45c4066990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c37fe700 1 --2- 172.21.15.29:0/1159239391 >> v2:172.21.15.163:3300/0 conn(0x7f45c4066520 0x7f45c4066990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c37fe700 1 -- 172.21.15.29:0/1159239391 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c4153340 con 0x7f45c4134e70 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c37fe700 1 --2- 172.21.15.29:0/1159239391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c4134e70 0x7f45c40c31a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f45c40c55f0 tx=0x7f45b40090f0).ready entity=mon.0 client_cookie=2d961c007db90bb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c27fc700 1 -- 172.21.15.29:0/1159239391 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400b600 con 0x7f45c4134e70 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c27fc700 1 -- 172.21.15.29:0/1159239391 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b400b760 con 0x7f45c4134e70 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.630+0000 7f45c27fc700 1 -- 172.21.15.29:0/1159239391 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400bae0 con 0x7f45c4134e70 2024-01-24T14:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.631+0000 7f45ca15b700 1 -- 172.21.15.29:0/1159239391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c4134e70 msgr2=0x7f45c40c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.631+0000 7f45ca15b700 1 --2- 172.21.15.29:0/1159239391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c4134e70 0x7f45c40c31a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f45c40c55f0 tx=0x7f45b40090f0).stop 2024-01-24T14:13:50.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.631+0000 7f45ca15b700 1 -- 172.21.15.29:0/1159239391 shutdown_connections 2024-01-24T14:13:50.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.631+0000 7f45ca15b700 1 --2- 172.21.15.29:0/1159239391 >> v2:172.21.15.163:3300/0 conn(0x7f45c4066520 0x7f45c4066990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.631+0000 7f45ca15b700 1 --2- 172.21.15.29:0/1159239391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c4134e70 0x7f45c40c31a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.631+0000 7f45ca15b700 1 --2- 172.21.15.29:0/1159239391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4068900 0x7f45c4152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.631+0000 7f45ca15b700 1 -- 172.21.15.29:0/1159239391 >> 172.21.15.29:0/1159239391 conn(0x7f45c40c8ba0 msgr2=0x7f45c40c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:50.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.631+0000 7f45ca15b700 1 -- 172.21.15.29:0/1159239391 shutdown_connections 2024-01-24T14:13:50.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.631+0000 7f45ca15b700 1 -- 172.21.15.29:0/1159239391 wait complete. 2024-01-24T14:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.632+0000 7f45ca15b700 1 Processor -- start 2024-01-24T14:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.632+0000 7f45ca15b700 1 -- start start 2024-01-24T14:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.633+0000 7f45ca15b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c4066520 0x7f45c40ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.633+0000 7f45ca15b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4068900 0x7f45c414e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.633+0000 7f45ca15b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c4134e70 0x7f45c40c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.633+0000 7f45ca15b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c40ad750 con 0x7f45c4066520 2024-01-24T14:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.633+0000 7f45ca15b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c40acad0 con 0x7f45c4134e70 2024-01-24T14:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.633+0000 7f45ca15b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c40acdb0 con 0x7f45c4068900 2024-01-24T14:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.633+0000 7f45c37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c4066520 0x7f45c40ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45c37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c4066520 0x7f45c40ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35356/0 (socket says 172.21.15.29:35356) 2024-01-24T14:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45c37fe700 1 -- 172.21.15.29:0/1371661396 learned_addr learned my addr 172.21.15.29:0/1371661396 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45c2ffd700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4068900 0x7f45c414e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45c3fff700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c4134e70 0x7f45c40c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45c37fe700 1 -- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4068900 msgr2=0x7f45c414e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45c37fe700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4068900 0x7f45c414e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45c37fe700 1 -- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c4134e70 msgr2=0x7f45c40c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45c37fe700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c4134e70 0x7f45c40c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45c37fe700 1 -- 172.21.15.29:0/1371661396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b4009370 con 0x7f45c4066520 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45c37fe700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c4066520 0x7f45c40ac520 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f45b4009130 tx=0x7f45b4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45bb7fe700 1 -- 172.21.15.29:0/1371661396 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400b8d0 con 0x7f45c4066520 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45bb7fe700 1 -- 172.21.15.29:0/1371661396 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b400ba50 con 0x7f45c4066520 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45ca15b700 1 -- 172.21.15.29:0/1371661396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c40ad090 con 0x7f45c4066520 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45bb7fe700 1 -- 172.21.15.29:0/1371661396 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b401c670 con 0x7f45c4066520 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.634+0000 7f45ca15b700 1 -- 172.21.15.29:0/1371661396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c414e850 con 0x7f45c4066520 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.636+0000 7f45ca15b700 1 -- 172.21.15.29:0/1371661396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c40475e0 con 0x7f45c4066520 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.636+0000 7f45bb7fe700 1 -- 172.21.15.29:0/1371661396 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f45b401c9e0 con 0x7f45c4066520 2024-01-24T14:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.636+0000 7f45bb7fe700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f45a405f240 0x7f45a4061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.636+0000 7f45bb7fe700 1 -- 172.21.15.29:0/1371661396 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6849+0+0 (secure 0 0 0) 0x7f45b407c390 con 0x7f45c4066520 2024-01-24T14:13:50.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.637+0000 7f45c2ffd700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f45a405f240 0x7f45a4061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.637+0000 7f45c2ffd700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f45a405f240 0x7f45a4061700 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f45c4152530 tx=0x7f45ac008040).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:50.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.641+0000 7f45bb7fe700 1 -- 172.21.15.29:0/1371661396 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f45b4050c30 con 0x7f45c4066520 2024-01-24T14:13:50.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.823+0000 7f45ca15b700 1 -- 172.21.15.29:0/1371661396 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f45c4135480 con 0x7f45c4066520 2024-01-24T14:13:50.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.829+0000 7f45bb7fe700 1 -- 172.21.15.29:0/1371661396 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v13) v1 ==== 120+0+0 (secure 0 0 0) 0x7f45b4050820 con 0x7f45c4066520 2024-01-24T14:13:50.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.831+0000 7f45ca15b700 1 -- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f45a405f240 msgr2=0x7f45a4061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.831+0000 7f45ca15b700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f45a405f240 0x7f45a4061700 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f45c4152530 tx=0x7f45ac008040).stop 2024-01-24T14:13:50.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.831+0000 7f45ca15b700 1 -- 172.21.15.29:0/1371661396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c4066520 msgr2=0x7f45c40ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.831+0000 7f45ca15b700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c4066520 0x7f45c40ac520 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f45b4009130 tx=0x7f45b4017000).stop 2024-01-24T14:13:50.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.831+0000 7f45ca15b700 1 -- 172.21.15.29:0/1371661396 shutdown_connections 2024-01-24T14:13:50.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.831+0000 7f45ca15b700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c4134e70 0x7f45c40c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.831+0000 7f45ca15b700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f45a405f240 0x7f45a4061700 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.832+0000 7f45ca15b700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c4066520 0x7f45c40ac520 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.832+0000 7f45ca15b700 1 --2- 172.21.15.29:0/1371661396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4068900 0x7f45c414e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.832+0000 7f45ca15b700 1 -- 172.21.15.29:0/1371661396 >> 172.21.15.29:0/1371661396 conn(0x7f45c40c8ba0 msgr2=0x7f45c41526b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:50.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.832+0000 7f45ca15b700 1 -- 172.21.15.29:0/1371661396 shutdown_connections 2024-01-24T14:13:50.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.832+0000 7f45ca15b700 1 -- 172.21.15.29:0/1371661396 wait complete. 2024-01-24T14:13:50.845 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2024-01-24T14:13:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.976+0000 7f64acd57700 1 Processor -- start 2024-01-24T14:13:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.976+0000 7f64acd57700 1 -- start start 2024-01-24T14:13:50.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.977+0000 7f64acd57700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64a814e5a0 0x7f64a80a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.977+0000 7f64acd57700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f64a8068440 0x7f64a80a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.977+0000 7f64acd57700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a81545c0 con 0x7f64a80638c0 2024-01-24T14:13:50.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.977+0000 7f64acd57700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a8068920 con 0x7f64a8068440 2024-01-24T14:13:50.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.977+0000 7f64acd57700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a81551c0 con 0x7f64a814e5a0 2024-01-24T14:13:50.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.977+0000 7f64a6d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f64a80638c0 0x7f64a80664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46692/0 (socket says 172.21.15.29:46692) 2024-01-24T14:13:50.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.977+0000 7f64a6d9d700 1 -- 172.21.15.29:0/693311826 learned_addr learned my addr 172.21.15.29:0/693311826 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:50.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.977+0000 7f64a659c700 1 --2- 172.21.15.29:0/693311826 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64a814e5a0 0x7f64a80a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.977+0000 7f649ffff700 1 --2- 172.21.15.29:0/693311826 >> v2:172.21.15.163:3300/0 conn(0x7f64a8068440 0x7f64a80a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.978+0000 7f64a5d9b700 1 -- 172.21.15.29:0/693311826 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774369685 0 0) 0x7f64a81545c0 con 0x7f64a80638c0 2024-01-24T14:13:50.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.978+0000 7f64a5d9b700 1 -- 172.21.15.29:0/693311826 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64940033f0 con 0x7f64a80638c0 2024-01-24T14:13:50.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.978+0000 7f649ffff700 1 -- 172.21.15.29:0/693311826 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64a814e5a0 msgr2=0x7f64a80a7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:50.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.978+0000 7f649ffff700 1 --2- 172.21.15.29:0/693311826 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64a814e5a0 0x7f64a80a7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.978+0000 7f649ffff700 1 -- 172.21.15.29:0/693311826 >> v1:172.21.15.29:6789/0 conn(0x7f64a80638c0 legacy=0x7f64a80664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.978+0000 7f649ffff700 1 -- 172.21.15.29:0/693311826 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a81555e0 con 0x7f64a8068440 2024-01-24T14:13:50.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.978+0000 7f64a659c700 1 --2- 172.21.15.29:0/693311826 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64a814e5a0 0x7f64a80a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:50.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.978+0000 7f649ffff700 1 --2- 172.21.15.29:0/693311826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a8068440 0x7f64a80a7fb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f6490009760 tx=0x7f6490009060).ready entity=mon.1 client_cookie=f4df50670eb5c016 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:50.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.978+0000 7f64a5d9b700 1 -- 172.21.15.29:0/693311826 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649000d4a0 con 0x7f64a8068440 2024-01-24T14:13:50.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.978+0000 7f64a5d9b700 1 -- 172.21.15.29:0/693311826 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649000d600 con 0x7f64a8068440 2024-01-24T14:13:50.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.978+0000 7f64a5d9b700 1 -- 172.21.15.29:0/693311826 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649000d980 con 0x7f64a8068440 2024-01-24T14:13:50.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.979+0000 7f64acd57700 1 -- 172.21.15.29:0/693311826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a8068440 msgr2=0x7f64a80a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.979+0000 7f64acd57700 1 --2- 172.21.15.29:0/693311826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a8068440 0x7f64a80a7fb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f6490009760 tx=0x7f6490009060).stop 2024-01-24T14:13:50.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.979+0000 7f64acd57700 1 -- 172.21.15.29:0/693311826 shutdown_connections 2024-01-24T14:13:50.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.979+0000 7f64acd57700 1 --2- 172.21.15.29:0/693311826 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64a814e5a0 0x7f64a80a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.979+0000 7f64acd57700 1 --2- 172.21.15.29:0/693311826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a8068440 0x7f64a80a7fb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.979+0000 7f64acd57700 1 -- 172.21.15.29:0/693311826 >> 172.21.15.29:0/693311826 conn(0x7f64a80c8bb0 msgr2=0x7f64a80c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:50.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.979+0000 7f64acd57700 1 -- 172.21.15.29:0/693311826 shutdown_connections 2024-01-24T14:13:50.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.979+0000 7f64acd57700 1 -- 172.21.15.29:0/693311826 wait complete. 2024-01-24T14:13:50.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.980+0000 7f64acd57700 1 Processor -- start 2024-01-24T14:13:50.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.981+0000 7f64acd57700 1 -- start start 2024-01-24T14:13:50.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f64acd57700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64a80638c0 0x7f64a8163990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f64acd57700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64a8068440 0x7f64a8155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f64acd57700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a814e5a0 0x7f64a81562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f64acd57700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a81567a0 con 0x7f64a8068440 2024-01-24T14:13:50.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f64acd57700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a814dff0 con 0x7f64a814e5a0 2024-01-24T14:13:50.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f64acd57700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a8163e80 con 0x7f64a80638c0 2024-01-24T14:13:50.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f649ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64a8068440 0x7f64a8155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f649ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64a8068440 0x7f64a8155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35366/0 (socket says 172.21.15.29:35366) 2024-01-24T14:13:50.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f649ffff700 1 -- 172.21.15.29:0/595352202 learned_addr learned my addr 172.21.15.29:0/595352202 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:50.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f64a6d9d700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a814e5a0 0x7f64a81562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f64a659c700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64a80638c0 0x7f64a8163990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f649ffff700 1 -- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64a80638c0 msgr2=0x7f64a8163990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f649ffff700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64a80638c0 0x7f64a8163990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f649ffff700 1 -- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a814e5a0 msgr2=0x7f64a81562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f649ffff700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a814e5a0 0x7f64a81562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f649ffff700 1 -- 172.21.15.29:0/595352202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64980092a0 con 0x7f64a8068440 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f64a6d9d700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a814e5a0 0x7f64a81562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f649ffff700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64a8068440 0x7f64a8155dc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f6490009d50 tx=0x7f6490016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f649e7fc700 1 -- 172.21.15.29:0/595352202 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490000c90 con 0x7f64a8068440 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f649e7fc700 1 -- 172.21.15.29:0/595352202 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649000d770 con 0x7f64a8068440 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f64acd57700 1 -- 172.21.15.29:0/595352202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64900092a0 con 0x7f64a8068440 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.982+0000 7f649e7fc700 1 -- 172.21.15.29:0/595352202 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649001d660 con 0x7f64a8068440 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.983+0000 7f64acd57700 1 -- 172.21.15.29:0/595352202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a80ae070 con 0x7f64a8068440 2024-01-24T14:13:50.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.984+0000 7f64acd57700 1 -- 172.21.15.29:0/595352202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a80475e0 con 0x7f64a8068440 2024-01-24T14:13:50.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.984+0000 7f649e7fc700 1 -- 172.21.15.29:0/595352202 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f649001dae0 con 0x7f64a8068440 2024-01-24T14:13:50.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.984+0000 7f649e7fc700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f648805f270 0x7f6488061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:50.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.984+0000 7f649e7fc700 1 -- 172.21.15.29:0/595352202 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6849+0+0 (secure 0 0 0) 0x7f649007bf20 con 0x7f64a8068440 2024-01-24T14:13:50.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.985+0000 7f64a659c700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f648805f270 0x7f6488061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:50.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.985+0000 7f64a659c700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f648805f270 0x7f6488061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f6498009270 tx=0x7f6498008040).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:50.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:50.989+0000 7f649e7fc700 1 -- 172.21.15.29:0/595352202 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f649004cd90 con 0x7f64a8068440 2024-01-24T14:13:51.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:51.244+0000 7f64acd57700 1 -- 172.21.15.29:0/595352202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f64a8068d50 con 0x7f64a8068440 2024-01-24T14:13:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.129+0000 7f649e7fc700 1 -- 172.21.15.29:0/595352202 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v570) v1 ==== 156+0+0 (secure 0 0 0) 0x7f64900503b0 con 0x7f64a8068440 2024-01-24T14:13:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.132+0000 7f64acd57700 1 -- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f648805f270 msgr2=0x7f6488061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.132+0000 7f64acd57700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f648805f270 0x7f6488061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f6498009270 tx=0x7f6498008040).stop 2024-01-24T14:13:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.132+0000 7f64acd57700 1 -- 172.21.15.29:0/595352202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64a8068440 msgr2=0x7f64a8155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.132+0000 7f64acd57700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64a8068440 0x7f64a8155dc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f6490009d50 tx=0x7f6490016000).stop 2024-01-24T14:13:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.132+0000 7f64acd57700 1 -- 172.21.15.29:0/595352202 shutdown_connections 2024-01-24T14:13:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.132+0000 7f64acd57700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a814e5a0 0x7f64a81562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:52.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.132+0000 7f64acd57700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64a80638c0 0x7f64a8163990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:52.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.132+0000 7f64acd57700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f648805f270 0x7f6488061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:52.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.132+0000 7f64acd57700 1 --2- 172.21.15.29:0/595352202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64a8068440 0x7f64a8155dc0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:52.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.132+0000 7f64acd57700 1 -- 172.21.15.29:0/595352202 >> 172.21.15.29:0/595352202 conn(0x7f64a80c8bb0 msgr2=0x7f64a80bdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:52.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.132+0000 7f64acd57700 1 -- 172.21.15.29:0/595352202 shutdown_connections 2024-01-24T14:13:52.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.132+0000 7f64acd57700 1 -- 172.21.15.29:0/595352202 wait complete. 2024-01-24T14:13:52.147 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2024-01-24T14:13:52.148 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2024-01-24T14:13:52.148 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-01-24T14:13:52.148 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2024-01-24T14:13:52.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.287+0000 7ff4789bd700 1 Processor -- start 2024-01-24T14:13:52.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.287+0000 7ff4789bd700 1 -- start start 2024-01-24T14:13:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.288+0000 7ff4789bd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff4740c4a10 0x7ff4740bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.288+0000 7ff4789bd700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4741582b0 con 0x7ff474066c00 2024-01-24T14:13:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.288+0000 7ff4789bd700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff474157ca0 con 0x7ff4740670a0 2024-01-24T14:13:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.288+0000 7ff4789bd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff474063c60 con 0x7ff4740c4a10 2024-01-24T14:13:52.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.288+0000 7ff472d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff474066c00 0x7ff4740bdb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46698/0 (socket says 172.21.15.29:46698) 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.288+0000 7ff472d9d700 1 -- 172.21.15.29:0/3835929960 learned_addr learned my addr 172.21.15.29:0/3835929960 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.288+0000 7ff47259c700 1 --2- 172.21.15.29:0/3835929960 >> v2:172.21.15.163:3300/0 conn(0x7ff4740c4a10 0x7ff4740bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.288+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094231758 0 0) 0x7ff4741582b0 con 0x7ff474066c00 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.288+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff45c0033f0 con 0x7ff474066c00 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3102392341 0 0) 0x7ff45c0033f0 con 0x7ff474066c00 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 >> v2:172.21.15.163:3300/0 conn(0x7ff4740c4a10 msgr2=0x7ff4740bf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 --2- 172.21.15.29:0/3835929960 >> v2:172.21.15.163:3300/0 conn(0x7ff4740c4a10 0x7ff4740bf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 >> v1:172.21.15.163:6789/0 conn(0x7ff4740670a0 legacy=0x7ff4740664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff474157f40 con 0x7ff474066c00 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7ff46c0017f0 con 0x7ff474066c00 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 >> v1:172.21.15.29:6789/0 conn(0x7ff474066c00 legacy=0x7ff4740bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 --2- 172.21.15.29:0/3835929960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff45c0038e0 0x7ff45c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:52.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 --2- 172.21.15.29:0/3835929960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff45c004690 0x7ff45c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 --2- 172.21.15.29:0/3835929960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c005000 0x7ff45c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff474063c60 con 0x7ff45c0038e0 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c00bf60 con 0x7ff45c004690 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c00c120 con 0x7ff45c005000 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47259c700 1 --2- 172.21.15.29:0/3835929960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff45c0038e0 0x7ff45c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff471d9b700 1 --2- 172.21.15.29:0/3835929960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff45c004690 0x7ff45c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff472d9d700 1 --2- 172.21.15.29:0/3835929960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c005000 0x7ff45c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47259c700 1 -- 172.21.15.29:0/3835929960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c005000 msgr2=0x7ff45c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47259c700 1 --2- 172.21.15.29:0/3835929960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c005000 0x7ff45c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47259c700 1 -- 172.21.15.29:0/3835929960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff45c004690 msgr2=0x7ff45c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47259c700 1 --2- 172.21.15.29:0/3835929960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff45c004690 0x7ff45c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47259c700 1 -- 172.21.15.29:0/3835929960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c0033f0 con 0x7ff45c0038e0 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff472d9d700 1 --2- 172.21.15.29:0/3835929960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c005000 0x7ff45c00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff472d9d700 1 -- 172.21.15.29:0/3835929960 reap_dead start 2024-01-24T14:13:52.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47259c700 1 --2- 172.21.15.29:0/3835929960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff45c0038e0 0x7ff45c003d00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7ff464009a80 tx=0x7ff464009060).ready entity=mon.0 client_cookie=2d4bb0225303433c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:52.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464009430 con 0x7ff45c0038e0 2024-01-24T14:13:52.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46400e640 con 0x7ff45c0038e0 2024-01-24T14:13:52.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.289+0000 7ff47159a700 1 -- 172.21.15.29:0/3835929960 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46400e9c0 con 0x7ff45c0038e0 2024-01-24T14:13:52.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.290+0000 7ff4789bd700 1 -- 172.21.15.29:0/3835929960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff45c0038e0 msgr2=0x7ff45c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:52.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.290+0000 7ff4789bd700 1 --2- 172.21.15.29:0/3835929960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff45c0038e0 0x7ff45c003d00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7ff464009a80 tx=0x7ff464009060).stop 2024-01-24T14:13:52.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.290+0000 7ff4789bd700 1 -- 172.21.15.29:0/3835929960 shutdown_connections 2024-01-24T14:13:52.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.290+0000 7ff4789bd700 1 --2- 172.21.15.29:0/3835929960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff45c0038e0 0x7ff45c003d00 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:52.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.290+0000 7ff4789bd700 1 -- 172.21.15.29:0/3835929960 >> 172.21.15.29:0/3835929960 conn(0x7ff4740c8bb0 msgr2=0x7ff4740c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:52.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.290+0000 7ff4789bd700 1 -- 172.21.15.29:0/3835929960 shutdown_connections 2024-01-24T14:13:52.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.290+0000 7ff4789bd700 1 -- 172.21.15.29:0/3835929960 wait complete. 2024-01-24T14:13:52.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.291+0000 7ff4789bd700 1 Processor -- start 2024-01-24T14:13:52.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.291+0000 7ff4789bd700 1 -- start start 2024-01-24T14:13:52.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.292+0000 7ff4789bd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c0038e0 0x7ff474164f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:52.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.292+0000 7ff4789bd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4740c0c20 0x7ff4740c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:52.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.292+0000 7ff4789bd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4741567e0 0x7ff474156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:52.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.292+0000 7ff4789bd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740c2010 con 0x7ff4741567e0 2024-01-24T14:13:52.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.292+0000 7ff4789bd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff474157150 con 0x7ff4740c0c20 2024-01-24T14:13:52.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.292+0000 7ff4789bd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740c1400 con 0x7ff45c0038e0 2024-01-24T14:13:52.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.292+0000 7ff472d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4741567e0 0x7ff474156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.292+0000 7ff472d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4741567e0 0x7ff474156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35392/0 (socket says 172.21.15.29:35392) 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.292+0000 7ff47259c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c0038e0 0x7ff474164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.292+0000 7ff471d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4740c0c20 0x7ff4740c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.292+0000 7ff472d9d700 1 -- 172.21.15.29:0/592956857 learned_addr learned my addr 172.21.15.29:0/592956857 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.293+0000 7ff472d9d700 1 -- 172.21.15.29:0/592956857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c0038e0 msgr2=0x7ff474164f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.293+0000 7ff472d9d700 1 --2- 172.21.15.29:0/592956857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c0038e0 0x7ff474164f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.293+0000 7ff472d9d700 1 -- 172.21.15.29:0/592956857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4740c0c20 msgr2=0x7ff4740c3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.293+0000 7ff472d9d700 1 --2- 172.21.15.29:0/592956857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4740c0c20 0x7ff4740c3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.293+0000 7ff472d9d700 1 -- 172.21.15.29:0/592956857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740c16e0 con 0x7ff4741567e0 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.293+0000 7ff472d9d700 1 --2- 172.21.15.29:0/592956857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4741567e0 0x7ff474156c60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7ff474066140 tx=0x7ff4740c6780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.293+0000 7ff4627fc700 1 -- 172.21.15.29:0/592956857 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c00e830 con 0x7ff4741567e0 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.293+0000 7ff4627fc700 1 -- 172.21.15.29:0/592956857 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c00e990 con 0x7ff4741567e0 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.293+0000 7ff4627fc700 1 -- 172.21.15.29:0/592956857 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c00ed10 con 0x7ff4741567e0 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.293+0000 7ff4789bd700 1 -- 172.21.15.29:0/592956857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4740c19c0 con 0x7ff4741567e0 2024-01-24T14:13:52.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.293+0000 7ff4789bd700 1 -- 172.21.15.29:0/592956857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4740c10a0 con 0x7ff4741567e0 2024-01-24T14:13:52.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.294+0000 7ff4789bd700 1 -- 172.21.15.29:0/592956857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4740c1e60 con 0x7ff4741567e0 2024-01-24T14:13:52.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.295+0000 7ff4627fc700 1 -- 172.21.15.29:0/592956857 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff46c014640 con 0x7ff4741567e0 2024-01-24T14:13:52.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.295+0000 7ff4627fc700 1 --2- 172.21.15.29:0/592956857 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff45405f270 0x7ff454061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:52.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.295+0000 7ff4627fc700 1 -- 172.21.15.29:0/592956857 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6849+0+0 (secure 0 0 0) 0x7ff46c079e20 con 0x7ff4741567e0 2024-01-24T14:13:52.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.295+0000 7ff47259c700 1 --2- 172.21.15.29:0/592956857 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff45405f270 0x7ff454061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:52.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.296+0000 7ff47259c700 1 --2- 172.21.15.29:0/592956857 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff45405f270 0x7ff454061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7ff464006d50 tx=0x7ff46401a000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:52.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.300+0000 7ff4627fc700 1 -- 172.21.15.29:0/592956857 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff46c04ac30 con 0x7ff4741567e0 2024-01-24T14:13:52.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:52.550+0000 7ff4789bd700 1 -- 172.21.15.29:0/592956857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7ff4740c5020 con 0x7ff4741567e0 2024-01-24T14:13:53.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.130+0000 7ff4627fc700 1 -- 172.21.15.29:0/592956857 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v571) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff46c04e250 con 0x7ff4741567e0 2024-01-24T14:13:53.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.133+0000 7ff4789bd700 1 -- 172.21.15.29:0/592956857 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff45405f270 msgr2=0x7ff454061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.133+0000 7ff4789bd700 1 --2- 172.21.15.29:0/592956857 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff45405f270 0x7ff454061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7ff464006d50 tx=0x7ff46401a000).stop 2024-01-24T14:13:53.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.133+0000 7ff4789bd700 1 -- 172.21.15.29:0/592956857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4741567e0 msgr2=0x7ff474156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.133+0000 7ff4789bd700 1 --2- 172.21.15.29:0/592956857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4741567e0 0x7ff474156c60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7ff474066140 tx=0x7ff4740c6780).stop 2024-01-24T14:13:53.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.133+0000 7ff4789bd700 1 -- 172.21.15.29:0/592956857 shutdown_connections 2024-01-24T14:13:53.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.133+0000 7ff4789bd700 1 --2- 172.21.15.29:0/592956857 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff45405f270 0x7ff454061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.133+0000 7ff4789bd700 1 --2- 172.21.15.29:0/592956857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4741567e0 0x7ff474156c60 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.133+0000 7ff4789bd700 1 --2- 172.21.15.29:0/592956857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c0038e0 0x7ff474164f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.133+0000 7ff4789bd700 1 --2- 172.21.15.29:0/592956857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4740c0c20 0x7ff4740c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.133+0000 7ff4789bd700 1 -- 172.21.15.29:0/592956857 >> 172.21.15.29:0/592956857 conn(0x7ff4740c8bb0 msgr2=0x7ff4740ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.134+0000 7ff4789bd700 1 -- 172.21.15.29:0/592956857 shutdown_connections 2024-01-24T14:13:53.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.134+0000 7ff4789bd700 1 -- 172.21.15.29:0/592956857 wait complete. 2024-01-24T14:13:53.150 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-24T14:13:53.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.287+0000 7f226fb12700 1 Processor -- start 2024-01-24T14:13:53.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.287+0000 7f226fb12700 1 -- start start 2024-01-24T14:13:53.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.288+0000 7f226fb12700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268153f40 0x7f22680a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:53.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.288+0000 7f226fb12700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f22680c8290 0x7f22680627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:53.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.288+0000 7f226fb12700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226806a360 con 0x7f2268154320 2024-01-24T14:13:53.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.288+0000 7f226fb12700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22680bcef0 con 0x7f22680c8290 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.288+0000 7f226fb12700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2268069de0 con 0x7f2268153f40 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.288+0000 7f226d0ad700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f22680c8290 0x7f22680627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.288+0000 7f226d0ad700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f22680c8290 0x7f22680627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35408/0 (socket says 172.21.15.29:35408) 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.288+0000 7f226e0af700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2268154320 0x7f2268157d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46714/0 (socket says 172.21.15.29:46714) 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.288+0000 7f226d0ad700 1 -- 172.21.15.29:0/1002932518 learned_addr learned my addr 172.21.15.29:0/1002932518 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.288+0000 7f226d8ae700 1 --2- 172.21.15.29:0/1002932518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268153f40 0x7f22680a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.289+0000 7f226d0ad700 1 -- 172.21.15.29:0/1002932518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268153f40 msgr2=0x7f22680a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.289+0000 7f226d0ad700 1 --2- 172.21.15.29:0/1002932518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268153f40 0x7f22680a6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.289+0000 7f226d0ad700 1 -- 172.21.15.29:0/1002932518 >> v1:172.21.15.29:6789/0 conn(0x7f2268154320 legacy=0x7f2268157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.289+0000 7f226d0ad700 1 -- 172.21.15.29:0/1002932518 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226806a080 con 0x7f22680c8290 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.289+0000 7f226c8ac700 1 -- 172.21.15.29:0/1002932518 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3903682956 0 0) 0x7f226806a360 con 0x7f2268154320 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.289+0000 7f226d0ad700 1 --2- 172.21.15.29:0/1002932518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22680c8290 0x7f22680627b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f2264006fd0 tx=0x7f22640090f0).ready entity=mon.0 client_cookie=356000228dd26b85 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.289+0000 7f226c8ac700 1 -- 172.21.15.29:0/1002932518 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400b580 con 0x7f22680c8290 2024-01-24T14:13:53.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.289+0000 7f226c8ac700 1 -- 172.21.15.29:0/1002932518 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400b6e0 con 0x7f22680c8290 2024-01-24T14:13:53.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.289+0000 7f226c8ac700 1 -- 172.21.15.29:0/1002932518 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400ba80 con 0x7f22680c8290 2024-01-24T14:13:53.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.289+0000 7f226fb12700 1 -- 172.21.15.29:0/1002932518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22680c8290 msgr2=0x7f22680627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.289+0000 7f226fb12700 1 --2- 172.21.15.29:0/1002932518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22680c8290 0x7f22680627b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f2264006fd0 tx=0x7f22640090f0).stop 2024-01-24T14:13:53.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.290+0000 7f226fb12700 1 -- 172.21.15.29:0/1002932518 shutdown_connections 2024-01-24T14:13:53.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.290+0000 7f226fb12700 1 --2- 172.21.15.29:0/1002932518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268153f40 0x7f22680a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.290+0000 7f226fb12700 1 --2- 172.21.15.29:0/1002932518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22680c8290 0x7f22680627b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.290+0000 7f226fb12700 1 -- 172.21.15.29:0/1002932518 >> 172.21.15.29:0/1002932518 conn(0x7f2268064390 msgr2=0x7f22680c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:53.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.290+0000 7f226fb12700 1 -- 172.21.15.29:0/1002932518 shutdown_connections 2024-01-24T14:13:53.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.290+0000 7f226fb12700 1 -- 172.21.15.29:0/1002932518 wait complete. 2024-01-24T14:13:53.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.291+0000 7f226fb12700 1 Processor -- start 2024-01-24T14:13:53.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.291+0000 7f226fb12700 1 -- start start 2024-01-24T14:13:53.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226fb12700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22680c8290 0x7f22680c08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226fb12700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2268153f40 0x7f226814f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226fb12700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268154320 0x7f22680c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226fb12700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22680bf850 con 0x7f2268153f40 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226fb12700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22680bfb30 con 0x7f22680c8290 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226fb12700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22680bfe10 con 0x7f2268154320 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226d0ad700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2268153f40 0x7f226814f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226d0ad700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2268153f40 0x7f226814f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35422/0 (socket says 172.21.15.29:35422) 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226d0ad700 1 -- 172.21.15.29:0/1867524857 learned_addr learned my addr 172.21.15.29:0/1867524857 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226e0af700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268154320 0x7f22680c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226d8ae700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22680c8290 0x7f22680c08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226d0ad700 1 -- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268154320 msgr2=0x7f22680c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226d0ad700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268154320 0x7f22680c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226d0ad700 1 -- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22680c8290 msgr2=0x7f22680c08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226d0ad700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22680c8290 0x7f22680c08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226d0ad700 1 -- 172.21.15.29:0/1867524857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009370 con 0x7f2268153f40 2024-01-24T14:13:53.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.292+0000 7f226d0ad700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2268153f40 0x7f226814f770 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f2268064910 tx=0x7f2264017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:53.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.293+0000 7f2259ffb700 1 -- 172.21.15.29:0/1867524857 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400be40 con 0x7f2268153f40 2024-01-24T14:13:53.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.293+0000 7f2259ffb700 1 -- 172.21.15.29:0/1867524857 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400b850 con 0x7f2268153f40 2024-01-24T14:13:53.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.293+0000 7f226fb12700 1 -- 172.21.15.29:0/1867524857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22680c00f0 con 0x7f2268153f40 2024-01-24T14:13:53.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.293+0000 7f2259ffb700 1 -- 172.21.15.29:0/1867524857 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226401c420 con 0x7f2268153f40 2024-01-24T14:13:53.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.293+0000 7f226fb12700 1 -- 172.21.15.29:0/1867524857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226814fc60 con 0x7f2268153f40 2024-01-24T14:13:53.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.294+0000 7f226fb12700 1 -- 172.21.15.29:0/1867524857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22680c1d00 con 0x7f2268153f40 2024-01-24T14:13:53.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.294+0000 7f2259ffb700 1 -- 172.21.15.29:0/1867524857 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f226401c810 con 0x7f2268153f40 2024-01-24T14:13:53.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.295+0000 7f2259ffb700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f224c05f240 0x7f224c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:53.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.295+0000 7f2259ffb700 1 -- 172.21.15.29:0/1867524857 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6849+0+0 (secure 0 0 0) 0x7f226407c160 con 0x7f2268153f40 2024-01-24T14:13:53.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.295+0000 7f226d8ae700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f224c05f240 0x7f224c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:53.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.295+0000 7f226d8ae700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f224c05f240 0x7f224c061700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f22680a6810 tx=0x7f225c008040).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:53.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.299+0000 7f2259ffb700 1 -- 172.21.15.29:0/1867524857 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f226404cfd0 con 0x7f2268153f40 2024-01-24T14:13:53.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.479+0000 7f226fb12700 1 -- 172.21.15.29:0/1867524857 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f22680475e0 con 0x7f2268153f40 2024-01-24T14:13:53.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.483+0000 7f2259ffb700 1 -- 172.21.15.29:0/1867524857 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f226402a0c0 con 0x7f2268153f40 2024-01-24T14:13:53.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.485+0000 7f226fb12700 1 -- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f224c05f240 msgr2=0x7f224c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.485+0000 7f226fb12700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f224c05f240 0x7f224c061700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f22680a6810 tx=0x7f225c008040).stop 2024-01-24T14:13:53.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.485+0000 7f226fb12700 1 -- 172.21.15.29:0/1867524857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2268153f40 msgr2=0x7f226814f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.485+0000 7f226fb12700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2268153f40 0x7f226814f770 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f2268064910 tx=0x7f2264017000).stop 2024-01-24T14:13:53.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.485+0000 7f226fb12700 1 -- 172.21.15.29:0/1867524857 shutdown_connections 2024-01-24T14:13:53.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.485+0000 7f226fb12700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268154320 0x7f22680c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.485+0000 7f226fb12700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22680c8290 0x7f22680c08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.485+0000 7f226fb12700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f224c05f240 0x7f224c061700 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.485+0000 7f226fb12700 1 --2- 172.21.15.29:0/1867524857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2268153f40 0x7f226814f770 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.485+0000 7f226fb12700 1 -- 172.21.15.29:0/1867524857 >> 172.21.15.29:0/1867524857 conn(0x7f2268064390 msgr2=0x7f22680627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:53.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.485+0000 7f226fb12700 1 -- 172.21.15.29:0/1867524857 shutdown_connections 2024-01-24T14:13:53.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.485+0000 7f226fb12700 1 -- 172.21.15.29:0/1867524857 wait complete. 2024-01-24T14:13:53.499 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T14:13:53.499 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:13:53.547 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:13:53.601 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T14:13:53.601 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-24T14:13:53.602 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:13:53.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.737+0000 7fe41d002700 1 Processor -- start 2024-01-24T14:13:53.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.737+0000 7fe41d002700 1 -- start start 2024-01-24T14:13:53.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.738+0000 7fe41d002700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4180c4ca0 0x7fe4181550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:53.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.738+0000 7fe41d002700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe418065360 con 0x7fe4181540a0 2024-01-24T14:13:53.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.738+0000 7fe41d002700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4180a6ce0 con 0x7fe418062ec0 2024-01-24T14:13:53.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.738+0000 7fe41d002700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180a74f0 con 0x7fe4180c4ca0 2024-01-24T14:13:53.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.738+0000 7fe416d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe4181540a0 0x7fe4180627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38788/0 (socket says 172.21.15.29:38788) 2024-01-24T14:13:53.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.738+0000 7fe416d9d700 1 -- 172.21.15.29:0/2366771558 learned_addr learned my addr 172.21.15.29:0/2366771558 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:53.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.738+0000 7fe41759e700 1 --2- 172.21.15.29:0/2366771558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4180c4ca0 0x7fe4181550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:53.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.739+0000 7fe415d9b700 1 -- 172.21.15.29:0/2366771558 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378740331 0 0) 0x7fe418065360 con 0x7fe4181540a0 2024-01-24T14:13:53.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.739+0000 7fe415d9b700 1 -- 172.21.15.29:0/2366771558 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4040033f0 con 0x7fe4181540a0 2024-01-24T14:13:53.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.739+0000 7fe415d9b700 1 -- 172.21.15.29:0/2366771558 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513480051 0 0) 0x7fe4180a6ce0 con 0x7fe418062ec0 2024-01-24T14:13:53.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.739+0000 7fe415d9b700 1 -- 172.21.15.29:0/2366771558 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe418065360 con 0x7fe418062ec0 2024-01-24T14:13:53.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.739+0000 7fe41759e700 1 -- 172.21.15.29:0/2366771558 >> v1:172.21.15.163:6789/0 conn(0x7fe418062ec0 legacy=0x7fe418069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.739+0000 7fe41759e700 1 -- 172.21.15.29:0/2366771558 >> v1:172.21.15.29:6789/0 conn(0x7fe4181540a0 legacy=0x7fe4180627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.739+0000 7fe415d9b700 1 -- 172.21.15.29:0/2366771558 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3206290947 0 0) 0x7fe4040033f0 con 0x7fe4181540a0 2024-01-24T14:13:53.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.739+0000 7fe41759e700 1 -- 172.21.15.29:0/2366771558 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180636f0 con 0x7fe4180c4ca0 2024-01-24T14:13:53.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.739+0000 7fe41759e700 1 --2- 172.21.15.29:0/2366771558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4180c4ca0 0x7fe4181550f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fe410009e90 tx=0x7fe410009780).ready entity=mon.2 client_cookie=96462cb7592d393b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:53.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.739+0000 7fe415d9b700 1 -- 172.21.15.29:0/2366771558 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41000e070 con 0x7fe4180c4ca0 2024-01-24T14:13:53.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.739+0000 7fe415d9b700 1 -- 172.21.15.29:0/2366771558 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41000b7a0 con 0x7fe4180c4ca0 2024-01-24T14:13:53.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.739+0000 7fe415d9b700 1 -- 172.21.15.29:0/2366771558 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400001470 con 0x7fe4180c4ca0 2024-01-24T14:13:53.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.740+0000 7fe41d002700 1 -- 172.21.15.29:0/2366771558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4180c4ca0 msgr2=0x7fe4181550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.740+0000 7fe41d002700 1 --2- 172.21.15.29:0/2366771558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4180c4ca0 0x7fe4181550f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fe410009e90 tx=0x7fe410009780).stop 2024-01-24T14:13:53.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.740+0000 7fe41d002700 1 -- 172.21.15.29:0/2366771558 shutdown_connections 2024-01-24T14:13:53.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.740+0000 7fe41d002700 1 --2- 172.21.15.29:0/2366771558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4180c4ca0 0x7fe4181550f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.740+0000 7fe41d002700 1 -- 172.21.15.29:0/2366771558 >> 172.21.15.29:0/2366771558 conn(0x7fe4180c85c0 msgr2=0x7fe4180c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:53.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.740+0000 7fe41d002700 1 -- 172.21.15.29:0/2366771558 shutdown_connections 2024-01-24T14:13:53.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.740+0000 7fe41d002700 1 -- 172.21.15.29:0/2366771558 wait complete. 2024-01-24T14:13:53.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.742+0000 7fe41d002700 1 Processor -- start 2024-01-24T14:13:53.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.742+0000 7fe41d002700 1 -- start start 2024-01-24T14:13:53.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.743+0000 7fe41d002700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe418062ec0 0x7fe4180a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.743+0000 7fe41d002700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4180c4ca0 0x7fe41814e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.743+0000 7fe41d002700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4181540a0 0x7fe41814e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.743+0000 7fe41d002700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4181555e0 con 0x7fe4180c4ca0 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.743+0000 7fe41d002700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe418164940 con 0x7fe418062ec0 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.743+0000 7fe41d002700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41814ed70 con 0x7fe4181540a0 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.743+0000 7fe41659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4180c4ca0 0x7fe41814e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.743+0000 7fe41659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4180c4ca0 0x7fe41814e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34154/0 (socket says 172.21.15.29:34154) 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.743+0000 7fe41659c700 1 -- 172.21.15.29:0/2958834772 learned_addr learned my addr 172.21.15.29:0/2958834772 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.743+0000 7fe416d9d700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe418062ec0 0x7fe4180a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.743+0000 7fe41759e700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4181540a0 0x7fe41814e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.744+0000 7fe41659c700 1 -- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4181540a0 msgr2=0x7fe41814e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.744+0000 7fe41659c700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4181540a0 0x7fe41814e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.744+0000 7fe41659c700 1 -- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe418062ec0 msgr2=0x7fe4180a9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.744+0000 7fe41659c700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe418062ec0 0x7fe4180a9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.744+0000 7fe41659c700 1 -- 172.21.15.29:0/2958834772 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410009a20 con 0x7fe4180c4ca0 2024-01-24T14:13:53.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.744+0000 7fe41659c700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4180c4ca0 0x7fe41814e200 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fe40800c490 tx=0x7fe408007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:53.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.744+0000 7fe40effd700 1 -- 172.21.15.29:0/2958834772 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40800ce40 con 0x7fe4180c4ca0 2024-01-24T14:13:53.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.744+0000 7fe40effd700 1 -- 172.21.15.29:0/2958834772 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40800f470 con 0x7fe4180c4ca0 2024-01-24T14:13:53.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.744+0000 7fe41d002700 1 -- 172.21.15.29:0/2958834772 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41814f050 con 0x7fe4180c4ca0 2024-01-24T14:13:53.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.744+0000 7fe40effd700 1 -- 172.21.15.29:0/2958834772 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40800f830 con 0x7fe4180c4ca0 2024-01-24T14:13:53.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.744+0000 7fe41d002700 1 -- 172.21.15.29:0/2958834772 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41814f610 con 0x7fe4180c4ca0 2024-01-24T14:13:53.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.745+0000 7fe41d002700 1 -- 172.21.15.29:0/2958834772 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4180beb70 con 0x7fe4180c4ca0 2024-01-24T14:13:53.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.746+0000 7fe40effd700 1 -- 172.21.15.29:0/2958834772 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe40800fbe0 con 0x7fe4180c4ca0 2024-01-24T14:13:53.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.746+0000 7fe40effd700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3f805f270 0x7fe3f8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:53.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.746+0000 7fe40effd700 1 -- 172.21.15.29:0/2958834772 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6849+0+0 (secure 0 0 0) 0x7fe408079b90 con 0x7fe4180c4ca0 2024-01-24T14:13:53.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.746+0000 7fe416d9d700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3f805f270 0x7fe3f8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:53.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.747+0000 7fe416d9d700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3f805f270 0x7fe3f8061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe4000011e0 tx=0x7fe400000bc0).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:53.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.750+0000 7fe40effd700 1 -- 172.21.15.29:0/2958834772 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe40804aa00 con 0x7fe4180c4ca0 2024-01-24T14:13:53.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.977+0000 7fe41d002700 1 -- 172.21.15.29:0/2958834772 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4180a6f30 con 0x7fe4180c4ca0 2024-01-24T14:13:53.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.977+0000 7fe40effd700 1 -- 172.21.15.29:0/2958834772 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fe408007d80 con 0x7fe4180c4ca0 2024-01-24T14:13:53.978 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:53.978 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1149323432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1149323432},{"type":"v1","addr":"172.21.15.163:6835","nonce":1149323432}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6157,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2941255566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":2941255566},{"type":"v1","addr":"172.21.15.29:6835","nonce":2941255566}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6163,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2235282008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2235282008},{"type":"v1","addr":"172.21.15.163:6837","nonce":2235282008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":6732,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/305362175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":305362175},{"type":"v1","addr":"172.21.15.29:6833","nonce":305362175}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[]} 2024-01-24T14:13:53.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.979+0000 7fe41d002700 1 -- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3f805f270 msgr2=0x7fe3f8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.979+0000 7fe41d002700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3f805f270 0x7fe3f8061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe4000011e0 tx=0x7fe400000bc0).stop 2024-01-24T14:13:53.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.979+0000 7fe41d002700 1 -- 172.21.15.29:0/2958834772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4180c4ca0 msgr2=0x7fe41814e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:53.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.979+0000 7fe41d002700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4180c4ca0 0x7fe41814e200 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fe40800c490 tx=0x7fe408007c60).stop 2024-01-24T14:13:53.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.980+0000 7fe41d002700 1 -- 172.21.15.29:0/2958834772 shutdown_connections 2024-01-24T14:13:53.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.980+0000 7fe41d002700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4181540a0 0x7fe41814e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.980+0000 7fe41d002700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe418062ec0 0x7fe4180a9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.980+0000 7fe41d002700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3f805f270 0x7fe3f8061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.980+0000 7fe41d002700 1 --2- 172.21.15.29:0/2958834772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4180c4ca0 0x7fe41814e200 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:53.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.980+0000 7fe41d002700 1 -- 172.21.15.29:0/2958834772 >> 172.21.15.29:0/2958834772 conn(0x7fe4180c85c0 msgr2=0x7fe4180c5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:53.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.980+0000 7fe41d002700 1 -- 172.21.15.29:0/2958834772 shutdown_connections 2024-01-24T14:13:53.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:53.980+0000 7fe41d002700 1 -- 172.21.15.29:0/2958834772 wait complete. 2024-01-24T14:13:53.982 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 37 2024-01-24T14:13:53.994 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-24T14:13:53.994 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-24T14:13:53.994 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-24T14:13:53.994 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:13:53.995 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-24T14:13:53.995 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-24T14:13:53.995 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-24T14:13:53.995 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:13:53.995 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-24T14:13:53.995 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-24T14:13:53.995 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-24T14:13:53.996 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:13:53.996 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-24T14:13:53.996 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-24T14:13:53.996 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-24T14:13:53.996 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:13:54.004 INFO:tasks.ceph.mds.c:Stopped 2024-01-24T14:13:54.004 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-24T14:13:54.005 INFO:tasks.ceph.mds.a:Stopped 2024-01-24T14:13:54.005 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-24T14:13:54.007 INFO:tasks.ceph.mds.d:Stopped 2024-01-24T14:13:54.007 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-24T14:13:54.007 INFO:tasks.ceph.mds.b:Stopped 2024-01-24T14:13:54.008 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-24T14:13:54.047 INFO:tasks.ceph.mds.c:Started 2024-01-24T14:13:54.047 DEBUG:teuthology.parallel:result is None 2024-01-24T14:13:54.048 INFO:tasks.ceph.mds.a:Started 2024-01-24T14:13:54.049 DEBUG:teuthology.parallel:result is None 2024-01-24T14:13:54.049 INFO:tasks.ceph.mds.d:Started 2024-01-24T14:13:54.050 DEBUG:teuthology.parallel:result is None 2024-01-24T14:13:54.050 INFO:tasks.ceph.mds.b:Started 2024-01-24T14:13:54.050 DEBUG:teuthology.parallel:result is None 2024-01-24T14:13:54.051 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-24T14:13:54.134 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T14:13:54.135 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T14:13:54.136 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-01-24T14:13:54.136 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T14:13:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.191+0000 7f28e5107700 1 Processor -- start 2024-01-24T14:13:54.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.191+0000 7f28e5107700 1 -- start start 2024-01-24T14:13:54.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.192+0000 7f28e5107700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28e0154240 0x7f28e0064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:54.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.192+0000 7f28e5107700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28e00a9d20 0x7f28e00aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:54.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.192+0000 7f28e5107700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e01550e0 con 0x7f28e0157cf0 2024-01-24T14:13:54.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.192+0000 7f28e5107700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e00c0d10 con 0x7f28e0154240 2024-01-24T14:13:54.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.192+0000 7f28e5107700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e00aa670 con 0x7f28e00a9d20 2024-01-24T14:13:54.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.192+0000 7f28ded9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28e0154240 0x7f28e0064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:54.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.192+0000 7f28ded9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28e0154240 0x7f28e0064780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34186/0 (socket says 172.21.15.29:34186) 2024-01-24T14:13:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.193+0000 7f28ded9d700 1 -- 172.21.15.29:0/2426270849 learned_addr learned my addr 172.21.15.29:0/2426270849 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.192+0000 7f28df59e700 1 --1- 172.21.15.29:0/2426270849 >> v1:172.21.15.29:6789/0 conn(0x7f28e0157cf0 0x7f28e0158100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38794/0 (socket says 172.21.15.29:38794) 2024-01-24T14:13:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.193+0000 7f28de59c700 1 --2- 172.21.15.29:0/2426270849 >> v2:172.21.15.163:3300/0 conn(0x7f28e00a9d20 0x7f28e00aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.193+0000 7f28ded9d700 1 -- 172.21.15.29:0/2426270849 >> v2:172.21.15.163:3300/0 conn(0x7f28e00a9d20 msgr2=0x7f28e00aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.193+0000 7f28ded9d700 1 --2- 172.21.15.29:0/2426270849 >> v2:172.21.15.163:3300/0 conn(0x7f28e00a9d20 0x7f28e00aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.193+0000 7f28ded9d700 1 -- 172.21.15.29:0/2426270849 >> v1:172.21.15.29:6789/0 conn(0x7f28e0157cf0 legacy=0x7f28e0158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.193+0000 7f28ded9d700 1 -- 172.21.15.29:0/2426270849 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e00650d0 con 0x7f28e0154240 2024-01-24T14:13:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.193+0000 7f28ddd9b700 1 -- 172.21.15.29:0/2426270849 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103514829 0 0) 0x7f28e01550e0 con 0x7f28e0157cf0 2024-01-24T14:13:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.193+0000 7f28ded9d700 1 --2- 172.21.15.29:0/2426270849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e0154240 0x7f28e0064780 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f28e0084090 tx=0x7f28d0009840).ready entity=mon.0 client_cookie=f3f152c0011df7dc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.193+0000 7f28ddd9b700 1 -- 172.21.15.29:0/2426270849 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d000e070 con 0x7f28e0154240 2024-01-24T14:13:54.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.193+0000 7f28ddd9b700 1 -- 172.21.15.29:0/2426270849 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d000ba20 con 0x7f28e0154240 2024-01-24T14:13:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.193+0000 7f28ddd9b700 1 -- 172.21.15.29:0/2426270849 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d000bda0 con 0x7f28e0154240 2024-01-24T14:13:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.194+0000 7f28e5107700 1 -- 172.21.15.29:0/2426270849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e0154240 msgr2=0x7f28e0064780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.194+0000 7f28e5107700 1 --2- 172.21.15.29:0/2426270849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e0154240 0x7f28e0064780 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f28e0084090 tx=0x7f28d0009840).stop 2024-01-24T14:13:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.194+0000 7f28e5107700 1 -- 172.21.15.29:0/2426270849 shutdown_connections 2024-01-24T14:13:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.194+0000 7f28e5107700 1 --2- 172.21.15.29:0/2426270849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e0154240 0x7f28e0064780 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.194+0000 7f28e5107700 1 --2- 172.21.15.29:0/2426270849 >> v2:172.21.15.163:3300/0 conn(0x7f28e00a9d20 0x7f28e00aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.194+0000 7f28e5107700 1 -- 172.21.15.29:0/2426270849 >> 172.21.15.29:0/2426270849 conn(0x7f28e00c8310 msgr2=0x7f28e005e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.194+0000 7f28e5107700 1 -- 172.21.15.29:0/2426270849 shutdown_connections 2024-01-24T14:13:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.194+0000 7f28e5107700 1 -- 172.21.15.29:0/2426270849 wait complete. 2024-01-24T14:13:54.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.195+0000 7f28e5107700 1 Processor -- start 2024-01-24T14:13:54.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.195+0000 7f28e5107700 1 -- start start 2024-01-24T14:13:54.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28e5107700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28e00a9d20 0x7f28e00c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:54.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28e5107700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e0154240 0x7f28e00b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:54.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28e5107700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e0157cf0 0x7f28e00b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:54.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28e5107700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e00c2af0 con 0x7f28e0157cf0 2024-01-24T14:13:54.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28e5107700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e00c1ee0 con 0x7f28e00a9d20 2024-01-24T14:13:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28e5107700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e00c21c0 con 0x7f28e0154240 2024-01-24T14:13:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28df59e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e0157cf0 0x7f28e00b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28df59e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e0157cf0 0x7f28e00b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34192/0 (socket says 172.21.15.29:34192) 2024-01-24T14:13:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28df59e700 1 -- 172.21.15.29:0/75180379 learned_addr learned my addr 172.21.15.29:0/75180379 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28de59c700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e0154240 0x7f28e00b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28ded9d700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28e00a9d20 0x7f28e00c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28df59e700 1 -- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e0154240 msgr2=0x7f28e00b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28df59e700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e0154240 0x7f28e00b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28df59e700 1 -- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28e00a9d20 msgr2=0x7f28e00c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28df59e700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28e00a9d20 0x7f28e00c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.196+0000 7f28df59e700 1 -- 172.21.15.29:0/75180379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d0009ae0 con 0x7f28e0157cf0 2024-01-24T14:13:54.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.197+0000 7f28df59e700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e0157cf0 0x7f28e00b0a30 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f28d800c490 tx=0x7f28d8007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:54.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.197+0000 7f28c6ffd700 1 -- 172.21.15.29:0/75180379 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d80070d0 con 0x7f28e0157cf0 2024-01-24T14:13:54.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.197+0000 7f28c6ffd700 1 -- 172.21.15.29:0/75180379 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d8007230 con 0x7f28e0157cf0 2024-01-24T14:13:54.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.197+0000 7f28e5107700 1 -- 172.21.15.29:0/75180379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e00c24a0 con 0x7f28e0157cf0 2024-01-24T14:13:54.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.197+0000 7f28c6ffd700 1 -- 172.21.15.29:0/75180379 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d80056e0 con 0x7f28e0157cf0 2024-01-24T14:13:54.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.197+0000 7f28e5107700 1 -- 172.21.15.29:0/75180379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e0152e00 con 0x7f28e0157cf0 2024-01-24T14:13:54.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.198+0000 7f28e5107700 1 -- 172.21.15.29:0/75180379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e00c2ca0 con 0x7f28e0157cf0 2024-01-24T14:13:54.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.198+0000 7f28c6ffd700 1 -- 172.21.15.29:0/75180379 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f28d8005ad0 con 0x7f28e0157cf0 2024-01-24T14:13:54.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.199+0000 7f28c6ffd700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28cc05f270 0x7f28cc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:54.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.199+0000 7f28c6ffd700 1 -- 172.21.15.29:0/75180379 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6849+0+0 (secure 0 0 0) 0x7f28d8079d50 con 0x7f28e0157cf0 2024-01-24T14:13:54.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.199+0000 7f28ded9d700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28cc05f270 0x7f28cc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:54.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.199+0000 7f28ded9d700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28cc05f270 0x7f28cc061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f28d0009ab0 tx=0x7f28d0009840).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:54.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.203+0000 7f28c6ffd700 1 -- 172.21.15.29:0/75180379 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f28e00c2ca0 con 0x7f28e0157cf0 2024-01-24T14:13:54.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:54.404+0000 7f28e5107700 1 -- 172.21.15.29:0/75180379 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f28e00c2ca0 con 0x7f28e0157cf0 2024-01-24T14:13:55.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.187+0000 7f28c6ffd700 1 -- 172.21.15.29:0/75180379 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v101) v1 ==== 97+0+0 (secure 0 0 0) 0x7f28e00c2ca0 con 0x7f28e0157cf0 2024-01-24T14:13:55.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.190+0000 7f28e5107700 1 -- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28cc05f270 msgr2=0x7f28cc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.190+0000 7f28e5107700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28cc05f270 0x7f28cc061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f28d0009ab0 tx=0x7f28d0009840).stop 2024-01-24T14:13:55.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.190+0000 7f28e5107700 1 -- 172.21.15.29:0/75180379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e0157cf0 msgr2=0x7f28e00b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.190+0000 7f28e5107700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e0157cf0 0x7f28e00b0a30 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f28d800c490 tx=0x7f28d8007dc0).stop 2024-01-24T14:13:55.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.190+0000 7f28e5107700 1 -- 172.21.15.29:0/75180379 shutdown_connections 2024-01-24T14:13:55.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.190+0000 7f28e5107700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28cc05f270 0x7f28cc061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.190+0000 7f28e5107700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e0157cf0 0x7f28e00b0a30 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.190+0000 7f28e5107700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28e00a9d20 0x7f28e00c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.190+0000 7f28e5107700 1 --2- 172.21.15.29:0/75180379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e0154240 0x7f28e00b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.190+0000 7f28e5107700 1 -- 172.21.15.29:0/75180379 >> 172.21.15.29:0/75180379 conn(0x7f28e00c8310 msgr2=0x7f28e0155570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:55.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.190+0000 7f28e5107700 1 -- 172.21.15.29:0/75180379 shutdown_connections 2024-01-24T14:13:55.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.190+0000 7f28e5107700 1 -- 172.21.15.29:0/75180379 wait complete. 2024-01-24T14:13:55.192 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-01-24T14:13:55.206 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-24T14:13:55.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.342+0000 7f64d57a7700 1 Processor -- start 2024-01-24T14:13:55.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.342+0000 7f64d57a7700 1 -- start start 2024-01-24T14:13:55.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64d57a7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f64d014ae20 0x7f64d0154f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:55.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64d57a7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f64d00a81d0 0x7f64d0067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:55.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64d57a7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d01584a0 con 0x7f64d014b200 2024-01-24T14:13:55.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64d57a7700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d0065880 con 0x7f64d00a81d0 2024-01-24T14:13:55.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64d57a7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d0063e40 con 0x7f64d014ae20 2024-01-24T14:13:55.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64cf7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f64d00a81d0 0x7f64d0067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:55.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64cf7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f64d00a81d0 0x7f64d0067a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34194/0 (socket says 172.21.15.29:34194) 2024-01-24T14:13:55.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64cf7fe700 1 -- 172.21.15.29:0/2867735999 learned_addr learned my addr 172.21.15.29:0/2867735999 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:55.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64ceffd700 1 --2- 172.21.15.29:0/2867735999 >> v2:172.21.15.163:3300/0 conn(0x7f64d014ae20 0x7f64d0154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:55.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64cf7fe700 1 -- 172.21.15.29:0/2867735999 >> v2:172.21.15.163:3300/0 conn(0x7f64d014ae20 msgr2=0x7f64d0154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:55.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64cf7fe700 1 --2- 172.21.15.29:0/2867735999 >> v2:172.21.15.163:3300/0 conn(0x7f64d014ae20 0x7f64d0154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64cf7fe700 1 -- 172.21.15.29:0/2867735999 >> v1:172.21.15.163:6789/0 conn(0x7f64d014b200 legacy=0x7f64d00a7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.343+0000 7f64cf7fe700 1 -- 172.21.15.29:0/2867735999 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d0157b70 con 0x7f64d00a81d0 2024-01-24T14:13:55.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.344+0000 7f64cf7fe700 1 --2- 172.21.15.29:0/2867735999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d00a81d0 0x7f64d0067a30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f64d0069c70 tx=0x7f64c8009840).ready entity=mon.0 client_cookie=3dfadef4ca9c94b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:55.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.344+0000 7f64cdffb700 1 -- 172.21.15.29:0/2867735999 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800e070 con 0x7f64d00a81d0 2024-01-24T14:13:55.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.344+0000 7f64cdffb700 1 -- 172.21.15.29:0/2867735999 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c800ba50 con 0x7f64d00a81d0 2024-01-24T14:13:55.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.344+0000 7f64cdffb700 1 -- 172.21.15.29:0/2867735999 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800bdd0 con 0x7f64d00a81d0 2024-01-24T14:13:55.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.344+0000 7f64d57a7700 1 -- 172.21.15.29:0/2867735999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d00a81d0 msgr2=0x7f64d0067a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.344+0000 7f64d57a7700 1 --2- 172.21.15.29:0/2867735999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d00a81d0 0x7f64d0067a30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f64d0069c70 tx=0x7f64c8009840).stop 2024-01-24T14:13:55.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.344+0000 7f64d57a7700 1 -- 172.21.15.29:0/2867735999 shutdown_connections 2024-01-24T14:13:55.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.344+0000 7f64d57a7700 1 --2- 172.21.15.29:0/2867735999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d00a81d0 0x7f64d0067a30 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.344+0000 7f64d57a7700 1 --2- 172.21.15.29:0/2867735999 >> v2:172.21.15.163:3300/0 conn(0x7f64d014ae20 0x7f64d0154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.344+0000 7f64d57a7700 1 -- 172.21.15.29:0/2867735999 >> 172.21.15.29:0/2867735999 conn(0x7f64d00c8bb0 msgr2=0x7f64d00c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:55.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.344+0000 7f64d57a7700 1 -- 172.21.15.29:0/2867735999 shutdown_connections 2024-01-24T14:13:55.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.344+0000 7f64d57a7700 1 -- 172.21.15.29:0/2867735999 wait complete. 2024-01-24T14:13:55.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.346+0000 7f64d57a7700 1 Processor -- start 2024-01-24T14:13:55.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.346+0000 7f64d57a7700 1 -- start start 2024-01-24T14:13:55.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64d57a7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d00a81d0 0x7f64d00a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:55.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64d57a7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d014ae20 0x7f64d00a72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:55.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64d57a7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64d014b200 0x7f64d014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:55.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64d57a7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00a5c50 con 0x7f64d00a81d0 2024-01-24T14:13:55.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64d57a7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00a7790 con 0x7f64d014ae20 2024-01-24T14:13:55.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64d57a7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00a5fb0 con 0x7f64d014b200 2024-01-24T14:13:55.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64ceffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d00a81d0 0x7f64d00a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:55.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64ceffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d00a81d0 0x7f64d00a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34208/0 (socket says 172.21.15.29:34208) 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64ceffd700 1 -- 172.21.15.29:0/4211341177 learned_addr learned my addr 172.21.15.29:0/4211341177 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64ce7fc700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d014ae20 0x7f64d00a72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64cf7fe700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64d014b200 0x7f64d014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64ceffd700 1 -- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64d014b200 msgr2=0x7f64d014ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64ceffd700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64d014b200 0x7f64d014ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64ceffd700 1 -- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d014ae20 msgr2=0x7f64d00a72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64ceffd700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d014ae20 0x7f64d00a72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64ceffd700 1 -- 172.21.15.29:0/4211341177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8009ae0 con 0x7f64d00a81d0 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64ceffd700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d00a81d0 0x7f64d00a5760 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f64b800d890 tx=0x7f64b8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64c6ffd700 1 -- 172.21.15.29:0/4211341177 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8011590 con 0x7f64d00a81d0 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.347+0000 7f64c6ffd700 1 -- 172.21.15.29:0/4211341177 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b80116f0 con 0x7f64d00a81d0 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.348+0000 7f64d57a7700 1 -- 172.21.15.29:0/4211341177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d00a6e10 con 0x7f64d00a81d0 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.348+0000 7f64c6ffd700 1 -- 172.21.15.29:0/4211341177 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8011a70 con 0x7f64d00a81d0 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.348+0000 7f64d57a7700 1 -- 172.21.15.29:0/4211341177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d00a6280 con 0x7f64d00a81d0 2024-01-24T14:13:55.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.349+0000 7f64d57a7700 1 -- 172.21.15.29:0/4211341177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d00475e0 con 0x7f64d00a81d0 2024-01-24T14:13:55.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.349+0000 7f64c6ffd700 1 -- 172.21.15.29:0/4211341177 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f64b8012420 con 0x7f64d00a81d0 2024-01-24T14:13:55.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.349+0000 7f64c6ffd700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f64b005f270 0x7f64b0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:55.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.349+0000 7f64c6ffd700 1 -- 172.21.15.29:0/4211341177 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f64b8079df0 con 0x7f64d00a81d0 2024-01-24T14:13:55.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.350+0000 7f64ce7fc700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f64b005f270 0x7f64b0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:55.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.350+0000 7f64ce7fc700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f64b005f270 0x7f64b0061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f64d0152470 tx=0x7f64c0007d40).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:55.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.354+0000 7f64c6ffd700 1 -- 172.21.15.29:0/4211341177 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f64b804ae30 con 0x7f64d00a81d0 2024-01-24T14:13:55.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.608+0000 7f64d57a7700 1 -- 172.21.15.29:0/4211341177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f64d0157e50 con 0x7f64d00a81d0 2024-01-24T14:13:55.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.609+0000 7f64c6ffd700 1 -- 172.21.15.29:0/4211341177 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f64b804e450 con 0x7f64d00a81d0 2024-01-24T14:13:55.609 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew==", 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ==", 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw==", 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:55.611 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ==", 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ==", 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA==", 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:55.613 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw==", 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-01-24T14:13:55.614 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ==", 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw==", 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ==", 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A==", 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:55.616 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ==", 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-01-24T14:13:55.617 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw==", 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA==", 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:13:55.618 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g==", 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.619 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ==", 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlxK1bFxAAZpfJn+Tcpmq7R9Yq29FvVg==", 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:13:55.620 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw==", 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:55.621 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-01-24T14:13:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA==", 2024-01-24T14:13:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:13:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:13:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T14:13:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:13:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:55.622 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-01-24T14:13:55.622 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T14:13:55.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.611+0000 7f64d57a7700 1 -- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f64b005f270 msgr2=0x7f64b0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.611+0000 7f64d57a7700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f64b005f270 0x7f64b0061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f64d0152470 tx=0x7f64c0007d40).stop 2024-01-24T14:13:55.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.611+0000 7f64d57a7700 1 -- 172.21.15.29:0/4211341177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d00a81d0 msgr2=0x7f64d00a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.611+0000 7f64d57a7700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d00a81d0 0x7f64d00a5760 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f64b800d890 tx=0x7f64b8008040).stop 2024-01-24T14:13:55.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.611+0000 7f64d57a7700 1 -- 172.21.15.29:0/4211341177 shutdown_connections 2024-01-24T14:13:55.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.611+0000 7f64d57a7700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64d014b200 0x7f64d014ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.611+0000 7f64d57a7700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f64b005f270 0x7f64b0061730 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.611+0000 7f64d57a7700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d00a81d0 0x7f64d00a5760 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.611+0000 7f64d57a7700 1 --2- 172.21.15.29:0/4211341177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d014ae20 0x7f64d00a72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.611+0000 7f64d57a7700 1 -- 172.21.15.29:0/4211341177 >> 172.21.15.29:0/4211341177 conn(0x7f64d00c8bb0 msgr2=0x7f64d0153d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:55.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.611+0000 7f64d57a7700 1 -- 172.21.15.29:0/4211341177 shutdown_connections 2024-01-24T14:13:55.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.611+0000 7f64d57a7700 1 -- 172.21.15.29:0/4211341177 wait complete. 2024-01-24T14:13:55.625 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:13:55.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.755+0000 7f55bd16a700 1 Processor -- start 2024-01-24T14:13:55.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.755+0000 7f55bd16a700 1 -- start start 2024-01-24T14:13:55.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.755+0000 7f55bd16a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55b8066c00 0x7f55b80bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:55.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55bd16a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b80670a0 0x7f55b8158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:55.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55bd16a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b81582b0 con 0x7f55b80c4a10 2024-01-24T14:13:55.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55bd16a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b8157ca0 con 0x7f55b8066c00 2024-01-24T14:13:55.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55bd16a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8063c60 con 0x7f55b80670a0 2024-01-24T14:13:55.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55b759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b80670a0 0x7f55b8158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:55.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55b659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55b8066c00 0x7f55b80bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:55.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55b759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b80670a0 0x7f55b8158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:47658/0 (socket says 172.21.15.29:47658) 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55b759e700 1 -- 172.21.15.29:0/638194681 learned_addr learned my addr 172.21.15.29:0/638194681 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55b659c700 1 --2- 172.21.15.29:0/638194681 >> v2:172.21.15.163:3300/0 conn(0x7f55b8066c00 0x7f55b80bf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:42580/0 (socket says 172.21.15.29:42580) 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55b659c700 1 -- 172.21.15.29:0/638194681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b80670a0 msgr2=0x7f55b8158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55b659c700 1 --2- 172.21.15.29:0/638194681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b80670a0 0x7f55b8158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55b659c700 1 -- 172.21.15.29:0/638194681 >> v1:172.21.15.163:6789/0 conn(0x7f55b80c4a10 legacy=0x7f55b80664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55b659c700 1 -- 172.21.15.29:0/638194681 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8157f40 con 0x7f55b8066c00 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.756+0000 7f55b5d9b700 1 -- 172.21.15.29:0/638194681 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248821621 0 0) 0x7f55b81582b0 con 0x7f55b80c4a10 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.757+0000 7f55b659c700 1 --2- 172.21.15.29:0/638194681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b8066c00 0x7f55b80bf6d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f55ac009760 tx=0x7f55ac009060).ready entity=mon.1 client_cookie=7f6dd5e68a36d4bb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.757+0000 7f55b5d9b700 1 -- 172.21.15.29:0/638194681 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00f510 con 0x7f55b8066c00 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.757+0000 7f55b5d9b700 1 -- 172.21.15.29:0/638194681 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac00f670 con 0x7f55b8066c00 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.757+0000 7f55b5d9b700 1 -- 172.21.15.29:0/638194681 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00f9f0 con 0x7f55b8066c00 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.757+0000 7f55bd16a700 1 -- 172.21.15.29:0/638194681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b8066c00 msgr2=0x7f55b80bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.757+0000 7f55bd16a700 1 --2- 172.21.15.29:0/638194681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b8066c00 0x7f55b80bf6d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f55ac009760 tx=0x7f55ac009060).stop 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.757+0000 7f55bd16a700 1 -- 172.21.15.29:0/638194681 shutdown_connections 2024-01-24T14:13:55.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.757+0000 7f55bd16a700 1 --2- 172.21.15.29:0/638194681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b80670a0 0x7f55b8158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.757+0000 7f55bd16a700 1 --2- 172.21.15.29:0/638194681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b8066c00 0x7f55b80bf6d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.757+0000 7f55bd16a700 1 -- 172.21.15.29:0/638194681 >> 172.21.15.29:0/638194681 conn(0x7f55b80c8bb0 msgr2=0x7f55b80c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:55.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.757+0000 7f55bd16a700 1 -- 172.21.15.29:0/638194681 shutdown_connections 2024-01-24T14:13:55.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.757+0000 7f55bd16a700 1 -- 172.21.15.29:0/638194681 wait complete. 2024-01-24T14:13:55.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.759+0000 7f55bd16a700 1 Processor -- start 2024-01-24T14:13:55.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.759+0000 7f55bd16a700 1 -- start start 2024-01-24T14:13:55.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.759+0000 7f55bd16a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b8066c00 0x7f55b8152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:55.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.759+0000 7f55bd16a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b80670a0 0x7f55b80aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:55.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55bd16a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b80c4a10 0x7f55b81519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:55.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55bd16a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b81677c0 con 0x7f55b80670a0 2024-01-24T14:13:55.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55bd16a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8167490 con 0x7f55b80c4a10 2024-01-24T14:13:55.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55bd16a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b80a49b0 con 0x7f55b8066c00 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55b659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b80670a0 0x7f55b80aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55b659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b80670a0 0x7f55b80aaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34220/0 (socket says 172.21.15.29:34220) 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55b659c700 1 -- 172.21.15.29:0/2108156652 learned_addr learned my addr 172.21.15.29:0/2108156652 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55b759e700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b80c4a10 0x7f55b81519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55b6d9d700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b8066c00 0x7f55b8152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55b659c700 1 -- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b8066c00 msgr2=0x7f55b8152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55b659c700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b8066c00 0x7f55b8152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55b659c700 1 -- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b80c4a10 msgr2=0x7f55b81519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55b659c700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b80c4a10 0x7f55b81519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55b659c700 1 -- 172.21.15.29:0/2108156652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac0092a0 con 0x7f55b80670a0 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f55b659c700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b80670a0 0x7f55b80aaf90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f55ac009e00 tx=0x7f55ac017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f559effd700 1 -- 172.21.15.29:0/2108156652 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00fe80 con 0x7f55b80670a0 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f559effd700 1 -- 172.21.15.29:0/2108156652 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac00f7e0 con 0x7f55b80670a0 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.760+0000 7f559effd700 1 -- 172.21.15.29:0/2108156652 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac01b640 con 0x7f55b80670a0 2024-01-24T14:13:55.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.761+0000 7f55bd16a700 1 -- 172.21.15.29:0/2108156652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b80a4c90 con 0x7f55b80670a0 2024-01-24T14:13:55.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.761+0000 7f55bd16a700 1 -- 172.21.15.29:0/2108156652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b8151ec0 con 0x7f55b80670a0 2024-01-24T14:13:55.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.762+0000 7f55bd16a700 1 -- 172.21.15.29:0/2108156652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b8167930 con 0x7f55b80670a0 2024-01-24T14:13:55.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.762+0000 7f559effd700 1 -- 172.21.15.29:0/2108156652 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f55ac01ba30 con 0x7f55b80670a0 2024-01-24T14:13:55.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.762+0000 7f559effd700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55a405f270 0x7f55a4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:55.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.763+0000 7f559effd700 1 -- 172.21.15.29:0/2108156652 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f55ac07bbf0 con 0x7f55b80670a0 2024-01-24T14:13:55.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.763+0000 7f55b6d9d700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55a405f270 0x7f55a4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:55.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.763+0000 7f55b6d9d700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55a405f270 0x7f55a4061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f55b80a7900 tx=0x7f55a8009040).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:55.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.767+0000 7f559effd700 1 -- 172.21.15.29:0/2108156652 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f55ac082080 con 0x7f55b80670a0 2024-01-24T14:13:55.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.948+0000 7f55bd16a700 1 -- 172.21.15.29:0/2108156652 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55b80475e0 con 0x7f55b80670a0 2024-01-24T14:13:55.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.949+0000 7f559effd700 1 -- 172.21.15.29:0/2108156652 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+14956 (secure 0 0 0) 0x7f55ac01f020 con 0x7f55b80670a0 2024-01-24T14:13:55.950 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:55.950 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":101,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:13:55.184431+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:13:55.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.951+0000 7f55bd16a700 1 -- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55a405f270 msgr2=0x7f55a4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.951+0000 7f55bd16a700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55a405f270 0x7f55a4061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f55b80a7900 tx=0x7f55a8009040).stop 2024-01-24T14:13:55.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.951+0000 7f55bd16a700 1 -- 172.21.15.29:0/2108156652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b80670a0 msgr2=0x7f55b80aaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:55.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.951+0000 7f55bd16a700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b80670a0 0x7f55b80aaf90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f55ac009e00 tx=0x7f55ac017000).stop 2024-01-24T14:13:55.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.951+0000 7f55bd16a700 1 -- 172.21.15.29:0/2108156652 shutdown_connections 2024-01-24T14:13:55.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.951+0000 7f55bd16a700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b80c4a10 0x7f55b81519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.951+0000 7f55bd16a700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b8066c00 0x7f55b8152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.951+0000 7f55bd16a700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55a405f270 0x7f55a4061730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.951+0000 7f55bd16a700 1 --2- 172.21.15.29:0/2108156652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b80670a0 0x7f55b80aaf90 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:55.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.951+0000 7f55bd16a700 1 -- 172.21.15.29:0/2108156652 >> 172.21.15.29:0/2108156652 conn(0x7f55b80c8bb0 msgr2=0x7f55b80c55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:55.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.952+0000 7f55bd16a700 1 -- 172.21.15.29:0/2108156652 shutdown_connections 2024-01-24T14:13:55.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:55.952+0000 7f55bd16a700 1 -- 172.21.15.29:0/2108156652 wait complete. 2024-01-24T14:13:55.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T14:13:55.966 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T14:13:56.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.099+0000 7fdd8ed8b700 1 Processor -- start 2024-01-24T14:13:56.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.099+0000 7fdd8ed8b700 1 -- start start 2024-01-24T14:13:56.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.100+0000 7fdd8ed8b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd880664c0 0x7fdd88066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.100+0000 7fdd8ed8b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdd880bdb30 0x7fdd880bdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.100+0000 7fdd8ed8b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd880686a0 con 0x7fdd88134ea0 2024-01-24T14:13:56.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.100+0000 7fdd8ed8b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd88066e30 con 0x7fdd880bdb30 2024-01-24T14:13:56.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.100+0000 7fdd8ed8b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880c38e0 con 0x7fdd880664c0 2024-01-24T14:13:56.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.100+0000 7fdd87fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd880664c0 0x7fdd88066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.100+0000 7fdd8d328700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdd880bdb30 0x7fdd880bdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.100+0000 7fdd8d328700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdd880bdb30 0x7fdd880bdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:42596/0 (socket says 172.21.15.29:42596) 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.100+0000 7fdd8d328700 1 -- 172.21.15.29:0/732796203 learned_addr learned my addr 172.21.15.29:0/732796203 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.100+0000 7fdd87fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd880664c0 0x7fdd88066940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:47680/0 (socket says 172.21.15.29:47680) 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.100+0000 7fdd8cb27700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fdd88134ea0 0x7fdd880c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:40876/0 (socket says 172.21.15.29:40876) 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd8d328700 1 -- 172.21.15.29:0/732796203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd880664c0 msgr2=0x7fdd88066940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd8d328700 1 --2- 172.21.15.29:0/732796203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd880664c0 0x7fdd88066940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd8d328700 1 -- 172.21.15.29:0/732796203 >> v1:172.21.15.163:6789/0 conn(0x7fdd88134ea0 legacy=0x7fdd880c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd8d328700 1 -- 172.21.15.29:0/732796203 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd88068b30 con 0x7fdd880bdb30 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd877fe700 1 -- 172.21.15.29:0/732796203 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153424686 0 0) 0x7fdd880686a0 con 0x7fdd88134ea0 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd8d328700 1 --2- 172.21.15.29:0/732796203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd880bdb30 0x7fdd880bdf90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fdd80009f20 tx=0x7fdd80009840).ready entity=mon.1 client_cookie=3c267578913ba0fa server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd877fe700 1 -- 172.21.15.29:0/732796203 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8000e070 con 0x7fdd880bdb30 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd877fe700 1 -- 172.21.15.29:0/732796203 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8000b9d0 con 0x7fdd880bdb30 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd877fe700 1 -- 172.21.15.29:0/732796203 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8000bd50 con 0x7fdd880bdb30 2024-01-24T14:13:56.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/732796203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd880bdb30 msgr2=0x7fdd880bdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd8ed8b700 1 --2- 172.21.15.29:0/732796203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd880bdb30 0x7fdd880bdf90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fdd80009f20 tx=0x7fdd80009840).stop 2024-01-24T14:13:56.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/732796203 shutdown_connections 2024-01-24T14:13:56.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd8ed8b700 1 --2- 172.21.15.29:0/732796203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd880bdb30 0x7fdd880bdf90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd8ed8b700 1 --2- 172.21.15.29:0/732796203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd880664c0 0x7fdd88066940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.101+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/732796203 >> 172.21.15.29:0/732796203 conn(0x7fdd880c8bd0 msgr2=0x7fdd880c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:56.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.102+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/732796203 shutdown_connections 2024-01-24T14:13:56.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.102+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/732796203 wait complete. 2024-01-24T14:13:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.103+0000 7fdd8ed8b700 1 Processor -- start 2024-01-24T14:13:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.103+0000 7fdd8ed8b700 1 -- start start 2024-01-24T14:13:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8ed8b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd880664c0 0x7fdd880c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8ed8b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd880bdb30 0x7fdd880ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8ed8b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd88134ea0 0x7fdd88162740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8ed8b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880c21c0 con 0x7fdd880664c0 2024-01-24T14:13:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8ed8b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880ad0e0 con 0x7fdd880bdb30 2024-01-24T14:13:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8ed8b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880ace50 con 0x7fdd88134ea0 2024-01-24T14:13:56.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8cb27700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd880664c0 0x7fdd880c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8cb27700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd880664c0 0x7fdd880c0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34226/0 (socket says 172.21.15.29:34226) 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8cb27700 1 -- 172.21.15.29:0/3730248130 learned_addr learned my addr 172.21.15.29:0/3730248130 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8d328700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd88134ea0 0x7fdd88162740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd87fff700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd880bdb30 0x7fdd880ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8cb27700 1 -- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd88134ea0 msgr2=0x7fdd88162740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8cb27700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd88134ea0 0x7fdd88162740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8cb27700 1 -- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd880bdb30 msgr2=0x7fdd880ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8cb27700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd880bdb30 0x7fdd880ac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.104+0000 7fdd8cb27700 1 -- 172.21.15.29:0/3730248130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd80009ae0 con 0x7fdd880664c0 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.105+0000 7fdd8cb27700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd880664c0 0x7fdd880c0ef0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7fdd7c00c560 tx=0x7fdd7c007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.105+0000 7fdd84ff9700 1 -- 172.21.15.29:0/3730248130 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c0072e0 con 0x7fdd880664c0 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.105+0000 7fdd84ff9700 1 -- 172.21.15.29:0/3730248130 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7c007440 con 0x7fdd880664c0 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.105+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/3730248130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd880c1890 con 0x7fdd880664c0 2024-01-24T14:13:56.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.105+0000 7fdd84ff9700 1 -- 172.21.15.29:0/3730248130 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c0058f0 con 0x7fdd880664c0 2024-01-24T14:13:56.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.105+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/3730248130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd880c1e50 con 0x7fdd880664c0 2024-01-24T14:13:56.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.106+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/3730248130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd88065c50 con 0x7fdd880664c0 2024-01-24T14:13:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.106+0000 7fdd84ff9700 1 -- 172.21.15.29:0/3730248130 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdd7c005ce0 con 0x7fdd880664c0 2024-01-24T14:13:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.107+0000 7fdd84ff9700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdd6c05f270 0x7fdd6c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.107+0000 7fdd84ff9700 1 -- 172.21.15.29:0/3730248130 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6290+0+0 (secure 0 0 0) 0x7fdd7c079b80 con 0x7fdd880664c0 2024-01-24T14:13:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.107+0000 7fdd87fff700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdd6c05f270 0x7fdd6c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.108+0000 7fdd87fff700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdd6c05f270 0x7fdd6c061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fdd8814fb70 tx=0x7fdd74009060).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:56.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.111+0000 7fdd84ff9700 1 -- 172.21.15.29:0/3730248130 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdd7c04ac20 con 0x7fdd880664c0 2024-01-24T14:13:56.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.293+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/3730248130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdd880475e0 con 0x7fdd880664c0 2024-01-24T14:13:56.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.294+0000 7fdd84ff9700 1 -- 172.21.15.29:0/3730248130 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v101) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdd7c04e240 con 0x7fdd880664c0 2024-01-24T14:13:56.295 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T14:13:56.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.296+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdd6c05f270 msgr2=0x7fdd6c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.296+0000 7fdd8ed8b700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdd6c05f270 0x7fdd6c061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fdd8814fb70 tx=0x7fdd74009060).stop 2024-01-24T14:13:56.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.296+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/3730248130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd880664c0 msgr2=0x7fdd880c0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.296+0000 7fdd8ed8b700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd880664c0 0x7fdd880c0ef0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7fdd7c00c560 tx=0x7fdd7c007c90).stop 2024-01-24T14:13:56.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.296+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/3730248130 shutdown_connections 2024-01-24T14:13:56.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.296+0000 7fdd8ed8b700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd88134ea0 0x7fdd88162740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.296+0000 7fdd8ed8b700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdd6c05f270 0x7fdd6c061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.296+0000 7fdd8ed8b700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd880664c0 0x7fdd880c0ef0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.296+0000 7fdd8ed8b700 1 --2- 172.21.15.29:0/3730248130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd880bdb30 0x7fdd880ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.296+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/3730248130 >> 172.21.15.29:0/3730248130 conn(0x7fdd880c8bd0 msgr2=0x7fdd88150b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:56.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.296+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/3730248130 shutdown_connections 2024-01-24T14:13:56.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.296+0000 7fdd8ed8b700 1 -- 172.21.15.29:0/3730248130 wait complete. 2024-01-24T14:13:56.310 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T14:13:56.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.443+0000 7fbc8a17a700 1 Processor -- start 2024-01-24T14:13:56.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.443+0000 7fbc8a17a700 1 -- start start 2024-01-24T14:13:56.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.444+0000 7fbc8a17a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc840664c0 0x7fbc84066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.444+0000 7fbc8a17a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840bdb30 0x7fbc840bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.444+0000 7fbc8a17a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc840686a0 con 0x7fbc84134ea0 2024-01-24T14:13:56.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.444+0000 7fbc8a17a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc84066e30 con 0x7fbc840664c0 2024-01-24T14:13:56.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.444+0000 7fbc8a17a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840c38e0 con 0x7fbc840bdb30 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.444+0000 7fbc82ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc840664c0 0x7fbc84066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.444+0000 7fbc82ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc840664c0 0x7fbc84066940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34236/0 (socket says 172.21.15.29:34236) 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.444+0000 7fbc837fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbc84134ea0 0x7fbc840c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38796/0 (socket says 172.21.15.29:38796) 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.444+0000 7fbc82ffd700 1 -- 172.21.15.29:0/1229185829 learned_addr learned my addr 172.21.15.29:0/1229185829 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.444+0000 7fbc83fff700 1 --2- 172.21.15.29:0/1229185829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840bdb30 0x7fbc840bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.445+0000 7fbc82ffd700 1 -- 172.21.15.29:0/1229185829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840bdb30 msgr2=0x7fbc840bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.445+0000 7fbc82ffd700 1 --2- 172.21.15.29:0/1229185829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840bdb30 0x7fbc840bdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.445+0000 7fbc82ffd700 1 -- 172.21.15.29:0/1229185829 >> v1:172.21.15.29:6789/0 conn(0x7fbc84134ea0 legacy=0x7fbc840c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.445+0000 7fbc827fc700 1 -- 172.21.15.29:0/1229185829 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1353904161 0 0) 0x7fbc840686a0 con 0x7fbc84134ea0 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.445+0000 7fbc82ffd700 1 -- 172.21.15.29:0/1229185829 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc84068b30 con 0x7fbc840664c0 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.445+0000 7fbc82ffd700 1 --2- 172.21.15.29:0/1229185829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc840664c0 0x7fbc84066940 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7fbc6c006fd0 tx=0x7fbc6c0090f0).ready entity=mon.0 client_cookie=79345b3753d1891d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.445+0000 7fbc827fc700 1 -- 172.21.15.29:0/1229185829 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c00b600 con 0x7fbc840664c0 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.445+0000 7fbc827fc700 1 -- 172.21.15.29:0/1229185829 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6c00b760 con 0x7fbc840664c0 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.445+0000 7fbc827fc700 1 -- 172.21.15.29:0/1229185829 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c00bae0 con 0x7fbc840664c0 2024-01-24T14:13:56.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.445+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1229185829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc840664c0 msgr2=0x7fbc84066940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.445+0000 7fbc8a17a700 1 --2- 172.21.15.29:0/1229185829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc840664c0 0x7fbc84066940 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7fbc6c006fd0 tx=0x7fbc6c0090f0).stop 2024-01-24T14:13:56.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.446+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1229185829 shutdown_connections 2024-01-24T14:13:56.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.446+0000 7fbc8a17a700 1 --2- 172.21.15.29:0/1229185829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840bdb30 0x7fbc840bdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.446+0000 7fbc8a17a700 1 --2- 172.21.15.29:0/1229185829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc840664c0 0x7fbc84066940 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.446+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1229185829 >> 172.21.15.29:0/1229185829 conn(0x7fbc840c8bd0 msgr2=0x7fbc840c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:56.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.446+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1229185829 shutdown_connections 2024-01-24T14:13:56.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.446+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1229185829 wait complete. 2024-01-24T14:13:56.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.447+0000 7fbc8a17a700 1 Processor -- start 2024-01-24T14:13:56.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.447+0000 7fbc8a17a700 1 -- start start 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc8a17a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc840664c0 0x7fbc84163eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc8a17a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840bdb30 0x7fbc840aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc8a17a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc84134ea0 0x7fbc841679d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc8a17a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840aa110 con 0x7fbc840664c0 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc8a17a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840aaa90 con 0x7fbc84134ea0 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc8a17a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840a9500 con 0x7fbc840bdb30 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc837fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc840664c0 0x7fbc84163eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc837fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc840664c0 0x7fbc84163eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34242/0 (socket says 172.21.15.29:34242) 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc837fe700 1 -- 172.21.15.29:0/1724559936 learned_addr learned my addr 172.21.15.29:0/1724559936 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc82ffd700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840bdb30 0x7fbc840aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc83fff700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc84134ea0 0x7fbc841679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc837fe700 1 -- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840bdb30 msgr2=0x7fbc840aa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc837fe700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840bdb30 0x7fbc840aa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc837fe700 1 -- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc84134ea0 msgr2=0x7fbc841679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc837fe700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc84134ea0 0x7fbc841679d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc837fe700 1 -- 172.21.15.29:0/1724559936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c009370 con 0x7fbc840664c0 2024-01-24T14:13:56.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.448+0000 7fbc83fff700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc84134ea0 0x7fbc841679d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:56.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.449+0000 7fbc837fe700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc840664c0 0x7fbc84163eb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7fbc7400c560 tx=0x7fbc74007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:56.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.449+0000 7fbc7b7fe700 1 -- 172.21.15.29:0/1724559936 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc740072e0 con 0x7fbc840664c0 2024-01-24T14:13:56.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.449+0000 7fbc7b7fe700 1 -- 172.21.15.29:0/1724559936 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc74007440 con 0x7fbc840664c0 2024-01-24T14:13:56.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.449+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1724559936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc840a97e0 con 0x7fbc840664c0 2024-01-24T14:13:56.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.449+0000 7fbc7b7fe700 1 -- 172.21.15.29:0/1724559936 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc740058f0 con 0x7fbc840664c0 2024-01-24T14:13:56.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.449+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1724559936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc840a9da0 con 0x7fbc840664c0 2024-01-24T14:13:56.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.450+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1724559936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc840475e0 con 0x7fbc840664c0 2024-01-24T14:13:56.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.450+0000 7fbc7b7fe700 1 -- 172.21.15.29:0/1724559936 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbc74005ca0 con 0x7fbc840664c0 2024-01-24T14:13:56.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.450+0000 7fbc7b7fe700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbc6405f270 0x7fbc64061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.451+0000 7fbc7b7fe700 1 -- 172.21.15.29:0/1724559936 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6290+0+0 (secure 0 0 0) 0x7fbc74079cb0 con 0x7fbc840664c0 2024-01-24T14:13:56.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.451+0000 7fbc82ffd700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbc6405f270 0x7fbc64061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.451+0000 7fbc82ffd700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbc6405f270 0x7fbc64061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fbc6c009130 tx=0x7fbc6c017000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:56.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.455+0000 7fbc7b7fe700 1 -- 172.21.15.29:0/1724559936 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbc7404ad50 con 0x7fbc840664c0 2024-01-24T14:13:56.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.633+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1724559936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbc84067a90 con 0x7fbc840664c0 2024-01-24T14:13:56.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.634+0000 7fbc7b7fe700 1 -- 172.21.15.29:0/1724559936 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v101) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbc74018030 con 0x7fbc840664c0 2024-01-24T14:13:56.634 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T14:13:56.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.636+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbc6405f270 msgr2=0x7fbc64061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.636+0000 7fbc8a17a700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbc6405f270 0x7fbc64061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fbc6c009130 tx=0x7fbc6c017000).stop 2024-01-24T14:13:56.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.636+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1724559936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc840664c0 msgr2=0x7fbc84163eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.636+0000 7fbc8a17a700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc840664c0 0x7fbc84163eb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7fbc7400c560 tx=0x7fbc74007c90).stop 2024-01-24T14:13:56.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.636+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1724559936 shutdown_connections 2024-01-24T14:13:56.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.636+0000 7fbc8a17a700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc84134ea0 0x7fbc841679d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.636+0000 7fbc8a17a700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbc6405f270 0x7fbc64061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.636+0000 7fbc8a17a700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc840664c0 0x7fbc84163eb0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.636+0000 7fbc8a17a700 1 --2- 172.21.15.29:0/1724559936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840bdb30 0x7fbc840aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.636+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1724559936 >> 172.21.15.29:0/1724559936 conn(0x7fbc840c8bd0 msgr2=0x7fbc840bdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:56.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.636+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1724559936 shutdown_connections 2024-01-24T14:13:56.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.636+0000 7fbc8a17a700 1 -- 172.21.15.29:0/1724559936 wait complete. 2024-01-24T14:13:56.650 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-24T14:13:56.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.779+0000 7f924eaf8700 1 Processor -- start 2024-01-24T14:13:56.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.779+0000 7f924eaf8700 1 -- start start 2024-01-24T14:13:56.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.780+0000 7f924eaf8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9248066c00 0x7f92480bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.780+0000 7f924eaf8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f92480670a0 0x7f9248158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.780+0000 7f924eaf8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92481582b0 con 0x7f92480c4a10 2024-01-24T14:13:56.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.780+0000 7f924eaf8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9248157ca0 con 0x7f92480670a0 2024-01-24T14:13:56.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.780+0000 7f924eaf8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9248063c60 con 0x7f9248066c00 2024-01-24T14:13:56.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.780+0000 7f924d095700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f92480670a0 0x7f9248158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.780+0000 7f924c894700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f92480c4a10 0x7f92480664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38810/0 (socket says 172.21.15.29:38810) 2024-01-24T14:13:56.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.780+0000 7f924c894700 1 -- 172.21.15.29:0/1768609924 learned_addr learned my addr 172.21.15.29:0/1768609924 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.780+0000 7f9247fff700 1 --2- 172.21.15.29:0/1768609924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9248066c00 0x7f92480bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.781+0000 7f924d095700 1 -- 172.21.15.29:0/1768609924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9248066c00 msgr2=0x7f92480bf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.781+0000 7f924d095700 1 --2- 172.21.15.29:0/1768609924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9248066c00 0x7f92480bf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.781+0000 7f924d095700 1 -- 172.21.15.29:0/1768609924 >> v1:172.21.15.29:6789/0 conn(0x7f92480c4a10 legacy=0x7f92480664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.781+0000 7f924d095700 1 -- 172.21.15.29:0/1768609924 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9248157f40 con 0x7f92480670a0 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.781+0000 7f92477fe700 1 -- 172.21.15.29:0/1768609924 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033591407 0 0) 0x7f92481582b0 con 0x7f92480c4a10 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.781+0000 7f924d095700 1 --2- 172.21.15.29:0/1768609924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92480670a0 0x7f9248158740 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f9240009e60 tx=0x7f9240009780).ready entity=mon.0 client_cookie=aad49f77af12fb76 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.781+0000 7f92477fe700 1 -- 172.21.15.29:0/1768609924 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924000e070 con 0x7f92480670a0 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.781+0000 7f92477fe700 1 -- 172.21.15.29:0/1768609924 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924000bb60 con 0x7f92480670a0 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.781+0000 7f92477fe700 1 -- 172.21.15.29:0/1768609924 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9240012410 con 0x7f92480670a0 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.781+0000 7f924eaf8700 1 -- 172.21.15.29:0/1768609924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92480670a0 msgr2=0x7f9248158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.781+0000 7f924eaf8700 1 --2- 172.21.15.29:0/1768609924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92480670a0 0x7f9248158740 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f9240009e60 tx=0x7f9240009780).stop 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.782+0000 7f924eaf8700 1 -- 172.21.15.29:0/1768609924 shutdown_connections 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.782+0000 7f924eaf8700 1 --2- 172.21.15.29:0/1768609924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92480670a0 0x7f9248158740 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.782+0000 7f924eaf8700 1 --2- 172.21.15.29:0/1768609924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9248066c00 0x7f92480bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.782+0000 7f924eaf8700 1 -- 172.21.15.29:0/1768609924 >> 172.21.15.29:0/1768609924 conn(0x7f92480c8bb0 msgr2=0x7f92480c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:56.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.782+0000 7f924eaf8700 1 -- 172.21.15.29:0/1768609924 shutdown_connections 2024-01-24T14:13:56.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.782+0000 7f924eaf8700 1 -- 172.21.15.29:0/1768609924 wait complete. 2024-01-24T14:13:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.783+0000 7f924eaf8700 1 Processor -- start 2024-01-24T14:13:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.783+0000 7f924eaf8700 1 -- start start 2024-01-24T14:13:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924eaf8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9248066c00 0x7f92481645c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924eaf8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92480670a0 0x7f92480c4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924eaf8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92480c4a10 0x7f92480ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924eaf8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92480acb30 con 0x7f9248066c00 2024-01-24T14:13:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924eaf8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92480abf20 con 0x7f92480c4a10 2024-01-24T14:13:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924eaf8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92480ac200 con 0x7f92480670a0 2024-01-24T14:13:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924c894700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9248066c00 0x7f92481645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924c894700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9248066c00 0x7f92481645c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34262/0 (socket says 172.21.15.29:34262) 2024-01-24T14:13:56.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924c894700 1 -- 172.21.15.29:0/2040415666 learned_addr learned my addr 172.21.15.29:0/2040415666 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f9247fff700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92480670a0 0x7f92480c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924d095700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92480c4a10 0x7f92480ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924c894700 1 -- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92480670a0 msgr2=0x7f92480c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924c894700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92480670a0 0x7f92480c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924c894700 1 -- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92480c4a10 msgr2=0x7f92480ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924c894700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92480c4a10 0x7f92480ad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f924c894700 1 -- 172.21.15.29:0/2040415666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9240009a20 con 0x7f9248066c00 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.784+0000 7f9247fff700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92480670a0 0x7f92480c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.785+0000 7f924c894700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9248066c00 0x7f92481645c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f923c001eb0 tx=0x7f923c007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.785+0000 7f9244ff9700 1 -- 172.21.15.29:0/2040415666 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c0072e0 con 0x7f9248066c00 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.785+0000 7f9244ff9700 1 -- 172.21.15.29:0/2040415666 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c007440 con 0x7f9248066c00 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.785+0000 7f924eaf8700 1 -- 172.21.15.29:0/2040415666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92480ac4e0 con 0x7f9248066c00 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.785+0000 7f9244ff9700 1 -- 172.21.15.29:0/2040415666 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c0057a0 con 0x7f9248066c00 2024-01-24T14:13:56.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.785+0000 7f924eaf8700 1 -- 172.21.15.29:0/2040415666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9248164ab0 con 0x7f9248066c00 2024-01-24T14:13:56.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.786+0000 7f924eaf8700 1 -- 172.21.15.29:0/2040415666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92480475e0 con 0x7f9248066c00 2024-01-24T14:13:56.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.786+0000 7f9244ff9700 1 -- 172.21.15.29:0/2040415666 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f923c005b90 con 0x7f9248066c00 2024-01-24T14:13:56.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.786+0000 7f9244ff9700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f922c05f270 0x7f922c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:56.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.787+0000 7f9244ff9700 1 -- 172.21.15.29:0/2040415666 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f923c079c60 con 0x7f9248066c00 2024-01-24T14:13:56.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.787+0000 7f9247fff700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f922c05f270 0x7f922c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:56.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.787+0000 7f9247fff700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f922c05f270 0x7f922c061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f92480c07a0 tx=0x7f9234008040).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:56.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.791+0000 7f9244ff9700 1 -- 172.21.15.29:0/2040415666 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f923c04ad00 con 0x7f9248066c00 2024-01-24T14:13:56.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.974+0000 7f924eaf8700 1 -- 172.21.15.29:0/2040415666 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9248068ce0 con 0x7f9248066c00 2024-01-24T14:13:56.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.974+0000 7f9244ff9700 1 -- 172.21.15.29:0/2040415666 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) v1 ==== 84+0+143 (secure 0 0 0) 0x7f923c04e320 con 0x7f9248066c00 2024-01-24T14:13:56.975 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:56.975 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T14:13:56.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:56.975 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-01-24T14:13:56.975 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "device_health_metrics" 2024-01-24T14:13:56.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:13:56.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:13:56.976 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 2, 2024-01-24T14:13:56.976 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "rbd" 2024-01-24T14:13:56.976 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:13:56.976 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T14:13:56.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.976+0000 7f924eaf8700 1 -- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f922c05f270 msgr2=0x7f922c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.976+0000 7f924eaf8700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f922c05f270 0x7f922c061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f92480c07a0 tx=0x7f9234008040).stop 2024-01-24T14:13:56.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.976+0000 7f924eaf8700 1 -- 172.21.15.29:0/2040415666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9248066c00 msgr2=0x7f92481645c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:56.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.976+0000 7f924eaf8700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9248066c00 0x7f92481645c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f923c001eb0 tx=0x7f923c007e90).stop 2024-01-24T14:13:56.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.976+0000 7f924eaf8700 1 -- 172.21.15.29:0/2040415666 shutdown_connections 2024-01-24T14:13:56.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.976+0000 7f924eaf8700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92480c4a10 0x7f92480ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.976+0000 7f924eaf8700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f922c05f270 0x7f922c061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.976+0000 7f924eaf8700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9248066c00 0x7f92481645c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.976+0000 7f924eaf8700 1 --2- 172.21.15.29:0/2040415666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92480670a0 0x7f92480c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:56.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.976+0000 7f924eaf8700 1 -- 172.21.15.29:0/2040415666 >> 172.21.15.29:0/2040415666 conn(0x7f92480c8bb0 msgr2=0x7f92480c7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:56.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.976+0000 7f924eaf8700 1 -- 172.21.15.29:0/2040415666 shutdown_connections 2024-01-24T14:13:56.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:56.976+0000 7f924eaf8700 1 -- 172.21.15.29:0/2040415666 wait complete. 2024-01-24T14:13:56.991 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-24T14:13:56.992 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2024-01-24T14:13:57.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.129+0000 7f9f8446c700 1 Processor -- start 2024-01-24T14:13:57.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.129+0000 7f9f8446c700 1 -- start start 2024-01-24T14:13:57.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.130+0000 7f9f8446c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9f7c062d50 0x7f9f7c158720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.130+0000 7f9f8446c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7c0c3e00 con 0x7f9f7c0c2190 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.130+0000 7f9f8446c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7c0c2900 con 0x7f9f7c062970 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.130+0000 7f9f8446c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7c158c10 con 0x7f9f7c062d50 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.130+0000 7f9f81a07700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9f7c0c2190 0x7f9f7c0a4890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38816/0 (socket says 172.21.15.29:38816) 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.130+0000 7f9f81a07700 1 -- 172.21.15.29:0/1057596100 learned_addr learned my addr 172.21.15.29:0/1057596100 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f82a09700 1 --2- 172.21.15.29:0/1057596100 >> v2:172.21.15.163:3300/0 conn(0x7f9f7c062d50 0x7f9f7c158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202822427 0 0) 0x7f9f7c0c3e00 con 0x7f9f7c0c2190 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f680033f0 con 0x7f9f7c0c2190 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 88106260 0 0) 0x7f9f680033f0 con 0x7f9f7c0c2190 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 >> v2:172.21.15.163:3300/0 conn(0x7f9f7c062d50 msgr2=0x7f9f7c158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 --2- 172.21.15.29:0/1057596100 >> v2:172.21.15.163:3300/0 conn(0x7f9f7c062d50 0x7f9f7c158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 >> v1:172.21.15.163:6789/0 conn(0x7f9f7c062970 legacy=0x7f9f7c0c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c065720 con 0x7f9f7c0c2190 2024-01-24T14:13:57.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f9f6c001470 con 0x7f9f7c0c2190 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 >> v1:172.21.15.29:6789/0 conn(0x7f9f7c0c2190 legacy=0x7f9f7c0a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 --2- 172.21.15.29:0/1057596100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f680038e0 0x7f9f68003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 --2- 172.21.15.29:0/1057596100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 0x7f9f68004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 --2- 172.21.15.29:0/1057596100 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f68005000 0x7f9f6800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c158c10 con 0x7f9f68004690 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6800beb0 con 0x7f9f680038e0 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6800c070 con 0x7f9f68005000 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f81a07700 1 --2- 172.21.15.29:0/1057596100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 0x7f9f68004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f82208700 1 --2- 172.21.15.29:0/1057596100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f680038e0 0x7f9f68003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.131+0000 7f9f82a09700 1 --2- 172.21.15.29:0/1057596100 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f68005000 0x7f9f6800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f81a07700 1 -- 172.21.15.29:0/1057596100 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f68005000 msgr2=0x7f9f6800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f81a07700 1 --2- 172.21.15.29:0/1057596100 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f68005000 0x7f9f6800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f81a07700 1 -- 172.21.15.29:0/1057596100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f680038e0 msgr2=0x7f9f68003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f81a07700 1 --2- 172.21.15.29:0/1057596100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f680038e0 0x7f9f68003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:57.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f81a07700 1 -- 172.21.15.29:0/1057596100 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f680033f0 con 0x7f9f68004690 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f82a09700 1 --2- 172.21.15.29:0/1057596100 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f68005000 0x7f9f6800b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f82a09700 1 -- 172.21.15.29:0/1057596100 reap_dead start 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f81a07700 1 --2- 172.21.15.29:0/1057596100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 0x7f9f68004b10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f9f6c00c160 tx=0x7f9f7c0658b0).ready entity=mon.0 client_cookie=913eace60e9a0d03 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c00ca90 con 0x7f9f68004690 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c00cbf0 con 0x7f9f68004690 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f81206700 1 -- 172.21.15.29:0/1057596100 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c0070d0 con 0x7f9f68004690 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f8446c700 1 -- 172.21.15.29:0/1057596100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 msgr2=0x7f9f68004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f8446c700 1 --2- 172.21.15.29:0/1057596100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 0x7f9f68004b10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f9f6c00c160 tx=0x7f9f7c0658b0).stop 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f8446c700 1 -- 172.21.15.29:0/1057596100 shutdown_connections 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f8446c700 1 --2- 172.21.15.29:0/1057596100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 0x7f9f68004b10 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.132+0000 7f9f8446c700 1 -- 172.21.15.29:0/1057596100 >> 172.21.15.29:0/1057596100 conn(0x7f9f7c067b20 msgr2=0x7f9f7c0638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.133+0000 7f9f8446c700 1 -- 172.21.15.29:0/1057596100 shutdown_connections 2024-01-24T14:13:57.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.133+0000 7f9f8446c700 1 -- 172.21.15.29:0/1057596100 wait complete. 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.134+0000 7f9f8446c700 1 Processor -- start 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.134+0000 7f9f8446c700 1 -- start start 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.134+0000 7f9f8446c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 0x7f9f7c1506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.134+0000 7f9f8446c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f7c156990 0x7f9f7c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f8446c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c0ab790 0x7f9f7c0abc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f8446c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c0a7530 con 0x7f9f68004690 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f8446c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c0ac100 con 0x7f9f7c156990 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f8446c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c0a6920 con 0x7f9f7c0ab790 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f82208700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 0x7f9f7c1506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f82208700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 0x7f9f7c1506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34282/0 (socket says 172.21.15.29:34282) 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f82208700 1 -- 172.21.15.29:0/2966748524 learned_addr learned my addr 172.21.15.29:0/2966748524 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f82a09700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c0ab790 0x7f9f7c0abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:57.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f81a07700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f7c156990 0x7f9f7c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f82208700 1 -- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c0ab790 msgr2=0x7f9f7c0abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f82208700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c0ab790 0x7f9f7c0abc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f82208700 1 -- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f7c156990 msgr2=0x7f9f7c0a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f82208700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f7c156990 0x7f9f7c0a6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f82208700 1 -- 172.21.15.29:0/2966748524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c0a6c00 con 0x7f9f68004690 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f82208700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 0x7f9f7c1506b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f9f780028f0 tx=0x7f9f78007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f727fc700 1 -- 172.21.15.29:0/2966748524 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7800f640 con 0x7f9f68004690 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f727fc700 1 -- 172.21.15.29:0/2966748524 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7800f7a0 con 0x7f9f68004690 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f8446c700 1 -- 172.21.15.29:0/2966748524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7c0a6ee0 con 0x7f9f68004690 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f727fc700 1 -- 172.21.15.29:0/2966748524 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7800fb20 con 0x7f9f68004690 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.135+0000 7f9f8446c700 1 -- 172.21.15.29:0/2966748524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f7c0a6550 con 0x7f9f68004690 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.136+0000 7f9f8446c700 1 -- 172.21.15.29:0/2966748524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f7c0a7810 con 0x7f9f68004690 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.137+0000 7f9f727fc700 1 -- 172.21.15.29:0/2966748524 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9f7801d3b0 con 0x7f9f68004690 2024-01-24T14:13:57.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.137+0000 7f9f727fc700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6005f270 0x7f9f60061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:57.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.137+0000 7f9f727fc700 1 -- 172.21.15.29:0/2966748524 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f9f7807a0e0 con 0x7f9f68004690 2024-01-24T14:13:57.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.137+0000 7f9f81a07700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6005f270 0x7f9f60061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:57.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.138+0000 7f9f81a07700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6005f270 0x7f9f60061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9f7c0a4ac0 tx=0x7f9f6c015000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:57.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.141+0000 7f9f727fc700 1 -- 172.21.15.29:0/2966748524 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9f7804b1d0 con 0x7f9f68004690 2024-01-24T14:13:57.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:57.325+0000 7f9f8446c700 1 -- 172.21.15.29:0/2966748524 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f9f7c0a8c20 con 0x7f9f68004690 2024-01-24T14:13:58.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.151+0000 7f9f727fc700 1 -- 172.21.15.29:0/2966748524 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) v1 ==== 152+0+0 (secure 0 0 0) 0x7f9f7804e7f0 con 0x7f9f68004690 2024-01-24T14:13:58.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.155+0000 7f9f8446c700 1 -- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6005f270 msgr2=0x7f9f60061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:58.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.155+0000 7f9f8446c700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6005f270 0x7f9f60061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9f7c0a4ac0 tx=0x7f9f6c015000).stop 2024-01-24T14:13:58.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.155+0000 7f9f8446c700 1 -- 172.21.15.29:0/2966748524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 msgr2=0x7f9f7c1506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:58.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.155+0000 7f9f8446c700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 0x7f9f7c1506b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f9f780028f0 tx=0x7f9f78007bb0).stop 2024-01-24T14:13:58.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.156+0000 7f9f8446c700 1 -- 172.21.15.29:0/2966748524 shutdown_connections 2024-01-24T14:13:58.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.156+0000 7f9f8446c700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c0ab790 0x7f9f7c0abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:58.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.156+0000 7f9f8446c700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6005f270 0x7f9f60061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:58.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.156+0000 7f9f8446c700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f68004690 0x7f9f7c1506b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:58.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.156+0000 7f9f8446c700 1 --2- 172.21.15.29:0/2966748524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f7c156990 0x7f9f7c0a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:58.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.156+0000 7f9f8446c700 1 -- 172.21.15.29:0/2966748524 >> 172.21.15.29:0/2966748524 conn(0x7f9f7c067b20 msgr2=0x7f9f7c150da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:58.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.162+0000 7f9f8446c700 1 -- 172.21.15.29:0/2966748524 shutdown_connections 2024-01-24T14:13:58.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.162+0000 7f9f8446c700 1 -- 172.21.15.29:0/2966748524 wait complete. 2024-01-24T14:13:58.165 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-01-24T14:13:58.180 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-24T14:13:58.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.321+0000 7f1dcf813700 1 Processor -- start 2024-01-24T14:13:58.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.321+0000 7f1dcf813700 1 -- start start 2024-01-24T14:13:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcf813700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1dc8150c00 0x7f1dc80c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcf813700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1dc8150fe0 0x7f1dc8154920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcf813700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc81558a0 con 0x7f1dc8154e10 2024-01-24T14:13:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcf813700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc8066960 con 0x7f1dc8150c00 2024-01-24T14:13:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcf813700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc80c2fc0 con 0x7f1dc8150fe0 2024-01-24T14:13:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcd5af700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1dc8150c00 0x7f1dc80c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcd5af700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1dc8150c00 0x7f1dc80c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34290/0 (socket says 172.21.15.29:34290) 2024-01-24T14:13:58.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcd5af700 1 -- 172.21.15.29:0/3981714489 learned_addr learned my addr 172.21.15.29:0/3981714489 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcddb0700 1 --1- 172.21.15.29:0/3981714489 >> v1:172.21.15.29:6789/0 conn(0x7f1dc8154e10 0x7f1dc80c44c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38828/0 (socket says 172.21.15.29:38828) 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dccdae700 1 --2- 172.21.15.29:0/3981714489 >> v2:172.21.15.163:3300/0 conn(0x7f1dc8150fe0 0x7f1dc8154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcd5af700 1 -- 172.21.15.29:0/3981714489 >> v2:172.21.15.163:3300/0 conn(0x7f1dc8150fe0 msgr2=0x7f1dc8154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcd5af700 1 --2- 172.21.15.29:0/3981714489 >> v2:172.21.15.163:3300/0 conn(0x7f1dc8150fe0 0x7f1dc8154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcd5af700 1 -- 172.21.15.29:0/3981714489 >> v1:172.21.15.29:6789/0 conn(0x7f1dc8154e10 legacy=0x7f1dc80c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dbffff700 1 -- 172.21.15.29:0/3981714489 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075484298 0 0) 0x7f1dc81558a0 con 0x7f1dc8154e10 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcd5af700 1 -- 172.21.15.29:0/3981714489 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc81555c0 con 0x7f1dc8150c00 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.322+0000 7f1dcd5af700 1 --2- 172.21.15.29:0/3981714489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dc8150c00 0x7f1dc80c3450 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f1dc80c7b90 tx=0x7f1dc4009840).ready entity=mon.0 client_cookie=c514218c44ffb11c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.323+0000 7f1dbffff700 1 -- 172.21.15.29:0/3981714489 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400e070 con 0x7f1dc8150c00 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.323+0000 7f1dbffff700 1 -- 172.21.15.29:0/3981714489 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc400ba50 con 0x7f1dc8150c00 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.323+0000 7f1dbffff700 1 -- 172.21.15.29:0/3981714489 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400bdd0 con 0x7f1dc8150c00 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.323+0000 7f1dcf813700 1 -- 172.21.15.29:0/3981714489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dc8150c00 msgr2=0x7f1dc80c3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:58.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.323+0000 7f1dcf813700 1 --2- 172.21.15.29:0/3981714489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dc8150c00 0x7f1dc80c3450 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f1dc80c7b90 tx=0x7f1dc4009840).stop 2024-01-24T14:13:58.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.323+0000 7f1dcf813700 1 -- 172.21.15.29:0/3981714489 shutdown_connections 2024-01-24T14:13:58.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.323+0000 7f1dcf813700 1 --2- 172.21.15.29:0/3981714489 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dc8150c00 0x7f1dc80c3450 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:58.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.323+0000 7f1dcf813700 1 --2- 172.21.15.29:0/3981714489 >> v2:172.21.15.163:3300/0 conn(0x7f1dc8150fe0 0x7f1dc8154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:58.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.323+0000 7f1dcf813700 1 -- 172.21.15.29:0/3981714489 >> 172.21.15.29:0/3981714489 conn(0x7f1dc80638c0 msgr2=0x7f1dc8063cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:58.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.323+0000 7f1dcf813700 1 -- 172.21.15.29:0/3981714489 shutdown_connections 2024-01-24T14:13:58.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.323+0000 7f1dcf813700 1 -- 172.21.15.29:0/3981714489 wait complete. 2024-01-24T14:13:58.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.324+0000 7f1dcf813700 1 Processor -- start 2024-01-24T14:13:58.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.325+0000 7f1dcf813700 1 -- start start 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.325+0000 7f1dcf813700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dc8150c00 0x7f1dc8168220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.325+0000 7f1dcf813700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dc8150fe0 0x7f1dc8142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.325+0000 7f1dcf813700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc8154e10 0x7f1dc80c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dcf813700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80c0a70 con 0x7f1dc8150fe0 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dcf813700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80c5c90 con 0x7f1dc8150c00 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dcf813700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80c2320 con 0x7f1dc8154e10 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dccdae700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dc8150fe0 0x7f1dc8142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dccdae700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dc8150fe0 0x7f1dc8142e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34302/0 (socket says 172.21.15.29:34302) 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dccdae700 1 -- 172.21.15.29:0/1655715421 learned_addr learned my addr 172.21.15.29:0/1655715421 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dcddb0700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc8154e10 0x7f1dc80c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dcd5af700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dc8150c00 0x7f1dc8168220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dccdae700 1 -- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc8154e10 msgr2=0x7f1dc80c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dccdae700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc8154e10 0x7f1dc80c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:58.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dccdae700 1 -- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dc8150c00 msgr2=0x7f1dc8168220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:58.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dccdae700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dc8150c00 0x7f1dc8168220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:58.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dccdae700 1 -- 172.21.15.29:0/1655715421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc4009ae0 con 0x7f1dc8150fe0 2024-01-24T14:13:58.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dcd5af700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dc8150c00 0x7f1dc8168220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:58.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dcddb0700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc8154e10 0x7f1dc80c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:58.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dccdae700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dc8150fe0 0x7f1dc8142e20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f1db800d8c0 tx=0x7f1db8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:58.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dbd7fa700 1 -- 172.21.15.29:0/1655715421 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db80095b0 con 0x7f1dc8150fe0 2024-01-24T14:13:58.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dbd7fa700 1 -- 172.21.15.29:0/1655715421 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db8009710 con 0x7f1dc8150fe0 2024-01-24T14:13:58.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dcf813700 1 -- 172.21.15.29:0/1655715421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc80c04b0 con 0x7f1dc8150fe0 2024-01-24T14:13:58.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dbd7fa700 1 -- 172.21.15.29:0/1655715421 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8009a90 con 0x7f1dc8150fe0 2024-01-24T14:13:58.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.326+0000 7f1dcf813700 1 -- 172.21.15.29:0/1655715421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc80c5920 con 0x7f1dc8150fe0 2024-01-24T14:13:58.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.328+0000 7f1dcf813700 1 -- 172.21.15.29:0/1655715421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc80c5780 con 0x7f1dc8150fe0 2024-01-24T14:13:58.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.328+0000 7f1dbd7fa700 1 -- 172.21.15.29:0/1655715421 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1db8013480 con 0x7f1dc8150fe0 2024-01-24T14:13:58.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.329+0000 7f1dbd7fa700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1dac05f270 0x7f1dac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:58.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.329+0000 7f1dbd7fa700 1 -- 172.21.15.29:0/1655715421 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6660+0+0 (secure 0 0 0) 0x7f1db807a6d0 con 0x7f1dc8150fe0 2024-01-24T14:13:58.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.329+0000 7f1dcd5af700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1dac05f270 0x7f1dac061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:58.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.329+0000 7f1dcd5af700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1dac05f270 0x7f1dac061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f1dc80c7b90 tx=0x7f1dc4009840).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:58.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.333+0000 7f1dbd7fa700 1 -- 172.21.15.29:0/1655715421 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1dc80c5780 con 0x7f1dc8150fe0 2024-01-24T14:13:58.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:58.515+0000 7f1dcf813700 1 -- 172.21.15.29:0/1655715421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1dc80c5780 con 0x7f1dc8150fe0 2024-01-24T14:13:58.807 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:13:58.810 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:13:59.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.157+0000 7f1dbd7fa700 1 -- 172.21.15.29:0/1655715421 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1dc80c5780 con 0x7f1dc8150fe0 2024-01-24T14:13:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.163+0000 7f1dcf813700 1 -- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1dac05f270 msgr2=0x7f1dac061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.163+0000 7f1dcf813700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1dac05f270 0x7f1dac061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f1dc80c7b90 tx=0x7f1dc4009840).stop 2024-01-24T14:13:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.163+0000 7f1dcf813700 1 -- 172.21.15.29:0/1655715421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dc8150fe0 msgr2=0x7f1dc8142e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.163+0000 7f1dcf813700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dc8150fe0 0x7f1dc8142e20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f1db800d8c0 tx=0x7f1db8008040).stop 2024-01-24T14:13:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.164+0000 7f1dcf813700 1 -- 172.21.15.29:0/1655715421 shutdown_connections 2024-01-24T14:13:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.164+0000 7f1dcf813700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc8154e10 0x7f1dc80c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.164+0000 7f1dcf813700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dc8150c00 0x7f1dc8168220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.164+0000 7f1dcf813700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1dac05f270 0x7f1dac061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.164+0000 7f1dcf813700 1 --2- 172.21.15.29:0/1655715421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dc8150fe0 0x7f1dc8142e20 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.164+0000 7f1dcf813700 1 -- 172.21.15.29:0/1655715421 >> 172.21.15.29:0/1655715421 conn(0x7f1dc80638c0 msgr2=0x7f1dc80a94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:59.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.165+0000 7f1dcf813700 1 -- 172.21.15.29:0/1655715421 shutdown_connections 2024-01-24T14:13:59.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.166+0000 7f1dcf813700 1 -- 172.21.15.29:0/1655715421 wait complete. 2024-01-24T14:13:59.177 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-01-24T14:13:59.198 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-24T14:13:59.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.333+0000 7f35ec3c8700 1 Processor -- start 2024-01-24T14:13:59.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.333+0000 7f35ec3c8700 1 -- start start 2024-01-24T14:13:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ec3c8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35e4064760 0x7f35e4064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ec3c8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35e40bfd40 0x7f35e40c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ec3c8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e4152240 con 0x7f35e40c8490 2024-01-24T14:13:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ec3c8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e41586c0 con 0x7f35e40bfd40 2024-01-24T14:13:59.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ec3c8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e40aa670 con 0x7f35e4064760 2024-01-24T14:13:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ea965700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35e40bfd40 0x7f35e40c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ea965700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35e40bfd40 0x7f35e40c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34308/0 (socket says 172.21.15.29:34308) 2024-01-24T14:13:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35e9963700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f35e40c8490 0x7f35e40a9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38838/0 (socket says 172.21.15.29:38838) 2024-01-24T14:13:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ea965700 1 -- 172.21.15.29:0/279949141 learned_addr learned my addr 172.21.15.29:0/279949141 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ea164700 1 --2- 172.21.15.29:0/279949141 >> v2:172.21.15.163:3300/0 conn(0x7f35e4064760 0x7f35e4064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ea965700 1 -- 172.21.15.29:0/279949141 >> v2:172.21.15.163:3300/0 conn(0x7f35e4064760 msgr2=0x7f35e4064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ea965700 1 --2- 172.21.15.29:0/279949141 >> v2:172.21.15.163:3300/0 conn(0x7f35e4064760 0x7f35e4064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ea965700 1 -- 172.21.15.29:0/279949141 >> v1:172.21.15.29:6789/0 conn(0x7f35e40c8490 legacy=0x7f35e40a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ea965700 1 -- 172.21.15.29:0/279949141 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4157d90 con 0x7f35e40bfd40 2024-01-24T14:13:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35e9162700 1 -- 172.21.15.29:0/279949141 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011956177 0 0) 0x7f35e4152240 con 0x7f35e40c8490 2024-01-24T14:13:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35ea965700 1 --2- 172.21.15.29:0/279949141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e40bfd40 0x7f35e40c01d0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f35e4084450 tx=0x7f35dc009840).ready entity=mon.0 client_cookie=366548cd0d88ad5a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35e9162700 1 -- 172.21.15.29:0/279949141 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00e070 con 0x7f35e40bfd40 2024-01-24T14:13:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.335+0000 7f35e9162700 1 -- 172.21.15.29:0/279949141 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc00bae0 con 0x7f35e40bfd40 2024-01-24T14:13:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.336+0000 7f35e9162700 1 -- 172.21.15.29:0/279949141 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00be60 con 0x7f35e40bfd40 2024-01-24T14:13:59.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.336+0000 7f35ec3c8700 1 -- 172.21.15.29:0/279949141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e40bfd40 msgr2=0x7f35e40c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.336+0000 7f35ec3c8700 1 --2- 172.21.15.29:0/279949141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e40bfd40 0x7f35e40c01d0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f35e4084450 tx=0x7f35dc009840).stop 2024-01-24T14:13:59.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.336+0000 7f35ec3c8700 1 -- 172.21.15.29:0/279949141 shutdown_connections 2024-01-24T14:13:59.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.336+0000 7f35ec3c8700 1 --2- 172.21.15.29:0/279949141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e40bfd40 0x7f35e40c01d0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.336+0000 7f35ec3c8700 1 --2- 172.21.15.29:0/279949141 >> v2:172.21.15.163:3300/0 conn(0x7f35e4064760 0x7f35e4064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.336+0000 7f35ec3c8700 1 -- 172.21.15.29:0/279949141 >> 172.21.15.29:0/279949141 conn(0x7f35e40cb8b0 msgr2=0x7f35e40c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:59.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.336+0000 7f35ec3c8700 1 -- 172.21.15.29:0/279949141 shutdown_connections 2024-01-24T14:13:59.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.336+0000 7f35ec3c8700 1 -- 172.21.15.29:0/279949141 wait complete. 2024-01-24T14:13:59.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.337+0000 7f35ec3c8700 1 Processor -- start 2024-01-24T14:13:59.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.337+0000 7f35ec3c8700 1 -- start start 2024-01-24T14:13:59.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.338+0000 7f35ec3c8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35e4064760 0x7f35e4152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.338+0000 7f35ec3c8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35e40bfd40 0x7f35e4164520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.338+0000 7f35ec3c8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e40c8490 0x7f35e4164a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.338+0000 7f35ec3c8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4152a90 con 0x7f35e40c8490 2024-01-24T14:13:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.338+0000 7f35ec3c8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4152d70 con 0x7f35e40bfd40 2024-01-24T14:13:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.338+0000 7f35ec3c8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4153050 con 0x7f35e4064760 2024-01-24T14:13:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.338+0000 7f35ea965700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e40c8490 0x7f35e4164a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.338+0000 7f35ea965700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e40c8490 0x7f35e4164a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34310/0 (socket says 172.21.15.29:34310) 2024-01-24T14:13:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.338+0000 7f35ea965700 1 -- 172.21.15.29:0/799544977 learned_addr learned my addr 172.21.15.29:0/799544977 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.338+0000 7f35ea164700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35e4064760 0x7f35e4152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.338+0000 7f35e9963700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35e40bfd40 0x7f35e4164520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.339+0000 7f35ea965700 1 -- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35e4064760 msgr2=0x7f35e4152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.339+0000 7f35ea965700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35e4064760 0x7f35e4152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.339+0000 7f35ea965700 1 -- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35e40bfd40 msgr2=0x7f35e4164520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.339+0000 7f35ea965700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35e40bfd40 0x7f35e4164520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.339+0000 7f35ea965700 1 -- 172.21.15.29:0/799544977 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc009ae0 con 0x7f35e40c8490 2024-01-24T14:13:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.339+0000 7f35ea965700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e40c8490 0x7f35e4164a10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f35dc009fd0 tx=0x7f35dc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.339+0000 7f35d67fc700 1 -- 172.21.15.29:0/799544977 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc007170 con 0x7f35e40c8490 2024-01-24T14:13:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.339+0000 7f35d67fc700 1 -- 172.21.15.29:0/799544977 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc0092e0 con 0x7f35e40c8490 2024-01-24T14:13:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.339+0000 7f35d67fc700 1 -- 172.21.15.29:0/799544977 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc01d660 con 0x7f35e40c8490 2024-01-24T14:13:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.339+0000 7f35ec3c8700 1 -- 172.21.15.29:0/799544977 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e4153330 con 0x7f35e40c8490 2024-01-24T14:13:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.339+0000 7f35ec3c8700 1 -- 172.21.15.29:0/799544977 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e4152730 con 0x7f35e40c8490 2024-01-24T14:13:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.340+0000 7f35ec3c8700 1 -- 172.21.15.29:0/799544977 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e40475e0 con 0x7f35e40c8490 2024-01-24T14:13:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.341+0000 7f35d67fc700 1 -- 172.21.15.29:0/799544977 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f35dc01da90 con 0x7f35e40c8490 2024-01-24T14:13:59.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.341+0000 7f35d67fc700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f35c805f270 0x7f35c8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.341+0000 7f35d67fc700 1 -- 172.21.15.29:0/799544977 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7042+0+0 (secure 0 0 0) 0x7f35dc07c480 con 0x7f35e40c8490 2024-01-24T14:13:59.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.342+0000 7f35ea164700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f35c805f270 0x7f35c8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.342+0000 7f35ea164700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f35c805f270 0x7f35c8061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f35d800bbc0 tx=0x7f35d8008040).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:59.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.346+0000 7f35d67fc700 1 -- 172.21.15.29:0/799544977 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f35dc04d1a0 con 0x7f35e40c8490 2024-01-24T14:13:59.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.575+0000 7f35ec3c8700 1 -- 172.21.15.29:0/799544977 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f35e40657a0 con 0x7f35e40c8490 2024-01-24T14:13:59.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.581+0000 7f35d67fc700 1 -- 172.21.15.29:0/799544977 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v39) v1 ==== 174+0+0 (secure 0 0 0) 0x7f35dc0507c0 con 0x7f35e40c8490 2024-01-24T14:13:59.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.589+0000 7f35ec3c8700 1 -- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f35c805f270 msgr2=0x7f35c8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.589+0000 7f35ec3c8700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f35c805f270 0x7f35c8061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f35d800bbc0 tx=0x7f35d8008040).stop 2024-01-24T14:13:59.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.589+0000 7f35ec3c8700 1 -- 172.21.15.29:0/799544977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e40c8490 msgr2=0x7f35e4164a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.589+0000 7f35ec3c8700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e40c8490 0x7f35e4164a10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f35dc009fd0 tx=0x7f35dc009840).stop 2024-01-24T14:13:59.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.589+0000 7f35ec3c8700 1 -- 172.21.15.29:0/799544977 shutdown_connections 2024-01-24T14:13:59.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.589+0000 7f35ec3c8700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f35c805f270 0x7f35c8061730 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0x7f35d800bbc0 tx=0x7f35d8008040).stop 2024-01-24T14:13:59.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.589+0000 7f35ec3c8700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35e40c8490 0x7f35e4164a10 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.589+0000 7f35ec3c8700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35e4064760 0x7f35e4152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.589+0000 7f35ec3c8700 1 --2- 172.21.15.29:0/799544977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35e40bfd40 0x7f35e4164520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.589+0000 7f35ec3c8700 1 -- 172.21.15.29:0/799544977 >> 172.21.15.29:0/799544977 conn(0x7f35e40cb8b0 msgr2=0x7f35e40c5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:59.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.595+0000 7f35ec3c8700 1 -- 172.21.15.29:0/799544977 shutdown_connections 2024-01-24T14:13:59.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.595+0000 7f35ec3c8700 1 -- 172.21.15.29:0/799544977 wait complete. 2024-01-24T14:13:59.603 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 12 and data pool 13 2024-01-24T14:13:59.620 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:13:59.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.749+0000 7f9a28525700 1 Processor -- start 2024-01-24T14:13:59.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.749+0000 7f9a28525700 1 -- start start 2024-01-24T14:13:59.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.750+0000 7f9a28525700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200be520 0x7f9a200664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.750+0000 7f9a28525700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a20064120 con 0x7f9a200669e0 2024-01-24T14:13:59.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.750+0000 7f9a28525700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a200bf6a0 con 0x7f9a2013a5c0 2024-01-24T14:13:59.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.750+0000 7f9a28525700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20066e70 con 0x7f9a200be520 2024-01-24T14:13:59.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a26ac2700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9a200669e0 0x7f9a200c31b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38842/0 (socket says 172.21.15.29:38842) 2024-01-24T14:13:59.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a26ac2700 1 -- 172.21.15.29:0/2913463180 learned_addr learned my addr 172.21.15.29:0/2913463180 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:13:59.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a25ac0700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200be520 0x7f9a200664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758335835 0 0) 0x7f9a20064120 con 0x7f9a200669e0 2024-01-24T14:13:59.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a0c0033f0 con 0x7f9a200669e0 2024-01-24T14:13:59.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1237822271 0 0) 0x7f9a0c0033f0 con 0x7f9a200669e0 2024-01-24T14:13:59.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200be520 msgr2=0x7f9a200664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200be520 0x7f9a200664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 >> v1:172.21.15.163:6789/0 conn(0x7f9a2013a5c0 legacy=0x7f9a20063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a200bea90 con 0x7f9a200669e0 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f9a180017f0 con 0x7f9a200669e0 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a25ac0700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200be520 0x7f9a200664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 >> v1:172.21.15.29:6789/0 conn(0x7f9a200669e0 legacy=0x7f9a200c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0c003910 0x7f9a0c003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0c004690 0x7f9a0c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200be520 0x7f9a0c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20066e70 con 0x7f9a0c004690 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c00b8e0 con 0x7f9a0c003910 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.751+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c00ba20 con 0x7f9a200be520 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.752+0000 7f9a25ac0700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0c004690 0x7f9a0c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.752+0000 7f9a262c1700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0c003910 0x7f9a0c003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.752+0000 7f9a26ac2700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200be520 0x7f9a0c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.752+0000 7f9a25ac0700 1 -- 172.21.15.29:0/2913463180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200be520 msgr2=0x7f9a0c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.752+0000 7f9a25ac0700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200be520 0x7f9a0c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.752+0000 7f9a25ac0700 1 -- 172.21.15.29:0/2913463180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0c003910 msgr2=0x7f9a0c003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.752+0000 7f9a25ac0700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0c003910 0x7f9a0c003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.752+0000 7f9a25ac0700 1 -- 172.21.15.29:0/2913463180 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a100092a0 con 0x7f9a0c004690 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.752+0000 7f9a25ac0700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0c004690 0x7f9a0c004b10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f9a10009c70 tx=0x7f9a10009060).ready entity=mon.0 client_cookie=67e5f4a0b4ab1b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.752+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100095c0 con 0x7f9a0c004690 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.752+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1000e690 con 0x7f9a0c004690 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.752+0000 7f9a252bf700 1 -- 172.21.15.29:0/2913463180 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1000ea10 con 0x7f9a0c004690 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.753+0000 7f9a28525700 1 -- 172.21.15.29:0/2913463180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0c004690 msgr2=0x7f9a0c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.753+0000 7f9a28525700 1 --2- 172.21.15.29:0/2913463180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0c004690 0x7f9a0c004b10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f9a10009c70 tx=0x7f9a10009060).stop 2024-01-24T14:13:59.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.753+0000 7f9a26ac2700 1 -- 172.21.15.29:0/2913463180 reap_dead start 2024-01-24T14:13:59.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.753+0000 7f9a28525700 1 -- 172.21.15.29:0/2913463180 shutdown_connections 2024-01-24T14:13:59.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.753+0000 7f9a28525700 1 -- 172.21.15.29:0/2913463180 >> 172.21.15.29:0/2913463180 conn(0x7f9a200c8bb0 msgr2=0x7f9a200c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:59.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.753+0000 7f9a28525700 1 -- 172.21.15.29:0/2913463180 shutdown_connections 2024-01-24T14:13:59.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.753+0000 7f9a28525700 1 -- 172.21.15.29:0/2913463180 wait complete. 2024-01-24T14:13:59.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.754+0000 7f9a28525700 1 Processor -- start 2024-01-24T14:13:59.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.754+0000 7f9a28525700 1 -- start start 2024-01-24T14:13:59.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a28525700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0c004690 0x7f9a20157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a28525700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a200ac140 0x7f9a200ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a28525700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200aad80 0x7f9a200ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a28525700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20168660 con 0x7f9a0c004690 2024-01-24T14:13:59.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a28525700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20155d00 con 0x7f9a200ac140 2024-01-24T14:13:59.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a28525700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20155fa0 con 0x7f9a200aad80 2024-01-24T14:13:59.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a262c1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0c004690 0x7f9a20157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a262c1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0c004690 0x7f9a20157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34330/0 (socket says 172.21.15.29:34330) 2024-01-24T14:13:59.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a262c1700 1 -- 172.21.15.29:0/1107258771 learned_addr learned my addr 172.21.15.29:0/1107258771 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:13:59.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a26ac2700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200aad80 0x7f9a200ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a262c1700 1 -- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200aad80 msgr2=0x7f9a200ab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a262c1700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200aad80 0x7f9a200ab1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a25ac0700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a200ac140 0x7f9a200ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a262c1700 1 -- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a200ac140 msgr2=0x7f9a200ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a262c1700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a200ac140 0x7f9a200ac5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a262c1700 1 -- 172.21.15.29:0/1107258771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a20156280 con 0x7f9a0c004690 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a26ac2700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200aad80 0x7f9a200ab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a25ac0700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a200ac140 0x7f9a200ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.755+0000 7f9a262c1700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0c004690 0x7f9a20157040 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f9a1c0029d0 tx=0x7f9a1c007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.756+0000 7f9a167fc700 1 -- 172.21.15.29:0/1107258771 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00f8d0 con 0x7f9a0c004690 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.756+0000 7f9a167fc700 1 -- 172.21.15.29:0/1107258771 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c00fa30 con 0x7f9a0c004690 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.756+0000 7f9a167fc700 1 -- 172.21.15.29:0/1107258771 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00fdb0 con 0x7f9a0c004690 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.756+0000 7f9a28525700 1 -- 172.21.15.29:0/1107258771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a20156560 con 0x7f9a0c004690 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.756+0000 7f9a28525700 1 -- 172.21.15.29:0/1107258771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a20156b20 con 0x7f9a0c004690 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.757+0000 7f9a28525700 1 -- 172.21.15.29:0/1107258771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a20156d40 con 0x7f9a0c004690 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.757+0000 7f9a167fc700 1 -- 172.21.15.29:0/1107258771 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9a1c014660 con 0x7f9a0c004690 2024-01-24T14:13:59.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.758+0000 7f9a167fc700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a0405f270 0x7f9a04061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:13:59.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.758+0000 7f9a167fc700 1 -- 172.21.15.29:0/1107258771 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f9a1c07a5b0 con 0x7f9a0c004690 2024-01-24T14:13:59.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.758+0000 7f9a25ac0700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a0405f270 0x7f9a04061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:13:59.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.759+0000 7f9a25ac0700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a0405f270 0x7f9a04061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f9a20153040 tx=0x7f9a1001a000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:13:59.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.762+0000 7f9a167fc700 1 -- 172.21.15.29:0/1107258771 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9a1c04ab00 con 0x7f9a0c004690 2024-01-24T14:13:59.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.943+0000 7f9a28525700 1 -- 172.21.15.29:0/1107258771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a200475e0 con 0x7f9a0c004690 2024-01-24T14:13:59.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.944+0000 7f9a167fc700 1 -- 172.21.15.29:0/1107258771 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+18197 (secure 0 0 0) 0x7f9a1c04b3b0 con 0x7f9a0c004690 2024-01-24T14:13:59.945 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:13:59.945 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":104,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:13:59.576585+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:13:57.364021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-24T14:13:58.553943+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:13:59.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.946+0000 7f9a28525700 1 -- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a0405f270 msgr2=0x7f9a04061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.946+0000 7f9a28525700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a0405f270 0x7f9a04061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f9a20153040 tx=0x7f9a1001a000).stop 2024-01-24T14:13:59.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.946+0000 7f9a28525700 1 -- 172.21.15.29:0/1107258771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0c004690 msgr2=0x7f9a20157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:13:59.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.946+0000 7f9a28525700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0c004690 0x7f9a20157040 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f9a1c0029d0 tx=0x7f9a1c007b90).stop 2024-01-24T14:13:59.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.946+0000 7f9a28525700 1 -- 172.21.15.29:0/1107258771 shutdown_connections 2024-01-24T14:13:59.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.946+0000 7f9a28525700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a200aad80 0x7f9a200ab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.946+0000 7f9a28525700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a0405f270 0x7f9a04061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.946+0000 7f9a28525700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a0c004690 0x7f9a20157040 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.946+0000 7f9a28525700 1 --2- 172.21.15.29:0/1107258771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a200ac140 0x7f9a200ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:13:59.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.946+0000 7f9a28525700 1 -- 172.21.15.29:0/1107258771 >> 172.21.15.29:0/1107258771 conn(0x7f9a200c8bb0 msgr2=0x7f9a2015a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:13:59.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.946+0000 7f9a28525700 1 -- 172.21.15.29:0/1107258771 shutdown_connections 2024-01-24T14:13:59.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:13:59.946+0000 7f9a28525700 1 -- 172.21.15.29:0/1107258771 wait complete. 2024-01-24T14:13:59.961 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:14:00.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.090+0000 7f625dab7700 1 Processor -- start 2024-01-24T14:14:00.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.090+0000 7f625dab7700 1 -- start start 2024-01-24T14:14:00.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.091+0000 7f625dab7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6258138710 0x7f62580be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:00.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.091+0000 7f625dab7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f62580627b0 0x7f6258062c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:00.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.091+0000 7f625dab7700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6258065770 con 0x7f6258138330 2024-01-24T14:14:00.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.091+0000 7f625dab7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6258063130 con 0x7f62580627b0 2024-01-24T14:14:00.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.091+0000 7f625dab7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6258069cf0 con 0x7f6258138710 2024-01-24T14:14:00.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.091+0000 7f62577fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6258138330 0x7f62580bca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38844/0 (socket says 172.21.15.29:38844) 2024-01-24T14:14:00.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f62577fe700 1 -- 172.21.15.29:0/2880264428 learned_addr learned my addr 172.21.15.29:0/2880264428 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:00.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f6256ffd700 1 --2- 172.21.15.29:0/2880264428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6258138710 0x7f62580be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f6257fff700 1 --2- 172.21.15.29:0/2880264428 >> v2:172.21.15.163:3300/0 conn(0x7f62580627b0 0x7f6258062c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f62567fc700 1 -- 172.21.15.29:0/2880264428 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800743908 0 0) 0x7f6258065770 con 0x7f6258138330 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f62567fc700 1 -- 172.21.15.29:0/2880264428 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62400033f0 con 0x7f6258138330 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f62567fc700 1 -- 172.21.15.29:0/2880264428 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2839907491 0 0) 0x7f62400033f0 con 0x7f6258138330 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f6256ffd700 1 -- 172.21.15.29:0/2880264428 >> v2:172.21.15.163:3300/0 conn(0x7f62580627b0 msgr2=0x7f6258062c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f6256ffd700 1 --2- 172.21.15.29:0/2880264428 >> v2:172.21.15.163:3300/0 conn(0x7f62580627b0 0x7f6258062c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f6256ffd700 1 -- 172.21.15.29:0/2880264428 >> v1:172.21.15.29:6789/0 conn(0x7f6258138330 legacy=0x7f62580bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f6256ffd700 1 -- 172.21.15.29:0/2880264428 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6258069f60 con 0x7f6258138710 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f6257fff700 1 --2- 172.21.15.29:0/2880264428 >> v2:172.21.15.163:3300/0 conn(0x7f62580627b0 0x7f6258062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f6256ffd700 1 --2- 172.21.15.29:0/2880264428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6258138710 0x7f62580be3e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f624c009760 tx=0x7f624c009060).ready entity=mon.2 client_cookie=db4d26c8df29828a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.092+0000 7f62567fc700 1 -- 172.21.15.29:0/2880264428 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248001470 con 0x7f6258138710 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.093+0000 7f62567fc700 1 -- 172.21.15.29:0/2880264428 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c011400 con 0x7f6258138710 2024-01-24T14:14:00.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.093+0000 7f62567fc700 1 -- 172.21.15.29:0/2880264428 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c011780 con 0x7f6258138710 2024-01-24T14:14:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.093+0000 7f625dab7700 1 -- 172.21.15.29:0/2880264428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6258138710 msgr2=0x7f62580be3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.093+0000 7f625dab7700 1 --2- 172.21.15.29:0/2880264428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6258138710 0x7f62580be3e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f624c009760 tx=0x7f624c009060).stop 2024-01-24T14:14:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.093+0000 7f625dab7700 1 -- 172.21.15.29:0/2880264428 shutdown_connections 2024-01-24T14:14:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.093+0000 7f625dab7700 1 --2- 172.21.15.29:0/2880264428 >> v2:172.21.15.163:3300/0 conn(0x7f62580627b0 0x7f6258062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.093+0000 7f625dab7700 1 --2- 172.21.15.29:0/2880264428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6258138710 0x7f62580be3e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.093+0000 7f625dab7700 1 -- 172.21.15.29:0/2880264428 >> 172.21.15.29:0/2880264428 conn(0x7f62580c85c0 msgr2=0x7f62580c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:00.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.093+0000 7f625dab7700 1 -- 172.21.15.29:0/2880264428 shutdown_connections 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.093+0000 7f625dab7700 1 -- 172.21.15.29:0/2880264428 wait complete. 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.095+0000 7f625dab7700 1 Processor -- start 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.095+0000 7f625dab7700 1 -- start start 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.095+0000 7f625dab7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62580627b0 0x7f62581375e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.095+0000 7f625dab7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6258138330 0x7f62580c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.095+0000 7f625dab7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6258138710 0x7f6258065c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.095+0000 7f625dab7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62580a6850 con 0x7f6258138330 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.095+0000 7f625dab7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62580a5c40 con 0x7f62580627b0 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.095+0000 7f625dab7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62580a5f20 con 0x7f6258138710 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f6256ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6258138330 0x7f62580c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f6256ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6258138330 0x7f62580c3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34336/0 (socket says 172.21.15.29:34336) 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f6256ffd700 1 -- 172.21.15.29:0/550974107 learned_addr learned my addr 172.21.15.29:0/550974107 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f6257fff700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6258138710 0x7f6258065c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:00.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f62577fe700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62580627b0 0x7f62581375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:00.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f6256ffd700 1 -- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6258138710 msgr2=0x7f6258065c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:00.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f6256ffd700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6258138710 0x7f6258065c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f6256ffd700 1 -- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62580627b0 msgr2=0x7f62581375e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:00.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f6256ffd700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62580627b0 0x7f62581375e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f6256ffd700 1 -- 172.21.15.29:0/550974107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6250009a20 con 0x7f6258138330 2024-01-24T14:14:00.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f6256ffd700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6258138330 0x7f62580c3a80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f6258069e30 tx=0x7f624c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:00.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f623f7fe700 1 -- 172.21.15.29:0/550974107 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c010870 con 0x7f6258138330 2024-01-24T14:14:00.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f623f7fe700 1 -- 172.21.15.29:0/550974107 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c0109d0 con 0x7f6258138330 2024-01-24T14:14:00.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f625dab7700 1 -- 172.21.15.29:0/550974107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624c0092a0 con 0x7f6258138330 2024-01-24T14:14:00.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f623f7fe700 1 -- 172.21.15.29:0/550974107 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c010d70 con 0x7f6258138330 2024-01-24T14:14:00.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.096+0000 7f625dab7700 1 -- 172.21.15.29:0/550974107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62580a64e0 con 0x7f6258138330 2024-01-24T14:14:00.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.098+0000 7f625dab7700 1 -- 172.21.15.29:0/550974107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62580475e0 con 0x7f6258138330 2024-01-24T14:14:00.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.098+0000 7f623f7fe700 1 -- 172.21.15.29:0/550974107 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f624c01d640 con 0x7f6258138330 2024-01-24T14:14:00.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.098+0000 7f623f7fe700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f624405f270 0x7f6244061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:00.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.098+0000 7f623f7fe700 1 -- 172.21.15.29:0/550974107 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f624c07bf70 con 0x7f6258138330 2024-01-24T14:14:00.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.098+0000 7f62577fe700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f624405f270 0x7f6244061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:00.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.099+0000 7f62577fe700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f624405f270 0x7f6244061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f62480011e0 tx=0x7f6248001520).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:00.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.103+0000 7f623f7fe700 1 -- 172.21.15.29:0/550974107 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f624c0aab00 con 0x7f6258138330 2024-01-24T14:14:00.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.285+0000 7f625dab7700 1 -- 172.21.15.29:0/550974107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f625806a570 con 0x7f6258138330 2024-01-24T14:14:00.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.286+0000 7f623f7fe700 1 -- 172.21.15.29:0/550974107 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+18197 (secure 0 0 0) 0x7f624c04ccc0 con 0x7f6258138330 2024-01-24T14:14:00.287 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:00.287 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":104,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:13:59.576585+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:13:57.364021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-24T14:13:58.553943+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:14:00.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.288+0000 7f625dab7700 1 -- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f624405f270 msgr2=0x7f6244061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:00.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.288+0000 7f625dab7700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f624405f270 0x7f6244061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f62480011e0 tx=0x7f6248001520).stop 2024-01-24T14:14:00.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.288+0000 7f625dab7700 1 -- 172.21.15.29:0/550974107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6258138330 msgr2=0x7f62580c3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:00.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.288+0000 7f625dab7700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6258138330 0x7f62580c3a80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f6258069e30 tx=0x7f624c016000).stop 2024-01-24T14:14:00.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.288+0000 7f625dab7700 1 -- 172.21.15.29:0/550974107 shutdown_connections 2024-01-24T14:14:00.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.288+0000 7f625dab7700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6258138710 0x7f6258065c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.288+0000 7f625dab7700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62580627b0 0x7f62581375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.288+0000 7f625dab7700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f624405f270 0x7f6244061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.288+0000 7f625dab7700 1 --2- 172.21.15.29:0/550974107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6258138330 0x7f62580c3a80 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.288+0000 7f625dab7700 1 -- 172.21.15.29:0/550974107 >> 172.21.15.29:0/550974107 conn(0x7f62580c85c0 msgr2=0x7f62580c68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:00.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.288+0000 7f625dab7700 1 -- 172.21.15.29:0/550974107 shutdown_connections 2024-01-24T14:14:00.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.288+0000 7f625dab7700 1 -- 172.21.15.29:0/550974107 wait complete. 2024-01-24T14:14:00.303 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-24T14:14:00.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.436+0000 7fe22f7cc700 1 Processor -- start 2024-01-24T14:14:00.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.436+0000 7fe22f7cc700 1 -- start start 2024-01-24T14:14:00.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.437+0000 7fe22f7cc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe228069680 0x7fe228069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.437+0000 7fe22f7cc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe2281589e0 0x7fe22815add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.437+0000 7fe22f7cc700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2280c3e70 con 0x7fe2280692a0 2024-01-24T14:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.437+0000 7fe22f7cc700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2280c4300 con 0x7fe228069680 2024-01-24T14:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.437+0000 7fe22f7cc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2280c3260 con 0x7fe2281589e0 2024-01-24T14:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.437+0000 7fe22cd67700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe228069680 0x7fe228069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.437+0000 7fe22d568700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe2280692a0 0x7fe228065e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38856/0 (socket says 172.21.15.29:38856) 2024-01-24T14:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.437+0000 7fe22d568700 1 -- 172.21.15.29:0/2122855303 learned_addr learned my addr 172.21.15.29:0/2122855303 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.438+0000 7fe22dd69700 1 --2- 172.21.15.29:0/2122855303 >> v2:172.21.15.163:3300/0 conn(0x7fe2281589e0 0x7fe22815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.438+0000 7fe21ffff700 1 -- 172.21.15.29:0/2122855303 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903536029 0 0) 0x7fe2280c3e70 con 0x7fe2280692a0 2024-01-24T14:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.438+0000 7fe21ffff700 1 -- 172.21.15.29:0/2122855303 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2140033f0 con 0x7fe2280692a0 2024-01-24T14:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.438+0000 7fe22cd67700 1 -- 172.21.15.29:0/2122855303 >> v2:172.21.15.163:3300/0 conn(0x7fe2281589e0 msgr2=0x7fe22815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.438+0000 7fe22cd67700 1 --2- 172.21.15.29:0/2122855303 >> v2:172.21.15.163:3300/0 conn(0x7fe2281589e0 0x7fe22815add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.438+0000 7fe22cd67700 1 -- 172.21.15.29:0/2122855303 >> v1:172.21.15.29:6789/0 conn(0x7fe2280692a0 legacy=0x7fe228065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.438+0000 7fe22cd67700 1 -- 172.21.15.29:0/2122855303 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2280c3540 con 0x7fe228069680 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.438+0000 7fe22cd67700 1 --2- 172.21.15.29:0/2122855303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe228069680 0x7fe228069ae0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7fe228046ef0 tx=0x7fe224009060).ready entity=mon.0 client_cookie=f0a1077f75663251 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.438+0000 7fe21ffff700 1 -- 172.21.15.29:0/2122855303 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22400d4a0 con 0x7fe228069680 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.438+0000 7fe21ffff700 1 -- 172.21.15.29:0/2122855303 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22400d600 con 0x7fe228069680 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.438+0000 7fe21ffff700 1 -- 172.21.15.29:0/2122855303 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22400d980 con 0x7fe228069680 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.439+0000 7fe22f7cc700 1 -- 172.21.15.29:0/2122855303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe228069680 msgr2=0x7fe228069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.439+0000 7fe22f7cc700 1 --2- 172.21.15.29:0/2122855303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe228069680 0x7fe228069ae0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7fe228046ef0 tx=0x7fe224009060).stop 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.439+0000 7fe22f7cc700 1 -- 172.21.15.29:0/2122855303 shutdown_connections 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.439+0000 7fe22f7cc700 1 --2- 172.21.15.29:0/2122855303 >> v2:172.21.15.163:3300/0 conn(0x7fe2281589e0 0x7fe22815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.439+0000 7fe22f7cc700 1 --2- 172.21.15.29:0/2122855303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe228069680 0x7fe228069ae0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.439+0000 7fe22f7cc700 1 -- 172.21.15.29:0/2122855303 >> 172.21.15.29:0/2122855303 conn(0x7fe228064390 msgr2=0x7fe2280c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.439+0000 7fe22f7cc700 1 -- 172.21.15.29:0/2122855303 shutdown_connections 2024-01-24T14:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.439+0000 7fe22f7cc700 1 -- 172.21.15.29:0/2122855303 wait complete. 2024-01-24T14:14:00.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.441+0000 7fe22f7cc700 1 Processor -- start 2024-01-24T14:14:00.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.441+0000 7fe22f7cc700 1 -- start start 2024-01-24T14:14:00.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22f7cc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2280692a0 0x7fe2280c1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:00.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22f7cc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe228069680 0x7fe2280beb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:00.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22f7cc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2281589e0 0x7fe22813e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:00.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22f7cc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280bff60 con 0x7fe2280692a0 2024-01-24T14:14:00.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22f7cc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280bf080 con 0x7fe228069680 2024-01-24T14:14:00.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22f7cc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280bf2e0 con 0x7fe2281589e0 2024-01-24T14:14:00.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22d568700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2280692a0 0x7fe2280c1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:00.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22d568700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2280692a0 0x7fe2280c1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34356/0 (socket says 172.21.15.29:34356) 2024-01-24T14:14:00.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22d568700 1 -- 172.21.15.29:0/1375885621 learned_addr learned my addr 172.21.15.29:0/1375885621 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:00.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22dd69700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2281589e0 0x7fe22813e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22cd67700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe228069680 0x7fe2280beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22d568700 1 -- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2281589e0 msgr2=0x7fe22813e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22d568700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2281589e0 0x7fe22813e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22d568700 1 -- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe228069680 msgr2=0x7fe2280beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22d568700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe228069680 0x7fe2280beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22d568700 1 -- 172.21.15.29:0/1375885621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2240092a0 con 0x7fe2280692a0 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.442+0000 7fe22cd67700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe228069680 0x7fe2280beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.443+0000 7fe22d568700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2280692a0 0x7fe2280c1620 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7fe21800c590 tx=0x7fe218007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.443+0000 7fe21d7fa700 1 -- 172.21.15.29:0/1375885621 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218007e40 con 0x7fe2280692a0 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.443+0000 7fe21d7fa700 1 -- 172.21.15.29:0/1375885621 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2180072e0 con 0x7fe2280692a0 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.443+0000 7fe22f7cc700 1 -- 172.21.15.29:0/1375885621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2280bf5c0 con 0x7fe2280692a0 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.443+0000 7fe21d7fa700 1 -- 172.21.15.29:0/1375885621 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2180057a0 con 0x7fe2280692a0 2024-01-24T14:14:00.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.443+0000 7fe22f7cc700 1 -- 172.21.15.29:0/1375885621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2280bfb80 con 0x7fe2280692a0 2024-01-24T14:14:00.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.444+0000 7fe22f7cc700 1 -- 172.21.15.29:0/1375885621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2280475e0 con 0x7fe2280692a0 2024-01-24T14:14:00.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.445+0000 7fe21d7fa700 1 -- 172.21.15.29:0/1375885621 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe218005bd0 con 0x7fe2280692a0 2024-01-24T14:14:00.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.445+0000 7fe21d7fa700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe20c05f270 0x7fe20c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:00.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.445+0000 7fe21d7fa700 1 -- 172.21.15.29:0/1375885621 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fe21807a080 con 0x7fe2280692a0 2024-01-24T14:14:00.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.445+0000 7fe22cd67700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe20c05f270 0x7fe20c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:00.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.446+0000 7fe22cd67700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe20c05f270 0x7fe20c061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fe224009fd0 tx=0x7fe224016000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:00.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.449+0000 7fe21d7fa700 1 -- 172.21.15.29:0/1375885621 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe21804add0 con 0x7fe2280692a0 2024-01-24T14:14:00.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:00.686+0000 7fe22f7cc700 1 -- 172.21.15.29:0/1375885621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe2280c01d0 con 0x7fe2280692a0 2024-01-24T14:14:01.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.594+0000 7fe21d7fa700 1 -- 172.21.15.29:0/1375885621 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe2280c01d0 con 0x7fe2280692a0 2024-01-24T14:14:01.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.597+0000 7fe22f7cc700 1 -- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe20c05f270 msgr2=0x7fe20c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:01.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.597+0000 7fe22f7cc700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe20c05f270 0x7fe20c061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fe224009fd0 tx=0x7fe224016000).stop 2024-01-24T14:14:01.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.597+0000 7fe22f7cc700 1 -- 172.21.15.29:0/1375885621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2280692a0 msgr2=0x7fe2280c1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:01.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.597+0000 7fe22f7cc700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2280692a0 0x7fe2280c1620 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7fe21800c590 tx=0x7fe218007c80).stop 2024-01-24T14:14:01.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.597+0000 7fe22f7cc700 1 -- 172.21.15.29:0/1375885621 shutdown_connections 2024-01-24T14:14:01.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.597+0000 7fe22f7cc700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2281589e0 0x7fe22813e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:01.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.597+0000 7fe22f7cc700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe20c05f270 0x7fe20c061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:01.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.597+0000 7fe22f7cc700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2280692a0 0x7fe2280c1620 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:01.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.597+0000 7fe22f7cc700 1 --2- 172.21.15.29:0/1375885621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe228069680 0x7fe2280beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:01.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.597+0000 7fe22f7cc700 1 -- 172.21.15.29:0/1375885621 >> 172.21.15.29:0/1375885621 conn(0x7fe228064390 msgr2=0x7fe22814bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.597+0000 7fe22f7cc700 1 -- 172.21.15.29:0/1375885621 shutdown_connections 2024-01-24T14:14:01.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.597+0000 7fe22f7cc700 1 -- 172.21.15.29:0/1375885621 wait complete. 2024-01-24T14:14:01.615 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-24T14:14:01.615 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-24T14:14:01.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.756+0000 7fd005a5d700 1 Processor -- start 2024-01-24T14:14:01.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.757+0000 7fd005a5d700 1 -- start start 2024-01-24T14:14:01.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.757+0000 7fd005a5d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd000064760 0x7fd000064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:01.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.757+0000 7fd005a5d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd0000c8490 0x7fd0000a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:01.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.757+0000 7fd005a5d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd000152240 con 0x7fd000154700 2024-01-24T14:14:01.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.757+0000 7fd005a5d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0001586c0 con 0x7fd0000c8490 2024-01-24T14:14:01.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.757+0000 7fd005a5d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0000c0d50 con 0x7fd000064760 2024-01-24T14:14:01.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffe7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd0000c8490 0x7fd0000a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:01.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffe7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd0000c8490 0x7fd0000a9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34364/0 (socket says 172.21.15.29:34364) 2024-01-24T14:14:01.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcfff7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd000154700 0x7fd000154bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38860/0 (socket says 172.21.15.29:38860) 2024-01-24T14:14:01.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffe7fc700 1 -- 172.21.15.29:0/1574438866 learned_addr learned my addr 172.21.15.29:0/1574438866 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:01.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffeffd700 1 --2- 172.21.15.29:0/1574438866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd000064760 0x7fd000064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:01.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffe7fc700 1 -- 172.21.15.29:0/1574438866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd000064760 msgr2=0x7fd000064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:01.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffe7fc700 1 --2- 172.21.15.29:0/1574438866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd000064760 0x7fd000064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:01.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffe7fc700 1 -- 172.21.15.29:0/1574438866 >> v1:172.21.15.29:6789/0 conn(0x7fd000154700 legacy=0x7fd000154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffe7fc700 1 -- 172.21.15.29:0/1574438866 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0000aa670 con 0x7fd0000c8490 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffdffb700 1 -- 172.21.15.29:0/1574438866 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993833495 0 0) 0x7fd000152240 con 0x7fd000154700 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffe7fc700 1 --2- 172.21.15.29:0/1574438866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0000c8490 0x7fd0000a9d90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7fcfe8006fd0 tx=0x7fcfe80090f0).ready entity=mon.0 client_cookie=34df02a066cf9d6d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffdffb700 1 -- 172.21.15.29:0/1574438866 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800b610 con 0x7fd0000c8490 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffdffb700 1 -- 172.21.15.29:0/1574438866 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe800b770 con 0x7fd0000c8490 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.758+0000 7fcffdffb700 1 -- 172.21.15.29:0/1574438866 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800baf0 con 0x7fd0000c8490 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.759+0000 7fd005a5d700 1 -- 172.21.15.29:0/1574438866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0000c8490 msgr2=0x7fd0000a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.759+0000 7fd005a5d700 1 --2- 172.21.15.29:0/1574438866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0000c8490 0x7fd0000a9d90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7fcfe8006fd0 tx=0x7fcfe80090f0).stop 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.759+0000 7fd005a5d700 1 -- 172.21.15.29:0/1574438866 shutdown_connections 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.759+0000 7fd005a5d700 1 --2- 172.21.15.29:0/1574438866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd000064760 0x7fd000064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.759+0000 7fd005a5d700 1 --2- 172.21.15.29:0/1574438866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0000c8490 0x7fd0000a9d90 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.759+0000 7fd005a5d700 1 -- 172.21.15.29:0/1574438866 >> 172.21.15.29:0/1574438866 conn(0x7fd0000cb8b0 msgr2=0x7fd0000c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.759+0000 7fd005a5d700 1 -- 172.21.15.29:0/1574438866 shutdown_connections 2024-01-24T14:14:01.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.759+0000 7fd005a5d700 1 -- 172.21.15.29:0/1574438866 wait complete. 2024-01-24T14:14:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.760+0000 7fd005a5d700 1 Processor -- start 2024-01-24T14:14:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.761+0000 7fd005a5d700 1 -- start start 2024-01-24T14:14:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.761+0000 7fd005a5d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd000064760 0x7fd00014f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.761+0000 7fd005a5d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0000c8490 0x7fd0001566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.761+0000 7fd005a5d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000154700 0x7fd000156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.761+0000 7fd005a5d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0001570c0 con 0x7fd000064760 2024-01-24T14:14:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.761+0000 7fd005a5d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd000154b90 con 0x7fd000154700 2024-01-24T14:14:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.761+0000 7fd005a5d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd000154e70 con 0x7fd0000c8490 2024-01-24T14:14:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.761+0000 7fcffeffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd000064760 0x7fd00014f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.761+0000 7fcffeffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd000064760 0x7fd00014f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34370/0 (socket says 172.21.15.29:34370) 2024-01-24T14:14:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.761+0000 7fcffeffd700 1 -- 172.21.15.29:0/2285479936 learned_addr learned my addr 172.21.15.29:0/2285479936 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:01.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fcffe7fc700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0000c8490 0x7fd0001566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fcfff7fe700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000154700 0x7fd000156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fcffeffd700 1 -- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0000c8490 msgr2=0x7fd0001566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fcffeffd700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0000c8490 0x7fd0001566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fcffeffd700 1 -- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000154700 msgr2=0x7fd000156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fcffeffd700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000154700 0x7fd000156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fcffeffd700 1 -- 172.21.15.29:0/2285479936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8009370 con 0x7fd000064760 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fcffeffd700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd000064760 0x7fd00014f600 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7fcff000e950 tx=0x7fcff0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fcff6ffd700 1 -- 172.21.15.29:0/2285479936 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0009530 con 0x7fd000064760 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fcff6ffd700 1 -- 172.21.15.29:0/2285479936 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff0012600 con 0x7fd000064760 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fd005a5d700 1 -- 172.21.15.29:0/2285479936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0000c1e80 con 0x7fd000064760 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fcff6ffd700 1 -- 172.21.15.29:0/2285479936 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0012980 con 0x7fd000064760 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.762+0000 7fd005a5d700 1 -- 172.21.15.29:0/2285479936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0000af910 con 0x7fd000064760 2024-01-24T14:14:01.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.763+0000 7fd005a5d700 1 -- 172.21.15.29:0/2285479936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0000475e0 con 0x7fd000064760 2024-01-24T14:14:01.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.764+0000 7fcff6ffd700 1 -- 172.21.15.29:0/2285479936 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcff0012d30 con 0x7fd000064760 2024-01-24T14:14:01.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.764+0000 7fcff6ffd700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcfe005f240 0x7fcfe0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:01.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.764+0000 7fcff6ffd700 1 -- 172.21.15.29:0/2285479936 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fcff007ac90 con 0x7fd000064760 2024-01-24T14:14:01.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.765+0000 7fcffe7fc700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcfe005f240 0x7fcfe0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:01.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.765+0000 7fcffe7fc700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcfe005f240 0x7fcfe0061700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fcfe8006fd0 tx=0x7fcfe8017000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:01.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.769+0000 7fcff6ffd700 1 -- 172.21.15.29:0/2285479936 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcff004b9e0 con 0x7fd000064760 2024-01-24T14:14:01.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:01.998+0000 7fd005a5d700 1 -- 172.21.15.29:0/2285479936 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd000157d90 con 0x7fd000064760 2024-01-24T14:14:02.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.603+0000 7fcff6ffd700 1 -- 172.21.15.29:0/2285479936 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcff004f000 con 0x7fd000064760 2024-01-24T14:14:02.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.606+0000 7fd005a5d700 1 -- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcfe005f240 msgr2=0x7fcfe0061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:02.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.606+0000 7fd005a5d700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcfe005f240 0x7fcfe0061700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fcfe8006fd0 tx=0x7fcfe8017000).stop 2024-01-24T14:14:02.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.606+0000 7fd005a5d700 1 -- 172.21.15.29:0/2285479936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd000064760 msgr2=0x7fd00014f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:02.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.606+0000 7fd005a5d700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd000064760 0x7fd00014f600 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7fcff000e950 tx=0x7fcff0009060).stop 2024-01-24T14:14:02.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.606+0000 7fd005a5d700 1 -- 172.21.15.29:0/2285479936 shutdown_connections 2024-01-24T14:14:02.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.606+0000 7fd005a5d700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000154700 0x7fd000156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:02.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.606+0000 7fd005a5d700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcfe005f240 0x7fcfe0061700 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:02.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.606+0000 7fd005a5d700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd000064760 0x7fd00014f600 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:02.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.606+0000 7fd005a5d700 1 --2- 172.21.15.29:0/2285479936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0000c8490 0x7fd0001566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:02.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.606+0000 7fd005a5d700 1 -- 172.21.15.29:0/2285479936 >> 172.21.15.29:0/2285479936 conn(0x7fd0000cb8b0 msgr2=0x7fd0000c5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:02.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.606+0000 7fd005a5d700 1 -- 172.21.15.29:0/2285479936 shutdown_connections 2024-01-24T14:14:02.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.606+0000 7fd005a5d700 1 -- 172.21.15.29:0/2285479936 wait complete. 2024-01-24T14:14:02.622 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-24T14:14:02.622 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:14:02.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.756+0000 7f7ea27d0700 1 Processor -- start 2024-01-24T14:14:02.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.756+0000 7f7ea27d0700 1 -- start start 2024-01-24T14:14:02.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.757+0000 7f7ea27d0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e9c062970 0x7f7e9c062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:02.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.757+0000 7f7ea27d0700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c0bcf60 con 0x7f7e9c158600 2024-01-24T14:14:02.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.757+0000 7f7ea27d0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c0a6ec0 con 0x7f7e9c066b40 2024-01-24T14:14:02.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.757+0000 7f7ea27d0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c0bdba0 con 0x7f7e9c062970 2024-01-24T14:14:02.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.758+0000 7f7ea0d6d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7e9c158600 0x7f7e9c154d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38864/0 (socket says 172.21.15.29:38864) 2024-01-24T14:14:02.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.758+0000 7f7ea0d6d700 1 -- 172.21.15.29:0/1084499518 learned_addr learned my addr 172.21.15.29:0/1084499518 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:02.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.758+0000 7f7e9bfff700 1 --2- 172.21.15.29:0/1084499518 >> v2:172.21.15.163:3300/0 conn(0x7f7e9c062970 0x7f7e9c062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:02.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.758+0000 7f7e9affd700 1 -- 172.21.15.29:0/1084499518 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145726273 0 0) 0x7f7e9c0bcf60 con 0x7f7e9c158600 2024-01-24T14:14:02.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.758+0000 7f7e9affd700 1 -- 172.21.15.29:0/1084499518 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e840033f0 con 0x7f7e9c158600 2024-01-24T14:14:02.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.758+0000 7f7e9bfff700 1 -- 172.21.15.29:0/1084499518 >> v1:172.21.15.163:6789/0 conn(0x7f7e9c066b40 legacy=0x7f7e9c0c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:02.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.758+0000 7f7e9bfff700 1 -- 172.21.15.29:0/1084499518 >> v1:172.21.15.29:6789/0 conn(0x7f7e9c158600 legacy=0x7f7e9c154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:02.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.758+0000 7f7e9affd700 1 -- 172.21.15.29:0/1084499518 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3588892875 0 0) 0x7f7e840033f0 con 0x7f7e9c158600 2024-01-24T14:14:02.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.758+0000 7f7e9bfff700 1 -- 172.21.15.29:0/1084499518 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c0a71a0 con 0x7f7e9c062970 2024-01-24T14:14:02.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.758+0000 7f7e9bfff700 1 --2- 172.21.15.29:0/1084499518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e9c062970 0x7f7e9c062d50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f7e8c009760 tx=0x7f7e8c009060).ready entity=mon.1 client_cookie=986b7f78914fea38 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:02.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.758+0000 7f7e9affd700 1 -- 172.21.15.29:0/1084499518 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e940017f0 con 0x7f7e9c062970 2024-01-24T14:14:02.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.759+0000 7f7e9affd700 1 -- 172.21.15.29:0/1084499518 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c011400 con 0x7f7e9c062970 2024-01-24T14:14:02.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.759+0000 7f7e9affd700 1 -- 172.21.15.29:0/1084499518 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c011780 con 0x7f7e9c062970 2024-01-24T14:14:02.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.759+0000 7f7ea27d0700 1 -- 172.21.15.29:0/1084499518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e9c062970 msgr2=0x7f7e9c062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:02.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.759+0000 7f7ea27d0700 1 --2- 172.21.15.29:0/1084499518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e9c062970 0x7f7e9c062d50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f7e8c009760 tx=0x7f7e8c009060).stop 2024-01-24T14:14:02.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.759+0000 7f7ea27d0700 1 -- 172.21.15.29:0/1084499518 shutdown_connections 2024-01-24T14:14:02.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.759+0000 7f7ea27d0700 1 --2- 172.21.15.29:0/1084499518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e9c062970 0x7f7e9c062d50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:02.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.759+0000 7f7ea27d0700 1 -- 172.21.15.29:0/1084499518 >> 172.21.15.29:0/1084499518 conn(0x7f7e9c0c8580 msgr2=0x7f7e9c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:02.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.759+0000 7f7ea27d0700 1 -- 172.21.15.29:0/1084499518 shutdown_connections 2024-01-24T14:14:02.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.759+0000 7f7ea27d0700 1 -- 172.21.15.29:0/1084499518 wait complete. 2024-01-24T14:14:02.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.760+0000 7f7ea27d0700 1 Processor -- start 2024-01-24T14:14:02.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.761+0000 7f7ea27d0700 1 -- start start 2024-01-24T14:14:02.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.761+0000 7f7ea27d0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7e9c062970 0x7f7e9c1578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:02.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.761+0000 7f7ea27d0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e9c066b40 0x7f7e9c165680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:02.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.761+0000 7f7ea27d0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e9c158600 0x7f7e9c164500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:02.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.761+0000 7f7ea27d0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0a7480 con 0x7f7e9c158600 2024-01-24T14:14:02.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.761+0000 7f7ea27d0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0bf2d0 con 0x7f7e9c066b40 2024-01-24T14:14:02.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.761+0000 7f7ea27d0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0bf5b0 con 0x7f7e9c062970 2024-01-24T14:14:02.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.761+0000 7f7ea0d6d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e9c158600 0x7f7e9c164500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7ea0d6d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e9c158600 0x7f7e9c164500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34374/0 (socket says 172.21.15.29:34374) 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7ea0d6d700 1 -- 172.21.15.29:0/2692332472 learned_addr learned my addr 172.21.15.29:0/2692332472 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7e9bfff700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7e9c062970 0x7f7e9c1578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7e9b7fe700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e9c066b40 0x7f7e9c165680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7ea0d6d700 1 -- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7e9c062970 msgr2=0x7f7e9c1578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7ea0d6d700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7e9c062970 0x7f7e9c1578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7ea0d6d700 1 -- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e9c066b40 msgr2=0x7f7e9c165680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7ea0d6d700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e9c066b40 0x7f7e9c165680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7ea0d6d700 1 -- 172.21.15.29:0/2692332472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c0092a0 con 0x7f7e9c158600 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7ea0d6d700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e9c158600 0x7f7e9c164500 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f7e9400cb90 tx=0x7f7e94001d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7e83fff700 1 -- 172.21.15.29:0/2692332472 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e940020e0 con 0x7f7e9c158600 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7e83fff700 1 -- 172.21.15.29:0/2692332472 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e94004c40 con 0x7f7e9c158600 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7e83fff700 1 -- 172.21.15.29:0/2692332472 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94007170 con 0x7f7e9c158600 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7ea27d0700 1 -- 172.21.15.29:0/2692332472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c164d20 con 0x7f7e9c158600 2024-01-24T14:14:02.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.762+0000 7f7ea27d0700 1 -- 172.21.15.29:0/2692332472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c1652e0 con 0x7f7e9c158600 2024-01-24T14:14:02.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.763+0000 7f7ea27d0700 1 -- 172.21.15.29:0/2692332472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c0475e0 con 0x7f7e9c158600 2024-01-24T14:14:02.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.764+0000 7f7e83fff700 1 -- 172.21.15.29:0/2692332472 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7e940074e0 con 0x7f7e9c158600 2024-01-24T14:14:02.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.764+0000 7f7e83fff700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7e8805f240 0x7f7e88061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:02.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.764+0000 7f7e83fff700 1 -- 172.21.15.29:0/2692332472 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f7e9407a520 con 0x7f7e9c158600 2024-01-24T14:14:02.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.765+0000 7f7e9bfff700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7e8805f240 0x7f7e88061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:02.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.765+0000 7f7e9bfff700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7e8805f240 0x7f7e88061700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7e8c009760 tx=0x7f7e8c014000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:02.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.769+0000 7f7e83fff700 1 -- 172.21.15.29:0/2692332472 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7e9404b270 con 0x7f7e9c158600 2024-01-24T14:14:02.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.996+0000 7f7ea27d0700 1 -- 172.21.15.29:0/2692332472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e9c0bf6f0 con 0x7f7e9c158600 2024-01-24T14:14:02.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.996+0000 7f7e83fff700 1 -- 172.21.15.29:0/2692332472 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f7e9404e890 con 0x7f7e9c158600 2024-01-24T14:14:02.997 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:02.997 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4083072158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":4083072158},{"type":"v1","addr":"172.21.15.163:6837","nonce":4083072158}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/4114592089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4114592089},{"type":"v1","addr":"172.21.15.163:6835","nonce":4114592089}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/926851304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":926851304},{"type":"v1","addr":"172.21.15.29:6835","nonce":926851304}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:13:59.576939+0000","modified":"2024-01-24T14:14:02.594306+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6833/3854229544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3854229544},{"type":"v1","addr":"172.21.15.29:6833","nonce":3854229544}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-24T14:14:03.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.999+0000 7f7ea27d0700 1 -- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7e8805f240 msgr2=0x7f7e88061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.999+0000 7f7ea27d0700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7e8805f240 0x7f7e88061700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7e8c009760 tx=0x7f7e8c014000).stop 2024-01-24T14:14:03.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.999+0000 7f7ea27d0700 1 -- 172.21.15.29:0/2692332472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e9c158600 msgr2=0x7f7e9c164500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.999+0000 7f7ea27d0700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e9c158600 0x7f7e9c164500 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f7e9400cb90 tx=0x7f7e94001d90).stop 2024-01-24T14:14:03.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.999+0000 7f7ea27d0700 1 -- 172.21.15.29:0/2692332472 shutdown_connections 2024-01-24T14:14:03.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.999+0000 7f7ea27d0700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7e8805f240 0x7f7e88061700 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.999+0000 7f7ea27d0700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e9c158600 0x7f7e9c164500 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.999+0000 7f7ea27d0700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7e9c062970 0x7f7e9c1578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.999+0000 7f7ea27d0700 1 --2- 172.21.15.29:0/2692332472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e9c066b40 0x7f7e9c165680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.999+0000 7f7ea27d0700 1 -- 172.21.15.29:0/2692332472 >> 172.21.15.29:0/2692332472 conn(0x7f7e9c0c8580 msgr2=0x7f7e9c14d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:03.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.999+0000 7f7ea27d0700 1 -- 172.21.15.29:0/2692332472 shutdown_connections 2024-01-24T14:14:03.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:02.999+0000 7f7ea27d0700 1 -- 172.21.15.29:0/2692332472 wait complete. 2024-01-24T14:14:03.001 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 43 2024-01-24T14:14:03.013 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:14:03.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.142+0000 7f1bd11e4700 1 Processor -- start 2024-01-24T14:14:03.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.142+0000 7f1bd11e4700 1 -- start start 2024-01-24T14:14:03.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.143+0000 7f1bd11e4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1bcc0c41c0 0x7f1bcc15ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.143+0000 7f1bd11e4700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bcc064120 con 0x7f1bcc0be520 2024-01-24T14:14:03.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.143+0000 7f1bd11e4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bcc0bf6a0 con 0x7f1bcc13a5c0 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.143+0000 7f1bd11e4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc0bea90 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.143+0000 7f1bcb59e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1bcc0c41c0 0x7f1bcc15ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.143+0000 7f1bca59c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1bcc0be520 0x7f1bcc0664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38878/0 (socket says 172.21.15.29:38878) 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.143+0000 7f1bca59c700 1 -- 172.21.15.29:0/3247130789 learned_addr learned my addr 172.21.15.29:0/3247130789 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.143+0000 7f1bcb59e700 1 -- 172.21.15.29:0/3247130789 >> v1:172.21.15.163:6789/0 conn(0x7f1bcc13a5c0 legacy=0x7f1bcc063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.143+0000 7f1bcb59e700 1 -- 172.21.15.29:0/3247130789 >> v1:172.21.15.29:6789/0 conn(0x7f1bcc0be520 legacy=0x7f1bcc0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.143+0000 7f1bc9d9b700 1 -- 172.21.15.29:0/3247130789 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242071750 0 0) 0x7f1bcc064120 con 0x7f1bcc0be520 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.143+0000 7f1bcb59e700 1 -- 172.21.15.29:0/3247130789 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc0bed70 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.144+0000 7f1bcb59e700 1 --2- 172.21.15.29:0/3247130789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bcc0c41c0 0x7f1bcc15ec60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f1bc4009e60 tx=0x7f1bc4009780).ready entity=mon.0 client_cookie=1e4c60065737761c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.144+0000 7f1bc9d9b700 1 -- 172.21.15.29:0/3247130789 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc400e070 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.144+0000 7f1bc9d9b700 1 -- 172.21.15.29:0/3247130789 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc400bc30 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.144+0000 7f1bc9d9b700 1 -- 172.21.15.29:0/3247130789 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4012410 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.144+0000 7f1bd11e4700 1 -- 172.21.15.29:0/3247130789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bcc0c41c0 msgr2=0x7f1bcc15ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.144+0000 7f1bd11e4700 1 --2- 172.21.15.29:0/3247130789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bcc0c41c0 0x7f1bcc15ec60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f1bc4009e60 tx=0x7f1bc4009780).stop 2024-01-24T14:14:03.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.144+0000 7f1bd11e4700 1 -- 172.21.15.29:0/3247130789 shutdown_connections 2024-01-24T14:14:03.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.144+0000 7f1bd11e4700 1 --2- 172.21.15.29:0/3247130789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bcc0c41c0 0x7f1bcc15ec60 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.144+0000 7f1bd11e4700 1 -- 172.21.15.29:0/3247130789 >> 172.21.15.29:0/3247130789 conn(0x7f1bcc0c8bb0 msgr2=0x7f1bcc0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:03.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.144+0000 7f1bd11e4700 1 -- 172.21.15.29:0/3247130789 shutdown_connections 2024-01-24T14:14:03.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.144+0000 7f1bd11e4700 1 -- 172.21.15.29:0/3247130789 wait complete. 2024-01-24T14:14:03.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.146+0000 7f1bd11e4700 1 Processor -- start 2024-01-24T14:14:03.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.146+0000 7f1bd11e4700 1 -- start start 2024-01-24T14:14:03.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.147+0000 7f1bd11e4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0be520 0x7f1bcc13ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.147+0000 7f1bd11e4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bcc0c41c0 0x7f1bcc13cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.147+0000 7f1bd11e4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc13a5c0 0x7f1bcc167d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.147+0000 7f1bd11e4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc13d470 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.147+0000 7f1bd11e4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc139290 con 0x7f1bcc0be520 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.147+0000 7f1bd11e4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc164430 con 0x7f1bcc13a5c0 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bca59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bcc0c41c0 0x7f1bcc13cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bca59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bcc0c41c0 0x7f1bcc13cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34386/0 (socket says 172.21.15.29:34386) 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bca59c700 1 -- 172.21.15.29:0/973417711 learned_addr learned my addr 172.21.15.29:0/973417711 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bcad9d700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0be520 0x7f1bcc13ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bcb59e700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc13a5c0 0x7f1bcc167d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bca59c700 1 -- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc13a5c0 msgr2=0x7f1bcc167d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bca59c700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc13a5c0 0x7f1bcc167d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bca59c700 1 -- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0be520 msgr2=0x7f1bcc13ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bca59c700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0be520 0x7f1bcc13ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bca59c700 1 -- 172.21.15.29:0/973417711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc4009a20 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bcb59e700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc13a5c0 0x7f1bcc167d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:03.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bca59c700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bcc0c41c0 0x7f1bcc13cf80 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f1bbc00c4c0 tx=0x7f1bbc007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:03.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bc2ffd700 1 -- 172.21.15.29:0/973417711 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0072e0 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bc2ffd700 1 -- 172.21.15.29:0/973417711 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc007440 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bc2ffd700 1 -- 172.21.15.29:0/973417711 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0057a0 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bd11e4700 1 -- 172.21.15.29:0/973417711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc164cd0 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.148+0000 7f1bd11e4700 1 -- 172.21.15.29:0/973417711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc164680 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.149+0000 7f1bd11e4700 1 -- 172.21.15.29:0/973417711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc0475e0 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.150+0000 7f1bc2ffd700 1 -- 172.21.15.29:0/973417711 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1bbc005b90 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.150+0000 7f1bc2ffd700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1bac05f270 0x7f1bac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.150+0000 7f1bc2ffd700 1 -- 172.21.15.29:0/973417711 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f1bbc079fd0 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.150+0000 7f1bcad9d700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1bac05f270 0x7f1bac061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.151+0000 7f1bcad9d700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1bac05f270 0x7f1bac061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1bb4003860 tx=0x7f1bb4007d40).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:03.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.155+0000 7f1bc2ffd700 1 -- 172.21.15.29:0/973417711 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1bbc04ac60 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.334+0000 7f1bd11e4700 1 -- 172.21.15.29:0/973417711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bcc0bf050 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.335+0000 7f1bc2ffd700 1 -- 172.21.15.29:0/973417711 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+18184 (secure 0 0 0) 0x7f1bbc04e280 con 0x7f1bcc0c41c0 2024-01-24T14:14:03.336 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:03.336 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":105,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:14:00.578824+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:13:57.364021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-24T14:13:58.553943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:14:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.337+0000 7f1bd11e4700 1 -- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1bac05f270 msgr2=0x7f1bac061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.337+0000 7f1bd11e4700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1bac05f270 0x7f1bac061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1bb4003860 tx=0x7f1bb4007d40).stop 2024-01-24T14:14:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.337+0000 7f1bd11e4700 1 -- 172.21.15.29:0/973417711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bcc0c41c0 msgr2=0x7f1bcc13cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.337+0000 7f1bd11e4700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bcc0c41c0 0x7f1bcc13cf80 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f1bbc00c4c0 tx=0x7f1bbc007c10).stop 2024-01-24T14:14:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.337+0000 7f1bd11e4700 1 -- 172.21.15.29:0/973417711 shutdown_connections 2024-01-24T14:14:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.337+0000 7f1bd11e4700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc13a5c0 0x7f1bcc167d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.337+0000 7f1bd11e4700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0be520 0x7f1bcc13ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.337+0000 7f1bd11e4700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1bac05f270 0x7f1bac061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.337+0000 7f1bd11e4700 1 --2- 172.21.15.29:0/973417711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bcc0c41c0 0x7f1bcc13cf80 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.337+0000 7f1bd11e4700 1 -- 172.21.15.29:0/973417711 >> 172.21.15.29:0/973417711 conn(0x7f1bcc0c8bb0 msgr2=0x7f1bcc0addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:03.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.337+0000 7f1bd11e4700 1 -- 172.21.15.29:0/973417711 shutdown_connections 2024-01-24T14:14:03.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.337+0000 7f1bd11e4700 1 -- 172.21.15.29:0/973417711 wait complete. 2024-01-24T14:14:03.351 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-24T14:14:03.352 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:14:03.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.482+0000 7f74af106700 1 Processor -- start 2024-01-24T14:14:03.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.482+0000 7f74af106700 1 -- start start 2024-01-24T14:14:03.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.483+0000 7f74af106700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f74a80bd850 0x7f74a80bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.483+0000 7f74af106700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74a80c1870 0x7f74a80c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.483+0000 7f74af106700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a8064040 con 0x7f74a80627b0 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.483+0000 7f74af106700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a80be120 con 0x7f74a80bd850 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.483+0000 7f74af106700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80c21e0 con 0x7f74a80c1870 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.483+0000 7f74acea2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f74a80bd850 0x7f74a80bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.483+0000 7f74acea2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f74a80bd850 0x7f74a80bdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59916/0 (socket says 172.21.15.29:59916) 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.483+0000 7f74ad6a3700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f74a80627b0 0x7f74a8062c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39590/0 (socket says 172.21.15.29:39590) 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.483+0000 7f74acea2700 1 -- 172.21.15.29:0/2785458331 learned_addr learned my addr 172.21.15.29:0/2785458331 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.484+0000 7f74a7fff700 1 --2- 172.21.15.29:0/2785458331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74a80c1870 0x7f74a80c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.484+0000 7f74acea2700 1 -- 172.21.15.29:0/2785458331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74a80c1870 msgr2=0x7f74a80c1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.484+0000 7f74acea2700 1 --2- 172.21.15.29:0/2785458331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74a80c1870 0x7f74a80c1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.484+0000 7f74acea2700 1 -- 172.21.15.29:0/2785458331 >> v1:172.21.15.29:6789/0 conn(0x7f74a80627b0 legacy=0x7f74a8062c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.484+0000 7f74acea2700 1 -- 172.21.15.29:0/2785458331 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8063430 con 0x7f74a80bd850 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.484+0000 7f74a77fe700 1 -- 172.21.15.29:0/2785458331 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920913740 0 0) 0x7f74a8064040 con 0x7f74a80627b0 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.484+0000 7f74acea2700 1 --2- 172.21.15.29:0/2785458331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74a80bd850 0x7f74a80bdc30 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f749c006fd0 tx=0x7f749c0090f0).ready entity=mon.0 client_cookie=27e5074b3af028e6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.484+0000 7f74a77fe700 1 -- 172.21.15.29:0/2785458331 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00b580 con 0x7f74a80bd850 2024-01-24T14:14:03.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.484+0000 7f74a77fe700 1 -- 172.21.15.29:0/2785458331 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c00b6e0 con 0x7f74a80bd850 2024-01-24T14:14:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.484+0000 7f74a77fe700 1 -- 172.21.15.29:0/2785458331 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00ba60 con 0x7f74a80bd850 2024-01-24T14:14:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.485+0000 7f74af106700 1 -- 172.21.15.29:0/2785458331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74a80bd850 msgr2=0x7f74a80bdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.485+0000 7f74af106700 1 --2- 172.21.15.29:0/2785458331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74a80bd850 0x7f74a80bdc30 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f749c006fd0 tx=0x7f749c0090f0).stop 2024-01-24T14:14:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.485+0000 7f74af106700 1 -- 172.21.15.29:0/2785458331 shutdown_connections 2024-01-24T14:14:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.485+0000 7f74af106700 1 --2- 172.21.15.29:0/2785458331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74a80bd850 0x7f74a80bdc30 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.485+0000 7f74af106700 1 --2- 172.21.15.29:0/2785458331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74a80c1870 0x7f74a80c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.485+0000 7f74af106700 1 -- 172.21.15.29:0/2785458331 >> 172.21.15.29:0/2785458331 conn(0x7f74a8064aa0 msgr2=0x7f74a8064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.485+0000 7f74af106700 1 -- 172.21.15.29:0/2785458331 shutdown_connections 2024-01-24T14:14:03.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.485+0000 7f74af106700 1 -- 172.21.15.29:0/2785458331 wait complete. 2024-01-24T14:14:03.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.486+0000 7f74af106700 1 Processor -- start 2024-01-24T14:14:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.486+0000 7f74af106700 1 -- start start 2024-01-24T14:14:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.487+0000 7f74af106700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74a80627b0 0x7f74a8138fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.487+0000 7f74af106700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74a80bd850 0x7f74a8137930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.487+0000 7f74af106700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74a80c1870 0x7f74a80ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.487+0000 7f74af106700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80c1560 con 0x7f74a80627b0 2024-01-24T14:14:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.487+0000 7f74af106700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a81369b0 con 0x7f74a80c1870 2024-01-24T14:14:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.487+0000 7f74af106700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a8138b20 con 0x7f74a80bd850 2024-01-24T14:14:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.487+0000 7f74acea2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74a80627b0 0x7f74a8138fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.487+0000 7f74acea2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74a80627b0 0x7f74a8138fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59932/0 (socket says 172.21.15.29:59932) 2024-01-24T14:14:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.487+0000 7f74acea2700 1 -- 172.21.15.29:0/1216354831 learned_addr learned my addr 172.21.15.29:0/1216354831 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.487+0000 7f74a7fff700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74a80bd850 0x7f74a8137930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.487+0000 7f74ad6a3700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74a80c1870 0x7f74a80ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.488+0000 7f74acea2700 1 -- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74a80bd850 msgr2=0x7f74a8137930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.488+0000 7f74acea2700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74a80bd850 0x7f74a8137930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.488+0000 7f74acea2700 1 -- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74a80c1870 msgr2=0x7f74a80ab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.488+0000 7f74acea2700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74a80c1870 0x7f74a80ab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.488+0000 7f74acea2700 1 -- 172.21.15.29:0/1216354831 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c009370 con 0x7f74a80627b0 2024-01-24T14:14:03.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.488+0000 7f74acea2700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74a80627b0 0x7f74a8138fb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f74a80643c0 tx=0x7f749c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:03.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.488+0000 7f74a4ff9700 1 -- 172.21.15.29:0/1216354831 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c000bc0 con 0x7f74a80627b0 2024-01-24T14:14:03.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.488+0000 7f74a4ff9700 1 -- 172.21.15.29:0/1216354831 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c00b850 con 0x7f74a80627b0 2024-01-24T14:14:03.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.488+0000 7f74af106700 1 -- 172.21.15.29:0/1216354831 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a8137f10 con 0x7f74a80627b0 2024-01-24T14:14:03.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.488+0000 7f74a4ff9700 1 -- 172.21.15.29:0/1216354831 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c01c670 con 0x7f74a80627b0 2024-01-24T14:14:03.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.488+0000 7f74af106700 1 -- 172.21.15.29:0/1216354831 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a81384d0 con 0x7f74a80627b0 2024-01-24T14:14:03.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.489+0000 7f74af106700 1 -- 172.21.15.29:0/1216354831 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a8138e00 con 0x7f74a80627b0 2024-01-24T14:14:03.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.490+0000 7f74a4ff9700 1 -- 172.21.15.29:0/1216354831 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f749c01cc70 con 0x7f74a80627b0 2024-01-24T14:14:03.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.490+0000 7f74a4ff9700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f748c05f270 0x7f748c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.490+0000 7f74a4ff9700 1 -- 172.21.15.29:0/1216354831 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f749c07c530 con 0x7f74a80627b0 2024-01-24T14:14:03.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.490+0000 7f74a7fff700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f748c05f270 0x7f748c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.491+0000 7f74a7fff700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f748c05f270 0x7f748c061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f74a81394a0 tx=0x7f7494009060).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:03.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.494+0000 7f74a4ff9700 1 -- 172.21.15.29:0/1216354831 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f749c04d280 con 0x7f74a80627b0 2024-01-24T14:14:03.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.676+0000 7f74af106700 1 -- 172.21.15.29:0/1216354831 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74a8138c60 con 0x7f748c05f270 2024-01-24T14:14:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.687+0000 7f74a4ff9700 1 -- 172.21.15.29:0/1216354831 <== mgr.5864 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+354612 (secure 0 0 0) 0x7f7494009950 con 0x7f748c05f270 2024-01-24T14:14:03.692 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:03.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.691+0000 7f74af106700 1 -- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f748c05f270 msgr2=0x7f748c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.691+0000 7f74af106700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f748c05f270 0x7f748c061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f74a81394a0 tx=0x7f7494009060).stop 2024-01-24T14:14:03.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.691+0000 7f74af106700 1 -- 172.21.15.29:0/1216354831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74a80627b0 msgr2=0x7f74a8138fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.691+0000 7f74af106700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74a80627b0 0x7f74a8138fb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f74a80643c0 tx=0x7f749c017000).stop 2024-01-24T14:14:03.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.691+0000 7f74af106700 1 -- 172.21.15.29:0/1216354831 shutdown_connections 2024-01-24T14:14:03.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.691+0000 7f74af106700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74a80c1870 0x7f74a80ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.691+0000 7f74af106700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f748c05f270 0x7f748c061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.691+0000 7f74af106700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f74a80627b0 0x7f74a8138fb0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.691+0000 7f74af106700 1 --2- 172.21.15.29:0/1216354831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74a80bd850 0x7f74a8137930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.691+0000 7f74af106700 1 -- 172.21.15.29:0/1216354831 >> 172.21.15.29:0/1216354831 conn(0x7f74a8064aa0 msgr2=0x7f74a80c6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:03.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.691+0000 7f74af106700 1 -- 172.21.15.29:0/1216354831 shutdown_connections 2024-01-24T14:14:03.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.691+0000 7f74af106700 1 -- 172.21.15.29:0/1216354831 wait complete. 2024-01-24T14:14:03.696 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:14:03.705 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T14:14:02.106924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":588,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":118,"acting":118,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2423452,"kb_used_data":9820,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752551268,"statfs":{"total":773094113280,"available":770612498432,"internally_reserved":0,"allocated":10055680,"data_stored":5525004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12846,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-64,"num_write_kb":-34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000226"},"pg_stats":[{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615363+0000","last_change":"2024-01-24T14:13:59.610904+0000","last_active":"2024-01-24T14:13:59.615363+0000","last_peered":"2024-01-24T14:13:59.615363+0000","last_clean":"2024-01-24T14:13:59.615363+0000","last_became_active":"2024-01-24T14:13:59.610672+0000","last_became_peered":"2024-01-24T14:13:59.610672+0000","last_unstale":"2024-01-24T14:13:59.615363+0000","last_undegraded":"2024-01-24T14:13:59.615363+0000","last_fullsized":"2024-01-24T14:13:59.615363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3f","version":"104'3","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616800+0000","last_change":"2024-01-24T14:13:59.174978+0000","last_active":"2024-01-24T14:13:59.616800+0000","last_peered":"2024-01-24T14:13:59.616800+0000","last_clean":"2024-01-24T14:13:59.616800+0000","last_became_active":"2024-01-24T14:13:59.174684+0000","last_became_peered":"2024-01-24T14:13:59.174684+0000","last_unstale":"2024-01-24T14:13:59.616800+0000","last_undegraded":"2024-01-24T14:13:59.616800+0000","last_fullsized":"2024-01-24T14:13:59.616800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599212+0000","last_change":"2024-01-24T14:13:59.596129+0000","last_active":"2024-01-24T14:13:59.599212+0000","last_peered":"2024-01-24T14:13:59.599212+0000","last_clean":"2024-01-24T14:13:59.599212+0000","last_became_active":"2024-01-24T14:13:59.595830+0000","last_became_peered":"2024-01-24T14:13:59.595830+0000","last_unstale":"2024-01-24T14:13:59.599212+0000","last_undegraded":"2024-01-24T14:13:59.599212+0000","last_fullsized":"2024-01-24T14:13:59.599212+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591337+0000","last_change":"2024-01-24T14:13:59.190558+0000","last_active":"2024-01-24T14:13:59.591337+0000","last_peered":"2024-01-24T14:13:59.591337+0000","last_clean":"2024-01-24T14:13:59.591337+0000","last_became_active":"2024-01-24T14:13:59.187620+0000","last_became_peered":"2024-01-24T14:13:59.187620+0000","last_unstale":"2024-01-24T14:13:59.591337+0000","last_undegraded":"2024-01-24T14:13:59.591337+0000","last_fullsized":"2024-01-24T14:13:59.591337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616684+0000","last_change":"2024-01-24T14:13:59.609750+0000","last_active":"2024-01-24T14:13:59.616684+0000","last_peered":"2024-01-24T14:13:59.616684+0000","last_clean":"2024-01-24T14:13:59.616684+0000","last_became_active":"2024-01-24T14:13:59.609455+0000","last_became_peered":"2024-01-24T14:13:59.609455+0000","last_unstale":"2024-01-24T14:13:59.616684+0000","last_undegraded":"2024-01-24T14:13:59.616684+0000","last_fullsized":"2024-01-24T14:13:59.616684+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593252+0000","last_change":"2024-01-24T14:13:59.173164+0000","last_active":"2024-01-24T14:13:59.593252+0000","last_peered":"2024-01-24T14:13:59.593252+0000","last_clean":"2024-01-24T14:13:59.593252+0000","last_became_active":"2024-01-24T14:13:59.172816+0000","last_became_peered":"2024-01-24T14:13:59.172816+0000","last_unstale":"2024-01-24T14:13:59.593252+0000","last_undegraded":"2024-01-24T14:13:59.593252+0000","last_fullsized":"2024-01-24T14:13:59.593252+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612433+0000","last_change":"2024-01-24T14:13:59.603989+0000","last_active":"2024-01-24T14:13:59.612433+0000","last_peered":"2024-01-24T14:13:59.612433+0000","last_clean":"2024-01-24T14:13:59.612433+0000","last_became_active":"2024-01-24T14:13:59.603818+0000","last_became_peered":"2024-01-24T14:13:59.603818+0000","last_unstale":"2024-01-24T14:13:59.612433+0000","last_undegraded":"2024-01-24T14:13:59.612433+0000","last_fullsized":"2024-01-24T14:13:59.612433+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581206+0000","last_change":"2024-01-24T14:13:59.168769+0000","last_active":"2024-01-24T14:13:59.581206+0000","last_peered":"2024-01-24T14:13:59.581206+0000","last_clean":"2024-01-24T14:13:59.581206+0000","last_became_active":"2024-01-24T14:13:59.168470+0000","last_became_peered":"2024-01-24T14:13:59.168470+0000","last_unstale":"2024-01-24T14:13:59.581206+0000","last_undegraded":"2024-01-24T14:13:59.581206+0000","last_fullsized":"2024-01-24T14:13:59.581206+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.614739+0000","last_change":"2024-01-24T14:13:59.602824+0000","last_active":"2024-01-24T14:13:59.614739+0000","last_peered":"2024-01-24T14:13:59.614739+0000","last_clean":"2024-01-24T14:13:59.614739+0000","last_became_active":"2024-01-24T14:13:59.602330+0000","last_became_peered":"2024-01-24T14:13:59.602330+0000","last_unstale":"2024-01-24T14:13:59.614739+0000","last_undegraded":"2024-01-24T14:13:59.614739+0000","last_fullsized":"2024-01-24T14:13:59.614739+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612022+0000","last_change":"2024-01-24T14:13:59.603458+0000","last_active":"2024-01-24T14:13:59.612022+0000","last_peered":"2024-01-24T14:13:59.612022+0000","last_clean":"2024-01-24T14:13:59.612022+0000","last_became_active":"2024-01-24T14:13:59.600085+0000","last_became_peered":"2024-01-24T14:13:59.600085+0000","last_unstale":"2024-01-24T14:13:59.612022+0000","last_undegraded":"2024-01-24T14:13:59.612022+0000","last_fullsized":"2024-01-24T14:13:59.612022+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604880+0000","last_change":"2024-01-24T14:13:59.186304+0000","last_active":"2024-01-24T14:13:59.604880+0000","last_peered":"2024-01-24T14:13:59.604880+0000","last_clean":"2024-01-24T14:13:59.604880+0000","last_became_active":"2024-01-24T14:13:59.186017+0000","last_became_peered":"2024-01-24T14:13:59.186017+0000","last_unstale":"2024-01-24T14:13:59.604880+0000","last_undegraded":"2024-01-24T14:13:59.604880+0000","last_fullsized":"2024-01-24T14:13:59.604880+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613474+0000","last_change":"2024-01-24T14:13:59.603461+0000","last_active":"2024-01-24T14:13:59.613474+0000","last_peered":"2024-01-24T14:13:59.613474+0000","last_clean":"2024-01-24T14:13:59.613474+0000","last_became_active":"2024-01-24T14:13:59.603177+0000","last_became_peered":"2024-01-24T14:13:59.603177+0000","last_unstale":"2024-01-24T14:13:59.613474+0000","last_undegraded":"2024-01-24T14:13:59.613474+0000","last_fullsized":"2024-01-24T14:13:59.613474+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616811+0000","last_change":"2024-01-24T14:13:59.176815+0000","last_active":"2024-01-24T14:13:59.616811+0000","last_peered":"2024-01-24T14:13:59.616811+0000","last_clean":"2024-01-24T14:13:59.616811+0000","last_became_active":"2024-01-24T14:13:59.176478+0000","last_became_peered":"2024-01-24T14:13:59.176478+0000","last_unstale":"2024-01-24T14:13:59.616811+0000","last_undegraded":"2024-01-24T14:13:59.616811+0000","last_fullsized":"2024-01-24T14:13:59.616811+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612937+0000","last_change":"2024-01-24T14:13:59.603476+0000","last_active":"2024-01-24T14:13:59.612937+0000","last_peered":"2024-01-24T14:13:59.612937+0000","last_clean":"2024-01-24T14:13:59.612937+0000","last_became_active":"2024-01-24T14:13:59.603267+0000","last_became_peered":"2024-01-24T14:13:59.603267+0000","last_unstale":"2024-01-24T14:13:59.612937+0000","last_undegraded":"2024-01-24T14:13:59.612937+0000","last_fullsized":"2024-01-24T14:13:59.612937+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.583071+0000","last_change":"2024-01-24T14:13:59.174673+0000","last_active":"2024-01-24T14:13:59.583071+0000","last_peered":"2024-01-24T14:13:59.583071+0000","last_clean":"2024-01-24T14:13:59.583071+0000","last_became_active":"2024-01-24T14:13:59.174365+0000","last_became_peered":"2024-01-24T14:13:59.174365+0000","last_unstale":"2024-01-24T14:13:59.583071+0000","last_undegraded":"2024-01-24T14:13:59.583071+0000","last_fullsized":"2024-01-24T14:13:59.583071+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591250+0000","last_change":"2024-01-24T14:13:59.184862+0000","last_active":"2024-01-24T14:13:59.591250+0000","last_peered":"2024-01-24T14:13:59.591250+0000","last_clean":"2024-01-24T14:13:59.591250+0000","last_became_active":"2024-01-24T14:13:59.184560+0000","last_became_peered":"2024-01-24T14:13:59.184560+0000","last_unstale":"2024-01-24T14:13:59.591250+0000","last_undegraded":"2024-01-24T14:13:59.591250+0000","last_fullsized":"2024-01-24T14:13:59.591250+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599235+0000","last_change":"2024-01-24T14:13:59.597440+0000","last_active":"2024-01-24T14:13:59.599235+0000","last_peered":"2024-01-24T14:13:59.599235+0000","last_clean":"2024-01-24T14:13:59.599235+0000","last_became_active":"2024-01-24T14:13:59.597132+0000","last_became_peered":"2024-01-24T14:13:59.597132+0000","last_unstale":"2024-01-24T14:13:59.599235+0000","last_undegraded":"2024-01-24T14:13:59.599235+0000","last_fullsized":"2024-01-24T14:13:59.599235+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.582496+0000","last_change":"2024-01-24T14:13:59.185072+0000","last_active":"2024-01-24T14:13:59.582496+0000","last_peered":"2024-01-24T14:13:59.582496+0000","last_clean":"2024-01-24T14:13:59.582496+0000","last_became_active":"2024-01-24T14:13:59.184040+0000","last_became_peered":"2024-01-24T14:13:59.184040+0000","last_unstale":"2024-01-24T14:13:59.582496+0000","last_undegraded":"2024-01-24T14:13:59.582496+0000","last_fullsized":"2024-01-24T14:13:59.582496+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599036+0000","last_change":"2024-01-24T14:13:59.597766+0000","last_active":"2024-01-24T14:13:59.599036+0000","last_peered":"2024-01-24T14:13:59.599036+0000","last_clean":"2024-01-24T14:13:59.599036+0000","last_became_active":"2024-01-24T14:13:59.597476+0000","last_became_peered":"2024-01-24T14:13:59.597476+0000","last_unstale":"2024-01-24T14:13:59.599036+0000","last_undegraded":"2024-01-24T14:13:59.599036+0000","last_fullsized":"2024-01-24T14:13:59.599036+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602220+0000","last_change":"2024-01-24T14:13:59.598055+0000","last_active":"2024-01-24T14:13:59.602220+0000","last_peered":"2024-01-24T14:13:59.602220+0000","last_clean":"2024-01-24T14:13:59.602220+0000","last_became_active":"2024-01-24T14:13:59.597777+0000","last_became_peered":"2024-01-24T14:13:59.597777+0000","last_unstale":"2024-01-24T14:13:59.602220+0000","last_undegraded":"2024-01-24T14:13:59.602220+0000","last_fullsized":"2024-01-24T14:13:59.602220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581576+0000","last_change":"2024-01-24T14:13:59.185733+0000","last_active":"2024-01-24T14:13:59.581576+0000","last_peered":"2024-01-24T14:13:59.581576+0000","last_clean":"2024-01-24T14:13:59.581576+0000","last_became_active":"2024-01-24T14:13:59.185482+0000","last_became_peered":"2024-01-24T14:13:59.185482+0000","last_unstale":"2024-01-24T14:13:59.581576+0000","last_undegraded":"2024-01-24T14:13:59.581576+0000","last_fullsized":"2024-01-24T14:13:59.581576+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599233+0000","last_change":"2024-01-24T14:13:59.597585+0000","last_active":"2024-01-24T14:13:59.599233+0000","last_peered":"2024-01-24T14:13:59.599233+0000","last_clean":"2024-01-24T14:13:59.599233+0000","last_became_active":"2024-01-24T14:13:59.597114+0000","last_became_peered":"2024-01-24T14:13:59.597114+0000","last_unstale":"2024-01-24T14:13:59.599233+0000","last_undegraded":"2024-01-24T14:13:59.599233+0000","last_fullsized":"2024-01-24T14:13:59.599233+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601777+0000","last_change":"2024-01-24T14:13:59.600823+0000","last_active":"2024-01-24T14:13:59.601777+0000","last_peered":"2024-01-24T14:13:59.601777+0000","last_clean":"2024-01-24T14:13:59.601777+0000","last_became_active":"2024-01-24T14:13:59.600322+0000","last_became_peered":"2024-01-24T14:13:59.600322+0000","last_unstale":"2024-01-24T14:13:59.601777+0000","last_undegraded":"2024-01-24T14:13:59.601777+0000","last_fullsized":"2024-01-24T14:13:59.601777+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615401+0000","last_change":"2024-01-24T14:13:59.611324+0000","last_active":"2024-01-24T14:13:59.615401+0000","last_peered":"2024-01-24T14:13:59.615401+0000","last_clean":"2024-01-24T14:13:59.615401+0000","last_became_active":"2024-01-24T14:13:59.610992+0000","last_became_peered":"2024-01-24T14:13:59.610992+0000","last_unstale":"2024-01-24T14:13:59.615401+0000","last_undegraded":"2024-01-24T14:13:59.615401+0000","last_fullsized":"2024-01-24T14:13:59.615401+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596181+0000","last_change":"2024-01-24T14:13:59.176584+0000","last_active":"2024-01-24T14:13:59.596181+0000","last_peered":"2024-01-24T14:13:59.596181+0000","last_clean":"2024-01-24T14:13:59.596181+0000","last_became_active":"2024-01-24T14:13:59.176352+0000","last_became_peered":"2024-01-24T14:13:59.176352+0000","last_unstale":"2024-01-24T14:13:59.596181+0000","last_undegraded":"2024-01-24T14:13:59.596181+0000","last_fullsized":"2024-01-24T14:13:59.596181+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.582565+0000","last_change":"2024-01-24T14:13:59.177305+0000","last_active":"2024-01-24T14:13:59.582565+0000","last_peered":"2024-01-24T14:13:59.582565+0000","last_clean":"2024-01-24T14:13:59.582565+0000","last_became_active":"2024-01-24T14:13:59.177013+0000","last_became_peered":"2024-01-24T14:13:59.177013+0000","last_unstale":"2024-01-24T14:13:59.582565+0000","last_undegraded":"2024-01-24T14:13:59.582565+0000","last_fullsized":"2024-01-24T14:13:59.582565+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613859+0000","last_change":"2024-01-24T14:13:59.605686+0000","last_active":"2024-01-24T14:13:59.613859+0000","last_peered":"2024-01-24T14:13:59.613859+0000","last_clean":"2024-01-24T14:13:59.613859+0000","last_became_active":"2024-01-24T14:13:59.599468+0000","last_became_peered":"2024-01-24T14:13:59.599468+0000","last_unstale":"2024-01-24T14:13:59.613859+0000","last_undegraded":"2024-01-24T14:13:59.613859+0000","last_fullsized":"2024-01-24T14:13:59.613859+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613713+0000","last_change":"2024-01-24T14:13:59.604787+0000","last_active":"2024-01-24T14:13:59.613713+0000","last_peered":"2024-01-24T14:13:59.613713+0000","last_clean":"2024-01-24T14:13:59.613713+0000","last_became_active":"2024-01-24T14:13:59.604506+0000","last_became_peered":"2024-01-24T14:13:59.604506+0000","last_unstale":"2024-01-24T14:13:59.613713+0000","last_undegraded":"2024-01-24T14:13:59.613713+0000","last_fullsized":"2024-01-24T14:13:59.613713+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593610+0000","last_change":"2024-01-24T14:13:59.177542+0000","last_active":"2024-01-24T14:13:59.593610+0000","last_peered":"2024-01-24T14:13:59.593610+0000","last_clean":"2024-01-24T14:13:59.593610+0000","last_became_active":"2024-01-24T14:13:59.177246+0000","last_became_peered":"2024-01-24T14:13:59.177246+0000","last_unstale":"2024-01-24T14:13:59.593610+0000","last_undegraded":"2024-01-24T14:13:59.593610+0000","last_fullsized":"2024-01-24T14:13:59.593610+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591431+0000","last_change":"2024-01-24T14:13:59.187008+0000","last_active":"2024-01-24T14:13:59.591431+0000","last_peered":"2024-01-24T14:13:59.591431+0000","last_clean":"2024-01-24T14:13:59.591431+0000","last_became_active":"2024-01-24T14:13:59.186690+0000","last_became_peered":"2024-01-24T14:13:59.186690+0000","last_unstale":"2024-01-24T14:13:59.591431+0000","last_undegraded":"2024-01-24T14:13:59.591431+0000","last_fullsized":"2024-01-24T14:13:59.591431+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594464+0000","last_change":"2024-01-24T14:13:59.175676+0000","last_active":"2024-01-24T14:13:59.594464+0000","last_peered":"2024-01-24T14:13:59.594464+0000","last_clean":"2024-01-24T14:13:59.594464+0000","last_became_active":"2024-01-24T14:13:59.175396+0000","last_became_peered":"2024-01-24T14:13:59.175396+0000","last_unstale":"2024-01-24T14:13:59.594464+0000","last_undegraded":"2024-01-24T14:13:59.594464+0000","last_fullsized":"2024-01-24T14:13:59.594464+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613559+0000","last_change":"2024-01-24T14:13:59.604227+0000","last_active":"2024-01-24T14:13:59.613559+0000","last_peered":"2024-01-24T14:13:59.613559+0000","last_clean":"2024-01-24T14:13:59.613559+0000","last_became_active":"2024-01-24T14:13:59.603933+0000","last_became_peered":"2024-01-24T14:13:59.603933+0000","last_unstale":"2024-01-24T14:13:59.613559+0000","last_undegraded":"2024-01-24T14:13:59.613559+0000","last_fullsized":"2024-01-24T14:13:59.613559+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613839+0000","last_change":"2024-01-24T14:13:59.604215+0000","last_active":"2024-01-24T14:13:59.613839+0000","last_peered":"2024-01-24T14:13:59.613839+0000","last_clean":"2024-01-24T14:13:59.613839+0000","last_became_active":"2024-01-24T14:13:59.603918+0000","last_became_peered":"2024-01-24T14:13:59.603918+0000","last_unstale":"2024-01-24T14:13:59.613839+0000","last_undegraded":"2024-01-24T14:13:59.613839+0000","last_fullsized":"2024-01-24T14:13:59.613839+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616746+0000","last_change":"2024-01-24T14:13:59.610454+0000","last_active":"2024-01-24T14:13:59.616746+0000","last_peered":"2024-01-24T14:13:59.616746+0000","last_clean":"2024-01-24T14:13:59.616746+0000","last_became_active":"2024-01-24T14:13:59.610190+0000","last_became_peered":"2024-01-24T14:13:59.610190+0000","last_unstale":"2024-01-24T14:13:59.616746+0000","last_undegraded":"2024-01-24T14:13:59.616746+0000","last_fullsized":"2024-01-24T14:13:59.616746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591156+0000","last_change":"2024-01-24T14:13:59.184839+0000","last_active":"2024-01-24T14:13:59.591156+0000","last_peered":"2024-01-24T14:13:59.591156+0000","last_clean":"2024-01-24T14:13:59.591156+0000","last_became_active":"2024-01-24T14:13:59.184535+0000","last_became_peered":"2024-01-24T14:13:59.184535+0000","last_unstale":"2024-01-24T14:13:59.591156+0000","last_undegraded":"2024-01-24T14:13:59.591156+0000","last_fullsized":"2024-01-24T14:13:59.591156+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613963+0000","last_change":"2024-01-24T14:13:59.174910+0000","last_active":"2024-01-24T14:13:59.613963+0000","last_peered":"2024-01-24T14:13:59.613963+0000","last_clean":"2024-01-24T14:13:59.613963+0000","last_became_active":"2024-01-24T14:13:59.174603+0000","last_became_peered":"2024-01-24T14:13:59.174603+0000","last_unstale":"2024-01-24T14:13:59.613963+0000","last_undegraded":"2024-01-24T14:13:59.613963+0000","last_fullsized":"2024-01-24T14:13:59.613963+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.618795+0000","last_change":"2024-01-24T14:13:59.614234+0000","last_active":"2024-01-24T14:13:59.618795+0000","last_peered":"2024-01-24T14:13:59.618795+0000","last_clean":"2024-01-24T14:13:59.618795+0000","last_became_active":"2024-01-24T14:13:59.614018+0000","last_became_peered":"2024-01-24T14:13:59.614018+0000","last_unstale":"2024-01-24T14:13:59.618795+0000","last_undegraded":"2024-01-24T14:13:59.618795+0000","last_fullsized":"2024-01-24T14:13:59.618795+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613805+0000","last_change":"2024-01-24T14:13:59.605325+0000","last_active":"2024-01-24T14:13:59.613805+0000","last_peered":"2024-01-24T14:13:59.613805+0000","last_clean":"2024-01-24T14:13:59.613805+0000","last_became_active":"2024-01-24T14:13:59.605027+0000","last_became_peered":"2024-01-24T14:13:59.605027+0000","last_unstale":"2024-01-24T14:13:59.613805+0000","last_undegraded":"2024-01-24T14:13:59.613805+0000","last_fullsized":"2024-01-24T14:13:59.613805+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615224+0000","last_change":"2024-01-24T14:13:59.610385+0000","last_active":"2024-01-24T14:13:59.615224+0000","last_peered":"2024-01-24T14:13:59.615224+0000","last_clean":"2024-01-24T14:13:59.615224+0000","last_became_active":"2024-01-24T14:13:59.609240+0000","last_became_peered":"2024-01-24T14:13:59.609240+0000","last_unstale":"2024-01-24T14:13:59.615224+0000","last_undegraded":"2024-01-24T14:13:59.615224+0000","last_fullsized":"2024-01-24T14:13:59.615224+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601872+0000","last_change":"2024-01-24T14:13:59.178489+0000","last_active":"2024-01-24T14:13:59.601872+0000","last_peered":"2024-01-24T14:13:59.601872+0000","last_clean":"2024-01-24T14:13:59.601872+0000","last_became_active":"2024-01-24T14:13:59.178185+0000","last_became_peered":"2024-01-24T14:13:59.178185+0000","last_unstale":"2024-01-24T14:13:59.601872+0000","last_undegraded":"2024-01-24T14:13:59.601872+0000","last_fullsized":"2024-01-24T14:13:59.601872+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616653+0000","last_change":"2024-01-24T14:13:59.609527+0000","last_active":"2024-01-24T14:13:59.616653+0000","last_peered":"2024-01-24T14:13:59.616653+0000","last_clean":"2024-01-24T14:13:59.616653+0000","last_became_active":"2024-01-24T14:13:59.609180+0000","last_became_peered":"2024-01-24T14:13:59.609180+0000","last_unstale":"2024-01-24T14:13:59.616653+0000","last_undegraded":"2024-01-24T14:13:59.616653+0000","last_fullsized":"2024-01-24T14:13:59.616653+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591353+0000","last_change":"2024-01-24T14:13:59.187223+0000","last_active":"2024-01-24T14:13:59.591353+0000","last_peered":"2024-01-24T14:13:59.591353+0000","last_clean":"2024-01-24T14:13:59.591353+0000","last_became_active":"2024-01-24T14:13:59.186908+0000","last_became_peered":"2024-01-24T14:13:59.186908+0000","last_unstale":"2024-01-24T14:13:59.591353+0000","last_undegraded":"2024-01-24T14:13:59.591353+0000","last_fullsized":"2024-01-24T14:13:59.591353+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616523+0000","last_change":"2024-01-24T14:13:59.606110+0000","last_active":"2024-01-24T14:13:59.616523+0000","last_peered":"2024-01-24T14:13:59.616523+0000","last_clean":"2024-01-24T14:13:59.616523+0000","last_became_active":"2024-01-24T14:13:59.605783+0000","last_became_peered":"2024-01-24T14:13:59.605783+0000","last_unstale":"2024-01-24T14:13:59.616523+0000","last_undegraded":"2024-01-24T14:13:59.616523+0000","last_fullsized":"2024-01-24T14:13:59.616523+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581385+0000","last_change":"2024-01-24T14:13:59.173283+0000","last_active":"2024-01-24T14:13:59.581385+0000","last_peered":"2024-01-24T14:13:59.581385+0000","last_clean":"2024-01-24T14:13:59.581385+0000","last_became_active":"2024-01-24T14:13:59.172965+0000","last_became_peered":"2024-01-24T14:13:59.172965+0000","last_unstale":"2024-01-24T14:13:59.581385+0000","last_undegraded":"2024-01-24T14:13:59.581385+0000","last_fullsized":"2024-01-24T14:13:59.581385+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591632+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:13:59.591632+0000","last_peered":"2024-01-24T14:13:59.591632+0000","last_clean":"2024-01-24T14:13:59.591632+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:13:59.591632+0000","last_undegraded":"2024-01-24T14:13:59.591632+0000","last_fullsized":"2024-01-24T14:13:59.591632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":89,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-24T14:13:54.182751+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:13:54.182751+0000","last_peered":"2024-01-24T14:13:54.182751+0000","last_clean":"2024-01-24T14:13:54.182751+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:13:54.182751+0000","last_undegraded":"2024-01-24T14:13:54.182751+0000","last_fullsized":"2024-01-24T14:13:54.182751+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613485+0000","last_change":"2024-01-24T14:13:59.603439+0000","last_active":"2024-01-24T14:13:59.613485+0000","last_peered":"2024-01-24T14:13:59.613485+0000","last_clean":"2024-01-24T14:13:59.613485+0000","last_became_active":"2024-01-24T14:13:59.603116+0000","last_became_peered":"2024-01-24T14:13:59.603116+0000","last_unstale":"2024-01-24T14:13:59.613485+0000","last_undegraded":"2024-01-24T14:13:59.613485+0000","last_fullsized":"2024-01-24T14:13:59.613485+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581282+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:13:59.581282+0000","last_peered":"2024-01-24T14:13:59.581282+0000","last_clean":"2024-01-24T14:13:59.581282+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:13:59.581282+0000","last_undegraded":"2024-01-24T14:13:59.581282+0000","last_fullsized":"2024-01-24T14:13:59.581282+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613322+0000","last_change":"2024-01-24T14:13:59.601159+0000","last_active":"2024-01-24T14:13:59.613322+0000","last_peered":"2024-01-24T14:13:59.613322+0000","last_clean":"2024-01-24T14:13:59.613322+0000","last_became_active":"2024-01-24T14:13:59.600870+0000","last_became_peered":"2024-01-24T14:13:59.600870+0000","last_unstale":"2024-01-24T14:13:59.613322+0000","last_undegraded":"2024-01-24T14:13:59.613322+0000","last_fullsized":"2024-01-24T14:13:59.613322+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.595988+0000","last_change":"2024-01-24T14:13:59.187233+0000","last_active":"2024-01-24T14:13:59.595988+0000","last_peered":"2024-01-24T14:13:59.595988+0000","last_clean":"2024-01-24T14:13:59.595988+0000","last_became_active":"2024-01-24T14:13:59.186954+0000","last_became_peered":"2024-01-24T14:13:59.186954+0000","last_unstale":"2024-01-24T14:13:59.595988+0000","last_undegraded":"2024-01-24T14:13:59.595988+0000","last_fullsized":"2024-01-24T14:13:59.595988+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":89,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-24T14:13:54.182751+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:13:54.182751+0000","last_peered":"2024-01-24T14:13:54.182751+0000","last_clean":"2024-01-24T14:13:54.182751+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:13:54.182751+0000","last_undegraded":"2024-01-24T14:13:54.182751+0000","last_fullsized":"2024-01-24T14:13:54.182751+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594597+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:13:59.594597+0000","last_peered":"2024-01-24T14:13:59.594597+0000","last_clean":"2024-01-24T14:13:59.594597+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:13:59.594597+0000","last_undegraded":"2024-01-24T14:13:59.594597+0000","last_fullsized":"2024-01-24T14:13:59.594597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594511+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:13:59.594511+0000","last_peered":"2024-01-24T14:13:59.594511+0000","last_clean":"2024-01-24T14:13:59.594511+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:13:59.594511+0000","last_undegraded":"2024-01-24T14:13:59.594511+0000","last_fullsized":"2024-01-24T14:13:59.594511+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.608535+0000","last_change":"2024-01-24T14:13:59.600293+0000","last_active":"2024-01-24T14:13:59.608535+0000","last_peered":"2024-01-24T14:13:59.608535+0000","last_clean":"2024-01-24T14:13:59.608535+0000","last_became_active":"2024-01-24T14:13:59.599864+0000","last_became_peered":"2024-01-24T14:13:59.599864+0000","last_unstale":"2024-01-24T14:13:59.608535+0000","last_undegraded":"2024-01-24T14:13:59.608535+0000","last_fullsized":"2024-01-24T14:13:59.608535+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":87,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596080+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:13:59.596080+0000","last_peered":"2024-01-24T14:13:59.596080+0000","last_clean":"2024-01-24T14:13:59.596080+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:13:59.596080+0000","last_undegraded":"2024-01-24T14:13:59.596080+0000","last_fullsized":"2024-01-24T14:13:59.596080+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591436+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:13:59.591436+0000","last_peered":"2024-01-24T14:13:59.591436+0000","last_clean":"2024-01-24T14:13:59.591436+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:13:59.591436+0000","last_undegraded":"2024-01-24T14:13:59.591436+0000","last_fullsized":"2024-01-24T14:13:59.591436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593811+0000","last_change":"2024-01-24T14:13:59.587516+0000","last_active":"2024-01-24T14:13:59.593811+0000","last_peered":"2024-01-24T14:13:59.593811+0000","last_clean":"2024-01-24T14:13:59.593811+0000","last_became_active":"2024-01-24T14:13:59.587242+0000","last_became_peered":"2024-01-24T14:13:59.587242+0000","last_unstale":"2024-01-24T14:13:59.593811+0000","last_undegraded":"2024-01-24T14:13:59.593811+0000","last_fullsized":"2024-01-24T14:13:59.593811+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581221+0000","last_change":"2024-01-24T14:13:59.176122+0000","last_active":"2024-01-24T14:13:59.581221+0000","last_peered":"2024-01-24T14:13:59.581221+0000","last_clean":"2024-01-24T14:13:59.581221+0000","last_became_active":"2024-01-24T14:13:59.175794+0000","last_became_peered":"2024-01-24T14:13:59.175794+0000","last_unstale":"2024-01-24T14:13:59.581221+0000","last_undegraded":"2024-01-24T14:13:59.581221+0000","last_fullsized":"2024-01-24T14:13:59.581221+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":80,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-24T14:13:54.605774+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:13:54.605774+0000","last_peered":"2024-01-24T14:13:54.605774+0000","last_clean":"2024-01-24T14:13:54.605774+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:13:54.605774+0000","last_undegraded":"2024-01-24T14:13:54.605774+0000","last_fullsized":"2024-01-24T14:13:54.605774+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":56,"acting":56,"num_store_stats":3},{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":588,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":44,"acting":44,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575038,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1290240,"data_stored":710498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.26000000000000001},"max":{"1min":0.80600000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.32700000000000001},"max":{"1min":0.753,"5min":0.82599999999999996,"15min":0.83499999999999996},"last":0.58899999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.312},"max":{"1min":0.75600000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":3.2570000000000001},"last":0.53000000000000003}]},{"osd":2,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.51700000000000002,"5min":0.40400000000000003,"15min":0.32000000000000001},"max":{"1min":0.79100000000000004,"5min":0.872,"15min":0.876},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.39300000000000002,"15min":0.36799999999999999},"max":{"1min":0.80000000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66700000000000004}]},{"osd":3,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.36499999999999999},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.85099999999999998,"15min":3.0049999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48899999999999999},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.84699999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.60399999999999998,"5min":0.70599999999999996,"15min":0.83599999999999997},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.501,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.29999999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.80900000000000005},"last":0.79400000000000004}]},{"osd":6,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.621},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.80700000000000005,"15min":0.87},"last":0.72199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575038,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302400,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069440,"statfs":{"total":96636764160,"available":96327106560,"internally_reserved":0,"allocated":1236992,"data_stored":679077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.59499999999999997},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73699999999999999,"5min":0.85699999999999998,"15min":0.86899999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":0.84899999999999998,"15min":1.3859999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73499999999999999,"5min":0.77000000000000002,"15min":1.353},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.78700000000000003,"5min":1.032,"15min":1.032},"last":0.46999999999999997}]},{"osd":2,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60099999999999998,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.39100000000000001,"15min":0.34300000000000003},"max":{"1min":0.78600000000000003,"5min":0.88500000000000001,"15min":6.8129999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.4079999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.66900000000000004},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81699999999999995,"5min":0.82799999999999996,"15min":7.016},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.43099999999999999,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.77400000000000002,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.63800000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.504,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":1.165,"15min":1.165},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.314},"max":{"1min":0.65500000000000003,"5min":0.82299999999999995,"15min":6.8719999999999999},"last":0.624}]},{"osd":5,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":0.86699999999999999,"15min":6.976},"last":0.497},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80200000000000005,"5min":0.85699999999999998,"15min":7.8789999999999996},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.747,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":0.89600000000000002,"15min":6.9080000000000004},"last":0.66200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302456,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069384,"statfs":{"total":96636764160,"available":96327049216,"internally_reserved":0,"allocated":1294336,"data_stored":710068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.51900000000000002,"15min":0.443},"min":{"1min":0.311,"5min":0.245,"15min":0.23400000000000001},"max":{"1min":0.57899999999999996,"5min":15.298999999999999,"15min":15.298999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.55000000000000004,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.57499999999999996,"5min":14.654,"15min":14.654},"last":0.38}]},{"osd":2,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45900000000000002,"15min":0.45100000000000001},"min":{"1min":0.34899999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.59799999999999998,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.25800000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47099999999999997,"15min":0.45700000000000002},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88600000000000001,"5min":1.073,"15min":1.137},"last":0.441},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.45900000000000002,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.218,"15min":1.218},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":1.1040000000000001,"15min":0.83299999999999996},"min":{"1min":0.47999999999999998,"5min":0.45000000000000001,"15min":0.443},"max":{"1min":0.86799999999999999,"5min":38.808,"15min":38.808},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":1.077,"15min":0.81200000000000006},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.40500000000000003},"max":{"1min":0.81299999999999994,"5min":38.823,"15min":38.823},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.46000000000000002,"15min":0.43099999999999999},"max":{"1min":0.94899999999999995,"5min":1.0449999999999999,"15min":1.0509999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.40000000000000002},"max":{"1min":0.91300000000000003,"5min":1.1559999999999999,"15min":1.1879999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.52500000000000002,"5min":0.49399999999999999,"15min":0.45300000000000001},"max":{"1min":0.98499999999999999,"5min":1.105,"15min":1.155},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.54900000000000004,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":0.93500000000000005,"5min":1.123,"15min":1.131},"last":0.58499999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542335,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302396,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069444,"statfs":{"total":96636764160,"available":96327110656,"internally_reserved":0,"allocated":1232896,"data_stored":678116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39600000000000002},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.57499999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.379}]},{"osd":2,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44800000000000001,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.28199999999999997},"max":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.66700000000000004},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.28799999999999998},"max":{"1min":0.61399999999999999,"5min":0.754,"15min":0.79400000000000004},"last":0.51300000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.33000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54500000000000004,"5min":1.482,"15min":1.482},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.496},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":0.872,"15min":0.872},"last":0.503}]},{"osd":4,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81499999999999995,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.64700000000000002}]},{"osd":5,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.875,"5min":1.655,"15min":1.655},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.42999999999999999},"max":{"1min":0.77200000000000002,"5min":0.94799999999999995,"15min":1.0840000000000001},"last":0.68700000000000006}]},{"osd":6,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.875,"5min":0.90700000000000003,"15min":1.137},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":0.96499999999999997,"15min":1.117},"last":0.71999999999999997}]},{"osd":7,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47099999999999997},"max":{"1min":0.86899999999999999,"5min":0.96099999999999997,"15min":0.98199999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.49099999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.78500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575038,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306548,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94065292,"statfs":{"total":96636764160,"available":96322859008,"internally_reserved":0,"allocated":1290240,"data_stored":710014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.29499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.65800000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.24399999999999999},"max":{"1min":0.54500000000000004,"5min":0.65900000000000003,"15min":0.67100000000000004},"last":0.46500000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.32500000000000001,"5min":0.27800000000000002,"15min":0.27300000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.67200000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.255},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.55600000000000005}]},{"osd":3,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.45000000000000001},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64300000000000002,"5min":0.67700000000000005,"15min":0.71699999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.38900000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.72799999999999998},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.502,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85899999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.55400000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.92200000000000004,"5min":0.996,"15min":1.042},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.53400000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83599999999999997,"5min":1.036,"15min":1.0469999999999999},"last":0.622},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90000000000000002,"5min":1.107,"15min":1.107},"last":0.70199999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.82899999999999996,"5min":1.0569999999999999,"15min":1.1100000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.57299999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84699999999999998,"5min":1.141,"15min":1.141},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.52400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":0.96199999999999997,"15min":1.054},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.88900000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.81000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302400,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069440,"statfs":{"total":96636764160,"available":96327106560,"internally_reserved":0,"allocated":1236992,"data_stored":679077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45100000000000001,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.76400000000000001,"5min":7.593,"15min":7.593},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48299999999999998,"15min":0.437},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56200000000000006,"5min":8.4239999999999995,"15min":8.4239999999999995},"last":0.34699999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.69099999999999995},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.221,"15min":0.221},"max":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.68799999999999994},"last":0.371}]},{"osd":2,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.496},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.66500000000000004,"5min":0.76300000000000001,"15min":0.77200000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.78800000000000003},"last":0.44900000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64500000000000002,"15min":0.63500000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.495,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":0.81599999999999995,"5min":1.976,"15min":1.976},"last":0.53600000000000003}]},{"osd":5,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91500000000000004,"5min":1.53,"15min":1.53},"last":0.629},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.97999999999999998,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72799999999999998,"15min":0.72099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.72999999999999998},"min":{"1min":0.495,"5min":0.49199999999999999,"15min":0.45800000000000002},"max":{"1min":0.93500000000000005,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.61799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302400,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069440,"statfs":{"total":96636764160,"available":96327106560,"internally_reserved":0,"allocated":1236992,"data_stored":679077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78300000000000003,"5min":0.80100000000000005,"15min":0.88600000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.84399999999999997},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.83099999999999996,"15min":0.85299999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.755,"5min":0.79800000000000004,"15min":1.506},"last":0.52100000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.80200000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56399999999999995}]},{"osd":3,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72899999999999998,"5min":0.77800000000000002,"15min":0.80600000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.81000000000000005,"15min":0.86299999999999999},"last":0.626}]},{"osd":5,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.92000000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.496,"15min":0.5},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65700000000000003,"5min":0.72899999999999998,"15min":1.556},"last":0.61599999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.753,"5min":0.78500000000000003,"15min":0.86099999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76400000000000001,"5min":0.82099999999999995,"15min":1.0409999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.38500000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.88100000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.307,"15min":0.307},"max":{"1min":0.72699999999999998,"5min":0.81399999999999995,"15min":1.26},"last":0.69699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302400,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069440,"statfs":{"total":96636764160,"available":96327106560,"internally_reserved":0,"allocated":1236992,"data_stored":679077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64400000000000002,"5min":0.85199999999999998,"15min":1.101},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.309,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71199999999999997,"5min":0.82699999999999996,"15min":1.1439999999999999},"last":0.53700000000000003}]},{"osd":1,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56000000000000005,"15min":0.57699999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67200000000000004,"5min":0.84099999999999997,"15min":1.093},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69199999999999995,"5min":0.81299999999999994,"15min":1.1639999999999999},"last":0.436}]},{"osd":2,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.68400000000000005,"5min":0.84299999999999997,"15min":1.1100000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.35499999999999998},"max":{"1min":0.67400000000000004,"5min":0.83599999999999997,"15min":1.1220000000000001},"last":0.40600000000000003}]},{"osd":3,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.38500000000000001},"max":{"1min":0.72199999999999998,"5min":0.80500000000000005,"15min":1.139},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.496,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":1.0609999999999999},"last":0.61499999999999999}]},{"osd":4,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70199999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.505,"15min":0.503},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.90000000000000002},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.32900000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":1.1839999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77400000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.63800000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72999999999999998,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.5},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.33200000000000002},"max":{"1min":0.78200000000000003,"5min":0.95999999999999996,"15min":1.2030000000000001},"last":0.54500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:14:03.711 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:14:03.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.878+0000 7f301c359700 1 Processor -- start 2024-01-24T14:14:03.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.878+0000 7f301c359700 1 -- start start 2024-01-24T14:14:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301c359700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30140c4a10 0x7f30140bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301c359700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f30140670a0 0x7f3014158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301c359700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140664f0 0x7f3014066980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301c359700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30141582b0 con 0x7f30140c4a10 2024-01-24T14:14:03.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301c359700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3014066e70 con 0x7f30140670a0 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301c359700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3014157ca0 con 0x7f30140664f0 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301a0f5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30140c4a10 0x7f30140bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301a0f5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30140c4a10 0x7f30140bf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59938/0 (socket says 172.21.15.29:59938) 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301a0f5700 1 -- 172.21.15.29:0/4201419885 learned_addr learned my addr 172.21.15.29:0/4201419885 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301a8f6700 1 --2- 172.21.15.29:0/4201419885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140664f0 0x7f3014066980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f30198f4700 1 --2- 172.21.15.29:0/4201419885 >> v2:172.21.15.163:3300/0 conn(0x7f30140670a0 0x7f3014158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301a0f5700 1 -- 172.21.15.29:0/4201419885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140664f0 msgr2=0x7f3014066980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301a0f5700 1 --2- 172.21.15.29:0/4201419885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140664f0 0x7f3014066980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301a0f5700 1 -- 172.21.15.29:0/4201419885 >> v2:172.21.15.163:3300/0 conn(0x7f30140670a0 msgr2=0x7f3014158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301a0f5700 1 --2- 172.21.15.29:0/4201419885 >> v2:172.21.15.163:3300/0 conn(0x7f30140670a0 0x7f3014158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301a0f5700 1 -- 172.21.15.29:0/4201419885 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3014063c60 con 0x7f30140c4a10 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.879+0000 7f301a8f6700 1 --2- 172.21.15.29:0/4201419885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140664f0 0x7f3014066980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.880+0000 7f301a0f5700 1 --2- 172.21.15.29:0/4201419885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140c4a10 0x7f30140bf6d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f3014068cb0 tx=0x7f30100090f0).ready entity=mon.0 client_cookie=8a43134bca458fb0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:03.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.880+0000 7f30190f3700 1 -- 172.21.15.29:0/4201419885 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301000b520 con 0x7f30140c4a10 2024-01-24T14:14:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.880+0000 7f30190f3700 1 -- 172.21.15.29:0/4201419885 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301000b680 con 0x7f30140c4a10 2024-01-24T14:14:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.880+0000 7f30190f3700 1 -- 172.21.15.29:0/4201419885 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301000ba00 con 0x7f30140c4a10 2024-01-24T14:14:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.880+0000 7f301c359700 1 -- 172.21.15.29:0/4201419885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140c4a10 msgr2=0x7f30140bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.880+0000 7f301c359700 1 --2- 172.21.15.29:0/4201419885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140c4a10 0x7f30140bf6d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f3014068cb0 tx=0x7f30100090f0).stop 2024-01-24T14:14:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.880+0000 7f301c359700 1 -- 172.21.15.29:0/4201419885 shutdown_connections 2024-01-24T14:14:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.880+0000 7f301c359700 1 --2- 172.21.15.29:0/4201419885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140664f0 0x7f3014066980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.880+0000 7f301c359700 1 --2- 172.21.15.29:0/4201419885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140c4a10 0x7f30140bf6d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.880+0000 7f301c359700 1 --2- 172.21.15.29:0/4201419885 >> v2:172.21.15.163:3300/0 conn(0x7f30140670a0 0x7f3014158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.880+0000 7f301c359700 1 -- 172.21.15.29:0/4201419885 >> 172.21.15.29:0/4201419885 conn(0x7f30140c8bb0 msgr2=0x7f30140c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.881+0000 7f301c359700 1 -- 172.21.15.29:0/4201419885 shutdown_connections 2024-01-24T14:14:03.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.881+0000 7f301c359700 1 -- 172.21.15.29:0/4201419885 wait complete. 2024-01-24T14:14:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.882+0000 7f301c359700 1 Processor -- start 2024-01-24T14:14:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.882+0000 7f301c359700 1 -- start start 2024-01-24T14:14:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.882+0000 7f301c359700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140664f0 0x7f30140c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.882+0000 7f301c359700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30140670a0 0x7f30140c0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.882+0000 7f301c359700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140c4a10 0x7f30141474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301c359700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30140c23f0 con 0x7f30140c4a10 2024-01-24T14:14:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301c359700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30140c0850 con 0x7f30140670a0 2024-01-24T14:14:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301c359700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3014156eb0 con 0x7f30140664f0 2024-01-24T14:14:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301a8f6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140c4a10 0x7f30141474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301a8f6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140c4a10 0x7f30141474d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59946/0 (socket says 172.21.15.29:59946) 2024-01-24T14:14:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301a8f6700 1 -- 172.21.15.29:0/3945360982 learned_addr learned my addr 172.21.15.29:0/3945360982 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:03.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f30198f4700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30140670a0 0x7f30140c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301a0f5700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140664f0 0x7f30140c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301a8f6700 1 -- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140664f0 msgr2=0x7f30140c1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301a8f6700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140664f0 0x7f30140c1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301a8f6700 1 -- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30140670a0 msgr2=0x7f30140c0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301a8f6700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30140670a0 0x7f30140c0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301a8f6700 1 -- 172.21.15.29:0/3945360982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3010009370 con 0x7f30140c4a10 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301a8f6700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140c4a10 0x7f30141474d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f300c009fc0 tx=0x7f300c0099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f300a7fc700 1 -- 172.21.15.29:0/3945360982 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c00dbe0 con 0x7f30140c4a10 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f300a7fc700 1 -- 172.21.15.29:0/3945360982 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c00f040 con 0x7f30140c4a10 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301c359700 1 -- 172.21.15.29:0/3945360982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30140c0ab0 con 0x7f30140c4a10 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f300a7fc700 1 -- 172.21.15.29:0/3945360982 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c0133f0 con 0x7f30140c4a10 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.883+0000 7f301c359700 1 -- 172.21.15.29:0/3945360982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30141563d0 con 0x7f30140c4a10 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.884+0000 7f301c359700 1 -- 172.21.15.29:0/3945360982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30140475e0 con 0x7f30140c4a10 2024-01-24T14:14:03.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.885+0000 7f300a7fc700 1 -- 172.21.15.29:0/3945360982 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f300c0137e0 con 0x7f30140c4a10 2024-01-24T14:14:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.885+0000 7f300a7fc700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2ff805f270 0x7f2ff8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.885+0000 7f300a7fc700 1 -- 172.21.15.29:0/3945360982 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f300c07ad80 con 0x7f30140c4a10 2024-01-24T14:14:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.885+0000 7f301a0f5700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2ff805f270 0x7f2ff8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:03.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.886+0000 7f301a0f5700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2ff805f270 0x7f2ff8061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f301000b2e0 tx=0x7f3010017000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:03.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:03.890+0000 7f300a7fc700 1 -- 172.21.15.29:0/3945360982 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f300c04bad0 con 0x7f30140c4a10 2024-01-24T14:14:04.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.068+0000 7f301c359700 1 -- 172.21.15.29:0/3945360982 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3014063f10 con 0x7f2ff805f270 2024-01-24T14:14:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.078+0000 7f300a7fc700 1 -- 172.21.15.29:0/3945360982 <== mgr.5864 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+354612 (secure 0 0 0) 0x7f3014063f10 con 0x7f2ff805f270 2024-01-24T14:14:04.083 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:04.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.081+0000 7f301c359700 1 -- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2ff805f270 msgr2=0x7f2ff8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.081+0000 7f301c359700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2ff805f270 0x7f2ff8061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f301000b2e0 tx=0x7f3010017000).stop 2024-01-24T14:14:04.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.081+0000 7f301c359700 1 -- 172.21.15.29:0/3945360982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140c4a10 msgr2=0x7f30141474d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.081+0000 7f301c359700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140c4a10 0x7f30141474d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f300c009fc0 tx=0x7f300c0099b0).stop 2024-01-24T14:14:04.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.081+0000 7f301c359700 1 -- 172.21.15.29:0/3945360982 shutdown_connections 2024-01-24T14:14:04.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.081+0000 7f301c359700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2ff805f270 0x7f2ff8061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.081+0000 7f301c359700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140c4a10 0x7f30141474d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.081+0000 7f301c359700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140664f0 0x7f30140c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.081+0000 7f301c359700 1 --2- 172.21.15.29:0/3945360982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30140670a0 0x7f30140c0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.081+0000 7f301c359700 1 -- 172.21.15.29:0/3945360982 >> 172.21.15.29:0/3945360982 conn(0x7f30140c8bb0 msgr2=0x7f30140c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:04.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.081+0000 7f301c359700 1 -- 172.21.15.29:0/3945360982 shutdown_connections 2024-01-24T14:14:04.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.081+0000 7f301c359700 1 -- 172.21.15.29:0/3945360982 wait complete. 2024-01-24T14:14:04.087 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:14:04.095 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T14:14:02.106924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":588,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":118,"acting":118,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2423452,"kb_used_data":9820,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752551268,"statfs":{"total":773094113280,"available":770612498432,"internally_reserved":0,"allocated":10055680,"data_stored":5525004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12846,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-64,"num_write_kb":-34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000226"},"pg_stats":[{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615363+0000","last_change":"2024-01-24T14:13:59.610904+0000","last_active":"2024-01-24T14:13:59.615363+0000","last_peered":"2024-01-24T14:13:59.615363+0000","last_clean":"2024-01-24T14:13:59.615363+0000","last_became_active":"2024-01-24T14:13:59.610672+0000","last_became_peered":"2024-01-24T14:13:59.610672+0000","last_unstale":"2024-01-24T14:13:59.615363+0000","last_undegraded":"2024-01-24T14:13:59.615363+0000","last_fullsized":"2024-01-24T14:13:59.615363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3f","version":"104'3","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616800+0000","last_change":"2024-01-24T14:13:59.174978+0000","last_active":"2024-01-24T14:13:59.616800+0000","last_peered":"2024-01-24T14:13:59.616800+0000","last_clean":"2024-01-24T14:13:59.616800+0000","last_became_active":"2024-01-24T14:13:59.174684+0000","last_became_peered":"2024-01-24T14:13:59.174684+0000","last_unstale":"2024-01-24T14:13:59.616800+0000","last_undegraded":"2024-01-24T14:13:59.616800+0000","last_fullsized":"2024-01-24T14:13:59.616800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599212+0000","last_change":"2024-01-24T14:13:59.596129+0000","last_active":"2024-01-24T14:13:59.599212+0000","last_peered":"2024-01-24T14:13:59.599212+0000","last_clean":"2024-01-24T14:13:59.599212+0000","last_became_active":"2024-01-24T14:13:59.595830+0000","last_became_peered":"2024-01-24T14:13:59.595830+0000","last_unstale":"2024-01-24T14:13:59.599212+0000","last_undegraded":"2024-01-24T14:13:59.599212+0000","last_fullsized":"2024-01-24T14:13:59.599212+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591337+0000","last_change":"2024-01-24T14:13:59.190558+0000","last_active":"2024-01-24T14:13:59.591337+0000","last_peered":"2024-01-24T14:13:59.591337+0000","last_clean":"2024-01-24T14:13:59.591337+0000","last_became_active":"2024-01-24T14:13:59.187620+0000","last_became_peered":"2024-01-24T14:13:59.187620+0000","last_unstale":"2024-01-24T14:13:59.591337+0000","last_undegraded":"2024-01-24T14:13:59.591337+0000","last_fullsized":"2024-01-24T14:13:59.591337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616684+0000","last_change":"2024-01-24T14:13:59.609750+0000","last_active":"2024-01-24T14:13:59.616684+0000","last_peered":"2024-01-24T14:13:59.616684+0000","last_clean":"2024-01-24T14:13:59.616684+0000","last_became_active":"2024-01-24T14:13:59.609455+0000","last_became_peered":"2024-01-24T14:13:59.609455+0000","last_unstale":"2024-01-24T14:13:59.616684+0000","last_undegraded":"2024-01-24T14:13:59.616684+0000","last_fullsized":"2024-01-24T14:13:59.616684+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593252+0000","last_change":"2024-01-24T14:13:59.173164+0000","last_active":"2024-01-24T14:13:59.593252+0000","last_peered":"2024-01-24T14:13:59.593252+0000","last_clean":"2024-01-24T14:13:59.593252+0000","last_became_active":"2024-01-24T14:13:59.172816+0000","last_became_peered":"2024-01-24T14:13:59.172816+0000","last_unstale":"2024-01-24T14:13:59.593252+0000","last_undegraded":"2024-01-24T14:13:59.593252+0000","last_fullsized":"2024-01-24T14:13:59.593252+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612433+0000","last_change":"2024-01-24T14:13:59.603989+0000","last_active":"2024-01-24T14:13:59.612433+0000","last_peered":"2024-01-24T14:13:59.612433+0000","last_clean":"2024-01-24T14:13:59.612433+0000","last_became_active":"2024-01-24T14:13:59.603818+0000","last_became_peered":"2024-01-24T14:13:59.603818+0000","last_unstale":"2024-01-24T14:13:59.612433+0000","last_undegraded":"2024-01-24T14:13:59.612433+0000","last_fullsized":"2024-01-24T14:13:59.612433+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581206+0000","last_change":"2024-01-24T14:13:59.168769+0000","last_active":"2024-01-24T14:13:59.581206+0000","last_peered":"2024-01-24T14:13:59.581206+0000","last_clean":"2024-01-24T14:13:59.581206+0000","last_became_active":"2024-01-24T14:13:59.168470+0000","last_became_peered":"2024-01-24T14:13:59.168470+0000","last_unstale":"2024-01-24T14:13:59.581206+0000","last_undegraded":"2024-01-24T14:13:59.581206+0000","last_fullsized":"2024-01-24T14:13:59.581206+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.614739+0000","last_change":"2024-01-24T14:13:59.602824+0000","last_active":"2024-01-24T14:13:59.614739+0000","last_peered":"2024-01-24T14:13:59.614739+0000","last_clean":"2024-01-24T14:13:59.614739+0000","last_became_active":"2024-01-24T14:13:59.602330+0000","last_became_peered":"2024-01-24T14:13:59.602330+0000","last_unstale":"2024-01-24T14:13:59.614739+0000","last_undegraded":"2024-01-24T14:13:59.614739+0000","last_fullsized":"2024-01-24T14:13:59.614739+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612022+0000","last_change":"2024-01-24T14:13:59.603458+0000","last_active":"2024-01-24T14:13:59.612022+0000","last_peered":"2024-01-24T14:13:59.612022+0000","last_clean":"2024-01-24T14:13:59.612022+0000","last_became_active":"2024-01-24T14:13:59.600085+0000","last_became_peered":"2024-01-24T14:13:59.600085+0000","last_unstale":"2024-01-24T14:13:59.612022+0000","last_undegraded":"2024-01-24T14:13:59.612022+0000","last_fullsized":"2024-01-24T14:13:59.612022+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604880+0000","last_change":"2024-01-24T14:13:59.186304+0000","last_active":"2024-01-24T14:13:59.604880+0000","last_peered":"2024-01-24T14:13:59.604880+0000","last_clean":"2024-01-24T14:13:59.604880+0000","last_became_active":"2024-01-24T14:13:59.186017+0000","last_became_peered":"2024-01-24T14:13:59.186017+0000","last_unstale":"2024-01-24T14:13:59.604880+0000","last_undegraded":"2024-01-24T14:13:59.604880+0000","last_fullsized":"2024-01-24T14:13:59.604880+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613474+0000","last_change":"2024-01-24T14:13:59.603461+0000","last_active":"2024-01-24T14:13:59.613474+0000","last_peered":"2024-01-24T14:13:59.613474+0000","last_clean":"2024-01-24T14:13:59.613474+0000","last_became_active":"2024-01-24T14:13:59.603177+0000","last_became_peered":"2024-01-24T14:13:59.603177+0000","last_unstale":"2024-01-24T14:13:59.613474+0000","last_undegraded":"2024-01-24T14:13:59.613474+0000","last_fullsized":"2024-01-24T14:13:59.613474+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616811+0000","last_change":"2024-01-24T14:13:59.176815+0000","last_active":"2024-01-24T14:13:59.616811+0000","last_peered":"2024-01-24T14:13:59.616811+0000","last_clean":"2024-01-24T14:13:59.616811+0000","last_became_active":"2024-01-24T14:13:59.176478+0000","last_became_peered":"2024-01-24T14:13:59.176478+0000","last_unstale":"2024-01-24T14:13:59.616811+0000","last_undegraded":"2024-01-24T14:13:59.616811+0000","last_fullsized":"2024-01-24T14:13:59.616811+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612937+0000","last_change":"2024-01-24T14:13:59.603476+0000","last_active":"2024-01-24T14:13:59.612937+0000","last_peered":"2024-01-24T14:13:59.612937+0000","last_clean":"2024-01-24T14:13:59.612937+0000","last_became_active":"2024-01-24T14:13:59.603267+0000","last_became_peered":"2024-01-24T14:13:59.603267+0000","last_unstale":"2024-01-24T14:13:59.612937+0000","last_undegraded":"2024-01-24T14:13:59.612937+0000","last_fullsized":"2024-01-24T14:13:59.612937+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.583071+0000","last_change":"2024-01-24T14:13:59.174673+0000","last_active":"2024-01-24T14:13:59.583071+0000","last_peered":"2024-01-24T14:13:59.583071+0000","last_clean":"2024-01-24T14:13:59.583071+0000","last_became_active":"2024-01-24T14:13:59.174365+0000","last_became_peered":"2024-01-24T14:13:59.174365+0000","last_unstale":"2024-01-24T14:13:59.583071+0000","last_undegraded":"2024-01-24T14:13:59.583071+0000","last_fullsized":"2024-01-24T14:13:59.583071+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591250+0000","last_change":"2024-01-24T14:13:59.184862+0000","last_active":"2024-01-24T14:13:59.591250+0000","last_peered":"2024-01-24T14:13:59.591250+0000","last_clean":"2024-01-24T14:13:59.591250+0000","last_became_active":"2024-01-24T14:13:59.184560+0000","last_became_peered":"2024-01-24T14:13:59.184560+0000","last_unstale":"2024-01-24T14:13:59.591250+0000","last_undegraded":"2024-01-24T14:13:59.591250+0000","last_fullsized":"2024-01-24T14:13:59.591250+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599235+0000","last_change":"2024-01-24T14:13:59.597440+0000","last_active":"2024-01-24T14:13:59.599235+0000","last_peered":"2024-01-24T14:13:59.599235+0000","last_clean":"2024-01-24T14:13:59.599235+0000","last_became_active":"2024-01-24T14:13:59.597132+0000","last_became_peered":"2024-01-24T14:13:59.597132+0000","last_unstale":"2024-01-24T14:13:59.599235+0000","last_undegraded":"2024-01-24T14:13:59.599235+0000","last_fullsized":"2024-01-24T14:13:59.599235+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.582496+0000","last_change":"2024-01-24T14:13:59.185072+0000","last_active":"2024-01-24T14:13:59.582496+0000","last_peered":"2024-01-24T14:13:59.582496+0000","last_clean":"2024-01-24T14:13:59.582496+0000","last_became_active":"2024-01-24T14:13:59.184040+0000","last_became_peered":"2024-01-24T14:13:59.184040+0000","last_unstale":"2024-01-24T14:13:59.582496+0000","last_undegraded":"2024-01-24T14:13:59.582496+0000","last_fullsized":"2024-01-24T14:13:59.582496+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599036+0000","last_change":"2024-01-24T14:13:59.597766+0000","last_active":"2024-01-24T14:13:59.599036+0000","last_peered":"2024-01-24T14:13:59.599036+0000","last_clean":"2024-01-24T14:13:59.599036+0000","last_became_active":"2024-01-24T14:13:59.597476+0000","last_became_peered":"2024-01-24T14:13:59.597476+0000","last_unstale":"2024-01-24T14:13:59.599036+0000","last_undegraded":"2024-01-24T14:13:59.599036+0000","last_fullsized":"2024-01-24T14:13:59.599036+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602220+0000","last_change":"2024-01-24T14:13:59.598055+0000","last_active":"2024-01-24T14:13:59.602220+0000","last_peered":"2024-01-24T14:13:59.602220+0000","last_clean":"2024-01-24T14:13:59.602220+0000","last_became_active":"2024-01-24T14:13:59.597777+0000","last_became_peered":"2024-01-24T14:13:59.597777+0000","last_unstale":"2024-01-24T14:13:59.602220+0000","last_undegraded":"2024-01-24T14:13:59.602220+0000","last_fullsized":"2024-01-24T14:13:59.602220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581576+0000","last_change":"2024-01-24T14:13:59.185733+0000","last_active":"2024-01-24T14:13:59.581576+0000","last_peered":"2024-01-24T14:13:59.581576+0000","last_clean":"2024-01-24T14:13:59.581576+0000","last_became_active":"2024-01-24T14:13:59.185482+0000","last_became_peered":"2024-01-24T14:13:59.185482+0000","last_unstale":"2024-01-24T14:13:59.581576+0000","last_undegraded":"2024-01-24T14:13:59.581576+0000","last_fullsized":"2024-01-24T14:13:59.581576+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599233+0000","last_change":"2024-01-24T14:13:59.597585+0000","last_active":"2024-01-24T14:13:59.599233+0000","last_peered":"2024-01-24T14:13:59.599233+0000","last_clean":"2024-01-24T14:13:59.599233+0000","last_became_active":"2024-01-24T14:13:59.597114+0000","last_became_peered":"2024-01-24T14:13:59.597114+0000","last_unstale":"2024-01-24T14:13:59.599233+0000","last_undegraded":"2024-01-24T14:13:59.599233+0000","last_fullsized":"2024-01-24T14:13:59.599233+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601777+0000","last_change":"2024-01-24T14:13:59.600823+0000","last_active":"2024-01-24T14:13:59.601777+0000","last_peered":"2024-01-24T14:13:59.601777+0000","last_clean":"2024-01-24T14:13:59.601777+0000","last_became_active":"2024-01-24T14:13:59.600322+0000","last_became_peered":"2024-01-24T14:13:59.600322+0000","last_unstale":"2024-01-24T14:13:59.601777+0000","last_undegraded":"2024-01-24T14:13:59.601777+0000","last_fullsized":"2024-01-24T14:13:59.601777+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615401+0000","last_change":"2024-01-24T14:13:59.611324+0000","last_active":"2024-01-24T14:13:59.615401+0000","last_peered":"2024-01-24T14:13:59.615401+0000","last_clean":"2024-01-24T14:13:59.615401+0000","last_became_active":"2024-01-24T14:13:59.610992+0000","last_became_peered":"2024-01-24T14:13:59.610992+0000","last_unstale":"2024-01-24T14:13:59.615401+0000","last_undegraded":"2024-01-24T14:13:59.615401+0000","last_fullsized":"2024-01-24T14:13:59.615401+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596181+0000","last_change":"2024-01-24T14:13:59.176584+0000","last_active":"2024-01-24T14:13:59.596181+0000","last_peered":"2024-01-24T14:13:59.596181+0000","last_clean":"2024-01-24T14:13:59.596181+0000","last_became_active":"2024-01-24T14:13:59.176352+0000","last_became_peered":"2024-01-24T14:13:59.176352+0000","last_unstale":"2024-01-24T14:13:59.596181+0000","last_undegraded":"2024-01-24T14:13:59.596181+0000","last_fullsized":"2024-01-24T14:13:59.596181+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.582565+0000","last_change":"2024-01-24T14:13:59.177305+0000","last_active":"2024-01-24T14:13:59.582565+0000","last_peered":"2024-01-24T14:13:59.582565+0000","last_clean":"2024-01-24T14:13:59.582565+0000","last_became_active":"2024-01-24T14:13:59.177013+0000","last_became_peered":"2024-01-24T14:13:59.177013+0000","last_unstale":"2024-01-24T14:13:59.582565+0000","last_undegraded":"2024-01-24T14:13:59.582565+0000","last_fullsized":"2024-01-24T14:13:59.582565+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613859+0000","last_change":"2024-01-24T14:13:59.605686+0000","last_active":"2024-01-24T14:13:59.613859+0000","last_peered":"2024-01-24T14:13:59.613859+0000","last_clean":"2024-01-24T14:13:59.613859+0000","last_became_active":"2024-01-24T14:13:59.599468+0000","last_became_peered":"2024-01-24T14:13:59.599468+0000","last_unstale":"2024-01-24T14:13:59.613859+0000","last_undegraded":"2024-01-24T14:13:59.613859+0000","last_fullsized":"2024-01-24T14:13:59.613859+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613713+0000","last_change":"2024-01-24T14:13:59.604787+0000","last_active":"2024-01-24T14:13:59.613713+0000","last_peered":"2024-01-24T14:13:59.613713+0000","last_clean":"2024-01-24T14:13:59.613713+0000","last_became_active":"2024-01-24T14:13:59.604506+0000","last_became_peered":"2024-01-24T14:13:59.604506+0000","last_unstale":"2024-01-24T14:13:59.613713+0000","last_undegraded":"2024-01-24T14:13:59.613713+0000","last_fullsized":"2024-01-24T14:13:59.613713+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593610+0000","last_change":"2024-01-24T14:13:59.177542+0000","last_active":"2024-01-24T14:13:59.593610+0000","last_peered":"2024-01-24T14:13:59.593610+0000","last_clean":"2024-01-24T14:13:59.593610+0000","last_became_active":"2024-01-24T14:13:59.177246+0000","last_became_peered":"2024-01-24T14:13:59.177246+0000","last_unstale":"2024-01-24T14:13:59.593610+0000","last_undegraded":"2024-01-24T14:13:59.593610+0000","last_fullsized":"2024-01-24T14:13:59.593610+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591431+0000","last_change":"2024-01-24T14:13:59.187008+0000","last_active":"2024-01-24T14:13:59.591431+0000","last_peered":"2024-01-24T14:13:59.591431+0000","last_clean":"2024-01-24T14:13:59.591431+0000","last_became_active":"2024-01-24T14:13:59.186690+0000","last_became_peered":"2024-01-24T14:13:59.186690+0000","last_unstale":"2024-01-24T14:13:59.591431+0000","last_undegraded":"2024-01-24T14:13:59.591431+0000","last_fullsized":"2024-01-24T14:13:59.591431+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594464+0000","last_change":"2024-01-24T14:13:59.175676+0000","last_active":"2024-01-24T14:13:59.594464+0000","last_peered":"2024-01-24T14:13:59.594464+0000","last_clean":"2024-01-24T14:13:59.594464+0000","last_became_active":"2024-01-24T14:13:59.175396+0000","last_became_peered":"2024-01-24T14:13:59.175396+0000","last_unstale":"2024-01-24T14:13:59.594464+0000","last_undegraded":"2024-01-24T14:13:59.594464+0000","last_fullsized":"2024-01-24T14:13:59.594464+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613559+0000","last_change":"2024-01-24T14:13:59.604227+0000","last_active":"2024-01-24T14:13:59.613559+0000","last_peered":"2024-01-24T14:13:59.613559+0000","last_clean":"2024-01-24T14:13:59.613559+0000","last_became_active":"2024-01-24T14:13:59.603933+0000","last_became_peered":"2024-01-24T14:13:59.603933+0000","last_unstale":"2024-01-24T14:13:59.613559+0000","last_undegraded":"2024-01-24T14:13:59.613559+0000","last_fullsized":"2024-01-24T14:13:59.613559+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613839+0000","last_change":"2024-01-24T14:13:59.604215+0000","last_active":"2024-01-24T14:13:59.613839+0000","last_peered":"2024-01-24T14:13:59.613839+0000","last_clean":"2024-01-24T14:13:59.613839+0000","last_became_active":"2024-01-24T14:13:59.603918+0000","last_became_peered":"2024-01-24T14:13:59.603918+0000","last_unstale":"2024-01-24T14:13:59.613839+0000","last_undegraded":"2024-01-24T14:13:59.613839+0000","last_fullsized":"2024-01-24T14:13:59.613839+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616746+0000","last_change":"2024-01-24T14:13:59.610454+0000","last_active":"2024-01-24T14:13:59.616746+0000","last_peered":"2024-01-24T14:13:59.616746+0000","last_clean":"2024-01-24T14:13:59.616746+0000","last_became_active":"2024-01-24T14:13:59.610190+0000","last_became_peered":"2024-01-24T14:13:59.610190+0000","last_unstale":"2024-01-24T14:13:59.616746+0000","last_undegraded":"2024-01-24T14:13:59.616746+0000","last_fullsized":"2024-01-24T14:13:59.616746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591156+0000","last_change":"2024-01-24T14:13:59.184839+0000","last_active":"2024-01-24T14:13:59.591156+0000","last_peered":"2024-01-24T14:13:59.591156+0000","last_clean":"2024-01-24T14:13:59.591156+0000","last_became_active":"2024-01-24T14:13:59.184535+0000","last_became_peered":"2024-01-24T14:13:59.184535+0000","last_unstale":"2024-01-24T14:13:59.591156+0000","last_undegraded":"2024-01-24T14:13:59.591156+0000","last_fullsized":"2024-01-24T14:13:59.591156+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613963+0000","last_change":"2024-01-24T14:13:59.174910+0000","last_active":"2024-01-24T14:13:59.613963+0000","last_peered":"2024-01-24T14:13:59.613963+0000","last_clean":"2024-01-24T14:13:59.613963+0000","last_became_active":"2024-01-24T14:13:59.174603+0000","last_became_peered":"2024-01-24T14:13:59.174603+0000","last_unstale":"2024-01-24T14:13:59.613963+0000","last_undegraded":"2024-01-24T14:13:59.613963+0000","last_fullsized":"2024-01-24T14:13:59.613963+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.618795+0000","last_change":"2024-01-24T14:13:59.614234+0000","last_active":"2024-01-24T14:13:59.618795+0000","last_peered":"2024-01-24T14:13:59.618795+0000","last_clean":"2024-01-24T14:13:59.618795+0000","last_became_active":"2024-01-24T14:13:59.614018+0000","last_became_peered":"2024-01-24T14:13:59.614018+0000","last_unstale":"2024-01-24T14:13:59.618795+0000","last_undegraded":"2024-01-24T14:13:59.618795+0000","last_fullsized":"2024-01-24T14:13:59.618795+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613805+0000","last_change":"2024-01-24T14:13:59.605325+0000","last_active":"2024-01-24T14:13:59.613805+0000","last_peered":"2024-01-24T14:13:59.613805+0000","last_clean":"2024-01-24T14:13:59.613805+0000","last_became_active":"2024-01-24T14:13:59.605027+0000","last_became_peered":"2024-01-24T14:13:59.605027+0000","last_unstale":"2024-01-24T14:13:59.613805+0000","last_undegraded":"2024-01-24T14:13:59.613805+0000","last_fullsized":"2024-01-24T14:13:59.613805+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615224+0000","last_change":"2024-01-24T14:13:59.610385+0000","last_active":"2024-01-24T14:13:59.615224+0000","last_peered":"2024-01-24T14:13:59.615224+0000","last_clean":"2024-01-24T14:13:59.615224+0000","last_became_active":"2024-01-24T14:13:59.609240+0000","last_became_peered":"2024-01-24T14:13:59.609240+0000","last_unstale":"2024-01-24T14:13:59.615224+0000","last_undegraded":"2024-01-24T14:13:59.615224+0000","last_fullsized":"2024-01-24T14:13:59.615224+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601872+0000","last_change":"2024-01-24T14:13:59.178489+0000","last_active":"2024-01-24T14:13:59.601872+0000","last_peered":"2024-01-24T14:13:59.601872+0000","last_clean":"2024-01-24T14:13:59.601872+0000","last_became_active":"2024-01-24T14:13:59.178185+0000","last_became_peered":"2024-01-24T14:13:59.178185+0000","last_unstale":"2024-01-24T14:13:59.601872+0000","last_undegraded":"2024-01-24T14:13:59.601872+0000","last_fullsized":"2024-01-24T14:13:59.601872+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616653+0000","last_change":"2024-01-24T14:13:59.609527+0000","last_active":"2024-01-24T14:13:59.616653+0000","last_peered":"2024-01-24T14:13:59.616653+0000","last_clean":"2024-01-24T14:13:59.616653+0000","last_became_active":"2024-01-24T14:13:59.609180+0000","last_became_peered":"2024-01-24T14:13:59.609180+0000","last_unstale":"2024-01-24T14:13:59.616653+0000","last_undegraded":"2024-01-24T14:13:59.616653+0000","last_fullsized":"2024-01-24T14:13:59.616653+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591353+0000","last_change":"2024-01-24T14:13:59.187223+0000","last_active":"2024-01-24T14:13:59.591353+0000","last_peered":"2024-01-24T14:13:59.591353+0000","last_clean":"2024-01-24T14:13:59.591353+0000","last_became_active":"2024-01-24T14:13:59.186908+0000","last_became_peered":"2024-01-24T14:13:59.186908+0000","last_unstale":"2024-01-24T14:13:59.591353+0000","last_undegraded":"2024-01-24T14:13:59.591353+0000","last_fullsized":"2024-01-24T14:13:59.591353+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616523+0000","last_change":"2024-01-24T14:13:59.606110+0000","last_active":"2024-01-24T14:13:59.616523+0000","last_peered":"2024-01-24T14:13:59.616523+0000","last_clean":"2024-01-24T14:13:59.616523+0000","last_became_active":"2024-01-24T14:13:59.605783+0000","last_became_peered":"2024-01-24T14:13:59.605783+0000","last_unstale":"2024-01-24T14:13:59.616523+0000","last_undegraded":"2024-01-24T14:13:59.616523+0000","last_fullsized":"2024-01-24T14:13:59.616523+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581385+0000","last_change":"2024-01-24T14:13:59.173283+0000","last_active":"2024-01-24T14:13:59.581385+0000","last_peered":"2024-01-24T14:13:59.581385+0000","last_clean":"2024-01-24T14:13:59.581385+0000","last_became_active":"2024-01-24T14:13:59.172965+0000","last_became_peered":"2024-01-24T14:13:59.172965+0000","last_unstale":"2024-01-24T14:13:59.581385+0000","last_undegraded":"2024-01-24T14:13:59.581385+0000","last_fullsized":"2024-01-24T14:13:59.581385+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591632+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:13:59.591632+0000","last_peered":"2024-01-24T14:13:59.591632+0000","last_clean":"2024-01-24T14:13:59.591632+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:13:59.591632+0000","last_undegraded":"2024-01-24T14:13:59.591632+0000","last_fullsized":"2024-01-24T14:13:59.591632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":89,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-24T14:13:54.182751+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:13:54.182751+0000","last_peered":"2024-01-24T14:13:54.182751+0000","last_clean":"2024-01-24T14:13:54.182751+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:13:54.182751+0000","last_undegraded":"2024-01-24T14:13:54.182751+0000","last_fullsized":"2024-01-24T14:13:54.182751+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613485+0000","last_change":"2024-01-24T14:13:59.603439+0000","last_active":"2024-01-24T14:13:59.613485+0000","last_peered":"2024-01-24T14:13:59.613485+0000","last_clean":"2024-01-24T14:13:59.613485+0000","last_became_active":"2024-01-24T14:13:59.603116+0000","last_became_peered":"2024-01-24T14:13:59.603116+0000","last_unstale":"2024-01-24T14:13:59.613485+0000","last_undegraded":"2024-01-24T14:13:59.613485+0000","last_fullsized":"2024-01-24T14:13:59.613485+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581282+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:13:59.581282+0000","last_peered":"2024-01-24T14:13:59.581282+0000","last_clean":"2024-01-24T14:13:59.581282+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:13:59.581282+0000","last_undegraded":"2024-01-24T14:13:59.581282+0000","last_fullsized":"2024-01-24T14:13:59.581282+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613322+0000","last_change":"2024-01-24T14:13:59.601159+0000","last_active":"2024-01-24T14:13:59.613322+0000","last_peered":"2024-01-24T14:13:59.613322+0000","last_clean":"2024-01-24T14:13:59.613322+0000","last_became_active":"2024-01-24T14:13:59.600870+0000","last_became_peered":"2024-01-24T14:13:59.600870+0000","last_unstale":"2024-01-24T14:13:59.613322+0000","last_undegraded":"2024-01-24T14:13:59.613322+0000","last_fullsized":"2024-01-24T14:13:59.613322+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.595988+0000","last_change":"2024-01-24T14:13:59.187233+0000","last_active":"2024-01-24T14:13:59.595988+0000","last_peered":"2024-01-24T14:13:59.595988+0000","last_clean":"2024-01-24T14:13:59.595988+0000","last_became_active":"2024-01-24T14:13:59.186954+0000","last_became_peered":"2024-01-24T14:13:59.186954+0000","last_unstale":"2024-01-24T14:13:59.595988+0000","last_undegraded":"2024-01-24T14:13:59.595988+0000","last_fullsized":"2024-01-24T14:13:59.595988+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":89,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-24T14:13:54.182751+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:13:54.182751+0000","last_peered":"2024-01-24T14:13:54.182751+0000","last_clean":"2024-01-24T14:13:54.182751+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:13:54.182751+0000","last_undegraded":"2024-01-24T14:13:54.182751+0000","last_fullsized":"2024-01-24T14:13:54.182751+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594597+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:13:59.594597+0000","last_peered":"2024-01-24T14:13:59.594597+0000","last_clean":"2024-01-24T14:13:59.594597+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:13:59.594597+0000","last_undegraded":"2024-01-24T14:13:59.594597+0000","last_fullsized":"2024-01-24T14:13:59.594597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594511+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:13:59.594511+0000","last_peered":"2024-01-24T14:13:59.594511+0000","last_clean":"2024-01-24T14:13:59.594511+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:13:59.594511+0000","last_undegraded":"2024-01-24T14:13:59.594511+0000","last_fullsized":"2024-01-24T14:13:59.594511+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.608535+0000","last_change":"2024-01-24T14:13:59.600293+0000","last_active":"2024-01-24T14:13:59.608535+0000","last_peered":"2024-01-24T14:13:59.608535+0000","last_clean":"2024-01-24T14:13:59.608535+0000","last_became_active":"2024-01-24T14:13:59.599864+0000","last_became_peered":"2024-01-24T14:13:59.599864+0000","last_unstale":"2024-01-24T14:13:59.608535+0000","last_undegraded":"2024-01-24T14:13:59.608535+0000","last_fullsized":"2024-01-24T14:13:59.608535+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":87,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596080+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:13:59.596080+0000","last_peered":"2024-01-24T14:13:59.596080+0000","last_clean":"2024-01-24T14:13:59.596080+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:13:59.596080+0000","last_undegraded":"2024-01-24T14:13:59.596080+0000","last_fullsized":"2024-01-24T14:13:59.596080+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:59.150171+0000","last_change":"2024-01-24T14:13:59.150171+0000","last_active":"2024-01-24T14:13:59.150171+0000","last_peered":"2024-01-24T14:13:59.150171+0000","last_clean":"2024-01-24T14:13:59.150171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:59.150171+0000","last_undegraded":"2024-01-24T14:13:59.150171+0000","last_fullsized":"2024-01-24T14:13:59.150171+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:13:58.144841+0000","last_change":"2024-01-24T14:13:58.144841+0000","last_active":"2024-01-24T14:13:58.144841+0000","last_peered":"2024-01-24T14:13:58.144841+0000","last_clean":"2024-01-24T14:13:58.144841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:13:58.144841+0000","last_undegraded":"2024-01-24T14:13:58.144841+0000","last_fullsized":"2024-01-24T14:13:58.144841+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591436+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:13:59.591436+0000","last_peered":"2024-01-24T14:13:59.591436+0000","last_clean":"2024-01-24T14:13:59.591436+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:13:59.591436+0000","last_undegraded":"2024-01-24T14:13:59.591436+0000","last_fullsized":"2024-01-24T14:13:59.591436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593811+0000","last_change":"2024-01-24T14:13:59.587516+0000","last_active":"2024-01-24T14:13:59.593811+0000","last_peered":"2024-01-24T14:13:59.593811+0000","last_clean":"2024-01-24T14:13:59.593811+0000","last_became_active":"2024-01-24T14:13:59.587242+0000","last_became_peered":"2024-01-24T14:13:59.587242+0000","last_unstale":"2024-01-24T14:13:59.593811+0000","last_undegraded":"2024-01-24T14:13:59.593811+0000","last_fullsized":"2024-01-24T14:13:59.593811+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581221+0000","last_change":"2024-01-24T14:13:59.176122+0000","last_active":"2024-01-24T14:13:59.581221+0000","last_peered":"2024-01-24T14:13:59.581221+0000","last_clean":"2024-01-24T14:13:59.581221+0000","last_became_active":"2024-01-24T14:13:59.175794+0000","last_became_peered":"2024-01-24T14:13:59.175794+0000","last_unstale":"2024-01-24T14:13:59.581221+0000","last_undegraded":"2024-01-24T14:13:59.581221+0000","last_fullsized":"2024-01-24T14:13:59.581221+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":80,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-24T14:13:54.605774+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:13:54.605774+0000","last_peered":"2024-01-24T14:13:54.605774+0000","last_clean":"2024-01-24T14:13:54.605774+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:13:54.605774+0000","last_undegraded":"2024-01-24T14:13:54.605774+0000","last_fullsized":"2024-01-24T14:13:54.605774+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":56,"acting":56,"num_store_stats":3},{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":588,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":44,"acting":44,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575038,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1290240,"data_stored":710498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.26000000000000001},"max":{"1min":0.80600000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.32700000000000001},"max":{"1min":0.753,"5min":0.82599999999999996,"15min":0.83499999999999996},"last":0.58899999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.312},"max":{"1min":0.75600000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":3.2570000000000001},"last":0.53000000000000003}]},{"osd":2,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.51700000000000002,"5min":0.40400000000000003,"15min":0.32000000000000001},"max":{"1min":0.79100000000000004,"5min":0.872,"15min":0.876},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.39300000000000002,"15min":0.36799999999999999},"max":{"1min":0.80000000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66700000000000004}]},{"osd":3,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.36499999999999999},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.85099999999999998,"15min":3.0049999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48899999999999999},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.84699999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.60399999999999998,"5min":0.70599999999999996,"15min":0.83599999999999997},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.501,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.29999999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.80900000000000005},"last":0.79400000000000004}]},{"osd":6,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.621},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.80700000000000005,"15min":0.87},"last":0.72199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575038,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302400,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069440,"statfs":{"total":96636764160,"available":96327106560,"internally_reserved":0,"allocated":1236992,"data_stored":679077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.59499999999999997},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73699999999999999,"5min":0.85699999999999998,"15min":0.86899999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":0.84899999999999998,"15min":1.3859999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73499999999999999,"5min":0.77000000000000002,"15min":1.353},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.78700000000000003,"5min":1.032,"15min":1.032},"last":0.46999999999999997}]},{"osd":2,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60099999999999998,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.39100000000000001,"15min":0.34300000000000003},"max":{"1min":0.78600000000000003,"5min":0.88500000000000001,"15min":6.8129999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.4079999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.66900000000000004},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81699999999999995,"5min":0.82799999999999996,"15min":7.016},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.43099999999999999,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.77400000000000002,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.63800000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.504,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":1.165,"15min":1.165},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.314},"max":{"1min":0.65500000000000003,"5min":0.82299999999999995,"15min":6.8719999999999999},"last":0.624}]},{"osd":5,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":0.86699999999999999,"15min":6.976},"last":0.497},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80200000000000005,"5min":0.85699999999999998,"15min":7.8789999999999996},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.747,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":0.89600000000000002,"15min":6.9080000000000004},"last":0.66200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302456,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069384,"statfs":{"total":96636764160,"available":96327049216,"internally_reserved":0,"allocated":1294336,"data_stored":710068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.51900000000000002,"15min":0.443},"min":{"1min":0.311,"5min":0.245,"15min":0.23400000000000001},"max":{"1min":0.57899999999999996,"5min":15.298999999999999,"15min":15.298999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.55000000000000004,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.57499999999999996,"5min":14.654,"15min":14.654},"last":0.38}]},{"osd":2,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45900000000000002,"15min":0.45100000000000001},"min":{"1min":0.34899999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.59799999999999998,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.25800000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47099999999999997,"15min":0.45700000000000002},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88600000000000001,"5min":1.073,"15min":1.137},"last":0.441},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.45900000000000002,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.218,"15min":1.218},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":1.1040000000000001,"15min":0.83299999999999996},"min":{"1min":0.47999999999999998,"5min":0.45000000000000001,"15min":0.443},"max":{"1min":0.86799999999999999,"5min":38.808,"15min":38.808},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":1.077,"15min":0.81200000000000006},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.40500000000000003},"max":{"1min":0.81299999999999994,"5min":38.823,"15min":38.823},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.46000000000000002,"15min":0.43099999999999999},"max":{"1min":0.94899999999999995,"5min":1.0449999999999999,"15min":1.0509999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.40000000000000002},"max":{"1min":0.91300000000000003,"5min":1.1559999999999999,"15min":1.1879999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.52500000000000002,"5min":0.49399999999999999,"15min":0.45300000000000001},"max":{"1min":0.98499999999999999,"5min":1.105,"15min":1.155},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.54900000000000004,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":0.93500000000000005,"5min":1.123,"15min":1.131},"last":0.58499999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542335,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302396,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069444,"statfs":{"total":96636764160,"available":96327110656,"internally_reserved":0,"allocated":1232896,"data_stored":678116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39600000000000002},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.57499999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.379}]},{"osd":2,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44800000000000001,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.28199999999999997},"max":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.66700000000000004},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.28799999999999998},"max":{"1min":0.61399999999999999,"5min":0.754,"15min":0.79400000000000004},"last":0.51300000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.33000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54500000000000004,"5min":1.482,"15min":1.482},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.496},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":0.872,"15min":0.872},"last":0.503}]},{"osd":4,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81499999999999995,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.64700000000000002}]},{"osd":5,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.875,"5min":1.655,"15min":1.655},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.42999999999999999},"max":{"1min":0.77200000000000002,"5min":0.94799999999999995,"15min":1.0840000000000001},"last":0.68700000000000006}]},{"osd":6,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.875,"5min":0.90700000000000003,"15min":1.137},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":0.96499999999999997,"15min":1.117},"last":0.71999999999999997}]},{"osd":7,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47099999999999997},"max":{"1min":0.86899999999999999,"5min":0.96099999999999997,"15min":0.98199999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.49099999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.78500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575038,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306548,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94065292,"statfs":{"total":96636764160,"available":96322859008,"internally_reserved":0,"allocated":1290240,"data_stored":710014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.29499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.65800000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.24399999999999999},"max":{"1min":0.54500000000000004,"5min":0.65900000000000003,"15min":0.67100000000000004},"last":0.46500000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.32500000000000001,"5min":0.27800000000000002,"15min":0.27300000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.67200000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.255},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.55600000000000005}]},{"osd":3,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.45000000000000001},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64300000000000002,"5min":0.67700000000000005,"15min":0.71699999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.38900000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.72799999999999998},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.502,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85899999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.55400000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.92200000000000004,"5min":0.996,"15min":1.042},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.53400000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83599999999999997,"5min":1.036,"15min":1.0469999999999999},"last":0.622},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90000000000000002,"5min":1.107,"15min":1.107},"last":0.70199999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.82899999999999996,"5min":1.0569999999999999,"15min":1.1100000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.57299999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84699999999999998,"5min":1.141,"15min":1.141},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.52400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":0.96199999999999997,"15min":1.054},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.88900000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.81000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302400,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069440,"statfs":{"total":96636764160,"available":96327106560,"internally_reserved":0,"allocated":1236992,"data_stored":679077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45100000000000001,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.76400000000000001,"5min":7.593,"15min":7.593},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48299999999999998,"15min":0.437},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56200000000000006,"5min":8.4239999999999995,"15min":8.4239999999999995},"last":0.34699999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.69099999999999995},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.221,"15min":0.221},"max":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.68799999999999994},"last":0.371}]},{"osd":2,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.496},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.66500000000000004,"5min":0.76300000000000001,"15min":0.77200000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.78800000000000003},"last":0.44900000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64500000000000002,"15min":0.63500000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.495,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":0.81599999999999995,"5min":1.976,"15min":1.976},"last":0.53600000000000003}]},{"osd":5,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91500000000000004,"5min":1.53,"15min":1.53},"last":0.629},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.97999999999999998,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72799999999999998,"15min":0.72099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.72999999999999998},"min":{"1min":0.495,"5min":0.49199999999999999,"15min":0.45800000000000002},"max":{"1min":0.93500000000000005,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.61799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302400,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069440,"statfs":{"total":96636764160,"available":96327106560,"internally_reserved":0,"allocated":1236992,"data_stored":679077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78300000000000003,"5min":0.80100000000000005,"15min":0.88600000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.84399999999999997},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.83099999999999996,"15min":0.85299999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.755,"5min":0.79800000000000004,"15min":1.506},"last":0.52100000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.80200000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56399999999999995}]},{"osd":3,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72899999999999998,"5min":0.77800000000000002,"15min":0.80600000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.81000000000000005,"15min":0.86299999999999999},"last":0.626}]},{"osd":5,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.92000000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.496,"15min":0.5},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65700000000000003,"5min":0.72899999999999998,"15min":1.556},"last":0.61599999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.753,"5min":0.78500000000000003,"15min":0.86099999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76400000000000001,"5min":0.82099999999999995,"15min":1.0409999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.38500000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.88100000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.307,"15min":0.307},"max":{"1min":0.72699999999999998,"5min":0.81399999999999995,"15min":1.26},"last":0.69699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302400,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069440,"statfs":{"total":96636764160,"available":96327106560,"internally_reserved":0,"allocated":1236992,"data_stored":679077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64400000000000002,"5min":0.85199999999999998,"15min":1.101},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.309,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71199999999999997,"5min":0.82699999999999996,"15min":1.1439999999999999},"last":0.53700000000000003}]},{"osd":1,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56000000000000005,"15min":0.57699999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67200000000000004,"5min":0.84099999999999997,"15min":1.093},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69199999999999995,"5min":0.81299999999999994,"15min":1.1639999999999999},"last":0.436}]},{"osd":2,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.68400000000000005,"5min":0.84299999999999997,"15min":1.1100000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.35499999999999998},"max":{"1min":0.67400000000000004,"5min":0.83599999999999997,"15min":1.1220000000000001},"last":0.40600000000000003}]},{"osd":3,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.38500000000000001},"max":{"1min":0.72199999999999998,"5min":0.80500000000000005,"15min":1.139},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.496,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":1.0609999999999999},"last":0.61499999999999999}]},{"osd":4,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70199999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.505,"15min":0.503},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.90000000000000002},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.32900000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":1.1839999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77400000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.63800000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72999999999999998,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.5},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.33200000000000002},"max":{"1min":0.78200000000000003,"5min":0.95999999999999996,"15min":1.2030000000000001},"last":0.54500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:14:04.101 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:14:04.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.268+0000 7ff629b68700 1 Processor -- start 2024-01-24T14:14:04.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.269+0000 7ff629b68700 1 -- start start 2024-01-24T14:14:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.269+0000 7ff629b68700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff624150ba0 0x7ff624150f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.269+0000 7ff629b68700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6240c3c40 con 0x7ff6240c40d0 2024-01-24T14:14:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.269+0000 7ff629b68700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6240c2d90 con 0x7ff624066480 2024-01-24T14:14:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.269+0000 7ff629b68700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6240c3030 con 0x7ff624150ba0 2024-01-24T14:14:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.270+0000 7ff622ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff6240c40d0 0x7ff624065d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39592/0 (socket says 172.21.15.29:39592) 2024-01-24T14:14:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.270+0000 7ff622ffd700 1 -- 172.21.15.29:0/2098097550 learned_addr learned my addr 172.21.15.29:0/2098097550 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.270+0000 7ff6237fe700 1 --2- 172.21.15.29:0/2098097550 >> v2:172.21.15.163:3300/0 conn(0x7ff624150ba0 0x7ff624150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.270+0000 7ff6227fc700 1 -- 172.21.15.29:0/2098097550 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106028674 0 0) 0x7ff6240c3c40 con 0x7ff6240c40d0 2024-01-24T14:14:04.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.270+0000 7ff6227fc700 1 -- 172.21.15.29:0/2098097550 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff60c0033f0 con 0x7ff6240c40d0 2024-01-24T14:14:04.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.270+0000 7ff6237fe700 1 -- 172.21.15.29:0/2098097550 >> v1:172.21.15.163:6789/0 conn(0x7ff624066480 legacy=0x7ff624069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.270+0000 7ff6237fe700 1 -- 172.21.15.29:0/2098097550 >> v1:172.21.15.29:6789/0 conn(0x7ff6240c40d0 legacy=0x7ff624065d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.270+0000 7ff6237fe700 1 -- 172.21.15.29:0/2098097550 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6240c3310 con 0x7ff624150ba0 2024-01-24T14:14:04.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.270+0000 7ff6237fe700 1 --2- 172.21.15.29:0/2098097550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff624150ba0 0x7ff624150f80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7ff614009760 tx=0x7ff614009060).ready entity=mon.1 client_cookie=7b322c1fc916e7dd server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:04.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.271+0000 7ff6227fc700 1 -- 172.21.15.29:0/2098097550 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61400f510 con 0x7ff624150ba0 2024-01-24T14:14:04.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.271+0000 7ff6227fc700 1 -- 172.21.15.29:0/2098097550 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61400f670 con 0x7ff624150ba0 2024-01-24T14:14:04.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.271+0000 7ff6227fc700 1 -- 172.21.15.29:0/2098097550 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61400f9f0 con 0x7ff624150ba0 2024-01-24T14:14:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.271+0000 7ff629b68700 1 -- 172.21.15.29:0/2098097550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff624150ba0 msgr2=0x7ff624150f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.271+0000 7ff629b68700 1 --2- 172.21.15.29:0/2098097550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff624150ba0 0x7ff624150f80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7ff614009760 tx=0x7ff614009060).stop 2024-01-24T14:14:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.271+0000 7ff629b68700 1 -- 172.21.15.29:0/2098097550 shutdown_connections 2024-01-24T14:14:04.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.271+0000 7ff629b68700 1 --2- 172.21.15.29:0/2098097550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff624150ba0 0x7ff624150f80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.271+0000 7ff629b68700 1 -- 172.21.15.29:0/2098097550 >> 172.21.15.29:0/2098097550 conn(0x7ff6240c85c0 msgr2=0x7ff6240c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.271+0000 7ff629b68700 1 -- 172.21.15.29:0/2098097550 shutdown_connections 2024-01-24T14:14:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.271+0000 7ff629b68700 1 -- 172.21.15.29:0/2098097550 wait complete. 2024-01-24T14:14:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.273+0000 7ff629b68700 1 Processor -- start 2024-01-24T14:14:04.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.273+0000 7ff629b68700 1 -- start start 2024-01-24T14:14:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.273+0000 7ff629b68700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff624066480 0x7ff624139c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.273+0000 7ff629b68700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6240c40d0 0x7ff62414cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.273+0000 7ff629b68700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff624150ba0 0x7ff6240627b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.273+0000 7ff629b68700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624148eb0 con 0x7ff624150ba0 2024-01-24T14:14:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.273+0000 7ff629b68700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624149190 con 0x7ff624066480 2024-01-24T14:14:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.273+0000 7ff629b68700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff624149470 con 0x7ff6240c40d0 2024-01-24T14:14:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff623fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff624150ba0 0x7ff6240627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff623fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff624150ba0 0x7ff6240627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59952/0 (socket says 172.21.15.29:59952) 2024-01-24T14:14:04.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff623fff700 1 -- 172.21.15.29:0/1617375574 learned_addr learned my addr 172.21.15.29:0/1617375574 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff6237fe700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff624066480 0x7ff624139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff622ffd700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6240c40d0 0x7ff62414cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff623fff700 1 -- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6240c40d0 msgr2=0x7ff62414cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff623fff700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6240c40d0 0x7ff62414cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff623fff700 1 -- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff624066480 msgr2=0x7ff624139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff623fff700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff624066480 0x7ff624139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff623fff700 1 -- 172.21.15.29:0/1617375574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6140092a0 con 0x7ff624150ba0 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff623fff700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff624150ba0 0x7ff6240627b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7ff61c002230 tx=0x7ff61c0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff60b7fe700 1 -- 172.21.15.29:0/1617375574 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c010510 con 0x7ff624150ba0 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff60b7fe700 1 -- 172.21.15.29:0/1617375574 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61c010670 con 0x7ff624150ba0 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff629b68700 1 -- 172.21.15.29:0/1617375574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff624149750 con 0x7ff624150ba0 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff60b7fe700 1 -- 172.21.15.29:0/1617375574 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c0109f0 con 0x7ff624150ba0 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.274+0000 7ff629b68700 1 -- 172.21.15.29:0/1617375574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62413a150 con 0x7ff624150ba0 2024-01-24T14:14:04.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.275+0000 7ff629b68700 1 -- 172.21.15.29:0/1617375574 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62414d9d0 con 0x7ff624150ba0 2024-01-24T14:14:04.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.276+0000 7ff60b7fe700 1 -- 172.21.15.29:0/1617375574 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff61c01b400 con 0x7ff624150ba0 2024-01-24T14:14:04.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.276+0000 7ff60b7fe700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff61005f270 0x7ff610061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:04.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.276+0000 7ff60b7fe700 1 -- 172.21.15.29:0/1617375574 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7ff61c07a5c0 con 0x7ff624150ba0 2024-01-24T14:14:04.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.277+0000 7ff6237fe700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff61005f270 0x7ff610061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:04.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.277+0000 7ff6237fe700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff61005f270 0x7ff610061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ff61400f640 tx=0x7ff614017000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:04.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.281+0000 7ff60b7fe700 1 -- 172.21.15.29:0/1617375574 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff62414d9d0 con 0x7ff624150ba0 2024-01-24T14:14:04.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.455+0000 7ff629b68700 1 -- 172.21.15.29:0/1617375574 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff62414d9d0 con 0x7ff61005f270 2024-01-24T14:14:04.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.466+0000 7ff60b7fe700 1 -- 172.21.15.29:0/1617375574 <== mgr.5864 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361939 (secure 0 0 0) 0x7ff62414d9d0 con 0x7ff61005f270 2024-01-24T14:14:04.471 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.469+0000 7ff629b68700 1 -- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff61005f270 msgr2=0x7ff610061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.469+0000 7ff629b68700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff61005f270 0x7ff610061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ff61400f640 tx=0x7ff614017000).stop 2024-01-24T14:14:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.469+0000 7ff629b68700 1 -- 172.21.15.29:0/1617375574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff624150ba0 msgr2=0x7ff6240627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.469+0000 7ff629b68700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff624150ba0 0x7ff6240627b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7ff61c002230 tx=0x7ff61c0015e0).stop 2024-01-24T14:14:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.469+0000 7ff629b68700 1 -- 172.21.15.29:0/1617375574 shutdown_connections 2024-01-24T14:14:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.469+0000 7ff629b68700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff61005f270 0x7ff610061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.469+0000 7ff629b68700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff624150ba0 0x7ff6240627b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.469+0000 7ff629b68700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff624066480 0x7ff624139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.469+0000 7ff629b68700 1 --2- 172.21.15.29:0/1617375574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6240c40d0 0x7ff62414cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.469+0000 7ff629b68700 1 -- 172.21.15.29:0/1617375574 >> 172.21.15.29:0/1617375574 conn(0x7ff6240c85c0 msgr2=0x7ff6240c5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.469+0000 7ff629b68700 1 -- 172.21.15.29:0/1617375574 shutdown_connections 2024-01-24T14:14:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.469+0000 7ff629b68700 1 -- 172.21.15.29:0/1617375574 wait complete. 2024-01-24T14:14:04.474 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:14:04.483 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-01-24T14:14:04.107428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":228,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2423740,"kb_used_data":10108,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752550980,"statfs":{"total":773094113280,"available":770612203520,"internally_reserved":0,"allocated":10350592,"data_stored":5683203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11128,"num_objects":-9,"num_object_clones":0,"num_object_copies":-18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-36,"num_write_kb":-28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000729"},"pg_stats":[{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615363+0000","last_change":"2024-01-24T14:13:59.610904+0000","last_active":"2024-01-24T14:13:59.615363+0000","last_peered":"2024-01-24T14:13:59.615363+0000","last_clean":"2024-01-24T14:13:59.615363+0000","last_became_active":"2024-01-24T14:13:59.610672+0000","last_became_peered":"2024-01-24T14:13:59.610672+0000","last_unstale":"2024-01-24T14:13:59.615363+0000","last_undegraded":"2024-01-24T14:13:59.615363+0000","last_fullsized":"2024-01-24T14:13:59.615363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3f","version":"104'3","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616800+0000","last_change":"2024-01-24T14:13:59.174978+0000","last_active":"2024-01-24T14:13:59.616800+0000","last_peered":"2024-01-24T14:13:59.616800+0000","last_clean":"2024-01-24T14:13:59.616800+0000","last_became_active":"2024-01-24T14:13:59.174684+0000","last_became_peered":"2024-01-24T14:13:59.174684+0000","last_unstale":"2024-01-24T14:13:59.616800+0000","last_undegraded":"2024-01-24T14:13:59.616800+0000","last_fullsized":"2024-01-24T14:13:59.616800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605090+0000","last_change":"2024-01-24T14:13:59.600282+0000","last_active":"2024-01-24T14:13:59.605090+0000","last_peered":"2024-01-24T14:13:59.605090+0000","last_clean":"2024-01-24T14:13:59.605090+0000","last_became_active":"2024-01-24T14:13:59.600004+0000","last_became_peered":"2024-01-24T14:13:59.600004+0000","last_unstale":"2024-01-24T14:13:59.605090+0000","last_undegraded":"2024-01-24T14:13:59.605090+0000","last_fullsized":"2024-01-24T14:13:59.605090+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.600650+0000","last_change":"2024-01-24T14:13:59.171713+0000","last_active":"2024-01-24T14:13:59.600650+0000","last_peered":"2024-01-24T14:13:59.600650+0000","last_clean":"2024-01-24T14:13:59.600650+0000","last_became_active":"2024-01-24T14:13:59.171407+0000","last_became_peered":"2024-01-24T14:13:59.171407+0000","last_unstale":"2024-01-24T14:13:59.600650+0000","last_undegraded":"2024-01-24T14:13:59.600650+0000","last_fullsized":"2024-01-24T14:13:59.600650+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599212+0000","last_change":"2024-01-24T14:13:59.596129+0000","last_active":"2024-01-24T14:13:59.599212+0000","last_peered":"2024-01-24T14:13:59.599212+0000","last_clean":"2024-01-24T14:13:59.599212+0000","last_became_active":"2024-01-24T14:13:59.595830+0000","last_became_peered":"2024-01-24T14:13:59.595830+0000","last_unstale":"2024-01-24T14:13:59.599212+0000","last_undegraded":"2024-01-24T14:13:59.599212+0000","last_fullsized":"2024-01-24T14:13:59.599212+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591337+0000","last_change":"2024-01-24T14:13:59.190558+0000","last_active":"2024-01-24T14:13:59.591337+0000","last_peered":"2024-01-24T14:13:59.591337+0000","last_clean":"2024-01-24T14:13:59.591337+0000","last_became_active":"2024-01-24T14:13:59.187620+0000","last_became_peered":"2024-01-24T14:13:59.187620+0000","last_unstale":"2024-01-24T14:13:59.591337+0000","last_undegraded":"2024-01-24T14:13:59.591337+0000","last_fullsized":"2024-01-24T14:13:59.591337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616684+0000","last_change":"2024-01-24T14:13:59.609750+0000","last_active":"2024-01-24T14:13:59.616684+0000","last_peered":"2024-01-24T14:13:59.616684+0000","last_clean":"2024-01-24T14:13:59.616684+0000","last_became_active":"2024-01-24T14:13:59.609455+0000","last_became_peered":"2024-01-24T14:13:59.609455+0000","last_unstale":"2024-01-24T14:13:59.616684+0000","last_undegraded":"2024-01-24T14:13:59.616684+0000","last_fullsized":"2024-01-24T14:13:59.616684+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3c","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599540+0000","last_change":"2024-01-24T14:13:59.184784+0000","last_active":"2024-01-24T14:13:59.599540+0000","last_peered":"2024-01-24T14:13:59.599540+0000","last_clean":"2024-01-24T14:13:59.599540+0000","last_became_active":"2024-01-24T14:13:59.184482+0000","last_became_peered":"2024-01-24T14:13:59.184482+0000","last_unstale":"2024-01-24T14:13:59.599540+0000","last_undegraded":"2024-01-24T14:13:59.599540+0000","last_fullsized":"2024-01-24T14:13:59.599540+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602293+0000","last_change":"2024-01-24T14:13:59.596092+0000","last_active":"2024-01-24T14:13:59.602293+0000","last_peered":"2024-01-24T14:13:59.602293+0000","last_clean":"2024-01-24T14:13:59.602293+0000","last_became_active":"2024-01-24T14:13:59.595791+0000","last_became_peered":"2024-01-24T14:13:59.595791+0000","last_unstale":"2024-01-24T14:13:59.602293+0000","last_undegraded":"2024-01-24T14:13:59.602293+0000","last_fullsized":"2024-01-24T14:13:59.602293+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593252+0000","last_change":"2024-01-24T14:13:59.173164+0000","last_active":"2024-01-24T14:13:59.593252+0000","last_peered":"2024-01-24T14:13:59.593252+0000","last_clean":"2024-01-24T14:13:59.593252+0000","last_became_active":"2024-01-24T14:13:59.172816+0000","last_became_peered":"2024-01-24T14:13:59.172816+0000","last_unstale":"2024-01-24T14:13:59.593252+0000","last_undegraded":"2024-01-24T14:13:59.593252+0000","last_fullsized":"2024-01-24T14:13:59.593252+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602472+0000","last_change":"2024-01-24T14:13:59.599516+0000","last_active":"2024-01-24T14:13:59.602472+0000","last_peered":"2024-01-24T14:13:59.602472+0000","last_clean":"2024-01-24T14:13:59.602472+0000","last_became_active":"2024-01-24T14:13:59.599234+0000","last_became_peered":"2024-01-24T14:13:59.599234+0000","last_unstale":"2024-01-24T14:13:59.602472+0000","last_undegraded":"2024-01-24T14:13:59.602472+0000","last_fullsized":"2024-01-24T14:13:59.602472+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.3a","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.611338+0000","last_change":"2024-01-24T14:13:59.180992+0000","last_active":"2024-01-24T14:13:59.611338+0000","last_peered":"2024-01-24T14:13:59.611338+0000","last_clean":"2024-01-24T14:13:59.611338+0000","last_became_active":"2024-01-24T14:13:59.180627+0000","last_became_peered":"2024-01-24T14:13:59.180627+0000","last_unstale":"2024-01-24T14:13:59.611338+0000","last_undegraded":"2024-01-24T14:13:59.611338+0000","last_fullsized":"2024-01-24T14:13:59.611338+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602256+0000","last_change":"2024-01-24T14:13:59.596465+0000","last_active":"2024-01-24T14:13:59.602256+0000","last_peered":"2024-01-24T14:13:59.602256+0000","last_clean":"2024-01-24T14:13:59.602256+0000","last_became_active":"2024-01-24T14:13:59.596185+0000","last_became_peered":"2024-01-24T14:13:59.596185+0000","last_unstale":"2024-01-24T14:13:59.602256+0000","last_undegraded":"2024-01-24T14:13:59.602256+0000","last_fullsized":"2024-01-24T14:13:59.602256+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.590664+0000","last_change":"2024-01-24T14:13:59.179164+0000","last_active":"2024-01-24T14:13:59.590664+0000","last_peered":"2024-01-24T14:13:59.590664+0000","last_clean":"2024-01-24T14:13:59.590664+0000","last_became_active":"2024-01-24T14:13:59.178543+0000","last_became_peered":"2024-01-24T14:13:59.178543+0000","last_unstale":"2024-01-24T14:13:59.590664+0000","last_undegraded":"2024-01-24T14:13:59.590664+0000","last_fullsized":"2024-01-24T14:13:59.590664+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612433+0000","last_change":"2024-01-24T14:13:59.603989+0000","last_active":"2024-01-24T14:13:59.612433+0000","last_peered":"2024-01-24T14:13:59.612433+0000","last_clean":"2024-01-24T14:13:59.612433+0000","last_became_active":"2024-01-24T14:13:59.603818+0000","last_became_peered":"2024-01-24T14:13:59.603818+0000","last_unstale":"2024-01-24T14:13:59.612433+0000","last_undegraded":"2024-01-24T14:13:59.612433+0000","last_fullsized":"2024-01-24T14:13:59.612433+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581206+0000","last_change":"2024-01-24T14:13:59.168769+0000","last_active":"2024-01-24T14:13:59.581206+0000","last_peered":"2024-01-24T14:13:59.581206+0000","last_clean":"2024-01-24T14:13:59.581206+0000","last_became_active":"2024-01-24T14:13:59.168470+0000","last_became_peered":"2024-01-24T14:13:59.168470+0000","last_unstale":"2024-01-24T14:13:59.581206+0000","last_undegraded":"2024-01-24T14:13:59.581206+0000","last_fullsized":"2024-01-24T14:13:59.581206+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613168+0000","last_change":"2024-01-24T14:13:59.605311+0000","last_active":"2024-01-24T14:13:59.613168+0000","last_peered":"2024-01-24T14:13:59.613168+0000","last_clean":"2024-01-24T14:13:59.613168+0000","last_became_active":"2024-01-24T14:13:59.604996+0000","last_became_peered":"2024-01-24T14:13:59.604996+0000","last_unstale":"2024-01-24T14:13:59.613168+0000","last_undegraded":"2024-01-24T14:13:59.613168+0000","last_fullsized":"2024-01-24T14:13:59.613168+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588758+0000","last_change":"2024-01-24T14:13:59.173631+0000","last_active":"2024-01-24T14:13:59.588758+0000","last_peered":"2024-01-24T14:13:59.588758+0000","last_clean":"2024-01-24T14:13:59.588758+0000","last_became_active":"2024-01-24T14:13:59.173365+0000","last_became_peered":"2024-01-24T14:13:59.173365+0000","last_unstale":"2024-01-24T14:13:59.588758+0000","last_undegraded":"2024-01-24T14:13:59.588758+0000","last_fullsized":"2024-01-24T14:13:59.588758+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602306+0000","last_change":"2024-01-24T14:13:59.597783+0000","last_active":"2024-01-24T14:13:59.602306+0000","last_peered":"2024-01-24T14:13:59.602306+0000","last_clean":"2024-01-24T14:13:59.602306+0000","last_became_active":"2024-01-24T14:13:59.597507+0000","last_became_peered":"2024-01-24T14:13:59.597507+0000","last_unstale":"2024-01-24T14:13:59.602306+0000","last_undegraded":"2024-01-24T14:13:59.602306+0000","last_fullsized":"2024-01-24T14:13:59.602306+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585624+0000","last_change":"2024-01-24T14:13:59.185609+0000","last_active":"2024-01-24T14:13:59.585624+0000","last_peered":"2024-01-24T14:13:59.585624+0000","last_clean":"2024-01-24T14:13:59.585624+0000","last_became_active":"2024-01-24T14:13:59.185272+0000","last_became_peered":"2024-01-24T14:13:59.185272+0000","last_unstale":"2024-01-24T14:13:59.585624+0000","last_undegraded":"2024-01-24T14:13:59.585624+0000","last_fullsized":"2024-01-24T14:13:59.585624+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613203+0000","last_change":"2024-01-24T14:13:59.607137+0000","last_active":"2024-01-24T14:13:59.613203+0000","last_peered":"2024-01-24T14:13:59.613203+0000","last_clean":"2024-01-24T14:13:59.613203+0000","last_became_active":"2024-01-24T14:13:59.606837+0000","last_became_peered":"2024-01-24T14:13:59.606837+0000","last_unstale":"2024-01-24T14:13:59.613203+0000","last_undegraded":"2024-01-24T14:13:59.613203+0000","last_fullsized":"2024-01-24T14:13:59.613203+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597239+0000","last_change":"2024-01-24T14:13:59.174329+0000","last_active":"2024-01-24T14:13:59.597239+0000","last_peered":"2024-01-24T14:13:59.597239+0000","last_clean":"2024-01-24T14:13:59.597239+0000","last_became_active":"2024-01-24T14:13:59.173356+0000","last_became_peered":"2024-01-24T14:13:59.173356+0000","last_unstale":"2024-01-24T14:13:59.597239+0000","last_undegraded":"2024-01-24T14:13:59.597239+0000","last_fullsized":"2024-01-24T14:13:59.597239+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.614739+0000","last_change":"2024-01-24T14:13:59.602824+0000","last_active":"2024-01-24T14:13:59.614739+0000","last_peered":"2024-01-24T14:13:59.614739+0000","last_clean":"2024-01-24T14:13:59.614739+0000","last_became_active":"2024-01-24T14:13:59.602330+0000","last_became_peered":"2024-01-24T14:13:59.602330+0000","last_unstale":"2024-01-24T14:13:59.614739+0000","last_undegraded":"2024-01-24T14:13:59.614739+0000","last_fullsized":"2024-01-24T14:13:59.614739+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616384+0000","last_change":"2024-01-24T14:13:59.191263+0000","last_active":"2024-01-24T14:13:59.616384+0000","last_peered":"2024-01-24T14:13:59.616384+0000","last_clean":"2024-01-24T14:13:59.616384+0000","last_became_active":"2024-01-24T14:13:59.191014+0000","last_became_peered":"2024-01-24T14:13:59.191014+0000","last_unstale":"2024-01-24T14:13:59.616384+0000","last_undegraded":"2024-01-24T14:13:59.616384+0000","last_fullsized":"2024-01-24T14:13:59.616384+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612022+0000","last_change":"2024-01-24T14:13:59.603458+0000","last_active":"2024-01-24T14:13:59.612022+0000","last_peered":"2024-01-24T14:13:59.612022+0000","last_clean":"2024-01-24T14:13:59.612022+0000","last_became_active":"2024-01-24T14:13:59.600085+0000","last_became_peered":"2024-01-24T14:13:59.600085+0000","last_unstale":"2024-01-24T14:13:59.612022+0000","last_undegraded":"2024-01-24T14:13:59.612022+0000","last_fullsized":"2024-01-24T14:13:59.612022+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604880+0000","last_change":"2024-01-24T14:13:59.186304+0000","last_active":"2024-01-24T14:13:59.604880+0000","last_peered":"2024-01-24T14:13:59.604880+0000","last_clean":"2024-01-24T14:13:59.604880+0000","last_became_active":"2024-01-24T14:13:59.186017+0000","last_became_peered":"2024-01-24T14:13:59.186017+0000","last_unstale":"2024-01-24T14:13:59.604880+0000","last_undegraded":"2024-01-24T14:13:59.604880+0000","last_fullsized":"2024-01-24T14:13:59.604880+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613474+0000","last_change":"2024-01-24T14:13:59.603461+0000","last_active":"2024-01-24T14:13:59.613474+0000","last_peered":"2024-01-24T14:13:59.613474+0000","last_clean":"2024-01-24T14:13:59.613474+0000","last_became_active":"2024-01-24T14:13:59.603177+0000","last_became_peered":"2024-01-24T14:13:59.603177+0000","last_unstale":"2024-01-24T14:13:59.613474+0000","last_undegraded":"2024-01-24T14:13:59.613474+0000","last_fullsized":"2024-01-24T14:13:59.613474+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585657+0000","last_change":"2024-01-24T14:13:59.181115+0000","last_active":"2024-01-24T14:13:59.585657+0000","last_peered":"2024-01-24T14:13:59.585657+0000","last_clean":"2024-01-24T14:13:59.585657+0000","last_became_active":"2024-01-24T14:13:59.180641+0000","last_became_peered":"2024-01-24T14:13:59.180641+0000","last_unstale":"2024-01-24T14:13:59.585657+0000","last_undegraded":"2024-01-24T14:13:59.585657+0000","last_fullsized":"2024-01-24T14:13:59.585657+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.606266+0000","last_change":"2024-01-24T14:13:59.602768+0000","last_active":"2024-01-24T14:13:59.606266+0000","last_peered":"2024-01-24T14:13:59.606266+0000","last_clean":"2024-01-24T14:13:59.606266+0000","last_became_active":"2024-01-24T14:13:59.602510+0000","last_became_peered":"2024-01-24T14:13:59.602510+0000","last_unstale":"2024-01-24T14:13:59.606266+0000","last_undegraded":"2024-01-24T14:13:59.606266+0000","last_fullsized":"2024-01-24T14:13:59.606266+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596930+0000","last_change":"2024-01-24T14:13:59.188810+0000","last_active":"2024-01-24T14:13:59.596930+0000","last_peered":"2024-01-24T14:13:59.596930+0000","last_clean":"2024-01-24T14:13:59.596930+0000","last_became_active":"2024-01-24T14:13:59.188475+0000","last_became_peered":"2024-01-24T14:13:59.188475+0000","last_unstale":"2024-01-24T14:13:59.596930+0000","last_undegraded":"2024-01-24T14:13:59.596930+0000","last_fullsized":"2024-01-24T14:13:59.596930+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.608903+0000","last_change":"2024-01-24T14:13:59.604713+0000","last_active":"2024-01-24T14:13:59.608903+0000","last_peered":"2024-01-24T14:13:59.608903+0000","last_clean":"2024-01-24T14:13:59.608903+0000","last_became_active":"2024-01-24T14:13:59.604382+0000","last_became_peered":"2024-01-24T14:13:59.604382+0000","last_unstale":"2024-01-24T14:13:59.608903+0000","last_undegraded":"2024-01-24T14:13:59.608903+0000","last_fullsized":"2024-01-24T14:13:59.608903+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.30","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616811+0000","last_change":"2024-01-24T14:13:59.176815+0000","last_active":"2024-01-24T14:13:59.616811+0000","last_peered":"2024-01-24T14:13:59.616811+0000","last_clean":"2024-01-24T14:13:59.616811+0000","last_became_active":"2024-01-24T14:13:59.176478+0000","last_became_peered":"2024-01-24T14:13:59.176478+0000","last_unstale":"2024-01-24T14:13:59.616811+0000","last_undegraded":"2024-01-24T14:13:59.616811+0000","last_fullsized":"2024-01-24T14:13:59.616811+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612937+0000","last_change":"2024-01-24T14:13:59.603476+0000","last_active":"2024-01-24T14:13:59.612937+0000","last_peered":"2024-01-24T14:13:59.612937+0000","last_clean":"2024-01-24T14:13:59.612937+0000","last_became_active":"2024-01-24T14:13:59.603267+0000","last_became_peered":"2024-01-24T14:13:59.603267+0000","last_unstale":"2024-01-24T14:13:59.612937+0000","last_undegraded":"2024-01-24T14:13:59.612937+0000","last_fullsized":"2024-01-24T14:13:59.612937+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.583071+0000","last_change":"2024-01-24T14:13:59.174673+0000","last_active":"2024-01-24T14:13:59.583071+0000","last_peered":"2024-01-24T14:13:59.583071+0000","last_clean":"2024-01-24T14:13:59.583071+0000","last_became_active":"2024-01-24T14:13:59.174365+0000","last_became_peered":"2024-01-24T14:13:59.174365+0000","last_unstale":"2024-01-24T14:13:59.583071+0000","last_undegraded":"2024-01-24T14:13:59.583071+0000","last_fullsized":"2024-01-24T14:13:59.583071+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.603045+0000","last_change":"2024-01-24T14:13:59.598989+0000","last_active":"2024-01-24T14:13:59.603045+0000","last_peered":"2024-01-24T14:13:59.603045+0000","last_clean":"2024-01-24T14:13:59.603045+0000","last_became_active":"2024-01-24T14:13:59.598687+0000","last_became_peered":"2024-01-24T14:13:59.598687+0000","last_unstale":"2024-01-24T14:13:59.603045+0000","last_undegraded":"2024-01-24T14:13:59.603045+0000","last_fullsized":"2024-01-24T14:13:59.603045+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591250+0000","last_change":"2024-01-24T14:13:59.184862+0000","last_active":"2024-01-24T14:13:59.591250+0000","last_peered":"2024-01-24T14:13:59.591250+0000","last_clean":"2024-01-24T14:13:59.591250+0000","last_became_active":"2024-01-24T14:13:59.184560+0000","last_became_peered":"2024-01-24T14:13:59.184560+0000","last_unstale":"2024-01-24T14:13:59.591250+0000","last_undegraded":"2024-01-24T14:13:59.591250+0000","last_fullsized":"2024-01-24T14:13:59.591250+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599235+0000","last_change":"2024-01-24T14:13:59.597440+0000","last_active":"2024-01-24T14:13:59.599235+0000","last_peered":"2024-01-24T14:13:59.599235+0000","last_clean":"2024-01-24T14:13:59.599235+0000","last_became_active":"2024-01-24T14:13:59.597132+0000","last_became_peered":"2024-01-24T14:13:59.597132+0000","last_unstale":"2024-01-24T14:13:59.599235+0000","last_undegraded":"2024-01-24T14:13:59.599235+0000","last_fullsized":"2024-01-24T14:13:59.599235+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2d","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612688+0000","last_change":"2024-01-24T14:13:59.177820+0000","last_active":"2024-01-24T14:13:59.612688+0000","last_peered":"2024-01-24T14:13:59.612688+0000","last_clean":"2024-01-24T14:13:59.612688+0000","last_became_active":"2024-01-24T14:13:59.177501+0000","last_became_peered":"2024-01-24T14:13:59.177501+0000","last_unstale":"2024-01-24T14:13:59.612688+0000","last_undegraded":"2024-01-24T14:13:59.612688+0000","last_fullsized":"2024-01-24T14:13:59.612688+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613916+0000","last_change":"2024-01-24T14:13:59.607979+0000","last_active":"2024-01-24T14:13:59.613916+0000","last_peered":"2024-01-24T14:13:59.613916+0000","last_clean":"2024-01-24T14:13:59.613916+0000","last_became_active":"2024-01-24T14:13:59.607684+0000","last_became_peered":"2024-01-24T14:13:59.607684+0000","last_unstale":"2024-01-24T14:13:59.613916+0000","last_undegraded":"2024-01-24T14:13:59.613916+0000","last_fullsized":"2024-01-24T14:13:59.613916+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.582496+0000","last_change":"2024-01-24T14:13:59.185072+0000","last_active":"2024-01-24T14:13:59.582496+0000","last_peered":"2024-01-24T14:13:59.582496+0000","last_clean":"2024-01-24T14:13:59.582496+0000","last_became_active":"2024-01-24T14:13:59.184040+0000","last_became_peered":"2024-01-24T14:13:59.184040+0000","last_unstale":"2024-01-24T14:13:59.582496+0000","last_undegraded":"2024-01-24T14:13:59.582496+0000","last_fullsized":"2024-01-24T14:13:59.582496+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599036+0000","last_change":"2024-01-24T14:13:59.597766+0000","last_active":"2024-01-24T14:13:59.599036+0000","last_peered":"2024-01-24T14:13:59.599036+0000","last_clean":"2024-01-24T14:13:59.599036+0000","last_became_active":"2024-01-24T14:13:59.597476+0000","last_became_peered":"2024-01-24T14:13:59.597476+0000","last_unstale":"2024-01-24T14:13:59.599036+0000","last_undegraded":"2024-01-24T14:13:59.599036+0000","last_fullsized":"2024-01-24T14:13:59.599036+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599653+0000","last_change":"2024-01-24T14:13:59.181186+0000","last_active":"2024-01-24T14:13:59.599653+0000","last_peered":"2024-01-24T14:13:59.599653+0000","last_clean":"2024-01-24T14:13:59.599653+0000","last_became_active":"2024-01-24T14:13:59.180838+0000","last_became_peered":"2024-01-24T14:13:59.180838+0000","last_unstale":"2024-01-24T14:13:59.599653+0000","last_undegraded":"2024-01-24T14:13:59.599653+0000","last_fullsized":"2024-01-24T14:13:59.599653+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602220+0000","last_change":"2024-01-24T14:13:59.598055+0000","last_active":"2024-01-24T14:13:59.602220+0000","last_peered":"2024-01-24T14:13:59.602220+0000","last_clean":"2024-01-24T14:13:59.602220+0000","last_became_active":"2024-01-24T14:13:59.597777+0000","last_became_peered":"2024-01-24T14:13:59.597777+0000","last_unstale":"2024-01-24T14:13:59.602220+0000","last_undegraded":"2024-01-24T14:13:59.602220+0000","last_fullsized":"2024-01-24T14:13:59.602220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581576+0000","last_change":"2024-01-24T14:13:59.185733+0000","last_active":"2024-01-24T14:13:59.581576+0000","last_peered":"2024-01-24T14:13:59.581576+0000","last_clean":"2024-01-24T14:13:59.581576+0000","last_became_active":"2024-01-24T14:13:59.185482+0000","last_became_peered":"2024-01-24T14:13:59.185482+0000","last_unstale":"2024-01-24T14:13:59.581576+0000","last_undegraded":"2024-01-24T14:13:59.581576+0000","last_fullsized":"2024-01-24T14:13:59.581576+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599233+0000","last_change":"2024-01-24T14:13:59.597585+0000","last_active":"2024-01-24T14:13:59.599233+0000","last_peered":"2024-01-24T14:13:59.599233+0000","last_clean":"2024-01-24T14:13:59.599233+0000","last_became_active":"2024-01-24T14:13:59.597114+0000","last_became_peered":"2024-01-24T14:13:59.597114+0000","last_unstale":"2024-01-24T14:13:59.599233+0000","last_undegraded":"2024-01-24T14:13:59.599233+0000","last_fullsized":"2024-01-24T14:13:59.599233+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.587957+0000","last_change":"2024-01-24T14:13:59.180890+0000","last_active":"2024-01-24T14:13:59.587957+0000","last_peered":"2024-01-24T14:13:59.587957+0000","last_clean":"2024-01-24T14:13:59.587957+0000","last_became_active":"2024-01-24T14:13:59.180535+0000","last_became_peered":"2024-01-24T14:13:59.180535+0000","last_unstale":"2024-01-24T14:13:59.587957+0000","last_undegraded":"2024-01-24T14:13:59.587957+0000","last_fullsized":"2024-01-24T14:13:59.587957+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613282+0000","last_change":"2024-01-24T14:13:59.607645+0000","last_active":"2024-01-24T14:13:59.613282+0000","last_peered":"2024-01-24T14:13:59.613282+0000","last_clean":"2024-01-24T14:13:59.613282+0000","last_became_active":"2024-01-24T14:13:59.607354+0000","last_became_peered":"2024-01-24T14:13:59.607354+0000","last_unstale":"2024-01-24T14:13:59.613282+0000","last_undegraded":"2024-01-24T14:13:59.613282+0000","last_fullsized":"2024-01-24T14:13:59.613282+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597240+0000","last_change":"2024-01-24T14:13:59.187503+0000","last_active":"2024-01-24T14:13:59.597240+0000","last_peered":"2024-01-24T14:13:59.597240+0000","last_clean":"2024-01-24T14:13:59.597240+0000","last_became_active":"2024-01-24T14:13:59.187203+0000","last_became_peered":"2024-01-24T14:13:59.187203+0000","last_unstale":"2024-01-24T14:13:59.597240+0000","last_undegraded":"2024-01-24T14:13:59.597240+0000","last_fullsized":"2024-01-24T14:13:59.597240+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601777+0000","last_change":"2024-01-24T14:13:59.600823+0000","last_active":"2024-01-24T14:13:59.601777+0000","last_peered":"2024-01-24T14:13:59.601777+0000","last_clean":"2024-01-24T14:13:59.601777+0000","last_became_active":"2024-01-24T14:13:59.600322+0000","last_became_peered":"2024-01-24T14:13:59.600322+0000","last_unstale":"2024-01-24T14:13:59.601777+0000","last_undegraded":"2024-01-24T14:13:59.601777+0000","last_fullsized":"2024-01-24T14:13:59.601777+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597400+0000","last_change":"2024-01-24T14:13:59.187980+0000","last_active":"2024-01-24T14:13:59.597400+0000","last_peered":"2024-01-24T14:13:59.597400+0000","last_clean":"2024-01-24T14:13:59.597400+0000","last_became_active":"2024-01-24T14:13:59.187653+0000","last_became_peered":"2024-01-24T14:13:59.187653+0000","last_unstale":"2024-01-24T14:13:59.597400+0000","last_undegraded":"2024-01-24T14:13:59.597400+0000","last_fullsized":"2024-01-24T14:13:59.597400+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615401+0000","last_change":"2024-01-24T14:13:59.611324+0000","last_active":"2024-01-24T14:13:59.615401+0000","last_peered":"2024-01-24T14:13:59.615401+0000","last_clean":"2024-01-24T14:13:59.615401+0000","last_became_active":"2024-01-24T14:13:59.610992+0000","last_became_peered":"2024-01-24T14:13:59.610992+0000","last_unstale":"2024-01-24T14:13:59.615401+0000","last_undegraded":"2024-01-24T14:13:59.615401+0000","last_fullsized":"2024-01-24T14:13:59.615401+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585787+0000","last_change":"2024-01-24T14:13:59.183133+0000","last_active":"2024-01-24T14:13:59.585787+0000","last_peered":"2024-01-24T14:13:59.585787+0000","last_clean":"2024-01-24T14:13:59.585787+0000","last_became_active":"2024-01-24T14:13:59.182837+0000","last_became_peered":"2024-01-24T14:13:59.182837+0000","last_unstale":"2024-01-24T14:13:59.585787+0000","last_undegraded":"2024-01-24T14:13:59.585787+0000","last_fullsized":"2024-01-24T14:13:59.585787+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605252+0000","last_change":"2024-01-24T14:13:59.601562+0000","last_active":"2024-01-24T14:13:59.605252+0000","last_peered":"2024-01-24T14:13:59.605252+0000","last_clean":"2024-01-24T14:13:59.605252+0000","last_became_active":"2024-01-24T14:13:59.600361+0000","last_became_peered":"2024-01-24T14:13:59.600361+0000","last_unstale":"2024-01-24T14:13:59.605252+0000","last_undegraded":"2024-01-24T14:13:59.605252+0000","last_fullsized":"2024-01-24T14:13:59.605252+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596181+0000","last_change":"2024-01-24T14:13:59.176584+0000","last_active":"2024-01-24T14:13:59.596181+0000","last_peered":"2024-01-24T14:13:59.596181+0000","last_clean":"2024-01-24T14:13:59.596181+0000","last_became_active":"2024-01-24T14:13:59.176352+0000","last_became_peered":"2024-01-24T14:13:59.176352+0000","last_unstale":"2024-01-24T14:13:59.596181+0000","last_undegraded":"2024-01-24T14:13:59.596181+0000","last_fullsized":"2024-01-24T14:13:59.596181+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.606565+0000","last_change":"2024-01-24T14:13:59.605786+0000","last_active":"2024-01-24T14:13:59.606565+0000","last_peered":"2024-01-24T14:13:59.606565+0000","last_clean":"2024-01-24T14:13:59.606565+0000","last_became_active":"2024-01-24T14:13:59.605479+0000","last_became_peered":"2024-01-24T14:13:59.605479+0000","last_unstale":"2024-01-24T14:13:59.606565+0000","last_undegraded":"2024-01-24T14:13:59.606565+0000","last_fullsized":"2024-01-24T14:13:59.606565+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.582565+0000","last_change":"2024-01-24T14:13:59.177305+0000","last_active":"2024-01-24T14:13:59.582565+0000","last_peered":"2024-01-24T14:13:59.582565+0000","last_clean":"2024-01-24T14:13:59.582565+0000","last_became_active":"2024-01-24T14:13:59.177013+0000","last_became_peered":"2024-01-24T14:13:59.177013+0000","last_unstale":"2024-01-24T14:13:59.582565+0000","last_undegraded":"2024-01-24T14:13:59.582565+0000","last_fullsized":"2024-01-24T14:13:59.582565+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613859+0000","last_change":"2024-01-24T14:13:59.605686+0000","last_active":"2024-01-24T14:13:59.613859+0000","last_peered":"2024-01-24T14:13:59.613859+0000","last_clean":"2024-01-24T14:13:59.613859+0000","last_became_active":"2024-01-24T14:13:59.599468+0000","last_became_peered":"2024-01-24T14:13:59.599468+0000","last_unstale":"2024-01-24T14:13:59.613859+0000","last_undegraded":"2024-01-24T14:13:59.613859+0000","last_fullsized":"2024-01-24T14:13:59.613859+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597018+0000","last_change":"2024-01-24T14:13:59.176388+0000","last_active":"2024-01-24T14:13:59.597018+0000","last_peered":"2024-01-24T14:13:59.597018+0000","last_clean":"2024-01-24T14:13:59.597018+0000","last_became_active":"2024-01-24T14:13:59.176027+0000","last_became_peered":"2024-01-24T14:13:59.176027+0000","last_unstale":"2024-01-24T14:13:59.597018+0000","last_undegraded":"2024-01-24T14:13:59.597018+0000","last_fullsized":"2024-01-24T14:13:59.597018+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613713+0000","last_change":"2024-01-24T14:13:59.604787+0000","last_active":"2024-01-24T14:13:59.613713+0000","last_peered":"2024-01-24T14:13:59.613713+0000","last_clean":"2024-01-24T14:13:59.613713+0000","last_became_active":"2024-01-24T14:13:59.604506+0000","last_became_peered":"2024-01-24T14:13:59.604506+0000","last_unstale":"2024-01-24T14:13:59.613713+0000","last_undegraded":"2024-01-24T14:13:59.613713+0000","last_fullsized":"2024-01-24T14:13:59.613713+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593610+0000","last_change":"2024-01-24T14:13:59.177542+0000","last_active":"2024-01-24T14:13:59.593610+0000","last_peered":"2024-01-24T14:13:59.593610+0000","last_clean":"2024-01-24T14:13:59.593610+0000","last_became_active":"2024-01-24T14:13:59.177246+0000","last_became_peered":"2024-01-24T14:13:59.177246+0000","last_unstale":"2024-01-24T14:13:59.593610+0000","last_undegraded":"2024-01-24T14:13:59.593610+0000","last_fullsized":"2024-01-24T14:13:59.593610+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602015+0000","last_change":"2024-01-24T14:13:59.597273+0000","last_active":"2024-01-24T14:13:59.602015+0000","last_peered":"2024-01-24T14:13:59.602015+0000","last_clean":"2024-01-24T14:13:59.602015+0000","last_became_active":"2024-01-24T14:13:59.597053+0000","last_became_peered":"2024-01-24T14:13:59.597053+0000","last_unstale":"2024-01-24T14:13:59.602015+0000","last_undegraded":"2024-01-24T14:13:59.602015+0000","last_fullsized":"2024-01-24T14:13:59.602015+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591431+0000","last_change":"2024-01-24T14:13:59.187008+0000","last_active":"2024-01-24T14:13:59.591431+0000","last_peered":"2024-01-24T14:13:59.591431+0000","last_clean":"2024-01-24T14:13:59.591431+0000","last_became_active":"2024-01-24T14:13:59.186690+0000","last_became_peered":"2024-01-24T14:13:59.186690+0000","last_unstale":"2024-01-24T14:13:59.591431+0000","last_undegraded":"2024-01-24T14:13:59.591431+0000","last_fullsized":"2024-01-24T14:13:59.591431+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602216+0000","last_change":"2024-01-24T14:13:59.596466+0000","last_active":"2024-01-24T14:13:59.602216+0000","last_peered":"2024-01-24T14:13:59.602216+0000","last_clean":"2024-01-24T14:13:59.602216+0000","last_became_active":"2024-01-24T14:13:59.596179+0000","last_became_peered":"2024-01-24T14:13:59.596179+0000","last_unstale":"2024-01-24T14:13:59.602216+0000","last_undegraded":"2024-01-24T14:13:59.602216+0000","last_fullsized":"2024-01-24T14:13:59.602216+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594464+0000","last_change":"2024-01-24T14:13:59.175676+0000","last_active":"2024-01-24T14:13:59.594464+0000","last_peered":"2024-01-24T14:13:59.594464+0000","last_clean":"2024-01-24T14:13:59.594464+0000","last_became_active":"2024-01-24T14:13:59.175396+0000","last_became_peered":"2024-01-24T14:13:59.175396+0000","last_unstale":"2024-01-24T14:13:59.594464+0000","last_undegraded":"2024-01-24T14:13:59.594464+0000","last_fullsized":"2024-01-24T14:13:59.594464+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.606287+0000","last_change":"2024-01-24T14:13:59.602951+0000","last_active":"2024-01-24T14:13:59.606287+0000","last_peered":"2024-01-24T14:13:59.606287+0000","last_clean":"2024-01-24T14:13:59.606287+0000","last_became_active":"2024-01-24T14:13:59.602648+0000","last_became_peered":"2024-01-24T14:13:59.602648+0000","last_unstale":"2024-01-24T14:13:59.606287+0000","last_undegraded":"2024-01-24T14:13:59.606287+0000","last_fullsized":"2024-01-24T14:13:59.606287+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.600682+0000","last_change":"2024-01-24T14:13:59.179441+0000","last_active":"2024-01-24T14:13:59.600682+0000","last_peered":"2024-01-24T14:13:59.600682+0000","last_clean":"2024-01-24T14:13:59.600682+0000","last_became_active":"2024-01-24T14:13:59.178699+0000","last_became_peered":"2024-01-24T14:13:59.178699+0000","last_unstale":"2024-01-24T14:13:59.600682+0000","last_undegraded":"2024-01-24T14:13:59.600682+0000","last_fullsized":"2024-01-24T14:13:59.600682+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613325+0000","last_change":"2024-01-24T14:13:59.607754+0000","last_active":"2024-01-24T14:13:59.613325+0000","last_peered":"2024-01-24T14:13:59.613325+0000","last_clean":"2024-01-24T14:13:59.613325+0000","last_became_active":"2024-01-24T14:13:59.607437+0000","last_became_peered":"2024-01-24T14:13:59.607437+0000","last_unstale":"2024-01-24T14:13:59.613325+0000","last_undegraded":"2024-01-24T14:13:59.613325+0000","last_fullsized":"2024-01-24T14:13:59.613325+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605436+0000","last_change":"2024-01-24T14:13:59.180915+0000","last_active":"2024-01-24T14:13:59.605436+0000","last_peered":"2024-01-24T14:13:59.605436+0000","last_clean":"2024-01-24T14:13:59.605436+0000","last_became_active":"2024-01-24T14:13:59.180385+0000","last_became_peered":"2024-01-24T14:13:59.180385+0000","last_unstale":"2024-01-24T14:13:59.605436+0000","last_undegraded":"2024-01-24T14:13:59.605436+0000","last_fullsized":"2024-01-24T14:13:59.605436+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605231+0000","last_change":"2024-01-24T14:13:59.601292+0000","last_active":"2024-01-24T14:13:59.605231+0000","last_peered":"2024-01-24T14:13:59.605231+0000","last_clean":"2024-01-24T14:13:59.605231+0000","last_became_active":"2024-01-24T14:13:59.600834+0000","last_became_peered":"2024-01-24T14:13:59.600834+0000","last_unstale":"2024-01-24T14:13:59.605231+0000","last_undegraded":"2024-01-24T14:13:59.605231+0000","last_fullsized":"2024-01-24T14:13:59.605231+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615335+0000","last_change":"2024-01-24T14:13:59.175601+0000","last_active":"2024-01-24T14:13:59.615335+0000","last_peered":"2024-01-24T14:13:59.615335+0000","last_clean":"2024-01-24T14:13:59.615335+0000","last_became_active":"2024-01-24T14:13:59.173774+0000","last_became_peered":"2024-01-24T14:13:59.173774+0000","last_unstale":"2024-01-24T14:13:59.615335+0000","last_undegraded":"2024-01-24T14:13:59.615335+0000","last_fullsized":"2024-01-24T14:13:59.615335+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613559+0000","last_change":"2024-01-24T14:13:59.604227+0000","last_active":"2024-01-24T14:13:59.613559+0000","last_peered":"2024-01-24T14:13:59.613559+0000","last_clean":"2024-01-24T14:13:59.613559+0000","last_became_active":"2024-01-24T14:13:59.603933+0000","last_became_peered":"2024-01-24T14:13:59.603933+0000","last_unstale":"2024-01-24T14:13:59.613559+0000","last_undegraded":"2024-01-24T14:13:59.613559+0000","last_fullsized":"2024-01-24T14:13:59.613559+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597128+0000","last_change":"2024-01-24T14:13:59.169115+0000","last_active":"2024-01-24T14:13:59.597128+0000","last_peered":"2024-01-24T14:13:59.597128+0000","last_clean":"2024-01-24T14:13:59.597128+0000","last_became_active":"2024-01-24T14:13:59.168789+0000","last_became_peered":"2024-01-24T14:13:59.168789+0000","last_unstale":"2024-01-24T14:13:59.597128+0000","last_undegraded":"2024-01-24T14:13:59.597128+0000","last_fullsized":"2024-01-24T14:13:59.597128+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613839+0000","last_change":"2024-01-24T14:13:59.604215+0000","last_active":"2024-01-24T14:13:59.613839+0000","last_peered":"2024-01-24T14:13:59.613839+0000","last_clean":"2024-01-24T14:13:59.613839+0000","last_became_active":"2024-01-24T14:13:59.603918+0000","last_became_peered":"2024-01-24T14:13:59.603918+0000","last_unstale":"2024-01-24T14:13:59.613839+0000","last_undegraded":"2024-01-24T14:13:59.613839+0000","last_fullsized":"2024-01-24T14:13:59.613839+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588396+0000","last_change":"2024-01-24T14:13:59.181773+0000","last_active":"2024-01-24T14:13:59.588396+0000","last_peered":"2024-01-24T14:13:59.588396+0000","last_clean":"2024-01-24T14:13:59.588396+0000","last_became_active":"2024-01-24T14:13:59.181227+0000","last_became_peered":"2024-01-24T14:13:59.181227+0000","last_unstale":"2024-01-24T14:13:59.588396+0000","last_undegraded":"2024-01-24T14:13:59.588396+0000","last_fullsized":"2024-01-24T14:13:59.588396+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602292+0000","last_change":"2024-01-24T14:13:59.598283+0000","last_active":"2024-01-24T14:13:59.602292+0000","last_peered":"2024-01-24T14:13:59.602292+0000","last_clean":"2024-01-24T14:13:59.602292+0000","last_became_active":"2024-01-24T14:13:59.598020+0000","last_became_peered":"2024-01-24T14:13:59.598020+0000","last_unstale":"2024-01-24T14:13:59.602292+0000","last_undegraded":"2024-01-24T14:13:59.602292+0000","last_fullsized":"2024-01-24T14:13:59.602292+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588480+0000","last_change":"2024-01-24T14:13:59.180957+0000","last_active":"2024-01-24T14:13:59.588480+0000","last_peered":"2024-01-24T14:13:59.588480+0000","last_clean":"2024-01-24T14:13:59.588480+0000","last_became_active":"2024-01-24T14:13:59.180449+0000","last_became_peered":"2024-01-24T14:13:59.180449+0000","last_unstale":"2024-01-24T14:13:59.588480+0000","last_undegraded":"2024-01-24T14:13:59.588480+0000","last_fullsized":"2024-01-24T14:13:59.588480+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616746+0000","last_change":"2024-01-24T14:13:59.610454+0000","last_active":"2024-01-24T14:13:59.616746+0000","last_peered":"2024-01-24T14:13:59.616746+0000","last_clean":"2024-01-24T14:13:59.616746+0000","last_became_active":"2024-01-24T14:13:59.610190+0000","last_became_peered":"2024-01-24T14:13:59.610190+0000","last_unstale":"2024-01-24T14:13:59.616746+0000","last_undegraded":"2024-01-24T14:13:59.616746+0000","last_fullsized":"2024-01-24T14:13:59.616746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.600543+0000","last_change":"2024-01-24T14:13:59.177014+0000","last_active":"2024-01-24T14:13:59.600543+0000","last_peered":"2024-01-24T14:13:59.600543+0000","last_clean":"2024-01-24T14:13:59.600543+0000","last_became_active":"2024-01-24T14:13:59.176432+0000","last_became_peered":"2024-01-24T14:13:59.176432+0000","last_unstale":"2024-01-24T14:13:59.600543+0000","last_undegraded":"2024-01-24T14:13:59.600543+0000","last_fullsized":"2024-01-24T14:13:59.600543+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.608927+0000","last_change":"2024-01-24T14:13:59.604972+0000","last_active":"2024-01-24T14:13:59.608927+0000","last_peered":"2024-01-24T14:13:59.608927+0000","last_clean":"2024-01-24T14:13:59.608927+0000","last_became_active":"2024-01-24T14:13:59.604643+0000","last_became_peered":"2024-01-24T14:13:59.604643+0000","last_unstale":"2024-01-24T14:13:59.608927+0000","last_undegraded":"2024-01-24T14:13:59.608927+0000","last_fullsized":"2024-01-24T14:13:59.608927+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591156+0000","last_change":"2024-01-24T14:13:59.184839+0000","last_active":"2024-01-24T14:13:59.591156+0000","last_peered":"2024-01-24T14:13:59.591156+0000","last_clean":"2024-01-24T14:13:59.591156+0000","last_became_active":"2024-01-24T14:13:59.184535+0000","last_became_peered":"2024-01-24T14:13:59.184535+0000","last_unstale":"2024-01-24T14:13:59.591156+0000","last_undegraded":"2024-01-24T14:13:59.591156+0000","last_fullsized":"2024-01-24T14:13:59.591156+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604995+0000","last_change":"2024-01-24T14:13:59.600061+0000","last_active":"2024-01-24T14:13:59.604995+0000","last_peered":"2024-01-24T14:13:59.604995+0000","last_clean":"2024-01-24T14:13:59.604995+0000","last_became_active":"2024-01-24T14:13:59.599799+0000","last_became_peered":"2024-01-24T14:13:59.599799+0000","last_unstale":"2024-01-24T14:13:59.604995+0000","last_undegraded":"2024-01-24T14:13:59.604995+0000","last_fullsized":"2024-01-24T14:13:59.604995+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591563+0000","last_change":"2024-01-24T14:13:59.179655+0000","last_active":"2024-01-24T14:13:59.591563+0000","last_peered":"2024-01-24T14:13:59.591563+0000","last_clean":"2024-01-24T14:13:59.591563+0000","last_became_active":"2024-01-24T14:13:59.179068+0000","last_became_peered":"2024-01-24T14:13:59.179068+0000","last_unstale":"2024-01-24T14:13:59.591563+0000","last_undegraded":"2024-01-24T14:13:59.591563+0000","last_fullsized":"2024-01-24T14:13:59.591563+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602283+0000","last_change":"2024-01-24T14:13:59.594537+0000","last_active":"2024-01-24T14:13:59.602283+0000","last_peered":"2024-01-24T14:13:59.602283+0000","last_clean":"2024-01-24T14:13:59.602283+0000","last_became_active":"2024-01-24T14:13:59.594254+0000","last_became_peered":"2024-01-24T14:13:59.594254+0000","last_unstale":"2024-01-24T14:13:59.602283+0000","last_undegraded":"2024-01-24T14:13:59.602283+0000","last_fullsized":"2024-01-24T14:13:59.602283+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591800+0000","last_change":"2024-01-24T14:13:59.182373+0000","last_active":"2024-01-24T14:13:59.591800+0000","last_peered":"2024-01-24T14:13:59.591800+0000","last_clean":"2024-01-24T14:13:59.591800+0000","last_became_active":"2024-01-24T14:13:59.182077+0000","last_became_peered":"2024-01-24T14:13:59.182077+0000","last_unstale":"2024-01-24T14:13:59.591800+0000","last_undegraded":"2024-01-24T14:13:59.591800+0000","last_fullsized":"2024-01-24T14:13:59.591800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613637+0000","last_change":"2024-01-24T14:13:59.607656+0000","last_active":"2024-01-24T14:13:59.613637+0000","last_peered":"2024-01-24T14:13:59.613637+0000","last_clean":"2024-01-24T14:13:59.613637+0000","last_became_active":"2024-01-24T14:13:59.607365+0000","last_became_peered":"2024-01-24T14:13:59.607365+0000","last_unstale":"2024-01-24T14:13:59.613637+0000","last_undegraded":"2024-01-24T14:13:59.613637+0000","last_fullsized":"2024-01-24T14:13:59.613637+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"104'3","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601949+0000","last_change":"2024-01-24T14:13:59.182375+0000","last_active":"2024-01-24T14:13:59.601949+0000","last_peered":"2024-01-24T14:13:59.601949+0000","last_clean":"2024-01-24T14:13:59.601949+0000","last_became_active":"2024-01-24T14:13:59.182150+0000","last_became_peered":"2024-01-24T14:13:59.182150+0000","last_unstale":"2024-01-24T14:13:59.601949+0000","last_undegraded":"2024-01-24T14:13:59.601949+0000","last_fullsized":"2024-01-24T14:13:59.601949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612014+0000","last_change":"2024-01-24T14:13:59.608414+0000","last_active":"2024-01-24T14:13:59.612014+0000","last_peered":"2024-01-24T14:13:59.612014+0000","last_clean":"2024-01-24T14:13:59.612014+0000","last_became_active":"2024-01-24T14:13:59.608013+0000","last_became_peered":"2024-01-24T14:13:59.608013+0000","last_unstale":"2024-01-24T14:13:59.612014+0000","last_undegraded":"2024-01-24T14:13:59.612014+0000","last_fullsized":"2024-01-24T14:13:59.612014+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.607061+0000","last_change":"2024-01-24T14:13:59.178726+0000","last_active":"2024-01-24T14:13:59.607061+0000","last_peered":"2024-01-24T14:13:59.607061+0000","last_clean":"2024-01-24T14:13:59.607061+0000","last_became_active":"2024-01-24T14:13:59.178463+0000","last_became_peered":"2024-01-24T14:13:59.178463+0000","last_unstale":"2024-01-24T14:13:59.607061+0000","last_undegraded":"2024-01-24T14:13:59.607061+0000","last_fullsized":"2024-01-24T14:13:59.607061+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.614134+0000","last_change":"2024-01-24T14:13:59.605906+0000","last_active":"2024-01-24T14:13:59.614134+0000","last_peered":"2024-01-24T14:13:59.614134+0000","last_clean":"2024-01-24T14:13:59.614134+0000","last_became_active":"2024-01-24T14:13:59.605404+0000","last_became_peered":"2024-01-24T14:13:59.605404+0000","last_unstale":"2024-01-24T14:13:59.614134+0000","last_undegraded":"2024-01-24T14:13:59.614134+0000","last_fullsized":"2024-01-24T14:13:59.614134+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613963+0000","last_change":"2024-01-24T14:13:59.174910+0000","last_active":"2024-01-24T14:13:59.613963+0000","last_peered":"2024-01-24T14:13:59.613963+0000","last_clean":"2024-01-24T14:13:59.613963+0000","last_became_active":"2024-01-24T14:13:59.174603+0000","last_became_peered":"2024-01-24T14:13:59.174603+0000","last_unstale":"2024-01-24T14:13:59.613963+0000","last_undegraded":"2024-01-24T14:13:59.613963+0000","last_fullsized":"2024-01-24T14:13:59.613963+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.618795+0000","last_change":"2024-01-24T14:13:59.614234+0000","last_active":"2024-01-24T14:13:59.618795+0000","last_peered":"2024-01-24T14:13:59.618795+0000","last_clean":"2024-01-24T14:13:59.618795+0000","last_became_active":"2024-01-24T14:13:59.614018+0000","last_became_peered":"2024-01-24T14:13:59.614018+0000","last_unstale":"2024-01-24T14:13:59.618795+0000","last_undegraded":"2024-01-24T14:13:59.618795+0000","last_fullsized":"2024-01-24T14:13:59.618795+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585537+0000","last_change":"2024-01-24T14:13:59.183528+0000","last_active":"2024-01-24T14:13:59.585537+0000","last_peered":"2024-01-24T14:13:59.585537+0000","last_clean":"2024-01-24T14:13:59.585537+0000","last_became_active":"2024-01-24T14:13:59.183214+0000","last_became_peered":"2024-01-24T14:13:59.183214+0000","last_unstale":"2024-01-24T14:13:59.585537+0000","last_undegraded":"2024-01-24T14:13:59.585537+0000","last_fullsized":"2024-01-24T14:13:59.585537+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613805+0000","last_change":"2024-01-24T14:13:59.605325+0000","last_active":"2024-01-24T14:13:59.613805+0000","last_peered":"2024-01-24T14:13:59.613805+0000","last_clean":"2024-01-24T14:13:59.613805+0000","last_became_active":"2024-01-24T14:13:59.605027+0000","last_became_peered":"2024-01-24T14:13:59.605027+0000","last_unstale":"2024-01-24T14:13:59.613805+0000","last_undegraded":"2024-01-24T14:13:59.613805+0000","last_fullsized":"2024-01-24T14:13:59.613805+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.586227+0000","last_change":"2024-01-24T14:13:59.182994+0000","last_active":"2024-01-24T14:13:59.586227+0000","last_peered":"2024-01-24T14:13:59.586227+0000","last_clean":"2024-01-24T14:13:59.586227+0000","last_became_active":"2024-01-24T14:13:59.182682+0000","last_became_peered":"2024-01-24T14:13:59.182682+0000","last_unstale":"2024-01-24T14:13:59.586227+0000","last_undegraded":"2024-01-24T14:13:59.586227+0000","last_fullsized":"2024-01-24T14:13:59.586227+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615224+0000","last_change":"2024-01-24T14:13:59.610385+0000","last_active":"2024-01-24T14:13:59.615224+0000","last_peered":"2024-01-24T14:13:59.615224+0000","last_clean":"2024-01-24T14:13:59.615224+0000","last_became_active":"2024-01-24T14:13:59.609240+0000","last_became_peered":"2024-01-24T14:13:59.609240+0000","last_unstale":"2024-01-24T14:13:59.615224+0000","last_undegraded":"2024-01-24T14:13:59.615224+0000","last_fullsized":"2024-01-24T14:13:59.615224+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.617265+0000","last_change":"2024-01-24T14:13:59.180036+0000","last_active":"2024-01-24T14:13:59.617265+0000","last_peered":"2024-01-24T14:13:59.617265+0000","last_clean":"2024-01-24T14:13:59.617265+0000","last_became_active":"2024-01-24T14:13:59.179065+0000","last_became_peered":"2024-01-24T14:13:59.179065+0000","last_unstale":"2024-01-24T14:13:59.617265+0000","last_undegraded":"2024-01-24T14:13:59.617265+0000","last_fullsized":"2024-01-24T14:13:59.617265+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.603037+0000","last_change":"2024-01-24T14:13:59.600098+0000","last_active":"2024-01-24T14:13:59.603037+0000","last_peered":"2024-01-24T14:13:59.603037+0000","last_clean":"2024-01-24T14:13:59.603037+0000","last_became_active":"2024-01-24T14:13:59.599838+0000","last_became_peered":"2024-01-24T14:13:59.599838+0000","last_unstale":"2024-01-24T14:13:59.603037+0000","last_undegraded":"2024-01-24T14:13:59.603037+0000","last_fullsized":"2024-01-24T14:13:59.603037+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601872+0000","last_change":"2024-01-24T14:13:59.178489+0000","last_active":"2024-01-24T14:13:59.601872+0000","last_peered":"2024-01-24T14:13:59.601872+0000","last_clean":"2024-01-24T14:13:59.601872+0000","last_became_active":"2024-01-24T14:13:59.178185+0000","last_became_peered":"2024-01-24T14:13:59.178185+0000","last_unstale":"2024-01-24T14:13:59.601872+0000","last_undegraded":"2024-01-24T14:13:59.601872+0000","last_fullsized":"2024-01-24T14:13:59.601872+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616653+0000","last_change":"2024-01-24T14:13:59.609527+0000","last_active":"2024-01-24T14:13:59.616653+0000","last_peered":"2024-01-24T14:13:59.616653+0000","last_clean":"2024-01-24T14:13:59.616653+0000","last_became_active":"2024-01-24T14:13:59.609180+0000","last_became_peered":"2024-01-24T14:13:59.609180+0000","last_unstale":"2024-01-24T14:13:59.616653+0000","last_undegraded":"2024-01-24T14:13:59.616653+0000","last_fullsized":"2024-01-24T14:13:59.616653+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591353+0000","last_change":"2024-01-24T14:13:59.187223+0000","last_active":"2024-01-24T14:13:59.591353+0000","last_peered":"2024-01-24T14:13:59.591353+0000","last_clean":"2024-01-24T14:13:59.591353+0000","last_became_active":"2024-01-24T14:13:59.186908+0000","last_became_peered":"2024-01-24T14:13:59.186908+0000","last_unstale":"2024-01-24T14:13:59.591353+0000","last_undegraded":"2024-01-24T14:13:59.591353+0000","last_fullsized":"2024-01-24T14:13:59.591353+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604295+0000","last_change":"2024-01-24T14:13:59.597776+0000","last_active":"2024-01-24T14:13:59.604295+0000","last_peered":"2024-01-24T14:13:59.604295+0000","last_clean":"2024-01-24T14:13:59.604295+0000","last_became_active":"2024-01-24T14:13:59.597468+0000","last_became_peered":"2024-01-24T14:13:59.597468+0000","last_unstale":"2024-01-24T14:13:59.604295+0000","last_undegraded":"2024-01-24T14:13:59.604295+0000","last_fullsized":"2024-01-24T14:13:59.604295+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588522+0000","last_change":"2024-01-24T14:13:59.169565+0000","last_active":"2024-01-24T14:13:59.588522+0000","last_peered":"2024-01-24T14:13:59.588522+0000","last_clean":"2024-01-24T14:13:59.588522+0000","last_became_active":"2024-01-24T14:13:59.169300+0000","last_became_peered":"2024-01-24T14:13:59.169300+0000","last_unstale":"2024-01-24T14:13:59.588522+0000","last_undegraded":"2024-01-24T14:13:59.588522+0000","last_fullsized":"2024-01-24T14:13:59.588522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.610569+0000","last_change":"2024-01-24T14:13:59.607765+0000","last_active":"2024-01-24T14:13:59.610569+0000","last_peered":"2024-01-24T14:13:59.610569+0000","last_clean":"2024-01-24T14:13:59.610569+0000","last_became_active":"2024-01-24T14:13:59.607462+0000","last_became_peered":"2024-01-24T14:13:59.607462+0000","last_unstale":"2024-01-24T14:13:59.610569+0000","last_undegraded":"2024-01-24T14:13:59.610569+0000","last_fullsized":"2024-01-24T14:13:59.610569+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.590394+0000","last_change":"2024-01-24T14:13:59.179494+0000","last_active":"2024-01-24T14:13:59.590394+0000","last_peered":"2024-01-24T14:13:59.590394+0000","last_clean":"2024-01-24T14:13:59.590394+0000","last_became_active":"2024-01-24T14:13:59.178687+0000","last_became_peered":"2024-01-24T14:13:59.178687+0000","last_unstale":"2024-01-24T14:13:59.590394+0000","last_undegraded":"2024-01-24T14:13:59.590394+0000","last_fullsized":"2024-01-24T14:13:59.590394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.598793+0000","last_change":"2024-01-24T14:13:59.592141+0000","last_active":"2024-01-24T14:13:59.598793+0000","last_peered":"2024-01-24T14:13:59.598793+0000","last_clean":"2024-01-24T14:13:59.598793+0000","last_became_active":"2024-01-24T14:13:59.590960+0000","last_became_peered":"2024-01-24T14:13:59.590960+0000","last_unstale":"2024-01-24T14:13:59.598793+0000","last_undegraded":"2024-01-24T14:13:59.598793+0000","last_fullsized":"2024-01-24T14:13:59.598793+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597174+0000","last_change":"2024-01-24T14:13:59.186128+0000","last_active":"2024-01-24T14:13:59.597174+0000","last_peered":"2024-01-24T14:13:59.597174+0000","last_clean":"2024-01-24T14:13:59.597174+0000","last_became_active":"2024-01-24T14:13:59.177094+0000","last_became_peered":"2024-01-24T14:13:59.177094+0000","last_unstale":"2024-01-24T14:13:59.597174+0000","last_undegraded":"2024-01-24T14:13:59.597174+0000","last_fullsized":"2024-01-24T14:13:59.597174+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613687+0000","last_change":"2024-01-24T14:13:59.606315+0000","last_active":"2024-01-24T14:13:59.613687+0000","last_peered":"2024-01-24T14:13:59.613687+0000","last_clean":"2024-01-24T14:13:59.613687+0000","last_became_active":"2024-01-24T14:13:59.606045+0000","last_became_peered":"2024-01-24T14:13:59.606045+0000","last_unstale":"2024-01-24T14:13:59.613687+0000","last_undegraded":"2024-01-24T14:13:59.613687+0000","last_fullsized":"2024-01-24T14:13:59.613687+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588511+0000","last_change":"2024-01-24T14:13:59.180639+0000","last_active":"2024-01-24T14:13:59.588511+0000","last_peered":"2024-01-24T14:13:59.588511+0000","last_clean":"2024-01-24T14:13:59.588511+0000","last_became_active":"2024-01-24T14:13:59.180142+0000","last_became_peered":"2024-01-24T14:13:59.180142+0000","last_unstale":"2024-01-24T14:13:59.588511+0000","last_undegraded":"2024-01-24T14:13:59.588511+0000","last_fullsized":"2024-01-24T14:13:59.588511+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616523+0000","last_change":"2024-01-24T14:13:59.606110+0000","last_active":"2024-01-24T14:13:59.616523+0000","last_peered":"2024-01-24T14:13:59.616523+0000","last_clean":"2024-01-24T14:13:59.616523+0000","last_became_active":"2024-01-24T14:13:59.605783+0000","last_became_peered":"2024-01-24T14:13:59.605783+0000","last_unstale":"2024-01-24T14:13:59.616523+0000","last_undegraded":"2024-01-24T14:13:59.616523+0000","last_fullsized":"2024-01-24T14:13:59.616523+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.590527+0000","last_change":"2024-01-24T14:13:59.179345+0000","last_active":"2024-01-24T14:13:59.590527+0000","last_peered":"2024-01-24T14:13:59.590527+0000","last_clean":"2024-01-24T14:13:59.590527+0000","last_became_active":"2024-01-24T14:13:59.178902+0000","last_became_peered":"2024-01-24T14:13:59.178902+0000","last_unstale":"2024-01-24T14:13:59.590527+0000","last_undegraded":"2024-01-24T14:13:59.590527+0000","last_fullsized":"2024-01-24T14:13:59.590527+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605142+0000","last_change":"2024-01-24T14:13:59.600846+0000","last_active":"2024-01-24T14:13:59.605142+0000","last_peered":"2024-01-24T14:13:59.605142+0000","last_clean":"2024-01-24T14:13:59.605142+0000","last_became_active":"2024-01-24T14:13:59.600474+0000","last_became_peered":"2024-01-24T14:13:59.600474+0000","last_unstale":"2024-01-24T14:13:59.605142+0000","last_undegraded":"2024-01-24T14:13:59.605142+0000","last_fullsized":"2024-01-24T14:13:59.605142+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588283+0000","last_change":"2024-01-24T14:13:59.168449+0000","last_active":"2024-01-24T14:13:59.588283+0000","last_peered":"2024-01-24T14:13:59.588283+0000","last_clean":"2024-01-24T14:13:59.588283+0000","last_became_active":"2024-01-24T14:13:59.168136+0000","last_became_peered":"2024-01-24T14:13:59.168136+0000","last_unstale":"2024-01-24T14:13:59.588283+0000","last_undegraded":"2024-01-24T14:13:59.588283+0000","last_fullsized":"2024-01-24T14:13:59.588283+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.598763+0000","last_change":"2024-01-24T14:13:59.592342+0000","last_active":"2024-01-24T14:13:59.598763+0000","last_peered":"2024-01-24T14:13:59.598763+0000","last_clean":"2024-01-24T14:13:59.598763+0000","last_became_active":"2024-01-24T14:13:59.590942+0000","last_became_peered":"2024-01-24T14:13:59.590942+0000","last_unstale":"2024-01-24T14:13:59.598763+0000","last_undegraded":"2024-01-24T14:13:59.598763+0000","last_fullsized":"2024-01-24T14:13:59.598763+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581385+0000","last_change":"2024-01-24T14:13:59.173283+0000","last_active":"2024-01-24T14:13:59.581385+0000","last_peered":"2024-01-24T14:13:59.581385+0000","last_clean":"2024-01-24T14:13:59.581385+0000","last_became_active":"2024-01-24T14:13:59.172965+0000","last_became_peered":"2024-01-24T14:13:59.172965+0000","last_unstale":"2024-01-24T14:13:59.581385+0000","last_undegraded":"2024-01-24T14:13:59.581385+0000","last_fullsized":"2024-01-24T14:13:59.581385+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591632+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:13:59.591632+0000","last_peered":"2024-01-24T14:13:59.591632+0000","last_clean":"2024-01-24T14:13:59.591632+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:13:59.591632+0000","last_undegraded":"2024-01-24T14:13:59.591632+0000","last_fullsized":"2024-01-24T14:13:59.591632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.603074+0000","last_change":"2024-01-24T14:13:59.599657+0000","last_active":"2024-01-24T14:13:59.603074+0000","last_peered":"2024-01-24T14:13:59.603074+0000","last_clean":"2024-01-24T14:13:59.603074+0000","last_became_active":"2024-01-24T14:13:59.599353+0000","last_became_peered":"2024-01-24T14:13:59.599353+0000","last_unstale":"2024-01-24T14:13:59.603074+0000","last_undegraded":"2024-01-24T14:13:59.603074+0000","last_fullsized":"2024-01-24T14:13:59.603074+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597437+0000","last_change":"2024-01-24T14:13:59.179171+0000","last_active":"2024-01-24T14:13:59.597437+0000","last_peered":"2024-01-24T14:13:59.597437+0000","last_clean":"2024-01-24T14:13:59.597437+0000","last_became_active":"2024-01-24T14:13:59.176714+0000","last_became_peered":"2024-01-24T14:13:59.176714+0000","last_unstale":"2024-01-24T14:13:59.597437+0000","last_undegraded":"2024-01-24T14:13:59.597437+0000","last_fullsized":"2024-01-24T14:13:59.597437+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585663+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:13:59.585663+0000","last_peered":"2024-01-24T14:13:59.585663+0000","last_clean":"2024-01-24T14:13:59.585663+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:13:59.585663+0000","last_undegraded":"2024-01-24T14:13:59.585663+0000","last_fullsized":"2024-01-24T14:13:59.585663+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613485+0000","last_change":"2024-01-24T14:13:59.603439+0000","last_active":"2024-01-24T14:13:59.613485+0000","last_peered":"2024-01-24T14:13:59.613485+0000","last_clean":"2024-01-24T14:13:59.613485+0000","last_became_active":"2024-01-24T14:13:59.603116+0000","last_became_peered":"2024-01-24T14:13:59.603116+0000","last_unstale":"2024-01-24T14:13:59.613485+0000","last_undegraded":"2024-01-24T14:13:59.613485+0000","last_fullsized":"2024-01-24T14:13:59.613485+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602963+0000","last_change":"2024-01-24T14:13:59.181711+0000","last_active":"2024-01-24T14:13:59.602963+0000","last_peered":"2024-01-24T14:13:59.602963+0000","last_clean":"2024-01-24T14:13:59.602963+0000","last_became_active":"2024-01-24T14:13:59.176397+0000","last_became_peered":"2024-01-24T14:13:59.176397+0000","last_unstale":"2024-01-24T14:13:59.602963+0000","last_undegraded":"2024-01-24T14:13:59.602963+0000","last_fullsized":"2024-01-24T14:13:59.602963+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581282+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:13:59.581282+0000","last_peered":"2024-01-24T14:13:59.581282+0000","last_clean":"2024-01-24T14:13:59.581282+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:13:59.581282+0000","last_undegraded":"2024-01-24T14:13:59.581282+0000","last_fullsized":"2024-01-24T14:13:59.581282+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613322+0000","last_change":"2024-01-24T14:13:59.601159+0000","last_active":"2024-01-24T14:13:59.613322+0000","last_peered":"2024-01-24T14:13:59.613322+0000","last_clean":"2024-01-24T14:13:59.613322+0000","last_became_active":"2024-01-24T14:13:59.600870+0000","last_became_peered":"2024-01-24T14:13:59.600870+0000","last_unstale":"2024-01-24T14:13:59.613322+0000","last_undegraded":"2024-01-24T14:13:59.613322+0000","last_fullsized":"2024-01-24T14:13:59.613322+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.595988+0000","last_change":"2024-01-24T14:13:59.187233+0000","last_active":"2024-01-24T14:13:59.595988+0000","last_peered":"2024-01-24T14:13:59.595988+0000","last_clean":"2024-01-24T14:13:59.595988+0000","last_became_active":"2024-01-24T14:13:59.186954+0000","last_became_peered":"2024-01-24T14:13:59.186954+0000","last_unstale":"2024-01-24T14:13:59.595988+0000","last_undegraded":"2024-01-24T14:13:59.595988+0000","last_fullsized":"2024-01-24T14:13:59.595988+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585302+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:13:59.585302+0000","last_peered":"2024-01-24T14:13:59.585302+0000","last_clean":"2024-01-24T14:13:59.585302+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:13:59.585302+0000","last_undegraded":"2024-01-24T14:13:59.585302+0000","last_fullsized":"2024-01-24T14:13:59.585302+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594597+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:13:59.594597+0000","last_peered":"2024-01-24T14:13:59.594597+0000","last_clean":"2024-01-24T14:13:59.594597+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:13:59.594597+0000","last_undegraded":"2024-01-24T14:13:59.594597+0000","last_fullsized":"2024-01-24T14:13:59.594597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605110+0000","last_change":"2024-01-24T14:13:59.601219+0000","last_active":"2024-01-24T14:13:59.605110+0000","last_peered":"2024-01-24T14:13:59.605110+0000","last_clean":"2024-01-24T14:13:59.605110+0000","last_became_active":"2024-01-24T14:13:59.600941+0000","last_became_peered":"2024-01-24T14:13:59.600941+0000","last_unstale":"2024-01-24T14:13:59.605110+0000","last_undegraded":"2024-01-24T14:13:59.605110+0000","last_fullsized":"2024-01-24T14:13:59.605110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613432+0000","last_change":"2024-01-24T14:13:59.187354+0000","last_active":"2024-01-24T14:13:59.613432+0000","last_peered":"2024-01-24T14:13:59.613432+0000","last_clean":"2024-01-24T14:13:59.613432+0000","last_became_active":"2024-01-24T14:13:59.187034+0000","last_became_peered":"2024-01-24T14:13:59.187034+0000","last_unstale":"2024-01-24T14:13:59.613432+0000","last_undegraded":"2024-01-24T14:13:59.613432+0000","last_fullsized":"2024-01-24T14:13:59.613432+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594511+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:13:59.594511+0000","last_peered":"2024-01-24T14:13:59.594511+0000","last_clean":"2024-01-24T14:13:59.594511+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:13:59.594511+0000","last_undegraded":"2024-01-24T14:13:59.594511+0000","last_fullsized":"2024-01-24T14:13:59.594511+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.608535+0000","last_change":"2024-01-24T14:13:59.600293+0000","last_active":"2024-01-24T14:13:59.608535+0000","last_peered":"2024-01-24T14:13:59.608535+0000","last_clean":"2024-01-24T14:13:59.608535+0000","last_became_active":"2024-01-24T14:13:59.599864+0000","last_became_peered":"2024-01-24T14:13:59.599864+0000","last_unstale":"2024-01-24T14:13:59.608535+0000","last_undegraded":"2024-01-24T14:13:59.608535+0000","last_fullsized":"2024-01-24T14:13:59.608535+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613067+0000","last_change":"2024-01-24T14:13:59.173278+0000","last_active":"2024-01-24T14:13:59.613067+0000","last_peered":"2024-01-24T14:13:59.613067+0000","last_clean":"2024-01-24T14:13:59.613067+0000","last_became_active":"2024-01-24T14:13:59.172981+0000","last_became_peered":"2024-01-24T14:13:59.172981+0000","last_unstale":"2024-01-24T14:13:59.613067+0000","last_undegraded":"2024-01-24T14:13:59.613067+0000","last_fullsized":"2024-01-24T14:13:59.613067+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":87,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596080+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:13:59.596080+0000","last_peered":"2024-01-24T14:13:59.596080+0000","last_clean":"2024-01-24T14:13:59.596080+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:13:59.596080+0000","last_undegraded":"2024-01-24T14:13:59.596080+0000","last_fullsized":"2024-01-24T14:13:59.596080+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604651+0000","last_change":"2024-01-24T14:13:59.597915+0000","last_active":"2024-01-24T14:13:59.604651+0000","last_peered":"2024-01-24T14:13:59.604651+0000","last_clean":"2024-01-24T14:13:59.604651+0000","last_became_active":"2024-01-24T14:13:59.597457+0000","last_became_peered":"2024-01-24T14:13:59.597457+0000","last_unstale":"2024-01-24T14:13:59.604651+0000","last_undegraded":"2024-01-24T14:13:59.604651+0000","last_fullsized":"2024-01-24T14:13:59.604651+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585416+0000","last_change":"2024-01-24T14:13:59.187991+0000","last_active":"2024-01-24T14:13:59.585416+0000","last_peered":"2024-01-24T14:13:59.585416+0000","last_clean":"2024-01-24T14:13:59.585416+0000","last_became_active":"2024-01-24T14:13:59.187677+0000","last_became_peered":"2024-01-24T14:13:59.187677+0000","last_unstale":"2024-01-24T14:13:59.585416+0000","last_undegraded":"2024-01-24T14:13:59.585416+0000","last_fullsized":"2024-01-24T14:13:59.585416+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591436+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:13:59.591436+0000","last_peered":"2024-01-24T14:13:59.591436+0000","last_clean":"2024-01-24T14:13:59.591436+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:13:59.591436+0000","last_undegraded":"2024-01-24T14:13:59.591436+0000","last_fullsized":"2024-01-24T14:13:59.591436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593811+0000","last_change":"2024-01-24T14:13:59.587516+0000","last_active":"2024-01-24T14:13:59.593811+0000","last_peered":"2024-01-24T14:13:59.593811+0000","last_clean":"2024-01-24T14:13:59.593811+0000","last_became_active":"2024-01-24T14:13:59.587242+0000","last_became_peered":"2024-01-24T14:13:59.587242+0000","last_unstale":"2024-01-24T14:13:59.593811+0000","last_undegraded":"2024-01-24T14:13:59.593811+0000","last_fullsized":"2024-01-24T14:13:59.593811+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581221+0000","last_change":"2024-01-24T14:13:59.176122+0000","last_active":"2024-01-24T14:13:59.581221+0000","last_peered":"2024-01-24T14:13:59.581221+0000","last_clean":"2024-01-24T14:13:59.581221+0000","last_became_active":"2024-01-24T14:13:59.175794+0000","last_became_peered":"2024-01-24T14:13:59.175794+0000","last_unstale":"2024-01-24T14:13:59.581221+0000","last_undegraded":"2024-01-24T14:13:59.581221+0000","last_fullsized":"2024-01-24T14:13:59.581221+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":84,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591329+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:13:59.591329+0000","last_peered":"2024-01-24T14:13:59.591329+0000","last_clean":"2024-01-24T14:13:59.591329+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:13:59.591329+0000","last_undegraded":"2024-01-24T14:13:59.591329+0000","last_fullsized":"2024-01-24T14:13:59.591329+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575038,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1290240,"data_stored":710498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.26000000000000001},"max":{"1min":0.80600000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.32700000000000001},"max":{"1min":0.753,"5min":0.82599999999999996,"15min":0.83499999999999996},"last":0.58899999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.312},"max":{"1min":0.75600000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":3.2570000000000001},"last":0.53000000000000003}]},{"osd":2,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.51700000000000002,"5min":0.40400000000000003,"15min":0.32000000000000001},"max":{"1min":0.79100000000000004,"5min":0.872,"15min":0.876},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.39300000000000002,"15min":0.36799999999999999},"max":{"1min":0.80000000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66700000000000004}]},{"osd":3,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.36499999999999999},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.85099999999999998,"15min":3.0049999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48899999999999999},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.84699999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.60399999999999998,"5min":0.70599999999999996,"15min":0.83599999999999997},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.501,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.29999999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.80900000000000005},"last":0.79400000000000004}]},{"osd":6,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.621},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.80700000000000005,"15min":0.87},"last":0.72199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575039,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302456,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069384,"statfs":{"total":96636764160,"available":96327049216,"internally_reserved":0,"allocated":1294336,"data_stored":710056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.59499999999999997},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73699999999999999,"5min":0.85699999999999998,"15min":0.86899999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":0.84899999999999998,"15min":1.3859999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73499999999999999,"5min":0.77000000000000002,"15min":1.353},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.78700000000000003,"5min":1.032,"15min":1.032},"last":0.46999999999999997}]},{"osd":2,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60099999999999998,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.39100000000000001,"15min":0.34300000000000003},"max":{"1min":0.78600000000000003,"5min":0.88500000000000001,"15min":6.8129999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.4079999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.66900000000000004},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81699999999999995,"5min":0.82799999999999996,"15min":7.016},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.43099999999999999,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.77400000000000002,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.63800000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.504,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":1.165,"15min":1.165},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.314},"max":{"1min":0.65500000000000003,"5min":0.82299999999999995,"15min":6.8719999999999999},"last":0.624}]},{"osd":5,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":0.86699999999999999,"15min":6.976},"last":0.497},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80200000000000005,"5min":0.85699999999999998,"15min":7.8789999999999996},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.747,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":0.89600000000000002,"15min":6.9080000000000004},"last":0.66200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302456,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069384,"statfs":{"total":96636764160,"available":96327049216,"internally_reserved":0,"allocated":1294336,"data_stored":710068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.51900000000000002,"15min":0.443},"min":{"1min":0.311,"5min":0.245,"15min":0.23400000000000001},"max":{"1min":0.57899999999999996,"5min":15.298999999999999,"15min":15.298999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.55000000000000004,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.57499999999999996,"5min":14.654,"15min":14.654},"last":0.38}]},{"osd":2,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45900000000000002,"15min":0.45100000000000001},"min":{"1min":0.34899999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.59799999999999998,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.25800000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47099999999999997,"15min":0.45700000000000002},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88600000000000001,"5min":1.073,"15min":1.137},"last":0.441},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.45900000000000002,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.218,"15min":1.218},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":1.1040000000000001,"15min":0.83299999999999996},"min":{"1min":0.47999999999999998,"5min":0.45000000000000001,"15min":0.443},"max":{"1min":0.86799999999999999,"5min":38.808,"15min":38.808},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":1.077,"15min":0.81200000000000006},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.40500000000000003},"max":{"1min":0.81299999999999994,"5min":38.823,"15min":38.823},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.46000000000000002,"15min":0.43099999999999999},"max":{"1min":0.94899999999999995,"5min":1.0449999999999999,"15min":1.0509999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.40000000000000002},"max":{"1min":0.91300000000000003,"5min":1.1559999999999999,"15min":1.1879999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.52500000000000002,"5min":0.49399999999999999,"15min":0.45300000000000001},"max":{"1min":0.98499999999999999,"5min":1.105,"15min":1.155},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.54900000000000004,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":0.93500000000000005,"5min":1.123,"15min":1.131},"last":0.58499999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542336,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1290240,"data_stored":709909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39600000000000002},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.57499999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45000000000000001}]},{"osd":2,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44800000000000001,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.28199999999999997},"max":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.66700000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.28799999999999998},"max":{"1min":0.61399999999999999,"5min":0.754,"15min":0.79400000000000004},"last":0.53700000000000003}]},{"osd":3,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.33000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54500000000000004,"5min":1.482,"15min":1.482},"last":0.372},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.496},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":0.872,"15min":0.872},"last":0.46100000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81499999999999995,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.64400000000000002}]},{"osd":5,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.875,"5min":1.655,"15min":1.655},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.42999999999999999},"max":{"1min":0.77200000000000002,"5min":0.94799999999999995,"15min":1.0840000000000001},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.875,"5min":0.90700000000000003,"15min":1.137},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":0.96499999999999997,"15min":1.117},"last":0.70499999999999996}]},{"osd":7,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47099999999999997},"max":{"1min":0.86899999999999999,"5min":0.96099999999999997,"15min":0.98199999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.49099999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.59399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575038,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306548,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94065292,"statfs":{"total":96636764160,"available":96322859008,"internally_reserved":0,"allocated":1290240,"data_stored":710014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.29499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.65800000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.24399999999999999},"max":{"1min":0.54500000000000004,"5min":0.65900000000000003,"15min":0.67100000000000004},"last":0.46500000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.32500000000000001,"5min":0.27800000000000002,"15min":0.27300000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.67200000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.255},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.55600000000000005}]},{"osd":3,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.45000000000000001},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64300000000000002,"5min":0.67700000000000005,"15min":0.71699999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.38900000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.72799999999999998},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.502,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85899999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.55400000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.92200000000000004,"5min":0.996,"15min":1.042},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.53400000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83599999999999997,"5min":1.036,"15min":1.0469999999999999},"last":0.622},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90000000000000002,"5min":1.107,"15min":1.107},"last":0.70199999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.82899999999999996,"5min":1.0569999999999999,"15min":1.1100000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.57299999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84699999999999998,"5min":1.141,"15min":1.141},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.52400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":0.96199999999999997,"15min":1.054},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.88900000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.81000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302460,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069380,"statfs":{"total":96636764160,"available":96327045120,"internally_reserved":0,"allocated":1298432,"data_stored":710658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45100000000000001,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.76400000000000001,"5min":7.593,"15min":7.593},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48299999999999998,"15min":0.437},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56200000000000006,"5min":8.4239999999999995,"15min":8.4239999999999995},"last":0.33900000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.69099999999999995},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.221,"15min":0.221},"max":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.68799999999999994},"last":0.47399999999999998}]},{"osd":2,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.496},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.66500000000000004,"5min":0.76300000000000001,"15min":0.77200000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.78800000000000003},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64500000000000002,"15min":0.63500000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.495,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":0.81599999999999995,"5min":1.976,"15min":1.976},"last":0.52200000000000002}]},{"osd":5,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.49299999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91500000000000004,"5min":1.53,"15min":1.53},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.97999999999999998,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.65700000000000003}]},{"osd":7,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72799999999999998,"15min":0.72099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.72999999999999998},"min":{"1min":0.495,"5min":0.49199999999999999,"15min":0.45800000000000002},"max":{"1min":0.93500000000000005,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.62}]}]},{"osd":4,"up_from":13,"seq":55834575039,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1290240,"data_stored":710498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78300000000000003,"5min":0.80100000000000005,"15min":0.88600000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.84399999999999997},"last":0.51800000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.83099999999999996,"15min":0.85299999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.755,"5min":0.79800000000000004,"15min":1.506},"last":0.65000000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.80200000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.503},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.61499999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72899999999999998,"5min":0.77800000000000002,"15min":0.80600000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.81000000000000005,"15min":0.86299999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.92000000000000004},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.496,"15min":0.5},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65700000000000003,"5min":0.72899999999999998,"15min":1.556},"last":0.46899999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.753,"5min":0.78500000000000003,"15min":0.86099999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76400000000000001,"5min":0.82099999999999995,"15min":1.0409999999999999},"last":0.45100000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.38500000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.88100000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.307,"15min":0.307},"max":{"1min":0.72699999999999998,"5min":0.81399999999999995,"15min":1.26},"last":0.626}]}]},{"osd":5,"up_from":13,"seq":55834575039,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302464,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069376,"statfs":{"total":96636764160,"available":96327041024,"internally_reserved":0,"allocated":1302528,"data_stored":711502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64400000000000002,"5min":0.85199999999999998,"15min":1.101},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.309,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71199999999999997,"5min":0.82699999999999996,"15min":1.1439999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56000000000000005,"15min":0.57699999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67200000000000004,"5min":0.84099999999999997,"15min":1.093},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69199999999999995,"5min":0.81299999999999994,"15min":1.1639999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.68400000000000005,"5min":0.84299999999999997,"15min":1.1100000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.35499999999999998},"max":{"1min":0.67400000000000004,"5min":0.83599999999999997,"15min":1.1220000000000001},"last":0.58299999999999996}]},{"osd":3,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.38500000000000001},"max":{"1min":0.72199999999999998,"5min":0.80500000000000005,"15min":1.139},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.496,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":1.0609999999999999},"last":0.52000000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70199999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.505,"15min":0.503},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.90000000000000002},"last":0.42399999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.32900000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":1.1839999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77400000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55100000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72999999999999998,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.33200000000000002},"max":{"1min":0.78200000000000003,"5min":0.95999999999999996,"15min":1.2030000000000001},"last":0.49299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:14:04.520 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-24T14:14:04.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.656+0000 7f1b999b1700 1 Processor -- start 2024-01-24T14:14:04.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.656+0000 7f1b999b1700 1 -- start start 2024-01-24T14:14:04.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.656+0000 7f1b999b1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b941546c0 0x7f1b940c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:04.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b999b1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b94158740 0x7f1b940c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:04.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b999b1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b940ca090 con 0x7f1b941542e0 2024-01-24T14:14:04.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b999b1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b940ca2d0 con 0x7f1b941546c0 2024-01-24T14:14:04.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b999b1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940ca5b0 con 0x7f1b94158740 2024-01-24T14:14:04.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b927fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b941546c0 0x7f1b940c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:04.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b937fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b94158740 0x7f1b940c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:04.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b927fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b941546c0 0x7f1b940c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:37244/0 (socket says 172.21.15.29:37244) 2024-01-24T14:14:04.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b927fc700 1 -- 172.21.15.29:0/276095190 learned_addr learned my addr 172.21.15.29:0/276095190 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:04.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b937fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b94158740 0x7f1b940c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:47732/0 (socket says 172.21.15.29:47732) 2024-01-24T14:14:04.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b937fe700 1 -- 172.21.15.29:0/276095190 >> v2:172.21.15.163:3300/0 conn(0x7f1b941546c0 msgr2=0x7f1b940c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:04.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b937fe700 1 --2- 172.21.15.29:0/276095190 >> v2:172.21.15.163:3300/0 conn(0x7f1b941546c0 0x7f1b940c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b937fe700 1 -- 172.21.15.29:0/276095190 >> v1:172.21.15.163:6789/0 conn(0x7f1b941542e0 legacy=0x7f1b94158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b937fe700 1 -- 172.21.15.29:0/276095190 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b940ca890 con 0x7f1b94158740 2024-01-24T14:14:04.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b927fc700 1 --2- 172.21.15.29:0/276095190 >> v2:172.21.15.163:3300/0 conn(0x7f1b941546c0 0x7f1b940c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:04.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.657+0000 7f1b937fe700 1 --2- 172.21.15.29:0/276095190 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b94158740 0x7f1b940c9ba0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f1b8c009e60 tx=0x7f1b8c009780).ready entity=mon.2 client_cookie=d3284206a36fd276 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:04.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.658+0000 7f1b91ffb700 1 -- 172.21.15.29:0/276095190 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c00e070 con 0x7f1b94158740 2024-01-24T14:14:04.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.658+0000 7f1b91ffb700 1 -- 172.21.15.29:0/276095190 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c00bb60 con 0x7f1b94158740 2024-01-24T14:14:04.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.658+0000 7f1b91ffb700 1 -- 172.21.15.29:0/276095190 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c012410 con 0x7f1b94158740 2024-01-24T14:14:04.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.658+0000 7f1b999b1700 1 -- 172.21.15.29:0/276095190 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b94158740 msgr2=0x7f1b940c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.658+0000 7f1b999b1700 1 --2- 172.21.15.29:0/276095190 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b94158740 0x7f1b940c9ba0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f1b8c009e60 tx=0x7f1b8c009780).stop 2024-01-24T14:14:04.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.658+0000 7f1b999b1700 1 -- 172.21.15.29:0/276095190 shutdown_connections 2024-01-24T14:14:04.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.658+0000 7f1b999b1700 1 --2- 172.21.15.29:0/276095190 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b94158740 0x7f1b940c9ba0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.658+0000 7f1b999b1700 1 --2- 172.21.15.29:0/276095190 >> v2:172.21.15.163:3300/0 conn(0x7f1b941546c0 0x7f1b940c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.658+0000 7f1b999b1700 1 -- 172.21.15.29:0/276095190 >> 172.21.15.29:0/276095190 conn(0x7f1b94094460 msgr2=0x7f1b9405e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:04.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.658+0000 7f1b999b1700 1 -- 172.21.15.29:0/276095190 shutdown_connections 2024-01-24T14:14:04.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.658+0000 7f1b999b1700 1 -- 172.21.15.29:0/276095190 wait complete. 2024-01-24T14:14:04.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.660+0000 7f1b999b1700 1 Processor -- start 2024-01-24T14:14:04.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.660+0000 7f1b999b1700 1 -- start start 2024-01-24T14:14:04.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.660+0000 7f1b999b1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b941542e0 0x7f1b94150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:04.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.660+0000 7f1b999b1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941546c0 0x7f1b940c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:04.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.660+0000 7f1b999b1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b94158740 0x7f1b940ab900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:04.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.660+0000 7f1b999b1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940ab470 con 0x7f1b941546c0 2024-01-24T14:14:04.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.660+0000 7f1b999b1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b941514e0 con 0x7f1b94158740 2024-01-24T14:14:04.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.660+0000 7f1b999b1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940ad460 con 0x7f1b941542e0 2024-01-24T14:14:04.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b927fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941546c0 0x7f1b940c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:04.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b927fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941546c0 0x7f1b940c22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59956/0 (socket says 172.21.15.29:59956) 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b927fc700 1 -- 172.21.15.29:0/4230430973 learned_addr learned my addr 172.21.15.29:0/4230430973 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b92ffd700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b941542e0 0x7f1b94150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b937fe700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b94158740 0x7f1b940ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b927fc700 1 -- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b941542e0 msgr2=0x7f1b94150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b927fc700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b941542e0 0x7f1b94150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b927fc700 1 -- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b94158740 msgr2=0x7f1b940ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b927fc700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b94158740 0x7f1b940ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b927fc700 1 -- 172.21.15.29:0/4230430973 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b88009370 con 0x7f1b941546c0 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b927fc700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941546c0 0x7f1b940c22d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f1b8800ea20 tx=0x7f1b880090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b7affd700 1 -- 172.21.15.29:0/4230430973 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800ee60 con 0x7f1b941546c0 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b999b1700 1 -- 172.21.15.29:0/4230430973 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8c009a20 con 0x7f1b941546c0 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b7affd700 1 -- 172.21.15.29:0/4230430973 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8800c850 con 0x7f1b941546c0 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b7affd700 1 -- 172.21.15.29:0/4230430973 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800cbd0 con 0x7f1b941546c0 2024-01-24T14:14:04.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.661+0000 7f1b999b1700 1 -- 172.21.15.29:0/4230430973 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b940ad180 con 0x7f1b941546c0 2024-01-24T14:14:04.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.662+0000 7f1b999b1700 1 -- 172.21.15.29:0/4230430973 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b940835e0 con 0x7f1b941546c0 2024-01-24T14:14:04.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.663+0000 7f1b7affd700 1 -- 172.21.15.29:0/4230430973 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1b8801b400 con 0x7f1b941546c0 2024-01-24T14:14:04.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.663+0000 7f1b7affd700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b8005f270 0x7f1b80061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:04.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.663+0000 7f1b7affd700 1 -- 172.21.15.29:0/4230430973 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f1b8807ad70 con 0x7f1b941546c0 2024-01-24T14:14:04.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.663+0000 7f1b92ffd700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b8005f270 0x7f1b80061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:04.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.664+0000 7f1b92ffd700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b8005f270 0x7f1b80061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f1b84007fd0 tx=0x7f1b84007d30).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:04.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.668+0000 7f1b7affd700 1 -- 172.21.15.29:0/4230430973 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1b8804bbd0 con 0x7f1b941546c0 2024-01-24T14:14:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.917+0000 7f1b999b1700 1 -- 172.21.15.29:0/4230430973 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1b940475e0 con 0x7f1b941546c0 2024-01-24T14:14:04.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.917+0000 7f1b7affd700 1 -- 172.21.15.29:0/4230430973 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+953 (secure 0 0 0) 0x7f1b88016080 con 0x7f1b941546c0 2024-01-24T14:14:04.918 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:04.918 INFO:teuthology.orchestra.run.smithi029.stdout:{"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":964,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":105,"num_osds":8,"num_up_osds":8,"osd_up_since":1706104690,"num_in_osds":8,"osd_in_since":1706104683,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":137}],"num_pgs":137,"num_pools":4,"num_objects":21,"data_bytes":2306,"bytes_used":2481909760,"bytes_avail":770612203520,"bytes_total":773094113280},"fsmap":{"epoch":43,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":6807}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-01-24T14:12:14.050791+0000","services":{}},"progress_events":{}} 2024-01-24T14:14:04.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.919+0000 7f1b999b1700 1 -- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b8005f270 msgr2=0x7f1b80061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.919+0000 7f1b999b1700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b8005f270 0x7f1b80061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f1b84007fd0 tx=0x7f1b84007d30).stop 2024-01-24T14:14:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.919+0000 7f1b999b1700 1 -- 172.21.15.29:0/4230430973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941546c0 msgr2=0x7f1b940c22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.919+0000 7f1b999b1700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941546c0 0x7f1b940c22d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f1b8800ea20 tx=0x7f1b880090f0).stop 2024-01-24T14:14:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.919+0000 7f1b999b1700 1 -- 172.21.15.29:0/4230430973 shutdown_connections 2024-01-24T14:14:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.919+0000 7f1b999b1700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b94158740 0x7f1b940ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.919+0000 7f1b999b1700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b941542e0 0x7f1b94150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.919+0000 7f1b999b1700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1b8005f270 0x7f1b80061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.919+0000 7f1b999b1700 1 --2- 172.21.15.29:0/4230430973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941546c0 0x7f1b940c22d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.919+0000 7f1b999b1700 1 -- 172.21.15.29:0/4230430973 >> 172.21.15.29:0/4230430973 conn(0x7f1b94094460 msgr2=0x7f1b940c97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.919+0000 7f1b999b1700 1 -- 172.21.15.29:0/4230430973 shutdown_connections 2024-01-24T14:14:04.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:04.919+0000 7f1b999b1700 1 -- 172.21.15.29:0/4230430973 wait complete. 2024-01-24T14:14:04.933 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-24T14:14:04.933 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:14:05.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.063+0000 7fd18f47e700 1 Processor -- start 2024-01-24T14:14:05.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.063+0000 7fd18f47e700 1 -- start start 2024-01-24T14:14:05.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.064+0000 7fd18f47e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd188066b40 0x7fd188062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:05.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.064+0000 7fd18f47e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880bcf60 con 0x7fd188062970 2024-01-24T14:14:05.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.064+0000 7fd18f47e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880a6ec0 con 0x7fd188158600 2024-01-24T14:14:05.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.064+0000 7fd18f47e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880bdba0 con 0x7fd188066b40 2024-01-24T14:14:05.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.064+0000 7fd18d21a700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd188062970 0x7fd1880c4da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39596/0 (socket says 172.21.15.29:39596) 2024-01-24T14:14:05.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.064+0000 7fd18d21a700 1 -- 172.21.15.29:0/3599343814 learned_addr learned my addr 172.21.15.29:0/3599343814 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:05.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.064+0000 7fd18ca19700 1 --2- 172.21.15.29:0/3599343814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd188066b40 0x7fd188062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.064+0000 7fd17ffff700 1 -- 172.21.15.29:0/3599343814 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091460498 0 0) 0x7fd1880bcf60 con 0x7fd188062970 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.064+0000 7fd17ffff700 1 -- 172.21.15.29:0/3599343814 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1740033f0 con 0x7fd188062970 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.065+0000 7fd18ca19700 1 -- 172.21.15.29:0/3599343814 >> v1:172.21.15.163:6789/0 conn(0x7fd188158600 legacy=0x7fd188154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.065+0000 7fd17ffff700 1 -- 172.21.15.29:0/3599343814 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591489220 0 0) 0x7fd1880a6ec0 con 0x7fd188158600 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.065+0000 7fd18ca19700 1 -- 172.21.15.29:0/3599343814 >> v1:172.21.15.29:6789/0 conn(0x7fd188062970 legacy=0x7fd1880c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.065+0000 7fd18ca19700 1 -- 172.21.15.29:0/3599343814 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1880a71a0 con 0x7fd188066b40 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.065+0000 7fd18ca19700 1 --2- 172.21.15.29:0/3599343814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd188066b40 0x7fd188062d50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fd178009790 tx=0x7fd178009060).ready entity=mon.2 client_cookie=c11ffba2b1063b0f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.065+0000 7fd17ffff700 1 -- 172.21.15.29:0/3599343814 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184001470 con 0x7fd188066b40 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.065+0000 7fd17ffff700 1 -- 172.21.15.29:0/3599343814 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd178012410 con 0x7fd188066b40 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.065+0000 7fd17ffff700 1 -- 172.21.15.29:0/3599343814 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178012790 con 0x7fd188066b40 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.065+0000 7fd18f47e700 1 -- 172.21.15.29:0/3599343814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd188066b40 msgr2=0x7fd188062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.065+0000 7fd18f47e700 1 --2- 172.21.15.29:0/3599343814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd188066b40 0x7fd188062d50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fd178009790 tx=0x7fd178009060).stop 2024-01-24T14:14:05.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.065+0000 7fd18f47e700 1 -- 172.21.15.29:0/3599343814 shutdown_connections 2024-01-24T14:14:05.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.065+0000 7fd18f47e700 1 --2- 172.21.15.29:0/3599343814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd188066b40 0x7fd188062d50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:05.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.066+0000 7fd18f47e700 1 -- 172.21.15.29:0/3599343814 >> 172.21.15.29:0/3599343814 conn(0x7fd1880c8580 msgr2=0x7fd1880c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:05.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.066+0000 7fd18f47e700 1 -- 172.21.15.29:0/3599343814 shutdown_connections 2024-01-24T14:14:05.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.066+0000 7fd18f47e700 1 -- 172.21.15.29:0/3599343814 wait complete. 2024-01-24T14:14:05.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.067+0000 7fd18f47e700 1 Processor -- start 2024-01-24T14:14:05.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.067+0000 7fd18f47e700 1 -- start start 2024-01-24T14:14:05.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18f47e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd188062970 0x7fd1881567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:05.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18f47e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd188066b40 0x7fd1880aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18f47e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd188158600 0x7fd1880aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18f47e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880a7480 con 0x7fd188158600 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18f47e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880ab480 con 0x7fd188062970 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18f47e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1881579b0 con 0x7fd188066b40 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18da1b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd188158600 0x7fd1880aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18da1b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd188158600 0x7fd1880aaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59958/0 (socket says 172.21.15.29:59958) 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18da1b700 1 -- 172.21.15.29:0/2907228 learned_addr learned my addr 172.21.15.29:0/2907228 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18ca19700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd188066b40 0x7fd1880aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18d21a700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd188062970 0x7fd1881567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18da1b700 1 -- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd188066b40 msgr2=0x7fd1880aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18da1b700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd188066b40 0x7fd1880aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18da1b700 1 -- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd188062970 msgr2=0x7fd1881567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18da1b700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd188062970 0x7fd1881567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18da1b700 1 -- 172.21.15.29:0/2907228 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780092a0 con 0x7fd188158600 2024-01-24T14:14:05.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18d21a700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd188062970 0x7fd1881567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:05.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.068+0000 7fd18da1b700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd188158600 0x7fd1880aaf90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7fd18000c810 tx=0x7fd180001500).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:05.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.069+0000 7fd17d7fa700 1 -- 172.21.15.29:0/2907228 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180010510 con 0x7fd188158600 2024-01-24T14:14:05.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.069+0000 7fd17d7fa700 1 -- 172.21.15.29:0/2907228 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180010670 con 0x7fd188158600 2024-01-24T14:14:05.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.069+0000 7fd18f47e700 1 -- 172.21.15.29:0/2907228 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd188156da0 con 0x7fd188158600 2024-01-24T14:14:05.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.069+0000 7fd17d7fa700 1 -- 172.21.15.29:0/2907228 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1800109f0 con 0x7fd188158600 2024-01-24T14:14:05.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.069+0000 7fd18f47e700 1 -- 172.21.15.29:0/2907228 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd188157360 con 0x7fd188158600 2024-01-24T14:14:05.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.070+0000 7fd18f47e700 1 -- 172.21.15.29:0/2907228 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1881553f0 con 0x7fd188158600 2024-01-24T14:14:05.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.070+0000 7fd17d7fa700 1 -- 172.21.15.29:0/2907228 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd18001b400 con 0x7fd188158600 2024-01-24T14:14:05.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.071+0000 7fd17d7fa700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd16c05f240 0x7fd16c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:05.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.071+0000 7fd17d7fa700 1 -- 172.21.15.29:0/2907228 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fd18007a610 con 0x7fd188158600 2024-01-24T14:14:05.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.071+0000 7fd18d21a700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd16c05f240 0x7fd16c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:05.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.071+0000 7fd18d21a700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd16c05f240 0x7fd16c061700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fd1840011e0 tx=0x7fd184000bc0).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:05.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.075+0000 7fd17d7fa700 1 -- 172.21.15.29:0/2907228 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd18004b240 con 0x7fd188158600 2024-01-24T14:14:05.144 INFO:tasks.ceph.mds.a.smithi029.stderr:2024-01-24T14:14:05.143+0000 7fa2fdff7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T14:14:05.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.252+0000 7fd18f47e700 1 -- 172.21.15.29:0/2907228 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1880c7930 con 0x7fd16c05f240 2024-01-24T14:14:05.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.263+0000 7fd17d7fa700 1 -- 172.21.15.29:0/2907228 <== mgr.5864 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361939 (secure 0 0 0) 0x7fd1880c7930 con 0x7fd16c05f240 2024-01-24T14:14:05.268 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:05.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.266+0000 7fd18f47e700 1 -- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd16c05f240 msgr2=0x7fd16c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:05.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.266+0000 7fd18f47e700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd16c05f240 0x7fd16c061700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fd1840011e0 tx=0x7fd184000bc0).stop 2024-01-24T14:14:05.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.266+0000 7fd18f47e700 1 -- 172.21.15.29:0/2907228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd188158600 msgr2=0x7fd1880aaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:05.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.266+0000 7fd18f47e700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd188158600 0x7fd1880aaf90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7fd18000c810 tx=0x7fd180001500).stop 2024-01-24T14:14:05.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.266+0000 7fd18f47e700 1 -- 172.21.15.29:0/2907228 shutdown_connections 2024-01-24T14:14:05.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.266+0000 7fd18f47e700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd16c05f240 0x7fd16c061700 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:05.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.266+0000 7fd18f47e700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd188158600 0x7fd1880aaf90 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.266+0000 7fd18f47e700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd188062970 0x7fd1881567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.266+0000 7fd18f47e700 1 --2- 172.21.15.29:0/2907228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd188066b40 0x7fd1880aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.266+0000 7fd18f47e700 1 -- 172.21.15.29:0/2907228 >> 172.21.15.29:0/2907228 conn(0x7fd1880c8580 msgr2=0x7fd188154ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.267+0000 7fd18f47e700 1 -- 172.21.15.29:0/2907228 shutdown_connections 2024-01-24T14:14:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:05.267+0000 7fd18f47e700 1 -- 172.21.15.29:0/2907228 wait complete. 2024-01-24T14:14:05.272 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:14:05.281 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-01-24T14:14:04.107428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":228,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2423740,"kb_used_data":10108,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752550980,"statfs":{"total":773094113280,"available":770612203520,"internally_reserved":0,"allocated":10350592,"data_stored":5683203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11128,"num_objects":-9,"num_object_clones":0,"num_object_copies":-18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-36,"num_write_kb":-28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000729"},"pg_stats":[{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615363+0000","last_change":"2024-01-24T14:13:59.610904+0000","last_active":"2024-01-24T14:13:59.615363+0000","last_peered":"2024-01-24T14:13:59.615363+0000","last_clean":"2024-01-24T14:13:59.615363+0000","last_became_active":"2024-01-24T14:13:59.610672+0000","last_became_peered":"2024-01-24T14:13:59.610672+0000","last_unstale":"2024-01-24T14:13:59.615363+0000","last_undegraded":"2024-01-24T14:13:59.615363+0000","last_fullsized":"2024-01-24T14:13:59.615363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3f","version":"104'3","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616800+0000","last_change":"2024-01-24T14:13:59.174978+0000","last_active":"2024-01-24T14:13:59.616800+0000","last_peered":"2024-01-24T14:13:59.616800+0000","last_clean":"2024-01-24T14:13:59.616800+0000","last_became_active":"2024-01-24T14:13:59.174684+0000","last_became_peered":"2024-01-24T14:13:59.174684+0000","last_unstale":"2024-01-24T14:13:59.616800+0000","last_undegraded":"2024-01-24T14:13:59.616800+0000","last_fullsized":"2024-01-24T14:13:59.616800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605090+0000","last_change":"2024-01-24T14:13:59.600282+0000","last_active":"2024-01-24T14:13:59.605090+0000","last_peered":"2024-01-24T14:13:59.605090+0000","last_clean":"2024-01-24T14:13:59.605090+0000","last_became_active":"2024-01-24T14:13:59.600004+0000","last_became_peered":"2024-01-24T14:13:59.600004+0000","last_unstale":"2024-01-24T14:13:59.605090+0000","last_undegraded":"2024-01-24T14:13:59.605090+0000","last_fullsized":"2024-01-24T14:13:59.605090+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.600650+0000","last_change":"2024-01-24T14:13:59.171713+0000","last_active":"2024-01-24T14:13:59.600650+0000","last_peered":"2024-01-24T14:13:59.600650+0000","last_clean":"2024-01-24T14:13:59.600650+0000","last_became_active":"2024-01-24T14:13:59.171407+0000","last_became_peered":"2024-01-24T14:13:59.171407+0000","last_unstale":"2024-01-24T14:13:59.600650+0000","last_undegraded":"2024-01-24T14:13:59.600650+0000","last_fullsized":"2024-01-24T14:13:59.600650+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599212+0000","last_change":"2024-01-24T14:13:59.596129+0000","last_active":"2024-01-24T14:13:59.599212+0000","last_peered":"2024-01-24T14:13:59.599212+0000","last_clean":"2024-01-24T14:13:59.599212+0000","last_became_active":"2024-01-24T14:13:59.595830+0000","last_became_peered":"2024-01-24T14:13:59.595830+0000","last_unstale":"2024-01-24T14:13:59.599212+0000","last_undegraded":"2024-01-24T14:13:59.599212+0000","last_fullsized":"2024-01-24T14:13:59.599212+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591337+0000","last_change":"2024-01-24T14:13:59.190558+0000","last_active":"2024-01-24T14:13:59.591337+0000","last_peered":"2024-01-24T14:13:59.591337+0000","last_clean":"2024-01-24T14:13:59.591337+0000","last_became_active":"2024-01-24T14:13:59.187620+0000","last_became_peered":"2024-01-24T14:13:59.187620+0000","last_unstale":"2024-01-24T14:13:59.591337+0000","last_undegraded":"2024-01-24T14:13:59.591337+0000","last_fullsized":"2024-01-24T14:13:59.591337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616684+0000","last_change":"2024-01-24T14:13:59.609750+0000","last_active":"2024-01-24T14:13:59.616684+0000","last_peered":"2024-01-24T14:13:59.616684+0000","last_clean":"2024-01-24T14:13:59.616684+0000","last_became_active":"2024-01-24T14:13:59.609455+0000","last_became_peered":"2024-01-24T14:13:59.609455+0000","last_unstale":"2024-01-24T14:13:59.616684+0000","last_undegraded":"2024-01-24T14:13:59.616684+0000","last_fullsized":"2024-01-24T14:13:59.616684+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3c","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599540+0000","last_change":"2024-01-24T14:13:59.184784+0000","last_active":"2024-01-24T14:13:59.599540+0000","last_peered":"2024-01-24T14:13:59.599540+0000","last_clean":"2024-01-24T14:13:59.599540+0000","last_became_active":"2024-01-24T14:13:59.184482+0000","last_became_peered":"2024-01-24T14:13:59.184482+0000","last_unstale":"2024-01-24T14:13:59.599540+0000","last_undegraded":"2024-01-24T14:13:59.599540+0000","last_fullsized":"2024-01-24T14:13:59.599540+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602293+0000","last_change":"2024-01-24T14:13:59.596092+0000","last_active":"2024-01-24T14:13:59.602293+0000","last_peered":"2024-01-24T14:13:59.602293+0000","last_clean":"2024-01-24T14:13:59.602293+0000","last_became_active":"2024-01-24T14:13:59.595791+0000","last_became_peered":"2024-01-24T14:13:59.595791+0000","last_unstale":"2024-01-24T14:13:59.602293+0000","last_undegraded":"2024-01-24T14:13:59.602293+0000","last_fullsized":"2024-01-24T14:13:59.602293+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593252+0000","last_change":"2024-01-24T14:13:59.173164+0000","last_active":"2024-01-24T14:13:59.593252+0000","last_peered":"2024-01-24T14:13:59.593252+0000","last_clean":"2024-01-24T14:13:59.593252+0000","last_became_active":"2024-01-24T14:13:59.172816+0000","last_became_peered":"2024-01-24T14:13:59.172816+0000","last_unstale":"2024-01-24T14:13:59.593252+0000","last_undegraded":"2024-01-24T14:13:59.593252+0000","last_fullsized":"2024-01-24T14:13:59.593252+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602472+0000","last_change":"2024-01-24T14:13:59.599516+0000","last_active":"2024-01-24T14:13:59.602472+0000","last_peered":"2024-01-24T14:13:59.602472+0000","last_clean":"2024-01-24T14:13:59.602472+0000","last_became_active":"2024-01-24T14:13:59.599234+0000","last_became_peered":"2024-01-24T14:13:59.599234+0000","last_unstale":"2024-01-24T14:13:59.602472+0000","last_undegraded":"2024-01-24T14:13:59.602472+0000","last_fullsized":"2024-01-24T14:13:59.602472+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.3a","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.611338+0000","last_change":"2024-01-24T14:13:59.180992+0000","last_active":"2024-01-24T14:13:59.611338+0000","last_peered":"2024-01-24T14:13:59.611338+0000","last_clean":"2024-01-24T14:13:59.611338+0000","last_became_active":"2024-01-24T14:13:59.180627+0000","last_became_peered":"2024-01-24T14:13:59.180627+0000","last_unstale":"2024-01-24T14:13:59.611338+0000","last_undegraded":"2024-01-24T14:13:59.611338+0000","last_fullsized":"2024-01-24T14:13:59.611338+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602256+0000","last_change":"2024-01-24T14:13:59.596465+0000","last_active":"2024-01-24T14:13:59.602256+0000","last_peered":"2024-01-24T14:13:59.602256+0000","last_clean":"2024-01-24T14:13:59.602256+0000","last_became_active":"2024-01-24T14:13:59.596185+0000","last_became_peered":"2024-01-24T14:13:59.596185+0000","last_unstale":"2024-01-24T14:13:59.602256+0000","last_undegraded":"2024-01-24T14:13:59.602256+0000","last_fullsized":"2024-01-24T14:13:59.602256+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.590664+0000","last_change":"2024-01-24T14:13:59.179164+0000","last_active":"2024-01-24T14:13:59.590664+0000","last_peered":"2024-01-24T14:13:59.590664+0000","last_clean":"2024-01-24T14:13:59.590664+0000","last_became_active":"2024-01-24T14:13:59.178543+0000","last_became_peered":"2024-01-24T14:13:59.178543+0000","last_unstale":"2024-01-24T14:13:59.590664+0000","last_undegraded":"2024-01-24T14:13:59.590664+0000","last_fullsized":"2024-01-24T14:13:59.590664+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612433+0000","last_change":"2024-01-24T14:13:59.603989+0000","last_active":"2024-01-24T14:13:59.612433+0000","last_peered":"2024-01-24T14:13:59.612433+0000","last_clean":"2024-01-24T14:13:59.612433+0000","last_became_active":"2024-01-24T14:13:59.603818+0000","last_became_peered":"2024-01-24T14:13:59.603818+0000","last_unstale":"2024-01-24T14:13:59.612433+0000","last_undegraded":"2024-01-24T14:13:59.612433+0000","last_fullsized":"2024-01-24T14:13:59.612433+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581206+0000","last_change":"2024-01-24T14:13:59.168769+0000","last_active":"2024-01-24T14:13:59.581206+0000","last_peered":"2024-01-24T14:13:59.581206+0000","last_clean":"2024-01-24T14:13:59.581206+0000","last_became_active":"2024-01-24T14:13:59.168470+0000","last_became_peered":"2024-01-24T14:13:59.168470+0000","last_unstale":"2024-01-24T14:13:59.581206+0000","last_undegraded":"2024-01-24T14:13:59.581206+0000","last_fullsized":"2024-01-24T14:13:59.581206+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613168+0000","last_change":"2024-01-24T14:13:59.605311+0000","last_active":"2024-01-24T14:13:59.613168+0000","last_peered":"2024-01-24T14:13:59.613168+0000","last_clean":"2024-01-24T14:13:59.613168+0000","last_became_active":"2024-01-24T14:13:59.604996+0000","last_became_peered":"2024-01-24T14:13:59.604996+0000","last_unstale":"2024-01-24T14:13:59.613168+0000","last_undegraded":"2024-01-24T14:13:59.613168+0000","last_fullsized":"2024-01-24T14:13:59.613168+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588758+0000","last_change":"2024-01-24T14:13:59.173631+0000","last_active":"2024-01-24T14:13:59.588758+0000","last_peered":"2024-01-24T14:13:59.588758+0000","last_clean":"2024-01-24T14:13:59.588758+0000","last_became_active":"2024-01-24T14:13:59.173365+0000","last_became_peered":"2024-01-24T14:13:59.173365+0000","last_unstale":"2024-01-24T14:13:59.588758+0000","last_undegraded":"2024-01-24T14:13:59.588758+0000","last_fullsized":"2024-01-24T14:13:59.588758+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602306+0000","last_change":"2024-01-24T14:13:59.597783+0000","last_active":"2024-01-24T14:13:59.602306+0000","last_peered":"2024-01-24T14:13:59.602306+0000","last_clean":"2024-01-24T14:13:59.602306+0000","last_became_active":"2024-01-24T14:13:59.597507+0000","last_became_peered":"2024-01-24T14:13:59.597507+0000","last_unstale":"2024-01-24T14:13:59.602306+0000","last_undegraded":"2024-01-24T14:13:59.602306+0000","last_fullsized":"2024-01-24T14:13:59.602306+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585624+0000","last_change":"2024-01-24T14:13:59.185609+0000","last_active":"2024-01-24T14:13:59.585624+0000","last_peered":"2024-01-24T14:13:59.585624+0000","last_clean":"2024-01-24T14:13:59.585624+0000","last_became_active":"2024-01-24T14:13:59.185272+0000","last_became_peered":"2024-01-24T14:13:59.185272+0000","last_unstale":"2024-01-24T14:13:59.585624+0000","last_undegraded":"2024-01-24T14:13:59.585624+0000","last_fullsized":"2024-01-24T14:13:59.585624+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613203+0000","last_change":"2024-01-24T14:13:59.607137+0000","last_active":"2024-01-24T14:13:59.613203+0000","last_peered":"2024-01-24T14:13:59.613203+0000","last_clean":"2024-01-24T14:13:59.613203+0000","last_became_active":"2024-01-24T14:13:59.606837+0000","last_became_peered":"2024-01-24T14:13:59.606837+0000","last_unstale":"2024-01-24T14:13:59.613203+0000","last_undegraded":"2024-01-24T14:13:59.613203+0000","last_fullsized":"2024-01-24T14:13:59.613203+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597239+0000","last_change":"2024-01-24T14:13:59.174329+0000","last_active":"2024-01-24T14:13:59.597239+0000","last_peered":"2024-01-24T14:13:59.597239+0000","last_clean":"2024-01-24T14:13:59.597239+0000","last_became_active":"2024-01-24T14:13:59.173356+0000","last_became_peered":"2024-01-24T14:13:59.173356+0000","last_unstale":"2024-01-24T14:13:59.597239+0000","last_undegraded":"2024-01-24T14:13:59.597239+0000","last_fullsized":"2024-01-24T14:13:59.597239+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.614739+0000","last_change":"2024-01-24T14:13:59.602824+0000","last_active":"2024-01-24T14:13:59.614739+0000","last_peered":"2024-01-24T14:13:59.614739+0000","last_clean":"2024-01-24T14:13:59.614739+0000","last_became_active":"2024-01-24T14:13:59.602330+0000","last_became_peered":"2024-01-24T14:13:59.602330+0000","last_unstale":"2024-01-24T14:13:59.614739+0000","last_undegraded":"2024-01-24T14:13:59.614739+0000","last_fullsized":"2024-01-24T14:13:59.614739+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616384+0000","last_change":"2024-01-24T14:13:59.191263+0000","last_active":"2024-01-24T14:13:59.616384+0000","last_peered":"2024-01-24T14:13:59.616384+0000","last_clean":"2024-01-24T14:13:59.616384+0000","last_became_active":"2024-01-24T14:13:59.191014+0000","last_became_peered":"2024-01-24T14:13:59.191014+0000","last_unstale":"2024-01-24T14:13:59.616384+0000","last_undegraded":"2024-01-24T14:13:59.616384+0000","last_fullsized":"2024-01-24T14:13:59.616384+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612022+0000","last_change":"2024-01-24T14:13:59.603458+0000","last_active":"2024-01-24T14:13:59.612022+0000","last_peered":"2024-01-24T14:13:59.612022+0000","last_clean":"2024-01-24T14:13:59.612022+0000","last_became_active":"2024-01-24T14:13:59.600085+0000","last_became_peered":"2024-01-24T14:13:59.600085+0000","last_unstale":"2024-01-24T14:13:59.612022+0000","last_undegraded":"2024-01-24T14:13:59.612022+0000","last_fullsized":"2024-01-24T14:13:59.612022+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604880+0000","last_change":"2024-01-24T14:13:59.186304+0000","last_active":"2024-01-24T14:13:59.604880+0000","last_peered":"2024-01-24T14:13:59.604880+0000","last_clean":"2024-01-24T14:13:59.604880+0000","last_became_active":"2024-01-24T14:13:59.186017+0000","last_became_peered":"2024-01-24T14:13:59.186017+0000","last_unstale":"2024-01-24T14:13:59.604880+0000","last_undegraded":"2024-01-24T14:13:59.604880+0000","last_fullsized":"2024-01-24T14:13:59.604880+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613474+0000","last_change":"2024-01-24T14:13:59.603461+0000","last_active":"2024-01-24T14:13:59.613474+0000","last_peered":"2024-01-24T14:13:59.613474+0000","last_clean":"2024-01-24T14:13:59.613474+0000","last_became_active":"2024-01-24T14:13:59.603177+0000","last_became_peered":"2024-01-24T14:13:59.603177+0000","last_unstale":"2024-01-24T14:13:59.613474+0000","last_undegraded":"2024-01-24T14:13:59.613474+0000","last_fullsized":"2024-01-24T14:13:59.613474+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585657+0000","last_change":"2024-01-24T14:13:59.181115+0000","last_active":"2024-01-24T14:13:59.585657+0000","last_peered":"2024-01-24T14:13:59.585657+0000","last_clean":"2024-01-24T14:13:59.585657+0000","last_became_active":"2024-01-24T14:13:59.180641+0000","last_became_peered":"2024-01-24T14:13:59.180641+0000","last_unstale":"2024-01-24T14:13:59.585657+0000","last_undegraded":"2024-01-24T14:13:59.585657+0000","last_fullsized":"2024-01-24T14:13:59.585657+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.606266+0000","last_change":"2024-01-24T14:13:59.602768+0000","last_active":"2024-01-24T14:13:59.606266+0000","last_peered":"2024-01-24T14:13:59.606266+0000","last_clean":"2024-01-24T14:13:59.606266+0000","last_became_active":"2024-01-24T14:13:59.602510+0000","last_became_peered":"2024-01-24T14:13:59.602510+0000","last_unstale":"2024-01-24T14:13:59.606266+0000","last_undegraded":"2024-01-24T14:13:59.606266+0000","last_fullsized":"2024-01-24T14:13:59.606266+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596930+0000","last_change":"2024-01-24T14:13:59.188810+0000","last_active":"2024-01-24T14:13:59.596930+0000","last_peered":"2024-01-24T14:13:59.596930+0000","last_clean":"2024-01-24T14:13:59.596930+0000","last_became_active":"2024-01-24T14:13:59.188475+0000","last_became_peered":"2024-01-24T14:13:59.188475+0000","last_unstale":"2024-01-24T14:13:59.596930+0000","last_undegraded":"2024-01-24T14:13:59.596930+0000","last_fullsized":"2024-01-24T14:13:59.596930+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.608903+0000","last_change":"2024-01-24T14:13:59.604713+0000","last_active":"2024-01-24T14:13:59.608903+0000","last_peered":"2024-01-24T14:13:59.608903+0000","last_clean":"2024-01-24T14:13:59.608903+0000","last_became_active":"2024-01-24T14:13:59.604382+0000","last_became_peered":"2024-01-24T14:13:59.604382+0000","last_unstale":"2024-01-24T14:13:59.608903+0000","last_undegraded":"2024-01-24T14:13:59.608903+0000","last_fullsized":"2024-01-24T14:13:59.608903+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.30","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616811+0000","last_change":"2024-01-24T14:13:59.176815+0000","last_active":"2024-01-24T14:13:59.616811+0000","last_peered":"2024-01-24T14:13:59.616811+0000","last_clean":"2024-01-24T14:13:59.616811+0000","last_became_active":"2024-01-24T14:13:59.176478+0000","last_became_peered":"2024-01-24T14:13:59.176478+0000","last_unstale":"2024-01-24T14:13:59.616811+0000","last_undegraded":"2024-01-24T14:13:59.616811+0000","last_fullsized":"2024-01-24T14:13:59.616811+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612937+0000","last_change":"2024-01-24T14:13:59.603476+0000","last_active":"2024-01-24T14:13:59.612937+0000","last_peered":"2024-01-24T14:13:59.612937+0000","last_clean":"2024-01-24T14:13:59.612937+0000","last_became_active":"2024-01-24T14:13:59.603267+0000","last_became_peered":"2024-01-24T14:13:59.603267+0000","last_unstale":"2024-01-24T14:13:59.612937+0000","last_undegraded":"2024-01-24T14:13:59.612937+0000","last_fullsized":"2024-01-24T14:13:59.612937+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.583071+0000","last_change":"2024-01-24T14:13:59.174673+0000","last_active":"2024-01-24T14:13:59.583071+0000","last_peered":"2024-01-24T14:13:59.583071+0000","last_clean":"2024-01-24T14:13:59.583071+0000","last_became_active":"2024-01-24T14:13:59.174365+0000","last_became_peered":"2024-01-24T14:13:59.174365+0000","last_unstale":"2024-01-24T14:13:59.583071+0000","last_undegraded":"2024-01-24T14:13:59.583071+0000","last_fullsized":"2024-01-24T14:13:59.583071+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.603045+0000","last_change":"2024-01-24T14:13:59.598989+0000","last_active":"2024-01-24T14:13:59.603045+0000","last_peered":"2024-01-24T14:13:59.603045+0000","last_clean":"2024-01-24T14:13:59.603045+0000","last_became_active":"2024-01-24T14:13:59.598687+0000","last_became_peered":"2024-01-24T14:13:59.598687+0000","last_unstale":"2024-01-24T14:13:59.603045+0000","last_undegraded":"2024-01-24T14:13:59.603045+0000","last_fullsized":"2024-01-24T14:13:59.603045+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591250+0000","last_change":"2024-01-24T14:13:59.184862+0000","last_active":"2024-01-24T14:13:59.591250+0000","last_peered":"2024-01-24T14:13:59.591250+0000","last_clean":"2024-01-24T14:13:59.591250+0000","last_became_active":"2024-01-24T14:13:59.184560+0000","last_became_peered":"2024-01-24T14:13:59.184560+0000","last_unstale":"2024-01-24T14:13:59.591250+0000","last_undegraded":"2024-01-24T14:13:59.591250+0000","last_fullsized":"2024-01-24T14:13:59.591250+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599235+0000","last_change":"2024-01-24T14:13:59.597440+0000","last_active":"2024-01-24T14:13:59.599235+0000","last_peered":"2024-01-24T14:13:59.599235+0000","last_clean":"2024-01-24T14:13:59.599235+0000","last_became_active":"2024-01-24T14:13:59.597132+0000","last_became_peered":"2024-01-24T14:13:59.597132+0000","last_unstale":"2024-01-24T14:13:59.599235+0000","last_undegraded":"2024-01-24T14:13:59.599235+0000","last_fullsized":"2024-01-24T14:13:59.599235+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2d","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612688+0000","last_change":"2024-01-24T14:13:59.177820+0000","last_active":"2024-01-24T14:13:59.612688+0000","last_peered":"2024-01-24T14:13:59.612688+0000","last_clean":"2024-01-24T14:13:59.612688+0000","last_became_active":"2024-01-24T14:13:59.177501+0000","last_became_peered":"2024-01-24T14:13:59.177501+0000","last_unstale":"2024-01-24T14:13:59.612688+0000","last_undegraded":"2024-01-24T14:13:59.612688+0000","last_fullsized":"2024-01-24T14:13:59.612688+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613916+0000","last_change":"2024-01-24T14:13:59.607979+0000","last_active":"2024-01-24T14:13:59.613916+0000","last_peered":"2024-01-24T14:13:59.613916+0000","last_clean":"2024-01-24T14:13:59.613916+0000","last_became_active":"2024-01-24T14:13:59.607684+0000","last_became_peered":"2024-01-24T14:13:59.607684+0000","last_unstale":"2024-01-24T14:13:59.613916+0000","last_undegraded":"2024-01-24T14:13:59.613916+0000","last_fullsized":"2024-01-24T14:13:59.613916+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.582496+0000","last_change":"2024-01-24T14:13:59.185072+0000","last_active":"2024-01-24T14:13:59.582496+0000","last_peered":"2024-01-24T14:13:59.582496+0000","last_clean":"2024-01-24T14:13:59.582496+0000","last_became_active":"2024-01-24T14:13:59.184040+0000","last_became_peered":"2024-01-24T14:13:59.184040+0000","last_unstale":"2024-01-24T14:13:59.582496+0000","last_undegraded":"2024-01-24T14:13:59.582496+0000","last_fullsized":"2024-01-24T14:13:59.582496+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599036+0000","last_change":"2024-01-24T14:13:59.597766+0000","last_active":"2024-01-24T14:13:59.599036+0000","last_peered":"2024-01-24T14:13:59.599036+0000","last_clean":"2024-01-24T14:13:59.599036+0000","last_became_active":"2024-01-24T14:13:59.597476+0000","last_became_peered":"2024-01-24T14:13:59.597476+0000","last_unstale":"2024-01-24T14:13:59.599036+0000","last_undegraded":"2024-01-24T14:13:59.599036+0000","last_fullsized":"2024-01-24T14:13:59.599036+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599653+0000","last_change":"2024-01-24T14:13:59.181186+0000","last_active":"2024-01-24T14:13:59.599653+0000","last_peered":"2024-01-24T14:13:59.599653+0000","last_clean":"2024-01-24T14:13:59.599653+0000","last_became_active":"2024-01-24T14:13:59.180838+0000","last_became_peered":"2024-01-24T14:13:59.180838+0000","last_unstale":"2024-01-24T14:13:59.599653+0000","last_undegraded":"2024-01-24T14:13:59.599653+0000","last_fullsized":"2024-01-24T14:13:59.599653+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602220+0000","last_change":"2024-01-24T14:13:59.598055+0000","last_active":"2024-01-24T14:13:59.602220+0000","last_peered":"2024-01-24T14:13:59.602220+0000","last_clean":"2024-01-24T14:13:59.602220+0000","last_became_active":"2024-01-24T14:13:59.597777+0000","last_became_peered":"2024-01-24T14:13:59.597777+0000","last_unstale":"2024-01-24T14:13:59.602220+0000","last_undegraded":"2024-01-24T14:13:59.602220+0000","last_fullsized":"2024-01-24T14:13:59.602220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581576+0000","last_change":"2024-01-24T14:13:59.185733+0000","last_active":"2024-01-24T14:13:59.581576+0000","last_peered":"2024-01-24T14:13:59.581576+0000","last_clean":"2024-01-24T14:13:59.581576+0000","last_became_active":"2024-01-24T14:13:59.185482+0000","last_became_peered":"2024-01-24T14:13:59.185482+0000","last_unstale":"2024-01-24T14:13:59.581576+0000","last_undegraded":"2024-01-24T14:13:59.581576+0000","last_fullsized":"2024-01-24T14:13:59.581576+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599233+0000","last_change":"2024-01-24T14:13:59.597585+0000","last_active":"2024-01-24T14:13:59.599233+0000","last_peered":"2024-01-24T14:13:59.599233+0000","last_clean":"2024-01-24T14:13:59.599233+0000","last_became_active":"2024-01-24T14:13:59.597114+0000","last_became_peered":"2024-01-24T14:13:59.597114+0000","last_unstale":"2024-01-24T14:13:59.599233+0000","last_undegraded":"2024-01-24T14:13:59.599233+0000","last_fullsized":"2024-01-24T14:13:59.599233+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.587957+0000","last_change":"2024-01-24T14:13:59.180890+0000","last_active":"2024-01-24T14:13:59.587957+0000","last_peered":"2024-01-24T14:13:59.587957+0000","last_clean":"2024-01-24T14:13:59.587957+0000","last_became_active":"2024-01-24T14:13:59.180535+0000","last_became_peered":"2024-01-24T14:13:59.180535+0000","last_unstale":"2024-01-24T14:13:59.587957+0000","last_undegraded":"2024-01-24T14:13:59.587957+0000","last_fullsized":"2024-01-24T14:13:59.587957+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613282+0000","last_change":"2024-01-24T14:13:59.607645+0000","last_active":"2024-01-24T14:13:59.613282+0000","last_peered":"2024-01-24T14:13:59.613282+0000","last_clean":"2024-01-24T14:13:59.613282+0000","last_became_active":"2024-01-24T14:13:59.607354+0000","last_became_peered":"2024-01-24T14:13:59.607354+0000","last_unstale":"2024-01-24T14:13:59.613282+0000","last_undegraded":"2024-01-24T14:13:59.613282+0000","last_fullsized":"2024-01-24T14:13:59.613282+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597240+0000","last_change":"2024-01-24T14:13:59.187503+0000","last_active":"2024-01-24T14:13:59.597240+0000","last_peered":"2024-01-24T14:13:59.597240+0000","last_clean":"2024-01-24T14:13:59.597240+0000","last_became_active":"2024-01-24T14:13:59.187203+0000","last_became_peered":"2024-01-24T14:13:59.187203+0000","last_unstale":"2024-01-24T14:13:59.597240+0000","last_undegraded":"2024-01-24T14:13:59.597240+0000","last_fullsized":"2024-01-24T14:13:59.597240+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601777+0000","last_change":"2024-01-24T14:13:59.600823+0000","last_active":"2024-01-24T14:13:59.601777+0000","last_peered":"2024-01-24T14:13:59.601777+0000","last_clean":"2024-01-24T14:13:59.601777+0000","last_became_active":"2024-01-24T14:13:59.600322+0000","last_became_peered":"2024-01-24T14:13:59.600322+0000","last_unstale":"2024-01-24T14:13:59.601777+0000","last_undegraded":"2024-01-24T14:13:59.601777+0000","last_fullsized":"2024-01-24T14:13:59.601777+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597400+0000","last_change":"2024-01-24T14:13:59.187980+0000","last_active":"2024-01-24T14:13:59.597400+0000","last_peered":"2024-01-24T14:13:59.597400+0000","last_clean":"2024-01-24T14:13:59.597400+0000","last_became_active":"2024-01-24T14:13:59.187653+0000","last_became_peered":"2024-01-24T14:13:59.187653+0000","last_unstale":"2024-01-24T14:13:59.597400+0000","last_undegraded":"2024-01-24T14:13:59.597400+0000","last_fullsized":"2024-01-24T14:13:59.597400+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615401+0000","last_change":"2024-01-24T14:13:59.611324+0000","last_active":"2024-01-24T14:13:59.615401+0000","last_peered":"2024-01-24T14:13:59.615401+0000","last_clean":"2024-01-24T14:13:59.615401+0000","last_became_active":"2024-01-24T14:13:59.610992+0000","last_became_peered":"2024-01-24T14:13:59.610992+0000","last_unstale":"2024-01-24T14:13:59.615401+0000","last_undegraded":"2024-01-24T14:13:59.615401+0000","last_fullsized":"2024-01-24T14:13:59.615401+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585787+0000","last_change":"2024-01-24T14:13:59.183133+0000","last_active":"2024-01-24T14:13:59.585787+0000","last_peered":"2024-01-24T14:13:59.585787+0000","last_clean":"2024-01-24T14:13:59.585787+0000","last_became_active":"2024-01-24T14:13:59.182837+0000","last_became_peered":"2024-01-24T14:13:59.182837+0000","last_unstale":"2024-01-24T14:13:59.585787+0000","last_undegraded":"2024-01-24T14:13:59.585787+0000","last_fullsized":"2024-01-24T14:13:59.585787+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605252+0000","last_change":"2024-01-24T14:13:59.601562+0000","last_active":"2024-01-24T14:13:59.605252+0000","last_peered":"2024-01-24T14:13:59.605252+0000","last_clean":"2024-01-24T14:13:59.605252+0000","last_became_active":"2024-01-24T14:13:59.600361+0000","last_became_peered":"2024-01-24T14:13:59.600361+0000","last_unstale":"2024-01-24T14:13:59.605252+0000","last_undegraded":"2024-01-24T14:13:59.605252+0000","last_fullsized":"2024-01-24T14:13:59.605252+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596181+0000","last_change":"2024-01-24T14:13:59.176584+0000","last_active":"2024-01-24T14:13:59.596181+0000","last_peered":"2024-01-24T14:13:59.596181+0000","last_clean":"2024-01-24T14:13:59.596181+0000","last_became_active":"2024-01-24T14:13:59.176352+0000","last_became_peered":"2024-01-24T14:13:59.176352+0000","last_unstale":"2024-01-24T14:13:59.596181+0000","last_undegraded":"2024-01-24T14:13:59.596181+0000","last_fullsized":"2024-01-24T14:13:59.596181+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.606565+0000","last_change":"2024-01-24T14:13:59.605786+0000","last_active":"2024-01-24T14:13:59.606565+0000","last_peered":"2024-01-24T14:13:59.606565+0000","last_clean":"2024-01-24T14:13:59.606565+0000","last_became_active":"2024-01-24T14:13:59.605479+0000","last_became_peered":"2024-01-24T14:13:59.605479+0000","last_unstale":"2024-01-24T14:13:59.606565+0000","last_undegraded":"2024-01-24T14:13:59.606565+0000","last_fullsized":"2024-01-24T14:13:59.606565+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.582565+0000","last_change":"2024-01-24T14:13:59.177305+0000","last_active":"2024-01-24T14:13:59.582565+0000","last_peered":"2024-01-24T14:13:59.582565+0000","last_clean":"2024-01-24T14:13:59.582565+0000","last_became_active":"2024-01-24T14:13:59.177013+0000","last_became_peered":"2024-01-24T14:13:59.177013+0000","last_unstale":"2024-01-24T14:13:59.582565+0000","last_undegraded":"2024-01-24T14:13:59.582565+0000","last_fullsized":"2024-01-24T14:13:59.582565+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613859+0000","last_change":"2024-01-24T14:13:59.605686+0000","last_active":"2024-01-24T14:13:59.613859+0000","last_peered":"2024-01-24T14:13:59.613859+0000","last_clean":"2024-01-24T14:13:59.613859+0000","last_became_active":"2024-01-24T14:13:59.599468+0000","last_became_peered":"2024-01-24T14:13:59.599468+0000","last_unstale":"2024-01-24T14:13:59.613859+0000","last_undegraded":"2024-01-24T14:13:59.613859+0000","last_fullsized":"2024-01-24T14:13:59.613859+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597018+0000","last_change":"2024-01-24T14:13:59.176388+0000","last_active":"2024-01-24T14:13:59.597018+0000","last_peered":"2024-01-24T14:13:59.597018+0000","last_clean":"2024-01-24T14:13:59.597018+0000","last_became_active":"2024-01-24T14:13:59.176027+0000","last_became_peered":"2024-01-24T14:13:59.176027+0000","last_unstale":"2024-01-24T14:13:59.597018+0000","last_undegraded":"2024-01-24T14:13:59.597018+0000","last_fullsized":"2024-01-24T14:13:59.597018+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613713+0000","last_change":"2024-01-24T14:13:59.604787+0000","last_active":"2024-01-24T14:13:59.613713+0000","last_peered":"2024-01-24T14:13:59.613713+0000","last_clean":"2024-01-24T14:13:59.613713+0000","last_became_active":"2024-01-24T14:13:59.604506+0000","last_became_peered":"2024-01-24T14:13:59.604506+0000","last_unstale":"2024-01-24T14:13:59.613713+0000","last_undegraded":"2024-01-24T14:13:59.613713+0000","last_fullsized":"2024-01-24T14:13:59.613713+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593610+0000","last_change":"2024-01-24T14:13:59.177542+0000","last_active":"2024-01-24T14:13:59.593610+0000","last_peered":"2024-01-24T14:13:59.593610+0000","last_clean":"2024-01-24T14:13:59.593610+0000","last_became_active":"2024-01-24T14:13:59.177246+0000","last_became_peered":"2024-01-24T14:13:59.177246+0000","last_unstale":"2024-01-24T14:13:59.593610+0000","last_undegraded":"2024-01-24T14:13:59.593610+0000","last_fullsized":"2024-01-24T14:13:59.593610+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602015+0000","last_change":"2024-01-24T14:13:59.597273+0000","last_active":"2024-01-24T14:13:59.602015+0000","last_peered":"2024-01-24T14:13:59.602015+0000","last_clean":"2024-01-24T14:13:59.602015+0000","last_became_active":"2024-01-24T14:13:59.597053+0000","last_became_peered":"2024-01-24T14:13:59.597053+0000","last_unstale":"2024-01-24T14:13:59.602015+0000","last_undegraded":"2024-01-24T14:13:59.602015+0000","last_fullsized":"2024-01-24T14:13:59.602015+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591431+0000","last_change":"2024-01-24T14:13:59.187008+0000","last_active":"2024-01-24T14:13:59.591431+0000","last_peered":"2024-01-24T14:13:59.591431+0000","last_clean":"2024-01-24T14:13:59.591431+0000","last_became_active":"2024-01-24T14:13:59.186690+0000","last_became_peered":"2024-01-24T14:13:59.186690+0000","last_unstale":"2024-01-24T14:13:59.591431+0000","last_undegraded":"2024-01-24T14:13:59.591431+0000","last_fullsized":"2024-01-24T14:13:59.591431+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602216+0000","last_change":"2024-01-24T14:13:59.596466+0000","last_active":"2024-01-24T14:13:59.602216+0000","last_peered":"2024-01-24T14:13:59.602216+0000","last_clean":"2024-01-24T14:13:59.602216+0000","last_became_active":"2024-01-24T14:13:59.596179+0000","last_became_peered":"2024-01-24T14:13:59.596179+0000","last_unstale":"2024-01-24T14:13:59.602216+0000","last_undegraded":"2024-01-24T14:13:59.602216+0000","last_fullsized":"2024-01-24T14:13:59.602216+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594464+0000","last_change":"2024-01-24T14:13:59.175676+0000","last_active":"2024-01-24T14:13:59.594464+0000","last_peered":"2024-01-24T14:13:59.594464+0000","last_clean":"2024-01-24T14:13:59.594464+0000","last_became_active":"2024-01-24T14:13:59.175396+0000","last_became_peered":"2024-01-24T14:13:59.175396+0000","last_unstale":"2024-01-24T14:13:59.594464+0000","last_undegraded":"2024-01-24T14:13:59.594464+0000","last_fullsized":"2024-01-24T14:13:59.594464+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.606287+0000","last_change":"2024-01-24T14:13:59.602951+0000","last_active":"2024-01-24T14:13:59.606287+0000","last_peered":"2024-01-24T14:13:59.606287+0000","last_clean":"2024-01-24T14:13:59.606287+0000","last_became_active":"2024-01-24T14:13:59.602648+0000","last_became_peered":"2024-01-24T14:13:59.602648+0000","last_unstale":"2024-01-24T14:13:59.606287+0000","last_undegraded":"2024-01-24T14:13:59.606287+0000","last_fullsized":"2024-01-24T14:13:59.606287+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.600682+0000","last_change":"2024-01-24T14:13:59.179441+0000","last_active":"2024-01-24T14:13:59.600682+0000","last_peered":"2024-01-24T14:13:59.600682+0000","last_clean":"2024-01-24T14:13:59.600682+0000","last_became_active":"2024-01-24T14:13:59.178699+0000","last_became_peered":"2024-01-24T14:13:59.178699+0000","last_unstale":"2024-01-24T14:13:59.600682+0000","last_undegraded":"2024-01-24T14:13:59.600682+0000","last_fullsized":"2024-01-24T14:13:59.600682+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613325+0000","last_change":"2024-01-24T14:13:59.607754+0000","last_active":"2024-01-24T14:13:59.613325+0000","last_peered":"2024-01-24T14:13:59.613325+0000","last_clean":"2024-01-24T14:13:59.613325+0000","last_became_active":"2024-01-24T14:13:59.607437+0000","last_became_peered":"2024-01-24T14:13:59.607437+0000","last_unstale":"2024-01-24T14:13:59.613325+0000","last_undegraded":"2024-01-24T14:13:59.613325+0000","last_fullsized":"2024-01-24T14:13:59.613325+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605436+0000","last_change":"2024-01-24T14:13:59.180915+0000","last_active":"2024-01-24T14:13:59.605436+0000","last_peered":"2024-01-24T14:13:59.605436+0000","last_clean":"2024-01-24T14:13:59.605436+0000","last_became_active":"2024-01-24T14:13:59.180385+0000","last_became_peered":"2024-01-24T14:13:59.180385+0000","last_unstale":"2024-01-24T14:13:59.605436+0000","last_undegraded":"2024-01-24T14:13:59.605436+0000","last_fullsized":"2024-01-24T14:13:59.605436+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605231+0000","last_change":"2024-01-24T14:13:59.601292+0000","last_active":"2024-01-24T14:13:59.605231+0000","last_peered":"2024-01-24T14:13:59.605231+0000","last_clean":"2024-01-24T14:13:59.605231+0000","last_became_active":"2024-01-24T14:13:59.600834+0000","last_became_peered":"2024-01-24T14:13:59.600834+0000","last_unstale":"2024-01-24T14:13:59.605231+0000","last_undegraded":"2024-01-24T14:13:59.605231+0000","last_fullsized":"2024-01-24T14:13:59.605231+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615335+0000","last_change":"2024-01-24T14:13:59.175601+0000","last_active":"2024-01-24T14:13:59.615335+0000","last_peered":"2024-01-24T14:13:59.615335+0000","last_clean":"2024-01-24T14:13:59.615335+0000","last_became_active":"2024-01-24T14:13:59.173774+0000","last_became_peered":"2024-01-24T14:13:59.173774+0000","last_unstale":"2024-01-24T14:13:59.615335+0000","last_undegraded":"2024-01-24T14:13:59.615335+0000","last_fullsized":"2024-01-24T14:13:59.615335+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613559+0000","last_change":"2024-01-24T14:13:59.604227+0000","last_active":"2024-01-24T14:13:59.613559+0000","last_peered":"2024-01-24T14:13:59.613559+0000","last_clean":"2024-01-24T14:13:59.613559+0000","last_became_active":"2024-01-24T14:13:59.603933+0000","last_became_peered":"2024-01-24T14:13:59.603933+0000","last_unstale":"2024-01-24T14:13:59.613559+0000","last_undegraded":"2024-01-24T14:13:59.613559+0000","last_fullsized":"2024-01-24T14:13:59.613559+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597128+0000","last_change":"2024-01-24T14:13:59.169115+0000","last_active":"2024-01-24T14:13:59.597128+0000","last_peered":"2024-01-24T14:13:59.597128+0000","last_clean":"2024-01-24T14:13:59.597128+0000","last_became_active":"2024-01-24T14:13:59.168789+0000","last_became_peered":"2024-01-24T14:13:59.168789+0000","last_unstale":"2024-01-24T14:13:59.597128+0000","last_undegraded":"2024-01-24T14:13:59.597128+0000","last_fullsized":"2024-01-24T14:13:59.597128+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613839+0000","last_change":"2024-01-24T14:13:59.604215+0000","last_active":"2024-01-24T14:13:59.613839+0000","last_peered":"2024-01-24T14:13:59.613839+0000","last_clean":"2024-01-24T14:13:59.613839+0000","last_became_active":"2024-01-24T14:13:59.603918+0000","last_became_peered":"2024-01-24T14:13:59.603918+0000","last_unstale":"2024-01-24T14:13:59.613839+0000","last_undegraded":"2024-01-24T14:13:59.613839+0000","last_fullsized":"2024-01-24T14:13:59.613839+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588396+0000","last_change":"2024-01-24T14:13:59.181773+0000","last_active":"2024-01-24T14:13:59.588396+0000","last_peered":"2024-01-24T14:13:59.588396+0000","last_clean":"2024-01-24T14:13:59.588396+0000","last_became_active":"2024-01-24T14:13:59.181227+0000","last_became_peered":"2024-01-24T14:13:59.181227+0000","last_unstale":"2024-01-24T14:13:59.588396+0000","last_undegraded":"2024-01-24T14:13:59.588396+0000","last_fullsized":"2024-01-24T14:13:59.588396+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602292+0000","last_change":"2024-01-24T14:13:59.598283+0000","last_active":"2024-01-24T14:13:59.602292+0000","last_peered":"2024-01-24T14:13:59.602292+0000","last_clean":"2024-01-24T14:13:59.602292+0000","last_became_active":"2024-01-24T14:13:59.598020+0000","last_became_peered":"2024-01-24T14:13:59.598020+0000","last_unstale":"2024-01-24T14:13:59.602292+0000","last_undegraded":"2024-01-24T14:13:59.602292+0000","last_fullsized":"2024-01-24T14:13:59.602292+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588480+0000","last_change":"2024-01-24T14:13:59.180957+0000","last_active":"2024-01-24T14:13:59.588480+0000","last_peered":"2024-01-24T14:13:59.588480+0000","last_clean":"2024-01-24T14:13:59.588480+0000","last_became_active":"2024-01-24T14:13:59.180449+0000","last_became_peered":"2024-01-24T14:13:59.180449+0000","last_unstale":"2024-01-24T14:13:59.588480+0000","last_undegraded":"2024-01-24T14:13:59.588480+0000","last_fullsized":"2024-01-24T14:13:59.588480+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616746+0000","last_change":"2024-01-24T14:13:59.610454+0000","last_active":"2024-01-24T14:13:59.616746+0000","last_peered":"2024-01-24T14:13:59.616746+0000","last_clean":"2024-01-24T14:13:59.616746+0000","last_became_active":"2024-01-24T14:13:59.610190+0000","last_became_peered":"2024-01-24T14:13:59.610190+0000","last_unstale":"2024-01-24T14:13:59.616746+0000","last_undegraded":"2024-01-24T14:13:59.616746+0000","last_fullsized":"2024-01-24T14:13:59.616746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.600543+0000","last_change":"2024-01-24T14:13:59.177014+0000","last_active":"2024-01-24T14:13:59.600543+0000","last_peered":"2024-01-24T14:13:59.600543+0000","last_clean":"2024-01-24T14:13:59.600543+0000","last_became_active":"2024-01-24T14:13:59.176432+0000","last_became_peered":"2024-01-24T14:13:59.176432+0000","last_unstale":"2024-01-24T14:13:59.600543+0000","last_undegraded":"2024-01-24T14:13:59.600543+0000","last_fullsized":"2024-01-24T14:13:59.600543+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.608927+0000","last_change":"2024-01-24T14:13:59.604972+0000","last_active":"2024-01-24T14:13:59.608927+0000","last_peered":"2024-01-24T14:13:59.608927+0000","last_clean":"2024-01-24T14:13:59.608927+0000","last_became_active":"2024-01-24T14:13:59.604643+0000","last_became_peered":"2024-01-24T14:13:59.604643+0000","last_unstale":"2024-01-24T14:13:59.608927+0000","last_undegraded":"2024-01-24T14:13:59.608927+0000","last_fullsized":"2024-01-24T14:13:59.608927+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591156+0000","last_change":"2024-01-24T14:13:59.184839+0000","last_active":"2024-01-24T14:13:59.591156+0000","last_peered":"2024-01-24T14:13:59.591156+0000","last_clean":"2024-01-24T14:13:59.591156+0000","last_became_active":"2024-01-24T14:13:59.184535+0000","last_became_peered":"2024-01-24T14:13:59.184535+0000","last_unstale":"2024-01-24T14:13:59.591156+0000","last_undegraded":"2024-01-24T14:13:59.591156+0000","last_fullsized":"2024-01-24T14:13:59.591156+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604995+0000","last_change":"2024-01-24T14:13:59.600061+0000","last_active":"2024-01-24T14:13:59.604995+0000","last_peered":"2024-01-24T14:13:59.604995+0000","last_clean":"2024-01-24T14:13:59.604995+0000","last_became_active":"2024-01-24T14:13:59.599799+0000","last_became_peered":"2024-01-24T14:13:59.599799+0000","last_unstale":"2024-01-24T14:13:59.604995+0000","last_undegraded":"2024-01-24T14:13:59.604995+0000","last_fullsized":"2024-01-24T14:13:59.604995+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591563+0000","last_change":"2024-01-24T14:13:59.179655+0000","last_active":"2024-01-24T14:13:59.591563+0000","last_peered":"2024-01-24T14:13:59.591563+0000","last_clean":"2024-01-24T14:13:59.591563+0000","last_became_active":"2024-01-24T14:13:59.179068+0000","last_became_peered":"2024-01-24T14:13:59.179068+0000","last_unstale":"2024-01-24T14:13:59.591563+0000","last_undegraded":"2024-01-24T14:13:59.591563+0000","last_fullsized":"2024-01-24T14:13:59.591563+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602283+0000","last_change":"2024-01-24T14:13:59.594537+0000","last_active":"2024-01-24T14:13:59.602283+0000","last_peered":"2024-01-24T14:13:59.602283+0000","last_clean":"2024-01-24T14:13:59.602283+0000","last_became_active":"2024-01-24T14:13:59.594254+0000","last_became_peered":"2024-01-24T14:13:59.594254+0000","last_unstale":"2024-01-24T14:13:59.602283+0000","last_undegraded":"2024-01-24T14:13:59.602283+0000","last_fullsized":"2024-01-24T14:13:59.602283+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591800+0000","last_change":"2024-01-24T14:13:59.182373+0000","last_active":"2024-01-24T14:13:59.591800+0000","last_peered":"2024-01-24T14:13:59.591800+0000","last_clean":"2024-01-24T14:13:59.591800+0000","last_became_active":"2024-01-24T14:13:59.182077+0000","last_became_peered":"2024-01-24T14:13:59.182077+0000","last_unstale":"2024-01-24T14:13:59.591800+0000","last_undegraded":"2024-01-24T14:13:59.591800+0000","last_fullsized":"2024-01-24T14:13:59.591800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613637+0000","last_change":"2024-01-24T14:13:59.607656+0000","last_active":"2024-01-24T14:13:59.613637+0000","last_peered":"2024-01-24T14:13:59.613637+0000","last_clean":"2024-01-24T14:13:59.613637+0000","last_became_active":"2024-01-24T14:13:59.607365+0000","last_became_peered":"2024-01-24T14:13:59.607365+0000","last_unstale":"2024-01-24T14:13:59.613637+0000","last_undegraded":"2024-01-24T14:13:59.613637+0000","last_fullsized":"2024-01-24T14:13:59.613637+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"104'3","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601949+0000","last_change":"2024-01-24T14:13:59.182375+0000","last_active":"2024-01-24T14:13:59.601949+0000","last_peered":"2024-01-24T14:13:59.601949+0000","last_clean":"2024-01-24T14:13:59.601949+0000","last_became_active":"2024-01-24T14:13:59.182150+0000","last_became_peered":"2024-01-24T14:13:59.182150+0000","last_unstale":"2024-01-24T14:13:59.601949+0000","last_undegraded":"2024-01-24T14:13:59.601949+0000","last_fullsized":"2024-01-24T14:13:59.601949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612014+0000","last_change":"2024-01-24T14:13:59.608414+0000","last_active":"2024-01-24T14:13:59.612014+0000","last_peered":"2024-01-24T14:13:59.612014+0000","last_clean":"2024-01-24T14:13:59.612014+0000","last_became_active":"2024-01-24T14:13:59.608013+0000","last_became_peered":"2024-01-24T14:13:59.608013+0000","last_unstale":"2024-01-24T14:13:59.612014+0000","last_undegraded":"2024-01-24T14:13:59.612014+0000","last_fullsized":"2024-01-24T14:13:59.612014+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.607061+0000","last_change":"2024-01-24T14:13:59.178726+0000","last_active":"2024-01-24T14:13:59.607061+0000","last_peered":"2024-01-24T14:13:59.607061+0000","last_clean":"2024-01-24T14:13:59.607061+0000","last_became_active":"2024-01-24T14:13:59.178463+0000","last_became_peered":"2024-01-24T14:13:59.178463+0000","last_unstale":"2024-01-24T14:13:59.607061+0000","last_undegraded":"2024-01-24T14:13:59.607061+0000","last_fullsized":"2024-01-24T14:13:59.607061+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.614134+0000","last_change":"2024-01-24T14:13:59.605906+0000","last_active":"2024-01-24T14:13:59.614134+0000","last_peered":"2024-01-24T14:13:59.614134+0000","last_clean":"2024-01-24T14:13:59.614134+0000","last_became_active":"2024-01-24T14:13:59.605404+0000","last_became_peered":"2024-01-24T14:13:59.605404+0000","last_unstale":"2024-01-24T14:13:59.614134+0000","last_undegraded":"2024-01-24T14:13:59.614134+0000","last_fullsized":"2024-01-24T14:13:59.614134+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613963+0000","last_change":"2024-01-24T14:13:59.174910+0000","last_active":"2024-01-24T14:13:59.613963+0000","last_peered":"2024-01-24T14:13:59.613963+0000","last_clean":"2024-01-24T14:13:59.613963+0000","last_became_active":"2024-01-24T14:13:59.174603+0000","last_became_peered":"2024-01-24T14:13:59.174603+0000","last_unstale":"2024-01-24T14:13:59.613963+0000","last_undegraded":"2024-01-24T14:13:59.613963+0000","last_fullsized":"2024-01-24T14:13:59.613963+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.618795+0000","last_change":"2024-01-24T14:13:59.614234+0000","last_active":"2024-01-24T14:13:59.618795+0000","last_peered":"2024-01-24T14:13:59.618795+0000","last_clean":"2024-01-24T14:13:59.618795+0000","last_became_active":"2024-01-24T14:13:59.614018+0000","last_became_peered":"2024-01-24T14:13:59.614018+0000","last_unstale":"2024-01-24T14:13:59.618795+0000","last_undegraded":"2024-01-24T14:13:59.618795+0000","last_fullsized":"2024-01-24T14:13:59.618795+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585537+0000","last_change":"2024-01-24T14:13:59.183528+0000","last_active":"2024-01-24T14:13:59.585537+0000","last_peered":"2024-01-24T14:13:59.585537+0000","last_clean":"2024-01-24T14:13:59.585537+0000","last_became_active":"2024-01-24T14:13:59.183214+0000","last_became_peered":"2024-01-24T14:13:59.183214+0000","last_unstale":"2024-01-24T14:13:59.585537+0000","last_undegraded":"2024-01-24T14:13:59.585537+0000","last_fullsized":"2024-01-24T14:13:59.585537+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613805+0000","last_change":"2024-01-24T14:13:59.605325+0000","last_active":"2024-01-24T14:13:59.613805+0000","last_peered":"2024-01-24T14:13:59.613805+0000","last_clean":"2024-01-24T14:13:59.613805+0000","last_became_active":"2024-01-24T14:13:59.605027+0000","last_became_peered":"2024-01-24T14:13:59.605027+0000","last_unstale":"2024-01-24T14:13:59.613805+0000","last_undegraded":"2024-01-24T14:13:59.613805+0000","last_fullsized":"2024-01-24T14:13:59.613805+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.586227+0000","last_change":"2024-01-24T14:13:59.182994+0000","last_active":"2024-01-24T14:13:59.586227+0000","last_peered":"2024-01-24T14:13:59.586227+0000","last_clean":"2024-01-24T14:13:59.586227+0000","last_became_active":"2024-01-24T14:13:59.182682+0000","last_became_peered":"2024-01-24T14:13:59.182682+0000","last_unstale":"2024-01-24T14:13:59.586227+0000","last_undegraded":"2024-01-24T14:13:59.586227+0000","last_fullsized":"2024-01-24T14:13:59.586227+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615224+0000","last_change":"2024-01-24T14:13:59.610385+0000","last_active":"2024-01-24T14:13:59.615224+0000","last_peered":"2024-01-24T14:13:59.615224+0000","last_clean":"2024-01-24T14:13:59.615224+0000","last_became_active":"2024-01-24T14:13:59.609240+0000","last_became_peered":"2024-01-24T14:13:59.609240+0000","last_unstale":"2024-01-24T14:13:59.615224+0000","last_undegraded":"2024-01-24T14:13:59.615224+0000","last_fullsized":"2024-01-24T14:13:59.615224+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.617265+0000","last_change":"2024-01-24T14:13:59.180036+0000","last_active":"2024-01-24T14:13:59.617265+0000","last_peered":"2024-01-24T14:13:59.617265+0000","last_clean":"2024-01-24T14:13:59.617265+0000","last_became_active":"2024-01-24T14:13:59.179065+0000","last_became_peered":"2024-01-24T14:13:59.179065+0000","last_unstale":"2024-01-24T14:13:59.617265+0000","last_undegraded":"2024-01-24T14:13:59.617265+0000","last_fullsized":"2024-01-24T14:13:59.617265+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.603037+0000","last_change":"2024-01-24T14:13:59.600098+0000","last_active":"2024-01-24T14:13:59.603037+0000","last_peered":"2024-01-24T14:13:59.603037+0000","last_clean":"2024-01-24T14:13:59.603037+0000","last_became_active":"2024-01-24T14:13:59.599838+0000","last_became_peered":"2024-01-24T14:13:59.599838+0000","last_unstale":"2024-01-24T14:13:59.603037+0000","last_undegraded":"2024-01-24T14:13:59.603037+0000","last_fullsized":"2024-01-24T14:13:59.603037+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601872+0000","last_change":"2024-01-24T14:13:59.178489+0000","last_active":"2024-01-24T14:13:59.601872+0000","last_peered":"2024-01-24T14:13:59.601872+0000","last_clean":"2024-01-24T14:13:59.601872+0000","last_became_active":"2024-01-24T14:13:59.178185+0000","last_became_peered":"2024-01-24T14:13:59.178185+0000","last_unstale":"2024-01-24T14:13:59.601872+0000","last_undegraded":"2024-01-24T14:13:59.601872+0000","last_fullsized":"2024-01-24T14:13:59.601872+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616653+0000","last_change":"2024-01-24T14:13:59.609527+0000","last_active":"2024-01-24T14:13:59.616653+0000","last_peered":"2024-01-24T14:13:59.616653+0000","last_clean":"2024-01-24T14:13:59.616653+0000","last_became_active":"2024-01-24T14:13:59.609180+0000","last_became_peered":"2024-01-24T14:13:59.609180+0000","last_unstale":"2024-01-24T14:13:59.616653+0000","last_undegraded":"2024-01-24T14:13:59.616653+0000","last_fullsized":"2024-01-24T14:13:59.616653+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591353+0000","last_change":"2024-01-24T14:13:59.187223+0000","last_active":"2024-01-24T14:13:59.591353+0000","last_peered":"2024-01-24T14:13:59.591353+0000","last_clean":"2024-01-24T14:13:59.591353+0000","last_became_active":"2024-01-24T14:13:59.186908+0000","last_became_peered":"2024-01-24T14:13:59.186908+0000","last_unstale":"2024-01-24T14:13:59.591353+0000","last_undegraded":"2024-01-24T14:13:59.591353+0000","last_fullsized":"2024-01-24T14:13:59.591353+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604295+0000","last_change":"2024-01-24T14:13:59.597776+0000","last_active":"2024-01-24T14:13:59.604295+0000","last_peered":"2024-01-24T14:13:59.604295+0000","last_clean":"2024-01-24T14:13:59.604295+0000","last_became_active":"2024-01-24T14:13:59.597468+0000","last_became_peered":"2024-01-24T14:13:59.597468+0000","last_unstale":"2024-01-24T14:13:59.604295+0000","last_undegraded":"2024-01-24T14:13:59.604295+0000","last_fullsized":"2024-01-24T14:13:59.604295+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588522+0000","last_change":"2024-01-24T14:13:59.169565+0000","last_active":"2024-01-24T14:13:59.588522+0000","last_peered":"2024-01-24T14:13:59.588522+0000","last_clean":"2024-01-24T14:13:59.588522+0000","last_became_active":"2024-01-24T14:13:59.169300+0000","last_became_peered":"2024-01-24T14:13:59.169300+0000","last_unstale":"2024-01-24T14:13:59.588522+0000","last_undegraded":"2024-01-24T14:13:59.588522+0000","last_fullsized":"2024-01-24T14:13:59.588522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.610569+0000","last_change":"2024-01-24T14:13:59.607765+0000","last_active":"2024-01-24T14:13:59.610569+0000","last_peered":"2024-01-24T14:13:59.610569+0000","last_clean":"2024-01-24T14:13:59.610569+0000","last_became_active":"2024-01-24T14:13:59.607462+0000","last_became_peered":"2024-01-24T14:13:59.607462+0000","last_unstale":"2024-01-24T14:13:59.610569+0000","last_undegraded":"2024-01-24T14:13:59.610569+0000","last_fullsized":"2024-01-24T14:13:59.610569+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.590394+0000","last_change":"2024-01-24T14:13:59.179494+0000","last_active":"2024-01-24T14:13:59.590394+0000","last_peered":"2024-01-24T14:13:59.590394+0000","last_clean":"2024-01-24T14:13:59.590394+0000","last_became_active":"2024-01-24T14:13:59.178687+0000","last_became_peered":"2024-01-24T14:13:59.178687+0000","last_unstale":"2024-01-24T14:13:59.590394+0000","last_undegraded":"2024-01-24T14:13:59.590394+0000","last_fullsized":"2024-01-24T14:13:59.590394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.598793+0000","last_change":"2024-01-24T14:13:59.592141+0000","last_active":"2024-01-24T14:13:59.598793+0000","last_peered":"2024-01-24T14:13:59.598793+0000","last_clean":"2024-01-24T14:13:59.598793+0000","last_became_active":"2024-01-24T14:13:59.590960+0000","last_became_peered":"2024-01-24T14:13:59.590960+0000","last_unstale":"2024-01-24T14:13:59.598793+0000","last_undegraded":"2024-01-24T14:13:59.598793+0000","last_fullsized":"2024-01-24T14:13:59.598793+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597174+0000","last_change":"2024-01-24T14:13:59.186128+0000","last_active":"2024-01-24T14:13:59.597174+0000","last_peered":"2024-01-24T14:13:59.597174+0000","last_clean":"2024-01-24T14:13:59.597174+0000","last_became_active":"2024-01-24T14:13:59.177094+0000","last_became_peered":"2024-01-24T14:13:59.177094+0000","last_unstale":"2024-01-24T14:13:59.597174+0000","last_undegraded":"2024-01-24T14:13:59.597174+0000","last_fullsized":"2024-01-24T14:13:59.597174+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613687+0000","last_change":"2024-01-24T14:13:59.606315+0000","last_active":"2024-01-24T14:13:59.613687+0000","last_peered":"2024-01-24T14:13:59.613687+0000","last_clean":"2024-01-24T14:13:59.613687+0000","last_became_active":"2024-01-24T14:13:59.606045+0000","last_became_peered":"2024-01-24T14:13:59.606045+0000","last_unstale":"2024-01-24T14:13:59.613687+0000","last_undegraded":"2024-01-24T14:13:59.613687+0000","last_fullsized":"2024-01-24T14:13:59.613687+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588511+0000","last_change":"2024-01-24T14:13:59.180639+0000","last_active":"2024-01-24T14:13:59.588511+0000","last_peered":"2024-01-24T14:13:59.588511+0000","last_clean":"2024-01-24T14:13:59.588511+0000","last_became_active":"2024-01-24T14:13:59.180142+0000","last_became_peered":"2024-01-24T14:13:59.180142+0000","last_unstale":"2024-01-24T14:13:59.588511+0000","last_undegraded":"2024-01-24T14:13:59.588511+0000","last_fullsized":"2024-01-24T14:13:59.588511+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616523+0000","last_change":"2024-01-24T14:13:59.606110+0000","last_active":"2024-01-24T14:13:59.616523+0000","last_peered":"2024-01-24T14:13:59.616523+0000","last_clean":"2024-01-24T14:13:59.616523+0000","last_became_active":"2024-01-24T14:13:59.605783+0000","last_became_peered":"2024-01-24T14:13:59.605783+0000","last_unstale":"2024-01-24T14:13:59.616523+0000","last_undegraded":"2024-01-24T14:13:59.616523+0000","last_fullsized":"2024-01-24T14:13:59.616523+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.590527+0000","last_change":"2024-01-24T14:13:59.179345+0000","last_active":"2024-01-24T14:13:59.590527+0000","last_peered":"2024-01-24T14:13:59.590527+0000","last_clean":"2024-01-24T14:13:59.590527+0000","last_became_active":"2024-01-24T14:13:59.178902+0000","last_became_peered":"2024-01-24T14:13:59.178902+0000","last_unstale":"2024-01-24T14:13:59.590527+0000","last_undegraded":"2024-01-24T14:13:59.590527+0000","last_fullsized":"2024-01-24T14:13:59.590527+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605142+0000","last_change":"2024-01-24T14:13:59.600846+0000","last_active":"2024-01-24T14:13:59.605142+0000","last_peered":"2024-01-24T14:13:59.605142+0000","last_clean":"2024-01-24T14:13:59.605142+0000","last_became_active":"2024-01-24T14:13:59.600474+0000","last_became_peered":"2024-01-24T14:13:59.600474+0000","last_unstale":"2024-01-24T14:13:59.605142+0000","last_undegraded":"2024-01-24T14:13:59.605142+0000","last_fullsized":"2024-01-24T14:13:59.605142+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588283+0000","last_change":"2024-01-24T14:13:59.168449+0000","last_active":"2024-01-24T14:13:59.588283+0000","last_peered":"2024-01-24T14:13:59.588283+0000","last_clean":"2024-01-24T14:13:59.588283+0000","last_became_active":"2024-01-24T14:13:59.168136+0000","last_became_peered":"2024-01-24T14:13:59.168136+0000","last_unstale":"2024-01-24T14:13:59.588283+0000","last_undegraded":"2024-01-24T14:13:59.588283+0000","last_fullsized":"2024-01-24T14:13:59.588283+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.598763+0000","last_change":"2024-01-24T14:13:59.592342+0000","last_active":"2024-01-24T14:13:59.598763+0000","last_peered":"2024-01-24T14:13:59.598763+0000","last_clean":"2024-01-24T14:13:59.598763+0000","last_became_active":"2024-01-24T14:13:59.590942+0000","last_became_peered":"2024-01-24T14:13:59.590942+0000","last_unstale":"2024-01-24T14:13:59.598763+0000","last_undegraded":"2024-01-24T14:13:59.598763+0000","last_fullsized":"2024-01-24T14:13:59.598763+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581385+0000","last_change":"2024-01-24T14:13:59.173283+0000","last_active":"2024-01-24T14:13:59.581385+0000","last_peered":"2024-01-24T14:13:59.581385+0000","last_clean":"2024-01-24T14:13:59.581385+0000","last_became_active":"2024-01-24T14:13:59.172965+0000","last_became_peered":"2024-01-24T14:13:59.172965+0000","last_unstale":"2024-01-24T14:13:59.581385+0000","last_undegraded":"2024-01-24T14:13:59.581385+0000","last_fullsized":"2024-01-24T14:13:59.581385+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591632+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:13:59.591632+0000","last_peered":"2024-01-24T14:13:59.591632+0000","last_clean":"2024-01-24T14:13:59.591632+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:13:59.591632+0000","last_undegraded":"2024-01-24T14:13:59.591632+0000","last_fullsized":"2024-01-24T14:13:59.591632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.603074+0000","last_change":"2024-01-24T14:13:59.599657+0000","last_active":"2024-01-24T14:13:59.603074+0000","last_peered":"2024-01-24T14:13:59.603074+0000","last_clean":"2024-01-24T14:13:59.603074+0000","last_became_active":"2024-01-24T14:13:59.599353+0000","last_became_peered":"2024-01-24T14:13:59.599353+0000","last_unstale":"2024-01-24T14:13:59.603074+0000","last_undegraded":"2024-01-24T14:13:59.603074+0000","last_fullsized":"2024-01-24T14:13:59.603074+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597437+0000","last_change":"2024-01-24T14:13:59.179171+0000","last_active":"2024-01-24T14:13:59.597437+0000","last_peered":"2024-01-24T14:13:59.597437+0000","last_clean":"2024-01-24T14:13:59.597437+0000","last_became_active":"2024-01-24T14:13:59.176714+0000","last_became_peered":"2024-01-24T14:13:59.176714+0000","last_unstale":"2024-01-24T14:13:59.597437+0000","last_undegraded":"2024-01-24T14:13:59.597437+0000","last_fullsized":"2024-01-24T14:13:59.597437+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585663+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:13:59.585663+0000","last_peered":"2024-01-24T14:13:59.585663+0000","last_clean":"2024-01-24T14:13:59.585663+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:13:59.585663+0000","last_undegraded":"2024-01-24T14:13:59.585663+0000","last_fullsized":"2024-01-24T14:13:59.585663+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613485+0000","last_change":"2024-01-24T14:13:59.603439+0000","last_active":"2024-01-24T14:13:59.613485+0000","last_peered":"2024-01-24T14:13:59.613485+0000","last_clean":"2024-01-24T14:13:59.613485+0000","last_became_active":"2024-01-24T14:13:59.603116+0000","last_became_peered":"2024-01-24T14:13:59.603116+0000","last_unstale":"2024-01-24T14:13:59.613485+0000","last_undegraded":"2024-01-24T14:13:59.613485+0000","last_fullsized":"2024-01-24T14:13:59.613485+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602963+0000","last_change":"2024-01-24T14:13:59.181711+0000","last_active":"2024-01-24T14:13:59.602963+0000","last_peered":"2024-01-24T14:13:59.602963+0000","last_clean":"2024-01-24T14:13:59.602963+0000","last_became_active":"2024-01-24T14:13:59.176397+0000","last_became_peered":"2024-01-24T14:13:59.176397+0000","last_unstale":"2024-01-24T14:13:59.602963+0000","last_undegraded":"2024-01-24T14:13:59.602963+0000","last_fullsized":"2024-01-24T14:13:59.602963+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581282+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:13:59.581282+0000","last_peered":"2024-01-24T14:13:59.581282+0000","last_clean":"2024-01-24T14:13:59.581282+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:13:59.581282+0000","last_undegraded":"2024-01-24T14:13:59.581282+0000","last_fullsized":"2024-01-24T14:13:59.581282+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613322+0000","last_change":"2024-01-24T14:13:59.601159+0000","last_active":"2024-01-24T14:13:59.613322+0000","last_peered":"2024-01-24T14:13:59.613322+0000","last_clean":"2024-01-24T14:13:59.613322+0000","last_became_active":"2024-01-24T14:13:59.600870+0000","last_became_peered":"2024-01-24T14:13:59.600870+0000","last_unstale":"2024-01-24T14:13:59.613322+0000","last_undegraded":"2024-01-24T14:13:59.613322+0000","last_fullsized":"2024-01-24T14:13:59.613322+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.595988+0000","last_change":"2024-01-24T14:13:59.187233+0000","last_active":"2024-01-24T14:13:59.595988+0000","last_peered":"2024-01-24T14:13:59.595988+0000","last_clean":"2024-01-24T14:13:59.595988+0000","last_became_active":"2024-01-24T14:13:59.186954+0000","last_became_peered":"2024-01-24T14:13:59.186954+0000","last_unstale":"2024-01-24T14:13:59.595988+0000","last_undegraded":"2024-01-24T14:13:59.595988+0000","last_fullsized":"2024-01-24T14:13:59.595988+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585302+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:13:59.585302+0000","last_peered":"2024-01-24T14:13:59.585302+0000","last_clean":"2024-01-24T14:13:59.585302+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:13:59.585302+0000","last_undegraded":"2024-01-24T14:13:59.585302+0000","last_fullsized":"2024-01-24T14:13:59.585302+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594597+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:13:59.594597+0000","last_peered":"2024-01-24T14:13:59.594597+0000","last_clean":"2024-01-24T14:13:59.594597+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:13:59.594597+0000","last_undegraded":"2024-01-24T14:13:59.594597+0000","last_fullsized":"2024-01-24T14:13:59.594597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605110+0000","last_change":"2024-01-24T14:13:59.601219+0000","last_active":"2024-01-24T14:13:59.605110+0000","last_peered":"2024-01-24T14:13:59.605110+0000","last_clean":"2024-01-24T14:13:59.605110+0000","last_became_active":"2024-01-24T14:13:59.600941+0000","last_became_peered":"2024-01-24T14:13:59.600941+0000","last_unstale":"2024-01-24T14:13:59.605110+0000","last_undegraded":"2024-01-24T14:13:59.605110+0000","last_fullsized":"2024-01-24T14:13:59.605110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613432+0000","last_change":"2024-01-24T14:13:59.187354+0000","last_active":"2024-01-24T14:13:59.613432+0000","last_peered":"2024-01-24T14:13:59.613432+0000","last_clean":"2024-01-24T14:13:59.613432+0000","last_became_active":"2024-01-24T14:13:59.187034+0000","last_became_peered":"2024-01-24T14:13:59.187034+0000","last_unstale":"2024-01-24T14:13:59.613432+0000","last_undegraded":"2024-01-24T14:13:59.613432+0000","last_fullsized":"2024-01-24T14:13:59.613432+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594511+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:13:59.594511+0000","last_peered":"2024-01-24T14:13:59.594511+0000","last_clean":"2024-01-24T14:13:59.594511+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:13:59.594511+0000","last_undegraded":"2024-01-24T14:13:59.594511+0000","last_fullsized":"2024-01-24T14:13:59.594511+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.608535+0000","last_change":"2024-01-24T14:13:59.600293+0000","last_active":"2024-01-24T14:13:59.608535+0000","last_peered":"2024-01-24T14:13:59.608535+0000","last_clean":"2024-01-24T14:13:59.608535+0000","last_became_active":"2024-01-24T14:13:59.599864+0000","last_became_peered":"2024-01-24T14:13:59.599864+0000","last_unstale":"2024-01-24T14:13:59.608535+0000","last_undegraded":"2024-01-24T14:13:59.608535+0000","last_fullsized":"2024-01-24T14:13:59.608535+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613067+0000","last_change":"2024-01-24T14:13:59.173278+0000","last_active":"2024-01-24T14:13:59.613067+0000","last_peered":"2024-01-24T14:13:59.613067+0000","last_clean":"2024-01-24T14:13:59.613067+0000","last_became_active":"2024-01-24T14:13:59.172981+0000","last_became_peered":"2024-01-24T14:13:59.172981+0000","last_unstale":"2024-01-24T14:13:59.613067+0000","last_undegraded":"2024-01-24T14:13:59.613067+0000","last_fullsized":"2024-01-24T14:13:59.613067+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":87,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596080+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:13:59.596080+0000","last_peered":"2024-01-24T14:13:59.596080+0000","last_clean":"2024-01-24T14:13:59.596080+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:13:59.596080+0000","last_undegraded":"2024-01-24T14:13:59.596080+0000","last_fullsized":"2024-01-24T14:13:59.596080+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604651+0000","last_change":"2024-01-24T14:13:59.597915+0000","last_active":"2024-01-24T14:13:59.604651+0000","last_peered":"2024-01-24T14:13:59.604651+0000","last_clean":"2024-01-24T14:13:59.604651+0000","last_became_active":"2024-01-24T14:13:59.597457+0000","last_became_peered":"2024-01-24T14:13:59.597457+0000","last_unstale":"2024-01-24T14:13:59.604651+0000","last_undegraded":"2024-01-24T14:13:59.604651+0000","last_fullsized":"2024-01-24T14:13:59.604651+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585416+0000","last_change":"2024-01-24T14:13:59.187991+0000","last_active":"2024-01-24T14:13:59.585416+0000","last_peered":"2024-01-24T14:13:59.585416+0000","last_clean":"2024-01-24T14:13:59.585416+0000","last_became_active":"2024-01-24T14:13:59.187677+0000","last_became_peered":"2024-01-24T14:13:59.187677+0000","last_unstale":"2024-01-24T14:13:59.585416+0000","last_undegraded":"2024-01-24T14:13:59.585416+0000","last_fullsized":"2024-01-24T14:13:59.585416+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591436+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:13:59.591436+0000","last_peered":"2024-01-24T14:13:59.591436+0000","last_clean":"2024-01-24T14:13:59.591436+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:13:59.591436+0000","last_undegraded":"2024-01-24T14:13:59.591436+0000","last_fullsized":"2024-01-24T14:13:59.591436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593811+0000","last_change":"2024-01-24T14:13:59.587516+0000","last_active":"2024-01-24T14:13:59.593811+0000","last_peered":"2024-01-24T14:13:59.593811+0000","last_clean":"2024-01-24T14:13:59.593811+0000","last_became_active":"2024-01-24T14:13:59.587242+0000","last_became_peered":"2024-01-24T14:13:59.587242+0000","last_unstale":"2024-01-24T14:13:59.593811+0000","last_undegraded":"2024-01-24T14:13:59.593811+0000","last_fullsized":"2024-01-24T14:13:59.593811+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581221+0000","last_change":"2024-01-24T14:13:59.176122+0000","last_active":"2024-01-24T14:13:59.581221+0000","last_peered":"2024-01-24T14:13:59.581221+0000","last_clean":"2024-01-24T14:13:59.581221+0000","last_became_active":"2024-01-24T14:13:59.175794+0000","last_became_peered":"2024-01-24T14:13:59.175794+0000","last_unstale":"2024-01-24T14:13:59.581221+0000","last_undegraded":"2024-01-24T14:13:59.581221+0000","last_fullsized":"2024-01-24T14:13:59.581221+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":84,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591329+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:13:59.591329+0000","last_peered":"2024-01-24T14:13:59.591329+0000","last_clean":"2024-01-24T14:13:59.591329+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:13:59.591329+0000","last_undegraded":"2024-01-24T14:13:59.591329+0000","last_fullsized":"2024-01-24T14:13:59.591329+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575038,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1290240,"data_stored":710498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.26000000000000001},"max":{"1min":0.80600000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.32700000000000001},"max":{"1min":0.753,"5min":0.82599999999999996,"15min":0.83499999999999996},"last":0.58899999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.312},"max":{"1min":0.75600000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":3.2570000000000001},"last":0.53000000000000003}]},{"osd":2,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.51700000000000002,"5min":0.40400000000000003,"15min":0.32000000000000001},"max":{"1min":0.79100000000000004,"5min":0.872,"15min":0.876},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.39300000000000002,"15min":0.36799999999999999},"max":{"1min":0.80000000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66700000000000004}]},{"osd":3,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.36499999999999999},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.85099999999999998,"15min":3.0049999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48899999999999999},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.84699999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.60399999999999998,"5min":0.70599999999999996,"15min":0.83599999999999997},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.501,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.29999999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.80900000000000005},"last":0.79400000000000004}]},{"osd":6,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.621},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.80700000000000005,"15min":0.87},"last":0.72199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575039,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302456,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069384,"statfs":{"total":96636764160,"available":96327049216,"internally_reserved":0,"allocated":1294336,"data_stored":710056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.59499999999999997},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73699999999999999,"5min":0.85699999999999998,"15min":0.86899999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":0.84899999999999998,"15min":1.3859999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73499999999999999,"5min":0.77000000000000002,"15min":1.353},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.78700000000000003,"5min":1.032,"15min":1.032},"last":0.46999999999999997}]},{"osd":2,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60099999999999998,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.39100000000000001,"15min":0.34300000000000003},"max":{"1min":0.78600000000000003,"5min":0.88500000000000001,"15min":6.8129999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.4079999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.66900000000000004},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81699999999999995,"5min":0.82799999999999996,"15min":7.016},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.43099999999999999,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.77400000000000002,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.63800000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.504,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":1.165,"15min":1.165},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.314},"max":{"1min":0.65500000000000003,"5min":0.82299999999999995,"15min":6.8719999999999999},"last":0.624}]},{"osd":5,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":0.86699999999999999,"15min":6.976},"last":0.497},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80200000000000005,"5min":0.85699999999999998,"15min":7.8789999999999996},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.747,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":0.89600000000000002,"15min":6.9080000000000004},"last":0.66200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302456,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069384,"statfs":{"total":96636764160,"available":96327049216,"internally_reserved":0,"allocated":1294336,"data_stored":710068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.51900000000000002,"15min":0.443},"min":{"1min":0.311,"5min":0.245,"15min":0.23400000000000001},"max":{"1min":0.57899999999999996,"5min":15.298999999999999,"15min":15.298999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.55000000000000004,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.57499999999999996,"5min":14.654,"15min":14.654},"last":0.38}]},{"osd":2,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45900000000000002,"15min":0.45100000000000001},"min":{"1min":0.34899999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.59799999999999998,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.25800000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47099999999999997,"15min":0.45700000000000002},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88600000000000001,"5min":1.073,"15min":1.137},"last":0.441},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.45900000000000002,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.218,"15min":1.218},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":1.1040000000000001,"15min":0.83299999999999996},"min":{"1min":0.47999999999999998,"5min":0.45000000000000001,"15min":0.443},"max":{"1min":0.86799999999999999,"5min":38.808,"15min":38.808},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":1.077,"15min":0.81200000000000006},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.40500000000000003},"max":{"1min":0.81299999999999994,"5min":38.823,"15min":38.823},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.46000000000000002,"15min":0.43099999999999999},"max":{"1min":0.94899999999999995,"5min":1.0449999999999999,"15min":1.0509999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.40000000000000002},"max":{"1min":0.91300000000000003,"5min":1.1559999999999999,"15min":1.1879999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.52500000000000002,"5min":0.49399999999999999,"15min":0.45300000000000001},"max":{"1min":0.98499999999999999,"5min":1.105,"15min":1.155},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.54900000000000004,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":0.93500000000000005,"5min":1.123,"15min":1.131},"last":0.58499999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542336,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1290240,"data_stored":709909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39600000000000002},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.57499999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45000000000000001}]},{"osd":2,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44800000000000001,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.28199999999999997},"max":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.66700000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.28799999999999998},"max":{"1min":0.61399999999999999,"5min":0.754,"15min":0.79400000000000004},"last":0.53700000000000003}]},{"osd":3,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.33000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54500000000000004,"5min":1.482,"15min":1.482},"last":0.372},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.496},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":0.872,"15min":0.872},"last":0.46100000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81499999999999995,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.64400000000000002}]},{"osd":5,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.875,"5min":1.655,"15min":1.655},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.42999999999999999},"max":{"1min":0.77200000000000002,"5min":0.94799999999999995,"15min":1.0840000000000001},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.875,"5min":0.90700000000000003,"15min":1.137},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":0.96499999999999997,"15min":1.117},"last":0.70499999999999996}]},{"osd":7,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47099999999999997},"max":{"1min":0.86899999999999999,"5min":0.96099999999999997,"15min":0.98199999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.49099999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.59399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575038,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306548,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94065292,"statfs":{"total":96636764160,"available":96322859008,"internally_reserved":0,"allocated":1290240,"data_stored":710014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.29499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.65800000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.24399999999999999},"max":{"1min":0.54500000000000004,"5min":0.65900000000000003,"15min":0.67100000000000004},"last":0.46500000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.32500000000000001,"5min":0.27800000000000002,"15min":0.27300000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.67200000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.255},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.55600000000000005}]},{"osd":3,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.45000000000000001},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64300000000000002,"5min":0.67700000000000005,"15min":0.71699999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.38900000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.72799999999999998},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.502,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85899999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.55400000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.92200000000000004,"5min":0.996,"15min":1.042},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.53400000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83599999999999997,"5min":1.036,"15min":1.0469999999999999},"last":0.622},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90000000000000002,"5min":1.107,"15min":1.107},"last":0.70199999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.82899999999999996,"5min":1.0569999999999999,"15min":1.1100000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.57299999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84699999999999998,"5min":1.141,"15min":1.141},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.52400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":0.96199999999999997,"15min":1.054},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.88900000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.81000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302460,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069380,"statfs":{"total":96636764160,"available":96327045120,"internally_reserved":0,"allocated":1298432,"data_stored":710658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45100000000000001,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.76400000000000001,"5min":7.593,"15min":7.593},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48299999999999998,"15min":0.437},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56200000000000006,"5min":8.4239999999999995,"15min":8.4239999999999995},"last":0.33900000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.69099999999999995},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.221,"15min":0.221},"max":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.68799999999999994},"last":0.47399999999999998}]},{"osd":2,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.496},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.66500000000000004,"5min":0.76300000000000001,"15min":0.77200000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.78800000000000003},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64500000000000002,"15min":0.63500000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.495,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":0.81599999999999995,"5min":1.976,"15min":1.976},"last":0.52200000000000002}]},{"osd":5,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.49299999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91500000000000004,"5min":1.53,"15min":1.53},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.97999999999999998,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.65700000000000003}]},{"osd":7,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72799999999999998,"15min":0.72099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.72999999999999998},"min":{"1min":0.495,"5min":0.49199999999999999,"15min":0.45800000000000002},"max":{"1min":0.93500000000000005,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.62}]}]},{"osd":4,"up_from":13,"seq":55834575039,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1290240,"data_stored":710498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78300000000000003,"5min":0.80100000000000005,"15min":0.88600000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.84399999999999997},"last":0.51800000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.83099999999999996,"15min":0.85299999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.755,"5min":0.79800000000000004,"15min":1.506},"last":0.65000000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.80200000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.503},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.61499999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72899999999999998,"5min":0.77800000000000002,"15min":0.80600000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.81000000000000005,"15min":0.86299999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.92000000000000004},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.496,"15min":0.5},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65700000000000003,"5min":0.72899999999999998,"15min":1.556},"last":0.46899999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.753,"5min":0.78500000000000003,"15min":0.86099999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76400000000000001,"5min":0.82099999999999995,"15min":1.0409999999999999},"last":0.45100000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.38500000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.88100000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.307,"15min":0.307},"max":{"1min":0.72699999999999998,"5min":0.81399999999999995,"15min":1.26},"last":0.626}]}]},{"osd":5,"up_from":13,"seq":55834575039,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302464,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069376,"statfs":{"total":96636764160,"available":96327041024,"internally_reserved":0,"allocated":1302528,"data_stored":711502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64400000000000002,"5min":0.85199999999999998,"15min":1.101},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.309,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71199999999999997,"5min":0.82699999999999996,"15min":1.1439999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56000000000000005,"15min":0.57699999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67200000000000004,"5min":0.84099999999999997,"15min":1.093},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69199999999999995,"5min":0.81299999999999994,"15min":1.1639999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.68400000000000005,"5min":0.84299999999999997,"15min":1.1100000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.35499999999999998},"max":{"1min":0.67400000000000004,"5min":0.83599999999999997,"15min":1.1220000000000001},"last":0.58299999999999996}]},{"osd":3,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.38500000000000001},"max":{"1min":0.72199999999999998,"5min":0.80500000000000005,"15min":1.139},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.496,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":1.0609999999999999},"last":0.52000000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70199999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.505,"15min":0.503},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.90000000000000002},"last":0.42399999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.32900000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":1.1839999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77400000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55100000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72999999999999998,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.33200000000000002},"max":{"1min":0.78200000000000003,"5min":0.95999999999999996,"15min":1.2030000000000001},"last":0.49299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:14:08.287 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:14:08.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.425+0000 7f6f7962e700 1 Processor -- start 2024-01-24T14:14:08.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.425+0000 7f6f7962e700 1 -- start start 2024-01-24T14:14:08.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.426+0000 7f6f7962e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6f740c4a10 0x7f6f740bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.426+0000 7f6f7962e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6f740670a0 0x7f6f74158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.426+0000 7f6f7962e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f741582b0 con 0x7f6f740664f0 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.426+0000 7f6f7962e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f74157ca0 con 0x7f6f740670a0 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.426+0000 7f6f7962e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f74063c60 con 0x7f6f740c4a10 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.426+0000 7f6f727fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6f740670a0 0x7f6f74158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.426+0000 7f6f727fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6f740670a0 0x7f6f74158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59964/0 (socket says 172.21.15.29:59964) 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.426+0000 7f6f727fc700 1 -- 172.21.15.29:0/3831531203 learned_addr learned my addr 172.21.15.29:0/3831531203 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.427+0000 7f6f72ffd700 1 --2- 172.21.15.29:0/3831531203 >> v2:172.21.15.163:3300/0 conn(0x7f6f740c4a10 0x7f6f740bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.427+0000 7f6f727fc700 1 -- 172.21.15.29:0/3831531203 >> v2:172.21.15.163:3300/0 conn(0x7f6f740c4a10 msgr2=0x7f6f740bf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.427+0000 7f6f727fc700 1 --2- 172.21.15.29:0/3831531203 >> v2:172.21.15.163:3300/0 conn(0x7f6f740c4a10 0x7f6f740bf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.427+0000 7f6f727fc700 1 -- 172.21.15.29:0/3831531203 >> v1:172.21.15.163:6789/0 conn(0x7f6f740664f0 legacy=0x7f6f740bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.427+0000 7f6f727fc700 1 -- 172.21.15.29:0/3831531203 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f74157f40 con 0x7f6f740670a0 2024-01-24T14:14:08.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.427+0000 7f6f727fc700 1 --2- 172.21.15.29:0/3831531203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f740670a0 0x7f6f74158740 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f6f74069ca0 tx=0x7f6f640090f0).ready entity=mon.0 client_cookie=3b948d14bd23477c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:08.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.427+0000 7f6f71ffb700 1 -- 172.21.15.29:0/3831531203 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400b600 con 0x7f6f740670a0 2024-01-24T14:14:08.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.427+0000 7f6f71ffb700 1 -- 172.21.15.29:0/3831531203 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6400b760 con 0x7f6f740670a0 2024-01-24T14:14:08.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.427+0000 7f6f71ffb700 1 -- 172.21.15.29:0/3831531203 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400bae0 con 0x7f6f740670a0 2024-01-24T14:14:08.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.428+0000 7f6f7962e700 1 -- 172.21.15.29:0/3831531203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f740670a0 msgr2=0x7f6f74158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:08.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.428+0000 7f6f7962e700 1 --2- 172.21.15.29:0/3831531203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f740670a0 0x7f6f74158740 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f6f74069ca0 tx=0x7f6f640090f0).stop 2024-01-24T14:14:08.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.428+0000 7f6f7962e700 1 -- 172.21.15.29:0/3831531203 shutdown_connections 2024-01-24T14:14:08.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.428+0000 7f6f7962e700 1 --2- 172.21.15.29:0/3831531203 >> v2:172.21.15.163:3300/0 conn(0x7f6f740c4a10 0x7f6f740bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:08.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.428+0000 7f6f7962e700 1 --2- 172.21.15.29:0/3831531203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f740670a0 0x7f6f74158740 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:08.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.428+0000 7f6f7962e700 1 -- 172.21.15.29:0/3831531203 >> 172.21.15.29:0/3831531203 conn(0x7f6f740c8bb0 msgr2=0x7f6f740c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:08.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.428+0000 7f6f7962e700 1 -- 172.21.15.29:0/3831531203 shutdown_connections 2024-01-24T14:14:08.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.428+0000 7f6f7962e700 1 -- 172.21.15.29:0/3831531203 wait complete. 2024-01-24T14:14:08.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.430+0000 7f6f7962e700 1 Processor -- start 2024-01-24T14:14:08.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.430+0000 7f6f7962e700 1 -- start start 2024-01-24T14:14:08.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f7962e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f740664f0 0x7f6f741414e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:08.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f7962e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f740670a0 0x7f6f740adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:08.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f7962e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f740c4a10 0x7f6f740ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:08.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f7962e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740ae550 con 0x7f6f740670a0 2024-01-24T14:14:08.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f7962e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f74166a20 con 0x7f6f740c4a10 2024-01-24T14:14:08.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f7962e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f74166d00 con 0x7f6f740664f0 2024-01-24T14:14:08.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f727fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f740670a0 0x7f6f740adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:08.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f727fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f740670a0 0x7f6f740adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59974/0 (socket says 172.21.15.29:59974) 2024-01-24T14:14:08.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f727fc700 1 -- 172.21.15.29:0/1881887812 learned_addr learned my addr 172.21.15.29:0/1881887812 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f737fe700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f740c4a10 0x7f6f740ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f72ffd700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f740664f0 0x7f6f741414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f727fc700 1 -- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f740664f0 msgr2=0x7f6f741414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f727fc700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f740664f0 0x7f6f741414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f727fc700 1 -- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f740c4a10 msgr2=0x7f6f740ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f727fc700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f740c4a10 0x7f6f740ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.431+0000 7f6f727fc700 1 -- 172.21.15.29:0/1881887812 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64009370 con 0x7f6f740670a0 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.432+0000 7f6f727fc700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f740670a0 0x7f6f740adb70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f6f64009130 tx=0x7f6f64017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.432+0000 7f6f6affd700 1 -- 172.21.15.29:0/1881887812 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400b8d0 con 0x7f6f740670a0 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.432+0000 7f6f6affd700 1 -- 172.21.15.29:0/1881887812 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6400ba30 con 0x7f6f740670a0 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.432+0000 7f6f7962e700 1 -- 172.21.15.29:0/1881887812 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f74166fe0 con 0x7f6f740670a0 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.432+0000 7f6f6affd700 1 -- 172.21.15.29:0/1881887812 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6401c670 con 0x7f6f740670a0 2024-01-24T14:14:08.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.432+0000 7f6f7962e700 1 -- 172.21.15.29:0/1881887812 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f741675a0 con 0x7f6f740670a0 2024-01-24T14:14:08.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.433+0000 7f6f7962e700 1 -- 172.21.15.29:0/1881887812 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f74167790 con 0x7f6f740670a0 2024-01-24T14:14:08.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.433+0000 7f6f6affd700 1 -- 172.21.15.29:0/1881887812 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6f6401c9e0 con 0x7f6f740670a0 2024-01-24T14:14:08.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.434+0000 7f6f6affd700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6f5405f270 0x7f6f54061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:08.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.434+0000 7f6f6affd700 1 -- 172.21.15.29:0/1881887812 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f6f6407c500 con 0x7f6f740670a0 2024-01-24T14:14:08.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.434+0000 7f6f72ffd700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6f5405f270 0x7f6f54061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:08.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.434+0000 7f6f72ffd700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6f5405f270 0x7f6f54061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6f5c006fd0 tx=0x7f6f5c008040).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:08.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.438+0000 7f6f6affd700 1 -- 172.21.15.29:0/1881887812 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f6f6404d250 con 0x7f6f740670a0 2024-01-24T14:14:08.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.618+0000 7f6f7962e700 1 -- 172.21.15.29:0/1881887812 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f740475e0 con 0x7f6f5405f270 2024-01-24T14:14:08.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.630+0000 7f6f6affd700 1 -- 172.21.15.29:0/1881887812 <== mgr.5864 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361775 (secure 0 0 0) 0x7f6f740475e0 con 0x7f6f5405f270 2024-01-24T14:14:08.635 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.633+0000 7f6f7962e700 1 -- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6f5405f270 msgr2=0x7f6f54061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.633+0000 7f6f7962e700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6f5405f270 0x7f6f54061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6f5c006fd0 tx=0x7f6f5c008040).stop 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.633+0000 7f6f7962e700 1 -- 172.21.15.29:0/1881887812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f740670a0 msgr2=0x7f6f740adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.633+0000 7f6f7962e700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f740670a0 0x7f6f740adb70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f6f64009130 tx=0x7f6f64017000).stop 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.633+0000 7f6f7962e700 1 -- 172.21.15.29:0/1881887812 shutdown_connections 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.633+0000 7f6f7962e700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f740c4a10 0x7f6f740ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.633+0000 7f6f7962e700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f740664f0 0x7f6f741414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.633+0000 7f6f7962e700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6f5405f270 0x7f6f54061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.633+0000 7f6f7962e700 1 --2- 172.21.15.29:0/1881887812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f740670a0 0x7f6f740adb70 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.633+0000 7f6f7962e700 1 -- 172.21.15.29:0/1881887812 >> 172.21.15.29:0/1881887812 conn(0x7f6f740c8bb0 msgr2=0x7f6f741537c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.633+0000 7f6f7962e700 1 -- 172.21.15.29:0/1881887812 shutdown_connections 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.633+0000 7f6f7962e700 1 -- 172.21.15.29:0/1881887812 wait complete. 2024-01-24T14:14:08.638 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:14:08.648 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-01-24T14:14:08.108721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2423740,"kb_used_data":10108,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752550980,"statfs":{"total":773094113280,"available":770612203520,"internally_reserved":0,"allocated":10350592,"data_stored":5683203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11128,"num_objects":-8,"num_object_clones":0,"num_object_copies":-16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-35,"num_write_kb":-28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002022"},"pg_stats":[{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615363+0000","last_change":"2024-01-24T14:13:59.610904+0000","last_active":"2024-01-24T14:13:59.615363+0000","last_peered":"2024-01-24T14:13:59.615363+0000","last_clean":"2024-01-24T14:13:59.615363+0000","last_became_active":"2024-01-24T14:13:59.610672+0000","last_became_peered":"2024-01-24T14:13:59.610672+0000","last_unstale":"2024-01-24T14:13:59.615363+0000","last_undegraded":"2024-01-24T14:13:59.615363+0000","last_fullsized":"2024-01-24T14:13:59.615363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3f","version":"104'3","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616800+0000","last_change":"2024-01-24T14:13:59.174978+0000","last_active":"2024-01-24T14:13:59.616800+0000","last_peered":"2024-01-24T14:13:59.616800+0000","last_clean":"2024-01-24T14:13:59.616800+0000","last_became_active":"2024-01-24T14:13:59.174684+0000","last_became_peered":"2024-01-24T14:13:59.174684+0000","last_unstale":"2024-01-24T14:13:59.616800+0000","last_undegraded":"2024-01-24T14:13:59.616800+0000","last_fullsized":"2024-01-24T14:13:59.616800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605090+0000","last_change":"2024-01-24T14:13:59.600282+0000","last_active":"2024-01-24T14:13:59.605090+0000","last_peered":"2024-01-24T14:13:59.605090+0000","last_clean":"2024-01-24T14:13:59.605090+0000","last_became_active":"2024-01-24T14:13:59.600004+0000","last_became_peered":"2024-01-24T14:13:59.600004+0000","last_unstale":"2024-01-24T14:13:59.605090+0000","last_undegraded":"2024-01-24T14:13:59.605090+0000","last_fullsized":"2024-01-24T14:13:59.605090+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.600650+0000","last_change":"2024-01-24T14:13:59.171713+0000","last_active":"2024-01-24T14:13:59.600650+0000","last_peered":"2024-01-24T14:13:59.600650+0000","last_clean":"2024-01-24T14:13:59.600650+0000","last_became_active":"2024-01-24T14:13:59.171407+0000","last_became_peered":"2024-01-24T14:13:59.171407+0000","last_unstale":"2024-01-24T14:13:59.600650+0000","last_undegraded":"2024-01-24T14:13:59.600650+0000","last_fullsized":"2024-01-24T14:13:59.600650+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599212+0000","last_change":"2024-01-24T14:13:59.596129+0000","last_active":"2024-01-24T14:13:59.599212+0000","last_peered":"2024-01-24T14:13:59.599212+0000","last_clean":"2024-01-24T14:13:59.599212+0000","last_became_active":"2024-01-24T14:13:59.595830+0000","last_became_peered":"2024-01-24T14:13:59.595830+0000","last_unstale":"2024-01-24T14:13:59.599212+0000","last_undegraded":"2024-01-24T14:13:59.599212+0000","last_fullsized":"2024-01-24T14:13:59.599212+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591337+0000","last_change":"2024-01-24T14:13:59.190558+0000","last_active":"2024-01-24T14:13:59.591337+0000","last_peered":"2024-01-24T14:13:59.591337+0000","last_clean":"2024-01-24T14:13:59.591337+0000","last_became_active":"2024-01-24T14:13:59.187620+0000","last_became_peered":"2024-01-24T14:13:59.187620+0000","last_unstale":"2024-01-24T14:13:59.591337+0000","last_undegraded":"2024-01-24T14:13:59.591337+0000","last_fullsized":"2024-01-24T14:13:59.591337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616684+0000","last_change":"2024-01-24T14:13:59.609750+0000","last_active":"2024-01-24T14:13:59.616684+0000","last_peered":"2024-01-24T14:13:59.616684+0000","last_clean":"2024-01-24T14:13:59.616684+0000","last_became_active":"2024-01-24T14:13:59.609455+0000","last_became_peered":"2024-01-24T14:13:59.609455+0000","last_unstale":"2024-01-24T14:13:59.616684+0000","last_undegraded":"2024-01-24T14:13:59.616684+0000","last_fullsized":"2024-01-24T14:13:59.616684+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3c","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599540+0000","last_change":"2024-01-24T14:13:59.184784+0000","last_active":"2024-01-24T14:13:59.599540+0000","last_peered":"2024-01-24T14:13:59.599540+0000","last_clean":"2024-01-24T14:13:59.599540+0000","last_became_active":"2024-01-24T14:13:59.184482+0000","last_became_peered":"2024-01-24T14:13:59.184482+0000","last_unstale":"2024-01-24T14:13:59.599540+0000","last_undegraded":"2024-01-24T14:13:59.599540+0000","last_fullsized":"2024-01-24T14:13:59.599540+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602293+0000","last_change":"2024-01-24T14:13:59.596092+0000","last_active":"2024-01-24T14:13:59.602293+0000","last_peered":"2024-01-24T14:13:59.602293+0000","last_clean":"2024-01-24T14:13:59.602293+0000","last_became_active":"2024-01-24T14:13:59.595791+0000","last_became_peered":"2024-01-24T14:13:59.595791+0000","last_unstale":"2024-01-24T14:13:59.602293+0000","last_undegraded":"2024-01-24T14:13:59.602293+0000","last_fullsized":"2024-01-24T14:13:59.602293+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593252+0000","last_change":"2024-01-24T14:13:59.173164+0000","last_active":"2024-01-24T14:13:59.593252+0000","last_peered":"2024-01-24T14:13:59.593252+0000","last_clean":"2024-01-24T14:13:59.593252+0000","last_became_active":"2024-01-24T14:13:59.172816+0000","last_became_peered":"2024-01-24T14:13:59.172816+0000","last_unstale":"2024-01-24T14:13:59.593252+0000","last_undegraded":"2024-01-24T14:13:59.593252+0000","last_fullsized":"2024-01-24T14:13:59.593252+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602472+0000","last_change":"2024-01-24T14:13:59.599516+0000","last_active":"2024-01-24T14:13:59.602472+0000","last_peered":"2024-01-24T14:13:59.602472+0000","last_clean":"2024-01-24T14:13:59.602472+0000","last_became_active":"2024-01-24T14:13:59.599234+0000","last_became_peered":"2024-01-24T14:13:59.599234+0000","last_unstale":"2024-01-24T14:13:59.602472+0000","last_undegraded":"2024-01-24T14:13:59.602472+0000","last_fullsized":"2024-01-24T14:13:59.602472+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.3a","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.611338+0000","last_change":"2024-01-24T14:13:59.180992+0000","last_active":"2024-01-24T14:13:59.611338+0000","last_peered":"2024-01-24T14:13:59.611338+0000","last_clean":"2024-01-24T14:13:59.611338+0000","last_became_active":"2024-01-24T14:13:59.180627+0000","last_became_peered":"2024-01-24T14:13:59.180627+0000","last_unstale":"2024-01-24T14:13:59.611338+0000","last_undegraded":"2024-01-24T14:13:59.611338+0000","last_fullsized":"2024-01-24T14:13:59.611338+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602256+0000","last_change":"2024-01-24T14:13:59.596465+0000","last_active":"2024-01-24T14:13:59.602256+0000","last_peered":"2024-01-24T14:13:59.602256+0000","last_clean":"2024-01-24T14:13:59.602256+0000","last_became_active":"2024-01-24T14:13:59.596185+0000","last_became_peered":"2024-01-24T14:13:59.596185+0000","last_unstale":"2024-01-24T14:13:59.602256+0000","last_undegraded":"2024-01-24T14:13:59.602256+0000","last_fullsized":"2024-01-24T14:13:59.602256+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.590664+0000","last_change":"2024-01-24T14:13:59.179164+0000","last_active":"2024-01-24T14:13:59.590664+0000","last_peered":"2024-01-24T14:13:59.590664+0000","last_clean":"2024-01-24T14:13:59.590664+0000","last_became_active":"2024-01-24T14:13:59.178543+0000","last_became_peered":"2024-01-24T14:13:59.178543+0000","last_unstale":"2024-01-24T14:13:59.590664+0000","last_undegraded":"2024-01-24T14:13:59.590664+0000","last_fullsized":"2024-01-24T14:13:59.590664+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612433+0000","last_change":"2024-01-24T14:13:59.603989+0000","last_active":"2024-01-24T14:13:59.612433+0000","last_peered":"2024-01-24T14:13:59.612433+0000","last_clean":"2024-01-24T14:13:59.612433+0000","last_became_active":"2024-01-24T14:13:59.603818+0000","last_became_peered":"2024-01-24T14:13:59.603818+0000","last_unstale":"2024-01-24T14:13:59.612433+0000","last_undegraded":"2024-01-24T14:13:59.612433+0000","last_fullsized":"2024-01-24T14:13:59.612433+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581206+0000","last_change":"2024-01-24T14:13:59.168769+0000","last_active":"2024-01-24T14:13:59.581206+0000","last_peered":"2024-01-24T14:13:59.581206+0000","last_clean":"2024-01-24T14:13:59.581206+0000","last_became_active":"2024-01-24T14:13:59.168470+0000","last_became_peered":"2024-01-24T14:13:59.168470+0000","last_unstale":"2024-01-24T14:13:59.581206+0000","last_undegraded":"2024-01-24T14:13:59.581206+0000","last_fullsized":"2024-01-24T14:13:59.581206+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613168+0000","last_change":"2024-01-24T14:13:59.605311+0000","last_active":"2024-01-24T14:13:59.613168+0000","last_peered":"2024-01-24T14:13:59.613168+0000","last_clean":"2024-01-24T14:13:59.613168+0000","last_became_active":"2024-01-24T14:13:59.604996+0000","last_became_peered":"2024-01-24T14:13:59.604996+0000","last_unstale":"2024-01-24T14:13:59.613168+0000","last_undegraded":"2024-01-24T14:13:59.613168+0000","last_fullsized":"2024-01-24T14:13:59.613168+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588758+0000","last_change":"2024-01-24T14:13:59.173631+0000","last_active":"2024-01-24T14:13:59.588758+0000","last_peered":"2024-01-24T14:13:59.588758+0000","last_clean":"2024-01-24T14:13:59.588758+0000","last_became_active":"2024-01-24T14:13:59.173365+0000","last_became_peered":"2024-01-24T14:13:59.173365+0000","last_unstale":"2024-01-24T14:13:59.588758+0000","last_undegraded":"2024-01-24T14:13:59.588758+0000","last_fullsized":"2024-01-24T14:13:59.588758+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602306+0000","last_change":"2024-01-24T14:13:59.597783+0000","last_active":"2024-01-24T14:13:59.602306+0000","last_peered":"2024-01-24T14:13:59.602306+0000","last_clean":"2024-01-24T14:13:59.602306+0000","last_became_active":"2024-01-24T14:13:59.597507+0000","last_became_peered":"2024-01-24T14:13:59.597507+0000","last_unstale":"2024-01-24T14:13:59.602306+0000","last_undegraded":"2024-01-24T14:13:59.602306+0000","last_fullsized":"2024-01-24T14:13:59.602306+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585624+0000","last_change":"2024-01-24T14:13:59.185609+0000","last_active":"2024-01-24T14:13:59.585624+0000","last_peered":"2024-01-24T14:13:59.585624+0000","last_clean":"2024-01-24T14:13:59.585624+0000","last_became_active":"2024-01-24T14:13:59.185272+0000","last_became_peered":"2024-01-24T14:13:59.185272+0000","last_unstale":"2024-01-24T14:13:59.585624+0000","last_undegraded":"2024-01-24T14:13:59.585624+0000","last_fullsized":"2024-01-24T14:13:59.585624+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613203+0000","last_change":"2024-01-24T14:13:59.607137+0000","last_active":"2024-01-24T14:13:59.613203+0000","last_peered":"2024-01-24T14:13:59.613203+0000","last_clean":"2024-01-24T14:13:59.613203+0000","last_became_active":"2024-01-24T14:13:59.606837+0000","last_became_peered":"2024-01-24T14:13:59.606837+0000","last_unstale":"2024-01-24T14:13:59.613203+0000","last_undegraded":"2024-01-24T14:13:59.613203+0000","last_fullsized":"2024-01-24T14:13:59.613203+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597239+0000","last_change":"2024-01-24T14:13:59.174329+0000","last_active":"2024-01-24T14:13:59.597239+0000","last_peered":"2024-01-24T14:13:59.597239+0000","last_clean":"2024-01-24T14:13:59.597239+0000","last_became_active":"2024-01-24T14:13:59.173356+0000","last_became_peered":"2024-01-24T14:13:59.173356+0000","last_unstale":"2024-01-24T14:13:59.597239+0000","last_undegraded":"2024-01-24T14:13:59.597239+0000","last_fullsized":"2024-01-24T14:13:59.597239+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.614739+0000","last_change":"2024-01-24T14:13:59.602824+0000","last_active":"2024-01-24T14:13:59.614739+0000","last_peered":"2024-01-24T14:13:59.614739+0000","last_clean":"2024-01-24T14:13:59.614739+0000","last_became_active":"2024-01-24T14:13:59.602330+0000","last_became_peered":"2024-01-24T14:13:59.602330+0000","last_unstale":"2024-01-24T14:13:59.614739+0000","last_undegraded":"2024-01-24T14:13:59.614739+0000","last_fullsized":"2024-01-24T14:13:59.614739+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616384+0000","last_change":"2024-01-24T14:13:59.191263+0000","last_active":"2024-01-24T14:13:59.616384+0000","last_peered":"2024-01-24T14:13:59.616384+0000","last_clean":"2024-01-24T14:13:59.616384+0000","last_became_active":"2024-01-24T14:13:59.191014+0000","last_became_peered":"2024-01-24T14:13:59.191014+0000","last_unstale":"2024-01-24T14:13:59.616384+0000","last_undegraded":"2024-01-24T14:13:59.616384+0000","last_fullsized":"2024-01-24T14:13:59.616384+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612022+0000","last_change":"2024-01-24T14:13:59.603458+0000","last_active":"2024-01-24T14:13:59.612022+0000","last_peered":"2024-01-24T14:13:59.612022+0000","last_clean":"2024-01-24T14:13:59.612022+0000","last_became_active":"2024-01-24T14:13:59.600085+0000","last_became_peered":"2024-01-24T14:13:59.600085+0000","last_unstale":"2024-01-24T14:13:59.612022+0000","last_undegraded":"2024-01-24T14:13:59.612022+0000","last_fullsized":"2024-01-24T14:13:59.612022+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604880+0000","last_change":"2024-01-24T14:13:59.186304+0000","last_active":"2024-01-24T14:13:59.604880+0000","last_peered":"2024-01-24T14:13:59.604880+0000","last_clean":"2024-01-24T14:13:59.604880+0000","last_became_active":"2024-01-24T14:13:59.186017+0000","last_became_peered":"2024-01-24T14:13:59.186017+0000","last_unstale":"2024-01-24T14:13:59.604880+0000","last_undegraded":"2024-01-24T14:13:59.604880+0000","last_fullsized":"2024-01-24T14:13:59.604880+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613474+0000","last_change":"2024-01-24T14:13:59.603461+0000","last_active":"2024-01-24T14:13:59.613474+0000","last_peered":"2024-01-24T14:13:59.613474+0000","last_clean":"2024-01-24T14:13:59.613474+0000","last_became_active":"2024-01-24T14:13:59.603177+0000","last_became_peered":"2024-01-24T14:13:59.603177+0000","last_unstale":"2024-01-24T14:13:59.613474+0000","last_undegraded":"2024-01-24T14:13:59.613474+0000","last_fullsized":"2024-01-24T14:13:59.613474+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585657+0000","last_change":"2024-01-24T14:13:59.181115+0000","last_active":"2024-01-24T14:13:59.585657+0000","last_peered":"2024-01-24T14:13:59.585657+0000","last_clean":"2024-01-24T14:13:59.585657+0000","last_became_active":"2024-01-24T14:13:59.180641+0000","last_became_peered":"2024-01-24T14:13:59.180641+0000","last_unstale":"2024-01-24T14:13:59.585657+0000","last_undegraded":"2024-01-24T14:13:59.585657+0000","last_fullsized":"2024-01-24T14:13:59.585657+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.606266+0000","last_change":"2024-01-24T14:13:59.602768+0000","last_active":"2024-01-24T14:13:59.606266+0000","last_peered":"2024-01-24T14:13:59.606266+0000","last_clean":"2024-01-24T14:13:59.606266+0000","last_became_active":"2024-01-24T14:13:59.602510+0000","last_became_peered":"2024-01-24T14:13:59.602510+0000","last_unstale":"2024-01-24T14:13:59.606266+0000","last_undegraded":"2024-01-24T14:13:59.606266+0000","last_fullsized":"2024-01-24T14:13:59.606266+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596930+0000","last_change":"2024-01-24T14:13:59.188810+0000","last_active":"2024-01-24T14:13:59.596930+0000","last_peered":"2024-01-24T14:13:59.596930+0000","last_clean":"2024-01-24T14:13:59.596930+0000","last_became_active":"2024-01-24T14:13:59.188475+0000","last_became_peered":"2024-01-24T14:13:59.188475+0000","last_unstale":"2024-01-24T14:13:59.596930+0000","last_undegraded":"2024-01-24T14:13:59.596930+0000","last_fullsized":"2024-01-24T14:13:59.596930+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.608903+0000","last_change":"2024-01-24T14:13:59.604713+0000","last_active":"2024-01-24T14:13:59.608903+0000","last_peered":"2024-01-24T14:13:59.608903+0000","last_clean":"2024-01-24T14:13:59.608903+0000","last_became_active":"2024-01-24T14:13:59.604382+0000","last_became_peered":"2024-01-24T14:13:59.604382+0000","last_unstale":"2024-01-24T14:13:59.608903+0000","last_undegraded":"2024-01-24T14:13:59.608903+0000","last_fullsized":"2024-01-24T14:13:59.608903+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.30","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616811+0000","last_change":"2024-01-24T14:13:59.176815+0000","last_active":"2024-01-24T14:13:59.616811+0000","last_peered":"2024-01-24T14:13:59.616811+0000","last_clean":"2024-01-24T14:13:59.616811+0000","last_became_active":"2024-01-24T14:13:59.176478+0000","last_became_peered":"2024-01-24T14:13:59.176478+0000","last_unstale":"2024-01-24T14:13:59.616811+0000","last_undegraded":"2024-01-24T14:13:59.616811+0000","last_fullsized":"2024-01-24T14:13:59.616811+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612937+0000","last_change":"2024-01-24T14:13:59.603476+0000","last_active":"2024-01-24T14:13:59.612937+0000","last_peered":"2024-01-24T14:13:59.612937+0000","last_clean":"2024-01-24T14:13:59.612937+0000","last_became_active":"2024-01-24T14:13:59.603267+0000","last_became_peered":"2024-01-24T14:13:59.603267+0000","last_unstale":"2024-01-24T14:13:59.612937+0000","last_undegraded":"2024-01-24T14:13:59.612937+0000","last_fullsized":"2024-01-24T14:13:59.612937+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.583071+0000","last_change":"2024-01-24T14:13:59.174673+0000","last_active":"2024-01-24T14:13:59.583071+0000","last_peered":"2024-01-24T14:13:59.583071+0000","last_clean":"2024-01-24T14:13:59.583071+0000","last_became_active":"2024-01-24T14:13:59.174365+0000","last_became_peered":"2024-01-24T14:13:59.174365+0000","last_unstale":"2024-01-24T14:13:59.583071+0000","last_undegraded":"2024-01-24T14:13:59.583071+0000","last_fullsized":"2024-01-24T14:13:59.583071+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.603045+0000","last_change":"2024-01-24T14:13:59.598989+0000","last_active":"2024-01-24T14:13:59.603045+0000","last_peered":"2024-01-24T14:13:59.603045+0000","last_clean":"2024-01-24T14:13:59.603045+0000","last_became_active":"2024-01-24T14:13:59.598687+0000","last_became_peered":"2024-01-24T14:13:59.598687+0000","last_unstale":"2024-01-24T14:13:59.603045+0000","last_undegraded":"2024-01-24T14:13:59.603045+0000","last_fullsized":"2024-01-24T14:13:59.603045+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591250+0000","last_change":"2024-01-24T14:13:59.184862+0000","last_active":"2024-01-24T14:13:59.591250+0000","last_peered":"2024-01-24T14:13:59.591250+0000","last_clean":"2024-01-24T14:13:59.591250+0000","last_became_active":"2024-01-24T14:13:59.184560+0000","last_became_peered":"2024-01-24T14:13:59.184560+0000","last_unstale":"2024-01-24T14:13:59.591250+0000","last_undegraded":"2024-01-24T14:13:59.591250+0000","last_fullsized":"2024-01-24T14:13:59.591250+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599235+0000","last_change":"2024-01-24T14:13:59.597440+0000","last_active":"2024-01-24T14:13:59.599235+0000","last_peered":"2024-01-24T14:13:59.599235+0000","last_clean":"2024-01-24T14:13:59.599235+0000","last_became_active":"2024-01-24T14:13:59.597132+0000","last_became_peered":"2024-01-24T14:13:59.597132+0000","last_unstale":"2024-01-24T14:13:59.599235+0000","last_undegraded":"2024-01-24T14:13:59.599235+0000","last_fullsized":"2024-01-24T14:13:59.599235+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2d","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612688+0000","last_change":"2024-01-24T14:13:59.177820+0000","last_active":"2024-01-24T14:13:59.612688+0000","last_peered":"2024-01-24T14:13:59.612688+0000","last_clean":"2024-01-24T14:13:59.612688+0000","last_became_active":"2024-01-24T14:13:59.177501+0000","last_became_peered":"2024-01-24T14:13:59.177501+0000","last_unstale":"2024-01-24T14:13:59.612688+0000","last_undegraded":"2024-01-24T14:13:59.612688+0000","last_fullsized":"2024-01-24T14:13:59.612688+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613916+0000","last_change":"2024-01-24T14:13:59.607979+0000","last_active":"2024-01-24T14:13:59.613916+0000","last_peered":"2024-01-24T14:13:59.613916+0000","last_clean":"2024-01-24T14:13:59.613916+0000","last_became_active":"2024-01-24T14:13:59.607684+0000","last_became_peered":"2024-01-24T14:13:59.607684+0000","last_unstale":"2024-01-24T14:13:59.613916+0000","last_undegraded":"2024-01-24T14:13:59.613916+0000","last_fullsized":"2024-01-24T14:13:59.613916+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.582496+0000","last_change":"2024-01-24T14:13:59.185072+0000","last_active":"2024-01-24T14:13:59.582496+0000","last_peered":"2024-01-24T14:13:59.582496+0000","last_clean":"2024-01-24T14:13:59.582496+0000","last_became_active":"2024-01-24T14:13:59.184040+0000","last_became_peered":"2024-01-24T14:13:59.184040+0000","last_unstale":"2024-01-24T14:13:59.582496+0000","last_undegraded":"2024-01-24T14:13:59.582496+0000","last_fullsized":"2024-01-24T14:13:59.582496+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599036+0000","last_change":"2024-01-24T14:13:59.597766+0000","last_active":"2024-01-24T14:13:59.599036+0000","last_peered":"2024-01-24T14:13:59.599036+0000","last_clean":"2024-01-24T14:13:59.599036+0000","last_became_active":"2024-01-24T14:13:59.597476+0000","last_became_peered":"2024-01-24T14:13:59.597476+0000","last_unstale":"2024-01-24T14:13:59.599036+0000","last_undegraded":"2024-01-24T14:13:59.599036+0000","last_fullsized":"2024-01-24T14:13:59.599036+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599653+0000","last_change":"2024-01-24T14:13:59.181186+0000","last_active":"2024-01-24T14:13:59.599653+0000","last_peered":"2024-01-24T14:13:59.599653+0000","last_clean":"2024-01-24T14:13:59.599653+0000","last_became_active":"2024-01-24T14:13:59.180838+0000","last_became_peered":"2024-01-24T14:13:59.180838+0000","last_unstale":"2024-01-24T14:13:59.599653+0000","last_undegraded":"2024-01-24T14:13:59.599653+0000","last_fullsized":"2024-01-24T14:13:59.599653+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602220+0000","last_change":"2024-01-24T14:13:59.598055+0000","last_active":"2024-01-24T14:13:59.602220+0000","last_peered":"2024-01-24T14:13:59.602220+0000","last_clean":"2024-01-24T14:13:59.602220+0000","last_became_active":"2024-01-24T14:13:59.597777+0000","last_became_peered":"2024-01-24T14:13:59.597777+0000","last_unstale":"2024-01-24T14:13:59.602220+0000","last_undegraded":"2024-01-24T14:13:59.602220+0000","last_fullsized":"2024-01-24T14:13:59.602220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581576+0000","last_change":"2024-01-24T14:13:59.185733+0000","last_active":"2024-01-24T14:13:59.581576+0000","last_peered":"2024-01-24T14:13:59.581576+0000","last_clean":"2024-01-24T14:13:59.581576+0000","last_became_active":"2024-01-24T14:13:59.185482+0000","last_became_peered":"2024-01-24T14:13:59.185482+0000","last_unstale":"2024-01-24T14:13:59.581576+0000","last_undegraded":"2024-01-24T14:13:59.581576+0000","last_fullsized":"2024-01-24T14:13:59.581576+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.599233+0000","last_change":"2024-01-24T14:13:59.597585+0000","last_active":"2024-01-24T14:13:59.599233+0000","last_peered":"2024-01-24T14:13:59.599233+0000","last_clean":"2024-01-24T14:13:59.599233+0000","last_became_active":"2024-01-24T14:13:59.597114+0000","last_became_peered":"2024-01-24T14:13:59.597114+0000","last_unstale":"2024-01-24T14:13:59.599233+0000","last_undegraded":"2024-01-24T14:13:59.599233+0000","last_fullsized":"2024-01-24T14:13:59.599233+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.587957+0000","last_change":"2024-01-24T14:13:59.180890+0000","last_active":"2024-01-24T14:13:59.587957+0000","last_peered":"2024-01-24T14:13:59.587957+0000","last_clean":"2024-01-24T14:13:59.587957+0000","last_became_active":"2024-01-24T14:13:59.180535+0000","last_became_peered":"2024-01-24T14:13:59.180535+0000","last_unstale":"2024-01-24T14:13:59.587957+0000","last_undegraded":"2024-01-24T14:13:59.587957+0000","last_fullsized":"2024-01-24T14:13:59.587957+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613282+0000","last_change":"2024-01-24T14:13:59.607645+0000","last_active":"2024-01-24T14:13:59.613282+0000","last_peered":"2024-01-24T14:13:59.613282+0000","last_clean":"2024-01-24T14:13:59.613282+0000","last_became_active":"2024-01-24T14:13:59.607354+0000","last_became_peered":"2024-01-24T14:13:59.607354+0000","last_unstale":"2024-01-24T14:13:59.613282+0000","last_undegraded":"2024-01-24T14:13:59.613282+0000","last_fullsized":"2024-01-24T14:13:59.613282+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597240+0000","last_change":"2024-01-24T14:13:59.187503+0000","last_active":"2024-01-24T14:13:59.597240+0000","last_peered":"2024-01-24T14:13:59.597240+0000","last_clean":"2024-01-24T14:13:59.597240+0000","last_became_active":"2024-01-24T14:13:59.187203+0000","last_became_peered":"2024-01-24T14:13:59.187203+0000","last_unstale":"2024-01-24T14:13:59.597240+0000","last_undegraded":"2024-01-24T14:13:59.597240+0000","last_fullsized":"2024-01-24T14:13:59.597240+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601777+0000","last_change":"2024-01-24T14:13:59.600823+0000","last_active":"2024-01-24T14:13:59.601777+0000","last_peered":"2024-01-24T14:13:59.601777+0000","last_clean":"2024-01-24T14:13:59.601777+0000","last_became_active":"2024-01-24T14:13:59.600322+0000","last_became_peered":"2024-01-24T14:13:59.600322+0000","last_unstale":"2024-01-24T14:13:59.601777+0000","last_undegraded":"2024-01-24T14:13:59.601777+0000","last_fullsized":"2024-01-24T14:13:59.601777+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597400+0000","last_change":"2024-01-24T14:13:59.187980+0000","last_active":"2024-01-24T14:13:59.597400+0000","last_peered":"2024-01-24T14:13:59.597400+0000","last_clean":"2024-01-24T14:13:59.597400+0000","last_became_active":"2024-01-24T14:13:59.187653+0000","last_became_peered":"2024-01-24T14:13:59.187653+0000","last_unstale":"2024-01-24T14:13:59.597400+0000","last_undegraded":"2024-01-24T14:13:59.597400+0000","last_fullsized":"2024-01-24T14:13:59.597400+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615401+0000","last_change":"2024-01-24T14:13:59.611324+0000","last_active":"2024-01-24T14:13:59.615401+0000","last_peered":"2024-01-24T14:13:59.615401+0000","last_clean":"2024-01-24T14:13:59.615401+0000","last_became_active":"2024-01-24T14:13:59.610992+0000","last_became_peered":"2024-01-24T14:13:59.610992+0000","last_unstale":"2024-01-24T14:13:59.615401+0000","last_undegraded":"2024-01-24T14:13:59.615401+0000","last_fullsized":"2024-01-24T14:13:59.615401+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.26","version":"105'1","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-01-24T14:14:00.909690+0000","last_change":"2024-01-24T14:13:59.183133+0000","last_active":"2024-01-24T14:14:00.909690+0000","last_peered":"2024-01-24T14:14:00.909690+0000","last_clean":"2024-01-24T14:14:00.909690+0000","last_became_active":"2024-01-24T14:13:59.182837+0000","last_became_peered":"2024-01-24T14:13:59.182837+0000","last_unstale":"2024-01-24T14:14:00.909690+0000","last_undegraded":"2024-01-24T14:14:00.909690+0000","last_fullsized":"2024-01-24T14:14:00.909690+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605252+0000","last_change":"2024-01-24T14:13:59.601562+0000","last_active":"2024-01-24T14:13:59.605252+0000","last_peered":"2024-01-24T14:13:59.605252+0000","last_clean":"2024-01-24T14:13:59.605252+0000","last_became_active":"2024-01-24T14:13:59.600361+0000","last_became_peered":"2024-01-24T14:13:59.600361+0000","last_unstale":"2024-01-24T14:13:59.605252+0000","last_undegraded":"2024-01-24T14:13:59.605252+0000","last_fullsized":"2024-01-24T14:13:59.605252+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596181+0000","last_change":"2024-01-24T14:13:59.176584+0000","last_active":"2024-01-24T14:13:59.596181+0000","last_peered":"2024-01-24T14:13:59.596181+0000","last_clean":"2024-01-24T14:13:59.596181+0000","last_became_active":"2024-01-24T14:13:59.176352+0000","last_became_peered":"2024-01-24T14:13:59.176352+0000","last_unstale":"2024-01-24T14:13:59.596181+0000","last_undegraded":"2024-01-24T14:13:59.596181+0000","last_fullsized":"2024-01-24T14:13:59.596181+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.606565+0000","last_change":"2024-01-24T14:13:59.605786+0000","last_active":"2024-01-24T14:13:59.606565+0000","last_peered":"2024-01-24T14:13:59.606565+0000","last_clean":"2024-01-24T14:13:59.606565+0000","last_became_active":"2024-01-24T14:13:59.605479+0000","last_became_peered":"2024-01-24T14:13:59.605479+0000","last_unstale":"2024-01-24T14:13:59.606565+0000","last_undegraded":"2024-01-24T14:13:59.606565+0000","last_fullsized":"2024-01-24T14:13:59.606565+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.582565+0000","last_change":"2024-01-24T14:13:59.177305+0000","last_active":"2024-01-24T14:13:59.582565+0000","last_peered":"2024-01-24T14:13:59.582565+0000","last_clean":"2024-01-24T14:13:59.582565+0000","last_became_active":"2024-01-24T14:13:59.177013+0000","last_became_peered":"2024-01-24T14:13:59.177013+0000","last_unstale":"2024-01-24T14:13:59.582565+0000","last_undegraded":"2024-01-24T14:13:59.582565+0000","last_fullsized":"2024-01-24T14:13:59.582565+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613859+0000","last_change":"2024-01-24T14:13:59.605686+0000","last_active":"2024-01-24T14:13:59.613859+0000","last_peered":"2024-01-24T14:13:59.613859+0000","last_clean":"2024-01-24T14:13:59.613859+0000","last_became_active":"2024-01-24T14:13:59.599468+0000","last_became_peered":"2024-01-24T14:13:59.599468+0000","last_unstale":"2024-01-24T14:13:59.613859+0000","last_undegraded":"2024-01-24T14:13:59.613859+0000","last_fullsized":"2024-01-24T14:13:59.613859+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597018+0000","last_change":"2024-01-24T14:13:59.176388+0000","last_active":"2024-01-24T14:13:59.597018+0000","last_peered":"2024-01-24T14:13:59.597018+0000","last_clean":"2024-01-24T14:13:59.597018+0000","last_became_active":"2024-01-24T14:13:59.176027+0000","last_became_peered":"2024-01-24T14:13:59.176027+0000","last_unstale":"2024-01-24T14:13:59.597018+0000","last_undegraded":"2024-01-24T14:13:59.597018+0000","last_fullsized":"2024-01-24T14:13:59.597018+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613713+0000","last_change":"2024-01-24T14:13:59.604787+0000","last_active":"2024-01-24T14:13:59.613713+0000","last_peered":"2024-01-24T14:13:59.613713+0000","last_clean":"2024-01-24T14:13:59.613713+0000","last_became_active":"2024-01-24T14:13:59.604506+0000","last_became_peered":"2024-01-24T14:13:59.604506+0000","last_unstale":"2024-01-24T14:13:59.613713+0000","last_undegraded":"2024-01-24T14:13:59.613713+0000","last_fullsized":"2024-01-24T14:13:59.613713+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593610+0000","last_change":"2024-01-24T14:13:59.177542+0000","last_active":"2024-01-24T14:13:59.593610+0000","last_peered":"2024-01-24T14:13:59.593610+0000","last_clean":"2024-01-24T14:13:59.593610+0000","last_became_active":"2024-01-24T14:13:59.177246+0000","last_became_peered":"2024-01-24T14:13:59.177246+0000","last_unstale":"2024-01-24T14:13:59.593610+0000","last_undegraded":"2024-01-24T14:13:59.593610+0000","last_fullsized":"2024-01-24T14:13:59.593610+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602015+0000","last_change":"2024-01-24T14:13:59.597273+0000","last_active":"2024-01-24T14:13:59.602015+0000","last_peered":"2024-01-24T14:13:59.602015+0000","last_clean":"2024-01-24T14:13:59.602015+0000","last_became_active":"2024-01-24T14:13:59.597053+0000","last_became_peered":"2024-01-24T14:13:59.597053+0000","last_unstale":"2024-01-24T14:13:59.602015+0000","last_undegraded":"2024-01-24T14:13:59.602015+0000","last_fullsized":"2024-01-24T14:13:59.602015+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591431+0000","last_change":"2024-01-24T14:13:59.187008+0000","last_active":"2024-01-24T14:13:59.591431+0000","last_peered":"2024-01-24T14:13:59.591431+0000","last_clean":"2024-01-24T14:13:59.591431+0000","last_became_active":"2024-01-24T14:13:59.186690+0000","last_became_peered":"2024-01-24T14:13:59.186690+0000","last_unstale":"2024-01-24T14:13:59.591431+0000","last_undegraded":"2024-01-24T14:13:59.591431+0000","last_fullsized":"2024-01-24T14:13:59.591431+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602216+0000","last_change":"2024-01-24T14:13:59.596466+0000","last_active":"2024-01-24T14:13:59.602216+0000","last_peered":"2024-01-24T14:13:59.602216+0000","last_clean":"2024-01-24T14:13:59.602216+0000","last_became_active":"2024-01-24T14:13:59.596179+0000","last_became_peered":"2024-01-24T14:13:59.596179+0000","last_unstale":"2024-01-24T14:13:59.602216+0000","last_undegraded":"2024-01-24T14:13:59.602216+0000","last_fullsized":"2024-01-24T14:13:59.602216+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594464+0000","last_change":"2024-01-24T14:13:59.175676+0000","last_active":"2024-01-24T14:13:59.594464+0000","last_peered":"2024-01-24T14:13:59.594464+0000","last_clean":"2024-01-24T14:13:59.594464+0000","last_became_active":"2024-01-24T14:13:59.175396+0000","last_became_peered":"2024-01-24T14:13:59.175396+0000","last_unstale":"2024-01-24T14:13:59.594464+0000","last_undegraded":"2024-01-24T14:13:59.594464+0000","last_fullsized":"2024-01-24T14:13:59.594464+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.606287+0000","last_change":"2024-01-24T14:13:59.602951+0000","last_active":"2024-01-24T14:13:59.606287+0000","last_peered":"2024-01-24T14:13:59.606287+0000","last_clean":"2024-01-24T14:13:59.606287+0000","last_became_active":"2024-01-24T14:13:59.602648+0000","last_became_peered":"2024-01-24T14:13:59.602648+0000","last_unstale":"2024-01-24T14:13:59.606287+0000","last_undegraded":"2024-01-24T14:13:59.606287+0000","last_fullsized":"2024-01-24T14:13:59.606287+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.600682+0000","last_change":"2024-01-24T14:13:59.179441+0000","last_active":"2024-01-24T14:13:59.600682+0000","last_peered":"2024-01-24T14:13:59.600682+0000","last_clean":"2024-01-24T14:13:59.600682+0000","last_became_active":"2024-01-24T14:13:59.178699+0000","last_became_peered":"2024-01-24T14:13:59.178699+0000","last_unstale":"2024-01-24T14:13:59.600682+0000","last_undegraded":"2024-01-24T14:13:59.600682+0000","last_fullsized":"2024-01-24T14:13:59.600682+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613325+0000","last_change":"2024-01-24T14:13:59.607754+0000","last_active":"2024-01-24T14:13:59.613325+0000","last_peered":"2024-01-24T14:13:59.613325+0000","last_clean":"2024-01-24T14:13:59.613325+0000","last_became_active":"2024-01-24T14:13:59.607437+0000","last_became_peered":"2024-01-24T14:13:59.607437+0000","last_unstale":"2024-01-24T14:13:59.613325+0000","last_undegraded":"2024-01-24T14:13:59.613325+0000","last_fullsized":"2024-01-24T14:13:59.613325+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605436+0000","last_change":"2024-01-24T14:13:59.180915+0000","last_active":"2024-01-24T14:13:59.605436+0000","last_peered":"2024-01-24T14:13:59.605436+0000","last_clean":"2024-01-24T14:13:59.605436+0000","last_became_active":"2024-01-24T14:13:59.180385+0000","last_became_peered":"2024-01-24T14:13:59.180385+0000","last_unstale":"2024-01-24T14:13:59.605436+0000","last_undegraded":"2024-01-24T14:13:59.605436+0000","last_fullsized":"2024-01-24T14:13:59.605436+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605231+0000","last_change":"2024-01-24T14:13:59.601292+0000","last_active":"2024-01-24T14:13:59.605231+0000","last_peered":"2024-01-24T14:13:59.605231+0000","last_clean":"2024-01-24T14:13:59.605231+0000","last_became_active":"2024-01-24T14:13:59.600834+0000","last_became_peered":"2024-01-24T14:13:59.600834+0000","last_unstale":"2024-01-24T14:13:59.605231+0000","last_undegraded":"2024-01-24T14:13:59.605231+0000","last_fullsized":"2024-01-24T14:13:59.605231+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615335+0000","last_change":"2024-01-24T14:13:59.175601+0000","last_active":"2024-01-24T14:13:59.615335+0000","last_peered":"2024-01-24T14:13:59.615335+0000","last_clean":"2024-01-24T14:13:59.615335+0000","last_became_active":"2024-01-24T14:13:59.173774+0000","last_became_peered":"2024-01-24T14:13:59.173774+0000","last_unstale":"2024-01-24T14:13:59.615335+0000","last_undegraded":"2024-01-24T14:13:59.615335+0000","last_fullsized":"2024-01-24T14:13:59.615335+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613559+0000","last_change":"2024-01-24T14:13:59.604227+0000","last_active":"2024-01-24T14:13:59.613559+0000","last_peered":"2024-01-24T14:13:59.613559+0000","last_clean":"2024-01-24T14:13:59.613559+0000","last_became_active":"2024-01-24T14:13:59.603933+0000","last_became_peered":"2024-01-24T14:13:59.603933+0000","last_unstale":"2024-01-24T14:13:59.613559+0000","last_undegraded":"2024-01-24T14:13:59.613559+0000","last_fullsized":"2024-01-24T14:13:59.613559+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597128+0000","last_change":"2024-01-24T14:13:59.169115+0000","last_active":"2024-01-24T14:13:59.597128+0000","last_peered":"2024-01-24T14:13:59.597128+0000","last_clean":"2024-01-24T14:13:59.597128+0000","last_became_active":"2024-01-24T14:13:59.168789+0000","last_became_peered":"2024-01-24T14:13:59.168789+0000","last_unstale":"2024-01-24T14:13:59.597128+0000","last_undegraded":"2024-01-24T14:13:59.597128+0000","last_fullsized":"2024-01-24T14:13:59.597128+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613839+0000","last_change":"2024-01-24T14:13:59.604215+0000","last_active":"2024-01-24T14:13:59.613839+0000","last_peered":"2024-01-24T14:13:59.613839+0000","last_clean":"2024-01-24T14:13:59.613839+0000","last_became_active":"2024-01-24T14:13:59.603918+0000","last_became_peered":"2024-01-24T14:13:59.603918+0000","last_unstale":"2024-01-24T14:13:59.613839+0000","last_undegraded":"2024-01-24T14:13:59.613839+0000","last_fullsized":"2024-01-24T14:13:59.613839+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588396+0000","last_change":"2024-01-24T14:13:59.181773+0000","last_active":"2024-01-24T14:13:59.588396+0000","last_peered":"2024-01-24T14:13:59.588396+0000","last_clean":"2024-01-24T14:13:59.588396+0000","last_became_active":"2024-01-24T14:13:59.181227+0000","last_became_peered":"2024-01-24T14:13:59.181227+0000","last_unstale":"2024-01-24T14:13:59.588396+0000","last_undegraded":"2024-01-24T14:13:59.588396+0000","last_fullsized":"2024-01-24T14:13:59.588396+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602292+0000","last_change":"2024-01-24T14:13:59.598283+0000","last_active":"2024-01-24T14:13:59.602292+0000","last_peered":"2024-01-24T14:13:59.602292+0000","last_clean":"2024-01-24T14:13:59.602292+0000","last_became_active":"2024-01-24T14:13:59.598020+0000","last_became_peered":"2024-01-24T14:13:59.598020+0000","last_unstale":"2024-01-24T14:13:59.602292+0000","last_undegraded":"2024-01-24T14:13:59.602292+0000","last_fullsized":"2024-01-24T14:13:59.602292+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588480+0000","last_change":"2024-01-24T14:13:59.180957+0000","last_active":"2024-01-24T14:13:59.588480+0000","last_peered":"2024-01-24T14:13:59.588480+0000","last_clean":"2024-01-24T14:13:59.588480+0000","last_became_active":"2024-01-24T14:13:59.180449+0000","last_became_peered":"2024-01-24T14:13:59.180449+0000","last_unstale":"2024-01-24T14:13:59.588480+0000","last_undegraded":"2024-01-24T14:13:59.588480+0000","last_fullsized":"2024-01-24T14:13:59.588480+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616746+0000","last_change":"2024-01-24T14:13:59.610454+0000","last_active":"2024-01-24T14:13:59.616746+0000","last_peered":"2024-01-24T14:13:59.616746+0000","last_clean":"2024-01-24T14:13:59.616746+0000","last_became_active":"2024-01-24T14:13:59.610190+0000","last_became_peered":"2024-01-24T14:13:59.610190+0000","last_unstale":"2024-01-24T14:13:59.616746+0000","last_undegraded":"2024-01-24T14:13:59.616746+0000","last_fullsized":"2024-01-24T14:13:59.616746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.600543+0000","last_change":"2024-01-24T14:13:59.177014+0000","last_active":"2024-01-24T14:13:59.600543+0000","last_peered":"2024-01-24T14:13:59.600543+0000","last_clean":"2024-01-24T14:13:59.600543+0000","last_became_active":"2024-01-24T14:13:59.176432+0000","last_became_peered":"2024-01-24T14:13:59.176432+0000","last_unstale":"2024-01-24T14:13:59.600543+0000","last_undegraded":"2024-01-24T14:13:59.600543+0000","last_fullsized":"2024-01-24T14:13:59.600543+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.608927+0000","last_change":"2024-01-24T14:13:59.604972+0000","last_active":"2024-01-24T14:13:59.608927+0000","last_peered":"2024-01-24T14:13:59.608927+0000","last_clean":"2024-01-24T14:13:59.608927+0000","last_became_active":"2024-01-24T14:13:59.604643+0000","last_became_peered":"2024-01-24T14:13:59.604643+0000","last_unstale":"2024-01-24T14:13:59.608927+0000","last_undegraded":"2024-01-24T14:13:59.608927+0000","last_fullsized":"2024-01-24T14:13:59.608927+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591156+0000","last_change":"2024-01-24T14:13:59.184839+0000","last_active":"2024-01-24T14:13:59.591156+0000","last_peered":"2024-01-24T14:13:59.591156+0000","last_clean":"2024-01-24T14:13:59.591156+0000","last_became_active":"2024-01-24T14:13:59.184535+0000","last_became_peered":"2024-01-24T14:13:59.184535+0000","last_unstale":"2024-01-24T14:13:59.591156+0000","last_undegraded":"2024-01-24T14:13:59.591156+0000","last_fullsized":"2024-01-24T14:13:59.591156+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604995+0000","last_change":"2024-01-24T14:13:59.600061+0000","last_active":"2024-01-24T14:13:59.604995+0000","last_peered":"2024-01-24T14:13:59.604995+0000","last_clean":"2024-01-24T14:13:59.604995+0000","last_became_active":"2024-01-24T14:13:59.599799+0000","last_became_peered":"2024-01-24T14:13:59.599799+0000","last_unstale":"2024-01-24T14:13:59.604995+0000","last_undegraded":"2024-01-24T14:13:59.604995+0000","last_fullsized":"2024-01-24T14:13:59.604995+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591563+0000","last_change":"2024-01-24T14:13:59.179655+0000","last_active":"2024-01-24T14:13:59.591563+0000","last_peered":"2024-01-24T14:13:59.591563+0000","last_clean":"2024-01-24T14:13:59.591563+0000","last_became_active":"2024-01-24T14:13:59.179068+0000","last_became_peered":"2024-01-24T14:13:59.179068+0000","last_unstale":"2024-01-24T14:13:59.591563+0000","last_undegraded":"2024-01-24T14:13:59.591563+0000","last_fullsized":"2024-01-24T14:13:59.591563+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602283+0000","last_change":"2024-01-24T14:13:59.594537+0000","last_active":"2024-01-24T14:13:59.602283+0000","last_peered":"2024-01-24T14:13:59.602283+0000","last_clean":"2024-01-24T14:13:59.602283+0000","last_became_active":"2024-01-24T14:13:59.594254+0000","last_became_peered":"2024-01-24T14:13:59.594254+0000","last_unstale":"2024-01-24T14:13:59.602283+0000","last_undegraded":"2024-01-24T14:13:59.602283+0000","last_fullsized":"2024-01-24T14:13:59.602283+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591800+0000","last_change":"2024-01-24T14:13:59.182373+0000","last_active":"2024-01-24T14:13:59.591800+0000","last_peered":"2024-01-24T14:13:59.591800+0000","last_clean":"2024-01-24T14:13:59.591800+0000","last_became_active":"2024-01-24T14:13:59.182077+0000","last_became_peered":"2024-01-24T14:13:59.182077+0000","last_unstale":"2024-01-24T14:13:59.591800+0000","last_undegraded":"2024-01-24T14:13:59.591800+0000","last_fullsized":"2024-01-24T14:13:59.591800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613637+0000","last_change":"2024-01-24T14:13:59.607656+0000","last_active":"2024-01-24T14:13:59.613637+0000","last_peered":"2024-01-24T14:13:59.613637+0000","last_clean":"2024-01-24T14:13:59.613637+0000","last_became_active":"2024-01-24T14:13:59.607365+0000","last_became_peered":"2024-01-24T14:13:59.607365+0000","last_unstale":"2024-01-24T14:13:59.613637+0000","last_undegraded":"2024-01-24T14:13:59.613637+0000","last_fullsized":"2024-01-24T14:13:59.613637+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"104'3","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601949+0000","last_change":"2024-01-24T14:13:59.182375+0000","last_active":"2024-01-24T14:13:59.601949+0000","last_peered":"2024-01-24T14:13:59.601949+0000","last_clean":"2024-01-24T14:13:59.601949+0000","last_became_active":"2024-01-24T14:13:59.182150+0000","last_became_peered":"2024-01-24T14:13:59.182150+0000","last_unstale":"2024-01-24T14:13:59.601949+0000","last_undegraded":"2024-01-24T14:13:59.601949+0000","last_fullsized":"2024-01-24T14:13:59.601949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.612014+0000","last_change":"2024-01-24T14:13:59.608414+0000","last_active":"2024-01-24T14:13:59.612014+0000","last_peered":"2024-01-24T14:13:59.612014+0000","last_clean":"2024-01-24T14:13:59.612014+0000","last_became_active":"2024-01-24T14:13:59.608013+0000","last_became_peered":"2024-01-24T14:13:59.608013+0000","last_unstale":"2024-01-24T14:13:59.612014+0000","last_undegraded":"2024-01-24T14:13:59.612014+0000","last_fullsized":"2024-01-24T14:13:59.612014+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.607061+0000","last_change":"2024-01-24T14:13:59.178726+0000","last_active":"2024-01-24T14:13:59.607061+0000","last_peered":"2024-01-24T14:13:59.607061+0000","last_clean":"2024-01-24T14:13:59.607061+0000","last_became_active":"2024-01-24T14:13:59.178463+0000","last_became_peered":"2024-01-24T14:13:59.178463+0000","last_unstale":"2024-01-24T14:13:59.607061+0000","last_undegraded":"2024-01-24T14:13:59.607061+0000","last_fullsized":"2024-01-24T14:13:59.607061+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.614134+0000","last_change":"2024-01-24T14:13:59.605906+0000","last_active":"2024-01-24T14:13:59.614134+0000","last_peered":"2024-01-24T14:13:59.614134+0000","last_clean":"2024-01-24T14:13:59.614134+0000","last_became_active":"2024-01-24T14:13:59.605404+0000","last_became_peered":"2024-01-24T14:13:59.605404+0000","last_unstale":"2024-01-24T14:13:59.614134+0000","last_undegraded":"2024-01-24T14:13:59.614134+0000","last_fullsized":"2024-01-24T14:13:59.614134+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613963+0000","last_change":"2024-01-24T14:13:59.174910+0000","last_active":"2024-01-24T14:13:59.613963+0000","last_peered":"2024-01-24T14:13:59.613963+0000","last_clean":"2024-01-24T14:13:59.613963+0000","last_became_active":"2024-01-24T14:13:59.174603+0000","last_became_peered":"2024-01-24T14:13:59.174603+0000","last_unstale":"2024-01-24T14:13:59.613963+0000","last_undegraded":"2024-01-24T14:13:59.613963+0000","last_fullsized":"2024-01-24T14:13:59.613963+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.618795+0000","last_change":"2024-01-24T14:13:59.614234+0000","last_active":"2024-01-24T14:13:59.618795+0000","last_peered":"2024-01-24T14:13:59.618795+0000","last_clean":"2024-01-24T14:13:59.618795+0000","last_became_active":"2024-01-24T14:13:59.614018+0000","last_became_peered":"2024-01-24T14:13:59.614018+0000","last_unstale":"2024-01-24T14:13:59.618795+0000","last_undegraded":"2024-01-24T14:13:59.618795+0000","last_fullsized":"2024-01-24T14:13:59.618795+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585537+0000","last_change":"2024-01-24T14:13:59.183528+0000","last_active":"2024-01-24T14:13:59.585537+0000","last_peered":"2024-01-24T14:13:59.585537+0000","last_clean":"2024-01-24T14:13:59.585537+0000","last_became_active":"2024-01-24T14:13:59.183214+0000","last_became_peered":"2024-01-24T14:13:59.183214+0000","last_unstale":"2024-01-24T14:13:59.585537+0000","last_undegraded":"2024-01-24T14:13:59.585537+0000","last_fullsized":"2024-01-24T14:13:59.585537+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613805+0000","last_change":"2024-01-24T14:13:59.605325+0000","last_active":"2024-01-24T14:13:59.613805+0000","last_peered":"2024-01-24T14:13:59.613805+0000","last_clean":"2024-01-24T14:13:59.613805+0000","last_became_active":"2024-01-24T14:13:59.605027+0000","last_became_peered":"2024-01-24T14:13:59.605027+0000","last_unstale":"2024-01-24T14:13:59.613805+0000","last_undegraded":"2024-01-24T14:13:59.613805+0000","last_fullsized":"2024-01-24T14:13:59.613805+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.586227+0000","last_change":"2024-01-24T14:13:59.182994+0000","last_active":"2024-01-24T14:13:59.586227+0000","last_peered":"2024-01-24T14:13:59.586227+0000","last_clean":"2024-01-24T14:13:59.586227+0000","last_became_active":"2024-01-24T14:13:59.182682+0000","last_became_peered":"2024-01-24T14:13:59.182682+0000","last_unstale":"2024-01-24T14:13:59.586227+0000","last_undegraded":"2024-01-24T14:13:59.586227+0000","last_fullsized":"2024-01-24T14:13:59.586227+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.615224+0000","last_change":"2024-01-24T14:13:59.610385+0000","last_active":"2024-01-24T14:13:59.615224+0000","last_peered":"2024-01-24T14:13:59.615224+0000","last_clean":"2024-01-24T14:13:59.615224+0000","last_became_active":"2024-01-24T14:13:59.609240+0000","last_became_peered":"2024-01-24T14:13:59.609240+0000","last_unstale":"2024-01-24T14:13:59.615224+0000","last_undegraded":"2024-01-24T14:13:59.615224+0000","last_fullsized":"2024-01-24T14:13:59.615224+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.617265+0000","last_change":"2024-01-24T14:13:59.180036+0000","last_active":"2024-01-24T14:13:59.617265+0000","last_peered":"2024-01-24T14:13:59.617265+0000","last_clean":"2024-01-24T14:13:59.617265+0000","last_became_active":"2024-01-24T14:13:59.179065+0000","last_became_peered":"2024-01-24T14:13:59.179065+0000","last_unstale":"2024-01-24T14:13:59.617265+0000","last_undegraded":"2024-01-24T14:13:59.617265+0000","last_fullsized":"2024-01-24T14:13:59.617265+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.603037+0000","last_change":"2024-01-24T14:13:59.600098+0000","last_active":"2024-01-24T14:13:59.603037+0000","last_peered":"2024-01-24T14:13:59.603037+0000","last_clean":"2024-01-24T14:13:59.603037+0000","last_became_active":"2024-01-24T14:13:59.599838+0000","last_became_peered":"2024-01-24T14:13:59.599838+0000","last_unstale":"2024-01-24T14:13:59.603037+0000","last_undegraded":"2024-01-24T14:13:59.603037+0000","last_fullsized":"2024-01-24T14:13:59.603037+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.601872+0000","last_change":"2024-01-24T14:13:59.178489+0000","last_active":"2024-01-24T14:13:59.601872+0000","last_peered":"2024-01-24T14:13:59.601872+0000","last_clean":"2024-01-24T14:13:59.601872+0000","last_became_active":"2024-01-24T14:13:59.178185+0000","last_became_peered":"2024-01-24T14:13:59.178185+0000","last_unstale":"2024-01-24T14:13:59.601872+0000","last_undegraded":"2024-01-24T14:13:59.601872+0000","last_fullsized":"2024-01-24T14:13:59.601872+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616653+0000","last_change":"2024-01-24T14:13:59.609527+0000","last_active":"2024-01-24T14:13:59.616653+0000","last_peered":"2024-01-24T14:13:59.616653+0000","last_clean":"2024-01-24T14:13:59.616653+0000","last_became_active":"2024-01-24T14:13:59.609180+0000","last_became_peered":"2024-01-24T14:13:59.609180+0000","last_unstale":"2024-01-24T14:13:59.616653+0000","last_undegraded":"2024-01-24T14:13:59.616653+0000","last_fullsized":"2024-01-24T14:13:59.616653+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591353+0000","last_change":"2024-01-24T14:13:59.187223+0000","last_active":"2024-01-24T14:13:59.591353+0000","last_peered":"2024-01-24T14:13:59.591353+0000","last_clean":"2024-01-24T14:13:59.591353+0000","last_became_active":"2024-01-24T14:13:59.186908+0000","last_became_peered":"2024-01-24T14:13:59.186908+0000","last_unstale":"2024-01-24T14:13:59.591353+0000","last_undegraded":"2024-01-24T14:13:59.591353+0000","last_fullsized":"2024-01-24T14:13:59.591353+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604295+0000","last_change":"2024-01-24T14:13:59.597776+0000","last_active":"2024-01-24T14:13:59.604295+0000","last_peered":"2024-01-24T14:13:59.604295+0000","last_clean":"2024-01-24T14:13:59.604295+0000","last_became_active":"2024-01-24T14:13:59.597468+0000","last_became_peered":"2024-01-24T14:13:59.597468+0000","last_unstale":"2024-01-24T14:13:59.604295+0000","last_undegraded":"2024-01-24T14:13:59.604295+0000","last_fullsized":"2024-01-24T14:13:59.604295+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588522+0000","last_change":"2024-01-24T14:13:59.169565+0000","last_active":"2024-01-24T14:13:59.588522+0000","last_peered":"2024-01-24T14:13:59.588522+0000","last_clean":"2024-01-24T14:13:59.588522+0000","last_became_active":"2024-01-24T14:13:59.169300+0000","last_became_peered":"2024-01-24T14:13:59.169300+0000","last_unstale":"2024-01-24T14:13:59.588522+0000","last_undegraded":"2024-01-24T14:13:59.588522+0000","last_fullsized":"2024-01-24T14:13:59.588522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.610569+0000","last_change":"2024-01-24T14:13:59.607765+0000","last_active":"2024-01-24T14:13:59.610569+0000","last_peered":"2024-01-24T14:13:59.610569+0000","last_clean":"2024-01-24T14:13:59.610569+0000","last_became_active":"2024-01-24T14:13:59.607462+0000","last_became_peered":"2024-01-24T14:13:59.607462+0000","last_unstale":"2024-01-24T14:13:59.610569+0000","last_undegraded":"2024-01-24T14:13:59.610569+0000","last_fullsized":"2024-01-24T14:13:59.610569+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.590394+0000","last_change":"2024-01-24T14:13:59.179494+0000","last_active":"2024-01-24T14:13:59.590394+0000","last_peered":"2024-01-24T14:13:59.590394+0000","last_clean":"2024-01-24T14:13:59.590394+0000","last_became_active":"2024-01-24T14:13:59.178687+0000","last_became_peered":"2024-01-24T14:13:59.178687+0000","last_unstale":"2024-01-24T14:13:59.590394+0000","last_undegraded":"2024-01-24T14:13:59.590394+0000","last_fullsized":"2024-01-24T14:13:59.590394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.598793+0000","last_change":"2024-01-24T14:13:59.592141+0000","last_active":"2024-01-24T14:13:59.598793+0000","last_peered":"2024-01-24T14:13:59.598793+0000","last_clean":"2024-01-24T14:13:59.598793+0000","last_became_active":"2024-01-24T14:13:59.590960+0000","last_became_peered":"2024-01-24T14:13:59.590960+0000","last_unstale":"2024-01-24T14:13:59.598793+0000","last_undegraded":"2024-01-24T14:13:59.598793+0000","last_fullsized":"2024-01-24T14:13:59.598793+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597174+0000","last_change":"2024-01-24T14:13:59.186128+0000","last_active":"2024-01-24T14:13:59.597174+0000","last_peered":"2024-01-24T14:13:59.597174+0000","last_clean":"2024-01-24T14:13:59.597174+0000","last_became_active":"2024-01-24T14:13:59.177094+0000","last_became_peered":"2024-01-24T14:13:59.177094+0000","last_unstale":"2024-01-24T14:13:59.597174+0000","last_undegraded":"2024-01-24T14:13:59.597174+0000","last_fullsized":"2024-01-24T14:13:59.597174+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613687+0000","last_change":"2024-01-24T14:13:59.606315+0000","last_active":"2024-01-24T14:13:59.613687+0000","last_peered":"2024-01-24T14:13:59.613687+0000","last_clean":"2024-01-24T14:13:59.613687+0000","last_became_active":"2024-01-24T14:13:59.606045+0000","last_became_peered":"2024-01-24T14:13:59.606045+0000","last_unstale":"2024-01-24T14:13:59.613687+0000","last_undegraded":"2024-01-24T14:13:59.613687+0000","last_fullsized":"2024-01-24T14:13:59.613687+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588511+0000","last_change":"2024-01-24T14:13:59.180639+0000","last_active":"2024-01-24T14:13:59.588511+0000","last_peered":"2024-01-24T14:13:59.588511+0000","last_clean":"2024-01-24T14:13:59.588511+0000","last_became_active":"2024-01-24T14:13:59.180142+0000","last_became_peered":"2024-01-24T14:13:59.180142+0000","last_unstale":"2024-01-24T14:13:59.588511+0000","last_undegraded":"2024-01-24T14:13:59.588511+0000","last_fullsized":"2024-01-24T14:13:59.588511+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.616523+0000","last_change":"2024-01-24T14:13:59.606110+0000","last_active":"2024-01-24T14:13:59.616523+0000","last_peered":"2024-01-24T14:13:59.616523+0000","last_clean":"2024-01-24T14:13:59.616523+0000","last_became_active":"2024-01-24T14:13:59.605783+0000","last_became_peered":"2024-01-24T14:13:59.605783+0000","last_unstale":"2024-01-24T14:13:59.616523+0000","last_undegraded":"2024-01-24T14:13:59.616523+0000","last_fullsized":"2024-01-24T14:13:59.616523+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.590527+0000","last_change":"2024-01-24T14:13:59.179345+0000","last_active":"2024-01-24T14:13:59.590527+0000","last_peered":"2024-01-24T14:13:59.590527+0000","last_clean":"2024-01-24T14:13:59.590527+0000","last_became_active":"2024-01-24T14:13:59.178902+0000","last_became_peered":"2024-01-24T14:13:59.178902+0000","last_unstale":"2024-01-24T14:13:59.590527+0000","last_undegraded":"2024-01-24T14:13:59.590527+0000","last_fullsized":"2024-01-24T14:13:59.590527+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605142+0000","last_change":"2024-01-24T14:13:59.600846+0000","last_active":"2024-01-24T14:13:59.605142+0000","last_peered":"2024-01-24T14:13:59.605142+0000","last_clean":"2024-01-24T14:13:59.605142+0000","last_became_active":"2024-01-24T14:13:59.600474+0000","last_became_peered":"2024-01-24T14:13:59.600474+0000","last_unstale":"2024-01-24T14:13:59.605142+0000","last_undegraded":"2024-01-24T14:13:59.605142+0000","last_fullsized":"2024-01-24T14:13:59.605142+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.588283+0000","last_change":"2024-01-24T14:13:59.168449+0000","last_active":"2024-01-24T14:13:59.588283+0000","last_peered":"2024-01-24T14:13:59.588283+0000","last_clean":"2024-01-24T14:13:59.588283+0000","last_became_active":"2024-01-24T14:13:59.168136+0000","last_became_peered":"2024-01-24T14:13:59.168136+0000","last_unstale":"2024-01-24T14:13:59.588283+0000","last_undegraded":"2024-01-24T14:13:59.588283+0000","last_fullsized":"2024-01-24T14:13:59.588283+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.598763+0000","last_change":"2024-01-24T14:13:59.592342+0000","last_active":"2024-01-24T14:13:59.598763+0000","last_peered":"2024-01-24T14:13:59.598763+0000","last_clean":"2024-01-24T14:13:59.598763+0000","last_became_active":"2024-01-24T14:13:59.590942+0000","last_became_peered":"2024-01-24T14:13:59.590942+0000","last_unstale":"2024-01-24T14:13:59.598763+0000","last_undegraded":"2024-01-24T14:13:59.598763+0000","last_fullsized":"2024-01-24T14:13:59.598763+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581385+0000","last_change":"2024-01-24T14:13:59.173283+0000","last_active":"2024-01-24T14:13:59.581385+0000","last_peered":"2024-01-24T14:13:59.581385+0000","last_clean":"2024-01-24T14:13:59.581385+0000","last_became_active":"2024-01-24T14:13:59.172965+0000","last_became_peered":"2024-01-24T14:13:59.172965+0000","last_unstale":"2024-01-24T14:13:59.581385+0000","last_undegraded":"2024-01-24T14:13:59.581385+0000","last_fullsized":"2024-01-24T14:13:59.581385+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591632+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:13:59.591632+0000","last_peered":"2024-01-24T14:13:59.591632+0000","last_clean":"2024-01-24T14:13:59.591632+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:13:59.591632+0000","last_undegraded":"2024-01-24T14:13:59.591632+0000","last_fullsized":"2024-01-24T14:13:59.591632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.603074+0000","last_change":"2024-01-24T14:13:59.599657+0000","last_active":"2024-01-24T14:13:59.603074+0000","last_peered":"2024-01-24T14:13:59.603074+0000","last_clean":"2024-01-24T14:13:59.603074+0000","last_became_active":"2024-01-24T14:13:59.599353+0000","last_became_peered":"2024-01-24T14:13:59.599353+0000","last_unstale":"2024-01-24T14:13:59.603074+0000","last_undegraded":"2024-01-24T14:13:59.603074+0000","last_fullsized":"2024-01-24T14:13:59.603074+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.597437+0000","last_change":"2024-01-24T14:13:59.179171+0000","last_active":"2024-01-24T14:13:59.597437+0000","last_peered":"2024-01-24T14:13:59.597437+0000","last_clean":"2024-01-24T14:13:59.597437+0000","last_became_active":"2024-01-24T14:13:59.176714+0000","last_became_peered":"2024-01-24T14:13:59.176714+0000","last_unstale":"2024-01-24T14:13:59.597437+0000","last_undegraded":"2024-01-24T14:13:59.597437+0000","last_fullsized":"2024-01-24T14:13:59.597437+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585663+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:13:59.585663+0000","last_peered":"2024-01-24T14:13:59.585663+0000","last_clean":"2024-01-24T14:13:59.585663+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:13:59.585663+0000","last_undegraded":"2024-01-24T14:13:59.585663+0000","last_fullsized":"2024-01-24T14:13:59.585663+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613485+0000","last_change":"2024-01-24T14:13:59.603439+0000","last_active":"2024-01-24T14:13:59.613485+0000","last_peered":"2024-01-24T14:13:59.613485+0000","last_clean":"2024-01-24T14:13:59.613485+0000","last_became_active":"2024-01-24T14:13:59.603116+0000","last_became_peered":"2024-01-24T14:13:59.603116+0000","last_unstale":"2024-01-24T14:13:59.613485+0000","last_undegraded":"2024-01-24T14:13:59.613485+0000","last_fullsized":"2024-01-24T14:13:59.613485+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.602963+0000","last_change":"2024-01-24T14:13:59.181711+0000","last_active":"2024-01-24T14:13:59.602963+0000","last_peered":"2024-01-24T14:13:59.602963+0000","last_clean":"2024-01-24T14:13:59.602963+0000","last_became_active":"2024-01-24T14:13:59.176397+0000","last_became_peered":"2024-01-24T14:13:59.176397+0000","last_unstale":"2024-01-24T14:13:59.602963+0000","last_undegraded":"2024-01-24T14:13:59.602963+0000","last_fullsized":"2024-01-24T14:13:59.602963+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581282+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:13:59.581282+0000","last_peered":"2024-01-24T14:13:59.581282+0000","last_clean":"2024-01-24T14:13:59.581282+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:13:59.581282+0000","last_undegraded":"2024-01-24T14:13:59.581282+0000","last_fullsized":"2024-01-24T14:13:59.581282+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613322+0000","last_change":"2024-01-24T14:13:59.601159+0000","last_active":"2024-01-24T14:13:59.613322+0000","last_peered":"2024-01-24T14:13:59.613322+0000","last_clean":"2024-01-24T14:13:59.613322+0000","last_became_active":"2024-01-24T14:13:59.600870+0000","last_became_peered":"2024-01-24T14:13:59.600870+0000","last_unstale":"2024-01-24T14:13:59.613322+0000","last_undegraded":"2024-01-24T14:13:59.613322+0000","last_fullsized":"2024-01-24T14:13:59.613322+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.595988+0000","last_change":"2024-01-24T14:13:59.187233+0000","last_active":"2024-01-24T14:13:59.595988+0000","last_peered":"2024-01-24T14:13:59.595988+0000","last_clean":"2024-01-24T14:13:59.595988+0000","last_became_active":"2024-01-24T14:13:59.186954+0000","last_became_peered":"2024-01-24T14:13:59.186954+0000","last_unstale":"2024-01-24T14:13:59.595988+0000","last_undegraded":"2024-01-24T14:13:59.595988+0000","last_fullsized":"2024-01-24T14:13:59.595988+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585302+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:13:59.585302+0000","last_peered":"2024-01-24T14:13:59.585302+0000","last_clean":"2024-01-24T14:13:59.585302+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:13:59.585302+0000","last_undegraded":"2024-01-24T14:13:59.585302+0000","last_fullsized":"2024-01-24T14:13:59.585302+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594597+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:13:59.594597+0000","last_peered":"2024-01-24T14:13:59.594597+0000","last_clean":"2024-01-24T14:13:59.594597+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:13:59.594597+0000","last_undegraded":"2024-01-24T14:13:59.594597+0000","last_fullsized":"2024-01-24T14:13:59.594597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.605110+0000","last_change":"2024-01-24T14:13:59.601219+0000","last_active":"2024-01-24T14:13:59.605110+0000","last_peered":"2024-01-24T14:13:59.605110+0000","last_clean":"2024-01-24T14:13:59.605110+0000","last_became_active":"2024-01-24T14:13:59.600941+0000","last_became_peered":"2024-01-24T14:13:59.600941+0000","last_unstale":"2024-01-24T14:13:59.605110+0000","last_undegraded":"2024-01-24T14:13:59.605110+0000","last_fullsized":"2024-01-24T14:13:59.605110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613432+0000","last_change":"2024-01-24T14:13:59.187354+0000","last_active":"2024-01-24T14:13:59.613432+0000","last_peered":"2024-01-24T14:13:59.613432+0000","last_clean":"2024-01-24T14:13:59.613432+0000","last_became_active":"2024-01-24T14:13:59.187034+0000","last_became_peered":"2024-01-24T14:13:59.187034+0000","last_unstale":"2024-01-24T14:13:59.613432+0000","last_undegraded":"2024-01-24T14:13:59.613432+0000","last_fullsized":"2024-01-24T14:13:59.613432+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.594511+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:13:59.594511+0000","last_peered":"2024-01-24T14:13:59.594511+0000","last_clean":"2024-01-24T14:13:59.594511+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:13:59.594511+0000","last_undegraded":"2024-01-24T14:13:59.594511+0000","last_fullsized":"2024-01-24T14:13:59.594511+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.608535+0000","last_change":"2024-01-24T14:13:59.600293+0000","last_active":"2024-01-24T14:13:59.608535+0000","last_peered":"2024-01-24T14:13:59.608535+0000","last_clean":"2024-01-24T14:13:59.608535+0000","last_became_active":"2024-01-24T14:13:59.599864+0000","last_became_peered":"2024-01-24T14:13:59.599864+0000","last_unstale":"2024-01-24T14:13:59.608535+0000","last_undegraded":"2024-01-24T14:13:59.608535+0000","last_fullsized":"2024-01-24T14:13:59.608535+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.613067+0000","last_change":"2024-01-24T14:13:59.173278+0000","last_active":"2024-01-24T14:13:59.613067+0000","last_peered":"2024-01-24T14:13:59.613067+0000","last_clean":"2024-01-24T14:13:59.613067+0000","last_became_active":"2024-01-24T14:13:59.172981+0000","last_became_peered":"2024-01-24T14:13:59.172981+0000","last_unstale":"2024-01-24T14:13:59.613067+0000","last_undegraded":"2024-01-24T14:13:59.613067+0000","last_fullsized":"2024-01-24T14:13:59.613067+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":87,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.596080+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:13:59.596080+0000","last_peered":"2024-01-24T14:13:59.596080+0000","last_clean":"2024-01-24T14:13:59.596080+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:13:59.596080+0000","last_undegraded":"2024-01-24T14:13:59.596080+0000","last_fullsized":"2024-01-24T14:13:59.596080+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.604651+0000","last_change":"2024-01-24T14:13:59.597915+0000","last_active":"2024-01-24T14:13:59.604651+0000","last_peered":"2024-01-24T14:13:59.604651+0000","last_clean":"2024-01-24T14:13:59.604651+0000","last_became_active":"2024-01-24T14:13:59.597457+0000","last_became_peered":"2024-01-24T14:13:59.597457+0000","last_unstale":"2024-01-24T14:13:59.604651+0000","last_undegraded":"2024-01-24T14:13:59.604651+0000","last_fullsized":"2024-01-24T14:13:59.604651+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.585416+0000","last_change":"2024-01-24T14:13:59.187991+0000","last_active":"2024-01-24T14:13:59.585416+0000","last_peered":"2024-01-24T14:13:59.585416+0000","last_clean":"2024-01-24T14:13:59.585416+0000","last_became_active":"2024-01-24T14:13:59.187677+0000","last_became_peered":"2024-01-24T14:13:59.187677+0000","last_unstale":"2024-01-24T14:13:59.585416+0000","last_undegraded":"2024-01-24T14:13:59.585416+0000","last_fullsized":"2024-01-24T14:13:59.585416+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591436+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:13:59.591436+0000","last_peered":"2024-01-24T14:13:59.591436+0000","last_clean":"2024-01-24T14:13:59.591436+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:13:59.591436+0000","last_undegraded":"2024-01-24T14:13:59.591436+0000","last_fullsized":"2024-01-24T14:13:59.591436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.593811+0000","last_change":"2024-01-24T14:13:59.587516+0000","last_active":"2024-01-24T14:13:59.593811+0000","last_peered":"2024-01-24T14:13:59.593811+0000","last_clean":"2024-01-24T14:13:59.593811+0000","last_became_active":"2024-01-24T14:13:59.587242+0000","last_became_peered":"2024-01-24T14:13:59.587242+0000","last_unstale":"2024-01-24T14:13:59.593811+0000","last_undegraded":"2024-01-24T14:13:59.593811+0000","last_fullsized":"2024-01-24T14:13:59.593811+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:59.150171+0000","last_clean_scrub_stamp":"2024-01-24T14:13:59.150171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.581221+0000","last_change":"2024-01-24T14:13:59.176122+0000","last_active":"2024-01-24T14:13:59.581221+0000","last_peered":"2024-01-24T14:13:59.581221+0000","last_clean":"2024-01-24T14:13:59.581221+0000","last_became_active":"2024-01-24T14:13:59.175794+0000","last_became_peered":"2024-01-24T14:13:59.175794+0000","last_unstale":"2024-01-24T14:13:59.581221+0000","last_undegraded":"2024-01-24T14:13:59.581221+0000","last_fullsized":"2024-01-24T14:13:59.581221+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:13:58.144841+0000","last_clean_scrub_stamp":"2024-01-24T14:13:58.144841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":84,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-01-24T14:13:59.591329+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:13:59.591329+0000","last_peered":"2024-01-24T14:13:59.591329+0000","last_clean":"2024-01-24T14:13:59.591329+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:13:59.591329+0000","last_undegraded":"2024-01-24T14:13:59.591329+0000","last_fullsized":"2024-01-24T14:13:59.591329+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575039,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1290240,"data_stored":710498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.26000000000000001},"max":{"1min":0.80600000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.32700000000000001},"max":{"1min":0.753,"5min":0.82599999999999996,"15min":0.83499999999999996},"last":0.42199999999999999}]},{"osd":1,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.312},"max":{"1min":0.75600000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":3.2570000000000001},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.51700000000000002,"5min":0.40400000000000003,"15min":0.32000000000000001},"max":{"1min":0.79100000000000004,"5min":0.872,"15min":0.876},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.39300000000000002,"15min":0.36799999999999999},"max":{"1min":0.80000000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54500000000000004}]},{"osd":3,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.36499999999999999},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.85099999999999998,"15min":3.0049999999999999},"last":0.46400000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48899999999999999},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.84699999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.60399999999999998,"5min":0.70599999999999996,"15min":0.83599999999999997},"last":0.627}]},{"osd":5,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.501,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.29999999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.80900000000000005},"last":0.61699999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.80700000000000005,"15min":0.87},"last":0.64000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575040,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302456,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069384,"statfs":{"total":96636764160,"available":96327049216,"internally_reserved":0,"allocated":1294336,"data_stored":710056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.59499999999999997},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73699999999999999,"5min":0.85699999999999998,"15min":0.86899999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":0.84899999999999998,"15min":1.3859999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73499999999999999,"5min":0.77000000000000002,"15min":1.353},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.78700000000000003,"5min":1.032,"15min":1.032},"last":0.47699999999999998}]},{"osd":2,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60099999999999998,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.39100000000000001,"15min":0.34300000000000003},"max":{"1min":0.78600000000000003,"5min":0.88500000000000001,"15min":6.8129999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.4079999999999999},"last":0.49399999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.66900000000000004},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81699999999999995,"5min":0.82799999999999996,"15min":7.016},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.43099999999999999,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.77400000000000002,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.504,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":1.165,"15min":1.165},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.501,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.314},"max":{"1min":0.65500000000000003,"5min":0.82299999999999995,"15min":6.8719999999999999},"last":0.621}]},{"osd":5,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":0.86699999999999999,"15min":6.976},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80200000000000005,"5min":0.85699999999999998,"15min":7.8789999999999996},"last":0.60299999999999998}]},{"osd":7,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.747,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":0.89600000000000002,"15min":6.9080000000000004},"last":0.74099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575039,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302456,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069384,"statfs":{"total":96636764160,"available":96327049216,"internally_reserved":0,"allocated":1294336,"data_stored":710068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.51900000000000002,"15min":0.443},"min":{"1min":0.311,"5min":0.245,"15min":0.23400000000000001},"max":{"1min":0.57899999999999996,"5min":15.298999999999999,"15min":15.298999999999999},"last":0.373},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.55000000000000004,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.57499999999999996,"5min":14.654,"15min":14.654},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45900000000000002,"15min":0.45100000000000001},"min":{"1min":0.34899999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.59799999999999998,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.25800000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.42399999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47099999999999997,"15min":0.45700000000000002},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53500000000000003}]},{"osd":4,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88600000000000001,"5min":1.073,"15min":1.137},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.45900000000000002,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.218,"15min":1.218},"last":0.65700000000000003}]},{"osd":5,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":1.1040000000000001,"15min":0.83299999999999996},"min":{"1min":0.47999999999999998,"5min":0.45000000000000001,"15min":0.443},"max":{"1min":0.86799999999999999,"5min":38.808,"15min":38.808},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":1.077,"15min":0.81200000000000006},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.40500000000000003},"max":{"1min":0.81299999999999994,"5min":38.823,"15min":38.823},"last":0.71399999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.46000000000000002,"15min":0.43099999999999999},"max":{"1min":0.94899999999999995,"5min":1.0449999999999999,"15min":1.0509999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.40000000000000002},"max":{"1min":0.91300000000000003,"5min":1.1559999999999999,"15min":1.1879999999999999},"last":0.69499999999999995}]},{"osd":7,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.52500000000000002,"5min":0.49399999999999999,"15min":0.45300000000000001},"max":{"1min":0.98499999999999999,"5min":1.105,"15min":1.155},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.54900000000000004,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":0.93500000000000005,"5min":1.123,"15min":1.131},"last":0.74299999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542337,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1290240,"data_stored":709909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39600000000000002},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.57499999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.378},{"interface":"front","average":{"1min":0.432,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.41499999999999998}]},{"osd":2,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44800000000000001,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.28199999999999997},"max":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.66700000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.28799999999999998},"max":{"1min":0.61399999999999999,"5min":0.754,"15min":0.79400000000000004},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.33000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54500000000000004,"5min":1.482,"15min":1.482},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.496},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":0.872,"15min":0.872},"last":0.39800000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81499999999999995,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.78400000000000003}]},{"osd":5,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.875,"5min":1.655,"15min":1.655},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.42999999999999999},"max":{"1min":0.77200000000000002,"5min":0.94799999999999995,"15min":1.0840000000000001},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.875,"5min":0.90700000000000003,"15min":1.137},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":0.96499999999999997,"15min":1.117},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47099999999999997},"max":{"1min":0.86899999999999999,"5min":0.96099999999999997,"15min":0.98199999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.49099999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.61099999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575039,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306548,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94065292,"statfs":{"total":96636764160,"available":96322859008,"internally_reserved":0,"allocated":1290240,"data_stored":710014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.29499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.65800000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.24399999999999999},"max":{"1min":0.54500000000000004,"5min":0.65900000000000003,"15min":0.67100000000000004},"last":0.28199999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.32500000000000001,"5min":0.27800000000000002,"15min":0.27300000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.67200000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.255},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.35899999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.45000000000000001},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64300000000000002,"5min":0.67700000000000005,"15min":0.71699999999999997},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.38900000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.72799999999999998},"last":0.37}]},{"osd":4,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.502,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85899999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.501},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.55400000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.92200000000000004,"5min":0.996,"15min":1.042},"last":0.55600000000000005}]},{"osd":5,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.53400000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83599999999999997,"5min":1.036,"15min":1.0469999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90000000000000002,"5min":1.107,"15min":1.107},"last":0.46999999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.82899999999999996,"5min":1.0569999999999999,"15min":1.1100000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.57299999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84699999999999998,"5min":1.141,"15min":1.141},"last":0.66700000000000004}]},{"osd":7,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.52400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":0.96199999999999997,"15min":1.054},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.88900000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.629}]}]},{"osd":3,"up_from":13,"seq":55834575039,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302460,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069380,"statfs":{"total":96636764160,"available":96327045120,"internally_reserved":0,"allocated":1298432,"data_stored":710658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45100000000000001,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.76400000000000001,"5min":7.593,"15min":7.593},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48299999999999998,"15min":0.437},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56200000000000006,"5min":8.4239999999999995,"15min":8.4239999999999995},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.69099999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.221,"15min":0.221},"max":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.68799999999999994},"last":0.52100000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.496},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.66500000000000004,"5min":0.76300000000000001,"15min":0.77200000000000002},"last":0.439},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.78800000000000003},"last":0.54200000000000004}]},{"osd":4,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64500000000000002,"15min":0.63500000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.495,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":0.81599999999999995,"5min":1.976,"15min":1.976},"last":0.79400000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91500000000000004,"5min":1.53,"15min":1.53},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.97999999999999998,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.84099999999999997}]},{"osd":7,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72799999999999998,"15min":0.72099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.72999999999999998},"min":{"1min":0.495,"5min":0.49199999999999999,"15min":0.45800000000000002},"max":{"1min":0.93500000000000005,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.873}]}]},{"osd":4,"up_from":13,"seq":55834575040,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1290240,"data_stored":710498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78300000000000003,"5min":0.80100000000000005,"15min":0.88600000000000001},"last":0.621},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.84399999999999997},"last":0.56999999999999995}]},{"osd":1,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.83099999999999996,"15min":0.85299999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.755,"5min":0.79800000000000004,"15min":1.506},"last":0.59299999999999997}]},{"osd":2,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.80200000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.60699999999999998}]},{"osd":3,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72899999999999998,"5min":0.77800000000000002,"15min":0.80600000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.81000000000000005,"15min":0.86299999999999999},"last":0.55800000000000005}]},{"osd":5,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.92000000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.496,"15min":0.5},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65700000000000003,"5min":0.72899999999999998,"15min":1.556},"last":0.38300000000000001}]},{"osd":6,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.753,"5min":0.78500000000000003,"15min":0.86099999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76400000000000001,"5min":0.82099999999999995,"15min":1.0409999999999999},"last":0.45900000000000002}]},{"osd":7,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.38500000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.88100000000000001},"last":0.432},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.307,"15min":0.307},"max":{"1min":0.72699999999999998,"5min":0.81399999999999995,"15min":1.26},"last":0.442}]}]},{"osd":5,"up_from":13,"seq":55834575040,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302464,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069376,"statfs":{"total":96636764160,"available":96327041024,"internally_reserved":0,"allocated":1302528,"data_stored":711502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64400000000000002,"5min":0.85199999999999998,"15min":1.101},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.309,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71199999999999997,"5min":0.82699999999999996,"15min":1.1439999999999999},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56000000000000005,"15min":0.57699999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67200000000000004,"5min":0.84099999999999997,"15min":1.093},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69199999999999995,"5min":0.81299999999999994,"15min":1.1639999999999999},"last":0.64700000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.68400000000000005,"5min":0.84299999999999997,"15min":1.1100000000000001},"last":0.624},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.35499999999999998},"max":{"1min":0.67400000000000004,"5min":0.83599999999999997,"15min":1.1220000000000001},"last":0.63400000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.38500000000000001},"max":{"1min":0.72199999999999998,"5min":0.80500000000000005,"15min":1.139},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.496,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":1.0609999999999999},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70199999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.505,"15min":0.503},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.90000000000000002},"last":0.51400000000000001}]},{"osd":6,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.32900000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":1.1839999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77400000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.496}]},{"osd":7,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72999999999999998,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.33200000000000002},"max":{"1min":0.78200000000000003,"5min":0.95999999999999996,"15min":1.2030000000000001},"last":0.56599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:14:08.653 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-24T14:14:08.654 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-24T14:14:08.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.828+0000 7fb87378f700 1 Processor -- start 2024-01-24T14:14:08.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.828+0000 7fb87378f700 1 -- start start 2024-01-24T14:14:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.829+0000 7fb87378f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb86c0665f0 0x7fb86c062c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.829+0000 7fb87378f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c06a250 con 0x7fb86c1593a0 2024-01-24T14:14:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.829+0000 7fb87378f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c0c3e60 con 0x7fb86c0627b0 2024-01-24T14:14:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.829+0000 7fb87378f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb86c0c2970 con 0x7fb86c0665f0 2024-01-24T14:14:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.829+0000 7fb871d2c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb86c0665f0 0x7fb86c062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.829+0000 7fb87152b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb86c1593a0 0x7fb86c0c2260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39600/0 (socket says 172.21.15.29:39600) 2024-01-24T14:14:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.829+0000 7fb87152b700 1 -- 172.21.15.29:0/790151270 learned_addr learned my addr 172.21.15.29:0/790151270 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.830+0000 7fb863fff700 1 -- 172.21.15.29:0/790151270 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192337722 0 0) 0x7fb86c06a250 con 0x7fb86c1593a0 2024-01-24T14:14:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.830+0000 7fb863fff700 1 -- 172.21.15.29:0/790151270 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8580033f0 con 0x7fb86c1593a0 2024-01-24T14:14:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.830+0000 7fb871d2c700 1 -- 172.21.15.29:0/790151270 >> v1:172.21.15.163:6789/0 conn(0x7fb86c0627b0 legacy=0x7fb86c0a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:08.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.830+0000 7fb871d2c700 1 -- 172.21.15.29:0/790151270 >> v1:172.21.15.29:6789/0 conn(0x7fb86c1593a0 legacy=0x7fb86c0c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:08.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.830+0000 7fb871d2c700 1 -- 172.21.15.29:0/790151270 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c063120 con 0x7fb86c0665f0 2024-01-24T14:14:08.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.830+0000 7fb871d2c700 1 --2- 172.21.15.29:0/790151270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb86c0665f0 0x7fb86c062c30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7fb864009e60 tx=0x7fb864009780).ready entity=mon.0 client_cookie=d879ceb568000f81 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:08.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.830+0000 7fb863fff700 1 -- 172.21.15.29:0/790151270 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86400e070 con 0x7fb86c0665f0 2024-01-24T14:14:08.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.830+0000 7fb863fff700 1 -- 172.21.15.29:0/790151270 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86400bb60 con 0x7fb86c0665f0 2024-01-24T14:14:08.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.830+0000 7fb863fff700 1 -- 172.21.15.29:0/790151270 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864012410 con 0x7fb86c0665f0 2024-01-24T14:14:08.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.831+0000 7fb87378f700 1 -- 172.21.15.29:0/790151270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb86c0665f0 msgr2=0x7fb86c062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:08.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.831+0000 7fb87378f700 1 --2- 172.21.15.29:0/790151270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb86c0665f0 0x7fb86c062c30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7fb864009e60 tx=0x7fb864009780).stop 2024-01-24T14:14:08.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.831+0000 7fb87378f700 1 -- 172.21.15.29:0/790151270 shutdown_connections 2024-01-24T14:14:08.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.831+0000 7fb87378f700 1 --2- 172.21.15.29:0/790151270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb86c0665f0 0x7fb86c062c30 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:08.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.831+0000 7fb87378f700 1 -- 172.21.15.29:0/790151270 >> 172.21.15.29:0/790151270 conn(0x7fb86c0638c0 msgr2=0x7fb86c063cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:08.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.831+0000 7fb87378f700 1 -- 172.21.15.29:0/790151270 shutdown_connections 2024-01-24T14:14:08.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.831+0000 7fb87378f700 1 -- 172.21.15.29:0/790151270 wait complete. 2024-01-24T14:14:08.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.832+0000 7fb87378f700 1 Processor -- start 2024-01-24T14:14:08.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.832+0000 7fb87378f700 1 -- start start 2024-01-24T14:14:08.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.833+0000 7fb87378f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c0627b0 0x7fb86c14ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.833+0000 7fb87378f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb86c0665f0 0x7fb86c0a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.833+0000 7fb87378f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c1593a0 0x7fb86c0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.833+0000 7fb87378f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0bb720 con 0x7fb86c0665f0 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.833+0000 7fb87378f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0bb930 con 0x7fb86c0627b0 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.833+0000 7fb87378f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0bbc10 con 0x7fb86c1593a0 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.833+0000 7fb870d2a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb86c0665f0 0x7fb86c0a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.833+0000 7fb870d2a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb86c0665f0 0x7fb86c0a79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59992/0 (socket says 172.21.15.29:59992) 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.833+0000 7fb870d2a700 1 -- 172.21.15.29:0/3044128435 learned_addr learned my addr 172.21.15.29:0/3044128435 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.833+0000 7fb87152b700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c0627b0 0x7fb86c14ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.834+0000 7fb871d2c700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c1593a0 0x7fb86c0ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.834+0000 7fb870d2a700 1 -- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c1593a0 msgr2=0x7fb86c0ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.834+0000 7fb870d2a700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c1593a0 0x7fb86c0ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.834+0000 7fb870d2a700 1 -- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c0627b0 msgr2=0x7fb86c14ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:08.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.834+0000 7fb870d2a700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c0627b0 0x7fb86c14ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:08.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.834+0000 7fb870d2a700 1 -- 172.21.15.29:0/3044128435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb864009a20 con 0x7fb86c0665f0 2024-01-24T14:14:08.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.834+0000 7fb870d2a700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb86c0665f0 0x7fb86c0a79b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fb85c00c4c0 tx=0x7fb85c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:08.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.834+0000 7fb8617fa700 1 -- 172.21.15.29:0/3044128435 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00ce70 con 0x7fb86c0665f0 2024-01-24T14:14:08.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.834+0000 7fb8617fa700 1 -- 172.21.15.29:0/3044128435 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c001e90 con 0x7fb86c0665f0 2024-01-24T14:14:08.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.834+0000 7fb87378f700 1 -- 172.21.15.29:0/3044128435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c0bbef0 con 0x7fb86c0665f0 2024-01-24T14:14:08.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.834+0000 7fb8617fa700 1 -- 172.21.15.29:0/3044128435 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c007250 con 0x7fb86c0665f0 2024-01-24T14:14:08.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.834+0000 7fb87378f700 1 -- 172.21.15.29:0/3044128435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c0bc4b0 con 0x7fb86c0665f0 2024-01-24T14:14:08.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.835+0000 7fb87378f700 1 -- 172.21.15.29:0/3044128435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c0bc6a0 con 0x7fb86c0665f0 2024-01-24T14:14:08.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.836+0000 7fb8617fa700 1 -- 172.21.15.29:0/3044128435 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb85c0066a0 con 0x7fb86c0665f0 2024-01-24T14:14:08.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.836+0000 7fb8617fa700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb85005f270 0x7fb850061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:08.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.836+0000 7fb8617fa700 1 -- 172.21.15.29:0/3044128435 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fb85c07a020 con 0x7fb86c0665f0 2024-01-24T14:14:08.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.836+0000 7fb87152b700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb85005f270 0x7fb850061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:08.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.837+0000 7fb87152b700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb85005f270 0x7fb850061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fb868007fd0 tx=0x7fb868009040).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:08.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:08.840+0000 7fb8617fa700 1 -- 172.21.15.29:0/3044128435 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb85c04ad70 con 0x7fb86c0665f0 2024-01-24T14:14:09.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.095+0000 7fb87378f700 1 -- 172.21.15.29:0/3044128435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb86c0475e0 con 0x7fb86c0665f0 2024-01-24T14:14:09.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.099+0000 7fb8617fa700 1 -- 172.21.15.29:0/3044128435 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb85c04e390 con 0x7fb86c0665f0 2024-01-24T14:14:09.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.100+0000 7fb87378f700 1 -- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb85005f270 msgr2=0x7fb850061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:09.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.100+0000 7fb87378f700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb85005f270 0x7fb850061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fb868007fd0 tx=0x7fb868009040).stop 2024-01-24T14:14:09.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.100+0000 7fb87378f700 1 -- 172.21.15.29:0/3044128435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb86c0665f0 msgr2=0x7fb86c0a79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:09.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.100+0000 7fb87378f700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb86c0665f0 0x7fb86c0a79b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fb85c00c4c0 tx=0x7fb85c007e00).stop 2024-01-24T14:14:09.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.101+0000 7fb87378f700 1 -- 172.21.15.29:0/3044128435 shutdown_connections 2024-01-24T14:14:09.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.101+0000 7fb87378f700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c1593a0 0x7fb86c0ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.101+0000 7fb87378f700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c0627b0 0x7fb86c14ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.101+0000 7fb87378f700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb85005f270 0x7fb850061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.101+0000 7fb87378f700 1 --2- 172.21.15.29:0/3044128435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb86c0665f0 0x7fb86c0a79b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.101+0000 7fb87378f700 1 -- 172.21.15.29:0/3044128435 >> 172.21.15.29:0/3044128435 conn(0x7fb86c0638c0 msgr2=0x7fb86c159780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:09.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.101+0000 7fb87378f700 1 -- 172.21.15.29:0/3044128435 shutdown_connections 2024-01-24T14:14:09.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.101+0000 7fb87378f700 1 -- 172.21.15.29:0/3044128435 wait complete. 2024-01-24T14:14:09.102 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-01-24T14:14:09.115 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:14:09.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.244+0000 7f352cdbf700 1 Processor -- start 2024-01-24T14:14:09.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.245+0000 7f352cdbf700 1 -- start start 2024-01-24T14:14:09.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352cdbf700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35280bd850 0x7f35280bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:09.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352cdbf700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35280c1870 0x7f35280c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:09.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352cdbf700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3528064040 con 0x7f35280627b0 2024-01-24T14:14:09.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352cdbf700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35280be120 con 0x7f35280bd850 2024-01-24T14:14:09.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352cdbf700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35280c21e0 con 0x7f35280c1870 2024-01-24T14:14:09.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35280bd850 0x7f35280bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:09.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35280bd850 0x7f35280bdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59998/0 (socket says 172.21.15.29:59998) 2024-01-24T14:14:09.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352659c700 1 -- 172.21.15.29:0/2200094268 learned_addr learned my addr 172.21.15.29:0/2200094268 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:09.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f3525d9b700 1 --2- 172.21.15.29:0/2200094268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35280c1870 0x7f35280c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:09.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352659c700 1 -- 172.21.15.29:0/2200094268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35280c1870 msgr2=0x7f35280c1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:09.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352659c700 1 --2- 172.21.15.29:0/2200094268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35280c1870 0x7f35280c1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352659c700 1 -- 172.21.15.29:0/2200094268 >> v1:172.21.15.163:6789/0 conn(0x7f35280627b0 legacy=0x7f3528062c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352659c700 1 -- 172.21.15.29:0/2200094268 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3528063430 con 0x7f35280bd850 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.246+0000 7f352659c700 1 --2- 172.21.15.29:0/2200094268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35280bd850 0x7f35280bdc30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f3518006fd0 tx=0x7f35180090f0).ready entity=mon.0 client_cookie=8eb7a91dbca358f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.247+0000 7f352559a700 1 -- 172.21.15.29:0/2200094268 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351800b580 con 0x7f35280bd850 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.247+0000 7f352559a700 1 -- 172.21.15.29:0/2200094268 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351800b6e0 con 0x7f35280bd850 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.247+0000 7f352559a700 1 -- 172.21.15.29:0/2200094268 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351800ba60 con 0x7f35280bd850 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.247+0000 7f352cdbf700 1 -- 172.21.15.29:0/2200094268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35280bd850 msgr2=0x7f35280bdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.247+0000 7f352cdbf700 1 --2- 172.21.15.29:0/2200094268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35280bd850 0x7f35280bdc30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f3518006fd0 tx=0x7f35180090f0).stop 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.247+0000 7f352cdbf700 1 -- 172.21.15.29:0/2200094268 shutdown_connections 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.247+0000 7f352cdbf700 1 --2- 172.21.15.29:0/2200094268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35280bd850 0x7f35280bdc30 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.247+0000 7f352cdbf700 1 --2- 172.21.15.29:0/2200094268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35280c1870 0x7f35280c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.247+0000 7f352cdbf700 1 -- 172.21.15.29:0/2200094268 >> 172.21.15.29:0/2200094268 conn(0x7f3528064aa0 msgr2=0x7f3528064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.247+0000 7f352cdbf700 1 -- 172.21.15.29:0/2200094268 shutdown_connections 2024-01-24T14:14:09.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.247+0000 7f352cdbf700 1 -- 172.21.15.29:0/2200094268 wait complete. 2024-01-24T14:14:09.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.249+0000 7f352cdbf700 1 Processor -- start 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.249+0000 7f352cdbf700 1 -- start start 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.249+0000 7f352cdbf700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35280627b0 0x7f35281364a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.249+0000 7f352cdbf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35280bd850 0x7f3528069320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.249+0000 7f352cdbf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35280c1870 0x7f3528069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.249+0000 7f352cdbf700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3528069d00 con 0x7f35280bd850 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.249+0000 7f352cdbf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35280aa410 con 0x7f35280c1870 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.249+0000 7f352cdbf700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35281643e0 con 0x7f35280627b0 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f3525d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35280bd850 0x7f3528069320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f3525d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35280bd850 0x7f3528069320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60006/0 (socket says 172.21.15.29:60006) 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f3525d9b700 1 -- 172.21.15.29:0/3550029435 learned_addr learned my addr 172.21.15.29:0/3550029435 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f352659c700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35280627b0 0x7f35281364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f3526d9d700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35280c1870 0x7f3528069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:09.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f3525d9b700 1 -- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35280627b0 msgr2=0x7f35281364a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f3525d9b700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35280627b0 0x7f35281364a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f3525d9b700 1 -- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35280c1870 msgr2=0x7f3528069810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f3525d9b700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35280c1870 0x7f3528069810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f3525d9b700 1 -- 172.21.15.29:0/3550029435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518009370 con 0x7f35280bd850 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f3526d9d700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35280c1870 0x7f3528069810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f3525d9b700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35280bd850 0x7f3528069320 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f351c00e960 tx=0x7f351c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f35167fc700 1 -- 172.21.15.29:0/3550029435 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c009510 con 0x7f35280bd850 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f35167fc700 1 -- 172.21.15.29:0/3550029435 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c00c850 con 0x7f35280bd850 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f352cdbf700 1 -- 172.21.15.29:0/3550029435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35280aa6b0 con 0x7f35280bd850 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f35167fc700 1 -- 172.21.15.29:0/3550029435 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c00cbd0 con 0x7f35280bd850 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.250+0000 7f352cdbf700 1 -- 172.21.15.29:0/3550029435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35280c1120 con 0x7f35280bd850 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.251+0000 7f352cdbf700 1 -- 172.21.15.29:0/3550029435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35280aa840 con 0x7f35280bd850 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.252+0000 7f35167fc700 1 -- 172.21.15.29:0/3550029435 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f351c01b400 con 0x7f35280bd850 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.252+0000 7f35167fc700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f350805f270 0x7f3508061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.252+0000 7f35167fc700 1 -- 172.21.15.29:0/3550029435 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f351c07ac50 con 0x7f35280bd850 2024-01-24T14:14:09.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.252+0000 7f352659c700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f350805f270 0x7f3508061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:09.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.253+0000 7f352659c700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f350805f270 0x7f3508061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f3518009130 tx=0x7f3518017000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:09.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.257+0000 7f35167fc700 1 -- 172.21.15.29:0/3550029435 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f351c04b940 con 0x7f35280bd850 2024-01-24T14:14:09.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.482+0000 7f352cdbf700 1 -- 172.21.15.29:0/3550029435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35280475e0 con 0x7f35280bd850 2024-01-24T14:14:09.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.482+0000 7f35167fc700 1 -- 172.21.15.29:0/3550029435 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f351c04ef60 con 0x7f35280bd850 2024-01-24T14:14:09.483 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:09.483 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4083072158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":4083072158},{"type":"v1","addr":"172.21.15.163:6837","nonce":4083072158}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/4114592089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4114592089},{"type":"v1","addr":"172.21.15.163:6835","nonce":4114592089}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/926851304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":926851304},{"type":"v1","addr":"172.21.15.29:6835","nonce":926851304}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:13:59.576939+0000","modified":"2024-01-24T14:14:02.594306+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6833/3854229544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3854229544},{"type":"v1","addr":"172.21.15.29:6833","nonce":3854229544}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-24T14:14:09.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.485+0000 7f352cdbf700 1 -- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f350805f270 msgr2=0x7f3508061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.485+0000 7f352cdbf700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f350805f270 0x7f3508061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f3518009130 tx=0x7f3518017000).stop 2024-01-24T14:14:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.485+0000 7f352cdbf700 1 -- 172.21.15.29:0/3550029435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35280bd850 msgr2=0x7f3528069320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.485+0000 7f352cdbf700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35280bd850 0x7f3528069320 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f351c00e960 tx=0x7f351c009060).stop 2024-01-24T14:14:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.485+0000 7f352cdbf700 1 -- 172.21.15.29:0/3550029435 shutdown_connections 2024-01-24T14:14:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.485+0000 7f352cdbf700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35280c1870 0x7f3528069810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.485+0000 7f352cdbf700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35280627b0 0x7f35281364a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.485+0000 7f352cdbf700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f350805f270 0x7f3508061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.485+0000 7f352cdbf700 1 --2- 172.21.15.29:0/3550029435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35280bd850 0x7f3528069320 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.485+0000 7f352cdbf700 1 -- 172.21.15.29:0/3550029435 >> 172.21.15.29:0/3550029435 conn(0x7f3528064aa0 msgr2=0x7f35280bc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.485+0000 7f352cdbf700 1 -- 172.21.15.29:0/3550029435 shutdown_connections 2024-01-24T14:14:09.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:09.485+0000 7f352cdbf700 1 -- 172.21.15.29:0/3550029435 wait complete. 2024-01-24T14:14:09.487 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 43 2024-01-24T14:14:09.901 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T14:13:59.576939+0000', 'modified': '2024-01-24T14:14:02.594306+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6807}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6807': {'gid': 6807, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6833/3854229544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6832', 'nonce': 3854229544}, {'type': 'v1', 'addr': '172.21.15.29:6833', 'nonce': 3854229544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-24T14:14:09.901 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T14:14:09.901 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-24T14:14:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.040+0000 7fd5e3f86700 1 Processor -- start 2024-01-24T14:14:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.040+0000 7fd5e3f86700 1 -- start start 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.041+0000 7fd5e3f86700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd5dc151de0 0x7fd5dc155470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.041+0000 7fd5e3f86700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5dc158800 con 0x7fd5dc066bb0 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.041+0000 7fd5e3f86700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5dc157ed0 con 0x7fd5dc0638c0 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.041+0000 7fd5e3f86700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5dc1581b0 con 0x7fd5dc151de0 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.041+0000 7fd5e2523700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd5dc066bb0 0x7fd5dc13d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39608/0 (socket says 172.21.15.29:39608) 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.041+0000 7fd5e2523700 1 -- 172.21.15.29:0/3240683665 learned_addr learned my addr 172.21.15.29:0/3240683665 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.041+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/3240683665 >> v2:172.21.15.163:3300/0 conn(0x7fd5dc151de0 0x7fd5dc155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.042+0000 7fd5e0d20700 1 -- 172.21.15.29:0/3240683665 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119587476 0 0) 0x7fd5dc158800 con 0x7fd5dc066bb0 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.042+0000 7fd5e0d20700 1 -- 172.21.15.29:0/3240683665 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5c80033f0 con 0x7fd5dc066bb0 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.042+0000 7fd5e0d20700 1 -- 172.21.15.29:0/3240683665 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167226187 0 0) 0x7fd5dc157ed0 con 0x7fd5dc0638c0 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.042+0000 7fd5e0d20700 1 -- 172.21.15.29:0/3240683665 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5dc158800 con 0x7fd5dc0638c0 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.042+0000 7fd5e1d22700 1 -- 172.21.15.29:0/3240683665 >> v1:172.21.15.163:6789/0 conn(0x7fd5dc0638c0 legacy=0x7fd5dc0664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.042+0000 7fd5e1d22700 1 -- 172.21.15.29:0/3240683665 >> v1:172.21.15.29:6789/0 conn(0x7fd5dc066bb0 legacy=0x7fd5dc13d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.042+0000 7fd5e1d22700 1 -- 172.21.15.29:0/3240683665 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc158490 con 0x7fd5dc151de0 2024-01-24T14:14:10.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.042+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/3240683665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5dc151de0 0x7fd5dc155470 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fd5d8009760 tx=0x7fd5d8009060).ready entity=mon.1 client_cookie=94704419db4f6152 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:10.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.042+0000 7fd5e0d20700 1 -- 172.21.15.29:0/3240683665 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800d4a0 con 0x7fd5dc151de0 2024-01-24T14:14:10.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.042+0000 7fd5e0d20700 1 -- 172.21.15.29:0/3240683665 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d800d600 con 0x7fd5dc151de0 2024-01-24T14:14:10.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.042+0000 7fd5e0d20700 1 -- 172.21.15.29:0/3240683665 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800d980 con 0x7fd5dc151de0 2024-01-24T14:14:10.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.043+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3240683665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5dc151de0 msgr2=0x7fd5dc155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.043+0000 7fd5e3f86700 1 --2- 172.21.15.29:0/3240683665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5dc151de0 0x7fd5dc155470 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fd5d8009760 tx=0x7fd5d8009060).stop 2024-01-24T14:14:10.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.043+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3240683665 shutdown_connections 2024-01-24T14:14:10.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.043+0000 7fd5e3f86700 1 --2- 172.21.15.29:0/3240683665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5dc151de0 0x7fd5dc155470 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.043+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3240683665 >> 172.21.15.29:0/3240683665 conn(0x7fd5dc0c5640 msgr2=0x7fd5dc0c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:10.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.043+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3240683665 shutdown_connections 2024-01-24T14:14:10.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.043+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3240683665 wait complete. 2024-01-24T14:14:10.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.044+0000 7fd5e3f86700 1 Processor -- start 2024-01-24T14:14:10.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.044+0000 7fd5e3f86700 1 -- start start 2024-01-24T14:14:10.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.045+0000 7fd5e3f86700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5dc0638c0 0x7fd5dc0c15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.045+0000 7fd5e3f86700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5dc066bb0 0x7fd5dc150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.045+0000 7fd5e3f86700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5dc151de0 0x7fd5dc154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.045+0000 7fd5e3f86700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc0c2780 con 0x7fd5dc066bb0 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.045+0000 7fd5e3f86700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc151210 con 0x7fd5dc0638c0 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.045+0000 7fd5e3f86700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc0c1b70 con 0x7fd5dc151de0 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.045+0000 7fd5e1521700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5dc066bb0 0x7fd5dc150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.045+0000 7fd5e1521700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5dc066bb0 0x7fd5dc150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60022/0 (socket says 172.21.15.29:60022) 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.045+0000 7fd5e1521700 1 -- 172.21.15.29:0/3773649018 learned_addr learned my addr 172.21.15.29:0/3773649018 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.045+0000 7fd5e2523700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5dc151de0 0x7fd5dc154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.045+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5dc0638c0 0x7fd5dc0c15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.046+0000 7fd5e1521700 1 -- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5dc151de0 msgr2=0x7fd5dc154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.046+0000 7fd5e1521700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5dc151de0 0x7fd5dc154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.046+0000 7fd5e1521700 1 -- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5dc0638c0 msgr2=0x7fd5dc0c15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.046+0000 7fd5e1521700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5dc0638c0 0x7fd5dc0c15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.046+0000 7fd5e1521700 1 -- 172.21.15.29:0/3773649018 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d80092a0 con 0x7fd5dc066bb0 2024-01-24T14:14:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.046+0000 7fd5e1521700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5dc066bb0 0x7fd5dc150d20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7fd5d000c490 tx=0x7fd5d0007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.046+0000 7fd5cdffb700 1 -- 172.21.15.29:0/3773649018 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000ce40 con 0x7fd5dc066bb0 2024-01-24T14:14:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.046+0000 7fd5cdffb700 1 -- 172.21.15.29:0/3773649018 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d000f470 con 0x7fd5dc066bb0 2024-01-24T14:14:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.046+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3773649018 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5dc0c1e50 con 0x7fd5dc066bb0 2024-01-24T14:14:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.046+0000 7fd5cdffb700 1 -- 172.21.15.29:0/3773649018 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000f810 con 0x7fd5dc066bb0 2024-01-24T14:14:10.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.046+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3773649018 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5dc0c2410 con 0x7fd5dc066bb0 2024-01-24T14:14:10.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.048+0000 7fd5cdffb700 1 -- 172.21.15.29:0/3773649018 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd5d000fbc0 con 0x7fd5dc066bb0 2024-01-24T14:14:10.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.048+0000 7fd5cdffb700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5c005f270 0x7fd5c0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.048+0000 7fd5cdffb700 1 -- 172.21.15.29:0/3773649018 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fd5d0079cc0 con 0x7fd5dc066bb0 2024-01-24T14:14:10.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.048+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5c005f270 0x7fd5c0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.049+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5c005f270 0x7fd5c0061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd5d8006d60 tx=0x7fd5d8016000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:10.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.050+0000 7fd5f5951b80 1 Processor -- start 2024-01-24T14:14:10.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.050+0000 7fd5f5951b80 1 -- start start 2024-01-24T14:14:10.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.051+0000 7fd5f5951b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510ada1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.051+0000 7fd5f5951b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 0x56510ad9c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.051+0000 7fd5f5951b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ad9cdf0 0x56510ad9d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.051+0000 7fd5f5951b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510ad183d0 con 0x56510ada1620 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.051+0000 7fd5f5951b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510acddbe0 con 0x56510ad9cdf0 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.051+0000 7fd5f5951b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56510acf3260 con 0x56510adea2c0 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.051+0000 7fd5e1d22700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510ada1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.051+0000 7fd5e1d22700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510ada1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60026/0 (socket says 172.21.15.29:60026) 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.051+0000 7fd5e1d22700 1 -- 172.21.15.29:0/350868278 learned_addr learned my addr 172.21.15.29:0/350868278 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.051+0000 7fd5e1521700 1 --2- 172.21.15.29:0/350868278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 0x56510ad9c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.052+0000 7fd5e2523700 1 --2- 172.21.15.29:0/350868278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ad9cdf0 0x56510ad9d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.052+0000 7fd5e1d22700 1 -- 172.21.15.29:0/350868278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 msgr2=0x56510ad9c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.052+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/350868278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 0x56510ad9c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.052+0000 7fd5e1d22700 1 -- 172.21.15.29:0/350868278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ad9cdf0 msgr2=0x56510ad9d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.052+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/350868278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ad9cdf0 0x56510ad9d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.052+0000 7fd5e1d22700 1 -- 172.21.15.29:0/350868278 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56510acec2f0 con 0x56510ada1620 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.052+0000 7fd5e2523700 1 --2- 172.21.15.29:0/350868278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ad9cdf0 0x56510ad9d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:10.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.052+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/350868278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510ada1a00 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fd5d800ddf0 tx=0x7fd5d8020070).ready entity=mon.0 client_cookie=41023d8ef026be6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.052+0000 7fd5c6ffd700 1 -- 172.21.15.29:0/350868278 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d801e5e0 con 0x56510ada1620 2024-01-24T14:14:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.052+0000 7fd5c6ffd700 1 -- 172.21.15.29:0/350868278 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d801e740 con 0x56510ada1620 2024-01-24T14:14:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.052+0000 7fd5c6ffd700 1 -- 172.21.15.29:0/350868278 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d801eac0 con 0x56510ada1620 2024-01-24T14:14:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.053+0000 7fd5f5951b80 1 -- 172.21.15.29:0/350868278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 msgr2=0x56510ada1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.053+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/350868278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510ada1a00 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fd5d800ddf0 tx=0x7fd5d8020070).stop 2024-01-24T14:14:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.053+0000 7fd5f5951b80 1 -- 172.21.15.29:0/350868278 shutdown_connections 2024-01-24T14:14:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.053+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/350868278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ad9cdf0 0x56510ad9d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.053+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/350868278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510ada1a00 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.053+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/350868278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 0x56510ad9c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.053+0000 7fd5f5951b80 1 -- 172.21.15.29:0/350868278 >> 172.21.15.29:0/350868278 conn(0x56510ac0a2e0 msgr2=0x56510adea750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.053+0000 7fd5f5951b80 1 -- 172.21.15.29:0/350868278 shutdown_connections 2024-01-24T14:14:10.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.053+0000 7fd5f5951b80 1 -- 172.21.15.29:0/350868278 wait complete. 2024-01-24T14:14:10.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.053+0000 7fd5f5951b80 1 -- start start 2024-01-24T14:14:10.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.053+0000 7fd5f5951b80 1 Processor -- start 2024-01-24T14:14:10.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.054+0000 7fd5f5951b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ad9cdf0 0x56510adcbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.054+0000 7fd5f5951b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ada1620 0x56510adcc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.054+0000 7fd5f5951b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 0x56510adcf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.054+0000 7fd5f5951b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510acf33a0 con 0x56510ad9cdf0 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.054+0000 7fd5f5951b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510acf3620 con 0x56510ada1620 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5f5951b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56510acbe990 con 0x56510adea2c0 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5f5951b80 10 client.0 ll_register_callbacks cb 0x56510ad61610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5e1d22700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ad9cdf0 0x56510adcbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5e1d22700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ad9cdf0 0x56510adcbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60042/0 (socket says 172.21.15.29:60042) 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5e1d22700 1 -- 172.21.15.29:0/3104742934 learned_addr learned my addr 172.21.15.29:0/3104742934 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5e1521700 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ada1620 0x56510adcc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5e2523700 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 0x56510adcf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5e1d22700 1 -- 172.21.15.29:0/3104742934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 msgr2=0x56510adcf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:10.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 0x56510adcf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5e1d22700 1 -- 172.21.15.29:0/3104742934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ada1620 msgr2=0x56510adcc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ada1620 0x56510adcc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5e1d22700 1 -- 172.21.15.29:0/3104742934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5dc158490 con 0x56510ad9cdf0 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ad9cdf0 0x56510adcbf60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fd5d8009760 tx=0x7fd5d8029000).ready entity=mon.0 client_cookie=bf1c8b0d596aeba0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5bbfff700 1 -- 172.21.15.29:0/3104742934 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8028910 con 0x56510ad9cdf0 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5bbfff700 1 -- 172.21.15.29:0/3104742934 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fd5d8028a70 con 0x56510ad9cdf0 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3104742934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56510ad9bdf0 con 0x56510ad9cdf0 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.055+0000 7fd5bbfff700 1 -- 172.21.15.29:0/3104742934 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5d8028e40 con 0x56510ad9cdf0 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5f5951b80 10 client.6984 fetch_fsmap learned FSMap version 43 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3104742934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x56510ace7520 con 0x56510ad9cdf0 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5bbfff700 1 -- 172.21.15.29:0/3104742934 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56510ad9bdf0 con 0x56510ad9cdf0 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5bbfff700 1 -- 172.21.15.29:0/3104742934 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd5d8011400 con 0x56510ad9cdf0 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5f5951b80 10 client.6984 fetch_fsmap finished waiting for FSMap version 43 2024-01-24T14:14:10.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5f5951b80 10 client.6984 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5f5951b80 20 client.6984 populate_metadata read hostname 'smithi029' 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.29:6832/3854229544,v1:172.21.15.29:6833/3854229544] conn(0x56510adc9e40 0x56510adca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5f5951b80 4 client.6984 mds_command: new command op to 6807 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3104742934 --> [v2:172.21.15.29:6832/3854229544,v1:172.21.15.29:6833/3854229544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56510ace8d60 con 0x56510adc9e40 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5e1521700 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.29:6832/3854229544,v1:172.21.15.29:6833/3854229544] conn(0x56510adc9e40 0x56510adca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5bbfff700 10 client.6984 ms_handle_connect on v2:172.21.15.29:6832/3854229544 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.056+0000 7fd5e1521700 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.29:6832/3854229544,v1:172.21.15.29:6833/3854229544] conn(0x56510adc9e40 0x56510adca220 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=46d90e458d0cffdc server_cookie=528ad7b768869ccf in_seq=0 out_seq=0 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5bbfff700 1 -- 172.21.15.29:0/3104742934 <== mds.0 v2:172.21.15.29:6832/3854229544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x56510ace8d60 con 0x56510adc9e40 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5bbfff700 10 client.6984 handle_command_reply: tid=0 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5f5951b80 1 client.6984 shutdown 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3104742934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ad9cdf0 msgr2=0x56510adcbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ad9cdf0 0x56510adcbf60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fd5d8009760 tx=0x7fd5d8029000).stop 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3104742934 shutdown_connections 2024-01-24T14:14:10.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.29:6832/3854229544,v1:172.21.15.29:6833/3854229544] conn(0x56510adc9e40 0x56510adca220 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 0x56510adcf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ad9cdf0 0x56510adcbf60 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5bbfff700 0 client.6984 ms_handle_reset on v2:172.21.15.29:6832/3854229544 2024-01-24T14:14:10.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/3104742934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ada1620 0x56510adcc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3104742934 >> 172.21.15.29:0/3104742934 conn(0x56510ac0a2e0 msgr2=0x56510ad9f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:10.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3104742934 shutdown_connections 2024-01-24T14:14:10.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.057+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3104742934 wait complete. 2024-01-24T14:14:10.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.058+0000 7fd5f5951b80 20 client.6984 trim_cache size 0 max 16384 2024-01-24T14:14:10.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.075+0000 7fd5f5951b80 1 Processor -- start 2024-01-24T14:14:10.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.075+0000 7fd5f5951b80 1 -- start start 2024-01-24T14:14:10.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5f5951b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510ad9cdf0 0x56510adcb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5f5951b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510ade3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5f5951b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510adea2c0 0x56510adcdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5f5951b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510ada0760 con 0x56510ada1620 2024-01-24T14:14:10.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5f5951b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510adcffe0 con 0x56510adea2c0 2024-01-24T14:14:10.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5f5951b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56510ad960e0 con 0x56510ad9cdf0 2024-01-24T14:14:10.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5e1521700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510ade3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5e1521700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510ade3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60058/0 (socket says 172.21.15.29:60058) 2024-01-24T14:14:10.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5e1521700 1 -- 172.21.15.29:0/3626209809 learned_addr learned my addr 172.21.15.29:0/3626209809 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5e2523700 1 --2- 172.21.15.29:0/3626209809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510adea2c0 0x56510adcdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/3626209809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510ad9cdf0 0x56510adcb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5e1521700 1 -- 172.21.15.29:0/3626209809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510ad9cdf0 msgr2=0x56510adcb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5e1521700 1 --2- 172.21.15.29:0/3626209809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510ad9cdf0 0x56510adcb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5e1521700 1 -- 172.21.15.29:0/3626209809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510adea2c0 msgr2=0x56510adcdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5e1521700 1 --2- 172.21.15.29:0/3626209809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510adea2c0 0x56510adcdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.076+0000 7fd5e1521700 1 -- 172.21.15.29:0/3626209809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56510adeef30 con 0x56510ada1620 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5e1521700 1 --2- 172.21.15.29:0/3626209809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510ade3f70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fd5d004e600 tx=0x7fd5d001e040).ready entity=mon.0 client_cookie=26ec513fade3a614 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5c77fe700 1 -- 172.21.15.29:0/3626209809 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d004c810 con 0x56510ada1620 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5c77fe700 1 -- 172.21.15.29:0/3626209809 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d004ca70 con 0x56510ada1620 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5c77fe700 1 -- 172.21.15.29:0/3626209809 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d004dbc0 con 0x56510ada1620 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3626209809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 msgr2=0x56510ade3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/3626209809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510ade3f70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fd5d004e600 tx=0x7fd5d001e040).stop 2024-01-24T14:14:10.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3626209809 shutdown_connections 2024-01-24T14:14:10.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/3626209809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510adea2c0 0x56510adcdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/3626209809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510ad9cdf0 0x56510adcb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/3626209809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510ade3f70 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3626209809 >> 172.21.15.29:0/3626209809 conn(0x56510ac0a2e0 msgr2=0x56510adea750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:10.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3626209809 shutdown_connections 2024-01-24T14:14:10.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.077+0000 7fd5f5951b80 1 -- 172.21.15.29:0/3626209809 wait complete. 2024-01-24T14:14:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.078+0000 7fd5f5951b80 1 -- start start 2024-01-24T14:14:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.078+0000 7fd5f5951b80 1 Processor -- start 2024-01-24T14:14:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.079+0000 7fd5f5951b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ad9cdf0 0x56510ae45c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.079+0000 7fd5f5951b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510adea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.079+0000 7fd5f5951b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 0x56510ad98020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.079+0000 7fd5f5951b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510ae06710 con 0x56510ada1620 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.079+0000 7fd5f5951b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510ae03e00 con 0x56510ad9cdf0 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.079+0000 7fd5f5951b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56510ae4c4c0 con 0x56510adea2c0 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.079+0000 7fd5f5951b80 10 client.0 ll_register_callbacks cb 0x56510ad61610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.079+0000 7fd5e1521700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510adea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.079+0000 7fd5e1521700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510adea7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60074/0 (socket says 172.21.15.29:60074) 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.079+0000 7fd5e1521700 1 -- 172.21.15.29:0/115563855 learned_addr learned my addr 172.21.15.29:0/115563855 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.079+0000 7fd5e2523700 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 0x56510ad98020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.079+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ad9cdf0 0x56510ae45c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5e1521700 1 -- 172.21.15.29:0/115563855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 msgr2=0x56510ad98020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5e1521700 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 0x56510ad98020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5e1521700 1 -- 172.21.15.29:0/115563855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ad9cdf0 msgr2=0x56510ae45c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5e1521700 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ad9cdf0 0x56510ae45c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5e1521700 1 -- 172.21.15.29:0/115563855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5dc0c2410 con 0x56510ada1620 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5e1521700 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510adea7f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fd5d004e600 tx=0x7fd5d005c000).ready entity=mon.0 client_cookie=2172cee8eb116e50 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:10.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5bbfff700 1 -- 172.21.15.29:0/115563855 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d004dbc0 con 0x56510ada1620 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5bbfff700 1 -- 172.21.15.29:0/115563855 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fd5d004ade0 con 0x56510ada1620 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5f5951b80 1 -- 172.21.15.29:0/115563855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56510ad9cc80 con 0x56510ada1620 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5bbfff700 1 -- 172.21.15.29:0/115563855 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5d004afa0 con 0x56510ada1620 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5f5951b80 10 client.6990 fetch_fsmap learned FSMap version 43 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5f5951b80 1 -- 172.21.15.29:0/115563855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x56510ace7270 con 0x56510ada1620 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5bbfff700 1 -- 172.21.15.29:0/115563855 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56510ad9cc80 con 0x56510ada1620 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5bbfff700 1 -- 172.21.15.29:0/115563855 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 43) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd5d004a4c0 con 0x56510ada1620 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5f5951b80 10 client.6990 fetch_fsmap finished waiting for FSMap version 43 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5f5951b80 10 client.6990 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5f5951b80 20 client.6990 populate_metadata read hostname 'smithi029' 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.29:6832/3854229544,v1:172.21.15.29:6833/3854229544] conn(0x56510ae05750 0x56510ae05b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5f5951b80 4 client.6990 mds_command: new command op to 6807 tid=0[{"prefix": "status"}] 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.080+0000 7fd5f5951b80 1 -- 172.21.15.29:0/115563855 --> [v2:172.21.15.29:6832/3854229544,v1:172.21.15.29:6833/3854229544] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fd5d801e740 con 0x56510ae05750 2024-01-24T14:14:10.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.29:6832/3854229544,v1:172.21.15.29:6833/3854229544] conn(0x56510ae05750 0x56510ae05b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5bbfff700 10 client.6990 ms_handle_connect on v2:172.21.15.29:6832/3854229544 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5e1d22700 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.29:6832/3854229544,v1:172.21.15.29:6833/3854229544] conn(0x56510ae05750 0x56510ae05b30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b2fb0710b130410c server_cookie=849a91fbed6438c2 in_seq=0 out_seq=0 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5bbfff700 1 -- 172.21.15.29:0/115563855 <== mds.0 v2:172.21.15.29:6832/3854229544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7fd5d801e740 con 0x56510ae05750 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5bbfff700 10 client.6990 handle_command_reply: tid=0 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5f5951b80 1 client.6990 shutdown 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5f5951b80 1 -- 172.21.15.29:0/115563855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 msgr2=0x56510adea7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510adea7f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fd5d004e600 tx=0x7fd5d005c000).stop 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5f5951b80 1 -- 172.21.15.29:0/115563855 shutdown_connections 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.29:6832/3854229544,v1:172.21.15.29:6833/3854229544] conn(0x56510ae05750 0x56510ae05b30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56510adea2c0 0x56510ad98020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56510ad9cdf0 0x56510ae45c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5f5951b80 1 --2- 172.21.15.29:0/115563855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56510ada1620 0x56510adea7f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5bbfff700 0 client.6990 ms_handle_reset on v2:172.21.15.29:6832/3854229544 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5f5951b80 1 -- 172.21.15.29:0/115563855 >> 172.21.15.29:0/115563855 conn(0x56510ac0a2e0 msgr2=0x56510adc8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:10.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.081+0000 7fd5f5951b80 1 -- 172.21.15.29:0/115563855 shutdown_connections 2024-01-24T14:14:10.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5f5951b80 1 -- 172.21.15.29:0/115563855 wait complete. 2024-01-24T14:14:10.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5f5951b80 20 client.6990 trim_cache size 0 max 16384 2024-01-24T14:14:10.088 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T14:14:10.088 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T14:14:10.088 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T14:14:10.088 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 6807, 2024-01-24T14:14:10.088 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T14:14:10.088 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T14:14:10.088 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T14:14:10.088 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 10.503643290999999, 2024-01-24T14:14:10.089 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 43, 2024-01-24T14:14:10.089 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 105, 2024-01-24T14:14:10.089 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 105, 2024-01-24T14:14:10.089 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 15.943942558 2024-01-24T14:14:10.089 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T14:14:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5c005f270 msgr2=0x7fd5c0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5e3f86700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5c005f270 0x7fd5c0061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd5d8006d60 tx=0x7fd5d8016000).stop 2024-01-24T14:14:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3773649018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5dc066bb0 msgr2=0x7fd5dc150d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5e3f86700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5dc066bb0 0x7fd5dc150d20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7fd5d000c490 tx=0x7fd5d0007c90).stop 2024-01-24T14:14:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3773649018 shutdown_connections 2024-01-24T14:14:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5e3f86700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5dc151de0 0x7fd5dc154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5e3f86700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5dc0638c0 0x7fd5dc0c15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5e3f86700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd5c005f270 0x7fd5c0061730 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5e3f86700 1 --2- 172.21.15.29:0/3773649018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5dc066bb0 0x7fd5dc150d20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3773649018 >> 172.21.15.29:0/3773649018 conn(0x7fd5dc0c5640 msgr2=0x7fd5dc153460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3773649018 shutdown_connections 2024-01-24T14:14:10.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.082+0000 7fd5e3f86700 1 -- 172.21.15.29:0/3773649018 wait complete. 2024-01-24T14:14:10.096 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T14:14:10.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.211+0000 7fc11fbbd700 1 Processor -- start 2024-01-24T14:14:10.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.211+0000 7fc11fbbd700 1 -- start start 2024-01-24T14:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.212+0000 7fc11fbbd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc1180be420 0x7fc1180be8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.212+0000 7fc11fbbd700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1180a7490 con 0x7fc1180c6b50 2024-01-24T14:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.212+0000 7fc11fbbd700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1180bed90 con 0x7fc1181520e0 2024-01-24T14:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.212+0000 7fc11fbbd700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1180a8520 con 0x7fc1180be420 2024-01-24T14:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.213+0000 7fc11d158700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc1180be420 0x7fc1180be8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.213+0000 7fc11d158700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc1180be420 0x7fc1180be8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60084/0 (socket says 172.21.15.29:60084) 2024-01-24T14:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.213+0000 7fc11d158700 1 -- 172.21.15.29:0/3518714715 learned_addr learned my addr 172.21.15.29:0/3518714715 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.213+0000 7fc11e15a700 1 --1- 172.21.15.29:0/3518714715 >> v1:172.21.15.29:6789/0 conn(0x7fc1180c6b50 0x7fc1180c6f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39616/0 (socket says 172.21.15.29:39616) 2024-01-24T14:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.213+0000 7fc11d158700 1 -- 172.21.15.29:0/3518714715 >> v1:172.21.15.163:6789/0 conn(0x7fc1181520e0 legacy=0x7fc1180a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.213+0000 7fc11d158700 1 -- 172.21.15.29:0/3518714715 >> v1:172.21.15.29:6789/0 conn(0x7fc1180c6b50 legacy=0x7fc1180c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.213+0000 7fc11d158700 1 -- 172.21.15.29:0/3518714715 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1180a6880 con 0x7fc1180be420 2024-01-24T14:14:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.213+0000 7fc11c957700 1 -- 172.21.15.29:0/3518714715 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769027561 0 0) 0x7fc1180a7490 con 0x7fc1180c6b50 2024-01-24T14:14:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.213+0000 7fc11d158700 1 --2- 172.21.15.29:0/3518714715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1180be420 0x7fc1180be8a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7fc108006fd0 tx=0x7fc1080090f0).ready entity=mon.0 client_cookie=7e74d36155f1c8a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.213+0000 7fc11c957700 1 -- 172.21.15.29:0/3518714715 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800b600 con 0x7fc1180be420 2024-01-24T14:14:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.213+0000 7fc11c957700 1 -- 172.21.15.29:0/3518714715 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10800b760 con 0x7fc1180be420 2024-01-24T14:14:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.213+0000 7fc11c957700 1 -- 172.21.15.29:0/3518714715 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800bae0 con 0x7fc1180be420 2024-01-24T14:14:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.214+0000 7fc11fbbd700 1 -- 172.21.15.29:0/3518714715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1180be420 msgr2=0x7fc1180be8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.214+0000 7fc11fbbd700 1 --2- 172.21.15.29:0/3518714715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1180be420 0x7fc1180be8a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7fc108006fd0 tx=0x7fc1080090f0).stop 2024-01-24T14:14:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.214+0000 7fc11fbbd700 1 -- 172.21.15.29:0/3518714715 shutdown_connections 2024-01-24T14:14:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.214+0000 7fc11fbbd700 1 --2- 172.21.15.29:0/3518714715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1180be420 0x7fc1180be8a0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.214+0000 7fc11fbbd700 1 -- 172.21.15.29:0/3518714715 >> 172.21.15.29:0/3518714715 conn(0x7fc1180922f0 msgr2=0x7fc118062f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.214+0000 7fc11fbbd700 1 -- 172.21.15.29:0/3518714715 shutdown_connections 2024-01-24T14:14:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.214+0000 7fc11fbbd700 1 -- 172.21.15.29:0/3518714715 wait complete. 2024-01-24T14:14:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.215+0000 7fc11fbbd700 1 Processor -- start 2024-01-24T14:14:10.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.215+0000 7fc11fbbd700 1 -- start start 2024-01-24T14:14:10.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11fbbd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1180be420 0x7fc118142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11fbbd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1180c6b50 0x7fc1180c0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11fbbd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1181520e0 0x7fc1180c1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11fbbd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1181633f0 con 0x7fc1180be420 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11fbbd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180c17d0 con 0x7fc1180c6b50 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11fbbd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180a9550 con 0x7fc1181520e0 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11d959700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1180be420 0x7fc118142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11d959700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1180be420 0x7fc118142970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60088/0 (socket says 172.21.15.29:60088) 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11d959700 1 -- 172.21.15.29:0/1445607717 learned_addr learned my addr 172.21.15.29:0/1445607717 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11e15a700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1181520e0 0x7fc1180c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11d158700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1180c6b50 0x7fc1180c0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11d959700 1 -- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1181520e0 msgr2=0x7fc1180c1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11d959700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1181520e0 0x7fc1180c1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11d959700 1 -- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1180c6b50 msgr2=0x7fc1180c0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.216+0000 7fc11d959700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1180c6b50 0x7fc1180c0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.217+0000 7fc11d959700 1 -- 172.21.15.29:0/1445607717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc108009370 con 0x7fc1180be420 2024-01-24T14:14:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.217+0000 7fc11d158700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1180c6b50 0x7fc1180c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.217+0000 7fc11d959700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1180be420 0x7fc118142970 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7fc11400c440 tx=0x7fc114007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.217+0000 7fc10dffb700 1 -- 172.21.15.29:0/1445607717 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400ce40 con 0x7fc1180be420 2024-01-24T14:14:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.217+0000 7fc10dffb700 1 -- 172.21.15.29:0/1445607717 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc114007020 con 0x7fc1180be420 2024-01-24T14:14:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.217+0000 7fc10dffb700 1 -- 172.21.15.29:0/1445607717 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1140073c0 con 0x7fc1180be420 2024-01-24T14:14:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.217+0000 7fc11fbbd700 1 -- 172.21.15.29:0/1445607717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1180a8790 con 0x7fc1180be420 2024-01-24T14:14:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.217+0000 7fc11fbbd700 1 -- 172.21.15.29:0/1445607717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1180a8d50 con 0x7fc1180be420 2024-01-24T14:14:10.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.218+0000 7fc11fbbd700 1 -- 172.21.15.29:0/1445607717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11805f400 con 0x7fc1180be420 2024-01-24T14:14:10.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.218+0000 7fc10dffb700 1 -- 172.21.15.29:0/1445607717 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc114006900 con 0x7fc1180be420 2024-01-24T14:14:10.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.219+0000 7fc10dffb700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc0fc05f240 0x7fc0fc061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:10.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.219+0000 7fc10dffb700 1 -- 172.21.15.29:0/1445607717 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fc114079b00 con 0x7fc1180be420 2024-01-24T14:14:10.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.219+0000 7fc11d158700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc0fc05f240 0x7fc0fc061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:10.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.220+0000 7fc11d158700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc0fc05f240 0x7fc0fc061700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc108009130 tx=0x7fc108017000).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:10.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.223+0000 7fc10dffb700 1 -- 172.21.15.29:0/1445607717 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc114047b00 con 0x7fc1180be420 2024-01-24T14:14:10.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.449+0000 7fc11fbbd700 1 -- 172.21.15.29:0/1445607717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc1180a9360 con 0x7fc1180be420 2024-01-24T14:14:10.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.449+0000 7fc10dffb700 1 -- 172.21.15.29:0/1445607717 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc11404e280 con 0x7fc1180be420 2024-01-24T14:14:10.450 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-24T14:14:10.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.451+0000 7fc11fbbd700 1 -- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc0fc05f240 msgr2=0x7fc0fc061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.451+0000 7fc11fbbd700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc0fc05f240 0x7fc0fc061700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc108009130 tx=0x7fc108017000).stop 2024-01-24T14:14:10.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.451+0000 7fc11fbbd700 1 -- 172.21.15.29:0/1445607717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1180be420 msgr2=0x7fc118142970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:10.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.451+0000 7fc11fbbd700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1180be420 0x7fc118142970 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7fc11400c440 tx=0x7fc114007e50).stop 2024-01-24T14:14:10.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.451+0000 7fc11fbbd700 1 -- 172.21.15.29:0/1445607717 shutdown_connections 2024-01-24T14:14:10.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.451+0000 7fc11fbbd700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1181520e0 0x7fc1180c1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.451+0000 7fc11fbbd700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc0fc05f240 0x7fc0fc061700 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.451+0000 7fc11fbbd700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1180be420 0x7fc118142970 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.451+0000 7fc11fbbd700 1 --2- 172.21.15.29:0/1445607717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1180c6b50 0x7fc1180c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:10.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.451+0000 7fc11fbbd700 1 -- 172.21.15.29:0/1445607717 >> 172.21.15.29:0/1445607717 conn(0x7fc1180922f0 msgr2=0x7fc118062f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:10.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.452+0000 7fc11fbbd700 1 -- 172.21.15.29:0/1445607717 shutdown_connections 2024-01-24T14:14:10.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:10.452+0000 7fc11fbbd700 1 -- 172.21.15.29:0/1445607717 wait complete. 2024-01-24T14:14:10.464 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-24T14:14:10.464 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T14:14:10.464 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T14:14:10.464 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-24T14:14:10.464 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-24T14:14:10.465 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-24T14:14:10.465 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-24T14:14:10.465 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T14:14:10.465 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T14:14:10.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:10.465 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T14:14:10.484 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T14:14:10.485 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 41400sec preferred_lft 41400sec 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::24bc:1cff:fe57:e704/64 scope link 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:10.486 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T14:14:10.487 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T14:14:10.487 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::2034:aaff:fef3:f12b/64 scope link 2024-01-24T14:14:10.487 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:10.487 INFO:teuthology.orchestra.run.smithi029.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T14:14:10.487 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether ba:ca:42:4a:9e:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T14:14:10.487 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::b8ca:42ff:fe4a:9e52/64 scope link 2024-01-24T14:14:10.487 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:10.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:10.487 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T14:14:10.556 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T14:14:10.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:10.558 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T14:14:10.623 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T14:14:10.623 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T14:14:10.623 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T14:14:10.623 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T14:14:10.623 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T14:14:10.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:10.623 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T14:14:10.623 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T14:14:10.623 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T14:14:10.624 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T14:14:10.624 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T14:14:10.624 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T14:14:10.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:10.750 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T14:14:10.770 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-24T14:14:10.770 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T14:14:10.770 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T14:14:10.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:10.771 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T14:14:10.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:10.846 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-01-24T14:14:10.908 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T14:14:10.909 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:14:11.001 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:14:11.002 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T14:14:11.025 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:14:11.025 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T14:14:11.028 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T14:14:11.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:11.028 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-24T14:14:11.061 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-24T14:14:11.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.240+0000 7fea09768700 1 Processor -- start 2024-01-24T14:14:11.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.240+0000 7fea09768700 1 -- start start 2024-01-24T14:14:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.241+0000 7fea09768700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea04150bd0 0x7fea04065e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.241+0000 7fea09768700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fea04069240 0x7fea04150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.241+0000 7fea09768700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0406a350 con 0x7fea041514a0 2024-01-24T14:14:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.241+0000 7fea09768700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea04069740 con 0x7fea04069240 2024-01-24T14:14:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.241+0000 7fea09768700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea04069a20 con 0x7fea04150bd0 2024-01-24T14:14:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.242+0000 7fea027fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fea04069240 0x7fea04150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.242+0000 7fea02ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea04150bd0 0x7fea04065e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.242+0000 7fea037fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fea041514a0 0x7fea041548e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:48154/0 (socket says 172.21.15.29:48154) 2024-01-24T14:14:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.242+0000 7fea037fe700 1 -- 172.21.15.29:0/2768299914 learned_addr learned my addr 172.21.15.29:0/2768299914 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.242+0000 7fea027fc700 1 -- 172.21.15.29:0/2768299914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea04150bd0 msgr2=0x7fea04065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.242+0000 7fea027fc700 1 --2- 172.21.15.29:0/2768299914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea04150bd0 0x7fea04065e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.242+0000 7fea027fc700 1 -- 172.21.15.29:0/2768299914 >> v1:172.21.15.163:6789/0 conn(0x7fea041514a0 legacy=0x7fea041548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.242+0000 7fea027fc700 1 -- 172.21.15.29:0/2768299914 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04069d00 con 0x7fea04069240 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.242+0000 7fea01ffb700 1 -- 172.21.15.29:0/2768299914 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167841343 0 0) 0x7fea0406a350 con 0x7fea041514a0 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.242+0000 7fea02ffd700 1 --2- 172.21.15.29:0/2768299914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea04150bd0 0x7fea04065e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.242+0000 7fea027fc700 1 --2- 172.21.15.29:0/2768299914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea04069240 0x7fea04150fb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fe9f8009760 tx=0x7fe9f8009060).ready entity=mon.1 client_cookie=58897a9ff181c78 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.243+0000 7fea01ffb700 1 -- 172.21.15.29:0/2768299914 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800f510 con 0x7fea04069240 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.243+0000 7fea01ffb700 1 -- 172.21.15.29:0/2768299914 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f800f670 con 0x7fea04069240 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.243+0000 7fea01ffb700 1 -- 172.21.15.29:0/2768299914 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800f9f0 con 0x7fea04069240 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.243+0000 7fea09768700 1 -- 172.21.15.29:0/2768299914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea04069240 msgr2=0x7fea04150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.243+0000 7fea09768700 1 --2- 172.21.15.29:0/2768299914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea04069240 0x7fea04150fb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fe9f8009760 tx=0x7fe9f8009060).stop 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.243+0000 7fea09768700 1 -- 172.21.15.29:0/2768299914 shutdown_connections 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.243+0000 7fea09768700 1 --2- 172.21.15.29:0/2768299914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea04150bd0 0x7fea04065e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.243+0000 7fea09768700 1 --2- 172.21.15.29:0/2768299914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea04069240 0x7fea04150fb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.243+0000 7fea09768700 1 -- 172.21.15.29:0/2768299914 >> 172.21.15.29:0/2768299914 conn(0x7fea040c8580 msgr2=0x7fea040c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:11.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.243+0000 7fea09768700 1 -- 172.21.15.29:0/2768299914 shutdown_connections 2024-01-24T14:14:11.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.243+0000 7fea09768700 1 -- 172.21.15.29:0/2768299914 wait complete. 2024-01-24T14:14:11.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.244+0000 7fea09768700 1 Processor -- start 2024-01-24T14:14:11.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.245+0000 7fea09768700 1 -- start start 2024-01-24T14:14:11.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.245+0000 7fea09768700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea04069240 0x7fea040c0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:11.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.245+0000 7fea09768700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea04150bd0 0x7fea04063cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:11.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.245+0000 7fea09768700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea041514a0 0x7fea04165f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:11.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.245+0000 7fea09768700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea04142620 con 0x7fea04150bd0 2024-01-24T14:14:11.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.245+0000 7fea09768700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea04142170 con 0x7fea04069240 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.245+0000 7fea09768700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea040ace30 con 0x7fea041514a0 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.245+0000 7fea027fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea04150bd0 0x7fea04063cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fea027fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea04150bd0 0x7fea04063cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60098/0 (socket says 172.21.15.29:60098) 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fea027fc700 1 -- 172.21.15.29:0/2445541993 learned_addr learned my addr 172.21.15.29:0/2445541993 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fea037fe700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea041514a0 0x7fea04165f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fea02ffd700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea04069240 0x7fea040c0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fea027fc700 1 -- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea041514a0 msgr2=0x7fea04165f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fea027fc700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea041514a0 0x7fea04165f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fea027fc700 1 -- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea04069240 msgr2=0x7fea040c0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fea027fc700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea04069240 0x7fea040c0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fea027fc700 1 -- 172.21.15.29:0/2445541993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f40092a0 con 0x7fea04150bd0 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fea027fc700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea04150bd0 0x7fea04063cc0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7fea040649f0 tx=0x7fe9f8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:11.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fe9eaffd700 1 -- 172.21.15.29:0/2445541993 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800fe80 con 0x7fea04150bd0 2024-01-24T14:14:11.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fe9eaffd700 1 -- 172.21.15.29:0/2445541993 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f800f7e0 con 0x7fea04150bd0 2024-01-24T14:14:11.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fea09768700 1 -- 172.21.15.29:0/2445541993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f80092a0 con 0x7fea04150bd0 2024-01-24T14:14:11.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fe9eaffd700 1 -- 172.21.15.29:0/2445541993 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f801b640 con 0x7fea04150bd0 2024-01-24T14:14:11.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.246+0000 7fea09768700 1 -- 172.21.15.29:0/2445541993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea040c1620 con 0x7fea04150bd0 2024-01-24T14:14:11.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.247+0000 7fea09768700 1 -- 172.21.15.29:0/2445541993 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea040475e0 con 0x7fea04150bd0 2024-01-24T14:14:11.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.248+0000 7fe9eaffd700 1 -- 172.21.15.29:0/2445541993 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe9f801ba30 con 0x7fea04150bd0 2024-01-24T14:14:11.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.248+0000 7fe9eaffd700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe9f005f270 0x7fe9f0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:11.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.248+0000 7fe9eaffd700 1 -- 172.21.15.29:0/2445541993 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fe9f807c130 con 0x7fea04150bd0 2024-01-24T14:14:11.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.248+0000 7fea02ffd700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe9f005f270 0x7fe9f0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:11.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.249+0000 7fea02ffd700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe9f005f270 0x7fe9f0061730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fe9f4009dc0 tx=0x7fe9f4009060).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:11.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.253+0000 7fe9eaffd700 1 -- 172.21.15.29:0/2445541993 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe9f804ce80 con 0x7fea04150bd0 2024-01-24T14:14:11.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.459+0000 7fea09768700 1 -- 172.21.15.29:0/2445541993 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fea04069fe0 con 0x7fe9f005f270 2024-01-24T14:14:11.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.461+0000 7fe9eaffd700 1 -- 172.21.15.29:0/2445541993 <== mgr.5864 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fea04069fe0 con 0x7fe9f005f270 2024-01-24T14:14:11.462 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T14:14:11.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:11.462 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-01-24T14:14:11.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:11.462 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T14:14:11.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.463+0000 7fea09768700 1 -- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe9f005f270 msgr2=0x7fe9f0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.463+0000 7fea09768700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe9f005f270 0x7fe9f0061730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fe9f4009dc0 tx=0x7fe9f4009060).stop 2024-01-24T14:14:11.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.463+0000 7fea09768700 1 -- 172.21.15.29:0/2445541993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea04150bd0 msgr2=0x7fea04063cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.463+0000 7fea09768700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea04150bd0 0x7fea04063cc0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7fea040649f0 tx=0x7fe9f8017000).stop 2024-01-24T14:14:11.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.463+0000 7fea09768700 1 -- 172.21.15.29:0/2445541993 shutdown_connections 2024-01-24T14:14:11.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.463+0000 7fea09768700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea041514a0 0x7fea04165f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.463+0000 7fea09768700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea04069240 0x7fea040c0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.463+0000 7fea09768700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe9f005f270 0x7fe9f0061730 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.463+0000 7fea09768700 1 --2- 172.21.15.29:0/2445541993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea04150bd0 0x7fea04063cc0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.463+0000 7fea09768700 1 -- 172.21.15.29:0/2445541993 >> 172.21.15.29:0/2445541993 conn(0x7fea040c8580 msgr2=0x7fea040bcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:11.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.463+0000 7fea09768700 1 -- 172.21.15.29:0/2445541993 shutdown_connections 2024-01-24T14:14:11.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.463+0000 7fea09768700 1 -- 172.21.15.29:0/2445541993 wait complete. 2024-01-24T14:14:11.477 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-01-24T14:14:11.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.608+0000 7ff5f6772700 1 Processor -- start 2024-01-24T14:14:11.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.608+0000 7ff5f6772700 1 -- start start 2024-01-24T14:14:11.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.609+0000 7ff5f6772700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5f014b710 0x7ff5f00c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.609+0000 7ff5f6772700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f0155840 con 0x7ff5f014b330 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.609+0000 7ff5f6772700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f0069280 con 0x7ff5f01550c0 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.609+0000 7ff5f6772700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f00c3910 con 0x7ff5f014b710 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.609+0000 7ff5ef7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5f014b710 0x7ff5f00c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.609+0000 7ff5effff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff5f014b330 0x7ff5f01549b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39618/0 (socket says 172.21.15.29:39618) 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.609+0000 7ff5effff700 1 -- 172.21.15.29:0/3106122102 learned_addr learned my addr 172.21.15.29:0/3106122102 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.610+0000 7ff5ef7fe700 1 -- 172.21.15.29:0/3106122102 >> v1:172.21.15.163:6789/0 conn(0x7ff5f01550c0 legacy=0x7ff5f00c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.610+0000 7ff5ef7fe700 1 -- 172.21.15.29:0/3106122102 >> v1:172.21.15.29:6789/0 conn(0x7ff5f014b330 legacy=0x7ff5f01549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.610+0000 7ff5ef7fe700 1 -- 172.21.15.29:0/3106122102 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f00c3bf0 con 0x7ff5f014b710 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.610+0000 7ff5eeffd700 1 -- 172.21.15.29:0/3106122102 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258858225 0 0) 0x7ff5f0155840 con 0x7ff5f014b330 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.610+0000 7ff5ef7fe700 1 --2- 172.21.15.29:0/3106122102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f014b710 0x7ff5f00c2c50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7ff5e4009760 tx=0x7ff5e4009060).ready entity=mon.0 client_cookie=8a26adad3a101442 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.610+0000 7ff5eeffd700 1 -- 172.21.15.29:0/3106122102 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400b580 con 0x7ff5f014b710 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.610+0000 7ff5eeffd700 1 -- 172.21.15.29:0/3106122102 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e400b6e0 con 0x7ff5f014b710 2024-01-24T14:14:11.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.610+0000 7ff5eeffd700 1 -- 172.21.15.29:0/3106122102 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400ba60 con 0x7ff5f014b710 2024-01-24T14:14:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.610+0000 7ff5f6772700 1 -- 172.21.15.29:0/3106122102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f014b710 msgr2=0x7ff5f00c2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.610+0000 7ff5f6772700 1 --2- 172.21.15.29:0/3106122102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f014b710 0x7ff5f00c2c50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7ff5e4009760 tx=0x7ff5e4009060).stop 2024-01-24T14:14:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.611+0000 7ff5f6772700 1 -- 172.21.15.29:0/3106122102 shutdown_connections 2024-01-24T14:14:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.611+0000 7ff5f6772700 1 --2- 172.21.15.29:0/3106122102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f014b710 0x7ff5f00c2c50 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.611+0000 7ff5f6772700 1 -- 172.21.15.29:0/3106122102 >> 172.21.15.29:0/3106122102 conn(0x7ff5f0064390 msgr2=0x7ff5f00c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.611+0000 7ff5f6772700 1 -- 172.21.15.29:0/3106122102 shutdown_connections 2024-01-24T14:14:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.611+0000 7ff5f6772700 1 -- 172.21.15.29:0/3106122102 wait complete. 2024-01-24T14:14:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.612+0000 7ff5f6772700 1 Processor -- start 2024-01-24T14:14:11.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.612+0000 7ff5f6772700 1 -- start start 2024-01-24T14:14:11.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5f6772700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5f014b330 0x7ff5f0150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:11.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5f6772700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f014b710 0x7ff5f013f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:11.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5f6772700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5f01550c0 0x7ff5f00bd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:11.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5f6772700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f0168630 con 0x7ff5f014b710 2024-01-24T14:14:11.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5f6772700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f0167a20 con 0x7ff5f014b330 2024-01-24T14:14:11.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5f6772700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f0167d00 con 0x7ff5f01550c0 2024-01-24T14:14:11.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5ef7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f014b710 0x7ff5f013f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:11.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5ef7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f014b710 0x7ff5f013f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60118/0 (socket says 172.21.15.29:60118) 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5ef7fe700 1 -- 172.21.15.29:0/1366664120 learned_addr learned my addr 172.21.15.29:0/1366664120 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5effff700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5f014b330 0x7ff5f0150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5f4d0f700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5f01550c0 0x7ff5f00bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5ef7fe700 1 -- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5f01550c0 msgr2=0x7ff5f00bd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5ef7fe700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5f01550c0 0x7ff5f00bd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5ef7fe700 1 -- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5f014b330 msgr2=0x7ff5f0150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5ef7fe700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5f014b330 0x7ff5f0150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5ef7fe700 1 -- 172.21.15.29:0/1366664120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e40092a0 con 0x7ff5f014b710 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5effff700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5f014b330 0x7ff5f0150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.613+0000 7ff5f4d0f700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5f01550c0 0x7ff5f00bd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.614+0000 7ff5ef7fe700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f014b710 0x7ff5f013f0a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7ff5f005ee00 tx=0x7ff5e4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.614+0000 7ff5d7fff700 1 -- 172.21.15.29:0/1366664120 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400ba60 con 0x7ff5f014b710 2024-01-24T14:14:11.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.614+0000 7ff5d7fff700 1 -- 172.21.15.29:0/1366664120 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e4000c90 con 0x7ff5f014b710 2024-01-24T14:14:11.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.614+0000 7ff5d7fff700 1 -- 172.21.15.29:0/1366664120 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e401b660 con 0x7ff5f014b710 2024-01-24T14:14:11.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.614+0000 7ff5f6772700 1 -- 172.21.15.29:0/1366664120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f0167fe0 con 0x7ff5f014b710 2024-01-24T14:14:11.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.614+0000 7ff5f6772700 1 -- 172.21.15.29:0/1366664120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f0150a50 con 0x7ff5f014b710 2024-01-24T14:14:11.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.615+0000 7ff5f6772700 1 -- 172.21.15.29:0/1366664120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f00475e0 con 0x7ff5f014b710 2024-01-24T14:14:11.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.615+0000 7ff5d7fff700 1 -- 172.21.15.29:0/1366664120 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff5e401ba10 con 0x7ff5f014b710 2024-01-24T14:14:11.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.615+0000 7ff5d7fff700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5dc05f270 0x7ff5dc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:11.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.616+0000 7ff5d7fff700 1 -- 172.21.15.29:0/1366664120 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7ff5e407c420 con 0x7ff5f014b710 2024-01-24T14:14:11.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.616+0000 7ff5effff700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5dc05f270 0x7ff5dc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:11.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.616+0000 7ff5effff700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5dc05f270 0x7ff5dc061730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff5e0000c00 tx=0x7ff5e0009040).ready entity=mgr.5864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:11.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.620+0000 7ff5d7fff700 1 -- 172.21.15.29:0/1366664120 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff5e4050b40 con 0x7ff5f014b710 2024-01-24T14:14:11.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.847+0000 7ff5f6772700 1 -- 172.21.15.29:0/1366664120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7ff5f00c8820 con 0x7ff5f014b710 2024-01-24T14:14:11.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.861+0000 7ff5d7fff700 1 -- 172.21.15.29:0/1366664120 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 32) v1 ==== 75565+0+0 (secure 0 0 0) 0x7ff5e4079c70 con 0x7ff5f014b710 2024-01-24T14:14:11.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.862+0000 7ff5d7fff700 1 -- 172.21.15.29:0/1366664120 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v32) v1 ==== 92+0+0 (secure 0 0 0) 0x7ff5e4042a90 con 0x7ff5f014b710 2024-01-24T14:14:11.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.864+0000 7ff5f6772700 1 -- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5dc05f270 msgr2=0x7ff5dc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.864+0000 7ff5f6772700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5dc05f270 0x7ff5dc061730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff5e0000c00 tx=0x7ff5e0009040).stop 2024-01-24T14:14:11.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.864+0000 7ff5f6772700 1 -- 172.21.15.29:0/1366664120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f014b710 msgr2=0x7ff5f013f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:11.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.864+0000 7ff5f6772700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f014b710 0x7ff5f013f0a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7ff5f005ee00 tx=0x7ff5e4017000).stop 2024-01-24T14:14:11.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.865+0000 7ff5f6772700 1 -- 172.21.15.29:0/1366664120 shutdown_connections 2024-01-24T14:14:11.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.865+0000 7ff5f6772700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5f01550c0 0x7ff5f00bd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.865+0000 7ff5f6772700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5f014b330 0x7ff5f0150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.865+0000 7ff5f6772700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5dc05f270 0x7ff5dc061730 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.865+0000 7ff5f6772700 1 --2- 172.21.15.29:0/1366664120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5f014b710 0x7ff5f013f0a0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:11.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.865+0000 7ff5f6772700 1 -- 172.21.15.29:0/1366664120 >> 172.21.15.29:0/1366664120 conn(0x7ff5f0064390 msgr2=0x7ff5f014a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:11.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.865+0000 7ff5f6772700 1 -- 172.21.15.29:0/1366664120 shutdown_connections 2024-01-24T14:14:11.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:11.865+0000 7ff5f6772700 1 -- 172.21.15.29:0/1366664120 wait complete. 2024-01-24T14:14:11.879 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-01-24T14:14:12.008 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:12.007+0000 7fecbec05680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:14:12.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.014+0000 7fabe21b2700 1 Processor -- start 2024-01-24T14:14:12.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.014+0000 7fabe21b2700 1 -- start start 2024-01-24T14:14:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.015+0000 7fabe21b2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c8490 0x7fabdc0a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.015+0000 7fabe21b2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fabdc0bfd40 0x7fabdc0c01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.015+0000 7fabe21b2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc152240 con 0x7fabdc064760 2024-01-24T14:14:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.015+0000 7fabe21b2700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabdc1586c0 con 0x7fabdc0bfd40 2024-01-24T14:14:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.015+0000 7fabe21b2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0aa670 con 0x7fabdc0c8490 2024-01-24T14:14:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.015+0000 7fabdbfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fabdc0bfd40 0x7fabdc0c01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.015+0000 7fabdbfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fabdc0bfd40 0x7fabdc0c01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60132/0 (socket says 172.21.15.29:60132) 2024-01-24T14:14:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.015+0000 7fabdbfff700 1 -- 172.21.15.29:0/2305695628 learned_addr learned my addr 172.21.15.29:0/2305695628 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.015+0000 7fabd2ffd700 1 --2- 172.21.15.29:0/2305695628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c8490 0x7fabdc0a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.016+0000 7fabdbfff700 1 -- 172.21.15.29:0/2305695628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c8490 msgr2=0x7fabdc0a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.016+0000 7fabdbfff700 1 --2- 172.21.15.29:0/2305695628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c8490 0x7fabdc0a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:12.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.016+0000 7fabdbfff700 1 -- 172.21.15.29:0/2305695628 >> v1:172.21.15.163:6789/0 conn(0x7fabdc064760 legacy=0x7fabdc064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:12.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.016+0000 7fabdbfff700 1 -- 172.21.15.29:0/2305695628 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc157d90 con 0x7fabdc0bfd40 2024-01-24T14:14:12.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.016+0000 7fabd2ffd700 1 --2- 172.21.15.29:0/2305695628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c8490 0x7fabdc0a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:12.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.016+0000 7fabdbfff700 1 --2- 172.21.15.29:0/2305695628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc0bfd40 0x7fabdc0c01a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fabd4009f20 tx=0x7fabd4009840).ready entity=mon.0 client_cookie=15a436364935e71b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:12.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.016+0000 7fabdaffd700 1 -- 172.21.15.29:0/2305695628 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd400e070 con 0x7fabdc0bfd40 2024-01-24T14:14:12.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.016+0000 7fabdaffd700 1 -- 172.21.15.29:0/2305695628 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd400bb10 con 0x7fabdc0bfd40 2024-01-24T14:14:12.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.016+0000 7fabdaffd700 1 -- 172.21.15.29:0/2305695628 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd400be90 con 0x7fabdc0bfd40 2024-01-24T14:14:12.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.017+0000 7fabe21b2700 1 -- 172.21.15.29:0/2305695628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc0bfd40 msgr2=0x7fabdc0c01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:12.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.017+0000 7fabe21b2700 1 --2- 172.21.15.29:0/2305695628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc0bfd40 0x7fabdc0c01a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fabd4009f20 tx=0x7fabd4009840).stop 2024-01-24T14:14:12.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.017+0000 7fabe21b2700 1 -- 172.21.15.29:0/2305695628 shutdown_connections 2024-01-24T14:14:12.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.017+0000 7fabe21b2700 1 --2- 172.21.15.29:0/2305695628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc0bfd40 0x7fabdc0c01a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:12.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.017+0000 7fabe21b2700 1 --2- 172.21.15.29:0/2305695628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c8490 0x7fabdc0a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:12.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.017+0000 7fabe21b2700 1 -- 172.21.15.29:0/2305695628 >> 172.21.15.29:0/2305695628 conn(0x7fabdc0cb8b0 msgr2=0x7fabdc0c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:12.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.017+0000 7fabe21b2700 1 -- 172.21.15.29:0/2305695628 shutdown_connections 2024-01-24T14:14:12.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.017+0000 7fabe21b2700 1 -- 172.21.15.29:0/2305695628 wait complete. 2024-01-24T14:14:12.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.018+0000 7fabe21b2700 1 Processor -- start 2024-01-24T14:14:12.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.018+0000 7fabe21b2700 1 -- start start 2024-01-24T14:14:12.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.019+0000 7fabe21b2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc064760 0x7fabdc0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:12.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.019+0000 7fabe21b2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc0bfd40 0x7fabdc164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:12.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.019+0000 7fabe21b2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c8490 0x7fabdc1656d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:12.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.019+0000 7fabe21b2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0b0720 con 0x7fabdc064760 2024-01-24T14:14:12.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.019+0000 7fabe21b2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc1654a0 con 0x7fabdc0bfd40 2024-01-24T14:14:12.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.019+0000 7fabe21b2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc1523d0 con 0x7fabdc0c8490 2024-01-24T14:14:12.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.019+0000 7fabdb7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc064760 0x7fabdc0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:12.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.019+0000 7fabdb7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc064760 0x7fabdc0afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60146/0 (socket says 172.21.15.29:60146) 2024-01-24T14:14:12.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.019+0000 7fabdb7fe700 1 -- 172.21.15.29:0/1313838043 learned_addr learned my addr 172.21.15.29:0/1313838043 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:12.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.019+0000 7fabd2ffd700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc0bfd40 0x7fabdc164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:12.022 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:12.019+0000 7f116184c680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:14:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.019+0000 7fabdbfff700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c8490 0x7fabdc1656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabdbfff700 1 -- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc0bfd40 msgr2=0x7fabdc164fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabdbfff700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc0bfd40 0x7fabdc164fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabdbfff700 1 -- 172.21.15.29:0/1313838043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc064760 msgr2=0x7fabdc0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabdbfff700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc064760 0x7fabdc0afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabdbfff700 1 -- 172.21.15.29:0/1313838043 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd4009ae0 con 0x7fabdc0c8490 2024-01-24T14:14:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabd2ffd700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc0bfd40 0x7fabdc164fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabdb7fe700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc064760 0x7fabdc0afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabdbfff700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c8490 0x7fabdc1656d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fabd4013000 tx=0x7fabd4009840).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabd3fff700 1 -- 172.21.15.29:0/1313838043 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd40032f0 con 0x7fabdc0c8490 2024-01-24T14:14:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabd3fff700 1 -- 172.21.15.29:0/1313838043 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd40092e0 con 0x7fabdc0c8490 2024-01-24T14:14:12.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabe21b2700 1 -- 172.21.15.29:0/1313838043 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc152c70 con 0x7fabdc0c8490 2024-01-24T14:14:12.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabd3fff700 1 -- 172.21.15.29:0/1313838043 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd401e6a0 con 0x7fabdc0c8490 2024-01-24T14:14:12.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.020+0000 7fabe21b2700 1 -- 172.21.15.29:0/1313838043 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc0b0440 con 0x7fabdc0c8490 2024-01-24T14:14:12.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.021+0000 7fabe21b2700 1 -- 172.21.15.29:0/1313838043 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc152e00 con 0x7fabdc0c8490 2024-01-24T14:14:12.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.022+0000 7fabd3fff700 1 -- 172.21.15.29:0/1313838043 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 32) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fabd4003040 con 0x7fabdc0c8490 2024-01-24T14:14:12.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.022+0000 7fabd3fff700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabbc05f2c0 0x7fabbc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:12.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.022+0000 7fabdb7fe700 1 -- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabbc05f2c0 msgr2=0x7fabbc061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:12.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.022+0000 7fabd3fff700 1 -- 172.21.15.29:0/1313838043 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fabd407bd40 con 0x7fabdc0c8490 2024-01-24T14:14:12.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.022+0000 7fabdb7fe700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabbc05f2c0 0x7fabbc061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:12.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.026+0000 7fabd3fff700 1 -- 172.21.15.29:0/1313838043 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fabd404cac0 con 0x7fabdc0c8490 2024-01-24T14:14:12.072 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:12.072+0000 7fecbec05680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:14:12.093 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:12.093+0000 7f116184c680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:14:12.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.209+0000 7fabe21b2700 1 -- 172.21.15.29:0/1313838043 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fabdc0475e0 con 0x7fabdc0c8490 2024-01-24T14:14:12.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.215+0000 7fabd3fff700 1 -- 172.21.15.29:0/1313838043 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v15)=0 v15) v1 ==== 138+0+0 (secure 0 0 0) 0x7fabd4023030 con 0x7fabdc0c8490 2024-01-24T14:14:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.217+0000 7fabe21b2700 1 -- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabbc05f2c0 msgr2=0x7fabbc061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.217+0000 7fabe21b2700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabbc05f2c0 0x7fabbc061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.217+0000 7fabe21b2700 1 -- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c8490 msgr2=0x7fabdc1656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.217+0000 7fabe21b2700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c8490 0x7fabdc1656d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fabd4013000 tx=0x7fabd4009840).stop 2024-01-24T14:14:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.217+0000 7fabe21b2700 1 -- 172.21.15.29:0/1313838043 shutdown_connections 2024-01-24T14:14:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.217+0000 7fabe21b2700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c8490 0x7fabdc1656d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.217+0000 7fabe21b2700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabbc05f2c0 0x7fabbc061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:12.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.217+0000 7fabe21b2700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc064760 0x7fabdc0afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:12.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.217+0000 7fabe21b2700 1 --2- 172.21.15.29:0/1313838043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc0bfd40 0x7fabdc164fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:12.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.217+0000 7fabe21b2700 1 -- 172.21.15.29:0/1313838043 >> 172.21.15.29:0/1313838043 conn(0x7fabdc0cb8b0 msgr2=0x7fabdc157130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:12.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.217+0000 7fabe21b2700 1 -- 172.21.15.29:0/1313838043 shutdown_connections 2024-01-24T14:14:12.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.217+0000 7fabe21b2700 1 -- 172.21.15.29:0/1313838043 wait complete. 2024-01-24T14:14:12.232 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2024-01-24T14:14:12.345 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:12.345+0000 7fecbec05680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:14:12.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.366+0000 7f03465f7700 1 Processor -- start 2024-01-24T14:14:12.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.367+0000 7f03465f7700 1 -- start start 2024-01-24T14:14:12.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.367+0000 7f03465f7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0340151e50 0x7f03400c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:12.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.367+0000 7f03465f7700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0340154f90 con 0x7f0340064e90 2024-01-24T14:14:12.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.367+0000 7f03465f7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03400a9f10 con 0x7f03400c7f70 2024-01-24T14:14:12.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.367+0000 7f03465f7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03400aa1f0 con 0x7f0340151e50 2024-01-24T14:14:12.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.368+0000 7f0344b94700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0340064e90 0x7f03400bfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39620/0 (socket says 172.21.15.29:39620) 2024-01-24T14:14:12.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.368+0000 7f0344b94700 1 -- 172.21.15.29:0/536003181 learned_addr learned my addr 172.21.15.29:0/536003181 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:12.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.368+0000 7f033ffff700 1 --2- 172.21.15.29:0/536003181 >> v2:172.21.15.163:3300/0 conn(0x7f0340151e50 0x7f03400c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:12.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.368+0000 7f033effd700 1 -- 172.21.15.29:0/536003181 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238236226 0 0) 0x7f0340154f90 con 0x7f0340064e90 2024-01-24T14:14:12.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.368+0000 7f033effd700 1 -- 172.21.15.29:0/536003181 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03280033f0 con 0x7f0340064e90 2024-01-24T14:14:12.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.368+0000 7f033ffff700 1 -- 172.21.15.29:0/536003181 >> v1:172.21.15.163:6789/0 conn(0x7f03400c7f70 legacy=0x7f0340064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:12.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.368+0000 7f033ffff700 1 -- 172.21.15.29:0/536003181 >> v1:172.21.15.29:6789/0 conn(0x7f0340064e90 legacy=0x7f03400bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:12.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.368+0000 7f033ffff700 1 -- 172.21.15.29:0/536003181 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03400aa4d0 con 0x7f0340151e50 2024-01-24T14:14:12.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.368+0000 7f033ffff700 1 --2- 172.21.15.29:0/536003181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0340151e50 0x7f03400c53c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f0330009760 tx=0x7f0330009060).ready entity=mon.1 client_cookie=b1d60a386f1ca170 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:12.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.368+0000 7f033effd700 1 -- 172.21.15.29:0/536003181 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033000f510 con 0x7f0340151e50 2024-01-24T14:14:12.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.368+0000 7f033effd700 1 -- 172.21.15.29:0/536003181 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033000f670 con 0x7f0340151e50 2024-01-24T14:14:12.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.368+0000 7f033effd700 1 -- 172.21.15.29:0/536003181 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033000f9f0 con 0x7f0340151e50 2024-01-24T14:14:12.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.369+0000 7f03465f7700 1 -- 172.21.15.29:0/536003181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0340151e50 msgr2=0x7f03400c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:12.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.369+0000 7f03465f7700 1 --2- 172.21.15.29:0/536003181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0340151e50 0x7f03400c53c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f0330009760 tx=0x7f0330009060).stop 2024-01-24T14:14:12.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.369+0000 7f03465f7700 1 -- 172.21.15.29:0/536003181 shutdown_connections 2024-01-24T14:14:12.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.369+0000 7f03465f7700 1 --2- 172.21.15.29:0/536003181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0340151e50 0x7f03400c53c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:12.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.369+0000 7f03465f7700 1 -- 172.21.15.29:0/536003181 >> 172.21.15.29:0/536003181 conn(0x7f03400cb880 msgr2=0x7f03400c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:12.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.369+0000 7f03465f7700 1 -- 172.21.15.29:0/536003181 shutdown_connections 2024-01-24T14:14:12.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.369+0000 7f03465f7700 1 -- 172.21.15.29:0/536003181 wait complete. 2024-01-24T14:14:12.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.370+0000 7f03465f7700 1 Processor -- start 2024-01-24T14:14:12.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.371+0000 7f03465f7700 1 -- start start 2024-01-24T14:14:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.371+0000 7f03465f7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0340064e90 0x7f034014c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.371+0000 7f03465f7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03400c7f70 0x7f03400c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.371+0000 7f03465f7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0340151e50 0x7f03400c2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.371+0000 7f03465f7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03400c2840 con 0x7f0340064e90 2024-01-24T14:14:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.371+0000 7f03465f7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03400b05d0 con 0x7f03400c7f70 2024-01-24T14:14:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.371+0000 7f03465f7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03400af9c0 con 0x7f0340151e50 2024-01-24T14:14:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.371+0000 7f033ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0340064e90 0x7f034014c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f033ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0340064e90 0x7f034014c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60160/0 (socket says 172.21.15.29:60160) 2024-01-24T14:14:12.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f033ffff700 1 -- 172.21.15.29:0/1644665532 learned_addr learned my addr 172.21.15.29:0/1644665532 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f0344b94700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0340151e50 0x7f03400c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f033f7fe700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03400c7f70 0x7f03400c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f033ffff700 1 -- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0340151e50 msgr2=0x7f03400c2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f033ffff700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0340151e50 0x7f03400c2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f033ffff700 1 -- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03400c7f70 msgr2=0x7f03400c1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f033ffff700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03400c7f70 0x7f03400c1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f033ffff700 1 -- 172.21.15.29:0/1644665532 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03300092a0 con 0x7f0340064e90 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f033ffff700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0340064e90 0x7f034014c160 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f0330009e00 tx=0x7f0330017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f0327fff700 1 -- 172.21.15.29:0/1644665532 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033000f7e0 con 0x7f0340064e90 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f0327fff700 1 -- 172.21.15.29:0/1644665532 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033000f940 con 0x7f0340064e90 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f03465f7700 1 -- 172.21.15.29:0/1644665532 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03400afca0 con 0x7f0340064e90 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f0327fff700 1 -- 172.21.15.29:0/1644665532 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033001b690 con 0x7f0340064e90 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.372+0000 7f03465f7700 1 -- 172.21.15.29:0/1644665532 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03400b0260 con 0x7f0340064e90 2024-01-24T14:14:12.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.374+0000 7f03465f7700 1 -- 172.21.15.29:0/1644665532 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03400aa710 con 0x7f0340064e90 2024-01-24T14:14:12.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.374+0000 7f0327fff700 1 -- 172.21.15.29:0/1644665532 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 32) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f033001ba00 con 0x7f0340064e90 2024-01-24T14:14:12.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.374+0000 7f0327fff700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 0x7f032c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:12.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.374+0000 7f0327fff700 1 -- 172.21.15.29:0/1644665532 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f033007be10 con 0x7f0340064e90 2024-01-24T14:14:12.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.375+0000 7f033f7fe700 1 -- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 msgr2=0x7f032c061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:12.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.375+0000 7f033f7fe700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 0x7f032c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:12.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.379+0000 7f0327fff700 1 -- 172.21.15.29:0/1644665532 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f033004cb60 con 0x7f0340064e90 2024-01-24T14:14:12.412 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:12.412+0000 7f116184c680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:14:12.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.576+0000 7f033f7fe700 1 -- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 msgr2=0x7f032c061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:12.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.576+0000 7f033f7fe700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 0x7f032c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:14:12.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.590+0000 7f03465f7700 1 -- 172.21.15.29:0/1644665532 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0340083310 con 0x7f032c05f2c0 2024-01-24T14:14:12.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:12.958+0000 7fecbec05680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:14:12.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.977+0000 7f033f7fe700 1 -- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 msgr2=0x7f032c061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:12.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:12.977+0000 7f033f7fe700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 0x7f032c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-01-24T14:14:13.030 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:13.030+0000 7fecbec05680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:14:13.133 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:13.133+0000 7f116184c680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:14:13.208 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:13.207+0000 7f116184c680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:14:13.215 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:13.215+0000 7fecbec05680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:14:13.422 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:13.421+0000 7f116184c680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:14:13.635 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:13.635+0000 7fecbec05680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:14:13.738 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:13.738+0000 7fecbec05680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:14:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:13.778+0000 7f033f7fe700 1 -- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 msgr2=0x7f032c061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:13.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:13.778+0000 7f033f7fe700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 0x7f032c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-01-24T14:14:13.891 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:13.891+0000 7fecbec05680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:14:13.901 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:13.901+0000 7f116184c680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:14:13.966 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:13.966+0000 7fecbec05680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:14:14.019 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:14.018+0000 7f116184c680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:14:14.033 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:14.033+0000 7fecbec05680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:14:14.197 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:14.194+0000 7f116184c680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:14:14.280 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:14.279+0000 7f116184c680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:14:14.288 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:14.288+0000 7fecbec05680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:14:14.358 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:14.357+0000 7f116184c680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:14:14.469 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:14.469+0000 7fecbec05680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:14:14.651 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:14.651+0000 7f116184c680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:14:14.870 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:14.870+0000 7f116184c680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:14:15.115 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:15.115+0000 7fecbec05680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:14:15.185 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:15.185+0000 7fecbec05680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:14:15.260 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:15.260+0000 7fecbec05680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:14:15.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:15.380+0000 7f033f7fe700 1 -- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 msgr2=0x7f032c061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:15.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:15.380+0000 7f033f7fe700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 0x7f032c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-01-24T14:14:15.406 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:15.406+0000 7fecbec05680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:14:15.476 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:15.476+0000 7fecbec05680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:14:15.591 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:15.591+0000 7fecbec05680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:14:15.626 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:15.625+0000 7f116184c680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:14:15.702 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:15.702+0000 7fecbec05680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:14:15.704 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:15.704+0000 7f116184c680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:14:15.790 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:15.789+0000 7f116184c680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:14:15.958 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:15.957+0000 7f116184c680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:14:15.993 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:15.993+0000 7fecbec05680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:14:16.039 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:16.039+0000 7f116184c680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:14:16.064 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:16.064+0000 7fecbec05680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:14:16.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:16.083+0000 7f0327fff700 1 -- 172.21.15.29:0/1644665532 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 33) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f0330079660 con 0x7f0340064e90 2024-01-24T14:14:16.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:16.083+0000 7f0327fff700 1 -- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 msgr2=0x7f032c061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:16.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:16.083+0000 7f0327fff700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 0x7f032c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:16.140 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:16.140+0000 7fec35981700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:16.141 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:16.140+0000 7fec35981700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:16.141 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:16.140+0000 7fec35981700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:16.141 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:16.140+0000 7fec35981700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:16.141 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:16.140+0000 7fec35981700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:16.141 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:16.141+0000 7fec3297b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:16.141 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:16.141+0000 7fec3297b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:16.141 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:16.141+0000 7fec3297b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:16.141 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:16.141+0000 7fec3297b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:16.141 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:16.141+0000 7fec3297b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:16.179 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:16.179+0000 7f116184c680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:14:16.301 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:16.301+0000 7f116184c680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:14:16.635 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:16.635+0000 7f116184c680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:14:16.716 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:16.716+0000 7f116184c680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:14:17.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.087+0000 7f0327fff700 1 -- 172.21.15.29:0/1644665532 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mgrmap(e 34) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f0330006a40 con 0x7f0340064e90 2024-01-24T14:14:17.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.087+0000 7f0327fff700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 0x7f032c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:17.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.087+0000 7f0327fff700 1 -- 172.21.15.29:0/1644665532 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0340083310 con 0x7f032c05f2c0 2024-01-24T14:14:17.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.088+0000 7f033f7fe700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 0x7f032c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:17.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.089+0000 7f033f7fe700 1 --2- 172.21.15.29:0/1644665532 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f032c05f2c0 0x7f032c061700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f0340150d60 tx=0x7f0334007d40).ready entity=mgr.6721 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:17.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.091+0000 7f0327fff700 1 -- 172.21.15.29:0/1644665532 <== mgr.6721 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f0340083310 con 0x7f032c05f2c0 2024-01-24T14:14:17.092 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:17.092+0000 7fec3d991700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2024-01-24T14:14:17.094 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2024-01-24T14:14:17.098 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-01-24T14:14:17.099 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-01-24T14:14:17.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.235+0000 7f34a3e7d700 1 Processor -- start 2024-01-24T14:14:17.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.235+0000 7f34a3e7d700 1 -- start start 2024-01-24T14:14:17.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.236+0000 7f34a3e7d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f349c141c00 0x7f349c065420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:17.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.236+0000 7f34a3e7d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f349c062ec0 0x7f349c069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:17.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.236+0000 7f34a3e7d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349c15cf20 con 0x7f349c141fe0 2024-01-24T14:14:17.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.236+0000 7f34a3e7d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349c15ca60 con 0x7f349c062ec0 2024-01-24T14:14:17.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.236+0000 7f34a3e7d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0c8440 con 0x7f349c141c00 2024-01-24T14:14:17.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.237+0000 7f34a1418700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f349c141fe0 0x7f349c0627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35304/0 (socket says 172.21.15.29:35304) 2024-01-24T14:14:17.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.237+0000 7f34a1418700 1 -- 172.21.15.29:0/688633810 learned_addr learned my addr 172.21.15.29:0/688633810 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:17.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.237+0000 7f34a1c19700 1 --2- 172.21.15.29:0/688633810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f349c141c00 0x7f349c065420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:17.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.237+0000 7f34a241a700 1 --2- 172.21.15.29:0/688633810 >> v2:172.21.15.163:3300/0 conn(0x7f349c062ec0 0x7f349c069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:17.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.237+0000 7f34a0c17700 1 -- 172.21.15.29:0/688633810 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768865367 0 0) 0x7f349c15cf20 con 0x7f349c141fe0 2024-01-24T14:14:17.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.237+0000 7f34a0c17700 1 -- 172.21.15.29:0/688633810 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34880033f0 con 0x7f349c141fe0 2024-01-24T14:14:17.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.237+0000 7f34a241a700 1 -- 172.21.15.29:0/688633810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f349c141c00 msgr2=0x7f349c065420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:17.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.237+0000 7f34a241a700 1 --2- 172.21.15.29:0/688633810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f349c141c00 0x7f349c065420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:17.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.237+0000 7f34a241a700 1 -- 172.21.15.29:0/688633810 >> v1:172.21.15.29:6789/0 conn(0x7f349c141fe0 legacy=0x7f349c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:17.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.237+0000 7f34a241a700 1 -- 172.21.15.29:0/688633810 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c15cc40 con 0x7f349c062ec0 2024-01-24T14:14:17.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.237+0000 7f34a241a700 1 --2- 172.21.15.29:0/688633810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f349c062ec0 0x7f349c069520 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f3494009e60 tx=0x7f3494009780).ready entity=mon.1 client_cookie=2b9685cc9ffb029f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:17.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.237+0000 7f34a0c17700 1 -- 172.21.15.29:0/688633810 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349400e070 con 0x7f349c062ec0 2024-01-24T14:14:17.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.238+0000 7f34a0c17700 1 -- 172.21.15.29:0/688633810 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349400bab0 con 0x7f349c062ec0 2024-01-24T14:14:17.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.238+0000 7f34a0c17700 1 -- 172.21.15.29:0/688633810 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349400be30 con 0x7f349c062ec0 2024-01-24T14:14:17.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.238+0000 7f34a3e7d700 1 -- 172.21.15.29:0/688633810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f349c062ec0 msgr2=0x7f349c069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:17.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.238+0000 7f34a3e7d700 1 --2- 172.21.15.29:0/688633810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f349c062ec0 0x7f349c069520 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f3494009e60 tx=0x7f3494009780).stop 2024-01-24T14:14:17.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.238+0000 7f34a3e7d700 1 -- 172.21.15.29:0/688633810 shutdown_connections 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.238+0000 7f34a3e7d700 1 --2- 172.21.15.29:0/688633810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f349c062ec0 0x7f349c069520 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.238+0000 7f34a3e7d700 1 --2- 172.21.15.29:0/688633810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f349c141c00 0x7f349c065420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.238+0000 7f34a3e7d700 1 -- 172.21.15.29:0/688633810 >> 172.21.15.29:0/688633810 conn(0x7f349c064390 msgr2=0x7f349c0c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.238+0000 7f34a3e7d700 1 -- 172.21.15.29:0/688633810 shutdown_connections 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.238+0000 7f34a3e7d700 1 -- 172.21.15.29:0/688633810 wait complete. 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.239+0000 7f34a3e7d700 1 Processor -- start 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.239+0000 7f34a3e7d700 1 -- start start 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.240+0000 7f34a3e7d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f349c062ec0 0x7f349c0beb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.240+0000 7f34a3e7d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f349c141c00 0x7f349c168ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.240+0000 7f34a3e7d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f349c141fe0 0x7f349c0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.240+0000 7f34a3e7d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0a7840 con 0x7f349c062ec0 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.240+0000 7f34a3e7d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0aab30 con 0x7f349c141fe0 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.240+0000 7f34a3e7d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c168fb0 con 0x7f349c141c00 2024-01-24T14:14:17.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.240+0000 7f34a1c19700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f349c062ec0 0x7f349c0beb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.240+0000 7f34a1c19700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f349c062ec0 0x7f349c0beb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52798/0 (socket says 172.21.15.29:52798) 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.240+0000 7f34a1c19700 1 -- 172.21.15.29:0/2938257256 learned_addr learned my addr 172.21.15.29:0/2938257256 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.240+0000 7f34a1418700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f349c141c00 0x7f349c168ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.240+0000 7f34a241a700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f349c141fe0 0x7f349c0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.241+0000 7f34a1c19700 1 -- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f349c141c00 msgr2=0x7f349c168ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.241+0000 7f34a1c19700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f349c141c00 0x7f349c168ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.241+0000 7f34a1c19700 1 -- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f349c141fe0 msgr2=0x7f349c0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.241+0000 7f34a1c19700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f349c141fe0 0x7f349c0a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.241+0000 7f34a1c19700 1 -- 172.21.15.29:0/2938257256 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494009a20 con 0x7f349c062ec0 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.241+0000 7f34a1c19700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f349c062ec0 0x7f349c0beb70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f349800e930 tx=0x7f3498009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.241+0000 7f348dffb700 1 -- 172.21.15.29:0/2938257256 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349800c680 con 0x7f349c062ec0 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.241+0000 7f348dffb700 1 -- 172.21.15.29:0/2938257256 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349800c7e0 con 0x7f349c062ec0 2024-01-24T14:14:17.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.241+0000 7f34a3e7d700 1 -- 172.21.15.29:0/2938257256 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349c0bf150 con 0x7f349c062ec0 2024-01-24T14:14:17.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.241+0000 7f348dffb700 1 -- 172.21.15.29:0/2938257256 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349800cb60 con 0x7f349c062ec0 2024-01-24T14:14:17.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.241+0000 7f34a3e7d700 1 -- 172.21.15.29:0/2938257256 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349c0bf710 con 0x7f349c062ec0 2024-01-24T14:14:17.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.243+0000 7f34a3e7d700 1 -- 172.21.15.29:0/2938257256 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349c0aae10 con 0x7f349c062ec0 2024-01-24T14:14:17.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.243+0000 7f348dffb700 1 -- 172.21.15.29:0/2938257256 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 34) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f349801b400 con 0x7f349c062ec0 2024-01-24T14:14:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.243+0000 7f348dffb700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f348005f080 0x7f3480061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.243+0000 7f348dffb700 1 -- 172.21.15.29:0/2938257256 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f349807aa10 con 0x7f349c062ec0 2024-01-24T14:14:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.243+0000 7f34a1418700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f348005f080 0x7f3480061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:17.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.244+0000 7f34a1418700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f348005f080 0x7f3480061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f3490005750 tx=0x7f3490009040).ready entity=mgr.6721 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:17.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.248+0000 7f348dffb700 1 -- 172.21.15.29:0/2938257256 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f349804ba00 con 0x7f349c062ec0 2024-01-24T14:14:17.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:17.480+0000 7f34a3e7d700 1 -- 172.21.15.29:0/2938257256 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f349c0475e0 con 0x7f349c062ec0 2024-01-24T14:14:18.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.093+0000 7f348dffb700 1 -- 172.21.15.29:0/2938257256 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 35) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f34980784b0 con 0x7f349c062ec0 2024-01-24T14:14:18.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.094+0000 7f348dffb700 1 -- 172.21.15.29:0/2938257256 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v35) v1 ==== 93+0+0 (secure 0 0 0) 0x7f3498071e60 con 0x7f349c062ec0 2024-01-24T14:14:18.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.097+0000 7f34a3e7d700 1 -- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f348005f080 msgr2=0x7f3480061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:18.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.097+0000 7f34a3e7d700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f348005f080 0x7f3480061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f3490005750 tx=0x7f3490009040).stop 2024-01-24T14:14:18.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.097+0000 7f34a3e7d700 1 -- 172.21.15.29:0/2938257256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f349c062ec0 msgr2=0x7f349c0beb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:18.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.097+0000 7f34a3e7d700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f349c062ec0 0x7f349c0beb70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f349800e930 tx=0x7f3498009060).stop 2024-01-24T14:14:18.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.097+0000 7f34a3e7d700 1 -- 172.21.15.29:0/2938257256 shutdown_connections 2024-01-24T14:14:18.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.097+0000 7f34a3e7d700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f349c141fe0 0x7f349c0a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.097+0000 7f34a3e7d700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f348005f080 0x7f3480061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.097+0000 7f34a3e7d700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f349c062ec0 0x7f349c0beb70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.097+0000 7f34a3e7d700 1 --2- 172.21.15.29:0/2938257256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f349c141c00 0x7f349c168ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.097+0000 7f34a3e7d700 1 -- 172.21.15.29:0/2938257256 >> 172.21.15.29:0/2938257256 conn(0x7f349c064390 msgr2=0x7f349c0c7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:18.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.098+0000 7f34a3e7d700 1 -- 172.21.15.29:0/2938257256 shutdown_connections 2024-01-24T14:14:18.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.098+0000 7f34a3e7d700 1 -- 172.21.15.29:0/2938257256 wait complete. 2024-01-24T14:14:18.114 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T14:14:18.114 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:14:18.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:14:18.202 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T14:14:18.202 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-24T14:14:18.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:14:18.202 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-24T14:14:18.244 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:18.244+0000 7f5349dda680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:14:18.249 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T14:14:18.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:18.249 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-24T14:14:18.256 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:18.255+0000 7f0678022680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:14:18.267 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:14:18.307 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:18.307+0000 7f5349dda680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:14:18.329 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:18.329+0000 7f0678022680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:14:18.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.439+0000 7fca6d81b700 1 Processor -- start 2024-01-24T14:14:18.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.439+0000 7fca6d81b700 1 -- start start 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca6d81b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fca680c5410 0x7fca680aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca6d81b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fca68154750 0x7fca680631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca6d81b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca680aa6c0 con 0x7fca68154370 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca6d81b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca68154ba0 con 0x7fca680c5410 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca6d81b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca68064240 con 0x7fca68154750 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca667fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fca680c5410 0x7fca680aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca66ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fca68154370 0x7fca680ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35314/0 (socket says 172.21.15.29:35314) 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca66ffd700 1 -- 172.21.15.29:0/2591907902 learned_addr learned my addr 172.21.15.29:0/2591907902 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca677fe700 1 --2- 172.21.15.29:0/2591907902 >> v2:172.21.15.163:3300/0 conn(0x7fca68154750 0x7fca680631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca667fc700 1 -- 172.21.15.29:0/2591907902 >> v2:172.21.15.163:3300/0 conn(0x7fca68154750 msgr2=0x7fca680631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca667fc700 1 --2- 172.21.15.29:0/2591907902 >> v2:172.21.15.163:3300/0 conn(0x7fca68154750 0x7fca680631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca667fc700 1 -- 172.21.15.29:0/2591907902 >> v1:172.21.15.29:6789/0 conn(0x7fca68154370 legacy=0x7fca680ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca667fc700 1 -- 172.21.15.29:0/2591907902 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca680cb5c0 con 0x7fca680c5410 2024-01-24T14:14:18.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca65ffb700 1 -- 172.21.15.29:0/2591907902 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458491801 0 0) 0x7fca680aa6c0 con 0x7fca68154370 2024-01-24T14:14:18.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.440+0000 7fca667fc700 1 --2- 172.21.15.29:0/2591907902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca680c5410 0x7fca680aa0e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7fca6805f190 tx=0x7fca5c009060).ready entity=mon.0 client_cookie=ae0cd0dc5e97f4f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:18.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.441+0000 7fca65ffb700 1 -- 172.21.15.29:0/2591907902 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00f510 con 0x7fca680c5410 2024-01-24T14:14:18.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.441+0000 7fca65ffb700 1 -- 172.21.15.29:0/2591907902 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5c00f670 con 0x7fca680c5410 2024-01-24T14:14:18.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.441+0000 7fca65ffb700 1 -- 172.21.15.29:0/2591907902 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00f9f0 con 0x7fca680c5410 2024-01-24T14:14:18.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.441+0000 7fca6d81b700 1 -- 172.21.15.29:0/2591907902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca680c5410 msgr2=0x7fca680aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:18.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.441+0000 7fca6d81b700 1 --2- 172.21.15.29:0/2591907902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca680c5410 0x7fca680aa0e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7fca6805f190 tx=0x7fca5c009060).stop 2024-01-24T14:14:18.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.441+0000 7fca6d81b700 1 -- 172.21.15.29:0/2591907902 shutdown_connections 2024-01-24T14:14:18.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.441+0000 7fca6d81b700 1 --2- 172.21.15.29:0/2591907902 >> v2:172.21.15.163:3300/0 conn(0x7fca68154750 0x7fca680631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.441+0000 7fca6d81b700 1 --2- 172.21.15.29:0/2591907902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca680c5410 0x7fca680aa0e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.441+0000 7fca6d81b700 1 -- 172.21.15.29:0/2591907902 >> 172.21.15.29:0/2591907902 conn(0x7fca6805ec80 msgr2=0x7fca680cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:18.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.441+0000 7fca6d81b700 1 -- 172.21.15.29:0/2591907902 shutdown_connections 2024-01-24T14:14:18.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.441+0000 7fca6d81b700 1 -- 172.21.15.29:0/2591907902 wait complete. 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.443+0000 7fca6d81b700 1 Processor -- start 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.443+0000 7fca6d81b700 1 -- start start 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.443+0000 7fca6d81b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca680c5410 0x7fca68144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.443+0000 7fca6d81b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca68154370 0x7fca680aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.443+0000 7fca6d81b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca68154750 0x7fca680af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.443+0000 7fca6d81b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca680c2790 con 0x7fca680c5410 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca6d81b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca680c2a30 con 0x7fca68154750 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca6d81b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca68150ca0 con 0x7fca68154370 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca66ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca680c5410 0x7fca68144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca66ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca680c5410 0x7fca68144cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52838/0 (socket says 172.21.15.29:52838) 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca66ffd700 1 -- 172.21.15.29:0/2098112403 learned_addr learned my addr 172.21.15.29:0/2098112403 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca667fc700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca68154370 0x7fca680aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:18.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca677fe700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca68154750 0x7fca680af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:18.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca66ffd700 1 -- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca68154370 msgr2=0x7fca680aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:18.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca66ffd700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca68154370 0x7fca680aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca66ffd700 1 -- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca68154750 msgr2=0x7fca680af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:18.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca66ffd700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca68154750 0x7fca680af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca66ffd700 1 -- 172.21.15.29:0/2098112403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c0092a0 con 0x7fca680c5410 2024-01-24T14:14:18.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca677fe700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca68154750 0x7fca680af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:18.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca66ffd700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca680c5410 0x7fca68144cc0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7fca58001360 tx=0x7fca58007b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:18.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca4effd700 1 -- 172.21.15.29:0/2098112403 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5800ce60 con 0x7fca680c5410 2024-01-24T14:14:18.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.444+0000 7fca4effd700 1 -- 172.21.15.29:0/2098112403 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca580072e0 con 0x7fca680c5410 2024-01-24T14:14:18.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.445+0000 7fca6d81b700 1 -- 172.21.15.29:0/2098112403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca681451b0 con 0x7fca680c5410 2024-01-24T14:14:18.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.445+0000 7fca4effd700 1 -- 172.21.15.29:0/2098112403 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca58005670 con 0x7fca680c5410 2024-01-24T14:14:18.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.445+0000 7fca6d81b700 1 -- 172.21.15.29:0/2098112403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca680afe10 con 0x7fca680c5410 2024-01-24T14:14:18.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.446+0000 7fca6d81b700 1 -- 172.21.15.29:0/2098112403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca680475e0 con 0x7fca680c5410 2024-01-24T14:14:18.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.446+0000 7fca4effd700 1 -- 172.21.15.29:0/2098112403 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 35) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fca58005a20 con 0x7fca680c5410 2024-01-24T14:14:18.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.447+0000 7fca4effd700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca5405f030 0x7fca540614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:18.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.447+0000 7fca4effd700 1 -- 172.21.15.29:0/2098112403 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fca58079c90 con 0x7fca680c5410 2024-01-24T14:14:18.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.447+0000 7fca667fc700 1 -- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca5405f030 msgr2=0x7fca540614f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:18.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.447+0000 7fca667fc700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca5405f030 0x7fca540614f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:18.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.451+0000 7fca4effd700 1 -- 172.21.15.29:0/2098112403 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fca5804ac30 con 0x7fca680c5410 2024-01-24T14:14:18.582 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:18.582+0000 7f5349dda680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:14:18.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.648+0000 7fca667fc700 1 -- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca5405f030 msgr2=0x7fca540614f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:18.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.648+0000 7fca667fc700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca5405f030 0x7fca540614f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:14:18.649 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:18.649+0000 7f0678022680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:14:18.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.679+0000 7fca6d81b700 1 -- 172.21.15.29:0/2098112403 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca680cb8a0 con 0x7fca680c5410 2024-01-24T14:14:18.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.680+0000 7fca4effd700 1 -- 172.21.15.29:0/2098112403 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fca58018070 con 0x7fca680c5410 2024-01-24T14:14:18.680 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:18.680 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4083072158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":4083072158},{"type":"v1","addr":"172.21.15.163:6837","nonce":4083072158}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/4114592089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4114592089},{"type":"v1","addr":"172.21.15.163:6835","nonce":4114592089}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/926851304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":926851304},{"type":"v1","addr":"172.21.15.29:6835","nonce":926851304}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:13:59.576939+0000","modified":"2024-01-24T14:14:02.594306+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6833/3854229544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3854229544},{"type":"v1","addr":"172.21.15.29:6833","nonce":3854229544}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-24T14:14:18.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.682+0000 7fca6d81b700 1 -- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca5405f030 msgr2=0x7fca540614f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:18.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.682+0000 7fca6d81b700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca5405f030 0x7fca540614f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:18.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.682+0000 7fca6d81b700 1 -- 172.21.15.29:0/2098112403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca680c5410 msgr2=0x7fca68144cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:18.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.682+0000 7fca6d81b700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca680c5410 0x7fca68144cc0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7fca58001360 tx=0x7fca58007b00).stop 2024-01-24T14:14:18.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.682+0000 7fca6d81b700 1 -- 172.21.15.29:0/2098112403 shutdown_connections 2024-01-24T14:14:18.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.682+0000 7fca6d81b700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca68154750 0x7fca680af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.682+0000 7fca6d81b700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca5405f030 0x7fca540614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:18.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.682+0000 7fca6d81b700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca680c5410 0x7fca68144cc0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.682+0000 7fca6d81b700 1 --2- 172.21.15.29:0/2098112403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca68154370 0x7fca680aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.682+0000 7fca6d81b700 1 -- 172.21.15.29:0/2098112403 >> 172.21.15.29:0/2098112403 conn(0x7fca6805ec80 msgr2=0x7fca680ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:18.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.682+0000 7fca6d81b700 1 -- 172.21.15.29:0/2098112403 shutdown_connections 2024-01-24T14:14:18.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.682+0000 7fca6d81b700 1 -- 172.21.15.29:0/2098112403 wait complete. 2024-01-24T14:14:18.684 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 43 2024-01-24T14:14:18.698 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:14:18.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.868+0000 7f3bfd7d6700 1 Processor -- start 2024-01-24T14:14:18.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.868+0000 7f3bfd7d6700 1 -- start start 2024-01-24T14:14:18.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.869+0000 7f3bfd7d6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8062ec0 0x7f3bf81527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:18.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.869+0000 7f3bfd7d6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3bf8152cd0 0x7f3bf815be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:18.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.869+0000 7f3bfd7d6700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf8064d90 con 0x7f3bf8134f10 2024-01-24T14:14:18.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.869+0000 7f3bfd7d6700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf8064250 con 0x7f3bf8152cd0 2024-01-24T14:14:18.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.869+0000 7f3bfd7d6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf8065220 con 0x7f3bf8062ec0 2024-01-24T14:14:18.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.869+0000 7f3bf6ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3bf8134f10 0x7f3bf80627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35318/0 (socket says 172.21.15.29:35318) 2024-01-24T14:14:18.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf6ffd700 1 -- 172.21.15.29:0/4220731819 learned_addr learned my addr 172.21.15.29:0/4220731819 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:18.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf67fc700 1 --2- 172.21.15.29:0/4220731819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8062ec0 0x7f3bf81527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:18.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf77fe700 1 --2- 172.21.15.29:0/4220731819 >> v2:172.21.15.29:3300/0 conn(0x7f3bf8152cd0 0x7f3bf815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:18.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf5ffb700 1 -- 172.21.15.29:0/4220731819 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193405310 0 0) 0x7f3bf8064d90 con 0x7f3bf8134f10 2024-01-24T14:14:18.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf5ffb700 1 -- 172.21.15.29:0/4220731819 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be00033f0 con 0x7f3bf8134f10 2024-01-24T14:14:18.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf67fc700 1 -- 172.21.15.29:0/4220731819 >> v2:172.21.15.29:3300/0 conn(0x7f3bf8152cd0 msgr2=0x7f3bf815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:18.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf67fc700 1 --2- 172.21.15.29:0/4220731819 >> v2:172.21.15.29:3300/0 conn(0x7f3bf8152cd0 0x7f3bf815be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf67fc700 1 -- 172.21.15.29:0/4220731819 >> v1:172.21.15.29:6789/0 conn(0x7f3bf8134f10 legacy=0x7f3bf80627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:18.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf67fc700 1 -- 172.21.15.29:0/4220731819 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf8064460 con 0x7f3bf8062ec0 2024-01-24T14:14:18.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf77fe700 1 --2- 172.21.15.29:0/4220731819 >> v2:172.21.15.29:3300/0 conn(0x7f3bf8152cd0 0x7f3bf815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:18.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf67fc700 1 --2- 172.21.15.29:0/4220731819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8062ec0 0x7f3bf81527e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f3bec009760 tx=0x7f3bec009060).ready entity=mon.2 client_cookie=252efbb74a641156 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:18.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf5ffb700 1 -- 172.21.15.29:0/4220731819 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0090d0 con 0x7f3bf8062ec0 2024-01-24T14:14:18.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf5ffb700 1 -- 172.21.15.29:0/4220731819 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bec00d4a0 con 0x7f3bf8062ec0 2024-01-24T14:14:18.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.870+0000 7f3bf5ffb700 1 -- 172.21.15.29:0/4220731819 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec00d820 con 0x7f3bf8062ec0 2024-01-24T14:14:18.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.871+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/4220731819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8062ec0 msgr2=0x7f3bf81527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:18.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.871+0000 7f3bfd7d6700 1 --2- 172.21.15.29:0/4220731819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8062ec0 0x7f3bf81527e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f3bec009760 tx=0x7f3bec009060).stop 2024-01-24T14:14:18.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.871+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/4220731819 shutdown_connections 2024-01-24T14:14:18.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.871+0000 7f3bfd7d6700 1 --2- 172.21.15.29:0/4220731819 >> v2:172.21.15.29:3300/0 conn(0x7f3bf8152cd0 0x7f3bf815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.871+0000 7f3bfd7d6700 1 --2- 172.21.15.29:0/4220731819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8062ec0 0x7f3bf81527e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.871+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/4220731819 >> 172.21.15.29:0/4220731819 conn(0x7f3bf80c69f0 msgr2=0x7f3bf80c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:18.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.871+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/4220731819 shutdown_connections 2024-01-24T14:14:18.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.871+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/4220731819 wait complete. 2024-01-24T14:14:18.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.873+0000 7f3bfd7d6700 1 Processor -- start 2024-01-24T14:14:18.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.874+0000 7f3bfd7d6700 1 -- start start 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bfd7d6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bf8062ec0 0x7f3bf813a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bfd7d6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bf8134f10 0x7f3bf8138a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bfd7d6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8152cd0 0x7f3bf80a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bfd7d6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf8131d20 con 0x7f3bf8134f10 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bfd7d6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf8138f40 con 0x7f3bf8062ec0 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bfd7d6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80ac8c0 con 0x7f3bf8152cd0 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bf67fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bf8134f10 0x7f3bf8138a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bf67fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bf8134f10 0x7f3bf8138a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52862/0 (socket says 172.21.15.29:52862) 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bf67fc700 1 -- 172.21.15.29:0/1562310897 learned_addr learned my addr 172.21.15.29:0/1562310897 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bf6ffd700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bf8062ec0 0x7f3bf813a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bf77fe700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8152cd0 0x7f3bf80a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bf67fc700 1 -- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8152cd0 msgr2=0x7f3bf80a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:18.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bf67fc700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8152cd0 0x7f3bf80a9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bf67fc700 1 -- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bf8062ec0 msgr2=0x7f3bf813a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:18.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bf67fc700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bf8062ec0 0x7f3bf813a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:18.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.875+0000 7f3bf67fc700 1 -- 172.21.15.29:0/1562310897 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf0009a20 con 0x7f3bf8134f10 2024-01-24T14:14:18.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.876+0000 7f3bf67fc700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bf8134f10 0x7f3bf8138a50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f3bf8064390 tx=0x7f3bec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:18.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.876+0000 7f3bdeffd700 1 -- 172.21.15.29:0/1562310897 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec006910 con 0x7f3bf8134f10 2024-01-24T14:14:18.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.876+0000 7f3bdeffd700 1 -- 172.21.15.29:0/1562310897 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bec012920 con 0x7f3bf8134f10 2024-01-24T14:14:18.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.876+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/1562310897 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec0092a0 con 0x7f3bf8134f10 2024-01-24T14:14:18.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.876+0000 7f3bdeffd700 1 -- 172.21.15.29:0/1562310897 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec012ce0 con 0x7f3bf8134f10 2024-01-24T14:14:18.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.876+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/1562310897 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf8132250 con 0x7f3bf8134f10 2024-01-24T14:14:18.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.877+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/1562310897 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf81328a0 con 0x7f3bf8134f10 2024-01-24T14:14:18.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.877+0000 7f3bdeffd700 1 -- 172.21.15.29:0/1562310897 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 35) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f3bec01c630 con 0x7f3bf8134f10 2024-01-24T14:14:18.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.878+0000 7f3bdeffd700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3be405f000 0x7f3be40614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:18.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.878+0000 7f3bf6ffd700 1 -- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3be405f000 msgr2=0x7f3be40614c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:18.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.878+0000 7f3bdeffd700 1 -- 172.21.15.29:0/1562310897 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f3bec07b840 con 0x7f3bf8134f10 2024-01-24T14:14:18.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.878+0000 7f3bf6ffd700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3be405f000 0x7f3be40614c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:18.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:18.886+0000 7f3bdeffd700 1 -- 172.21.15.29:0/1562310897 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3bec04c7e0 con 0x7f3bf8134f10 2024-01-24T14:14:19.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.079+0000 7f3bf6ffd700 1 -- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3be405f000 msgr2=0x7f3be40614c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:19.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.079+0000 7f3bf6ffd700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3be405f000 0x7f3be40614c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:14:19.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.097+0000 7f3bdeffd700 1 -- 172.21.15.29:0/1562310897 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 36) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f3bec0792e0 con 0x7f3bf8134f10 2024-01-24T14:14:19.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.100+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/1562310897 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bf80475e0 con 0x7f3bf8134f10 2024-01-24T14:14:19.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.101+0000 7f3bdeffd700 1 -- 172.21.15.29:0/1562310897 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+18557 (secure 0 0 0) 0x7f3bec0429f0 con 0x7f3bf8134f10 2024-01-24T14:14:19.103 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:19.103 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":106,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:14:16.078701+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:13:57.364021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-24T14:13:58.553943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3810716912":"2024-01-25T14:14:16.078667+0000","172.21.15.163:6833/43241":"2024-01-25T14:14:16.078667+0000","172.21.15.163:6832/43241":"2024-01-25T14:14:16.078667+0000","172.21.15.163:0/3154477280":"2024-01-25T14:14:16.078667+0000","172.21.15.163:0/1381714363":"2024-01-25T14:14:16.078667+0000","172.21.15.163:0/3261466368":"2024-01-25T14:14:16.078667+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:14:19.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.104+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3be405f000 msgr2=0x7f3be40614c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:19.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.104+0000 7f3bfd7d6700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3be405f000 0x7f3be40614c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:19.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.104+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/1562310897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bf8134f10 msgr2=0x7f3bf8138a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:19.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.104+0000 7f3bfd7d6700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bf8134f10 0x7f3bf8138a50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f3bf8064390 tx=0x7f3bec016000).stop 2024-01-24T14:14:19.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.104+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/1562310897 shutdown_connections 2024-01-24T14:14:19.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.104+0000 7f3bfd7d6700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8152cd0 0x7f3bf80a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.104+0000 7f3bfd7d6700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bf8062ec0 0x7f3bf813a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.104+0000 7f3bfd7d6700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3be405f000 0x7f3be40614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:19.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.104+0000 7f3bfd7d6700 1 --2- 172.21.15.29:0/1562310897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bf8134f10 0x7f3bf8138a50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.104+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/1562310897 >> 172.21.15.29:0/1562310897 conn(0x7f3bf80c69f0 msgr2=0x7f3bf8150b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:19.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.104+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/1562310897 shutdown_connections 2024-01-24T14:14:19.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.104+0000 7f3bfd7d6700 1 -- 172.21.15.29:0/1562310897 wait complete. 2024-01-24T14:14:19.119 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T14:13:57.364021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T14:13:58.553943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T14:14:19.119 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T14:14:19.200 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:19.200+0000 7f5349dda680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:14:19.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.255+0000 7fb1a9f1d700 1 Processor -- start 2024-01-24T14:14:19.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.255+0000 7fb1a9f1d700 1 -- start start 2024-01-24T14:14:19.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.257+0000 7fb1a9f1d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb1a4134ea0 0x7fb1a40c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.257+0000 7fb1a9f1d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1a40c36c0 0x7fb1a40664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.257+0000 7fb1a9f1d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a40686a0 con 0x7fb1a40669b0 2024-01-24T14:14:19.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.257+0000 7fb1a9f1d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a4066e30 con 0x7fb1a4134ea0 2024-01-24T14:14:19.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.257+0000 7fb1a9f1d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4068b30 con 0x7fb1a40c36c0 2024-01-24T14:14:19.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.257+0000 7fb19affd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1a40c36c0 0x7fb1a40664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:19.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.257+0000 7fb1a37fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb1a4134ea0 0x7fb1a40c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:19.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.257+0000 7fb1a3fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb1a40669b0 0x7fb1a41559f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:48884/0 (socket says 172.21.15.29:48884) 2024-01-24T14:14:19.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.257+0000 7fb1a3fff700 1 -- 172.21.15.29:0/1968579852 learned_addr learned my addr 172.21.15.29:0/1968579852 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:19.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.257+0000 7fb19affd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1a40c36c0 0x7fb1a40664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:54564/0 (socket says 172.21.15.29:54564) 2024-01-24T14:14:19.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.258+0000 7fb1a37fe700 1 -- 172.21.15.29:0/1968579852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1a40c36c0 msgr2=0x7fb1a40664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:19.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.258+0000 7fb1a37fe700 1 --2- 172.21.15.29:0/1968579852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1a40c36c0 0x7fb1a40664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.258+0000 7fb1a37fe700 1 -- 172.21.15.29:0/1968579852 >> v1:172.21.15.163:6789/0 conn(0x7fb1a40669b0 legacy=0x7fb1a41559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:19.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.258+0000 7fb1a2ffd700 1 -- 172.21.15.29:0/1968579852 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796787045 0 0) 0x7fb1a40686a0 con 0x7fb1a40669b0 2024-01-24T14:14:19.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.258+0000 7fb1a37fe700 1 -- 172.21.15.29:0/1968579852 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4067a90 con 0x7fb1a4134ea0 2024-01-24T14:14:19.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.258+0000 7fb19affd700 1 --2- 172.21.15.29:0/1968579852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1a40c36c0 0x7fb1a40664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:19.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.258+0000 7fb1a37fe700 1 --2- 172.21.15.29:0/1968579852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4134ea0 0x7fb1a40c31d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fb194009760 tx=0x7fb194009060).ready entity=mon.1 client_cookie=69efe257602be8b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:19.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.258+0000 7fb1a2ffd700 1 -- 172.21.15.29:0/1968579852 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400f510 con 0x7fb1a4134ea0 2024-01-24T14:14:19.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.258+0000 7fb1a2ffd700 1 -- 172.21.15.29:0/1968579852 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19400f670 con 0x7fb1a4134ea0 2024-01-24T14:14:19.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.258+0000 7fb1a2ffd700 1 -- 172.21.15.29:0/1968579852 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400f9f0 con 0x7fb1a4134ea0 2024-01-24T14:14:19.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.259+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1968579852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4134ea0 msgr2=0x7fb1a40c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:19.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.259+0000 7fb1a9f1d700 1 --2- 172.21.15.29:0/1968579852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4134ea0 0x7fb1a40c31d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fb194009760 tx=0x7fb194009060).stop 2024-01-24T14:14:19.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.259+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1968579852 shutdown_connections 2024-01-24T14:14:19.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.259+0000 7fb1a9f1d700 1 --2- 172.21.15.29:0/1968579852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4134ea0 0x7fb1a40c31d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.259+0000 7fb1a9f1d700 1 --2- 172.21.15.29:0/1968579852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1a40c36c0 0x7fb1a40664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.259+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1968579852 >> 172.21.15.29:0/1968579852 conn(0x7fb1a40c8bd0 msgr2=0x7fb1a40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:19.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.259+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1968579852 shutdown_connections 2024-01-24T14:14:19.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.259+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1968579852 wait complete. 2024-01-24T14:14:19.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.260+0000 7fb1a9f1d700 1 Processor -- start 2024-01-24T14:14:19.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.260+0000 7fb1a9f1d700 1 -- start start 2024-01-24T14:14:19.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.261+0000 7fb1a9f1d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1a40669b0 0x7fb1a414d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.261+0000 7fb1a9f1d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1a40c36c0 0x7fb1a4163200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.261+0000 7fb1a9f1d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4134ea0 0x7fb1a414f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.261+0000 7fb1a9f1d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4136eb0 con 0x7fb1a40669b0 2024-01-24T14:14:19.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.261+0000 7fb1a9f1d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4136580 con 0x7fb1a4134ea0 2024-01-24T14:14:19.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.261+0000 7fb1a9f1d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4136860 con 0x7fb1a40c36c0 2024-01-24T14:14:19.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.261+0000 7fb1a37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1a40669b0 0x7fb1a414d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:19.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.261+0000 7fb19affd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1a40c36c0 0x7fb1a4163200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:19.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.261+0000 7fb1a37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1a40669b0 0x7fb1a414d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52864/0 (socket says 172.21.15.29:52864) 2024-01-24T14:14:19.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.261+0000 7fb1a37fe700 1 -- 172.21.15.29:0/1387116138 learned_addr learned my addr 172.21.15.29:0/1387116138 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:19.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.261+0000 7fb1a3fff700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4134ea0 0x7fb1a414f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:19.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.262+0000 7fb1a37fe700 1 -- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1a40c36c0 msgr2=0x7fb1a4163200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:19.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.262+0000 7fb1a37fe700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1a40c36c0 0x7fb1a4163200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.262+0000 7fb1a37fe700 1 -- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4134ea0 msgr2=0x7fb1a414f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:19.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.262+0000 7fb1a37fe700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4134ea0 0x7fb1a414f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.262+0000 7fb1a37fe700 1 -- 172.21.15.29:0/1387116138 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0092a0 con 0x7fb1a40669b0 2024-01-24T14:14:19.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.262+0000 7fb1a3fff700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4134ea0 0x7fb1a414f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:19.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.262+0000 7fb1a37fe700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1a40669b0 0x7fb1a414d080 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7fb194009e00 tx=0x7fb194017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:19.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.262+0000 7fb19bfff700 1 -- 172.21.15.29:0/1387116138 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19401a4d0 con 0x7fb1a40669b0 2024-01-24T14:14:19.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.262+0000 7fb19bfff700 1 -- 172.21.15.29:0/1387116138 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19401e070 con 0x7fb1a40669b0 2024-01-24T14:14:19.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.262+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1387116138 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1940092a0 con 0x7fb1a40669b0 2024-01-24T14:14:19.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.262+0000 7fb19bfff700 1 -- 172.21.15.29:0/1387116138 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19401a8a0 con 0x7fb1a40669b0 2024-01-24T14:14:19.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.262+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1387116138 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a41636f0 con 0x7fb1a40669b0 2024-01-24T14:14:19.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.263+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1387116138 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a40475e0 con 0x7fb1a40669b0 2024-01-24T14:14:19.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.263+0000 7fb19bfff700 1 -- 172.21.15.29:0/1387116138 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 36) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fb194020030 con 0x7fb1a40669b0 2024-01-24T14:14:19.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.264+0000 7fb19bfff700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb18405f300 0x7fb1840617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.264+0000 7fb19bfff700 1 -- 172.21.15.29:0/1387116138 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fb19407b8c0 con 0x7fb1a40669b0 2024-01-24T14:14:19.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.264+0000 7fb19affd700 1 -- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb18405f300 msgr2=0x7fb1840617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:19.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.264+0000 7fb19affd700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb18405f300 0x7fb1840617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:19.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.268+0000 7fb19bfff700 1 -- 172.21.15.29:0/1387116138 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb19404c490 con 0x7fb1a40669b0 2024-01-24T14:14:19.273 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:19.273+0000 7f5349dda680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:14:19.414 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:19.413+0000 7f0678022680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:14:19.459 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:19.459+0000 7f5349dda680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:14:19.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.465+0000 7fb19affd700 1 -- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb18405f300 msgr2=0x7fb1840617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:19.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.465+0000 7fb19affd700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb18405f300 0x7fb1840617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:14:19.489 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:19.489+0000 7f0678022680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:14:19.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.495+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1387116138 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb1a4067d70 con 0x7fb1a40669b0 2024-01-24T14:14:19.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.496+0000 7fb19bfff700 1 -- 172.21.15.29:0/1387116138 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v106) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb194028030 con 0x7fb1a40669b0 2024-01-24T14:14:19.496 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T14:14:19.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.498+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb18405f300 msgr2=0x7fb1840617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:19.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.498+0000 7fb1a9f1d700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb18405f300 0x7fb1840617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:19.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.498+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1387116138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1a40669b0 msgr2=0x7fb1a414d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:19.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.498+0000 7fb1a9f1d700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1a40669b0 0x7fb1a414d080 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7fb194009e00 tx=0x7fb194017000).stop 2024-01-24T14:14:19.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.498+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1387116138 shutdown_connections 2024-01-24T14:14:19.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.498+0000 7fb1a9f1d700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4134ea0 0x7fb1a414f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.498+0000 7fb1a9f1d700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb18405f300 0x7fb1840617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:19.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.498+0000 7fb1a9f1d700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1a40669b0 0x7fb1a414d080 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.498+0000 7fb1a9f1d700 1 --2- 172.21.15.29:0/1387116138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1a40c36c0 0x7fb1a4163200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.498+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1387116138 >> 172.21.15.29:0/1387116138 conn(0x7fb1a40c8bd0 msgr2=0x7fb1a41535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:19.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.498+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1387116138 shutdown_connections 2024-01-24T14:14:19.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.498+0000 7fb1a9f1d700 1 -- 172.21.15.29:0/1387116138 wait complete. 2024-01-24T14:14:19.513 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T14:14:19.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.657+0000 7f2bc94eb700 1 Processor -- start 2024-01-24T14:14:19.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.657+0000 7f2bc94eb700 1 -- start start 2024-01-24T14:14:19.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.658+0000 7f2bc94eb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2bc4064760 0x7f2bc4064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.658+0000 7f2bc94eb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c8490 0x7f2bc40a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.658+0000 7f2bc94eb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2bc4154700 0x7f2bc4154b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.658+0000 7f2bc94eb700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc4152240 con 0x7f2bc4064760 2024-01-24T14:14:19.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.658+0000 7f2bc94eb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc41586c0 con 0x7f2bc4154700 2024-01-24T14:14:19.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.658+0000 7f2bc94eb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc40aa670 con 0x7f2bc40c8490 2024-01-24T14:14:19.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc2ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2bc4064760 0x7f2bc4064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:19.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc2ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2bc4064760 0x7f2bc4064b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52878/0 (socket says 172.21.15.29:52878) 2024-01-24T14:14:19.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc2ffd700 1 -- 172.21.15.29:0/3124764006 learned_addr learned my addr 172.21.15.29:0/3124764006 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:19.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc27fc700 1 --2- 172.21.15.29:0/3124764006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c8490 0x7f2bc40a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:19.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc37fe700 1 --2- 172.21.15.29:0/3124764006 >> v2:172.21.15.163:3300/0 conn(0x7f2bc4154700 0x7f2bc4154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:19.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc2ffd700 1 -- 172.21.15.29:0/3124764006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c8490 msgr2=0x7f2bc40a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc2ffd700 1 --2- 172.21.15.29:0/3124764006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c8490 0x7f2bc40a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc2ffd700 1 -- 172.21.15.29:0/3124764006 >> v2:172.21.15.163:3300/0 conn(0x7f2bc4154700 msgr2=0x7f2bc4154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc2ffd700 1 --2- 172.21.15.29:0/3124764006 >> v2:172.21.15.163:3300/0 conn(0x7f2bc4154700 0x7f2bc4154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc2ffd700 1 -- 172.21.15.29:0/3124764006 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc4157d90 con 0x7f2bc4064760 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc2ffd700 1 --2- 172.21.15.29:0/3124764006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc4064760 0x7f2bc4064b40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f2bc4084340 tx=0x7f2bb40090f0).ready entity=mon.0 client_cookie=531f653317ddb59a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc1ffb700 1 -- 172.21.15.29:0/3124764006 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb400f510 con 0x7f2bc4064760 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc1ffb700 1 -- 172.21.15.29:0/3124764006 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb400f670 con 0x7f2bc4064760 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.659+0000 7f2bc1ffb700 1 -- 172.21.15.29:0/3124764006 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb400f9f0 con 0x7f2bc4064760 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.660+0000 7f2bc94eb700 1 -- 172.21.15.29:0/3124764006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc4064760 msgr2=0x7f2bc4064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.660+0000 7f2bc94eb700 1 --2- 172.21.15.29:0/3124764006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc4064760 0x7f2bc4064b40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f2bc4084340 tx=0x7f2bb40090f0).stop 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.660+0000 7f2bc94eb700 1 -- 172.21.15.29:0/3124764006 shutdown_connections 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.660+0000 7f2bc94eb700 1 --2- 172.21.15.29:0/3124764006 >> v2:172.21.15.163:3300/0 conn(0x7f2bc4154700 0x7f2bc4154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.660+0000 7f2bc94eb700 1 --2- 172.21.15.29:0/3124764006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc4064760 0x7f2bc4064b40 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.660+0000 7f2bc94eb700 1 --2- 172.21.15.29:0/3124764006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc40c8490 0x7f2bc40a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.660+0000 7f2bc94eb700 1 -- 172.21.15.29:0/3124764006 >> 172.21.15.29:0/3124764006 conn(0x7f2bc40cb8b0 msgr2=0x7f2bc40c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:19.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.660+0000 7f2bc94eb700 1 -- 172.21.15.29:0/3124764006 shutdown_connections 2024-01-24T14:14:19.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.660+0000 7f2bc94eb700 1 -- 172.21.15.29:0/3124764006 wait complete. 2024-01-24T14:14:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.662+0000 7f2bc94eb700 1 Processor -- start 2024-01-24T14:14:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.662+0000 7f2bc94eb700 1 -- start start 2024-01-24T14:14:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.662+0000 7f2bc94eb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc4064760 0x7f2bc4163ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.662+0000 7f2bc94eb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc40c8490 0x7f2bc4167d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.662+0000 7f2bc94eb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc4154700 0x7f2bc4168270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.662+0000 7f2bc94eb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4168760 con 0x7f2bc4154700 2024-01-24T14:14:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.662+0000 7f2bc94eb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc40abf10 con 0x7f2bc40c8490 2024-01-24T14:14:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.662+0000 7f2bc94eb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc40ac1f0 con 0x7f2bc4064760 2024-01-24T14:14:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc4154700 0x7f2bc4168270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc4154700 0x7f2bc4168270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52880/0 (socket says 172.21.15.29:52880) 2024-01-24T14:14:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc37fe700 1 -- 172.21.15.29:0/2583407603 learned_addr learned my addr 172.21.15.29:0/2583407603 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:19.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc2ffd700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc4064760 0x7f2bc4163ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc27fc700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc40c8490 0x7f2bc4167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc37fe700 1 -- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc4064760 msgr2=0x7f2bc4163ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc37fe700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc4064760 0x7f2bc4163ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc37fe700 1 -- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc40c8490 msgr2=0x7f2bc4167d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc37fe700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc40c8490 0x7f2bc4167d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc37fe700 1 -- 172.21.15.29:0/2583407603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb4009370 con 0x7f2bc4154700 2024-01-24T14:14:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc37fe700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc4154700 0x7f2bc4168270 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f2bbc009df0 tx=0x7f2bbc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2baaffd700 1 -- 172.21.15.29:0/2583407603 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc00caa0 con 0x7f2bc4154700 2024-01-24T14:14:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2baaffd700 1 -- 172.21.15.29:0/2583407603 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc00f040 con 0x7f2bc4154700 2024-01-24T14:14:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc94eb700 1 -- 172.21.15.29:0/2583407603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc40ac4d0 con 0x7f2bc4154700 2024-01-24T14:14:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2baaffd700 1 -- 172.21.15.29:0/2583407603 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc00ce10 con 0x7f2bc4154700 2024-01-24T14:14:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.663+0000 7f2bc94eb700 1 -- 172.21.15.29:0/2583407603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc41644e0 con 0x7f2bc4154700 2024-01-24T14:14:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.665+0000 7f2baaffd700 1 -- 172.21.15.29:0/2583407603 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 36) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f2bbc013610 con 0x7f2bc4154700 2024-01-24T14:14:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.665+0000 7f2bc94eb700 1 -- 172.21.15.29:0/2583407603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc4164700 con 0x7f2bc4154700 2024-01-24T14:14:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.665+0000 7f2baaffd700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2bb005f300 0x7f2bb00617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.665+0000 7f2bc2ffd700 1 -- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2bb005f300 msgr2=0x7f2bb00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.665+0000 7f2baaffd700 1 -- 172.21.15.29:0/2583407603 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f2bbc07b180 con 0x7f2bc4154700 2024-01-24T14:14:19.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.665+0000 7f2bc2ffd700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2bb005f300 0x7f2bb00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:19.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.670+0000 7f2baaffd700 1 -- 172.21.15.29:0/2583407603 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2bc4164700 con 0x7f2bc4154700 2024-01-24T14:14:19.719 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:19.719+0000 7f0678022680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:14:19.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.853+0000 7f2bc94eb700 1 -- 172.21.15.29:0/2583407603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2bc4164700 con 0x7f2bc4154700 2024-01-24T14:14:19.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.853+0000 7f2baaffd700 1 -- 172.21.15.29:0/2583407603 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v106) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2bc4164700 con 0x7f2bc4154700 2024-01-24T14:14:19.854 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T14:14:19.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.855+0000 7f2bc94eb700 1 -- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2bb005f300 msgr2=0x7f2bb00617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:19.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.855+0000 7f2bc94eb700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2bb005f300 0x7f2bb00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:19.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.855+0000 7f2bc94eb700 1 -- 172.21.15.29:0/2583407603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc4154700 msgr2=0x7f2bc4168270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:19.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.855+0000 7f2bc94eb700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc4154700 0x7f2bc4168270 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f2bbc009df0 tx=0x7f2bbc00b040).stop 2024-01-24T14:14:19.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.855+0000 7f2bc94eb700 1 -- 172.21.15.29:0/2583407603 shutdown_connections 2024-01-24T14:14:19.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.855+0000 7f2bc94eb700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2bb005f300 0x7f2bb00617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:19.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.855+0000 7f2bc94eb700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bc4154700 0x7f2bc4168270 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.855+0000 7f2bc94eb700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc4064760 0x7f2bc4163ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.855+0000 7f2bc94eb700 1 --2- 172.21.15.29:0/2583407603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc40c8490 0x7f2bc4167d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:19.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.855+0000 7f2bc94eb700 1 -- 172.21.15.29:0/2583407603 >> 172.21.15.29:0/2583407603 conn(0x7f2bc40cb8b0 msgr2=0x7f2bc4153b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:19.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.855+0000 7f2bc94eb700 1 -- 172.21.15.29:0/2583407603 shutdown_connections 2024-01-24T14:14:19.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:19.855+0000 7f2bc94eb700 1 -- 172.21.15.29:0/2583407603 wait complete. 2024-01-24T14:14:19.870 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T14:14:19.880 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:19.880+0000 7f5349dda680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:14:19.983 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:19.982+0000 7f5349dda680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:14:20.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.007+0000 7ff6a36c9700 1 Processor -- start 2024-01-24T14:14:20.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.007+0000 7ff6a36c9700 1 -- start start 2024-01-24T14:14:20.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.008+0000 7ff6a36c9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff69c13a5e0 0x7ff69c0639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.008+0000 7ff6a36c9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff69c063eb0 0x7ff69c0664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.008+0000 7ff6a36c9700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69c066e00 con 0x7ff69c0669b0 2024-01-24T14:14:20.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.008+0000 7ff6a36c9700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69c0679f0 con 0x7ff69c063eb0 2024-01-24T14:14:20.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.008+0000 7ff6a36c9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c067cd0 con 0x7ff69c13a5e0 2024-01-24T14:14:20.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.008+0000 7ff6a0c64700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff69c063eb0 0x7ff69c0664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.008+0000 7ff6a1c66700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff69c0669b0 0x7ff69c157bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35320/0 (socket says 172.21.15.29:35320) 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.008+0000 7ff6a0c64700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff69c063eb0 0x7ff69c0664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52892/0 (socket says 172.21.15.29:52892) 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.008+0000 7ff6a1c66700 1 -- 172.21.15.29:0/2941266714 learned_addr learned my addr 172.21.15.29:0/2941266714 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.008+0000 7ff6a1465700 1 --2- 172.21.15.29:0/2941266714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff69c13a5e0 0x7ff69c0639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.008+0000 7ff6a0c64700 1 -- 172.21.15.29:0/2941266714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff69c13a5e0 msgr2=0x7ff69c0639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.008+0000 7ff6a0c64700 1 --2- 172.21.15.29:0/2941266714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff69c13a5e0 0x7ff69c0639c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff6a0c64700 1 -- 172.21.15.29:0/2941266714 >> v1:172.21.15.29:6789/0 conn(0x7ff69c0669b0 legacy=0x7ff69c157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff6a0c64700 1 -- 172.21.15.29:0/2941266714 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c067fb0 con 0x7ff69c063eb0 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff693fff700 1 -- 172.21.15.29:0/2941266714 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082458327 0 0) 0x7ff69c066e00 con 0x7ff69c0669b0 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff6a0c64700 1 --2- 172.21.15.29:0/2941266714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff69c063eb0 0x7ff69c0664c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7ff68c009760 tx=0x7ff68c009060).ready entity=mon.0 client_cookie=53c1b2bc5476418 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff693fff700 1 -- 172.21.15.29:0/2941266714 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c00f510 con 0x7ff69c063eb0 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff693fff700 1 -- 172.21.15.29:0/2941266714 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c00f670 con 0x7ff69c063eb0 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff693fff700 1 -- 172.21.15.29:0/2941266714 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c00f9f0 con 0x7ff69c063eb0 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff6a36c9700 1 -- 172.21.15.29:0/2941266714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff69c063eb0 msgr2=0x7ff69c0664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff6a36c9700 1 --2- 172.21.15.29:0/2941266714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff69c063eb0 0x7ff69c0664c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7ff68c009760 tx=0x7ff68c009060).stop 2024-01-24T14:14:20.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff6a36c9700 1 -- 172.21.15.29:0/2941266714 shutdown_connections 2024-01-24T14:14:20.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff6a36c9700 1 --2- 172.21.15.29:0/2941266714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff69c13a5e0 0x7ff69c0639c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff6a36c9700 1 --2- 172.21.15.29:0/2941266714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff69c063eb0 0x7ff69c0664c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.009+0000 7ff6a36c9700 1 -- 172.21.15.29:0/2941266714 >> 172.21.15.29:0/2941266714 conn(0x7ff69c0c8bd0 msgr2=0x7ff69c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:20.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.010+0000 7ff6a36c9700 1 -- 172.21.15.29:0/2941266714 shutdown_connections 2024-01-24T14:14:20.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.010+0000 7ff6a36c9700 1 -- 172.21.15.29:0/2941266714 wait complete. 2024-01-24T14:14:20.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.011+0000 7ff6a36c9700 1 Processor -- start 2024-01-24T14:14:20.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.011+0000 7ff6a36c9700 1 -- start start 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.012+0000 7ff6a36c9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff69c063eb0 0x7ff69c138dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a36c9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff69c0669b0 0x7ff69c0ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a36c9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff69c13a5e0 0x7ff69c0a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a36c9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0a9a90 con 0x7ff69c13a5e0 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a36c9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0abd40 con 0x7ff69c063eb0 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a36c9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0a91f0 con 0x7ff69c0669b0 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a1c66700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff69c13a5e0 0x7ff69c0a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a1c66700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff69c13a5e0 0x7ff69c0a9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52900/0 (socket says 172.21.15.29:52900) 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a1c66700 1 -- 172.21.15.29:0/1727133898 learned_addr learned my addr 172.21.15.29:0/1727133898 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a0c64700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff69c0669b0 0x7ff69c0ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a1465700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff69c063eb0 0x7ff69c138dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a1c66700 1 -- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff69c0669b0 msgr2=0x7ff69c0ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a1c66700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff69c0669b0 0x7ff69c0ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a1c66700 1 -- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff69c063eb0 msgr2=0x7ff69c138dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a1c66700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff69c063eb0 0x7ff69c138dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a1c66700 1 -- 172.21.15.29:0/1727133898 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c0092a0 con 0x7ff69c13a5e0 2024-01-24T14:14:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a1465700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff69c063eb0 0x7ff69c138dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6a1c66700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff69c13a5e0 0x7ff69c0a9f00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7ff69400c7e0 tx=0x7ff694007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.013+0000 7ff6917fa700 1 -- 172.21.15.29:0/1727133898 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694005d30 con 0x7ff69c13a5e0 2024-01-24T14:14:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.014+0000 7ff6917fa700 1 -- 172.21.15.29:0/1727133898 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff694005e90 con 0x7ff69c13a5e0 2024-01-24T14:14:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.014+0000 7ff6917fa700 1 -- 172.21.15.29:0/1727133898 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6940072e0 con 0x7ff69c13a5e0 2024-01-24T14:14:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.014+0000 7ff6a36c9700 1 -- 172.21.15.29:0/1727133898 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c0a94d0 con 0x7ff69c13a5e0 2024-01-24T14:14:20.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.014+0000 7ff6a36c9700 1 -- 172.21.15.29:0/1727133898 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c0a8d60 con 0x7ff69c13a5e0 2024-01-24T14:14:20.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.015+0000 7ff6917fa700 1 -- 172.21.15.29:0/1727133898 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 36) v1 ==== 75605+0+0 (secure 0 0 0) 0x7ff6940076d0 con 0x7ff69c13a5e0 2024-01-24T14:14:20.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.015+0000 7ff6a36c9700 1 -- 172.21.15.29:0/1727133898 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69c0a8fb0 con 0x7ff69c13a5e0 2024-01-24T14:14:20.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.015+0000 7ff6917fa700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff68005f300 0x7ff6800617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.016+0000 7ff6917fa700 1 -- 172.21.15.29:0/1727133898 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7400+0+0 (secure 0 0 0) 0x7ff69407a430 con 0x7ff69c13a5e0 2024-01-24T14:14:20.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.016+0000 7ff6a1465700 1 -- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff68005f300 msgr2=0x7ff6800617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:20.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.016+0000 7ff6a1465700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff68005f300 0x7ff6800617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:20.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.020+0000 7ff6917fa700 1 -- 172.21.15.29:0/1727133898 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff69c0a8fb0 con 0x7ff69c13a5e0 2024-01-24T14:14:20.140 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:20.139+0000 7f5349dda680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:14:20.215 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:20.214+0000 7f5349dda680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:14:20.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.217+0000 7ff6a1465700 1 -- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff68005f300 msgr2=0x7ff6800617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:20.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.217+0000 7ff6a1465700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff68005f300 0x7ff6800617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:14:20.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.223+0000 7ff6a36c9700 1 -- 172.21.15.29:0/1727133898 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff69c0a8fb0 con 0x7ff69c13a5e0 2024-01-24T14:14:20.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.224+0000 7ff6917fa700 1 -- 172.21.15.29:0/1727133898 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff69c0a8fb0 con 0x7ff69c13a5e0 2024-01-24T14:14:20.224 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T14:14:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.226+0000 7ff6a36c9700 1 -- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff68005f300 msgr2=0x7ff6800617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.226+0000 7ff6a36c9700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff68005f300 0x7ff6800617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.226+0000 7ff6a36c9700 1 -- 172.21.15.29:0/1727133898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff69c13a5e0 msgr2=0x7ff69c0a9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.226+0000 7ff6a36c9700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff69c13a5e0 0x7ff69c0a9f00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7ff69400c7e0 tx=0x7ff694007f90).stop 2024-01-24T14:14:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.226+0000 7ff6a36c9700 1 -- 172.21.15.29:0/1727133898 shutdown_connections 2024-01-24T14:14:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.226+0000 7ff6a36c9700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff68005f300 0x7ff6800617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:20.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.226+0000 7ff6a36c9700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff69c13a5e0 0x7ff69c0a9f00 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.226+0000 7ff6a36c9700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff69c063eb0 0x7ff69c138dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.226+0000 7ff6a36c9700 1 --2- 172.21.15.29:0/1727133898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff69c0669b0 0x7ff69c0ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.226+0000 7ff6a36c9700 1 -- 172.21.15.29:0/1727133898 >> 172.21.15.29:0/1727133898 conn(0x7ff69c0c8bd0 msgr2=0x7ff69c154e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.226+0000 7ff6a36c9700 1 -- 172.21.15.29:0/1727133898 shutdown_connections 2024-01-24T14:14:20.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.226+0000 7ff6a36c9700 1 -- 172.21.15.29:0/1727133898 wait complete. 2024-01-24T14:14:20.228 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:20.226+0000 7f0678022680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:14:20.245 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T14:14:20.282 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:20.281+0000 7f5349dda680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:14:20.359 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:20.358+0000 7f0678022680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:14:20.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.382+0000 7ff5ed44f700 1 Processor -- start 2024-01-24T14:14:20.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.382+0000 7ff5ed44f700 1 -- start start 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.383+0000 7ff5ed44f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5e8066c30 0x7ff5e813d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.383+0000 7ff5ed44f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5e8157de0 0x7ff5e8158230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.383+0000 7ff5ed44f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e8158b50 con 0x7ff5e813d580 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.383+0000 7ff5ed44f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e8068470 con 0x7ff5e8157de0 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.383+0000 7ff5ed44f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e8067b40 con 0x7ff5e8066c30 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.383+0000 7ff5e77fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5e8157de0 0x7ff5e8158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.383+0000 7ff5e77fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5e8157de0 0x7ff5e8158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52904/0 (socket says 172.21.15.29:52904) 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.383+0000 7ff5e77fe700 1 -- 172.21.15.29:0/1144572115 learned_addr learned my addr 172.21.15.29:0/1144572115 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.383+0000 7ff5e67fc700 1 --2- 172.21.15.29:0/1144572115 >> v2:172.21.15.163:3300/0 conn(0x7ff5e8066c30 0x7ff5e813d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.384+0000 7ff5e77fe700 1 -- 172.21.15.29:0/1144572115 >> v2:172.21.15.163:3300/0 conn(0x7ff5e8066c30 msgr2=0x7ff5e813d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.384+0000 7ff5e77fe700 1 --2- 172.21.15.29:0/1144572115 >> v2:172.21.15.163:3300/0 conn(0x7ff5e8066c30 0x7ff5e813d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.384+0000 7ff5e77fe700 1 -- 172.21.15.29:0/1144572115 >> v1:172.21.15.163:6789/0 conn(0x7ff5e813d580 legacy=0x7ff5e8066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.384+0000 7ff5e77fe700 1 -- 172.21.15.29:0/1144572115 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8067e20 con 0x7ff5e8157de0 2024-01-24T14:14:20.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.384+0000 7ff5e67fc700 1 --2- 172.21.15.29:0/1144572115 >> v2:172.21.15.163:3300/0 conn(0x7ff5e8066c30 0x7ff5e813d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:20.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.384+0000 7ff5e77fe700 1 --2- 172.21.15.29:0/1144572115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5e8157de0 0x7ff5e8158230 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7ff5e8046ec0 tx=0x7ff5e0009840).ready entity=mon.0 client_cookie=d76364121ef390a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:20.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.384+0000 7ff5e5ffb700 1 -- 172.21.15.29:0/1144572115 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000e070 con 0x7ff5e8157de0 2024-01-24T14:14:20.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.384+0000 7ff5e5ffb700 1 -- 172.21.15.29:0/1144572115 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e000ba20 con 0x7ff5e8157de0 2024-01-24T14:14:20.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.384+0000 7ff5e5ffb700 1 -- 172.21.15.29:0/1144572115 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000bda0 con 0x7ff5e8157de0 2024-01-24T14:14:20.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.385+0000 7ff5ed44f700 1 -- 172.21.15.29:0/1144572115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5e8157de0 msgr2=0x7ff5e8158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.385+0000 7ff5ed44f700 1 --2- 172.21.15.29:0/1144572115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5e8157de0 0x7ff5e8158230 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7ff5e8046ec0 tx=0x7ff5e0009840).stop 2024-01-24T14:14:20.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.385+0000 7ff5ed44f700 1 -- 172.21.15.29:0/1144572115 shutdown_connections 2024-01-24T14:14:20.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.385+0000 7ff5ed44f700 1 --2- 172.21.15.29:0/1144572115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5e8157de0 0x7ff5e8158230 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.385+0000 7ff5ed44f700 1 --2- 172.21.15.29:0/1144572115 >> v2:172.21.15.163:3300/0 conn(0x7ff5e8066c30 0x7ff5e813d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.385+0000 7ff5ed44f700 1 -- 172.21.15.29:0/1144572115 >> 172.21.15.29:0/1144572115 conn(0x7ff5e80c8bc0 msgr2=0x7ff5e80c8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:20.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.385+0000 7ff5ed44f700 1 -- 172.21.15.29:0/1144572115 shutdown_connections 2024-01-24T14:14:20.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.385+0000 7ff5ed44f700 1 -- 172.21.15.29:0/1144572115 wait complete. 2024-01-24T14:14:20.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.386+0000 7ff5ed44f700 1 Processor -- start 2024-01-24T14:14:20.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.386+0000 7ff5ed44f700 1 -- start start 2024-01-24T14:14:20.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.387+0000 7ff5ed44f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e8066c30 0x7ff5e80aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.387+0000 7ff5ed44f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e813d580 0x7ff5e80a7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.387+0000 7ff5ed44f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5e8157de0 0x7ff5e80c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.387+0000 7ff5ed44f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80a9c50 con 0x7ff5e8157de0 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.387+0000 7ff5ed44f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80ab4c0 con 0x7ff5e813d580 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.387+0000 7ff5ed44f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8164f70 con 0x7ff5e8066c30 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.387+0000 7ff5e77fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5e8157de0 0x7ff5e80c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.387+0000 7ff5e77fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5e8157de0 0x7ff5e80c6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52906/0 (socket says 172.21.15.29:52906) 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.387+0000 7ff5e77fe700 1 -- 172.21.15.29:0/2472937199 learned_addr learned my addr 172.21.15.29:0/2472937199 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.387+0000 7ff5e6ffd700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e8066c30 0x7ff5e80aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.387+0000 7ff5e67fc700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e813d580 0x7ff5e80a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.388+0000 7ff5e77fe700 1 -- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e8066c30 msgr2=0x7ff5e80aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.388+0000 7ff5e77fe700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e8066c30 0x7ff5e80aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.388+0000 7ff5e77fe700 1 -- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e813d580 msgr2=0x7ff5e80a7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.388+0000 7ff5e77fe700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e813d580 0x7ff5e80a7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.388+0000 7ff5e77fe700 1 -- 172.21.15.29:0/2472937199 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc0092a0 con 0x7ff5e8157de0 2024-01-24T14:14:20.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.388+0000 7ff5e77fe700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5e8157de0 0x7ff5e80c6ed0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7ff5e000b6c0 tx=0x7ff5e0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:20.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.388+0000 7ff5ceffd700 1 -- 172.21.15.29:0/2472937199 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00092e0 con 0x7ff5e8157de0 2024-01-24T14:14:20.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.388+0000 7ff5ceffd700 1 -- 172.21.15.29:0/2472937199 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e001e470 con 0x7ff5e8157de0 2024-01-24T14:14:20.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.388+0000 7ff5ed44f700 1 -- 172.21.15.29:0/2472937199 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e0009ae0 con 0x7ff5e8157de0 2024-01-24T14:14:20.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.388+0000 7ff5ceffd700 1 -- 172.21.15.29:0/2472937199 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e001e830 con 0x7ff5e8157de0 2024-01-24T14:14:20.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.388+0000 7ff5ed44f700 1 -- 172.21.15.29:0/2472937199 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e80a9600 con 0x7ff5e8157de0 2024-01-24T14:14:20.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.389+0000 7ff5ed44f700 1 -- 172.21.15.29:0/2472937199 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e80a9dc0 con 0x7ff5e8157de0 2024-01-24T14:14:20.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.390+0000 7ff5ceffd700 1 -- 172.21.15.29:0/2472937199 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 36) v1 ==== 75605+0+0 (secure 0 0 0) 0x7ff5e0003040 con 0x7ff5e8157de0 2024-01-24T14:14:20.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.390+0000 7ff5ceffd700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5d405f300 0x7ff5d40617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.390+0000 7ff5ceffd700 1 -- 172.21.15.29:0/2472937199 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7400+0+0 (secure 0 0 0) 0x7ff5e007bf20 con 0x7ff5e8157de0 2024-01-24T14:14:20.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.390+0000 7ff5e6ffd700 1 -- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5d405f300 msgr2=0x7ff5d40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:20.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.390+0000 7ff5e6ffd700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5d405f300 0x7ff5d40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:20.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.398+0000 7ff5ceffd700 1 -- 172.21.15.29:0/2472937199 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff5e00aab00 con 0x7ff5e8157de0 2024-01-24T14:14:20.540 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:20.539+0000 7f5349dda680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:14:20.542 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:20.541+0000 7f0678022680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:14:20.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.581+0000 7ff5ed44f700 1 -- 172.21.15.29:0/2472937199 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5e80475e0 con 0x7ff5e8157de0 2024-01-24T14:14:20.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.582+0000 7ff5ceffd700 1 -- 172.21.15.29:0/2472937199 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5e0023020 con 0x7ff5e8157de0 2024-01-24T14:14:20.583 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T14:14:20.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.584+0000 7ff5ed44f700 1 -- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5d405f300 msgr2=0x7ff5d40617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:20.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.584+0000 7ff5ed44f700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5d405f300 0x7ff5d40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:20.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.584+0000 7ff5ed44f700 1 -- 172.21.15.29:0/2472937199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5e8157de0 msgr2=0x7ff5e80c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.584+0000 7ff5ed44f700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5e8157de0 0x7ff5e80c6ed0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7ff5e000b6c0 tx=0x7ff5e0009840).stop 2024-01-24T14:14:20.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.584+0000 7ff5ed44f700 1 -- 172.21.15.29:0/2472937199 shutdown_connections 2024-01-24T14:14:20.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.584+0000 7ff5ed44f700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5d405f300 0x7ff5d40617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:20.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.584+0000 7ff5ed44f700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5e8157de0 0x7ff5e80c6ed0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.584+0000 7ff5ed44f700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e8066c30 0x7ff5e80aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.584+0000 7ff5ed44f700 1 --2- 172.21.15.29:0/2472937199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e813d580 0x7ff5e80a7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.584+0000 7ff5ed44f700 1 -- 172.21.15.29:0/2472937199 >> 172.21.15.29:0/2472937199 conn(0x7ff5e80c8bc0 msgr2=0x7ff5e80a7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:20.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.584+0000 7ff5ed44f700 1 -- 172.21.15.29:0/2472937199 shutdown_connections 2024-01-24T14:14:20.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.584+0000 7ff5ed44f700 1 -- 172.21.15.29:0/2472937199 wait complete. 2024-01-24T14:14:20.599 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:14:20.632 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:20.631+0000 7f0678022680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:14:20.712 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:20.712+0000 7f0678022680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:14:20.722 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:20.721+0000 7f5349dda680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:14:20.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.730+0000 7fb838144700 1 Processor -- start 2024-01-24T14:14:20.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.730+0000 7fb838144700 1 -- start start 2024-01-24T14:14:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.731+0000 7fb838144700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb8300c5410 0x7fb8300aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.731+0000 7fb838144700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb830154750 0x7fb8300631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.731+0000 7fb838144700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8300aa6c0 con 0x7fb830154370 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.731+0000 7fb838144700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb830064240 con 0x7fb8300c5410 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.731+0000 7fb838144700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300cb5c0 con 0x7fb830154750 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.731+0000 7fb8356df700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb8300c5410 0x7fb8300aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.731+0000 7fb835ee0700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb830154370 0x7fb8300ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35336/0 (socket says 172.21.15.29:35336) 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.731+0000 7fb835ee0700 1 -- 172.21.15.29:0/2494547986 learned_addr learned my addr 172.21.15.29:0/2494547986 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.731+0000 7fb8356df700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb8300c5410 0x7fb8300aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52916/0 (socket says 172.21.15.29:52916) 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.731+0000 7fb8366e1700 1 --2- 172.21.15.29:0/2494547986 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb830154750 0x7fb8300631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.732+0000 7fb8356df700 1 -- 172.21.15.29:0/2494547986 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb830154750 msgr2=0x7fb8300631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.732+0000 7fb8356df700 1 --2- 172.21.15.29:0/2494547986 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb830154750 0x7fb8300631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.732+0000 7fb8356df700 1 -- 172.21.15.29:0/2494547986 >> v1:172.21.15.29:6789/0 conn(0x7fb830154370 legacy=0x7fb8300ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.732+0000 7fb8356df700 1 -- 172.21.15.29:0/2494547986 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8300cb8a0 con 0x7fb8300c5410 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.732+0000 7fb834ede700 1 -- 172.21.15.29:0/2494547986 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047378433 0 0) 0x7fb8300aa6c0 con 0x7fb830154370 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.732+0000 7fb8366e1700 1 --2- 172.21.15.29:0/2494547986 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb830154750 0x7fb8300631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.732+0000 7fb8356df700 1 --2- 172.21.15.29:0/2494547986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8300c5410 0x7fb8300aa0e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7fb820009760 tx=0x7fb820009060).ready entity=mon.0 client_cookie=b71d5a33f3a7780d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.732+0000 7fb834ede700 1 -- 172.21.15.29:0/2494547986 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82000b580 con 0x7fb8300c5410 2024-01-24T14:14:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.732+0000 7fb834ede700 1 -- 172.21.15.29:0/2494547986 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82000b6e0 con 0x7fb8300c5410 2024-01-24T14:14:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.732+0000 7fb834ede700 1 -- 172.21.15.29:0/2494547986 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82000ba60 con 0x7fb8300c5410 2024-01-24T14:14:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.733+0000 7fb838144700 1 -- 172.21.15.29:0/2494547986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8300c5410 msgr2=0x7fb8300aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.733+0000 7fb838144700 1 --2- 172.21.15.29:0/2494547986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8300c5410 0x7fb8300aa0e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7fb820009760 tx=0x7fb820009060).stop 2024-01-24T14:14:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.733+0000 7fb838144700 1 -- 172.21.15.29:0/2494547986 shutdown_connections 2024-01-24T14:14:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.733+0000 7fb838144700 1 --2- 172.21.15.29:0/2494547986 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb830154750 0x7fb8300631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.733+0000 7fb838144700 1 --2- 172.21.15.29:0/2494547986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8300c5410 0x7fb8300aa0e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.733+0000 7fb838144700 1 -- 172.21.15.29:0/2494547986 >> 172.21.15.29:0/2494547986 conn(0x7fb83005ec80 msgr2=0x7fb8300cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.733+0000 7fb838144700 1 -- 172.21.15.29:0/2494547986 shutdown_connections 2024-01-24T14:14:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.733+0000 7fb838144700 1 -- 172.21.15.29:0/2494547986 wait complete. 2024-01-24T14:14:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.734+0000 7fb838144700 1 Processor -- start 2024-01-24T14:14:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.734+0000 7fb838144700 1 -- start start 2024-01-24T14:14:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb838144700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8300c5410 0x7fb830154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb838144700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb830154370 0x7fb830155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb838144700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb830154750 0x7fb830164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb838144700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300c4e10 con 0x7fb830154750 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb838144700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb830155bf0 con 0x7fb830154370 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb838144700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb830155ed0 con 0x7fb8300c5410 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb8366e1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb830154750 0x7fb830164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb835ee0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8300c5410 0x7fb830154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb8366e1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb830154750 0x7fb830164ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52932/0 (socket says 172.21.15.29:52932) 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb8366e1700 1 -- 172.21.15.29:0/866002819 learned_addr learned my addr 172.21.15.29:0/866002819 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb8356df700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb830154370 0x7fb830155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb8366e1700 1 -- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8300c5410 msgr2=0x7fb830154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb8366e1700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8300c5410 0x7fb830154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb8366e1700 1 -- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb830154370 msgr2=0x7fb830155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb8366e1700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb830154370 0x7fb830155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb8366e1700 1 -- 172.21.15.29:0/866002819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8200092a0 con 0x7fb830154750 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.735+0000 7fb8366e1700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb830154750 0x7fb830164ed0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fb828009fc0 tx=0x7fb8280099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.736+0000 7fb825ffb700 1 -- 172.21.15.29:0/866002819 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82800dbe0 con 0x7fb830154750 2024-01-24T14:14:20.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.736+0000 7fb825ffb700 1 -- 172.21.15.29:0/866002819 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82800f040 con 0x7fb830154750 2024-01-24T14:14:20.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.736+0000 7fb838144700 1 -- 172.21.15.29:0/866002819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8301561b0 con 0x7fb830154750 2024-01-24T14:14:20.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.736+0000 7fb825ffb700 1 -- 172.21.15.29:0/866002819 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8280133f0 con 0x7fb830154750 2024-01-24T14:14:20.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.736+0000 7fb838144700 1 -- 172.21.15.29:0/866002819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb830155860 con 0x7fb830154750 2024-01-24T14:14:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.737+0000 7fb825ffb700 1 -- 172.21.15.29:0/866002819 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 36) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fb8280137e0 con 0x7fb830154750 2024-01-24T14:14:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.737+0000 7fb838144700 1 -- 172.21.15.29:0/866002819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8300475e0 con 0x7fb830154750 2024-01-24T14:14:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.738+0000 7fb825ffb700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb81405f300 0x7fb8140617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.738+0000 7fb825ffb700 1 -- 172.21.15.29:0/866002819 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fb82807b050 con 0x7fb830154750 2024-01-24T14:14:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.738+0000 7fb835ee0700 1 -- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb81405f300 msgr2=0x7fb8140617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.738+0000 7fb835ee0700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb81405f300 0x7fb8140617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:20.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.742+0000 7fb825ffb700 1 -- 172.21.15.29:0/866002819 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb82804bb10 con 0x7fb830154750 2024-01-24T14:14:20.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.939+0000 7fb835ee0700 1 -- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb81405f300 msgr2=0x7fb8140617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:20.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:20.939+0000 7fb835ee0700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb81405f300 0x7fb8140617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:14:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.021+0000 7fb838144700 1 -- 172.21.15.29:0/866002819 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8300cbb80 con 0x7fb830154750 2024-01-24T14:14:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.022+0000 7fb825ffb700 1 -- 172.21.15.29:0/866002819 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fb82804f130 con 0x7fb830154750 2024-01-24T14:14:21.022 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:21.023 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4083072158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":4083072158},{"type":"v1","addr":"172.21.15.163:6837","nonce":4083072158}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/4114592089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4114592089},{"type":"v1","addr":"172.21.15.163:6835","nonce":4114592089}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/926851304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":926851304},{"type":"v1","addr":"172.21.15.29:6835","nonce":926851304}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:13:59.576939+0000","modified":"2024-01-24T14:14:02.594306+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6833/3854229544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3854229544},{"type":"v1","addr":"172.21.15.29:6833","nonce":3854229544}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-24T14:14:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.024+0000 7fb838144700 1 -- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb81405f300 msgr2=0x7fb8140617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.024+0000 7fb838144700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb81405f300 0x7fb8140617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.024+0000 7fb838144700 1 -- 172.21.15.29:0/866002819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb830154750 msgr2=0x7fb830164ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.024+0000 7fb838144700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb830154750 0x7fb830164ed0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fb828009fc0 tx=0x7fb8280099b0).stop 2024-01-24T14:14:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.024+0000 7fb838144700 1 -- 172.21.15.29:0/866002819 shutdown_connections 2024-01-24T14:14:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.024+0000 7fb838144700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb81405f300 0x7fb8140617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.024+0000 7fb838144700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb830154750 0x7fb830164ed0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.024+0000 7fb838144700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8300c5410 0x7fb830154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.024+0000 7fb838144700 1 --2- 172.21.15.29:0/866002819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb830154370 0x7fb830155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.024+0000 7fb838144700 1 -- 172.21.15.29:0/866002819 >> 172.21.15.29:0/866002819 conn(0x7fb83005ec80 msgr2=0x7fb83014d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.024+0000 7fb838144700 1 -- 172.21.15.29:0/866002819 shutdown_connections 2024-01-24T14:14:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.024+0000 7fb838144700 1 -- 172.21.15.29:0/866002819 wait complete. 2024-01-24T14:14:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 43 2024-01-24T14:14:21.038 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:14:21.052 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:21.052+0000 7f0678022680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:14:21.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.186+0000 7f3cc89c8700 1 Processor -- start 2024-01-24T14:14:21.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.186+0000 7f3cc89c8700 1 -- start start 2024-01-24T14:14:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.187+0000 7f3cc89c8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cc4154370 0x7f3cc40c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.187+0000 7f3cc89c8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3cc40aa0e0 0x7f3cc4154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.187+0000 7f3cc89c8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc40aa6c0 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.187+0000 7f3cc89c8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc4064240 con 0x7f3cc40aa0e0 2024-01-24T14:14:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.187+0000 7f3cc89c8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40cb5c0 con 0x7f3cc4154370 2024-01-24T14:14:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.187+0000 7f3cc1d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3cc40aa0e0 0x7f3cc4154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.188+0000 7f3cc1d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3cc40aa0e0 0x7f3cc4154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52948/0 (socket says 172.21.15.29:52948) 2024-01-24T14:14:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.188+0000 7f3cc1d9b700 1 -- 172.21.15.29:0/4000019300 learned_addr learned my addr 172.21.15.29:0/4000019300 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.188+0000 7f3cc259c700 1 --2- 172.21.15.29:0/4000019300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cc4154370 0x7f3cc40c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.188+0000 7f3cc1d9b700 1 -- 172.21.15.29:0/4000019300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cc4154370 msgr2=0x7f3cc40c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.188+0000 7f3cc1d9b700 1 --2- 172.21.15.29:0/4000019300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cc4154370 0x7f3cc40c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.188+0000 7f3cc1d9b700 1 -- 172.21.15.29:0/4000019300 >> v1:172.21.15.163:6789/0 conn(0x7f3cc40ab1f0 legacy=0x7f3cc40631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.188+0000 7f3cc1d9b700 1 -- 172.21.15.29:0/4000019300 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc40cb8a0 con 0x7f3cc40aa0e0 2024-01-24T14:14:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.188+0000 7f3cc1d9b700 1 --2- 172.21.15.29:0/4000019300 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cc40aa0e0 0x7f3cc4154750 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f3cb8006fd0 tx=0x7f3cb80090f0).ready entity=mon.0 client_cookie=c57b96b0d3d93b93 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.188+0000 7f3cc159a700 1 -- 172.21.15.29:0/4000019300 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800b580 con 0x7f3cc40aa0e0 2024-01-24T14:14:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.188+0000 7f3cc159a700 1 -- 172.21.15.29:0/4000019300 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb800b6e0 con 0x7f3cc40aa0e0 2024-01-24T14:14:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.188+0000 7f3cc159a700 1 -- 172.21.15.29:0/4000019300 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800ba60 con 0x7f3cc40aa0e0 2024-01-24T14:14:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.189+0000 7f3cc89c8700 1 -- 172.21.15.29:0/4000019300 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cc40aa0e0 msgr2=0x7f3cc4154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.189+0000 7f3cc89c8700 1 --2- 172.21.15.29:0/4000019300 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cc40aa0e0 0x7f3cc4154750 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f3cb8006fd0 tx=0x7f3cb80090f0).stop 2024-01-24T14:14:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.189+0000 7f3cc89c8700 1 -- 172.21.15.29:0/4000019300 shutdown_connections 2024-01-24T14:14:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.189+0000 7f3cc89c8700 1 --2- 172.21.15.29:0/4000019300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cc4154370 0x7f3cc40c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.189+0000 7f3cc89c8700 1 --2- 172.21.15.29:0/4000019300 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cc40aa0e0 0x7f3cc4154750 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.189+0000 7f3cc89c8700 1 -- 172.21.15.29:0/4000019300 >> 172.21.15.29:0/4000019300 conn(0x7f3cc405ec80 msgr2=0x7f3cc40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:21.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.189+0000 7f3cc89c8700 1 -- 172.21.15.29:0/4000019300 shutdown_connections 2024-01-24T14:14:21.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.189+0000 7f3cc89c8700 1 -- 172.21.15.29:0/4000019300 wait complete. 2024-01-24T14:14:21.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.190+0000 7f3cc89c8700 1 Processor -- start 2024-01-24T14:14:21.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.190+0000 7f3cc89c8700 1 -- start start 2024-01-24T14:14:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.191+0000 7f3cc89c8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc40aa0e0 0x7f3cc40ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.191+0000 7f3cc89c8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cc40ab1f0 0x7f3cc4168150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.191+0000 7f3cc89c8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cc4154370 0x7f3cc40c2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.191+0000 7f3cc89c8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40af120 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.191+0000 7f3cc89c8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4154e80 con 0x7f3cc40aa0e0 2024-01-24T14:14:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.191+0000 7f3cc89c8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4155160 con 0x7f3cc4154370 2024-01-24T14:14:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.191+0000 7f3cc1d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cc40ab1f0 0x7f3cc4168150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.191+0000 7f3cc1d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cc40ab1f0 0x7f3cc4168150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52952/0 (socket says 172.21.15.29:52952) 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.191+0000 7f3cc1d9b700 1 -- 172.21.15.29:0/3290123497 learned_addr learned my addr 172.21.15.29:0/3290123497 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.191+0000 7f3cc259c700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc40aa0e0 0x7f3cc40ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.191+0000 7f3cc2d9d700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cc4154370 0x7f3cc40c2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.192+0000 7f3cc1d9b700 1 -- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cc4154370 msgr2=0x7f3cc40c2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.192+0000 7f3cc1d9b700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cc4154370 0x7f3cc40c2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.192+0000 7f3cc1d9b700 1 -- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc40aa0e0 msgr2=0x7f3cc40ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.192+0000 7f3cc1d9b700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc40aa0e0 0x7f3cc40ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.192+0000 7f3cc1d9b700 1 -- 172.21.15.29:0/3290123497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb8009370 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.192+0000 7f3cc259c700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc40aa0e0 0x7f3cc40ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.192+0000 7f3cc1d9b700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cc40ab1f0 0x7f3cc4168150 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f3cb8009130 tx=0x7f3cb8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.192+0000 7f3cb27fc700 1 -- 172.21.15.29:0/3290123497 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8000bc0 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.192+0000 7f3cb27fc700 1 -- 172.21.15.29:0/3290123497 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb800b850 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.192+0000 7f3cc89c8700 1 -- 172.21.15.29:0/3290123497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc4155440 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.192+0000 7f3cb27fc700 1 -- 172.21.15.29:0/3290123497 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8000d20 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.192+0000 7f3cc89c8700 1 -- 172.21.15.29:0/3290123497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc4168c20 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.194+0000 7f3cc89c8700 1 -- 172.21.15.29:0/3290123497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc4168e10 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.194+0000 7f3cb27fc700 1 -- 172.21.15.29:0/3290123497 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 36) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f3cb801cb10 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.194+0000 7f3cb27fc700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3ca405f2d0 0x7f3ca4061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.194+0000 7f3cb27fc700 1 -- 172.21.15.29:0/3290123497 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f3cb807c760 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.195+0000 7f3cc259c700 1 -- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3ca405f2d0 msgr2=0x7f3ca4061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:21.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.195+0000 7f3cc259c700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3ca405f2d0 0x7f3ca4061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:21.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.199+0000 7f3cb27fc700 1 -- 172.21.15.29:0/3290123497 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3cc4168e10 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.271 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:21.270+0000 7f0678022680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:14:21.378 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:21.377+0000 7f5349dda680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:14:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.384+0000 7f3cc89c8700 1 -- 172.21.15.29:0/3290123497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cc4168e10 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.385+0000 7f3cb27fc700 1 -- 172.21.15.29:0/3290123497 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+18557 (secure 0 0 0) 0x7f3cc4168e10 con 0x7f3cc40ab1f0 2024-01-24T14:14:21.386 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:21.386 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":106,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:14:16.078701+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:13:57.364021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-24T14:13:58.553943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3810716912":"2024-01-25T14:14:16.078667+0000","172.21.15.163:6833/43241":"2024-01-25T14:14:16.078667+0000","172.21.15.163:6832/43241":"2024-01-25T14:14:16.078667+0000","172.21.15.163:0/3154477280":"2024-01-25T14:14:16.078667+0000","172.21.15.163:0/1381714363":"2024-01-25T14:14:16.078667+0000","172.21.15.163:0/3261466368":"2024-01-25T14:14:16.078667+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:14:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.387+0000 7f3cc89c8700 1 -- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3ca405f2d0 msgr2=0x7f3ca4061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.387+0000 7f3cc89c8700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3ca405f2d0 0x7f3ca4061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.387+0000 7f3cc89c8700 1 -- 172.21.15.29:0/3290123497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cc40ab1f0 msgr2=0x7f3cc4168150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.387+0000 7f3cc89c8700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cc40ab1f0 0x7f3cc4168150 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f3cb8009130 tx=0x7f3cb8017000).stop 2024-01-24T14:14:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.387+0000 7f3cc89c8700 1 -- 172.21.15.29:0/3290123497 shutdown_connections 2024-01-24T14:14:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.387+0000 7f3cc89c8700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cc4154370 0x7f3cc40c2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.387+0000 7f3cc89c8700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc40aa0e0 0x7f3cc40ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.387+0000 7f3cc89c8700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3ca405f2d0 0x7f3ca4061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:21.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.387+0000 7f3cc89c8700 1 --2- 172.21.15.29:0/3290123497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cc40ab1f0 0x7f3cc4168150 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.387+0000 7f3cc89c8700 1 -- 172.21.15.29:0/3290123497 >> 172.21.15.29:0/3290123497 conn(0x7f3cc405ec80 msgr2=0x7f3cc4151810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:21.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.387+0000 7f3cc89c8700 1 -- 172.21.15.29:0/3290123497 shutdown_connections 2024-01-24T14:14:21.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.387+0000 7f3cc89c8700 1 -- 172.21.15.29:0/3290123497 wait complete. 2024-01-24T14:14:21.402 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T14:14:21.402 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:14:21.447 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:21.446+0000 7f5349dda680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:14:21.521 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:21.520+0000 7f5349dda680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:14:21.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.532+0000 7fb616c93700 1 Processor -- start 2024-01-24T14:14:21.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.533+0000 7fb616c93700 1 -- start start 2024-01-24T14:14:21.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.533+0000 7fb616c93700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb610150d40 0x7fb610069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.533+0000 7fb616c93700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb610065ec0 0x7fb610154930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.533+0000 7fb616c93700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb610154e20 0x7fb610151120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb616c93700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6100c2ce0 con 0x7fb610150d40 2024-01-24T14:14:21.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb616c93700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6101552b0 con 0x7fb610065ec0 2024-01-24T14:14:21.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb616c93700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb610069890 con 0x7fb610154e20 2024-01-24T14:14:21.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb614a2f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb610150d40 0x7fb610069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb614a2f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb610150d40 0x7fb610069240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52968/0 (socket says 172.21.15.29:52968) 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb614a2f700 1 -- 172.21.15.29:0/3019012573 learned_addr learned my addr 172.21.15.29:0/3019012573 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb615230700 1 --2- 172.21.15.29:0/3019012573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb610154e20 0x7fb610151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb60ffff700 1 --2- 172.21.15.29:0/3019012573 >> v2:172.21.15.163:3300/0 conn(0x7fb610065ec0 0x7fb610154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb614a2f700 1 -- 172.21.15.29:0/3019012573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb610154e20 msgr2=0x7fb610151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb614a2f700 1 --2- 172.21.15.29:0/3019012573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb610154e20 0x7fb610151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb614a2f700 1 -- 172.21.15.29:0/3019012573 >> v2:172.21.15.163:3300/0 conn(0x7fb610065ec0 msgr2=0x7fb610154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb614a2f700 1 --2- 172.21.15.29:0/3019012573 >> v2:172.21.15.163:3300/0 conn(0x7fb610065ec0 0x7fb610154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb614a2f700 1 -- 172.21.15.29:0/3019012573 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb610069b70 con 0x7fb610150d40 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb614a2f700 1 --2- 172.21.15.29:0/3019012573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb610150d40 0x7fb610069240 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7fb610068cd0 tx=0x7fb6040090f0).ready entity=mon.0 client_cookie=ee4fafb229f4a609 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb60f7fe700 1 -- 172.21.15.29:0/3019012573 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60400f510 con 0x7fb610150d40 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.534+0000 7fb60f7fe700 1 -- 172.21.15.29:0/3019012573 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60400f670 con 0x7fb610150d40 2024-01-24T14:14:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.535+0000 7fb60f7fe700 1 -- 172.21.15.29:0/3019012573 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60400f9f0 con 0x7fb610150d40 2024-01-24T14:14:21.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.535+0000 7fb616c93700 1 -- 172.21.15.29:0/3019012573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb610150d40 msgr2=0x7fb610069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.535+0000 7fb616c93700 1 --2- 172.21.15.29:0/3019012573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb610150d40 0x7fb610069240 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7fb610068cd0 tx=0x7fb6040090f0).stop 2024-01-24T14:14:21.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.535+0000 7fb616c93700 1 -- 172.21.15.29:0/3019012573 shutdown_connections 2024-01-24T14:14:21.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.535+0000 7fb616c93700 1 --2- 172.21.15.29:0/3019012573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb610154e20 0x7fb610151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.535+0000 7fb616c93700 1 --2- 172.21.15.29:0/3019012573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb610150d40 0x7fb610069240 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.535+0000 7fb616c93700 1 --2- 172.21.15.29:0/3019012573 >> v2:172.21.15.163:3300/0 conn(0x7fb610065ec0 0x7fb610154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.535+0000 7fb616c93700 1 -- 172.21.15.29:0/3019012573 >> 172.21.15.29:0/3019012573 conn(0x7fb6100c8580 msgr2=0x7fb6100c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:21.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.535+0000 7fb616c93700 1 -- 172.21.15.29:0/3019012573 shutdown_connections 2024-01-24T14:14:21.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.535+0000 7fb616c93700 1 -- 172.21.15.29:0/3019012573 wait complete. 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.536+0000 7fb616c93700 1 Processor -- start 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.537+0000 7fb616c93700 1 -- start start 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.537+0000 7fb616c93700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb610065ec0 0x7fb61014e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.537+0000 7fb616c93700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb610150d40 0x7fb61013f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.537+0000 7fb616c93700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb610154e20 0x7fb6100ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.537+0000 7fb616c93700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61014c5f0 con 0x7fb610154e20 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.537+0000 7fb616c93700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61014e290 con 0x7fb610150d40 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.537+0000 7fb616c93700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61014eb70 con 0x7fb610065ec0 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb615230700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb610154e20 0x7fb6100ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb614a2f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb610065ec0 0x7fb61014e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb615230700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb610154e20 0x7fb6100ab2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52976/0 (socket says 172.21.15.29:52976) 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb615230700 1 -- 172.21.15.29:0/2500273103 learned_addr learned my addr 172.21.15.29:0/2500273103 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb60ffff700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb610150d40 0x7fb61013f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb615230700 1 -- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb610065ec0 msgr2=0x7fb61014e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb615230700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb610065ec0 0x7fb61014e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb615230700 1 -- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb610150d40 msgr2=0x7fb61013f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb615230700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb610150d40 0x7fb61013f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb615230700 1 -- 172.21.15.29:0/2500273103 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb604009370 con 0x7fb610154e20 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb615230700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb610154e20 0x7fb6100ab2d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7fb608009df0 tx=0x7fb60800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb60cff9700 1 -- 172.21.15.29:0/2500273103 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60800caa0 con 0x7fb610154e20 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb60cff9700 1 -- 172.21.15.29:0/2500273103 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60800f040 con 0x7fb610154e20 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb616c93700 1 -- 172.21.15.29:0/2500273103 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6100c00d0 con 0x7fb610154e20 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb60cff9700 1 -- 172.21.15.29:0/2500273103 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60800ce10 con 0x7fb610154e20 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.538+0000 7fb616c93700 1 -- 172.21.15.29:0/2500273103 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6100c0690 con 0x7fb610154e20 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.539+0000 7fb616c93700 1 -- 172.21.15.29:0/2500273103 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6100475e0 con 0x7fb610154e20 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.540+0000 7fb60cff9700 1 -- 172.21.15.29:0/2500273103 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 36) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fb608013610 con 0x7fb610154e20 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.540+0000 7fb60cff9700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5f405f300 0x7fb5f40617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.540+0000 7fb60cff9700 1 -- 172.21.15.29:0/2500273103 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fb60807b1a0 con 0x7fb610154e20 2024-01-24T14:14:21.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.540+0000 7fb614a2f700 1 -- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5f405f300 msgr2=0x7fb5f40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:21.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.540+0000 7fb614a2f700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5f405f300 0x7fb5f40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:21.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.545+0000 7fb60cff9700 1 -- 172.21.15.29:0/2500273103 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb60804bc60 con 0x7fb610154e20 2024-01-24T14:14:21.671 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:21.670+0000 7f5349dda680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:14:21.741 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:21.740+0000 7f5349dda680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:14:21.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.741+0000 7fb614a2f700 1 -- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5f405f300 msgr2=0x7fb5f40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:21.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.741+0000 7fb614a2f700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5f405f300 0x7fb5f40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:14:21.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.770+0000 7fb616c93700 1 -- 172.21.15.29:0/2500273103 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb61006b350 con 0x7fb610154e20 2024-01-24T14:14:21.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.771+0000 7fb60cff9700 1 -- 172.21.15.29:0/2500273103 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fb60804f280 con 0x7fb610154e20 2024-01-24T14:14:21.771 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:21.771 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4083072158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":4083072158},{"type":"v1","addr":"172.21.15.163:6837","nonce":4083072158}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/4114592089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4114592089},{"type":"v1","addr":"172.21.15.163:6835","nonce":4114592089}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/926851304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":926851304},{"type":"v1","addr":"172.21.15.29:6835","nonce":926851304}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:13:59.576939+0000","modified":"2024-01-24T14:14:02.594306+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6833/3854229544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3854229544},{"type":"v1","addr":"172.21.15.29:6833","nonce":3854229544}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-24T14:14:21.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.773+0000 7fb616c93700 1 -- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5f405f300 msgr2=0x7fb5f40617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:21.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.773+0000 7fb616c93700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5f405f300 0x7fb5f40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:21.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.773+0000 7fb616c93700 1 -- 172.21.15.29:0/2500273103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb610154e20 msgr2=0x7fb6100ab2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.773+0000 7fb616c93700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb610154e20 0x7fb6100ab2d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7fb608009df0 tx=0x7fb60800b040).stop 2024-01-24T14:14:21.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.773+0000 7fb616c93700 1 -- 172.21.15.29:0/2500273103 shutdown_connections 2024-01-24T14:14:21.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.773+0000 7fb616c93700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5f405f300 0x7fb5f40617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:21.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.773+0000 7fb616c93700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb610154e20 0x7fb6100ab2d0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.773+0000 7fb616c93700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb610065ec0 0x7fb61014e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.773+0000 7fb616c93700 1 --2- 172.21.15.29:0/2500273103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb610150d40 0x7fb61013f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.773+0000 7fb616c93700 1 -- 172.21.15.29:0/2500273103 >> 172.21.15.29:0/2500273103 conn(0x7fb6100c8580 msgr2=0x7fb6100627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:21.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.773+0000 7fb616c93700 1 -- 172.21.15.29:0/2500273103 shutdown_connections 2024-01-24T14:14:21.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.773+0000 7fb616c93700 1 -- 172.21.15.29:0/2500273103 wait complete. 2024-01-24T14:14:21.775 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 43 2024-01-24T14:14:21.788 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:14:21.857 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:21.856+0000 7f5349dda680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:14:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.921+0000 7f10a50c5700 1 Processor -- start 2024-01-24T14:14:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.921+0000 7f10a50c5700 1 -- start start 2024-01-24T14:14:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.922+0000 7f10a50c5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10a0154370 0x7f10a00c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.922+0000 7f10a50c5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f10a00aa0e0 0x7f10a0154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.922+0000 7f10a50c5700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a00aa6c0 con 0x7f10a00ab1f0 2024-01-24T14:14:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.922+0000 7f10a50c5700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a0064240 con 0x7f10a00aa0e0 2024-01-24T14:14:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.922+0000 7f10a50c5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00cb5c0 con 0x7f10a0154370 2024-01-24T14:14:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109f59e700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f10a00ab1f0 0x7f10a00631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35338/0 (socket says 172.21.15.29:35338) 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109f59e700 1 -- 172.21.15.29:0/506567770 learned_addr learned my addr 172.21.15.29:0/506567770 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109ed9d700 1 --2- 172.21.15.29:0/506567770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10a0154370 0x7f10a00c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109e59c700 1 --2- 172.21.15.29:0/506567770 >> v2:172.21.15.163:3300/0 conn(0x7f10a00aa0e0 0x7f10a0154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109dd9b700 1 -- 172.21.15.29:0/506567770 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809607418 0 0) 0x7f10a00aa6c0 con 0x7f10a00ab1f0 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109dd9b700 1 -- 172.21.15.29:0/506567770 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10880033f0 con 0x7f10a00ab1f0 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109e59c700 1 -- 172.21.15.29:0/506567770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10a0154370 msgr2=0x7f10a00c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109e59c700 1 --2- 172.21.15.29:0/506567770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10a0154370 0x7f10a00c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109e59c700 1 -- 172.21.15.29:0/506567770 >> v1:172.21.15.29:6789/0 conn(0x7f10a00ab1f0 legacy=0x7f10a00631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109e59c700 1 -- 172.21.15.29:0/506567770 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a00cb8a0 con 0x7f10a00aa0e0 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109ed9d700 1 --2- 172.21.15.29:0/506567770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10a0154370 0x7f10a00c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109e59c700 1 --2- 172.21.15.29:0/506567770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10a00aa0e0 0x7f10a0154750 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f1094009760 tx=0x7f1094009060).ready entity=mon.1 client_cookie=8666a2aa5eea588b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109dd9b700 1 -- 172.21.15.29:0/506567770 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109400d4a0 con 0x7f10a00aa0e0 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109dd9b700 1 -- 172.21.15.29:0/506567770 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109400d600 con 0x7f10a00aa0e0 2024-01-24T14:14:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.923+0000 7f109dd9b700 1 -- 172.21.15.29:0/506567770 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109400d980 con 0x7f10a00aa0e0 2024-01-24T14:14:21.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.924+0000 7f10a50c5700 1 -- 172.21.15.29:0/506567770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10a00aa0e0 msgr2=0x7f10a0154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.924+0000 7f10a50c5700 1 --2- 172.21.15.29:0/506567770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10a00aa0e0 0x7f10a0154750 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f1094009760 tx=0x7f1094009060).stop 2024-01-24T14:14:21.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.924+0000 7f10a50c5700 1 -- 172.21.15.29:0/506567770 shutdown_connections 2024-01-24T14:14:21.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.924+0000 7f10a50c5700 1 --2- 172.21.15.29:0/506567770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10a0154370 0x7f10a00c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.924+0000 7f10a50c5700 1 --2- 172.21.15.29:0/506567770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10a00aa0e0 0x7f10a0154750 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.924+0000 7f10a50c5700 1 -- 172.21.15.29:0/506567770 >> 172.21.15.29:0/506567770 conn(0x7f10a005ec80 msgr2=0x7f10a00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:21.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.924+0000 7f10a50c5700 1 -- 172.21.15.29:0/506567770 shutdown_connections 2024-01-24T14:14:21.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.924+0000 7f10a50c5700 1 -- 172.21.15.29:0/506567770 wait complete. 2024-01-24T14:14:21.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.926+0000 7f10a50c5700 1 Processor -- start 2024-01-24T14:14:21.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.926+0000 7f10a50c5700 1 -- start start 2024-01-24T14:14:21.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f10a50c5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10a00aa0e0 0x7f10a00c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f10a50c5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10a00ab1f0 0x7f10a00aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f10a50c5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10a0154370 0x7f10a00af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f10a50c5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a014c420 con 0x7f10a00ab1f0 2024-01-24T14:14:21.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f10a50c5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00ad750 con 0x7f10a00aa0e0 2024-01-24T14:14:21.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f10a50c5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00ca930 con 0x7f10a0154370 2024-01-24T14:14:21.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f109e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10a00ab1f0 0x7f10a00aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f109e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10a00ab1f0 0x7f10a00aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52986/0 (socket says 172.21.15.29:52986) 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f109e59c700 1 -- 172.21.15.29:0/2762615050 learned_addr learned my addr 172.21.15.29:0/2762615050 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f109f59e700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10a0154370 0x7f10a00af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f109ed9d700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10a00aa0e0 0x7f10a00c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f109ed9d700 1 -- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10a0154370 msgr2=0x7f10a00af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f109ed9d700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10a0154370 0x7f10a00af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f109ed9d700 1 -- 172.21.15.29:0/2762615050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10a00ab1f0 msgr2=0x7f10a00aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f109ed9d700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10a00ab1f0 0x7f10a00aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f109ed9d700 1 -- 172.21.15.29:0/2762615050 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1090009250 con 0x7f10a00aa0e0 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f109e59c700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10a00ab1f0 0x7f10a00aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.927+0000 7f109ed9d700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10a00aa0e0 0x7f10a00c8eb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f109000e950 tx=0x7f1090006f40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.928+0000 7f1086ffd700 1 -- 172.21.15.29:0/2762615050 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090009560 con 0x7f10a00aa0e0 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.928+0000 7f1086ffd700 1 -- 172.21.15.29:0/2762615050 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1090005880 con 0x7f10a00aa0e0 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.928+0000 7f1086ffd700 1 -- 172.21.15.29:0/2762615050 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090005c00 con 0x7f10a00aa0e0 2024-01-24T14:14:21.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.928+0000 7f10a50c5700 1 -- 172.21.15.29:0/2762615050 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10940092a0 con 0x7f10a00aa0e0 2024-01-24T14:14:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.928+0000 7f10a50c5700 1 -- 172.21.15.29:0/2762615050 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a00c9e50 con 0x7f10a00aa0e0 2024-01-24T14:14:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.929+0000 7f10a50c5700 1 -- 172.21.15.29:0/2762615050 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a00ca780 con 0x7f10a00aa0e0 2024-01-24T14:14:21.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.929+0000 7f1086ffd700 1 -- 172.21.15.29:0/2762615050 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 36) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f1090006020 con 0x7f10a00aa0e0 2024-01-24T14:14:21.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.929+0000 7f1086ffd700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f108c05f2d0 0x7f108c061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:21.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.930+0000 7f1086ffd700 1 -- 172.21.15.29:0/2762615050 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f109007a930 con 0x7f10a00aa0e0 2024-01-24T14:14:21.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.930+0000 7f109e59c700 1 -- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f108c05f2d0 msgr2=0x7f108c061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:21.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.930+0000 7f109e59c700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f108c05f2d0 0x7f108c061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:21.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:21.934+0000 7f1086ffd700 1 -- 172.21.15.29:0/2762615050 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f109004eed0 con 0x7f10a00aa0e0 2024-01-24T14:14:21.967 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:21.966+0000 7f5349dda680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:14:22.051 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:22.050+0000 7f0678022680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:14:22.130 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:22.130+0000 7f0678022680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:14:22.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.131+0000 7f109e59c700 1 -- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f108c05f2d0 msgr2=0x7f108c061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:22.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.131+0000 7f109e59c700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f108c05f2d0 0x7f108c061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:14:22.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.162+0000 7f10a50c5700 1 -- 172.21.15.29:0/2762615050 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10a00475e0 con 0x7f10a00aa0e0 2024-01-24T14:14:22.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.163+0000 7f1086ffd700 1 -- 172.21.15.29:0/2762615050 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f1090016030 con 0x7f10a00aa0e0 2024-01-24T14:14:22.164 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:22.164 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4083072158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":4083072158},{"type":"v1","addr":"172.21.15.163:6837","nonce":4083072158}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/4114592089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4114592089},{"type":"v1","addr":"172.21.15.163:6835","nonce":4114592089}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/926851304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":926851304},{"type":"v1","addr":"172.21.15.29:6835","nonce":926851304}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:13:59.576939+0000","modified":"2024-01-24T14:14:02.594306+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6833/3854229544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3854229544},{"type":"v1","addr":"172.21.15.29:6833","nonce":3854229544}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-24T14:14:22.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.165+0000 7f10a50c5700 1 -- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f108c05f2d0 msgr2=0x7f108c061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:22.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.165+0000 7f10a50c5700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f108c05f2d0 0x7f108c061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.165+0000 7f10a50c5700 1 -- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10a00aa0e0 msgr2=0x7f10a00c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.165+0000 7f10a50c5700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10a00aa0e0 0x7f10a00c8eb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f109000e950 tx=0x7f1090006f40).stop 2024-01-24T14:14:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.165+0000 7f10a50c5700 1 -- 172.21.15.29:0/2762615050 shutdown_connections 2024-01-24T14:14:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.165+0000 7f10a50c5700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10a0154370 0x7f10a00af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.165+0000 7f10a50c5700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10a00aa0e0 0x7f10a00c8eb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.165+0000 7f10a50c5700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f108c05f2d0 0x7f108c061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.165+0000 7f10a50c5700 1 --2- 172.21.15.29:0/2762615050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10a00ab1f0 0x7f10a00aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.165+0000 7f10a50c5700 1 -- 172.21.15.29:0/2762615050 >> 172.21.15.29:0/2762615050 conn(0x7f10a005ec80 msgr2=0x7f10a015aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.165+0000 7f10a50c5700 1 -- 172.21.15.29:0/2762615050 shutdown_connections 2024-01-24T14:14:22.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.166+0000 7f10a50c5700 1 -- 172.21.15.29:0/2762615050 wait complete. 2024-01-24T14:14:22.168 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 43 2024-01-24T14:14:22.180 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:14:22.220 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:22.219+0000 7f0678022680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:14:22.258 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:22.257+0000 7f5349dda680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:14:22.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.311+0000 7f19534e6700 1 Processor -- start 2024-01-24T14:14:22.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.311+0000 7f19534e6700 1 -- start start 2024-01-24T14:14:22.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.312+0000 7f19534e6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f194c0c5410 0x7f194c0aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:22.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.312+0000 7f19534e6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f194c154750 0x7f194c0631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:22.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.312+0000 7f19534e6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c0aa6c0 con 0x7f194c154370 2024-01-24T14:14:22.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.312+0000 7f19534e6700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c064240 con 0x7f194c154750 2024-01-24T14:14:22.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.312+0000 7f19534e6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0cb5c0 con 0x7f194c0c5410 2024-01-24T14:14:22.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.312+0000 7f1951a83700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f194c154750 0x7f194c0631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:22.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.312+0000 7f1951a83700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f194c154750 0x7f194c0631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52990/0 (socket says 172.21.15.29:52990) 2024-01-24T14:14:22.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.312+0000 7f1951a83700 1 -- 172.21.15.29:0/2873401714 learned_addr learned my addr 172.21.15.29:0/2873401714 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:22.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.313+0000 7f1950a81700 1 --2- 172.21.15.29:0/2873401714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f194c0c5410 0x7f194c0aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:22.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.313+0000 7f1951a83700 1 -- 172.21.15.29:0/2873401714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f194c0c5410 msgr2=0x7f194c0aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:22.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.313+0000 7f1951a83700 1 --2- 172.21.15.29:0/2873401714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f194c0c5410 0x7f194c0aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.313+0000 7f1951a83700 1 -- 172.21.15.29:0/2873401714 >> v1:172.21.15.163:6789/0 conn(0x7f194c154370 legacy=0x7f194c0ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:22.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.313+0000 7f1951a83700 1 -- 172.21.15.29:0/2873401714 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0cb8a0 con 0x7f194c154750 2024-01-24T14:14:22.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.313+0000 7f1943fff700 1 -- 172.21.15.29:0/2873401714 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831974595 0 0) 0x7f194c0aa6c0 con 0x7f194c154370 2024-01-24T14:14:22.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.313+0000 7f1951a83700 1 --2- 172.21.15.29:0/2873401714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f194c154750 0x7f194c0631d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f1944009f20 tx=0x7f1944009840).ready entity=mon.0 client_cookie=ef05a6c0fbfabe1a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:22.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.313+0000 7f1943fff700 1 -- 172.21.15.29:0/2873401714 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194400e070 con 0x7f194c154750 2024-01-24T14:14:22.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.313+0000 7f1943fff700 1 -- 172.21.15.29:0/2873401714 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194400b9d0 con 0x7f194c154750 2024-01-24T14:14:22.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.313+0000 7f1943fff700 1 -- 172.21.15.29:0/2873401714 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194400bd50 con 0x7f194c154750 2024-01-24T14:14:22.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.314+0000 7f19534e6700 1 -- 172.21.15.29:0/2873401714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f194c154750 msgr2=0x7f194c0631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:22.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.314+0000 7f19534e6700 1 --2- 172.21.15.29:0/2873401714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f194c154750 0x7f194c0631d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f1944009f20 tx=0x7f1944009840).stop 2024-01-24T14:14:22.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.314+0000 7f19534e6700 1 -- 172.21.15.29:0/2873401714 shutdown_connections 2024-01-24T14:14:22.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.314+0000 7f19534e6700 1 --2- 172.21.15.29:0/2873401714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f194c154750 0x7f194c0631d0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.314+0000 7f19534e6700 1 --2- 172.21.15.29:0/2873401714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f194c0c5410 0x7f194c0aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.314+0000 7f19534e6700 1 -- 172.21.15.29:0/2873401714 >> 172.21.15.29:0/2873401714 conn(0x7f194c05ec80 msgr2=0x7f194c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:22.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.314+0000 7f19534e6700 1 -- 172.21.15.29:0/2873401714 shutdown_connections 2024-01-24T14:14:22.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.314+0000 7f19534e6700 1 -- 172.21.15.29:0/2873401714 wait complete. 2024-01-24T14:14:22.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.315+0000 7f19534e6700 1 Processor -- start 2024-01-24T14:14:22.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.315+0000 7f19534e6700 1 -- start start 2024-01-24T14:14:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.316+0000 7f19534e6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f194c0c5410 0x7f194c146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.316+0000 7f19534e6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f194c154370 0x7f194c0c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.316+0000 7f19534e6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f194c154750 0x7f194c0c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.316+0000 7f19534e6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0c2ad0 con 0x7f194c154750 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.316+0000 7f19534e6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0c3020 con 0x7f194c154370 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.316+0000 7f19534e6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0c4010 con 0x7f194c0c5410 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.316+0000 7f1951a83700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f194c154750 0x7f194c0c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.316+0000 7f1951a83700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f194c154750 0x7f194c0c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52992/0 (socket says 172.21.15.29:52992) 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.316+0000 7f1951a83700 1 -- 172.21.15.29:0/3036115186 learned_addr learned my addr 172.21.15.29:0/3036115186 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.316+0000 7f1951282700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f194c0c5410 0x7f194c146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.316+0000 7f1950a81700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f194c154370 0x7f194c0c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.317+0000 7f1951a83700 1 -- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f194c0c5410 msgr2=0x7f194c146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.317+0000 7f1951a83700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f194c0c5410 0x7f194c146e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.317+0000 7f1951a83700 1 -- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f194c154370 msgr2=0x7f194c0c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.317+0000 7f1951a83700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f194c154370 0x7f194c0c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.317+0000 7f1951a83700 1 -- 172.21.15.29:0/3036115186 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1944009ae0 con 0x7f194c154750 2024-01-24T14:14:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.317+0000 7f1951a83700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f194c154750 0x7f194c0c3530 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f194400b610 tx=0x7f1944009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:22.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.317+0000 7f19417fa700 1 -- 172.21.15.29:0/3036115186 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194400bd50 con 0x7f194c154750 2024-01-24T14:14:22.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.317+0000 7f19417fa700 1 -- 172.21.15.29:0/3036115186 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19440092e0 con 0x7f194c154750 2024-01-24T14:14:22.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.317+0000 7f19534e6700 1 -- 172.21.15.29:0/3036115186 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c0c42f0 con 0x7f194c154750 2024-01-24T14:14:22.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.317+0000 7f19417fa700 1 -- 172.21.15.29:0/3036115186 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194401d6b0 con 0x7f194c154750 2024-01-24T14:14:22.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.317+0000 7f19534e6700 1 -- 172.21.15.29:0/3036115186 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c0c3a20 con 0x7f194c154750 2024-01-24T14:14:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.318+0000 7f19534e6700 1 -- 172.21.15.29:0/3036115186 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c0475e0 con 0x7f194c154750 2024-01-24T14:14:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.319+0000 7f19417fa700 1 -- 172.21.15.29:0/3036115186 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 36) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f1944003040 con 0x7f194c154750 2024-01-24T14:14:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.319+0000 7f19417fa700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f193005f2d0 0x7f1930061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.319+0000 7f19417fa700 1 -- 172.21.15.29:0/3036115186 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f194407bbd0 con 0x7f194c154750 2024-01-24T14:14:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.319+0000 7f1951282700 1 -- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f193005f2d0 msgr2=0x7f1930061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.319+0000 7f1951282700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f193005f2d0 0x7f1930061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:22.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.323+0000 7f19417fa700 1 -- 172.21.15.29:0/3036115186 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f194404c740 con 0x7f194c154750 2024-01-24T14:14:22.329 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:22.328+0000 7f5349dda680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:14:22.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.351+0000 7f19417fa700 1 -- 172.21.15.29:0/3036115186 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 37) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f1944079190 con 0x7f194c154750 2024-01-24T14:14:22.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.351+0000 7f19417fa700 1 -- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f193005f2d0 msgr2=0x7f1930061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:22.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.351+0000 7f19417fa700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f193005f2d0 0x7f1930061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:22.399 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:22.398+0000 7f52c5b60700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:22.400 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:22.398+0000 7f52c5b60700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:22.400 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:22.398+0000 7f52c5b60700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:22.400 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:22.398+0000 7f52c5b60700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:22.400 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:22.398+0000 7f52c5b60700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:22.400 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:22.399+0000 7f52c2359700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:22.400 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:22.399+0000 7f52c2359700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:22.400 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:22.399+0000 7f52c2359700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:22.400 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:22.399+0000 7f52c2359700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:22.400 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:22.399+0000 7f52c2359700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:22.401 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:22.399+0000 7f0678022680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:14:22.483 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:22.483+0000 7f0678022680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:14:22.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.515+0000 7f19534e6700 1 -- 172.21.15.29:0/3036115186 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f194c0cbb80 con 0x7f194c154750 2024-01-24T14:14:22.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.516+0000 7f19417fa700 1 -- 172.21.15.29:0/3036115186 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+18870 (secure 0 0 0) 0x7f19440309d0 con 0x7f194c154750 2024-01-24T14:14:22.517 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:22.517 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":107,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:14:22.344610+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:13:57.364021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-24T14:13:58.553943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/1800281818":"2024-01-25T14:14:22.344573+0000","172.21.15.163:0/746108515":"2024-01-25T14:14:22.344573+0000","172.21.15.163:0/2204115969":"2024-01-25T14:14:22.344573+0000","172.21.15.163:0/3810716912":"2024-01-25T14:14:16.078667+0000","172.21.15.163:0/3695077808":"2024-01-25T14:14:22.344573+0000","172.21.15.163:0/778512655":"2024-01-25T14:14:22.344573+0000","172.21.15.163:6833/43241":"2024-01-25T14:14:16.078667+0000","172.21.15.163:6832/43241":"2024-01-25T14:14:16.078667+0000","172.21.15.163:0/3154477280":"2024-01-25T14:14:16.078667+0000","172.21.15.163:0/1381714363":"2024-01-25T14:14:16.078667+0000","172.21.15.163:0/3261466368":"2024-01-25T14:14:16.078667+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:14:22.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.518+0000 7f19534e6700 1 -- 172.21.15.29:0/3036115186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f194c154750 msgr2=0x7f194c0c3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:22.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.518+0000 7f19534e6700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f194c154750 0x7f194c0c3530 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f194400b610 tx=0x7f1944009840).stop 2024-01-24T14:14:22.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.518+0000 7f19534e6700 1 -- 172.21.15.29:0/3036115186 shutdown_connections 2024-01-24T14:14:22.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.518+0000 7f19534e6700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f193005f2d0 0x7f1930061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:22.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.519+0000 7f19534e6700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f194c154750 0x7f194c0c3530 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.519+0000 7f19534e6700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f194c0c5410 0x7f194c146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.519+0000 7f19534e6700 1 --2- 172.21.15.29:0/3036115186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f194c154370 0x7f194c0c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.519+0000 7f19534e6700 1 -- 172.21.15.29:0/3036115186 >> 172.21.15.29:0/3036115186 conn(0x7f194c05ec80 msgr2=0x7f194c0ad480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:22.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.519+0000 7f19534e6700 1 -- 172.21.15.29:0/3036115186 shutdown_connections 2024-01-24T14:14:22.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.519+0000 7f19534e6700 1 -- 172.21.15.29:0/3036115186 wait complete. 2024-01-24T14:14:22.533 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-24T14:14:22.622 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:22.622+0000 7f0678022680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:14:22.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.706+0000 7f805e286700 1 Processor -- start 2024-01-24T14:14:22.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.706+0000 7f805e286700 1 -- start start 2024-01-24T14:14:22.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.707+0000 7f805e286700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8058158920 0x7f80580664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:22.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.707+0000 7f805e286700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8058155290 con 0x7f8058066990 2024-01-24T14:14:22.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.707+0000 7f805e286700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8058066e20 con 0x7f80580bf6c0 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.707+0000 7f805e286700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8058065160 con 0x7f8058158920 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.707+0000 7f805c823700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8058066990 0x7f80580bdb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35342/0 (socket says 172.21.15.29:35342) 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.707+0000 7f805c823700 1 -- 172.21.15.29:0/3837789305 learned_addr learned my addr 172.21.15.29:0/3837789305 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.707+0000 7f80577fe700 1 --2- 172.21.15.29:0/3837789305 >> v2:172.21.15.163:3300/0 conn(0x7f8058158920 0x7f80580664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.707+0000 7f8056ffd700 1 -- 172.21.15.29:0/3837789305 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482512303 0 0) 0x7f8058155290 con 0x7f8058066990 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.707+0000 7f8056ffd700 1 -- 172.21.15.29:0/3837789305 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80400033f0 con 0x7f8058066990 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.708+0000 7f80577fe700 1 -- 172.21.15.29:0/3837789305 >> v1:172.21.15.163:6789/0 conn(0x7f80580bf6c0 legacy=0x7f80580638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.708+0000 7f8056ffd700 1 -- 172.21.15.29:0/3837789305 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3212681776 0 0) 0x7f80400033f0 con 0x7f8058066990 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.708+0000 7f80577fe700 1 -- 172.21.15.29:0/3837789305 >> v1:172.21.15.29:6789/0 conn(0x7f8058066990 legacy=0x7f80580bdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.708+0000 7f80577fe700 1 -- 172.21.15.29:0/3837789305 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8058155720 con 0x7f8058158920 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.708+0000 7f80577fe700 1 --2- 172.21.15.29:0/3837789305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8058158920 0x7f80580664a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f804c009760 tx=0x7f804c009060).ready entity=mon.1 client_cookie=91cef9f40fbe7da3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.708+0000 7f8056ffd700 1 -- 172.21.15.29:0/3837789305 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80500017f0 con 0x7f8058158920 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.708+0000 7f8056ffd700 1 -- 172.21.15.29:0/3837789305 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c012410 con 0x7f8058158920 2024-01-24T14:14:22.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.708+0000 7f8056ffd700 1 -- 172.21.15.29:0/3837789305 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c012790 con 0x7f8058158920 2024-01-24T14:14:22.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.708+0000 7f805e286700 1 -- 172.21.15.29:0/3837789305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8058158920 msgr2=0x7f80580664a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.708+0000 7f805e286700 1 --2- 172.21.15.29:0/3837789305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8058158920 0x7f80580664a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f804c009760 tx=0x7f804c009060).stop 2024-01-24T14:14:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.709+0000 7f805e286700 1 -- 172.21.15.29:0/3837789305 shutdown_connections 2024-01-24T14:14:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.709+0000 7f805e286700 1 --2- 172.21.15.29:0/3837789305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8058158920 0x7f80580664a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.709+0000 7f805e286700 1 -- 172.21.15.29:0/3837789305 >> 172.21.15.29:0/3837789305 conn(0x7f80580c5640 msgr2=0x7f80580c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.709+0000 7f805e286700 1 -- 172.21.15.29:0/3837789305 shutdown_connections 2024-01-24T14:14:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.709+0000 7f805e286700 1 -- 172.21.15.29:0/3837789305 wait complete. 2024-01-24T14:14:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.710+0000 7f805e286700 1 Processor -- start 2024-01-24T14:14:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.711+0000 7f805e286700 1 -- start start 2024-01-24T14:14:22.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f805e286700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8058066990 0x7f80580a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:22.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f805e286700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80580bf6c0 0x7f80580aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:22.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f805e286700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8058158920 0x7f80580a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f805e286700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80580a8200 con 0x7f80580bf6c0 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f805e286700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80580a7f20 con 0x7f8058158920 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f805e286700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80580aa510 con 0x7f8058066990 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f80577fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80580bf6c0 0x7f80580aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f80577fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80580bf6c0 0x7f80580aaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53006/0 (socket says 172.21.15.29:53006) 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f80577fe700 1 -- 172.21.15.29:0/116740818 learned_addr learned my addr 172.21.15.29:0/116740818 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f805c823700 1 --2- 172.21.15.29:0/116740818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8058158920 0x7f80580a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f8057fff700 1 --2- 172.21.15.29:0/116740818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8058066990 0x7f80580a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f80577fe700 1 -- 172.21.15.29:0/116740818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8058066990 msgr2=0x7f80580a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f80577fe700 1 --2- 172.21.15.29:0/116740818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8058066990 0x7f80580a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f80577fe700 1 -- 172.21.15.29:0/116740818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8058158920 msgr2=0x7f80580a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f80577fe700 1 --2- 172.21.15.29:0/116740818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8058158920 0x7f80580a74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f80577fe700 1 -- 172.21.15.29:0/116740818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c0092a0 con 0x7f80580bf6c0 2024-01-24T14:14:22.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f805c823700 1 --2- 172.21.15.29:0/116740818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8058158920 0x7f80580a74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:22.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f80577fe700 1 --2- 172.21.15.29:0/116740818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80580bf6c0 0x7f80580aaff0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f804c009e00 tx=0x7f804c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:22.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.712+0000 7f803ffff700 1 -- 172.21.15.29:0/116740818 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c011950 con 0x7f80580bf6c0 2024-01-24T14:14:22.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.713+0000 7f803ffff700 1 -- 172.21.15.29:0/116740818 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c011ab0 con 0x7f80580bf6c0 2024-01-24T14:14:22.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.713+0000 7f805e286700 1 -- 172.21.15.29:0/116740818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80580aa7f0 con 0x7f80580bf6c0 2024-01-24T14:14:22.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.713+0000 7f803ffff700 1 -- 172.21.15.29:0/116740818 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c011e50 con 0x7f80580bf6c0 2024-01-24T14:14:22.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.713+0000 7f805e286700 1 -- 172.21.15.29:0/116740818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80580a5f20 con 0x7f80580bf6c0 2024-01-24T14:14:22.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.714+0000 7f805e286700 1 -- 172.21.15.29:0/116740818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80580aa370 con 0x7f80580bf6c0 2024-01-24T14:14:22.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.714+0000 7f803ffff700 1 -- 172.21.15.29:0/116740818 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 37) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f804c01d600 con 0x7f80580bf6c0 2024-01-24T14:14:22.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.715+0000 7f803ffff700 1 -- 172.21.15.29:0/116740818 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7615+0+0 (secure 0 0 0) 0x7f804c07bda0 con 0x7f80580bf6c0 2024-01-24T14:14:22.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.719+0000 7f803ffff700 1 -- 172.21.15.29:0/116740818 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f80580aa370 con 0x7f80580bf6c0 2024-01-24T14:14:22.750 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:22.749+0000 7f0678022680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:14:22.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:22.944+0000 7f805e286700 1 -- 172.21.15.29:0/116740818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f80580aad10 con 0x7f80580bf6c0 2024-01-24T14:14:23.089 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:23.088+0000 7f0678022680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:14:23.173 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:23.172+0000 7f0678022680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:14:23.346 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-01-24T14:14:23.346+0000 7fe09bfc3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-24T14:14:23.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.349+0000 7f803ffff700 1 -- 172.21.15.29:0/116740818 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7f80580aad10 con 0x7f80580bf6c0 2024-01-24T14:14:23.352 INFO:tasks.ceph.mds.a.smithi029.stderr: -1380> 2024-01-24T14:14:05.143+0000 7fa2fdff7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T14:14:23.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.352+0000 7f805e286700 1 -- 172.21.15.29:0/116740818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80580bf6c0 msgr2=0x7f80580aaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:23.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.352+0000 7f805e286700 1 --2- 172.21.15.29:0/116740818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80580bf6c0 0x7f80580aaff0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f804c009e00 tx=0x7f804c015000).stop 2024-01-24T14:14:23.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.352+0000 7f805e286700 1 -- 172.21.15.29:0/116740818 shutdown_connections 2024-01-24T14:14:23.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.352+0000 7f805e286700 1 --2- 172.21.15.29:0/116740818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8058158920 0x7f80580a74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:23.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.352+0000 7f805e286700 1 --2- 172.21.15.29:0/116740818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8058066990 0x7f80580a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:23.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.352+0000 7f805e286700 1 --2- 172.21.15.29:0/116740818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80580bf6c0 0x7f80580aaff0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:23.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.352+0000 7f805e286700 1 -- 172.21.15.29:0/116740818 >> 172.21.15.29:0/116740818 conn(0x7f80580c5640 msgr2=0x7f8058155c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:23.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.352+0000 7f805e286700 1 -- 172.21.15.29:0/116740818 shutdown_connections 2024-01-24T14:14:23.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.352+0000 7f805e286700 1 -- 172.21.15.29:0/116740818 wait complete. 2024-01-24T14:14:23.354 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-24T14:14:23.371 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-24T14:14:23.380 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T14:14:23.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.503+0000 7fa76fca1700 1 Processor -- start 2024-01-24T14:14:23.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.503+0000 7fa76fca1700 1 -- start start 2024-01-24T14:14:23.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.504+0000 7fa76fca1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa76814b330 0x7fa7681549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:23.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.504+0000 7fa76fca1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa7680c3360 0x7fa768154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:23.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.504+0000 7fa76fca1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa768155840 con 0x7fa76814b710 2024-01-24T14:14:23.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.504+0000 7fa76fca1700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa768155390 con 0x7fa7680c3360 2024-01-24T14:14:23.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.504+0000 7fa76fca1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa768069280 con 0x7fa76814b330 2024-01-24T14:14:23.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.504+0000 7fa76e23e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa7680c3360 0x7fa768154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.504+0000 7fa76e23e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa7680c3360 0x7fa768154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49804/0 (socket says 172.21.15.29:49804) 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.504+0000 7fa76e23e700 1 -- 172.21.15.29:0/3578948855 learned_addr learned my addr 172.21.15.29:0/3578948855 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.504+0000 7fa76da3d700 1 --2- 172.21.15.29:0/3578948855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa76814b330 0x7fa7681549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76e23e700 1 -- 172.21.15.29:0/3578948855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa76814b330 msgr2=0x7fa7681549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76e23e700 1 --2- 172.21.15.29:0/3578948855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa76814b330 0x7fa7681549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76e23e700 1 -- 172.21.15.29:0/3578948855 >> v1:172.21.15.163:6789/0 conn(0x7fa76814b710 legacy=0x7fa7680c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76e23e700 1 -- 172.21.15.29:0/3578948855 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7680c3910 con 0x7fa7680c3360 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76e23e700 1 --2- 172.21.15.29:0/3578948855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7680c3360 0x7fa768154ea0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7fa760009f20 tx=0x7fa760009840).ready entity=mon.0 client_cookie=5d0050c00b6afd7e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76ca3b700 1 -- 172.21.15.29:0/3578948855 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76000e070 con 0x7fa7680c3360 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76ca3b700 1 -- 172.21.15.29:0/3578948855 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76000ba80 con 0x7fa7680c3360 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76ca3b700 1 -- 172.21.15.29:0/3578948855 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76000be00 con 0x7fa7680c3360 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76fca1700 1 -- 172.21.15.29:0/3578948855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7680c3360 msgr2=0x7fa768154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76fca1700 1 --2- 172.21.15.29:0/3578948855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7680c3360 0x7fa768154ea0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7fa760009f20 tx=0x7fa760009840).stop 2024-01-24T14:14:23.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76fca1700 1 -- 172.21.15.29:0/3578948855 shutdown_connections 2024-01-24T14:14:23.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76fca1700 1 --2- 172.21.15.29:0/3578948855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa76814b330 0x7fa7681549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:23.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76fca1700 1 --2- 172.21.15.29:0/3578948855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7680c3360 0x7fa768154ea0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:23.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.505+0000 7fa76fca1700 1 -- 172.21.15.29:0/3578948855 >> 172.21.15.29:0/3578948855 conn(0x7fa768064390 msgr2=0x7fa7680c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:23.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.506+0000 7fa76fca1700 1 -- 172.21.15.29:0/3578948855 shutdown_connections 2024-01-24T14:14:23.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.506+0000 7fa76fca1700 1 -- 172.21.15.29:0/3578948855 wait complete. 2024-01-24T14:14:23.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.507+0000 7fa76fca1700 1 Processor -- start 2024-01-24T14:14:23.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.507+0000 7fa76fca1700 1 -- start start 2024-01-24T14:14:23.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76fca1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7680c3360 0x7fa768152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:23.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76fca1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76814b330 0x7fa768152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:23.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76fca1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76814b710 0x7fa768145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:23.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76fca1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7681539a0 con 0x7fa76814b330 2024-01-24T14:14:23.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76fca1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa768154440 con 0x7fa76814b710 2024-01-24T14:14:23.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76fca1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa768140b90 con 0x7fa7680c3360 2024-01-24T14:14:23.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76d23c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76814b330 0x7fa768152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:23.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76d23c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76814b330 0x7fa768152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49812/0 (socket says 172.21.15.29:49812) 2024-01-24T14:14:23.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76d23c700 1 -- 172.21.15.29:0/3033498908 learned_addr learned my addr 172.21.15.29:0/3033498908 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:23.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76e23e700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76814b710 0x7fa768145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:23.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76da3d700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7680c3360 0x7fa768152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76d23c700 1 -- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7680c3360 msgr2=0x7fa768152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76d23c700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7680c3360 0x7fa768152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76d23c700 1 -- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76814b710 msgr2=0x7fa768145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76d23c700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76814b710 0x7fa768145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76d23c700 1 -- 172.21.15.29:0/3033498908 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa760009ae0 con 0x7fa76814b330 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76e23e700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76814b710 0x7fa768145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76d23c700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76814b330 0x7fa768152a80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7fa75c00c440 tx=0x7fa75c007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa759ffb700 1 -- 172.21.15.29:0/3033498908 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00ce90 con 0x7fa76814b330 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa759ffb700 1 -- 172.21.15.29:0/3033498908 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c007020 con 0x7fa76814b330 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa759ffb700 1 -- 172.21.15.29:0/3033498908 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0073c0 con 0x7fa76814b330 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.508+0000 7fa76fca1700 1 -- 172.21.15.29:0/3033498908 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7680abce0 con 0x7fa76814b330 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.509+0000 7fa76fca1700 1 -- 172.21.15.29:0/3033498908 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7681533e0 con 0x7fa76814b330 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.510+0000 7fa76fca1700 1 -- 172.21.15.29:0/3033498908 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7680c1ec0 con 0x7fa76814b330 2024-01-24T14:14:23.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.510+0000 7fa759ffb700 1 -- 172.21.15.29:0/3033498908 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 38) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fa75c0068b0 con 0x7fa76814b330 2024-01-24T14:14:23.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.510+0000 7fa759ffb700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa74c05f030 0x7fa74c0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:23.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.510+0000 7fa759ffb700 1 -- 172.21.15.29:0/3033498908 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7701+0+0 (secure 0 0 0) 0x7fa75c079fc0 con 0x7fa76814b330 2024-01-24T14:14:23.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.511+0000 7fa76da3d700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa74c05f030 0x7fa74c0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:23.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.511+0000 7fa76da3d700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa74c05f030 0x7fa74c0614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fa76400bb20 tx=0x7fa764008040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:23.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.515+0000 7fa759ffb700 1 -- 172.21.15.29:0/3033498908 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa75c04ae30 con 0x7fa76814b330 2024-01-24T14:14:23.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:23.748+0000 7fa76fca1700 1 -- 172.21.15.29:0/3033498908 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa7680475e0 con 0x7fa76814b330 2024-01-24T14:14:24.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.362+0000 7fa759ffb700 1 -- 172.21.15.29:0/3033498908 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa75c04e450 con 0x7fa76814b330 2024-01-24T14:14:24.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.365+0000 7fa76fca1700 1 -- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa74c05f030 msgr2=0x7fa74c0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:24.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.365+0000 7fa76fca1700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa74c05f030 0x7fa74c0614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fa76400bb20 tx=0x7fa764008040).stop 2024-01-24T14:14:24.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.365+0000 7fa76fca1700 1 -- 172.21.15.29:0/3033498908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76814b330 msgr2=0x7fa768152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:24.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.365+0000 7fa76fca1700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76814b330 0x7fa768152a80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7fa75c00c440 tx=0x7fa75c007da0).stop 2024-01-24T14:14:24.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.365+0000 7fa76fca1700 1 -- 172.21.15.29:0/3033498908 shutdown_connections 2024-01-24T14:14:24.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.365+0000 7fa76fca1700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76814b710 0x7fa768145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:24.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.365+0000 7fa76fca1700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7680c3360 0x7fa768152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:24.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.365+0000 7fa76fca1700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa74c05f030 0x7fa74c0614f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:24.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.365+0000 7fa76fca1700 1 --2- 172.21.15.29:0/3033498908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa76814b330 0x7fa768152a80 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:24.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.365+0000 7fa76fca1700 1 -- 172.21.15.29:0/3033498908 >> 172.21.15.29:0/3033498908 conn(0x7fa768064390 msgr2=0x7fa7680c58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:24.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.365+0000 7fa76fca1700 1 -- 172.21.15.29:0/3033498908 shutdown_connections 2024-01-24T14:14:24.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.365+0000 7fa76fca1700 1 -- 172.21.15.29:0/3033498908 wait complete. 2024-01-24T14:14:24.382 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-24T14:14:24.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.565+0000 7f785ea2e700 1 Processor -- start 2024-01-24T14:14:24.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.565+0000 7f785ea2e700 1 -- start start 2024-01-24T14:14:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.566+0000 7f785ea2e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7858066b60 0x7f78580c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.566+0000 7f785ea2e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f78580684c0 0x7f7858153090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.566+0000 7f785ea2e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7858067a70 con 0x7f785814fd10 2024-01-24T14:14:24.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.566+0000 7f785ea2e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78580c3d50 con 0x7f78580684c0 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.566+0000 7f785ea2e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78581500f0 con 0x7f7858066b60 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.566+0000 7f785cfcb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f78580684c0 0x7f7858153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.566+0000 7f7857fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f785814fd10 0x7f7858066450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40330/0 (socket says 172.21.15.29:40330) 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.566+0000 7f7857fff700 1 -- 172.21.15.29:0/3140045884 learned_addr learned my addr 172.21.15.29:0/3140045884 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.567+0000 7f78577fe700 1 --2- 172.21.15.29:0/3140045884 >> v2:172.21.15.163:3300/0 conn(0x7f7858066b60 0x7f78580c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.567+0000 7f785cfcb700 1 -- 172.21.15.29:0/3140045884 >> v2:172.21.15.163:3300/0 conn(0x7f7858066b60 msgr2=0x7f78580c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.567+0000 7f785cfcb700 1 --2- 172.21.15.29:0/3140045884 >> v2:172.21.15.163:3300/0 conn(0x7f7858066b60 0x7f78580c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.567+0000 7f785cfcb700 1 -- 172.21.15.29:0/3140045884 >> v1:172.21.15.29:6789/0 conn(0x7f785814fd10 legacy=0x7f7858066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.567+0000 7f785cfcb700 1 -- 172.21.15.29:0/3140045884 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7858153580 con 0x7f78580684c0 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.567+0000 7f7856ffd700 1 -- 172.21.15.29:0/3140045884 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909133783 0 0) 0x7f7858067a70 con 0x7f785814fd10 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.567+0000 7f78577fe700 1 --2- 172.21.15.29:0/3140045884 >> v2:172.21.15.163:3300/0 conn(0x7f7858066b60 0x7f78580c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.567+0000 7f785cfcb700 1 --2- 172.21.15.29:0/3140045884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78580684c0 0x7f7858153090 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f78580c3180 tx=0x7f7850009780).ready entity=mon.0 client_cookie=1574cc007827914e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.567+0000 7f7856ffd700 1 -- 172.21.15.29:0/3140045884 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785000e070 con 0x7f78580684c0 2024-01-24T14:14:24.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.567+0000 7f7856ffd700 1 -- 172.21.15.29:0/3140045884 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785000bc30 con 0x7f78580684c0 2024-01-24T14:14:24.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.567+0000 7f7856ffd700 1 -- 172.21.15.29:0/3140045884 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7850012410 con 0x7f78580684c0 2024-01-24T14:14:24.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.568+0000 7f785ea2e700 1 -- 172.21.15.29:0/3140045884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78580684c0 msgr2=0x7f7858153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:24.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.568+0000 7f785ea2e700 1 --2- 172.21.15.29:0/3140045884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78580684c0 0x7f7858153090 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f78580c3180 tx=0x7f7850009780).stop 2024-01-24T14:14:24.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.568+0000 7f785ea2e700 1 -- 172.21.15.29:0/3140045884 shutdown_connections 2024-01-24T14:14:24.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.568+0000 7f785ea2e700 1 --2- 172.21.15.29:0/3140045884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78580684c0 0x7f7858153090 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:24.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.568+0000 7f785ea2e700 1 --2- 172.21.15.29:0/3140045884 >> v2:172.21.15.163:3300/0 conn(0x7f7858066b60 0x7f78580c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:24.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.568+0000 7f785ea2e700 1 -- 172.21.15.29:0/3140045884 >> 172.21.15.29:0/3140045884 conn(0x7f78580c8bd0 msgr2=0x7f78580c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:24.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.568+0000 7f785ea2e700 1 -- 172.21.15.29:0/3140045884 shutdown_connections 2024-01-24T14:14:24.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.568+0000 7f785ea2e700 1 -- 172.21.15.29:0/3140045884 wait complete. 2024-01-24T14:14:24.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.569+0000 7f785ea2e700 1 Processor -- start 2024-01-24T14:14:24.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.569+0000 7f785ea2e700 1 -- start start 2024-01-24T14:14:24.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.570+0000 7f785ea2e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7858066b60 0x7f785813ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:24.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.570+0000 7f785ea2e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78580684c0 0x7f78580c1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:24.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.570+0000 7f785ea2e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785814fd10 0x7f78581518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:24.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.570+0000 7f785ea2e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785814d200 con 0x7f7858066b60 2024-01-24T14:14:24.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.570+0000 7f785ea2e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7858063af0 con 0x7f78580684c0 2024-01-24T14:14:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.570+0000 7f785ea2e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785813bce0 con 0x7f785814fd10 2024-01-24T14:14:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.570+0000 7f7857fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7858066b60 0x7f785813ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.570+0000 7f7857fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7858066b60 0x7f785813ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49836/0 (socket says 172.21.15.29:49836) 2024-01-24T14:14:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.570+0000 7f7857fff700 1 -- 172.21.15.29:0/2906352838 learned_addr learned my addr 172.21.15.29:0/2906352838 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.570+0000 7f78577fe700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78580684c0 0x7f78580c1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.570+0000 7f785cfcb700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785814fd10 0x7f78581518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.571+0000 7f7857fff700 1 -- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785814fd10 msgr2=0x7f78581518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.571+0000 7f7857fff700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785814fd10 0x7f78581518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.571+0000 7f7857fff700 1 -- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78580684c0 msgr2=0x7f78580c1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.571+0000 7f7857fff700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78580684c0 0x7f78580c1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.571+0000 7f7857fff700 1 -- 172.21.15.29:0/2906352838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7850009a20 con 0x7f7858066b60 2024-01-24T14:14:24.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.571+0000 7f7857fff700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7858066b60 0x7f785813ac10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f784000c460 tx=0x7f7840007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:24.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.571+0000 7f784ffff700 1 -- 172.21.15.29:0/2906352838 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400070d0 con 0x7f7858066b60 2024-01-24T14:14:24.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.571+0000 7f784ffff700 1 -- 172.21.15.29:0/2906352838 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7840007230 con 0x7f7858066b60 2024-01-24T14:14:24.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.571+0000 7f785ea2e700 1 -- 172.21.15.29:0/2906352838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78580ab1d0 con 0x7f7858066b60 2024-01-24T14:14:24.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.571+0000 7f784ffff700 1 -- 172.21.15.29:0/2906352838 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400075b0 con 0x7f7858066b60 2024-01-24T14:14:24.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.571+0000 7f785ea2e700 1 -- 172.21.15.29:0/2906352838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785813b720 con 0x7f7858066b60 2024-01-24T14:14:24.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.572+0000 7f785ea2e700 1 -- 172.21.15.29:0/2906352838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7858067530 con 0x7f7858066b60 2024-01-24T14:14:24.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.573+0000 7f784ffff700 1 -- 172.21.15.29:0/2906352838 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 38) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f78400059b0 con 0x7f7858066b60 2024-01-24T14:14:24.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.573+0000 7f784ffff700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f783805f000 0x7f78380614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:24.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.573+0000 7f784ffff700 1 -- 172.21.15.29:0/2906352838 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7701+0+0 (secure 0 0 0) 0x7f7840079ce0 con 0x7f7858066b60 2024-01-24T14:14:24.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.573+0000 7f78577fe700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f783805f000 0x7f78380614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:24.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.574+0000 7f78577fe700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f783805f000 0x7f78380614c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f7858065aa0 tx=0x7f7848008040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:24.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.577+0000 7f784ffff700 1 -- 172.21.15.29:0/2906352838 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7858067530 con 0x7f7858066b60 2024-01-24T14:14:24.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:24.761+0000 7f785ea2e700 1 -- 172.21.15.29:0/2906352838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7858067530 con 0x7f7858066b60 2024-01-24T14:14:25.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.360+0000 7f784ffff700 1 -- 172.21.15.29:0/2906352838 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7840077650 con 0x7f7858066b60 2024-01-24T14:14:25.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.374+0000 7f784ffff700 1 -- 172.21.15.29:0/2906352838 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7840075f00 con 0x7f7858066b60 2024-01-24T14:14:25.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.378+0000 7f785ea2e700 1 -- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f783805f000 msgr2=0x7f78380614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:25.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.378+0000 7f785ea2e700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f783805f000 0x7f78380614c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f7858065aa0 tx=0x7f7848008040).stop 2024-01-24T14:14:25.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.378+0000 7f785ea2e700 1 -- 172.21.15.29:0/2906352838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7858066b60 msgr2=0x7f785813ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:25.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.378+0000 7f785ea2e700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7858066b60 0x7f785813ac10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f784000c460 tx=0x7f7840007e90).stop 2024-01-24T14:14:25.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.378+0000 7f785ea2e700 1 -- 172.21.15.29:0/2906352838 shutdown_connections 2024-01-24T14:14:25.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.378+0000 7f785ea2e700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785814fd10 0x7f78581518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:25.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.378+0000 7f785ea2e700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f783805f000 0x7f78380614c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:25.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.378+0000 7f785ea2e700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7858066b60 0x7f785813ac10 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:25.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.378+0000 7f785ea2e700 1 --2- 172.21.15.29:0/2906352838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78580684c0 0x7f78580c1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:25.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.378+0000 7f785ea2e700 1 -- 172.21.15.29:0/2906352838 >> 172.21.15.29:0/2906352838 conn(0x7f78580c8bd0 msgr2=0x7f785814e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:25.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.379+0000 7f785ea2e700 1 -- 172.21.15.29:0/2906352838 shutdown_connections 2024-01-24T14:14:25.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.379+0000 7f785ea2e700 1 -- 172.21.15.29:0/2906352838 wait complete. 2024-01-24T14:14:25.381 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-01-24T14:14:25.398 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-24T14:14:25.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.535+0000 7fa10ae71700 1 Processor -- start 2024-01-24T14:14:25.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.535+0000 7fa10ae71700 1 -- start start 2024-01-24T14:14:25.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa10ae71700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa104066b60 0x7fa1040c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:25.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa10ae71700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1040684c0 0x7fa104153090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:25.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa10ae71700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa104067a70 con 0x7fa10414fd10 2024-01-24T14:14:25.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa10ae71700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1040c3d50 con 0x7fa104066b60 2024-01-24T14:14:25.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa10ae71700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1041500f0 con 0x7fa1040684c0 2024-01-24T14:14:25.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa103fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa104066b60 0x7fa1040c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa10940e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1040684c0 0x7fa104153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa108c0d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa10414fd10 0x7fa104066450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:48380/0 (socket says 172.21.15.29:48380) 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa108c0d700 1 -- 172.21.15.29:0/2104387130 learned_addr learned my addr 172.21.15.29:0/2104387130 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa10940e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1040684c0 0x7fa104153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:34524/0 (socket says 172.21.15.29:34524) 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa103fff700 1 -- 172.21.15.29:0/2104387130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1040684c0 msgr2=0x7fa104153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa103fff700 1 --2- 172.21.15.29:0/2104387130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1040684c0 0x7fa104153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa103fff700 1 -- 172.21.15.29:0/2104387130 >> v1:172.21.15.163:6789/0 conn(0x7fa10414fd10 legacy=0x7fa104066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa103fff700 1 -- 172.21.15.29:0/2104387130 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa104153580 con 0x7fa104066b60 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.536+0000 7fa1037fe700 1 -- 172.21.15.29:0/2104387130 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988148031 0 0) 0x7fa104067a70 con 0x7fa10414fd10 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.537+0000 7fa103fff700 1 --2- 172.21.15.29:0/2104387130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa104066b60 0x7fa1040c31d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fa0f0009760 tx=0x7fa0f0009060).ready entity=mon.1 client_cookie=6b269bcbbae4e82 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.537+0000 7fa1037fe700 1 -- 172.21.15.29:0/2104387130 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f000f510 con 0x7fa104066b60 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.537+0000 7fa1037fe700 1 -- 172.21.15.29:0/2104387130 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f000f670 con 0x7fa104066b60 2024-01-24T14:14:25.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.537+0000 7fa1037fe700 1 -- 172.21.15.29:0/2104387130 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f000f9f0 con 0x7fa104066b60 2024-01-24T14:14:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.537+0000 7fa10ae71700 1 -- 172.21.15.29:0/2104387130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa104066b60 msgr2=0x7fa1040c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:25.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.537+0000 7fa10ae71700 1 --2- 172.21.15.29:0/2104387130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa104066b60 0x7fa1040c31d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fa0f0009760 tx=0x7fa0f0009060).stop 2024-01-24T14:14:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.537+0000 7fa10ae71700 1 -- 172.21.15.29:0/2104387130 shutdown_connections 2024-01-24T14:14:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.537+0000 7fa10ae71700 1 --2- 172.21.15.29:0/2104387130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1040684c0 0x7fa104153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.537+0000 7fa10ae71700 1 --2- 172.21.15.29:0/2104387130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa104066b60 0x7fa1040c31d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.537+0000 7fa10ae71700 1 -- 172.21.15.29:0/2104387130 >> 172.21.15.29:0/2104387130 conn(0x7fa1040c8bd0 msgr2=0x7fa1040c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.537+0000 7fa10ae71700 1 -- 172.21.15.29:0/2104387130 shutdown_connections 2024-01-24T14:14:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.537+0000 7fa10ae71700 1 -- 172.21.15.29:0/2104387130 wait complete. 2024-01-24T14:14:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.539+0000 7fa10ae71700 1 Processor -- start 2024-01-24T14:14:25.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.539+0000 7fa10ae71700 1 -- start start 2024-01-24T14:14:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa10ae71700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa104066b60 0x7fa1040ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa10ae71700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1040684c0 0x7fa1040c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa10ae71700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa10414fd10 0x7fa104064b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa10ae71700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040c1150 con 0x7fa104066b60 2024-01-24T14:14:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa10ae71700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040c5ae0 con 0x7fa1040684c0 2024-01-24T14:14:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa10ae71700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa104065440 con 0x7fa10414fd10 2024-01-24T14:14:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa108c0d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa104066b60 0x7fa1040ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:25.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa108c0d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa104066b60 0x7fa1040ab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49846/0 (socket says 172.21.15.29:49846) 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa108c0d700 1 -- 172.21.15.29:0/3979500531 learned_addr learned my addr 172.21.15.29:0/3979500531 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa10940e700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa10414fd10 0x7fa104064b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa103fff700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1040684c0 0x7fa1040c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa108c0d700 1 -- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa10414fd10 msgr2=0x7fa104064b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa108c0d700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa10414fd10 0x7fa104064b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa108c0d700 1 -- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1040684c0 msgr2=0x7fa1040c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa108c0d700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1040684c0 0x7fa1040c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa108c0d700 1 -- 172.21.15.29:0/3979500531 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00092a0 con 0x7fa104066b60 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.540+0000 7fa108c0d700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa104066b60 0x7fa1040ab0c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7fa0f8001eb0 tx=0x7fa0f8007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.541+0000 7fa100ff9700 1 -- 172.21.15.29:0/3979500531 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80070d0 con 0x7fa104066b60 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.541+0000 7fa100ff9700 1 -- 172.21.15.29:0/3979500531 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f8007230 con 0x7fa104066b60 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.541+0000 7fa100ff9700 1 -- 172.21.15.29:0/3979500531 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80075d0 con 0x7fa104066b60 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.541+0000 7fa10ae71700 1 -- 172.21.15.29:0/3979500531 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1040c13f0 con 0x7fa104066b60 2024-01-24T14:14:25.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.541+0000 7fa10ae71700 1 -- 172.21.15.29:0/3979500531 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1040c19b0 con 0x7fa104066b60 2024-01-24T14:14:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.542+0000 7fa10ae71700 1 -- 172.21.15.29:0/3979500531 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1040680f0 con 0x7fa104066b60 2024-01-24T14:14:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.543+0000 7fa100ff9700 1 -- 172.21.15.29:0/3979500531 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa0f8005bc0 con 0x7fa104066b60 2024-01-24T14:14:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.543+0000 7fa100ff9700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa0e805f270 0x7fa0e8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:25.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.543+0000 7fa100ff9700 1 -- 172.21.15.29:0/3979500531 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7263+0+0 (secure 0 0 0) 0x7fa0f807a610 con 0x7fa104066b60 2024-01-24T14:14:25.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.543+0000 7fa103fff700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa0e805f270 0x7fa0e8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:25.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.544+0000 7fa103fff700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa0e805f270 0x7fa0e8061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fa104066f60 tx=0x7fa0f0017000).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:25.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.547+0000 7fa100ff9700 1 -- 172.21.15.29:0/3979500531 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa0f804b3b0 con 0x7fa104066b60 2024-01-24T14:14:25.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:25.728+0000 7fa10ae71700 1 -- 172.21.15.29:0/3979500531 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa10406ae00 con 0x7fa104066b60 2024-01-24T14:14:26.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.374+0000 7fa100ff9700 1 -- 172.21.15.29:0/3979500531 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa0f804e9d0 con 0x7fa104066b60 2024-01-24T14:14:26.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.378+0000 7fa10ae71700 1 -- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa0e805f270 msgr2=0x7fa0e8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:26.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.378+0000 7fa10ae71700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa0e805f270 0x7fa0e8061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fa104066f60 tx=0x7fa0f0017000).stop 2024-01-24T14:14:26.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.378+0000 7fa10ae71700 1 -- 172.21.15.29:0/3979500531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa104066b60 msgr2=0x7fa1040ab0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:26.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.378+0000 7fa10ae71700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa104066b60 0x7fa1040ab0c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7fa0f8001eb0 tx=0x7fa0f8007e90).stop 2024-01-24T14:14:26.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.378+0000 7fa10ae71700 1 -- 172.21.15.29:0/3979500531 shutdown_connections 2024-01-24T14:14:26.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.378+0000 7fa10ae71700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa10414fd10 0x7fa104064b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.378+0000 7fa10ae71700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa0e805f270 0x7fa0e8061730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.378+0000 7fa10ae71700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa104066b60 0x7fa1040ab0c0 secure :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0x7fa0f8001eb0 tx=0x7fa0f8007e90).stop 2024-01-24T14:14:26.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.378+0000 7fa10ae71700 1 --2- 172.21.15.29:0/3979500531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1040684c0 0x7fa1040c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.378+0000 7fa10ae71700 1 -- 172.21.15.29:0/3979500531 >> 172.21.15.29:0/3979500531 conn(0x7fa1040c8bd0 msgr2=0x7fa10414d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:26.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.379+0000 7fa10ae71700 1 -- 172.21.15.29:0/3979500531 shutdown_connections 2024-01-24T14:14:26.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.379+0000 7fa10ae71700 1 -- 172.21.15.29:0/3979500531 wait complete. 2024-01-24T14:14:26.389 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-01-24T14:14:26.403 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.539+0000 7f263567f700 1 Processor -- start 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.539+0000 7f263567f700 1 -- start start 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.540+0000 7f263567f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2630155160 0x7f26300627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.540+0000 7f263567f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2630062ca0 0x7f2630069280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.540+0000 7f263567f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2630154cd0 con 0x7f263014b140 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.540+0000 7f263567f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26301555c0 con 0x7f2630155160 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.540+0000 7f263567f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2630063130 con 0x7f2630062ca0 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.541+0000 7f262e7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2630155160 0x7f26300627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.541+0000 7f262e7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2630155160 0x7f26300627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49850/0 (socket says 172.21.15.29:49850) 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.541+0000 7f262e7fc700 1 -- 172.21.15.29:0/3998835248 learned_addr learned my addr 172.21.15.29:0/3998835248 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.541+0000 7f262f7fe700 1 --2- 172.21.15.29:0/3998835248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2630062ca0 0x7f2630069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.541+0000 7f262e7fc700 1 -- 172.21.15.29:0/3998835248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2630062ca0 msgr2=0x7f2630069280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:26.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.541+0000 7f262e7fc700 1 --2- 172.21.15.29:0/3998835248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2630062ca0 0x7f2630069280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.541+0000 7f262e7fc700 1 -- 172.21.15.29:0/3998835248 >> v1:172.21.15.163:6789/0 conn(0x7f263014b140 legacy=0x7f26300bca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.541+0000 7f262e7fc700 1 -- 172.21.15.29:0/3998835248 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26301546a0 con 0x7f2630155160 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.541+0000 7f262e7fc700 1 --2- 172.21.15.29:0/3998835248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2630155160 0x7f26300627b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f2618006fd0 tx=0x7f26180090f0).ready entity=mon.0 client_cookie=cb36308778848a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.541+0000 7f262dffb700 1 -- 172.21.15.29:0/3998835248 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261800b600 con 0x7f2630155160 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.541+0000 7f262dffb700 1 -- 172.21.15.29:0/3998835248 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261800b760 con 0x7f2630155160 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.541+0000 7f262dffb700 1 -- 172.21.15.29:0/3998835248 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261800bae0 con 0x7f2630155160 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.542+0000 7f263567f700 1 -- 172.21.15.29:0/3998835248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2630155160 msgr2=0x7f26300627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.542+0000 7f263567f700 1 --2- 172.21.15.29:0/3998835248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2630155160 0x7f26300627b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f2618006fd0 tx=0x7f26180090f0).stop 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.542+0000 7f263567f700 1 -- 172.21.15.29:0/3998835248 shutdown_connections 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.542+0000 7f263567f700 1 --2- 172.21.15.29:0/3998835248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2630062ca0 0x7f2630069280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.542+0000 7f263567f700 1 --2- 172.21.15.29:0/3998835248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2630155160 0x7f26300627b0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.542+0000 7f263567f700 1 -- 172.21.15.29:0/3998835248 >> 172.21.15.29:0/3998835248 conn(0x7f2630064390 msgr2=0x7f26300c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.542+0000 7f263567f700 1 -- 172.21.15.29:0/3998835248 shutdown_connections 2024-01-24T14:14:26.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.542+0000 7f263567f700 1 -- 172.21.15.29:0/3998835248 wait complete. 2024-01-24T14:14:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.543+0000 7f263567f700 1 Processor -- start 2024-01-24T14:14:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.543+0000 7f263567f700 1 -- start start 2024-01-24T14:14:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.544+0000 7f263567f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2630062ca0 0x7f2630148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.544+0000 7f263567f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f263014b140 0x7f26300a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.544+0000 7f263567f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2630155160 0x7f26300a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.544+0000 7f263567f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26300a6230 con 0x7f263014b140 2024-01-24T14:14:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.544+0000 7f263567f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2630156160 con 0x7f2630062ca0 2024-01-24T14:14:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.544+0000 7f263567f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2630155870 con 0x7f2630155160 2024-01-24T14:14:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.544+0000 7f262e7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f263014b140 0x7f26300a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.544+0000 7f262e7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f263014b140 0x7f26300a5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49858/0 (socket says 172.21.15.29:49858) 2024-01-24T14:14:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.544+0000 7f262e7fc700 1 -- 172.21.15.29:0/1413553326 learned_addr learned my addr 172.21.15.29:0/1413553326 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:26.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.544+0000 7f262effd700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2630062ca0 0x7f2630148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.544+0000 7f262f7fe700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2630155160 0x7f26300a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.545+0000 7f262e7fc700 1 -- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2630155160 msgr2=0x7f26300a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.545+0000 7f262e7fc700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2630155160 0x7f26300a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.545+0000 7f262e7fc700 1 -- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2630062ca0 msgr2=0x7f2630148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.545+0000 7f262e7fc700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2630062ca0 0x7f2630148770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.545+0000 7f262e7fc700 1 -- 172.21.15.29:0/1413553326 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2618009370 con 0x7f263014b140 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.545+0000 7f262e7fc700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f263014b140 0x7f26300a5850 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f263005ee00 tx=0x7f2618017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.545+0000 7f2626ffd700 1 -- 172.21.15.29:0/1413553326 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261800bea0 con 0x7f263014b140 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.545+0000 7f2626ffd700 1 -- 172.21.15.29:0/1413553326 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261800b8d0 con 0x7f263014b140 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.545+0000 7f2626ffd700 1 -- 172.21.15.29:0/1413553326 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261801c670 con 0x7f263014b140 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.545+0000 7f263567f700 1 -- 172.21.15.29:0/1413553326 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2630155b10 con 0x7f263014b140 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.545+0000 7f263567f700 1 -- 172.21.15.29:0/1413553326 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2630148c60 con 0x7f263014b140 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.546+0000 7f263567f700 1 -- 172.21.15.29:0/1413553326 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26301562f0 con 0x7f263014b140 2024-01-24T14:14:26.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.546+0000 7f2626ffd700 1 -- 172.21.15.29:0/1413553326 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f261801c9e0 con 0x7f263014b140 2024-01-24T14:14:26.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.547+0000 7f2626ffd700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f261005f270 0x7f2610061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.547+0000 7f2626ffd700 1 -- 172.21.15.29:0/1413553326 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6849+0+0 (secure 0 0 0) 0x7f261807c480 con 0x7f263014b140 2024-01-24T14:14:26.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.547+0000 7f262effd700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f261005f270 0x7f2610061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.548+0000 7f262effd700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f261005f270 0x7f2610061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f26200038b0 tx=0x7f2620007d20).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:26.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.551+0000 7f2626ffd700 1 -- 172.21.15.29:0/1413553326 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f261804d2f0 con 0x7f263014b140 2024-01-24T14:14:26.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.732+0000 7f263567f700 1 -- 172.21.15.29:0/1413553326 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f26300475e0 con 0x7f263014b140 2024-01-24T14:14:26.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.736+0000 7f2626ffd700 1 -- 172.21.15.29:0/1413553326 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2618050910 con 0x7f263014b140 2024-01-24T14:14:26.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.738+0000 7f263567f700 1 -- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f261005f270 msgr2=0x7f2610061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:26.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.738+0000 7f263567f700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f261005f270 0x7f2610061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f26200038b0 tx=0x7f2620007d20).stop 2024-01-24T14:14:26.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.738+0000 7f263567f700 1 -- 172.21.15.29:0/1413553326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f263014b140 msgr2=0x7f26300a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:26.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.738+0000 7f263567f700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f263014b140 0x7f26300a5850 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f263005ee00 tx=0x7f2618017000).stop 2024-01-24T14:14:26.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.738+0000 7f263567f700 1 -- 172.21.15.29:0/1413553326 shutdown_connections 2024-01-24T14:14:26.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.738+0000 7f263567f700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2630155160 0x7f26300a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.738+0000 7f263567f700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2630062ca0 0x7f2630148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.738+0000 7f263567f700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f261005f270 0x7f2610061730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.738+0000 7f263567f700 1 --2- 172.21.15.29:0/1413553326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f263014b140 0x7f26300a5850 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.738+0000 7f263567f700 1 -- 172.21.15.29:0/1413553326 >> 172.21.15.29:0/1413553326 conn(0x7f2630064390 msgr2=0x7f2630152250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:26.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.738+0000 7f263567f700 1 -- 172.21.15.29:0/1413553326 shutdown_connections 2024-01-24T14:14:26.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.738+0000 7f263567f700 1 -- 172.21.15.29:0/1413553326 wait complete. 2024-01-24T14:14:26.752 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-01-24T14:14:26.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.884+0000 7f2de0cac700 1 Processor -- start 2024-01-24T14:14:26.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.884+0000 7f2de0cac700 1 -- start start 2024-01-24T14:14:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.885+0000 7f2de0cac700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc14e530 0x7f2ddc155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.885+0000 7f2de0cac700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2ddc0c0e20 0x7f2ddc068a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.885+0000 7f2de0cac700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ddc158960 con 0x7f2ddc05e1f0 2024-01-24T14:14:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.885+0000 7f2de0cac700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ddc0c1b90 con 0x7f2ddc0c0e20 2024-01-24T14:14:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.885+0000 7f2de0cac700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc0c4760 con 0x7f2ddc14e530 2024-01-24T14:14:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.885+0000 7f2dd9d9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2ddc05e1f0 0x7f2ddc0c0710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40340/0 (socket says 172.21.15.29:40340) 2024-01-24T14:14:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.885+0000 7f2dd9d9b700 1 -- 172.21.15.29:0/3067788983 learned_addr learned my addr 172.21.15.29:0/3067788983 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.885+0000 7f2dda59c700 1 --2- 172.21.15.29:0/3067788983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc14e530 0x7f2ddc155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.885+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874829361 0 0) 0x7f2ddc158960 con 0x7f2ddc05e1f0 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.885+0000 7f2ddad9d700 1 --2- 172.21.15.29:0/3067788983 >> v2:172.21.15.163:3300/0 conn(0x7f2ddc0c0e20 0x7f2ddc068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.885+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dc40033f0 con 0x7f2ddc05e1f0 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 144524791 0 0) 0x7f2dc40033f0 con 0x7f2ddc05e1f0 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc14e530 msgr2=0x7f2ddc155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 --2- 172.21.15.29:0/3067788983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc14e530 0x7f2ddc155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 >> v2:172.21.15.163:3300/0 conn(0x7f2ddc0c0e20 msgr2=0x7f2ddc068a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 --2- 172.21.15.29:0/3067788983 >> v2:172.21.15.163:3300/0 conn(0x7f2ddc0c0e20 0x7f2ddc068a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc155650 con 0x7f2ddc05e1f0 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f2dd00015f0 con 0x7f2ddc05e1f0 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 >> v1:172.21.15.29:6789/0 conn(0x7f2ddc05e1f0 legacy=0x7f2ddc0c0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 --2- 172.21.15.29:0/3067788983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc40038e0 0x7f2dc4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 --2- 172.21.15.29:0/3067788983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc4004690 0x7f2dc4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 --2- 172.21.15.29:0/3067788983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc14e530 0x7f2dc4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc0c1b90 con 0x7f2dc4004690 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40044b0 con 0x7f2dc40038e0 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc400b910 con 0x7f2ddc14e530 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd9d9b700 1 --2- 172.21.15.29:0/3067788983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc4004690 0x7f2dc4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dda59c700 1 --2- 172.21.15.29:0/3067788983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc40038e0 0x7f2dc4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2ddad9d700 1 --2- 172.21.15.29:0/3067788983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc14e530 0x7f2dc4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd9d9b700 1 -- 172.21.15.29:0/3067788983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc14e530 msgr2=0x7f2dc4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd9d9b700 1 --2- 172.21.15.29:0/3067788983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc14e530 0x7f2dc4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd9d9b700 1 -- 172.21.15.29:0/3067788983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc40038e0 msgr2=0x7f2dc4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd9d9b700 1 --2- 172.21.15.29:0/3067788983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc40038e0 0x7f2dc4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd9d9b700 1 -- 172.21.15.29:0/3067788983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc40033f0 con 0x7f2dc4004690 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.886+0000 7f2dd9d9b700 1 --2- 172.21.15.29:0/3067788983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc4004690 0x7f2dc4004b10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7f2dd000c160 tx=0x7f2ddc1588d0).ready entity=mon.0 client_cookie=fe7a89b843e91000 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.887+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0007a40 con 0x7f2dc4004690 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.887+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd000ca90 con 0x7f2dc4004690 2024-01-24T14:14:26.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.887+0000 7f2dd959a700 1 -- 172.21.15.29:0/3067788983 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000ce30 con 0x7f2dc4004690 2024-01-24T14:14:26.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.887+0000 7f2de0cac700 1 -- 172.21.15.29:0/3067788983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc4004690 msgr2=0x7f2dc4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:26.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.887+0000 7f2de0cac700 1 --2- 172.21.15.29:0/3067788983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc4004690 0x7f2dc4004b10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7f2dd000c160 tx=0x7f2ddc1588d0).stop 2024-01-24T14:14:26.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.887+0000 7f2ddad9d700 1 -- 172.21.15.29:0/3067788983 reap_dead start 2024-01-24T14:14:26.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.887+0000 7f2de0cac700 1 -- 172.21.15.29:0/3067788983 shutdown_connections 2024-01-24T14:14:26.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.887+0000 7f2de0cac700 1 -- 172.21.15.29:0/3067788983 >> 172.21.15.29:0/3067788983 conn(0x7f2ddc0c7d40 msgr2=0x7f2ddc0c3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:26.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.887+0000 7f2de0cac700 1 -- 172.21.15.29:0/3067788983 shutdown_connections 2024-01-24T14:14:26.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.887+0000 7f2de0cac700 1 -- 172.21.15.29:0/3067788983 wait complete. 2024-01-24T14:14:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.888+0000 7f2de0cac700 1 Processor -- start 2024-01-24T14:14:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.888+0000 7f2de0cac700 1 -- start start 2024-01-24T14:14:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.889+0000 7f2de0cac700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc4004690 0x7f2ddc14dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.889+0000 7f2de0cac700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc0ac3b0 0x7f2ddc14f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.889+0000 7f2de0cac700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ddc156970 0x7f2ddc156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.889+0000 7f2de0cac700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc151c30 con 0x7f2dc4004690 2024-01-24T14:14:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.889+0000 7f2de0cac700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc1572e0 con 0x7f2ddc156970 2024-01-24T14:14:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.889+0000 7f2de0cac700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc1504a0 con 0x7f2ddc0ac3b0 2024-01-24T14:14:26.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.889+0000 7f2dda59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc4004690 0x7f2ddc14dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.889+0000 7f2dda59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc4004690 0x7f2ddc14dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49890/0 (socket says 172.21.15.29:49890) 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.889+0000 7f2dda59c700 1 -- 172.21.15.29:0/1892460806 learned_addr learned my addr 172.21.15.29:0/1892460806 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.889+0000 7f2dd9d9b700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc0ac3b0 0x7f2ddc14f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.889+0000 7f2ddad9d700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ddc156970 0x7f2ddc156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.890+0000 7f2dda59c700 1 -- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc0ac3b0 msgr2=0x7f2ddc14f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.890+0000 7f2dda59c700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc0ac3b0 0x7f2ddc14f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.890+0000 7f2dda59c700 1 -- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ddc156970 msgr2=0x7f2ddc156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.890+0000 7f2dda59c700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ddc156970 0x7f2ddc156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.890+0000 7f2dda59c700 1 -- 172.21.15.29:0/1892460806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc151900 con 0x7f2dc4004690 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.890+0000 7f2dda59c700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc4004690 0x7f2ddc14dda0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f2dcc011a50 tx=0x7f2dcc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.890+0000 7f2dca7fc700 1 -- 172.21.15.29:0/1892460806 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc011e90 con 0x7f2dc4004690 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.890+0000 7f2dca7fc700 1 -- 172.21.15.29:0/1892460806 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc00f8c0 con 0x7f2dc4004690 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.890+0000 7f2de0cac700 1 -- 172.21.15.29:0/1892460806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ddc14fc00 con 0x7f2dc4004690 2024-01-24T14:14:26.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.890+0000 7f2dca7fc700 1 -- 172.21.15.29:0/1892460806 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00fc40 con 0x7f2dc4004690 2024-01-24T14:14:26.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.890+0000 7f2de0cac700 1 -- 172.21.15.29:0/1892460806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ddc1501c0 con 0x7f2dc4004690 2024-01-24T14:14:26.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.891+0000 7f2de0cac700 1 -- 172.21.15.29:0/1892460806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ddc0475e0 con 0x7f2dc4004690 2024-01-24T14:14:26.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.892+0000 7f2dca7fc700 1 -- 172.21.15.29:0/1892460806 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2dcc014440 con 0x7f2dc4004690 2024-01-24T14:14:26.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.892+0000 7f2dca7fc700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2dbc05f270 0x7f2dbc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:26.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.892+0000 7f2dca7fc700 1 -- 172.21.15.29:0/1892460806 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6849+0+0 (secure 0 0 0) 0x7f2dcc07b120 con 0x7f2dc4004690 2024-01-24T14:14:26.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.892+0000 7f2dd9d9b700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2dbc05f270 0x7f2dbc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:26.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.893+0000 7f2dd9d9b700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2dbc05f270 0x7f2dbc061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2dd0000ef0 tx=0x7f2dd0015000).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:26.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:26.896+0000 7f2dca7fc700 1 -- 172.21.15.29:0/1892460806 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2dcc04bf30 con 0x7f2dc4004690 2024-01-24T14:14:27.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.078+0000 7f2de0cac700 1 -- 172.21.15.29:0/1892460806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f2ddc0c12e0 con 0x7f2dc4004690 2024-01-24T14:14:27.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.082+0000 7f2dca7fc700 1 -- 172.21.15.29:0/1892460806 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v17) v1 ==== 120+0+0 (secure 0 0 0) 0x7f2dcc04f550 con 0x7f2dc4004690 2024-01-24T14:14:27.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.084+0000 7f2de0cac700 1 -- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2dbc05f270 msgr2=0x7f2dbc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:27.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.084+0000 7f2de0cac700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2dbc05f270 0x7f2dbc061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2dd0000ef0 tx=0x7f2dd0015000).stop 2024-01-24T14:14:27.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.084+0000 7f2de0cac700 1 -- 172.21.15.29:0/1892460806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc4004690 msgr2=0x7f2ddc14dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:27.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.084+0000 7f2de0cac700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc4004690 0x7f2ddc14dda0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f2dcc011a50 tx=0x7f2dcc009060).stop 2024-01-24T14:14:27.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.084+0000 7f2de0cac700 1 -- 172.21.15.29:0/1892460806 shutdown_connections 2024-01-24T14:14:27.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.084+0000 7f2de0cac700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ddc156970 0x7f2ddc156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:27.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.084+0000 7f2de0cac700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2dbc05f270 0x7f2dbc061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:27.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.084+0000 7f2de0cac700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2dc4004690 0x7f2ddc14dda0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:27.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.084+0000 7f2de0cac700 1 --2- 172.21.15.29:0/1892460806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ddc0ac3b0 0x7f2ddc14f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:27.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.084+0000 7f2de0cac700 1 -- 172.21.15.29:0/1892460806 >> 172.21.15.29:0/1892460806 conn(0x7f2ddc0c7d40 msgr2=0x7f2ddc0a9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:27.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.084+0000 7f2de0cac700 1 -- 172.21.15.29:0/1892460806 shutdown_connections 2024-01-24T14:14:27.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.084+0000 7f2de0cac700 1 -- 172.21.15.29:0/1892460806 wait complete. 2024-01-24T14:14:27.098 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2024-01-24T14:14:27.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.227+0000 7f9cf8927700 1 Processor -- start 2024-01-24T14:14:27.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.228+0000 7f9cf8927700 1 -- start start 2024-01-24T14:14:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.228+0000 7f9cf8927700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9cf4138330 0x7f9cf4138710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf8927700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9cf40bca50 0x7f9cf40be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf8927700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf4065770 con 0x7f9cf40627b0 2024-01-24T14:14:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf8927700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf4069cf0 con 0x7f9cf40bca50 2024-01-24T14:14:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf8927700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf4069f60 con 0x7f9cf4138330 2024-01-24T14:14:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf1d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9cf40bca50 0x7f9cf40be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf2d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9cf40627b0 0x7f9cf4155ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40350/0 (socket says 172.21.15.29:40350) 2024-01-24T14:14:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf2d9d700 1 -- 172.21.15.29:0/1778146180 learned_addr learned my addr 172.21.15.29:0/1778146180 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf259c700 1 --2- 172.21.15.29:0/1778146180 >> v2:172.21.15.163:3300/0 conn(0x7f9cf4138330 0x7f9cf4138710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf1d9b700 1 -- 172.21.15.29:0/1778146180 >> v2:172.21.15.163:3300/0 conn(0x7f9cf4138330 msgr2=0x7f9cf4138710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:27.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf1d9b700 1 --2- 172.21.15.29:0/1778146180 >> v2:172.21.15.163:3300/0 conn(0x7f9cf4138330 0x7f9cf4138710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf1d9b700 1 -- 172.21.15.29:0/1778146180 >> v1:172.21.15.29:6789/0 conn(0x7f9cf40627b0 legacy=0x7f9cf4155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf1d9b700 1 -- 172.21.15.29:0/1778146180 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf406a240 con 0x7f9cf40bca50 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf159a700 1 -- 172.21.15.29:0/1778146180 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282652774 0 0) 0x7f9cf4065770 con 0x7f9cf40627b0 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf1d9b700 1 --2- 172.21.15.29:0/1778146180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cf40bca50 0x7f9cf40be3e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f9cf40649f0 tx=0x7f9ce8009060).ready entity=mon.0 client_cookie=1e4f36684bbd9473 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf159a700 1 -- 172.21.15.29:0/1778146180 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800b580 con 0x7f9cf40bca50 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf159a700 1 -- 172.21.15.29:0/1778146180 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce800b6e0 con 0x7f9cf40bca50 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.229+0000 7f9cf159a700 1 -- 172.21.15.29:0/1778146180 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800ba60 con 0x7f9cf40bca50 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.230+0000 7f9cf8927700 1 -- 172.21.15.29:0/1778146180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cf40bca50 msgr2=0x7f9cf40be3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.230+0000 7f9cf8927700 1 --2- 172.21.15.29:0/1778146180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cf40bca50 0x7f9cf40be3e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f9cf40649f0 tx=0x7f9ce8009060).stop 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.230+0000 7f9cf8927700 1 -- 172.21.15.29:0/1778146180 shutdown_connections 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.230+0000 7f9cf8927700 1 --2- 172.21.15.29:0/1778146180 >> v2:172.21.15.163:3300/0 conn(0x7f9cf4138330 0x7f9cf4138710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.230+0000 7f9cf8927700 1 --2- 172.21.15.29:0/1778146180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cf40bca50 0x7f9cf40be3e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:27.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.230+0000 7f9cf8927700 1 -- 172.21.15.29:0/1778146180 >> 172.21.15.29:0/1778146180 conn(0x7f9cf40c8580 msgr2=0x7f9cf40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:27.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.230+0000 7f9cf8927700 1 -- 172.21.15.29:0/1778146180 shutdown_connections 2024-01-24T14:14:27.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.230+0000 7f9cf8927700 1 -- 172.21.15.29:0/1778146180 wait complete. 2024-01-24T14:14:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.231+0000 7f9cf8927700 1 Processor -- start 2024-01-24T14:14:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.231+0000 7f9cf8927700 1 -- start start 2024-01-24T14:14:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.232+0000 7f9cf8927700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cf40627b0 0x7f9cf4136920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:27.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.232+0000 7f9cf8927700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cf40bca50 0x7f9cf41568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.232+0000 7f9cf8927700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cf4138330 0x7f9cf4156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.232+0000 7f9cf8927700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf41573b0 con 0x7f9cf40bca50 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.232+0000 7f9cf8927700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40ac440 con 0x7f9cf40627b0 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.232+0000 7f9cf8927700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40abb10 con 0x7f9cf4138330 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.232+0000 7f9cf1d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cf40bca50 0x7f9cf41568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.232+0000 7f9cf1d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cf40bca50 0x7f9cf41568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49898/0 (socket says 172.21.15.29:49898) 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.232+0000 7f9cf1d9b700 1 -- 172.21.15.29:0/3731375039 learned_addr learned my addr 172.21.15.29:0/3731375039 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9cf259c700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cf40627b0 0x7f9cf4136920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9cf2d9d700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cf4138330 0x7f9cf4156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9cf1d9b700 1 -- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cf4138330 msgr2=0x7f9cf4156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9cf1d9b700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cf4138330 0x7f9cf4156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9cf1d9b700 1 -- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cf40627b0 msgr2=0x7f9cf4136920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9cf1d9b700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cf40627b0 0x7f9cf4136920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:27.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9cf1d9b700 1 -- 172.21.15.29:0/3731375039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce80092a0 con 0x7f9cf40bca50 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9cf2d9d700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cf4138330 0x7f9cf4156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9cf1d9b700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cf40bca50 0x7f9cf41568d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f9ce8009e80 tx=0x7f9ce8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9ce27fc700 1 -- 172.21.15.29:0/3731375039 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800b850 con 0x7f9cf40bca50 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9ce27fc700 1 -- 172.21.15.29:0/3731375039 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce800b9b0 con 0x7f9cf40bca50 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9cf8927700 1 -- 172.21.15.29:0/3731375039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf40abdf0 con 0x7f9cf40bca50 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9ce27fc700 1 -- 172.21.15.29:0/3731375039 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce801b640 con 0x7f9cf40bca50 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.233+0000 7f9cf8927700 1 -- 172.21.15.29:0/3731375039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf40ac8d0 con 0x7f9cf40bca50 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.234+0000 7f9cf8927700 1 -- 172.21.15.29:0/3731375039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf40ac5f0 con 0x7f9cf40bca50 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.235+0000 7f9ce27fc700 1 -- 172.21.15.29:0/3731375039 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9ce801ba30 con 0x7f9cf40bca50 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.235+0000 7f9ce27fc700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9cd405f270 0x7f9cd4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.235+0000 7f9ce27fc700 1 -- 172.21.15.29:0/3731375039 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6849+0+0 (secure 0 0 0) 0x7f9ce807c430 con 0x7f9cf40bca50 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.235+0000 7f9cf259c700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9cd405f270 0x7f9cd4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:27.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.236+0000 7f9cf259c700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9cd405f270 0x7f9cd4061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f9cf4063520 tx=0x7f9ce4008040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:27.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.240+0000 7f9ce27fc700 1 -- 172.21.15.29:0/3731375039 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9cf40ac5f0 con 0x7f9cf40bca50 2024-01-24T14:14:27.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:27.494+0000 7f9cf8927700 1 -- 172.21.15.29:0/3731375039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f9cf40ac5b0 con 0x7f9cf40bca50 2024-01-24T14:14:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.383+0000 7f9ce27fc700 1 -- 172.21.15.29:0/3731375039 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v601) v1 ==== 166+0+0 (secure 0 0 0) 0x7f9ce804d2a0 con 0x7f9cf40bca50 2024-01-24T14:14:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.386+0000 7f9cf8927700 1 -- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9cd405f270 msgr2=0x7f9cd4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.386+0000 7f9cf8927700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9cd405f270 0x7f9cd4061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f9cf4063520 tx=0x7f9ce4008040).stop 2024-01-24T14:14:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.386+0000 7f9cf8927700 1 -- 172.21.15.29:0/3731375039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cf40bca50 msgr2=0x7f9cf41568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.386+0000 7f9cf8927700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cf40bca50 0x7f9cf41568d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f9ce8009e80 tx=0x7f9ce8017000).stop 2024-01-24T14:14:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.386+0000 7f9cf8927700 1 -- 172.21.15.29:0/3731375039 shutdown_connections 2024-01-24T14:14:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.386+0000 7f9cf8927700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cf4138330 0x7f9cf4156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.386+0000 7f9cf8927700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cf40627b0 0x7f9cf4136920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.386+0000 7f9cf8927700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9cd405f270 0x7f9cd4061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.386+0000 7f9cf8927700 1 --2- 172.21.15.29:0/3731375039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cf40bca50 0x7f9cf41568d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:28.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.386+0000 7f9cf8927700 1 -- 172.21.15.29:0/3731375039 >> 172.21.15.29:0/3731375039 conn(0x7f9cf40c8580 msgr2=0x7f9cf4158550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:28.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.386+0000 7f9cf8927700 1 -- 172.21.15.29:0/3731375039 shutdown_connections 2024-01-24T14:14:28.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.386+0000 7f9cf8927700 1 -- 172.21.15.29:0/3731375039 wait complete. 2024-01-24T14:14:28.402 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2024-01-24T14:14:28.402 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2024-01-24T14:14:28.402 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-01-24T14:14:28.403 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2024-01-24T14:14:28.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.537+0000 7f171d9e8700 1 Processor -- start 2024-01-24T14:14:28.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.537+0000 7f171d9e8700 1 -- start start 2024-01-24T14:14:28.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.538+0000 7f171d9e8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1718154370 0x7f17180c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:28.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.538+0000 7f171d9e8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17180aa0e0 0x7f1718154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:28.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.538+0000 7f171d9e8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17180aa6c0 con 0x7f17180ab1f0 2024-01-24T14:14:28.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.538+0000 7f171d9e8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1718064240 con 0x7f17180aa0e0 2024-01-24T14:14:28.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.538+0000 7f171d9e8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17180cb5c0 con 0x7f1718154370 2024-01-24T14:14:28.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.538+0000 7f17167fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17180aa0e0 0x7f1718154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:28.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.538+0000 7f17167fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17180aa0e0 0x7f1718154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49912/0 (socket says 172.21.15.29:49912) 2024-01-24T14:14:28.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.538+0000 7f17167fc700 1 -- 172.21.15.29:0/1038070264 learned_addr learned my addr 172.21.15.29:0/1038070264 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:28.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.538+0000 7f1716ffd700 1 --2- 172.21.15.29:0/1038070264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1718154370 0x7f17180c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:28.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f17167fc700 1 -- 172.21.15.29:0/1038070264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1718154370 msgr2=0x7f17180c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:28.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f17167fc700 1 --2- 172.21.15.29:0/1038070264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1718154370 0x7f17180c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:28.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f17167fc700 1 -- 172.21.15.29:0/1038070264 >> v1:172.21.15.163:6789/0 conn(0x7f17180ab1f0 legacy=0x7f17180631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:28.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f17167fc700 1 -- 172.21.15.29:0/1038070264 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17180cb8a0 con 0x7f17180aa0e0 2024-01-24T14:14:28.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f17167fc700 1 --2- 172.21.15.29:0/1038070264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17180aa0e0 0x7f1718154750 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f1700006fd0 tx=0x7f17000090f0).ready entity=mon.0 client_cookie=a342a1d84a170517 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:28.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f1715ffb700 1 -- 172.21.15.29:0/1038070264 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170000b580 con 0x7f17180aa0e0 2024-01-24T14:14:28.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f1715ffb700 1 -- 172.21.15.29:0/1038070264 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170000b6e0 con 0x7f17180aa0e0 2024-01-24T14:14:28.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f1715ffb700 1 -- 172.21.15.29:0/1038070264 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170000ba60 con 0x7f17180aa0e0 2024-01-24T14:14:28.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f171d9e8700 1 -- 172.21.15.29:0/1038070264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17180aa0e0 msgr2=0x7f1718154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:28.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f171d9e8700 1 --2- 172.21.15.29:0/1038070264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17180aa0e0 0x7f1718154750 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f1700006fd0 tx=0x7f17000090f0).stop 2024-01-24T14:14:28.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f171d9e8700 1 -- 172.21.15.29:0/1038070264 shutdown_connections 2024-01-24T14:14:28.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f171d9e8700 1 --2- 172.21.15.29:0/1038070264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1718154370 0x7f17180c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:28.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f171d9e8700 1 --2- 172.21.15.29:0/1038070264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17180aa0e0 0x7f1718154750 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:28.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.539+0000 7f171d9e8700 1 -- 172.21.15.29:0/1038070264 >> 172.21.15.29:0/1038070264 conn(0x7f171805ec80 msgr2=0x7f17180cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:28.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.540+0000 7f171d9e8700 1 -- 172.21.15.29:0/1038070264 shutdown_connections 2024-01-24T14:14:28.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.540+0000 7f171d9e8700 1 -- 172.21.15.29:0/1038070264 wait complete. 2024-01-24T14:14:28.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.541+0000 7f171d9e8700 1 Processor -- start 2024-01-24T14:14:28.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.541+0000 7f171d9e8700 1 -- start start 2024-01-24T14:14:28.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.542+0000 7f171d9e8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17180aa0e0 0x7f1718159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:28.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.542+0000 7f171d9e8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17180ab1f0 0x7f1718168630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:28.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.542+0000 7f171d9e8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1718154370 0x7f1718152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:28.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.542+0000 7f171d9e8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1718165020 con 0x7f17180aa0e0 2024-01-24T14:14:28.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f171d9e8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1718150450 con 0x7f17180ab1f0 2024-01-24T14:14:28.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f171d9e8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1718165180 con 0x7f1718154370 2024-01-24T14:14:28.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f1716ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17180aa0e0 0x7f1718159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:28.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f1716ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17180aa0e0 0x7f1718159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49920/0 (socket says 172.21.15.29:49920) 2024-01-24T14:14:28.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f1716ffd700 1 -- 172.21.15.29:0/1004880651 learned_addr learned my addr 172.21.15.29:0/1004880651 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:28.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f17177fe700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1718154370 0x7f1718152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:28.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f17167fc700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17180ab1f0 0x7f1718168630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:28.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f1716ffd700 1 -- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1718154370 msgr2=0x7f1718152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:28.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f1716ffd700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1718154370 0x7f1718152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:28.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f1716ffd700 1 -- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17180ab1f0 msgr2=0x7f1718168630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:28.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f1716ffd700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17180ab1f0 0x7f1718168630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:28.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f1716ffd700 1 -- 172.21.15.29:0/1004880651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1700009370 con 0x7f17180aa0e0 2024-01-24T14:14:28.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f17177fe700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1718154370 0x7f1718152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:28.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f17167fc700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17180ab1f0 0x7f1718168630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:28.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f1716ffd700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17180aa0e0 0x7f1718159d20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f170800e8d0 tx=0x7f1708006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:28.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f170effd700 1 -- 172.21.15.29:0/1004880651 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170800c5e0 con 0x7f17180aa0e0 2024-01-24T14:14:28.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f170effd700 1 -- 172.21.15.29:0/1004880651 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170800c740 con 0x7f17180aa0e0 2024-01-24T14:14:28.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f171d9e8700 1 -- 172.21.15.29:0/1004880651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17181571c0 con 0x7f17180aa0e0 2024-01-24T14:14:28.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f170effd700 1 -- 172.21.15.29:0/1004880651 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170800cac0 con 0x7f17180aa0e0 2024-01-24T14:14:28.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.543+0000 7f171d9e8700 1 -- 172.21.15.29:0/1004880651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1718157780 con 0x7f17180aa0e0 2024-01-24T14:14:28.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.545+0000 7f171d9e8700 1 -- 172.21.15.29:0/1004880651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17180caad0 con 0x7f17180aa0e0 2024-01-24T14:14:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.545+0000 7f170effd700 1 -- 172.21.15.29:0/1004880651 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f170801b400 con 0x7f17180aa0e0 2024-01-24T14:14:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.545+0000 7f170effd700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f16f805f240 0x7f16f8061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.545+0000 7f170effd700 1 -- 172.21.15.29:0/1004880651 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6849+0+0 (secure 0 0 0) 0x7f170807aaa0 con 0x7f17180aa0e0 2024-01-24T14:14:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.546+0000 7f17167fc700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f16f805f240 0x7f16f8061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:28.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.546+0000 7f17167fc700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f16f805f240 0x7f16f8061700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f1700006fd0 tx=0x7f1700017000).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:28.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.550+0000 7f170effd700 1 -- 172.21.15.29:0/1004880651 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f170804b8b0 con 0x7f17180aa0e0 2024-01-24T14:14:28.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:28.802+0000 7f171d9e8700 1 -- 172.21.15.29:0/1004880651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f17180475e0 con 0x7f17180aa0e0 2024-01-24T14:14:28.886 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:14:28.889 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:14:29.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.386+0000 7f170effd700 1 -- 172.21.15.29:0/1004880651 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v602) v1 ==== 186+0+0 (secure 0 0 0) 0x7f1708016070 con 0x7f17180aa0e0 2024-01-24T14:14:29.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.388+0000 7f171d9e8700 1 -- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f16f805f240 msgr2=0x7f16f8061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.388+0000 7f171d9e8700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f16f805f240 0x7f16f8061700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f1700006fd0 tx=0x7f1700017000).stop 2024-01-24T14:14:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.388+0000 7f171d9e8700 1 -- 172.21.15.29:0/1004880651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17180aa0e0 msgr2=0x7f1718159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.388+0000 7f171d9e8700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17180aa0e0 0x7f1718159d20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f170800e8d0 tx=0x7f1708006f40).stop 2024-01-24T14:14:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.388+0000 7f171d9e8700 1 -- 172.21.15.29:0/1004880651 shutdown_connections 2024-01-24T14:14:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.388+0000 7f171d9e8700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1718154370 0x7f1718152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.388+0000 7f171d9e8700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f16f805f240 0x7f16f8061700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.388+0000 7f171d9e8700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17180aa0e0 0x7f1718159d20 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.388+0000 7f171d9e8700 1 --2- 172.21.15.29:0/1004880651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17180ab1f0 0x7f1718168630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.388+0000 7f171d9e8700 1 -- 172.21.15.29:0/1004880651 >> 172.21.15.29:0/1004880651 conn(0x7f171805ec80 msgr2=0x7f1718151890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.388+0000 7f171d9e8700 1 -- 172.21.15.29:0/1004880651 shutdown_connections 2024-01-24T14:14:29.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.389+0000 7f171d9e8700 1 -- 172.21.15.29:0/1004880651 wait complete. 2024-01-24T14:14:29.405 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-24T14:14:29.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.543+0000 7fbba881b700 1 Processor -- start 2024-01-24T14:14:29.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.543+0000 7fbba881b700 1 -- start start 2024-01-24T14:14:29.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.544+0000 7fbba881b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbba40692a0 0x7fbba4065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.544+0000 7fbba881b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbba4066370 0x7fbba4069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.544+0000 7fbba881b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba40c3e70 con 0x7fbba4069b70 2024-01-24T14:14:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.544+0000 7fbba881b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba40c4300 con 0x7fbba4066370 2024-01-24T14:14:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.544+0000 7fbba881b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba40c3260 con 0x7fbba40692a0 2024-01-24T14:14:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.544+0000 7fbba1d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbba4066370 0x7fbba4069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.544+0000 7fbba1d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbba4066370 0x7fbba4069680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49924/0 (socket says 172.21.15.29:49924) 2024-01-24T14:14:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.544+0000 7fbba1d9b700 1 -- 172.21.15.29:0/122708955 learned_addr learned my addr 172.21.15.29:0/122708955 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.544+0000 7fbba259c700 1 --2- 172.21.15.29:0/122708955 >> v2:172.21.15.163:3300/0 conn(0x7fbba40692a0 0x7fbba4065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.545+0000 7fbba1d9b700 1 -- 172.21.15.29:0/122708955 >> v2:172.21.15.163:3300/0 conn(0x7fbba40692a0 msgr2=0x7fbba4065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.545+0000 7fbba1d9b700 1 --2- 172.21.15.29:0/122708955 >> v2:172.21.15.163:3300/0 conn(0x7fbba40692a0 0x7fbba4065e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.545+0000 7fbba1d9b700 1 -- 172.21.15.29:0/122708955 >> v1:172.21.15.163:6789/0 conn(0x7fbba4069b70 legacy=0x7fbba40bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:29.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.545+0000 7fbba1d9b700 1 -- 172.21.15.29:0/122708955 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40c3540 con 0x7fbba4066370 2024-01-24T14:14:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.545+0000 7fbba1d9b700 1 --2- 172.21.15.29:0/122708955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbba4066370 0x7fbba4069680 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7fbba405ed00 tx=0x7fbb940090f0).ready entity=mon.0 client_cookie=97205e763fb45851 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.545+0000 7fbba159a700 1 -- 172.21.15.29:0/122708955 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9400b520 con 0x7fbba4066370 2024-01-24T14:14:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.545+0000 7fbba159a700 1 -- 172.21.15.29:0/122708955 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9400b680 con 0x7fbba4066370 2024-01-24T14:14:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.545+0000 7fbba159a700 1 -- 172.21.15.29:0/122708955 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9400ba00 con 0x7fbba4066370 2024-01-24T14:14:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.545+0000 7fbba881b700 1 -- 172.21.15.29:0/122708955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbba4066370 msgr2=0x7fbba4069680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.545+0000 7fbba881b700 1 --2- 172.21.15.29:0/122708955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbba4066370 0x7fbba4069680 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7fbba405ed00 tx=0x7fbb940090f0).stop 2024-01-24T14:14:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.546+0000 7fbba881b700 1 -- 172.21.15.29:0/122708955 shutdown_connections 2024-01-24T14:14:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.546+0000 7fbba881b700 1 --2- 172.21.15.29:0/122708955 >> v2:172.21.15.163:3300/0 conn(0x7fbba40692a0 0x7fbba4065e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.546+0000 7fbba881b700 1 --2- 172.21.15.29:0/122708955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbba4066370 0x7fbba4069680 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.546+0000 7fbba881b700 1 -- 172.21.15.29:0/122708955 >> 172.21.15.29:0/122708955 conn(0x7fbba4064390 msgr2=0x7fbba40c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:29.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.546+0000 7fbba881b700 1 -- 172.21.15.29:0/122708955 shutdown_connections 2024-01-24T14:14:29.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.546+0000 7fbba881b700 1 -- 172.21.15.29:0/122708955 wait complete. 2024-01-24T14:14:29.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.547+0000 7fbba881b700 1 Processor -- start 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.547+0000 7fbba881b700 1 -- start start 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.548+0000 7fbba881b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbba4066370 0x7fbba414b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.548+0000 7fbba881b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba40692a0 0x7fbba414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.548+0000 7fbba881b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbba4069b70 0x7fbba414ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.548+0000 7fbba881b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba414d0b0 con 0x7fbba4066370 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.548+0000 7fbba881b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba414deb0 con 0x7fbba40692a0 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.548+0000 7fbba881b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba414bb90 con 0x7fbba4069b70 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbba4066370 0x7fbba414b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbba4066370 0x7fbba414b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49932/0 (socket says 172.21.15.29:49932) 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba259c700 1 -- 172.21.15.29:0/4177626433 learned_addr learned my addr 172.21.15.29:0/4177626433 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba1d9b700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba40692a0 0x7fbba414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba2d9d700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbba4069b70 0x7fbba414ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba259c700 1 -- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbba4069b70 msgr2=0x7fbba414ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:29.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba259c700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbba4069b70 0x7fbba414ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba259c700 1 -- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba40692a0 msgr2=0x7fbba414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba259c700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba40692a0 0x7fbba414cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba259c700 1 -- 172.21.15.29:0/4177626433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94009370 con 0x7fbba4066370 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba2d9d700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbba4069b70 0x7fbba414ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba259c700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbba4066370 0x7fbba414b360 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7fbb8c00e930 tx=0x7fbb8c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbb9a7fc700 1 -- 172.21.15.29:0/4177626433 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c00c670 con 0x7fbba4066370 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbb9a7fc700 1 -- 172.21.15.29:0/4177626433 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c00c7d0 con 0x7fbba4066370 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbb9a7fc700 1 -- 172.21.15.29:0/4177626433 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c00cb50 con 0x7fbba4066370 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba881b700 1 -- 172.21.15.29:0/4177626433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba414b140 con 0x7fbba4066370 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.549+0000 7fbba881b700 1 -- 172.21.15.29:0/4177626433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba414d580 con 0x7fbba4066370 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.550+0000 7fbba881b700 1 -- 172.21.15.29:0/4177626433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba414e190 con 0x7fbba4066370 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.551+0000 7fbb9a7fc700 1 -- 172.21.15.29:0/4177626433 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbb8c01b400 con 0x7fbba4066370 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.551+0000 7fbb9a7fc700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbb8405f270 0x7fbb84061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.551+0000 7fbb9a7fc700 1 -- 172.21.15.29:0/4177626433 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6849+0+0 (secure 0 0 0) 0x7fbb8c07ab70 con 0x7fbba4066370 2024-01-24T14:14:29.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.551+0000 7fbba1d9b700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbb8405f270 0x7fbb84061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:29.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.552+0000 7fbba1d9b700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbb8405f270 0x7fbb84061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fbb94009130 tx=0x7fbb94017000).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.555+0000 7fbb9a7fc700 1 -- 172.21.15.29:0/4177626433 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbb8c04ba30 con 0x7fbba4066370 2024-01-24T14:14:29.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.740+0000 7fbba881b700 1 -- 172.21.15.29:0/4177626433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbba40475e0 con 0x7fbba4066370 2024-01-24T14:14:29.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.743+0000 7fbb9a7fc700 1 -- 172.21.15.29:0/4177626433 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbb8c04f050 con 0x7fbba4066370 2024-01-24T14:14:29.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.745+0000 7fbba881b700 1 -- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbb8405f270 msgr2=0x7fbb84061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:29.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.745+0000 7fbba881b700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbb8405f270 0x7fbb84061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fbb94009130 tx=0x7fbb94017000).stop 2024-01-24T14:14:29.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.745+0000 7fbba881b700 1 -- 172.21.15.29:0/4177626433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbba4066370 msgr2=0x7fbba414b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:29.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.745+0000 7fbba881b700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbba4066370 0x7fbba414b360 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7fbb8c00e930 tx=0x7fbb8c009060).stop 2024-01-24T14:14:29.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.745+0000 7fbba881b700 1 -- 172.21.15.29:0/4177626433 shutdown_connections 2024-01-24T14:14:29.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.745+0000 7fbba881b700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbba4069b70 0x7fbba414ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.745+0000 7fbba881b700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbb8405f270 0x7fbb84061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.745+0000 7fbba881b700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbba4066370 0x7fbba414b360 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.745+0000 7fbba881b700 1 --2- 172.21.15.29:0/4177626433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba40692a0 0x7fbba414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.745+0000 7fbba881b700 1 -- 172.21.15.29:0/4177626433 >> 172.21.15.29:0/4177626433 conn(0x7fbba4064390 msgr2=0x7fbba40c6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:29.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.746+0000 7fbba881b700 1 -- 172.21.15.29:0/4177626433 shutdown_connections 2024-01-24T14:14:29.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.746+0000 7fbba881b700 1 -- 172.21.15.29:0/4177626433 wait complete. 2024-01-24T14:14:29.759 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T14:14:29.760 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:14:29.802 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:14:29.856 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T14:14:29.856 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-24T14:14:29.857 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:14:29.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.992+0000 7f9768835700 1 Processor -- start 2024-01-24T14:14:29.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.992+0000 7f9768835700 1 -- start start 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.993+0000 7f9768835700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9764134cc0 0x7f97640c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.993+0000 7f9768835700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97641350a0 0x7f9764152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.993+0000 7f9768835700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9764068360 con 0x7f97640664f0 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.993+0000 7f9768835700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9764153360 con 0x7f9764134cc0 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.993+0000 7f9768835700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97640c3820 con 0x7f97641350a0 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.993+0000 7f976259c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9764134cc0 0x7f97640c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.994+0000 7f9762d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f97640664f0 0x7f9764155a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40354/0 (socket says 172.21.15.29:40354) 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.994+0000 7f9762d9d700 1 -- 172.21.15.29:0/601009371 learned_addr learned my addr 172.21.15.29:0/601009371 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.994+0000 7f9761d9b700 1 --2- 172.21.15.29:0/601009371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97641350a0 0x7f9764152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.994+0000 7f976259c700 1 -- 172.21.15.29:0/601009371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97641350a0 msgr2=0x7f9764152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.994+0000 7f976259c700 1 --2- 172.21.15.29:0/601009371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97641350a0 0x7f9764152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.994+0000 7f976259c700 1 -- 172.21.15.29:0/601009371 >> v1:172.21.15.29:6789/0 conn(0x7f97640664f0 legacy=0x7f9764155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:29.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.994+0000 7f976259c700 1 -- 172.21.15.29:0/601009371 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9764067a30 con 0x7f9764134cc0 2024-01-24T14:14:29.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.994+0000 7f976159a700 1 -- 172.21.15.29:0/601009371 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119916996 0 0) 0x7f9764068360 con 0x7f97640664f0 2024-01-24T14:14:29.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.994+0000 7f976259c700 1 --2- 172.21.15.29:0/601009371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9764134cc0 0x7f97640c31b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f9758009760 tx=0x7f9758009060).ready entity=mon.0 client_cookie=140d54179de2ab83 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:29.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.994+0000 7f976159a700 1 -- 172.21.15.29:0/601009371 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975800f510 con 0x7f9764134cc0 2024-01-24T14:14:29.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.994+0000 7f976159a700 1 -- 172.21.15.29:0/601009371 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975800f670 con 0x7f9764134cc0 2024-01-24T14:14:29.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.994+0000 7f976159a700 1 -- 172.21.15.29:0/601009371 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975800f9f0 con 0x7f9764134cc0 2024-01-24T14:14:29.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.995+0000 7f9768835700 1 -- 172.21.15.29:0/601009371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9764134cc0 msgr2=0x7f97640c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:29.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.995+0000 7f9768835700 1 --2- 172.21.15.29:0/601009371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9764134cc0 0x7f97640c31b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f9758009760 tx=0x7f9758009060).stop 2024-01-24T14:14:29.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.995+0000 7f9768835700 1 -- 172.21.15.29:0/601009371 shutdown_connections 2024-01-24T14:14:29.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.995+0000 7f9768835700 1 --2- 172.21.15.29:0/601009371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9764134cc0 0x7f97640c31b0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.995+0000 7f9768835700 1 --2- 172.21.15.29:0/601009371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97641350a0 0x7f9764152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:29.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.995+0000 7f9768835700 1 -- 172.21.15.29:0/601009371 >> 172.21.15.29:0/601009371 conn(0x7f97640c8bb0 msgr2=0x7f97640c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:29.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.995+0000 7f9768835700 1 -- 172.21.15.29:0/601009371 shutdown_connections 2024-01-24T14:14:29.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.995+0000 7f9768835700 1 -- 172.21.15.29:0/601009371 wait complete. 2024-01-24T14:14:29.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.996+0000 7f9768835700 1 Processor -- start 2024-01-24T14:14:29.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.996+0000 7f9768835700 1 -- start start 2024-01-24T14:14:29.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f9768835700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97640664f0 0x7f976414cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:29.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f9768835700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9764134cc0 0x7f9764164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f9768835700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97641350a0 0x7f976414f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f9768835700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976414f980 con 0x7f97640664f0 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f9768835700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9764164760 con 0x7f97641350a0 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f9768835700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9764164130 con 0x7f9764134cc0 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f976259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97640664f0 0x7f976414cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f976259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97640664f0 0x7f976414cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49952/0 (socket says 172.21.15.29:49952) 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f976259c700 1 -- 172.21.15.29:0/1896351170 learned_addr learned my addr 172.21.15.29:0/1896351170 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f976259c700 1 -- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9764134cc0 msgr2=0x7f9764164bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f9761d9b700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9764134cc0 0x7f9764164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f9762d9d700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97641350a0 0x7f976414f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f976259c700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9764134cc0 0x7f9764164bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f976259c700 1 -- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97641350a0 msgr2=0x7f976414f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f976259c700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97641350a0 0x7f976414f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:30.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f976259c700 1 -- 172.21.15.29:0/1896351170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580092a0 con 0x7f97640664f0 2024-01-24T14:14:30.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f9761d9b700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9764134cc0 0x7f9764164bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:30.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.997+0000 7f9762d9d700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97641350a0 0x7f976414f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:30.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.998+0000 7f976259c700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97640664f0 0x7f976414cdb0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7f976406a130 tx=0x7f9758017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:30.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.998+0000 7f97527fc700 1 -- 172.21.15.29:0/1896351170 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975800f7e0 con 0x7f97640664f0 2024-01-24T14:14:30.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.998+0000 7f97527fc700 1 -- 172.21.15.29:0/1896351170 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975800f940 con 0x7f97640664f0 2024-01-24T14:14:30.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.998+0000 7f9768835700 1 -- 172.21.15.29:0/1896351170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976414d400 con 0x7f97640664f0 2024-01-24T14:14:30.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.998+0000 7f97527fc700 1 -- 172.21.15.29:0/1896351170 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975801b660 con 0x7f97640664f0 2024-01-24T14:14:30.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.998+0000 7f9768835700 1 -- 172.21.15.29:0/1896351170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97641643f0 con 0x7f97640664f0 2024-01-24T14:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.999+0000 7f9768835700 1 -- 172.21.15.29:0/1896351170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9764164a40 con 0x7f97640664f0 2024-01-24T14:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:29.999+0000 7f97527fc700 1 -- 172.21.15.29:0/1896351170 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f975801ba50 con 0x7f97640664f0 2024-01-24T14:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.000+0000 7f97527fc700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f974405f240 0x7f9744061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.000+0000 7f97527fc700 1 -- 172.21.15.29:0/1896351170 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6849+0+0 (secure 0 0 0) 0x7f975807bfa0 con 0x7f97640664f0 2024-01-24T14:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.000+0000 7f9761d9b700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f974405f240 0x7f9744061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.000+0000 7f9761d9b700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f974405f240 0x7f9744061700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9754006d80 tx=0x7f9754008040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:30.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.004+0000 7f97527fc700 1 -- 172.21.15.29:0/1896351170 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f97580aab00 con 0x7f97640664f0 2024-01-24T14:14:30.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.233+0000 7f9768835700 1 -- 172.21.15.29:0/1896351170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97640475e0 con 0x7f97640664f0 2024-01-24T14:14:30.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.233+0000 7f97527fc700 1 -- 172.21.15.29:0/1896351170 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f9758029020 con 0x7f97640664f0 2024-01-24T14:14:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:30.234 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4083072158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":4083072158},{"type":"v1","addr":"172.21.15.163:6837","nonce":4083072158}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/4114592089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4114592089},{"type":"v1","addr":"172.21.15.163:6835","nonce":4114592089}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/926851304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":926851304},{"type":"v1","addr":"172.21.15.29:6835","nonce":926851304}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":7182,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3662577235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3662577235},{"type":"v1","addr":"172.21.15.29:6833","nonce":3662577235}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[]} 2024-01-24T14:14:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.235+0000 7f9768835700 1 -- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f974405f240 msgr2=0x7f9744061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.235+0000 7f9768835700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f974405f240 0x7f9744061700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9754006d80 tx=0x7f9754008040).stop 2024-01-24T14:14:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.235+0000 7f9768835700 1 -- 172.21.15.29:0/1896351170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97640664f0 msgr2=0x7f976414cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.235+0000 7f9768835700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97640664f0 0x7f976414cdb0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7f976406a130 tx=0x7f9758017000).stop 2024-01-24T14:14:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.235+0000 7f9768835700 1 -- 172.21.15.29:0/1896351170 shutdown_connections 2024-01-24T14:14:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.235+0000 7f9768835700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97641350a0 0x7f976414f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.235+0000 7f9768835700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f974405f240 0x7f9744061700 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.235+0000 7f9768835700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97640664f0 0x7f976414cdb0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.235+0000 7f9768835700 1 --2- 172.21.15.29:0/1896351170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9764134cc0 0x7f9764164bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.235+0000 7f9768835700 1 -- 172.21.15.29:0/1896351170 >> 172.21.15.29:0/1896351170 conn(0x7f97640c8bb0 msgr2=0x7f9764151300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.235+0000 7f9768835700 1 -- 172.21.15.29:0/1896351170 shutdown_connections 2024-01-24T14:14:30.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.235+0000 7f9768835700 1 -- 172.21.15.29:0/1896351170 wait complete. 2024-01-24T14:14:30.238 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 45 2024-01-24T14:14:30.249 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-24T14:14:30.250 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-24T14:14:30.250 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-24T14:14:30.250 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:14:30.250 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-24T14:14:30.250 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-24T14:14:30.250 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-24T14:14:30.250 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:14:30.251 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-24T14:14:30.251 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-24T14:14:30.251 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-24T14:14:30.251 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:14:30.251 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-24T14:14:30.251 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-24T14:14:30.251 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-24T14:14:30.252 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:14:30.260 INFO:tasks.ceph.mds.a:Stopped 2024-01-24T14:14:30.260 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-24T14:14:30.261 INFO:tasks.ceph.mds.c:Stopped 2024-01-24T14:14:30.261 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-24T14:14:30.262 INFO:tasks.ceph.mds.b:Stopped 2024-01-24T14:14:30.263 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-24T14:14:30.263 INFO:tasks.ceph.mds.d:Stopped 2024-01-24T14:14:30.263 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-24T14:14:30.303 INFO:tasks.ceph.mds.a:Started 2024-01-24T14:14:30.303 DEBUG:teuthology.parallel:result is None 2024-01-24T14:14:30.304 INFO:tasks.ceph.mds.c:Started 2024-01-24T14:14:30.305 DEBUG:teuthology.parallel:result is None 2024-01-24T14:14:30.306 INFO:tasks.ceph.mds.b:Started 2024-01-24T14:14:30.306 DEBUG:teuthology.parallel:result is None 2024-01-24T14:14:30.306 INFO:tasks.ceph.mds.d:Started 2024-01-24T14:14:30.306 DEBUG:teuthology.parallel:result is None 2024-01-24T14:14:30.307 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-24T14:14:30.390 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T14:14:30.390 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T14:14:30.391 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-01-24T14:14:30.390+0000 7f6167040700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-01-24T14:14:30.393 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T14:14:30.393 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-01-24T14:14:30.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.447+0000 7fe635b02700 1 Processor -- start 2024-01-24T14:14:30.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.447+0000 7fe635b02700 1 -- start start 2024-01-24T14:14:30.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.448+0000 7fe635b02700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630135350 0x7fe6300664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:30.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.448+0000 7fe635b02700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe630152d00 0x7fe6300bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:30.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.448+0000 7fe635b02700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe630156460 con 0x7fe630066990 2024-01-24T14:14:30.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.448+0000 7fe635b02700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6301531b0 con 0x7fe630152d00 2024-01-24T14:14:30.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.448+0000 7fe635b02700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe630155c00 con 0x7fe630135350 2024-01-24T14:14:30.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.448+0000 7fe62effd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe630066990 0x7fe6301525f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40364/0 (socket says 172.21.15.29:40364) 2024-01-24T14:14:30.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.448+0000 7fe62effd700 1 -- 172.21.15.29:0/2949527319 learned_addr learned my addr 172.21.15.29:0/2949527319 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:30.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62f7fe700 1 --2- 172.21.15.29:0/2949527319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630135350 0x7fe6300664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:30.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62ffff700 1 --2- 172.21.15.29:0/2949527319 >> v2:172.21.15.163:3300/0 conn(0x7fe630152d00 0x7fe6300bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:30.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62e7fc700 1 -- 172.21.15.29:0/2949527319 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3278600431 0 0) 0x7fe630156460 con 0x7fe630066990 2024-01-24T14:14:30.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62e7fc700 1 -- 172.21.15.29:0/2949527319 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6180033f0 con 0x7fe630066990 2024-01-24T14:14:30.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62ffff700 1 -- 172.21.15.29:0/2949527319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630135350 msgr2=0x7fe6300664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62ffff700 1 --2- 172.21.15.29:0/2949527319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630135350 0x7fe6300664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62ffff700 1 -- 172.21.15.29:0/2949527319 >> v1:172.21.15.29:6789/0 conn(0x7fe630066990 legacy=0x7fe6301525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62ffff700 1 -- 172.21.15.29:0/2949527319 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe630066e10 con 0x7fe630152d00 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62f7fe700 1 --2- 172.21.15.29:0/2949527319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630135350 0x7fe6300664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62ffff700 1 --2- 172.21.15.29:0/2949527319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630152d00 0x7fe6300bdb00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fe628009e60 tx=0x7fe628009780).ready entity=mon.1 client_cookie=9b06fd0de385ba9c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62e7fc700 1 -- 172.21.15.29:0/2949527319 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62800e070 con 0x7fe630152d00 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62e7fc700 1 -- 172.21.15.29:0/2949527319 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62800bab0 con 0x7fe630152d00 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.449+0000 7fe62e7fc700 1 -- 172.21.15.29:0/2949527319 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62800be30 con 0x7fe630152d00 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.450+0000 7fe635b02700 1 -- 172.21.15.29:0/2949527319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630152d00 msgr2=0x7fe6300bdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.450+0000 7fe635b02700 1 --2- 172.21.15.29:0/2949527319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630152d00 0x7fe6300bdb00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fe628009e60 tx=0x7fe628009780).stop 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.450+0000 7fe635b02700 1 -- 172.21.15.29:0/2949527319 shutdown_connections 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.450+0000 7fe635b02700 1 --2- 172.21.15.29:0/2949527319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630152d00 0x7fe6300bdb00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.450+0000 7fe635b02700 1 --2- 172.21.15.29:0/2949527319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630135350 0x7fe6300664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:30.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.450+0000 7fe635b02700 1 -- 172.21.15.29:0/2949527319 >> 172.21.15.29:0/2949527319 conn(0x7fe6300c5640 msgr2=0x7fe6300c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:30.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.450+0000 7fe635b02700 1 -- 172.21.15.29:0/2949527319 shutdown_connections 2024-01-24T14:14:30.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.450+0000 7fe635b02700 1 -- 172.21.15.29:0/2949527319 wait complete. 2024-01-24T14:14:30.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.451+0000 7fe635b02700 1 Processor -- start 2024-01-24T14:14:30.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.451+0000 7fe635b02700 1 -- start start 2024-01-24T14:14:30.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.452+0000 7fe635b02700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630066990 0x7fe6300ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:30.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.452+0000 7fe635b02700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630135350 0x7fe6301383d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:30.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.452+0000 7fe635b02700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe630152d00 0x7fe630165590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.452+0000 7fe635b02700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6300645e0 con 0x7fe630152d00 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.452+0000 7fe635b02700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe630063960 con 0x7fe630135350 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.452+0000 7fe635b02700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe630063c40 con 0x7fe630066990 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.452+0000 7fe62f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630066990 0x7fe6300ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.452+0000 7fe62ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe630152d00 0x7fe630165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.452+0000 7fe62effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630135350 0x7fe6301383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.452+0000 7fe62f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630066990 0x7fe6300ac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:34696/0 (socket says 172.21.15.29:34696) 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.452+0000 7fe62f7fe700 1 -- 172.21.15.29:0/324710218 learned_addr learned my addr 172.21.15.29:0/324710218 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.453+0000 7fe62f7fe700 1 -- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630135350 msgr2=0x7fe6301383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.453+0000 7fe62f7fe700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630135350 0x7fe6301383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.453+0000 7fe62f7fe700 1 -- 172.21.15.29:0/324710218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe630152d00 msgr2=0x7fe630165590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.453+0000 7fe62f7fe700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe630152d00 0x7fe630165590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.453+0000 7fe62f7fe700 1 -- 172.21.15.29:0/324710218 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6200092a0 con 0x7fe630066990 2024-01-24T14:14:30.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.453+0000 7fe62ffff700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe630152d00 0x7fe630165590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:30.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.453+0000 7fe62f7fe700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630066990 0x7fe6300ac1b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fe62000ea20 tx=0x7fe620009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:30.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.453+0000 7fe6177fe700 1 -- 172.21.15.29:0/324710218 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62000ee60 con 0x7fe630066990 2024-01-24T14:14:30.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.453+0000 7fe6177fe700 1 -- 172.21.15.29:0/324710218 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62000c850 con 0x7fe630066990 2024-01-24T14:14:30.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.453+0000 7fe6177fe700 1 -- 172.21.15.29:0/324710218 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62000cbd0 con 0x7fe630066990 2024-01-24T14:14:30.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.453+0000 7fe635b02700 1 -- 172.21.15.29:0/324710218 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe628009a20 con 0x7fe630066990 2024-01-24T14:14:30.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.453+0000 7fe635b02700 1 -- 172.21.15.29:0/324710218 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe630064200 con 0x7fe630066990 2024-01-24T14:14:30.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.454+0000 7fe635b02700 1 -- 172.21.15.29:0/324710218 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe630064850 con 0x7fe630066990 2024-01-24T14:14:30.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.454+0000 7fe6177fe700 1 -- 172.21.15.29:0/324710218 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe62001b400 con 0x7fe630066990 2024-01-24T14:14:30.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.455+0000 7fe6177fe700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe61c05f270 0x7fe61c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:30.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.455+0000 7fe6177fe700 1 -- 172.21.15.29:0/324710218 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6849+0+0 (secure 0 0 0) 0x7fe62007ab90 con 0x7fe630066990 2024-01-24T14:14:30.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.455+0000 7fe62effd700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe61c05f270 0x7fe61c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:30.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.455+0000 7fe62effd700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe61c05f270 0x7fe61c061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe630065a10 tx=0x7fe624009040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:30.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.460+0000 7fe6177fe700 1 -- 172.21.15.29:0/324710218 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe62004ba50 con 0x7fe630066990 2024-01-24T14:14:30.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:30.656+0000 7fe635b02700 1 -- 172.21.15.29:0/324710218 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe6300475e0 con 0x7fe630066990 2024-01-24T14:14:31.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.427+0000 7fe6177fe700 1 -- 172.21.15.29:0/324710218 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v116) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe62004f070 con 0x7fe630066990 2024-01-24T14:14:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.430+0000 7fe635b02700 1 -- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe61c05f270 msgr2=0x7fe61c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.430+0000 7fe635b02700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe61c05f270 0x7fe61c061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe630065a10 tx=0x7fe624009040).stop 2024-01-24T14:14:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.430+0000 7fe635b02700 1 -- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630066990 msgr2=0x7fe6300ac1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.430+0000 7fe635b02700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630066990 0x7fe6300ac1b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fe62000ea20 tx=0x7fe620009060).stop 2024-01-24T14:14:31.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.431+0000 7fe635b02700 1 -- 172.21.15.29:0/324710218 shutdown_connections 2024-01-24T14:14:31.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.431+0000 7fe635b02700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe61c05f270 0x7fe61c061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:31.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.431+0000 7fe635b02700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe630152d00 0x7fe630165590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:31.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.431+0000 7fe635b02700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe630066990 0x7fe6300ac1b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:31.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.431+0000 7fe635b02700 1 --2- 172.21.15.29:0/324710218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630135350 0x7fe6301383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:31.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.431+0000 7fe635b02700 1 -- 172.21.15.29:0/324710218 >> 172.21.15.29:0/324710218 conn(0x7fe6300c5640 msgr2=0x7fe63014b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:31.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.431+0000 7fe635b02700 1 -- 172.21.15.29:0/324710218 shutdown_connections 2024-01-24T14:14:31.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.431+0000 7fe635b02700 1 -- 172.21.15.29:0/324710218 wait complete. 2024-01-24T14:14:31.432 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-01-24T14:14:31.448 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-24T14:14:31.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.585+0000 7f0cec97f700 1 Processor -- start 2024-01-24T14:14:31.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.585+0000 7f0cec97f700 1 -- start start 2024-01-24T14:14:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.586+0000 7f0cec97f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0ce8069240 0x7f0ce8065ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.586+0000 7f0cec97f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce80c2ce0 con 0x7f0ce8150d40 2024-01-24T14:14:31.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.586+0000 7f0cec97f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce80663b0 con 0x7f0ce8155040 2024-01-24T14:14:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.586+0000 7f0cec97f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce8069890 con 0x7f0ce8069240 2024-01-24T14:14:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.586+0000 7f0ce6d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0ce8069240 0x7f0ce8065ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.586+0000 7f0ce659c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0ce8150d40 0x7f0ce8154930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40376/0 (socket says 172.21.15.29:40376) 2024-01-24T14:14:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.586+0000 7f0ce659c700 1 -- 172.21.15.29:0/3623755188 learned_addr learned my addr 172.21.15.29:0/3623755188 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.586+0000 7f0ce6d9d700 1 -- 172.21.15.29:0/3623755188 >> v1:172.21.15.163:6789/0 conn(0x7f0ce8155040 legacy=0x7f0ce80c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.586+0000 7f0ce6d9d700 1 -- 172.21.15.29:0/3623755188 >> v1:172.21.15.29:6789/0 conn(0x7f0ce8150d40 legacy=0x7f0ce8154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.586+0000 7f0ce6d9d700 1 -- 172.21.15.29:0/3623755188 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce8069b70 con 0x7f0ce8069240 2024-01-24T14:14:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.586+0000 7f0ce559a700 1 -- 172.21.15.29:0/3623755188 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803325587 0 0) 0x7f0ce80c2ce0 con 0x7f0ce8150d40 2024-01-24T14:14:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.587+0000 7f0ce6d9d700 1 --2- 172.21.15.29:0/3623755188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ce8069240 0x7f0ce8065ec0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7f0ce0009e60 tx=0x7f0ce0009780).ready entity=mon.0 client_cookie=fa0ee0efe56ba802 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.587+0000 7f0ce559a700 1 -- 172.21.15.29:0/3623755188 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000e070 con 0x7f0ce8069240 2024-01-24T14:14:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.587+0000 7f0ce559a700 1 -- 172.21.15.29:0/3623755188 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce000bc30 con 0x7f0ce8069240 2024-01-24T14:14:31.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.587+0000 7f0ce559a700 1 -- 172.21.15.29:0/3623755188 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0012410 con 0x7f0ce8069240 2024-01-24T14:14:31.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.587+0000 7f0cec97f700 1 -- 172.21.15.29:0/3623755188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ce8069240 msgr2=0x7f0ce8065ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:31.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.587+0000 7f0cec97f700 1 --2- 172.21.15.29:0/3623755188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ce8069240 0x7f0ce8065ec0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7f0ce0009e60 tx=0x7f0ce0009780).stop 2024-01-24T14:14:31.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.587+0000 7f0cec97f700 1 -- 172.21.15.29:0/3623755188 shutdown_connections 2024-01-24T14:14:31.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.587+0000 7f0cec97f700 1 --2- 172.21.15.29:0/3623755188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ce8069240 0x7f0ce8065ec0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:31.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.587+0000 7f0cec97f700 1 -- 172.21.15.29:0/3623755188 >> 172.21.15.29:0/3623755188 conn(0x7f0ce80c8580 msgr2=0x7f0ce80c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:31.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.587+0000 7f0cec97f700 1 -- 172.21.15.29:0/3623755188 shutdown_connections 2024-01-24T14:14:31.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.587+0000 7f0cec97f700 1 -- 172.21.15.29:0/3623755188 wait complete. 2024-01-24T14:14:31.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.589+0000 7f0cec97f700 1 Processor -- start 2024-01-24T14:14:31.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.589+0000 7f0cec97f700 1 -- start start 2024-01-24T14:14:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0cec97f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ce8069240 0x7f0ce80bfab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:31.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0cec97f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ce8150d40 0x7f0ce8153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0cec97f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ce8155040 0x7f0ce8149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0cec97f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce80c0780 con 0x7f0ce8150d40 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0cec97f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce80c0a60 con 0x7f0ce8069240 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0cec97f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce80c0d40 con 0x7f0ce8155040 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0ce5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ce8150d40 0x7f0ce8153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0ce5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ce8150d40 0x7f0ce8153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50012/0 (socket says 172.21.15.29:50012) 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0ce5d9b700 1 -- 172.21.15.29:0/2400558671 learned_addr learned my addr 172.21.15.29:0/2400558671 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0ce659c700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ce8069240 0x7f0ce80bfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0ce6d9d700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ce8155040 0x7f0ce8149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0ce5d9b700 1 -- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ce8155040 msgr2=0x7f0ce8149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0ce5d9b700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ce8155040 0x7f0ce8149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0ce5d9b700 1 -- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ce8069240 msgr2=0x7f0ce80bfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.590+0000 7f0ce5d9b700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ce8069240 0x7f0ce80bfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:31.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.591+0000 7f0ce5d9b700 1 -- 172.21.15.29:0/2400558671 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce0009a20 con 0x7f0ce8150d40 2024-01-24T14:14:31.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.591+0000 7f0ce659c700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ce8069240 0x7f0ce80bfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:31.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.591+0000 7f0ce5d9b700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ce8150d40 0x7f0ce8153b00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f0cdc00c440 tx=0x7f0cdc007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:31.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.591+0000 7f0cd67fc700 1 -- 172.21.15.29:0/2400558671 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc007020 con 0x7f0ce8150d40 2024-01-24T14:14:31.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.591+0000 7f0cd67fc700 1 -- 172.21.15.29:0/2400558671 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc007180 con 0x7f0ce8150d40 2024-01-24T14:14:31.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.591+0000 7f0cec97f700 1 -- 172.21.15.29:0/2400558671 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce80c1020 con 0x7f0ce8150d40 2024-01-24T14:14:31.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.591+0000 7f0cd67fc700 1 -- 172.21.15.29:0/2400558671 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc007520 con 0x7f0ce8150d40 2024-01-24T14:14:31.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.591+0000 7f0cec97f700 1 -- 172.21.15.29:0/2400558671 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce80bee00 con 0x7f0ce8150d40 2024-01-24T14:14:31.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.592+0000 7f0cec97f700 1 -- 172.21.15.29:0/2400558671 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce80475e0 con 0x7f0ce8150d40 2024-01-24T14:14:31.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.592+0000 7f0cd67fc700 1 -- 172.21.15.29:0/2400558671 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0cdc0068b0 con 0x7f0ce8150d40 2024-01-24T14:14:31.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.593+0000 7f0cd67fc700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0cc805f270 0x7f0cc8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:31.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.593+0000 7f0cd67fc700 1 -- 172.21.15.29:0/2400558671 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f0cdc079f30 con 0x7f0ce8150d40 2024-01-24T14:14:31.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.593+0000 7f0ce659c700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0cc805f270 0x7f0cc8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:31.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.594+0000 7f0ce659c700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0cc805f270 0x7f0cc8061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f0cd80011e0 tx=0x7f0cd8009040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:31.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.597+0000 7f0cd67fc700 1 -- 172.21.15.29:0/2400558671 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0cdc04af10 con 0x7f0ce8150d40 2024-01-24T14:14:31.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.850+0000 7f0cec97f700 1 -- 172.21.15.29:0/2400558671 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0ce80c51a0 con 0x7f0ce8150d40 2024-01-24T14:14:31.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.850+0000 7f0cd67fc700 1 -- 172.21.15.29:0/2400558671 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0cdc04e530 con 0x7f0ce8150d40 2024-01-24T14:14:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:31.851 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T14:14:31.851 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew==", 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-01-24T14:14:31.852 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ==", 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw==", 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:14:31.853 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ==", 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-01-24T14:14:31.854 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ==", 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA==", 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw==", 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ==", 2024-01-24T14:14:31.856 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw==", 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ==", 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A==", 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:14:31.858 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ==", 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-01-24T14:14:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw==", 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA==", 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:14:31.860 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g==", 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-01-24T14:14:31.861 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ==", 2024-01-24T14:14:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:14:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:14:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:14:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:14:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlxK1bFxAAZpfJn+Tcpmq7R9Yq29FvVg==", 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw==", 2024-01-24T14:14:31.863 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:14:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T14:14:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-01-24T14:14:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA==", 2024-01-24T14:14:31.864 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:14:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:14:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T14:14:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:14:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:31.865 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-01-24T14:14:31.866 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T14:14:31.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.853+0000 7f0cec97f700 1 -- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0cc805f270 msgr2=0x7f0cc8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:31.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.853+0000 7f0cec97f700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0cc805f270 0x7f0cc8061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f0cd80011e0 tx=0x7f0cd8009040).stop 2024-01-24T14:14:31.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.853+0000 7f0cec97f700 1 -- 172.21.15.29:0/2400558671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ce8150d40 msgr2=0x7f0ce8153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:31.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.853+0000 7f0cec97f700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ce8150d40 0x7f0ce8153b00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f0cdc00c440 tx=0x7f0cdc007dd0).stop 2024-01-24T14:14:31.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.853+0000 7f0cec97f700 1 -- 172.21.15.29:0/2400558671 shutdown_connections 2024-01-24T14:14:31.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.853+0000 7f0cec97f700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ce8155040 0x7f0ce8149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:31.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.853+0000 7f0cec97f700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ce8069240 0x7f0ce80bfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:31.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.853+0000 7f0cec97f700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0cc805f270 0x7f0cc8061730 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:31.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.853+0000 7f0cec97f700 1 --2- 172.21.15.29:0/2400558671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ce8150d40 0x7f0ce8153b00 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:31.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.853+0000 7f0cec97f700 1 -- 172.21.15.29:0/2400558671 >> 172.21.15.29:0/2400558671 conn(0x7f0ce80c8580 msgr2=0x7f0ce8149320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:31.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.853+0000 7f0cec97f700 1 -- 172.21.15.29:0/2400558671 shutdown_connections 2024-01-24T14:14:31.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:31.853+0000 7f0cec97f700 1 -- 172.21.15.29:0/2400558671 wait complete. 2024-01-24T14:14:31.870 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:14:32.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.000+0000 7fb162528700 1 Processor -- start 2024-01-24T14:14:32.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.000+0000 7fb162528700 1 -- start start 2024-01-24T14:14:32.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb162528700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb15c1543c0 0x7fb15c0bcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb162528700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb15c0bd120 0x7fb15c0c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb162528700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb15c05e850 con 0x7fb15c153fe0 2024-01-24T14:14:32.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb162528700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb15c0a4f90 con 0x7fb15c1543c0 2024-01-24T14:14:32.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb162528700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb15c0631f0 con 0x7fb15c0bd120 2024-01-24T14:14:32.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb15bfff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb15c153fe0 0x7fb15c0a4880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40390/0 (socket says 172.21.15.29:40390) 2024-01-24T14:14:32.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb15bfff700 1 -- 172.21.15.29:0/2900220826 learned_addr learned my addr 172.21.15.29:0/2900220826 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:32.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb15b7fe700 1 --2- 172.21.15.29:0/2900220826 >> v2:172.21.15.29:3300/0 conn(0x7fb15c1543c0 0x7fb15c0bcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb160ac5700 1 --2- 172.21.15.29:0/2900220826 >> v2:172.21.15.163:3300/0 conn(0x7fb15c0bd120 0x7fb15c0c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb15affd700 1 -- 172.21.15.29:0/2900220826 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300935262 0 0) 0x7fb15c05e850 con 0x7fb15c153fe0 2024-01-24T14:14:32.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb15affd700 1 -- 172.21.15.29:0/2900220826 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1440033f0 con 0x7fb15c153fe0 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb15b7fe700 1 -- 172.21.15.29:0/2900220826 >> v2:172.21.15.163:3300/0 conn(0x7fb15c0bd120 msgr2=0x7fb15c0c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb15b7fe700 1 --2- 172.21.15.29:0/2900220826 >> v2:172.21.15.163:3300/0 conn(0x7fb15c0bd120 0x7fb15c0c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb15b7fe700 1 -- 172.21.15.29:0/2900220826 >> v1:172.21.15.29:6789/0 conn(0x7fb15c153fe0 legacy=0x7fb15c0a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb15b7fe700 1 -- 172.21.15.29:0/2900220826 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c0bd7e0 con 0x7fb15c1543c0 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.001+0000 7fb15b7fe700 1 --2- 172.21.15.29:0/2900220826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb15c1543c0 0x7fb15c0bcc30 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7fb15c065ac0 tx=0x7fb150009060).ready entity=mon.0 client_cookie=5bdfddc51d33906e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.002+0000 7fb15affd700 1 -- 172.21.15.29:0/2900220826 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15000f510 con 0x7fb15c1543c0 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.002+0000 7fb15affd700 1 -- 172.21.15.29:0/2900220826 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15000f670 con 0x7fb15c1543c0 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.002+0000 7fb15affd700 1 -- 172.21.15.29:0/2900220826 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15000f9f0 con 0x7fb15c1543c0 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.002+0000 7fb162528700 1 -- 172.21.15.29:0/2900220826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb15c1543c0 msgr2=0x7fb15c0bcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.002+0000 7fb162528700 1 --2- 172.21.15.29:0/2900220826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb15c1543c0 0x7fb15c0bcc30 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7fb15c065ac0 tx=0x7fb150009060).stop 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.002+0000 7fb162528700 1 -- 172.21.15.29:0/2900220826 shutdown_connections 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.002+0000 7fb162528700 1 --2- 172.21.15.29:0/2900220826 >> v2:172.21.15.163:3300/0 conn(0x7fb15c0bd120 0x7fb15c0c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.002+0000 7fb162528700 1 --2- 172.21.15.29:0/2900220826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb15c1543c0 0x7fb15c0bcc30 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.002+0000 7fb162528700 1 -- 172.21.15.29:0/2900220826 >> 172.21.15.29:0/2900220826 conn(0x7fb15c064aa0 msgr2=0x7fb15c064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:32.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.002+0000 7fb162528700 1 -- 172.21.15.29:0/2900220826 shutdown_connections 2024-01-24T14:14:32.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.002+0000 7fb162528700 1 -- 172.21.15.29:0/2900220826 wait complete. 2024-01-24T14:14:32.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.004+0000 7fb162528700 1 Processor -- start 2024-01-24T14:14:32.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.004+0000 7fb162528700 1 -- start start 2024-01-24T14:14:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.005+0000 7fb162528700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb15c0bd120 0x7fb15c155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.005+0000 7fb162528700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb15c153fe0 0x7fb15c151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.005+0000 7fb162528700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb15c1543c0 0x7fb15c1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.005+0000 7fb162528700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0a8530 con 0x7fb15c0bd120 2024-01-24T14:14:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.005+0000 7fb162528700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0aa220 con 0x7fb15c1543c0 2024-01-24T14:14:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.005+0000 7fb162528700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c151780 con 0x7fb15c153fe0 2024-01-24T14:14:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.005+0000 7fb15bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb15c0bd120 0x7fb15c155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.005+0000 7fb15bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb15c0bd120 0x7fb15c155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50038/0 (socket says 172.21.15.29:50038) 2024-01-24T14:14:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.005+0000 7fb15bfff700 1 -- 172.21.15.29:0/2336464127 learned_addr learned my addr 172.21.15.29:0/2336464127 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:32.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb160ac5700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb15c1543c0 0x7fb15c1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb15b7fe700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb15c153fe0 0x7fb15c151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb15bfff700 1 -- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb15c153fe0 msgr2=0x7fb15c151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb15bfff700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb15c153fe0 0x7fb15c151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb15bfff700 1 -- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb15c1543c0 msgr2=0x7fb15c1571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb15bfff700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb15c1543c0 0x7fb15c1571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb15bfff700 1 -- 172.21.15.29:0/2336464127 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1500092a0 con 0x7fb15c0bd120 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb15b7fe700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb15c153fe0 0x7fb15c151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb160ac5700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb15c1543c0 0x7fb15c1571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb15bfff700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb15c0bd120 0x7fb15c155310 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7fb14c00c590 tx=0x7fb14c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb143fff700 1 -- 172.21.15.29:0/2336464127 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c007e40 con 0x7fb15c0bd120 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb143fff700 1 -- 172.21.15.29:0/2336464127 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14c0072e0 con 0x7fb15c0bd120 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb162528700 1 -- 172.21.15.29:0/2336464127 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c155950 con 0x7fb15c0bd120 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb143fff700 1 -- 172.21.15.29:0/2336464127 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0057a0 con 0x7fb15c0bd120 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.006+0000 7fb162528700 1 -- 172.21.15.29:0/2336464127 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c155f10 con 0x7fb15c0bd120 2024-01-24T14:14:32.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.007+0000 7fb162528700 1 -- 172.21.15.29:0/2336464127 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15c0475e0 con 0x7fb15c0bd120 2024-01-24T14:14:32.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.007+0000 7fb143fff700 1 -- 172.21.15.29:0/2336464127 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb14c005bd0 con 0x7fb15c0bd120 2024-01-24T14:14:32.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.008+0000 7fb143fff700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb14805f270 0x7fb148061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.008+0000 7fb143fff700 1 -- 172.21.15.29:0/2336464127 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6290+0+0 (secure 0 0 0) 0x7fb14c079d60 con 0x7fb15c0bd120 2024-01-24T14:14:32.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.008+0000 7fb15b7fe700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb14805f270 0x7fb148061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.009+0000 7fb15b7fe700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb14805f270 0x7fb148061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb15c06ae30 tx=0x7fb150017000).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:32.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.012+0000 7fb143fff700 1 -- 172.21.15.29:0/2336464127 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb14c04ae00 con 0x7fb15c0bd120 2024-01-24T14:14:32.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.194+0000 7fb162528700 1 -- 172.21.15.29:0/2336464127 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb15c05e330 con 0x7fb15c0bd120 2024-01-24T14:14:32.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.195+0000 7fb143fff700 1 -- 172.21.15.29:0/2336464127 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+14965 (secure 0 0 0) 0x7fb14c04e420 con 0x7fb15c0bd120 2024-01-24T14:14:32.196 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:32.196 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":116,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:14:31.420298+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:14:32.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.198+0000 7fb162528700 1 -- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb14805f270 msgr2=0x7fb148061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.198+0000 7fb162528700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb14805f270 0x7fb148061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb15c06ae30 tx=0x7fb150017000).stop 2024-01-24T14:14:32.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.198+0000 7fb162528700 1 -- 172.21.15.29:0/2336464127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb15c0bd120 msgr2=0x7fb15c155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.198+0000 7fb162528700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb15c0bd120 0x7fb15c155310 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7fb14c00c590 tx=0x7fb14c007c80).stop 2024-01-24T14:14:32.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.198+0000 7fb162528700 1 -- 172.21.15.29:0/2336464127 shutdown_connections 2024-01-24T14:14:32.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.198+0000 7fb162528700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb15c1543c0 0x7fb15c1571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.198+0000 7fb162528700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb14805f270 0x7fb148061730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.198+0000 7fb162528700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb15c0bd120 0x7fb15c155310 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.198+0000 7fb162528700 1 --2- 172.21.15.29:0/2336464127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb15c153fe0 0x7fb15c151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.198+0000 7fb162528700 1 -- 172.21.15.29:0/2336464127 >> 172.21.15.29:0/2336464127 conn(0x7fb15c064aa0 msgr2=0x7fb15c0bc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:32.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.198+0000 7fb162528700 1 -- 172.21.15.29:0/2336464127 shutdown_connections 2024-01-24T14:14:32.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.198+0000 7fb162528700 1 -- 172.21.15.29:0/2336464127 wait complete. 2024-01-24T14:14:32.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T14:14:32.212 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T14:14:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.344+0000 7f1788cd6700 1 Processor -- start 2024-01-24T14:14:32.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.344+0000 7f1788cd6700 1 -- start start 2024-01-24T14:14:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.345+0000 7f1788cd6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17840c8290 0x7f17840627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.345+0000 7f1788cd6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1784154320 0x7f1784062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.345+0000 7f1788cd6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178406a360 con 0x7f1784153f40 2024-01-24T14:14:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.345+0000 7f1788cd6700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1784063190 con 0x7f17840c8290 2024-01-24T14:14:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.345+0000 7f1788cd6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17840a74c0 con 0x7f1784154320 2024-01-24T14:14:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f1781d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17840c8290 0x7f17840627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f1781d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17840c8290 0x7f17840627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50046/0 (socket says 172.21.15.29:50046) 2024-01-24T14:14:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f1781d9b700 1 -- 172.21.15.29:0/3242925459 learned_addr learned my addr 172.21.15.29:0/3242925459 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f1782d9d700 1 --2- 172.21.15.29:0/3242925459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1784154320 0x7f1784062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f1781d9b700 1 -- 172.21.15.29:0/3242925459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1784154320 msgr2=0x7f1784062ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f1781d9b700 1 --2- 172.21.15.29:0/3242925459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1784154320 0x7f1784062ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f1781d9b700 1 -- 172.21.15.29:0/3242925459 >> v1:172.21.15.163:6789/0 conn(0x7f1784153f40 legacy=0x7f17840a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f1781d9b700 1 -- 172.21.15.29:0/3242925459 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17840bcef0 con 0x7f17840c8290 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f1782d9d700 1 --2- 172.21.15.29:0/3242925459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1784154320 0x7f1784062ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f1781d9b700 1 --2- 172.21.15.29:0/3242925459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17840c8290 0x7f17840627b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f1778006fd0 tx=0x7f17780090f0).ready entity=mon.0 client_cookie=3ecaf1f8007c9d65 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f178159a700 1 -- 172.21.15.29:0/3242925459 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177800b600 con 0x7f17840c8290 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f178159a700 1 -- 172.21.15.29:0/3242925459 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177800b760 con 0x7f17840c8290 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.346+0000 7f178159a700 1 -- 172.21.15.29:0/3242925459 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177800bb00 con 0x7f17840c8290 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.347+0000 7f1788cd6700 1 -- 172.21.15.29:0/3242925459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17840c8290 msgr2=0x7f17840627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.347+0000 7f1788cd6700 1 --2- 172.21.15.29:0/3242925459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17840c8290 0x7f17840627b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f1778006fd0 tx=0x7f17780090f0).stop 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.347+0000 7f1788cd6700 1 -- 172.21.15.29:0/3242925459 shutdown_connections 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.347+0000 7f1788cd6700 1 --2- 172.21.15.29:0/3242925459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1784154320 0x7f1784062ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.347+0000 7f1788cd6700 1 --2- 172.21.15.29:0/3242925459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17840c8290 0x7f17840627b0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.347+0000 7f1788cd6700 1 -- 172.21.15.29:0/3242925459 >> 172.21.15.29:0/3242925459 conn(0x7f1784064390 msgr2=0x7f17840c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:32.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.347+0000 7f1788cd6700 1 -- 172.21.15.29:0/3242925459 shutdown_connections 2024-01-24T14:14:32.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.347+0000 7f1788cd6700 1 -- 172.21.15.29:0/3242925459 wait complete. 2024-01-24T14:14:32.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.348+0000 7f1788cd6700 1 Processor -- start 2024-01-24T14:14:32.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.348+0000 7f1788cd6700 1 -- start start 2024-01-24T14:14:32.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.349+0000 7f1788cd6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17840c8290 0x7f1784155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.349+0000 7f1788cd6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1784153f40 0x7f1784150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.349+0000 7f1788cd6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1784154320 0x7f178414b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.349+0000 7f1788cd6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178414c5d0 con 0x7f1784154320 2024-01-24T14:14:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.349+0000 7f1788cd6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178414bd40 con 0x7f1784153f40 2024-01-24T14:14:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.349+0000 7f1788cd6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178414bfc0 con 0x7f17840c8290 2024-01-24T14:14:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.349+0000 7f1782d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1784154320 0x7f178414b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.349+0000 7f1782d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1784154320 0x7f178414b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50054/0 (socket says 172.21.15.29:50054) 2024-01-24T14:14:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.349+0000 7f178259c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17840c8290 0x7f1784155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.349+0000 7f1782d9d700 1 -- 172.21.15.29:0/3199122317 learned_addr learned my addr 172.21.15.29:0/3199122317 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.349+0000 7f1781d9b700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1784153f40 0x7f1784150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.350+0000 7f1782d9d700 1 -- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17840c8290 msgr2=0x7f1784155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.350+0000 7f1782d9d700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17840c8290 0x7f1784155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.350+0000 7f1782d9d700 1 -- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1784153f40 msgr2=0x7f1784150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.350+0000 7f1782d9d700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1784153f40 0x7f1784150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.350+0000 7f1782d9d700 1 -- 172.21.15.29:0/3199122317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1778009370 con 0x7f1784154320 2024-01-24T14:14:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.350+0000 7f1782d9d700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1784154320 0x7f178414b850 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f177c00d180 tx=0x7f177c009a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.350+0000 7f17727fc700 1 -- 172.21.15.29:0/3199122317 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c00dcc0 con 0x7f1784154320 2024-01-24T14:14:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.350+0000 7f17727fc700 1 -- 172.21.15.29:0/3199122317 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c00f040 con 0x7f1784154320 2024-01-24T14:14:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.350+0000 7f1788cd6700 1 -- 172.21.15.29:0/3199122317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17840c0300 con 0x7f1784154320 2024-01-24T14:14:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.350+0000 7f17727fc700 1 -- 172.21.15.29:0/3199122317 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c013620 con 0x7f1784154320 2024-01-24T14:14:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.350+0000 7f1788cd6700 1 -- 172.21.15.29:0/3199122317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17840c05e0 con 0x7f1784154320 2024-01-24T14:14:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.351+0000 7f1788cd6700 1 -- 172.21.15.29:0/3199122317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17840475e0 con 0x7f1784154320 2024-01-24T14:14:32.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.352+0000 7f17727fc700 1 -- 172.21.15.29:0/3199122317 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f177c013a10 con 0x7f1784154320 2024-01-24T14:14:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.353+0000 7f17727fc700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f176405f270 0x7f1764061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.353+0000 7f17727fc700 1 -- 172.21.15.29:0/3199122317 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f177c07aa90 con 0x7f1784154320 2024-01-24T14:14:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.353+0000 7f178259c700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f176405f270 0x7f1764061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.353+0000 7f178259c700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f176405f270 0x7f1764061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f1784152f60 tx=0x7f1774009040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:32.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.357+0000 7f17727fc700 1 -- 172.21.15.29:0/3199122317 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f177c04bad0 con 0x7f1784154320 2024-01-24T14:14:32.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.539+0000 7f1788cd6700 1 -- 172.21.15.29:0/3199122317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f178414c710 con 0x7f1784154320 2024-01-24T14:14:32.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.540+0000 7f17727fc700 1 -- 172.21.15.29:0/3199122317 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v116) v1 ==== 110+0+10 (secure 0 0 0) 0x7f177c04f0f0 con 0x7f1784154320 2024-01-24T14:14:32.541 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T14:14:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.542+0000 7f1788cd6700 1 -- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f176405f270 msgr2=0x7f1764061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.542+0000 7f1788cd6700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f176405f270 0x7f1764061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f1784152f60 tx=0x7f1774009040).stop 2024-01-24T14:14:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.542+0000 7f1788cd6700 1 -- 172.21.15.29:0/3199122317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1784154320 msgr2=0x7f178414b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.542+0000 7f1788cd6700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1784154320 0x7f178414b850 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f177c00d180 tx=0x7f177c009a50).stop 2024-01-24T14:14:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.542+0000 7f1788cd6700 1 -- 172.21.15.29:0/3199122317 shutdown_connections 2024-01-24T14:14:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.542+0000 7f1788cd6700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f176405f270 0x7f1764061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.542+0000 7f1788cd6700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1784154320 0x7f178414b850 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.542+0000 7f1788cd6700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17840c8290 0x7f1784155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.542+0000 7f1788cd6700 1 --2- 172.21.15.29:0/3199122317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1784153f40 0x7f1784150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.542+0000 7f1788cd6700 1 -- 172.21.15.29:0/3199122317 >> 172.21.15.29:0/3199122317 conn(0x7f1784064390 msgr2=0x7f17840c2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:32.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.542+0000 7f1788cd6700 1 -- 172.21.15.29:0/3199122317 shutdown_connections 2024-01-24T14:14:32.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.542+0000 7f1788cd6700 1 -- 172.21.15.29:0/3199122317 wait complete. 2024-01-24T14:14:32.555 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T14:14:32.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.688+0000 7f5f8e944700 1 Processor -- start 2024-01-24T14:14:32.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.688+0000 7f5f8e944700 1 -- start start 2024-01-24T14:14:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.689+0000 7f5f8e944700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5f88064760 0x7f5f88064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.689+0000 7f5f8e944700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5f880c8490 0x7f5f880a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.689+0000 7f5f8e944700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f88152240 con 0x7f5f88154700 2024-01-24T14:14:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.689+0000 7f5f8e944700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f881586c0 con 0x7f5f880c8490 2024-01-24T14:14:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.689+0000 7f5f8e944700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f880c0d50 con 0x7f5f88064760 2024-01-24T14:14:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.689+0000 7f5f877fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5f880c8490 0x7f5f880a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.689+0000 7f5f8cee1700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5f88154700 0x7f5f88154b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40394/0 (socket says 172.21.15.29:40394) 2024-01-24T14:14:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.689+0000 7f5f8cee1700 1 -- 172.21.15.29:0/1284376740 learned_addr learned my addr 172.21.15.29:0/1284376740 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.689+0000 7f5f87fff700 1 --2- 172.21.15.29:0/1284376740 >> v2:172.21.15.163:3300/0 conn(0x7f5f88064760 0x7f5f88064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f877fe700 1 -- 172.21.15.29:0/1284376740 >> v2:172.21.15.163:3300/0 conn(0x7f5f88064760 msgr2=0x7f5f88064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:32.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f877fe700 1 --2- 172.21.15.29:0/1284376740 >> v2:172.21.15.163:3300/0 conn(0x7f5f88064760 0x7f5f88064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f877fe700 1 -- 172.21.15.29:0/1284376740 >> v1:172.21.15.29:6789/0 conn(0x7f5f88154700 legacy=0x7f5f88154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f877fe700 1 -- 172.21.15.29:0/1284376740 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f880aa670 con 0x7f5f880c8490 2024-01-24T14:14:32.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f86ffd700 1 -- 172.21.15.29:0/1284376740 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365580047 0 0) 0x7f5f88152240 con 0x7f5f88154700 2024-01-24T14:14:32.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f877fe700 1 --2- 172.21.15.29:0/1284376740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f880c8490 0x7f5f880a9d90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f5f88084450 tx=0x7f5f70009060).ready entity=mon.0 client_cookie=9dbd2dcd6721d33f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:32.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f86ffd700 1 -- 172.21.15.29:0/1284376740 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000b610 con 0x7f5f880c8490 2024-01-24T14:14:32.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f86ffd700 1 -- 172.21.15.29:0/1284376740 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7000b770 con 0x7f5f880c8490 2024-01-24T14:14:32.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f86ffd700 1 -- 172.21.15.29:0/1284376740 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000baf0 con 0x7f5f880c8490 2024-01-24T14:14:32.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f8e944700 1 -- 172.21.15.29:0/1284376740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f880c8490 msgr2=0x7f5f880a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f8e944700 1 --2- 172.21.15.29:0/1284376740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f880c8490 0x7f5f880a9d90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f5f88084450 tx=0x7f5f70009060).stop 2024-01-24T14:14:32.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f8e944700 1 -- 172.21.15.29:0/1284376740 shutdown_connections 2024-01-24T14:14:32.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f8e944700 1 --2- 172.21.15.29:0/1284376740 >> v2:172.21.15.163:3300/0 conn(0x7f5f88064760 0x7f5f88064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f8e944700 1 --2- 172.21.15.29:0/1284376740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f880c8490 0x7f5f880a9d90 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f8e944700 1 -- 172.21.15.29:0/1284376740 >> 172.21.15.29:0/1284376740 conn(0x7f5f880cb8b0 msgr2=0x7f5f880c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:32.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.690+0000 7f5f8e944700 1 -- 172.21.15.29:0/1284376740 shutdown_connections 2024-01-24T14:14:32.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.691+0000 7f5f8e944700 1 -- 172.21.15.29:0/1284376740 wait complete. 2024-01-24T14:14:32.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.692+0000 7f5f8e944700 1 Processor -- start 2024-01-24T14:14:32.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.692+0000 7f5f8e944700 1 -- start start 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f8e944700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f88064760 0x7f5f88165400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f8e944700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f880c8490 0x7f5f88156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f8e944700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f88154700 0x7f5f88163e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f8e944700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880ab310 con 0x7f5f88064760 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f8e944700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880ab5f0 con 0x7f5f88154700 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f8e944700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f88157330 con 0x7f5f880c8490 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f87fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f88064760 0x7f5f88165400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f87fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f88064760 0x7f5f88165400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50064/0 (socket says 172.21.15.29:50064) 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f87fff700 1 -- 172.21.15.29:0/793805554 learned_addr learned my addr 172.21.15.29:0/793805554 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f877fe700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f880c8490 0x7f5f88156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f8cee1700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f88154700 0x7f5f88163e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f87fff700 1 -- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f880c8490 msgr2=0x7f5f88156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f87fff700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f880c8490 0x7f5f88156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f87fff700 1 -- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f88154700 msgr2=0x7f5f88163e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f87fff700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f88154700 0x7f5f88163e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f87fff700 1 -- 172.21.15.29:0/793805554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700092a0 con 0x7f5f88064760 2024-01-24T14:14:32.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f8cee1700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f88154700 0x7f5f88163e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:32.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f87fff700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f88064760 0x7f5f88165400 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f5f7800d890 tx=0x7f5f78008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:32.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.693+0000 7f5f7ffff700 1 -- 172.21.15.29:0/793805554 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f780125f0 con 0x7f5f88064760 2024-01-24T14:14:32.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.694+0000 7f5f7ffff700 1 -- 172.21.15.29:0/793805554 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f78012750 con 0x7f5f88064760 2024-01-24T14:14:32.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.694+0000 7f5f8e944700 1 -- 172.21.15.29:0/793805554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f88157610 con 0x7f5f88064760 2024-01-24T14:14:32.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.694+0000 7f5f7ffff700 1 -- 172.21.15.29:0/793805554 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78012ad0 con 0x7f5f88064760 2024-01-24T14:14:32.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.694+0000 7f5f8e944700 1 -- 172.21.15.29:0/793805554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f88156f10 con 0x7f5f88064760 2024-01-24T14:14:32.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.695+0000 7f5f8e944700 1 -- 172.21.15.29:0/793805554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f880c10c0 con 0x7f5f88064760 2024-01-24T14:14:32.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.696+0000 7f5f7ffff700 1 -- 172.21.15.29:0/793805554 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5f78013480 con 0x7f5f88064760 2024-01-24T14:14:32.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.696+0000 7f5f7ffff700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f6805f270 0x7f5f68061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:32.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.696+0000 7f5f7ffff700 1 -- 172.21.15.29:0/793805554 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f5f7807ad00 con 0x7f5f88064760 2024-01-24T14:14:32.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.696+0000 7f5f877fe700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f6805f270 0x7f5f68061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:32.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.697+0000 7f5f877fe700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f6805f270 0x7f5f68061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f5f70009fd0 tx=0x7f5f70017000).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:32.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.700+0000 7f5f7ffff700 1 -- 172.21.15.29:0/793805554 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5f7804bd00 con 0x7f5f88064760 2024-01-24T14:14:32.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.879+0000 7f5f8e944700 1 -- 172.21.15.29:0/793805554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5f880475e0 con 0x7f5f88064760 2024-01-24T14:14:32.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.880+0000 7f5f7ffff700 1 -- 172.21.15.29:0/793805554 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v116) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5f7804f320 con 0x7f5f88064760 2024-01-24T14:14:32.880 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T14:14:32.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.882+0000 7f5f8e944700 1 -- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f6805f270 msgr2=0x7f5f68061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.882+0000 7f5f8e944700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f6805f270 0x7f5f68061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f5f70009fd0 tx=0x7f5f70017000).stop 2024-01-24T14:14:32.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.882+0000 7f5f8e944700 1 -- 172.21.15.29:0/793805554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f88064760 msgr2=0x7f5f88165400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:32.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.882+0000 7f5f8e944700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f88064760 0x7f5f88165400 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f5f7800d890 tx=0x7f5f78008040).stop 2024-01-24T14:14:32.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.882+0000 7f5f8e944700 1 -- 172.21.15.29:0/793805554 shutdown_connections 2024-01-24T14:14:32.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.882+0000 7f5f8e944700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f88154700 0x7f5f88163e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.882+0000 7f5f8e944700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f6805f270 0x7f5f68061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.882+0000 7f5f8e944700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f88064760 0x7f5f88165400 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.882+0000 7f5f8e944700 1 --2- 172.21.15.29:0/793805554 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f880c8490 0x7f5f88156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:32.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.882+0000 7f5f8e944700 1 -- 172.21.15.29:0/793805554 >> 172.21.15.29:0/793805554 conn(0x7f5f880cb8b0 msgr2=0x7f5f88152750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:32.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.882+0000 7f5f8e944700 1 -- 172.21.15.29:0/793805554 shutdown_connections 2024-01-24T14:14:32.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:32.882+0000 7f5f8e944700 1 -- 172.21.15.29:0/793805554 wait complete. 2024-01-24T14:14:32.896 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-24T14:14:33.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.026+0000 7f5f2d8c3700 1 Processor -- start 2024-01-24T14:14:33.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.026+0000 7f5f2d8c3700 1 -- start start 2024-01-24T14:14:33.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.026+0000 7f5f2d8c3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5f280c5410 0x7f5f280aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.026+0000 7f5f2d8c3700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f280aa6c0 con 0x7f5f28154370 2024-01-24T14:14:33.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f2d8c3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f28064240 con 0x7f5f28154750 2024-01-24T14:14:33.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f2d8c3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f28154ba0 con 0x7f5f280c5410 2024-01-24T14:14:33.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f26ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5f28154370 0x7f5f280ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40410/0 (socket says 172.21.15.29:40410) 2024-01-24T14:14:33.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f26ffd700 1 -- 172.21.15.29:0/3458511718 learned_addr learned my addr 172.21.15.29:0/3458511718 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:33.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f267fc700 1 --2- 172.21.15.29:0/3458511718 >> v2:172.21.15.163:3300/0 conn(0x7f5f280c5410 0x7f5f280aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933903737 0 0) 0x7f5f280aa6c0 con 0x7f5f28154370 2024-01-24T14:14:33.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f100033f0 con 0x7f5f28154370 2024-01-24T14:14:33.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 517974548 0 0) 0x7f5f100033f0 con 0x7f5f28154370 2024-01-24T14:14:33.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 >> v2:172.21.15.163:3300/0 conn(0x7f5f280c5410 msgr2=0x7f5f280aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:33.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 --2- 172.21.15.29:0/3458511718 >> v2:172.21.15.163:3300/0 conn(0x7f5f280c5410 0x7f5f280aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 >> v1:172.21.15.163:6789/0 conn(0x7f5f28154750 legacy=0x7f5f280631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:33.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f280cb5c0 con 0x7f5f28154370 2024-01-24T14:14:33.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f5f180015f0 con 0x7f5f28154370 2024-01-24T14:14:33.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 >> v1:172.21.15.29:6789/0 conn(0x7f5f28154370 legacy=0x7f5f280ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:33.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 --2- 172.21.15.29:0/3458511718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 0x7f5f10003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 --2- 172.21.15.29:0/3458511718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f10004690 0x7f5f10004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 --2- 172.21.15.29:0/3458511718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f10005000 0x7f5f1000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28154ba0 con 0x7f5f100038e0 2024-01-24T14:14:33.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1000bf60 con 0x7f5f10004690 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1000c0f0 con 0x7f5f10005000 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.027+0000 7f5f26ffd700 1 --2- 172.21.15.29:0/3458511718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 0x7f5f10003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f267fc700 1 --2- 172.21.15.29:0/3458511718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f10004690 0x7f5f10004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f277fe700 1 --2- 172.21.15.29:0/3458511718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f10005000 0x7f5f1000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f26ffd700 1 -- 172.21.15.29:0/3458511718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f10005000 msgr2=0x7f5f1000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f26ffd700 1 --2- 172.21.15.29:0/3458511718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f10005000 0x7f5f1000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f26ffd700 1 -- 172.21.15.29:0/3458511718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f10004690 msgr2=0x7f5f10004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f26ffd700 1 --2- 172.21.15.29:0/3458511718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f10004690 0x7f5f10004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f26ffd700 1 -- 172.21.15.29:0/3458511718 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f100033f0 con 0x7f5f100038e0 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f277fe700 1 --2- 172.21.15.29:0/3458511718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f10005000 0x7f5f1000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f277fe700 1 -- 172.21.15.29:0/3458511718 reap_dead start 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f26ffd700 1 --2- 172.21.15.29:0/3458511718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 0x7f5f10003d00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f5f1800c190 tx=0x7f5f280643d0).ready entity=mon.0 client_cookie=1deed1d2dc680059 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800ca80 con 0x7f5f100038e0 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1800cbe0 con 0x7f5f100038e0 2024-01-24T14:14:33.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.028+0000 7f5f25ffb700 1 -- 172.21.15.29:0/3458511718 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180070d0 con 0x7f5f100038e0 2024-01-24T14:14:33.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.029+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/3458511718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 msgr2=0x7f5f10003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:33.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.029+0000 7f5f2d8c3700 1 --2- 172.21.15.29:0/3458511718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 0x7f5f10003d00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f5f1800c190 tx=0x7f5f280643d0).stop 2024-01-24T14:14:33.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.029+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/3458511718 shutdown_connections 2024-01-24T14:14:33.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.029+0000 7f5f2d8c3700 1 --2- 172.21.15.29:0/3458511718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 0x7f5f10003d00 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.029+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/3458511718 >> 172.21.15.29:0/3458511718 conn(0x7f5f2805ec80 msgr2=0x7f5f280cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:33.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.029+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/3458511718 shutdown_connections 2024-01-24T14:14:33.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.029+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/3458511718 wait complete. 2024-01-24T14:14:33.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.031+0000 7f5f2d8c3700 1 Processor -- start 2024-01-24T14:14:33.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.031+0000 7f5f2d8c3700 1 -- start start 2024-01-24T14:14:33.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.031+0000 7f5f2d8c3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 0x7f5f28167f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.031+0000 7f5f2d8c3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f281596f0 0x7f5f2814dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.031+0000 7f5f2d8c3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f2814be00 0x7f5f280c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.031+0000 7f5f2d8c3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28144790 con 0x7f5f100038e0 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.031+0000 7f5f2d8c3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28168470 con 0x7f5f281596f0 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.031+0000 7f5f2d8c3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f281566a0 con 0x7f5f2814be00 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f26ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 0x7f5f28167f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f26ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 0x7f5f28167f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50084/0 (socket says 172.21.15.29:50084) 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f26ffd700 1 -- 172.21.15.29:0/2779568299 learned_addr learned my addr 172.21.15.29:0/2779568299 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f277fe700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f2814be00 0x7f5f280c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f267fc700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f281596f0 0x7f5f2814dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f26ffd700 1 -- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f2814be00 msgr2=0x7f5f280c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f26ffd700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f2814be00 0x7f5f280c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f26ffd700 1 -- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f281596f0 msgr2=0x7f5f2814dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f26ffd700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f281596f0 0x7f5f2814dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f26ffd700 1 -- 172.21.15.29:0/2779568299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f28159b70 con 0x7f5f100038e0 2024-01-24T14:14:33.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f26ffd700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 0x7f5f28167f80 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f5f1800c780 tx=0x7f5f18015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:33.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f0effd700 1 -- 172.21.15.29:0/2779568299 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18000f70 con 0x7f5f100038e0 2024-01-24T14:14:33.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f0effd700 1 -- 172.21.15.29:0/2779568299 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18005e40 con 0x7f5f100038e0 2024-01-24T14:14:33.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f0effd700 1 -- 172.21.15.29:0/2779568299 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18007480 con 0x7f5f100038e0 2024-01-24T14:14:33.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/2779568299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f28168690 con 0x7f5f100038e0 2024-01-24T14:14:33.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.032+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/2779568299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f28168c50 con 0x7f5f100038e0 2024-01-24T14:14:33.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.033+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/2779568299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f280475e0 con 0x7f5f100038e0 2024-01-24T14:14:33.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.034+0000 7f5f0effd700 1 -- 172.21.15.29:0/2779568299 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5f1801c030 con 0x7f5f100038e0 2024-01-24T14:14:33.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.034+0000 7f5f0effd700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f1405f270 0x7f5f14061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.034+0000 7f5f0effd700 1 -- 172.21.15.29:0/2779568299 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f5f1807abd0 con 0x7f5f100038e0 2024-01-24T14:14:33.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.034+0000 7f5f267fc700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f1405f270 0x7f5f14061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.035+0000 7f5f267fc700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f1405f270 0x7f5f14061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5f280b0f80 tx=0x7f5f1c009060).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:33.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.039+0000 7f5f0effd700 1 -- 172.21.15.29:0/2779568299 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5f1804bc70 con 0x7f5f100038e0 2024-01-24T14:14:33.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.223+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/2779568299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5f280cb8a0 con 0x7f5f100038e0 2024-01-24T14:14:33.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.224+0000 7f5f0effd700 1 -- 172.21.15.29:0/2779568299 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v116) v1 ==== 84+0+143 (secure 0 0 0) 0x7f5f1804f290 con 0x7f5f100038e0 2024-01-24T14:14:33.225 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:33.225 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T14:14:33.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:33.225 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-01-24T14:14:33.225 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "device_health_metrics" 2024-01-24T14:14:33.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:14:33.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:33.225 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 2, 2024-01-24T14:14:33.225 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "rbd" 2024-01-24T14:14:33.225 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:33.226 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T14:14:33.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.226+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f1405f270 msgr2=0x7f5f14061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:33.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.226+0000 7f5f2d8c3700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f1405f270 0x7f5f14061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5f280b0f80 tx=0x7f5f1c009060).stop 2024-01-24T14:14:33.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.226+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/2779568299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 msgr2=0x7f5f28167f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:33.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.226+0000 7f5f2d8c3700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 0x7f5f28167f80 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f5f1800c780 tx=0x7f5f18015000).stop 2024-01-24T14:14:33.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.226+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/2779568299 shutdown_connections 2024-01-24T14:14:33.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.226+0000 7f5f2d8c3700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f2814be00 0x7f5f280c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.226+0000 7f5f2d8c3700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f1405f270 0x7f5f14061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.226+0000 7f5f2d8c3700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f100038e0 0x7f5f28167f80 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.226+0000 7f5f2d8c3700 1 --2- 172.21.15.29:0/2779568299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f281596f0 0x7f5f2814dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.226+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/2779568299 >> 172.21.15.29:0/2779568299 conn(0x7f5f2805ec80 msgr2=0x7f5f28151ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:33.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.226+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/2779568299 shutdown_connections 2024-01-24T14:14:33.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.226+0000 7f5f2d8c3700 1 -- 172.21.15.29:0/2779568299 wait complete. 2024-01-24T14:14:33.240 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-24T14:14:33.240 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2024-01-24T14:14:33.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.374+0000 7fbf5a153700 1 Processor -- start 2024-01-24T14:14:33.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.374+0000 7fbf5a153700 1 -- start start 2024-01-24T14:14:33.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.375+0000 7fbf5a153700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbf54066450 0x7fbf54066830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.375+0000 7fbf5a153700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf54152540 0x7fbf541529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.375+0000 7fbf5a153700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf54155a10 0x7fbf54155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.375+0000 7fbf5a153700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf54153300 con 0x7fbf54155a10 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.375+0000 7fbf5a153700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf54067b10 con 0x7fbf54066450 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.375+0000 7fbf5a153700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54067df0 con 0x7fbf54152540 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.375+0000 7fbf53fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf54155a10 0x7fbf54155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.375+0000 7fbf53fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf54155a10 0x7fbf54155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50098/0 (socket says 172.21.15.29:50098) 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.375+0000 7fbf53fff700 1 -- 172.21.15.29:0/496937266 learned_addr learned my addr 172.21.15.29:0/496937266 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.375+0000 7fbf4affd700 1 --2- 172.21.15.29:0/496937266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf54152540 0x7fbf541529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.375+0000 7fbf537fe700 1 --2- 172.21.15.29:0/496937266 >> v2:172.21.15.163:3300/0 conn(0x7fbf54066450 0x7fbf54066830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf53fff700 1 -- 172.21.15.29:0/496937266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf54152540 msgr2=0x7fbf541529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf53fff700 1 --2- 172.21.15.29:0/496937266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf54152540 0x7fbf541529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf53fff700 1 -- 172.21.15.29:0/496937266 >> v2:172.21.15.163:3300/0 conn(0x7fbf54066450 msgr2=0x7fbf54066830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:33.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf53fff700 1 --2- 172.21.15.29:0/496937266 >> v2:172.21.15.163:3300/0 conn(0x7fbf54066450 0x7fbf54066830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf53fff700 1 -- 172.21.15.29:0/496937266 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf540680d0 con 0x7fbf54155a10 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf4affd700 1 --2- 172.21.15.29:0/496937266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf54152540 0x7fbf541529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf53fff700 1 --2- 172.21.15.29:0/496937266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf54155a10 0x7fbf54155ea0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7fbf5405fa70 tx=0x7fbf4c009840).ready entity=mon.0 client_cookie=5bf06e909c158dd2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf52ffd700 1 -- 172.21.15.29:0/496937266 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00e070 con 0x7fbf54155a10 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf52ffd700 1 -- 172.21.15.29:0/496937266 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c00ba50 con 0x7fbf54155a10 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf52ffd700 1 -- 172.21.15.29:0/496937266 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00bdd0 con 0x7fbf54155a10 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf5a153700 1 -- 172.21.15.29:0/496937266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf54155a10 msgr2=0x7fbf54155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf5a153700 1 --2- 172.21.15.29:0/496937266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf54155a10 0x7fbf54155ea0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7fbf5405fa70 tx=0x7fbf4c009840).stop 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf5a153700 1 -- 172.21.15.29:0/496937266 shutdown_connections 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf5a153700 1 --2- 172.21.15.29:0/496937266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf54155a10 0x7fbf54155ea0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf5a153700 1 --2- 172.21.15.29:0/496937266 >> v2:172.21.15.163:3300/0 conn(0x7fbf54066450 0x7fbf54066830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf5a153700 1 --2- 172.21.15.29:0/496937266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf54152540 0x7fbf541529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.376+0000 7fbf5a153700 1 -- 172.21.15.29:0/496937266 >> 172.21.15.29:0/496937266 conn(0x7fbf540c8bd0 msgr2=0x7fbf540c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.377+0000 7fbf5a153700 1 -- 172.21.15.29:0/496937266 shutdown_connections 2024-01-24T14:14:33.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.377+0000 7fbf5a153700 1 -- 172.21.15.29:0/496937266 wait complete. 2024-01-24T14:14:33.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.378+0000 7fbf5a153700 1 Processor -- start 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.379+0000 7fbf5a153700 1 -- start start 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.379+0000 7fbf5a153700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf54066450 0x7fbf5414f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.379+0000 7fbf5a153700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf54152540 0x7fbf540c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.379+0000 7fbf5a153700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf54155a10 0x7fbf540c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.379+0000 7fbf5a153700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54134030 con 0x7fbf54155a10 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf5a153700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5414d730 con 0x7fbf54152540 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf5a153700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540ab450 con 0x7fbf54066450 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf53fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf54155a10 0x7fbf540c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf53fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf54155a10 0x7fbf540c2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50100/0 (socket says 172.21.15.29:50100) 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf53fff700 1 -- 172.21.15.29:0/448013021 learned_addr learned my addr 172.21.15.29:0/448013021 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf4affd700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf54152540 0x7fbf540c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf537fe700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf54066450 0x7fbf5414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf53fff700 1 -- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf54066450 msgr2=0x7fbf5414f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:33.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf53fff700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf54066450 0x7fbf5414f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf53fff700 1 -- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf54152540 msgr2=0x7fbf540c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf53fff700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf54152540 0x7fbf540c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf53fff700 1 -- 172.21.15.29:0/448013021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c009ae0 con 0x7fbf54155a10 2024-01-24T14:14:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf537fe700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf54066450 0x7fbf5414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf4affd700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf54152540 0x7fbf540c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf53fff700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf54155a10 0x7fbf540c2250 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7fbf4c009fd0 tx=0x7fbf4c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf4bfff700 1 -- 172.21.15.29:0/448013021 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0032f0 con 0x7fbf54155a10 2024-01-24T14:14:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf4bfff700 1 -- 172.21.15.29:0/448013021 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c009500 con 0x7fbf54155a10 2024-01-24T14:14:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf4bfff700 1 -- 172.21.15.29:0/448013021 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c01e6a0 con 0x7fbf54155a10 2024-01-24T14:14:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf5a153700 1 -- 172.21.15.29:0/448013021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5414f900 con 0x7fbf54155a10 2024-01-24T14:14:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.380+0000 7fbf5a153700 1 -- 172.21.15.29:0/448013021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5414cd90 con 0x7fbf54155a10 2024-01-24T14:14:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.381+0000 7fbf5a153700 1 -- 172.21.15.29:0/448013021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5414d870 con 0x7fbf54155a10 2024-01-24T14:14:33.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.382+0000 7fbf4bfff700 1 -- 172.21.15.29:0/448013021 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbf4c003040 con 0x7fbf54155a10 2024-01-24T14:14:33.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.382+0000 7fbf4bfff700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf3405f270 0x7fbf34061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:33.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.382+0000 7fbf4bfff700 1 -- 172.21.15.29:0/448013021 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6290+0+0 (secure 0 0 0) 0x7fbf4c07bbc0 con 0x7fbf54155a10 2024-01-24T14:14:33.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.382+0000 7fbf537fe700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf3405f270 0x7fbf34061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:33.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.383+0000 7fbf537fe700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf3405f270 0x7fbf34061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fbf44009810 tx=0x7fbf44008040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:33.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.386+0000 7fbf4bfff700 1 -- 172.21.15.29:0/448013021 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbf5414d870 con 0x7fbf54155a10 2024-01-24T14:14:33.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:33.571+0000 7fbf5a153700 1 -- 172.21.15.29:0/448013021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fbf5414d870 con 0x7fbf54155a10 2024-01-24T14:14:34.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.433+0000 7fbf4bfff700 1 -- 172.21.15.29:0/448013021 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v117) v1 ==== 152+0+0 (secure 0 0 0) 0x7fbf5414d870 con 0x7fbf54155a10 2024-01-24T14:14:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.437+0000 7fbf5a153700 1 -- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf3405f270 msgr2=0x7fbf34061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.437+0000 7fbf5a153700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf3405f270 0x7fbf34061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fbf44009810 tx=0x7fbf44008040).stop 2024-01-24T14:14:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.437+0000 7fbf5a153700 1 -- 172.21.15.29:0/448013021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf54155a10 msgr2=0x7fbf540c2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:34.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.437+0000 7fbf5a153700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf54155a10 0x7fbf540c2250 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7fbf4c009fd0 tx=0x7fbf4c009840).stop 2024-01-24T14:14:34.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.443+0000 7fbf5a153700 1 -- 172.21.15.29:0/448013021 shutdown_connections 2024-01-24T14:14:34.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.443+0000 7fbf5a153700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf3405f270 0x7fbf34061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:34.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.443+0000 7fbf5a153700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf54155a10 0x7fbf540c2250 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:34.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.443+0000 7fbf5a153700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf54066450 0x7fbf5414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:34.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.443+0000 7fbf5a153700 1 --2- 172.21.15.29:0/448013021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf54152540 0x7fbf540c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:34.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.443+0000 7fbf5a153700 1 -- 172.21.15.29:0/448013021 >> 172.21.15.29:0/448013021 conn(0x7fbf540c8bd0 msgr2=0x7fbf541529a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:34.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.443+0000 7fbf5a153700 1 -- 172.21.15.29:0/448013021 shutdown_connections 2024-01-24T14:14:34.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.444+0000 7fbf5a153700 1 -- 172.21.15.29:0/448013021 wait complete. 2024-01-24T14:14:34.446 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-01-24T14:14:34.461 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-24T14:14:34.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.601+0000 7f7937ec7700 1 Processor -- start 2024-01-24T14:14:34.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.601+0000 7f7937ec7700 1 -- start start 2024-01-24T14:14:34.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7937ec7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7930066b00 0x7f79300c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:34.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7937ec7700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79300c6be0 con 0x7f793013ba60 2024-01-24T14:14:34.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7937ec7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79300c33e0 con 0x7f7930066720 2024-01-24T14:14:34.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7937ec7700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7930066f40 con 0x7f7930066b00 2024-01-24T14:14:34.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7936464700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7930066b00 0x7f79300c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:34.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7935462700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f793013ba60 0x7f79300c7070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46950/0 (socket says 172.21.15.29:46950) 2024-01-24T14:14:34.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7936464700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7930066b00 0x7f79300c4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41310/0 (socket says 172.21.15.29:41310) 2024-01-24T14:14:34.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7935462700 1 -- 172.21.15.29:0/994340530 learned_addr learned my addr 172.21.15.29:0/994340530 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7936464700 1 -- 172.21.15.29:0/994340530 >> v1:172.21.15.163:6789/0 conn(0x7f7930066720 legacy=0x7f793013b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7936464700 1 -- 172.21.15.29:0/994340530 >> v1:172.21.15.29:6789/0 conn(0x7f793013ba60 legacy=0x7f79300c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7934c61700 1 -- 172.21.15.29:0/994340530 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571750139 0 0) 0x7f79300c6be0 con 0x7f793013ba60 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7936464700 1 -- 172.21.15.29:0/994340530 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79300c68b0 con 0x7f7930066b00 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.602+0000 7f7936464700 1 --2- 172.21.15.29:0/994340530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7930066b00 0x7f79300c4ee0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f7928009760 tx=0x7f7928009060).ready entity=mon.0 client_cookie=7ab74ae94452ea1e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.603+0000 7f7934c61700 1 -- 172.21.15.29:0/994340530 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800b580 con 0x7f7930066b00 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.603+0000 7f7934c61700 1 -- 172.21.15.29:0/994340530 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792800b6e0 con 0x7f7930066b00 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.603+0000 7f7934c61700 1 -- 172.21.15.29:0/994340530 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800ba60 con 0x7f7930066b00 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.603+0000 7f7937ec7700 1 -- 172.21.15.29:0/994340530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7930066b00 msgr2=0x7f79300c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.603+0000 7f7937ec7700 1 --2- 172.21.15.29:0/994340530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7930066b00 0x7f79300c4ee0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f7928009760 tx=0x7f7928009060).stop 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.603+0000 7f7937ec7700 1 -- 172.21.15.29:0/994340530 shutdown_connections 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.603+0000 7f7937ec7700 1 --2- 172.21.15.29:0/994340530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7930066b00 0x7f79300c4ee0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:34.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.603+0000 7f7937ec7700 1 -- 172.21.15.29:0/994340530 >> 172.21.15.29:0/994340530 conn(0x7f793005ec80 msgr2=0x7f79300c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:34.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.603+0000 7f7937ec7700 1 -- 172.21.15.29:0/994340530 shutdown_connections 2024-01-24T14:14:34.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.603+0000 7f7937ec7700 1 -- 172.21.15.29:0/994340530 wait complete. 2024-01-24T14:14:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.605+0000 7f7937ec7700 1 Processor -- start 2024-01-24T14:14:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.605+0000 7f7937ec7700 1 -- start start 2024-01-24T14:14:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.605+0000 7f7937ec7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7930066720 0x7f7930168410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.605+0000 7f7937ec7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7930066b00 0x7f7930137bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.605+0000 7f7937ec7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f793013ba60 0x7f79300bdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.605+0000 7f7937ec7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7930168900 con 0x7f7930066b00 2024-01-24T14:14:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.605+0000 7f7937ec7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79301380b0 con 0x7f793013ba60 2024-01-24T14:14:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.605+0000 7f7937ec7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7930165390 con 0x7f7930066720 2024-01-24T14:14:34.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.605+0000 7f7935462700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7930066b00 0x7f7930137bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7935462700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7930066b00 0x7f7930137bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41312/0 (socket says 172.21.15.29:41312) 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7935462700 1 -- 172.21.15.29:0/4265282375 learned_addr learned my addr 172.21.15.29:0/4265282375 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7936464700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f793013ba60 0x7f79300bdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7935c63700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7930066720 0x7f7930168410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7935462700 1 -- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7930066720 msgr2=0x7f7930168410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7935462700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7930066720 0x7f7930168410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7935462700 1 -- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f793013ba60 msgr2=0x7f79300bdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7935462700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f793013ba60 0x7f79300bdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7935462700 1 -- 172.21.15.29:0/4265282375 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79280092a0 con 0x7f7930066b00 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7935c63700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7930066720 0x7f7930168410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7936464700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f793013ba60 0x7f79300bdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7935462700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7930066b00 0x7f7930137bc0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f792400c490 tx=0x7f7924007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f792dffb700 1 -- 172.21.15.29:0/4265282375 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79240072e0 con 0x7f7930066b00 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f792dffb700 1 -- 172.21.15.29:0/4265282375 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7924007440 con 0x7f7930066b00 2024-01-24T14:14:34.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f792dffb700 1 -- 172.21.15.29:0/4265282375 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79240057a0 con 0x7f7930066b00 2024-01-24T14:14:34.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.606+0000 7f7937ec7700 1 -- 172.21.15.29:0/4265282375 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7930164780 con 0x7f7930066b00 2024-01-24T14:14:34.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.607+0000 7f7937ec7700 1 -- 172.21.15.29:0/4265282375 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7930164d40 con 0x7f7930066b00 2024-01-24T14:14:34.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.608+0000 7f7937ec7700 1 -- 172.21.15.29:0/4265282375 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79300475e0 con 0x7f7930066b00 2024-01-24T14:14:34.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.608+0000 7f792dffb700 1 -- 172.21.15.29:0/4265282375 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7924005b90 con 0x7f7930066b00 2024-01-24T14:14:34.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.608+0000 7f792dffb700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f791c05f270 0x7f791c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:34.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.608+0000 7f792dffb700 1 -- 172.21.15.29:0/4265282375 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6660+0+0 (secure 0 0 0) 0x7f7924079d50 con 0x7f7930066b00 2024-01-24T14:14:34.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.609+0000 7f7935c63700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f791c05f270 0x7f791c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:34.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.609+0000 7f7935c63700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f791c05f270 0x7f791c061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7920007fd0 tx=0x7f7920009040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:34.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.613+0000 7f792dffb700 1 -- 172.21.15.29:0/4265282375 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f792404ac80 con 0x7f7930066b00 2024-01-24T14:14:34.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:34.795+0000 7f7937ec7700 1 -- 172.21.15.29:0/4265282375 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f79300c3680 con 0x7f7930066b00 2024-01-24T14:14:35.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.442+0000 7f792dffb700 1 -- 172.21.15.29:0/4265282375 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v118) v1 ==== 170+0+0 (secure 0 0 0) 0x7f792404e2a0 con 0x7f7930066b00 2024-01-24T14:14:35.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.449+0000 7f7937ec7700 1 -- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f791c05f270 msgr2=0x7f791c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:35.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.449+0000 7f7937ec7700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f791c05f270 0x7f791c061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7920007fd0 tx=0x7f7920009040).stop 2024-01-24T14:14:35.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.449+0000 7f7937ec7700 1 -- 172.21.15.29:0/4265282375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7930066b00 msgr2=0x7f7930137bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:35.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.449+0000 7f7937ec7700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7930066b00 0x7f7930137bc0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f792400c490 tx=0x7f7924007e10).stop 2024-01-24T14:14:35.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.452+0000 7f7937ec7700 1 -- 172.21.15.29:0/4265282375 shutdown_connections 2024-01-24T14:14:35.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.452+0000 7f7937ec7700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f793013ba60 0x7f79300bdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.452+0000 7f7937ec7700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7930066720 0x7f7930168410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.452+0000 7f7937ec7700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f791c05f270 0x7f791c061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.452+0000 7f7937ec7700 1 --2- 172.21.15.29:0/4265282375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7930066b00 0x7f7930137bc0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.452+0000 7f7937ec7700 1 -- 172.21.15.29:0/4265282375 >> 172.21.15.29:0/4265282375 conn(0x7f793005ec80 msgr2=0x7f79300a8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:35.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.458+0000 7f7937ec7700 1 -- 172.21.15.29:0/4265282375 shutdown_connections 2024-01-24T14:14:35.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.458+0000 7f7937ec7700 1 -- 172.21.15.29:0/4265282375 wait complete. 2024-01-24T14:14:35.464 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-01-24T14:14:35.490 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-24T14:14:35.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.624+0000 7f72dd9fd700 1 Processor -- start 2024-01-24T14:14:35.625 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.624+0000 7f72dd9fd700 1 -- start start 2024-01-24T14:14:35.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.625+0000 7f72dd9fd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72d8064760 0x7f72d8064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:35.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.625+0000 7f72dd9fd700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d8152240 con 0x7f72d80c8490 2024-01-24T14:14:35.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.625+0000 7f72dd9fd700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d81586c0 con 0x7f72d80bfd40 2024-01-24T14:14:35.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.625+0000 7f72dd9fd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d80aa670 con 0x7f72d8064760 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.625+0000 7f72d67fc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f72d80c8490 0x7f72d80a9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46954/0 (socket says 172.21.15.29:46954) 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.625+0000 7f72d67fc700 1 -- 172.21.15.29:0/1374997433 learned_addr learned my addr 172.21.15.29:0/1374997433 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.625+0000 7f72d6ffd700 1 --2- 172.21.15.29:0/1374997433 >> v2:172.21.15.163:3300/0 conn(0x7f72d8064760 0x7f72d8064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.625+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759979089 0 0) 0x7f72d8152240 con 0x7f72d80c8490 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.625+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c00033f0 con 0x7f72d80c8490 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.625+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 686681744 0 0) 0x7f72c00033f0 con 0x7f72d80c8490 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 >> v2:172.21.15.163:3300/0 conn(0x7f72d8064760 msgr2=0x7f72d8064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 --2- 172.21.15.29:0/1374997433 >> v2:172.21.15.163:3300/0 conn(0x7f72d8064760 0x7f72d8064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 >> v1:172.21.15.163:6789/0 conn(0x7f72d80bfd40 legacy=0x7f72d80c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d8157d90 con 0x7f72d80c8490 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f72cc001470 con 0x7f72d80c8490 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 >> v1:172.21.15.29:6789/0 conn(0x7f72d80c8490 legacy=0x7f72d80a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:35.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 --2- 172.21.15.29:0/1374997433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72c00038e0 0x7f72c0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 --2- 172.21.15.29:0/1374997433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 0x7f72c0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 --2- 172.21.15.29:0/1374997433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c0005000 0x7f72c000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80aa670 con 0x7f72c0004690 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c000beb0 con 0x7f72c0005000 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c000c070 con 0x7f72c00038e0 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d67fc700 1 --2- 172.21.15.29:0/1374997433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 0x7f72c0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d6ffd700 1 --2- 172.21.15.29:0/1374997433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72c00038e0 0x7f72c0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d77fe700 1 --2- 172.21.15.29:0/1374997433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c0005000 0x7f72c000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d67fc700 1 -- 172.21.15.29:0/1374997433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72c00038e0 msgr2=0x7f72c0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d67fc700 1 --2- 172.21.15.29:0/1374997433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72c00038e0 0x7f72c0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d67fc700 1 -- 172.21.15.29:0/1374997433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c0005000 msgr2=0x7f72c000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d67fc700 1 --2- 172.21.15.29:0/1374997433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c0005000 0x7f72c000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d67fc700 1 -- 172.21.15.29:0/1374997433 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c00033f0 con 0x7f72c0004690 2024-01-24T14:14:35.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d77fe700 1 -- 172.21.15.29:0/1374997433 reap_dead start 2024-01-24T14:14:35.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d67fc700 1 --2- 172.21.15.29:0/1374997433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 0x7f72c0004b10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f72cc00c190 tx=0x7f72cc001e50).ready entity=mon.0 client_cookie=728341234447bc78 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:35.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.626+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc007a20 con 0x7f72c0004690 2024-01-24T14:14:35.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.627+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc00cac0 con 0x7f72c0004690 2024-01-24T14:14:35.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.627+0000 7f72d5ffb700 1 -- 172.21.15.29:0/1374997433 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc00ce40 con 0x7f72c0004690 2024-01-24T14:14:35.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.627+0000 7f72dd9fd700 1 -- 172.21.15.29:0/1374997433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 msgr2=0x7f72c0004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:35.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.627+0000 7f72dd9fd700 1 --2- 172.21.15.29:0/1374997433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 0x7f72c0004b10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f72cc00c190 tx=0x7f72cc001e50).stop 2024-01-24T14:14:35.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.627+0000 7f72dd9fd700 1 -- 172.21.15.29:0/1374997433 shutdown_connections 2024-01-24T14:14:35.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.627+0000 7f72dd9fd700 1 --2- 172.21.15.29:0/1374997433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 0x7f72c0004b10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.627+0000 7f72dd9fd700 1 -- 172.21.15.29:0/1374997433 >> 172.21.15.29:0/1374997433 conn(0x7f72d80cb8b0 msgr2=0x7f72d80c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:35.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.627+0000 7f72dd9fd700 1 -- 172.21.15.29:0/1374997433 shutdown_connections 2024-01-24T14:14:35.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.627+0000 7f72dd9fd700 1 -- 172.21.15.29:0/1374997433 wait complete. 2024-01-24T14:14:35.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.629+0000 7f72dd9fd700 1 Processor -- start 2024-01-24T14:14:35.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.629+0000 7f72dd9fd700 1 -- start start 2024-01-24T14:14:35.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.630+0000 7f72dd9fd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 0x7f72d814f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:35.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.630+0000 7f72dd9fd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d8150ce0 0x7f72d8151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:35.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.630+0000 7f72dd9fd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d8152bd0 0x7f72d8153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:35.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.630+0000 7f72dd9fd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d814f720 con 0x7f72c0004690 2024-01-24T14:14:35.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.630+0000 7f72dd9fd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d8151650 con 0x7f72d8150ce0 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.630+0000 7f72dd9fd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d8164cf0 con 0x7f72d8152bd0 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.630+0000 7f72d6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 0x7f72d814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.630+0000 7f72d6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 0x7f72d814f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41342/0 (socket says 172.21.15.29:41342) 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.630+0000 7f72d6ffd700 1 -- 172.21.15.29:0/3867261796 learned_addr learned my addr 172.21.15.29:0/3867261796 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.630+0000 7f72d77fe700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d8152bd0 0x7f72d8153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.630+0000 7f72d67fc700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d8150ce0 0x7f72d8151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.631+0000 7f72d6ffd700 1 -- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d8152bd0 msgr2=0x7f72d8153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.631+0000 7f72d6ffd700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d8152bd0 0x7f72d8153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.631+0000 7f72d6ffd700 1 -- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d8150ce0 msgr2=0x7f72d8151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.631+0000 7f72d6ffd700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d8150ce0 0x7f72d8151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.631+0000 7f72d6ffd700 1 -- 172.21.15.29:0/3867261796 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d8164fd0 con 0x7f72c0004690 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.631+0000 7f72d67fc700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d8150ce0 0x7f72d8151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.631+0000 7f72d6ffd700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 0x7f72d814f230 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f72c8011a20 tx=0x7f72c8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:35.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.631+0000 7f72beffd700 1 -- 172.21.15.29:0/3867261796 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c800f940 con 0x7f72c0004690 2024-01-24T14:14:35.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.631+0000 7f72beffd700 1 -- 172.21.15.29:0/3867261796 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c800faa0 con 0x7f72c0004690 2024-01-24T14:14:35.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.631+0000 7f72dd9fd700 1 -- 172.21.15.29:0/3867261796 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d81652b0 con 0x7f72c0004690 2024-01-24T14:14:35.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.631+0000 7f72beffd700 1 -- 172.21.15.29:0/3867261796 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c800fe20 con 0x7f72c0004690 2024-01-24T14:14:35.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.631+0000 7f72dd9fd700 1 -- 172.21.15.29:0/3867261796 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d81649e0 con 0x7f72c0004690 2024-01-24T14:14:35.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.632+0000 7f72dd9fd700 1 -- 172.21.15.29:0/3867261796 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d80475e0 con 0x7f72c0004690 2024-01-24T14:14:35.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.633+0000 7f72beffd700 1 -- 172.21.15.29:0/3867261796 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f72c8014650 con 0x7f72c0004690 2024-01-24T14:14:35.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.633+0000 7f72beffd700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f72c405f270 0x7f72c4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:35.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.633+0000 7f72beffd700 1 -- 172.21.15.29:0/3867261796 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7042+0+0 (secure 0 0 0) 0x7f72c807b2f0 con 0x7f72c0004690 2024-01-24T14:14:35.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.633+0000 7f72d67fc700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f72c405f270 0x7f72c4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:35.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.634+0000 7f72d67fc700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f72c405f270 0x7f72c4061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f72d80ca110 tx=0x7f72cc015000).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:35.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.638+0000 7f72beffd700 1 -- 172.21.15.29:0/3867261796 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f72c804c0a0 con 0x7f72c0004690 2024-01-24T14:14:35.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.868+0000 7f72dd9fd700 1 -- 172.21.15.29:0/3867261796 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f72d8158070 con 0x7f72c0004690 2024-01-24T14:14:35.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.875+0000 7f72beffd700 1 -- 172.21.15.29:0/3867261796 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v47) v1 ==== 174+0+0 (secure 0 0 0) 0x7f72c804f6c0 con 0x7f72c0004690 2024-01-24T14:14:35.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.878+0000 7f72dd9fd700 1 -- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f72c405f270 msgr2=0x7f72c4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:35.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.878+0000 7f72dd9fd700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f72c405f270 0x7f72c4061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f72d80ca110 tx=0x7f72cc015000).stop 2024-01-24T14:14:35.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.878+0000 7f72dd9fd700 1 -- 172.21.15.29:0/3867261796 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 msgr2=0x7f72d814f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:35.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.878+0000 7f72dd9fd700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 0x7f72d814f230 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f72c8011a20 tx=0x7f72c8009060).stop 2024-01-24T14:14:35.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.879+0000 7f72dd9fd700 1 -- 172.21.15.29:0/3867261796 shutdown_connections 2024-01-24T14:14:35.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.879+0000 7f72dd9fd700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d8152bd0 0x7f72d8153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.879+0000 7f72dd9fd700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f72c405f270 0x7f72c4061730 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0x7f72d80ca110 tx=0x7f72cc015000).stop 2024-01-24T14:14:35.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.879+0000 7f72dd9fd700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c0004690 0x7f72d814f230 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.879+0000 7f72dd9fd700 1 --2- 172.21.15.29:0/3867261796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d8150ce0 0x7f72d8151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:35.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.879+0000 7f72dd9fd700 1 -- 172.21.15.29:0/3867261796 >> 172.21.15.29:0/3867261796 conn(0x7f72d80cb8b0 msgr2=0x7f72d80c5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:35.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.883+0000 7f72dd9fd700 1 -- 172.21.15.29:0/3867261796 shutdown_connections 2024-01-24T14:14:35.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:35.889+0000 7f72dd9fd700 1 -- 172.21.15.29:0/3867261796 wait complete. 2024-01-24T14:14:35.902 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 14 and data pool 15 2024-01-24T14:14:35.914 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:14:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.042+0000 7f1cb319d700 1 Processor -- start 2024-01-24T14:14:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.043+0000 7f1cb319d700 1 -- start start 2024-01-24T14:14:36.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.043+0000 7f1cb319d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1cac155b00 0x7f1cac0664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.043+0000 7f1cb319d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac066e90 con 0x7f1cac1565f0 2024-01-24T14:14:36.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.043+0000 7f1cb319d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac0c3910 con 0x7f1cac0669e0 2024-01-24T14:14:36.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.043+0000 7f1cb319d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cac0c5500 con 0x7f1cac155b00 2024-01-24T14:14:36.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.044+0000 7f1cb173a700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1cac1565f0 0x7f1cac0bdeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46956/0 (socket says 172.21.15.29:46956) 2024-01-24T14:14:36.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.044+0000 7f1cb173a700 1 -- 172.21.15.29:0/3542707453 learned_addr learned my addr 172.21.15.29:0/3542707453 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:36.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.044+0000 7f1cb0f39700 1 --2- 172.21.15.29:0/3542707453 >> v2:172.21.15.163:3300/0 conn(0x7f1cac155b00 0x7f1cac0664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.044+0000 7f1cab7fe700 1 -- 172.21.15.29:0/3542707453 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008647544 0 0) 0x7f1ca4001490 con 0x7f1cac1565f0 2024-01-24T14:14:36.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.044+0000 7f1cab7fe700 1 -- 172.21.15.29:0/3542707453 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c9c0033f0 con 0x7f1cac1565f0 2024-01-24T14:14:36.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.044+0000 7f1cb0f39700 1 -- 172.21.15.29:0/3542707453 >> v1:172.21.15.163:6789/0 conn(0x7f1cac0669e0 legacy=0x7f1cac155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.044+0000 7f1cab7fe700 1 -- 172.21.15.29:0/3542707453 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1156179519 0 0) 0x7f1c9c0033f0 con 0x7f1cac1565f0 2024-01-24T14:14:36.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.044+0000 7f1cb0f39700 1 -- 172.21.15.29:0/3542707453 >> v1:172.21.15.29:6789/0 conn(0x7f1cac1565f0 legacy=0x7f1cac0bdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.044+0000 7f1cb0f39700 1 -- 172.21.15.29:0/3542707453 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0c5220 con 0x7f1cac155b00 2024-01-24T14:14:36.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.044+0000 7f1cb0f39700 1 --2- 172.21.15.29:0/3542707453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cac155b00 0x7f1cac0664f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f1ca0009760 tx=0x7f1ca0009060).ready entity=mon.1 client_cookie=ba56668313717731 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:36.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.045+0000 7f1cab7fe700 1 -- 172.21.15.29:0/3542707453 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4001980 con 0x7f1cac155b00 2024-01-24T14:14:36.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.045+0000 7f1cab7fe700 1 -- 172.21.15.29:0/3542707453 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca0011400 con 0x7f1cac155b00 2024-01-24T14:14:36.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.045+0000 7f1cab7fe700 1 -- 172.21.15.29:0/3542707453 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0011780 con 0x7f1cac155b00 2024-01-24T14:14:36.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.045+0000 7f1cb319d700 1 -- 172.21.15.29:0/3542707453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cac155b00 msgr2=0x7f1cac0664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.045+0000 7f1cb319d700 1 --2- 172.21.15.29:0/3542707453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cac155b00 0x7f1cac0664f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f1ca0009760 tx=0x7f1ca0009060).stop 2024-01-24T14:14:36.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.045+0000 7f1cb319d700 1 -- 172.21.15.29:0/3542707453 shutdown_connections 2024-01-24T14:14:36.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.045+0000 7f1cb319d700 1 --2- 172.21.15.29:0/3542707453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cac155b00 0x7f1cac0664f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.045+0000 7f1cb319d700 1 -- 172.21.15.29:0/3542707453 >> 172.21.15.29:0/3542707453 conn(0x7f1cac0c8bb0 msgr2=0x7f1cac0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:36.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.045+0000 7f1cb319d700 1 -- 172.21.15.29:0/3542707453 shutdown_connections 2024-01-24T14:14:36.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.045+0000 7f1cb319d700 1 -- 172.21.15.29:0/3542707453 wait complete. 2024-01-24T14:14:36.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.046+0000 7f1cb319d700 1 Processor -- start 2024-01-24T14:14:36.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.047+0000 7f1cb319d700 1 -- start start 2024-01-24T14:14:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.047+0000 7f1cb319d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cac0669e0 0x7f1cac0abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.047+0000 7f1cb319d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cac155b00 0x7f1cac0a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cb319d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1cac1565f0 0x7f1cac16ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cb319d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac15f4b0 con 0x7f1cac155b00 2024-01-24T14:14:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cb319d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0acea0 con 0x7f1cac0669e0 2024-01-24T14:14:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cb319d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0ac030 con 0x7f1cac1565f0 2024-01-24T14:14:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cabfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cac155b00 0x7f1cac0a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cabfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cac155b00 0x7f1cac0a7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41348/0 (socket says 172.21.15.29:41348) 2024-01-24T14:14:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cabfff700 1 -- 172.21.15.29:0/4151469435 learned_addr learned my addr 172.21.15.29:0/4151469435 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cb173a700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1cac1565f0 0x7f1cac16ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cb0f39700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cac0669e0 0x7f1cac0abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cabfff700 1 -- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1cac1565f0 msgr2=0x7f1cac16ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cabfff700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1cac1565f0 0x7f1cac16ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cabfff700 1 -- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cac0669e0 msgr2=0x7f1cac0abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cabfff700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cac0669e0 0x7f1cac0abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cabfff700 1 -- 172.21.15.29:0/4151469435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca00092a0 con 0x7f1cac155b00 2024-01-24T14:14:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cabfff700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cac155b00 0x7f1cac0a7b20 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f1c9800c4c0 tx=0x7f1c98007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1ca8ff9700 1 -- 172.21.15.29:0/4151469435 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98007020 con 0x7f1cac155b00 2024-01-24T14:14:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1ca8ff9700 1 -- 172.21.15.29:0/4151469435 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c98001e90 con 0x7f1cac155b00 2024-01-24T14:14:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1ca8ff9700 1 -- 172.21.15.29:0/4151469435 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c980073b0 con 0x7f1cac155b00 2024-01-24T14:14:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.048+0000 7f1cb319d700 1 -- 172.21.15.29:0/4151469435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac0ac290 con 0x7f1cac155b00 2024-01-24T14:14:36.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.049+0000 7f1cb319d700 1 -- 172.21.15.29:0/4151469435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac0ac850 con 0x7f1cac155b00 2024-01-24T14:14:36.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.050+0000 7f1cb319d700 1 -- 172.21.15.29:0/4151469435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac0ad030 con 0x7f1cac155b00 2024-01-24T14:14:36.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.050+0000 7f1ca8ff9700 1 -- 172.21.15.29:0/4151469435 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1c980068b0 con 0x7f1cac155b00 2024-01-24T14:14:36.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.050+0000 7f1ca8ff9700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c9005f270 0x7f1c90061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.051+0000 7f1ca8ff9700 1 -- 172.21.15.29:0/4151469435 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f1c9807a220 con 0x7f1cac155b00 2024-01-24T14:14:36.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.051+0000 7f1cb0f39700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c9005f270 0x7f1c90061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.051+0000 7f1cb0f39700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c9005f270 0x7f1c90061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f1ca0009760 tx=0x7f1ca0014000).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:36.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.055+0000 7f1ca8ff9700 1 -- 172.21.15.29:0/4151469435 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1c9804aee0 con 0x7f1cac155b00 2024-01-24T14:14:36.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.237+0000 7f1cb319d700 1 -- 172.21.15.29:0/4151469435 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cac0475e0 con 0x7f1cac155b00 2024-01-24T14:14:36.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.238+0000 7f1ca8ff9700 1 -- 172.21.15.29:0/4151469435 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+18198 (secure 0 0 0) 0x7f1c9804e500 con 0x7f1cac155b00 2024-01-24T14:14:36.239 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:36.239 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":119,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:14:35.869851+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:14:33.610829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-24T14:14:34.834814+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:14:36.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.240+0000 7f1cb319d700 1 -- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c9005f270 msgr2=0x7f1c90061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.240+0000 7f1cb319d700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c9005f270 0x7f1c90061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f1ca0009760 tx=0x7f1ca0014000).stop 2024-01-24T14:14:36.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.240+0000 7f1cb319d700 1 -- 172.21.15.29:0/4151469435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cac155b00 msgr2=0x7f1cac0a7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.240+0000 7f1cb319d700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cac155b00 0x7f1cac0a7b20 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f1c9800c4c0 tx=0x7f1c98007e00).stop 2024-01-24T14:14:36.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.240+0000 7f1cb319d700 1 -- 172.21.15.29:0/4151469435 shutdown_connections 2024-01-24T14:14:36.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.240+0000 7f1cb319d700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1cac1565f0 0x7f1cac16ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.240+0000 7f1cb319d700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cac0669e0 0x7f1cac0abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.240+0000 7f1cb319d700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1c9005f270 0x7f1c90061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.240+0000 7f1cb319d700 1 --2- 172.21.15.29:0/4151469435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1cac155b00 0x7f1cac0a7b20 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.240+0000 7f1cb319d700 1 -- 172.21.15.29:0/4151469435 >> 172.21.15.29:0/4151469435 conn(0x7f1cac0c8bb0 msgr2=0x7f1cac13b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:36.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.240+0000 7f1cb319d700 1 -- 172.21.15.29:0/4151469435 shutdown_connections 2024-01-24T14:14:36.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.240+0000 7f1cb319d700 1 -- 172.21.15.29:0/4151469435 wait complete. 2024-01-24T14:14:36.255 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:14:36.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.383+0000 7f786c5fa700 1 Processor -- start 2024-01-24T14:14:36.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.383+0000 7f786c5fa700 1 -- start start 2024-01-24T14:14:36.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786c5fa700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7864150cb0 0x7f7864069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786c5fa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7864065e50 0x7f7864154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786c5fa700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7864154e40 0x7f7864151090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786c5fa700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78640c2c70 con 0x7f7864154e40 2024-01-24T14:14:36.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786c5fa700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786406a3a0 con 0x7f7864150cb0 2024-01-24T14:14:36.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786c5fa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78641552d0 con 0x7f7864065e50 2024-01-24T14:14:36.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786ab97700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7864154e40 0x7f7864151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786ab97700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7864154e40 0x7f7864151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41358/0 (socket says 172.21.15.29:41358) 2024-01-24T14:14:36.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786ab97700 1 -- 172.21.15.29:0/400320953 learned_addr learned my addr 172.21.15.29:0/400320953 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:36.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f7869b95700 1 --2- 172.21.15.29:0/400320953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7864065e50 0x7f7864154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786a396700 1 --2- 172.21.15.29:0/400320953 >> v2:172.21.15.163:3300/0 conn(0x7f7864150cb0 0x7f7864069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786ab97700 1 -- 172.21.15.29:0/400320953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7864065e50 msgr2=0x7f7864154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:36.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786ab97700 1 --2- 172.21.15.29:0/400320953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7864065e50 0x7f7864154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786ab97700 1 -- 172.21.15.29:0/400320953 >> v2:172.21.15.163:3300/0 conn(0x7f7864150cb0 msgr2=0x7f7864069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:36.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786ab97700 1 --2- 172.21.15.29:0/400320953 >> v2:172.21.15.163:3300/0 conn(0x7f7864150cb0 0x7f7864069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786ab97700 1 -- 172.21.15.29:0/400320953 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864069820 con 0x7f7864154e40 2024-01-24T14:14:36.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f786ab97700 1 --2- 172.21.15.29:0/400320953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7864154e40 0x7f7864151090 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f786406a8e0 tx=0x7f785c00ba50).ready entity=mon.0 client_cookie=b980d1b58a8f4a6f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:36.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f7869394700 1 -- 172.21.15.29:0/400320953 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c010070 con 0x7f7864154e40 2024-01-24T14:14:36.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.385+0000 7f7869394700 1 -- 172.21.15.29:0/400320953 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c00dd30 con 0x7f7864154e40 2024-01-24T14:14:36.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.386+0000 7f7869394700 1 -- 172.21.15.29:0/400320953 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c00dea0 con 0x7f7864154e40 2024-01-24T14:14:36.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.386+0000 7f786c5fa700 1 -- 172.21.15.29:0/400320953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7864154e40 msgr2=0x7f7864151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.386+0000 7f786c5fa700 1 --2- 172.21.15.29:0/400320953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7864154e40 0x7f7864151090 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f786406a8e0 tx=0x7f785c00ba50).stop 2024-01-24T14:14:36.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.386+0000 7f786c5fa700 1 -- 172.21.15.29:0/400320953 shutdown_connections 2024-01-24T14:14:36.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.386+0000 7f786c5fa700 1 --2- 172.21.15.29:0/400320953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7864154e40 0x7f7864151090 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.386+0000 7f786c5fa700 1 --2- 172.21.15.29:0/400320953 >> v2:172.21.15.163:3300/0 conn(0x7f7864150cb0 0x7f7864069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.386+0000 7f786c5fa700 1 --2- 172.21.15.29:0/400320953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7864065e50 0x7f7864154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.386+0000 7f786c5fa700 1 -- 172.21.15.29:0/400320953 >> 172.21.15.29:0/400320953 conn(0x7f78640c85c0 msgr2=0x7f78640c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:36.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.386+0000 7f786c5fa700 1 -- 172.21.15.29:0/400320953 shutdown_connections 2024-01-24T14:14:36.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.386+0000 7f786c5fa700 1 -- 172.21.15.29:0/400320953 wait complete. 2024-01-24T14:14:36.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.387+0000 7f786c5fa700 1 Processor -- start 2024-01-24T14:14:36.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.387+0000 7f786c5fa700 1 -- start start 2024-01-24T14:14:36.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.388+0000 7f786c5fa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7864065e50 0x7f78640ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.388+0000 7f786c5fa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7864150cb0 0x7f786414a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.388+0000 7f786c5fa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7864154e40 0x7f78640aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.388+0000 7f786c5fa700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786414aca0 con 0x7f7864065e50 2024-01-24T14:14:36.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.388+0000 7f786c5fa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78640ac150 con 0x7f7864150cb0 2024-01-24T14:14:36.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.388+0000 7f786c5fa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78640c1eb0 con 0x7f7864154e40 2024-01-24T14:14:36.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.388+0000 7f786a396700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7864065e50 0x7f78640ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.388+0000 7f786a396700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7864065e50 0x7f78640ac5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41368/0 (socket says 172.21.15.29:41368) 2024-01-24T14:14:36.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.388+0000 7f786a396700 1 -- 172.21.15.29:0/1825615730 learned_addr learned my addr 172.21.15.29:0/1825615730 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:36.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f7869b95700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7864150cb0 0x7f786414a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f786ab97700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7864154e40 0x7f78640aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f786a396700 1 -- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7864154e40 msgr2=0x7f78640aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f786a396700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7864154e40 0x7f78640aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f786a396700 1 -- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7864150cb0 msgr2=0x7f786414a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f786a396700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7864150cb0 0x7f786414a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f786a396700 1 -- 172.21.15.29:0/1825615730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c00bcf0 con 0x7f7864065e50 2024-01-24T14:14:36.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f786a396700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7864065e50 0x7f78640ac5e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f786000e930 tx=0x7f7860009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:36.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f785a7fc700 1 -- 172.21.15.29:0/1825615730 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786000c680 con 0x7f7864065e50 2024-01-24T14:14:36.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f785a7fc700 1 -- 172.21.15.29:0/1825615730 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786000c7e0 con 0x7f7864065e50 2024-01-24T14:14:36.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f786c5fa700 1 -- 172.21.15.29:0/1825615730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78640ab540 con 0x7f7864065e50 2024-01-24T14:14:36.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f785a7fc700 1 -- 172.21.15.29:0/1825615730 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786000cb60 con 0x7f7864065e50 2024-01-24T14:14:36.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.389+0000 7f786c5fa700 1 -- 172.21.15.29:0/1825615730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78640abb00 con 0x7f7864065e50 2024-01-24T14:14:36.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.390+0000 7f786c5fa700 1 -- 172.21.15.29:0/1825615730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7864066230 con 0x7f7864065e50 2024-01-24T14:14:36.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.391+0000 7f785a7fc700 1 -- 172.21.15.29:0/1825615730 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f786001b400 con 0x7f7864065e50 2024-01-24T14:14:36.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.391+0000 7f785a7fc700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f784805f270 0x7f7848061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.391+0000 7f785a7fc700 1 -- 172.21.15.29:0/1825615730 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f786007ac90 con 0x7f7864065e50 2024-01-24T14:14:36.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.391+0000 7f7869b95700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f784805f270 0x7f7848061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.392+0000 7f7869b95700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f784805f270 0x7f7848061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f78640c1940 tx=0x7f7854009060).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:36.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.396+0000 7f785a7fc700 1 -- 172.21.15.29:0/1825615730 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f786004ba30 con 0x7f7864065e50 2024-01-24T14:14:36.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.579+0000 7f786c5fa700 1 -- 172.21.15.29:0/1825615730 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78640475e0 con 0x7f7864065e50 2024-01-24T14:14:36.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.580+0000 7f785a7fc700 1 -- 172.21.15.29:0/1825615730 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+18198 (secure 0 0 0) 0x7f786004f050 con 0x7f7864065e50 2024-01-24T14:14:36.581 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:36.582 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":119,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:14:35.869851+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:14:33.610829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-24T14:14:34.834814+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:14:36.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.582+0000 7f786c5fa700 1 -- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f784805f270 msgr2=0x7f7848061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.582+0000 7f786c5fa700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f784805f270 0x7f7848061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f78640c1940 tx=0x7f7854009060).stop 2024-01-24T14:14:36.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.582+0000 7f786c5fa700 1 -- 172.21.15.29:0/1825615730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7864065e50 msgr2=0x7f78640ac5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.582+0000 7f786c5fa700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7864065e50 0x7f78640ac5e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f786000e930 tx=0x7f7860009060).stop 2024-01-24T14:14:36.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.582+0000 7f786c5fa700 1 -- 172.21.15.29:0/1825615730 shutdown_connections 2024-01-24T14:14:36.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.582+0000 7f786c5fa700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7864154e40 0x7f78640aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.582+0000 7f786c5fa700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f784805f270 0x7f7848061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.582+0000 7f786c5fa700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7864065e50 0x7f78640ac5e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.582+0000 7f786c5fa700 1 --2- 172.21.15.29:0/1825615730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7864150cb0 0x7f786414a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.582+0000 7f786c5fa700 1 -- 172.21.15.29:0/1825615730 >> 172.21.15.29:0/1825615730 conn(0x7f78640c85c0 msgr2=0x7f78640c3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:36.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.582+0000 7f786c5fa700 1 -- 172.21.15.29:0/1825615730 shutdown_connections 2024-01-24T14:14:36.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.582+0000 7f786c5fa700 1 -- 172.21.15.29:0/1825615730 wait complete. 2024-01-24T14:14:36.597 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-24T14:14:36.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.731+0000 7fa23160a700 1 Processor -- start 2024-01-24T14:14:36.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.731+0000 7fa23160a700 1 -- start start 2024-01-24T14:14:36.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.732+0000 7fa23160a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa22c0c8490 0x7fa22c0a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.732+0000 7fa23160a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa22c0bfd40 0x7fa22c0c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.732+0000 7fa23160a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22c152240 con 0x7fa22c064760 2024-01-24T14:14:36.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.732+0000 7fa23160a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22c1586c0 con 0x7fa22c0c8490 2024-01-24T14:14:36.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.732+0000 7fa23160a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c0aa670 con 0x7fa22c0bfd40 2024-01-24T14:14:36.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.732+0000 7fa22affd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa22c064760 0x7fa22c064b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46962/0 (socket says 172.21.15.29:46962) 2024-01-24T14:14:36.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.732+0000 7fa22affd700 1 -- 172.21.15.29:0/2652209762 learned_addr learned my addr 172.21.15.29:0/2652209762 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:36.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.732+0000 7fa22b7fe700 1 --2- 172.21.15.29:0/2652209762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa22c0bfd40 0x7fa22c0c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.732+0000 7fa22a7fc700 1 --2- 172.21.15.29:0/2652209762 >> v2:172.21.15.163:3300/0 conn(0x7fa22c0c8490 0x7fa22c0a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.732+0000 7fa229ffb700 1 -- 172.21.15.29:0/2652209762 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2841111322 0 0) 0x7fa22c152240 con 0x7fa22c064760 2024-01-24T14:14:36.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.732+0000 7fa229ffb700 1 -- 172.21.15.29:0/2652209762 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2180033f0 con 0x7fa22c064760 2024-01-24T14:14:36.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa22b7fe700 1 -- 172.21.15.29:0/2652209762 >> v2:172.21.15.163:3300/0 conn(0x7fa22c0c8490 msgr2=0x7fa22c0a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:36.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa22b7fe700 1 --2- 172.21.15.29:0/2652209762 >> v2:172.21.15.163:3300/0 conn(0x7fa22c0c8490 0x7fa22c0a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa22b7fe700 1 -- 172.21.15.29:0/2652209762 >> v1:172.21.15.29:6789/0 conn(0x7fa22c064760 legacy=0x7fa22c064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa22b7fe700 1 -- 172.21.15.29:0/2652209762 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c157d90 con 0x7fa22c0bfd40 2024-01-24T14:14:36.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa22a7fc700 1 --2- 172.21.15.29:0/2652209762 >> v2:172.21.15.163:3300/0 conn(0x7fa22c0c8490 0x7fa22c0a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:36.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa22b7fe700 1 --2- 172.21.15.29:0/2652209762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa22c0bfd40 0x7fa22c0c01d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fa224009e60 tx=0x7fa224009780).ready entity=mon.2 client_cookie=b353e389c3b22907 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:36.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa229ffb700 1 -- 172.21.15.29:0/2652209762 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400e070 con 0x7fa22c0bfd40 2024-01-24T14:14:36.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa229ffb700 1 -- 172.21.15.29:0/2652209762 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22400bb40 con 0x7fa22c0bfd40 2024-01-24T14:14:36.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa229ffb700 1 -- 172.21.15.29:0/2652209762 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224012410 con 0x7fa22c0bfd40 2024-01-24T14:14:36.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa23160a700 1 -- 172.21.15.29:0/2652209762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa22c0bfd40 msgr2=0x7fa22c0c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa23160a700 1 --2- 172.21.15.29:0/2652209762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa22c0bfd40 0x7fa22c0c01d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fa224009e60 tx=0x7fa224009780).stop 2024-01-24T14:14:36.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa23160a700 1 -- 172.21.15.29:0/2652209762 shutdown_connections 2024-01-24T14:14:36.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa23160a700 1 --2- 172.21.15.29:0/2652209762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa22c0bfd40 0x7fa22c0c01d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa23160a700 1 --2- 172.21.15.29:0/2652209762 >> v2:172.21.15.163:3300/0 conn(0x7fa22c0c8490 0x7fa22c0a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.733+0000 7fa23160a700 1 -- 172.21.15.29:0/2652209762 >> 172.21.15.29:0/2652209762 conn(0x7fa22c0cb8b0 msgr2=0x7fa22c0c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:36.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.734+0000 7fa23160a700 1 -- 172.21.15.29:0/2652209762 shutdown_connections 2024-01-24T14:14:36.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.734+0000 7fa23160a700 1 -- 172.21.15.29:0/2652209762 wait complete. 2024-01-24T14:14:36.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.735+0000 7fa23160a700 1 Processor -- start 2024-01-24T14:14:36.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.735+0000 7fa23160a700 1 -- start start 2024-01-24T14:14:36.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa23160a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa22c064760 0x7fa22c0b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa23160a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa22c0bfd40 0x7fa22c146730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa23160a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa22c0c8490 0x7fa22c154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa23160a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c167d00 con 0x7fa22c0c8490 2024-01-24T14:14:36.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa23160a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c167fa0 con 0x7fa22c064760 2024-01-24T14:14:36.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa23160a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c168280 con 0x7fa22c0bfd40 2024-01-24T14:14:36.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa22b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa22c0c8490 0x7fa22c154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa22b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa22c0c8490 0x7fa22c154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41372/0 (socket says 172.21.15.29:41372) 2024-01-24T14:14:36.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa22b7fe700 1 -- 172.21.15.29:0/405378713 learned_addr learned my addr 172.21.15.29:0/405378713 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:36.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa22a7fc700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa22c0bfd40 0x7fa22c146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa22affd700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa22c064760 0x7fa22c0b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa22b7fe700 1 -- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa22c0bfd40 msgr2=0x7fa22c146730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa22b7fe700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa22c0bfd40 0x7fa22c146730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa22b7fe700 1 -- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa22c064760 msgr2=0x7fa22c0b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:36.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa22b7fe700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa22c064760 0x7fa22c0b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:36.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa22b7fe700 1 -- 172.21.15.29:0/405378713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2140092a0 con 0x7fa22c0c8490 2024-01-24T14:14:36.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa22affd700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa22c064760 0x7fa22c0b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:14:36.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.736+0000 7fa22b7fe700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa22c0c8490 0x7fa22c154390 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7fa22400b610 tx=0x7fa224009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:36.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.737+0000 7fa222ffd700 1 -- 172.21.15.29:0/405378713 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22401d700 con 0x7fa22c0c8490 2024-01-24T14:14:36.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.737+0000 7fa222ffd700 1 -- 172.21.15.29:0/405378713 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22401d860 con 0x7fa22c0c8490 2024-01-24T14:14:36.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.737+0000 7fa23160a700 1 -- 172.21.15.29:0/405378713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa224009a20 con 0x7fa22c0c8490 2024-01-24T14:14:36.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.737+0000 7fa222ffd700 1 -- 172.21.15.29:0/405378713 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22401dbe0 con 0x7fa22c0c8490 2024-01-24T14:14:36.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.737+0000 7fa23160a700 1 -- 172.21.15.29:0/405378713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c168840 con 0x7fa22c0c8490 2024-01-24T14:14:36.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.738+0000 7fa23160a700 1 -- 172.21.15.29:0/405378713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22c0c1e60 con 0x7fa22c0c8490 2024-01-24T14:14:36.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.738+0000 7fa222ffd700 1 -- 172.21.15.29:0/405378713 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa224003040 con 0x7fa22c0c8490 2024-01-24T14:14:36.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.739+0000 7fa222ffd700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa20c05f270 0x7fa20c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:36.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.739+0000 7fa222ffd700 1 -- 172.21.15.29:0/405378713 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fa22407c7a0 con 0x7fa22c0c8490 2024-01-24T14:14:36.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.739+0000 7fa22affd700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa20c05f270 0x7fa20c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:36.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.739+0000 7fa22affd700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa20c05f270 0x7fa20c061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa21c005750 tx=0x7fa21c009040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:36.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.743+0000 7fa222ffd700 1 -- 172.21.15.29:0/405378713 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa22404d4f0 con 0x7fa22c0c8490 2024-01-24T14:14:36.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:36.974+0000 7fa23160a700 1 -- 172.21.15.29:0/405378713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa22c0475e0 con 0x7fa22c0c8490 2024-01-24T14:14:37.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.906+0000 7fa222ffd700 1 -- 172.21.15.29:0/405378713 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa224050b10 con 0x7fa22c0c8490 2024-01-24T14:14:37.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.908+0000 7fa23160a700 1 -- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa20c05f270 msgr2=0x7fa20c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:37.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.908+0000 7fa23160a700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa20c05f270 0x7fa20c061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa21c005750 tx=0x7fa21c009040).stop 2024-01-24T14:14:37.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.908+0000 7fa23160a700 1 -- 172.21.15.29:0/405378713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa22c0c8490 msgr2=0x7fa22c154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:37.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.908+0000 7fa23160a700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa22c0c8490 0x7fa22c154390 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7fa22400b610 tx=0x7fa224009780).stop 2024-01-24T14:14:37.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.908+0000 7fa23160a700 1 -- 172.21.15.29:0/405378713 shutdown_connections 2024-01-24T14:14:37.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.908+0000 7fa23160a700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa20c05f270 0x7fa20c061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:37.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.908+0000 7fa23160a700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa22c0c8490 0x7fa22c154390 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:37.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.908+0000 7fa23160a700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa22c064760 0x7fa22c0b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:37.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.908+0000 7fa23160a700 1 --2- 172.21.15.29:0/405378713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa22c0bfd40 0x7fa22c146730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:37.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.908+0000 7fa23160a700 1 -- 172.21.15.29:0/405378713 >> 172.21.15.29:0/405378713 conn(0x7fa22c0cb8b0 msgr2=0x7fa22c155d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:37.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.908+0000 7fa23160a700 1 -- 172.21.15.29:0/405378713 shutdown_connections 2024-01-24T14:14:37.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:37.908+0000 7fa23160a700 1 -- 172.21.15.29:0/405378713 wait complete. 2024-01-24T14:14:37.924 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-24T14:14:37.925 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-24T14:14:38.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.060+0000 7f944c555700 1 Processor -- start 2024-01-24T14:14:38.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.060+0000 7f944c555700 1 -- start start 2024-01-24T14:14:38.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.061+0000 7f944c555700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f944413a5e0 0x7f94440639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:38.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.061+0000 7f944c555700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 0x7f94440664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:38.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.061+0000 7f944c555700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9444066e20 con 0x7f94440669b0 2024-01-24T14:14:38.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.061+0000 7f944c555700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94440679f0 con 0x7f944413a5e0 2024-01-24T14:14:38.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.061+0000 7f944c555700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9444067cd0 con 0x7f9444063eb0 2024-01-24T14:14:38.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f9449af0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 0x7f94440664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:38.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f944a2f1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f944413a5e0 0x7f94440639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:38.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f9449af0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 0x7f94440664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:53760/0 (socket says 172.21.15.29:53760) 2024-01-24T14:14:38.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f944aaf2700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f94440669b0 0x7f9444157bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:59860/0 (socket says 172.21.15.29:59860) 2024-01-24T14:14:38.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f9449af0700 1 -- 172.21.15.29:0/970600265 learned_addr learned my addr 172.21.15.29:0/970600265 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:38.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f944a2f1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f944413a5e0 0x7f94440639c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:52208/0 (socket says 172.21.15.29:52208) 2024-01-24T14:14:38.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f9449af0700 1 -- 172.21.15.29:0/970600265 >> v2:172.21.15.163:3300/0 conn(0x7f944413a5e0 msgr2=0x7f94440639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:38.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f9449af0700 1 --2- 172.21.15.29:0/970600265 >> v2:172.21.15.163:3300/0 conn(0x7f944413a5e0 0x7f94440639c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f9449af0700 1 -- 172.21.15.29:0/970600265 >> v1:172.21.15.163:6789/0 conn(0x7f94440669b0 legacy=0x7f9444157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f9449af0700 1 -- 172.21.15.29:0/970600265 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9444067fb0 con 0x7f9444063eb0 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f94492ef700 1 -- 172.21.15.29:0/970600265 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110484747 0 0) 0x7f9444066e20 con 0x7f94440669b0 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f944a2f1700 1 --2- 172.21.15.29:0/970600265 >> v2:172.21.15.163:3300/0 conn(0x7f944413a5e0 0x7f94440639c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.062+0000 7f9449af0700 1 --2- 172.21.15.29:0/970600265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 0x7f94440664c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f9440006fd0 tx=0x7f94400090f0).ready entity=mon.2 client_cookie=a619953e1ac6957d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.063+0000 7f94492ef700 1 -- 172.21.15.29:0/970600265 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944000b580 con 0x7f9444063eb0 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.063+0000 7f94492ef700 1 -- 172.21.15.29:0/970600265 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944000b6e0 con 0x7f9444063eb0 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.063+0000 7f94492ef700 1 -- 172.21.15.29:0/970600265 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944000ba60 con 0x7f9444063eb0 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.063+0000 7f944c555700 1 -- 172.21.15.29:0/970600265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 msgr2=0x7f94440664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.063+0000 7f944c555700 1 --2- 172.21.15.29:0/970600265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 0x7f94440664c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f9440006fd0 tx=0x7f94400090f0).stop 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.063+0000 7f944c555700 1 -- 172.21.15.29:0/970600265 shutdown_connections 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.063+0000 7f944c555700 1 --2- 172.21.15.29:0/970600265 >> v2:172.21.15.163:3300/0 conn(0x7f944413a5e0 0x7f94440639c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:38.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.063+0000 7f944c555700 1 --2- 172.21.15.29:0/970600265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 0x7f94440664c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:38.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.063+0000 7f944c555700 1 -- 172.21.15.29:0/970600265 >> 172.21.15.29:0/970600265 conn(0x7f94440c8bd0 msgr2=0x7f94440c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:38.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.063+0000 7f944c555700 1 -- 172.21.15.29:0/970600265 shutdown_connections 2024-01-24T14:14:38.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.063+0000 7f944c555700 1 -- 172.21.15.29:0/970600265 wait complete. 2024-01-24T14:14:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.064+0000 7f944c555700 1 Processor -- start 2024-01-24T14:14:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.065+0000 7f944c555700 1 -- start start 2024-01-24T14:14:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.065+0000 7f944c555700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 0x7f9444165510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.065+0000 7f944c555700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94440669b0 0x7f9444153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:38.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.065+0000 7f944c555700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f944413a5e0 0x7f94440ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.065+0000 7f944c555700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94440ae840 con 0x7f94440669b0 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.065+0000 7f944c555700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94440adc30 con 0x7f944413a5e0 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.065+0000 7f944c555700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94440adf10 con 0x7f9444063eb0 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.065+0000 7f9449af0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94440669b0 0x7f9444153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f9449af0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94440669b0 0x7f9444153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41376/0 (socket says 172.21.15.29:41376) 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f9449af0700 1 -- 172.21.15.29:0/386295856 learned_addr learned my addr 172.21.15.29:0/386295856 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f944a2f1700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 0x7f9444165510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f944aaf2700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f944413a5e0 0x7f94440ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f944a2f1700 1 -- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f944413a5e0 msgr2=0x7f94440ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f944a2f1700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f944413a5e0 0x7f94440ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f944a2f1700 1 -- 172.21.15.29:0/386295856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94440669b0 msgr2=0x7f9444153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f944a2f1700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94440669b0 0x7f9444153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f944a2f1700 1 -- 172.21.15.29:0/386295856 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380092a0 con 0x7f9444063eb0 2024-01-24T14:14:38.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f9449af0700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94440669b0 0x7f9444153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f944a2f1700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 0x7f9444165510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f943800e980 tx=0x7f9438009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f94367fc700 1 -- 172.21.15.29:0/386295856 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438012600 con 0x7f9444063eb0 2024-01-24T14:14:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f94367fc700 1 -- 172.21.15.29:0/386295856 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9438012760 con 0x7f9444063eb0 2024-01-24T14:14:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f944c555700 1 -- 172.21.15.29:0/386295856 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9440009370 con 0x7f9444063eb0 2024-01-24T14:14:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.066+0000 7f94367fc700 1 -- 172.21.15.29:0/386295856 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438012ae0 con 0x7f9444063eb0 2024-01-24T14:14:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.067+0000 7f944c555700 1 -- 172.21.15.29:0/386295856 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94440ae4d0 con 0x7f9444063eb0 2024-01-24T14:14:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.068+0000 7f944c555700 1 -- 172.21.15.29:0/386295856 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94440abcb0 con 0x7f9444063eb0 2024-01-24T14:14:38.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.068+0000 7f94367fc700 1 -- 172.21.15.29:0/386295856 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f943801b400 con 0x7f9444063eb0 2024-01-24T14:14:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.068+0000 7f94367fc700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f942805f270 0x7f9428061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.068+0000 7f94367fc700 1 -- 172.21.15.29:0/386295856 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f943807ac60 con 0x7f9444063eb0 2024-01-24T14:14:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.069+0000 7f9449af0700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f942805f270 0x7f9428061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:38.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.069+0000 7f9449af0700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f942805f270 0x7f9428061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f94440a8400 tx=0x7f9440017000).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:38.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.073+0000 7f94367fc700 1 -- 172.21.15.29:0/386295856 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f943804b950 con 0x7f9444063eb0 2024-01-24T14:14:38.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.299+0000 7f944c555700 1 -- 172.21.15.29:0/386295856 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f94440acfe0 con 0x7f9444063eb0 2024-01-24T14:14:38.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.912+0000 7f94367fc700 1 -- 172.21.15.29:0/386295856 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v51) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9438016020 con 0x7f9444063eb0 2024-01-24T14:14:38.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.914+0000 7f944c555700 1 -- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f942805f270 msgr2=0x7f9428061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:38.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.914+0000 7f944c555700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f942805f270 0x7f9428061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f94440a8400 tx=0x7f9440017000).stop 2024-01-24T14:14:38.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.914+0000 7f944c555700 1 -- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 msgr2=0x7f9444165510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:38.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.914+0000 7f944c555700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 0x7f9444165510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f943800e980 tx=0x7f9438009060).stop 2024-01-24T14:14:38.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.914+0000 7f944c555700 1 -- 172.21.15.29:0/386295856 shutdown_connections 2024-01-24T14:14:38.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.914+0000 7f944c555700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f944413a5e0 0x7f94440ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:38.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.914+0000 7f944c555700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9444063eb0 0x7f9444165510 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:38.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.914+0000 7f944c555700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f942805f270 0x7f9428061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:38.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.914+0000 7f944c555700 1 --2- 172.21.15.29:0/386295856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94440669b0 0x7f9444153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:38.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.914+0000 7f944c555700 1 -- 172.21.15.29:0/386295856 >> 172.21.15.29:0/386295856 conn(0x7f94440c8bd0 msgr2=0x7f9444155c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:38.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.914+0000 7f944c555700 1 -- 172.21.15.29:0/386295856 shutdown_connections 2024-01-24T14:14:38.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:38.914+0000 7f944c555700 1 -- 172.21.15.29:0/386295856 wait complete. 2024-01-24T14:14:38.930 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-24T14:14:38.930 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:14:39.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.063+0000 7f748ea53700 1 Processor -- start 2024-01-24T14:14:39.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.063+0000 7f748ea53700 1 -- start start 2024-01-24T14:14:39.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.064+0000 7f748ea53700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7488062ec0 0x7f74880c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.064+0000 7f748ea53700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74881550f0 0x7f7488069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.064+0000 7f748ea53700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7488065360 con 0x7f74881540a0 2024-01-24T14:14:39.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.064+0000 7f748ea53700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74880a74f0 con 0x7f7488062ec0 2024-01-24T14:14:39.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.064+0000 7f748ea53700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74880a6ce0 con 0x7f74881550f0 2024-01-24T14:14:39.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.064+0000 7f7487fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f74881540a0 0x7f74880627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46972/0 (socket says 172.21.15.29:46972) 2024-01-24T14:14:39.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.064+0000 7f7487fff700 1 -- 172.21.15.29:0/1443330185 learned_addr learned my addr 172.21.15.29:0/1443330185 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:39.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.064+0000 7f74877fe700 1 --2- 172.21.15.29:0/1443330185 >> v2:172.21.15.163:3300/0 conn(0x7f7488062ec0 0x7f74880c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.064+0000 7f748cff0700 1 --2- 172.21.15.29:0/1443330185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74881550f0 0x7f7488069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.064+0000 7f7486ffd700 1 -- 172.21.15.29:0/1443330185 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588224395 0 0) 0x7f7488065360 con 0x7f74881540a0 2024-01-24T14:14:39.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.065+0000 7f7486ffd700 1 -- 172.21.15.29:0/1443330185 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74740033f0 con 0x7f74881540a0 2024-01-24T14:14:39.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.065+0000 7f7486ffd700 1 -- 172.21.15.29:0/1443330185 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2023096852 0 0) 0x7f74740033f0 con 0x7f74881540a0 2024-01-24T14:14:39.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.065+0000 7f748cff0700 1 -- 172.21.15.29:0/1443330185 >> v2:172.21.15.163:3300/0 conn(0x7f7488062ec0 msgr2=0x7f74880c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:39.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.065+0000 7f748cff0700 1 --2- 172.21.15.29:0/1443330185 >> v2:172.21.15.163:3300/0 conn(0x7f7488062ec0 0x7f74880c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.065+0000 7f748cff0700 1 -- 172.21.15.29:0/1443330185 >> v1:172.21.15.29:6789/0 conn(0x7f74881540a0 legacy=0x7f74880627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.065+0000 7f748cff0700 1 -- 172.21.15.29:0/1443330185 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74880636f0 con 0x7f74881550f0 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.065+0000 7f74877fe700 1 --2- 172.21.15.29:0/1443330185 >> v2:172.21.15.163:3300/0 conn(0x7f7488062ec0 0x7f74880c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.065+0000 7f748cff0700 1 --2- 172.21.15.29:0/1443330185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74881550f0 0x7f7488069240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f7480009e60 tx=0x7f7480009780).ready entity=mon.2 client_cookie=2e9008adaa13fdb6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.065+0000 7f7486ffd700 1 -- 172.21.15.29:0/1443330185 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748000e070 con 0x7f74881550f0 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.065+0000 7f7486ffd700 1 -- 172.21.15.29:0/1443330185 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748000b7a0 con 0x7f74881550f0 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.065+0000 7f7486ffd700 1 -- 172.21.15.29:0/1443330185 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470001470 con 0x7f74881550f0 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.066+0000 7f748ea53700 1 -- 172.21.15.29:0/1443330185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74881550f0 msgr2=0x7f7488069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.066+0000 7f748ea53700 1 --2- 172.21.15.29:0/1443330185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74881550f0 0x7f7488069240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f7480009e60 tx=0x7f7480009780).stop 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.066+0000 7f748ea53700 1 -- 172.21.15.29:0/1443330185 shutdown_connections 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.066+0000 7f748ea53700 1 --2- 172.21.15.29:0/1443330185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74881550f0 0x7f7488069240 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.066+0000 7f748ea53700 1 --2- 172.21.15.29:0/1443330185 >> v2:172.21.15.163:3300/0 conn(0x7f7488062ec0 0x7f74880c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.066+0000 7f748ea53700 1 -- 172.21.15.29:0/1443330185 >> 172.21.15.29:0/1443330185 conn(0x7f74880c85c0 msgr2=0x7f74880c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:39.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.066+0000 7f748ea53700 1 -- 172.21.15.29:0/1443330185 shutdown_connections 2024-01-24T14:14:39.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.066+0000 7f748ea53700 1 -- 172.21.15.29:0/1443330185 wait complete. 2024-01-24T14:14:39.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.067+0000 7f748ea53700 1 Processor -- start 2024-01-24T14:14:39.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.067+0000 7f748ea53700 1 -- start start 2024-01-24T14:14:39.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.068+0000 7f748ea53700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488062ec0 0x7f74881641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.068+0000 7f748ea53700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74881540a0 0x7f74880beb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.068+0000 7f748ea53700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74881550f0 0x7f74880c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.068+0000 7f748ea53700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880c0120 con 0x7f7488062ec0 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.068+0000 7f748ea53700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74881646d0 con 0x7f74881550f0 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.068+0000 7f748ea53700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74880bf510 con 0x7f74881540a0 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.068+0000 7f7487fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488062ec0 0x7f74881641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.068+0000 7f7487fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488062ec0 0x7f74881641e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41382/0 (socket says 172.21.15.29:41382) 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.068+0000 7f7487fff700 1 -- 172.21.15.29:0/1162089273 learned_addr learned my addr 172.21.15.29:0/1162089273 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.068+0000 7f74877fe700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74881540a0 0x7f74880beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.068+0000 7f748cff0700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74881550f0 0x7f74880c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.069+0000 7f7487fff700 1 -- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74881540a0 msgr2=0x7f74880beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.069+0000 7f7487fff700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74881540a0 0x7f74880beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.069+0000 7f7487fff700 1 -- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74881550f0 msgr2=0x7f74880c05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.069+0000 7f7487fff700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74881550f0 0x7f74880c05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.069+0000 7f7487fff700 1 -- 172.21.15.29:0/1162089273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74780092a0 con 0x7f7488062ec0 2024-01-24T14:14:39.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.069+0000 7f7487fff700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488062ec0 0x7f74881641e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f747000c840 tx=0x7f7470001ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:39.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.069+0000 7f747ffff700 1 -- 172.21.15.29:0/1162089273 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470007170 con 0x7f7488062ec0 2024-01-24T14:14:39.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.069+0000 7f747ffff700 1 -- 172.21.15.29:0/1162089273 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74700072d0 con 0x7f7488062ec0 2024-01-24T14:14:39.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.069+0000 7f747ffff700 1 -- 172.21.15.29:0/1162089273 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74700048c0 con 0x7f7488062ec0 2024-01-24T14:14:39.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.069+0000 7f748ea53700 1 -- 172.21.15.29:0/1162089273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7480009a20 con 0x7f7488062ec0 2024-01-24T14:14:39.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.069+0000 7f748ea53700 1 -- 172.21.15.29:0/1162089273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74880bfad0 con 0x7f7488062ec0 2024-01-24T14:14:39.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.070+0000 7f748ea53700 1 -- 172.21.15.29:0/1162089273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74880c0400 con 0x7f7488062ec0 2024-01-24T14:14:39.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.070+0000 7f747ffff700 1 -- 172.21.15.29:0/1162089273 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7470004c70 con 0x7f7488062ec0 2024-01-24T14:14:39.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.071+0000 7f747ffff700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f746805f270 0x7f7468061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.071+0000 7f747ffff700 1 -- 172.21.15.29:0/1162089273 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f747007a400 con 0x7f7488062ec0 2024-01-24T14:14:39.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.071+0000 7f74877fe700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f746805f270 0x7f7468061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.072+0000 7f74877fe700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f746805f270 0x7f7468061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7478009270 tx=0x7f7478008040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:39.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.075+0000 7f747ffff700 1 -- 172.21.15.29:0/1162089273 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f747004b150 con 0x7f7488062ec0 2024-01-24T14:14:39.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.304+0000 7f748ea53700 1 -- 172.21.15.29:0/1162089273 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f748806a5f0 con 0x7f7488062ec0 2024-01-24T14:14:39.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.304+0000 7f747ffff700 1 -- 172.21.15.29:0/1162089273 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f747004e770 con 0x7f7488062ec0 2024-01-24T14:14:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:39.305 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6341,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4230829295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4230829295},{"type":"v1","addr":"172.21.15.29:6835","nonce":4230829295}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6344,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3194389571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3194389571},{"type":"v1","addr":"172.21.15.163:6836","nonce":3194389571}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2473497045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2473497045},{"type":"v1","addr":"172.21.15.163:6837","nonce":2473497045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:14:35.870228+0000","modified":"2024-01-24T14:14:38.905491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7254":{"gid":7254,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/3097015540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3097015540},{"type":"v1","addr":"172.21.15.29:6834","nonce":3097015540}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-24T14:14:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.306+0000 7f748ea53700 1 -- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f746805f270 msgr2=0x7f7468061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.306+0000 7f748ea53700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f746805f270 0x7f7468061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7478009270 tx=0x7f7478008040).stop 2024-01-24T14:14:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.306+0000 7f748ea53700 1 -- 172.21.15.29:0/1162089273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488062ec0 msgr2=0x7f74881641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.306+0000 7f748ea53700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488062ec0 0x7f74881641e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f747000c840 tx=0x7f7470001ab0).stop 2024-01-24T14:14:39.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.306+0000 7f748ea53700 1 -- 172.21.15.29:0/1162089273 shutdown_connections 2024-01-24T14:14:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.306+0000 7f748ea53700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74881550f0 0x7f74880c05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.306+0000 7f748ea53700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f746805f270 0x7f7468061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.306+0000 7f748ea53700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488062ec0 0x7f74881641e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.306+0000 7f748ea53700 1 --2- 172.21.15.29:0/1162089273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74881540a0 0x7f74880beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.306+0000 7f748ea53700 1 -- 172.21.15.29:0/1162089273 >> 172.21.15.29:0/1162089273 conn(0x7f74880c85c0 msgr2=0x7f7488069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.306+0000 7f748ea53700 1 -- 172.21.15.29:0/1162089273 shutdown_connections 2024-01-24T14:14:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.306+0000 7f748ea53700 1 -- 172.21.15.29:0/1162089273 wait complete. 2024-01-24T14:14:39.308 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 51 2024-01-24T14:14:39.321 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:14:39.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.450+0000 7f5612284700 1 Processor -- start 2024-01-24T14:14:39.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.450+0000 7f5612284700 1 -- start start 2024-01-24T14:14:39.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.450+0000 7f5612284700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f560c138710 0x7f560c0be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f5612284700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f560c0627b0 0x7f560c062c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f5612284700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560c065770 con 0x7f560c138330 2024-01-24T14:14:39.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f5612284700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c069cf0 con 0x7f560c0627b0 2024-01-24T14:14:39.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f5612284700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c0630f0 con 0x7f560c138710 2024-01-24T14:14:39.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f5610821700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f560c0627b0 0x7f560c062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f560bfff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f560c138330 0x7f560c0bca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46986/0 (socket says 172.21.15.29:46986) 2024-01-24T14:14:39.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f560bfff700 1 -- 172.21.15.29:0/1479416921 learned_addr learned my addr 172.21.15.29:0/1479416921 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:39.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f5610821700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f560c0627b0 0x7f560c062c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41390/0 (socket says 172.21.15.29:41390) 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f560b7fe700 1 --2- 172.21.15.29:0/1479416921 >> v2:172.21.15.163:3300/0 conn(0x7f560c138710 0x7f560c0be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f5610821700 1 -- 172.21.15.29:0/1479416921 >> v2:172.21.15.163:3300/0 conn(0x7f560c138710 msgr2=0x7f560c0be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f5610821700 1 --2- 172.21.15.29:0/1479416921 >> v2:172.21.15.163:3300/0 conn(0x7f560c138710 0x7f560c0be3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f5610821700 1 -- 172.21.15.29:0/1479416921 >> v1:172.21.15.29:6789/0 conn(0x7f560c138330 legacy=0x7f560c0bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f560affd700 1 -- 172.21.15.29:0/1479416921 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253722603 0 0) 0x7f560c065770 con 0x7f560c138330 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f5610821700 1 -- 172.21.15.29:0/1479416921 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c069f60 con 0x7f560c0627b0 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f5610821700 1 --2- 172.21.15.29:0/1479416921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f560c0627b0 0x7f560c062c00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f560c0aa530 tx=0x7f5604009780).ready entity=mon.0 client_cookie=bc7b4ca013dd166c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f560affd700 1 -- 172.21.15.29:0/1479416921 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400e070 con 0x7f560c0627b0 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f560affd700 1 -- 172.21.15.29:0/1479416921 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560400bc30 con 0x7f560c0627b0 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.451+0000 7f560affd700 1 -- 172.21.15.29:0/1479416921 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604012410 con 0x7f560c0627b0 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.452+0000 7f5612284700 1 -- 172.21.15.29:0/1479416921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f560c0627b0 msgr2=0x7f560c062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.452+0000 7f5612284700 1 --2- 172.21.15.29:0/1479416921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f560c0627b0 0x7f560c062c00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f560c0aa530 tx=0x7f5604009780).stop 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.452+0000 7f5612284700 1 -- 172.21.15.29:0/1479416921 shutdown_connections 2024-01-24T14:14:39.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.452+0000 7f5612284700 1 --2- 172.21.15.29:0/1479416921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f560c0627b0 0x7f560c062c00 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.452+0000 7f5612284700 1 --2- 172.21.15.29:0/1479416921 >> v2:172.21.15.163:3300/0 conn(0x7f560c138710 0x7f560c0be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.452+0000 7f5612284700 1 -- 172.21.15.29:0/1479416921 >> 172.21.15.29:0/1479416921 conn(0x7f560c0c85c0 msgr2=0x7f560c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:39.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.452+0000 7f5612284700 1 -- 172.21.15.29:0/1479416921 shutdown_connections 2024-01-24T14:14:39.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.452+0000 7f5612284700 1 -- 172.21.15.29:0/1479416921 wait complete. 2024-01-24T14:14:39.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.454+0000 7f5612284700 1 Processor -- start 2024-01-24T14:14:39.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.454+0000 7f5612284700 1 -- start start 2024-01-24T14:14:39.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.455+0000 7f5612284700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f560c0627b0 0x7f560c0c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.455+0000 7f5612284700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f560c138330 0x7f560c0ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.455+0000 7f5612284700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f560c138710 0x7f560c137480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.455+0000 7f5612284700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0ac5d0 con 0x7f560c138330 2024-01-24T14:14:39.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.455+0000 7f5612284700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0adaa0 con 0x7f560c0627b0 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.455+0000 7f5612284700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0bf4a0 con 0x7f560c138710 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.455+0000 7f560b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f560c138330 0x7f560c0ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.455+0000 7f560b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f560c138330 0x7f560c0ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41404/0 (socket says 172.21.15.29:41404) 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.455+0000 7f560b7fe700 1 -- 172.21.15.29:0/1544807580 learned_addr learned my addr 172.21.15.29:0/1544807580 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.455+0000 7f5610821700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f560c138710 0x7f560c137480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.455+0000 7f560bfff700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f560c0627b0 0x7f560c0c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.456+0000 7f560b7fe700 1 -- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f560c138710 msgr2=0x7f560c137480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.456+0000 7f560b7fe700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f560c138710 0x7f560c137480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.456+0000 7f560b7fe700 1 -- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f560c0627b0 msgr2=0x7f560c0c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.456+0000 7f560b7fe700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f560c0627b0 0x7f560c0c1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.456+0000 7f560b7fe700 1 -- 172.21.15.29:0/1544807580 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5604009a20 con 0x7f560c138330 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.456+0000 7f560b7fe700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f560c138330 0x7f560c0ad5b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f560000d890 tx=0x7f5600008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.456+0000 7f55f3fff700 1 -- 172.21.15.29:0/1544807580 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56000125f0 con 0x7f560c138330 2024-01-24T14:14:39.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.456+0000 7f55f3fff700 1 -- 172.21.15.29:0/1544807580 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5600012750 con 0x7f560c138330 2024-01-24T14:14:39.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.456+0000 7f5612284700 1 -- 172.21.15.29:0/1544807580 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c0bf780 con 0x7f560c138330 2024-01-24T14:14:39.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.456+0000 7f55f3fff700 1 -- 172.21.15.29:0/1544807580 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600012ad0 con 0x7f560c138330 2024-01-24T14:14:39.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.456+0000 7f5612284700 1 -- 172.21.15.29:0/1544807580 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c0bfa60 con 0x7f560c138330 2024-01-24T14:14:39.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.457+0000 7f5612284700 1 -- 172.21.15.29:0/1544807580 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c0475e0 con 0x7f560c138330 2024-01-24T14:14:39.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.457+0000 7f55f3fff700 1 -- 172.21.15.29:0/1544807580 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5600013480 con 0x7f560c138330 2024-01-24T14:14:39.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.458+0000 7f55f3fff700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55f805f270 0x7f55f8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.458+0000 7f55f3fff700 1 -- 172.21.15.29:0/1544807580 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f560007a840 con 0x7f560c138330 2024-01-24T14:14:39.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.458+0000 7f560bfff700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55f805f270 0x7f55f8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.459+0000 7f560bfff700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55f805f270 0x7f55f8061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f55fc0011e0 tx=0x7f55fc009040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:39.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.462+0000 7f55f3fff700 1 -- 172.21.15.29:0/1544807580 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f560004b5f0 con 0x7f560c138330 2024-01-24T14:14:39.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.642+0000 7f5612284700 1 -- 172.21.15.29:0/1544807580 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f560c06a570 con 0x7f560c138330 2024-01-24T14:14:39.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.643+0000 7f55f3fff700 1 -- 172.21.15.29:0/1544807580 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f560004ec10 con 0x7f560c138330 2024-01-24T14:14:39.644 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:39.644 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":120,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:14:36.889111+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:14:33.610829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-24T14:14:34.834814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:14:39.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.645+0000 7f5612284700 1 -- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55f805f270 msgr2=0x7f55f8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.645+0000 7f5612284700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55f805f270 0x7f55f8061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f55fc0011e0 tx=0x7f55fc009040).stop 2024-01-24T14:14:39.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.645+0000 7f5612284700 1 -- 172.21.15.29:0/1544807580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f560c138330 msgr2=0x7f560c0ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.645+0000 7f5612284700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f560c138330 0x7f560c0ad5b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f560000d890 tx=0x7f5600008040).stop 2024-01-24T14:14:39.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.645+0000 7f5612284700 1 -- 172.21.15.29:0/1544807580 shutdown_connections 2024-01-24T14:14:39.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.645+0000 7f5612284700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f560c138710 0x7f560c137480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.645+0000 7f5612284700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f560c0627b0 0x7f560c0c1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.645+0000 7f5612284700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f55f805f270 0x7f55f8061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.645+0000 7f5612284700 1 --2- 172.21.15.29:0/1544807580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f560c138330 0x7f560c0ad5b0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.645+0000 7f5612284700 1 -- 172.21.15.29:0/1544807580 >> 172.21.15.29:0/1544807580 conn(0x7f560c0c85c0 msgr2=0x7f560c0c44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:39.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.645+0000 7f5612284700 1 -- 172.21.15.29:0/1544807580 shutdown_connections 2024-01-24T14:14:39.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.645+0000 7f5612284700 1 -- 172.21.15.29:0/1544807580 wait complete. 2024-01-24T14:14:39.660 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-24T14:14:39.660 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:14:39.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.790+0000 7f1afcae7700 1 Processor -- start 2024-01-24T14:14:39.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.790+0000 7f1afcae7700 1 -- start start 2024-01-24T14:14:39.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.791+0000 7f1afcae7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1af8154190 0x7f1af8069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.791+0000 7f1afcae7700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af806a2c0 con 0x7f1af8062d50 2024-01-24T14:14:39.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.791+0000 7f1afcae7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af80bd190 con 0x7f1af8062970 2024-01-24T14:14:39.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.791+0000 7f1afcae7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af8154620 con 0x7f1af8154190 2024-01-24T14:14:39.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.791+0000 7f1af5d9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1af8062d50 0x7f1af8065360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46992/0 (socket says 172.21.15.29:46992) 2024-01-24T14:14:39.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.791+0000 7f1af5d9b700 1 -- 172.21.15.29:0/1727653126 learned_addr learned my addr 172.21.15.29:0/1727653126 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.791+0000 7f1af6d9d700 1 --2- 172.21.15.29:0/1727653126 >> v2:172.21.15.163:3300/0 conn(0x7f1af8154190 0x7f1af8069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.791+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120038411 0 0) 0x7f1af806a2c0 con 0x7f1af8062d50 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.791+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ae00033f0 con 0x7f1af8062d50 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.791+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3456981710 0 0) 0x7f1ae00033f0 con 0x7f1af8062d50 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.791+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 >> v2:172.21.15.163:3300/0 conn(0x7f1af8154190 msgr2=0x7f1af8069dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 --2- 172.21.15.29:0/1727653126 >> v2:172.21.15.163:3300/0 conn(0x7f1af8154190 0x7f1af8069dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 >> v1:172.21.15.163:6789/0 conn(0x7f1af8062970 legacy=0x7f1af8153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af80c4e60 con 0x7f1af8062d50 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f1aec001470 con 0x7f1af8062d50 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af6d9d700 1 --2- 172.21.15.29:0/1727653126 >> v2:172.21.15.163:3300/0 conn(0x7f1af8154190 0x7f1af8069dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 >> v1:172.21.15.29:6789/0 conn(0x7f1af8062d50 legacy=0x7f1af8065360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 --2- 172.21.15.29:0/1727653126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae00038e0 0x7f1ae0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 --2- 172.21.15.29:0/1727653126 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ae0004690 0x7f1ae0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 --2- 172.21.15.29:0/1727653126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae0005050 0x7f1ae000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af8154620 con 0x7f1ae0005050 2024-01-24T14:14:39.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae000bfe0 con 0x7f1ae00038e0 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae000c1a0 con 0x7f1ae0004690 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af6d9d700 1 --2- 172.21.15.29:0/1727653126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae0005050 0x7f1ae000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af659c700 1 --2- 172.21.15.29:0/1727653126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae00038e0 0x7f1ae0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af5d9b700 1 --2- 172.21.15.29:0/1727653126 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ae0004690 0x7f1ae0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af6d9d700 1 -- 172.21.15.29:0/1727653126 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ae0004690 msgr2=0x7f1ae0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af6d9d700 1 --2- 172.21.15.29:0/1727653126 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ae0004690 0x7f1ae0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af6d9d700 1 -- 172.21.15.29:0/1727653126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae00038e0 msgr2=0x7f1ae0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af6d9d700 1 --2- 172.21.15.29:0/1727653126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae00038e0 0x7f1ae0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af6d9d700 1 -- 172.21.15.29:0/1727653126 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae00033f0 con 0x7f1ae0005050 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af6d9d700 1 -- 172.21.15.29:0/1727653126 reap_dead start 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af6d9d700 1 --2- 172.21.15.29:0/1727653126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae0005050 0x7f1ae000b8e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7f1af00094d0 tx=0x7f1af000b040).ready entity=mon.0 client_cookie=13244e199c75f528 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0017070 con 0x7f1ae0005050 2024-01-24T14:14:39.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af000f040 con 0x7f1ae0005050 2024-01-24T14:14:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.792+0000 7f1af559a700 1 -- 172.21.15.29:0/1727653126 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af000ee70 con 0x7f1ae0005050 2024-01-24T14:14:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.793+0000 7f1afcae7700 1 -- 172.21.15.29:0/1727653126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae0005050 msgr2=0x7f1ae000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.793+0000 7f1afcae7700 1 --2- 172.21.15.29:0/1727653126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae0005050 0x7f1ae000b8e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7f1af00094d0 tx=0x7f1af000b040).stop 2024-01-24T14:14:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.793+0000 7f1afcae7700 1 -- 172.21.15.29:0/1727653126 shutdown_connections 2024-01-24T14:14:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.793+0000 7f1afcae7700 1 --2- 172.21.15.29:0/1727653126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ae0005050 0x7f1ae000b8e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.793+0000 7f1afcae7700 1 -- 172.21.15.29:0/1727653126 >> 172.21.15.29:0/1727653126 conn(0x7f1af80c85c0 msgr2=0x7f1af80c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.793+0000 7f1afcae7700 1 -- 172.21.15.29:0/1727653126 shutdown_connections 2024-01-24T14:14:39.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.793+0000 7f1afcae7700 1 -- 172.21.15.29:0/1727653126 wait complete. 2024-01-24T14:14:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.794+0000 7f1afcae7700 1 Processor -- start 2024-01-24T14:14:39.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.794+0000 7f1afcae7700 1 -- start start 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.795+0000 7f1afcae7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae0005050 0x7f1af80a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.795+0000 7f1afcae7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1af814d620 0x7f1af80aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.795+0000 7f1afcae7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af80635a0 0x7f1af8063a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.795+0000 7f1afcae7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80a9bf0 con 0x7f1af80635a0 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.795+0000 7f1afcae7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80aaf90 con 0x7f1ae0005050 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.795+0000 7f1afcae7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af814daa0 con 0x7f1af814d620 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.795+0000 7f1af6d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af80635a0 0x7f1af8063a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.795+0000 7f1af6d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af80635a0 0x7f1af8063a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41430/0 (socket says 172.21.15.29:41430) 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.795+0000 7f1af6d9d700 1 -- 172.21.15.29:0/1756310764 learned_addr learned my addr 172.21.15.29:0/1756310764 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.795+0000 7f1af5d9b700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1af814d620 0x7f1af80aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.795+0000 7f1af659c700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae0005050 0x7f1af80a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.796+0000 7f1af6d9d700 1 -- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1af814d620 msgr2=0x7f1af80aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.796+0000 7f1af6d9d700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1af814d620 0x7f1af80aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.796+0000 7f1af6d9d700 1 -- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae0005050 msgr2=0x7f1af80a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.796+0000 7f1af6d9d700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae0005050 0x7f1af80a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.796+0000 7f1af6d9d700 1 -- 172.21.15.29:0/1756310764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af8063f10 con 0x7f1af80635a0 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.796+0000 7f1af6d9d700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af80635a0 0x7f1af8063a20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f1af000e980 tx=0x7f1af0019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.796+0000 7f1ae67fc700 1 -- 172.21.15.29:0/1756310764 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0009c50 con 0x7f1af80635a0 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.796+0000 7f1ae67fc700 1 -- 172.21.15.29:0/1756310764 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af001e640 con 0x7f1af80635a0 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.796+0000 7f1afcae7700 1 -- 172.21.15.29:0/1756310764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af80ab180 con 0x7f1af80635a0 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.796+0000 7f1ae67fc700 1 -- 172.21.15.29:0/1756310764 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af001e7f0 con 0x7f1af80635a0 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.796+0000 7f1afcae7700 1 -- 172.21.15.29:0/1756310764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af80ab740 con 0x7f1af80635a0 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.797+0000 7f1afcae7700 1 -- 172.21.15.29:0/1756310764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af80475e0 con 0x7f1af80635a0 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.797+0000 7f1ae67fc700 1 -- 172.21.15.29:0/1756310764 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1af0024030 con 0x7f1af80635a0 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.798+0000 7f1ae67fc700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ad805f270 0x7f1ad8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.798+0000 7f1ae67fc700 1 -- 172.21.15.29:0/1756310764 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f1af007c8b0 con 0x7f1af80635a0 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.798+0000 7f1af659c700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ad805f270 0x7f1ad8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:39.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.798+0000 7f1af659c700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ad805f270 0x7f1ad8061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1ae8007fd0 tx=0x7f1ae8007bb0).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:39.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.802+0000 7f1ae67fc700 1 -- 172.21.15.29:0/1756310764 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1af004d600 con 0x7f1af80635a0 2024-01-24T14:14:39.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.980+0000 7f1afcae7700 1 -- 172.21.15.29:0/1756310764 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1af8063380 con 0x7f1ad805f270 2024-01-24T14:14:39.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.990+0000 7f1ae67fc700 1 -- 172.21.15.29:0/1756310764 <== mgr.6790 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+350111 (secure 0 0 0) 0x7f1af8063380 con 0x7f1ad805f270 2024-01-24T14:14:39.995 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:39.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.994+0000 7f1afcae7700 1 -- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ad805f270 msgr2=0x7f1ad8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.994+0000 7f1afcae7700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ad805f270 0x7f1ad8061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1ae8007fd0 tx=0x7f1ae8007bb0).stop 2024-01-24T14:14:39.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.994+0000 7f1afcae7700 1 -- 172.21.15.29:0/1756310764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af80635a0 msgr2=0x7f1af8063a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:39.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.994+0000 7f1afcae7700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af80635a0 0x7f1af8063a20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f1af000e980 tx=0x7f1af0019000).stop 2024-01-24T14:14:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.994+0000 7f1afcae7700 1 -- 172.21.15.29:0/1756310764 shutdown_connections 2024-01-24T14:14:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.994+0000 7f1afcae7700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ad805f270 0x7f1ad8061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.994+0000 7f1afcae7700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1af80635a0 0x7f1af8063a20 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.994+0000 7f1afcae7700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae0005050 0x7f1af80a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.994+0000 7f1afcae7700 1 --2- 172.21.15.29:0/1756310764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1af814d620 0x7f1af80aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.994+0000 7f1afcae7700 1 -- 172.21.15.29:0/1756310764 >> 172.21.15.29:0/1756310764 conn(0x7f1af80c85c0 msgr2=0x7f1af80c2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.994+0000 7f1afcae7700 1 -- 172.21.15.29:0/1756310764 shutdown_connections 2024-01-24T14:14:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:39.994+0000 7f1afcae7700 1 -- 172.21.15.29:0/1756310764 wait complete. 2024-01-24T14:14:39.999 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:14:40.009 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T14:14:38.363513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2424796,"kb_used_data":11164,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752549924,"statfs":{"total":773094113280,"available":770611122176,"internally_reserved":0,"allocated":11431936,"data_stored":6340015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5550,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.007771"},"pg_stats":[{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":92,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-24T14:14:30.584895+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:14:30.584895+0000","last_peered":"2024-01-24T14:14:30.584895+0000","last_clean":"2024-01-24T14:14:30.584895+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:14:30.584895+0000","last_undegraded":"2024-01-24T14:14:30.584895+0000","last_fullsized":"2024-01-24T14:14:30.584895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":103,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-24T14:14:30.418526+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:14:30.418526+0000","last_peered":"2024-01-24T14:14:30.418526+0000","last_clean":"2024-01-24T14:14:30.418526+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:14:30.418526+0000","last_undegraded":"2024-01-24T14:14:30.418526+0000","last_fullsized":"2024-01-24T14:14:30.418526+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":99,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-24T14:14:30.418281+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:14:30.418281+0000","last_peered":"2024-01-24T14:14:30.418281+0000","last_clean":"2024-01-24T14:14:30.418281+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:14:30.418281+0000","last_undegraded":"2024-01-24T14:14:30.418281+0000","last_fullsized":"2024-01-24T14:14:30.418281+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":103,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-24T14:14:30.418526+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:14:30.418526+0000","last_peered":"2024-01-24T14:14:30.418526+0000","last_clean":"2024-01-24T14:14:30.418526+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:14:30.418526+0000","last_undegraded":"2024-01-24T14:14:30.418526+0000","last_fullsized":"2024-01-24T14:14:30.418526+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-24T14:14:26.396359+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:14:26.396359+0000","last_peered":"2024-01-24T14:14:26.396359+0000","last_clean":"2024-01-24T14:14:26.396359+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:14:26.396359+0000","last_undegraded":"2024-01-24T14:14:26.396359+0000","last_fullsized":"2024-01-24T14:14:26.396359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":94,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-24T14:14:26.396242+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:14:26.396242+0000","last_peered":"2024-01-24T14:14:26.396242+0000","last_clean":"2024-01-24T14:14:26.396242+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:14:26.396242+0000","last_undegraded":"2024-01-24T14:14:26.396242+0000","last_fullsized":"2024-01-24T14:14:26.396242+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-24T14:14:26.397055+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:14:26.397055+0000","last_peered":"2024-01-24T14:14:26.397055+0000","last_clean":"2024-01-24T14:14:26.397055+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:14:26.397055+0000","last_undegraded":"2024-01-24T14:14:26.397055+0000","last_fullsized":"2024-01-24T14:14:26.397055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":92,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-24T14:14:30.584878+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:14:30.584878+0000","last_peered":"2024-01-24T14:14:30.584878+0000","last_clean":"2024-01-24T14:14:30.584878+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:14:30.584878+0000","last_undegraded":"2024-01-24T14:14:30.584878+0000","last_fullsized":"2024-01-24T14:14:30.584878+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:35.432490+0000","last_change":"2024-01-24T14:14:35.432490+0000","last_active":"2024-01-24T14:14:35.432490+0000","last_peered":"2024-01-24T14:14:35.432490+0000","last_clean":"2024-01-24T14:14:35.432490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:35.432490+0000","last_undegraded":"2024-01-24T14:14:35.432490+0000","last_fullsized":"2024-01-24T14:14:35.432490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:14:34.424934+0000","last_change":"2024-01-24T14:14:34.424934+0000","last_active":"2024-01-24T14:14:34.424934+0000","last_peered":"2024-01-24T14:14:34.424934+0000","last_clean":"2024-01-24T14:14:34.424934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:14:34.424934+0000","last_undegraded":"2024-01-24T14:14:34.424934+0000","last_fullsized":"2024-01-24T14:14:34.424934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":91,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-24T14:14:26.396229+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:14:26.396229+0000","last_peered":"2024-01-24T14:14:26.396229+0000","last_clean":"2024-01-24T14:14:26.396229+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:14:26.396229+0000","last_undegraded":"2024-01-24T14:14:26.396229+0000","last_fullsized":"2024-01-24T14:14:26.396229+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575043,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1429504,"data_stored":792622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.26000000000000001},"max":{"1min":0.80600000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.32700000000000001},"max":{"1min":0.753,"5min":0.82599999999999996,"15min":0.83499999999999996},"last":0.64200000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.312},"max":{"1min":0.75600000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":3.2570000000000001},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.51700000000000002,"5min":0.40400000000000003,"15min":0.32000000000000001},"max":{"1min":0.79100000000000004,"5min":0.872,"15min":0.876},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.39300000000000002,"15min":0.36799999999999999},"max":{"1min":0.80000000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.44900000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.36499999999999999},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.85099999999999998,"15min":3.0049999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48899999999999999},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.84699999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.60399999999999998,"5min":0.70599999999999996,"15min":0.83599999999999997},"last":0.48999999999999999}]},{"osd":5,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.501,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.29999999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.80900000000000005},"last":0.75700000000000001}]},{"osd":6,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.80700000000000005,"15min":0.87},"last":0.54300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1429504,"data_stored":792622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.59499999999999997},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73699999999999999,"5min":0.85699999999999998,"15min":0.86899999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":0.84899999999999998,"15min":1.3859999999999999},"last":0.499}]},{"osd":1,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73499999999999999,"5min":0.77000000000000002,"15min":1.353},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.78700000000000003,"5min":1.032,"15min":1.032},"last":0.48099999999999998}]},{"osd":2,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60099999999999998,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.39100000000000001,"15min":0.34300000000000003},"max":{"1min":0.78600000000000003,"5min":0.88500000000000001,"15min":6.8129999999999997},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.4079999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.66900000000000004},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81699999999999995,"5min":0.82799999999999996,"15min":7.016},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.43099999999999999,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.77400000000000002,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.53600000000000003}]},{"osd":4,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.504,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":1.165,"15min":1.165},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.501,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.314},"max":{"1min":0.65500000000000003,"5min":0.82299999999999995,"15min":6.8719999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":0.86699999999999999,"15min":6.976},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80200000000000005,"5min":0.85699999999999998,"15min":7.8789999999999996},"last":0.54800000000000004}]},{"osd":7,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.747,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":0.89600000000000002,"15min":6.9080000000000004},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834575043,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1429504,"data_stored":792622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.51900000000000002,"15min":0.443},"min":{"1min":0.311,"5min":0.245,"15min":0.23400000000000001},"max":{"1min":0.57899999999999996,"5min":15.298999999999999,"15min":15.298999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.55000000000000004,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.57499999999999996,"5min":14.654,"15min":14.654},"last":0.31900000000000001}]},{"osd":2,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45900000000000002,"15min":0.45100000000000001},"min":{"1min":0.34899999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.59799999999999998,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.25800000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47099999999999997,"15min":0.45700000000000002},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.377},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.33700000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88600000000000001,"5min":1.073,"15min":1.137},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.45900000000000002,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.218,"15min":1.218},"last":0.503}]},{"osd":5,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":1.1040000000000001,"15min":0.83299999999999996},"min":{"1min":0.47999999999999998,"5min":0.45000000000000001,"15min":0.443},"max":{"1min":0.86799999999999999,"5min":38.808,"15min":38.808},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":1.077,"15min":0.81200000000000006},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.40500000000000003},"max":{"1min":0.81299999999999994,"5min":38.823,"15min":38.823},"last":0.53900000000000003}]},{"osd":6,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.46000000000000002,"15min":0.43099999999999999},"max":{"1min":0.94899999999999995,"5min":1.0449999999999999,"15min":1.0509999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.40000000000000002},"max":{"1min":0.91300000000000003,"5min":1.1559999999999999,"15min":1.1879999999999999},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.52500000000000002,"5min":0.49399999999999999,"15min":0.45300000000000001},"max":{"1min":0.98499999999999999,"5min":1.105,"15min":1.155},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.54900000000000004,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":0.93500000000000005,"5min":1.123,"15min":1.131},"last":0.71399999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542341,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302584,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069256,"statfs":{"total":96636764160,"available":96326918144,"internally_reserved":0,"allocated":1425408,"data_stored":791661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39600000000000002},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.57499999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.30599999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44800000000000001,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.28199999999999997},"max":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.66700000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.28799999999999998},"max":{"1min":0.61399999999999999,"5min":0.754,"15min":0.79400000000000004},"last":0.498}]},{"osd":3,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.33000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54500000000000004,"5min":1.482,"15min":1.482},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.496},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":0.872,"15min":0.872},"last":0.56200000000000006}]},{"osd":4,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81499999999999995,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.45500000000000002}]},{"osd":5,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.875,"5min":1.655,"15min":1.655},"last":0.434},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.42999999999999999},"max":{"1min":0.77200000000000002,"5min":0.94799999999999995,"15min":1.0840000000000001},"last":0.74299999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.875,"5min":0.90700000000000003,"15min":1.137},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":0.96499999999999997,"15min":1.117},"last":0.44400000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47099999999999997},"max":{"1min":0.86899999999999999,"5min":0.96099999999999997,"15min":0.98199999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.49099999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.47999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306684,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94065156,"statfs":{"total":96636764160,"available":96322719744,"internally_reserved":0,"allocated":1429504,"data_stored":792622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.29499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.65800000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.24399999999999999},"max":{"1min":0.54500000000000004,"5min":0.65900000000000003,"15min":0.67100000000000004},"last":0.375}]},{"osd":1,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.32500000000000001,"5min":0.27800000000000002,"15min":0.27300000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.67200000000000004},"last":0.5},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.255},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.43099999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.45000000000000001},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64300000000000002,"5min":0.67700000000000005,"15min":0.71699999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.38900000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.72799999999999998},"last":0.41599999999999998}]},{"osd":4,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.502,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85899999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.55400000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.92200000000000004,"5min":0.996,"15min":1.042},"last":0.79400000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.53400000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83599999999999997,"5min":1.036,"15min":1.0469999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90000000000000002,"5min":1.107,"15min":1.107},"last":0.77700000000000002}]},{"osd":6,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.82899999999999996,"5min":1.0569999999999999,"15min":1.1100000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.57299999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84699999999999998,"5min":1.141,"15min":1.141},"last":0.88300000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.52400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":0.96199999999999997,"15min":1.054},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.88900000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.83199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1429504,"data_stored":792622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45100000000000001,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.76400000000000001,"5min":7.593,"15min":7.593},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48299999999999998,"15min":0.437},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56200000000000006,"5min":8.4239999999999995,"15min":8.4239999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.69099999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.221,"15min":0.221},"max":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.68799999999999994},"last":0.39800000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.496},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.66500000000000004,"5min":0.76300000000000001,"15min":0.77200000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.78800000000000003},"last":0.44}]},{"osd":4,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64500000000000002,"15min":0.63500000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.495,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":0.81599999999999995,"5min":1.976,"15min":1.976},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91500000000000004,"5min":1.53,"15min":1.53},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.97999999999999998,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.57699999999999996}]},{"osd":7,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72799999999999998,"15min":0.72099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.72999999999999998},"min":{"1min":0.495,"5min":0.49199999999999999,"15min":0.45800000000000002},"max":{"1min":0.93500000000000005,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.54600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302588,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1429504,"data_stored":792622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78300000000000003,"5min":0.80100000000000005,"15min":0.88600000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.84399999999999997},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.83099999999999996,"15min":0.85299999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.755,"5min":0.79800000000000004,"15min":1.506},"last":0.55200000000000005}]},{"osd":2,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.80200000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.78700000000000003}]},{"osd":3,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72899999999999998,"5min":0.77800000000000002,"15min":0.80600000000000005},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.81000000000000005,"15min":0.86299999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.92000000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.496,"15min":0.5},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65700000000000003,"5min":0.72899999999999998,"15min":1.556},"last":0.71299999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.753,"5min":0.78500000000000003,"15min":0.86099999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76400000000000001,"5min":0.82099999999999995,"15min":1.0409999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.38500000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.88100000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.307,"15min":0.307},"max":{"1min":0.72699999999999998,"5min":0.81399999999999995,"15min":1.26},"last":0.81499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575044,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1429504,"data_stored":792622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64400000000000002,"5min":0.85199999999999998,"15min":1.101},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.309,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71199999999999997,"5min":0.82699999999999996,"15min":1.1439999999999999},"last":0.436}]},{"osd":1,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56000000000000005,"15min":0.57699999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67200000000000004,"5min":0.84099999999999997,"15min":1.093},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69199999999999995,"5min":0.81299999999999994,"15min":1.1639999999999999},"last":0.38100000000000001}]},{"osd":2,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.68400000000000005,"5min":0.84299999999999997,"15min":1.1100000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.35499999999999998},"max":{"1min":0.67400000000000004,"5min":0.83599999999999997,"15min":1.1220000000000001},"last":0.45900000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.38500000000000001},"max":{"1min":0.72199999999999998,"5min":0.80500000000000005,"15min":1.139},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.496,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":1.0609999999999999},"last":0.496}]},{"osd":4,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70199999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.505,"15min":0.503},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.90000000000000002},"last":0.41999999999999998}]},{"osd":6,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.32900000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":1.1839999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77400000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60799999999999998}]},{"osd":7,"last update":"Wed Jan 24 14:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72999999999999998,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.33200000000000002},"max":{"1min":0.78200000000000003,"5min":0.95999999999999996,"15min":1.2030000000000001},"last":0.59599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:14:40.019 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:14:40.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.181+0000 7fbf3524e700 1 Processor -- start 2024-01-24T14:14:40.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.181+0000 7fbf3524e700 1 -- start start 2024-01-24T14:14:40.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.182+0000 7fbf3524e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf300c4a10 0x7fbf300bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.182+0000 7fbf3524e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf300670a0 0x7fbf30158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.182+0000 7fbf3524e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbf300664f0 0x7fbf300669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.182+0000 7fbf3524e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf30066e90 con 0x7fbf300c4a10 2024-01-24T14:14:40.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.182+0000 7fbf3524e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf301582b0 con 0x7fbf300664f0 2024-01-24T14:14:40.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.182+0000 7fbf3524e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf30157ca0 con 0x7fbf300670a0 2024-01-24T14:14:40.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.182+0000 7fbf2ed9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf300c4a10 0x7fbf300bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.182+0000 7fbf2ed9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf300c4a10 0x7fbf300bf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41440/0 (socket says 172.21.15.29:41440) 2024-01-24T14:14:40.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.182+0000 7fbf2ed9d700 1 -- 172.21.15.29:0/294086068 learned_addr learned my addr 172.21.15.29:0/294086068 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:40.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.182+0000 7fbf2e59c700 1 --2- 172.21.15.29:0/294086068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf300670a0 0x7fbf30158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.183+0000 7fbf2f59e700 1 --2- 172.21.15.29:0/294086068 >> v2:172.21.15.163:3300/0 conn(0x7fbf300664f0 0x7fbf300669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.183+0000 7fbf2ed9d700 1 -- 172.21.15.29:0/294086068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf300670a0 msgr2=0x7fbf30158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:40.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.183+0000 7fbf2ed9d700 1 --2- 172.21.15.29:0/294086068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf300670a0 0x7fbf30158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.183+0000 7fbf2ed9d700 1 -- 172.21.15.29:0/294086068 >> v2:172.21.15.163:3300/0 conn(0x7fbf300664f0 msgr2=0x7fbf300669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:40.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.183+0000 7fbf2ed9d700 1 --2- 172.21.15.29:0/294086068 >> v2:172.21.15.163:3300/0 conn(0x7fbf300664f0 0x7fbf300669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.183+0000 7fbf2ed9d700 1 -- 172.21.15.29:0/294086068 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf30063c60 con 0x7fbf300c4a10 2024-01-24T14:14:40.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.183+0000 7fbf2ed9d700 1 --2- 172.21.15.29:0/294086068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf300c4a10 0x7fbf300bf6d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7fbf30065a00 tx=0x7fbf180090f0).ready entity=mon.0 client_cookie=2d49eb5eead05837 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:40.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.183+0000 7fbf2dd9b700 1 -- 172.21.15.29:0/294086068 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800b600 con 0x7fbf300c4a10 2024-01-24T14:14:40.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.183+0000 7fbf2dd9b700 1 -- 172.21.15.29:0/294086068 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1800b760 con 0x7fbf300c4a10 2024-01-24T14:14:40.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.183+0000 7fbf2dd9b700 1 -- 172.21.15.29:0/294086068 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800bae0 con 0x7fbf300c4a10 2024-01-24T14:14:40.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.183+0000 7fbf3524e700 1 -- 172.21.15.29:0/294086068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf300c4a10 msgr2=0x7fbf300bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:40.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.183+0000 7fbf3524e700 1 --2- 172.21.15.29:0/294086068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf300c4a10 0x7fbf300bf6d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7fbf30065a00 tx=0x7fbf180090f0).stop 2024-01-24T14:14:40.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.184+0000 7fbf3524e700 1 -- 172.21.15.29:0/294086068 shutdown_connections 2024-01-24T14:14:40.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.184+0000 7fbf3524e700 1 --2- 172.21.15.29:0/294086068 >> v2:172.21.15.163:3300/0 conn(0x7fbf300664f0 0x7fbf300669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.184+0000 7fbf3524e700 1 --2- 172.21.15.29:0/294086068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf300c4a10 0x7fbf300bf6d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.184+0000 7fbf3524e700 1 --2- 172.21.15.29:0/294086068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf300670a0 0x7fbf30158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.184+0000 7fbf3524e700 1 -- 172.21.15.29:0/294086068 >> 172.21.15.29:0/294086068 conn(0x7fbf300c8bb0 msgr2=0x7fbf300c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:40.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.184+0000 7fbf3524e700 1 -- 172.21.15.29:0/294086068 shutdown_connections 2024-01-24T14:14:40.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.184+0000 7fbf3524e700 1 -- 172.21.15.29:0/294086068 wait complete. 2024-01-24T14:14:40.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.185+0000 7fbf3524e700 1 Processor -- start 2024-01-24T14:14:40.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.186+0000 7fbf3524e700 1 -- start start 2024-01-24T14:14:40.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.186+0000 7fbf3524e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf300664f0 0x7fbf30149af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.186+0000 7fbf3524e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf300670a0 0x7fbf301556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.186+0000 7fbf3524e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf300c4a10 0x7fbf30155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.186+0000 7fbf3524e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf30156e20 con 0x7fbf300664f0 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.186+0000 7fbf3524e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf30150300 con 0x7fbf300670a0 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.186+0000 7fbf3524e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf30156210 con 0x7fbf300c4a10 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.186+0000 7fbf2ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf300664f0 0x7fbf30149af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.186+0000 7fbf2ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf300664f0 0x7fbf30149af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41448/0 (socket says 172.21.15.29:41448) 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.186+0000 7fbf2ed9d700 1 -- 172.21.15.29:0/3405342007 learned_addr learned my addr 172.21.15.29:0/3405342007 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf2f59e700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf300c4a10 0x7fbf30155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf2e59c700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf300670a0 0x7fbf301556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf2ed9d700 1 -- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf300c4a10 msgr2=0x7fbf30155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf2ed9d700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf300c4a10 0x7fbf30155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf2ed9d700 1 -- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf300670a0 msgr2=0x7fbf301556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf2ed9d700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf300670a0 0x7fbf301556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf2ed9d700 1 -- 172.21.15.29:0/3405342007 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf18009370 con 0x7fbf300664f0 2024-01-24T14:14:40.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf2ed9d700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf300664f0 0x7fbf30149af0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7fbf300c69b0 tx=0x7fbf18017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:40.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf26ffd700 1 -- 172.21.15.29:0/3405342007 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800bea0 con 0x7fbf300664f0 2024-01-24T14:14:40.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf26ffd700 1 -- 172.21.15.29:0/3405342007 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1800b8d0 con 0x7fbf300664f0 2024-01-24T14:14:40.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf3524e700 1 -- 172.21.15.29:0/3405342007 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf301564f0 con 0x7fbf300664f0 2024-01-24T14:14:40.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf26ffd700 1 -- 172.21.15.29:0/3405342007 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1801c670 con 0x7fbf300664f0 2024-01-24T14:14:40.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.187+0000 7fbf3524e700 1 -- 172.21.15.29:0/3405342007 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf30156ab0 con 0x7fbf300664f0 2024-01-24T14:14:40.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.188+0000 7fbf3524e700 1 -- 172.21.15.29:0/3405342007 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf300c4df0 con 0x7fbf300664f0 2024-01-24T14:14:40.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.189+0000 7fbf26ffd700 1 -- 172.21.15.29:0/3405342007 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbf1801c9e0 con 0x7fbf300664f0 2024-01-24T14:14:40.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.189+0000 7fbf26ffd700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf1005f270 0x7fbf10061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.189+0000 7fbf26ffd700 1 -- 172.21.15.29:0/3405342007 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fbf1807c550 con 0x7fbf300664f0 2024-01-24T14:14:40.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.189+0000 7fbf2e59c700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf1005f270 0x7fbf10061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.190+0000 7fbf2e59c700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf1005f270 0x7fbf10061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fbf300c8380 tx=0x7fbf20008040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:40.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.193+0000 7fbf26ffd700 1 -- 172.21.15.29:0/3405342007 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbf18050cd0 con 0x7fbf300664f0 2024-01-24T14:14:40.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.368+0000 7fbf3524e700 1 -- 172.21.15.29:0/3405342007 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf30156f60 con 0x7fbf1005f270 2024-01-24T14:14:40.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.379+0000 7fbf26ffd700 1 -- 172.21.15.29:0/3405342007 <== mgr.6790 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361750 (secure 0 0 0) 0x7fbf2000b7a0 con 0x7fbf1005f270 2024-01-24T14:14:40.383 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:40.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.382+0000 7fbf3524e700 1 -- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf1005f270 msgr2=0x7fbf10061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.382+0000 7fbf3524e700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf1005f270 0x7fbf10061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fbf300c8380 tx=0x7fbf20008040).stop 2024-01-24T14:14:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.382+0000 7fbf3524e700 1 -- 172.21.15.29:0/3405342007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf300664f0 msgr2=0x7fbf30149af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.382+0000 7fbf3524e700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf300664f0 0x7fbf30149af0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7fbf300c69b0 tx=0x7fbf18017000).stop 2024-01-24T14:14:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.382+0000 7fbf3524e700 1 -- 172.21.15.29:0/3405342007 shutdown_connections 2024-01-24T14:14:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.382+0000 7fbf3524e700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf300c4a10 0x7fbf30155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.382+0000 7fbf3524e700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbf1005f270 0x7fbf10061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.382+0000 7fbf3524e700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf300664f0 0x7fbf30149af0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.382+0000 7fbf3524e700 1 --2- 172.21.15.29:0/3405342007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf300670a0 0x7fbf301556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.382+0000 7fbf3524e700 1 -- 172.21.15.29:0/3405342007 >> 172.21.15.29:0/3405342007 conn(0x7fbf300c8bb0 msgr2=0x7fbf30153650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.382+0000 7fbf3524e700 1 -- 172.21.15.29:0/3405342007 shutdown_connections 2024-01-24T14:14:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.382+0000 7fbf3524e700 1 -- 172.21.15.29:0/3405342007 wait complete. 2024-01-24T14:14:40.387 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:14:40.396 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-01-24T14:14:40.363966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":230,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425244,"kb_used_data":11612,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752549476,"statfs":{"total":773094113280,"available":770610663424,"internally_reserved":0,"allocated":11890688,"data_stored":6591563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5550,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.007771"},"pg_stats":[{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.903641+0000","last_change":"2024-01-24T14:14:35.898948+0000","last_active":"2024-01-24T14:14:35.903641+0000","last_peered":"2024-01-24T14:14:35.903641+0000","last_clean":"2024-01-24T14:14:35.903641+0000","last_became_active":"2024-01-24T14:14:35.898676+0000","last_became_peered":"2024-01-24T14:14:35.898676+0000","last_unstale":"2024-01-24T14:14:35.903641+0000","last_undegraded":"2024-01-24T14:14:35.903641+0000","last_fullsized":"2024-01-24T14:14:35.903641+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.3f","version":"119'3","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.904895+0000","last_change":"2024-01-24T14:14:35.477526+0000","last_active":"2024-01-24T14:14:35.904895+0000","last_peered":"2024-01-24T14:14:35.904895+0000","last_clean":"2024-01-24T14:14:35.904895+0000","last_became_active":"2024-01-24T14:14:35.477220+0000","last_became_peered":"2024-01-24T14:14:35.477220+0000","last_unstale":"2024-01-24T14:14:35.904895+0000","last_undegraded":"2024-01-24T14:14:35.904895+0000","last_fullsized":"2024-01-24T14:14:35.904895+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.902693+0000","last_change":"2024-01-24T14:14:35.890268+0000","last_active":"2024-01-24T14:14:35.902693+0000","last_peered":"2024-01-24T14:14:35.902693+0000","last_clean":"2024-01-24T14:14:35.902693+0000","last_became_active":"2024-01-24T14:14:35.889983+0000","last_became_peered":"2024-01-24T14:14:35.889983+0000","last_unstale":"2024-01-24T14:14:35.902693+0000","last_undegraded":"2024-01-24T14:14:35.902693+0000","last_fullsized":"2024-01-24T14:14:35.902693+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.875812+0000","last_change":"2024-01-24T14:14:35.460481+0000","last_active":"2024-01-24T14:14:35.875812+0000","last_peered":"2024-01-24T14:14:35.875812+0000","last_clean":"2024-01-24T14:14:35.875812+0000","last_became_active":"2024-01-24T14:14:35.460166+0000","last_became_peered":"2024-01-24T14:14:35.460166+0000","last_unstale":"2024-01-24T14:14:35.875812+0000","last_undegraded":"2024-01-24T14:14:35.875812+0000","last_fullsized":"2024-01-24T14:14:35.875812+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.909578+0000","last_change":"2024-01-24T14:14:35.904285+0000","last_active":"2024-01-24T14:14:35.909578+0000","last_peered":"2024-01-24T14:14:35.909578+0000","last_clean":"2024-01-24T14:14:35.909578+0000","last_became_active":"2024-01-24T14:14:35.904007+0000","last_became_peered":"2024-01-24T14:14:35.904007+0000","last_unstale":"2024-01-24T14:14:35.909578+0000","last_undegraded":"2024-01-24T14:14:35.909578+0000","last_fullsized":"2024-01-24T14:14:35.909578+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.878492+0000","last_change":"2024-01-24T14:14:35.458430+0000","last_active":"2024-01-24T14:14:35.878492+0000","last_peered":"2024-01-24T14:14:35.878492+0000","last_clean":"2024-01-24T14:14:35.878492+0000","last_became_active":"2024-01-24T14:14:35.458153+0000","last_became_peered":"2024-01-24T14:14:35.458153+0000","last_unstale":"2024-01-24T14:14:35.878492+0000","last_undegraded":"2024-01-24T14:14:35.878492+0000","last_fullsized":"2024-01-24T14:14:35.878492+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.905144+0000","last_change":"2024-01-24T14:14:35.889398+0000","last_active":"2024-01-24T14:14:35.905144+0000","last_peered":"2024-01-24T14:14:35.905144+0000","last_clean":"2024-01-24T14:14:35.905144+0000","last_became_active":"2024-01-24T14:14:35.889068+0000","last_became_peered":"2024-01-24T14:14:35.889068+0000","last_unstale":"2024-01-24T14:14:35.905144+0000","last_undegraded":"2024-01-24T14:14:35.905144+0000","last_fullsized":"2024-01-24T14:14:35.905144+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3c","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.903105+0000","last_change":"2024-01-24T14:14:35.476630+0000","last_active":"2024-01-24T14:14:35.903105+0000","last_peered":"2024-01-24T14:14:35.903105+0000","last_clean":"2024-01-24T14:14:35.903105+0000","last_became_active":"2024-01-24T14:14:35.476281+0000","last_became_peered":"2024-01-24T14:14:35.476281+0000","last_unstale":"2024-01-24T14:14:35.903105+0000","last_undegraded":"2024-01-24T14:14:35.903105+0000","last_fullsized":"2024-01-24T14:14:35.903105+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.899804+0000","last_change":"2024-01-24T14:14:35.888470+0000","last_active":"2024-01-24T14:14:35.899804+0000","last_peered":"2024-01-24T14:14:35.899804+0000","last_clean":"2024-01-24T14:14:35.899804+0000","last_became_active":"2024-01-24T14:14:35.888134+0000","last_became_peered":"2024-01-24T14:14:35.888134+0000","last_unstale":"2024-01-24T14:14:35.899804+0000","last_undegraded":"2024-01-24T14:14:35.899804+0000","last_fullsized":"2024-01-24T14:14:35.899804+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.895782+0000","last_change":"2024-01-24T14:14:35.461527+0000","last_active":"2024-01-24T14:14:35.895782+0000","last_peered":"2024-01-24T14:14:35.895782+0000","last_clean":"2024-01-24T14:14:35.895782+0000","last_became_active":"2024-01-24T14:14:35.460515+0000","last_became_peered":"2024-01-24T14:14:35.460515+0000","last_unstale":"2024-01-24T14:14:35.895782+0000","last_undegraded":"2024-01-24T14:14:35.895782+0000","last_fullsized":"2024-01-24T14:14:35.895782+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.899690+0000","last_change":"2024-01-24T14:14:35.897090+0000","last_active":"2024-01-24T14:14:35.899690+0000","last_peered":"2024-01-24T14:14:35.899690+0000","last_clean":"2024-01-24T14:14:35.899690+0000","last_became_active":"2024-01-24T14:14:35.896824+0000","last_became_peered":"2024-01-24T14:14:35.896824+0000","last_unstale":"2024-01-24T14:14:35.899690+0000","last_undegraded":"2024-01-24T14:14:35.899690+0000","last_fullsized":"2024-01-24T14:14:35.899690+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3a","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.903937+0000","last_change":"2024-01-24T14:14:35.473282+0000","last_active":"2024-01-24T14:14:35.903937+0000","last_peered":"2024-01-24T14:14:35.903937+0000","last_clean":"2024-01-24T14:14:35.903937+0000","last_became_active":"2024-01-24T14:14:35.472951+0000","last_became_peered":"2024-01-24T14:14:35.472951+0000","last_unstale":"2024-01-24T14:14:35.903937+0000","last_undegraded":"2024-01-24T14:14:35.903937+0000","last_fullsized":"2024-01-24T14:14:35.903937+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.906805+0000","last_change":"2024-01-24T14:14:35.895723+0000","last_active":"2024-01-24T14:14:35.906805+0000","last_peered":"2024-01-24T14:14:35.906805+0000","last_clean":"2024-01-24T14:14:35.906805+0000","last_became_active":"2024-01-24T14:14:35.894927+0000","last_became_peered":"2024-01-24T14:14:35.894927+0000","last_unstale":"2024-01-24T14:14:35.906805+0000","last_undegraded":"2024-01-24T14:14:35.906805+0000","last_fullsized":"2024-01-24T14:14:35.906805+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.879370+0000","last_change":"2024-01-24T14:14:35.472743+0000","last_active":"2024-01-24T14:14:35.879370+0000","last_peered":"2024-01-24T14:14:35.879370+0000","last_clean":"2024-01-24T14:14:35.879370+0000","last_became_active":"2024-01-24T14:14:35.472469+0000","last_became_peered":"2024-01-24T14:14:35.472469+0000","last_unstale":"2024-01-24T14:14:35.879370+0000","last_undegraded":"2024-01-24T14:14:35.879370+0000","last_fullsized":"2024-01-24T14:14:35.879370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.899777+0000","last_change":"2024-01-24T14:14:35.887913+0000","last_active":"2024-01-24T14:14:35.899777+0000","last_peered":"2024-01-24T14:14:35.899777+0000","last_clean":"2024-01-24T14:14:35.899777+0000","last_became_active":"2024-01-24T14:14:35.887610+0000","last_became_peered":"2024-01-24T14:14:35.887610+0000","last_unstale":"2024-01-24T14:14:35.899777+0000","last_undegraded":"2024-01-24T14:14:35.899777+0000","last_fullsized":"2024-01-24T14:14:35.899777+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.882772+0000","last_change":"2024-01-24T14:14:35.469403+0000","last_active":"2024-01-24T14:14:35.882772+0000","last_peered":"2024-01-24T14:14:35.882772+0000","last_clean":"2024-01-24T14:14:35.882772+0000","last_became_active":"2024-01-24T14:14:35.468941+0000","last_became_peered":"2024-01-24T14:14:35.468941+0000","last_unstale":"2024-01-24T14:14:35.882772+0000","last_undegraded":"2024-01-24T14:14:35.882772+0000","last_fullsized":"2024-01-24T14:14:35.882772+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.898573+0000","last_change":"2024-01-24T14:14:35.890265+0000","last_active":"2024-01-24T14:14:35.898573+0000","last_peered":"2024-01-24T14:14:35.898573+0000","last_clean":"2024-01-24T14:14:35.898573+0000","last_became_active":"2024-01-24T14:14:35.886849+0000","last_became_peered":"2024-01-24T14:14:35.886849+0000","last_unstale":"2024-01-24T14:14:35.898573+0000","last_undegraded":"2024-01-24T14:14:35.898573+0000","last_fullsized":"2024-01-24T14:14:35.898573+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.881837+0000","last_change":"2024-01-24T14:14:35.458117+0000","last_active":"2024-01-24T14:14:35.881837+0000","last_peered":"2024-01-24T14:14:35.881837+0000","last_clean":"2024-01-24T14:14:35.881837+0000","last_became_active":"2024-01-24T14:14:35.457769+0000","last_became_peered":"2024-01-24T14:14:35.457769+0000","last_unstale":"2024-01-24T14:14:35.881837+0000","last_undegraded":"2024-01-24T14:14:35.881837+0000","last_fullsized":"2024-01-24T14:14:35.881837+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.904644+0000","last_change":"2024-01-24T14:14:35.898082+0000","last_active":"2024-01-24T14:14:35.904644+0000","last_peered":"2024-01-24T14:14:35.904644+0000","last_clean":"2024-01-24T14:14:35.904644+0000","last_became_active":"2024-01-24T14:14:35.897787+0000","last_became_peered":"2024-01-24T14:14:35.897787+0000","last_unstale":"2024-01-24T14:14:35.904644+0000","last_undegraded":"2024-01-24T14:14:35.904644+0000","last_fullsized":"2024-01-24T14:14:35.904644+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.876755+0000","last_change":"2024-01-24T14:14:35.476488+0000","last_active":"2024-01-24T14:14:35.876755+0000","last_peered":"2024-01-24T14:14:35.876755+0000","last_clean":"2024-01-24T14:14:35.876755+0000","last_became_active":"2024-01-24T14:14:35.476146+0000","last_became_peered":"2024-01-24T14:14:35.476146+0000","last_unstale":"2024-01-24T14:14:35.876755+0000","last_undegraded":"2024-01-24T14:14:35.876755+0000","last_fullsized":"2024-01-24T14:14:35.876755+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.907177+0000","last_change":"2024-01-24T14:14:35.904177+0000","last_active":"2024-01-24T14:14:35.907177+0000","last_peered":"2024-01-24T14:14:35.907177+0000","last_clean":"2024-01-24T14:14:35.907177+0000","last_became_active":"2024-01-24T14:14:35.903890+0000","last_became_peered":"2024-01-24T14:14:35.903890+0000","last_unstale":"2024-01-24T14:14:35.907177+0000","last_undegraded":"2024-01-24T14:14:35.907177+0000","last_fullsized":"2024-01-24T14:14:35.907177+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.881863+0000","last_change":"2024-01-24T14:14:35.475207+0000","last_active":"2024-01-24T14:14:35.881863+0000","last_peered":"2024-01-24T14:14:35.881863+0000","last_clean":"2024-01-24T14:14:35.881863+0000","last_became_active":"2024-01-24T14:14:35.474986+0000","last_became_peered":"2024-01-24T14:14:35.474986+0000","last_unstale":"2024-01-24T14:14:35.881863+0000","last_undegraded":"2024-01-24T14:14:35.881863+0000","last_fullsized":"2024-01-24T14:14:35.881863+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901823+0000","last_change":"2024-01-24T14:14:35.894283+0000","last_active":"2024-01-24T14:14:35.901823+0000","last_peered":"2024-01-24T14:14:35.901823+0000","last_clean":"2024-01-24T14:14:35.901823+0000","last_became_active":"2024-01-24T14:14:35.893894+0000","last_became_peered":"2024-01-24T14:14:35.893894+0000","last_unstale":"2024-01-24T14:14:35.901823+0000","last_undegraded":"2024-01-24T14:14:35.901823+0000","last_fullsized":"2024-01-24T14:14:35.901823+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.34","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.911374+0000","last_change":"2024-01-24T14:14:35.463091+0000","last_active":"2024-01-24T14:14:35.911374+0000","last_peered":"2024-01-24T14:14:35.911374+0000","last_clean":"2024-01-24T14:14:35.911374+0000","last_became_active":"2024-01-24T14:14:35.462809+0000","last_became_peered":"2024-01-24T14:14:35.462809+0000","last_unstale":"2024-01-24T14:14:35.911374+0000","last_undegraded":"2024-01-24T14:14:35.911374+0000","last_fullsized":"2024-01-24T14:14:35.911374+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.893857+0000","last_change":"2024-01-24T14:14:35.880132+0000","last_active":"2024-01-24T14:14:35.893857+0000","last_peered":"2024-01-24T14:14:35.893857+0000","last_clean":"2024-01-24T14:14:35.893857+0000","last_became_active":"2024-01-24T14:14:35.879845+0000","last_became_peered":"2024-01-24T14:14:35.879845+0000","last_unstale":"2024-01-24T14:14:35.893857+0000","last_undegraded":"2024-01-24T14:14:35.893857+0000","last_fullsized":"2024-01-24T14:14:35.893857+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.902834+0000","last_change":"2024-01-24T14:14:35.471904+0000","last_active":"2024-01-24T14:14:35.902834+0000","last_peered":"2024-01-24T14:14:35.902834+0000","last_clean":"2024-01-24T14:14:35.902834+0000","last_became_active":"2024-01-24T14:14:35.471611+0000","last_became_peered":"2024-01-24T14:14:35.471611+0000","last_unstale":"2024-01-24T14:14:35.902834+0000","last_undegraded":"2024-01-24T14:14:35.902834+0000","last_fullsized":"2024-01-24T14:14:35.902834+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.909364+0000","last_change":"2024-01-24T14:14:35.898736+0000","last_active":"2024-01-24T14:14:35.909364+0000","last_peered":"2024-01-24T14:14:35.909364+0000","last_clean":"2024-01-24T14:14:35.909364+0000","last_became_active":"2024-01-24T14:14:35.898484+0000","last_became_peered":"2024-01-24T14:14:35.898484+0000","last_unstale":"2024-01-24T14:14:35.909364+0000","last_undegraded":"2024-01-24T14:14:35.909364+0000","last_fullsized":"2024-01-24T14:14:35.909364+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.883040+0000","last_change":"2024-01-24T14:14:35.469846+0000","last_active":"2024-01-24T14:14:35.883040+0000","last_peered":"2024-01-24T14:14:35.883040+0000","last_clean":"2024-01-24T14:14:35.883040+0000","last_became_active":"2024-01-24T14:14:35.469555+0000","last_became_peered":"2024-01-24T14:14:35.469555+0000","last_unstale":"2024-01-24T14:14:35.883040+0000","last_undegraded":"2024-01-24T14:14:35.883040+0000","last_fullsized":"2024-01-24T14:14:35.883040+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.902575+0000","last_change":"2024-01-24T14:14:35.898362+0000","last_active":"2024-01-24T14:14:35.902575+0000","last_peered":"2024-01-24T14:14:35.902575+0000","last_clean":"2024-01-24T14:14:35.902575+0000","last_became_active":"2024-01-24T14:14:35.897938+0000","last_became_peered":"2024-01-24T14:14:35.897938+0000","last_unstale":"2024-01-24T14:14:35.902575+0000","last_undegraded":"2024-01-24T14:14:35.902575+0000","last_fullsized":"2024-01-24T14:14:35.902575+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.881978+0000","last_change":"2024-01-24T14:14:35.472729+0000","last_active":"2024-01-24T14:14:35.881978+0000","last_peered":"2024-01-24T14:14:35.881978+0000","last_clean":"2024-01-24T14:14:35.881978+0000","last_became_active":"2024-01-24T14:14:35.472412+0000","last_became_peered":"2024-01-24T14:14:35.472412+0000","last_unstale":"2024-01-24T14:14:35.881978+0000","last_undegraded":"2024-01-24T14:14:35.881978+0000","last_fullsized":"2024-01-24T14:14:35.881978+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.909591+0000","last_change":"2024-01-24T14:14:35.903104+0000","last_active":"2024-01-24T14:14:35.909591+0000","last_peered":"2024-01-24T14:14:35.909591+0000","last_clean":"2024-01-24T14:14:35.909591+0000","last_became_active":"2024-01-24T14:14:35.902815+0000","last_became_peered":"2024-01-24T14:14:35.902815+0000","last_unstale":"2024-01-24T14:14:35.909591+0000","last_undegraded":"2024-01-24T14:14:35.909591+0000","last_fullsized":"2024-01-24T14:14:35.909591+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.30","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.910688+0000","last_change":"2024-01-24T14:14:35.477526+0000","last_active":"2024-01-24T14:14:35.910688+0000","last_peered":"2024-01-24T14:14:35.910688+0000","last_clean":"2024-01-24T14:14:35.910688+0000","last_became_active":"2024-01-24T14:14:35.476727+0000","last_became_peered":"2024-01-24T14:14:35.476727+0000","last_unstale":"2024-01-24T14:14:35.910688+0000","last_undegraded":"2024-01-24T14:14:35.910688+0000","last_fullsized":"2024-01-24T14:14:35.910688+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.909506+0000","last_change":"2024-01-24T14:14:35.902769+0000","last_active":"2024-01-24T14:14:35.909506+0000","last_peered":"2024-01-24T14:14:35.909506+0000","last_clean":"2024-01-24T14:14:35.909506+0000","last_became_active":"2024-01-24T14:14:35.902473+0000","last_became_peered":"2024-01-24T14:14:35.902473+0000","last_unstale":"2024-01-24T14:14:35.909506+0000","last_undegraded":"2024-01-24T14:14:35.909506+0000","last_fullsized":"2024-01-24T14:14:35.909506+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.888726+0000","last_change":"2024-01-24T14:14:35.479068+0000","last_active":"2024-01-24T14:14:35.888726+0000","last_peered":"2024-01-24T14:14:35.888726+0000","last_clean":"2024-01-24T14:14:35.888726+0000","last_became_active":"2024-01-24T14:14:35.478709+0000","last_became_peered":"2024-01-24T14:14:35.478709+0000","last_unstale":"2024-01-24T14:14:35.888726+0000","last_undegraded":"2024-01-24T14:14:35.888726+0000","last_fullsized":"2024-01-24T14:14:35.888726+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.905744+0000","last_change":"2024-01-24T14:14:35.894412+0000","last_active":"2024-01-24T14:14:35.905744+0000","last_peered":"2024-01-24T14:14:35.905744+0000","last_clean":"2024-01-24T14:14:35.905744+0000","last_became_active":"2024-01-24T14:14:35.894120+0000","last_became_peered":"2024-01-24T14:14:35.894120+0000","last_unstale":"2024-01-24T14:14:35.905744+0000","last_undegraded":"2024-01-24T14:14:35.905744+0000","last_fullsized":"2024-01-24T14:14:35.905744+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.878888+0000","last_change":"2024-01-24T14:14:35.457790+0000","last_active":"2024-01-24T14:14:35.878888+0000","last_peered":"2024-01-24T14:14:35.878888+0000","last_clean":"2024-01-24T14:14:35.878888+0000","last_became_active":"2024-01-24T14:14:35.457477+0000","last_became_peered":"2024-01-24T14:14:35.457477+0000","last_unstale":"2024-01-24T14:14:35.878888+0000","last_undegraded":"2024-01-24T14:14:35.878888+0000","last_fullsized":"2024-01-24T14:14:35.878888+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.904759+0000","last_change":"2024-01-24T14:14:35.898468+0000","last_active":"2024-01-24T14:14:35.904759+0000","last_peered":"2024-01-24T14:14:35.904759+0000","last_clean":"2024-01-24T14:14:35.904759+0000","last_became_active":"2024-01-24T14:14:35.898104+0000","last_became_peered":"2024-01-24T14:14:35.898104+0000","last_unstale":"2024-01-24T14:14:35.904759+0000","last_undegraded":"2024-01-24T14:14:35.904759+0000","last_fullsized":"2024-01-24T14:14:35.904759+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2d","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.905570+0000","last_change":"2024-01-24T14:14:35.474972+0000","last_active":"2024-01-24T14:14:35.905570+0000","last_peered":"2024-01-24T14:14:35.905570+0000","last_clean":"2024-01-24T14:14:35.905570+0000","last_became_active":"2024-01-24T14:14:35.474787+0000","last_became_peered":"2024-01-24T14:14:35.474787+0000","last_unstale":"2024-01-24T14:14:35.905570+0000","last_undegraded":"2024-01-24T14:14:35.905570+0000","last_fullsized":"2024-01-24T14:14:35.905570+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901604+0000","last_change":"2024-01-24T14:14:35.897263+0000","last_active":"2024-01-24T14:14:35.901604+0000","last_peered":"2024-01-24T14:14:35.901604+0000","last_clean":"2024-01-24T14:14:35.901604+0000","last_became_active":"2024-01-24T14:14:35.896947+0000","last_became_peered":"2024-01-24T14:14:35.896947+0000","last_unstale":"2024-01-24T14:14:35.901604+0000","last_undegraded":"2024-01-24T14:14:35.901604+0000","last_fullsized":"2024-01-24T14:14:35.901604+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.884523+0000","last_change":"2024-01-24T14:14:35.469350+0000","last_active":"2024-01-24T14:14:35.884523+0000","last_peered":"2024-01-24T14:14:35.884523+0000","last_clean":"2024-01-24T14:14:35.884523+0000","last_became_active":"2024-01-24T14:14:35.468548+0000","last_became_peered":"2024-01-24T14:14:35.468548+0000","last_unstale":"2024-01-24T14:14:35.884523+0000","last_undegraded":"2024-01-24T14:14:35.884523+0000","last_fullsized":"2024-01-24T14:14:35.884523+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.903193+0000","last_change":"2024-01-24T14:14:35.898954+0000","last_active":"2024-01-24T14:14:35.903193+0000","last_peered":"2024-01-24T14:14:35.903193+0000","last_clean":"2024-01-24T14:14:35.903193+0000","last_became_active":"2024-01-24T14:14:35.898624+0000","last_became_peered":"2024-01-24T14:14:35.898624+0000","last_unstale":"2024-01-24T14:14:35.903193+0000","last_undegraded":"2024-01-24T14:14:35.903193+0000","last_fullsized":"2024-01-24T14:14:35.903193+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.900030+0000","last_change":"2024-01-24T14:14:35.458517+0000","last_active":"2024-01-24T14:14:35.900030+0000","last_peered":"2024-01-24T14:14:35.900030+0000","last_clean":"2024-01-24T14:14:35.900030+0000","last_became_active":"2024-01-24T14:14:35.457982+0000","last_became_peered":"2024-01-24T14:14:35.457982+0000","last_unstale":"2024-01-24T14:14:35.900030+0000","last_undegraded":"2024-01-24T14:14:35.900030+0000","last_fullsized":"2024-01-24T14:14:35.900030+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901497+0000","last_change":"2024-01-24T14:14:35.893829+0000","last_active":"2024-01-24T14:14:35.901497+0000","last_peered":"2024-01-24T14:14:35.901497+0000","last_clean":"2024-01-24T14:14:35.901497+0000","last_became_active":"2024-01-24T14:14:35.884029+0000","last_became_peered":"2024-01-24T14:14:35.884029+0000","last_unstale":"2024-01-24T14:14:35.901497+0000","last_undegraded":"2024-01-24T14:14:35.901497+0000","last_fullsized":"2024-01-24T14:14:35.901497+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.883145+0000","last_change":"2024-01-24T14:14:35.469777+0000","last_active":"2024-01-24T14:14:35.883145+0000","last_peered":"2024-01-24T14:14:35.883145+0000","last_clean":"2024-01-24T14:14:35.883145+0000","last_became_active":"2024-01-24T14:14:35.469490+0000","last_became_peered":"2024-01-24T14:14:35.469490+0000","last_unstale":"2024-01-24T14:14:35.883145+0000","last_undegraded":"2024-01-24T14:14:35.883145+0000","last_fullsized":"2024-01-24T14:14:35.883145+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.910006+0000","last_change":"2024-01-24T14:14:35.905300+0000","last_active":"2024-01-24T14:14:35.910006+0000","last_peered":"2024-01-24T14:14:35.910006+0000","last_clean":"2024-01-24T14:14:35.910006+0000","last_became_active":"2024-01-24T14:14:35.904975+0000","last_became_peered":"2024-01-24T14:14:35.904975+0000","last_unstale":"2024-01-24T14:14:35.910006+0000","last_undegraded":"2024-01-24T14:14:35.910006+0000","last_fullsized":"2024-01-24T14:14:35.910006+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.878142+0000","last_change":"2024-01-24T14:14:35.476909+0000","last_active":"2024-01-24T14:14:35.878142+0000","last_peered":"2024-01-24T14:14:35.878142+0000","last_clean":"2024-01-24T14:14:35.878142+0000","last_became_active":"2024-01-24T14:14:35.476548+0000","last_became_peered":"2024-01-24T14:14:35.476548+0000","last_unstale":"2024-01-24T14:14:35.878142+0000","last_undegraded":"2024-01-24T14:14:35.878142+0000","last_fullsized":"2024-01-24T14:14:35.878142+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.903234+0000","last_change":"2024-01-24T14:14:35.897210+0000","last_active":"2024-01-24T14:14:35.903234+0000","last_peered":"2024-01-24T14:14:35.903234+0000","last_clean":"2024-01-24T14:14:35.903234+0000","last_became_active":"2024-01-24T14:14:35.896913+0000","last_became_peered":"2024-01-24T14:14:35.896913+0000","last_unstale":"2024-01-24T14:14:35.903234+0000","last_undegraded":"2024-01-24T14:14:35.903234+0000","last_fullsized":"2024-01-24T14:14:35.903234+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.882143+0000","last_change":"2024-01-24T14:14:35.457853+0000","last_active":"2024-01-24T14:14:35.882143+0000","last_peered":"2024-01-24T14:14:35.882143+0000","last_clean":"2024-01-24T14:14:35.882143+0000","last_became_active":"2024-01-24T14:14:35.457527+0000","last_became_peered":"2024-01-24T14:14:35.457527+0000","last_unstale":"2024-01-24T14:14:35.882143+0000","last_undegraded":"2024-01-24T14:14:35.882143+0000","last_fullsized":"2024-01-24T14:14:35.882143+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.906003+0000","last_change":"2024-01-24T14:14:35.903514+0000","last_active":"2024-01-24T14:14:35.906003+0000","last_peered":"2024-01-24T14:14:35.906003+0000","last_clean":"2024-01-24T14:14:35.906003+0000","last_became_active":"2024-01-24T14:14:35.903243+0000","last_became_peered":"2024-01-24T14:14:35.903243+0000","last_unstale":"2024-01-24T14:14:35.906003+0000","last_undegraded":"2024-01-24T14:14:35.906003+0000","last_fullsized":"2024-01-24T14:14:35.906003+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.876025+0000","last_change":"2024-01-24T14:14:35.459962+0000","last_active":"2024-01-24T14:14:35.876025+0000","last_peered":"2024-01-24T14:14:35.876025+0000","last_clean":"2024-01-24T14:14:35.876025+0000","last_became_active":"2024-01-24T14:14:35.459676+0000","last_became_peered":"2024-01-24T14:14:35.459676+0000","last_unstale":"2024-01-24T14:14:35.876025+0000","last_undegraded":"2024-01-24T14:14:35.876025+0000","last_fullsized":"2024-01-24T14:14:35.876025+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.898767+0000","last_change":"2024-01-24T14:14:35.889832+0000","last_active":"2024-01-24T14:14:35.898767+0000","last_peered":"2024-01-24T14:14:35.898767+0000","last_clean":"2024-01-24T14:14:35.898767+0000","last_became_active":"2024-01-24T14:14:35.889489+0000","last_became_peered":"2024-01-24T14:14:35.889489+0000","last_unstale":"2024-01-24T14:14:35.898767+0000","last_undegraded":"2024-01-24T14:14:35.898767+0000","last_fullsized":"2024-01-24T14:14:35.898767+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.880858+0000","last_change":"2024-01-24T14:14:35.472866+0000","last_active":"2024-01-24T14:14:35.880858+0000","last_peered":"2024-01-24T14:14:35.880858+0000","last_clean":"2024-01-24T14:14:35.880858+0000","last_became_active":"2024-01-24T14:14:35.472601+0000","last_became_peered":"2024-01-24T14:14:35.472601+0000","last_unstale":"2024-01-24T14:14:35.880858+0000","last_undegraded":"2024-01-24T14:14:35.880858+0000","last_fullsized":"2024-01-24T14:14:35.880858+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.898605+0000","last_change":"2024-01-24T14:14:35.891125+0000","last_active":"2024-01-24T14:14:35.898605+0000","last_peered":"2024-01-24T14:14:35.898605+0000","last_clean":"2024-01-24T14:14:35.898605+0000","last_became_active":"2024-01-24T14:14:35.890828+0000","last_became_peered":"2024-01-24T14:14:35.890828+0000","last_unstale":"2024-01-24T14:14:35.898605+0000","last_undegraded":"2024-01-24T14:14:35.898605+0000","last_fullsized":"2024-01-24T14:14:35.898605+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.876039+0000","last_change":"2024-01-24T14:14:35.462737+0000","last_active":"2024-01-24T14:14:35.876039+0000","last_peered":"2024-01-24T14:14:35.876039+0000","last_clean":"2024-01-24T14:14:35.876039+0000","last_became_active":"2024-01-24T14:14:35.462363+0000","last_became_peered":"2024-01-24T14:14:35.462363+0000","last_unstale":"2024-01-24T14:14:35.876039+0000","last_undegraded":"2024-01-24T14:14:35.876039+0000","last_fullsized":"2024-01-24T14:14:35.876039+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.902445+0000","last_change":"2024-01-24T14:14:35.897362+0000","last_active":"2024-01-24T14:14:35.902445+0000","last_peered":"2024-01-24T14:14:35.902445+0000","last_clean":"2024-01-24T14:14:35.902445+0000","last_became_active":"2024-01-24T14:14:35.897060+0000","last_became_peered":"2024-01-24T14:14:35.897060+0000","last_unstale":"2024-01-24T14:14:35.902445+0000","last_undegraded":"2024-01-24T14:14:35.902445+0000","last_fullsized":"2024-01-24T14:14:35.902445+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.888512+0000","last_change":"2024-01-24T14:14:35.470100+0000","last_active":"2024-01-24T14:14:35.888512+0000","last_peered":"2024-01-24T14:14:35.888512+0000","last_clean":"2024-01-24T14:14:35.888512+0000","last_became_active":"2024-01-24T14:14:35.469660+0000","last_became_peered":"2024-01-24T14:14:35.469660+0000","last_unstale":"2024-01-24T14:14:35.888512+0000","last_undegraded":"2024-01-24T14:14:35.888512+0000","last_fullsized":"2024-01-24T14:14:35.888512+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901650+0000","last_change":"2024-01-24T14:14:35.897256+0000","last_active":"2024-01-24T14:14:35.901650+0000","last_peered":"2024-01-24T14:14:35.901650+0000","last_clean":"2024-01-24T14:14:35.901650+0000","last_became_active":"2024-01-24T14:14:35.897067+0000","last_became_peered":"2024-01-24T14:14:35.897067+0000","last_unstale":"2024-01-24T14:14:35.901650+0000","last_undegraded":"2024-01-24T14:14:35.901650+0000","last_fullsized":"2024-01-24T14:14:35.901650+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.879773+0000","last_change":"2024-01-24T14:14:35.465153+0000","last_active":"2024-01-24T14:14:35.879773+0000","last_peered":"2024-01-24T14:14:35.879773+0000","last_clean":"2024-01-24T14:14:35.879773+0000","last_became_active":"2024-01-24T14:14:35.464888+0000","last_became_peered":"2024-01-24T14:14:35.464888+0000","last_unstale":"2024-01-24T14:14:35.879773+0000","last_undegraded":"2024-01-24T14:14:35.879773+0000","last_fullsized":"2024-01-24T14:14:35.879773+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.907206+0000","last_change":"2024-01-24T14:14:35.905129+0000","last_active":"2024-01-24T14:14:35.907206+0000","last_peered":"2024-01-24T14:14:35.907206+0000","last_clean":"2024-01-24T14:14:35.907206+0000","last_became_active":"2024-01-24T14:14:35.904831+0000","last_became_peered":"2024-01-24T14:14:35.904831+0000","last_unstale":"2024-01-24T14:14:35.907206+0000","last_undegraded":"2024-01-24T14:14:35.907206+0000","last_fullsized":"2024-01-24T14:14:35.907206+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.883256+0000","last_change":"2024-01-24T14:14:35.458390+0000","last_active":"2024-01-24T14:14:35.883256+0000","last_peered":"2024-01-24T14:14:35.883256+0000","last_clean":"2024-01-24T14:14:35.883256+0000","last_became_active":"2024-01-24T14:14:35.457833+0000","last_became_peered":"2024-01-24T14:14:35.457833+0000","last_unstale":"2024-01-24T14:14:35.883256+0000","last_undegraded":"2024-01-24T14:14:35.883256+0000","last_fullsized":"2024-01-24T14:14:35.883256+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.906183+0000","last_change":"2024-01-24T14:14:35.902258+0000","last_active":"2024-01-24T14:14:35.906183+0000","last_peered":"2024-01-24T14:14:35.906183+0000","last_clean":"2024-01-24T14:14:35.906183+0000","last_became_active":"2024-01-24T14:14:35.901947+0000","last_became_peered":"2024-01-24T14:14:35.901947+0000","last_unstale":"2024-01-24T14:14:35.906183+0000","last_undegraded":"2024-01-24T14:14:35.906183+0000","last_fullsized":"2024-01-24T14:14:35.906183+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.875271+0000","last_change":"2024-01-24T14:14:35.476641+0000","last_active":"2024-01-24T14:14:35.875271+0000","last_peered":"2024-01-24T14:14:35.875271+0000","last_clean":"2024-01-24T14:14:35.875271+0000","last_became_active":"2024-01-24T14:14:35.476286+0000","last_became_peered":"2024-01-24T14:14:35.476286+0000","last_unstale":"2024-01-24T14:14:35.875271+0000","last_undegraded":"2024-01-24T14:14:35.875271+0000","last_fullsized":"2024-01-24T14:14:35.875271+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.909493+0000","last_change":"2024-01-24T14:14:35.900663+0000","last_active":"2024-01-24T14:14:35.909493+0000","last_peered":"2024-01-24T14:14:35.909493+0000","last_clean":"2024-01-24T14:14:35.909493+0000","last_became_active":"2024-01-24T14:14:35.900369+0000","last_became_peered":"2024-01-24T14:14:35.900369+0000","last_unstale":"2024-01-24T14:14:35.909493+0000","last_undegraded":"2024-01-24T14:14:35.909493+0000","last_fullsized":"2024-01-24T14:14:35.909493+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.884908+0000","last_change":"2024-01-24T14:14:35.477702+0000","last_active":"2024-01-24T14:14:35.884908+0000","last_peered":"2024-01-24T14:14:35.884908+0000","last_clean":"2024-01-24T14:14:35.884908+0000","last_became_active":"2024-01-24T14:14:35.477048+0000","last_became_peered":"2024-01-24T14:14:35.477048+0000","last_unstale":"2024-01-24T14:14:35.884908+0000","last_undegraded":"2024-01-24T14:14:35.884908+0000","last_fullsized":"2024-01-24T14:14:35.884908+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901670+0000","last_change":"2024-01-24T14:14:35.897376+0000","last_active":"2024-01-24T14:14:35.901670+0000","last_peered":"2024-01-24T14:14:35.901670+0000","last_clean":"2024-01-24T14:14:35.901670+0000","last_became_active":"2024-01-24T14:14:35.896807+0000","last_became_peered":"2024-01-24T14:14:35.896807+0000","last_unstale":"2024-01-24T14:14:35.901670+0000","last_undegraded":"2024-01-24T14:14:35.901670+0000","last_fullsized":"2024-01-24T14:14:35.901670+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.905681+0000","last_change":"2024-01-24T14:14:35.475976+0000","last_active":"2024-01-24T14:14:35.905681+0000","last_peered":"2024-01-24T14:14:35.905681+0000","last_clean":"2024-01-24T14:14:35.905681+0000","last_became_active":"2024-01-24T14:14:35.475730+0000","last_became_peered":"2024-01-24T14:14:35.475730+0000","last_unstale":"2024-01-24T14:14:35.905681+0000","last_undegraded":"2024-01-24T14:14:35.905681+0000","last_fullsized":"2024-01-24T14:14:35.905681+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.909181+0000","last_change":"2024-01-24T14:14:35.893588+0000","last_active":"2024-01-24T14:14:35.909181+0000","last_peered":"2024-01-24T14:14:35.909181+0000","last_clean":"2024-01-24T14:14:35.909181+0000","last_became_active":"2024-01-24T14:14:35.892982+0000","last_became_peered":"2024-01-24T14:14:35.892982+0000","last_unstale":"2024-01-24T14:14:35.909181+0000","last_undegraded":"2024-01-24T14:14:35.909181+0000","last_fullsized":"2024-01-24T14:14:35.909181+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.902976+0000","last_change":"2024-01-24T14:14:35.472741+0000","last_active":"2024-01-24T14:14:35.902976+0000","last_peered":"2024-01-24T14:14:35.902976+0000","last_clean":"2024-01-24T14:14:35.902976+0000","last_became_active":"2024-01-24T14:14:35.472484+0000","last_became_peered":"2024-01-24T14:14:35.472484+0000","last_unstale":"2024-01-24T14:14:35.902976+0000","last_undegraded":"2024-01-24T14:14:35.902976+0000","last_fullsized":"2024-01-24T14:14:35.902976+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.903175+0000","last_change":"2024-01-24T14:14:35.895271+0000","last_active":"2024-01-24T14:14:35.903175+0000","last_peered":"2024-01-24T14:14:35.903175+0000","last_clean":"2024-01-24T14:14:35.903175+0000","last_became_active":"2024-01-24T14:14:35.894913+0000","last_became_peered":"2024-01-24T14:14:35.894913+0000","last_unstale":"2024-01-24T14:14:35.903175+0000","last_undegraded":"2024-01-24T14:14:35.903175+0000","last_fullsized":"2024-01-24T14:14:35.903175+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.903417+0000","last_change":"2024-01-24T14:14:35.467622+0000","last_active":"2024-01-24T14:14:35.903417+0000","last_peered":"2024-01-24T14:14:35.903417+0000","last_clean":"2024-01-24T14:14:35.903417+0000","last_became_active":"2024-01-24T14:14:35.467023+0000","last_became_peered":"2024-01-24T14:14:35.467023+0000","last_unstale":"2024-01-24T14:14:35.903417+0000","last_undegraded":"2024-01-24T14:14:35.903417+0000","last_fullsized":"2024-01-24T14:14:35.903417+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.904673+0000","last_change":"2024-01-24T14:14:35.898447+0000","last_active":"2024-01-24T14:14:35.904673+0000","last_peered":"2024-01-24T14:14:35.904673+0000","last_clean":"2024-01-24T14:14:35.904673+0000","last_became_active":"2024-01-24T14:14:35.898047+0000","last_became_peered":"2024-01-24T14:14:35.898047+0000","last_unstale":"2024-01-24T14:14:35.904673+0000","last_undegraded":"2024-01-24T14:14:35.904673+0000","last_fullsized":"2024-01-24T14:14:35.904673+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.885841+0000","last_change":"2024-01-24T14:14:35.465989+0000","last_active":"2024-01-24T14:14:35.885841+0000","last_peered":"2024-01-24T14:14:35.885841+0000","last_clean":"2024-01-24T14:14:35.885841+0000","last_became_active":"2024-01-24T14:14:35.465671+0000","last_became_peered":"2024-01-24T14:14:35.465671+0000","last_unstale":"2024-01-24T14:14:35.885841+0000","last_undegraded":"2024-01-24T14:14:35.885841+0000","last_fullsized":"2024-01-24T14:14:35.885841+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.896590+0000","last_change":"2024-01-24T14:14:35.893927+0000","last_active":"2024-01-24T14:14:35.896590+0000","last_peered":"2024-01-24T14:14:35.896590+0000","last_clean":"2024-01-24T14:14:35.896590+0000","last_became_active":"2024-01-24T14:14:35.893635+0000","last_became_peered":"2024-01-24T14:14:35.893635+0000","last_unstale":"2024-01-24T14:14:35.896590+0000","last_undegraded":"2024-01-24T14:14:35.896590+0000","last_fullsized":"2024-01-24T14:14:35.896590+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.882227+0000","last_change":"2024-01-24T14:14:35.467858+0000","last_active":"2024-01-24T14:14:35.882227+0000","last_peered":"2024-01-24T14:14:35.882227+0000","last_clean":"2024-01-24T14:14:35.882227+0000","last_became_active":"2024-01-24T14:14:35.467443+0000","last_became_peered":"2024-01-24T14:14:35.467443+0000","last_unstale":"2024-01-24T14:14:35.882227+0000","last_undegraded":"2024-01-24T14:14:35.882227+0000","last_fullsized":"2024-01-24T14:14:35.882227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.900722+0000","last_change":"2024-01-24T14:14:35.891240+0000","last_active":"2024-01-24T14:14:35.900722+0000","last_peered":"2024-01-24T14:14:35.900722+0000","last_clean":"2024-01-24T14:14:35.900722+0000","last_became_active":"2024-01-24T14:14:35.889909+0000","last_became_peered":"2024-01-24T14:14:35.889909+0000","last_unstale":"2024-01-24T14:14:35.900722+0000","last_undegraded":"2024-01-24T14:14:35.900722+0000","last_fullsized":"2024-01-24T14:14:35.900722+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.885498+0000","last_change":"2024-01-24T14:14:35.472931+0000","last_active":"2024-01-24T14:14:35.885498+0000","last_peered":"2024-01-24T14:14:35.885498+0000","last_clean":"2024-01-24T14:14:35.885498+0000","last_became_active":"2024-01-24T14:14:35.472596+0000","last_became_peered":"2024-01-24T14:14:35.472596+0000","last_unstale":"2024-01-24T14:14:35.885498+0000","last_undegraded":"2024-01-24T14:14:35.885498+0000","last_fullsized":"2024-01-24T14:14:35.885498+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.909620+0000","last_change":"2024-01-24T14:14:35.906271+0000","last_active":"2024-01-24T14:14:35.909620+0000","last_peered":"2024-01-24T14:14:35.909620+0000","last_clean":"2024-01-24T14:14:35.909620+0000","last_became_active":"2024-01-24T14:14:35.906002+0000","last_became_peered":"2024-01-24T14:14:35.906002+0000","last_unstale":"2024-01-24T14:14:35.909620+0000","last_undegraded":"2024-01-24T14:14:35.909620+0000","last_fullsized":"2024-01-24T14:14:35.909620+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.878202+0000","last_change":"2024-01-24T14:14:35.475679+0000","last_active":"2024-01-24T14:14:35.878202+0000","last_peered":"2024-01-24T14:14:35.878202+0000","last_clean":"2024-01-24T14:14:35.878202+0000","last_became_active":"2024-01-24T14:14:35.475377+0000","last_became_peered":"2024-01-24T14:14:35.475377+0000","last_unstale":"2024-01-24T14:14:35.878202+0000","last_undegraded":"2024-01-24T14:14:35.878202+0000","last_fullsized":"2024-01-24T14:14:35.878202+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.898786+0000","last_change":"2024-01-24T14:14:35.889368+0000","last_active":"2024-01-24T14:14:35.898786+0000","last_peered":"2024-01-24T14:14:35.898786+0000","last_clean":"2024-01-24T14:14:35.898786+0000","last_became_active":"2024-01-24T14:14:35.889067+0000","last_became_peered":"2024-01-24T14:14:35.889067+0000","last_unstale":"2024-01-24T14:14:35.898786+0000","last_undegraded":"2024-01-24T14:14:35.898786+0000","last_fullsized":"2024-01-24T14:14:35.898786+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.882872+0000","last_change":"2024-01-24T14:14:35.473022+0000","last_active":"2024-01-24T14:14:35.882872+0000","last_peered":"2024-01-24T14:14:35.882872+0000","last_clean":"2024-01-24T14:14:35.882872+0000","last_became_active":"2024-01-24T14:14:35.472556+0000","last_became_peered":"2024-01-24T14:14:35.472556+0000","last_unstale":"2024-01-24T14:14:35.882872+0000","last_undegraded":"2024-01-24T14:14:35.882872+0000","last_fullsized":"2024-01-24T14:14:35.882872+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.903604+0000","last_change":"2024-01-24T14:14:35.896333+0000","last_active":"2024-01-24T14:14:35.903604+0000","last_peered":"2024-01-24T14:14:35.903604+0000","last_clean":"2024-01-24T14:14:35.903604+0000","last_became_active":"2024-01-24T14:14:35.896083+0000","last_became_peered":"2024-01-24T14:14:35.896083+0000","last_unstale":"2024-01-24T14:14:35.903604+0000","last_undegraded":"2024-01-24T14:14:35.903604+0000","last_fullsized":"2024-01-24T14:14:35.903604+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.882253+0000","last_change":"2024-01-24T14:14:35.468602+0000","last_active":"2024-01-24T14:14:35.882253+0000","last_peered":"2024-01-24T14:14:35.882253+0000","last_clean":"2024-01-24T14:14:35.882253+0000","last_became_active":"2024-01-24T14:14:35.468297+0000","last_became_peered":"2024-01-24T14:14:35.468297+0000","last_unstale":"2024-01-24T14:14:35.882253+0000","last_undegraded":"2024-01-24T14:14:35.882253+0000","last_fullsized":"2024-01-24T14:14:35.882253+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.898809+0000","last_change":"2024-01-24T14:14:35.889589+0000","last_active":"2024-01-24T14:14:35.898809+0000","last_peered":"2024-01-24T14:14:35.898809+0000","last_clean":"2024-01-24T14:14:35.898809+0000","last_became_active":"2024-01-24T14:14:35.889192+0000","last_became_peered":"2024-01-24T14:14:35.889192+0000","last_unstale":"2024-01-24T14:14:35.898809+0000","last_undegraded":"2024-01-24T14:14:35.898809+0000","last_fullsized":"2024-01-24T14:14:35.898809+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.878999+0000","last_change":"2024-01-24T14:14:35.475697+0000","last_active":"2024-01-24T14:14:35.878999+0000","last_peered":"2024-01-24T14:14:35.878999+0000","last_clean":"2024-01-24T14:14:35.878999+0000","last_became_active":"2024-01-24T14:14:35.475370+0000","last_became_peered":"2024-01-24T14:14:35.475370+0000","last_unstale":"2024-01-24T14:14:35.878999+0000","last_undegraded":"2024-01-24T14:14:35.878999+0000","last_fullsized":"2024-01-24T14:14:35.878999+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.904799+0000","last_change":"2024-01-24T14:14:35.901374+0000","last_active":"2024-01-24T14:14:35.904799+0000","last_peered":"2024-01-24T14:14:35.904799+0000","last_clean":"2024-01-24T14:14:35.904799+0000","last_became_active":"2024-01-24T14:14:35.901078+0000","last_became_peered":"2024-01-24T14:14:35.901078+0000","last_unstale":"2024-01-24T14:14:35.904799+0000","last_undegraded":"2024-01-24T14:14:35.904799+0000","last_fullsized":"2024-01-24T14:14:35.904799+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"119'3","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901751+0000","last_change":"2024-01-24T14:14:35.463909+0000","last_active":"2024-01-24T14:14:35.901751+0000","last_peered":"2024-01-24T14:14:35.901751+0000","last_clean":"2024-01-24T14:14:35.901751+0000","last_became_active":"2024-01-24T14:14:35.463582+0000","last_became_peered":"2024-01-24T14:14:35.463582+0000","last_unstale":"2024-01-24T14:14:35.901751+0000","last_undegraded":"2024-01-24T14:14:35.901751+0000","last_fullsized":"2024-01-24T14:14:35.901751+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901842+0000","last_change":"2024-01-24T14:14:35.895171+0000","last_active":"2024-01-24T14:14:35.901842+0000","last_peered":"2024-01-24T14:14:35.901842+0000","last_clean":"2024-01-24T14:14:35.901842+0000","last_became_active":"2024-01-24T14:14:35.894913+0000","last_became_peered":"2024-01-24T14:14:35.894913+0000","last_unstale":"2024-01-24T14:14:35.901842+0000","last_undegraded":"2024-01-24T14:14:35.901842+0000","last_fullsized":"2024-01-24T14:14:35.901842+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.908795+0000","last_change":"2024-01-24T14:14:35.458672+0000","last_active":"2024-01-24T14:14:35.908795+0000","last_peered":"2024-01-24T14:14:35.908795+0000","last_clean":"2024-01-24T14:14:35.908795+0000","last_became_active":"2024-01-24T14:14:35.458402+0000","last_became_peered":"2024-01-24T14:14:35.458402+0000","last_unstale":"2024-01-24T14:14:35.908795+0000","last_undegraded":"2024-01-24T14:14:35.908795+0000","last_fullsized":"2024-01-24T14:14:35.908795+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.908698+0000","last_change":"2024-01-24T14:14:35.891871+0000","last_active":"2024-01-24T14:14:35.908698+0000","last_peered":"2024-01-24T14:14:35.908698+0000","last_clean":"2024-01-24T14:14:35.908698+0000","last_became_active":"2024-01-24T14:14:35.890112+0000","last_became_peered":"2024-01-24T14:14:35.890112+0000","last_unstale":"2024-01-24T14:14:35.908698+0000","last_undegraded":"2024-01-24T14:14:35.908698+0000","last_fullsized":"2024-01-24T14:14:35.908698+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901446+0000","last_change":"2024-01-24T14:14:35.454335+0000","last_active":"2024-01-24T14:14:35.901446+0000","last_peered":"2024-01-24T14:14:35.901446+0000","last_clean":"2024-01-24T14:14:35.901446+0000","last_became_active":"2024-01-24T14:14:35.454026+0000","last_became_peered":"2024-01-24T14:14:35.454026+0000","last_unstale":"2024-01-24T14:14:35.901446+0000","last_undegraded":"2024-01-24T14:14:35.901446+0000","last_fullsized":"2024-01-24T14:14:35.901446+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.902898+0000","last_change":"2024-01-24T14:14:35.897884+0000","last_active":"2024-01-24T14:14:35.902898+0000","last_peered":"2024-01-24T14:14:35.902898+0000","last_clean":"2024-01-24T14:14:35.902898+0000","last_became_active":"2024-01-24T14:14:35.897621+0000","last_became_peered":"2024-01-24T14:14:35.897621+0000","last_unstale":"2024-01-24T14:14:35.902898+0000","last_undegraded":"2024-01-24T14:14:35.902898+0000","last_fullsized":"2024-01-24T14:14:35.902898+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.876531+0000","last_change":"2024-01-24T14:14:35.476562+0000","last_active":"2024-01-24T14:14:35.876531+0000","last_peered":"2024-01-24T14:14:35.876531+0000","last_clean":"2024-01-24T14:14:35.876531+0000","last_became_active":"2024-01-24T14:14:35.476126+0000","last_became_peered":"2024-01-24T14:14:35.476126+0000","last_unstale":"2024-01-24T14:14:35.876531+0000","last_undegraded":"2024-01-24T14:14:35.876531+0000","last_fullsized":"2024-01-24T14:14:35.876531+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.898652+0000","last_change":"2024-01-24T14:14:35.889337+0000","last_active":"2024-01-24T14:14:35.898652+0000","last_peered":"2024-01-24T14:14:35.898652+0000","last_clean":"2024-01-24T14:14:35.898652+0000","last_became_active":"2024-01-24T14:14:35.889027+0000","last_became_peered":"2024-01-24T14:14:35.889027+0000","last_unstale":"2024-01-24T14:14:35.898652+0000","last_undegraded":"2024-01-24T14:14:35.898652+0000","last_fullsized":"2024-01-24T14:14:35.898652+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.882543+0000","last_change":"2024-01-24T14:14:35.460916+0000","last_active":"2024-01-24T14:14:35.882543+0000","last_peered":"2024-01-24T14:14:35.882543+0000","last_clean":"2024-01-24T14:14:35.882543+0000","last_became_active":"2024-01-24T14:14:35.460644+0000","last_became_peered":"2024-01-24T14:14:35.460644+0000","last_unstale":"2024-01-24T14:14:35.882543+0000","last_undegraded":"2024-01-24T14:14:35.882543+0000","last_fullsized":"2024-01-24T14:14:35.882543+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901536+0000","last_change":"2024-01-24T14:14:35.896511+0000","last_active":"2024-01-24T14:14:35.901536+0000","last_peered":"2024-01-24T14:14:35.901536+0000","last_clean":"2024-01-24T14:14:35.901536+0000","last_became_active":"2024-01-24T14:14:35.894513+0000","last_became_peered":"2024-01-24T14:14:35.894513+0000","last_unstale":"2024-01-24T14:14:35.901536+0000","last_undegraded":"2024-01-24T14:14:35.901536+0000","last_fullsized":"2024-01-24T14:14:35.901536+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.904427+0000","last_change":"2024-01-24T14:14:35.469220+0000","last_active":"2024-01-24T14:14:35.904427+0000","last_peered":"2024-01-24T14:14:35.904427+0000","last_clean":"2024-01-24T14:14:35.904427+0000","last_became_active":"2024-01-24T14:14:35.468609+0000","last_became_peered":"2024-01-24T14:14:35.468609+0000","last_unstale":"2024-01-24T14:14:35.904427+0000","last_undegraded":"2024-01-24T14:14:35.904427+0000","last_fullsized":"2024-01-24T14:14:35.904427+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.903623+0000","last_change":"2024-01-24T14:14:35.898847+0000","last_active":"2024-01-24T14:14:35.903623+0000","last_peered":"2024-01-24T14:14:35.903623+0000","last_clean":"2024-01-24T14:14:35.903623+0000","last_became_active":"2024-01-24T14:14:35.898585+0000","last_became_peered":"2024-01-24T14:14:35.898585+0000","last_unstale":"2024-01-24T14:14:35.903623+0000","last_undegraded":"2024-01-24T14:14:35.903623+0000","last_fullsized":"2024-01-24T14:14:35.903623+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.898040+0000","last_change":"2024-01-24T14:14:35.457503+0000","last_active":"2024-01-24T14:14:35.898040+0000","last_peered":"2024-01-24T14:14:35.898040+0000","last_clean":"2024-01-24T14:14:35.898040+0000","last_became_active":"2024-01-24T14:14:35.457228+0000","last_became_peered":"2024-01-24T14:14:35.457228+0000","last_unstale":"2024-01-24T14:14:35.898040+0000","last_undegraded":"2024-01-24T14:14:35.898040+0000","last_fullsized":"2024-01-24T14:14:35.898040+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.902715+0000","last_change":"2024-01-24T14:14:35.897914+0000","last_active":"2024-01-24T14:14:35.902715+0000","last_peered":"2024-01-24T14:14:35.902715+0000","last_clean":"2024-01-24T14:14:35.902715+0000","last_became_active":"2024-01-24T14:14:35.897609+0000","last_became_peered":"2024-01-24T14:14:35.897609+0000","last_unstale":"2024-01-24T14:14:35.902715+0000","last_undegraded":"2024-01-24T14:14:35.902715+0000","last_fullsized":"2024-01-24T14:14:35.902715+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.888436+0000","last_change":"2024-01-24T14:14:35.467092+0000","last_active":"2024-01-24T14:14:35.888436+0000","last_peered":"2024-01-24T14:14:35.888436+0000","last_clean":"2024-01-24T14:14:35.888436+0000","last_became_active":"2024-01-24T14:14:35.466784+0000","last_became_peered":"2024-01-24T14:14:35.466784+0000","last_unstale":"2024-01-24T14:14:35.888436+0000","last_undegraded":"2024-01-24T14:14:35.888436+0000","last_fullsized":"2024-01-24T14:14:35.888436+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.903155+0000","last_change":"2024-01-24T14:14:35.891394+0000","last_active":"2024-01-24T14:14:35.903155+0000","last_peered":"2024-01-24T14:14:35.903155+0000","last_clean":"2024-01-24T14:14:35.903155+0000","last_became_active":"2024-01-24T14:14:35.891073+0000","last_became_peered":"2024-01-24T14:14:35.891073+0000","last_unstale":"2024-01-24T14:14:35.903155+0000","last_undegraded":"2024-01-24T14:14:35.903155+0000","last_fullsized":"2024-01-24T14:14:35.903155+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.885623+0000","last_change":"2024-01-24T14:14:35.467430+0000","last_active":"2024-01-24T14:14:35.885623+0000","last_peered":"2024-01-24T14:14:35.885623+0000","last_clean":"2024-01-24T14:14:35.885623+0000","last_became_active":"2024-01-24T14:14:35.466608+0000","last_became_peered":"2024-01-24T14:14:35.466608+0000","last_unstale":"2024-01-24T14:14:35.885623+0000","last_undegraded":"2024-01-24T14:14:35.885623+0000","last_fullsized":"2024-01-24T14:14:35.885623+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.899320+0000","last_change":"2024-01-24T14:14:35.894406+0000","last_active":"2024-01-24T14:14:35.899320+0000","last_peered":"2024-01-24T14:14:35.899320+0000","last_clean":"2024-01-24T14:14:35.899320+0000","last_became_active":"2024-01-24T14:14:35.894082+0000","last_became_peered":"2024-01-24T14:14:35.894082+0000","last_unstale":"2024-01-24T14:14:35.899320+0000","last_undegraded":"2024-01-24T14:14:35.899320+0000","last_fullsized":"2024-01-24T14:14:35.899320+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.877043+0000","last_change":"2024-01-24T14:14:35.459822+0000","last_active":"2024-01-24T14:14:35.877043+0000","last_peered":"2024-01-24T14:14:35.877043+0000","last_clean":"2024-01-24T14:14:35.877043+0000","last_became_active":"2024-01-24T14:14:35.459415+0000","last_became_peered":"2024-01-24T14:14:35.459415+0000","last_unstale":"2024-01-24T14:14:35.877043+0000","last_undegraded":"2024-01-24T14:14:35.877043+0000","last_fullsized":"2024-01-24T14:14:35.877043+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901401+0000","last_change":"2024-01-24T14:14:35.892422+0000","last_active":"2024-01-24T14:14:35.901401+0000","last_peered":"2024-01-24T14:14:35.901401+0000","last_clean":"2024-01-24T14:14:35.901401+0000","last_became_active":"2024-01-24T14:14:35.892151+0000","last_became_peered":"2024-01-24T14:14:35.892151+0000","last_unstale":"2024-01-24T14:14:35.901401+0000","last_undegraded":"2024-01-24T14:14:35.901401+0000","last_fullsized":"2024-01-24T14:14:35.901401+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.885397+0000","last_change":"2024-01-24T14:14:35.475606+0000","last_active":"2024-01-24T14:14:35.885397+0000","last_peered":"2024-01-24T14:14:35.885397+0000","last_clean":"2024-01-24T14:14:35.885397+0000","last_became_active":"2024-01-24T14:14:35.475405+0000","last_became_peered":"2024-01-24T14:14:35.475405+0000","last_unstale":"2024-01-24T14:14:35.885397+0000","last_undegraded":"2024-01-24T14:14:35.885397+0000","last_fullsized":"2024-01-24T14:14:35.885397+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.898707+0000","last_change":"2024-01-24T14:14:35.889991+0000","last_active":"2024-01-24T14:14:35.898707+0000","last_peered":"2024-01-24T14:14:35.898707+0000","last_clean":"2024-01-24T14:14:35.898707+0000","last_became_active":"2024-01-24T14:14:35.889666+0000","last_became_peered":"2024-01-24T14:14:35.889666+0000","last_unstale":"2024-01-24T14:14:35.898707+0000","last_undegraded":"2024-01-24T14:14:35.898707+0000","last_fullsized":"2024-01-24T14:14:35.898707+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.875923+0000","last_change":"2024-01-24T14:14:35.476434+0000","last_active":"2024-01-24T14:14:35.875923+0000","last_peered":"2024-01-24T14:14:35.875923+0000","last_clean":"2024-01-24T14:14:35.875923+0000","last_became_active":"2024-01-24T14:14:35.476160+0000","last_became_peered":"2024-01-24T14:14:35.476160+0000","last_unstale":"2024-01-24T14:14:35.875923+0000","last_undegraded":"2024-01-24T14:14:35.875923+0000","last_fullsized":"2024-01-24T14:14:35.875923+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901873+0000","last_change":"2024-01-24T14:14:35.899429+0000","last_active":"2024-01-24T14:14:35.901873+0000","last_peered":"2024-01-24T14:14:35.901873+0000","last_clean":"2024-01-24T14:14:35.901873+0000","last_became_active":"2024-01-24T14:14:35.894599+0000","last_became_peered":"2024-01-24T14:14:35.894599+0000","last_unstale":"2024-01-24T14:14:35.901873+0000","last_undegraded":"2024-01-24T14:14:35.901873+0000","last_fullsized":"2024-01-24T14:14:35.901873+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.885278+0000","last_change":"2024-01-24T14:14:35.475127+0000","last_active":"2024-01-24T14:14:35.885278+0000","last_peered":"2024-01-24T14:14:35.885278+0000","last_clean":"2024-01-24T14:14:35.885278+0000","last_became_active":"2024-01-24T14:14:35.474574+0000","last_became_peered":"2024-01-24T14:14:35.474574+0000","last_unstale":"2024-01-24T14:14:35.885278+0000","last_undegraded":"2024-01-24T14:14:35.885278+0000","last_fullsized":"2024-01-24T14:14:35.885278+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.899521+0000","last_change":"2024-01-24T14:14:35.895046+0000","last_active":"2024-01-24T14:14:35.899521+0000","last_peered":"2024-01-24T14:14:35.899521+0000","last_clean":"2024-01-24T14:14:35.899521+0000","last_became_active":"2024-01-24T14:14:35.894807+0000","last_became_peered":"2024-01-24T14:14:35.894807+0000","last_unstale":"2024-01-24T14:14:35.899521+0000","last_undegraded":"2024-01-24T14:14:35.899521+0000","last_fullsized":"2024-01-24T14:14:35.899521+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.876240+0000","last_change":"2024-01-24T14:14:35.478171+0000","last_active":"2024-01-24T14:14:35.876240+0000","last_peered":"2024-01-24T14:14:35.876240+0000","last_clean":"2024-01-24T14:14:35.876240+0000","last_became_active":"2024-01-24T14:14:35.477856+0000","last_became_peered":"2024-01-24T14:14:35.477856+0000","last_unstale":"2024-01-24T14:14:35.876240+0000","last_undegraded":"2024-01-24T14:14:35.876240+0000","last_fullsized":"2024-01-24T14:14:35.876240+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.890583+0000","last_change":"2024-01-24T14:14:35.887730+0000","last_active":"2024-01-24T14:14:35.890583+0000","last_peered":"2024-01-24T14:14:35.890583+0000","last_clean":"2024-01-24T14:14:35.890583+0000","last_became_active":"2024-01-24T14:14:35.886793+0000","last_became_peered":"2024-01-24T14:14:35.886793+0000","last_unstale":"2024-01-24T14:14:35.890583+0000","last_undegraded":"2024-01-24T14:14:35.890583+0000","last_fullsized":"2024-01-24T14:14:35.890583+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.885043+0000","last_change":"2024-01-24T14:14:35.477495+0000","last_active":"2024-01-24T14:14:35.885043+0000","last_peered":"2024-01-24T14:14:35.885043+0000","last_clean":"2024-01-24T14:14:35.885043+0000","last_became_active":"2024-01-24T14:14:35.476274+0000","last_became_peered":"2024-01-24T14:14:35.476274+0000","last_unstale":"2024-01-24T14:14:35.885043+0000","last_undegraded":"2024-01-24T14:14:35.885043+0000","last_fullsized":"2024-01-24T14:14:35.885043+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":96,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.886234+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:14:35.886234+0000","last_peered":"2024-01-24T14:14:35.886234+0000","last_clean":"2024-01-24T14:14:35.886234+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:14:35.886234+0000","last_undegraded":"2024-01-24T14:14:35.886234+0000","last_fullsized":"2024-01-24T14:14:35.886234+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.899497+0000","last_change":"2024-01-24T14:14:35.898244+0000","last_active":"2024-01-24T14:14:35.899497+0000","last_peered":"2024-01-24T14:14:35.899497+0000","last_clean":"2024-01-24T14:14:35.899497+0000","last_became_active":"2024-01-24T14:14:35.898055+0000","last_became_peered":"2024-01-24T14:14:35.898055+0000","last_unstale":"2024-01-24T14:14:35.899497+0000","last_undegraded":"2024-01-24T14:14:35.899497+0000","last_fullsized":"2024-01-24T14:14:35.899497+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.902445+0000","last_change":"2024-01-24T14:14:35.463322+0000","last_active":"2024-01-24T14:14:35.902445+0000","last_peered":"2024-01-24T14:14:35.902445+0000","last_clean":"2024-01-24T14:14:35.902445+0000","last_became_active":"2024-01-24T14:14:35.462920+0000","last_became_peered":"2024-01-24T14:14:35.462920+0000","last_unstale":"2024-01-24T14:14:35.902445+0000","last_undegraded":"2024-01-24T14:14:35.902445+0000","last_fullsized":"2024-01-24T14:14:35.902445+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":107,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.875717+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:14:35.875717+0000","last_peered":"2024-01-24T14:14:35.875717+0000","last_clean":"2024-01-24T14:14:35.875717+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:14:35.875717+0000","last_undegraded":"2024-01-24T14:14:35.875717+0000","last_fullsized":"2024-01-24T14:14:35.875717+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.888611+0000","last_change":"2024-01-24T14:14:35.883624+0000","last_active":"2024-01-24T14:14:35.888611+0000","last_peered":"2024-01-24T14:14:35.888611+0000","last_clean":"2024-01-24T14:14:35.888611+0000","last_became_active":"2024-01-24T14:14:35.883326+0000","last_became_peered":"2024-01-24T14:14:35.883326+0000","last_unstale":"2024-01-24T14:14:35.888611+0000","last_undegraded":"2024-01-24T14:14:35.888611+0000","last_fullsized":"2024-01-24T14:14:35.888611+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.877224+0000","last_change":"2024-01-24T14:14:35.462928+0000","last_active":"2024-01-24T14:14:35.877224+0000","last_peered":"2024-01-24T14:14:35.877224+0000","last_clean":"2024-01-24T14:14:35.877224+0000","last_became_active":"2024-01-24T14:14:35.462446+0000","last_became_peered":"2024-01-24T14:14:35.462446+0000","last_unstale":"2024-01-24T14:14:35.877224+0000","last_undegraded":"2024-01-24T14:14:35.877224+0000","last_fullsized":"2024-01-24T14:14:35.877224+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":103,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.875425+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:14:35.875425+0000","last_peered":"2024-01-24T14:14:35.875425+0000","last_clean":"2024-01-24T14:14:35.875425+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:14:35.875425+0000","last_undegraded":"2024-01-24T14:14:35.875425+0000","last_fullsized":"2024-01-24T14:14:35.875425+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901634+0000","last_change":"2024-01-24T14:14:35.893953+0000","last_active":"2024-01-24T14:14:35.901634+0000","last_peered":"2024-01-24T14:14:35.901634+0000","last_clean":"2024-01-24T14:14:35.901634+0000","last_became_active":"2024-01-24T14:14:35.893649+0000","last_became_peered":"2024-01-24T14:14:35.893649+0000","last_unstale":"2024-01-24T14:14:35.901634+0000","last_undegraded":"2024-01-24T14:14:35.901634+0000","last_fullsized":"2024-01-24T14:14:35.901634+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.908776+0000","last_change":"2024-01-24T14:14:35.477665+0000","last_active":"2024-01-24T14:14:35.908776+0000","last_peered":"2024-01-24T14:14:35.908776+0000","last_clean":"2024-01-24T14:14:35.908776+0000","last_became_active":"2024-01-24T14:14:35.477322+0000","last_became_peered":"2024-01-24T14:14:35.477322+0000","last_unstale":"2024-01-24T14:14:35.908776+0000","last_undegraded":"2024-01-24T14:14:35.908776+0000","last_fullsized":"2024-01-24T14:14:35.908776+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":107,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.875585+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:14:35.875585+0000","last_peered":"2024-01-24T14:14:35.875585+0000","last_clean":"2024-01-24T14:14:35.875585+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:14:35.875585+0000","last_undegraded":"2024-01-24T14:14:35.875585+0000","last_fullsized":"2024-01-24T14:14:35.875585+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.879226+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:14:35.879226+0000","last_peered":"2024-01-24T14:14:35.879226+0000","last_clean":"2024-01-24T14:14:35.879226+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:14:35.879226+0000","last_undegraded":"2024-01-24T14:14:35.879226+0000","last_fullsized":"2024-01-24T14:14:35.879226+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901803+0000","last_change":"2024-01-24T14:14:35.893909+0000","last_active":"2024-01-24T14:14:35.901803+0000","last_peered":"2024-01-24T14:14:35.901803+0000","last_clean":"2024-01-24T14:14:35.901803+0000","last_became_active":"2024-01-24T14:14:35.893604+0000","last_became_peered":"2024-01-24T14:14:35.893604+0000","last_unstale":"2024-01-24T14:14:35.901803+0000","last_undegraded":"2024-01-24T14:14:35.901803+0000","last_fullsized":"2024-01-24T14:14:35.901803+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.885944+0000","last_change":"2024-01-24T14:14:35.467201+0000","last_active":"2024-01-24T14:14:35.885944+0000","last_peered":"2024-01-24T14:14:35.885944+0000","last_clean":"2024-01-24T14:14:35.885944+0000","last_became_active":"2024-01-24T14:14:35.464512+0000","last_became_peered":"2024-01-24T14:14:35.464512+0000","last_unstale":"2024-01-24T14:14:35.885944+0000","last_undegraded":"2024-01-24T14:14:35.885944+0000","last_fullsized":"2024-01-24T14:14:35.885944+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":98,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.881504+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:14:35.881504+0000","last_peered":"2024-01-24T14:14:35.881504+0000","last_clean":"2024-01-24T14:14:35.881504+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:14:35.881504+0000","last_undegraded":"2024-01-24T14:14:35.881504+0000","last_fullsized":"2024-01-24T14:14:35.881504+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901886+0000","last_change":"2024-01-24T14:14:35.899805+0000","last_active":"2024-01-24T14:14:35.901886+0000","last_peered":"2024-01-24T14:14:35.901886+0000","last_clean":"2024-01-24T14:14:35.901886+0000","last_became_active":"2024-01-24T14:14:35.899500+0000","last_became_peered":"2024-01-24T14:14:35.899500+0000","last_unstale":"2024-01-24T14:14:35.901886+0000","last_undegraded":"2024-01-24T14:14:35.901886+0000","last_fullsized":"2024-01-24T14:14:35.901886+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.875356+0000","last_change":"2024-01-24T14:14:35.456864+0000","last_active":"2024-01-24T14:14:35.875356+0000","last_peered":"2024-01-24T14:14:35.875356+0000","last_clean":"2024-01-24T14:14:35.875356+0000","last_became_active":"2024-01-24T14:14:35.456577+0000","last_became_peered":"2024-01-24T14:14:35.456577+0000","last_unstale":"2024-01-24T14:14:35.875356+0000","last_undegraded":"2024-01-24T14:14:35.875356+0000","last_fullsized":"2024-01-24T14:14:35.875356+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":98,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.882816+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:14:35.882816+0000","last_peered":"2024-01-24T14:14:35.882816+0000","last_clean":"2024-01-24T14:14:35.882816+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:14:35.882816+0000","last_undegraded":"2024-01-24T14:14:35.882816+0000","last_fullsized":"2024-01-24T14:14:35.882816+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901517+0000","last_change":"2024-01-24T14:14:35.894078+0000","last_active":"2024-01-24T14:14:35.901517+0000","last_peered":"2024-01-24T14:14:35.901517+0000","last_clean":"2024-01-24T14:14:35.901517+0000","last_became_active":"2024-01-24T14:14:35.893834+0000","last_became_peered":"2024-01-24T14:14:35.893834+0000","last_unstale":"2024-01-24T14:14:35.901517+0000","last_undegraded":"2024-01-24T14:14:35.901517+0000","last_fullsized":"2024-01-24T14:14:35.901517+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.875629+0000","last_change":"2024-01-24T14:14:35.476605+0000","last_active":"2024-01-24T14:14:35.875629+0000","last_peered":"2024-01-24T14:14:35.875629+0000","last_clean":"2024-01-24T14:14:35.875629+0000","last_became_active":"2024-01-24T14:14:35.476308+0000","last_became_peered":"2024-01-24T14:14:35.476308+0000","last_unstale":"2024-01-24T14:14:35.875629+0000","last_undegraded":"2024-01-24T14:14:35.875629+0000","last_fullsized":"2024-01-24T14:14:35.875629+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":96,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.882575+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:14:35.882575+0000","last_peered":"2024-01-24T14:14:35.882575+0000","last_clean":"2024-01-24T14:14:35.882575+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:14:35.882575+0000","last_undegraded":"2024-01-24T14:14:35.882575+0000","last_fullsized":"2024-01-24T14:14:35.882575+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.901425+0000","last_change":"2024-01-24T14:14:35.893977+0000","last_active":"2024-01-24T14:14:35.901425+0000","last_peered":"2024-01-24T14:14:35.901425+0000","last_clean":"2024-01-24T14:14:35.901425+0000","last_became_active":"2024-01-24T14:14:35.893643+0000","last_became_peered":"2024-01-24T14:14:35.893643+0000","last_unstale":"2024-01-24T14:14:35.901425+0000","last_undegraded":"2024-01-24T14:14:35.901425+0000","last_fullsized":"2024-01-24T14:14:35.901425+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:35.432490+0000","last_clean_scrub_stamp":"2024-01-24T14:14:35.432490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.903223+0000","last_change":"2024-01-24T14:14:35.467253+0000","last_active":"2024-01-24T14:14:35.903223+0000","last_peered":"2024-01-24T14:14:35.903223+0000","last_clean":"2024-01-24T14:14:35.903223+0000","last_became_active":"2024-01-24T14:14:35.466900+0000","last_became_peered":"2024-01-24T14:14:35.466900+0000","last_unstale":"2024-01-24T14:14:35.903223+0000","last_undegraded":"2024-01-24T14:14:35.903223+0000","last_fullsized":"2024-01-24T14:14:35.903223+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:14:34.424934+0000","last_clean_scrub_stamp":"2024-01-24T14:14:34.424934+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":95,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-24T14:14:35.877163+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:14:35.877163+0000","last_peered":"2024-01-24T14:14:35.877163+0000","last_clean":"2024-01-24T14:14:35.877163+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:14:35.877163+0000","last_undegraded":"2024-01-24T14:14:35.877163+0000","last_fullsized":"2024-01-24T14:14:35.877163+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575044,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302644,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069196,"statfs":{"total":96636764160,"available":96326856704,"internally_reserved":0,"allocated":1486848,"data_stored":823601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.26000000000000001},"max":{"1min":0.80600000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.32700000000000001},"max":{"1min":0.753,"5min":0.82599999999999996,"15min":0.83499999999999996},"last":0.66400000000000003}]},{"osd":1,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.312},"max":{"1min":0.75600000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":3.2570000000000001},"last":0.61899999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.51700000000000002,"5min":0.40400000000000003,"15min":0.32000000000000001},"max":{"1min":0.79100000000000004,"5min":0.872,"15min":0.876},"last":0.629},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.39300000000000002,"15min":0.36799999999999999},"max":{"1min":0.80000000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.36499999999999999},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.85099999999999998,"15min":3.0049999999999999},"last":0.60499999999999998}]},{"osd":4,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48899999999999999},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.84699999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.60399999999999998,"5min":0.70599999999999996,"15min":0.83599999999999997},"last":0.46600000000000003}]},{"osd":5,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.501,"15min":0.497},"min":{"1min":0.307,"5min":0.307,"15min":0.29999999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.80900000000000005},"last":0.50900000000000001}]},{"osd":6,"last update":"Wed Jan 24 14:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.495},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.80700000000000005,"15min":0.87},"last":0.57699999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302636,"kb_used_data":1444,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069204,"statfs":{"total":96636764160,"available":96326864896,"internally_reserved":0,"allocated":1478656,"data_stored":823489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.59499999999999997},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73699999999999999,"5min":0.85699999999999998,"15min":0.86899999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":0.84899999999999998,"15min":1.3859999999999999},"last":0.58599999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73499999999999999,"5min":0.77000000000000002,"15min":1.353},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.78700000000000003,"5min":1.032,"15min":1.032},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60099999999999998,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.39100000000000001,"15min":0.34300000000000003},"max":{"1min":0.78600000000000003,"5min":0.88500000000000001,"15min":6.8129999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":1.4079999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.66900000000000004},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81699999999999995,"5min":0.82799999999999996,"15min":7.016},"last":0.496},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.43099999999999999,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.77400000000000002,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.57099999999999995}]},{"osd":4,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.504,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":1.165,"15min":1.165},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.314},"max":{"1min":0.65500000000000003,"5min":0.82299999999999995,"15min":6.8719999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":0.86699999999999999,"15min":6.976},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80200000000000005,"5min":0.85699999999999998,"15min":7.8789999999999996},"last":0.56200000000000006}]},{"osd":7,"last update":"Wed Jan 24 14:13:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.747,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":0.89600000000000002,"15min":6.9080000000000004},"last":0.71499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302640,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069200,"statfs":{"total":96636764160,"available":96326860800,"internally_reserved":0,"allocated":1482752,"data_stored":823559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.51900000000000002,"15min":0.443},"min":{"1min":0.311,"5min":0.245,"15min":0.23400000000000001},"max":{"1min":0.57899999999999996,"5min":15.298999999999999,"15min":15.298999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.55000000000000004,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.57499999999999996,"5min":14.654,"15min":14.654},"last":0.375}]},{"osd":2,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45900000000000002,"15min":0.45100000000000001},"min":{"1min":0.34899999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.59799999999999998,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.25800000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.441}]},{"osd":3,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47099999999999997,"15min":0.45700000000000002},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.39600000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88600000000000001,"5min":1.073,"15min":1.137},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.45900000000000002,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.218,"15min":1.218},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":1.1040000000000001,"15min":0.83299999999999996},"min":{"1min":0.47999999999999998,"5min":0.45000000000000001,"15min":0.443},"max":{"1min":0.86799999999999999,"5min":38.808,"15min":38.808},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":1.077,"15min":0.81200000000000006},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.40500000000000003},"max":{"1min":0.81299999999999994,"5min":38.823,"15min":38.823},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.46000000000000002,"15min":0.43099999999999999},"max":{"1min":0.94899999999999995,"5min":1.0449999999999999,"15min":1.0509999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.40000000000000002},"max":{"1min":0.91300000000000003,"5min":1.1559999999999999,"15min":1.1879999999999999},"last":0.79900000000000004}]},{"osd":7,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.52500000000000002,"5min":0.49399999999999999,"15min":0.45300000000000001},"max":{"1min":0.98499999999999999,"5min":1.105,"15min":1.155},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.54900000000000004,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":0.93500000000000005,"5min":1.123,"15min":1.131},"last":0.77300000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542342,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302636,"kb_used_data":1444,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069204,"statfs":{"total":96636764160,"available":96326864896,"internally_reserved":0,"allocated":1478656,"data_stored":823082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39600000000000002},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.57499999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.36199999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44800000000000001,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.28199999999999997},"max":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.66700000000000004},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.28799999999999998},"max":{"1min":0.61399999999999999,"5min":0.754,"15min":0.79400000000000004},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.33000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54500000000000004,"5min":1.482,"15min":1.482},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.496},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":0.872,"15min":0.872},"last":0.378}]},{"osd":4,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81499999999999995,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.875,"5min":1.655,"15min":1.655},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.42999999999999999},"max":{"1min":0.77200000000000002,"5min":0.94799999999999995,"15min":1.0840000000000001},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.875,"5min":0.90700000000000003,"15min":1.137},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":0.96499999999999997,"15min":1.117},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed Jan 24 14:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47099999999999997},"max":{"1min":0.86899999999999999,"5min":0.96099999999999997,"15min":0.98199999999999998},"last":0.746},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.49099999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.59699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306740,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94065100,"statfs":{"total":96636764160,"available":96322662400,"internally_reserved":0,"allocated":1486848,"data_stored":824415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.29499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.65800000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.24399999999999999},"max":{"1min":0.54500000000000004,"5min":0.65900000000000003,"15min":0.67100000000000004},"last":0.308}]},{"osd":1,"last update":"Wed Jan 24 14:14:34 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41999999999999998,"15min":0.41499999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56299999999999994,"5min":0.64400000000000002,"15min":0.67200000000000004},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.45200000000000001,"15min":0.46100000000000002},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.29699999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.37}]},{"osd":3,"last update":"Wed Jan 24 14:14:34 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.443,"15min":0.45200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.629,"5min":0.67700000000000005,"15min":0.71699999999999997},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.46600000000000003,"15min":0.47099999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.30299999999999999},"max":{"1min":0.66000000000000003,"5min":0.70799999999999996,"15min":0.72799999999999998},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:14:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.624,"15min":0.64200000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93500000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.438},"max":{"1min":0.82299999999999995,"5min":0.98599999999999999,"15min":1.042},"last":0.57999999999999996}]},{"osd":5,"last update":"Wed Jan 24 14:14:34 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66100000000000003,"15min":0.67900000000000005},"min":{"1min":0.46200000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86299999999999999,"5min":1.036,"15min":1.0469999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65400000000000003,"15min":0.67100000000000004},"min":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88800000000000001,"5min":1.107,"15min":1.107},"last":0.56999999999999995}]},{"osd":6,"last update":"Wed Jan 24 14:14:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68000000000000005,"15min":0.69499999999999995},"min":{"1min":0.48699999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94799999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.46800000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88300000000000001,"5min":1.141,"15min":1.141},"last":0.622}]},{"osd":7,"last update":"Wed Jan 24 14:14:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69599999999999995,"15min":0.71599999999999997},"min":{"1min":0.45700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.91100000000000003,"5min":0.96199999999999997,"15min":1.0409999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.71199999999999997,"15min":0.72399999999999998},"min":{"1min":0.503,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92000000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.67200000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1499136,"data_stored":825601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45100000000000001,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.76400000000000001,"5min":7.593,"15min":7.593},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.48299999999999998,"15min":0.437},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56200000000000006,"5min":8.4239999999999995,"15min":8.4239999999999995},"last":0.40000000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.69099999999999995},"last":0.378},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.221,"15min":0.221},"max":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.68799999999999994},"last":0.371}]},{"osd":2,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.496},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.66500000000000004,"5min":0.76300000000000001,"15min":0.77200000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.78800000000000003},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64500000000000002,"15min":0.63500000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.495,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":0.81599999999999995,"5min":1.976,"15min":1.976},"last":0.52300000000000002}]},{"osd":5,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91500000000000004,"5min":1.53,"15min":1.53},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.97999999999999998,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.50700000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:13:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72799999999999998,"15min":0.72099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.72999999999999998},"min":{"1min":0.495,"5min":0.49199999999999999,"15min":0.45800000000000002},"max":{"1min":0.93500000000000005,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.59199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575045,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1499136,"data_stored":824327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78300000000000003,"5min":0.80100000000000005,"15min":0.88600000000000001},"last":0.495},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.84399999999999997},"last":0.51900000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.83099999999999996,"15min":0.85299999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.755,"5min":0.79800000000000004,"15min":1.506},"last":0.53900000000000003}]},{"osd":2,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.80200000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45200000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.72899999999999998,"5min":0.77800000000000002,"15min":0.80600000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36499999999999999},"max":{"1min":0.79300000000000004,"5min":0.81000000000000005,"15min":0.86299999999999999},"last":0.52400000000000002}]},{"osd":5,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.92000000000000004},"last":0.5},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.496,"15min":0.5},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65700000000000003,"5min":0.72899999999999998,"15min":1.556},"last":0.435}]},{"osd":6,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.753,"5min":0.78500000000000003,"15min":0.86099999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76400000000000001,"5min":0.82099999999999995,"15min":1.0409999999999999},"last":0.47499999999999998}]},{"osd":7,"last update":"Wed Jan 24 14:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.38500000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.88100000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.307,"15min":0.307},"max":{"1min":0.72699999999999998,"5min":0.81399999999999995,"15min":1.26},"last":0.55300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575045,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302636,"kb_used_data":1444,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069204,"statfs":{"total":96636764160,"available":96326864896,"internally_reserved":0,"allocated":1478656,"data_stored":823489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:14:35 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.53500000000000003,"15min":0.56399999999999995},"min":{"1min":0.34300000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.67900000000000005,"5min":0.85199999999999998,"15min":1.0800000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.36299999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76400000000000001,"5min":0.82699999999999996,"15min":0.84399999999999997},"last":0.46400000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:14:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54100000000000004,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57299999999999995,"15min":0.58999999999999997},"min":{"1min":0.374,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70999999999999996,"5min":0.81299999999999994,"15min":1.1639999999999999},"last":0.42399999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:14:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.39100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66800000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56699999999999995,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77600000000000002,"5min":0.83599999999999997,"15min":0.83699999999999997},"last":0.39400000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:14:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.58399999999999996,"15min":0.60099999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.749,"5min":0.80500000000000005,"15min":1.139},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.44800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.72799999999999998,"5min":0.83099999999999996,"15min":0.85399999999999998},"last":0.501}]},{"osd":4,"last update":"Wed Jan 24 14:14:35 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.34000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65400000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.498,"15min":0.499},"min":{"1min":0.35599999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.60099999999999998,"5min":0.84999999999999998,"15min":0.90000000000000002},"last":0.44700000000000001}]},{"osd":6,"last update":"Wed Jan 24 14:14:35 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.504,"15min":0.504},"min":{"1min":0.307,"5min":0.253,"15min":0.253},"max":{"1min":0.72499999999999998,"5min":0.86399999999999999,"15min":1.1839999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.36599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69099999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55800000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:14:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.53000000000000003,"15min":0.53700000000000003},"min":{"1min":0.34200000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.63900000000000001,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.70299999999999996,"5min":0.95999999999999996,"15min":1.2030000000000001},"last":0.57399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:14:40.402 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-24T14:14:40.402 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-24T14:14:40.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.576+0000 7f60cc28e700 1 Processor -- start 2024-01-24T14:14:40.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.576+0000 7f60cc28e700 1 -- start start 2024-01-24T14:14:40.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.577+0000 7f60cc28e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f60c414e6a0 0x7f60c40c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.577+0000 7f60cc28e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f60c414ea80 0x7f60c4154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.577+0000 7f60cc28e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60c4158620 0x7f60c40627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.577+0000 7f60cc28e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c4158190 con 0x7f60c414ea80 2024-01-24T14:14:40.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.577+0000 7f60cc28e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c4155110 con 0x7f60c414e6a0 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.577+0000 7f60cc28e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41553f0 con 0x7f60c4158620 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.577+0000 7f60c9829700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f60c414ea80 0x7f60c4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.577+0000 7f60c9829700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f60c414ea80 0x7f60c4154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41456/0 (socket says 172.21.15.29:41456) 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.577+0000 7f60c9829700 1 -- 172.21.15.29:0/3885549344 learned_addr learned my addr 172.21.15.29:0/3885549344 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.577+0000 7f60ca82b700 1 --2- 172.21.15.29:0/3885549344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60c4158620 0x7f60c40627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.577+0000 7f60ca02a700 1 --2- 172.21.15.29:0/3885549344 >> v2:172.21.15.163:3300/0 conn(0x7f60c414e6a0 0x7f60c40c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.578+0000 7f60c9829700 1 -- 172.21.15.29:0/3885549344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60c4158620 msgr2=0x7f60c40627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.578+0000 7f60c9829700 1 --2- 172.21.15.29:0/3885549344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60c4158620 0x7f60c40627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.578+0000 7f60c9829700 1 -- 172.21.15.29:0/3885549344 >> v2:172.21.15.163:3300/0 conn(0x7f60c414e6a0 msgr2=0x7f60c40c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.578+0000 7f60c9829700 1 --2- 172.21.15.29:0/3885549344 >> v2:172.21.15.163:3300/0 conn(0x7f60c414e6a0 0x7f60c40c2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.578+0000 7f60c9829700 1 -- 172.21.15.29:0/3885549344 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c41556d0 con 0x7f60c414ea80 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.578+0000 7f60c9829700 1 --2- 172.21.15.29:0/3885549344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60c414ea80 0x7f60c4154890 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f60c40beb90 tx=0x7f60c00090f0).ready entity=mon.0 client_cookie=7f035aff6e8ad6e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.578+0000 7f60c9028700 1 -- 172.21.15.29:0/3885549344 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c000f510 con 0x7f60c414ea80 2024-01-24T14:14:40.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.578+0000 7f60c9028700 1 -- 172.21.15.29:0/3885549344 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c000f670 con 0x7f60c414ea80 2024-01-24T14:14:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.578+0000 7f60c9028700 1 -- 172.21.15.29:0/3885549344 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c000fa10 con 0x7f60c414ea80 2024-01-24T14:14:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.578+0000 7f60cc28e700 1 -- 172.21.15.29:0/3885549344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60c414ea80 msgr2=0x7f60c4154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.578+0000 7f60cc28e700 1 --2- 172.21.15.29:0/3885549344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60c414ea80 0x7f60c4154890 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f60c40beb90 tx=0x7f60c00090f0).stop 2024-01-24T14:14:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.579+0000 7f60cc28e700 1 -- 172.21.15.29:0/3885549344 shutdown_connections 2024-01-24T14:14:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.579+0000 7f60cc28e700 1 --2- 172.21.15.29:0/3885549344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60c4158620 0x7f60c40627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.579+0000 7f60cc28e700 1 --2- 172.21.15.29:0/3885549344 >> v2:172.21.15.163:3300/0 conn(0x7f60c414e6a0 0x7f60c40c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.579+0000 7f60cc28e700 1 --2- 172.21.15.29:0/3885549344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60c414ea80 0x7f60c4154890 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.579+0000 7f60cc28e700 1 -- 172.21.15.29:0/3885549344 >> 172.21.15.29:0/3885549344 conn(0x7f60c40c7bc0 msgr2=0x7f60c40c7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.579+0000 7f60cc28e700 1 -- 172.21.15.29:0/3885549344 shutdown_connections 2024-01-24T14:14:40.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.579+0000 7f60cc28e700 1 -- 172.21.15.29:0/3885549344 wait complete. 2024-01-24T14:14:40.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.580+0000 7f60cc28e700 1 Processor -- start 2024-01-24T14:14:40.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.580+0000 7f60cc28e700 1 -- start start 2024-01-24T14:14:40.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60cc28e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60c414e6a0 0x7f60c414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60cc28e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60c414ea80 0x7f60c40a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60cc28e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60c4158620 0x7f60c40a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60cc28e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c40aa010 con 0x7f60c4158620 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60cc28e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c414b6d0 con 0x7f60c414e6a0 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60cc28e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c40a9400 con 0x7f60c414ea80 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60ca82b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60c4158620 0x7f60c40a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60ca82b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60c4158620 0x7f60c40a5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41460/0 (socket says 172.21.15.29:41460) 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60ca82b700 1 -- 172.21.15.29:0/3578054473 learned_addr learned my addr 172.21.15.29:0/3578054473 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60ca02a700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60c414e6a0 0x7f60c414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60c9829700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60c414ea80 0x7f60c40a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60ca82b700 1 -- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60c414ea80 msgr2=0x7f60c40a8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60ca82b700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60c414ea80 0x7f60c40a8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60ca82b700 1 -- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60c414e6a0 msgr2=0x7f60c414b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:40.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60ca82b700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60c414e6a0 0x7f60c414b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60ca82b700 1 -- 172.21.15.29:0/3578054473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c0009370 con 0x7f60c4158620 2024-01-24T14:14:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60ca82b700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60c4158620 0x7f60c40a5850 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f60bc009df0 tx=0x7f60bc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60b67fc700 1 -- 172.21.15.29:0/3578054473 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc00cb50 con 0x7f60c4158620 2024-01-24T14:14:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.581+0000 7f60b67fc700 1 -- 172.21.15.29:0/3578054473 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc00f040 con 0x7f60c4158620 2024-01-24T14:14:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.582+0000 7f60cc28e700 1 -- 172.21.15.29:0/3578054473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c40a96e0 con 0x7f60c4158620 2024-01-24T14:14:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.582+0000 7f60b67fc700 1 -- 172.21.15.29:0/3578054473 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc00cec0 con 0x7f60c4158620 2024-01-24T14:14:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.582+0000 7f60cc28e700 1 -- 172.21.15.29:0/3578054473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c40a9ca0 con 0x7f60c4158620 2024-01-24T14:14:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.583+0000 7f60cc28e700 1 -- 172.21.15.29:0/3578054473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c40475e0 con 0x7f60c4158620 2024-01-24T14:14:40.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.583+0000 7f60b67fc700 1 -- 172.21.15.29:0/3578054473 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f60bc013610 con 0x7f60c4158620 2024-01-24T14:14:40.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.584+0000 7f60b67fc700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f60a805f270 0x7f60a8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.584+0000 7f60b67fc700 1 -- 172.21.15.29:0/3578054473 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f60bc07b050 con 0x7f60c4158620 2024-01-24T14:14:40.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.584+0000 7f60ca02a700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f60a805f270 0x7f60a8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.584+0000 7f60ca02a700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f60a805f270 0x7f60a8061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f60b8009bc0 tx=0x7f60b8008040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:40.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.588+0000 7f60b67fc700 1 -- 172.21.15.29:0/3578054473 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f60bc0a9b00 con 0x7f60c4158620 2024-01-24T14:14:40.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.841+0000 7f60cc28e700 1 -- 172.21.15.29:0/3578054473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f60c40aa150 con 0x7f60c4158620 2024-01-24T14:14:40.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.844+0000 7f60b67fc700 1 -- 172.21.15.29:0/3578054473 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7f60bc04bce0 con 0x7f60c4158620 2024-01-24T14:14:40.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.846+0000 7f60cc28e700 1 -- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f60a805f270 msgr2=0x7f60a8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:40.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.846+0000 7f60cc28e700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f60a805f270 0x7f60a8061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f60b8009bc0 tx=0x7f60b8008040).stop 2024-01-24T14:14:40.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.846+0000 7f60cc28e700 1 -- 172.21.15.29:0/3578054473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60c4158620 msgr2=0x7f60c40a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:40.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.846+0000 7f60cc28e700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60c4158620 0x7f60c40a5850 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f60bc009df0 tx=0x7f60bc00b040).stop 2024-01-24T14:14:40.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.846+0000 7f60cc28e700 1 -- 172.21.15.29:0/3578054473 shutdown_connections 2024-01-24T14:14:40.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.846+0000 7f60cc28e700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f60a805f270 0x7f60a8061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.846+0000 7f60cc28e700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60c4158620 0x7f60c40a5850 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.846+0000 7f60cc28e700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60c414e6a0 0x7f60c414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.846+0000 7f60cc28e700 1 --2- 172.21.15.29:0/3578054473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60c414ea80 0x7f60c40a8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.846+0000 7f60cc28e700 1 -- 172.21.15.29:0/3578054473 >> 172.21.15.29:0/3578054473 conn(0x7f60c40c7bc0 msgr2=0x7f60c414d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:40.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.847+0000 7f60cc28e700 1 -- 172.21.15.29:0/3578054473 shutdown_connections 2024-01-24T14:14:40.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.847+0000 7f60cc28e700 1 -- 172.21.15.29:0/3578054473 wait complete. 2024-01-24T14:14:40.848 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-01-24T14:14:40.860 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:14:40.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.990+0000 7fdda94ee700 1 Processor -- start 2024-01-24T14:14:40.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.990+0000 7fdda94ee700 1 -- start start 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.991+0000 7fdda94ee700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdda4066c00 0x7fdda40bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.991+0000 7fdda94ee700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda41582b0 con 0x7fdda40670a0 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.991+0000 7fdda94ee700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda4157ca0 con 0x7fdda40c4a10 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.991+0000 7fdda94ee700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda4063c60 con 0x7fdda4066c00 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.992+0000 7fdda27fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdda4066c00 0x7fdda40bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.992+0000 7fdda37fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdda40670a0 0x7fdda40bdb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:47004/0 (socket says 172.21.15.29:47004) 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.992+0000 7fdda37fe700 1 -- 172.21.15.29:0/2102248509 learned_addr learned my addr 172.21.15.29:0/2102248509 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.992+0000 7fdda27fc700 1 -- 172.21.15.29:0/2102248509 >> v1:172.21.15.163:6789/0 conn(0x7fdda40c4a10 legacy=0x7fdda40664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.992+0000 7fdda27fc700 1 -- 172.21.15.29:0/2102248509 >> v1:172.21.15.29:6789/0 conn(0x7fdda40670a0 legacy=0x7fdda40bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.992+0000 7fdda1ffb700 1 -- 172.21.15.29:0/2102248509 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596080359 0 0) 0x7fdda41582b0 con 0x7fdda40670a0 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.992+0000 7fdda27fc700 1 -- 172.21.15.29:0/2102248509 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda4157f40 con 0x7fdda4066c00 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.992+0000 7fdda27fc700 1 --2- 172.21.15.29:0/2102248509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda4066c00 0x7fdda40bf6d0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7fdd98009760 tx=0x7fdd98009060).ready entity=mon.0 client_cookie=f7feea4f219a3a00 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:40.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.992+0000 7fdda1ffb700 1 -- 172.21.15.29:0/2102248509 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9800b520 con 0x7fdda4066c00 2024-01-24T14:14:40.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.992+0000 7fdda1ffb700 1 -- 172.21.15.29:0/2102248509 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9800b680 con 0x7fdda4066c00 2024-01-24T14:14:40.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.992+0000 7fdda1ffb700 1 -- 172.21.15.29:0/2102248509 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9800ba00 con 0x7fdda4066c00 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.993+0000 7fdda94ee700 1 -- 172.21.15.29:0/2102248509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda4066c00 msgr2=0x7fdda40bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.993+0000 7fdda94ee700 1 --2- 172.21.15.29:0/2102248509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda4066c00 0x7fdda40bf6d0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7fdd98009760 tx=0x7fdd98009060).stop 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.993+0000 7fdda94ee700 1 -- 172.21.15.29:0/2102248509 shutdown_connections 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.993+0000 7fdda94ee700 1 --2- 172.21.15.29:0/2102248509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda4066c00 0x7fdda40bf6d0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.993+0000 7fdda94ee700 1 -- 172.21.15.29:0/2102248509 >> 172.21.15.29:0/2102248509 conn(0x7fdda40c8bb0 msgr2=0x7fdda40c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.993+0000 7fdda94ee700 1 -- 172.21.15.29:0/2102248509 shutdown_connections 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.993+0000 7fdda94ee700 1 -- 172.21.15.29:0/2102248509 wait complete. 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.994+0000 7fdda94ee700 1 Processor -- start 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.994+0000 7fdda94ee700 1 -- start start 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.995+0000 7fdda94ee700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4066c00 0x7fdda4164480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.995+0000 7fdda94ee700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdda40670a0 0x7fdda4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.995+0000 7fdda94ee700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda40c4a10 0x7fdda40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:40.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.995+0000 7fdda94ee700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda41549d0 con 0x7fdda40c4a10 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.995+0000 7fdda94ee700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda4154cb0 con 0x7fdda40670a0 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.995+0000 7fdda94ee700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda4154f90 con 0x7fdda4066c00 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.995+0000 7fdda37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda40c4a10 0x7fdda40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.995+0000 7fdda37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda40c4a10 0x7fdda40a62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41472/0 (socket says 172.21.15.29:41472) 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.995+0000 7fdda37fe700 1 -- 172.21.15.29:0/2222732422 learned_addr learned my addr 172.21.15.29:0/2222732422 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.995+0000 7fdda2ffd700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4066c00 0x7fdda4164480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.996+0000 7fdda27fc700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdda40670a0 0x7fdda4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.996+0000 7fdda37fe700 1 -- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4066c00 msgr2=0x7fdda4164480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.996+0000 7fdda37fe700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4066c00 0x7fdda4164480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.996+0000 7fdda37fe700 1 -- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdda40670a0 msgr2=0x7fdda4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.996+0000 7fdda37fe700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdda40670a0 0x7fdda4155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.996+0000 7fdda37fe700 1 -- 172.21.15.29:0/2222732422 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd980092a0 con 0x7fdda40c4a10 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.996+0000 7fdda37fe700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda40c4a10 0x7fdda40a62f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7fdd9c00c7e0 tx=0x7fdd9c007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:41.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.996+0000 7fdd8affd700 1 -- 172.21.15.29:0/2222732422 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c005d30 con 0x7fdda40c4a10 2024-01-24T14:14:41.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.996+0000 7fdd8affd700 1 -- 172.21.15.29:0/2222732422 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9c005e90 con 0x7fdda40c4a10 2024-01-24T14:14:41.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.996+0000 7fdda94ee700 1 -- 172.21.15.29:0/2222732422 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda4155270 con 0x7fdda40c4a10 2024-01-24T14:14:41.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.996+0000 7fdd8affd700 1 -- 172.21.15.29:0/2222732422 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c007320 con 0x7fdda40c4a10 2024-01-24T14:14:41.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.996+0000 7fdda94ee700 1 -- 172.21.15.29:0/2222732422 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda4164970 con 0x7fdda40c4a10 2024-01-24T14:14:41.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.997+0000 7fdda94ee700 1 -- 172.21.15.29:0/2222732422 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda4167b40 con 0x7fdda40c4a10 2024-01-24T14:14:41.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.998+0000 7fdd8affd700 1 -- 172.21.15.29:0/2222732422 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdd9c007710 con 0x7fdda40c4a10 2024-01-24T14:14:41.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.998+0000 7fdd8affd700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdd9005f240 0x7fdd90061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.998+0000 7fdd8affd700 1 -- 172.21.15.29:0/2222732422 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fdd9c07a860 con 0x7fdda40c4a10 2024-01-24T14:14:41.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.998+0000 7fdda2ffd700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdd9005f240 0x7fdd90061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:40.999+0000 7fdda2ffd700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdd9005f240 0x7fdd90061700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fdda40c4450 tx=0x7fdd94009040).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:41.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.003+0000 7fdd8affd700 1 -- 172.21.15.29:0/2222732422 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdda4167b40 con 0x7fdda40c4a10 2024-01-24T14:14:41.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.228+0000 7fdda94ee700 1 -- 172.21.15.29:0/2222732422 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdda4167b40 con 0x7fdda40c4a10 2024-01-24T14:14:41.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.229+0000 7fdd8affd700 1 -- 172.21.15.29:0/2222732422 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fdda4167b40 con 0x7fdda40c4a10 2024-01-24T14:14:41.230 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:41.230 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6341,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4230829295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4230829295},{"type":"v1","addr":"172.21.15.29:6835","nonce":4230829295}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6344,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3194389571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3194389571},{"type":"v1","addr":"172.21.15.163:6836","nonce":3194389571}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2473497045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2473497045},{"type":"v1","addr":"172.21.15.163:6837","nonce":2473497045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:14:35.870228+0000","modified":"2024-01-24T14:14:38.905491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7254":{"gid":7254,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/3097015540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3097015540},{"type":"v1","addr":"172.21.15.29:6834","nonce":3097015540}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-24T14:14:41.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.231+0000 7fdda94ee700 1 -- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdd9005f240 msgr2=0x7fdd90061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.231+0000 7fdda94ee700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdd9005f240 0x7fdd90061700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fdda40c4450 tx=0x7fdd94009040).stop 2024-01-24T14:14:41.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.231+0000 7fdda94ee700 1 -- 172.21.15.29:0/2222732422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda40c4a10 msgr2=0x7fdda40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.231+0000 7fdda94ee700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda40c4a10 0x7fdda40a62f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7fdd9c00c7e0 tx=0x7fdd9c007f90).stop 2024-01-24T14:14:41.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.231+0000 7fdda94ee700 1 -- 172.21.15.29:0/2222732422 shutdown_connections 2024-01-24T14:14:41.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.231+0000 7fdda94ee700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdd9005f240 0x7fdd90061700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.231+0000 7fdda94ee700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdda40c4a10 0x7fdda40a62f0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.231+0000 7fdda94ee700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4066c00 0x7fdda4164480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.231+0000 7fdda94ee700 1 --2- 172.21.15.29:0/2222732422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdda40670a0 0x7fdda4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.231+0000 7fdda94ee700 1 -- 172.21.15.29:0/2222732422 >> 172.21.15.29:0/2222732422 conn(0x7fdda40c8bb0 msgr2=0x7fdda40c73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:41.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.231+0000 7fdda94ee700 1 -- 172.21.15.29:0/2222732422 shutdown_connections 2024-01-24T14:14:41.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.231+0000 7fdda94ee700 1 -- 172.21.15.29:0/2222732422 wait complete. 2024-01-24T14:14:41.233 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 51 2024-01-24T14:14:41.399 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-01-24T14:14:41.399+0000 7f6161034700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T14:14:41.648 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T14:14:35.870228+0000', 'modified': '2024-01-24T14:14:38.905491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7254}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7254': {'gid': 7254, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6834/3097015540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 3097015540}, {'type': 'v1', 'addr': '172.21.15.29:6834', 'nonce': 3097015540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-24T14:14:41.648 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T14:14:41.648 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-24T14:14:41.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.789+0000 7fad9e769700 1 Processor -- start 2024-01-24T14:14:41.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.789+0000 7fad9e769700 1 -- start start 2024-01-24T14:14:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.790+0000 7fad9e769700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fad9813de40 0x7fad980bd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.790+0000 7fad9e769700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fad9813e220 0x7fad980639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.790+0000 7fad9e769700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad98063540 con 0x7fad98063ec0 2024-01-24T14:14:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.790+0000 7fad9e769700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad98062930 con 0x7fad9813de40 2024-01-24T14:14:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.790+0000 7fad9e769700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad98062c10 con 0x7fad9813e220 2024-01-24T14:14:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.790+0000 7fad97fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fad9813de40 0x7fad980bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.790+0000 7fad9cd06700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fad98063ec0 0x7fad980c1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:47008/0 (socket says 172.21.15.29:47008) 2024-01-24T14:14:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.790+0000 7fad9cd06700 1 -- 172.21.15.29:0/2614538692 learned_addr learned my addr 172.21.15.29:0/2614538692 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:41.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.790+0000 7fad977fe700 1 --2- 172.21.15.29:0/2614538692 >> v2:172.21.15.163:3300/0 conn(0x7fad9813e220 0x7fad980639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad97fff700 1 -- 172.21.15.29:0/2614538692 >> v2:172.21.15.163:3300/0 conn(0x7fad9813e220 msgr2=0x7fad980639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad97fff700 1 --2- 172.21.15.29:0/2614538692 >> v2:172.21.15.163:3300/0 conn(0x7fad9813e220 0x7fad980639d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad97fff700 1 -- 172.21.15.29:0/2614538692 >> v1:172.21.15.29:6789/0 conn(0x7fad98063ec0 legacy=0x7fad980c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad97fff700 1 -- 172.21.15.29:0/2614538692 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad98062ef0 con 0x7fad9813de40 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad96ffd700 1 -- 172.21.15.29:0/2614538692 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205646850 0 0) 0x7fad98063540 con 0x7fad98063ec0 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad97fff700 1 --2- 172.21.15.29:0/2614538692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad9813de40 0x7fad980bd060 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fad980688a0 tx=0x7fad88009060).ready entity=mon.0 client_cookie=445d907623f970d5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad96ffd700 1 -- 172.21.15.29:0/2614538692 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8800f510 con 0x7fad9813de40 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad96ffd700 1 -- 172.21.15.29:0/2614538692 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8800f670 con 0x7fad9813de40 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad96ffd700 1 -- 172.21.15.29:0/2614538692 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8800f9f0 con 0x7fad9813de40 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad9e769700 1 -- 172.21.15.29:0/2614538692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad9813de40 msgr2=0x7fad980bd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad9e769700 1 --2- 172.21.15.29:0/2614538692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad9813de40 0x7fad980bd060 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fad980688a0 tx=0x7fad88009060).stop 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad9e769700 1 -- 172.21.15.29:0/2614538692 shutdown_connections 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad9e769700 1 --2- 172.21.15.29:0/2614538692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad9813de40 0x7fad980bd060 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad9e769700 1 --2- 172.21.15.29:0/2614538692 >> v2:172.21.15.163:3300/0 conn(0x7fad9813e220 0x7fad980639d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.791+0000 7fad9e769700 1 -- 172.21.15.29:0/2614538692 >> 172.21.15.29:0/2614538692 conn(0x7fad980c3cd0 msgr2=0x7fad980c40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.792+0000 7fad9e769700 1 -- 172.21.15.29:0/2614538692 shutdown_connections 2024-01-24T14:14:41.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.792+0000 7fad9e769700 1 -- 172.21.15.29:0/2614538692 wait complete. 2024-01-24T14:14:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.793+0000 7fad9e769700 1 Processor -- start 2024-01-24T14:14:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.793+0000 7fad9e769700 1 -- start start 2024-01-24T14:14:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9e769700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad98063ec0 0x7fad98158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9e769700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad9813de40 0x7fad980c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9e769700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad9813e220 0x7fad980abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9e769700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad981582b0 con 0x7fad9813e220 2024-01-24T14:14:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9e769700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad981576a0 con 0x7fad9813de40 2024-01-24T14:14:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9e769700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad98157980 con 0x7fad98063ec0 2024-01-24T14:14:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9cd06700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad9813e220 0x7fad980abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9cd06700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad9813e220 0x7fad980abe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41488/0 (socket says 172.21.15.29:41488) 2024-01-24T14:14:41.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9cd06700 1 -- 172.21.15.29:0/2541819134 learned_addr learned my addr 172.21.15.29:0/2541819134 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad97fff700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad98063ec0 0x7fad98158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad977fe700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad9813de40 0x7fad980c1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9cd06700 1 -- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad98063ec0 msgr2=0x7fad98158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9cd06700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad98063ec0 0x7fad98158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9cd06700 1 -- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad9813de40 msgr2=0x7fad980c1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9cd06700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad9813de40 0x7fad980c1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9cd06700 1 -- 172.21.15.29:0/2541819134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad880092a0 con 0x7fad9813e220 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.794+0000 7fad9cd06700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad9813e220 0x7fad980abe50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7fad9000c7e0 tx=0x7fad90007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.795+0000 7fad8ffff700 1 -- 172.21.15.29:0/2541819134 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90005d30 con 0x7fad9813e220 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.795+0000 7fad8ffff700 1 -- 172.21.15.29:0/2541819134 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad90005e90 con 0x7fad9813e220 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.795+0000 7fad9e769700 1 -- 172.21.15.29:0/2541819134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad98157c60 con 0x7fad9813e220 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.795+0000 7fad8ffff700 1 -- 172.21.15.29:0/2541819134 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90007320 con 0x7fad9813e220 2024-01-24T14:14:41.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.795+0000 7fad9e769700 1 -- 172.21.15.29:0/2541819134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad98158c30 con 0x7fad9813e220 2024-01-24T14:14:41.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.796+0000 7fad8ffff700 1 -- 172.21.15.29:0/2541819134 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fad90007710 con 0x7fad9813e220 2024-01-24T14:14:41.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.797+0000 7fad8ffff700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad7805f240 0x7fad78061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.797+0000 7fad8ffff700 1 -- 172.21.15.29:0/2541819134 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fad9007a960 con 0x7fad9813e220 2024-01-24T14:14:41.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.797+0000 7fad97fff700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad7805f240 0x7fad78061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.797+0000 7fad97fff700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad7805f240 0x7fad78061700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fad8800f640 tx=0x7fad88017000).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.799+0000 7fadb0134b80 1 Processor -- start 2024-01-24T14:14:41.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.799+0000 7fadb0134b80 1 -- start start 2024-01-24T14:14:41.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fadb0134b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 0x560e56c69a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fadb0134b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56cb22c0 0x560e56c64900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fadb0134b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c64df0 0x560e56c65270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fadb0134b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e56be03d0 con 0x560e56c69620 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fadb0134b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e56ba5be0 con 0x560e56cb22c0 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fadb0134b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e56bbb260 con 0x560e56c64df0 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fad97fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 0x560e56c69a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fad97fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 0x560e56c69a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41492/0 (socket says 172.21.15.29:41492) 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fad97fff700 1 -- 172.21.15.29:0/2976101783 learned_addr learned my addr 172.21.15.29:0/2976101783 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fad9cd06700 1 --2- 172.21.15.29:0/2976101783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c64df0 0x560e56c65270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fad977fe700 1 --2- 172.21.15.29:0/2976101783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56cb22c0 0x560e56c64900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fad97fff700 1 -- 172.21.15.29:0/2976101783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c64df0 msgr2=0x560e56c65270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.800+0000 7fad97fff700 1 --2- 172.21.15.29:0/2976101783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c64df0 0x560e56c65270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.801+0000 7fad97fff700 1 -- 172.21.15.29:0/2976101783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56cb22c0 msgr2=0x560e56c64900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.801+0000 7fad97fff700 1 --2- 172.21.15.29:0/2976101783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56cb22c0 0x560e56c64900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.801+0000 7fad97fff700 1 -- 172.21.15.29:0/2976101783 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e56bb42f0 con 0x560e56c69620 2024-01-24T14:14:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.801+0000 7fad97fff700 1 --2- 172.21.15.29:0/2976101783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 0x560e56c69a00 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7fad88009dd0 tx=0x7fad8801d040).ready entity=mon.0 client_cookie=27e72e9655aae596 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.801+0000 7fad8d7fa700 1 -- 172.21.15.29:0/2976101783 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8801b8a0 con 0x560e56c69620 2024-01-24T14:14:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.801+0000 7fad8d7fa700 1 -- 172.21.15.29:0/2976101783 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8801ba00 con 0x560e56c69620 2024-01-24T14:14:41.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.801+0000 7fad8d7fa700 1 -- 172.21.15.29:0/2976101783 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8801bd80 con 0x560e56c69620 2024-01-24T14:14:41.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.802+0000 7fadb0134b80 1 -- 172.21.15.29:0/2976101783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 msgr2=0x560e56c69a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.802+0000 7fadb0134b80 1 --2- 172.21.15.29:0/2976101783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 0x560e56c69a00 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7fad88009dd0 tx=0x7fad8801d040).stop 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.802+0000 7fadb0134b80 1 -- 172.21.15.29:0/2976101783 shutdown_connections 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.802+0000 7fadb0134b80 1 --2- 172.21.15.29:0/2976101783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c64df0 0x560e56c65270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.802+0000 7fadb0134b80 1 --2- 172.21.15.29:0/2976101783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 0x560e56c69a00 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.802+0000 7fadb0134b80 1 --2- 172.21.15.29:0/2976101783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56cb22c0 0x560e56c64900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.802+0000 7fadb0134b80 1 -- 172.21.15.29:0/2976101783 >> 172.21.15.29:0/2976101783 conn(0x560e56ad22e0 msgr2=0x560e56cb2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.802+0000 7fadb0134b80 1 -- 172.21.15.29:0/2976101783 shutdown_connections 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.802+0000 7fadb0134b80 1 -- 172.21.15.29:0/2976101783 wait complete. 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.802+0000 7fadb0134b80 1 -- start start 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.803+0000 7fadb0134b80 1 Processor -- start 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.803+0000 7fadb0134b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56c64df0 0x560e56c91420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.803+0000 7fadb0134b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 0x560e56c94580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.803+0000 7fadb0134b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56cb22c0 0x560e56cb4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.803+0000 7fadb0134b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e56bbb3a0 con 0x560e56c69620 2024-01-24T14:14:41.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.803+0000 7fadb0134b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e56bbb620 con 0x560e56c64df0 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.803+0000 7fadb0134b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e56b86990 con 0x560e56cb22c0 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.803+0000 7fadb0134b80 10 client.0 ll_register_callbacks cb 0x560e56c29610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.803+0000 7fad977fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 0x560e56c94580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad977fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 0x560e56c94580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41508/0 (socket says 172.21.15.29:41508) 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad977fe700 1 -- 172.21.15.29:0/1302462948 learned_addr learned my addr 172.21.15.29:0/1302462948 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad9cd06700 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56cb22c0 0x560e56cb4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad97fff700 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56c64df0 0x560e56c91420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad977fe700 1 -- 172.21.15.29:0/1302462948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56cb22c0 msgr2=0x560e56cb4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad977fe700 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56cb22c0 0x560e56cb4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad977fe700 1 -- 172.21.15.29:0/1302462948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56c64df0 msgr2=0x560e56c91420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad977fe700 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56c64df0 0x560e56c91420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad977fe700 1 -- 172.21.15.29:0/1302462948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad98062ef0 con 0x560e56c69620 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad97fff700 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56c64df0 0x560e56c91420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:41.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad977fe700 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 0x560e56c94580 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7fad80014450 tx=0x7fad80008040).ready entity=mon.0 client_cookie=bd138b60a7fbbc2d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad767fc700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad767fc700 1 -- 172.21.15.29:0/1302462948 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80014ee0 con 0x560e56c69620 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad767fc700 1 -- 172.21.15.29:0/1302462948 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fad80016640 con 0x560e56c69620 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fadb0134b80 1 -- 172.21.15.29:0/1302462948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e56c63df0 con 0x560e56c69620 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fad767fc700 1 -- 172.21.15.29:0/1302462948 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad80016a50 con 0x560e56c69620 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fadb0134b80 10 client.7407 fetch_fsmap learned FSMap version 51 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.804+0000 7fadb0134b80 1 -- 172.21.15.29:0/1302462948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x560e56baf520 con 0x560e56c69620 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.805+0000 7fad767fc700 1 -- 172.21.15.29:0/1302462948 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560e56c63df0 con 0x560e56c69620 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.805+0000 7fad767fc700 1 -- 172.21.15.29:0/1302462948 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 51) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fad80016bb0 con 0x560e56c69620 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.805+0000 7fadb0134b80 10 client.7407 fetch_fsmap finished waiting for FSMap version 51 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.805+0000 7fadb0134b80 10 client.7407 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.805+0000 7fadb0134b80 20 client.7407 populate_metadata read hostname 'smithi029' 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.805+0000 7fadb0134b80 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.29:6833/3097015540,v1:172.21.15.29:6834/3097015540] conn(0x560e56c96880 0x560e56ccb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.805+0000 7fadb0134b80 4 client.7407 mds_command: new command op to 7254 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T14:14:41.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.805+0000 7fadb0134b80 1 -- 172.21.15.29:0/1302462948 --> [v2:172.21.15.29:6833/3097015540,v1:172.21.15.29:6834/3097015540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e56bb0d60 con 0x560e56c96880 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.805+0000 7fad97fff700 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.29:6833/3097015540,v1:172.21.15.29:6834/3097015540] conn(0x560e56c96880 0x560e56ccb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.805+0000 7fad767fc700 10 client.7407 ms_handle_connect on v2:172.21.15.29:6833/3097015540 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.805+0000 7fad97fff700 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.29:6833/3097015540,v1:172.21.15.29:6834/3097015540] conn(0x560e56c96880 0x560e56ccb9b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=725b9753eceb77d4 server_cookie=fa7b4d4f003798e7 in_seq=0 out_seq=0 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fad767fc700 1 -- 172.21.15.29:0/1302462948 <== mds.0 v2:172.21.15.29:6833/3097015540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x560e56bb0d60 con 0x560e56c96880 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fad767fc700 10 client.7407 handle_command_reply: tid=0 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fadb0134b80 1 client.7407 shutdown 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fadb0134b80 1 -- 172.21.15.29:0/1302462948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 msgr2=0x560e56c94580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fadb0134b80 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 0x560e56c94580 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7fad80014450 tx=0x7fad80008040).stop 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fadb0134b80 1 -- 172.21.15.29:0/1302462948 shutdown_connections 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fadb0134b80 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56cb22c0 0x560e56cb4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fadb0134b80 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56c64df0 0x560e56c91420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fadb0134b80 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.29:6833/3097015540,v1:172.21.15.29:6834/3097015540] conn(0x560e56c96880 0x560e56ccb9b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fadb0134b80 1 --2- 172.21.15.29:0/1302462948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c69620 0x560e56c94580 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fadb0134b80 1 -- 172.21.15.29:0/1302462948 >> 172.21.15.29:0/1302462948 conn(0x560e56ad22e0 msgr2=0x560e56c67690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:41.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fad767fc700 0 client.7407 ms_handle_reset on v2:172.21.15.29:6833/3097015540 2024-01-24T14:14:41.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fadb0134b80 1 -- 172.21.15.29:0/1302462948 shutdown_connections 2024-01-24T14:14:41.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fadb0134b80 1 -- 172.21.15.29:0/1302462948 wait complete. 2024-01-24T14:14:41.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.806+0000 7fadb0134b80 20 client.7407 trim_cache size 0 max 16384 2024-01-24T14:14:41.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.824+0000 7fadb0134b80 1 Processor -- start 2024-01-24T14:14:41.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.824+0000 7fadb0134b80 1 -- start start 2024-01-24T14:14:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fadb0134b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c64df0 0x560e56c65f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fadb0134b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56c69620 0x560e56c67510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fadb0134b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56cb22c0 0x560e56c67a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fadb0134b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e56c96fc0 con 0x560e56cb22c0 2024-01-24T14:14:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fadb0134b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e56c6c2b0 con 0x560e56c69620 2024-01-24T14:14:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fadb0134b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e56c95260 con 0x560e56c64df0 2024-01-24T14:14:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fad9cd06700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56cb22c0 0x560e56c67a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fad9cd06700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56cb22c0 0x560e56c67a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41520/0 (socket says 172.21.15.29:41520) 2024-01-24T14:14:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fad9cd06700 1 -- 172.21.15.29:0/2060343421 learned_addr learned my addr 172.21.15.29:0/2060343421 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fad97fff700 1 --2- 172.21.15.29:0/2060343421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c64df0 0x560e56c65f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fad977fe700 1 --2- 172.21.15.29:0/2060343421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56c69620 0x560e56c67510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fad9cd06700 1 -- 172.21.15.29:0/2060343421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c64df0 msgr2=0x560e56c65f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.825+0000 7fad9cd06700 1 --2- 172.21.15.29:0/2060343421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c64df0 0x560e56c65f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.826+0000 7fad9cd06700 1 -- 172.21.15.29:0/2060343421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56c69620 msgr2=0x560e56c67510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.826+0000 7fad9cd06700 1 --2- 172.21.15.29:0/2060343421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56c69620 0x560e56c67510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.826+0000 7fad9cd06700 1 -- 172.21.15.29:0/2060343421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e56baf270 con 0x560e56cb22c0 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.826+0000 7fad97fff700 1 --2- 172.21.15.29:0/2060343421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c64df0 0x560e56c65f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.826+0000 7fad9cd06700 1 --2- 172.21.15.29:0/2060343421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56cb22c0 0x560e56c67a00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7fad900788b0 tx=0x7fad90056040).ready entity=mon.0 client_cookie=ce612412dd8d8aad server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.826+0000 7fad8dffb700 1 -- 172.21.15.29:0/2060343421 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90078930 con 0x560e56cb22c0 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.826+0000 7fad8dffb700 1 -- 172.21.15.29:0/2060343421 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9004d650 con 0x560e56cb22c0 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.826+0000 7fad8dffb700 1 -- 172.21.15.29:0/2060343421 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9004b9c0 con 0x560e56cb22c0 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.826+0000 7fadb0134b80 1 -- 172.21.15.29:0/2060343421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56cb22c0 msgr2=0x560e56c67a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.826+0000 7fadb0134b80 1 --2- 172.21.15.29:0/2060343421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56cb22c0 0x560e56c67a00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7fad900788b0 tx=0x7fad90056040).stop 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.827+0000 7fadb0134b80 1 -- 172.21.15.29:0/2060343421 shutdown_connections 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.827+0000 7fadb0134b80 1 --2- 172.21.15.29:0/2060343421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56cb22c0 0x560e56c67a00 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.827+0000 7fadb0134b80 1 --2- 172.21.15.29:0/2060343421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c64df0 0x560e56c65f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.827+0000 7fadb0134b80 1 --2- 172.21.15.29:0/2060343421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56c69620 0x560e56c67510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.827+0000 7fadb0134b80 1 -- 172.21.15.29:0/2060343421 >> 172.21.15.29:0/2060343421 conn(0x560e56ad22e0 msgr2=0x560e56cb2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:41.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.827+0000 7fadb0134b80 1 -- 172.21.15.29:0/2060343421 shutdown_connections 2024-01-24T14:14:41.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.827+0000 7fadb0134b80 1 -- 172.21.15.29:0/2060343421 wait complete. 2024-01-24T14:14:41.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.827+0000 7fadb0134b80 1 -- start start 2024-01-24T14:14:41.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.827+0000 7fadb0134b80 1 Processor -- start 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.828+0000 7fadb0134b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c64df0 0x560e56c66760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.828+0000 7fadb0134b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c69620 0x560e56d13730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.828+0000 7fadb0134b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56cb22c0 0x560e56c60020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.828+0000 7fadb0134b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e56d153b0 con 0x560e56c64df0 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.828+0000 7fadb0134b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e56cc9b30 con 0x560e56cb22c0 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.828+0000 7fadb0134b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e56c69ea0 con 0x560e56c69620 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.828+0000 7fadb0134b80 10 client.0 ll_register_callbacks cb 0x560e56c29610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.828+0000 7fad97fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c64df0 0x560e56c66760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.828+0000 7fad97fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c64df0 0x560e56c66760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41522/0 (socket says 172.21.15.29:41522) 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.828+0000 7fad97fff700 1 -- 172.21.15.29:0/3977916662 learned_addr learned my addr 172.21.15.29:0/3977916662 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.828+0000 7fad977fe700 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c69620 0x560e56d13730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.828+0000 7fad9cd06700 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56cb22c0 0x560e56c60020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fad97fff700 1 -- 172.21.15.29:0/3977916662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c69620 msgr2=0x560e56d13730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:41.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fad97fff700 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c69620 0x560e56d13730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fad97fff700 1 -- 172.21.15.29:0/3977916662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56cb22c0 msgr2=0x560e56c60020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fad97fff700 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56cb22c0 0x560e56c60020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fad97fff700 1 -- 172.21.15.29:0/3977916662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad98158c30 con 0x560e56c64df0 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fad97fff700 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c64df0 0x560e56c66760 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x560e56cb2720 tx=0x7fad8801d040).ready entity=mon.0 client_cookie=2c34dcca544422ae server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fad767fc700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fad767fc700 1 -- 172.21.15.29:0/3977916662 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad88034040 con 0x560e56c64df0 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fad767fc700 1 -- 172.21.15.29:0/3977916662 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fad8800fb40 con 0x560e56c64df0 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fadb0134b80 1 -- 172.21.15.29:0/3977916662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad8801ba00 con 0x560e56c64df0 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fad767fc700 1 -- 172.21.15.29:0/3977916662 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad8801bd80 con 0x560e56c64df0 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fadb0134b80 10 client.7413 fetch_fsmap learned FSMap version 51 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fadb0134b80 1 -- 172.21.15.29:0/3977916662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x560e56a28f80 con 0x560e56c64df0 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fad767fc700 1 -- 172.21.15.29:0/3977916662 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8801ba00 con 0x560e56c64df0 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fad767fc700 1 -- 172.21.15.29:0/3977916662 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 51) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fad88037070 con 0x560e56c64df0 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fadb0134b80 10 client.7413 fetch_fsmap finished waiting for FSMap version 51 2024-01-24T14:14:41.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fadb0134b80 10 client.7413 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.829+0000 7fadb0134b80 20 client.7413 populate_metadata read hostname 'smithi029' 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.29:6833/3097015540,v1:172.21.15.29:6834/3097015540] conn(0x560e56d13ff0 0x560e56d143d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 4 client.7413 mds_command: new command op to 7254 tid=0[{"prefix": "status"}] 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 1 -- 172.21.15.29:0/3977916662 --> [v2:172.21.15.29:6833/3097015540,v1:172.21.15.29:6834/3097015540] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560e56ba77d0 con 0x560e56d13ff0 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fad977fe700 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.29:6833/3097015540,v1:172.21.15.29:6834/3097015540] conn(0x560e56d13ff0 0x560e56d143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fad767fc700 10 client.7413 ms_handle_connect on v2:172.21.15.29:6833/3097015540 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fad977fe700 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.29:6833/3097015540,v1:172.21.15.29:6834/3097015540] conn(0x560e56d13ff0 0x560e56d143d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=95bd4f792866491d server_cookie=1f24bb274c63b184 in_seq=0 out_seq=0 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fad767fc700 1 -- 172.21.15.29:0/3977916662 <== mds.0 v2:172.21.15.29:6833/3097015540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x560e56ba77d0 con 0x560e56d13ff0 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fad767fc700 10 client.7413 handle_command_reply: tid=0 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 1 client.7413 shutdown 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 1 -- 172.21.15.29:0/3977916662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c64df0 msgr2=0x560e56c66760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c64df0 0x560e56c66760 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x560e56cb2720 tx=0x7fad8801d040).stop 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 1 -- 172.21.15.29:0/3977916662 shutdown_connections 2024-01-24T14:14:41.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e56cb22c0 0x560e56c60020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.29:6833/3097015540,v1:172.21.15.29:6834/3097015540] conn(0x560e56d13ff0 0x560e56d143d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e56c64df0 0x560e56c66760 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 1 --2- 172.21.15.29:0/3977916662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560e56c69620 0x560e56d13730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fad767fc700 0 client.7413 ms_handle_reset on v2:172.21.15.29:6833/3097015540 2024-01-24T14:14:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 1 -- 172.21.15.29:0/3977916662 >> 172.21.15.29:0/3977916662 conn(0x560e56ad22e0 msgr2=0x560e56c91250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.830+0000 7fadb0134b80 1 -- 172.21.15.29:0/3977916662 shutdown_connections 2024-01-24T14:14:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fadb0134b80 1 -- 172.21.15.29:0/3977916662 wait complete. 2024-01-24T14:14:41.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fadb0134b80 20 client.7413 trim_cache size 0 max 16384 2024-01-24T14:14:41.837 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T14:14:41.837 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T14:14:41.837 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T14:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 7254, 2024-01-24T14:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T14:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T14:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T14:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 5.95968126, 2024-01-24T14:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 51, 2024-01-24T14:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 120, 2024-01-24T14:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 120, 2024-01-24T14:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 11.438693383 2024-01-24T14:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T14:14:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fad9e769700 1 -- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad7805f240 msgr2=0x7fad78061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fad9e769700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad7805f240 0x7fad78061700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fad8800f640 tx=0x7fad88017000).stop 2024-01-24T14:14:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fad9e769700 1 -- 172.21.15.29:0/2541819134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad9813e220 msgr2=0x7fad980abe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fad9e769700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad9813e220 0x7fad980abe50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7fad9000c7e0 tx=0x7fad90007f90).stop 2024-01-24T14:14:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fad9e769700 1 -- 172.21.15.29:0/2541819134 shutdown_connections 2024-01-24T14:14:41.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fad9e769700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fad7805f240 0x7fad78061700 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fad9e769700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fad9813e220 0x7fad980abe50 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fad9e769700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad98063ec0 0x7fad98158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fad9e769700 1 --2- 172.21.15.29:0/2541819134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad9813de40 0x7fad980c1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fad9e769700 1 -- 172.21.15.29:0/2541819134 >> 172.21.15.29:0/2541819134 conn(0x7fad980c3cd0 msgr2=0x7fad98069700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:41.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fad9e769700 1 -- 172.21.15.29:0/2541819134 shutdown_connections 2024-01-24T14:14:41.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.831+0000 7fad9e769700 1 -- 172.21.15.29:0/2541819134 wait complete. 2024-01-24T14:14:41.845 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T14:14:41.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.961+0000 7f7bbb91f700 1 Processor -- start 2024-01-24T14:14:41.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.961+0000 7f7bbb91f700 1 -- start start 2024-01-24T14:14:41.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.962+0000 7f7bbb91f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7bb41520e0 0x7f7bb40a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.962+0000 7f7bbb91f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7bb40beb30 0x7f7bb40c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.962+0000 7f7bbb91f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb40a7490 con 0x7f7bb40a8300 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.962+0000 7f7bbb91f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb40a6880 con 0x7f7bb40beb30 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.962+0000 7f7bbb91f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb40a6b60 con 0x7f7bb41520e0 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.962+0000 7f7bb9ebc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7bb40beb30 0x7f7bb40c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.963+0000 7f7bb9ebc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7bb40beb30 0x7f7bb40c6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41524/0 (socket says 172.21.15.29:41524) 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.963+0000 7f7bb9ebc700 1 -- 172.21.15.29:0/1693720902 learned_addr learned my addr 172.21.15.29:0/1693720902 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.963+0000 7f7bb96bb700 1 --2- 172.21.15.29:0/1693720902 >> v2:172.21.15.163:3300/0 conn(0x7f7bb41520e0 0x7f7bb40a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.963+0000 7f7bb9ebc700 1 -- 172.21.15.29:0/1693720902 >> v2:172.21.15.163:3300/0 conn(0x7f7bb41520e0 msgr2=0x7f7bb40a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.963+0000 7f7bb9ebc700 1 --2- 172.21.15.29:0/1693720902 >> v2:172.21.15.163:3300/0 conn(0x7f7bb41520e0 0x7f7bb40a7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.963+0000 7f7bb9ebc700 1 -- 172.21.15.29:0/1693720902 >> v1:172.21.15.163:6789/0 conn(0x7f7bb40a8300 legacy=0x7f7bb40be420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.963+0000 7f7bb9ebc700 1 -- 172.21.15.29:0/1693720902 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb40a6e40 con 0x7f7bb40beb30 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.963+0000 7f7bb9ebc700 1 --2- 172.21.15.29:0/1693720902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bb40beb30 0x7f7bb40c6b50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f7bb40691d0 tx=0x7f7bac009840).ready entity=mon.0 client_cookie=9658a2bfd0e9b143 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:41.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.963+0000 7f7ba7fff700 1 -- 172.21.15.29:0/1693720902 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00e070 con 0x7f7bb40beb30 2024-01-24T14:14:41.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.963+0000 7f7ba7fff700 1 -- 172.21.15.29:0/1693720902 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac00ba20 con 0x7f7bb40beb30 2024-01-24T14:14:41.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.963+0000 7f7ba7fff700 1 -- 172.21.15.29:0/1693720902 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00bda0 con 0x7f7bb40beb30 2024-01-24T14:14:41.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.964+0000 7f7bbb91f700 1 -- 172.21.15.29:0/1693720902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bb40beb30 msgr2=0x7f7bb40c6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.964+0000 7f7bbb91f700 1 --2- 172.21.15.29:0/1693720902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bb40beb30 0x7f7bb40c6b50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f7bb40691d0 tx=0x7f7bac009840).stop 2024-01-24T14:14:41.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.964+0000 7f7bbb91f700 1 -- 172.21.15.29:0/1693720902 shutdown_connections 2024-01-24T14:14:41.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.964+0000 7f7bbb91f700 1 --2- 172.21.15.29:0/1693720902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bb40beb30 0x7f7bb40c6b50 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.964+0000 7f7bbb91f700 1 --2- 172.21.15.29:0/1693720902 >> v2:172.21.15.163:3300/0 conn(0x7f7bb41520e0 0x7f7bb40a7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.964+0000 7f7bbb91f700 1 -- 172.21.15.29:0/1693720902 >> 172.21.15.29:0/1693720902 conn(0x7f7bb40922f0 msgr2=0x7f7bb4062f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:41.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.964+0000 7f7bbb91f700 1 -- 172.21.15.29:0/1693720902 shutdown_connections 2024-01-24T14:14:41.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.964+0000 7f7bbb91f700 1 -- 172.21.15.29:0/1693720902 wait complete. 2024-01-24T14:14:41.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.965+0000 7f7bbb91f700 1 Processor -- start 2024-01-24T14:14:41.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.965+0000 7f7bbb91f700 1 -- start start 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.966+0000 7f7bbb91f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bb40a8300 0x7f7bb41506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.966+0000 7f7bbb91f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bb40beb30 0x7f7bb40abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.966+0000 7f7bbb91f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bb41520e0 0x7f7bb40ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.966+0000 7f7bbb91f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40ac5e0 con 0x7f7bb40beb30 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.966+0000 7f7bbb91f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb41633a0 con 0x7f7bb40a8300 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.966+0000 7f7bbb91f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4150bd0 con 0x7f7bb41520e0 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.966+0000 7f7bb8eba700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bb40beb30 0x7f7bb40abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.966+0000 7f7bb9ebc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bb41520e0 0x7f7bb40ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.966+0000 7f7bb96bb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bb40a8300 0x7f7bb41506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.966+0000 7f7bb8eba700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bb40beb30 0x7f7bb40abc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41532/0 (socket says 172.21.15.29:41532) 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.966+0000 7f7bb9ebc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bb41520e0 0x7f7bb40ac0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:53926/0 (socket says 172.21.15.29:53926) 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.966+0000 7f7bb9ebc700 1 -- 172.21.15.29:0/2761014975 learned_addr learned my addr 172.21.15.29:0/2761014975 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.967+0000 7f7bb9ebc700 1 -- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bb40a8300 msgr2=0x7f7bb41506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.967+0000 7f7bb9ebc700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bb40a8300 0x7f7bb41506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.967+0000 7f7bb9ebc700 1 -- 172.21.15.29:0/2761014975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bb40beb30 msgr2=0x7f7bb40abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:41.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.967+0000 7f7bb9ebc700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bb40beb30 0x7f7bb40abc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:41.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.967+0000 7f7bb9ebc700 1 -- 172.21.15.29:0/2761014975 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac009ae0 con 0x7f7bb41520e0 2024-01-24T14:14:41.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.967+0000 7f7bb8eba700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bb40beb30 0x7f7bb40abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:41.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.967+0000 7f7bb9ebc700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bb41520e0 0x7f7bb40ac0f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f7bac00b6c0 tx=0x7f7bac009840).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:41.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.967+0000 7f7ba57fa700 1 -- 172.21.15.29:0/2761014975 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0092e0 con 0x7f7bb41520e0 2024-01-24T14:14:41.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.967+0000 7f7ba57fa700 1 -- 172.21.15.29:0/2761014975 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac01e470 con 0x7f7bb41520e0 2024-01-24T14:14:41.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.967+0000 7f7bbb91f700 1 -- 172.21.15.29:0/2761014975 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb4162790 con 0x7f7bb41520e0 2024-01-24T14:14:41.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.967+0000 7f7ba57fa700 1 -- 172.21.15.29:0/2761014975 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac01e830 con 0x7f7bb41520e0 2024-01-24T14:14:41.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.967+0000 7f7bbb91f700 1 -- 172.21.15.29:0/2761014975 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb4162d50 con 0x7f7bb41520e0 2024-01-24T14:14:41.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.968+0000 7f7bbb91f700 1 -- 172.21.15.29:0/2761014975 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb405f400 con 0x7f7bb41520e0 2024-01-24T14:14:41.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.968+0000 7f7ba57fa700 1 -- 172.21.15.29:0/2761014975 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7bac003040 con 0x7f7bb41520e0 2024-01-24T14:14:41.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.969+0000 7f7ba57fa700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7b9805f270 0x7f7b98061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:41.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.969+0000 7f7ba57fa700 1 -- 172.21.15.29:0/2761014975 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f7bac07bbb0 con 0x7f7bb41520e0 2024-01-24T14:14:41.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.969+0000 7f7bb96bb700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7b9805f270 0x7f7b98061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:41.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.969+0000 7f7bb96bb700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7b9805f270 0x7f7b98061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7bb40a7760 tx=0x7f7bb0006f40).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:41.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:41.973+0000 7f7ba57fa700 1 -- 172.21.15.29:0/2761014975 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7bac04c9b0 con 0x7f7bb41520e0 2024-01-24T14:14:42.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.200+0000 7f7bbb91f700 1 -- 172.21.15.29:0/2761014975 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7bb4069e00 con 0x7f7bb41520e0 2024-01-24T14:14:42.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.200+0000 7f7ba57fa700 1 -- 172.21.15.29:0/2761014975 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7bac023070 con 0x7f7bb41520e0 2024-01-24T14:14:42.201 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-24T14:14:42.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.202+0000 7f7bbb91f700 1 -- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7b9805f270 msgr2=0x7f7b98061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:42.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.202+0000 7f7bbb91f700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7b9805f270 0x7f7b98061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7bb40a7760 tx=0x7f7bb0006f40).stop 2024-01-24T14:14:42.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.202+0000 7f7bbb91f700 1 -- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bb41520e0 msgr2=0x7f7bb40ac0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:42.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.202+0000 7f7bbb91f700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bb41520e0 0x7f7bb40ac0f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f7bac00b6c0 tx=0x7f7bac009840).stop 2024-01-24T14:14:42.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.202+0000 7f7bbb91f700 1 -- 172.21.15.29:0/2761014975 shutdown_connections 2024-01-24T14:14:42.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.202+0000 7f7bbb91f700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bb41520e0 0x7f7bb40ac0f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:42.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.202+0000 7f7bbb91f700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bb40a8300 0x7f7bb41506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:42.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.202+0000 7f7bbb91f700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7b9805f270 0x7f7b98061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:42.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.202+0000 7f7bbb91f700 1 --2- 172.21.15.29:0/2761014975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bb40beb30 0x7f7bb40abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:42.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.202+0000 7f7bbb91f700 1 -- 172.21.15.29:0/2761014975 >> 172.21.15.29:0/2761014975 conn(0x7f7bb40922f0 msgr2=0x7f7bb40c5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:42.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.203+0000 7f7bbb91f700 1 -- 172.21.15.29:0/2761014975 shutdown_connections 2024-01-24T14:14:42.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.203+0000 7f7bbb91f700 1 -- 172.21.15.29:0/2761014975 wait complete. 2024-01-24T14:14:42.215 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-24T14:14:42.215 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T14:14:42.215 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T14:14:42.215 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-24T14:14:42.216 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-24T14:14:42.216 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-24T14:14:42.216 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-24T14:14:42.216 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T14:14:42.216 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T14:14:42.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:42.216 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T14:14:42.235 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T14:14:42.236 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T14:14:42.236 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T14:14:42.236 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:42.236 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T14:14:42.236 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:42.236 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:14:42.236 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T14:14:42.236 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T14:14:42.236 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T14:14:42.236 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T14:14:42.236 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T14:14:42.236 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 41368sec preferred_lft 41368sec 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::24bc:1cff:fe57:e704/64 scope link 2024-01-24T14:14:42.237 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:42.238 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T14:14:42.238 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T14:14:42.238 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::2034:aaff:fef3:f12b/64 scope link 2024-01-24T14:14:42.238 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:42.238 INFO:teuthology.orchestra.run.smithi029.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T14:14:42.238 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether ba:ca:42:4a:9e:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T14:14:42.238 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::b8ca:42ff:fe4a:9e52/64 scope link 2024-01-24T14:14:42.238 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:14:42.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:42.238 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T14:14:42.307 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T14:14:42.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:42.309 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T14:14:42.372 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T14:14:42.372 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T14:14:42.373 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T14:14:42.373 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T14:14:42.373 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T14:14:42.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:42.373 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T14:14:42.373 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T14:14:42.373 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T14:14:42.373 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T14:14:42.373 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T14:14:42.373 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T14:14:42.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:42.498 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T14:14:42.518 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-24T14:14:42.518 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T14:14:42.518 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T14:14:42.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:42.518 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T14:14:42.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:42.594 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-01-24T14:14:42.657 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T14:14:42.657 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:14:42.750 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:14:42.750 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T14:14:42.772 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:14:42.772 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T14:14:42.774 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T14:14:42.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:14:42.774 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-24T14:14:42.805 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-24T14:14:42.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.984+0000 7f4fb0f36700 1 Processor -- start 2024-01-24T14:14:42.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.984+0000 7f4fb0f36700 1 -- start start 2024-01-24T14:14:42.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.985+0000 7f4fb0f36700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4fac151de0 0x7f4fac155470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:42.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.985+0000 7f4fb0f36700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac158800 con 0x7f4fac0638c0 2024-01-24T14:14:42.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.985+0000 7f4fb0f36700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac157ed0 con 0x7f4fac066bb0 2024-01-24T14:14:42.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.985+0000 7f4fb0f36700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac1581b0 con 0x7f4fac151de0 2024-01-24T14:14:42.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.985+0000 7f4faa59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4fac151de0 0x7f4fac155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:42.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.985+0000 7f4fa1d9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4fac0638c0 0x7f4fac0664a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:47022/0 (socket says 172.21.15.29:47022) 2024-01-24T14:14:42.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.985+0000 7f4fa1d9b700 1 -- 172.21.15.29:0/1372747491 learned_addr learned my addr 172.21.15.29:0/1372747491 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:42.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.985+0000 7f4faa59c700 1 -- 172.21.15.29:0/1372747491 >> v1:172.21.15.163:6789/0 conn(0x7f4fac066bb0 legacy=0x7f4fac13d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:42.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.985+0000 7f4faa59c700 1 -- 172.21.15.29:0/1372747491 >> v1:172.21.15.29:6789/0 conn(0x7f4fac0638c0 legacy=0x7f4fac0664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:42.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.985+0000 7f4fa9d9b700 1 -- 172.21.15.29:0/1372747491 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985913351 0 0) 0x7f4fac158800 con 0x7f4fac0638c0 2024-01-24T14:14:42.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.985+0000 7f4faa59c700 1 -- 172.21.15.29:0/1372747491 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac158490 con 0x7f4fac151de0 2024-01-24T14:14:42.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.986+0000 7f4faa59c700 1 --2- 172.21.15.29:0/1372747491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fac151de0 0x7f4fac155470 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f4f9c009760 tx=0x7f4f9c009060).ready entity=mon.0 client_cookie=7512c0ce70172786 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:42.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.986+0000 7f4fa9d9b700 1 -- 172.21.15.29:0/1372747491 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00b580 con 0x7f4fac151de0 2024-01-24T14:14:42.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.986+0000 7f4fa9d9b700 1 -- 172.21.15.29:0/1372747491 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c00b6e0 con 0x7f4fac151de0 2024-01-24T14:14:42.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.986+0000 7f4fa9d9b700 1 -- 172.21.15.29:0/1372747491 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00ba60 con 0x7f4fac151de0 2024-01-24T14:14:42.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.986+0000 7f4fb0f36700 1 -- 172.21.15.29:0/1372747491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fac151de0 msgr2=0x7f4fac155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:42.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.986+0000 7f4fb0f36700 1 --2- 172.21.15.29:0/1372747491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fac151de0 0x7f4fac155470 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f4f9c009760 tx=0x7f4f9c009060).stop 2024-01-24T14:14:42.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.986+0000 7f4fb0f36700 1 -- 172.21.15.29:0/1372747491 shutdown_connections 2024-01-24T14:14:42.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.986+0000 7f4fb0f36700 1 --2- 172.21.15.29:0/1372747491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fac151de0 0x7f4fac155470 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:42.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.986+0000 7f4fb0f36700 1 -- 172.21.15.29:0/1372747491 >> 172.21.15.29:0/1372747491 conn(0x7f4fac0c5640 msgr2=0x7f4fac0c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:42.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.986+0000 7f4fb0f36700 1 -- 172.21.15.29:0/1372747491 shutdown_connections 2024-01-24T14:14:42.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.986+0000 7f4fb0f36700 1 -- 172.21.15.29:0/1372747491 wait complete. 2024-01-24T14:14:42.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.988+0000 7f4fb0f36700 1 Processor -- start 2024-01-24T14:14:42.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.988+0000 7f4fb0f36700 1 -- start start 2024-01-24T14:14:42.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fb0f36700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fac0638c0 0x7f4fac0a8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:42.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fb0f36700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fac066bb0 0x7f4fac164860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:42.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fb0f36700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fac151de0 0x7f4fac0aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:42.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fb0f36700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac165830 con 0x7f4fac066bb0 2024-01-24T14:14:42.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fb0f36700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0a8970 con 0x7f4fac151de0 2024-01-24T14:14:42.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fb0f36700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac165270 con 0x7f4fac0638c0 2024-01-24T14:14:42.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fa1d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fac066bb0 0x7f4fac164860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:42.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fa1d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fac066bb0 0x7f4fac164860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41554/0 (socket says 172.21.15.29:41554) 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fa1d9b700 1 -- 172.21.15.29:0/969485502 learned_addr learned my addr 172.21.15.29:0/969485502 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4faa59c700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fac0638c0 0x7f4fac0a8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4faad9d700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fac151de0 0x7f4fac0aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fa1d9b700 1 -- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fac0638c0 msgr2=0x7f4fac0a8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fa1d9b700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fac0638c0 0x7f4fac0a8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fa1d9b700 1 -- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fac151de0 msgr2=0x7f4fac0aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fa1d9b700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fac151de0 0x7f4fac0aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fa1d9b700 1 -- 172.21.15.29:0/969485502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c0092a0 con 0x7f4fac066bb0 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fa1d9b700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fac066bb0 0x7f4fac164860 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f4f9400c4c0 tx=0x7f4f94007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fa2ffd700 1 -- 172.21.15.29:0/969485502 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9400ce30 con 0x7f4fac066bb0 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.989+0000 7f4fa2ffd700 1 -- 172.21.15.29:0/969485502 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f940072e0 con 0x7f4fac066bb0 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.990+0000 7f4fb0f36700 1 -- 172.21.15.29:0/969485502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac165550 con 0x7f4fac066bb0 2024-01-24T14:14:42.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.990+0000 7f4fa2ffd700 1 -- 172.21.15.29:0/969485502 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f940057a0 con 0x7f4fac066bb0 2024-01-24T14:14:42.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.990+0000 7f4fb0f36700 1 -- 172.21.15.29:0/969485502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac0a92f0 con 0x7f4fac066bb0 2024-01-24T14:14:42.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.991+0000 7f4fb0f36700 1 -- 172.21.15.29:0/969485502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fac0a8ae0 con 0x7f4fac066bb0 2024-01-24T14:14:42.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.991+0000 7f4fa2ffd700 1 -- 172.21.15.29:0/969485502 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4f94005bd0 con 0x7f4fac066bb0 2024-01-24T14:14:42.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.991+0000 7f4fa2ffd700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4f8c05f270 0x7f4f8c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:42.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.991+0000 7f4fa2ffd700 1 -- 172.21.15.29:0/969485502 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f4f94079f60 con 0x7f4fac066bb0 2024-01-24T14:14:42.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.992+0000 7f4faa59c700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4f8c05f270 0x7f4f8c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:42.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.992+0000 7f4faa59c700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4f8c05f270 0x7f4f8c061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f4f9c00b340 tx=0x7f4f9c017000).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:42.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:42.996+0000 7f4fa2ffd700 1 -- 172.21.15.29:0/969485502 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4f9404acb0 con 0x7f4fac066bb0 2024-01-24T14:14:43.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.207+0000 7f4fb0f36700 1 -- 172.21.15.29:0/969485502 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fac0475e0 con 0x7f4f8c05f270 2024-01-24T14:14:43.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.209+0000 7f4fa2ffd700 1 -- 172.21.15.29:0/969485502 <== mgr.6790 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4fac0475e0 con 0x7f4f8c05f270 2024-01-24T14:14:43.209 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T14:14:43.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:14:43.210 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-01-24T14:14:43.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:14:43.210 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T14:14:43.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.211+0000 7f4fb0f36700 1 -- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4f8c05f270 msgr2=0x7f4f8c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:43.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.211+0000 7f4fb0f36700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4f8c05f270 0x7f4f8c061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f4f9c00b340 tx=0x7f4f9c017000).stop 2024-01-24T14:14:43.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.211+0000 7f4fb0f36700 1 -- 172.21.15.29:0/969485502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fac066bb0 msgr2=0x7f4fac164860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:43.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.211+0000 7f4fb0f36700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fac066bb0 0x7f4fac164860 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f4f9400c4c0 tx=0x7f4f94007c10).stop 2024-01-24T14:14:43.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.211+0000 7f4fb0f36700 1 -- 172.21.15.29:0/969485502 shutdown_connections 2024-01-24T14:14:43.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.211+0000 7f4fb0f36700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fac151de0 0x7f4fac0aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.211+0000 7f4fb0f36700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fac0638c0 0x7f4fac0a8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.211+0000 7f4fb0f36700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4f8c05f270 0x7f4f8c061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.211+0000 7f4fb0f36700 1 --2- 172.21.15.29:0/969485502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fac066bb0 0x7f4fac164860 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.211+0000 7f4fb0f36700 1 -- 172.21.15.29:0/969485502 >> 172.21.15.29:0/969485502 conn(0x7f4fac0c5640 msgr2=0x7f4fac0679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.211+0000 7f4fb0f36700 1 -- 172.21.15.29:0/969485502 shutdown_connections 2024-01-24T14:14:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.211+0000 7f4fb0f36700 1 -- 172.21.15.29:0/969485502 wait complete. 2024-01-24T14:14:43.225 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-01-24T14:14:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.356+0000 7f963ec49700 1 Processor -- start 2024-01-24T14:14:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.356+0000 7f963ec49700 1 -- start start 2024-01-24T14:14:43.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.357+0000 7f963ec49700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9638157530 0x7f9638068470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:43.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.357+0000 7f963ec49700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9638068960 0x7f9638066520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:43.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.357+0000 7f963ec49700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9638066ec0 con 0x7f9638066a10 2024-01-24T14:14:43.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.357+0000 7f963ec49700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9638065040 con 0x7f9638068960 2024-01-24T14:14:43.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.357+0000 7f963ec49700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9638064140 con 0x7f9638157530 2024-01-24T14:14:43.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.357+0000 7f963d1e6700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9638066a10 0x7f9638157c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:47026/0 (socket says 172.21.15.29:47026) 2024-01-24T14:14:43.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.357+0000 7f963d1e6700 1 -- 172.21.15.29:0/3700596869 learned_addr learned my addr 172.21.15.29:0/3700596869 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:14:43.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f963c9e5700 1 --2- 172.21.15.29:0/3700596869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9638157530 0x7f9638068470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:43.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f9637fff700 1 --2- 172.21.15.29:0/3700596869 >> v2:172.21.15.163:3300/0 conn(0x7f9638068960 0x7f9638066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f96377fe700 1 -- 172.21.15.29:0/3700596869 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289593211 0 0) 0x7f9630001490 con 0x7f9638066a10 2024-01-24T14:14:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f96377fe700 1 -- 172.21.15.29:0/3700596869 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96200033f0 con 0x7f9638066a10 2024-01-24T14:14:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f963c9e5700 1 -- 172.21.15.29:0/3700596869 >> v2:172.21.15.163:3300/0 conn(0x7f9638068960 msgr2=0x7f9638066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f963c9e5700 1 --2- 172.21.15.29:0/3700596869 >> v2:172.21.15.163:3300/0 conn(0x7f9638068960 0x7f9638066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f963c9e5700 1 -- 172.21.15.29:0/3700596869 >> v1:172.21.15.29:6789/0 conn(0x7f9638066a10 legacy=0x7f9638157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f963c9e5700 1 -- 172.21.15.29:0/3700596869 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96380be080 con 0x7f9638157530 2024-01-24T14:14:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f9637fff700 1 --2- 172.21.15.29:0/3700596869 >> v2:172.21.15.163:3300/0 conn(0x7f9638068960 0x7f9638066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f963c9e5700 1 --2- 172.21.15.29:0/3700596869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9638157530 0x7f9638068470 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f9628009760 tx=0x7f9628009060).ready entity=mon.2 client_cookie=7aac3dde197483e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f96377fe700 1 -- 172.21.15.29:0/3700596869 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962800d4a0 con 0x7f9638157530 2024-01-24T14:14:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f96377fe700 1 -- 172.21.15.29:0/3700596869 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962800d600 con 0x7f9638157530 2024-01-24T14:14:43.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.358+0000 7f96377fe700 1 -- 172.21.15.29:0/3700596869 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962800d980 con 0x7f9638157530 2024-01-24T14:14:43.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.359+0000 7f963ec49700 1 -- 172.21.15.29:0/3700596869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9638157530 msgr2=0x7f9638068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:43.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.359+0000 7f963ec49700 1 --2- 172.21.15.29:0/3700596869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9638157530 0x7f9638068470 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f9628009760 tx=0x7f9628009060).stop 2024-01-24T14:14:43.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.359+0000 7f963ec49700 1 -- 172.21.15.29:0/3700596869 shutdown_connections 2024-01-24T14:14:43.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.359+0000 7f963ec49700 1 --2- 172.21.15.29:0/3700596869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9638157530 0x7f9638068470 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.359+0000 7f963ec49700 1 --2- 172.21.15.29:0/3700596869 >> v2:172.21.15.163:3300/0 conn(0x7f9638068960 0x7f9638066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.359+0000 7f963ec49700 1 -- 172.21.15.29:0/3700596869 >> 172.21.15.29:0/3700596869 conn(0x7f96380c8ba0 msgr2=0x7f96380c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:43.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.359+0000 7f963ec49700 1 -- 172.21.15.29:0/3700596869 shutdown_connections 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.359+0000 7f963ec49700 1 -- 172.21.15.29:0/3700596869 wait complete. 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.360+0000 7f963ec49700 1 Processor -- start 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.361+0000 7f963ec49700 1 -- start start 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.361+0000 7f963ec49700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9638066a10 0x7f96380a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.361+0000 7f963ec49700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9638068960 0x7f963814fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.361+0000 7f963ec49700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9638157530 0x7f96380a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.361+0000 7f963ec49700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96380c41a0 con 0x7f9638066a10 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.361+0000 7f963ec49700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96380a54a0 con 0x7f9638068960 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.361+0000 7f963ec49700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96380c3590 con 0x7f9638157530 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.361+0000 7f963c9e5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9638066a10 0x7f96380a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.361+0000 7f963c9e5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9638066a10 0x7f96380a73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41560/0 (socket says 172.21.15.29:41560) 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.361+0000 7f963c9e5700 1 -- 172.21.15.29:0/435977037 learned_addr learned my addr 172.21.15.29:0/435977037 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:43.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f9637fff700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9638068960 0x7f963814fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f963d1e6700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9638157530 0x7f96380a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f963c9e5700 1 -- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9638157530 msgr2=0x7f96380a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f963c9e5700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9638157530 0x7f96380a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f963c9e5700 1 -- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9638068960 msgr2=0x7f963814fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f963c9e5700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9638068960 0x7f963814fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f963c9e5700 1 -- 172.21.15.29:0/435977037 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c0092a0 con 0x7f9638066a10 2024-01-24T14:14:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f963c9e5700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9638066a10 0x7f96380a73a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7f9628011fb0 tx=0x7f9628016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f9634ff9700 1 -- 172.21.15.29:0/435977037 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962800dd30 con 0x7f9638066a10 2024-01-24T14:14:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f9634ff9700 1 -- 172.21.15.29:0/435977037 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962800de90 con 0x7f9638066a10 2024-01-24T14:14:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f963ec49700 1 -- 172.21.15.29:0/435977037 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96280092a0 con 0x7f9638066a10 2024-01-24T14:14:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f9634ff9700 1 -- 172.21.15.29:0/435977037 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962800d980 con 0x7f9638066a10 2024-01-24T14:14:43.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.362+0000 7f963ec49700 1 -- 172.21.15.29:0/435977037 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96380c3b50 con 0x7f9638066a10 2024-01-24T14:14:43.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.363+0000 7f963ec49700 1 -- 172.21.15.29:0/435977037 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96380475e0 con 0x7f9638066a10 2024-01-24T14:14:43.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.365+0000 7f9634ff9700 1 -- 172.21.15.29:0/435977037 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 39) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f962801d640 con 0x7f9638066a10 2024-01-24T14:14:43.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.365+0000 7f9634ff9700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f962405f270 0x7f9624061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:43.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.365+0000 7f9634ff9700 1 -- 172.21.15.29:0/435977037 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f962807c960 con 0x7f9638066a10 2024-01-24T14:14:43.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.365+0000 7f9637fff700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f962405f270 0x7f9624061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:43.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.366+0000 7f9637fff700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f962405f270 0x7f9624061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f96380c5330 tx=0x7f962c009060).ready entity=mgr.6790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:43.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.369+0000 7f9634ff9700 1 -- 172.21.15.29:0/435977037 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f962804d770 con 0x7f9638066a10 2024-01-24T14:14:43.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.601+0000 7f963ec49700 1 -- 172.21.15.29:0/435977037 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f963806c230 con 0x7f9638066a10 2024-01-24T14:14:43.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.927+0000 7f9634ff9700 1 -- 172.21.15.29:0/435977037 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 40) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f962807a270 con 0x7f9638066a10 2024-01-24T14:14:43.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.930+0000 7f9634ff9700 1 -- 172.21.15.29:0/435977037 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v40) v1 ==== 92+0+0 (secure 0 0 0) 0x7f9628048030 con 0x7f9638066a10 2024-01-24T14:14:43.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.932+0000 7f963ec49700 1 -- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f962405f270 msgr2=0x7f9624061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:43.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.932+0000 7f963ec49700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f962405f270 0x7f9624061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f96380c5330 tx=0x7f962c009060).stop 2024-01-24T14:14:43.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.932+0000 7f963ec49700 1 -- 172.21.15.29:0/435977037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9638066a10 msgr2=0x7f96380a73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:43.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.932+0000 7f963ec49700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9638066a10 0x7f96380a73a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7f9628011fb0 tx=0x7f9628016000).stop 2024-01-24T14:14:43.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.932+0000 7f963ec49700 1 -- 172.21.15.29:0/435977037 shutdown_connections 2024-01-24T14:14:43.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.932+0000 7f963ec49700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9638157530 0x7f96380a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.932+0000 7f963ec49700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f962405f270 0x7f9624061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.932+0000 7f963ec49700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9638066a10 0x7f96380a73a0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.932+0000 7f963ec49700 1 --2- 172.21.15.29:0/435977037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9638068960 0x7f963814fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:43.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.932+0000 7f963ec49700 1 -- 172.21.15.29:0/435977037 >> 172.21.15.29:0/435977037 conn(0x7f96380c8ba0 msgr2=0x7f9638150e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:43.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.933+0000 7f963ec49700 1 -- 172.21.15.29:0/435977037 shutdown_connections 2024-01-24T14:14:43.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:43.933+0000 7f963ec49700 1 -- 172.21.15.29:0/435977037 wait complete. 2024-01-24T14:14:43.950 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-01-24T14:14:44.075 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:44.075+0000 7ffb488e5680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:14:44.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.087+0000 7fbd35583700 1 Processor -- start 2024-01-24T14:14:44.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.087+0000 7fbd35583700 1 -- start start 2024-01-24T14:14:44.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.088+0000 7fbd35583700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd300664c0 0x7fbd30066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:44.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.088+0000 7fbd35583700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbd300bdb30 0x7fbd300bdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:44.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.088+0000 7fbd35583700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd300686a0 con 0x7fbd30134ea0 2024-01-24T14:14:44.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.088+0000 7fbd35583700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd30066e30 con 0x7fbd300bdb30 2024-01-24T14:14:44.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.088+0000 7fbd35583700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300c38e0 con 0x7fbd300664c0 2024-01-24T14:14:44.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.088+0000 7fbd2f7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbd300bdb30 0x7fbd300bdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:44.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2f7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbd300bdb30 0x7fbd300bdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43188/0 (socket says 172.21.15.29:43188) 2024-01-24T14:14:44.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2f7fe700 1 -- 172.21.15.29:0/2903799483 learned_addr learned my addr 172.21.15.29:0/2903799483 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:44.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2effd700 1 --1- 172.21.15.29:0/2903799483 >> v1:172.21.15.163:6789/0 conn(0x7fbd30134ea0 0x7fbd300c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:36350/0 (socket says 172.21.15.29:36350) 2024-01-24T14:14:44.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2e7fc700 1 --2- 172.21.15.29:0/2903799483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd300664c0 0x7fbd30066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:44.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2dffb700 1 -- 172.21.15.29:0/2903799483 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100261995 0 0) 0x7fbd300686a0 con 0x7fbd30134ea0 2024-01-24T14:14:44.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2f7fe700 1 -- 172.21.15.29:0/2903799483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd300664c0 msgr2=0x7fbd30066940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:44.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2f7fe700 1 --2- 172.21.15.29:0/2903799483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd300664c0 0x7fbd30066940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2f7fe700 1 -- 172.21.15.29:0/2903799483 >> v1:172.21.15.163:6789/0 conn(0x7fbd30134ea0 legacy=0x7fbd300c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:44.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2f7fe700 1 -- 172.21.15.29:0/2903799483 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd30068b30 con 0x7fbd300bdb30 2024-01-24T14:14:44.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2e7fc700 1 --2- 172.21.15.29:0/2903799483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd300664c0 0x7fbd30066940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:44.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2f7fe700 1 --2- 172.21.15.29:0/2903799483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd300bdb30 0x7fbd300bdf90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7fbd28009f20 tx=0x7fbd28009840).ready entity=mon.0 client_cookie=d76c9dcbda286e6e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:44.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2dffb700 1 -- 172.21.15.29:0/2903799483 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2800e070 con 0x7fbd300bdb30 2024-01-24T14:14:44.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2dffb700 1 -- 172.21.15.29:0/2903799483 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2800b9d0 con 0x7fbd300bdb30 2024-01-24T14:14:44.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.089+0000 7fbd2dffb700 1 -- 172.21.15.29:0/2903799483 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2800bd50 con 0x7fbd300bdb30 2024-01-24T14:14:44.093 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:44.090+0000 7fd2661ce680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:14:44.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.090+0000 7fbd35583700 1 -- 172.21.15.29:0/2903799483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd300bdb30 msgr2=0x7fbd300bdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:44.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.090+0000 7fbd35583700 1 --2- 172.21.15.29:0/2903799483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd300bdb30 0x7fbd300bdf90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7fbd28009f20 tx=0x7fbd28009840).stop 2024-01-24T14:14:44.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.090+0000 7fbd35583700 1 -- 172.21.15.29:0/2903799483 shutdown_connections 2024-01-24T14:14:44.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.090+0000 7fbd35583700 1 --2- 172.21.15.29:0/2903799483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd300bdb30 0x7fbd300bdf90 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.090+0000 7fbd35583700 1 --2- 172.21.15.29:0/2903799483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd300664c0 0x7fbd30066940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.090+0000 7fbd35583700 1 -- 172.21.15.29:0/2903799483 >> 172.21.15.29:0/2903799483 conn(0x7fbd300c8bd0 msgr2=0x7fbd300c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:44.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.090+0000 7fbd35583700 1 -- 172.21.15.29:0/2903799483 shutdown_connections 2024-01-24T14:14:44.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.090+0000 7fbd35583700 1 -- 172.21.15.29:0/2903799483 wait complete. 2024-01-24T14:14:44.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.092+0000 7fbd35583700 1 Processor -- start 2024-01-24T14:14:44.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.092+0000 7fbd35583700 1 -- start start 2024-01-24T14:14:44.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.092+0000 7fbd35583700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd300664c0 0x7fbd300ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:44.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.092+0000 7fbd35583700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd300bdb30 0x7fbd3014cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:44.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.092+0000 7fbd35583700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30134ea0 0x7fbd300abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:44.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.092+0000 7fbd35583700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300647d0 con 0x7fbd300664c0 2024-01-24T14:14:44.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.092+0000 7fbd35583700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd30063ea0 con 0x7fbd30134ea0 2024-01-24T14:14:44.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.092+0000 7fbd35583700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd30064180 con 0x7fbd300bdb30 2024-01-24T14:14:44.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.092+0000 7fbd2effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd300664c0 0x7fbd300ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:44.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd2effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd300664c0 0x7fbd300ab3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43192/0 (socket says 172.21.15.29:43192) 2024-01-24T14:14:44.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd2effd700 1 -- 172.21.15.29:0/2498787058 learned_addr learned my addr 172.21.15.29:0/2498787058 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd2e7fc700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd300bdb30 0x7fbd3014cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd2f7fe700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30134ea0 0x7fbd300abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd2effd700 1 -- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd300bdb30 msgr2=0x7fbd3014cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd2effd700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd300bdb30 0x7fbd3014cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd2effd700 1 -- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30134ea0 msgr2=0x7fbd300abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd2effd700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30134ea0 0x7fbd300abb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd2effd700 1 -- 172.21.15.29:0/2498787058 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd240092a0 con 0x7fbd300664c0 2024-01-24T14:14:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd2e7fc700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd300bdb30 0x7fbd3014cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd2effd700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd300664c0 0x7fbd300ab3a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fbd2000c560 tx=0x7fbd20007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd16ffd700 1 -- 172.21.15.29:0/2498787058 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd200072e0 con 0x7fbd300664c0 2024-01-24T14:14:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd16ffd700 1 -- 172.21.15.29:0/2498787058 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd20007440 con 0x7fbd300664c0 2024-01-24T14:14:44.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd35583700 1 -- 172.21.15.29:0/2498787058 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd28009ae0 con 0x7fbd300664c0 2024-01-24T14:14:44.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd16ffd700 1 -- 172.21.15.29:0/2498787058 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd200058f0 con 0x7fbd300664c0 2024-01-24T14:14:44.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.093+0000 7fbd35583700 1 -- 172.21.15.29:0/2498787058 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3014d020 con 0x7fbd300664c0 2024-01-24T14:14:44.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.094+0000 7fbd35583700 1 -- 172.21.15.29:0/2498787058 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd300475e0 con 0x7fbd300664c0 2024-01-24T14:14:44.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.095+0000 7fbd16ffd700 1 -- 172.21.15.29:0/2498787058 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 40) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fbd20005ce0 con 0x7fbd300664c0 2024-01-24T14:14:44.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.095+0000 7fbd16ffd700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbd1c05f2c0 0x7fbd1c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:44.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.095+0000 7fbd16ffd700 1 -- 172.21.15.29:0/2498787058 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fbd2007a060 con 0x7fbd300664c0 2024-01-24T14:14:44.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.095+0000 7fbd2e7fc700 1 -- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbd1c05f2c0 msgr2=0x7fbd1c061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:44.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.095+0000 7fbd2e7fc700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbd1c05f2c0 0x7fbd1c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:44.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.100+0000 7fbd16ffd700 1 -- 172.21.15.29:0/2498787058 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbd2004adb0 con 0x7fbd300664c0 2024-01-24T14:14:44.139 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:44.140+0000 7ffb488e5680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:14:44.164 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:44.164+0000 7fd2661ce680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:14:44.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.284+0000 7fbd35583700 1 -- 172.21.15.29:0/2498787058 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fbd30067a90 con 0x7fbd300664c0 2024-01-24T14:14:44.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.287+0000 7fbd16ffd700 1 -- 172.21.15.29:0/2498787058 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v19)=0 v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbd20018070 con 0x7fbd300664c0 2024-01-24T14:14:44.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.289+0000 7fbd35583700 1 -- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbd1c05f2c0 msgr2=0x7fbd1c061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:44.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.289+0000 7fbd35583700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbd1c05f2c0 0x7fbd1c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:44.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.289+0000 7fbd35583700 1 -- 172.21.15.29:0/2498787058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd300664c0 msgr2=0x7fbd300ab3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:44.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.289+0000 7fbd35583700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd300664c0 0x7fbd300ab3a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fbd2000c560 tx=0x7fbd20007c90).stop 2024-01-24T14:14:44.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.289+0000 7fbd35583700 1 -- 172.21.15.29:0/2498787058 shutdown_connections 2024-01-24T14:14:44.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.289+0000 7fbd35583700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30134ea0 0x7fbd300abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.289+0000 7fbd35583700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbd1c05f2c0 0x7fbd1c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:44.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.289+0000 7fbd35583700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd300664c0 0x7fbd300ab3a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.289+0000 7fbd35583700 1 --2- 172.21.15.29:0/2498787058 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd300bdb30 0x7fbd3014cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.289+0000 7fbd35583700 1 -- 172.21.15.29:0/2498787058 >> 172.21.15.29:0/2498787058 conn(0x7fbd300c8bd0 msgr2=0x7fbd30150b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:44.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.289+0000 7fbd35583700 1 -- 172.21.15.29:0/2498787058 shutdown_connections 2024-01-24T14:14:44.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.289+0000 7fbd35583700 1 -- 172.21.15.29:0/2498787058 wait complete. 2024-01-24T14:14:44.304 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /bad-path 1M 2024-01-24T14:14:44.416 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:44.416+0000 7ffb488e5680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:14:44.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.441+0000 7fb8467a1700 1 Processor -- start 2024-01-24T14:14:44.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.441+0000 7fb8467a1700 1 -- start start 2024-01-24T14:14:44.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.442+0000 7fb8467a1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb840154fd0 0x7fb8400638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:44.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.442+0000 7fb8467a1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb840066b60 0x7fb840063db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:44.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.442+0000 7fb8467a1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb840067c10 con 0x7fb8401553b0 2024-01-24T14:14:44.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.442+0000 7fb8467a1700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb840067ef0 con 0x7fb840066b60 2024-01-24T14:14:44.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.442+0000 7fb8467a1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400681d0 con 0x7fb840154fd0 2024-01-24T14:14:44.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.443+0000 7fb844d3e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb840066b60 0x7fb840063db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:44.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.443+0000 7fb844d3e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb840066b60 0x7fb840063db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43194/0 (socket says 172.21.15.29:43194) 2024-01-24T14:14:44.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.443+0000 7fb844d3e700 1 -- 172.21.15.29:0/1364724556 learned_addr learned my addr 172.21.15.29:0/1364724556 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:44.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.443+0000 7fb83ffff700 1 --2- 172.21.15.29:0/1364724556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb840154fd0 0x7fb8400638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:44.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.443+0000 7fb844d3e700 1 -- 172.21.15.29:0/1364724556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb840154fd0 msgr2=0x7fb8400638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.443+0000 7fb844d3e700 1 --2- 172.21.15.29:0/1364724556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb840154fd0 0x7fb8400638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.443+0000 7fb844d3e700 1 -- 172.21.15.29:0/1364724556 >> v1:172.21.15.163:6789/0 conn(0x7fb8401553b0 legacy=0x7fb840066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.443+0000 7fb844d3e700 1 -- 172.21.15.29:0/1364724556 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8400684b0 con 0x7fb840066b60 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.443+0000 7fb844d3e700 1 --2- 172.21.15.29:0/1364724556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb840066b60 0x7fb840063db0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7fb838009f20 tx=0x7fb838009840).ready entity=mon.0 client_cookie=9f38820570e0d4f4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.443+0000 7fb83effd700 1 -- 172.21.15.29:0/1364724556 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83800e070 con 0x7fb840066b60 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.443+0000 7fb83effd700 1 -- 172.21.15.29:0/1364724556 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83800b9d0 con 0x7fb840066b60 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.443+0000 7fb83effd700 1 -- 172.21.15.29:0/1364724556 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83800bd50 con 0x7fb840066b60 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.444+0000 7fb8467a1700 1 -- 172.21.15.29:0/1364724556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb840066b60 msgr2=0x7fb840063db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.444+0000 7fb8467a1700 1 --2- 172.21.15.29:0/1364724556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb840066b60 0x7fb840063db0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7fb838009f20 tx=0x7fb838009840).stop 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.444+0000 7fb8467a1700 1 -- 172.21.15.29:0/1364724556 shutdown_connections 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.444+0000 7fb8467a1700 1 --2- 172.21.15.29:0/1364724556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb840154fd0 0x7fb8400638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.444+0000 7fb8467a1700 1 --2- 172.21.15.29:0/1364724556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb840066b60 0x7fb840063db0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.444+0000 7fb8467a1700 1 -- 172.21.15.29:0/1364724556 >> 172.21.15.29:0/1364724556 conn(0x7fb8400c8bd0 msgr2=0x7fb8400c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:44.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.444+0000 7fb8467a1700 1 -- 172.21.15.29:0/1364724556 shutdown_connections 2024-01-24T14:14:44.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.444+0000 7fb8467a1700 1 -- 172.21.15.29:0/1364724556 wait complete. 2024-01-24T14:14:44.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.446+0000 7fb8467a1700 1 Processor -- start 2024-01-24T14:14:44.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.446+0000 7fb8467a1700 1 -- start start 2024-01-24T14:14:44.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.447+0000 7fb8467a1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb840066b60 0x7fb8400c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:44.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.447+0000 7fb8467a1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb840154fd0 0x7fb8400aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:44.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.447+0000 7fb8467a1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8401553b0 0x7fb840164660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:44.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.447+0000 7fb8467a1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400c8010 con 0x7fb840154fd0 2024-01-24T14:14:44.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.447+0000 7fb8467a1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400ab350 con 0x7fb840066b60 2024-01-24T14:14:44.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.447+0000 7fb8467a1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400c2a20 con 0x7fb8401553b0 2024-01-24T14:14:44.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.447+0000 7fb83ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb840066b60 0x7fb8400c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:44.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.447+0000 7fb83f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb840154fd0 0x7fb8400aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:44.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.447+0000 7fb83ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb840066b60 0x7fb8400c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:33124/0 (socket says 172.21.15.29:33124) 2024-01-24T14:14:44.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.447+0000 7fb83ffff700 1 -- 172.21.15.29:0/569897407 learned_addr learned my addr 172.21.15.29:0/569897407 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:44.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.447+0000 7fb844d3e700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8401553b0 0x7fb840164660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:44.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.448+0000 7fb83ffff700 1 -- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8401553b0 msgr2=0x7fb840164660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:44.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.448+0000 7fb83ffff700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8401553b0 0x7fb840164660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.448+0000 7fb83ffff700 1 -- 172.21.15.29:0/569897407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb840154fd0 msgr2=0x7fb8400aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:44.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.448+0000 7fb83ffff700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb840154fd0 0x7fb8400aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:44.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.448+0000 7fb83ffff700 1 -- 172.21.15.29:0/569897407 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb838009ae0 con 0x7fb840066b60 2024-01-24T14:14:44.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.448+0000 7fb83f7fe700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb840154fd0 0x7fb8400aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:44.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.448+0000 7fb83ffff700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb840066b60 0x7fb8400c84a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fb83000e930 tx=0x7fb830009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:44.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.448+0000 7fb837fff700 1 -- 172.21.15.29:0/569897407 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb830012600 con 0x7fb840066b60 2024-01-24T14:14:44.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.448+0000 7fb837fff700 1 -- 172.21.15.29:0/569897407 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb830012760 con 0x7fb840066b60 2024-01-24T14:14:44.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.448+0000 7fb8467a1700 1 -- 172.21.15.29:0/569897407 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8400c76e0 con 0x7fb840066b60 2024-01-24T14:14:44.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.448+0000 7fb837fff700 1 -- 172.21.15.29:0/569897407 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb830012ae0 con 0x7fb840066b60 2024-01-24T14:14:44.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.448+0000 7fb8467a1700 1 -- 172.21.15.29:0/569897407 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8400c7ca0 con 0x7fb840066b60 2024-01-24T14:14:44.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.449+0000 7fb8467a1700 1 -- 172.21.15.29:0/569897407 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8400c82f0 con 0x7fb840066b60 2024-01-24T14:14:44.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.449+0000 7fb837fff700 1 -- 172.21.15.29:0/569897407 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 40) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fb83001b400 con 0x7fb840066b60 2024-01-24T14:14:44.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.450+0000 7fb837fff700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 0x7fb820061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:44.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.450+0000 7fb83f7fe700 1 -- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 msgr2=0x7fb820061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:44.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.450+0000 7fb837fff700 1 -- 172.21.15.29:0/569897407 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fb83007acf0 con 0x7fb840066b60 2024-01-24T14:14:44.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.450+0000 7fb83f7fe700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 0x7fb820061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:14:44.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.454+0000 7fb837fff700 1 -- 172.21.15.29:0/569897407 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb83004b920 con 0x7fb840066b60 2024-01-24T14:14:44.486 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:44.486+0000 7fd2661ce680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:14:44.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.651+0000 7fb83f7fe700 1 -- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 msgr2=0x7fb820061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:44.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.651+0000 7fb83f7fe700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 0x7fb820061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:14:44.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:44.662+0000 7fb8467a1700 1 -- 172.21.15.29:0/569897407 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8400c2b90 con 0x7fb82005f2c0 2024-01-24T14:14:45.035 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:45.034+0000 7ffb488e5680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:14:45.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:45.052+0000 7fb83f7fe700 1 -- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 msgr2=0x7fb820061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:45.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:45.052+0000 7fb83f7fe700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 0x7fb820061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-01-24T14:14:45.106 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:45.106+0000 7ffb488e5680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:14:45.202 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:45.202+0000 7fd2661ce680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:14:45.276 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:45.277+0000 7fd2661ce680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:14:45.295 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:45.295+0000 7ffb488e5680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:14:45.491 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:45.492+0000 7fd2661ce680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:14:45.718 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:45.718+0000 7ffb488e5680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:14:45.822 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:45.822+0000 7ffb488e5680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:14:45.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:45.853+0000 7fb83f7fe700 1 -- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 msgr2=0x7fb820061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:45.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:45.853+0000 7fb83f7fe700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 0x7fb820061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-01-24T14:14:45.969 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:45.970+0000 7fd2661ce680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:14:45.976 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:45.976+0000 7ffb488e5680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:14:46.052 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:46.052+0000 7ffb488e5680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:14:46.086 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:46.087+0000 7fd2661ce680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:14:46.120 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:46.120+0000 7ffb488e5680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:14:46.263 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:46.263+0000 7fd2661ce680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:14:46.350 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:46.348+0000 7fd2661ce680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:14:46.378 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:46.378+0000 7ffb488e5680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:14:46.424 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:46.424+0000 7fd2661ce680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:14:46.563 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:46.562+0000 7ffb488e5680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:14:46.718 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:46.718+0000 7fd2661ce680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:14:46.929 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:46.929+0000 7fd2661ce680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:14:47.218 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:47.218+0000 7ffb488e5680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:14:47.287 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:47.287+0000 7ffb488e5680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:14:47.364 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:47.364+0000 7ffb488e5680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:14:47.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:47.454+0000 7fb83f7fe700 1 -- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 msgr2=0x7fb820061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:14:47.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:47.454+0000 7fb83f7fe700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 0x7fb820061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-01-24T14:14:47.512 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:47.512+0000 7ffb488e5680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:14:47.583 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:47.583+0000 7ffb488e5680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:14:47.678 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:47.679+0000 7fd2661ce680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:14:47.698 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:47.699+0000 7ffb488e5680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:14:47.757 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:47.758+0000 7fd2661ce680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:14:47.807 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:47.806+0000 7ffb488e5680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:14:47.843 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:47.843+0000 7fd2661ce680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:14:48.010 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:48.010+0000 7fd2661ce680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:14:48.091 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:48.091+0000 7fd2661ce680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:14:48.098 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:48.098+0000 7ffb488e5680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:14:48.170 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:48.169+0000 7ffb488e5680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:14:48.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:48.191+0000 7fb837fff700 1 -- 172.21.15.29:0/569897407 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mgrmap(e 41) v1 ==== 75287+0+0 (secure 0 0 0) 0x7fb830078480 con 0x7fb840066b60 2024-01-24T14:14:48.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:48.192+0000 7fb837fff700 1 -- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 msgr2=0x7fb820061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:14:48.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:48.192+0000 7fb837fff700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 0x7fb820061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:14:48.226 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:48.227+0000 7fd2661ce680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:14:48.257 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:48.257+0000 7ffabc65b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:48.258 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:48.257+0000 7ffabc65b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:48.258 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:48.257+0000 7ffabc65b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:48.258 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:48.257+0000 7ffabc65b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:48.258 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:48.257+0000 7ffabc65b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:48.258 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:48.258+0000 7ffabf661700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:48.258 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:48.258+0000 7ffabf661700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:48.259 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:48.258+0000 7ffabf661700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:48.259 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:48.258+0000 7ffabf661700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:48.259 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:14:48.258+0000 7ffabf661700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:14:48.353 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:48.354+0000 7fd2661ce680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:14:48.688 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:48.689+0000 7fd2661ce680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:14:48.770 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:14:48.770+0000 7fd2661ce680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:14:49.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.198+0000 7fb837fff700 1 -- 172.21.15.29:0/569897407 <== mon.1 v2:172.21.15.163:3300/0 8 ==== mgrmap(e 42) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fb830078800 con 0x7fb840066b60 2024-01-24T14:14:49.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.198+0000 7fb837fff700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 0x7fb820061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:49.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.198+0000 7fb837fff700 1 -- 172.21.15.29:0/569897407 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb820065e50 con 0x7fb82005f2c0 2024-01-24T14:14:49.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.199+0000 7fb83f7fe700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 0x7fb820061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:49.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.200+0000 7fb83f7fe700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 0x7fb820061700 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fb8400c8ae0 tx=0x7fb828007d20).ready entity=mgr.7117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:49.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.225+0000 7fb837fff700 1 -- 172.21.15.29:0/569897407 <== mgr.7117 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb820065e50 con 0x7fb82005f2c0 2024-01-24T14:14:49.225 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /bad-path 2024-01-24T14:14:49.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.228+0000 7fb8467a1700 1 -- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 msgr2=0x7fb820061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:49.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.228+0000 7fb8467a1700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 0x7fb820061700 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fb8400c8ae0 tx=0x7fb828007d20).stop 2024-01-24T14:14:49.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.228+0000 7fb8467a1700 1 -- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb840066b60 msgr2=0x7fb8400c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:49.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.228+0000 7fb8467a1700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb840066b60 0x7fb8400c84a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fb83000e930 tx=0x7fb830009060).stop 2024-01-24T14:14:49.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.228+0000 7fb8467a1700 1 -- 172.21.15.29:0/569897407 shutdown_connections 2024-01-24T14:14:49.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.228+0000 7fb8467a1700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8401553b0 0x7fb840164660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.228+0000 7fb8467a1700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb840066b60 0x7fb8400c84a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.228+0000 7fb8467a1700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb82005f2c0 0x7fb820061700 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.228+0000 7fb8467a1700 1 --2- 172.21.15.29:0/569897407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb840154fd0 0x7fb8400aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.228+0000 7fb8467a1700 1 -- 172.21.15.29:0/569897407 >> 172.21.15.29:0/569897407 conn(0x7fb8400c8bd0 msgr2=0x7fb840063880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:49.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.228+0000 7fb8467a1700 1 -- 172.21.15.29:0/569897407 shutdown_connections 2024-01-24T14:14:49.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.228+0000 7fb8467a1700 1 -- 172.21.15.29:0/569897407 wait complete. 2024-01-24T14:14:49.243 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2024-01-24T14:14:49.244 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2024-01-24T14:14:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.382+0000 7f584e06b700 1 Processor -- start 2024-01-24T14:14:49.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.382+0000 7f584e06b700 1 -- start start 2024-01-24T14:14:49.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f584e06b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5848066450 0x7f5848066830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:49.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f584e06b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5848152540 0x7f58481529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:49.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f584e06b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5848153300 con 0x7f5848155a10 2024-01-24T14:14:49.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f584e06b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5848067b10 con 0x7f5848066450 2024-01-24T14:14:49.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f584e06b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5848067df0 con 0x7f5848152540 2024-01-24T14:14:49.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f58477fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5848066450 0x7f5848066830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:49.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f58477fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5848066450 0x7f5848066830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:33132/0 (socket says 172.21.15.29:33132) 2024-01-24T14:14:49.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f58477fe700 1 -- 172.21.15.29:0/103152356 learned_addr learned my addr 172.21.15.29:0/103152356 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:49.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f5846ffd700 1 --2- 172.21.15.29:0/103152356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5848152540 0x7f58481529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:49.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f58477fe700 1 -- 172.21.15.29:0/103152356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5848152540 msgr2=0x7f58481529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:14:49.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f58477fe700 1 --2- 172.21.15.29:0/103152356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5848152540 0x7f58481529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f58477fe700 1 -- 172.21.15.29:0/103152356 >> v1:172.21.15.163:6789/0 conn(0x7f5848155a10 legacy=0x7f5848155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:49.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f58477fe700 1 -- 172.21.15.29:0/103152356 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480680d0 con 0x7f5848066450 2024-01-24T14:14:49.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.383+0000 7f5846ffd700 1 --2- 172.21.15.29:0/103152356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5848152540 0x7f58481529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:14:49.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.384+0000 7f58477fe700 1 --2- 172.21.15.29:0/103152356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5848066450 0x7f5848066830 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f5830006fd0 tx=0x7f58300090f0).ready entity=mon.1 client_cookie=fd34bd3cae18c07f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:49.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.384+0000 7f58467fc700 1 -- 172.21.15.29:0/103152356 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583000b580 con 0x7f5848066450 2024-01-24T14:14:49.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.384+0000 7f58467fc700 1 -- 172.21.15.29:0/103152356 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583000b6e0 con 0x7f5848066450 2024-01-24T14:14:49.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.384+0000 7f58467fc700 1 -- 172.21.15.29:0/103152356 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583000ba60 con 0x7f5848066450 2024-01-24T14:14:49.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.384+0000 7f584e06b700 1 -- 172.21.15.29:0/103152356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5848066450 msgr2=0x7f5848066830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:49.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.384+0000 7f584e06b700 1 --2- 172.21.15.29:0/103152356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5848066450 0x7f5848066830 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f5830006fd0 tx=0x7f58300090f0).stop 2024-01-24T14:14:49.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.384+0000 7f584e06b700 1 -- 172.21.15.29:0/103152356 shutdown_connections 2024-01-24T14:14:49.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.384+0000 7f584e06b700 1 --2- 172.21.15.29:0/103152356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5848066450 0x7f5848066830 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.384+0000 7f584e06b700 1 --2- 172.21.15.29:0/103152356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5848152540 0x7f58481529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.384+0000 7f584e06b700 1 -- 172.21.15.29:0/103152356 >> 172.21.15.29:0/103152356 conn(0x7f58480c8bd0 msgr2=0x7f58480c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:49.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.384+0000 7f584e06b700 1 -- 172.21.15.29:0/103152356 shutdown_connections 2024-01-24T14:14:49.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.384+0000 7f584e06b700 1 -- 172.21.15.29:0/103152356 wait complete. 2024-01-24T14:14:49.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.386+0000 7f584e06b700 1 Processor -- start 2024-01-24T14:14:49.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.386+0000 7f584e06b700 1 -- start start 2024-01-24T14:14:49.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f584e06b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5848066450 0x7f58480c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:49.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f584e06b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5848152540 0x7f584814f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:49.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f584e06b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5848155a10 0x7f584814f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:49.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f584e06b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480c2340 con 0x7f5848066450 2024-01-24T14:14:49.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f584e06b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480c1730 con 0x7f5848155a10 2024-01-24T14:14:49.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f584e06b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58480c1a10 con 0x7f5848152540 2024-01-24T14:14:49.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f58477fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5848066450 0x7f58480c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:49.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f58477fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5848066450 0x7f58480c27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43212/0 (socket says 172.21.15.29:43212) 2024-01-24T14:14:49.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f58477fe700 1 -- 172.21.15.29:0/885901782 learned_addr learned my addr 172.21.15.29:0/885901782 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:14:49.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f5846ffd700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5848152540 0x7f584814f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:49.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f5847fff700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5848155a10 0x7f584814f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:49.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f58477fe700 1 -- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5848152540 msgr2=0x7f584814f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:49.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f58477fe700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5848152540 0x7f584814f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f58477fe700 1 -- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5848155a10 msgr2=0x7f584814f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:49.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f58477fe700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5848155a10 0x7f584814f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f58477fe700 1 -- 172.21.15.29:0/885901782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58380092a0 con 0x7f5848066450 2024-01-24T14:14:49.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.387+0000 7f58477fe700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5848066450 0x7f58480c27d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f5830009fd0 tx=0x7f5830017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:49.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.388+0000 7f583f7fe700 1 -- 172.21.15.29:0/885901782 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830000bc0 con 0x7f5848066450 2024-01-24T14:14:49.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.388+0000 7f583f7fe700 1 -- 172.21.15.29:0/885901782 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583000b850 con 0x7f5848066450 2024-01-24T14:14:49.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.388+0000 7f584e06b700 1 -- 172.21.15.29:0/885901782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5830009370 con 0x7f5848066450 2024-01-24T14:14:49.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.388+0000 7f583f7fe700 1 -- 172.21.15.29:0/885901782 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830000d20 con 0x7f5848066450 2024-01-24T14:14:49.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.388+0000 7f584e06b700 1 -- 172.21.15.29:0/885901782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58480c1fd0 con 0x7f5848066450 2024-01-24T14:14:49.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.389+0000 7f584e06b700 1 -- 172.21.15.29:0/885901782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58480660a0 con 0x7f5848066450 2024-01-24T14:14:49.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.389+0000 7f583f7fe700 1 -- 172.21.15.29:0/885901782 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 42) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f583001bac0 con 0x7f5848066450 2024-01-24T14:14:49.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.389+0000 7f583f7fe700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f582805f080 0x7f5828061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:14:49.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.390+0000 7f583f7fe700 1 -- 172.21.15.29:0/885901782 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f583007bfe0 con 0x7f5848066450 2024-01-24T14:14:49.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.390+0000 7f5846ffd700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f582805f080 0x7f5828061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:14:49.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.390+0000 7f5846ffd700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f582805f080 0x7f5828061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f58480bfab0 tx=0x7f5838009060).ready entity=mgr.7117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:14:49.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.394+0000 7f583f7fe700 1 -- 172.21.15.29:0/885901782 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f58480660a0 con 0x7f5848066450 2024-01-24T14:14:49.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.603+0000 7f584e06b700 1 -- 172.21.15.29:0/885901782 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58480c2480 con 0x7f582805f080 2024-01-24T14:14:49.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.606+0000 7f583f7fe700 1 -- 172.21.15.29:0/885901782 <== mgr.7117 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7f5838011a20 con 0x7f582805f080 2024-01-24T14:14:49.606 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:14:49.606 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T14:14:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2024-01-24T14:14:49.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.608+0000 7f584e06b700 1 -- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f582805f080 msgr2=0x7f5828061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:49.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.608+0000 7f584e06b700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f582805f080 0x7f5828061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f58480bfab0 tx=0x7f5838009060).stop 2024-01-24T14:14:49.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.608+0000 7f584e06b700 1 -- 172.21.15.29:0/885901782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5848066450 msgr2=0x7f58480c27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:14:49.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.608+0000 7f584e06b700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5848066450 0x7f58480c27d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f5830009fd0 tx=0x7f5830017000).stop 2024-01-24T14:14:49.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.608+0000 7f584e06b700 1 -- 172.21.15.29:0/885901782 shutdown_connections 2024-01-24T14:14:49.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.608+0000 7f584e06b700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5848155a10 0x7f584814f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.608+0000 7f584e06b700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f582805f080 0x7f5828061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.608+0000 7f584e06b700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5848066450 0x7f58480c27d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.608+0000 7f584e06b700 1 --2- 172.21.15.29:0/885901782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5848152540 0x7f584814f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:14:49.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.608+0000 7f584e06b700 1 -- 172.21.15.29:0/885901782 >> 172.21.15.29:0/885901782 conn(0x7f58480c8bd0 msgr2=0x7f58480c7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:14:49.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.608+0000 7f584e06b700 1 -- 172.21.15.29:0/885901782 shutdown_connections 2024-01-24T14:14:49.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:14:49.608+0000 7f584e06b700 1 -- 172.21.15.29:0/885901782 wait complete. 2024-01-24T14:14:49.621 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T14:14:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2024-01-24T14:14:58.964 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:14:58.967 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:15:00.230 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:00.229+0000 7ffaaee40700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:00.230 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:00.229+0000 7ffaaee40700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:00.230 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:00.229+0000 7ffaaee40700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:00.230 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:00.229+0000 7ffaaee40700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:00.230 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:00.229+0000 7ffaaee40700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:29.038 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:15:29.041 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:15:49.623 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2024-01-24T14:15:49.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.766+0000 7f9787d98700 1 Processor -- start 2024-01-24T14:15:49.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.766+0000 7f9787d98700 1 -- start start 2024-01-24T14:15:49.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.767+0000 7f9787d98700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9780134ce0 0x7f97801350c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:49.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.767+0000 7f9787d98700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9780068720 con 0x7f9780152f80 2024-01-24T14:15:49.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.767+0000 7f9787d98700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9780068bb0 con 0x7f9780066450 2024-01-24T14:15:49.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.767+0000 7f9787d98700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978014c9e0 con 0x7f9780134ce0 2024-01-24T14:15:49.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.767+0000 7f9786335700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9780152f80 0x7f978015bdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38708/0 (socket says 172.21.15.29:38708) 2024-01-24T14:15:49.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.767+0000 7f9786335700 1 -- 172.21.15.29:0/3456823967 learned_addr learned my addr 172.21.15.29:0/3456823967 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:49.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9785b34700 1 --2- 172.21.15.29:0/3456823967 >> v2:172.21.15.163:3300/0 conn(0x7f9780134ce0 0x7f97801350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:49.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499405994 0 0) 0x7f9780068720 con 0x7f9780152f80 2024-01-24T14:15:49.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f976c0033f0 con 0x7f9780152f80 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2284205001 0 0) 0x7f976c0033f0 con 0x7f9780152f80 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 >> v2:172.21.15.163:3300/0 conn(0x7f9780134ce0 msgr2=0x7f97801350c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 --2- 172.21.15.29:0/3456823967 >> v2:172.21.15.163:3300/0 conn(0x7f9780134ce0 0x7f97801350c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 >> v1:172.21.15.163:6789/0 conn(0x7f9780066450 legacy=0x7f9780152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9780067b10 con 0x7f9780152f80 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f97780017f0 con 0x7f9780152f80 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 >> v1:172.21.15.29:6789/0 conn(0x7f9780152f80 legacy=0x7f978015bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 --2- 172.21.15.29:0/3456823967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f976c0038e0 0x7f976c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 --2- 172.21.15.29:0/3456823967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f976c004690 0x7f976c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 --2- 172.21.15.29:0/3456823967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f976c005000 0x7f976c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978014c9e0 con 0x7f976c0038e0 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c00beb0 con 0x7f976c005000 2024-01-24T14:15:49.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c00c070 con 0x7f976c004690 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9785b34700 1 --2- 172.21.15.29:0/3456823967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f976c0038e0 0x7f976c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9785333700 1 --2- 172.21.15.29:0/3456823967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f976c004690 0x7f976c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.768+0000 7f9786335700 1 --2- 172.21.15.29:0/3456823967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f976c005000 0x7f976c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.769+0000 7f9785b34700 1 -- 172.21.15.29:0/3456823967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f976c004690 msgr2=0x7f976c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.769+0000 7f9785b34700 1 --2- 172.21.15.29:0/3456823967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f976c004690 0x7f976c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.769+0000 7f9785b34700 1 -- 172.21.15.29:0/3456823967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f976c005000 msgr2=0x7f976c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.769+0000 7f9785b34700 1 --2- 172.21.15.29:0/3456823967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f976c005000 0x7f976c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.769+0000 7f9785b34700 1 -- 172.21.15.29:0/3456823967 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c0033f0 con 0x7f976c0038e0 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.769+0000 7f9786335700 1 -- 172.21.15.29:0/3456823967 reap_dead start 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.769+0000 7f9785b34700 1 --2- 172.21.15.29:0/3456823967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f976c0038e0 0x7f976c003d00 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f9770009a80 tx=0x7f9770009060).ready entity=mon.0 client_cookie=8a18e215a6e4b3e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.769+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770009430 con 0x7f976c0038e0 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.769+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977000e640 con 0x7f976c0038e0 2024-01-24T14:15:49.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.769+0000 7f9784b32700 1 -- 172.21.15.29:0/3456823967 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977000e9c0 con 0x7f976c0038e0 2024-01-24T14:15:49.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.770+0000 7f9787d98700 1 -- 172.21.15.29:0/3456823967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f976c0038e0 msgr2=0x7f976c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:49.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.770+0000 7f9787d98700 1 --2- 172.21.15.29:0/3456823967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f976c0038e0 0x7f976c003d00 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f9770009a80 tx=0x7f9770009060).stop 2024-01-24T14:15:49.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.770+0000 7f9787d98700 1 -- 172.21.15.29:0/3456823967 shutdown_connections 2024-01-24T14:15:49.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.770+0000 7f9787d98700 1 --2- 172.21.15.29:0/3456823967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f976c0038e0 0x7f976c003d00 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:49.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.770+0000 7f9787d98700 1 -- 172.21.15.29:0/3456823967 >> 172.21.15.29:0/3456823967 conn(0x7f97800c8bd0 msgr2=0x7f97800c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:49.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.770+0000 7f9787d98700 1 -- 172.21.15.29:0/3456823967 shutdown_connections 2024-01-24T14:15:49.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.770+0000 7f9787d98700 1 -- 172.21.15.29:0/3456823967 wait complete. 2024-01-24T14:15:49.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.771+0000 7f9787d98700 1 Processor -- start 2024-01-24T14:15:49.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.771+0000 7f9787d98700 1 -- start start 2024-01-24T14:15:49.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9787d98700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f976c0038e0 0x7f9780138100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:49.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9787d98700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97800c29a0 0x7f97800ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:49.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9787d98700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9780165900 0x7f9780165d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:49.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9787d98700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97801385f0 con 0x7f97800c29a0 2024-01-24T14:15:49.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9787d98700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9780166270 con 0x7f9780165900 2024-01-24T14:15:49.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9787d98700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97800c2e20 con 0x7f976c0038e0 2024-01-24T14:15:49.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9785333700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97800c29a0 0x7f97800ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9785333700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97800c29a0 0x7f97800ae4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33054/0 (socket says 172.21.15.29:33054) 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9785333700 1 -- 172.21.15.29:0/2195733485 learned_addr learned my addr 172.21.15.29:0/2195733485 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9785b34700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f976c0038e0 0x7f9780138100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9786335700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9780165900 0x7f9780165d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9785333700 1 -- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f976c0038e0 msgr2=0x7f9780138100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9785333700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f976c0038e0 0x7f9780138100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9785333700 1 -- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9780165900 msgr2=0x7f9780165d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9785333700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9780165900 0x7f9780165d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.772+0000 7f9785333700 1 -- 172.21.15.29:0/2195733485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97800c1bc0 con 0x7f97800c29a0 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.773+0000 7f9785333700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97800c29a0 0x7f97800ae4e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f977c002920 tx=0x7f977c007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.773+0000 7f9775ffb700 1 -- 172.21.15.29:0/2195733485 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c011590 con 0x7f97800c29a0 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.773+0000 7f9775ffb700 1 -- 172.21.15.29:0/2195733485 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c0116f0 con 0x7f97800c29a0 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.773+0000 7f9787d98700 1 -- 172.21.15.29:0/2195733485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97800c1ea0 con 0x7f97800c29a0 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.773+0000 7f9775ffb700 1 -- 172.21.15.29:0/2195733485 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c011a70 con 0x7f97800c29a0 2024-01-24T14:15:49.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.773+0000 7f9787d98700 1 -- 172.21.15.29:0/2195733485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9780135670 con 0x7f97800c29a0 2024-01-24T14:15:49.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.774+0000 7f9787d98700 1 -- 172.21.15.29:0/2195733485 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9780135890 con 0x7f97800c29a0 2024-01-24T14:15:49.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.774+0000 7f9775ffb700 1 -- 172.21.15.29:0/2195733485 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 44) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f977c01c3b0 con 0x7f97800c29a0 2024-01-24T14:15:49.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.774+0000 7f9775ffb700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f976405f3e0 0x7f97640618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:49.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.775+0000 7f9775ffb700 1 -- 172.21.15.29:0/2195733485 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7f977c07a5a0 con 0x7f97800c29a0 2024-01-24T14:15:49.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.775+0000 7f9785b34700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f976405f3e0 0x7f97640618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:49.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.775+0000 7f9785b34700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f976405f3e0 0x7f97640618a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9770006d50 tx=0x7f977001a000).ready entity=mgr.7117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:49.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.779+0000 7f9775ffb700 1 -- 172.21.15.29:0/2195733485 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f977c04afa0 con 0x7f97800c29a0 2024-01-24T14:15:49.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.986+0000 7f9787d98700 1 -- 172.21.15.29:0/2195733485 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97800475e0 con 0x7f976405f3e0 2024-01-24T14:15:49.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.988+0000 7f9775ffb700 1 -- 172.21.15.29:0/2195733485 <== mgr.7117 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f97800475e0 con 0x7f976405f3e0 2024-01-24T14:15:49.989 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:15:49.989 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T14:14:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2024-01-24T14:15:49.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.990+0000 7f9787d98700 1 -- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f976405f3e0 msgr2=0x7f97640618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:49.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.990+0000 7f9787d98700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f976405f3e0 0x7f97640618a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9770006d50 tx=0x7f977001a000).stop 2024-01-24T14:15:49.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.990+0000 7f9787d98700 1 -- 172.21.15.29:0/2195733485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97800c29a0 msgr2=0x7f97800ae4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:49.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.990+0000 7f9787d98700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97800c29a0 0x7f97800ae4e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f977c002920 tx=0x7f977c007bb0).stop 2024-01-24T14:15:49.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.990+0000 7f9787d98700 1 -- 172.21.15.29:0/2195733485 shutdown_connections 2024-01-24T14:15:49.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.990+0000 7f9787d98700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9780165900 0x7f9780165d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:49.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.990+0000 7f9787d98700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f976c0038e0 0x7f9780138100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:49.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.990+0000 7f9787d98700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f976405f3e0 0x7f97640618a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:49.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.990+0000 7f9787d98700 1 --2- 172.21.15.29:0/2195733485 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97800c29a0 0x7f97800ae4e0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:49.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.990+0000 7f9787d98700 1 -- 172.21.15.29:0/2195733485 >> 172.21.15.29:0/2195733485 conn(0x7f97800c8bd0 msgr2=0x7f978014b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:49.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.990+0000 7f9787d98700 1 -- 172.21.15.29:0/2195733485 shutdown_connections 2024-01-24T14:15:49.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:49.991+0000 7f9787d98700 1 -- 172.21.15.29:0/2195733485 wait complete. 2024-01-24T14:15:50.004 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T14:14:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2024-01-24T14:15:50.005 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /bad-path 2024-01-24T14:15:50.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.138+0000 7ff26843d700 1 Processor -- start 2024-01-24T14:15:50.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.138+0000 7ff26843d700 1 -- start start 2024-01-24T14:15:50.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.139+0000 7ff26843d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2600ab430 0x7ff2600acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:50.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.139+0000 7ff26843d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2600aa420 con 0x7ff260069220 2024-01-24T14:15:50.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.139+0000 7ff26843d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2600aafa0 con 0x7ff260063c80 2024-01-24T14:15:50.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.139+0000 7ff26843d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2600698c0 con 0x7ff2600ab430 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.139+0000 7ff2659d8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2600ab430 0x7ff2600acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.139+0000 7ff2669da700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff260069220 0x7ff2600c3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38720/0 (socket says 172.21.15.29:38720) 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.139+0000 7ff2669da700 1 -- 172.21.15.29:0/3490256893 learned_addr learned my addr 172.21.15.29:0/3490256893 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.139+0000 7ff2659d8700 1 -- 172.21.15.29:0/3490256893 >> v1:172.21.15.163:6789/0 conn(0x7ff260063c80 legacy=0x7ff260068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.140+0000 7ff2659d8700 1 -- 172.21.15.29:0/3490256893 >> v1:172.21.15.29:6789/0 conn(0x7ff260069220 legacy=0x7ff2600c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.140+0000 7ff2659d8700 1 -- 172.21.15.29:0/3490256893 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2600aa670 con 0x7ff2600ab430 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.140+0000 7ff2651d7700 1 -- 172.21.15.29:0/3490256893 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886610191 0 0) 0x7ff2600aa420 con 0x7ff260069220 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.140+0000 7ff2659d8700 1 --2- 172.21.15.29:0/3490256893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2600ab430 0x7ff2600acaa0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7ff250009760 tx=0x7ff250009060).ready entity=mon.0 client_cookie=7044c0cfa28ba5e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.140+0000 7ff2651d7700 1 -- 172.21.15.29:0/3490256893 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25000b520 con 0x7ff2600ab430 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.140+0000 7ff2651d7700 1 -- 172.21.15.29:0/3490256893 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25000b680 con 0x7ff2600ab430 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.140+0000 7ff2651d7700 1 -- 172.21.15.29:0/3490256893 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25000ba00 con 0x7ff2600ab430 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.140+0000 7ff26843d700 1 -- 172.21.15.29:0/3490256893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2600ab430 msgr2=0x7ff2600acaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:50.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.140+0000 7ff26843d700 1 --2- 172.21.15.29:0/3490256893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2600ab430 0x7ff2600acaa0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7ff250009760 tx=0x7ff250009060).stop 2024-01-24T14:15:50.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.140+0000 7ff26843d700 1 -- 172.21.15.29:0/3490256893 shutdown_connections 2024-01-24T14:15:50.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.140+0000 7ff26843d700 1 --2- 172.21.15.29:0/3490256893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2600ab430 0x7ff2600acaa0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:50.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.140+0000 7ff26843d700 1 -- 172.21.15.29:0/3490256893 >> 172.21.15.29:0/3490256893 conn(0x7ff2600c7280 msgr2=0x7ff2600c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:50.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.141+0000 7ff26843d700 1 -- 172.21.15.29:0/3490256893 shutdown_connections 2024-01-24T14:15:50.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.141+0000 7ff26843d700 1 -- 172.21.15.29:0/3490256893 wait complete. 2024-01-24T14:15:50.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.142+0000 7ff26843d700 1 Processor -- start 2024-01-24T14:15:50.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.142+0000 7ff26843d700 1 -- start start 2024-01-24T14:15:50.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff26843d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260063c80 0x7ff26014a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:50.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff26843d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff260069220 0x7ff26014f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:50.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff26843d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff26014b6b0 0x7ff2600c1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:50.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff26843d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26014bbb0 con 0x7ff26014b6b0 2024-01-24T14:15:50.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff26843d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26014be90 con 0x7ff260063c80 2024-01-24T14:15:50.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff26843d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26014c170 con 0x7ff260069220 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff2669da700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff26014b6b0 0x7ff2600c1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff2669da700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff26014b6b0 0x7ff2600c1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33066/0 (socket says 172.21.15.29:33066) 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff2669da700 1 -- 172.21.15.29:0/239308602 learned_addr learned my addr 172.21.15.29:0/239308602 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff2659d8700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff260069220 0x7ff26014f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff2661d9700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260063c80 0x7ff26014a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff2669da700 1 -- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff260069220 msgr2=0x7ff26014f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff2669da700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff260069220 0x7ff26014f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff2669da700 1 -- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260063c80 msgr2=0x7ff26014a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff2669da700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260063c80 0x7ff26014a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff2669da700 1 -- 172.21.15.29:0/239308602 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2500092a0 con 0x7ff26014b6b0 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff2661d9700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260063c80 0x7ff26014a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.143+0000 7ff2669da700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff26014b6b0 0x7ff2600c1560 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7ff25800c790 tx=0x7ff258007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:50.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.144+0000 7ff2567fc700 1 -- 172.21.15.29:0/239308602 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258005d30 con 0x7ff26014b6b0 2024-01-24T14:15:50.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.144+0000 7ff2567fc700 1 -- 172.21.15.29:0/239308602 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff258007110 con 0x7ff26014b6b0 2024-01-24T14:15:50.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.144+0000 7ff26843d700 1 -- 172.21.15.29:0/239308602 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26014c450 con 0x7ff26014b6b0 2024-01-24T14:15:50.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.144+0000 7ff2567fc700 1 -- 172.21.15.29:0/239308602 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2580074d0 con 0x7ff26014b6b0 2024-01-24T14:15:50.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.144+0000 7ff26843d700 1 -- 172.21.15.29:0/239308602 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26014ab30 con 0x7ff26014b6b0 2024-01-24T14:15:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.145+0000 7ff26843d700 1 -- 172.21.15.29:0/239308602 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26014e440 con 0x7ff26014b6b0 2024-01-24T14:15:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.145+0000 7ff2567fc700 1 -- 172.21.15.29:0/239308602 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 44) v1 ==== 75679+0+0 (secure 0 0 0) 0x7ff258007840 con 0x7ff26014b6b0 2024-01-24T14:15:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.146+0000 7ff2567fc700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff24405f3e0 0x7ff2440618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.146+0000 7ff2567fc700 1 -- 172.21.15.29:0/239308602 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7ff25807abd0 con 0x7ff26014b6b0 2024-01-24T14:15:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.146+0000 7ff2661d9700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff24405f3e0 0x7ff2440618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:50.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.146+0000 7ff2661d9700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff24405f3e0 0x7ff2440618a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff25c002780 tx=0x7ff25c009040).ready entity=mgr.7117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:50.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.150+0000 7ff2567fc700 1 -- 172.21.15.29:0/239308602 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff26014e440 con 0x7ff26014b6b0 2024-01-24T14:15:50.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.363+0000 7ff26843d700 1 -- 172.21.15.29:0/239308602 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7ff26014e440 con 0x7ff24405f3e0 2024-01-24T14:15:50.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.368+0000 7ff2567fc700 1 -- 172.21.15.29:0/239308602 <== mgr.7117 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7ff26014e440 con 0x7ff24405f3e0 2024-01-24T14:15:50.368 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /bad-path 2024-01-24T14:15:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.369+0000 7ff26843d700 1 -- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff24405f3e0 msgr2=0x7ff2440618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.369+0000 7ff26843d700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff24405f3e0 0x7ff2440618a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff25c002780 tx=0x7ff25c009040).stop 2024-01-24T14:15:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.370+0000 7ff26843d700 1 -- 172.21.15.29:0/239308602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff26014b6b0 msgr2=0x7ff2600c1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.370+0000 7ff26843d700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff26014b6b0 0x7ff2600c1560 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7ff25800c790 tx=0x7ff258007f30).stop 2024-01-24T14:15:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.370+0000 7ff26843d700 1 -- 172.21.15.29:0/239308602 shutdown_connections 2024-01-24T14:15:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.370+0000 7ff26843d700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff24405f3e0 0x7ff2440618a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.370+0000 7ff26843d700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff26014b6b0 0x7ff2600c1560 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.370+0000 7ff26843d700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260063c80 0x7ff26014a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.370+0000 7ff26843d700 1 --2- 172.21.15.29:0/239308602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff260069220 0x7ff26014f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.370+0000 7ff26843d700 1 -- 172.21.15.29:0/239308602 >> 172.21.15.29:0/239308602 conn(0x7ff2600c7280 msgr2=0x7ff2600c0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.370+0000 7ff26843d700 1 -- 172.21.15.29:0/239308602 shutdown_connections 2024-01-24T14:15:50.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.370+0000 7ff26843d700 1 -- 172.21.15.29:0/239308602 wait complete. 2024-01-24T14:15:50.383 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2024-01-24T14:15:50.384 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-01-24T14:15:50.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.516+0000 7f18e30b8700 1 Processor -- start 2024-01-24T14:15:50.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.516+0000 7f18e30b8700 1 -- start start 2024-01-24T14:15:50.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18e30b8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18dc134e70 0x7f18dc0c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:50.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18e30b8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f18dc068900 0x7f18dc152e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:50.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18e30b8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18dc068470 con 0x7f18dc066520 2024-01-24T14:15:50.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18e30b8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc14c910 con 0x7f18dc068900 2024-01-24T14:15:50.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18e30b8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc153340 con 0x7f18dc134e70 2024-01-24T14:15:50.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18dbfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f18dc068900 0x7f18dc152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:50.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18dbfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f18dc068900 0x7f18dc152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33070/0 (socket says 172.21.15.29:33070) 2024-01-24T14:15:50.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18dbfff700 1 -- 172.21.15.29:0/3232886206 learned_addr learned my addr 172.21.15.29:0/3232886206 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:50.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18e1655700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f18dc066520 0x7f18dc1559e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38736/0 (socket says 172.21.15.29:38736) 2024-01-24T14:15:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18e0e54700 1 --2- 172.21.15.29:0/3232886206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18dc134e70 0x7f18dc0c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18dbfff700 1 -- 172.21.15.29:0/3232886206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18dc134e70 msgr2=0x7f18dc0c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18dbfff700 1 --2- 172.21.15.29:0/3232886206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18dc134e70 0x7f18dc0c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18dbfff700 1 -- 172.21.15.29:0/3232886206 >> v1:172.21.15.29:6789/0 conn(0x7f18dc066520 legacy=0x7f18dc1559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18dbfff700 1 -- 172.21.15.29:0/3232886206 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0c3860 con 0x7f18dc068900 2024-01-24T14:15:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18db7fe700 1 -- 172.21.15.29:0/3232886206 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095409718 0 0) 0x7f18dc068470 con 0x7f18dc066520 2024-01-24T14:15:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.517+0000 7f18dbfff700 1 --2- 172.21.15.29:0/3232886206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18dc068900 0x7f18dc152e50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f18c8006fd0 tx=0x7f18c80090f0).ready entity=mon.0 client_cookie=29336e73ba63b572 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.518+0000 7f18db7fe700 1 -- 172.21.15.29:0/3232886206 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c800b600 con 0x7f18dc068900 2024-01-24T14:15:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.518+0000 7f18db7fe700 1 -- 172.21.15.29:0/3232886206 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c800b760 con 0x7f18dc068900 2024-01-24T14:15:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.518+0000 7f18db7fe700 1 -- 172.21.15.29:0/3232886206 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c800bae0 con 0x7f18dc068900 2024-01-24T14:15:50.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.518+0000 7f18e30b8700 1 -- 172.21.15.29:0/3232886206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18dc068900 msgr2=0x7f18dc152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:50.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.518+0000 7f18e30b8700 1 --2- 172.21.15.29:0/3232886206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18dc068900 0x7f18dc152e50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f18c8006fd0 tx=0x7f18c80090f0).stop 2024-01-24T14:15:50.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.518+0000 7f18e30b8700 1 -- 172.21.15.29:0/3232886206 shutdown_connections 2024-01-24T14:15:50.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.518+0000 7f18e30b8700 1 --2- 172.21.15.29:0/3232886206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18dc134e70 0x7f18dc0c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:50.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.518+0000 7f18e30b8700 1 --2- 172.21.15.29:0/3232886206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18dc068900 0x7f18dc152e50 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:50.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.518+0000 7f18e30b8700 1 -- 172.21.15.29:0/3232886206 >> 172.21.15.29:0/3232886206 conn(0x7f18dc0c8ba0 msgr2=0x7f18dc0c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:50.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.518+0000 7f18e30b8700 1 -- 172.21.15.29:0/3232886206 shutdown_connections 2024-01-24T14:15:50.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.518+0000 7f18e30b8700 1 -- 172.21.15.29:0/3232886206 wait complete. 2024-01-24T14:15:50.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.519+0000 7f18e30b8700 1 Processor -- start 2024-01-24T14:15:50.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.520+0000 7f18e30b8700 1 -- start start 2024-01-24T14:15:50.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.520+0000 7f18e30b8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18dc066520 0x7f18dc134480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e30b8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18dc068900 0x7f18dc0abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e30b8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18dc134e70 0x7f18dc0ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e30b8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc156be0 con 0x7f18dc066520 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e30b8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc156e50 con 0x7f18dc134e70 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e30b8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc157130 con 0x7f18dc068900 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e0e54700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18dc066520 0x7f18dc134480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e0e54700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18dc066520 0x7f18dc134480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33086/0 (socket says 172.21.15.29:33086) 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e0e54700 1 -- 172.21.15.29:0/461329754 learned_addr learned my addr 172.21.15.29:0/461329754 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18dbfff700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18dc068900 0x7f18dc0abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e1655700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18dc134e70 0x7f18dc0ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e0e54700 1 -- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18dc068900 msgr2=0x7f18dc0abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e0e54700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18dc068900 0x7f18dc0abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:50.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e0e54700 1 -- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18dc134e70 msgr2=0x7f18dc0ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:50.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e0e54700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18dc134e70 0x7f18dc0ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:50.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e0e54700 1 -- 172.21.15.29:0/461329754 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8009370 con 0x7f18dc066520 2024-01-24T14:15:50.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e0e54700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18dc066520 0x7f18dc134480 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f18d000d890 tx=0x7f18d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:50.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18d8ff9700 1 -- 172.21.15.29:0/461329754 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d00125f0 con 0x7f18dc066520 2024-01-24T14:15:50.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18d8ff9700 1 -- 172.21.15.29:0/461329754 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d0012750 con 0x7f18dc066520 2024-01-24T14:15:50.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18d8ff9700 1 -- 172.21.15.29:0/461329754 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0012ad0 con 0x7f18dc066520 2024-01-24T14:15:50.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e30b8700 1 -- 172.21.15.29:0/461329754 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc157410 con 0x7f18dc066520 2024-01-24T14:15:50.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.521+0000 7f18e30b8700 1 -- 172.21.15.29:0/461329754 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc134970 con 0x7f18dc066520 2024-01-24T14:15:50.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.523+0000 7f18e30b8700 1 -- 172.21.15.29:0/461329754 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc0475e0 con 0x7f18dc066520 2024-01-24T14:15:50.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.523+0000 7f18d8ff9700 1 -- 172.21.15.29:0/461329754 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 44) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f18d0013480 con 0x7f18dc066520 2024-01-24T14:15:50.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.523+0000 7f18d8ff9700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f18c005f3e0 0x7f18c00618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:50.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.524+0000 7f18d8ff9700 1 -- 172.21.15.29:0/461329754 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7f18d007a8e0 con 0x7f18dc066520 2024-01-24T14:15:50.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.524+0000 7f18dbfff700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f18c005f3e0 0x7f18c00618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:50.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.524+0000 7f18dbfff700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f18c005f3e0 0x7f18c00618a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f18c8006fd0 tx=0x7f18c8017000).ready entity=mgr.7117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:50.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.528+0000 7f18d8ff9700 1 -- 172.21.15.29:0/461329754 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f18d004b2e0 con 0x7f18dc066520 2024-01-24T14:15:50.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:50.759+0000 7f18e30b8700 1 -- 172.21.15.29:0/461329754 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f18dc068df0 con 0x7f18dc066520 2024-01-24T14:15:51.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.342+0000 7f18d8ff9700 1 -- 172.21.15.29:0/461329754 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 45) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f18d0077e40 con 0x7f18dc066520 2024-01-24T14:15:51.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.343+0000 7f18d8ff9700 1 -- 172.21.15.29:0/461329754 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v45) v1 ==== 93+0+0 (secure 0 0 0) 0x7f18d0073170 con 0x7f18dc066520 2024-01-24T14:15:51.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.346+0000 7f18e30b8700 1 -- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f18c005f3e0 msgr2=0x7f18c00618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:51.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.346+0000 7f18e30b8700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f18c005f3e0 0x7f18c00618a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f18c8006fd0 tx=0x7f18c8017000).stop 2024-01-24T14:15:51.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.346+0000 7f18e30b8700 1 -- 172.21.15.29:0/461329754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18dc066520 msgr2=0x7f18dc134480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:51.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.346+0000 7f18e30b8700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18dc066520 0x7f18dc134480 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f18d000d890 tx=0x7f18d0008040).stop 2024-01-24T14:15:51.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.346+0000 7f18e30b8700 1 -- 172.21.15.29:0/461329754 shutdown_connections 2024-01-24T14:15:51.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.346+0000 7f18e30b8700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18dc134e70 0x7f18dc0ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:51.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.346+0000 7f18e30b8700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f18c005f3e0 0x7f18c00618a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:51.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.346+0000 7f18e30b8700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18dc066520 0x7f18dc134480 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:51.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.346+0000 7f18e30b8700 1 --2- 172.21.15.29:0/461329754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18dc068900 0x7f18dc0abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:51.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.346+0000 7f18e30b8700 1 -- 172.21.15.29:0/461329754 >> 172.21.15.29:0/461329754 conn(0x7f18dc0c8ba0 msgr2=0x7f18dc1541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:51.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.346+0000 7f18e30b8700 1 -- 172.21.15.29:0/461329754 shutdown_connections 2024-01-24T14:15:51.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.346+0000 7f18e30b8700 1 -- 172.21.15.29:0/461329754 wait complete. 2024-01-24T14:15:51.362 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T14:15:51.363 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:15:51.408 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:15:51.450 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T14:15:51.450 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-24T14:15:51.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:15:51.450 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-24T14:15:51.490 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T14:15:51.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:15:51.491 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-24T14:15:51.492 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:51.492+0000 7f91b8078680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:15:51.505 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:51.505+0000 7fadf8e6f680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:15:51.523 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:15:51.555 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:51.554+0000 7f91b8078680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:15:51.581 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:51.581+0000 7fadf8e6f680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:15:51.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.696+0000 7f87a5ced700 1 Processor -- start 2024-01-24T14:15:51.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.697+0000 7f87a5ced700 1 -- start start 2024-01-24T14:15:51.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.697+0000 7f87a5ced700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f87a0134cc0 0x7f87a00c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:51.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.697+0000 7f87a5ced700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f87a01350a0 0x7f87a0152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:51.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.697+0000 7f87a5ced700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a0068360 con 0x7f87a00664f0 2024-01-24T14:15:51.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.697+0000 7f87a5ced700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a0153360 con 0x7f87a01350a0 2024-01-24T14:15:51.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.697+0000 7f87a5ced700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a00c3820 con 0x7f87a0134cc0 2024-01-24T14:15:51.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.698+0000 7f879effd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f87a01350a0 0x7f87a0152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:51.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.698+0000 7f879effd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f87a01350a0 0x7f87a0152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33116/0 (socket says 172.21.15.29:33116) 2024-01-24T14:15:51.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.698+0000 7f879effd700 1 -- 172.21.15.29:0/818820839 learned_addr learned my addr 172.21.15.29:0/818820839 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:51.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.698+0000 7f879f7fe700 1 --2- 172.21.15.29:0/818820839 >> v2:172.21.15.163:3300/0 conn(0x7f87a0134cc0 0x7f87a00c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.698+0000 7f879effd700 1 -- 172.21.15.29:0/818820839 >> v2:172.21.15.163:3300/0 conn(0x7f87a0134cc0 msgr2=0x7f87a00c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.698+0000 7f879effd700 1 --2- 172.21.15.29:0/818820839 >> v2:172.21.15.163:3300/0 conn(0x7f87a0134cc0 0x7f87a00c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.698+0000 7f879effd700 1 -- 172.21.15.29:0/818820839 >> v1:172.21.15.163:6789/0 conn(0x7f87a00664f0 legacy=0x7f87a0155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.698+0000 7f879e7fc700 1 -- 172.21.15.29:0/818820839 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284144151 0 0) 0x7f87a0068360 con 0x7f87a00664f0 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.698+0000 7f879effd700 1 -- 172.21.15.29:0/818820839 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a0067a30 con 0x7f87a01350a0 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.698+0000 7f879effd700 1 --2- 172.21.15.29:0/818820839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87a01350a0 0x7f87a0152e70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f87a00aee60 tx=0x7f87880090f0).ready entity=mon.0 client_cookie=463d006e04dbb871 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.698+0000 7f879e7fc700 1 -- 172.21.15.29:0/818820839 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878800b520 con 0x7f87a01350a0 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.698+0000 7f879e7fc700 1 -- 172.21.15.29:0/818820839 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878800b680 con 0x7f87a01350a0 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.699+0000 7f879e7fc700 1 -- 172.21.15.29:0/818820839 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878800ba00 con 0x7f87a01350a0 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.699+0000 7f87a5ced700 1 -- 172.21.15.29:0/818820839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87a01350a0 msgr2=0x7f87a0152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.699+0000 7f87a5ced700 1 --2- 172.21.15.29:0/818820839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87a01350a0 0x7f87a0152e70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f87a00aee60 tx=0x7f87880090f0).stop 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.699+0000 7f87a5ced700 1 -- 172.21.15.29:0/818820839 shutdown_connections 2024-01-24T14:15:51.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.699+0000 7f87a5ced700 1 --2- 172.21.15.29:0/818820839 >> v2:172.21.15.163:3300/0 conn(0x7f87a0134cc0 0x7f87a00c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:51.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.699+0000 7f87a5ced700 1 --2- 172.21.15.29:0/818820839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87a01350a0 0x7f87a0152e70 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:51.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.699+0000 7f87a5ced700 1 -- 172.21.15.29:0/818820839 >> 172.21.15.29:0/818820839 conn(0x7f87a00c8bb0 msgr2=0x7f87a00c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:51.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.699+0000 7f87a5ced700 1 -- 172.21.15.29:0/818820839 shutdown_connections 2024-01-24T14:15:51.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.699+0000 7f87a5ced700 1 -- 172.21.15.29:0/818820839 wait complete. 2024-01-24T14:15:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.701+0000 7f87a5ced700 1 Processor -- start 2024-01-24T14:15:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.701+0000 7f87a5ced700 1 -- start start 2024-01-24T14:15:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.702+0000 7f87a5ced700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87a00664f0 0x7f87a00bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:51.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.702+0000 7f87a5ced700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0134cc0 0x7f87a0164f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.702+0000 7f87a5ced700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87a01350a0 0x7f87a0137c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.702+0000 7f87a5ced700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a0165440 con 0x7f87a01350a0 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.702+0000 7f87a5ced700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a0136ee0 con 0x7f87a0134cc0 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.702+0000 7f87a5ced700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00abb60 con 0x7f87a00664f0 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.702+0000 7f879ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87a01350a0 0x7f87a0137c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.702+0000 7f879ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87a01350a0 0x7f87a0137c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33128/0 (socket says 172.21.15.29:33128) 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.702+0000 7f879ffff700 1 -- 172.21.15.29:0/2875417251 learned_addr learned my addr 172.21.15.29:0/2875417251 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.702+0000 7f879f7fe700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87a00664f0 0x7f87a00bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.702+0000 7f879effd700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0134cc0 0x7f87a0164f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.703+0000 7f879ffff700 1 -- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87a00664f0 msgr2=0x7f87a00bf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.703+0000 7f879ffff700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87a00664f0 0x7f87a00bf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.703+0000 7f879ffff700 1 -- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0134cc0 msgr2=0x7f87a0164f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:51.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.703+0000 7f879ffff700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0134cc0 0x7f87a0164f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.703+0000 7f879ffff700 1 -- 172.21.15.29:0/2875417251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8788009370 con 0x7f87a01350a0 2024-01-24T14:15:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.703+0000 7f879ffff700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87a01350a0 0x7f87a0137c40 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f879800c490 tx=0x7f8798007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.703+0000 7f87977fe700 1 -- 172.21.15.29:0/2875417251 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798012400 con 0x7f87a01350a0 2024-01-24T14:15:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.703+0000 7f87977fe700 1 -- 172.21.15.29:0/2875417251 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798012580 con 0x7f87a01350a0 2024-01-24T14:15:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.703+0000 7f87a5ced700 1 -- 172.21.15.29:0/2875417251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a0137170 con 0x7f87a01350a0 2024-01-24T14:15:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.703+0000 7f87977fe700 1 -- 172.21.15.29:0/2875417251 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798012940 con 0x7f87a01350a0 2024-01-24T14:15:51.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.703+0000 7f87a5ced700 1 -- 172.21.15.29:0/2875417251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a0137730 con 0x7f87a01350a0 2024-01-24T14:15:51.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.705+0000 7f87a5ced700 1 -- 172.21.15.29:0/2875417251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a00475e0 con 0x7f87a01350a0 2024-01-24T14:15:51.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.705+0000 7f87977fe700 1 -- 172.21.15.29:0/2875417251 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 45) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f879801b3b0 con 0x7f87a01350a0 2024-01-24T14:15:51.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.706+0000 7f87977fe700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f878005f390 0x7f8780061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:51.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.706+0000 7f87977fe700 1 -- 172.21.15.29:0/2875417251 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7f879807a0a0 con 0x7f87a01350a0 2024-01-24T14:15:51.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.706+0000 7f879f7fe700 1 -- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f878005f390 msgr2=0x7f8780061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:51.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.706+0000 7f879f7fe700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f878005f390 0x7f8780061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:15:51.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.710+0000 7f87977fe700 1 -- 172.21.15.29:0/2875417251 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8798048b00 con 0x7f87a01350a0 2024-01-24T14:15:51.832 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:51.831+0000 7f91b8078680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:15:51.900 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:51.899+0000 7fadf8e6f680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:15:51.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.907+0000 7f879f7fe700 1 -- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f878005f390 msgr2=0x7f8780061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:51.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.907+0000 7f879f7fe700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f878005f390 0x7f8780061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:15:51.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.938+0000 7f87a5ced700 1 -- 172.21.15.29:0/2875417251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87a0068920 con 0x7f87a01350a0 2024-01-24T14:15:51.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.939+0000 7f87977fe700 1 -- 172.21.15.29:0/2875417251 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f879804aa90 con 0x7f87a01350a0 2024-01-24T14:15:51.939 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:15:51.940 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6341,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4230829295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4230829295},{"type":"v1","addr":"172.21.15.29:6835","nonce":4230829295}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6344,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3194389571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3194389571},{"type":"v1","addr":"172.21.15.163:6836","nonce":3194389571}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2473497045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2473497045},{"type":"v1","addr":"172.21.15.163:6837","nonce":2473497045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:14:35.870228+0000","modified":"2024-01-24T14:14:38.905491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7254":{"gid":7254,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/3097015540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3097015540},{"type":"v1","addr":"172.21.15.29:6834","nonce":3097015540}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-24T14:15:51.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.941+0000 7f87a5ced700 1 -- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f878005f390 msgr2=0x7f8780061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:15:51.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.941+0000 7f87a5ced700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f878005f390 0x7f8780061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:51.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.941+0000 7f87a5ced700 1 -- 172.21.15.29:0/2875417251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87a01350a0 msgr2=0x7f87a0137c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:51.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.941+0000 7f87a5ced700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87a01350a0 0x7f87a0137c40 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f879800c490 tx=0x7f8798007bf0).stop 2024-01-24T14:15:51.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.941+0000 7f87a5ced700 1 -- 172.21.15.29:0/2875417251 shutdown_connections 2024-01-24T14:15:51.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.941+0000 7f87a5ced700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f878005f390 0x7f8780061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:51.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.941+0000 7f87a5ced700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87a01350a0 0x7f87a0137c40 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:51.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.941+0000 7f87a5ced700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87a00664f0 0x7f87a00bf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:51.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.941+0000 7f87a5ced700 1 --2- 172.21.15.29:0/2875417251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0134cc0 0x7f87a0164f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:51.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.941+0000 7f87a5ced700 1 -- 172.21.15.29:0/2875417251 >> 172.21.15.29:0/2875417251 conn(0x7f87a00c8bb0 msgr2=0x7f87a00c6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:51.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.941+0000 7f87a5ced700 1 -- 172.21.15.29:0/2875417251 shutdown_connections 2024-01-24T14:15:51.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:51.941+0000 7f87a5ced700 1 -- 172.21.15.29:0/2875417251 wait complete. 2024-01-24T14:15:51.943 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 51 2024-01-24T14:15:51.956 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:15:52.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.127+0000 7fca89f76700 1 Processor -- start 2024-01-24T14:15:52.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.127+0000 7fca89f76700 1 -- start start 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.128+0000 7fca89f76700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fca840638e0 0x7fca840aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.128+0000 7fca89f76700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca840aa6c0 con 0x7fca840c5410 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.128+0000 7fca89f76700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca84064240 con 0x7fca84154370 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.128+0000 7fca89f76700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca840cb5c0 con 0x7fca840638e0 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.128+0000 7fca83fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fca840638e0 0x7fca840aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.128+0000 7fca82ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fca840c5410 0x7fca840631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38764/0 (socket says 172.21.15.29:38764) 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.128+0000 7fca82ffd700 1 -- 172.21.15.29:0/1086686014 learned_addr learned my addr 172.21.15.29:0/1086686014 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.129+0000 7fca83fff700 1 -- 172.21.15.29:0/1086686014 >> v1:172.21.15.163:6789/0 conn(0x7fca84154370 legacy=0x7fca840ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.129+0000 7fca83fff700 1 -- 172.21.15.29:0/1086686014 >> v1:172.21.15.29:6789/0 conn(0x7fca840c5410 legacy=0x7fca840631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.129+0000 7fca83fff700 1 -- 172.21.15.29:0/1086686014 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca840cb8a0 con 0x7fca840638e0 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.129+0000 7fca827fc700 1 -- 172.21.15.29:0/1086686014 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250280238 0 0) 0x7fca840aa6c0 con 0x7fca840c5410 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.129+0000 7fca83fff700 1 --2- 172.21.15.29:0/1086686014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca840638e0 0x7fca840aa0e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7fca7c009e60 tx=0x7fca7c009780).ready entity=mon.0 client_cookie=e2a434b40a7312a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.129+0000 7fca827fc700 1 -- 172.21.15.29:0/1086686014 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c00e070 con 0x7fca840638e0 2024-01-24T14:15:52.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.129+0000 7fca827fc700 1 -- 172.21.15.29:0/1086686014 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7c00bc30 con 0x7fca840638e0 2024-01-24T14:15:52.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.129+0000 7fca827fc700 1 -- 172.21.15.29:0/1086686014 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c012410 con 0x7fca840638e0 2024-01-24T14:15:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.130+0000 7fca89f76700 1 -- 172.21.15.29:0/1086686014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca840638e0 msgr2=0x7fca840aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.130+0000 7fca89f76700 1 --2- 172.21.15.29:0/1086686014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca840638e0 0x7fca840aa0e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7fca7c009e60 tx=0x7fca7c009780).stop 2024-01-24T14:15:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.130+0000 7fca89f76700 1 -- 172.21.15.29:0/1086686014 shutdown_connections 2024-01-24T14:15:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.130+0000 7fca89f76700 1 --2- 172.21.15.29:0/1086686014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca840638e0 0x7fca840aa0e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.130+0000 7fca89f76700 1 -- 172.21.15.29:0/1086686014 >> 172.21.15.29:0/1086686014 conn(0x7fca8405ec80 msgr2=0x7fca840cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.130+0000 7fca89f76700 1 -- 172.21.15.29:0/1086686014 shutdown_connections 2024-01-24T14:15:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.130+0000 7fca89f76700 1 -- 172.21.15.29:0/1086686014 wait complete. 2024-01-24T14:15:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.131+0000 7fca89f76700 1 Processor -- start 2024-01-24T14:15:52.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.131+0000 7fca89f76700 1 -- start start 2024-01-24T14:15:52.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca89f76700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca840638e0 0x7fca8414c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca89f76700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca840c5410 0x7fca840adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca89f76700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca84154370 0x7fca84154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca89f76700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca840ac3c0 con 0x7fca840638e0 2024-01-24T14:15:52.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca89f76700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8414ce20 con 0x7fca840c5410 2024-01-24T14:15:52.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca89f76700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca840ab7b0 con 0x7fca84154370 2024-01-24T14:15:52.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca837fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca840638e0 0x7fca8414c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca837fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca840638e0 0x7fca8414c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33146/0 (socket says 172.21.15.29:33146) 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca837fe700 1 -- 172.21.15.29:0/3042164900 learned_addr learned my addr 172.21.15.29:0/3042164900 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca82ffd700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca840c5410 0x7fca840adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca83fff700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca84154370 0x7fca84154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca837fe700 1 -- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca84154370 msgr2=0x7fca84154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca837fe700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca84154370 0x7fca84154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca837fe700 1 -- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca840c5410 msgr2=0x7fca840adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca837fe700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca840c5410 0x7fca840adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.132+0000 7fca837fe700 1 -- 172.21.15.29:0/3042164900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c009a20 con 0x7fca840638e0 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.133+0000 7fca837fe700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca840638e0 0x7fca8414c5f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7fca7400c440 tx=0x7fca74007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.133+0000 7fca6b7fe700 1 -- 172.21.15.29:0/3042164900 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca74007020 con 0x7fca840638e0 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.133+0000 7fca6b7fe700 1 -- 172.21.15.29:0/3042164900 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca74007180 con 0x7fca840638e0 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.133+0000 7fca6b7fe700 1 -- 172.21.15.29:0/3042164900 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca74007520 con 0x7fca840638e0 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.133+0000 7fca89f76700 1 -- 172.21.15.29:0/3042164900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca840aba90 con 0x7fca840638e0 2024-01-24T14:15:52.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.133+0000 7fca89f76700 1 -- 172.21.15.29:0/3042164900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca840ac050 con 0x7fca840638e0 2024-01-24T14:15:52.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.134+0000 7fca89f76700 1 -- 172.21.15.29:0/3042164900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca840475e0 con 0x7fca840638e0 2024-01-24T14:15:52.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.134+0000 7fca6b7fe700 1 -- 172.21.15.29:0/3042164900 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 45) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fca74006900 con 0x7fca840638e0 2024-01-24T14:15:52.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.135+0000 7fca6b7fe700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca7005f390 0x7fca70061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.135+0000 7fca6b7fe700 1 -- 172.21.15.29:0/3042164900 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7fca74079de0 con 0x7fca840638e0 2024-01-24T14:15:52.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.135+0000 7fca82ffd700 1 -- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca7005f390 msgr2=0x7fca70061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:52.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.135+0000 7fca82ffd700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca7005f390 0x7fca70061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:15:52.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.139+0000 7fca6b7fe700 1 -- 172.21.15.29:0/3042164900 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fca74047b00 con 0x7fca840638e0 2024-01-24T14:15:52.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.324+0000 7fca89f76700 1 -- 172.21.15.29:0/3042164900 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca840cbb80 con 0x7fca840638e0 2024-01-24T14:15:52.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.325+0000 7fca6b7fe700 1 -- 172.21.15.29:0/3042164900 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+18752 (secure 0 0 0) 0x7fca7404a840 con 0x7fca840638e0 2024-01-24T14:15:52.326 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:15:52.326 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":123,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:15:49.269646+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:14:33.610829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-24T14:14:34.834814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.18","mappings":[{"from":1,"to":7}]},{"pgid":"14.29","mappings":[{"from":1,"to":2}]},{"pgid":"14.36","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/2260886250":"2024-01-25T14:14:48.183216+0000","172.21.15.163:6833/43241":"2024-01-25T14:14:48.183216+0000","172.21.15.163:6832/43241":"2024-01-25T14:14:48.183216+0000","172.21.15.163:0/2891867602":"2024-01-25T14:14:48.183216+0000","172.21.15.163:0/3822418593":"2024-01-25T14:14:48.183216+0000","172.21.15.163:0/4052979774":"2024-01-25T14:14:48.183216+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:15:52.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.327+0000 7fca89f76700 1 -- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca7005f390 msgr2=0x7fca70061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:15:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.327+0000 7fca89f76700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca7005f390 0x7fca70061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.327+0000 7fca89f76700 1 -- 172.21.15.29:0/3042164900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca840638e0 msgr2=0x7fca8414c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.327+0000 7fca89f76700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca840638e0 0x7fca8414c5f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7fca7400c440 tx=0x7fca74007e50).stop 2024-01-24T14:15:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.327+0000 7fca89f76700 1 -- 172.21.15.29:0/3042164900 shutdown_connections 2024-01-24T14:15:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.327+0000 7fca89f76700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca84154370 0x7fca84154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.327+0000 7fca89f76700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fca7005f390 0x7fca70061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.327+0000 7fca89f76700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca840638e0 0x7fca8414c5f0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.327+0000 7fca89f76700 1 --2- 172.21.15.29:0/3042164900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca840c5410 0x7fca840adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.327+0000 7fca89f76700 1 -- 172.21.15.29:0/3042164900 >> 172.21.15.29:0/3042164900 conn(0x7fca8405ec80 msgr2=0x7fca840c7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.327+0000 7fca89f76700 1 -- 172.21.15.29:0/3042164900 shutdown_connections 2024-01-24T14:15:52.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.328+0000 7fca89f76700 1 -- 172.21.15.29:0/3042164900 wait complete. 2024-01-24T14:15:52.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T14:14:33.610829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T14:14:34.834814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T14:15:52.343 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T14:15:52.449 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:52.448+0000 7f91b8078680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:15:52.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.477+0000 7fa5e7e2a700 1 Processor -- start 2024-01-24T14:15:52.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.477+0000 7fa5e7e2a700 1 -- start start 2024-01-24T14:15:52.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.479+0000 7fa5e7e2a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e0062ec0 0x7fa5e006a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.479+0000 7fa5e7e2a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa5e00c8270 0x7fa5e01391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.479+0000 7fa5e7e2a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e00bcda0 con 0x7fa5e01384f0 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.479+0000 7fa5e7e2a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e0065f80 con 0x7fa5e00c8270 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.479+0000 7fa5e7e2a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e0066260 con 0x7fa5e0062ec0 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.479+0000 7fa5e53c5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e0062ec0 0x7fa5e006a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.479+0000 7fa5e63c7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa5e00c8270 0x7fa5e01391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.479+0000 7fa5e5bc6700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa5e01384f0 0x7fa5e00627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:39764/0 (socket says 172.21.15.29:39764) 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.479+0000 7fa5e5bc6700 1 -- 172.21.15.29:0/576883773 learned_addr learned my addr 172.21.15.29:0/576883773 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.480+0000 7fa5e4bc4700 1 -- 172.21.15.29:0/576883773 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903448325 0 0) 0x7fa5e00bcda0 con 0x7fa5e01384f0 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.480+0000 7fa5e53c5700 1 -- 172.21.15.29:0/576883773 >> v2:172.21.15.163:3300/0 conn(0x7fa5e00c8270 msgr2=0x7fa5e01391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.480+0000 7fa5e53c5700 1 --2- 172.21.15.29:0/576883773 >> v2:172.21.15.163:3300/0 conn(0x7fa5e00c8270 0x7fa5e01391d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.480+0000 7fa5e53c5700 1 -- 172.21.15.29:0/576883773 >> v1:172.21.15.163:6789/0 conn(0x7fa5e01384f0 legacy=0x7fa5e00627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.480+0000 7fa5e53c5700 1 -- 172.21.15.29:0/576883773 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e00bca50 con 0x7fa5e0062ec0 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.480+0000 7fa5e63c7700 1 --2- 172.21.15.29:0/576883773 >> v2:172.21.15.163:3300/0 conn(0x7fa5e00c8270 0x7fa5e01391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.480+0000 7fa5e53c5700 1 --2- 172.21.15.29:0/576883773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e0062ec0 0x7fa5e006a460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fa5d0009760 tx=0x7fa5d0009060).ready entity=mon.2 client_cookie=453517282c337726 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.480+0000 7fa5e4bc4700 1 -- 172.21.15.29:0/576883773 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d00090d0 con 0x7fa5e0062ec0 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.480+0000 7fa5e4bc4700 1 -- 172.21.15.29:0/576883773 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d000f510 con 0x7fa5e0062ec0 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.480+0000 7fa5e4bc4700 1 -- 172.21.15.29:0/576883773 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000f890 con 0x7fa5e0062ec0 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.481+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/576883773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e0062ec0 msgr2=0x7fa5e006a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.481+0000 7fa5e7e2a700 1 --2- 172.21.15.29:0/576883773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e0062ec0 0x7fa5e006a460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fa5d0009760 tx=0x7fa5d0009060).stop 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.481+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/576883773 shutdown_connections 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.481+0000 7fa5e7e2a700 1 --2- 172.21.15.29:0/576883773 >> v2:172.21.15.163:3300/0 conn(0x7fa5e00c8270 0x7fa5e01391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.481+0000 7fa5e7e2a700 1 --2- 172.21.15.29:0/576883773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e0062ec0 0x7fa5e006a460 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.481+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/576883773 >> 172.21.15.29:0/576883773 conn(0x7fa5e0064390 msgr2=0x7fa5e00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.481+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/576883773 shutdown_connections 2024-01-24T14:15:52.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.481+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/576883773 wait complete. 2024-01-24T14:15:52.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.483+0000 7fa5e7e2a700 1 Processor -- start 2024-01-24T14:15:52.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.483+0000 7fa5e7e2a700 1 -- start start 2024-01-24T14:15:52.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.484+0000 7fa5e7e2a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e0062ec0 0x7fa5e00ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.484+0000 7fa5e7e2a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e00c8270 0x7fa5e01519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.484+0000 7fa5e7e2a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e01384f0 0x7fa5e00ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.484+0000 7fa5e7e2a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e0155310 con 0x7fa5e01384f0 2024-01-24T14:15:52.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.484+0000 7fa5e7e2a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e0151eb0 con 0x7fa5e0062ec0 2024-01-24T14:15:52.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.484+0000 7fa5e7e2a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e00abc70 con 0x7fa5e00c8270 2024-01-24T14:15:52.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.484+0000 7fa5e63c7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e01384f0 0x7fa5e00ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.484+0000 7fa5e53c5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e00c8270 0x7fa5e01519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.484+0000 7fa5e5bc6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e0062ec0 0x7fa5e00ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.484+0000 7fa5e63c7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e01384f0 0x7fa5e00ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33162/0 (socket says 172.21.15.29:33162) 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.484+0000 7fa5e63c7700 1 -- 172.21.15.29:0/3507097382 learned_addr learned my addr 172.21.15.29:0/3507097382 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.485+0000 7fa5e53c5700 1 -- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e0062ec0 msgr2=0x7fa5e00ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.485+0000 7fa5e53c5700 1 --2- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e0062ec0 0x7fa5e00ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.485+0000 7fa5e53c5700 1 -- 172.21.15.29:0/3507097382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e01384f0 msgr2=0x7fa5e00ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.485+0000 7fa5e53c5700 1 --2- 172.21.15.29:0/3507097382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e01384f0 0x7fa5e00ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.485+0000 7fa5e53c5700 1 -- 172.21.15.29:0/3507097382 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d8009a20 con 0x7fa5e00c8270 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.485+0000 7fa5e53c5700 1 --2- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e00c8270 0x7fa5e01519c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fa5d0014000 tx=0x7fa5d0017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.485+0000 7fa5d5ffb700 1 -- 172.21.15.29:0/3507097382 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000f890 con 0x7fa5e00c8270 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.485+0000 7fa5d5ffb700 1 -- 172.21.15.29:0/3507097382 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d0000d10 con 0x7fa5e00c8270 2024-01-24T14:15:52.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.485+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/3507097382 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d00092a0 con 0x7fa5e00c8270 2024-01-24T14:15:52.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.485+0000 7fa5d5ffb700 1 -- 172.21.15.29:0/3507097382 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000fea0 con 0x7fa5e00c8270 2024-01-24T14:15:52.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.485+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/3507097382 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e00ac230 con 0x7fa5e00c8270 2024-01-24T14:15:52.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.487+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/3507097382 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e0155480 con 0x7fa5e00c8270 2024-01-24T14:15:52.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.487+0000 7fa5d5ffb700 1 -- 172.21.15.29:0/3507097382 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 45) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fa5d001b620 con 0x7fa5e00c8270 2024-01-24T14:15:52.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.487+0000 7fa5d5ffb700 1 --2- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa5c405f390 0x7fa5c4061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.487+0000 7fa5e5bc6700 1 -- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa5c405f390 msgr2=0x7fa5c4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:52.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.487+0000 7fa5d5ffb700 1 -- 172.21.15.29:0/3507097382 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7fa5d007c170 con 0x7fa5e00c8270 2024-01-24T14:15:52.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.487+0000 7fa5e5bc6700 1 --2- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa5c405f390 0x7fa5c4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:15:52.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.492+0000 7fa5d5ffb700 1 -- 172.21.15.29:0/3507097382 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa5d004cb10 con 0x7fa5e00c8270 2024-01-24T14:15:52.523 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:52.522+0000 7f91b8078680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:15:52.632 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:52.631+0000 7fadf8e6f680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:15:52.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.688+0000 7fa5e5bc6700 1 -- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa5c405f390 msgr2=0x7fa5c4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:52.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.688+0000 7fa5e5bc6700 1 --2- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa5c405f390 0x7fa5c4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:15:52.708 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:52.707+0000 7f91b8078680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:15:52.721 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:52.720+0000 7fadf8e6f680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:15:52.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.734+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/3507097382 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa5e00475e0 con 0x7fa5e00c8270 2024-01-24T14:15:52.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.735+0000 7fa5d5ffb700 1 -- 172.21.15.29:0/3507097382 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v123) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa5d001f0b0 con 0x7fa5e00c8270 2024-01-24T14:15:52.735 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T14:15:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.737+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa5c405f390 msgr2=0x7fa5c4061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:15:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.737+0000 7fa5e7e2a700 1 --2- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa5c405f390 0x7fa5c4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.737+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e00c8270 msgr2=0x7fa5e01519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.737+0000 7fa5e7e2a700 1 --2- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e00c8270 0x7fa5e01519c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fa5d0014000 tx=0x7fa5d0017000).stop 2024-01-24T14:15:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.737+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/3507097382 shutdown_connections 2024-01-24T14:15:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.737+0000 7fa5e7e2a700 1 --2- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa5c405f390 0x7fa5c4061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.737+0000 7fa5e7e2a700 1 --2- 172.21.15.29:0/3507097382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5e01384f0 0x7fa5e00ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.737+0000 7fa5e7e2a700 1 --2- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e0062ec0 0x7fa5e00ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.737+0000 7fa5e7e2a700 1 --2- 172.21.15.29:0/3507097382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e00c8270 0x7fa5e01519c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.737+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/3507097382 >> 172.21.15.29:0/3507097382 conn(0x7fa5e0064390 msgr2=0x7fa5e0137530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:52.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.737+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/3507097382 shutdown_connections 2024-01-24T14:15:52.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.737+0000 7fa5e7e2a700 1 -- 172.21.15.29:0/3507097382 wait complete. 2024-01-24T14:15:52.751 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T14:15:52.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.886+0000 7fe3efffc700 1 Processor -- start 2024-01-24T14:15:52.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.887+0000 7fe3efffc700 1 -- start start 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.888+0000 7fe3efffc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e8064760 0x7fe3e8064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.888+0000 7fe3efffc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe3e80c8490 0x7fe3e80a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.888+0000 7fe3efffc700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e8152240 con 0x7fe3e8154700 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.888+0000 7fe3efffc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e81586c0 con 0x7fe3e80c8490 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.888+0000 7fe3efffc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80c0d50 con 0x7fe3e8064760 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.888+0000 7fe3ee599700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe3e8154700 0x7fe3e8154bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38768/0 (socket says 172.21.15.29:38768) 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.888+0000 7fe3ee599700 1 -- 172.21.15.29:0/378259400 learned_addr learned my addr 172.21.15.29:0/378259400 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.888+0000 7fe3edd98700 1 --2- 172.21.15.29:0/378259400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e8064760 0x7fe3e8064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.888+0000 7fe3ed597700 1 --2- 172.21.15.29:0/378259400 >> v2:172.21.15.163:3300/0 conn(0x7fe3e80c8490 0x7fe3e80a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.888+0000 7fe3ecd96700 1 -- 172.21.15.29:0/378259400 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289057322 0 0) 0x7fe3e8152240 con 0x7fe3e8154700 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.888+0000 7fe3ecd96700 1 -- 172.21.15.29:0/378259400 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3d40033f0 con 0x7fe3e8154700 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.889+0000 7fe3edd98700 1 -- 172.21.15.29:0/378259400 >> v2:172.21.15.163:3300/0 conn(0x7fe3e80c8490 msgr2=0x7fe3e80a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.889+0000 7fe3edd98700 1 --2- 172.21.15.29:0/378259400 >> v2:172.21.15.163:3300/0 conn(0x7fe3e80c8490 0x7fe3e80a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.889+0000 7fe3edd98700 1 -- 172.21.15.29:0/378259400 >> v1:172.21.15.29:6789/0 conn(0x7fe3e8154700 legacy=0x7fe3e8154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.889+0000 7fe3edd98700 1 -- 172.21.15.29:0/378259400 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e80aa670 con 0x7fe3e8064760 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.889+0000 7fe3ed597700 1 --2- 172.21.15.29:0/378259400 >> v2:172.21.15.163:3300/0 conn(0x7fe3e80c8490 0x7fe3e80a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.889+0000 7fe3edd98700 1 --2- 172.21.15.29:0/378259400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e8064760 0x7fe3e8064b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fe3e4009760 tx=0x7fe3e4009060).ready entity=mon.2 client_cookie=b0602b80b21216c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.889+0000 7fe3ecd96700 1 -- 172.21.15.29:0/378259400 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400f510 con 0x7fe3e8064760 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.889+0000 7fe3ecd96700 1 -- 172.21.15.29:0/378259400 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e400f670 con 0x7fe3e8064760 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.889+0000 7fe3ecd96700 1 -- 172.21.15.29:0/378259400 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400f9f0 con 0x7fe3e8064760 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.890+0000 7fe3efffc700 1 -- 172.21.15.29:0/378259400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e8064760 msgr2=0x7fe3e8064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.890+0000 7fe3efffc700 1 --2- 172.21.15.29:0/378259400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e8064760 0x7fe3e8064b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fe3e4009760 tx=0x7fe3e4009060).stop 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.890+0000 7fe3efffc700 1 -- 172.21.15.29:0/378259400 shutdown_connections 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.890+0000 7fe3efffc700 1 --2- 172.21.15.29:0/378259400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e8064760 0x7fe3e8064b40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.890+0000 7fe3efffc700 1 --2- 172.21.15.29:0/378259400 >> v2:172.21.15.163:3300/0 conn(0x7fe3e80c8490 0x7fe3e80a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.890+0000 7fe3efffc700 1 -- 172.21.15.29:0/378259400 >> 172.21.15.29:0/378259400 conn(0x7fe3e80cb8b0 msgr2=0x7fe3e80c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.890+0000 7fe3efffc700 1 -- 172.21.15.29:0/378259400 shutdown_connections 2024-01-24T14:15:52.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.890+0000 7fe3efffc700 1 -- 172.21.15.29:0/378259400 wait complete. 2024-01-24T14:15:52.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.891+0000 7fe3efffc700 1 Processor -- start 2024-01-24T14:15:52.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.892+0000 7fe3efffc700 1 -- start start 2024-01-24T14:15:52.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.892+0000 7fe3efffc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e8064760 0x7fe3e81554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3efffc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3e80c8490 0x7fe3e80c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3efffc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e8154700 0x7fe3e81446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3efffc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e81559e0 con 0x7fe3e80c8490 2024-01-24T14:15:52.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3efffc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80c4310 con 0x7fe3e8154700 2024-01-24T14:15:52.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3efffc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8155c00 con 0x7fe3e8064760 2024-01-24T14:15:52.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ed597700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3e80c8490 0x7fe3e80c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ed597700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3e80c8490 0x7fe3e80c3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33164/0 (socket says 172.21.15.29:33164) 2024-01-24T14:15:52.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3edd98700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e8064760 0x7fe3e81554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ed597700 1 -- 172.21.15.29:0/642289411 learned_addr learned my addr 172.21.15.29:0/642289411 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ee599700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e8154700 0x7fe3e81446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ed597700 1 -- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e8064760 msgr2=0x7fe3e81554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ed597700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e8064760 0x7fe3e81554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ed597700 1 -- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e8154700 msgr2=0x7fe3e81446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ed597700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e8154700 0x7fe3e81446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ed597700 1 -- 172.21.15.29:0/642289411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d80092a0 con 0x7fe3e80c8490 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3edd98700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e8064760 0x7fe3e81554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ed597700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3e80c8490 0x7fe3e80c3e20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7fe3d800e9d0 tx=0x7fe3d8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ddffb700 1 -- 172.21.15.29:0/642289411 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800c6a0 con 0x7fe3e80c8490 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ddffb700 1 -- 172.21.15.29:0/642289411 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d800c800 con 0x7fe3e80c8490 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3ddffb700 1 -- 172.21.15.29:0/642289411 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800cb80 con 0x7fe3e80c8490 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.893+0000 7fe3efffc700 1 -- 172.21.15.29:0/642289411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e40092a0 con 0x7fe3e80c8490 2024-01-24T14:15:52.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.894+0000 7fe3efffc700 1 -- 172.21.15.29:0/642289411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e81564a0 con 0x7fe3e80c8490 2024-01-24T14:15:52.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.895+0000 7fe3ddffb700 1 -- 172.21.15.29:0/642289411 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 45) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fe3d801b400 con 0x7fe3e80c8490 2024-01-24T14:15:52.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.895+0000 7fe3efffc700 1 -- 172.21.15.29:0/642289411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e80475e0 con 0x7fe3e80c8490 2024-01-24T14:15:52.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.895+0000 7fe3ddffb700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3cc05f360 0x7fe3cc061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:52.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.896+0000 7fe3ddffb700 1 -- 172.21.15.29:0/642289411 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7fe3d807afc0 con 0x7fe3e80c8490 2024-01-24T14:15:52.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.896+0000 7fe3edd98700 1 -- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3cc05f360 msgr2=0x7fe3cc061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:52.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.896+0000 7fe3edd98700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3cc05f360 0x7fe3cc061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:15:52.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:52.900+0000 7fe3ddffb700 1 -- 172.21.15.29:0/642289411 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe3d804b9c0 con 0x7fe3e80c8490 2024-01-24T14:15:52.949 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:52.948+0000 7fadf8e6f680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:15:53.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.088+0000 7fe3efffc700 1 -- 172.21.15.29:0/642289411 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe3e8157d90 con 0x7fe3e80c8490 2024-01-24T14:15:53.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.089+0000 7fe3ddffb700 1 -- 172.21.15.29:0/642289411 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v123) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe3d8016020 con 0x7fe3e80c8490 2024-01-24T14:15:53.089 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T14:15:53.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.091+0000 7fe3efffc700 1 -- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3cc05f360 msgr2=0x7fe3cc061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:15:53.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.091+0000 7fe3efffc700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3cc05f360 0x7fe3cc061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:53.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.091+0000 7fe3efffc700 1 -- 172.21.15.29:0/642289411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3e80c8490 msgr2=0x7fe3e80c3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.091+0000 7fe3efffc700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3e80c8490 0x7fe3e80c3e20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7fe3d800e9d0 tx=0x7fe3d8009060).stop 2024-01-24T14:15:53.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.091+0000 7fe3efffc700 1 -- 172.21.15.29:0/642289411 shutdown_connections 2024-01-24T14:15:53.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.091+0000 7fe3efffc700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e8154700 0x7fe3e81446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.091+0000 7fe3efffc700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e8064760 0x7fe3e81554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.091+0000 7fe3efffc700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3cc05f360 0x7fe3cc061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:53.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.091+0000 7fe3efffc700 1 --2- 172.21.15.29:0/642289411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3e80c8490 0x7fe3e80c3e20 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.091+0000 7fe3efffc700 1 -- 172.21.15.29:0/642289411 >> 172.21.15.29:0/642289411 conn(0x7fe3e80cb8b0 msgr2=0x7fe3e8152240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:53.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.091+0000 7fe3efffc700 1 -- 172.21.15.29:0/642289411 shutdown_connections 2024-01-24T14:15:53.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.091+0000 7fe3efffc700 1 -- 172.21.15.29:0/642289411 wait complete. 2024-01-24T14:15:53.105 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T14:15:53.131 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:53.130+0000 7f91b8078680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:15:53.234 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:53.233+0000 7f91b8078680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:15:53.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.241+0000 7f5a4b91f700 1 Processor -- start 2024-01-24T14:15:53.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.242+0000 7f5a4b91f700 1 -- start start 2024-01-24T14:15:53.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.242+0000 7f5a4b91f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a440692a0 0x7f5a44065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.242+0000 7f5a4b91f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a44066370 0x7f5a44069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.242+0000 7f5a4b91f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a440c3e70 con 0x7f5a44069b70 2024-01-24T14:15:53.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.242+0000 7f5a4b91f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a440c4300 con 0x7f5a440692a0 2024-01-24T14:15:53.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.242+0000 7f5a4b91f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a440c3260 con 0x7f5a44066370 2024-01-24T14:15:53.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.243+0000 7f5a49ebc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5a44069b70 0x7f5a440bcd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38776/0 (socket says 172.21.15.29:38776) 2024-01-24T14:15:53.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.243+0000 7f5a49ebc700 1 -- 172.21.15.29:0/226392197 learned_addr learned my addr 172.21.15.29:0/226392197 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:53.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.243+0000 7f5a496bb700 1 --2- 172.21.15.29:0/226392197 >> v2:172.21.15.163:3300/0 conn(0x7f5a440692a0 0x7f5a44065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.243+0000 7f5a48eba700 1 --2- 172.21.15.29:0/226392197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a44066370 0x7f5a44069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.243+0000 7f5a3bfff700 1 -- 172.21.15.29:0/226392197 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440266914 0 0) 0x7f5a440c3e70 con 0x7f5a44069b70 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.243+0000 7f5a3bfff700 1 -- 172.21.15.29:0/226392197 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a300033f0 con 0x7f5a44069b70 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.243+0000 7f5a496bb700 1 -- 172.21.15.29:0/226392197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a44066370 msgr2=0x7f5a44069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.243+0000 7f5a496bb700 1 --2- 172.21.15.29:0/226392197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a44066370 0x7f5a44069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.243+0000 7f5a496bb700 1 -- 172.21.15.29:0/226392197 >> v1:172.21.15.29:6789/0 conn(0x7f5a44069b70 legacy=0x7f5a440bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.243+0000 7f5a496bb700 1 -- 172.21.15.29:0/226392197 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a440c3540 con 0x7f5a440692a0 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.243+0000 7f5a48eba700 1 --2- 172.21.15.29:0/226392197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a44066370 0x7f5a44069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.243+0000 7f5a496bb700 1 --2- 172.21.15.29:0/226392197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a440692a0 0x7f5a44065e80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f5a40009760 tx=0x7f5a40009060).ready entity=mon.1 client_cookie=657928c354e341b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.244+0000 7f5a3bfff700 1 -- 172.21.15.29:0/226392197 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000d4a0 con 0x7f5a440692a0 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.244+0000 7f5a3bfff700 1 -- 172.21.15.29:0/226392197 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4000d600 con 0x7f5a440692a0 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.244+0000 7f5a3bfff700 1 -- 172.21.15.29:0/226392197 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000d980 con 0x7f5a440692a0 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.244+0000 7f5a4b91f700 1 -- 172.21.15.29:0/226392197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a440692a0 msgr2=0x7f5a44065e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.244+0000 7f5a4b91f700 1 --2- 172.21.15.29:0/226392197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a440692a0 0x7f5a44065e80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f5a40009760 tx=0x7f5a40009060).stop 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.244+0000 7f5a4b91f700 1 -- 172.21.15.29:0/226392197 shutdown_connections 2024-01-24T14:15:53.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.244+0000 7f5a4b91f700 1 --2- 172.21.15.29:0/226392197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a440692a0 0x7f5a44065e80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.244+0000 7f5a4b91f700 1 --2- 172.21.15.29:0/226392197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a44066370 0x7f5a44069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.244+0000 7f5a4b91f700 1 -- 172.21.15.29:0/226392197 >> 172.21.15.29:0/226392197 conn(0x7f5a44064390 msgr2=0x7f5a440c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:53.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.244+0000 7f5a4b91f700 1 -- 172.21.15.29:0/226392197 shutdown_connections 2024-01-24T14:15:53.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.244+0000 7f5a4b91f700 1 -- 172.21.15.29:0/226392197 wait complete. 2024-01-24T14:15:53.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.245+0000 7f5a4b91f700 1 Processor -- start 2024-01-24T14:15:53.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.246+0000 7f5a4b91f700 1 -- start start 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.246+0000 7f5a4b91f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a44066370 0x7f5a44153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.246+0000 7f5a4b91f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a440692a0 0x7f5a440ab370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.246+0000 7f5a4b91f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a44069b70 0x7f5a440ab860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.246+0000 7f5a4b91f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a440abd50 con 0x7f5a44066370 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a4b91f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a440acb60 con 0x7f5a440692a0 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a4b91f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a440abf50 con 0x7f5a44069b70 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a496bb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a44066370 0x7f5a44153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a496bb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a44066370 0x7f5a44153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:33176/0 (socket says 172.21.15.29:33176) 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a496bb700 1 -- 172.21.15.29:0/860224426 learned_addr learned my addr 172.21.15.29:0/860224426 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a48eba700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a440692a0 0x7f5a440ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a49ebc700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a44069b70 0x7f5a440ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a496bb700 1 -- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a44069b70 msgr2=0x7f5a440ab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a496bb700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a44069b70 0x7f5a440ab860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a496bb700 1 -- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a440692a0 msgr2=0x7f5a440ab370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a496bb700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a440692a0 0x7f5a440ab370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a496bb700 1 -- 172.21.15.29:0/860224426 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a340092a0 con 0x7f5a44066370 2024-01-24T14:15:53.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a496bb700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a44066370 0x7f5a44153c40 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f5a40009d50 tx=0x7f5a40016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:53.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a397fa700 1 -- 172.21.15.29:0/860224426 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000dd30 con 0x7f5a44066370 2024-01-24T14:15:53.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a397fa700 1 -- 172.21.15.29:0/860224426 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4000de90 con 0x7f5a44066370 2024-01-24T14:15:53.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a4b91f700 1 -- 172.21.15.29:0/860224426 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a400092a0 con 0x7f5a44066370 2024-01-24T14:15:53.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.247+0000 7f5a397fa700 1 -- 172.21.15.29:0/860224426 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000d980 con 0x7f5a44066370 2024-01-24T14:15:53.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.248+0000 7f5a4b91f700 1 -- 172.21.15.29:0/860224426 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a440ac510 con 0x7f5a44066370 2024-01-24T14:15:53.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.249+0000 7f5a4b91f700 1 -- 172.21.15.29:0/860224426 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a440475e0 con 0x7f5a44066370 2024-01-24T14:15:53.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.249+0000 7f5a397fa700 1 -- 172.21.15.29:0/860224426 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 45) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f5a4001d640 con 0x7f5a44066370 2024-01-24T14:15:53.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.249+0000 7f5a397fa700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5a2805f390 0x7f5a28061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.249+0000 7f5a397fa700 1 -- 172.21.15.29:0/860224426 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7f5a4007c2b0 con 0x7f5a44066370 2024-01-24T14:15:53.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.249+0000 7f5a48eba700 1 -- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5a2805f390 msgr2=0x7f5a28061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:53.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.249+0000 7f5a48eba700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5a2805f390 0x7f5a28061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:15:53.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.254+0000 7f5a397fa700 1 -- 172.21.15.29:0/860224426 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5a4004cd10 con 0x7f5a44066370 2024-01-24T14:15:53.390 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:53.389+0000 7f91b8078680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:15:53.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.437+0000 7f5a4b91f700 1 -- 172.21.15.29:0/860224426 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a440c6ce0 con 0x7f5a44066370 2024-01-24T14:15:53.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.437+0000 7f5a397fa700 1 -- 172.21.15.29:0/860224426 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a40050330 con 0x7f5a44066370 2024-01-24T14:15:53.438 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T14:15:53.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.440+0000 7f5a4b91f700 1 -- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5a2805f390 msgr2=0x7f5a28061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:15:53.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.440+0000 7f5a4b91f700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5a2805f390 0x7f5a28061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:53.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.440+0000 7f5a4b91f700 1 -- 172.21.15.29:0/860224426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a44066370 msgr2=0x7f5a44153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.440+0000 7f5a4b91f700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a44066370 0x7f5a44153c40 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f5a40009d50 tx=0x7f5a40016000).stop 2024-01-24T14:15:53.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.440+0000 7f5a4b91f700 1 -- 172.21.15.29:0/860224426 shutdown_connections 2024-01-24T14:15:53.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.440+0000 7f5a4b91f700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a44069b70 0x7f5a440ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.440+0000 7f5a4b91f700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5a2805f390 0x7f5a28061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:53.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.440+0000 7f5a4b91f700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5a44066370 0x7f5a44153c40 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.440+0000 7f5a4b91f700 1 --2- 172.21.15.29:0/860224426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a440692a0 0x7f5a440ab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.440+0000 7f5a4b91f700 1 -- 172.21.15.29:0/860224426 >> 172.21.15.29:0/860224426 conn(0x7f5a44064390 msgr2=0x7f5a440c6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:53.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.440+0000 7f5a4b91f700 1 -- 172.21.15.29:0/860224426 shutdown_connections 2024-01-24T14:15:53.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.440+0000 7f5a4b91f700 1 -- 172.21.15.29:0/860224426 wait complete. 2024-01-24T14:15:53.443 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:53.443+0000 7fadf8e6f680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:15:53.455 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T14:15:53.466 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:53.465+0000 7f91b8078680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:15:53.534 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:53.533+0000 7f91b8078680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:15:53.568 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:53.568+0000 7fadf8e6f680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:15:53.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.588+0000 7f1aa901c700 1 Processor -- start 2024-01-24T14:15:53.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.588+0000 7f1aa901c700 1 -- start start 2024-01-24T14:15:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa901c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1aa413a5e0 0x7f1aa40639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa901c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1aa4066bd0 0x7f1aa40bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa901c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa40679f0 con 0x7f1aa4063eb0 2024-01-24T14:15:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa901c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa4067cd0 con 0x7f1aa413a5e0 2024-01-24T14:15:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa901c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa4067fb0 con 0x7f1aa4066bd0 2024-01-24T14:15:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa2d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1aa413a5e0 0x7f1aa40639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa259c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1aa4063eb0 0x7f1aa40664c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39596/0 (socket says 172.21.15.29:39596) 2024-01-24T14:15:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa259c700 1 -- 172.21.15.29:0/2980134735 learned_addr learned my addr 172.21.15.29:0/2980134735 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa359e700 1 --2- 172.21.15.29:0/2980134735 >> v2:172.21.15.163:3300/0 conn(0x7f1aa4066bd0 0x7f1aa40bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa2d9d700 1 -- 172.21.15.29:0/2980134735 >> v2:172.21.15.163:3300/0 conn(0x7f1aa4066bd0 msgr2=0x7f1aa40bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa2d9d700 1 --2- 172.21.15.29:0/2980134735 >> v2:172.21.15.163:3300/0 conn(0x7f1aa4066bd0 0x7f1aa40bdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa2d9d700 1 -- 172.21.15.29:0/2980134735 >> v1:172.21.15.29:6789/0 conn(0x7f1aa4063eb0 legacy=0x7f1aa40664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa2d9d700 1 -- 172.21.15.29:0/2980134735 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4068290 con 0x7f1aa413a5e0 2024-01-24T14:15:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa1d9b700 1 -- 172.21.15.29:0/2980134735 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600197114 0 0) 0x7f1aa40679f0 con 0x7f1aa4063eb0 2024-01-24T14:15:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.589+0000 7f1aa2d9d700 1 --2- 172.21.15.29:0/2980134735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa413a5e0 0x7f1aa40639c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f1aa406a250 tx=0x7f1a94009060).ready entity=mon.0 client_cookie=5c552b79c5df6493 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.590+0000 7f1aa1d9b700 1 -- 172.21.15.29:0/2980134735 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400f510 con 0x7f1aa413a5e0 2024-01-24T14:15:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.590+0000 7f1aa1d9b700 1 -- 172.21.15.29:0/2980134735 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9400f670 con 0x7f1aa413a5e0 2024-01-24T14:15:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.590+0000 7f1aa1d9b700 1 -- 172.21.15.29:0/2980134735 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400f9f0 con 0x7f1aa413a5e0 2024-01-24T14:15:53.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.590+0000 7f1aa901c700 1 -- 172.21.15.29:0/2980134735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa413a5e0 msgr2=0x7f1aa40639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.590+0000 7f1aa901c700 1 --2- 172.21.15.29:0/2980134735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa413a5e0 0x7f1aa40639c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f1aa406a250 tx=0x7f1a94009060).stop 2024-01-24T14:15:53.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.590+0000 7f1aa901c700 1 -- 172.21.15.29:0/2980134735 shutdown_connections 2024-01-24T14:15:53.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.590+0000 7f1aa901c700 1 --2- 172.21.15.29:0/2980134735 >> v2:172.21.15.163:3300/0 conn(0x7f1aa4066bd0 0x7f1aa40bdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.590+0000 7f1aa901c700 1 --2- 172.21.15.29:0/2980134735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa413a5e0 0x7f1aa40639c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.590+0000 7f1aa901c700 1 -- 172.21.15.29:0/2980134735 >> 172.21.15.29:0/2980134735 conn(0x7f1aa40c8bd0 msgr2=0x7f1aa40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:53.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.590+0000 7f1aa901c700 1 -- 172.21.15.29:0/2980134735 shutdown_connections 2024-01-24T14:15:53.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.590+0000 7f1aa901c700 1 -- 172.21.15.29:0/2980134735 wait complete. 2024-01-24T14:15:53.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.592+0000 7f1aa901c700 1 Processor -- start 2024-01-24T14:15:53.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.592+0000 7f1aa901c700 1 -- start start 2024-01-24T14:15:53.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.592+0000 7f1aa901c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa4063eb0 0x7f1aa413b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.592+0000 7f1aa901c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1aa4066bd0 0x7f1aa4139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.592+0000 7f1aa901c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1aa413a5e0 0x7f1aa40a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.592+0000 7f1aa901c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa41686f0 con 0x7f1aa4063eb0 2024-01-24T14:15:53.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.592+0000 7f1aa901c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa4167950 con 0x7f1aa4066bd0 2024-01-24T14:15:53.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.592+0000 7f1aa901c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa4167c10 con 0x7f1aa413a5e0 2024-01-24T14:15:53.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa2d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa4063eb0 0x7f1aa413b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa2d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa4063eb0 0x7f1aa413b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38870/0 (socket says 172.21.15.29:38870) 2024-01-24T14:15:53.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa2d9d700 1 -- 172.21.15.29:0/623210785 learned_addr learned my addr 172.21.15.29:0/623210785 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:53.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa259c700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1aa4066bd0 0x7f1aa4139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa359e700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1aa413a5e0 0x7f1aa40a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa2d9d700 1 -- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1aa413a5e0 msgr2=0x7f1aa40a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa2d9d700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1aa413a5e0 0x7f1aa40a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa2d9d700 1 -- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1aa4066bd0 msgr2=0x7f1aa4139670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa2d9d700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1aa4066bd0 0x7f1aa4139670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa2d9d700 1 -- 172.21.15.29:0/623210785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a940092a0 con 0x7f1aa4063eb0 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa259c700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1aa4066bd0 0x7f1aa4139670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa2d9d700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa4063eb0 0x7f1aa413b8e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f1a94009dd0 tx=0x7f1a94017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1a8affd700 1 -- 172.21.15.29:0/623210785 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400f7e0 con 0x7f1aa4063eb0 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1a8affd700 1 -- 172.21.15.29:0/623210785 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9400f940 con 0x7f1aa4063eb0 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa901c700 1 -- 172.21.15.29:0/623210785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa4167ef0 con 0x7f1aa4063eb0 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1a8affd700 1 -- 172.21.15.29:0/623210785 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9401b690 con 0x7f1aa4063eb0 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.593+0000 7f1aa901c700 1 -- 172.21.15.29:0/623210785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa4139b60 con 0x7f1aa4063eb0 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.595+0000 7f1aa901c700 1 -- 172.21.15.29:0/623210785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa4168410 con 0x7f1aa4063eb0 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.595+0000 7f1a8affd700 1 -- 172.21.15.29:0/623210785 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 45) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f1a9401ba40 con 0x7f1aa4063eb0 2024-01-24T14:15:53.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.596+0000 7f1a8affd700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a9005f390 0x7f1a90061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.596+0000 7f1a8affd700 1 -- 172.21.15.29:0/623210785 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7f1a9407c0a0 con 0x7f1aa4063eb0 2024-01-24T14:15:53.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.596+0000 7f1aa259c700 1 -- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a9005f390 msgr2=0x7f1a90061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:53.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.596+0000 7f1aa259c700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a9005f390 0x7f1a90061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:15:53.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.600+0000 7f1a8affd700 1 -- 172.21.15.29:0/623210785 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1a9404cb00 con 0x7f1aa4063eb0 2024-01-24T14:15:53.768 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:53.767+0000 7fadf8e6f680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:15:53.790 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:53.789+0000 7f91b8078680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:15:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.797+0000 7f1aa259c700 1 -- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a9005f390 msgr2=0x7f1a90061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.797+0000 7f1aa259c700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a9005f390 0x7f1a90061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:15:53.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.804+0000 7f1aa901c700 1 -- 172.21.15.29:0/623210785 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1aa40475e0 con 0x7f1aa4063eb0 2024-01-24T14:15:53.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.804+0000 7f1a8affd700 1 -- 172.21.15.29:0/623210785 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a940280a0 con 0x7f1aa4063eb0 2024-01-24T14:15:53.805 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T14:15:53.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.806+0000 7f1aa901c700 1 -- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a9005f390 msgr2=0x7f1a90061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:15:53.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.806+0000 7f1aa901c700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a9005f390 0x7f1a90061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:53.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.806+0000 7f1aa901c700 1 -- 172.21.15.29:0/623210785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa4063eb0 msgr2=0x7f1aa413b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.806+0000 7f1aa901c700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa4063eb0 0x7f1aa413b8e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f1a94009dd0 tx=0x7f1a94017000).stop 2024-01-24T14:15:53.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.806+0000 7f1aa901c700 1 -- 172.21.15.29:0/623210785 shutdown_connections 2024-01-24T14:15:53.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.806+0000 7f1aa901c700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1aa413a5e0 0x7f1aa40a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.806+0000 7f1aa901c700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1a9005f390 0x7f1a90061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:53.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.806+0000 7f1aa901c700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa4063eb0 0x7f1aa413b8e0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.806+0000 7f1aa901c700 1 --2- 172.21.15.29:0/623210785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1aa4066bd0 0x7f1aa4139670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.806+0000 7f1aa901c700 1 -- 172.21.15.29:0/623210785 >> 172.21.15.29:0/623210785 conn(0x7f1aa40c8bd0 msgr2=0x7f1aa4155750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:53.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.806+0000 7f1aa901c700 1 -- 172.21.15.29:0/623210785 shutdown_connections 2024-01-24T14:15:53.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.806+0000 7f1aa901c700 1 -- 172.21.15.29:0/623210785 wait complete. 2024-01-24T14:15:53.821 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:15:53.857 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:53.856+0000 7fadf8e6f680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:15:53.937 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:53.937+0000 7fadf8e6f680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:15:53.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.951+0000 7fcca57da700 1 Processor -- start 2024-01-24T14:15:53.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.951+0000 7fcca57da700 1 -- start start 2024-01-24T14:15:53.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcca57da700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcca00c5410 0x7fcca00631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcca57da700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca0063b50 con 0x7fcca00636c0 2024-01-24T14:15:53.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcca57da700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca00cb730 con 0x7fcca01544a0 2024-01-24T14:15:53.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcca57da700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca00cbcf0 con 0x7fcca00c5410 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcc9f7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcca00636c0 0x7fcca00c03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39610/0 (socket says 172.21.15.29:39610) 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcc9f7fe700 1 -- 172.21.15.29:0/1383251501 learned_addr learned my addr 172.21.15.29:0/1383251501 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcc9e7fc700 1 --2- 172.21.15.29:0/1383251501 >> v2:172.21.15.163:3300/0 conn(0x7fcca00c5410 0x7fcca00631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcc9dffb700 1 -- 172.21.15.29:0/1383251501 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557566294 0 0) 0x7fcca0063b50 con 0x7fcca00636c0 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcc9dffb700 1 -- 172.21.15.29:0/1383251501 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc880033f0 con 0x7fcca00636c0 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcc9dffb700 1 -- 172.21.15.29:0/1383251501 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 85006235 0 0) 0x7fcca00cb730 con 0x7fcca01544a0 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcc9dffb700 1 -- 172.21.15.29:0/1383251501 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcca0063b50 con 0x7fcca01544a0 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcc9e7fc700 1 -- 172.21.15.29:0/1383251501 >> v1:172.21.15.163:6789/0 conn(0x7fcca01544a0 legacy=0x7fcca00a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcc9e7fc700 1 -- 172.21.15.29:0/1383251501 >> v1:172.21.15.29:6789/0 conn(0x7fcca00636c0 legacy=0x7fcca00c03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.953+0000 7fcc9e7fc700 1 -- 172.21.15.29:0/1383251501 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca00cba10 con 0x7fcca00c5410 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.954+0000 7fcc9e7fc700 1 --2- 172.21.15.29:0/1383251501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca00c5410 0x7fcca00631d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fcc94009760 tx=0x7fcc94009060).ready entity=mon.1 client_cookie=386631dc991044b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.954+0000 7fcc9dffb700 1 -- 172.21.15.29:0/1383251501 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94009120 con 0x7fcca00c5410 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.954+0000 7fcc9dffb700 1 -- 172.21.15.29:0/1383251501 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9400d4a0 con 0x7fcca00c5410 2024-01-24T14:15:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.954+0000 7fcc9dffb700 1 -- 172.21.15.29:0/1383251501 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400d820 con 0x7fcca00c5410 2024-01-24T14:15:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.954+0000 7fcca57da700 1 -- 172.21.15.29:0/1383251501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca00c5410 msgr2=0x7fcca00631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.954+0000 7fcca57da700 1 --2- 172.21.15.29:0/1383251501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca00c5410 0x7fcca00631d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fcc94009760 tx=0x7fcc94009060).stop 2024-01-24T14:15:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.954+0000 7fcca57da700 1 -- 172.21.15.29:0/1383251501 shutdown_connections 2024-01-24T14:15:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.954+0000 7fcca57da700 1 --2- 172.21.15.29:0/1383251501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca00c5410 0x7fcca00631d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.954+0000 7fcca57da700 1 -- 172.21.15.29:0/1383251501 >> 172.21.15.29:0/1383251501 conn(0x7fcca005ec80 msgr2=0x7fcca00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.954+0000 7fcca57da700 1 -- 172.21.15.29:0/1383251501 shutdown_connections 2024-01-24T14:15:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.955+0000 7fcca57da700 1 -- 172.21.15.29:0/1383251501 wait complete. 2024-01-24T14:15:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.956+0000 7fcca57da700 1 Processor -- start 2024-01-24T14:15:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.956+0000 7fcca57da700 1 -- start start 2024-01-24T14:15:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcca57da700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca00636c0 0x7fcca014ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcca57da700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcca00c5410 0x7fcca014cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcca57da700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca01544a0 0x7fcca0138600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcca57da700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca014a6c0 con 0x7fcca00636c0 2024-01-24T14:15:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcca57da700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca00c3c00 con 0x7fcca01544a0 2024-01-24T14:15:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcca57da700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca00c3ee0 con 0x7fcca00c5410 2024-01-24T14:15:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcc9effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca00636c0 0x7fcca014ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcc9effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca00636c0 0x7fcca014ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38872/0 (socket says 172.21.15.29:38872) 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcc9effd700 1 -- 172.21.15.29:0/1242657912 learned_addr learned my addr 172.21.15.29:0/1242657912 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcc9f7fe700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca01544a0 0x7fcca0138600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcc9e7fc700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcca00c5410 0x7fcca014cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcc9effd700 1 -- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcca00c5410 msgr2=0x7fcca014cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcc9effd700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcca00c5410 0x7fcca014cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcc9effd700 1 -- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca01544a0 msgr2=0x7fcca0138600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcc9effd700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca01544a0 0x7fcca0138600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.957+0000 7fcc9effd700 1 -- 172.21.15.29:0/1242657912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc940092a0 con 0x7fcca00636c0 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.958+0000 7fcc9effd700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca00636c0 0x7fcca014ace0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7fcc9000c440 tx=0x7fcc90007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.958+0000 7fcc86ffd700 1 -- 172.21.15.29:0/1242657912 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90012400 con 0x7fcca00636c0 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.958+0000 7fcc86ffd700 1 -- 172.21.15.29:0/1242657912 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc90012560 con 0x7fcca00636c0 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.958+0000 7fcc86ffd700 1 -- 172.21.15.29:0/1242657912 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90012920 con 0x7fcca00636c0 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.958+0000 7fcca57da700 1 -- 172.21.15.29:0/1242657912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca00c41c0 con 0x7fcca00636c0 2024-01-24T14:15:53.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.958+0000 7fcca57da700 1 -- 172.21.15.29:0/1242657912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca00c4780 con 0x7fcca00636c0 2024-01-24T14:15:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.959+0000 7fcca57da700 1 -- 172.21.15.29:0/1242657912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca00af220 con 0x7fcca00636c0 2024-01-24T14:15:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.959+0000 7fcc86ffd700 1 -- 172.21.15.29:0/1242657912 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 45) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fcc9001b3b0 con 0x7fcca00636c0 2024-01-24T14:15:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.960+0000 7fcc86ffd700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8c05f360 0x7fcc8c061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.960+0000 7fcc86ffd700 1 -- 172.21.15.29:0/1242657912 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7fcc9007a380 con 0x7fcca00636c0 2024-01-24T14:15:53.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.960+0000 7fcc9e7fc700 1 -- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8c05f360 msgr2=0x7fcc8c061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:53.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.960+0000 7fcc9e7fc700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8c05f360 0x7fcc8c061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:15:53.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:53.964+0000 7fcc86ffd700 1 -- 172.21.15.29:0/1242657912 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcc9004acc0 con 0x7fcca00636c0 2024-01-24T14:15:53.973 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:53.972+0000 7f91b8078680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:15:54.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.161+0000 7fcc9e7fc700 1 -- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8c05f360 msgr2=0x7fcc8c061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:54.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.161+0000 7fcc9e7fc700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8c05f360 0x7fcc8c061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:15:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.193+0000 7fcca57da700 1 -- 172.21.15.29:0/1242657912 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcca00475e0 con 0x7fcca00636c0 2024-01-24T14:15:54.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.194+0000 7fcc86ffd700 1 -- 172.21.15.29:0/1242657912 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fcc9004e2e0 con 0x7fcca00636c0 2024-01-24T14:15:54.195 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:15:54.195 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6341,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4230829295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4230829295},{"type":"v1","addr":"172.21.15.29:6835","nonce":4230829295}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6344,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3194389571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3194389571},{"type":"v1","addr":"172.21.15.163:6836","nonce":3194389571}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2473497045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2473497045},{"type":"v1","addr":"172.21.15.163:6837","nonce":2473497045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:14:35.870228+0000","modified":"2024-01-24T14:14:38.905491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7254":{"gid":7254,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/3097015540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3097015540},{"type":"v1","addr":"172.21.15.29:6834","nonce":3097015540}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-24T14:15:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.196+0000 7fcca57da700 1 -- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8c05f360 msgr2=0x7fcc8c061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:15:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.196+0000 7fcca57da700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8c05f360 0x7fcc8c061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.196+0000 7fcca57da700 1 -- 172.21.15.29:0/1242657912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca00636c0 msgr2=0x7fcca014ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.196+0000 7fcca57da700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca00636c0 0x7fcca014ace0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7fcc9000c440 tx=0x7fcc90007c80).stop 2024-01-24T14:15:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.196+0000 7fcca57da700 1 -- 172.21.15.29:0/1242657912 shutdown_connections 2024-01-24T14:15:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.196+0000 7fcca57da700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca01544a0 0x7fcca0138600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.196+0000 7fcca57da700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8c05f360 0x7fcc8c061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.196+0000 7fcca57da700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca00636c0 0x7fcca014ace0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.196+0000 7fcca57da700 1 --2- 172.21.15.29:0/1242657912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcca00c5410 0x7fcca014cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.196+0000 7fcca57da700 1 -- 172.21.15.29:0/1242657912 >> 172.21.15.29:0/1242657912 conn(0x7fcca005ec80 msgr2=0x7fcca00c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:54.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.196+0000 7fcca57da700 1 -- 172.21.15.29:0/1242657912 shutdown_connections 2024-01-24T14:15:54.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.196+0000 7fcca57da700 1 -- 172.21.15.29:0/1242657912 wait complete. 2024-01-24T14:15:54.198 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 51 2024-01-24T14:15:54.211 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:15:54.239 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:54.238+0000 7fadf8e6f680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:15:54.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.372+0000 7f1468b92700 1 Processor -- start 2024-01-24T14:15:54.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.372+0000 7f1468b92700 1 -- start start 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.373+0000 7f1468b92700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14641540a0 0x7f14640c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.373+0000 7f1468b92700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1464065360 con 0x7f14641550f0 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.373+0000 7f1468b92700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14640a6ce0 con 0x7f1464062ec0 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.373+0000 7f1468b92700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14640a74f0 con 0x7f14641540a0 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.373+0000 7f1461d9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f14641550f0 0x7f14640627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39622/0 (socket says 172.21.15.29:39622) 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.373+0000 7f1461d9b700 1 -- 172.21.15.29:0/116860937 learned_addr learned my addr 172.21.15.29:0/116860937 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.373+0000 7f146259c700 1 --2- 172.21.15.29:0/116860937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14641540a0 0x7f14640c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.373+0000 7f146159a700 1 -- 172.21.15.29:0/116860937 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593208687 0 0) 0x7f1464065360 con 0x7f14641550f0 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.373+0000 7f146159a700 1 -- 172.21.15.29:0/116860937 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f144c0033f0 con 0x7f14641550f0 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.374+0000 7f146159a700 1 -- 172.21.15.29:0/116860937 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904722554 0 0) 0x7f14640a6ce0 con 0x7f1464062ec0 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.374+0000 7f146259c700 1 -- 172.21.15.29:0/116860937 >> v1:172.21.15.163:6789/0 conn(0x7f1464062ec0 legacy=0x7f1464069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.374+0000 7f146259c700 1 -- 172.21.15.29:0/116860937 >> v1:172.21.15.29:6789/0 conn(0x7f14641550f0 legacy=0x7f14640627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:54.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.374+0000 7f146259c700 1 -- 172.21.15.29:0/116860937 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14640636f0 con 0x7f14641540a0 2024-01-24T14:15:54.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.374+0000 7f146259c700 1 --2- 172.21.15.29:0/116860937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14641540a0 0x7f14640c4ca0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f1454009790 tx=0x7f1454009060).ready entity=mon.2 client_cookie=1bfa4ffe2d4ab96b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:54.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.374+0000 7f146159a700 1 -- 172.21.15.29:0/116860937 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400f510 con 0x7f14641540a0 2024-01-24T14:15:54.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.374+0000 7f146159a700 1 -- 172.21.15.29:0/116860937 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145400f670 con 0x7f14641540a0 2024-01-24T14:15:54.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.374+0000 7f146159a700 1 -- 172.21.15.29:0/116860937 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400f9f0 con 0x7f14641540a0 2024-01-24T14:15:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.375+0000 7f1468b92700 1 -- 172.21.15.29:0/116860937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14641540a0 msgr2=0x7f14640c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.375+0000 7f1468b92700 1 --2- 172.21.15.29:0/116860937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14641540a0 0x7f14640c4ca0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f1454009790 tx=0x7f1454009060).stop 2024-01-24T14:15:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.375+0000 7f1468b92700 1 -- 172.21.15.29:0/116860937 shutdown_connections 2024-01-24T14:15:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.375+0000 7f1468b92700 1 --2- 172.21.15.29:0/116860937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14641540a0 0x7f14640c4ca0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.375+0000 7f1468b92700 1 -- 172.21.15.29:0/116860937 >> 172.21.15.29:0/116860937 conn(0x7f14640c85c0 msgr2=0x7f14640c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.375+0000 7f1468b92700 1 -- 172.21.15.29:0/116860937 shutdown_connections 2024-01-24T14:15:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.375+0000 7f1468b92700 1 -- 172.21.15.29:0/116860937 wait complete. 2024-01-24T14:15:54.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.376+0000 7f1468b92700 1 Processor -- start 2024-01-24T14:15:54.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.376+0000 7f1468b92700 1 -- start start 2024-01-24T14:15:54.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1468b92700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1464062ec0 0x7f14641680f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:54.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1468b92700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14641540a0 0x7f14640c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:54.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1468b92700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14641550f0 0x7f14640c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:54.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1468b92700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14640c3d90 con 0x7f14641550f0 2024-01-24T14:15:54.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1468b92700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14640ad040 con 0x7f1464062ec0 2024-01-24T14:15:54.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1468b92700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14640ad320 con 0x7f14641540a0 2024-01-24T14:15:54.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1462d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14641550f0 0x7f14640c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:54.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1462d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14641550f0 0x7f14640c38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38876/0 (socket says 172.21.15.29:38876) 2024-01-24T14:15:54.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1462d9d700 1 -- 172.21.15.29:0/281383274 learned_addr learned my addr 172.21.15.29:0/281383274 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:54.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f146259c700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1464062ec0 0x7f14641680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1461d9b700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14641540a0 0x7f14640c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1462d9d700 1 -- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14641540a0 msgr2=0x7f14640c33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1462d9d700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14641540a0 0x7f14640c33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.377+0000 7f1462d9d700 1 -- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1464062ec0 msgr2=0x7f14641680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.378+0000 7f1462d9d700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1464062ec0 0x7f14641680f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.378+0000 7f1462d9d700 1 -- 172.21.15.29:0/281383274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14540092a0 con 0x7f14641550f0 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.378+0000 7f146259c700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1464062ec0 0x7f14641680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.378+0000 7f1462d9d700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14641550f0 0x7f14640c38a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7f145c00c7e0 tx=0x7f145c007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.378+0000 7f14527fc700 1 -- 172.21.15.29:0/281383274 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c00e4a0 con 0x7f14641550f0 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.378+0000 7f14527fc700 1 -- 172.21.15.29:0/281383274 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c00e620 con 0x7f14641550f0 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.378+0000 7f1468b92700 1 -- 172.21.15.29:0/281383274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14640ad600 con 0x7f14641550f0 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.378+0000 7f14527fc700 1 -- 172.21.15.29:0/281383274 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c00e9e0 con 0x7f14641550f0 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.378+0000 7f1468b92700 1 -- 172.21.15.29:0/281383274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14641685e0 con 0x7f14641550f0 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.379+0000 7f1468b92700 1 -- 172.21.15.29:0/281383274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14641687d0 con 0x7f14641550f0 2024-01-24T14:15:54.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.379+0000 7f14527fc700 1 -- 172.21.15.29:0/281383274 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 45) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f145c0123f0 con 0x7f14641550f0 2024-01-24T14:15:54.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.380+0000 7f14527fc700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f144405f390 0x7f1444061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:54.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.380+0000 7f14527fc700 1 -- 172.21.15.29:0/281383274 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7f145c07a450 con 0x7f14641550f0 2024-01-24T14:15:54.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.380+0000 7f146259c700 1 -- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f144405f390 msgr2=0x7f1444061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:54.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.380+0000 7f146259c700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f144405f390 0x7f1444061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:15:54.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.384+0000 7f14527fc700 1 -- 172.21.15.29:0/281383274 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f145c047b00 con 0x7f14641550f0 2024-01-24T14:15:54.457 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:54.456+0000 7fadf8e6f680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:15:54.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.569+0000 7f1468b92700 1 -- 172.21.15.29:0/281383274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14640a76c0 con 0x7f14641550f0 2024-01-24T14:15:54.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.570+0000 7f14527fc700 1 -- 172.21.15.29:0/281383274 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+18752 (secure 0 0 0) 0x7f145c04e880 con 0x7f14641550f0 2024-01-24T14:15:54.571 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:15:54.571 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":123,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:15:49.269646+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:14:33.610829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-24T14:14:34.834814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.18","mappings":[{"from":1,"to":7}]},{"pgid":"14.29","mappings":[{"from":1,"to":2}]},{"pgid":"14.36","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/2260886250":"2024-01-25T14:14:48.183216+0000","172.21.15.163:6833/43241":"2024-01-25T14:14:48.183216+0000","172.21.15.163:6832/43241":"2024-01-25T14:14:48.183216+0000","172.21.15.163:0/2891867602":"2024-01-25T14:14:48.183216+0000","172.21.15.163:0/3822418593":"2024-01-25T14:14:48.183216+0000","172.21.15.163:0/4052979774":"2024-01-25T14:14:48.183216+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:15:54.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.572+0000 7f1468b92700 1 -- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f144405f390 msgr2=0x7f1444061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:15:54.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.572+0000 7f1468b92700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f144405f390 0x7f1444061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:54.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.572+0000 7f1468b92700 1 -- 172.21.15.29:0/281383274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14641550f0 msgr2=0x7f14640c38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:54.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.572+0000 7f1468b92700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14641550f0 0x7f14640c38a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7f145c00c7e0 tx=0x7f145c007f10).stop 2024-01-24T14:15:54.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.572+0000 7f1468b92700 1 -- 172.21.15.29:0/281383274 shutdown_connections 2024-01-24T14:15:54.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.572+0000 7f1468b92700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f144405f390 0x7f1444061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:54.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.572+0000 7f1468b92700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14641550f0 0x7f14640c38a0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.572+0000 7f1468b92700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1464062ec0 0x7f14641680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.572+0000 7f1468b92700 1 --2- 172.21.15.29:0/281383274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14641540a0 0x7f14640c33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.572+0000 7f1468b92700 1 -- 172.21.15.29:0/281383274 >> 172.21.15.29:0/281383274 conn(0x7f14640c85c0 msgr2=0x7f14640c6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:54.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.572+0000 7f1468b92700 1 -- 172.21.15.29:0/281383274 shutdown_connections 2024-01-24T14:15:54.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.572+0000 7f1468b92700 1 -- 172.21.15.29:0/281383274 wait complete. 2024-01-24T14:15:54.586 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T14:15:54.587 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:15:54.627 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:54.626+0000 7f91b8078680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:15:54.696 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:54.695+0000 7f91b8078680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:15:54.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.717+0000 7f80b5c53700 1 Processor -- start 2024-01-24T14:15:54.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.717+0000 7f80b5c53700 1 -- start start 2024-01-24T14:15:54.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.718+0000 7f80b5c53700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 0x7f80b00c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:54.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.718+0000 7f80b5c53700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f80b00aa0e0 0x7f80b0154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:54.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.718+0000 7f80b5c53700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b00aa6c0 con 0x7f80b00ab1f0 2024-01-24T14:15:54.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.718+0000 7f80b5c53700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b0064240 con 0x7f80b00aa0e0 2024-01-24T14:15:54.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.718+0000 7f80b5c53700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00cb5c0 con 0x7f80b0154370 2024-01-24T14:15:54.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.718+0000 7f80af7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 0x7f80b00c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:54.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.718+0000 7f80a6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f80b00aa0e0 0x7f80b0154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:54.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.718+0000 7f80affff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f80b00ab1f0 0x7f80b00631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:35952/0 (socket says 172.21.15.29:35952) 2024-01-24T14:15:54.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.718+0000 7f80affff700 1 -- 172.21.15.29:0/134942431 learned_addr learned my addr 172.21.15.29:0/134942431 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.719+0000 7f80af7fe700 1 -- 172.21.15.29:0/134942431 >> v2:172.21.15.163:3300/0 conn(0x7f80b00aa0e0 msgr2=0x7f80b0154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.719+0000 7f80af7fe700 1 --2- 172.21.15.29:0/134942431 >> v2:172.21.15.163:3300/0 conn(0x7f80b00aa0e0 0x7f80b0154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.719+0000 7f80af7fe700 1 -- 172.21.15.29:0/134942431 >> v1:172.21.15.163:6789/0 conn(0x7f80b00ab1f0 legacy=0x7f80b00631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.719+0000 7f80af7fe700 1 -- 172.21.15.29:0/134942431 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b00cb8a0 con 0x7f80b0154370 2024-01-24T14:15:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.719+0000 7f80a6ffd700 1 --2- 172.21.15.29:0/134942431 >> v2:172.21.15.163:3300/0 conn(0x7f80b00aa0e0 0x7f80b0154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.719+0000 7f80af7fe700 1 --2- 172.21.15.29:0/134942431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 0x7f80b00c5410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f80a0006fd0 tx=0x7f80a0006f40).ready entity=mon.2 client_cookie=5ca0903a70390605 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.719+0000 7f80aeffd700 1 -- 172.21.15.29:0/134942431 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0005880 con 0x7f80b0154370 2024-01-24T14:15:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.719+0000 7f80aeffd700 1 -- 172.21.15.29:0/134942431 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a00059e0 con 0x7f80b0154370 2024-01-24T14:15:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.719+0000 7f80aeffd700 1 -- 172.21.15.29:0/134942431 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0005d60 con 0x7f80b0154370 2024-01-24T14:15:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.720+0000 7f80b5c53700 1 -- 172.21.15.29:0/134942431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 msgr2=0x7f80b00c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.720+0000 7f80b5c53700 1 --2- 172.21.15.29:0/134942431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 0x7f80b00c5410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f80a0006fd0 tx=0x7f80a0006f40).stop 2024-01-24T14:15:54.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.720+0000 7f80b5c53700 1 -- 172.21.15.29:0/134942431 shutdown_connections 2024-01-24T14:15:54.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.720+0000 7f80b5c53700 1 --2- 172.21.15.29:0/134942431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 0x7f80b00c5410 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.720+0000 7f80b5c53700 1 --2- 172.21.15.29:0/134942431 >> v2:172.21.15.163:3300/0 conn(0x7f80b00aa0e0 0x7f80b0154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.720+0000 7f80b5c53700 1 -- 172.21.15.29:0/134942431 >> 172.21.15.29:0/134942431 conn(0x7f80b005ec80 msgr2=0x7f80b00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:54.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.720+0000 7f80b5c53700 1 -- 172.21.15.29:0/134942431 shutdown_connections 2024-01-24T14:15:54.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.720+0000 7f80b5c53700 1 -- 172.21.15.29:0/134942431 wait complete. 2024-01-24T14:15:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.721+0000 7f80b5c53700 1 Processor -- start 2024-01-24T14:15:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.721+0000 7f80b5c53700 1 -- start start 2024-01-24T14:15:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80b5c53700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80b00aa0e0 0x7f80b013fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:54.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80b5c53700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80b00ab1f0 0x7f80b014fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80b5c53700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 0x7f80b00c12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80b5c53700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b01505a0 con 0x7f80b00ab1f0 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80b5c53700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b01503b0 con 0x7f80b00aa0e0 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80b5c53700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b0156800 con 0x7f80b0154370 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80a6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80b00ab1f0 0x7f80b014fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80affff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 0x7f80b00c12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80af7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80b00aa0e0 0x7f80b013fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80a6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80b00ab1f0 0x7f80b014fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38892/0 (socket says 172.21.15.29:38892) 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80a6ffd700 1 -- 172.21.15.29:0/3312118577 learned_addr learned my addr 172.21.15.29:0/3312118577 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80af7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80b00aa0e0 0x7f80b013fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:40074/0 (socket says 172.21.15.29:40074) 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.722+0000 7f80affff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 0x7f80b00c12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:50060/0 (socket says 172.21.15.29:50060) 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80a6ffd700 1 -- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 msgr2=0x7f80b00c12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80a6ffd700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 0x7f80b00c12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80a6ffd700 1 -- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80b00aa0e0 msgr2=0x7f80b013fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80a6ffd700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80b00aa0e0 0x7f80b013fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80a6ffd700 1 -- 172.21.15.29:0/3312118577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80980092a0 con 0x7f80b00ab1f0 2024-01-24T14:15:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80af7fe700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80b00aa0e0 0x7f80b013fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80affff700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 0x7f80b00c12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80a6ffd700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80b00ab1f0 0x7f80b014fec0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f809800e9d0 tx=0x7f8098009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80a7fff700 1 -- 172.21.15.29:0/3312118577 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809800c6a0 con 0x7f80b00ab1f0 2024-01-24T14:15:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80a7fff700 1 -- 172.21.15.29:0/3312118577 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809800c800 con 0x7f80b00ab1f0 2024-01-24T14:15:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80a7fff700 1 -- 172.21.15.29:0/3312118577 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809800cb80 con 0x7f80b00ab1f0 2024-01-24T14:15:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80b5c53700 1 -- 172.21.15.29:0/3312118577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a0009250 con 0x7f80b00ab1f0 2024-01-24T14:15:54.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.723+0000 7f80b5c53700 1 -- 172.21.15.29:0/3312118577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b014eed0 con 0x7f80b00ab1f0 2024-01-24T14:15:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.724+0000 7f80b5c53700 1 -- 172.21.15.29:0/3312118577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b0156990 con 0x7f80b00ab1f0 2024-01-24T14:15:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.725+0000 7f80a7fff700 1 -- 172.21.15.29:0/3312118577 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 45) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f8098025480 con 0x7f80b00ab1f0 2024-01-24T14:15:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.725+0000 7f80a7fff700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f809005f360 0x7f8090061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.725+0000 7f80a7fff700 1 -- 172.21.15.29:0/3312118577 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7f809807b410 con 0x7f80b00ab1f0 2024-01-24T14:15:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.725+0000 7f80af7fe700 1 -- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f809005f360 msgr2=0x7f8090061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:54.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.725+0000 7f80af7fe700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f809005f360 0x7f8090061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:15:54.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.730+0000 7f80a7fff700 1 -- 172.21.15.29:0/3312118577 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f809804bed0 con 0x7f80b00ab1f0 2024-01-24T14:15:54.773 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:54.772+0000 7f91b8078680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:15:54.921 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:54.920+0000 7f91b8078680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:15:54.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.926+0000 7f80af7fe700 1 -- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f809005f360 msgr2=0x7f8090061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:54.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.926+0000 7f80af7fe700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f809005f360 0x7f8090061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:15:54.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.959+0000 7f80b5c53700 1 -- 172.21.15.29:0/3312118577 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80b00cbb80 con 0x7f80b00ab1f0 2024-01-24T14:15:54.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.959+0000 7f80a7fff700 1 -- 172.21.15.29:0/3312118577 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f8098016030 con 0x7f80b00ab1f0 2024-01-24T14:15:54.960 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:15:54.960 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6341,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4230829295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4230829295},{"type":"v1","addr":"172.21.15.29:6835","nonce":4230829295}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6344,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3194389571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3194389571},{"type":"v1","addr":"172.21.15.163:6836","nonce":3194389571}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2473497045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2473497045},{"type":"v1","addr":"172.21.15.163:6837","nonce":2473497045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:14:35.870228+0000","modified":"2024-01-24T14:14:38.905491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7254":{"gid":7254,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/3097015540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3097015540},{"type":"v1","addr":"172.21.15.29:6834","nonce":3097015540}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-24T14:15:54.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.961+0000 7f80b5c53700 1 -- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f809005f360 msgr2=0x7f8090061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:15:54.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.961+0000 7f80b5c53700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f809005f360 0x7f8090061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:54.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.961+0000 7f80b5c53700 1 -- 172.21.15.29:0/3312118577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80b00ab1f0 msgr2=0x7f80b014fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:54.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.961+0000 7f80b5c53700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80b00ab1f0 0x7f80b014fec0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f809800e9d0 tx=0x7f8098009060).stop 2024-01-24T14:15:54.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.961+0000 7f80b5c53700 1 -- 172.21.15.29:0/3312118577 shutdown_connections 2024-01-24T14:15:54.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.961+0000 7f80b5c53700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b0154370 0x7f80b00c12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.961+0000 7f80b5c53700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80b00aa0e0 0x7f80b013fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.961+0000 7f80b5c53700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f809005f360 0x7f8090061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:54.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.961+0000 7f80b5c53700 1 --2- 172.21.15.29:0/3312118577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80b00ab1f0 0x7f80b014fec0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:54.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.962+0000 7f80b5c53700 1 -- 172.21.15.29:0/3312118577 >> 172.21.15.29:0/3312118577 conn(0x7f80b005ec80 msgr2=0x7f80b01519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:54.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.962+0000 7f80b5c53700 1 -- 172.21.15.29:0/3312118577 shutdown_connections 2024-01-24T14:15:54.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:54.962+0000 7f80b5c53700 1 -- 172.21.15.29:0/3312118577 wait complete. 2024-01-24T14:15:54.963 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 51 2024-01-24T14:15:54.976 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:15:54.991 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:54.990+0000 7f91b8078680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:15:55.108 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.107+0000 7f91b8078680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:15:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.110+0000 7fab1764a700 1 Processor -- start 2024-01-24T14:15:55.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.110+0000 7fab1764a700 1 -- start start 2024-01-24T14:15:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab1764a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab10154370 0x7fab100c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab1764a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab100aa0e0 0x7fab10154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab1764a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab100aa6c0 con 0x7fab100ab1f0 2024-01-24T14:15:55.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab1764a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab10064240 con 0x7fab100aa0e0 2024-01-24T14:15:55.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab1764a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab100cb5c0 con 0x7fab10154370 2024-01-24T14:15:55.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab14be5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab100aa0e0 0x7fab10154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab15be7700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fab100ab1f0 0x7fab100631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39638/0 (socket says 172.21.15.29:39638) 2024-01-24T14:15:55.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab15be7700 1 -- 172.21.15.29:0/2680188448 learned_addr learned my addr 172.21.15.29:0/2680188448 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:55.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab153e6700 1 --2- 172.21.15.29:0/2680188448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab10154370 0x7fab100c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab14be5700 1 -- 172.21.15.29:0/2680188448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab10154370 msgr2=0x7fab100c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:55.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab14be5700 1 --2- 172.21.15.29:0/2680188448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab10154370 0x7fab100c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab14be5700 1 -- 172.21.15.29:0/2680188448 >> v1:172.21.15.29:6789/0 conn(0x7fab100ab1f0 legacy=0x7fab100631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.111+0000 7fab14be5700 1 -- 172.21.15.29:0/2680188448 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab100cb8a0 con 0x7fab100aa0e0 2024-01-24T14:15:55.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.112+0000 7fab14be5700 1 --2- 172.21.15.29:0/2680188448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab100aa0e0 0x7fab10154750 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7fab00009760 tx=0x7fab00009060).ready entity=mon.0 client_cookie=f1584fd0f462cf1c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:55.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.112+0000 7fab07fff700 1 -- 172.21.15.29:0/2680188448 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0000f510 con 0x7fab100aa0e0 2024-01-24T14:15:55.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.112+0000 7fab07fff700 1 -- 172.21.15.29:0/2680188448 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0000f670 con 0x7fab100aa0e0 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.112+0000 7fab07fff700 1 -- 172.21.15.29:0/2680188448 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0000f9f0 con 0x7fab100aa0e0 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.113+0000 7fab1764a700 1 -- 172.21.15.29:0/2680188448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab100aa0e0 msgr2=0x7fab10154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.113+0000 7fab1764a700 1 --2- 172.21.15.29:0/2680188448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab100aa0e0 0x7fab10154750 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7fab00009760 tx=0x7fab00009060).stop 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.113+0000 7fab1764a700 1 -- 172.21.15.29:0/2680188448 shutdown_connections 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.113+0000 7fab1764a700 1 --2- 172.21.15.29:0/2680188448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab10154370 0x7fab100c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.113+0000 7fab1764a700 1 --2- 172.21.15.29:0/2680188448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab100aa0e0 0x7fab10154750 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.113+0000 7fab1764a700 1 -- 172.21.15.29:0/2680188448 >> 172.21.15.29:0/2680188448 conn(0x7fab1005ec80 msgr2=0x7fab100cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.113+0000 7fab1764a700 1 -- 172.21.15.29:0/2680188448 shutdown_connections 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.113+0000 7fab1764a700 1 -- 172.21.15.29:0/2680188448 wait complete. 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.114+0000 7fab1764a700 1 Processor -- start 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.114+0000 7fab1764a700 1 -- start start 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.115+0000 7fab1764a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab100aa0e0 0x7fab10159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.115+0000 7fab1764a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab100ab1f0 0x7fab10168630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.115+0000 7fab1764a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab10154370 0x7fab10152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.115+0000 7fab1764a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab10165020 con 0x7fab10154370 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.115+0000 7fab1764a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab10150450 con 0x7fab100ab1f0 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.115+0000 7fab1764a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab10165180 con 0x7fab100aa0e0 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.115+0000 7fab15be7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab10154370 0x7fab10152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.115+0000 7fab15be7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab10154370 0x7fab10152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38902/0 (socket says 172.21.15.29:38902) 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.115+0000 7fab153e6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab100aa0e0 0x7fab10159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.115+0000 7fab15be7700 1 -- 172.21.15.29:0/3280560648 learned_addr learned my addr 172.21.15.29:0/3280560648 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.115+0000 7fab14be5700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab100ab1f0 0x7fab10168630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.116+0000 7fab153e6700 1 -- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab100ab1f0 msgr2=0x7fab10168630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.116+0000 7fab153e6700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab100ab1f0 0x7fab10168630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.116+0000 7fab153e6700 1 -- 172.21.15.29:0/3280560648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab10154370 msgr2=0x7fab10152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.116+0000 7fab153e6700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab10154370 0x7fab10152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.116+0000 7fab153e6700 1 -- 172.21.15.29:0/3280560648 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab000092a0 con 0x7fab100aa0e0 2024-01-24T14:15:55.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.116+0000 7fab15be7700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab10154370 0x7fab10152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:55.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.116+0000 7fab153e6700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab100aa0e0 0x7fab10159d20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fab0c00e8d0 tx=0x7fab0c006f40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:55.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.116+0000 7fab057fa700 1 -- 172.21.15.29:0/3280560648 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c012600 con 0x7fab100aa0e0 2024-01-24T14:15:55.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.116+0000 7fab057fa700 1 -- 172.21.15.29:0/3280560648 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c012760 con 0x7fab100aa0e0 2024-01-24T14:15:55.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.116+0000 7fab057fa700 1 -- 172.21.15.29:0/3280560648 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c012ae0 con 0x7fab100aa0e0 2024-01-24T14:15:55.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.116+0000 7fab1764a700 1 -- 172.21.15.29:0/3280560648 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab101571c0 con 0x7fab100aa0e0 2024-01-24T14:15:55.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.116+0000 7fab1764a700 1 -- 172.21.15.29:0/3280560648 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab10157780 con 0x7fab100aa0e0 2024-01-24T14:15:55.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.117+0000 7fab1764a700 1 -- 172.21.15.29:0/3280560648 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab100caad0 con 0x7fab100aa0e0 2024-01-24T14:15:55.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.117+0000 7fab057fa700 1 -- 172.21.15.29:0/3280560648 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 45) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fab0c01b400 con 0x7fab100aa0e0 2024-01-24T14:15:55.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.118+0000 7fab057fa700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaf405f360 0x7faaf4061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.118+0000 7fab14be5700 1 -- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaf405f360 msgr2=0x7faaf4061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:55.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.118+0000 7fab057fa700 1 -- 172.21.15.29:0/3280560648 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7fab0c07ad80 con 0x7fab100aa0e0 2024-01-24T14:15:55.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.118+0000 7fab14be5700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaf405f360 0x7faaf4061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:15:55.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.122+0000 7fab057fa700 1 -- 172.21.15.29:0/3280560648 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fab0c04b7e0 con 0x7fab100aa0e0 2024-01-24T14:15:55.218 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.217+0000 7f91b8078680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:15:55.232 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:55.231+0000 7fadf8e6f680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:15:55.312 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:55.311+0000 7fadf8e6f680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:15:55.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.319+0000 7fab14be5700 1 -- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaf405f360 msgr2=0x7faaf4061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:55.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.319+0000 7fab14be5700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaf405f360 0x7faaf4061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:15:55.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.349+0000 7fab1764a700 1 -- 172.21.15.29:0/3280560648 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab100475e0 con 0x7fab100aa0e0 2024-01-24T14:15:55.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.350+0000 7fab057fa700 1 -- 172.21.15.29:0/3280560648 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fab0c04ee00 con 0x7fab100aa0e0 2024-01-24T14:15:55.351 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:15:55.351 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6341,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4230829295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4230829295},{"type":"v1","addr":"172.21.15.29:6835","nonce":4230829295}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6344,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3194389571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3194389571},{"type":"v1","addr":"172.21.15.163:6836","nonce":3194389571}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2473497045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2473497045},{"type":"v1","addr":"172.21.15.163:6837","nonce":2473497045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:14:35.870228+0000","modified":"2024-01-24T14:14:38.905491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7254":{"gid":7254,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/3097015540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3097015540},{"type":"v1","addr":"172.21.15.29:6834","nonce":3097015540}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-24T14:15:55.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.352+0000 7fab1764a700 1 -- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaf405f360 msgr2=0x7faaf4061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:15:55.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.352+0000 7fab1764a700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaf405f360 0x7faaf4061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:55.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.352+0000 7fab1764a700 1 -- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab100aa0e0 msgr2=0x7fab10159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.352+0000 7fab1764a700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab100aa0e0 0x7fab10159d20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fab0c00e8d0 tx=0x7fab0c006f40).stop 2024-01-24T14:15:55.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.352+0000 7fab1764a700 1 -- 172.21.15.29:0/3280560648 shutdown_connections 2024-01-24T14:15:55.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.352+0000 7fab1764a700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaf405f360 0x7faaf4061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:55.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.352+0000 7fab1764a700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab10154370 0x7fab10152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.352+0000 7fab1764a700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab100aa0e0 0x7fab10159d20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.352+0000 7fab1764a700 1 --2- 172.21.15.29:0/3280560648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab100ab1f0 0x7fab10168630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.352+0000 7fab1764a700 1 -- 172.21.15.29:0/3280560648 >> 172.21.15.29:0/3280560648 conn(0x7fab1005ec80 msgr2=0x7fab10151890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:55.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.352+0000 7fab1764a700 1 -- 172.21.15.29:0/3280560648 shutdown_connections 2024-01-24T14:15:55.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.352+0000 7fab1764a700 1 -- 172.21.15.29:0/3280560648 wait complete. 2024-01-24T14:15:55.354 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 51 2024-01-24T14:15:55.366 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:15:55.403 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:55.402+0000 7fadf8e6f680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:15:55.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.501+0000 7f63415cc700 1 Processor -- start 2024-01-24T14:15:55.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.501+0000 7f63415cc700 1 -- start start 2024-01-24T14:15:55.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.502+0000 7f63415cc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 0x7f633c0638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.502+0000 7f63415cc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f633c066c00 0x7f633c0bf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.502+0000 7f63415cc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633c068400 con 0x7f633c154060 2024-01-24T14:15:55.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.502+0000 7f63415cc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633c0bfad0 con 0x7f633c066c00 2024-01-24T14:15:55.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.502+0000 7f63415cc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c1544c0 con 0x7f633c153c80 2024-01-24T14:15:55.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.502+0000 7f633affd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 0x7f633c0638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.502+0000 7f633affd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 0x7f633c0638c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:50090/0 (socket says 172.21.15.29:50090) 2024-01-24T14:15:55.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.502+0000 7f633b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f633c066c00 0x7f633c0bf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.502+0000 7f633affd700 1 -- 172.21.15.29:0/2850404273 learned_addr learned my addr 172.21.15.29:0/2850404273 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:55.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.502+0000 7f633a7fc700 1 --1- 172.21.15.29:0/2850404273 >> v1:172.21.15.163:6789/0 conn(0x7f633c154060 0x7f633c0664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:35968/0 (socket says 172.21.15.29:35968) 2024-01-24T14:15:55.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.503+0000 7f633affd700 1 -- 172.21.15.29:0/2850404273 >> v2:172.21.15.163:3300/0 conn(0x7f633c066c00 msgr2=0x7f633c0bf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.503+0000 7f633affd700 1 --2- 172.21.15.29:0/2850404273 >> v2:172.21.15.163:3300/0 conn(0x7f633c066c00 0x7f633c0bf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.503+0000 7f633affd700 1 -- 172.21.15.29:0/2850404273 >> v1:172.21.15.163:6789/0 conn(0x7f633c154060 legacy=0x7f633c0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.503+0000 7f633affd700 1 -- 172.21.15.29:0/2850404273 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c067ad0 con 0x7f633c153c80 2024-01-24T14:15:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.503+0000 7f633affd700 1 --2- 172.21.15.29:0/2850404273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 0x7f633c0638c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f632c006fd0 tx=0x7f632c0090f0).ready entity=mon.2 client_cookie=2361485502230f29 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.503+0000 7f6339ffb700 1 -- 172.21.15.29:0/2850404273 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00b580 con 0x7f633c153c80 2024-01-24T14:15:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.503+0000 7f6339ffb700 1 -- 172.21.15.29:0/2850404273 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c00b6e0 con 0x7f633c153c80 2024-01-24T14:15:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.503+0000 7f6339ffb700 1 -- 172.21.15.29:0/2850404273 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00ba60 con 0x7f633c153c80 2024-01-24T14:15:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.503+0000 7f63415cc700 1 -- 172.21.15.29:0/2850404273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 msgr2=0x7f633c0638c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.504+0000 7f63415cc700 1 --2- 172.21.15.29:0/2850404273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 0x7f633c0638c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f632c006fd0 tx=0x7f632c0090f0).stop 2024-01-24T14:15:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.504+0000 7f63415cc700 1 -- 172.21.15.29:0/2850404273 shutdown_connections 2024-01-24T14:15:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.504+0000 7f63415cc700 1 --2- 172.21.15.29:0/2850404273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 0x7f633c0638c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.504+0000 7f63415cc700 1 --2- 172.21.15.29:0/2850404273 >> v2:172.21.15.163:3300/0 conn(0x7f633c066c00 0x7f633c0bf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.504+0000 7f63415cc700 1 -- 172.21.15.29:0/2850404273 >> 172.21.15.29:0/2850404273 conn(0x7f633c0c8bb0 msgr2=0x7f633c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.504+0000 7f63415cc700 1 -- 172.21.15.29:0/2850404273 shutdown_connections 2024-01-24T14:15:55.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.504+0000 7f63415cc700 1 -- 172.21.15.29:0/2850404273 wait complete. 2024-01-24T14:15:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.505+0000 7f63415cc700 1 Processor -- start 2024-01-24T14:15:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.505+0000 7f63415cc700 1 -- start start 2024-01-24T14:15:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f63415cc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f633c066c00 0x7f633c13ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f63415cc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 0x7f633c0bfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f63415cc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f633c154060 0x7f633c1525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f63415cc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c13b0f0 con 0x7f633c154060 2024-01-24T14:15:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f63415cc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c155a30 con 0x7f633c066c00 2024-01-24T14:15:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f63415cc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c155d10 con 0x7f633c153c80 2024-01-24T14:15:55.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f633b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f633c154060 0x7f633c1525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f633a7fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 0x7f633c0bfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f633affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f633c066c00 0x7f633c13ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f633b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f633c154060 0x7f633c1525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38912/0 (socket says 172.21.15.29:38912) 2024-01-24T14:15:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f633b7fe700 1 -- 172.21.15.29:0/1939142643 learned_addr learned my addr 172.21.15.29:0/1939142643 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f633a7fc700 1 -- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f633c066c00 msgr2=0x7f633c13ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f633a7fc700 1 --2- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f633c066c00 0x7f633c13ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f633a7fc700 1 -- 172.21.15.29:0/1939142643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f633c154060 msgr2=0x7f633c1525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.506+0000 7f633a7fc700 1 --2- 172.21.15.29:0/1939142643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f633c154060 0x7f633c1525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.507+0000 7f633a7fc700 1 -- 172.21.15.29:0/1939142643 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c009370 con 0x7f633c153c80 2024-01-24T14:15:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.507+0000 7f633a7fc700 1 --2- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 0x7f633c0bfde0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f633c065aa0 tx=0x7f6330007d30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.507+0000 7f6322ffd700 1 -- 172.21.15.29:0/1939142643 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300070d0 con 0x7f633c153c80 2024-01-24T14:15:55.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.507+0000 7f6322ffd700 1 -- 172.21.15.29:0/1939142643 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6330007230 con 0x7f633c153c80 2024-01-24T14:15:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.507+0000 7f63415cc700 1 -- 172.21.15.29:0/1939142643 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c155ff0 con 0x7f633c153c80 2024-01-24T14:15:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.507+0000 7f6322ffd700 1 -- 172.21.15.29:0/1939142643 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300075b0 con 0x7f633c153c80 2024-01-24T14:15:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.507+0000 7f63415cc700 1 -- 172.21.15.29:0/1939142643 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c1565b0 con 0x7f633c153c80 2024-01-24T14:15:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.508+0000 7f63415cc700 1 -- 172.21.15.29:0/1939142643 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c1532e0 con 0x7f633c153c80 2024-01-24T14:15:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.508+0000 7f6322ffd700 1 -- 172.21.15.29:0/1939142643 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 45) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f63300058f0 con 0x7f633c153c80 2024-01-24T14:15:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.509+0000 7f6322ffd700 1 --2- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f632805f390 0x7f6328061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.509+0000 7f633affd700 1 -- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f632805f390 msgr2=0x7f6328061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:15:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.509+0000 7f6322ffd700 1 -- 172.21.15.29:0/1939142643 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7516+0+0 (secure 0 0 0) 0x7f633007a0b0 con 0x7f633c153c80 2024-01-24T14:15:55.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.509+0000 7f633affd700 1 --2- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f632805f390 0x7f6328061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:15:55.513 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.512+0000 7f91b8078680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:15:55.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.513+0000 7f6322ffd700 1 -- 172.21.15.29:0/1939142643 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f633004ab10 con 0x7f633c153c80 2024-01-24T14:15:55.580 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:55.580+0000 7fadf8e6f680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:15:55.585 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.584+0000 7f91b8078680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:15:55.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.609+0000 7f6322ffd700 1 -- 172.21.15.29:0/1939142643 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mgrmap(e 46) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f6330077610 con 0x7f633c153c80 2024-01-24T14:15:55.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.609+0000 7f6322ffd700 1 -- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f632805f390 msgr2=0x7f6328061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:15:55.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.609+0000 7f6322ffd700 1 --2- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f632805f390 0x7f6328061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:55.665 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.664+0000 7f91345ff700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:55.666 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.664+0000 7f91345ff700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:55.666 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.664+0000 7f91345ff700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:55.666 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.664+0000 7f91345ff700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:55.666 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.664+0000 7f91345ff700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:55.666 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.664+0000 7f91315f9700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:55.666 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.664+0000 7f91315f9700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:55.666 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.664+0000 7f91315f9700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:55.666 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.664+0000 7f91315f9700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:55.666 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:15:55.664+0000 7f91315f9700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:15:55.671 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:55.671+0000 7fadf8e6f680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:15:55.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.706+0000 7f63415cc700 1 -- 172.21.15.29:0/1939142643 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f633c155e50 con 0x7f633c153c80 2024-01-24T14:15:55.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.708+0000 7f6322ffd700 1 -- 172.21.15.29:0/1939142643 <== mon.2 v2:172.21.15.163:3301/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+19130 (secure 0 0 0) 0x7f633c155e50 con 0x7f633c153c80 2024-01-24T14:15:55.709 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:15:55.709 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":124,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:15:55.599467+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:14:33.610829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-24T14:14:34.834814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.18","mappings":[{"from":1,"to":7}]},{"pgid":"14.29","mappings":[{"from":1,"to":2}]},{"pgid":"14.36","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/4094435573":"2024-01-25T14:15:55.599392+0000","172.21.15.163:0/2938540236":"2024-01-25T14:15:55.599392+0000","172.21.15.163:0/3363496559":"2024-01-25T14:15:55.599392+0000","172.21.15.163:0/1675876377":"2024-01-25T14:15:55.599392+0000","172.21.15.163:0/1347250197":"2024-01-25T14:15:55.599392+0000","172.21.15.163:0/3059185022":"2024-01-25T14:15:55.599392+0000","172.21.15.163:0/2260886250":"2024-01-25T14:14:48.183216+0000","172.21.15.163:6833/43241":"2024-01-25T14:14:48.183216+0000","172.21.15.163:6832/43241":"2024-01-25T14:14:48.183216+0000","172.21.15.163:0/2891867602":"2024-01-25T14:14:48.183216+0000","172.21.15.163:0/3822418593":"2024-01-25T14:14:48.183216+0000","172.21.15.163:0/4052979774":"2024-01-25T14:14:48.183216+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:15:55.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.710+0000 7f63415cc700 1 -- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 msgr2=0x7f633c0bfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.710+0000 7f63415cc700 1 --2- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 0x7f633c0bfde0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f633c065aa0 tx=0x7f6330007d30).stop 2024-01-24T14:15:55.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.710+0000 7f63415cc700 1 -- 172.21.15.29:0/1939142643 shutdown_connections 2024-01-24T14:15:55.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.710+0000 7f63415cc700 1 --2- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f632805f390 0x7f6328061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:15:55.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.710+0000 7f63415cc700 1 --2- 172.21.15.29:0/1939142643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f633c154060 0x7f633c1525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.710+0000 7f63415cc700 1 --2- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f633c066c00 0x7f633c13ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.710+0000 7f63415cc700 1 --2- 172.21.15.29:0/1939142643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f633c153c80 0x7f633c0bfde0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.710+0000 7f63415cc700 1 -- 172.21.15.29:0/1939142643 >> 172.21.15.29:0/1939142643 conn(0x7f633c0c8bb0 msgr2=0x7f633c14d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:55.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.710+0000 7f63415cc700 1 -- 172.21.15.29:0/1939142643 shutdown_connections 2024-01-24T14:15:55.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.710+0000 7f63415cc700 1 -- 172.21.15.29:0/1939142643 wait complete. 2024-01-24T14:15:55.725 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-24T14:15:55.808 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:55.807+0000 7fadf8e6f680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:15:55.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.898+0000 7f1ad2d5e700 1 Processor -- start 2024-01-24T14:15:55.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.898+0000 7f1ad2d5e700 1 -- start start 2024-01-24T14:15:55.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad2d5e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1acc151aa0 0x7f1acc0a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad2d5e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1acc151e80 0x7f1acc0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad2d5e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1acc065360 con 0x7f1acc0a71d0 2024-01-24T14:15:55.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad2d5e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1acc0bcab0 con 0x7f1acc151e80 2024-01-24T14:15:55.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad2d5e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0bcd90 con 0x7f1acc151aa0 2024-01-24T14:15:55.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad12fb700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1acc0a71d0 0x7f1acc157c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39640/0 (socket says 172.21.15.29:39640) 2024-01-24T14:15:55.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad12fb700 1 -- 172.21.15.29:0/1182735903 learned_addr learned my addr 172.21.15.29:0/1182735903 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:55.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad0afa700 1 --2- 172.21.15.29:0/1182735903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1acc151aa0 0x7f1acc0a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1acbfff700 1 --2- 172.21.15.29:0/1182735903 >> v2:172.21.15.163:3300/0 conn(0x7f1acc151e80 0x7f1acc0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad0afa700 1 -- 172.21.15.29:0/1182735903 >> v2:172.21.15.163:3300/0 conn(0x7f1acc151e80 msgr2=0x7f1acc0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1acb7fe700 1 -- 172.21.15.29:0/1182735903 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429142531 0 0) 0x7f1acc065360 con 0x7f1acc0a71d0 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad0afa700 1 --2- 172.21.15.29:0/1182735903 >> v2:172.21.15.163:3300/0 conn(0x7f1acc151e80 0x7f1acc0627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad0afa700 1 -- 172.21.15.29:0/1182735903 >> v1:172.21.15.29:6789/0 conn(0x7f1acc0a71d0 legacy=0x7f1acc157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad0afa700 1 -- 172.21.15.29:0/1182735903 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc0bd070 con 0x7f1acc151aa0 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1acbfff700 1 --2- 172.21.15.29:0/1182735903 >> v2:172.21.15.163:3300/0 conn(0x7f1acc151e80 0x7f1acc0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.900+0000 7f1ad0afa700 1 --2- 172.21.15.29:0/1182735903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1acc151aa0 0x7f1acc0a6ce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f1ac4009e60 tx=0x7f1ac4009780).ready entity=mon.2 client_cookie=2c3c8a7c445e2c84 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.901+0000 7f1acb7fe700 1 -- 172.21.15.29:0/1182735903 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac400e070 con 0x7f1acc151aa0 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.901+0000 7f1acb7fe700 1 -- 172.21.15.29:0/1182735903 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac400bb60 con 0x7f1acc151aa0 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.901+0000 7f1acb7fe700 1 -- 172.21.15.29:0/1182735903 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4012410 con 0x7f1acc151aa0 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.901+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/1182735903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1acc151aa0 msgr2=0x7f1acc0a6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.901+0000 7f1ad2d5e700 1 --2- 172.21.15.29:0/1182735903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1acc151aa0 0x7f1acc0a6ce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f1ac4009e60 tx=0x7f1ac4009780).stop 2024-01-24T14:15:55.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.901+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/1182735903 shutdown_connections 2024-01-24T14:15:55.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.901+0000 7f1ad2d5e700 1 --2- 172.21.15.29:0/1182735903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1acc151aa0 0x7f1acc0a6ce0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.901+0000 7f1ad2d5e700 1 --2- 172.21.15.29:0/1182735903 >> v2:172.21.15.163:3300/0 conn(0x7f1acc151e80 0x7f1acc0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.901+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/1182735903 >> 172.21.15.29:0/1182735903 conn(0x7f1acc0c8580 msgr2=0x7f1acc0c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:55.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.901+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/1182735903 shutdown_connections 2024-01-24T14:15:55.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.902+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/1182735903 wait complete. 2024-01-24T14:15:55.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.903+0000 7f1ad2d5e700 1 Processor -- start 2024-01-24T14:15:55.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.903+0000 7f1ad2d5e700 1 -- start start 2024-01-24T14:15:55.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.904+0000 7f1ad2d5e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1acc0a71d0 0x7f1acc13be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.904+0000 7f1ad2d5e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1acc151aa0 0x7f1acc153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.904+0000 7f1ad2d5e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1acc151e80 0x7f1acc153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:55.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.904+0000 7f1ad2d5e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0aa580 con 0x7f1acc151aa0 2024-01-24T14:15:55.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.904+0000 7f1ad2d5e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc154410 con 0x7f1acc0a71d0 2024-01-24T14:15:55.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.904+0000 7f1ad2d5e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc1643e0 con 0x7f1acc151e80 2024-01-24T14:15:55.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.904+0000 7f1ad0afa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1acc0a71d0 0x7f1acc13be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.904+0000 7f1acbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1acc151aa0 0x7f1acc153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.904+0000 7f1ad12fb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1acc151e80 0x7f1acc153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:55.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.904+0000 7f1ad0afa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1acc0a71d0 0x7f1acc13be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:40136/0 (socket says 172.21.15.29:40136) 2024-01-24T14:15:55.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.904+0000 7f1acbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1acc151aa0 0x7f1acc153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38926/0 (socket says 172.21.15.29:38926) 2024-01-24T14:15:55.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.904+0000 7f1ad0afa700 1 -- 172.21.15.29:0/649277174 learned_addr learned my addr 172.21.15.29:0/649277174 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:55.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1ad12fb700 1 -- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1acc0a71d0 msgr2=0x7f1acc13be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1ad12fb700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1acc0a71d0 0x7f1acc13be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1ad12fb700 1 -- 172.21.15.29:0/649277174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1acc151aa0 msgr2=0x7f1acc153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:55.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1ad12fb700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1acc151aa0 0x7f1acc153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:55.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1ad12fb700 1 -- 172.21.15.29:0/649277174 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00092a0 con 0x7f1acc151e80 2024-01-24T14:15:55.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1ad0afa700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1acc0a71d0 0x7f1acc13be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:55.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1acbfff700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1acc151aa0 0x7f1acc153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:55.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1ad12fb700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1acc151e80 0x7f1acc153f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f1abc001eb0 tx=0x7f1abc007b30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:55.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1ac8ff9700 1 -- 172.21.15.29:0/649277174 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0070d0 con 0x7f1acc151e80 2024-01-24T14:15:55.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1ac8ff9700 1 -- 172.21.15.29:0/649277174 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc007250 con 0x7f1acc151e80 2024-01-24T14:15:55.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1ac8ff9700 1 -- 172.21.15.29:0/649277174 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc007610 con 0x7f1acc151e80 2024-01-24T14:15:55.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/649277174 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac4009a20 con 0x7f1acc151e80 2024-01-24T14:15:55.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.905+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/649277174 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc164980 con 0x7f1acc151e80 2024-01-24T14:15:55.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.907+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/649277174 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1acc1652b0 con 0x7f1acc151e80 2024-01-24T14:15:55.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.907+0000 7f1ac8ff9700 1 -- 172.21.15.29:0/649277174 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 46) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f1abc0059b0 con 0x7f1acc151e80 2024-01-24T14:15:55.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.907+0000 7f1ac8ff9700 1 -- 172.21.15.29:0/649277174 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 7774+0+0 (secure 0 0 0) 0x7f1abc079aa0 con 0x7f1acc151e80 2024-01-24T14:15:55.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:55.911+0000 7f1ac8ff9700 1 -- 172.21.15.29:0/649277174 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1abc04a8d0 con 0x7f1acc151e80 2024-01-24T14:15:55.939 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:55.938+0000 7fadf8e6f680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:15:56.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.141+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/649277174 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1acc0475e0 con 0x7f1acc151e80 2024-01-24T14:15:56.277 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:56.276+0000 7fadf8e6f680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:15:56.360 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:15:56.360+0000 7fadf8e6f680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:15:56.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.615+0000 7f1ac8ff9700 1 -- 172.21.15.29:0/649277174 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mgrmap(e 47) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f1abc0773d0 con 0x7f1acc151e80 2024-01-24T14:15:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.616+0000 7f1ac8ff9700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ab80302f0 0x7f1ab8068be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.616+0000 7f1ad0afa700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ab80302f0 0x7f1ab8068be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:56.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.617+0000 7f1ad0afa700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ab80302f0 0x7f1ab8068be0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f1ac4009fd0 tx=0x7f1ac4009780).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:56.855 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-01-24T14:15:56.854+0000 7fe09bfc3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-24T14:15:56.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.860+0000 7f1ac8ff9700 1 -- 172.21.15.29:0/649277174 <== mon.2 v2:172.21.15.163:3301/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1abc0732d0 con 0x7f1acc151e80 2024-01-24T14:15:56.861 INFO:tasks.ceph.mds.c.smithi029.stderr: -7332> 2024-01-24T14:14:30.390+0000 7f6167040700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-01-24T14:15:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.862+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ab80302f0 msgr2=0x7f1ab8068be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:56.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.862+0000 7f1ad2d5e700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ab80302f0 0x7f1ab8068be0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f1ac4009fd0 tx=0x7f1ac4009780).stop 2024-01-24T14:15:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.862+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1acc151e80 msgr2=0x7f1acc153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.862+0000 7f1ad2d5e700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1acc151e80 0x7f1acc153f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f1abc001eb0 tx=0x7f1abc007b30).stop 2024-01-24T14:15:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.862+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/649277174 shutdown_connections 2024-01-24T14:15:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.862+0000 7f1ad2d5e700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1acc151e80 0x7f1acc153f20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.862+0000 7f1ad2d5e700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1acc0a71d0 0x7f1acc13be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.862+0000 7f1ad2d5e700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1ab80302f0 0x7f1ab8068be0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.862+0000 7f1ad2d5e700 1 --2- 172.21.15.29:0/649277174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1acc151aa0 0x7f1acc153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.863+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/649277174 >> 172.21.15.29:0/649277174 conn(0x7f1acc0c8580 msgr2=0x7f1acc063570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.863+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/649277174 shutdown_connections 2024-01-24T14:15:56.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:56.863+0000 7f1ad2d5e700 1 -- 172.21.15.29:0/649277174 wait complete. 2024-01-24T14:15:56.864 INFO:tasks.ceph.mds.c.smithi029.stderr: -6119> 2024-01-24T14:14:41.399+0000 7f6161034700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T14:15:56.865 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-24T14:15:56.879 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-24T14:15:56.894 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T14:15:57.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.029+0000 7f69dfae7700 1 Processor -- start 2024-01-24T14:15:57.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.029+0000 7f69dfae7700 1 -- start start 2024-01-24T14:15:57.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.030+0000 7f69dfae7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69d8066520 0x7f69d8066960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:57.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.030+0000 7f69dfae7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f69d80be510 0x7f69d80be9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:57.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.030+0000 7f69dfae7700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d80be080 con 0x7f69d8157530 2024-01-24T14:15:57.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.030+0000 7f69dfae7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d8065040 con 0x7f69d80be510 2024-01-24T14:15:57.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.030+0000 7f69dfae7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d8064140 con 0x7f69d8066520 2024-01-24T14:15:57.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.030+0000 7f69dd883700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f69d8157530 0x7f69d8068470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39650/0 (socket says 172.21.15.29:39650) 2024-01-24T14:15:57.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.030+0000 7f69dd883700 1 -- 172.21.15.29:0/357479382 learned_addr learned my addr 172.21.15.29:0/357479382 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:57.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.030+0000 7f69dd082700 1 --2- 172.21.15.29:0/357479382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69d8066520 0x7f69d8066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:57.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.030+0000 7f69de084700 1 --2- 172.21.15.29:0/357479382 >> v2:172.21.15.163:3300/0 conn(0x7f69d80be510 0x7f69d80be9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:57.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.030+0000 7f69dc881700 1 -- 172.21.15.29:0/357479382 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917338587 0 0) 0x7f69d80be080 con 0x7f69d8157530 2024-01-24T14:15:57.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.030+0000 7f69dc881700 1 -- 172.21.15.29:0/357479382 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69c40033f0 con 0x7f69d8157530 2024-01-24T14:15:57.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.031+0000 7f69dc881700 1 -- 172.21.15.29:0/357479382 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1794785400 0 0) 0x7f69c40033f0 con 0x7f69d8157530 2024-01-24T14:15:57.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.031+0000 7f69dd082700 1 -- 172.21.15.29:0/357479382 >> v2:172.21.15.163:3300/0 conn(0x7f69d80be510 msgr2=0x7f69d80be9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:57.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.031+0000 7f69dd082700 1 --2- 172.21.15.29:0/357479382 >> v2:172.21.15.163:3300/0 conn(0x7f69d80be510 0x7f69d80be9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:57.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.031+0000 7f69dd082700 1 -- 172.21.15.29:0/357479382 >> v1:172.21.15.29:6789/0 conn(0x7f69d8157530 legacy=0x7f69d8068470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:57.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.031+0000 7f69dd082700 1 -- 172.21.15.29:0/357479382 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d8067380 con 0x7f69d8066520 2024-01-24T14:15:57.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.031+0000 7f69de084700 1 --2- 172.21.15.29:0/357479382 >> v2:172.21.15.163:3300/0 conn(0x7f69d80be510 0x7f69d80be9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:15:57.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.031+0000 7f69dd082700 1 --2- 172.21.15.29:0/357479382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69d8066520 0x7f69d8066960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f69d4009760 tx=0x7f69d4009060).ready entity=mon.2 client_cookie=55d3f91cc7c13805 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:57.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.031+0000 7f69dc881700 1 -- 172.21.15.29:0/357479382 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8001470 con 0x7f69d8066520 2024-01-24T14:15:57.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.031+0000 7f69dc881700 1 -- 172.21.15.29:0/357479382 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d4011400 con 0x7f69d8066520 2024-01-24T14:15:57.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.031+0000 7f69dc881700 1 -- 172.21.15.29:0/357479382 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d4011780 con 0x7f69d8066520 2024-01-24T14:15:57.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.032+0000 7f69dfae7700 1 -- 172.21.15.29:0/357479382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69d8066520 msgr2=0x7f69d8066960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:57.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.032+0000 7f69dfae7700 1 --2- 172.21.15.29:0/357479382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69d8066520 0x7f69d8066960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f69d4009760 tx=0x7f69d4009060).stop 2024-01-24T14:15:57.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.032+0000 7f69dfae7700 1 -- 172.21.15.29:0/357479382 shutdown_connections 2024-01-24T14:15:57.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.032+0000 7f69dfae7700 1 --2- 172.21.15.29:0/357479382 >> v2:172.21.15.163:3300/0 conn(0x7f69d80be510 0x7f69d80be9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:57.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.032+0000 7f69dfae7700 1 --2- 172.21.15.29:0/357479382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69d8066520 0x7f69d8066960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:57.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.032+0000 7f69dfae7700 1 -- 172.21.15.29:0/357479382 >> 172.21.15.29:0/357479382 conn(0x7f69d80c8ba0 msgr2=0x7f69d80c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:57.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.032+0000 7f69dfae7700 1 -- 172.21.15.29:0/357479382 shutdown_connections 2024-01-24T14:15:57.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.032+0000 7f69dfae7700 1 -- 172.21.15.29:0/357479382 wait complete. 2024-01-24T14:15:57.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.033+0000 7f69dfae7700 1 Processor -- start 2024-01-24T14:15:57.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.033+0000 7f69dfae7700 1 -- start start 2024-01-24T14:15:57.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dfae7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69d8066520 0x7f69d80c1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:57.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dfae7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69d80be510 0x7f69d814b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:57.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dfae7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69d8157530 0x7f69d80bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:57.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dfae7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80c12f0 con 0x7f69d8066520 2024-01-24T14:15:57.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dfae7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80c06e0 con 0x7f69d80be510 2024-01-24T14:15:57.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dfae7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80c09c0 con 0x7f69d8157530 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dd883700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69d8066520 0x7f69d80c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dd883700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69d8066520 0x7f69d80c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38964/0 (socket says 172.21.15.29:38964) 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dd883700 1 -- 172.21.15.29:0/3991283134 learned_addr learned my addr 172.21.15.29:0/3991283134 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dd082700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69d80be510 0x7f69d814b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69de084700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69d8157530 0x7f69d80bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dd883700 1 -- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69d8157530 msgr2=0x7f69d80bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dd883700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69d8157530 0x7f69d80bfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dd883700 1 -- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69d80be510 msgr2=0x7f69d814b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dd883700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69d80be510 0x7f69d814b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69dd883700 1 -- 172.21.15.29:0/3991283134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d0009a20 con 0x7f69d8066520 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.034+0000 7f69de084700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69d8157530 0x7f69d80bfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.035+0000 7f69dd883700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69d8066520 0x7f69d80c1780 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f69c800c810 tx=0x7f69c8001ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.035+0000 7f69cdffb700 1 -- 172.21.15.29:0/3991283134 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8007170 con 0x7f69d8066520 2024-01-24T14:15:57.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.035+0000 7f69cdffb700 1 -- 172.21.15.29:0/3991283134 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c80072d0 con 0x7f69d8066520 2024-01-24T14:15:57.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.035+0000 7f69cdffb700 1 -- 172.21.15.29:0/3991283134 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c80048c0 con 0x7f69d8066520 2024-01-24T14:15:57.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.035+0000 7f69dfae7700 1 -- 172.21.15.29:0/3991283134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d40092a0 con 0x7f69d8066520 2024-01-24T14:15:57.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.035+0000 7f69dfae7700 1 -- 172.21.15.29:0/3991283134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d80c0f80 con 0x7f69d8066520 2024-01-24T14:15:57.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.036+0000 7f69dfae7700 1 -- 172.21.15.29:0/3991283134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d8066fb0 con 0x7f69d8066520 2024-01-24T14:15:57.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.036+0000 7f69cdffb700 1 -- 172.21.15.29:0/3991283134 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 47) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f69c8004c70 con 0x7f69d8066520 2024-01-24T14:15:57.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.037+0000 7f69cdffb700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69bc05f000 0x7f69bc0614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:57.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.037+0000 7f69cdffb700 1 -- 172.21.15.29:0/3991283134 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7860+0+0 (secure 0 0 0) 0x7f69c807a220 con 0x7f69d8066520 2024-01-24T14:15:57.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.037+0000 7f69dd082700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69bc05f000 0x7f69bc0614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:57.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.038+0000 7f69dd082700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69bc05f000 0x7f69bc0614c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f69d80c55b0 tx=0x7f69d4016000).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:57.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.041+0000 7f69cdffb700 1 -- 172.21.15.29:0/3991283134 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f69c804b0a0 con 0x7f69d8066520 2024-01-24T14:15:57.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.271+0000 7f69dfae7700 1 -- 172.21.15.29:0/3991283134 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f69d80c15d0 con 0x7f69d8066520 2024-01-24T14:15:57.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.859+0000 7f69cdffb700 1 -- 172.21.15.29:0/3991283134 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) v1 ==== 104+0+0 (secure 0 0 0) 0x7f69c804e6c0 con 0x7f69d8066520 2024-01-24T14:15:57.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.862+0000 7f69dfae7700 1 -- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69bc05f000 msgr2=0x7f69bc0614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:57.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.862+0000 7f69dfae7700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69bc05f000 0x7f69bc0614c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f69d80c55b0 tx=0x7f69d4016000).stop 2024-01-24T14:15:57.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.862+0000 7f69dfae7700 1 -- 172.21.15.29:0/3991283134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69d8066520 msgr2=0x7f69d80c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:57.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.862+0000 7f69dfae7700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69d8066520 0x7f69d80c1780 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f69c800c810 tx=0x7f69c8001ab0).stop 2024-01-24T14:15:57.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.862+0000 7f69dfae7700 1 -- 172.21.15.29:0/3991283134 shutdown_connections 2024-01-24T14:15:57.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.862+0000 7f69dfae7700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69d8157530 0x7f69d80bfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:57.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.862+0000 7f69dfae7700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f69bc05f000 0x7f69bc0614c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:57.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.862+0000 7f69dfae7700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69d8066520 0x7f69d80c1780 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:57.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.862+0000 7f69dfae7700 1 --2- 172.21.15.29:0/3991283134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69d80be510 0x7f69d814b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:57.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.862+0000 7f69dfae7700 1 -- 172.21.15.29:0/3991283134 >> 172.21.15.29:0/3991283134 conn(0x7f69d80c8ba0 msgr2=0x7f69d80c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:57.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.862+0000 7f69dfae7700 1 -- 172.21.15.29:0/3991283134 shutdown_connections 2024-01-24T14:15:57.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:57.862+0000 7f69dfae7700 1 -- 172.21.15.29:0/3991283134 wait complete. 2024-01-24T14:15:57.878 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-24T14:15:58.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.016+0000 7f28e3f4f700 1 Processor -- start 2024-01-24T14:15:58.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.016+0000 7f28e3f4f700 1 -- start start 2024-01-24T14:15:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e3f4f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28dc13a6f0 0x7f28dc068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e3f4f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28dc0638c0 0x7f28dc066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e3f4f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28dc0643f0 con 0x7f28dc066940 2024-01-24T14:15:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e3f4f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28dc0bdb30 con 0x7f28dc13a6f0 2024-01-24T14:15:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e3f4f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0be3d0 con 0x7f28dc0638c0 2024-01-24T14:15:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e1ceb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28dc13a6f0 0x7f28dc068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e24ec700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f28dc066940 0x7f28dc0be860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39666/0 (socket says 172.21.15.29:39666) 2024-01-24T14:15:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e24ec700 1 -- 172.21.15.29:0/54693178 learned_addr learned my addr 172.21.15.29:0/54693178 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e14ea700 1 --2- 172.21.15.29:0/54693178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28dc0638c0 0x7f28dc066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e1ceb700 1 -- 172.21.15.29:0/54693178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28dc0638c0 msgr2=0x7f28dc066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:58.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e1ceb700 1 --2- 172.21.15.29:0/54693178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28dc0638c0 0x7f28dc066450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e1ceb700 1 -- 172.21.15.29:0/54693178 >> v1:172.21.15.29:6789/0 conn(0x7f28dc066940 legacy=0x7f28dc0be860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e1ceb700 1 -- 172.21.15.29:0/54693178 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc066dd0 con 0x7f28dc13a6f0 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e0ce9700 1 -- 172.21.15.29:0/54693178 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164381018 0 0) 0x7f28dc0643f0 con 0x7f28dc066940 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.017+0000 7f28e1ceb700 1 --2- 172.21.15.29:0/54693178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28dc13a6f0 0x7f28dc068660 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f28d0009760 tx=0x7f28d0009060).ready entity=mon.0 client_cookie=32d45ad0dbdbeada server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.018+0000 7f28e0ce9700 1 -- 172.21.15.29:0/54693178 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d000b580 con 0x7f28dc13a6f0 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.018+0000 7f28e0ce9700 1 -- 172.21.15.29:0/54693178 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d000b6e0 con 0x7f28dc13a6f0 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.018+0000 7f28e0ce9700 1 -- 172.21.15.29:0/54693178 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d000ba60 con 0x7f28dc13a6f0 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.018+0000 7f28e3f4f700 1 -- 172.21.15.29:0/54693178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28dc13a6f0 msgr2=0x7f28dc068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.018+0000 7f28e3f4f700 1 --2- 172.21.15.29:0/54693178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28dc13a6f0 0x7f28dc068660 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f28d0009760 tx=0x7f28d0009060).stop 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.018+0000 7f28e3f4f700 1 -- 172.21.15.29:0/54693178 shutdown_connections 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.018+0000 7f28e3f4f700 1 --2- 172.21.15.29:0/54693178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28dc13a6f0 0x7f28dc068660 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.018+0000 7f28e3f4f700 1 --2- 172.21.15.29:0/54693178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28dc0638c0 0x7f28dc066450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:58.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.018+0000 7f28e3f4f700 1 -- 172.21.15.29:0/54693178 >> 172.21.15.29:0/54693178 conn(0x7f28dc0c8bd0 msgr2=0x7f28dc0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:58.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.018+0000 7f28e3f4f700 1 -- 172.21.15.29:0/54693178 shutdown_connections 2024-01-24T14:15:58.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.018+0000 7f28e3f4f700 1 -- 172.21.15.29:0/54693178 wait complete. 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.019+0000 7f28e3f4f700 1 Processor -- start 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.020+0000 7f28e3f4f700 1 -- start start 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.020+0000 7f28e3f4f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28dc0638c0 0x7f28dc0adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.020+0000 7f28e3f4f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28dc066940 0x7f28dc0abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.020+0000 7f28e3f4f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28dc13a6f0 0x7f28dc0ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.020+0000 7f28e3f4f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0aebc0 con 0x7f28dc0638c0 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.020+0000 7f28e3f4f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0ac0c0 con 0x7f28dc066940 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.020+0000 7f28e3f4f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0ae600 con 0x7f28dc13a6f0 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.020+0000 7f28e1ceb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28dc0638c0 0x7f28dc0adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.020+0000 7f28e1ceb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28dc0638c0 0x7f28dc0adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38984/0 (socket says 172.21.15.29:38984) 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.020+0000 7f28e1ceb700 1 -- 172.21.15.29:0/1302521191 learned_addr learned my addr 172.21.15.29:0/1302521191 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28e24ec700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28dc13a6f0 0x7f28dc0ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:58.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28e14ea700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28dc066940 0x7f28dc0abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:58.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28e1ceb700 1 -- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28dc13a6f0 msgr2=0x7f28dc0ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:58.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28e1ceb700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28dc13a6f0 0x7f28dc0ac4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:58.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28e1ceb700 1 -- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28dc066940 msgr2=0x7f28dc0abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:58.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28e1ceb700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28dc066940 0x7f28dc0abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:58.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28e1ceb700 1 -- 172.21.15.29:0/1302521191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d00092a0 con 0x7f28dc0638c0 2024-01-24T14:15:58.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28e1ceb700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28dc0638c0 0x7f28dc0adf90 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f28d0009eb0 tx=0x7f28d0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:58.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28cdffb700 1 -- 172.21.15.29:0/1302521191 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d000b850 con 0x7f28dc0638c0 2024-01-24T14:15:58.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28cdffb700 1 -- 172.21.15.29:0/1302521191 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d000b9b0 con 0x7f28dc0638c0 2024-01-24T14:15:58.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28e3f4f700 1 -- 172.21.15.29:0/1302521191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28dc0ae8e0 con 0x7f28dc0638c0 2024-01-24T14:15:58.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28cdffb700 1 -- 172.21.15.29:0/1302521191 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d001b690 con 0x7f28dc0638c0 2024-01-24T14:15:58.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.021+0000 7f28e3f4f700 1 -- 172.21.15.29:0/1302521191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc0a8a40 con 0x7f28dc0638c0 2024-01-24T14:15:58.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.022+0000 7f28e3f4f700 1 -- 172.21.15.29:0/1302521191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28dc06ae00 con 0x7f28dc0638c0 2024-01-24T14:15:58.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.023+0000 7f28cdffb700 1 -- 172.21.15.29:0/1302521191 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 47) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f28d001ba40 con 0x7f28dc0638c0 2024-01-24T14:15:58.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.023+0000 7f28cdffb700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28c005f030 0x7f28c00614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:58.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.023+0000 7f28cdffb700 1 -- 172.21.15.29:0/1302521191 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 7860+0+0 (secure 0 0 0) 0x7f28d007bfa0 con 0x7f28dc0638c0 2024-01-24T14:15:58.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.023+0000 7f28e14ea700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28c005f030 0x7f28c00614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:58.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.024+0000 7f28e14ea700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28c005f030 0x7f28c00614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f28dc163f00 tx=0x7f28d8008040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:58.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.028+0000 7f28cdffb700 1 -- 172.21.15.29:0/1302521191 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f28d004cd70 con 0x7f28dc0638c0 2024-01-24T14:15:58.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.209+0000 7f28e3f4f700 1 -- 172.21.15.29:0/1302521191 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28dc0475e0 con 0x7f28dc0638c0 2024-01-24T14:15:58.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.616+0000 7f28cdffb700 1 -- 172.21.15.29:0/1302521191 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f28d0079870 con 0x7f28dc0638c0 2024-01-24T14:15:58.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.863+0000 7f28cdffb700 1 -- 172.21.15.29:0/1302521191 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) v1 ==== 181+0+0 (secure 0 0 0) 0x7f28d00425d0 con 0x7f28dc0638c0 2024-01-24T14:15:58.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.868+0000 7f28e3f4f700 1 -- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28c005f030 msgr2=0x7f28c00614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:58.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.868+0000 7f28e3f4f700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28c005f030 0x7f28c00614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f28dc163f00 tx=0x7f28d8008040).stop 2024-01-24T14:15:58.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.868+0000 7f28e3f4f700 1 -- 172.21.15.29:0/1302521191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28dc0638c0 msgr2=0x7f28dc0adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:58.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.868+0000 7f28e3f4f700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28dc0638c0 0x7f28dc0adf90 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f28d0009eb0 tx=0x7f28d0017000).stop 2024-01-24T14:15:58.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.869+0000 7f28e3f4f700 1 -- 172.21.15.29:0/1302521191 shutdown_connections 2024-01-24T14:15:58.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.869+0000 7f28e3f4f700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28dc13a6f0 0x7f28dc0ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:58.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.869+0000 7f28e3f4f700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f28c005f030 0x7f28c00614f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:58.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.869+0000 7f28e3f4f700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28dc0638c0 0x7f28dc0adf90 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:58.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.869+0000 7f28e3f4f700 1 --2- 172.21.15.29:0/1302521191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28dc066940 0x7f28dc0abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:58.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.869+0000 7f28e3f4f700 1 -- 172.21.15.29:0/1302521191 >> 172.21.15.29:0/1302521191 conn(0x7f28dc0c8bd0 msgr2=0x7f28dc0c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:58.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.870+0000 7f28e3f4f700 1 -- 172.21.15.29:0/1302521191 shutdown_connections 2024-01-24T14:15:58.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:58.870+0000 7f28e3f4f700 1 -- 172.21.15.29:0/1302521191 wait complete. 2024-01-24T14:15:58.872 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-01-24T14:15:58.889 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-24T14:15:59.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.029+0000 7f63dd3d9700 1 Processor -- start 2024-01-24T14:15:59.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.029+0000 7f63dd3d9700 1 -- start start 2024-01-24T14:15:59.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.030+0000 7f63dd3d9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f63d814eb10 0x7f63d80c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:59.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.030+0000 7f63dd3d9700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d8154dc0 con 0x7f63d814e730 2024-01-24T14:15:59.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.030+0000 7f63dd3d9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d80c5470 con 0x7f63d80c7f70 2024-01-24T14:15:59.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.030+0000 7f63dd3d9700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d80c6300 con 0x7f63d814eb10 2024-01-24T14:15:59.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.030+0000 7f63d67fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f63d814eb10 0x7f63d80c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:59.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.030+0000 7f63d6ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f63d814e730 0x7f63d8155250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39682/0 (socket says 172.21.15.29:39682) 2024-01-24T14:15:59.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.030+0000 7f63d6ffd700 1 -- 172.21.15.29:0/444803600 learned_addr learned my addr 172.21.15.29:0/444803600 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:15:59.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.030+0000 7f63d67fc700 1 -- 172.21.15.29:0/444803600 >> v1:172.21.15.163:6789/0 conn(0x7f63d80c7f70 legacy=0x7f63d80a9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:15:59.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.030+0000 7f63d67fc700 1 -- 172.21.15.29:0/444803600 >> v1:172.21.15.29:6789/0 conn(0x7f63d814e730 legacy=0x7f63d8155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.030+0000 7f63d5ffb700 1 -- 172.21.15.29:0/444803600 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869662166 0 0) 0x7f63d8154dc0 con 0x7f63d814e730 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.030+0000 7f63d67fc700 1 -- 172.21.15.29:0/444803600 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80c6ad0 con 0x7f63d814eb10 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.030+0000 7f63d67fc700 1 --2- 172.21.15.29:0/444803600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63d814eb10 0x7f63d80c5e10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f63cc009760 tx=0x7f63cc009060).ready entity=mon.0 client_cookie=818f3679bc49c60 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.031+0000 7f63d5ffb700 1 -- 172.21.15.29:0/444803600 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc00b520 con 0x7f63d814eb10 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.031+0000 7f63d5ffb700 1 -- 172.21.15.29:0/444803600 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63cc00b680 con 0x7f63d814eb10 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.031+0000 7f63d5ffb700 1 -- 172.21.15.29:0/444803600 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc00ba00 con 0x7f63d814eb10 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.031+0000 7f63dd3d9700 1 -- 172.21.15.29:0/444803600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63d814eb10 msgr2=0x7f63d80c5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.031+0000 7f63dd3d9700 1 --2- 172.21.15.29:0/444803600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63d814eb10 0x7f63d80c5e10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f63cc009760 tx=0x7f63cc009060).stop 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.031+0000 7f63dd3d9700 1 -- 172.21.15.29:0/444803600 shutdown_connections 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.031+0000 7f63dd3d9700 1 --2- 172.21.15.29:0/444803600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63d814eb10 0x7f63d80c5e10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.031+0000 7f63dd3d9700 1 -- 172.21.15.29:0/444803600 >> 172.21.15.29:0/444803600 conn(0x7f63d80cb880 msgr2=0x7f63d80c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.031+0000 7f63dd3d9700 1 -- 172.21.15.29:0/444803600 shutdown_connections 2024-01-24T14:15:59.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.031+0000 7f63dd3d9700 1 -- 172.21.15.29:0/444803600 wait complete. 2024-01-24T14:15:59.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.033+0000 7f63dd3d9700 1 Processor -- start 2024-01-24T14:15:59.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.033+0000 7f63dd3d9700 1 -- start start 2024-01-24T14:15:59.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.033+0000 7f63dd3d9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63d80c7f70 0x7f63d814c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:59.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.033+0000 7f63dd3d9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d814e730 0x7f63d80c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:59.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.033+0000 7f63dd3d9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d814eb10 0x7f63d80ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:59.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.033+0000 7f63dd3d9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80ca5c0 con 0x7f63d80c7f70 2024-01-24T14:15:59.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.033+0000 7f63dd3d9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8165250 con 0x7f63d814e730 2024-01-24T14:15:59.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.033+0000 7f63dd3d9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8164920 con 0x7f63d814eb10 2024-01-24T14:15:59.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63d6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63d80c7f70 0x7f63d814c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:59.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63d6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63d80c7f70 0x7f63d814c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39006/0 (socket says 172.21.15.29:39006) 2024-01-24T14:15:59.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63d6ffd700 1 -- 172.21.15.29:0/2008879612 learned_addr learned my addr 172.21.15.29:0/2008879612 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:15:59.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63d67fc700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d814e730 0x7f63d80c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:59.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63d77fe700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d814eb10 0x7f63d80ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:59.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63d6ffd700 1 -- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d814eb10 msgr2=0x7f63d80ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:59.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63d6ffd700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d814eb10 0x7f63d80ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:59.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63d6ffd700 1 -- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d814e730 msgr2=0x7f63d80c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:59.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63d6ffd700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d814e730 0x7f63d80c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:59.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63d6ffd700 1 -- 172.21.15.29:0/2008879612 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc0092a0 con 0x7f63d80c7f70 2024-01-24T14:15:59.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63d6ffd700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63d80c7f70 0x7f63d814c020 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f63c800c4e0 tx=0x7f63c8007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:59.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63beffd700 1 -- 172.21.15.29:0/2008879612 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c800ce90 con 0x7f63d80c7f70 2024-01-24T14:15:59.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63beffd700 1 -- 172.21.15.29:0/2008879612 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c80072e0 con 0x7f63d80c7f70 2024-01-24T14:15:59.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63beffd700 1 -- 172.21.15.29:0/2008879612 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8007660 con 0x7f63d80c7f70 2024-01-24T14:15:59.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63dd3d9700 1 -- 172.21.15.29:0/2008879612 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d8164c00 con 0x7f63d80c7f70 2024-01-24T14:15:59.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.034+0000 7f63dd3d9700 1 -- 172.21.15.29:0/2008879612 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d81645d0 con 0x7f63d80c7f70 2024-01-24T14:15:59.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.036+0000 7f63dd3d9700 1 -- 172.21.15.29:0/2008879612 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d8083310 con 0x7f63d80c7f70 2024-01-24T14:15:59.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.036+0000 7f63beffd700 1 -- 172.21.15.29:0/2008879612 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f63c80059b0 con 0x7f63d80c7f70 2024-01-24T14:15:59.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.036+0000 7f63beffd700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f63c405f270 0x7f63c4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:15:59.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.036+0000 7f63beffd700 1 -- 172.21.15.29:0/2008879612 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 7306+0+0 (secure 0 0 0) 0x7f63c8079ff0 con 0x7f63d80c7f70 2024-01-24T14:15:59.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.037+0000 7f63d67fc700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f63c405f270 0x7f63c4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:15:59.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.037+0000 7f63d67fc700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f63c405f270 0x7f63c4061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f63d814ef40 tx=0x7f63cc017000).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:15:59.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.041+0000 7f63beffd700 1 -- 172.21.15.29:0/2008879612 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f63c804abe0 con 0x7f63d80c7f70 2024-01-24T14:15:59.106 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:15:59.109 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:15:59.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.224+0000 7f63dd3d9700 1 -- 172.21.15.29:0/2008879612 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f63d8165530 con 0x7f63d80c7f70 2024-01-24T14:15:59.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.864+0000 7f63beffd700 1 -- 172.21.15.29:0/2008879612 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v128) v1 ==== 169+0+0 (secure 0 0 0) 0x7f63c804e200 con 0x7f63d80c7f70 2024-01-24T14:15:59.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.868+0000 7f63dd3d9700 1 -- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f63c405f270 msgr2=0x7f63c4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.868+0000 7f63dd3d9700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f63c405f270 0x7f63c4061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f63d814ef40 tx=0x7f63cc017000).stop 2024-01-24T14:15:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.868+0000 7f63dd3d9700 1 -- 172.21.15.29:0/2008879612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63d80c7f70 msgr2=0x7f63d814c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:15:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.869+0000 7f63dd3d9700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63d80c7f70 0x7f63d814c020 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f63c800c4e0 tx=0x7f63c8007c70).stop 2024-01-24T14:15:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.869+0000 7f63dd3d9700 1 -- 172.21.15.29:0/2008879612 shutdown_connections 2024-01-24T14:15:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.869+0000 7f63dd3d9700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d814eb10 0x7f63d80ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.869+0000 7f63dd3d9700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f63c405f270 0x7f63c4061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.869+0000 7f63dd3d9700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63d80c7f70 0x7f63d814c020 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.869+0000 7f63dd3d9700 1 --2- 172.21.15.29:0/2008879612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d814e730 0x7f63d80c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:15:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.869+0000 7f63dd3d9700 1 -- 172.21.15.29:0/2008879612 >> 172.21.15.29:0/2008879612 conn(0x7f63d80cb880 msgr2=0x7f63d80ad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:15:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.869+0000 7f63dd3d9700 1 -- 172.21.15.29:0/2008879612 shutdown_connections 2024-01-24T14:15:59.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:15:59.869+0000 7f63dd3d9700 1 -- 172.21.15.29:0/2008879612 wait complete. 2024-01-24T14:15:59.871 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-01-24T14:15:59.886 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-24T14:16:00.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.023+0000 7f3b2e6fa700 1 Processor -- start 2024-01-24T14:16:00.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.023+0000 7f3b2e6fa700 1 -- start start 2024-01-24T14:16:00.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b2e6fa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b28157530 0x7f3b28068470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b2e6fa700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3b28068960 0x7f3b28066520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b2e6fa700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b28066ec0 con 0x7f3b28066a10 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b2e6fa700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b28065040 con 0x7f3b28068960 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b2e6fa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b28064140 con 0x7f3b28157530 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b277fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3b28068960 0x7f3b28066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b2cc97700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3b28066a10 0x7f3b28157c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39684/0 (socket says 172.21.15.29:39684) 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b2cc97700 1 -- 172.21.15.29:0/4288270806 learned_addr learned my addr 172.21.15.29:0/4288270806 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b27fff700 1 --2- 172.21.15.29:0/4288270806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b28157530 0x7f3b28068470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b277fe700 1 -- 172.21.15.29:0/4288270806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b28157530 msgr2=0x7f3b28068470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b277fe700 1 --2- 172.21.15.29:0/4288270806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b28157530 0x7f3b28068470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b277fe700 1 -- 172.21.15.29:0/4288270806 >> v1:172.21.15.29:6789/0 conn(0x7f3b28066a10 legacy=0x7f3b28157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b26ffd700 1 -- 172.21.15.29:0/4288270806 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527614986 0 0) 0x7f3b20001490 con 0x7f3b28066a10 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b277fe700 1 -- 172.21.15.29:0/4288270806 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b280be080 con 0x7f3b28068960 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.024+0000 7f3b277fe700 1 --2- 172.21.15.29:0/4288270806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b28068960 0x7f3b28066520 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f3b1c009760 tx=0x7f3b1c009060).ready entity=mon.0 client_cookie=f9623cc55b827388 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.025+0000 7f3b26ffd700 1 -- 172.21.15.29:0/4288270806 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c00b580 con 0x7f3b28068960 2024-01-24T14:16:00.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.025+0000 7f3b26ffd700 1 -- 172.21.15.29:0/4288270806 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c00b6e0 con 0x7f3b28068960 2024-01-24T14:16:00.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.025+0000 7f3b26ffd700 1 -- 172.21.15.29:0/4288270806 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c00ba60 con 0x7f3b28068960 2024-01-24T14:16:00.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.025+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/4288270806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b28068960 msgr2=0x7f3b28066520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.025+0000 7f3b2e6fa700 1 --2- 172.21.15.29:0/4288270806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b28068960 0x7f3b28066520 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f3b1c009760 tx=0x7f3b1c009060).stop 2024-01-24T14:16:00.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.025+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/4288270806 shutdown_connections 2024-01-24T14:16:00.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.025+0000 7f3b2e6fa700 1 --2- 172.21.15.29:0/4288270806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b28157530 0x7f3b28068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.025+0000 7f3b2e6fa700 1 --2- 172.21.15.29:0/4288270806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b28068960 0x7f3b28066520 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.025+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/4288270806 >> 172.21.15.29:0/4288270806 conn(0x7f3b280c8ba0 msgr2=0x7f3b280c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:00.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.025+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/4288270806 shutdown_connections 2024-01-24T14:16:00.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.025+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/4288270806 wait complete. 2024-01-24T14:16:00.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.026+0000 7f3b2e6fa700 1 Processor -- start 2024-01-24T14:16:00.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.027+0000 7f3b2e6fa700 1 -- start start 2024-01-24T14:16:00.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.027+0000 7f3b2e6fa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b28066a10 0x7f3b280c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.027+0000 7f3b2e6fa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b28068960 0x7f3b280a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.027+0000 7f3b2e6fa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b28157530 0x7f3b280a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.027+0000 7f3b2e6fa700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280a5260 con 0x7f3b28157530 2024-01-24T14:16:00.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.027+0000 7f3b2e6fa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b281548f0 con 0x7f3b28068960 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.027+0000 7f3b2e6fa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b28153ce0 con 0x7f3b28066a10 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b2cc97700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b28157530 0x7f3b280a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b2cc97700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b28157530 0x7f3b280a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39038/0 (socket says 172.21.15.29:39038) 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b2cc97700 1 -- 172.21.15.29:0/1584730342 learned_addr learned my addr 172.21.15.29:0/1584730342 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b27fff700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b28066a10 0x7f3b280c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b277fe700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b28068960 0x7f3b280a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b2cc97700 1 -- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b28066a10 msgr2=0x7f3b280c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b2cc97700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b28066a10 0x7f3b280c3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b2cc97700 1 -- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b28068960 msgr2=0x7f3b280a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b2cc97700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b28068960 0x7f3b280a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b2cc97700 1 -- 172.21.15.29:0/1584730342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c0092a0 con 0x7f3b28157530 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b2cc97700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b28157530 0x7f3b280a4d70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f3b2000c7e0 tx=0x7f3b20007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b0ffff700 1 -- 172.21.15.29:0/1584730342 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20005d30 con 0x7f3b28157530 2024-01-24T14:16:00.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b0ffff700 1 -- 172.21.15.29:0/1584730342 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b20005e90 con 0x7f3b28157530 2024-01-24T14:16:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/1584730342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b28153fc0 con 0x7f3b28157530 2024-01-24T14:16:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b0ffff700 1 -- 172.21.15.29:0/1584730342 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20007320 con 0x7f3b28157530 2024-01-24T14:16:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.028+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/1584730342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b28154600 con 0x7f3b28157530 2024-01-24T14:16:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.029+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/1584730342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b28154a80 con 0x7f3b28157530 2024-01-24T14:16:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.030+0000 7f3b0ffff700 1 -- 172.21.15.29:0/1584730342 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3b200076d0 con 0x7f3b28157530 2024-01-24T14:16:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.030+0000 7f3b0ffff700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3b1405f2a0 0x7f3b14061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.030+0000 7f3b0ffff700 1 -- 172.21.15.29:0/1584730342 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6892+0+0 (secure 0 0 0) 0x7f3b2007a5b0 con 0x7f3b28157530 2024-01-24T14:16:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.030+0000 7f3b27fff700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3b1405f2a0 0x7f3b14061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.031+0000 7f3b27fff700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3b1405f2a0 0x7f3b14061760 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f3b1800a940 tx=0x7f3b18008040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:00.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.034+0000 7f3b0ffff700 1 -- 172.21.15.29:0/1584730342 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3b2004b400 con 0x7f3b28157530 2024-01-24T14:16:00.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.219+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/1584730342 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3b28066ec0 con 0x7f3b28157530 2024-01-24T14:16:00.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.223+0000 7f3b0ffff700 1 -- 172.21.15.29:0/1584730342 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3b28066ec0 con 0x7f3b28157530 2024-01-24T14:16:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.225+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3b1405f2a0 msgr2=0x7f3b14061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.225+0000 7f3b2e6fa700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3b1405f2a0 0x7f3b14061760 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f3b1800a940 tx=0x7f3b18008040).stop 2024-01-24T14:16:00.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.225+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/1584730342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b28157530 msgr2=0x7f3b280a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.225+0000 7f3b2e6fa700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b28157530 0x7f3b280a4d70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f3b2000c7e0 tx=0x7f3b20007dd0).stop 2024-01-24T14:16:00.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.225+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/1584730342 shutdown_connections 2024-01-24T14:16:00.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.225+0000 7f3b2e6fa700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3b1405f2a0 0x7f3b14061760 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.225+0000 7f3b2e6fa700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b28157530 0x7f3b280a4d70 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.225+0000 7f3b2e6fa700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b28066a10 0x7f3b280c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.225+0000 7f3b2e6fa700 1 --2- 172.21.15.29:0/1584730342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b28068960 0x7f3b280a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.225+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/1584730342 >> 172.21.15.29:0/1584730342 conn(0x7f3b280c8ba0 msgr2=0x7f3b28150fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:00.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.226+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/1584730342 shutdown_connections 2024-01-24T14:16:00.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.226+0000 7f3b2e6fa700 1 -- 172.21.15.29:0/1584730342 wait complete. 2024-01-24T14:16:00.240 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-01-24T14:16:00.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.373+0000 7f26c2c6d700 1 Processor -- start 2024-01-24T14:16:00.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.373+0000 7f26c2c6d700 1 -- start start 2024-01-24T14:16:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.374+0000 7f26c2c6d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26bc0663b0 0x7f26bc152f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.374+0000 7f26c2c6d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26bc0be4b0 0x7f26bc069b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.374+0000 7f26c2c6d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26bc069690 con 0x7f26bc06a010 2024-01-24T14:16:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.374+0000 7f26c2c6d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26bc0be930 con 0x7f26bc0be4b0 2024-01-24T14:16:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.374+0000 7f26c2c6d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc0bca30 con 0x7f26bc0663b0 2024-01-24T14:16:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.374+0000 7f26c0a09700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26bc0663b0 0x7f26bc152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.374+0000 7f26bbfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26bc0be4b0 0x7f26bc069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.374+0000 7f26c120a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f26bc06a010 0x7f26bc065420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:35982/0 (socket says 172.21.15.29:35982) 2024-01-24T14:16:00.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.374+0000 7f26c120a700 1 -- 172.21.15.29:0/3081303312 learned_addr learned my addr 172.21.15.29:0/3081303312 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.375+0000 7f26bb7fe700 1 -- 172.21.15.29:0/3081303312 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605091121 0 0) 0x7f26bc069690 con 0x7f26bc06a010 2024-01-24T14:16:00.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.375+0000 7f26bb7fe700 1 -- 172.21.15.29:0/3081303312 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26ac0033f0 con 0x7f26bc06a010 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.375+0000 7f26bbfff700 1 -- 172.21.15.29:0/3081303312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26bc0663b0 msgr2=0x7f26bc152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.375+0000 7f26bbfff700 1 --2- 172.21.15.29:0/3081303312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26bc0663b0 0x7f26bc152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.375+0000 7f26bbfff700 1 -- 172.21.15.29:0/3081303312 >> v1:172.21.15.163:6789/0 conn(0x7f26bc06a010 legacy=0x7f26bc065420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.375+0000 7f26bbfff700 1 -- 172.21.15.29:0/3081303312 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc0bcc40 con 0x7f26bc0be4b0 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.375+0000 7f26c0a09700 1 --2- 172.21.15.29:0/3081303312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26bc0663b0 0x7f26bc152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.375+0000 7f26bbfff700 1 --2- 172.21.15.29:0/3081303312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26bc0be4b0 0x7f26bc069b20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f26a8009760 tx=0x7f26a8009060).ready entity=mon.1 client_cookie=a431842fccf739d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.375+0000 7f26bb7fe700 1 -- 172.21.15.29:0/3081303312 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a80090d0 con 0x7f26bc0be4b0 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.375+0000 7f26bb7fe700 1 -- 172.21.15.29:0/3081303312 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a800d4a0 con 0x7f26bc0be4b0 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.375+0000 7f26bb7fe700 1 -- 172.21.15.29:0/3081303312 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a800d820 con 0x7f26bc0be4b0 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.376+0000 7f26c2c6d700 1 -- 172.21.15.29:0/3081303312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26bc0be4b0 msgr2=0x7f26bc069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.376+0000 7f26c2c6d700 1 --2- 172.21.15.29:0/3081303312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26bc0be4b0 0x7f26bc069b20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f26a8009760 tx=0x7f26a8009060).stop 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.376+0000 7f26c2c6d700 1 -- 172.21.15.29:0/3081303312 shutdown_connections 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.376+0000 7f26c2c6d700 1 --2- 172.21.15.29:0/3081303312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26bc0663b0 0x7f26bc152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.376+0000 7f26c2c6d700 1 --2- 172.21.15.29:0/3081303312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26bc0be4b0 0x7f26bc069b20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.376+0000 7f26c2c6d700 1 -- 172.21.15.29:0/3081303312 >> 172.21.15.29:0/3081303312 conn(0x7f26bc064390 msgr2=0x7f26bc0c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.376+0000 7f26c2c6d700 1 -- 172.21.15.29:0/3081303312 shutdown_connections 2024-01-24T14:16:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.376+0000 7f26c2c6d700 1 -- 172.21.15.29:0/3081303312 wait complete. 2024-01-24T14:16:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.377+0000 7f26c2c6d700 1 Processor -- start 2024-01-24T14:16:00.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.377+0000 7f26c2c6d700 1 -- start start 2024-01-24T14:16:00.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26c2c6d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26bc0663b0 0x7f26bc135780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26c2c6d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26bc06a010 0x7f26bc0a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26c2c6d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26bc0be4b0 0x7f26bc0a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26c2c6d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc0a5260 con 0x7f26bc06a010 2024-01-24T14:16:00.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26c2c6d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc0a5540 con 0x7f26bc0663b0 2024-01-24T14:16:00.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26c2c6d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc0a5820 con 0x7f26bc0be4b0 2024-01-24T14:16:00.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26bbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26bc06a010 0x7f26bc0a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26bbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26bc06a010 0x7f26bc0a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39040/0 (socket says 172.21.15.29:39040) 2024-01-24T14:16:00.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26bbfff700 1 -- 172.21.15.29:0/667332833 learned_addr learned my addr 172.21.15.29:0/667332833 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:00.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26c0a09700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26bc0663b0 0x7f26bc135780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26c120a700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26bc0be4b0 0x7f26bc0a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26bbfff700 1 -- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26bc0be4b0 msgr2=0x7f26bc0a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26bbfff700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26bc0be4b0 0x7f26bc0a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26bbfff700 1 -- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26bc0663b0 msgr2=0x7f26bc135780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.378+0000 7f26bbfff700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26bc0663b0 0x7f26bc135780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.379+0000 7f26bbfff700 1 -- 172.21.15.29:0/667332833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b00092a0 con 0x7f26bc06a010 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.379+0000 7f26c120a700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26bc0be4b0 0x7f26bc0a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.379+0000 7f26bbfff700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26bc06a010 0x7f26bc0a4880 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f26a8010000 tx=0x7f26a8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.379+0000 7f26b8ff9700 1 -- 172.21.15.29:0/667332833 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a80127f0 con 0x7f26bc06a010 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.379+0000 7f26b8ff9700 1 -- 172.21.15.29:0/667332833 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a8012950 con 0x7f26bc06a010 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.379+0000 7f26c2c6d700 1 -- 172.21.15.29:0/667332833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a80092a0 con 0x7f26bc06a010 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.379+0000 7f26b8ff9700 1 -- 172.21.15.29:0/667332833 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8012cd0 con 0x7f26bc06a010 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.379+0000 7f26c2c6d700 1 -- 172.21.15.29:0/667332833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26bc0c2170 con 0x7f26bc06a010 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.380+0000 7f26c2c6d700 1 -- 172.21.15.29:0/667332833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26bc0c2390 con 0x7f26bc06a010 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.381+0000 7f26b8ff9700 1 -- 172.21.15.29:0/667332833 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f26a801c630 con 0x7f26bc06a010 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.381+0000 7f26b8ff9700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f26a005f240 0x7f26a0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.381+0000 7f26b8ff9700 1 -- 172.21.15.29:0/667332833 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6892+0+0 (secure 0 0 0) 0x7f26a807baa0 con 0x7f26bc06a010 2024-01-24T14:16:00.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.381+0000 7f26c0a09700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f26a005f240 0x7f26a0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.382+0000 7f26c0a09700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f26a005f240 0x7f26a0061700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f26b0009dc0 tx=0x7f26b0009060).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:00.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.386+0000 7f26b8ff9700 1 -- 172.21.15.29:0/667332833 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f26a804c8f0 con 0x7f26bc06a010 2024-01-24T14:16:00.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.574+0000 7f26c2c6d700 1 -- 172.21.15.29:0/667332833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f26bc0475e0 con 0x7f26bc06a010 2024-01-24T14:16:00.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.578+0000 7f26b8ff9700 1 -- 172.21.15.29:0/667332833 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v21) v1 ==== 120+0+0 (secure 0 0 0) 0x7f26a804ff10 con 0x7f26bc06a010 2024-01-24T14:16:00.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.580+0000 7f26c2c6d700 1 -- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f26a005f240 msgr2=0x7f26a0061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.580+0000 7f26c2c6d700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f26a005f240 0x7f26a0061700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f26b0009dc0 tx=0x7f26b0009060).stop 2024-01-24T14:16:00.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.580+0000 7f26c2c6d700 1 -- 172.21.15.29:0/667332833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26bc06a010 msgr2=0x7f26bc0a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.580+0000 7f26c2c6d700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26bc06a010 0x7f26bc0a4880 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f26a8010000 tx=0x7f26a8016000).stop 2024-01-24T14:16:00.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.580+0000 7f26c2c6d700 1 -- 172.21.15.29:0/667332833 shutdown_connections 2024-01-24T14:16:00.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.580+0000 7f26c2c6d700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26bc0be4b0 0x7f26bc0a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.580+0000 7f26c2c6d700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26bc0663b0 0x7f26bc135780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.580+0000 7f26c2c6d700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f26a005f240 0x7f26a0061700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.580+0000 7f26c2c6d700 1 --2- 172.21.15.29:0/667332833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26bc06a010 0x7f26bc0a4880 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.580+0000 7f26c2c6d700 1 -- 172.21.15.29:0/667332833 >> 172.21.15.29:0/667332833 conn(0x7f26bc064390 msgr2=0x7f26bc150330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:00.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.580+0000 7f26c2c6d700 1 -- 172.21.15.29:0/667332833 shutdown_connections 2024-01-24T14:16:00.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.580+0000 7f26c2c6d700 1 -- 172.21.15.29:0/667332833 wait complete. 2024-01-24T14:16:00.594 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2024-01-24T14:16:00.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.724+0000 7f0abff99700 1 Processor -- start 2024-01-24T14:16:00.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.725+0000 7f0abff99700 1 -- start start 2024-01-24T14:16:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.725+0000 7f0abff99700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0ab8134cc0 0x7f0ab80c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.725+0000 7f0abff99700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0ab81350a0 0x7f0ab8152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.725+0000 7f0abff99700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab8068360 con 0x7f0ab80664f0 2024-01-24T14:16:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.725+0000 7f0abff99700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab8153360 con 0x7f0ab81350a0 2024-01-24T14:16:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.725+0000 7f0abff99700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab80c3820 con 0x7f0ab8134cc0 2024-01-24T14:16:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abd534700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0ab81350a0 0x7f0ab8152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abe536700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0ab80664f0 0x7f0ab8155a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39700/0 (socket says 172.21.15.29:39700) 2024-01-24T14:16:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abe536700 1 -- 172.21.15.29:0/1636461349 learned_addr learned my addr 172.21.15.29:0/1636461349 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abdd35700 1 --2- 172.21.15.29:0/1636461349 >> v2:172.21.15.163:3300/0 conn(0x7f0ab8134cc0 0x7f0ab80c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abcd33700 1 -- 172.21.15.29:0/1636461349 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3278301727 0 0) 0x7f0ab8068360 con 0x7f0ab80664f0 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abcd33700 1 -- 172.21.15.29:0/1636461349 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa40033f0 con 0x7f0ab80664f0 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abd534700 1 -- 172.21.15.29:0/1636461349 >> v2:172.21.15.163:3300/0 conn(0x7f0ab8134cc0 msgr2=0x7f0ab80c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abd534700 1 --2- 172.21.15.29:0/1636461349 >> v2:172.21.15.163:3300/0 conn(0x7f0ab8134cc0 0x7f0ab80c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abd534700 1 -- 172.21.15.29:0/1636461349 >> v1:172.21.15.29:6789/0 conn(0x7f0ab80664f0 legacy=0x7f0ab8155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abd534700 1 -- 172.21.15.29:0/1636461349 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab8067a30 con 0x7f0ab81350a0 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abd534700 1 --2- 172.21.15.29:0/1636461349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab81350a0 0x7f0ab8152e70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f0ab80aee60 tx=0x7f0aa8009060).ready entity=mon.0 client_cookie=b38d6eaa46860227 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abcd33700 1 -- 172.21.15.29:0/1636461349 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa800d4a0 con 0x7f0ab81350a0 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abcd33700 1 -- 172.21.15.29:0/1636461349 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa800d600 con 0x7f0ab81350a0 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.726+0000 7f0abcd33700 1 -- 172.21.15.29:0/1636461349 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa800d980 con 0x7f0ab81350a0 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.727+0000 7f0abff99700 1 -- 172.21.15.29:0/1636461349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab81350a0 msgr2=0x7f0ab8152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.727+0000 7f0abff99700 1 --2- 172.21.15.29:0/1636461349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab81350a0 0x7f0ab8152e70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f0ab80aee60 tx=0x7f0aa8009060).stop 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.727+0000 7f0abff99700 1 -- 172.21.15.29:0/1636461349 shutdown_connections 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.727+0000 7f0abff99700 1 --2- 172.21.15.29:0/1636461349 >> v2:172.21.15.163:3300/0 conn(0x7f0ab8134cc0 0x7f0ab80c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.727+0000 7f0abff99700 1 --2- 172.21.15.29:0/1636461349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab81350a0 0x7f0ab8152e70 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.727+0000 7f0abff99700 1 -- 172.21.15.29:0/1636461349 >> 172.21.15.29:0/1636461349 conn(0x7f0ab80c8bb0 msgr2=0x7f0ab80c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:00.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.727+0000 7f0abff99700 1 -- 172.21.15.29:0/1636461349 shutdown_connections 2024-01-24T14:16:00.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.727+0000 7f0abff99700 1 -- 172.21.15.29:0/1636461349 wait complete. 2024-01-24T14:16:00.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.728+0000 7f0abff99700 1 Processor -- start 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.728+0000 7f0abff99700 1 -- start start 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.729+0000 7f0abff99700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ab80664f0 0x7f0ab80aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.729+0000 7f0abff99700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8134cc0 0x7f0ab814a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.729+0000 7f0abff99700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ab81350a0 0x7f0ab8150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.729+0000 7f0abff99700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8168450 con 0x7f0ab8134cc0 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.729+0000 7f0abff99700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80aa7a0 con 0x7f0ab80664f0 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.729+0000 7f0abff99700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8167560 con 0x7f0ab81350a0 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.729+0000 7f0abd534700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8134cc0 0x7f0ab814a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.729+0000 7f0abd534700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8134cc0 0x7f0ab814a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39054/0 (socket says 172.21.15.29:39054) 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.729+0000 7f0abd534700 1 -- 172.21.15.29:0/627901047 learned_addr learned my addr 172.21.15.29:0/627901047 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0abe536700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ab81350a0 0x7f0ab8150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0abdd35700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ab80664f0 0x7f0ab80aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0abd534700 1 -- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ab81350a0 msgr2=0x7f0ab8150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0abd534700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ab81350a0 0x7f0ab8150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0abd534700 1 -- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ab80664f0 msgr2=0x7f0ab80aa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0abd534700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ab80664f0 0x7f0ab80aa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0abd534700 1 -- 172.21.15.29:0/627901047 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa80092a0 con 0x7f0ab8134cc0 2024-01-24T14:16:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0abd534700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8134cc0 0x7f0ab814a580 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f0aa8009fd0 tx=0x7f0aa8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0aadffb700 1 -- 172.21.15.29:0/627901047 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8000c90 con 0x7f0ab8134cc0 2024-01-24T14:16:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0aadffb700 1 -- 172.21.15.29:0/627901047 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa800d770 con 0x7f0ab8134cc0 2024-01-24T14:16:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0aadffb700 1 -- 172.21.15.29:0/627901047 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa801d430 con 0x7f0ab8134cc0 2024-01-24T14:16:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0abff99700 1 -- 172.21.15.29:0/627901047 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab8167840 con 0x7f0ab8134cc0 2024-01-24T14:16:00.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.730+0000 7f0abff99700 1 -- 172.21.15.29:0/627901047 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab8167e00 con 0x7f0ab8134cc0 2024-01-24T14:16:00.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.732+0000 7f0abff99700 1 -- 172.21.15.29:0/627901047 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab80475e0 con 0x7f0ab8134cc0 2024-01-24T14:16:00.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.732+0000 7f0aadffb700 1 -- 172.21.15.29:0/627901047 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0aa801d820 con 0x7f0ab8134cc0 2024-01-24T14:16:00.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.732+0000 7f0aadffb700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0a9c05f270 0x7f0a9c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:00.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.732+0000 7f0aadffb700 1 -- 172.21.15.29:0/627901047 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6892+0+0 (secure 0 0 0) 0x7f0aa807bf10 con 0x7f0ab8134cc0 2024-01-24T14:16:00.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.733+0000 7f0abdd35700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0a9c05f270 0x7f0a9c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:00.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.733+0000 7f0abdd35700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0a9c05f270 0x7f0a9c061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f0ab4009bc0 tx=0x7f0ab4008040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:00.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.737+0000 7f0aadffb700 1 -- 172.21.15.29:0/627901047 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0aa804cd60 con 0x7f0ab8134cc0 2024-01-24T14:16:00.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:00.989+0000 7f0abff99700 1 -- 172.21.15.29:0/627901047 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f0ab80be3a0 con 0x7f0ab8134cc0 2024-01-24T14:16:01.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.871+0000 7f0aadffb700 1 -- 172.21.15.29:0/627901047 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v663) v1 ==== 183+0+0 (secure 0 0 0) 0x7f0aa8050380 con 0x7f0ab8134cc0 2024-01-24T14:16:01.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.874+0000 7f0abff99700 1 -- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0a9c05f270 msgr2=0x7f0a9c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:01.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.874+0000 7f0abff99700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0a9c05f270 0x7f0a9c061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f0ab4009bc0 tx=0x7f0ab4008040).stop 2024-01-24T14:16:01.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.874+0000 7f0abff99700 1 -- 172.21.15.29:0/627901047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8134cc0 msgr2=0x7f0ab814a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.874+0000 7f0abff99700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8134cc0 0x7f0ab814a580 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f0aa8009fd0 tx=0x7f0aa8016000).stop 2024-01-24T14:16:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.874+0000 7f0abff99700 1 -- 172.21.15.29:0/627901047 shutdown_connections 2024-01-24T14:16:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.874+0000 7f0abff99700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ab81350a0 0x7f0ab8150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.874+0000 7f0abff99700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ab80664f0 0x7f0ab80aa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.874+0000 7f0abff99700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0a9c05f270 0x7f0a9c061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.874+0000 7f0abff99700 1 --2- 172.21.15.29:0/627901047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8134cc0 0x7f0ab814a580 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.874+0000 7f0abff99700 1 -- 172.21.15.29:0/627901047 >> 172.21.15.29:0/627901047 conn(0x7f0ab80c8bb0 msgr2=0x7f0ab814b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.874+0000 7f0abff99700 1 -- 172.21.15.29:0/627901047 shutdown_connections 2024-01-24T14:16:01.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:01.874+0000 7f0abff99700 1 -- 172.21.15.29:0/627901047 wait complete. 2024-01-24T14:16:01.890 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2024-01-24T14:16:01.890 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2024-01-24T14:16:01.890 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-01-24T14:16:01.891 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2024-01-24T14:16:02.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.024+0000 7f2b75917700 1 Processor -- start 2024-01-24T14:16:02.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.024+0000 7f2b75917700 1 -- start start 2024-01-24T14:16:02.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.025+0000 7f2b75917700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b70153c80 0x7f2b700638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:02.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.025+0000 7f2b75917700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2b70066c00 0x7f2b700bf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:02.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.025+0000 7f2b75917700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b70068400 con 0x7f2b70154060 2024-01-24T14:16:02.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.025+0000 7f2b75917700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b700bfad0 con 0x7f2b70066c00 2024-01-24T14:16:02.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.025+0000 7f2b75917700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b701544c0 con 0x7f2b70153c80 2024-01-24T14:16:02.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.025+0000 7f2b6f7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2b70066c00 0x7f2b700bf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:02.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.025+0000 7f2b6f7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2b70066c00 0x7f2b700bf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39068/0 (socket says 172.21.15.29:39068) 2024-01-24T14:16:02.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.025+0000 7f2b6f7fe700 1 -- 172.21.15.29:0/2099954797 learned_addr learned my addr 172.21.15.29:0/2099954797 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:02.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.025+0000 7f2b6effd700 1 --2- 172.21.15.29:0/2099954797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b70153c80 0x7f2b700638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:02.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.026+0000 7f2b6f7fe700 1 -- 172.21.15.29:0/2099954797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b70153c80 msgr2=0x7f2b700638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:02.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.026+0000 7f2b6f7fe700 1 --2- 172.21.15.29:0/2099954797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b70153c80 0x7f2b700638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:02.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.026+0000 7f2b6f7fe700 1 -- 172.21.15.29:0/2099954797 >> v1:172.21.15.163:6789/0 conn(0x7f2b70154060 legacy=0x7f2b700664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:02.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.026+0000 7f2b6f7fe700 1 -- 172.21.15.29:0/2099954797 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b70067ad0 con 0x7f2b70066c00 2024-01-24T14:16:02.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.026+0000 7f2b6dffb700 1 -- 172.21.15.29:0/2099954797 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041778742 0 0) 0x7f2b70068400 con 0x7f2b70154060 2024-01-24T14:16:02.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.026+0000 7f2b6f7fe700 1 --2- 172.21.15.29:0/2099954797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b70066c00 0x7f2b700bf5e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f2b68009f50 tx=0x7f2b68009840).ready entity=mon.0 client_cookie=818d2dc8b141332 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:02.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.026+0000 7f2b6dffb700 1 -- 172.21.15.29:0/2099954797 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6800e070 con 0x7f2b70066c00 2024-01-24T14:16:02.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.026+0000 7f2b6dffb700 1 -- 172.21.15.29:0/2099954797 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6800b9c0 con 0x7f2b70066c00 2024-01-24T14:16:02.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.026+0000 7f2b6dffb700 1 -- 172.21.15.29:0/2099954797 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6800bd40 con 0x7f2b70066c00 2024-01-24T14:16:02.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.027+0000 7f2b75917700 1 -- 172.21.15.29:0/2099954797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b70066c00 msgr2=0x7f2b700bf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:02.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.027+0000 7f2b75917700 1 --2- 172.21.15.29:0/2099954797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b70066c00 0x7f2b700bf5e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f2b68009f50 tx=0x7f2b68009840).stop 2024-01-24T14:16:02.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.027+0000 7f2b75917700 1 -- 172.21.15.29:0/2099954797 shutdown_connections 2024-01-24T14:16:02.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.027+0000 7f2b75917700 1 --2- 172.21.15.29:0/2099954797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b70153c80 0x7f2b700638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:02.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.027+0000 7f2b75917700 1 --2- 172.21.15.29:0/2099954797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b70066c00 0x7f2b700bf5e0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:02.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.027+0000 7f2b75917700 1 -- 172.21.15.29:0/2099954797 >> 172.21.15.29:0/2099954797 conn(0x7f2b700c8bb0 msgr2=0x7f2b700c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:02.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.027+0000 7f2b75917700 1 -- 172.21.15.29:0/2099954797 shutdown_connections 2024-01-24T14:16:02.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.027+0000 7f2b75917700 1 -- 172.21.15.29:0/2099954797 wait complete. 2024-01-24T14:16:02.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.028+0000 7f2b75917700 1 Processor -- start 2024-01-24T14:16:02.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.029+0000 7f2b75917700 1 -- start start 2024-01-24T14:16:02.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b75917700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b70066c00 0x7f2b7013ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b75917700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b70153c80 0x7f2b701575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b75917700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b70154060 0x7f2b70157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b75917700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70157fc0 con 0x7f2b70153c80 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b75917700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b700c2830 con 0x7f2b70066c00 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b75917700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b700c0d00 con 0x7f2b70154060 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b6e7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b70153c80 0x7f2b701575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b6e7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b70153c80 0x7f2b701575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39084/0 (socket says 172.21.15.29:39084) 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b6e7fc700 1 -- 172.21.15.29:0/1391807270 learned_addr learned my addr 172.21.15.29:0/1391807270 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b6effd700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b70066c00 0x7f2b7013ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b6f7fe700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b70154060 0x7f2b70157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b6e7fc700 1 -- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b70154060 msgr2=0x7f2b70157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b6e7fc700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b70154060 0x7f2b70157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b6e7fc700 1 -- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b70066c00 msgr2=0x7f2b7013ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:02.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b6e7fc700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b70066c00 0x7f2b7013ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:02.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b6e7fc700 1 -- 172.21.15.29:0/1391807270 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b68009ae0 con 0x7f2b70153c80 2024-01-24T14:16:02.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b6e7fc700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b70153c80 0x7f2b701575e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f2b5800c490 tx=0x7f2b58007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:02.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b66ffd700 1 -- 172.21.15.29:0/1391807270 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58001ea0 con 0x7f2b70153c80 2024-01-24T14:16:02.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.030+0000 7f2b66ffd700 1 -- 172.21.15.29:0/1391807270 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b58007020 con 0x7f2b70153c80 2024-01-24T14:16:02.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.031+0000 7f2b75917700 1 -- 172.21.15.29:0/1391807270 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b700c1f00 con 0x7f2b70153c80 2024-01-24T14:16:02.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.031+0000 7f2b66ffd700 1 -- 172.21.15.29:0/1391807270 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b580073c0 con 0x7f2b70153c80 2024-01-24T14:16:02.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.031+0000 7f2b75917700 1 -- 172.21.15.29:0/1391807270 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b700c24c0 con 0x7f2b70153c80 2024-01-24T14:16:02.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.032+0000 7f2b75917700 1 -- 172.21.15.29:0/1391807270 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b700475e0 con 0x7f2b70153c80 2024-01-24T14:16:02.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.032+0000 7f2b66ffd700 1 -- 172.21.15.29:0/1391807270 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2b58006ac0 con 0x7f2b70153c80 2024-01-24T14:16:02.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.032+0000 7f2b66ffd700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b5005f270 0x7f2b50061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:02.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.033+0000 7f2b66ffd700 1 -- 172.21.15.29:0/1391807270 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6892+0+0 (secure 0 0 0) 0x7f2b5807a0e0 con 0x7f2b70153c80 2024-01-24T14:16:02.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.033+0000 7f2b6effd700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b5005f270 0x7f2b50061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:02.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.033+0000 7f2b6effd700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b5005f270 0x7f2b50061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f2b60009c30 tx=0x7f2b60009060).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:02.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.037+0000 7f2b66ffd700 1 -- 172.21.15.29:0/1391807270 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2b5804af30 con 0x7f2b70153c80 2024-01-24T14:16:02.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.289+0000 7f2b75917700 1 -- 172.21.15.29:0/1391807270 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f2b70067db0 con 0x7f2b70153c80 2024-01-24T14:16:02.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.874+0000 7f2b66ffd700 1 -- 172.21.15.29:0/1391807270 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v664) v1 ==== 151+0+0 (secure 0 0 0) 0x7f2b5804e550 con 0x7f2b70153c80 2024-01-24T14:16:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.876+0000 7f2b75917700 1 -- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b5005f270 msgr2=0x7f2b50061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:02.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.876+0000 7f2b75917700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b5005f270 0x7f2b50061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f2b60009c30 tx=0x7f2b60009060).stop 2024-01-24T14:16:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.876+0000 7f2b75917700 1 -- 172.21.15.29:0/1391807270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b70153c80 msgr2=0x7f2b701575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.876+0000 7f2b75917700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b70153c80 0x7f2b701575e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f2b5800c490 tx=0x7f2b58007dd0).stop 2024-01-24T14:16:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.876+0000 7f2b75917700 1 -- 172.21.15.29:0/1391807270 shutdown_connections 2024-01-24T14:16:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.876+0000 7f2b75917700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b70154060 0x7f2b70157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.876+0000 7f2b75917700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b70066c00 0x7f2b7013ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.876+0000 7f2b75917700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b5005f270 0x7f2b50061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.876+0000 7f2b75917700 1 --2- 172.21.15.29:0/1391807270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b70153c80 0x7f2b701575e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.876+0000 7f2b75917700 1 -- 172.21.15.29:0/1391807270 >> 172.21.15.29:0/1391807270 conn(0x7f2b700c8bb0 msgr2=0x7f2b700ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.877+0000 7f2b75917700 1 -- 172.21.15.29:0/1391807270 shutdown_connections 2024-01-24T14:16:02.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:02.877+0000 7f2b75917700 1 -- 172.21.15.29:0/1391807270 wait complete. 2024-01-24T14:16:02.893 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-24T14:16:03.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.028+0000 7feb77392700 1 Processor -- start 2024-01-24T14:16:03.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.029+0000 7feb77392700 1 -- start start 2024-01-24T14:16:03.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.029+0000 7feb77392700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feb701384f0 0x7feb7006a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.029+0000 7feb77392700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb700c8270 0x7feb700627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.029+0000 7feb77392700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb70063150 con 0x7feb70062ca0 2024-01-24T14:16:03.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.029+0000 7feb77392700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb700bcda0 con 0x7feb701384f0 2024-01-24T14:16:03.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.029+0000 7feb77392700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb70065f80 con 0x7feb700c8270 2024-01-24T14:16:03.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb7592f700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7feb70062ca0 0x7feb700692a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:39716/0 (socket says 172.21.15.29:39716) 2024-01-24T14:16:03.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb7592f700 1 -- 172.21.15.29:0/2393627667 learned_addr learned my addr 172.21.15.29:0/2393627667 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:03.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb7492d700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb700c8270 0x7feb700627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb7512e700 1 --2- 172.21.15.29:0/2393627667 >> v2:172.21.15.163:3300/0 conn(0x7feb701384f0 0x7feb7006a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506762374 0 0) 0x7feb68001490 con 0x7feb70062ca0 2024-01-24T14:16:03.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb5c0033f0 con 0x7feb70062ca0 2024-01-24T14:16:03.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3012621170 0 0) 0x7feb5c0033f0 con 0x7feb70062ca0 2024-01-24T14:16:03.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb700c8270 msgr2=0x7feb700627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:03.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb700c8270 0x7feb700627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 >> v2:172.21.15.163:3300/0 conn(0x7feb701384f0 msgr2=0x7feb7006a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:03.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 --2- 172.21.15.29:0/2393627667 >> v2:172.21.15.163:3300/0 conn(0x7feb701384f0 0x7feb7006a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb70066260 con 0x7feb70062ca0 2024-01-24T14:16:03.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7feb68001980 con 0x7feb70062ca0 2024-01-24T14:16:03.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb7512e700 1 --2- 172.21.15.29:0/2393627667 >> v2:172.21.15.163:3300/0 conn(0x7feb701384f0 0x7feb7006a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:16:03.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 >> v1:172.21.15.29:6789/0 conn(0x7feb70062ca0 legacy=0x7feb700692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb5c0038e0 0x7feb5c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5c004690 0x7feb5c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb5c005000 0x7feb5c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb700bcda0 con 0x7feb5c004690 2024-01-24T14:16:03.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb70065f80 con 0x7feb5c0038e0 2024-01-24T14:16:03.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c00be00 con 0x7feb5c005000 2024-01-24T14:16:03.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.030+0000 7feb7492d700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb700c8270 0x7feb700627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:16:03.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb7512e700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb5c0038e0 0x7feb5c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb7492d700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5c004690 0x7feb5c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb7592f700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb5c005000 0x7feb5c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb7512e700 1 -- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb5c005000 msgr2=0x7feb5c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:03.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb7512e700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb5c005000 0x7feb5c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb7512e700 1 -- 172.21.15.29:0/2393627667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5c004690 msgr2=0x7feb5c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:03.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb7512e700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5c004690 0x7feb5c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb7512e700 1 -- 172.21.15.29:0/2393627667 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c0033f0 con 0x7feb5c0038e0 2024-01-24T14:16:03.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb7492d700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5c004690 0x7feb5c004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:16:03.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb7512e700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb5c0038e0 0x7feb5c003d00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7feb64009ca0 tx=0x7feb64009060).ready entity=mon.1 client_cookie=1d8cb2cb0a2a314a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:03.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb640095c0 con 0x7feb5c0038e0 2024-01-24T14:16:03.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6400e770 con 0x7feb5c0038e0 2024-01-24T14:16:03.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.031+0000 7feb63fff700 1 -- 172.21.15.29:0/2393627667 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6400eaf0 con 0x7feb5c0038e0 2024-01-24T14:16:03.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.032+0000 7feb77392700 1 -- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb5c0038e0 msgr2=0x7feb5c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.032+0000 7feb77392700 1 --2- 172.21.15.29:0/2393627667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb5c0038e0 0x7feb5c003d00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7feb64009ca0 tx=0x7feb64009060).stop 2024-01-24T14:16:03.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.032+0000 7feb7592f700 1 -- 172.21.15.29:0/2393627667 reap_dead start 2024-01-24T14:16:03.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.032+0000 7feb77392700 1 -- 172.21.15.29:0/2393627667 shutdown_connections 2024-01-24T14:16:03.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.032+0000 7feb77392700 1 -- 172.21.15.29:0/2393627667 >> 172.21.15.29:0/2393627667 conn(0x7feb70064390 msgr2=0x7feb700c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:03.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.032+0000 7feb77392700 1 -- 172.21.15.29:0/2393627667 shutdown_connections 2024-01-24T14:16:03.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.032+0000 7feb77392700 1 -- 172.21.15.29:0/2393627667 wait complete. 2024-01-24T14:16:03.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.033+0000 7feb77392700 1 Processor -- start 2024-01-24T14:16:03.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.033+0000 7feb77392700 1 -- start start 2024-01-24T14:16:03.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.034+0000 7feb77392700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5c0038e0 0x7feb701367b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.034+0000 7feb77392700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb700aa380 0x7feb700a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.034+0000 7feb77392700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb700a9180 0x7feb700a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.034+0000 7feb77392700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb70136ca0 con 0x7feb5c0038e0 2024-01-24T14:16:03.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.034+0000 7feb77392700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb700a5180 con 0x7feb700a9180 2024-01-24T14:16:03.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.034+0000 7feb77392700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb700a86a0 con 0x7feb700aa380 2024-01-24T14:16:03.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.034+0000 7feb7512e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5c0038e0 0x7feb701367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.034+0000 7feb7512e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5c0038e0 0x7feb701367b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39102/0 (socket says 172.21.15.29:39102) 2024-01-24T14:16:03.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.034+0000 7feb7512e700 1 -- 172.21.15.29:0/4257230600 learned_addr learned my addr 172.21.15.29:0/4257230600 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:03.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb7492d700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb700aa380 0x7feb700a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb7592f700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb700a9180 0x7feb700a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb7512e700 1 -- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb700aa380 msgr2=0x7feb700a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb7512e700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb700aa380 0x7feb700a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb7512e700 1 -- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb700a9180 msgr2=0x7feb700a9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb7512e700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb700a9180 0x7feb700a9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb7512e700 1 -- 172.21.15.29:0/4257230600 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb700a8980 con 0x7feb5c0038e0 2024-01-24T14:16:03.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb7592f700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb700a9180 0x7feb700a9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:16:03.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb7512e700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5c0038e0 0x7feb701367b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7feb5c00bdd0 tx=0x7feb6401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:03.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb617fa700 1 -- 172.21.15.29:0/4257230600 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb64020070 con 0x7feb5c0038e0 2024-01-24T14:16:03.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb617fa700 1 -- 172.21.15.29:0/4257230600 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb640095c0 con 0x7feb5c0038e0 2024-01-24T14:16:03.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb77392700 1 -- 172.21.15.29:0/4257230600 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb700a8c60 con 0x7feb5c0038e0 2024-01-24T14:16:03.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb617fa700 1 -- 172.21.15.29:0/4257230600 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6400eaf0 con 0x7feb5c0038e0 2024-01-24T14:16:03.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.035+0000 7feb77392700 1 -- 172.21.15.29:0/4257230600 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb700a4d70 con 0x7feb5c0038e0 2024-01-24T14:16:03.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.036+0000 7feb77392700 1 -- 172.21.15.29:0/4257230600 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb700a8fd0 con 0x7feb5c0038e0 2024-01-24T14:16:03.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.037+0000 7feb617fa700 1 -- 172.21.15.29:0/4257230600 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7feb64025920 con 0x7feb5c0038e0 2024-01-24T14:16:03.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.037+0000 7feb617fa700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feb5405f270 0x7feb54061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.037+0000 7feb617fa700 1 -- 172.21.15.29:0/4257230600 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6892+0+0 (secure 0 0 0) 0x7feb6407bce0 con 0x7feb5c0038e0 2024-01-24T14:16:03.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.037+0000 7feb7492d700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feb5405f270 0x7feb54061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.038+0000 7feb7492d700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feb5405f270 0x7feb54061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7feb70157b80 tx=0x7feb6c008040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:03.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.042+0000 7feb617fa700 1 -- 172.21.15.29:0/4257230600 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7feb6404cb30 con 0x7feb5c0038e0 2024-01-24T14:16:03.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.224+0000 7feb77392700 1 -- 172.21.15.29:0/4257230600 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7feb700a8df0 con 0x7feb5c0038e0 2024-01-24T14:16:03.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.227+0000 7feb617fa700 1 -- 172.21.15.29:0/4257230600 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7feb6402a020 con 0x7feb5c0038e0 2024-01-24T14:16:03.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.229+0000 7feb77392700 1 -- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feb5405f270 msgr2=0x7feb54061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.229+0000 7feb77392700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feb5405f270 0x7feb54061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7feb70157b80 tx=0x7feb6c008040).stop 2024-01-24T14:16:03.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.229+0000 7feb77392700 1 -- 172.21.15.29:0/4257230600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5c0038e0 msgr2=0x7feb701367b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.229+0000 7feb77392700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5c0038e0 0x7feb701367b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7feb5c00bdd0 tx=0x7feb6401a000).stop 2024-01-24T14:16:03.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.230+0000 7feb77392700 1 -- 172.21.15.29:0/4257230600 shutdown_connections 2024-01-24T14:16:03.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.230+0000 7feb77392700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb700a9180 0x7feb700a9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.230+0000 7feb77392700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feb5405f270 0x7feb54061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.230+0000 7feb77392700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5c0038e0 0x7feb701367b0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.230+0000 7feb77392700 1 --2- 172.21.15.29:0/4257230600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb700aa380 0x7feb700a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.230+0000 7feb77392700 1 -- 172.21.15.29:0/4257230600 >> 172.21.15.29:0/4257230600 conn(0x7feb70064390 msgr2=0x7feb7013a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:03.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.230+0000 7feb77392700 1 -- 172.21.15.29:0/4257230600 shutdown_connections 2024-01-24T14:16:03.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.230+0000 7feb77392700 1 -- 172.21.15.29:0/4257230600 wait complete. 2024-01-24T14:16:03.244 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T14:16:03.245 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:16:03.290 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:16:03.337 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T14:16:03.337 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-24T14:16:03.337 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:16:03.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.471+0000 7ff98fea1700 1 Processor -- start 2024-01-24T14:16:03.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.472+0000 7ff98fea1700 1 -- start start 2024-01-24T14:16:03.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.472+0000 7ff98fea1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff988066b40 0x7ff988062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.472+0000 7ff98fea1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff988158600 0x7ff988154d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.472+0000 7ff98fea1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9880bcf60 con 0x7ff988062970 2024-01-24T14:16:03.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.472+0000 7ff98fea1700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9880bdba0 con 0x7ff988066b40 2024-01-24T14:16:03.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.472+0000 7ff98fea1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880a7480 con 0x7ff988158600 2024-01-24T14:16:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.473+0000 7ff98d43c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff988066b40 0x7ff988062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.473+0000 7ff98d43c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff988066b40 0x7ff988062d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39170/0 (socket says 172.21.15.29:39170) 2024-01-24T14:16:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.473+0000 7ff98d43c700 1 -- 172.21.15.29:0/745138627 learned_addr learned my addr 172.21.15.29:0/745138627 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.473+0000 7ff98e43e700 1 --2- 172.21.15.29:0/745138627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff988158600 0x7ff988154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.473+0000 7ff98d43c700 1 -- 172.21.15.29:0/745138627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff988158600 msgr2=0x7ff988154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.473+0000 7ff98d43c700 1 --2- 172.21.15.29:0/745138627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff988158600 0x7ff988154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.473+0000 7ff98d43c700 1 -- 172.21.15.29:0/745138627 >> v1:172.21.15.163:6789/0 conn(0x7ff988062970 legacy=0x7ff9880c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.473+0000 7ff98d43c700 1 -- 172.21.15.29:0/745138627 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9880a6ec0 con 0x7ff988066b40 2024-01-24T14:16:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.473+0000 7ff98d43c700 1 --2- 172.21.15.29:0/745138627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff988066b40 0x7ff988062d50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7ff97c006fd0 tx=0x7ff97c0090f0).ready entity=mon.0 client_cookie=a6a2af11ea9f8952 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.473+0000 7ff98cc3b700 1 -- 172.21.15.29:0/745138627 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c00b600 con 0x7ff988066b40 2024-01-24T14:16:03.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.473+0000 7ff98cc3b700 1 -- 172.21.15.29:0/745138627 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97c00b760 con 0x7ff988066b40 2024-01-24T14:16:03.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.473+0000 7ff98cc3b700 1 -- 172.21.15.29:0/745138627 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c00bae0 con 0x7ff988066b40 2024-01-24T14:16:03.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.474+0000 7ff98fea1700 1 -- 172.21.15.29:0/745138627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff988066b40 msgr2=0x7ff988062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.474+0000 7ff98fea1700 1 --2- 172.21.15.29:0/745138627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff988066b40 0x7ff988062d50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7ff97c006fd0 tx=0x7ff97c0090f0).stop 2024-01-24T14:16:03.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.474+0000 7ff98fea1700 1 -- 172.21.15.29:0/745138627 shutdown_connections 2024-01-24T14:16:03.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.474+0000 7ff98fea1700 1 --2- 172.21.15.29:0/745138627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff988158600 0x7ff988154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.474+0000 7ff98fea1700 1 --2- 172.21.15.29:0/745138627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff988066b40 0x7ff988062d50 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.474+0000 7ff98fea1700 1 -- 172.21.15.29:0/745138627 >> 172.21.15.29:0/745138627 conn(0x7ff9880c8580 msgr2=0x7ff9880c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:03.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.474+0000 7ff98fea1700 1 -- 172.21.15.29:0/745138627 shutdown_connections 2024-01-24T14:16:03.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.474+0000 7ff98fea1700 1 -- 172.21.15.29:0/745138627 wait complete. 2024-01-24T14:16:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.475+0000 7ff98fea1700 1 Processor -- start 2024-01-24T14:16:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.475+0000 7ff98fea1700 1 -- start start 2024-01-24T14:16:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.476+0000 7ff98fea1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff988062970 0x7ff9880a9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.476+0000 7ff98fea1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff988066b40 0x7ff9880bf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.476+0000 7ff98fea1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff988158600 0x7ff9880c3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.476+0000 7ff98fea1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880bf1e0 con 0x7ff988066b40 2024-01-24T14:16:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.476+0000 7ff98fea1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880c0b30 con 0x7ff988062970 2024-01-24T14:16:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.476+0000 7ff98fea1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9881650e0 con 0x7ff988158600 2024-01-24T14:16:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.476+0000 7ff98d43c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff988066b40 0x7ff9880bf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.476+0000 7ff98d43c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff988066b40 0x7ff9880bf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39182/0 (socket says 172.21.15.29:39182) 2024-01-24T14:16:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.476+0000 7ff98d43c700 1 -- 172.21.15.29:0/2244254480 learned_addr learned my addr 172.21.15.29:0/2244254480 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:03.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.476+0000 7ff98dc3d700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff988062970 0x7ff9880a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.476+0000 7ff98e43e700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff988158600 0x7ff9880c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.477+0000 7ff98d43c700 1 -- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff988158600 msgr2=0x7ff9880c3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.477+0000 7ff98d43c700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff988158600 0x7ff9880c3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.477+0000 7ff98d43c700 1 -- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff988062970 msgr2=0x7ff9880a9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.477+0000 7ff98d43c700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff988062970 0x7ff9880a9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.477+0000 7ff98d43c700 1 -- 172.21.15.29:0/2244254480 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c009370 con 0x7ff988066b40 2024-01-24T14:16:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.477+0000 7ff98d43c700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff988066b40 0x7ff9880bf670 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7ff97c011000 tx=0x7ff97c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.477+0000 7ff985ffb700 1 -- 172.21.15.29:0/2244254480 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c00bea0 con 0x7ff988066b40 2024-01-24T14:16:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.477+0000 7ff985ffb700 1 -- 172.21.15.29:0/2244254480 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97c00b8d0 con 0x7ff988066b40 2024-01-24T14:16:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.477+0000 7ff98fea1700 1 -- 172.21.15.29:0/2244254480 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9881653c0 con 0x7ff988066b40 2024-01-24T14:16:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.477+0000 7ff985ffb700 1 -- 172.21.15.29:0/2244254480 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c01c670 con 0x7ff988066b40 2024-01-24T14:16:03.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.477+0000 7ff98fea1700 1 -- 172.21.15.29:0/2244254480 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9880bee70 con 0x7ff988066b40 2024-01-24T14:16:03.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.478+0000 7ff98fea1700 1 -- 172.21.15.29:0/2244254480 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9880bf350 con 0x7ff988066b40 2024-01-24T14:16:03.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.479+0000 7ff985ffb700 1 -- 172.21.15.29:0/2244254480 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff97c01ca20 con 0x7ff988066b40 2024-01-24T14:16:03.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.479+0000 7ff985ffb700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff97405f270 0x7ff974061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.479+0000 7ff985ffb700 1 -- 172.21.15.29:0/2244254480 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6892+0+0 (secure 0 0 0) 0x7ff97c07c320 con 0x7ff988066b40 2024-01-24T14:16:03.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.479+0000 7ff98dc3d700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff97405f270 0x7ff974061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.480+0000 7ff98dc3d700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff97405f270 0x7ff974061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ff980004290 tx=0x7ff9800014c0).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:03.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.484+0000 7ff985ffb700 1 -- 172.21.15.29:0/2244254480 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff97c050ba0 con 0x7ff988066b40 2024-01-24T14:16:03.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.711+0000 7ff98fea1700 1 -- 172.21.15.29:0/2244254480 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9880475e0 con 0x7ff988066b40 2024-01-24T14:16:03.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.711+0000 7ff985ffb700 1 -- 172.21.15.29:0/2244254480 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+3535 (secure 0 0 0) 0x7ff97c02a090 con 0x7ff988066b40 2024-01-24T14:16:03.712 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:16:03.712 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6341,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4230829295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4230829295},{"type":"v1","addr":"172.21.15.29:6835","nonce":4230829295}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6344,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3194389571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3194389571},{"type":"v1","addr":"172.21.15.163:6836","nonce":3194389571}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2473497045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2473497045},{"type":"v1","addr":"172.21.15.163:6837","nonce":2473497045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46},{"gid":7635,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/3039620149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3039620149},{"type":"v1","addr":"172.21.15.29:6834","nonce":3039620149}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[]} 2024-01-24T14:16:03.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.713+0000 7ff98fea1700 1 -- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff97405f270 msgr2=0x7ff974061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.713+0000 7ff98fea1700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff97405f270 0x7ff974061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ff980004290 tx=0x7ff9800014c0).stop 2024-01-24T14:16:03.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.713+0000 7ff98fea1700 1 -- 172.21.15.29:0/2244254480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff988066b40 msgr2=0x7ff9880bf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.713+0000 7ff98fea1700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff988066b40 0x7ff9880bf670 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7ff97c011000 tx=0x7ff97c017000).stop 2024-01-24T14:16:03.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.713+0000 7ff98fea1700 1 -- 172.21.15.29:0/2244254480 shutdown_connections 2024-01-24T14:16:03.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.713+0000 7ff98fea1700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff988158600 0x7ff9880c3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.713+0000 7ff98fea1700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff988062970 0x7ff9880a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.713+0000 7ff98fea1700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff97405f270 0x7ff974061730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.713+0000 7ff98fea1700 1 --2- 172.21.15.29:0/2244254480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff988066b40 0x7ff9880bf670 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.713+0000 7ff98fea1700 1 -- 172.21.15.29:0/2244254480 >> 172.21.15.29:0/2244254480 conn(0x7ff9880c8580 msgr2=0x7ff9881530b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:03.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.713+0000 7ff98fea1700 1 -- 172.21.15.29:0/2244254480 shutdown_connections 2024-01-24T14:16:03.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.713+0000 7ff98fea1700 1 -- 172.21.15.29:0/2244254480 wait complete. 2024-01-24T14:16:03.715 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 53 2024-01-24T14:16:03.728 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-24T14:16:03.728 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-24T14:16:03.728 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-24T14:16:03.728 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:16:03.728 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-24T14:16:03.728 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-24T14:16:03.729 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-24T14:16:03.729 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:16:03.729 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-24T14:16:03.729 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-24T14:16:03.729 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-24T14:16:03.729 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:16:03.729 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-24T14:16:03.730 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-24T14:16:03.730 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-24T14:16:03.730 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:16:03.740 INFO:tasks.ceph.mds.c:Stopped 2024-01-24T14:16:03.740 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-24T14:16:03.741 INFO:tasks.ceph.mds.a:Stopped 2024-01-24T14:16:03.741 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-24T14:16:03.742 INFO:tasks.ceph.mds.d:Stopped 2024-01-24T14:16:03.742 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-24T14:16:03.743 INFO:tasks.ceph.mds.b:Stopped 2024-01-24T14:16:03.743 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-24T14:16:03.783 INFO:tasks.ceph.mds.c:Started 2024-01-24T14:16:03.783 DEBUG:teuthology.parallel:result is None 2024-01-24T14:16:03.784 INFO:tasks.ceph.mds.a:Started 2024-01-24T14:16:03.785 DEBUG:teuthology.parallel:result is None 2024-01-24T14:16:03.785 INFO:tasks.ceph.mds.d:Started 2024-01-24T14:16:03.785 DEBUG:teuthology.parallel:result is None 2024-01-24T14:16:03.786 INFO:tasks.ceph.mds.b:Started 2024-01-24T14:16:03.786 DEBUG:teuthology.parallel:result is None 2024-01-24T14:16:03.787 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-24T14:16:03.868 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-01-24T14:16:03.868 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T14:16:03.869 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T14:16:03.870 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T14:16:03.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.928+0000 7f27cbc77700 1 Processor -- start 2024-01-24T14:16:03.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.928+0000 7f27cbc77700 1 -- start start 2024-01-24T14:16:03.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27cbc77700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27c41524d0 0x7f27c4066160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27cbc77700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f27c40647f0 0x7f27c414f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27cbc77700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c4064bf0 con 0x7f27c4064410 2024-01-24T14:16:03.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27cbc77700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c406a190 con 0x7f27c41524d0 2024-01-24T14:16:03.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27cbc77700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c4068550 con 0x7f27c40647f0 2024-01-24T14:16:03.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27c9212700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27c41524d0 0x7f27c4066160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27c9212700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27c41524d0 0x7f27c4066160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39218/0 (socket says 172.21.15.29:39218) 2024-01-24T14:16:03.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27c9212700 1 -- 172.21.15.29:0/792634642 learned_addr learned my addr 172.21.15.29:0/792634642 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27ca214700 1 --2- 172.21.15.29:0/792634642 >> v2:172.21.15.163:3300/0 conn(0x7f27c40647f0 0x7f27c414f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27c9212700 1 -- 172.21.15.29:0/792634642 >> v2:172.21.15.163:3300/0 conn(0x7f27c40647f0 msgr2=0x7f27c414f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27c9212700 1 --2- 172.21.15.29:0/792634642 >> v2:172.21.15.163:3300/0 conn(0x7f27c40647f0 0x7f27c414f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27c9212700 1 -- 172.21.15.29:0/792634642 >> v1:172.21.15.163:6789/0 conn(0x7f27c4064410 legacy=0x7f27c4069a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27c9212700 1 -- 172.21.15.29:0/792634642 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c40695f0 con 0x7f27c41524d0 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.929+0000 7f27c9212700 1 --2- 172.21.15.29:0/792634642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c41524d0 0x7f27c4066160 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f27c413bb70 tx=0x7f27b40090f0).ready entity=mon.0 client_cookie=c6e78dc6fc7f2d0a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.930+0000 7f27c8a11700 1 -- 172.21.15.29:0/792634642 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b400b600 con 0x7f27c41524d0 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.930+0000 7f27c8a11700 1 -- 172.21.15.29:0/792634642 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b400b760 con 0x7f27c41524d0 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.930+0000 7f27c8a11700 1 -- 172.21.15.29:0/792634642 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b400bae0 con 0x7f27c41524d0 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.930+0000 7f27cbc77700 1 -- 172.21.15.29:0/792634642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c41524d0 msgr2=0x7f27c4066160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.930+0000 7f27cbc77700 1 --2- 172.21.15.29:0/792634642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c41524d0 0x7f27c4066160 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f27c413bb70 tx=0x7f27b40090f0).stop 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.930+0000 7f27cbc77700 1 -- 172.21.15.29:0/792634642 shutdown_connections 2024-01-24T14:16:03.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.930+0000 7f27cbc77700 1 --2- 172.21.15.29:0/792634642 >> v2:172.21.15.163:3300/0 conn(0x7f27c40647f0 0x7f27c414f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.930+0000 7f27cbc77700 1 --2- 172.21.15.29:0/792634642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c41524d0 0x7f27c4066160 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.930+0000 7f27cbc77700 1 -- 172.21.15.29:0/792634642 >> 172.21.15.29:0/792634642 conn(0x7f27c40c7480 msgr2=0x7f27c40c3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:03.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.930+0000 7f27cbc77700 1 -- 172.21.15.29:0/792634642 shutdown_connections 2024-01-24T14:16:03.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.930+0000 7f27cbc77700 1 -- 172.21.15.29:0/792634642 wait complete. 2024-01-24T14:16:03.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.931+0000 7f27cbc77700 1 Processor -- start 2024-01-24T14:16:03.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.931+0000 7f27cbc77700 1 -- start start 2024-01-24T14:16:03.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.932+0000 7f27cbc77700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27c4064410 0x7f27c414ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.932+0000 7f27cbc77700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c40647f0 0x7f27c4151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.932+0000 7f27cbc77700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27c41524d0 0x7f27c4151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.932+0000 7f27cbc77700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c4152370 con 0x7f27c40647f0 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.932+0000 7f27cbc77700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c4134cb0 con 0x7f27c41524d0 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.932+0000 7f27cbc77700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c4138550 con 0x7f27c4064410 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.932+0000 7f27c9212700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c40647f0 0x7f27c4151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.932+0000 7f27c9212700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c40647f0 0x7f27c4151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39226/0 (socket says 172.21.15.29:39226) 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.932+0000 7f27c9212700 1 -- 172.21.15.29:0/3211135729 learned_addr learned my addr 172.21.15.29:0/3211135729 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.932+0000 7f27c9a13700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27c4064410 0x7f27c414ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.932+0000 7f27ca214700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27c41524d0 0x7f27c4151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.933+0000 7f27c9212700 1 -- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27c4064410 msgr2=0x7f27c414ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.933+0000 7f27c9212700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27c4064410 0x7f27c414ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.933+0000 7f27c9212700 1 -- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27c41524d0 msgr2=0x7f27c4151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.933+0000 7f27c9212700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27c41524d0 0x7f27c4151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.933+0000 7f27c9212700 1 -- 172.21.15.29:0/3211135729 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b4009370 con 0x7f27c40647f0 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.933+0000 7f27c9212700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c40647f0 0x7f27c4151990 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f27c4063880 tx=0x7f27b4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:03.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.933+0000 7f27b9ffb700 1 -- 172.21.15.29:0/3211135729 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b400bea0 con 0x7f27c40647f0 2024-01-24T14:16:03.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.933+0000 7f27b9ffb700 1 -- 172.21.15.29:0/3211135729 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b400b8d0 con 0x7f27c40647f0 2024-01-24T14:16:03.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.933+0000 7f27cbc77700 1 -- 172.21.15.29:0/3211135729 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c41346f0 con 0x7f27c40647f0 2024-01-24T14:16:03.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.933+0000 7f27b9ffb700 1 -- 172.21.15.29:0/3211135729 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b401c670 con 0x7f27c40647f0 2024-01-24T14:16:03.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.933+0000 7f27cbc77700 1 -- 172.21.15.29:0/3211135729 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c41387f0 con 0x7f27c40647f0 2024-01-24T14:16:03.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.934+0000 7f27cbc77700 1 -- 172.21.15.29:0/3211135729 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c41383b0 con 0x7f27c40647f0 2024-01-24T14:16:03.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.935+0000 7f27b9ffb700 1 -- 172.21.15.29:0/3211135729 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f27b401c9e0 con 0x7f27c40647f0 2024-01-24T14:16:03.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.935+0000 7f27b9ffb700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f27a805f270 0x7f27a8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:03.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.935+0000 7f27b9ffb700 1 -- 172.21.15.29:0/3211135729 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6892+0+0 (secure 0 0 0) 0x7f27b407be50 con 0x7f27c40647f0 2024-01-24T14:16:03.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.935+0000 7f27c9a13700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f27a805f270 0x7f27a8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:03.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.936+0000 7f27c9a13700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f27a805f270 0x7f27a8061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f27c0003860 tx=0x7f27c0007d40).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:03.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:03.940+0000 7f27b9ffb700 1 -- 172.21.15.29:0/3211135729 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f27c41383b0 con 0x7f27c40647f0 2024-01-24T14:16:04.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.137+0000 7f27cbc77700 1 -- 172.21.15.29:0/3211135729 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f27c41383b0 con 0x7f27c40647f0 2024-01-24T14:16:04.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.899+0000 7f27b9ffb700 1 -- 172.21.15.29:0/3211135729 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v133) v1 ==== 97+0+0 (secure 0 0 0) 0x7f27c41383b0 con 0x7f27c40647f0 2024-01-24T14:16:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.902+0000 7f27cbc77700 1 -- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f27a805f270 msgr2=0x7f27a8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.902+0000 7f27cbc77700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f27a805f270 0x7f27a8061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f27c0003860 tx=0x7f27c0007d40).stop 2024-01-24T14:16:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.902+0000 7f27cbc77700 1 -- 172.21.15.29:0/3211135729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c40647f0 msgr2=0x7f27c4151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.902+0000 7f27cbc77700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c40647f0 0x7f27c4151990 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f27c4063880 tx=0x7f27b4017000).stop 2024-01-24T14:16:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.902+0000 7f27cbc77700 1 -- 172.21.15.29:0/3211135729 shutdown_connections 2024-01-24T14:16:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.902+0000 7f27cbc77700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27c41524d0 0x7f27c4151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.902+0000 7f27cbc77700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27c4064410 0x7f27c414ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.902+0000 7f27cbc77700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f27a805f270 0x7f27a8061730 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.902+0000 7f27cbc77700 1 --2- 172.21.15.29:0/3211135729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c40647f0 0x7f27c4151990 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:04.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.902+0000 7f27cbc77700 1 -- 172.21.15.29:0/3211135729 >> 172.21.15.29:0/3211135729 conn(0x7f27c40c7480 msgr2=0x7f27c40bb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.902+0000 7f27cbc77700 1 -- 172.21.15.29:0/3211135729 shutdown_connections 2024-01-24T14:16:04.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:04.902+0000 7f27cbc77700 1 -- 172.21.15.29:0/3211135729 wait complete. 2024-01-24T14:16:04.904 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-01-24T14:16:04.918 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-24T14:16:05.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.051+0000 7fce045e8700 1 Processor -- start 2024-01-24T14:16:05.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.051+0000 7fce045e8700 1 -- start start 2024-01-24T14:16:05.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.052+0000 7fce045e8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcdfc066c00 0x7fcdfc0bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.052+0000 7fce045e8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdfc1582b0 con 0x7fcdfc0670a0 2024-01-24T14:16:05.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.052+0000 7fce045e8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdfc157ca0 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.052+0000 7fce045e8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdfc063c60 con 0x7fcdfc066c00 2024-01-24T14:16:05.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.052+0000 7fce01b83700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcdfc066c00 0x7fcdfc0bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.052+0000 7fce02b85700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcdfc0670a0 0x7fcdfc0bdb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52362/0 (socket says 172.21.15.29:52362) 2024-01-24T14:16:05.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.052+0000 7fce01b83700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcdfc066c00 0x7fcdfc0bf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39232/0 (socket says 172.21.15.29:39232) 2024-01-24T14:16:05.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.052+0000 7fce02b85700 1 -- 172.21.15.29:0/531099472 learned_addr learned my addr 172.21.15.29:0/531099472 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:05.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.053+0000 7fce01b83700 1 -- 172.21.15.29:0/531099472 >> v1:172.21.15.163:6789/0 conn(0x7fcdfc0c4a10 legacy=0x7fcdfc0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.053+0000 7fce01b83700 1 -- 172.21.15.29:0/531099472 >> v1:172.21.15.29:6789/0 conn(0x7fcdfc0670a0 legacy=0x7fcdfc0bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.053+0000 7fce01382700 1 -- 172.21.15.29:0/531099472 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666582684 0 0) 0x7fcdfc1582b0 con 0x7fcdfc0670a0 2024-01-24T14:16:05.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.053+0000 7fce01b83700 1 -- 172.21.15.29:0/531099472 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc157f40 con 0x7fcdfc066c00 2024-01-24T14:16:05.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.053+0000 7fce01b83700 1 --2- 172.21.15.29:0/531099472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdfc066c00 0x7fcdfc0bf6d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7fcdf8009760 tx=0x7fcdf8009060).ready entity=mon.0 client_cookie=a7374d1d2274da18 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:05.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.053+0000 7fce01382700 1 -- 172.21.15.29:0/531099472 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf800b580 con 0x7fcdfc066c00 2024-01-24T14:16:05.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.053+0000 7fce01382700 1 -- 172.21.15.29:0/531099472 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf800b6e0 con 0x7fcdfc066c00 2024-01-24T14:16:05.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.053+0000 7fce01382700 1 -- 172.21.15.29:0/531099472 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf800ba60 con 0x7fcdfc066c00 2024-01-24T14:16:05.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.053+0000 7fce045e8700 1 -- 172.21.15.29:0/531099472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdfc066c00 msgr2=0x7fcdfc0bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.053+0000 7fce045e8700 1 --2- 172.21.15.29:0/531099472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdfc066c00 0x7fcdfc0bf6d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7fcdf8009760 tx=0x7fcdf8009060).stop 2024-01-24T14:16:05.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.054+0000 7fce045e8700 1 -- 172.21.15.29:0/531099472 shutdown_connections 2024-01-24T14:16:05.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.054+0000 7fce045e8700 1 --2- 172.21.15.29:0/531099472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdfc066c00 0x7fcdfc0bf6d0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.054+0000 7fce045e8700 1 -- 172.21.15.29:0/531099472 >> 172.21.15.29:0/531099472 conn(0x7fcdfc0c8bb0 msgr2=0x7fcdfc0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:05.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.054+0000 7fce045e8700 1 -- 172.21.15.29:0/531099472 shutdown_connections 2024-01-24T14:16:05.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.054+0000 7fce045e8700 1 -- 172.21.15.29:0/531099472 wait complete. 2024-01-24T14:16:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.055+0000 7fce045e8700 1 Processor -- start 2024-01-24T14:16:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.055+0000 7fce045e8700 1 -- start start 2024-01-24T14:16:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.056+0000 7fce045e8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdfc066c00 0x7fcdfc155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.056+0000 7fce045e8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdfc0670a0 0x7fcdfc0a8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.056+0000 7fce045e8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdfc0c4a10 0x7fcdfc152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.056+0000 7fce045e8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0a9a80 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.056+0000 7fce045e8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0a8e70 con 0x7fcdfc066c00 2024-01-24T14:16:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.056+0000 7fce045e8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0a9150 con 0x7fcdfc0670a0 2024-01-24T14:16:05.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce02b85700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdfc0c4a10 0x7fcdfc152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce02b85700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdfc0c4a10 0x7fcdfc152ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39244/0 (socket says 172.21.15.29:39244) 2024-01-24T14:16:05.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce02b85700 1 -- 172.21.15.29:0/4085730892 learned_addr learned my addr 172.21.15.29:0/4085730892 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:05.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce02384700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdfc066c00 0x7fcdfc155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce01b83700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdfc0670a0 0x7fcdfc0a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce02b85700 1 -- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdfc0670a0 msgr2=0x7fcdfc0a8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce02b85700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdfc0670a0 0x7fcdfc0a8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce02b85700 1 -- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdfc066c00 msgr2=0x7fcdfc155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce02b85700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdfc066c00 0x7fcdfc155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce02b85700 1 -- 172.21.15.29:0/4085730892 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80092a0 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce01b83700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdfc0670a0 0x7fcdfc0a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:16:05.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce02b85700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdfc0c4a10 0x7fcdfc152ba0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fcdf400c790 tx=0x7fcdf4007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:05.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fcdee7fc700 1 -- 172.21.15.29:0/4085730892 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4005d30 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fcdee7fc700 1 -- 172.21.15.29:0/4085730892 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf4007110 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce045e8700 1 -- 172.21.15.29:0/4085730892 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdfc0a9430 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fcdee7fc700 1 -- 172.21.15.29:0/4085730892 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40074d0 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.057+0000 7fce045e8700 1 -- 172.21.15.29:0/4085730892 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc0a8b80 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.058+0000 7fce045e8700 1 -- 172.21.15.29:0/4085730892 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdfc0a9bf0 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.059+0000 7fcdee7fc700 1 -- 172.21.15.29:0/4085730892 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcdf4007840 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.059+0000 7fcdee7fc700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcde005f270 0x7fcde0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.059+0000 7fcdee7fc700 1 -- 172.21.15.29:0/4085730892 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6290+0+0 (secure 0 0 0) 0x7fcdf407a4b0 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.059+0000 7fce02384700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcde005f270 0x7fcde0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.060+0000 7fce02384700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcde005f270 0x7fcde0061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcdf0003630 tx=0x7fcdf0009040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:05.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.063+0000 7fcdee7fc700 1 -- 172.21.15.29:0/4085730892 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcdf404b550 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.316+0000 7fce045e8700 1 -- 172.21.15.29:0/4085730892 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcdfc0475e0 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.316+0000 7fcdee7fc700 1 -- 172.21.15.29:0/4085730892 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fcdf404eb70 con 0x7fcdfc0c4a10 2024-01-24T14:16:05.317 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:16:05.317 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T14:16:05.317 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-01-24T14:16:05.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.317 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew==", 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ==", 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:16:05.318 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw==", 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.319 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.320 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-01-24T14:16:05.320 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ==", 2024-01-24T14:16:05.320 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.320 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:16:05.320 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:16:05.320 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:16:05.320 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.320 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ==", 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA==", 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.321 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw==", 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.322 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ==", 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw==", 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:16:05.323 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ==", 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.324 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A==", 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ==", 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.325 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw==", 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA==", 2024-01-24T14:16:05.326 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g==", 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:16:05.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ==", 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-01-24T14:16:05.328 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlxK1bFxAAZpfJn+Tcpmq7R9Yq29FvVg==", 2024-01-24T14:16:05.329 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.329 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:16:05.329 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:16:05.329 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:16:05.329 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.329 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.329 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-01-24T14:16:05.329 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw==", 2024-01-24T14:16:05.329 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.329 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:16:05.330 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T14:16:05.330 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.330 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:05.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:05.330 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-01-24T14:16:05.330 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA==", 2024-01-24T14:16:05.330 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:16:05.330 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:16:05.330 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T14:16:05.330 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:16:05.330 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.331 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:05.331 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-01-24T14:16:05.331 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T14:16:05.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.318+0000 7fce045e8700 1 -- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcde005f270 msgr2=0x7fcde0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.318+0000 7fce045e8700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcde005f270 0x7fcde0061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcdf0003630 tx=0x7fcdf0009040).stop 2024-01-24T14:16:05.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.318+0000 7fce045e8700 1 -- 172.21.15.29:0/4085730892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdfc0c4a10 msgr2=0x7fcdfc152ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.318+0000 7fce045e8700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdfc0c4a10 0x7fcdfc152ba0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fcdf400c790 tx=0x7fcdf4007f30).stop 2024-01-24T14:16:05.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.319+0000 7fce045e8700 1 -- 172.21.15.29:0/4085730892 shutdown_connections 2024-01-24T14:16:05.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.319+0000 7fce045e8700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcde005f270 0x7fcde0061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.319+0000 7fce045e8700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcdfc0c4a10 0x7fcdfc152ba0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.319+0000 7fce045e8700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdfc066c00 0x7fcdfc155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.319+0000 7fce045e8700 1 --2- 172.21.15.29:0/4085730892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdfc0670a0 0x7fcdfc0a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.319+0000 7fce045e8700 1 -- 172.21.15.29:0/4085730892 >> 172.21.15.29:0/4085730892 conn(0x7fcdfc0c8bb0 msgr2=0x7fcdfc0c3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:05.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.319+0000 7fce045e8700 1 -- 172.21.15.29:0/4085730892 shutdown_connections 2024-01-24T14:16:05.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.319+0000 7fce045e8700 1 -- 172.21.15.29:0/4085730892 wait complete. 2024-01-24T14:16:05.334 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:16:05.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.464+0000 7f79e5da4700 1 Processor -- start 2024-01-24T14:16:05.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.464+0000 7f79e5da4700 1 -- start start 2024-01-24T14:16:05.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79e5da4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f79e0062ec0 0x7f79e00c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79e5da4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79e01550f0 0x7f79e0069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79e5da4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e0065360 con 0x7f79e01540a0 2024-01-24T14:16:05.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79e5da4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e00a74f0 con 0x7f79e0062ec0 2024-01-24T14:16:05.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79e5da4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e00a6ce0 con 0x7f79e01550f0 2024-01-24T14:16:05.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79deffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f79e0062ec0 0x7f79e00c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79deffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f79e0062ec0 0x7f79e00c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39246/0 (socket says 172.21.15.29:39246) 2024-01-24T14:16:05.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79deffd700 1 -- 172.21.15.29:0/2144130429 learned_addr learned my addr 172.21.15.29:0/2144130429 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:05.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79dffff700 1 --2- 172.21.15.29:0/2144130429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79e01550f0 0x7f79e0069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79deffd700 1 -- 172.21.15.29:0/2144130429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79e01550f0 msgr2=0x7f79e0069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:05.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79deffd700 1 --2- 172.21.15.29:0/2144130429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79e01550f0 0x7f79e0069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79deffd700 1 -- 172.21.15.29:0/2144130429 >> v1:172.21.15.163:6789/0 conn(0x7f79e01540a0 legacy=0x7f79e00627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.465+0000 7f79deffd700 1 -- 172.21.15.29:0/2144130429 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e00636f0 con 0x7f79e0062ec0 2024-01-24T14:16:05.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.466+0000 7f79deffd700 1 --2- 172.21.15.29:0/2144130429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e0062ec0 0x7f79e00c4ca0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f79d4006fd0 tx=0x7f79d40090f0).ready entity=mon.0 client_cookie=314089c0432608f7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:05.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.466+0000 7f79de7fc700 1 -- 172.21.15.29:0/2144130429 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400b600 con 0x7f79e0062ec0 2024-01-24T14:16:05.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.466+0000 7f79de7fc700 1 -- 172.21.15.29:0/2144130429 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d400b760 con 0x7f79e0062ec0 2024-01-24T14:16:05.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.466+0000 7f79de7fc700 1 -- 172.21.15.29:0/2144130429 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400bae0 con 0x7f79e0062ec0 2024-01-24T14:16:05.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.466+0000 7f79e5da4700 1 -- 172.21.15.29:0/2144130429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e0062ec0 msgr2=0x7f79e00c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.466+0000 7f79e5da4700 1 --2- 172.21.15.29:0/2144130429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e0062ec0 0x7f79e00c4ca0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f79d4006fd0 tx=0x7f79d40090f0).stop 2024-01-24T14:16:05.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.466+0000 7f79e5da4700 1 -- 172.21.15.29:0/2144130429 shutdown_connections 2024-01-24T14:16:05.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.466+0000 7f79e5da4700 1 --2- 172.21.15.29:0/2144130429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79e01550f0 0x7f79e0069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.466+0000 7f79e5da4700 1 --2- 172.21.15.29:0/2144130429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e0062ec0 0x7f79e00c4ca0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.466+0000 7f79e5da4700 1 -- 172.21.15.29:0/2144130429 >> 172.21.15.29:0/2144130429 conn(0x7f79e00c85c0 msgr2=0x7f79e00c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:05.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.466+0000 7f79e5da4700 1 -- 172.21.15.29:0/2144130429 shutdown_connections 2024-01-24T14:16:05.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.466+0000 7f79e5da4700 1 -- 172.21.15.29:0/2144130429 wait complete. 2024-01-24T14:16:05.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.468+0000 7f79e5da4700 1 Processor -- start 2024-01-24T14:16:05.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.468+0000 7f79e5da4700 1 -- start start 2024-01-24T14:16:05.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79e5da4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79e0062ec0 0x7f79e0168600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79e5da4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e01540a0 0x7f79e01528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79e5da4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79e01550f0 0x7f79e00a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79e5da4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e0165650 con 0x7f79e01540a0 2024-01-24T14:16:05.472 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79e5da4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e0165370 con 0x7f79e01550f0 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79e5da4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e0168af0 con 0x7f79e0062ec0 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79deffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e01540a0 0x7f79e01528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79deffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e01540a0 0x7f79e01528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39252/0 (socket says 172.21.15.29:39252) 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79deffd700 1 -- 172.21.15.29:0/524339856 learned_addr learned my addr 172.21.15.29:0/524339856 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79df7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79e0062ec0 0x7f79e0168600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79dffff700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79e01550f0 0x7f79e00a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79deffd700 1 -- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79e0062ec0 msgr2=0x7f79e0168600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79deffd700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79e0062ec0 0x7f79e0168600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79deffd700 1 -- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79e01550f0 msgr2=0x7f79e00a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79deffd700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79e01550f0 0x7f79e00a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79deffd700 1 -- 172.21.15.29:0/524339856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4009370 con 0x7f79e01540a0 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79deffd700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e01540a0 0x7f79e01528f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f79d4011000 tx=0x7f79d4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.469+0000 7f79c77fe700 1 -- 172.21.15.29:0/524339856 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400bea0 con 0x7f79e01540a0 2024-01-24T14:16:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.470+0000 7f79c77fe700 1 -- 172.21.15.29:0/524339856 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d400b8d0 con 0x7f79e01540a0 2024-01-24T14:16:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.470+0000 7f79e5da4700 1 -- 172.21.15.29:0/524339856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e0164a40 con 0x7f79e01540a0 2024-01-24T14:16:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.470+0000 7f79c77fe700 1 -- 172.21.15.29:0/524339856 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d401b660 con 0x7f79e01540a0 2024-01-24T14:16:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.470+0000 7f79e5da4700 1 -- 172.21.15.29:0/524339856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e0165000 con 0x7f79e01540a0 2024-01-24T14:16:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.471+0000 7f79e5da4700 1 -- 172.21.15.29:0/524339856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e0165830 con 0x7f79e01540a0 2024-01-24T14:16:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.471+0000 7f79c77fe700 1 -- 172.21.15.29:0/524339856 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f79d401ba10 con 0x7f79e01540a0 2024-01-24T14:16:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.471+0000 7f79c77fe700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f79cc05f270 0x7f79cc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.471+0000 7f79c77fe700 1 -- 172.21.15.29:0/524339856 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f79d407be00 con 0x7f79e01540a0 2024-01-24T14:16:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.472+0000 7f79df7fe700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f79cc05f270 0x7f79cc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.472+0000 7f79df7fe700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f79cc05f270 0x7f79cc061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f79d0003860 tx=0x7f79d0007d40).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:05.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.476+0000 7f79c77fe700 1 -- 172.21.15.29:0/524339856 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f79e0165830 con 0x7f79e01540a0 2024-01-24T14:16:05.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.657+0000 7f79e5da4700 1 -- 172.21.15.29:0/524339856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79e0165830 con 0x7f79e01540a0 2024-01-24T14:16:05.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.658+0000 7f79c77fe700 1 -- 172.21.15.29:0/524339856 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+14965 (secure 0 0 0) 0x7f79e0165830 con 0x7f79e01540a0 2024-01-24T14:16:05.658 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:16:05.658 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":133,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:16:04.896505+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:16:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.659+0000 7f79e5da4700 1 -- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f79cc05f270 msgr2=0x7f79cc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.659+0000 7f79e5da4700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f79cc05f270 0x7f79cc061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f79d0003860 tx=0x7f79d0007d40).stop 2024-01-24T14:16:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.659+0000 7f79e5da4700 1 -- 172.21.15.29:0/524339856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e01540a0 msgr2=0x7f79e01528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.659+0000 7f79e5da4700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e01540a0 0x7f79e01528f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f79d4011000 tx=0x7f79d4017000).stop 2024-01-24T14:16:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.660+0000 7f79e5da4700 1 -- 172.21.15.29:0/524339856 shutdown_connections 2024-01-24T14:16:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.660+0000 7f79e5da4700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79e01550f0 0x7f79e00a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.660+0000 7f79e5da4700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79e0062ec0 0x7f79e0168600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.660+0000 7f79e5da4700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f79cc05f270 0x7f79cc061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.660+0000 7f79e5da4700 1 --2- 172.21.15.29:0/524339856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e01540a0 0x7f79e01528f0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.660+0000 7f79e5da4700 1 -- 172.21.15.29:0/524339856 >> 172.21.15.29:0/524339856 conn(0x7f79e00c85c0 msgr2=0x7f79e00c20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.660+0000 7f79e5da4700 1 -- 172.21.15.29:0/524339856 shutdown_connections 2024-01-24T14:16:05.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.660+0000 7f79e5da4700 1 -- 172.21.15.29:0/524339856 wait complete. 2024-01-24T14:16:05.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T14:16:05.675 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T14:16:05.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.806+0000 7fbd10b8b700 1 Processor -- start 2024-01-24T14:16:05.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.806+0000 7fbd10b8b700 1 -- start start 2024-01-24T14:16:05.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.807+0000 7fbd10b8b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbd0c069680 0x7fbd0c069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.807+0000 7fbd10b8b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd0c1589e0 0x7fbd0c15add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.807+0000 7fbd10b8b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0c0c3e70 con 0x7fbd0c0692a0 2024-01-24T14:16:05.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.807+0000 7fbd10b8b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0c0c4300 con 0x7fbd0c069680 2024-01-24T14:16:05.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.807+0000 7fbd10b8b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c0c3260 con 0x7fbd0c1589e0 2024-01-24T14:16:05.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd09d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbd0c069680 0x7fbd0c069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd09d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbd0c069680 0x7fbd0c069ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39256/0 (socket says 172.21.15.29:39256) 2024-01-24T14:16:05.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd09d9b700 1 -- 172.21.15.29:0/2826321453 learned_addr learned my addr 172.21.15.29:0/2826321453 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:05.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd0ad9d700 1 --2- 172.21.15.29:0/2826321453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd0c1589e0 0x7fbd0c15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd09d9b700 1 -- 172.21.15.29:0/2826321453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd0c1589e0 msgr2=0x7fbd0c15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:05.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd09d9b700 1 --2- 172.21.15.29:0/2826321453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd0c1589e0 0x7fbd0c15add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd0959a700 1 -- 172.21.15.29:0/2826321453 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048528663 0 0) 0x7fbd0c0c3e70 con 0x7fbd0c0692a0 2024-01-24T14:16:05.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd09d9b700 1 -- 172.21.15.29:0/2826321453 >> v1:172.21.15.163:6789/0 conn(0x7fbd0c0692a0 legacy=0x7fbd0c065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd09d9b700 1 -- 172.21.15.29:0/2826321453 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0c3540 con 0x7fbd0c069680 2024-01-24T14:16:05.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd09d9b700 1 --2- 172.21.15.29:0/2826321453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd0c069680 0x7fbd0c069ae0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7fbd00006fd0 tx=0x7fbd000090f0).ready entity=mon.0 client_cookie=2d404d54fc2a91f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:05.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd0959a700 1 -- 172.21.15.29:0/2826321453 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000b580 con 0x7fbd0c069680 2024-01-24T14:16:05.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd0959a700 1 -- 172.21.15.29:0/2826321453 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0000b6e0 con 0x7fbd0c069680 2024-01-24T14:16:05.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.808+0000 7fbd0959a700 1 -- 172.21.15.29:0/2826321453 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000ba60 con 0x7fbd0c069680 2024-01-24T14:16:05.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.809+0000 7fbd10b8b700 1 -- 172.21.15.29:0/2826321453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd0c069680 msgr2=0x7fbd0c069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.809+0000 7fbd10b8b700 1 --2- 172.21.15.29:0/2826321453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd0c069680 0x7fbd0c069ae0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7fbd00006fd0 tx=0x7fbd000090f0).stop 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.809+0000 7fbd10b8b700 1 -- 172.21.15.29:0/2826321453 shutdown_connections 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.809+0000 7fbd10b8b700 1 --2- 172.21.15.29:0/2826321453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd0c1589e0 0x7fbd0c15add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.809+0000 7fbd10b8b700 1 --2- 172.21.15.29:0/2826321453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd0c069680 0x7fbd0c069ae0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.809+0000 7fbd10b8b700 1 -- 172.21.15.29:0/2826321453 >> 172.21.15.29:0/2826321453 conn(0x7fbd0c064390 msgr2=0x7fbd0c0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.809+0000 7fbd10b8b700 1 -- 172.21.15.29:0/2826321453 shutdown_connections 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.809+0000 7fbd10b8b700 1 -- 172.21.15.29:0/2826321453 wait complete. 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.810+0000 7fbd10b8b700 1 Processor -- start 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.810+0000 7fbd10b8b700 1 -- start start 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.811+0000 7fbd10b8b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd0c0692a0 0x7fbd0c1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.811+0000 7fbd10b8b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd0c069680 0x7fbd0c152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.811+0000 7fbd10b8b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd0c1589e0 0x7fbd0c147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.811+0000 7fbd10b8b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c1533f0 con 0x7fbd0c0692a0 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.811+0000 7fbd10b8b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c1541f0 con 0x7fbd0c069680 2024-01-24T14:16:05.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.811+0000 7fbd10b8b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c1535e0 con 0x7fbd0c1589e0 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.811+0000 7fbd0a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd0c0692a0 0x7fbd0c1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.811+0000 7fbd0a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd0c0692a0 0x7fbd0c1528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39262/0 (socket says 172.21.15.29:39262) 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.811+0000 7fbd0a59c700 1 -- 172.21.15.29:0/1689467545 learned_addr learned my addr 172.21.15.29:0/1689467545 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.811+0000 7fbd09d9b700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd0c069680 0x7fbd0c152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.811+0000 7fbd0ad9d700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd0c1589e0 0x7fbd0c147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.812+0000 7fbd0a59c700 1 -- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd0c1589e0 msgr2=0x7fbd0c147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.812+0000 7fbd0a59c700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd0c1589e0 0x7fbd0c147580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.812+0000 7fbd0a59c700 1 -- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd0c069680 msgr2=0x7fbd0c152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.812+0000 7fbd0a59c700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd0c069680 0x7fbd0c152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.812+0000 7fbd0a59c700 1 -- 172.21.15.29:0/1689467545 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00009370 con 0x7fbd0c0692a0 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.812+0000 7fbd0a59c700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd0c0692a0 0x7fbd0c1528d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7fbcfc001eb0 tx=0x7fbcfc007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.812+0000 7fbcfa7fc700 1 -- 172.21.15.29:0/1689467545 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00ce60 con 0x7fbd0c0692a0 2024-01-24T14:16:05.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.812+0000 7fbcfa7fc700 1 -- 172.21.15.29:0/1689467545 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc00f470 con 0x7fbd0c0692a0 2024-01-24T14:16:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.812+0000 7fbcfa7fc700 1 -- 172.21.15.29:0/1689467545 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00f810 con 0x7fbd0c0692a0 2024-01-24T14:16:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.812+0000 7fbd10b8b700 1 -- 172.21.15.29:0/1689467545 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c1538c0 con 0x7fbd0c0692a0 2024-01-24T14:16:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.812+0000 7fbd10b8b700 1 -- 172.21.15.29:0/1689467545 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c153e80 con 0x7fbd0c0692a0 2024-01-24T14:16:05.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.813+0000 7fbd10b8b700 1 -- 172.21.15.29:0/1689467545 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0c0475e0 con 0x7fbd0c0692a0 2024-01-24T14:16:05.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.814+0000 7fbcfa7fc700 1 -- 172.21.15.29:0/1689467545 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbcfc00fb80 con 0x7fbd0c0692a0 2024-01-24T14:16:05.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.815+0000 7fbcfa7fc700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbcec05f270 0x7fbcec061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:05.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.815+0000 7fbcfa7fc700 1 -- 172.21.15.29:0/1689467545 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6290+0+0 (secure 0 0 0) 0x7fbcfc079f30 con 0x7fbd0c0692a0 2024-01-24T14:16:05.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.815+0000 7fbd09d9b700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbcec05f270 0x7fbcec061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:05.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.816+0000 7fbd09d9b700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbcec05f270 0x7fbcec061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fbd00009fd0 tx=0x7fbd00017000).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:05.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:05.818+0000 7fbcfa7fc700 1 -- 172.21.15.29:0/1689467545 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbcfc04b090 con 0x7fbd0c0692a0 2024-01-24T14:16:06.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.000+0000 7fbd10b8b700 1 -- 172.21.15.29:0/1689467545 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbd0c0c3820 con 0x7fbd0c0692a0 2024-01-24T14:16:06.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.001+0000 7fbcfa7fc700 1 -- 172.21.15.29:0/1689467545 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v133) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbcfc04e6b0 con 0x7fbd0c0692a0 2024-01-24T14:16:06.002 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T14:16:06.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.003+0000 7fbd10b8b700 1 -- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbcec05f270 msgr2=0x7fbcec061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.003+0000 7fbd10b8b700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbcec05f270 0x7fbcec061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fbd00009fd0 tx=0x7fbd00017000).stop 2024-01-24T14:16:06.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.003+0000 7fbd10b8b700 1 -- 172.21.15.29:0/1689467545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd0c0692a0 msgr2=0x7fbd0c1528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.003+0000 7fbd10b8b700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd0c0692a0 0x7fbd0c1528d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7fbcfc001eb0 tx=0x7fbcfc007c70).stop 2024-01-24T14:16:06.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.003+0000 7fbd10b8b700 1 -- 172.21.15.29:0/1689467545 shutdown_connections 2024-01-24T14:16:06.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.003+0000 7fbd10b8b700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd0c1589e0 0x7fbd0c147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.003+0000 7fbd10b8b700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbcec05f270 0x7fbcec061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.003+0000 7fbd10b8b700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd0c0692a0 0x7fbd0c1528d0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.003+0000 7fbd10b8b700 1 --2- 172.21.15.29:0/1689467545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd0c069680 0x7fbd0c152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.003+0000 7fbd10b8b700 1 -- 172.21.15.29:0/1689467545 >> 172.21.15.29:0/1689467545 conn(0x7fbd0c064390 msgr2=0x7fbd0c0acca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:06.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.003+0000 7fbd10b8b700 1 -- 172.21.15.29:0/1689467545 shutdown_connections 2024-01-24T14:16:06.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.003+0000 7fbd10b8b700 1 -- 172.21.15.29:0/1689467545 wait complete. 2024-01-24T14:16:06.017 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T14:16:06.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.151+0000 7ff1748d0700 1 Processor -- start 2024-01-24T14:16:06.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.151+0000 7ff1748d0700 1 -- start start 2024-01-24T14:16:06.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.152+0000 7ff1748d0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170066950 0x7ff1700be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.152+0000 7ff1748d0700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1700640d0 con 0x7ff1700664c0 2024-01-24T14:16:06.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.152+0000 7ff1748d0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1700679f0 con 0x7ff17013a5e0 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.152+0000 7ff1748d0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff170067cd0 con 0x7ff170066950 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.152+0000 7ff16dd9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff1700664c0 0x7ff1700bdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52366/0 (socket says 172.21.15.29:52366) 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.152+0000 7ff16dd9b700 1 -- 172.21.15.29:0/957231443 learned_addr learned my addr 172.21.15.29:0/957231443 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.152+0000 7ff16ed9d700 1 --2- 172.21.15.29:0/957231443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170066950 0x7ff1700be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.152+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143879293 0 0) 0x7ff1700640d0 con 0x7ff1700664c0 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.152+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1580033f0 con 0x7ff1700664c0 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 177082378 0 0) 0x7ff1580033f0 con 0x7ff1700664c0 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170066950 msgr2=0x7ff1700be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 --2- 172.21.15.29:0/957231443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170066950 0x7ff1700be240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 >> v1:172.21.15.163:6789/0 conn(0x7ff17013a5e0 legacy=0x7ff1700639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff170067fb0 con 0x7ff1700664c0 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7ff164001470 con 0x7ff1700664c0 2024-01-24T14:16:06.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 >> v1:172.21.15.29:6789/0 conn(0x7ff1700664c0 legacy=0x7ff1700bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 --2- 172.21.15.29:0/957231443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158003910 0x7ff158003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 --2- 172.21.15.29:0/957231443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff158004690 0x7ff158004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 --2- 172.21.15.29:0/957231443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170066950 0x7ff158005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff170067cd0 con 0x7ff158004690 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15800b8e0 con 0x7ff158003910 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15800baa0 con 0x7ff170066950 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16dd9b700 1 --2- 172.21.15.29:0/957231443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff158004690 0x7ff158004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16e59c700 1 --2- 172.21.15.29:0/957231443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158003910 0x7ff158003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16ed9d700 1 --2- 172.21.15.29:0/957231443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170066950 0x7ff158005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16dd9b700 1 -- 172.21.15.29:0/957231443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170066950 msgr2=0x7ff158005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16dd9b700 1 --2- 172.21.15.29:0/957231443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170066950 0x7ff158005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16dd9b700 1 -- 172.21.15.29:0/957231443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158003910 msgr2=0x7ff158003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:06.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16dd9b700 1 --2- 172.21.15.29:0/957231443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158003910 0x7ff158003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16dd9b700 1 -- 172.21.15.29:0/957231443 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1580033f0 con 0x7ff158004690 2024-01-24T14:16:06.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.153+0000 7ff16dd9b700 1 --2- 172.21.15.29:0/957231443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff158004690 0x7ff158004b10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7ff16400c0e0 tx=0x7ff164001da0).ready entity=mon.0 client_cookie=4b9855d04df39243 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:06.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.154+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164007970 con 0x7ff158004690 2024-01-24T14:16:06.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.154+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16400ca10 con 0x7ff158004690 2024-01-24T14:16:06.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.154+0000 7ff16d59a700 1 -- 172.21.15.29:0/957231443 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16400cd90 con 0x7ff158004690 2024-01-24T14:16:06.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.154+0000 7ff1748d0700 1 -- 172.21.15.29:0/957231443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff158004690 msgr2=0x7ff158004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.154+0000 7ff1748d0700 1 --2- 172.21.15.29:0/957231443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff158004690 0x7ff158004b10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7ff16400c0e0 tx=0x7ff164001da0).stop 2024-01-24T14:16:06.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.154+0000 7ff16ed9d700 1 -- 172.21.15.29:0/957231443 reap_dead start 2024-01-24T14:16:06.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.154+0000 7ff1748d0700 1 -- 172.21.15.29:0/957231443 shutdown_connections 2024-01-24T14:16:06.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.154+0000 7ff1748d0700 1 -- 172.21.15.29:0/957231443 >> 172.21.15.29:0/957231443 conn(0x7ff1700c8bd0 msgr2=0x7ff1700c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:06.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.154+0000 7ff1748d0700 1 -- 172.21.15.29:0/957231443 shutdown_connections 2024-01-24T14:16:06.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.154+0000 7ff1748d0700 1 -- 172.21.15.29:0/957231443 wait complete. 2024-01-24T14:16:06.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.155+0000 7ff1748d0700 1 Processor -- start 2024-01-24T14:16:06.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.156+0000 7ff1748d0700 1 -- start start 2024-01-24T14:16:06.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.156+0000 7ff1748d0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158004690 0x7ff170152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.156+0000 7ff1748d0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170164000 0x7ff1700aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.156+0000 7ff1748d0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1700ab210 0x7ff1700a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.156+0000 7ff1748d0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1700a89d0 con 0x7ff1700ab210 2024-01-24T14:16:06.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.156+0000 7ff1748d0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1700a83f0 con 0x7ff158004690 2024-01-24T14:16:06.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.156+0000 7ff1748d0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1700ab670 con 0x7ff170164000 2024-01-24T14:16:06.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff16ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1700ab210 0x7ff1700a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff16ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1700ab210 0x7ff1700a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39272/0 (socket says 172.21.15.29:39272) 2024-01-24T14:16:06.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff16ed9d700 1 -- 172.21.15.29:0/2941192859 learned_addr learned my addr 172.21.15.29:0/2941192859 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:06.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff16e59c700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158004690 0x7ff170152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff16dd9b700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170164000 0x7ff1700aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff16ed9d700 1 -- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170164000 msgr2=0x7ff1700aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff16ed9d700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170164000 0x7ff1700aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff16ed9d700 1 -- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158004690 msgr2=0x7ff170152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff16ed9d700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158004690 0x7ff170152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff16ed9d700 1 -- 172.21.15.29:0/2941192859 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1700a55d0 con 0x7ff1700ab210 2024-01-24T14:16:06.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff16ed9d700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1700ab210 0x7ff1700a4880 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7ff1680094d0 tx=0x7ff16800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:06.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff15e7fc700 1 -- 172.21.15.29:0/2941192859 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16800e040 con 0x7ff1700ab210 2024-01-24T14:16:06.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff15e7fc700 1 -- 172.21.15.29:0/2941192859 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16800f040 con 0x7ff1700ab210 2024-01-24T14:16:06.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff15e7fc700 1 -- 172.21.15.29:0/2941192859 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16800fe50 con 0x7ff1700ab210 2024-01-24T14:16:06.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff1748d0700 1 -- 172.21.15.29:0/2941192859 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1700a8660 con 0x7ff1700ab210 2024-01-24T14:16:06.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.157+0000 7ff1748d0700 1 -- 172.21.15.29:0/2941192859 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff170152d60 con 0x7ff1700ab210 2024-01-24T14:16:06.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.159+0000 7ff1748d0700 1 -- 172.21.15.29:0/2941192859 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1700a8b40 con 0x7ff1700ab210 2024-01-24T14:16:06.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.159+0000 7ff15e7fc700 1 -- 172.21.15.29:0/2941192859 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff168015660 con 0x7ff1700ab210 2024-01-24T14:16:06.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.159+0000 7ff15e7fc700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff15005f270 0x7ff150061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.159+0000 7ff15e7fc700 1 -- 172.21.15.29:0/2941192859 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6290+0+0 (secure 0 0 0) 0x7ff16807b010 con 0x7ff1700ab210 2024-01-24T14:16:06.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.159+0000 7ff16e59c700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff15005f270 0x7ff150061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.160+0000 7ff16e59c700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff15005f270 0x7ff150061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff1600078e0 tx=0x7ff160009040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:06.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.164+0000 7ff15e7fc700 1 -- 172.21.15.29:0/2941192859 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff16804c0b0 con 0x7ff1700ab210 2024-01-24T14:16:06.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.344+0000 7ff1748d0700 1 -- 172.21.15.29:0/2941192859 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff1700475e0 con 0x7ff1700ab210 2024-01-24T14:16:06.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.345+0000 7ff15e7fc700 1 -- 172.21.15.29:0/2941192859 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v133) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff16804f6d0 con 0x7ff1700ab210 2024-01-24T14:16:06.346 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T14:16:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.347+0000 7ff1748d0700 1 -- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff15005f270 msgr2=0x7ff150061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.347+0000 7ff1748d0700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff15005f270 0x7ff150061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff1600078e0 tx=0x7ff160009040).stop 2024-01-24T14:16:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.347+0000 7ff1748d0700 1 -- 172.21.15.29:0/2941192859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1700ab210 msgr2=0x7ff1700a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.347+0000 7ff1748d0700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1700ab210 0x7ff1700a4880 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7ff1680094d0 tx=0x7ff16800b040).stop 2024-01-24T14:16:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.347+0000 7ff1748d0700 1 -- 172.21.15.29:0/2941192859 shutdown_connections 2024-01-24T14:16:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.347+0000 7ff1748d0700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff15005f270 0x7ff150061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.347+0000 7ff1748d0700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1700ab210 0x7ff1700a4880 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.347+0000 7ff1748d0700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158004690 0x7ff170152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.347+0000 7ff1748d0700 1 --2- 172.21.15.29:0/2941192859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff170164000 0x7ff1700aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.347+0000 7ff1748d0700 1 -- 172.21.15.29:0/2941192859 >> 172.21.15.29:0/2941192859 conn(0x7ff1700c8bd0 msgr2=0x7ff1701536f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.347+0000 7ff1748d0700 1 -- 172.21.15.29:0/2941192859 shutdown_connections 2024-01-24T14:16:06.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.347+0000 7ff1748d0700 1 -- 172.21.15.29:0/2941192859 wait complete. 2024-01-24T14:16:06.362 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-24T14:16:06.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.491+0000 7faa4f27b700 1 Processor -- start 2024-01-24T14:16:06.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.492+0000 7faa4f27b700 1 -- start start 2024-01-24T14:16:06.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa4f27b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa4813a5c0 0x7faa48063a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa4f27b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faa480be520 0x7faa480664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa4f27b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa48066e90 con 0x7faa480669e0 2024-01-24T14:16:06.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa4f27b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa480bf6a0 con 0x7faa480be520 2024-01-24T14:16:06.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa4f27b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa480bea90 con 0x7faa4813a5c0 2024-01-24T14:16:06.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa4d818700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7faa480669e0 0x7faa48067a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52376/0 (socket says 172.21.15.29:52376) 2024-01-24T14:16:06.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa4d818700 1 -- 172.21.15.29:0/293874549 learned_addr learned my addr 172.21.15.29:0/293874549 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:06.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa4c816700 1 --2- 172.21.15.29:0/293874549 >> v2:172.21.15.163:3300/0 conn(0x7faa480be520 0x7faa480664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa4d017700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa4813a5c0 0x7faa48063a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316993622 0 0) 0x7faa40001490 con 0x7faa480669e0 2024-01-24T14:16:06.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa340033f0 con 0x7faa480669e0 2024-01-24T14:16:06.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3837020326 0 0) 0x7faa340033f0 con 0x7faa480669e0 2024-01-24T14:16:06.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa4813a5c0 msgr2=0x7faa48063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:06.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa3ffff700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa4813a5c0 0x7faa48063a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 >> v2:172.21.15.163:3300/0 conn(0x7faa480be520 msgr2=0x7faa480664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:06.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa3ffff700 1 --2- 172.21.15.29:0/293874549 >> v2:172.21.15.163:3300/0 conn(0x7faa480be520 0x7faa480664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa480bed70 con 0x7faa480669e0 2024-01-24T14:16:06.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7faa40001980 con 0x7faa480669e0 2024-01-24T14:16:06.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa4c816700 1 --2- 172.21.15.29:0/293874549 >> v2:172.21.15.163:3300/0 conn(0x7faa480be520 0x7faa480664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:16:06.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 >> v1:172.21.15.29:6789/0 conn(0x7faa480669e0 legacy=0x7faa48067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.493+0000 7faa3ffff700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa340038e0 0x7faa34003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa3ffff700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa4813a5c0 0x7faa34004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa3ffff700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa34004b80 0x7faa34005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa480bf6a0 con 0x7faa340038e0 2024-01-24T14:16:06.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa480bea90 con 0x7faa34004b80 2024-01-24T14:16:06.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3400b8e0 con 0x7faa4813a5c0 2024-01-24T14:16:06.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa4d017700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa340038e0 0x7faa34003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa4c816700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa4813a5c0 0x7faa34004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa4d818700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa34004b80 0x7faa34005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa4d017700 1 -- 172.21.15.29:0/293874549 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa4813a5c0 msgr2=0x7faa34004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:06.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa4d017700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa4813a5c0 0x7faa34004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa4d017700 1 -- 172.21.15.29:0/293874549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa34004b80 msgr2=0x7faa34005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:06.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa4d017700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa34004b80 0x7faa34005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa4d017700 1 -- 172.21.15.29:0/293874549 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa340033f0 con 0x7faa340038e0 2024-01-24T14:16:06.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa4d818700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa34004b80 0x7faa34005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:06.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa4d017700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa340038e0 0x7faa34003d00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7faa44009a60 tx=0x7faa44009060).ready entity=mon.0 client_cookie=a3d41cd9661616f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:06.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44009ea0 con 0x7faa340038e0 2024-01-24T14:16:06.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa44009430 con 0x7faa340038e0 2024-01-24T14:16:06.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.494+0000 7faa3ffff700 1 -- 172.21.15.29:0/293874549 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4400e870 con 0x7faa340038e0 2024-01-24T14:16:06.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.495+0000 7faa4f27b700 1 -- 172.21.15.29:0/293874549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa340038e0 msgr2=0x7faa34003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.495+0000 7faa4f27b700 1 --2- 172.21.15.29:0/293874549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa340038e0 0x7faa34003d00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7faa44009a60 tx=0x7faa44009060).stop 2024-01-24T14:16:06.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.495+0000 7faa4d818700 1 -- 172.21.15.29:0/293874549 reap_dead start 2024-01-24T14:16:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.495+0000 7faa4f27b700 1 -- 172.21.15.29:0/293874549 shutdown_connections 2024-01-24T14:16:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.495+0000 7faa4f27b700 1 -- 172.21.15.29:0/293874549 >> 172.21.15.29:0/293874549 conn(0x7faa480c8bb0 msgr2=0x7faa480c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.495+0000 7faa4f27b700 1 -- 172.21.15.29:0/293874549 shutdown_connections 2024-01-24T14:16:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.495+0000 7faa4f27b700 1 -- 172.21.15.29:0/293874549 wait complete. 2024-01-24T14:16:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.496+0000 7faa4f27b700 1 Processor -- start 2024-01-24T14:16:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.496+0000 7faa4f27b700 1 -- start start 2024-01-24T14:16:06.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.497+0000 7faa4f27b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa340038e0 0x7faa48138610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.497+0000 7faa4f27b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa48155b40 0x7faa48155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.497+0000 7faa4f27b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa480ad8e0 0x7faa480add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.497+0000 7faa4f27b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa481564b0 con 0x7faa340038e0 2024-01-24T14:16:06.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.497+0000 7faa4f27b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa480ae250 con 0x7faa48155b40 2024-01-24T14:16:06.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.497+0000 7faa4f27b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa48138b00 con 0x7faa480ad8e0 2024-01-24T14:16:06.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.497+0000 7faa4d017700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa340038e0 0x7faa48138610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.497+0000 7faa4d017700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa340038e0 0x7faa48138610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39290/0 (socket says 172.21.15.29:39290) 2024-01-24T14:16:06.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.497+0000 7faa4d017700 1 -- 172.21.15.29:0/3754731601 learned_addr learned my addr 172.21.15.29:0/3754731601 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:06.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa4d818700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa480ad8e0 0x7faa480add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa4c816700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa48155b40 0x7faa48155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa4d017700 1 -- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa480ad8e0 msgr2=0x7faa480add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa4d017700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa480ad8e0 0x7faa480add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa4d017700 1 -- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa48155b40 msgr2=0x7faa48155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa4d017700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa48155b40 0x7faa48155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa4d017700 1 -- 172.21.15.29:0/3754731601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa48156700 con 0x7faa340038e0 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa4d017700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa340038e0 0x7faa48138610 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7faa4401b520 tx=0x7faa4401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa3d7fa700 1 -- 172.21.15.29:0/3754731601 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44020070 con 0x7faa340038e0 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa3d7fa700 1 -- 172.21.15.29:0/3754731601 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4401b9e0 con 0x7faa340038e0 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa4f27b700 1 -- 172.21.15.29:0/3754731601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa481569e0 con 0x7faa340038e0 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa3d7fa700 1 -- 172.21.15.29:0/3754731601 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4401bda0 con 0x7faa340038e0 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.498+0000 7faa4f27b700 1 -- 172.21.15.29:0/3754731601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa48156fa0 con 0x7faa340038e0 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.499+0000 7faa4f27b700 1 -- 172.21.15.29:0/3754731601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa480475e0 con 0x7faa340038e0 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.500+0000 7faa3d7fa700 1 -- 172.21.15.29:0/3754731601 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7faa440123f0 con 0x7faa340038e0 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.500+0000 7faa3d7fa700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa2c05f270 0x7faa2c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.500+0000 7faa3d7fa700 1 -- 172.21.15.29:0/3754731601 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6290+0+0 (secure 0 0 0) 0x7faa4407b810 con 0x7faa340038e0 2024-01-24T14:16:06.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.500+0000 7faa4c816700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa2c05f270 0x7faa2c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.501+0000 7faa4c816700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa2c05f270 0x7faa2c061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7faa48064390 tx=0x7faa38009060).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:06.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.505+0000 7faa3d7fa700 1 -- 172.21.15.29:0/3754731601 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7faa4404c850 con 0x7faa340038e0 2024-01-24T14:16:06.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.689+0000 7faa4f27b700 1 -- 172.21.15.29:0/3754731601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7faa480bf050 con 0x7faa340038e0 2024-01-24T14:16:06.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.689+0000 7faa3d7fa700 1 -- 172.21.15.29:0/3754731601 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v133) v1 ==== 84+0+143 (secure 0 0 0) 0x7faa44029020 con 0x7faa340038e0 2024-01-24T14:16:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:16:06.690 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T14:16:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-01-24T14:16:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "device_health_metrics" 2024-01-24T14:16:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:16:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 2, 2024-01-24T14:16:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "rbd" 2024-01-24T14:16:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:06.691 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T14:16:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.691+0000 7faa4f27b700 1 -- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa2c05f270 msgr2=0x7faa2c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.691+0000 7faa4f27b700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa2c05f270 0x7faa2c061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7faa48064390 tx=0x7faa38009060).stop 2024-01-24T14:16:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.691+0000 7faa4f27b700 1 -- 172.21.15.29:0/3754731601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa340038e0 msgr2=0x7faa48138610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.691+0000 7faa4f27b700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa340038e0 0x7faa48138610 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7faa4401b520 tx=0x7faa4401a000).stop 2024-01-24T14:16:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.691+0000 7faa4f27b700 1 -- 172.21.15.29:0/3754731601 shutdown_connections 2024-01-24T14:16:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.691+0000 7faa4f27b700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa480ad8e0 0x7faa480add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.691+0000 7faa4f27b700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa2c05f270 0x7faa2c061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.691+0000 7faa4f27b700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa340038e0 0x7faa48138610 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.691+0000 7faa4f27b700 1 --2- 172.21.15.29:0/3754731601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa48155b40 0x7faa48155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.691+0000 7faa4f27b700 1 -- 172.21.15.29:0/3754731601 >> 172.21.15.29:0/3754731601 conn(0x7faa480c8bb0 msgr2=0x7faa48138e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.691+0000 7faa4f27b700 1 -- 172.21.15.29:0/3754731601 shutdown_connections 2024-01-24T14:16:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.691+0000 7faa4f27b700 1 -- 172.21.15.29:0/3754731601 wait complete. 2024-01-24T14:16:06.705 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-24T14:16:06.706 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2024-01-24T14:16:06.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.839+0000 7fd12823d700 1 Processor -- start 2024-01-24T14:16:06.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.840+0000 7fd12823d700 1 -- start start 2024-01-24T14:16:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd12823d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd12014f8f0 0x7fd120152490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd12823d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd12014fcd0 0x7fd120156190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd12823d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12006a690 con 0x7fd120156680 2024-01-24T14:16:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd12823d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd120155a60 con 0x7fd12014f8f0 2024-01-24T14:16:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd12823d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd120155d00 con 0x7fd12014fcd0 2024-01-24T14:16:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd125fd9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd12014f8f0 0x7fd120152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd125fd9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd12014f8f0 0x7fd120152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39306/0 (socket says 172.21.15.29:39306) 2024-01-24T14:16:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd125fd9700 1 -- 172.21.15.29:0/3510407334 learned_addr learned my addr 172.21.15.29:0/3510407334 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd1267da700 1 --1- 172.21.15.29:0/3510407334 >> v1:172.21.15.29:6789/0 conn(0x7fd120156680 0x7fd120065410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52384/0 (socket says 172.21.15.29:52384) 2024-01-24T14:16:06.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd1257d8700 1 --2- 172.21.15.29:0/3510407334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd12014fcd0 0x7fd120156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd125fd9700 1 -- 172.21.15.29:0/3510407334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd12014fcd0 msgr2=0x7fd120156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd125fd9700 1 --2- 172.21.15.29:0/3510407334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd12014fcd0 0x7fd120156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd125fd9700 1 -- 172.21.15.29:0/3510407334 >> v1:172.21.15.29:6789/0 conn(0x7fd120156680 legacy=0x7fd120065410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd124fd7700 1 -- 172.21.15.29:0/3510407334 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485322241 0 0) 0x7fd118001490 con 0x7fd120156680 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd125fd9700 1 -- 172.21.15.29:0/3510407334 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd120152cf0 con 0x7fd12014f8f0 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd125fd9700 1 --2- 172.21.15.29:0/3510407334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd12014f8f0 0x7fd120152490 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7fd11c006fd0 tx=0x7fd11c0090f0).ready entity=mon.0 client_cookie=3aa36a7912b9716c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.841+0000 7fd124fd7700 1 -- 172.21.15.29:0/3510407334 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00b600 con 0x7fd12014f8f0 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.842+0000 7fd124fd7700 1 -- 172.21.15.29:0/3510407334 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c00b760 con 0x7fd12014f8f0 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.842+0000 7fd124fd7700 1 -- 172.21.15.29:0/3510407334 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00bae0 con 0x7fd12014f8f0 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.842+0000 7fd12823d700 1 -- 172.21.15.29:0/3510407334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd12014f8f0 msgr2=0x7fd120152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.842+0000 7fd12823d700 1 --2- 172.21.15.29:0/3510407334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd12014f8f0 0x7fd120152490 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7fd11c006fd0 tx=0x7fd11c0090f0).stop 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.842+0000 7fd12823d700 1 -- 172.21.15.29:0/3510407334 shutdown_connections 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.842+0000 7fd12823d700 1 --2- 172.21.15.29:0/3510407334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd12014f8f0 0x7fd120152490 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.842+0000 7fd12823d700 1 --2- 172.21.15.29:0/3510407334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd12014fcd0 0x7fd120156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.842+0000 7fd12823d700 1 -- 172.21.15.29:0/3510407334 >> 172.21.15.29:0/3510407334 conn(0x7fd120067b20 msgr2=0x7fd1200638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:06.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.842+0000 7fd12823d700 1 -- 172.21.15.29:0/3510407334 shutdown_connections 2024-01-24T14:16:06.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.842+0000 7fd12823d700 1 -- 172.21.15.29:0/3510407334 wait complete. 2024-01-24T14:16:06.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.844+0000 7fd12823d700 1 Processor -- start 2024-01-24T14:16:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.844+0000 7fd12823d700 1 -- start start 2024-01-24T14:16:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.844+0000 7fd12823d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12014f8f0 0x7fd1200acb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.844+0000 7fd12823d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd12014fcd0 0x7fd12014c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd12823d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd120156680 0x7fd1201643e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd12823d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200ad050 con 0x7fd120156680 2024-01-24T14:16:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd12823d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12014ca30 con 0x7fd12014f8f0 2024-01-24T14:16:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd12823d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200ac0d0 con 0x7fd12014fcd0 2024-01-24T14:16:06.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd1267da700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd120156680 0x7fd1201643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd1267da700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd120156680 0x7fd1201643e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39316/0 (socket says 172.21.15.29:39316) 2024-01-24T14:16:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd1267da700 1 -- 172.21.15.29:0/711287557 learned_addr learned my addr 172.21.15.29:0/711287557 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd125fd9700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12014f8f0 0x7fd1200acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd1257d8700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd12014fcd0 0x7fd12014c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd1267da700 1 -- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd12014fcd0 msgr2=0x7fd12014c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd1267da700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd12014fcd0 0x7fd12014c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd1267da700 1 -- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12014f8f0 msgr2=0x7fd1200acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd1267da700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12014f8f0 0x7fd1200acb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd1267da700 1 -- 172.21.15.29:0/711287557 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c009370 con 0x7fd120156680 2024-01-24T14:16:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd1267da700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd120156680 0x7fd1201643e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7fd11800c7e0 tx=0x7fd118007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd111ffb700 1 -- 172.21.15.29:0/711287557 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118005c70 con 0x7fd120156680 2024-01-24T14:16:06.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd111ffb700 1 -- 172.21.15.29:0/711287557 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd118005dd0 con 0x7fd120156680 2024-01-24T14:16:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd111ffb700 1 -- 172.21.15.29:0/711287557 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118007340 con 0x7fd120156680 2024-01-24T14:16:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.845+0000 7fd12823d700 1 -- 172.21.15.29:0/711287557 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1200ab4c0 con 0x7fd120156680 2024-01-24T14:16:06.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.846+0000 7fd12823d700 1 -- 172.21.15.29:0/711287557 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1200aba80 con 0x7fd120156680 2024-01-24T14:16:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.847+0000 7fd12823d700 1 -- 172.21.15.29:0/711287557 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1200475e0 con 0x7fd120156680 2024-01-24T14:16:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.847+0000 7fd111ffb700 1 -- 172.21.15.29:0/711287557 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd1180076b0 con 0x7fd120156680 2024-01-24T14:16:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.847+0000 7fd111ffb700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd10405f270 0x7fd104061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.847+0000 7fd111ffb700 1 -- 172.21.15.29:0/711287557 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6290+0+0 (secure 0 0 0) 0x7fd11807a270 con 0x7fd120156680 2024-01-24T14:16:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.848+0000 7fd125fd9700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd10405f270 0x7fd104061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.848+0000 7fd125fd9700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd10405f270 0x7fd104061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fd11c009130 tx=0x7fd11c017000).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:06.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:06.852+0000 7fd111ffb700 1 -- 172.21.15.29:0/711287557 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd11804b310 con 0x7fd120156680 2024-01-24T14:16:07.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.036+0000 7fd12823d700 1 -- 172.21.15.29:0/711287557 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fd12006a690 con 0x7fd120156680 2024-01-24T14:16:07.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.907+0000 7fd111ffb700 1 -- 172.21.15.29:0/711287557 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v134) v1 ==== 152+0+0 (secure 0 0 0) 0x7fd12006a690 con 0x7fd120156680 2024-01-24T14:16:07.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.911+0000 7fd12823d700 1 -- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd10405f270 msgr2=0x7fd104061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:07.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.911+0000 7fd12823d700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd10405f270 0x7fd104061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fd11c009130 tx=0x7fd11c017000).stop 2024-01-24T14:16:07.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.911+0000 7fd12823d700 1 -- 172.21.15.29:0/711287557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd120156680 msgr2=0x7fd1201643e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:07.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.911+0000 7fd12823d700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd120156680 0x7fd1201643e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7fd11800c7e0 tx=0x7fd118007dd0).stop 2024-01-24T14:16:07.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.911+0000 7fd12823d700 1 -- 172.21.15.29:0/711287557 shutdown_connections 2024-01-24T14:16:07.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.911+0000 7fd12823d700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd10405f270 0x7fd104061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:07.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.911+0000 7fd12823d700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd120156680 0x7fd1201643e0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:07.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.911+0000 7fd12823d700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12014f8f0 0x7fd1200acb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:07.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.911+0000 7fd12823d700 1 --2- 172.21.15.29:0/711287557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd12014fcd0 0x7fd12014c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:07.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.911+0000 7fd12823d700 1 -- 172.21.15.29:0/711287557 >> 172.21.15.29:0/711287557 conn(0x7fd120067b20 msgr2=0x7fd1200a9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:07.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.912+0000 7fd12823d700 1 -- 172.21.15.29:0/711287557 shutdown_connections 2024-01-24T14:16:07.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:07.917+0000 7fd12823d700 1 -- 172.21.15.29:0/711287557 wait complete. 2024-01-24T14:16:07.920 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-01-24T14:16:07.934 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-24T14:16:08.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.075+0000 7fb2b57e7700 1 Processor -- start 2024-01-24T14:16:08.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.075+0000 7fb2b57e7700 1 -- start start 2024-01-24T14:16:08.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.076+0000 7fb2b57e7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb2b0066560 0x7fb2b0066970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.076+0000 7fb2b57e7700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b0063f60 con 0x7fb2b0153dd0 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.076+0000 7fb2b57e7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b0154a60 con 0x7fb2b0063ad0 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.076+0000 7fb2b57e7700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b0154d40 con 0x7fb2b0066560 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.076+0000 7fb2aeffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb2b0153dd0 0x7fb2b01541b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52390/0 (socket says 172.21.15.29:52390) 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.077+0000 7fb2aeffd700 1 -- 172.21.15.29:0/1329469965 learned_addr learned my addr 172.21.15.29:0/1329469965 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.077+0000 7fb2af7fe700 1 --2- 172.21.15.29:0/1329469965 >> v2:172.21.15.29:3300/0 conn(0x7fb2b0066560 0x7fb2b0066970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.077+0000 7fb2adffb700 1 -- 172.21.15.29:0/1329469965 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850105091 0 0) 0x7fb2b0063f60 con 0x7fb2b0153dd0 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.077+0000 7fb2adffb700 1 -- 172.21.15.29:0/1329469965 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb29c0033f0 con 0x7fb2b0153dd0 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.077+0000 7fb2af7fe700 1 -- 172.21.15.29:0/1329469965 >> v1:172.21.15.163:6789/0 conn(0x7fb2b0063ad0 legacy=0x7fb2b00c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.077+0000 7fb2af7fe700 1 -- 172.21.15.29:0/1329469965 >> v1:172.21.15.29:6789/0 conn(0x7fb2b0153dd0 legacy=0x7fb2b01541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.077+0000 7fb2af7fe700 1 -- 172.21.15.29:0/1329469965 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b0155020 con 0x7fb2b0066560 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.077+0000 7fb2af7fe700 1 --2- 172.21.15.29:0/1329469965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b0066560 0x7fb2b0066970 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7fb2a8009e60 tx=0x7fb2a8009780).ready entity=mon.0 client_cookie=cb6ea514ec78d83b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:08.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.077+0000 7fb2adffb700 1 -- 172.21.15.29:0/1329469965 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a800e070 con 0x7fb2b0066560 2024-01-24T14:16:08.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.077+0000 7fb2adffb700 1 -- 172.21.15.29:0/1329469965 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a800bb20 con 0x7fb2b0066560 2024-01-24T14:16:08.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.077+0000 7fb2adffb700 1 -- 172.21.15.29:0/1329469965 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a800bea0 con 0x7fb2b0066560 2024-01-24T14:16:08.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.078+0000 7fb2b57e7700 1 -- 172.21.15.29:0/1329469965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b0066560 msgr2=0x7fb2b0066970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:08.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.078+0000 7fb2b57e7700 1 --2- 172.21.15.29:0/1329469965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b0066560 0x7fb2b0066970 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7fb2a8009e60 tx=0x7fb2a8009780).stop 2024-01-24T14:16:08.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.078+0000 7fb2b57e7700 1 -- 172.21.15.29:0/1329469965 shutdown_connections 2024-01-24T14:16:08.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.078+0000 7fb2b57e7700 1 --2- 172.21.15.29:0/1329469965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b0066560 0x7fb2b0066970 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:08.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.078+0000 7fb2b57e7700 1 -- 172.21.15.29:0/1329469965 >> 172.21.15.29:0/1329469965 conn(0x7fb2b005ec80 msgr2=0x7fb2b00c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:08.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.078+0000 7fb2b57e7700 1 -- 172.21.15.29:0/1329469965 shutdown_connections 2024-01-24T14:16:08.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.078+0000 7fb2b57e7700 1 -- 172.21.15.29:0/1329469965 wait complete. 2024-01-24T14:16:08.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.079+0000 7fb2b57e7700 1 Processor -- start 2024-01-24T14:16:08.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.079+0000 7fb2b57e7700 1 -- start start 2024-01-24T14:16:08.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.080+0000 7fb2b57e7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2b0063ad0 0x7fb2b0164660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:08.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.080+0000 7fb2b57e7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b0066560 0x7fb2b01685b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:08.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.080+0000 7fb2b57e7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2b0153dd0 0x7fb2b01673a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:08.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.080+0000 7fb2b57e7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b00acbe0 con 0x7fb2b0066560 2024-01-24T14:16:08.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.080+0000 7fb2b57e7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b0164b50 con 0x7fb2b0153dd0 2024-01-24T14:16:08.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.080+0000 7fb2b57e7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b00abfd0 con 0x7fb2b0063ad0 2024-01-24T14:16:08.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.080+0000 7fb2ae7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b0066560 0x7fb2b01685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:08.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.080+0000 7fb2ae7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b0066560 0x7fb2b01685b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39328/0 (socket says 172.21.15.29:39328) 2024-01-24T14:16:08.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.080+0000 7fb2ae7fc700 1 -- 172.21.15.29:0/2442014787 learned_addr learned my addr 172.21.15.29:0/2442014787 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:08.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.080+0000 7fb2aeffd700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2b0063ad0 0x7fb2b0164660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:08.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.080+0000 7fb2af7fe700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2b0153dd0 0x7fb2b01673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:08.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.081+0000 7fb2ae7fc700 1 -- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2b0063ad0 msgr2=0x7fb2b0164660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:08.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.081+0000 7fb2ae7fc700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2b0063ad0 0x7fb2b0164660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:08.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.081+0000 7fb2ae7fc700 1 -- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2b0153dd0 msgr2=0x7fb2b01673a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:08.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.081+0000 7fb2ae7fc700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2b0153dd0 0x7fb2b01673a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:08.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.081+0000 7fb2ae7fc700 1 -- 172.21.15.29:0/2442014787 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a8009a20 con 0x7fb2b0066560 2024-01-24T14:16:08.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.081+0000 7fb2ae7fc700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b0066560 0x7fb2b01685b0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7fb29800c440 tx=0x7fb298007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:08.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.081+0000 7fb2a6ffd700 1 -- 172.21.15.29:0/2442014787 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298012400 con 0x7fb2b0066560 2024-01-24T14:16:08.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.081+0000 7fb2a6ffd700 1 -- 172.21.15.29:0/2442014787 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb298012580 con 0x7fb2b0066560 2024-01-24T14:16:08.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.081+0000 7fb2b57e7700 1 -- 172.21.15.29:0/2442014787 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b00ac2b0 con 0x7fb2b0066560 2024-01-24T14:16:08.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.081+0000 7fb2a6ffd700 1 -- 172.21.15.29:0/2442014787 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298012940 con 0x7fb2b0066560 2024-01-24T14:16:08.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.081+0000 7fb2b57e7700 1 -- 172.21.15.29:0/2442014787 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b00ac870 con 0x7fb2b0066560 2024-01-24T14:16:08.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.082+0000 7fb2b57e7700 1 -- 172.21.15.29:0/2442014787 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b00acd50 con 0x7fb2b0066560 2024-01-24T14:16:08.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.082+0000 7fb2a6ffd700 1 -- 172.21.15.29:0/2442014787 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb298012d30 con 0x7fb2b0066560 2024-01-24T14:16:08.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.083+0000 7fb2a6ffd700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb29005f270 0x7fb290061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:08.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.083+0000 7fb2a6ffd700 1 -- 172.21.15.29:0/2442014787 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6660+0+0 (secure 0 0 0) 0x7fb298079950 con 0x7fb2b0066560 2024-01-24T14:16:08.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.083+0000 7fb2aeffd700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb29005f270 0x7fb290061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:08.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.084+0000 7fb2aeffd700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb29005f270 0x7fb290061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb2a0007fd0 tx=0x7fb2a0009040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:08.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.088+0000 7fb2a6ffd700 1 -- 172.21.15.29:0/2442014787 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb298048b00 con 0x7fb2b0066560 2024-01-24T14:16:08.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.269+0000 7fb2b57e7700 1 -- 172.21.15.29:0/2442014787 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb2b00475e0 con 0x7fb2b0066560 2024-01-24T14:16:08.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.914+0000 7fb2a6ffd700 1 -- 172.21.15.29:0/2442014787 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v135) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb29804e360 con 0x7fb2b0066560 2024-01-24T14:16:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.924+0000 7fb2b57e7700 1 -- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb29005f270 msgr2=0x7fb290061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.924+0000 7fb2b57e7700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb29005f270 0x7fb290061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb2a0007fd0 tx=0x7fb2a0009040).stop 2024-01-24T14:16:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.924+0000 7fb2b57e7700 1 -- 172.21.15.29:0/2442014787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b0066560 msgr2=0x7fb2b01685b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.924+0000 7fb2b57e7700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b0066560 0x7fb2b01685b0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7fb29800c440 tx=0x7fb298007de0).stop 2024-01-24T14:16:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.924+0000 7fb2b57e7700 1 -- 172.21.15.29:0/2442014787 shutdown_connections 2024-01-24T14:16:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.924+0000 7fb2b57e7700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2b0153dd0 0x7fb2b01673a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.924+0000 7fb2b57e7700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2b0063ad0 0x7fb2b0164660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.924+0000 7fb2b57e7700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb29005f270 0x7fb290061730 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0x7fb2a0007fd0 tx=0x7fb2a0009040).stop 2024-01-24T14:16:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.924+0000 7fb2b57e7700 1 --2- 172.21.15.29:0/2442014787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b0066560 0x7fb2b01685b0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:08.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.925+0000 7fb2b57e7700 1 -- 172.21.15.29:0/2442014787 >> 172.21.15.29:0/2442014787 conn(0x7fb2b005ec80 msgr2=0x7fb2b0157050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:08.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.926+0000 7fb2b57e7700 1 -- 172.21.15.29:0/2442014787 shutdown_connections 2024-01-24T14:16:08.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:08.930+0000 7fb2b57e7700 1 -- 172.21.15.29:0/2442014787 wait complete. 2024-01-24T14:16:08.932 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-01-24T14:16:08.953 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-24T14:16:09.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.088+0000 7f2f345a8700 1 Processor -- start 2024-01-24T14:16:09.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.089+0000 7f2f345a8700 1 -- start start 2024-01-24T14:16:09.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f345a8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2f2c157380 0x7f2f2c063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f345a8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c157760 0x7f2f2c157c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f345a8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2c0627b0 con 0x7f2f2c158110 2024-01-24T14:16:09.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f345a8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2c062d70 con 0x7f2f2c157380 2024-01-24T14:16:09.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f345a8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c1586e0 con 0x7f2f2c157760 2024-01-24T14:16:09.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f32344700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2f2c157380 0x7f2f2c063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f32344700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2f2c157380 0x7f2f2c063200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39334/0 (socket says 172.21.15.29:39334) 2024-01-24T14:16:09.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f32344700 1 -- 172.21.15.29:0/1934796647 learned_addr learned my addr 172.21.15.29:0/1934796647 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f31b43700 1 --2- 172.21.15.29:0/1934796647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c157760 0x7f2f2c157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f32344700 1 -- 172.21.15.29:0/1934796647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c157760 msgr2=0x7f2f2c157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f32344700 1 --2- 172.21.15.29:0/1934796647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c157760 0x7f2f2c157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f32344700 1 -- 172.21.15.29:0/1934796647 >> v1:172.21.15.163:6789/0 conn(0x7f2f2c158110 legacy=0x7f2f2c064b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f32344700 1 -- 172.21.15.29:0/1934796647 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c1589c0 con 0x7f2f2c157380 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.090+0000 7f2f32344700 1 --2- 172.21.15.29:0/1934796647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c157380 0x7f2f2c063200 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f2f20006fd0 tx=0x7f2f200090f0).ready entity=mon.0 client_cookie=768f78813024dbcb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.091+0000 7f2f31342700 1 -- 172.21.15.29:0/1934796647 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2000b580 con 0x7f2f2c157380 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.091+0000 7f2f31342700 1 -- 172.21.15.29:0/1934796647 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2000b6e0 con 0x7f2f2c157380 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.091+0000 7f2f31342700 1 -- 172.21.15.29:0/1934796647 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2000ba80 con 0x7f2f2c157380 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.091+0000 7f2f345a8700 1 -- 172.21.15.29:0/1934796647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c157380 msgr2=0x7f2f2c063200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.091+0000 7f2f345a8700 1 --2- 172.21.15.29:0/1934796647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c157380 0x7f2f2c063200 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f2f20006fd0 tx=0x7f2f200090f0).stop 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.091+0000 7f2f345a8700 1 -- 172.21.15.29:0/1934796647 shutdown_connections 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.091+0000 7f2f345a8700 1 --2- 172.21.15.29:0/1934796647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c157380 0x7f2f2c063200 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.091+0000 7f2f345a8700 1 --2- 172.21.15.29:0/1934796647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c157760 0x7f2f2c157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.091+0000 7f2f345a8700 1 -- 172.21.15.29:0/1934796647 >> 172.21.15.29:0/1934796647 conn(0x7f2f2c0c7260 msgr2=0x7f2f2c0c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:09.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.091+0000 7f2f345a8700 1 -- 172.21.15.29:0/1934796647 shutdown_connections 2024-01-24T14:16:09.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.091+0000 7f2f345a8700 1 -- 172.21.15.29:0/1934796647 wait complete. 2024-01-24T14:16:09.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.093+0000 7f2f345a8700 1 Processor -- start 2024-01-24T14:16:09.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.093+0000 7f2f345a8700 1 -- start start 2024-01-24T14:16:09.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.093+0000 7f2f345a8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2c157380 0x7f2f2c142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.093+0000 7f2f345a8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c157760 0x7f2f2c0aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.093+0000 7f2f345a8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c158110 0x7f2f2c0ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.093+0000 7f2f345a8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0ab730 con 0x7f2f2c158110 2024-01-24T14:16:09.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.093+0000 7f2f345a8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c168d10 con 0x7f2f2c157380 2024-01-24T14:16:09.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.093+0000 7f2f345a8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c1536e0 con 0x7f2f2c157760 2024-01-24T14:16:09.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f32b45700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c158110 0x7f2f2c0ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f32b45700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c158110 0x7f2f2c0ab240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39342/0 (socket says 172.21.15.29:39342) 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f32b45700 1 -- 172.21.15.29:0/2705465319 learned_addr learned my addr 172.21.15.29:0/2705465319 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f32344700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2c157380 0x7f2f2c142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f31b43700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c157760 0x7f2f2c0aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f32b45700 1 -- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c157760 msgr2=0x7f2f2c0aad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f32b45700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c157760 0x7f2f2c0aad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f32b45700 1 -- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2c157380 msgr2=0x7f2f2c142b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f32b45700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2c157380 0x7f2f2c142b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f32b45700 1 -- 172.21.15.29:0/2705465319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f20009370 con 0x7f2f2c158110 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f31b43700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c157760 0x7f2f2c0aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f32b45700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c158110 0x7f2f2c0ab240 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f2f2400c7c0 tx=0x7f2f24001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f1e7fc700 1 -- 172.21.15.29:0/2705465319 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2400e4a0 con 0x7f2f2c158110 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f1e7fc700 1 -- 172.21.15.29:0/2705465319 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2400e600 con 0x7f2f2c158110 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f1e7fc700 1 -- 172.21.15.29:0/2705465319 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2400e980 con 0x7f2f2c158110 2024-01-24T14:16:09.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f345a8700 1 -- 172.21.15.29:0/2705465319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c152db0 con 0x7f2f2c158110 2024-01-24T14:16:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.094+0000 7f2f345a8700 1 -- 172.21.15.29:0/2705465319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c153370 con 0x7f2f2c158110 2024-01-24T14:16:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.095+0000 7f2f345a8700 1 -- 172.21.15.29:0/2705465319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c153890 con 0x7f2f2c158110 2024-01-24T14:16:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.096+0000 7f2f1e7fc700 1 -- 172.21.15.29:0/2705465319 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2f24012400 con 0x7f2f2c158110 2024-01-24T14:16:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.096+0000 7f2f1e7fc700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2f1005f270 0x7f2f10061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.096+0000 7f2f1e7fc700 1 -- 172.21.15.29:0/2705465319 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 7042+0+0 (secure 0 0 0) 0x7f2f24079d30 con 0x7f2f2c158110 2024-01-24T14:16:09.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.097+0000 7f2f32344700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2f1005f270 0x7f2f10061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.097+0000 7f2f32344700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2f1005f270 0x7f2f10061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2f20009fd0 tx=0x7f2f20017000).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:09.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.101+0000 7f2f1e7fc700 1 -- 172.21.15.29:0/2705465319 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2f24048b00 con 0x7f2f2c158110 2024-01-24T14:16:09.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.330+0000 7f2f345a8700 1 -- 172.21.15.29:0/2705465319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2f2c0475e0 con 0x7f2f2c158110 2024-01-24T14:16:09.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.336+0000 7f2f1e7fc700 1 -- 172.21.15.29:0/2705465319 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v55) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2f24017030 con 0x7f2f2c158110 2024-01-24T14:16:09.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.339+0000 7f2f345a8700 1 -- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2f1005f270 msgr2=0x7f2f10061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.339+0000 7f2f345a8700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2f1005f270 0x7f2f10061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2f20009fd0 tx=0x7f2f20017000).stop 2024-01-24T14:16:09.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.340+0000 7f2f345a8700 1 -- 172.21.15.29:0/2705465319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c158110 msgr2=0x7f2f2c0ab240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.340+0000 7f2f345a8700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c158110 0x7f2f2c0ab240 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f2f2400c7c0 tx=0x7f2f24001510).stop 2024-01-24T14:16:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.340+0000 7f2f345a8700 1 -- 172.21.15.29:0/2705465319 shutdown_connections 2024-01-24T14:16:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.340+0000 7f2f345a8700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2f1005f270 0x7f2f10061730 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0x7f2f20009fd0 tx=0x7f2f20017000).stop 2024-01-24T14:16:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.340+0000 7f2f345a8700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f2c158110 0x7f2f2c0ab240 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.340+0000 7f2f345a8700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2c157380 0x7f2f2c142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.340+0000 7f2f345a8700 1 --2- 172.21.15.29:0/2705465319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2c157760 0x7f2f2c0aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.340+0000 7f2f345a8700 1 -- 172.21.15.29:0/2705465319 >> 172.21.15.29:0/2705465319 conn(0x7f2f2c0c7260 msgr2=0x7f2f2c156540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.340+0000 7f2f345a8700 1 -- 172.21.15.29:0/2705465319 shutdown_connections 2024-01-24T14:16:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.340+0000 7f2f345a8700 1 -- 172.21.15.29:0/2705465319 wait complete. 2024-01-24T14:16:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 16 and data pool 17 2024-01-24T14:16:09.367 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:16:09.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.499+0000 7f8ccda2e700 1 Processor -- start 2024-01-24T14:16:09.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.499+0000 7f8ccda2e700 1 -- start start 2024-01-24T14:16:09.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8ccda2e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8cc8154370 0x7f8cc80c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8ccda2e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8cc8154750 0x7f8cc80631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8ccda2e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc80aa6c0 con 0x7f8cc80aa0e0 2024-01-24T14:16:09.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8ccda2e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc8154ba0 con 0x7f8cc8154750 2024-01-24T14:16:09.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8ccda2e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc8064240 con 0x7f8cc8154370 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8cc77fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8cc8154750 0x7f8cc80631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8cc77fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8cc8154750 0x7f8cc80631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39352/0 (socket says 172.21.15.29:39352) 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8cc77fe700 1 -- 172.21.15.29:0/326742997 learned_addr learned my addr 172.21.15.29:0/326742997 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8cc6ffd700 1 --2- 172.21.15.29:0/326742997 >> v2:172.21.15.163:3300/0 conn(0x7f8cc8154370 0x7f8cc80c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8cc77fe700 1 -- 172.21.15.29:0/326742997 >> v2:172.21.15.163:3300/0 conn(0x7f8cc8154370 msgr2=0x7f8cc80c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8cc77fe700 1 --2- 172.21.15.29:0/326742997 >> v2:172.21.15.163:3300/0 conn(0x7f8cc8154370 0x7f8cc80c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8cc77fe700 1 -- 172.21.15.29:0/326742997 >> v1:172.21.15.163:6789/0 conn(0x7f8cc80aa0e0 legacy=0x7f8cc80ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.500+0000 7f8cc77fe700 1 -- 172.21.15.29:0/326742997 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc80cb5c0 con 0x7f8cc8154750 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.501+0000 7f8cc77fe700 1 --2- 172.21.15.29:0/326742997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cc8154750 0x7f8cc80631d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f8cc8084050 tx=0x7f8cc0009840).ready entity=mon.0 client_cookie=cedbfe5d5ec4c434 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.501+0000 7f8cc67fc700 1 -- 172.21.15.29:0/326742997 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000e070 con 0x7f8cc8154750 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.501+0000 7f8cc67fc700 1 -- 172.21.15.29:0/326742997 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc000ba50 con 0x7f8cc8154750 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.501+0000 7f8cc67fc700 1 -- 172.21.15.29:0/326742997 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000bdd0 con 0x7f8cc8154750 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.501+0000 7f8ccda2e700 1 -- 172.21.15.29:0/326742997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cc8154750 msgr2=0x7f8cc80631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.501+0000 7f8ccda2e700 1 --2- 172.21.15.29:0/326742997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cc8154750 0x7f8cc80631d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f8cc8084050 tx=0x7f8cc0009840).stop 2024-01-24T14:16:09.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.501+0000 7f8ccda2e700 1 -- 172.21.15.29:0/326742997 shutdown_connections 2024-01-24T14:16:09.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.501+0000 7f8ccda2e700 1 --2- 172.21.15.29:0/326742997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cc8154750 0x7f8cc80631d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.501+0000 7f8ccda2e700 1 --2- 172.21.15.29:0/326742997 >> v2:172.21.15.163:3300/0 conn(0x7f8cc8154370 0x7f8cc80c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.501+0000 7f8ccda2e700 1 -- 172.21.15.29:0/326742997 >> 172.21.15.29:0/326742997 conn(0x7f8cc805ec80 msgr2=0x7f8cc80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:09.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.501+0000 7f8ccda2e700 1 -- 172.21.15.29:0/326742997 shutdown_connections 2024-01-24T14:16:09.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.501+0000 7f8ccda2e700 1 -- 172.21.15.29:0/326742997 wait complete. 2024-01-24T14:16:09.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.503+0000 7f8ccda2e700 1 Processor -- start 2024-01-24T14:16:09.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.503+0000 7f8ccda2e700 1 -- start start 2024-01-24T14:16:09.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.503+0000 7f8ccda2e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cc80aa0e0 0x7f8cc8144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.503+0000 7f8ccda2e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cc8154370 0x7f8cc8151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.503+0000 7f8ccda2e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cc8154750 0x7f8cc8151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8ccda2e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc8152450 con 0x7f8cc8154370 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8ccda2e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc8144530 con 0x7f8cc8154750 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8ccda2e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80c2bf0 con 0x7f8cc80aa0e0 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cbe7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cc8154370 0x7f8cc8151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cbe7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cc8154370 0x7f8cc8151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39362/0 (socket says 172.21.15.29:39362) 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cbe7fc700 1 -- 172.21.15.29:0/4272469932 learned_addr learned my addr 172.21.15.29:0/4272469932 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cc77fe700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cc8154750 0x7f8cc8151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cc6ffd700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cc80aa0e0 0x7f8cc8144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cbe7fc700 1 -- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cc80aa0e0 msgr2=0x7f8cc8144040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cbe7fc700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cc80aa0e0 0x7f8cc8144040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cbe7fc700 1 -- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cc8154750 msgr2=0x7f8cc8151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cbe7fc700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cc8154750 0x7f8cc8151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cbe7fc700 1 -- 172.21.15.29:0/4272469932 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0009ae0 con 0x7f8cc8154370 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cbe7fc700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cc8154370 0x7f8cc8151a70 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7f8cb000c4c0 tx=0x7f8cb0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:09.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cbf7fe700 1 -- 172.21.15.29:0/4272469932 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb000ce70 con 0x7f8cc8154370 2024-01-24T14:16:09.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cbf7fe700 1 -- 172.21.15.29:0/4272469932 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb0001e90 con 0x7f8cc8154370 2024-01-24T14:16:09.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8ccda2e700 1 -- 172.21.15.29:0/4272469932 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc814cdf0 con 0x7f8cc8154370 2024-01-24T14:16:09.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.504+0000 7f8cbf7fe700 1 -- 172.21.15.29:0/4272469932 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0007250 con 0x7f8cc8154370 2024-01-24T14:16:09.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.505+0000 7f8ccda2e700 1 -- 172.21.15.29:0/4272469932 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc814d3b0 con 0x7f8cc8154370 2024-01-24T14:16:09.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.506+0000 7f8ccda2e700 1 -- 172.21.15.29:0/4272469932 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc80475e0 con 0x7f8cc8154370 2024-01-24T14:16:09.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.506+0000 7f8cbf7fe700 1 -- 172.21.15.29:0/4272469932 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8cb00066a0 con 0x7f8cc8154370 2024-01-24T14:16:09.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.506+0000 7f8cbf7fe700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8ca805f270 0x7f8ca8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.506+0000 7f8cbf7fe700 1 -- 172.21.15.29:0/4272469932 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f8cb007a070 con 0x7f8cc8154370 2024-01-24T14:16:09.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.507+0000 7f8cc6ffd700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8ca805f270 0x7f8ca8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.507+0000 7f8cc6ffd700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8ca805f270 0x7f8ca8061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8cb8006fd0 tx=0x7f8cb8008040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:09.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.511+0000 7f8cbf7fe700 1 -- 172.21.15.29:0/4272469932 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8cb004adc0 con 0x7f8cc8154370 2024-01-24T14:16:09.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.691+0000 7f8ccda2e700 1 -- 172.21.15.29:0/4272469932 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cc80cb8a0 con 0x7f8cc8154370 2024-01-24T14:16:09.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.692+0000 7f8cbf7fe700 1 -- 172.21.15.29:0/4272469932 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+18198 (secure 0 0 0) 0x7f8cb004e3e0 con 0x7f8cc8154370 2024-01-24T14:16:09.693 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:16:09.693 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":136,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:16:09.332007+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:16:07.075847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-24T14:16:08.308597+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:16:09.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.694+0000 7f8ccda2e700 1 -- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8ca805f270 msgr2=0x7f8ca8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.695+0000 7f8ccda2e700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8ca805f270 0x7f8ca8061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8cb8006fd0 tx=0x7f8cb8008040).stop 2024-01-24T14:16:09.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.695+0000 7f8ccda2e700 1 -- 172.21.15.29:0/4272469932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cc8154370 msgr2=0x7f8cc8151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.695+0000 7f8ccda2e700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cc8154370 0x7f8cc8151a70 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7f8cb000c4c0 tx=0x7f8cb0007e00).stop 2024-01-24T14:16:09.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.695+0000 7f8ccda2e700 1 -- 172.21.15.29:0/4272469932 shutdown_connections 2024-01-24T14:16:09.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.695+0000 7f8ccda2e700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cc8154750 0x7f8cc8151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.695+0000 7f8ccda2e700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cc80aa0e0 0x7f8cc8144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.695+0000 7f8ccda2e700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8ca805f270 0x7f8ca8061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.695+0000 7f8ccda2e700 1 --2- 172.21.15.29:0/4272469932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cc8154370 0x7f8cc8151a70 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.695+0000 7f8ccda2e700 1 -- 172.21.15.29:0/4272469932 >> 172.21.15.29:0/4272469932 conn(0x7f8cc805ec80 msgr2=0x7f8cc80c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:09.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.695+0000 7f8ccda2e700 1 -- 172.21.15.29:0/4272469932 shutdown_connections 2024-01-24T14:16:09.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.695+0000 7f8ccda2e700 1 -- 172.21.15.29:0/4272469932 wait complete. 2024-01-24T14:16:09.709 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:16:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.839+0000 7f94bb0f4700 1 Processor -- start 2024-01-24T14:16:09.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.839+0000 7f94bb0f4700 1 -- start start 2024-01-24T14:16:09.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94bb0f4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b4066c00 0x7f94b40c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94bb0f4700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b4068360 con 0x7f94b4134cc0 2024-01-24T14:16:09.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94bb0f4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b40c3820 con 0x7f94b41350a0 2024-01-24T14:16:09.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94bb0f4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b4067a30 con 0x7f94b4066c00 2024-01-24T14:16:09.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b8e90700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f94b4134cc0 0x7f94b40664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52402/0 (socket says 172.21.15.29:52402) 2024-01-24T14:16:09.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b8e90700 1 -- 172.21.15.29:0/1884554310 learned_addr learned my addr 172.21.15.29:0/1884554310 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:09.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b3fff700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b4066c00 0x7f94b40c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606094870 0 0) 0x7f94b4068360 con 0x7f94b4134cc0 2024-01-24T14:16:09.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a40033f0 con 0x7f94b4134cc0 2024-01-24T14:16:09.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3420796670 0 0) 0x7f94a40033f0 con 0x7f94b4134cc0 2024-01-24T14:16:09.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b4066c00 msgr2=0x7f94b40c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:09.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b37fe700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b4066c00 0x7f94b40c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 >> v1:172.21.15.163:6789/0 conn(0x7f94b41350a0 legacy=0x7f94b415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4067d10 con 0x7f94b4134cc0 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f94a8001470 con 0x7f94b4134cc0 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 >> v1:172.21.15.29:6789/0 conn(0x7f94b4134cc0 legacy=0x7f94b40664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b37fe700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94a4003910 0x7f94a4003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b37fe700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b4066c00 0x7f94a4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.840+0000 7f94b37fe700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4004b80 0x7f94a4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b4067a30 con 0x7f94a4003910 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a400b990 con 0x7f94a4004b80 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a400bb20 con 0x7f94b4066c00 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b8e90700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94a4003910 0x7f94a4003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b3fff700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b4066c00 0x7f94a4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b9691700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4004b80 0x7f94a4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b8e90700 1 -- 172.21.15.29:0/1884554310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b4066c00 msgr2=0x7f94a4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b8e90700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b4066c00 0x7f94a4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b8e90700 1 -- 172.21.15.29:0/1884554310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4004b80 msgr2=0x7f94a4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b8e90700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4004b80 0x7f94a4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b8e90700 1 -- 172.21.15.29:0/1884554310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a40033f0 con 0x7f94a4003910 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b9691700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4004b80 0x7f94a4005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b8e90700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94a4003910 0x7f94a4003d30 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f94a8007fc0 tx=0x7f94a8001d80).ready entity=mon.0 client_cookie=36c5ad19fcb912ff server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a800c9e0 con 0x7f94a4003910 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a800cb40 con 0x7f94a4003910 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.841+0000 7f94b37fe700 1 -- 172.21.15.29:0/1884554310 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a80070d0 con 0x7f94a4003910 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.842+0000 7f94bb0f4700 1 -- 172.21.15.29:0/1884554310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94a4003910 msgr2=0x7f94a4003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.842+0000 7f94bb0f4700 1 --2- 172.21.15.29:0/1884554310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94a4003910 0x7f94a4003d30 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f94a8007fc0 tx=0x7f94a8001d80).stop 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.842+0000 7f94b9691700 1 -- 172.21.15.29:0/1884554310 reap_dead start 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.842+0000 7f94bb0f4700 1 -- 172.21.15.29:0/1884554310 shutdown_connections 2024-01-24T14:16:09.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.842+0000 7f94bb0f4700 1 -- 172.21.15.29:0/1884554310 >> 172.21.15.29:0/1884554310 conn(0x7f94b40c8bb0 msgr2=0x7f94b40c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.842+0000 7f94bb0f4700 1 -- 172.21.15.29:0/1884554310 shutdown_connections 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.842+0000 7f94bb0f4700 1 -- 172.21.15.29:0/1884554310 wait complete. 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.843+0000 7f94bb0f4700 1 Processor -- start 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.844+0000 7f94bb0f4700 1 -- start start 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.844+0000 7f94bb0f4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4003910 0x7f94b4151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.844+0000 7f94bb0f4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b414f780 0x7f94b414fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.844+0000 7f94bb0f4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b4137850 0x7f94b4137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.844+0000 7f94bb0f4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b414be40 con 0x7f94b4137850 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.844+0000 7f94bb0f4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b41654f0 con 0x7f94a4003910 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.844+0000 7f94bb0f4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b41657d0 con 0x7f94b414f780 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.844+0000 7f94b9691700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b4137850 0x7f94b4137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b9691700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b4137850 0x7f94b4137cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39382/0 (socket says 172.21.15.29:39382) 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b9691700 1 -- 172.21.15.29:0/670769255 learned_addr learned my addr 172.21.15.29:0/670769255 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b8e90700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4003910 0x7f94b4151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b3fff700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b414f780 0x7f94b414fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b9691700 1 -- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b414f780 msgr2=0x7f94b414fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b9691700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b414f780 0x7f94b414fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b9691700 1 -- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4003910 msgr2=0x7f94b4151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:09.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b9691700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4003910 0x7f94b4151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:09.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b9691700 1 -- 172.21.15.29:0/670769255 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4165ab0 con 0x7f94b4137850 2024-01-24T14:16:09.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b9691700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b4137850 0x7f94b4137cd0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7f94ac000f80 tx=0x7f94ac009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:09.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b0ff9700 1 -- 172.21.15.29:0/670769255 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac010520 con 0x7f94b4137850 2024-01-24T14:16:09.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b0ff9700 1 -- 172.21.15.29:0/670769255 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac010680 con 0x7f94b4137850 2024-01-24T14:16:09.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94b0ff9700 1 -- 172.21.15.29:0/670769255 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac010a20 con 0x7f94b4137850 2024-01-24T14:16:09.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94bb0f4700 1 -- 172.21.15.29:0/670769255 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b4165d90 con 0x7f94b4137850 2024-01-24T14:16:09.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.845+0000 7f94bb0f4700 1 -- 172.21.15.29:0/670769255 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b41518b0 con 0x7f94b4137850 2024-01-24T14:16:09.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.846+0000 7f94bb0f4700 1 -- 172.21.15.29:0/670769255 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b40475e0 con 0x7f94b4137850 2024-01-24T14:16:09.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.847+0000 7f94b0ff9700 1 -- 172.21.15.29:0/670769255 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f94ac003070 con 0x7f94b4137850 2024-01-24T14:16:09.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.847+0000 7f94b0ff9700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f949805f270 0x7f9498061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:09.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.847+0000 7f94b0ff9700 1 -- 172.21.15.29:0/670769255 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f94ac07a290 con 0x7f94b4137850 2024-01-24T14:16:09.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.848+0000 7f94b8e90700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f949805f270 0x7f9498061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:09.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.848+0000 7f94b8e90700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f949805f270 0x7f9498061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f94a8007da0 tx=0x7f94a8015000).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:09.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:09.851+0000 7f94b0ff9700 1 -- 172.21.15.29:0/670769255 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f94ac04afe0 con 0x7f94b4137850 2024-01-24T14:16:10.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.036+0000 7f94bb0f4700 1 -- 172.21.15.29:0/670769255 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94b406b190 con 0x7f94b4137850 2024-01-24T14:16:10.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.037+0000 7f94b0ff9700 1 -- 172.21.15.29:0/670769255 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+18198 (secure 0 0 0) 0x7f94ac04e600 con 0x7f94b4137850 2024-01-24T14:16:10.038 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:16:10.038 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":136,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:16:09.332007+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:16:07.075847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-24T14:16:08.308597+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:16:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.039+0000 7f94bb0f4700 1 -- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f949805f270 msgr2=0x7f9498061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.039+0000 7f94bb0f4700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f949805f270 0x7f9498061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f94a8007da0 tx=0x7f94a8015000).stop 2024-01-24T14:16:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.039+0000 7f94bb0f4700 1 -- 172.21.15.29:0/670769255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b4137850 msgr2=0x7f94b4137cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.039+0000 7f94bb0f4700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b4137850 0x7f94b4137cd0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7f94ac000f80 tx=0x7f94ac009040).stop 2024-01-24T14:16:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.039+0000 7f94bb0f4700 1 -- 172.21.15.29:0/670769255 shutdown_connections 2024-01-24T14:16:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.039+0000 7f94bb0f4700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f949805f270 0x7f9498061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:10.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.039+0000 7f94bb0f4700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94b4137850 0x7f94b4137cd0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.039+0000 7f94bb0f4700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4003910 0x7f94b4151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.039+0000 7f94bb0f4700 1 --2- 172.21.15.29:0/670769255 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94b414f780 0x7f94b414fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.039+0000 7f94bb0f4700 1 -- 172.21.15.29:0/670769255 >> 172.21.15.29:0/670769255 conn(0x7f94b40c8bb0 msgr2=0x7f94b4152230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.039+0000 7f94bb0f4700 1 -- 172.21.15.29:0/670769255 shutdown_connections 2024-01-24T14:16:10.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.039+0000 7f94bb0f4700 1 -- 172.21.15.29:0/670769255 wait complete. 2024-01-24T14:16:10.054 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-24T14:16:10.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.186+0000 7f914afb2700 1 Processor -- start 2024-01-24T14:16:10.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.186+0000 7f914afb2700 1 -- start start 2024-01-24T14:16:10.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.187+0000 7f914afb2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f91440664c0 0x7f9144066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:10.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.187+0000 7f914afb2700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91440686a0 con 0x7f9144134ea0 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.187+0000 7f914afb2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9144066e30 con 0x7f91440bdb30 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.187+0000 7f914afb2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91440c38e0 con 0x7f91440664c0 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.187+0000 7f9148d4e700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9144134ea0 0x7f91440c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52418/0 (socket says 172.21.15.29:52418) 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.187+0000 7f9148d4e700 1 -- 172.21.15.29:0/3581453259 learned_addr learned my addr 172.21.15.29:0/3581453259 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f9143fff700 1 --2- 172.21.15.29:0/3581453259 >> v2:172.21.15.163:3300/0 conn(0x7f91440664c0 0x7f9144066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176283548 0 0) 0x7f91440686a0 con 0x7f9144134ea0 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91340033f0 con 0x7f9144134ea0 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855192767 0 0) 0x7f9144066e30 con 0x7f91440bdb30 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91440686a0 con 0x7f91440bdb30 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2381765101 0 0) 0x7f91340033f0 con 0x7f9144134ea0 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 >> v2:172.21.15.163:3300/0 conn(0x7f91440664c0 msgr2=0x7f9144066940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 --2- 172.21.15.29:0/3581453259 >> v2:172.21.15.163:3300/0 conn(0x7f91440664c0 0x7f9144066940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:10.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 >> v1:172.21.15.163:6789/0 conn(0x7f91440bdb30 legacy=0x7f91440bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9144068b30 con 0x7f9144134ea0 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f9138001470 con 0x7f9144134ea0 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f9143fff700 1 --2- 172.21.15.29:0/3581453259 >> v2:172.21.15.163:3300/0 conn(0x7f91440664c0 0x7f9144066940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 >> v1:172.21.15.29:6789/0 conn(0x7f9144134ea0 legacy=0x7f91440c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 --2- 172.21.15.29:0/3581453259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9134003a70 0x7f9134003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 --2- 172.21.15.29:0/3581453259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9134004690 0x7f9134004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 --2- 172.21.15.29:0/3581453259 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9134005000 0x7f913400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c38e0 con 0x7f9134004690 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913400be80 con 0x7f9134003a70 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913400c040 con 0x7f9134005000 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f9143fff700 1 --2- 172.21.15.29:0/3581453259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9134004690 0x7f9134004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f9148d4e700 1 --2- 172.21.15.29:0/3581453259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9134003a70 0x7f9134003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.188+0000 7f914954f700 1 --2- 172.21.15.29:0/3581453259 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9134005000 0x7f913400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f9143fff700 1 -- 172.21.15.29:0/3581453259 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9134005000 msgr2=0x7f913400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f9143fff700 1 --2- 172.21.15.29:0/3581453259 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9134005000 0x7f913400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f9143fff700 1 -- 172.21.15.29:0/3581453259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9134003a70 msgr2=0x7f9134003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f9143fff700 1 --2- 172.21.15.29:0/3581453259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9134003a70 0x7f9134003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f9143fff700 1 -- 172.21.15.29:0/3581453259 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91340033f0 con 0x7f9134004690 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f914954f700 1 -- 172.21.15.29:0/3581453259 reap_dead start 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f9143fff700 1 --2- 172.21.15.29:0/3581453259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9134004690 0x7f9134004b10 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f9130006d80 tx=0x7f9130009060).ready entity=mon.0 client_cookie=795038d49e473141 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300095c0 con 0x7f9134004690 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913000e710 con 0x7f9134004690 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f91437fe700 1 -- 172.21.15.29:0/3581453259 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913000ea90 con 0x7f9134004690 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f914afb2700 1 -- 172.21.15.29:0/3581453259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9134004690 msgr2=0x7f9134004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f914afb2700 1 --2- 172.21.15.29:0/3581453259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9134004690 0x7f9134004b10 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f9130006d80 tx=0x7f9130009060).stop 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f914afb2700 1 -- 172.21.15.29:0/3581453259 shutdown_connections 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f914afb2700 1 --2- 172.21.15.29:0/3581453259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9134004690 0x7f9134004b10 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f914afb2700 1 -- 172.21.15.29:0/3581453259 >> 172.21.15.29:0/3581453259 conn(0x7f91440c8bd0 msgr2=0x7f91440c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f914afb2700 1 -- 172.21.15.29:0/3581453259 shutdown_connections 2024-01-24T14:16:10.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.189+0000 7f914afb2700 1 -- 172.21.15.29:0/3581453259 wait complete. 2024-01-24T14:16:10.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.191+0000 7f914afb2700 1 Processor -- start 2024-01-24T14:16:10.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.191+0000 7f914afb2700 1 -- start start 2024-01-24T14:16:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f914afb2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9134004690 0x7f9144162da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f914afb2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91440ae530 0x7f91440c1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f914afb2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91440c1820 0x7f91440aa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f914afb2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c2920 con 0x7f91440ae530 2024-01-24T14:16:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f914afb2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c1d10 con 0x7f91440c1820 2024-01-24T14:16:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f914afb2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c1ff0 con 0x7f9134004690 2024-01-24T14:16:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f9143fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91440ae530 0x7f91440c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f9143fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91440ae530 0x7f91440c1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39396/0 (socket says 172.21.15.29:39396) 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f9143fff700 1 -- 172.21.15.29:0/3491556983 learned_addr learned my addr 172.21.15.29:0/3491556983 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f9148d4e700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9134004690 0x7f9144162da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f914954f700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91440c1820 0x7f91440aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f9143fff700 1 -- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9134004690 msgr2=0x7f9144162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f9143fff700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9134004690 0x7f9144162da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f9143fff700 1 -- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91440c1820 msgr2=0x7f91440aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f9143fff700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91440c1820 0x7f91440aa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f9143fff700 1 -- 172.21.15.29:0/3491556983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440c22d0 con 0x7f91440ae530 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f914954f700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91440c1820 0x7f91440aa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.192+0000 7f9143fff700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91440ae530 0x7f91440c1330 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f913000e4b0 tx=0x7f913001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.193+0000 7f9140ff9700 1 -- 172.21.15.29:0/3491556983 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130000be0 con 0x7f91440ae530 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.193+0000 7f9140ff9700 1 -- 172.21.15.29:0/3491556983 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913000e880 con 0x7f91440ae530 2024-01-24T14:16:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.193+0000 7f914afb2700 1 -- 172.21.15.29:0/3491556983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91440c25b0 con 0x7f91440ae530 2024-01-24T14:16:10.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.193+0000 7f9140ff9700 1 -- 172.21.15.29:0/3491556983 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130024600 con 0x7f91440ae530 2024-01-24T14:16:10.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.193+0000 7f914afb2700 1 -- 172.21.15.29:0/3491556983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91440ab030 con 0x7f91440ae530 2024-01-24T14:16:10.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.194+0000 7f914afb2700 1 -- 172.21.15.29:0/3491556983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91440475e0 con 0x7f91440ae530 2024-01-24T14:16:10.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.194+0000 7f9140ff9700 1 -- 172.21.15.29:0/3491556983 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f91300249b0 con 0x7f91440ae530 2024-01-24T14:16:10.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.194+0000 7f9140ff9700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f912805f270 0x7f9128061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:10.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.194+0000 7f9140ff9700 1 -- 172.21.15.29:0/3491556983 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f913007bb70 con 0x7f91440ae530 2024-01-24T14:16:10.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.195+0000 7f9148d4e700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f912805f270 0x7f9128061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:10.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.195+0000 7f9148d4e700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f912805f270 0x7f9128061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f9138000ff0 tx=0x7f9138001e10).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:10.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.199+0000 7f9140ff9700 1 -- 172.21.15.29:0/3491556983 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f913004c970 con 0x7f91440ae530 2024-01-24T14:16:10.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:10.433+0000 7f914afb2700 1 -- 172.21.15.29:0/3491556983 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9144067a90 con 0x7f91440ae530 2024-01-24T14:16:11.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.348+0000 7f9140ff9700 1 -- 172.21.15.29:0/3491556983 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v58) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9130016020 con 0x7f91440ae530 2024-01-24T14:16:11.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.351+0000 7f914afb2700 1 -- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f912805f270 msgr2=0x7f9128061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:11.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.351+0000 7f914afb2700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f912805f270 0x7f9128061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f9138000ff0 tx=0x7f9138001e10).stop 2024-01-24T14:16:11.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.351+0000 7f914afb2700 1 -- 172.21.15.29:0/3491556983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91440ae530 msgr2=0x7f91440c1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:11.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.351+0000 7f914afb2700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91440ae530 0x7f91440c1330 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f913000e4b0 tx=0x7f913001a000).stop 2024-01-24T14:16:11.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.351+0000 7f914afb2700 1 -- 172.21.15.29:0/3491556983 shutdown_connections 2024-01-24T14:16:11.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.351+0000 7f914afb2700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91440c1820 0x7f91440aa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:11.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.351+0000 7f914afb2700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9134004690 0x7f9144162da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:11.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.351+0000 7f914afb2700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f912805f270 0x7f9128061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:11.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.351+0000 7f914afb2700 1 --2- 172.21.15.29:0/3491556983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91440ae530 0x7f91440c1330 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:11.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.351+0000 7f914afb2700 1 -- 172.21.15.29:0/3491556983 >> 172.21.15.29:0/3491556983 conn(0x7f91440c8bd0 msgr2=0x7f91440c5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:11.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.351+0000 7f914afb2700 1 -- 172.21.15.29:0/3491556983 shutdown_connections 2024-01-24T14:16:11.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.351+0000 7f914afb2700 1 -- 172.21.15.29:0/3491556983 wait complete. 2024-01-24T14:16:11.367 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-24T14:16:11.367 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-24T14:16:11.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.503+0000 7f2b9756b700 1 Processor -- start 2024-01-24T14:16:11.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.503+0000 7f2b9756b700 1 -- start start 2024-01-24T14:16:11.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b9756b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2b900664c0 0x7f2b90066950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b9756b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b900bdb30 0x7f2b900bdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b9756b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b90066e40 con 0x7f2b9013a5e0 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b9756b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b900640d0 con 0x7f2b900664c0 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b9756b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b900679f0 con 0x7f2b900bdb30 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b94b06700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2b900664c0 0x7f2b90066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b94b06700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2b900664c0 0x7f2b90066950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39402/0 (socket says 172.21.15.29:39402) 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b94b06700 1 -- 172.21.15.29:0/434967757 learned_addr learned my addr 172.21.15.29:0/434967757 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b95b08700 1 --2- 172.21.15.29:0/434967757 >> v2:172.21.15.163:3300/0 conn(0x7f2b900bdb30 0x7f2b900bdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b94b06700 1 -- 172.21.15.29:0/434967757 >> v2:172.21.15.163:3300/0 conn(0x7f2b900bdb30 msgr2=0x7f2b900bdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b94b06700 1 --2- 172.21.15.29:0/434967757 >> v2:172.21.15.163:3300/0 conn(0x7f2b900bdb30 0x7f2b900bdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b94b06700 1 -- 172.21.15.29:0/434967757 >> v1:172.21.15.29:6789/0 conn(0x7f2b9013a5e0 legacy=0x7f2b900639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b94b06700 1 -- 172.21.15.29:0/434967757 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90067cd0 con 0x7f2b900664c0 2024-01-24T14:16:11.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.504+0000 7f2b87fff700 1 -- 172.21.15.29:0/434967757 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402119168 0 0) 0x7f2b90066e40 con 0x7f2b9013a5e0 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.505+0000 7f2b94b06700 1 --2- 172.21.15.29:0/434967757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b900664c0 0x7f2b90066950 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7f2b900c9950 tx=0x7f2b800090f0).ready entity=mon.0 client_cookie=b3911e762a8bf57d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.505+0000 7f2b87fff700 1 -- 172.21.15.29:0/434967757 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8000b600 con 0x7f2b900664c0 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.505+0000 7f2b87fff700 1 -- 172.21.15.29:0/434967757 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8000b760 con 0x7f2b900664c0 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.505+0000 7f2b87fff700 1 -- 172.21.15.29:0/434967757 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8000bae0 con 0x7f2b900664c0 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.505+0000 7f2b9756b700 1 -- 172.21.15.29:0/434967757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b900664c0 msgr2=0x7f2b90066950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.505+0000 7f2b9756b700 1 --2- 172.21.15.29:0/434967757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b900664c0 0x7f2b90066950 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7f2b900c9950 tx=0x7f2b800090f0).stop 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.505+0000 7f2b9756b700 1 -- 172.21.15.29:0/434967757 shutdown_connections 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.505+0000 7f2b9756b700 1 --2- 172.21.15.29:0/434967757 >> v2:172.21.15.163:3300/0 conn(0x7f2b900bdb30 0x7f2b900bdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.505+0000 7f2b9756b700 1 --2- 172.21.15.29:0/434967757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b900664c0 0x7f2b90066950 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.505+0000 7f2b9756b700 1 -- 172.21.15.29:0/434967757 >> 172.21.15.29:0/434967757 conn(0x7f2b900c8bd0 msgr2=0x7f2b900c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.505+0000 7f2b9756b700 1 -- 172.21.15.29:0/434967757 shutdown_connections 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.505+0000 7f2b9756b700 1 -- 172.21.15.29:0/434967757 wait complete. 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.507+0000 7f2b9756b700 1 Processor -- start 2024-01-24T14:16:11.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.507+0000 7f2b9756b700 1 -- start start 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b9756b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b900664c0 0x7f2b901384d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b9756b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b900bdb30 0x7f2b900a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b9756b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b9013a5e0 0x7f2b90155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b9756b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900ae120 con 0x7f2b900bdb30 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b9756b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900ad4a0 con 0x7f2b9013a5e0 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b9756b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900ad780 con 0x7f2b900664c0 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b94b06700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b900bdb30 0x7f2b900a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b94b06700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b900bdb30 0x7f2b900a8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39414/0 (socket says 172.21.15.29:39414) 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b94b06700 1 -- 172.21.15.29:0/3947839640 learned_addr learned my addr 172.21.15.29:0/3947839640 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b94b06700 1 -- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b900664c0 msgr2=0x7f2b901384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b95b08700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b9013a5e0 0x7f2b90155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b95307700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b900664c0 0x7f2b901384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b94b06700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b900664c0 0x7f2b901384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:11.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b94b06700 1 -- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b9013a5e0 msgr2=0x7f2b90155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b94b06700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b9013a5e0 0x7f2b90155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b94b06700 1 -- 172.21.15.29:0/3947839640 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b80009370 con 0x7f2b900bdb30 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b95b08700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b9013a5e0 0x7f2b90155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b95307700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b900664c0 0x7f2b901384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.508+0000 7f2b94b06700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b900bdb30 0x7f2b900a8c20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7f2b80009130 tx=0x7f2b80017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.509+0000 7f2b857fa700 1 -- 172.21.15.29:0/3947839640 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8000bae0 con 0x7f2b900bdb30 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.509+0000 7f2b857fa700 1 -- 172.21.15.29:0/3947839640 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b80000c90 con 0x7f2b900bdb30 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.509+0000 7f2b9756b700 1 -- 172.21.15.29:0/3947839640 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b900ada60 con 0x7f2b900bdb30 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.509+0000 7f2b857fa700 1 -- 172.21.15.29:0/3947839640 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8001b660 con 0x7f2b900bdb30 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.509+0000 7f2b9756b700 1 -- 172.21.15.29:0/3947839640 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b900aeb60 con 0x7f2b900bdb30 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.510+0000 7f2b9756b700 1 -- 172.21.15.29:0/3947839640 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b900ae260 con 0x7f2b900bdb30 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.510+0000 7f2b857fa700 1 -- 172.21.15.29:0/3947839640 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2b8001ba10 con 0x7f2b900bdb30 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.510+0000 7f2b857fa700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b7405f270 0x7f2b74061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.511+0000 7f2b857fa700 1 -- 172.21.15.29:0/3947839640 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f2b8007bc70 con 0x7f2b900bdb30 2024-01-24T14:16:11.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.511+0000 7f2b95307700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b7405f270 0x7f2b74061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:11.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.511+0000 7f2b95307700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b7405f270 0x7f2b74061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f2b8c0011e0 tx=0x7f2b8c009040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:11.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.515+0000 7f2b857fa700 1 -- 172.21.15.29:0/3947839640 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2b900ae260 con 0x7f2b900bdb30 2024-01-24T14:16:11.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:11.744+0000 7f2b9756b700 1 -- 172.21.15.29:0/3947839640 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2b900ae260 con 0x7f2b900bdb30 2024-01-24T14:16:12.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.353+0000 7f2b857fa700 1 -- 172.21.15.29:0/3947839640 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v59) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2b900ae260 con 0x7f2b900bdb30 2024-01-24T14:16:12.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.355+0000 7f2b9756b700 1 -- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b7405f270 msgr2=0x7f2b74061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.355+0000 7f2b9756b700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b7405f270 0x7f2b74061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f2b8c0011e0 tx=0x7f2b8c009040).stop 2024-01-24T14:16:12.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.355+0000 7f2b9756b700 1 -- 172.21.15.29:0/3947839640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b900bdb30 msgr2=0x7f2b900a8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.355+0000 7f2b9756b700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b900bdb30 0x7f2b900a8c20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7f2b80009130 tx=0x7f2b80017000).stop 2024-01-24T14:16:12.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.355+0000 7f2b9756b700 1 -- 172.21.15.29:0/3947839640 shutdown_connections 2024-01-24T14:16:12.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.355+0000 7f2b9756b700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b9013a5e0 0x7f2b90155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.355+0000 7f2b9756b700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b900664c0 0x7f2b901384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.355+0000 7f2b9756b700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2b7405f270 0x7f2b74061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.355+0000 7f2b9756b700 1 --2- 172.21.15.29:0/3947839640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b900bdb30 0x7f2b900a8c20 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.355+0000 7f2b9756b700 1 -- 172.21.15.29:0/3947839640 >> 172.21.15.29:0/3947839640 conn(0x7f2b900c8bd0 msgr2=0x7f2b901398e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:12.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.356+0000 7f2b9756b700 1 -- 172.21.15.29:0/3947839640 shutdown_connections 2024-01-24T14:16:12.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.356+0000 7f2b9756b700 1 -- 172.21.15.29:0/3947839640 wait complete. 2024-01-24T14:16:12.373 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-24T14:16:12.373 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:16:12.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.508+0000 7f4855c31700 1 Processor -- start 2024-01-24T14:16:12.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.508+0000 7f4855c31700 1 -- start start 2024-01-24T14:16:12.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.509+0000 7f4855c31700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f485013d7a0 0x7f48500a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:12.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.509+0000 7f4855c31700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48500c1420 con 0x7f4850064c80 2024-01-24T14:16:12.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.509+0000 7f4855c31700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48500c1700 con 0x7f485013db80 2024-01-24T14:16:12.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.509+0000 7f4855c31700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48500c19e0 con 0x7f485013d7a0 2024-01-24T14:16:12.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.509+0000 7f484ffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4850064c80 0x7f48500c24c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52430/0 (socket says 172.21.15.29:52430) 2024-01-24T14:16:12.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.509+0000 7f484ffff700 1 -- 172.21.15.29:0/4085741260 learned_addr learned my addr 172.21.15.29:0/4085741260 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:12.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.509+0000 7f484f7fe700 1 --2- 172.21.15.29:0/4085741260 >> v2:172.21.15.163:3300/0 conn(0x7f485013d7a0 0x7f48500a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:12.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.510+0000 7f484e7fc700 1 -- 172.21.15.29:0/4085741260 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889297223 0 0) 0x7f48500c1420 con 0x7f4850064c80 2024-01-24T14:16:12.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.510+0000 7f484e7fc700 1 -- 172.21.15.29:0/4085741260 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48380033f0 con 0x7f4850064c80 2024-01-24T14:16:12.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.510+0000 7f484e7fc700 1 -- 172.21.15.29:0/4085741260 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 883027128 0 0) 0x7f48380033f0 con 0x7f4850064c80 2024-01-24T14:16:12.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.510+0000 7f484f7fe700 1 -- 172.21.15.29:0/4085741260 >> v1:172.21.15.163:6789/0 conn(0x7f485013db80 legacy=0x7f4850064570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.510+0000 7f484f7fe700 1 -- 172.21.15.29:0/4085741260 >> v1:172.21.15.29:6789/0 conn(0x7f4850064c80 legacy=0x7f48500c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.510+0000 7f484f7fe700 1 -- 172.21.15.29:0/4085741260 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48500c1cc0 con 0x7f485013d7a0 2024-01-24T14:16:12.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.510+0000 7f484f7fe700 1 --2- 172.21.15.29:0/4085741260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f485013d7a0 0x7f48500a4880 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f4840009760 tx=0x7f4840009060).ready entity=mon.1 client_cookie=aed7f5648d021d16 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:12.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.510+0000 7f484e7fc700 1 -- 172.21.15.29:0/4085741260 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48480017f0 con 0x7f485013d7a0 2024-01-24T14:16:12.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.510+0000 7f484e7fc700 1 -- 172.21.15.29:0/4085741260 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4840009130 con 0x7f485013d7a0 2024-01-24T14:16:12.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.510+0000 7f484e7fc700 1 -- 172.21.15.29:0/4085741260 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840012620 con 0x7f485013d7a0 2024-01-24T14:16:12.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.511+0000 7f4855c31700 1 -- 172.21.15.29:0/4085741260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f485013d7a0 msgr2=0x7f48500a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.511+0000 7f4855c31700 1 --2- 172.21.15.29:0/4085741260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f485013d7a0 0x7f48500a4880 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f4840009760 tx=0x7f4840009060).stop 2024-01-24T14:16:12.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.511+0000 7f4855c31700 1 -- 172.21.15.29:0/4085741260 shutdown_connections 2024-01-24T14:16:12.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.511+0000 7f4855c31700 1 --2- 172.21.15.29:0/4085741260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f485013d7a0 0x7f48500a4880 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.511+0000 7f4855c31700 1 -- 172.21.15.29:0/4085741260 >> 172.21.15.29:0/4085741260 conn(0x7f4850063280 msgr2=0x7f48500c6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:12.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.511+0000 7f4855c31700 1 -- 172.21.15.29:0/4085741260 shutdown_connections 2024-01-24T14:16:12.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.511+0000 7f4855c31700 1 -- 172.21.15.29:0/4085741260 wait complete. 2024-01-24T14:16:12.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.512+0000 7f4855c31700 1 Processor -- start 2024-01-24T14:16:12.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.513+0000 7f4855c31700 1 -- start start 2024-01-24T14:16:12.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.513+0000 7f4855c31700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4850064c80 0x7f4850133980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:12.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.513+0000 7f4855c31700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f485013d7a0 0x7f48500c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:12.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.513+0000 7f4855c31700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f485013db80 0x7f48500c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:12.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.513+0000 7f4855c31700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4850157120 con 0x7f4850064c80 2024-01-24T14:16:12.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.513+0000 7f4855c31700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48500c5e70 con 0x7f485013d7a0 2024-01-24T14:16:12.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.513+0000 7f4855c31700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4850157b70 con 0x7f485013db80 2024-01-24T14:16:12.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.513+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4850064c80 0x7f4850133980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:12.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.513+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4850064c80 0x7f4850133980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39426/0 (socket says 172.21.15.29:39426) 2024-01-24T14:16:12.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.513+0000 7f484f7fe700 1 -- 172.21.15.29:0/215987815 learned_addr learned my addr 172.21.15.29:0/215987815 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:12.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f484ffff700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f485013db80 0x7f48500c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:12.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f484effd700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f485013d7a0 0x7f48500c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:12.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f484f7fe700 1 -- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f485013db80 msgr2=0x7f48500c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f484f7fe700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f485013db80 0x7f48500c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f484f7fe700 1 -- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f485013d7a0 msgr2=0x7f48500c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f484f7fe700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f485013d7a0 0x7f48500c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f484f7fe700 1 -- 172.21.15.29:0/215987815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48400092a0 con 0x7f4850064c80 2024-01-24T14:16:12.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f484f7fe700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4850064c80 0x7f4850133980 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7f4840000e50 tx=0x7f4840015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:12.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f48377fe700 1 -- 172.21.15.29:0/215987815 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840011a20 con 0x7f4850064c80 2024-01-24T14:16:12.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f48377fe700 1 -- 172.21.15.29:0/215987815 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4840011b80 con 0x7f4850064c80 2024-01-24T14:16:12.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f4855c31700 1 -- 172.21.15.29:0/215987815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48501586f0 con 0x7f4850064c80 2024-01-24T14:16:12.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f48377fe700 1 -- 172.21.15.29:0/215987815 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484001d3f0 con 0x7f4850064c80 2024-01-24T14:16:12.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.514+0000 7f4855c31700 1 -- 172.21.15.29:0/215987815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4850157dc0 con 0x7f4850064c80 2024-01-24T14:16:12.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.515+0000 7f4855c31700 1 -- 172.21.15.29:0/215987815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48501589d0 con 0x7f4850064c80 2024-01-24T14:16:12.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.516+0000 7f48377fe700 1 -- 172.21.15.29:0/215987815 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f484001d760 con 0x7f4850064c80 2024-01-24T14:16:12.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.516+0000 7f48377fe700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f483c05f270 0x7f483c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:12.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.516+0000 7f48377fe700 1 -- 172.21.15.29:0/215987815 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f484007c150 con 0x7f4850064c80 2024-01-24T14:16:12.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.516+0000 7f484effd700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f483c05f270 0x7f483c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:12.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.517+0000 7f484effd700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f483c05f270 0x7f483c061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f48501576e0 tx=0x7f4844009040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:12.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.521+0000 7f48377fe700 1 -- 172.21.15.29:0/215987815 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f484004cea0 con 0x7f4850064c80 2024-01-24T14:16:12.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.746+0000 7f4855c31700 1 -- 172.21.15.29:0/215987815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48501589d0 con 0x7f4850064c80 2024-01-24T14:16:12.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.746+0000 7f48377fe700 1 -- 172.21.15.29:0/215987815 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f48400504c0 con 0x7f4850064c80 2024-01-24T14:16:12.748 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:16:12.748 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1579003637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1579003637},{"type":"v1","addr":"172.21.15.163:6836","nonce":1579003637}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2823175689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2823175689},{"type":"v1","addr":"172.21.15.163:6837","nonce":2823175689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2560003609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2560003609},{"type":"v1","addr":"172.21.15.29:6834","nonce":2560003609}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:16:09.332377+0000","modified":"2024-01-24T14:16:12.347265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1458876872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1458876872},{"type":"v1","addr":"172.21.15.29:6835","nonce":1458876872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-24T14:16:12.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.748+0000 7f4855c31700 1 -- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f483c05f270 msgr2=0x7f483c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.748+0000 7f4855c31700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f483c05f270 0x7f483c061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f48501576e0 tx=0x7f4844009040).stop 2024-01-24T14:16:12.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.748+0000 7f4855c31700 1 -- 172.21.15.29:0/215987815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4850064c80 msgr2=0x7f4850133980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.748+0000 7f4855c31700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4850064c80 0x7f4850133980 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7f4840000e50 tx=0x7f4840015000).stop 2024-01-24T14:16:12.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.748+0000 7f4855c31700 1 -- 172.21.15.29:0/215987815 shutdown_connections 2024-01-24T14:16:12.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.748+0000 7f4855c31700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f485013db80 0x7f48500c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.748+0000 7f4855c31700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f483c05f270 0x7f483c061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.748+0000 7f4855c31700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4850064c80 0x7f4850133980 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.748+0000 7f4855c31700 1 --2- 172.21.15.29:0/215987815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f485013d7a0 0x7f48500c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.748+0000 7f4855c31700 1 -- 172.21.15.29:0/215987815 >> 172.21.15.29:0/215987815 conn(0x7f4850063280 msgr2=0x7f48500c66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:12.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.749+0000 7f4855c31700 1 -- 172.21.15.29:0/215987815 shutdown_connections 2024-01-24T14:16:12.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.749+0000 7f4855c31700 1 -- 172.21.15.29:0/215987815 wait complete. 2024-01-24T14:16:12.751 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 59 2024-01-24T14:16:12.763 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:16:12.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.892+0000 7fb1ba38f700 1 Processor -- start 2024-01-24T14:16:12.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.893+0000 7fb1ba38f700 1 -- start start 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.893+0000 7fb1ba38f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb1b40666b0 0x7fb1b4066a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.893+0000 7fb1ba38f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1b40c4d40 0x7fb1b413b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.893+0000 7fb1ba38f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b40ac070 con 0x7fb1b40abc30 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.893+0000 7fb1ba38f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b40c5490 con 0x7fb1b40666b0 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.893+0000 7fb1ba38f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b40bdd40 con 0x7fb1b40c4d40 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.894+0000 7fb1b3fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb1b40666b0 0x7fb1b4066a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.894+0000 7fb1b3fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb1b40666b0 0x7fb1b4066a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39440/0 (socket says 172.21.15.29:39440) 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.894+0000 7fb1b3fff700 1 -- 172.21.15.29:0/2162949516 learned_addr learned my addr 172.21.15.29:0/2162949516 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.894+0000 7fb1b892c700 1 --2- 172.21.15.29:0/2162949516 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1b40c4d40 0x7fb1b413b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.894+0000 7fb1b3fff700 1 -- 172.21.15.29:0/2162949516 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1b40c4d40 msgr2=0x7fb1b413b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.894+0000 7fb1b3fff700 1 --2- 172.21.15.29:0/2162949516 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1b40c4d40 0x7fb1b413b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.894+0000 7fb1b3fff700 1 -- 172.21.15.29:0/2162949516 >> v1:172.21.15.163:6789/0 conn(0x7fb1b40abc30 legacy=0x7fb1b41563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.894+0000 7fb1b3fff700 1 -- 172.21.15.29:0/2162949516 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b40c38a0 con 0x7fb1b40666b0 2024-01-24T14:16:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.894+0000 7fb1b3fff700 1 --2- 172.21.15.29:0/2162949516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1b40666b0 0x7fb1b4066a90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7fb1a4006fd0 tx=0x7fb1a40090f0).ready entity=mon.0 client_cookie=80365ce66142db8a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.894+0000 7fb1b2ffd700 1 -- 172.21.15.29:0/2162949516 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a400b600 con 0x7fb1b40666b0 2024-01-24T14:16:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.894+0000 7fb1b2ffd700 1 -- 172.21.15.29:0/2162949516 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a400b760 con 0x7fb1b40666b0 2024-01-24T14:16:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.894+0000 7fb1b2ffd700 1 -- 172.21.15.29:0/2162949516 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a400bae0 con 0x7fb1b40666b0 2024-01-24T14:16:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.895+0000 7fb1ba38f700 1 -- 172.21.15.29:0/2162949516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1b40666b0 msgr2=0x7fb1b4066a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.895+0000 7fb1ba38f700 1 --2- 172.21.15.29:0/2162949516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1b40666b0 0x7fb1b4066a90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7fb1a4006fd0 tx=0x7fb1a40090f0).stop 2024-01-24T14:16:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.895+0000 7fb1ba38f700 1 -- 172.21.15.29:0/2162949516 shutdown_connections 2024-01-24T14:16:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.895+0000 7fb1ba38f700 1 --2- 172.21.15.29:0/2162949516 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1b40c4d40 0x7fb1b413b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.895+0000 7fb1ba38f700 1 --2- 172.21.15.29:0/2162949516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1b40666b0 0x7fb1b4066a90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.895+0000 7fb1ba38f700 1 -- 172.21.15.29:0/2162949516 >> 172.21.15.29:0/2162949516 conn(0x7fb1b40c8bb0 msgr2=0x7fb1b40c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.895+0000 7fb1ba38f700 1 -- 172.21.15.29:0/2162949516 shutdown_connections 2024-01-24T14:16:12.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.895+0000 7fb1ba38f700 1 -- 172.21.15.29:0/2162949516 wait complete. 2024-01-24T14:16:12.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.896+0000 7fb1ba38f700 1 Processor -- start 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.896+0000 7fb1ba38f700 1 -- start start 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.897+0000 7fb1ba38f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1b40666b0 0x7fb1b40c1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.897+0000 7fb1ba38f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1b40abc30 0x7fb1b41560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.897+0000 7fb1ba38f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1b40c4d40 0x7fb1b413a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.897+0000 7fb1ba38f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b40c2ac0 con 0x7fb1b40c4d40 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.897+0000 7fb1ba38f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b41565c0 con 0x7fb1b40abc30 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.897+0000 7fb1ba38f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b40a8f80 con 0x7fb1b40666b0 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.897+0000 7fb1b892c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1b40c4d40 0x7fb1b413a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.897+0000 7fb1b892c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1b40c4d40 0x7fb1b413a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39448/0 (socket says 172.21.15.29:39448) 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.897+0000 7fb1b892c700 1 -- 172.21.15.29:0/3472722940 learned_addr learned my addr 172.21.15.29:0/3472722940 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.897+0000 7fb1b37fe700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1b40abc30 0x7fb1b41560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.897+0000 7fb1b3fff700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1b40666b0 0x7fb1b40c1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.898+0000 7fb1b892c700 1 -- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1b40666b0 msgr2=0x7fb1b40c1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.898+0000 7fb1b892c700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1b40666b0 0x7fb1b40c1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.898+0000 7fb1b892c700 1 -- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1b40abc30 msgr2=0x7fb1b41560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.898+0000 7fb1b892c700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1b40abc30 0x7fb1b41560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.898+0000 7fb1b892c700 1 -- 172.21.15.29:0/3472722940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4009370 con 0x7fb1b40c4d40 2024-01-24T14:16:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.898+0000 7fb1b892c700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1b40c4d40 0x7fb1b413a630 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7fb1ac007f80 tx=0x7fb1ac009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.898+0000 7fb19bfff700 1 -- 172.21.15.29:0/3472722940 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00ec80 con 0x7fb1b40c4d40 2024-01-24T14:16:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.898+0000 7fb19bfff700 1 -- 172.21.15.29:0/3472722940 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac00ede0 con 0x7fb1b40c4d40 2024-01-24T14:16:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.898+0000 7fb1ba38f700 1 -- 172.21.15.29:0/3472722940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b40a9240 con 0x7fb1b40c4d40 2024-01-24T14:16:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.898+0000 7fb19bfff700 1 -- 172.21.15.29:0/3472722940 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac014610 con 0x7fb1b40c4d40 2024-01-24T14:16:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.898+0000 7fb1ba38f700 1 -- 172.21.15.29:0/3472722940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b40c2500 con 0x7fb1b40c4d40 2024-01-24T14:16:12.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.899+0000 7fb1ba38f700 1 -- 172.21.15.29:0/3472722940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b40a93d0 con 0x7fb1b40c4d40 2024-01-24T14:16:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.899+0000 7fb19bfff700 1 -- 172.21.15.29:0/3472722940 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb1ac014980 con 0x7fb1b40c4d40 2024-01-24T14:16:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.900+0000 7fb19bfff700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1a005f270 0x7fb1a0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.900+0000 7fb19bfff700 1 -- 172.21.15.29:0/3472722940 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fb1ac07abf0 con 0x7fb1b40c4d40 2024-01-24T14:16:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.900+0000 7fb1b3fff700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1a005f270 0x7fb1a0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:12.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.901+0000 7fb1b3fff700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1a005f270 0x7fb1a0061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fb1a4009130 tx=0x7fb1a4017000).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:12.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:12.904+0000 7fb19bfff700 1 -- 172.21.15.29:0/3472722940 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb1ac081080 con 0x7fb1b40c4d40 2024-01-24T14:16:13.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.090+0000 7fb1ba38f700 1 -- 172.21.15.29:0/3472722940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1b40475e0 con 0x7fb1b40c4d40 2024-01-24T14:16:13.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.091+0000 7fb19bfff700 1 -- 172.21.15.29:0/3472722940 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+18185 (secure 0 0 0) 0x7fb1ac04b940 con 0x7fb1b40c4d40 2024-01-24T14:16:13.092 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:16:13.092 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":137,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:16:10.334589+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:16:07.075847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-24T14:16:08.308597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:16:13.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.093+0000 7fb1ba38f700 1 -- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1a005f270 msgr2=0x7fb1a0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.093+0000 7fb1ba38f700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1a005f270 0x7fb1a0061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fb1a4009130 tx=0x7fb1a4017000).stop 2024-01-24T14:16:13.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.093+0000 7fb1ba38f700 1 -- 172.21.15.29:0/3472722940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1b40c4d40 msgr2=0x7fb1b413a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.093+0000 7fb1ba38f700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1b40c4d40 0x7fb1b413a630 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7fb1ac007f80 tx=0x7fb1ac009860).stop 2024-01-24T14:16:13.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.093+0000 7fb1ba38f700 1 -- 172.21.15.29:0/3472722940 shutdown_connections 2024-01-24T14:16:13.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.093+0000 7fb1ba38f700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1a005f270 0x7fb1a0061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.093+0000 7fb1ba38f700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1b40c4d40 0x7fb1b413a630 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.093+0000 7fb1ba38f700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1b40666b0 0x7fb1b40c1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.093+0000 7fb1ba38f700 1 --2- 172.21.15.29:0/3472722940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1b40abc30 0x7fb1b41560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.093+0000 7fb1ba38f700 1 -- 172.21.15.29:0/3472722940 >> 172.21.15.29:0/3472722940 conn(0x7fb1b40c8bb0 msgr2=0x7fb1b40a7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:13.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.093+0000 7fb1ba38f700 1 -- 172.21.15.29:0/3472722940 shutdown_connections 2024-01-24T14:16:13.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.094+0000 7fb1ba38f700 1 -- 172.21.15.29:0/3472722940 wait complete. 2024-01-24T14:16:13.108 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-24T14:16:13.108 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:16:13.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.238+0000 7f7c92f8f700 1 Processor -- start 2024-01-24T14:16:13.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.238+0000 7f7c92f8f700 1 -- start start 2024-01-24T14:16:13.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.239+0000 7f7c92f8f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c8c154190 0x7f7c8c069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.239+0000 7f7c92f8f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8c06a2c0 con 0x7f7c8c062d50 2024-01-24T14:16:13.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.239+0000 7f7c92f8f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8c0bd190 con 0x7f7c8c062970 2024-01-24T14:16:13.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.239+0000 7f7c92f8f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8c154620 con 0x7f7c8c154190 2024-01-24T14:16:13.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.239+0000 7f7c9152c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c8c154190 0x7f7c8c069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.239+0000 7f7c8bfff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7c8c062d50 0x7f7c8c065360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52440/0 (socket says 172.21.15.29:52440) 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.239+0000 7f7c9152c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c8c154190 0x7f7c8c069dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39462/0 (socket says 172.21.15.29:39462) 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.239+0000 7f7c8bfff700 1 -- 172.21.15.29:0/3625604541 learned_addr learned my addr 172.21.15.29:0/3625604541 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.239+0000 7f7c9152c700 1 -- 172.21.15.29:0/3625604541 >> v1:172.21.15.163:6789/0 conn(0x7f7c8c062970 legacy=0x7f7c8c153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.239+0000 7f7c9152c700 1 -- 172.21.15.29:0/3625604541 >> v1:172.21.15.29:6789/0 conn(0x7f7c8c062d50 legacy=0x7f7c8c065360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.239+0000 7f7c9152c700 1 -- 172.21.15.29:0/3625604541 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8c0c4e60 con 0x7f7c8c154190 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.239+0000 7f7c9152c700 1 --2- 172.21.15.29:0/3625604541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8c154190 0x7f7c8c069dd0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7f7c84009e60 tx=0x7f7c84009780).ready entity=mon.0 client_cookie=18b8267ae0ec6b90 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.240+0000 7f7c8b7fe700 1 -- 172.21.15.29:0/3625604541 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8400e070 con 0x7f7c8c154190 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.240+0000 7f7c8b7fe700 1 -- 172.21.15.29:0/3625604541 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8400bc30 con 0x7f7c8c154190 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.240+0000 7f7c8b7fe700 1 -- 172.21.15.29:0/3625604541 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84012410 con 0x7f7c8c154190 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.240+0000 7f7c92f8f700 1 -- 172.21.15.29:0/3625604541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8c154190 msgr2=0x7f7c8c069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.240+0000 7f7c92f8f700 1 --2- 172.21.15.29:0/3625604541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8c154190 0x7f7c8c069dd0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7f7c84009e60 tx=0x7f7c84009780).stop 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.240+0000 7f7c92f8f700 1 -- 172.21.15.29:0/3625604541 shutdown_connections 2024-01-24T14:16:13.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.240+0000 7f7c92f8f700 1 --2- 172.21.15.29:0/3625604541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8c154190 0x7f7c8c069dd0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.240+0000 7f7c92f8f700 1 -- 172.21.15.29:0/3625604541 >> 172.21.15.29:0/3625604541 conn(0x7f7c8c0c85c0 msgr2=0x7f7c8c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:13.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.240+0000 7f7c92f8f700 1 -- 172.21.15.29:0/3625604541 shutdown_connections 2024-01-24T14:16:13.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.240+0000 7f7c92f8f700 1 -- 172.21.15.29:0/3625604541 wait complete. 2024-01-24T14:16:13.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.242+0000 7f7c92f8f700 1 Processor -- start 2024-01-24T14:16:13.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.242+0000 7f7c92f8f700 1 -- start start 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c92f8f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c8c062970 0x7f7c8c13d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c92f8f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8c062d50 0x7f7c8c13d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c92f8f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c8c154190 0x7f7c8c0a8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c92f8f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c13dad0 con 0x7f7c8c062d50 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c92f8f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c0bf090 con 0x7f7c8c062970 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c92f8f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c0bf370 con 0x7f7c8c154190 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c8bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8c062d50 0x7f7c8c13d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c8bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8c062d50 0x7f7c8c13d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39464/0 (socket says 172.21.15.29:39464) 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c8bfff700 1 -- 172.21.15.29:0/1921086120 learned_addr learned my addr 172.21.15.29:0/1921086120 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c90d2b700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c8c062970 0x7f7c8c13d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c9152c700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c8c154190 0x7f7c8c0a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c8bfff700 1 -- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c8c154190 msgr2=0x7f7c8c0a8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c8bfff700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c8c154190 0x7f7c8c0a8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c8bfff700 1 -- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c8c062970 msgr2=0x7f7c8c13d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c8bfff700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c8c062970 0x7f7c8c13d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c8bfff700 1 -- 172.21.15.29:0/1921086120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c84009a20 con 0x7f7c8c062d50 2024-01-24T14:16:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c9152c700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c8c154190 0x7f7c8c0a8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c90d2b700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c8c062970 0x7f7c8c13d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c8bfff700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8c062d50 0x7f7c8c13d5e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f7c78007fd0 tx=0x7f7c78007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c88ff9700 1 -- 172.21.15.29:0/1921086120 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780072e0 con 0x7f7c8c062d50 2024-01-24T14:16:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c88ff9700 1 -- 172.21.15.29:0/1921086120 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c78007440 con 0x7f7c8c062d50 2024-01-24T14:16:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c92f8f700 1 -- 172.21.15.29:0/1921086120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8c0bf650 con 0x7f7c8c062d50 2024-01-24T14:16:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.243+0000 7f7c88ff9700 1 -- 172.21.15.29:0/1921086120 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780057a0 con 0x7f7c8c062d50 2024-01-24T14:16:13.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.244+0000 7f7c92f8f700 1 -- 172.21.15.29:0/1921086120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8c0bfc10 con 0x7f7c8c062d50 2024-01-24T14:16:13.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.245+0000 7f7c92f8f700 1 -- 172.21.15.29:0/1921086120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8c0bfe30 con 0x7f7c8c062d50 2024-01-24T14:16:13.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.245+0000 7f7c88ff9700 1 -- 172.21.15.29:0/1921086120 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7c78005b90 con 0x7f7c8c062d50 2024-01-24T14:16:13.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.245+0000 7f7c88ff9700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c7005f270 0x7f7c70061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.246+0000 7f7c88ff9700 1 -- 172.21.15.29:0/1921086120 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f7c78079f90 con 0x7f7c8c062d50 2024-01-24T14:16:13.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.246+0000 7f7c90d2b700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c7005f270 0x7f7c70061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.246+0000 7f7c90d2b700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c7005f270 0x7f7c70061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7c80000c00 tx=0x7f7c80009040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:13.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.250+0000 7f7c88ff9700 1 -- 172.21.15.29:0/1921086120 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7c7804ace0 con 0x7f7c8c062d50 2024-01-24T14:16:13.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.428+0000 7f7c92f8f700 1 -- 172.21.15.29:0/1921086120 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c8c063380 con 0x7f7c7005f270 2024-01-24T14:16:13.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.439+0000 7f7c88ff9700 1 -- 172.21.15.29:0/1921086120 <== mgr.7195 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357681 (secure 0 0 0) 0x7f7c8c063380 con 0x7f7c7005f270 2024-01-24T14:16:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:16:13.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.442+0000 7f7c92f8f700 1 -- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c7005f270 msgr2=0x7f7c70061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.442+0000 7f7c92f8f700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c7005f270 0x7f7c70061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7c80000c00 tx=0x7f7c80009040).stop 2024-01-24T14:16:13.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.443+0000 7f7c92f8f700 1 -- 172.21.15.29:0/1921086120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8c062d50 msgr2=0x7f7c8c13d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.443+0000 7f7c92f8f700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8c062d50 0x7f7c8c13d5e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f7c78007fd0 tx=0x7f7c78007df0).stop 2024-01-24T14:16:13.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.443+0000 7f7c92f8f700 1 -- 172.21.15.29:0/1921086120 shutdown_connections 2024-01-24T14:16:13.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.443+0000 7f7c92f8f700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c8c154190 0x7f7c8c0a8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.443+0000 7f7c92f8f700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c8c062970 0x7f7c8c13d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.443+0000 7f7c92f8f700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c7005f270 0x7f7c70061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.443+0000 7f7c92f8f700 1 --2- 172.21.15.29:0/1921086120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c8c062d50 0x7f7c8c13d5e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.443+0000 7f7c92f8f700 1 -- 172.21.15.29:0/1921086120 >> 172.21.15.29:0/1921086120 conn(0x7f7c8c0c85c0 msgr2=0x7f7c8c0a5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:13.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.443+0000 7f7c92f8f700 1 -- 172.21.15.29:0/1921086120 shutdown_connections 2024-01-24T14:16:13.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.443+0000 7f7c92f8f700 1 -- 172.21.15.29:0/1921086120 wait complete. 2024-01-24T14:16:13.448 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:16:13.457 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T14:16:11.617551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1414,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":178,"acting":178,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2430844,"kb_used_data":13116,"kb_used_omap":0,"kb_used_meta":2417664,"kb_avail":752543876,"statfs":{"total":773094113280,"available":770604929024,"internally_reserved":0,"allocated":13430784,"data_stored":7518464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2475687936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6814,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-61,"num_write_kb":-29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.241883"},"pg_stats":[{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3f","version":"136'3","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.376003+0000","last_change":"2024-01-24T14:16:08.938387+0000","last_active":"2024-01-24T14:16:09.376003+0000","last_peered":"2024-01-24T14:16:09.376003+0000","last_clean":"2024-01-24T14:16:09.376003+0000","last_became_active":"2024-01-24T14:16:08.937623+0000","last_became_peered":"2024-01-24T14:16:08.937623+0000","last_unstale":"2024-01-24T14:16:09.376003+0000","last_undegraded":"2024-01-24T14:16:09.376003+0000","last_fullsized":"2024-01-24T14:16:09.376003+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373723+0000","last_change":"2024-01-24T14:16:09.369632+0000","last_active":"2024-01-24T14:16:09.373723+0000","last_peered":"2024-01-24T14:16:09.373723+0000","last_clean":"2024-01-24T14:16:09.373723+0000","last_became_active":"2024-01-24T14:16:09.369361+0000","last_became_peered":"2024-01-24T14:16:09.369361+0000","last_unstale":"2024-01-24T14:16:09.373723+0000","last_undegraded":"2024-01-24T14:16:09.373723+0000","last_fullsized":"2024-01-24T14:16:09.373723+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.377049+0000","last_change":"2024-01-24T14:16:09.373720+0000","last_active":"2024-01-24T14:16:09.377049+0000","last_peered":"2024-01-24T14:16:09.377049+0000","last_clean":"2024-01-24T14:16:09.377049+0000","last_became_active":"2024-01-24T14:16:09.369847+0000","last_became_peered":"2024-01-24T14:16:09.369847+0000","last_unstale":"2024-01-24T14:16:09.377049+0000","last_undegraded":"2024-01-24T14:16:09.377049+0000","last_fullsized":"2024-01-24T14:16:09.377049+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.346582+0000","last_change":"2024-01-24T14:16:08.942908+0000","last_active":"2024-01-24T14:16:09.346582+0000","last_peered":"2024-01-24T14:16:09.346582+0000","last_clean":"2024-01-24T14:16:09.346582+0000","last_became_active":"2024-01-24T14:16:08.942564+0000","last_became_peered":"2024-01-24T14:16:08.942564+0000","last_unstale":"2024-01-24T14:16:09.346582+0000","last_undegraded":"2024-01-24T14:16:09.346582+0000","last_fullsized":"2024-01-24T14:16:09.346582+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.353245+0000","last_change":"2024-01-24T14:16:08.936366+0000","last_active":"2024-01-24T14:16:09.353245+0000","last_peered":"2024-01-24T14:16:09.353245+0000","last_clean":"2024-01-24T14:16:09.353245+0000","last_became_active":"2024-01-24T14:16:08.932759+0000","last_became_peered":"2024-01-24T14:16:08.932759+0000","last_unstale":"2024-01-24T14:16:09.353245+0000","last_undegraded":"2024-01-24T14:16:09.353245+0000","last_fullsized":"2024-01-24T14:16:09.353245+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367519+0000","last_change":"2024-01-24T14:16:09.362617+0000","last_active":"2024-01-24T14:16:09.367519+0000","last_peered":"2024-01-24T14:16:09.367519+0000","last_clean":"2024-01-24T14:16:09.367519+0000","last_became_active":"2024-01-24T14:16:09.362309+0000","last_became_peered":"2024-01-24T14:16:09.362309+0000","last_unstale":"2024-01-24T14:16:09.367519+0000","last_undegraded":"2024-01-24T14:16:09.367519+0000","last_fullsized":"2024-01-24T14:16:09.367519+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3b","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360187+0000","last_change":"2024-01-24T14:16:08.930724+0000","last_active":"2024-01-24T14:16:09.360187+0000","last_peered":"2024-01-24T14:16:09.360187+0000","last_clean":"2024-01-24T14:16:09.360187+0000","last_became_active":"2024-01-24T14:16:08.929739+0000","last_became_peered":"2024-01-24T14:16:08.929739+0000","last_unstale":"2024-01-24T14:16:09.360187+0000","last_undegraded":"2024-01-24T14:16:09.360187+0000","last_fullsized":"2024-01-24T14:16:09.360187+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.357159+0000","last_change":"2024-01-24T14:16:09.350328+0000","last_active":"2024-01-24T14:16:09.357159+0000","last_peered":"2024-01-24T14:16:09.357159+0000","last_clean":"2024-01-24T14:16:09.357159+0000","last_became_active":"2024-01-24T14:16:09.350048+0000","last_became_peered":"2024-01-24T14:16:09.350048+0000","last_unstale":"2024-01-24T14:16:09.357159+0000","last_undegraded":"2024-01-24T14:16:09.357159+0000","last_fullsized":"2024-01-24T14:16:09.357159+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.364819+0000","last_change":"2024-01-24T14:16:09.361939+0000","last_active":"2024-01-24T14:16:09.364819+0000","last_peered":"2024-01-24T14:16:09.364819+0000","last_clean":"2024-01-24T14:16:09.364819+0000","last_became_active":"2024-01-24T14:16:09.361601+0000","last_became_peered":"2024-01-24T14:16:09.361601+0000","last_unstale":"2024-01-24T14:16:09.364819+0000","last_undegraded":"2024-01-24T14:16:09.364819+0000","last_fullsized":"2024-01-24T14:16:09.364819+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336782+0000","last_change":"2024-01-24T14:16:08.919648+0000","last_active":"2024-01-24T14:16:09.336782+0000","last_peered":"2024-01-24T14:16:09.336782+0000","last_clean":"2024-01-24T14:16:09.336782+0000","last_became_active":"2024-01-24T14:16:08.918195+0000","last_became_peered":"2024-01-24T14:16:08.918195+0000","last_unstale":"2024-01-24T14:16:09.336782+0000","last_undegraded":"2024-01-24T14:16:09.336782+0000","last_fullsized":"2024-01-24T14:16:09.336782+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373200+0000","last_change":"2024-01-24T14:16:09.365741+0000","last_active":"2024-01-24T14:16:09.373200+0000","last_peered":"2024-01-24T14:16:09.373200+0000","last_clean":"2024-01-24T14:16:09.373200+0000","last_became_active":"2024-01-24T14:16:09.365444+0000","last_became_peered":"2024-01-24T14:16:09.365444+0000","last_unstale":"2024-01-24T14:16:09.373200+0000","last_undegraded":"2024-01-24T14:16:09.373200+0000","last_fullsized":"2024-01-24T14:16:09.373200+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373747+0000","last_change":"2024-01-24T14:16:09.370446+0000","last_active":"2024-01-24T14:16:09.373747+0000","last_peered":"2024-01-24T14:16:09.373747+0000","last_clean":"2024-01-24T14:16:09.373747+0000","last_became_active":"2024-01-24T14:16:09.369908+0000","last_became_peered":"2024-01-24T14:16:09.369908+0000","last_unstale":"2024-01-24T14:16:09.373747+0000","last_undegraded":"2024-01-24T14:16:09.373747+0000","last_fullsized":"2024-01-24T14:16:09.373747+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350675+0000","last_change":"2024-01-24T14:16:08.929539+0000","last_active":"2024-01-24T14:16:09.350675+0000","last_peered":"2024-01-24T14:16:09.350675+0000","last_clean":"2024-01-24T14:16:09.350675+0000","last_became_active":"2024-01-24T14:16:08.929003+0000","last_became_peered":"2024-01-24T14:16:08.929003+0000","last_unstale":"2024-01-24T14:16:09.350675+0000","last_undegraded":"2024-01-24T14:16:09.350675+0000","last_fullsized":"2024-01-24T14:16:09.350675+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373705+0000","last_change":"2024-01-24T14:16:09.366328+0000","last_active":"2024-01-24T14:16:09.373705+0000","last_peered":"2024-01-24T14:16:09.373705+0000","last_clean":"2024-01-24T14:16:09.373705+0000","last_became_active":"2024-01-24T14:16:09.365624+0000","last_became_peered":"2024-01-24T14:16:09.365624+0000","last_unstale":"2024-01-24T14:16:09.373705+0000","last_undegraded":"2024-01-24T14:16:09.373705+0000","last_fullsized":"2024-01-24T14:16:09.373705+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.343446+0000","last_change":"2024-01-24T14:16:08.932084+0000","last_active":"2024-01-24T14:16:09.343446+0000","last_peered":"2024-01-24T14:16:09.343446+0000","last_clean":"2024-01-24T14:16:09.343446+0000","last_became_active":"2024-01-24T14:16:08.931666+0000","last_became_peered":"2024-01-24T14:16:08.931666+0000","last_unstale":"2024-01-24T14:16:09.343446+0000","last_undegraded":"2024-01-24T14:16:09.343446+0000","last_fullsized":"2024-01-24T14:16:09.343446+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350347+0000","last_change":"2024-01-24T14:16:08.939889+0000","last_active":"2024-01-24T14:16:09.350347+0000","last_peered":"2024-01-24T14:16:09.350347+0000","last_clean":"2024-01-24T14:16:09.350347+0000","last_became_active":"2024-01-24T14:16:08.939593+0000","last_became_peered":"2024-01-24T14:16:08.939593+0000","last_unstale":"2024-01-24T14:16:09.350347+0000","last_undegraded":"2024-01-24T14:16:09.350347+0000","last_fullsized":"2024-01-24T14:16:09.350347+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373521+0000","last_change":"2024-01-24T14:16:09.366785+0000","last_active":"2024-01-24T14:16:09.373521+0000","last_peered":"2024-01-24T14:16:09.373521+0000","last_clean":"2024-01-24T14:16:09.373521+0000","last_became_active":"2024-01-24T14:16:09.360406+0000","last_became_peered":"2024-01-24T14:16:09.360406+0000","last_unstale":"2024-01-24T14:16:09.373521+0000","last_undegraded":"2024-01-24T14:16:09.373521+0000","last_fullsized":"2024-01-24T14:16:09.373521+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367775+0000","last_change":"2024-01-24T14:16:09.363463+0000","last_active":"2024-01-24T14:16:09.367775+0000","last_peered":"2024-01-24T14:16:09.367775+0000","last_clean":"2024-01-24T14:16:09.367775+0000","last_became_active":"2024-01-24T14:16:09.363031+0000","last_became_peered":"2024-01-24T14:16:09.363031+0000","last_unstale":"2024-01-24T14:16:09.367775+0000","last_undegraded":"2024-01-24T14:16:09.367775+0000","last_fullsized":"2024-01-24T14:16:09.367775+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.356728+0000","last_change":"2024-01-24T14:16:08.944006+0000","last_active":"2024-01-24T14:16:09.356728+0000","last_peered":"2024-01-24T14:16:09.356728+0000","last_clean":"2024-01-24T14:16:09.356728+0000","last_became_active":"2024-01-24T14:16:08.943326+0000","last_became_peered":"2024-01-24T14:16:08.943326+0000","last_unstale":"2024-01-24T14:16:09.356728+0000","last_undegraded":"2024-01-24T14:16:09.356728+0000","last_fullsized":"2024-01-24T14:16:09.356728+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.369265+0000","last_change":"2024-01-24T14:16:08.945145+0000","last_active":"2024-01-24T14:16:09.369265+0000","last_peered":"2024-01-24T14:16:09.369265+0000","last_clean":"2024-01-24T14:16:09.369265+0000","last_became_active":"2024-01-24T14:16:08.944841+0000","last_became_peered":"2024-01-24T14:16:08.944841+0000","last_unstale":"2024-01-24T14:16:09.369265+0000","last_undegraded":"2024-01-24T14:16:09.369265+0000","last_fullsized":"2024-01-24T14:16:09.369265+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.363705+0000","last_change":"2024-01-24T14:16:09.361865+0000","last_active":"2024-01-24T14:16:09.363705+0000","last_peered":"2024-01-24T14:16:09.363705+0000","last_clean":"2024-01-24T14:16:09.363705+0000","last_became_active":"2024-01-24T14:16:09.361658+0000","last_became_peered":"2024-01-24T14:16:09.361658+0000","last_unstale":"2024-01-24T14:16:09.363705+0000","last_undegraded":"2024-01-24T14:16:09.363705+0000","last_fullsized":"2024-01-24T14:16:09.363705+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2b","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.349695+0000","last_change":"2024-01-24T14:16:08.938203+0000","last_active":"2024-01-24T14:16:09.349695+0000","last_peered":"2024-01-24T14:16:09.349695+0000","last_clean":"2024-01-24T14:16:09.349695+0000","last_became_active":"2024-01-24T14:16:08.937888+0000","last_became_peered":"2024-01-24T14:16:08.937888+0000","last_unstale":"2024-01-24T14:16:09.349695+0000","last_undegraded":"2024-01-24T14:16:09.349695+0000","last_fullsized":"2024-01-24T14:16:09.349695+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350511+0000","last_change":"2024-01-24T14:16:08.943928+0000","last_active":"2024-01-24T14:16:09.350511+0000","last_peered":"2024-01-24T14:16:09.350511+0000","last_clean":"2024-01-24T14:16:09.350511+0000","last_became_active":"2024-01-24T14:16:08.943463+0000","last_became_peered":"2024-01-24T14:16:08.943463+0000","last_unstale":"2024-01-24T14:16:09.350511+0000","last_undegraded":"2024-01-24T14:16:09.350511+0000","last_fullsized":"2024-01-24T14:16:09.350511+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.362435+0000","last_change":"2024-01-24T14:16:09.355594+0000","last_active":"2024-01-24T14:16:09.362435+0000","last_peered":"2024-01-24T14:16:09.362435+0000","last_clean":"2024-01-24T14:16:09.362435+0000","last_became_active":"2024-01-24T14:16:09.355308+0000","last_became_peered":"2024-01-24T14:16:09.355308+0000","last_unstale":"2024-01-24T14:16:09.362435+0000","last_undegraded":"2024-01-24T14:16:09.362435+0000","last_fullsized":"2024-01-24T14:16:09.362435+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.351402+0000","last_change":"2024-01-24T14:16:08.938907+0000","last_active":"2024-01-24T14:16:09.351402+0000","last_peered":"2024-01-24T14:16:09.351402+0000","last_clean":"2024-01-24T14:16:09.351402+0000","last_became_active":"2024-01-24T14:16:08.938612+0000","last_became_peered":"2024-01-24T14:16:08.938612+0000","last_unstale":"2024-01-24T14:16:09.351402+0000","last_undegraded":"2024-01-24T14:16:09.351402+0000","last_fullsized":"2024-01-24T14:16:09.351402+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.349170+0000","last_change":"2024-01-24T14:16:08.935501+0000","last_active":"2024-01-24T14:16:09.349170+0000","last_peered":"2024-01-24T14:16:09.349170+0000","last_clean":"2024-01-24T14:16:09.349170+0000","last_became_active":"2024-01-24T14:16:08.935240+0000","last_became_peered":"2024-01-24T14:16:08.935240+0000","last_unstale":"2024-01-24T14:16:09.349170+0000","last_undegraded":"2024-01-24T14:16:09.349170+0000","last_fullsized":"2024-01-24T14:16:09.349170+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373150+0000","last_change":"2024-01-24T14:16:09.363397+0000","last_active":"2024-01-24T14:16:09.373150+0000","last_peered":"2024-01-24T14:16:09.373150+0000","last_clean":"2024-01-24T14:16:09.373150+0000","last_became_active":"2024-01-24T14:16:09.363107+0000","last_became_peered":"2024-01-24T14:16:09.363107+0000","last_unstale":"2024-01-24T14:16:09.373150+0000","last_undegraded":"2024-01-24T14:16:09.373150+0000","last_fullsized":"2024-01-24T14:16:09.373150+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373437+0000","last_change":"2024-01-24T14:16:09.370522+0000","last_active":"2024-01-24T14:16:09.373437+0000","last_peered":"2024-01-24T14:16:09.373437+0000","last_clean":"2024-01-24T14:16:09.373437+0000","last_became_active":"2024-01-24T14:16:09.370203+0000","last_became_peered":"2024-01-24T14:16:09.370203+0000","last_unstale":"2024-01-24T14:16:09.373437+0000","last_undegraded":"2024-01-24T14:16:09.373437+0000","last_fullsized":"2024-01-24T14:16:09.373437+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.343560+0000","last_change":"2024-01-24T14:16:08.935364+0000","last_active":"2024-01-24T14:16:09.343560+0000","last_peered":"2024-01-24T14:16:09.343560+0000","last_clean":"2024-01-24T14:16:09.343560+0000","last_became_active":"2024-01-24T14:16:08.935043+0000","last_became_peered":"2024-01-24T14:16:08.935043+0000","last_unstale":"2024-01-24T14:16:09.343560+0000","last_undegraded":"2024-01-24T14:16:09.343560+0000","last_fullsized":"2024-01-24T14:16:09.343560+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.374548+0000","last_change":"2024-01-24T14:16:09.371809+0000","last_active":"2024-01-24T14:16:09.374548+0000","last_peered":"2024-01-24T14:16:09.374548+0000","last_clean":"2024-01-24T14:16:09.374548+0000","last_became_active":"2024-01-24T14:16:09.371419+0000","last_became_peered":"2024-01-24T14:16:09.371419+0000","last_unstale":"2024-01-24T14:16:09.374548+0000","last_undegraded":"2024-01-24T14:16:09.374548+0000","last_fullsized":"2024-01-24T14:16:09.374548+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360483+0000","last_change":"2024-01-24T14:16:09.353820+0000","last_active":"2024-01-24T14:16:09.360483+0000","last_peered":"2024-01-24T14:16:09.360483+0000","last_clean":"2024-01-24T14:16:09.360483+0000","last_became_active":"2024-01-24T14:16:09.353300+0000","last_became_peered":"2024-01-24T14:16:09.353300+0000","last_unstale":"2024-01-24T14:16:09.360483+0000","last_undegraded":"2024-01-24T14:16:09.360483+0000","last_fullsized":"2024-01-24T14:16:09.360483+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336483+0000","last_change":"2024-01-24T14:16:08.922917+0000","last_active":"2024-01-24T14:16:09.336483+0000","last_peered":"2024-01-24T14:16:09.336483+0000","last_clean":"2024-01-24T14:16:09.336483+0000","last_became_active":"2024-01-24T14:16:08.922601+0000","last_became_peered":"2024-01-24T14:16:08.922601+0000","last_unstale":"2024-01-24T14:16:09.336483+0000","last_undegraded":"2024-01-24T14:16:09.336483+0000","last_fullsized":"2024-01-24T14:16:09.336483+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367500+0000","last_change":"2024-01-24T14:16:09.362620+0000","last_active":"2024-01-24T14:16:09.367500+0000","last_peered":"2024-01-24T14:16:09.367500+0000","last_clean":"2024-01-24T14:16:09.367500+0000","last_became_active":"2024-01-24T14:16:09.362292+0000","last_became_peered":"2024-01-24T14:16:09.362292+0000","last_unstale":"2024-01-24T14:16:09.367500+0000","last_undegraded":"2024-01-24T14:16:09.367500+0000","last_fullsized":"2024-01-24T14:16:09.367500+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373265+0000","last_change":"2024-01-24T14:16:09.366331+0000","last_active":"2024-01-24T14:16:09.373265+0000","last_peered":"2024-01-24T14:16:09.373265+0000","last_clean":"2024-01-24T14:16:09.373265+0000","last_became_active":"2024-01-24T14:16:09.365809+0000","last_became_peered":"2024-01-24T14:16:09.365809+0000","last_unstale":"2024-01-24T14:16:09.373265+0000","last_undegraded":"2024-01-24T14:16:09.373265+0000","last_fullsized":"2024-01-24T14:16:09.373265+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336883+0000","last_change":"2024-01-24T14:16:08.928995+0000","last_active":"2024-01-24T14:16:09.336883+0000","last_peered":"2024-01-24T14:16:09.336883+0000","last_clean":"2024-01-24T14:16:09.336883+0000","last_became_active":"2024-01-24T14:16:08.928664+0000","last_became_peered":"2024-01-24T14:16:08.928664+0000","last_unstale":"2024-01-24T14:16:09.336883+0000","last_undegraded":"2024-01-24T14:16:09.336883+0000","last_fullsized":"2024-01-24T14:16:09.336883+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.363319+0000","last_change":"2024-01-24T14:16:09.359848+0000","last_active":"2024-01-24T14:16:09.363319+0000","last_peered":"2024-01-24T14:16:09.363319+0000","last_clean":"2024-01-24T14:16:09.363319+0000","last_became_active":"2024-01-24T14:16:09.359438+0000","last_became_peered":"2024-01-24T14:16:09.359438+0000","last_unstale":"2024-01-24T14:16:09.363319+0000","last_undegraded":"2024-01-24T14:16:09.363319+0000","last_fullsized":"2024-01-24T14:16:09.363319+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.353921+0000","last_change":"2024-01-24T14:16:08.938305+0000","last_active":"2024-01-24T14:16:09.353921+0000","last_peered":"2024-01-24T14:16:09.353921+0000","last_clean":"2024-01-24T14:16:09.353921+0000","last_became_active":"2024-01-24T14:16:08.937907+0000","last_became_peered":"2024-01-24T14:16:08.937907+0000","last_unstale":"2024-01-24T14:16:09.353921+0000","last_undegraded":"2024-01-24T14:16:09.353921+0000","last_fullsized":"2024-01-24T14:16:09.353921+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367746+0000","last_change":"2024-01-24T14:16:09.364666+0000","last_active":"2024-01-24T14:16:09.367746+0000","last_peered":"2024-01-24T14:16:09.367746+0000","last_clean":"2024-01-24T14:16:09.367746+0000","last_became_active":"2024-01-24T14:16:09.364380+0000","last_became_peered":"2024-01-24T14:16:09.364380+0000","last_unstale":"2024-01-24T14:16:09.367746+0000","last_undegraded":"2024-01-24T14:16:09.367746+0000","last_fullsized":"2024-01-24T14:16:09.367746+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350397+0000","last_change":"2024-01-24T14:16:08.931977+0000","last_active":"2024-01-24T14:16:09.350397+0000","last_peered":"2024-01-24T14:16:09.350397+0000","last_clean":"2024-01-24T14:16:09.350397+0000","last_became_active":"2024-01-24T14:16:08.931639+0000","last_became_peered":"2024-01-24T14:16:08.931639+0000","last_unstale":"2024-01-24T14:16:09.350397+0000","last_undegraded":"2024-01-24T14:16:09.350397+0000","last_fullsized":"2024-01-24T14:16:09.350397+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336246+0000","last_change":"2024-01-24T14:16:08.931939+0000","last_active":"2024-01-24T14:16:09.336246+0000","last_peered":"2024-01-24T14:16:09.336246+0000","last_clean":"2024-01-24T14:16:09.336246+0000","last_became_active":"2024-01-24T14:16:08.931609+0000","last_became_peered":"2024-01-24T14:16:08.931609+0000","last_unstale":"2024-01-24T14:16:09.336246+0000","last_undegraded":"2024-01-24T14:16:09.336246+0000","last_fullsized":"2024-01-24T14:16:09.336246+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373093+0000","last_change":"2024-01-24T14:16:09.363363+0000","last_active":"2024-01-24T14:16:09.373093+0000","last_peered":"2024-01-24T14:16:09.373093+0000","last_clean":"2024-01-24T14:16:09.373093+0000","last_became_active":"2024-01-24T14:16:09.363039+0000","last_became_peered":"2024-01-24T14:16:09.363039+0000","last_unstale":"2024-01-24T14:16:09.373093+0000","last_undegraded":"2024-01-24T14:16:09.373093+0000","last_fullsized":"2024-01-24T14:16:09.373093+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.340588+0000","last_change":"2024-01-24T14:16:08.947795+0000","last_active":"2024-01-24T14:16:09.340588+0000","last_peered":"2024-01-24T14:16:09.340588+0000","last_clean":"2024-01-24T14:16:09.340588+0000","last_became_active":"2024-01-24T14:16:08.947549+0000","last_became_peered":"2024-01-24T14:16:08.947549+0000","last_unstale":"2024-01-24T14:16:09.340588+0000","last_undegraded":"2024-01-24T14:16:09.340588+0000","last_fullsized":"2024-01-24T14:16:09.340588+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.340926+0000","last_change":"2024-01-24T14:16:08.931616+0000","last_active":"2024-01-24T14:16:09.340926+0000","last_peered":"2024-01-24T14:16:09.340926+0000","last_clean":"2024-01-24T14:16:09.340926+0000","last_became_active":"2024-01-24T14:16:08.931287+0000","last_became_peered":"2024-01-24T14:16:08.931287+0000","last_unstale":"2024-01-24T14:16:09.340926+0000","last_undegraded":"2024-01-24T14:16:09.340926+0000","last_fullsized":"2024-01-24T14:16:09.340926+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373545+0000","last_change":"2024-01-24T14:16:09.367674+0000","last_active":"2024-01-24T14:16:09.373545+0000","last_peered":"2024-01-24T14:16:09.373545+0000","last_clean":"2024-01-24T14:16:09.373545+0000","last_became_active":"2024-01-24T14:16:09.367413+0000","last_became_peered":"2024-01-24T14:16:09.367413+0000","last_unstale":"2024-01-24T14:16:09.373545+0000","last_undegraded":"2024-01-24T14:16:09.373545+0000","last_fullsized":"2024-01-24T14:16:09.373545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.364800+0000","last_change":"2024-01-24T14:16:09.361864+0000","last_active":"2024-01-24T14:16:09.364800+0000","last_peered":"2024-01-24T14:16:09.364800+0000","last_clean":"2024-01-24T14:16:09.364800+0000","last_became_active":"2024-01-24T14:16:09.361500+0000","last_became_peered":"2024-01-24T14:16:09.361500+0000","last_unstale":"2024-01-24T14:16:09.364800+0000","last_undegraded":"2024-01-24T14:16:09.364800+0000","last_fullsized":"2024-01-24T14:16:09.364800+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.355492+0000","last_change":"2024-01-24T14:16:08.939533+0000","last_active":"2024-01-24T14:16:09.355492+0000","last_peered":"2024-01-24T14:16:09.355492+0000","last_clean":"2024-01-24T14:16:09.355492+0000","last_became_active":"2024-01-24T14:16:08.939114+0000","last_became_peered":"2024-01-24T14:16:08.939114+0000","last_unstale":"2024-01-24T14:16:09.355492+0000","last_undegraded":"2024-01-24T14:16:09.355492+0000","last_fullsized":"2024-01-24T14:16:09.355492+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.369114+0000","last_change":"2024-01-24T14:16:09.368019+0000","last_active":"2024-01-24T14:16:09.369114+0000","last_peered":"2024-01-24T14:16:09.369114+0000","last_clean":"2024-01-24T14:16:09.369114+0000","last_became_active":"2024-01-24T14:16:09.367477+0000","last_became_peered":"2024-01-24T14:16:09.367477+0000","last_unstale":"2024-01-24T14:16:09.369114+0000","last_undegraded":"2024-01-24T14:16:09.369114+0000","last_fullsized":"2024-01-24T14:16:09.369114+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.374688+0000","last_change":"2024-01-24T14:16:08.944181+0000","last_active":"2024-01-24T14:16:09.374688+0000","last_peered":"2024-01-24T14:16:09.374688+0000","last_clean":"2024-01-24T14:16:09.374688+0000","last_became_active":"2024-01-24T14:16:08.943722+0000","last_became_peered":"2024-01-24T14:16:08.943722+0000","last_unstale":"2024-01-24T14:16:09.374688+0000","last_undegraded":"2024-01-24T14:16:09.374688+0000","last_fullsized":"2024-01-24T14:16:09.374688+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.376650+0000","last_change":"2024-01-24T14:16:09.375052+0000","last_active":"2024-01-24T14:16:09.376650+0000","last_peered":"2024-01-24T14:16:09.376650+0000","last_clean":"2024-01-24T14:16:09.376650+0000","last_became_active":"2024-01-24T14:16:09.374764+0000","last_became_peered":"2024-01-24T14:16:09.374764+0000","last_unstale":"2024-01-24T14:16:09.376650+0000","last_undegraded":"2024-01-24T14:16:09.376650+0000","last_fullsized":"2024-01-24T14:16:09.376650+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.352207+0000","last_change":"2024-01-24T14:16:08.927699+0000","last_active":"2024-01-24T14:16:09.352207+0000","last_peered":"2024-01-24T14:16:09.352207+0000","last_clean":"2024-01-24T14:16:09.352207+0000","last_became_active":"2024-01-24T14:16:08.927375+0000","last_became_peered":"2024-01-24T14:16:08.927375+0000","last_unstale":"2024-01-24T14:16:09.352207+0000","last_undegraded":"2024-01-24T14:16:09.352207+0000","last_fullsized":"2024-01-24T14:16:09.352207+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.340477+0000","last_change":"2024-01-24T14:16:08.938132+0000","last_active":"2024-01-24T14:16:09.340477+0000","last_peered":"2024-01-24T14:16:09.340477+0000","last_clean":"2024-01-24T14:16:09.340477+0000","last_became_active":"2024-01-24T14:16:08.937761+0000","last_became_peered":"2024-01-24T14:16:08.937761+0000","last_unstale":"2024-01-24T14:16:09.340477+0000","last_undegraded":"2024-01-24T14:16:09.340477+0000","last_fullsized":"2024-01-24T14:16:09.340477+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.357408+0000","last_change":"2024-01-24T14:16:08.942263+0000","last_active":"2024-01-24T14:16:09.357408+0000","last_peered":"2024-01-24T14:16:09.357408+0000","last_clean":"2024-01-24T14:16:09.357408+0000","last_became_active":"2024-01-24T14:16:08.930700+0000","last_became_peered":"2024-01-24T14:16:08.930700+0000","last_unstale":"2024-01-24T14:16:09.357408+0000","last_undegraded":"2024-01-24T14:16:09.357408+0000","last_fullsized":"2024-01-24T14:16:09.357408+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.377044+0000","last_change":"2024-01-24T14:16:09.373474+0000","last_active":"2024-01-24T14:16:09.377044+0000","last_peered":"2024-01-24T14:16:09.377044+0000","last_clean":"2024-01-24T14:16:09.377044+0000","last_became_active":"2024-01-24T14:16:09.373140+0000","last_became_peered":"2024-01-24T14:16:09.373140+0000","last_unstale":"2024-01-24T14:16:09.377044+0000","last_undegraded":"2024-01-24T14:16:09.377044+0000","last_fullsized":"2024-01-24T14:16:09.377044+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336238+0000","last_change":"2024-01-24T14:16:08.931869+0000","last_active":"2024-01-24T14:16:09.336238+0000","last_peered":"2024-01-24T14:16:09.336238+0000","last_clean":"2024-01-24T14:16:09.336238+0000","last_became_active":"2024-01-24T14:16:08.931534+0000","last_became_peered":"2024-01-24T14:16:08.931534+0000","last_unstale":"2024-01-24T14:16:09.336238+0000","last_undegraded":"2024-01-24T14:16:09.336238+0000","last_fullsized":"2024-01-24T14:16:09.336238+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.349081+0000","last_change":"2024-01-24T14:16:08.934952+0000","last_active":"2024-01-24T14:16:09.349081+0000","last_peered":"2024-01-24T14:16:09.349081+0000","last_clean":"2024-01-24T14:16:09.349081+0000","last_became_active":"2024-01-24T14:16:08.934586+0000","last_became_peered":"2024-01-24T14:16:08.934586+0000","last_unstale":"2024-01-24T14:16:09.349081+0000","last_undegraded":"2024-01-24T14:16:09.349081+0000","last_fullsized":"2024-01-24T14:16:09.349081+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373572+0000","last_change":"2024-01-24T14:16:09.369996+0000","last_active":"2024-01-24T14:16:09.373572+0000","last_peered":"2024-01-24T14:16:09.373572+0000","last_clean":"2024-01-24T14:16:09.373572+0000","last_became_active":"2024-01-24T14:16:09.369677+0000","last_became_peered":"2024-01-24T14:16:09.369677+0000","last_unstale":"2024-01-24T14:16:09.373572+0000","last_undegraded":"2024-01-24T14:16:09.373572+0000","last_fullsized":"2024-01-24T14:16:09.373572+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.355605+0000","last_change":"2024-01-24T14:16:08.939792+0000","last_active":"2024-01-24T14:16:09.355605+0000","last_peered":"2024-01-24T14:16:09.355605+0000","last_clean":"2024-01-24T14:16:09.355605+0000","last_became_active":"2024-01-24T14:16:08.939486+0000","last_became_peered":"2024-01-24T14:16:08.939486+0000","last_unstale":"2024-01-24T14:16:09.355605+0000","last_undegraded":"2024-01-24T14:16:09.355605+0000","last_fullsized":"2024-01-24T14:16:09.355605+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.369957+0000","last_change":"2024-01-24T14:16:08.945586+0000","last_active":"2024-01-24T14:16:09.369957+0000","last_peered":"2024-01-24T14:16:09.369957+0000","last_clean":"2024-01-24T14:16:09.369957+0000","last_became_active":"2024-01-24T14:16:08.945238+0000","last_became_peered":"2024-01-24T14:16:08.945238+0000","last_unstale":"2024-01-24T14:16:09.369957+0000","last_undegraded":"2024-01-24T14:16:09.369957+0000","last_fullsized":"2024-01-24T14:16:09.369957+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373766+0000","last_change":"2024-01-24T14:16:09.371183+0000","last_active":"2024-01-24T14:16:09.373766+0000","last_peered":"2024-01-24T14:16:09.373766+0000","last_clean":"2024-01-24T14:16:09.373766+0000","last_became_active":"2024-01-24T14:16:09.369722+0000","last_became_peered":"2024-01-24T14:16:09.369722+0000","last_unstale":"2024-01-24T14:16:09.373766+0000","last_undegraded":"2024-01-24T14:16:09.373766+0000","last_fullsized":"2024-01-24T14:16:09.373766+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373728+0000","last_change":"2024-01-24T14:16:09.366503+0000","last_active":"2024-01-24T14:16:09.373728+0000","last_peered":"2024-01-24T14:16:09.373728+0000","last_clean":"2024-01-24T14:16:09.373728+0000","last_became_active":"2024-01-24T14:16:09.366048+0000","last_became_peered":"2024-01-24T14:16:09.366048+0000","last_unstale":"2024-01-24T14:16:09.373728+0000","last_undegraded":"2024-01-24T14:16:09.373728+0000","last_fullsized":"2024-01-24T14:16:09.373728+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.346959+0000","last_change":"2024-01-24T14:16:08.934921+0000","last_active":"2024-01-24T14:16:09.346959+0000","last_peered":"2024-01-24T14:16:09.346959+0000","last_clean":"2024-01-24T14:16:09.346959+0000","last_became_active":"2024-01-24T14:16:08.934003+0000","last_became_peered":"2024-01-24T14:16:08.934003+0000","last_unstale":"2024-01-24T14:16:09.346959+0000","last_undegraded":"2024-01-24T14:16:09.346959+0000","last_fullsized":"2024-01-24T14:16:09.346959+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.341038+0000","last_change":"2024-01-24T14:16:08.936624+0000","last_active":"2024-01-24T14:16:09.341038+0000","last_peered":"2024-01-24T14:16:09.341038+0000","last_clean":"2024-01-24T14:16:09.341038+0000","last_became_active":"2024-01-24T14:16:08.933113+0000","last_became_peered":"2024-01-24T14:16:08.933113+0000","last_unstale":"2024-01-24T14:16:09.341038+0000","last_undegraded":"2024-01-24T14:16:09.341038+0000","last_fullsized":"2024-01-24T14:16:09.341038+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.340705+0000","last_change":"2024-01-24T14:16:08.937210+0000","last_active":"2024-01-24T14:16:09.340705+0000","last_peered":"2024-01-24T14:16:09.340705+0000","last_clean":"2024-01-24T14:16:09.340705+0000","last_became_active":"2024-01-24T14:16:08.936853+0000","last_became_peered":"2024-01-24T14:16:08.936853+0000","last_unstale":"2024-01-24T14:16:09.340705+0000","last_undegraded":"2024-01-24T14:16:09.340705+0000","last_fullsized":"2024-01-24T14:16:09.340705+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336642+0000","last_change":"2024-01-24T14:16:08.932135+0000","last_active":"2024-01-24T14:16:09.336642+0000","last_peered":"2024-01-24T14:16:09.336642+0000","last_clean":"2024-01-24T14:16:09.336642+0000","last_became_active":"2024-01-24T14:16:08.931750+0000","last_became_peered":"2024-01-24T14:16:08.931750+0000","last_unstale":"2024-01-24T14:16:09.336642+0000","last_undegraded":"2024-01-24T14:16:09.336642+0000","last_fullsized":"2024-01-24T14:16:09.336642+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373240+0000","last_change":"2024-01-24T14:16:09.366166+0000","last_active":"2024-01-24T14:16:09.373240+0000","last_peered":"2024-01-24T14:16:09.373240+0000","last_clean":"2024-01-24T14:16:09.373240+0000","last_became_active":"2024-01-24T14:16:09.365494+0000","last_became_peered":"2024-01-24T14:16:09.365494+0000","last_unstale":"2024-01-24T14:16:09.373240+0000","last_undegraded":"2024-01-24T14:16:09.373240+0000","last_fullsized":"2024-01-24T14:16:09.373240+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.355725+0000","last_change":"2024-01-24T14:16:08.929461+0000","last_active":"2024-01-24T14:16:09.355725+0000","last_peered":"2024-01-24T14:16:09.355725+0000","last_clean":"2024-01-24T14:16:09.355725+0000","last_became_active":"2024-01-24T14:16:08.927699+0000","last_became_peered":"2024-01-24T14:16:08.927699+0000","last_unstale":"2024-01-24T14:16:09.355725+0000","last_undegraded":"2024-01-24T14:16:09.355725+0000","last_fullsized":"2024-01-24T14:16:09.355725+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:08.906152+0000","last_change":"2024-01-24T14:16:08.906152+0000","last_active":"2024-01-24T14:16:08.906152+0000","last_peered":"2024-01-24T14:16:08.906152+0000","last_clean":"2024-01-24T14:16:08.906152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:08.906152+0000","last_undegraded":"2024-01-24T14:16:08.906152+0000","last_fullsized":"2024-01-24T14:16:08.906152+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.339976+0000","last_change":"2024-01-24T14:16:08.932446+0000","last_active":"2024-01-24T14:16:09.339976+0000","last_peered":"2024-01-24T14:16:09.339976+0000","last_clean":"2024-01-24T14:16:09.339976+0000","last_became_active":"2024-01-24T14:16:08.932185+0000","last_became_peered":"2024-01-24T14:16:08.932185+0000","last_unstale":"2024-01-24T14:16:09.339976+0000","last_undegraded":"2024-01-24T14:16:09.339976+0000","last_fullsized":"2024-01-24T14:16:09.339976+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350412+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:16:09.350412+0000","last_peered":"2024-01-24T14:16:09.350412+0000","last_clean":"2024-01-24T14:16:09.350412+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:16:09.350412+0000","last_undegraded":"2024-01-24T14:16:09.350412+0000","last_fullsized":"2024-01-24T14:16:09.350412+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360556+0000","last_change":"2024-01-24T14:16:09.349071+0000","last_active":"2024-01-24T14:16:09.360556+0000","last_peered":"2024-01-24T14:16:09.360556+0000","last_clean":"2024-01-24T14:16:09.360556+0000","last_became_active":"2024-01-24T14:16:09.347445+0000","last_became_peered":"2024-01-24T14:16:09.347445+0000","last_unstale":"2024-01-24T14:16:09.360556+0000","last_undegraded":"2024-01-24T14:16:09.360556+0000","last_fullsized":"2024-01-24T14:16:09.360556+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"136'4","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.361554+0000","last_change":"2024-01-24T14:16:08.933952+0000","last_active":"2024-01-24T14:16:09.361554+0000","last_peered":"2024-01-24T14:16:09.361554+0000","last_clean":"2024-01-24T14:16:09.361554+0000","last_became_active":"2024-01-24T14:16:08.933637+0000","last_became_peered":"2024-01-24T14:16:08.933637+0000","last_unstale":"2024-01-24T14:16:09.361554+0000","last_undegraded":"2024-01-24T14:16:09.361554+0000","last_fullsized":"2024-01-24T14:16:09.361554+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":124,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.342299+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:16:09.342299+0000","last_peered":"2024-01-24T14:16:09.342299+0000","last_clean":"2024-01-24T14:16:09.342299+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:16:09.342299+0000","last_undegraded":"2024-01-24T14:16:09.342299+0000","last_fullsized":"2024-01-24T14:16:09.342299+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360431+0000","last_change":"2024-01-24T14:16:09.346481+0000","last_active":"2024-01-24T14:16:09.360431+0000","last_peered":"2024-01-24T14:16:09.360431+0000","last_clean":"2024-01-24T14:16:09.360431+0000","last_became_active":"2024-01-24T14:16:09.346181+0000","last_became_peered":"2024-01-24T14:16:09.346181+0000","last_unstale":"2024-01-24T14:16:09.360431+0000","last_undegraded":"2024-01-24T14:16:09.360431+0000","last_fullsized":"2024-01-24T14:16:09.360431+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.363300+0000","last_change":"2024-01-24T14:16:08.936284+0000","last_active":"2024-01-24T14:16:09.363300+0000","last_peered":"2024-01-24T14:16:09.363300+0000","last_clean":"2024-01-24T14:16:09.363300+0000","last_became_active":"2024-01-24T14:16:08.935984+0000","last_became_peered":"2024-01-24T14:16:08.935984+0000","last_unstale":"2024-01-24T14:16:09.363300+0000","last_undegraded":"2024-01-24T14:16:09.363300+0000","last_fullsized":"2024-01-24T14:16:09.363300+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":119,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336449+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:16:09.336449+0000","last_peered":"2024-01-24T14:16:09.336449+0000","last_clean":"2024-01-24T14:16:09.336449+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:16:09.336449+0000","last_undegraded":"2024-01-24T14:16:09.336449+0000","last_fullsized":"2024-01-24T14:16:09.336449+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.362415+0000","last_change":"2024-01-24T14:16:09.355510+0000","last_active":"2024-01-24T14:16:09.362415+0000","last_peered":"2024-01-24T14:16:09.362415+0000","last_clean":"2024-01-24T14:16:09.362415+0000","last_became_active":"2024-01-24T14:16:09.355210+0000","last_became_peered":"2024-01-24T14:16:09.355210+0000","last_unstale":"2024-01-24T14:16:09.362415+0000","last_undegraded":"2024-01-24T14:16:09.362415+0000","last_fullsized":"2024-01-24T14:16:09.362415+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.341631+0000","last_change":"2024-01-24T14:16:08.942955+0000","last_active":"2024-01-24T14:16:09.341631+0000","last_peered":"2024-01-24T14:16:09.341631+0000","last_clean":"2024-01-24T14:16:09.341631+0000","last_became_active":"2024-01-24T14:16:08.942771+0000","last_became_peered":"2024-01-24T14:16:08.942771+0000","last_unstale":"2024-01-24T14:16:09.341631+0000","last_undegraded":"2024-01-24T14:16:09.341631+0000","last_fullsized":"2024-01-24T14:16:09.341631+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":124,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.340794+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:16:09.340794+0000","last_peered":"2024-01-24T14:16:09.340794+0000","last_clean":"2024-01-24T14:16:09.340794+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:16:09.340794+0000","last_undegraded":"2024-01-24T14:16:09.340794+0000","last_fullsized":"2024-01-24T14:16:09.340794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-01-24T14:16:04.619759+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:16:04.619759+0000","last_peered":"2024-01-24T14:16:04.619759+0000","last_clean":"2024-01-24T14:16:04.619759+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:16:04.619759+0000","last_undegraded":"2024-01-24T14:16:04.619759+0000","last_fullsized":"2024-01-24T14:16:04.619759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.362815+0000","last_change":"2024-01-24T14:16:09.356779+0000","last_active":"2024-01-24T14:16:09.362815+0000","last_peered":"2024-01-24T14:16:09.362815+0000","last_clean":"2024-01-24T14:16:09.362815+0000","last_became_active":"2024-01-24T14:16:09.356407+0000","last_became_peered":"2024-01-24T14:16:09.356407+0000","last_unstale":"2024-01-24T14:16:09.362815+0000","last_undegraded":"2024-01-24T14:16:09.362815+0000","last_fullsized":"2024-01-24T14:16:09.362815+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T14:16:07.900978+0000","last_change":"2024-01-24T14:16:07.900978+0000","last_active":"2024-01-24T14:16:07.900978+0000","last_peered":"2024-01-24T14:16:07.900978+0000","last_clean":"2024-01-24T14:16:07.900978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T14:16:07.900978+0000","last_undegraded":"2024-01-24T14:16:07.900978+0000","last_fullsized":"2024-01-24T14:16:07.900978+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-01-24T14:16:04.619757+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:16:04.619757+0000","last_peered":"2024-01-24T14:16:04.619757+0000","last_clean":"2024-01-24T14:16:04.619757+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:16:04.619757+0000","last_undegraded":"2024-01-24T14:16:04.619757+0000","last_fullsized":"2024-01-24T14:16:04.619757+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.358344+0000","last_change":"2024-01-24T14:16:09.350990+0000","last_active":"2024-01-24T14:16:09.358344+0000","last_peered":"2024-01-24T14:16:09.358344+0000","last_clean":"2024-01-24T14:16:09.358344+0000","last_became_active":"2024-01-24T14:16:09.350707+0000","last_became_peered":"2024-01-24T14:16:09.350707+0000","last_unstale":"2024-01-24T14:16:09.358344+0000","last_undegraded":"2024-01-24T14:16:09.358344+0000","last_fullsized":"2024-01-24T14:16:09.358344+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336786+0000","last_change":"2024-01-24T14:16:08.932090+0000","last_active":"2024-01-24T14:16:09.336786+0000","last_peered":"2024-01-24T14:16:09.336786+0000","last_clean":"2024-01-24T14:16:09.336786+0000","last_became_active":"2024-01-24T14:16:08.931350+0000","last_became_peered":"2024-01-24T14:16:08.931350+0000","last_unstale":"2024-01-24T14:16:09.336786+0000","last_undegraded":"2024-01-24T14:16:09.336786+0000","last_fullsized":"2024-01-24T14:16:09.336786+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-01-24T14:16:04.619791+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:16:04.619791+0000","last_peered":"2024-01-24T14:16:04.619791+0000","last_clean":"2024-01-24T14:16:04.619791+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:16:04.619791+0000","last_undegraded":"2024-01-24T14:16:04.619791+0000","last_fullsized":"2024-01-24T14:16:04.619791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360295+0000","last_change":"2024-01-24T14:16:09.355470+0000","last_active":"2024-01-24T14:16:09.360295+0000","last_peered":"2024-01-24T14:16:09.360295+0000","last_clean":"2024-01-24T14:16:09.360295+0000","last_became_active":"2024-01-24T14:16:09.355204+0000","last_became_peered":"2024-01-24T14:16:09.355204+0000","last_unstale":"2024-01-24T14:16:09.360295+0000","last_undegraded":"2024-01-24T14:16:09.360295+0000","last_fullsized":"2024-01-24T14:16:09.360295+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.374172+0000","last_change":"2024-01-24T14:16:08.946057+0000","last_active":"2024-01-24T14:16:09.374172+0000","last_peered":"2024-01-24T14:16:09.374172+0000","last_clean":"2024-01-24T14:16:09.374172+0000","last_became_active":"2024-01-24T14:16:08.945836+0000","last_became_peered":"2024-01-24T14:16:08.945836+0000","last_unstale":"2024-01-24T14:16:09.374172+0000","last_undegraded":"2024-01-24T14:16:09.374172+0000","last_fullsized":"2024-01-24T14:16:09.374172+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350665+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:16:09.350665+0000","last_peered":"2024-01-24T14:16:09.350665+0000","last_clean":"2024-01-24T14:16:09.350665+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:16:09.350665+0000","last_undegraded":"2024-01-24T14:16:09.350665+0000","last_fullsized":"2024-01-24T14:16:09.350665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.362950+0000","last_change":"2024-01-24T14:16:09.353064+0000","last_active":"2024-01-24T14:16:09.362950+0000","last_peered":"2024-01-24T14:16:09.362950+0000","last_clean":"2024-01-24T14:16:09.362950+0000","last_became_active":"2024-01-24T14:16:09.352722+0000","last_became_peered":"2024-01-24T14:16:09.352722+0000","last_unstale":"2024-01-24T14:16:09.362950+0000","last_undegraded":"2024-01-24T14:16:09.362950+0000","last_fullsized":"2024-01-24T14:16:09.362950+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"136'3","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.372384+0000","last_change":"2024-01-24T14:16:08.934729+0000","last_active":"2024-01-24T14:16:09.372384+0000","last_peered":"2024-01-24T14:16:09.372384+0000","last_clean":"2024-01-24T14:16:09.372384+0000","last_became_active":"2024-01-24T14:16:08.934448+0000","last_became_peered":"2024-01-24T14:16:08.934448+0000","last_unstale":"2024-01-24T14:16:09.372384+0000","last_undegraded":"2024-01-24T14:16:09.372384+0000","last_fullsized":"2024-01-24T14:16:09.372384+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":105,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-01-24T14:16:04.619742+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:16:04.619742+0000","last_peered":"2024-01-24T14:16:04.619742+0000","last_clean":"2024-01-24T14:16:04.619742+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:16:04.619742+0000","last_undegraded":"2024-01-24T14:16:04.619742+0000","last_fullsized":"2024-01-24T14:16:04.619742+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":74,"acting":74,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":1414,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":86,"acting":86,"num_store_stats":5},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575061,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302800,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069040,"statfs":{"total":96636764160,"available":96326696960,"internally_reserved":0,"allocated":1646592,"data_stored":922987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.441,"5min":0.36799999999999999,"15min":0.26000000000000001},"max":{"1min":0.80200000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.32700000000000001},"max":{"1min":0.82199999999999995,"5min":0.82499999999999996,"15min":0.83499999999999996},"last":0.49399999999999999}]},{"osd":1,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.39800000000000002,"15min":0.371},"max":{"1min":0.81299999999999994,"5min":0.85099999999999998,"15min":0.85599999999999998},"last":0.626},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77800000000000002,"5min":0.80600000000000005,"15min":0.80900000000000005},"last":0.46500000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40300000000000002},"max":{"1min":0.78000000000000003,"5min":0.872,"15min":0.876},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.60499999999999998},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.36799999999999999},"max":{"1min":0.77700000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59499999999999997}]},{"osd":3,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.47799999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.752,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.39300000000000002,"5min":0.38,"15min":0.35499999999999998},"max":{"1min":0.83199999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52400000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.60899999999999999,"5min":0.77000000000000002,"15min":0.84699999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47999999999999998,"15min":0.49099999999999999},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68200000000000005,"5min":0.71199999999999997,"15min":0.745},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.502,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.72499999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.42699999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71499999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.73499999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.34100000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80100000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.42099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.745,"5min":0.80700000000000005,"15min":0.87},"last":0.56399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575062,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302788,"kb_used_data":1596,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069052,"statfs":{"total":96636764160,"available":96326709248,"internally_reserved":0,"allocated":1634304,"data_stored":916015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.32700000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.752,"15min":0.85699999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58299999999999996,"15min":0.59299999999999997},"min":{"1min":0.441,"5min":0.35299999999999998,"15min":0.29999999999999999},"max":{"1min":0.82799999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47199999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.59199999999999997},"min":{"1min":0.432,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.88},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.42399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73899999999999999,"5min":0.78700000000000003,"15min":1.032},"last":0.45300000000000001}]},{"osd":2,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.75600000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.52100000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.62},"min":{"1min":0.41299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76200000000000001,"5min":0.82799999999999996,"15min":0.89200000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.81499999999999995,"5min":0.86799999999999999,"15min":1.2430000000000001},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.375,"5min":0.32200000000000001,"15min":0.28899999999999998},"max":{"1min":0.77400000000000002,"5min":0.83899999999999997,"15min":1.165},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.314},"max":{"1min":0.71499999999999997,"5min":0.74399999999999999,"15min":0.83199999999999996},"last":0.69499999999999995}]},{"osd":5,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.377,"5min":0.313,"15min":0.313},"max":{"1min":0.745,"5min":0.81100000000000005,"15min":0.86699999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.36499999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.745,"5min":0.80200000000000005,"15min":0.85699999999999998},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71899999999999997,"5min":0.75900000000000001,"15min":1.2070000000000001},"last":0.627},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.318,"15min":0.318},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.89600000000000002},"last":0.71499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575062,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302856,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068984,"statfs":{"total":96636764160,"available":96326639616,"internally_reserved":0,"allocated":1703936,"data_stored":953978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.50800000000000001,"15min":0.442},"min":{"1min":0.26400000000000001,"5min":0.245,"15min":0.23400000000000001},"max":{"1min":0.50800000000000001,"5min":15.298999999999999,"15min":15.298999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.54100000000000004,"15min":0.46999999999999997},"min":{"1min":0.31900000000000001,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.59399999999999997,"5min":14.654,"15min":14.654},"last":0.35799999999999998}]},{"osd":2,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.45200000000000001},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.54100000000000004,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":0.62,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46899999999999997}]},{"osd":3,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.31,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.55500000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.48699999999999999,"15min":0.48399999999999999},"min":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.56200000000000006,"5min":1.651,"15min":1.651},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64700000000000002,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":1.137},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.46500000000000002,"5min":0.45900000000000002,"15min":0.36499999999999999},"max":{"1min":0.89100000000000001,"5min":0.995,"15min":1.218},"last":0.54500000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69099999999999995,"15min":0.83899999999999997},"min":{"1min":0.53500000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83699999999999997,"5min":0.94999999999999996,"15min":38.808},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.81200000000000006},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.40500000000000003},"max":{"1min":0.89400000000000002,"5min":0.95099999999999996,"15min":38.823},"last":0.503}]},{"osd":6,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.44500000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.0509999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70699999999999996,"15min":0.72699999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.40000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1439999999999999,"15min":1.1879999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68500000000000005,"15min":0.69899999999999995},"min":{"1min":0.51300000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88300000000000001,"5min":0.995,"15min":1.155},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72699999999999998,"15min":0.751},"min":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.49399999999999999},"max":{"1min":0.92900000000000005,"5min":1.103,"15min":1.131},"last":0.69399999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542360,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306952,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064888,"statfs":{"total":96636764160,"available":96322445312,"internally_reserved":0,"allocated":1703936,"data_stored":946548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38700000000000001,"15min":0.39600000000000002},"min":{"1min":0.26900000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.59799999999999998,"5min":0.60899999999999999,"15min":0.69399999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.42699999999999999,"15min":0.42799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.223},"max":{"1min":0.48799999999999999,"5min":0.623,"15min":0.71699999999999997},"last":0.5}]},{"osd":2,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45200000000000001,"15min":0.44},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.28999999999999998},"max":{"1min":0.57699999999999996,"5min":0.66100000000000003,"15min":0.66700000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.46000000000000002},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.61599999999999999,"5min":0.754,"15min":0.79400000000000004},"last":0.45100000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.437,"15min":0.434},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.25900000000000001},"max":{"1min":0.55800000000000005,"5min":1.482,"15min":1.482},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49399999999999999,"15min":0.499},"min":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.32100000000000001},"max":{"1min":0.621,"5min":0.872,"15min":0.872},"last":0.59199999999999997}]},{"osd":4,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64400000000000002,"15min":0.65000000000000002},"min":{"1min":0.52400000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77400000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.50700000000000001,"5min":0.41899999999999998,"15min":0.40000000000000002},"max":{"1min":0.90000000000000002,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.52700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93200000000000005,"5min":1.655,"15min":1.655},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.68200000000000005},"min":{"1min":0.51000000000000001,"5min":0.46300000000000002,"15min":0.42999999999999999},"max":{"1min":0.86899999999999999,"5min":1.234,"15min":1.234},"last":0.79500000000000004}]},{"osd":6,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.57699999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90100000000000002,"5min":1.196,"15min":1.196},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92200000000000004,"5min":0.96499999999999997,"15min":1.117},"last":0.57899999999999996}]},{"osd":7,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.56200000000000006,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88700000000000001,"5min":0.93999999999999995,"15min":0.98199999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69299999999999995,"15min":0.69799999999999995},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.879,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.84099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575062,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1683456,"data_stored":946129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.24199999999999999},"max":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.65800000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42599999999999999,"15min":0.434},"min":{"1min":0.33500000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.67100000000000004},"last":0.40300000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42699999999999999,"15min":0.41599999999999998},"min":{"1min":0.29499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61099999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45300000000000001,"15min":0.46000000000000002},"min":{"1min":0.35399999999999998,"5min":0.30399999999999999,"15min":0.29699999999999999},"max":{"1min":0.58299999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.36399999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44800000000000001,"15min":0.45400000000000001},"min":{"1min":0.318,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58899999999999997,"5min":0.67700000000000005,"15min":0.71699999999999997},"last":0.372},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30299999999999999},"max":{"1min":0.628,"5min":0.70799999999999996,"15min":0.72799999999999998},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.628,"15min":0.64400000000000002},"min":{"1min":0.45400000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96699999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.748},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.65900000000000003,"15min":0.67200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.96599999999999997,"15min":1.042},"last":0.65200000000000002}]},{"osd":5,"last update":"Wed Jan 24 14:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65800000000000003,"15min":0.67900000000000005},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91500000000000004,"5min":1.036,"15min":1.0469999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85099999999999998,"5min":1.107,"15min":1.107},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.48899999999999999,"5min":0.48699999999999999,"15min":0.45600000000000002},"max":{"1min":0.874,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0529999999999999,"5min":1.141,"15min":1.141},"last":0.73399999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.71799999999999997},"min":{"1min":0.504,"5min":0.435,"15min":0.435},"max":{"1min":0.89800000000000002,"5min":0.96199999999999997,"15min":1.0409999999999999},"last":0.623},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71699999999999997,"15min":0.72799999999999998},"min":{"1min":0.497,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97999999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.70999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575062,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":955944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.46500000000000002,"15min":0.41799999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.22900000000000001},"max":{"1min":0.58899999999999997,"5min":7.593,"15min":7.593},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.499,"15min":0.44400000000000001},"min":{"1min":0.29399999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.60699999999999998,"5min":8.4239999999999995,"15min":8.4239999999999995},"last":0.38900000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55100000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45200000000000001,"15min":0.45500000000000002},"min":{"1min":0.32800000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.66200000000000003,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.442}]},{"osd":2,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48999999999999999,"15min":0.496},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.311},"max":{"1min":0.65000000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.68400000000000005,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.47599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.006,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65300000000000002,"15min":0.66200000000000003},"min":{"1min":0.497,"5min":0.437,"15min":0.42999999999999999},"max":{"1min":0.93799999999999994,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.755}]},{"osd":5,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.39900000000000002},"max":{"1min":1.03,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66600000000000004,"15min":0.68899999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97799999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.77800000000000002}]},{"osd":6,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68500000000000005,"15min":0.68999999999999995},"min":{"1min":0.53400000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.501,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.99099999999999999,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.72299999999999998}]},{"osd":7,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.44600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.97399999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.56299999999999994,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95799999999999996,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.64300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575062,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302800,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069040,"statfs":{"total":96636764160,"available":96326696960,"internally_reserved":0,"allocated":1646592,"data_stored":922987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.74299999999999999,"5min":0.83199999999999996,"15min":0.88600000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.439,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.74199999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.36699999999999999,"15min":0.33600000000000002},"max":{"1min":0.78500000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.44800000000000001,"5min":0.38200000000000001,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.46500000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74099999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80700000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.57499999999999996}]},{"osd":3,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.41999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.71599999999999997,"5min":0.79200000000000004,"15min":0.80600000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60399999999999998,"15min":0.61499999999999999},"min":{"1min":0.48899999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78300000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66200000000000003}]},{"osd":5,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.497,"15min":0.47899999999999998},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.27300000000000002},"max":{"1min":0.70599999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29199999999999998},"max":{"1min":0.66500000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.67100000000000004}]},{"osd":6,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50900000000000001,"15min":0.48899999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64400000000000002,"5min":0.79700000000000004,"15min":0.80700000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33200000000000002},"max":{"1min":0.70699999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60899999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.65700000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.622},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.72299999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575063,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":953876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53800000000000003,"15min":0.56799999999999995},"min":{"1min":0.41199999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70999999999999996,"5min":0.81399999999999995,"15min":1.0800000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.58699999999999997},"min":{"1min":0.42299999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54700000000000004,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79100000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.57899999999999996,"15min":0.59699999999999998},"min":{"1min":0.436,"5min":0.35199999999999998,"15min":0.29799999999999999},"max":{"1min":0.78500000000000003,"5min":0.79000000000000004,"15min":1.1639999999999999},"last":0.54600000000000004}]},{"osd":2,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.58999999999999997},"min":{"1min":0.39500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56699999999999995,"15min":0.58899999999999997},"min":{"1min":0.41999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.71699999999999997,"5min":0.77700000000000002,"15min":0.83699999999999997},"last":0.53100000000000003}]},{"osd":3,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58699999999999997,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":1.139},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.59299999999999997,"15min":0.61599999999999999},"min":{"1min":0.42899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.80800000000000005,"5min":0.83099999999999996,"15min":0.85399999999999998},"last":0.56599999999999995}]},{"osd":4,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.33700000000000002,"15min":0.30299999999999999},"max":{"1min":0.76800000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.498,"15min":0.501},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.30399999999999999},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.90000000000000002},"last":0.58399999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.505,"15min":0.505},"min":{"1min":0.36099999999999999,"5min":0.307,"15min":0.253},"max":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":1.1839999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.32000000000000001},"max":{"1min":0.72799999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60899999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.82799999999999996,"5min":0.95999999999999996,"15min":1.2030000000000001},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:16:13.464 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:16:13.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.630+0000 7fd4eeb15700 1 Processor -- start 2024-01-24T14:16:13.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.630+0000 7fd4eeb15700 1 -- start start 2024-01-24T14:16:13.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.631+0000 7fd4eeb15700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e8062ec0 0x7fd4e80c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.631+0000 7fd4eeb15700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e8065360 con 0x7fd4e81550f0 2024-01-24T14:16:13.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.631+0000 7fd4eeb15700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e80a6ce0 con 0x7fd4e81540a0 2024-01-24T14:16:13.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.631+0000 7fd4eeb15700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80a74f0 con 0x7fd4e8062ec0 2024-01-24T14:16:13.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.631+0000 7fd4ed0b2700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd4e81550f0 0x7fd4e8069240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48392/0 (socket says 172.21.15.29:48392) 2024-01-24T14:16:13.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.631+0000 7fd4ed0b2700 1 -- 172.21.15.29:0/884421383 learned_addr learned my addr 172.21.15.29:0/884421383 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:13.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.631+0000 7fd4e7fff700 1 --2- 172.21.15.29:0/884421383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e8062ec0 0x7fd4e80c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.631+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810500068 0 0) 0x7fd4e8065360 con 0x7fd4e81550f0 2024-01-24T14:16:13.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.631+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4d80033f0 con 0x7fd4e81550f0 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.631+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 27385830 0 0) 0x7fd4d80033f0 con 0x7fd4e81550f0 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.631+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e8062ec0 msgr2=0x7fd4e80c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.631+0000 7fd4e77fe700 1 --2- 172.21.15.29:0/884421383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e8062ec0 0x7fd4e80c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 >> v1:172.21.15.163:6789/0 conn(0x7fd4e81540a0 legacy=0x7fd4e80627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e80636f0 con 0x7fd4e81550f0 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fd4e00017f0 con 0x7fd4e81550f0 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 >> v1:172.21.15.29:6789/0 conn(0x7fd4e81550f0 legacy=0x7fd4e8069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 --2- 172.21.15.29:0/884421383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4d8003910 0x7fd4d8003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 --2- 172.21.15.29:0/884421383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e8062ec0 0x7fd4d8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 --2- 172.21.15.29:0/884421383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d8004b80 0x7fd4d8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80a74f0 con 0x7fd4d8004b80 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d800b8e0 con 0x7fd4d8003910 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d800ba50 con 0x7fd4e8062ec0 2024-01-24T14:16:13.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4ed0b2700 1 --2- 172.21.15.29:0/884421383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d8004b80 0x7fd4d8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4ec8b1700 1 --2- 172.21.15.29:0/884421383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4d8003910 0x7fd4d8003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e7fff700 1 --2- 172.21.15.29:0/884421383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e8062ec0 0x7fd4d8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4ed0b2700 1 -- 172.21.15.29:0/884421383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e8062ec0 msgr2=0x7fd4d8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4ed0b2700 1 --2- 172.21.15.29:0/884421383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e8062ec0 0x7fd4d8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4ed0b2700 1 -- 172.21.15.29:0/884421383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4d8003910 msgr2=0x7fd4d8003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4ed0b2700 1 --2- 172.21.15.29:0/884421383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4d8003910 0x7fd4d8003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4ed0b2700 1 -- 172.21.15.29:0/884421383 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d80033f0 con 0x7fd4d8004b80 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4ed0b2700 1 --2- 172.21.15.29:0/884421383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d8004b80 0x7fd4d8005000 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7fd4e000c5f0 tx=0x7fd4e0001d90).ready entity=mon.0 client_cookie=c5070a700e63aef7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000ce60 con 0x7fd4d8004b80 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e0001fc0 con 0x7fd4d8004b80 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.632+0000 7fd4e77fe700 1 -- 172.21.15.29:0/884421383 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0005d50 con 0x7fd4d8004b80 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.633+0000 7fd4eeb15700 1 -- 172.21.15.29:0/884421383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d8004b80 msgr2=0x7fd4d8005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.633+0000 7fd4eeb15700 1 --2- 172.21.15.29:0/884421383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d8004b80 0x7fd4d8005000 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7fd4e000c5f0 tx=0x7fd4e0001d90).stop 2024-01-24T14:16:13.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.633+0000 7fd4ed0b2700 1 -- 172.21.15.29:0/884421383 reap_dead start 2024-01-24T14:16:13.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.633+0000 7fd4eeb15700 1 -- 172.21.15.29:0/884421383 shutdown_connections 2024-01-24T14:16:13.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.633+0000 7fd4eeb15700 1 -- 172.21.15.29:0/884421383 >> 172.21.15.29:0/884421383 conn(0x7fd4e80c85c0 msgr2=0x7fd4e80c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:13.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.633+0000 7fd4eeb15700 1 -- 172.21.15.29:0/884421383 shutdown_connections 2024-01-24T14:16:13.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.633+0000 7fd4eeb15700 1 -- 172.21.15.29:0/884421383 wait complete. 2024-01-24T14:16:13.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.634+0000 7fd4eeb15700 1 Processor -- start 2024-01-24T14:16:13.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.635+0000 7fd4eeb15700 1 -- start start 2024-01-24T14:16:13.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.635+0000 7fd4eeb15700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d8004b80 0x7fd4e814f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.635+0000 7fd4eeb15700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e814f5d0 0x7fd4e814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.635+0000 7fd4eeb15700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4e814dfb0 0x7fd4e80c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.635+0000 7fd4eeb15700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c2f90 con 0x7fd4d8004b80 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.635+0000 7fd4eeb15700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c3270 con 0x7fd4e814dfb0 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.635+0000 7fd4eeb15700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c3550 con 0x7fd4e814f5d0 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4ec8b1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d8004b80 0x7fd4e814f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4ec8b1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d8004b80 0x7fd4e814f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51016/0 (socket says 172.21.15.29:51016) 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4ec8b1700 1 -- 172.21.15.29:0/1173989033 learned_addr learned my addr 172.21.15.29:0/1173989033 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4ed0b2700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4e814dfb0 0x7fd4e80c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4e7fff700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e814f5d0 0x7fd4e814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4ec8b1700 1 -- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e814f5d0 msgr2=0x7fd4e814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4ec8b1700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e814f5d0 0x7fd4e814dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4ec8b1700 1 -- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4e814dfb0 msgr2=0x7fd4e80c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4ec8b1700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4e814dfb0 0x7fd4e80c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4ec8b1700 1 -- 172.21.15.29:0/1173989033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e80c3830 con 0x7fd4d8004b80 2024-01-24T14:16:13.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4ed0b2700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4e814dfb0 0x7fd4e80c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:16:13.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4ec8b1700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d8004b80 0x7fd4e814f0e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fd4dc002a00 tx=0x7fd4dc007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:13.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4e4ff9700 1 -- 172.21.15.29:0/1173989033 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00f960 con 0x7fd4d8004b80 2024-01-24T14:16:13.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4e4ff9700 1 -- 172.21.15.29:0/1173989033 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc00fac0 con 0x7fd4d8004b80 2024-01-24T14:16:13.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4eeb15700 1 -- 172.21.15.29:0/1173989033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e80c3b10 con 0x7fd4d8004b80 2024-01-24T14:16:13.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4e4ff9700 1 -- 172.21.15.29:0/1173989033 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00fe40 con 0x7fd4d8004b80 2024-01-24T14:16:13.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.636+0000 7fd4eeb15700 1 -- 172.21.15.29:0/1173989033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e80c2790 con 0x7fd4d8004b80 2024-01-24T14:16:13.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.637+0000 7fd4eeb15700 1 -- 172.21.15.29:0/1173989033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e806a3c0 con 0x7fd4d8004b80 2024-01-24T14:16:13.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.638+0000 7fd4e4ff9700 1 -- 172.21.15.29:0/1173989033 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd4dc014660 con 0x7fd4d8004b80 2024-01-24T14:16:13.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.638+0000 7fd4e4ff9700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4cc05f270 0x7fd4cc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:13.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.638+0000 7fd4e4ff9700 1 -- 172.21.15.29:0/1173989033 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fd4dc07a7a0 con 0x7fd4d8004b80 2024-01-24T14:16:13.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.639+0000 7fd4e7fff700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4cc05f270 0x7fd4cc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:13.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.639+0000 7fd4e7fff700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4cc05f270 0x7fd4cc061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fd4e81527a0 tx=0x7fd4d4008040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:13.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.643+0000 7fd4e4ff9700 1 -- 172.21.15.29:0/1173989033 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd4dc04ab00 con 0x7fd4d8004b80 2024-01-24T14:16:13.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.823+0000 7fd4eeb15700 1 -- 172.21.15.29:0/1173989033 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4e80c3d00 con 0x7fd4cc05f270 2024-01-24T14:16:13.834 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.833+0000 7fd4e4ff9700 1 -- 172.21.15.29:0/1173989033 <== mgr.7195 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+362062 (secure 0 0 0) 0x7fd4e80c3d00 con 0x7fd4cc05f270 2024-01-24T14:16:13.838 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:16:13.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.837+0000 7fd4eeb15700 1 -- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4cc05f270 msgr2=0x7fd4cc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.837+0000 7fd4eeb15700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4cc05f270 0x7fd4cc061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fd4e81527a0 tx=0x7fd4d4008040).stop 2024-01-24T14:16:13.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.837+0000 7fd4eeb15700 1 -- 172.21.15.29:0/1173989033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d8004b80 msgr2=0x7fd4e814f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.837+0000 7fd4eeb15700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d8004b80 0x7fd4e814f0e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fd4dc002a00 tx=0x7fd4dc007b90).stop 2024-01-24T14:16:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.837+0000 7fd4eeb15700 1 -- 172.21.15.29:0/1173989033 shutdown_connections 2024-01-24T14:16:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.837+0000 7fd4eeb15700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4e814dfb0 0x7fd4e80c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.837+0000 7fd4eeb15700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4cc05f270 0x7fd4cc061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.837+0000 7fd4eeb15700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d8004b80 0x7fd4e814f0e0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.837+0000 7fd4eeb15700 1 --2- 172.21.15.29:0/1173989033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4e814f5d0 0x7fd4e814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.837+0000 7fd4eeb15700 1 -- 172.21.15.29:0/1173989033 >> 172.21.15.29:0/1173989033 conn(0x7fd4e80c85c0 msgr2=0x7fd4e80627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.837+0000 7fd4eeb15700 1 -- 172.21.15.29:0/1173989033 shutdown_connections 2024-01-24T14:16:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:13.837+0000 7fd4eeb15700 1 -- 172.21.15.29:0/1173989033 wait complete. 2024-01-24T14:16:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:16:13.851 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-01-24T14:16:13.618042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":240,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2431000,"kb_used_data":13272,"kb_used_omap":0,"kb_used_meta":2417664,"kb_avail":752543720,"statfs":{"total":773094113280,"available":770604769280,"internally_reserved":0,"allocated":13590528,"data_stored":7611308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2475687936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5922,"num_objects":-19,"num_object_clones":0,"num_object_copies":-38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.242373"},"pg_stats":[{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.359929+0000","last_change":"2024-01-24T14:16:09.352831+0000","last_active":"2024-01-24T14:16:09.359929+0000","last_peered":"2024-01-24T14:16:09.359929+0000","last_clean":"2024-01-24T14:16:09.359929+0000","last_became_active":"2024-01-24T14:16:09.352551+0000","last_became_peered":"2024-01-24T14:16:09.352551+0000","last_unstale":"2024-01-24T14:16:09.359929+0000","last_undegraded":"2024-01-24T14:16:09.359929+0000","last_fullsized":"2024-01-24T14:16:09.359929+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3f","version":"136'3","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.376003+0000","last_change":"2024-01-24T14:16:08.938387+0000","last_active":"2024-01-24T14:16:09.376003+0000","last_peered":"2024-01-24T14:16:09.376003+0000","last_clean":"2024-01-24T14:16:09.376003+0000","last_became_active":"2024-01-24T14:16:08.937623+0000","last_became_peered":"2024-01-24T14:16:08.937623+0000","last_unstale":"2024-01-24T14:16:09.376003+0000","last_undegraded":"2024-01-24T14:16:09.376003+0000","last_fullsized":"2024-01-24T14:16:09.376003+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373723+0000","last_change":"2024-01-24T14:16:09.369632+0000","last_active":"2024-01-24T14:16:09.373723+0000","last_peered":"2024-01-24T14:16:09.373723+0000","last_clean":"2024-01-24T14:16:09.373723+0000","last_became_active":"2024-01-24T14:16:09.369361+0000","last_became_peered":"2024-01-24T14:16:09.369361+0000","last_unstale":"2024-01-24T14:16:09.373723+0000","last_undegraded":"2024-01-24T14:16:09.373723+0000","last_fullsized":"2024-01-24T14:16:09.373723+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350167+0000","last_change":"2024-01-24T14:16:08.926041+0000","last_active":"2024-01-24T14:16:09.350167+0000","last_peered":"2024-01-24T14:16:09.350167+0000","last_clean":"2024-01-24T14:16:09.350167+0000","last_became_active":"2024-01-24T14:16:08.925750+0000","last_became_peered":"2024-01-24T14:16:08.925750+0000","last_unstale":"2024-01-24T14:16:09.350167+0000","last_undegraded":"2024-01-24T14:16:09.350167+0000","last_fullsized":"2024-01-24T14:16:09.350167+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.377049+0000","last_change":"2024-01-24T14:16:09.373720+0000","last_active":"2024-01-24T14:16:09.377049+0000","last_peered":"2024-01-24T14:16:09.377049+0000","last_clean":"2024-01-24T14:16:09.377049+0000","last_became_active":"2024-01-24T14:16:09.369847+0000","last_became_peered":"2024-01-24T14:16:09.369847+0000","last_unstale":"2024-01-24T14:16:09.377049+0000","last_undegraded":"2024-01-24T14:16:09.377049+0000","last_fullsized":"2024-01-24T14:16:09.377049+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.346582+0000","last_change":"2024-01-24T14:16:08.942908+0000","last_active":"2024-01-24T14:16:09.346582+0000","last_peered":"2024-01-24T14:16:09.346582+0000","last_clean":"2024-01-24T14:16:09.346582+0000","last_became_active":"2024-01-24T14:16:08.942564+0000","last_became_peered":"2024-01-24T14:16:08.942564+0000","last_unstale":"2024-01-24T14:16:09.346582+0000","last_undegraded":"2024-01-24T14:16:09.346582+0000","last_fullsized":"2024-01-24T14:16:09.346582+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367050+0000","last_change":"2024-01-24T14:16:09.362212+0000","last_active":"2024-01-24T14:16:09.367050+0000","last_peered":"2024-01-24T14:16:09.367050+0000","last_clean":"2024-01-24T14:16:09.367050+0000","last_became_active":"2024-01-24T14:16:09.361761+0000","last_became_peered":"2024-01-24T14:16:09.361761+0000","last_unstale":"2024-01-24T14:16:09.367050+0000","last_undegraded":"2024-01-24T14:16:09.367050+0000","last_fullsized":"2024-01-24T14:16:09.367050+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3c","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.353245+0000","last_change":"2024-01-24T14:16:08.936366+0000","last_active":"2024-01-24T14:16:09.353245+0000","last_peered":"2024-01-24T14:16:09.353245+0000","last_clean":"2024-01-24T14:16:09.353245+0000","last_became_active":"2024-01-24T14:16:08.932759+0000","last_became_peered":"2024-01-24T14:16:08.932759+0000","last_unstale":"2024-01-24T14:16:09.353245+0000","last_undegraded":"2024-01-24T14:16:09.353245+0000","last_fullsized":"2024-01-24T14:16:09.353245+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367519+0000","last_change":"2024-01-24T14:16:09.362617+0000","last_active":"2024-01-24T14:16:09.367519+0000","last_peered":"2024-01-24T14:16:09.367519+0000","last_clean":"2024-01-24T14:16:09.367519+0000","last_became_active":"2024-01-24T14:16:09.362309+0000","last_became_peered":"2024-01-24T14:16:09.362309+0000","last_unstale":"2024-01-24T14:16:09.367519+0000","last_undegraded":"2024-01-24T14:16:09.367519+0000","last_fullsized":"2024-01-24T14:16:09.367519+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3b","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360187+0000","last_change":"2024-01-24T14:16:08.930724+0000","last_active":"2024-01-24T14:16:09.360187+0000","last_peered":"2024-01-24T14:16:09.360187+0000","last_clean":"2024-01-24T14:16:09.360187+0000","last_became_active":"2024-01-24T14:16:08.929739+0000","last_became_peered":"2024-01-24T14:16:08.929739+0000","last_unstale":"2024-01-24T14:16:09.360187+0000","last_undegraded":"2024-01-24T14:16:09.360187+0000","last_fullsized":"2024-01-24T14:16:09.360187+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.357159+0000","last_change":"2024-01-24T14:16:09.350328+0000","last_active":"2024-01-24T14:16:09.357159+0000","last_peered":"2024-01-24T14:16:09.357159+0000","last_clean":"2024-01-24T14:16:09.357159+0000","last_became_active":"2024-01-24T14:16:09.350048+0000","last_became_peered":"2024-01-24T14:16:09.350048+0000","last_unstale":"2024-01-24T14:16:09.357159+0000","last_undegraded":"2024-01-24T14:16:09.357159+0000","last_fullsized":"2024-01-24T14:16:09.357159+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3a","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360086+0000","last_change":"2024-01-24T14:16:08.930376+0000","last_active":"2024-01-24T14:16:09.360086+0000","last_peered":"2024-01-24T14:16:09.360086+0000","last_clean":"2024-01-24T14:16:09.360086+0000","last_became_active":"2024-01-24T14:16:08.930096+0000","last_became_peered":"2024-01-24T14:16:08.930096+0000","last_unstale":"2024-01-24T14:16:09.360086+0000","last_undegraded":"2024-01-24T14:16:09.360086+0000","last_fullsized":"2024-01-24T14:16:09.360086+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.361032+0000","last_change":"2024-01-24T14:16:09.356412+0000","last_active":"2024-01-24T14:16:09.361032+0000","last_peered":"2024-01-24T14:16:09.361032+0000","last_clean":"2024-01-24T14:16:09.361032+0000","last_became_active":"2024-01-24T14:16:09.356087+0000","last_became_peered":"2024-01-24T14:16:09.356087+0000","last_unstale":"2024-01-24T14:16:09.361032+0000","last_undegraded":"2024-01-24T14:16:09.361032+0000","last_fullsized":"2024-01-24T14:16:09.361032+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.353185+0000","last_change":"2024-01-24T14:16:08.930911+0000","last_active":"2024-01-24T14:16:09.353185+0000","last_peered":"2024-01-24T14:16:09.353185+0000","last_clean":"2024-01-24T14:16:09.353185+0000","last_became_active":"2024-01-24T14:16:08.930624+0000","last_became_peered":"2024-01-24T14:16:08.930624+0000","last_unstale":"2024-01-24T14:16:09.353185+0000","last_undegraded":"2024-01-24T14:16:09.353185+0000","last_fullsized":"2024-01-24T14:16:09.353185+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.364819+0000","last_change":"2024-01-24T14:16:09.361939+0000","last_active":"2024-01-24T14:16:09.364819+0000","last_peered":"2024-01-24T14:16:09.364819+0000","last_clean":"2024-01-24T14:16:09.364819+0000","last_became_active":"2024-01-24T14:16:09.361601+0000","last_became_peered":"2024-01-24T14:16:09.361601+0000","last_unstale":"2024-01-24T14:16:09.364819+0000","last_undegraded":"2024-01-24T14:16:09.364819+0000","last_fullsized":"2024-01-24T14:16:09.364819+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336782+0000","last_change":"2024-01-24T14:16:08.919648+0000","last_active":"2024-01-24T14:16:09.336782+0000","last_peered":"2024-01-24T14:16:09.336782+0000","last_clean":"2024-01-24T14:16:09.336782+0000","last_became_active":"2024-01-24T14:16:08.918195+0000","last_became_peered":"2024-01-24T14:16:08.918195+0000","last_unstale":"2024-01-24T14:16:09.336782+0000","last_undegraded":"2024-01-24T14:16:09.336782+0000","last_fullsized":"2024-01-24T14:16:09.336782+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373200+0000","last_change":"2024-01-24T14:16:09.365741+0000","last_active":"2024-01-24T14:16:09.373200+0000","last_peered":"2024-01-24T14:16:09.373200+0000","last_clean":"2024-01-24T14:16:09.373200+0000","last_became_active":"2024-01-24T14:16:09.365444+0000","last_became_peered":"2024-01-24T14:16:09.365444+0000","last_unstale":"2024-01-24T14:16:09.373200+0000","last_undegraded":"2024-01-24T14:16:09.373200+0000","last_fullsized":"2024-01-24T14:16:09.373200+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350071+0000","last_change":"2024-01-24T14:16:08.932527+0000","last_active":"2024-01-24T14:16:09.350071+0000","last_peered":"2024-01-24T14:16:09.350071+0000","last_clean":"2024-01-24T14:16:09.350071+0000","last_became_active":"2024-01-24T14:16:08.932249+0000","last_became_peered":"2024-01-24T14:16:08.932249+0000","last_unstale":"2024-01-24T14:16:09.350071+0000","last_undegraded":"2024-01-24T14:16:09.350071+0000","last_fullsized":"2024-01-24T14:16:09.350071+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373747+0000","last_change":"2024-01-24T14:16:09.370446+0000","last_active":"2024-01-24T14:16:09.373747+0000","last_peered":"2024-01-24T14:16:09.373747+0000","last_clean":"2024-01-24T14:16:09.373747+0000","last_became_active":"2024-01-24T14:16:09.369908+0000","last_became_peered":"2024-01-24T14:16:09.369908+0000","last_unstale":"2024-01-24T14:16:09.373747+0000","last_undegraded":"2024-01-24T14:16:09.373747+0000","last_fullsized":"2024-01-24T14:16:09.373747+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350675+0000","last_change":"2024-01-24T14:16:08.929539+0000","last_active":"2024-01-24T14:16:09.350675+0000","last_peered":"2024-01-24T14:16:09.350675+0000","last_clean":"2024-01-24T14:16:09.350675+0000","last_became_active":"2024-01-24T14:16:08.929003+0000","last_became_peered":"2024-01-24T14:16:08.929003+0000","last_unstale":"2024-01-24T14:16:09.350675+0000","last_undegraded":"2024-01-24T14:16:09.350675+0000","last_fullsized":"2024-01-24T14:16:09.350675+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373705+0000","last_change":"2024-01-24T14:16:09.366328+0000","last_active":"2024-01-24T14:16:09.373705+0000","last_peered":"2024-01-24T14:16:09.373705+0000","last_clean":"2024-01-24T14:16:09.373705+0000","last_became_active":"2024-01-24T14:16:09.365624+0000","last_became_peered":"2024-01-24T14:16:09.365624+0000","last_unstale":"2024-01-24T14:16:09.373705+0000","last_undegraded":"2024-01-24T14:16:09.373705+0000","last_fullsized":"2024-01-24T14:16:09.373705+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.343446+0000","last_change":"2024-01-24T14:16:08.932084+0000","last_active":"2024-01-24T14:16:09.343446+0000","last_peered":"2024-01-24T14:16:09.343446+0000","last_clean":"2024-01-24T14:16:09.343446+0000","last_became_active":"2024-01-24T14:16:08.931666+0000","last_became_peered":"2024-01-24T14:16:08.931666+0000","last_unstale":"2024-01-24T14:16:09.343446+0000","last_undegraded":"2024-01-24T14:16:09.343446+0000","last_fullsized":"2024-01-24T14:16:09.343446+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.365061+0000","last_change":"2024-01-24T14:16:09.359372+0000","last_active":"2024-01-24T14:16:09.365061+0000","last_peered":"2024-01-24T14:16:09.365061+0000","last_clean":"2024-01-24T14:16:09.365061+0000","last_became_active":"2024-01-24T14:16:09.359080+0000","last_became_peered":"2024-01-24T14:16:09.359080+0000","last_unstale":"2024-01-24T14:16:09.365061+0000","last_undegraded":"2024-01-24T14:16:09.365061+0000","last_fullsized":"2024-01-24T14:16:09.365061+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.34","version":"136'4","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367954+0000","last_change":"2024-01-24T14:16:08.939673+0000","last_active":"2024-01-24T14:16:09.367954+0000","last_peered":"2024-01-24T14:16:09.367954+0000","last_clean":"2024-01-24T14:16:09.367954+0000","last_became_active":"2024-01-24T14:16:08.939397+0000","last_became_peered":"2024-01-24T14:16:08.939397+0000","last_unstale":"2024-01-24T14:16:09.367954+0000","last_undegraded":"2024-01-24T14:16:09.367954+0000","last_fullsized":"2024-01-24T14:16:09.367954+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.366973+0000","last_change":"2024-01-24T14:16:09.360974+0000","last_active":"2024-01-24T14:16:09.366973+0000","last_peered":"2024-01-24T14:16:09.366973+0000","last_clean":"2024-01-24T14:16:09.366973+0000","last_became_active":"2024-01-24T14:16:09.360689+0000","last_became_peered":"2024-01-24T14:16:09.360689+0000","last_unstale":"2024-01-24T14:16:09.366973+0000","last_undegraded":"2024-01-24T14:16:09.366973+0000","last_fullsized":"2024-01-24T14:16:09.366973+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.33","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350347+0000","last_change":"2024-01-24T14:16:08.939889+0000","last_active":"2024-01-24T14:16:09.350347+0000","last_peered":"2024-01-24T14:16:09.350347+0000","last_clean":"2024-01-24T14:16:09.350347+0000","last_became_active":"2024-01-24T14:16:08.939593+0000","last_became_peered":"2024-01-24T14:16:08.939593+0000","last_unstale":"2024-01-24T14:16:09.350347+0000","last_undegraded":"2024-01-24T14:16:09.350347+0000","last_fullsized":"2024-01-24T14:16:09.350347+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.366976+0000","last_change":"2024-01-24T14:16:09.361170+0000","last_active":"2024-01-24T14:16:09.366976+0000","last_peered":"2024-01-24T14:16:09.366976+0000","last_clean":"2024-01-24T14:16:09.366976+0000","last_became_active":"2024-01-24T14:16:09.357943+0000","last_became_peered":"2024-01-24T14:16:09.357943+0000","last_unstale":"2024-01-24T14:16:09.366976+0000","last_undegraded":"2024-01-24T14:16:09.366976+0000","last_fullsized":"2024-01-24T14:16:09.366976+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.351361+0000","last_change":"2024-01-24T14:16:08.936955+0000","last_active":"2024-01-24T14:16:09.351361+0000","last_peered":"2024-01-24T14:16:09.351361+0000","last_clean":"2024-01-24T14:16:09.351361+0000","last_became_active":"2024-01-24T14:16:08.936696+0000","last_became_peered":"2024-01-24T14:16:08.936696+0000","last_unstale":"2024-01-24T14:16:09.351361+0000","last_undegraded":"2024-01-24T14:16:09.351361+0000","last_fullsized":"2024-01-24T14:16:09.351361+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373521+0000","last_change":"2024-01-24T14:16:09.366785+0000","last_active":"2024-01-24T14:16:09.373521+0000","last_peered":"2024-01-24T14:16:09.373521+0000","last_clean":"2024-01-24T14:16:09.373521+0000","last_became_active":"2024-01-24T14:16:09.360406+0000","last_became_peered":"2024-01-24T14:16:09.360406+0000","last_unstale":"2024-01-24T14:16:09.373521+0000","last_undegraded":"2024-01-24T14:16:09.373521+0000","last_fullsized":"2024-01-24T14:16:09.373521+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350688+0000","last_change":"2024-01-24T14:16:08.938951+0000","last_active":"2024-01-24T14:16:09.350688+0000","last_peered":"2024-01-24T14:16:09.350688+0000","last_clean":"2024-01-24T14:16:09.350688+0000","last_became_active":"2024-01-24T14:16:08.938492+0000","last_became_peered":"2024-01-24T14:16:08.938492+0000","last_unstale":"2024-01-24T14:16:09.350688+0000","last_undegraded":"2024-01-24T14:16:09.350688+0000","last_fullsized":"2024-01-24T14:16:09.350688+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367775+0000","last_change":"2024-01-24T14:16:09.363463+0000","last_active":"2024-01-24T14:16:09.367775+0000","last_peered":"2024-01-24T14:16:09.367775+0000","last_clean":"2024-01-24T14:16:09.367775+0000","last_became_active":"2024-01-24T14:16:09.363031+0000","last_became_peered":"2024-01-24T14:16:09.363031+0000","last_unstale":"2024-01-24T14:16:09.367775+0000","last_undegraded":"2024-01-24T14:16:09.367775+0000","last_fullsized":"2024-01-24T14:16:09.367775+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.30","version":"136'4","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.363280+0000","last_change":"2024-01-24T14:16:08.938028+0000","last_active":"2024-01-24T14:16:09.363280+0000","last_peered":"2024-01-24T14:16:09.363280+0000","last_clean":"2024-01-24T14:16:09.363280+0000","last_became_active":"2024-01-24T14:16:08.937766+0000","last_became_peered":"2024-01-24T14:16:08.937766+0000","last_unstale":"2024-01-24T14:16:09.363280+0000","last_undegraded":"2024-01-24T14:16:09.363280+0000","last_fullsized":"2024-01-24T14:16:09.363280+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.365741+0000","last_change":"2024-01-24T14:16:09.360739+0000","last_active":"2024-01-24T14:16:09.365741+0000","last_peered":"2024-01-24T14:16:09.365741+0000","last_clean":"2024-01-24T14:16:09.365741+0000","last_became_active":"2024-01-24T14:16:09.360496+0000","last_became_peered":"2024-01-24T14:16:09.360496+0000","last_unstale":"2024-01-24T14:16:09.365741+0000","last_undegraded":"2024-01-24T14:16:09.365741+0000","last_fullsized":"2024-01-24T14:16:09.365741+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.356728+0000","last_change":"2024-01-24T14:16:08.944006+0000","last_active":"2024-01-24T14:16:09.356728+0000","last_peered":"2024-01-24T14:16:09.356728+0000","last_clean":"2024-01-24T14:16:09.356728+0000","last_became_active":"2024-01-24T14:16:08.943326+0000","last_became_peered":"2024-01-24T14:16:08.943326+0000","last_unstale":"2024-01-24T14:16:09.356728+0000","last_undegraded":"2024-01-24T14:16:09.356728+0000","last_fullsized":"2024-01-24T14:16:09.356728+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.365007+0000","last_change":"2024-01-24T14:16:09.359628+0000","last_active":"2024-01-24T14:16:09.365007+0000","last_peered":"2024-01-24T14:16:09.365007+0000","last_clean":"2024-01-24T14:16:09.365007+0000","last_became_active":"2024-01-24T14:16:09.359292+0000","last_became_peered":"2024-01-24T14:16:09.359292+0000","last_unstale":"2024-01-24T14:16:09.365007+0000","last_undegraded":"2024-01-24T14:16:09.365007+0000","last_fullsized":"2024-01-24T14:16:09.365007+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350286+0000","last_change":"2024-01-24T14:16:08.930363+0000","last_active":"2024-01-24T14:16:09.350286+0000","last_peered":"2024-01-24T14:16:09.350286+0000","last_clean":"2024-01-24T14:16:09.350286+0000","last_became_active":"2024-01-24T14:16:08.929928+0000","last_became_peered":"2024-01-24T14:16:08.929928+0000","last_unstale":"2024-01-24T14:16:09.350286+0000","last_undegraded":"2024-01-24T14:16:09.350286+0000","last_fullsized":"2024-01-24T14:16:09.350286+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360235+0000","last_change":"2024-01-24T14:16:09.355008+0000","last_active":"2024-01-24T14:16:09.360235+0000","last_peered":"2024-01-24T14:16:09.360235+0000","last_clean":"2024-01-24T14:16:09.360235+0000","last_became_active":"2024-01-24T14:16:09.354487+0000","last_became_peered":"2024-01-24T14:16:09.354487+0000","last_unstale":"2024-01-24T14:16:09.360235+0000","last_undegraded":"2024-01-24T14:16:09.360235+0000","last_fullsized":"2024-01-24T14:16:09.360235+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2d","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.369265+0000","last_change":"2024-01-24T14:16:08.945145+0000","last_active":"2024-01-24T14:16:09.369265+0000","last_peered":"2024-01-24T14:16:09.369265+0000","last_clean":"2024-01-24T14:16:09.369265+0000","last_became_active":"2024-01-24T14:16:08.944841+0000","last_became_peered":"2024-01-24T14:16:08.944841+0000","last_unstale":"2024-01-24T14:16:09.369265+0000","last_undegraded":"2024-01-24T14:16:09.369265+0000","last_fullsized":"2024-01-24T14:16:09.369265+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.366956+0000","last_change":"2024-01-24T14:16:09.361089+0000","last_active":"2024-01-24T14:16:09.366956+0000","last_peered":"2024-01-24T14:16:09.366956+0000","last_clean":"2024-01-24T14:16:09.366956+0000","last_became_active":"2024-01-24T14:16:09.359699+0000","last_became_peered":"2024-01-24T14:16:09.359699+0000","last_unstale":"2024-01-24T14:16:09.366956+0000","last_undegraded":"2024-01-24T14:16:09.366956+0000","last_fullsized":"2024-01-24T14:16:09.366956+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.342944+0000","last_change":"2024-01-24T14:16:08.937052+0000","last_active":"2024-01-24T14:16:09.342944+0000","last_peered":"2024-01-24T14:16:09.342944+0000","last_clean":"2024-01-24T14:16:09.342944+0000","last_became_active":"2024-01-24T14:16:08.936769+0000","last_became_peered":"2024-01-24T14:16:08.936769+0000","last_unstale":"2024-01-24T14:16:09.342944+0000","last_undegraded":"2024-01-24T14:16:09.342944+0000","last_fullsized":"2024-01-24T14:16:09.342944+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.363705+0000","last_change":"2024-01-24T14:16:09.361865+0000","last_active":"2024-01-24T14:16:09.363705+0000","last_peered":"2024-01-24T14:16:09.363705+0000","last_clean":"2024-01-24T14:16:09.363705+0000","last_became_active":"2024-01-24T14:16:09.361658+0000","last_became_peered":"2024-01-24T14:16:09.361658+0000","last_unstale":"2024-01-24T14:16:09.363705+0000","last_undegraded":"2024-01-24T14:16:09.363705+0000","last_fullsized":"2024-01-24T14:16:09.363705+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2b","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.349695+0000","last_change":"2024-01-24T14:16:08.938203+0000","last_active":"2024-01-24T14:16:09.349695+0000","last_peered":"2024-01-24T14:16:09.349695+0000","last_clean":"2024-01-24T14:16:09.349695+0000","last_became_active":"2024-01-24T14:16:08.937888+0000","last_became_peered":"2024-01-24T14:16:08.937888+0000","last_unstale":"2024-01-24T14:16:09.349695+0000","last_undegraded":"2024-01-24T14:16:09.349695+0000","last_fullsized":"2024-01-24T14:16:09.349695+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.359915+0000","last_change":"2024-01-24T14:16:09.352145+0000","last_active":"2024-01-24T14:16:09.359915+0000","last_peered":"2024-01-24T14:16:09.359915+0000","last_clean":"2024-01-24T14:16:09.359915+0000","last_became_active":"2024-01-24T14:16:09.351776+0000","last_became_peered":"2024-01-24T14:16:09.351776+0000","last_unstale":"2024-01-24T14:16:09.359915+0000","last_undegraded":"2024-01-24T14:16:09.359915+0000","last_fullsized":"2024-01-24T14:16:09.359915+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350511+0000","last_change":"2024-01-24T14:16:08.943928+0000","last_active":"2024-01-24T14:16:09.350511+0000","last_peered":"2024-01-24T14:16:09.350511+0000","last_clean":"2024-01-24T14:16:09.350511+0000","last_became_active":"2024-01-24T14:16:08.943463+0000","last_became_peered":"2024-01-24T14:16:08.943463+0000","last_unstale":"2024-01-24T14:16:09.350511+0000","last_undegraded":"2024-01-24T14:16:09.350511+0000","last_fullsized":"2024-01-24T14:16:09.350511+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.362435+0000","last_change":"2024-01-24T14:16:09.355594+0000","last_active":"2024-01-24T14:16:09.362435+0000","last_peered":"2024-01-24T14:16:09.362435+0000","last_clean":"2024-01-24T14:16:09.362435+0000","last_became_active":"2024-01-24T14:16:09.355308+0000","last_became_peered":"2024-01-24T14:16:09.355308+0000","last_unstale":"2024-01-24T14:16:09.362435+0000","last_undegraded":"2024-01-24T14:16:09.362435+0000","last_fullsized":"2024-01-24T14:16:09.362435+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.351402+0000","last_change":"2024-01-24T14:16:08.938907+0000","last_active":"2024-01-24T14:16:09.351402+0000","last_peered":"2024-01-24T14:16:09.351402+0000","last_clean":"2024-01-24T14:16:09.351402+0000","last_became_active":"2024-01-24T14:16:08.938612+0000","last_became_peered":"2024-01-24T14:16:08.938612+0000","last_unstale":"2024-01-24T14:16:09.351402+0000","last_undegraded":"2024-01-24T14:16:09.351402+0000","last_fullsized":"2024-01-24T14:16:09.351402+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360147+0000","last_change":"2024-01-24T14:16:09.354905+0000","last_active":"2024-01-24T14:16:09.360147+0000","last_peered":"2024-01-24T14:16:09.360147+0000","last_clean":"2024-01-24T14:16:09.360147+0000","last_became_active":"2024-01-24T14:16:09.354602+0000","last_became_peered":"2024-01-24T14:16:09.354602+0000","last_unstale":"2024-01-24T14:16:09.360147+0000","last_undegraded":"2024-01-24T14:16:09.360147+0000","last_fullsized":"2024-01-24T14:16:09.360147+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.349170+0000","last_change":"2024-01-24T14:16:08.935501+0000","last_active":"2024-01-24T14:16:09.349170+0000","last_peered":"2024-01-24T14:16:09.349170+0000","last_clean":"2024-01-24T14:16:09.349170+0000","last_became_active":"2024-01-24T14:16:08.935240+0000","last_became_peered":"2024-01-24T14:16:08.935240+0000","last_unstale":"2024-01-24T14:16:09.349170+0000","last_undegraded":"2024-01-24T14:16:09.349170+0000","last_fullsized":"2024-01-24T14:16:09.349170+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373150+0000","last_change":"2024-01-24T14:16:09.363397+0000","last_active":"2024-01-24T14:16:09.373150+0000","last_peered":"2024-01-24T14:16:09.373150+0000","last_clean":"2024-01-24T14:16:09.373150+0000","last_became_active":"2024-01-24T14:16:09.363107+0000","last_became_peered":"2024-01-24T14:16:09.363107+0000","last_unstale":"2024-01-24T14:16:09.373150+0000","last_undegraded":"2024-01-24T14:16:09.373150+0000","last_fullsized":"2024-01-24T14:16:09.373150+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.342438+0000","last_change":"2024-01-24T14:16:08.940206+0000","last_active":"2024-01-24T14:16:09.342438+0000","last_peered":"2024-01-24T14:16:09.342438+0000","last_clean":"2024-01-24T14:16:09.342438+0000","last_became_active":"2024-01-24T14:16:08.939945+0000","last_became_peered":"2024-01-24T14:16:08.939945+0000","last_unstale":"2024-01-24T14:16:09.342438+0000","last_undegraded":"2024-01-24T14:16:09.342438+0000","last_fullsized":"2024-01-24T14:16:09.342438+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.366117+0000","last_change":"2024-01-24T14:16:09.363983+0000","last_active":"2024-01-24T14:16:09.366117+0000","last_peered":"2024-01-24T14:16:09.366117+0000","last_clean":"2024-01-24T14:16:09.366117+0000","last_became_active":"2024-01-24T14:16:09.363697+0000","last_became_peered":"2024-01-24T14:16:09.363697+0000","last_unstale":"2024-01-24T14:16:09.366117+0000","last_undegraded":"2024-01-24T14:16:09.366117+0000","last_fullsized":"2024-01-24T14:16:09.366117+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.341202+0000","last_change":"2024-01-24T14:16:08.926623+0000","last_active":"2024-01-24T14:16:09.341202+0000","last_peered":"2024-01-24T14:16:09.341202+0000","last_clean":"2024-01-24T14:16:09.341202+0000","last_became_active":"2024-01-24T14:16:08.926299+0000","last_became_peered":"2024-01-24T14:16:08.926299+0000","last_unstale":"2024-01-24T14:16:09.341202+0000","last_undegraded":"2024-01-24T14:16:09.341202+0000","last_fullsized":"2024-01-24T14:16:09.341202+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373437+0000","last_change":"2024-01-24T14:16:09.370522+0000","last_active":"2024-01-24T14:16:09.373437+0000","last_peered":"2024-01-24T14:16:09.373437+0000","last_clean":"2024-01-24T14:16:09.373437+0000","last_became_active":"2024-01-24T14:16:09.370203+0000","last_became_peered":"2024-01-24T14:16:09.370203+0000","last_unstale":"2024-01-24T14:16:09.373437+0000","last_undegraded":"2024-01-24T14:16:09.373437+0000","last_fullsized":"2024-01-24T14:16:09.373437+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.343560+0000","last_change":"2024-01-24T14:16:08.935364+0000","last_active":"2024-01-24T14:16:09.343560+0000","last_peered":"2024-01-24T14:16:09.343560+0000","last_clean":"2024-01-24T14:16:09.343560+0000","last_became_active":"2024-01-24T14:16:08.935043+0000","last_became_peered":"2024-01-24T14:16:08.935043+0000","last_unstale":"2024-01-24T14:16:09.343560+0000","last_undegraded":"2024-01-24T14:16:09.343560+0000","last_fullsized":"2024-01-24T14:16:09.343560+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.374548+0000","last_change":"2024-01-24T14:16:09.371809+0000","last_active":"2024-01-24T14:16:09.374548+0000","last_peered":"2024-01-24T14:16:09.374548+0000","last_clean":"2024-01-24T14:16:09.374548+0000","last_became_active":"2024-01-24T14:16:09.371419+0000","last_became_peered":"2024-01-24T14:16:09.371419+0000","last_unstale":"2024-01-24T14:16:09.374548+0000","last_undegraded":"2024-01-24T14:16:09.374548+0000","last_fullsized":"2024-01-24T14:16:09.374548+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.348136+0000","last_change":"2024-01-24T14:16:08.937712+0000","last_active":"2024-01-24T14:16:09.348136+0000","last_peered":"2024-01-24T14:16:09.348136+0000","last_clean":"2024-01-24T14:16:09.348136+0000","last_became_active":"2024-01-24T14:16:08.937386+0000","last_became_peered":"2024-01-24T14:16:08.937386+0000","last_unstale":"2024-01-24T14:16:09.348136+0000","last_undegraded":"2024-01-24T14:16:09.348136+0000","last_fullsized":"2024-01-24T14:16:09.348136+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360483+0000","last_change":"2024-01-24T14:16:09.353820+0000","last_active":"2024-01-24T14:16:09.360483+0000","last_peered":"2024-01-24T14:16:09.360483+0000","last_clean":"2024-01-24T14:16:09.360483+0000","last_became_active":"2024-01-24T14:16:09.353300+0000","last_became_peered":"2024-01-24T14:16:09.353300+0000","last_unstale":"2024-01-24T14:16:09.360483+0000","last_undegraded":"2024-01-24T14:16:09.360483+0000","last_fullsized":"2024-01-24T14:16:09.360483+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336483+0000","last_change":"2024-01-24T14:16:08.922917+0000","last_active":"2024-01-24T14:16:09.336483+0000","last_peered":"2024-01-24T14:16:09.336483+0000","last_clean":"2024-01-24T14:16:09.336483+0000","last_became_active":"2024-01-24T14:16:08.922601+0000","last_became_peered":"2024-01-24T14:16:08.922601+0000","last_unstale":"2024-01-24T14:16:09.336483+0000","last_undegraded":"2024-01-24T14:16:09.336483+0000","last_fullsized":"2024-01-24T14:16:09.336483+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.362763+0000","last_change":"2024-01-24T14:16:09.357298+0000","last_active":"2024-01-24T14:16:09.362763+0000","last_peered":"2024-01-24T14:16:09.362763+0000","last_clean":"2024-01-24T14:16:09.362763+0000","last_became_active":"2024-01-24T14:16:09.356947+0000","last_became_peered":"2024-01-24T14:16:09.356947+0000","last_unstale":"2024-01-24T14:16:09.362763+0000","last_undegraded":"2024-01-24T14:16:09.362763+0000","last_fullsized":"2024-01-24T14:16:09.362763+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.342328+0000","last_change":"2024-01-24T14:16:08.937052+0000","last_active":"2024-01-24T14:16:09.342328+0000","last_peered":"2024-01-24T14:16:09.342328+0000","last_clean":"2024-01-24T14:16:09.342328+0000","last_became_active":"2024-01-24T14:16:08.936727+0000","last_became_peered":"2024-01-24T14:16:08.936727+0000","last_unstale":"2024-01-24T14:16:09.342328+0000","last_undegraded":"2024-01-24T14:16:09.342328+0000","last_fullsized":"2024-01-24T14:16:09.342328+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367500+0000","last_change":"2024-01-24T14:16:09.362620+0000","last_active":"2024-01-24T14:16:09.367500+0000","last_peered":"2024-01-24T14:16:09.367500+0000","last_clean":"2024-01-24T14:16:09.367500+0000","last_became_active":"2024-01-24T14:16:09.362292+0000","last_became_peered":"2024-01-24T14:16:09.362292+0000","last_unstale":"2024-01-24T14:16:09.367500+0000","last_undegraded":"2024-01-24T14:16:09.367500+0000","last_fullsized":"2024-01-24T14:16:09.367500+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.353289+0000","last_change":"2024-01-24T14:16:08.945388+0000","last_active":"2024-01-24T14:16:09.353289+0000","last_peered":"2024-01-24T14:16:09.353289+0000","last_clean":"2024-01-24T14:16:09.353289+0000","last_became_active":"2024-01-24T14:16:08.945096+0000","last_became_peered":"2024-01-24T14:16:08.945096+0000","last_unstale":"2024-01-24T14:16:09.353289+0000","last_undegraded":"2024-01-24T14:16:09.353289+0000","last_fullsized":"2024-01-24T14:16:09.353289+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373265+0000","last_change":"2024-01-24T14:16:09.366331+0000","last_active":"2024-01-24T14:16:09.373265+0000","last_peered":"2024-01-24T14:16:09.373265+0000","last_clean":"2024-01-24T14:16:09.373265+0000","last_became_active":"2024-01-24T14:16:09.365809+0000","last_became_peered":"2024-01-24T14:16:09.365809+0000","last_unstale":"2024-01-24T14:16:09.373265+0000","last_undegraded":"2024-01-24T14:16:09.373265+0000","last_fullsized":"2024-01-24T14:16:09.373265+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336883+0000","last_change":"2024-01-24T14:16:08.928995+0000","last_active":"2024-01-24T14:16:09.336883+0000","last_peered":"2024-01-24T14:16:09.336883+0000","last_clean":"2024-01-24T14:16:09.336883+0000","last_became_active":"2024-01-24T14:16:08.928664+0000","last_became_peered":"2024-01-24T14:16:08.928664+0000","last_unstale":"2024-01-24T14:16:09.336883+0000","last_undegraded":"2024-01-24T14:16:09.336883+0000","last_fullsized":"2024-01-24T14:16:09.336883+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.363319+0000","last_change":"2024-01-24T14:16:09.359848+0000","last_active":"2024-01-24T14:16:09.363319+0000","last_peered":"2024-01-24T14:16:09.363319+0000","last_clean":"2024-01-24T14:16:09.363319+0000","last_became_active":"2024-01-24T14:16:09.359438+0000","last_became_peered":"2024-01-24T14:16:09.359438+0000","last_unstale":"2024-01-24T14:16:09.363319+0000","last_undegraded":"2024-01-24T14:16:09.363319+0000","last_fullsized":"2024-01-24T14:16:09.363319+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.353921+0000","last_change":"2024-01-24T14:16:08.938305+0000","last_active":"2024-01-24T14:16:09.353921+0000","last_peered":"2024-01-24T14:16:09.353921+0000","last_clean":"2024-01-24T14:16:09.353921+0000","last_became_active":"2024-01-24T14:16:08.937907+0000","last_became_peered":"2024-01-24T14:16:08.937907+0000","last_unstale":"2024-01-24T14:16:09.353921+0000","last_undegraded":"2024-01-24T14:16:09.353921+0000","last_fullsized":"2024-01-24T14:16:09.353921+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.363693+0000","last_change":"2024-01-24T14:16:09.359168+0000","last_active":"2024-01-24T14:16:09.363693+0000","last_peered":"2024-01-24T14:16:09.363693+0000","last_clean":"2024-01-24T14:16:09.363693+0000","last_became_active":"2024-01-24T14:16:09.358852+0000","last_became_peered":"2024-01-24T14:16:09.358852+0000","last_unstale":"2024-01-24T14:16:09.363693+0000","last_undegraded":"2024-01-24T14:16:09.363693+0000","last_fullsized":"2024-01-24T14:16:09.363693+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360862+0000","last_change":"2024-01-24T14:16:08.938717+0000","last_active":"2024-01-24T14:16:09.360862+0000","last_peered":"2024-01-24T14:16:09.360862+0000","last_clean":"2024-01-24T14:16:09.360862+0000","last_became_active":"2024-01-24T14:16:08.938423+0000","last_became_peered":"2024-01-24T14:16:08.938423+0000","last_unstale":"2024-01-24T14:16:09.360862+0000","last_undegraded":"2024-01-24T14:16:09.360862+0000","last_fullsized":"2024-01-24T14:16:09.360862+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367746+0000","last_change":"2024-01-24T14:16:09.364666+0000","last_active":"2024-01-24T14:16:09.367746+0000","last_peered":"2024-01-24T14:16:09.367746+0000","last_clean":"2024-01-24T14:16:09.367746+0000","last_became_active":"2024-01-24T14:16:09.364380+0000","last_became_peered":"2024-01-24T14:16:09.364380+0000","last_unstale":"2024-01-24T14:16:09.367746+0000","last_undegraded":"2024-01-24T14:16:09.367746+0000","last_fullsized":"2024-01-24T14:16:09.367746+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350397+0000","last_change":"2024-01-24T14:16:08.931977+0000","last_active":"2024-01-24T14:16:09.350397+0000","last_peered":"2024-01-24T14:16:09.350397+0000","last_clean":"2024-01-24T14:16:09.350397+0000","last_became_active":"2024-01-24T14:16:08.931639+0000","last_became_peered":"2024-01-24T14:16:08.931639+0000","last_unstale":"2024-01-24T14:16:09.350397+0000","last_undegraded":"2024-01-24T14:16:09.350397+0000","last_fullsized":"2024-01-24T14:16:09.350397+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367068+0000","last_change":"2024-01-24T14:16:09.363042+0000","last_active":"2024-01-24T14:16:09.367068+0000","last_peered":"2024-01-24T14:16:09.367068+0000","last_clean":"2024-01-24T14:16:09.367068+0000","last_became_active":"2024-01-24T14:16:09.362673+0000","last_became_peered":"2024-01-24T14:16:09.362673+0000","last_unstale":"2024-01-24T14:16:09.367068+0000","last_undegraded":"2024-01-24T14:16:09.367068+0000","last_fullsized":"2024-01-24T14:16:09.367068+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336246+0000","last_change":"2024-01-24T14:16:08.931939+0000","last_active":"2024-01-24T14:16:09.336246+0000","last_peered":"2024-01-24T14:16:09.336246+0000","last_clean":"2024-01-24T14:16:09.336246+0000","last_became_active":"2024-01-24T14:16:08.931609+0000","last_became_peered":"2024-01-24T14:16:08.931609+0000","last_unstale":"2024-01-24T14:16:09.336246+0000","last_undegraded":"2024-01-24T14:16:09.336246+0000","last_fullsized":"2024-01-24T14:16:09.336246+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373093+0000","last_change":"2024-01-24T14:16:09.363363+0000","last_active":"2024-01-24T14:16:09.373093+0000","last_peered":"2024-01-24T14:16:09.373093+0000","last_clean":"2024-01-24T14:16:09.373093+0000","last_became_active":"2024-01-24T14:16:09.363039+0000","last_became_peered":"2024-01-24T14:16:09.363039+0000","last_unstale":"2024-01-24T14:16:09.373093+0000","last_undegraded":"2024-01-24T14:16:09.373093+0000","last_fullsized":"2024-01-24T14:16:09.373093+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.340588+0000","last_change":"2024-01-24T14:16:08.947795+0000","last_active":"2024-01-24T14:16:09.340588+0000","last_peered":"2024-01-24T14:16:09.340588+0000","last_clean":"2024-01-24T14:16:09.340588+0000","last_became_active":"2024-01-24T14:16:08.947549+0000","last_became_peered":"2024-01-24T14:16:08.947549+0000","last_unstale":"2024-01-24T14:16:09.340588+0000","last_undegraded":"2024-01-24T14:16:09.340588+0000","last_fullsized":"2024-01-24T14:16:09.340588+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.359967+0000","last_change":"2024-01-24T14:16:09.353818+0000","last_active":"2024-01-24T14:16:09.359967+0000","last_peered":"2024-01-24T14:16:09.359967+0000","last_clean":"2024-01-24T14:16:09.359967+0000","last_became_active":"2024-01-24T14:16:09.353536+0000","last_became_peered":"2024-01-24T14:16:09.353536+0000","last_unstale":"2024-01-24T14:16:09.359967+0000","last_undegraded":"2024-01-24T14:16:09.359967+0000","last_fullsized":"2024-01-24T14:16:09.359967+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.340926+0000","last_change":"2024-01-24T14:16:08.931616+0000","last_active":"2024-01-24T14:16:09.340926+0000","last_peered":"2024-01-24T14:16:09.340926+0000","last_clean":"2024-01-24T14:16:09.340926+0000","last_became_active":"2024-01-24T14:16:08.931287+0000","last_became_peered":"2024-01-24T14:16:08.931287+0000","last_unstale":"2024-01-24T14:16:09.340926+0000","last_undegraded":"2024-01-24T14:16:09.340926+0000","last_fullsized":"2024-01-24T14:16:09.340926+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373545+0000","last_change":"2024-01-24T14:16:09.367674+0000","last_active":"2024-01-24T14:16:09.373545+0000","last_peered":"2024-01-24T14:16:09.373545+0000","last_clean":"2024-01-24T14:16:09.373545+0000","last_became_active":"2024-01-24T14:16:09.367413+0000","last_became_peered":"2024-01-24T14:16:09.367413+0000","last_unstale":"2024-01-24T14:16:09.373545+0000","last_undegraded":"2024-01-24T14:16:09.373545+0000","last_fullsized":"2024-01-24T14:16:09.373545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.342828+0000","last_change":"2024-01-24T14:16:08.931252+0000","last_active":"2024-01-24T14:16:09.342828+0000","last_peered":"2024-01-24T14:16:09.342828+0000","last_clean":"2024-01-24T14:16:09.342828+0000","last_became_active":"2024-01-24T14:16:08.930913+0000","last_became_peered":"2024-01-24T14:16:08.930913+0000","last_unstale":"2024-01-24T14:16:09.342828+0000","last_undegraded":"2024-01-24T14:16:09.342828+0000","last_fullsized":"2024-01-24T14:16:09.342828+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.364800+0000","last_change":"2024-01-24T14:16:09.361864+0000","last_active":"2024-01-24T14:16:09.364800+0000","last_peered":"2024-01-24T14:16:09.364800+0000","last_clean":"2024-01-24T14:16:09.364800+0000","last_became_active":"2024-01-24T14:16:09.361500+0000","last_became_peered":"2024-01-24T14:16:09.361500+0000","last_unstale":"2024-01-24T14:16:09.364800+0000","last_undegraded":"2024-01-24T14:16:09.364800+0000","last_fullsized":"2024-01-24T14:16:09.364800+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.355492+0000","last_change":"2024-01-24T14:16:08.939533+0000","last_active":"2024-01-24T14:16:09.355492+0000","last_peered":"2024-01-24T14:16:09.355492+0000","last_clean":"2024-01-24T14:16:09.355492+0000","last_became_active":"2024-01-24T14:16:08.939114+0000","last_became_peered":"2024-01-24T14:16:08.939114+0000","last_unstale":"2024-01-24T14:16:09.355492+0000","last_undegraded":"2024-01-24T14:16:09.355492+0000","last_fullsized":"2024-01-24T14:16:09.355492+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.369114+0000","last_change":"2024-01-24T14:16:09.368019+0000","last_active":"2024-01-24T14:16:09.369114+0000","last_peered":"2024-01-24T14:16:09.369114+0000","last_clean":"2024-01-24T14:16:09.369114+0000","last_became_active":"2024-01-24T14:16:09.367477+0000","last_became_peered":"2024-01-24T14:16:09.367477+0000","last_unstale":"2024-01-24T14:16:09.369114+0000","last_undegraded":"2024-01-24T14:16:09.369114+0000","last_fullsized":"2024-01-24T14:16:09.369114+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.374688+0000","last_change":"2024-01-24T14:16:08.944181+0000","last_active":"2024-01-24T14:16:09.374688+0000","last_peered":"2024-01-24T14:16:09.374688+0000","last_clean":"2024-01-24T14:16:09.374688+0000","last_became_active":"2024-01-24T14:16:08.943722+0000","last_became_peered":"2024-01-24T14:16:08.943722+0000","last_unstale":"2024-01-24T14:16:09.374688+0000","last_undegraded":"2024-01-24T14:16:09.374688+0000","last_fullsized":"2024-01-24T14:16:09.374688+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.376650+0000","last_change":"2024-01-24T14:16:09.375052+0000","last_active":"2024-01-24T14:16:09.376650+0000","last_peered":"2024-01-24T14:16:09.376650+0000","last_clean":"2024-01-24T14:16:09.376650+0000","last_became_active":"2024-01-24T14:16:09.374764+0000","last_became_peered":"2024-01-24T14:16:09.374764+0000","last_unstale":"2024-01-24T14:16:09.376650+0000","last_undegraded":"2024-01-24T14:16:09.376650+0000","last_fullsized":"2024-01-24T14:16:09.376650+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.352207+0000","last_change":"2024-01-24T14:16:08.927699+0000","last_active":"2024-01-24T14:16:09.352207+0000","last_peered":"2024-01-24T14:16:09.352207+0000","last_clean":"2024-01-24T14:16:09.352207+0000","last_became_active":"2024-01-24T14:16:08.927375+0000","last_became_peered":"2024-01-24T14:16:08.927375+0000","last_unstale":"2024-01-24T14:16:09.352207+0000","last_undegraded":"2024-01-24T14:16:09.352207+0000","last_fullsized":"2024-01-24T14:16:09.352207+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.375867+0000","last_change":"2024-01-24T14:16:09.374839+0000","last_active":"2024-01-24T14:16:09.375867+0000","last_peered":"2024-01-24T14:16:09.375867+0000","last_clean":"2024-01-24T14:16:09.375867+0000","last_became_active":"2024-01-24T14:16:09.374363+0000","last_became_peered":"2024-01-24T14:16:09.374363+0000","last_unstale":"2024-01-24T14:16:09.375867+0000","last_undegraded":"2024-01-24T14:16:09.375867+0000","last_fullsized":"2024-01-24T14:16:09.375867+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.349482+0000","last_change":"2024-01-24T14:16:08.932533+0000","last_active":"2024-01-24T14:16:09.349482+0000","last_peered":"2024-01-24T14:16:09.349482+0000","last_clean":"2024-01-24T14:16:09.349482+0000","last_became_active":"2024-01-24T14:16:08.932252+0000","last_became_peered":"2024-01-24T14:16:08.932252+0000","last_unstale":"2024-01-24T14:16:09.349482+0000","last_undegraded":"2024-01-24T14:16:09.349482+0000","last_fullsized":"2024-01-24T14:16:09.349482+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367025+0000","last_change":"2024-01-24T14:16:09.362032+0000","last_active":"2024-01-24T14:16:09.367025+0000","last_peered":"2024-01-24T14:16:09.367025+0000","last_clean":"2024-01-24T14:16:09.367025+0000","last_became_active":"2024-01-24T14:16:09.361366+0000","last_became_peered":"2024-01-24T14:16:09.361366+0000","last_unstale":"2024-01-24T14:16:09.367025+0000","last_undegraded":"2024-01-24T14:16:09.367025+0000","last_fullsized":"2024-01-24T14:16:09.367025+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.340477+0000","last_change":"2024-01-24T14:16:08.938132+0000","last_active":"2024-01-24T14:16:09.340477+0000","last_peered":"2024-01-24T14:16:09.340477+0000","last_clean":"2024-01-24T14:16:09.340477+0000","last_became_active":"2024-01-24T14:16:08.937761+0000","last_became_peered":"2024-01-24T14:16:08.937761+0000","last_unstale":"2024-01-24T14:16:09.340477+0000","last_undegraded":"2024-01-24T14:16:09.340477+0000","last_fullsized":"2024-01-24T14:16:09.340477+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367348+0000","last_change":"2024-01-24T14:16:09.364901+0000","last_active":"2024-01-24T14:16:09.367348+0000","last_peered":"2024-01-24T14:16:09.367348+0000","last_clean":"2024-01-24T14:16:09.367348+0000","last_became_active":"2024-01-24T14:16:09.364555+0000","last_became_peered":"2024-01-24T14:16:09.364555+0000","last_unstale":"2024-01-24T14:16:09.367348+0000","last_undegraded":"2024-01-24T14:16:09.367348+0000","last_fullsized":"2024-01-24T14:16:09.367348+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.357408+0000","last_change":"2024-01-24T14:16:08.942263+0000","last_active":"2024-01-24T14:16:09.357408+0000","last_peered":"2024-01-24T14:16:09.357408+0000","last_clean":"2024-01-24T14:16:09.357408+0000","last_became_active":"2024-01-24T14:16:08.930700+0000","last_became_peered":"2024-01-24T14:16:08.930700+0000","last_unstale":"2024-01-24T14:16:09.357408+0000","last_undegraded":"2024-01-24T14:16:09.357408+0000","last_fullsized":"2024-01-24T14:16:09.357408+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.377044+0000","last_change":"2024-01-24T14:16:09.373474+0000","last_active":"2024-01-24T14:16:09.377044+0000","last_peered":"2024-01-24T14:16:09.377044+0000","last_clean":"2024-01-24T14:16:09.377044+0000","last_became_active":"2024-01-24T14:16:09.373140+0000","last_became_peered":"2024-01-24T14:16:09.373140+0000","last_unstale":"2024-01-24T14:16:09.377044+0000","last_undegraded":"2024-01-24T14:16:09.377044+0000","last_fullsized":"2024-01-24T14:16:09.377044+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336238+0000","last_change":"2024-01-24T14:16:08.931869+0000","last_active":"2024-01-24T14:16:09.336238+0000","last_peered":"2024-01-24T14:16:09.336238+0000","last_clean":"2024-01-24T14:16:09.336238+0000","last_became_active":"2024-01-24T14:16:08.931534+0000","last_became_peered":"2024-01-24T14:16:08.931534+0000","last_unstale":"2024-01-24T14:16:09.336238+0000","last_undegraded":"2024-01-24T14:16:09.336238+0000","last_fullsized":"2024-01-24T14:16:09.336238+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.361053+0000","last_change":"2024-01-24T14:16:09.358187+0000","last_active":"2024-01-24T14:16:09.361053+0000","last_peered":"2024-01-24T14:16:09.361053+0000","last_clean":"2024-01-24T14:16:09.361053+0000","last_became_active":"2024-01-24T14:16:09.357853+0000","last_became_peered":"2024-01-24T14:16:09.357853+0000","last_unstale":"2024-01-24T14:16:09.361053+0000","last_undegraded":"2024-01-24T14:16:09.361053+0000","last_fullsized":"2024-01-24T14:16:09.361053+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.349081+0000","last_change":"2024-01-24T14:16:08.934952+0000","last_active":"2024-01-24T14:16:09.349081+0000","last_peered":"2024-01-24T14:16:09.349081+0000","last_clean":"2024-01-24T14:16:09.349081+0000","last_became_active":"2024-01-24T14:16:08.934586+0000","last_became_peered":"2024-01-24T14:16:08.934586+0000","last_unstale":"2024-01-24T14:16:09.349081+0000","last_undegraded":"2024-01-24T14:16:09.349081+0000","last_fullsized":"2024-01-24T14:16:09.349081+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373572+0000","last_change":"2024-01-24T14:16:09.369996+0000","last_active":"2024-01-24T14:16:09.373572+0000","last_peered":"2024-01-24T14:16:09.373572+0000","last_clean":"2024-01-24T14:16:09.373572+0000","last_became_active":"2024-01-24T14:16:09.369677+0000","last_became_peered":"2024-01-24T14:16:09.369677+0000","last_unstale":"2024-01-24T14:16:09.373572+0000","last_undegraded":"2024-01-24T14:16:09.373572+0000","last_fullsized":"2024-01-24T14:16:09.373572+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.355605+0000","last_change":"2024-01-24T14:16:08.939792+0000","last_active":"2024-01-24T14:16:09.355605+0000","last_peered":"2024-01-24T14:16:09.355605+0000","last_clean":"2024-01-24T14:16:09.355605+0000","last_became_active":"2024-01-24T14:16:08.939486+0000","last_became_peered":"2024-01-24T14:16:08.939486+0000","last_unstale":"2024-01-24T14:16:09.355605+0000","last_undegraded":"2024-01-24T14:16:09.355605+0000","last_fullsized":"2024-01-24T14:16:09.355605+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.375752+0000","last_change":"2024-01-24T14:16:09.374526+0000","last_active":"2024-01-24T14:16:09.375752+0000","last_peered":"2024-01-24T14:16:09.375752+0000","last_clean":"2024-01-24T14:16:09.375752+0000","last_became_active":"2024-01-24T14:16:09.374201+0000","last_became_peered":"2024-01-24T14:16:09.374201+0000","last_unstale":"2024-01-24T14:16:09.375752+0000","last_undegraded":"2024-01-24T14:16:09.375752+0000","last_fullsized":"2024-01-24T14:16:09.375752+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.369957+0000","last_change":"2024-01-24T14:16:08.945586+0000","last_active":"2024-01-24T14:16:09.369957+0000","last_peered":"2024-01-24T14:16:09.369957+0000","last_clean":"2024-01-24T14:16:09.369957+0000","last_became_active":"2024-01-24T14:16:08.945238+0000","last_became_peered":"2024-01-24T14:16:08.945238+0000","last_unstale":"2024-01-24T14:16:09.369957+0000","last_undegraded":"2024-01-24T14:16:09.369957+0000","last_fullsized":"2024-01-24T14:16:09.369957+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373766+0000","last_change":"2024-01-24T14:16:09.371183+0000","last_active":"2024-01-24T14:16:09.373766+0000","last_peered":"2024-01-24T14:16:09.373766+0000","last_clean":"2024-01-24T14:16:09.373766+0000","last_became_active":"2024-01-24T14:16:09.369722+0000","last_became_peered":"2024-01-24T14:16:09.369722+0000","last_unstale":"2024-01-24T14:16:09.373766+0000","last_undegraded":"2024-01-24T14:16:09.373766+0000","last_fullsized":"2024-01-24T14:16:09.373766+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.355429+0000","last_change":"2024-01-24T14:16:08.932584+0000","last_active":"2024-01-24T14:16:09.355429+0000","last_peered":"2024-01-24T14:16:09.355429+0000","last_clean":"2024-01-24T14:16:09.355429+0000","last_became_active":"2024-01-24T14:16:08.931906+0000","last_became_peered":"2024-01-24T14:16:08.931906+0000","last_unstale":"2024-01-24T14:16:09.355429+0000","last_undegraded":"2024-01-24T14:16:09.355429+0000","last_fullsized":"2024-01-24T14:16:09.355429+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373728+0000","last_change":"2024-01-24T14:16:09.366503+0000","last_active":"2024-01-24T14:16:09.373728+0000","last_peered":"2024-01-24T14:16:09.373728+0000","last_clean":"2024-01-24T14:16:09.373728+0000","last_became_active":"2024-01-24T14:16:09.366048+0000","last_became_peered":"2024-01-24T14:16:09.366048+0000","last_unstale":"2024-01-24T14:16:09.373728+0000","last_undegraded":"2024-01-24T14:16:09.373728+0000","last_fullsized":"2024-01-24T14:16:09.373728+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.346959+0000","last_change":"2024-01-24T14:16:08.934921+0000","last_active":"2024-01-24T14:16:09.346959+0000","last_peered":"2024-01-24T14:16:09.346959+0000","last_clean":"2024-01-24T14:16:09.346959+0000","last_became_active":"2024-01-24T14:16:08.934003+0000","last_became_peered":"2024-01-24T14:16:08.934003+0000","last_unstale":"2024-01-24T14:16:09.346959+0000","last_undegraded":"2024-01-24T14:16:09.346959+0000","last_fullsized":"2024-01-24T14:16:09.346959+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.375755+0000","last_change":"2024-01-24T14:16:09.374603+0000","last_active":"2024-01-24T14:16:09.375755+0000","last_peered":"2024-01-24T14:16:09.375755+0000","last_clean":"2024-01-24T14:16:09.375755+0000","last_became_active":"2024-01-24T14:16:09.374160+0000","last_became_peered":"2024-01-24T14:16:09.374160+0000","last_unstale":"2024-01-24T14:16:09.375755+0000","last_undegraded":"2024-01-24T14:16:09.375755+0000","last_fullsized":"2024-01-24T14:16:09.375755+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.341038+0000","last_change":"2024-01-24T14:16:08.936624+0000","last_active":"2024-01-24T14:16:09.341038+0000","last_peered":"2024-01-24T14:16:09.341038+0000","last_clean":"2024-01-24T14:16:09.341038+0000","last_became_active":"2024-01-24T14:16:08.933113+0000","last_became_peered":"2024-01-24T14:16:08.933113+0000","last_unstale":"2024-01-24T14:16:09.341038+0000","last_undegraded":"2024-01-24T14:16:09.341038+0000","last_fullsized":"2024-01-24T14:16:09.341038+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.362451+0000","last_change":"2024-01-24T14:16:09.352155+0000","last_active":"2024-01-24T14:16:09.362451+0000","last_peered":"2024-01-24T14:16:09.362451+0000","last_clean":"2024-01-24T14:16:09.362451+0000","last_became_active":"2024-01-24T14:16:09.351796+0000","last_became_peered":"2024-01-24T14:16:09.351796+0000","last_unstale":"2024-01-24T14:16:09.362451+0000","last_undegraded":"2024-01-24T14:16:09.362451+0000","last_fullsized":"2024-01-24T14:16:09.362451+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.340705+0000","last_change":"2024-01-24T14:16:08.937210+0000","last_active":"2024-01-24T14:16:09.340705+0000","last_peered":"2024-01-24T14:16:09.340705+0000","last_clean":"2024-01-24T14:16:09.340705+0000","last_became_active":"2024-01-24T14:16:08.936853+0000","last_became_peered":"2024-01-24T14:16:08.936853+0000","last_unstale":"2024-01-24T14:16:09.340705+0000","last_undegraded":"2024-01-24T14:16:09.340705+0000","last_fullsized":"2024-01-24T14:16:09.340705+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.351530+0000","last_change":"2024-01-24T14:16:09.345747+0000","last_active":"2024-01-24T14:16:09.351530+0000","last_peered":"2024-01-24T14:16:09.351530+0000","last_clean":"2024-01-24T14:16:09.351530+0000","last_became_active":"2024-01-24T14:16:09.345296+0000","last_became_peered":"2024-01-24T14:16:09.345296+0000","last_unstale":"2024-01-24T14:16:09.351530+0000","last_undegraded":"2024-01-24T14:16:09.351530+0000","last_fullsized":"2024-01-24T14:16:09.351530+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.342454+0000","last_change":"2024-01-24T14:16:08.930529+0000","last_active":"2024-01-24T14:16:09.342454+0000","last_peered":"2024-01-24T14:16:09.342454+0000","last_clean":"2024-01-24T14:16:09.342454+0000","last_became_active":"2024-01-24T14:16:08.930165+0000","last_became_peered":"2024-01-24T14:16:08.930165+0000","last_unstale":"2024-01-24T14:16:09.342454+0000","last_undegraded":"2024-01-24T14:16:09.342454+0000","last_fullsized":"2024-01-24T14:16:09.342454+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.367309+0000","last_change":"2024-01-24T14:16:09.364267+0000","last_active":"2024-01-24T14:16:09.367309+0000","last_peered":"2024-01-24T14:16:09.367309+0000","last_clean":"2024-01-24T14:16:09.367309+0000","last_became_active":"2024-01-24T14:16:09.363981+0000","last_became_peered":"2024-01-24T14:16:09.363981+0000","last_unstale":"2024-01-24T14:16:09.367309+0000","last_undegraded":"2024-01-24T14:16:09.367309+0000","last_fullsized":"2024-01-24T14:16:09.367309+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336642+0000","last_change":"2024-01-24T14:16:08.932135+0000","last_active":"2024-01-24T14:16:09.336642+0000","last_peered":"2024-01-24T14:16:09.336642+0000","last_clean":"2024-01-24T14:16:09.336642+0000","last_became_active":"2024-01-24T14:16:08.931750+0000","last_became_peered":"2024-01-24T14:16:08.931750+0000","last_unstale":"2024-01-24T14:16:09.336642+0000","last_undegraded":"2024-01-24T14:16:09.336642+0000","last_fullsized":"2024-01-24T14:16:09.336642+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.373240+0000","last_change":"2024-01-24T14:16:09.366166+0000","last_active":"2024-01-24T14:16:09.373240+0000","last_peered":"2024-01-24T14:16:09.373240+0000","last_clean":"2024-01-24T14:16:09.373240+0000","last_became_active":"2024-01-24T14:16:09.365494+0000","last_became_peered":"2024-01-24T14:16:09.365494+0000","last_unstale":"2024-01-24T14:16:09.373240+0000","last_undegraded":"2024-01-24T14:16:09.373240+0000","last_fullsized":"2024-01-24T14:16:09.373240+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.355725+0000","last_change":"2024-01-24T14:16:08.929461+0000","last_active":"2024-01-24T14:16:09.355725+0000","last_peered":"2024-01-24T14:16:09.355725+0000","last_clean":"2024-01-24T14:16:09.355725+0000","last_became_active":"2024-01-24T14:16:08.927699+0000","last_became_peered":"2024-01-24T14:16:08.927699+0000","last_unstale":"2024-01-24T14:16:09.355725+0000","last_undegraded":"2024-01-24T14:16:09.355725+0000","last_fullsized":"2024-01-24T14:16:09.355725+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.375777+0000","last_change":"2024-01-24T14:16:09.374585+0000","last_active":"2024-01-24T14:16:09.375777+0000","last_peered":"2024-01-24T14:16:09.375777+0000","last_clean":"2024-01-24T14:16:09.375777+0000","last_became_active":"2024-01-24T14:16:09.374267+0000","last_became_peered":"2024-01-24T14:16:09.374267+0000","last_unstale":"2024-01-24T14:16:09.375777+0000","last_undegraded":"2024-01-24T14:16:09.375777+0000","last_fullsized":"2024-01-24T14:16:09.375777+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.339976+0000","last_change":"2024-01-24T14:16:08.932446+0000","last_active":"2024-01-24T14:16:09.339976+0000","last_peered":"2024-01-24T14:16:09.339976+0000","last_clean":"2024-01-24T14:16:09.339976+0000","last_became_active":"2024-01-24T14:16:08.932185+0000","last_became_peered":"2024-01-24T14:16:08.932185+0000","last_unstale":"2024-01-24T14:16:09.339976+0000","last_undegraded":"2024-01-24T14:16:09.339976+0000","last_fullsized":"2024-01-24T14:16:09.339976+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350412+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:16:09.350412+0000","last_peered":"2024-01-24T14:16:09.350412+0000","last_clean":"2024-01-24T14:16:09.350412+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:16:09.350412+0000","last_undegraded":"2024-01-24T14:16:09.350412+0000","last_fullsized":"2024-01-24T14:16:09.350412+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360556+0000","last_change":"2024-01-24T14:16:09.349071+0000","last_active":"2024-01-24T14:16:09.360556+0000","last_peered":"2024-01-24T14:16:09.360556+0000","last_clean":"2024-01-24T14:16:09.360556+0000","last_became_active":"2024-01-24T14:16:09.347445+0000","last_became_peered":"2024-01-24T14:16:09.347445+0000","last_unstale":"2024-01-24T14:16:09.360556+0000","last_undegraded":"2024-01-24T14:16:09.360556+0000","last_fullsized":"2024-01-24T14:16:09.360556+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"136'4","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.361554+0000","last_change":"2024-01-24T14:16:08.933952+0000","last_active":"2024-01-24T14:16:09.361554+0000","last_peered":"2024-01-24T14:16:09.361554+0000","last_clean":"2024-01-24T14:16:09.361554+0000","last_became_active":"2024-01-24T14:16:08.933637+0000","last_became_peered":"2024-01-24T14:16:08.933637+0000","last_unstale":"2024-01-24T14:16:09.361554+0000","last_undegraded":"2024-01-24T14:16:09.361554+0000","last_fullsized":"2024-01-24T14:16:09.361554+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":124,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.342299+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:16:09.342299+0000","last_peered":"2024-01-24T14:16:09.342299+0000","last_clean":"2024-01-24T14:16:09.342299+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:16:09.342299+0000","last_undegraded":"2024-01-24T14:16:09.342299+0000","last_fullsized":"2024-01-24T14:16:09.342299+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360431+0000","last_change":"2024-01-24T14:16:09.346481+0000","last_active":"2024-01-24T14:16:09.360431+0000","last_peered":"2024-01-24T14:16:09.360431+0000","last_clean":"2024-01-24T14:16:09.360431+0000","last_became_active":"2024-01-24T14:16:09.346181+0000","last_became_peered":"2024-01-24T14:16:09.346181+0000","last_unstale":"2024-01-24T14:16:09.360431+0000","last_undegraded":"2024-01-24T14:16:09.360431+0000","last_fullsized":"2024-01-24T14:16:09.360431+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.363300+0000","last_change":"2024-01-24T14:16:08.936284+0000","last_active":"2024-01-24T14:16:09.363300+0000","last_peered":"2024-01-24T14:16:09.363300+0000","last_clean":"2024-01-24T14:16:09.363300+0000","last_became_active":"2024-01-24T14:16:08.935984+0000","last_became_peered":"2024-01-24T14:16:08.935984+0000","last_unstale":"2024-01-24T14:16:09.363300+0000","last_undegraded":"2024-01-24T14:16:09.363300+0000","last_fullsized":"2024-01-24T14:16:09.363300+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":119,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336449+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:16:09.336449+0000","last_peered":"2024-01-24T14:16:09.336449+0000","last_clean":"2024-01-24T14:16:09.336449+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:16:09.336449+0000","last_undegraded":"2024-01-24T14:16:09.336449+0000","last_fullsized":"2024-01-24T14:16:09.336449+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.362415+0000","last_change":"2024-01-24T14:16:09.355510+0000","last_active":"2024-01-24T14:16:09.362415+0000","last_peered":"2024-01-24T14:16:09.362415+0000","last_clean":"2024-01-24T14:16:09.362415+0000","last_became_active":"2024-01-24T14:16:09.355210+0000","last_became_peered":"2024-01-24T14:16:09.355210+0000","last_unstale":"2024-01-24T14:16:09.362415+0000","last_undegraded":"2024-01-24T14:16:09.362415+0000","last_fullsized":"2024-01-24T14:16:09.362415+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.341631+0000","last_change":"2024-01-24T14:16:08.942955+0000","last_active":"2024-01-24T14:16:09.341631+0000","last_peered":"2024-01-24T14:16:09.341631+0000","last_clean":"2024-01-24T14:16:09.341631+0000","last_became_active":"2024-01-24T14:16:08.942771+0000","last_became_peered":"2024-01-24T14:16:08.942771+0000","last_unstale":"2024-01-24T14:16:09.341631+0000","last_undegraded":"2024-01-24T14:16:09.341631+0000","last_fullsized":"2024-01-24T14:16:09.341631+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":124,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.340794+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:16:09.340794+0000","last_peered":"2024-01-24T14:16:09.340794+0000","last_clean":"2024-01-24T14:16:09.340794+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:16:09.340794+0000","last_undegraded":"2024-01-24T14:16:09.340794+0000","last_fullsized":"2024-01-24T14:16:09.340794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.352665+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:16:09.352665+0000","last_peered":"2024-01-24T14:16:09.352665+0000","last_clean":"2024-01-24T14:16:09.352665+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:16:09.352665+0000","last_undegraded":"2024-01-24T14:16:09.352665+0000","last_fullsized":"2024-01-24T14:16:09.352665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.362815+0000","last_change":"2024-01-24T14:16:09.356779+0000","last_active":"2024-01-24T14:16:09.362815+0000","last_peered":"2024-01-24T14:16:09.362815+0000","last_clean":"2024-01-24T14:16:09.362815+0000","last_became_active":"2024-01-24T14:16:09.356407+0000","last_became_peered":"2024-01-24T14:16:09.356407+0000","last_unstale":"2024-01-24T14:16:09.362815+0000","last_undegraded":"2024-01-24T14:16:09.362815+0000","last_fullsized":"2024-01-24T14:16:09.362815+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350631+0000","last_change":"2024-01-24T14:16:08.938442+0000","last_active":"2024-01-24T14:16:09.350631+0000","last_peered":"2024-01-24T14:16:09.350631+0000","last_clean":"2024-01-24T14:16:09.350631+0000","last_became_active":"2024-01-24T14:16:08.938179+0000","last_became_peered":"2024-01-24T14:16:08.938179+0000","last_unstale":"2024-01-24T14:16:09.350631+0000","last_undegraded":"2024-01-24T14:16:09.350631+0000","last_fullsized":"2024-01-24T14:16:09.350631+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":112,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.352554+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:16:09.352554+0000","last_peered":"2024-01-24T14:16:09.352554+0000","last_clean":"2024-01-24T14:16:09.352554+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:16:09.352554+0000","last_undegraded":"2024-01-24T14:16:09.352554+0000","last_fullsized":"2024-01-24T14:16:09.352554+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.358344+0000","last_change":"2024-01-24T14:16:09.350990+0000","last_active":"2024-01-24T14:16:09.358344+0000","last_peered":"2024-01-24T14:16:09.358344+0000","last_clean":"2024-01-24T14:16:09.358344+0000","last_became_active":"2024-01-24T14:16:09.350707+0000","last_became_peered":"2024-01-24T14:16:09.350707+0000","last_unstale":"2024-01-24T14:16:09.358344+0000","last_undegraded":"2024-01-24T14:16:09.358344+0000","last_fullsized":"2024-01-24T14:16:09.358344+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.336786+0000","last_change":"2024-01-24T14:16:08.932090+0000","last_active":"2024-01-24T14:16:09.336786+0000","last_peered":"2024-01-24T14:16:09.336786+0000","last_clean":"2024-01-24T14:16:09.336786+0000","last_became_active":"2024-01-24T14:16:08.931350+0000","last_became_peered":"2024-01-24T14:16:08.931350+0000","last_unstale":"2024-01-24T14:16:09.336786+0000","last_undegraded":"2024-01-24T14:16:09.336786+0000","last_fullsized":"2024-01-24T14:16:09.336786+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":112,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.354062+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:16:09.354062+0000","last_peered":"2024-01-24T14:16:09.354062+0000","last_clean":"2024-01-24T14:16:09.354062+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:16:09.354062+0000","last_undegraded":"2024-01-24T14:16:09.354062+0000","last_fullsized":"2024-01-24T14:16:09.354062+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.360295+0000","last_change":"2024-01-24T14:16:09.355470+0000","last_active":"2024-01-24T14:16:09.360295+0000","last_peered":"2024-01-24T14:16:09.360295+0000","last_clean":"2024-01-24T14:16:09.360295+0000","last_became_active":"2024-01-24T14:16:09.355204+0000","last_became_peered":"2024-01-24T14:16:09.355204+0000","last_unstale":"2024-01-24T14:16:09.360295+0000","last_undegraded":"2024-01-24T14:16:09.360295+0000","last_fullsized":"2024-01-24T14:16:09.360295+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.374172+0000","last_change":"2024-01-24T14:16:08.946057+0000","last_active":"2024-01-24T14:16:09.374172+0000","last_peered":"2024-01-24T14:16:09.374172+0000","last_clean":"2024-01-24T14:16:09.374172+0000","last_became_active":"2024-01-24T14:16:08.945836+0000","last_became_peered":"2024-01-24T14:16:08.945836+0000","last_unstale":"2024-01-24T14:16:09.374172+0000","last_undegraded":"2024-01-24T14:16:09.374172+0000","last_fullsized":"2024-01-24T14:16:09.374172+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":111,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350665+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:16:09.350665+0000","last_peered":"2024-01-24T14:16:09.350665+0000","last_clean":"2024-01-24T14:16:09.350665+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:16:09.350665+0000","last_undegraded":"2024-01-24T14:16:09.350665+0000","last_fullsized":"2024-01-24T14:16:09.350665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.362950+0000","last_change":"2024-01-24T14:16:09.353064+0000","last_active":"2024-01-24T14:16:09.362950+0000","last_peered":"2024-01-24T14:16:09.362950+0000","last_clean":"2024-01-24T14:16:09.362950+0000","last_became_active":"2024-01-24T14:16:09.352722+0000","last_became_peered":"2024-01-24T14:16:09.352722+0000","last_unstale":"2024-01-24T14:16:09.362950+0000","last_undegraded":"2024-01-24T14:16:09.362950+0000","last_fullsized":"2024-01-24T14:16:09.362950+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:08.906152+0000","last_clean_scrub_stamp":"2024-01-24T14:16:08.906152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"136'3","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.372384+0000","last_change":"2024-01-24T14:16:08.934729+0000","last_active":"2024-01-24T14:16:09.372384+0000","last_peered":"2024-01-24T14:16:09.372384+0000","last_clean":"2024-01-24T14:16:09.372384+0000","last_became_active":"2024-01-24T14:16:08.934448+0000","last_became_peered":"2024-01-24T14:16:08.934448+0000","last_unstale":"2024-01-24T14:16:09.372384+0000","last_undegraded":"2024-01-24T14:16:09.372384+0000","last_fullsized":"2024-01-24T14:16:09.372384+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:16:07.900978+0000","last_clean_scrub_stamp":"2024-01-24T14:16:07.900978+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":109,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-01-24T14:16:09.350181+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:16:09.350181+0000","last_peered":"2024-01-24T14:16:09.350181+0000","last_clean":"2024-01-24T14:16:09.350181+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:16:09.350181+0000","last_undegraded":"2024-01-24T14:16:09.350181+0000","last_fullsized":"2024-01-24T14:16:09.350181+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575062,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302840,"kb_used_data":1648,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069000,"statfs":{"total":96636764160,"available":96326656000,"internally_reserved":0,"allocated":1687552,"data_stored":946163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.441,"5min":0.36799999999999999,"15min":0.26000000000000001},"max":{"1min":0.80200000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.32700000000000001},"max":{"1min":0.82199999999999995,"5min":0.82499999999999996,"15min":0.83499999999999996},"last":0.72199999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.39800000000000002,"15min":0.371},"max":{"1min":0.81299999999999994,"5min":0.85099999999999998,"15min":0.85599999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77800000000000002,"5min":0.80600000000000005,"15min":0.80900000000000005},"last":0.59399999999999997}]},{"osd":2,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40300000000000002},"max":{"1min":0.78000000000000003,"5min":0.872,"15min":0.876},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.60499999999999998},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.36799999999999999},"max":{"1min":0.77700000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51400000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.47799999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.752,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.39300000000000002,"5min":0.38,"15min":0.35499999999999998},"max":{"1min":0.83199999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63900000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.60899999999999999,"5min":0.77000000000000002,"15min":0.84699999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47999999999999998,"15min":0.49099999999999999},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68200000000000005,"5min":0.71199999999999997,"15min":0.745},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.502,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.72499999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.42699999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71499999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.749}]},{"osd":6,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.34100000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80100000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.42099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.745,"5min":0.80700000000000005,"15min":0.87},"last":0.66300000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575063,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":953924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.32700000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.752,"15min":0.85699999999999998},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58299999999999996,"15min":0.59299999999999997},"min":{"1min":0.441,"5min":0.35299999999999998,"15min":0.29999999999999999},"max":{"1min":0.82799999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48999999999999999}]},{"osd":1,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.59199999999999997},"min":{"1min":0.432,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.88},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.42399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73899999999999999,"5min":0.78700000000000003,"15min":1.032},"last":0.501}]},{"osd":2,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.75600000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.52500000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.62},"min":{"1min":0.41299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76200000000000001,"5min":0.82799999999999996,"15min":0.89200000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.81499999999999995,"5min":0.86799999999999999,"15min":1.2430000000000001},"last":0.48199999999999998}]},{"osd":4,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.375,"5min":0.32200000000000001,"15min":0.28899999999999998},"max":{"1min":0.77400000000000002,"5min":0.83899999999999997,"15min":1.165},"last":0.443},{"interface":"front","average":{"1min":0.503,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.314},"max":{"1min":0.71499999999999997,"5min":0.74399999999999999,"15min":0.83199999999999996},"last":0.35899999999999999}]},{"osd":5,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.377,"5min":0.313,"15min":0.313},"max":{"1min":0.745,"5min":0.81100000000000005,"15min":0.86699999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.36499999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.745,"5min":0.80200000000000005,"15min":0.85699999999999998},"last":0.41099999999999998}]},{"osd":7,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71899999999999997,"5min":0.75900000000000001,"15min":1.2070000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.318,"15min":0.318},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.89600000000000002},"last":0.47499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575062,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302856,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068984,"statfs":{"total":96636764160,"available":96326639616,"internally_reserved":0,"allocated":1703936,"data_stored":953978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.50800000000000001,"15min":0.442},"min":{"1min":0.26400000000000001,"5min":0.245,"15min":0.23400000000000001},"max":{"1min":0.50800000000000001,"5min":15.298999999999999,"15min":15.298999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.54100000000000004,"15min":0.46999999999999997},"min":{"1min":0.31900000000000001,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.59399999999999997,"5min":14.654,"15min":14.654},"last":0.35799999999999998}]},{"osd":2,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.45200000000000001},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.54100000000000004,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":0.62,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46899999999999997}]},{"osd":3,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.31,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.55500000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.48699999999999999,"15min":0.48399999999999999},"min":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.56200000000000006,"5min":1.651,"15min":1.651},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64700000000000002,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":1.137},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.46500000000000002,"5min":0.45900000000000002,"15min":0.36499999999999999},"max":{"1min":0.89100000000000001,"5min":0.995,"15min":1.218},"last":0.54500000000000004}]},{"osd":5,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69099999999999995,"15min":0.83899999999999997},"min":{"1min":0.53500000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83699999999999997,"5min":0.94999999999999996,"15min":38.808},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.81200000000000006},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.40500000000000003},"max":{"1min":0.89400000000000002,"5min":0.95099999999999996,"15min":38.823},"last":0.503}]},{"osd":6,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.44500000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.0509999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70699999999999996,"15min":0.72699999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.40000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1439999999999999,"15min":1.1879999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68500000000000005,"15min":0.69899999999999995},"min":{"1min":0.51300000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88300000000000001,"5min":0.995,"15min":1.155},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72699999999999998,"15min":0.751},"min":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.49399999999999999},"max":{"1min":0.92900000000000005,"5min":1.103,"15min":1.131},"last":0.69399999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542360,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306952,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064888,"statfs":{"total":96636764160,"available":96322445312,"internally_reserved":0,"allocated":1703936,"data_stored":946548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38700000000000001,"15min":0.39600000000000002},"min":{"1min":0.26900000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.59799999999999998,"5min":0.60899999999999999,"15min":0.69399999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.42699999999999999,"15min":0.42799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.223},"max":{"1min":0.48799999999999999,"5min":0.623,"15min":0.71699999999999997},"last":0.5}]},{"osd":2,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45200000000000001,"15min":0.44},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.28999999999999998},"max":{"1min":0.57699999999999996,"5min":0.66100000000000003,"15min":0.66700000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.46000000000000002},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.61599999999999999,"5min":0.754,"15min":0.79400000000000004},"last":0.45100000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.437,"15min":0.434},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.25900000000000001},"max":{"1min":0.55800000000000005,"5min":1.482,"15min":1.482},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49399999999999999,"15min":0.499},"min":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.32100000000000001},"max":{"1min":0.621,"5min":0.872,"15min":0.872},"last":0.59199999999999997}]},{"osd":4,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64400000000000002,"15min":0.65000000000000002},"min":{"1min":0.52400000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77400000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.50700000000000001,"5min":0.41899999999999998,"15min":0.40000000000000002},"max":{"1min":0.90000000000000002,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.52700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93200000000000005,"5min":1.655,"15min":1.655},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.68200000000000005},"min":{"1min":0.51000000000000001,"5min":0.46300000000000002,"15min":0.42999999999999999},"max":{"1min":0.86899999999999999,"5min":1.234,"15min":1.234},"last":0.79500000000000004}]},{"osd":6,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.57699999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90100000000000002,"5min":1.196,"15min":1.196},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92200000000000004,"5min":0.96499999999999997,"15min":1.117},"last":0.57899999999999996}]},{"osd":7,"last update":"Wed Jan 24 14:15:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.56200000000000006,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88700000000000001,"5min":0.93999999999999995,"15min":0.98199999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69299999999999995,"15min":0.69799999999999995},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.879,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.84099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575062,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1683456,"data_stored":946129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.24199999999999999},"max":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.65800000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42599999999999999,"15min":0.434},"min":{"1min":0.33500000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.67100000000000004},"last":0.40300000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42699999999999999,"15min":0.41599999999999998},"min":{"1min":0.29499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61099999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45300000000000001,"15min":0.46000000000000002},"min":{"1min":0.35399999999999998,"5min":0.30399999999999999,"15min":0.29699999999999999},"max":{"1min":0.58299999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.36399999999999999}]},{"osd":3,"last update":"Wed Jan 24 14:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44800000000000001,"15min":0.45400000000000001},"min":{"1min":0.318,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58899999999999997,"5min":0.67700000000000005,"15min":0.71699999999999997},"last":0.372},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30299999999999999},"max":{"1min":0.628,"5min":0.70799999999999996,"15min":0.72799999999999998},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.628,"15min":0.64400000000000002},"min":{"1min":0.45400000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96699999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.748},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.65900000000000003,"15min":0.67200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.96599999999999997,"15min":1.042},"last":0.65200000000000002}]},{"osd":5,"last update":"Wed Jan 24 14:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65800000000000003,"15min":0.67900000000000005},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91500000000000004,"5min":1.036,"15min":1.0469999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85099999999999998,"5min":1.107,"15min":1.107},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed Jan 24 14:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.48899999999999999,"5min":0.48699999999999999,"15min":0.45600000000000002},"max":{"1min":0.874,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0529999999999999,"5min":1.141,"15min":1.141},"last":0.73399999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.71799999999999997},"min":{"1min":0.504,"5min":0.435,"15min":0.435},"max":{"1min":0.89800000000000002,"5min":0.96199999999999997,"15min":1.0409999999999999},"last":0.623},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71699999999999997,"15min":0.72799999999999998},"min":{"1min":0.497,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97999999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.70999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575062,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":955944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.46500000000000002,"15min":0.41799999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.22900000000000001},"max":{"1min":0.58899999999999997,"5min":7.593,"15min":7.593},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.499,"15min":0.44400000000000001},"min":{"1min":0.29399999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.60699999999999998,"5min":8.4239999999999995,"15min":8.4239999999999995},"last":0.38900000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55100000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45200000000000001,"15min":0.45500000000000002},"min":{"1min":0.32800000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.66200000000000003,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.442}]},{"osd":2,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48999999999999999,"15min":0.496},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.311},"max":{"1min":0.65000000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.68400000000000005,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.47599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.006,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65300000000000002,"15min":0.66200000000000003},"min":{"1min":0.497,"5min":0.437,"15min":0.42999999999999999},"max":{"1min":0.93799999999999994,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.755}]},{"osd":5,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.39900000000000002},"max":{"1min":1.03,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66600000000000004,"15min":0.68899999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97799999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.77800000000000002}]},{"osd":6,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68500000000000005,"15min":0.68999999999999995},"min":{"1min":0.53400000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.501,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.99099999999999999,"5min":3.1800000000000002,"15min":3.1800000000000002},"last":0.72299999999999998}]},{"osd":7,"last update":"Wed Jan 24 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.44600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.97399999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.56299999999999994,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95799999999999996,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":0.64300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575063,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":954746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.74299999999999999,"5min":0.83199999999999996,"15min":0.88600000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.439,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.74199999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58799999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.36699999999999999,"15min":0.33600000000000002},"max":{"1min":0.78500000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.44800000000000001,"5min":0.38200000000000001,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.40300000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74099999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80700000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.47299999999999998}]},{"osd":3,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.41999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.71599999999999997,"5min":0.79200000000000004,"15min":0.80600000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60399999999999998,"15min":0.61499999999999999},"min":{"1min":0.48899999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78300000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.42899999999999999}]},{"osd":5,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.497,"15min":0.47899999999999998},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.27300000000000002},"max":{"1min":0.70599999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29199999999999998},"max":{"1min":0.66500000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.45000000000000001}]},{"osd":6,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50900000000000001,"15min":0.48899999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64400000000000002,"5min":0.79700000000000004,"15min":0.80700000000000005},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33200000000000002},"max":{"1min":0.70699999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.56399999999999995}]},{"osd":7,"last update":"Wed Jan 24 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.65700000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.72299999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.495}]}]},{"osd":5,"up_from":13,"seq":55834575063,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":953876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53800000000000003,"15min":0.56799999999999995},"min":{"1min":0.41199999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70999999999999996,"5min":0.81399999999999995,"15min":1.0800000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.58699999999999997},"min":{"1min":0.42299999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54700000000000004,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79100000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.57899999999999996,"15min":0.59699999999999998},"min":{"1min":0.436,"5min":0.35199999999999998,"15min":0.29799999999999999},"max":{"1min":0.78500000000000003,"5min":0.79000000000000004,"15min":1.1639999999999999},"last":0.54600000000000004}]},{"osd":2,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.58999999999999997},"min":{"1min":0.39500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56699999999999995,"15min":0.58899999999999997},"min":{"1min":0.41999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.71699999999999997,"5min":0.77700000000000002,"15min":0.83699999999999997},"last":0.53100000000000003}]},{"osd":3,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58699999999999997,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":1.139},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.59299999999999997,"15min":0.61599999999999999},"min":{"1min":0.42899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.80800000000000005,"5min":0.83099999999999996,"15min":0.85399999999999998},"last":0.56599999999999995}]},{"osd":4,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.33700000000000002,"15min":0.30299999999999999},"max":{"1min":0.76800000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.498,"15min":0.501},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.30399999999999999},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.90000000000000002},"last":0.58399999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.505,"15min":0.505},"min":{"1min":0.36099999999999999,"5min":0.307,"15min":0.253},"max":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":1.1839999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.32000000000000001},"max":{"1min":0.72799999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60899999999999999}]},{"osd":7,"last update":"Wed Jan 24 14:15:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.82799999999999996,"5min":0.95999999999999996,"15min":1.2030000000000001},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:16:13.857 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-24T14:16:13.858 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-24T14:16:14.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.030+0000 7f36500a7700 1 Processor -- start 2024-01-24T14:16:14.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.030+0000 7f36500a7700 1 -- start start 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.031+0000 7f36500a7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3648150fe0 0x7f3648154920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.031+0000 7f36500a7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3648154e10 0x7f36480c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.031+0000 7f36500a7700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36480c3b60 con 0x7f3648150c00 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.031+0000 7f36500a7700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3648066960 con 0x7f3648154e10 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.031+0000 7f36500a7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36481558a0 con 0x7f3648150fe0 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.031+0000 7f364e644700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3648154e10 0x7f36480c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.031+0000 7f364e644700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3648154e10 0x7f36480c44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51018/0 (socket says 172.21.15.29:51018) 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.031+0000 7f364de43700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3648150c00 0x7f36480c3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48396/0 (socket says 172.21.15.29:48396) 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.031+0000 7f364e644700 1 -- 172.21.15.29:0/4138728286 learned_addr learned my addr 172.21.15.29:0/4138728286 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.032+0000 7f364d642700 1 --2- 172.21.15.29:0/4138728286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3648150fe0 0x7f3648154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.032+0000 7f364e644700 1 -- 172.21.15.29:0/4138728286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3648150fe0 msgr2=0x7f3648154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.032+0000 7f364e644700 1 --2- 172.21.15.29:0/4138728286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3648150fe0 0x7f3648154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.032+0000 7f364e644700 1 -- 172.21.15.29:0/4138728286 >> v1:172.21.15.29:6789/0 conn(0x7f3648150c00 legacy=0x7f36480c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.032+0000 7f364e644700 1 -- 172.21.15.29:0/4138728286 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480c2fc0 con 0x7f3648154e10 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.032+0000 7f364ce41700 1 -- 172.21.15.29:0/4138728286 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505262578 0 0) 0x7f36480c3b60 con 0x7f3648150c00 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.032+0000 7f364e644700 1 --2- 172.21.15.29:0/4138728286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3648154e10 0x7f36480c44c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7f3640009f20 tx=0x7f3640009840).ready entity=mon.0 client_cookie=dd857241d20e40cb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.032+0000 7f364ce41700 1 -- 172.21.15.29:0/4138728286 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364000e070 con 0x7f3648154e10 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.032+0000 7f364ce41700 1 -- 172.21.15.29:0/4138728286 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364000ba80 con 0x7f3648154e10 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.032+0000 7f364ce41700 1 -- 172.21.15.29:0/4138728286 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364000be00 con 0x7f3648154e10 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.033+0000 7f36500a7700 1 -- 172.21.15.29:0/4138728286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3648154e10 msgr2=0x7f36480c44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.033+0000 7f36500a7700 1 --2- 172.21.15.29:0/4138728286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3648154e10 0x7f36480c44c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7f3640009f20 tx=0x7f3640009840).stop 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.033+0000 7f36500a7700 1 -- 172.21.15.29:0/4138728286 shutdown_connections 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.033+0000 7f36500a7700 1 --2- 172.21.15.29:0/4138728286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3648154e10 0x7f36480c44c0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.033+0000 7f36500a7700 1 --2- 172.21.15.29:0/4138728286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3648150fe0 0x7f3648154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.033+0000 7f36500a7700 1 -- 172.21.15.29:0/4138728286 >> 172.21.15.29:0/4138728286 conn(0x7f36480638c0 msgr2=0x7f3648063cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.033+0000 7f36500a7700 1 -- 172.21.15.29:0/4138728286 shutdown_connections 2024-01-24T14:16:14.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.033+0000 7f36500a7700 1 -- 172.21.15.29:0/4138728286 wait complete. 2024-01-24T14:16:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.034+0000 7f36500a7700 1 Processor -- start 2024-01-24T14:16:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.034+0000 7f36500a7700 1 -- start start 2024-01-24T14:16:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f36500a7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3648150c00 0x7f3648142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f36500a7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3648150fe0 0x7f3648153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f36500a7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3648154e10 0x7f36481541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f36500a7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36480c68c0 con 0x7f3648150c00 2024-01-24T14:16:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f36500a7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36480c6430 con 0x7f3648150fe0 2024-01-24T14:16:14.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f36500a7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36480c5820 con 0x7f3648154e10 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f364de43700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3648150c00 0x7f3648142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f364de43700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3648150c00 0x7f3648142fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51034/0 (socket says 172.21.15.29:51034) 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f364de43700 1 -- 172.21.15.29:0/1570863620 learned_addr learned my addr 172.21.15.29:0/1570863620 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f364e644700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3648154e10 0x7f36481541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f364de43700 1 -- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3648154e10 msgr2=0x7f36481541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f364d642700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3648150fe0 0x7f3648153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f364de43700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3648154e10 0x7f36481541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f364de43700 1 -- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3648150fe0 msgr2=0x7f3648153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f364de43700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3648150fe0 0x7f3648153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f364de43700 1 -- 172.21.15.29:0/1570863620 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3640009ae0 con 0x7f3648150c00 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f364e644700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3648154e10 0x7f36481541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.035+0000 7f364d642700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3648150fe0 0x7f3648153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:16:14.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.036+0000 7f364de43700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3648150c00 0x7f3648142fe0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f364400c560 tx=0x7f3644007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:14.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.036+0000 7f3639ffb700 1 -- 172.21.15.29:0/1570863620 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36440072e0 con 0x7f3648150c00 2024-01-24T14:16:14.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.036+0000 7f3639ffb700 1 -- 172.21.15.29:0/1570863620 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3644007440 con 0x7f3648150c00 2024-01-24T14:16:14.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.036+0000 7f36500a7700 1 -- 172.21.15.29:0/1570863620 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36480c5b00 con 0x7f3648150c00 2024-01-24T14:16:14.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.036+0000 7f3639ffb700 1 -- 172.21.15.29:0/1570863620 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644005930 con 0x7f3648150c00 2024-01-24T14:16:14.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.036+0000 7f36500a7700 1 -- 172.21.15.29:0/1570863620 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36480c60c0 con 0x7f3648150c00 2024-01-24T14:16:14.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.037+0000 7f36500a7700 1 -- 172.21.15.29:0/1570863620 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36480c6710 con 0x7f3648150c00 2024-01-24T14:16:14.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.037+0000 7f3639ffb700 1 -- 172.21.15.29:0/1570863620 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3644005ce0 con 0x7f3648150c00 2024-01-24T14:16:14.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.038+0000 7f3639ffb700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f362c05f270 0x7f362c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:14.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.038+0000 7f3639ffb700 1 -- 172.21.15.29:0/1570863620 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f3644079fc0 con 0x7f3648150c00 2024-01-24T14:16:14.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.038+0000 7f364d642700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f362c05f270 0x7f362c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:14.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.039+0000 7f364d642700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f362c05f270 0x7f362c061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f36480bb4c0 tx=0x7f363c008040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:14.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.042+0000 7f3639ffb700 1 -- 172.21.15.29:0/1570863620 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f364404ad10 con 0x7f3648150c00 2024-01-24T14:16:14.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.294+0000 7f36500a7700 1 -- 172.21.15.29:0/1570863620 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f36480475e0 con 0x7f3648150c00 2024-01-24T14:16:14.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.299+0000 7f3639ffb700 1 -- 172.21.15.29:0/1570863620 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3644018070 con 0x7f3648150c00 2024-01-24T14:16:14.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.301+0000 7f36500a7700 1 -- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f362c05f270 msgr2=0x7f362c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.301+0000 7f36500a7700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f362c05f270 0x7f362c061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f36480bb4c0 tx=0x7f363c008040).stop 2024-01-24T14:16:14.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.301+0000 7f36500a7700 1 -- 172.21.15.29:0/1570863620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3648150c00 msgr2=0x7f3648142fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.301+0000 7f36500a7700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3648150c00 0x7f3648142fe0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f364400c560 tx=0x7f3644007c90).stop 2024-01-24T14:16:14.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.301+0000 7f36500a7700 1 -- 172.21.15.29:0/1570863620 shutdown_connections 2024-01-24T14:16:14.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.301+0000 7f36500a7700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3648154e10 0x7f36481541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.301+0000 7f36500a7700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f362c05f270 0x7f362c061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.301+0000 7f36500a7700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3648150c00 0x7f3648142fe0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.301+0000 7f36500a7700 1 --2- 172.21.15.29:0/1570863620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3648150fe0 0x7f3648153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.301+0000 7f36500a7700 1 -- 172.21.15.29:0/1570863620 >> 172.21.15.29:0/1570863620 conn(0x7f36480638c0 msgr2=0x7f36480bfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:14.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.301+0000 7f36500a7700 1 -- 172.21.15.29:0/1570863620 shutdown_connections 2024-01-24T14:16:14.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.301+0000 7f36500a7700 1 -- 172.21.15.29:0/1570863620 wait complete. 2024-01-24T14:16:14.303 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-01-24T14:16:14.315 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:16:14.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.446+0000 7ff14cc32700 1 Processor -- start 2024-01-24T14:16:14.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.446+0000 7ff14cc32700 1 -- start start 2024-01-24T14:16:14.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.447+0000 7ff14cc32700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff148066c00 0x7ff148155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:14.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.447+0000 7ff14cc32700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1480c3910 con 0x7ff148155b00 2024-01-24T14:16:14.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.447+0000 7ff14cc32700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1480c5220 con 0x7ff1481563d0 2024-01-24T14:16:14.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.447+0000 7ff14cc32700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1480c5500 con 0x7ff148066c00 2024-01-24T14:16:14.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.447+0000 7ff14659c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff148155b00 0x7ff1480664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48402/0 (socket says 172.21.15.29:48402) 2024-01-24T14:16:14.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.447+0000 7ff14659c700 1 -- 172.21.15.29:0/1612984921 learned_addr learned my addr 172.21.15.29:0/1612984921 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:14.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.447+0000 7ff145d9b700 1 --2- 172.21.15.29:0/1612984921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff148066c00 0x7ff148155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:14.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.447+0000 7ff14559a700 1 -- 172.21.15.29:0/1612984921 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328784620 0 0) 0x7ff1480c3910 con 0x7ff148155b00 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.447+0000 7ff14559a700 1 -- 172.21.15.29:0/1612984921 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1300033f0 con 0x7ff148155b00 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.447+0000 7ff14559a700 1 -- 172.21.15.29:0/1612984921 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058485917 0 0) 0x7ff1480c5220 con 0x7ff1481563d0 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.447+0000 7ff14559a700 1 -- 172.21.15.29:0/1612984921 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1480c3910 con 0x7ff1481563d0 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.448+0000 7ff145d9b700 1 -- 172.21.15.29:0/1612984921 >> v1:172.21.15.163:6789/0 conn(0x7ff1481563d0 legacy=0x7ff1480bdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.448+0000 7ff145d9b700 1 -- 172.21.15.29:0/1612984921 >> v1:172.21.15.29:6789/0 conn(0x7ff148155b00 legacy=0x7ff1480664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.448+0000 7ff145d9b700 1 -- 172.21.15.29:0/1612984921 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1480c3350 con 0x7ff148066c00 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.448+0000 7ff145d9b700 1 --2- 172.21.15.29:0/1612984921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff148066c00 0x7ff148155ee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7ff13c009790 tx=0x7ff13c009060).ready entity=mon.2 client_cookie=d4826e522c2b442c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.448+0000 7ff14559a700 1 -- 172.21.15.29:0/1612984921 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c00f510 con 0x7ff148066c00 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.448+0000 7ff14559a700 1 -- 172.21.15.29:0/1612984921 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13c00f670 con 0x7ff148066c00 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.448+0000 7ff14559a700 1 -- 172.21.15.29:0/1612984921 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c00f9f0 con 0x7ff148066c00 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.448+0000 7ff14cc32700 1 -- 172.21.15.29:0/1612984921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff148066c00 msgr2=0x7ff148155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.448+0000 7ff14cc32700 1 --2- 172.21.15.29:0/1612984921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff148066c00 0x7ff148155ee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7ff13c009790 tx=0x7ff13c009060).stop 2024-01-24T14:16:14.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.449+0000 7ff14cc32700 1 -- 172.21.15.29:0/1612984921 shutdown_connections 2024-01-24T14:16:14.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.449+0000 7ff14cc32700 1 --2- 172.21.15.29:0/1612984921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff148066c00 0x7ff148155ee0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.449+0000 7ff14cc32700 1 -- 172.21.15.29:0/1612984921 >> 172.21.15.29:0/1612984921 conn(0x7ff1480c8bb0 msgr2=0x7ff1480c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:14.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.449+0000 7ff14cc32700 1 -- 172.21.15.29:0/1612984921 shutdown_connections 2024-01-24T14:16:14.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.449+0000 7ff14cc32700 1 -- 172.21.15.29:0/1612984921 wait complete. 2024-01-24T14:16:14.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.450+0000 7ff14cc32700 1 Processor -- start 2024-01-24T14:16:14.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.450+0000 7ff14cc32700 1 -- start start 2024-01-24T14:16:14.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.451+0000 7ff14cc32700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff148066c00 0x7ff14813a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:14.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.451+0000 7ff14cc32700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff148155b00 0x7ff148150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:14.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.451+0000 7ff14cc32700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1481563d0 0x7ff148151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:14.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.451+0000 7ff14cc32700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff148151610 con 0x7ff1481563d0 2024-01-24T14:16:14.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.451+0000 7ff14cc32700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff148162d80 con 0x7ff148155b00 2024-01-24T14:16:14.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.451+0000 7ff14cc32700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff148162170 con 0x7ff148066c00 2024-01-24T14:16:14.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff146d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1481563d0 0x7ff148151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:14.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff146d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1481563d0 0x7ff148151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51042/0 (socket says 172.21.15.29:51042) 2024-01-24T14:16:14.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff146d9d700 1 -- 172.21.15.29:0/3042392854 learned_addr learned my addr 172.21.15.29:0/3042392854 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:14.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff145d9b700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff148155b00 0x7ff148150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:14.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff14659c700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff148066c00 0x7ff14813a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:14.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff146d9d700 1 -- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff148066c00 msgr2=0x7ff14813a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff146d9d700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff148066c00 0x7ff14813a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff146d9d700 1 -- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff148155b00 msgr2=0x7ff148150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff146d9d700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff148155b00 0x7ff148150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff146d9d700 1 -- 172.21.15.29:0/3042392854 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c0092a0 con 0x7ff1481563d0 2024-01-24T14:16:14.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff146d9d700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1481563d0 0x7ff148151120 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7ff14000c7c0 tx=0x7ff14000cad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:14.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff1367fc700 1 -- 172.21.15.29:0/3042392854 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff140002240 con 0x7ff1481563d0 2024-01-24T14:16:14.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff1367fc700 1 -- 172.21.15.29:0/3042392854 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1400023a0 con 0x7ff1481563d0 2024-01-24T14:16:14.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff14cc32700 1 -- 172.21.15.29:0/3042392854 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff148162450 con 0x7ff1481563d0 2024-01-24T14:16:14.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff1367fc700 1 -- 172.21.15.29:0/3042392854 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1400017e0 con 0x7ff1481563d0 2024-01-24T14:16:14.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.452+0000 7ff14cc32700 1 -- 172.21.15.29:0/3042392854 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff148162a10 con 0x7ff1481563d0 2024-01-24T14:16:14.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.454+0000 7ff14cc32700 1 -- 172.21.15.29:0/3042392854 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1480475e0 con 0x7ff1481563d0 2024-01-24T14:16:14.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.454+0000 7ff1367fc700 1 -- 172.21.15.29:0/3042392854 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff140002510 con 0x7ff1481563d0 2024-01-24T14:16:14.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.454+0000 7ff1367fc700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff12805f270 0x7ff128061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:14.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.454+0000 7ff1367fc700 1 -- 172.21.15.29:0/3042392854 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7ff14007a5f0 con 0x7ff1481563d0 2024-01-24T14:16:14.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.454+0000 7ff14659c700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff12805f270 0x7ff128061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:14.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.455+0000 7ff14659c700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff12805f270 0x7ff128061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff138007fd0 tx=0x7ff138009040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:14.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.459+0000 7ff1367fc700 1 -- 172.21.15.29:0/3042392854 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff14004ed70 con 0x7ff1481563d0 2024-01-24T14:16:14.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.684+0000 7ff14cc32700 1 -- 172.21.15.29:0/3042392854 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1480c3630 con 0x7ff1481563d0 2024-01-24T14:16:14.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.684+0000 7ff1367fc700 1 -- 172.21.15.29:0/3042392854 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4497 (secure 0 0 0) 0x7ff14004e960 con 0x7ff1481563d0 2024-01-24T14:16:14.685 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:16:14.685 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1579003637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1579003637},{"type":"v1","addr":"172.21.15.163:6836","nonce":1579003637}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2823175689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2823175689},{"type":"v1","addr":"172.21.15.163:6837","nonce":2823175689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2560003609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2560003609},{"type":"v1","addr":"172.21.15.29:6834","nonce":2560003609}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:16:09.332377+0000","modified":"2024-01-24T14:16:12.347265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1458876872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1458876872},{"type":"v1","addr":"172.21.15.29:6835","nonce":1458876872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-24T14:16:14.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.686+0000 7ff14cc32700 1 -- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff12805f270 msgr2=0x7ff128061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.686+0000 7ff14cc32700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff12805f270 0x7ff128061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff138007fd0 tx=0x7ff138009040).stop 2024-01-24T14:16:14.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.686+0000 7ff14cc32700 1 -- 172.21.15.29:0/3042392854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1481563d0 msgr2=0x7ff148151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:14.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.686+0000 7ff14cc32700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1481563d0 0x7ff148151120 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7ff14000c7c0 tx=0x7ff14000cad0).stop 2024-01-24T14:16:14.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.687+0000 7ff14cc32700 1 -- 172.21.15.29:0/3042392854 shutdown_connections 2024-01-24T14:16:14.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.687+0000 7ff14cc32700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff12805f270 0x7ff128061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.687+0000 7ff14cc32700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1481563d0 0x7ff148151120 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.687+0000 7ff14cc32700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff148066c00 0x7ff14813a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.687+0000 7ff14cc32700 1 --2- 172.21.15.29:0/3042392854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff148155b00 0x7ff148150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:14.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.687+0000 7ff14cc32700 1 -- 172.21.15.29:0/3042392854 >> 172.21.15.29:0/3042392854 conn(0x7ff1480c8bb0 msgr2=0x7ff1480bde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:14.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.687+0000 7ff14cc32700 1 -- 172.21.15.29:0/3042392854 shutdown_connections 2024-01-24T14:16:14.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:14.687+0000 7ff14cc32700 1 -- 172.21.15.29:0/3042392854 wait complete. 2024-01-24T14:16:14.689 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 59 2024-01-24T14:16:14.884 INFO:tasks.ceph.mds.a.smithi029.stderr:2024-01-24T14:16:14.884+0000 7f788d1d1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T14:16:15.104 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T14:16:09.332377+0000', 'modified': '2024-01-24T14:16:12.347265+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7728}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7728': {'gid': 7728, 'name': 'a', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/1458876872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 1458876872}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 1458876872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-24T14:16:15.104 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T14:16:15.105 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-24T14:16:15.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.245+0000 7f103e2e9700 1 Processor -- start 2024-01-24T14:16:15.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.245+0000 7f103e2e9700 1 -- start start 2024-01-24T14:16:15.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.246+0000 7f103e2e9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1038069240 0x7f1038150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.246+0000 7f103e2e9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10381514a0 0x7f1038151950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.246+0000 7f103e2e9700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f103806a350 con 0x7f1038150bd0 2024-01-24T14:16:15.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.246+0000 7f103e2e9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1038069740 con 0x7f1038069240 2024-01-24T14:16:15.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.246+0000 7f103e2e9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1038069a20 con 0x7f10381514a0 2024-01-24T14:16:15.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.246+0000 7f1037fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1038150bd0 0x7f1038065e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48404/0 (socket says 172.21.15.29:48404) 2024-01-24T14:16:15.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.246+0000 7f1037fff700 1 -- 172.21.15.29:0/42036927 learned_addr learned my addr 172.21.15.29:0/42036927 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:15.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.246+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389513192 0 0) 0x7f103806a350 con 0x7f1038150bd0 2024-01-24T14:16:15.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.246+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10240033f0 con 0x7f1038150bd0 2024-01-24T14:16:15.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.246+0000 7f10377fe700 1 --2- 172.21.15.29:0/42036927 >> v2:172.21.15.163:3300/0 conn(0x7f1038069240 0x7f1038150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.246+0000 7f103c886700 1 --2- 172.21.15.29:0/42036927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10381514a0 0x7f1038151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 831882346 0 0) 0x7f10240033f0 con 0x7f1038150bd0 2024-01-24T14:16:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10381514a0 msgr2=0x7f1038151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 --2- 172.21.15.29:0/42036927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10381514a0 0x7f1038151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 >> v2:172.21.15.163:3300/0 conn(0x7f1038069240 msgr2=0x7f1038150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 --2- 172.21.15.29:0/42036927 >> v2:172.21.15.163:3300/0 conn(0x7f1038069240 0x7f1038150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1038069d00 con 0x7f1038150bd0 2024-01-24T14:16:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f1028001470 con 0x7f1038150bd0 2024-01-24T14:16:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 >> v1:172.21.15.29:6789/0 conn(0x7f1038150bd0 legacy=0x7f1038065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 --2- 172.21.15.29:0/42036927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10240038e0 0x7f1024003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 --2- 172.21.15.29:0/42036927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10381514a0 0x7f1024004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 --2- 172.21.15.29:0/42036927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1024004b80 0x7f1024005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1038069740 con 0x7f10240038e0 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1038069a20 con 0x7f1024004b80 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102400b8e0 con 0x7f10381514a0 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1037fff700 1 --2- 172.21.15.29:0/42036927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10240038e0 0x7f1024003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f10377fe700 1 --2- 172.21.15.29:0/42036927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10381514a0 0x7f1024004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f103c886700 1 --2- 172.21.15.29:0/42036927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1024004b80 0x7f1024005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1037fff700 1 -- 172.21.15.29:0/42036927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10381514a0 msgr2=0x7f1024004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1037fff700 1 --2- 172.21.15.29:0/42036927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10381514a0 0x7f1024004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1037fff700 1 -- 172.21.15.29:0/42036927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1024004b80 msgr2=0x7f1024005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1037fff700 1 --2- 172.21.15.29:0/42036927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1024004b80 0x7f1024005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1037fff700 1 -- 172.21.15.29:0/42036927 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10240033f0 con 0x7f10240038e0 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.247+0000 7f1037fff700 1 --2- 172.21.15.29:0/42036927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10240038e0 0x7f1024003d00 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f102800c210 tx=0x7f1028001e90).ready entity=mon.0 client_cookie=c3aaf3da8a36a5db server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.248+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028007a60 con 0x7f10240038e0 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.248+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102800cae0 con 0x7f10240038e0 2024-01-24T14:16:15.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.248+0000 7f1036ffd700 1 -- 172.21.15.29:0/42036927 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102800ce60 con 0x7f10240038e0 2024-01-24T14:16:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.248+0000 7f103e2e9700 1 -- 172.21.15.29:0/42036927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10240038e0 msgr2=0x7f1024003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.248+0000 7f103e2e9700 1 --2- 172.21.15.29:0/42036927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10240038e0 0x7f1024003d00 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f102800c210 tx=0x7f1028001e90).stop 2024-01-24T14:16:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.248+0000 7f103c886700 1 -- 172.21.15.29:0/42036927 reap_dead start 2024-01-24T14:16:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.248+0000 7f103e2e9700 1 -- 172.21.15.29:0/42036927 shutdown_connections 2024-01-24T14:16:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.248+0000 7f103e2e9700 1 -- 172.21.15.29:0/42036927 >> 172.21.15.29:0/42036927 conn(0x7f10380c8580 msgr2=0x7f10380c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.248+0000 7f103e2e9700 1 -- 172.21.15.29:0/42036927 shutdown_connections 2024-01-24T14:16:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.248+0000 7f103e2e9700 1 -- 172.21.15.29:0/42036927 wait complete. 2024-01-24T14:16:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.250+0000 7f103e2e9700 1 Processor -- start 2024-01-24T14:16:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.250+0000 7f103e2e9700 1 -- start start 2024-01-24T14:16:15.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f103e2e9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10240038e0 0x7f10380c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f103e2e9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10380c3cb0 0x7f103814af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f103e2e9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10380bd130 0x7f10380bd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f103e2e9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1038165020 con 0x7f10240038e0 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f103e2e9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10380bda80 con 0x7f10380bd130 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f103e2e9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10381646f0 con 0x7f10380c3cb0 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f1037fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10240038e0 0x7f10380c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f1037fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10240038e0 0x7f10380c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51068/0 (socket says 172.21.15.29:51068) 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f1037fff700 1 -- 172.21.15.29:0/2033784928 learned_addr learned my addr 172.21.15.29:0/2033784928 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f1037fff700 1 -- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10380c3cb0 msgr2=0x7f103814af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f1037fff700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10380c3cb0 0x7f103814af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f103c886700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10380bd130 0x7f10380bd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f1037fff700 1 -- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10380bd130 msgr2=0x7f10380bd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f1037fff700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10380bd130 0x7f10380bd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f1037fff700 1 -- 172.21.15.29:0/2033784928 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10381649d0 con 0x7f10240038e0 2024-01-24T14:16:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f1037fff700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10240038e0 0x7f10380c4530 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f1024004490 tx=0x7f1028015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f102ffff700 1 -- 172.21.15.29:0/2033784928 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028007640 con 0x7f10240038e0 2024-01-24T14:16:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f102ffff700 1 -- 172.21.15.29:0/2033784928 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10280046b0 con 0x7f10240038e0 2024-01-24T14:16:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f103e2e9700 1 -- 172.21.15.29:0/2033784928 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1038164cb0 con 0x7f10240038e0 2024-01-24T14:16:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f102ffff700 1 -- 172.21.15.29:0/2033784928 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028004a50 con 0x7f10240038e0 2024-01-24T14:16:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.251+0000 7f103e2e9700 1 -- 172.21.15.29:0/2033784928 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10380c4a20 con 0x7f10240038e0 2024-01-24T14:16:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.253+0000 7f102ffff700 1 -- 172.21.15.29:0/2033784928 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1028013450 con 0x7f10240038e0 2024-01-24T14:16:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.253+0000 7f102ffff700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f101805f240 0x7f1018061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.253+0000 7f102ffff700 1 -- 172.21.15.29:0/2033784928 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f102807b620 con 0x7f10240038e0 2024-01-24T14:16:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.253+0000 7f10377fe700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f101805f240 0x7f1018061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.254+0000 7f10377fe700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f101805f240 0x7f1018061700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1020006d80 tx=0x7f1020009060).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.256+0000 7f104fcb4b80 1 Processor -- start 2024-01-24T14:16:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.256+0000 7f104fcb4b80 1 -- start start 2024-01-24T14:16:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f104fcb4b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 0x558dc367ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f104fcb4b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc36c52c0 0x558dc3677900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f104fcb4b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc3678270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f104fcb4b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc35f33d0 con 0x558dc367c620 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f104fcb4b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc35b8be0 con 0x558dc36c52c0 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f104fcb4b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558dc35ce260 con 0x558dc3677df0 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f1037fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 0x558dc367ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f1037fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 0x558dc367ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51082/0 (socket says 172.21.15.29:51082) 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f1037fff700 1 -- 172.21.15.29:0/2995832873 learned_addr learned my addr 172.21.15.29:0/2995832873 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f103c886700 1 --2- 172.21.15.29:0/2995832873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc3678270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f1037fff700 1 -- 172.21.15.29:0/2995832873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 msgr2=0x558dc3678270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f10377fe700 1 --2- 172.21.15.29:0/2995832873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc36c52c0 0x558dc3677900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f1037fff700 1 --2- 172.21.15.29:0/2995832873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc3678270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f1037fff700 1 -- 172.21.15.29:0/2995832873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc36c52c0 msgr2=0x558dc3677900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f1037fff700 1 --2- 172.21.15.29:0/2995832873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc36c52c0 0x558dc3677900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f1037fff700 1 -- 172.21.15.29:0/2995832873 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dc35c72f0 con 0x558dc367c620 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f103c886700 1 --2- 172.21.15.29:0/2995832873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc3678270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f1037fff700 1 --2- 172.21.15.29:0/2995832873 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 0x558dc367ca00 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f1028000fd0 tx=0x7f102800f000).ready entity=mon.0 client_cookie=ef94f69c7b4526a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.257+0000 7f102d7fa700 1 -- 172.21.15.29:0/2995832873 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028004e40 con 0x558dc367c620 2024-01-24T14:16:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.258+0000 7f102d7fa700 1 -- 172.21.15.29:0/2995832873 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102804ff70 con 0x558dc367c620 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.258+0000 7f102d7fa700 1 -- 172.21.15.29:0/2995832873 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028055710 con 0x558dc367c620 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.258+0000 7f104fcb4b80 1 -- 172.21.15.29:0/2995832873 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 msgr2=0x558dc367ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.258+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/2995832873 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 0x558dc367ca00 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f1028000fd0 tx=0x7f102800f000).stop 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.258+0000 7f104fcb4b80 1 -- 172.21.15.29:0/2995832873 shutdown_connections 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.258+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/2995832873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc3678270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.258+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/2995832873 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 0x558dc367ca00 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.258+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/2995832873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc36c52c0 0x558dc3677900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.258+0000 7f104fcb4b80 1 -- 172.21.15.29:0/2995832873 >> 172.21.15.29:0/2995832873 conn(0x558dc34e52e0 msgr2=0x558dc36c5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.258+0000 7f104fcb4b80 1 -- 172.21.15.29:0/2995832873 shutdown_connections 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.258+0000 7f104fcb4b80 1 -- 172.21.15.29:0/2995832873 wait complete. 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.259+0000 7f104fcb4b80 1 -- start start 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.259+0000 7f104fcb4b80 1 Processor -- start 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f104fcb4b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc36a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f104fcb4b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 0x558dc3694040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f104fcb4b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc36c52c0 0x558dc36aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f104fcb4b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc35ce3a0 con 0x558dc367c620 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f104fcb4b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc35ce620 con 0x558dc36c52c0 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f104fcb4b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558dc3599990 con 0x558dc3677df0 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f104fcb4b80 10 client.0 ll_register_callbacks cb 0x558dc363c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f10377fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 0x558dc3694040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f10377fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 0x558dc3694040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51094/0 (socket says 172.21.15.29:51094) 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f10377fe700 1 -- 172.21.15.29:0/4105037896 learned_addr learned my addr 172.21.15.29:0/4105037896 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f103c886700 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc36c52c0 0x558dc36aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f1037fff700 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc36a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f10377fe700 1 -- 172.21.15.29:0/4105037896 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 msgr2=0x558dc36a90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f10377fe700 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc36a90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f10377fe700 1 -- 172.21.15.29:0/4105037896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc36c52c0 msgr2=0x558dc36aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f10377fe700 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc36c52c0 0x558dc36aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f10377fe700 1 -- 172.21.15.29:0/4105037896 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10380c4a20 con 0x558dc367c620 2024-01-24T14:16:15.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.260+0000 7f103c886700 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc36c52c0 0x558dc36aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f10377fe700 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 0x558dc3694040 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f1020006c40 tx=0x7f1020017000).ready entity=mon.0 client_cookie=422d55a753f081c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f10167fc700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f10167fc700 1 -- 172.21.15.29:0/4105037896 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020014c60 con 0x558dc367c620 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f104fcb4b80 1 -- 172.21.15.29:0/4105037896 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558dc3676df0 con 0x558dc367c620 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f10167fc700 1 -- 172.21.15.29:0/4105037896 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f1020014dc0 con 0x558dc367c620 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f10167fc700 1 -- 172.21.15.29:0/4105037896 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f102001f670 con 0x558dc367c620 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f104fcb4b80 10 client.7872 fetch_fsmap learned FSMap version 59 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f104fcb4b80 1 -- 172.21.15.29:0/4105037896 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x558dc35c2520 con 0x558dc367c620 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f10167fc700 1 -- 172.21.15.29:0/4105037896 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558dc3676df0 con 0x558dc367c620 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f10167fc700 1 -- 172.21.15.29:0/4105037896 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 59) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f102001f7d0 con 0x558dc367c620 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f104fcb4b80 10 client.7872 fetch_fsmap finished waiting for FSMap version 59 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f104fcb4b80 10 client.7872 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-24T14:16:15.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f104fcb4b80 20 client.7872 populate_metadata read hostname 'smithi029' 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.29:6833/1458876872,v1:172.21.15.29:6835/1458876872] conn(0x558dc36a72c0 0x558dc36a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f104fcb4b80 4 client.7872 mds_command: new command op to 7728 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.261+0000 7f104fcb4b80 1 -- 172.21.15.29:0/4105037896 --> [v2:172.21.15.29:6833/1458876872,v1:172.21.15.29:6835/1458876872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dc35c3d60 con 0x558dc36a72c0 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.262+0000 7f1037fff700 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.29:6833/1458876872,v1:172.21.15.29:6835/1458876872] conn(0x558dc36a72c0 0x558dc36a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.262+0000 7f10167fc700 10 client.7872 ms_handle_connect on v2:172.21.15.29:6833/1458876872 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.262+0000 7f1037fff700 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.29:6833/1458876872,v1:172.21.15.29:6835/1458876872] conn(0x558dc36a72c0 0x558dc36a76a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e0072fa06f765c2e server_cookie=5a35cc9a9b41d8f0 in_seq=0 out_seq=0 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.262+0000 7f10167fc700 1 -- 172.21.15.29:0/4105037896 <== mds.0 v2:172.21.15.29:6833/1458876872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x558dc35c3d60 con 0x558dc36a72c0 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.262+0000 7f10167fc700 10 client.7872 handle_command_reply: tid=0 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f104fcb4b80 1 client.7872 shutdown 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f104fcb4b80 1 -- 172.21.15.29:0/4105037896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 msgr2=0x558dc3694040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 0x558dc3694040 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f1020006c40 tx=0x7f1020017000).stop 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f104fcb4b80 1 -- 172.21.15.29:0/4105037896 shutdown_connections 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.29:6833/1458876872,v1:172.21.15.29:6835/1458876872] conn(0x558dc36a72c0 0x558dc36a76a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc36c52c0 0x558dc36aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc36a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/4105037896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc367c620 0x558dc3694040 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f10167fc700 0 client.7872 ms_handle_reset on v2:172.21.15.29:6833/1458876872 2024-01-24T14:16:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f104fcb4b80 1 -- 172.21.15.29:0/4105037896 >> 172.21.15.29:0/4105037896 conn(0x558dc34e52e0 msgr2=0x558dc367ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f104fcb4b80 1 -- 172.21.15.29:0/4105037896 shutdown_connections 2024-01-24T14:16:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f104fcb4b80 1 -- 172.21.15.29:0/4105037896 wait complete. 2024-01-24T14:16:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.263+0000 7f104fcb4b80 20 client.7872 trim_cache size 0 max 16384 2024-01-24T14:16:15.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.281+0000 7f104fcb4b80 1 Processor -- start 2024-01-24T14:16:15.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.281+0000 7f104fcb4b80 1 -- start start 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f104fcb4b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc3677df0 0x558dc36c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f104fcb4b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc367c620 0x558dc36a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f104fcb4b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 0x558dc36a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f104fcb4b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc36a7a00 con 0x558dc36c52c0 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f104fcb4b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc367c2d0 con 0x558dc3677df0 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f104fcb4b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558dc3694140 con 0x558dc367c620 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f103c886700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 0x558dc36a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f103c886700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 0x558dc36a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51098/0 (socket says 172.21.15.29:51098) 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f103c886700 1 -- 172.21.15.29:0/2067248151 learned_addr learned my addr 172.21.15.29:0/2067248151 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f1037fff700 1 --2- 172.21.15.29:0/2067248151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc3677df0 0x558dc36c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f10377fe700 1 --2- 172.21.15.29:0/2067248151 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc367c620 0x558dc36a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f103c886700 1 -- 172.21.15.29:0/2067248151 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc367c620 msgr2=0x558dc36a3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f103c886700 1 --2- 172.21.15.29:0/2067248151 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc367c620 0x558dc36a3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f103c886700 1 -- 172.21.15.29:0/2067248151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc3677df0 msgr2=0x558dc36c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f103c886700 1 --2- 172.21.15.29:0/2067248151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc3677df0 0x558dc36c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f103c886700 1 -- 172.21.15.29:0/2067248151 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dc35c2270 con 0x558dc36c52c0 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f1037fff700 1 --2- 172.21.15.29:0/2067248151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc3677df0 0x558dc36c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f103c886700 1 --2- 172.21.15.29:0/2067248151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 0x558dc36a43f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f1030015520 tx=0x7f103000b040).ready entity=mon.0 client_cookie=106d15bd11a85222 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f102dffb700 1 -- 172.21.15.29:0/2067248151 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103000e040 con 0x558dc36c52c0 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f102dffb700 1 -- 172.21.15.29:0/2067248151 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103000f040 con 0x558dc36c52c0 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.282+0000 7f102dffb700 1 -- 172.21.15.29:0/2067248151 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030018870 con 0x558dc36c52c0 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.283+0000 7f104fcb4b80 1 -- 172.21.15.29:0/2067248151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 msgr2=0x558dc36a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.283+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/2067248151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 0x558dc36a43f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f1030015520 tx=0x7f103000b040).stop 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.283+0000 7f104fcb4b80 1 -- 172.21.15.29:0/2067248151 shutdown_connections 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.283+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/2067248151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 0x558dc36a43f0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.283+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/2067248151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc3677df0 0x558dc36c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.283+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/2067248151 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc367c620 0x558dc36a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.283+0000 7f104fcb4b80 1 -- 172.21.15.29:0/2067248151 >> 172.21.15.29:0/2067248151 conn(0x558dc34e52e0 msgr2=0x558dc367a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:15.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.283+0000 7f104fcb4b80 1 -- 172.21.15.29:0/2067248151 shutdown_connections 2024-01-24T14:16:15.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.283+0000 7f104fcb4b80 1 -- 172.21.15.29:0/2067248151 wait complete. 2024-01-24T14:16:15.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.284+0000 7f104fcb4b80 1 -- start start 2024-01-24T14:16:15.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.284+0000 7f104fcb4b80 1 Processor -- start 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f104fcb4b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc3726420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f104fcb4b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc367c620 0x558dc36c0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f104fcb4b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 0x558dc36bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f104fcb4b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc36da260 con 0x558dc36c52c0 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f104fcb4b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc36a8810 con 0x558dc367c620 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f104fcb4b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558dc36c03e0 con 0x558dc3677df0 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f104fcb4b80 10 client.0 ll_register_callbacks cb 0x558dc363c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f103c886700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 0x558dc36bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f103c886700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 0x558dc36bfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51108/0 (socket says 172.21.15.29:51108) 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f103c886700 1 -- 172.21.15.29:0/1989913621 learned_addr learned my addr 172.21.15.29:0/1989913621 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f1037fff700 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc3726420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f10377fe700 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc367c620 0x558dc36c0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f103c886700 1 -- 172.21.15.29:0/1989913621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 msgr2=0x558dc3726420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f103c886700 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc3726420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f103c886700 1 -- 172.21.15.29:0/1989913621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc367c620 msgr2=0x558dc36c0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f103c886700 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc367c620 0x558dc36c0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f103c886700 1 -- 172.21.15.29:0/1989913621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10300152d0 con 0x558dc36c52c0 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f103c886700 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 0x558dc36bfd30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f103001a000 tx=0x7f1030020000).ready entity=mon.0 client_cookie=f866f67f7dea504a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f10167fc700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f10167fc700 1 -- 172.21.15.29:0/1989913621 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103000e040 con 0x558dc36c52c0 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f10167fc700 1 -- 172.21.15.29:0/1989913621 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f103002a430 con 0x558dc36c52c0 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.285+0000 7f104fcb4b80 1 -- 172.21.15.29:0/1989913621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f102804ff70 con 0x558dc36c52c0 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f10167fc700 1 -- 172.21.15.29:0/1989913621 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f103002a800 con 0x558dc36c52c0 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f104fcb4b80 10 client.7878 fetch_fsmap learned FSMap version 59 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f104fcb4b80 1 -- 172.21.15.29:0/1989913621 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x558dc343bf80 con 0x558dc36c52c0 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f10167fc700 1 -- 172.21.15.29:0/1989913621 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102804ff70 con 0x558dc36c52c0 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f10167fc700 1 -- 172.21.15.29:0/1989913621 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 59) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f103002a960 con 0x558dc36c52c0 2024-01-24T14:16:15.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f104fcb4b80 10 client.7878 fetch_fsmap finished waiting for FSMap version 59 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f104fcb4b80 10 client.7878 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f104fcb4b80 20 client.7878 populate_metadata read hostname 'smithi029' 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.29:6833/1458876872,v1:172.21.15.29:6835/1458876872] conn(0x558dc3693080 0x558dc3693460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f104fcb4b80 4 client.7878 mds_command: new command op to 7728 tid=0[{"prefix": "status"}] 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f104fcb4b80 1 -- 172.21.15.29:0/1989913621 --> [v2:172.21.15.29:6833/1458876872,v1:172.21.15.29:6835/1458876872] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558dc35ba7d0 con 0x558dc3693080 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f1037fff700 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.29:6833/1458876872,v1:172.21.15.29:6835/1458876872] conn(0x558dc3693080 0x558dc3693460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f10167fc700 10 client.7878 ms_handle_connect on v2:172.21.15.29:6833/1458876872 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f1037fff700 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.29:6833/1458876872,v1:172.21.15.29:6835/1458876872] conn(0x558dc3693080 0x558dc3693460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=cd1eb9f2840c3a7d server_cookie=578b946dbb4e924 in_seq=0 out_seq=0 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f10167fc700 1 -- 172.21.15.29:0/1989913621 <== mds.0 v2:172.21.15.29:6833/1458876872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558dc35ba7d0 con 0x558dc3693080 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.286+0000 7f10167fc700 10 client.7878 handle_command_reply: tid=0 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f104fcb4b80 1 client.7878 shutdown 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f104fcb4b80 1 -- 172.21.15.29:0/1989913621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 msgr2=0x558dc36bfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 0x558dc36bfd30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f103001a000 tx=0x7f1030020000).stop 2024-01-24T14:16:15.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f104fcb4b80 1 -- 172.21.15.29:0/1989913621 shutdown_connections 2024-01-24T14:16:15.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.29:6833/1458876872,v1:172.21.15.29:6835/1458876872] conn(0x558dc3693080 0x558dc3693460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558dc36c52c0 0x558dc36bfd30 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558dc3677df0 0x558dc3726420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f10167fc700 0 client.7878 ms_handle_reset on v2:172.21.15.29:6833/1458876872 2024-01-24T14:16:15.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f104fcb4b80 1 --2- 172.21.15.29:0/1989913621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558dc367c620 0x558dc36c0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f104fcb4b80 1 -- 172.21.15.29:0/1989913621 >> 172.21.15.29:0/1989913621 conn(0x558dc34e52e0 msgr2=0x558dc36c5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:15.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f104fcb4b80 1 -- 172.21.15.29:0/1989913621 shutdown_connections 2024-01-24T14:16:15.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f104fcb4b80 1 -- 172.21.15.29:0/1989913621 wait complete. 2024-01-24T14:16:15.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f104fcb4b80 20 client.7878 trim_cache size 0 max 16384 2024-01-24T14:16:15.294 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T14:16:15.295 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T14:16:15.295 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T14:16:15.295 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 7728, 2024-01-24T14:16:15.295 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T14:16:15.295 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T14:16:15.295 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T14:16:15.295 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 5.9533009310000002, 2024-01-24T14:16:15.295 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 59, 2024-01-24T14:16:15.295 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 137, 2024-01-24T14:16:15.295 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 137, 2024-01-24T14:16:15.295 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 11.414895524 2024-01-24T14:16:15.295 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T14:16:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f103e2e9700 1 -- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f101805f240 msgr2=0x7f1018061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f103e2e9700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f101805f240 0x7f1018061700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1020006d80 tx=0x7f1020009060).stop 2024-01-24T14:16:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f103e2e9700 1 -- 172.21.15.29:0/2033784928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10240038e0 msgr2=0x7f10380c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.287+0000 7f103e2e9700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10240038e0 0x7f10380c4530 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f1024004490 tx=0x7f1028015000).stop 2024-01-24T14:16:15.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.288+0000 7f103e2e9700 1 -- 172.21.15.29:0/2033784928 shutdown_connections 2024-01-24T14:16:15.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.288+0000 7f103e2e9700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10380bd130 0x7f10380bd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.288+0000 7f103e2e9700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f101805f240 0x7f1018061700 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.288+0000 7f103e2e9700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10240038e0 0x7f10380c4530 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.288+0000 7f103e2e9700 1 --2- 172.21.15.29:0/2033784928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10380c3cb0 0x7f103814af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:16:15.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.288+0000 7f103e2e9700 1 -- 172.21.15.29:0/2033784928 >> 172.21.15.29:0/2033784928 conn(0x7f10380c8580 msgr2=0x7f10380c58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:15.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.288+0000 7f103e2e9700 1 -- 172.21.15.29:0/2033784928 shutdown_connections 2024-01-24T14:16:15.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.288+0000 7f103e2e9700 1 -- 172.21.15.29:0/2033784928 wait complete. 2024-01-24T14:16:15.301 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T14:16:15.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.417+0000 7f6992672700 1 Processor -- start 2024-01-24T14:16:15.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.417+0000 7f6992672700 1 -- start start 2024-01-24T14:16:15.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6992672700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c0be420 0x7f698c0be8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6992672700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f698c0c6b50 0x7f698c0c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6992672700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698c0a7490 con 0x7f698c1520e0 2024-01-24T14:16:15.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6992672700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698c0bed90 con 0x7f698c0c6b50 2024-01-24T14:16:15.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6992672700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698c0a8520 con 0x7f698c0be420 2024-01-24T14:16:15.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6990c0f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f698c0c6b50 0x7f698c0c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6990c0f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f698c0c6b50 0x7f698c0c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51120/0 (socket says 172.21.15.29:51120) 2024-01-24T14:16:15.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6990c0f700 1 -- 172.21.15.29:0/2227361928 learned_addr learned my addr 172.21.15.29:0/2227361928 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:15.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f698b7fe700 1 --2- 172.21.15.29:0/2227361928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c0be420 0x7f698c0be8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6990c0f700 1 -- 172.21.15.29:0/2227361928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c0be420 msgr2=0x7f698c0be8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6990c0f700 1 --2- 172.21.15.29:0/2227361928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c0be420 0x7f698c0be8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6990c0f700 1 -- 172.21.15.29:0/2227361928 >> v1:172.21.15.163:6789/0 conn(0x7f698c1520e0 legacy=0x7f698c0a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6990c0f700 1 -- 172.21.15.29:0/2227361928 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c0a6880 con 0x7f698c0c6b50 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.418+0000 7f6990c0f700 1 --2- 172.21.15.29:0/2227361928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0c6b50 0x7f698c0c6fe0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f6984009f20 tx=0x7f6984009840).ready entity=mon.0 client_cookie=4f370b0b90916bad server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.419+0000 7f698affd700 1 -- 172.21.15.29:0/2227361928 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698400e070 con 0x7f698c0c6b50 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.419+0000 7f698affd700 1 -- 172.21.15.29:0/2227361928 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698400ba80 con 0x7f698c0c6b50 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.419+0000 7f698affd700 1 -- 172.21.15.29:0/2227361928 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698400be00 con 0x7f698c0c6b50 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.419+0000 7f6992672700 1 -- 172.21.15.29:0/2227361928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0c6b50 msgr2=0x7f698c0c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.419+0000 7f6992672700 1 --2- 172.21.15.29:0/2227361928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0c6b50 0x7f698c0c6fe0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f6984009f20 tx=0x7f6984009840).stop 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.419+0000 7f6992672700 1 -- 172.21.15.29:0/2227361928 shutdown_connections 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.419+0000 7f6992672700 1 --2- 172.21.15.29:0/2227361928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0c6b50 0x7f698c0c6fe0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.419+0000 7f6992672700 1 --2- 172.21.15.29:0/2227361928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c0be420 0x7f698c0be8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.419+0000 7f6992672700 1 -- 172.21.15.29:0/2227361928 >> 172.21.15.29:0/2227361928 conn(0x7f698c0922f0 msgr2=0x7f698c062f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:15.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.419+0000 7f6992672700 1 -- 172.21.15.29:0/2227361928 shutdown_connections 2024-01-24T14:16:15.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.419+0000 7f6992672700 1 -- 172.21.15.29:0/2227361928 wait complete. 2024-01-24T14:16:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.420+0000 7f6992672700 1 Processor -- start 2024-01-24T14:16:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.421+0000 7f6992672700 1 -- start start 2024-01-24T14:16:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.421+0000 7f6992672700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0be420 0x7f698c1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.421+0000 7f6992672700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c0c6b50 0x7f698c164340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.421+0000 7f6992672700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c1520e0 0x7f698c064160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.421+0000 7f6992672700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c063cd0 con 0x7f698c0be420 2024-01-24T14:16:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.421+0000 7f6992672700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c1507c0 con 0x7f698c1520e0 2024-01-24T14:16:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.421+0000 7f6992672700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698c164c60 con 0x7f698c0c6b50 2024-01-24T14:16:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f698bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0be420 0x7f698c1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f698bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0be420 0x7f698c1502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51128/0 (socket says 172.21.15.29:51128) 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f698bfff700 1 -- 172.21.15.29:0/4117393099 learned_addr learned my addr 172.21.15.29:0/4117393099 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f698b7fe700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c0c6b50 0x7f698c164340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f6990c0f700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c1520e0 0x7f698c064160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f698bfff700 1 -- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c0c6b50 msgr2=0x7f698c164340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f698bfff700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c0c6b50 0x7f698c164340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f698bfff700 1 -- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c1520e0 msgr2=0x7f698c064160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f698bfff700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c1520e0 0x7f698c064160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f698bfff700 1 -- 172.21.15.29:0/4117393099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6984009ae0 con 0x7f698c0be420 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f6990c0f700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c1520e0 0x7f698c064160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f698b7fe700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c0c6b50 0x7f698c164340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f698bfff700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0be420 0x7f698c1502d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7f697c00c4c0 tx=0x7f697c007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f6973fff700 1 -- 172.21.15.29:0/4117393099 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c007020 con 0x7f698c0be420 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f6973fff700 1 -- 172.21.15.29:0/4117393099 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c007180 con 0x7f698c0be420 2024-01-24T14:16:15.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f6992672700 1 -- 172.21.15.29:0/4117393099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698c164f40 con 0x7f698c0be420 2024-01-24T14:16:15.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f6973fff700 1 -- 172.21.15.29:0/4117393099 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c007520 con 0x7f698c0be420 2024-01-24T14:16:15.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.422+0000 7f6992672700 1 -- 172.21.15.29:0/4117393099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698c063960 con 0x7f698c0be420 2024-01-24T14:16:15.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.423+0000 7f6992672700 1 -- 172.21.15.29:0/4117393099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698c05f400 con 0x7f698c0be420 2024-01-24T14:16:15.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.424+0000 7f6973fff700 1 -- 172.21.15.29:0/4117393099 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f697c006900 con 0x7f698c0be420 2024-01-24T14:16:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.424+0000 7f6973fff700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f697805f270 0x7f6978061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.425+0000 7f6973fff700 1 -- 172.21.15.29:0/4117393099 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f697c07a8b0 con 0x7f698c0be420 2024-01-24T14:16:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.425+0000 7f698b7fe700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f697805f270 0x7f6978061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.425+0000 7f698b7fe700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f697805f270 0x7f6978061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f6980006fd0 tx=0x7f6980008040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:15.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.428+0000 7f6973fff700 1 -- 172.21.15.29:0/4117393099 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f697c04b6c0 con 0x7f698c0be420 2024-01-24T14:16:15.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.656+0000 7f6992672700 1 -- 172.21.15.29:0/4117393099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f698c063fb0 con 0x7f698c0be420 2024-01-24T14:16:15.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.657+0000 7f6973fff700 1 -- 172.21.15.29:0/4117393099 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) v1 ==== 53+0+73 (secure 0 0 0) 0x7f697c04ece0 con 0x7f698c0be420 2024-01-24T14:16:15.658 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-24T14:16:15.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.659+0000 7f6992672700 1 -- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f697805f270 msgr2=0x7f6978061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.659+0000 7f6992672700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f697805f270 0x7f6978061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f6980006fd0 tx=0x7f6980008040).stop 2024-01-24T14:16:15.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.659+0000 7f6992672700 1 -- 172.21.15.29:0/4117393099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0be420 msgr2=0x7f698c1502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:15.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.659+0000 7f6992672700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0be420 0x7f698c1502d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7f697c00c4c0 tx=0x7f697c007df0).stop 2024-01-24T14:16:15.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.659+0000 7f6992672700 1 -- 172.21.15.29:0/4117393099 shutdown_connections 2024-01-24T14:16:15.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.659+0000 7f6992672700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c1520e0 0x7f698c064160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.659+0000 7f6992672700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f697805f270 0x7f6978061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.659+0000 7f6992672700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0be420 0x7f698c1502d0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.659+0000 7f6992672700 1 --2- 172.21.15.29:0/4117393099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c0c6b50 0x7f698c164340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:15.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.659+0000 7f6992672700 1 -- 172.21.15.29:0/4117393099 >> 172.21.15.29:0/4117393099 conn(0x7f698c0922f0 msgr2=0x7f698c0c5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:15.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.659+0000 7f6992672700 1 -- 172.21.15.29:0/4117393099 shutdown_connections 2024-01-24T14:16:15.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:15.659+0000 7f6992672700 1 -- 172.21.15.29:0/4117393099 wait complete. 2024-01-24T14:16:15.671 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-24T14:16:15.672 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T14:16:15.672 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T14:16:15.672 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-24T14:16:15.672 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-24T14:16:15.672 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-24T14:16:15.672 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-24T14:16:15.672 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T14:16:15.672 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T14:16:15.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:16:15.672 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T14:16:15.692 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T14:16:15.692 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T14:16:15.692 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T14:16:15.692 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:16:15.692 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T14:16:15.692 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:16:15.692 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 41275sec preferred_lft 41275sec 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:16:15.693 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::24bc:1cff:fe57:e704/64 scope link 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::2034:aaff:fef3:f12b/64 scope link 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether ba:ca:42:4a:9e:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T14:16:15.694 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::b8ca:42ff:fe4a:9e52/64 scope link 2024-01-24T14:16:15.695 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:16:15.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:16:15.695 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T14:16:15.763 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T14:16:15.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:16:15.764 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T14:16:15.829 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T14:16:15.829 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T14:16:15.829 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T14:16:15.829 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T14:16:15.829 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T14:16:15.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:16:15.830 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T14:16:15.830 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T14:16:15.830 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T14:16:15.830 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T14:16:15.830 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T14:16:15.830 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T14:16:15.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:16:15.954 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T14:16:15.974 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-24T14:16:15.975 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T14:16:15.975 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T14:16:15.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:16:15.975 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T14:16:16.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:16:16.049 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-01-24T14:16:16.112 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T14:16:16.112 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:16:16.204 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:16:16.204 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T14:16:16.225 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:16:16.225 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T14:16:16.228 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T14:16:16.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:16:16.228 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-24T14:16:16.260 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-24T14:16:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.440+0000 7f51db061700 1 Processor -- start 2024-01-24T14:16:16.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.441+0000 7f51db061700 1 -- start start 2024-01-24T14:16:16.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.441+0000 7f51db061700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f51d4154070 0x7f51d40a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.441+0000 7f51db061700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51d4154450 0x7f51d4066750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.441+0000 7f51db061700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51d40c4ce0 0x7f51d40627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.441+0000 7f51db061700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d40bd160 con 0x7f51d4154450 2024-01-24T14:16:16.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.441+0000 7f51db061700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d40bcba0 con 0x7f51d4154070 2024-01-24T14:16:16.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.441+0000 7f51db061700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40bce80 con 0x7f51d40c4ce0 2024-01-24T14:16:16.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d3fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51d4154450 0x7f51d4066750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d3fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51d4154450 0x7f51d4066750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51140/0 (socket says 172.21.15.29:51140) 2024-01-24T14:16:16.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d3fff700 1 -- 172.21.15.29:0/3487736286 learned_addr learned my addr 172.21.15.29:0/3487736286 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:16.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d95fe700 1 --2- 172.21.15.29:0/3487736286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51d40c4ce0 0x7f51d40627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d8dfd700 1 --2- 172.21.15.29:0/3487736286 >> v2:172.21.15.163:3300/0 conn(0x7f51d4154070 0x7f51d40a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d3fff700 1 -- 172.21.15.29:0/3487736286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51d40c4ce0 msgr2=0x7f51d40627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d3fff700 1 --2- 172.21.15.29:0/3487736286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51d40c4ce0 0x7f51d40627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d3fff700 1 -- 172.21.15.29:0/3487736286 >> v2:172.21.15.163:3300/0 conn(0x7f51d4154070 msgr2=0x7f51d40a6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d3fff700 1 --2- 172.21.15.29:0/3487736286 >> v2:172.21.15.163:3300/0 conn(0x7f51d4154070 0x7f51d40a6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d3fff700 1 -- 172.21.15.29:0/3487736286 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d4065f50 con 0x7f51d4154450 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d95fe700 1 --2- 172.21.15.29:0/3487736286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51d40c4ce0 0x7f51d40627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d8dfd700 1 --2- 172.21.15.29:0/3487736286 >> v2:172.21.15.163:3300/0 conn(0x7f51d4154070 0x7f51d40a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d3fff700 1 --2- 172.21.15.29:0/3487736286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51d4154450 0x7f51d4066750 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f51d40c58c0 tx=0x7f51c00090f0).ready entity=mon.0 client_cookie=f615428703341e94 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d37fe700 1 -- 172.21.15.29:0/3487736286 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c000b600 con 0x7f51d4154450 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d37fe700 1 -- 172.21.15.29:0/3487736286 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c000b760 con 0x7f51d4154450 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.442+0000 7f51d37fe700 1 -- 172.21.15.29:0/3487736286 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c000bae0 con 0x7f51d4154450 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.443+0000 7f51db061700 1 -- 172.21.15.29:0/3487736286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51d4154450 msgr2=0x7f51d4066750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.443+0000 7f51db061700 1 --2- 172.21.15.29:0/3487736286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51d4154450 0x7f51d4066750 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f51d40c58c0 tx=0x7f51c00090f0).stop 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.443+0000 7f51db061700 1 -- 172.21.15.29:0/3487736286 shutdown_connections 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.443+0000 7f51db061700 1 --2- 172.21.15.29:0/3487736286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51d40c4ce0 0x7f51d40627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.443+0000 7f51db061700 1 --2- 172.21.15.29:0/3487736286 >> v2:172.21.15.163:3300/0 conn(0x7f51d4154070 0x7f51d40a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.443+0000 7f51db061700 1 --2- 172.21.15.29:0/3487736286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51d4154450 0x7f51d4066750 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.443+0000 7f51db061700 1 -- 172.21.15.29:0/3487736286 >> 172.21.15.29:0/3487736286 conn(0x7f51d40c8590 msgr2=0x7f51d40c4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:16.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.443+0000 7f51db061700 1 -- 172.21.15.29:0/3487736286 shutdown_connections 2024-01-24T14:16:16.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.443+0000 7f51db061700 1 -- 172.21.15.29:0/3487736286 wait complete. 2024-01-24T14:16:16.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.444+0000 7f51db061700 1 Processor -- start 2024-01-24T14:16:16.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.445+0000 7f51db061700 1 -- start start 2024-01-24T14:16:16.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.445+0000 7f51db061700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51d40c4ce0 0x7f51d4147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.445+0000 7f51db061700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51d4154070 0x7f51d4164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.445+0000 7f51db061700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51d4154450 0x7f51d40abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.445+0000 7f51db061700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d41651c0 con 0x7f51d4154070 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.445+0000 7f51db061700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40ab820 con 0x7f51d4154450 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.445+0000 7f51db061700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40aac10 con 0x7f51d40c4ce0 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.445+0000 7f51d3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51d4154070 0x7f51d4164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.445+0000 7f51d3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51d4154070 0x7f51d4164cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51152/0 (socket says 172.21.15.29:51152) 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d3fff700 1 -- 172.21.15.29:0/1723742200 learned_addr learned my addr 172.21.15.29:0/1723742200 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d8dfd700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51d40c4ce0 0x7f51d4147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d95fe700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51d4154450 0x7f51d40abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d3fff700 1 -- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51d40c4ce0 msgr2=0x7f51d4147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d3fff700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51d40c4ce0 0x7f51d4147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d3fff700 1 -- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51d4154450 msgr2=0x7f51d40abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:16.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d3fff700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51d4154450 0x7f51d40abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d3fff700 1 -- 172.21.15.29:0/1723742200 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c0009370 con 0x7f51d4154070 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d95fe700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51d4154450 0x7f51d40abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d3fff700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51d4154070 0x7f51d4164cd0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7f51d4068a50 tx=0x7f51c0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d0ff9700 1 -- 172.21.15.29:0/1723742200 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c000b8d0 con 0x7f51d4154070 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d0ff9700 1 -- 172.21.15.29:0/1723742200 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c000ba30 con 0x7f51d4154070 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51d0ff9700 1 -- 172.21.15.29:0/1723742200 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c001b660 con 0x7f51d4154070 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51db061700 1 -- 172.21.15.29:0/1723742200 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d40aaef0 con 0x7f51d4154070 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.446+0000 7f51db061700 1 -- 172.21.15.29:0/1723742200 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d40ab4b0 con 0x7f51d4154070 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.447+0000 7f51db061700 1 -- 172.21.15.29:0/1723742200 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d40ab990 con 0x7f51d4154070 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.448+0000 7f51d0ff9700 1 -- 172.21.15.29:0/1723742200 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f51c001ba50 con 0x7f51d4154070 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.449+0000 7f51d0ff9700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f51b805f240 0x7f51b8061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.449+0000 7f51d0ff9700 1 -- 172.21.15.29:0/1723742200 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f51c007c780 con 0x7f51d4154070 2024-01-24T14:16:16.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.449+0000 7f51d8dfd700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f51b805f240 0x7f51b8061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.449+0000 7f51d8dfd700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f51b805f240 0x7f51b8061700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f51c8006e00 tx=0x7f51c8008040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:16.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.453+0000 7f51d0ff9700 1 -- 172.21.15.29:0/1723742200 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f51c004d590 con 0x7f51d4154070 2024-01-24T14:16:16.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.661+0000 7f51db061700 1 -- 172.21.15.29:0/1723742200 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f51d4066230 con 0x7f51b805f240 2024-01-24T14:16:16.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.663+0000 7f51d0ff9700 1 -- 172.21.15.29:0/1723742200 <== mgr.7195 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f51d4066230 con 0x7f51b805f240 2024-01-24T14:16:16.663 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T14:16:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:16:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-01-24T14:16:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:16:16.663 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T14:16:16.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.664+0000 7f51db061700 1 -- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f51b805f240 msgr2=0x7f51b8061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.664+0000 7f51db061700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f51b805f240 0x7f51b8061700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f51c8006e00 tx=0x7f51c8008040).stop 2024-01-24T14:16:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.664+0000 7f51db061700 1 -- 172.21.15.29:0/1723742200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51d4154070 msgr2=0x7f51d4164cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.664+0000 7f51db061700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51d4154070 0x7f51d4164cd0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7f51d4068a50 tx=0x7f51c0017000).stop 2024-01-24T14:16:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.664+0000 7f51db061700 1 -- 172.21.15.29:0/1723742200 shutdown_connections 2024-01-24T14:16:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.664+0000 7f51db061700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51d4154450 0x7f51d40abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.664+0000 7f51db061700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51d40c4ce0 0x7f51d4147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.664+0000 7f51db061700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f51b805f240 0x7f51b8061700 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.664+0000 7f51db061700 1 --2- 172.21.15.29:0/1723742200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51d4154070 0x7f51d4164cd0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.664+0000 7f51db061700 1 -- 172.21.15.29:0/1723742200 >> 172.21.15.29:0/1723742200 conn(0x7f51d40c8590 msgr2=0x7f51d40633a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.664+0000 7f51db061700 1 -- 172.21.15.29:0/1723742200 shutdown_connections 2024-01-24T14:16:16.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.665+0000 7f51db061700 1 -- 172.21.15.29:0/1723742200 wait complete. 2024-01-24T14:16:16.678 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-01-24T14:16:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.809+0000 7ff2804b1700 1 Processor -- start 2024-01-24T14:16:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.809+0000 7ff2804b1700 1 -- start start 2024-01-24T14:16:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.810+0000 7ff2804b1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff278134e70 0x7ff2780c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.810+0000 7ff2804b1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff278068900 0x7ff278152e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.810+0000 7ff2804b1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff278066520 0x7ff2780669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.810+0000 7ff2804b1700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27814c910 con 0x7ff278068900 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff2804b1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff278068470 con 0x7ff278066520 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff2804b1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff278066ea0 con 0x7ff278134e70 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27da4c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff278068900 0x7ff278152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27da4c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff278068900 0x7ff278152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51160/0 (socket says 172.21.15.29:51160) 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27da4c700 1 -- 172.21.15.29:0/2908924265 learned_addr learned my addr 172.21.15.29:0/2908924265 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27ea4e700 1 --2- 172.21.15.29:0/2908924265 >> v2:172.21.15.163:3300/0 conn(0x7ff278066520 0x7ff2780669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27e24d700 1 --2- 172.21.15.29:0/2908924265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff278134e70 0x7ff2780c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27da4c700 1 -- 172.21.15.29:0/2908924265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff278134e70 msgr2=0x7ff2780c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27da4c700 1 --2- 172.21.15.29:0/2908924265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff278134e70 0x7ff2780c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27da4c700 1 -- 172.21.15.29:0/2908924265 >> v2:172.21.15.163:3300/0 conn(0x7ff278066520 msgr2=0x7ff2780669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27da4c700 1 --2- 172.21.15.29:0/2908924265 >> v2:172.21.15.163:3300/0 conn(0x7ff278066520 0x7ff2780669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27da4c700 1 -- 172.21.15.29:0/2908924265 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278153340 con 0x7ff278068900 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27ea4e700 1 --2- 172.21.15.29:0/2908924265 >> v2:172.21.15.163:3300/0 conn(0x7ff278066520 0x7ff2780669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27da4c700 1 --2- 172.21.15.29:0/2908924265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff278068900 0x7ff278152e50 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7ff2780c55f0 tx=0x7ff2740090f0).ready entity=mon.0 client_cookie=b12db6be9bbd117f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27d24b700 1 -- 172.21.15.29:0/2908924265 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400b600 con 0x7ff278068900 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27d24b700 1 -- 172.21.15.29:0/2908924265 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27400b760 con 0x7ff278068900 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.811+0000 7ff27d24b700 1 -- 172.21.15.29:0/2908924265 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400bae0 con 0x7ff278068900 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.812+0000 7ff2804b1700 1 -- 172.21.15.29:0/2908924265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff278068900 msgr2=0x7ff278152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.812+0000 7ff2804b1700 1 --2- 172.21.15.29:0/2908924265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff278068900 0x7ff278152e50 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7ff2780c55f0 tx=0x7ff2740090f0).stop 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.812+0000 7ff2804b1700 1 -- 172.21.15.29:0/2908924265 shutdown_connections 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.812+0000 7ff2804b1700 1 --2- 172.21.15.29:0/2908924265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff278134e70 0x7ff2780c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.812+0000 7ff2804b1700 1 --2- 172.21.15.29:0/2908924265 >> v2:172.21.15.163:3300/0 conn(0x7ff278066520 0x7ff2780669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.812+0000 7ff2804b1700 1 --2- 172.21.15.29:0/2908924265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff278068900 0x7ff278152e50 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.812+0000 7ff2804b1700 1 -- 172.21.15.29:0/2908924265 >> 172.21.15.29:0/2908924265 conn(0x7ff2780c8ba0 msgr2=0x7ff2780c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:16.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.812+0000 7ff2804b1700 1 -- 172.21.15.29:0/2908924265 shutdown_connections 2024-01-24T14:16:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.812+0000 7ff2804b1700 1 -- 172.21.15.29:0/2908924265 wait complete. 2024-01-24T14:16:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.813+0000 7ff2804b1700 1 Processor -- start 2024-01-24T14:16:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.813+0000 7ff2804b1700 1 -- start start 2024-01-24T14:16:16.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.814+0000 7ff2804b1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff278066520 0x7ff27814ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.814+0000 7ff2804b1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff278068900 0x7ff2780abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.814+0000 7ff2804b1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff278134e70 0x7ff2780ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.814+0000 7ff2804b1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278136570 con 0x7ff278134e70 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.814+0000 7ff2804b1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278136e10 con 0x7ff278068900 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.814+0000 7ff2804b1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2781383c0 con 0x7ff278066520 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.814+0000 7ff27ea4e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff278134e70 0x7ff2780ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.814+0000 7ff27ea4e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff278134e70 0x7ff2780ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51172/0 (socket says 172.21.15.29:51172) 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.814+0000 7ff27ea4e700 1 -- 172.21.15.29:0/2543944935 learned_addr learned my addr 172.21.15.29:0/2543944935 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff27e24d700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff278066520 0x7ff27814ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff27da4c700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff278068900 0x7ff2780abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff27ea4e700 1 -- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff278066520 msgr2=0x7ff27814ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff27ea4e700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff278066520 0x7ff27814ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff27ea4e700 1 -- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff278068900 msgr2=0x7ff2780abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff27ea4e700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff278068900 0x7ff2780abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff27ea4e700 1 -- 172.21.15.29:0/2543944935 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff274009370 con 0x7ff278134e70 2024-01-24T14:16:16.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff27ea4e700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff278134e70 0x7ff2780ac040 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7ff27000d100 tx=0x7ff2700099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff26a7fc700 1 -- 172.21.15.29:0/2543944935 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27000dda0 con 0x7ff278134e70 2024-01-24T14:16:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff26a7fc700 1 -- 172.21.15.29:0/2543944935 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27000f040 con 0x7ff278134e70 2024-01-24T14:16:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff2804b1700 1 -- 172.21.15.29:0/2543944935 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2780ac6f0 con 0x7ff278134e70 2024-01-24T14:16:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff26a7fc700 1 -- 172.21.15.29:0/2543944935 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270013600 con 0x7ff278134e70 2024-01-24T14:16:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.815+0000 7ff2804b1700 1 -- 172.21.15.29:0/2543944935 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff278136aa0 con 0x7ff278134e70 2024-01-24T14:16:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.816+0000 7ff2804b1700 1 -- 172.21.15.29:0/2543944935 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff278136fc0 con 0x7ff278134e70 2024-01-24T14:16:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.817+0000 7ff26a7fc700 1 -- 172.21.15.29:0/2543944935 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 48) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff2700139f0 con 0x7ff278134e70 2024-01-24T14:16:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.818+0000 7ff26a7fc700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff25c05f270 0x7ff25c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.818+0000 7ff26a7fc700 1 -- 172.21.15.29:0/2543944935 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7ff27007bb30 con 0x7ff278134e70 2024-01-24T14:16:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.818+0000 7ff27e24d700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff25c05f270 0x7ff25c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:16.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.819+0000 7ff27e24d700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff25c05f270 0x7ff25c061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff26c00bb20 tx=0x7ff26c008040).ready entity=mgr.7195 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:16.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:16.822+0000 7ff26a7fc700 1 -- 172.21.15.29:0/2543944935 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff27004c7e0 con 0x7ff278134e70 2024-01-24T14:16:17.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.052+0000 7ff2804b1700 1 -- 172.21.15.29:0/2543944935 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7ff278136fc0 con 0x7ff278134e70 2024-01-24T14:16:17.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.369+0000 7ff26a7fc700 1 -- 172.21.15.29:0/2543944935 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 49) v1 ==== 75565+0+0 (secure 0 0 0) 0x7ff270079340 con 0x7ff278134e70 2024-01-24T14:16:17.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.371+0000 7ff26a7fc700 1 -- 172.21.15.29:0/2543944935 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v49) v1 ==== 92+0+0 (secure 0 0 0) 0x7ff270076770 con 0x7ff278134e70 2024-01-24T14:16:17.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.373+0000 7ff2804b1700 1 -- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff25c05f270 msgr2=0x7ff25c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:17.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.374+0000 7ff2804b1700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff25c05f270 0x7ff25c061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff26c00bb20 tx=0x7ff26c008040).stop 2024-01-24T14:16:17.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.374+0000 7ff2804b1700 1 -- 172.21.15.29:0/2543944935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff278134e70 msgr2=0x7ff2780ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:17.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.374+0000 7ff2804b1700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff278134e70 0x7ff2780ac040 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7ff27000d100 tx=0x7ff2700099b0).stop 2024-01-24T14:16:17.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.374+0000 7ff2804b1700 1 -- 172.21.15.29:0/2543944935 shutdown_connections 2024-01-24T14:16:17.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.374+0000 7ff2804b1700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff25c05f270 0x7ff25c061730 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.374+0000 7ff2804b1700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff278134e70 0x7ff2780ac040 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.374+0000 7ff2804b1700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff278066520 0x7ff27814ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.374+0000 7ff2804b1700 1 --2- 172.21.15.29:0/2543944935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff278068900 0x7ff2780abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.374+0000 7ff2804b1700 1 -- 172.21.15.29:0/2543944935 >> 172.21.15.29:0/2543944935 conn(0x7ff2780c8ba0 msgr2=0x7ff2781344d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:17.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.374+0000 7ff2804b1700 1 -- 172.21.15.29:0/2543944935 shutdown_connections 2024-01-24T14:16:17.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.374+0000 7ff2804b1700 1 -- 172.21.15.29:0/2543944935 wait complete. 2024-01-24T14:16:17.392 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-01-24T14:16:17.511 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:17.511+0000 7f6959d4f680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:16:17.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.529+0000 7f0b56fd9700 1 Processor -- start 2024-01-24T14:16:17.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.529+0000 7f0b56fd9700 1 -- start start 2024-01-24T14:16:17.530 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:17.530+0000 7fdee7390680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:16:17.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.530+0000 7f0b56fd9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0b50064760 0x7f0b50064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:17.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.530+0000 7f0b56fd9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0b500bfd40 0x7f0b500c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:17.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.530+0000 7f0b56fd9700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b50152240 con 0x7f0b500c8490 2024-01-24T14:16:17.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.530+0000 7f0b56fd9700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b501586c0 con 0x7f0b500bfd40 2024-01-24T14:16:17.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.530+0000 7f0b56fd9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b500aa670 con 0x7f0b50064760 2024-01-24T14:16:17.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.530+0000 7f0b55576700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0b500bfd40 0x7f0b500c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:17.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.530+0000 7f0b55576700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0b500bfd40 0x7f0b500c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51194/0 (socket says 172.21.15.29:51194) 2024-01-24T14:16:17.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.530+0000 7f0b55576700 1 -- 172.21.15.29:0/1320579513 learned_addr learned my addr 172.21.15.29:0/1320579513 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:17.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.530+0000 7f0b4ffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0b500c8490 0x7f0b500a9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48410/0 (socket says 172.21.15.29:48410) 2024-01-24T14:16:17.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.530+0000 7f0b54d75700 1 --2- 172.21.15.29:0/1320579513 >> v2:172.21.15.163:3300/0 conn(0x7f0b50064760 0x7f0b50064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:17.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.531+0000 7f0b55576700 1 -- 172.21.15.29:0/1320579513 >> v2:172.21.15.163:3300/0 conn(0x7f0b50064760 msgr2=0x7f0b50064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.531+0000 7f0b55576700 1 --2- 172.21.15.29:0/1320579513 >> v2:172.21.15.163:3300/0 conn(0x7f0b50064760 0x7f0b50064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.531+0000 7f0b55576700 1 -- 172.21.15.29:0/1320579513 >> v1:172.21.15.29:6789/0 conn(0x7f0b500c8490 legacy=0x7f0b500a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.531+0000 7f0b55576700 1 -- 172.21.15.29:0/1320579513 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b50157d90 con 0x7f0b500bfd40 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.531+0000 7f0b4f7fe700 1 -- 172.21.15.29:0/1320579513 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713146310 0 0) 0x7f0b50152240 con 0x7f0b500c8490 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.531+0000 7f0b55576700 1 --2- 172.21.15.29:0/1320579513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b500bfd40 0x7f0b500c01d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f0b50084450 tx=0x7f0b48009840).ready entity=mon.0 client_cookie=80b6578283bdf6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.531+0000 7f0b4f7fe700 1 -- 172.21.15.29:0/1320579513 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4800e070 con 0x7f0b500bfd40 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.531+0000 7f0b4f7fe700 1 -- 172.21.15.29:0/1320579513 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4800bab0 con 0x7f0b500bfd40 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.531+0000 7f0b4f7fe700 1 -- 172.21.15.29:0/1320579513 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4800be30 con 0x7f0b500bfd40 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.532+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1320579513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b500bfd40 msgr2=0x7f0b500c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.532+0000 7f0b56fd9700 1 --2- 172.21.15.29:0/1320579513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b500bfd40 0x7f0b500c01d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f0b50084450 tx=0x7f0b48009840).stop 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.532+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1320579513 shutdown_connections 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.532+0000 7f0b56fd9700 1 --2- 172.21.15.29:0/1320579513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b500bfd40 0x7f0b500c01d0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.532+0000 7f0b56fd9700 1 --2- 172.21.15.29:0/1320579513 >> v2:172.21.15.163:3300/0 conn(0x7f0b50064760 0x7f0b50064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.532+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1320579513 >> 172.21.15.29:0/1320579513 conn(0x7f0b500cb8b0 msgr2=0x7f0b500c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:17.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.532+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1320579513 shutdown_connections 2024-01-24T14:16:17.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.532+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1320579513 wait complete. 2024-01-24T14:16:17.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.533+0000 7f0b56fd9700 1 Processor -- start 2024-01-24T14:16:17.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.533+0000 7f0b56fd9700 1 -- start start 2024-01-24T14:16:17.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.534+0000 7f0b56fd9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b50064760 0x7f0b500add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.534+0000 7f0b56fd9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b500bfd40 0x7f0b5014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.534+0000 7f0b56fd9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b500c8490 0x7f0b500af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.534+0000 7f0b56fd9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b500ae670 con 0x7f0b500c8490 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.534+0000 7f0b56fd9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b500ae950 con 0x7f0b50064760 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.534+0000 7f0b56fd9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b500aec30 con 0x7f0b500bfd40 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.534+0000 7f0b55576700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b500c8490 0x7f0b500af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.534+0000 7f0b54d75700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b50064760 0x7f0b500add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.534+0000 7f0b4ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b500bfd40 0x7f0b5014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.534+0000 7f0b55576700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b500c8490 0x7f0b500af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51198/0 (socket says 172.21.15.29:51198) 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.534+0000 7f0b55576700 1 -- 172.21.15.29:0/1628260726 learned_addr learned my addr 172.21.15.29:0/1628260726 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b54d75700 1 -- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b500bfd40 msgr2=0x7f0b5014f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b54d75700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b500bfd40 0x7f0b5014f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b54d75700 1 -- 172.21.15.29:0/1628260726 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b500c8490 msgr2=0x7f0b500af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:17.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b54d75700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b500c8490 0x7f0b500af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b54d75700 1 -- 172.21.15.29:0/1628260726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b48009ae0 con 0x7f0b50064760 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b4ffff700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b500bfd40 0x7f0b5014f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b55576700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b500c8490 0x7f0b500af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b54d75700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b50064760 0x7f0b500add90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f0b4400e900 tx=0x7f0b44009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b4cff9700 1 -- 172.21.15.29:0/1628260726 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44009450 con 0x7f0b50064760 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b4cff9700 1 -- 172.21.15.29:0/1628260726 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44012600 con 0x7f0b50064760 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1628260726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b500aef10 con 0x7f0b50064760 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b4cff9700 1 -- 172.21.15.29:0/1628260726 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44012980 con 0x7f0b50064760 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.535+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1628260726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b500ae280 con 0x7f0b50064760 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.536+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1628260726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5006a2d0 con 0x7f0b50064760 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.536+0000 7f0b4cff9700 1 -- 172.21.15.29:0/1628260726 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 49) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f0b44012d30 con 0x7f0b50064760 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.537+0000 7f0b4cff9700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0b3405f290 0x7f0b34061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.537+0000 7f0b4ffff700 1 -- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0b3405f290 msgr2=0x7f0b34061750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.537+0000 7f0b4cff9700 1 -- 172.21.15.29:0/1628260726 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f0b4407abe0 con 0x7f0b50064760 2024-01-24T14:16:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.537+0000 7f0b4ffff700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0b3405f290 0x7f0b34061750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:16:17.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.541+0000 7f0b4cff9700 1 -- 172.21.15.29:0/1628260726 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0b4404b870 con 0x7f0b50064760 2024-01-24T14:16:17.576 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:17.576+0000 7f6959d4f680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:16:17.605 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:17.605+0000 7fdee7390680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:16:17.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.727+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1628260726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f0b500475e0 con 0x7f0b50064760 2024-01-24T14:16:17.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.732+0000 7f0b4cff9700 1 -- 172.21.15.29:0/1628260726 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0b4404ee90 con 0x7f0b50064760 2024-01-24T14:16:17.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.734+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0b3405f290 msgr2=0x7f0b34061750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:16:17.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.734+0000 7f0b56fd9700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0b3405f290 0x7f0b34061750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:16:17.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.734+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b50064760 msgr2=0x7f0b500add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:17.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.734+0000 7f0b56fd9700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b50064760 0x7f0b500add90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f0b4400e900 tx=0x7f0b44009060).stop 2024-01-24T14:16:17.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.734+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1628260726 shutdown_connections 2024-01-24T14:16:17.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.734+0000 7f0b56fd9700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0b3405f290 0x7f0b34061750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:16:17.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.734+0000 7f0b56fd9700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b500c8490 0x7f0b500af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.734+0000 7f0b56fd9700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b50064760 0x7f0b500add90 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.734+0000 7f0b56fd9700 1 --2- 172.21.15.29:0/1628260726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b500bfd40 0x7f0b5014f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.734+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1628260726 >> 172.21.15.29:0/1628260726 conn(0x7f0b500cb8b0 msgr2=0x7f0b500c5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:17.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.734+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1628260726 shutdown_connections 2024-01-24T14:16:17.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.734+0000 7f0b56fd9700 1 -- 172.21.15.29:0/1628260726 wait complete. 2024-01-24T14:16:17.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:17.749 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-01-24T14:16:17.772 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2024-01-24T14:16:17.851 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:17.851+0000 7f6959d4f680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:16:17.922 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:17.921+0000 7fdee7390680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:16:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.948+0000 7f2d73965700 1 Processor -- start 2024-01-24T14:16:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.948+0000 7f2d73965700 1 -- start start 2024-01-24T14:16:17.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.949+0000 7f2d73965700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2d6c0c7f70 0x7f2d6c0c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:17.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.949+0000 7f2d73965700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c154f90 con 0x7f2d6c151e50 2024-01-24T14:16:17.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.949+0000 7f2d73965700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c0a9f10 con 0x7f2d6c064e90 2024-01-24T14:16:17.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.949+0000 7f2d73965700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c0aa1f0 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.949+0000 7f2d71f02700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2d6c0c7f70 0x7f2d6c0c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:17.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.949+0000 7f2d71701700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2d6c151e50 0x7f2d6c064780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48424/0 (socket says 172.21.15.29:48424) 2024-01-24T14:16:17.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.949+0000 7f2d71701700 1 -- 172.21.15.29:0/3577078261 learned_addr learned my addr 172.21.15.29:0/3577078261 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:17.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.950+0000 7f2d71f02700 1 -- 172.21.15.29:0/3577078261 >> v1:172.21.15.163:6789/0 conn(0x7f2d6c064e90 legacy=0x7f2d6c0bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:17.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.950+0000 7f2d71f02700 1 -- 172.21.15.29:0/3577078261 >> v1:172.21.15.29:6789/0 conn(0x7f2d6c151e50 legacy=0x7f2d6c064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:17.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.950+0000 7f2d71f02700 1 -- 172.21.15.29:0/3577078261 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0aa4d0 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.950+0000 7f2d71f02700 1 --2- 172.21.15.29:0/3577078261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c0c53c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f2d64009e60 tx=0x7f2d64009780).ready entity=mon.0 client_cookie=82583d03d4bcbec1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:17.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.950+0000 7f2d63fff700 1 -- 172.21.15.29:0/3577078261 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400e070 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.950+0000 7f2d63fff700 1 -- 172.21.15.29:0/3577078261 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6400bc30 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.950+0000 7f2d63fff700 1 -- 172.21.15.29:0/3577078261 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64012410 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.951+0000 7f2d73965700 1 -- 172.21.15.29:0/3577078261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d6c0c7f70 msgr2=0x7f2d6c0c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.951+0000 7f2d73965700 1 --2- 172.21.15.29:0/3577078261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c0c53c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f2d64009e60 tx=0x7f2d64009780).stop 2024-01-24T14:16:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.951+0000 7f2d73965700 1 -- 172.21.15.29:0/3577078261 shutdown_connections 2024-01-24T14:16:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.951+0000 7f2d73965700 1 --2- 172.21.15.29:0/3577078261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c0c53c0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.951+0000 7f2d73965700 1 -- 172.21.15.29:0/3577078261 >> 172.21.15.29:0/3577078261 conn(0x7f2d6c0cb880 msgr2=0x7f2d6c0c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.951+0000 7f2d73965700 1 -- 172.21.15.29:0/3577078261 shutdown_connections 2024-01-24T14:16:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.951+0000 7f2d73965700 1 -- 172.21.15.29:0/3577078261 wait complete. 2024-01-24T14:16:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.952+0000 7f2d73965700 1 Processor -- start 2024-01-24T14:16:17.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.952+0000 7f2d73965700 1 -- start start 2024-01-24T14:16:17.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.953+0000 7f2d73965700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c064e90 0x7f2d6c159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:17.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.953+0000 7f2d73965700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c1644b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:17.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.953+0000 7f2d73965700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d6c151e50 0x7f2d6c14d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:17.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.953+0000 7f2d73965700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c1654c0 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.953+0000 7f2d73965700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0adfa0 con 0x7f2d6c151e50 2024-01-24T14:16:17.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.953+0000 7f2d73965700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c1649a0 con 0x7f2d6c064e90 2024-01-24T14:16:17.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.953+0000 7f2d70f00700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c1644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:17.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.953+0000 7f2d70f00700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c1644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51226/0 (socket says 172.21.15.29:51226) 2024-01-24T14:16:17.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.953+0000 7f2d70f00700 1 -- 172.21.15.29:0/1878858319 learned_addr learned my addr 172.21.15.29:0/1878858319 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:17.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.953+0000 7f2d71701700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c064e90 0x7f2d6c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:17.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.953+0000 7f2d71f02700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d6c151e50 0x7f2d6c14d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:17.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.954+0000 7f2d70f00700 1 -- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c064e90 msgr2=0x7f2d6c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:17.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.954+0000 7f2d70f00700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c064e90 0x7f2d6c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.954+0000 7f2d70f00700 1 -- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d6c151e50 msgr2=0x7f2d6c14d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:17.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.954+0000 7f2d70f00700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d6c151e50 0x7f2d6c14d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:17.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.954+0000 7f2d70f00700 1 -- 172.21.15.29:0/1878858319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64009a20 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.954+0000 7f2d70f00700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c1644b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f2d5c00c4f0 tx=0x7f2d5c007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:17.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.954+0000 7f2d617fa700 1 -- 172.21.15.29:0/1878858319 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c00ce00 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.954+0000 7f2d617fa700 1 -- 172.21.15.29:0/1878858319 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c007020 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.954+0000 7f2d73965700 1 -- 172.21.15.29:0/1878858319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c14d9b0 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.954+0000 7f2d617fa700 1 -- 172.21.15.29:0/1878858319 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c0073c0 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.954+0000 7f2d73965700 1 -- 172.21.15.29:0/1878858319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c164f00 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.955+0000 7f2d73965700 1 -- 172.21.15.29:0/1878858319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c0aa710 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.956+0000 7f2d617fa700 1 -- 172.21.15.29:0/1878858319 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 49) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f2d5c0068b0 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.957+0000 7f2d617fa700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 0x7f2d50061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:17.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.957+0000 7f2d71701700 1 -- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 msgr2=0x7f2d50061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:16:17.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.957+0000 7f2d617fa700 1 -- 172.21.15.29:0/1878858319 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f2d5c07a160 con 0x7f2d6c0c7f70 2024-01-24T14:16:17.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.957+0000 7f2d71701700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 0x7f2d50061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:16:17.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:17.964+0000 7f2d617fa700 1 -- 172.21.15.29:0/1878858319 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2d5c04aeb0 con 0x7f2d6c0c7f70 2024-01-24T14:16:18.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:18.158+0000 7f2d71701700 1 -- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 msgr2=0x7f2d50061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:16:18.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:18.158+0000 7f2d71701700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 0x7f2d50061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:16:18.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:18.172+0000 7f2d73965700 1 -- 172.21.15.29:0/1878858319 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d6c0ae150 con 0x7f2d5005f2c0 2024-01-24T14:16:18.474 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:18.474+0000 7f6959d4f680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:16:18.546 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:18.546+0000 7f6959d4f680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:16:18.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:18.559+0000 7f2d71701700 1 -- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 msgr2=0x7f2d50061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:16:18.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:18.559+0000 7f2d71701700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 0x7f2d50061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-01-24T14:16:18.647 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:18.646+0000 7fdee7390680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:16:18.722 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:18.721+0000 7fdee7390680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:16:18.735 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:18.735+0000 7f6959d4f680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:16:18.935 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:18.935+0000 7fdee7390680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:16:19.155 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:19.155+0000 7f6959d4f680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:16:19.259 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:19.259+0000 7f6959d4f680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:16:19.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:19.360+0000 7f2d71701700 1 -- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 msgr2=0x7f2d50061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:16:19.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:19.360+0000 7f2d71701700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 0x7f2d50061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-01-24T14:16:19.412 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:19.411+0000 7fdee7390680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:16:19.412 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:19.413+0000 7f6959d4f680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:16:19.486 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:19.486+0000 7f6959d4f680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:16:19.529 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:19.529+0000 7fdee7390680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:16:19.554 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:19.553+0000 7f6959d4f680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:16:19.705 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:19.704+0000 7fdee7390680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:16:19.790 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:19.789+0000 7fdee7390680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:16:19.811 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:19.811+0000 7f6959d4f680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:16:19.867 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:19.866+0000 7fdee7390680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:16:19.994 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:19.994+0000 7f6959d4f680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:16:20.160 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:20.159+0000 7fdee7390680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:16:20.373 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:20.372+0000 7fdee7390680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:16:20.649 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:20.649+0000 7f6959d4f680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:16:20.717 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:20.717+0000 7f6959d4f680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:16:20.792 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:20.792+0000 7f6959d4f680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:16:20.941 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:20.941+0000 7f6959d4f680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:16:20.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:20.962+0000 7f2d71701700 1 -- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 msgr2=0x7f2d50061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:16:20.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:20.962+0000 7f2d71701700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 0x7f2d50061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-01-24T14:16:21.011 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.011+0000 7f6959d4f680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:16:21.126 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:21.126+0000 7fdee7390680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:16:21.127 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.127+0000 7f6959d4f680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:16:21.204 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:21.204+0000 7fdee7390680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:16:21.235 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.235+0000 7f6959d4f680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:16:21.290 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:21.289+0000 7fdee7390680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:16:21.458 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:21.457+0000 7fdee7390680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:16:21.525 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.524+0000 7f6959d4f680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:16:21.538 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:21.537+0000 7fdee7390680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:16:21.596 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.596+0000 7f6959d4f680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:16:21.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:21.615+0000 7f2d617fa700 1 -- 172.21.15.29:0/1878858319 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 50) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f2d5c0779b0 con 0x7f2d6c0c7f70 2024-01-24T14:16:21.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:21.616+0000 7f2d617fa700 1 -- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 msgr2=0x7f2d50061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:16:21.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:21.616+0000 7f2d617fa700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 0x7f2d50061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:16:21.674 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:21.674+0000 7fdee7390680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:16:21.682 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.683+0000 7f68ccac3700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:16:21.683 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.683+0000 7f68ccac3700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:16:21.683 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.683+0000 7f68ccac3700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:16:21.683 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.683+0000 7f68ccac3700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:16:21.683 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.683+0000 7f68ccac3700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:16:21.684 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.684+0000 7f68d0acb700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:16:21.684 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.684+0000 7f68d0acb700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:16:21.684 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.684+0000 7f68d0acb700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:16:21.684 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.684+0000 7f68d0acb700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:16:21.684 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:16:21.684+0000 7f68d0acb700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:16:21.798 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:21.797+0000 7fdee7390680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:16:22.133 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:22.133+0000 7fdee7390680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:16:22.214 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:16:22.213+0000 7fdee7390680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:16:22.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.618+0000 7f2d617fa700 1 -- 172.21.15.29:0/1878858319 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mgrmap(e 51) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f2d5c04b280 con 0x7f2d6c0c7f70 2024-01-24T14:16:22.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.619+0000 7f2d617fa700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 0x7f2d50061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:22.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.619+0000 7f2d617fa700 1 -- 172.21.15.29:0/1878858319 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d6c0ae150 con 0x7f2d5005f2c0 2024-01-24T14:16:22.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.619+0000 7f2d71701700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 0x7f2d50061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:22.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.620+0000 7f2d71701700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 0x7f2d50061700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f2d68007fd0 tx=0x7f2d68007d90).ready entity=mgr.6791 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:22.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.635+0000 7f2d617fa700 1 -- 172.21.15.29:0/1878858319 <== mgr.6791 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2d6c0ae150 con 0x7f2d5005f2c0 2024-01-24T14:16:22.635 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path snap_test_dir1 2024-01-24T14:16:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.637+0000 7f2d73965700 1 -- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 msgr2=0x7f2d50061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.637+0000 7f2d73965700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 0x7f2d50061700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f2d68007fd0 tx=0x7f2d68007d90).stop 2024-01-24T14:16:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.637+0000 7f2d73965700 1 -- 172.21.15.29:0/1878858319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d6c0c7f70 msgr2=0x7f2d6c1644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.637+0000 7f2d73965700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c1644b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f2d5c00c4f0 tx=0x7f2d5c007dd0).stop 2024-01-24T14:16:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.637+0000 7f2d73965700 1 -- 172.21.15.29:0/1878858319 shutdown_connections 2024-01-24T14:16:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.637+0000 7f2d73965700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d6c151e50 0x7f2d6c14d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.637+0000 7f2d73965700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c064e90 0x7f2d6c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.637+0000 7f2d73965700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2d5005f2c0 0x7f2d50061700 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.637+0000 7f2d73965700 1 --2- 172.21.15.29:0/1878858319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c1644b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.637+0000 7f2d73965700 1 -- 172.21.15.29:0/1878858319 >> 172.21.15.29:0/1878858319 conn(0x7f2d6c0cb880 msgr2=0x7f2d6c0ab2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.637+0000 7f2d73965700 1 -- 172.21.15.29:0/1878858319 shutdown_connections 2024-01-24T14:16:22.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.637+0000 7f2d73965700 1 -- 172.21.15.29:0/1878858319 wait complete. 2024-01-24T14:16:22.652 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-01-24T14:16:22.653 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2024-01-24-14_17_00 in ~38s... 2024-01-24T14:16:22.653 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2024-01-24T14:16:22.653 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2024-01-24T14:16:22.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.792+0000 7fa481630700 1 Processor -- start 2024-01-24T14:16:22.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.792+0000 7fa481630700 1 -- start start 2024-01-24T14:16:22.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa481630700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa47c13a540 0x7fa47c0638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:22.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa481630700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c0c31b0 con 0x7fa47c066b80 2024-01-24T14:16:22.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa481630700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c0c60a0 con 0x7fa47c13a920 2024-01-24T14:16:22.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa481630700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c067360 con 0x7fa47c13a540 2024-01-24T14:16:22.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa47b7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa47c066b80 0x7fa47c0c3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:48432/0 (socket says 172.21.15.29:48432) 2024-01-24T14:16:22.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa47b7fe700 1 -- 172.21.15.29:0/1329193840 learned_addr learned my addr 172.21.15.29:0/1329193840 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:16:22.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa47affd700 1 --2- 172.21.15.29:0/1329193840 >> v2:172.21.15.163:3300/0 conn(0x7fa47c13a540 0x7fa47c0638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:22.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa479ffb700 1 -- 172.21.15.29:0/1329193840 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869711517 0 0) 0x7fa47c0c31b0 con 0x7fa47c066b80 2024-01-24T14:16:22.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa479ffb700 1 -- 172.21.15.29:0/1329193840 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4640033f0 con 0x7fa47c066b80 2024-01-24T14:16:22.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa479ffb700 1 -- 172.21.15.29:0/1329193840 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399652054 0 0) 0x7fa47c0c60a0 con 0x7fa47c13a920 2024-01-24T14:16:22.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa479ffb700 1 -- 172.21.15.29:0/1329193840 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa47c0c31b0 con 0x7fa47c13a920 2024-01-24T14:16:22.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa479ffb700 1 -- 172.21.15.29:0/1329193840 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1749787638 0 0) 0x7fa4640033f0 con 0x7fa47c066b80 2024-01-24T14:16:22.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa47affd700 1 -- 172.21.15.29:0/1329193840 >> v1:172.21.15.163:6789/0 conn(0x7fa47c13a920 legacy=0x7fa47c066470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:22.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa47affd700 1 -- 172.21.15.29:0/1329193840 >> v1:172.21.15.29:6789/0 conn(0x7fa47c066b80 legacy=0x7fa47c0c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:22.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.793+0000 7fa47affd700 1 -- 172.21.15.29:0/1329193840 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0c6380 con 0x7fa47c13a540 2024-01-24T14:16:22.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.794+0000 7fa47affd700 1 --2- 172.21.15.29:0/1329193840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa47c13a540 0x7fa47c0638c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7fa46c009760 tx=0x7fa46c009060).ready entity=mon.1 client_cookie=cf3f771de2502dbf server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:22.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.794+0000 7fa479ffb700 1 -- 172.21.15.29:0/1329193840 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740017f0 con 0x7fa47c13a540 2024-01-24T14:16:22.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.794+0000 7fa479ffb700 1 -- 172.21.15.29:0/1329193840 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c0090d0 con 0x7fa47c13a540 2024-01-24T14:16:22.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.794+0000 7fa479ffb700 1 -- 172.21.15.29:0/1329193840 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c011610 con 0x7fa47c13a540 2024-01-24T14:16:22.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.794+0000 7fa481630700 1 -- 172.21.15.29:0/1329193840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa47c13a540 msgr2=0x7fa47c0638c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:22.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.794+0000 7fa481630700 1 --2- 172.21.15.29:0/1329193840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa47c13a540 0x7fa47c0638c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7fa46c009760 tx=0x7fa46c009060).stop 2024-01-24T14:16:22.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.794+0000 7fa481630700 1 -- 172.21.15.29:0/1329193840 shutdown_connections 2024-01-24T14:16:22.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.794+0000 7fa481630700 1 --2- 172.21.15.29:0/1329193840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa47c13a540 0x7fa47c0638c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:22.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.794+0000 7fa481630700 1 -- 172.21.15.29:0/1329193840 >> 172.21.15.29:0/1329193840 conn(0x7fa47c05e1f0 msgr2=0x7fa47c05e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:22.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.794+0000 7fa481630700 1 -- 172.21.15.29:0/1329193840 shutdown_connections 2024-01-24T14:16:22.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.795+0000 7fa481630700 1 -- 172.21.15.29:0/1329193840 wait complete. 2024-01-24T14:16:22.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.796+0000 7fa481630700 1 Processor -- start 2024-01-24T14:16:22.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.796+0000 7fa481630700 1 -- start start 2024-01-24T14:16:22.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa481630700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa47c066b80 0x7fa47c1387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:22.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa481630700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa47c13a540 0x7fa47c154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:22.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa481630700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa47c13a920 0x7fa47c1684a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa481630700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c13d120 con 0x7fa47c066b80 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa481630700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c168990 con 0x7fa47c13a540 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa481630700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0ac040 con 0x7fa47c13a920 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa47affd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa47c066b80 0x7fa47c1387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa47affd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa47c066b80 0x7fa47c1387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51230/0 (socket says 172.21.15.29:51230) 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa47affd700 1 -- 172.21.15.29:0/475974295 learned_addr learned my addr 172.21.15.29:0/475974295 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa47a7fc700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa47c13a540 0x7fa47c154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa47b7fe700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa47c13a920 0x7fa47c1684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa47affd700 1 -- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa47c13a920 msgr2=0x7fa47c1684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa47affd700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa47c13a920 0x7fa47c1684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa47affd700 1 -- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa47c13a540 msgr2=0x7fa47c154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa47affd700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa47c13a540 0x7fa47c154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:22.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.797+0000 7fa47affd700 1 -- 172.21.15.29:0/475974295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c0092a0 con 0x7fa47c066b80 2024-01-24T14:16:22.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.798+0000 7fa47b7fe700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa47c13a920 0x7fa47c1684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:16:22.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.798+0000 7fa47affd700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa47c066b80 0x7fa47c1387d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7fa47c069e30 tx=0x7fa46c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:22.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.798+0000 7fa462ffd700 1 -- 172.21.15.29:0/475974295 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0107c0 con 0x7fa47c066b80 2024-01-24T14:16:22.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.798+0000 7fa462ffd700 1 -- 172.21.15.29:0/475974295 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c010920 con 0x7fa47c066b80 2024-01-24T14:16:22.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.798+0000 7fa481630700 1 -- 172.21.15.29:0/475974295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c0ac320 con 0x7fa47c066b80 2024-01-24T14:16:22.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.798+0000 7fa462ffd700 1 -- 172.21.15.29:0/475974295 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c010ca0 con 0x7fa47c066b80 2024-01-24T14:16:22.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.798+0000 7fa481630700 1 -- 172.21.15.29:0/475974295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c0ac8e0 con 0x7fa47c066b80 2024-01-24T14:16:22.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.799+0000 7fa481630700 1 -- 172.21.15.29:0/475974295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47c0475e0 con 0x7fa47c066b80 2024-01-24T14:16:22.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.799+0000 7fa462ffd700 1 -- 172.21.15.29:0/475974295 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 51) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fa46c01d640 con 0x7fa47c066b80 2024-01-24T14:16:22.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.800+0000 7fa462ffd700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa46805f080 0x7fa468061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:16:22.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.800+0000 7fa462ffd700 1 -- 172.21.15.29:0/475974295 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fa46c07ba10 con 0x7fa47c066b80 2024-01-24T14:16:22.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.800+0000 7fa47a7fc700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa46805f080 0x7fa468061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:16:22.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.800+0000 7fa47a7fc700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa46805f080 0x7fa468061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa47c13c8e0 tx=0x7fa470007bd0).ready entity=mgr.6791 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:16:22.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:22.804+0000 7fa462ffd700 1 -- 172.21.15.29:0/475974295 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa46c04c9b0 con 0x7fa47c066b80 2024-01-24T14:16:23.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.012+0000 7fa481630700 1 -- 172.21.15.29:0/475974295 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa47c0c3400 con 0x7fa46805f080 2024-01-24T14:16:23.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.015+0000 7fa462ffd700 1 -- 172.21.15.29:0/475974295 <== mgr.6791 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fa47c0c3400 con 0x7fa46805f080 2024-01-24T14:16:23.016 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:16:23.016 INFO:teuthology.orchestra.run.smithi029.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2024-01-24T14:16:23.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.017+0000 7fa481630700 1 -- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa46805f080 msgr2=0x7fa468061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:23.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.017+0000 7fa481630700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa46805f080 0x7fa468061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa47c13c8e0 tx=0x7fa470007bd0).stop 2024-01-24T14:16:23.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.017+0000 7fa481630700 1 -- 172.21.15.29:0/475974295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa47c066b80 msgr2=0x7fa47c1387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:16:23.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.017+0000 7fa481630700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa47c066b80 0x7fa47c1387d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7fa47c069e30 tx=0x7fa46c016000).stop 2024-01-24T14:16:23.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.017+0000 7fa481630700 1 -- 172.21.15.29:0/475974295 shutdown_connections 2024-01-24T14:16:23.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.017+0000 7fa481630700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa47c13a920 0x7fa47c1684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:23.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.017+0000 7fa481630700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa46805f080 0x7fa468061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:23.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.017+0000 7fa481630700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa47c066b80 0x7fa47c1387d0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:23.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.017+0000 7fa481630700 1 --2- 172.21.15.29:0/475974295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa47c13a540 0x7fa47c154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:16:23.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.017+0000 7fa481630700 1 -- 172.21.15.29:0/475974295 >> 172.21.15.29:0/475974295 conn(0x7fa47c05e1f0 msgr2=0x7fa47c1393f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:16:23.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.017+0000 7fa481630700 1 -- 172.21.15.29:0/475974295 shutdown_connections 2024-01-24T14:16:23.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:16:23.017+0000 7fa481630700 1 -- 172.21.15.29:0/475974295 wait complete. 2024-01-24T14:16:23.031 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2024-01-24T14:16:23.031 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2024-01-24T14:16:23.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:23.032 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:24.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:24.053 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:25.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:25.077 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:26.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:26.100 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:27.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:27.124 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:28.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:28.148 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:29.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:29.173 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:29.175 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:16:29.177 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:16:30.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:30.196 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:31.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:31.221 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:32.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:32.246 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:33.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:33.270 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:34.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:34.294 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:35.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:35.319 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:36.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:36.343 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:37.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:37.366 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:38.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:38.391 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:39.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:39.415 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:40.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:40.439 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:41.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:41.464 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:42.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:42.489 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:43.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:43.513 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:44.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:44.537 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:45.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:45.561 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:46.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:46.586 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:47.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:47.609 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:48.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:48.634 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:49.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:49.659 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:50.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:50.683 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:51.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:51.707 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:52.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:52.732 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:53.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:53.756 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:54.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:54.781 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:55.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:55.805 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:56.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:56.830 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:57.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:57.854 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:58.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:58.878 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:16:59.300 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:16:59.302 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:16:59.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:16:59.902 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:17:00.640 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:00.640+0000 7f68c02aa700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:00.640 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:00.640+0000 7f68c02aa700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:00.640 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:00.640+0000 7f68c02aa700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:00.640 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:00.640+0000 7f68c02aa700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:00.640 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:00.640+0000 7f68c02aa700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:00.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:17:00.927 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:17:00.949 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_17_00_UTC 2024-01-24T14:17:00.950 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2024-01-24-14_17_00_UTC'] 2024-01-24T14:17:00.950 DEBUG:tasks.cephfs.test_snap_schedules:exec=1706105782.652968, now = 1706105820.95038, timo = 38 2024-01-24T14:17:01.951 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2024-01-24T14:17:02.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.092+0000 7f7d20915700 1 Processor -- start 2024-01-24T14:17:02.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.092+0000 7f7d20915700 1 -- start start 2024-01-24T14:17:02.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.093+0000 7f7d20915700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7d1c066450 0x7f7d1c066830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:02.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.093+0000 7f7d20915700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1c1530b0 con 0x7f7d1c159da0 2024-01-24T14:17:02.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.093+0000 7f7d20915700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1c153300 con 0x7f7d1c152540 2024-01-24T14:17:02.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.093+0000 7f7d20915700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1c067b10 con 0x7f7d1c066450 2024-01-24T14:17:02.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.094+0000 7f7d1ad9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7d1c159da0 0x7f7d1c15c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:59798/0 (socket says 172.21.15.29:59798) 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.094+0000 7f7d1ad9d700 1 -- 172.21.15.29:0/1785783109 learned_addr learned my addr 172.21.15.29:0/1785783109 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.094+0000 7f7d1a59c700 1 --2- 172.21.15.29:0/1785783109 >> v2:172.21.15.163:3300/0 conn(0x7f7d1c066450 0x7f7d1c066830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.094+0000 7f7d1959a700 1 -- 172.21.15.29:0/1785783109 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403392676 0 0) 0x7f7d1c1530b0 con 0x7f7d1c159da0 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.094+0000 7f7d1959a700 1 -- 172.21.15.29:0/1785783109 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d040033f0 con 0x7f7d1c159da0 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.094+0000 7f7d1959a700 1 -- 172.21.15.29:0/1785783109 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969396824 0 0) 0x7f7d1c153300 con 0x7f7d1c152540 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.094+0000 7f7d1959a700 1 -- 172.21.15.29:0/1785783109 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d1c1530b0 con 0x7f7d1c152540 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.094+0000 7f7d1959a700 1 -- 172.21.15.29:0/1785783109 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1358369587 0 0) 0x7f7d040033f0 con 0x7f7d1c159da0 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.094+0000 7f7d1a59c700 1 -- 172.21.15.29:0/1785783109 >> v1:172.21.15.163:6789/0 conn(0x7f7d1c152540 legacy=0x7f7d1c1529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.094+0000 7f7d1a59c700 1 -- 172.21.15.29:0/1785783109 >> v1:172.21.15.29:6789/0 conn(0x7f7d1c159da0 legacy=0x7f7d1c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.094+0000 7f7d1a59c700 1 -- 172.21.15.29:0/1785783109 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c067df0 con 0x7f7d1c066450 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.094+0000 7f7d1a59c700 1 --2- 172.21.15.29:0/1785783109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d1c066450 0x7f7d1c066830 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f7d10009760 tx=0x7f7d10009060).ready entity=mon.1 client_cookie=d3864345593f6c0d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.095+0000 7f7d1959a700 1 -- 172.21.15.29:0/1785783109 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d140017f0 con 0x7f7d1c066450 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.095+0000 7f7d1959a700 1 -- 172.21.15.29:0/1785783109 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d10011400 con 0x7f7d1c066450 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.095+0000 7f7d1959a700 1 -- 172.21.15.29:0/1785783109 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10011780 con 0x7f7d1c066450 2024-01-24T14:17:02.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.095+0000 7f7d20915700 1 -- 172.21.15.29:0/1785783109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d1c066450 msgr2=0x7f7d1c066830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.095+0000 7f7d20915700 1 --2- 172.21.15.29:0/1785783109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d1c066450 0x7f7d1c066830 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f7d10009760 tx=0x7f7d10009060).stop 2024-01-24T14:17:02.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.095+0000 7f7d20915700 1 -- 172.21.15.29:0/1785783109 shutdown_connections 2024-01-24T14:17:02.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.095+0000 7f7d20915700 1 --2- 172.21.15.29:0/1785783109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d1c066450 0x7f7d1c066830 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.095+0000 7f7d20915700 1 -- 172.21.15.29:0/1785783109 >> 172.21.15.29:0/1785783109 conn(0x7f7d1c0c8bd0 msgr2=0x7f7d1c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:02.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.095+0000 7f7d20915700 1 -- 172.21.15.29:0/1785783109 shutdown_connections 2024-01-24T14:17:02.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.095+0000 7f7d20915700 1 -- 172.21.15.29:0/1785783109 wait complete. 2024-01-24T14:17:02.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.096+0000 7f7d20915700 1 Processor -- start 2024-01-24T14:17:02.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.096+0000 7f7d20915700 1 -- start start 2024-01-24T14:17:02.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.097+0000 7f7d20915700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c066450 0x7f7d1c0c2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:02.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.097+0000 7f7d20915700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d1c152540 0x7f7d1c153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:02.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.097+0000 7f7d20915700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d1c159da0 0x7f7d1c0c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:02.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.097+0000 7f7d20915700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c14bfd0 con 0x7f7d1c066450 2024-01-24T14:17:02.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.097+0000 7f7d20915700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0c0640 con 0x7f7d1c152540 2024-01-24T14:17:02.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d20915700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c153a60 con 0x7f7d1c159da0 2024-01-24T14:17:02.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d1a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c066450 0x7f7d1c0c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:02.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d1a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c066450 0x7f7d1c0c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43780/0 (socket says 172.21.15.29:43780) 2024-01-24T14:17:02.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d1a59c700 1 -- 172.21.15.29:0/1960139782 learned_addr learned my addr 172.21.15.29:0/1960139782 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d19d9b700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d1c152540 0x7f7d1c153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d1ad9d700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d1c159da0 0x7f7d1c0c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d1a59c700 1 -- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d1c159da0 msgr2=0x7f7d1c0c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d1a59c700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d1c159da0 0x7f7d1c0c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d1a59c700 1 -- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d1c152540 msgr2=0x7f7d1c153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d1a59c700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d1c152540 0x7f7d1c153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d1a59c700 1 -- 172.21.15.29:0/1960139782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d100092a0 con 0x7f7d1c066450 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d1ad9d700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d1c159da0 0x7f7d1c0c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d19d9b700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d1c152540 0x7f7d1c153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d1a59c700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c066450 0x7f7d1c0c2760 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f7d100118a0 tx=0x7f7d10016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d0a7fc700 1 -- 172.21.15.29:0/1960139782 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d100108e0 con 0x7f7d1c066450 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d0a7fc700 1 -- 172.21.15.29:0/1960139782 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d10010a40 con 0x7f7d1c066450 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d20915700 1 -- 172.21.15.29:0/1960139782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c153d30 con 0x7f7d1c066450 2024-01-24T14:17:02.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d0a7fc700 1 -- 172.21.15.29:0/1960139782 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10010de0 con 0x7f7d1c066450 2024-01-24T14:17:02.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.098+0000 7f7d20915700 1 -- 172.21.15.29:0/1960139782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c1542f0 con 0x7f7d1c066450 2024-01-24T14:17:02.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.099+0000 7f7d20915700 1 -- 172.21.15.29:0/1960139782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1c0c0410 con 0x7f7d1c066450 2024-01-24T14:17:02.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.100+0000 7f7d0a7fc700 1 -- 172.21.15.29:0/1960139782 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 53) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f7d1001d640 con 0x7f7d1c066450 2024-01-24T14:17:02.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.100+0000 7f7d0a7fc700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7cfc05f3e0 0x7f7cfc0618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:02.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.100+0000 7f7d0a7fc700 1 -- 172.21.15.29:0/1960139782 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f7d1007c010 con 0x7f7d1c066450 2024-01-24T14:17:02.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.100+0000 7f7d19d9b700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7cfc05f3e0 0x7f7cfc0618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:02.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.101+0000 7f7d19d9b700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7cfc05f3e0 0x7f7cfc0618a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7d1c159ca0 tx=0x7f7d0c009040).ready entity=mgr.6791 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:02.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.105+0000 7f7d0a7fc700 1 -- 172.21.15.29:0/1960139782 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7d1004cae0 con 0x7f7d1c066450 2024-01-24T14:17:02.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.313+0000 7f7d20915700 1 -- 172.21.15.29:0/1960139782 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d1c0475e0 con 0x7f7cfc05f3e0 2024-01-24T14:17:02.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.318+0000 7f7d0a7fc700 1 -- 172.21.15.29:0/1960139782 <== mgr.6791 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f7d1c0475e0 con 0x7f7cfc05f3e0 2024-01-24T14:17:02.319 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path snap_test_dir1 2024-01-24T14:17:02.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.320+0000 7f7d20915700 1 -- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7cfc05f3e0 msgr2=0x7f7cfc0618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.320+0000 7f7d20915700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7cfc05f3e0 0x7f7cfc0618a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7d1c159ca0 tx=0x7f7d0c009040).stop 2024-01-24T14:17:02.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.320+0000 7f7d20915700 1 -- 172.21.15.29:0/1960139782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c066450 msgr2=0x7f7d1c0c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.320+0000 7f7d20915700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c066450 0x7f7d1c0c2760 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f7d100118a0 tx=0x7f7d10016000).stop 2024-01-24T14:17:02.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.320+0000 7f7d20915700 1 -- 172.21.15.29:0/1960139782 shutdown_connections 2024-01-24T14:17:02.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.320+0000 7f7d20915700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d1c159da0 0x7f7d1c0c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.320+0000 7f7d20915700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7cfc05f3e0 0x7f7cfc0618a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.320+0000 7f7d20915700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d1c066450 0x7f7d1c0c2760 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.320+0000 7f7d20915700 1 --2- 172.21.15.29:0/1960139782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d1c152540 0x7f7d1c153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.320+0000 7f7d20915700 1 -- 172.21.15.29:0/1960139782 >> 172.21.15.29:0/1960139782 conn(0x7f7d1c0c8bd0 msgr2=0x7f7d1c134430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:02.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.320+0000 7f7d20915700 1 -- 172.21.15.29:0/1960139782 shutdown_connections 2024-01-24T14:17:02.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.320+0000 7f7d20915700 1 -- 172.21.15.29:0/1960139782 wait complete. 2024-01-24T14:17:02.334 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-01-24T14:17:02.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:17:02.334 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T14:17:02.354 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_17_00_UTC 2024-01-24T14:17:02.354 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_17_00_UTC 2024-01-24T14:17:02.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:17:02.354 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_17_00_UTC) 2024-01-24T14:17:02.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:17:02.420 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-01-24T14:17:02.486 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-01-24T14:17:02.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.666+0000 7fdc4fff1700 1 Processor -- start 2024-01-24T14:17:02.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.666+0000 7fdc4fff1700 1 -- start start 2024-01-24T14:17:02.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.667+0000 7fdc4fff1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdc4813b660 0x7fdc48063a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:02.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.667+0000 7fdc4fff1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdc480beb10 0x7fdc480669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:02.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.667+0000 7fdc4fff1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc48067aa0 con 0x7fdc48066520 2024-01-24T14:17:02.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.667+0000 7fdc4fff1700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc48068900 con 0x7fdc4813b660 2024-01-24T14:17:02.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.667+0000 7fdc4fff1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc48066e90 con 0x7fdc480beb10 2024-01-24T14:17:02.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.667+0000 7fdc4dd8d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdc4813b660 0x7fdc48063a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:02.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.667+0000 7fdc4dd8d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdc4813b660 0x7fdc48063a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43782/0 (socket says 172.21.15.29:43782) 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.667+0000 7fdc4dd8d700 1 -- 172.21.15.29:0/3956792861 learned_addr learned my addr 172.21.15.29:0/3956792861 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.667+0000 7fdc4e58e700 1 --2- 172.21.15.29:0/3956792861 >> v2:172.21.15.163:3300/0 conn(0x7fdc480beb10 0x7fdc480669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.667+0000 7fdc4dd8d700 1 -- 172.21.15.29:0/3956792861 >> v2:172.21.15.163:3300/0 conn(0x7fdc480beb10 msgr2=0x7fdc480669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.667+0000 7fdc4dd8d700 1 --2- 172.21.15.29:0/3956792861 >> v2:172.21.15.163:3300/0 conn(0x7fdc480beb10 0x7fdc480669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.667+0000 7fdc4dd8d700 1 -- 172.21.15.29:0/3956792861 >> v1:172.21.15.163:6789/0 conn(0x7fdc48066520 legacy=0x7fdc4815c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.668+0000 7fdc4dd8d700 1 -- 172.21.15.29:0/3956792861 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4813ba40 con 0x7fdc4813b660 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.668+0000 7fdc4dd8d700 1 --2- 172.21.15.29:0/3956792861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc4813b660 0x7fdc48063a00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7fdc480c9890 tx=0x7fdc440090f0).ready entity=mon.0 client_cookie=b29fee26791b4f0b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.668+0000 7fdc4cd8b700 1 -- 172.21.15.29:0/3956792861 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4400b520 con 0x7fdc4813b660 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.668+0000 7fdc4cd8b700 1 -- 172.21.15.29:0/3956792861 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4400b680 con 0x7fdc4813b660 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.668+0000 7fdc4cd8b700 1 -- 172.21.15.29:0/3956792861 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4400ba00 con 0x7fdc4813b660 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.668+0000 7fdc4fff1700 1 -- 172.21.15.29:0/3956792861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc4813b660 msgr2=0x7fdc48063a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.668+0000 7fdc4fff1700 1 --2- 172.21.15.29:0/3956792861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc4813b660 0x7fdc48063a00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7fdc480c9890 tx=0x7fdc440090f0).stop 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.668+0000 7fdc4fff1700 1 -- 172.21.15.29:0/3956792861 shutdown_connections 2024-01-24T14:17:02.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.668+0000 7fdc4fff1700 1 --2- 172.21.15.29:0/3956792861 >> v2:172.21.15.163:3300/0 conn(0x7fdc480beb10 0x7fdc480669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.668+0000 7fdc4fff1700 1 --2- 172.21.15.29:0/3956792861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc4813b660 0x7fdc48063a00 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.668+0000 7fdc4fff1700 1 -- 172.21.15.29:0/3956792861 >> 172.21.15.29:0/3956792861 conn(0x7fdc480c8ba0 msgr2=0x7fdc480c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:02.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.669+0000 7fdc4fff1700 1 -- 172.21.15.29:0/3956792861 shutdown_connections 2024-01-24T14:17:02.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.669+0000 7fdc4fff1700 1 -- 172.21.15.29:0/3956792861 wait complete. 2024-01-24T14:17:02.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.670+0000 7fdc4fff1700 1 Processor -- start 2024-01-24T14:17:02.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.670+0000 7fdc4fff1700 1 -- start start 2024-01-24T14:17:02.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4fff1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc48066520 0x7fdc48168bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:02.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4fff1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc480beb10 0x7fdc480ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:02.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4fff1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc4813b660 0x7fdc480ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:02.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4fff1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480a8c70 con 0x7fdc4813b660 2024-01-24T14:17:02.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4fff1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480ae030 con 0x7fdc48066520 2024-01-24T14:17:02.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4fff1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480ae310 con 0x7fdc480beb10 2024-01-24T14:17:02.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4e58e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc4813b660 0x7fdc480ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:02.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4e58e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc4813b660 0x7fdc480ad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43788/0 (socket says 172.21.15.29:43788) 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4e58e700 1 -- 172.21.15.29:0/560133518 learned_addr learned my addr 172.21.15.29:0/560133518 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4dd8d700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc48066520 0x7fdc48168bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4d58c700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc480beb10 0x7fdc480ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4e58e700 1 -- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc480beb10 msgr2=0x7fdc480ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4e58e700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc480beb10 0x7fdc480ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4e58e700 1 -- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc48066520 msgr2=0x7fdc48168bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4e58e700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc48066520 0x7fdc48168bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4e58e700 1 -- 172.21.15.29:0/560133518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc44009370 con 0x7fdc4813b660 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4dd8d700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc48066520 0x7fdc48168bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4e58e700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc4813b660 0x7fdc480ad9e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7fdc40009d70 tx=0x7fdc4000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc3dffb700 1 -- 172.21.15.29:0/560133518 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000cb50 con 0x7fdc4813b660 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc3dffb700 1 -- 172.21.15.29:0/560133518 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4000f040 con 0x7fdc4813b660 2024-01-24T14:17:02.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc4fff1700 1 -- 172.21.15.29:0/560133518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc480a8ec0 con 0x7fdc4813b660 2024-01-24T14:17:02.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.671+0000 7fdc3dffb700 1 -- 172.21.15.29:0/560133518 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40013400 con 0x7fdc4813b660 2024-01-24T14:17:02.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.672+0000 7fdc4fff1700 1 -- 172.21.15.29:0/560133518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4813f1b0 con 0x7fdc4813b660 2024-01-24T14:17:02.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.673+0000 7fdc4fff1700 1 -- 172.21.15.29:0/560133518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc480475e0 con 0x7fdc4813b660 2024-01-24T14:17:02.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.673+0000 7fdc3dffb700 1 -- 172.21.15.29:0/560133518 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 53) v1 ==== 75679+0+0 (secure 0 0 0) 0x7fdc400137b0 con 0x7fdc4813b660 2024-01-24T14:17:02.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.673+0000 7fdc3dffb700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdc2c05f3e0 0x7fdc2c0618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:02.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.674+0000 7fdc3dffb700 1 -- 172.21.15.29:0/560133518 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fdc4007b3f0 con 0x7fdc4813b660 2024-01-24T14:17:02.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.674+0000 7fdc4dd8d700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdc2c05f3e0 0x7fdc2c0618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:02.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.674+0000 7fdc4dd8d700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdc2c05f3e0 0x7fdc2c0618a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fdc44009130 tx=0x7fdc44017000).ready entity=mgr.6791 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:02.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.678+0000 7fdc3dffb700 1 -- 172.21.15.29:0/560133518 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdc4004bdb0 con 0x7fdc4813b660 2024-01-24T14:17:02.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.907+0000 7fdc4fff1700 1 -- 172.21.15.29:0/560133518 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fdc48067cf0 con 0x7fdc4813b660 2024-01-24T14:17:02.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.943+0000 7fdc3dffb700 1 -- 172.21.15.29:0/560133518 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 54) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fdc400788b0 con 0x7fdc4813b660 2024-01-24T14:17:02.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.945+0000 7fdc3dffb700 1 -- 172.21.15.29:0/560133518 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v54) v1 ==== 93+0+0 (secure 0 0 0) 0x7fdc400715f0 con 0x7fdc4813b660 2024-01-24T14:17:02.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.947+0000 7fdc4fff1700 1 -- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdc2c05f3e0 msgr2=0x7fdc2c0618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.947+0000 7fdc4fff1700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdc2c05f3e0 0x7fdc2c0618a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fdc44009130 tx=0x7fdc44017000).stop 2024-01-24T14:17:02.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.947+0000 7fdc4fff1700 1 -- 172.21.15.29:0/560133518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc4813b660 msgr2=0x7fdc480ad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:02.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.947+0000 7fdc4fff1700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc4813b660 0x7fdc480ad9e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7fdc40009d70 tx=0x7fdc4000b040).stop 2024-01-24T14:17:02.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.947+0000 7fdc4fff1700 1 -- 172.21.15.29:0/560133518 shutdown_connections 2024-01-24T14:17:02.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.947+0000 7fdc4fff1700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdc2c05f3e0 0x7fdc2c0618a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.947+0000 7fdc4fff1700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc4813b660 0x7fdc480ad9e0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.947+0000 7fdc4fff1700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc48066520 0x7fdc48168bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.947+0000 7fdc4fff1700 1 --2- 172.21.15.29:0/560133518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc480beb10 0x7fdc480ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:02.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.947+0000 7fdc4fff1700 1 -- 172.21.15.29:0/560133518 >> 172.21.15.29:0/560133518 conn(0x7fdc480c8ba0 msgr2=0x7fdc48159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:02.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.947+0000 7fdc4fff1700 1 -- 172.21.15.29:0/560133518 shutdown_connections 2024-01-24T14:17:02.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:02.947+0000 7fdc4fff1700 1 -- 172.21.15.29:0/560133518 wait complete. 2024-01-24T14:17:02.961 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T14:17:02.962 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:17:03.008 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:17:03.043 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T14:17:03.044 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-24T14:17:03.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:17:03.044 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-24T14:17:03.089 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:03.089+0000 7fc883c12680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:17:03.096 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T14:17:03.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:17:03.097 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-24T14:17:03.104 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:03.104+0000 7f44ddf45680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:17:03.115 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:17:03.153 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:03.153+0000 7fc883c12680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:17:03.179 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:03.178+0000 7f44ddf45680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:17:03.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.288+0000 7fc2457cf700 1 Processor -- start 2024-01-24T14:17:03.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.288+0000 7fc2457cf700 1 -- start start 2024-01-24T14:17:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc2457cf700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc240134f10 0x7fc2400627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc2457cf700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc240062ca0 0x7fc2401527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc2457cf700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc240064d90 con 0x7fc240152cd0 2024-01-24T14:17:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc2457cf700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc240064250 con 0x7fc240134f10 2024-01-24T14:17:03.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc2457cf700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc240063120 con 0x7fc240062ca0 2024-01-24T14:17:03.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc23effd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc240134f10 0x7fc2400627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:03.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc23effd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc240134f10 0x7fc2400627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43804/0 (socket says 172.21.15.29:43804) 2024-01-24T14:17:03.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc23effd700 1 -- 172.21.15.29:0/3469621558 learned_addr learned my addr 172.21.15.29:0/3469621558 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:03.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc23e7fc700 1 --2- 172.21.15.29:0/3469621558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc240062ca0 0x7fc2401527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:03.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc23effd700 1 -- 172.21.15.29:0/3469621558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc240062ca0 msgr2=0x7fc2401527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc23effd700 1 --2- 172.21.15.29:0/3469621558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc240062ca0 0x7fc2401527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc23effd700 1 -- 172.21.15.29:0/3469621558 >> v1:172.21.15.163:6789/0 conn(0x7fc240152cd0 legacy=0x7fc240155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.289+0000 7fc23effd700 1 -- 172.21.15.29:0/3469621558 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc240065220 con 0x7fc240134f10 2024-01-24T14:17:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.290+0000 7fc23effd700 1 --2- 172.21.15.29:0/3469621558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240134f10 0x7fc2400627b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7fc230006fd0 tx=0x7fc2300090f0).ready entity=mon.0 client_cookie=94453dbc4a9db19f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.290+0000 7fc23dffb700 1 -- 172.21.15.29:0/3469621558 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23000b580 con 0x7fc240134f10 2024-01-24T14:17:03.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.290+0000 7fc23dffb700 1 -- 172.21.15.29:0/3469621558 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23000b6e0 con 0x7fc240134f10 2024-01-24T14:17:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.290+0000 7fc23dffb700 1 -- 172.21.15.29:0/3469621558 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23000ba80 con 0x7fc240134f10 2024-01-24T14:17:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.290+0000 7fc2457cf700 1 -- 172.21.15.29:0/3469621558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240134f10 msgr2=0x7fc2400627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.290+0000 7fc2457cf700 1 --2- 172.21.15.29:0/3469621558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240134f10 0x7fc2400627b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7fc230006fd0 tx=0x7fc2300090f0).stop 2024-01-24T14:17:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.290+0000 7fc2457cf700 1 -- 172.21.15.29:0/3469621558 shutdown_connections 2024-01-24T14:17:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.290+0000 7fc2457cf700 1 --2- 172.21.15.29:0/3469621558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240134f10 0x7fc2400627b0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.290+0000 7fc2457cf700 1 --2- 172.21.15.29:0/3469621558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc240062ca0 0x7fc2401527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.290+0000 7fc2457cf700 1 -- 172.21.15.29:0/3469621558 >> 172.21.15.29:0/3469621558 conn(0x7fc2400c69f0 msgr2=0x7fc2400c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:03.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.290+0000 7fc2457cf700 1 -- 172.21.15.29:0/3469621558 shutdown_connections 2024-01-24T14:17:03.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.290+0000 7fc2457cf700 1 -- 172.21.15.29:0/3469621558 wait complete. 2024-01-24T14:17:03.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.292+0000 7fc2457cf700 1 Processor -- start 2024-01-24T14:17:03.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.292+0000 7fc2457cf700 1 -- start start 2024-01-24T14:17:03.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc2457cf700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc240062ca0 0x7fc240137b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:03.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc2457cf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240134f10 0x7fc2400bf940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:03.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc2457cf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc240152cd0 0x7fc2400c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:03.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc2457cf700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc240133ee0 con 0x7fc240134f10 2024-01-24T14:17:03.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc2457cf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400bec50 con 0x7fc240152cd0 2024-01-24T14:17:03.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc2457cf700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400bfe30 con 0x7fc240062ca0 2024-01-24T14:17:03.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc23e7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240134f10 0x7fc2400bf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:03.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc23e7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240134f10 0x7fc2400bf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43814/0 (socket says 172.21.15.29:43814) 2024-01-24T14:17:03.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc23e7fc700 1 -- 172.21.15.29:0/3637388984 learned_addr learned my addr 172.21.15.29:0/3637388984 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:03.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc23effd700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc240062ca0 0x7fc240137b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:03.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc23f7fe700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc240152cd0 0x7fc2400c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:03.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc23e7fc700 1 -- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc240062ca0 msgr2=0x7fc240137b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:03.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc23e7fc700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc240062ca0 0x7fc240137b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc23e7fc700 1 -- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc240152cd0 msgr2=0x7fc2400c0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc23e7fc700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc240152cd0 0x7fc2400c0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc23e7fc700 1 -- 172.21.15.29:0/3637388984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc230009370 con 0x7fc240134f10 2024-01-24T14:17:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.293+0000 7fc23e7fc700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240134f10 0x7fc2400bf940 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7fc23400e900 tx=0x7fc234009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.294+0000 7fc226ffd700 1 -- 172.21.15.29:0/3637388984 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc234012600 con 0x7fc240134f10 2024-01-24T14:17:03.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.294+0000 7fc226ffd700 1 -- 172.21.15.29:0/3637388984 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc234012760 con 0x7fc240134f10 2024-01-24T14:17:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.294+0000 7fc226ffd700 1 -- 172.21.15.29:0/3637388984 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc234012ae0 con 0x7fc240134f10 2024-01-24T14:17:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.294+0000 7fc2457cf700 1 -- 172.21.15.29:0/3637388984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2400bf4b0 con 0x7fc240134f10 2024-01-24T14:17:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.294+0000 7fc2457cf700 1 -- 172.21.15.29:0/3637388984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2400bee60 con 0x7fc240134f10 2024-01-24T14:17:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.295+0000 7fc2457cf700 1 -- 172.21.15.29:0/3637388984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2400bf640 con 0x7fc240134f10 2024-01-24T14:17:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.295+0000 7fc226ffd700 1 -- 172.21.15.29:0/3637388984 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 54) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fc23401b400 con 0x7fc240134f10 2024-01-24T14:17:03.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.296+0000 7fc226ffd700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc22c05f390 0x7fc22c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.296+0000 7fc23effd700 1 -- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc22c05f390 msgr2=0x7fc22c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.296+0000 7fc226ffd700 1 -- 172.21.15.29:0/3637388984 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fc23407adb0 con 0x7fc240134f10 2024-01-24T14:17:03.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.296+0000 7fc23effd700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc22c05f390 0x7fc22c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:03.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.300+0000 7fc226ffd700 1 -- 172.21.15.29:0/3637388984 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc23404b820 con 0x7fc240134f10 2024-01-24T14:17:03.427 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:03.427+0000 7fc883c12680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:17:03.496 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:03.496+0000 7f44ddf45680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:17:03.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.497+0000 7fc23effd700 1 -- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc22c05f390 msgr2=0x7fc22c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:03.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.497+0000 7fc23effd700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc22c05f390 0x7fc22c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:17:03.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.526+0000 7fc2457cf700 1 -- 172.21.15.29:0/3637388984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2400656c0 con 0x7fc240134f10 2024-01-24T14:17:03.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.527+0000 7fc226ffd700 1 -- 172.21.15.29:0/3637388984 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fc234016070 con 0x7fc240134f10 2024-01-24T14:17:03.528 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:03.528 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1579003637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1579003637},{"type":"v1","addr":"172.21.15.163:6836","nonce":1579003637}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2823175689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2823175689},{"type":"v1","addr":"172.21.15.163:6837","nonce":2823175689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2560003609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2560003609},{"type":"v1","addr":"172.21.15.29:6834","nonce":2560003609}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:16:09.332377+0000","modified":"2024-01-24T14:16:12.347265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1458876872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1458876872},{"type":"v1","addr":"172.21.15.29:6835","nonce":1458876872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-24T14:17:03.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.529+0000 7fc2457cf700 1 -- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc22c05f390 msgr2=0x7fc22c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:03.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.529+0000 7fc2457cf700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc22c05f390 0x7fc22c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:03.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.529+0000 7fc2457cf700 1 -- 172.21.15.29:0/3637388984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240134f10 msgr2=0x7fc2400bf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:03.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.529+0000 7fc2457cf700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240134f10 0x7fc2400bf940 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7fc23400e900 tx=0x7fc234009060).stop 2024-01-24T14:17:03.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.529+0000 7fc2457cf700 1 -- 172.21.15.29:0/3637388984 shutdown_connections 2024-01-24T14:17:03.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.529+0000 7fc2457cf700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc240152cd0 0x7fc2400c0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.529+0000 7fc2457cf700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc240062ca0 0x7fc240137b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.529+0000 7fc2457cf700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc22c05f390 0x7fc22c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:03.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.529+0000 7fc2457cf700 1 --2- 172.21.15.29:0/3637388984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc240134f10 0x7fc2400bf940 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.529+0000 7fc2457cf700 1 -- 172.21.15.29:0/3637388984 >> 172.21.15.29:0/3637388984 conn(0x7fc2400c69f0 msgr2=0x7fc2400bb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:03.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.529+0000 7fc2457cf700 1 -- 172.21.15.29:0/3637388984 shutdown_connections 2024-01-24T14:17:03.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.529+0000 7fc2457cf700 1 -- 172.21.15.29:0/3637388984 wait complete. 2024-01-24T14:17:03.531 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 59 2024-01-24T14:17:03.544 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:17:03.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.715+0000 7f8be40cc700 1 Processor -- start 2024-01-24T14:17:03.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.716+0000 7f8be40cc700 1 -- start start 2024-01-24T14:17:03.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.716+0000 7f8be40cc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8bdc13a5c0 0x7f8bdc063a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:03.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.716+0000 7f8be40cc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bdc066c00 0x7f8bdc0c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:03.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.716+0000 7f8be40cc700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bdc0bf6a0 con 0x7f8bdc0be520 2024-01-24T14:17:03.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.716+0000 7f8be40cc700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bdc0bea90 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.716+0000 7f8be40cc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc0bed70 con 0x7f8bdc066c00 2024-01-24T14:17:03.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.717+0000 7f8be1e68700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8bdc13a5c0 0x7f8bdc063a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:03.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.717+0000 7f8be1667700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8bdc0be520 0x7f8bdc0664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38500/0 (socket says 172.21.15.29:38500) 2024-01-24T14:17:03.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.717+0000 7f8be1667700 1 -- 172.21.15.29:0/1608154987 learned_addr learned my addr 172.21.15.29:0/1608154987 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:03.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.717+0000 7f8be1e68700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8bdc13a5c0 0x7f8bdc063a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49214/0 (socket says 172.21.15.29:49214) 2024-01-24T14:17:03.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.717+0000 7f8be2669700 1 --2- 172.21.15.29:0/1608154987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bdc066c00 0x7f8bdc0c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.717+0000 7f8be1e68700 1 -- 172.21.15.29:0/1608154987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bdc066c00 msgr2=0x7f8bdc0c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.717+0000 7f8be1e68700 1 --2- 172.21.15.29:0/1608154987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bdc066c00 0x7f8bdc0c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.717+0000 7f8be1e68700 1 -- 172.21.15.29:0/1608154987 >> v1:172.21.15.29:6789/0 conn(0x7f8bdc0be520 legacy=0x7f8bdc0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.717+0000 7f8be0e66700 1 -- 172.21.15.29:0/1608154987 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725251455 0 0) 0x7f8bdc0bf6a0 con 0x7f8bdc0be520 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.717+0000 7f8be1e68700 1 -- 172.21.15.29:0/1608154987 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc0bf050 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.717+0000 7f8be1e68700 1 --2- 172.21.15.29:0/1608154987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bdc13a5c0 0x7f8bdc063a10 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f8bd8009760 tx=0x7f8bd8009060).ready entity=mon.0 client_cookie=85057c3dc22bb644 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.717+0000 7f8be0e66700 1 -- 172.21.15.29:0/1608154987 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd800f510 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.718+0000 7f8be0e66700 1 -- 172.21.15.29:0/1608154987 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd800f670 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.718+0000 7f8be0e66700 1 -- 172.21.15.29:0/1608154987 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd800f9f0 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.718+0000 7f8be40cc700 1 -- 172.21.15.29:0/1608154987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bdc13a5c0 msgr2=0x7f8bdc063a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.718+0000 7f8be40cc700 1 --2- 172.21.15.29:0/1608154987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bdc13a5c0 0x7f8bdc063a10 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f8bd8009760 tx=0x7f8bd8009060).stop 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.718+0000 7f8be40cc700 1 -- 172.21.15.29:0/1608154987 shutdown_connections 2024-01-24T14:17:03.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.718+0000 7f8be40cc700 1 --2- 172.21.15.29:0/1608154987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bdc066c00 0x7f8bdc0c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.718+0000 7f8be40cc700 1 --2- 172.21.15.29:0/1608154987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bdc13a5c0 0x7f8bdc063a10 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.718+0000 7f8be40cc700 1 -- 172.21.15.29:0/1608154987 >> 172.21.15.29:0/1608154987 conn(0x7f8bdc0c8bb0 msgr2=0x7f8bdc0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:03.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.718+0000 7f8be40cc700 1 -- 172.21.15.29:0/1608154987 shutdown_connections 2024-01-24T14:17:03.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.718+0000 7f8be40cc700 1 -- 172.21.15.29:0/1608154987 wait complete. 2024-01-24T14:17:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.720+0000 7f8be40cc700 1 Processor -- start 2024-01-24T14:17:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.720+0000 7f8be40cc700 1 -- start start 2024-01-24T14:17:03.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.720+0000 7f8be40cc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bdc066c00 0x7f8bdc1671e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.720+0000 7f8be40cc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bdc0be520 0x7f8bdc0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.720+0000 7f8be40cc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bdc13a5c0 0x7f8bdc0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.720+0000 7f8be40cc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc0a5d20 con 0x7f8bdc0be520 2024-01-24T14:17:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.720+0000 7f8be40cc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc0a7540 con 0x7f8bdc066c00 2024-01-24T14:17:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.720+0000 7f8be40cc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc0a6a80 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8be1667700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bdc0be520 0x7f8bdc0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:03.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8be1667700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bdc0be520 0x7f8bdc0a5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49226/0 (socket says 172.21.15.29:49226) 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8be2669700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bdc13a5c0 0x7f8bdc0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8be1667700 1 -- 172.21.15.29:0/3774427233 learned_addr learned my addr 172.21.15.29:0/3774427233 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8be1e68700 1 --2- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bdc066c00 0x7f8bdc1671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8be2669700 1 -- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bdc066c00 msgr2=0x7f8bdc1671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8be2669700 1 --2- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bdc066c00 0x7f8bdc1671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8be2669700 1 -- 172.21.15.29:0/3774427233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bdc0be520 msgr2=0x7f8bdc0a5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8be2669700 1 --2- 172.21.15.29:0/3774427233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bdc0be520 0x7f8bdc0a5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8be2669700 1 -- 172.21.15.29:0/3774427233 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd80092a0 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8be2669700 1 --2- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bdc13a5c0 0x7f8bdc0a5830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f8bd4009e20 tx=0x7f8bd400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8bd1ffb700 1 -- 172.21.15.29:0/3774427233 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400caa0 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8bd1ffb700 1 -- 172.21.15.29:0/3774427233 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd400f040 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8be40cc700 1 -- 172.21.15.29:0/3774427233 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bdc0a5f00 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.721+0000 7f8bd1ffb700 1 -- 172.21.15.29:0/3774427233 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400ce10 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.722+0000 7f8be40cc700 1 -- 172.21.15.29:0/3774427233 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc0a64c0 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.723+0000 7f8be40cc700 1 -- 172.21.15.29:0/3774427233 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bdc1642d0 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.723+0000 7f8bd1ffb700 1 -- 172.21.15.29:0/3774427233 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 54) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f8bd4013610 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.723+0000 7f8bd1ffb700 1 --2- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8bc005f390 0x7f8bc0061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.723+0000 7f8bd1ffb700 1 -- 172.21.15.29:0/3774427233 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f8bd407b310 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.723+0000 7f8be1e68700 1 -- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8bc005f390 msgr2=0x7f8bc0061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.724+0000 7f8be1e68700 1 --2- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8bc005f390 0x7f8bc0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:03.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.728+0000 7f8bd1ffb700 1 -- 172.21.15.29:0/3774427233 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8bdc1642d0 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.911+0000 7f8be40cc700 1 -- 172.21.15.29:0/3774427233 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bdc1642d0 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.912+0000 7f8bd1ffb700 1 -- 172.21.15.29:0/3774427233 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18556 (secure 0 0 0) 0x7f8bdc1642d0 con 0x7f8bdc13a5c0 2024-01-24T14:17:03.914 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:03.914 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":138,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:16:21.610043+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:16:07.075847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-24T14:16:08.308597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6833/43241":"2024-01-25T14:16:21.610007+0000","172.21.15.163:6832/43241":"2024-01-25T14:16:21.610007+0000","172.21.15.163:0/187931593":"2024-01-25T14:16:21.610007+0000","172.21.15.163:0/455194783":"2024-01-25T14:16:21.610007+0000","172.21.15.163:0/1937450853":"2024-01-25T14:16:21.610007+0000","172.21.15.163:0/3338508841":"2024-01-25T14:16:21.610007+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:17:03.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.915+0000 7f8be40cc700 1 -- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8bc005f390 msgr2=0x7f8bc0061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:03.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.915+0000 7f8be40cc700 1 --2- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8bc005f390 0x7f8bc0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:03.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.915+0000 7f8be40cc700 1 -- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bdc13a5c0 msgr2=0x7f8bdc0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:03.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.915+0000 7f8be40cc700 1 --2- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bdc13a5c0 0x7f8bdc0a5830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f8bd4009e20 tx=0x7f8bd400b040).stop 2024-01-24T14:17:03.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.915+0000 7f8be40cc700 1 -- 172.21.15.29:0/3774427233 shutdown_connections 2024-01-24T14:17:03.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.915+0000 7f8be40cc700 1 --2- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bdc13a5c0 0x7f8bdc0a5830 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.915+0000 7f8be40cc700 1 --2- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bdc066c00 0x7f8bdc1671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.915+0000 7f8be40cc700 1 --2- 172.21.15.29:0/3774427233 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8bc005f390 0x7f8bc0061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:03.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.915+0000 7f8be40cc700 1 --2- 172.21.15.29:0/3774427233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bdc0be520 0x7f8bdc0a5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:03.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.915+0000 7f8be40cc700 1 -- 172.21.15.29:0/3774427233 >> 172.21.15.29:0/3774427233 conn(0x7f8bdc0c8bb0 msgr2=0x7f8bdc0ad6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:03.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.915+0000 7f8be40cc700 1 -- 172.21.15.29:0/3774427233 shutdown_connections 2024-01-24T14:17:03.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:03.915+0000 7f8be40cc700 1 -- 172.21.15.29:0/3774427233 wait complete. 2024-01-24T14:17:03.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T14:16:07.075847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T14:16:08.308597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T14:17:03.930 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T14:17:04.047 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:04.046+0000 7fc883c12680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:17:04.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.070+0000 7ff441ad1700 1 Processor -- start 2024-01-24T14:17:04.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.070+0000 7ff441ad1700 1 -- start start 2024-01-24T14:17:04.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff441ad1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff43c0c8290 0x7ff43c0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff441ad1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff43c154320 0x7ff43c062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff441ad1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c06a360 con 0x7ff43c153f40 2024-01-24T14:17:04.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff441ad1700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c063190 con 0x7ff43c154320 2024-01-24T14:17:04.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff441ad1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c0a74c0 con 0x7ff43c0c8290 2024-01-24T14:17:04.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff43bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff43c154320 0x7ff43c062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff43bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff43c154320 0x7ff43c062ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49232/0 (socket says 172.21.15.29:49232) 2024-01-24T14:17:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff43bfff700 1 -- 172.21.15.29:0/2050395791 learned_addr learned my addr 172.21.15.29:0/2050395791 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff43affd700 1 --2- 172.21.15.29:0/2050395791 >> v2:172.21.15.163:3300/0 conn(0x7ff43c0c8290 0x7ff43c0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff43bfff700 1 -- 172.21.15.29:0/2050395791 >> v2:172.21.15.163:3300/0 conn(0x7ff43c0c8290 msgr2=0x7ff43c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff43bfff700 1 --2- 172.21.15.29:0/2050395791 >> v2:172.21.15.163:3300/0 conn(0x7ff43c0c8290 0x7ff43c0627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff43bfff700 1 -- 172.21.15.29:0/2050395791 >> v1:172.21.15.163:6789/0 conn(0x7ff43c153f40 legacy=0x7ff43c0a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff43bfff700 1 -- 172.21.15.29:0/2050395791 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0bcef0 con 0x7ff43c154320 2024-01-24T14:17:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.071+0000 7ff43bfff700 1 --2- 172.21.15.29:0/2050395791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c154320 0x7ff43c062ca0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7ff43c0c58f0 tx=0x7ff434009840).ready entity=mon.0 client_cookie=965b3ee52792049d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.072+0000 7ff43a7fc700 1 -- 172.21.15.29:0/2050395791 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43400e070 con 0x7ff43c154320 2024-01-24T14:17:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.072+0000 7ff43a7fc700 1 -- 172.21.15.29:0/2050395791 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43400ba20 con 0x7ff43c154320 2024-01-24T14:17:04.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.072+0000 7ff43a7fc700 1 -- 172.21.15.29:0/2050395791 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43400bda0 con 0x7ff43c154320 2024-01-24T14:17:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.072+0000 7ff441ad1700 1 -- 172.21.15.29:0/2050395791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c154320 msgr2=0x7ff43c062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.072+0000 7ff441ad1700 1 --2- 172.21.15.29:0/2050395791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c154320 0x7ff43c062ca0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7ff43c0c58f0 tx=0x7ff434009840).stop 2024-01-24T14:17:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.072+0000 7ff441ad1700 1 -- 172.21.15.29:0/2050395791 shutdown_connections 2024-01-24T14:17:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.072+0000 7ff441ad1700 1 --2- 172.21.15.29:0/2050395791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c154320 0x7ff43c062ca0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.072+0000 7ff441ad1700 1 --2- 172.21.15.29:0/2050395791 >> v2:172.21.15.163:3300/0 conn(0x7ff43c0c8290 0x7ff43c0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.072+0000 7ff441ad1700 1 -- 172.21.15.29:0/2050395791 >> 172.21.15.29:0/2050395791 conn(0x7ff43c064390 msgr2=0x7ff43c0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.072+0000 7ff441ad1700 1 -- 172.21.15.29:0/2050395791 shutdown_connections 2024-01-24T14:17:04.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.072+0000 7ff441ad1700 1 -- 172.21.15.29:0/2050395791 wait complete. 2024-01-24T14:17:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.074+0000 7ff441ad1700 1 Processor -- start 2024-01-24T14:17:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.074+0000 7ff441ad1700 1 -- start start 2024-01-24T14:17:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff441ad1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff43c0c8290 0x7ff43c0c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff441ad1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff43c153f40 0x7ff43c0beed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff441ad1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c154320 0x7ff43c0aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff441ad1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0c25c0 con 0x7ff43c154320 2024-01-24T14:17:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff441ad1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0c3410 con 0x7ff43c153f40 2024-01-24T14:17:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff441ad1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0bf3c0 con 0x7ff43c0c8290 2024-01-24T14:17:04.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff43bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c154320 0x7ff43c0aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff43bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c154320 0x7ff43c0aceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49248/0 (socket says 172.21.15.29:49248) 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff43bfff700 1 -- 172.21.15.29:0/3446596469 learned_addr learned my addr 172.21.15.29:0/3446596469 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff43affd700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff43c153f40 0x7ff43c0beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff43b7fe700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff43c0c8290 0x7ff43c0c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff43bfff700 1 -- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff43c0c8290 msgr2=0x7ff43c0c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff43bfff700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff43c0c8290 0x7ff43c0c20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff43bfff700 1 -- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff43c153f40 msgr2=0x7ff43c0beed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff43bfff700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff43c153f40 0x7ff43c0beed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff43bfff700 1 -- 172.21.15.29:0/3446596469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff434009ae0 con 0x7ff43c154320 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.075+0000 7ff43affd700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff43c153f40 0x7ff43c0beed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.076+0000 7ff43bfff700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c154320 0x7ff43c0aceb0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7ff43400b6c0 tx=0x7ff43c0bce60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.076+0000 7ff4337fe700 1 -- 172.21.15.29:0/3446596469 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434007080 con 0x7ff43c154320 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.076+0000 7ff4337fe700 1 -- 172.21.15.29:0/3446596469 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4340092e0 con 0x7ff43c154320 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.076+0000 7ff441ad1700 1 -- 172.21.15.29:0/3446596469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c0c2800 con 0x7ff43c154320 2024-01-24T14:17:04.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.076+0000 7ff4337fe700 1 -- 172.21.15.29:0/3446596469 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43401d690 con 0x7ff43c154320 2024-01-24T14:17:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.076+0000 7ff441ad1700 1 -- 172.21.15.29:0/3446596469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c0c2dc0 con 0x7ff43c154320 2024-01-24T14:17:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.077+0000 7ff441ad1700 1 -- 172.21.15.29:0/3446596469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c0475e0 con 0x7ff43c154320 2024-01-24T14:17:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.078+0000 7ff4337fe700 1 -- 172.21.15.29:0/3446596469 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 54) v1 ==== 75662+0+0 (secure 0 0 0) 0x7ff434003040 con 0x7ff43c154320 2024-01-24T14:17:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.078+0000 7ff4337fe700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff41c05f390 0x7ff41c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.079+0000 7ff43b7fe700 1 -- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff41c05f390 msgr2=0x7ff41c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.079+0000 7ff4337fe700 1 -- 172.21.15.29:0/3446596469 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7456+0+0 (secure 0 0 0) 0x7ff43407c730 con 0x7ff43c154320 2024-01-24T14:17:04.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.079+0000 7ff43b7fe700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff41c05f390 0x7ff41c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:04.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.082+0000 7ff4337fe700 1 -- 172.21.15.29:0/3446596469 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff43404d150 con 0x7ff43c154320 2024-01-24T14:17:04.120 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:04.120+0000 7fc883c12680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:17:04.233 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:04.233+0000 7f44ddf45680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:17:04.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.280+0000 7ff43b7fe700 1 -- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff41c05f390 msgr2=0x7ff41c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:04.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.280+0000 7ff43b7fe700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff41c05f390 0x7ff41c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:17:04.308 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:04.308+0000 7fc883c12680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:17:04.310 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:04.310+0000 7f44ddf45680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:17:04.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.311+0000 7ff441ad1700 1 -- 172.21.15.29:0/3446596469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff43c06a540 con 0x7ff43c154320 2024-01-24T14:17:04.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.312+0000 7ff4337fe700 1 -- 172.21.15.29:0/3446596469 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v139) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff43402c030 con 0x7ff43c154320 2024-01-24T14:17:04.313 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T14:17:04.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.314+0000 7ff441ad1700 1 -- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff41c05f390 msgr2=0x7ff41c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:04.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.314+0000 7ff441ad1700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff41c05f390 0x7ff41c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:04.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.314+0000 7ff441ad1700 1 -- 172.21.15.29:0/3446596469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c154320 msgr2=0x7ff43c0aceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.314+0000 7ff441ad1700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c154320 0x7ff43c0aceb0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7ff43400b6c0 tx=0x7ff43c0bce60).stop 2024-01-24T14:17:04.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.314+0000 7ff441ad1700 1 -- 172.21.15.29:0/3446596469 shutdown_connections 2024-01-24T14:17:04.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.314+0000 7ff441ad1700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff41c05f390 0x7ff41c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:04.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.314+0000 7ff441ad1700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff43c154320 0x7ff43c0aceb0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.314+0000 7ff441ad1700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff43c0c8290 0x7ff43c0c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.314+0000 7ff441ad1700 1 --2- 172.21.15.29:0/3446596469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff43c153f40 0x7ff43c0beed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.314+0000 7ff441ad1700 1 -- 172.21.15.29:0/3446596469 >> 172.21.15.29:0/3446596469 conn(0x7ff43c064390 msgr2=0x7ff43c0a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:04.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.314+0000 7ff441ad1700 1 -- 172.21.15.29:0/3446596469 shutdown_connections 2024-01-24T14:17:04.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.314+0000 7ff441ad1700 1 -- 172.21.15.29:0/3446596469 wait complete. 2024-01-24T14:17:04.329 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T14:17:04.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.468+0000 7f295ecda700 1 Processor -- start 2024-01-24T14:17:04.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.469+0000 7f295ecda700 1 -- start start 2024-01-24T14:17:04.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.469+0000 7f295ecda700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f295815d080 0x7f295815f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.469+0000 7f295ecda700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2958152240 con 0x7f29580c8490 2024-01-24T14:17:04.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.469+0000 7f295ecda700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29581586c0 con 0x7f2958064760 2024-01-24T14:17:04.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.469+0000 7f295ecda700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29580c0d50 con 0x7f295815d080 2024-01-24T14:17:04.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.470+0000 7f295d277700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f295815d080 0x7f295815f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.470+0000 7f2957fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f29580c8490 0x7f29580a9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38512/0 (socket says 172.21.15.29:38512) 2024-01-24T14:17:04.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.470+0000 7f2957fff700 1 -- 172.21.15.29:0/2302488849 learned_addr learned my addr 172.21.15.29:0/2302488849 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:04.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.470+0000 7f295d277700 1 -- 172.21.15.29:0/2302488849 >> v1:172.21.15.163:6789/0 conn(0x7f2958064760 legacy=0x7f2958064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.470+0000 7f295d277700 1 -- 172.21.15.29:0/2302488849 >> v1:172.21.15.29:6789/0 conn(0x7f29580c8490 legacy=0x7f29580a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.470+0000 7f29577fe700 1 -- 172.21.15.29:0/2302488849 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093063090 0 0) 0x7f2958152240 con 0x7f29580c8490 2024-01-24T14:17:04.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.470+0000 7f295d277700 1 -- 172.21.15.29:0/2302488849 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29580aa670 con 0x7f295815d080 2024-01-24T14:17:04.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.470+0000 7f295d277700 1 --2- 172.21.15.29:0/2302488849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295815d080 0x7f295815f470 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f2950009e60 tx=0x7f2950009780).ready entity=mon.0 client_cookie=641236ce3f50f070 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:04.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.470+0000 7f29577fe700 1 -- 172.21.15.29:0/2302488849 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295000e070 con 0x7f295815d080 2024-01-24T14:17:04.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.470+0000 7f29577fe700 1 -- 172.21.15.29:0/2302488849 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295000bcc0 con 0x7f295815d080 2024-01-24T14:17:04.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.470+0000 7f29577fe700 1 -- 172.21.15.29:0/2302488849 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950012620 con 0x7f295815d080 2024-01-24T14:17:04.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.471+0000 7f295ecda700 1 -- 172.21.15.29:0/2302488849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295815d080 msgr2=0x7f295815f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.471+0000 7f295ecda700 1 --2- 172.21.15.29:0/2302488849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295815d080 0x7f295815f470 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f2950009e60 tx=0x7f2950009780).stop 2024-01-24T14:17:04.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.471+0000 7f295ecda700 1 -- 172.21.15.29:0/2302488849 shutdown_connections 2024-01-24T14:17:04.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.471+0000 7f295ecda700 1 --2- 172.21.15.29:0/2302488849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295815d080 0x7f295815f470 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.471+0000 7f295ecda700 1 -- 172.21.15.29:0/2302488849 >> 172.21.15.29:0/2302488849 conn(0x7f29580cb8b0 msgr2=0x7f29580c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:04.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.471+0000 7f295ecda700 1 -- 172.21.15.29:0/2302488849 shutdown_connections 2024-01-24T14:17:04.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.471+0000 7f295ecda700 1 -- 172.21.15.29:0/2302488849 wait complete. 2024-01-24T14:17:04.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.473+0000 7f295ecda700 1 Processor -- start 2024-01-24T14:17:04.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.473+0000 7f295ecda700 1 -- start start 2024-01-24T14:17:04.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.474+0000 7f295ecda700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2958064760 0x7f295814df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.474+0000 7f295ecda700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29580c8490 0x7f295814bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.474+0000 7f295ecda700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295815d080 0x7f295814c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.474+0000 7f295ecda700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295814e410 con 0x7f295815d080 2024-01-24T14:17:04.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.474+0000 7f295ecda700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295814c4e0 con 0x7f29580c8490 2024-01-24T14:17:04.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.474+0000 7f295ecda700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295814b860 con 0x7f2958064760 2024-01-24T14:17:04.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.474+0000 7f295d277700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295815d080 0x7f295814c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.474+0000 7f295ca76700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2958064760 0x7f295814df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.474+0000 7f2957fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29580c8490 0x7f295814bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.474+0000 7f295ca76700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2958064760 0x7f295814df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:58102/0 (socket says 172.21.15.29:58102) 2024-01-24T14:17:04.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.474+0000 7f295d277700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295815d080 0x7f295814c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49270/0 (socket says 172.21.15.29:49270) 2024-01-24T14:17:04.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.474+0000 7f295ca76700 1 -- 172.21.15.29:0/3241478507 learned_addr learned my addr 172.21.15.29:0/3241478507 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:04.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.475+0000 7f2957fff700 1 -- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2958064760 msgr2=0x7f295814df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.475+0000 7f2957fff700 1 --2- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2958064760 0x7f295814df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.475+0000 7f2957fff700 1 -- 172.21.15.29:0/3241478507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295815d080 msgr2=0x7f295814c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.475+0000 7f2957fff700 1 --2- 172.21.15.29:0/3241478507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295815d080 0x7f295814c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.475+0000 7f2957fff700 1 -- 172.21.15.29:0/3241478507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2950009a20 con 0x7f29580c8490 2024-01-24T14:17:04.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.475+0000 7f295ca76700 1 --2- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2958064760 0x7f295814df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:04.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.475+0000 7f2957fff700 1 --2- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29580c8490 0x7f295814bff0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f2944000c00 tx=0x7f2944007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:04.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.475+0000 7f2954ff9700 1 -- 172.21.15.29:0/3241478507 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29440070d0 con 0x7f29580c8490 2024-01-24T14:17:04.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.475+0000 7f2954ff9700 1 -- 172.21.15.29:0/3241478507 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2944007250 con 0x7f29580c8490 2024-01-24T14:17:04.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.475+0000 7f295ecda700 1 -- 172.21.15.29:0/3241478507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295814e620 con 0x7f29580c8490 2024-01-24T14:17:04.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.475+0000 7f2954ff9700 1 -- 172.21.15.29:0/3241478507 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944007610 con 0x7f29580c8490 2024-01-24T14:17:04.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.475+0000 7f295ecda700 1 -- 172.21.15.29:0/3241478507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29580c4590 con 0x7f29580c8490 2024-01-24T14:17:04.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.476+0000 7f295ecda700 1 -- 172.21.15.29:0/3241478507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29580475e0 con 0x7f29580c8490 2024-01-24T14:17:04.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.477+0000 7f2954ff9700 1 -- 172.21.15.29:0/3241478507 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 54) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f29440059b0 con 0x7f29580c8490 2024-01-24T14:17:04.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.477+0000 7f2954ff9700 1 --2- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f293c05f390 0x7f293c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.478+0000 7f295ca76700 1 -- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f293c05f390 msgr2=0x7f293c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:04.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.478+0000 7f2954ff9700 1 -- 172.21.15.29:0/3241478507 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7456+0+0 (secure 0 0 0) 0x7f294407a090 con 0x7f29580c8490 2024-01-24T14:17:04.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.478+0000 7f295ca76700 1 --2- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f293c05f390 0x7f293c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:04.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.482+0000 7f2954ff9700 1 -- 172.21.15.29:0/3241478507 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f294404ab30 con 0x7f29580c8490 2024-01-24T14:17:04.535 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:04.534+0000 7f44ddf45680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:17:04.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.664+0000 7f295ecda700 1 -- 172.21.15.29:0/3241478507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f29580657a0 con 0x7f29580c8490 2024-01-24T14:17:04.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.665+0000 7f2954ff9700 1 -- 172.21.15.29:0/3241478507 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v139) v1 ==== 92+0+10 (secure 0 0 0) 0x7f294404e150 con 0x7f29580c8490 2024-01-24T14:17:04.666 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T14:17:04.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.667+0000 7f295ecda700 1 -- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f293c05f390 msgr2=0x7f293c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:04.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.667+0000 7f295ecda700 1 --2- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f293c05f390 0x7f293c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:04.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.667+0000 7f295ecda700 1 -- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29580c8490 msgr2=0x7f295814bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.667+0000 7f295ecda700 1 --2- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29580c8490 0x7f295814bff0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f2944000c00 tx=0x7f2944007b90).stop 2024-01-24T14:17:04.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.667+0000 7f295ecda700 1 -- 172.21.15.29:0/3241478507 shutdown_connections 2024-01-24T14:17:04.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.667+0000 7f295ecda700 1 --2- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f293c05f390 0x7f293c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:04.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.667+0000 7f295ecda700 1 --2- 172.21.15.29:0/3241478507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295815d080 0x7f295814c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.667+0000 7f295ecda700 1 --2- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2958064760 0x7f295814df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.667+0000 7f295ecda700 1 --2- 172.21.15.29:0/3241478507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29580c8490 0x7f295814bff0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.667+0000 7f295ecda700 1 -- 172.21.15.29:0/3241478507 >> 172.21.15.29:0/3241478507 conn(0x7f29580cb8b0 msgr2=0x7f29580ab620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:04.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.667+0000 7f295ecda700 1 -- 172.21.15.29:0/3241478507 shutdown_connections 2024-01-24T14:17:04.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.668+0000 7f295ecda700 1 -- 172.21.15.29:0/3241478507 wait complete. 2024-01-24T14:17:04.682 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T14:17:04.734 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:04.734+0000 7fc883c12680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:17:04.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.820+0000 7fb20387b700 1 Processor -- start 2024-01-24T14:17:04.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.820+0000 7fb20387b700 1 -- start start 2024-01-24T14:17:04.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb20387b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb1fc0692a0 0x7fb1fc065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb20387b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc066370 0x7fb1fc069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb20387b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc0c3e70 con 0x7fb1fc069b70 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb20387b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1fc0c4300 con 0x7fb1fc0692a0 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb20387b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0c3260 con 0x7fb1fc066370 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb200e16700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc066370 0x7fb1fc069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb201617700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb1fc0692a0 0x7fb1fc065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb201617700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb1fc0692a0 0x7fb1fc065e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:59594/0 (socket says 172.21.15.29:59594) 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb201e18700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb1fc069b70 0x7fb1fc0bcd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:59214/0 (socket says 172.21.15.29:59214) 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb200e16700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc066370 0x7fb1fc069680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:58114/0 (socket says 172.21.15.29:58114) 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb201617700 1 -- 172.21.15.29:0/2618374307 learned_addr learned my addr 172.21.15.29:0/2618374307 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb200e16700 1 -- 172.21.15.29:0/2618374307 >> v2:172.21.15.163:3300/0 conn(0x7fb1fc0692a0 msgr2=0x7fb1fc065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.821+0000 7fb200e16700 1 --2- 172.21.15.29:0/2618374307 >> v2:172.21.15.163:3300/0 conn(0x7fb1fc0692a0 0x7fb1fc065e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.822+0000 7fb200e16700 1 -- 172.21.15.29:0/2618374307 >> v1:172.21.15.163:6789/0 conn(0x7fb1fc069b70 legacy=0x7fb1fc0bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.822+0000 7fb200e16700 1 -- 172.21.15.29:0/2618374307 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc0c3540 con 0x7fb1fc066370 2024-01-24T14:17:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.822+0000 7fb1f3fff700 1 -- 172.21.15.29:0/2618374307 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925330437 0 0) 0x7fb1fc0c3e70 con 0x7fb1fc069b70 2024-01-24T14:17:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.822+0000 7fb201617700 1 --2- 172.21.15.29:0/2618374307 >> v2:172.21.15.163:3300/0 conn(0x7fb1fc0692a0 0x7fb1fc065e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.822+0000 7fb200e16700 1 --2- 172.21.15.29:0/2618374307 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc066370 0x7fb1fc069680 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fb1ec009760 tx=0x7fb1ec009060).ready entity=mon.2 client_cookie=454a7c91a547c43b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.822+0000 7fb1f3fff700 1 -- 172.21.15.29:0/2618374307 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec00f510 con 0x7fb1fc066370 2024-01-24T14:17:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.822+0000 7fb1f3fff700 1 -- 172.21.15.29:0/2618374307 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec00f670 con 0x7fb1fc066370 2024-01-24T14:17:04.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.822+0000 7fb1f3fff700 1 -- 172.21.15.29:0/2618374307 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec00f9f0 con 0x7fb1fc066370 2024-01-24T14:17:04.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.823+0000 7fb20387b700 1 -- 172.21.15.29:0/2618374307 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc066370 msgr2=0x7fb1fc069680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.823+0000 7fb20387b700 1 --2- 172.21.15.29:0/2618374307 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc066370 0x7fb1fc069680 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fb1ec009760 tx=0x7fb1ec009060).stop 2024-01-24T14:17:04.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.823+0000 7fb20387b700 1 -- 172.21.15.29:0/2618374307 shutdown_connections 2024-01-24T14:17:04.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.823+0000 7fb20387b700 1 --2- 172.21.15.29:0/2618374307 >> v2:172.21.15.163:3300/0 conn(0x7fb1fc0692a0 0x7fb1fc065e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.823+0000 7fb20387b700 1 --2- 172.21.15.29:0/2618374307 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc066370 0x7fb1fc069680 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.823+0000 7fb20387b700 1 -- 172.21.15.29:0/2618374307 >> 172.21.15.29:0/2618374307 conn(0x7fb1fc064390 msgr2=0x7fb1fc0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:04.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.823+0000 7fb20387b700 1 -- 172.21.15.29:0/2618374307 shutdown_connections 2024-01-24T14:17:04.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.823+0000 7fb20387b700 1 -- 172.21.15.29:0/2618374307 wait complete. 2024-01-24T14:17:04.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.824+0000 7fb20387b700 1 Processor -- start 2024-01-24T14:17:04.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.824+0000 7fb20387b700 1 -- start start 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.825+0000 7fb20387b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1fc066370 0x7fb1fc0c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.825+0000 7fb20387b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc0692a0 0x7fb1fc133b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.825+0000 7fb20387b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc069b70 0x7fb1fc134070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.825+0000 7fb20387b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc134560 con 0x7fb1fc066370 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.825+0000 7fb20387b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0bfc50 con 0x7fb1fc0692a0 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.825+0000 7fb20387b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0bff30 con 0x7fb1fc069b70 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.825+0000 7fb201617700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1fc066370 0x7fb1fc0c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.825+0000 7fb201e18700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc069b70 0x7fb1fc134070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.825+0000 7fb201617700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1fc066370 0x7fb1fc0c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49278/0 (socket says 172.21.15.29:49278) 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.825+0000 7fb200e16700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc0692a0 0x7fb1fc133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.825+0000 7fb201617700 1 -- 172.21.15.29:0/3390220830 learned_addr learned my addr 172.21.15.29:0/3390220830 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.826+0000 7fb201e18700 1 -- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc0692a0 msgr2=0x7fb1fc133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.826+0000 7fb201e18700 1 --2- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc0692a0 0x7fb1fc133b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.826+0000 7fb201e18700 1 -- 172.21.15.29:0/3390220830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1fc066370 msgr2=0x7fb1fc0c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.826+0000 7fb201e18700 1 --2- 172.21.15.29:0/3390220830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1fc066370 0x7fb1fc0c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.826+0000 7fb201e18700 1 -- 172.21.15.29:0/3390220830 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f8009370 con 0x7fb1fc069b70 2024-01-24T14:17:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.826+0000 7fb200e16700 1 --2- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc0692a0 0x7fb1fc133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.826+0000 7fb201e18700 1 --2- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc069b70 0x7fb1fc134070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fb1f400c7e0 tx=0x7fb1f4007eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.826+0000 7fb1f17fa700 1 -- 172.21.15.29:0/3390220830 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4005c70 con 0x7fb1fc069b70 2024-01-24T14:17:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.826+0000 7fb1f17fa700 1 -- 172.21.15.29:0/3390220830 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f4005df0 con 0x7fb1fc069b70 2024-01-24T14:17:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.826+0000 7fb20387b700 1 -- 172.21.15.29:0/3390220830 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ec0092a0 con 0x7fb1fc069b70 2024-01-24T14:17:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.826+0000 7fb1f17fa700 1 -- 172.21.15.29:0/3390220830 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4007360 con 0x7fb1fc069b70 2024-01-24T14:17:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.826+0000 7fb20387b700 1 -- 172.21.15.29:0/3390220830 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1fc0ac310 con 0x7fb1fc069b70 2024-01-24T14:17:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.827+0000 7fb20387b700 1 -- 172.21.15.29:0/3390220830 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1fc0ac4a0 con 0x7fb1fc069b70 2024-01-24T14:17:04.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.827+0000 7fb1f17fa700 1 -- 172.21.15.29:0/3390220830 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 54) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fb1f4007710 con 0x7fb1fc069b70 2024-01-24T14:17:04.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.828+0000 7fb1f17fa700 1 --2- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1e005f390 0x7fb1e0061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:04.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.828+0000 7fb201617700 1 -- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1e005f390 msgr2=0x7fb1e0061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:04.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.828+0000 7fb1f17fa700 1 -- 172.21.15.29:0/3390220830 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7456+0+0 (secure 0 0 0) 0x7fb1f407aaa0 con 0x7fb1fc069b70 2024-01-24T14:17:04.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.828+0000 7fb201617700 1 --2- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1e005f390 0x7fb1e0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:04.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:04.832+0000 7fb1f17fa700 1 -- 172.21.15.29:0/3390220830 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb1f404b4a0 con 0x7fb1fc069b70 2024-01-24T14:17:04.838 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:04.837+0000 7fc883c12680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:17:04.995 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:04.994+0000 7fc883c12680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:17:05.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.019+0000 7fb20387b700 1 -- 172.21.15.29:0/3390220830 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1fc0c05b0 con 0x7fb1fc069b70 2024-01-24T14:17:05.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.020+0000 7fb1f17fa700 1 -- 172.21.15.29:0/3390220830 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb1f404eac0 con 0x7fb1fc069b70 2024-01-24T14:17:05.020 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T14:17:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.022+0000 7fb20387b700 1 -- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1e005f390 msgr2=0x7fb1e0061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.022+0000 7fb20387b700 1 --2- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1e005f390 0x7fb1e0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.022+0000 7fb20387b700 1 -- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc069b70 msgr2=0x7fb1fc134070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.022+0000 7fb20387b700 1 --2- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc069b70 0x7fb1fc134070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fb1f400c7e0 tx=0x7fb1f4007eb0).stop 2024-01-24T14:17:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.022+0000 7fb20387b700 1 -- 172.21.15.29:0/3390220830 shutdown_connections 2024-01-24T14:17:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.022+0000 7fb20387b700 1 --2- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1fc069b70 0x7fb1fc134070 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.022+0000 7fb20387b700 1 --2- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb1e005f390 0x7fb1e0061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.022+0000 7fb20387b700 1 --2- 172.21.15.29:0/3390220830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1fc066370 0x7fb1fc0c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.022+0000 7fb20387b700 1 --2- 172.21.15.29:0/3390220830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc0692a0 0x7fb1fc133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.022+0000 7fb20387b700 1 -- 172.21.15.29:0/3390220830 >> 172.21.15.29:0/3390220830 conn(0x7fb1fc064390 msgr2=0x7fb1fc0c5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.022+0000 7fb20387b700 1 -- 172.21.15.29:0/3390220830 shutdown_connections 2024-01-24T14:17:05.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.022+0000 7fb20387b700 1 -- 172.21.15.29:0/3390220830 wait complete. 2024-01-24T14:17:05.033 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:05.033+0000 7f44ddf45680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:17:05.036 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T14:17:05.070 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:05.069+0000 7fc883c12680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:17:05.138 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:05.136+0000 7fc883c12680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:17:05.158 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:05.157+0000 7f44ddf45680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:17:05.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.169+0000 7f4ffd295700 1 Processor -- start 2024-01-24T14:17:05.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.169+0000 7f4ffd295700 1 -- start start 2024-01-24T14:17:05.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.170+0000 7f4ffd295700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4ff8153f40 0x7f4ff80a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.170+0000 7f4ffd295700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ff8062ec0 0x7f4ff8065440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.170+0000 7f4ffd295700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff806a360 con 0x7f4ff80c8290 2024-01-24T14:17:05.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.170+0000 7f4ffd295700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff80bcef0 con 0x7f4ff8153f40 2024-01-24T14:17:05.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.170+0000 7f4ffd295700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff8069de0 con 0x7f4ff8062ec0 2024-01-24T14:17:05.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.170+0000 7f4ff6ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4ff8153f40 0x7f4ff80a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff6ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4ff8153f40 0x7f4ff80a6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49282/0 (socket says 172.21.15.29:49282) 2024-01-24T14:17:05.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff67fc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4ff80c8290 0x7f4ff80627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38516/0 (socket says 172.21.15.29:38516) 2024-01-24T14:17:05.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff6ffd700 1 -- 172.21.15.29:0/2364316805 learned_addr learned my addr 172.21.15.29:0/2364316805 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:05.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff77fe700 1 --2- 172.21.15.29:0/2364316805 >> v2:172.21.15.163:3300/0 conn(0x7f4ff8062ec0 0x7f4ff8065440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff6ffd700 1 -- 172.21.15.29:0/2364316805 >> v2:172.21.15.163:3300/0 conn(0x7f4ff8062ec0 msgr2=0x7f4ff8065440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff6ffd700 1 --2- 172.21.15.29:0/2364316805 >> v2:172.21.15.163:3300/0 conn(0x7f4ff8062ec0 0x7f4ff8065440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff6ffd700 1 -- 172.21.15.29:0/2364316805 >> v1:172.21.15.29:6789/0 conn(0x7f4ff80c8290 legacy=0x7f4ff80627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff6ffd700 1 -- 172.21.15.29:0/2364316805 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff806a080 con 0x7f4ff8153f40 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff5ffb700 1 -- 172.21.15.29:0/2364316805 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210241993 0 0) 0x7f4ff806a360 con 0x7f4ff80c8290 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff6ffd700 1 --2- 172.21.15.29:0/2364316805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ff8153f40 0x7f4ff80a6db0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f4ff80bce60 tx=0x7f4fe80090f0).ready entity=mon.0 client_cookie=9f1f116be4e7bbfb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff5ffb700 1 -- 172.21.15.29:0/2364316805 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800b520 con 0x7f4ff8153f40 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff5ffb700 1 -- 172.21.15.29:0/2364316805 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe800b680 con 0x7f4ff8153f40 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.171+0000 7f4ff5ffb700 1 -- 172.21.15.29:0/2364316805 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800ba20 con 0x7f4ff8153f40 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.172+0000 7f4ffd295700 1 -- 172.21.15.29:0/2364316805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ff8153f40 msgr2=0x7f4ff80a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.172+0000 7f4ffd295700 1 --2- 172.21.15.29:0/2364316805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ff8153f40 0x7f4ff80a6db0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f4ff80bce60 tx=0x7f4fe80090f0).stop 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.172+0000 7f4ffd295700 1 -- 172.21.15.29:0/2364316805 shutdown_connections 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.172+0000 7f4ffd295700 1 --2- 172.21.15.29:0/2364316805 >> v2:172.21.15.163:3300/0 conn(0x7f4ff8062ec0 0x7f4ff8065440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.172+0000 7f4ffd295700 1 --2- 172.21.15.29:0/2364316805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ff8153f40 0x7f4ff80a6db0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.172+0000 7f4ffd295700 1 -- 172.21.15.29:0/2364316805 >> 172.21.15.29:0/2364316805 conn(0x7f4ff8064390 msgr2=0x7f4ff80c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:05.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.172+0000 7f4ffd295700 1 -- 172.21.15.29:0/2364316805 shutdown_connections 2024-01-24T14:17:05.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.172+0000 7f4ffd295700 1 -- 172.21.15.29:0/2364316805 wait complete. 2024-01-24T14:17:05.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.173+0000 7f4ffd295700 1 Processor -- start 2024-01-24T14:17:05.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.174+0000 7f4ffd295700 1 -- start start 2024-01-24T14:17:05.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.174+0000 7f4ffd295700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ff8062ec0 0x7f4ff814edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.174+0000 7f4ffd295700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ff80c8290 0x7f4ff80a7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.174+0000 7f4ffd295700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ff8153f40 0x7f4ff80beb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.174+0000 7f4ffd295700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff80a8e30 con 0x7f4ff80c8290 2024-01-24T14:17:05.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.174+0000 7f4ffd295700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff80a92c0 con 0x7f4ff8062ec0 2024-01-24T14:17:05.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.174+0000 7f4ffd295700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff80a8220 con 0x7f4ff8153f40 2024-01-24T14:17:05.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.174+0000 7f4ff67fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ff80c8290 0x7f4ff80a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.174+0000 7f4ff67fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ff80c8290 0x7f4ff80a7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49286/0 (socket says 172.21.15.29:49286) 2024-01-24T14:17:05.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.174+0000 7f4ff67fc700 1 -- 172.21.15.29:0/1459268002 learned_addr learned my addr 172.21.15.29:0/1459268002 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:05.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.174+0000 7f4ff6ffd700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ff8062ec0 0x7f4ff814edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.175+0000 7f4ff77fe700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ff8153f40 0x7f4ff80beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.175+0000 7f4ff67fc700 1 -- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ff8153f40 msgr2=0x7f4ff80beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.175+0000 7f4ff67fc700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ff8153f40 0x7f4ff80beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.175+0000 7f4ff67fc700 1 -- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ff8062ec0 msgr2=0x7f4ff814edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.175+0000 7f4ff67fc700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ff8062ec0 0x7f4ff814edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.175+0000 7f4ff67fc700 1 -- 172.21.15.29:0/1459268002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe8009370 con 0x7f4ff80c8290 2024-01-24T14:17:05.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.175+0000 7f4ff67fc700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ff80c8290 0x7f4ff80a7840 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f4fe000c4e0 tx=0x7f4fe0007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:05.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.175+0000 7f4feeffd700 1 -- 172.21.15.29:0/1459268002 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe000ce50 con 0x7f4ff80c8290 2024-01-24T14:17:05.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.175+0000 7f4feeffd700 1 -- 172.21.15.29:0/1459268002 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe00072e0 con 0x7f4ff80c8290 2024-01-24T14:17:05.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.175+0000 7f4ffd295700 1 -- 172.21.15.29:0/1459268002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff80a8500 con 0x7f4ff80c8290 2024-01-24T14:17:05.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.175+0000 7f4feeffd700 1 -- 172.21.15.29:0/1459268002 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00056e0 con 0x7f4ff80c8290 2024-01-24T14:17:05.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.175+0000 7f4ffd295700 1 -- 172.21.15.29:0/1459268002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff80a8ac0 con 0x7f4ff80c8290 2024-01-24T14:17:05.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.176+0000 7f4ffd295700 1 -- 172.21.15.29:0/1459268002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff80a8fe0 con 0x7f4ff80c8290 2024-01-24T14:17:05.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.177+0000 7f4feeffd700 1 -- 172.21.15.29:0/1459268002 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 54) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f4fe0005ad0 con 0x7f4ff80c8290 2024-01-24T14:17:05.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.177+0000 7f4feeffd700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4fd805f390 0x7f4fd8061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.177+0000 7f4ff6ffd700 1 -- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4fd805f390 msgr2=0x7f4fd8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:05.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.177+0000 7f4feeffd700 1 -- 172.21.15.29:0/1459268002 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7456+0+0 (secure 0 0 0) 0x7f4fe007a2b0 con 0x7f4ff80c8290 2024-01-24T14:17:05.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.177+0000 7f4ff6ffd700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4fd805f390 0x7f4fd8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:05.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.182+0000 7f4feeffd700 1 -- 172.21.15.29:0/1459268002 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4ff80a8fe0 con 0x7f4ff80c8290 2024-01-24T14:17:05.341 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:05.340+0000 7f44ddf45680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:17:05.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.366+0000 7f4ffd295700 1 -- 172.21.15.29:0/1459268002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ff80475e0 con 0x7f4ff80c8290 2024-01-24T14:17:05.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.367+0000 7f4feeffd700 1 -- 172.21.15.29:0/1459268002 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fe0018020 con 0x7f4ff80c8290 2024-01-24T14:17:05.367 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T14:17:05.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.370+0000 7f4ffd295700 1 -- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4fd805f390 msgr2=0x7f4fd8061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:05.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.370+0000 7f4ffd295700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4fd805f390 0x7f4fd8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:05.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.370+0000 7f4ffd295700 1 -- 172.21.15.29:0/1459268002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ff80c8290 msgr2=0x7f4ff80a7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.370+0000 7f4ffd295700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ff80c8290 0x7f4ff80a7840 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f4fe000c4e0 tx=0x7f4fe0007c60).stop 2024-01-24T14:17:05.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.370+0000 7f4ffd295700 1 -- 172.21.15.29:0/1459268002 shutdown_connections 2024-01-24T14:17:05.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.370+0000 7f4ffd295700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ff8153f40 0x7f4ff80beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.370+0000 7f4ffd295700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ff8062ec0 0x7f4ff814edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.370+0000 7f4ffd295700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4fd805f390 0x7f4fd8061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:05.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.370+0000 7f4ffd295700 1 --2- 172.21.15.29:0/1459268002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ff80c8290 0x7f4ff80a7840 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.370+0000 7f4ffd295700 1 -- 172.21.15.29:0/1459268002 >> 172.21.15.29:0/1459268002 conn(0x7f4ff8064390 msgr2=0x7f4ff80c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:05.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.370+0000 7f4ffd295700 1 -- 172.21.15.29:0/1459268002 shutdown_connections 2024-01-24T14:17:05.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.370+0000 7f4ffd295700 1 -- 172.21.15.29:0/1459268002 wait complete. 2024-01-24T14:17:05.390 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:17:05.398 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:05.397+0000 7fc883c12680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:17:05.439 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:05.439+0000 7f44ddf45680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:17:05.518 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:05.517+0000 7f44ddf45680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:17:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.520+0000 7f8fccf0b700 1 Processor -- start 2024-01-24T14:17:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.521+0000 7f8fccf0b700 1 -- start start 2024-01-24T14:17:05.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fccf0b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8fc80638e0 0x7f8fc80aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fccf0b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc80aa6c0 con 0x7f8fc8154370 2024-01-24T14:17:05.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fccf0b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc8064240 con 0x7f8fc80c5410 2024-01-24T14:17:05.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fccf0b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc80cb5c0 con 0x7f8fc80638e0 2024-01-24T14:17:05.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fc659c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8fc8154370 0x7f8fc80ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38518/0 (socket says 172.21.15.29:38518) 2024-01-24T14:17:05.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fc659c700 1 -- 172.21.15.29:0/1869481030 learned_addr learned my addr 172.21.15.29:0/1869481030 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:05.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fc6d9d700 1 --2- 172.21.15.29:0/1869481030 >> v2:172.21.15.163:3300/0 conn(0x7f8fc80638e0 0x7f8fc80aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fc559a700 1 -- 172.21.15.29:0/1869481030 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890428195 0 0) 0x7f8fc80aa6c0 con 0x7f8fc8154370 2024-01-24T14:17:05.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fc559a700 1 -- 172.21.15.29:0/1869481030 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb00033f0 con 0x7f8fc8154370 2024-01-24T14:17:05.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fc559a700 1 -- 172.21.15.29:0/1869481030 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1575001133 0 0) 0x7f8fb00033f0 con 0x7f8fc8154370 2024-01-24T14:17:05.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fc6d9d700 1 -- 172.21.15.29:0/1869481030 >> v1:172.21.15.163:6789/0 conn(0x7f8fc80c5410 legacy=0x7f8fc80631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fc6d9d700 1 -- 172.21.15.29:0/1869481030 >> v1:172.21.15.29:6789/0 conn(0x7f8fc8154370 legacy=0x7f8fc80ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.522+0000 7f8fc6d9d700 1 -- 172.21.15.29:0/1869481030 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc80cb8a0 con 0x7f8fc80638e0 2024-01-24T14:17:05.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.523+0000 7f8fc6d9d700 1 --2- 172.21.15.29:0/1869481030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fc80638e0 0x7f8fc80aa0e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f8fc0009e60 tx=0x7f8fc0009780).ready entity=mon.1 client_cookie=2db7e4dd514632b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:05.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.523+0000 7f8fc559a700 1 -- 172.21.15.29:0/1869481030 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc000e070 con 0x7f8fc80638e0 2024-01-24T14:17:05.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.523+0000 7f8fc559a700 1 -- 172.21.15.29:0/1869481030 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc000b850 con 0x7f8fc80638e0 2024-01-24T14:17:05.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.523+0000 7f8fc559a700 1 -- 172.21.15.29:0/1869481030 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80015f0 con 0x7f8fc80638e0 2024-01-24T14:17:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.523+0000 7f8fccf0b700 1 -- 172.21.15.29:0/1869481030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fc80638e0 msgr2=0x7f8fc80aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.523+0000 7f8fccf0b700 1 --2- 172.21.15.29:0/1869481030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fc80638e0 0x7f8fc80aa0e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f8fc0009e60 tx=0x7f8fc0009780).stop 2024-01-24T14:17:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.524+0000 7f8fccf0b700 1 -- 172.21.15.29:0/1869481030 shutdown_connections 2024-01-24T14:17:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.524+0000 7f8fccf0b700 1 --2- 172.21.15.29:0/1869481030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fc80638e0 0x7f8fc80aa0e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.524+0000 7f8fccf0b700 1 -- 172.21.15.29:0/1869481030 >> 172.21.15.29:0/1869481030 conn(0x7f8fc805ec80 msgr2=0x7f8fc80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:05.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.524+0000 7f8fccf0b700 1 -- 172.21.15.29:0/1869481030 shutdown_connections 2024-01-24T14:17:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.524+0000 7f8fccf0b700 1 -- 172.21.15.29:0/1869481030 wait complete. 2024-01-24T14:17:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.525+0000 7f8fccf0b700 1 Processor -- start 2024-01-24T14:17:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.525+0000 7f8fccf0b700 1 -- start start 2024-01-24T14:17:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fccf0b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fc80638e0 0x7f8fc80c3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fccf0b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc80c5410 0x7f8fc8147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fccf0b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fc8154370 0x7f8fc80ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fccf0b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc80afc10 con 0x7f8fc80c5410 2024-01-24T14:17:05.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fccf0b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc80af000 con 0x7f8fc80638e0 2024-01-24T14:17:05.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fccf0b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc80af2e0 con 0x7f8fc8154370 2024-01-24T14:17:05.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fc5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc80c5410 0x7f8fc8147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fc5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc80c5410 0x7f8fc8147bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49300/0 (socket says 172.21.15.29:49300) 2024-01-24T14:17:05.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fc5d9b700 1 -- 172.21.15.29:0/3600412565 learned_addr learned my addr 172.21.15.29:0/3600412565 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:05.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fc659c700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fc80638e0 0x7f8fc80c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fc6d9d700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fc8154370 0x7f8fc80ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fc5d9b700 1 -- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fc8154370 msgr2=0x7f8fc80ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fc5d9b700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fc8154370 0x7f8fc80ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fc5d9b700 1 -- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fc80638e0 msgr2=0x7f8fc80c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fc5d9b700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fc80638e0 0x7f8fc80c3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fc5d9b700 1 -- 172.21.15.29:0/3600412565 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0009a20 con 0x7f8fc80c5410 2024-01-24T14:17:05.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.526+0000 7f8fc5d9b700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc80c5410 0x7f8fc8147bf0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f8fbc00c5e0 tx=0x7f8fbc007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:05.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.527+0000 7f8fb67fc700 1 -- 172.21.15.29:0/3600412565 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00f470 con 0x7f8fc80c5410 2024-01-24T14:17:05.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.527+0000 7f8fb67fc700 1 -- 172.21.15.29:0/3600412565 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc00f5d0 con 0x7f8fc80c5410 2024-01-24T14:17:05.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.527+0000 7f8fccf0b700 1 -- 172.21.15.29:0/3600412565 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc80af5c0 con 0x7f8fc80c5410 2024-01-24T14:17:05.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.527+0000 7f8fb67fc700 1 -- 172.21.15.29:0/3600412565 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00f950 con 0x7f8fc80c5410 2024-01-24T14:17:05.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.527+0000 7f8fccf0b700 1 -- 172.21.15.29:0/3600412565 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc81480e0 con 0x7f8fc80c5410 2024-01-24T14:17:05.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.528+0000 7f8fccf0b700 1 -- 172.21.15.29:0/3600412565 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc80affa0 con 0x7f8fc80c5410 2024-01-24T14:17:05.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.528+0000 7f8fb67fc700 1 -- 172.21.15.29:0/3600412565 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 54) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f8fbc00fd40 con 0x7f8fc80c5410 2024-01-24T14:17:05.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.529+0000 7f8fb67fc700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8fa805f360 0x7f8fa8061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.529+0000 7f8fc659c700 1 -- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8fa805f360 msgr2=0x7f8fa8061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:05.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.529+0000 7f8fb67fc700 1 -- 172.21.15.29:0/3600412565 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7456+0+0 (secure 0 0 0) 0x7f8fbc07a100 con 0x7f8fc80c5410 2024-01-24T14:17:05.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.529+0000 7f8fc659c700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8fa805f360 0x7f8fa8061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:05.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.533+0000 7f8fb67fc700 1 -- 172.21.15.29:0/3600412565 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8fbc047b00 con 0x7f8fc80c5410 2024-01-24T14:17:05.584 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:05.583+0000 7fc883c12680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:17:05.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.730+0000 7f8fc659c700 1 -- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8fa805f360 msgr2=0x7f8fa8061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:05.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.730+0000 7f8fc659c700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8fa805f360 0x7f8fa8061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:17:05.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.760+0000 7f8fccf0b700 1 -- 172.21.15.29:0/3600412565 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fc80afa60 con 0x7f8fc80c5410 2024-01-24T14:17:05.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.760+0000 7f8fb67fc700 1 -- 172.21.15.29:0/3600412565 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f8fc80affa0 con 0x7f8fc80c5410 2024-01-24T14:17:05.761 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:05.761 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1579003637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1579003637},{"type":"v1","addr":"172.21.15.163:6836","nonce":1579003637}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2823175689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2823175689},{"type":"v1","addr":"172.21.15.163:6837","nonce":2823175689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2560003609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2560003609},{"type":"v1","addr":"172.21.15.29:6834","nonce":2560003609}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:16:09.332377+0000","modified":"2024-01-24T14:16:12.347265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1458876872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1458876872},{"type":"v1","addr":"172.21.15.29:6835","nonce":1458876872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-24T14:17:05.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.762+0000 7f8fccf0b700 1 -- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8fa805f360 msgr2=0x7f8fa8061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:05.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.762+0000 7f8fccf0b700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8fa805f360 0x7f8fa8061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:05.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.762+0000 7f8fccf0b700 1 -- 172.21.15.29:0/3600412565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc80c5410 msgr2=0x7f8fc8147bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.762+0000 7f8fccf0b700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc80c5410 0x7f8fc8147bf0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f8fbc00c5e0 tx=0x7f8fbc007d80).stop 2024-01-24T14:17:05.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.762+0000 7f8fccf0b700 1 -- 172.21.15.29:0/3600412565 shutdown_connections 2024-01-24T14:17:05.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.762+0000 7f8fccf0b700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fc8154370 0x7f8fc80ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.762+0000 7f8fccf0b700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fc80638e0 0x7f8fc80c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.762+0000 7f8fccf0b700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8fa805f360 0x7f8fa8061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:05.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.762+0000 7f8fccf0b700 1 --2- 172.21.15.29:0/3600412565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fc80c5410 0x7f8fc8147bf0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.762+0000 7f8fccf0b700 1 -- 172.21.15.29:0/3600412565 >> 172.21.15.29:0/3600412565 conn(0x7f8fc805ec80 msgr2=0x7f8fc8153570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:05.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.762+0000 7f8fccf0b700 1 -- 172.21.15.29:0/3600412565 shutdown_connections 2024-01-24T14:17:05.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.762+0000 7f8fccf0b700 1 -- 172.21.15.29:0/3600412565 wait complete. 2024-01-24T14:17:05.765 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 59 2024-01-24T14:17:05.777 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:17:05.820 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:05.819+0000 7f44ddf45680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:17:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.908+0000 7f499b6da700 1 Processor -- start 2024-01-24T14:17:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.908+0000 7f499b6da700 1 -- start start 2024-01-24T14:17:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.909+0000 7f499b6da700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4994154370 0x7f49940c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.909+0000 7f499b6da700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4994154750 0x7f49940631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.909+0000 7f499b6da700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49940aa6c0 con 0x7f49940aa0e0 2024-01-24T14:17:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.909+0000 7f499b6da700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4994064240 con 0x7f4994154370 2024-01-24T14:17:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.909+0000 7f499b6da700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49940cb5c0 con 0x7f4994154750 2024-01-24T14:17:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.909+0000 7f4998c75700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f49940aa0e0 0x7f49940ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38524/0 (socket says 172.21.15.29:38524) 2024-01-24T14:17:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.909+0000 7f4998c75700 1 -- 172.21.15.29:0/1836787434 learned_addr learned my addr 172.21.15.29:0/1836787434 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.909+0000 7f4999476700 1 --2- 172.21.15.29:0/1836787434 >> v2:172.21.15.163:3300/0 conn(0x7f4994154370 0x7f49940c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.909+0000 7f4999c77700 1 --2- 172.21.15.29:0/1836787434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4994154750 0x7f49940631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.910+0000 7f498bfff700 1 -- 172.21.15.29:0/1836787434 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306528065 0 0) 0x7f49940aa6c0 con 0x7f49940aa0e0 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.910+0000 7f498bfff700 1 -- 172.21.15.29:0/1836787434 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49800033f0 con 0x7f49940aa0e0 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.910+0000 7f4999476700 1 -- 172.21.15.29:0/1836787434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4994154750 msgr2=0x7f49940631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.910+0000 7f4999476700 1 --2- 172.21.15.29:0/1836787434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4994154750 0x7f49940631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.910+0000 7f4999476700 1 -- 172.21.15.29:0/1836787434 >> v1:172.21.15.29:6789/0 conn(0x7f49940aa0e0 legacy=0x7f49940ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.910+0000 7f4999476700 1 -- 172.21.15.29:0/1836787434 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49940cb8a0 con 0x7f4994154370 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.910+0000 7f4999c77700 1 --2- 172.21.15.29:0/1836787434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4994154750 0x7f49940631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.910+0000 7f4999476700 1 --2- 172.21.15.29:0/1836787434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4994154370 0x7f49940c5410 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f4990006fd0 tx=0x7f4990006f40).ready entity=mon.1 client_cookie=9dfefb74a663fc6e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.910+0000 7f498bfff700 1 -- 172.21.15.29:0/1836787434 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990005880 con 0x7f4994154370 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.910+0000 7f498bfff700 1 -- 172.21.15.29:0/1836787434 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49900059e0 con 0x7f4994154370 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.910+0000 7f498bfff700 1 -- 172.21.15.29:0/1836787434 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990005d60 con 0x7f4994154370 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.911+0000 7f499b6da700 1 -- 172.21.15.29:0/1836787434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4994154370 msgr2=0x7f49940c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.911+0000 7f499b6da700 1 --2- 172.21.15.29:0/1836787434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4994154370 0x7f49940c5410 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f4990006fd0 tx=0x7f4990006f40).stop 2024-01-24T14:17:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.911+0000 7f499b6da700 1 -- 172.21.15.29:0/1836787434 shutdown_connections 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.911+0000 7f499b6da700 1 --2- 172.21.15.29:0/1836787434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4994154750 0x7f49940631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.911+0000 7f499b6da700 1 --2- 172.21.15.29:0/1836787434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4994154370 0x7f49940c5410 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.911+0000 7f499b6da700 1 -- 172.21.15.29:0/1836787434 >> 172.21.15.29:0/1836787434 conn(0x7f499405ec80 msgr2=0x7f49940cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.911+0000 7f499b6da700 1 -- 172.21.15.29:0/1836787434 shutdown_connections 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.911+0000 7f499b6da700 1 -- 172.21.15.29:0/1836787434 wait complete. 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.913+0000 7f499b6da700 1 Processor -- start 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.913+0000 7f499b6da700 1 -- start start 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f499b6da700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49940aa0e0 0x7f49940c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f499b6da700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4994154370 0x7f4994158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f499b6da700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4994154750 0x7f4994151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f499b6da700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49941458a0 con 0x7f4994154370 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f499b6da700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4994144c90 con 0x7f4994154750 2024-01-24T14:17:05.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f499b6da700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4994144f70 con 0x7f49940aa0e0 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f4998c75700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4994154370 0x7f4994158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f4998c75700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4994154370 0x7f4994158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49310/0 (socket says 172.21.15.29:49310) 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f4998c75700 1 -- 172.21.15.29:0/2776446868 learned_addr learned my addr 172.21.15.29:0/2776446868 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f4999c77700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4994154750 0x7f4994151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f4999476700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49940aa0e0 0x7f49940c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f4998c75700 1 -- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49940aa0e0 msgr2=0x7f49940c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f4998c75700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49940aa0e0 0x7f49940c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f4998c75700 1 -- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4994154750 msgr2=0x7f4994151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f4998c75700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4994154750 0x7f4994151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f4998c75700 1 -- 172.21.15.29:0/2776446868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c009a20 con 0x7f4994154370 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.914+0000 7f4999c77700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4994154750 0x7f4994151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.915+0000 7f4998c75700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4994154370 0x7f4994158b20 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f498400c590 tx=0x7f4984007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:05.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.915+0000 7f49897fa700 1 -- 172.21.15.29:0/2776446868 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49840072e0 con 0x7f4994154370 2024-01-24T14:17:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.915+0000 7f49897fa700 1 -- 172.21.15.29:0/2776446868 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4984007440 con 0x7f4994154370 2024-01-24T14:17:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.915+0000 7f499b6da700 1 -- 172.21.15.29:0/2776446868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4990009250 con 0x7f4994154370 2024-01-24T14:17:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.915+0000 7f49897fa700 1 -- 172.21.15.29:0/2776446868 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49840057a0 con 0x7f4994154370 2024-01-24T14:17:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.915+0000 7f499b6da700 1 -- 172.21.15.29:0/2776446868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4994145530 con 0x7f4994154370 2024-01-24T14:17:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.916+0000 7f499b6da700 1 -- 172.21.15.29:0/2776446868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49940475e0 con 0x7f4994154370 2024-01-24T14:17:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.916+0000 7f49897fa700 1 -- 172.21.15.29:0/2776446868 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 54) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f4984005bd0 con 0x7f4994154370 2024-01-24T14:17:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.917+0000 7f49897fa700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f497805f360 0x7f4978061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.917+0000 7f4999476700 1 -- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f497805f360 msgr2=0x7f4978061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.917+0000 7f49897fa700 1 -- 172.21.15.29:0/2776446868 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7456+0+0 (secure 0 0 0) 0x7f498407a380 con 0x7f4994154370 2024-01-24T14:17:05.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.917+0000 7f4999476700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f497805f360 0x7f4978061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:05.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:05.921+0000 7f49897fa700 1 -- 172.21.15.29:0/2776446868 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f498404ae20 con 0x7f4994154370 2024-01-24T14:17:06.038 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:06.037+0000 7f44ddf45680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:17:06.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.104+0000 7f499b6da700 1 -- 172.21.15.29:0/2776446868 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49940cbb80 con 0x7f4994154370 2024-01-24T14:17:06.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.105+0000 7f49897fa700 1 -- 172.21.15.29:0/2776446868 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+18668 (secure 0 0 0) 0x7f498404e440 con 0x7f4994154370 2024-01-24T14:17:06.106 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:06.106 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":139,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:17:03.941964+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:16:07.075847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-24T14:16:08.308597+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":139,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6833/43241":"2024-01-25T14:16:21.610007+0000","172.21.15.163:6832/43241":"2024-01-25T14:16:21.610007+0000","172.21.15.163:0/187931593":"2024-01-25T14:16:21.610007+0000","172.21.15.163:0/455194783":"2024-01-25T14:16:21.610007+0000","172.21.15.163:0/1937450853":"2024-01-25T14:16:21.610007+0000","172.21.15.163:0/3338508841":"2024-01-25T14:16:21.610007+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:17:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.107+0000 7f499b6da700 1 -- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f497805f360 msgr2=0x7f4978061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.107+0000 7f499b6da700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f497805f360 0x7f4978061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.107+0000 7f499b6da700 1 -- 172.21.15.29:0/2776446868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4994154370 msgr2=0x7f4994158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.107+0000 7f499b6da700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4994154370 0x7f4994158b20 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f498400c590 tx=0x7f4984007c80).stop 2024-01-24T14:17:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.108+0000 7f499b6da700 1 -- 172.21.15.29:0/2776446868 shutdown_connections 2024-01-24T14:17:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.108+0000 7f499b6da700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4994154750 0x7f4994151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.108+0000 7f499b6da700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49940aa0e0 0x7f49940c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.108+0000 7f499b6da700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f497805f360 0x7f4978061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.108+0000 7f499b6da700 1 --2- 172.21.15.29:0/2776446868 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4994154370 0x7f4994158b20 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.108+0000 7f499b6da700 1 -- 172.21.15.29:0/2776446868 >> 172.21.15.29:0/2776446868 conn(0x7f499405ec80 msgr2=0x7f49941534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.108+0000 7f499b6da700 1 -- 172.21.15.29:0/2776446868 shutdown_connections 2024-01-24T14:17:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.108+0000 7f499b6da700 1 -- 172.21.15.29:0/2776446868 wait complete. 2024-01-24T14:17:06.122 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T14:17:06.123 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:17:06.240 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:06.239+0000 7fc883c12680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:17:06.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.293+0000 7f4356a3a700 1 Processor -- start 2024-01-24T14:17:06.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.293+0000 7f4356a3a700 1 -- start start 2024-01-24T14:17:06.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.294+0000 7f4356a3a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f435014eb80 0x7f4350066210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.294+0000 7f4356a3a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4350151b10 0x7f43500c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.294+0000 7f4356a3a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435006a5a0 con 0x7f4350069240 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.294+0000 7f4356a3a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43500696c0 con 0x7f435014eb80 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.294+0000 7f4356a3a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4350069990 con 0x7f4350151b10 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.294+0000 7f434ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f435014eb80 0x7f4350066210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.294+0000 7f434f7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4350069240 0x7f4350151400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38526/0 (socket says 172.21.15.29:38526) 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.294+0000 7f434f7fe700 1 -- 172.21.15.29:0/276032926 learned_addr learned my addr 172.21.15.29:0/276032926 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.294+0000 7f4354fd7700 1 --2- 172.21.15.29:0/276032926 >> v2:172.21.15.163:3300/0 conn(0x7f4350151b10 0x7f43500c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.295+0000 7f434ffff700 1 -- 172.21.15.29:0/276032926 >> v2:172.21.15.163:3300/0 conn(0x7f4350151b10 msgr2=0x7f43500c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.295+0000 7f434ffff700 1 --2- 172.21.15.29:0/276032926 >> v2:172.21.15.163:3300/0 conn(0x7f4350151b10 0x7f43500c30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.295+0000 7f434ffff700 1 -- 172.21.15.29:0/276032926 >> v1:172.21.15.29:6789/0 conn(0x7f4350069240 legacy=0x7f4350151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.295+0000 7f434ffff700 1 -- 172.21.15.29:0/276032926 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4350069c70 con 0x7f435014eb80 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.295+0000 7f434effd700 1 -- 172.21.15.29:0/276032926 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183713350 0 0) 0x7f435006a5a0 con 0x7f4350069240 2024-01-24T14:17:06.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.295+0000 7f434ffff700 1 --2- 172.21.15.29:0/276032926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f435014eb80 0x7f4350066210 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f4350066950 tx=0x7f4340009060).ready entity=mon.0 client_cookie=49f6e58838d443c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:06.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.295+0000 7f434effd700 1 -- 172.21.15.29:0/276032926 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000f510 con 0x7f435014eb80 2024-01-24T14:17:06.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.295+0000 7f434effd700 1 -- 172.21.15.29:0/276032926 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434000f670 con 0x7f435014eb80 2024-01-24T14:17:06.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.295+0000 7f434effd700 1 -- 172.21.15.29:0/276032926 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000f9f0 con 0x7f435014eb80 2024-01-24T14:17:06.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.296+0000 7f4356a3a700 1 -- 172.21.15.29:0/276032926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f435014eb80 msgr2=0x7f4350066210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:06.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.296+0000 7f4356a3a700 1 --2- 172.21.15.29:0/276032926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f435014eb80 0x7f4350066210 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f4350066950 tx=0x7f4340009060).stop 2024-01-24T14:17:06.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.296+0000 7f4356a3a700 1 -- 172.21.15.29:0/276032926 shutdown_connections 2024-01-24T14:17:06.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.296+0000 7f4356a3a700 1 --2- 172.21.15.29:0/276032926 >> v2:172.21.15.163:3300/0 conn(0x7f4350151b10 0x7f43500c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.296+0000 7f4356a3a700 1 --2- 172.21.15.29:0/276032926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f435014eb80 0x7f4350066210 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.296+0000 7f4356a3a700 1 -- 172.21.15.29:0/276032926 >> 172.21.15.29:0/276032926 conn(0x7f43500c85c0 msgr2=0x7f43500c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:06.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.296+0000 7f4356a3a700 1 -- 172.21.15.29:0/276032926 shutdown_connections 2024-01-24T14:17:06.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.296+0000 7f4356a3a700 1 -- 172.21.15.29:0/276032926 wait complete. 2024-01-24T14:17:06.300 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.297+0000 7f4356a3a700 1 Processor -- start 2024-01-24T14:17:06.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.297+0000 7f4356a3a700 1 -- start start 2024-01-24T14:17:06.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.298+0000 7f4356a3a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4350069240 0x7f4350144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:06.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.298+0000 7f4356a3a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f435014eb80 0x7f4350152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:06.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.298+0000 7f4356a3a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4350151b10 0x7f4350153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:06.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.298+0000 7f4356a3a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43500c15d0 con 0x7f435014eb80 2024-01-24T14:17:06.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.298+0000 7f4356a3a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43500c06e0 con 0x7f4350069240 2024-01-24T14:17:06.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.298+0000 7f4356a3a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43500c09c0 con 0x7f4350151b10 2024-01-24T14:17:06.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.298+0000 7f434f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f435014eb80 0x7f4350152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:06.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.298+0000 7f434ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4350069240 0x7f4350144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:06.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.298+0000 7f434f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f435014eb80 0x7f4350152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49328/0 (socket says 172.21.15.29:49328) 2024-01-24T14:17:06.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.298+0000 7f434f7fe700 1 -- 172.21.15.29:0/2643686470 learned_addr learned my addr 172.21.15.29:0/2643686470 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:06.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.298+0000 7f4354fd7700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4350151b10 0x7f4350153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:06.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.299+0000 7f434ffff700 1 -- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4350151b10 msgr2=0x7f4350153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:06.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.299+0000 7f434ffff700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4350151b10 0x7f4350153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.299+0000 7f434ffff700 1 -- 172.21.15.29:0/2643686470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f435014eb80 msgr2=0x7f4350152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:06.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.299+0000 7f434ffff700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f435014eb80 0x7f4350152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.299+0000 7f434ffff700 1 -- 172.21.15.29:0/2643686470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43400092a0 con 0x7f4350069240 2024-01-24T14:17:06.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.299+0000 7f434ffff700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4350069240 0x7f4350144bf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f4340014000 tx=0x7f4340017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:06.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.299+0000 7f4337fff700 1 -- 172.21.15.29:0/2643686470 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340020070 con 0x7f4350069240 2024-01-24T14:17:06.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.299+0000 7f4337fff700 1 -- 172.21.15.29:0/2643686470 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4340000d10 con 0x7f4350069240 2024-01-24T14:17:06.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.299+0000 7f4356a3a700 1 -- 172.21.15.29:0/2643686470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43500c0ca0 con 0x7f4350069240 2024-01-24T14:17:06.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.299+0000 7f4337fff700 1 -- 172.21.15.29:0/2643686470 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000f9f0 con 0x7f4350069240 2024-01-24T14:17:06.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.299+0000 7f4356a3a700 1 -- 172.21.15.29:0/2643686470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43500c1260 con 0x7f4350069240 2024-01-24T14:17:06.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.300+0000 7f4337fff700 1 -- 172.21.15.29:0/2643686470 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 54) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f4340011400 con 0x7f4350069240 2024-01-24T14:17:06.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.300+0000 7f4356a3a700 1 -- 172.21.15.29:0/2643686470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43500475e0 con 0x7f4350069240 2024-01-24T14:17:06.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.301+0000 7f4337fff700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f433c05f390 0x7f433c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:06.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.301+0000 7f434f7fe700 1 -- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f433c05f390 msgr2=0x7f433c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:06.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.301+0000 7f4337fff700 1 -- 172.21.15.29:0/2643686470 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7456+0+0 (secure 0 0 0) 0x7f434007f960 con 0x7f4350069240 2024-01-24T14:17:06.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.301+0000 7f434f7fe700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f433c05f390 0x7f433c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:06.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.305+0000 7f4337fff700 1 -- 172.21.15.29:0/2643686470 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f43400503a0 con 0x7f4350069240 2024-01-24T14:17:06.310 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:06.309+0000 7fc883c12680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:17:06.387 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:06.386+0000 7fc883c12680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:17:06.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.502+0000 7f434f7fe700 1 -- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f433c05f390 msgr2=0x7f433c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:06.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.502+0000 7f434f7fe700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f433c05f390 0x7f433c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:17:06.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.531+0000 7f4356a3a700 1 -- 172.21.15.29:0/2643686470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43500aa5b0 con 0x7f4350069240 2024-01-24T14:17:06.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.532+0000 7f4337fff700 1 -- 172.21.15.29:0/2643686470 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f43400539c0 con 0x7f4350069240 2024-01-24T14:17:06.533 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:06.533 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1579003637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1579003637},{"type":"v1","addr":"172.21.15.163:6836","nonce":1579003637}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2823175689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2823175689},{"type":"v1","addr":"172.21.15.163:6837","nonce":2823175689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2560003609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2560003609},{"type":"v1","addr":"172.21.15.29:6834","nonce":2560003609}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:16:09.332377+0000","modified":"2024-01-24T14:16:12.347265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1458876872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1458876872},{"type":"v1","addr":"172.21.15.29:6835","nonce":1458876872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-24T14:17:06.535 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:06.533+0000 7fc883c12680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:17:06.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.534+0000 7f4356a3a700 1 -- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f433c05f390 msgr2=0x7f433c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:06.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.534+0000 7f4356a3a700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f433c05f390 0x7f433c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:06.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.534+0000 7f4356a3a700 1 -- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4350069240 msgr2=0x7f4350144bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:06.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.534+0000 7f4356a3a700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4350069240 0x7f4350144bf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f4340014000 tx=0x7f4340017000).stop 2024-01-24T14:17:06.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.534+0000 7f4356a3a700 1 -- 172.21.15.29:0/2643686470 shutdown_connections 2024-01-24T14:17:06.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.534+0000 7f4356a3a700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4350151b10 0x7f4350153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.534+0000 7f4356a3a700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4350069240 0x7f4350144bf0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.534+0000 7f4356a3a700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f433c05f390 0x7f433c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:06.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.534+0000 7f4356a3a700 1 --2- 172.21.15.29:0/2643686470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f435014eb80 0x7f4350152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.534+0000 7f4356a3a700 1 -- 172.21.15.29:0/2643686470 >> 172.21.15.29:0/2643686470 conn(0x7f43500c85c0 msgr2=0x7f43500c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:06.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.534+0000 7f4356a3a700 1 -- 172.21.15.29:0/2643686470 shutdown_connections 2024-01-24T14:17:06.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.534+0000 7f4356a3a700 1 -- 172.21.15.29:0/2643686470 wait complete. 2024-01-24T14:17:06.537 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 59 2024-01-24T14:17:06.549 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:17:06.606 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:06.605+0000 7fc883c12680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:17:06.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.683+0000 7f7c7a0d2700 1 Processor -- start 2024-01-24T14:17:06.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.683+0000 7f7c7a0d2700 1 -- start start 2024-01-24T14:17:06.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.684+0000 7f7c7a0d2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c74154370 0x7f7c740c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:06.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.684+0000 7f7c7a0d2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c74154750 0x7f7c740631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:06.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.684+0000 7f7c7a0d2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c740aa6c0 con 0x7f7c740aa0e0 2024-01-24T14:17:06.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.684+0000 7f7c7a0d2700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c74064240 con 0x7f7c74154750 2024-01-24T14:17:06.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.684+0000 7f7c7a0d2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740cb5c0 con 0x7f7c74154370 2024-01-24T14:17:06.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.684+0000 7f7c73fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c74154750 0x7f7c740631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:06.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.684+0000 7f7c73fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c74154750 0x7f7c740631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49332/0 (socket says 172.21.15.29:49332) 2024-01-24T14:17:06.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.684+0000 7f7c73fff700 1 -- 172.21.15.29:0/3180730049 learned_addr learned my addr 172.21.15.29:0/3180730049 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:06.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.685+0000 7f7c737fe700 1 --2- 172.21.15.29:0/3180730049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c74154370 0x7f7c740c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:06.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.685+0000 7f7c73fff700 1 -- 172.21.15.29:0/3180730049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c74154370 msgr2=0x7f7c740c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:06.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.685+0000 7f7c73fff700 1 --2- 172.21.15.29:0/3180730049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c74154370 0x7f7c740c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.685+0000 7f7c73fff700 1 -- 172.21.15.29:0/3180730049 >> v1:172.21.15.163:6789/0 conn(0x7f7c740aa0e0 legacy=0x7f7c740ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.685+0000 7f7c73fff700 1 -- 172.21.15.29:0/3180730049 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740cb8a0 con 0x7f7c74154750 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.685+0000 7f7c72ffd700 1 -- 172.21.15.29:0/3180730049 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299869550 0 0) 0x7f7c740aa6c0 con 0x7f7c740aa0e0 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.685+0000 7f7c73fff700 1 --2- 172.21.15.29:0/3180730049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c74154750 0x7f7c740631d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f7c6c009f20 tx=0x7f7c6c009840).ready entity=mon.0 client_cookie=70a01c15fca91c21 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.685+0000 7f7c72ffd700 1 -- 172.21.15.29:0/3180730049 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00e070 con 0x7f7c74154750 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.685+0000 7f7c72ffd700 1 -- 172.21.15.29:0/3180730049 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6c00b9d0 con 0x7f7c74154750 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.685+0000 7f7c72ffd700 1 -- 172.21.15.29:0/3180730049 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00bd50 con 0x7f7c74154750 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.686+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/3180730049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c74154750 msgr2=0x7f7c740631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.686+0000 7f7c7a0d2700 1 --2- 172.21.15.29:0/3180730049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c74154750 0x7f7c740631d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f7c6c009f20 tx=0x7f7c6c009840).stop 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.686+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/3180730049 shutdown_connections 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.686+0000 7f7c7a0d2700 1 --2- 172.21.15.29:0/3180730049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c74154370 0x7f7c740c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.686+0000 7f7c7a0d2700 1 --2- 172.21.15.29:0/3180730049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c74154750 0x7f7c740631d0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.686+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/3180730049 >> 172.21.15.29:0/3180730049 conn(0x7f7c7405ec80 msgr2=0x7f7c740cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:06.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.686+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/3180730049 shutdown_connections 2024-01-24T14:17:06.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.686+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/3180730049 wait complete. 2024-01-24T14:17:06.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.688+0000 7f7c7a0d2700 1 Processor -- start 2024-01-24T14:17:06.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.688+0000 7f7c7a0d2700 1 -- start start 2024-01-24T14:17:06.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c7a0d2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c740aa0e0 0x7f7c740c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:06.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c7a0d2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c74154370 0x7f7c740c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:06.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c7a0d2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c74154750 0x7f7c741418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:06.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c7a0d2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740ae560 con 0x7f7c740aa0e0 2024-01-24T14:17:06.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c7a0d2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740c2de0 con 0x7f7c74154750 2024-01-24T14:17:06.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c7a0d2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740ae820 con 0x7f7c74154370 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c737fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c740aa0e0 0x7f7c740c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c737fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c740aa0e0 0x7f7c740c3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49346/0 (socket says 172.21.15.29:49346) 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c737fe700 1 -- 172.21.15.29:0/2578766168 learned_addr learned my addr 172.21.15.29:0/2578766168 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c73fff700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c74154750 0x7f7c741418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c6bfff700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c74154370 0x7f7c740c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c73fff700 1 -- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c74154370 msgr2=0x7f7c740c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c73fff700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c74154370 0x7f7c740c1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c73fff700 1 -- 172.21.15.29:0/2578766168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c740aa0e0 msgr2=0x7f7c740c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c73fff700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c740aa0e0 0x7f7c740c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c73fff700 1 -- 172.21.15.29:0/2578766168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c009ae0 con 0x7f7c74154750 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.689+0000 7f7c737fe700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c740aa0e0 0x7f7c740c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.690+0000 7f7c73fff700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c74154750 0x7f7c741418c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f7c6c013000 tx=0x7f7c6c009840).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.690+0000 7f7c6b7fe700 1 -- 172.21.15.29:0/2578766168 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00bd50 con 0x7f7c74154750 2024-01-24T14:17:06.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.690+0000 7f7c6b7fe700 1 -- 172.21.15.29:0/2578766168 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6c00b640 con 0x7f7c74154750 2024-01-24T14:17:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.690+0000 7f7c6b7fe700 1 -- 172.21.15.29:0/2578766168 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c01c630 con 0x7f7c74154750 2024-01-24T14:17:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.690+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/2578766168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c740c3aa0 con 0x7f7c74154750 2024-01-24T14:17:06.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.690+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/2578766168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c740c2540 con 0x7f7c74154750 2024-01-24T14:17:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.691+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/2578766168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c74157f90 con 0x7f7c74154750 2024-01-24T14:17:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.691+0000 7f7c6b7fe700 1 -- 172.21.15.29:0/2578766168 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 54) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f7c6c003040 con 0x7f7c74154750 2024-01-24T14:17:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.692+0000 7f7c6b7fe700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c5405f390 0x7f7c54061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.692+0000 7f7c6b7fe700 1 -- 172.21.15.29:0/2578766168 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7456+0+0 (secure 0 0 0) 0x7f7c6c07b620 con 0x7f7c74154750 2024-01-24T14:17:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.692+0000 7f7c737fe700 1 -- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c5405f390 msgr2=0x7f7c54061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:06.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.692+0000 7f7c737fe700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c5405f390 0x7f7c54061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:06.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.696+0000 7f7c6b7fe700 1 -- 172.21.15.29:0/2578766168 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7c6c04bfb0 con 0x7f7c74154750 2024-01-24T14:17:06.722 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:06.721+0000 7fc883c12680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:17:06.812 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:06.811+0000 7f44ddf45680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:17:06.831 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:06.830+0000 7fc883c12680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:17:06.891 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:06.891+0000 7f44ddf45680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:17:06.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.893+0000 7f7c737fe700 1 -- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c5405f390 msgr2=0x7f7c54061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:06.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.893+0000 7f7c737fe700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c5405f390 0x7f7c54061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:17:06.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.925+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/2578766168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c740c2c40 con 0x7f7c74154750 2024-01-24T14:17:06.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.926+0000 7f7c6b7fe700 1 -- 172.21.15.29:0/2578766168 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f7c740c2c40 con 0x7f7c74154750 2024-01-24T14:17:06.926 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:06.927 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1579003637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1579003637},{"type":"v1","addr":"172.21.15.163:6836","nonce":1579003637}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2823175689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2823175689},{"type":"v1","addr":"172.21.15.163:6837","nonce":2823175689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2560003609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2560003609},{"type":"v1","addr":"172.21.15.29:6834","nonce":2560003609}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:16:09.332377+0000","modified":"2024-01-24T14:16:12.347265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1458876872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1458876872},{"type":"v1","addr":"172.21.15.29:6835","nonce":1458876872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-24T14:17:06.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.928+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c5405f390 msgr2=0x7f7c54061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:06.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.928+0000 7f7c7a0d2700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c5405f390 0x7f7c54061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:06.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.928+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c74154750 msgr2=0x7f7c741418c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:06.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.928+0000 7f7c7a0d2700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c74154750 0x7f7c741418c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f7c6c013000 tx=0x7f7c6c009840).stop 2024-01-24T14:17:06.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.928+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/2578766168 shutdown_connections 2024-01-24T14:17:06.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.928+0000 7f7c7a0d2700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c74154750 0x7f7c741418c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.928+0000 7f7c7a0d2700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c5405f390 0x7f7c54061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:06.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.928+0000 7f7c7a0d2700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c740aa0e0 0x7f7c740c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.928+0000 7f7c7a0d2700 1 --2- 172.21.15.29:0/2578766168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c74154370 0x7f7c740c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:06.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.928+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/2578766168 >> 172.21.15.29:0/2578766168 conn(0x7f7c7405ec80 msgr2=0x7f7c74151a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:06.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.928+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/2578766168 shutdown_connections 2024-01-24T14:17:06.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:06.928+0000 7f7c7a0d2700 1 -- 172.21.15.29:0/2578766168 wait complete. 2024-01-24T14:17:06.930 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 59 2024-01-24T14:17:06.943 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:17:06.982 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:06.981+0000 7f44ddf45680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:17:07.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.074+0000 7f791240d700 1 Processor -- start 2024-01-24T14:17:07.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.074+0000 7f791240d700 1 -- start start 2024-01-24T14:17:07.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f791240d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f790c13a5c0 0x7f790c063a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:07.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f791240d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f790c0be520 0x7f790c0664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f791240d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f790c0669e0 0x7f790c067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f791240d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790c0bf6a0 con 0x7f790c0be520 2024-01-24T14:17:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f791240d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790c066e50 con 0x7f790c13a5c0 2024-01-24T14:17:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f791240d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790c0bea90 con 0x7f790c0669e0 2024-01-24T14:17:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f790b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f790c0be520 0x7f790c0664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f79109aa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f790c0669e0 0x7f790c067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f790b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f790c0be520 0x7f790c0664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49362/0 (socket says 172.21.15.29:49362) 2024-01-24T14:17:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f790bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f790c13a5c0 0x7f790c063a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f790b7fe700 1 -- 172.21.15.29:0/1282093649 learned_addr learned my addr 172.21.15.29:0/1282093649 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f790bfff700 1 -- 172.21.15.29:0/1282093649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f790c0669e0 msgr2=0x7f790c067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f790bfff700 1 --2- 172.21.15.29:0/1282093649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f790c0669e0 0x7f790c067a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f790bfff700 1 -- 172.21.15.29:0/1282093649 >> v2:172.21.15.29:3300/0 conn(0x7f790c0be520 msgr2=0x7f790c0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f790bfff700 1 --2- 172.21.15.29:0/1282093649 >> v2:172.21.15.29:3300/0 conn(0x7f790c0be520 0x7f790c0664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f790bfff700 1 -- 172.21.15.29:0/1282093649 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c0bed70 con 0x7f790c13a5c0 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f790b7fe700 1 --2- 172.21.15.29:0/1282093649 >> v2:172.21.15.29:3300/0 conn(0x7f790c0be520 0x7f790c0664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f79109aa700 1 --2- 172.21.15.29:0/1282093649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f790c0669e0 0x7f790c067a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.075+0000 7f790bfff700 1 --2- 172.21.15.29:0/1282093649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f790c13a5c0 0x7f790c063a10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f790c069560 tx=0x7f78fc009060).ready entity=mon.1 client_cookie=4d5ca0b24302df67 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.076+0000 7f790affd700 1 -- 172.21.15.29:0/1282093649 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00d4a0 con 0x7f790c13a5c0 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.076+0000 7f790affd700 1 -- 172.21.15.29:0/1282093649 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc00d600 con 0x7f790c13a5c0 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.076+0000 7f790affd700 1 -- 172.21.15.29:0/1282093649 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00d980 con 0x7f790c13a5c0 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.076+0000 7f791240d700 1 -- 172.21.15.29:0/1282093649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f790c13a5c0 msgr2=0x7f790c063a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.076+0000 7f791240d700 1 --2- 172.21.15.29:0/1282093649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f790c13a5c0 0x7f790c063a10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f790c069560 tx=0x7f78fc009060).stop 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.076+0000 7f791240d700 1 -- 172.21.15.29:0/1282093649 shutdown_connections 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.076+0000 7f791240d700 1 --2- 172.21.15.29:0/1282093649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f790c0669e0 0x7f790c067a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.076+0000 7f791240d700 1 --2- 172.21.15.29:0/1282093649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f790c13a5c0 0x7f790c063a10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.076+0000 7f791240d700 1 --2- 172.21.15.29:0/1282093649 >> v2:172.21.15.29:3300/0 conn(0x7f790c0be520 0x7f790c0664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.076+0000 7f791240d700 1 -- 172.21.15.29:0/1282093649 >> 172.21.15.29:0/1282093649 conn(0x7f790c0c8bb0 msgr2=0x7f790c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:07.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.076+0000 7f791240d700 1 -- 172.21.15.29:0/1282093649 shutdown_connections 2024-01-24T14:17:07.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.076+0000 7f791240d700 1 -- 172.21.15.29:0/1282093649 wait complete. 2024-01-24T14:17:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.077+0000 7f791240d700 1 Processor -- start 2024-01-24T14:17:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.078+0000 7f791240d700 1 -- start start 2024-01-24T14:17:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.078+0000 7f791240d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f790c0669e0 0x7f790c1393f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:07.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f791240d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f790c0be520 0x7f790c0a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:07.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f791240d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f790c13a5c0 0x7f790c0a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:07.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f791240d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c0a76b0 con 0x7f790c0be520 2024-01-24T14:17:07.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f791240d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c154a10 con 0x7f790c13a5c0 2024-01-24T14:17:07.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f791240d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790c0a5710 con 0x7f790c0669e0 2024-01-24T14:17:07.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f790b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f790c0be520 0x7f790c0a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:07.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f790bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f790c0669e0 0x7f790c1393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:07.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f790b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f790c0be520 0x7f790c0a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49372/0 (socket says 172.21.15.29:49372) 2024-01-24T14:17:07.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f79109aa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f790c13a5c0 0x7f790c0a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:07.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f790b7fe700 1 -- 172.21.15.29:0/3012427809 learned_addr learned my addr 172.21.15.29:0/3012427809 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:07.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f790b7fe700 1 -- 172.21.15.29:0/3012427809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f790c0669e0 msgr2=0x7f790c1393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f790b7fe700 1 --2- 172.21.15.29:0/3012427809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f790c0669e0 0x7f790c1393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f790b7fe700 1 -- 172.21.15.29:0/3012427809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f790c13a5c0 msgr2=0x7f790c0a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f790b7fe700 1 --2- 172.21.15.29:0/3012427809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f790c13a5c0 0x7f790c0a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f790b7fe700 1 -- 172.21.15.29:0/3012427809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7904009a20 con 0x7f790c0be520 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f790b7fe700 1 --2- 172.21.15.29:0/3012427809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f790c0be520 0x7f790c0a4ba0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f7900006d00 tx=0x7f79000090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f78f3fff700 1 -- 172.21.15.29:0/3012427809 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790000c6a0 con 0x7f790c0be520 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f791240d700 1 -- 172.21.15.29:0/3012427809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7900009370 con 0x7f790c0be520 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.079+0000 7f78f3fff700 1 -- 172.21.15.29:0/3012427809 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790000c800 con 0x7f790c0be520 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.080+0000 7f78f3fff700 1 -- 172.21.15.29:0/3012427809 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790000cb80 con 0x7f790c0be520 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.080+0000 7f791240d700 1 -- 172.21.15.29:0/3012427809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc0092a0 con 0x7f790c0be520 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.081+0000 7f78f3fff700 1 -- 172.21.15.29:0/3012427809 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 54) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f790001b400 con 0x7f790c0be520 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.081+0000 7f791240d700 1 -- 172.21.15.29:0/3012427809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790c0475e0 con 0x7f790c0be520 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.081+0000 7f78f3fff700 1 --2- 172.21.15.29:0/3012427809 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f78f805f390 0x7f78f8061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:07.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.081+0000 7f790bfff700 1 -- 172.21.15.29:0/3012427809 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f78f805f390 msgr2=0x7f78f8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:07.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.081+0000 7f78f3fff700 1 -- 172.21.15.29:0/3012427809 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 7456+0+0 (secure 0 0 0) 0x7f790007b060 con 0x7f790c0be520 2024-01-24T14:17:07.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.081+0000 7f790bfff700 1 --2- 172.21.15.29:0/3012427809 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f78f805f390 0x7f78f8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:07.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.086+0000 7f78f3fff700 1 -- 172.21.15.29:0/3012427809 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f790004baa0 con 0x7f790c0be520 2024-01-24T14:17:07.125 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:07.124+0000 7fc883c12680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:17:07.179 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:07.178+0000 7f44ddf45680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:17:07.197 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:07.196+0000 7fc883c12680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:17:07.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.219+0000 7f78f3fff700 1 -- 172.21.15.29:0/3012427809 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 55) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f7900078600 con 0x7f790c0be520 2024-01-24T14:17:07.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.219+0000 7f78f3fff700 1 -- 172.21.15.29:0/3012427809 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f78f805f390 msgr2=0x7f78f8061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:07.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.219+0000 7f78f3fff700 1 --2- 172.21.15.29:0/3012427809 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f78f805f390 0x7f78f8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:07.267 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:07.266+0000 7f44ddf45680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:17:07.279 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:07.278+0000 7fc7fd193700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:07.279 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:07.278+0000 7fc7fd193700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:07.280 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:07.278+0000 7fc7fd193700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:07.280 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:07.278+0000 7fc7fd193700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:07.280 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:07.278+0000 7fc7fd193700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:07.281 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:07.279+0000 7fc7ff998700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:07.281 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:07.279+0000 7fc7ff998700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:07.281 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:07.279+0000 7fc7ff998700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:07.281 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:07.279+0000 7fc7ff998700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:07.281 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:07.279+0000 7fc7ff998700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:07.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.308+0000 7f791240d700 1 -- 172.21.15.29:0/3012427809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f790c0bf0c0 con 0x7f790c0be520 2024-01-24T14:17:07.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.309+0000 7f78f3fff700 1 -- 172.21.15.29:0/3012427809 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+19002 (secure 0 0 0) 0x7f790006d760 con 0x7f790c0be520 2024-01-24T14:17:07.310 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:07.310 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":140,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:17:07.211367+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:16:07.075847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-24T14:16:08.308597+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":139,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3198290428":"2024-01-25T14:17:07.211304+0000","172.21.15.163:0/4092356087":"2024-01-25T14:17:07.211304+0000","172.21.15.163:0/3145122938":"2024-01-25T14:17:07.211304+0000","172.21.15.163:0/4248625770":"2024-01-25T14:17:07.211304+0000","172.21.15.163:0/4124865118":"2024-01-25T14:17:07.211304+0000","172.21.15.163:0/2828921925":"2024-01-25T14:17:07.211304+0000","172.21.15.163:6833/43241":"2024-01-25T14:16:21.610007+0000","172.21.15.163:6832/43241":"2024-01-25T14:16:21.610007+0000","172.21.15.163:0/187931593":"2024-01-25T14:16:21.610007+0000","172.21.15.163:0/455194783":"2024-01-25T14:16:21.610007+0000","172.21.15.163:0/1937450853":"2024-01-25T14:16:21.610007+0000","172.21.15.163:0/3338508841":"2024-01-25T14:16:21.610007+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:17:07.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.311+0000 7f791240d700 1 -- 172.21.15.29:0/3012427809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f790c0be520 msgr2=0x7f790c0a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:07.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.311+0000 7f791240d700 1 --2- 172.21.15.29:0/3012427809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f790c0be520 0x7f790c0a4ba0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f7900006d00 tx=0x7f79000090f0).stop 2024-01-24T14:17:07.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.311+0000 7f791240d700 1 -- 172.21.15.29:0/3012427809 shutdown_connections 2024-01-24T14:17:07.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.311+0000 7f791240d700 1 --2- 172.21.15.29:0/3012427809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f790c13a5c0 0x7f790c0a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.311+0000 7f791240d700 1 --2- 172.21.15.29:0/3012427809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f790c0669e0 0x7f790c1393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.311+0000 7f791240d700 1 --2- 172.21.15.29:0/3012427809 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f78f805f390 0x7f78f8061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:07.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.311+0000 7f791240d700 1 --2- 172.21.15.29:0/3012427809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f790c0be520 0x7f790c0a4ba0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.311+0000 7f791240d700 1 -- 172.21.15.29:0/3012427809 >> 172.21.15.29:0/3012427809 conn(0x7f790c0c8bb0 msgr2=0x7f790c152c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:07.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.312+0000 7f791240d700 1 -- 172.21.15.29:0/3012427809 shutdown_connections 2024-01-24T14:17:07.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.312+0000 7f791240d700 1 -- 172.21.15.29:0/3012427809 wait complete. 2024-01-24T14:17:07.326 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-24T14:17:07.406 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:07.406+0000 7f44ddf45680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:17:07.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.498+0000 7f6884703700 1 Processor -- start 2024-01-24T14:17:07.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.498+0000 7f6884703700 1 -- start start 2024-01-24T14:17:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.499+0000 7f6884703700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f687c0a9d20 0x7f687c0aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.499+0000 7f6884703700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0bfd00 0x7f687c0c0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.499+0000 7f6884703700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c064e90 con 0x7f687c154240 2024-01-24T14:17:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.499+0000 7f6884703700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687c1550e0 con 0x7f687c0a9d20 2024-01-24T14:17:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.499+0000 7f6884703700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0aa670 con 0x7f687c0bfd00 2024-01-24T14:17:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.499+0000 7f6881c9e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f687c0a9d20 0x7f687c0aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.499+0000 7f6881c9e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f687c0a9d20 0x7f687c0aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49374/0 (socket says 172.21.15.29:49374) 2024-01-24T14:17:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.499+0000 7f6881c9e700 1 -- 172.21.15.29:0/2889410463 learned_addr learned my addr 172.21.15.29:0/2889410463 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.500+0000 7f6882ca0700 1 --2- 172.21.15.29:0/2889410463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0bfd00 0x7f687c0c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:07.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.500+0000 7f6881c9e700 1 -- 172.21.15.29:0/2889410463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0bfd00 msgr2=0x7f687c0c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.500+0000 7f6881c9e700 1 --2- 172.21.15.29:0/2889410463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0bfd00 0x7f687c0c0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.500+0000 7f6881c9e700 1 -- 172.21.15.29:0/2889410463 >> v1:172.21.15.163:6789/0 conn(0x7f687c154240 legacy=0x7f687c064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.500+0000 7f6881c9e700 1 -- 172.21.15.29:0/2889410463 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c0650d0 con 0x7f687c0a9d20 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.500+0000 7f6881c9e700 1 --2- 172.21.15.29:0/2889410463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0a9d20 0x7f687c0aa180 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f686c006fd0 tx=0x7f686c0090f0).ready entity=mon.0 client_cookie=e122c0375c6b6df7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.500+0000 7f688149d700 1 -- 172.21.15.29:0/2889410463 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c00b600 con 0x7f687c0a9d20 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.500+0000 7f688149d700 1 -- 172.21.15.29:0/2889410463 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686c00b760 con 0x7f687c0a9d20 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.500+0000 7f688149d700 1 -- 172.21.15.29:0/2889410463 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c00bae0 con 0x7f687c0a9d20 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.501+0000 7f6884703700 1 -- 172.21.15.29:0/2889410463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0a9d20 msgr2=0x7f687c0aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.501+0000 7f6884703700 1 --2- 172.21.15.29:0/2889410463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0a9d20 0x7f687c0aa180 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f686c006fd0 tx=0x7f686c0090f0).stop 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.501+0000 7f6884703700 1 -- 172.21.15.29:0/2889410463 shutdown_connections 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.501+0000 7f6884703700 1 --2- 172.21.15.29:0/2889410463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0bfd00 0x7f687c0c0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.501+0000 7f6884703700 1 --2- 172.21.15.29:0/2889410463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0a9d20 0x7f687c0aa180 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.501+0000 7f6884703700 1 -- 172.21.15.29:0/2889410463 >> 172.21.15.29:0/2889410463 conn(0x7f687c0c8310 msgr2=0x7f687c05e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:07.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.501+0000 7f6884703700 1 -- 172.21.15.29:0/2889410463 shutdown_connections 2024-01-24T14:17:07.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.501+0000 7f6884703700 1 -- 172.21.15.29:0/2889410463 wait complete. 2024-01-24T14:17:07.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.503+0000 7f6884703700 1 Processor -- start 2024-01-24T14:17:07.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.503+0000 7f6884703700 1 -- start start 2024-01-24T14:17:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.504+0000 7f6884703700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0a9d20 0x7f687c14fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.504+0000 7f6884703700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0bfd00 0x7f687c1501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.504+0000 7f6884703700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f687c154240 0x7f687c0c3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:07.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.504+0000 7f6884703700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0c3500 con 0x7f687c0a9d20 2024-01-24T14:17:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.504+0000 7f6884703700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c150770 con 0x7f687c154240 2024-01-24T14:17:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.504+0000 7f6884703700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c150a50 con 0x7f687c0bfd00 2024-01-24T14:17:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.504+0000 7f688249f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0a9d20 0x7f687c14fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f688249f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0a9d20 0x7f687c14fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49378/0 (socket says 172.21.15.29:49378) 2024-01-24T14:17:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f688249f700 1 -- 172.21.15.29:0/1201684357 learned_addr learned my addr 172.21.15.29:0/1201684357 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f6881c9e700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0bfd00 0x7f687c1501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f6882ca0700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f687c154240 0x7f687c0c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f688249f700 1 -- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0bfd00 msgr2=0x7f687c1501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f688249f700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0bfd00 0x7f687c1501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f688249f700 1 -- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f687c154240 msgr2=0x7f687c0c3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:07.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f688249f700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f687c154240 0x7f687c0c3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:07.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f688249f700 1 -- 172.21.15.29:0/1201684357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c009370 con 0x7f687c0a9d20 2024-01-24T14:17:07.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f688249f700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0a9d20 0x7f687c14fce0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f687800c440 tx=0x7f6878007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:07.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f68727fc700 1 -- 172.21.15.29:0/1201684357 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878007020 con 0x7f687c0a9d20 2024-01-24T14:17:07.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f68727fc700 1 -- 172.21.15.29:0/1201684357 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6878007180 con 0x7f687c0a9d20 2024-01-24T14:17:07.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f68727fc700 1 -- 172.21.15.29:0/1201684357 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878007520 con 0x7f687c0a9d20 2024-01-24T14:17:07.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f6884703700 1 -- 172.21.15.29:0/1201684357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687c0c2eb0 con 0x7f687c0a9d20 2024-01-24T14:17:07.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.505+0000 7f6884703700 1 -- 172.21.15.29:0/1201684357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c0b0c20 con 0x7f687c0a9d20 2024-01-24T14:17:07.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.506+0000 7f6884703700 1 -- 172.21.15.29:0/1201684357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687c0c37e0 con 0x7f687c0a9d20 2024-01-24T14:17:07.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.507+0000 7f68727fc700 1 -- 172.21.15.29:0/1201684357 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 55) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f68780068b0 con 0x7f687c0a9d20 2024-01-24T14:17:07.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.507+0000 7f68727fc700 1 -- 172.21.15.29:0/1201684357 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 7686+0+0 (secure 0 0 0) 0x7f68780797c0 con 0x7f687c0a9d20 2024-01-24T14:17:07.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.512+0000 7f68727fc700 1 -- 172.21.15.29:0/1201684357 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f6878047b00 con 0x7f687c0a9d20 2024-01-24T14:17:07.537 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:07.536+0000 7f44ddf45680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:17:07.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:07.740+0000 7f6884703700 1 -- 172.21.15.29:0/1201684357 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f687c064e90 con 0x7f687c0a9d20 2024-01-24T14:17:07.875 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:07.875+0000 7f44ddf45680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:17:07.960 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:07.959+0000 7f44ddf45680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:17:08.215 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-01-24T14:17:08.215+0000 7fe09bfc3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-24T14:17:08.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.220+0000 7f68727fc700 1 -- 172.21.15.29:0/1201684357 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 56) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f6878077030 con 0x7f687c0a9d20 2024-01-24T14:17:08.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.220+0000 7f68727fc700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f68600302f0 0x7f6860068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:08.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.223+0000 7f6881c9e700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f68600302f0 0x7f6860068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:08.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.224+0000 7f6881c9e700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f68600302f0 0x7f6860068a60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f686c009130 tx=0x7f686c017000).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:08.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.231+0000 7f68727fc700 1 -- 172.21.15.29:0/1201684357 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6878070de0 con 0x7f687c0a9d20 2024-01-24T14:17:08.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.234+0000 7f6884703700 1 -- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f68600302f0 msgr2=0x7f6860068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:08.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.234+0000 7f6884703700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f68600302f0 0x7f6860068a60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f686c009130 tx=0x7f686c017000).stop 2024-01-24T14:17:08.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.234+0000 7f6884703700 1 -- 172.21.15.29:0/1201684357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0a9d20 msgr2=0x7f687c14fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:08.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.234+0000 7f6884703700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0a9d20 0x7f687c14fce0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f687800c440 tx=0x7f6878007e00).stop 2024-01-24T14:17:08.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.234+0000 7f6884703700 1 -- 172.21.15.29:0/1201684357 shutdown_connections 2024-01-24T14:17:08.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.234+0000 7f6884703700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f687c154240 0x7f687c0c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:08.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.234+0000 7f6884703700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f68600302f0 0x7f6860068a60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:08.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.234+0000 7f6884703700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f687c0a9d20 0x7f687c14fce0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:08.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.234+0000 7f6884703700 1 --2- 172.21.15.29:0/1201684357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f687c0bfd00 0x7f687c1501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:08.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.234+0000 7f6884703700 1 -- 172.21.15.29:0/1201684357 >> 172.21.15.29:0/1201684357 conn(0x7f687c0c8310 msgr2=0x7f687c0aae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:08.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.234+0000 7f6884703700 1 -- 172.21.15.29:0/1201684357 shutdown_connections 2024-01-24T14:17:08.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.234+0000 7f6884703700 1 -- 172.21.15.29:0/1201684357 wait complete. 2024-01-24T14:17:08.236 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-24T14:17:08.237 INFO:tasks.ceph.mds.a.smithi029.stderr: -7524> 2024-01-24T14:16:14.884+0000 7f788d1d1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T14:17:08.250 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-24T14:17:08.268 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T14:17:08.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.401+0000 7f4484095700 1 Processor -- start 2024-01-24T14:17:08.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.402+0000 7f4484095700 1 -- start start 2024-01-24T14:17:08.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.402+0000 7f4484095700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f447c1550c0 0x7f447c0c3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:08.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.402+0000 7f4484095700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c155840 con 0x7f447c14b710 2024-01-24T14:17:08.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.402+0000 7f4484095700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c069280 con 0x7f447c14b330 2024-01-24T14:17:08.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.402+0000 7f4484095700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0c3910 con 0x7f447c1550c0 2024-01-24T14:17:08.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4481630700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f447c14b710 0x7f447c0c2c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38540/0 (socket says 172.21.15.29:38540) 2024-01-24T14:17:08.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4481630700 1 -- 172.21.15.29:0/773897535 learned_addr learned my addr 172.21.15.29:0/773897535 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:08.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4482632700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f447c1550c0 0x7f447c0c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572534632 0 0) 0x7f447c155840 con 0x7f447c14b710 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44680033f0 con 0x7f447c14b710 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491773071 0 0) 0x7f447c069280 con 0x7f447c14b330 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f447c155840 con 0x7f447c14b330 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2966774499 0 0) 0x7f44680033f0 con 0x7f447c14b710 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f447c1550c0 msgr2=0x7f447c0c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f447c1550c0 0x7f447c0c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 >> v1:172.21.15.163:6789/0 conn(0x7f447c14b330 legacy=0x7f447c1549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c0c3bf0 con 0x7f447c14b710 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f4478001490 con 0x7f447c14b710 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4482632700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f447c1550c0 0x7f447c0c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 >> v1:172.21.15.29:6789/0 conn(0x7f447c14b710 legacy=0x7f447c0c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:08.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4468003aa0 0x7f4468003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468004690 0x7f4468004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f447c1550c0 0x7f4468005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0c3910 con 0x7f4468003aa0 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446800b8e0 con 0x7f4468004690 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446800ba70 con 0x7f447c1550c0 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.403+0000 7f4481e31700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4468003aa0 0x7f4468003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.404+0000 7f4482632700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f447c1550c0 0x7f4468005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.404+0000 7f4481630700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468004690 0x7f4468004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.404+0000 7f4481e31700 1 -- 172.21.15.29:0/773897535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f447c1550c0 msgr2=0x7f4468005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.404+0000 7f4481e31700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f447c1550c0 0x7f4468005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.404+0000 7f4481e31700 1 -- 172.21.15.29:0/773897535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468004690 msgr2=0x7f4468004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.404+0000 7f4481e31700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468004690 0x7f4468004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:08.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.404+0000 7f4481e31700 1 -- 172.21.15.29:0/773897535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4474009a20 con 0x7f4468003aa0 2024-01-24T14:17:08.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.404+0000 7f4481e31700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4468003aa0 0x7f4468003ea0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f4470001890 tx=0x7f4470007bb0).ready entity=mon.0 client_cookie=2f9a6343436cf024 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:08.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.404+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470007020 con 0x7f4468003aa0 2024-01-24T14:17:08.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.404+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4470007180 con 0x7f4468003aa0 2024-01-24T14:17:08.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.404+0000 7f4480e2f700 1 -- 172.21.15.29:0/773897535 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470007500 con 0x7f4468003aa0 2024-01-24T14:17:08.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.405+0000 7f4484095700 1 -- 172.21.15.29:0/773897535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4468003aa0 msgr2=0x7f4468003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:08.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.405+0000 7f4484095700 1 --2- 172.21.15.29:0/773897535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4468003aa0 0x7f4468003ea0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f4470001890 tx=0x7f4470007bb0).stop 2024-01-24T14:17:08.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.405+0000 7f4482632700 1 -- 172.21.15.29:0/773897535 reap_dead start 2024-01-24T14:17:08.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.405+0000 7f4484095700 1 -- 172.21.15.29:0/773897535 shutdown_connections 2024-01-24T14:17:08.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.405+0000 7f4484095700 1 -- 172.21.15.29:0/773897535 >> 172.21.15.29:0/773897535 conn(0x7f447c064390 msgr2=0x7f447c0c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:08.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.405+0000 7f4484095700 1 -- 172.21.15.29:0/773897535 shutdown_connections 2024-01-24T14:17:08.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.405+0000 7f4484095700 1 -- 172.21.15.29:0/773897535 wait complete. 2024-01-24T14:17:08.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.406+0000 7f4484095700 1 Processor -- start 2024-01-24T14:17:08.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.406+0000 7f4484095700 1 -- start start 2024-01-24T14:17:08.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4484095700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4468003aa0 0x7f447c160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:08.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4484095700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f447c0ad2a0 0x7f447c0c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:08.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4484095700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f447c0ad720 0x7f447c0c3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:08.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4484095700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0c35f0 con 0x7f447c0ad720 2024-01-24T14:17:08.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4484095700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0bfc70 con 0x7f447c0ad2a0 2024-01-24T14:17:08.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4484095700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c163ac0 con 0x7f4468003aa0 2024-01-24T14:17:08.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4482632700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f447c0ad720 0x7f447c0c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4482632700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f447c0ad720 0x7f447c0c3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49430/0 (socket says 172.21.15.29:49430) 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4482632700 1 -- 172.21.15.29:0/3749474402 learned_addr learned my addr 172.21.15.29:0/3749474402 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4481e31700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4468003aa0 0x7f447c160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4481630700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f447c0ad2a0 0x7f447c0c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4482632700 1 -- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4468003aa0 msgr2=0x7f447c160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4482632700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4468003aa0 0x7f447c160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4482632700 1 -- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f447c0ad2a0 msgr2=0x7f447c0c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.407+0000 7f4482632700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f447c0ad2a0 0x7f447c0c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.408+0000 7f4482632700 1 -- 172.21.15.29:0/3749474402 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c163da0 con 0x7f447c0ad720 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.408+0000 7f4482632700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f447c0ad720 0x7f447c0c3100 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f44740094d0 tx=0x7f447400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.408+0000 7f446dffb700 1 -- 172.21.15.29:0/3749474402 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447400e040 con 0x7f447c0ad720 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.408+0000 7f446dffb700 1 -- 172.21.15.29:0/3749474402 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447400f040 con 0x7f447c0ad720 2024-01-24T14:17:08.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.408+0000 7f4484095700 1 -- 172.21.15.29:0/3749474402 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c164080 con 0x7f447c0ad720 2024-01-24T14:17:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.408+0000 7f446dffb700 1 -- 172.21.15.29:0/3749474402 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447400fd90 con 0x7f447c0ad720 2024-01-24T14:17:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.408+0000 7f4484095700 1 -- 172.21.15.29:0/3749474402 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c1610a0 con 0x7f447c0ad720 2024-01-24T14:17:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.409+0000 7f4484095700 1 -- 172.21.15.29:0/3749474402 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c0475e0 con 0x7f447c0ad720 2024-01-24T14:17:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.410+0000 7f446dffb700 1 -- 172.21.15.29:0/3749474402 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 56) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f4474015660 con 0x7f447c0ad720 2024-01-24T14:17:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.410+0000 7f446dffb700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f446005f030 0x7f44600614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.410+0000 7f446dffb700 1 -- 172.21.15.29:0/3749474402 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 7772+0+0 (secure 0 0 0) 0x7f447407b200 con 0x7f447c0ad720 2024-01-24T14:17:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.410+0000 7f4481e31700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f446005f030 0x7f44600614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:08.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.411+0000 7f4481e31700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f446005f030 0x7f44600614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f4470001670 tx=0x7f4470016000).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:08.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.414+0000 7f446dffb700 1 -- 172.21.15.29:0/3749474402 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f447404c030 con 0x7f447c0ad720 2024-01-24T14:17:08.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:08.643+0000 7f4484095700 1 -- 172.21.15.29:0/3749474402 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f447c0c8820 con 0x7f447c0ad720 2024-01-24T14:17:09.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.240+0000 7f446dffb700 1 -- 172.21.15.29:0/3749474402 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f447404f650 con 0x7f447c0ad720 2024-01-24T14:17:09.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.243+0000 7f4484095700 1 -- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f446005f030 msgr2=0x7f44600614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:09.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.243+0000 7f4484095700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f446005f030 0x7f44600614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f4470001670 tx=0x7f4470016000).stop 2024-01-24T14:17:09.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.243+0000 7f4484095700 1 -- 172.21.15.29:0/3749474402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f447c0ad720 msgr2=0x7f447c0c3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:09.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.243+0000 7f4484095700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f447c0ad720 0x7f447c0c3100 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f44740094d0 tx=0x7f447400b040).stop 2024-01-24T14:17:09.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.243+0000 7f4484095700 1 -- 172.21.15.29:0/3749474402 shutdown_connections 2024-01-24T14:17:09.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.243+0000 7f4484095700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f446005f030 0x7f44600614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:09.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.243+0000 7f4484095700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f447c0ad720 0x7f447c0c3100 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:09.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.243+0000 7f4484095700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4468003aa0 0x7f447c160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:09.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.243+0000 7f4484095700 1 --2- 172.21.15.29:0/3749474402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f447c0ad2a0 0x7f447c0c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:09.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.243+0000 7f4484095700 1 -- 172.21.15.29:0/3749474402 >> 172.21.15.29:0/3749474402 conn(0x7f447c064390 msgr2=0x7f447c0c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:09.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.243+0000 7f4484095700 1 -- 172.21.15.29:0/3749474402 shutdown_connections 2024-01-24T14:17:09.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.243+0000 7f4484095700 1 -- 172.21.15.29:0/3749474402 wait complete. 2024-01-24T14:17:09.259 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-24T14:17:09.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.399+0000 7f9d9fb37700 1 Processor -- start 2024-01-24T14:17:09.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.399+0000 7f9d9fb37700 1 -- start start 2024-01-24T14:17:09.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.400+0000 7f9d9fb37700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d98134e80 0x7f9d980c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:09.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.400+0000 7f9d9fb37700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9d981560e0 0x7f9d980bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:09.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.400+0000 7f9d9fb37700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d980c5b40 con 0x7f9d980c31b0 2024-01-24T14:17:09.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.400+0000 7f9d9fb37700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d980ab3a0 con 0x7f9d981560e0 2024-01-24T14:17:09.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.400+0000 7f9d9fb37700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d980c64c0 con 0x7f9d98134e80 2024-01-24T14:17:09.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.400+0000 7f9d9d8d3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d98134e80 0x7f9d980c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:09.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.400+0000 7f9d9e0d4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9d981560e0 0x7f9d980bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:09.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.400+0000 7f9d9d0d2700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f9d980c31b0 0x7f9d981559d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:59264/0 (socket says 172.21.15.29:59264) 2024-01-24T14:17:09.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.400+0000 7f9d9d0d2700 1 -- 172.21.15.29:0/3998819700 learned_addr learned my addr 172.21.15.29:0/3998819700 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:09.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.401+0000 7f9d9c8d1700 1 -- 172.21.15.29:0/3998819700 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984628888 0 0) 0x7f9d980c5b40 con 0x7f9d980c31b0 2024-01-24T14:17:09.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.401+0000 7f9d9c8d1700 1 -- 172.21.15.29:0/3998819700 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d840033f0 con 0x7f9d980c31b0 2024-01-24T14:17:09.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.401+0000 7f9d9d8d3700 1 -- 172.21.15.29:0/3998819700 >> v2:172.21.15.163:3300/0 conn(0x7f9d981560e0 msgr2=0x7f9d980bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:09.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.401+0000 7f9d9d8d3700 1 --2- 172.21.15.29:0/3998819700 >> v2:172.21.15.163:3300/0 conn(0x7f9d981560e0 0x7f9d980bdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:09.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.401+0000 7f9d9d8d3700 1 -- 172.21.15.29:0/3998819700 >> v1:172.21.15.163:6789/0 conn(0x7f9d980c31b0 legacy=0x7f9d981559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:09.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.401+0000 7f9d9d8d3700 1 -- 172.21.15.29:0/3998819700 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d980c4f30 con 0x7f9d98134e80 2024-01-24T14:17:09.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.401+0000 7f9d9e0d4700 1 --2- 172.21.15.29:0/3998819700 >> v2:172.21.15.163:3300/0 conn(0x7f9d981560e0 0x7f9d980bdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:09.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.401+0000 7f9d9d8d3700 1 --2- 172.21.15.29:0/3998819700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d98134e80 0x7f9d980c5fd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f9d94009760 tx=0x7f9d94009060).ready entity=mon.2 client_cookie=7ff199a61d78691d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:09.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.401+0000 7f9d9c8d1700 1 -- 172.21.15.29:0/3998819700 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9400d4a0 con 0x7f9d98134e80 2024-01-24T14:17:09.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.401+0000 7f9d9c8d1700 1 -- 172.21.15.29:0/3998819700 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9400d600 con 0x7f9d98134e80 2024-01-24T14:17:09.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.401+0000 7f9d9c8d1700 1 -- 172.21.15.29:0/3998819700 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9400d980 con 0x7f9d98134e80 2024-01-24T14:17:09.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.402+0000 7f9d9fb37700 1 -- 172.21.15.29:0/3998819700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d98134e80 msgr2=0x7f9d980c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:09.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.402+0000 7f9d9fb37700 1 --2- 172.21.15.29:0/3998819700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d98134e80 0x7f9d980c5fd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f9d94009760 tx=0x7f9d94009060).stop 2024-01-24T14:17:09.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.402+0000 7f9d9fb37700 1 -- 172.21.15.29:0/3998819700 shutdown_connections 2024-01-24T14:17:09.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.402+0000 7f9d9fb37700 1 --2- 172.21.15.29:0/3998819700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d98134e80 0x7f9d980c5fd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:09.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.402+0000 7f9d9fb37700 1 --2- 172.21.15.29:0/3998819700 >> v2:172.21.15.163:3300/0 conn(0x7f9d981560e0 0x7f9d980bdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:09.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.402+0000 7f9d9fb37700 1 -- 172.21.15.29:0/3998819700 >> 172.21.15.29:0/3998819700 conn(0x7f9d9805e1f0 msgr2=0x7f9d9805e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:09.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.402+0000 7f9d9fb37700 1 -- 172.21.15.29:0/3998819700 shutdown_connections 2024-01-24T14:17:09.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.402+0000 7f9d9fb37700 1 -- 172.21.15.29:0/3998819700 wait complete. 2024-01-24T14:17:09.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.403+0000 7f9d9fb37700 1 Processor -- start 2024-01-24T14:17:09.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.403+0000 7f9d9fb37700 1 -- start start 2024-01-24T14:17:09.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9fb37700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d980c31b0 0x7f9d981340a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9fb37700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d98134e80 0x7f9d980638c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9fb37700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d981560e0 0x7f9d98135590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9fb37700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d98151700 con 0x7f9d980c31b0 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9fb37700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d980642a0 con 0x7f9d98134e80 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9fb37700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d98064580 con 0x7f9d981560e0 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9d8d3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d980c31b0 0x7f9d981340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9d8d3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d980c31b0 0x7f9d981340a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49432/0 (socket says 172.21.15.29:49432) 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9d8d3700 1 -- 172.21.15.29:0/2758902694 learned_addr learned my addr 172.21.15.29:0/2758902694 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9e0d4700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d981560e0 0x7f9d98135590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9d0d2700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d98134e80 0x7f9d980638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9d8d3700 1 -- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d981560e0 msgr2=0x7f9d98135590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9d8d3700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d981560e0 0x7f9d98135590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9d8d3700 1 -- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d98134e80 msgr2=0x7f9d980638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9d8d3700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d98134e80 0x7f9d980638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.404+0000 7f9d9d8d3700 1 -- 172.21.15.29:0/2758902694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d90009a20 con 0x7f9d980c31b0 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.405+0000 7f9d9d8d3700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d980c31b0 0x7f9d981340a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7f9d980c6600 tx=0x7f9d94016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.405+0000 7f9d8dffb700 1 -- 172.21.15.29:0/2758902694 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94000c90 con 0x7f9d980c31b0 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.405+0000 7f9d8dffb700 1 -- 172.21.15.29:0/2758902694 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9400d770 con 0x7f9d980c31b0 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.405+0000 7f9d9fb37700 1 -- 172.21.15.29:0/2758902694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d940092a0 con 0x7f9d980c31b0 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.405+0000 7f9d8dffb700 1 -- 172.21.15.29:0/2758902694 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9401d430 con 0x7f9d980c31b0 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.405+0000 7f9d9fb37700 1 -- 172.21.15.29:0/2758902694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d98151390 con 0x7f9d980c31b0 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.406+0000 7f9d9fb37700 1 -- 172.21.15.29:0/2758902694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d980475e0 con 0x7f9d980c31b0 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.406+0000 7f9d8dffb700 1 -- 172.21.15.29:0/2758902694 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 56) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f9d9401d820 con 0x7f9d980c31b0 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.407+0000 7f9d8dffb700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9d7c05f030 0x7f9d7c0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.407+0000 7f9d8dffb700 1 -- 172.21.15.29:0/2758902694 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 7772+0+0 (secure 0 0 0) 0x7f9d9407bfe0 con 0x7f9d980c31b0 2024-01-24T14:17:09.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.407+0000 7f9d9d0d2700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9d7c05f030 0x7f9d7c0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:09.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.407+0000 7f9d9d0d2700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9d7c05f030 0x7f9d7c0614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9d88007fd0 tx=0x7f9d88009040).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:09.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.411+0000 7f9d8dffb700 1 -- 172.21.15.29:0/2758902694 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9d9404cce0 con 0x7f9d980c31b0 2024-01-24T14:17:09.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.592+0000 7f9d9fb37700 1 -- 172.21.15.29:0/2758902694 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9d980c5210 con 0x7f9d980c31b0 2024-01-24T14:17:09.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:09.963+0000 7f9d8dffb700 1 -- 172.21.15.29:0/2758902694 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9d94079840 con 0x7f9d980c31b0 2024-01-24T14:17:10.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.242+0000 7f9d8dffb700 1 -- 172.21.15.29:0/2758902694 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v143) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9d94042a00 con 0x7f9d980c31b0 2024-01-24T14:17:10.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.246+0000 7f9d9fb37700 1 -- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9d7c05f030 msgr2=0x7f9d7c0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:10.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.246+0000 7f9d9fb37700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9d7c05f030 0x7f9d7c0614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9d88007fd0 tx=0x7f9d88009040).stop 2024-01-24T14:17:10.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.246+0000 7f9d9fb37700 1 -- 172.21.15.29:0/2758902694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d980c31b0 msgr2=0x7f9d981340a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:10.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.246+0000 7f9d9fb37700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d980c31b0 0x7f9d981340a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7f9d980c6600 tx=0x7f9d94016000).stop 2024-01-24T14:17:10.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.246+0000 7f9d9fb37700 1 -- 172.21.15.29:0/2758902694 shutdown_connections 2024-01-24T14:17:10.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.246+0000 7f9d9fb37700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d981560e0 0x7f9d98135590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.246+0000 7f9d9fb37700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9d7c05f030 0x7f9d7c0614f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.246+0000 7f9d9fb37700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d980c31b0 0x7f9d981340a0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.246+0000 7f9d9fb37700 1 --2- 172.21.15.29:0/2758902694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d98134e80 0x7f9d980638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.246+0000 7f9d9fb37700 1 -- 172.21.15.29:0/2758902694 >> 172.21.15.29:0/2758902694 conn(0x7f9d9805e1f0 msgr2=0x7f9d980c70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:10.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.246+0000 7f9d9fb37700 1 -- 172.21.15.29:0/2758902694 shutdown_connections 2024-01-24T14:17:10.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.246+0000 7f9d9fb37700 1 -- 172.21.15.29:0/2758902694 wait complete. 2024-01-24T14:17:10.250 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-01-24T14:17:10.263 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-24T14:17:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.401+0000 7fcbb75c4700 1 Processor -- start 2024-01-24T14:17:10.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.401+0000 7fcbb75c4700 1 -- start start 2024-01-24T14:17:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.402+0000 7fcbb75c4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb0152540 0x7fcbb01529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:10.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.402+0000 7fcbb75c4700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb0153300 con 0x7fcbb0159da0 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.402+0000 7fcbb75c4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb0067b10 con 0x7fcbb0066450 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.402+0000 7fcbb75c4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0067df0 con 0x7fcbb0152540 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.402+0000 7fcbb5b61700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcbb0159da0 0x7fcbb015c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38544/0 (socket says 172.21.15.29:38544) 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.402+0000 7fcbb5b61700 1 -- 172.21.15.29:0/228039430 learned_addr learned my addr 172.21.15.29:0/228039430 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.402+0000 7fcbb4b5f700 1 --2- 172.21.15.29:0/228039430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb0152540 0x7fcbb01529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923619575 0 0) 0x7fcbb0153300 con 0x7fcbb0159da0 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb9c0033f0 con 0x7fcbb0159da0 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3904869953 0 0) 0x7fcb9c0033f0 con 0x7fcbb0159da0 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb0152540 msgr2=0x7fcbb01529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 --2- 172.21.15.29:0/228039430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb0152540 0x7fcbb01529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 >> v1:172.21.15.163:6789/0 conn(0x7fcbb0066450 legacy=0x7fcbb0066830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb00680d0 con 0x7fcbb0159da0 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fcba80017f0 con 0x7fcbb0159da0 2024-01-24T14:17:10.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 >> v1:172.21.15.29:6789/0 conn(0x7fcbb0159da0 legacy=0x7fcbb015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 --2- 172.21.15.29:0/228039430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb9c003910 0x7fcb9c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 --2- 172.21.15.29:0/228039430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb0152540 0x7fcb9c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 --2- 172.21.15.29:0/228039430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb9c004b80 0x7fcb9c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0067df0 con 0x7fcb9c003910 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c00b910 con 0x7fcb9c004b80 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c00bad0 con 0x7fcbb0152540 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcbb5360700 1 --2- 172.21.15.29:0/228039430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb9c003910 0x7fcb9c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcbb4b5f700 1 --2- 172.21.15.29:0/228039430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb0152540 0x7fcb9c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcbb5b61700 1 --2- 172.21.15.29:0/228039430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb9c004b80 0x7fcb9c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcbb5360700 1 -- 172.21.15.29:0/228039430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb0152540 msgr2=0x7fcb9c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcbb5360700 1 --2- 172.21.15.29:0/228039430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb0152540 0x7fcb9c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcbb5360700 1 -- 172.21.15.29:0/228039430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb9c004b80 msgr2=0x7fcb9c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcbb5360700 1 --2- 172.21.15.29:0/228039430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb9c004b80 0x7fcb9c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.403+0000 7fcbb5360700 1 -- 172.21.15.29:0/228039430 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c0033f0 con 0x7fcb9c003910 2024-01-24T14:17:10.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.404+0000 7fcbb5360700 1 --2- 172.21.15.29:0/228039430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb9c003910 0x7fcb9c003d30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7fcba4001730 tx=0x7fcba4007bb0).ready entity=mon.0 client_cookie=2efd6961422dcc89 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:10.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.404+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4007020 con 0x7fcb9c003910 2024-01-24T14:17:10.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.404+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba4007180 con 0x7fcb9c003910 2024-01-24T14:17:10.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.404+0000 7fcba3fff700 1 -- 172.21.15.29:0/228039430 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4007500 con 0x7fcb9c003910 2024-01-24T14:17:10.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.404+0000 7fcbb75c4700 1 -- 172.21.15.29:0/228039430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb9c003910 msgr2=0x7fcb9c003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:10.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.404+0000 7fcbb75c4700 1 --2- 172.21.15.29:0/228039430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb9c003910 0x7fcb9c003d30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7fcba4001730 tx=0x7fcba4007bb0).stop 2024-01-24T14:17:10.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.404+0000 7fcbb5b61700 1 -- 172.21.15.29:0/228039430 reap_dead start 2024-01-24T14:17:10.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.404+0000 7fcbb75c4700 1 -- 172.21.15.29:0/228039430 shutdown_connections 2024-01-24T14:17:10.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.404+0000 7fcbb75c4700 1 -- 172.21.15.29:0/228039430 >> 172.21.15.29:0/228039430 conn(0x7fcbb00c8bd0 msgr2=0x7fcbb00c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:10.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.404+0000 7fcbb75c4700 1 -- 172.21.15.29:0/228039430 shutdown_connections 2024-01-24T14:17:10.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.404+0000 7fcbb75c4700 1 -- 172.21.15.29:0/228039430 wait complete. 2024-01-24T14:17:10.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.406+0000 7fcbb75c4700 1 Processor -- start 2024-01-24T14:17:10.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.406+0000 7fcbb75c4700 1 -- start start 2024-01-24T14:17:10.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb75c4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb9c003910 0x7fcbb01540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:10.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb75c4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbb01358b0 0x7fcbb0154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb75c4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb014ba50 0x7fcbb014ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb75c4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0155400 con 0x7fcb9c003910 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb75c4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0157520 con 0x7fcbb01358b0 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb75c4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0155640 con 0x7fcbb014ba50 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb5360700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb9c003910 0x7fcbb01540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb5360700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb9c003910 0x7fcbb01540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49446/0 (socket says 172.21.15.29:49446) 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb5360700 1 -- 172.21.15.29:0/3054298368 learned_addr learned my addr 172.21.15.29:0/3054298368 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb5b61700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb014ba50 0x7fcbb014ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb4b5f700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbb01358b0 0x7fcbb0154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb5360700 1 -- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb014ba50 msgr2=0x7fcbb014ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb5360700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb014ba50 0x7fcbb014ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb5360700 1 -- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbb01358b0 msgr2=0x7fcbb0154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:10.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb5360700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbb01358b0 0x7fcbb0154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb5360700 1 -- 172.21.15.29:0/3054298368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb0156bf0 con 0x7fcb9c003910 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb5b61700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb014ba50 0x7fcbb014ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb4b5f700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbb01358b0 0x7fcbb0154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb5360700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb9c003910 0x7fcbb01540d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7fcb9c00b8e0 tx=0x7fcba4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcba17fa700 1 -- 172.21.15.29:0/3054298368 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4013670 con 0x7fcb9c003910 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcba17fa700 1 -- 172.21.15.29:0/3054298368 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba40137d0 con 0x7fcb9c003910 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcba17fa700 1 -- 172.21.15.29:0/3054298368 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4013b50 con 0x7fcb9c003910 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.407+0000 7fcbb75c4700 1 -- 172.21.15.29:0/3054298368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb0156ed0 con 0x7fcb9c003910 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.408+0000 7fcbb75c4700 1 -- 172.21.15.29:0/3054298368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb014bed0 con 0x7fcb9c003910 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.409+0000 7fcbb75c4700 1 -- 172.21.15.29:0/3054298368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb00660a0 con 0x7fcb9c003910 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.409+0000 7fcba17fa700 1 -- 172.21.15.29:0/3054298368 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcba401f400 con 0x7fcb9c003910 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.409+0000 7fcba17fa700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcb9405f270 0x7fcb94061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.409+0000 7fcba17fa700 1 -- 172.21.15.29:0/3054298368 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 7334+0+0 (secure 0 0 0) 0x7fcba407b3b0 con 0x7fcb9c003910 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.410+0000 7fcbb4b5f700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcb9405f270 0x7fcb94061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:10.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.410+0000 7fcbb4b5f700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcb9405f270 0x7fcb94061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fcbac006c40 tx=0x7fcbac008040).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:10.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.414+0000 7fcba17fa700 1 -- 172.21.15.29:0/3054298368 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcbb00660a0 con 0x7fcb9c003910 2024-01-24T14:17:10.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.600+0000 7fcbb75c4700 1 -- 172.21.15.29:0/3054298368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcbb00660a0 con 0x7fcb9c003910 2024-01-24T14:17:10.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.873+0000 7fcba17fa700 1 -- 172.21.15.29:0/3054298368 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v144) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcbb00660a0 con 0x7fcb9c003910 2024-01-24T14:17:10.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.877+0000 7fcbb75c4700 1 -- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcb9405f270 msgr2=0x7fcb94061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:10.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.877+0000 7fcbb75c4700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcb9405f270 0x7fcb94061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fcbac006c40 tx=0x7fcbac008040).stop 2024-01-24T14:17:10.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.877+0000 7fcbb75c4700 1 -- 172.21.15.29:0/3054298368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb9c003910 msgr2=0x7fcbb01540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:10.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.877+0000 7fcbb75c4700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb9c003910 0x7fcbb01540d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7fcb9c00b8e0 tx=0x7fcba4015000).stop 2024-01-24T14:17:10.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.877+0000 7fcbb75c4700 1 -- 172.21.15.29:0/3054298368 shutdown_connections 2024-01-24T14:17:10.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.877+0000 7fcbb75c4700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb014ba50 0x7fcbb014ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.877+0000 7fcbb75c4700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcb9405f270 0x7fcb94061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.877+0000 7fcbb75c4700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb9c003910 0x7fcbb01540d0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.877+0000 7fcbb75c4700 1 --2- 172.21.15.29:0/3054298368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbb01358b0 0x7fcbb0154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:10.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.877+0000 7fcbb75c4700 1 -- 172.21.15.29:0/3054298368 >> 172.21.15.29:0/3054298368 conn(0x7fcbb00c8bd0 msgr2=0x7fcbb0134190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:10.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.877+0000 7fcbb75c4700 1 -- 172.21.15.29:0/3054298368 shutdown_connections 2024-01-24T14:17:10.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:10.877+0000 7fcbb75c4700 1 -- 172.21.15.29:0/3054298368 wait complete. 2024-01-24T14:17:10.879 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-01-24T14:17:10.894 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-24T14:17:11.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.031+0000 7fbeb94fc700 1 Processor -- start 2024-01-24T14:17:11.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.031+0000 7fbeb94fc700 1 -- start start 2024-01-24T14:17:11.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.032+0000 7fbeb94fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbeb40c5af0 0x7fbeb40c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.032+0000 7fbeb94fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbeb40bfd20 0x7fbeb40c0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.032+0000 7fbeb94fc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb40c6820 con 0x7fbeb4064740 2024-01-24T14:17:11.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.032+0000 7fbeb94fc700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb41556b0 con 0x7fbeb40c5af0 2024-01-24T14:17:11.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.032+0000 7fbeb94fc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb4155220 con 0x7fbeb40bfd20 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.032+0000 7fbeb27fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbeb40c5af0 0x7fbeb40c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.032+0000 7fbeb27fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbeb40c5af0 0x7fbeb40c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49448/0 (socket says 172.21.15.29:49448) 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.032+0000 7fbeb27fc700 1 -- 172.21.15.29:0/3525220317 learned_addr learned my addr 172.21.15.29:0/3525220317 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.032+0000 7fbeb37fe700 1 --2- 172.21.15.29:0/3525220317 >> v2:172.21.15.163:3300/0 conn(0x7fbeb40bfd20 0x7fbeb40c0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.033+0000 7fbeb27fc700 1 -- 172.21.15.29:0/3525220317 >> v2:172.21.15.163:3300/0 conn(0x7fbeb40bfd20 msgr2=0x7fbeb40c0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.033+0000 7fbeb27fc700 1 --2- 172.21.15.29:0/3525220317 >> v2:172.21.15.163:3300/0 conn(0x7fbeb40bfd20 0x7fbeb40c0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.033+0000 7fbeb27fc700 1 -- 172.21.15.29:0/3525220317 >> v1:172.21.15.163:6789/0 conn(0x7fbeb4064740 legacy=0x7fbeb4064b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.033+0000 7fbeb27fc700 1 -- 172.21.15.29:0/3525220317 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb4154680 con 0x7fbeb40c5af0 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.033+0000 7fbeb37fe700 1 --2- 172.21.15.29:0/3525220317 >> v2:172.21.15.163:3300/0 conn(0x7fbeb40bfd20 0x7fbeb40c0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.033+0000 7fbeb27fc700 1 --2- 172.21.15.29:0/3525220317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb40c5af0 0x7fbeb40c5f70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7fbeb40aaa00 tx=0x7fbea40090f0).ready entity=mon.0 client_cookie=1f47d06aca6bb09 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.033+0000 7fbeb1ffb700 1 -- 172.21.15.29:0/3525220317 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400b600 con 0x7fbeb40c5af0 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.033+0000 7fbeb1ffb700 1 -- 172.21.15.29:0/3525220317 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea400b760 con 0x7fbeb40c5af0 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.033+0000 7fbeb1ffb700 1 -- 172.21.15.29:0/3525220317 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400bae0 con 0x7fbeb40c5af0 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.034+0000 7fbeb94fc700 1 -- 172.21.15.29:0/3525220317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb40c5af0 msgr2=0x7fbeb40c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.034+0000 7fbeb94fc700 1 --2- 172.21.15.29:0/3525220317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb40c5af0 0x7fbeb40c5f70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7fbeb40aaa00 tx=0x7fbea40090f0).stop 2024-01-24T14:17:11.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.034+0000 7fbeb94fc700 1 -- 172.21.15.29:0/3525220317 shutdown_connections 2024-01-24T14:17:11.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.034+0000 7fbeb94fc700 1 --2- 172.21.15.29:0/3525220317 >> v2:172.21.15.163:3300/0 conn(0x7fbeb40bfd20 0x7fbeb40c0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.034+0000 7fbeb94fc700 1 --2- 172.21.15.29:0/3525220317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb40c5af0 0x7fbeb40c5f70 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.034+0000 7fbeb94fc700 1 -- 172.21.15.29:0/3525220317 >> 172.21.15.29:0/3525220317 conn(0x7fbeb40cb890 msgr2=0x7fbeb40c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:11.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.034+0000 7fbeb94fc700 1 -- 172.21.15.29:0/3525220317 shutdown_connections 2024-01-24T14:17:11.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.034+0000 7fbeb94fc700 1 -- 172.21.15.29:0/3525220317 wait complete. 2024-01-24T14:17:11.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.035+0000 7fbeb94fc700 1 Processor -- start 2024-01-24T14:17:11.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.035+0000 7fbeb94fc700 1 -- start start 2024-01-24T14:17:11.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb94fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb4064740 0x7fbeb4150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb94fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbeb40bfd20 0x7fbeb4063140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb94fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbeb40c5af0 0x7fbeb4063630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb94fc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb4063b20 con 0x7fbeb4064740 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb94fc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb4167cd0 con 0x7fbeb40bfd20 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb94fc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb4167fb0 con 0x7fbeb40c5af0 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb2ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb4064740 0x7fbeb4150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb2ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb4064740 0x7fbeb4150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49450/0 (socket says 172.21.15.29:49450) 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb2ffd700 1 -- 172.21.15.29:0/2218491572 learned_addr learned my addr 172.21.15.29:0/2218491572 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb27fc700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbeb40bfd20 0x7fbeb4063140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb37fe700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbeb40c5af0 0x7fbeb4063630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb2ffd700 1 -- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbeb40c5af0 msgr2=0x7fbeb4063630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb2ffd700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbeb40c5af0 0x7fbeb4063630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb2ffd700 1 -- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbeb40bfd20 msgr2=0x7fbeb4063140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb2ffd700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbeb40bfd20 0x7fbeb4063140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb2ffd700 1 -- 172.21.15.29:0/2218491572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea4009370 con 0x7fbeb4064740 2024-01-24T14:17:11.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.036+0000 7fbeb37fe700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbeb40c5af0 0x7fbeb4063630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:17:11.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.037+0000 7fbeb2ffd700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb4064740 0x7fbeb4150e40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fbe9c00c4c0 tx=0x7fbe9c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:11.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.037+0000 7fbeaaffd700 1 -- 172.21.15.29:0/2218491572 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c007020 con 0x7fbeb4064740 2024-01-24T14:17:11.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.037+0000 7fbeaaffd700 1 -- 172.21.15.29:0/2218491572 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c001e90 con 0x7fbeb4064740 2024-01-24T14:17:11.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.037+0000 7fbeb94fc700 1 -- 172.21.15.29:0/2218491572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb4168290 con 0x7fbeb4064740 2024-01-24T14:17:11.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.037+0000 7fbeaaffd700 1 -- 172.21.15.29:0/2218491572 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c0073b0 con 0x7fbeb4064740 2024-01-24T14:17:11.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.037+0000 7fbeb94fc700 1 -- 172.21.15.29:0/2218491572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb4168850 con 0x7fbeb4064740 2024-01-24T14:17:11.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.038+0000 7fbeb94fc700 1 -- 172.21.15.29:0/2218491572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb40afd40 con 0x7fbeb4064740 2024-01-24T14:17:11.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.038+0000 7fbeaaffd700 1 -- 172.21.15.29:0/2218491572 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbe9c0068b0 con 0x7fbeb4064740 2024-01-24T14:17:11.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.039+0000 7fbeaaffd700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbe9405f270 0x7fbe94061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.039+0000 7fbeaaffd700 1 -- 172.21.15.29:0/2218491572 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6920+0+0 (secure 0 0 0) 0x7fbe9c079f10 con 0x7fbeb4064740 2024-01-24T14:17:11.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.039+0000 7fbeb27fc700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbe9405f270 0x7fbe94061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.040+0000 7fbeb27fc700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbe9405f270 0x7fbe94061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fbeb40aaa00 tx=0x7fbea4017000).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:11.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.043+0000 7fbeaaffd700 1 -- 172.21.15.29:0/2218491572 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbe9c04ad40 con 0x7fbeb4064740 2024-01-24T14:17:11.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.224+0000 7fbeb94fc700 1 -- 172.21.15.29:0/2218491572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbeb40c6820 con 0x7fbeb4064740 2024-01-24T14:17:11.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.228+0000 7fbeaaffd700 1 -- 172.21.15.29:0/2218491572 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbeb40c6820 con 0x7fbeb4064740 2024-01-24T14:17:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.229+0000 7fbeb94fc700 1 -- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbe9405f270 msgr2=0x7fbe94061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.229+0000 7fbeb94fc700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbe9405f270 0x7fbe94061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fbeb40aaa00 tx=0x7fbea4017000).stop 2024-01-24T14:17:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.230+0000 7fbeb94fc700 1 -- 172.21.15.29:0/2218491572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb4064740 msgr2=0x7fbeb4150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.230+0000 7fbeb94fc700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb4064740 0x7fbeb4150e40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fbe9c00c4c0 tx=0x7fbe9c007e00).stop 2024-01-24T14:17:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.230+0000 7fbeb94fc700 1 -- 172.21.15.29:0/2218491572 shutdown_connections 2024-01-24T14:17:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.230+0000 7fbeb94fc700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbeb40c5af0 0x7fbeb4063630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.230+0000 7fbeb94fc700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fbe9405f270 0x7fbe94061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.230+0000 7fbeb94fc700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbeb4064740 0x7fbeb4150e40 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.230+0000 7fbeb94fc700 1 --2- 172.21.15.29:0/2218491572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbeb40bfd20 0x7fbeb4063140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.230+0000 7fbeb94fc700 1 -- 172.21.15.29:0/2218491572 >> 172.21.15.29:0/2218491572 conn(0x7fbeb40cb890 msgr2=0x7fbeb40c8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.230+0000 7fbeb94fc700 1 -- 172.21.15.29:0/2218491572 shutdown_connections 2024-01-24T14:17:11.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.230+0000 7fbeb94fc700 1 -- 172.21.15.29:0/2218491572 wait complete. 2024-01-24T14:17:11.244 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-01-24T14:17:11.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.375+0000 7f299a0ce700 1 Processor -- start 2024-01-24T14:17:11.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.375+0000 7f299a0ce700 1 -- start start 2024-01-24T14:17:11.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.376+0000 7f299a0ce700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f29940663b0 0x7f2994152f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.376+0000 7f299a0ce700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2994065420 0x7f29940658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.376+0000 7f299a0ce700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2994069690 con 0x7f29940be4b0 2024-01-24T14:17:11.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.376+0000 7f299a0ce700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29940be930 con 0x7f29940663b0 2024-01-24T14:17:11.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.376+0000 7f299a0ce700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29940bca30 con 0x7f2994065420 2024-01-24T14:17:11.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.377+0000 7f29937fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f29940663b0 0x7f2994152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.377+0000 7f2993fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2994065420 0x7f29940658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.377+0000 7f2992ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f29940be4b0 0x7f2994069b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:59300/0 (socket says 172.21.15.29:59300) 2024-01-24T14:17:11.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.377+0000 7f2992ffd700 1 -- 172.21.15.29:0/88574523 learned_addr learned my addr 172.21.15.29:0/88574523 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:11.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.377+0000 7f2993fff700 1 -- 172.21.15.29:0/88574523 >> v2:172.21.15.163:3300/0 conn(0x7f29940663b0 msgr2=0x7f2994152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:11.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.377+0000 7f2993fff700 1 --2- 172.21.15.29:0/88574523 >> v2:172.21.15.163:3300/0 conn(0x7f29940663b0 0x7f2994152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.377+0000 7f2993fff700 1 -- 172.21.15.29:0/88574523 >> v1:172.21.15.163:6789/0 conn(0x7f29940be4b0 legacy=0x7f2994069b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.377+0000 7f2993fff700 1 -- 172.21.15.29:0/88574523 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29940bcc40 con 0x7f2994065420 2024-01-24T14:17:11.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.377+0000 7f29937fe700 1 --2- 172.21.15.29:0/88574523 >> v2:172.21.15.163:3300/0 conn(0x7f29940663b0 0x7f2994152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:11.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.377+0000 7f2993fff700 1 --2- 172.21.15.29:0/88574523 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2994065420 0x7f29940658b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f298c009e60 tx=0x7f298c009780).ready entity=mon.2 client_cookie=24d107e99b7c414 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:11.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.377+0000 7f29927fc700 1 -- 172.21.15.29:0/88574523 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c00e070 con 0x7f2994065420 2024-01-24T14:17:11.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.378+0000 7f29927fc700 1 -- 172.21.15.29:0/88574523 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c00bb60 con 0x7f2994065420 2024-01-24T14:17:11.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.378+0000 7f29927fc700 1 -- 172.21.15.29:0/88574523 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c012410 con 0x7f2994065420 2024-01-24T14:17:11.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.378+0000 7f299a0ce700 1 -- 172.21.15.29:0/88574523 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2994065420 msgr2=0x7f29940658b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.378+0000 7f299a0ce700 1 --2- 172.21.15.29:0/88574523 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2994065420 0x7f29940658b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f298c009e60 tx=0x7f298c009780).stop 2024-01-24T14:17:11.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.378+0000 7f299a0ce700 1 -- 172.21.15.29:0/88574523 shutdown_connections 2024-01-24T14:17:11.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.378+0000 7f299a0ce700 1 --2- 172.21.15.29:0/88574523 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2994065420 0x7f29940658b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.378+0000 7f299a0ce700 1 --2- 172.21.15.29:0/88574523 >> v2:172.21.15.163:3300/0 conn(0x7f29940663b0 0x7f2994152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.378+0000 7f299a0ce700 1 -- 172.21.15.29:0/88574523 >> 172.21.15.29:0/88574523 conn(0x7f2994064390 msgr2=0x7f29940c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:11.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.378+0000 7f299a0ce700 1 -- 172.21.15.29:0/88574523 shutdown_connections 2024-01-24T14:17:11.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.378+0000 7f299a0ce700 1 -- 172.21.15.29:0/88574523 wait complete. 2024-01-24T14:17:11.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.380+0000 7f299a0ce700 1 Processor -- start 2024-01-24T14:17:11.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.380+0000 7f299a0ce700 1 -- start start 2024-01-24T14:17:11.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.380+0000 7f299a0ce700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2994065420 0x7f2994134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.380+0000 7f299a0ce700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29940663b0 0x7f29940c03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.380+0000 7f299a0ce700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29940be4b0 0x7f29940c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.380+0000 7f299a0ce700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29940c0dd0 con 0x7f2994065420 2024-01-24T14:17:11.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.380+0000 7f299a0ce700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2994150090 con 0x7f29940be4b0 2024-01-24T14:17:11.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.380+0000 7f299a0ce700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2994150bd0 con 0x7f29940663b0 2024-01-24T14:17:11.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f29937fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2994065420 0x7f2994134e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f29937fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2994065420 0x7f2994134e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49452/0 (socket says 172.21.15.29:49452) 2024-01-24T14:17:11.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f29937fe700 1 -- 172.21.15.29:0/2505720850 learned_addr learned my addr 172.21.15.29:0/2505720850 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:11.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f2992ffd700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29940663b0 0x7f29940c03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f2993fff700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29940be4b0 0x7f29940c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f29937fe700 1 -- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29940663b0 msgr2=0x7f29940c03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f29937fe700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29940663b0 0x7f29940c03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f29937fe700 1 -- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29940be4b0 msgr2=0x7f29940c08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f29937fe700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29940be4b0 0x7f29940c08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f29937fe700 1 -- 172.21.15.29:0/2505720850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29840092a0 con 0x7f2994065420 2024-01-24T14:17:11.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f29937fe700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2994065420 0x7f2994134e00 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7f298400ea20 tx=0x7f2984009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:11.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f298b7fe700 1 -- 172.21.15.29:0/2505720850 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298400ee60 con 0x7f2994065420 2024-01-24T14:17:11.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f298b7fe700 1 -- 172.21.15.29:0/2505720850 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298400c7f0 con 0x7f2994065420 2024-01-24T14:17:11.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f299a0ce700 1 -- 172.21.15.29:0/2505720850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298c009a20 con 0x7f2994065420 2024-01-24T14:17:11.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f298b7fe700 1 -- 172.21.15.29:0/2505720850 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298400cb70 con 0x7f2994065420 2024-01-24T14:17:11.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.381+0000 7f299a0ce700 1 -- 172.21.15.29:0/2505720850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2994135340 con 0x7f2994065420 2024-01-24T14:17:11.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.383+0000 7f299a0ce700 1 -- 172.21.15.29:0/2505720850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2994150eb0 con 0x7f2994065420 2024-01-24T14:17:11.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.383+0000 7f298b7fe700 1 -- 172.21.15.29:0/2505720850 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f298401b400 con 0x7f2994065420 2024-01-24T14:17:11.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.383+0000 7f298b7fe700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f297405f270 0x7f2974061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.383+0000 7f298b7fe700 1 -- 172.21.15.29:0/2505720850 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6920+0+0 (secure 0 0 0) 0x7f298407ab60 con 0x7f2994065420 2024-01-24T14:17:11.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.384+0000 7f2992ffd700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f297405f270 0x7f2974061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.384+0000 7f2992ffd700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f297405f270 0x7f2974061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f2994063130 tx=0x7f297c009040).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:11.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.388+0000 7f298b7fe700 1 -- 172.21.15.29:0/2505720850 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f298404b930 con 0x7f2994065420 2024-01-24T14:17:11.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.571+0000 7f299a0ce700 1 -- 172.21.15.29:0/2505720850 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f299413a7b0 con 0x7f2994065420 2024-01-24T14:17:11.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.575+0000 7f298b7fe700 1 -- 172.21.15.29:0/2505720850 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v25) v1 ==== 120+0+0 (secure 0 0 0) 0x7f298404ef50 con 0x7f2994065420 2024-01-24T14:17:11.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.577+0000 7f299a0ce700 1 -- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f297405f270 msgr2=0x7f2974061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.577+0000 7f299a0ce700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f297405f270 0x7f2974061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f2994063130 tx=0x7f297c009040).stop 2024-01-24T14:17:11.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.577+0000 7f299a0ce700 1 -- 172.21.15.29:0/2505720850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2994065420 msgr2=0x7f2994134e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.577+0000 7f299a0ce700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2994065420 0x7f2994134e00 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7f298400ea20 tx=0x7f2984009060).stop 2024-01-24T14:17:11.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.577+0000 7f299a0ce700 1 -- 172.21.15.29:0/2505720850 shutdown_connections 2024-01-24T14:17:11.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.577+0000 7f299a0ce700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29940be4b0 0x7f29940c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.577+0000 7f299a0ce700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f297405f270 0x7f2974061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.577+0000 7f299a0ce700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2994065420 0x7f2994134e00 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.577+0000 7f299a0ce700 1 --2- 172.21.15.29:0/2505720850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29940663b0 0x7f29940c03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.577+0000 7f299a0ce700 1 -- 172.21.15.29:0/2505720850 >> 172.21.15.29:0/2505720850 conn(0x7f2994064390 msgr2=0x7f29940c6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:11.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.577+0000 7f299a0ce700 1 -- 172.21.15.29:0/2505720850 shutdown_connections 2024-01-24T14:17:11.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.577+0000 7f299a0ce700 1 -- 172.21.15.29:0/2505720850 wait complete. 2024-01-24T14:17:11.592 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2024-01-24T14:17:11.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.727+0000 7f467bf94700 1 Processor -- start 2024-01-24T14:17:11.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.727+0000 7f467bf94700 1 -- start start 2024-01-24T14:17:11.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.728+0000 7f467bf94700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46740627b0 0x7f4674062c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.728+0000 7f467bf94700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4674065770 con 0x7f4674138330 2024-01-24T14:17:11.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.728+0000 7f467bf94700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4674063130 con 0x7f4674138710 2024-01-24T14:17:11.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.728+0000 7f467bf94700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4674069cf0 con 0x7f46740627b0 2024-01-24T14:17:11.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.728+0000 7f4679d30700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4674138330 0x7f46740bca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38556/0 (socket says 172.21.15.29:38556) 2024-01-24T14:17:11.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.728+0000 7f4679d30700 1 -- 172.21.15.29:0/3408650226 learned_addr learned my addr 172.21.15.29:0/3408650226 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:11.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f467a531700 1 --2- 172.21.15.29:0/3408650226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46740627b0 0x7f4674062c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572679407 0 0) 0x7f4674065770 con 0x7f4674138330 2024-01-24T14:17:11.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46600033f0 con 0x7f4674138330 2024-01-24T14:17:11.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 346133567 0 0) 0x7f46600033f0 con 0x7f4674138330 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46740627b0 msgr2=0x7f4674062c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 --2- 172.21.15.29:0/3408650226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46740627b0 0x7f4674062c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 >> v1:172.21.15.163:6789/0 conn(0x7f4674138710 legacy=0x7f46740be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4674069f60 con 0x7f4674138330 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f4668001470 con 0x7f4674138330 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 >> v1:172.21.15.29:6789/0 conn(0x7f4674138330 legacy=0x7f46740bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 --2- 172.21.15.29:0/3408650226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4660003910 0x7f4660003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 --2- 172.21.15.29:0/3408650226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46740627b0 0x7f4660004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 --2- 172.21.15.29:0/3408650226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4660004b80 0x7f4660005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4674069cf0 con 0x7f4660004b80 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466000b8e0 con 0x7f4660003910 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466000ba50 con 0x7f46740627b0 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f467a531700 1 --2- 172.21.15.29:0/3408650226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4660004b80 0x7f4660005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f4679d30700 1 --2- 172.21.15.29:0/3408650226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4660003910 0x7f4660003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.729+0000 7f467952f700 1 --2- 172.21.15.29:0/3408650226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46740627b0 0x7f4660004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.730+0000 7f467a531700 1 -- 172.21.15.29:0/3408650226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46740627b0 msgr2=0x7f4660004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.730+0000 7f467a531700 1 --2- 172.21.15.29:0/3408650226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46740627b0 0x7f4660004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.730+0000 7f467a531700 1 -- 172.21.15.29:0/3408650226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4660003910 msgr2=0x7f4660003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.730+0000 7f467a531700 1 --2- 172.21.15.29:0/3408650226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4660003910 0x7f4660003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.730+0000 7f467a531700 1 -- 172.21.15.29:0/3408650226 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46600033f0 con 0x7f4660004b80 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.730+0000 7f467a531700 1 --2- 172.21.15.29:0/3408650226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4660004b80 0x7f4660005000 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f466c00e210 tx=0x7f466c00b040).ready entity=mon.0 client_cookie=4b19d5d5ede5a57e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.730+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c009ac0 con 0x7f4660004b80 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.730+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c00f040 con 0x7f4660004b80 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.730+0000 7f4678d2e700 1 -- 172.21.15.29:0/3408650226 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c00ec70 con 0x7f4660004b80 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.731+0000 7f467bf94700 1 -- 172.21.15.29:0/3408650226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4660004b80 msgr2=0x7f4660005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.731+0000 7f467bf94700 1 --2- 172.21.15.29:0/3408650226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4660004b80 0x7f4660005000 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f466c00e210 tx=0x7f466c00b040).stop 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.731+0000 7f467a531700 1 -- 172.21.15.29:0/3408650226 reap_dead start 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.731+0000 7f467bf94700 1 -- 172.21.15.29:0/3408650226 shutdown_connections 2024-01-24T14:17:11.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.731+0000 7f467bf94700 1 -- 172.21.15.29:0/3408650226 >> 172.21.15.29:0/3408650226 conn(0x7f46740c85c0 msgr2=0x7f46740c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:11.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.731+0000 7f467bf94700 1 -- 172.21.15.29:0/3408650226 shutdown_connections 2024-01-24T14:17:11.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.731+0000 7f467bf94700 1 -- 172.21.15.29:0/3408650226 wait complete. 2024-01-24T14:17:11.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.732+0000 7f467bf94700 1 Processor -- start 2024-01-24T14:17:11.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.732+0000 7f467bf94700 1 -- start start 2024-01-24T14:17:11.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.733+0000 7f467bf94700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4660004b80 0x7f4674155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.733+0000 7f467bf94700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f467414f9a0 0x7f46740ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.733+0000 7f467bf94700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46740ace60 0x7f4674153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.733+0000 7f467bf94700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4674161c10 con 0x7f46740ace60 2024-01-24T14:17:11.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.733+0000 7f467bf94700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4674162a30 con 0x7f4660004b80 2024-01-24T14:17:11.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.733+0000 7f467bf94700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4674155720 con 0x7f467414f9a0 2024-01-24T14:17:11.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.733+0000 7f4679d30700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4660004b80 0x7f4674155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.734+0000 7f4679d30700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4660004b80 0x7f4674155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:59876/0 (socket says 172.21.15.29:59876) 2024-01-24T14:17:11.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.734+0000 7f4679d30700 1 -- 172.21.15.29:0/2147670178 learned_addr learned my addr 172.21.15.29:0/2147670178 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:11.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.734+0000 7f467a531700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46740ace60 0x7f4674153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.734+0000 7f467952f700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f467414f9a0 0x7f46740ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.734+0000 7f4679d30700 1 -- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f467414f9a0 msgr2=0x7f46740ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.734+0000 7f4679d30700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f467414f9a0 0x7f46740ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.734+0000 7f4679d30700 1 -- 172.21.15.29:0/2147670178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46740ace60 msgr2=0x7f4674153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:11.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.734+0000 7f4679d30700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46740ace60 0x7f4674153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:11.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.734+0000 7f4679d30700 1 -- 172.21.15.29:0/2147670178 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46740ad2c0 con 0x7f4660004b80 2024-01-24T14:17:11.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.734+0000 7f467a531700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46740ace60 0x7f4674153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:11.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.736+0000 7f4679d30700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4660004b80 0x7f4674155230 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f4668004a10 tx=0x7f4668001e70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:11.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.737+0000 7f4665ffb700 1 -- 172.21.15.29:0/2147670178 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466800f6c0 con 0x7f4660004b80 2024-01-24T14:17:11.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.737+0000 7f4665ffb700 1 -- 172.21.15.29:0/2147670178 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466800f820 con 0x7f4660004b80 2024-01-24T14:17:11.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.737+0000 7f4665ffb700 1 -- 172.21.15.29:0/2147670178 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466800fba0 con 0x7f4660004b80 2024-01-24T14:17:11.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.737+0000 7f467bf94700 1 -- 172.21.15.29:0/2147670178 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4674161e20 con 0x7f4660004b80 2024-01-24T14:17:11.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.738+0000 7f467bf94700 1 -- 172.21.15.29:0/2147670178 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46741623e0 con 0x7f4660004b80 2024-01-24T14:17:11.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.741+0000 7f4665ffb700 1 -- 172.21.15.29:0/2147670178 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f466801c360 con 0x7f4660004b80 2024-01-24T14:17:11.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.741+0000 7f467bf94700 1 -- 172.21.15.29:0/2147670178 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46740475e0 con 0x7f4660004b80 2024-01-24T14:17:11.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.741+0000 7f4665ffb700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f465805f270 0x7f4658061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:11.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.741+0000 7f4665ffb700 1 -- 172.21.15.29:0/2147670178 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6920+0+0 (secure 0 0 0) 0x7f4668079c90 con 0x7f4660004b80 2024-01-24T14:17:11.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.742+0000 7f467952f700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f465805f270 0x7f4658061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:11.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.745+0000 7f467952f700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f465805f270 0x7f4658061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f46740c2060 tx=0x7f4670007b90).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:11.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:11.749+0000 7f4665ffb700 1 -- 172.21.15.29:0/2147670178 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f466804aa60 con 0x7f4660004b80 2024-01-24T14:17:12.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.005+0000 7f467bf94700 1 -- 172.21.15.29:0/2147670178 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f467406a570 con 0x7f4660004b80 2024-01-24T14:17:12.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.968+0000 7f4665ffb700 1 -- 172.21.15.29:0/2147670178 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v712) v1 ==== 148+0+0 (secure 0 0 0) 0x7f4668019020 con 0x7f4660004b80 2024-01-24T14:17:12.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.970+0000 7f467bf94700 1 -- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f465805f270 msgr2=0x7f4658061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:12.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.970+0000 7f467bf94700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f465805f270 0x7f4658061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f46740c2060 tx=0x7f4670007b90).stop 2024-01-24T14:17:12.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.970+0000 7f467bf94700 1 -- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4660004b80 msgr2=0x7f4674155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:12.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.970+0000 7f467bf94700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4660004b80 0x7f4674155230 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f4668004a10 tx=0x7f4668001e70).stop 2024-01-24T14:17:12.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.970+0000 7f467bf94700 1 -- 172.21.15.29:0/2147670178 shutdown_connections 2024-01-24T14:17:12.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.971+0000 7f467bf94700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f465805f270 0x7f4658061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:12.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.971+0000 7f467bf94700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46740ace60 0x7f4674153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:12.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.971+0000 7f467bf94700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4660004b80 0x7f4674155230 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:12.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.971+0000 7f467bf94700 1 --2- 172.21.15.29:0/2147670178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f467414f9a0 0x7f46740ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:12.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.971+0000 7f467bf94700 1 -- 172.21.15.29:0/2147670178 >> 172.21.15.29:0/2147670178 conn(0x7f46740c85c0 msgr2=0x7f46740aac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:12.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.971+0000 7f467bf94700 1 -- 172.21.15.29:0/2147670178 shutdown_connections 2024-01-24T14:17:12.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:12.971+0000 7f467bf94700 1 -- 172.21.15.29:0/2147670178 wait complete. 2024-01-24T14:17:12.987 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2024-01-24T14:17:12.987 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2024-01-24T14:17:12.987 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-01-24T14:17:12.987 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2024-01-24T14:17:13.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.121+0000 7fb0b47e0700 1 Processor -- start 2024-01-24T14:17:13.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.121+0000 7fb0b47e0700 1 -- start start 2024-01-24T14:17:13.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.122+0000 7fb0b47e0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb0ac062ec0 0x7fb0ac0c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:13.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.122+0000 7fb0b47e0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0ac1550f0 0x7fb0ac069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:13.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.122+0000 7fb0b47e0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ac065360 con 0x7fb0ac1540a0 2024-01-24T14:17:13.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.122+0000 7fb0b47e0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ac0a74f0 con 0x7fb0ac062ec0 2024-01-24T14:17:13.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.122+0000 7fb0b47e0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0a6ce0 con 0x7fb0ac1550f0 2024-01-24T14:17:13.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.123+0000 7fb0b1d7b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb0ac062ec0 0x7fb0ac0c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:13.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.123+0000 7fb0b1d7b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb0ac062ec0 0x7fb0ac0c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49492/0 (socket says 172.21.15.29:49492) 2024-01-24T14:17:13.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.123+0000 7fb0b1d7b700 1 -- 172.21.15.29:0/64332971 learned_addr learned my addr 172.21.15.29:0/64332971 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:13.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.123+0000 7fb0b2d7d700 1 --2- 172.21.15.29:0/64332971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0ac1550f0 0x7fb0ac069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:13.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.123+0000 7fb0b1d7b700 1 -- 172.21.15.29:0/64332971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0ac1550f0 msgr2=0x7fb0ac069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:13.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.123+0000 7fb0b1d7b700 1 --2- 172.21.15.29:0/64332971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0ac1550f0 0x7fb0ac069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:13.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.123+0000 7fb0b1d7b700 1 -- 172.21.15.29:0/64332971 >> v1:172.21.15.163:6789/0 conn(0x7fb0ac1540a0 legacy=0x7fb0ac0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.123+0000 7fb0b1d7b700 1 -- 172.21.15.29:0/64332971 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac0636f0 con 0x7fb0ac062ec0 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.123+0000 7fb0b1d7b700 1 --2- 172.21.15.29:0/64332971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0ac062ec0 0x7fb0ac0c4ca0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7fb09c006fd0 tx=0x7fb09c0090f0).ready entity=mon.0 client_cookie=5c80e1c0b0c86a46 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.123+0000 7fb0b157a700 1 -- 172.21.15.29:0/64332971 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c00b600 con 0x7fb0ac062ec0 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.123+0000 7fb0b157a700 1 -- 172.21.15.29:0/64332971 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c00b760 con 0x7fb0ac062ec0 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.123+0000 7fb0b157a700 1 -- 172.21.15.29:0/64332971 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c00bae0 con 0x7fb0ac062ec0 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.124+0000 7fb0b47e0700 1 -- 172.21.15.29:0/64332971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0ac062ec0 msgr2=0x7fb0ac0c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.124+0000 7fb0b47e0700 1 --2- 172.21.15.29:0/64332971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0ac062ec0 0x7fb0ac0c4ca0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7fb09c006fd0 tx=0x7fb09c0090f0).stop 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.124+0000 7fb0b47e0700 1 -- 172.21.15.29:0/64332971 shutdown_connections 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.124+0000 7fb0b47e0700 1 --2- 172.21.15.29:0/64332971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0ac1550f0 0x7fb0ac069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.124+0000 7fb0b47e0700 1 --2- 172.21.15.29:0/64332971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0ac062ec0 0x7fb0ac0c4ca0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.124+0000 7fb0b47e0700 1 -- 172.21.15.29:0/64332971 >> 172.21.15.29:0/64332971 conn(0x7fb0ac0c85c0 msgr2=0x7fb0ac0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.124+0000 7fb0b47e0700 1 -- 172.21.15.29:0/64332971 shutdown_connections 2024-01-24T14:17:13.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.124+0000 7fb0b47e0700 1 -- 172.21.15.29:0/64332971 wait complete. 2024-01-24T14:17:13.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.125+0000 7fb0b47e0700 1 Processor -- start 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.125+0000 7fb0b47e0700 1 -- start start 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.126+0000 7fb0b47e0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0ac062ec0 0x7fb0ac0a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.126+0000 7fb0b47e0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0ac1540a0 0x7fb0ac14cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.126+0000 7fb0b47e0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0ac1550f0 0x7fb0ac152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.126+0000 7fb0b47e0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0acd90 con 0x7fb0ac062ec0 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.126+0000 7fb0b47e0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0c0800 con 0x7fb0ac1540a0 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.126+0000 7fb0b47e0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0bfed0 con 0x7fb0ac1550f0 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.126+0000 7fb0b257c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0ac062ec0 0x7fb0ac0a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.126+0000 7fb0b257c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0ac062ec0 0x7fb0ac0a7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49494/0 (socket says 172.21.15.29:49494) 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.126+0000 7fb0b257c700 1 -- 172.21.15.29:0/1665257347 learned_addr learned my addr 172.21.15.29:0/1665257347 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.126+0000 7fb0b2d7d700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0ac1550f0 0x7fb0ac152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.126+0000 7fb0b1d7b700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0ac1540a0 0x7fb0ac14cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.127+0000 7fb0b257c700 1 -- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0ac1550f0 msgr2=0x7fb0ac152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:13.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.127+0000 7fb0b257c700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0ac1550f0 0x7fb0ac152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:13.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.127+0000 7fb0b257c700 1 -- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0ac1540a0 msgr2=0x7fb0ac14cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:13.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.127+0000 7fb0b257c700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0ac1540a0 0x7fb0ac14cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:13.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.127+0000 7fb0b257c700 1 -- 172.21.15.29:0/1665257347 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09c009370 con 0x7fb0ac062ec0 2024-01-24T14:17:13.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.127+0000 7fb0b257c700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0ac062ec0 0x7fb0ac0a7630 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7fb0a800b460 tx=0x7fb0a8006f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:13.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.127+0000 7fb0a27fc700 1 -- 172.21.15.29:0/1665257347 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8012410 con 0x7fb0ac062ec0 2024-01-24T14:17:13.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.127+0000 7fb0a27fc700 1 -- 172.21.15.29:0/1665257347 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8012590 con 0x7fb0ac062ec0 2024-01-24T14:17:13.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.127+0000 7fb0b47e0700 1 -- 172.21.15.29:0/1665257347 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ac0c01b0 con 0x7fb0ac062ec0 2024-01-24T14:17:13.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.127+0000 7fb0a27fc700 1 -- 172.21.15.29:0/1665257347 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8012950 con 0x7fb0ac062ec0 2024-01-24T14:17:13.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.127+0000 7fb0b47e0700 1 -- 172.21.15.29:0/1665257347 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ac14d4b0 con 0x7fb0ac062ec0 2024-01-24T14:17:13.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.128+0000 7fb0a27fc700 1 -- 172.21.15.29:0/1665257347 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb0a8012cc0 con 0x7fb0ac062ec0 2024-01-24T14:17:13.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.129+0000 7fb0b47e0700 1 -- 172.21.15.29:0/1665257347 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ac0475e0 con 0x7fb0ac062ec0 2024-01-24T14:17:13.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.129+0000 7fb0a27fc700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb09005f270 0x7fb090061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:13.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.129+0000 7fb0a27fc700 1 -- 172.21.15.29:0/1665257347 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6920+0+0 (secure 0 0 0) 0x7fb0a807a3a0 con 0x7fb0ac062ec0 2024-01-24T14:17:13.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.129+0000 7fb0b1d7b700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb09005f270 0x7fb090061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:13.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.130+0000 7fb0b1d7b700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb09005f270 0x7fb090061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fb0ac0632c0 tx=0x7fb09c017000).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:13.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.134+0000 7fb0a27fc700 1 -- 172.21.15.29:0/1665257347 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb0a804b1d0 con 0x7fb0ac062ec0 2024-01-24T14:17:13.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.387+0000 7fb0b47e0700 1 -- 172.21.15.29:0/1665257347 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7fb0ac06a5f0 con 0x7fb0ac062ec0 2024-01-24T14:17:13.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.971+0000 7fb0a27fc700 1 -- 172.21.15.29:0/1665257347 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v713) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb0a804e7f0 con 0x7fb0ac062ec0 2024-01-24T14:17:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.973+0000 7fb0b47e0700 1 -- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb09005f270 msgr2=0x7fb090061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.973+0000 7fb0b47e0700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb09005f270 0x7fb090061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fb0ac0632c0 tx=0x7fb09c017000).stop 2024-01-24T14:17:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.973+0000 7fb0b47e0700 1 -- 172.21.15.29:0/1665257347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0ac062ec0 msgr2=0x7fb0ac0a7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.973+0000 7fb0b47e0700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0ac062ec0 0x7fb0ac0a7630 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7fb0a800b460 tx=0x7fb0a8006f20).stop 2024-01-24T14:17:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.973+0000 7fb0b47e0700 1 -- 172.21.15.29:0/1665257347 shutdown_connections 2024-01-24T14:17:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.973+0000 7fb0b47e0700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0ac1550f0 0x7fb0ac152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.973+0000 7fb0b47e0700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb09005f270 0x7fb090061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:13.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.973+0000 7fb0b47e0700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0ac062ec0 0x7fb0ac0a7630 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.973+0000 7fb0b47e0700 1 --2- 172.21.15.29:0/1665257347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0ac1540a0 0x7fb0ac14cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.973+0000 7fb0b47e0700 1 -- 172.21.15.29:0/1665257347 >> 172.21.15.29:0/1665257347 conn(0x7fb0ac0c85c0 msgr2=0x7fb0ac0627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.973+0000 7fb0b47e0700 1 -- 172.21.15.29:0/1665257347 shutdown_connections 2024-01-24T14:17:13.975 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:13.973+0000 7fb0b47e0700 1 -- 172.21.15.29:0/1665257347 wait complete. 2024-01-24T14:17:13.989 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-24T14:17:14.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.127+0000 7f50f991a700 1 Processor -- start 2024-01-24T14:17:14.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.127+0000 7f50f991a700 1 -- start start 2024-01-24T14:17:14.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f991a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f4064760 0x7f50f4064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f991a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f50f40c8490 0x7f50f40a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f991a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f50f4154700 0x7f50f4154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f991a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f4152240 con 0x7f50f40c8490 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f991a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f41586c0 con 0x7f50f4154700 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f991a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40aa670 con 0x7f50f4064760 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f27fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f50f40c8490 0x7f50f40a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f27fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f50f40c8490 0x7f50f40a9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47608/0 (socket says 172.21.15.29:47608) 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f27fc700 1 -- 172.21.15.29:0/3221952566 learned_addr learned my addr 172.21.15.29:0/3221952566 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f2ffd700 1 --2- 172.21.15.29:0/3221952566 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f4064760 0x7f50f4064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f37fe700 1 --2- 172.21.15.29:0/3221952566 >> v2:172.21.15.163:3300/0 conn(0x7f50f4154700 0x7f50f4154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f27fc700 1 -- 172.21.15.29:0/3221952566 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f4064760 msgr2=0x7f50f4064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f27fc700 1 --2- 172.21.15.29:0/3221952566 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f4064760 0x7f50f4064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f27fc700 1 -- 172.21.15.29:0/3221952566 >> v2:172.21.15.163:3300/0 conn(0x7f50f4154700 msgr2=0x7f50f4154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f27fc700 1 --2- 172.21.15.29:0/3221952566 >> v2:172.21.15.163:3300/0 conn(0x7f50f4154700 0x7f50f4154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f27fc700 1 -- 172.21.15.29:0/3221952566 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f4157d90 con 0x7f50f40c8490 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.128+0000 7f50f27fc700 1 --2- 172.21.15.29:0/3221952566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f40c8490 0x7f50f40a9d90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f50f4084340 tx=0x7f50e80090f0).ready entity=mon.0 client_cookie=13821a9fb73be7ea server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.129+0000 7f50f1ffb700 1 -- 172.21.15.29:0/3221952566 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e800f510 con 0x7f50f40c8490 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.129+0000 7f50f1ffb700 1 -- 172.21.15.29:0/3221952566 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e800f670 con 0x7f50f40c8490 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.129+0000 7f50f1ffb700 1 -- 172.21.15.29:0/3221952566 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e800f9f0 con 0x7f50f40c8490 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.129+0000 7f50f991a700 1 -- 172.21.15.29:0/3221952566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f40c8490 msgr2=0x7f50f40a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.129+0000 7f50f991a700 1 --2- 172.21.15.29:0/3221952566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f40c8490 0x7f50f40a9d90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f50f4084340 tx=0x7f50e80090f0).stop 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.129+0000 7f50f991a700 1 -- 172.21.15.29:0/3221952566 shutdown_connections 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.129+0000 7f50f991a700 1 --2- 172.21.15.29:0/3221952566 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f4064760 0x7f50f4064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.129+0000 7f50f991a700 1 --2- 172.21.15.29:0/3221952566 >> v2:172.21.15.163:3300/0 conn(0x7f50f4154700 0x7f50f4154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.129+0000 7f50f991a700 1 --2- 172.21.15.29:0/3221952566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f40c8490 0x7f50f40a9d90 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.129+0000 7f50f991a700 1 -- 172.21.15.29:0/3221952566 >> 172.21.15.29:0/3221952566 conn(0x7f50f40cb8b0 msgr2=0x7f50f40c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:14.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.129+0000 7f50f991a700 1 -- 172.21.15.29:0/3221952566 shutdown_connections 2024-01-24T14:17:14.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.129+0000 7f50f991a700 1 -- 172.21.15.29:0/3221952566 wait complete. 2024-01-24T14:17:14.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.131+0000 7f50f991a700 1 Processor -- start 2024-01-24T14:17:14.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.131+0000 7f50f991a700 1 -- start start 2024-01-24T14:17:14.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.131+0000 7f50f991a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f4064760 0x7f50f414d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:14.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.131+0000 7f50f991a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f40c8490 0x7f50f4154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:14.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.131+0000 7f50f991a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50f4154700 0x7f50f41550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:14.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f991a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40ad7a0 con 0x7f50f40c8490 2024-01-24T14:17:14.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f991a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f4155ce0 con 0x7f50f4154700 2024-01-24T14:17:14.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f991a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f41563b0 con 0x7f50f4064760 2024-01-24T14:17:14.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f27fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f40c8490 0x7f50f4154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:14.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f27fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f40c8490 0x7f50f4154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47622/0 (socket says 172.21.15.29:47622) 2024-01-24T14:17:14.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f27fc700 1 -- 172.21.15.29:0/3093035309 learned_addr learned my addr 172.21.15.29:0/3093035309 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:14.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f37fe700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50f4154700 0x7f50f41550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:14.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f2ffd700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f4064760 0x7f50f414d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:14.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f27fc700 1 -- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f4064760 msgr2=0x7f50f414d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:14.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f27fc700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f4064760 0x7f50f414d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f27fc700 1 -- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50f4154700 msgr2=0x7f50f41550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:14.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f27fc700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50f4154700 0x7f50f41550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f27fc700 1 -- 172.21.15.29:0/3093035309 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e8009370 con 0x7f50f40c8490 2024-01-24T14:17:14.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f37fe700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50f4154700 0x7f50f41550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:17:14.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f2ffd700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f4064760 0x7f50f414d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:17:14.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f27fc700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f40c8490 0x7f50f4154bb0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f50e8011000 tx=0x7f50e8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:14.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50daffd700 1 -- 172.21.15.29:0/3093035309 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e800f7e0 con 0x7f50f40c8490 2024-01-24T14:17:14.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50daffd700 1 -- 172.21.15.29:0/3093035309 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e800f940 con 0x7f50f40c8490 2024-01-24T14:17:14.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50daffd700 1 -- 172.21.15.29:0/3093035309 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e801c670 con 0x7f50f40c8490 2024-01-24T14:17:14.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f991a700 1 -- 172.21.15.29:0/3093035309 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f4155720 con 0x7f50f40c8490 2024-01-24T14:17:14.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.132+0000 7f50f991a700 1 -- 172.21.15.29:0/3093035309 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f4156040 con 0x7f50f40c8490 2024-01-24T14:17:14.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.133+0000 7f50f991a700 1 -- 172.21.15.29:0/3093035309 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f40475e0 con 0x7f50f40c8490 2024-01-24T14:17:14.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.134+0000 7f50daffd700 1 -- 172.21.15.29:0/3093035309 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f50e801ca20 con 0x7f50f40c8490 2024-01-24T14:17:14.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.134+0000 7f50daffd700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f50e005f270 0x7f50e0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:14.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.134+0000 7f50daffd700 1 -- 172.21.15.29:0/3093035309 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6920+0+0 (secure 0 0 0) 0x7f50e807c5b0 con 0x7f50f40c8490 2024-01-24T14:17:14.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.135+0000 7f50f2ffd700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f50e005f270 0x7f50e0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:14.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.135+0000 7f50f2ffd700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f50e005f270 0x7f50e0061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f50e4006d80 tx=0x7f50e4008040).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:14.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.139+0000 7f50daffd700 1 -- 172.21.15.29:0/3093035309 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f50e804d3e0 con 0x7f50f40c8490 2024-01-24T14:17:14.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.321+0000 7f50f991a700 1 -- 172.21.15.29:0/3093035309 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f50f41561d0 con 0x7f50f40c8490 2024-01-24T14:17:14.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.325+0000 7f50daffd700 1 -- 172.21.15.29:0/3093035309 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f50e802a0c0 con 0x7f50f40c8490 2024-01-24T14:17:14.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.327+0000 7f50f991a700 1 -- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f50e005f270 msgr2=0x7f50e0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:14.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.327+0000 7f50f991a700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f50e005f270 0x7f50e0061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f50e4006d80 tx=0x7f50e4008040).stop 2024-01-24T14:17:14.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.327+0000 7f50f991a700 1 -- 172.21.15.29:0/3093035309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f40c8490 msgr2=0x7f50f4154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:14.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.327+0000 7f50f991a700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f40c8490 0x7f50f4154bb0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f50e8011000 tx=0x7f50e8017000).stop 2024-01-24T14:17:14.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.327+0000 7f50f991a700 1 -- 172.21.15.29:0/3093035309 shutdown_connections 2024-01-24T14:17:14.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.327+0000 7f50f991a700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50f4154700 0x7f50f41550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.327+0000 7f50f991a700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f4064760 0x7f50f414d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.327+0000 7f50f991a700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f50e005f270 0x7f50e0061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.327+0000 7f50f991a700 1 --2- 172.21.15.29:0/3093035309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50f40c8490 0x7f50f4154bb0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.327+0000 7f50f991a700 1 -- 172.21.15.29:0/3093035309 >> 172.21.15.29:0/3093035309 conn(0x7f50f40cb8b0 msgr2=0x7f50f40c0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:14.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.327+0000 7f50f991a700 1 -- 172.21.15.29:0/3093035309 shutdown_connections 2024-01-24T14:17:14.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.327+0000 7f50f991a700 1 -- 172.21.15.29:0/3093035309 wait complete. 2024-01-24T14:17:14.341 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T14:17:14.342 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:17:14.392 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T14:17:14.439 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T14:17:14.439 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-24T14:17:14.439 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:17:14.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.574+0000 7f2173d83700 1 Processor -- start 2024-01-24T14:17:14.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.574+0000 7f2173d83700 1 -- start start 2024-01-24T14:17:14.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.575+0000 7f2173d83700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f216c066c00 0x7f216c159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:14.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.575+0000 7f2173d83700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0bdc40 con 0x7f216c15b8a0 2024-01-24T14:17:14.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.575+0000 7f2173d83700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0bdf20 con 0x7f216c067110 2024-01-24T14:17:14.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.575+0000 7f2173d83700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c0be200 con 0x7f216c066c00 2024-01-24T14:17:14.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.575+0000 7f2172320700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f216c066c00 0x7f216c159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:14.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.575+0000 7f2171b1f700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f216c15b8a0 0x7f216c0639e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44086/0 (socket says 172.21.15.29:44086) 2024-01-24T14:17:14.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.575+0000 7f2171b1f700 1 -- 172.21.15.29:0/1107802920 learned_addr learned my addr 172.21.15.29:0/1107802920 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:14.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.576+0000 7f2172320700 1 -- 172.21.15.29:0/1107802920 >> v1:172.21.15.163:6789/0 conn(0x7f216c067110 legacy=0x7f216c0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:14.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.576+0000 7f2172320700 1 -- 172.21.15.29:0/1107802920 >> v1:172.21.15.29:6789/0 conn(0x7f216c15b8a0 legacy=0x7f216c0639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:14.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.576+0000 7f2172320700 1 -- 172.21.15.29:0/1107802920 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0be4e0 con 0x7f216c066c00 2024-01-24T14:17:14.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.576+0000 7f2172320700 1 --2- 172.21.15.29:0/1107802920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216c066c00 0x7f216c159c20 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f2164009e60 tx=0x7f2164009780).ready entity=mon.0 client_cookie=915761a9ba3e581f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:14.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.576+0000 7f2170b1d700 1 -- 172.21.15.29:0/1107802920 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400e070 con 0x7f216c066c00 2024-01-24T14:17:14.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.576+0000 7f2170b1d700 1 -- 172.21.15.29:0/1107802920 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216400bb30 con 0x7f216c066c00 2024-01-24T14:17:14.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.576+0000 7f2170b1d700 1 -- 172.21.15.29:0/1107802920 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400beb0 con 0x7f216c066c00 2024-01-24T14:17:14.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.577+0000 7f2173d83700 1 -- 172.21.15.29:0/1107802920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216c066c00 msgr2=0x7f216c159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:14.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.577+0000 7f2173d83700 1 --2- 172.21.15.29:0/1107802920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216c066c00 0x7f216c159c20 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f2164009e60 tx=0x7f2164009780).stop 2024-01-24T14:17:14.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.577+0000 7f2173d83700 1 -- 172.21.15.29:0/1107802920 shutdown_connections 2024-01-24T14:17:14.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.577+0000 7f2173d83700 1 --2- 172.21.15.29:0/1107802920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216c066c00 0x7f216c159c20 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.577+0000 7f2173d83700 1 -- 172.21.15.29:0/1107802920 >> 172.21.15.29:0/1107802920 conn(0x7f216c0c8bb0 msgr2=0x7f216c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:14.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.577+0000 7f2173d83700 1 -- 172.21.15.29:0/1107802920 shutdown_connections 2024-01-24T14:17:14.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.577+0000 7f2173d83700 1 -- 172.21.15.29:0/1107802920 wait complete. 2024-01-24T14:17:14.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.578+0000 7f2173d83700 1 Processor -- start 2024-01-24T14:17:14.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.578+0000 7f2173d83700 1 -- start start 2024-01-24T14:17:14.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2173d83700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f216c066c00 0x7f216c14d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2173d83700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f216c067110 0x7f216c14e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2173d83700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216c15b8a0 0x7f216c14b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2173d83700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c13ca40 con 0x7f216c15b8a0 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2173d83700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c168300 con 0x7f216c067110 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2173d83700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c13c110 con 0x7f216c066c00 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2172320700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216c15b8a0 0x7f216c14b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2172320700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216c15b8a0 0x7f216c14b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47632/0 (socket says 172.21.15.29:47632) 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2172320700 1 -- 172.21.15.29:0/3782935657 learned_addr learned my addr 172.21.15.29:0/3782935657 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f217131e700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f216c067110 0x7f216c14e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2171b1f700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f216c066c00 0x7f216c14d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2172320700 1 -- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f216c066c00 msgr2=0x7f216c14d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2172320700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f216c066c00 0x7f216c14d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2172320700 1 -- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f216c067110 msgr2=0x7f216c14e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2172320700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f216c067110 0x7f216c14e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.579+0000 7f2172320700 1 -- 172.21.15.29:0/3782935657 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2164009a20 con 0x7f216c15b8a0 2024-01-24T14:17:14.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.580+0000 7f2172320700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216c15b8a0 0x7f216c14b980 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f2164011000 tx=0x7f2164009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:14.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.580+0000 7f2161ffb700 1 -- 172.21.15.29:0/3782935657 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164021070 con 0x7f216c15b8a0 2024-01-24T14:17:14.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.580+0000 7f2161ffb700 1 -- 172.21.15.29:0/3782935657 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216401d630 con 0x7f216c15b8a0 2024-01-24T14:17:14.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.580+0000 7f2173d83700 1 -- 172.21.15.29:0/3782935657 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c13c3f0 con 0x7f216c15b8a0 2024-01-24T14:17:14.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.580+0000 7f2161ffb700 1 -- 172.21.15.29:0/3782935657 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216401d9f0 con 0x7f216c15b8a0 2024-01-24T14:17:14.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.580+0000 7f2173d83700 1 -- 172.21.15.29:0/3782935657 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c14e9d0 con 0x7f216c15b8a0 2024-01-24T14:17:14.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.581+0000 7f2173d83700 1 -- 172.21.15.29:0/3782935657 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c13cbf0 con 0x7f216c15b8a0 2024-01-24T14:17:14.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.582+0000 7f2161ffb700 1 -- 172.21.15.29:0/3782935657 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2164003040 con 0x7f216c15b8a0 2024-01-24T14:17:14.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.582+0000 7f2161ffb700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f215005f270 0x7f2150061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:14.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.582+0000 7f2161ffb700 1 -- 172.21.15.29:0/3782935657 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6920+0+0 (secure 0 0 0) 0x7f216407be80 con 0x7f216c15b8a0 2024-01-24T14:17:14.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.582+0000 7f2171b1f700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f215005f270 0x7f2150061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:14.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.583+0000 7f2171b1f700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f215005f270 0x7f2150061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f215c007fd0 tx=0x7f215c007d30).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:14.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.586+0000 7f2161ffb700 1 -- 172.21.15.29:0/3782935657 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f216c13cbf0 con 0x7f216c15b8a0 2024-01-24T14:17:14.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.810+0000 7f2173d83700 1 -- 172.21.15.29:0/3782935657 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f216c0475e0 con 0x7f216c15b8a0 2024-01-24T14:17:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.810+0000 7f2161ffb700 1 -- 172.21.15.29:0/3782935657 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f216404ccb0 con 0x7f216c15b8a0 2024-01-24T14:17:14.811 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:14.811 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":61,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1579003637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1579003637},{"type":"v1","addr":"172.21.15.163:6836","nonce":1579003637}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2823175689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2823175689},{"type":"v1","addr":"172.21.15.163:6837","nonce":2823175689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2560003609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2560003609},{"type":"v1","addr":"172.21.15.29:6834","nonce":2560003609}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":8091,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/113821825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":113821825},{"type":"v1","addr":"172.21.15.29:6835","nonce":113821825}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[]} 2024-01-24T14:17:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.812+0000 7f2173d83700 1 -- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f215005f270 msgr2=0x7f2150061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.812+0000 7f2173d83700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f215005f270 0x7f2150061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f215c007fd0 tx=0x7f215c007d30).stop 2024-01-24T14:17:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.812+0000 7f2173d83700 1 -- 172.21.15.29:0/3782935657 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216c15b8a0 msgr2=0x7f216c14b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.812+0000 7f2173d83700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216c15b8a0 0x7f216c14b980 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f2164011000 tx=0x7f2164009780).stop 2024-01-24T14:17:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.812+0000 7f2173d83700 1 -- 172.21.15.29:0/3782935657 shutdown_connections 2024-01-24T14:17:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.812+0000 7f2173d83700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f215005f270 0x7f2150061730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.812+0000 7f2173d83700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216c15b8a0 0x7f216c14b980 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.812+0000 7f2173d83700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f216c066c00 0x7f216c14d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.812+0000 7f2173d83700 1 --2- 172.21.15.29:0/3782935657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f216c067110 0x7f216c14e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:14.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.812+0000 7f2173d83700 1 -- 172.21.15.29:0/3782935657 >> 172.21.15.29:0/3782935657 conn(0x7f216c0c8bb0 msgr2=0x7f216c15ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:14.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.812+0000 7f2173d83700 1 -- 172.21.15.29:0/3782935657 shutdown_connections 2024-01-24T14:17:14.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:14.812+0000 7f2173d83700 1 -- 172.21.15.29:0/3782935657 wait complete. 2024-01-24T14:17:14.814 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 61 2024-01-24T14:17:14.827 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-24T14:17:14.827 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-24T14:17:14.827 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-24T14:17:14.827 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:17:14.827 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-24T14:17:14.827 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-24T14:17:14.827 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-24T14:17:14.828 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:17:14.828 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-24T14:17:14.828 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-24T14:17:14.828 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-24T14:17:14.828 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:17:14.828 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-24T14:17:14.828 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-24T14:17:14.829 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-24T14:17:14.829 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T14:17:14.837 INFO:tasks.ceph.mds.a:Stopped 2024-01-24T14:17:14.837 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-24T14:17:14.838 INFO:tasks.ceph.mds.c:Stopped 2024-01-24T14:17:14.838 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-24T14:17:14.839 INFO:tasks.ceph.mds.d:Stopped 2024-01-24T14:17:14.839 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-24T14:17:14.840 INFO:tasks.ceph.mds.b:Stopped 2024-01-24T14:17:14.840 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-24T14:17:14.879 INFO:tasks.ceph.mds.a:Started 2024-01-24T14:17:14.879 DEBUG:teuthology.parallel:result is None 2024-01-24T14:17:14.880 INFO:tasks.ceph.mds.c:Started 2024-01-24T14:17:14.880 DEBUG:teuthology.parallel:result is None 2024-01-24T14:17:14.881 INFO:tasks.ceph.mds.d:Started 2024-01-24T14:17:14.881 DEBUG:teuthology.parallel:result is None 2024-01-24T14:17:14.882 INFO:tasks.ceph.mds.b:Started 2024-01-24T14:17:14.882 DEBUG:teuthology.parallel:result is None 2024-01-24T14:17:14.882 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-24T14:17:14.964 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-01-24T14:17:14.964 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T14:17:14.966 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T14:17:14.966 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T14:17:15.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.023+0000 7f1fac724700 1 Processor -- start 2024-01-24T14:17:15.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.023+0000 7f1fac724700 1 -- start start 2024-01-24T14:17:15.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.024+0000 7f1fac724700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1fa4135350 0x7f1fa40664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:15.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1fac724700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa4066990 0x7f1fa41525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:15.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1fac724700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa4156460 con 0x7f1fa4152ae0 2024-01-24T14:17:15.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1fac724700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa4155c00 con 0x7f1fa4135350 2024-01-24T14:17:15.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1fac724700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4066e10 con 0x7f1fa4066990 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1faacc1700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1fa4152ae0 0x7f1fa40679a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44088/0 (socket says 172.21.15.29:44088) 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1faacc1700 1 -- 172.21.15.29:0/3778354111 learned_addr learned my addr 172.21.15.29:0/3778354111 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1fa9cbf700 1 --2- 172.21.15.29:0/3778354111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa4066990 0x7f1fa41525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1faa4c0700 1 --2- 172.21.15.29:0/3778354111 >> v2:172.21.15.163:3300/0 conn(0x7f1fa4135350 0x7f1fa40664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1fa94be700 1 -- 172.21.15.29:0/3778354111 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871620628 0 0) 0x7f1fa4156460 con 0x7f1fa4152ae0 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1fa94be700 1 -- 172.21.15.29:0/3778354111 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f900033f0 con 0x7f1fa4152ae0 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1fa9cbf700 1 -- 172.21.15.29:0/3778354111 >> v2:172.21.15.163:3300/0 conn(0x7f1fa4135350 msgr2=0x7f1fa40664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1fa9cbf700 1 --2- 172.21.15.29:0/3778354111 >> v2:172.21.15.163:3300/0 conn(0x7f1fa4135350 0x7f1fa40664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1fa9cbf700 1 -- 172.21.15.29:0/3778354111 >> v1:172.21.15.29:6789/0 conn(0x7f1fa4152ae0 legacy=0x7f1fa40679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1fa9cbf700 1 -- 172.21.15.29:0/3778354111 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa4153390 con 0x7f1fa4066990 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.025+0000 7f1fa9cbf700 1 --2- 172.21.15.29:0/3778354111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa4066990 0x7f1fa41525f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f1f94009760 tx=0x7f1f94009060).ready entity=mon.2 client_cookie=bdd2cfdc3f50bb24 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.026+0000 7f1fa94be700 1 -- 172.21.15.29:0/3778354111 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400d4a0 con 0x7f1fa4066990 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.026+0000 7f1fa94be700 1 -- 172.21.15.29:0/3778354111 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9400d600 con 0x7f1fa4066990 2024-01-24T14:17:15.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.026+0000 7f1fa94be700 1 -- 172.21.15.29:0/3778354111 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400d980 con 0x7f1fa4066990 2024-01-24T14:17:15.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.026+0000 7f1fac724700 1 -- 172.21.15.29:0/3778354111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa4066990 msgr2=0x7f1fa41525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:15.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.026+0000 7f1fac724700 1 --2- 172.21.15.29:0/3778354111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa4066990 0x7f1fa41525f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f1f94009760 tx=0x7f1f94009060).stop 2024-01-24T14:17:15.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.026+0000 7f1fac724700 1 -- 172.21.15.29:0/3778354111 shutdown_connections 2024-01-24T14:17:15.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.026+0000 7f1fac724700 1 --2- 172.21.15.29:0/3778354111 >> v2:172.21.15.163:3300/0 conn(0x7f1fa4135350 0x7f1fa40664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:15.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.026+0000 7f1fac724700 1 --2- 172.21.15.29:0/3778354111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa4066990 0x7f1fa41525f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:15.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.026+0000 7f1fac724700 1 -- 172.21.15.29:0/3778354111 >> 172.21.15.29:0/3778354111 conn(0x7f1fa40c5640 msgr2=0x7f1fa40c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:15.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.026+0000 7f1fac724700 1 -- 172.21.15.29:0/3778354111 shutdown_connections 2024-01-24T14:17:15.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.026+0000 7f1fac724700 1 -- 172.21.15.29:0/3778354111 wait complete. 2024-01-24T14:17:15.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.028+0000 7f1fac724700 1 Processor -- start 2024-01-24T14:17:15.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.028+0000 7f1fac724700 1 -- start start 2024-01-24T14:17:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.028+0000 7f1fac724700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa4066990 0x7f1fa40a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.028+0000 7f1fac724700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa4135350 0x7f1fa4137f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.028+0000 7f1fac724700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fa4152ae0 0x7f1fa4167a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.028+0000 7f1fac724700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4064b10 con 0x7f1fa4152ae0 2024-01-24T14:17:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.028+0000 7f1fac724700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa40a8410 con 0x7f1fa4066990 2024-01-24T14:17:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.028+0000 7f1fac724700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa40ab670 con 0x7f1fa4135350 2024-01-24T14:17:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1faacc1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fa4152ae0 0x7f1fa4167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1faacc1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fa4152ae0 0x7f1fa4167a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47668/0 (socket says 172.21.15.29:47668) 2024-01-24T14:17:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1faacc1700 1 -- 172.21.15.29:0/830350438 learned_addr learned my addr 172.21.15.29:0/830350438 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1faa4c0700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa4066990 0x7f1fa40a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:15.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1fa9cbf700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa4135350 0x7f1fa4137f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1faacc1700 1 -- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa4135350 msgr2=0x7f1fa4137f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1faacc1700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa4135350 0x7f1fa4137f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1faacc1700 1 -- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa4066990 msgr2=0x7f1fa40a7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1faacc1700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa4066990 0x7f1fa40a7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1faacc1700 1 -- 172.21.15.29:0/830350438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f940092a0 con 0x7f1fa4152ae0 2024-01-24T14:17:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1faacc1700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fa4152ae0 0x7f1fa4167a50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f1f9c001460 tx=0x7f1f9c0015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1f9a7fc700 1 -- 172.21.15.29:0/830350438 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c005d30 con 0x7f1fa4152ae0 2024-01-24T14:17:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1f9a7fc700 1 -- 172.21.15.29:0/830350438 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c005e90 con 0x7f1fa4152ae0 2024-01-24T14:17:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1fac724700 1 -- 172.21.15.29:0/830350438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa40a8e40 con 0x7f1fa4152ae0 2024-01-24T14:17:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1f9a7fc700 1 -- 172.21.15.29:0/830350438 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0072e0 con 0x7f1fa4152ae0 2024-01-24T14:17:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.029+0000 7f1fac724700 1 -- 172.21.15.29:0/830350438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa4064450 con 0x7f1fa4152ae0 2024-01-24T14:17:15.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.030+0000 7f1fac724700 1 -- 172.21.15.29:0/830350438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa40475e0 con 0x7f1fa4152ae0 2024-01-24T14:17:15.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.031+0000 7f1f9a7fc700 1 -- 172.21.15.29:0/830350438 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1f9c0076d0 con 0x7f1fa4152ae0 2024-01-24T14:17:15.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.031+0000 7f1f9a7fc700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1f8805f270 0x7f1f88061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:15.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.031+0000 7f1f9a7fc700 1 -- 172.21.15.29:0/830350438 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6892+0+0 (secure 0 0 0) 0x7f1f9c079f10 con 0x7f1fa4152ae0 2024-01-24T14:17:15.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.031+0000 7f1faa4c0700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1f8805f270 0x7f1f88061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:15.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.032+0000 7f1faa4c0700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1f8805f270 0x7f1f88061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f1fa0009c30 tx=0x7f1fa0009060).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:15.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.035+0000 7f1f9a7fc700 1 -- 172.21.15.29:0/830350438 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1f9c04ae20 con 0x7f1fa4152ae0 2024-01-24T14:17:15.041 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-01-24T14:17:15.041 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-01-24T14:17:15.041 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-01-24T14:17:15.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.232+0000 7f1fac724700 1 -- 172.21.15.29:0/830350438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1fa40a8e40 con 0x7f1fa4152ae0 2024-01-24T14:17:15.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:15.997+0000 7f1f9a7fc700 1 -- 172.21.15.29:0/830350438 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v149) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1fa40a8e40 con 0x7f1fa4152ae0 2024-01-24T14:17:16.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.001+0000 7f1fac724700 1 -- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1f8805f270 msgr2=0x7f1f88061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.001+0000 7f1fac724700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1f8805f270 0x7f1f88061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f1fa0009c30 tx=0x7f1fa0009060).stop 2024-01-24T14:17:16.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.001+0000 7f1fac724700 1 -- 172.21.15.29:0/830350438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fa4152ae0 msgr2=0x7f1fa4167a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.001+0000 7f1fac724700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fa4152ae0 0x7f1fa4167a50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f1f9c001460 tx=0x7f1f9c0015b0).stop 2024-01-24T14:17:16.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.001+0000 7f1fac724700 1 -- 172.21.15.29:0/830350438 shutdown_connections 2024-01-24T14:17:16.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.001+0000 7f1fac724700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1f8805f270 0x7f1f88061730 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.001+0000 7f1fac724700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fa4152ae0 0x7f1fa4167a50 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.001+0000 7f1fac724700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa4066990 0x7f1fa40a7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.001+0000 7f1fac724700 1 --2- 172.21.15.29:0/830350438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa4135350 0x7f1fa4137f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.001+0000 7f1fac724700 1 -- 172.21.15.29:0/830350438 >> 172.21.15.29:0/830350438 conn(0x7f1fa40c5640 msgr2=0x7f1fa4151870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:16.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.001+0000 7f1fac724700 1 -- 172.21.15.29:0/830350438 shutdown_connections 2024-01-24T14:17:16.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.001+0000 7f1fac724700 1 -- 172.21.15.29:0/830350438 wait complete. 2024-01-24T14:17:16.003 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-01-24T14:17:16.019 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-24T14:17:16.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.155+0000 7faacba24700 1 Processor -- start 2024-01-24T14:17:16.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.155+0000 7faacba24700 1 -- start start 2024-01-24T14:17:16.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.156+0000 7faacba24700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faac414e840 0x7faac40c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.156+0000 7faacba24700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faac4062ec0 0x7faac40bc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.156+0000 7faacba24700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac40c2210 con 0x7faac40c1d60 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.156+0000 7faacba24700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac40632f0 con 0x7faac414e840 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.156+0000 7faacba24700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac40640f0 con 0x7faac4062ec0 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.156+0000 7faac97c0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faac414e840 0x7faac40c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.156+0000 7faac97c0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faac414e840 0x7faac40c1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47682/0 (socket says 172.21.15.29:47682) 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.156+0000 7faac97c0700 1 -- 172.21.15.29:0/3856973981 learned_addr learned my addr 172.21.15.29:0/3856973981 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.156+0000 7faac9fc1700 1 --2- 172.21.15.29:0/3856973981 >> v2:172.21.15.163:3300/0 conn(0x7faac4062ec0 0x7faac40bc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.156+0000 7faac97c0700 1 -- 172.21.15.29:0/3856973981 >> v2:172.21.15.163:3300/0 conn(0x7faac4062ec0 msgr2=0x7faac40bc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.156+0000 7faac97c0700 1 --2- 172.21.15.29:0/3856973981 >> v2:172.21.15.163:3300/0 conn(0x7faac4062ec0 0x7faac40bc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.157+0000 7faac97c0700 1 -- 172.21.15.29:0/3856973981 >> v1:172.21.15.163:6789/0 conn(0x7faac40c1d60 legacy=0x7faac40627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.157+0000 7faac97c0700 1 -- 172.21.15.29:0/3856973981 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac40634e0 con 0x7faac414e840 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.157+0000 7faac97c0700 1 --2- 172.21.15.29:0/3856973981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac414e840 0x7faac40c1870 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7faac4157d80 tx=0x7faac00090f0).ready entity=mon.0 client_cookie=72e988a5b870301a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:16.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.157+0000 7faabbfff700 1 -- 172.21.15.29:0/3856973981 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac000b520 con 0x7faac414e840 2024-01-24T14:17:16.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.157+0000 7faabbfff700 1 -- 172.21.15.29:0/3856973981 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac000b680 con 0x7faac414e840 2024-01-24T14:17:16.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.157+0000 7faabbfff700 1 -- 172.21.15.29:0/3856973981 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac000ba00 con 0x7faac414e840 2024-01-24T14:17:16.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.157+0000 7faacba24700 1 -- 172.21.15.29:0/3856973981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac414e840 msgr2=0x7faac40c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.157+0000 7faacba24700 1 --2- 172.21.15.29:0/3856973981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac414e840 0x7faac40c1870 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7faac4157d80 tx=0x7faac00090f0).stop 2024-01-24T14:17:16.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.157+0000 7faacba24700 1 -- 172.21.15.29:0/3856973981 shutdown_connections 2024-01-24T14:17:16.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.157+0000 7faacba24700 1 --2- 172.21.15.29:0/3856973981 >> v2:172.21.15.163:3300/0 conn(0x7faac4062ec0 0x7faac40bc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.157+0000 7faacba24700 1 --2- 172.21.15.29:0/3856973981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac414e840 0x7faac40c1870 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.157+0000 7faacba24700 1 -- 172.21.15.29:0/3856973981 >> 172.21.15.29:0/3856973981 conn(0x7faac4064aa0 msgr2=0x7faac4064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:16.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.158+0000 7faacba24700 1 -- 172.21.15.29:0/3856973981 shutdown_connections 2024-01-24T14:17:16.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.158+0000 7faacba24700 1 -- 172.21.15.29:0/3856973981 wait complete. 2024-01-24T14:17:16.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.159+0000 7faacba24700 1 Processor -- start 2024-01-24T14:17:16.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.159+0000 7faacba24700 1 -- start start 2024-01-24T14:17:16.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faacba24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac4062ec0 0x7faac41351c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faacba24700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac40c1d60 0x7faac40a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faacba24700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac414e840 0x7faac40a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faacba24700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac40a69f0 con 0x7faac40c1d60 2024-01-24T14:17:16.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faacba24700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac41356b0 con 0x7faac4062ec0 2024-01-24T14:17:16.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faacba24700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac40a8b00 con 0x7faac414e840 2024-01-24T14:17:16.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faac8fbf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac40c1d60 0x7faac40a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faac8fbf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac40c1d60 0x7faac40a6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47684/0 (socket says 172.21.15.29:47684) 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faac8fbf700 1 -- 172.21.15.29:0/2266264712 learned_addr learned my addr 172.21.15.29:0/2266264712 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faac9fc1700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac414e840 0x7faac40a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faac97c0700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac4062ec0 0x7faac41351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faac8fbf700 1 -- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac414e840 msgr2=0x7faac40a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faac8fbf700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac414e840 0x7faac40a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faac8fbf700 1 -- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac4062ec0 msgr2=0x7faac41351c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faac8fbf700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac4062ec0 0x7faac41351c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faac8fbf700 1 -- 172.21.15.29:0/2266264712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac0009370 con 0x7faac40c1d60 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.160+0000 7faac8fbf700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac40c1d60 0x7faac40a6010 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7faab400c4c0 tx=0x7faab4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.161+0000 7faab97fa700 1 -- 172.21.15.29:0/2266264712 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4007020 con 0x7faac40c1d60 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.161+0000 7faab97fa700 1 -- 172.21.15.29:0/2266264712 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab4001e90 con 0x7faac40c1d60 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.161+0000 7faacba24700 1 -- 172.21.15.29:0/2266264712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac40a8de0 con 0x7faac40c1d60 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.161+0000 7faab97fa700 1 -- 172.21.15.29:0/2266264712 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab40073b0 con 0x7faac40c1d60 2024-01-24T14:17:16.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.161+0000 7faacba24700 1 -- 172.21.15.29:0/2266264712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac40a93a0 con 0x7faac40c1d60 2024-01-24T14:17:16.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.162+0000 7faacba24700 1 -- 172.21.15.29:0/2266264712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac4155250 con 0x7faac40c1d60 2024-01-24T14:17:16.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.162+0000 7faab97fa700 1 -- 172.21.15.29:0/2266264712 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7faab40068b0 con 0x7faac40c1d60 2024-01-24T14:17:16.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.163+0000 7faab97fa700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaa805f270 0x7faaa8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.163+0000 7faab97fa700 1 -- 172.21.15.29:0/2266264712 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6290+0+0 (secure 0 0 0) 0x7faab4079cd0 con 0x7faac40c1d60 2024-01-24T14:17:16.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.163+0000 7faac97c0700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaa805f270 0x7faaa8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.163+0000 7faac97c0700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaa805f270 0x7faaa8061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7faac0009130 tx=0x7faac0017000).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:16.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.167+0000 7faab97fa700 1 -- 172.21.15.29:0/2266264712 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7faac4155250 con 0x7faac40c1d60 2024-01-24T14:17:16.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.416+0000 7faacba24700 1 -- 172.21.15.29:0/2266264712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7faac40c2210 con 0x7faac40c1d60 2024-01-24T14:17:16.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.417+0000 7faab97fa700 1 -- 172.21.15.29:0/2266264712 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7faac40c2210 con 0x7faac40c1d60 2024-01-24T14:17:16.417 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew==", 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.418 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ==", 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw==", 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:17:16.419 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ==", 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ==", 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA==", 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.421 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw==", 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ==", 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw==", 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:17:16.423 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ==", 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.424 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A==", 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ==", 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw==", 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA==", 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.426 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g==", 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:17:16.427 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ==", 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-01-24T14:17:16.428 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlxK1bFxAAZpfJn+Tcpmq7R9Yq29FvVg==", 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw==", 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.429 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:16.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:16.430 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-01-24T14:17:16.430 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA==", 2024-01-24T14:17:16.430 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T14:17:16.430 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T14:17:16.430 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T14:17:16.430 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T14:17:16.430 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.430 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:16.430 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-01-24T14:17:16.430 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T14:17:16.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.418+0000 7faacba24700 1 -- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaa805f270 msgr2=0x7faaa8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.418+0000 7faacba24700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaa805f270 0x7faaa8061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7faac0009130 tx=0x7faac0017000).stop 2024-01-24T14:17:16.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.418+0000 7faacba24700 1 -- 172.21.15.29:0/2266264712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac40c1d60 msgr2=0x7faac40a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.418+0000 7faacba24700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac40c1d60 0x7faac40a6010 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7faab400c4c0 tx=0x7faab4007e00).stop 2024-01-24T14:17:16.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.419+0000 7faacba24700 1 -- 172.21.15.29:0/2266264712 shutdown_connections 2024-01-24T14:17:16.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.419+0000 7faacba24700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac414e840 0x7faac40a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.419+0000 7faacba24700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac4062ec0 0x7faac41351c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.419+0000 7faacba24700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faaa805f270 0x7faaa8061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.419+0000 7faacba24700 1 --2- 172.21.15.29:0/2266264712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac40c1d60 0x7faac40a6010 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.419+0000 7faacba24700 1 -- 172.21.15.29:0/2266264712 >> 172.21.15.29:0/2266264712 conn(0x7faac4064aa0 msgr2=0x7faac40a4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:16.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.419+0000 7faacba24700 1 -- 172.21.15.29:0/2266264712 shutdown_connections 2024-01-24T14:17:16.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.419+0000 7faacba24700 1 -- 172.21.15.29:0/2266264712 wait complete. 2024-01-24T14:17:16.433 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:17:16.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.562+0000 7fc94e19e700 1 Processor -- start 2024-01-24T14:17:16.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.563+0000 7fc94e19e700 1 -- start start 2024-01-24T14:17:16.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94e19e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc948150a80 0x7fc948150e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94e19e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc948069070 0x7fc9480694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94e19e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc9480c3140 0x7fc9480c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94e19e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9480c5f50 con 0x7fc948150a80 2024-01-24T14:17:16.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94e19e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9480699e0 con 0x7fc9480c3140 2024-01-24T14:17:16.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94e19e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480c3de0 con 0x7fc948069070 2024-01-24T14:17:16.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94d19c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc948150a80 0x7fc948150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94d19c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc948150a80 0x7fc948150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47690/0 (socket says 172.21.15.29:47690) 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94d19c700 1 -- 172.21.15.29:0/3150917348 learned_addr learned my addr 172.21.15.29:0/3150917348 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94d99d700 1 --2- 172.21.15.29:0/3150917348 >> v2:172.21.15.163:3300/0 conn(0x7fc9480c3140 0x7fc9480c35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94c99b700 1 --2- 172.21.15.29:0/3150917348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc948069070 0x7fc9480694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94d19c700 1 -- 172.21.15.29:0/3150917348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc948069070 msgr2=0x7fc9480694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94d19c700 1 --2- 172.21.15.29:0/3150917348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc948069070 0x7fc9480694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94d19c700 1 -- 172.21.15.29:0/3150917348 >> v2:172.21.15.163:3300/0 conn(0x7fc9480c3140 msgr2=0x7fc9480c35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94d19c700 1 --2- 172.21.15.29:0/3150917348 >> v2:172.21.15.163:3300/0 conn(0x7fc9480c3140 0x7fc9480c35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94d19c700 1 -- 172.21.15.29:0/3150917348 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480c5620 con 0x7fc948150a80 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94d99d700 1 --2- 172.21.15.29:0/3150917348 >> v2:172.21.15.163:3300/0 conn(0x7fc9480c3140 0x7fc9480c35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.564+0000 7fc94d19c700 1 --2- 172.21.15.29:0/3150917348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc948150a80 0x7fc948150e60 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7fc94806a6f0 tx=0x7fc93c0090f0).ready entity=mon.0 client_cookie=d62392ad1a5a5926 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.565+0000 7fc93bfff700 1 -- 172.21.15.29:0/3150917348 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00b600 con 0x7fc948150a80 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.565+0000 7fc93bfff700 1 -- 172.21.15.29:0/3150917348 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c00b760 con 0x7fc948150a80 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.565+0000 7fc93bfff700 1 -- 172.21.15.29:0/3150917348 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00bae0 con 0x7fc948150a80 2024-01-24T14:17:16.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.565+0000 7fc94e19e700 1 -- 172.21.15.29:0/3150917348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc948150a80 msgr2=0x7fc948150e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.565+0000 7fc94e19e700 1 --2- 172.21.15.29:0/3150917348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc948150a80 0x7fc948150e60 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7fc94806a6f0 tx=0x7fc93c0090f0).stop 2024-01-24T14:17:16.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.565+0000 7fc94e19e700 1 -- 172.21.15.29:0/3150917348 shutdown_connections 2024-01-24T14:17:16.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.565+0000 7fc94e19e700 1 --2- 172.21.15.29:0/3150917348 >> v2:172.21.15.163:3300/0 conn(0x7fc9480c3140 0x7fc9480c35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.565+0000 7fc94e19e700 1 --2- 172.21.15.29:0/3150917348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc948150a80 0x7fc948150e60 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.565+0000 7fc94e19e700 1 --2- 172.21.15.29:0/3150917348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc948069070 0x7fc9480694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.565+0000 7fc94e19e700 1 -- 172.21.15.29:0/3150917348 >> 172.21.15.29:0/3150917348 conn(0x7fc9480c43c0 msgr2=0x7fc9480c47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:16.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.565+0000 7fc94e19e700 1 -- 172.21.15.29:0/3150917348 shutdown_connections 2024-01-24T14:17:16.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.565+0000 7fc94e19e700 1 -- 172.21.15.29:0/3150917348 wait complete. 2024-01-24T14:17:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.566+0000 7fc94e19e700 1 Processor -- start 2024-01-24T14:17:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.567+0000 7fc94e19e700 1 -- start start 2024-01-24T14:17:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.567+0000 7fc94e19e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc948069070 0x7fc94814f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94e19e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9480c3140 0x7fc9480627b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94e19e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc948150a80 0x7fc948062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94e19e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc948063190 con 0x7fc948069070 2024-01-24T14:17:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94e19e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480c0bd0 con 0x7fc948150a80 2024-01-24T14:17:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94e19e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480c1470 con 0x7fc9480c3140 2024-01-24T14:17:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94d19c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc948069070 0x7fc94814f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94d19c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc948069070 0x7fc94814f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47700/0 (socket says 172.21.15.29:47700) 2024-01-24T14:17:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94d19c700 1 -- 172.21.15.29:0/3031876914 learned_addr learned my addr 172.21.15.29:0/3031876914 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:16.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94d99d700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc948150a80 0x7fc948062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94c99b700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9480c3140 0x7fc9480627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94d19c700 1 -- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9480c3140 msgr2=0x7fc9480627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94d19c700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9480c3140 0x7fc9480627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94d19c700 1 -- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc948150a80 msgr2=0x7fc948062ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94d19c700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc948150a80 0x7fc948062ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94d19c700 1 -- 172.21.15.29:0/3031876914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c009370 con 0x7fc948069070 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94d99d700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc948150a80 0x7fc948062ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94d19c700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc948069070 0x7fc94814f020 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7fc93c009130 tx=0x7fc93c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc9397fa700 1 -- 172.21.15.29:0/3031876914 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00bae0 con 0x7fc948069070 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc9397fa700 1 -- 172.21.15.29:0/3031876914 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c000c90 con 0x7fc948069070 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc94e19e700 1 -- 172.21.15.29:0/3031876914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9481631f0 con 0x7fc948069070 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.568+0000 7fc9397fa700 1 -- 172.21.15.29:0/3031876914 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c01c670 con 0x7fc948069070 2024-01-24T14:17:16.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.569+0000 7fc94e19e700 1 -- 172.21.15.29:0/3031876914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9480c1100 con 0x7fc948069070 2024-01-24T14:17:16.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.569+0000 7fc94e19e700 1 -- 172.21.15.29:0/3031876914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9480683b0 con 0x7fc948069070 2024-01-24T14:17:16.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.570+0000 7fc9397fa700 1 -- 172.21.15.29:0/3031876914 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc93c01ca20 con 0x7fc948069070 2024-01-24T14:17:16.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.570+0000 7fc9397fa700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc92c05f270 0x7fc92c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.570+0000 7fc9397fa700 1 -- 172.21.15.29:0/3031876914 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6290+0+0 (secure 0 0 0) 0x7fc93c07c2f0 con 0x7fc948069070 2024-01-24T14:17:16.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.571+0000 7fc94c99b700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc92c05f270 0x7fc92c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.571+0000 7fc94c99b700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc92c05f270 0x7fc92c061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc9480aa3c0 tx=0x7fc944008040).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:16.575 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.575+0000 7fc9397fa700 1 -- 172.21.15.29:0/3031876914 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc93c050dc0 con 0x7fc948069070 2024-01-24T14:17:16.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.756+0000 7fc94e19e700 1 -- 172.21.15.29:0/3031876914 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9480c1750 con 0x7fc948069070 2024-01-24T14:17:16.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.757+0000 7fc9397fa700 1 -- 172.21.15.29:0/3031876914 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+14965 (secure 0 0 0) 0x7fc93c02a090 con 0x7fc948069070 2024-01-24T14:17:16.760 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:16.760 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":149,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:17:15.992733+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:17:16.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.759+0000 7fc94e19e700 1 -- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc92c05f270 msgr2=0x7fc92c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.759+0000 7fc94e19e700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc92c05f270 0x7fc92c061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc9480aa3c0 tx=0x7fc944008040).stop 2024-01-24T14:17:16.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.759+0000 7fc94e19e700 1 -- 172.21.15.29:0/3031876914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc948069070 msgr2=0x7fc94814f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.759+0000 7fc94e19e700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc948069070 0x7fc94814f020 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7fc93c009130 tx=0x7fc93c017000).stop 2024-01-24T14:17:16.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.759+0000 7fc94e19e700 1 -- 172.21.15.29:0/3031876914 shutdown_connections 2024-01-24T14:17:16.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.759+0000 7fc94e19e700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc948150a80 0x7fc948062ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.759+0000 7fc94e19e700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc92c05f270 0x7fc92c061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.759+0000 7fc94e19e700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc948069070 0x7fc94814f020 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.759+0000 7fc94e19e700 1 --2- 172.21.15.29:0/3031876914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9480c3140 0x7fc9480627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.759+0000 7fc94e19e700 1 -- 172.21.15.29:0/3031876914 >> 172.21.15.29:0/3031876914 conn(0x7fc9480c43c0 msgr2=0x7fc9480ad3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:16.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.759+0000 7fc94e19e700 1 -- 172.21.15.29:0/3031876914 shutdown_connections 2024-01-24T14:17:16.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.759+0000 7fc94e19e700 1 -- 172.21.15.29:0/3031876914 wait complete. 2024-01-24T14:17:16.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T14:17:16.774 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T14:17:16.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.909+0000 7f6d36b72700 1 Processor -- start 2024-01-24T14:17:16.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.909+0000 7f6d36b72700 1 -- start start 2024-01-24T14:17:16.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.910+0000 7f6d36b72700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6d30153f40 0x7f6d300a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.910+0000 7f6d36b72700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6d30062ec0 0x7f6d30065440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.910+0000 7f6d36b72700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3006a360 con 0x7f6d300c8290 2024-01-24T14:17:16.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.910+0000 7f6d36b72700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d300bcef0 con 0x7f6d30062ec0 2024-01-24T14:17:16.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.910+0000 7f6d36b72700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d30069de0 con 0x7f6d30153f40 2024-01-24T14:17:16.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.910+0000 7f6d2ffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6d300c8290 0x7f6d300627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44100/0 (socket says 172.21.15.29:44100) 2024-01-24T14:17:16.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.910+0000 7f6d2ffff700 1 -- 172.21.15.29:0/1053150950 learned_addr learned my addr 172.21.15.29:0/1053150950 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:16.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.910+0000 7f6d3510f700 1 --2- 172.21.15.29:0/1053150950 >> v2:172.21.15.163:3300/0 conn(0x7f6d30062ec0 0x7f6d30065440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.910+0000 7f6d3490e700 1 --2- 172.21.15.29:0/1053150950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6d30153f40 0x7f6d300a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.911+0000 7f6d2f7fe700 1 -- 172.21.15.29:0/1053150950 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068000476 0 0) 0x7f6d3006a360 con 0x7f6d300c8290 2024-01-24T14:17:16.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.911+0000 7f6d2f7fe700 1 -- 172.21.15.29:0/1053150950 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d200033f0 con 0x7f6d300c8290 2024-01-24T14:17:16.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.911+0000 7f6d3490e700 1 -- 172.21.15.29:0/1053150950 >> v2:172.21.15.163:3300/0 conn(0x7f6d30062ec0 msgr2=0x7f6d30065440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:16.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.911+0000 7f6d3490e700 1 --2- 172.21.15.29:0/1053150950 >> v2:172.21.15.163:3300/0 conn(0x7f6d30062ec0 0x7f6d30065440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.911+0000 7f6d3490e700 1 -- 172.21.15.29:0/1053150950 >> v1:172.21.15.29:6789/0 conn(0x7f6d300c8290 legacy=0x7f6d300627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.911+0000 7f6d3490e700 1 -- 172.21.15.29:0/1053150950 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3006a080 con 0x7f6d30153f40 2024-01-24T14:17:16.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.911+0000 7f6d3510f700 1 --2- 172.21.15.29:0/1053150950 >> v2:172.21.15.163:3300/0 conn(0x7f6d30062ec0 0x7f6d30065440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:16.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.911+0000 7f6d3490e700 1 --2- 172.21.15.29:0/1053150950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6d30153f40 0x7f6d300a6db0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f6d24009760 tx=0x7f6d24009060).ready entity=mon.2 client_cookie=9c2ff589b08503a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:16.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.911+0000 7f6d2f7fe700 1 -- 172.21.15.29:0/1053150950 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d240090d0 con 0x7f6d30153f40 2024-01-24T14:17:16.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.911+0000 7f6d2f7fe700 1 -- 172.21.15.29:0/1053150950 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2400d4a0 con 0x7f6d30153f40 2024-01-24T14:17:16.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.911+0000 7f6d2f7fe700 1 -- 172.21.15.29:0/1053150950 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2400d820 con 0x7f6d30153f40 2024-01-24T14:17:16.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.912+0000 7f6d36b72700 1 -- 172.21.15.29:0/1053150950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6d30153f40 msgr2=0x7f6d300a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.912+0000 7f6d36b72700 1 --2- 172.21.15.29:0/1053150950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6d30153f40 0x7f6d300a6db0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f6d24009760 tx=0x7f6d24009060).stop 2024-01-24T14:17:16.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.912+0000 7f6d36b72700 1 -- 172.21.15.29:0/1053150950 shutdown_connections 2024-01-24T14:17:16.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.912+0000 7f6d36b72700 1 --2- 172.21.15.29:0/1053150950 >> v2:172.21.15.163:3300/0 conn(0x7f6d30062ec0 0x7f6d30065440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.912+0000 7f6d36b72700 1 --2- 172.21.15.29:0/1053150950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6d30153f40 0x7f6d300a6db0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.912+0000 7f6d36b72700 1 -- 172.21.15.29:0/1053150950 >> 172.21.15.29:0/1053150950 conn(0x7f6d30064390 msgr2=0x7f6d300c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:16.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.912+0000 7f6d36b72700 1 -- 172.21.15.29:0/1053150950 shutdown_connections 2024-01-24T14:17:16.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.912+0000 7f6d36b72700 1 -- 172.21.15.29:0/1053150950 wait complete. 2024-01-24T14:17:16.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.913+0000 7f6d36b72700 1 Processor -- start 2024-01-24T14:17:16.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.914+0000 7f6d36b72700 1 -- start start 2024-01-24T14:17:16.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d36b72700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d30062ec0 0x7f6d300a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d36b72700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d300c8290 0x7f6d301517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d36b72700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6d30153f40 0x7f6d30151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d36b72700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d30152190 con 0x7f6d300c8290 2024-01-24T14:17:16.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d36b72700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d30165740 con 0x7f6d30062ec0 2024-01-24T14:17:16.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d36b72700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d300a7890 con 0x7f6d30153f40 2024-01-24T14:17:16.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d2ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d300c8290 0x7f6d301517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d2ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d300c8290 0x7f6d301517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47702/0 (socket says 172.21.15.29:47702) 2024-01-24T14:17:16.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d2ffff700 1 -- 172.21.15.29:0/4041197407 learned_addr learned my addr 172.21.15.29:0/4041197407 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d3490e700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d30062ec0 0x7f6d300a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d3510f700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6d30153f40 0x7f6d30151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d2ffff700 1 -- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6d30153f40 msgr2=0x7f6d30151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d2ffff700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6d30153f40 0x7f6d30151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d2ffff700 1 -- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d30062ec0 msgr2=0x7f6d300a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d2ffff700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d30062ec0 0x7f6d300a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d2ffff700 1 -- 172.21.15.29:0/4041197407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d28009a20 con 0x7f6d300c8290 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.915+0000 7f6d3490e700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d30062ec0 0x7f6d300a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.916+0000 7f6d2ffff700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d300c8290 0x7f6d301517b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f6d1c00c410 tx=0x7f6d1c007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.916+0000 7f6d2cff9700 1 -- 172.21.15.29:0/4041197407 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00ce80 con 0x7f6d300c8290 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.916+0000 7f6d2cff9700 1 -- 172.21.15.29:0/4041197407 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c0072e0 con 0x7f6d300c8290 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.916+0000 7f6d36b72700 1 -- 172.21.15.29:0/4041197407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d240092a0 con 0x7f6d300c8290 2024-01-24T14:17:16.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.916+0000 7f6d2cff9700 1 -- 172.21.15.29:0/4041197407 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0057a0 con 0x7f6d300c8290 2024-01-24T14:17:16.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.916+0000 7f6d36b72700 1 -- 172.21.15.29:0/4041197407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d300a6ff0 con 0x7f6d300c8290 2024-01-24T14:17:16.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.917+0000 7f6d36b72700 1 -- 172.21.15.29:0/4041197407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d300475e0 con 0x7f6d300c8290 2024-01-24T14:17:16.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.917+0000 7f6d2cff9700 1 -- 172.21.15.29:0/4041197407 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6d1c005b90 con 0x7f6d300c8290 2024-01-24T14:17:16.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.918+0000 7f6d2cff9700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6d1405f270 0x7f6d14061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:16.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.918+0000 7f6d2cff9700 1 -- 172.21.15.29:0/4041197407 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f6d1c079b60 con 0x7f6d300c8290 2024-01-24T14:17:16.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.918+0000 7f6d3490e700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6d1405f270 0x7f6d14061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:16.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.919+0000 7f6d3490e700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6d1405f270 0x7f6d14061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f6d2400d260 tx=0x7f6d24016000).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:16.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:16.922+0000 7f6d2cff9700 1 -- 172.21.15.29:0/4041197407 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f6d1c04acc0 con 0x7f6d300c8290 2024-01-24T14:17:17.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.104+0000 7f6d36b72700 1 -- 172.21.15.29:0/4041197407 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6d300a76f0 con 0x7f6d300c8290 2024-01-24T14:17:17.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.104+0000 7f6d2cff9700 1 -- 172.21.15.29:0/4041197407 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v149) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6d300a76f0 con 0x7f6d300c8290 2024-01-24T14:17:17.105 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T14:17:17.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.106+0000 7f6d36b72700 1 -- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6d1405f270 msgr2=0x7f6d14061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.106+0000 7f6d36b72700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6d1405f270 0x7f6d14061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f6d2400d260 tx=0x7f6d24016000).stop 2024-01-24T14:17:17.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.106+0000 7f6d36b72700 1 -- 172.21.15.29:0/4041197407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d300c8290 msgr2=0x7f6d301517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.106+0000 7f6d36b72700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d300c8290 0x7f6d301517b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f6d1c00c410 tx=0x7f6d1c007bd0).stop 2024-01-24T14:17:17.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.106+0000 7f6d36b72700 1 -- 172.21.15.29:0/4041197407 shutdown_connections 2024-01-24T14:17:17.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.106+0000 7f6d36b72700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6d30153f40 0x7f6d30151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.106+0000 7f6d36b72700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d30062ec0 0x7f6d300a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.106+0000 7f6d36b72700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6d1405f270 0x7f6d14061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.106+0000 7f6d36b72700 1 --2- 172.21.15.29:0/4041197407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d300c8290 0x7f6d301517b0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.106+0000 7f6d36b72700 1 -- 172.21.15.29:0/4041197407 >> 172.21.15.29:0/4041197407 conn(0x7f6d30064390 msgr2=0x7f6d300acab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:17.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.106+0000 7f6d36b72700 1 -- 172.21.15.29:0/4041197407 shutdown_connections 2024-01-24T14:17:17.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.106+0000 7f6d36b72700 1 -- 172.21.15.29:0/4041197407 wait complete. 2024-01-24T14:17:17.121 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T14:17:17.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.253+0000 7feed7fec700 1 Processor -- start 2024-01-24T14:17:17.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.253+0000 7feed7fec700 1 -- start start 2024-01-24T14:17:17.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.254+0000 7feed7fec700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feed0157380 0x7feed0063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.254+0000 7feed7fec700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0064b60 0x7feed0064fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.254+0000 7feed7fec700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed00627b0 con 0x7feed0157760 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.254+0000 7feed7fec700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed0062d70 con 0x7feed0157380 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.254+0000 7feed7fec700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed00654c0 con 0x7feed0064b60 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.254+0000 7feed6589700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0064b60 0x7feed0064fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.254+0000 7feed5d88700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feed0157380 0x7feed0063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.254+0000 7feed6589700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0064b60 0x7feed0064fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:52318/0 (socket says 172.21.15.29:52318) 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.254+0000 7feed6589700 1 -- 172.21.15.29:0/771887585 learned_addr learned my addr 172.21.15.29:0/771887585 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.255+0000 7feed5d88700 1 -- 172.21.15.29:0/771887585 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0064b60 msgr2=0x7feed0064fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.255+0000 7feed5d88700 1 --2- 172.21.15.29:0/771887585 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0064b60 0x7feed0064fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.255+0000 7feed5d88700 1 -- 172.21.15.29:0/771887585 >> v1:172.21.15.163:6789/0 conn(0x7feed0157760 legacy=0x7feed0157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.255+0000 7feed5d88700 1 -- 172.21.15.29:0/771887585 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed01586e0 con 0x7feed0157380 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.255+0000 7feed4d86700 1 -- 172.21.15.29:0/771887585 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265236441 0 0) 0x7feed00627b0 con 0x7feed0157760 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.255+0000 7feed6589700 1 --2- 172.21.15.29:0/771887585 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0064b60 0x7feed0064fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:17.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.255+0000 7feed5d88700 1 --2- 172.21.15.29:0/771887585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feed0157380 0x7feed0063200 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7feecc009760 tx=0x7feecc009060).ready entity=mon.1 client_cookie=51547ecc3f50a6a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.255+0000 7feed4d86700 1 -- 172.21.15.29:0/771887585 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc0090d0 con 0x7feed0157380 2024-01-24T14:17:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.255+0000 7feed4d86700 1 -- 172.21.15.29:0/771887585 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc00f510 con 0x7feed0157380 2024-01-24T14:17:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.255+0000 7feed4d86700 1 -- 172.21.15.29:0/771887585 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc00f890 con 0x7feed0157380 2024-01-24T14:17:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.256+0000 7feed7fec700 1 -- 172.21.15.29:0/771887585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feed0157380 msgr2=0x7feed0063200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.256+0000 7feed7fec700 1 --2- 172.21.15.29:0/771887585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feed0157380 0x7feed0063200 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7feecc009760 tx=0x7feecc009060).stop 2024-01-24T14:17:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.256+0000 7feed7fec700 1 -- 172.21.15.29:0/771887585 shutdown_connections 2024-01-24T14:17:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.256+0000 7feed7fec700 1 --2- 172.21.15.29:0/771887585 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0064b60 0x7feed0064fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.256+0000 7feed7fec700 1 --2- 172.21.15.29:0/771887585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feed0157380 0x7feed0063200 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.256+0000 7feed7fec700 1 -- 172.21.15.29:0/771887585 >> 172.21.15.29:0/771887585 conn(0x7feed00c7260 msgr2=0x7feed00c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.256+0000 7feed7fec700 1 -- 172.21.15.29:0/771887585 shutdown_connections 2024-01-24T14:17:17.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.256+0000 7feed7fec700 1 -- 172.21.15.29:0/771887585 wait complete. 2024-01-24T14:17:17.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.257+0000 7feed7fec700 1 Processor -- start 2024-01-24T14:17:17.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.257+0000 7feed7fec700 1 -- start start 2024-01-24T14:17:17.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.258+0000 7feed7fec700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feed0064b60 0x7feed00c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.258+0000 7feed7fec700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0157380 0x7feed0143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.258+0000 7feed7fec700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed0157760 0x7feed0154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.258+0000 7feed7fec700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00c22e0 con 0x7feed0157760 2024-01-24T14:17:17.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.258+0000 7feed7fec700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00bf220 con 0x7feed0064b60 2024-01-24T14:17:17.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.258+0000 7feed7fec700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed00bf500 con 0x7feed0157380 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.258+0000 7feed5d88700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feed0064b60 0x7feed00c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.258+0000 7feed5587700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0157380 0x7feed0143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.258+0000 7feed5d88700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feed0064b60 0x7feed00c07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:46294/0 (socket says 172.21.15.29:46294) 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.258+0000 7feed5d88700 1 -- 172.21.15.29:0/3830674012 learned_addr learned my addr 172.21.15.29:0/3830674012 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feed6589700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed0157760 0x7feed0154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feed5d88700 1 -- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0157380 msgr2=0x7feed0143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feed5d88700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0157380 0x7feed0143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feed5d88700 1 -- 172.21.15.29:0/3830674012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed0157760 msgr2=0x7feed0154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feed5d88700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed0157760 0x7feed0154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feed5d88700 1 -- 172.21.15.29:0/3830674012 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec8009ae0 con 0x7feed0064b60 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feed5587700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0157380 0x7feed0143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feed6589700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed0157760 0x7feed0154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feed5d88700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feed0064b60 0x7feed00c07a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7feecc009e00 tx=0x7feecc017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:17.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feec5ffb700 1 -- 172.21.15.29:0/3830674012 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc00f890 con 0x7feed0064b60 2024-01-24T14:17:17.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feec5ffb700 1 -- 172.21.15.29:0/3830674012 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc00fc50 con 0x7feed0064b60 2024-01-24T14:17:17.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feed7fec700 1 -- 172.21.15.29:0/3830674012 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feecc0092a0 con 0x7feed0064b60 2024-01-24T14:17:17.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feec5ffb700 1 -- 172.21.15.29:0/3830674012 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc01b410 con 0x7feed0064b60 2024-01-24T14:17:17.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.259+0000 7feed7fec700 1 -- 172.21.15.29:0/3830674012 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed00bfac0 con 0x7feed0064b60 2024-01-24T14:17:17.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.260+0000 7feed7fec700 1 -- 172.21.15.29:0/3830674012 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed00c2610 con 0x7feed0064b60 2024-01-24T14:17:17.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.261+0000 7feec5ffb700 1 -- 172.21.15.29:0/3830674012 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7feecc01b7c0 con 0x7feed0064b60 2024-01-24T14:17:17.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.261+0000 7feec5ffb700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feeb405f270 0x7feeb4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.261+0000 7feec5ffb700 1 -- 172.21.15.29:0/3830674012 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6290+0+0 (secure 0 0 0) 0x7feecc07be60 con 0x7feed0064b60 2024-01-24T14:17:17.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.262+0000 7feed5587700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feeb405f270 0x7feeb4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.262+0000 7feed5587700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feeb405f270 0x7feeb4061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7feed0154280 tx=0x7feec0009040).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:17.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.266+0000 7feec5ffb700 1 -- 172.21.15.29:0/3830674012 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7feecc04cea0 con 0x7feed0064b60 2024-01-24T14:17:17.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.446+0000 7feed7fec700 1 -- 172.21.15.29:0/3830674012 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feed00c2450 con 0x7feed0064b60 2024-01-24T14:17:17.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.446+0000 7feec5ffb700 1 -- 172.21.15.29:0/3830674012 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v149) v1 ==== 92+0+10 (secure 0 0 0) 0x7feed00c2450 con 0x7feed0064b60 2024-01-24T14:17:17.447 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T14:17:17.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.448+0000 7feed7fec700 1 -- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feeb405f270 msgr2=0x7feeb4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.448+0000 7feed7fec700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feeb405f270 0x7feeb4061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7feed0154280 tx=0x7feec0009040).stop 2024-01-24T14:17:17.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.448+0000 7feed7fec700 1 -- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feed0064b60 msgr2=0x7feed00c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.448+0000 7feed7fec700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feed0064b60 0x7feed00c07a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7feecc009e00 tx=0x7feecc017000).stop 2024-01-24T14:17:17.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.449+0000 7feed7fec700 1 -- 172.21.15.29:0/3830674012 shutdown_connections 2024-01-24T14:17:17.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.449+0000 7feed7fec700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feeb405f270 0x7feeb4061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.449+0000 7feed7fec700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed0157760 0x7feed0154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.449+0000 7feed7fec700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feed0064b60 0x7feed00c07a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.449+0000 7feed7fec700 1 --2- 172.21.15.29:0/3830674012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feed0157380 0x7feed0143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.449+0000 7feed7fec700 1 -- 172.21.15.29:0/3830674012 >> 172.21.15.29:0/3830674012 conn(0x7feed00c7260 msgr2=0x7feed00aa8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:17.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.449+0000 7feed7fec700 1 -- 172.21.15.29:0/3830674012 shutdown_connections 2024-01-24T14:17:17.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.449+0000 7feed7fec700 1 -- 172.21.15.29:0/3830674012 wait complete. 2024-01-24T14:17:17.463 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-24T14:17:17.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.592+0000 7f792bcd3700 1 Processor -- start 2024-01-24T14:17:17.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.592+0000 7f792bcd3700 1 -- start start 2024-01-24T14:17:17.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.593+0000 7f792bcd3700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f79240c41c0 0x7f792415ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.593+0000 7f792bcd3700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7924064120 con 0x7f792413a5c0 2024-01-24T14:17:17.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.593+0000 7f792bcd3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79240bf6a0 con 0x7f79240be520 2024-01-24T14:17:17.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.593+0000 7f792bcd3700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79240bea90 con 0x7f79240c41c0 2024-01-24T14:17:17.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.593+0000 7f792a270700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f79240c41c0 0x7f792415ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.593+0000 7f7929a6f700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f792413a5c0 0x7f7924063a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44102/0 (socket says 172.21.15.29:44102) 2024-01-24T14:17:17.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.593+0000 7f7929a6f700 1 -- 172.21.15.29:0/2763598118 learned_addr learned my addr 172.21.15.29:0/2763598118 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:17.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.594+0000 7f792a270700 1 -- 172.21.15.29:0/2763598118 >> v1:172.21.15.163:6789/0 conn(0x7f79240be520 legacy=0x7f79240664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.594+0000 7f792a270700 1 -- 172.21.15.29:0/2763598118 >> v1:172.21.15.29:6789/0 conn(0x7f792413a5c0 legacy=0x7f7924063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.594+0000 7f7928a6d700 1 -- 172.21.15.29:0/2763598118 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687118844 0 0) 0x7f7924064120 con 0x7f792413a5c0 2024-01-24T14:17:17.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.594+0000 7f792a270700 1 -- 172.21.15.29:0/2763598118 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79240bed70 con 0x7f79240c41c0 2024-01-24T14:17:17.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.594+0000 7f792a270700 1 --2- 172.21.15.29:0/2763598118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79240c41c0 0x7f792415ec60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f791c009e60 tx=0x7f791c009780).ready entity=mon.0 client_cookie=de4035a09a045801 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:17.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.594+0000 7f7928a6d700 1 -- 172.21.15.29:0/2763598118 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c00e070 con 0x7f79240c41c0 2024-01-24T14:17:17.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.594+0000 7f7928a6d700 1 -- 172.21.15.29:0/2763598118 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c00bc30 con 0x7f79240c41c0 2024-01-24T14:17:17.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.594+0000 7f7928a6d700 1 -- 172.21.15.29:0/2763598118 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c012410 con 0x7f79240c41c0 2024-01-24T14:17:17.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.595+0000 7f792bcd3700 1 -- 172.21.15.29:0/2763598118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79240c41c0 msgr2=0x7f792415ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.595+0000 7f792bcd3700 1 --2- 172.21.15.29:0/2763598118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79240c41c0 0x7f792415ec60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f791c009e60 tx=0x7f791c009780).stop 2024-01-24T14:17:17.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.595+0000 7f792bcd3700 1 -- 172.21.15.29:0/2763598118 shutdown_connections 2024-01-24T14:17:17.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.595+0000 7f792bcd3700 1 --2- 172.21.15.29:0/2763598118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79240c41c0 0x7f792415ec60 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.595+0000 7f792bcd3700 1 -- 172.21.15.29:0/2763598118 >> 172.21.15.29:0/2763598118 conn(0x7f79240c8bb0 msgr2=0x7f79240c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:17.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.595+0000 7f792bcd3700 1 -- 172.21.15.29:0/2763598118 shutdown_connections 2024-01-24T14:17:17.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.595+0000 7f792bcd3700 1 -- 172.21.15.29:0/2763598118 wait complete. 2024-01-24T14:17:17.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.596+0000 7f792bcd3700 1 Processor -- start 2024-01-24T14:17:17.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.596+0000 7f792bcd3700 1 -- start start 2024-01-24T14:17:17.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792bcd3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79240be520 0x7f7924167780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792bcd3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79240c41c0 0x7f7924139430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792bcd3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f792413a5c0 0x7f7924151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792bcd3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79241549b0 con 0x7f792413a5c0 2024-01-24T14:17:17.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792bcd3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7924167fd0 con 0x7f79240c41c0 2024-01-24T14:17:17.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792bcd3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79241682b0 con 0x7f79240be520 2024-01-24T14:17:17.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792a270700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f792413a5c0 0x7f7924151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792a270700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f792413a5c0 0x7f7924151e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47718/0 (socket says 172.21.15.29:47718) 2024-01-24T14:17:17.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792a270700 1 -- 172.21.15.29:0/171240782 learned_addr learned my addr 172.21.15.29:0/171240782 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:17.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f7929a6f700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79240be520 0x7f7924167780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792926e700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79240c41c0 0x7f7924139430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792a270700 1 -- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79240be520 msgr2=0x7f7924167780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792a270700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79240be520 0x7f7924167780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792a270700 1 -- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79240c41c0 msgr2=0x7f7924139430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792a270700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79240c41c0 0x7f7924139430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792a270700 1 -- 172.21.15.29:0/171240782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c009a20 con 0x7f792413a5c0 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792926e700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79240c41c0 0x7f7924139430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.597+0000 7f792a270700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f792413a5c0 0x7f7924151e40 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f791c00b6c0 tx=0x7f791c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.598+0000 7f7915ffb700 1 -- 172.21.15.29:0/171240782 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c0064d0 con 0x7f792413a5c0 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.598+0000 7f7915ffb700 1 -- 172.21.15.29:0/171240782 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c006630 con 0x7f792413a5c0 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.598+0000 7f7915ffb700 1 -- 172.21.15.29:0/171240782 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c0069d0 con 0x7f792413a5c0 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.598+0000 7f792bcd3700 1 -- 172.21.15.29:0/171240782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7924168590 con 0x7f792413a5c0 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.598+0000 7f792bcd3700 1 -- 172.21.15.29:0/171240782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7924167c70 con 0x7f792413a5c0 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.599+0000 7f792bcd3700 1 -- 172.21.15.29:0/171240782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79240475e0 con 0x7f792413a5c0 2024-01-24T14:17:17.601 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.599+0000 7f7915ffb700 1 -- 172.21.15.29:0/171240782 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f791c01d630 con 0x7f792413a5c0 2024-01-24T14:17:17.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.600+0000 7f7915ffb700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f790805f270 0x7f7908061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.600+0000 7f7915ffb700 1 -- 172.21.15.29:0/171240782 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f791c07cb00 con 0x7f792413a5c0 2024-01-24T14:17:17.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.600+0000 7f7929a6f700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f790805f270 0x7f7908061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.600+0000 7f7929a6f700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f790805f270 0x7f7908061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7918007fd0 tx=0x7f7918009040).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:17.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.604+0000 7f7915ffb700 1 -- 172.21.15.29:0/171240782 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f791c04db20 con 0x7f792413a5c0 2024-01-24T14:17:17.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.789+0000 7f792bcd3700 1 -- 172.21.15.29:0/171240782 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7924154af0 con 0x7f792413a5c0 2024-01-24T14:17:17.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.789+0000 7f7915ffb700 1 -- 172.21.15.29:0/171240782 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v149) v1 ==== 84+0+143 (secure 0 0 0) 0x7f791c07c020 con 0x7f792413a5c0 2024-01-24T14:17:17.790 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:17.790 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T14:17:17.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:17.790 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-01-24T14:17:17.790 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "device_health_metrics" 2024-01-24T14:17:17.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T14:17:17.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:17.791 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 2, 2024-01-24T14:17:17.791 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "rbd" 2024-01-24T14:17:17.791 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:17.791 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T14:17:17.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.791+0000 7f792bcd3700 1 -- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f790805f270 msgr2=0x7f7908061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.791+0000 7f792bcd3700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f790805f270 0x7f7908061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7918007fd0 tx=0x7f7918009040).stop 2024-01-24T14:17:17.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.791+0000 7f792bcd3700 1 -- 172.21.15.29:0/171240782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f792413a5c0 msgr2=0x7f7924151e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.791+0000 7f792bcd3700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f792413a5c0 0x7f7924151e40 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f791c00b6c0 tx=0x7f791c009780).stop 2024-01-24T14:17:17.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.791+0000 7f792bcd3700 1 -- 172.21.15.29:0/171240782 shutdown_connections 2024-01-24T14:17:17.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.791+0000 7f792bcd3700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f790805f270 0x7f7908061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.791+0000 7f792bcd3700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f792413a5c0 0x7f7924151e40 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.791+0000 7f792bcd3700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79240be520 0x7f7924167780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.791+0000 7f792bcd3700 1 --2- 172.21.15.29:0/171240782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79240c41c0 0x7f7924139430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.791+0000 7f792bcd3700 1 -- 172.21.15.29:0/171240782 >> 172.21.15.29:0/171240782 conn(0x7f79240c8bb0 msgr2=0x7f7924153a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:17.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.791+0000 7f792bcd3700 1 -- 172.21.15.29:0/171240782 shutdown_connections 2024-01-24T14:17:17.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.791+0000 7f792bcd3700 1 -- 172.21.15.29:0/171240782 wait complete. 2024-01-24T14:17:17.806 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-24T14:17:17.806 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2024-01-24T14:17:17.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.940+0000 7f886cf30700 1 Processor -- start 2024-01-24T14:17:17.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.940+0000 7f886cf30700 1 -- start start 2024-01-24T14:17:17.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.941+0000 7f886cf30700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8868151c70 0x7f8868152050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.941+0000 7f886cf30700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8868156410 0x7f8868069ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.941+0000 7f886cf30700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f886806a3d0 0x7f88680627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.941+0000 7f886cf30700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8868069a50 con 0x7f8868156410 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.941+0000 7f886cf30700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88680a77d0 con 0x7f886806a3d0 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.941+0000 7f886cf30700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88680a7ab0 con 0x7f8868151c70 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.941+0000 7f8865d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8868156410 0x7f8868069ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.941+0000 7f8865d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8868156410 0x7f8868069ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47732/0 (socket says 172.21.15.29:47732) 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.941+0000 7f8865d9b700 1 -- 172.21.15.29:0/1783374962 learned_addr learned my addr 172.21.15.29:0/1783374962 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.942+0000 7f8866d9d700 1 --2- 172.21.15.29:0/1783374962 >> v2:172.21.15.163:3300/0 conn(0x7f886806a3d0 0x7f88680627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.942+0000 7f886659c700 1 --2- 172.21.15.29:0/1783374962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8868151c70 0x7f8868152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.942+0000 7f8865d9b700 1 -- 172.21.15.29:0/1783374962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8868151c70 msgr2=0x7f8868152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.942+0000 7f8865d9b700 1 --2- 172.21.15.29:0/1783374962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8868151c70 0x7f8868152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.942+0000 7f8865d9b700 1 -- 172.21.15.29:0/1783374962 >> v2:172.21.15.163:3300/0 conn(0x7f886806a3d0 msgr2=0x7f88680627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:17.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.942+0000 7f8865d9b700 1 --2- 172.21.15.29:0/1783374962 >> v2:172.21.15.163:3300/0 conn(0x7f886806a3d0 0x7f88680627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.942+0000 7f8865d9b700 1 -- 172.21.15.29:0/1783374962 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8868069400 con 0x7f8868156410 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.942+0000 7f8865d9b700 1 --2- 172.21.15.29:0/1783374962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8868156410 0x7f8868069ee0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f88681559a0 tx=0x7f88500090f0).ready entity=mon.0 client_cookie=a3a309323ad7f37e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.942+0000 7f886559a700 1 -- 172.21.15.29:0/1783374962 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885000b600 con 0x7f8868156410 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.942+0000 7f886559a700 1 -- 172.21.15.29:0/1783374962 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885000b760 con 0x7f8868156410 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.942+0000 7f886559a700 1 -- 172.21.15.29:0/1783374962 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885000bae0 con 0x7f8868156410 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.943+0000 7f886cf30700 1 -- 172.21.15.29:0/1783374962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8868156410 msgr2=0x7f8868069ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.943+0000 7f886cf30700 1 --2- 172.21.15.29:0/1783374962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8868156410 0x7f8868069ee0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f88681559a0 tx=0x7f88500090f0).stop 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.943+0000 7f886cf30700 1 -- 172.21.15.29:0/1783374962 shutdown_connections 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.943+0000 7f886cf30700 1 --2- 172.21.15.29:0/1783374962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8868151c70 0x7f8868152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.943+0000 7f886cf30700 1 --2- 172.21.15.29:0/1783374962 >> v2:172.21.15.163:3300/0 conn(0x7f886806a3d0 0x7f88680627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.943+0000 7f886cf30700 1 --2- 172.21.15.29:0/1783374962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8868156410 0x7f8868069ee0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.943+0000 7f886cf30700 1 -- 172.21.15.29:0/1783374962 >> 172.21.15.29:0/1783374962 conn(0x7f8868067b20 msgr2=0x7f88680638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.943+0000 7f886cf30700 1 -- 172.21.15.29:0/1783374962 shutdown_connections 2024-01-24T14:17:17.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.943+0000 7f886cf30700 1 -- 172.21.15.29:0/1783374962 wait complete. 2024-01-24T14:17:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.944+0000 7f886cf30700 1 Processor -- start 2024-01-24T14:17:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.944+0000 7f886cf30700 1 -- start start 2024-01-24T14:17:17.947 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f886cf30700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f886806a3d0 0x7f8868164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f886cf30700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8868151c70 0x7f88680c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f886cf30700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8868156410 0x7f88680a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f886cf30700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88680a5880 con 0x7f8868151c70 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f886cf30700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88681650c0 con 0x7f8868156410 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f886cf30700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88680c3cb0 con 0x7f886806a3d0 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f8865d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8868151c70 0x7f88680c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f8865d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8868151c70 0x7f88680c37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47742/0 (socket says 172.21.15.29:47742) 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f8865d9b700 1 -- 172.21.15.29:0/678671286 learned_addr learned my addr 172.21.15.29:0/678671286 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f886659c700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f886806a3d0 0x7f8868164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f8866d9d700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8868156410 0x7f88680a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f8865d9b700 1 -- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f886806a3d0 msgr2=0x7f8868164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f8865d9b700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f886806a3d0 0x7f8868164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f8865d9b700 1 -- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8868156410 msgr2=0x7f88680a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f8865d9b700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8868156410 0x7f88680a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.945+0000 7f8865d9b700 1 -- 172.21.15.29:0/678671286 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8850009370 con 0x7f8868151c70 2024-01-24T14:17:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.946+0000 7f8865d9b700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8868151c70 0x7f88680c37c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f8850009130 tx=0x7f8850017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.946+0000 7f885e7fc700 1 -- 172.21.15.29:0/678671286 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885000bea0 con 0x7f8868151c70 2024-01-24T14:17:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.946+0000 7f885e7fc700 1 -- 172.21.15.29:0/678671286 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885000b8d0 con 0x7f8868151c70 2024-01-24T14:17:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.946+0000 7f885e7fc700 1 -- 172.21.15.29:0/678671286 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885001b640 con 0x7f8868151c70 2024-01-24T14:17:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.946+0000 7f886cf30700 1 -- 172.21.15.29:0/678671286 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88680a4c70 con 0x7f8868151c70 2024-01-24T14:17:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.946+0000 7f886cf30700 1 -- 172.21.15.29:0/678671286 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88680a5230 con 0x7f8868151c70 2024-01-24T14:17:17.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.947+0000 7f886cf30700 1 -- 172.21.15.29:0/678671286 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88680a5a30 con 0x7f8868151c70 2024-01-24T14:17:17.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.947+0000 7f885e7fc700 1 -- 172.21.15.29:0/678671286 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f885001b9b0 con 0x7f8868151c70 2024-01-24T14:17:17.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.948+0000 7f885e7fc700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f884805f270 0x7f8848061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:17.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.948+0000 7f885e7fc700 1 -- 172.21.15.29:0/678671286 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f885007bbe0 con 0x7f8868151c70 2024-01-24T14:17:17.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.948+0000 7f886659c700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f884805f270 0x7f8848061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:17.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.949+0000 7f886659c700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f884805f270 0x7f8848061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f886814ef90 tx=0x7f8858008040).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:17.952 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:17.952+0000 7f885e7fc700 1 -- 172.21.15.29:0/678671286 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f88680a5a30 con 0x7f8868151c70 2024-01-24T14:17:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.136+0000 7f886cf30700 1 -- 172.21.15.29:0/678671286 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f88680475e0 con 0x7f8868151c70 2024-01-24T14:17:18.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.994+0000 7f885e7fc700 1 -- 172.21.15.29:0/678671286 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v150) v1 ==== 152+0+0 (secure 0 0 0) 0x7f885004cbc0 con 0x7f8868151c70 2024-01-24T14:17:18.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.997+0000 7f886cf30700 1 -- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f884805f270 msgr2=0x7f8848061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:18.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.998+0000 7f886cf30700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f884805f270 0x7f8848061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f886814ef90 tx=0x7f8858008040).stop 2024-01-24T14:17:18.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.998+0000 7f886cf30700 1 -- 172.21.15.29:0/678671286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8868151c70 msgr2=0x7f88680c37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:18.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.998+0000 7f886cf30700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8868151c70 0x7f88680c37c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f8850009130 tx=0x7f8850017000).stop 2024-01-24T14:17:18.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.998+0000 7f886cf30700 1 -- 172.21.15.29:0/678671286 shutdown_connections 2024-01-24T14:17:18.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.998+0000 7f886cf30700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8868156410 0x7f88680a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:18.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.998+0000 7f886cf30700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f886806a3d0 0x7f8868164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:18.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.998+0000 7f886cf30700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f884805f270 0x7f8848061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:18.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.998+0000 7f886cf30700 1 --2- 172.21.15.29:0/678671286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8868151c70 0x7f88680c37c0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:18.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.998+0000 7f886cf30700 1 -- 172.21.15.29:0/678671286 >> 172.21.15.29:0/678671286 conn(0x7f8868067b20 msgr2=0x7f88680c5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:19.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.998+0000 7f886cf30700 1 -- 172.21.15.29:0/678671286 shutdown_connections 2024-01-24T14:17:19.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:18.999+0000 7f886cf30700 1 -- 172.21.15.29:0/678671286 wait complete. 2024-01-24T14:17:19.004 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-01-24T14:17:19.020 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-24T14:17:19.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.159+0000 7f7d1642f700 1 Processor -- start 2024-01-24T14:17:19.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.159+0000 7f7d1642f700 1 -- start start 2024-01-24T14:17:19.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.160+0000 7f7d1642f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7d10064c40 0x7f7d10065080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:19.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.160+0000 7f7d1642f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1005e790 con 0x7f7d10152740 2024-01-24T14:17:19.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.160+0000 7f7d1642f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d100ac0f0 con 0x7f7d100aa4f0 2024-01-24T14:17:19.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.160+0000 7f7d1642f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d100aac60 con 0x7f7d10064c40 2024-01-24T14:17:19.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.161+0000 7f7d0ffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7d10152740 0x7f7d100ace70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44104/0 (socket says 172.21.15.29:44104) 2024-01-24T14:17:19.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.161+0000 7f7d0ffff700 1 -- 172.21.15.29:0/3565604786 learned_addr learned my addr 172.21.15.29:0/3565604786 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:19.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.161+0000 7f7d149cc700 1 --2- 172.21.15.29:0/3565604786 >> v2:172.21.15.163:3300/0 conn(0x7f7d10064c40 0x7f7d10065080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:19.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.161+0000 7f7d0effd700 1 -- 172.21.15.29:0/3565604786 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347090203 0 0) 0x7f7d1005e790 con 0x7f7d10152740 2024-01-24T14:17:19.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.161+0000 7f7d0effd700 1 -- 172.21.15.29:0/3565604786 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cf80033f0 con 0x7f7d10152740 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.161+0000 7f7d0effd700 1 -- 172.21.15.29:0/3565604786 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 733524825 0 0) 0x7f7d100ac0f0 con 0x7f7d100aa4f0 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.161+0000 7f7d0effd700 1 -- 172.21.15.29:0/3565604786 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d1005e790 con 0x7f7d100aa4f0 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.161+0000 7f7d0effd700 1 -- 172.21.15.29:0/3565604786 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2316708709 0 0) 0x7f7cf80033f0 con 0x7f7d10152740 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.161+0000 7f7d149cc700 1 -- 172.21.15.29:0/3565604786 >> v1:172.21.15.163:6789/0 conn(0x7f7d100aa4f0 legacy=0x7f7d10152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.161+0000 7f7d149cc700 1 -- 172.21.15.29:0/3565604786 >> v1:172.21.15.29:6789/0 conn(0x7f7d10152740 legacy=0x7f7d100ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.161+0000 7f7d149cc700 1 -- 172.21.15.29:0/3565604786 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d10065570 con 0x7f7d10064c40 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.161+0000 7f7d149cc700 1 --2- 172.21.15.29:0/3565604786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d10064c40 0x7f7d10065080 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f7d08009e60 tx=0x7f7d08009780).ready entity=mon.1 client_cookie=4b23eb3719c07692 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.162+0000 7f7d0effd700 1 -- 172.21.15.29:0/3565604786 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0800e070 con 0x7f7d10064c40 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.162+0000 7f7d0effd700 1 -- 172.21.15.29:0/3565604786 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0800b7a0 con 0x7f7d10064c40 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.162+0000 7f7d0effd700 1 -- 172.21.15.29:0/3565604786 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000015f0 con 0x7f7d10064c40 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.162+0000 7f7d1642f700 1 -- 172.21.15.29:0/3565604786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d10064c40 msgr2=0x7f7d10065080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.162+0000 7f7d1642f700 1 --2- 172.21.15.29:0/3565604786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d10064c40 0x7f7d10065080 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f7d08009e60 tx=0x7f7d08009780).stop 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.162+0000 7f7d1642f700 1 -- 172.21.15.29:0/3565604786 shutdown_connections 2024-01-24T14:17:19.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.162+0000 7f7d1642f700 1 --2- 172.21.15.29:0/3565604786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d10064c40 0x7f7d10065080 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:19.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.162+0000 7f7d1642f700 1 -- 172.21.15.29:0/3565604786 >> 172.21.15.29:0/3565604786 conn(0x7f7d100c7330 msgr2=0x7f7d100c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:19.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.162+0000 7f7d1642f700 1 -- 172.21.15.29:0/3565604786 shutdown_connections 2024-01-24T14:17:19.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.162+0000 7f7d1642f700 1 -- 172.21.15.29:0/3565604786 wait complete. 2024-01-24T14:17:19.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.163+0000 7f7d1642f700 1 Processor -- start 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.164+0000 7f7d1642f700 1 -- start start 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.164+0000 7f7d1642f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d10064c40 0x7f7d1014e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.164+0000 7f7d1642f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d10152740 0x7f7d100c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.164+0000 7f7d1642f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d100c0ab0 0x7f7d100c0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.164+0000 7f7d1642f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100c2950 con 0x7f7d100c0ab0 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.164+0000 7f7d1642f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100c32d0 con 0x7f7d10064c40 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.164+0000 7f7d1642f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100c1d40 con 0x7f7d10152740 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7d149cc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d100c0ab0 0x7f7d100c0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7d149cc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d100c0ab0 0x7f7d100c0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47754/0 (socket says 172.21.15.29:47754) 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7d149cc700 1 -- 172.21.15.29:0/1887351512 learned_addr learned my addr 172.21.15.29:0/1887351512 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7d0ffff700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d10064c40 0x7f7d1014e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7d0f7fe700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d10152740 0x7f7d100c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7d149cc700 1 -- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d10152740 msgr2=0x7f7d100c2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:19.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7d149cc700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d10152740 0x7f7d100c2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:19.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7d149cc700 1 -- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d10064c40 msgr2=0x7f7d1014e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:19.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7d149cc700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d10064c40 0x7f7d1014e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:19.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7d149cc700 1 -- 172.21.15.29:0/1887351512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d08009a20 con 0x7f7d100c0ab0 2024-01-24T14:17:19.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7d149cc700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d100c0ab0 0x7f7d100c0f30 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f7d08012dd0 tx=0x7f7d08009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:19.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7cf7fff700 1 -- 172.21.15.29:0/1887351512 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0801d440 con 0x7f7d100c0ab0 2024-01-24T14:17:19.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7cf7fff700 1 -- 172.21.15.29:0/1887351512 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0801d5a0 con 0x7f7d100c0ab0 2024-01-24T14:17:19.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7cf7fff700 1 -- 172.21.15.29:0/1887351512 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0801d920 con 0x7f7d100c0ab0 2024-01-24T14:17:19.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.165+0000 7f7d1642f700 1 -- 172.21.15.29:0/1887351512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d100c2020 con 0x7f7d100c0ab0 2024-01-24T14:17:19.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.166+0000 7f7d1642f700 1 -- 172.21.15.29:0/1887351512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d100c25e0 con 0x7f7d100c0ab0 2024-01-24T14:17:19.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.167+0000 7f7d1642f700 1 -- 172.21.15.29:0/1887351512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d100c2b00 con 0x7f7d100c0ab0 2024-01-24T14:17:19.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.167+0000 7f7cf7fff700 1 -- 172.21.15.29:0/1887351512 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7d080263b0 con 0x7f7d100c0ab0 2024-01-24T14:17:19.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.167+0000 7f7cf7fff700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7cfc05f270 0x7f7cfc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:19.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.168+0000 7f7cf7fff700 1 -- 172.21.15.29:0/1887351512 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6660+0+0 (secure 0 0 0) 0x7f7d0807bf60 con 0x7f7d100c0ab0 2024-01-24T14:17:19.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.168+0000 7f7d0ffff700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7cfc05f270 0x7f7cfc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:19.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.168+0000 7f7d0ffff700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7cfc05f270 0x7f7cfc061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f7d00001130 tx=0x7f7d000017a0).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:19.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.172+0000 7f7cf7fff700 1 -- 172.21.15.29:0/1887351512 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7d080aab00 con 0x7f7d100c0ab0 2024-01-24T14:17:19.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:19.358+0000 7f7d1642f700 1 -- 172.21.15.29:0/1887351512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7d100475e0 con 0x7f7d100c0ab0 2024-01-24T14:17:20.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.002+0000 7f7cf7fff700 1 -- 172.21.15.29:0/1887351512 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v151) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7d08022070 con 0x7f7d100c0ab0 2024-01-24T14:17:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.008+0000 7f7d1642f700 1 -- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7cfc05f270 msgr2=0x7f7cfc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.008+0000 7f7d1642f700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7cfc05f270 0x7f7cfc061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f7d00001130 tx=0x7f7d000017a0).stop 2024-01-24T14:17:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.008+0000 7f7d1642f700 1 -- 172.21.15.29:0/1887351512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d100c0ab0 msgr2=0x7f7d100c0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.008+0000 7f7d1642f700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d100c0ab0 0x7f7d100c0f30 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f7d08012dd0 tx=0x7f7d08009780).stop 2024-01-24T14:17:20.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.008+0000 7f7d1642f700 1 -- 172.21.15.29:0/1887351512 shutdown_connections 2024-01-24T14:17:20.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.008+0000 7f7d1642f700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7cfc05f270 0x7f7cfc061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.008+0000 7f7d1642f700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d100c0ab0 0x7f7d100c0f30 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.008+0000 7f7d1642f700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d10064c40 0x7f7d1014e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.008+0000 7f7d1642f700 1 --2- 172.21.15.29:0/1887351512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d10152740 0x7f7d100c2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.008+0000 7f7d1642f700 1 -- 172.21.15.29:0/1887351512 >> 172.21.15.29:0/1887351512 conn(0x7f7d100c7330 msgr2=0x7f7d10068ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:20.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.010+0000 7f7d1642f700 1 -- 172.21.15.29:0/1887351512 shutdown_connections 2024-01-24T14:17:20.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.010+0000 7f7d1642f700 1 -- 172.21.15.29:0/1887351512 wait complete. 2024-01-24T14:17:20.015 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-01-24T14:17:20.040 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-24T14:17:20.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.174+0000 7ff5e83f1700 1 Processor -- start 2024-01-24T14:17:20.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.174+0000 7ff5e83f1700 1 -- start start 2024-01-24T14:17:20.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e83f1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e0062ec0 0x7ff5e006a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e83f1700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e00bcda0 con 0x7ff5e01384f0 2024-01-24T14:17:20.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e83f1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e0065f80 con 0x7ff5e00c8270 2024-01-24T14:17:20.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e83f1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e0066260 con 0x7ff5e0062ec0 2024-01-24T14:17:20.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e618d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff5e01384f0 0x7ff5e00627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44106/0 (socket says 172.21.15.29:44106) 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e618d700 1 -- 172.21.15.29:0/1757764820 learned_addr learned my addr 172.21.15.29:0/1757764820 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e598c700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e0062ec0 0x7ff5e006a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819286512 0 0) 0x7ff5e00bcda0 con 0x7ff5e01384f0 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5cc0033f0 con 0x7ff5e01384f0 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097363651 0 0) 0x7ff5e0065f80 con 0x7ff5e00c8270 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5e00bcda0 con 0x7ff5e00c8270 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2537110916 0 0) 0x7ff5cc0033f0 con 0x7ff5e01384f0 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e0062ec0 msgr2=0x7ff5e006a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e518b700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e0062ec0 0x7ff5e006a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 >> v1:172.21.15.163:6789/0 conn(0x7ff5e00c8270 legacy=0x7ff5e01391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e00bca50 con 0x7ff5e01384f0 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.176+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7ff5d4001470 con 0x7ff5e01384f0 2024-01-24T14:17:20.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 >> v1:172.21.15.29:6789/0 conn(0x7ff5e01384f0 legacy=0x7ff5e00627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e518b700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e0062ec0 0x7ff5cc003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e518b700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc003fb0 0x7ff5cc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e518b700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5cc004b80 0x7ff5cc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e0066260 con 0x7ff5cc003fb0 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc00b9f0 con 0x7ff5cc004b80 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc00bbb0 con 0x7ff5e0062ec0 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e598c700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc003fb0 0x7ff5cc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e618d700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e0062ec0 0x7ff5cc003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e698e700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5cc004b80 0x7ff5cc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e598c700 1 -- 172.21.15.29:0/1757764820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e0062ec0 msgr2=0x7ff5cc003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e598c700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e0062ec0 0x7ff5cc003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e598c700 1 -- 172.21.15.29:0/1757764820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5cc004b80 msgr2=0x7ff5cc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e598c700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5cc004b80 0x7ff5cc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e598c700 1 -- 172.21.15.29:0/1757764820 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc0033f0 con 0x7ff5cc003fb0 2024-01-24T14:17:20.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e698e700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5cc004b80 0x7ff5cc005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e598c700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc003fb0 0x7ff5cc004690 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7ff5e00632c0 tx=0x7ff5dc009060).ready entity=mon.0 client_cookie=62d446d2b19bddf7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc009430 con 0x7ff5cc003fb0 2024-01-24T14:17:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5dc00e640 con 0x7ff5cc003fb0 2024-01-24T14:17:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.177+0000 7ff5e518b700 1 -- 172.21.15.29:0/1757764820 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc00e9c0 con 0x7ff5cc003fb0 2024-01-24T14:17:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.178+0000 7ff5e83f1700 1 -- 172.21.15.29:0/1757764820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc003fb0 msgr2=0x7ff5cc004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.178+0000 7ff5e83f1700 1 --2- 172.21.15.29:0/1757764820 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc003fb0 0x7ff5cc004690 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7ff5e00632c0 tx=0x7ff5dc009060).stop 2024-01-24T14:17:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.178+0000 7ff5e698e700 1 -- 172.21.15.29:0/1757764820 reap_dead start 2024-01-24T14:17:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.178+0000 7ff5e83f1700 1 -- 172.21.15.29:0/1757764820 shutdown_connections 2024-01-24T14:17:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.178+0000 7ff5e83f1700 1 -- 172.21.15.29:0/1757764820 >> 172.21.15.29:0/1757764820 conn(0x7ff5e0064390 msgr2=0x7ff5e00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.178+0000 7ff5e83f1700 1 -- 172.21.15.29:0/1757764820 shutdown_connections 2024-01-24T14:17:20.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.178+0000 7ff5e83f1700 1 -- 172.21.15.29:0/1757764820 wait complete. 2024-01-24T14:17:20.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.179+0000 7ff5e83f1700 1 Processor -- start 2024-01-24T14:17:20.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.179+0000 7ff5e83f1700 1 -- start start 2024-01-24T14:17:20.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.180+0000 7ff5e83f1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc003fb0 0x7ff5e00a6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.180+0000 7ff5e83f1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e00c0780 0x7ff5e00ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.180+0000 7ff5e83f1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e00a9780 0x7ff5e00a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.180+0000 7ff5e83f1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e00a7400 con 0x7ff5cc003fb0 2024-01-24T14:17:20.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.180+0000 7ff5e83f1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e00aa0f0 con 0x7ff5e00a9780 2024-01-24T14:17:20.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.180+0000 7ff5e83f1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e00becc0 con 0x7ff5e00c0780 2024-01-24T14:17:20.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.180+0000 7ff5e618d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc003fb0 0x7ff5e00a6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.180+0000 7ff5e618d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc003fb0 0x7ff5e00a6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47782/0 (socket says 172.21.15.29:47782) 2024-01-24T14:17:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.180+0000 7ff5e618d700 1 -- 172.21.15.29:0/3394570525 learned_addr learned my addr 172.21.15.29:0/3394570525 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.180+0000 7ff5e598c700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e00c0780 0x7ff5e00ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.180+0000 7ff5e698e700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e00a9780 0x7ff5e00a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.181+0000 7ff5e618d700 1 -- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e00c0780 msgr2=0x7ff5e00ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.181+0000 7ff5e618d700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e00c0780 0x7ff5e00ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.181+0000 7ff5e618d700 1 -- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e00a9780 msgr2=0x7ff5e00a9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.181+0000 7ff5e618d700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e00a9780 0x7ff5e00a9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.181+0000 7ff5e618d700 1 -- 172.21.15.29:0/3394570525 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e00befa0 con 0x7ff5cc003fb0 2024-01-24T14:17:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.181+0000 7ff5e618d700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc003fb0 0x7ff5e00a6f10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7ff5d4004810 tx=0x7ff5d4001e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.181+0000 7ff5d27fc700 1 -- 172.21.15.29:0/3394570525 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4001470 con 0x7ff5cc003fb0 2024-01-24T14:17:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.181+0000 7ff5d27fc700 1 -- 172.21.15.29:0/3394570525 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d4011590 con 0x7ff5cc003fb0 2024-01-24T14:17:20.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.181+0000 7ff5e83f1700 1 -- 172.21.15.29:0/3394570525 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e00bf280 con 0x7ff5cc003fb0 2024-01-24T14:17:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.181+0000 7ff5d27fc700 1 -- 172.21.15.29:0/3394570525 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4011910 con 0x7ff5cc003fb0 2024-01-24T14:17:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.181+0000 7ff5e83f1700 1 -- 172.21.15.29:0/3394570525 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e00bf840 con 0x7ff5cc003fb0 2024-01-24T14:17:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.182+0000 7ff5e83f1700 1 -- 172.21.15.29:0/3394570525 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e00475e0 con 0x7ff5cc003fb0 2024-01-24T14:17:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.183+0000 7ff5d27fc700 1 -- 172.21.15.29:0/3394570525 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff5d4011c80 con 0x7ff5cc003fb0 2024-01-24T14:17:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.183+0000 7ff5d27fc700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5c405f270 0x7ff5c4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.183+0000 7ff5d27fc700 1 -- 172.21.15.29:0/3394570525 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 7042+0+0 (secure 0 0 0) 0x7ff5d407a840 con 0x7ff5cc003fb0 2024-01-24T14:17:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.183+0000 7ff5e598c700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5c405f270 0x7ff5c4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.184+0000 7ff5e598c700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5c405f270 0x7ff5c4061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7ff5e00632c0 tx=0x7ff5dc01a000).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:20.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.188+0000 7ff5d27fc700 1 -- 172.21.15.29:0/3394570525 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff5d404b5f0 con 0x7ff5cc003fb0 2024-01-24T14:17:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.412+0000 7ff5e83f1700 1 -- 172.21.15.29:0/3394570525 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff5e005efa0 con 0x7ff5cc003fb0 2024-01-24T14:17:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.418+0000 7ff5d27fc700 1 -- 172.21.15.29:0/3394570525 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v63) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff5d404ec10 con 0x7ff5cc003fb0 2024-01-24T14:17:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.425+0000 7ff5e83f1700 1 -- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5c405f270 msgr2=0x7ff5c4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.425+0000 7ff5e83f1700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5c405f270 0x7ff5c4061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7ff5e00632c0 tx=0x7ff5dc01a000).stop 2024-01-24T14:17:20.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.425+0000 7ff5e83f1700 1 -- 172.21.15.29:0/3394570525 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc003fb0 msgr2=0x7ff5e00a6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.425+0000 7ff5e83f1700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc003fb0 0x7ff5e00a6f10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7ff5d4004810 tx=0x7ff5d4001e10).stop 2024-01-24T14:17:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.425+0000 7ff5e83f1700 1 -- 172.21.15.29:0/3394570525 shutdown_connections 2024-01-24T14:17:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.425+0000 7ff5e83f1700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e00a9780 0x7ff5e00a9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.425+0000 7ff5e83f1700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5c405f270 0x7ff5c4061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.425+0000 7ff5e83f1700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5cc003fb0 0x7ff5e00a6f10 secure :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 rx=0x7ff5d4004810 tx=0x7ff5d4001e10).stop 2024-01-24T14:17:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.425+0000 7ff5e83f1700 1 --2- 172.21.15.29:0/3394570525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e00c0780 0x7ff5e00ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.425+0000 7ff5e83f1700 1 -- 172.21.15.29:0/3394570525 >> 172.21.15.29:0/3394570525 conn(0x7ff5e0064390 msgr2=0x7ff5e0139900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:20.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.428+0000 7ff5e83f1700 1 -- 172.21.15.29:0/3394570525 shutdown_connections 2024-01-24T14:17:20.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.432+0000 7ff5e83f1700 1 -- 172.21.15.29:0/3394570525 wait complete. 2024-01-24T14:17:20.444 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 18 and data pool 19 2024-01-24T14:17:20.455 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:17:20.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.584+0000 7fddf551d700 1 Processor -- start 2024-01-24T14:17:20.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.585+0000 7fddf551d700 1 -- start start 2024-01-24T14:17:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.585+0000 7fddf551d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fddf00638e0 0x7fddf015f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.585+0000 7fddf551d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf00cb730 con 0x7fddf00c5410 2024-01-24T14:17:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.585+0000 7fddf551d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf00cba10 con 0x7fddf01544a0 2024-01-24T14:17:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.585+0000 7fddf551d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00cbcf0 con 0x7fddf00638e0 2024-01-24T14:17:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.586+0000 7fddee7fc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fddf00c5410 0x7fddf00631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44118/0 (socket says 172.21.15.29:44118) 2024-01-24T14:17:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.586+0000 7fddee7fc700 1 -- 172.21.15.29:0/4244391840 learned_addr learned my addr 172.21.15.29:0/4244391840 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.586+0000 7fddef7fe700 1 --2- 172.21.15.29:0/4244391840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fddf00638e0 0x7fddf015f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.586+0000 7fddedffb700 1 -- 172.21.15.29:0/4244391840 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811254674 0 0) 0x7fddf00cb730 con 0x7fddf00c5410 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.586+0000 7fddedffb700 1 -- 172.21.15.29:0/4244391840 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddd80033f0 con 0x7fddf00c5410 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.586+0000 7fddedffb700 1 -- 172.21.15.29:0/4244391840 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268301699 0 0) 0x7fddf00cba10 con 0x7fddf01544a0 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.586+0000 7fddedffb700 1 -- 172.21.15.29:0/4244391840 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddf00cb730 con 0x7fddf01544a0 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.586+0000 7fddedffb700 1 -- 172.21.15.29:0/4244391840 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3705347753 0 0) 0x7fddd80033f0 con 0x7fddf00c5410 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.586+0000 7fddef7fe700 1 -- 172.21.15.29:0/4244391840 >> v1:172.21.15.163:6789/0 conn(0x7fddf01544a0 legacy=0x7fddf00a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.586+0000 7fddef7fe700 1 -- 172.21.15.29:0/4244391840 >> v1:172.21.15.29:6789/0 conn(0x7fddf00c5410 legacy=0x7fddf00631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.586+0000 7fddef7fe700 1 -- 172.21.15.29:0/4244391840 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf00bfea0 con 0x7fddf00638e0 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.586+0000 7fddef7fe700 1 --2- 172.21.15.29:0/4244391840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fddf00638e0 0x7fddf015f4d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fdde8009e90 tx=0x7fdde8009780).ready entity=mon.2 client_cookie=df24eb71652a98b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.587+0000 7fddedffb700 1 -- 172.21.15.29:0/4244391840 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde800e070 con 0x7fddf00638e0 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.587+0000 7fddedffb700 1 -- 172.21.15.29:0/4244391840 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde800b7a0 con 0x7fddf00638e0 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.587+0000 7fddedffb700 1 -- 172.21.15.29:0/4244391840 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4001470 con 0x7fddf00638e0 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.587+0000 7fddf551d700 1 -- 172.21.15.29:0/4244391840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fddf00638e0 msgr2=0x7fddf015f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.587+0000 7fddf551d700 1 --2- 172.21.15.29:0/4244391840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fddf00638e0 0x7fddf015f4d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fdde8009e90 tx=0x7fdde8009780).stop 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.587+0000 7fddf551d700 1 -- 172.21.15.29:0/4244391840 shutdown_connections 2024-01-24T14:17:20.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.587+0000 7fddf551d700 1 --2- 172.21.15.29:0/4244391840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fddf00638e0 0x7fddf015f4d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.587+0000 7fddf551d700 1 -- 172.21.15.29:0/4244391840 >> 172.21.15.29:0/4244391840 conn(0x7fddf005ec80 msgr2=0x7fddf00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:20.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.587+0000 7fddf551d700 1 -- 172.21.15.29:0/4244391840 shutdown_connections 2024-01-24T14:17:20.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.587+0000 7fddf551d700 1 -- 172.21.15.29:0/4244391840 wait complete. 2024-01-24T14:17:20.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.589+0000 7fddf551d700 1 Processor -- start 2024-01-24T14:17:20.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.589+0000 7fddf551d700 1 -- start start 2024-01-24T14:17:20.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.589+0000 7fddf551d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddf00638e0 0x7fddf00c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.589+0000 7fddf551d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fddf00c5410 0x7fddf014c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.589+0000 7fddf551d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddf01544a0 0x7fddf014f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.589+0000 7fddf551d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00c2e80 con 0x7fddf01544a0 2024-01-24T14:17:20.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddf551d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00c3ca0 con 0x7fddf00638e0 2024-01-24T14:17:20.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddf551d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00c3090 con 0x7fddf00c5410 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddef7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddf01544a0 0x7fddf014f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddef7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddf01544a0 0x7fddf014f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47788/0 (socket says 172.21.15.29:47788) 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddef7fe700 1 -- 172.21.15.29:0/1637990217 learned_addr learned my addr 172.21.15.29:0/1637990217 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddee7fc700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fddf00c5410 0x7fddf014c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddeeffd700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddf00638e0 0x7fddf00c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddef7fe700 1 -- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fddf00c5410 msgr2=0x7fddf014c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddef7fe700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fddf00c5410 0x7fddf014c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddef7fe700 1 -- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddf00638e0 msgr2=0x7fddf00c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddef7fe700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddf00638e0 0x7fddf00c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddef7fe700 1 -- 172.21.15.29:0/1637990217 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde8009a20 con 0x7fddf01544a0 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddee7fc700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fddf00c5410 0x7fddf014c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddef7fe700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddf01544a0 0x7fddf014f2b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7fdde800b610 tx=0x7fdde8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddd6ffd700 1 -- 172.21.15.29:0/1637990217 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde800bd80 con 0x7fddf01544a0 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddd6ffd700 1 -- 172.21.15.29:0/1637990217 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde801e480 con 0x7fddf01544a0 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddf551d700 1 -- 172.21.15.29:0/1637990217 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf00c3370 con 0x7fddf01544a0 2024-01-24T14:17:20.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddd6ffd700 1 -- 172.21.15.29:0/1637990217 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde800bee0 con 0x7fddf01544a0 2024-01-24T14:17:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.590+0000 7fddf551d700 1 -- 172.21.15.29:0/1637990217 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf00c3930 con 0x7fddf01544a0 2024-01-24T14:17:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.591+0000 7fddf551d700 1 -- 172.21.15.29:0/1637990217 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf00c3e30 con 0x7fddf01544a0 2024-01-24T14:17:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.592+0000 7fddd6ffd700 1 -- 172.21.15.29:0/1637990217 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdde8003040 con 0x7fddf01544a0 2024-01-24T14:17:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.592+0000 7fddd6ffd700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdddc05f270 0x7fdddc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.593+0000 7fddd6ffd700 1 -- 172.21.15.29:0/1637990217 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fdde807c1b0 con 0x7fddf01544a0 2024-01-24T14:17:20.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.593+0000 7fddeeffd700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdddc05f270 0x7fdddc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.593+0000 7fddeeffd700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdddc05f270 0x7fdddc061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fdde0001170 tx=0x7fdde0007bb0).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:20.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.597+0000 7fddd6ffd700 1 -- 172.21.15.29:0/1637990217 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdde804ced0 con 0x7fddf01544a0 2024-01-24T14:17:20.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.780+0000 7fddf551d700 1 -- 172.21.15.29:0/1637990217 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddf00475e0 con 0x7fddf01544a0 2024-01-24T14:17:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.781+0000 7fddd6ffd700 1 -- 172.21.15.29:0/1637990217 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+18198 (secure 0 0 0) 0x7fdde8023090 con 0x7fddf01544a0 2024-01-24T14:17:20.782 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:20.782 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":152,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:17:20.413954+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:17:18.175741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-24T14:17:19.397106+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:17:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.783+0000 7fddf551d700 1 -- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdddc05f270 msgr2=0x7fdddc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.783+0000 7fddf551d700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdddc05f270 0x7fdddc061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fdde0001170 tx=0x7fdde0007bb0).stop 2024-01-24T14:17:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.783+0000 7fddf551d700 1 -- 172.21.15.29:0/1637990217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddf01544a0 msgr2=0x7fddf014f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.783+0000 7fddf551d700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddf01544a0 0x7fddf014f2b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7fdde800b610 tx=0x7fdde8009780).stop 2024-01-24T14:17:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.783+0000 7fddf551d700 1 -- 172.21.15.29:0/1637990217 shutdown_connections 2024-01-24T14:17:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.783+0000 7fddf551d700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdddc05f270 0x7fdddc061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.783+0000 7fddf551d700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fddf01544a0 0x7fddf014f2b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.783+0000 7fddf551d700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddf00638e0 0x7fddf00c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.783+0000 7fddf551d700 1 --2- 172.21.15.29:0/1637990217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fddf00c5410 0x7fddf014c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.783+0000 7fddf551d700 1 -- 172.21.15.29:0/1637990217 >> 172.21.15.29:0/1637990217 conn(0x7fddf005ec80 msgr2=0x7fddf00c9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.783+0000 7fddf551d700 1 -- 172.21.15.29:0/1637990217 shutdown_connections 2024-01-24T14:17:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.783+0000 7fddf551d700 1 -- 172.21.15.29:0/1637990217 wait complete. 2024-01-24T14:17:20.798 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:17:20.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.928+0000 7ff52744f700 1 Processor -- start 2024-01-24T14:17:20.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.928+0000 7ff52744f700 1 -- start start 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.929+0000 7ff52744f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff520066c00 0x7ff5200bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.929+0000 7ff52744f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5200670a0 0x7ff520158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.929+0000 7ff52744f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5201582b0 con 0x7ff5200c4a10 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.929+0000 7ff52744f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff520157ca0 con 0x7ff5200670a0 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.929+0000 7ff52744f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff520063c60 con 0x7ff520066c00 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff5259ec700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5200670a0 0x7ff520158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff5259ec700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff5200670a0 0x7ff520158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47792/0 (socket says 172.21.15.29:47792) 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff5259ec700 1 -- 172.21.15.29:0/1331128856 learned_addr learned my addr 172.21.15.29:0/1331128856 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff5249ea700 1 --2- 172.21.15.29:0/1331128856 >> v2:172.21.15.163:3300/0 conn(0x7ff520066c00 0x7ff5200bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff5259ec700 1 -- 172.21.15.29:0/1331128856 >> v2:172.21.15.163:3300/0 conn(0x7ff520066c00 msgr2=0x7ff5200bf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff5259ec700 1 --2- 172.21.15.29:0/1331128856 >> v2:172.21.15.163:3300/0 conn(0x7ff520066c00 0x7ff5200bf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff5259ec700 1 -- 172.21.15.29:0/1331128856 >> v1:172.21.15.163:6789/0 conn(0x7ff5200c4a10 legacy=0x7ff5200664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff5259ec700 1 -- 172.21.15.29:0/1331128856 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff520157f40 con 0x7ff5200670a0 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff5249ea700 1 --2- 172.21.15.29:0/1331128856 >> v2:172.21.15.163:3300/0 conn(0x7ff520066c00 0x7ff5200bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff5259ec700 1 --2- 172.21.15.29:0/1331128856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5200670a0 0x7ff520158740 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7ff520068cb0 tx=0x7ff518009840).ready entity=mon.0 client_cookie=79f9c414118aaf24 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff517fff700 1 -- 172.21.15.29:0/1331128856 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51800e070 con 0x7ff5200670a0 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff517fff700 1 -- 172.21.15.29:0/1331128856 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51800ba50 con 0x7ff5200670a0 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.930+0000 7ff517fff700 1 -- 172.21.15.29:0/1331128856 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51800bdd0 con 0x7ff5200670a0 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.931+0000 7ff52744f700 1 -- 172.21.15.29:0/1331128856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5200670a0 msgr2=0x7ff520158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.931+0000 7ff52744f700 1 --2- 172.21.15.29:0/1331128856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5200670a0 0x7ff520158740 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7ff520068cb0 tx=0x7ff518009840).stop 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.931+0000 7ff52744f700 1 -- 172.21.15.29:0/1331128856 shutdown_connections 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.931+0000 7ff52744f700 1 --2- 172.21.15.29:0/1331128856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5200670a0 0x7ff520158740 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.931+0000 7ff52744f700 1 --2- 172.21.15.29:0/1331128856 >> v2:172.21.15.163:3300/0 conn(0x7ff520066c00 0x7ff5200bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.931+0000 7ff52744f700 1 -- 172.21.15.29:0/1331128856 >> 172.21.15.29:0/1331128856 conn(0x7ff5200c8bb0 msgr2=0x7ff5200c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.931+0000 7ff52744f700 1 -- 172.21.15.29:0/1331128856 shutdown_connections 2024-01-24T14:17:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.931+0000 7ff52744f700 1 -- 172.21.15.29:0/1331128856 wait complete. 2024-01-24T14:17:20.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.932+0000 7ff52744f700 1 Processor -- start 2024-01-24T14:17:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.932+0000 7ff52744f700 1 -- start start 2024-01-24T14:17:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.933+0000 7ff52744f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff520066c00 0x7ff5200aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.933+0000 7ff52744f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5200670a0 0x7ff520156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.933+0000 7ff52744f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5200c4a10 0x7ff5200a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.933+0000 7ff52744f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200aaab0 con 0x7ff5200670a0 2024-01-24T14:17:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.933+0000 7ff52744f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200a9ea0 con 0x7ff5200c4a10 2024-01-24T14:17:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.933+0000 7ff52744f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200aa180 con 0x7ff520066c00 2024-01-24T14:17:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.933+0000 7ff5249ea700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5200670a0 0x7ff520156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.933+0000 7ff5249ea700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5200670a0 0x7ff520156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47794/0 (socket says 172.21.15.29:47794) 2024-01-24T14:17:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.933+0000 7ff5249ea700 1 -- 172.21.15.29:0/3360667540 learned_addr learned my addr 172.21.15.29:0/3360667540 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff5251eb700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff520066c00 0x7ff5200aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff5259ec700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5200c4a10 0x7ff5200a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff5249ea700 1 -- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff520066c00 msgr2=0x7ff5200aaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff5249ea700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff520066c00 0x7ff5200aaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff5249ea700 1 -- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5200c4a10 msgr2=0x7ff5200a76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff5249ea700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5200c4a10 0x7ff5200a76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff5249ea700 1 -- 172.21.15.29:0/3360667540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff518009ae0 con 0x7ff5200670a0 2024-01-24T14:17:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff5249ea700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5200670a0 0x7ff520156950 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7ff51c00d860 tx=0x7ff51c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff5157fa700 1 -- 172.21.15.29:0/3360667540 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0095f0 con 0x7ff5200670a0 2024-01-24T14:17:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff5157fa700 1 -- 172.21.15.29:0/3360667540 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c009750 con 0x7ff5200670a0 2024-01-24T14:17:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff52744f700 1 -- 172.21.15.29:0/3360667540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5200aa460 con 0x7ff5200670a0 2024-01-24T14:17:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff5157fa700 1 -- 172.21.15.29:0/3360667540 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c009ad0 con 0x7ff5200670a0 2024-01-24T14:17:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.934+0000 7ff52744f700 1 -- 172.21.15.29:0/3360667540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff520153a00 con 0x7ff5200670a0 2024-01-24T14:17:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.935+0000 7ff52744f700 1 -- 172.21.15.29:0/3360667540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5200475e0 con 0x7ff5200670a0 2024-01-24T14:17:20.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.936+0000 7ff5157fa700 1 -- 172.21.15.29:0/3360667540 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff51c012420 con 0x7ff5200670a0 2024-01-24T14:17:20.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.936+0000 7ff5157fa700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff50405f270 0x7ff504061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:20.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.936+0000 7ff5157fa700 1 -- 172.21.15.29:0/3360667540 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 7142+0+0 (secure 0 0 0) 0x7ff51c07a3d0 con 0x7ff5200670a0 2024-01-24T14:17:20.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.936+0000 7ff5251eb700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff50405f270 0x7ff504061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:20.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.937+0000 7ff5251eb700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff50405f270 0x7ff504061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff510003630 tx=0x7ff510009040).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:20.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:20.941+0000 7ff5157fa700 1 -- 172.21.15.29:0/3360667540 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff51c04b0c0 con 0x7ff5200670a0 2024-01-24T14:17:20.982 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-01-24T14:17:20.982+0000 7f61367c9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T14:17:21.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.124+0000 7ff52744f700 1 -- 172.21.15.29:0/3360667540 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff520068ce0 con 0x7ff5200670a0 2024-01-24T14:17:21.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.125+0000 7ff5157fa700 1 -- 172.21.15.29:0/3360667540 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+18198 (secure 0 0 0) 0x7ff51c04e6e0 con 0x7ff5200670a0 2024-01-24T14:17:21.126 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:21.126 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":152,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:17:20.413954+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:17:18.175741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-24T14:17:19.397106+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:17:21.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.127+0000 7ff52744f700 1 -- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff50405f270 msgr2=0x7ff504061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:21.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.127+0000 7ff52744f700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff50405f270 0x7ff504061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff510003630 tx=0x7ff510009040).stop 2024-01-24T14:17:21.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.127+0000 7ff52744f700 1 -- 172.21.15.29:0/3360667540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5200670a0 msgr2=0x7ff520156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:21.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.127+0000 7ff52744f700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5200670a0 0x7ff520156950 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7ff51c00d860 tx=0x7ff51c008040).stop 2024-01-24T14:17:21.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.127+0000 7ff52744f700 1 -- 172.21.15.29:0/3360667540 shutdown_connections 2024-01-24T14:17:21.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.127+0000 7ff52744f700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5200c4a10 0x7ff5200a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:21.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.127+0000 7ff52744f700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff520066c00 0x7ff5200aaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:21.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.127+0000 7ff52744f700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff50405f270 0x7ff504061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:21.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.127+0000 7ff52744f700 1 --2- 172.21.15.29:0/3360667540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5200670a0 0x7ff520156950 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:21.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.127+0000 7ff52744f700 1 -- 172.21.15.29:0/3360667540 >> 172.21.15.29:0/3360667540 conn(0x7ff5200c8bb0 msgr2=0x7ff5200c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:21.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.127+0000 7ff52744f700 1 -- 172.21.15.29:0/3360667540 shutdown_connections 2024-01-24T14:17:21.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.127+0000 7ff52744f700 1 -- 172.21.15.29:0/3360667540 wait complete. 2024-01-24T14:17:21.142 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-24T14:17:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.274+0000 7f3783ec6700 1 Processor -- start 2024-01-24T14:17:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.275+0000 7f3783ec6700 1 -- start start 2024-01-24T14:17:21.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3783ec6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c13a5e0 0x7f377c0639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:21.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3783ec6700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377c0679f0 con 0x7f377c063eb0 2024-01-24T14:17:21.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3783ec6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377c067cd0 con 0x7f377c066bd0 2024-01-24T14:17:21.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3783ec6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377c067fb0 con 0x7f377c13a5e0 2024-01-24T14:17:21.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3781461700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f377c063eb0 0x7f377c0664c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44120/0 (socket says 172.21.15.29:44120) 2024-01-24T14:17:21.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3781461700 1 -- 172.21.15.29:0/2672237693 learned_addr learned my addr 172.21.15.29:0/2672237693 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:21.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3781c62700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c13a5e0 0x7f377c0639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:21.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049877346 0 0) 0x7f377c0679f0 con 0x7f377c063eb0 2024-01-24T14:17:21.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37680033f0 con 0x7f377c063eb0 2024-01-24T14:17:21.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 6678024 0 0) 0x7f37680033f0 con 0x7f377c063eb0 2024-01-24T14:17:21.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c13a5e0 msgr2=0x7f377c0639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:21.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3780c60700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c13a5e0 0x7f377c0639c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:21.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 >> v1:172.21.15.163:6789/0 conn(0x7f377c066bd0 legacy=0x7f377c0bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:21.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c068290 con 0x7f377c063eb0 2024-01-24T14:17:21.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f376c001470 con 0x7f377c063eb0 2024-01-24T14:17:21.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.276+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 >> v1:172.21.15.29:6789/0 conn(0x7f377c063eb0 legacy=0x7f377c0664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:21.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3780c60700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3768003910 0x7f3768003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3780c60700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3768004220 0x7f37680046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3780c60700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c13a5e0 0x7f3768004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c067fb0 con 0x7f3768003910 2024-01-24T14:17:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376800b8e0 con 0x7f3768004220 2024-01-24T14:17:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376800ba20 con 0x7f377c13a5e0 2024-01-24T14:17:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3781c62700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3768003910 0x7f3768003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3781461700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3768004220 0x7f37680046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3782463700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c13a5e0 0x7f3768004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3781c62700 1 -- 172.21.15.29:0/2672237693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c13a5e0 msgr2=0x7f3768004ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3781c62700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c13a5e0 0x7f3768004ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3781c62700 1 -- 172.21.15.29:0/2672237693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3768004220 msgr2=0x7f37680046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3781c62700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3768004220 0x7f37680046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3781c62700 1 -- 172.21.15.29:0/2672237693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37680033f0 con 0x7f3768003910 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3782463700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c13a5e0 0x7f3768004ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3781c62700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3768003910 0x7f3768003cf0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7f3778009a60 tx=0x7f3778009060).ready entity=mon.0 client_cookie=c4cc017047b814ba server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3778009ea0 con 0x7f3768003910 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3778009430 con 0x7f3768003910 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.277+0000 7f3780c60700 1 -- 172.21.15.29:0/2672237693 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377800e870 con 0x7f3768003910 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.278+0000 7f3783ec6700 1 -- 172.21.15.29:0/2672237693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3768003910 msgr2=0x7f3768003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.278+0000 7f3783ec6700 1 --2- 172.21.15.29:0/2672237693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3768003910 0x7f3768003cf0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7f3778009a60 tx=0x7f3778009060).stop 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.278+0000 7f3782463700 1 -- 172.21.15.29:0/2672237693 reap_dead start 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.278+0000 7f3783ec6700 1 -- 172.21.15.29:0/2672237693 shutdown_connections 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.278+0000 7f3783ec6700 1 -- 172.21.15.29:0/2672237693 >> 172.21.15.29:0/2672237693 conn(0x7f377c0c8bd0 msgr2=0x7f377c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.278+0000 7f3783ec6700 1 -- 172.21.15.29:0/2672237693 shutdown_connections 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.278+0000 7f3783ec6700 1 -- 172.21.15.29:0/2672237693 wait complete. 2024-01-24T14:17:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.279+0000 7f3783ec6700 1 Processor -- start 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.279+0000 7f3783ec6700 1 -- start start 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.280+0000 7f3783ec6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3768003910 0x7f377c0aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.280+0000 7f3783ec6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f377c0ab2b0 0x7f377c0a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.280+0000 7f3783ec6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c0a9210 0x7f377c0a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.280+0000 7f3783ec6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c163f40 con 0x7f377c0ab2b0 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.280+0000 7f3783ec6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c154ec0 con 0x7f3768003910 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.280+0000 7f3783ec6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377c0a97b0 con 0x7f377c0a9210 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.280+0000 7f3781461700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f377c0ab2b0 0x7f377c0a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.280+0000 7f3781461700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f377c0ab2b0 0x7f377c0a8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47806/0 (socket says 172.21.15.29:47806) 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.280+0000 7f3781461700 1 -- 172.21.15.29:0/3042489998 learned_addr learned my addr 172.21.15.29:0/3042489998 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.280+0000 7f3781c62700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3768003910 0x7f377c0aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.280+0000 7f3782463700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c0a9210 0x7f377c0a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.281+0000 7f3781461700 1 -- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c0a9210 msgr2=0x7f377c0a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.281+0000 7f3781461700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c0a9210 0x7f377c0a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.281+0000 7f3781461700 1 -- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3768003910 msgr2=0x7f377c0aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:21.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.281+0000 7f3781461700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3768003910 0x7f377c0aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:21.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.281+0000 7f3781461700 1 -- 172.21.15.29:0/3042489998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c0a9a90 con 0x7f377c0ab2b0 2024-01-24T14:17:21.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.281+0000 7f3781461700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f377c0ab2b0 0x7f377c0a8d20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f376c001b70 tx=0x7f376c008b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:21.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.281+0000 7f3771ffb700 1 -- 172.21.15.29:0/3042489998 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c008e60 con 0x7f377c0ab2b0 2024-01-24T14:17:21.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.281+0000 7f3771ffb700 1 -- 172.21.15.29:0/3042489998 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c00f7d0 con 0x7f377c0ab2b0 2024-01-24T14:17:21.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.281+0000 7f3783ec6700 1 -- 172.21.15.29:0/3042489998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377c0a9d70 con 0x7f377c0ab2b0 2024-01-24T14:17:21.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.281+0000 7f3771ffb700 1 -- 172.21.15.29:0/3042489998 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c00fb70 con 0x7f377c0ab2b0 2024-01-24T14:17:21.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.281+0000 7f3783ec6700 1 -- 172.21.15.29:0/3042489998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c0a6670 con 0x7f377c0ab2b0 2024-01-24T14:17:21.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.282+0000 7f3783ec6700 1 -- 172.21.15.29:0/3042489998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377c0a9f90 con 0x7f377c0ab2b0 2024-01-24T14:17:21.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.282+0000 7f3771ffb700 1 -- 172.21.15.29:0/3042489998 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f376c016440 con 0x7f377c0ab2b0 2024-01-24T14:17:21.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.283+0000 7f3771ffb700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f376005f240 0x7f3760061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:21.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.283+0000 7f3771ffb700 1 -- 172.21.15.29:0/3042489998 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f376c07a460 con 0x7f377c0ab2b0 2024-01-24T14:17:21.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.283+0000 7f3781c62700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f376005f240 0x7f3760061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:21.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.284+0000 7f3781c62700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f376005f240 0x7f3760061700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f3778009820 tx=0x7f377801a000).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:21.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.287+0000 7f3771ffb700 1 -- 172.21.15.29:0/3042489998 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f376c04b150 con 0x7f377c0ab2b0 2024-01-24T14:17:21.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:21.517+0000 7f3783ec6700 1 -- 172.21.15.29:0/3042489998 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f377c0475e0 con 0x7f377c0ab2b0 2024-01-24T14:17:21.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-01-24T14:17:21.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-01-24T14:17:21.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-01-24T14:17:22.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.419+0000 7f3771ffb700 1 -- 172.21.15.29:0/3042489998 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) v1 ==== 118+0+0 (secure 0 0 0) 0x7f376c01b020 con 0x7f377c0ab2b0 2024-01-24T14:17:22.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.422+0000 7f3783ec6700 1 -- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f376005f240 msgr2=0x7f3760061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:22.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.422+0000 7f3783ec6700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f376005f240 0x7f3760061700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f3778009820 tx=0x7f377801a000).stop 2024-01-24T14:17:22.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.422+0000 7f3783ec6700 1 -- 172.21.15.29:0/3042489998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f377c0ab2b0 msgr2=0x7f377c0a8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:22.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.422+0000 7f3783ec6700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f377c0ab2b0 0x7f377c0a8d20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f376c001b70 tx=0x7f376c008b50).stop 2024-01-24T14:17:22.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.422+0000 7f3783ec6700 1 -- 172.21.15.29:0/3042489998 shutdown_connections 2024-01-24T14:17:22.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.422+0000 7f3783ec6700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f377c0a9210 0x7f377c0a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:22.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.422+0000 7f3783ec6700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3768003910 0x7f377c0aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:22.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.422+0000 7f3783ec6700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f376005f240 0x7f3760061700 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:22.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.422+0000 7f3783ec6700 1 --2- 172.21.15.29:0/3042489998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f377c0ab2b0 0x7f377c0a8d20 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:22.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.422+0000 7f3783ec6700 1 -- 172.21.15.29:0/3042489998 >> 172.21.15.29:0/3042489998 conn(0x7f377c0c8bd0 msgr2=0x7f377c0bf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:22.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.422+0000 7f3783ec6700 1 -- 172.21.15.29:0/3042489998 shutdown_connections 2024-01-24T14:17:22.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.422+0000 7f3783ec6700 1 -- 172.21.15.29:0/3042489998 wait complete. 2024-01-24T14:17:22.438 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-24T14:17:22.438 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-24T14:17:22.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.575+0000 7f7c8a8fb700 1 Processor -- start 2024-01-24T14:17:22.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.575+0000 7f7c8a8fb700 1 -- start start 2024-01-24T14:17:22.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.576+0000 7f7c8a8fb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c84153f40 0x7f7c840a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:22.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.576+0000 7f7c8a8fb700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8406a360 con 0x7f7c840c8290 2024-01-24T14:17:22.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.576+0000 7f7c8a8fb700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c840bcef0 con 0x7f7c84062ec0 2024-01-24T14:17:22.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.576+0000 7f7c8a8fb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c84069de0 con 0x7f7c84153f40 2024-01-24T14:17:22.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.577+0000 7f7c837fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7c840c8290 0x7f7c840627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:44122/0 (socket says 172.21.15.29:44122) 2024-01-24T14:17:22.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.577+0000 7f7c837fe700 1 -- 172.21.15.29:0/1242295341 learned_addr learned my addr 172.21.15.29:0/1242295341 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:22.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.577+0000 7f7c83fff700 1 --2- 172.21.15.29:0/1242295341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c84153f40 0x7f7c840a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:22.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.577+0000 7f7c82ffd700 1 -- 172.21.15.29:0/1242295341 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706504706 0 0) 0x7f7c8406a360 con 0x7f7c840c8290 2024-01-24T14:17:22.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.577+0000 7f7c82ffd700 1 -- 172.21.15.29:0/1242295341 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c6c0033f0 con 0x7f7c840c8290 2024-01-24T14:17:22.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.577+0000 7f7c82ffd700 1 -- 172.21.15.29:0/1242295341 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641597304 0 0) 0x7f7c840bcef0 con 0x7f7c84062ec0 2024-01-24T14:17:22.579 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.577+0000 7f7c82ffd700 1 -- 172.21.15.29:0/1242295341 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c8406a360 con 0x7f7c84062ec0 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.577+0000 7f7c82ffd700 1 -- 172.21.15.29:0/1242295341 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 13962264 0 0) 0x7f7c6c0033f0 con 0x7f7c840c8290 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.577+0000 7f7c83fff700 1 -- 172.21.15.29:0/1242295341 >> v1:172.21.15.163:6789/0 conn(0x7f7c84062ec0 legacy=0x7f7c84065440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.577+0000 7f7c83fff700 1 -- 172.21.15.29:0/1242295341 >> v1:172.21.15.29:6789/0 conn(0x7f7c840c8290 legacy=0x7f7c840627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.577+0000 7f7c83fff700 1 -- 172.21.15.29:0/1242295341 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8406a080 con 0x7f7c84153f40 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.577+0000 7f7c83fff700 1 --2- 172.21.15.29:0/1242295341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c84153f40 0x7f7c840a6db0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f7c74009790 tx=0x7f7c74009060).ready entity=mon.2 client_cookie=8625f510b3629d6a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.578+0000 7f7c82ffd700 1 -- 172.21.15.29:0/1242295341 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78001470 con 0x7f7c84153f40 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.578+0000 7f7c82ffd700 1 -- 172.21.15.29:0/1242295341 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c740090d0 con 0x7f7c84153f40 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.578+0000 7f7c82ffd700 1 -- 172.21.15.29:0/1242295341 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74011610 con 0x7f7c84153f40 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.578+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/1242295341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c84153f40 msgr2=0x7f7c840a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.578+0000 7f7c8a8fb700 1 --2- 172.21.15.29:0/1242295341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c84153f40 0x7f7c840a6db0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f7c74009790 tx=0x7f7c74009060).stop 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.578+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/1242295341 shutdown_connections 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.578+0000 7f7c8a8fb700 1 --2- 172.21.15.29:0/1242295341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c84153f40 0x7f7c840a6db0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.578+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/1242295341 >> 172.21.15.29:0/1242295341 conn(0x7f7c84064390 msgr2=0x7f7c840c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:22.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.578+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/1242295341 shutdown_connections 2024-01-24T14:17:22.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.578+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/1242295341 wait complete. 2024-01-24T14:17:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.579+0000 7f7c8a8fb700 1 Processor -- start 2024-01-24T14:17:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.580+0000 7f7c8a8fb700 1 -- start start 2024-01-24T14:17:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.580+0000 7f7c8a8fb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c84062ec0 0x7f7c840c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.580+0000 7f7c8a8fb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c840c8290 0x7f7c840beb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.580+0000 7f7c8a8fb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c84153f40 0x7f7c840bf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.580+0000 7f7c8a8fb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840bf570 con 0x7f7c84062ec0 2024-01-24T14:17:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.580+0000 7f7c8a8fb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840a8350 con 0x7f7c84153f40 2024-01-24T14:17:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.580+0000 7f7c8a8fb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c84152fa0 con 0x7f7c840c8290 2024-01-24T14:17:22.583 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.580+0000 7f7c83fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c84062ec0 0x7f7c840c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c83fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c84062ec0 0x7f7c840c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:47816/0 (socket says 172.21.15.29:47816) 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c83fff700 1 -- 172.21.15.29:0/3430059183 learned_addr learned my addr 172.21.15.29:0/3430059183 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c837fe700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c840c8290 0x7f7c840beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c88e98700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c84153f40 0x7f7c840bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c83fff700 1 -- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c840c8290 msgr2=0x7f7c840beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c83fff700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c840c8290 0x7f7c840beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c83fff700 1 -- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c84153f40 msgr2=0x7f7c840bf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c83fff700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c84153f40 0x7f7c840bf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c83fff700 1 -- 172.21.15.29:0/3430059183 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740092a0 con 0x7f7c84062ec0 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c88e98700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c84153f40 0x7f7c840bf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c83fff700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c84062ec0 0x7f7c840c35b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7f7c74009d80 tx=0x7f7c74016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c6bfff700 1 -- 172.21.15.29:0/3430059183 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74010740 con 0x7f7c84062ec0 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c6bfff700 1 -- 172.21.15.29:0/3430059183 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c740108a0 con 0x7f7c84062ec0 2024-01-24T14:17:22.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/3430059183 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c840a7a20 con 0x7f7c84062ec0 2024-01-24T14:17:22.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c6bfff700 1 -- 172.21.15.29:0/3430059183 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74010c20 con 0x7f7c84062ec0 2024-01-24T14:17:22.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.581+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/3430059183 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c840a7fe0 con 0x7f7c84062ec0 2024-01-24T14:17:22.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.582+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/3430059183 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c840475e0 con 0x7f7c84062ec0 2024-01-24T14:17:22.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.584+0000 7f7c6bfff700 1 -- 172.21.15.29:0/3430059183 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7c7401d640 con 0x7f7c84062ec0 2024-01-24T14:17:22.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.584+0000 7f7c6bfff700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c7005f270 0x7f7c70061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:22.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.584+0000 7f7c6bfff700 1 -- 172.21.15.29:0/3430059183 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f7c7407baa0 con 0x7f7c84062ec0 2024-01-24T14:17:22.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.584+0000 7f7c837fe700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c7005f270 0x7f7c70061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:22.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.585+0000 7f7c837fe700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c7005f270 0x7f7c70061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7c840a7010 tx=0x7f7c78001a50).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:22.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.588+0000 7f7c6bfff700 1 -- 172.21.15.29:0/3430059183 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7c7404c960 con 0x7f7c84062ec0 2024-01-24T14:17:22.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:22.818+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/3430059183 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7c8406a540 con 0x7f7c84062ec0 2024-01-24T14:17:23.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.424+0000 7f7c6bfff700 1 -- 172.21.15.29:0/3430059183 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7c7402c020 con 0x7f7c84062ec0 2024-01-24T14:17:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.426+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c7005f270 msgr2=0x7f7c70061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.426+0000 7f7c8a8fb700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c7005f270 0x7f7c70061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7c840a7010 tx=0x7f7c78001a50).stop 2024-01-24T14:17:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.427+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/3430059183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c84062ec0 msgr2=0x7f7c840c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.427+0000 7f7c8a8fb700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c84062ec0 0x7f7c840c35b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7f7c74009d80 tx=0x7f7c74016000).stop 2024-01-24T14:17:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.427+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/3430059183 shutdown_connections 2024-01-24T14:17:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.427+0000 7f7c8a8fb700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c84153f40 0x7f7c840bf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.427+0000 7f7c8a8fb700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7c7005f270 0x7f7c70061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.427+0000 7f7c8a8fb700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c84062ec0 0x7f7c840c35b0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.427+0000 7f7c8a8fb700 1 --2- 172.21.15.29:0/3430059183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c840c8290 0x7f7c840beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.427+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/3430059183 >> 172.21.15.29:0/3430059183 conn(0x7f7c84064390 msgr2=0x7f7c840c6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:23.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.427+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/3430059183 shutdown_connections 2024-01-24T14:17:23.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.427+0000 7f7c8a8fb700 1 -- 172.21.15.29:0/3430059183 wait complete. 2024-01-24T14:17:23.443 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-24T14:17:23.443 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:17:23.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.577+0000 7f5454f7f700 1 Processor -- start 2024-01-24T14:17:23.578 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.577+0000 7f5454f7f700 1 -- start start 2024-01-24T14:17:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.578+0000 7f5454f7f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5450155b00 0x7f54500664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.578+0000 7f5454f7f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54501565f0 0x7f54500bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.578+0000 7f5454f7f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5450066e90 con 0x7f54500669e0 2024-01-24T14:17:23.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.578+0000 7f5454f7f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54500c5500 con 0x7f5450155b00 2024-01-24T14:17:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.578+0000 7f5454f7f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54500c3910 con 0x7f54501565f0 2024-01-24T14:17:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.578+0000 7f544ed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54501565f0 0x7f54500bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.578+0000 7f544e59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5450155b00 0x7f54500664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.578+0000 7f544ed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54501565f0 0x7f54500bdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:41690/0 (socket says 172.21.15.29:41690) 2024-01-24T14:17:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.578+0000 7f544ed9d700 1 -- 172.21.15.29:0/1258524257 learned_addr learned my addr 172.21.15.29:0/1258524257 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.578+0000 7f544dd9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f54500669e0 0x7f5450155ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:35754/0 (socket says 172.21.15.29:35754) 2024-01-24T14:17:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.579+0000 7f544e59c700 1 -- 172.21.15.29:0/1258524257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54501565f0 msgr2=0x7f54500bdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.579+0000 7f544e59c700 1 --2- 172.21.15.29:0/1258524257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54501565f0 0x7f54500bdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.579+0000 7f544e59c700 1 -- 172.21.15.29:0/1258524257 >> v1:172.21.15.163:6789/0 conn(0x7f54500669e0 legacy=0x7f5450155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.579+0000 7f544d59a700 1 -- 172.21.15.29:0/1258524257 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784953402 0 0) 0x7f5444001110 con 0x7f54500669e0 2024-01-24T14:17:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.579+0000 7f544e59c700 1 -- 172.21.15.29:0/1258524257 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54500c5220 con 0x7f5450155b00 2024-01-24T14:17:23.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.579+0000 7f544e59c700 1 --2- 172.21.15.29:0/1258524257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5450155b00 0x7f54500664f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f5440009760 tx=0x7f5440009060).ready entity=mon.1 client_cookie=44fe8c4c3360bf56 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:23.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.579+0000 7f544d59a700 1 -- 172.21.15.29:0/1258524257 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544000f510 con 0x7f5450155b00 2024-01-24T14:17:23.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.579+0000 7f544d59a700 1 -- 172.21.15.29:0/1258524257 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544000f670 con 0x7f5450155b00 2024-01-24T14:17:23.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.579+0000 7f544d59a700 1 -- 172.21.15.29:0/1258524257 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544000f9f0 con 0x7f5450155b00 2024-01-24T14:17:23.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.579+0000 7f5454f7f700 1 -- 172.21.15.29:0/1258524257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5450155b00 msgr2=0x7f54500664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:23.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.579+0000 7f5454f7f700 1 --2- 172.21.15.29:0/1258524257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5450155b00 0x7f54500664f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f5440009760 tx=0x7f5440009060).stop 2024-01-24T14:17:23.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.580+0000 7f5454f7f700 1 -- 172.21.15.29:0/1258524257 shutdown_connections 2024-01-24T14:17:23.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.580+0000 7f5454f7f700 1 --2- 172.21.15.29:0/1258524257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54501565f0 0x7f54500bdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.580+0000 7f5454f7f700 1 --2- 172.21.15.29:0/1258524257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5450155b00 0x7f54500664f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.580+0000 7f5454f7f700 1 -- 172.21.15.29:0/1258524257 >> 172.21.15.29:0/1258524257 conn(0x7f54500c8bb0 msgr2=0x7f54500c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:23.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.580+0000 7f5454f7f700 1 -- 172.21.15.29:0/1258524257 shutdown_connections 2024-01-24T14:17:23.582 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.580+0000 7f5454f7f700 1 -- 172.21.15.29:0/1258524257 wait complete. 2024-01-24T14:17:23.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.581+0000 7f5454f7f700 1 Processor -- start 2024-01-24T14:17:23.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.581+0000 7f5454f7f700 1 -- start start 2024-01-24T14:17:23.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f5454f7f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54500669e0 0x7f545013a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:23.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f5454f7f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5450155b00 0x7f54500abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:23.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f5454f7f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54501565f0 0x7f54500ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:23.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f5454f7f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54500ac690 con 0x7f54500669e0 2024-01-24T14:17:23.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f5454f7f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54500651e0 con 0x7f54501565f0 2024-01-24T14:17:23.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f5454f7f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54500645d0 con 0x7f5450155b00 2024-01-24T14:17:23.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f544e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54500669e0 0x7f545013a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f544e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54500669e0 0x7f545013a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56360/0 (socket says 172.21.15.29:56360) 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f544e59c700 1 -- 172.21.15.29:0/358882903 learned_addr learned my addr 172.21.15.29:0/358882903 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f544ed9d700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54501565f0 0x7f54500ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f544dd9b700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5450155b00 0x7f54500abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f544e59c700 1 -- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5450155b00 msgr2=0x7f54500abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f544e59c700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5450155b00 0x7f54500abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f544e59c700 1 -- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54501565f0 msgr2=0x7f54500ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f544e59c700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54501565f0 0x7f54500ac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f544e59c700 1 -- 172.21.15.29:0/358882903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400092a0 con 0x7f54500669e0 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.582+0000 7f544e59c700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54500669e0 0x7f545013a6d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f54500c69b0 tx=0x7f5440017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.583+0000 7f543e7fc700 1 -- 172.21.15.29:0/358882903 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544000f7e0 con 0x7f54500669e0 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.583+0000 7f543e7fc700 1 -- 172.21.15.29:0/358882903 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544000f940 con 0x7f54500669e0 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.583+0000 7f5454f7f700 1 -- 172.21.15.29:0/358882903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54500648b0 con 0x7f54500669e0 2024-01-24T14:17:23.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.583+0000 7f543e7fc700 1 -- 172.21.15.29:0/358882903 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544001b660 con 0x7f54500669e0 2024-01-24T14:17:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.583+0000 7f5454f7f700 1 -- 172.21.15.29:0/358882903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5450064e70 con 0x7f54500669e0 2024-01-24T14:17:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.584+0000 7f5454f7f700 1 -- 172.21.15.29:0/358882903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54500c3290 con 0x7f54500669e0 2024-01-24T14:17:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.584+0000 7f543e7fc700 1 -- 172.21.15.29:0/358882903 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f544001ba50 con 0x7f54500669e0 2024-01-24T14:17:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.584+0000 7f543e7fc700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f543005f270 0x7f5430061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.585+0000 7f543e7fc700 1 -- 172.21.15.29:0/358882903 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f544007c230 con 0x7f54500669e0 2024-01-24T14:17:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.585+0000 7f544dd9b700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f543005f270 0x7f5430061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:23.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.585+0000 7f544dd9b700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f543005f270 0x7f5430061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f54500bf5f0 tx=0x7f5444007d70).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:23.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.589+0000 7f543e7fc700 1 -- 172.21.15.29:0/358882903 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f544004cec0 con 0x7f54500669e0 2024-01-24T14:17:23.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.813+0000 7f5454f7f700 1 -- 172.21.15.29:0/358882903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54500475e0 con 0x7f54500669e0 2024-01-24T14:17:23.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.814+0000 7f543e7fc700 1 -- 172.21.15.29:0/358882903 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f5440028090 con 0x7f54500669e0 2024-01-24T14:17:23.815 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:23.815 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2106677945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2106677945},{"type":"v1","addr":"172.21.15.163:6836","nonce":2106677945}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7786,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1964541525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1964541525},{"type":"v1","addr":"172.21.15.163:6837","nonce":1964541525}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4162831288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4162831288},{"type":"v1","addr":"172.21.15.29:6835","nonce":4162831288}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:17:20.414311+0000","modified":"2024-01-24T14:17:23.419003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8166":{"gid":8166,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/1312733772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1312733772},{"type":"v1","addr":"172.21.15.29:6834","nonce":1312733772}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-24T14:17:23.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.816+0000 7f5454f7f700 1 -- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f543005f270 msgr2=0x7f5430061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:23.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.816+0000 7f5454f7f700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f543005f270 0x7f5430061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f54500bf5f0 tx=0x7f5444007d70).stop 2024-01-24T14:17:23.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.816+0000 7f5454f7f700 1 -- 172.21.15.29:0/358882903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54500669e0 msgr2=0x7f545013a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:23.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.816+0000 7f5454f7f700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54500669e0 0x7f545013a6d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f54500c69b0 tx=0x7f5440017000).stop 2024-01-24T14:17:23.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.816+0000 7f5454f7f700 1 -- 172.21.15.29:0/358882903 shutdown_connections 2024-01-24T14:17:23.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.816+0000 7f5454f7f700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54501565f0 0x7f54500ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.816+0000 7f5454f7f700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f543005f270 0x7f5430061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.816+0000 7f5454f7f700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54500669e0 0x7f545013a6d0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.816+0000 7f5454f7f700 1 --2- 172.21.15.29:0/358882903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5450155b00 0x7f54500abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.816+0000 7f5454f7f700 1 -- 172.21.15.29:0/358882903 >> 172.21.15.29:0/358882903 conn(0x7f54500c8bb0 msgr2=0x7f54500a8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:23.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.816+0000 7f5454f7f700 1 -- 172.21.15.29:0/358882903 shutdown_connections 2024-01-24T14:17:23.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.816+0000 7f5454f7f700 1 -- 172.21.15.29:0/358882903 wait complete. 2024-01-24T14:17:23.818 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 66 2024-01-24T14:17:23.830 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T14:17:23.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.960+0000 7f726eeb3700 1 Processor -- start 2024-01-24T14:17:23.961 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.960+0000 7f726eeb3700 1 -- start start 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726eeb3700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7268134cc0 0x7f72680c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726eeb3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7268066c00 0x7f7268152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726eeb3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7268068360 con 0x7f72681350a0 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726eeb3700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7268153360 con 0x7f7268134cc0 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726eeb3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72680c3820 con 0x7f7268066c00 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726cc4f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7268134cc0 0x7f72680c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726cc4f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7268134cc0 0x7f72680c31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56376/0 (socket says 172.21.15.29:56376) 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726cc4f700 1 -- 172.21.15.29:0/3086969180 learned_addr learned my addr 172.21.15.29:0/3086969180 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726d450700 1 --2- 172.21.15.29:0/3086969180 >> v2:172.21.15.163:3300/0 conn(0x7f7268066c00 0x7f7268152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726cc4f700 1 -- 172.21.15.29:0/3086969180 >> v2:172.21.15.163:3300/0 conn(0x7f7268066c00 msgr2=0x7f7268152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726cc4f700 1 --2- 172.21.15.29:0/3086969180 >> v2:172.21.15.163:3300/0 conn(0x7f7268066c00 0x7f7268152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726cc4f700 1 -- 172.21.15.29:0/3086969180 >> v1:172.21.15.163:6789/0 conn(0x7f72681350a0 legacy=0x7f72680664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:23.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726cc4f700 1 -- 172.21.15.29:0/3086969180 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7268067a30 con 0x7f7268134cc0 2024-01-24T14:17:23.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f726cc4f700 1 --2- 172.21.15.29:0/3086969180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7268134cc0 0x7f72680c31b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f7268152430 tx=0x7f725c0090f0).ready entity=mon.0 client_cookie=d61b7745f1ff48e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:23.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.961+0000 7f72677fe700 1 -- 172.21.15.29:0/3086969180 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c00b5a0 con 0x7f7268134cc0 2024-01-24T14:17:23.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.962+0000 7f72677fe700 1 -- 172.21.15.29:0/3086969180 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c00b700 con 0x7f7268134cc0 2024-01-24T14:17:23.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.962+0000 7f72677fe700 1 -- 172.21.15.29:0/3086969180 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c00ba80 con 0x7f7268134cc0 2024-01-24T14:17:23.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.962+0000 7f726eeb3700 1 -- 172.21.15.29:0/3086969180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7268134cc0 msgr2=0x7f72680c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:23.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.962+0000 7f726eeb3700 1 --2- 172.21.15.29:0/3086969180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7268134cc0 0x7f72680c31b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f7268152430 tx=0x7f725c0090f0).stop 2024-01-24T14:17:23.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.962+0000 7f726eeb3700 1 -- 172.21.15.29:0/3086969180 shutdown_connections 2024-01-24T14:17:23.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.962+0000 7f726eeb3700 1 --2- 172.21.15.29:0/3086969180 >> v2:172.21.15.163:3300/0 conn(0x7f7268066c00 0x7f7268152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.962+0000 7f726eeb3700 1 --2- 172.21.15.29:0/3086969180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7268134cc0 0x7f72680c31b0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.962+0000 7f726eeb3700 1 -- 172.21.15.29:0/3086969180 >> 172.21.15.29:0/3086969180 conn(0x7f72680c8bb0 msgr2=0x7f72680c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:23.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.962+0000 7f726eeb3700 1 -- 172.21.15.29:0/3086969180 shutdown_connections 2024-01-24T14:17:23.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.962+0000 7f726eeb3700 1 -- 172.21.15.29:0/3086969180 wait complete. 2024-01-24T14:17:23.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.964+0000 7f726eeb3700 1 Processor -- start 2024-01-24T14:17:23.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.964+0000 7f726eeb3700 1 -- start start 2024-01-24T14:17:23.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.965+0000 7f726eeb3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7268066c00 0x7f7268166320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:23.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.965+0000 7f726eeb3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7268134cc0 0x7f726814ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:23.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.965+0000 7f726eeb3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72681350a0 0x7f7268134490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:23.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.965+0000 7f726eeb3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72680a9b30 con 0x7f7268134cc0 2024-01-24T14:17:23.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.965+0000 7f726eeb3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72680a97e0 con 0x7f7268066c00 2024-01-24T14:17:23.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.965+0000 7f726eeb3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72680c2b60 con 0x7f72681350a0 2024-01-24T14:17:23.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.965+0000 7f7267fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7268134cc0 0x7f726814ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.965+0000 7f726cc4f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7268066c00 0x7f7268166320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.965+0000 7f7267fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7268134cc0 0x7f726814ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56384/0 (socket says 172.21.15.29:56384) 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.965+0000 7f7267fff700 1 -- 172.21.15.29:0/1750617818 learned_addr learned my addr 172.21.15.29:0/1750617818 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.965+0000 7f726d450700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72681350a0 0x7f7268134490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.966+0000 7f7267fff700 1 -- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72681350a0 msgr2=0x7f7268134490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.966+0000 7f7267fff700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72681350a0 0x7f7268134490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.966+0000 7f7267fff700 1 -- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7268066c00 msgr2=0x7f7268166320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.966+0000 7f7267fff700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7268066c00 0x7f7268166320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.966+0000 7f7267fff700 1 -- 172.21.15.29:0/1750617818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c009370 con 0x7f7268134cc0 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.966+0000 7f726cc4f700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7268066c00 0x7f7268166320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.966+0000 7f7267fff700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7268134cc0 0x7f726814ad20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7f725400b460 tx=0x7f7254006f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.966+0000 7f7264ff9700 1 -- 172.21.15.29:0/1750617818 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254012410 con 0x7f7268134cc0 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.966+0000 7f7264ff9700 1 -- 172.21.15.29:0/1750617818 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7254012590 con 0x7f7268134cc0 2024-01-24T14:17:23.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.966+0000 7f726eeb3700 1 -- 172.21.15.29:0/1750617818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72680c2e40 con 0x7f7268134cc0 2024-01-24T14:17:23.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.966+0000 7f7264ff9700 1 -- 172.21.15.29:0/1750617818 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254012950 con 0x7f7268134cc0 2024-01-24T14:17:23.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.966+0000 7f726eeb3700 1 -- 172.21.15.29:0/1750617818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7268136af0 con 0x7f7268134cc0 2024-01-24T14:17:23.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.967+0000 7f7264ff9700 1 -- 172.21.15.29:0/1750617818 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7254012cc0 con 0x7f7268134cc0 2024-01-24T14:17:23.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.967+0000 7f726eeb3700 1 -- 172.21.15.29:0/1750617818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72680475e0 con 0x7f7268134cc0 2024-01-24T14:17:23.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.968+0000 7f7264ff9700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f724c05f240 0x7f724c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:23.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.968+0000 7f7264ff9700 1 -- 172.21.15.29:0/1750617818 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f725407a480 con 0x7f7268134cc0 2024-01-24T14:17:23.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.968+0000 7f726cc4f700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f724c05f240 0x7f724c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:23.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.969+0000 7f726cc4f700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f724c05f240 0x7f724c061700 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f725c009130 tx=0x7f725c015000).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:23.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:23.972+0000 7f7264ff9700 1 -- 172.21.15.29:0/1750617818 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f725404b1d0 con 0x7f7268134cc0 2024-01-24T14:17:24.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.157+0000 7f726eeb3700 1 -- 172.21.15.29:0/1750617818 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72680ab6f0 con 0x7f7268134cc0 2024-01-24T14:17:24.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.158+0000 7f7264ff9700 1 -- 172.21.15.29:0/1750617818 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f725404e7f0 con 0x7f7268134cc0 2024-01-24T14:17:24.159 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:24.159 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":153,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T14:17:21.415753+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:17:18.175741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-24T14:17:19.397106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T14:17:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.160+0000 7f726eeb3700 1 -- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f724c05f240 msgr2=0x7f724c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.160+0000 7f726eeb3700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f724c05f240 0x7f724c061700 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f725c009130 tx=0x7f725c015000).stop 2024-01-24T14:17:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.160+0000 7f726eeb3700 1 -- 172.21.15.29:0/1750617818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7268134cc0 msgr2=0x7f726814ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.160+0000 7f726eeb3700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7268134cc0 0x7f726814ad20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7f725400b460 tx=0x7f7254006f20).stop 2024-01-24T14:17:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.160+0000 7f726eeb3700 1 -- 172.21.15.29:0/1750617818 shutdown_connections 2024-01-24T14:17:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.160+0000 7f726eeb3700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72681350a0 0x7f7268134490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.160+0000 7f726eeb3700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7268066c00 0x7f7268166320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.160+0000 7f726eeb3700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f724c05f240 0x7f724c061700 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.160+0000 7f726eeb3700 1 --2- 172.21.15.29:0/1750617818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7268134cc0 0x7f726814ad20 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.160+0000 7f726eeb3700 1 -- 172.21.15.29:0/1750617818 >> 172.21.15.29:0/1750617818 conn(0x7f72680c8bb0 msgr2=0x7f7268066fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.160+0000 7f726eeb3700 1 -- 172.21.15.29:0/1750617818 shutdown_connections 2024-01-24T14:17:24.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.160+0000 7f726eeb3700 1 -- 172.21.15.29:0/1750617818 wait complete. 2024-01-24T14:17:24.174 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-24T14:17:24.175 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:17:24.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.304+0000 7f856217d700 1 Processor -- start 2024-01-24T14:17:24.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.304+0000 7f856217d700 1 -- start start 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.305+0000 7f856217d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f855c0638c0 0x7f855c154060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.305+0000 7f856217d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855c068400 con 0x7f855c153c80 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.305+0000 7f856217d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855c067ad0 con 0x7f855c066c00 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.305+0000 7f856217d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855c067db0 con 0x7f855c0638c0 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.305+0000 7f855bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f855c0638c0 0x7f855c154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.305+0000 7f855b7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f855c153c80 0x7f855c0664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41656/0 (socket says 172.21.15.29:41656) 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.305+0000 7f855b7fe700 1 -- 172.21.15.29:0/144780048 learned_addr learned my addr 172.21.15.29:0/144780048 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.306+0000 7f855bfff700 1 -- 172.21.15.29:0/144780048 >> v1:172.21.15.163:6789/0 conn(0x7f855c066c00 legacy=0x7f855c0bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.306+0000 7f855bfff700 1 -- 172.21.15.29:0/144780048 >> v1:172.21.15.29:6789/0 conn(0x7f855c153c80 legacy=0x7f855c0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.306+0000 7f855a7fc700 1 -- 172.21.15.29:0/144780048 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554712084 0 0) 0x7f855c067ad0 con 0x7f855c066c00 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.306+0000 7f855bfff700 1 -- 172.21.15.29:0/144780048 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c068090 con 0x7f855c0638c0 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.306+0000 7f855bfff700 1 --2- 172.21.15.29:0/144780048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855c0638c0 0x7f855c154060 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f8554009e90 tx=0x7f8554009780).ready entity=mon.0 client_cookie=258e00f5058a4fa4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.306+0000 7f855a7fc700 1 -- 172.21.15.29:0/144780048 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855400e070 con 0x7f855c0638c0 2024-01-24T14:17:24.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.306+0000 7f855a7fc700 1 -- 172.21.15.29:0/144780048 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855400bb30 con 0x7f855c0638c0 2024-01-24T14:17:24.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.306+0000 7f855a7fc700 1 -- 172.21.15.29:0/144780048 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855400beb0 con 0x7f855c0638c0 2024-01-24T14:17:24.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.307+0000 7f856217d700 1 -- 172.21.15.29:0/144780048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855c0638c0 msgr2=0x7f855c154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.307+0000 7f856217d700 1 --2- 172.21.15.29:0/144780048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855c0638c0 0x7f855c154060 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f8554009e90 tx=0x7f8554009780).stop 2024-01-24T14:17:24.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.307+0000 7f856217d700 1 -- 172.21.15.29:0/144780048 shutdown_connections 2024-01-24T14:17:24.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.307+0000 7f856217d700 1 --2- 172.21.15.29:0/144780048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855c0638c0 0x7f855c154060 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.307+0000 7f856217d700 1 -- 172.21.15.29:0/144780048 >> 172.21.15.29:0/144780048 conn(0x7f855c0c8bb0 msgr2=0x7f855c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:24.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.307+0000 7f856217d700 1 -- 172.21.15.29:0/144780048 shutdown_connections 2024-01-24T14:17:24.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.307+0000 7f856217d700 1 -- 172.21.15.29:0/144780048 wait complete. 2024-01-24T14:17:24.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.308+0000 7f856217d700 1 Processor -- start 2024-01-24T14:17:24.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.308+0000 7f856217d700 1 -- start start 2024-01-24T14:17:24.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.309+0000 7f856217d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c0638c0 0x7f855c13ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:24.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.309+0000 7f856217d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f855c066c00 0x7f855c0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:24.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.309+0000 7f856217d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855c153c80 0x7f855c0c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:24.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.309+0000 7f856217d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0c87a0 con 0x7f855c153c80 2024-01-24T14:17:24.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.309+0000 7f856217d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0c4870 con 0x7f855c066c00 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.309+0000 7f856217d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0c4b10 con 0x7f855c0638c0 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.309+0000 7f855bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855c153c80 0x7f855c0c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.309+0000 7f855bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855c153c80 0x7f855c0c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56402/0 (socket says 172.21.15.29:56402) 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.309+0000 7f855bfff700 1 -- 172.21.15.29:0/511053584 learned_addr learned my addr 172.21.15.29:0/511053584 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.309+0000 7f855affd700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f855c066c00 0x7f855c0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.309+0000 7f855b7fe700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c0638c0 0x7f855c13ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.310+0000 7f855bfff700 1 -- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c0638c0 msgr2=0x7f855c13ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.310+0000 7f855bfff700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c0638c0 0x7f855c13ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.310+0000 7f855bfff700 1 -- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f855c066c00 msgr2=0x7f855c0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.310+0000 7f855bfff700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f855c066c00 0x7f855c0c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.310+0000 7f855bfff700 1 -- 172.21.15.29:0/511053584 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8554009a20 con 0x7f855c153c80 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.310+0000 7f855bfff700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855c153c80 0x7f855c0c82b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f855c0c69b0 tx=0x7f855c064440).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.310+0000 7f85437fe700 1 -- 172.21.15.29:0/511053584 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554009460 con 0x7f855c153c80 2024-01-24T14:17:24.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.310+0000 7f85437fe700 1 -- 172.21.15.29:0/511053584 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855401e480 con 0x7f855c153c80 2024-01-24T14:17:24.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.310+0000 7f856217d700 1 -- 172.21.15.29:0/511053584 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855c0c4df0 con 0x7f855c153c80 2024-01-24T14:17:24.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.310+0000 7f85437fe700 1 -- 172.21.15.29:0/511053584 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855401e800 con 0x7f855c153c80 2024-01-24T14:17:24.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.310+0000 7f856217d700 1 -- 172.21.15.29:0/511053584 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c13f0b0 con 0x7f855c153c80 2024-01-24T14:17:24.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.311+0000 7f856217d700 1 -- 172.21.15.29:0/511053584 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855c0475e0 con 0x7f855c153c80 2024-01-24T14:17:24.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.311+0000 7f85437fe700 1 -- 172.21.15.29:0/511053584 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f855401eb70 con 0x7f855c153c80 2024-01-24T14:17:24.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.312+0000 7f85437fe700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f854805f270 0x7f8548061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:24.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.312+0000 7f85437fe700 1 -- 172.21.15.29:0/511053584 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f855407cbf0 con 0x7f855c153c80 2024-01-24T14:17:24.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.312+0000 7f855b7fe700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f854805f270 0x7f8548061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:24.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.312+0000 7f855b7fe700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f854805f270 0x7f8548061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f855c1571a0 tx=0x7f854c009040).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:24.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.316+0000 7f85437fe700 1 -- 172.21.15.29:0/511053584 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8554083080 con 0x7f855c153c80 2024-01-24T14:17:24.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.496+0000 7f856217d700 1 -- 172.21.15.29:0/511053584 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f855c0475e0 con 0x7f854805f270 2024-01-24T14:17:24.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.507+0000 7f85437fe700 1 -- 172.21.15.29:0/511053584 <== mgr.6851 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+362242 (secure 0 0 0) 0x7f855c0475e0 con 0x7f854805f270 2024-01-24T14:17:24.513 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:24.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.511+0000 7f856217d700 1 -- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f854805f270 msgr2=0x7f8548061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.511+0000 7f856217d700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f854805f270 0x7f8548061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f855c1571a0 tx=0x7f854c009040).stop 2024-01-24T14:17:24.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.511+0000 7f856217d700 1 -- 172.21.15.29:0/511053584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855c153c80 msgr2=0x7f855c0c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.511+0000 7f856217d700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855c153c80 0x7f855c0c82b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f855c0c69b0 tx=0x7f855c064440).stop 2024-01-24T14:17:24.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.511+0000 7f856217d700 1 -- 172.21.15.29:0/511053584 shutdown_connections 2024-01-24T14:17:24.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.511+0000 7f856217d700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f854805f270 0x7f8548061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.511+0000 7f856217d700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f855c153c80 0x7f855c0c82b0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.511+0000 7f856217d700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c0638c0 0x7f855c13ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.511+0000 7f856217d700 1 --2- 172.21.15.29:0/511053584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f855c066c00 0x7f855c0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.511+0000 7f856217d700 1 -- 172.21.15.29:0/511053584 >> 172.21.15.29:0/511053584 conn(0x7f855c0c8bb0 msgr2=0x7f855c0c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:24.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.511+0000 7f856217d700 1 -- 172.21.15.29:0/511053584 shutdown_connections 2024-01-24T14:17:24.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.511+0000 7f856217d700 1 -- 172.21.15.29:0/511053584 wait complete. 2024-01-24T14:17:24.517 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:17:24.526 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T14:17:23.231362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2436732,"kb_used_data":14908,"kb_used_omap":0,"kb_used_meta":2421760,"kb_avail":752537988,"statfs":{"total":773094113280,"available":770598899712,"internally_reserved":0,"allocated":15265792,"data_stored":8612443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2479882240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20936,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-71,"num_write_kb":-47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.016822"},"pg_stats":[{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447956+0000","last_change":"2024-01-24T14:17:20.444978+0000","last_active":"2024-01-24T14:17:20.447956+0000","last_peered":"2024-01-24T14:17:20.447956+0000","last_clean":"2024-01-24T14:17:20.447956+0000","last_became_active":"2024-01-24T14:17:20.444519+0000","last_became_peered":"2024-01-24T14:17:20.444519+0000","last_unstale":"2024-01-24T14:17:20.447956+0000","last_undegraded":"2024-01-24T14:17:20.447956+0000","last_fullsized":"2024-01-24T14:17:20.447956+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.3f","version":"152'3","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.446762+0000","last_change":"2024-01-24T14:17:20.024233+0000","last_active":"2024-01-24T14:17:20.446762+0000","last_peered":"2024-01-24T14:17:20.446762+0000","last_clean":"2024-01-24T14:17:20.446762+0000","last_became_active":"2024-01-24T14:17:20.023950+0000","last_became_peered":"2024-01-24T14:17:20.023950+0000","last_unstale":"2024-01-24T14:17:20.446762+0000","last_undegraded":"2024-01-24T14:17:20.446762+0000","last_fullsized":"2024-01-24T14:17:20.446762+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448650+0000","last_change":"2024-01-24T14:17:20.442921+0000","last_active":"2024-01-24T14:17:20.448650+0000","last_peered":"2024-01-24T14:17:20.448650+0000","last_clean":"2024-01-24T14:17:20.448650+0000","last_became_active":"2024-01-24T14:17:20.439984+0000","last_became_peered":"2024-01-24T14:17:20.439984+0000","last_unstale":"2024-01-24T14:17:20.448650+0000","last_undegraded":"2024-01-24T14:17:20.448650+0000","last_fullsized":"2024-01-24T14:17:20.448650+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423287+0000","last_change":"2024-01-24T14:17:20.022892+0000","last_active":"2024-01-24T14:17:20.423287+0000","last_peered":"2024-01-24T14:17:20.423287+0000","last_clean":"2024-01-24T14:17:20.423287+0000","last_became_active":"2024-01-24T14:17:20.022218+0000","last_became_peered":"2024-01-24T14:17:20.022218+0000","last_unstale":"2024-01-24T14:17:20.423287+0000","last_undegraded":"2024-01-24T14:17:20.423287+0000","last_fullsized":"2024-01-24T14:17:20.423287+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.452232+0000","last_change":"2024-01-24T14:17:20.450262+0000","last_active":"2024-01-24T14:17:20.452232+0000","last_peered":"2024-01-24T14:17:20.452232+0000","last_clean":"2024-01-24T14:17:20.452232+0000","last_became_active":"2024-01-24T14:17:20.449899+0000","last_became_peered":"2024-01-24T14:17:20.449899+0000","last_unstale":"2024-01-24T14:17:20.452232+0000","last_undegraded":"2024-01-24T14:17:20.452232+0000","last_fullsized":"2024-01-24T14:17:20.452232+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423290+0000","last_change":"2024-01-24T14:17:20.021408+0000","last_active":"2024-01-24T14:17:20.423290+0000","last_peered":"2024-01-24T14:17:20.423290+0000","last_clean":"2024-01-24T14:17:20.423290+0000","last_became_active":"2024-01-24T14:17:20.021089+0000","last_became_peered":"2024-01-24T14:17:20.021089+0000","last_unstale":"2024-01-24T14:17:20.423290+0000","last_undegraded":"2024-01-24T14:17:20.423290+0000","last_fullsized":"2024-01-24T14:17:20.423290+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.445387+0000","last_change":"2024-01-24T14:17:20.442458+0000","last_active":"2024-01-24T14:17:20.445387+0000","last_peered":"2024-01-24T14:17:20.445387+0000","last_clean":"2024-01-24T14:17:20.445387+0000","last_became_active":"2024-01-24T14:17:20.442169+0000","last_became_peered":"2024-01-24T14:17:20.442169+0000","last_unstale":"2024-01-24T14:17:20.445387+0000","last_undegraded":"2024-01-24T14:17:20.445387+0000","last_fullsized":"2024-01-24T14:17:20.445387+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.3c","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449478+0000","last_change":"2024-01-24T14:17:20.026370+0000","last_active":"2024-01-24T14:17:20.449478+0000","last_peered":"2024-01-24T14:17:20.449478+0000","last_clean":"2024-01-24T14:17:20.449478+0000","last_became_active":"2024-01-24T14:17:20.026096+0000","last_became_peered":"2024-01-24T14:17:20.026096+0000","last_unstale":"2024-01-24T14:17:20.449478+0000","last_undegraded":"2024-01-24T14:17:20.449478+0000","last_fullsized":"2024-01-24T14:17:20.449478+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.439849+0000","last_change":"2024-01-24T14:17:20.430898+0000","last_active":"2024-01-24T14:17:20.439849+0000","last_peered":"2024-01-24T14:17:20.439849+0000","last_clean":"2024-01-24T14:17:20.439849+0000","last_became_active":"2024-01-24T14:17:20.430614+0000","last_became_peered":"2024-01-24T14:17:20.430614+0000","last_unstale":"2024-01-24T14:17:20.439849+0000","last_undegraded":"2024-01-24T14:17:20.439849+0000","last_fullsized":"2024-01-24T14:17:20.439849+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3b","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.436687+0000","last_change":"2024-01-24T14:17:20.021474+0000","last_active":"2024-01-24T14:17:20.436687+0000","last_peered":"2024-01-24T14:17:20.436687+0000","last_clean":"2024-01-24T14:17:20.436687+0000","last_became_active":"2024-01-24T14:17:20.021202+0000","last_became_peered":"2024-01-24T14:17:20.021202+0000","last_unstale":"2024-01-24T14:17:20.436687+0000","last_undegraded":"2024-01-24T14:17:20.436687+0000","last_fullsized":"2024-01-24T14:17:20.436687+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451840+0000","last_change":"2024-01-24T14:17:20.445712+0000","last_active":"2024-01-24T14:17:20.451840+0000","last_peered":"2024-01-24T14:17:20.451840+0000","last_clean":"2024-01-24T14:17:20.451840+0000","last_became_active":"2024-01-24T14:17:20.444488+0000","last_became_peered":"2024-01-24T14:17:20.444488+0000","last_unstale":"2024-01-24T14:17:20.451840+0000","last_undegraded":"2024-01-24T14:17:20.451840+0000","last_fullsized":"2024-01-24T14:17:20.451840+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3a","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.440681+0000","last_change":"2024-01-24T14:17:20.035007+0000","last_active":"2024-01-24T14:17:20.440681+0000","last_peered":"2024-01-24T14:17:20.440681+0000","last_clean":"2024-01-24T14:17:20.440681+0000","last_became_active":"2024-01-24T14:17:20.034701+0000","last_became_peered":"2024-01-24T14:17:20.034701+0000","last_unstale":"2024-01-24T14:17:20.440681+0000","last_undegraded":"2024-01-24T14:17:20.440681+0000","last_fullsized":"2024-01-24T14:17:20.440681+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449011+0000","last_change":"2024-01-24T14:17:20.444292+0000","last_active":"2024-01-24T14:17:20.449011+0000","last_peered":"2024-01-24T14:17:20.449011+0000","last_clean":"2024-01-24T14:17:20.449011+0000","last_became_active":"2024-01-24T14:17:20.444013+0000","last_became_peered":"2024-01-24T14:17:20.444013+0000","last_unstale":"2024-01-24T14:17:20.449011+0000","last_undegraded":"2024-01-24T14:17:20.449011+0000","last_fullsized":"2024-01-24T14:17:20.449011+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.430390+0000","last_change":"2024-01-24T14:17:20.034106+0000","last_active":"2024-01-24T14:17:20.430390+0000","last_peered":"2024-01-24T14:17:20.430390+0000","last_clean":"2024-01-24T14:17:20.430390+0000","last_became_active":"2024-01-24T14:17:20.023239+0000","last_became_peered":"2024-01-24T14:17:20.023239+0000","last_unstale":"2024-01-24T14:17:20.430390+0000","last_undegraded":"2024-01-24T14:17:20.430390+0000","last_fullsized":"2024-01-24T14:17:20.430390+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.442509+0000","last_change":"2024-01-24T14:17:20.438409+0000","last_active":"2024-01-24T14:17:20.442509+0000","last_peered":"2024-01-24T14:17:20.442509+0000","last_clean":"2024-01-24T14:17:20.442509+0000","last_became_active":"2024-01-24T14:17:20.438111+0000","last_became_peered":"2024-01-24T14:17:20.438111+0000","last_unstale":"2024-01-24T14:17:20.442509+0000","last_undegraded":"2024-01-24T14:17:20.442509+0000","last_fullsized":"2024-01-24T14:17:20.442509+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.424880+0000","last_change":"2024-01-24T14:17:20.022961+0000","last_active":"2024-01-24T14:17:20.424880+0000","last_peered":"2024-01-24T14:17:20.424880+0000","last_clean":"2024-01-24T14:17:20.424880+0000","last_became_active":"2024-01-24T14:17:20.022632+0000","last_became_peered":"2024-01-24T14:17:20.022632+0000","last_unstale":"2024-01-24T14:17:20.424880+0000","last_undegraded":"2024-01-24T14:17:20.424880+0000","last_fullsized":"2024-01-24T14:17:20.424880+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447932+0000","last_change":"2024-01-24T14:17:20.444806+0000","last_active":"2024-01-24T14:17:20.447932+0000","last_peered":"2024-01-24T14:17:20.447932+0000","last_clean":"2024-01-24T14:17:20.447932+0000","last_became_active":"2024-01-24T14:17:20.444167+0000","last_became_peered":"2024-01-24T14:17:20.444167+0000","last_unstale":"2024-01-24T14:17:20.447932+0000","last_undegraded":"2024-01-24T14:17:20.447932+0000","last_fullsized":"2024-01-24T14:17:20.447932+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.429341+0000","last_change":"2024-01-24T14:17:20.030297+0000","last_active":"2024-01-24T14:17:20.429341+0000","last_peered":"2024-01-24T14:17:20.429341+0000","last_clean":"2024-01-24T14:17:20.429341+0000","last_became_active":"2024-01-24T14:17:20.030015+0000","last_became_peered":"2024-01-24T14:17:20.030015+0000","last_unstale":"2024-01-24T14:17:20.429341+0000","last_undegraded":"2024-01-24T14:17:20.429341+0000","last_fullsized":"2024-01-24T14:17:20.429341+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449450+0000","last_change":"2024-01-24T14:17:20.442943+0000","last_active":"2024-01-24T14:17:20.449450+0000","last_peered":"2024-01-24T14:17:20.449450+0000","last_clean":"2024-01-24T14:17:20.449450+0000","last_became_active":"2024-01-24T14:17:20.442070+0000","last_became_peered":"2024-01-24T14:17:20.442070+0000","last_unstale":"2024-01-24T14:17:20.449450+0000","last_undegraded":"2024-01-24T14:17:20.449450+0000","last_fullsized":"2024-01-24T14:17:20.449450+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.435235+0000","last_change":"2024-01-24T14:17:20.034877+0000","last_active":"2024-01-24T14:17:20.435235+0000","last_peered":"2024-01-24T14:17:20.435235+0000","last_clean":"2024-01-24T14:17:20.435235+0000","last_became_active":"2024-01-24T14:17:20.034540+0000","last_became_peered":"2024-01-24T14:17:20.034540+0000","last_unstale":"2024-01-24T14:17:20.435235+0000","last_undegraded":"2024-01-24T14:17:20.435235+0000","last_fullsized":"2024-01-24T14:17:20.435235+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.445476+0000","last_change":"2024-01-24T14:17:20.442359+0000","last_active":"2024-01-24T14:17:20.445476+0000","last_peered":"2024-01-24T14:17:20.445476+0000","last_clean":"2024-01-24T14:17:20.445476+0000","last_became_active":"2024-01-24T14:17:20.442098+0000","last_became_peered":"2024-01-24T14:17:20.442098+0000","last_unstale":"2024-01-24T14:17:20.445476+0000","last_undegraded":"2024-01-24T14:17:20.445476+0000","last_fullsized":"2024-01-24T14:17:20.445476+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423831+0000","last_change":"2024-01-24T14:17:20.020488+0000","last_active":"2024-01-24T14:17:20.423831+0000","last_peered":"2024-01-24T14:17:20.423831+0000","last_clean":"2024-01-24T14:17:20.423831+0000","last_became_active":"2024-01-24T14:17:20.020210+0000","last_became_peered":"2024-01-24T14:17:20.020210+0000","last_unstale":"2024-01-24T14:17:20.423831+0000","last_undegraded":"2024-01-24T14:17:20.423831+0000","last_fullsized":"2024-01-24T14:17:20.423831+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448508+0000","last_change":"2024-01-24T14:17:20.444334+0000","last_active":"2024-01-24T14:17:20.448508+0000","last_peered":"2024-01-24T14:17:20.448508+0000","last_clean":"2024-01-24T14:17:20.448508+0000","last_became_active":"2024-01-24T14:17:20.444044+0000","last_became_peered":"2024-01-24T14:17:20.444044+0000","last_unstale":"2024-01-24T14:17:20.448508+0000","last_undegraded":"2024-01-24T14:17:20.448508+0000","last_fullsized":"2024-01-24T14:17:20.448508+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.34","version":"152'4","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.454447+0000","last_change":"2024-01-24T14:17:20.035154+0000","last_active":"2024-01-24T14:17:20.454447+0000","last_peered":"2024-01-24T14:17:20.454447+0000","last_clean":"2024-01-24T14:17:20.454447+0000","last_became_active":"2024-01-24T14:17:20.034810+0000","last_became_peered":"2024-01-24T14:17:20.034810+0000","last_unstale":"2024-01-24T14:17:20.454447+0000","last_undegraded":"2024-01-24T14:17:20.454447+0000","last_fullsized":"2024-01-24T14:17:20.454447+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447900+0000","last_change":"2024-01-24T14:17:20.442809+0000","last_active":"2024-01-24T14:17:20.447900+0000","last_peered":"2024-01-24T14:17:20.447900+0000","last_clean":"2024-01-24T14:17:20.447900+0000","last_became_active":"2024-01-24T14:17:20.442478+0000","last_became_peered":"2024-01-24T14:17:20.442478+0000","last_unstale":"2024-01-24T14:17:20.447900+0000","last_undegraded":"2024-01-24T14:17:20.447900+0000","last_fullsized":"2024-01-24T14:17:20.447900+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.33","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.444609+0000","last_change":"2024-01-24T14:17:20.030771+0000","last_active":"2024-01-24T14:17:20.444609+0000","last_peered":"2024-01-24T14:17:20.444609+0000","last_clean":"2024-01-24T14:17:20.444609+0000","last_became_active":"2024-01-24T14:17:20.030457+0000","last_became_peered":"2024-01-24T14:17:20.030457+0000","last_unstale":"2024-01-24T14:17:20.444609+0000","last_undegraded":"2024-01-24T14:17:20.444609+0000","last_fullsized":"2024-01-24T14:17:20.444609+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449149+0000","last_change":"2024-01-24T14:17:20.447685+0000","last_active":"2024-01-24T14:17:20.449149+0000","last_peered":"2024-01-24T14:17:20.449149+0000","last_clean":"2024-01-24T14:17:20.449149+0000","last_became_active":"2024-01-24T14:17:20.442539+0000","last_became_peered":"2024-01-24T14:17:20.442539+0000","last_unstale":"2024-01-24T14:17:20.449149+0000","last_undegraded":"2024-01-24T14:17:20.449149+0000","last_fullsized":"2024-01-24T14:17:20.449149+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423022+0000","last_change":"2024-01-24T14:17:20.025463+0000","last_active":"2024-01-24T14:17:20.423022+0000","last_peered":"2024-01-24T14:17:20.423022+0000","last_clean":"2024-01-24T14:17:20.423022+0000","last_became_active":"2024-01-24T14:17:20.024916+0000","last_became_peered":"2024-01-24T14:17:20.024916+0000","last_unstale":"2024-01-24T14:17:20.423022+0000","last_undegraded":"2024-01-24T14:17:20.423022+0000","last_fullsized":"2024-01-24T14:17:20.423022+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.439262+0000","last_change":"2024-01-24T14:17:20.429709+0000","last_active":"2024-01-24T14:17:20.439262+0000","last_peered":"2024-01-24T14:17:20.439262+0000","last_clean":"2024-01-24T14:17:20.439262+0000","last_became_active":"2024-01-24T14:17:20.429425+0000","last_became_peered":"2024-01-24T14:17:20.429425+0000","last_unstale":"2024-01-24T14:17:20.439262+0000","last_undegraded":"2024-01-24T14:17:20.439262+0000","last_fullsized":"2024-01-24T14:17:20.439262+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.422164+0000","last_change":"2024-01-24T14:17:20.020973+0000","last_active":"2024-01-24T14:17:20.422164+0000","last_peered":"2024-01-24T14:17:20.422164+0000","last_clean":"2024-01-24T14:17:20.422164+0000","last_became_active":"2024-01-24T14:17:20.020647+0000","last_became_peered":"2024-01-24T14:17:20.020647+0000","last_unstale":"2024-01-24T14:17:20.422164+0000","last_undegraded":"2024-01-24T14:17:20.422164+0000","last_fullsized":"2024-01-24T14:17:20.422164+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448628+0000","last_change":"2024-01-24T14:17:20.439117+0000","last_active":"2024-01-24T14:17:20.448628+0000","last_peered":"2024-01-24T14:17:20.448628+0000","last_clean":"2024-01-24T14:17:20.448628+0000","last_became_active":"2024-01-24T14:17:20.438789+0000","last_became_peered":"2024-01-24T14:17:20.438789+0000","last_unstale":"2024-01-24T14:17:20.448628+0000","last_undegraded":"2024-01-24T14:17:20.448628+0000","last_fullsized":"2024-01-24T14:17:20.448628+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.30","version":"152'4","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449555+0000","last_change":"2024-01-24T14:17:20.036630+0000","last_active":"2024-01-24T14:17:20.449555+0000","last_peered":"2024-01-24T14:17:20.449555+0000","last_clean":"2024-01-24T14:17:20.449555+0000","last_became_active":"2024-01-24T14:17:20.036294+0000","last_became_peered":"2024-01-24T14:17:20.036294+0000","last_unstale":"2024-01-24T14:17:20.449555+0000","last_undegraded":"2024-01-24T14:17:20.449555+0000","last_fullsized":"2024-01-24T14:17:20.449555+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.442311+0000","last_change":"2024-01-24T14:17:20.435992+0000","last_active":"2024-01-24T14:17:20.442311+0000","last_peered":"2024-01-24T14:17:20.442311+0000","last_clean":"2024-01-24T14:17:20.442311+0000","last_became_active":"2024-01-24T14:17:20.435732+0000","last_became_peered":"2024-01-24T14:17:20.435732+0000","last_unstale":"2024-01-24T14:17:20.442311+0000","last_undegraded":"2024-01-24T14:17:20.442311+0000","last_fullsized":"2024-01-24T14:17:20.442311+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.435330+0000","last_change":"2024-01-24T14:17:20.031041+0000","last_active":"2024-01-24T14:17:20.435330+0000","last_peered":"2024-01-24T14:17:20.435330+0000","last_clean":"2024-01-24T14:17:20.435330+0000","last_became_active":"2024-01-24T14:17:20.030681+0000","last_became_peered":"2024-01-24T14:17:20.030681+0000","last_unstale":"2024-01-24T14:17:20.435330+0000","last_undegraded":"2024-01-24T14:17:20.435330+0000","last_fullsized":"2024-01-24T14:17:20.435330+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.443121+0000","last_change":"2024-01-24T14:17:20.438829+0000","last_active":"2024-01-24T14:17:20.443121+0000","last_peered":"2024-01-24T14:17:20.443121+0000","last_clean":"2024-01-24T14:17:20.443121+0000","last_became_active":"2024-01-24T14:17:20.438547+0000","last_became_peered":"2024-01-24T14:17:20.438547+0000","last_unstale":"2024-01-24T14:17:20.443121+0000","last_undegraded":"2024-01-24T14:17:20.443121+0000","last_fullsized":"2024-01-24T14:17:20.443121+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423424+0000","last_change":"2024-01-24T14:17:20.022692+0000","last_active":"2024-01-24T14:17:20.423424+0000","last_peered":"2024-01-24T14:17:20.423424+0000","last_clean":"2024-01-24T14:17:20.423424+0000","last_became_active":"2024-01-24T14:17:20.021854+0000","last_became_peered":"2024-01-24T14:17:20.021854+0000","last_unstale":"2024-01-24T14:17:20.423424+0000","last_undegraded":"2024-01-24T14:17:20.423424+0000","last_fullsized":"2024-01-24T14:17:20.423424+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.439645+0000","last_change":"2024-01-24T14:17:20.433756+0000","last_active":"2024-01-24T14:17:20.439645+0000","last_peered":"2024-01-24T14:17:20.439645+0000","last_clean":"2024-01-24T14:17:20.439645+0000","last_became_active":"2024-01-24T14:17:20.433447+0000","last_became_peered":"2024-01-24T14:17:20.433447+0000","last_unstale":"2024-01-24T14:17:20.439645+0000","last_undegraded":"2024-01-24T14:17:20.439645+0000","last_fullsized":"2024-01-24T14:17:20.439645+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449394+0000","last_change":"2024-01-24T14:17:20.020567+0000","last_active":"2024-01-24T14:17:20.449394+0000","last_peered":"2024-01-24T14:17:20.449394+0000","last_clean":"2024-01-24T14:17:20.449394+0000","last_became_active":"2024-01-24T14:17:20.020147+0000","last_became_peered":"2024-01-24T14:17:20.020147+0000","last_unstale":"2024-01-24T14:17:20.449394+0000","last_undegraded":"2024-01-24T14:17:20.449394+0000","last_fullsized":"2024-01-24T14:17:20.449394+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448483+0000","last_change":"2024-01-24T14:17:20.441062+0000","last_active":"2024-01-24T14:17:20.448483+0000","last_peered":"2024-01-24T14:17:20.448483+0000","last_clean":"2024-01-24T14:17:20.448483+0000","last_became_active":"2024-01-24T14:17:20.440697+0000","last_became_peered":"2024-01-24T14:17:20.440697+0000","last_unstale":"2024-01-24T14:17:20.448483+0000","last_undegraded":"2024-01-24T14:17:20.448483+0000","last_fullsized":"2024-01-24T14:17:20.448483+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.418915+0000","last_change":"2024-01-24T14:17:20.036805+0000","last_active":"2024-01-24T14:17:20.418915+0000","last_peered":"2024-01-24T14:17:20.418915+0000","last_clean":"2024-01-24T14:17:20.418915+0000","last_became_active":"2024-01-24T14:17:20.036507+0000","last_became_peered":"2024-01-24T14:17:20.036507+0000","last_unstale":"2024-01-24T14:17:20.418915+0000","last_undegraded":"2024-01-24T14:17:20.418915+0000","last_fullsized":"2024-01-24T14:17:20.418915+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.446534+0000","last_change":"2024-01-24T14:17:20.437696+0000","last_active":"2024-01-24T14:17:20.446534+0000","last_peered":"2024-01-24T14:17:20.446534+0000","last_clean":"2024-01-24T14:17:20.446534+0000","last_became_active":"2024-01-24T14:17:20.437442+0000","last_became_peered":"2024-01-24T14:17:20.437442+0000","last_unstale":"2024-01-24T14:17:20.446534+0000","last_undegraded":"2024-01-24T14:17:20.446534+0000","last_fullsized":"2024-01-24T14:17:20.446534+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2b","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.444221+0000","last_change":"2024-01-24T14:17:20.027646+0000","last_active":"2024-01-24T14:17:20.444221+0000","last_peered":"2024-01-24T14:17:20.444221+0000","last_clean":"2024-01-24T14:17:20.444221+0000","last_became_active":"2024-01-24T14:17:20.027185+0000","last_became_peered":"2024-01-24T14:17:20.027185+0000","last_unstale":"2024-01-24T14:17:20.444221+0000","last_undegraded":"2024-01-24T14:17:20.444221+0000","last_fullsized":"2024-01-24T14:17:20.444221+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451858+0000","last_change":"2024-01-24T14:17:20.445917+0000","last_active":"2024-01-24T14:17:20.451858+0000","last_peered":"2024-01-24T14:17:20.451858+0000","last_clean":"2024-01-24T14:17:20.451858+0000","last_became_active":"2024-01-24T14:17:20.444661+0000","last_became_peered":"2024-01-24T14:17:20.444661+0000","last_unstale":"2024-01-24T14:17:20.451858+0000","last_undegraded":"2024-01-24T14:17:20.451858+0000","last_fullsized":"2024-01-24T14:17:20.451858+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.422612+0000","last_change":"2024-01-24T14:17:20.025674+0000","last_active":"2024-01-24T14:17:20.422612+0000","last_peered":"2024-01-24T14:17:20.422612+0000","last_clean":"2024-01-24T14:17:20.422612+0000","last_became_active":"2024-01-24T14:17:20.025356+0000","last_became_peered":"2024-01-24T14:17:20.025356+0000","last_unstale":"2024-01-24T14:17:20.422612+0000","last_undegraded":"2024-01-24T14:17:20.422612+0000","last_fullsized":"2024-01-24T14:17:20.422612+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448431+0000","last_change":"2024-01-24T14:17:20.444234+0000","last_active":"2024-01-24T14:17:20.448431+0000","last_peered":"2024-01-24T14:17:20.448431+0000","last_clean":"2024-01-24T14:17:20.448431+0000","last_became_active":"2024-01-24T14:17:20.443949+0000","last_became_peered":"2024-01-24T14:17:20.443949+0000","last_unstale":"2024-01-24T14:17:20.448431+0000","last_undegraded":"2024-01-24T14:17:20.448431+0000","last_fullsized":"2024-01-24T14:17:20.448431+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.435257+0000","last_change":"2024-01-24T14:17:20.028043+0000","last_active":"2024-01-24T14:17:20.435257+0000","last_peered":"2024-01-24T14:17:20.435257+0000","last_clean":"2024-01-24T14:17:20.435257+0000","last_became_active":"2024-01-24T14:17:20.027027+0000","last_became_peered":"2024-01-24T14:17:20.027027+0000","last_unstale":"2024-01-24T14:17:20.435257+0000","last_undegraded":"2024-01-24T14:17:20.435257+0000","last_fullsized":"2024-01-24T14:17:20.435257+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.446080+0000","last_change":"2024-01-24T14:17:20.442564+0000","last_active":"2024-01-24T14:17:20.446080+0000","last_peered":"2024-01-24T14:17:20.446080+0000","last_clean":"2024-01-24T14:17:20.446080+0000","last_became_active":"2024-01-24T14:17:20.442009+0000","last_became_peered":"2024-01-24T14:17:20.442009+0000","last_unstale":"2024-01-24T14:17:20.446080+0000","last_undegraded":"2024-01-24T14:17:20.446080+0000","last_fullsized":"2024-01-24T14:17:20.446080+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.421736+0000","last_change":"2024-01-24T14:17:20.023624+0000","last_active":"2024-01-24T14:17:20.421736+0000","last_peered":"2024-01-24T14:17:20.421736+0000","last_clean":"2024-01-24T14:17:20.421736+0000","last_became_active":"2024-01-24T14:17:20.022507+0000","last_became_peered":"2024-01-24T14:17:20.022507+0000","last_unstale":"2024-01-24T14:17:20.421736+0000","last_undegraded":"2024-01-24T14:17:20.421736+0000","last_fullsized":"2024-01-24T14:17:20.421736+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.433245+0000","last_change":"2024-01-24T14:17:20.427976+0000","last_active":"2024-01-24T14:17:20.433245+0000","last_peered":"2024-01-24T14:17:20.433245+0000","last_clean":"2024-01-24T14:17:20.433245+0000","last_became_active":"2024-01-24T14:17:20.427671+0000","last_became_peered":"2024-01-24T14:17:20.427671+0000","last_unstale":"2024-01-24T14:17:20.433245+0000","last_undegraded":"2024-01-24T14:17:20.433245+0000","last_fullsized":"2024-01-24T14:17:20.433245+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.424720+0000","last_change":"2024-01-24T14:17:20.022793+0000","last_active":"2024-01-24T14:17:20.424720+0000","last_peered":"2024-01-24T14:17:20.424720+0000","last_clean":"2024-01-24T14:17:20.424720+0000","last_became_active":"2024-01-24T14:17:20.022536+0000","last_became_peered":"2024-01-24T14:17:20.022536+0000","last_unstale":"2024-01-24T14:17:20.424720+0000","last_undegraded":"2024-01-24T14:17:20.424720+0000","last_fullsized":"2024-01-24T14:17:20.424720+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.453076+0000","last_change":"2024-01-24T14:17:20.450587+0000","last_active":"2024-01-24T14:17:20.453076+0000","last_peered":"2024-01-24T14:17:20.453076+0000","last_clean":"2024-01-24T14:17:20.453076+0000","last_became_active":"2024-01-24T14:17:20.450189+0000","last_became_peered":"2024-01-24T14:17:20.450189+0000","last_unstale":"2024-01-24T14:17:20.453076+0000","last_undegraded":"2024-01-24T14:17:20.453076+0000","last_fullsized":"2024-01-24T14:17:20.453076+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.419653+0000","last_change":"2024-01-24T14:17:20.024118+0000","last_active":"2024-01-24T14:17:20.419653+0000","last_peered":"2024-01-24T14:17:20.419653+0000","last_clean":"2024-01-24T14:17:20.419653+0000","last_became_active":"2024-01-24T14:17:20.023883+0000","last_became_peered":"2024-01-24T14:17:20.023883+0000","last_unstale":"2024-01-24T14:17:20.419653+0000","last_undegraded":"2024-01-24T14:17:20.419653+0000","last_fullsized":"2024-01-24T14:17:20.419653+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.438262+0000","last_change":"2024-01-24T14:17:20.429107+0000","last_active":"2024-01-24T14:17:20.438262+0000","last_peered":"2024-01-24T14:17:20.438262+0000","last_clean":"2024-01-24T14:17:20.438262+0000","last_became_active":"2024-01-24T14:17:20.428735+0000","last_became_peered":"2024-01-24T14:17:20.428735+0000","last_unstale":"2024-01-24T14:17:20.438262+0000","last_undegraded":"2024-01-24T14:17:20.438262+0000","last_fullsized":"2024-01-24T14:17:20.438262+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.433103+0000","last_change":"2024-01-24T14:17:20.020397+0000","last_active":"2024-01-24T14:17:20.433103+0000","last_peered":"2024-01-24T14:17:20.433103+0000","last_clean":"2024-01-24T14:17:20.433103+0000","last_became_active":"2024-01-24T14:17:20.019754+0000","last_became_peered":"2024-01-24T14:17:20.019754+0000","last_unstale":"2024-01-24T14:17:20.433103+0000","last_undegraded":"2024-01-24T14:17:20.433103+0000","last_fullsized":"2024-01-24T14:17:20.433103+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447507+0000","last_change":"2024-01-24T14:17:20.436591+0000","last_active":"2024-01-24T14:17:20.447507+0000","last_peered":"2024-01-24T14:17:20.447507+0000","last_clean":"2024-01-24T14:17:20.447507+0000","last_became_active":"2024-01-24T14:17:20.436320+0000","last_became_peered":"2024-01-24T14:17:20.436320+0000","last_unstale":"2024-01-24T14:17:20.447507+0000","last_undegraded":"2024-01-24T14:17:20.447507+0000","last_fullsized":"2024-01-24T14:17:20.447507+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.431563+0000","last_change":"2024-01-24T14:17:20.036853+0000","last_active":"2024-01-24T14:17:20.431563+0000","last_peered":"2024-01-24T14:17:20.431563+0000","last_clean":"2024-01-24T14:17:20.431563+0000","last_became_active":"2024-01-24T14:17:20.036567+0000","last_became_peered":"2024-01-24T14:17:20.036567+0000","last_unstale":"2024-01-24T14:17:20.431563+0000","last_undegraded":"2024-01-24T14:17:20.431563+0000","last_fullsized":"2024-01-24T14:17:20.431563+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447330+0000","last_change":"2024-01-24T14:17:20.437674+0000","last_active":"2024-01-24T14:17:20.447330+0000","last_peered":"2024-01-24T14:17:20.447330+0000","last_clean":"2024-01-24T14:17:20.447330+0000","last_became_active":"2024-01-24T14:17:20.437276+0000","last_became_peered":"2024-01-24T14:17:20.437276+0000","last_unstale":"2024-01-24T14:17:20.447330+0000","last_undegraded":"2024-01-24T14:17:20.447330+0000","last_fullsized":"2024-01-24T14:17:20.447330+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.419037+0000","last_change":"2024-01-24T14:17:20.024288+0000","last_active":"2024-01-24T14:17:20.419037+0000","last_peered":"2024-01-24T14:17:20.419037+0000","last_clean":"2024-01-24T14:17:20.419037+0000","last_became_active":"2024-01-24T14:17:20.023983+0000","last_became_peered":"2024-01-24T14:17:20.023983+0000","last_unstale":"2024-01-24T14:17:20.419037+0000","last_undegraded":"2024-01-24T14:17:20.419037+0000","last_fullsized":"2024-01-24T14:17:20.419037+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449128+0000","last_change":"2024-01-24T14:17:20.447184+0000","last_active":"2024-01-24T14:17:20.449128+0000","last_peered":"2024-01-24T14:17:20.449128+0000","last_clean":"2024-01-24T14:17:20.449128+0000","last_became_active":"2024-01-24T14:17:20.439268+0000","last_became_peered":"2024-01-24T14:17:20.439268+0000","last_unstale":"2024-01-24T14:17:20.449128+0000","last_undegraded":"2024-01-24T14:17:20.449128+0000","last_fullsized":"2024-01-24T14:17:20.449128+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423744+0000","last_change":"2024-01-24T14:17:20.030298+0000","last_active":"2024-01-24T14:17:20.423744+0000","last_peered":"2024-01-24T14:17:20.423744+0000","last_clean":"2024-01-24T14:17:20.423744+0000","last_became_active":"2024-01-24T14:17:20.029681+0000","last_became_peered":"2024-01-24T14:17:20.029681+0000","last_unstale":"2024-01-24T14:17:20.423744+0000","last_undegraded":"2024-01-24T14:17:20.423744+0000","last_fullsized":"2024-01-24T14:17:20.423744+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.439786+0000","last_change":"2024-01-24T14:17:20.429845+0000","last_active":"2024-01-24T14:17:20.439786+0000","last_peered":"2024-01-24T14:17:20.439786+0000","last_clean":"2024-01-24T14:17:20.439786+0000","last_became_active":"2024-01-24T14:17:20.429504+0000","last_became_peered":"2024-01-24T14:17:20.429504+0000","last_unstale":"2024-01-24T14:17:20.439786+0000","last_undegraded":"2024-01-24T14:17:20.439786+0000","last_fullsized":"2024-01-24T14:17:20.439786+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.422557+0000","last_change":"2024-01-24T14:17:20.018423+0000","last_active":"2024-01-24T14:17:20.422557+0000","last_peered":"2024-01-24T14:17:20.422557+0000","last_clean":"2024-01-24T14:17:20.422557+0000","last_became_active":"2024-01-24T14:17:20.017590+0000","last_became_peered":"2024-01-24T14:17:20.017590+0000","last_unstale":"2024-01-24T14:17:20.422557+0000","last_undegraded":"2024-01-24T14:17:20.422557+0000","last_fullsized":"2024-01-24T14:17:20.422557+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451741+0000","last_change":"2024-01-24T14:17:20.445375+0000","last_active":"2024-01-24T14:17:20.451741+0000","last_peered":"2024-01-24T14:17:20.451741+0000","last_clean":"2024-01-24T14:17:20.451741+0000","last_became_active":"2024-01-24T14:17:20.444280+0000","last_became_peered":"2024-01-24T14:17:20.444280+0000","last_unstale":"2024-01-24T14:17:20.451741+0000","last_undegraded":"2024-01-24T14:17:20.451741+0000","last_fullsized":"2024-01-24T14:17:20.451741+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.421857+0000","last_change":"2024-01-24T14:17:20.018602+0000","last_active":"2024-01-24T14:17:20.421857+0000","last_peered":"2024-01-24T14:17:20.421857+0000","last_clean":"2024-01-24T14:17:20.421857+0000","last_became_active":"2024-01-24T14:17:20.018276+0000","last_became_peered":"2024-01-24T14:17:20.018276+0000","last_unstale":"2024-01-24T14:17:20.421857+0000","last_undegraded":"2024-01-24T14:17:20.421857+0000","last_fullsized":"2024-01-24T14:17:20.421857+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.445523+0000","last_change":"2024-01-24T14:17:20.438173+0000","last_active":"2024-01-24T14:17:20.445523+0000","last_peered":"2024-01-24T14:17:20.445523+0000","last_clean":"2024-01-24T14:17:20.445523+0000","last_became_active":"2024-01-24T14:17:20.437862+0000","last_became_peered":"2024-01-24T14:17:20.437862+0000","last_unstale":"2024-01-24T14:17:20.445523+0000","last_undegraded":"2024-01-24T14:17:20.445523+0000","last_fullsized":"2024-01-24T14:17:20.445523+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447434+0000","last_change":"2024-01-24T14:17:20.029454+0000","last_active":"2024-01-24T14:17:20.447434+0000","last_peered":"2024-01-24T14:17:20.447434+0000","last_clean":"2024-01-24T14:17:20.447434+0000","last_became_active":"2024-01-24T14:17:20.029263+0000","last_became_peered":"2024-01-24T14:17:20.029263+0000","last_unstale":"2024-01-24T14:17:20.447434+0000","last_undegraded":"2024-01-24T14:17:20.447434+0000","last_fullsized":"2024-01-24T14:17:20.447434+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451925+0000","last_change":"2024-01-24T14:17:20.449441+0000","last_active":"2024-01-24T14:17:20.451925+0000","last_peered":"2024-01-24T14:17:20.451925+0000","last_clean":"2024-01-24T14:17:20.451925+0000","last_became_active":"2024-01-24T14:17:20.449177+0000","last_became_peered":"2024-01-24T14:17:20.449177+0000","last_unstale":"2024-01-24T14:17:20.451925+0000","last_undegraded":"2024-01-24T14:17:20.451925+0000","last_fullsized":"2024-01-24T14:17:20.451925+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.437391+0000","last_change":"2024-01-24T14:17:20.036605+0000","last_active":"2024-01-24T14:17:20.437391+0000","last_peered":"2024-01-24T14:17:20.437391+0000","last_clean":"2024-01-24T14:17:20.437391+0000","last_became_active":"2024-01-24T14:17:20.036362+0000","last_became_peered":"2024-01-24T14:17:20.036362+0000","last_unstale":"2024-01-24T14:17:20.437391+0000","last_undegraded":"2024-01-24T14:17:20.437391+0000","last_fullsized":"2024-01-24T14:17:20.437391+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.454289+0000","last_change":"2024-01-24T14:17:20.449329+0000","last_active":"2024-01-24T14:17:20.454289+0000","last_peered":"2024-01-24T14:17:20.454289+0000","last_clean":"2024-01-24T14:17:20.454289+0000","last_became_active":"2024-01-24T14:17:20.448991+0000","last_became_peered":"2024-01-24T14:17:20.448991+0000","last_unstale":"2024-01-24T14:17:20.454289+0000","last_undegraded":"2024-01-24T14:17:20.454289+0000","last_fullsized":"2024-01-24T14:17:20.454289+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.441220+0000","last_change":"2024-01-24T14:17:20.034832+0000","last_active":"2024-01-24T14:17:20.441220+0000","last_peered":"2024-01-24T14:17:20.441220+0000","last_clean":"2024-01-24T14:17:20.441220+0000","last_became_active":"2024-01-24T14:17:20.034429+0000","last_became_peered":"2024-01-24T14:17:20.034429+0000","last_unstale":"2024-01-24T14:17:20.441220+0000","last_undegraded":"2024-01-24T14:17:20.441220+0000","last_fullsized":"2024-01-24T14:17:20.441220+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451665+0000","last_change":"2024-01-24T14:17:20.445208+0000","last_active":"2024-01-24T14:17:20.451665+0000","last_peered":"2024-01-24T14:17:20.451665+0000","last_clean":"2024-01-24T14:17:20.451665+0000","last_became_active":"2024-01-24T14:17:20.444202+0000","last_became_peered":"2024-01-24T14:17:20.444202+0000","last_unstale":"2024-01-24T14:17:20.451665+0000","last_undegraded":"2024-01-24T14:17:20.451665+0000","last_fullsized":"2024-01-24T14:17:20.451665+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.434492+0000","last_change":"2024-01-24T14:17:20.035895+0000","last_active":"2024-01-24T14:17:20.434492+0000","last_peered":"2024-01-24T14:17:20.434492+0000","last_clean":"2024-01-24T14:17:20.434492+0000","last_became_active":"2024-01-24T14:17:20.035366+0000","last_became_peered":"2024-01-24T14:17:20.035366+0000","last_unstale":"2024-01-24T14:17:20.434492+0000","last_undegraded":"2024-01-24T14:17:20.434492+0000","last_fullsized":"2024-01-24T14:17:20.434492+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.441332+0000","last_change":"2024-01-24T14:17:20.436366+0000","last_active":"2024-01-24T14:17:20.441332+0000","last_peered":"2024-01-24T14:17:20.441332+0000","last_clean":"2024-01-24T14:17:20.441332+0000","last_became_active":"2024-01-24T14:17:20.436044+0000","last_became_peered":"2024-01-24T14:17:20.436044+0000","last_unstale":"2024-01-24T14:17:20.441332+0000","last_undegraded":"2024-01-24T14:17:20.441332+0000","last_fullsized":"2024-01-24T14:17:20.441332+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.433900+0000","last_change":"2024-01-24T14:17:20.027356+0000","last_active":"2024-01-24T14:17:20.433900+0000","last_peered":"2024-01-24T14:17:20.433900+0000","last_clean":"2024-01-24T14:17:20.433900+0000","last_became_active":"2024-01-24T14:17:20.027010+0000","last_became_peered":"2024-01-24T14:17:20.027010+0000","last_unstale":"2024-01-24T14:17:20.433900+0000","last_undegraded":"2024-01-24T14:17:20.433900+0000","last_fullsized":"2024-01-24T14:17:20.433900+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.442818+0000","last_change":"2024-01-24T14:17:20.438826+0000","last_active":"2024-01-24T14:17:20.442818+0000","last_peered":"2024-01-24T14:17:20.442818+0000","last_clean":"2024-01-24T14:17:20.442818+0000","last_became_active":"2024-01-24T14:17:20.438507+0000","last_became_peered":"2024-01-24T14:17:20.438507+0000","last_unstale":"2024-01-24T14:17:20.442818+0000","last_undegraded":"2024-01-24T14:17:20.442818+0000","last_fullsized":"2024-01-24T14:17:20.442818+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.425217+0000","last_change":"2024-01-24T14:17:20.030030+0000","last_active":"2024-01-24T14:17:20.425217+0000","last_peered":"2024-01-24T14:17:20.425217+0000","last_clean":"2024-01-24T14:17:20.425217+0000","last_became_active":"2024-01-24T14:17:20.029271+0000","last_became_peered":"2024-01-24T14:17:20.029271+0000","last_unstale":"2024-01-24T14:17:20.425217+0000","last_undegraded":"2024-01-24T14:17:20.425217+0000","last_fullsized":"2024-01-24T14:17:20.425217+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449030+0000","last_change":"2024-01-24T14:17:20.444435+0000","last_active":"2024-01-24T14:17:20.449030+0000","last_peered":"2024-01-24T14:17:20.449030+0000","last_clean":"2024-01-24T14:17:20.449030+0000","last_became_active":"2024-01-24T14:17:20.444107+0000","last_became_peered":"2024-01-24T14:17:20.444107+0000","last_unstale":"2024-01-24T14:17:20.449030+0000","last_undegraded":"2024-01-24T14:17:20.449030+0000","last_fullsized":"2024-01-24T14:17:20.449030+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.422268+0000","last_change":"2024-01-24T14:17:20.023190+0000","last_active":"2024-01-24T14:17:20.422268+0000","last_peered":"2024-01-24T14:17:20.422268+0000","last_clean":"2024-01-24T14:17:20.422268+0000","last_became_active":"2024-01-24T14:17:20.022894+0000","last_became_peered":"2024-01-24T14:17:20.022894+0000","last_unstale":"2024-01-24T14:17:20.422268+0000","last_undegraded":"2024-01-24T14:17:20.422268+0000","last_fullsized":"2024-01-24T14:17:20.422268+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447836+0000","last_change":"2024-01-24T14:17:20.442688+0000","last_active":"2024-01-24T14:17:20.447836+0000","last_peered":"2024-01-24T14:17:20.447836+0000","last_clean":"2024-01-24T14:17:20.447836+0000","last_became_active":"2024-01-24T14:17:20.442384+0000","last_became_peered":"2024-01-24T14:17:20.442384+0000","last_unstale":"2024-01-24T14:17:20.447836+0000","last_undegraded":"2024-01-24T14:17:20.447836+0000","last_fullsized":"2024-01-24T14:17:20.447836+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.433616+0000","last_change":"2024-01-24T14:17:20.030600+0000","last_active":"2024-01-24T14:17:20.433616+0000","last_peered":"2024-01-24T14:17:20.433616+0000","last_clean":"2024-01-24T14:17:20.433616+0000","last_became_active":"2024-01-24T14:17:20.030260+0000","last_became_peered":"2024-01-24T14:17:20.030260+0000","last_unstale":"2024-01-24T14:17:20.433616+0000","last_undegraded":"2024-01-24T14:17:20.433616+0000","last_fullsized":"2024-01-24T14:17:20.433616+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.442803+0000","last_change":"2024-01-24T14:17:20.437473+0000","last_active":"2024-01-24T14:17:20.442803+0000","last_peered":"2024-01-24T14:17:20.442803+0000","last_clean":"2024-01-24T14:17:20.442803+0000","last_became_active":"2024-01-24T14:17:20.437211+0000","last_became_peered":"2024-01-24T14:17:20.437211+0000","last_unstale":"2024-01-24T14:17:20.442803+0000","last_undegraded":"2024-01-24T14:17:20.442803+0000","last_fullsized":"2024-01-24T14:17:20.442803+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.446096+0000","last_change":"2024-01-24T14:17:20.015954+0000","last_active":"2024-01-24T14:17:20.446096+0000","last_peered":"2024-01-24T14:17:20.446096+0000","last_clean":"2024-01-24T14:17:20.446096+0000","last_became_active":"2024-01-24T14:17:20.015682+0000","last_became_peered":"2024-01-24T14:17:20.015682+0000","last_unstale":"2024-01-24T14:17:20.446096+0000","last_undegraded":"2024-01-24T14:17:20.446096+0000","last_fullsized":"2024-01-24T14:17:20.446096+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.441771+0000","last_change":"2024-01-24T14:17:20.440811+0000","last_active":"2024-01-24T14:17:20.441771+0000","last_peered":"2024-01-24T14:17:20.441771+0000","last_clean":"2024-01-24T14:17:20.441771+0000","last_became_active":"2024-01-24T14:17:20.440486+0000","last_became_peered":"2024-01-24T14:17:20.440486+0000","last_unstale":"2024-01-24T14:17:20.441771+0000","last_undegraded":"2024-01-24T14:17:20.441771+0000","last_fullsized":"2024-01-24T14:17:20.441771+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.441487+0000","last_change":"2024-01-24T14:17:20.037718+0000","last_active":"2024-01-24T14:17:20.441487+0000","last_peered":"2024-01-24T14:17:20.441487+0000","last_clean":"2024-01-24T14:17:20.441487+0000","last_became_active":"2024-01-24T14:17:20.037463+0000","last_became_peered":"2024-01-24T14:17:20.037463+0000","last_unstale":"2024-01-24T14:17:20.441487+0000","last_undegraded":"2024-01-24T14:17:20.441487+0000","last_fullsized":"2024-01-24T14:17:20.441487+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.437312+0000","last_change":"2024-01-24T14:17:20.429721+0000","last_active":"2024-01-24T14:17:20.437312+0000","last_peered":"2024-01-24T14:17:20.437312+0000","last_clean":"2024-01-24T14:17:20.437312+0000","last_became_active":"2024-01-24T14:17:20.429442+0000","last_became_peered":"2024-01-24T14:17:20.429442+0000","last_unstale":"2024-01-24T14:17:20.437312+0000","last_undegraded":"2024-01-24T14:17:20.437312+0000","last_fullsized":"2024-01-24T14:17:20.437312+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423945+0000","last_change":"2024-01-24T14:17:20.024636+0000","last_active":"2024-01-24T14:17:20.423945+0000","last_peered":"2024-01-24T14:17:20.423945+0000","last_clean":"2024-01-24T14:17:20.423945+0000","last_became_active":"2024-01-24T14:17:20.024408+0000","last_became_peered":"2024-01-24T14:17:20.024408+0000","last_unstale":"2024-01-24T14:17:20.423945+0000","last_undegraded":"2024-01-24T14:17:20.423945+0000","last_fullsized":"2024-01-24T14:17:20.423945+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447999+0000","last_change":"2024-01-24T14:17:20.445755+0000","last_active":"2024-01-24T14:17:20.447999+0000","last_peered":"2024-01-24T14:17:20.447999+0000","last_clean":"2024-01-24T14:17:20.447999+0000","last_became_active":"2024-01-24T14:17:20.445267+0000","last_became_peered":"2024-01-24T14:17:20.445267+0000","last_unstale":"2024-01-24T14:17:20.447999+0000","last_undegraded":"2024-01-24T14:17:20.447999+0000","last_fullsized":"2024-01-24T14:17:20.447999+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.421578+0000","last_change":"2024-01-24T14:17:20.024180+0000","last_active":"2024-01-24T14:17:20.421578+0000","last_peered":"2024-01-24T14:17:20.421578+0000","last_clean":"2024-01-24T14:17:20.421578+0000","last_became_active":"2024-01-24T14:17:20.023866+0000","last_became_peered":"2024-01-24T14:17:20.023866+0000","last_unstale":"2024-01-24T14:17:20.421578+0000","last_undegraded":"2024-01-24T14:17:20.421578+0000","last_fullsized":"2024-01-24T14:17:20.421578+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449039+0000","last_change":"2024-01-24T14:17:20.443241+0000","last_active":"2024-01-24T14:17:20.449039+0000","last_peered":"2024-01-24T14:17:20.449039+0000","last_clean":"2024-01-24T14:17:20.449039+0000","last_became_active":"2024-01-24T14:17:20.439755+0000","last_became_peered":"2024-01-24T14:17:20.439755+0000","last_unstale":"2024-01-24T14:17:20.449039+0000","last_undegraded":"2024-01-24T14:17:20.449039+0000","last_fullsized":"2024-01-24T14:17:20.449039+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.437747+0000","last_change":"2024-01-24T14:17:20.018026+0000","last_active":"2024-01-24T14:17:20.437747+0000","last_peered":"2024-01-24T14:17:20.437747+0000","last_clean":"2024-01-24T14:17:20.437747+0000","last_became_active":"2024-01-24T14:17:20.017715+0000","last_became_peered":"2024-01-24T14:17:20.017715+0000","last_unstale":"2024-01-24T14:17:20.437747+0000","last_undegraded":"2024-01-24T14:17:20.437747+0000","last_fullsized":"2024-01-24T14:17:20.437747+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.442133+0000","last_change":"2024-01-24T14:17:20.434915+0000","last_active":"2024-01-24T14:17:20.442133+0000","last_peered":"2024-01-24T14:17:20.442133+0000","last_clean":"2024-01-24T14:17:20.442133+0000","last_became_active":"2024-01-24T14:17:20.434607+0000","last_became_peered":"2024-01-24T14:17:20.434607+0000","last_unstale":"2024-01-24T14:17:20.442133+0000","last_undegraded":"2024-01-24T14:17:20.442133+0000","last_fullsized":"2024-01-24T14:17:20.442133+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423873+0000","last_change":"2024-01-24T14:17:20.025281+0000","last_active":"2024-01-24T14:17:20.423873+0000","last_peered":"2024-01-24T14:17:20.423873+0000","last_clean":"2024-01-24T14:17:20.423873+0000","last_became_active":"2024-01-24T14:17:20.024504+0000","last_became_peered":"2024-01-24T14:17:20.024504+0000","last_unstale":"2024-01-24T14:17:20.423873+0000","last_undegraded":"2024-01-24T14:17:20.423873+0000","last_fullsized":"2024-01-24T14:17:20.423873+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447824+0000","last_change":"2024-01-24T14:17:20.441907+0000","last_active":"2024-01-24T14:17:20.447824+0000","last_peered":"2024-01-24T14:17:20.447824+0000","last_clean":"2024-01-24T14:17:20.447824+0000","last_became_active":"2024-01-24T14:17:20.441615+0000","last_became_peered":"2024-01-24T14:17:20.441615+0000","last_unstale":"2024-01-24T14:17:20.447824+0000","last_undegraded":"2024-01-24T14:17:20.447824+0000","last_fullsized":"2024-01-24T14:17:20.447824+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423190+0000","last_change":"2024-01-24T14:17:20.028922+0000","last_active":"2024-01-24T14:17:20.423190+0000","last_peered":"2024-01-24T14:17:20.423190+0000","last_clean":"2024-01-24T14:17:20.423190+0000","last_became_active":"2024-01-24T14:17:20.028549+0000","last_became_peered":"2024-01-24T14:17:20.028549+0000","last_unstale":"2024-01-24T14:17:20.423190+0000","last_undegraded":"2024-01-24T14:17:20.423190+0000","last_fullsized":"2024-01-24T14:17:20.423190+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451842+0000","last_change":"2024-01-24T14:17:20.445971+0000","last_active":"2024-01-24T14:17:20.451842+0000","last_peered":"2024-01-24T14:17:20.451842+0000","last_clean":"2024-01-24T14:17:20.451842+0000","last_became_active":"2024-01-24T14:17:20.445160+0000","last_became_peered":"2024-01-24T14:17:20.445160+0000","last_unstale":"2024-01-24T14:17:20.451842+0000","last_undegraded":"2024-01-24T14:17:20.451842+0000","last_fullsized":"2024-01-24T14:17:20.451842+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.430666+0000","last_change":"2024-01-24T14:17:20.037299+0000","last_active":"2024-01-24T14:17:20.430666+0000","last_peered":"2024-01-24T14:17:20.430666+0000","last_clean":"2024-01-24T14:17:20.430666+0000","last_became_active":"2024-01-24T14:17:20.036972+0000","last_became_peered":"2024-01-24T14:17:20.036972+0000","last_unstale":"2024-01-24T14:17:20.430666+0000","last_undegraded":"2024-01-24T14:17:20.430666+0000","last_fullsized":"2024-01-24T14:17:20.430666+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.435012+0000","last_change":"2024-01-24T14:17:20.429204+0000","last_active":"2024-01-24T14:17:20.435012+0000","last_peered":"2024-01-24T14:17:20.435012+0000","last_clean":"2024-01-24T14:17:20.435012+0000","last_became_active":"2024-01-24T14:17:20.428903+0000","last_became_peered":"2024-01-24T14:17:20.428903+0000","last_unstale":"2024-01-24T14:17:20.435012+0000","last_undegraded":"2024-01-24T14:17:20.435012+0000","last_fullsized":"2024-01-24T14:17:20.435012+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.443079+0000","last_change":"2024-01-24T14:17:20.017724+0000","last_active":"2024-01-24T14:17:20.443079+0000","last_peered":"2024-01-24T14:17:20.443079+0000","last_clean":"2024-01-24T14:17:20.443079+0000","last_became_active":"2024-01-24T14:17:20.017445+0000","last_became_peered":"2024-01-24T14:17:20.017445+0000","last_unstale":"2024-01-24T14:17:20.443079+0000","last_undegraded":"2024-01-24T14:17:20.443079+0000","last_fullsized":"2024-01-24T14:17:20.443079+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448140+0000","last_change":"2024-01-24T14:17:20.438796+0000","last_active":"2024-01-24T14:17:20.448140+0000","last_peered":"2024-01-24T14:17:20.448140+0000","last_clean":"2024-01-24T14:17:20.448140+0000","last_became_active":"2024-01-24T14:17:20.438504+0000","last_became_peered":"2024-01-24T14:17:20.438504+0000","last_unstale":"2024-01-24T14:17:20.448140+0000","last_undegraded":"2024-01-24T14:17:20.448140+0000","last_fullsized":"2024-01-24T14:17:20.448140+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.428283+0000","last_change":"2024-01-24T14:17:20.028340+0000","last_active":"2024-01-24T14:17:20.428283+0000","last_peered":"2024-01-24T14:17:20.428283+0000","last_clean":"2024-01-24T14:17:20.428283+0000","last_became_active":"2024-01-24T14:17:20.028046+0000","last_became_peered":"2024-01-24T14:17:20.028046+0000","last_unstale":"2024-01-24T14:17:20.428283+0000","last_undegraded":"2024-01-24T14:17:20.428283+0000","last_fullsized":"2024-01-24T14:17:20.428283+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.453080+0000","last_change":"2024-01-24T14:17:20.450691+0000","last_active":"2024-01-24T14:17:20.453080+0000","last_peered":"2024-01-24T14:17:20.453080+0000","last_clean":"2024-01-24T14:17:20.453080+0000","last_became_active":"2024-01-24T14:17:20.450409+0000","last_became_peered":"2024-01-24T14:17:20.450409+0000","last_unstale":"2024-01-24T14:17:20.453080+0000","last_undegraded":"2024-01-24T14:17:20.453080+0000","last_fullsized":"2024-01-24T14:17:20.453080+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.435092+0000","last_change":"2024-01-24T14:17:20.030946+0000","last_active":"2024-01-24T14:17:20.435092+0000","last_peered":"2024-01-24T14:17:20.435092+0000","last_clean":"2024-01-24T14:17:20.435092+0000","last_became_active":"2024-01-24T14:17:20.030458+0000","last_became_peered":"2024-01-24T14:17:20.030458+0000","last_unstale":"2024-01-24T14:17:20.435092+0000","last_undegraded":"2024-01-24T14:17:20.435092+0000","last_fullsized":"2024-01-24T14:17:20.435092+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.442721+0000","last_change":"2024-01-24T14:17:20.431613+0000","last_active":"2024-01-24T14:17:20.442721+0000","last_peered":"2024-01-24T14:17:20.442721+0000","last_clean":"2024-01-24T14:17:20.442721+0000","last_became_active":"2024-01-24T14:17:20.431320+0000","last_became_peered":"2024-01-24T14:17:20.431320+0000","last_unstale":"2024-01-24T14:17:20.442721+0000","last_undegraded":"2024-01-24T14:17:20.442721+0000","last_fullsized":"2024-01-24T14:17:20.442721+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.431682+0000","last_change":"2024-01-24T14:17:20.034473+0000","last_active":"2024-01-24T14:17:20.431682+0000","last_peered":"2024-01-24T14:17:20.431682+0000","last_clean":"2024-01-24T14:17:20.431682+0000","last_became_active":"2024-01-24T14:17:20.034154+0000","last_became_peered":"2024-01-24T14:17:20.034154+0000","last_unstale":"2024-01-24T14:17:20.431682+0000","last_undegraded":"2024-01-24T14:17:20.431682+0000","last_fullsized":"2024-01-24T14:17:20.431682+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451812+0000","last_change":"2024-01-24T14:17:20.445791+0000","last_active":"2024-01-24T14:17:20.451812+0000","last_peered":"2024-01-24T14:17:20.451812+0000","last_clean":"2024-01-24T14:17:20.451812+0000","last_became_active":"2024-01-24T14:17:20.444443+0000","last_became_peered":"2024-01-24T14:17:20.444443+0000","last_unstale":"2024-01-24T14:17:20.451812+0000","last_undegraded":"2024-01-24T14:17:20.451812+0000","last_fullsized":"2024-01-24T14:17:20.451812+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.433921+0000","last_change":"2024-01-24T14:17:20.026060+0000","last_active":"2024-01-24T14:17:20.433921+0000","last_peered":"2024-01-24T14:17:20.433921+0000","last_clean":"2024-01-24T14:17:20.433921+0000","last_became_active":"2024-01-24T14:17:20.025751+0000","last_became_peered":"2024-01-24T14:17:20.025751+0000","last_unstale":"2024-01-24T14:17:20.433921+0000","last_undegraded":"2024-01-24T14:17:20.433921+0000","last_fullsized":"2024-01-24T14:17:20.433921+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.440817+0000","last_change":"2024-01-24T14:17:20.432753+0000","last_active":"2024-01-24T14:17:20.440817+0000","last_peered":"2024-01-24T14:17:20.440817+0000","last_clean":"2024-01-24T14:17:20.440817+0000","last_became_active":"2024-01-24T14:17:20.432407+0000","last_became_peered":"2024-01-24T14:17:20.432407+0000","last_unstale":"2024-01-24T14:17:20.440817+0000","last_undegraded":"2024-01-24T14:17:20.440817+0000","last_fullsized":"2024-01-24T14:17:20.440817+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.428882+0000","last_change":"2024-01-24T14:17:20.034798+0000","last_active":"2024-01-24T14:17:20.428882+0000","last_peered":"2024-01-24T14:17:20.428882+0000","last_clean":"2024-01-24T14:17:20.428882+0000","last_became_active":"2024-01-24T14:17:20.034552+0000","last_became_peered":"2024-01-24T14:17:20.034552+0000","last_unstale":"2024-01-24T14:17:20.428882+0000","last_undegraded":"2024-01-24T14:17:20.428882+0000","last_fullsized":"2024-01-24T14:17:20.428882+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.445827+0000","last_change":"2024-01-24T14:17:20.442888+0000","last_active":"2024-01-24T14:17:20.445827+0000","last_peered":"2024-01-24T14:17:20.445827+0000","last_clean":"2024-01-24T14:17:20.445827+0000","last_became_active":"2024-01-24T14:17:20.442596+0000","last_became_peered":"2024-01-24T14:17:20.442596+0000","last_unstale":"2024-01-24T14:17:20.445827+0000","last_undegraded":"2024-01-24T14:17:20.445827+0000","last_fullsized":"2024-01-24T14:17:20.445827+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423297+0000","last_change":"2024-01-24T14:17:20.022465+0000","last_active":"2024-01-24T14:17:20.423297+0000","last_peered":"2024-01-24T14:17:20.423297+0000","last_clean":"2024-01-24T14:17:20.423297+0000","last_became_active":"2024-01-24T14:17:20.022142+0000","last_became_peered":"2024-01-24T14:17:20.022142+0000","last_unstale":"2024-01-24T14:17:20.423297+0000","last_undegraded":"2024-01-24T14:17:20.423297+0000","last_fullsized":"2024-01-24T14:17:20.423297+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.453069+0000","last_change":"2024-01-24T14:17:20.450415+0000","last_active":"2024-01-24T14:17:20.453069+0000","last_peered":"2024-01-24T14:17:20.453069+0000","last_clean":"2024-01-24T14:17:20.453069+0000","last_became_active":"2024-01-24T14:17:20.449996+0000","last_became_peered":"2024-01-24T14:17:20.449996+0000","last_unstale":"2024-01-24T14:17:20.453069+0000","last_undegraded":"2024-01-24T14:17:20.453069+0000","last_fullsized":"2024-01-24T14:17:20.453069+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.430608+0000","last_change":"2024-01-24T14:17:20.021844+0000","last_active":"2024-01-24T14:17:20.430608+0000","last_peered":"2024-01-24T14:17:20.430608+0000","last_clean":"2024-01-24T14:17:20.430608+0000","last_became_active":"2024-01-24T14:17:20.021506+0000","last_became_peered":"2024-01-24T14:17:20.021506+0000","last_unstale":"2024-01-24T14:17:20.430608+0000","last_undegraded":"2024-01-24T14:17:20.430608+0000","last_fullsized":"2024-01-24T14:17:20.430608+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451761+0000","last_change":"2024-01-24T14:17:20.445477+0000","last_active":"2024-01-24T14:17:20.451761+0000","last_peered":"2024-01-24T14:17:20.451761+0000","last_clean":"2024-01-24T14:17:20.451761+0000","last_became_active":"2024-01-24T14:17:20.444918+0000","last_became_peered":"2024-01-24T14:17:20.444918+0000","last_unstale":"2024-01-24T14:17:20.451761+0000","last_undegraded":"2024-01-24T14:17:20.451761+0000","last_fullsized":"2024-01-24T14:17:20.451761+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.450254+0000","last_change":"2024-01-24T14:17:20.020286+0000","last_active":"2024-01-24T14:17:20.450254+0000","last_peered":"2024-01-24T14:17:20.450254+0000","last_clean":"2024-01-24T14:17:20.450254+0000","last_became_active":"2024-01-24T14:17:20.020025+0000","last_became_peered":"2024-01-24T14:17:20.020025+0000","last_unstale":"2024-01-24T14:17:20.450254+0000","last_undegraded":"2024-01-24T14:17:20.450254+0000","last_fullsized":"2024-01-24T14:17:20.450254+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.434613+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:17:20.434613+0000","last_peered":"2024-01-24T14:17:20.434613+0000","last_clean":"2024-01-24T14:17:20.434613+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:17:20.434613+0000","last_undegraded":"2024-01-24T14:17:20.434613+0000","last_fullsized":"2024-01-24T14:17:20.434613+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449140+0000","last_change":"2024-01-24T14:17:20.447404+0000","last_active":"2024-01-24T14:17:20.449140+0000","last_peered":"2024-01-24T14:17:20.449140+0000","last_clean":"2024-01-24T14:17:20.449140+0000","last_became_active":"2024-01-24T14:17:20.439657+0000","last_became_peered":"2024-01-24T14:17:20.439657+0000","last_unstale":"2024-01-24T14:17:20.449140+0000","last_undegraded":"2024-01-24T14:17:20.449140+0000","last_fullsized":"2024-01-24T14:17:20.449140+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.418426+0000","last_change":"2024-01-24T14:17:20.020233+0000","last_active":"2024-01-24T14:17:20.418426+0000","last_peered":"2024-01-24T14:17:20.418426+0000","last_clean":"2024-01-24T14:17:20.418426+0000","last_became_active":"2024-01-24T14:17:20.019923+0000","last_became_peered":"2024-01-24T14:17:20.019923+0000","last_unstale":"2024-01-24T14:17:20.418426+0000","last_undegraded":"2024-01-24T14:17:20.418426+0000","last_fullsized":"2024-01-24T14:17:20.418426+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.421302+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:17:20.421302+0000","last_peered":"2024-01-24T14:17:20.421302+0000","last_clean":"2024-01-24T14:17:20.421302+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:17:20.421302+0000","last_undegraded":"2024-01-24T14:17:20.421302+0000","last_fullsized":"2024-01-24T14:17:20.421302+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447849+0000","last_change":"2024-01-24T14:17:20.442874+0000","last_active":"2024-01-24T14:17:20.447849+0000","last_peered":"2024-01-24T14:17:20.447849+0000","last_clean":"2024-01-24T14:17:20.447849+0000","last_became_active":"2024-01-24T14:17:20.442594+0000","last_became_peered":"2024-01-24T14:17:20.442594+0000","last_unstale":"2024-01-24T14:17:20.447849+0000","last_undegraded":"2024-01-24T14:17:20.447849+0000","last_fullsized":"2024-01-24T14:17:20.447849+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.434688+0000","last_change":"2024-01-24T14:17:20.029762+0000","last_active":"2024-01-24T14:17:20.434688+0000","last_peered":"2024-01-24T14:17:20.434688+0000","last_clean":"2024-01-24T14:17:20.434688+0000","last_became_active":"2024-01-24T14:17:20.029474+0000","last_became_peered":"2024-01-24T14:17:20.029474+0000","last_unstale":"2024-01-24T14:17:20.434688+0000","last_undegraded":"2024-01-24T14:17:20.434688+0000","last_fullsized":"2024-01-24T14:17:20.434688+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.418040+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:17:20.418040+0000","last_peered":"2024-01-24T14:17:20.418040+0000","last_clean":"2024-01-24T14:17:20.418040+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:17:20.418040+0000","last_undegraded":"2024-01-24T14:17:20.418040+0000","last_fullsized":"2024-01-24T14:17:20.418040+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451788+0000","last_change":"2024-01-24T14:17:20.445595+0000","last_active":"2024-01-24T14:17:20.451788+0000","last_peered":"2024-01-24T14:17:20.451788+0000","last_clean":"2024-01-24T14:17:20.451788+0000","last_became_active":"2024-01-24T14:17:20.444716+0000","last_became_peered":"2024-01-24T14:17:20.444716+0000","last_unstale":"2024-01-24T14:17:20.451788+0000","last_undegraded":"2024-01-24T14:17:20.451788+0000","last_fullsized":"2024-01-24T14:17:20.451788+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449022+0000","last_change":"2024-01-24T14:17:20.027508+0000","last_active":"2024-01-24T14:17:20.449022+0000","last_peered":"2024-01-24T14:17:20.449022+0000","last_clean":"2024-01-24T14:17:20.449022+0000","last_became_active":"2024-01-24T14:17:20.027246+0000","last_became_peered":"2024-01-24T14:17:20.027246+0000","last_unstale":"2024-01-24T14:17:20.449022+0000","last_undegraded":"2024-01-24T14:17:20.449022+0000","last_fullsized":"2024-01-24T14:17:20.449022+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.422230+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:17:20.422230+0000","last_peered":"2024-01-24T14:17:20.422230+0000","last_clean":"2024-01-24T14:17:20.422230+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:17:20.422230+0000","last_undegraded":"2024-01-24T14:17:20.422230+0000","last_fullsized":"2024-01-24T14:17:20.422230+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.427575+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:17:20.427575+0000","last_peered":"2024-01-24T14:17:20.427575+0000","last_clean":"2024-01-24T14:17:20.427575+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:17:20.427575+0000","last_undegraded":"2024-01-24T14:17:20.427575+0000","last_fullsized":"2024-01-24T14:17:20.427575+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448634+0000","last_change":"2024-01-24T14:17:20.442710+0000","last_active":"2024-01-24T14:17:20.448634+0000","last_peered":"2024-01-24T14:17:20.448634+0000","last_clean":"2024-01-24T14:17:20.448634+0000","last_became_active":"2024-01-24T14:17:20.442434+0000","last_became_peered":"2024-01-24T14:17:20.442434+0000","last_unstale":"2024-01-24T14:17:20.448634+0000","last_undegraded":"2024-01-24T14:17:20.448634+0000","last_fullsized":"2024-01-24T14:17:20.448634+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"152'4","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447271+0000","last_change":"2024-01-24T14:17:20.027163+0000","last_active":"2024-01-24T14:17:20.447271+0000","last_peered":"2024-01-24T14:17:20.447271+0000","last_clean":"2024-01-24T14:17:20.447271+0000","last_became_active":"2024-01-24T14:17:20.026862+0000","last_became_peered":"2024-01-24T14:17:20.026862+0000","last_unstale":"2024-01-24T14:17:20.447271+0000","last_undegraded":"2024-01-24T14:17:20.447271+0000","last_fullsized":"2024-01-24T14:17:20.447271+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.427474+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:17:20.427474+0000","last_peered":"2024-01-24T14:17:20.427474+0000","last_clean":"2024-01-24T14:17:20.427474+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:17:20.427474+0000","last_undegraded":"2024-01-24T14:17:20.427474+0000","last_fullsized":"2024-01-24T14:17:20.427474+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.454308+0000","last_change":"2024-01-24T14:17:20.449377+0000","last_active":"2024-01-24T14:17:20.454308+0000","last_peered":"2024-01-24T14:17:20.454308+0000","last_clean":"2024-01-24T14:17:20.454308+0000","last_became_active":"2024-01-24T14:17:20.449015+0000","last_became_peered":"2024-01-24T14:17:20.449015+0000","last_unstale":"2024-01-24T14:17:20.454308+0000","last_undegraded":"2024-01-24T14:17:20.454308+0000","last_fullsized":"2024-01-24T14:17:20.454308+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"152'3","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.440656+0000","last_change":"2024-01-24T14:17:20.024502+0000","last_active":"2024-01-24T14:17:20.440656+0000","last_peered":"2024-01-24T14:17:20.440656+0000","last_clean":"2024-01-24T14:17:20.440656+0000","last_became_active":"2024-01-24T14:17:20.024226+0000","last_became_peered":"2024-01-24T14:17:20.024226+0000","last_unstale":"2024-01-24T14:17:20.440656+0000","last_undegraded":"2024-01-24T14:17:20.440656+0000","last_fullsized":"2024-01-24T14:17:20.440656+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.428398+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:17:20.428398+0000","last_peered":"2024-01-24T14:17:20.428398+0000","last_clean":"2024-01-24T14:17:20.428398+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:17:20.428398+0000","last_undegraded":"2024-01-24T14:17:20.428398+0000","last_fullsized":"2024-01-24T14:17:20.428398+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.441037+0000","last_change":"2024-01-24T14:17:20.437167+0000","last_active":"2024-01-24T14:17:20.441037+0000","last_peered":"2024-01-24T14:17:20.441037+0000","last_clean":"2024-01-24T14:17:20.441037+0000","last_became_active":"2024-01-24T14:17:20.436887+0000","last_became_peered":"2024-01-24T14:17:20.436887+0000","last_unstale":"2024-01-24T14:17:20.441037+0000","last_undegraded":"2024-01-24T14:17:20.441037+0000","last_fullsized":"2024-01-24T14:17:20.441037+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423534+0000","last_change":"2024-01-24T14:17:20.023079+0000","last_active":"2024-01-24T14:17:20.423534+0000","last_peered":"2024-01-24T14:17:20.423534+0000","last_clean":"2024-01-24T14:17:20.423534+0000","last_became_active":"2024-01-24T14:17:20.022752+0000","last_became_peered":"2024-01-24T14:17:20.022752+0000","last_unstale":"2024-01-24T14:17:20.423534+0000","last_undegraded":"2024-01-24T14:17:20.423534+0000","last_fullsized":"2024-01-24T14:17:20.423534+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.435186+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:17:20.435186+0000","last_peered":"2024-01-24T14:17:20.435186+0000","last_clean":"2024-01-24T14:17:20.435186+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:17:20.435186+0000","last_undegraded":"2024-01-24T14:17:20.435186+0000","last_fullsized":"2024-01-24T14:17:20.435186+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448135+0000","last_change":"2024-01-24T14:17:20.439321+0000","last_active":"2024-01-24T14:17:20.448135+0000","last_peered":"2024-01-24T14:17:20.448135+0000","last_clean":"2024-01-24T14:17:20.448135+0000","last_became_active":"2024-01-24T14:17:20.439019+0000","last_became_peered":"2024-01-24T14:17:20.439019+0000","last_unstale":"2024-01-24T14:17:20.448135+0000","last_undegraded":"2024-01-24T14:17:20.448135+0000","last_fullsized":"2024-01-24T14:17:20.448135+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.424818+0000","last_change":"2024-01-24T14:17:20.018330+0000","last_active":"2024-01-24T14:17:20.424818+0000","last_peered":"2024-01-24T14:17:20.424818+0000","last_clean":"2024-01-24T14:17:20.424818+0000","last_became_active":"2024-01-24T14:17:20.017816+0000","last_became_peered":"2024-01-24T14:17:20.017816+0000","last_unstale":"2024-01-24T14:17:20.424818+0000","last_undegraded":"2024-01-24T14:17:20.424818+0000","last_fullsized":"2024-01-24T14:17:20.424818+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.432947+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:17:20.432947+0000","last_peered":"2024-01-24T14:17:20.432947+0000","last_clean":"2024-01-24T14:17:20.432947+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:17:20.432947+0000","last_undegraded":"2024-01-24T14:17:20.432947+0000","last_fullsized":"2024-01-24T14:17:20.432947+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575076,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303068,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068772,"statfs":{"total":96636764160,"available":96326422528,"internally_reserved":0,"allocated":1921024,"data_stored":1076869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.26000000000000001},"max":{"1min":0.76300000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.82499999999999996,"15min":0.83399999999999996},"last":0.60899999999999999}]},{"osd":1,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77000000000000002,"5min":0.85099999999999998,"15min":0.85599999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.57399999999999995,"15min":0.58099999999999996},"min":{"1min":0.376,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.753,"5min":0.80600000000000005,"15min":0.80900000000000005},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72499999999999998,"5min":0.872,"15min":0.872},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81499999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.67200000000000004}]},{"osd":3,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60399999999999998,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77300000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.35499999999999998},"max":{"1min":0.67600000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47999999999999998,"15min":0.48499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48199999999999998,"15min":0.48899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.71199999999999997,"15min":0.745},"last":0.627}]},{"osd":5,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.502,"15min":0.496},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.621,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.33100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.749,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73199999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.39700000000000002,"5min":0.36299999999999999,"15min":0.36099999999999999},"max":{"1min":0.77500000000000002,"5min":0.80700000000000005,"15min":0.83399999999999996},"last":0.77000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575077,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303060,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068780,"statfs":{"total":96636764160,"available":96326430720,"internally_reserved":0,"allocated":1912832,"data_stored":1076813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.59399999999999997},"min":{"1min":0.38900000000000001,"5min":0.32700000000000001,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.752,"15min":0.85699999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.58999999999999997},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.29999999999999999},"max":{"1min":0.75700000000000001,"5min":0.82799999999999996,"15min":0.84899999999999998},"last":0.47599999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56599999999999995,"15min":0.58799999999999997},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71299999999999997,"5min":0.78700000000000003,"15min":0.88},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.40600000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76300000000000001,"5min":0.78700000000000003,"15min":1.032},"last":0.38100000000000001}]},{"osd":2,"last update":"Wed Jan 24 14:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.60499999999999998},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.88500000000000001},"last":0.503},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58599999999999997,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76700000000000002,"5min":0.871,"15min":0.871},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed Jan 24 14:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59999999999999998,"15min":0.61699999999999999},"min":{"1min":0.45800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.89200000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.60099999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78700000000000003,"5min":0.81499999999999995,"15min":1.2430000000000001},"last":0.51700000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48399999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.28899999999999998},"max":{"1min":0.73899999999999999,"5min":0.83899999999999997,"15min":1.165},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.505,"15min":0.51200000000000001},"min":{"1min":0.34899999999999998,"5min":0.32900000000000001,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.74399999999999999,"15min":0.82299999999999995},"last":0.627}]},{"osd":5,"last update":"Wed Jan 24 14:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.80800000000000005,"5min":0.81100000000000005,"15min":0.86699999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.34300000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68700000000000006,"5min":0.80200000000000005,"15min":0.85699999999999998},"last":0.55100000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.38200000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":1.2070000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.45800000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.89600000000000002},"last":0.60499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575076,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307148,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064692,"statfs":{"total":96636764160,"available":96322244608,"internally_reserved":0,"allocated":1904640,"data_stored":1076653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39100000000000001,"15min":0.441},"min":{"1min":0.27900000000000003,"5min":0.249,"15min":0.23400000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":15.298999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.42399999999999999,"15min":0.46899999999999997},"min":{"1min":0.311,"5min":0.28299999999999997,"15min":0.254},"max":{"1min":0.54400000000000004,"5min":0.67500000000000004,"15min":14.654},"last":0.42299999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.33800000000000002,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.44700000000000001},"min":{"1min":0.316,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":0.56399999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.34999999999999998}]},{"osd":3,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"min":{"1min":0.35199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72799999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47999999999999998,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63400000000000001,"5min":1.651,"15min":1.651},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64100000000000001,"15min":0.66400000000000003},"min":{"1min":0.45400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.47499999999999998,"5min":0.46500000000000002,"15min":0.36499999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.218},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69499999999999995,"15min":0.83599999999999997},"min":{"1min":0.51900000000000002,"5min":0.46600000000000003,"15min":0.45000000000000001},"max":{"1min":1.159,"5min":1.159,"15min":38.808},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.81000000000000005},"min":{"1min":0.47499999999999998,"5min":0.46800000000000003,"15min":0.40500000000000003},"max":{"1min":1.046,"5min":1.046,"15min":38.823},"last":0.55900000000000005}]},{"osd":6,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67700000000000005,"15min":0.68400000000000005},"min":{"1min":0.51700000000000002,"5min":0.48799999999999999,"15min":0.44500000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69899999999999995,"15min":0.71999999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.40000000000000002},"max":{"1min":0.995,"5min":1.1439999999999999,"15min":1.157},"last":0.55000000000000004}]},{"osd":7,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68300000000000005,"15min":0.69499999999999995},"min":{"1min":0.55200000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72399999999999998,"15min":0.747},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0549999999999999,"5min":1.103,"15min":1.131},"last":0.58499999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542374,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307144,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064696,"statfs":{"total":96636764160,"available":96322248704,"internally_reserved":0,"allocated":1900544,"data_stored":1076030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.30099999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.55200000000000005,"5min":0.60899999999999999,"15min":0.69399999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.32600000000000001,"5min":0.28699999999999998,"15min":0.223},"max":{"1min":0.55300000000000005,"5min":0.60099999999999998,"15min":0.71699999999999997},"last":0.29099999999999998}]},{"osd":2,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.34399999999999997,"5min":0.29699999999999999,"15min":0.28999999999999998},"max":{"1min":0.61699999999999999,"5min":0.66100000000000003,"15min":0.66700000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.46500000000000002,"15min":0.46200000000000002},"min":{"1min":0.36699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.79400000000000004},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.436},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.25900000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":1.482},"last":0.376},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.498,"15min":0.502},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69299999999999995,"5min":0.82499999999999996,"15min":0.872},"last":0.32200000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64100000000000001,"15min":0.65000000000000002},"min":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83799999999999997,"5min":0.91300000000000003,"15min":0.95899999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.438,"15min":0.40000000000000002},"max":{"1min":0.82299999999999995,"5min":0.90000000000000002,"15min":1.5669999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.51600000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":1.655},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.53300000000000003,"5min":0.46300000000000002,"15min":0.42999999999999999},"max":{"1min":0.878,"5min":1.234,"15min":1.234},"last":0.53100000000000003}]},{"osd":6,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69299999999999995,"15min":0.69199999999999995},"min":{"1min":0.56499999999999995,"5min":0.47999999999999998,"15min":0.45500000000000002},"max":{"1min":0.82999999999999996,"5min":1.196,"15min":1.196},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83299999999999996,"5min":0.96499999999999997,"15min":1.117},"last":0.54100000000000004}]},{"osd":7,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.55700000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88700000000000001,"5min":0.93999999999999995,"15min":0.98199999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88200000000000001,"5min":1.2110000000000001,"15min":1.6160000000000001},"last":0.46100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575076,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307148,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064692,"statfs":{"total":96636764160,"available":96322244608,"internally_reserved":0,"allocated":1904640,"data_stored":1076133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.31,"5min":0.26700000000000002,"15min":0.24199999999999999},"max":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.65800000000000003},"last":0.31},{"interface":"front","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.435},"min":{"1min":0.35899999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63500000000000001,"5min":0.64500000000000002,"15min":0.65900000000000003},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.432,"15min":0.41799999999999998},"min":{"1min":0.33700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.433,"5min":0.44900000000000001,"15min":0.45900000000000002},"min":{"1min":0.34599999999999997,"5min":0.30399999999999999,"15min":0.29699999999999999},"max":{"1min":0.59899999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.44600000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44800000000000001,"15min":0.45200000000000001},"min":{"1min":0.36899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.52900000000000003,"5min":0.64300000000000002,"15min":0.71699999999999997},"last":0.379},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.47499999999999998,"15min":0.47299999999999998},"min":{"1min":0.38800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.72799999999999998},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63600000000000001,"15min":0.64100000000000001},"min":{"1min":0.45800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92400000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.47899999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.96599999999999997,"15min":1.042},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Jan 24 14:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66200000000000003,"15min":0.67700000000000005},"min":{"1min":0.47699999999999998,"5min":0.46000000000000002,"15min":0.42599999999999999},"max":{"1min":0.95799999999999996,"5min":1.036,"15min":1.0469999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66600000000000004,"15min":0.66900000000000004},"min":{"1min":0.50700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91200000000000003,"5min":1.107,"15min":1.107},"last":0.72199999999999998}]},{"osd":6,"last update":"Wed Jan 24 14:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67900000000000005,"15min":0.69399999999999995},"min":{"1min":0.53400000000000003,"5min":0.48699999999999999,"15min":0.45600000000000002},"max":{"1min":0.98299999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.51800000000000002,"5min":0.46800000000000003,"15min":0.45800000000000002},"max":{"1min":0.90200000000000002,"5min":1.141,"15min":1.141},"last":0.83299999999999996}]},{"osd":7,"last update":"Wed Jan 24 14:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69699999999999995,"15min":0.71399999999999997},"min":{"1min":0.53500000000000003,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":1.0409999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71599999999999997,"15min":0.72499999999999998},"min":{"1min":0.56100000000000005,"5min":0.497,"15min":0.49199999999999999},"max":{"1min":0.93700000000000006,"5min":0.97999999999999998,"15min":1.0860000000000001},"last":0.86299999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575076,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303052,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068788,"statfs":{"total":96636764160,"available":96326438912,"internally_reserved":0,"allocated":1904640,"data_stored":1076189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40100000000000002,"15min":0.41699999999999998},"min":{"1min":0.28000000000000003,"5min":0.23400000000000001,"15min":0.22900000000000001},"max":{"1min":0.499,"5min":2.1200000000000001,"15min":7.593},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.433,"15min":0.443},"min":{"1min":0.28199999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52300000000000002,"5min":2.4820000000000002,"15min":8.4239999999999995},"last":0.39200000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41199999999999998,"15min":0.40999999999999998},"min":{"1min":0.28999999999999998,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.45000000000000001,"15min":0.45300000000000001},"min":{"1min":0.33900000000000002,"5min":0.32800000000000001,"15min":0.221},"max":{"1min":0.627,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.34300000000000003}]},{"osd":2,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.76300000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.38500000000000001,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.70699999999999996,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94299999999999995,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.437},"max":{"1min":0.84199999999999997,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.61899999999999999}]},{"osd":5,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.68799999999999994},"min":{"1min":0.47699999999999998,"5min":0.42799999999999999,"15min":0.39900000000000002},"max":{"1min":0.90400000000000003,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.41899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1,"5min":1,"15min":1.0429999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68300000000000005,"15min":0.68999999999999995},"min":{"1min":0.51300000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0409999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":0.99299999999999999,"5min":2.6110000000000002,"15min":3.1800000000000002},"last":0.495}]},{"osd":7,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":2.4220000000000002,"15min":3.0059999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.73599999999999999},"min":{"1min":0.504,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0640000000000001,"5min":2.556,"15min":2.9929999999999999},"last":0.51700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575077,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303056,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068784,"statfs":{"total":96636764160,"available":96326434816,"internally_reserved":0,"allocated":1908736,"data_stored":1077545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.58499999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73199999999999998,"5min":0.83199999999999996,"15min":0.88600000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.59799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.40699999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58299999999999996,"15min":0.59899999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66600000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57699999999999996,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.36299999999999999},"max":{"1min":0.71799999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.45000000000000001}]},{"osd":2,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.84199999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59099999999999997,"15min":0.60599999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.57999999999999996}]},{"osd":3,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.80600000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60299999999999998,"15min":0.61299999999999999},"min":{"1min":0.42299999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76600000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.49199999999999999}]},{"osd":5,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.27300000000000002},"max":{"1min":0.64800000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.495},"min":{"1min":0.32400000000000001,"5min":0.30499999999999999,"15min":0.29199999999999998},"max":{"1min":0.67100000000000004,"5min":0.71299999999999997,"15min":0.72899999999999998},"last":0.621}]},{"osd":6,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50800000000000001,"15min":0.49299999999999999},"min":{"1min":0.379,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64400000000000002,"5min":0.79700000000000004,"15min":0.80700000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.36799999999999999,"5min":0.35999999999999999,"15min":0.33200000000000002},"max":{"1min":0.77800000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.63500000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.40699999999999997,"5min":0.33800000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.74299999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575077,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303056,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068784,"statfs":{"total":96636764160,"available":96326434816,"internally_reserved":0,"allocated":1908736,"data_stored":1076211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53400000000000003,"15min":0.56699999999999995},"min":{"1min":0.36499999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":1.0800000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56999999999999995,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.309,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.54700000000000004}]},{"osd":1,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.57699999999999996},"min":{"1min":0.41299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.84099999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.59499999999999997},"min":{"1min":0.44500000000000001,"5min":0.35199999999999998,"15min":0.29799999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":1.1639999999999999},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.437,"5min":0.39100000000000001,"15min":0.33900000000000002},"max":{"1min":0.70799999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.58899999999999997},"min":{"1min":0.38200000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83699999999999997},"last":0.626}]},{"osd":3,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":0.60599999999999998},"min":{"1min":0.45500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":1.139},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.374},"max":{"1min":0.69999999999999996,"5min":0.80800000000000005,"15min":0.85399999999999998},"last":0.59499999999999997}]},{"osd":4,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.35799999999999998,"5min":0.33900000000000002,"15min":0.30299999999999999},"max":{"1min":0.71399999999999997,"5min":0.76800000000000002,"15min":0.89400000000000002},"last":0.505},{"interface":"front","average":{"1min":0.503,"5min":0.496,"15min":0.501},"min":{"1min":0.34300000000000003,"5min":0.32200000000000001,"15min":0.30399999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.90000000000000002},"last":0.57399999999999995}]},{"osd":6,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.253},"max":{"1min":0.72199999999999998,"5min":0.79500000000000004,"15min":1.1839999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.32000000000000001},"max":{"1min":0.747,"5min":0.80700000000000005,"15min":0.92800000000000005},"last":0.63800000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52400000000000002,"15min":0.53400000000000003},"min":{"1min":0.371,"5min":0.34200000000000003,"15min":0.33100000000000002},"max":{"1min":0.746,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.76800000000000002,"5min":0.83999999999999997,"15min":1.2030000000000001},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:17:24.534 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T14:17:24.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.699+0000 7f6749b02700 1 Processor -- start 2024-01-24T14:17:24.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.699+0000 7f6749b02700 1 -- start start 2024-01-24T14:17:24.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.699+0000 7f6749b02700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6744134cc0 0x7f67440c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:24.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.699+0000 7f6749b02700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f67441350a0 0x7f6744152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:24.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.699+0000 7f6749b02700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6744068360 con 0x7f67440664f0 2024-01-24T14:17:24.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.699+0000 7f6749b02700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6744153360 con 0x7f67441350a0 2024-01-24T14:17:24.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.699+0000 7f6749b02700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67440c3820 con 0x7f6744134cc0 2024-01-24T14:17:24.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f6742ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f67441350a0 0x7f6744152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:24.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f6742ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f67441350a0 0x7f6744152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56414/0 (socket says 172.21.15.29:56414) 2024-01-24T14:17:24.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f6742ffd700 1 -- 172.21.15.29:0/876969757 learned_addr learned my addr 172.21.15.29:0/876969757 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:24.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f6743fff700 1 --1- 172.21.15.29:0/876969757 >> v1:172.21.15.29:6789/0 conn(0x7f67440664f0 0x7f6744155a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41666/0 (socket says 172.21.15.29:41666) 2024-01-24T14:17:24.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f67437fe700 1 --2- 172.21.15.29:0/876969757 >> v2:172.21.15.163:3300/0 conn(0x7f6744134cc0 0x7f67440c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f6742ffd700 1 -- 172.21.15.29:0/876969757 >> v2:172.21.15.163:3300/0 conn(0x7f6744134cc0 msgr2=0x7f67440c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f6742ffd700 1 --2- 172.21.15.29:0/876969757 >> v2:172.21.15.163:3300/0 conn(0x7f6744134cc0 0x7f67440c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f6742ffd700 1 -- 172.21.15.29:0/876969757 >> v1:172.21.15.29:6789/0 conn(0x7f67440664f0 legacy=0x7f6744155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f67427fc700 1 -- 172.21.15.29:0/876969757 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 682860014 0 0) 0x7f6744068360 con 0x7f67440664f0 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f6742ffd700 1 -- 172.21.15.29:0/876969757 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6744067a30 con 0x7f67441350a0 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f67437fe700 1 --2- 172.21.15.29:0/876969757 >> v2:172.21.15.163:3300/0 conn(0x7f6744134cc0 0x7f67440c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f6742ffd700 1 --2- 172.21.15.29:0/876969757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67441350a0 0x7f6744152e70 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f67440aee60 tx=0x7f67380090f0).ready entity=mon.0 client_cookie=e248d1e62c7918bb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f67427fc700 1 -- 172.21.15.29:0/876969757 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673800b600 con 0x7f67441350a0 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f67427fc700 1 -- 172.21.15.29:0/876969757 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673800b760 con 0x7f67441350a0 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.700+0000 7f67427fc700 1 -- 172.21.15.29:0/876969757 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673800bae0 con 0x7f67441350a0 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.701+0000 7f6749b02700 1 -- 172.21.15.29:0/876969757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67441350a0 msgr2=0x7f6744152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.701+0000 7f6749b02700 1 --2- 172.21.15.29:0/876969757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67441350a0 0x7f6744152e70 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f67440aee60 tx=0x7f67380090f0).stop 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.701+0000 7f6749b02700 1 -- 172.21.15.29:0/876969757 shutdown_connections 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.701+0000 7f6749b02700 1 --2- 172.21.15.29:0/876969757 >> v2:172.21.15.163:3300/0 conn(0x7f6744134cc0 0x7f67440c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.701+0000 7f6749b02700 1 --2- 172.21.15.29:0/876969757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67441350a0 0x7f6744152e70 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.701+0000 7f6749b02700 1 -- 172.21.15.29:0/876969757 >> 172.21.15.29:0/876969757 conn(0x7f67440c8bb0 msgr2=0x7f67440c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:24.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.701+0000 7f6749b02700 1 -- 172.21.15.29:0/876969757 shutdown_connections 2024-01-24T14:17:24.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.701+0000 7f6749b02700 1 -- 172.21.15.29:0/876969757 wait complete. 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.702+0000 7f6749b02700 1 Processor -- start 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.703+0000 7f6749b02700 1 -- start start 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.703+0000 7f6749b02700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67440664f0 0x7f67441573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.703+0000 7f6749b02700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6744134cc0 0x7f674414a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.703+0000 7f6749b02700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67441350a0 0x7f6744155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.703+0000 7f6749b02700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6744162420 con 0x7f67441350a0 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.703+0000 7f6749b02700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6744156290 con 0x7f67440664f0 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.703+0000 7f6749b02700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6744156570 con 0x7f6744134cc0 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.703+0000 7f6743fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67441350a0 0x7f6744155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.703+0000 7f6743fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67441350a0 0x7f6744155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56424/0 (socket says 172.21.15.29:56424) 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f6743fff700 1 -- 172.21.15.29:0/2705084368 learned_addr learned my addr 172.21.15.29:0/2705084368 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f67437fe700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67440664f0 0x7f67441573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f6742ffd700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6744134cc0 0x7f674414a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:24.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f6743fff700 1 -- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6744134cc0 msgr2=0x7f674414a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f6743fff700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6744134cc0 0x7f674414a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f6743fff700 1 -- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67440664f0 msgr2=0x7f67441573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f6743fff700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67440664f0 0x7f67441573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f6743fff700 1 -- 172.21.15.29:0/2705084368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6738009370 con 0x7f67441350a0 2024-01-24T14:17:24.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f6743fff700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67441350a0 0x7f6744155990 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f673c00c810 tx=0x7f673c007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:24.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f672b7fe700 1 -- 172.21.15.29:0/2705084368 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c005c70 con 0x7f67441350a0 2024-01-24T14:17:24.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f672b7fe700 1 -- 172.21.15.29:0/2705084368 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c005df0 con 0x7f67441350a0 2024-01-24T14:17:24.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f6749b02700 1 -- 172.21.15.29:0/2705084368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6744156850 con 0x7f67441350a0 2024-01-24T14:17:24.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f672b7fe700 1 -- 172.21.15.29:0/2705084368 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c007360 con 0x7f67441350a0 2024-01-24T14:17:24.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.704+0000 7f6749b02700 1 -- 172.21.15.29:0/2705084368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674414aaf0 con 0x7f67441350a0 2024-01-24T14:17:24.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.706+0000 7f6749b02700 1 -- 172.21.15.29:0/2705084368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67441569e0 con 0x7f67441350a0 2024-01-24T14:17:24.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.706+0000 7f672b7fe700 1 -- 172.21.15.29:0/2705084368 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f673c007750 con 0x7f67441350a0 2024-01-24T14:17:24.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.706+0000 7f672b7fe700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f240 0x7f6730061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:24.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.706+0000 7f672b7fe700 1 -- 172.21.15.29:0/2705084368 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f673c07a860 con 0x7f67441350a0 2024-01-24T14:17:24.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.706+0000 7f67437fe700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f240 0x7f6730061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:24.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.707+0000 7f67437fe700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f240 0x7f6730061700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f6734006fd0 tx=0x7f6734008040).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:24.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.711+0000 7f672b7fe700 1 -- 172.21.15.29:0/2705084368 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f673c04b5b0 con 0x7f67441350a0 2024-01-24T14:17:24.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.888+0000 7f6749b02700 1 -- 172.21.15.29:0/2705084368 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67440475e0 con 0x7f673005f240 2024-01-24T14:17:24.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.899+0000 7f672b7fe700 1 -- 172.21.15.29:0/2705084368 <== mgr.6851 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+362242 (secure 0 0 0) 0x7f67440475e0 con 0x7f673005f240 2024-01-24T14:17:24.904 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.903+0000 7f6749b02700 1 -- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f240 msgr2=0x7f6730061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.903+0000 7f6749b02700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f240 0x7f6730061700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f6734006fd0 tx=0x7f6734008040).stop 2024-01-24T14:17:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.903+0000 7f6749b02700 1 -- 172.21.15.29:0/2705084368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67441350a0 msgr2=0x7f6744155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.903+0000 7f6749b02700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67441350a0 0x7f6744155990 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f673c00c810 tx=0x7f673c007f10).stop 2024-01-24T14:17:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.903+0000 7f6749b02700 1 -- 172.21.15.29:0/2705084368 shutdown_connections 2024-01-24T14:17:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.903+0000 7f6749b02700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f673005f240 0x7f6730061700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.903+0000 7f6749b02700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67441350a0 0x7f6744155990 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.903+0000 7f6749b02700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67440664f0 0x7f67441573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.903+0000 7f6749b02700 1 --2- 172.21.15.29:0/2705084368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6744134cc0 0x7f674414a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.903+0000 7f6749b02700 1 -- 172.21.15.29:0/2705084368 >> 172.21.15.29:0/2705084368 conn(0x7f67440c8bb0 msgr2=0x7f67440c7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:24.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.903+0000 7f6749b02700 1 -- 172.21.15.29:0/2705084368 shutdown_connections 2024-01-24T14:17:24.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:24.903+0000 7f6749b02700 1 -- 172.21.15.29:0/2705084368 wait complete. 2024-01-24T14:17:24.908 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T14:17:24.917 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T14:17:23.231362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2436732,"kb_used_data":14908,"kb_used_omap":0,"kb_used_meta":2421760,"kb_avail":752537988,"statfs":{"total":773094113280,"available":770598899712,"internally_reserved":0,"allocated":15265792,"data_stored":8612443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2479882240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20936,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-71,"num_write_kb":-47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.016822"},"pg_stats":[{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447956+0000","last_change":"2024-01-24T14:17:20.444978+0000","last_active":"2024-01-24T14:17:20.447956+0000","last_peered":"2024-01-24T14:17:20.447956+0000","last_clean":"2024-01-24T14:17:20.447956+0000","last_became_active":"2024-01-24T14:17:20.444519+0000","last_became_peered":"2024-01-24T14:17:20.444519+0000","last_unstale":"2024-01-24T14:17:20.447956+0000","last_undegraded":"2024-01-24T14:17:20.447956+0000","last_fullsized":"2024-01-24T14:17:20.447956+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.3f","version":"152'3","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.446762+0000","last_change":"2024-01-24T14:17:20.024233+0000","last_active":"2024-01-24T14:17:20.446762+0000","last_peered":"2024-01-24T14:17:20.446762+0000","last_clean":"2024-01-24T14:17:20.446762+0000","last_became_active":"2024-01-24T14:17:20.023950+0000","last_became_peered":"2024-01-24T14:17:20.023950+0000","last_unstale":"2024-01-24T14:17:20.446762+0000","last_undegraded":"2024-01-24T14:17:20.446762+0000","last_fullsized":"2024-01-24T14:17:20.446762+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448650+0000","last_change":"2024-01-24T14:17:20.442921+0000","last_active":"2024-01-24T14:17:20.448650+0000","last_peered":"2024-01-24T14:17:20.448650+0000","last_clean":"2024-01-24T14:17:20.448650+0000","last_became_active":"2024-01-24T14:17:20.439984+0000","last_became_peered":"2024-01-24T14:17:20.439984+0000","last_unstale":"2024-01-24T14:17:20.448650+0000","last_undegraded":"2024-01-24T14:17:20.448650+0000","last_fullsized":"2024-01-24T14:17:20.448650+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423287+0000","last_change":"2024-01-24T14:17:20.022892+0000","last_active":"2024-01-24T14:17:20.423287+0000","last_peered":"2024-01-24T14:17:20.423287+0000","last_clean":"2024-01-24T14:17:20.423287+0000","last_became_active":"2024-01-24T14:17:20.022218+0000","last_became_peered":"2024-01-24T14:17:20.022218+0000","last_unstale":"2024-01-24T14:17:20.423287+0000","last_undegraded":"2024-01-24T14:17:20.423287+0000","last_fullsized":"2024-01-24T14:17:20.423287+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.452232+0000","last_change":"2024-01-24T14:17:20.450262+0000","last_active":"2024-01-24T14:17:20.452232+0000","last_peered":"2024-01-24T14:17:20.452232+0000","last_clean":"2024-01-24T14:17:20.452232+0000","last_became_active":"2024-01-24T14:17:20.449899+0000","last_became_peered":"2024-01-24T14:17:20.449899+0000","last_unstale":"2024-01-24T14:17:20.452232+0000","last_undegraded":"2024-01-24T14:17:20.452232+0000","last_fullsized":"2024-01-24T14:17:20.452232+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423290+0000","last_change":"2024-01-24T14:17:20.021408+0000","last_active":"2024-01-24T14:17:20.423290+0000","last_peered":"2024-01-24T14:17:20.423290+0000","last_clean":"2024-01-24T14:17:20.423290+0000","last_became_active":"2024-01-24T14:17:20.021089+0000","last_became_peered":"2024-01-24T14:17:20.021089+0000","last_unstale":"2024-01-24T14:17:20.423290+0000","last_undegraded":"2024-01-24T14:17:20.423290+0000","last_fullsized":"2024-01-24T14:17:20.423290+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.445387+0000","last_change":"2024-01-24T14:17:20.442458+0000","last_active":"2024-01-24T14:17:20.445387+0000","last_peered":"2024-01-24T14:17:20.445387+0000","last_clean":"2024-01-24T14:17:20.445387+0000","last_became_active":"2024-01-24T14:17:20.442169+0000","last_became_peered":"2024-01-24T14:17:20.442169+0000","last_unstale":"2024-01-24T14:17:20.445387+0000","last_undegraded":"2024-01-24T14:17:20.445387+0000","last_fullsized":"2024-01-24T14:17:20.445387+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.3c","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449478+0000","last_change":"2024-01-24T14:17:20.026370+0000","last_active":"2024-01-24T14:17:20.449478+0000","last_peered":"2024-01-24T14:17:20.449478+0000","last_clean":"2024-01-24T14:17:20.449478+0000","last_became_active":"2024-01-24T14:17:20.026096+0000","last_became_peered":"2024-01-24T14:17:20.026096+0000","last_unstale":"2024-01-24T14:17:20.449478+0000","last_undegraded":"2024-01-24T14:17:20.449478+0000","last_fullsized":"2024-01-24T14:17:20.449478+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.439849+0000","last_change":"2024-01-24T14:17:20.430898+0000","last_active":"2024-01-24T14:17:20.439849+0000","last_peered":"2024-01-24T14:17:20.439849+0000","last_clean":"2024-01-24T14:17:20.439849+0000","last_became_active":"2024-01-24T14:17:20.430614+0000","last_became_peered":"2024-01-24T14:17:20.430614+0000","last_unstale":"2024-01-24T14:17:20.439849+0000","last_undegraded":"2024-01-24T14:17:20.439849+0000","last_fullsized":"2024-01-24T14:17:20.439849+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3b","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.436687+0000","last_change":"2024-01-24T14:17:20.021474+0000","last_active":"2024-01-24T14:17:20.436687+0000","last_peered":"2024-01-24T14:17:20.436687+0000","last_clean":"2024-01-24T14:17:20.436687+0000","last_became_active":"2024-01-24T14:17:20.021202+0000","last_became_peered":"2024-01-24T14:17:20.021202+0000","last_unstale":"2024-01-24T14:17:20.436687+0000","last_undegraded":"2024-01-24T14:17:20.436687+0000","last_fullsized":"2024-01-24T14:17:20.436687+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451840+0000","last_change":"2024-01-24T14:17:20.445712+0000","last_active":"2024-01-24T14:17:20.451840+0000","last_peered":"2024-01-24T14:17:20.451840+0000","last_clean":"2024-01-24T14:17:20.451840+0000","last_became_active":"2024-01-24T14:17:20.444488+0000","last_became_peered":"2024-01-24T14:17:20.444488+0000","last_unstale":"2024-01-24T14:17:20.451840+0000","last_undegraded":"2024-01-24T14:17:20.451840+0000","last_fullsized":"2024-01-24T14:17:20.451840+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3a","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.440681+0000","last_change":"2024-01-24T14:17:20.035007+0000","last_active":"2024-01-24T14:17:20.440681+0000","last_peered":"2024-01-24T14:17:20.440681+0000","last_clean":"2024-01-24T14:17:20.440681+0000","last_became_active":"2024-01-24T14:17:20.034701+0000","last_became_peered":"2024-01-24T14:17:20.034701+0000","last_unstale":"2024-01-24T14:17:20.440681+0000","last_undegraded":"2024-01-24T14:17:20.440681+0000","last_fullsized":"2024-01-24T14:17:20.440681+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449011+0000","last_change":"2024-01-24T14:17:20.444292+0000","last_active":"2024-01-24T14:17:20.449011+0000","last_peered":"2024-01-24T14:17:20.449011+0000","last_clean":"2024-01-24T14:17:20.449011+0000","last_became_active":"2024-01-24T14:17:20.444013+0000","last_became_peered":"2024-01-24T14:17:20.444013+0000","last_unstale":"2024-01-24T14:17:20.449011+0000","last_undegraded":"2024-01-24T14:17:20.449011+0000","last_fullsized":"2024-01-24T14:17:20.449011+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.430390+0000","last_change":"2024-01-24T14:17:20.034106+0000","last_active":"2024-01-24T14:17:20.430390+0000","last_peered":"2024-01-24T14:17:20.430390+0000","last_clean":"2024-01-24T14:17:20.430390+0000","last_became_active":"2024-01-24T14:17:20.023239+0000","last_became_peered":"2024-01-24T14:17:20.023239+0000","last_unstale":"2024-01-24T14:17:20.430390+0000","last_undegraded":"2024-01-24T14:17:20.430390+0000","last_fullsized":"2024-01-24T14:17:20.430390+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.442509+0000","last_change":"2024-01-24T14:17:20.438409+0000","last_active":"2024-01-24T14:17:20.442509+0000","last_peered":"2024-01-24T14:17:20.442509+0000","last_clean":"2024-01-24T14:17:20.442509+0000","last_became_active":"2024-01-24T14:17:20.438111+0000","last_became_peered":"2024-01-24T14:17:20.438111+0000","last_unstale":"2024-01-24T14:17:20.442509+0000","last_undegraded":"2024-01-24T14:17:20.442509+0000","last_fullsized":"2024-01-24T14:17:20.442509+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.424880+0000","last_change":"2024-01-24T14:17:20.022961+0000","last_active":"2024-01-24T14:17:20.424880+0000","last_peered":"2024-01-24T14:17:20.424880+0000","last_clean":"2024-01-24T14:17:20.424880+0000","last_became_active":"2024-01-24T14:17:20.022632+0000","last_became_peered":"2024-01-24T14:17:20.022632+0000","last_unstale":"2024-01-24T14:17:20.424880+0000","last_undegraded":"2024-01-24T14:17:20.424880+0000","last_fullsized":"2024-01-24T14:17:20.424880+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447932+0000","last_change":"2024-01-24T14:17:20.444806+0000","last_active":"2024-01-24T14:17:20.447932+0000","last_peered":"2024-01-24T14:17:20.447932+0000","last_clean":"2024-01-24T14:17:20.447932+0000","last_became_active":"2024-01-24T14:17:20.444167+0000","last_became_peered":"2024-01-24T14:17:20.444167+0000","last_unstale":"2024-01-24T14:17:20.447932+0000","last_undegraded":"2024-01-24T14:17:20.447932+0000","last_fullsized":"2024-01-24T14:17:20.447932+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.429341+0000","last_change":"2024-01-24T14:17:20.030297+0000","last_active":"2024-01-24T14:17:20.429341+0000","last_peered":"2024-01-24T14:17:20.429341+0000","last_clean":"2024-01-24T14:17:20.429341+0000","last_became_active":"2024-01-24T14:17:20.030015+0000","last_became_peered":"2024-01-24T14:17:20.030015+0000","last_unstale":"2024-01-24T14:17:20.429341+0000","last_undegraded":"2024-01-24T14:17:20.429341+0000","last_fullsized":"2024-01-24T14:17:20.429341+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449450+0000","last_change":"2024-01-24T14:17:20.442943+0000","last_active":"2024-01-24T14:17:20.449450+0000","last_peered":"2024-01-24T14:17:20.449450+0000","last_clean":"2024-01-24T14:17:20.449450+0000","last_became_active":"2024-01-24T14:17:20.442070+0000","last_became_peered":"2024-01-24T14:17:20.442070+0000","last_unstale":"2024-01-24T14:17:20.449450+0000","last_undegraded":"2024-01-24T14:17:20.449450+0000","last_fullsized":"2024-01-24T14:17:20.449450+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.435235+0000","last_change":"2024-01-24T14:17:20.034877+0000","last_active":"2024-01-24T14:17:20.435235+0000","last_peered":"2024-01-24T14:17:20.435235+0000","last_clean":"2024-01-24T14:17:20.435235+0000","last_became_active":"2024-01-24T14:17:20.034540+0000","last_became_peered":"2024-01-24T14:17:20.034540+0000","last_unstale":"2024-01-24T14:17:20.435235+0000","last_undegraded":"2024-01-24T14:17:20.435235+0000","last_fullsized":"2024-01-24T14:17:20.435235+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.445476+0000","last_change":"2024-01-24T14:17:20.442359+0000","last_active":"2024-01-24T14:17:20.445476+0000","last_peered":"2024-01-24T14:17:20.445476+0000","last_clean":"2024-01-24T14:17:20.445476+0000","last_became_active":"2024-01-24T14:17:20.442098+0000","last_became_peered":"2024-01-24T14:17:20.442098+0000","last_unstale":"2024-01-24T14:17:20.445476+0000","last_undegraded":"2024-01-24T14:17:20.445476+0000","last_fullsized":"2024-01-24T14:17:20.445476+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423831+0000","last_change":"2024-01-24T14:17:20.020488+0000","last_active":"2024-01-24T14:17:20.423831+0000","last_peered":"2024-01-24T14:17:20.423831+0000","last_clean":"2024-01-24T14:17:20.423831+0000","last_became_active":"2024-01-24T14:17:20.020210+0000","last_became_peered":"2024-01-24T14:17:20.020210+0000","last_unstale":"2024-01-24T14:17:20.423831+0000","last_undegraded":"2024-01-24T14:17:20.423831+0000","last_fullsized":"2024-01-24T14:17:20.423831+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448508+0000","last_change":"2024-01-24T14:17:20.444334+0000","last_active":"2024-01-24T14:17:20.448508+0000","last_peered":"2024-01-24T14:17:20.448508+0000","last_clean":"2024-01-24T14:17:20.448508+0000","last_became_active":"2024-01-24T14:17:20.444044+0000","last_became_peered":"2024-01-24T14:17:20.444044+0000","last_unstale":"2024-01-24T14:17:20.448508+0000","last_undegraded":"2024-01-24T14:17:20.448508+0000","last_fullsized":"2024-01-24T14:17:20.448508+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.34","version":"152'4","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.454447+0000","last_change":"2024-01-24T14:17:20.035154+0000","last_active":"2024-01-24T14:17:20.454447+0000","last_peered":"2024-01-24T14:17:20.454447+0000","last_clean":"2024-01-24T14:17:20.454447+0000","last_became_active":"2024-01-24T14:17:20.034810+0000","last_became_peered":"2024-01-24T14:17:20.034810+0000","last_unstale":"2024-01-24T14:17:20.454447+0000","last_undegraded":"2024-01-24T14:17:20.454447+0000","last_fullsized":"2024-01-24T14:17:20.454447+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447900+0000","last_change":"2024-01-24T14:17:20.442809+0000","last_active":"2024-01-24T14:17:20.447900+0000","last_peered":"2024-01-24T14:17:20.447900+0000","last_clean":"2024-01-24T14:17:20.447900+0000","last_became_active":"2024-01-24T14:17:20.442478+0000","last_became_peered":"2024-01-24T14:17:20.442478+0000","last_unstale":"2024-01-24T14:17:20.447900+0000","last_undegraded":"2024-01-24T14:17:20.447900+0000","last_fullsized":"2024-01-24T14:17:20.447900+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.33","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.444609+0000","last_change":"2024-01-24T14:17:20.030771+0000","last_active":"2024-01-24T14:17:20.444609+0000","last_peered":"2024-01-24T14:17:20.444609+0000","last_clean":"2024-01-24T14:17:20.444609+0000","last_became_active":"2024-01-24T14:17:20.030457+0000","last_became_peered":"2024-01-24T14:17:20.030457+0000","last_unstale":"2024-01-24T14:17:20.444609+0000","last_undegraded":"2024-01-24T14:17:20.444609+0000","last_fullsized":"2024-01-24T14:17:20.444609+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449149+0000","last_change":"2024-01-24T14:17:20.447685+0000","last_active":"2024-01-24T14:17:20.449149+0000","last_peered":"2024-01-24T14:17:20.449149+0000","last_clean":"2024-01-24T14:17:20.449149+0000","last_became_active":"2024-01-24T14:17:20.442539+0000","last_became_peered":"2024-01-24T14:17:20.442539+0000","last_unstale":"2024-01-24T14:17:20.449149+0000","last_undegraded":"2024-01-24T14:17:20.449149+0000","last_fullsized":"2024-01-24T14:17:20.449149+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423022+0000","last_change":"2024-01-24T14:17:20.025463+0000","last_active":"2024-01-24T14:17:20.423022+0000","last_peered":"2024-01-24T14:17:20.423022+0000","last_clean":"2024-01-24T14:17:20.423022+0000","last_became_active":"2024-01-24T14:17:20.024916+0000","last_became_peered":"2024-01-24T14:17:20.024916+0000","last_unstale":"2024-01-24T14:17:20.423022+0000","last_undegraded":"2024-01-24T14:17:20.423022+0000","last_fullsized":"2024-01-24T14:17:20.423022+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.439262+0000","last_change":"2024-01-24T14:17:20.429709+0000","last_active":"2024-01-24T14:17:20.439262+0000","last_peered":"2024-01-24T14:17:20.439262+0000","last_clean":"2024-01-24T14:17:20.439262+0000","last_became_active":"2024-01-24T14:17:20.429425+0000","last_became_peered":"2024-01-24T14:17:20.429425+0000","last_unstale":"2024-01-24T14:17:20.439262+0000","last_undegraded":"2024-01-24T14:17:20.439262+0000","last_fullsized":"2024-01-24T14:17:20.439262+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.422164+0000","last_change":"2024-01-24T14:17:20.020973+0000","last_active":"2024-01-24T14:17:20.422164+0000","last_peered":"2024-01-24T14:17:20.422164+0000","last_clean":"2024-01-24T14:17:20.422164+0000","last_became_active":"2024-01-24T14:17:20.020647+0000","last_became_peered":"2024-01-24T14:17:20.020647+0000","last_unstale":"2024-01-24T14:17:20.422164+0000","last_undegraded":"2024-01-24T14:17:20.422164+0000","last_fullsized":"2024-01-24T14:17:20.422164+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448628+0000","last_change":"2024-01-24T14:17:20.439117+0000","last_active":"2024-01-24T14:17:20.448628+0000","last_peered":"2024-01-24T14:17:20.448628+0000","last_clean":"2024-01-24T14:17:20.448628+0000","last_became_active":"2024-01-24T14:17:20.438789+0000","last_became_peered":"2024-01-24T14:17:20.438789+0000","last_unstale":"2024-01-24T14:17:20.448628+0000","last_undegraded":"2024-01-24T14:17:20.448628+0000","last_fullsized":"2024-01-24T14:17:20.448628+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.30","version":"152'4","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449555+0000","last_change":"2024-01-24T14:17:20.036630+0000","last_active":"2024-01-24T14:17:20.449555+0000","last_peered":"2024-01-24T14:17:20.449555+0000","last_clean":"2024-01-24T14:17:20.449555+0000","last_became_active":"2024-01-24T14:17:20.036294+0000","last_became_peered":"2024-01-24T14:17:20.036294+0000","last_unstale":"2024-01-24T14:17:20.449555+0000","last_undegraded":"2024-01-24T14:17:20.449555+0000","last_fullsized":"2024-01-24T14:17:20.449555+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.442311+0000","last_change":"2024-01-24T14:17:20.435992+0000","last_active":"2024-01-24T14:17:20.442311+0000","last_peered":"2024-01-24T14:17:20.442311+0000","last_clean":"2024-01-24T14:17:20.442311+0000","last_became_active":"2024-01-24T14:17:20.435732+0000","last_became_peered":"2024-01-24T14:17:20.435732+0000","last_unstale":"2024-01-24T14:17:20.442311+0000","last_undegraded":"2024-01-24T14:17:20.442311+0000","last_fullsized":"2024-01-24T14:17:20.442311+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.435330+0000","last_change":"2024-01-24T14:17:20.031041+0000","last_active":"2024-01-24T14:17:20.435330+0000","last_peered":"2024-01-24T14:17:20.435330+0000","last_clean":"2024-01-24T14:17:20.435330+0000","last_became_active":"2024-01-24T14:17:20.030681+0000","last_became_peered":"2024-01-24T14:17:20.030681+0000","last_unstale":"2024-01-24T14:17:20.435330+0000","last_undegraded":"2024-01-24T14:17:20.435330+0000","last_fullsized":"2024-01-24T14:17:20.435330+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.443121+0000","last_change":"2024-01-24T14:17:20.438829+0000","last_active":"2024-01-24T14:17:20.443121+0000","last_peered":"2024-01-24T14:17:20.443121+0000","last_clean":"2024-01-24T14:17:20.443121+0000","last_became_active":"2024-01-24T14:17:20.438547+0000","last_became_peered":"2024-01-24T14:17:20.438547+0000","last_unstale":"2024-01-24T14:17:20.443121+0000","last_undegraded":"2024-01-24T14:17:20.443121+0000","last_fullsized":"2024-01-24T14:17:20.443121+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423424+0000","last_change":"2024-01-24T14:17:20.022692+0000","last_active":"2024-01-24T14:17:20.423424+0000","last_peered":"2024-01-24T14:17:20.423424+0000","last_clean":"2024-01-24T14:17:20.423424+0000","last_became_active":"2024-01-24T14:17:20.021854+0000","last_became_peered":"2024-01-24T14:17:20.021854+0000","last_unstale":"2024-01-24T14:17:20.423424+0000","last_undegraded":"2024-01-24T14:17:20.423424+0000","last_fullsized":"2024-01-24T14:17:20.423424+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.439645+0000","last_change":"2024-01-24T14:17:20.433756+0000","last_active":"2024-01-24T14:17:20.439645+0000","last_peered":"2024-01-24T14:17:20.439645+0000","last_clean":"2024-01-24T14:17:20.439645+0000","last_became_active":"2024-01-24T14:17:20.433447+0000","last_became_peered":"2024-01-24T14:17:20.433447+0000","last_unstale":"2024-01-24T14:17:20.439645+0000","last_undegraded":"2024-01-24T14:17:20.439645+0000","last_fullsized":"2024-01-24T14:17:20.439645+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449394+0000","last_change":"2024-01-24T14:17:20.020567+0000","last_active":"2024-01-24T14:17:20.449394+0000","last_peered":"2024-01-24T14:17:20.449394+0000","last_clean":"2024-01-24T14:17:20.449394+0000","last_became_active":"2024-01-24T14:17:20.020147+0000","last_became_peered":"2024-01-24T14:17:20.020147+0000","last_unstale":"2024-01-24T14:17:20.449394+0000","last_undegraded":"2024-01-24T14:17:20.449394+0000","last_fullsized":"2024-01-24T14:17:20.449394+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448483+0000","last_change":"2024-01-24T14:17:20.441062+0000","last_active":"2024-01-24T14:17:20.448483+0000","last_peered":"2024-01-24T14:17:20.448483+0000","last_clean":"2024-01-24T14:17:20.448483+0000","last_became_active":"2024-01-24T14:17:20.440697+0000","last_became_peered":"2024-01-24T14:17:20.440697+0000","last_unstale":"2024-01-24T14:17:20.448483+0000","last_undegraded":"2024-01-24T14:17:20.448483+0000","last_fullsized":"2024-01-24T14:17:20.448483+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.418915+0000","last_change":"2024-01-24T14:17:20.036805+0000","last_active":"2024-01-24T14:17:20.418915+0000","last_peered":"2024-01-24T14:17:20.418915+0000","last_clean":"2024-01-24T14:17:20.418915+0000","last_became_active":"2024-01-24T14:17:20.036507+0000","last_became_peered":"2024-01-24T14:17:20.036507+0000","last_unstale":"2024-01-24T14:17:20.418915+0000","last_undegraded":"2024-01-24T14:17:20.418915+0000","last_fullsized":"2024-01-24T14:17:20.418915+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.446534+0000","last_change":"2024-01-24T14:17:20.437696+0000","last_active":"2024-01-24T14:17:20.446534+0000","last_peered":"2024-01-24T14:17:20.446534+0000","last_clean":"2024-01-24T14:17:20.446534+0000","last_became_active":"2024-01-24T14:17:20.437442+0000","last_became_peered":"2024-01-24T14:17:20.437442+0000","last_unstale":"2024-01-24T14:17:20.446534+0000","last_undegraded":"2024-01-24T14:17:20.446534+0000","last_fullsized":"2024-01-24T14:17:20.446534+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2b","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.444221+0000","last_change":"2024-01-24T14:17:20.027646+0000","last_active":"2024-01-24T14:17:20.444221+0000","last_peered":"2024-01-24T14:17:20.444221+0000","last_clean":"2024-01-24T14:17:20.444221+0000","last_became_active":"2024-01-24T14:17:20.027185+0000","last_became_peered":"2024-01-24T14:17:20.027185+0000","last_unstale":"2024-01-24T14:17:20.444221+0000","last_undegraded":"2024-01-24T14:17:20.444221+0000","last_fullsized":"2024-01-24T14:17:20.444221+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451858+0000","last_change":"2024-01-24T14:17:20.445917+0000","last_active":"2024-01-24T14:17:20.451858+0000","last_peered":"2024-01-24T14:17:20.451858+0000","last_clean":"2024-01-24T14:17:20.451858+0000","last_became_active":"2024-01-24T14:17:20.444661+0000","last_became_peered":"2024-01-24T14:17:20.444661+0000","last_unstale":"2024-01-24T14:17:20.451858+0000","last_undegraded":"2024-01-24T14:17:20.451858+0000","last_fullsized":"2024-01-24T14:17:20.451858+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.422612+0000","last_change":"2024-01-24T14:17:20.025674+0000","last_active":"2024-01-24T14:17:20.422612+0000","last_peered":"2024-01-24T14:17:20.422612+0000","last_clean":"2024-01-24T14:17:20.422612+0000","last_became_active":"2024-01-24T14:17:20.025356+0000","last_became_peered":"2024-01-24T14:17:20.025356+0000","last_unstale":"2024-01-24T14:17:20.422612+0000","last_undegraded":"2024-01-24T14:17:20.422612+0000","last_fullsized":"2024-01-24T14:17:20.422612+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448431+0000","last_change":"2024-01-24T14:17:20.444234+0000","last_active":"2024-01-24T14:17:20.448431+0000","last_peered":"2024-01-24T14:17:20.448431+0000","last_clean":"2024-01-24T14:17:20.448431+0000","last_became_active":"2024-01-24T14:17:20.443949+0000","last_became_peered":"2024-01-24T14:17:20.443949+0000","last_unstale":"2024-01-24T14:17:20.448431+0000","last_undegraded":"2024-01-24T14:17:20.448431+0000","last_fullsized":"2024-01-24T14:17:20.448431+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.435257+0000","last_change":"2024-01-24T14:17:20.028043+0000","last_active":"2024-01-24T14:17:20.435257+0000","last_peered":"2024-01-24T14:17:20.435257+0000","last_clean":"2024-01-24T14:17:20.435257+0000","last_became_active":"2024-01-24T14:17:20.027027+0000","last_became_peered":"2024-01-24T14:17:20.027027+0000","last_unstale":"2024-01-24T14:17:20.435257+0000","last_undegraded":"2024-01-24T14:17:20.435257+0000","last_fullsized":"2024-01-24T14:17:20.435257+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.446080+0000","last_change":"2024-01-24T14:17:20.442564+0000","last_active":"2024-01-24T14:17:20.446080+0000","last_peered":"2024-01-24T14:17:20.446080+0000","last_clean":"2024-01-24T14:17:20.446080+0000","last_became_active":"2024-01-24T14:17:20.442009+0000","last_became_peered":"2024-01-24T14:17:20.442009+0000","last_unstale":"2024-01-24T14:17:20.446080+0000","last_undegraded":"2024-01-24T14:17:20.446080+0000","last_fullsized":"2024-01-24T14:17:20.446080+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.421736+0000","last_change":"2024-01-24T14:17:20.023624+0000","last_active":"2024-01-24T14:17:20.421736+0000","last_peered":"2024-01-24T14:17:20.421736+0000","last_clean":"2024-01-24T14:17:20.421736+0000","last_became_active":"2024-01-24T14:17:20.022507+0000","last_became_peered":"2024-01-24T14:17:20.022507+0000","last_unstale":"2024-01-24T14:17:20.421736+0000","last_undegraded":"2024-01-24T14:17:20.421736+0000","last_fullsized":"2024-01-24T14:17:20.421736+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.433245+0000","last_change":"2024-01-24T14:17:20.427976+0000","last_active":"2024-01-24T14:17:20.433245+0000","last_peered":"2024-01-24T14:17:20.433245+0000","last_clean":"2024-01-24T14:17:20.433245+0000","last_became_active":"2024-01-24T14:17:20.427671+0000","last_became_peered":"2024-01-24T14:17:20.427671+0000","last_unstale":"2024-01-24T14:17:20.433245+0000","last_undegraded":"2024-01-24T14:17:20.433245+0000","last_fullsized":"2024-01-24T14:17:20.433245+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.424720+0000","last_change":"2024-01-24T14:17:20.022793+0000","last_active":"2024-01-24T14:17:20.424720+0000","last_peered":"2024-01-24T14:17:20.424720+0000","last_clean":"2024-01-24T14:17:20.424720+0000","last_became_active":"2024-01-24T14:17:20.022536+0000","last_became_peered":"2024-01-24T14:17:20.022536+0000","last_unstale":"2024-01-24T14:17:20.424720+0000","last_undegraded":"2024-01-24T14:17:20.424720+0000","last_fullsized":"2024-01-24T14:17:20.424720+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.453076+0000","last_change":"2024-01-24T14:17:20.450587+0000","last_active":"2024-01-24T14:17:20.453076+0000","last_peered":"2024-01-24T14:17:20.453076+0000","last_clean":"2024-01-24T14:17:20.453076+0000","last_became_active":"2024-01-24T14:17:20.450189+0000","last_became_peered":"2024-01-24T14:17:20.450189+0000","last_unstale":"2024-01-24T14:17:20.453076+0000","last_undegraded":"2024-01-24T14:17:20.453076+0000","last_fullsized":"2024-01-24T14:17:20.453076+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.419653+0000","last_change":"2024-01-24T14:17:20.024118+0000","last_active":"2024-01-24T14:17:20.419653+0000","last_peered":"2024-01-24T14:17:20.419653+0000","last_clean":"2024-01-24T14:17:20.419653+0000","last_became_active":"2024-01-24T14:17:20.023883+0000","last_became_peered":"2024-01-24T14:17:20.023883+0000","last_unstale":"2024-01-24T14:17:20.419653+0000","last_undegraded":"2024-01-24T14:17:20.419653+0000","last_fullsized":"2024-01-24T14:17:20.419653+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.438262+0000","last_change":"2024-01-24T14:17:20.429107+0000","last_active":"2024-01-24T14:17:20.438262+0000","last_peered":"2024-01-24T14:17:20.438262+0000","last_clean":"2024-01-24T14:17:20.438262+0000","last_became_active":"2024-01-24T14:17:20.428735+0000","last_became_peered":"2024-01-24T14:17:20.428735+0000","last_unstale":"2024-01-24T14:17:20.438262+0000","last_undegraded":"2024-01-24T14:17:20.438262+0000","last_fullsized":"2024-01-24T14:17:20.438262+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.433103+0000","last_change":"2024-01-24T14:17:20.020397+0000","last_active":"2024-01-24T14:17:20.433103+0000","last_peered":"2024-01-24T14:17:20.433103+0000","last_clean":"2024-01-24T14:17:20.433103+0000","last_became_active":"2024-01-24T14:17:20.019754+0000","last_became_peered":"2024-01-24T14:17:20.019754+0000","last_unstale":"2024-01-24T14:17:20.433103+0000","last_undegraded":"2024-01-24T14:17:20.433103+0000","last_fullsized":"2024-01-24T14:17:20.433103+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447507+0000","last_change":"2024-01-24T14:17:20.436591+0000","last_active":"2024-01-24T14:17:20.447507+0000","last_peered":"2024-01-24T14:17:20.447507+0000","last_clean":"2024-01-24T14:17:20.447507+0000","last_became_active":"2024-01-24T14:17:20.436320+0000","last_became_peered":"2024-01-24T14:17:20.436320+0000","last_unstale":"2024-01-24T14:17:20.447507+0000","last_undegraded":"2024-01-24T14:17:20.447507+0000","last_fullsized":"2024-01-24T14:17:20.447507+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.431563+0000","last_change":"2024-01-24T14:17:20.036853+0000","last_active":"2024-01-24T14:17:20.431563+0000","last_peered":"2024-01-24T14:17:20.431563+0000","last_clean":"2024-01-24T14:17:20.431563+0000","last_became_active":"2024-01-24T14:17:20.036567+0000","last_became_peered":"2024-01-24T14:17:20.036567+0000","last_unstale":"2024-01-24T14:17:20.431563+0000","last_undegraded":"2024-01-24T14:17:20.431563+0000","last_fullsized":"2024-01-24T14:17:20.431563+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447330+0000","last_change":"2024-01-24T14:17:20.437674+0000","last_active":"2024-01-24T14:17:20.447330+0000","last_peered":"2024-01-24T14:17:20.447330+0000","last_clean":"2024-01-24T14:17:20.447330+0000","last_became_active":"2024-01-24T14:17:20.437276+0000","last_became_peered":"2024-01-24T14:17:20.437276+0000","last_unstale":"2024-01-24T14:17:20.447330+0000","last_undegraded":"2024-01-24T14:17:20.447330+0000","last_fullsized":"2024-01-24T14:17:20.447330+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.419037+0000","last_change":"2024-01-24T14:17:20.024288+0000","last_active":"2024-01-24T14:17:20.419037+0000","last_peered":"2024-01-24T14:17:20.419037+0000","last_clean":"2024-01-24T14:17:20.419037+0000","last_became_active":"2024-01-24T14:17:20.023983+0000","last_became_peered":"2024-01-24T14:17:20.023983+0000","last_unstale":"2024-01-24T14:17:20.419037+0000","last_undegraded":"2024-01-24T14:17:20.419037+0000","last_fullsized":"2024-01-24T14:17:20.419037+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449128+0000","last_change":"2024-01-24T14:17:20.447184+0000","last_active":"2024-01-24T14:17:20.449128+0000","last_peered":"2024-01-24T14:17:20.449128+0000","last_clean":"2024-01-24T14:17:20.449128+0000","last_became_active":"2024-01-24T14:17:20.439268+0000","last_became_peered":"2024-01-24T14:17:20.439268+0000","last_unstale":"2024-01-24T14:17:20.449128+0000","last_undegraded":"2024-01-24T14:17:20.449128+0000","last_fullsized":"2024-01-24T14:17:20.449128+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423744+0000","last_change":"2024-01-24T14:17:20.030298+0000","last_active":"2024-01-24T14:17:20.423744+0000","last_peered":"2024-01-24T14:17:20.423744+0000","last_clean":"2024-01-24T14:17:20.423744+0000","last_became_active":"2024-01-24T14:17:20.029681+0000","last_became_peered":"2024-01-24T14:17:20.029681+0000","last_unstale":"2024-01-24T14:17:20.423744+0000","last_undegraded":"2024-01-24T14:17:20.423744+0000","last_fullsized":"2024-01-24T14:17:20.423744+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.439786+0000","last_change":"2024-01-24T14:17:20.429845+0000","last_active":"2024-01-24T14:17:20.439786+0000","last_peered":"2024-01-24T14:17:20.439786+0000","last_clean":"2024-01-24T14:17:20.439786+0000","last_became_active":"2024-01-24T14:17:20.429504+0000","last_became_peered":"2024-01-24T14:17:20.429504+0000","last_unstale":"2024-01-24T14:17:20.439786+0000","last_undegraded":"2024-01-24T14:17:20.439786+0000","last_fullsized":"2024-01-24T14:17:20.439786+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.422557+0000","last_change":"2024-01-24T14:17:20.018423+0000","last_active":"2024-01-24T14:17:20.422557+0000","last_peered":"2024-01-24T14:17:20.422557+0000","last_clean":"2024-01-24T14:17:20.422557+0000","last_became_active":"2024-01-24T14:17:20.017590+0000","last_became_peered":"2024-01-24T14:17:20.017590+0000","last_unstale":"2024-01-24T14:17:20.422557+0000","last_undegraded":"2024-01-24T14:17:20.422557+0000","last_fullsized":"2024-01-24T14:17:20.422557+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451741+0000","last_change":"2024-01-24T14:17:20.445375+0000","last_active":"2024-01-24T14:17:20.451741+0000","last_peered":"2024-01-24T14:17:20.451741+0000","last_clean":"2024-01-24T14:17:20.451741+0000","last_became_active":"2024-01-24T14:17:20.444280+0000","last_became_peered":"2024-01-24T14:17:20.444280+0000","last_unstale":"2024-01-24T14:17:20.451741+0000","last_undegraded":"2024-01-24T14:17:20.451741+0000","last_fullsized":"2024-01-24T14:17:20.451741+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.421857+0000","last_change":"2024-01-24T14:17:20.018602+0000","last_active":"2024-01-24T14:17:20.421857+0000","last_peered":"2024-01-24T14:17:20.421857+0000","last_clean":"2024-01-24T14:17:20.421857+0000","last_became_active":"2024-01-24T14:17:20.018276+0000","last_became_peered":"2024-01-24T14:17:20.018276+0000","last_unstale":"2024-01-24T14:17:20.421857+0000","last_undegraded":"2024-01-24T14:17:20.421857+0000","last_fullsized":"2024-01-24T14:17:20.421857+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.445523+0000","last_change":"2024-01-24T14:17:20.438173+0000","last_active":"2024-01-24T14:17:20.445523+0000","last_peered":"2024-01-24T14:17:20.445523+0000","last_clean":"2024-01-24T14:17:20.445523+0000","last_became_active":"2024-01-24T14:17:20.437862+0000","last_became_peered":"2024-01-24T14:17:20.437862+0000","last_unstale":"2024-01-24T14:17:20.445523+0000","last_undegraded":"2024-01-24T14:17:20.445523+0000","last_fullsized":"2024-01-24T14:17:20.445523+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447434+0000","last_change":"2024-01-24T14:17:20.029454+0000","last_active":"2024-01-24T14:17:20.447434+0000","last_peered":"2024-01-24T14:17:20.447434+0000","last_clean":"2024-01-24T14:17:20.447434+0000","last_became_active":"2024-01-24T14:17:20.029263+0000","last_became_peered":"2024-01-24T14:17:20.029263+0000","last_unstale":"2024-01-24T14:17:20.447434+0000","last_undegraded":"2024-01-24T14:17:20.447434+0000","last_fullsized":"2024-01-24T14:17:20.447434+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451925+0000","last_change":"2024-01-24T14:17:20.449441+0000","last_active":"2024-01-24T14:17:20.451925+0000","last_peered":"2024-01-24T14:17:20.451925+0000","last_clean":"2024-01-24T14:17:20.451925+0000","last_became_active":"2024-01-24T14:17:20.449177+0000","last_became_peered":"2024-01-24T14:17:20.449177+0000","last_unstale":"2024-01-24T14:17:20.451925+0000","last_undegraded":"2024-01-24T14:17:20.451925+0000","last_fullsized":"2024-01-24T14:17:20.451925+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.437391+0000","last_change":"2024-01-24T14:17:20.036605+0000","last_active":"2024-01-24T14:17:20.437391+0000","last_peered":"2024-01-24T14:17:20.437391+0000","last_clean":"2024-01-24T14:17:20.437391+0000","last_became_active":"2024-01-24T14:17:20.036362+0000","last_became_peered":"2024-01-24T14:17:20.036362+0000","last_unstale":"2024-01-24T14:17:20.437391+0000","last_undegraded":"2024-01-24T14:17:20.437391+0000","last_fullsized":"2024-01-24T14:17:20.437391+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.454289+0000","last_change":"2024-01-24T14:17:20.449329+0000","last_active":"2024-01-24T14:17:20.454289+0000","last_peered":"2024-01-24T14:17:20.454289+0000","last_clean":"2024-01-24T14:17:20.454289+0000","last_became_active":"2024-01-24T14:17:20.448991+0000","last_became_peered":"2024-01-24T14:17:20.448991+0000","last_unstale":"2024-01-24T14:17:20.454289+0000","last_undegraded":"2024-01-24T14:17:20.454289+0000","last_fullsized":"2024-01-24T14:17:20.454289+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.441220+0000","last_change":"2024-01-24T14:17:20.034832+0000","last_active":"2024-01-24T14:17:20.441220+0000","last_peered":"2024-01-24T14:17:20.441220+0000","last_clean":"2024-01-24T14:17:20.441220+0000","last_became_active":"2024-01-24T14:17:20.034429+0000","last_became_peered":"2024-01-24T14:17:20.034429+0000","last_unstale":"2024-01-24T14:17:20.441220+0000","last_undegraded":"2024-01-24T14:17:20.441220+0000","last_fullsized":"2024-01-24T14:17:20.441220+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451665+0000","last_change":"2024-01-24T14:17:20.445208+0000","last_active":"2024-01-24T14:17:20.451665+0000","last_peered":"2024-01-24T14:17:20.451665+0000","last_clean":"2024-01-24T14:17:20.451665+0000","last_became_active":"2024-01-24T14:17:20.444202+0000","last_became_peered":"2024-01-24T14:17:20.444202+0000","last_unstale":"2024-01-24T14:17:20.451665+0000","last_undegraded":"2024-01-24T14:17:20.451665+0000","last_fullsized":"2024-01-24T14:17:20.451665+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.434492+0000","last_change":"2024-01-24T14:17:20.035895+0000","last_active":"2024-01-24T14:17:20.434492+0000","last_peered":"2024-01-24T14:17:20.434492+0000","last_clean":"2024-01-24T14:17:20.434492+0000","last_became_active":"2024-01-24T14:17:20.035366+0000","last_became_peered":"2024-01-24T14:17:20.035366+0000","last_unstale":"2024-01-24T14:17:20.434492+0000","last_undegraded":"2024-01-24T14:17:20.434492+0000","last_fullsized":"2024-01-24T14:17:20.434492+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.441332+0000","last_change":"2024-01-24T14:17:20.436366+0000","last_active":"2024-01-24T14:17:20.441332+0000","last_peered":"2024-01-24T14:17:20.441332+0000","last_clean":"2024-01-24T14:17:20.441332+0000","last_became_active":"2024-01-24T14:17:20.436044+0000","last_became_peered":"2024-01-24T14:17:20.436044+0000","last_unstale":"2024-01-24T14:17:20.441332+0000","last_undegraded":"2024-01-24T14:17:20.441332+0000","last_fullsized":"2024-01-24T14:17:20.441332+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.433900+0000","last_change":"2024-01-24T14:17:20.027356+0000","last_active":"2024-01-24T14:17:20.433900+0000","last_peered":"2024-01-24T14:17:20.433900+0000","last_clean":"2024-01-24T14:17:20.433900+0000","last_became_active":"2024-01-24T14:17:20.027010+0000","last_became_peered":"2024-01-24T14:17:20.027010+0000","last_unstale":"2024-01-24T14:17:20.433900+0000","last_undegraded":"2024-01-24T14:17:20.433900+0000","last_fullsized":"2024-01-24T14:17:20.433900+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.442818+0000","last_change":"2024-01-24T14:17:20.438826+0000","last_active":"2024-01-24T14:17:20.442818+0000","last_peered":"2024-01-24T14:17:20.442818+0000","last_clean":"2024-01-24T14:17:20.442818+0000","last_became_active":"2024-01-24T14:17:20.438507+0000","last_became_peered":"2024-01-24T14:17:20.438507+0000","last_unstale":"2024-01-24T14:17:20.442818+0000","last_undegraded":"2024-01-24T14:17:20.442818+0000","last_fullsized":"2024-01-24T14:17:20.442818+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.425217+0000","last_change":"2024-01-24T14:17:20.030030+0000","last_active":"2024-01-24T14:17:20.425217+0000","last_peered":"2024-01-24T14:17:20.425217+0000","last_clean":"2024-01-24T14:17:20.425217+0000","last_became_active":"2024-01-24T14:17:20.029271+0000","last_became_peered":"2024-01-24T14:17:20.029271+0000","last_unstale":"2024-01-24T14:17:20.425217+0000","last_undegraded":"2024-01-24T14:17:20.425217+0000","last_fullsized":"2024-01-24T14:17:20.425217+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449030+0000","last_change":"2024-01-24T14:17:20.444435+0000","last_active":"2024-01-24T14:17:20.449030+0000","last_peered":"2024-01-24T14:17:20.449030+0000","last_clean":"2024-01-24T14:17:20.449030+0000","last_became_active":"2024-01-24T14:17:20.444107+0000","last_became_peered":"2024-01-24T14:17:20.444107+0000","last_unstale":"2024-01-24T14:17:20.449030+0000","last_undegraded":"2024-01-24T14:17:20.449030+0000","last_fullsized":"2024-01-24T14:17:20.449030+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.422268+0000","last_change":"2024-01-24T14:17:20.023190+0000","last_active":"2024-01-24T14:17:20.422268+0000","last_peered":"2024-01-24T14:17:20.422268+0000","last_clean":"2024-01-24T14:17:20.422268+0000","last_became_active":"2024-01-24T14:17:20.022894+0000","last_became_peered":"2024-01-24T14:17:20.022894+0000","last_unstale":"2024-01-24T14:17:20.422268+0000","last_undegraded":"2024-01-24T14:17:20.422268+0000","last_fullsized":"2024-01-24T14:17:20.422268+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447836+0000","last_change":"2024-01-24T14:17:20.442688+0000","last_active":"2024-01-24T14:17:20.447836+0000","last_peered":"2024-01-24T14:17:20.447836+0000","last_clean":"2024-01-24T14:17:20.447836+0000","last_became_active":"2024-01-24T14:17:20.442384+0000","last_became_peered":"2024-01-24T14:17:20.442384+0000","last_unstale":"2024-01-24T14:17:20.447836+0000","last_undegraded":"2024-01-24T14:17:20.447836+0000","last_fullsized":"2024-01-24T14:17:20.447836+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.433616+0000","last_change":"2024-01-24T14:17:20.030600+0000","last_active":"2024-01-24T14:17:20.433616+0000","last_peered":"2024-01-24T14:17:20.433616+0000","last_clean":"2024-01-24T14:17:20.433616+0000","last_became_active":"2024-01-24T14:17:20.030260+0000","last_became_peered":"2024-01-24T14:17:20.030260+0000","last_unstale":"2024-01-24T14:17:20.433616+0000","last_undegraded":"2024-01-24T14:17:20.433616+0000","last_fullsized":"2024-01-24T14:17:20.433616+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.442803+0000","last_change":"2024-01-24T14:17:20.437473+0000","last_active":"2024-01-24T14:17:20.442803+0000","last_peered":"2024-01-24T14:17:20.442803+0000","last_clean":"2024-01-24T14:17:20.442803+0000","last_became_active":"2024-01-24T14:17:20.437211+0000","last_became_peered":"2024-01-24T14:17:20.437211+0000","last_unstale":"2024-01-24T14:17:20.442803+0000","last_undegraded":"2024-01-24T14:17:20.442803+0000","last_fullsized":"2024-01-24T14:17:20.442803+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.446096+0000","last_change":"2024-01-24T14:17:20.015954+0000","last_active":"2024-01-24T14:17:20.446096+0000","last_peered":"2024-01-24T14:17:20.446096+0000","last_clean":"2024-01-24T14:17:20.446096+0000","last_became_active":"2024-01-24T14:17:20.015682+0000","last_became_peered":"2024-01-24T14:17:20.015682+0000","last_unstale":"2024-01-24T14:17:20.446096+0000","last_undegraded":"2024-01-24T14:17:20.446096+0000","last_fullsized":"2024-01-24T14:17:20.446096+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.441771+0000","last_change":"2024-01-24T14:17:20.440811+0000","last_active":"2024-01-24T14:17:20.441771+0000","last_peered":"2024-01-24T14:17:20.441771+0000","last_clean":"2024-01-24T14:17:20.441771+0000","last_became_active":"2024-01-24T14:17:20.440486+0000","last_became_peered":"2024-01-24T14:17:20.440486+0000","last_unstale":"2024-01-24T14:17:20.441771+0000","last_undegraded":"2024-01-24T14:17:20.441771+0000","last_fullsized":"2024-01-24T14:17:20.441771+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.441487+0000","last_change":"2024-01-24T14:17:20.037718+0000","last_active":"2024-01-24T14:17:20.441487+0000","last_peered":"2024-01-24T14:17:20.441487+0000","last_clean":"2024-01-24T14:17:20.441487+0000","last_became_active":"2024-01-24T14:17:20.037463+0000","last_became_peered":"2024-01-24T14:17:20.037463+0000","last_unstale":"2024-01-24T14:17:20.441487+0000","last_undegraded":"2024-01-24T14:17:20.441487+0000","last_fullsized":"2024-01-24T14:17:20.441487+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.437312+0000","last_change":"2024-01-24T14:17:20.429721+0000","last_active":"2024-01-24T14:17:20.437312+0000","last_peered":"2024-01-24T14:17:20.437312+0000","last_clean":"2024-01-24T14:17:20.437312+0000","last_became_active":"2024-01-24T14:17:20.429442+0000","last_became_peered":"2024-01-24T14:17:20.429442+0000","last_unstale":"2024-01-24T14:17:20.437312+0000","last_undegraded":"2024-01-24T14:17:20.437312+0000","last_fullsized":"2024-01-24T14:17:20.437312+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423945+0000","last_change":"2024-01-24T14:17:20.024636+0000","last_active":"2024-01-24T14:17:20.423945+0000","last_peered":"2024-01-24T14:17:20.423945+0000","last_clean":"2024-01-24T14:17:20.423945+0000","last_became_active":"2024-01-24T14:17:20.024408+0000","last_became_peered":"2024-01-24T14:17:20.024408+0000","last_unstale":"2024-01-24T14:17:20.423945+0000","last_undegraded":"2024-01-24T14:17:20.423945+0000","last_fullsized":"2024-01-24T14:17:20.423945+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447999+0000","last_change":"2024-01-24T14:17:20.445755+0000","last_active":"2024-01-24T14:17:20.447999+0000","last_peered":"2024-01-24T14:17:20.447999+0000","last_clean":"2024-01-24T14:17:20.447999+0000","last_became_active":"2024-01-24T14:17:20.445267+0000","last_became_peered":"2024-01-24T14:17:20.445267+0000","last_unstale":"2024-01-24T14:17:20.447999+0000","last_undegraded":"2024-01-24T14:17:20.447999+0000","last_fullsized":"2024-01-24T14:17:20.447999+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.421578+0000","last_change":"2024-01-24T14:17:20.024180+0000","last_active":"2024-01-24T14:17:20.421578+0000","last_peered":"2024-01-24T14:17:20.421578+0000","last_clean":"2024-01-24T14:17:20.421578+0000","last_became_active":"2024-01-24T14:17:20.023866+0000","last_became_peered":"2024-01-24T14:17:20.023866+0000","last_unstale":"2024-01-24T14:17:20.421578+0000","last_undegraded":"2024-01-24T14:17:20.421578+0000","last_fullsized":"2024-01-24T14:17:20.421578+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449039+0000","last_change":"2024-01-24T14:17:20.443241+0000","last_active":"2024-01-24T14:17:20.449039+0000","last_peered":"2024-01-24T14:17:20.449039+0000","last_clean":"2024-01-24T14:17:20.449039+0000","last_became_active":"2024-01-24T14:17:20.439755+0000","last_became_peered":"2024-01-24T14:17:20.439755+0000","last_unstale":"2024-01-24T14:17:20.449039+0000","last_undegraded":"2024-01-24T14:17:20.449039+0000","last_fullsized":"2024-01-24T14:17:20.449039+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.437747+0000","last_change":"2024-01-24T14:17:20.018026+0000","last_active":"2024-01-24T14:17:20.437747+0000","last_peered":"2024-01-24T14:17:20.437747+0000","last_clean":"2024-01-24T14:17:20.437747+0000","last_became_active":"2024-01-24T14:17:20.017715+0000","last_became_peered":"2024-01-24T14:17:20.017715+0000","last_unstale":"2024-01-24T14:17:20.437747+0000","last_undegraded":"2024-01-24T14:17:20.437747+0000","last_fullsized":"2024-01-24T14:17:20.437747+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.442133+0000","last_change":"2024-01-24T14:17:20.434915+0000","last_active":"2024-01-24T14:17:20.442133+0000","last_peered":"2024-01-24T14:17:20.442133+0000","last_clean":"2024-01-24T14:17:20.442133+0000","last_became_active":"2024-01-24T14:17:20.434607+0000","last_became_peered":"2024-01-24T14:17:20.434607+0000","last_unstale":"2024-01-24T14:17:20.442133+0000","last_undegraded":"2024-01-24T14:17:20.442133+0000","last_fullsized":"2024-01-24T14:17:20.442133+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423873+0000","last_change":"2024-01-24T14:17:20.025281+0000","last_active":"2024-01-24T14:17:20.423873+0000","last_peered":"2024-01-24T14:17:20.423873+0000","last_clean":"2024-01-24T14:17:20.423873+0000","last_became_active":"2024-01-24T14:17:20.024504+0000","last_became_peered":"2024-01-24T14:17:20.024504+0000","last_unstale":"2024-01-24T14:17:20.423873+0000","last_undegraded":"2024-01-24T14:17:20.423873+0000","last_fullsized":"2024-01-24T14:17:20.423873+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447824+0000","last_change":"2024-01-24T14:17:20.441907+0000","last_active":"2024-01-24T14:17:20.447824+0000","last_peered":"2024-01-24T14:17:20.447824+0000","last_clean":"2024-01-24T14:17:20.447824+0000","last_became_active":"2024-01-24T14:17:20.441615+0000","last_became_peered":"2024-01-24T14:17:20.441615+0000","last_unstale":"2024-01-24T14:17:20.447824+0000","last_undegraded":"2024-01-24T14:17:20.447824+0000","last_fullsized":"2024-01-24T14:17:20.447824+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423190+0000","last_change":"2024-01-24T14:17:20.028922+0000","last_active":"2024-01-24T14:17:20.423190+0000","last_peered":"2024-01-24T14:17:20.423190+0000","last_clean":"2024-01-24T14:17:20.423190+0000","last_became_active":"2024-01-24T14:17:20.028549+0000","last_became_peered":"2024-01-24T14:17:20.028549+0000","last_unstale":"2024-01-24T14:17:20.423190+0000","last_undegraded":"2024-01-24T14:17:20.423190+0000","last_fullsized":"2024-01-24T14:17:20.423190+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451842+0000","last_change":"2024-01-24T14:17:20.445971+0000","last_active":"2024-01-24T14:17:20.451842+0000","last_peered":"2024-01-24T14:17:20.451842+0000","last_clean":"2024-01-24T14:17:20.451842+0000","last_became_active":"2024-01-24T14:17:20.445160+0000","last_became_peered":"2024-01-24T14:17:20.445160+0000","last_unstale":"2024-01-24T14:17:20.451842+0000","last_undegraded":"2024-01-24T14:17:20.451842+0000","last_fullsized":"2024-01-24T14:17:20.451842+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.430666+0000","last_change":"2024-01-24T14:17:20.037299+0000","last_active":"2024-01-24T14:17:20.430666+0000","last_peered":"2024-01-24T14:17:20.430666+0000","last_clean":"2024-01-24T14:17:20.430666+0000","last_became_active":"2024-01-24T14:17:20.036972+0000","last_became_peered":"2024-01-24T14:17:20.036972+0000","last_unstale":"2024-01-24T14:17:20.430666+0000","last_undegraded":"2024-01-24T14:17:20.430666+0000","last_fullsized":"2024-01-24T14:17:20.430666+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.435012+0000","last_change":"2024-01-24T14:17:20.429204+0000","last_active":"2024-01-24T14:17:20.435012+0000","last_peered":"2024-01-24T14:17:20.435012+0000","last_clean":"2024-01-24T14:17:20.435012+0000","last_became_active":"2024-01-24T14:17:20.428903+0000","last_became_peered":"2024-01-24T14:17:20.428903+0000","last_unstale":"2024-01-24T14:17:20.435012+0000","last_undegraded":"2024-01-24T14:17:20.435012+0000","last_fullsized":"2024-01-24T14:17:20.435012+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.443079+0000","last_change":"2024-01-24T14:17:20.017724+0000","last_active":"2024-01-24T14:17:20.443079+0000","last_peered":"2024-01-24T14:17:20.443079+0000","last_clean":"2024-01-24T14:17:20.443079+0000","last_became_active":"2024-01-24T14:17:20.017445+0000","last_became_peered":"2024-01-24T14:17:20.017445+0000","last_unstale":"2024-01-24T14:17:20.443079+0000","last_undegraded":"2024-01-24T14:17:20.443079+0000","last_fullsized":"2024-01-24T14:17:20.443079+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448140+0000","last_change":"2024-01-24T14:17:20.438796+0000","last_active":"2024-01-24T14:17:20.448140+0000","last_peered":"2024-01-24T14:17:20.448140+0000","last_clean":"2024-01-24T14:17:20.448140+0000","last_became_active":"2024-01-24T14:17:20.438504+0000","last_became_peered":"2024-01-24T14:17:20.438504+0000","last_unstale":"2024-01-24T14:17:20.448140+0000","last_undegraded":"2024-01-24T14:17:20.448140+0000","last_fullsized":"2024-01-24T14:17:20.448140+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.428283+0000","last_change":"2024-01-24T14:17:20.028340+0000","last_active":"2024-01-24T14:17:20.428283+0000","last_peered":"2024-01-24T14:17:20.428283+0000","last_clean":"2024-01-24T14:17:20.428283+0000","last_became_active":"2024-01-24T14:17:20.028046+0000","last_became_peered":"2024-01-24T14:17:20.028046+0000","last_unstale":"2024-01-24T14:17:20.428283+0000","last_undegraded":"2024-01-24T14:17:20.428283+0000","last_fullsized":"2024-01-24T14:17:20.428283+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.453080+0000","last_change":"2024-01-24T14:17:20.450691+0000","last_active":"2024-01-24T14:17:20.453080+0000","last_peered":"2024-01-24T14:17:20.453080+0000","last_clean":"2024-01-24T14:17:20.453080+0000","last_became_active":"2024-01-24T14:17:20.450409+0000","last_became_peered":"2024-01-24T14:17:20.450409+0000","last_unstale":"2024-01-24T14:17:20.453080+0000","last_undegraded":"2024-01-24T14:17:20.453080+0000","last_fullsized":"2024-01-24T14:17:20.453080+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.435092+0000","last_change":"2024-01-24T14:17:20.030946+0000","last_active":"2024-01-24T14:17:20.435092+0000","last_peered":"2024-01-24T14:17:20.435092+0000","last_clean":"2024-01-24T14:17:20.435092+0000","last_became_active":"2024-01-24T14:17:20.030458+0000","last_became_peered":"2024-01-24T14:17:20.030458+0000","last_unstale":"2024-01-24T14:17:20.435092+0000","last_undegraded":"2024-01-24T14:17:20.435092+0000","last_fullsized":"2024-01-24T14:17:20.435092+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.442721+0000","last_change":"2024-01-24T14:17:20.431613+0000","last_active":"2024-01-24T14:17:20.442721+0000","last_peered":"2024-01-24T14:17:20.442721+0000","last_clean":"2024-01-24T14:17:20.442721+0000","last_became_active":"2024-01-24T14:17:20.431320+0000","last_became_peered":"2024-01-24T14:17:20.431320+0000","last_unstale":"2024-01-24T14:17:20.442721+0000","last_undegraded":"2024-01-24T14:17:20.442721+0000","last_fullsized":"2024-01-24T14:17:20.442721+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.431682+0000","last_change":"2024-01-24T14:17:20.034473+0000","last_active":"2024-01-24T14:17:20.431682+0000","last_peered":"2024-01-24T14:17:20.431682+0000","last_clean":"2024-01-24T14:17:20.431682+0000","last_became_active":"2024-01-24T14:17:20.034154+0000","last_became_peered":"2024-01-24T14:17:20.034154+0000","last_unstale":"2024-01-24T14:17:20.431682+0000","last_undegraded":"2024-01-24T14:17:20.431682+0000","last_fullsized":"2024-01-24T14:17:20.431682+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451812+0000","last_change":"2024-01-24T14:17:20.445791+0000","last_active":"2024-01-24T14:17:20.451812+0000","last_peered":"2024-01-24T14:17:20.451812+0000","last_clean":"2024-01-24T14:17:20.451812+0000","last_became_active":"2024-01-24T14:17:20.444443+0000","last_became_peered":"2024-01-24T14:17:20.444443+0000","last_unstale":"2024-01-24T14:17:20.451812+0000","last_undegraded":"2024-01-24T14:17:20.451812+0000","last_fullsized":"2024-01-24T14:17:20.451812+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.433921+0000","last_change":"2024-01-24T14:17:20.026060+0000","last_active":"2024-01-24T14:17:20.433921+0000","last_peered":"2024-01-24T14:17:20.433921+0000","last_clean":"2024-01-24T14:17:20.433921+0000","last_became_active":"2024-01-24T14:17:20.025751+0000","last_became_peered":"2024-01-24T14:17:20.025751+0000","last_unstale":"2024-01-24T14:17:20.433921+0000","last_undegraded":"2024-01-24T14:17:20.433921+0000","last_fullsized":"2024-01-24T14:17:20.433921+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.440817+0000","last_change":"2024-01-24T14:17:20.432753+0000","last_active":"2024-01-24T14:17:20.440817+0000","last_peered":"2024-01-24T14:17:20.440817+0000","last_clean":"2024-01-24T14:17:20.440817+0000","last_became_active":"2024-01-24T14:17:20.432407+0000","last_became_peered":"2024-01-24T14:17:20.432407+0000","last_unstale":"2024-01-24T14:17:20.440817+0000","last_undegraded":"2024-01-24T14:17:20.440817+0000","last_fullsized":"2024-01-24T14:17:20.440817+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.428882+0000","last_change":"2024-01-24T14:17:20.034798+0000","last_active":"2024-01-24T14:17:20.428882+0000","last_peered":"2024-01-24T14:17:20.428882+0000","last_clean":"2024-01-24T14:17:20.428882+0000","last_became_active":"2024-01-24T14:17:20.034552+0000","last_became_peered":"2024-01-24T14:17:20.034552+0000","last_unstale":"2024-01-24T14:17:20.428882+0000","last_undegraded":"2024-01-24T14:17:20.428882+0000","last_fullsized":"2024-01-24T14:17:20.428882+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.445827+0000","last_change":"2024-01-24T14:17:20.442888+0000","last_active":"2024-01-24T14:17:20.445827+0000","last_peered":"2024-01-24T14:17:20.445827+0000","last_clean":"2024-01-24T14:17:20.445827+0000","last_became_active":"2024-01-24T14:17:20.442596+0000","last_became_peered":"2024-01-24T14:17:20.442596+0000","last_unstale":"2024-01-24T14:17:20.445827+0000","last_undegraded":"2024-01-24T14:17:20.445827+0000","last_fullsized":"2024-01-24T14:17:20.445827+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423297+0000","last_change":"2024-01-24T14:17:20.022465+0000","last_active":"2024-01-24T14:17:20.423297+0000","last_peered":"2024-01-24T14:17:20.423297+0000","last_clean":"2024-01-24T14:17:20.423297+0000","last_became_active":"2024-01-24T14:17:20.022142+0000","last_became_peered":"2024-01-24T14:17:20.022142+0000","last_unstale":"2024-01-24T14:17:20.423297+0000","last_undegraded":"2024-01-24T14:17:20.423297+0000","last_fullsized":"2024-01-24T14:17:20.423297+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.453069+0000","last_change":"2024-01-24T14:17:20.450415+0000","last_active":"2024-01-24T14:17:20.453069+0000","last_peered":"2024-01-24T14:17:20.453069+0000","last_clean":"2024-01-24T14:17:20.453069+0000","last_became_active":"2024-01-24T14:17:20.449996+0000","last_became_peered":"2024-01-24T14:17:20.449996+0000","last_unstale":"2024-01-24T14:17:20.453069+0000","last_undegraded":"2024-01-24T14:17:20.453069+0000","last_fullsized":"2024-01-24T14:17:20.453069+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.430608+0000","last_change":"2024-01-24T14:17:20.021844+0000","last_active":"2024-01-24T14:17:20.430608+0000","last_peered":"2024-01-24T14:17:20.430608+0000","last_clean":"2024-01-24T14:17:20.430608+0000","last_became_active":"2024-01-24T14:17:20.021506+0000","last_became_peered":"2024-01-24T14:17:20.021506+0000","last_unstale":"2024-01-24T14:17:20.430608+0000","last_undegraded":"2024-01-24T14:17:20.430608+0000","last_fullsized":"2024-01-24T14:17:20.430608+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451761+0000","last_change":"2024-01-24T14:17:20.445477+0000","last_active":"2024-01-24T14:17:20.451761+0000","last_peered":"2024-01-24T14:17:20.451761+0000","last_clean":"2024-01-24T14:17:20.451761+0000","last_became_active":"2024-01-24T14:17:20.444918+0000","last_became_peered":"2024-01-24T14:17:20.444918+0000","last_unstale":"2024-01-24T14:17:20.451761+0000","last_undegraded":"2024-01-24T14:17:20.451761+0000","last_fullsized":"2024-01-24T14:17:20.451761+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.450254+0000","last_change":"2024-01-24T14:17:20.020286+0000","last_active":"2024-01-24T14:17:20.450254+0000","last_peered":"2024-01-24T14:17:20.450254+0000","last_clean":"2024-01-24T14:17:20.450254+0000","last_became_active":"2024-01-24T14:17:20.020025+0000","last_became_peered":"2024-01-24T14:17:20.020025+0000","last_unstale":"2024-01-24T14:17:20.450254+0000","last_undegraded":"2024-01-24T14:17:20.450254+0000","last_fullsized":"2024-01-24T14:17:20.450254+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.434613+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T14:17:20.434613+0000","last_peered":"2024-01-24T14:17:20.434613+0000","last_clean":"2024-01-24T14:17:20.434613+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T14:17:20.434613+0000","last_undegraded":"2024-01-24T14:17:20.434613+0000","last_fullsized":"2024-01-24T14:17:20.434613+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449140+0000","last_change":"2024-01-24T14:17:20.447404+0000","last_active":"2024-01-24T14:17:20.449140+0000","last_peered":"2024-01-24T14:17:20.449140+0000","last_clean":"2024-01-24T14:17:20.449140+0000","last_became_active":"2024-01-24T14:17:20.439657+0000","last_became_peered":"2024-01-24T14:17:20.439657+0000","last_unstale":"2024-01-24T14:17:20.449140+0000","last_undegraded":"2024-01-24T14:17:20.449140+0000","last_fullsized":"2024-01-24T14:17:20.449140+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.418426+0000","last_change":"2024-01-24T14:17:20.020233+0000","last_active":"2024-01-24T14:17:20.418426+0000","last_peered":"2024-01-24T14:17:20.418426+0000","last_clean":"2024-01-24T14:17:20.418426+0000","last_became_active":"2024-01-24T14:17:20.019923+0000","last_became_peered":"2024-01-24T14:17:20.019923+0000","last_unstale":"2024-01-24T14:17:20.418426+0000","last_undegraded":"2024-01-24T14:17:20.418426+0000","last_fullsized":"2024-01-24T14:17:20.418426+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.421302+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T14:17:20.421302+0000","last_peered":"2024-01-24T14:17:20.421302+0000","last_clean":"2024-01-24T14:17:20.421302+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T14:17:20.421302+0000","last_undegraded":"2024-01-24T14:17:20.421302+0000","last_fullsized":"2024-01-24T14:17:20.421302+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447849+0000","last_change":"2024-01-24T14:17:20.442874+0000","last_active":"2024-01-24T14:17:20.447849+0000","last_peered":"2024-01-24T14:17:20.447849+0000","last_clean":"2024-01-24T14:17:20.447849+0000","last_became_active":"2024-01-24T14:17:20.442594+0000","last_became_peered":"2024-01-24T14:17:20.442594+0000","last_unstale":"2024-01-24T14:17:20.447849+0000","last_undegraded":"2024-01-24T14:17:20.447849+0000","last_fullsized":"2024-01-24T14:17:20.447849+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.434688+0000","last_change":"2024-01-24T14:17:20.029762+0000","last_active":"2024-01-24T14:17:20.434688+0000","last_peered":"2024-01-24T14:17:20.434688+0000","last_clean":"2024-01-24T14:17:20.434688+0000","last_became_active":"2024-01-24T14:17:20.029474+0000","last_became_peered":"2024-01-24T14:17:20.029474+0000","last_unstale":"2024-01-24T14:17:20.434688+0000","last_undegraded":"2024-01-24T14:17:20.434688+0000","last_fullsized":"2024-01-24T14:17:20.434688+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.418040+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T14:17:20.418040+0000","last_peered":"2024-01-24T14:17:20.418040+0000","last_clean":"2024-01-24T14:17:20.418040+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T14:17:20.418040+0000","last_undegraded":"2024-01-24T14:17:20.418040+0000","last_fullsized":"2024-01-24T14:17:20.418040+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.451788+0000","last_change":"2024-01-24T14:17:20.445595+0000","last_active":"2024-01-24T14:17:20.451788+0000","last_peered":"2024-01-24T14:17:20.451788+0000","last_clean":"2024-01-24T14:17:20.451788+0000","last_became_active":"2024-01-24T14:17:20.444716+0000","last_became_peered":"2024-01-24T14:17:20.444716+0000","last_unstale":"2024-01-24T14:17:20.451788+0000","last_undegraded":"2024-01-24T14:17:20.451788+0000","last_fullsized":"2024-01-24T14:17:20.451788+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.449022+0000","last_change":"2024-01-24T14:17:20.027508+0000","last_active":"2024-01-24T14:17:20.449022+0000","last_peered":"2024-01-24T14:17:20.449022+0000","last_clean":"2024-01-24T14:17:20.449022+0000","last_became_active":"2024-01-24T14:17:20.027246+0000","last_became_peered":"2024-01-24T14:17:20.027246+0000","last_unstale":"2024-01-24T14:17:20.449022+0000","last_undegraded":"2024-01-24T14:17:20.449022+0000","last_fullsized":"2024-01-24T14:17:20.449022+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.422230+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T14:17:20.422230+0000","last_peered":"2024-01-24T14:17:20.422230+0000","last_clean":"2024-01-24T14:17:20.422230+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T14:17:20.422230+0000","last_undegraded":"2024-01-24T14:17:20.422230+0000","last_fullsized":"2024-01-24T14:17:20.422230+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.427575+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T14:17:20.427575+0000","last_peered":"2024-01-24T14:17:20.427575+0000","last_clean":"2024-01-24T14:17:20.427575+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T14:17:20.427575+0000","last_undegraded":"2024-01-24T14:17:20.427575+0000","last_fullsized":"2024-01-24T14:17:20.427575+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448634+0000","last_change":"2024-01-24T14:17:20.442710+0000","last_active":"2024-01-24T14:17:20.448634+0000","last_peered":"2024-01-24T14:17:20.448634+0000","last_clean":"2024-01-24T14:17:20.448634+0000","last_became_active":"2024-01-24T14:17:20.442434+0000","last_became_peered":"2024-01-24T14:17:20.442434+0000","last_unstale":"2024-01-24T14:17:20.448634+0000","last_undegraded":"2024-01-24T14:17:20.448634+0000","last_fullsized":"2024-01-24T14:17:20.448634+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"152'4","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.447271+0000","last_change":"2024-01-24T14:17:20.027163+0000","last_active":"2024-01-24T14:17:20.447271+0000","last_peered":"2024-01-24T14:17:20.447271+0000","last_clean":"2024-01-24T14:17:20.447271+0000","last_became_active":"2024-01-24T14:17:20.026862+0000","last_became_peered":"2024-01-24T14:17:20.026862+0000","last_unstale":"2024-01-24T14:17:20.447271+0000","last_undegraded":"2024-01-24T14:17:20.447271+0000","last_fullsized":"2024-01-24T14:17:20.447271+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.427474+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T14:17:20.427474+0000","last_peered":"2024-01-24T14:17:20.427474+0000","last_clean":"2024-01-24T14:17:20.427474+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T14:17:20.427474+0000","last_undegraded":"2024-01-24T14:17:20.427474+0000","last_fullsized":"2024-01-24T14:17:20.427474+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.454308+0000","last_change":"2024-01-24T14:17:20.449377+0000","last_active":"2024-01-24T14:17:20.454308+0000","last_peered":"2024-01-24T14:17:20.454308+0000","last_clean":"2024-01-24T14:17:20.454308+0000","last_became_active":"2024-01-24T14:17:20.449015+0000","last_became_peered":"2024-01-24T14:17:20.449015+0000","last_unstale":"2024-01-24T14:17:20.454308+0000","last_undegraded":"2024-01-24T14:17:20.454308+0000","last_fullsized":"2024-01-24T14:17:20.454308+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"152'3","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.440656+0000","last_change":"2024-01-24T14:17:20.024502+0000","last_active":"2024-01-24T14:17:20.440656+0000","last_peered":"2024-01-24T14:17:20.440656+0000","last_clean":"2024-01-24T14:17:20.440656+0000","last_became_active":"2024-01-24T14:17:20.024226+0000","last_became_peered":"2024-01-24T14:17:20.024226+0000","last_unstale":"2024-01-24T14:17:20.440656+0000","last_undegraded":"2024-01-24T14:17:20.440656+0000","last_fullsized":"2024-01-24T14:17:20.440656+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.428398+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T14:17:20.428398+0000","last_peered":"2024-01-24T14:17:20.428398+0000","last_clean":"2024-01-24T14:17:20.428398+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T14:17:20.428398+0000","last_undegraded":"2024-01-24T14:17:20.428398+0000","last_fullsized":"2024-01-24T14:17:20.428398+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.441037+0000","last_change":"2024-01-24T14:17:20.437167+0000","last_active":"2024-01-24T14:17:20.441037+0000","last_peered":"2024-01-24T14:17:20.441037+0000","last_clean":"2024-01-24T14:17:20.441037+0000","last_became_active":"2024-01-24T14:17:20.436887+0000","last_became_peered":"2024-01-24T14:17:20.436887+0000","last_unstale":"2024-01-24T14:17:20.441037+0000","last_undegraded":"2024-01-24T14:17:20.441037+0000","last_fullsized":"2024-01-24T14:17:20.441037+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.423534+0000","last_change":"2024-01-24T14:17:20.023079+0000","last_active":"2024-01-24T14:17:20.423534+0000","last_peered":"2024-01-24T14:17:20.423534+0000","last_clean":"2024-01-24T14:17:20.423534+0000","last_became_active":"2024-01-24T14:17:20.022752+0000","last_became_peered":"2024-01-24T14:17:20.022752+0000","last_unstale":"2024-01-24T14:17:20.423534+0000","last_undegraded":"2024-01-24T14:17:20.423534+0000","last_fullsized":"2024-01-24T14:17:20.423534+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.435186+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T14:17:20.435186+0000","last_peered":"2024-01-24T14:17:20.435186+0000","last_clean":"2024-01-24T14:17:20.435186+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T14:17:20.435186+0000","last_undegraded":"2024-01-24T14:17:20.435186+0000","last_fullsized":"2024-01-24T14:17:20.435186+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.448135+0000","last_change":"2024-01-24T14:17:20.439321+0000","last_active":"2024-01-24T14:17:20.448135+0000","last_peered":"2024-01-24T14:17:20.448135+0000","last_clean":"2024-01-24T14:17:20.448135+0000","last_became_active":"2024-01-24T14:17:20.439019+0000","last_became_peered":"2024-01-24T14:17:20.439019+0000","last_unstale":"2024-01-24T14:17:20.448135+0000","last_undegraded":"2024-01-24T14:17:20.448135+0000","last_fullsized":"2024-01-24T14:17:20.448135+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:19.993314+0000","last_clean_scrub_stamp":"2024-01-24T14:17:19.993314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.424818+0000","last_change":"2024-01-24T14:17:20.018330+0000","last_active":"2024-01-24T14:17:20.424818+0000","last_peered":"2024-01-24T14:17:20.424818+0000","last_clean":"2024-01-24T14:17:20.424818+0000","last_became_active":"2024-01-24T14:17:20.017816+0000","last_became_peered":"2024-01-24T14:17:20.017816+0000","last_unstale":"2024-01-24T14:17:20.424818+0000","last_undegraded":"2024-01-24T14:17:20.424818+0000","last_fullsized":"2024-01-24T14:17:20.424818+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T14:17:18.986552+0000","last_clean_scrub_stamp":"2024-01-24T14:17:18.986552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-01-24T14:17:20.432947+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T14:17:20.432947+0000","last_peered":"2024-01-24T14:17:20.432947+0000","last_clean":"2024-01-24T14:17:20.432947+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T14:17:20.432947+0000","last_undegraded":"2024-01-24T14:17:20.432947+0000","last_fullsized":"2024-01-24T14:17:20.432947+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575076,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303068,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068772,"statfs":{"total":96636764160,"available":96326422528,"internally_reserved":0,"allocated":1921024,"data_stored":1076869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.26000000000000001},"max":{"1min":0.76300000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.82499999999999996,"15min":0.83399999999999996},"last":0.60899999999999999}]},{"osd":1,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77000000000000002,"5min":0.85099999999999998,"15min":0.85599999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.57399999999999995,"15min":0.58099999999999996},"min":{"1min":0.376,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.753,"5min":0.80600000000000005,"15min":0.80900000000000005},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72499999999999998,"5min":0.872,"15min":0.872},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81499999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.67200000000000004}]},{"osd":3,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60399999999999998,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77300000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.35499999999999998},"max":{"1min":0.67600000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47999999999999998,"15min":0.48499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48199999999999998,"15min":0.48899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.71199999999999997,"15min":0.745},"last":0.627}]},{"osd":5,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.502,"15min":0.496},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.621,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.33100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.749,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed Jan 24 14:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73199999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.39700000000000002,"5min":0.36299999999999999,"15min":0.36099999999999999},"max":{"1min":0.77500000000000002,"5min":0.80700000000000005,"15min":0.83399999999999996},"last":0.77000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575077,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303060,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068780,"statfs":{"total":96636764160,"available":96326430720,"internally_reserved":0,"allocated":1912832,"data_stored":1076813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.59399999999999997},"min":{"1min":0.38900000000000001,"5min":0.32700000000000001,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.752,"15min":0.85699999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.58999999999999997},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.29999999999999999},"max":{"1min":0.75700000000000001,"5min":0.82799999999999996,"15min":0.84899999999999998},"last":0.47599999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56599999999999995,"15min":0.58799999999999997},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71299999999999997,"5min":0.78700000000000003,"15min":0.88},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.40600000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76300000000000001,"5min":0.78700000000000003,"15min":1.032},"last":0.38100000000000001}]},{"osd":2,"last update":"Wed Jan 24 14:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.60499999999999998},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.88500000000000001},"last":0.503},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58599999999999997,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76700000000000002,"5min":0.871,"15min":0.871},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed Jan 24 14:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59999999999999998,"15min":0.61699999999999999},"min":{"1min":0.45800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.89200000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.60099999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78700000000000003,"5min":0.81499999999999995,"15min":1.2430000000000001},"last":0.51700000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48399999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.28899999999999998},"max":{"1min":0.73899999999999999,"5min":0.83899999999999997,"15min":1.165},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.505,"15min":0.51200000000000001},"min":{"1min":0.34899999999999998,"5min":0.32900000000000001,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.74399999999999999,"15min":0.82299999999999995},"last":0.627}]},{"osd":5,"last update":"Wed Jan 24 14:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.80800000000000005,"5min":0.81100000000000005,"15min":0.86699999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.34300000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68700000000000006,"5min":0.80200000000000005,"15min":0.85699999999999998},"last":0.55100000000000005}]},{"osd":7,"last update":"Wed Jan 24 14:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.38200000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":1.2070000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.45800000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.89600000000000002},"last":0.60499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575076,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307148,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064692,"statfs":{"total":96636764160,"available":96322244608,"internally_reserved":0,"allocated":1904640,"data_stored":1076653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39100000000000001,"15min":0.441},"min":{"1min":0.27900000000000003,"5min":0.249,"15min":0.23400000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":15.298999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.42399999999999999,"15min":0.46899999999999997},"min":{"1min":0.311,"5min":0.28299999999999997,"15min":0.254},"max":{"1min":0.54400000000000004,"5min":0.67500000000000004,"15min":14.654},"last":0.42299999999999999}]},{"osd":2,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.33800000000000002,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.70799999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.44700000000000001},"min":{"1min":0.316,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":0.56399999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.34999999999999998}]},{"osd":3,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"min":{"1min":0.35199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72799999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47999999999999998,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63400000000000001,"5min":1.651,"15min":1.651},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64100000000000001,"15min":0.66400000000000003},"min":{"1min":0.45400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.47499999999999998,"5min":0.46500000000000002,"15min":0.36499999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.218},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69499999999999995,"15min":0.83599999999999997},"min":{"1min":0.51900000000000002,"5min":0.46600000000000003,"15min":0.45000000000000001},"max":{"1min":1.159,"5min":1.159,"15min":38.808},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.81000000000000005},"min":{"1min":0.47499999999999998,"5min":0.46800000000000003,"15min":0.40500000000000003},"max":{"1min":1.046,"5min":1.046,"15min":38.823},"last":0.55900000000000005}]},{"osd":6,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67700000000000005,"15min":0.68400000000000005},"min":{"1min":0.51700000000000002,"5min":0.48799999999999999,"15min":0.44500000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69899999999999995,"15min":0.71999999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.40000000000000002},"max":{"1min":0.995,"5min":1.1439999999999999,"15min":1.157},"last":0.55000000000000004}]},{"osd":7,"last update":"Wed Jan 24 14:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68300000000000005,"15min":0.69499999999999995},"min":{"1min":0.55200000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72399999999999998,"15min":0.747},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0549999999999999,"5min":1.103,"15min":1.131},"last":0.58499999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542374,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307144,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064696,"statfs":{"total":96636764160,"available":96322248704,"internally_reserved":0,"allocated":1900544,"data_stored":1076030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.30099999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.55200000000000005,"5min":0.60899999999999999,"15min":0.69399999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.32600000000000001,"5min":0.28699999999999998,"15min":0.223},"max":{"1min":0.55300000000000005,"5min":0.60099999999999998,"15min":0.71699999999999997},"last":0.29099999999999998}]},{"osd":2,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.34399999999999997,"5min":0.29699999999999999,"15min":0.28999999999999998},"max":{"1min":0.61699999999999999,"5min":0.66100000000000003,"15min":0.66700000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.46500000000000002,"15min":0.46200000000000002},"min":{"1min":0.36699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.79400000000000004},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.436},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.25900000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":1.482},"last":0.376},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.498,"15min":0.502},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69299999999999995,"5min":0.82499999999999996,"15min":0.872},"last":0.32200000000000001}]},{"osd":4,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64100000000000001,"15min":0.65000000000000002},"min":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83799999999999997,"5min":0.91300000000000003,"15min":0.95899999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.438,"15min":0.40000000000000002},"max":{"1min":0.82299999999999995,"5min":0.90000000000000002,"15min":1.5669999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.51600000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":1.655},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.53300000000000003,"5min":0.46300000000000002,"15min":0.42999999999999999},"max":{"1min":0.878,"5min":1.234,"15min":1.234},"last":0.53100000000000003}]},{"osd":6,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69299999999999995,"15min":0.69199999999999995},"min":{"1min":0.56499999999999995,"5min":0.47999999999999998,"15min":0.45500000000000002},"max":{"1min":0.82999999999999996,"5min":1.196,"15min":1.196},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83299999999999996,"5min":0.96499999999999997,"15min":1.117},"last":0.54100000000000004}]},{"osd":7,"last update":"Wed Jan 24 14:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.55700000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88700000000000001,"5min":0.93999999999999995,"15min":0.98199999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88200000000000001,"5min":1.2110000000000001,"15min":1.6160000000000001},"last":0.46100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575076,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307148,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064692,"statfs":{"total":96636764160,"available":96322244608,"internally_reserved":0,"allocated":1904640,"data_stored":1076133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.31,"5min":0.26700000000000002,"15min":0.24199999999999999},"max":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.65800000000000003},"last":0.31},{"interface":"front","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.435},"min":{"1min":0.35899999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63500000000000001,"5min":0.64500000000000002,"15min":0.65900000000000003},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed Jan 24 14:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.432,"15min":0.41799999999999998},"min":{"1min":0.33700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.433,"5min":0.44900000000000001,"15min":0.45900000000000002},"min":{"1min":0.34599999999999997,"5min":0.30399999999999999,"15min":0.29699999999999999},"max":{"1min":0.59899999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.44600000000000001}]},{"osd":3,"last update":"Wed Jan 24 14:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44800000000000001,"15min":0.45200000000000001},"min":{"1min":0.36899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.52900000000000003,"5min":0.64300000000000002,"15min":0.71699999999999997},"last":0.379},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.47499999999999998,"15min":0.47299999999999998},"min":{"1min":0.38800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.72799999999999998},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Jan 24 14:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63600000000000001,"15min":0.64100000000000001},"min":{"1min":0.45800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92400000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.47899999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.96599999999999997,"15min":1.042},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Jan 24 14:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66200000000000003,"15min":0.67700000000000005},"min":{"1min":0.47699999999999998,"5min":0.46000000000000002,"15min":0.42599999999999999},"max":{"1min":0.95799999999999996,"5min":1.036,"15min":1.0469999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66600000000000004,"15min":0.66900000000000004},"min":{"1min":0.50700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91200000000000003,"5min":1.107,"15min":1.107},"last":0.72199999999999998}]},{"osd":6,"last update":"Wed Jan 24 14:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67900000000000005,"15min":0.69399999999999995},"min":{"1min":0.53400000000000003,"5min":0.48699999999999999,"15min":0.45600000000000002},"max":{"1min":0.98299999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.51800000000000002,"5min":0.46800000000000003,"15min":0.45800000000000002},"max":{"1min":0.90200000000000002,"5min":1.141,"15min":1.141},"last":0.83299999999999996}]},{"osd":7,"last update":"Wed Jan 24 14:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69699999999999995,"15min":0.71399999999999997},"min":{"1min":0.53500000000000003,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":1.0409999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71599999999999997,"15min":0.72499999999999998},"min":{"1min":0.56100000000000005,"5min":0.497,"15min":0.49199999999999999},"max":{"1min":0.93700000000000006,"5min":0.97999999999999998,"15min":1.0860000000000001},"last":0.86299999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575076,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303052,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068788,"statfs":{"total":96636764160,"available":96326438912,"internally_reserved":0,"allocated":1904640,"data_stored":1076189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40100000000000002,"15min":0.41699999999999998},"min":{"1min":0.28000000000000003,"5min":0.23400000000000001,"15min":0.22900000000000001},"max":{"1min":0.499,"5min":2.1200000000000001,"15min":7.593},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.433,"15min":0.443},"min":{"1min":0.28199999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52300000000000002,"5min":2.4820000000000002,"15min":8.4239999999999995},"last":0.39200000000000002}]},{"osd":1,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41199999999999998,"15min":0.40999999999999998},"min":{"1min":0.28999999999999998,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.45000000000000001,"15min":0.45300000000000001},"min":{"1min":0.33900000000000002,"5min":0.32800000000000001,"15min":0.221},"max":{"1min":0.627,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.34300000000000003}]},{"osd":2,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.76300000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.38500000000000001,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.70699999999999996,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94299999999999995,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.437},"max":{"1min":0.84199999999999997,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.61899999999999999}]},{"osd":5,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.68799999999999994},"min":{"1min":0.47699999999999998,"5min":0.42799999999999999,"15min":0.39900000000000002},"max":{"1min":0.90400000000000003,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.41899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1,"5min":1,"15min":1.0429999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68300000000000005,"15min":0.68999999999999995},"min":{"1min":0.51300000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0409999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":0.99299999999999999,"5min":2.6110000000000002,"15min":3.1800000000000002},"last":0.495}]},{"osd":7,"last update":"Wed Jan 24 14:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":2.4220000000000002,"15min":3.0059999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.73599999999999999},"min":{"1min":0.504,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0640000000000001,"5min":2.556,"15min":2.9929999999999999},"last":0.51700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575077,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303056,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068784,"statfs":{"total":96636764160,"available":96326434816,"internally_reserved":0,"allocated":1908736,"data_stored":1077545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.58499999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73199999999999998,"5min":0.83199999999999996,"15min":0.88600000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.59799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.40699999999999997}]},{"osd":1,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58299999999999996,"15min":0.59899999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66600000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57699999999999996,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.36299999999999999},"max":{"1min":0.71799999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.45000000000000001}]},{"osd":2,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.84199999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59099999999999997,"15min":0.60599999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.57999999999999996}]},{"osd":3,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.80600000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60299999999999998,"15min":0.61299999999999999},"min":{"1min":0.42299999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76600000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.49199999999999999}]},{"osd":5,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.27300000000000002},"max":{"1min":0.64800000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.495},"min":{"1min":0.32400000000000001,"5min":0.30499999999999999,"15min":0.29199999999999998},"max":{"1min":0.67100000000000004,"5min":0.71299999999999997,"15min":0.72899999999999998},"last":0.621}]},{"osd":6,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50800000000000001,"15min":0.49299999999999999},"min":{"1min":0.379,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64400000000000002,"5min":0.79700000000000004,"15min":0.80700000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.36799999999999999,"5min":0.35999999999999999,"15min":0.33200000000000002},"max":{"1min":0.77800000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.63500000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.40699999999999997,"5min":0.33800000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.74299999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575077,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303056,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068784,"statfs":{"total":96636764160,"available":96326434816,"internally_reserved":0,"allocated":1908736,"data_stored":1076211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53400000000000003,"15min":0.56699999999999995},"min":{"1min":0.36499999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":1.0800000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56999999999999995,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.309,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.54700000000000004}]},{"osd":1,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.57699999999999996},"min":{"1min":0.41299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.84099999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.59499999999999997},"min":{"1min":0.44500000000000001,"5min":0.35199999999999998,"15min":0.29799999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":1.1639999999999999},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.437,"5min":0.39100000000000001,"15min":0.33900000000000002},"max":{"1min":0.70799999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.58899999999999997},"min":{"1min":0.38200000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83699999999999997},"last":0.626}]},{"osd":3,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":0.60599999999999998},"min":{"1min":0.45500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":1.139},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.374},"max":{"1min":0.69999999999999996,"5min":0.80800000000000005,"15min":0.85399999999999998},"last":0.59499999999999997}]},{"osd":4,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.35799999999999998,"5min":0.33900000000000002,"15min":0.30299999999999999},"max":{"1min":0.71399999999999997,"5min":0.76800000000000002,"15min":0.89400000000000002},"last":0.505},{"interface":"front","average":{"1min":0.503,"5min":0.496,"15min":0.501},"min":{"1min":0.34300000000000003,"5min":0.32200000000000001,"15min":0.30399999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.90000000000000002},"last":0.57399999999999995}]},{"osd":6,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.253},"max":{"1min":0.72199999999999998,"5min":0.79500000000000004,"15min":1.1839999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.32000000000000001},"max":{"1min":0.747,"5min":0.80700000000000005,"15min":0.92800000000000005},"last":0.63800000000000001}]},{"osd":7,"last update":"Wed Jan 24 14:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52400000000000002,"15min":0.53400000000000003},"min":{"1min":0.371,"5min":0.34200000000000003,"15min":0.33100000000000002},"max":{"1min":0.746,"5min":0.81999999999999995,"15min":0.93700000000000006},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.76800000000000002,"5min":0.83999999999999997,"15min":1.2030000000000001},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T14:17:24.922 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-24T14:17:24.923 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-24T14:17:25.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.099+0000 7fcf7b50d700 1 Processor -- start 2024-01-24T14:17:25.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.099+0000 7fcf7b50d700 1 -- start start 2024-01-24T14:17:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.100+0000 7fcf7b50d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7413dcc0 0x7fcf74065a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.100+0000 7fcf7b50d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcf74069280 0x7fcf74157df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.100+0000 7fcf7b50d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf740655c0 con 0x7fcf7413e0a0 2024-01-24T14:17:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.100+0000 7fcf7b50d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf74158b60 con 0x7fcf74069280 2024-01-24T14:17:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.100+0000 7fcf7b50d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74064c90 con 0x7fcf7413dcc0 2024-01-24T14:17:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.100+0000 7fcf78aa8700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcf7413e0a0 0x7fcf74068b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41672/0 (socket says 172.21.15.29:41672) 2024-01-24T14:17:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.100+0000 7fcf78aa8700 1 -- 172.21.15.29:0/3901492998 learned_addr learned my addr 172.21.15.29:0/3901492998 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf792a9700 1 --2- 172.21.15.29:0/3901492998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7413dcc0 0x7fcf74065a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf79aaa700 1 --2- 172.21.15.29:0/3901492998 >> v2:172.21.15.163:3300/0 conn(0x7fcf74069280 0x7fcf74157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf6bfff700 1 -- 172.21.15.29:0/3901492998 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304492233 0 0) 0x7fcf740655c0 con 0x7fcf7413e0a0 2024-01-24T14:17:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf6bfff700 1 -- 172.21.15.29:0/3901492998 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf600033f0 con 0x7fcf7413e0a0 2024-01-24T14:17:25.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf79aaa700 1 -- 172.21.15.29:0/3901492998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7413dcc0 msgr2=0x7fcf74065a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf79aaa700 1 --2- 172.21.15.29:0/3901492998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7413dcc0 0x7fcf74065a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf79aaa700 1 -- 172.21.15.29:0/3901492998 >> v1:172.21.15.29:6789/0 conn(0x7fcf7413e0a0 legacy=0x7fcf74068b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf6bfff700 1 -- 172.21.15.29:0/3901492998 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2807307341 0 0) 0x7fcf600033f0 con 0x7fcf7413e0a0 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf79aaa700 1 -- 172.21.15.29:0/3901492998 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf74064f70 con 0x7fcf74069280 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf79aaa700 1 --2- 172.21.15.29:0/3901492998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf74069280 0x7fcf74157df0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fcf6c009e60 tx=0x7fcf6c009780).ready entity=mon.1 client_cookie=1eea720f68234858 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf6bfff700 1 -- 172.21.15.29:0/3901492998 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00e070 con 0x7fcf74069280 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf6bfff700 1 -- 172.21.15.29:0/3901492998 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c00b7a0 con 0x7fcf74069280 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.101+0000 7fcf6bfff700 1 -- 172.21.15.29:0/3901492998 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64001470 con 0x7fcf74069280 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.102+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3901492998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf74069280 msgr2=0x7fcf74157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.102+0000 7fcf7b50d700 1 --2- 172.21.15.29:0/3901492998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf74069280 0x7fcf74157df0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fcf6c009e60 tx=0x7fcf6c009780).stop 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.102+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3901492998 shutdown_connections 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.102+0000 7fcf7b50d700 1 --2- 172.21.15.29:0/3901492998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf74069280 0x7fcf74157df0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.102+0000 7fcf7b50d700 1 --2- 172.21.15.29:0/3901492998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7413dcc0 0x7fcf74065a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.102+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3901492998 >> 172.21.15.29:0/3901492998 conn(0x7fcf740c7350 msgr2=0x7fcf740c7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:25.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.102+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3901492998 shutdown_connections 2024-01-24T14:17:25.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.102+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3901492998 wait complete. 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.103+0000 7fcf7b50d700 1 Processor -- start 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.103+0000 7fcf7b50d700 1 -- start start 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.104+0000 7fcf7b50d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74069280 0x7fcf740c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.104+0000 7fcf7b50d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7413dcc0 0x7fcf741598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.104+0000 7fcf7b50d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7413e0a0 0x7fcf741636b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.104+0000 7fcf7b50d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740c3f10 con 0x7fcf74069280 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.104+0000 7fcf7b50d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74159de0 con 0x7fcf7413e0a0 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.104+0000 7fcf7b50d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740c35e0 con 0x7fcf7413dcc0 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.104+0000 7fcf792a9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74069280 0x7fcf740c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.104+0000 7fcf792a9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74069280 0x7fcf740c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56438/0 (socket says 172.21.15.29:56438) 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.104+0000 7fcf792a9700 1 -- 172.21.15.29:0/3759413972 learned_addr learned my addr 172.21.15.29:0/3759413972 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf78aa8700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7413dcc0 0x7fcf741598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf79aaa700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7413e0a0 0x7fcf741636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf792a9700 1 -- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7413dcc0 msgr2=0x7fcf741598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf792a9700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7413dcc0 0x7fcf741598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf792a9700 1 -- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7413e0a0 msgr2=0x7fcf741636b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf792a9700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7413e0a0 0x7fcf741636b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf792a9700 1 -- 172.21.15.29:0/3759413972 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c009a20 con 0x7fcf74069280 2024-01-24T14:17:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf792a9700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74069280 0x7fcf740c2f20 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7fcf7000e930 tx=0x7fcf70009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf697fa700 1 -- 172.21.15.29:0/3759413972 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70012600 con 0x7fcf74069280 2024-01-24T14:17:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf697fa700 1 -- 172.21.15.29:0/3759413972 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf70012760 con 0x7fcf74069280 2024-01-24T14:17:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3759413972 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf740c38c0 con 0x7fcf74069280 2024-01-24T14:17:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf697fa700 1 -- 172.21.15.29:0/3759413972 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70012ae0 con 0x7fcf74069280 2024-01-24T14:17:25.108 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.105+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3759413972 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf740ab000 con 0x7fcf74069280 2024-01-24T14:17:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.106+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3759413972 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf740c4b50 con 0x7fcf74069280 2024-01-24T14:17:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.107+0000 7fcf697fa700 1 -- 172.21.15.29:0/3759413972 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcf7001b400 con 0x7fcf74069280 2024-01-24T14:17:25.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.107+0000 7fcf697fa700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcf5805f270 0x7fcf58061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.107+0000 7fcf697fa700 1 -- 172.21.15.29:0/3759413972 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fcf7007ac20 con 0x7fcf74069280 2024-01-24T14:17:25.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.107+0000 7fcf78aa8700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcf5805f270 0x7fcf58061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.108+0000 7fcf78aa8700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcf5805f270 0x7fcf58061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fcf7413cf00 tx=0x7fcf640018a0).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:25.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.111+0000 7fcf697fa700 1 -- 172.21.15.29:0/3759413972 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcf7004b910 con 0x7fcf74069280 2024-01-24T14:17:25.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.364+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3759413972 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcf740ab3e0 con 0x7fcf74069280 2024-01-24T14:17:25.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.368+0000 7fcf697fa700 1 -- 172.21.15.29:0/3759413972 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+0 (secure 0 0 0) 0x7fcf7004ef30 con 0x7fcf74069280 2024-01-24T14:17:25.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.370+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcf5805f270 msgr2=0x7fcf58061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:25.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.370+0000 7fcf7b50d700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcf5805f270 0x7fcf58061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fcf7413cf00 tx=0x7fcf640018a0).stop 2024-01-24T14:17:25.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.370+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3759413972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74069280 msgr2=0x7fcf740c2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:25.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.370+0000 7fcf7b50d700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74069280 0x7fcf740c2f20 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7fcf7000e930 tx=0x7fcf70009060).stop 2024-01-24T14:17:25.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.370+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3759413972 shutdown_connections 2024-01-24T14:17:25.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.370+0000 7fcf7b50d700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7413e0a0 0x7fcf741636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.370+0000 7fcf7b50d700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcf5805f270 0x7fcf58061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.370+0000 7fcf7b50d700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74069280 0x7fcf740c2f20 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.370+0000 7fcf7b50d700 1 --2- 172.21.15.29:0/3759413972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7413dcc0 0x7fcf741598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.370+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3759413972 >> 172.21.15.29:0/3759413972 conn(0x7fcf740c7350 msgr2=0x7fcf740627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:25.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.370+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3759413972 shutdown_connections 2024-01-24T14:17:25.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.370+0000 7fcf7b50d700 1 -- 172.21.15.29:0/3759413972 wait complete. 2024-01-24T14:17:25.372 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-01-24T14:17:25.384 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T14:17:25.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.515+0000 7f35bf18b700 1 Processor -- start 2024-01-24T14:17:25.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.515+0000 7f35bf18b700 1 -- start start 2024-01-24T14:17:25.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.516+0000 7f35bf18b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b813a5c0 0x7f35b8063a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.516+0000 7f35bf18b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35b80be520 0x7f35b80664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.516+0000 7f35bf18b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35b80669e0 0x7f35b8067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.516+0000 7f35bf18b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b8066e90 con 0x7f35b80be520 2024-01-24T14:17:25.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.516+0000 7f35bf18b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b80bf6a0 con 0x7f35b80669e0 2024-01-24T14:17:25.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.516+0000 7f35bf18b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80bea90 con 0x7f35b813a5c0 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35b7fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35b80be520 0x7f35b80664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35b7fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35b80be520 0x7f35b80664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56452/0 (socket says 172.21.15.29:56452) 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35b7fff700 1 -- 172.21.15.29:0/3439885928 learned_addr learned my addr 172.21.15.29:0/3439885928 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35bd728700 1 --2- 172.21.15.29:0/3439885928 >> v2:172.21.15.163:3300/0 conn(0x7f35b80669e0 0x7f35b8067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35bcf27700 1 --2- 172.21.15.29:0/3439885928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b813a5c0 0x7f35b8063a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35b7fff700 1 -- 172.21.15.29:0/3439885928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b813a5c0 msgr2=0x7f35b8063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35b7fff700 1 --2- 172.21.15.29:0/3439885928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b813a5c0 0x7f35b8063a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35b7fff700 1 -- 172.21.15.29:0/3439885928 >> v2:172.21.15.163:3300/0 conn(0x7f35b80669e0 msgr2=0x7f35b8067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35b7fff700 1 --2- 172.21.15.29:0/3439885928 >> v2:172.21.15.163:3300/0 conn(0x7f35b80669e0 0x7f35b8067a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35b7fff700 1 -- 172.21.15.29:0/3439885928 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b80bed70 con 0x7f35b80be520 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35bd728700 1 --2- 172.21.15.29:0/3439885928 >> v2:172.21.15.163:3300/0 conn(0x7f35b80669e0 0x7f35b8067a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35b7fff700 1 --2- 172.21.15.29:0/3439885928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35b80be520 0x7f35b80664f0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f35b80c5620 tx=0x7f35a40090f0).ready entity=mon.0 client_cookie=696a35b75231e045 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35b77fe700 1 -- 172.21.15.29:0/3439885928 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a400b600 con 0x7f35b80be520 2024-01-24T14:17:25.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35b77fe700 1 -- 172.21.15.29:0/3439885928 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a400b760 con 0x7f35b80be520 2024-01-24T14:17:25.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.517+0000 7f35b77fe700 1 -- 172.21.15.29:0/3439885928 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a400bae0 con 0x7f35b80be520 2024-01-24T14:17:25.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.518+0000 7f35bf18b700 1 -- 172.21.15.29:0/3439885928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35b80be520 msgr2=0x7f35b80664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:25.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.518+0000 7f35bf18b700 1 --2- 172.21.15.29:0/3439885928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35b80be520 0x7f35b80664f0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f35b80c5620 tx=0x7f35a40090f0).stop 2024-01-24T14:17:25.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.518+0000 7f35bf18b700 1 -- 172.21.15.29:0/3439885928 shutdown_connections 2024-01-24T14:17:25.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.518+0000 7f35bf18b700 1 --2- 172.21.15.29:0/3439885928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b813a5c0 0x7f35b8063a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.518+0000 7f35bf18b700 1 --2- 172.21.15.29:0/3439885928 >> v2:172.21.15.163:3300/0 conn(0x7f35b80669e0 0x7f35b8067a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.518+0000 7f35bf18b700 1 --2- 172.21.15.29:0/3439885928 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35b80be520 0x7f35b80664f0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.518+0000 7f35bf18b700 1 -- 172.21.15.29:0/3439885928 >> 172.21.15.29:0/3439885928 conn(0x7f35b80c8bb0 msgr2=0x7f35b80c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:25.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.518+0000 7f35bf18b700 1 -- 172.21.15.29:0/3439885928 shutdown_connections 2024-01-24T14:17:25.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.518+0000 7f35bf18b700 1 -- 172.21.15.29:0/3439885928 wait complete. 2024-01-24T14:17:25.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.519+0000 7f35bf18b700 1 Processor -- start 2024-01-24T14:17:25.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.519+0000 7f35bf18b700 1 -- start start 2024-01-24T14:17:25.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.520+0000 7f35bf18b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35b80669e0 0x7f35b8138910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.520+0000 7f35bf18b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b80be520 0x7f35b80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.520+0000 7f35bf18b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b813a5c0 0x7f35b80a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.520+0000 7f35bf18b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80a6ad0 con 0x7f35b80669e0 2024-01-24T14:17:25.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.520+0000 7f35bf18b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b8168630 con 0x7f35b80be520 2024-01-24T14:17:25.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.520+0000 7f35bf18b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b8167a20 con 0x7f35b813a5c0 2024-01-24T14:17:25.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.520+0000 7f35bcf27700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35b80669e0 0x7f35b8138910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.520+0000 7f35bcf27700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35b80669e0 0x7f35b8138910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56454/0 (socket says 172.21.15.29:56454) 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.520+0000 7f35bcf27700 1 -- 172.21.15.29:0/3546276371 learned_addr learned my addr 172.21.15.29:0/3546276371 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35bd728700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b813a5c0 0x7f35b80a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35b7fff700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b80be520 0x7f35b80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35bcf27700 1 -- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b813a5c0 msgr2=0x7f35b80a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35bcf27700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b813a5c0 0x7f35b80a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35bcf27700 1 -- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b80be520 msgr2=0x7f35b80a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35bcf27700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b80be520 0x7f35b80a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35bcf27700 1 -- 172.21.15.29:0/3546276371 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a4009370 con 0x7f35b80669e0 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35bd728700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b813a5c0 0x7f35b80a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35b7fff700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b80be520 0x7f35b80a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35bcf27700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35b80669e0 0x7f35b8138910 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f35ac00c850 tx=0x7f35ac008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35b4ff9700 1 -- 172.21.15.29:0/3546276371 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac010580 con 0x7f35b80669e0 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35b4ff9700 1 -- 172.21.15.29:0/3546276371 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ac0106e0 con 0x7f35b80669e0 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35bf18b700 1 -- 172.21.15.29:0/3546276371 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b8167d00 con 0x7f35b80669e0 2024-01-24T14:17:25.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35b4ff9700 1 -- 172.21.15.29:0/3546276371 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac010a60 con 0x7f35b80669e0 2024-01-24T14:17:25.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.521+0000 7f35bf18b700 1 -- 172.21.15.29:0/3546276371 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b81682c0 con 0x7f35b80669e0 2024-01-24T14:17:25.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.522+0000 7f35bf18b700 1 -- 172.21.15.29:0/3546276371 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b80475e0 con 0x7f35b80669e0 2024-01-24T14:17:25.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.523+0000 7f35b4ff9700 1 -- 172.21.15.29:0/3546276371 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f35ac01b460 con 0x7f35b80669e0 2024-01-24T14:17:25.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.523+0000 7f35b4ff9700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f359c05f270 0x7f359c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:25.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.523+0000 7f35b4ff9700 1 -- 172.21.15.29:0/3546276371 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f35ac07a730 con 0x7f35b80669e0 2024-01-24T14:17:25.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.523+0000 7f35b7fff700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f359c05f270 0x7f359c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:25.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.524+0000 7f35b7fff700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f359c05f270 0x7f359c061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f35b80c5620 tx=0x7f35a4017000).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:25.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.527+0000 7f35b4ff9700 1 -- 172.21.15.29:0/3546276371 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f35ac04b420 con 0x7f35b80669e0 2024-01-24T14:17:25.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.754+0000 7f35bf18b700 1 -- 172.21.15.29:0/3546276371 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35b80bf050 con 0x7f35b80669e0 2024-01-24T14:17:25.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.754+0000 7f35b4ff9700 1 -- 172.21.15.29:0/3546276371 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f35ac04ea40 con 0x7f35b80669e0 2024-01-24T14:17:25.755 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:25.755 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2106677945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2106677945},{"type":"v1","addr":"172.21.15.163:6836","nonce":2106677945}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7786,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1964541525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1964541525},{"type":"v1","addr":"172.21.15.163:6837","nonce":1964541525}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4162831288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4162831288},{"type":"v1","addr":"172.21.15.29:6835","nonce":4162831288}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:17:20.414311+0000","modified":"2024-01-24T14:17:23.419003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8166":{"gid":8166,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/1312733772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1312733772},{"type":"v1","addr":"172.21.15.29:6834","nonce":1312733772}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-24T14:17:25.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.756+0000 7f35bf18b700 1 -- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f359c05f270 msgr2=0x7f359c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.756+0000 7f35bf18b700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f359c05f270 0x7f359c061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f35b80c5620 tx=0x7f35a4017000).stop 2024-01-24T14:17:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.756+0000 7f35bf18b700 1 -- 172.21.15.29:0/3546276371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35b80669e0 msgr2=0x7f35b8138910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.756+0000 7f35bf18b700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35b80669e0 0x7f35b8138910 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f35ac00c850 tx=0x7f35ac008040).stop 2024-01-24T14:17:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.756+0000 7f35bf18b700 1 -- 172.21.15.29:0/3546276371 shutdown_connections 2024-01-24T14:17:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.756+0000 7f35bf18b700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b813a5c0 0x7f35b80a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.756+0000 7f35bf18b700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f359c05f270 0x7f359c061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.756+0000 7f35bf18b700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35b80669e0 0x7f35b8138910 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.756+0000 7f35bf18b700 1 --2- 172.21.15.29:0/3546276371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b80be520 0x7f35b80a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.757+0000 7f35bf18b700 1 -- 172.21.15.29:0/3546276371 >> 172.21.15.29:0/3546276371 conn(0x7f35b80c8bb0 msgr2=0x7f35b80c67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.757+0000 7f35bf18b700 1 -- 172.21.15.29:0/3546276371 shutdown_connections 2024-01-24T14:17:25.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:25.757+0000 7f35bf18b700 1 -- 172.21.15.29:0/3546276371 wait complete. 2024-01-24T14:17:25.759 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 66 2024-01-24T14:17:26.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T14:17:20.414311+0000', 'modified': '2024-01-24T14:17:23.419003+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8166}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8166': {'gid': 8166, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6834/1312733772', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6832', 'nonce': 1312733772}, {'type': 'v1', 'addr': '172.21.15.29:6834', 'nonce': 1312733772}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-24T14:17:26.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T14:17:26.174 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-24T14:17:26.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.312+0000 7f7f38db4700 1 Processor -- start 2024-01-24T14:17:26.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.312+0000 7f7f38db4700 1 -- start start 2024-01-24T14:17:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.313+0000 7f7f38db4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7f3414b020 0x7f7f340c2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.313+0000 7f7f38db4700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f340c2970 con 0x7f7f3414ac40 2024-01-24T14:17:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.313+0000 7f7f38db4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f340c2c50 con 0x7f7f340691b0 2024-01-24T14:17:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.313+0000 7f7f38db4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f340c2f30 con 0x7f7f3414b020 2024-01-24T14:17:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.314+0000 7f7f32d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7f3414b020 0x7f7f340c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.314+0000 7f7f3259c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7f3414ac40 0x7f7f34068aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41678/0 (socket says 172.21.15.29:41678) 2024-01-24T14:17:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.314+0000 7f7f3259c700 1 -- 172.21.15.29:0/2541977762 learned_addr learned my addr 172.21.15.29:0/2541977762 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:26.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.314+0000 7f7f32d9d700 1 -- 172.21.15.29:0/2541977762 >> v1:172.21.15.163:6789/0 conn(0x7f7f340691b0 legacy=0x7f7f340c1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.314+0000 7f7f32d9d700 1 -- 172.21.15.29:0/2541977762 >> v1:172.21.15.29:6789/0 conn(0x7f7f3414ac40 legacy=0x7f7f34068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.314+0000 7f7f3159a700 1 -- 172.21.15.29:0/2541977762 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269210695 0 0) 0x7f7f340c2970 con 0x7f7f3414ac40 2024-01-24T14:17:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.314+0000 7f7f32d9d700 1 -- 172.21.15.29:0/2541977762 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f340c3210 con 0x7f7f3414b020 2024-01-24T14:17:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.314+0000 7f7f32d9d700 1 --2- 172.21.15.29:0/2541977762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f3414b020 0x7f7f340c2350 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f7f2c009e60 tx=0x7f7f2c009780).ready entity=mon.0 client_cookie=41dde9b318bd70e7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.314+0000 7f7f3159a700 1 -- 172.21.15.29:0/2541977762 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c00e070 con 0x7f7f3414b020 2024-01-24T14:17:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.314+0000 7f7f3159a700 1 -- 172.21.15.29:0/2541977762 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c00bc30 con 0x7f7f3414b020 2024-01-24T14:17:26.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.314+0000 7f7f3159a700 1 -- 172.21.15.29:0/2541977762 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c012410 con 0x7f7f3414b020 2024-01-24T14:17:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.315+0000 7f7f38db4700 1 -- 172.21.15.29:0/2541977762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f3414b020 msgr2=0x7f7f340c2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.315+0000 7f7f38db4700 1 --2- 172.21.15.29:0/2541977762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f3414b020 0x7f7f340c2350 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f7f2c009e60 tx=0x7f7f2c009780).stop 2024-01-24T14:17:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.315+0000 7f7f38db4700 1 -- 172.21.15.29:0/2541977762 shutdown_connections 2024-01-24T14:17:26.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.315+0000 7f7f38db4700 1 --2- 172.21.15.29:0/2541977762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f3414b020 0x7f7f340c2350 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.315+0000 7f7f38db4700 1 -- 172.21.15.29:0/2541977762 >> 172.21.15.29:0/2541977762 conn(0x7f7f340c7480 msgr2=0x7f7f340c3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:26.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.315+0000 7f7f38db4700 1 -- 172.21.15.29:0/2541977762 shutdown_connections 2024-01-24T14:17:26.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.315+0000 7f7f38db4700 1 -- 172.21.15.29:0/2541977762 wait complete. 2024-01-24T14:17:26.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.316+0000 7f7f38db4700 1 Processor -- start 2024-01-24T14:17:26.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.316+0000 7f7f38db4700 1 -- start start 2024-01-24T14:17:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.317+0000 7f7f38db4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f340691b0 0x7f7f34140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.317+0000 7f7f38db4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f3414ac40 0x7f7f3413f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.317+0000 7f7f38db4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f3414b020 0x7f7f340bbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.317+0000 7f7f38db4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f34141cf0 con 0x7f7f340691b0 2024-01-24T14:17:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.317+0000 7f7f38db4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f340a9a70 con 0x7f7f3414ac40 2024-01-24T14:17:26.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.317+0000 7f7f38db4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f34141450 con 0x7f7f3414b020 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.317+0000 7f7f3259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f340691b0 0x7f7f34140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.317+0000 7f7f3259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f340691b0 0x7f7f34140ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56468/0 (socket says 172.21.15.29:56468) 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.317+0000 7f7f3259c700 1 -- 172.21.15.29:0/1147915176 learned_addr learned my addr 172.21.15.29:0/1147915176 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f31d9b700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f3414ac40 0x7f7f3413f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f32d9d700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f3414b020 0x7f7f340bbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f3259c700 1 -- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f3414b020 msgr2=0x7f7f340bbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f3259c700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f3414b020 0x7f7f340bbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f3259c700 1 -- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f3414ac40 msgr2=0x7f7f3413f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f3259c700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f3414ac40 0x7f7f3413f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f3259c700 1 -- 172.21.15.29:0/1147915176 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c009a20 con 0x7f7f340691b0 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f3259c700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f340691b0 0x7f7f34140ae0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f7f1c00c4e0 tx=0x7f7f1c007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f2a7fc700 1 -- 172.21.15.29:0/1147915176 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00ce90 con 0x7f7f340691b0 2024-01-24T14:17:26.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f2a7fc700 1 -- 172.21.15.29:0/1147915176 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c0072e0 con 0x7f7f340691b0 2024-01-24T14:17:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f38db4700 1 -- 172.21.15.29:0/1147915176 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f34141730 con 0x7f7f340691b0 2024-01-24T14:17:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f2a7fc700 1 -- 172.21.15.29:0/1147915176 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c007660 con 0x7f7f340691b0 2024-01-24T14:17:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.318+0000 7f7f38db4700 1 -- 172.21.15.29:0/1147915176 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f34140fd0 con 0x7f7f340691b0 2024-01-24T14:17:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.320+0000 7f7f2a7fc700 1 -- 172.21.15.29:0/1147915176 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7f1c0059b0 con 0x7f7f340691b0 2024-01-24T14:17:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.320+0000 7f7f2a7fc700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7f1405f270 0x7f7f14061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.320+0000 7f7f2a7fc700 1 -- 172.21.15.29:0/1147915176 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f7f1c079e90 con 0x7f7f340691b0 2024-01-24T14:17:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.320+0000 7f7f31d9b700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7f1405f270 0x7f7f14061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.321+0000 7f7f31d9b700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7f1405f270 0x7f7f14061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f7f34062bf0 tx=0x7f7f24007d40).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.322+0000 7f7f4a77fb80 1 Processor -- start 2024-01-24T14:17:26.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.323+0000 7f7f4a77fb80 1 -- start start 2024-01-24T14:17:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f4a77fb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c57e620 0x55798c57ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f4a77fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c5c72c0 0x55798c579900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f4a77fb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 0x55798c57a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f4a77fb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798c4f53d0 con 0x55798c579df0 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f4a77fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798c4babe0 con 0x55798c5c72c0 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f4a77fb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55798c4d0260 con 0x55798c57e620 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f32d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 0x55798c57a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f32d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 0x55798c57a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56478/0 (socket says 172.21.15.29:56478) 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f32d9d700 1 -- 172.21.15.29:0/269016982 learned_addr learned my addr 172.21.15.29:0/269016982 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f3259c700 1 --2- 172.21.15.29:0/269016982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c57e620 0x55798c57ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f31d9b700 1 --2- 172.21.15.29:0/269016982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c5c72c0 0x55798c579900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f32d9d700 1 -- 172.21.15.29:0/269016982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c57e620 msgr2=0x55798c57ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f32d9d700 1 --2- 172.21.15.29:0/269016982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c57e620 0x55798c57ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f32d9d700 1 -- 172.21.15.29:0/269016982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c5c72c0 msgr2=0x55798c579900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f32d9d700 1 --2- 172.21.15.29:0/269016982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c5c72c0 0x55798c579900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f32d9d700 1 -- 172.21.15.29:0/269016982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798c4c92f0 con 0x55798c579df0 2024-01-24T14:17:26.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.324+0000 7f7f32d9d700 1 --2- 172.21.15.29:0/269016982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 0x55798c57a270 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f7f2c00b6c0 tx=0x7f7f2c009780).ready entity=mon.0 client_cookie=af47334819d56bcb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.325+0000 7f7f137fe700 1 -- 172.21.15.29:0/269016982 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c012780 con 0x55798c579df0 2024-01-24T14:17:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.325+0000 7f7f137fe700 1 -- 172.21.15.29:0/269016982 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c0128e0 con 0x55798c579df0 2024-01-24T14:17:26.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.325+0000 7f7f137fe700 1 -- 172.21.15.29:0/269016982 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c012c80 con 0x55798c579df0 2024-01-24T14:17:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.325+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/269016982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 msgr2=0x55798c57a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.325+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/269016982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 0x55798c57a270 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f7f2c00b6c0 tx=0x7f7f2c009780).stop 2024-01-24T14:17:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.325+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/269016982 shutdown_connections 2024-01-24T14:17:26.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.325+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/269016982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 0x55798c57a270 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.325+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/269016982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c57e620 0x55798c57ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.325+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/269016982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c5c72c0 0x55798c579900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.325+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/269016982 >> 172.21.15.29:0/269016982 conn(0x55798c3e72e0 msgr2=0x55798c5c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.325+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/269016982 shutdown_connections 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.325+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/269016982 wait complete. 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.326+0000 7f7f4a77fb80 1 -- start start 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.326+0000 7f7f4a77fb80 1 Processor -- start 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.327+0000 7f7f4a77fb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 0x55798c5aa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.327+0000 7f7f4a77fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c57e620 0x55798c5aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.327+0000 7f7f4a77fb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c5c72c0 0x55798c5ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.327+0000 7f7f4a77fb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798c4d03a0 con 0x55798c579df0 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.327+0000 7f7f4a77fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798c4d0620 con 0x55798c57e620 2024-01-24T14:17:26.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.327+0000 7f7f4a77fb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55798c49b990 con 0x55798c5c72c0 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.327+0000 7f7f4a77fb80 10 client.0 ll_register_callbacks cb 0x55798c53e610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.327+0000 7f7f3259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 0x55798c5aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.327+0000 7f7f3259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 0x55798c5aa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56494/0 (socket says 172.21.15.29:56494) 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.327+0000 7f7f3259c700 1 -- 172.21.15.29:0/1298025305 learned_addr learned my addr 172.21.15.29:0/1298025305 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.327+0000 7f7f31d9b700 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c57e620 0x55798c5aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f32d9d700 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c5c72c0 0x55798c5ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f3259c700 1 -- 172.21.15.29:0/1298025305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c5c72c0 msgr2=0x55798c5ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f3259c700 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c5c72c0 0x55798c5ab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f3259c700 1 -- 172.21.15.29:0/1298025305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c57e620 msgr2=0x55798c5aabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f3259c700 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c57e620 0x55798c5aabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f3259c700 1 -- 172.21.15.29:0/1298025305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f340c3210 con 0x55798c579df0 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f3259c700 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 0x55798c5aa700 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f7f1c077ea0 tx=0x7f7f1c050040).ready entity=mon.0 client_cookie=14b503961d292240 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:26.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f10ff9700 1 -- 172.21.15.29:0/1298025305 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c005c60 con 0x55798c579df0 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f10ff9700 1 -- 172.21.15.29:0/1298025305 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f7f1c005dc0 con 0x55798c579df0 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/1298025305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55798c578df0 con 0x55798c579df0 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f10ff9700 1 -- 172.21.15.29:0/1298025305 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f1c01c560 con 0x55798c579df0 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f4a77fb80 10 client.8298 fetch_fsmap learned FSMap version 66 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/1298025305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x55798c4c4520 con 0x55798c579df0 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.328+0000 7f7f10ff9700 1 -- 172.21.15.29:0/1298025305 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55798c578df0 con 0x55798c579df0 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.329+0000 7f7f10ff9700 1 -- 172.21.15.29:0/1298025305 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 66) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7f1c057070 con 0x55798c579df0 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.329+0000 7f7f4a77fb80 10 client.8298 fetch_fsmap finished waiting for FSMap version 66 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.329+0000 7f7f4a77fb80 10 client.8298 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.329+0000 7f7f4a77fb80 20 client.8298 populate_metadata read hostname 'smithi029' 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.329+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.29:6832/1312733772,v1:172.21.15.29:6834/1312733772] conn(0x55798c5d3a50 0x55798c5d3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.329+0000 7f7f4a77fb80 4 client.8298 mds_command: new command op to 8166 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T14:17:26.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.329+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/1298025305 --> [v2:172.21.15.29:6832/1312733772,v1:172.21.15.29:6834/1312733772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55798c4c5d60 con 0x55798c5d3a50 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.329+0000 7f7f31d9b700 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.29:6832/1312733772,v1:172.21.15.29:6834/1312733772] conn(0x55798c5d3a50 0x55798c5d3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.329+0000 7f7f10ff9700 10 client.8298 ms_handle_connect on v2:172.21.15.29:6832/1312733772 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.329+0000 7f7f31d9b700 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.29:6832/1312733772,v1:172.21.15.29:6834/1312733772] conn(0x55798c5d3a50 0x55798c5d3e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=9e0a4a3872f6cb7c server_cookie=a6d101102bf836ed in_seq=0 out_seq=0 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f10ff9700 1 -- 172.21.15.29:0/1298025305 <== mds.0 v2:172.21.15.29:6832/1312733772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x55798c4c5d60 con 0x55798c5d3a50 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f10ff9700 10 client.8298 handle_command_reply: tid=0 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f4a77fb80 1 client.8298 shutdown 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/1298025305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 msgr2=0x55798c5aa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 0x55798c5aa700 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f7f1c077ea0 tx=0x7f7f1c050040).stop 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/1298025305 shutdown_connections 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.29:6832/1312733772,v1:172.21.15.29:6834/1312733772] conn(0x55798c5d3a50 0x55798c5d3e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c5c72c0 0x55798c5ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c579df0 0x55798c5aa700 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/1298025305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c57e620 0x55798c5aabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f10ff9700 0 client.8298 ms_handle_reset on v2:172.21.15.29:6832/1312733772 2024-01-24T14:17:26.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/1298025305 >> 172.21.15.29:0/1298025305 conn(0x55798c3e72e0 msgr2=0x55798c5c3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:26.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/1298025305 shutdown_connections 2024-01-24T14:17:26.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/1298025305 wait complete. 2024-01-24T14:17:26.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.330+0000 7f7f4a77fb80 20 client.8298 trim_cache size 0 max 16384 2024-01-24T14:17:26.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.348+0000 7f7f4a77fb80 1 Processor -- start 2024-01-24T14:17:26.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.348+0000 7f7f4a77fb80 1 -- start start 2024-01-24T14:17:26.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f4a77fb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c579df0 0x55798c57ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f4a77fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c57e620 0x55798c5797c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f4a77fb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c5c72c0 0x55798c5d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f4a77fb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798c57e030 con 0x55798c5c72c0 2024-01-24T14:17:26.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f4a77fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798c5abeb0 con 0x55798c57e620 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f4a77fb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55798c5d27a0 con 0x55798c579df0 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f32d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c5c72c0 0x55798c5d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f32d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c5c72c0 0x55798c5d2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56504/0 (socket says 172.21.15.29:56504) 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f32d9d700 1 -- 172.21.15.29:0/2723437348 learned_addr learned my addr 172.21.15.29:0/2723437348 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f3259c700 1 --2- 172.21.15.29:0/2723437348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c579df0 0x55798c57ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f31d9b700 1 --2- 172.21.15.29:0/2723437348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c57e620 0x55798c5797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f32d9d700 1 -- 172.21.15.29:0/2723437348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c579df0 msgr2=0x55798c57ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f32d9d700 1 --2- 172.21.15.29:0/2723437348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c579df0 0x55798c57ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f32d9d700 1 -- 172.21.15.29:0/2723437348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c57e620 msgr2=0x55798c5797c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f32d9d700 1 --2- 172.21.15.29:0/2723437348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c57e620 0x55798c5797c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f32d9d700 1 -- 172.21.15.29:0/2723437348 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798c5c1940 con 0x55798c5c72c0 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f32d9d700 1 --2- 172.21.15.29:0/2723437348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c5c72c0 0x55798c5d2b70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7f7f2c009bf0 tx=0x7f7f2c01f000).ready entity=mon.0 client_cookie=2242393e4a6fec5c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f13fff700 1 -- 172.21.15.29:0/2723437348 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c02a070 con 0x55798c5c72c0 2024-01-24T14:17:26.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f13fff700 1 -- 172.21.15.29:0/2723437348 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c00be20 con 0x55798c5c72c0 2024-01-24T14:17:26.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.349+0000 7f7f13fff700 1 -- 172.21.15.29:0/2723437348 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c02d070 con 0x55798c5c72c0 2024-01-24T14:17:26.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.350+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/2723437348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c5c72c0 msgr2=0x55798c5d2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.350+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/2723437348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c5c72c0 0x55798c5d2b70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7f7f2c009bf0 tx=0x7f7f2c01f000).stop 2024-01-24T14:17:26.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.350+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/2723437348 shutdown_connections 2024-01-24T14:17:26.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.350+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/2723437348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c5c72c0 0x55798c5d2b70 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.350+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/2723437348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c579df0 0x55798c57ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.350+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/2723437348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c57e620 0x55798c5797c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.350+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/2723437348 >> 172.21.15.29:0/2723437348 conn(0x55798c3e72e0 msgr2=0x55798c5c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:26.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.350+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/2723437348 shutdown_connections 2024-01-24T14:17:26.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.350+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/2723437348 wait complete. 2024-01-24T14:17:26.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.351+0000 7f7f4a77fb80 1 -- start start 2024-01-24T14:17:26.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.351+0000 7f7f4a77fb80 1 Processor -- start 2024-01-24T14:17:26.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f4a77fb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c579df0 0x55798c57ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f4a77fb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c57e620 0x55798c5d6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f4a77fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c5c72c0 0x55798c62b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f4a77fb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798c5d2ec0 con 0x55798c57e620 2024-01-24T14:17:26.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f4a77fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798c5db8c0 con 0x55798c5c72c0 2024-01-24T14:17:26.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f4a77fb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55798c5db550 con 0x55798c579df0 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f4a77fb80 10 client.0 ll_register_callbacks cb 0x55798c53e610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f31d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c57e620 0x55798c5d6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f31d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c57e620 0x55798c5d6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56506/0 (socket says 172.21.15.29:56506) 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f31d9b700 1 -- 172.21.15.29:0/649219082 learned_addr learned my addr 172.21.15.29:0/649219082 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f3259c700 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c579df0 0x55798c57ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f32d9d700 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c5c72c0 0x55798c62b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f31d9b700 1 -- 172.21.15.29:0/649219082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c579df0 msgr2=0x55798c57ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f31d9b700 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c579df0 0x55798c57ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f31d9b700 1 -- 172.21.15.29:0/649219082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c5c72c0 msgr2=0x55798c62b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f31d9b700 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c5c72c0 0x55798c62b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f31d9b700 1 -- 172.21.15.29:0/649219082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55798c4c92f0 con 0x55798c57e620 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f32d9d700 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c5c72c0 0x55798c62b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f31d9b700 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c57e620 0x55798c5d6940 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x55798c580d90 tx=0x7f7f2401b040).ready entity=mon.0 client_cookie=e5c77820fe255a5f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.352+0000 7f7f10ff9700 1 -- 172.21.15.29:0/649219082 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f24029030 con 0x55798c57e620 2024-01-24T14:17:26.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f10ff9700 1 -- 172.21.15.29:0/649219082 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f7f2402a070 con 0x55798c57e620 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/649219082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f2c0128e0 con 0x55798c57e620 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f10ff9700 1 -- 172.21.15.29:0/649219082 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f24003040 con 0x55798c57e620 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f4a77fb80 10 client.8304 fetch_fsmap learned FSMap version 66 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/649219082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x55798c4c4270 con 0x55798c57e620 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f10ff9700 1 -- 172.21.15.29:0/649219082 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c0128e0 con 0x55798c57e620 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f10ff9700 1 -- 172.21.15.29:0/649219082 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 66) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7f24025760 con 0x55798c57e620 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f4a77fb80 10 client.8304 fetch_fsmap finished waiting for FSMap version 66 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f4a77fb80 10 client.8304 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f4a77fb80 20 client.8304 populate_metadata read hostname 'smithi029' 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.29:6832/1312733772,v1:172.21.15.29:6834/1312733772] conn(0x55798c5dbe70 0x55798c5dc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f4a77fb80 4 client.8304 mds_command: new command op to 8166 tid=0[{"prefix": "status"}] 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/649219082 --> [v2:172.21.15.29:6832/1312733772,v1:172.21.15.29:6834/1312733772] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55798c4bc7d0 con 0x55798c5dbe70 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f3259c700 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.29:6832/1312733772,v1:172.21.15.29:6834/1312733772] conn(0x55798c5dbe70 0x55798c5dc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f10ff9700 10 client.8304 ms_handle_connect on v2:172.21.15.29:6832/1312733772 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.353+0000 7f7f3259c700 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.29:6832/1312733772,v1:172.21.15.29:6834/1312733772] conn(0x55798c5dbe70 0x55798c5dc250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a288ab38eb221bbd server_cookie=973b7ad2270d9f05 in_seq=0 out_seq=0 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f10ff9700 1 -- 172.21.15.29:0/649219082 <== mds.0 v2:172.21.15.29:6832/1312733772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55798c4bc7d0 con 0x55798c5dbe70 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f10ff9700 10 client.8304 handle_command_reply: tid=0 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f4a77fb80 1 client.8304 shutdown 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/649219082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c57e620 msgr2=0x55798c5d6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c57e620 0x55798c5d6940 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x55798c580d90 tx=0x7f7f2401b040).stop 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/649219082 shutdown_connections 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.29:6832/1312733772,v1:172.21.15.29:6834/1312733772] conn(0x55798c5dbe70 0x55798c5dc250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55798c5c72c0 0x55798c62b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f10ff9700 0 client.8304 ms_handle_reset on v2:172.21.15.29:6832/1312733772 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55798c579df0 0x55798c57ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f4a77fb80 1 --2- 172.21.15.29:0/649219082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55798c57e620 0x55798c5d6940 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/649219082 >> 172.21.15.29:0/649219082 conn(0x55798c3e72e0 msgr2=0x55798c57c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:26.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/649219082 shutdown_connections 2024-01-24T14:17:26.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f4a77fb80 1 -- 172.21.15.29:0/649219082 wait complete. 2024-01-24T14:17:26.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.354+0000 7f7f4a77fb80 20 client.8304 trim_cache size 0 max 16384 2024-01-24T14:17:26.361 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T14:17:26.361 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T14:17:26.361 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T14:17:26.361 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 8166, 2024-01-24T14:17:26.361 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T14:17:26.361 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T14:17:26.361 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T14:17:26.361 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 5.9390128139999998, 2024-01-24T14:17:26.361 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 66, 2024-01-24T14:17:26.362 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 153, 2024-01-24T14:17:26.362 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 153, 2024-01-24T14:17:26.362 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 11.386431912000001 2024-01-24T14:17:26.362 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T14:17:26.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.355+0000 7f7f38db4700 1 -- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7f1405f270 msgr2=0x7f7f14061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.355+0000 7f7f38db4700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7f1405f270 0x7f7f14061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f7f34062bf0 tx=0x7f7f24007d40).stop 2024-01-24T14:17:26.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.355+0000 7f7f38db4700 1 -- 172.21.15.29:0/1147915176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f340691b0 msgr2=0x7f7f34140ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.355+0000 7f7f38db4700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f340691b0 0x7f7f34140ae0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f7f1c00c4e0 tx=0x7f7f1c007c70).stop 2024-01-24T14:17:26.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.355+0000 7f7f38db4700 1 -- 172.21.15.29:0/1147915176 shutdown_connections 2024-01-24T14:17:26.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.355+0000 7f7f38db4700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f3414b020 0x7f7f340bbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.355+0000 7f7f38db4700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7f1405f270 0x7f7f14061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.355+0000 7f7f38db4700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f340691b0 0x7f7f34140ae0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.355+0000 7f7f38db4700 1 --2- 172.21.15.29:0/1147915176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f3414ac40 0x7f7f3413f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.355+0000 7f7f38db4700 1 -- 172.21.15.29:0/1147915176 >> 172.21.15.29:0/1147915176 conn(0x7f7f340c7480 msgr2=0x7f7f340bcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:26.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.355+0000 7f7f38db4700 1 -- 172.21.15.29:0/1147915176 shutdown_connections 2024-01-24T14:17:26.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.355+0000 7f7f38db4700 1 -- 172.21.15.29:0/1147915176 wait complete. 2024-01-24T14:17:26.372 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T14:17:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.487+0000 7ff1875ba700 1 Processor -- start 2024-01-24T14:17:26.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.487+0000 7ff1875ba700 1 -- start start 2024-01-24T14:17:26.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.488+0000 7ff1875ba700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff18014e280 0x7ff18005e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.488+0000 7ff1875ba700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff18014e660 0x7ff1800bc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.488+0000 7ff1875ba700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff180063100 con 0x7ff180157c50 2024-01-24T14:17:26.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.488+0000 7ff1875ba700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1800bca20 con 0x7ff18014e280 2024-01-24T14:17:26.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.488+0000 7ff1875ba700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff180064120 con 0x7ff18014e660 2024-01-24T14:17:26.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.488+0000 7ff185356700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff18014e280 0x7ff18005e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.488+0000 7ff185356700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff18014e280 0x7ff18005e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56520/0 (socket says 172.21.15.29:56520) 2024-01-24T14:17:26.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.488+0000 7ff185356700 1 -- 172.21.15.29:0/1364972016 learned_addr learned my addr 172.21.15.29:0/1364972016 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:26.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.488+0000 7ff184b55700 1 --2- 172.21.15.29:0/1364972016 >> v2:172.21.15.163:3300/0 conn(0x7ff18014e660 0x7ff1800bc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.489+0000 7ff185356700 1 -- 172.21.15.29:0/1364972016 >> v2:172.21.15.163:3300/0 conn(0x7ff18014e660 msgr2=0x7ff1800bc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:26.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.489+0000 7ff185356700 1 --2- 172.21.15.29:0/1364972016 >> v2:172.21.15.163:3300/0 conn(0x7ff18014e660 0x7ff1800bc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.489+0000 7ff185356700 1 -- 172.21.15.29:0/1364972016 >> v1:172.21.15.163:6789/0 conn(0x7ff180157c50 legacy=0x7ff180154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.489+0000 7ff185356700 1 -- 172.21.15.29:0/1364972016 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff180063360 con 0x7ff18014e280 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.489+0000 7ff184b55700 1 --2- 172.21.15.29:0/1364972016 >> v2:172.21.15.163:3300/0 conn(0x7ff18014e660 0x7ff1800bc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.489+0000 7ff185356700 1 --2- 172.21.15.29:0/1364972016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff18014e280 0x7ff18005e470 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7ff180137800 tx=0x7ff17c0090f0).ready entity=mon.0 client_cookie=97cbdfe8b4855b9a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.489+0000 7ff177fff700 1 -- 172.21.15.29:0/1364972016 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00b600 con 0x7ff18014e280 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.489+0000 7ff177fff700 1 -- 172.21.15.29:0/1364972016 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c00b760 con 0x7ff18014e280 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.489+0000 7ff177fff700 1 -- 172.21.15.29:0/1364972016 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00bae0 con 0x7ff18014e280 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.490+0000 7ff1875ba700 1 -- 172.21.15.29:0/1364972016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff18014e280 msgr2=0x7ff18005e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.490+0000 7ff1875ba700 1 --2- 172.21.15.29:0/1364972016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff18014e280 0x7ff18005e470 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7ff180137800 tx=0x7ff17c0090f0).stop 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.490+0000 7ff1875ba700 1 -- 172.21.15.29:0/1364972016 shutdown_connections 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.490+0000 7ff1875ba700 1 --2- 172.21.15.29:0/1364972016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff18014e280 0x7ff18005e470 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.490+0000 7ff1875ba700 1 --2- 172.21.15.29:0/1364972016 >> v2:172.21.15.163:3300/0 conn(0x7ff18014e660 0x7ff1800bc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.490+0000 7ff1875ba700 1 -- 172.21.15.29:0/1364972016 >> 172.21.15.29:0/1364972016 conn(0x7ff1800c3d30 msgr2=0x7ff1800c4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.490+0000 7ff1875ba700 1 -- 172.21.15.29:0/1364972016 shutdown_connections 2024-01-24T14:17:26.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.490+0000 7ff1875ba700 1 -- 172.21.15.29:0/1364972016 wait complete. 2024-01-24T14:17:26.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.491+0000 7ff1875ba700 1 Processor -- start 2024-01-24T14:17:26.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.491+0000 7ff1875ba700 1 -- start start 2024-01-24T14:17:26.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff1875ba700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff18014e280 0x7ff18014d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff1875ba700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff18014e660 0x7ff1800aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff1875ba700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff180157c50 0x7ff180159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff1875ba700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18012a150 con 0x7ff18014e280 2024-01-24T14:17:26.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff1875ba700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800a8400 con 0x7ff18014e660 2024-01-24T14:17:26.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff1875ba700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18014d650 con 0x7ff180157c50 2024-01-24T14:17:26.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff185356700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff18014e280 0x7ff18014d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff185356700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff18014e280 0x7ff18014d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56536/0 (socket says 172.21.15.29:56536) 2024-01-24T14:17:26.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff185356700 1 -- 172.21.15.29:0/2917162272 learned_addr learned my addr 172.21.15.29:0/2917162272 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:26.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff184b55700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff18014e660 0x7ff1800aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff185b57700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff180157c50 0x7ff180159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff185356700 1 -- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff180157c50 msgr2=0x7ff180159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff185356700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff180157c50 0x7ff180159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff185356700 1 -- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff18014e660 msgr2=0x7ff1800aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff185356700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff18014e660 0x7ff1800aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff185356700 1 -- 172.21.15.29:0/2917162272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c009370 con 0x7ff18014e280 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.492+0000 7ff185b57700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff180157c50 0x7ff180159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.493+0000 7ff185356700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff18014e280 0x7ff18014d160 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7ff180063290 tx=0x7ff17c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.493+0000 7ff1757fa700 1 -- 172.21.15.29:0/2917162272 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00b8d0 con 0x7ff18014e280 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.493+0000 7ff1757fa700 1 -- 172.21.15.29:0/2917162272 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c00ba30 con 0x7ff18014e280 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.493+0000 7ff1757fa700 1 -- 172.21.15.29:0/2917162272 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c01b690 con 0x7ff18014e280 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.493+0000 7ff1875ba700 1 -- 172.21.15.29:0/2917162272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1800a7ad0 con 0x7ff18014e280 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.493+0000 7ff1875ba700 1 -- 172.21.15.29:0/2917162272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1800a8090 con 0x7ff18014e280 2024-01-24T14:17:26.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.494+0000 7ff1875ba700 1 -- 172.21.15.29:0/2917162272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff180067b10 con 0x7ff18014e280 2024-01-24T14:17:26.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.494+0000 7ff1757fa700 1 -- 172.21.15.29:0/2917162272 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff17c01ba40 con 0x7ff18014e280 2024-01-24T14:17:26.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.495+0000 7ff1757fa700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff16405f270 0x7ff164061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:26.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.495+0000 7ff1757fa700 1 -- 172.21.15.29:0/2917162272 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7ff17c07bfb0 con 0x7ff18014e280 2024-01-24T14:17:26.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.495+0000 7ff184b55700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff16405f270 0x7ff164061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:26.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.496+0000 7ff184b55700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff16405f270 0x7ff164061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff1800a7250 tx=0x7ff170008040).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:26.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.499+0000 7ff1757fa700 1 -- 172.21.15.29:0/2917162272 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff17c04cd00 con 0x7ff18014e280 2024-01-24T14:17:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.724+0000 7ff1875ba700 1 -- 172.21.15.29:0/2917162272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff180046d80 con 0x7ff18014e280 2024-01-24T14:17:26.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.725+0000 7ff1757fa700 1 -- 172.21.15.29:0/2917162272 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff17c050320 con 0x7ff18014e280 2024-01-24T14:17:26.726 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-24T14:17:26.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.727+0000 7ff1875ba700 1 -- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff16405f270 msgr2=0x7ff164061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.727+0000 7ff1875ba700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff16405f270 0x7ff164061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff1800a7250 tx=0x7ff170008040).stop 2024-01-24T14:17:26.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.727+0000 7ff1875ba700 1 -- 172.21.15.29:0/2917162272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff18014e280 msgr2=0x7ff18014d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:26.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.727+0000 7ff1875ba700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff18014e280 0x7ff18014d160 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7ff180063290 tx=0x7ff17c017000).stop 2024-01-24T14:17:26.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.727+0000 7ff1875ba700 1 -- 172.21.15.29:0/2917162272 shutdown_connections 2024-01-24T14:17:26.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.727+0000 7ff1875ba700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff180157c50 0x7ff180159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.727+0000 7ff1875ba700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff16405f270 0x7ff164061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.727+0000 7ff1875ba700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff18014e280 0x7ff18014d160 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.727+0000 7ff1875ba700 1 --2- 172.21.15.29:0/2917162272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff18014e660 0x7ff1800aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:26.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.727+0000 7ff1875ba700 1 -- 172.21.15.29:0/2917162272 >> 172.21.15.29:0/2917162272 conn(0x7ff1800c3d30 msgr2=0x7ff1800a4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:26.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.727+0000 7ff1875ba700 1 -- 172.21.15.29:0/2917162272 shutdown_connections 2024-01-24T14:17:26.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:26.727+0000 7ff1875ba700 1 -- 172.21.15.29:0/2917162272 wait complete. 2024-01-24T14:17:26.740 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-24T14:17:26.740 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T14:17:26.740 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T14:17:26.740 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-24T14:17:26.740 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-24T14:17:26.740 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-24T14:17:26.740 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-24T14:17:26.740 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T14:17:26.740 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T14:17:26.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:17:26.741 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T14:17:26.760 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T14:17:26.760 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T14:17:26.760 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T14:17:26.760 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:17:26.760 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 41204sec preferred_lft 41204sec 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T14:17:26.761 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::24bc:1cff:fe57:e704/64 scope link 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::2034:aaff:fef3:f12b/64 scope link 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:17:26.762 INFO:teuthology.orchestra.run.smithi029.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T14:17:26.763 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether ba:ca:42:4a:9e:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T14:17:26.763 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::b8ca:42ff:fe4a:9e52/64 scope link 2024-01-24T14:17:26.763 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T14:17:26.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:17:26.763 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T14:17:26.831 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T14:17:26.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:17:26.832 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T14:17:26.897 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T14:17:26.897 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T14:17:26.897 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T14:17:26.897 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T14:17:26.897 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T14:17:26.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:17:26.898 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T14:17:26.898 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T14:17:26.898 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T14:17:26.898 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T14:17:26.898 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T14:17:26.898 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T14:17:27.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:17:27.021 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T14:17:27.041 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-24T14:17:27.041 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T14:17:27.042 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T14:17:27.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:17:27.042 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T14:17:27.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:17:27.118 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-01-24T14:17:27.181 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T14:17:27.181 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:17:27.274 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:17:27.274 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T14:17:27.296 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T14:17:27.296 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T14:17:27.299 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T14:17:27.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T14:17:27.299 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-24T14:17:27.332 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-24T14:17:27.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.514+0000 7f9393635700 1 Processor -- start 2024-01-24T14:17:27.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.514+0000 7f9393635700 1 -- start start 2024-01-24T14:17:27.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9393635700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f938c151e80 0x7f938c0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:27.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9393635700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f938c062ca0 0x7f938c069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:27.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9393635700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938c063150 con 0x7f938c151aa0 2024-01-24T14:17:27.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9393635700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938c0a73f0 con 0x7f938c062ca0 2024-01-24T14:17:27.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9393635700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938c065360 con 0x7f938c151e80 2024-01-24T14:17:27.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9391bd2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f938c062ca0 0x7f938c069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9391bd2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f938c062ca0 0x7f938c069240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56546/0 (socket says 172.21.15.29:56546) 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9391bd2700 1 -- 172.21.15.29:0/2032147322 learned_addr learned my addr 172.21.15.29:0/2032147322 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f93913d1700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f938c151aa0 0x7f938c0a6ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41682/0 (socket says 172.21.15.29:41682) 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9390bd0700 1 --2- 172.21.15.29:0/2032147322 >> v2:172.21.15.163:3300/0 conn(0x7f938c151e80 0x7f938c0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9391bd2700 1 -- 172.21.15.29:0/2032147322 >> v2:172.21.15.163:3300/0 conn(0x7f938c151e80 msgr2=0x7f938c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9391bd2700 1 --2- 172.21.15.29:0/2032147322 >> v2:172.21.15.163:3300/0 conn(0x7f938c151e80 0x7f938c0627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9391bd2700 1 -- 172.21.15.29:0/2032147322 >> v1:172.21.15.29:6789/0 conn(0x7f938c151aa0 legacy=0x7f938c0a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9383fff700 1 -- 172.21.15.29:0/2032147322 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581624990 0 0) 0x7f9388001110 con 0x7f938c151aa0 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9391bd2700 1 -- 172.21.15.29:0/2032147322 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c0bcab0 con 0x7f938c062ca0 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.515+0000 7f9391bd2700 1 --2- 172.21.15.29:0/2032147322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f938c062ca0 0x7f938c069240 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f938c06a540 tx=0x7f9384009840).ready entity=mon.0 client_cookie=31dd5d42b0b6b32b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.516+0000 7f9383fff700 1 -- 172.21.15.29:0/2032147322 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938400e070 con 0x7f938c062ca0 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.516+0000 7f9383fff700 1 -- 172.21.15.29:0/2032147322 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938400ba50 con 0x7f938c062ca0 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.516+0000 7f9383fff700 1 -- 172.21.15.29:0/2032147322 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938400bdd0 con 0x7f938c062ca0 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.516+0000 7f9393635700 1 -- 172.21.15.29:0/2032147322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f938c062ca0 msgr2=0x7f938c069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:27.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.516+0000 7f9393635700 1 --2- 172.21.15.29:0/2032147322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f938c062ca0 0x7f938c069240 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f938c06a540 tx=0x7f9384009840).stop 2024-01-24T14:17:27.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.516+0000 7f9393635700 1 -- 172.21.15.29:0/2032147322 shutdown_connections 2024-01-24T14:17:27.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.516+0000 7f9393635700 1 --2- 172.21.15.29:0/2032147322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f938c062ca0 0x7f938c069240 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.516+0000 7f9393635700 1 --2- 172.21.15.29:0/2032147322 >> v2:172.21.15.163:3300/0 conn(0x7f938c151e80 0x7f938c0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.516+0000 7f9393635700 1 -- 172.21.15.29:0/2032147322 >> 172.21.15.29:0/2032147322 conn(0x7f938c0c8580 msgr2=0x7f938c0c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:27.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.516+0000 7f9393635700 1 -- 172.21.15.29:0/2032147322 shutdown_connections 2024-01-24T14:17:27.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.516+0000 7f9393635700 1 -- 172.21.15.29:0/2032147322 wait complete. 2024-01-24T14:17:27.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.518+0000 7f9393635700 1 Processor -- start 2024-01-24T14:17:27.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.518+0000 7f9393635700 1 -- start start 2024-01-24T14:17:27.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9393635700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f938c062ca0 0x7f938c1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:27.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9393635700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f938c151aa0 0x7f938c165100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:27.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9393635700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f938c151e80 0x7f938c0a8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:27.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9393635700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1567e0 con 0x7f938c151aa0 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9393635700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c155b60 con 0x7f938c151e80 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9393635700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c155e40 con 0x7f938c062ca0 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9390bd0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f938c151aa0 0x7f938c165100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9390bd0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f938c151aa0 0x7f938c165100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56550/0 (socket says 172.21.15.29:56550) 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9390bd0700 1 -- 172.21.15.29:0/3074188806 learned_addr learned my addr 172.21.15.29:0/3074188806 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9391bd2700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f938c151e80 0x7f938c0a8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f93913d1700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f938c062ca0 0x7f938c1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9390bd0700 1 -- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f938c062ca0 msgr2=0x7f938c1555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9390bd0700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f938c062ca0 0x7f938c1555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9390bd0700 1 -- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f938c151e80 msgr2=0x7f938c0a8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9390bd0700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f938c151e80 0x7f938c0a8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9390bd0700 1 -- 172.21.15.29:0/3074188806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9384009ae0 con 0x7f938c151aa0 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9391bd2700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f938c151e80 0x7f938c0a8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9390bd0700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f938c151aa0 0x7f938c165100 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f937c00d890 tx=0x7f937c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:27.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f93817fa700 1 -- 172.21.15.29:0/3074188806 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c0125f0 con 0x7f938c151aa0 2024-01-24T14:17:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f93817fa700 1 -- 172.21.15.29:0/3074188806 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937c012750 con 0x7f938c151aa0 2024-01-24T14:17:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f9393635700 1 -- 172.21.15.29:0/3074188806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938c156120 con 0x7f938c151aa0 2024-01-24T14:17:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.519+0000 7f93817fa700 1 -- 172.21.15.29:0/3074188806 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c012ad0 con 0x7f938c151aa0 2024-01-24T14:17:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.520+0000 7f9393635700 1 -- 172.21.15.29:0/3074188806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938c1655f0 con 0x7f938c151aa0 2024-01-24T14:17:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.521+0000 7f9393635700 1 -- 172.21.15.29:0/3074188806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938c156920 con 0x7f938c151aa0 2024-01-24T14:17:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.521+0000 7f93817fa700 1 -- 172.21.15.29:0/3074188806 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f937c013480 con 0x7f938c151aa0 2024-01-24T14:17:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.521+0000 7f93817fa700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f937005f240 0x7f9370061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.521+0000 7f93817fa700 1 -- 172.21.15.29:0/3074188806 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f937c07a5b0 con 0x7f938c151aa0 2024-01-24T14:17:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.522+0000 7f93913d1700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f937005f240 0x7f9370061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:27.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.522+0000 7f93913d1700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f937005f240 0x7f9370061700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f938c167bb0 tx=0x7f9388007180).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:27.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.526+0000 7f93817fa700 1 -- 172.21.15.29:0/3074188806 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f938c156920 con 0x7f938c151aa0 2024-01-24T14:17:27.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.736+0000 7f9393635700 1 -- 172.21.15.29:0/3074188806 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f938c156920 con 0x7f937005f240 2024-01-24T14:17:27.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.738+0000 7f93817fa700 1 -- 172.21.15.29:0/3074188806 <== mgr.6851 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f938c156920 con 0x7f937005f240 2024-01-24T14:17:27.739 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T14:17:27.739 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T14:17:27.739 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-01-24T14:17:27.739 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T14:17:27.739 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T14:17:27.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.740+0000 7f9393635700 1 -- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f937005f240 msgr2=0x7f9370061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:27.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.740+0000 7f9393635700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f937005f240 0x7f9370061700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f938c167bb0 tx=0x7f9388007180).stop 2024-01-24T14:17:27.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.740+0000 7f9393635700 1 -- 172.21.15.29:0/3074188806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f938c151aa0 msgr2=0x7f938c165100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:27.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.740+0000 7f9393635700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f938c151aa0 0x7f938c165100 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f937c00d890 tx=0x7f937c008040).stop 2024-01-24T14:17:27.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.740+0000 7f9393635700 1 -- 172.21.15.29:0/3074188806 shutdown_connections 2024-01-24T14:17:27.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.740+0000 7f9393635700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f938c151e80 0x7f938c0a8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.740+0000 7f9393635700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f938c062ca0 0x7f938c1555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.740+0000 7f9393635700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f937005f240 0x7f9370061700 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.740+0000 7f9393635700 1 --2- 172.21.15.29:0/3074188806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f938c151aa0 0x7f938c165100 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.740+0000 7f9393635700 1 -- 172.21.15.29:0/3074188806 >> 172.21.15.29:0/3074188806 conn(0x7f938c0c8580 msgr2=0x7f938c0c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:27.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.740+0000 7f9393635700 1 -- 172.21.15.29:0/3074188806 shutdown_connections 2024-01-24T14:17:27.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.740+0000 7f9393635700 1 -- 172.21.15.29:0/3074188806 wait complete. 2024-01-24T14:17:27.754 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-01-24T14:17:27.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.885+0000 7f23a7d31700 1 Processor -- start 2024-01-24T14:17:27.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.886+0000 7f23a7d31700 1 -- start start 2024-01-24T14:17:27.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.886+0000 7f23a7d31700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23a0066c30 0x7f23a00c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:27.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.886+0000 7f23a7d31700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0068900 0x7f23a0152e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:27.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.886+0000 7f23a7d31700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a0068470 con 0x7f23a0134e70 2024-01-24T14:17:27.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a7d31700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a014c910 con 0x7f23a0066c30 2024-01-24T14:17:27.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a7d31700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0153340 con 0x7f23a0068900 2024-01-24T14:17:27.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a62ce700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0068900 0x7f23a0152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:27.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a52cc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23a0066c30 0x7f23a00c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:27.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a52cc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23a0066c30 0x7f23a00c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:50634/0 (socket says 172.21.15.29:50634) 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a52cc700 1 -- 172.21.15.29:0/329223898 learned_addr learned my addr 172.21.15.29:0/329223898 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a62ce700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0068900 0x7f23a0152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:41824/0 (socket says 172.21.15.29:41824) 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a5acd700 1 --1- 172.21.15.29:0/329223898 >> v1:172.21.15.163:6789/0 conn(0x7f23a0134e70 0x7f23a0066520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:35808/0 (socket says 172.21.15.29:35808) 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a52cc700 1 -- 172.21.15.29:0/329223898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0068900 msgr2=0x7f23a0152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a52cc700 1 --2- 172.21.15.29:0/329223898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0068900 0x7f23a0152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a52cc700 1 -- 172.21.15.29:0/329223898 >> v1:172.21.15.163:6789/0 conn(0x7f23a0134e70 legacy=0x7f23a0066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a4acb700 1 -- 172.21.15.29:0/329223898 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248542046 0 0) 0x7f23a0068470 con 0x7f23a0134e70 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a52cc700 1 -- 172.21.15.29:0/329223898 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00c3860 con 0x7f23a0066c30 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a62ce700 1 --2- 172.21.15.29:0/329223898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0068900 0x7f23a0152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.887+0000 7f23a52cc700 1 --2- 172.21.15.29:0/329223898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a0066c30 0x7f23a00c31a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f2394006fd0 tx=0x7f23940090f0).ready entity=mon.1 client_cookie=be372d60382e4fb3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.888+0000 7f23a4acb700 1 -- 172.21.15.29:0/329223898 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239400b580 con 0x7f23a0066c30 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.888+0000 7f23a4acb700 1 -- 172.21.15.29:0/329223898 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239400b6e0 con 0x7f23a0066c30 2024-01-24T14:17:27.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.888+0000 7f23a4acb700 1 -- 172.21.15.29:0/329223898 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239400ba60 con 0x7f23a0066c30 2024-01-24T14:17:27.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.888+0000 7f23a7d31700 1 -- 172.21.15.29:0/329223898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a0066c30 msgr2=0x7f23a00c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:27.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.888+0000 7f23a7d31700 1 --2- 172.21.15.29:0/329223898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a0066c30 0x7f23a00c31a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f2394006fd0 tx=0x7f23940090f0).stop 2024-01-24T14:17:27.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.888+0000 7f23a7d31700 1 -- 172.21.15.29:0/329223898 shutdown_connections 2024-01-24T14:17:27.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.888+0000 7f23a7d31700 1 --2- 172.21.15.29:0/329223898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0068900 0x7f23a0152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.888+0000 7f23a7d31700 1 --2- 172.21.15.29:0/329223898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a0066c30 0x7f23a00c31a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.888+0000 7f23a7d31700 1 -- 172.21.15.29:0/329223898 >> 172.21.15.29:0/329223898 conn(0x7f23a00c8ba0 msgr2=0x7f23a00c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:27.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.888+0000 7f23a7d31700 1 -- 172.21.15.29:0/329223898 shutdown_connections 2024-01-24T14:17:27.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.888+0000 7f23a7d31700 1 -- 172.21.15.29:0/329223898 wait complete. 2024-01-24T14:17:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.889+0000 7f23a7d31700 1 Processor -- start 2024-01-24T14:17:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.890+0000 7f23a7d31700 1 -- start start 2024-01-24T14:17:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.890+0000 7f23a7d31700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a0066c30 0x7f23a0162430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.890+0000 7f23a7d31700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a0068900 0x7f23a00ab220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.890+0000 7f23a7d31700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0134e70 0x7f23a01496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.890+0000 7f23a7d31700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00c0d40 con 0x7f23a0066c30 2024-01-24T14:17:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.890+0000 7f23a7d31700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a01629e0 con 0x7f23a0068900 2024-01-24T14:17:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.890+0000 7f23a7d31700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0162cc0 con 0x7f23a0134e70 2024-01-24T14:17:27.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.890+0000 7f23a5acd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a0066c30 0x7f23a0162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a5acd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a0066c30 0x7f23a0162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56552/0 (socket says 172.21.15.29:56552) 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a5acd700 1 -- 172.21.15.29:0/2509945520 learned_addr learned my addr 172.21.15.29:0/2509945520 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a62ce700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0134e70 0x7f23a01496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a52cc700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a0068900 0x7f23a00ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a5acd700 1 -- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0134e70 msgr2=0x7f23a01496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a5acd700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0134e70 0x7f23a01496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a5acd700 1 -- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a0068900 msgr2=0x7f23a00ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a5acd700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a0068900 0x7f23a00ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a5acd700 1 -- 172.21.15.29:0/2509945520 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2398009a20 con 0x7f23a0066c30 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a62ce700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0134e70 0x7f23a01496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a52cc700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a0068900 0x7f23a00ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a5acd700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a0066c30 0x7f23a0162430 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f239c00c440 tx=0x7f239c007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:27.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f2391ffb700 1 -- 172.21.15.29:0/2509945520 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c0072e0 con 0x7f23a0066c30 2024-01-24T14:17:27.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f2391ffb700 1 -- 172.21.15.29:0/2509945520 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239c007440 con 0x7f23a0066c30 2024-01-24T14:17:27.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f2391ffb700 1 -- 172.21.15.29:0/2509945520 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c005910 con 0x7f23a0066c30 2024-01-24T14:17:27.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.891+0000 7f23a7d31700 1 -- 172.21.15.29:0/2509945520 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2394009370 con 0x7f23a0066c30 2024-01-24T14:17:27.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.892+0000 7f23a7d31700 1 -- 172.21.15.29:0/2509945520 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a0163280 con 0x7f23a0066c30 2024-01-24T14:17:27.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.893+0000 7f23a7d31700 1 -- 172.21.15.29:0/2509945520 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a00475e0 con 0x7f23a0066c30 2024-01-24T14:17:27.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.893+0000 7f2391ffb700 1 -- 172.21.15.29:0/2509945520 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 57) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f239c005d00 con 0x7f23a0066c30 2024-01-24T14:17:27.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.893+0000 7f2391ffb700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f238405f2a0 0x7f2384061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:27.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.893+0000 7f2391ffb700 1 -- 172.21.15.29:0/2509945520 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f239c079e20 con 0x7f23a0066c30 2024-01-24T14:17:27.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.894+0000 7f23a52cc700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f238405f2a0 0x7f2384061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:27.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.894+0000 7f23a52cc700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f238405f2a0 0x7f2384061760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f2394006fd0 tx=0x7f2394017000).ready entity=mgr.6851 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:27.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:27.898+0000 7f2391ffb700 1 -- 172.21.15.29:0/2509945520 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f239c04ab70 con 0x7f23a0066c30 2024-01-24T14:17:28.133 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.133+0000 7f23a7d31700 1 -- 172.21.15.29:0/2509945520 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f23a0068df0 con 0x7f23a0066c30 2024-01-24T14:17:28.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.442+0000 7f2391ffb700 1 -- 172.21.15.29:0/2509945520 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 58) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f239c077670 con 0x7f23a0066c30 2024-01-24T14:17:28.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.443+0000 7f2391ffb700 1 -- 172.21.15.29:0/2509945520 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v58) v1 ==== 92+0+0 (secure 0 0 0) 0x7f239c075f60 con 0x7f23a0066c30 2024-01-24T14:17:28.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.446+0000 7f23a7d31700 1 -- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f238405f2a0 msgr2=0x7f2384061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:28.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.446+0000 7f23a7d31700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f238405f2a0 0x7f2384061760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f2394006fd0 tx=0x7f2394017000).stop 2024-01-24T14:17:28.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.446+0000 7f23a7d31700 1 -- 172.21.15.29:0/2509945520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a0066c30 msgr2=0x7f23a0162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:28.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.446+0000 7f23a7d31700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a0066c30 0x7f23a0162430 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f239c00c440 tx=0x7f239c007e30).stop 2024-01-24T14:17:28.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.447+0000 7f23a7d31700 1 -- 172.21.15.29:0/2509945520 shutdown_connections 2024-01-24T14:17:28.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.447+0000 7f23a7d31700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a0134e70 0x7f23a01496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:28.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.447+0000 7f23a7d31700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f238405f2a0 0x7f2384061760 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:28.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.447+0000 7f23a7d31700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a0066c30 0x7f23a0162430 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:28.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.447+0000 7f23a7d31700 1 --2- 172.21.15.29:0/2509945520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a0068900 0x7f23a00ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:28.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.447+0000 7f23a7d31700 1 -- 172.21.15.29:0/2509945520 >> 172.21.15.29:0/2509945520 conn(0x7f23a00c8ba0 msgr2=0x7f23a00bf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:28.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.447+0000 7f23a7d31700 1 -- 172.21.15.29:0/2509945520 shutdown_connections 2024-01-24T14:17:28.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.447+0000 7f23a7d31700 1 -- 172.21.15.29:0/2509945520 wait complete. 2024-01-24T14:17:28.464 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-01-24T14:17:28.588 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:28.588+0000 7fc8824dc680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:17:28.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.603+0000 7fcc9a39f700 1 Processor -- start 2024-01-24T14:17:28.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.603+0000 7fcc9a39f700 1 -- start start 2024-01-24T14:17:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.604+0000 7fcc9a39f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcc9413a5e0 0x7fcc940639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.604+0000 7fcc9a39f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc94066bd0 0x7fcc940bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.604+0000 7fcc9a39f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc940679f0 con 0x7fcc94063eb0 2024-01-24T14:17:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.604+0000 7fcc9a39f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc94067cd0 con 0x7fcc94066bd0 2024-01-24T14:17:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.604+0000 7fcc9a39f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc94067fb0 con 0x7fcc9413a5e0 2024-01-24T14:17:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.604+0000 7fcc9893c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc94066bd0 0x7fcc940bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.604+0000 7fcc93fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcc9413a5e0 0x7fcc940639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.604+0000 7fcc9893c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc94066bd0 0x7fcc940bdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56568/0 (socket says 172.21.15.29:56568) 2024-01-24T14:17:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.604+0000 7fcc9893c700 1 -- 172.21.15.29:0/775216985 learned_addr learned my addr 172.21.15.29:0/775216985 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.605+0000 7fcc93fff700 1 -- 172.21.15.29:0/775216985 >> v2:172.21.15.29:3300/0 conn(0x7fcc94066bd0 msgr2=0x7fcc940bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.605+0000 7fcc93fff700 1 --2- 172.21.15.29:0/775216985 >> v2:172.21.15.29:3300/0 conn(0x7fcc94066bd0 0x7fcc940bdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:28.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.605+0000 7fcc93fff700 1 -- 172.21.15.29:0/775216985 >> v1:172.21.15.163:6789/0 conn(0x7fcc94063eb0 legacy=0x7fcc940664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:28.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.605+0000 7fcc93fff700 1 -- 172.21.15.29:0/775216985 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc94068290 con 0x7fcc9413a5e0 2024-01-24T14:17:28.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.605+0000 7fcc9893c700 1 --2- 172.21.15.29:0/775216985 >> v2:172.21.15.29:3300/0 conn(0x7fcc94066bd0 0x7fcc940bdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:28.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.605+0000 7fcc92ffd700 1 -- 172.21.15.29:0/775216985 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881449828 0 0) 0x7fcc940679f0 con 0x7fcc94063eb0 2024-01-24T14:17:28.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.605+0000 7fcc93fff700 1 --2- 172.21.15.29:0/775216985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc9413a5e0 0x7fcc940639c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fcc94069ba0 tx=0x7fcc84009060).ready entity=mon.1 client_cookie=2d03821f4e9a6d14 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:28.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.605+0000 7fcc92ffd700 1 -- 172.21.15.29:0/775216985 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8400f510 con 0x7fcc9413a5e0 2024-01-24T14:17:28.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.605+0000 7fcc92ffd700 1 -- 172.21.15.29:0/775216985 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8400f670 con 0x7fcc9413a5e0 2024-01-24T14:17:28.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.605+0000 7fcc92ffd700 1 -- 172.21.15.29:0/775216985 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8400f9f0 con 0x7fcc9413a5e0 2024-01-24T14:17:28.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.605+0000 7fcc9a39f700 1 -- 172.21.15.29:0/775216985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc9413a5e0 msgr2=0x7fcc940639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:28.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.605+0000 7fcc9a39f700 1 --2- 172.21.15.29:0/775216985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc9413a5e0 0x7fcc940639c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fcc94069ba0 tx=0x7fcc84009060).stop 2024-01-24T14:17:28.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.606+0000 7fcc9a39f700 1 -- 172.21.15.29:0/775216985 shutdown_connections 2024-01-24T14:17:28.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.606+0000 7fcc9a39f700 1 --2- 172.21.15.29:0/775216985 >> v2:172.21.15.29:3300/0 conn(0x7fcc94066bd0 0x7fcc940bdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:28.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.606+0000 7fcc9a39f700 1 --2- 172.21.15.29:0/775216985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc9413a5e0 0x7fcc940639c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:28.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.606+0000 7fcc9a39f700 1 -- 172.21.15.29:0/775216985 >> 172.21.15.29:0/775216985 conn(0x7fcc940c8bd0 msgr2=0x7fcc940c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:28.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.606+0000 7fcc9a39f700 1 -- 172.21.15.29:0/775216985 shutdown_connections 2024-01-24T14:17:28.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.606+0000 7fcc9a39f700 1 -- 172.21.15.29:0/775216985 wait complete. 2024-01-24T14:17:28.609 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:28.607+0000 7f63b22af680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T14:17:28.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.607+0000 7fcc9a39f700 1 Processor -- start 2024-01-24T14:17:28.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.608+0000 7fcc9a39f700 1 -- start start 2024-01-24T14:17:28.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc9a39f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc94063eb0 0x7fcc940ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:28.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc9a39f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc94066bd0 0x7fcc94137ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc9a39f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9413a5e0 0x7fcc9413ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc9a39f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940ad200 con 0x7fcc9413a5e0 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc9a39f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940ac630 con 0x7fcc94066bd0 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc9a39f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940ac8d0 con 0x7fcc94063eb0 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc937fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc94066bd0 0x7fcc94137ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc9893c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9413a5e0 0x7fcc9413ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc93fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc94063eb0 0x7fcc940ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc937fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc94066bd0 0x7fcc94137ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:50686/0 (socket says 172.21.15.29:50686) 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc937fe700 1 -- 172.21.15.29:0/3076807174 learned_addr learned my addr 172.21.15.29:0/3076807174 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc93fff700 1 -- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc94066bd0 msgr2=0x7fcc94137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc93fff700 1 --2- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc94066bd0 0x7fcc94137ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc93fff700 1 -- 172.21.15.29:0/3076807174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9413a5e0 msgr2=0x7fcc9413ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:28.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc93fff700 1 --2- 172.21.15.29:0/3076807174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9413a5e0 0x7fcc9413ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:28.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc93fff700 1 -- 172.21.15.29:0/3076807174 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c009ae0 con 0x7fcc94063eb0 2024-01-24T14:17:28.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc9893c700 1 --2- 172.21.15.29:0/3076807174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9413a5e0 0x7fcc9413ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:28.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc93fff700 1 --2- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc94063eb0 0x7fcc940ad690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fcc84014000 tx=0x7fcc84017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:28.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.609+0000 7fcc7bfff700 1 -- 172.21.15.29:0/3076807174 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8400f7e0 con 0x7fcc94063eb0 2024-01-24T14:17:28.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.610+0000 7fcc7bfff700 1 -- 172.21.15.29:0/3076807174 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8400f940 con 0x7fcc94063eb0 2024-01-24T14:17:28.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.610+0000 7fcc9a39f700 1 -- 172.21.15.29:0/3076807174 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc840092a0 con 0x7fcc94063eb0 2024-01-24T14:17:28.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.610+0000 7fcc7bfff700 1 -- 172.21.15.29:0/3076807174 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8401b690 con 0x7fcc94063eb0 2024-01-24T14:17:28.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.610+0000 7fcc9a39f700 1 -- 172.21.15.29:0/3076807174 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc940ace90 con 0x7fcc94063eb0 2024-01-24T14:17:28.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.611+0000 7fcc9a39f700 1 -- 172.21.15.29:0/3076807174 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc940adb80 con 0x7fcc94063eb0 2024-01-24T14:17:28.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.611+0000 7fcc7bfff700 1 -- 172.21.15.29:0/3076807174 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 58) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fcc8401ba00 con 0x7fcc94063eb0 2024-01-24T14:17:28.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.611+0000 7fcc7bfff700 1 --2- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8005f2c0 0x7fcc80061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:28.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.611+0000 7fcc937fe700 1 -- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8005f2c0 msgr2=0x7fcc80061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:28.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.611+0000 7fcc7bfff700 1 -- 172.21.15.29:0/3076807174 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fcc8407bcf0 con 0x7fcc94063eb0 2024-01-24T14:17:28.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.611+0000 7fcc937fe700 1 --2- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8005f2c0 0x7fcc80061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:28.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.616+0000 7fcc7bfff700 1 -- 172.21.15.29:0/3076807174 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcc8404ca40 con 0x7fcc94063eb0 2024-01-24T14:17:28.652 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:28.652+0000 7fc8824dc680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:17:28.681 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:28.681+0000 7f63b22af680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T14:17:28.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.800+0000 7fcc9a39f700 1 -- 172.21.15.29:0/3076807174 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fcc940475e0 con 0x7fcc94063eb0 2024-01-24T14:17:28.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.804+0000 7fcc7bfff700 1 -- 172.21.15.29:0/3076807174 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcc84028020 con 0x7fcc94063eb0 2024-01-24T14:17:28.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.806+0000 7fcc9a39f700 1 -- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8005f2c0 msgr2=0x7fcc80061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:28.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.806+0000 7fcc9a39f700 1 --2- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8005f2c0 0x7fcc80061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:28.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.806+0000 7fcc9a39f700 1 -- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc94063eb0 msgr2=0x7fcc940ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:28.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.806+0000 7fcc9a39f700 1 --2- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc94063eb0 0x7fcc940ad690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fcc84014000 tx=0x7fcc84017000).stop 2024-01-24T14:17:28.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.806+0000 7fcc9a39f700 1 -- 172.21.15.29:0/3076807174 shutdown_connections 2024-01-24T14:17:28.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.806+0000 7fcc9a39f700 1 --2- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fcc8005f2c0 0x7fcc80061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:28.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.806+0000 7fcc9a39f700 1 --2- 172.21.15.29:0/3076807174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9413a5e0 0x7fcc9413ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:28.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.806+0000 7fcc9a39f700 1 --2- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc94063eb0 0x7fcc940ad690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:28.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.806+0000 7fcc9a39f700 1 --2- 172.21.15.29:0/3076807174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc94066bd0 0x7fcc94137ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:28.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.806+0000 7fcc9a39f700 1 -- 172.21.15.29:0/3076807174 >> 172.21.15.29:0/3076807174 conn(0x7fcc940c8bd0 msgr2=0x7fcc94156c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:28.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.806+0000 7fcc9a39f700 1 -- 172.21.15.29:0/3076807174 shutdown_connections 2024-01-24T14:17:28.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:28.806+0000 7fcc9a39f700 1 -- 172.21.15.29:0/3076807174 wait complete. 2024-01-24T14:17:28.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T14:17:28.821 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-01-24T14:17:28.845 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1 1M 2024-01-24T14:17:28.926 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:28.926+0000 7fc8824dc680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:17:29.000 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:28.999+0000 7f63b22af680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T14:17:29.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.021+0000 7f4c53d42700 1 Processor -- start 2024-01-24T14:17:29.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.021+0000 7f4c53d42700 1 -- start start 2024-01-24T14:17:29.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c53d42700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c4c064e90 0x7f4c4c0c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:29.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c53d42700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4c154f90 con 0x7f4c4c151e50 2024-01-24T14:17:29.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c53d42700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4c0a9f10 con 0x7f4c4c0c7f70 2024-01-24T14:17:29.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c53d42700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0aa1f0 con 0x7f4c4c064e90 2024-01-24T14:17:29.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c51ade700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4c4c151e50 0x7f4c4c064780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41704/0 (socket says 172.21.15.29:41704) 2024-01-24T14:17:29.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c51ade700 1 -- 172.21.15.29:0/2273654200 learned_addr learned my addr 172.21.15.29:0/2273654200 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:29.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c512dd700 1 --2- 172.21.15.29:0/2273654200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c4c064e90 0x7f4c4c0c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:29.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c50adc700 1 -- 172.21.15.29:0/2273654200 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964702538 0 0) 0x7f4c4c154f90 con 0x7f4c4c151e50 2024-01-24T14:17:29.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c50adc700 1 -- 172.21.15.29:0/2273654200 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c380033f0 con 0x7f4c4c151e50 2024-01-24T14:17:29.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c512dd700 1 -- 172.21.15.29:0/2273654200 >> v1:172.21.15.163:6789/0 conn(0x7f4c4c0c7f70 legacy=0x7f4c4c0bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:29.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c50adc700 1 -- 172.21.15.29:0/2273654200 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731713145 0 0) 0x7f4c4c0a9f10 con 0x7f4c4c0c7f70 2024-01-24T14:17:29.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c512dd700 1 -- 172.21.15.29:0/2273654200 >> v1:172.21.15.29:6789/0 conn(0x7f4c4c151e50 legacy=0x7f4c4c064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:29.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.023+0000 7f4c512dd700 1 -- 172.21.15.29:0/2273654200 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c0aa4d0 con 0x7f4c4c064e90 2024-01-24T14:17:29.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.024+0000 7f4c512dd700 1 --2- 172.21.15.29:0/2273654200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c4c064e90 0x7f4c4c0c53c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f4c3c009790 tx=0x7f4c3c009060).ready entity=mon.2 client_cookie=1cc2f44113587e97 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:29.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.024+0000 7f4c50adc700 1 -- 172.21.15.29:0/2273654200 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00f510 con 0x7f4c4c064e90 2024-01-24T14:17:29.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.024+0000 7f4c50adc700 1 -- 172.21.15.29:0/2273654200 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c00f670 con 0x7f4c4c064e90 2024-01-24T14:17:29.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.024+0000 7f4c50adc700 1 -- 172.21.15.29:0/2273654200 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00f9f0 con 0x7f4c4c064e90 2024-01-24T14:17:29.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.024+0000 7f4c53d42700 1 -- 172.21.15.29:0/2273654200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c4c064e90 msgr2=0x7f4c4c0c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:29.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.024+0000 7f4c53d42700 1 --2- 172.21.15.29:0/2273654200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c4c064e90 0x7f4c4c0c53c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f4c3c009790 tx=0x7f4c3c009060).stop 2024-01-24T14:17:29.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.024+0000 7f4c53d42700 1 -- 172.21.15.29:0/2273654200 shutdown_connections 2024-01-24T14:17:29.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.024+0000 7f4c53d42700 1 --2- 172.21.15.29:0/2273654200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c4c064e90 0x7f4c4c0c53c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:29.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.024+0000 7f4c53d42700 1 -- 172.21.15.29:0/2273654200 >> 172.21.15.29:0/2273654200 conn(0x7f4c4c0cb880 msgr2=0x7f4c4c0c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:29.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.025+0000 7f4c53d42700 1 -- 172.21.15.29:0/2273654200 shutdown_connections 2024-01-24T14:17:29.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.025+0000 7f4c53d42700 1 -- 172.21.15.29:0/2273654200 wait complete. 2024-01-24T14:17:29.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.026+0000 7f4c53d42700 1 Processor -- start 2024-01-24T14:17:29.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.026+0000 7f4c53d42700 1 -- start start 2024-01-24T14:17:29.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.027+0000 7f4c53d42700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c4c064e90 0x7f4c4c14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:29.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.027+0000 7f4c53d42700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c4c0c7f70 0x7f4c4c168660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:29.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.027+0000 7f4c53d42700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c4c151e50 0x7f4c4c0ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:29.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.027+0000 7f4c53d42700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c1657d0 con 0x7f4c4c064e90 2024-01-24T14:17:29.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.027+0000 7f4c53d42700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c164bc0 con 0x7f4c4c0c7f70 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.027+0000 7f4c53d42700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c164ea0 con 0x7f4c4c151e50 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c51ade700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c4c064e90 0x7f4c4c14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c51ade700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c4c064e90 0x7f4c4c14c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56584/0 (socket says 172.21.15.29:56584) 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c51ade700 1 -- 172.21.15.29:0/84679528 learned_addr learned my addr 172.21.15.29:0/84679528 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c512dd700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c4c0c7f70 0x7f4c4c168660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c522df700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c4c151e50 0x7f4c4c0ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c51ade700 1 -- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c4c151e50 msgr2=0x7f4c4c0ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c51ade700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c4c151e50 0x7f4c4c0ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c51ade700 1 -- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c4c0c7f70 msgr2=0x7f4c4c168660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c51ade700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c4c0c7f70 0x7f4c4c168660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c51ade700 1 -- 172.21.15.29:0/84679528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0092a0 con 0x7f4c4c064e90 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c522df700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c4c151e50 0x7f4c4c0ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:29.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c51ade700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c4c064e90 0x7f4c4c14c1e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f4c4800c410 tx=0x7f4c48007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:29.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c41ffb700 1 -- 172.21.15.29:0/84679528 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c480070d0 con 0x7f4c4c064e90 2024-01-24T14:17:29.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c41ffb700 1 -- 172.21.15.29:0/84679528 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c48007230 con 0x7f4c4c064e90 2024-01-24T14:17:29.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c41ffb700 1 -- 172.21.15.29:0/84679528 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c480075d0 con 0x7f4c4c064e90 2024-01-24T14:17:29.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.028+0000 7f4c53d42700 1 -- 172.21.15.29:0/84679528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c165180 con 0x7f4c4c064e90 2024-01-24T14:17:29.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.029+0000 7f4c53d42700 1 -- 172.21.15.29:0/84679528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c164870 con 0x7f4c4c064e90 2024-01-24T14:17:29.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.030+0000 7f4c53d42700 1 -- 172.21.15.29:0/84679528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c0aa710 con 0x7f4c4c064e90 2024-01-24T14:17:29.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.030+0000 7f4c41ffb700 1 -- 172.21.15.29:0/84679528 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 58) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f4c480059b0 con 0x7f4c4c064e90 2024-01-24T14:17:29.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.030+0000 7f4c41ffb700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 0x7f4c30061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:29.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.031+0000 7f4c41ffb700 1 -- 172.21.15.29:0/84679528 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f4c48079dc0 con 0x7f4c4c064e90 2024-01-24T14:17:29.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.031+0000 7f4c512dd700 1 -- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 msgr2=0x7f4c30061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:29.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.031+0000 7f4c512dd700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 0x7f4c30061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T14:17:29.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.035+0000 7f4c41ffb700 1 -- 172.21.15.29:0/84679528 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4c4804ab10 con 0x7f4c4c064e90 2024-01-24T14:17:29.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.232+0000 7f4c512dd700 1 -- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 msgr2=0x7f4c30061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:29.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.232+0000 7f4c512dd700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 0x7f4c30061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T14:17:29.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.245+0000 7f4c53d42700 1 -- 172.21.15.29:0/84679528 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c4c0ad880 con 0x7f4c3005f2c0 2024-01-24T14:17:29.378 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:17:29.381 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:17:29.546 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:29.546+0000 7fc8824dc680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:17:29.619 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:29.619+0000 7fc8824dc680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:17:29.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.633+0000 7f4c512dd700 1 -- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 msgr2=0x7f4c30061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:29.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:29.633+0000 7f4c512dd700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 0x7f4c30061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-01-24T14:17:29.728 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:29.727+0000 7f63b22af680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T14:17:29.803 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:29.802+0000 7f63b22af680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T14:17:29.805 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:29.805+0000 7fc8824dc680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:17:30.018 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:30.018+0000 7f63b22af680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T14:17:30.225 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:30.225+0000 7fc8824dc680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:17:30.329 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:30.329+0000 7fc8824dc680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:17:30.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:30.434+0000 7f4c512dd700 1 -- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 msgr2=0x7f4c30061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:30.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:30.434+0000 7f4c512dd700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 0x7f4c30061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-01-24T14:17:30.483 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:30.482+0000 7fc8824dc680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:17:30.498 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:30.498+0000 7f63b22af680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T14:17:30.558 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:30.558+0000 7fc8824dc680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:17:30.616 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:30.615+0000 7f63b22af680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:17:30.624 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:30.624+0000 7fc8824dc680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:17:30.793 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:30.792+0000 7f63b22af680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T14:17:30.879 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:30.878+0000 7f63b22af680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T14:17:30.880 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:30.880+0000 7fc8824dc680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:17:30.956 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:30.956+0000 7f63b22af680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T14:17:31.063 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:31.063+0000 7fc8824dc680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:17:31.253 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:31.252+0000 7f63b22af680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T14:17:31.465 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:31.464+0000 7f63b22af680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T14:17:31.712 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:31.712+0000 7fc8824dc680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:17:31.781 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:31.781+0000 7fc8824dc680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:17:31.856 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:31.855+0000 7fc8824dc680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:17:32.003 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.003+0000 7fc8824dc680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:17:32.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:32.036+0000 7f4c512dd700 1 -- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 msgr2=0x7f4c30061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T14:17:32.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:32.036+0000 7f4c512dd700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 0x7f4c30061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-01-24T14:17:32.075 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.075+0000 7fc8824dc680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:17:32.190 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.190+0000 7fc8824dc680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:17:32.219 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:32.218+0000 7f63b22af680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T14:17:32.298 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:32.298+0000 7f63b22af680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T14:17:32.299 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.299+0000 7fc8824dc680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:17:32.384 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:32.383+0000 7f63b22af680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T14:17:32.553 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:32.552+0000 7f63b22af680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T14:17:32.589 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.589+0000 7fc8824dc680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:17:32.635 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:32.634+0000 7f63b22af680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T14:17:32.660 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.660+0000 7fc8824dc680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:17:32.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:32.680+0000 7f4c41ffb700 1 -- 172.21.15.29:0/84679528 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 59) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f4c48077610 con 0x7f4c4c064e90 2024-01-24T14:17:32.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:32.680+0000 7f4c41ffb700 1 -- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 msgr2=0x7f4c30061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T14:17:32.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:32.680+0000 7f4c41ffb700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 0x7f4c30061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T14:17:32.749 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.749+0000 7fc7f8a57700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:32.749 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.749+0000 7fc7f8a57700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:32.749 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.749+0000 7fc7f8a57700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:32.749 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.749+0000 7fc7f8a57700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:32.749 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.749+0000 7fc7f8a57700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:32.749 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.749+0000 7fc7f5a51700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:32.749 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.749+0000 7fc7f5a51700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:32.750 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.749+0000 7fc7f5a51700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:32.750 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.749+0000 7fc7f5a51700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:32.750 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:17:32.749+0000 7fc7f5a51700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:17:32.774 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:32.774+0000 7f63b22af680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T14:17:32.898 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:32.898+0000 7f63b22af680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T14:17:33.233 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:33.232+0000 7f63b22af680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T14:17:33.315 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T14:17:33.314+0000 7f63b22af680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T14:17:33.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.683+0000 7f4c41ffb700 1 -- 172.21.15.29:0/84679528 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mgrmap(e 60) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f4c48077990 con 0x7f4c4c064e90 2024-01-24T14:17:33.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.683+0000 7f4c41ffb700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 0x7f4c30061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:33.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.684+0000 7f4c41ffb700 1 -- 172.21.15.29:0/84679528 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c4c0ad880 con 0x7f4c3005f2c0 2024-01-24T14:17:33.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.684+0000 7f4c512dd700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 0x7f4c30061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:33.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.686+0000 7f4c512dd700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 0x7f4c30061700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f4c4c065290 tx=0x7f4c3c017000).ready entity=mgr.7097 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:33.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.694+0000 7f4c41ffb700 1 -- 172.21.15.29:0/84679528 <== mgr.7097 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f4c4c0ad880 con 0x7f4c3005f2c0 2024-01-24T14:17:33.695 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /snap_test_dir1 2024-01-24T14:17:33.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.697+0000 7f4c53d42700 1 -- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 msgr2=0x7f4c30061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:33.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.697+0000 7f4c53d42700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 0x7f4c30061700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f4c4c065290 tx=0x7f4c3c017000).stop 2024-01-24T14:17:33.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.697+0000 7f4c53d42700 1 -- 172.21.15.29:0/84679528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c4c064e90 msgr2=0x7f4c4c14c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:33.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.697+0000 7f4c53d42700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c4c064e90 0x7f4c4c14c1e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f4c4800c410 tx=0x7f4c48007c70).stop 2024-01-24T14:17:33.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.697+0000 7f4c53d42700 1 -- 172.21.15.29:0/84679528 shutdown_connections 2024-01-24T14:17:33.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.697+0000 7f4c53d42700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c4c151e50 0x7f4c4c0ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:33.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.697+0000 7f4c53d42700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4c3005f2c0 0x7f4c30061700 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:33.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.697+0000 7f4c53d42700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c4c064e90 0x7f4c4c14c1e0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:33.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.697+0000 7f4c53d42700 1 --2- 172.21.15.29:0/84679528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c4c0c7f70 0x7f4c4c168660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:33.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.697+0000 7f4c53d42700 1 -- 172.21.15.29:0/84679528 >> 172.21.15.29:0/84679528 conn(0x7f4c4c0cb880 msgr2=0x7f4c4c152440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:33.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.697+0000 7f4c53d42700 1 -- 172.21.15.29:0/84679528 shutdown_connections 2024-01-24T14:17:33.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.697+0000 7f4c53d42700 1 -- 172.21.15.29:0/84679528 wait complete. 2024-01-24T14:17:33.713 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2024-01-24T14:17:33.713 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs /snap_test_dir1 55n 2024-01-24T14:17:33.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.853+0000 7f9a9bf38700 1 Processor -- start 2024-01-24T14:17:33.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.854+0000 7f9a9bf38700 1 -- start start 2024-01-24T14:17:33.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.854+0000 7f9a9bf38700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a940692a0 0x7f9a94065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:33.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a9bf38700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a94066370 0x7f9a94069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a9bf38700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a940c3e70 con 0x7f9a94069b70 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a9bf38700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a940c4300 con 0x7f9a94066370 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a9bf38700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940c3260 con 0x7f9a940692a0 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a9a4d5700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9a94069b70 0x7f9a940bcd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58800/0 (socket says 172.21.15.29:58800) 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a9a4d5700 1 -- 172.21.15.29:0/3617112941 learned_addr learned my addr 172.21.15.29:0/3617112941 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a99cd4700 1 --2- 172.21.15.29:0/3617112941 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a940692a0 0x7f9a94065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a994d3700 1 --2- 172.21.15.29:0/3617112941 >> v2:172.21.15.163:3300/0 conn(0x7f9a94066370 0x7f9a94069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a98cd2700 1 -- 172.21.15.29:0/3617112941 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956839384 0 0) 0x7f9a940c3e70 con 0x7f9a94069b70 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a98cd2700 1 -- 172.21.15.29:0/3617112941 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a800033f0 con 0x7f9a94069b70 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a98cd2700 1 -- 172.21.15.29:0/3617112941 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3701191450 0 0) 0x7f9a800033f0 con 0x7f9a94069b70 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a99cd4700 1 -- 172.21.15.29:0/3617112941 >> v2:172.21.15.163:3300/0 conn(0x7f9a94066370 msgr2=0x7f9a94069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a99cd4700 1 --2- 172.21.15.29:0/3617112941 >> v2:172.21.15.163:3300/0 conn(0x7f9a94066370 0x7f9a94069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:33.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a99cd4700 1 -- 172.21.15.29:0/3617112941 >> v1:172.21.15.29:6789/0 conn(0x7f9a94069b70 legacy=0x7f9a940bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:33.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a99cd4700 1 -- 172.21.15.29:0/3617112941 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a940c3540 con 0x7f9a940692a0 2024-01-24T14:17:33.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.855+0000 7f9a994d3700 1 --2- 172.21.15.29:0/3617112941 >> v2:172.21.15.163:3300/0 conn(0x7f9a94066370 0x7f9a94069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T14:17:33.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.856+0000 7f9a99cd4700 1 --2- 172.21.15.29:0/3617112941 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a940692a0 0x7f9a94065e80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f9a88009760 tx=0x7f9a88009060).ready entity=mon.2 client_cookie=441f446b310dd467 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:33.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.856+0000 7f9a98cd2700 1 -- 172.21.15.29:0/3617112941 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0017f0 con 0x7f9a940692a0 2024-01-24T14:17:33.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.856+0000 7f9a98cd2700 1 -- 172.21.15.29:0/3617112941 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a88011400 con 0x7f9a940692a0 2024-01-24T14:17:33.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.856+0000 7f9a98cd2700 1 -- 172.21.15.29:0/3617112941 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88011780 con 0x7f9a940692a0 2024-01-24T14:17:33.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.856+0000 7f9a9bf38700 1 -- 172.21.15.29:0/3617112941 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a940692a0 msgr2=0x7f9a94065e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:33.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.856+0000 7f9a9bf38700 1 --2- 172.21.15.29:0/3617112941 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a940692a0 0x7f9a94065e80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f9a88009760 tx=0x7f9a88009060).stop 2024-01-24T14:17:33.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.856+0000 7f9a9bf38700 1 -- 172.21.15.29:0/3617112941 shutdown_connections 2024-01-24T14:17:33.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.856+0000 7f9a9bf38700 1 --2- 172.21.15.29:0/3617112941 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a940692a0 0x7f9a94065e80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:33.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.856+0000 7f9a9bf38700 1 --2- 172.21.15.29:0/3617112941 >> v2:172.21.15.163:3300/0 conn(0x7f9a94066370 0x7f9a94069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:33.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.856+0000 7f9a9bf38700 1 -- 172.21.15.29:0/3617112941 >> 172.21.15.29:0/3617112941 conn(0x7f9a94064390 msgr2=0x7f9a940c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:33.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.856+0000 7f9a9bf38700 1 -- 172.21.15.29:0/3617112941 shutdown_connections 2024-01-24T14:17:33.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.856+0000 7f9a9bf38700 1 -- 172.21.15.29:0/3617112941 wait complete. 2024-01-24T14:17:33.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.858+0000 7f9a9bf38700 1 Processor -- start 2024-01-24T14:17:33.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.858+0000 7f9a9bf38700 1 -- start start 2024-01-24T14:17:33.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.858+0000 7f9a9bf38700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a94066370 0x7f9a940bf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.858+0000 7f9a9bf38700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a940692a0 0x7f9a9413c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.858+0000 7f9a9bf38700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a94069b70 0x7f9a9413d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.858+0000 7f9a9bf38700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940bf310 con 0x7f9a940692a0 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.858+0000 7f9a9bf38700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a94164ec0 con 0x7f9a94066370 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.858+0000 7f9a9bf38700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940bfc90 con 0x7f9a94069b70 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a994d3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a940692a0 0x7f9a9413c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a994d3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a940692a0 0x7f9a9413c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44316/0 (socket says 172.21.15.29:44316) 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a994d3700 1 -- 172.21.15.29:0/285326168 learned_addr learned my addr 172.21.15.29:0/285326168 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a99cd4700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a94066370 0x7f9a940bf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a9a4d5700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a94069b70 0x7f9a9413d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a994d3700 1 -- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a94069b70 msgr2=0x7f9a9413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a994d3700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a94069b70 0x7f9a9413d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a994d3700 1 -- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a94066370 msgr2=0x7f9a940bf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a994d3700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a94066370 0x7f9a940bf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:33.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a994d3700 1 -- 172.21.15.29:0/285326168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900092a0 con 0x7f9a940692a0 2024-01-24T14:17:33.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a994d3700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a940692a0 0x7f9a9413c740 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7f9a9000e960 tx=0x7f9a90009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:33.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a85ffb700 1 -- 172.21.15.29:0/285326168 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90012600 con 0x7f9a940692a0 2024-01-24T14:17:33.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a85ffb700 1 -- 172.21.15.29:0/285326168 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a90012760 con 0x7f9a940692a0 2024-01-24T14:17:33.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a9bf38700 1 -- 172.21.15.29:0/285326168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a880092a0 con 0x7f9a940692a0 2024-01-24T14:17:33.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.859+0000 7f9a85ffb700 1 -- 172.21.15.29:0/285326168 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90012ae0 con 0x7f9a940692a0 2024-01-24T14:17:33.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.860+0000 7f9a9bf38700 1 -- 172.21.15.29:0/285326168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a940becc0 con 0x7f9a940692a0 2024-01-24T14:17:33.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.861+0000 7f9a9bf38700 1 -- 172.21.15.29:0/285326168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a940475e0 con 0x7f9a940692a0 2024-01-24T14:17:33.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.861+0000 7f9a85ffb700 1 -- 172.21.15.29:0/285326168 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 60) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f9a9001b400 con 0x7f9a940692a0 2024-01-24T14:17:33.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.861+0000 7f9a85ffb700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a7805f080 0x7f9a78061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:33.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.862+0000 7f9a85ffb700 1 -- 172.21.15.29:0/285326168 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 7400+0+0 (secure 0 0 0) 0x7f9a9007a8a0 con 0x7f9a940692a0 2024-01-24T14:17:33.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.862+0000 7f9a99cd4700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a7805f080 0x7f9a78061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:33.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.862+0000 7f9a99cd4700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a7805f080 0x7f9a78061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f9a88009760 tx=0x7f9a88016000).ready entity=mgr.7097 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:33.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:33.866+0000 7f9a85ffb700 1 -- 172.21.15.29:0/285326168 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9a9004b7e0 con 0x7f9a940692a0 2024-01-24T14:17:34.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.076+0000 7f9a9bf38700 1 -- 172.21.15.29:0/285326168 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a940c2e10 con 0x7f9a7805f080 2024-01-24T14:17:34.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.081+0000 7f9a85ffb700 1 -- 172.21.15.29:0/285326168 <== mgr.7097 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7f9a940c2e10 con 0x7f9a7805f080 2024-01-24T14:17:34.082 INFO:teuthology.orchestra.run.smithi029.stdout:Retention added to path /snap_test_dir1 2024-01-24T14:17:34.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.083+0000 7f9a9bf38700 1 -- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a7805f080 msgr2=0x7f9a78061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:34.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.083+0000 7f9a9bf38700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a7805f080 0x7f9a78061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f9a88009760 tx=0x7f9a88016000).stop 2024-01-24T14:17:34.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.083+0000 7f9a9bf38700 1 -- 172.21.15.29:0/285326168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a940692a0 msgr2=0x7f9a9413c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.083+0000 7f9a9bf38700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a940692a0 0x7f9a9413c740 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7f9a9000e960 tx=0x7f9a90009060).stop 2024-01-24T14:17:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.083+0000 7f9a9bf38700 1 -- 172.21.15.29:0/285326168 shutdown_connections 2024-01-24T14:17:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.083+0000 7f9a9bf38700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a94069b70 0x7f9a9413d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.083+0000 7f9a9bf38700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a94066370 0x7f9a940bf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.083+0000 7f9a9bf38700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9a7805f080 0x7f9a78061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.083+0000 7f9a9bf38700 1 --2- 172.21.15.29:0/285326168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a940692a0 0x7f9a9413c740 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.083+0000 7f9a9bf38700 1 -- 172.21.15.29:0/285326168 >> 172.21.15.29:0/285326168 conn(0x7f9a94064390 msgr2=0x7f9a940c6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.083+0000 7f9a9bf38700 1 -- 172.21.15.29:0/285326168 shutdown_connections 2024-01-24T14:17:34.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.083+0000 7f9a9bf38700 1 -- 172.21.15.29:0/285326168 wait complete. 2024-01-24T14:17:34.097 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2024-01-24T14:17:34.098 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2024-01-24T14:17:34.098 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json /snap_test_dir1 2024-01-24T14:17:34.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.232+0000 7fa724799700 1 Processor -- start 2024-01-24T14:17:34.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.232+0000 7fa724799700 1 -- start start 2024-01-24T14:17:34.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa724799700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa71c13a6f0 0x7fa71c068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:34.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa724799700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa71c0638c0 0x7fa71c066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:34.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa724799700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa71c066940 0x7fa71c0be860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:34.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa724799700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71c0643f0 con 0x7fa71c13a6f0 2024-01-24T14:17:34.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa724799700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71c0bdb30 con 0x7fa71c066940 2024-01-24T14:17:34.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa724799700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c0be3d0 con 0x7fa71c0638c0 2024-01-24T14:17:34.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa722535700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa71c13a6f0 0x7fa71c068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa722535700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa71c13a6f0 0x7fa71c068660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44328/0 (socket says 172.21.15.29:44328) 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa722535700 1 -- 172.21.15.29:0/1591226382 learned_addr learned my addr 172.21.15.29:0/1591226382 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa721d34700 1 --2- 172.21.15.29:0/1591226382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa71c0638c0 0x7fa71c066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa722d36700 1 --2- 172.21.15.29:0/1591226382 >> v2:172.21.15.163:3300/0 conn(0x7fa71c066940 0x7fa71c0be860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa722535700 1 -- 172.21.15.29:0/1591226382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa71c0638c0 msgr2=0x7fa71c066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa722535700 1 --2- 172.21.15.29:0/1591226382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa71c0638c0 0x7fa71c066450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa722535700 1 -- 172.21.15.29:0/1591226382 >> v2:172.21.15.163:3300/0 conn(0x7fa71c066940 msgr2=0x7fa71c0be860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa722535700 1 --2- 172.21.15.29:0/1591226382 >> v2:172.21.15.163:3300/0 conn(0x7fa71c066940 0x7fa71c0be860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa722535700 1 -- 172.21.15.29:0/1591226382 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c066dd0 con 0x7fa71c13a6f0 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.233+0000 7fa722535700 1 --2- 172.21.15.29:0/1591226382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa71c13a6f0 0x7fa71c068660 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7fa71c0a84b0 tx=0x7fa70c0090f0).ready entity=mon.0 client_cookie=4abd1ce0e072cf4d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.234+0000 7fa721533700 1 -- 172.21.15.29:0/1591226382 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c00f510 con 0x7fa71c13a6f0 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.234+0000 7fa721533700 1 -- 172.21.15.29:0/1591226382 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70c00f670 con 0x7fa71c13a6f0 2024-01-24T14:17:34.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.234+0000 7fa721533700 1 -- 172.21.15.29:0/1591226382 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c00f9f0 con 0x7fa71c13a6f0 2024-01-24T14:17:34.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.234+0000 7fa724799700 1 -- 172.21.15.29:0/1591226382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa71c13a6f0 msgr2=0x7fa71c068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:34.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.234+0000 7fa724799700 1 --2- 172.21.15.29:0/1591226382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa71c13a6f0 0x7fa71c068660 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7fa71c0a84b0 tx=0x7fa70c0090f0).stop 2024-01-24T14:17:34.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.234+0000 7fa724799700 1 -- 172.21.15.29:0/1591226382 shutdown_connections 2024-01-24T14:17:34.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.234+0000 7fa724799700 1 --2- 172.21.15.29:0/1591226382 >> v2:172.21.15.163:3300/0 conn(0x7fa71c066940 0x7fa71c0be860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.234+0000 7fa724799700 1 --2- 172.21.15.29:0/1591226382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa71c13a6f0 0x7fa71c068660 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.234+0000 7fa724799700 1 --2- 172.21.15.29:0/1591226382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa71c0638c0 0x7fa71c066450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.234+0000 7fa724799700 1 -- 172.21.15.29:0/1591226382 >> 172.21.15.29:0/1591226382 conn(0x7fa71c0c8bd0 msgr2=0x7fa71c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:34.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.234+0000 7fa724799700 1 -- 172.21.15.29:0/1591226382 shutdown_connections 2024-01-24T14:17:34.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.234+0000 7fa724799700 1 -- 172.21.15.29:0/1591226382 wait complete. 2024-01-24T14:17:34.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.236+0000 7fa724799700 1 Processor -- start 2024-01-24T14:17:34.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.236+0000 7fa724799700 1 -- start start 2024-01-24T14:17:34.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.237+0000 7fa724799700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa71c0638c0 0x7fa71c156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:34.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.237+0000 7fa724799700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa71c066940 0x7fa71c0ac550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:34.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.237+0000 7fa724799700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa71c13a6f0 0x7fa71c0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:34.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.237+0000 7fa724799700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c164880 con 0x7fa71c0638c0 2024-01-24T14:17:34.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.237+0000 7fa724799700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c164b60 con 0x7fa71c066940 2024-01-24T14:17:34.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.237+0000 7fa724799700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c164e40 con 0x7fa71c13a6f0 2024-01-24T14:17:34.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.237+0000 7fa722535700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa71c0638c0 0x7fa71c156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:34.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.237+0000 7fa722535700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa71c0638c0 0x7fa71c156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44332/0 (socket says 172.21.15.29:44332) 2024-01-24T14:17:34.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.237+0000 7fa722535700 1 -- 172.21.15.29:0/1551056980 learned_addr learned my addr 172.21.15.29:0/1551056980 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T14:17:34.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.237+0000 7fa721d34700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa71c066940 0x7fa71c0ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:34.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.237+0000 7fa722d36700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa71c13a6f0 0x7fa71c0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:34.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.238+0000 7fa722535700 1 -- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa71c13a6f0 msgr2=0x7fa71c0adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:34.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.238+0000 7fa722535700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa71c13a6f0 0x7fa71c0adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.238+0000 7fa722535700 1 -- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa71c066940 msgr2=0x7fa71c0ac550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:34.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.238+0000 7fa722535700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa71c066940 0x7fa71c0ac550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.238+0000 7fa722535700 1 -- 172.21.15.29:0/1551056980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c009370 con 0x7fa71c0638c0 2024-01-24T14:17:34.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.238+0000 7fa722535700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa71c0638c0 0x7fa71c156eb0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7fa71c068ff0 tx=0x7fa70c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:34.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.238+0000 7fa7127fc700 1 -- 172.21.15.29:0/1551056980 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c00f7e0 con 0x7fa71c0638c0 2024-01-24T14:17:34.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.238+0000 7fa7127fc700 1 -- 172.21.15.29:0/1551056980 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70c00f940 con 0x7fa71c0638c0 2024-01-24T14:17:34.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.238+0000 7fa724799700 1 -- 172.21.15.29:0/1551056980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71c165120 con 0x7fa71c0638c0 2024-01-24T14:17:34.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.238+0000 7fa7127fc700 1 -- 172.21.15.29:0/1551056980 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c01c670 con 0x7fa71c0638c0 2024-01-24T14:17:34.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.238+0000 7fa724799700 1 -- 172.21.15.29:0/1551056980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71c152cc0 con 0x7fa71c0638c0 2024-01-24T14:17:34.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.239+0000 7fa724799700 1 -- 172.21.15.29:0/1551056980 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa71c06ae00 con 0x7fa71c0638c0 2024-01-24T14:17:34.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.239+0000 7fa7127fc700 1 -- 172.21.15.29:0/1551056980 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 60) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fa70c01ca60 con 0x7fa71c0638c0 2024-01-24T14:17:34.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.240+0000 7fa7127fc700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa70005f080 0x7fa700061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T14:17:34.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.240+0000 7fa7127fc700 1 -- 172.21.15.29:0/1551056980 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 7400+0+0 (secure 0 0 0) 0x7fa70c07c0e0 con 0x7fa71c0638c0 2024-01-24T14:17:34.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.240+0000 7fa721d34700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa70005f080 0x7fa700061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T14:17:34.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.241+0000 7fa721d34700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa70005f080 0x7fa700061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fa71c157610 tx=0x7fa718008040).ready entity=mgr.7097 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T14:17:34.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.245+0000 7fa7127fc700 1 -- 172.21.15.29:0/1551056980 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa70c04d130 con 0x7fa71c0638c0 2024-01-24T14:17:34.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.451+0000 7fa724799700 1 -- 172.21.15.29:0/1551056980 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa71c152cc0 con 0x7fa70005f080 2024-01-24T14:17:34.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.455+0000 7fa7127fc700 1 -- 172.21.15.29:0/1551056980 <== mgr.7097 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7fa71c152cc0 con 0x7fa70005f080 2024-01-24T14:17:34.455 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T14:17:34.455 INFO:teuthology.orchestra.run.smithi029.stdout:{"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2024-01-24T14:17:34.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.456+0000 7fa724799700 1 -- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa70005f080 msgr2=0x7fa700061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:34.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.456+0000 7fa724799700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa70005f080 0x7fa700061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fa71c157610 tx=0x7fa718008040).stop 2024-01-24T14:17:34.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.456+0000 7fa724799700 1 -- 172.21.15.29:0/1551056980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa71c0638c0 msgr2=0x7fa71c156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T14:17:34.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.456+0000 7fa724799700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa71c0638c0 0x7fa71c156eb0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7fa71c068ff0 tx=0x7fa70c017000).stop 2024-01-24T14:17:34.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.456+0000 7fa724799700 1 -- 172.21.15.29:0/1551056980 shutdown_connections 2024-01-24T14:17:34.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.456+0000 7fa724799700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa71c13a6f0 0x7fa71c0adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.456+0000 7fa724799700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa70005f080 0x7fa700061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.456+0000 7fa724799700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa71c0638c0 0x7fa71c156eb0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.456+0000 7fa724799700 1 --2- 172.21.15.29:0/1551056980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa71c066940 0x7fa71c0ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T14:17:34.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.456+0000 7fa724799700 1 -- 172.21.15.29:0/1551056980 >> 172.21.15.29:0/1551056980 conn(0x7fa71c0c8bd0 msgr2=0x7fa71c0c6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T14:17:34.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.456+0000 7fa724799700 1 -- 172.21.15.29:0/1551056980 shutdown_connections 2024-01-24T14:17:34.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T14:17:34.456+0000 7fa724799700 1 -- 172.21.15.29:0/1551056980 wait complete. 2024-01-24T14:17:34.470 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2024-01-24T14:17:34.470 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1M'], 'retention': [{'n': 55}]} 2024-01-24T14:17:59.457 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:17:59.460 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:18:00.086 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:18:00.085+0000 7fc7e8236700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:18:00.087 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:18:00.085+0000 7fc7e8236700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:18:00.087 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:18:00.085+0000 7fc7e8236700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:18:00.087 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:18:00.085+0000 7fc7e8236700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:18:00.087 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T14:18:00.085+0000 7fc7e8236700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T14:18:29.538 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:18:29.541 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:18:59.626 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:18:59.630 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:19:29.755 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:19:29.758 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:19:59.837 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:19:59.841 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:20:29.910 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:20:29.914 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:20:59.994 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:20:59.997 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:21:30.067 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:21:30.070 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:22:00.161 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:22:00.164 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:22:30.230 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:22:30.234 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:23:00.319 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:23:00.323 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:23:30.393 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:23:30.397 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:24:00.498 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:24:00.501 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:24:30.580 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:24:30.583 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:25:00.655 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:25:00.659 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:25:30.747 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:25:30.750 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:26:00.875 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:26:00.878 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:26:30.991 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:26:30.994 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:27:01.056 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:27:01.059 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:27:31.147 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:27:31.150 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:28:01.268 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:28:01.271 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:28:31.353 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:28:31.356 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:29:01.423 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:29:01.427 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:29:31.512 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:29:31.516 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:30:01.648 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:30:01.651 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:30:31.715 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:30:31.719 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:31:01.809 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:31:01.812 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:31:31.885 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:31:31.888 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:32:02.042 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:32:02.046 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:32:32.125 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:32:32.129 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:33:02.226 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:33:02.230 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:33:32.310 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:33:32.313 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:34:02.466 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:34:02.470 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:34:32.552 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:34:32.555 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:35:02.642 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:35:02.646 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:35:32.729 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:35:32.732 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:36:02.817 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:36:02.820 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:36:32.926 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:36:32.930 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:37:03.017 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:37:03.020 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:37:33.101 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:37:33.104 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:38:03.209 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:38:03.212 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:38:33.293 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:38:33.296 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:39:03.376 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:39:03.384 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:39:33.460 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:39:33.463 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:40:03.576 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:40:03.580 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:40:33.653 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:40:33.656 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:41:03.749 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:41:03.753 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:41:33.834 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:41:33.837 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:42:03.952 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:42:03.955 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:42:34.049 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:42:34.052 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:43:04.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:43:04.144 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:43:34.223 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:43:34.226 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:44:04.355 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:44:04.358 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:44:34.439 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:44:34.443 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:45:04.529 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:45:04.532 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:45:34.596 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:45:34.600 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:46:04.748 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:46:04.751 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:46:34.813 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:46:34.817 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:47:04.870 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:47:04.874 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:47:34.954 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:47:34.957 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:48:05.137 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:48:05.141 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:48:35.215 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:48:35.218 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:49:05.295 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:49:05.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:49:35.378 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:49:35.381 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:50:05.502 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:50:05.505 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:50:35.569 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:50:35.573 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:51:05.649 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:51:05.652 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:51:35.747 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:51:35.751 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:52:05.946 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:52:05.949 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:52:36.012 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:52:36.016 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:53:06.095 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:53:06.098 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:53:36.195 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:53:36.198 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:54:06.286 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:54:06.289 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:54:36.379 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:54:36.382 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:55:06.460 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:55:06.464 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:55:36.560 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:55:36.564 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:56:06.647 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:56:06.650 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:56:36.709 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:56:36.712 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:57:06.781 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:57:06.785 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:57:36.866 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:57:36.869 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:58:06.972 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:58:06.975 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:58:37.045 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:58:37.048 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:59:07.132 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:59:07.136 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:59:37.198 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T14:59:37.202 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:00:07.276 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:00:07.279 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:00:37.365 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:00:37.368 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:01:07.436 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:01:07.441 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:01:37.545 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:01:37.548 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:02:07.629 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:02:07.632 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:02:37.707 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:02:37.711 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:03:07.789 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:03:07.792 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:03:37.884 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:03:37.887 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:04:07.958 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:04:07.962 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:04:38.041 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:04:38.044 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:05:08.110 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:05:08.113 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:05:38.181 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:05:38.184 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:06:08.260 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:06:08.264 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:06:38.349 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:06:38.353 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:07:08.420 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:07:08.424 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:07:38.510 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:07:38.513 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:08:08.594 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:08:08.597 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:08:38.713 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:08:38.717 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:09:08.791 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:09:08.794 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:09:38.945 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:09:38.948 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:10:09.022 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:10:09.026 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:10:39.116 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:10:39.119 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:11:09.180 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:11:09.184 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:11:39.250 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:11:39.253 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:12:09.340 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:12:09.343 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:12:39.421 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:12:39.424 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:13:09.498 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:13:09.501 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:13:15.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:15.472 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:13:15.499 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_19_00_UTC 2024-01-24T15:13:15.499 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_20_00_UTC 2024-01-24T15:13:15.499 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_21_00_UTC 2024-01-24T15:13:15.499 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_22_00_UTC 2024-01-24T15:13:15.499 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_23_00_UTC 2024-01-24T15:13:15.499 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_24_00_UTC 2024-01-24T15:13:15.499 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_25_00_UTC 2024-01-24T15:13:15.499 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_26_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_27_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_28_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_29_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_30_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_31_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_32_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_33_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_34_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_35_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_36_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_37_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_38_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_39_00_UTC 2024-01-24T15:13:15.500 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_40_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_41_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_42_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_43_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_44_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_45_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_46_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_47_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_48_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_49_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_50_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_51_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_52_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_53_00_UTC 2024-01-24T15:13:15.501 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_54_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_55_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_56_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_57_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_58_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_59_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_00_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_01_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_02_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_03_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_04_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_05_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_06_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_07_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_08_00_UTC 2024-01-24T15:13:15.502 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_09_00_UTC 2024-01-24T15:13:15.503 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_10_00_UTC 2024-01-24T15:13:15.503 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_11_00_UTC 2024-01-24T15:13:15.503 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_12_00_UTC 2024-01-24T15:13:15.503 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_13_00_UTC 2024-01-24T15:13:15.503 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-14_19_00_UTC', 'scheduled-2024-01-24-14_20_00_UTC', 'scheduled-2024-01-24-14_21_00_UTC', 'scheduled-2024-01-24-14_22_00_UTC', 'scheduled-2024-01-24-14_23_00_UTC', 'scheduled-2024-01-24-14_24_00_UTC', 'scheduled-2024-01-24-14_25_00_UTC', 'scheduled-2024-01-24-14_26_00_UTC', 'scheduled-2024-01-24-14_27_00_UTC', 'scheduled-2024-01-24-14_28_00_UTC', 'scheduled-2024-01-24-14_29_00_UTC', 'scheduled-2024-01-24-14_30_00_UTC', 'scheduled-2024-01-24-14_31_00_UTC', 'scheduled-2024-01-24-14_32_00_UTC', 'scheduled-2024-01-24-14_33_00_UTC', 'scheduled-2024-01-24-14_34_00_UTC', 'scheduled-2024-01-24-14_35_00_UTC', 'scheduled-2024-01-24-14_36_00_UTC', 'scheduled-2024-01-24-14_37_00_UTC', 'scheduled-2024-01-24-14_38_00_UTC', 'scheduled-2024-01-24-14_39_00_UTC', 'scheduled-2024-01-24-14_40_00_UTC', 'scheduled-2024-01-24-14_41_00_UTC', 'scheduled-2024-01-24-14_42_00_UTC', 'scheduled-2024-01-24-14_43_00_UTC', 'scheduled-2024-01-24-14_44_00_UTC', 'scheduled-2024-01-24-14_45_00_UTC', 'scheduled-2024-01-24-14_46_00_UTC', 'scheduled-2024-01-24-14_47_00_UTC', 'scheduled-2024-01-24-14_48_00_UTC', 'scheduled-2024-01-24-14_49_00_UTC', 'scheduled-2024-01-24-14_50_00_UTC', 'scheduled-2024-01-24-14_51_00_UTC', 'scheduled-2024-01-24-14_52_00_UTC', 'scheduled-2024-01-24-14_53_00_UTC', 'scheduled-2024-01-24-14_54_00_UTC', 'scheduled-2024-01-24-14_55_00_UTC', 'scheduled-2024-01-24-14_56_00_UTC', 'scheduled-2024-01-24-14_57_00_UTC', 'scheduled-2024-01-24-14_58_00_UTC', 'scheduled-2024-01-24-14_59_00_UTC', 'scheduled-2024-01-24-15_00_00_UTC', 'scheduled-2024-01-24-15_01_00_UTC', 'scheduled-2024-01-24-15_02_00_UTC', 'scheduled-2024-01-24-15_03_00_UTC', 'scheduled-2024-01-24-15_04_00_UTC', 'scheduled-2024-01-24-15_05_00_UTC', 'scheduled-2024-01-24-15_06_00_UTC', 'scheduled-2024-01-24-15_07_00_UTC', 'scheduled-2024-01-24-15_08_00_UTC', 'scheduled-2024-01-24-15_09_00_UTC', 'scheduled-2024-01-24-15_10_00_UTC', 'scheduled-2024-01-24-15_11_00_UTC', 'scheduled-2024-01-24-15_12_00_UTC', 'scheduled-2024-01-24-15_13_00_UTC'] 2024-01-24T15:13:15.503 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1 2024-01-24T15:13:15.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.679+0000 7f6a864c1700 1 Processor -- start 2024-01-24T15:13:15.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.679+0000 7f6a864c1700 1 -- start start 2024-01-24T15:13:15.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.680+0000 7f6a864c1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a8014b0a0 0x7f6a8014b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:15.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.680+0000 7f6a864c1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a800c4b00 0x7f6a80154a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:15.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.680+0000 7f6a864c1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a801557e0 con 0x7f6a800c3660 2024-01-24T15:13:15.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.680+0000 7f6a864c1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a800c28a0 con 0x7f6a8014b0a0 2024-01-24T15:13:15.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.680+0000 7f6a864c1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800c2b80 con 0x7f6a800c4b00 2024-01-24T15:13:15.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.680+0000 7f6a7ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a8014b0a0 0x7f6a8014b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:15.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.680+0000 7f6a84a5e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a800c4b00 0x7f6a80154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:15.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.680+0000 7f6a7ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a8014b0a0 0x7f6a8014b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:56160/0 (socket says 172.21.15.29:56160) 2024-01-24T15:13:15.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.680+0000 7f6a7f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6a800c3660 0x7f6a800c43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:34594/0 (socket says 172.21.15.29:34594) 2024-01-24T15:13:15.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.680+0000 7f6a84a5e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a800c4b00 0x7f6a80154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:36106/0 (socket says 172.21.15.29:36106) 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.680+0000 7f6a7f7fe700 1 -- 172.21.15.29:0/866666137 learned_addr learned my addr 172.21.15.29:0/866666137 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.681+0000 7f6a7ffff700 1 -- 172.21.15.29:0/866666137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a800c4b00 msgr2=0x7f6a80154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.681+0000 7f6a7ffff700 1 --2- 172.21.15.29:0/866666137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a800c4b00 0x7f6a80154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.681+0000 7f6a7ffff700 1 -- 172.21.15.29:0/866666137 >> v1:172.21.15.163:6789/0 conn(0x7f6a800c3660 legacy=0x7f6a800c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.681+0000 7f6a7ffff700 1 -- 172.21.15.29:0/866666137 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a800c2e60 con 0x7f6a8014b0a0 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.681+0000 7f6a7effd700 1 -- 172.21.15.29:0/866666137 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725818542 0 0) 0x7f6a801557e0 con 0x7f6a800c3660 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.681+0000 7f6a84a5e700 1 --2- 172.21.15.29:0/866666137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a800c4b00 0x7f6a80154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.681+0000 7f6a7ffff700 1 --2- 172.21.15.29:0/866666137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a8014b0a0 0x7f6a8014b480 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f6a70009760 tx=0x7f6a70009060).ready entity=mon.1 client_cookie=7e678733041df7a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.681+0000 7f6a7effd700 1 -- 172.21.15.29:0/866666137 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7000f510 con 0x7f6a8014b0a0 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.681+0000 7f6a7effd700 1 -- 172.21.15.29:0/866666137 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7000f670 con 0x7f6a8014b0a0 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.681+0000 7f6a7effd700 1 -- 172.21.15.29:0/866666137 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7000f9f0 con 0x7f6a8014b0a0 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.682+0000 7f6a864c1700 1 -- 172.21.15.29:0/866666137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a8014b0a0 msgr2=0x7f6a8014b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.682+0000 7f6a864c1700 1 --2- 172.21.15.29:0/866666137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a8014b0a0 0x7f6a8014b480 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f6a70009760 tx=0x7f6a70009060).stop 2024-01-24T15:13:15.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.682+0000 7f6a864c1700 1 -- 172.21.15.29:0/866666137 shutdown_connections 2024-01-24T15:13:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.682+0000 7f6a864c1700 1 --2- 172.21.15.29:0/866666137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a800c4b00 0x7f6a80154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.682+0000 7f6a864c1700 1 --2- 172.21.15.29:0/866666137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a8014b0a0 0x7f6a8014b480 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.682+0000 7f6a864c1700 1 -- 172.21.15.29:0/866666137 >> 172.21.15.29:0/866666137 conn(0x7f6a80067b20 msgr2=0x7f6a800638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.682+0000 7f6a864c1700 1 -- 172.21.15.29:0/866666137 shutdown_connections 2024-01-24T15:13:15.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.682+0000 7f6a864c1700 1 -- 172.21.15.29:0/866666137 wait complete. 2024-01-24T15:13:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.683+0000 7f6a864c1700 1 Processor -- start 2024-01-24T15:13:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.683+0000 7f6a864c1700 1 -- start start 2024-01-24T15:13:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.684+0000 7f6a864c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a800c3660 0x7f6a801685e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.684+0000 7f6a864c1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a800c4b00 0x7f6a800c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.684+0000 7f6a864c1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a8014b0a0 0x7f6a800c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:15.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.684+0000 7f6a864c1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a80148a90 con 0x7f6a8014b0a0 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.684+0000 7f6a864c1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800c72a0 con 0x7f6a800c3660 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.684+0000 7f6a864c1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a80167fd0 con 0x7f6a800c4b00 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.684+0000 7f6a84a5e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a8014b0a0 0x7f6a800c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.684+0000 7f6a84a5e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a8014b0a0 0x7f6a800c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58792/0 (socket says 172.21.15.29:58792) 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.684+0000 7f6a7ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a800c3660 0x7f6a801685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.684+0000 7f6a84a5e700 1 -- 172.21.15.29:0/1992444974 learned_addr learned my addr 172.21.15.29:0/1992444974 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.684+0000 7f6a7f7fe700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a800c4b00 0x7f6a800c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.685+0000 7f6a84a5e700 1 -- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a800c4b00 msgr2=0x7f6a800c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.685+0000 7f6a84a5e700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a800c4b00 0x7f6a800c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.685+0000 7f6a84a5e700 1 -- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a800c3660 msgr2=0x7f6a801685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.685+0000 7f6a84a5e700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a800c3660 0x7f6a801685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.685+0000 7f6a84a5e700 1 -- 172.21.15.29:0/1992444974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a78009a20 con 0x7f6a8014b0a0 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.685+0000 7f6a7ffff700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a800c3660 0x7f6a801685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:15.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.685+0000 7f6a84a5e700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a8014b0a0 0x7f6a800c6db0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f6a78009fc0 tx=0x7f6a7800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:15.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.685+0000 7f6a77fff700 1 -- 172.21.15.29:0/1992444974 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78009d20 con 0x7f6a8014b0a0 2024-01-24T15:13:15.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.685+0000 7f6a77fff700 1 -- 172.21.15.29:0/1992444974 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7800f040 con 0x7f6a8014b0a0 2024-01-24T15:13:15.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.685+0000 7f6a864c1700 1 -- 172.21.15.29:0/1992444974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a700092a0 con 0x7f6a8014b0a0 2024-01-24T15:13:15.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.685+0000 7f6a77fff700 1 -- 172.21.15.29:0/1992444974 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7800cde0 con 0x7f6a8014b0a0 2024-01-24T15:13:15.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.685+0000 7f6a864c1700 1 -- 172.21.15.29:0/1992444974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a80165300 con 0x7f6a8014b0a0 2024-01-24T15:13:15.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.686+0000 7f6a864c1700 1 -- 172.21.15.29:0/1992444974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a800475e0 con 0x7f6a8014b0a0 2024-01-24T15:13:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.687+0000 7f6a77fff700 1 -- 172.21.15.29:0/1992444974 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 63) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f6a78013610 con 0x7f6a8014b0a0 2024-01-24T15:13:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.687+0000 7f6a77fff700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6a6005f2c0 0x7f6a60061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.688+0000 7f6a77fff700 1 -- 172.21.15.29:0/1992444974 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 7546+0+0 (secure 0 0 0) 0x7f6a7807b250 con 0x7f6a8014b0a0 2024-01-24T15:13:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.688+0000 7f6a7ffff700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6a6005f2c0 0x7f6a60061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:15.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.688+0000 7f6a7ffff700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6a6005f2c0 0x7f6a60061780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f6a70009fd0 tx=0x7f6a70017000).ready entity=mgr.7097 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:15.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.692+0000 7f6a77fff700 1 -- 172.21.15.29:0/1992444974 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f6a7804be10 con 0x7f6a8014b0a0 2024-01-24T15:13:15.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.899+0000 7f6a864c1700 1 -- 172.21.15.29:0/1992444974 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a800643a0 con 0x7f6a6005f2c0 2024-01-24T15:13:15.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.901+0000 7f6a77fff700 1 -- 172.21.15.29:0/1992444974 <== mgr.7097 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f6a800643a0 con 0x7f6a6005f2c0 2024-01-24T15:13:15.903 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:15.903 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T14:17:33", "first": "2024-01-24T14:18:00", "last": "2024-01-24T15:13:00", "last_pruned": "2024-01-24T15:13:00", "created_count": 56, "pruned_count": 1, "active": true}] 2024-01-24T15:13:15.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.902+0000 7f6a864c1700 1 -- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6a6005f2c0 msgr2=0x7f6a60061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:15.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.902+0000 7f6a864c1700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6a6005f2c0 0x7f6a60061780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f6a70009fd0 tx=0x7f6a70017000).stop 2024-01-24T15:13:15.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.902+0000 7f6a864c1700 1 -- 172.21.15.29:0/1992444974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a8014b0a0 msgr2=0x7f6a800c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:15.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.902+0000 7f6a864c1700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a8014b0a0 0x7f6a800c6db0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f6a78009fc0 tx=0x7f6a7800b040).stop 2024-01-24T15:13:15.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.903+0000 7f6a864c1700 1 -- 172.21.15.29:0/1992444974 shutdown_connections 2024-01-24T15:13:15.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.903+0000 7f6a864c1700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6a6005f2c0 0x7f6a60061780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:15.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.903+0000 7f6a864c1700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a8014b0a0 0x7f6a800c6db0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:15.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.903+0000 7f6a864c1700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a800c3660 0x7f6a801685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:15.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.903+0000 7f6a864c1700 1 --2- 172.21.15.29:0/1992444974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a800c4b00 0x7f6a800c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:15.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.903+0000 7f6a864c1700 1 -- 172.21.15.29:0/1992444974 >> 172.21.15.29:0/1992444974 conn(0x7f6a80067b20 msgr2=0x7f6a800627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:15.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.903+0000 7f6a864c1700 1 -- 172.21.15.29:0/1992444974 shutdown_connections 2024-01-24T15:13:15.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:15.903+0000 7f6a864c1700 1 -- 172.21.15.29:0/1992444974 wait complete. 2024-01-24T15:13:15.918 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T14:17:33", "first": "2024-01-24T14:18:00", "last": "2024-01-24T15:13:00", "last_pruned": "2024-01-24T15:13:00", "created_count": 56, "pruned_count": 1, "active": true}] 2024-01-24T15:13:15.918 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1M', 'retention': {'n': 55}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T14:17:33', 'first': '2024-01-24T14:18:00', 'last': '2024-01-24T15:13:00', 'last_pruned': '2024-01-24T15:13:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2024-01-24T15:13:15.918 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2024-01-24T15:13:15.918 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2024-01-24T15:13:15.918 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1 2024-01-24T15:13:16.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.050+0000 7f412fb7c700 1 Processor -- start 2024-01-24T15:13:16.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.050+0000 7f412fb7c700 1 -- start start 2024-01-24T15:13:16.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.051+0000 7f412fb7c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41281340a0 0x7f4128151e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:16.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.051+0000 7f412fb7c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f41280c3b20 0x7f41280bcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:16.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.051+0000 7f412fb7c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41280bb940 con 0x7f4128133cc0 2024-01-24T15:13:16.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.051+0000 7f412fb7c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41280c2520 con 0x7f41280c3b20 2024-01-24T15:13:16.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.051+0000 7f412fb7c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412814b940 con 0x7f41281340a0 2024-01-24T15:13:16.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.051+0000 7f412d918700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4128133cc0 0x7f41280c3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38388/0 (socket says 172.21.15.29:38388) 2024-01-24T15:13:16.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.051+0000 7f412e119700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f41280c3b20 0x7f41280bcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:16.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.051+0000 7f412d918700 1 -- 172.21.15.29:0/151771073 learned_addr learned my addr 172.21.15.29:0/151771073 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:16.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.051+0000 7f412d117700 1 --2- 172.21.15.29:0/151771073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41281340a0 0x7f4128151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:16.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.051+0000 7f412e119700 1 -- 172.21.15.29:0/151771073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41281340a0 msgr2=0x7f4128151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:16.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.051+0000 7f412e119700 1 --2- 172.21.15.29:0/151771073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41281340a0 0x7f4128151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:16.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412e119700 1 -- 172.21.15.29:0/151771073 >> v1:172.21.15.29:6789/0 conn(0x7f4128133cc0 legacy=0x7f41280c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:16.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412e119700 1 -- 172.21.15.29:0/151771073 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128152360 con 0x7f41280c3b20 2024-01-24T15:13:16.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412c916700 1 -- 172.21.15.29:0/151771073 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281590370 0 0) 0x7f41280bb940 con 0x7f4128133cc0 2024-01-24T15:13:16.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412e119700 1 --2- 172.21.15.29:0/151771073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41280c3b20 0x7f41280bcae0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f4120009e60 tx=0x7f4120009780).ready entity=mon.0 client_cookie=238cc43ccf1c56c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:16.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412c916700 1 -- 172.21.15.29:0/151771073 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000e070 con 0x7f41280c3b20 2024-01-24T15:13:16.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412c916700 1 -- 172.21.15.29:0/151771073 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412000bb60 con 0x7f41280c3b20 2024-01-24T15:13:16.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412c916700 1 -- 172.21.15.29:0/151771073 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120012410 con 0x7f41280c3b20 2024-01-24T15:13:16.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412fb7c700 1 -- 172.21.15.29:0/151771073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41280c3b20 msgr2=0x7f41280bcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:16.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412fb7c700 1 --2- 172.21.15.29:0/151771073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41280c3b20 0x7f41280bcae0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f4120009e60 tx=0x7f4120009780).stop 2024-01-24T15:13:16.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412fb7c700 1 -- 172.21.15.29:0/151771073 shutdown_connections 2024-01-24T15:13:16.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412fb7c700 1 --2- 172.21.15.29:0/151771073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41280c3b20 0x7f41280bcae0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:16.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412fb7c700 1 --2- 172.21.15.29:0/151771073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41281340a0 0x7f4128151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:16.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.052+0000 7f412fb7c700 1 -- 172.21.15.29:0/151771073 >> 172.21.15.29:0/151771073 conn(0x7f41280627b0 msgr2=0x7f4128062bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:16.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.053+0000 7f412fb7c700 1 -- 172.21.15.29:0/151771073 shutdown_connections 2024-01-24T15:13:16.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.053+0000 7f412fb7c700 1 -- 172.21.15.29:0/151771073 wait complete. 2024-01-24T15:13:16.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.054+0000 7f412fb7c700 1 Processor -- start 2024-01-24T15:13:16.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.054+0000 7f412fb7c700 1 -- start start 2024-01-24T15:13:16.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412fb7c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41280c3b20 0x7f41280ab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:16.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412fb7c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4128133cc0 0x7f41280c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:16.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412fb7c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41281340a0 0x7f41280a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:16.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412fb7c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41280c5f60 con 0x7f4128133cc0 2024-01-24T15:13:16.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412fb7c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41280c6d50 con 0x7f41281340a0 2024-01-24T15:13:16.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412fb7c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41280ab4f0 con 0x7f41280c3b20 2024-01-24T15:13:16.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412d117700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4128133cc0 0x7f41280c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:16.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412d117700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4128133cc0 0x7f41280c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58800/0 (socket says 172.21.15.29:58800) 2024-01-24T15:13:16.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412d117700 1 -- 172.21.15.29:0/6099429 learned_addr learned my addr 172.21.15.29:0/6099429 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:16.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412d918700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41280c3b20 0x7f41280ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412e119700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41281340a0 0x7f41280a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412d117700 1 -- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41280c3b20 msgr2=0x7f41280ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412d117700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41280c3b20 0x7f41280ab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412d117700 1 -- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41281340a0 msgr2=0x7f41280a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412d117700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41281340a0 0x7f41280a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412d117700 1 -- 172.21.15.29:0/6099429 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4120009a20 con 0x7f4128133cc0 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f412d117700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4128133cc0 0x7f41280c71e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f412400e900 tx=0x7f4124009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.055+0000 7f4119ffb700 1 -- 172.21.15.29:0/6099429 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4124009390 con 0x7f4128133cc0 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.056+0000 7f4119ffb700 1 -- 172.21.15.29:0/6099429 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4124012600 con 0x7f4128133cc0 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.056+0000 7f4119ffb700 1 -- 172.21.15.29:0/6099429 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4124012980 con 0x7f4128133cc0 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.056+0000 7f412fb7c700 1 -- 172.21.15.29:0/6099429 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41280c6140 con 0x7f4128133cc0 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.056+0000 7f412fb7c700 1 -- 172.21.15.29:0/6099429 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41280c6700 con 0x7f4128133cc0 2024-01-24T15:13:16.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.057+0000 7f412fb7c700 1 -- 172.21.15.29:0/6099429 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41280c6ee0 con 0x7f4128133cc0 2024-01-24T15:13:16.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.057+0000 7f4119ffb700 1 -- 172.21.15.29:0/6099429 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 63) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f4124012d30 con 0x7f4128133cc0 2024-01-24T15:13:16.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.057+0000 7f4119ffb700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f410c05f2c0 0x7f410c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:16.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.057+0000 7f4119ffb700 1 -- 172.21.15.29:0/6099429 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 7546+0+0 (secure 0 0 0) 0x7f412407ac40 con 0x7f4128133cc0 2024-01-24T15:13:16.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.058+0000 7f412d918700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f410c05f2c0 0x7f410c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:16.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.058+0000 7f412d918700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f410c05f2c0 0x7f410c061780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f411c0011e0 tx=0x7f411c007bb0).ready entity=mgr.7097 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:16.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.062+0000 7f4119ffb700 1 -- 172.21.15.29:0/6099429 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4124080020 con 0x7f4128133cc0 2024-01-24T15:13:16.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.268+0000 7f412fb7c700 1 -- 172.21.15.29:0/6099429 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f41280475e0 con 0x7f410c05f2c0 2024-01-24T15:13:16.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.272+0000 7f4119ffb700 1 -- 172.21.15.29:0/6099429 <== mgr.7097 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7f41280475e0 con 0x7f410c05f2c0 2024-01-24T15:13:16.274 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /snap_test_dir1 2024-01-24T15:13:16.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.274+0000 7f412fb7c700 1 -- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f410c05f2c0 msgr2=0x7f410c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:16.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.274+0000 7f412fb7c700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f410c05f2c0 0x7f410c061780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f411c0011e0 tx=0x7f411c007bb0).stop 2024-01-24T15:13:16.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.274+0000 7f412fb7c700 1 -- 172.21.15.29:0/6099429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4128133cc0 msgr2=0x7f41280c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:16.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.274+0000 7f412fb7c700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4128133cc0 0x7f41280c71e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f412400e900 tx=0x7f4124009060).stop 2024-01-24T15:13:16.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.274+0000 7f412fb7c700 1 -- 172.21.15.29:0/6099429 shutdown_connections 2024-01-24T15:13:16.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.274+0000 7f412fb7c700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41281340a0 0x7f41280a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:16.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.274+0000 7f412fb7c700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41280c3b20 0x7f41280ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:16.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.274+0000 7f412fb7c700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f410c05f2c0 0x7f410c061780 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:16.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.274+0000 7f412fb7c700 1 --2- 172.21.15.29:0/6099429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4128133cc0 0x7f41280c71e0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:16.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.274+0000 7f412fb7c700 1 -- 172.21.15.29:0/6099429 >> 172.21.15.29:0/6099429 conn(0x7f41280627b0 msgr2=0x7f4128062b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:16.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.274+0000 7f412fb7c700 1 -- 172.21.15.29:0/6099429 shutdown_connections 2024-01-24T15:13:16.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:16.274+0000 7f412fb7c700 1 -- 172.21.15.29:0/6099429 wait complete. 2024-01-24T15:13:16.289 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2024-01-24T15:13:16.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:16.289 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:13:16.310 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_19_00_UTC 2024-01-24T15:13:16.311 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_20_00_UTC 2024-01-24T15:13:16.311 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_21_00_UTC 2024-01-24T15:13:16.311 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_22_00_UTC 2024-01-24T15:13:16.311 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_23_00_UTC 2024-01-24T15:13:16.311 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_24_00_UTC 2024-01-24T15:13:16.311 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_25_00_UTC 2024-01-24T15:13:16.311 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_26_00_UTC 2024-01-24T15:13:16.311 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_27_00_UTC 2024-01-24T15:13:16.311 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_28_00_UTC 2024-01-24T15:13:16.312 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_29_00_UTC 2024-01-24T15:13:16.312 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_30_00_UTC 2024-01-24T15:13:16.312 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_31_00_UTC 2024-01-24T15:13:16.312 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_32_00_UTC 2024-01-24T15:13:16.312 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_33_00_UTC 2024-01-24T15:13:16.312 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_34_00_UTC 2024-01-24T15:13:16.312 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_35_00_UTC 2024-01-24T15:13:16.312 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_36_00_UTC 2024-01-24T15:13:16.312 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_37_00_UTC 2024-01-24T15:13:16.312 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_38_00_UTC 2024-01-24T15:13:16.312 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_39_00_UTC 2024-01-24T15:13:16.312 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_40_00_UTC 2024-01-24T15:13:16.313 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_41_00_UTC 2024-01-24T15:13:16.313 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_42_00_UTC 2024-01-24T15:13:16.313 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_43_00_UTC 2024-01-24T15:13:16.313 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_44_00_UTC 2024-01-24T15:13:16.313 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_45_00_UTC 2024-01-24T15:13:16.313 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_46_00_UTC 2024-01-24T15:13:16.313 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_47_00_UTC 2024-01-24T15:13:16.313 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_48_00_UTC 2024-01-24T15:13:16.313 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_49_00_UTC 2024-01-24T15:13:16.313 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_50_00_UTC 2024-01-24T15:13:16.313 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_51_00_UTC 2024-01-24T15:13:16.313 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_52_00_UTC 2024-01-24T15:13:16.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_53_00_UTC 2024-01-24T15:13:16.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_54_00_UTC 2024-01-24T15:13:16.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_55_00_UTC 2024-01-24T15:13:16.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_56_00_UTC 2024-01-24T15:13:16.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_57_00_UTC 2024-01-24T15:13:16.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_58_00_UTC 2024-01-24T15:13:16.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-14_59_00_UTC 2024-01-24T15:13:16.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_00_00_UTC 2024-01-24T15:13:16.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_01_00_UTC 2024-01-24T15:13:16.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_02_00_UTC 2024-01-24T15:13:16.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_03_00_UTC 2024-01-24T15:13:16.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_04_00_UTC 2024-01-24T15:13:16.315 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_05_00_UTC 2024-01-24T15:13:16.315 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_06_00_UTC 2024-01-24T15:13:16.315 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_07_00_UTC 2024-01-24T15:13:16.315 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_08_00_UTC 2024-01-24T15:13:16.315 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_09_00_UTC 2024-01-24T15:13:16.315 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_10_00_UTC 2024-01-24T15:13:16.315 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_11_00_UTC 2024-01-24T15:13:16.315 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_12_00_UTC 2024-01-24T15:13:16.315 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_13_00_UTC 2024-01-24T15:13:16.315 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_19_00_UTC 2024-01-24T15:13:16.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:16.316 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_19_00_UTC) 2024-01-24T15:13:16.380 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_20_00_UTC 2024-01-24T15:13:16.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:16.380 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_20_00_UTC) 2024-01-24T15:13:16.450 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_21_00_UTC 2024-01-24T15:13:16.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:16.451 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_21_00_UTC) 2024-01-24T15:13:16.520 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_22_00_UTC 2024-01-24T15:13:16.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:16.520 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_22_00_UTC) 2024-01-24T15:13:16.589 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_23_00_UTC 2024-01-24T15:13:16.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:16.589 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_23_00_UTC) 2024-01-24T15:13:16.660 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_24_00_UTC 2024-01-24T15:13:16.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:16.660 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_24_00_UTC) 2024-01-24T15:13:16.730 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_25_00_UTC 2024-01-24T15:13:16.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:16.730 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_25_00_UTC) 2024-01-24T15:13:16.802 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_26_00_UTC 2024-01-24T15:13:16.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:16.803 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_26_00_UTC) 2024-01-24T15:13:16.872 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_27_00_UTC 2024-01-24T15:13:16.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:16.872 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_27_00_UTC) 2024-01-24T15:13:16.942 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_28_00_UTC 2024-01-24T15:13:16.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:16.942 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_28_00_UTC) 2024-01-24T15:13:17.012 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_29_00_UTC 2024-01-24T15:13:17.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.012 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_29_00_UTC) 2024-01-24T15:13:17.081 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_30_00_UTC 2024-01-24T15:13:17.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.081 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_30_00_UTC) 2024-01-24T15:13:17.150 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_31_00_UTC 2024-01-24T15:13:17.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.150 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_31_00_UTC) 2024-01-24T15:13:17.220 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_32_00_UTC 2024-01-24T15:13:17.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.221 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_32_00_UTC) 2024-01-24T15:13:17.290 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_33_00_UTC 2024-01-24T15:13:17.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.290 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_33_00_UTC) 2024-01-24T15:13:17.358 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_34_00_UTC 2024-01-24T15:13:17.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.359 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_34_00_UTC) 2024-01-24T15:13:17.428 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_35_00_UTC 2024-01-24T15:13:17.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.429 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_35_00_UTC) 2024-01-24T15:13:17.501 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_36_00_UTC 2024-01-24T15:13:17.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.502 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_36_00_UTC) 2024-01-24T15:13:17.572 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_37_00_UTC 2024-01-24T15:13:17.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.572 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_37_00_UTC) 2024-01-24T15:13:17.648 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_38_00_UTC 2024-01-24T15:13:17.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.648 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_38_00_UTC) 2024-01-24T15:13:17.717 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_39_00_UTC 2024-01-24T15:13:17.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.718 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_39_00_UTC) 2024-01-24T15:13:17.793 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_40_00_UTC 2024-01-24T15:13:17.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.794 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_40_00_UTC) 2024-01-24T15:13:17.865 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_41_00_UTC 2024-01-24T15:13:17.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.865 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_41_00_UTC) 2024-01-24T15:13:17.935 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_42_00_UTC 2024-01-24T15:13:17.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:17.935 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_42_00_UTC) 2024-01-24T15:13:18.005 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_43_00_UTC 2024-01-24T15:13:18.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.005 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_43_00_UTC) 2024-01-24T15:13:18.078 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_44_00_UTC 2024-01-24T15:13:18.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.078 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_44_00_UTC) 2024-01-24T15:13:18.147 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_45_00_UTC 2024-01-24T15:13:18.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.147 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_45_00_UTC) 2024-01-24T15:13:18.216 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_46_00_UTC 2024-01-24T15:13:18.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.216 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_46_00_UTC) 2024-01-24T15:13:18.285 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_47_00_UTC 2024-01-24T15:13:18.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.286 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_47_00_UTC) 2024-01-24T15:13:18.355 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_48_00_UTC 2024-01-24T15:13:18.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.355 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_48_00_UTC) 2024-01-24T15:13:18.423 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_49_00_UTC 2024-01-24T15:13:18.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.424 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_49_00_UTC) 2024-01-24T15:13:18.493 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_50_00_UTC 2024-01-24T15:13:18.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.493 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_50_00_UTC) 2024-01-24T15:13:18.562 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_51_00_UTC 2024-01-24T15:13:18.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.562 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_51_00_UTC) 2024-01-24T15:13:18.657 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_52_00_UTC 2024-01-24T15:13:18.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.658 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_52_00_UTC) 2024-01-24T15:13:18.710 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_53_00_UTC 2024-01-24T15:13:18.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.710 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_53_00_UTC) 2024-01-24T15:13:18.768 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_54_00_UTC 2024-01-24T15:13:18.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.768 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_54_00_UTC) 2024-01-24T15:13:18.792 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_55_00_UTC 2024-01-24T15:13:18.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.793 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_55_00_UTC) 2024-01-24T15:13:18.858 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_56_00_UTC 2024-01-24T15:13:18.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.859 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_56_00_UTC) 2024-01-24T15:13:18.927 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_57_00_UTC 2024-01-24T15:13:18.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.927 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_57_00_UTC) 2024-01-24T15:13:18.995 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_58_00_UTC 2024-01-24T15:13:18.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:18.995 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_58_00_UTC) 2024-01-24T15:13:19.062 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-14_59_00_UTC 2024-01-24T15:13:19.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.062 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-14_59_00_UTC) 2024-01-24T15:13:19.132 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_00_00_UTC 2024-01-24T15:13:19.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.132 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_00_00_UTC) 2024-01-24T15:13:19.200 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_01_00_UTC 2024-01-24T15:13:19.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.200 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_01_00_UTC) 2024-01-24T15:13:19.268 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_02_00_UTC 2024-01-24T15:13:19.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.269 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_02_00_UTC) 2024-01-24T15:13:19.336 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_03_00_UTC 2024-01-24T15:13:19.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.336 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_03_00_UTC) 2024-01-24T15:13:19.403 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_04_00_UTC 2024-01-24T15:13:19.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.404 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_04_00_UTC) 2024-01-24T15:13:19.470 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_05_00_UTC 2024-01-24T15:13:19.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.471 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_05_00_UTC) 2024-01-24T15:13:19.538 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_06_00_UTC 2024-01-24T15:13:19.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.538 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_06_00_UTC) 2024-01-24T15:13:19.605 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_07_00_UTC 2024-01-24T15:13:19.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.606 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_07_00_UTC) 2024-01-24T15:13:19.687 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_08_00_UTC 2024-01-24T15:13:19.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.687 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_08_00_UTC) 2024-01-24T15:13:19.754 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_09_00_UTC 2024-01-24T15:13:19.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.754 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_09_00_UTC) 2024-01-24T15:13:19.820 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_10_00_UTC 2024-01-24T15:13:19.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.820 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_10_00_UTC) 2024-01-24T15:13:19.886 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_11_00_UTC 2024-01-24T15:13:19.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.887 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_11_00_UTC) 2024-01-24T15:13:19.965 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_12_00_UTC 2024-01-24T15:13:19.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:19.966 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_12_00_UTC) 2024-01-24T15:13:20.032 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_13_00_UTC 2024-01-24T15:13:20.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:20.032 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_13_00_UTC) 2024-01-24T15:13:20.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:20.100 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-01-24T15:13:20.168 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-01-24T15:13:20.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.348+0000 7f2cfc140700 1 Processor -- start 2024-01-24T15:13:20.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.348+0000 7f2cfc140700 1 -- start start 2024-01-24T15:13:20.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.349+0000 7f2cfc140700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2cf414b330 0x7f2cf41549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:20.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.349+0000 7f2cfc140700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf414b710 0x7f2cf4154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:20.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.349+0000 7f2cfc140700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf4155840 con 0x7f2cf40c2c50 2024-01-24T15:13:20.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.349+0000 7f2cfc140700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf4155390 con 0x7f2cf414b330 2024-01-24T15:13:20.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.349+0000 7f2cfc140700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf4069280 con 0x7f2cf414b710 2024-01-24T15:13:20.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.349+0000 7f2cf9edc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2cf414b330 0x7f2cf41549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:20.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.349+0000 7f2cf9edc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2cf414b330 0x7f2cf41549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58812/0 (socket says 172.21.15.29:58812) 2024-01-24T15:13:20.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.349+0000 7f2cf9edc700 1 -- 172.21.15.29:0/1347602609 learned_addr learned my addr 172.21.15.29:0/1347602609 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:20.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.349+0000 7f2cf96db700 1 --2- 172.21.15.29:0/1347602609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf414b710 0x7f2cf4154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:20.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.349+0000 7f2cf9edc700 1 -- 172.21.15.29:0/1347602609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf414b710 msgr2=0x7f2cf4154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:20.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.349+0000 7f2cf9edc700 1 --2- 172.21.15.29:0/1347602609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf414b710 0x7f2cf4154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:20.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.349+0000 7f2cf9edc700 1 -- 172.21.15.29:0/1347602609 >> v1:172.21.15.163:6789/0 conn(0x7f2cf40c2c50 legacy=0x7f2cf40c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:20.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.350+0000 7f2cf9edc700 1 -- 172.21.15.29:0/1347602609 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40c3910 con 0x7f2cf414b330 2024-01-24T15:13:20.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.350+0000 7f2cf96db700 1 --2- 172.21.15.29:0/1347602609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf414b710 0x7f2cf4154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:20.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.350+0000 7f2cf9edc700 1 --2- 172.21.15.29:0/1347602609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cf414b330 0x7f2cf41549b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7f2cf0006fd0 tx=0x7f2cf00090f0).ready entity=mon.0 client_cookie=73149c334fe53fad server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:20.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.350+0000 7f2cf8eda700 1 -- 172.21.15.29:0/1347602609 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf000b600 con 0x7f2cf414b330 2024-01-24T15:13:20.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.350+0000 7f2cf8eda700 1 -- 172.21.15.29:0/1347602609 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf000b760 con 0x7f2cf414b330 2024-01-24T15:13:20.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.350+0000 7f2cf8eda700 1 -- 172.21.15.29:0/1347602609 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf000bae0 con 0x7f2cf414b330 2024-01-24T15:13:20.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.350+0000 7f2cfc140700 1 -- 172.21.15.29:0/1347602609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cf414b330 msgr2=0x7f2cf41549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:20.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.350+0000 7f2cfc140700 1 --2- 172.21.15.29:0/1347602609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cf414b330 0x7f2cf41549b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7f2cf0006fd0 tx=0x7f2cf00090f0).stop 2024-01-24T15:13:20.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.350+0000 7f2cfc140700 1 -- 172.21.15.29:0/1347602609 shutdown_connections 2024-01-24T15:13:20.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.350+0000 7f2cfc140700 1 --2- 172.21.15.29:0/1347602609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cf414b330 0x7f2cf41549b0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:20.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.350+0000 7f2cfc140700 1 --2- 172.21.15.29:0/1347602609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf414b710 0x7f2cf4154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:20.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.350+0000 7f2cfc140700 1 -- 172.21.15.29:0/1347602609 >> 172.21.15.29:0/1347602609 conn(0x7f2cf4064390 msgr2=0x7f2cf40c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:20.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.351+0000 7f2cfc140700 1 -- 172.21.15.29:0/1347602609 shutdown_connections 2024-01-24T15:13:20.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.351+0000 7f2cfc140700 1 -- 172.21.15.29:0/1347602609 wait complete. 2024-01-24T15:13:20.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.352+0000 7f2cfc140700 1 Processor -- start 2024-01-24T15:13:20.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.352+0000 7f2cfc140700 1 -- start start 2024-01-24T15:13:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.353+0000 7f2cfc140700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40c2c50 0x7f2cf413e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.353+0000 7f2cfc140700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cf414b330 0x7f2cf41400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.353+0000 7f2cfc140700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf414b710 0x7f2cf4140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.353+0000 7f2cfc140700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40aaeb0 con 0x7f2cf414b330 2024-01-24T15:13:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.353+0000 7f2cfc140700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf4140bc0 con 0x7f2cf40c2c50 2024-01-24T15:13:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.353+0000 7f2cfc140700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf4140e60 con 0x7f2cf414b710 2024-01-24T15:13:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.353+0000 7f2cf96db700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cf414b330 0x7f2cf41400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.353+0000 7f2cf96db700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cf414b330 0x7f2cf41400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58828/0 (socket says 172.21.15.29:58828) 2024-01-24T15:13:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.353+0000 7f2cf96db700 1 -- 172.21.15.29:0/3739720791 learned_addr learned my addr 172.21.15.29:0/3739720791 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:20.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2cf9edc700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40c2c50 0x7f2cf413e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2cf96db700 1 -- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf414b710 msgr2=0x7f2cf4140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2cfa6dd700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf414b710 0x7f2cf4140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2cf96db700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf414b710 0x7f2cf4140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2cf96db700 1 -- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40c2c50 msgr2=0x7f2cf413e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2cf96db700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40c2c50 0x7f2cf413e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2cf96db700 1 -- 172.21.15.29:0/3739720791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf0009370 con 0x7f2cf414b330 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2cf9edc700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40c2c50 0x7f2cf413e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2cfa6dd700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf414b710 0x7f2cf4140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2cf96db700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cf414b330 0x7f2cf41400a0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7f2ce400d860 tx=0x7f2ce4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2ce9ffb700 1 -- 172.21.15.29:0/3739720791 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4011590 con 0x7f2cf414b330 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2ce9ffb700 1 -- 172.21.15.29:0/3739720791 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce40116f0 con 0x7f2cf414b330 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2cfc140700 1 -- 172.21.15.29:0/3739720791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf4141140 con 0x7f2cf414b330 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2ce9ffb700 1 -- 172.21.15.29:0/3739720791 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4011a70 con 0x7f2cf414b330 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.354+0000 7f2cfc140700 1 -- 172.21.15.29:0/3739720791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf413e7e0 con 0x7f2cf414b330 2024-01-24T15:13:20.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.355+0000 7f2cfc140700 1 -- 172.21.15.29:0/3739720791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf40ab060 con 0x7f2cf414b330 2024-01-24T15:13:20.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.356+0000 7f2ce9ffb700 1 -- 172.21.15.29:0/3739720791 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 63) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f2ce4012420 con 0x7f2cf414b330 2024-01-24T15:13:20.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.356+0000 7f2ce9ffb700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2cd805f2c0 0x7f2cd8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:20.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.356+0000 7f2ce9ffb700 1 -- 172.21.15.29:0/3739720791 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7546+0+0 (secure 0 0 0) 0x7f2ce407a4f0 con 0x7f2cf414b330 2024-01-24T15:13:20.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.356+0000 7f2cf9edc700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2cd805f2c0 0x7f2cd8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:20.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.357+0000 7f2cf9edc700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2cd805f2c0 0x7f2cd8061780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f2cf0009130 tx=0x7f2cf0017000).ready entity=mgr.7097 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:20.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.361+0000 7f2ce9ffb700 1 -- 172.21.15.29:0/3739720791 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2cf40ab060 con 0x7f2cf414b330 2024-01-24T15:13:20.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.594+0000 7f2cfc140700 1 -- 172.21.15.29:0/3739720791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f2cf406a950 con 0x7f2cf414b330 2024-01-24T15:13:20.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.627+0000 7f2ce9ffb700 1 -- 172.21.15.29:0/3739720791 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 64) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f2ce4077bb0 con 0x7f2cf414b330 2024-01-24T15:13:20.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.628+0000 7f2ce9ffb700 1 -- 172.21.15.29:0/3739720791 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v64) v1 ==== 93+0+0 (secure 0 0 0) 0x7f2ce4072420 con 0x7f2cf414b330 2024-01-24T15:13:20.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.631+0000 7f2cfc140700 1 -- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2cd805f2c0 msgr2=0x7f2cd8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:20.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.631+0000 7f2cfc140700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2cd805f2c0 0x7f2cd8061780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f2cf0009130 tx=0x7f2cf0017000).stop 2024-01-24T15:13:20.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.631+0000 7f2cfc140700 1 -- 172.21.15.29:0/3739720791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cf414b330 msgr2=0x7f2cf41400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:20.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.631+0000 7f2cfc140700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cf414b330 0x7f2cf41400a0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7f2ce400d860 tx=0x7f2ce4008040).stop 2024-01-24T15:13:20.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.631+0000 7f2cfc140700 1 -- 172.21.15.29:0/3739720791 shutdown_connections 2024-01-24T15:13:20.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.631+0000 7f2cfc140700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf414b710 0x7f2cf4140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:20.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.631+0000 7f2cfc140700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40c2c50 0x7f2cf413e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:20.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.631+0000 7f2cfc140700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2cd805f2c0 0x7f2cd8061780 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:20.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.631+0000 7f2cfc140700 1 --2- 172.21.15.29:0/3739720791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2cf414b330 0x7f2cf41400a0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:20.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.631+0000 7f2cfc140700 1 -- 172.21.15.29:0/3739720791 >> 172.21.15.29:0/3739720791 conn(0x7f2cf4064390 msgr2=0x7f2cf4153630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:20.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.631+0000 7f2cfc140700 1 -- 172.21.15.29:0/3739720791 shutdown_connections 2024-01-24T15:13:20.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:20.631+0000 7f2cfc140700 1 -- 172.21.15.29:0/3739720791 wait complete. 2024-01-24T15:13:20.654 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T15:13:20.654 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:13:20.717 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:13:20.764 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T15:13:20.764 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-24T15:13:20.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:20.764 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-24T15:13:20.813 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:20.811+0000 7f9a21a0e680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:13:20.821 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T15:13:20.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:13:20.821 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-24T15:13:20.827 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:20.826+0000 7f6508dce680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:13:20.844 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:13:20.895 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:20.892+0000 7f9a21a0e680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:13:20.907 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:20.906+0000 7f6508dce680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:13:21.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.016+0000 7f3fe045c700 1 Processor -- start 2024-01-24T15:13:21.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.016+0000 7f3fe045c700 1 -- start start 2024-01-24T15:13:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.017+0000 7f3fe045c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3fd80c5410 0x7f3fd80aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.017+0000 7f3fe045c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fd8154750 0x7f3fd80631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fe045c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd80aa6c0 con 0x7f3fd8154370 2024-01-24T15:13:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fe045c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd8064240 con 0x7f3fd80c5410 2024-01-24T15:13:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fe045c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80cb5c0 con 0x7f3fd8154750 2024-01-24T15:13:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fdd9f7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3fd80c5410 0x7f3fd80aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fdd9f7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3fd80c5410 0x7f3fd80aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58842/0 (socket says 172.21.15.29:58842) 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fde1f8700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3fd8154370 0x7f3fd80ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38402/0 (socket says 172.21.15.29:38402) 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fdd9f7700 1 -- 172.21.15.29:0/3394529804 learned_addr learned my addr 172.21.15.29:0/3394529804 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fde9f9700 1 --2- 172.21.15.29:0/3394529804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fd8154750 0x7f3fd80631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fdd9f7700 1 -- 172.21.15.29:0/3394529804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fd8154750 msgr2=0x7f3fd80631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fdd9f7700 1 --2- 172.21.15.29:0/3394529804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fd8154750 0x7f3fd80631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fdd9f7700 1 -- 172.21.15.29:0/3394529804 >> v1:172.21.15.29:6789/0 conn(0x7f3fd8154370 legacy=0x7f3fd80ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fdd9f7700 1 -- 172.21.15.29:0/3394529804 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd80cb8a0 con 0x7f3fd80c5410 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fdd1f6700 1 -- 172.21.15.29:0/3394529804 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242168395 0 0) 0x7f3fd80aa6c0 con 0x7f3fd8154370 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fdd9f7700 1 --2- 172.21.15.29:0/3394529804 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fd80c5410 0x7f3fd80aa0e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f3fc8006fd0 tx=0x7f3fc80090f0).ready entity=mon.0 client_cookie=20a701fbe64ae9fd server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.018+0000 7f3fdd1f6700 1 -- 172.21.15.29:0/3394529804 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc800b580 con 0x7f3fd80c5410 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.019+0000 7f3fdd1f6700 1 -- 172.21.15.29:0/3394529804 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc800b6e0 con 0x7f3fd80c5410 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.019+0000 7f3fdd1f6700 1 -- 172.21.15.29:0/3394529804 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc800ba60 con 0x7f3fd80c5410 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.019+0000 7f3fe045c700 1 -- 172.21.15.29:0/3394529804 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fd80c5410 msgr2=0x7f3fd80aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.019+0000 7f3fe045c700 1 --2- 172.21.15.29:0/3394529804 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fd80c5410 0x7f3fd80aa0e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f3fc8006fd0 tx=0x7f3fc80090f0).stop 2024-01-24T15:13:21.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.019+0000 7f3fe045c700 1 -- 172.21.15.29:0/3394529804 shutdown_connections 2024-01-24T15:13:21.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.019+0000 7f3fe045c700 1 --2- 172.21.15.29:0/3394529804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fd8154750 0x7f3fd80631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.019+0000 7f3fe045c700 1 --2- 172.21.15.29:0/3394529804 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fd80c5410 0x7f3fd80aa0e0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.019+0000 7f3fe045c700 1 -- 172.21.15.29:0/3394529804 >> 172.21.15.29:0/3394529804 conn(0x7f3fd805ec80 msgr2=0x7f3fd80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:21.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.019+0000 7f3fe045c700 1 -- 172.21.15.29:0/3394529804 shutdown_connections 2024-01-24T15:13:21.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.019+0000 7f3fe045c700 1 -- 172.21.15.29:0/3394529804 wait complete. 2024-01-24T15:13:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.021+0000 7f3fe045c700 1 Processor -- start 2024-01-24T15:13:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.021+0000 7f3fe045c700 1 -- start start 2024-01-24T15:13:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.021+0000 7f3fe045c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fd80c5410 0x7f3fd814cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.021+0000 7f3fe045c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fd8154370 0x7f3fd80addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.021+0000 7f3fe045c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fd8154750 0x7f3fd80ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.021+0000 7f3fe045c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80ae7a0 con 0x7f3fd80c5410 2024-01-24T15:13:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.021+0000 7f3fe045c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd8155a40 con 0x7f3fd8154370 2024-01-24T15:13:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.021+0000 7f3fe045c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd8159350 con 0x7f3fd8154750 2024-01-24T15:13:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.021+0000 7f3fde1f8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fd80c5410 0x7f3fd814cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fde1f8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fd80c5410 0x7f3fd814cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58844/0 (socket says 172.21.15.29:58844) 2024-01-24T15:13:21.025 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fde1f8700 1 -- 172.21.15.29:0/2989405779 learned_addr learned my addr 172.21.15.29:0/2989405779 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fde9f9700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fd8154750 0x7f3fd80ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fdd9f7700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fd8154370 0x7f3fd80addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fde1f8700 1 -- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fd8154750 msgr2=0x7f3fd80ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fde1f8700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fd8154750 0x7f3fd80ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fde1f8700 1 -- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fd8154370 msgr2=0x7f3fd80addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fde1f8700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fd8154370 0x7f3fd80addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fde1f8700 1 -- 172.21.15.29:0/2989405779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8009370 con 0x7f3fd80c5410 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fde1f8700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fd80c5410 0x7f3fd814cf00 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f3fd400c810 tx=0x7f3fd4007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fce7fc700 1 -- 172.21.15.29:0/2989405779 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd40072e0 con 0x7f3fd80c5410 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fce7fc700 1 -- 172.21.15.29:0/2989405779 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd4007440 con 0x7f3fd80c5410 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fe045c700 1 -- 172.21.15.29:0/2989405779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd8159630 con 0x7f3fd80c5410 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fce7fc700 1 -- 172.21.15.29:0/2989405779 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4005670 con 0x7f3fd80c5410 2024-01-24T15:13:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.022+0000 7f3fe045c700 1 -- 172.21.15.29:0/2989405779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd8155c50 con 0x7f3fd80c5410 2024-01-24T15:13:21.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.024+0000 7f3fce7fc700 1 -- 172.21.15.29:0/2989405779 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 64) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f3fd4005a20 con 0x7f3fd80c5410 2024-01-24T15:13:21.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.024+0000 7f3fe045c700 1 -- 172.21.15.29:0/2989405779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd8159b50 con 0x7f3fd80c5410 2024-01-24T15:13:21.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.024+0000 7f3fce7fc700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3fbc05f270 0x7f3fbc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.024+0000 7f3fdd9f7700 1 -- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3fbc05f270 msgr2=0x7f3fbc061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:21.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.024+0000 7f3fce7fc700 1 -- 172.21.15.29:0/2989405779 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 7546+0+0 (secure 0 0 0) 0x7f3fd407a010 con 0x7f3fd80c5410 2024-01-24T15:13:21.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.024+0000 7f3fdd9f7700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3fbc05f270 0x7f3fbc061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:21.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.029+0000 7f3fce7fc700 1 -- 172.21.15.29:0/2989405779 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3fd404abd0 con 0x7f3fd80c5410 2024-01-24T15:13:21.195 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:21.192+0000 7f9a21a0e680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:13:21.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.225+0000 7f3fdd9f7700 1 -- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3fbc05f270 msgr2=0x7f3fbc061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:21.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.225+0000 7f3fdd9f7700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3fbc05f270 0x7f3fbc061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:13:21.279 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:21.277+0000 7f6508dce680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:13:21.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.313+0000 7f3fe045c700 1 -- 172.21.15.29:0/2989405779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fd80475e0 con 0x7f3fd80c5410 2024-01-24T15:13:21.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.314+0000 7f3fce7fc700 1 -- 172.21.15.29:0/2989405779 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f3fd404e1f0 con 0x7f3fd80c5410 2024-01-24T15:13:21.316 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:21.316 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2106677945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2106677945},{"type":"v1","addr":"172.21.15.163:6836","nonce":2106677945}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7786,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1964541525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1964541525},{"type":"v1","addr":"172.21.15.163:6837","nonce":1964541525}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4162831288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4162831288},{"type":"v1","addr":"172.21.15.29:6835","nonce":4162831288}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:17:20.414311+0000","modified":"2024-01-24T14:17:23.419003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8166":{"gid":8166,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/1312733772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1312733772},{"type":"v1","addr":"172.21.15.29:6834","nonce":1312733772}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-24T15:13:21.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.316+0000 7f3fe045c700 1 -- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3fbc05f270 msgr2=0x7f3fbc061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:21.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.316+0000 7f3fe045c700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3fbc05f270 0x7f3fbc061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:21.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.316+0000 7f3fe045c700 1 -- 172.21.15.29:0/2989405779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fd80c5410 msgr2=0x7f3fd814cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.316+0000 7f3fe045c700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fd80c5410 0x7f3fd814cf00 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f3fd400c810 tx=0x7f3fd4007ba0).stop 2024-01-24T15:13:21.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.316+0000 7f3fe045c700 1 -- 172.21.15.29:0/2989405779 shutdown_connections 2024-01-24T15:13:21.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.316+0000 7f3fe045c700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fd8154750 0x7f3fd80ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.316+0000 7f3fe045c700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3fbc05f270 0x7f3fbc061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:21.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.316+0000 7f3fe045c700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fd80c5410 0x7f3fd814cf00 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.316+0000 7f3fe045c700 1 --2- 172.21.15.29:0/2989405779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fd8154370 0x7f3fd80addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.316+0000 7f3fe045c700 1 -- 172.21.15.29:0/2989405779 >> 172.21.15.29:0/2989405779 conn(0x7f3fd805ec80 msgr2=0x7f3fd814d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:21.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.316+0000 7f3fe045c700 1 -- 172.21.15.29:0/2989405779 shutdown_connections 2024-01-24T15:13:21.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.316+0000 7f3fe045c700 1 -- 172.21.15.29:0/2989405779 wait complete. 2024-01-24T15:13:21.320 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 66 2024-01-24T15:13:21.332 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:13:21.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.515+0000 7ff614850700 1 Processor -- start 2024-01-24T15:13:21.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.515+0000 7ff614850700 1 -- start start 2024-01-24T15:13:21.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.516+0000 7ff614850700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6101540a0 0x7ff6100c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.516+0000 7ff614850700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff610062ec0 0x7ff610069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.516+0000 7ff614850700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff610065360 con 0x7ff6101550f0 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.516+0000 7ff614850700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6100a74f0 con 0x7ff610062ec0 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.516+0000 7ff614850700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100a6ce0 con 0x7ff6101540a0 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60ed9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff610062ec0 0x7ff610069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60ed9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff610062ec0 0x7ff610069240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58854/0 (socket says 172.21.15.29:58854) 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60ed9d700 1 -- 172.21.15.29:0/2493163271 learned_addr learned my addr 172.21.15.29:0/2493163271 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60e59c700 1 --2- 172.21.15.29:0/2493163271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6101540a0 0x7ff6100c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60d59a700 1 -- 172.21.15.29:0/2493163271 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129451604 0 0) 0x7ff610065360 con 0x7ff6101550f0 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60d59a700 1 -- 172.21.15.29:0/2493163271 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f80033f0 con 0x7ff6101550f0 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60ed9d700 1 -- 172.21.15.29:0/2493163271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6101540a0 msgr2=0x7ff6100c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60ed9d700 1 --2- 172.21.15.29:0/2493163271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6101540a0 0x7ff6100c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60ed9d700 1 -- 172.21.15.29:0/2493163271 >> v1:172.21.15.163:6789/0 conn(0x7ff6101550f0 legacy=0x7ff6100627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60ed9d700 1 -- 172.21.15.29:0/2493163271 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6100636f0 con 0x7ff610062ec0 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60e59c700 1 --2- 172.21.15.29:0/2493163271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6101540a0 0x7ff6100c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60ed9d700 1 --2- 172.21.15.29:0/2493163271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff610062ec0 0x7ff610069240 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7ff608009f20 tx=0x7ff608009840).ready entity=mon.0 client_cookie=d4603f9e2463a801 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60d59a700 1 -- 172.21.15.29:0/2493163271 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800e070 con 0x7ff610062ec0 2024-01-24T15:13:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60d59a700 1 -- 172.21.15.29:0/2493163271 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60800b970 con 0x7ff610062ec0 2024-01-24T15:13:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.517+0000 7ff60d59a700 1 -- 172.21.15.29:0/2493163271 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800bcf0 con 0x7ff610062ec0 2024-01-24T15:13:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.518+0000 7ff614850700 1 -- 172.21.15.29:0/2493163271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff610062ec0 msgr2=0x7ff610069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.518+0000 7ff614850700 1 --2- 172.21.15.29:0/2493163271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff610062ec0 0x7ff610069240 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7ff608009f20 tx=0x7ff608009840).stop 2024-01-24T15:13:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.518+0000 7ff614850700 1 -- 172.21.15.29:0/2493163271 shutdown_connections 2024-01-24T15:13:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.518+0000 7ff614850700 1 --2- 172.21.15.29:0/2493163271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6101540a0 0x7ff6100c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.518+0000 7ff614850700 1 --2- 172.21.15.29:0/2493163271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff610062ec0 0x7ff610069240 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.518+0000 7ff614850700 1 -- 172.21.15.29:0/2493163271 >> 172.21.15.29:0/2493163271 conn(0x7ff6100c85c0 msgr2=0x7ff6100c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.518+0000 7ff614850700 1 -- 172.21.15.29:0/2493163271 shutdown_connections 2024-01-24T15:13:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.518+0000 7ff614850700 1 -- 172.21.15.29:0/2493163271 wait complete. 2024-01-24T15:13:21.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.520+0000 7ff614850700 1 Processor -- start 2024-01-24T15:13:21.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.520+0000 7ff614850700 1 -- start start 2024-01-24T15:13:21.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.521+0000 7ff614850700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff610062ec0 0x7ff610168660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.521+0000 7ff614850700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6101540a0 0x7ff61014f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.521+0000 7ff614850700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6101550f0 0x7ff61014f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.521+0000 7ff614850700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff610168240 con 0x7ff6101540a0 2024-01-24T15:13:21.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.521+0000 7ff614850700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61014fbb0 con 0x7ff6101550f0 2024-01-24T15:13:21.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.521+0000 7ff614850700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100adb50 con 0x7ff610062ec0 2024-01-24T15:13:21.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.521+0000 7ff60dd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6101540a0 0x7ff61014f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.521+0000 7ff60e59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff610062ec0 0x7ff610168660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.521+0000 7ff60e59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff610062ec0 0x7ff610168660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:36200/0 (socket says 172.21.15.29:36200) 2024-01-24T15:13:21.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.521+0000 7ff60ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6101550f0 0x7ff61014f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.525 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.521+0000 7ff60dd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6101540a0 0x7ff61014f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58856/0 (socket says 172.21.15.29:58856) 2024-01-24T15:13:21.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.521+0000 7ff60e59c700 1 -- 172.21.15.29:0/3304072450 learned_addr learned my addr 172.21.15.29:0/3304072450 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:21.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.522+0000 7ff60e59c700 1 -- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6101550f0 msgr2=0x7ff61014f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.522+0000 7ff60e59c700 1 --2- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6101550f0 0x7ff61014f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.522+0000 7ff60e59c700 1 -- 172.21.15.29:0/3304072450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6101540a0 msgr2=0x7ff61014f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.522+0000 7ff60e59c700 1 --2- 172.21.15.29:0/3304072450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6101540a0 0x7ff61014f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.522+0000 7ff60e59c700 1 -- 172.21.15.29:0/3304072450 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6000092a0 con 0x7ff610062ec0 2024-01-24T15:13:21.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.522+0000 7ff60dd9b700 1 --2- 172.21.15.29:0/3304072450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6101540a0 0x7ff61014f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:21.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.522+0000 7ff60e59c700 1 --2- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff610062ec0 0x7ff610168660 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7ff60000ea20 tx=0x7ff600009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:21.526 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.522+0000 7ff5fe7fc700 1 -- 172.21.15.29:0/3304072450 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000ee60 con 0x7ff610062ec0 2024-01-24T15:13:21.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.522+0000 7ff5fe7fc700 1 -- 172.21.15.29:0/3304072450 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60000c850 con 0x7ff610062ec0 2024-01-24T15:13:21.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.522+0000 7ff614850700 1 -- 172.21.15.29:0/3304072450 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff608009ae0 con 0x7ff610062ec0 2024-01-24T15:13:21.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.522+0000 7ff5fe7fc700 1 -- 172.21.15.29:0/3304072450 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000cbd0 con 0x7ff610062ec0 2024-01-24T15:13:21.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.522+0000 7ff614850700 1 -- 172.21.15.29:0/3304072450 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff610167c10 con 0x7ff610062ec0 2024-01-24T15:13:21.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.523+0000 7ff614850700 1 -- 172.21.15.29:0/3304072450 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6101684b0 con 0x7ff610062ec0 2024-01-24T15:13:21.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.523+0000 7ff5fe7fc700 1 -- 172.21.15.29:0/3304072450 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 64) v1 ==== 75560+0+0 (secure 0 0 0) 0x7ff60001b400 con 0x7ff610062ec0 2024-01-24T15:13:21.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.524+0000 7ff5fe7fc700 1 --2- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5f005f270 0x7ff5f0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.524+0000 7ff60dd9b700 1 -- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5f005f270 msgr2=0x7ff5f0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:21.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.524+0000 7ff5fe7fc700 1 -- 172.21.15.29:0/3304072450 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 7546+0+0 (secure 0 0 0) 0x7ff60007ae70 con 0x7ff610062ec0 2024-01-24T15:13:21.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.524+0000 7ff60dd9b700 1 --2- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5f005f270 0x7ff5f0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.532+0000 7ff5fe7fc700 1 -- 172.21.15.29:0/3304072450 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff60004bb40 con 0x7ff610062ec0 2024-01-24T15:13:21.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.719+0000 7ff614850700 1 -- 172.21.15.29:0/3304072450 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6100475e0 con 0x7ff610062ec0 2024-01-24T15:13:21.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.720+0000 7ff5fe7fc700 1 -- 172.21.15.29:0/3304072450 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+18890 (secure 0 0 0) 0x7ff60004f160 con 0x7ff610062ec0 2024-01-24T15:13:21.722 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:21.723 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":163,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:13:21.618361+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:17:18.175741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-24T14:17:19.397106+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":163,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.3","mappings":[{"from":2,"to":0}]},{"pgid":"19.16","mappings":[{"from":2,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/1044393967":"2024-01-25T14:17:32.674163+0000","172.21.15.163:6833/43241":"2024-01-25T14:17:32.674163+0000","172.21.15.163:6832/43241":"2024-01-25T14:17:32.674163+0000","172.21.15.163:0/3112762339":"2024-01-25T14:17:32.674163+0000","172.21.15.163:0/2817247896":"2024-01-25T14:17:32.674163+0000","172.21.15.163:0/221549441":"2024-01-25T14:17:32.674163+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":37},{"begin":66,"length":47}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":56,"length":1},{"begin":112,"length":1}]}],"new_purged_snaps":[{"pool":19,"snaps":[{"begin":3,"length":17},{"begin":59,"length":7}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:13:21.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.724+0000 7ff614850700 1 -- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5f005f270 msgr2=0x7ff5f0061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:21.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.724+0000 7ff614850700 1 --2- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5f005f270 0x7ff5f0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:21.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.724+0000 7ff614850700 1 -- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff610062ec0 msgr2=0x7ff610168660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.724+0000 7ff614850700 1 --2- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff610062ec0 0x7ff610168660 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7ff60000ea20 tx=0x7ff600009060).stop 2024-01-24T15:13:21.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.724+0000 7ff614850700 1 -- 172.21.15.29:0/3304072450 shutdown_connections 2024-01-24T15:13:21.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.724+0000 7ff614850700 1 --2- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6101550f0 0x7ff61014f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.724+0000 7ff614850700 1 --2- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff610062ec0 0x7ff610168660 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.724+0000 7ff614850700 1 --2- 172.21.15.29:0/3304072450 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff5f005f270 0x7ff5f0061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:21.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.724+0000 7ff614850700 1 --2- 172.21.15.29:0/3304072450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6101540a0 0x7ff61014f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.724+0000 7ff614850700 1 -- 172.21.15.29:0/3304072450 >> 172.21.15.29:0/3304072450 conn(0x7ff6100c85c0 msgr2=0x7ff610153460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:21.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.724+0000 7ff614850700 1 -- 172.21.15.29:0/3304072450 shutdown_connections 2024-01-24T15:13:21.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.724+0000 7ff614850700 1 -- 172.21.15.29:0/3304072450 wait complete. 2024-01-24T15:13:21.745 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T14:17:18.175741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '152', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T14:17:19.397106+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 163, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T15:13:21.745 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T15:13:21.836 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:21.833+0000 7f9a21a0e680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:13:21.908 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:21.905+0000 7f9a21a0e680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:13:21.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.910+0000 7fc4cf8b9700 1 Processor -- start 2024-01-24T15:13:21.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.910+0000 7fc4cf8b9700 1 -- start start 2024-01-24T15:13:21.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4cf8b9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c8156240 0x7fc4c815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4cf8b9700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c8063fc0 con 0x7fc4c8155670 2024-01-24T15:13:21.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4cf8b9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c8156920 con 0x7fc4c8155290 2024-01-24T15:13:21.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4cf8b9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c8065530 con 0x7fc4c8156240 2024-01-24T15:13:21.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4cce54700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc4c8155670 0x7fc4c8155b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38412/0 (socket says 172.21.15.29:38412) 2024-01-24T15:13:21.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4cce54700 1 -- 172.21.15.29:0/1480093524 learned_addr learned my addr 172.21.15.29:0/1480093524 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4cde56700 1 --2- 172.21.15.29:0/1480093524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c8156240 0x7fc4c815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4bffff700 1 -- 172.21.15.29:0/1480093524 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737388844 0 0) 0x7fc4c8063fc0 con 0x7fc4c8155670 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4bffff700 1 -- 172.21.15.29:0/1480093524 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4b40033f0 con 0x7fc4c8155670 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4bffff700 1 -- 172.21.15.29:0/1480093524 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835417508 0 0) 0x7fc4c8156920 con 0x7fc4c8155290 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4bffff700 1 -- 172.21.15.29:0/1480093524 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c8063fc0 con 0x7fc4c8155290 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4cde56700 1 -- 172.21.15.29:0/1480093524 >> v1:172.21.15.163:6789/0 conn(0x7fc4c8155290 legacy=0x7fc4c80bc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4cde56700 1 -- 172.21.15.29:0/1480093524 >> v1:172.21.15.29:6789/0 conn(0x7fc4c8155670 legacy=0x7fc4c8155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.912+0000 7fc4cde56700 1 -- 172.21.15.29:0/1480093524 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c8065710 con 0x7fc4c8156240 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.913+0000 7fc4cde56700 1 --2- 172.21.15.29:0/1480093524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c8156240 0x7fc4c815ce70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fc4c0009e90 tx=0x7fc4c0009780).ready entity=mon.2 client_cookie=a80ff2c4bdd973e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.913+0000 7fc4bffff700 1 -- 172.21.15.29:0/1480093524 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c000e070 con 0x7fc4c8156240 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.913+0000 7fc4bffff700 1 -- 172.21.15.29:0/1480093524 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c000bab0 con 0x7fc4c8156240 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.913+0000 7fc4bffff700 1 -- 172.21.15.29:0/1480093524 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c000be30 con 0x7fc4c8156240 2024-01-24T15:13:21.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.913+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/1480093524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c8156240 msgr2=0x7fc4c815ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.913+0000 7fc4cf8b9700 1 --2- 172.21.15.29:0/1480093524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c8156240 0x7fc4c815ce70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fc4c0009e90 tx=0x7fc4c0009780).stop 2024-01-24T15:13:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.913+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/1480093524 shutdown_connections 2024-01-24T15:13:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.913+0000 7fc4cf8b9700 1 --2- 172.21.15.29:0/1480093524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c8156240 0x7fc4c815ce70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.913+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/1480093524 >> 172.21.15.29:0/1480093524 conn(0x7fc4c80c7260 msgr2=0x7fc4c80c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.914+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/1480093524 shutdown_connections 2024-01-24T15:13:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.914+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/1480093524 wait complete. 2024-01-24T15:13:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.915+0000 7fc4cf8b9700 1 Processor -- start 2024-01-24T15:13:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.915+0000 7fc4cf8b9700 1 -- start start 2024-01-24T15:13:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cf8b9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c8155290 0x7fc4c8143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cf8b9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4c8155670 0x7fc4c80a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cf8b9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c8156240 0x7fc4c80c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cf8b9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c8168280 con 0x7fc4c8155670 2024-01-24T15:13:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cf8b9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c81676b0 con 0x7fc4c8155290 2024-01-24T15:13:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cf8b9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c8167950 con 0x7fc4c8156240 2024-01-24T15:13:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cce54700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4c8155670 0x7fc4c80a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cce54700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4c8155670 0x7fc4c80a8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58860/0 (socket says 172.21.15.29:58860) 2024-01-24T15:13:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cce54700 1 -- 172.21.15.29:0/3322522731 learned_addr learned my addr 172.21.15.29:0/3322522731 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cd655700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c8155290 0x7fc4c8143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cde56700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c8156240 0x7fc4c80c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cd655700 1 -- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c8156240 msgr2=0x7fc4c80c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cd655700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c8156240 0x7fc4c80c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cd655700 1 -- 172.21.15.29:0/3322522731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4c8155670 msgr2=0x7fc4c80a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:21.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cd655700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4c8155670 0x7fc4c80a8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.916+0000 7fc4cd655700 1 -- 172.21.15.29:0/3322522731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c0009a20 con 0x7fc4c8155290 2024-01-24T15:13:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.917+0000 7fc4cd655700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c8155290 0x7fc4c8143cc0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fc4c400c420 tx=0x7fc4c4007bd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.917+0000 7fc4bd7fa700 1 -- 172.21.15.29:0/3322522731 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4012400 con 0x7fc4c8155290 2024-01-24T15:13:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.917+0000 7fc4bd7fa700 1 -- 172.21.15.29:0/3322522731 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c4012560 con 0x7fc4c8155290 2024-01-24T15:13:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.917+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/3322522731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c8167c30 con 0x7fc4c8155290 2024-01-24T15:13:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.917+0000 7fc4bd7fa700 1 -- 172.21.15.29:0/3322522731 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40128e0 con 0x7fc4c8155290 2024-01-24T15:13:21.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.917+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/3322522731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c81441b0 con 0x7fc4c8155290 2024-01-24T15:13:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.918+0000 7fc4bd7fa700 1 -- 172.21.15.29:0/3322522731 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 64) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fc4c4012c90 con 0x7fc4c8155290 2024-01-24T15:13:21.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.918+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/3322522731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c80475e0 con 0x7fc4c8155290 2024-01-24T15:13:21.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.919+0000 7fc4bd7fa700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc4ac05f270 0x7fc4ac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:21.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.919+0000 7fc4bd7fa700 1 -- 172.21.15.29:0/3322522731 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 7590+0+0 (secure 0 0 0) 0x7fc4c4079cf0 con 0x7fc4c8155290 2024-01-24T15:13:21.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.919+0000 7fc4cce54700 1 -- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc4ac05f270 msgr2=0x7fc4ac061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:21.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.919+0000 7fc4cce54700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc4ac05f270 0x7fc4ac061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:21.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:21.923+0000 7fc4bd7fa700 1 -- 172.21.15.29:0/3322522731 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc4c4048b00 con 0x7fc4c8155290 2024-01-24T15:13:22.085 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:22.083+0000 7f6508dce680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:13:22.104 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:22.102+0000 7f9a21a0e680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:13:22.160 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:22.158+0000 7f6508dce680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:13:22.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.159+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/3322522731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc4c80659f0 con 0x7fc4c8155290 2024-01-24T15:13:22.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.160+0000 7fc4bd7fa700 1 -- 172.21.15.29:0/3322522731 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v163) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc4c404a870 con 0x7fc4c8155290 2024-01-24T15:13:22.162 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T15:13:22.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.162+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc4ac05f270 msgr2=0x7fc4ac061730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-24T15:13:22.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.162+0000 7fc4cf8b9700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc4ac05f270 0x7fc4ac061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:22.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.162+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c8155290 msgr2=0x7fc4c8143cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:22.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.162+0000 7fc4cf8b9700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c8155290 0x7fc4c8143cc0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fc4c400c420 tx=0x7fc4c4007bd0).stop 2024-01-24T15:13:22.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.162+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/3322522731 shutdown_connections 2024-01-24T15:13:22.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.162+0000 7fc4cf8b9700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c8156240 0x7fc4c80c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.162+0000 7fc4cf8b9700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c8155290 0x7fc4c8143cc0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.162+0000 7fc4cf8b9700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc4ac05f270 0x7fc4ac061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:22.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.162+0000 7fc4cf8b9700 1 --2- 172.21.15.29:0/3322522731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4c8155670 0x7fc4c80a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.162+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/3322522731 >> 172.21.15.29:0/3322522731 conn(0x7fc4c80c7260 msgr2=0x7fc4c80c58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:22.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.163+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/3322522731 shutdown_connections 2024-01-24T15:13:22.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.163+0000 7fc4cf8b9700 1 -- 172.21.15.29:0/3322522731 wait complete. 2024-01-24T15:13:22.178 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T15:13:22.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.313+0000 7ff0c1e3a700 1 Processor -- start 2024-01-24T15:13:22.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.313+0000 7ff0c1e3a700 1 -- start start 2024-01-24T15:13:22.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0c1e3a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff0bc064760 0x7ff0bc064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:22.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0c1e3a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0bc152240 con 0x7ff0bc0c8490 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0c1e3a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0bc1586c0 con 0x7ff0bc0bfd40 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0c1e3a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0bc0aa670 con 0x7ff0bc064760 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0b2ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff0bc0c8490 0x7ff0bc0a9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38414/0 (socket says 172.21.15.29:38414) 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0b2ffd700 1 -- 172.21.15.29:0/4123557944 learned_addr learned my addr 172.21.15.29:0/4123557944 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0bb7fe700 1 --2- 172.21.15.29:0/4123557944 >> v2:172.21.15.163:3300/0 conn(0x7ff0bc064760 0x7ff0bc064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0baffd700 1 -- 172.21.15.29:0/4123557944 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299484595 0 0) 0x7ff0bc152240 con 0x7ff0bc0c8490 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0baffd700 1 -- 172.21.15.29:0/4123557944 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0a80033f0 con 0x7ff0bc0c8490 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0baffd700 1 -- 172.21.15.29:0/4123557944 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687358780 0 0) 0x7ff0bc1586c0 con 0x7ff0bc0bfd40 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0baffd700 1 -- 172.21.15.29:0/4123557944 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0bc152240 con 0x7ff0bc0bfd40 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0bb7fe700 1 -- 172.21.15.29:0/4123557944 >> v1:172.21.15.163:6789/0 conn(0x7ff0bc0bfd40 legacy=0x7ff0bc0c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0bb7fe700 1 -- 172.21.15.29:0/4123557944 >> v1:172.21.15.29:6789/0 conn(0x7ff0bc0c8490 legacy=0x7ff0bc0a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.314+0000 7ff0bb7fe700 1 -- 172.21.15.29:0/4123557944 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc157d90 con 0x7ff0bc064760 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.315+0000 7ff0bb7fe700 1 --2- 172.21.15.29:0/4123557944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0bc064760 0x7ff0bc064b40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7ff0ac009760 tx=0x7ff0ac009060).ready entity=mon.1 client_cookie=5c8bd2434d04b78a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:22.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.315+0000 7ff0baffd700 1 -- 172.21.15.29:0/4123557944 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac00f510 con 0x7ff0bc064760 2024-01-24T15:13:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.315+0000 7ff0baffd700 1 -- 172.21.15.29:0/4123557944 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac00f670 con 0x7ff0bc064760 2024-01-24T15:13:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.315+0000 7ff0baffd700 1 -- 172.21.15.29:0/4123557944 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac00f9f0 con 0x7ff0bc064760 2024-01-24T15:13:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.315+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/4123557944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0bc064760 msgr2=0x7ff0bc064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.315+0000 7ff0c1e3a700 1 --2- 172.21.15.29:0/4123557944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0bc064760 0x7ff0bc064b40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7ff0ac009760 tx=0x7ff0ac009060).stop 2024-01-24T15:13:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.315+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/4123557944 shutdown_connections 2024-01-24T15:13:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.315+0000 7ff0c1e3a700 1 --2- 172.21.15.29:0/4123557944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0bc064760 0x7ff0bc064b40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.315+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/4123557944 >> 172.21.15.29:0/4123557944 conn(0x7ff0bc0cb8b0 msgr2=0x7ff0bc0c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.315+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/4123557944 shutdown_connections 2024-01-24T15:13:22.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.315+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/4123557944 wait complete. 2024-01-24T15:13:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.317+0000 7ff0c1e3a700 1 Processor -- start 2024-01-24T15:13:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.317+0000 7ff0c1e3a700 1 -- start start 2024-01-24T15:13:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0c1e3a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0bc064760 0x7ff0bc1650a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0c1e3a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0bc0bfd40 0x7ff0bc0c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0c1e3a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0bc0c8490 0x7ff0bc0af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:22.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0c1e3a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0c4780 con 0x7ff0bc064760 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0c1e3a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0c44b0 con 0x7ff0bc0bfd40 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0c1e3a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0adec0 con 0x7ff0bc0c8490 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0bb7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0bc064760 0x7ff0bc1650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0bb7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0bc064760 0x7ff0bc1650a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58876/0 (socket says 172.21.15.29:58876) 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0bb7fe700 1 -- 172.21.15.29:0/1142308720 learned_addr learned my addr 172.21.15.29:0/1142308720 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0bbfff700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0bc0c8490 0x7ff0bc0af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0b2ffd700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0bc0bfd40 0x7ff0bc0c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0bb7fe700 1 -- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0bc0c8490 msgr2=0x7ff0bc0af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0bb7fe700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0bc0c8490 0x7ff0bc0af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0bb7fe700 1 -- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0bc0bfd40 msgr2=0x7ff0bc0c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0bb7fe700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0bc0bfd40 0x7ff0bc0c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0bb7fe700 1 -- 172.21.15.29:0/1142308720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac0092a0 con 0x7ff0bc064760 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0b2ffd700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0bc0bfd40 0x7ff0bc0c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:22.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.318+0000 7ff0bb7fe700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0bc064760 0x7ff0bc1650a0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7ff0ac009d50 tx=0x7ff0ac017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:22.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.319+0000 7ff0b3fff700 1 -- 172.21.15.29:0/1142308720 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac0115d0 con 0x7ff0bc064760 2024-01-24T15:13:22.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.319+0000 7ff0b3fff700 1 -- 172.21.15.29:0/1142308720 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac01e070 con 0x7ff0bc064760 2024-01-24T15:13:22.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.319+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/1142308720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0bc0ae1a0 con 0x7ff0bc064760 2024-01-24T15:13:22.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.319+0000 7ff0b3fff700 1 -- 172.21.15.29:0/1142308720 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac011960 con 0x7ff0bc064760 2024-01-24T15:13:22.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.319+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/1142308720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0bc0ae760 con 0x7ff0bc064760 2024-01-24T15:13:22.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.320+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/1142308720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0bc0475e0 con 0x7ff0bc064760 2024-01-24T15:13:22.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.320+0000 7ff0b3fff700 1 -- 172.21.15.29:0/1142308720 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 64) v1 ==== 75560+0+0 (secure 0 0 0) 0x7ff0ac020030 con 0x7ff0bc064760 2024-01-24T15:13:22.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.321+0000 7ff0b3fff700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff09c05f240 0x7ff09c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:22.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.321+0000 7ff0b3fff700 1 -- 172.21.15.29:0/1142308720 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 7590+0+0 (secure 0 0 0) 0x7ff0ac07ba40 con 0x7ff0bc064760 2024-01-24T15:13:22.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.325+0000 7ff0b3fff700 1 -- 172.21.15.29:0/1142308720 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff0ac04c570 con 0x7ff0bc064760 2024-01-24T15:13:22.382 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:22.380+0000 7f6508dce680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:13:22.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.507+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/1142308720 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff0bc0c7d40 con 0x7ff0bc064760 2024-01-24T15:13:22.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.507+0000 7ff0b3fff700 1 -- 172.21.15.29:0/1142308720 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v163) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff0ac04fb90 con 0x7ff0bc064760 2024-01-24T15:13:22.510 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T15:13:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.510+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff09c05f240 msgr2=0x7ff09c061700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-24T15:13:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.510+0000 7ff0c1e3a700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff09c05f240 0x7ff09c061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:22.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.510+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/1142308720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0bc064760 msgr2=0x7ff0bc1650a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:22.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.510+0000 7ff0c1e3a700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0bc064760 0x7ff0bc1650a0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7ff0ac009d50 tx=0x7ff0ac017000).stop 2024-01-24T15:13:22.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.510+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/1142308720 shutdown_connections 2024-01-24T15:13:22.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.510+0000 7ff0c1e3a700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0bc0c8490 0x7ff0bc0af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.510+0000 7ff0c1e3a700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff09c05f240 0x7ff09c061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:22.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.510+0000 7ff0c1e3a700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0bc064760 0x7ff0bc1650a0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.510+0000 7ff0c1e3a700 1 --2- 172.21.15.29:0/1142308720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0bc0bfd40 0x7ff0bc0c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.510+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/1142308720 >> 172.21.15.29:0/1142308720 conn(0x7ff0bc0cb8b0 msgr2=0x7ff0bc1557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:22.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.510+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/1142308720 shutdown_connections 2024-01-24T15:13:22.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.510+0000 7ff0c1e3a700 1 -- 172.21.15.29:0/1142308720 wait complete. 2024-01-24T15:13:22.526 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T15:13:22.527 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:22.524+0000 7f9a21a0e680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:13:22.634 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:22.632+0000 7f9a21a0e680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:13:22.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.667+0000 7f9f82489700 1 Processor -- start 2024-01-24T15:13:22.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.667+0000 7f9f82489700 1 -- start start 2024-01-24T15:13:22.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f82489700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c064760 0x7f9f7c064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:22.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f82489700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9f7c0c8490 0x7f9f7c0a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:22.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f82489700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7c152240 con 0x7f9f7c154700 2024-01-24T15:13:22.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f82489700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7c1586c0 con 0x7f9f7c0c8490 2024-01-24T15:13:22.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f82489700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c0c0d50 con 0x7f9f7c064760 2024-01-24T15:13:22.671 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f7bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c064760 0x7f9f7c064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f7bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c064760 0x7f9f7c064b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:36228/0 (socket says 172.21.15.29:36228) 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f7b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9f7c0c8490 0x7f9f7c0a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f7bfff700 1 -- 172.21.15.29:0/2129903760 learned_addr learned my addr 172.21.15.29:0/2129903760 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f7bfff700 1 -- 172.21.15.29:0/2129903760 >> v2:172.21.15.163:3300/0 conn(0x7f9f7c0c8490 msgr2=0x7f9f7c0a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f7bfff700 1 --2- 172.21.15.29:0/2129903760 >> v2:172.21.15.163:3300/0 conn(0x7f9f7c0c8490 0x7f9f7c0a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f7bfff700 1 -- 172.21.15.29:0/2129903760 >> v1:172.21.15.163:6789/0 conn(0x7f9f7c154700 legacy=0x7f9f7c154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f7bfff700 1 -- 172.21.15.29:0/2129903760 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c0aa670 con 0x7f9f7c064760 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.668+0000 7f9f7bfff700 1 --2- 172.21.15.29:0/2129903760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c064760 0x7f9f7c064b40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f9f6c006fd0 tx=0x7f9f6c0090f0).ready entity=mon.2 client_cookie=8a7cb34ed7cf9b6f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.669+0000 7f9f7affd700 1 -- 172.21.15.29:0/2129903760 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c00b610 con 0x7f9f7c064760 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.669+0000 7f9f7affd700 1 -- 172.21.15.29:0/2129903760 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c00b770 con 0x7f9f7c064760 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.669+0000 7f9f7affd700 1 -- 172.21.15.29:0/2129903760 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c00baf0 con 0x7f9f7c064760 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.669+0000 7f9f82489700 1 -- 172.21.15.29:0/2129903760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c064760 msgr2=0x7f9f7c064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.669+0000 7f9f82489700 1 --2- 172.21.15.29:0/2129903760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c064760 0x7f9f7c064b40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f9f6c006fd0 tx=0x7f9f6c0090f0).stop 2024-01-24T15:13:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.669+0000 7f9f82489700 1 -- 172.21.15.29:0/2129903760 shutdown_connections 2024-01-24T15:13:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.669+0000 7f9f82489700 1 --2- 172.21.15.29:0/2129903760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c064760 0x7f9f7c064b40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.669+0000 7f9f82489700 1 --2- 172.21.15.29:0/2129903760 >> v2:172.21.15.163:3300/0 conn(0x7f9f7c0c8490 0x7f9f7c0a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.669+0000 7f9f82489700 1 -- 172.21.15.29:0/2129903760 >> 172.21.15.29:0/2129903760 conn(0x7f9f7c0cb8b0 msgr2=0x7f9f7c0c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.669+0000 7f9f82489700 1 -- 172.21.15.29:0/2129903760 shutdown_connections 2024-01-24T15:13:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.669+0000 7f9f82489700 1 -- 172.21.15.29:0/2129903760 wait complete. 2024-01-24T15:13:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.670+0000 7f9f82489700 1 Processor -- start 2024-01-24T15:13:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.671+0000 7f9f82489700 1 -- start start 2024-01-24T15:13:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.671+0000 7f9f82489700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f7c064760 0x7f9f7c157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.671+0000 7f9f82489700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f7c0c8490 0x7f9f7c0b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.671+0000 7f9f82489700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c154700 0x7f9f7c1457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.671+0000 7f9f82489700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c0b0ee0 con 0x7f9f7c064760 2024-01-24T15:13:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.671+0000 7f9f82489700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c1691c0 con 0x7f9f7c0c8490 2024-01-24T15:13:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.671+0000 7f9f82489700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c156080 con 0x7f9f7c154700 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f7bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f7c064760 0x7f9f7c157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f7bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f7c064760 0x7f9f7c157120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58888/0 (socket says 172.21.15.29:58888) 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f7bfff700 1 -- 172.21.15.29:0/516981379 learned_addr learned my addr 172.21.15.29:0/516981379 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f7b7fe700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f7c0c8490 0x7f9f7c0b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f80a26700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c154700 0x7f9f7c1457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f7b7fe700 1 -- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c154700 msgr2=0x7f9f7c1457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f7b7fe700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c154700 0x7f9f7c1457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f7b7fe700 1 -- 172.21.15.29:0/516981379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f7c064760 msgr2=0x7f9f7c157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f7b7fe700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f7c064760 0x7f9f7c157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f7b7fe700 1 -- 172.21.15.29:0/516981379 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c009370 con 0x7f9f7c0c8490 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f7b7fe700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f7c0c8490 0x7f9f7c0b09f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f9f7000e8d0 tx=0x7f9f70009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f63fff700 1 -- 172.21.15.29:0/516981379 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70009470 con 0x7f9f7c0c8490 2024-01-24T15:13:22.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f63fff700 1 -- 172.21.15.29:0/516981379 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f70012600 con 0x7f9f7c0c8490 2024-01-24T15:13:22.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f82489700 1 -- 172.21.15.29:0/516981379 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7c156360 con 0x7f9f7c0c8490 2024-01-24T15:13:22.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.672+0000 7f9f63fff700 1 -- 172.21.15.29:0/516981379 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70012980 con 0x7f9f7c0c8490 2024-01-24T15:13:22.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.673+0000 7f9f82489700 1 -- 172.21.15.29:0/516981379 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f7c156920 con 0x7f9f7c0c8490 2024-01-24T15:13:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.674+0000 7f9f82489700 1 -- 172.21.15.29:0/516981379 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f7c156f70 con 0x7f9f7c0c8490 2024-01-24T15:13:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.674+0000 7f9f63fff700 1 -- 172.21.15.29:0/516981379 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 64) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f9f7001b400 con 0x7f9f7c0c8490 2024-01-24T15:13:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.674+0000 7f9f63fff700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6805f240 0x7f9f68061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.674+0000 7f9f7bfff700 1 -- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6805f240 msgr2=0x7f9f68061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.674+0000 7f9f63fff700 1 -- 172.21.15.29:0/516981379 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 7546+0+0 (secure 0 0 0) 0x7f9f7007abf0 con 0x7f9f7c0c8490 2024-01-24T15:13:22.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.675+0000 7f9f7bfff700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6805f240 0x7f9f68061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:22.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.679+0000 7f9f63fff700 1 -- 172.21.15.29:0/516981379 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9f70081080 con 0x7f9f7c0c8490 2024-01-24T15:13:22.792 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:22.790+0000 7f9a21a0e680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:13:22.866 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.864+0000 7f9f82489700 1 -- 172.21.15.29:0/516981379 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f7c0c12c0 con 0x7f9f7c0c8490 2024-01-24T15:13:22.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.865+0000 7f9f63fff700 1 -- 172.21.15.29:0/516981379 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v164) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f7004b750 con 0x7f9f7c0c8490 2024-01-24T15:13:22.867 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T15:13:22.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.867+0000 7f9f82489700 1 -- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6805f240 msgr2=0x7f9f68061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:22.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.867+0000 7f9f82489700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6805f240 0x7f9f68061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:22.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.867+0000 7f9f82489700 1 -- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f7c0c8490 msgr2=0x7f9f7c0b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:22.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.867+0000 7f9f82489700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f7c0c8490 0x7f9f7c0b09f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f9f7000e8d0 tx=0x7f9f70009060).stop 2024-01-24T15:13:22.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.867+0000 7f9f82489700 1 -- 172.21.15.29:0/516981379 shutdown_connections 2024-01-24T15:13:22.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.867+0000 7f9f82489700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f7c154700 0x7f9f7c1457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.867+0000 7f9f82489700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6805f240 0x7f9f68061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:22.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.867+0000 7f9f82489700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f7c064760 0x7f9f7c157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.867+0000 7f9f82489700 1 --2- 172.21.15.29:0/516981379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f7c0c8490 0x7f9f7c0b09f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:22.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.867+0000 7f9f82489700 1 -- 172.21.15.29:0/516981379 >> 172.21.15.29:0/516981379 conn(0x7f9f7c0cb8b0 msgr2=0x7f9f7c1523d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:22.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.867+0000 7f9f82489700 1 -- 172.21.15.29:0/516981379 shutdown_connections 2024-01-24T15:13:22.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:22.867+0000 7f9f82489700 1 -- 172.21.15.29:0/516981379 wait complete. 2024-01-24T15:13:22.871 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:22.868+0000 7f9a21a0e680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:13:22.879 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:22.878+0000 7f6508dce680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:13:22.882 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T15:13:22.940 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:22.936+0000 7f9a21a0e680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:13:23.007 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:23.005+0000 7f6508dce680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:13:23.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.018+0000 7ff645862700 1 Processor -- start 2024-01-24T15:13:23.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.018+0000 7ff645862700 1 -- start start 2024-01-24T15:13:23.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.019+0000 7ff645862700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6400692a0 0x7ff640065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.019+0000 7ff645862700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff640066370 0x7ff640069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.019+0000 7ff645862700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff640069b70 0x7ff6400bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.019+0000 7ff645862700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6400c3e70 con 0x7ff6400692a0 2024-01-24T15:13:23.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.019+0000 7ff645862700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6400c4300 con 0x7ff640066370 2024-01-24T15:13:23.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.019+0000 7ff645862700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6400c3260 con 0x7ff640069b70 2024-01-24T15:13:23.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.019+0000 7ff63effd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6400692a0 0x7ff640065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.022 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.019+0000 7ff63effd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6400692a0 0x7ff640065e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58896/0 (socket says 172.21.15.29:58896) 2024-01-24T15:13:23.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.019+0000 7ff63effd700 1 -- 172.21.15.29:0/2580706663 learned_addr learned my addr 172.21.15.29:0/2580706663 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:23.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.019+0000 7ff63f7fe700 1 --2- 172.21.15.29:0/2580706663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff640069b70 0x7ff6400bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.019+0000 7ff63e7fc700 1 --2- 172.21.15.29:0/2580706663 >> v2:172.21.15.163:3300/0 conn(0x7ff640066370 0x7ff640069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.020+0000 7ff63effd700 1 -- 172.21.15.29:0/2580706663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff640069b70 msgr2=0x7ff6400bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:23.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.020+0000 7ff63effd700 1 --2- 172.21.15.29:0/2580706663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff640069b70 0x7ff6400bcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.020+0000 7ff63effd700 1 -- 172.21.15.29:0/2580706663 >> v2:172.21.15.163:3300/0 conn(0x7ff640066370 msgr2=0x7ff640069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:23.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.020+0000 7ff63effd700 1 --2- 172.21.15.29:0/2580706663 >> v2:172.21.15.163:3300/0 conn(0x7ff640066370 0x7ff640069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.020+0000 7ff63effd700 1 -- 172.21.15.29:0/2580706663 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400c3540 con 0x7ff6400692a0 2024-01-24T15:13:23.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.020+0000 7ff63effd700 1 --2- 172.21.15.29:0/2580706663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6400692a0 0x7ff640065e80 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7ff6400c8b30 tx=0x7ff6280090f0).ready entity=mon.0 client_cookie=30493f08706e46c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:23.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.020+0000 7ff63dffb700 1 -- 172.21.15.29:0/2580706663 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62800f510 con 0x7ff6400692a0 2024-01-24T15:13:23.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.020+0000 7ff63dffb700 1 -- 172.21.15.29:0/2580706663 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62800f670 con 0x7ff6400692a0 2024-01-24T15:13:23.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.020+0000 7ff63dffb700 1 -- 172.21.15.29:0/2580706663 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62800fa10 con 0x7ff6400692a0 2024-01-24T15:13:23.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.021+0000 7ff645862700 1 -- 172.21.15.29:0/2580706663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6400692a0 msgr2=0x7ff640065e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.021+0000 7ff645862700 1 --2- 172.21.15.29:0/2580706663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6400692a0 0x7ff640065e80 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7ff6400c8b30 tx=0x7ff6280090f0).stop 2024-01-24T15:13:23.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.021+0000 7ff645862700 1 -- 172.21.15.29:0/2580706663 shutdown_connections 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.021+0000 7ff645862700 1 --2- 172.21.15.29:0/2580706663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff640069b70 0x7ff6400bcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.021+0000 7ff645862700 1 --2- 172.21.15.29:0/2580706663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6400692a0 0x7ff640065e80 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.021+0000 7ff645862700 1 --2- 172.21.15.29:0/2580706663 >> v2:172.21.15.163:3300/0 conn(0x7ff640066370 0x7ff640069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.021+0000 7ff645862700 1 -- 172.21.15.29:0/2580706663 >> 172.21.15.29:0/2580706663 conn(0x7ff640064390 msgr2=0x7ff6400c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.021+0000 7ff645862700 1 -- 172.21.15.29:0/2580706663 shutdown_connections 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.021+0000 7ff645862700 1 -- 172.21.15.29:0/2580706663 wait complete. 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.022+0000 7ff645862700 1 Processor -- start 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.022+0000 7ff645862700 1 -- start start 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.023+0000 7ff645862700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff640066370 0x7ff6401645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.023+0000 7ff645862700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6400692a0 0x7ff640157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.023+0000 7ff645862700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff640069b70 0x7ff6400c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.023+0000 7ff645862700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640167d40 con 0x7ff6400692a0 2024-01-24T15:13:23.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.023+0000 7ff645862700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640164b70 con 0x7ff640066370 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.023+0000 7ff645862700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff640164e50 con 0x7ff640069b70 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.023+0000 7ff63e7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6400692a0 0x7ff640157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.023+0000 7ff63effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff640066370 0x7ff6401645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.023+0000 7ff63f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff640069b70 0x7ff6400c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.023+0000 7ff63effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff640066370 0x7ff6401645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:56268/0 (socket says 172.21.15.29:56268) 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.023+0000 7ff63effd700 1 -- 172.21.15.29:0/162722122 learned_addr learned my addr 172.21.15.29:0/162722122 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff63e7fc700 1 -- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff640069b70 msgr2=0x7ff6400c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff63e7fc700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff640069b70 0x7ff6400c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff63e7fc700 1 -- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff640066370 msgr2=0x7ff6401645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff63e7fc700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff640066370 0x7ff6401645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff63e7fc700 1 -- 172.21.15.29:0/162722122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff628009370 con 0x7ff6400692a0 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff63effd700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff640066370 0x7ff6401645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff63f7fe700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff640069b70 0x7ff6400c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff63e7fc700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6400692a0 0x7ff640157200 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7ff63000e900 tx=0x7ff630009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:23.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff636ffd700 1 -- 172.21.15.29:0/162722122 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630012600 con 0x7ff6400692a0 2024-01-24T15:13:23.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff636ffd700 1 -- 172.21.15.29:0/162722122 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff630012760 con 0x7ff6400692a0 2024-01-24T15:13:23.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff636ffd700 1 -- 172.21.15.29:0/162722122 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630012ae0 con 0x7ff6400692a0 2024-01-24T15:13:23.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff645862700 1 -- 172.21.15.29:0/162722122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff640165130 con 0x7ff6400692a0 2024-01-24T15:13:23.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.024+0000 7ff645862700 1 -- 172.21.15.29:0/162722122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6401576f0 con 0x7ff6400692a0 2024-01-24T15:13:23.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.025+0000 7ff645862700 1 -- 172.21.15.29:0/162722122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff640167e80 con 0x7ff6400692a0 2024-01-24T15:13:23.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.026+0000 7ff636ffd700 1 -- 172.21.15.29:0/162722122 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 64) v1 ==== 75560+0+0 (secure 0 0 0) 0x7ff63001b400 con 0x7ff6400692a0 2024-01-24T15:13:23.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.026+0000 7ff636ffd700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff62005f270 0x7ff620061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.026+0000 7ff636ffd700 1 -- 172.21.15.29:0/162722122 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 7546+0+0 (secure 0 0 0) 0x7ff63007ae40 con 0x7ff6400692a0 2024-01-24T15:13:23.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.026+0000 7ff63effd700 1 -- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff62005f270 msgr2=0x7ff620061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:23.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.026+0000 7ff63effd700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff62005f270 0x7ff620061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:23.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.031+0000 7ff636ffd700 1 -- 172.21.15.29:0/162722122 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff640167e80 con 0x7ff6400692a0 2024-01-24T15:13:23.190 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:23.188+0000 7f6508dce680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:13:23.200 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:23.198+0000 7f9a21a0e680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:13:23.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.215+0000 7ff645862700 1 -- 172.21.15.29:0/162722122 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff640167e80 con 0x7ff6400692a0 2024-01-24T15:13:23.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.215+0000 7ff636ffd700 1 -- 172.21.15.29:0/162722122 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v164) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff640167e80 con 0x7ff6400692a0 2024-01-24T15:13:23.217 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T15:13:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.217+0000 7ff645862700 1 -- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff62005f270 msgr2=0x7ff620061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.217+0000 7ff645862700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff62005f270 0x7ff620061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.217+0000 7ff645862700 1 -- 172.21.15.29:0/162722122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6400692a0 msgr2=0x7ff640157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.217+0000 7ff645862700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6400692a0 0x7ff640157200 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7ff63000e900 tx=0x7ff630009060).stop 2024-01-24T15:13:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.217+0000 7ff645862700 1 -- 172.21.15.29:0/162722122 shutdown_connections 2024-01-24T15:13:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.217+0000 7ff645862700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff640069b70 0x7ff6400c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.217+0000 7ff645862700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff640066370 0x7ff6401645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.217+0000 7ff645862700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff62005f270 0x7ff620061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:23.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.217+0000 7ff645862700 1 --2- 172.21.15.29:0/162722122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6400692a0 0x7ff640157200 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.217+0000 7ff645862700 1 -- 172.21.15.29:0/162722122 >> 172.21.15.29:0/162722122 conn(0x7ff640064390 msgr2=0x7ff6400c7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:23.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.218+0000 7ff645862700 1 -- 172.21.15.29:0/162722122 shutdown_connections 2024-01-24T15:13:23.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.218+0000 7ff645862700 1 -- 172.21.15.29:0/162722122 wait complete. 2024-01-24T15:13:23.234 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:13:23.280 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:23.278+0000 7f6508dce680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:13:23.360 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:23.358+0000 7f6508dce680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:13:23.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.363+0000 7fa2b7445700 1 Processor -- start 2024-01-24T15:13:23.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.363+0000 7fa2b7445700 1 -- start start 2024-01-24T15:13:23.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.364+0000 7fa2b7445700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa2b00c5410 0x7fa2b00aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.364+0000 7fa2b7445700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b00aa6c0 con 0x7fa2b0154750 2024-01-24T15:13:23.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.364+0000 7fa2b7445700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b0064240 con 0x7fa2b0154370 2024-01-24T15:13:23.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.364+0000 7fa2b7445700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b0154ba0 con 0x7fa2b00c5410 2024-01-24T15:13:23.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.364+0000 7fa2b49e0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa2b00c5410 0x7fa2b00aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.364+0000 7fa2b59e2700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa2b0154750 0x7fa2b00631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38422/0 (socket says 172.21.15.29:38422) 2024-01-24T15:13:23.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.364+0000 7fa2b59e2700 1 -- 172.21.15.29:0/471456661 learned_addr learned my addr 172.21.15.29:0/471456661 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:23.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.365+0000 7fa2b49e0700 1 -- 172.21.15.29:0/471456661 >> v1:172.21.15.163:6789/0 conn(0x7fa2b0154370 legacy=0x7fa2b00ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.365+0000 7fa2b49e0700 1 -- 172.21.15.29:0/471456661 >> v1:172.21.15.29:6789/0 conn(0x7fa2b0154750 legacy=0x7fa2b00631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.365+0000 7fa2b49e0700 1 -- 172.21.15.29:0/471456661 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b00cb5c0 con 0x7fa2b00c5410 2024-01-24T15:13:23.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.365+0000 7fa2a7fff700 1 -- 172.21.15.29:0/471456661 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954789543 0 0) 0x7fa2b00aa6c0 con 0x7fa2b0154750 2024-01-24T15:13:23.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.365+0000 7fa2b49e0700 1 --2- 172.21.15.29:0/471456661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b00c5410 0x7fa2b00aa0e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7fa2a0009760 tx=0x7fa2a0009060).ready entity=mon.0 client_cookie=5ec2c78b106b78a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:23.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.365+0000 7fa2a7fff700 1 -- 172.21.15.29:0/471456661 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000b580 con 0x7fa2b00c5410 2024-01-24T15:13:23.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.365+0000 7fa2a7fff700 1 -- 172.21.15.29:0/471456661 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a000b6e0 con 0x7fa2b00c5410 2024-01-24T15:13:23.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.365+0000 7fa2a7fff700 1 -- 172.21.15.29:0/471456661 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000ba60 con 0x7fa2b00c5410 2024-01-24T15:13:23.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.366+0000 7fa2b7445700 1 -- 172.21.15.29:0/471456661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b00c5410 msgr2=0x7fa2b00aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.366+0000 7fa2b7445700 1 --2- 172.21.15.29:0/471456661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b00c5410 0x7fa2b00aa0e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7fa2a0009760 tx=0x7fa2a0009060).stop 2024-01-24T15:13:23.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.366+0000 7fa2b7445700 1 -- 172.21.15.29:0/471456661 shutdown_connections 2024-01-24T15:13:23.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.366+0000 7fa2b7445700 1 --2- 172.21.15.29:0/471456661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b00c5410 0x7fa2b00aa0e0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.366+0000 7fa2b7445700 1 -- 172.21.15.29:0/471456661 >> 172.21.15.29:0/471456661 conn(0x7fa2b005ec80 msgr2=0x7fa2b00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:23.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.366+0000 7fa2b7445700 1 -- 172.21.15.29:0/471456661 shutdown_connections 2024-01-24T15:13:23.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.366+0000 7fa2b7445700 1 -- 172.21.15.29:0/471456661 wait complete. 2024-01-24T15:13:23.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.367+0000 7fa2b7445700 1 Processor -- start 2024-01-24T15:13:23.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.367+0000 7fa2b7445700 1 -- start start 2024-01-24T15:13:23.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.368+0000 7fa2b7445700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2b00c5410 0x7fa2b014c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.368+0000 7fa2b7445700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b0154370 0x7fa2b0156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.368+0000 7fa2b7445700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2b0154750 0x7fa2b01592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.368+0000 7fa2b7445700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00addc0 con 0x7fa2b0154370 2024-01-24T15:13:23.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.368+0000 7fa2b7445700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00ae0a0 con 0x7fa2b0154750 2024-01-24T15:13:23.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.368+0000 7fa2b7445700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b014cc10 con 0x7fa2b00c5410 2024-01-24T15:13:23.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2b49e0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b0154370 0x7fa2b0156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2b49e0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b0154370 0x7fa2b0156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58908/0 (socket says 172.21.15.29:58908) 2024-01-24T15:13:23.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2b49e0700 1 -- 172.21.15.29:0/3703741237 learned_addr learned my addr 172.21.15.29:0/3703741237 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:23.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2b59e2700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2b0154750 0x7fa2b01592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2b51e1700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2b00c5410 0x7fa2b014c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2b49e0700 1 -- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2b00c5410 msgr2=0x7fa2b014c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2b49e0700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2b00c5410 0x7fa2b014c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2b49e0700 1 -- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2b0154750 msgr2=0x7fa2b01592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2b49e0700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2b0154750 0x7fa2b01592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2b49e0700 1 -- 172.21.15.29:0/3703741237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a00092a0 con 0x7fa2b0154370 2024-01-24T15:13:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2b49e0700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b0154370 0x7fa2b0156790 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7fa2b00c8920 tx=0x7fa2a0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2a57fa700 1 -- 172.21.15.29:0/3703741237 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000b850 con 0x7fa2b0154370 2024-01-24T15:13:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2a57fa700 1 -- 172.21.15.29:0/3703741237 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a000b9d0 con 0x7fa2b0154370 2024-01-24T15:13:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2b7445700 1 -- 172.21.15.29:0/3703741237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b0157360 con 0x7fa2b0154370 2024-01-24T15:13:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.369+0000 7fa2a57fa700 1 -- 172.21.15.29:0/3703741237 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a001b660 con 0x7fa2b0154370 2024-01-24T15:13:23.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.370+0000 7fa2b7445700 1 -- 172.21.15.29:0/3703741237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b0157920 con 0x7fa2b0154370 2024-01-24T15:13:23.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.371+0000 7fa2b7445700 1 -- 172.21.15.29:0/3703741237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b00ad450 con 0x7fa2b0154370 2024-01-24T15:13:23.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.371+0000 7fa2a57fa700 1 -- 172.21.15.29:0/3703741237 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 64) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fa2a001ba90 con 0x7fa2b0154370 2024-01-24T15:13:23.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.371+0000 7fa2a57fa700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa29405f270 0x7fa294061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.371+0000 7fa2a57fa700 1 -- 172.21.15.29:0/3703741237 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 7546+0+0 (secure 0 0 0) 0x7fa2a007c390 con 0x7fa2b0154370 2024-01-24T15:13:23.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.371+0000 7fa2b51e1700 1 -- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa29405f270 msgr2=0x7fa294061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:23.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.371+0000 7fa2b51e1700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa29405f270 0x7fa294061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:23.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.376+0000 7fa2a57fa700 1 -- 172.21.15.29:0/3703741237 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa2a004cf50 con 0x7fa2b0154370 2024-01-24T15:13:23.387 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:23.385+0000 7f9a21a0e680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:13:23.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.572+0000 7fa2b51e1700 1 -- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa29405f270 msgr2=0x7fa294061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:23.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.572+0000 7fa2b51e1700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa29405f270 0x7fa294061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:13:23.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.606+0000 7fa2b7445700 1 -- 172.21.15.29:0/3703741237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2b00475e0 con 0x7fa2b0154370 2024-01-24T15:13:23.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.606+0000 7fa2a57fa700 1 -- 172.21.15.29:0/3703741237 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fa2a0050570 con 0x7fa2b0154370 2024-01-24T15:13:23.609 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:23.609 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2106677945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2106677945},{"type":"v1","addr":"172.21.15.163:6836","nonce":2106677945}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7786,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1964541525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1964541525},{"type":"v1","addr":"172.21.15.163:6837","nonce":1964541525}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4162831288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4162831288},{"type":"v1","addr":"172.21.15.29:6835","nonce":4162831288}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:17:20.414311+0000","modified":"2024-01-24T14:17:23.419003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8166":{"gid":8166,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/1312733772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1312733772},{"type":"v1","addr":"172.21.15.29:6834","nonce":1312733772}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-24T15:13:23.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.609+0000 7fa2b7445700 1 -- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa29405f270 msgr2=0x7fa294061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:23.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.609+0000 7fa2b7445700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa29405f270 0x7fa294061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:23.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.609+0000 7fa2b7445700 1 -- 172.21.15.29:0/3703741237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b0154370 msgr2=0x7fa2b0156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.609+0000 7fa2b7445700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b0154370 0x7fa2b0156790 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7fa2b00c8920 tx=0x7fa2a0017000).stop 2024-01-24T15:13:23.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.609+0000 7fa2b7445700 1 -- 172.21.15.29:0/3703741237 shutdown_connections 2024-01-24T15:13:23.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.609+0000 7fa2b7445700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2b0154750 0x7fa2b01592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.609+0000 7fa2b7445700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2b00c5410 0x7fa2b014c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.609+0000 7fa2b7445700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa29405f270 0x7fa294061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:23.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.609+0000 7fa2b7445700 1 --2- 172.21.15.29:0/3703741237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b0154370 0x7fa2b0156790 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.609+0000 7fa2b7445700 1 -- 172.21.15.29:0/3703741237 >> 172.21.15.29:0/3703741237 conn(0x7fa2b005ec80 msgr2=0x7fa2b00c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:23.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.609+0000 7fa2b7445700 1 -- 172.21.15.29:0/3703741237 shutdown_connections 2024-01-24T15:13:23.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.609+0000 7fa2b7445700 1 -- 172.21.15.29:0/3703741237 wait complete. 2024-01-24T15:13:23.612 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 66 2024-01-24T15:13:23.624 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:13:23.666 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:23.665+0000 7f6508dce680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:13:23.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.755+0000 7f3dc915d700 1 Processor -- start 2024-01-24T15:13:23.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.755+0000 7f3dc915d700 1 -- start start 2024-01-24T15:13:23.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc915d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3dc4138330 0x7f3dc4138710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc915d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3dc40627b0 0x7f3dc4062c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc915d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc4065770 con 0x7f3dc40bca50 2024-01-24T15:13:23.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc915d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc4063130 con 0x7f3dc40627b0 2024-01-24T15:13:23.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc915d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc4069cf0 con 0x7f3dc4138330 2024-01-24T15:13:23.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc359e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3dc40627b0 0x7f3dc4062c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc359e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3dc40627b0 0x7f3dc4062c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35562/0 (socket says 172.21.15.29:35562) 2024-01-24T15:13:23.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc259c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3dc40bca50 0x7f3dc40be3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52322/0 (socket says 172.21.15.29:52322) 2024-01-24T15:13:23.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc359e700 1 -- 172.21.15.29:0/35886172 learned_addr learned my addr 172.21.15.29:0/35886172 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:23.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc2d9d700 1 --2- 172.21.15.29:0/35886172 >> v2:172.21.15.163:3300/0 conn(0x7f3dc4138330 0x7f3dc4138710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc359e700 1 -- 172.21.15.29:0/35886172 >> v2:172.21.15.163:3300/0 conn(0x7f3dc4138330 msgr2=0x7f3dc4138710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:23.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc359e700 1 --2- 172.21.15.29:0/35886172 >> v2:172.21.15.163:3300/0 conn(0x7f3dc4138330 0x7f3dc4138710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc359e700 1 -- 172.21.15.29:0/35886172 >> v1:172.21.15.29:6789/0 conn(0x7f3dc40bca50 legacy=0x7f3dc40be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.756+0000 7f3dc1d9b700 1 -- 172.21.15.29:0/35886172 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532339142 0 0) 0x7f3dc4065770 con 0x7f3dc40bca50 2024-01-24T15:13:23.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.757+0000 7f3dc359e700 1 -- 172.21.15.29:0/35886172 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc4069f60 con 0x7f3dc40627b0 2024-01-24T15:13:23.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.757+0000 7f3dc359e700 1 --2- 172.21.15.29:0/35886172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3dc40627b0 0x7f3dc4062c40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7f3dc40649f0 tx=0x7f3dbc009840).ready entity=mon.0 client_cookie=22bb5ffeec49a1fd server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:23.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.757+0000 7f3dc1d9b700 1 -- 172.21.15.29:0/35886172 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00e070 con 0x7f3dc40627b0 2024-01-24T15:13:23.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.757+0000 7f3dc1d9b700 1 -- 172.21.15.29:0/35886172 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc00ba20 con 0x7f3dc40627b0 2024-01-24T15:13:23.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.757+0000 7f3dc1d9b700 1 -- 172.21.15.29:0/35886172 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00bda0 con 0x7f3dc40627b0 2024-01-24T15:13:23.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.757+0000 7f3dc915d700 1 -- 172.21.15.29:0/35886172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3dc40627b0 msgr2=0x7f3dc4062c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.757+0000 7f3dc915d700 1 --2- 172.21.15.29:0/35886172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3dc40627b0 0x7f3dc4062c40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7f3dc40649f0 tx=0x7f3dbc009840).stop 2024-01-24T15:13:23.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.758+0000 7f3dc915d700 1 -- 172.21.15.29:0/35886172 shutdown_connections 2024-01-24T15:13:23.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.758+0000 7f3dc915d700 1 --2- 172.21.15.29:0/35886172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3dc40627b0 0x7f3dc4062c40 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.758+0000 7f3dc915d700 1 --2- 172.21.15.29:0/35886172 >> v2:172.21.15.163:3300/0 conn(0x7f3dc4138330 0x7f3dc4138710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.758+0000 7f3dc915d700 1 -- 172.21.15.29:0/35886172 >> 172.21.15.29:0/35886172 conn(0x7f3dc40c8580 msgr2=0x7f3dc40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:23.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.758+0000 7f3dc915d700 1 -- 172.21.15.29:0/35886172 shutdown_connections 2024-01-24T15:13:23.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.758+0000 7f3dc915d700 1 -- 172.21.15.29:0/35886172 wait complete. 2024-01-24T15:13:23.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.760+0000 7f3dc915d700 1 Processor -- start 2024-01-24T15:13:23.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.760+0000 7f3dc915d700 1 -- start start 2024-01-24T15:13:23.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.761+0000 7f3dc915d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dc40627b0 0x7f3dc40ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc915d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3dc40bca50 0x7f3dc40a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc915d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dc4138330 0x7f3dc40a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc915d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc40adbe0 con 0x7f3dc40bca50 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc915d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc40a5b00 con 0x7f3dc40627b0 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc915d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc40a5d80 con 0x7f3dc4138330 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3dc40bca50 0x7f3dc40a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3dc40bca50 0x7f3dc40a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35574/0 (socket says 172.21.15.29:35574) 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc259c700 1 -- 172.21.15.29:0/4247576359 learned_addr learned my addr 172.21.15.29:0/4247576359 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc2d9d700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dc40627b0 0x7f3dc40ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc359e700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dc4138330 0x7f3dc40a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc259c700 1 -- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dc4138330 msgr2=0x7f3dc40a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc259c700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dc4138330 0x7f3dc40a6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc259c700 1 -- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dc40627b0 msgr2=0x7f3dc40ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc259c700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dc40627b0 0x7f3dc40ad6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc259c700 1 -- 172.21.15.29:0/4247576359 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc009ae0 con 0x7f3dc40bca50 2024-01-24T15:13:23.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc2d9d700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dc40627b0 0x7f3dc40ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:23.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3dc259c700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3dc40bca50 0x7f3dc40a5610 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7f3db800c560 tx=0x7f3db8007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:23.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.762+0000 7f3daaffd700 1 -- 172.21.15.29:0/4247576359 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db80072e0 con 0x7f3dc40bca50 2024-01-24T15:13:23.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.763+0000 7f3daaffd700 1 -- 172.21.15.29:0/4247576359 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db8007440 con 0x7f3dc40bca50 2024-01-24T15:13:23.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.763+0000 7f3dc915d700 1 -- 172.21.15.29:0/4247576359 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc40a6060 con 0x7f3dc40bca50 2024-01-24T15:13:23.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.763+0000 7f3daaffd700 1 -- 172.21.15.29:0/4247576359 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db80058f0 con 0x7f3dc40bca50 2024-01-24T15:13:23.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.763+0000 7f3dc915d700 1 -- 172.21.15.29:0/4247576359 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc40a6620 con 0x7f3dc40bca50 2024-01-24T15:13:23.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.764+0000 7f3dc915d700 1 -- 172.21.15.29:0/4247576359 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc40475e0 con 0x7f3dc40bca50 2024-01-24T15:13:23.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.764+0000 7f3daaffd700 1 -- 172.21.15.29:0/4247576359 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 64) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f3db8005ce0 con 0x7f3dc40bca50 2024-01-24T15:13:23.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.764+0000 7f3daaffd700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3db005f240 0x7f3db0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:23.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.765+0000 7f3daaffd700 1 -- 172.21.15.29:0/4247576359 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 7546+0+0 (secure 0 0 0) 0x7f3db807a130 con 0x7f3dc40bca50 2024-01-24T15:13:23.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.765+0000 7f3dc2d9d700 1 -- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3db005f240 msgr2=0x7f3db0061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:23.768 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.765+0000 7f3dc2d9d700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3db005f240 0x7f3db0061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:23.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.769+0000 7f3daaffd700 1 -- 172.21.15.29:0/4247576359 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3db804ac30 con 0x7f3dc40bca50 2024-01-24T15:13:23.883 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:23.881+0000 7f6508dce680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:13:23.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.951+0000 7f3dc915d700 1 -- 172.21.15.29:0/4247576359 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dc406a240 con 0x7f3dc40bca50 2024-01-24T15:13:23.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.953+0000 7f3daaffd700 1 -- 172.21.15.29:0/4247576359 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+18821 (secure 0 0 0) 0x7f3db804e250 con 0x7f3dc40bca50 2024-01-24T15:13:23.955 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:23.955 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":164,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:13:22.619750+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:17:18.175741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-24T14:17:19.397106+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":164,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.3","mappings":[{"from":2,"to":0}]},{"pgid":"19.16","mappings":[{"from":2,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/1044393967":"2024-01-25T14:17:32.674163+0000","172.21.15.163:6833/43241":"2024-01-25T14:17:32.674163+0000","172.21.15.163:6832/43241":"2024-01-25T14:17:32.674163+0000","172.21.15.163:0/3112762339":"2024-01-25T14:17:32.674163+0000","172.21.15.163:0/2817247896":"2024-01-25T14:17:32.674163+0000","172.21.15.163:0/221549441":"2024-01-25T14:17:32.674163+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:13:23.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.955+0000 7f3dc915d700 1 -- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3db005f240 msgr2=0x7f3db0061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:23.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.955+0000 7f3dc915d700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3db005f240 0x7f3db0061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:23.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.955+0000 7f3dc915d700 1 -- 172.21.15.29:0/4247576359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3dc40bca50 msgr2=0x7f3dc40a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:23.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.955+0000 7f3dc915d700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3dc40bca50 0x7f3dc40a5610 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7f3db800c560 tx=0x7f3db8007c90).stop 2024-01-24T15:13:23.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.955+0000 7f3dc915d700 1 -- 172.21.15.29:0/4247576359 shutdown_connections 2024-01-24T15:13:23.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.955+0000 7f3dc915d700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dc4138330 0x7f3dc40a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.955+0000 7f3dc915d700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dc40627b0 0x7f3dc40ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.955+0000 7f3dc915d700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3db005f240 0x7f3db0061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:23.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.955+0000 7f3dc915d700 1 --2- 172.21.15.29:0/4247576359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3dc40bca50 0x7f3dc40a5610 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:23.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.955+0000 7f3dc915d700 1 -- 172.21.15.29:0/4247576359 >> 172.21.15.29:0/4247576359 conn(0x7f3dc40c8580 msgr2=0x7f3dc40c5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:23.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.955+0000 7f3dc915d700 1 -- 172.21.15.29:0/4247576359 shutdown_connections 2024-01-24T15:13:23.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:23.955+0000 7f3dc915d700 1 -- 172.21.15.29:0/4247576359 wait complete. 2024-01-24T15:13:23.973 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T15:13:23.973 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:13:24.046 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:24.044+0000 7f9a21a0e680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:13:24.115 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:24.113+0000 7f9a21a0e680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:13:24.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.142+0000 7f5e7cb56700 1 Processor -- start 2024-01-24T15:13:24.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.142+0000 7f5e7cb56700 1 -- start start 2024-01-24T15:13:24.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e7cb56700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e78154370 0x7f5e780c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e7cb56700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5e78154750 0x7f5e780631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e7cb56700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e780aa6c0 con 0x7f5e780aa0e0 2024-01-24T15:13:24.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e7cb56700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e78064240 con 0x7f5e78154750 2024-01-24T15:13:24.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e7cb56700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780cb5c0 con 0x7f5e78154370 2024-01-24T15:13:24.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e76d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5e78154750 0x7f5e780631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e76d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5e78154750 0x7f5e780631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35584/0 (socket says 172.21.15.29:35584) 2024-01-24T15:13:24.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e76d9d700 1 -- 172.21.15.29:0/2722325338 learned_addr learned my addr 172.21.15.29:0/2722325338 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:24.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e7659c700 1 --2- 172.21.15.29:0/2722325338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e78154370 0x7f5e780c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e76d9d700 1 -- 172.21.15.29:0/2722325338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e78154370 msgr2=0x7f5e780c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:24.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e7559a700 1 -- 172.21.15.29:0/2722325338 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425881194 0 0) 0x7f5e780aa6c0 con 0x7f5e780aa0e0 2024-01-24T15:13:24.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e76d9d700 1 --2- 172.21.15.29:0/2722325338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e78154370 0x7f5e780c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e76d9d700 1 -- 172.21.15.29:0/2722325338 >> v1:172.21.15.163:6789/0 conn(0x7f5e780aa0e0 legacy=0x7f5e780ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.144+0000 7f5e76d9d700 1 -- 172.21.15.29:0/2722325338 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e780cb8a0 con 0x7f5e78154750 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.145+0000 7f5e76d9d700 1 --2- 172.21.15.29:0/2722325338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e78154750 0x7f5e780631d0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7f5e70009f20 tx=0x7f5e70009840).ready entity=mon.0 client_cookie=ef915aea218dcd83 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.145+0000 7f5e7559a700 1 -- 172.21.15.29:0/2722325338 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000e070 con 0x7f5e78154750 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.145+0000 7f5e7559a700 1 -- 172.21.15.29:0/2722325338 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7000b9d0 con 0x7f5e78154750 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.145+0000 7f5e7559a700 1 -- 172.21.15.29:0/2722325338 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000bd50 con 0x7f5e78154750 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.145+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2722325338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e78154750 msgr2=0x7f5e780631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.145+0000 7f5e7cb56700 1 --2- 172.21.15.29:0/2722325338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e78154750 0x7f5e780631d0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7f5e70009f20 tx=0x7f5e70009840).stop 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.146+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2722325338 shutdown_connections 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.146+0000 7f5e7cb56700 1 --2- 172.21.15.29:0/2722325338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e78154370 0x7f5e780c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.146+0000 7f5e7cb56700 1 --2- 172.21.15.29:0/2722325338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e78154750 0x7f5e780631d0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.146+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2722325338 >> 172.21.15.29:0/2722325338 conn(0x7f5e7805ec80 msgr2=0x7f5e780cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.146+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2722325338 shutdown_connections 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.146+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2722325338 wait complete. 2024-01-24T15:13:24.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.147+0000 7f5e7cb56700 1 Processor -- start 2024-01-24T15:13:24.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.147+0000 7f5e7cb56700 1 -- start start 2024-01-24T15:13:24.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e7cb56700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e780aa0e0 0x7f5e7814e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e7cb56700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e78154370 0x7f5e78150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e7cb56700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e78154750 0x7f5e78150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e7cb56700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e78150f40 con 0x7f5e78154370 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e7cb56700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780c27c0 con 0x7f5e78154750 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e7cb56700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780c1eb0 con 0x7f5e780aa0e0 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e75d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e78154370 0x7f5e78150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e75d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e78154370 0x7f5e78150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35586/0 (socket says 172.21.15.29:35586) 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e75d9b700 1 -- 172.21.15.29:0/2301949712 learned_addr learned my addr 172.21.15.29:0/2301949712 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e7659c700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e780aa0e0 0x7f5e7814e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e76d9d700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e78154750 0x7f5e78150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e75d9b700 1 -- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e780aa0e0 msgr2=0x7f5e7814e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e75d9b700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e780aa0e0 0x7f5e7814e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e75d9b700 1 -- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e78154750 msgr2=0x7f5e78150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e75d9b700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e78154750 0x7f5e78150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e75d9b700 1 -- 172.21.15.29:0/2301949712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e70009ae0 con 0x7f5e78154370 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e75d9b700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e78154370 0x7f5e78150560 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f5e6c007fd0 tx=0x7f5e6c007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:24.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.148+0000 7f5e667fc700 1 -- 172.21.15.29:0/2301949712 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00ce20 con 0x7f5e78154370 2024-01-24T15:13:24.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.149+0000 7f5e667fc700 1 -- 172.21.15.29:0/2301949712 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c00f470 con 0x7f5e78154370 2024-01-24T15:13:24.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.149+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2301949712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e780c4a70 con 0x7f5e78154370 2024-01-24T15:13:24.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.149+0000 7f5e667fc700 1 -- 172.21.15.29:0/2301949712 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00f830 con 0x7f5e78154370 2024-01-24T15:13:24.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.149+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2301949712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e780c2450 con 0x7f5e78154370 2024-01-24T15:13:24.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.150+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2301949712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e780475e0 con 0x7f5e78154370 2024-01-24T15:13:24.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.150+0000 7f5e667fc700 1 -- 172.21.15.29:0/2301949712 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 64) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f5e6c00fba0 con 0x7f5e78154370 2024-01-24T15:13:24.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.150+0000 7f5e667fc700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5e5805f270 0x7f5e58061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.151+0000 7f5e667fc700 1 -- 172.21.15.29:0/2301949712 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 7546+0+0 (secure 0 0 0) 0x7f5e6c079e20 con 0x7f5e78154370 2024-01-24T15:13:24.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.151+0000 7f5e7659c700 1 -- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5e5805f270 msgr2=0x7f5e58061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:24.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.151+0000 7f5e7659c700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5e5805f270 0x7f5e58061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:24.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.155+0000 7f5e667fc700 1 -- 172.21.15.29:0/2301949712 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5e6c04a9e0 con 0x7f5e78154370 2024-01-24T15:13:24.192 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:24.190+0000 7f9a21a0e680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:13:24.339 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:24.336+0000 7f9a21a0e680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:13:24.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.352+0000 7f5e7659c700 1 -- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5e5805f270 msgr2=0x7f5e58061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:24.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.352+0000 7f5e7659c700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5e5805f270 0x7f5e58061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:13:24.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.383+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2301949712 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e780cbb80 con 0x7f5e78154370 2024-01-24T15:13:24.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.384+0000 7f5e667fc700 1 -- 172.21.15.29:0/2301949712 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f5e6c04e000 con 0x7f5e78154370 2024-01-24T15:13:24.386 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:24.386 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2106677945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2106677945},{"type":"v1","addr":"172.21.15.163:6836","nonce":2106677945}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7786,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1964541525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1964541525},{"type":"v1","addr":"172.21.15.163:6837","nonce":1964541525}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4162831288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4162831288},{"type":"v1","addr":"172.21.15.29:6835","nonce":4162831288}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:17:20.414311+0000","modified":"2024-01-24T14:17:23.419003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8166":{"gid":8166,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/1312733772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1312733772},{"type":"v1","addr":"172.21.15.29:6834","nonce":1312733772}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-24T15:13:24.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.386+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5e5805f270 msgr2=0x7f5e58061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:24.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.386+0000 7f5e7cb56700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5e5805f270 0x7f5e58061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:24.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.386+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2301949712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e78154370 msgr2=0x7f5e78150560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.386+0000 7f5e7cb56700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e78154370 0x7f5e78150560 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f5e6c007fd0 tx=0x7f5e6c007c40).stop 2024-01-24T15:13:24.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.386+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2301949712 shutdown_connections 2024-01-24T15:13:24.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.386+0000 7f5e7cb56700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e78154750 0x7f5e78150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.386+0000 7f5e7cb56700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e780aa0e0 0x7f5e7814e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.386+0000 7f5e7cb56700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5e5805f270 0x7f5e58061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:24.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.386+0000 7f5e7cb56700 1 --2- 172.21.15.29:0/2301949712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e78154370 0x7f5e78150560 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.386+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2301949712 >> 172.21.15.29:0/2301949712 conn(0x7f5e7805ec80 msgr2=0x7f5e7814fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:24.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.386+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2301949712 shutdown_connections 2024-01-24T15:13:24.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.386+0000 7f5e7cb56700 1 -- 172.21.15.29:0/2301949712 wait complete. 2024-01-24T15:13:24.389 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 66 2024-01-24T15:13:24.401 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:13:24.410 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:24.408+0000 7f9a21a0e680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:13:24.525 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:24.523+0000 7f9a21a0e680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:13:24.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.534+0000 7f8e746e1700 1 Processor -- start 2024-01-24T15:13:24.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.534+0000 7f8e746e1700 1 -- start start 2024-01-24T15:13:24.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.535+0000 7f8e746e1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e6c13a5c0 0x7f8e6c063a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.535+0000 7f8e746e1700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e6c0bf6a0 con 0x7f8e6c066c00 2024-01-24T15:13:24.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.535+0000 7f8e746e1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e6c0bea90 con 0x7f8e6c0be520 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.535+0000 7f8e746e1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0bed70 con 0x7f8e6c13a5c0 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.535+0000 7f8e72c7e700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8e6c066c00 0x7f8e6c0c31b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52326/0 (socket says 172.21.15.29:52326) 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.535+0000 7f8e72c7e700 1 -- 172.21.15.29:0/3494472993 learned_addr learned my addr 172.21.15.29:0/3494472993 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.536+0000 7f8e7247d700 1 --2- 172.21.15.29:0/3494472993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e6c13a5c0 0x7f8e6c063a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.536+0000 7f8e7147b700 1 -- 172.21.15.29:0/3494472993 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099897313 0 0) 0x7f8e6c0bf6a0 con 0x7f8e6c066c00 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.536+0000 7f8e7147b700 1 -- 172.21.15.29:0/3494472993 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e540033f0 con 0x7f8e6c066c00 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.536+0000 7f8e7147b700 1 -- 172.21.15.29:0/3494472993 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962623987 0 0) 0x7f8e6c0bea90 con 0x7f8e6c0be520 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.536+0000 7f8e7247d700 1 -- 172.21.15.29:0/3494472993 >> v1:172.21.15.163:6789/0 conn(0x7f8e6c0be520 legacy=0x7f8e6c0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.536+0000 7f8e7247d700 1 -- 172.21.15.29:0/3494472993 >> v1:172.21.15.29:6789/0 conn(0x7f8e6c066c00 legacy=0x7f8e6c0c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.536+0000 7f8e7247d700 1 -- 172.21.15.29:0/3494472993 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c0bf050 con 0x7f8e6c13a5c0 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.536+0000 7f8e7247d700 1 --2- 172.21.15.29:0/3494472993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e6c13a5c0 0x7f8e6c063a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f8e64009790 tx=0x7f8e64009060).ready entity=mon.2 client_cookie=57d81c2cfae5101 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.536+0000 7f8e7147b700 1 -- 172.21.15.29:0/3494472993 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6400f510 con 0x7f8e6c13a5c0 2024-01-24T15:13:24.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.536+0000 7f8e7147b700 1 -- 172.21.15.29:0/3494472993 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6400f670 con 0x7f8e6c13a5c0 2024-01-24T15:13:24.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.536+0000 7f8e7147b700 1 -- 172.21.15.29:0/3494472993 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6400f9f0 con 0x7f8e6c13a5c0 2024-01-24T15:13:24.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.537+0000 7f8e746e1700 1 -- 172.21.15.29:0/3494472993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e6c13a5c0 msgr2=0x7f8e6c063a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.537+0000 7f8e746e1700 1 --2- 172.21.15.29:0/3494472993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e6c13a5c0 0x7f8e6c063a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f8e64009790 tx=0x7f8e64009060).stop 2024-01-24T15:13:24.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.537+0000 7f8e746e1700 1 -- 172.21.15.29:0/3494472993 shutdown_connections 2024-01-24T15:13:24.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.537+0000 7f8e746e1700 1 --2- 172.21.15.29:0/3494472993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e6c13a5c0 0x7f8e6c063a10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.537+0000 7f8e746e1700 1 -- 172.21.15.29:0/3494472993 >> 172.21.15.29:0/3494472993 conn(0x7f8e6c0c8bb0 msgr2=0x7f8e6c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:24.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.537+0000 7f8e746e1700 1 -- 172.21.15.29:0/3494472993 shutdown_connections 2024-01-24T15:13:24.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.537+0000 7f8e746e1700 1 -- 172.21.15.29:0/3494472993 wait complete. 2024-01-24T15:13:24.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.538+0000 7f8e746e1700 1 Processor -- start 2024-01-24T15:13:24.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.539+0000 7f8e746e1700 1 -- start start 2024-01-24T15:13:24.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.539+0000 7f8e746e1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6c066c00 0x7f8e6c157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.539+0000 7f8e746e1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e6c0be520 0x7f8e6c1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.539+0000 7f8e746e1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e6c13a5c0 0x7f8e6c0a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.539+0000 7f8e746e1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c158590 con 0x7f8e6c066c00 2024-01-24T15:13:24.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.539+0000 7f8e746e1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0a7340 con 0x7f8e6c0be520 2024-01-24T15:13:24.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.539+0000 7f8e746e1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0a9720 con 0x7f8e6c13a5c0 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e7247d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6c066c00 0x7f8e6c157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e7247d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6c066c00 0x7f8e6c157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35602/0 (socket says 172.21.15.29:35602) 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e7247d700 1 -- 172.21.15.29:0/2575117617 learned_addr learned my addr 172.21.15.29:0/2575117617 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e71c7c700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e6c0be520 0x7f8e6c1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e72c7e700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e6c13a5c0 0x7f8e6c0a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e7247d700 1 -- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e6c13a5c0 msgr2=0x7f8e6c0a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e7247d700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e6c13a5c0 0x7f8e6c0a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e7247d700 1 -- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e6c0be520 msgr2=0x7f8e6c1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e7247d700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e6c0be520 0x7f8e6c1580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e7247d700 1 -- 172.21.15.29:0/2575117617 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e640092a0 con 0x7f8e6c066c00 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e7247d700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6c066c00 0x7f8e6c157bb0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f8e6c06a130 tx=0x7f8e64017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e6a7fc700 1 -- 172.21.15.29:0/2575117617 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6400fe80 con 0x7f8e6c066c00 2024-01-24T15:13:24.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e6a7fc700 1 -- 172.21.15.29:0/2575117617 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6400f7e0 con 0x7f8e6c066c00 2024-01-24T15:13:24.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e746e1700 1 -- 172.21.15.29:0/2575117617 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6c0a8df0 con 0x7f8e6c066c00 2024-01-24T15:13:24.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e6a7fc700 1 -- 172.21.15.29:0/2575117617 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6401b640 con 0x7f8e6c066c00 2024-01-24T15:13:24.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.540+0000 7f8e746e1700 1 -- 172.21.15.29:0/2575117617 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6c0a93b0 con 0x7f8e6c066c00 2024-01-24T15:13:24.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.542+0000 7f8e6a7fc700 1 -- 172.21.15.29:0/2575117617 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 64) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f8e6401b9b0 con 0x7f8e6c066c00 2024-01-24T15:13:24.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.542+0000 7f8e746e1700 1 -- 172.21.15.29:0/2575117617 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6c0a9a00 con 0x7f8e6c066c00 2024-01-24T15:13:24.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.542+0000 7f8e6a7fc700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e5805f270 0x7f8e58061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.542+0000 7f8e6a7fc700 1 -- 172.21.15.29:0/2575117617 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 7546+0+0 (secure 0 0 0) 0x7f8e6407c2a0 con 0x7f8e6c066c00 2024-01-24T15:13:24.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.543+0000 7f8e71c7c700 1 -- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e5805f270 msgr2=0x7f8e58061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:24.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.543+0000 7f8e71c7c700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e5805f270 0x7f8e58061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:24.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.547+0000 7f8e6a7fc700 1 -- 172.21.15.29:0/2575117617 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8e6404ce60 con 0x7f8e6c066c00 2024-01-24T15:13:24.636 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:24.634+0000 7f9a21a0e680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:13:24.656 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:24.654+0000 7f6508dce680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:13:24.736 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:24.734+0000 7f6508dce680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:13:24.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.744+0000 7f8e71c7c700 1 -- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e5805f270 msgr2=0x7f8e58061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:24.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.744+0000 7f8e71c7c700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e5805f270 0x7f8e58061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:13:24.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.777+0000 7f8e746e1700 1 -- 172.21.15.29:0/2575117617 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e6c0475e0 con 0x7f8e6c066c00 2024-01-24T15:13:24.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.778+0000 7f8e6a7fc700 1 -- 172.21.15.29:0/2575117617 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f8e64050480 con 0x7f8e6c066c00 2024-01-24T15:13:24.780 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:24.780 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2106677945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2106677945},{"type":"v1","addr":"172.21.15.163:6836","nonce":2106677945}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7786,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1964541525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1964541525},{"type":"v1","addr":"172.21.15.163:6837","nonce":1964541525}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4162831288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4162831288},{"type":"v1","addr":"172.21.15.29:6835","nonce":4162831288}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T14:17:20.414311+0000","modified":"2024-01-24T14:17:23.419003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8166":{"gid":8166,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/1312733772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1312733772},{"type":"v1","addr":"172.21.15.29:6834","nonce":1312733772}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-24T15:13:24.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.780+0000 7f8e746e1700 1 -- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e5805f270 msgr2=0x7f8e58061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:24.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.780+0000 7f8e746e1700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e5805f270 0x7f8e58061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:24.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.780+0000 7f8e746e1700 1 -- 172.21.15.29:0/2575117617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6c066c00 msgr2=0x7f8e6c157bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.780+0000 7f8e746e1700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6c066c00 0x7f8e6c157bb0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f8e6c06a130 tx=0x7f8e64017000).stop 2024-01-24T15:13:24.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.780+0000 7f8e746e1700 1 -- 172.21.15.29:0/2575117617 shutdown_connections 2024-01-24T15:13:24.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.780+0000 7f8e746e1700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e6c13a5c0 0x7f8e6c0a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.780+0000 7f8e746e1700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f8e5805f270 0x7f8e58061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:24.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.780+0000 7f8e746e1700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e6c066c00 0x7f8e6c157bb0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.780+0000 7f8e746e1700 1 --2- 172.21.15.29:0/2575117617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e6c0be520 0x7f8e6c1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.780+0000 7f8e746e1700 1 -- 172.21.15.29:0/2575117617 >> 172.21.15.29:0/2575117617 conn(0x7f8e6c0c8bb0 msgr2=0x7f8e6c0c7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:24.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.780+0000 7f8e746e1700 1 -- 172.21.15.29:0/2575117617 shutdown_connections 2024-01-24T15:13:24.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.780+0000 7f8e746e1700 1 -- 172.21.15.29:0/2575117617 wait complete. 2024-01-24T15:13:24.787 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 66 2024-01-24T15:13:24.796 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:13:24.824 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:24.822+0000 7f6508dce680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:13:24.926 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.924+0000 7f4548312700 1 Processor -- start 2024-01-24T15:13:24.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.924+0000 7f4548312700 1 -- start start 2024-01-24T15:13:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.925+0000 7f4548312700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4540154370 0x7f45400c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.925+0000 7f4548312700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45400aa0e0 0x7f4540154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f4548312700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45400aa6c0 con 0x7f45400ab1f0 2024-01-24T15:13:24.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f4548312700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4540064240 con 0x7f45400aa0e0 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f4548312700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45400cb5c0 con 0x7f4540154370 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f45458ad700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45400aa0e0 0x7f4540154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f45458ad700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45400aa0e0 0x7f4540154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35610/0 (socket says 172.21.15.29:35610) 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f45458ad700 1 -- 172.21.15.29:0/2214588391 learned_addr learned my addr 172.21.15.29:0/2214588391 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f45460ae700 1 --2- 172.21.15.29:0/2214588391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4540154370 0x7f45400c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f45458ad700 1 -- 172.21.15.29:0/2214588391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4540154370 msgr2=0x7f45400c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f45458ad700 1 --2- 172.21.15.29:0/2214588391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4540154370 0x7f45400c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f45458ad700 1 -- 172.21.15.29:0/2214588391 >> v1:172.21.15.163:6789/0 conn(0x7f45400ab1f0 legacy=0x7f45400631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f45458ad700 1 -- 172.21.15.29:0/2214588391 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45400cb8a0 con 0x7f45400aa0e0 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f45458ad700 1 --2- 172.21.15.29:0/2214588391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45400aa0e0 0x7f4540154750 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f4534006fd0 tx=0x7f45340090f0).ready entity=mon.0 client_cookie=da37e105ac508e10 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f45450ac700 1 -- 172.21.15.29:0/2214588391 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400b580 con 0x7f45400aa0e0 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f45450ac700 1 -- 172.21.15.29:0/2214588391 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453400b6e0 con 0x7f45400aa0e0 2024-01-24T15:13:24.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.926+0000 7f45450ac700 1 -- 172.21.15.29:0/2214588391 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400ba60 con 0x7f45400aa0e0 2024-01-24T15:13:24.930 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:24.927+0000 7f9a21a0e680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:13:24.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.927+0000 7f4548312700 1 -- 172.21.15.29:0/2214588391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45400aa0e0 msgr2=0x7f4540154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.927+0000 7f4548312700 1 --2- 172.21.15.29:0/2214588391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45400aa0e0 0x7f4540154750 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f4534006fd0 tx=0x7f45340090f0).stop 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.927+0000 7f4548312700 1 -- 172.21.15.29:0/2214588391 shutdown_connections 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.927+0000 7f4548312700 1 --2- 172.21.15.29:0/2214588391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4540154370 0x7f45400c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.927+0000 7f4548312700 1 --2- 172.21.15.29:0/2214588391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45400aa0e0 0x7f4540154750 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.927+0000 7f4548312700 1 -- 172.21.15.29:0/2214588391 >> 172.21.15.29:0/2214588391 conn(0x7f454005ec80 msgr2=0x7f45400cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.927+0000 7f4548312700 1 -- 172.21.15.29:0/2214588391 shutdown_connections 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.927+0000 7f4548312700 1 -- 172.21.15.29:0/2214588391 wait complete. 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.928+0000 7f4548312700 1 Processor -- start 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.929+0000 7f4548312700 1 -- start start 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.929+0000 7f4548312700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45400aa0e0 0x7f4540159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.929+0000 7f4548312700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45400ab1f0 0x7f4540168630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.929+0000 7f4548312700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4540154370 0x7f4540152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.929+0000 7f4548312700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4540165020 con 0x7f45400aa0e0 2024-01-24T15:13:24.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.929+0000 7f4548312700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4540150450 con 0x7f4540154370 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.929+0000 7f4548312700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4540165180 con 0x7f45400ab1f0 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.929+0000 7f45460ae700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45400aa0e0 0x7f4540159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f45460ae700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45400aa0e0 0x7f4540159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35624/0 (socket says 172.21.15.29:35624) 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f45460ae700 1 -- 172.21.15.29:0/1893065716 learned_addr learned my addr 172.21.15.29:0/1893065716 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f45458ad700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45400ab1f0 0x7f4540168630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f45468af700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4540154370 0x7f4540152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f45460ae700 1 -- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45400ab1f0 msgr2=0x7f4540168630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f45460ae700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45400ab1f0 0x7f4540168630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f45460ae700 1 -- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4540154370 msgr2=0x7f4540152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f45460ae700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4540154370 0x7f4540152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f45460ae700 1 -- 172.21.15.29:0/1893065716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4534009370 con 0x7f45400aa0e0 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f45460ae700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45400aa0e0 0x7f4540159d20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f453800e8d0 tx=0x7f4538006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f453e7fc700 1 -- 172.21.15.29:0/1893065716 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538012600 con 0x7f45400aa0e0 2024-01-24T15:13:24.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f453e7fc700 1 -- 172.21.15.29:0/1893065716 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4538012760 con 0x7f45400aa0e0 2024-01-24T15:13:24.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f4548312700 1 -- 172.21.15.29:0/1893065716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45401571c0 con 0x7f45400aa0e0 2024-01-24T15:13:24.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f453e7fc700 1 -- 172.21.15.29:0/1893065716 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538012ae0 con 0x7f45400aa0e0 2024-01-24T15:13:24.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.930+0000 7f4548312700 1 -- 172.21.15.29:0/1893065716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4540157780 con 0x7f45400aa0e0 2024-01-24T15:13:24.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.931+0000 7f4548312700 1 -- 172.21.15.29:0/1893065716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45400caad0 con 0x7f45400aa0e0 2024-01-24T15:13:24.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.932+0000 7f453e7fc700 1 -- 172.21.15.29:0/1893065716 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 64) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f453801b400 con 0x7f45400aa0e0 2024-01-24T15:13:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.932+0000 7f453e7fc700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f452c05f240 0x7f452c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.932+0000 7f453e7fc700 1 -- 172.21.15.29:0/1893065716 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 7546+0+0 (secure 0 0 0) 0x7f453807ac70 con 0x7f45400aa0e0 2024-01-24T15:13:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.932+0000 7f45458ad700 1 -- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f452c05f240 msgr2=0x7f452c061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:24.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.932+0000 7f45458ad700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f452c05f240 0x7f452c061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:24.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:24.936+0000 7f453e7fc700 1 -- 172.21.15.29:0/1893065716 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f453804b7d0 con 0x7f45400aa0e0 2024-01-24T15:13:24.997 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:24.995+0000 7f6508dce680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:13:25.001 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:24.999+0000 7f9a21a0e680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:13:25.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.024+0000 7f453e7fc700 1 -- 172.21.15.29:0/1893065716 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 65) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f4538078330 con 0x7f45400aa0e0 2024-01-24T15:13:25.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.024+0000 7f453e7fc700 1 -- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f452c05f240 msgr2=0x7f452c061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:25.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.024+0000 7f453e7fc700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f452c05f240 0x7f452c061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:25.087 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:25.085+0000 7f6508dce680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:13:25.087 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:25.085+0000 7f999d794700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:25.087 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:25.085+0000 7f999d794700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:25.087 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:25.085+0000 7f999d794700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:25.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:25.085+0000 7f999d794700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:25.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:25.085+0000 7f999d794700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:25.089 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:25.087+0000 7f999978c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:25.089 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:25.087+0000 7f999978c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:25.090 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:25.087+0000 7f999978c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:25.090 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:25.087+0000 7f999978c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:25.090 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:25.087+0000 7f999978c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:25.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.129+0000 7f4548312700 1 -- 172.21.15.29:0/1893065716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45400475e0 con 0x7f45400aa0e0 2024-01-24T15:13:25.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.130+0000 7f453e7fc700 1 -- 172.21.15.29:0/1893065716 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+18998 (secure 0 0 0) 0x7f453806cfc0 con 0x7f45400aa0e0 2024-01-24T15:13:25.132 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:25.132 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":165,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:13:25.014209+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-24T14:17:18.175741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-24T14:17:19.397106+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":164,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.3","mappings":[{"from":2,"to":0}]},{"pgid":"19.16","mappings":[{"from":2,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/81313215":"2024-01-25T15:13:25.014170+0000","172.21.15.163:0/1609152438":"2024-01-25T15:13:25.014170+0000","172.21.15.163:0/1946204":"2024-01-25T15:13:25.014170+0000","172.21.15.163:0/3056918050":"2024-01-25T15:13:25.014170+0000","172.21.15.163:0/1044393967":"2024-01-25T14:17:32.674163+0000","172.21.15.163:6833/43241":"2024-01-25T14:17:32.674163+0000","172.21.15.163:6832/43241":"2024-01-25T14:17:32.674163+0000","172.21.15.163:0/3112762339":"2024-01-25T14:17:32.674163+0000","172.21.15.163:0/2817247896":"2024-01-25T14:17:32.674163+0000","172.21.15.163:0/221549441":"2024-01-25T14:17:32.674163+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:13:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.132+0000 7f4548312700 1 -- 172.21.15.29:0/1893065716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45400aa0e0 msgr2=0x7f4540159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.132+0000 7f4548312700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45400aa0e0 0x7f4540159d20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f453800e8d0 tx=0x7f4538006f40).stop 2024-01-24T15:13:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.132+0000 7f4548312700 1 -- 172.21.15.29:0/1893065716 shutdown_connections 2024-01-24T15:13:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.132+0000 7f4548312700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4540154370 0x7f4540152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.132+0000 7f4548312700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f452c05f240 0x7f452c061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.132+0000 7f4548312700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45400aa0e0 0x7f4540159d20 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.132+0000 7f4548312700 1 --2- 172.21.15.29:0/1893065716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45400ab1f0 0x7f4540168630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.132+0000 7f4548312700 1 -- 172.21.15.29:0/1893065716 >> 172.21.15.29:0/1893065716 conn(0x7f454005ec80 msgr2=0x7f4540151890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.132+0000 7f4548312700 1 -- 172.21.15.29:0/1893065716 shutdown_connections 2024-01-24T15:13:25.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.132+0000 7f4548312700 1 -- 172.21.15.29:0/1893065716 wait complete. 2024-01-24T15:13:25.148 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-24T15:13:25.228 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:25.226+0000 7f6508dce680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:13:25.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.318+0000 7fb4c0a3e700 1 Processor -- start 2024-01-24T15:13:25.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.318+0000 7fb4c0a3e700 1 -- start start 2024-01-24T15:13:25.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.319+0000 7fb4c0a3e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb4bc13de40 0x7fb4bc0bd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:25.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.319+0000 7fb4c0a3e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4bc0c1830 0x7fb4bc0c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.319+0000 7fb4c0a3e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc063540 con 0x7fb4bc13e220 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.319+0000 7fb4c0a3e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4bc0c2160 con 0x7fb4bc13de40 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.319+0000 7fb4c0a3e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc0640e0 con 0x7fb4bc0c1830 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4bad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4bc0c1830 0x7fb4bc0c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4ba59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb4bc13de40 0x7fb4bc0bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4b9d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb4bc13e220 0x7fb4bc0639d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:40746/0 (socket says 172.21.15.29:40746) 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4bad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4bc0c1830 0x7fb4bc0c1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:56422/0 (socket says 172.21.15.29:56422) 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4b9d9b700 1 -- 172.21.15.29:0/183022982 learned_addr learned my addr 172.21.15.29:0/183022982 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4b959a700 1 -- 172.21.15.29:0/183022982 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303143255 0 0) 0x7fb4bc063540 con 0x7fb4bc13e220 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4bad9d700 1 -- 172.21.15.29:0/183022982 >> v2:172.21.15.163:3300/0 conn(0x7fb4bc13de40 msgr2=0x7fb4bc0bd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4bad9d700 1 --2- 172.21.15.29:0/183022982 >> v2:172.21.15.163:3300/0 conn(0x7fb4bc13de40 0x7fb4bc0bd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4bad9d700 1 -- 172.21.15.29:0/183022982 >> v1:172.21.15.163:6789/0 conn(0x7fb4bc13e220 legacy=0x7fb4bc0639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:25.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4bad9d700 1 -- 172.21.15.29:0/183022982 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc062930 con 0x7fb4bc0c1830 2024-01-24T15:13:25.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4bad9d700 1 --2- 172.21.15.29:0/183022982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4bc0c1830 0x7fb4bc0c1c70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fb4b4009e60 tx=0x7fb4b4009780).ready entity=mon.2 client_cookie=e35f3d2675d61bfe server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:25.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4b959a700 1 -- 172.21.15.29:0/183022982 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400e070 con 0x7fb4bc0c1830 2024-01-24T15:13:25.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4b959a700 1 -- 172.21.15.29:0/183022982 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b400bb60 con 0x7fb4bc0c1830 2024-01-24T15:13:25.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.320+0000 7fb4b959a700 1 -- 172.21.15.29:0/183022982 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4012410 con 0x7fb4bc0c1830 2024-01-24T15:13:25.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.321+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/183022982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4bc0c1830 msgr2=0x7fb4bc0c1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:25.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.321+0000 7fb4c0a3e700 1 --2- 172.21.15.29:0/183022982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4bc0c1830 0x7fb4bc0c1c70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fb4b4009e60 tx=0x7fb4b4009780).stop 2024-01-24T15:13:25.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.321+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/183022982 shutdown_connections 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.321+0000 7fb4c0a3e700 1 --2- 172.21.15.29:0/183022982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4bc0c1830 0x7fb4bc0c1c70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.321+0000 7fb4c0a3e700 1 --2- 172.21.15.29:0/183022982 >> v2:172.21.15.163:3300/0 conn(0x7fb4bc13de40 0x7fb4bc0bd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.321+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/183022982 >> 172.21.15.29:0/183022982 conn(0x7fb4bc0c3cd0 msgr2=0x7fb4bc0c40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.321+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/183022982 shutdown_connections 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.321+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/183022982 wait complete. 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.322+0000 7fb4c0a3e700 1 Processor -- start 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.322+0000 7fb4c0a3e700 1 -- start start 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.323+0000 7fb4c0a3e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4bc0c1830 0x7fb4bc1570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.323+0000 7fb4c0a3e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4bc13de40 0x7fb4bc06a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.323+0000 7fb4c0a3e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4bc13e220 0x7fb4bc135040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.323+0000 7fb4c0a3e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc068db0 con 0x7fb4bc0c1830 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.323+0000 7fb4c0a3e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc06a650 con 0x7fb4bc13de40 2024-01-24T15:13:25.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.323+0000 7fb4c0a3e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc13ac30 con 0x7fb4bc13e220 2024-01-24T15:13:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.323+0000 7fb4ba59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4bc0c1830 0x7fb4bc1570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.323+0000 7fb4b9d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4bc13de40 0x7fb4bc06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.323+0000 7fb4ba59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4bc0c1830 0x7fb4bc1570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35630/0 (socket says 172.21.15.29:35630) 2024-01-24T15:13:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.323+0000 7fb4ba59c700 1 -- 172.21.15.29:0/3805330560 learned_addr learned my addr 172.21.15.29:0/3805330560 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.323+0000 7fb4bad9d700 1 --2- 172.21.15.29:0/3805330560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4bc13e220 0x7fb4bc135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.324+0000 7fb4ba59c700 1 -- 172.21.15.29:0/3805330560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4bc13e220 msgr2=0x7fb4bc135040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.324+0000 7fb4ba59c700 1 --2- 172.21.15.29:0/3805330560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4bc13e220 0x7fb4bc135040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.324+0000 7fb4ba59c700 1 -- 172.21.15.29:0/3805330560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4bc13de40 msgr2=0x7fb4bc06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.324+0000 7fb4ba59c700 1 --2- 172.21.15.29:0/3805330560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4bc13de40 0x7fb4bc06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.324+0000 7fb4ba59c700 1 -- 172.21.15.29:0/3805330560 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b4009a20 con 0x7fb4bc0c1830 2024-01-24T15:13:25.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.324+0000 7fb4b9d9b700 1 --2- 172.21.15.29:0/3805330560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4bc13de40 0x7fb4bc06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.324+0000 7fb4ba59c700 1 --2- 172.21.15.29:0/3805330560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4bc0c1830 0x7fb4bc1570b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7fb4b000e930 tx=0x7fb4b0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.324+0000 7fb4aa7fc700 1 -- 172.21.15.29:0/3805330560 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0012600 con 0x7fb4bc0c1830 2024-01-24T15:13:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.324+0000 7fb4aa7fc700 1 -- 172.21.15.29:0/3805330560 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b0012760 con 0x7fb4bc0c1830 2024-01-24T15:13:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.324+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/3805330560 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc13af10 con 0x7fb4bc0c1830 2024-01-24T15:13:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.324+0000 7fb4aa7fc700 1 -- 172.21.15.29:0/3805330560 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0012ae0 con 0x7fb4bc0c1830 2024-01-24T15:13:25.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.324+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/3805330560 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc135530 con 0x7fb4bc0c1830 2024-01-24T15:13:25.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.325+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/3805330560 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc0475e0 con 0x7fb4bc0c1830 2024-01-24T15:13:25.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.325+0000 7fb4aa7fc700 1 -- 172.21.15.29:0/3805330560 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 65) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fb4b001b400 con 0x7fb4bc0c1830 2024-01-24T15:13:25.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.326+0000 7fb4aa7fc700 1 -- 172.21.15.29:0/3805330560 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 7674+0+0 (secure 0 0 0) 0x7fb4b007a9b0 con 0x7fb4bc0c1830 2024-01-24T15:13:25.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.330+0000 7fb4aa7fc700 1 -- 172.21.15.29:0/3805330560 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb4b004b7e0 con 0x7fb4bc0c1830 2024-01-24T15:13:25.358 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:25.356+0000 7f6508dce680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:13:25.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:25.556+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/3805330560 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb4bc05e570 con 0x7fb4bc0c1830 2024-01-24T15:13:25.697 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:25.695+0000 7f6508dce680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:13:25.778 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:25.776+0000 7f6508dce680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:13:26.023 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-01-24T15:13:26.018+0000 7fe09bfc3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-24T15:13:26.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.022+0000 7fb4aa7fc700 1 -- 172.21.15.29:0/3805330560 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb4b004ee00 con 0x7fb4bc0c1830 2024-01-24T15:13:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.026+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/3805330560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4bc0c1830 msgr2=0x7fb4bc1570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.026+0000 7fb4c0a3e700 1 --2- 172.21.15.29:0/3805330560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4bc0c1830 0x7fb4bc1570b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7fb4b000e930 tx=0x7fb4b0009060).stop 2024-01-24T15:13:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.026+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/3805330560 shutdown_connections 2024-01-24T15:13:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.026+0000 7fb4c0a3e700 1 --2- 172.21.15.29:0/3805330560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4bc13e220 0x7fb4bc135040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.026+0000 7fb4c0a3e700 1 --2- 172.21.15.29:0/3805330560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4bc0c1830 0x7fb4bc1570b0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.026+0000 7fb4c0a3e700 1 --2- 172.21.15.29:0/3805330560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4bc13de40 0x7fb4bc06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.026+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/3805330560 >> 172.21.15.29:0/3805330560 conn(0x7fb4bc0c3cd0 msgr2=0x7fb4bc0a64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.027+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/3805330560 shutdown_connections 2024-01-24T15:13:26.029 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.027+0000 7fb4c0a3e700 1 -- 172.21.15.29:0/3805330560 wait complete. 2024-01-24T15:13:26.030 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-24T15:13:26.050 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-24T15:13:26.070 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T15:13:26.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.200+0000 7f4eee925700 1 Processor -- start 2024-01-24T15:13:26.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.200+0000 7f4eee925700 1 -- start start 2024-01-24T15:13:26.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.201+0000 7f4eee925700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee80663b0 0x7f4ee8152f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:26.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.201+0000 7f4eee925700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4ee8065420 0x7f4ee80658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:26.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.201+0000 7f4eee925700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee8069690 con 0x7f4ee80be4b0 2024-01-24T15:13:26.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.201+0000 7f4eee925700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee80be930 con 0x7f4ee8065420 2024-01-24T15:13:26.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.201+0000 7f4eee925700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee8065dc0 con 0x7f4ee80663b0 2024-01-24T15:13:26.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.201+0000 7f4eecec2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4ee8065420 0x7f4ee80658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:26.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.201+0000 7f4eecec2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4ee8065420 0x7f4ee80658d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35652/0 (socket says 172.21.15.29:35652) 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.201+0000 7f4eecec2700 1 -- 172.21.15.29:0/4054298077 learned_addr learned my addr 172.21.15.29:0/4054298077 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.201+0000 7f4ee7fff700 1 --2- 172.21.15.29:0/4054298077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee80663b0 0x7f4ee8152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4eecec2700 1 -- 172.21.15.29:0/4054298077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee80663b0 msgr2=0x7f4ee8152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4eecec2700 1 --2- 172.21.15.29:0/4054298077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee80663b0 0x7f4ee8152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4eecec2700 1 -- 172.21.15.29:0/4054298077 >> v1:172.21.15.163:6789/0 conn(0x7f4ee80be4b0 legacy=0x7f4ee8069b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4eecec2700 1 -- 172.21.15.29:0/4054298077 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee80bcc40 con 0x7f4ee8065420 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4eecec2700 1 --2- 172.21.15.29:0/4054298077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee8065420 0x7f4ee80658d0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f4ee0009f20 tx=0x7f4ee0009840).ready entity=mon.0 client_cookie=379e03a1ef89af2d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4ee6ffd700 1 -- 172.21.15.29:0/4054298077 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee000e070 con 0x7f4ee8065420 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4ee6ffd700 1 -- 172.21.15.29:0/4054298077 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee000ba40 con 0x7f4ee8065420 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4ee6ffd700 1 -- 172.21.15.29:0/4054298077 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee000bdc0 con 0x7f4ee8065420 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4eee925700 1 -- 172.21.15.29:0/4054298077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee8065420 msgr2=0x7f4ee80658d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4eee925700 1 --2- 172.21.15.29:0/4054298077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee8065420 0x7f4ee80658d0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f4ee0009f20 tx=0x7f4ee0009840).stop 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4eee925700 1 -- 172.21.15.29:0/4054298077 shutdown_connections 2024-01-24T15:13:26.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4eee925700 1 --2- 172.21.15.29:0/4054298077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee80663b0 0x7f4ee8152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:26.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4eee925700 1 --2- 172.21.15.29:0/4054298077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee8065420 0x7f4ee80658d0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:26.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.202+0000 7f4eee925700 1 -- 172.21.15.29:0/4054298077 >> 172.21.15.29:0/4054298077 conn(0x7f4ee8064390 msgr2=0x7f4ee80c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:26.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.203+0000 7f4eee925700 1 -- 172.21.15.29:0/4054298077 shutdown_connections 2024-01-24T15:13:26.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.203+0000 7f4eee925700 1 -- 172.21.15.29:0/4054298077 wait complete. 2024-01-24T15:13:26.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.204+0000 7f4eee925700 1 Processor -- start 2024-01-24T15:13:26.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.204+0000 7f4eee925700 1 -- start start 2024-01-24T15:13:26.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4eee925700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee8065420 0x7f4ee813b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4eee925700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80663b0 0x7f4ee8167b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4eee925700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee80be4b0 0x7f4ee8168020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4eee925700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee8168510 con 0x7f4ee80663b0 2024-01-24T15:13:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4eee925700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee81399b0 con 0x7f4ee8065420 2024-01-24T15:13:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4eee925700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee8139c90 con 0x7f4ee80be4b0 2024-01-24T15:13:26.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4ee77fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80663b0 0x7f4ee8167b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4ee77fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80663b0 0x7f4ee8167b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35664/0 (socket says 172.21.15.29:35664) 2024-01-24T15:13:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4ee77fe700 1 -- 172.21.15.29:0/3032695305 learned_addr learned my addr 172.21.15.29:0/3032695305 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4eecec2700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee80be4b0 0x7f4ee8168020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4ee7fff700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee8065420 0x7f4ee813b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4ee77fe700 1 -- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee80be4b0 msgr2=0x7f4ee8168020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:26.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4ee77fe700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee80be4b0 0x7f4ee8168020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4ee77fe700 1 -- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee8065420 msgr2=0x7f4ee813b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4ee77fe700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee8065420 0x7f4ee813b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4ee77fe700 1 -- 172.21.15.29:0/3032695305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee0009ae0 con 0x7f4ee80663b0 2024-01-24T15:13:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.205+0000 7f4ee77fe700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80663b0 0x7f4ee8167b30 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f4edc00b490 tx=0x7f4edc006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.206+0000 7f4ecffff700 1 -- 172.21.15.29:0/3032695305 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc012410 con 0x7f4ee80663b0 2024-01-24T15:13:26.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.206+0000 7f4ecffff700 1 -- 172.21.15.29:0/3032695305 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4edc012590 con 0x7f4ee80663b0 2024-01-24T15:13:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.206+0000 7f4eee925700 1 -- 172.21.15.29:0/3032695305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee80a54d0 con 0x7f4ee80663b0 2024-01-24T15:13:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.206+0000 7f4ecffff700 1 -- 172.21.15.29:0/3032695305 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc012950 con 0x7f4ee80663b0 2024-01-24T15:13:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.206+0000 7f4eee925700 1 -- 172.21.15.29:0/3032695305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee81688e0 con 0x7f4ee80663b0 2024-01-24T15:13:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.207+0000 7f4eee925700 1 -- 172.21.15.29:0/3032695305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee8066790 con 0x7f4ee80663b0 2024-01-24T15:13:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.207+0000 7f4ecffff700 1 -- 172.21.15.29:0/3032695305 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 66) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f4edc01b3e0 con 0x7f4ee80663b0 2024-01-24T15:13:26.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.207+0000 7f4ecffff700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ed405f030 0x7f4ed40614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:26.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.208+0000 7f4ecffff700 1 -- 172.21.15.29:0/3032695305 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 7760+0+0 (secure 0 0 0) 0x7f4edc07a520 con 0x7f4ee80663b0 2024-01-24T15:13:26.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.208+0000 7f4ee7fff700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ed405f030 0x7f4ed40614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:26.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.208+0000 7f4ee7fff700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ed405f030 0x7f4ed40614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f4ed800bbc0 tx=0x7f4ed8008040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:26.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.212+0000 7f4ecffff700 1 -- 172.21.15.29:0/3032695305 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4edc04b300 con 0x7f4ee80663b0 2024-01-24T15:13:26.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:26.438+0000 7f4eee925700 1 -- 172.21.15.29:0/3032695305 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4ee80475e0 con 0x7f4ee80663b0 2024-01-24T15:13:27.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.033+0000 7f4ecffff700 1 -- 172.21.15.29:0/3032695305 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4edc04e920 con 0x7f4ee80663b0 2024-01-24T15:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.035+0000 7f4eee925700 1 -- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ed405f030 msgr2=0x7f4ed40614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.035+0000 7f4eee925700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ed405f030 0x7f4ed40614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f4ed800bbc0 tx=0x7f4ed8008040).stop 2024-01-24T15:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.036+0000 7f4eee925700 1 -- 172.21.15.29:0/3032695305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80663b0 msgr2=0x7f4ee8167b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.036+0000 7f4eee925700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80663b0 0x7f4ee8167b30 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f4edc00b490 tx=0x7f4edc006ef0).stop 2024-01-24T15:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.036+0000 7f4eee925700 1 -- 172.21.15.29:0/3032695305 shutdown_connections 2024-01-24T15:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.036+0000 7f4eee925700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee80be4b0 0x7f4ee8168020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.036+0000 7f4eee925700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee8065420 0x7f4ee813b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:27.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.036+0000 7f4eee925700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f4ed405f030 0x7f4ed40614f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:27.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.036+0000 7f4eee925700 1 --2- 172.21.15.29:0/3032695305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ee80663b0 0x7f4ee8167b30 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:27.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.036+0000 7f4eee925700 1 -- 172.21.15.29:0/3032695305 >> 172.21.15.29:0/3032695305 conn(0x7f4ee8064390 msgr2=0x7f4ee80c7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:27.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.036+0000 7f4eee925700 1 -- 172.21.15.29:0/3032695305 shutdown_connections 2024-01-24T15:13:27.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.036+0000 7f4eee925700 1 -- 172.21.15.29:0/3032695305 wait complete. 2024-01-24T15:13:27.054 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-24T15:13:27.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.190+0000 7fd4caea8700 1 Processor -- start 2024-01-24T15:13:27.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.190+0000 7fd4caea8700 1 -- start start 2024-01-24T15:13:27.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.191+0000 7fd4caea8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c40bf870 0x7fd4c4068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:27.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.191+0000 7fd4caea8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4c4066450 0x7fd4c4066900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:27.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.191+0000 7fd4caea8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c40643f0 con 0x7fd4c413a6f0 2024-01-24T15:13:27.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.191+0000 7fd4caea8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c4066df0 con 0x7fd4c4066450 2024-01-24T15:13:27.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.191+0000 7fd4caea8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c40bdb30 con 0x7fd4c40bf870 2024-01-24T15:13:27.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.191+0000 7fd4c9445700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4c4066450 0x7fd4c4066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:27.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.191+0000 7fd4c9445700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4c4066450 0x7fd4c4066900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35678/0 (socket says 172.21.15.29:35678) 2024-01-24T15:13:27.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.192+0000 7fd4c9445700 1 -- 172.21.15.29:0/2937846156 learned_addr learned my addr 172.21.15.29:0/2937846156 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:27.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.192+0000 7fd4c3fff700 1 --2- 172.21.15.29:0/2937846156 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c40bf870 0x7fd4c4068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:27.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.192+0000 7fd4c9445700 1 -- 172.21.15.29:0/2937846156 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c40bf870 msgr2=0x7fd4c4068660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:27.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.192+0000 7fd4c9445700 1 --2- 172.21.15.29:0/2937846156 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c40bf870 0x7fd4c4068660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:27.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.192+0000 7fd4c9445700 1 -- 172.21.15.29:0/2937846156 >> v1:172.21.15.163:6789/0 conn(0x7fd4c413a6f0 legacy=0x7fd4c40638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:27.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.192+0000 7fd4c9445700 1 -- 172.21.15.29:0/2937846156 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c40be3d0 con 0x7fd4c4066450 2024-01-24T15:13:27.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.192+0000 7fd4c9445700 1 --2- 172.21.15.29:0/2937846156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4c4066450 0x7fd4c4066900 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7fd4bc009f20 tx=0x7fd4bc009840).ready entity=mon.0 client_cookie=978252be7c05cd85 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:27.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.192+0000 7fd4c37fe700 1 -- 172.21.15.29:0/2937846156 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc00e070 con 0x7fd4c4066450 2024-01-24T15:13:27.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.192+0000 7fd4c37fe700 1 -- 172.21.15.29:0/2937846156 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc00b9d0 con 0x7fd4c4066450 2024-01-24T15:13:27.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.192+0000 7fd4c37fe700 1 -- 172.21.15.29:0/2937846156 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc00bd50 con 0x7fd4c4066450 2024-01-24T15:13:27.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.193+0000 7fd4caea8700 1 -- 172.21.15.29:0/2937846156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4c4066450 msgr2=0x7fd4c4066900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:27.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.193+0000 7fd4caea8700 1 --2- 172.21.15.29:0/2937846156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4c4066450 0x7fd4c4066900 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7fd4bc009f20 tx=0x7fd4bc009840).stop 2024-01-24T15:13:27.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.193+0000 7fd4caea8700 1 -- 172.21.15.29:0/2937846156 shutdown_connections 2024-01-24T15:13:27.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.193+0000 7fd4caea8700 1 --2- 172.21.15.29:0/2937846156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4c4066450 0x7fd4c4066900 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:27.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.193+0000 7fd4caea8700 1 --2- 172.21.15.29:0/2937846156 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c40bf870 0x7fd4c4068660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:27.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.193+0000 7fd4caea8700 1 -- 172.21.15.29:0/2937846156 >> 172.21.15.29:0/2937846156 conn(0x7fd4c40c8bd0 msgr2=0x7fd4c40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:27.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.193+0000 7fd4caea8700 1 -- 172.21.15.29:0/2937846156 shutdown_connections 2024-01-24T15:13:27.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.193+0000 7fd4caea8700 1 -- 172.21.15.29:0/2937846156 wait complete. 2024-01-24T15:13:27.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.194+0000 7fd4caea8700 1 Processor -- start 2024-01-24T15:13:27.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.194+0000 7fd4caea8700 1 -- start start 2024-01-24T15:13:27.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.195+0000 7fd4caea8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c4066450 0x7fd4c4139b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:27.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.195+0000 7fd4caea8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c40bf870 0x7fd4c413d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:27.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.195+0000 7fd4caea8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4c413a6f0 0x7fd4c413d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:27.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.195+0000 7fd4caea8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c40ade50 con 0x7fd4c413a6f0 2024-01-24T15:13:27.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.195+0000 7fd4caea8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c413dbd0 con 0x7fd4c40bf870 2024-01-24T15:13:27.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.195+0000 7fd4caea8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c40ad580 con 0x7fd4c4066450 2024-01-24T15:13:27.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.195+0000 7fd4c9445700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4c413a6f0 0x7fd4c413d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:27.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.195+0000 7fd4c9445700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4c413a6f0 0x7fd4c413d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35692/0 (socket says 172.21.15.29:35692) 2024-01-24T15:13:27.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.195+0000 7fd4c9445700 1 -- 172.21.15.29:0/1599252704 learned_addr learned my addr 172.21.15.29:0/1599252704 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.195+0000 7fd4c3fff700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c40bf870 0x7fd4c413d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.195+0000 7fd4c8c44700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c4066450 0x7fd4c4139b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.196+0000 7fd4c9445700 1 -- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c4066450 msgr2=0x7fd4c4139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.196+0000 7fd4c9445700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c4066450 0x7fd4c4139b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.196+0000 7fd4c9445700 1 -- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c40bf870 msgr2=0x7fd4c413d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.196+0000 7fd4c9445700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c40bf870 0x7fd4c413d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.196+0000 7fd4c9445700 1 -- 172.21.15.29:0/1599252704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc009ae0 con 0x7fd4c413a6f0 2024-01-24T15:13:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.196+0000 7fd4c9445700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4c413a6f0 0x7fd4c413d6e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7fd4bc00be70 tx=0x7fd4bc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.196+0000 7fd4c0ff9700 1 -- 172.21.15.29:0/1599252704 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0092e0 con 0x7fd4c413a6f0 2024-01-24T15:13:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.196+0000 7fd4c0ff9700 1 -- 172.21.15.29:0/1599252704 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc01e470 con 0x7fd4c413a6f0 2024-01-24T15:13:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.196+0000 7fd4caea8700 1 -- 172.21.15.29:0/1599252704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c40aeb00 con 0x7fd4c413a6f0 2024-01-24T15:13:27.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.196+0000 7fd4c0ff9700 1 -- 172.21.15.29:0/1599252704 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc01e830 con 0x7fd4c413a6f0 2024-01-24T15:13:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.196+0000 7fd4caea8700 1 -- 172.21.15.29:0/1599252704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c40ada70 con 0x7fd4c413a6f0 2024-01-24T15:13:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.197+0000 7fd4c0ff9700 1 -- 172.21.15.29:0/1599252704 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 66) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fd4bc003040 con 0x7fd4c413a6f0 2024-01-24T15:13:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.198+0000 7fd4caea8700 1 -- 172.21.15.29:0/1599252704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c406ae00 con 0x7fd4c413a6f0 2024-01-24T15:13:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.198+0000 7fd4c0ff9700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4a805f030 0x7fd4a80614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.198+0000 7fd4c0ff9700 1 -- 172.21.15.29:0/1599252704 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 7760+0+0 (secure 0 0 0) 0x7fd4bc07b980 con 0x7fd4c413a6f0 2024-01-24T15:13:27.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.198+0000 7fd4c8c44700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4a805f030 0x7fd4a80614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:27.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.199+0000 7fd4c8c44700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4a805f030 0x7fd4a80614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fd4b8003860 tx=0x7fd4b8007d40).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:27.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.203+0000 7fd4c0ff9700 1 -- 172.21.15.29:0/1599252704 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd4bc04c870 con 0x7fd4c413a6f0 2024-01-24T15:13:27.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.387+0000 7fd4caea8700 1 -- 172.21.15.29:0/1599252704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd4c40475e0 con 0x7fd4c413a6f0 2024-01-24T15:13:27.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:27.650+0000 7fd4c0ff9700 1 -- 172.21.15.29:0/1599252704 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd4bc0792c0 con 0x7fd4c413a6f0 2024-01-24T15:13:28.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.037+0000 7fd4c0ff9700 1 -- 172.21.15.29:0/1599252704 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v168) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd4bc042910 con 0x7fd4c413a6f0 2024-01-24T15:13:28.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.041+0000 7fd4caea8700 1 -- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4a805f030 msgr2=0x7fd4a80614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:28.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.041+0000 7fd4caea8700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4a805f030 0x7fd4a80614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fd4b8003860 tx=0x7fd4b8007d40).stop 2024-01-24T15:13:28.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.041+0000 7fd4caea8700 1 -- 172.21.15.29:0/1599252704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4c413a6f0 msgr2=0x7fd4c413d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:28.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.041+0000 7fd4caea8700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4c413a6f0 0x7fd4c413d6e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7fd4bc00be70 tx=0x7fd4bc009840).stop 2024-01-24T15:13:28.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.042+0000 7fd4caea8700 1 -- 172.21.15.29:0/1599252704 shutdown_connections 2024-01-24T15:13:28.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.042+0000 7fd4caea8700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd4a805f030 0x7fd4a80614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:28.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.042+0000 7fd4caea8700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4c413a6f0 0x7fd4c413d6e0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:28.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.042+0000 7fd4caea8700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c4066450 0x7fd4c4139b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:28.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.042+0000 7fd4caea8700 1 --2- 172.21.15.29:0/1599252704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c40bf870 0x7fd4c413d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:28.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.042+0000 7fd4caea8700 1 -- 172.21.15.29:0/1599252704 >> 172.21.15.29:0/1599252704 conn(0x7fd4c40c8bd0 msgr2=0x7fd4c40c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:28.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.042+0000 7fd4caea8700 1 -- 172.21.15.29:0/1599252704 shutdown_connections 2024-01-24T15:13:28.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.042+0000 7fd4caea8700 1 -- 172.21.15.29:0/1599252704 wait complete. 2024-01-24T15:13:28.045 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-01-24T15:13:28.063 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-24T15:13:28.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.203+0000 7f86628a5700 1 Processor -- start 2024-01-24T15:13:28.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.203+0000 7f86628a5700 1 -- start start 2024-01-24T15:13:28.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.204+0000 7f86628a5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f865c151eb0 0x7f865c067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:28.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.204+0000 7f86628a5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f865c069260 0x7f865c0696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:28.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.204+0000 7f86628a5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c0bd2b0 con 0x7f865c0c3d80 2024-01-24T15:13:28.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.204+0000 7f86628a5700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865c0bca50 con 0x7f865c151eb0 2024-01-24T15:13:28.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.204+0000 7f86628a5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c069be0 con 0x7f865c069260 2024-01-24T15:13:28.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.204+0000 7f8660e42700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f865c069260 0x7f865c0696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:28.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.205+0000 7f8660e42700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f865c069260 0x7f865c0696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:56482/0 (socket says 172.21.15.29:56482) 2024-01-24T15:13:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.205+0000 7f865b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f865c151eb0 0x7f865c067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.205+0000 7f8660e42700 1 -- 172.21.15.29:0/2133576665 learned_addr learned my addr 172.21.15.29:0/2133576665 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.205+0000 7f865bfff700 1 --1- 172.21.15.29:0/2133576665 >> v1:172.21.15.163:6789/0 conn(0x7f865c0c3d80 0x7f865c0c20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:40782/0 (socket says 172.21.15.29:40782) 2024-01-24T15:13:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.205+0000 7f8660e42700 1 -- 172.21.15.29:0/2133576665 >> v2:172.21.15.163:3300/0 conn(0x7f865c151eb0 msgr2=0x7f865c067140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.205+0000 7f8660e42700 1 --2- 172.21.15.29:0/2133576665 >> v2:172.21.15.163:3300/0 conn(0x7f865c151eb0 0x7f865c067140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.205+0000 7f8660e42700 1 -- 172.21.15.29:0/2133576665 >> v1:172.21.15.163:6789/0 conn(0x7f865c0c3d80 legacy=0x7f865c0c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.205+0000 7f8660e42700 1 -- 172.21.15.29:0/2133576665 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c152330 con 0x7f865c069260 2024-01-24T15:13:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.205+0000 7f8660e42700 1 --2- 172.21.15.29:0/2133576665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f865c069260 0x7f865c0696f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f8654009f20 tx=0x7f8654009840).ready entity=mon.2 client_cookie=81cf2fbd0b53d454 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.205+0000 7f865affd700 1 -- 172.21.15.29:0/2133576665 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865400e070 con 0x7f865c069260 2024-01-24T15:13:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.206+0000 7f865affd700 1 -- 172.21.15.29:0/2133576665 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865400b9d0 con 0x7f865c069260 2024-01-24T15:13:28.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.206+0000 7f865affd700 1 -- 172.21.15.29:0/2133576665 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865400bd50 con 0x7f865c069260 2024-01-24T15:13:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.206+0000 7f86628a5700 1 -- 172.21.15.29:0/2133576665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f865c069260 msgr2=0x7f865c0696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.206+0000 7f86628a5700 1 --2- 172.21.15.29:0/2133576665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f865c069260 0x7f865c0696f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f8654009f20 tx=0x7f8654009840).stop 2024-01-24T15:13:28.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.206+0000 7f86628a5700 1 -- 172.21.15.29:0/2133576665 shutdown_connections 2024-01-24T15:13:28.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.206+0000 7f86628a5700 1 --2- 172.21.15.29:0/2133576665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f865c069260 0x7f865c0696f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:28.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.206+0000 7f86628a5700 1 --2- 172.21.15.29:0/2133576665 >> v2:172.21.15.163:3300/0 conn(0x7f865c151eb0 0x7f865c067140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:28.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.206+0000 7f86628a5700 1 -- 172.21.15.29:0/2133576665 >> 172.21.15.29:0/2133576665 conn(0x7f865c0c7af0 msgr2=0x7f865c0c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:28.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.206+0000 7f86628a5700 1 -- 172.21.15.29:0/2133576665 shutdown_connections 2024-01-24T15:13:28.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.206+0000 7f86628a5700 1 -- 172.21.15.29:0/2133576665 wait complete. 2024-01-24T15:13:28.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.208+0000 7f86628a5700 1 Processor -- start 2024-01-24T15:13:28.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.208+0000 7f86628a5700 1 -- start start 2024-01-24T15:13:28.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.209+0000 7f86628a5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f865c069260 0x7f865c0ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:28.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.209+0000 7f86628a5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f865c0c3d80 0x7f865c0ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:28.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.209+0000 7f86628a5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f865c151eb0 0x7f865c0abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:28.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.209+0000 7f86628a5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1574d0 con 0x7f865c0c3d80 2024-01-24T15:13:28.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.209+0000 7f86628a5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c156950 con 0x7f865c151eb0 2024-01-24T15:13:28.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.209+0000 7f86628a5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c156c30 con 0x7f865c069260 2024-01-24T15:13:28.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.209+0000 7f865b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f865c0c3d80 0x7f865c0ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:28.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.209+0000 7f865b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f865c0c3d80 0x7f865c0ab5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35706/0 (socket says 172.21.15.29:35706) 2024-01-24T15:13:28.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.209+0000 7f865b7fe700 1 -- 172.21.15.29:0/1780427704 learned_addr learned my addr 172.21.15.29:0/1780427704 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:28.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.209+0000 7f8660e42700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f865c151eb0 0x7f865c0abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:28.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.209+0000 7f865bfff700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f865c069260 0x7f865c0ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:28.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.210+0000 7f865b7fe700 1 -- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f865c069260 msgr2=0x7f865c0ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:28.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.210+0000 7f865b7fe700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f865c069260 0x7f865c0ab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:28.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.210+0000 7f865b7fe700 1 -- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f865c151eb0 msgr2=0x7f865c0abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:28.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.210+0000 7f865b7fe700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f865c151eb0 0x7f865c0abac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:28.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.210+0000 7f865b7fe700 1 -- 172.21.15.29:0/1780427704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8654009ae0 con 0x7f865c0c3d80 2024-01-24T15:13:28.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.210+0000 7f865b7fe700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f865c0c3d80 0x7f865c0ab5d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f864400e930 tx=0x7f8644009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:28.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.210+0000 7f8653fff700 1 -- 172.21.15.29:0/1780427704 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644012600 con 0x7f865c0c3d80 2024-01-24T15:13:28.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.210+0000 7f8653fff700 1 -- 172.21.15.29:0/1780427704 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8644012760 con 0x7f865c0c3d80 2024-01-24T15:13:28.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.210+0000 7f86628a5700 1 -- 172.21.15.29:0/1780427704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c156f10 con 0x7f865c0c3d80 2024-01-24T15:13:28.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.210+0000 7f8653fff700 1 -- 172.21.15.29:0/1780427704 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644012ae0 con 0x7f865c0c3d80 2024-01-24T15:13:28.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.210+0000 7f86628a5700 1 -- 172.21.15.29:0/1780427704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c136010 con 0x7f865c0c3d80 2024-01-24T15:13:28.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.211+0000 7f86628a5700 1 -- 172.21.15.29:0/1780427704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c136290 con 0x7f865c0c3d80 2024-01-24T15:13:28.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.212+0000 7f8653fff700 1 -- 172.21.15.29:0/1780427704 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f864401b400 con 0x7f865c0c3d80 2024-01-24T15:13:28.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.212+0000 7f8653fff700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f863c05f270 0x7f863c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:28.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.212+0000 7f8653fff700 1 -- 172.21.15.29:0/1780427704 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 7322+0+0 (secure 0 0 0) 0x7f864407acd0 con 0x7f865c0c3d80 2024-01-24T15:13:28.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.212+0000 7f865bfff700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f863c05f270 0x7f863c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:28.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.213+0000 7f865bfff700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f863c05f270 0x7f863c061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f864c007fd0 tx=0x7f864c007d30).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:28.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.216+0000 7f8653fff700 1 -- 172.21.15.29:0/1780427704 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f864404b910 con 0x7f865c0c3d80 2024-01-24T15:13:28.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:28.408+0000 7f86628a5700 1 -- 172.21.15.29:0/1780427704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f865c0aadb0 con 0x7f865c0c3d80 2024-01-24T15:13:29.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.040+0000 7f8653fff700 1 -- 172.21.15.29:0/1780427704 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v169) v1 ==== 169+0+0 (secure 0 0 0) 0x7f864404ef30 con 0x7f865c0c3d80 2024-01-24T15:13:29.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.045+0000 7f86628a5700 1 -- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f863c05f270 msgr2=0x7f863c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.045+0000 7f86628a5700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f863c05f270 0x7f863c061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f864c007fd0 tx=0x7f864c007d30).stop 2024-01-24T15:13:29.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.045+0000 7f86628a5700 1 -- 172.21.15.29:0/1780427704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f865c0c3d80 msgr2=0x7f865c0ab5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.045+0000 7f86628a5700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f865c0c3d80 0x7f865c0ab5d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f864400e930 tx=0x7f8644009060).stop 2024-01-24T15:13:29.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.046+0000 7f86628a5700 1 -- 172.21.15.29:0/1780427704 shutdown_connections 2024-01-24T15:13:29.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.046+0000 7f86628a5700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f865c151eb0 0x7f865c0abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.046+0000 7f86628a5700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f865c069260 0x7f865c0ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.046+0000 7f86628a5700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f863c05f270 0x7f863c061730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.046+0000 7f86628a5700 1 --2- 172.21.15.29:0/1780427704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f865c0c3d80 0x7f865c0ab5d0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.046+0000 7f86628a5700 1 -- 172.21.15.29:0/1780427704 >> 172.21.15.29:0/1780427704 conn(0x7f865c0c7af0 msgr2=0x7f865c14efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:29.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.046+0000 7f86628a5700 1 -- 172.21.15.29:0/1780427704 shutdown_connections 2024-01-24T15:13:29.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.046+0000 7f86628a5700 1 -- 172.21.15.29:0/1780427704 wait complete. 2024-01-24T15:13:29.049 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-01-24T15:13:29.065 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-24T15:13:29.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.202+0000 7f18440be700 1 Processor -- start 2024-01-24T15:13:29.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.202+0000 7f18440be700 1 -- start start 2024-01-24T15:13:29.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.203+0000 7f18440be700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f183c134e70 0x7f183c0c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.203+0000 7f18440be700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183c068470 con 0x7f183c066c30 2024-01-24T15:13:29.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.203+0000 7f18440be700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183c0c3860 con 0x7f183c068900 2024-01-24T15:13:29.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.203+0000 7f18440be700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183c14c910 con 0x7f183c134e70 2024-01-24T15:13:29.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.203+0000 7f184265b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f183c066c30 0x7f183c0bdb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52344/0 (socket says 172.21.15.29:52344) 2024-01-24T15:13:29.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.203+0000 7f184265b700 1 -- 172.21.15.29:0/951427117 learned_addr learned my addr 172.21.15.29:0/951427117 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:29.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.203+0000 7f1841e5a700 1 --2- 172.21.15.29:0/951427117 >> v2:172.21.15.163:3300/0 conn(0x7f183c134e70 0x7f183c0c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.204+0000 7f1840e58700 1 -- 172.21.15.29:0/951427117 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84423958 0 0) 0x7f183c068470 con 0x7f183c066c30 2024-01-24T15:13:29.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.204+0000 7f1840e58700 1 -- 172.21.15.29:0/951427117 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18280033f0 con 0x7f183c066c30 2024-01-24T15:13:29.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.204+0000 7f1841e5a700 1 -- 172.21.15.29:0/951427117 >> v1:172.21.15.163:6789/0 conn(0x7f183c068900 legacy=0x7f183c066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.204+0000 7f1840e58700 1 -- 172.21.15.29:0/951427117 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2311324003 0 0) 0x7f18280033f0 con 0x7f183c066c30 2024-01-24T15:13:29.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.204+0000 7f1841e5a700 1 -- 172.21.15.29:0/951427117 >> v1:172.21.15.29:6789/0 conn(0x7f183c066c30 legacy=0x7f183c0bdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.204+0000 7f1841e5a700 1 -- 172.21.15.29:0/951427117 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c067b40 con 0x7f183c134e70 2024-01-24T15:13:29.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.204+0000 7f1841e5a700 1 --2- 172.21.15.29:0/951427117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c134e70 0x7f183c0c31a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f182c009760 tx=0x7f182c009060).ready entity=mon.1 client_cookie=c9162dd3f9e01a09 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:29.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.204+0000 7f1840e58700 1 -- 172.21.15.29:0/951427117 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18340017f0 con 0x7f183c134e70 2024-01-24T15:13:29.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.204+0000 7f1840e58700 1 -- 172.21.15.29:0/951427117 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c012410 con 0x7f183c134e70 2024-01-24T15:13:29.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.204+0000 7f1840e58700 1 -- 172.21.15.29:0/951427117 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c012790 con 0x7f183c134e70 2024-01-24T15:13:29.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.205+0000 7f18440be700 1 -- 172.21.15.29:0/951427117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c134e70 msgr2=0x7f183c0c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.205+0000 7f18440be700 1 --2- 172.21.15.29:0/951427117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c134e70 0x7f183c0c31a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f182c009760 tx=0x7f182c009060).stop 2024-01-24T15:13:29.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.205+0000 7f18440be700 1 -- 172.21.15.29:0/951427117 shutdown_connections 2024-01-24T15:13:29.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.205+0000 7f18440be700 1 --2- 172.21.15.29:0/951427117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c134e70 0x7f183c0c31a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.205+0000 7f18440be700 1 -- 172.21.15.29:0/951427117 >> 172.21.15.29:0/951427117 conn(0x7f183c0c8ba0 msgr2=0x7f183c0c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:29.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.205+0000 7f18440be700 1 -- 172.21.15.29:0/951427117 shutdown_connections 2024-01-24T15:13:29.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.205+0000 7f18440be700 1 -- 172.21.15.29:0/951427117 wait complete. 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.206+0000 7f18440be700 1 Processor -- start 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.206+0000 7f18440be700 1 -- start start 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f18440be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c066c30 0x7f183c165290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f18440be700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f183c068900 0x7f183c152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f18440be700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f183c134e70 0x7f183c0abec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f18440be700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c152930 con 0x7f183c134e70 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f18440be700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0ac3b0 con 0x7f183c066c30 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f18440be700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183c1533e0 con 0x7f183c068900 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f184265b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f183c134e70 0x7f183c0abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f184265b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f183c134e70 0x7f183c0abec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35720/0 (socket says 172.21.15.29:35720) 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f184265b700 1 -- 172.21.15.29:0/1570164317 learned_addr learned my addr 172.21.15.29:0/1570164317 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f1841659700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f183c068900 0x7f183c152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f1841e5a700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c066c30 0x7f183c165290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f184265b700 1 -- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f183c068900 msgr2=0x7f183c152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f184265b700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f183c068900 0x7f183c152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f184265b700 1 -- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c066c30 msgr2=0x7f183c165290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f184265b700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c066c30 0x7f183c165290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f184265b700 1 -- 172.21.15.29:0/1570164317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0092a0 con 0x7f183c134e70 2024-01-24T15:13:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.207+0000 7f1841e5a700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c066c30 0x7f183c165290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.208+0000 7f184265b700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f183c134e70 0x7f183c0abec0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f183400cbc0 tx=0x7f18340018e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.208+0000 7f1831ffb700 1 -- 172.21.15.29:0/1570164317 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834004c40 con 0x7f183c134e70 2024-01-24T15:13:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.208+0000 7f1831ffb700 1 -- 172.21.15.29:0/1570164317 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1834004da0 con 0x7f183c134e70 2024-01-24T15:13:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.208+0000 7f1831ffb700 1 -- 172.21.15.29:0/1570164317 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18340073a0 con 0x7f183c134e70 2024-01-24T15:13:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.208+0000 7f18440be700 1 -- 172.21.15.29:0/1570164317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183c154210 con 0x7f183c134e70 2024-01-24T15:13:29.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.208+0000 7f18440be700 1 -- 172.21.15.29:0/1570164317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183c152e20 con 0x7f183c134e70 2024-01-24T15:13:29.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.209+0000 7f1831ffb700 1 -- 172.21.15.29:0/1570164317 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1834007750 con 0x7f183c134e70 2024-01-24T15:13:29.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.209+0000 7f18440be700 1 -- 172.21.15.29:0/1570164317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183c0475e0 con 0x7f183c134e70 2024-01-24T15:13:29.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.210+0000 7f1831ffb700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f182005f270 0x7f1820061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.210+0000 7f1831ffb700 1 -- 172.21.15.29:0/1570164317 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6850+0+0 (secure 0 0 0) 0x7f183407ac10 con 0x7f183c134e70 2024-01-24T15:13:29.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.210+0000 7f1841e5a700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f182005f270 0x7f1820061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.210+0000 7f1841e5a700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f182005f270 0x7f1820061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f182c006d30 tx=0x7f182c015000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:29.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.214+0000 7f1831ffb700 1 -- 172.21.15.29:0/1570164317 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f183404b480 con 0x7f183c134e70 2024-01-24T15:13:29.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.397+0000 7f18440be700 1 -- 172.21.15.29:0/1570164317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f183c068df0 con 0x7f183c134e70 2024-01-24T15:13:29.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.400+0000 7f1831ffb700 1 -- 172.21.15.29:0/1570164317 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f183404eaa0 con 0x7f183c134e70 2024-01-24T15:13:29.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.403+0000 7f18440be700 1 -- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f182005f270 msgr2=0x7f1820061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.403+0000 7f18440be700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f182005f270 0x7f1820061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f182c006d30 tx=0x7f182c015000).stop 2024-01-24T15:13:29.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.403+0000 7f18440be700 1 -- 172.21.15.29:0/1570164317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f183c134e70 msgr2=0x7f183c0abec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.403+0000 7f18440be700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f183c134e70 0x7f183c0abec0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f183400cbc0 tx=0x7f18340018e0).stop 2024-01-24T15:13:29.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.403+0000 7f18440be700 1 -- 172.21.15.29:0/1570164317 shutdown_connections 2024-01-24T15:13:29.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.403+0000 7f18440be700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f182005f270 0x7f1820061730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.403+0000 7f18440be700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f183c134e70 0x7f183c0abec0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.403+0000 7f18440be700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c066c30 0x7f183c165290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.403+0000 7f18440be700 1 --2- 172.21.15.29:0/1570164317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f183c068900 0x7f183c152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.403+0000 7f18440be700 1 -- 172.21.15.29:0/1570164317 >> 172.21.15.29:0/1570164317 conn(0x7f183c0c8ba0 msgr2=0x7f183c06ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:29.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.403+0000 7f18440be700 1 -- 172.21.15.29:0/1570164317 shutdown_connections 2024-01-24T15:13:29.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.403+0000 7f18440be700 1 -- 172.21.15.29:0/1570164317 wait complete. 2024-01-24T15:13:29.418 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-01-24T15:13:29.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.548+0000 7f314d571700 1 Processor -- start 2024-01-24T15:13:29.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.548+0000 7f314d571700 1 -- start start 2024-01-24T15:13:29.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.549+0000 7f314d571700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480c5af0 0x7f31480c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.549+0000 7f314d571700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31480c6820 con 0x7f31480bfd20 2024-01-24T15:13:29.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.549+0000 7f314d571700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3148155220 con 0x7f3148064740 2024-01-24T15:13:29.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.549+0000 7f314d571700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31481556b0 con 0x7f31480c5af0 2024-01-24T15:13:29.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.549+0000 7f31477fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f31480bfd20 0x7f31480c01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52350/0 (socket says 172.21.15.29:52350) 2024-01-24T15:13:29.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.549+0000 7f31477fe700 1 -- 172.21.15.29:0/2964719514 learned_addr learned my addr 172.21.15.29:0/2964719514 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:29.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.549+0000 7f31467fc700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480c5af0 0x7f31480c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188130501 0 0) 0x7f3140001490 con 0x7f31480bfd20 2024-01-24T15:13:29.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31300033f0 con 0x7f31480bfd20 2024-01-24T15:13:29.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209427063 0 0) 0x7f3148155220 con 0x7f3148064740 2024-01-24T15:13:29.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3140001490 con 0x7f3148064740 2024-01-24T15:13:29.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1495085395 0 0) 0x7f31300033f0 con 0x7f31480bfd20 2024-01-24T15:13:29.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480c5af0 msgr2=0x7f31480c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:29.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480c5af0 0x7f31480c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 >> v1:172.21.15.163:6789/0 conn(0x7f3148064740 legacy=0x7f3148064b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3148154680 con 0x7f31480bfd20 2024-01-24T15:13:29.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f3140001980 con 0x7f31480bfd20 2024-01-24T15:13:29.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f31467fc700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480c5af0 0x7f31480c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:29.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 >> v1:172.21.15.29:6789/0 conn(0x7f31480bfd20 legacy=0x7f31480c01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480c5af0 0x7f3130003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3130003f90 0x7f3130004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3130004b80 0x7f3130005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31481556b0 con 0x7f3130004b80 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313000b940 con 0x7f3130003f90 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313000bb00 con 0x7f31480c5af0 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f31477fe700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3130004b80 0x7f3130005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f31467fc700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3130003f90 0x7f3130004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f3146ffd700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480c5af0 0x7f3130003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f31477fe700 1 -- 172.21.15.29:0/2964719514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480c5af0 msgr2=0x7f3130003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f31477fe700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480c5af0 0x7f3130003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f31477fe700 1 -- 172.21.15.29:0/2964719514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3130003f90 msgr2=0x7f3130004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.550+0000 7f31477fe700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3130003f90 0x7f3130004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.551+0000 7f31477fe700 1 -- 172.21.15.29:0/2964719514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313c0092a0 con 0x7f3130004b80 2024-01-24T15:13:29.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.551+0000 7f31477fe700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3130004b80 0x7f3130005000 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f314000c630 tx=0x7f3140007c00).ready entity=mon.0 client_cookie=70a364c9abcffa87 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:29.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.551+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140005b20 con 0x7f3130004b80 2024-01-24T15:13:29.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.551+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3140005c80 con 0x7f3130004b80 2024-01-24T15:13:29.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.551+0000 7f3145ffb700 1 -- 172.21.15.29:0/2964719514 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31400070d0 con 0x7f3130004b80 2024-01-24T15:13:29.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.551+0000 7f314d571700 1 -- 172.21.15.29:0/2964719514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3130004b80 msgr2=0x7f3130005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.551+0000 7f314d571700 1 --2- 172.21.15.29:0/2964719514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3130004b80 0x7f3130005000 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f314000c630 tx=0x7f3140007c00).stop 2024-01-24T15:13:29.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.551+0000 7f31477fe700 1 -- 172.21.15.29:0/2964719514 reap_dead start 2024-01-24T15:13:29.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.551+0000 7f314d571700 1 -- 172.21.15.29:0/2964719514 shutdown_connections 2024-01-24T15:13:29.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.551+0000 7f314d571700 1 -- 172.21.15.29:0/2964719514 >> 172.21.15.29:0/2964719514 conn(0x7f31480cb890 msgr2=0x7f31480c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:29.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.551+0000 7f314d571700 1 -- 172.21.15.29:0/2964719514 shutdown_connections 2024-01-24T15:13:29.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.551+0000 7f314d571700 1 -- 172.21.15.29:0/2964719514 wait complete. 2024-01-24T15:13:29.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.553+0000 7f314d571700 1 Processor -- start 2024-01-24T15:13:29.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.553+0000 7f314d571700 1 -- start start 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f314d571700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3148150d50 0x7f3148151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f314d571700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31480ad250 0x7f31480ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f314d571700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31480abd90 0x7f31480ac210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f314d571700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31480af970 con 0x7f31480abd90 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f314d571700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31480c37d0 con 0x7f31480ad250 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f314d571700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31480ac700 con 0x7f3148150d50 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f31477fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31480abd90 0x7f31480ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f31477fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31480abd90 0x7f31480ac210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35738/0 (socket says 172.21.15.29:35738) 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f31477fe700 1 -- 172.21.15.29:0/557900138 learned_addr learned my addr 172.21.15.29:0/557900138 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f3146ffd700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3148150d50 0x7f3148151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f31467fc700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31480ad250 0x7f31480ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f31477fe700 1 -- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3148150d50 msgr2=0x7f3148151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f31477fe700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3148150d50 0x7f3148151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f31477fe700 1 -- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31480ad250 msgr2=0x7f31480ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f31477fe700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31480ad250 0x7f31480ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f31477fe700 1 -- 172.21.15.29:0/557900138 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31480adf10 con 0x7f31480abd90 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f31477fe700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31480abd90 0x7f31480ac210 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f314000cc20 tx=0x7f3140015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.554+0000 7f312effd700 1 -- 172.21.15.29:0/557900138 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140002120 con 0x7f31480abd90 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.555+0000 7f312effd700 1 -- 172.21.15.29:0/557900138 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31400070d0 con 0x7f31480abd90 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.555+0000 7f314d571700 1 -- 172.21.15.29:0/557900138 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31480ae1f0 con 0x7f31480abd90 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.555+0000 7f312effd700 1 -- 172.21.15.29:0/557900138 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140004af0 con 0x7f31480abd90 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.555+0000 7f314d571700 1 -- 172.21.15.29:0/557900138 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31480aebc0 con 0x7f31480abd90 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.556+0000 7f314d571700 1 -- 172.21.15.29:0/557900138 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31480475e0 con 0x7f31480abd90 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.556+0000 7f312effd700 1 -- 172.21.15.29:0/557900138 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3140013450 con 0x7f31480abd90 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.556+0000 7f312effd700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f313405f270 0x7f3134061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.557+0000 7f312effd700 1 -- 172.21.15.29:0/557900138 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6850+0+0 (secure 0 0 0) 0x7f314007b370 con 0x7f31480abd90 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.557+0000 7f3146ffd700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f313405f270 0x7f3134061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.557+0000 7f3146ffd700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f313405f270 0x7f3134061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f3138000c00 tx=0x7f3138007bb0).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:29.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.561+0000 7f312effd700 1 -- 172.21.15.29:0/557900138 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f314004c170 con 0x7f31480abd90 2024-01-24T15:13:29.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.750+0000 7f314d571700 1 -- 172.21.15.29:0/557900138 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f31480c6820 con 0x7f31480abd90 2024-01-24T15:13:29.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.754+0000 7f312effd700 1 -- 172.21.15.29:0/557900138 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v29) v1 ==== 120+0+0 (secure 0 0 0) 0x7f31480c6820 con 0x7f31480abd90 2024-01-24T15:13:29.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.756+0000 7f314d571700 1 -- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f313405f270 msgr2=0x7f3134061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.756+0000 7f314d571700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f313405f270 0x7f3134061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f3138000c00 tx=0x7f3138007bb0).stop 2024-01-24T15:13:29.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.756+0000 7f314d571700 1 -- 172.21.15.29:0/557900138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31480abd90 msgr2=0x7f31480ac210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.756+0000 7f314d571700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31480abd90 0x7f31480ac210 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f314000cc20 tx=0x7f3140015000).stop 2024-01-24T15:13:29.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.756+0000 7f314d571700 1 -- 172.21.15.29:0/557900138 shutdown_connections 2024-01-24T15:13:29.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.756+0000 7f314d571700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f313405f270 0x7f3134061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.756+0000 7f314d571700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31480abd90 0x7f31480ac210 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.756+0000 7f314d571700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3148150d50 0x7f3148151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.756+0000 7f314d571700 1 --2- 172.21.15.29:0/557900138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31480ad250 0x7f31480ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.756+0000 7f314d571700 1 -- 172.21.15.29:0/557900138 >> 172.21.15.29:0/557900138 conn(0x7f31480cb890 msgr2=0x7f31480aa090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:29.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.756+0000 7f314d571700 1 -- 172.21.15.29:0/557900138 shutdown_connections 2024-01-24T15:13:29.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.756+0000 7f314d571700 1 -- 172.21.15.29:0/557900138 wait complete. 2024-01-24T15:13:29.772 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2024-01-24T15:13:29.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.900+0000 7f6245e00700 1 Processor -- start 2024-01-24T15:13:29.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.900+0000 7f6245e00700 1 -- start start 2024-01-24T15:13:29.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.901+0000 7f6245e00700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6240138710 0x7f62400be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.901+0000 7f6245e00700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f62400627b0 0x7f6240062c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.901+0000 7f6245e00700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6240065770 con 0x7f6240138330 2024-01-24T15:13:29.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.901+0000 7f6245e00700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6240063130 con 0x7f6240138710 2024-01-24T15:13:29.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.901+0000 7f6245e00700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6240069cf0 con 0x7f62400627b0 2024-01-24T15:13:29.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.901+0000 7f623effd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6240138710 0x7f62400be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.901+0000 7f623f7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6240138330 0x7f62400bca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52362/0 (socket says 172.21.15.29:52362) 2024-01-24T15:13:29.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.901+0000 7f623f7fe700 1 -- 172.21.15.29:0/3630841640 learned_addr learned my addr 172.21.15.29:0/3630841640 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:29.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.902+0000 7f623ffff700 1 --2- 172.21.15.29:0/3630841640 >> v2:172.21.15.163:3300/0 conn(0x7f62400627b0 0x7f6240062c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.902+0000 7f623effd700 1 -- 172.21.15.29:0/3630841640 >> v2:172.21.15.163:3300/0 conn(0x7f62400627b0 msgr2=0x7f6240062c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.902+0000 7f623effd700 1 --2- 172.21.15.29:0/3630841640 >> v2:172.21.15.163:3300/0 conn(0x7f62400627b0 0x7f6240062c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.902+0000 7f623effd700 1 -- 172.21.15.29:0/3630841640 >> v1:172.21.15.29:6789/0 conn(0x7f6240138330 legacy=0x7f62400bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.902+0000 7f623e7fc700 1 -- 172.21.15.29:0/3630841640 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768733252 0 0) 0x7f6240065770 con 0x7f6240138330 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.902+0000 7f623effd700 1 -- 172.21.15.29:0/3630841640 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6240069f60 con 0x7f6240138710 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.902+0000 7f623ffff700 1 --2- 172.21.15.29:0/3630841640 >> v2:172.21.15.163:3300/0 conn(0x7f62400627b0 0x7f6240062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.902+0000 7f623effd700 1 --2- 172.21.15.29:0/3630841640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6240138710 0x7f62400be3e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f62400c2060 tx=0x7f6228009060).ready entity=mon.0 client_cookie=2edea2dd79b31fe7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.902+0000 7f623e7fc700 1 -- 172.21.15.29:0/3630841640 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622800b580 con 0x7f6240138710 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.902+0000 7f623e7fc700 1 -- 172.21.15.29:0/3630841640 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622800b6e0 con 0x7f6240138710 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.902+0000 7f623e7fc700 1 -- 172.21.15.29:0/3630841640 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622800ba60 con 0x7f6240138710 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.903+0000 7f6245e00700 1 -- 172.21.15.29:0/3630841640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6240138710 msgr2=0x7f62400be3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.903+0000 7f6245e00700 1 --2- 172.21.15.29:0/3630841640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6240138710 0x7f62400be3e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f62400c2060 tx=0x7f6228009060).stop 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.903+0000 7f6245e00700 1 -- 172.21.15.29:0/3630841640 shutdown_connections 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.903+0000 7f6245e00700 1 --2- 172.21.15.29:0/3630841640 >> v2:172.21.15.163:3300/0 conn(0x7f62400627b0 0x7f6240062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.903+0000 7f6245e00700 1 --2- 172.21.15.29:0/3630841640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6240138710 0x7f62400be3e0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.903+0000 7f6245e00700 1 -- 172.21.15.29:0/3630841640 >> 172.21.15.29:0/3630841640 conn(0x7f62400c8580 msgr2=0x7f62400c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:29.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.903+0000 7f6245e00700 1 -- 172.21.15.29:0/3630841640 shutdown_connections 2024-01-24T15:13:29.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.903+0000 7f6245e00700 1 -- 172.21.15.29:0/3630841640 wait complete. 2024-01-24T15:13:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.904+0000 7f6245e00700 1 Processor -- start 2024-01-24T15:13:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.904+0000 7f6245e00700 1 -- start start 2024-01-24T15:13:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f6245e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62400627b0 0x7f62400c3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f6245e00700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6240138330 0x7f62400c1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f6245e00700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6240138710 0x7f62400ab520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f6245e00700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62400c1870 con 0x7f6240138710 2024-01-24T15:13:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f6245e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62400c25d0 con 0x7f62400627b0 2024-01-24T15:13:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f6245e00700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62400c28b0 con 0x7f6240138330 2024-01-24T15:13:29.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f623ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6240138710 0x7f62400ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f623ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6240138710 0x7f62400ab520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35758/0 (socket says 172.21.15.29:35758) 2024-01-24T15:13:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f623ffff700 1 -- 172.21.15.29:0/4122490653 learned_addr learned my addr 172.21.15.29:0/4122490653 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f623f7fe700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62400627b0 0x7f62400c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f623effd700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6240138330 0x7f62400c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f623ffff700 1 -- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6240138330 msgr2=0x7f62400c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f623ffff700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6240138330 0x7f62400c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f623ffff700 1 -- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62400627b0 msgr2=0x7f62400c3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f623ffff700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62400627b0 0x7f62400c3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:29.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f623ffff700 1 -- 172.21.15.29:0/4122490653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62280092a0 con 0x7f6240138710 2024-01-24T15:13:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.905+0000 7f623effd700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6240138330 0x7f62400c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.906+0000 7f623ffff700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6240138710 0x7f62400ab520 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f623800d180 tx=0x7f6238009a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.906+0000 7f62377fe700 1 -- 172.21.15.29:0/4122490653 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623800dd70 con 0x7f6240138710 2024-01-24T15:13:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.906+0000 7f62377fe700 1 -- 172.21.15.29:0/4122490653 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623800f040 con 0x7f6240138710 2024-01-24T15:13:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.906+0000 7f6245e00700 1 -- 172.21.15.29:0/4122490653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62400c2b90 con 0x7f6240138710 2024-01-24T15:13:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.906+0000 7f62377fe700 1 -- 172.21.15.29:0/4122490653 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238013600 con 0x7f6240138710 2024-01-24T15:13:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.906+0000 7f6245e00700 1 -- 172.21.15.29:0/4122490653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62400c3150 con 0x7f6240138710 2024-01-24T15:13:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.907+0000 7f6245e00700 1 -- 172.21.15.29:0/4122490653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6240068d80 con 0x7f6240138710 2024-01-24T15:13:29.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.907+0000 7f62377fe700 1 -- 172.21.15.29:0/4122490653 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f62380139f0 con 0x7f6240138710 2024-01-24T15:13:29.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.908+0000 7f62377fe700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f622005f270 0x7f6220061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:29.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.908+0000 7f62377fe700 1 -- 172.21.15.29:0/4122490653 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6850+0+0 (secure 0 0 0) 0x7f623807af00 con 0x7f6240138710 2024-01-24T15:13:29.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.908+0000 7f623f7fe700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f622005f270 0x7f6220061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:29.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.908+0000 7f623f7fe700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f622005f270 0x7f6220061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f62300011e0 tx=0x7f6230007bb0).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:29.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:29.912+0000 7f62377fe700 1 -- 172.21.15.29:0/4122490653 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f623804bc50 con 0x7f6240138710 2024-01-24T15:13:30.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:30.167+0000 7f6245e00700 1 -- 172.21.15.29:0/4122490653 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f62400c3340 con 0x7f6240138710 2024-01-24T15:13:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.045+0000 7f62377fe700 1 -- 172.21.15.29:0/4122490653 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2421) v1 ==== 178+0+0 (secure 0 0 0) 0x7f623804f270 con 0x7f6240138710 2024-01-24T15:13:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.047+0000 7f6245e00700 1 -- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f622005f270 msgr2=0x7f6220061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.047+0000 7f6245e00700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f622005f270 0x7f6220061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f62300011e0 tx=0x7f6230007bb0).stop 2024-01-24T15:13:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.047+0000 7f6245e00700 1 -- 172.21.15.29:0/4122490653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6240138710 msgr2=0x7f62400ab520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.047+0000 7f6245e00700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6240138710 0x7f62400ab520 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f623800d180 tx=0x7f6238009a50).stop 2024-01-24T15:13:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.048+0000 7f6245e00700 1 -- 172.21.15.29:0/4122490653 shutdown_connections 2024-01-24T15:13:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.048+0000 7f6245e00700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f622005f270 0x7f6220061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.048+0000 7f6245e00700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6240138710 0x7f62400ab520 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.048+0000 7f6245e00700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62400627b0 0x7f62400c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.048+0000 7f6245e00700 1 --2- 172.21.15.29:0/4122490653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6240138330 0x7f62400c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:31.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.048+0000 7f6245e00700 1 -- 172.21.15.29:0/4122490653 >> 172.21.15.29:0/4122490653 conn(0x7f62400c8580 msgr2=0x7f62400a7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.048+0000 7f6245e00700 1 -- 172.21.15.29:0/4122490653 shutdown_connections 2024-01-24T15:13:31.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.048+0000 7f6245e00700 1 -- 172.21.15.29:0/4122490653 wait complete. 2024-01-24T15:13:31.065 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2024-01-24T15:13:31.065 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2024-01-24T15:13:31.065 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-01-24T15:13:31.066 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2024-01-24T15:13:31.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.199+0000 7f47576c0700 1 Processor -- start 2024-01-24T15:13:31.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.199+0000 7f47576c0700 1 -- start start 2024-01-24T15:13:31.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.200+0000 7f47576c0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4750066c00 0x7f4750155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:31.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.200+0000 7f47576c0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47501563d0 0x7f47500bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:31.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.200+0000 7f47576c0700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500c3910 con 0x7f4750155b00 2024-01-24T15:13:31.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.200+0000 7f47576c0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47500c5500 con 0x7f4750066c00 2024-01-24T15:13:31.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.200+0000 7f47576c0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47500c5220 con 0x7f47501563d0 2024-01-24T15:13:31.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.200+0000 7f475545c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4750155b00 0x7f47500664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52376/0 (socket says 172.21.15.29:52376) 2024-01-24T15:13:31.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.200+0000 7f475545c700 1 -- 172.21.15.29:0/1685820030 learned_addr learned my addr 172.21.15.29:0/1685820030 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:31.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4754c5b700 1 --2- 172.21.15.29:0/1685820030 >> v2:172.21.15.163:3300/0 conn(0x7f4750066c00 0x7f4750155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:31.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4755c5d700 1 --2- 172.21.15.29:0/1685820030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47501563d0 0x7f47500bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:31.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592269898 0 0) 0x7f47500c3910 con 0x7f4750155b00 2024-01-24T15:13:31.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f473c0033f0 con 0x7f4750155b00 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1190064042 0 0) 0x7f473c0033f0 con 0x7f4750155b00 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47501563d0 msgr2=0x7f47500bdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 --2- 172.21.15.29:0/1685820030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47501563d0 0x7f47500bdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 >> v2:172.21.15.163:3300/0 conn(0x7f4750066c00 msgr2=0x7f4750155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 --2- 172.21.15.29:0/1685820030 >> v2:172.21.15.163:3300/0 conn(0x7f4750066c00 0x7f4750155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47500c3350 con 0x7f4750155b00 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f4740001470 con 0x7f4750155b00 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 >> v1:172.21.15.29:6789/0 conn(0x7f4750155b00 legacy=0x7f47500664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 --2- 172.21.15.29:0/1685820030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f473c0038e0 0x7f473c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 --2- 172.21.15.29:0/1685820030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f473c004690 0x7f473c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 --2- 172.21.15.29:0/1685820030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47501563d0 0x7f473c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47500c5220 con 0x7f473c0038e0 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0044b0 con 0x7f473c004690 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c00b910 con 0x7f47501563d0 2024-01-24T15:13:31.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4754c5b700 1 --2- 172.21.15.29:0/1685820030 >> v2:172.21.15.163:3300/0 conn(0x7f4750066c00 0x7f4750155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f475545c700 1 --2- 172.21.15.29:0/1685820030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f473c0038e0 0x7f473c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.201+0000 7f4755c5d700 1 --2- 172.21.15.29:0/1685820030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47501563d0 0x7f473c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.202+0000 7f4754c5b700 1 --2- 172.21.15.29:0/1685820030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f473c004690 0x7f473c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.202+0000 7f475545c700 1 -- 172.21.15.29:0/1685820030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47501563d0 msgr2=0x7f473c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.202+0000 7f475545c700 1 --2- 172.21.15.29:0/1685820030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47501563d0 0x7f473c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.202+0000 7f475545c700 1 -- 172.21.15.29:0/1685820030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f473c004690 msgr2=0x7f473c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.202+0000 7f475545c700 1 --2- 172.21.15.29:0/1685820030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f473c004690 0x7f473c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.202+0000 7f475545c700 1 -- 172.21.15.29:0/1685820030 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c0033f0 con 0x7f473c0038e0 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.202+0000 7f475545c700 1 --2- 172.21.15.29:0/1685820030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f473c0038e0 0x7f473c003d00 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f474000c210 tx=0x7f4740001e90).ready entity=mon.0 client_cookie=30870e5f23657989 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.202+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740007a60 con 0x7f473c0038e0 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.202+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474000cae0 con 0x7f473c0038e0 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.202+0000 7f4747fff700 1 -- 172.21.15.29:0/1685820030 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474000cea0 con 0x7f473c0038e0 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.203+0000 7f47576c0700 1 -- 172.21.15.29:0/1685820030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f473c0038e0 msgr2=0x7f473c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:31.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.203+0000 7f47576c0700 1 --2- 172.21.15.29:0/1685820030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f473c0038e0 0x7f473c003d00 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f474000c210 tx=0x7f4740001e90).stop 2024-01-24T15:13:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.203+0000 7f4755c5d700 1 -- 172.21.15.29:0/1685820030 reap_dead start 2024-01-24T15:13:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.203+0000 7f47576c0700 1 -- 172.21.15.29:0/1685820030 shutdown_connections 2024-01-24T15:13:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.203+0000 7f47576c0700 1 -- 172.21.15.29:0/1685820030 >> 172.21.15.29:0/1685820030 conn(0x7f47500c8bb0 msgr2=0x7f47500c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.203+0000 7f47576c0700 1 -- 172.21.15.29:0/1685820030 shutdown_connections 2024-01-24T15:13:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.203+0000 7f47576c0700 1 -- 172.21.15.29:0/1685820030 wait complete. 2024-01-24T15:13:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.204+0000 7f47576c0700 1 Processor -- start 2024-01-24T15:13:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.204+0000 7f47576c0700 1 -- start start 2024-01-24T15:13:31.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f47576c0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f473c0038e0 0x7f4750138210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:31.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f47576c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f475014a670 0x7f475014aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:31.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f47576c0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4750167530 0x7f47501679b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f47576c0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750168490 con 0x7f473c0038e0 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f47576c0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750064040 con 0x7f475014a670 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f47576c0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475014afe0 con 0x7f4750167530 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f475545c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f473c0038e0 0x7f4750138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f475545c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f473c0038e0 0x7f4750138210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35776/0 (socket says 172.21.15.29:35776) 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f475545c700 1 -- 172.21.15.29:0/901470552 learned_addr learned my addr 172.21.15.29:0/901470552 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f4755c5d700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4750167530 0x7f47501679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f4754c5b700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f475014a670 0x7f475014aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f475545c700 1 -- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4750167530 msgr2=0x7f47501679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f475545c700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4750167530 0x7f47501679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f475545c700 1 -- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f475014a670 msgr2=0x7f475014aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f475545c700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f475014a670 0x7f475014aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.205+0000 7f475545c700 1 -- 172.21.15.29:0/901470552 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47501681b0 con 0x7f473c0038e0 2024-01-24T15:13:31.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.206+0000 7f4755c5d700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4750167530 0x7f47501679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.206+0000 7f475545c700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f473c0038e0 0x7f4750138210 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f4740000e40 tx=0x7f4740015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.206+0000 7f47457fa700 1 -- 172.21.15.29:0/901470552 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400046b0 con 0x7f473c0038e0 2024-01-24T15:13:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.206+0000 7f47457fa700 1 -- 172.21.15.29:0/901470552 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4740004810 con 0x7f473c0038e0 2024-01-24T15:13:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.206+0000 7f47576c0700 1 -- 172.21.15.29:0/901470552 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47500639f0 con 0x7f473c0038e0 2024-01-24T15:13:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.206+0000 7f47457fa700 1 -- 172.21.15.29:0/901470552 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740004bd0 con 0x7f473c0038e0 2024-01-24T15:13:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.206+0000 7f47576c0700 1 -- 172.21.15.29:0/901470552 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4750167ea0 con 0x7f473c0038e0 2024-01-24T15:13:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.207+0000 7f47576c0700 1 -- 172.21.15.29:0/901470552 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47500475e0 con 0x7f473c0038e0 2024-01-24T15:13:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.207+0000 7f47457fa700 1 -- 172.21.15.29:0/901470552 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4740020450 con 0x7f473c0038e0 2024-01-24T15:13:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.208+0000 7f47457fa700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f473405f270 0x7f4734061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.208+0000 7f47457fa700 1 -- 172.21.15.29:0/901470552 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6850+0+0 (secure 0 0 0) 0x7f474007b640 con 0x7f473c0038e0 2024-01-24T15:13:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.208+0000 7f4754c5b700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f473405f270 0x7f4734061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:31.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.208+0000 7f4754c5b700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f473405f270 0x7f4734061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f474c006c40 tx=0x7f474c008040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:31.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.212+0000 7f47457fa700 1 -- 172.21.15.29:0/901470552 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f474004c450 con 0x7f473c0038e0 2024-01-24T15:13:31.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:31.463+0000 7f47576c0700 1 -- 172.21.15.29:0/901470552 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f47500c3630 con 0x7f473c0038e0 2024-01-24T15:13:32.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.047+0000 7f47457fa700 1 -- 172.21.15.29:0/901470552 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2422) v1 ==== 177+0+0 (secure 0 0 0) 0x7f474001a020 con 0x7f473c0038e0 2024-01-24T15:13:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.049+0000 7f47576c0700 1 -- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f473405f270 msgr2=0x7f4734061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.049+0000 7f47576c0700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f473405f270 0x7f4734061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f474c006c40 tx=0x7f474c008040).stop 2024-01-24T15:13:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.049+0000 7f47576c0700 1 -- 172.21.15.29:0/901470552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f473c0038e0 msgr2=0x7f4750138210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.049+0000 7f47576c0700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f473c0038e0 0x7f4750138210 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f4740000e40 tx=0x7f4740015000).stop 2024-01-24T15:13:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.049+0000 7f47576c0700 1 -- 172.21.15.29:0/901470552 shutdown_connections 2024-01-24T15:13:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.049+0000 7f47576c0700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4750167530 0x7f47501679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.049+0000 7f47576c0700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f473405f270 0x7f4734061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.049+0000 7f47576c0700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f473c0038e0 0x7f4750138210 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.049+0000 7f47576c0700 1 --2- 172.21.15.29:0/901470552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f475014a670 0x7f475014aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.049+0000 7f47576c0700 1 -- 172.21.15.29:0/901470552 >> 172.21.15.29:0/901470552 conn(0x7f47500c8bb0 msgr2=0x7f47500aae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:32.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.050+0000 7f47576c0700 1 -- 172.21.15.29:0/901470552 shutdown_connections 2024-01-24T15:13:32.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.050+0000 7f47576c0700 1 -- 172.21.15.29:0/901470552 wait complete. 2024-01-24T15:13:32.067 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-24T15:13:32.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.202+0000 7fa96aba2700 1 Processor -- start 2024-01-24T15:13:32.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.202+0000 7fa96aba2700 1 -- start start 2024-01-24T15:13:32.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.203+0000 7fa96aba2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa964153f40 0x7fa9640a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:32.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.203+0000 7fa96aba2700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96406a360 con 0x7fa9640c8290 2024-01-24T15:13:32.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.203+0000 7fa96aba2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9640bcef0 con 0x7fa964062ec0 2024-01-24T15:13:32.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.203+0000 7fa96aba2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa964069de0 con 0x7fa964153f40 2024-01-24T15:13:32.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.203+0000 7fa963fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa9640c8290 0x7fa9640627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52378/0 (socket says 172.21.15.29:52378) 2024-01-24T15:13:32.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.203+0000 7fa963fff700 1 -- 172.21.15.29:0/2457723506 learned_addr learned my addr 172.21.15.29:0/2457723506 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:32.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.203+0000 7fa96893e700 1 --2- 172.21.15.29:0/2457723506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa964153f40 0x7fa9640a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:32.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.203+0000 7fa9637fe700 1 -- 172.21.15.29:0/2457723506 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2851440876 0 0) 0x7fa96406a360 con 0x7fa9640c8290 2024-01-24T15:13:32.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.203+0000 7fa9637fe700 1 -- 172.21.15.29:0/2457723506 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9540033f0 con 0x7fa9640c8290 2024-01-24T15:13:32.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.204+0000 7fa9637fe700 1 -- 172.21.15.29:0/2457723506 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431835581 0 0) 0x7fa9640bcef0 con 0x7fa964062ec0 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.204+0000 7fa9637fe700 1 -- 172.21.15.29:0/2457723506 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa96406a360 con 0x7fa964062ec0 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.204+0000 7fa96893e700 1 -- 172.21.15.29:0/2457723506 >> v1:172.21.15.163:6789/0 conn(0x7fa964062ec0 legacy=0x7fa964065440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.204+0000 7fa96893e700 1 -- 172.21.15.29:0/2457723506 >> v1:172.21.15.29:6789/0 conn(0x7fa9640c8290 legacy=0x7fa9640627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.204+0000 7fa96893e700 1 -- 172.21.15.29:0/2457723506 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96406a080 con 0x7fa964153f40 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.204+0000 7fa96893e700 1 --2- 172.21.15.29:0/2457723506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa964153f40 0x7fa9640a6db0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fa958009790 tx=0x7fa958009060).ready entity=mon.2 client_cookie=6e8404d400a39ae3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.204+0000 7fa9637fe700 1 -- 172.21.15.29:0/2457723506 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9580090d0 con 0x7fa964153f40 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.204+0000 7fa9637fe700 1 -- 172.21.15.29:0/2457723506 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95800f510 con 0x7fa964153f40 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.204+0000 7fa9637fe700 1 -- 172.21.15.29:0/2457723506 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800f890 con 0x7fa964153f40 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.205+0000 7fa96aba2700 1 -- 172.21.15.29:0/2457723506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa964153f40 msgr2=0x7fa9640a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.205+0000 7fa96aba2700 1 --2- 172.21.15.29:0/2457723506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa964153f40 0x7fa9640a6db0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fa958009790 tx=0x7fa958009060).stop 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.205+0000 7fa96aba2700 1 -- 172.21.15.29:0/2457723506 shutdown_connections 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.205+0000 7fa96aba2700 1 --2- 172.21.15.29:0/2457723506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa964153f40 0x7fa9640a6db0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.205+0000 7fa96aba2700 1 -- 172.21.15.29:0/2457723506 >> 172.21.15.29:0/2457723506 conn(0x7fa964064390 msgr2=0x7fa9640c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.205+0000 7fa96aba2700 1 -- 172.21.15.29:0/2457723506 shutdown_connections 2024-01-24T15:13:32.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.205+0000 7fa96aba2700 1 -- 172.21.15.29:0/2457723506 wait complete. 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.206+0000 7fa96aba2700 1 Processor -- start 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.206+0000 7fa96aba2700 1 -- start start 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96aba2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa964062ec0 0x7fa9640beb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96aba2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa9640c8290 0x7fa9641678a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96aba2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa964153f40 0x7fa96414f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96aba2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9640c39d0 con 0x7fa964153f40 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96aba2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa964151190 con 0x7fa964062ec0 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96aba2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9640bf080 con 0x7fa9640c8290 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96913f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa964153f40 0x7fa96414f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96913f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa964153f40 0x7fa96414f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35782/0 (socket says 172.21.15.29:35782) 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96913f700 1 -- 172.21.15.29:0/2279303628 learned_addr learned my addr 172.21.15.29:0/2279303628 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa963fff700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa9640c8290 0x7fa9641678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:32.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96893e700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa964062ec0 0x7fa9640beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96913f700 1 -- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa9640c8290 msgr2=0x7fa9641678a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96913f700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa9640c8290 0x7fa9641678a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96913f700 1 -- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa964062ec0 msgr2=0x7fa9640beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96913f700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa964062ec0 0x7fa9640beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.207+0000 7fa96913f700 1 -- 172.21.15.29:0/2279303628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9580092a0 con 0x7fa964153f40 2024-01-24T15:13:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.208+0000 7fa96913f700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa964153f40 0x7fa96414f080 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7fa95c00c7c0 tx=0x7fa95c007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.208+0000 7fa960ff9700 1 -- 172.21.15.29:0/2279303628 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c00e4a0 con 0x7fa964153f40 2024-01-24T15:13:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.208+0000 7fa960ff9700 1 -- 172.21.15.29:0/2279303628 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95c00e600 con 0x7fa964153f40 2024-01-24T15:13:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.208+0000 7fa96aba2700 1 -- 172.21.15.29:0/2279303628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa964150840 con 0x7fa964153f40 2024-01-24T15:13:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.208+0000 7fa960ff9700 1 -- 172.21.15.29:0/2279303628 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c00e980 con 0x7fa964153f40 2024-01-24T15:13:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.208+0000 7fa96aba2700 1 -- 172.21.15.29:0/2279303628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa964150e00 con 0x7fa964153f40 2024-01-24T15:13:32.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.209+0000 7fa96aba2700 1 -- 172.21.15.29:0/2279303628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9640475e0 con 0x7fa964153f40 2024-01-24T15:13:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.209+0000 7fa960ff9700 1 -- 172.21.15.29:0/2279303628 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa95c00ed30 con 0x7fa964153f40 2024-01-24T15:13:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.210+0000 7fa960ff9700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa94805f270 0x7fa948061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.210+0000 7fa960ff9700 1 -- 172.21.15.29:0/2279303628 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6850+0+0 (secure 0 0 0) 0x7fa95c079d30 con 0x7fa964153f40 2024-01-24T15:13:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.210+0000 7fa96893e700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa94805f270 0x7fa948061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:32.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.210+0000 7fa96893e700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa94805f270 0x7fa948061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa958009fd0 tx=0x7fa958017000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:32.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.214+0000 7fa960ff9700 1 -- 172.21.15.29:0/2279303628 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa95c048b00 con 0x7fa964153f40 2024-01-24T15:13:32.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.397+0000 7fa96aba2700 1 -- 172.21.15.29:0/2279303628 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa96406a540 con 0x7fa964153f40 2024-01-24T15:13:32.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.401+0000 7fa960ff9700 1 -- 172.21.15.29:0/2279303628 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa95c017030 con 0x7fa964153f40 2024-01-24T15:13:32.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.403+0000 7fa96aba2700 1 -- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa94805f270 msgr2=0x7fa948061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.403+0000 7fa96aba2700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa94805f270 0x7fa948061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa958009fd0 tx=0x7fa958017000).stop 2024-01-24T15:13:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.403+0000 7fa96aba2700 1 -- 172.21.15.29:0/2279303628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa964153f40 msgr2=0x7fa96414f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.403+0000 7fa96aba2700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa964153f40 0x7fa96414f080 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7fa95c00c7c0 tx=0x7fa95c007f50).stop 2024-01-24T15:13:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.403+0000 7fa96aba2700 1 -- 172.21.15.29:0/2279303628 shutdown_connections 2024-01-24T15:13:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.403+0000 7fa96aba2700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa94805f270 0x7fa948061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.403+0000 7fa96aba2700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa964153f40 0x7fa96414f080 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.403+0000 7fa96aba2700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa964062ec0 0x7fa9640beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.403+0000 7fa96aba2700 1 --2- 172.21.15.29:0/2279303628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa9640c8290 0x7fa9641678a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.403+0000 7fa96aba2700 1 -- 172.21.15.29:0/2279303628 >> 172.21.15.29:0/2279303628 conn(0x7fa964064390 msgr2=0x7fa9640a5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.403+0000 7fa96aba2700 1 -- 172.21.15.29:0/2279303628 shutdown_connections 2024-01-24T15:13:32.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.403+0000 7fa96aba2700 1 -- 172.21.15.29:0/2279303628 wait complete. 2024-01-24T15:13:32.419 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T15:13:32.419 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:13:32.464 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:13:32.516 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T15:13:32.516 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-24T15:13:32.516 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:13:32.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.650+0000 7f06161f0700 1 Processor -- start 2024-01-24T15:13:32.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.650+0000 7f06161f0700 1 -- start start 2024-01-24T15:13:32.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.650+0000 7f06161f0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06101540a0 0x7f06100c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f06161f0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0610062ec0 0x7f0610069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f06161f0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0610065360 con 0x7f06101550f0 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f06161f0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06100a74f0 con 0x7f0610062ec0 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f06161f0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100a6ce0 con 0x7f06101540a0 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f060ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0610062ec0 0x7f0610069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f060ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0610062ec0 0x7f0610069240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35784/0 (socket says 172.21.15.29:35784) 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f060ffff700 1 -- 172.21.15.29:0/1166490316 learned_addr learned my addr 172.21.15.29:0/1166490316 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f060f7fe700 1 --2- 172.21.15.29:0/1166490316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06101540a0 0x7f06100c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f060ffff700 1 -- 172.21.15.29:0/1166490316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06101540a0 msgr2=0x7f06100c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f060ffff700 1 --2- 172.21.15.29:0/1166490316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06101540a0 0x7f06100c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f060ffff700 1 -- 172.21.15.29:0/1166490316 >> v1:172.21.15.163:6789/0 conn(0x7f06101550f0 legacy=0x7f06100627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f060ffff700 1 -- 172.21.15.29:0/1166490316 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06100636f0 con 0x7f0610062ec0 2024-01-24T15:13:32.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f060ffff700 1 --2- 172.21.15.29:0/1166490316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0610062ec0 0x7f0610069240 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f0608009f20 tx=0x7f0608009840).ready entity=mon.0 client_cookie=4339fb641a682276 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:32.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f060e7fc700 1 -- 172.21.15.29:0/1166490316 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060800e070 con 0x7f0610062ec0 2024-01-24T15:13:32.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f060e7fc700 1 -- 172.21.15.29:0/1166490316 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060800ba80 con 0x7f0610062ec0 2024-01-24T15:13:32.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.651+0000 7f060e7fc700 1 -- 172.21.15.29:0/1166490316 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060800be00 con 0x7f0610062ec0 2024-01-24T15:13:32.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.652+0000 7f06161f0700 1 -- 172.21.15.29:0/1166490316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0610062ec0 msgr2=0x7f0610069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.652+0000 7f06161f0700 1 --2- 172.21.15.29:0/1166490316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0610062ec0 0x7f0610069240 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f0608009f20 tx=0x7f0608009840).stop 2024-01-24T15:13:32.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.652+0000 7f06161f0700 1 -- 172.21.15.29:0/1166490316 shutdown_connections 2024-01-24T15:13:32.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.652+0000 7f06161f0700 1 --2- 172.21.15.29:0/1166490316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06101540a0 0x7f06100c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.652+0000 7f06161f0700 1 --2- 172.21.15.29:0/1166490316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0610062ec0 0x7f0610069240 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.652+0000 7f06161f0700 1 -- 172.21.15.29:0/1166490316 >> 172.21.15.29:0/1166490316 conn(0x7f06100c85c0 msgr2=0x7f06100c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:32.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.652+0000 7f06161f0700 1 -- 172.21.15.29:0/1166490316 shutdown_connections 2024-01-24T15:13:32.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.652+0000 7f06161f0700 1 -- 172.21.15.29:0/1166490316 wait complete. 2024-01-24T15:13:32.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.653+0000 7f06161f0700 1 Processor -- start 2024-01-24T15:13:32.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.654+0000 7f06161f0700 1 -- start start 2024-01-24T15:13:32.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.654+0000 7f06161f0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0610062ec0 0x7f06101530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:32.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.654+0000 7f06161f0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06101540a0 0x7f0610164650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:32.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.654+0000 7f06161f0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06101550f0 0x7f06100abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:32.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.654+0000 7f06161f0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06100aba70 con 0x7f06101540a0 2024-01-24T15:13:32.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.654+0000 7f06161f0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06100ace20 con 0x7f06101550f0 2024-01-24T15:13:32.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.654+0000 7f06161f0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100aae60 con 0x7f0610062ec0 2024-01-24T15:13:32.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.654+0000 7f060effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06101540a0 0x7f0610164650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.654+0000 7f060effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06101540a0 0x7f0610164650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35790/0 (socket says 172.21.15.29:35790) 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.654+0000 7f060effd700 1 -- 172.21.15.29:0/1555822474 learned_addr learned my addr 172.21.15.29:0/1555822474 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f060f7fe700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0610062ec0 0x7f06101530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f060ffff700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06101550f0 0x7f06100abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f060effd700 1 -- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0610062ec0 msgr2=0x7f06101530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f060effd700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0610062ec0 0x7f06101530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f060effd700 1 -- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06101550f0 msgr2=0x7f06100abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f060effd700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06101550f0 0x7f06100abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f060effd700 1 -- 172.21.15.29:0/1555822474 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0608009ae0 con 0x7f06101540a0 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f060ffff700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06101550f0 0x7f06100abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f060effd700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06101540a0 0x7f0610164650 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f060400c440 tx=0x7f0604007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f05f77fe700 1 -- 172.21.15.29:0/1555822474 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060400ce40 con 0x7f06101540a0 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f05f77fe700 1 -- 172.21.15.29:0/1555822474 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0604007020 con 0x7f06101540a0 2024-01-24T15:13:32.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f06161f0700 1 -- 172.21.15.29:0/1555822474 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06100ab140 con 0x7f06101540a0 2024-01-24T15:13:32.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f05f77fe700 1 -- 172.21.15.29:0/1555822474 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06040073c0 con 0x7f06101540a0 2024-01-24T15:13:32.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.655+0000 7f06161f0700 1 -- 172.21.15.29:0/1555822474 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06100ab780 con 0x7f06101540a0 2024-01-24T15:13:32.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.656+0000 7f06161f0700 1 -- 172.21.15.29:0/1555822474 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06100abc20 con 0x7f06101540a0 2024-01-24T15:13:32.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.657+0000 7f05f77fe700 1 -- 172.21.15.29:0/1555822474 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f06040068b0 con 0x7f06101540a0 2024-01-24T15:13:32.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.657+0000 7f05f77fe700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f05fc05f270 0x7f05fc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:32.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.657+0000 7f05f77fe700 1 -- 172.21.15.29:0/1555822474 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6850+0+0 (secure 0 0 0) 0x7f060407a040 con 0x7f06101540a0 2024-01-24T15:13:32.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.657+0000 7f060f7fe700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f05fc05f270 0x7f05fc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:32.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.658+0000 7f060f7fe700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f05fc05f270 0x7f05fc061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f060000bbc0 tx=0x7f0600008040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:32.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.661+0000 7f05f77fe700 1 -- 172.21.15.29:0/1555822474 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f06100abc20 con 0x7f06101540a0 2024-01-24T15:13:32.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.886+0000 7f06161f0700 1 -- 172.21.15.29:0/1555822474 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06100475e0 con 0x7f06101540a0 2024-01-24T15:13:32.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.887+0000 7f05f77fe700 1 -- 172.21.15.29:0/1555822474 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f060404aeb0 con 0x7f06101540a0 2024-01-24T15:13:32.889 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:32.889 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":68,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2106677945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2106677945},{"type":"v1","addr":"172.21.15.163:6836","nonce":2106677945}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7786,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1964541525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1964541525},{"type":"v1","addr":"172.21.15.163:6837","nonce":1964541525}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/4162831288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4162831288},{"type":"v1","addr":"172.21.15.29:6835","nonce":4162831288}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4003775427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":4003775427},{"type":"v1","addr":"172.21.15.29:6834","nonce":4003775427}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[]} 2024-01-24T15:13:32.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.889+0000 7f06161f0700 1 -- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f05fc05f270 msgr2=0x7f05fc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.889+0000 7f06161f0700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f05fc05f270 0x7f05fc061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f060000bbc0 tx=0x7f0600008040).stop 2024-01-24T15:13:32.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.889+0000 7f06161f0700 1 -- 172.21.15.29:0/1555822474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06101540a0 msgr2=0x7f0610164650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:32.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.889+0000 7f06161f0700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06101540a0 0x7f0610164650 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f060400c440 tx=0x7f0604007dd0).stop 2024-01-24T15:13:32.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.889+0000 7f06161f0700 1 -- 172.21.15.29:0/1555822474 shutdown_connections 2024-01-24T15:13:32.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.889+0000 7f06161f0700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06101550f0 0x7f06100abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.889+0000 7f06161f0700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0610062ec0 0x7f06101530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.889+0000 7f06161f0700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f05fc05f270 0x7f05fc061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.889+0000 7f06161f0700 1 --2- 172.21.15.29:0/1555822474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06101540a0 0x7f0610164650 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:32.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.889+0000 7f06161f0700 1 -- 172.21.15.29:0/1555822474 >> 172.21.15.29:0/1555822474 conn(0x7f06100c85c0 msgr2=0x7f0610069240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:32.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.889+0000 7f06161f0700 1 -- 172.21.15.29:0/1555822474 shutdown_connections 2024-01-24T15:13:32.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:32.889+0000 7f06161f0700 1 -- 172.21.15.29:0/1555822474 wait complete. 2024-01-24T15:13:32.892 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 68 2024-01-24T15:13:32.905 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-24T15:13:32.905 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-24T15:13:32.905 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-24T15:13:32.905 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:13:32.906 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-24T15:13:32.906 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-24T15:13:32.906 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-24T15:13:32.906 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:13:32.907 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-24T15:13:32.907 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-24T15:13:32.907 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-24T15:13:32.907 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:13:32.907 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-24T15:13:32.907 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-24T15:13:32.907 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-24T15:13:32.907 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:13:32.916 INFO:tasks.ceph.mds.c:Stopped 2024-01-24T15:13:32.916 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-24T15:13:32.917 INFO:tasks.ceph.mds.a:Stopped 2024-01-24T15:13:32.917 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-24T15:13:32.919 INFO:tasks.ceph.mds.b:Stopped 2024-01-24T15:13:32.919 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-24T15:13:32.920 INFO:tasks.ceph.mds.d:Stopped 2024-01-24T15:13:32.920 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-24T15:13:32.964 INFO:tasks.ceph.mds.b:Started 2024-01-24T15:13:32.965 DEBUG:teuthology.parallel:result is None 2024-01-24T15:13:32.966 INFO:tasks.ceph.mds.c:Started 2024-01-24T15:13:32.966 INFO:tasks.ceph.mds.d:Started 2024-01-24T15:13:32.966 DEBUG:teuthology.parallel:result is None 2024-01-24T15:13:32.967 DEBUG:teuthology.parallel:result is None 2024-01-24T15:13:32.967 INFO:tasks.ceph.mds.a:Started 2024-01-24T15:13:32.967 DEBUG:teuthology.parallel:result is None 2024-01-24T15:13:32.968 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-24T15:13:33.046 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T15:13:33.046 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-01-24T15:13:33.053 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T15:13:33.053 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T15:13:33.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.109+0000 7f46d2dfc700 1 Processor -- start 2024-01-24T15:13:33.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.109+0000 7f46d2dfc700 1 -- start start 2024-01-24T15:13:33.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46d2dfc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46cc154240 0x7f46cc064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:33.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46d2dfc700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc1550e0 con 0x7f46cc0aa180 2024-01-24T15:13:33.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46d2dfc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc158b10 con 0x7f46cc0a9d20 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46d2dfc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46cc0c0d10 con 0x7f46cc154240 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46d1399700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f46cc0aa180 0x7f46cc158400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52398/0 (socket says 172.21.15.29:52398) 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46d1399700 1 -- 172.21.15.29:0/1937122624 learned_addr learned my addr 172.21.15.29:0/1937122624 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46d0b98700 1 --2- 172.21.15.29:0/1937122624 >> v2:172.21.15.163:3300/0 conn(0x7f46cc154240 0x7f46cc064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46cb7fe700 1 -- 172.21.15.29:0/1937122624 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445872407 0 0) 0x7f46cc1550e0 con 0x7f46cc0aa180 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46cb7fe700 1 -- 172.21.15.29:0/1937122624 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46bc0033f0 con 0x7f46cc0aa180 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46cb7fe700 1 -- 172.21.15.29:0/1937122624 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477248967 0 0) 0x7f46cc158b10 con 0x7f46cc0a9d20 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46cb7fe700 1 -- 172.21.15.29:0/1937122624 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46cc1550e0 con 0x7f46cc0a9d20 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46d0b98700 1 -- 172.21.15.29:0/1937122624 >> v1:172.21.15.163:6789/0 conn(0x7f46cc0a9d20 legacy=0x7f46cc157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46cb7fe700 1 -- 172.21.15.29:0/1937122624 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3406274927 0 0) 0x7f46bc0033f0 con 0x7f46cc0aa180 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46d0b98700 1 -- 172.21.15.29:0/1937122624 >> v1:172.21.15.29:6789/0 conn(0x7f46cc0aa180 legacy=0x7f46cc158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.110+0000 7f46d0b98700 1 -- 172.21.15.29:0/1937122624 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc0650d0 con 0x7f46cc154240 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.111+0000 7f46d0b98700 1 --2- 172.21.15.29:0/1937122624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc154240 0x7f46cc064780 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f46c0009760 tx=0x7f46c0009060).ready entity=mon.1 client_cookie=fdb43a1c8fd979c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:33.114 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.111+0000 7f46cb7fe700 1 -- 172.21.15.29:0/1937122624 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c40017f0 con 0x7f46cc154240 2024-01-24T15:13:33.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.111+0000 7f46cb7fe700 1 -- 172.21.15.29:0/1937122624 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c0011400 con 0x7f46cc154240 2024-01-24T15:13:33.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.111+0000 7f46cb7fe700 1 -- 172.21.15.29:0/1937122624 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0011780 con 0x7f46cc154240 2024-01-24T15:13:33.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.111+0000 7f46d2dfc700 1 -- 172.21.15.29:0/1937122624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc154240 msgr2=0x7f46cc064780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:33.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.111+0000 7f46d2dfc700 1 --2- 172.21.15.29:0/1937122624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc154240 0x7f46cc064780 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f46c0009760 tx=0x7f46c0009060).stop 2024-01-24T15:13:33.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.111+0000 7f46d2dfc700 1 -- 172.21.15.29:0/1937122624 shutdown_connections 2024-01-24T15:13:33.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.111+0000 7f46d2dfc700 1 --2- 172.21.15.29:0/1937122624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc154240 0x7f46cc064780 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:33.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.111+0000 7f46d2dfc700 1 -- 172.21.15.29:0/1937122624 >> 172.21.15.29:0/1937122624 conn(0x7f46cc0c8310 msgr2=0x7f46cc05e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:33.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.111+0000 7f46d2dfc700 1 -- 172.21.15.29:0/1937122624 shutdown_connections 2024-01-24T15:13:33.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.111+0000 7f46d2dfc700 1 -- 172.21.15.29:0/1937122624 wait complete. 2024-01-24T15:13:33.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.113+0000 7f46d2dfc700 1 Processor -- start 2024-01-24T15:13:33.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.113+0000 7f46d2dfc700 1 -- start start 2024-01-24T15:13:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d2dfc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46cc0a9d20 0x7f46cc148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d2dfc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc0aa180 0x7f46cc0ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d2dfc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46cc154240 0x7f46cc156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d2dfc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc0c2770 con 0x7f46cc154240 2024-01-24T15:13:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d2dfc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc0c2440 con 0x7f46cc0aa180 2024-01-24T15:13:33.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d2dfc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc0af0b0 con 0x7f46cc0a9d20 2024-01-24T15:13:33.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d1399700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46cc154240 0x7f46cc156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:33.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d1399700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46cc154240 0x7f46cc156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35832/0 (socket says 172.21.15.29:35832) 2024-01-24T15:13:33.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d1399700 1 -- 172.21.15.29:0/2516834168 learned_addr learned my addr 172.21.15.29:0/2516834168 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:33.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46cbfff700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc0aa180 0x7f46cc0ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:33.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d0b98700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46cc0a9d20 0x7f46cc148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:33.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d1399700 1 -- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46cc0a9d20 msgr2=0x7f46cc148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:33.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d1399700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46cc0a9d20 0x7f46cc148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:33.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d1399700 1 -- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc0aa180 msgr2=0x7f46cc0ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:33.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d1399700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc0aa180 0x7f46cc0ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:33.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d1399700 1 -- 172.21.15.29:0/2516834168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c00092a0 con 0x7f46cc154240 2024-01-24T15:13:33.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.114+0000 7f46d1399700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46cc154240 0x7f46cc156a40 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f46c400cb50 tx=0x7f46c4001e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:33.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.115+0000 7f46c8ff9700 1 -- 172.21.15.29:0/2516834168 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c4004c40 con 0x7f46cc154240 2024-01-24T15:13:33.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.115+0000 7f46c8ff9700 1 -- 172.21.15.29:0/2516834168 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c4004da0 con 0x7f46cc154240 2024-01-24T15:13:33.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.115+0000 7f46d2dfc700 1 -- 172.21.15.29:0/2516834168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc0af390 con 0x7f46cc154240 2024-01-24T15:13:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.115+0000 7f46c8ff9700 1 -- 172.21.15.29:0/2516834168 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c40073c0 con 0x7f46cc154240 2024-01-24T15:13:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.115+0000 7f46d2dfc700 1 -- 172.21.15.29:0/2516834168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc148d00 con 0x7f46cc154240 2024-01-24T15:13:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.116+0000 7f46d2dfc700 1 -- 172.21.15.29:0/2516834168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc0c28e0 con 0x7f46cc154240 2024-01-24T15:13:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.116+0000 7f46c8ff9700 1 -- 172.21.15.29:0/2516834168 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f46c4007770 con 0x7f46cc154240 2024-01-24T15:13:33.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.116+0000 7f46c8ff9700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f46b005f270 0x7f46b0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:33.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.117+0000 7f46c8ff9700 1 -- 172.21.15.29:0/2516834168 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6850+0+0 (secure 0 0 0) 0x7f46c407a340 con 0x7f46cc154240 2024-01-24T15:13:33.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.117+0000 7f46d0b98700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f46b005f270 0x7f46b0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:33.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.117+0000 7f46d0b98700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f46b005f270 0x7f46b0061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f46c0009760 tx=0x7f46c0016000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:33.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.121+0000 7f46c8ff9700 1 -- 172.21.15.29:0/2516834168 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f46c404b150 con 0x7f46cc154240 2024-01-24T15:13:33.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:33.323+0000 7f46d2dfc700 1 -- 172.21.15.29:0/2516834168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f46cc0475e0 con 0x7f46cc154240 2024-01-24T15:13:34.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.087+0000 7f46c8ff9700 1 -- 172.21.15.29:0/2516834168 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v174) v1 ==== 97+0+0 (secure 0 0 0) 0x7f46c4001f10 con 0x7f46cc154240 2024-01-24T15:13:34.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.089+0000 7f46d2dfc700 1 -- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f46b005f270 msgr2=0x7f46b0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.089+0000 7f46d2dfc700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f46b005f270 0x7f46b0061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f46c0009760 tx=0x7f46c0016000).stop 2024-01-24T15:13:34.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.089+0000 7f46d2dfc700 1 -- 172.21.15.29:0/2516834168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46cc154240 msgr2=0x7f46cc156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.089+0000 7f46d2dfc700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46cc154240 0x7f46cc156a40 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f46c400cb50 tx=0x7f46c4001e80).stop 2024-01-24T15:13:34.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.089+0000 7f46d2dfc700 1 -- 172.21.15.29:0/2516834168 shutdown_connections 2024-01-24T15:13:34.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.089+0000 7f46d2dfc700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f46b005f270 0x7f46b0061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.089+0000 7f46d2dfc700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46cc154240 0x7f46cc156a40 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.089+0000 7f46d2dfc700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46cc0a9d20 0x7f46cc148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.089+0000 7f46d2dfc700 1 --2- 172.21.15.29:0/2516834168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc0aa180 0x7f46cc0ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.089+0000 7f46d2dfc700 1 -- 172.21.15.29:0/2516834168 >> 172.21.15.29:0/2516834168 conn(0x7f46cc0c8310 msgr2=0x7f46cc0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:34.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.090+0000 7f46d2dfc700 1 -- 172.21.15.29:0/2516834168 shutdown_connections 2024-01-24T15:13:34.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.090+0000 7f46d2dfc700 1 -- 172.21.15.29:0/2516834168 wait complete. 2024-01-24T15:13:34.093 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-01-24T15:13:34.107 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-24T15:13:34.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.239+0000 7fc9e5bbd700 1 Processor -- start 2024-01-24T15:13:34.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.239+0000 7fc9e5bbd700 1 -- start start 2024-01-24T15:13:34.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9e5bbd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc9e0138710 0x7fc9e00be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:34.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9e5bbd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc9e00627b0 0x7fc9e0062c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:34.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9e5bbd700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e0065770 con 0x7fc9e0138330 2024-01-24T15:13:34.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9e5bbd700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e0063130 con 0x7fc9e00627b0 2024-01-24T15:13:34.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9e5bbd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e0069cf0 con 0x7fc9e0138710 2024-01-24T15:13:34.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9dffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc9e00627b0 0x7fc9e0062c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:34.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9dffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc9e00627b0 0x7fc9e0062c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51590/0 (socket says 172.21.15.29:51590) 2024-01-24T15:13:34.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9dffff700 1 -- 172.21.15.29:0/2654743340 learned_addr learned my addr 172.21.15.29:0/2654743340 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:34.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9deffd700 1 --2- 172.21.15.29:0/2654743340 >> v2:172.21.15.163:3300/0 conn(0x7fc9e0138710 0x7fc9e00be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:34.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9dffff700 1 -- 172.21.15.29:0/2654743340 >> v2:172.21.15.163:3300/0 conn(0x7fc9e0138710 msgr2=0x7fc9e00be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:34.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9dffff700 1 --2- 172.21.15.29:0/2654743340 >> v2:172.21.15.163:3300/0 conn(0x7fc9e0138710 0x7fc9e00be3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9dffff700 1 -- 172.21.15.29:0/2654743340 >> v1:172.21.15.163:6789/0 conn(0x7fc9e0138330 legacy=0x7fc9e00bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.240+0000 7fc9dffff700 1 -- 172.21.15.29:0/2654743340 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e0069f60 con 0x7fc9e00627b0 2024-01-24T15:13:34.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.241+0000 7fc9dffff700 1 --2- 172.21.15.29:0/2654743340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9e00627b0 0x7fc9e0062c40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7fc9e0069c20 tx=0x7fc9d8009840).ready entity=mon.0 client_cookie=f2a0ff2e852259f4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:34.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.241+0000 7fc9de7fc700 1 -- 172.21.15.29:0/2654743340 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d800e070 con 0x7fc9e00627b0 2024-01-24T15:13:34.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.241+0000 7fc9de7fc700 1 -- 172.21.15.29:0/2654743340 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d800ba50 con 0x7fc9e00627b0 2024-01-24T15:13:34.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.241+0000 7fc9de7fc700 1 -- 172.21.15.29:0/2654743340 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d800bdd0 con 0x7fc9e00627b0 2024-01-24T15:13:34.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.241+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/2654743340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9e00627b0 msgr2=0x7fc9e0062c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.241+0000 7fc9e5bbd700 1 --2- 172.21.15.29:0/2654743340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9e00627b0 0x7fc9e0062c40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7fc9e0069c20 tx=0x7fc9d8009840).stop 2024-01-24T15:13:34.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.241+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/2654743340 shutdown_connections 2024-01-24T15:13:34.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.241+0000 7fc9e5bbd700 1 --2- 172.21.15.29:0/2654743340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9e00627b0 0x7fc9e0062c40 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.241+0000 7fc9e5bbd700 1 --2- 172.21.15.29:0/2654743340 >> v2:172.21.15.163:3300/0 conn(0x7fc9e0138710 0x7fc9e00be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.241+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/2654743340 >> 172.21.15.29:0/2654743340 conn(0x7fc9e00c8580 msgr2=0x7fc9e00c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:34.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.241+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/2654743340 shutdown_connections 2024-01-24T15:13:34.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.241+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/2654743340 wait complete. 2024-01-24T15:13:34.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.243+0000 7fc9e5bbd700 1 Processor -- start 2024-01-24T15:13:34.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.243+0000 7fc9e5bbd700 1 -- start start 2024-01-24T15:13:34.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.244+0000 7fc9e5bbd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e00627b0 0x7fc9e0137680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:34.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.244+0000 7fc9e5bbd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9e0138330 0x7fc9e00beb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:34.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.244+0000 7fc9e5bbd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e0138710 0x7fc9e00ac380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:34.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.244+0000 7fc9e5bbd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00bfb30 con 0x7fc9e0138330 2024-01-24T15:13:34.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.244+0000 7fc9e5bbd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00bf080 con 0x7fc9e0138710 2024-01-24T15:13:34.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.244+0000 7fc9e5bbd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e0162370 con 0x7fc9e00627b0 2024-01-24T15:13:34.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.244+0000 7fc9deffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9e0138330 0x7fc9e00beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:34.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9deffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9e0138330 0x7fc9e00beb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51604/0 (socket says 172.21.15.29:51604) 2024-01-24T15:13:34.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9deffd700 1 -- 172.21.15.29:0/3074635679 learned_addr learned my addr 172.21.15.29:0/3074635679 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:34.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9df7fe700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e00627b0 0x7fc9e0137680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:34.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9dffff700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e0138710 0x7fc9e00ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:34.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9deffd700 1 -- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e00627b0 msgr2=0x7fc9e0137680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9deffd700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e00627b0 0x7fc9e0137680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9deffd700 1 -- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e0138710 msgr2=0x7fc9e00ac380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9deffd700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e0138710 0x7fc9e00ac380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9deffd700 1 -- 172.21.15.29:0/3074635679 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d8009ae0 con 0x7fc9e0138330 2024-01-24T15:13:34.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9deffd700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9e0138330 0x7fc9e00beb90 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7fc9c800d890 tx=0x7fc9c8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:34.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9d77fe700 1 -- 172.21.15.29:0/3074635679 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8011590 con 0x7fc9e0138330 2024-01-24T15:13:34.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9d77fe700 1 -- 172.21.15.29:0/3074635679 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c80116f0 con 0x7fc9e0138330 2024-01-24T15:13:34.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9d77fe700 1 -- 172.21.15.29:0/3074635679 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8011a70 con 0x7fc9e0138330 2024-01-24T15:13:34.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/3074635679 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e0162640 con 0x7fc9e0138330 2024-01-24T15:13:34.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.245+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/3074635679 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e00bf310 con 0x7fc9e0138330 2024-01-24T15:13:34.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.246+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/3074635679 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e01627d0 con 0x7fc9e0138330 2024-01-24T15:13:34.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.247+0000 7fc9d77fe700 1 -- 172.21.15.29:0/3074635679 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc9c8012420 con 0x7fc9e0138330 2024-01-24T15:13:34.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.247+0000 7fc9d77fe700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc9c005f270 0x7fc9c0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:34.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.247+0000 7fc9d77fe700 1 -- 172.21.15.29:0/3074635679 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6334+0+0 (secure 0 0 0) 0x7fc9c8079d50 con 0x7fc9e0138330 2024-01-24T15:13:34.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.248+0000 7fc9df7fe700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc9c005f270 0x7fc9c0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:34.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.248+0000 7fc9df7fe700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc9c005f270 0x7fc9c0061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fc9e0063700 tx=0x7fc9d0009040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:34.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.252+0000 7fc9d77fe700 1 -- 172.21.15.29:0/3074635679 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc9c804ad70 con 0x7fc9e0138330 2024-01-24T15:13:34.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.506+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/3074635679 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc9e00475e0 con 0x7fc9e0138330 2024-01-24T15:13:34.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.506+0000 7fc9d77fe700 1 -- 172.21.15.29:0/3074635679 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v43) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc9c8011e60 con 0x7fc9e0138330 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew==", 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-01-24T15:13:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ==", 2024-01-24T15:13:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:13:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:13:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:13:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw==", 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ==", 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:13:34.511 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ==", 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.513 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-01-24T15:13:34.513 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA==", 2024-01-24T15:13:34.513 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.513 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:13:34.513 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:13:34.513 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.513 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.514 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-01-24T15:13:34.514 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw==", 2024-01-24T15:13:34.514 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.514 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:13:34.514 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:13:34.514 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.514 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.514 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-01-24T15:13:34.515 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ==", 2024-01-24T15:13:34.515 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.515 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:13:34.515 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:13:34.515 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.515 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.515 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-01-24T15:13:34.515 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw==", 2024-01-24T15:13:34.516 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.516 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:13:34.516 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:13:34.516 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.516 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.516 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-01-24T15:13:34.516 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ==", 2024-01-24T15:13:34.517 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.517 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:13:34.517 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:13:34.517 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.517 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.517 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-01-24T15:13:34.517 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A==", 2024-01-24T15:13:34.517 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.517 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:13:34.518 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:13:34.518 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.518 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.518 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-01-24T15:13:34.518 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ==", 2024-01-24T15:13:34.518 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.518 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:13:34.518 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:13:34.519 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.519 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.519 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-01-24T15:13:34.519 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw==", 2024-01-24T15:13:34.519 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.519 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:13:34.519 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-01-24T15:13:34.519 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-24T15:13:34.519 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.520 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-01-24T15:13:34.520 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA==", 2024-01-24T15:13:34.520 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.520 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:13:34.520 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T15:13:34.520 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T15:13:34.520 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T15:13:34.520 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.521 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-01-24T15:13:34.521 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g==", 2024-01-24T15:13:34.521 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.521 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:13:34.521 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T15:13:34.521 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T15:13:34.521 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T15:13:34.521 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.522 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-01-24T15:13:34.522 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ==", 2024-01-24T15:13:34.522 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.522 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:13:34.522 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T15:13:34.522 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T15:13:34.522 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T15:13:34.522 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.523 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-01-24T15:13:34.523 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlxK1bFxAAZpfJn+Tcpmq7R9Yq29FvVg==", 2024-01-24T15:13:34.523 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.523 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T15:13:34.523 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:13:34.523 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:13:34.523 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.524 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.524 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-01-24T15:13:34.524 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw==", 2024-01-24T15:13:34.524 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.524 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T15:13:34.524 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T15:13:34.524 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.525 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:34.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:34.525 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-01-24T15:13:34.525 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA==", 2024-01-24T15:13:34.525 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:13:34.525 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T15:13:34.525 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T15:13:34.525 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:13:34.525 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.526 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:34.526 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-01-24T15:13:34.526 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:13:34.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.508+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc9c005f270 msgr2=0x7fc9c0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.508+0000 7fc9e5bbd700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc9c005f270 0x7fc9c0061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fc9e0063700 tx=0x7fc9d0009040).stop 2024-01-24T15:13:34.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.509+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/3074635679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9e0138330 msgr2=0x7fc9e00beb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.509+0000 7fc9e5bbd700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9e0138330 0x7fc9e00beb90 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7fc9c800d890 tx=0x7fc9c8008040).stop 2024-01-24T15:13:34.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.509+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/3074635679 shutdown_connections 2024-01-24T15:13:34.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.509+0000 7fc9e5bbd700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e0138710 0x7fc9e00ac380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.509+0000 7fc9e5bbd700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e00627b0 0x7fc9e0137680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.509+0000 7fc9e5bbd700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc9c005f270 0x7fc9c0061730 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.509+0000 7fc9e5bbd700 1 --2- 172.21.15.29:0/3074635679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9e0138330 0x7fc9e00beb90 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.509+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/3074635679 >> 172.21.15.29:0/3074635679 conn(0x7fc9e00c8580 msgr2=0x7fc9e00c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:34.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.509+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/3074635679 shutdown_connections 2024-01-24T15:13:34.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.509+0000 7fc9e5bbd700 1 -- 172.21.15.29:0/3074635679 wait complete. 2024-01-24T15:13:34.529 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:13:34.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.657+0000 7f90cf4e5700 1 Processor -- start 2024-01-24T15:13:34.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.657+0000 7f90cf4e5700 1 -- start start 2024-01-24T15:13:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.658+0000 7f90cf4e5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c8155b00 0x7f90c80664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:34.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.658+0000 7f90cf4e5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f90c80669e0 0x7f90c8155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.658+0000 7f90cf4e5700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c8066e60 con 0x7f90c81563d0 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.658+0000 7f90cf4e5700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c80c5500 con 0x7f90c80669e0 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.658+0000 7f90cf4e5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80c3910 con 0x7f90c8155b00 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.658+0000 7f90cca80700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f90c80669e0 0x7f90c8155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.658+0000 7f90cca80700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f90c80669e0 0x7f90c8155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51620/0 (socket says 172.21.15.29:51620) 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.658+0000 7f90cca80700 1 -- 172.21.15.29:0/3745675092 learned_addr learned my addr 172.21.15.29:0/3745675092 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.658+0000 7f90cd281700 1 --2- 172.21.15.29:0/3745675092 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c8155b00 0x7f90c80664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.659+0000 7f90cca80700 1 -- 172.21.15.29:0/3745675092 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c8155b00 msgr2=0x7f90c80664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.659+0000 7f90cca80700 1 --2- 172.21.15.29:0/3745675092 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c8155b00 0x7f90c80664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.659+0000 7f90cca80700 1 -- 172.21.15.29:0/3745675092 >> v1:172.21.15.29:6789/0 conn(0x7f90c81563d0 legacy=0x7f90c8067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.659+0000 7f90cca80700 1 -- 172.21.15.29:0/3745675092 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c80c5220 con 0x7f90c80669e0 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.659+0000 7f90cca80700 1 --2- 172.21.15.29:0/3745675092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90c80669e0 0x7f90c8155ee0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f90c4006fd0 tx=0x7f90c40090f0).ready entity=mon.0 client_cookie=a24c76958acb7b62 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.659+0000 7f90bbfff700 1 -- 172.21.15.29:0/3745675092 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c400b600 con 0x7f90c80669e0 2024-01-24T15:13:34.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.659+0000 7f90bbfff700 1 -- 172.21.15.29:0/3745675092 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c400b760 con 0x7f90c80669e0 2024-01-24T15:13:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.659+0000 7f90bbfff700 1 -- 172.21.15.29:0/3745675092 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c400bae0 con 0x7f90c80669e0 2024-01-24T15:13:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.659+0000 7f90cf4e5700 1 -- 172.21.15.29:0/3745675092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90c80669e0 msgr2=0x7f90c8155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.659+0000 7f90cf4e5700 1 --2- 172.21.15.29:0/3745675092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90c80669e0 0x7f90c8155ee0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f90c4006fd0 tx=0x7f90c40090f0).stop 2024-01-24T15:13:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.660+0000 7f90cf4e5700 1 -- 172.21.15.29:0/3745675092 shutdown_connections 2024-01-24T15:13:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.660+0000 7f90cf4e5700 1 --2- 172.21.15.29:0/3745675092 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c8155b00 0x7f90c80664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.660+0000 7f90cf4e5700 1 --2- 172.21.15.29:0/3745675092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90c80669e0 0x7f90c8155ee0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.660+0000 7f90cf4e5700 1 -- 172.21.15.29:0/3745675092 >> 172.21.15.29:0/3745675092 conn(0x7f90c80c8bb0 msgr2=0x7f90c80c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.660+0000 7f90cf4e5700 1 -- 172.21.15.29:0/3745675092 shutdown_connections 2024-01-24T15:13:34.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.660+0000 7f90cf4e5700 1 -- 172.21.15.29:0/3745675092 wait complete. 2024-01-24T15:13:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.661+0000 7f90cf4e5700 1 Processor -- start 2024-01-24T15:13:34.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.661+0000 7f90cf4e5700 1 -- start start 2024-01-24T15:13:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cf4e5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90c80669e0 0x7f90c80c0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cf4e5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c8155b00 0x7f90c813aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cf4e5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c81563d0 0x7f90c80acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cf4e5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c8153ce0 con 0x7f90c80669e0 2024-01-24T15:13:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cf4e5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c8153fc0 con 0x7f90c8155b00 2024-01-24T15:13:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cf4e5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c81542a0 con 0x7f90c81563d0 2024-01-24T15:13:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cd281700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90c80669e0 0x7f90c80c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cd281700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90c80669e0 0x7f90c80c0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51628/0 (socket says 172.21.15.29:51628) 2024-01-24T15:13:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cd281700 1 -- 172.21.15.29:0/2994646393 learned_addr learned my addr 172.21.15.29:0/2994646393 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cda82700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c81563d0 0x7f90c80acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:34.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cd281700 1 -- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c81563d0 msgr2=0x7f90c80acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cd281700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c81563d0 0x7f90c80acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cd281700 1 -- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c8155b00 msgr2=0x7f90c813aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cd281700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c8155b00 0x7f90c813aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cd281700 1 -- 172.21.15.29:0/2994646393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c4009370 con 0x7f90c80669e0 2024-01-24T15:13:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cd281700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90c80669e0 0x7f90c80c0070 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f90bc00d890 tx=0x7f90bc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90b97fa700 1 -- 172.21.15.29:0/2994646393 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc0125f0 con 0x7f90c80669e0 2024-01-24T15:13:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90b97fa700 1 -- 172.21.15.29:0/2994646393 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc012750 con 0x7f90c80669e0 2024-01-24T15:13:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90cf4e5700 1 -- 172.21.15.29:0/2994646393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c8154580 con 0x7f90c80669e0 2024-01-24T15:13:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.663+0000 7f90b97fa700 1 -- 172.21.15.29:0/2994646393 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc012ad0 con 0x7f90c80669e0 2024-01-24T15:13:34.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.664+0000 7f90cf4e5700 1 -- 172.21.15.29:0/2994646393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c80c0560 con 0x7f90c80669e0 2024-01-24T15:13:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.665+0000 7f90cf4e5700 1 -- 172.21.15.29:0/2994646393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c80475e0 con 0x7f90c80669e0 2024-01-24T15:13:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.665+0000 7f90b97fa700 1 -- 172.21.15.29:0/2994646393 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f90bc013480 con 0x7f90c80669e0 2024-01-24T15:13:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.665+0000 7f90b97fa700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f90ac05f270 0x7f90ac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.665+0000 7f90b97fa700 1 -- 172.21.15.29:0/2994646393 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6334+0+0 (secure 0 0 0) 0x7f90bc07a2a0 con 0x7f90c80669e0 2024-01-24T15:13:34.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.666+0000 7f90cca80700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f90ac05f270 0x7f90ac061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:34.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.666+0000 7f90cca80700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f90ac05f270 0x7f90ac061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f90c4006fd0 tx=0x7f90c4017000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:34.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.670+0000 7f90b97fa700 1 -- 172.21.15.29:0/2994646393 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f90bc04b2c0 con 0x7f90c80669e0 2024-01-24T15:13:34.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.853+0000 7f90cf4e5700 1 -- 172.21.15.29:0/2994646393 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90c8153e20 con 0x7f90c80669e0 2024-01-24T15:13:34.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.854+0000 7f90b97fa700 1 -- 172.21.15.29:0/2994646393 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+15037 (secure 0 0 0) 0x7f90c8153e20 con 0x7f90c80669e0 2024-01-24T15:13:34.856 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:34.856 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":174,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:13:34.084178+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:13:34.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.856+0000 7f90cf4e5700 1 -- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f90ac05f270 msgr2=0x7f90ac061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.856+0000 7f90cf4e5700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f90ac05f270 0x7f90ac061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f90c4006fd0 tx=0x7f90c4017000).stop 2024-01-24T15:13:34.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.856+0000 7f90cf4e5700 1 -- 172.21.15.29:0/2994646393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90c80669e0 msgr2=0x7f90c80c0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:34.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.856+0000 7f90cf4e5700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90c80669e0 0x7f90c80c0070 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f90bc00d890 tx=0x7f90bc008040).stop 2024-01-24T15:13:34.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.856+0000 7f90cf4e5700 1 -- 172.21.15.29:0/2994646393 shutdown_connections 2024-01-24T15:13:34.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.856+0000 7f90cf4e5700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c81563d0 0x7f90c80acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.856+0000 7f90cf4e5700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f90ac05f270 0x7f90ac061730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.856+0000 7f90cf4e5700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90c80669e0 0x7f90c80c0070 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:34.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.856+0000 7f90cf4e5700 1 --2- 172.21.15.29:0/2994646393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c8155b00 0x7f90c813aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:34.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.856+0000 7f90cf4e5700 1 -- 172.21.15.29:0/2994646393 >> 172.21.15.29:0/2994646393 conn(0x7f90c80c8bb0 msgr2=0x7f90c80be710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:34.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.856+0000 7f90cf4e5700 1 -- 172.21.15.29:0/2994646393 shutdown_connections 2024-01-24T15:13:34.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:34.856+0000 7f90cf4e5700 1 -- 172.21.15.29:0/2994646393 wait complete. 2024-01-24T15:13:34.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T15:13:34.873 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T15:13:35.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.005+0000 7f06d2cb8700 1 Processor -- start 2024-01-24T15:13:35.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.005+0000 7f06d2cb8700 1 -- start start 2024-01-24T15:13:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d2cb8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f06cc134ea0 0x7f06cc0c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d2cb8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06cc0c36c0 0x7f06cc0664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d2cb8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06cc0686a0 con 0x7f06cc0669b0 2024-01-24T15:13:35.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d2cb8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06cc066e30 con 0x7f06cc134ea0 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d2cb8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc068b30 con 0x7f06cc0c36c0 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d0a54700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f06cc134ea0 0x7f06cc0c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d0a54700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f06cc134ea0 0x7f06cc0c31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51636/0 (socket says 172.21.15.29:51636) 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d0a54700 1 -- 172.21.15.29:0/1767187006 learned_addr learned my addr 172.21.15.29:0/1767187006 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d1255700 1 --1- 172.21.15.29:0/1767187006 >> v1:172.21.15.29:6789/0 conn(0x7f06cc0669b0 0x7f06cc1559f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41102/0 (socket says 172.21.15.29:41102) 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06cbfff700 1 --2- 172.21.15.29:0/1767187006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06cc0c36c0 0x7f06cc0664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d0a54700 1 -- 172.21.15.29:0/1767187006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06cc0c36c0 msgr2=0x7f06cc0664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d0a54700 1 --2- 172.21.15.29:0/1767187006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06cc0c36c0 0x7f06cc0664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d0a54700 1 -- 172.21.15.29:0/1767187006 >> v1:172.21.15.29:6789/0 conn(0x7f06cc0669b0 legacy=0x7f06cc1559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d0a54700 1 -- 172.21.15.29:0/1767187006 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc067a90 con 0x7f06cc134ea0 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06d0a54700 1 --2- 172.21.15.29:0/1767187006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06cc134ea0 0x7f06cc0c31d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f06c0006fd0 tx=0x7f06c00090f0).ready entity=mon.0 client_cookie=53c694ced591e48 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.006+0000 7f06cb7fe700 1 -- 172.21.15.29:0/1767187006 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c000b600 con 0x7f06cc134ea0 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.007+0000 7f06cb7fe700 1 -- 172.21.15.29:0/1767187006 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c000b760 con 0x7f06cc134ea0 2024-01-24T15:13:35.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.007+0000 7f06cb7fe700 1 -- 172.21.15.29:0/1767187006 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c000bae0 con 0x7f06cc134ea0 2024-01-24T15:13:35.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.007+0000 7f06d2cb8700 1 -- 172.21.15.29:0/1767187006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06cc134ea0 msgr2=0x7f06cc0c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.007+0000 7f06d2cb8700 1 --2- 172.21.15.29:0/1767187006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06cc134ea0 0x7f06cc0c31d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f06c0006fd0 tx=0x7f06c00090f0).stop 2024-01-24T15:13:35.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.007+0000 7f06d2cb8700 1 -- 172.21.15.29:0/1767187006 shutdown_connections 2024-01-24T15:13:35.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.007+0000 7f06d2cb8700 1 --2- 172.21.15.29:0/1767187006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06cc134ea0 0x7f06cc0c31d0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.007+0000 7f06d2cb8700 1 --2- 172.21.15.29:0/1767187006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06cc0c36c0 0x7f06cc0664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.007+0000 7f06d2cb8700 1 -- 172.21.15.29:0/1767187006 >> 172.21.15.29:0/1767187006 conn(0x7f06cc0c8bd0 msgr2=0x7f06cc0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:35.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.007+0000 7f06d2cb8700 1 -- 172.21.15.29:0/1767187006 shutdown_connections 2024-01-24T15:13:35.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.007+0000 7f06d2cb8700 1 -- 172.21.15.29:0/1767187006 wait complete. 2024-01-24T15:13:35.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.008+0000 7f06d2cb8700 1 Processor -- start 2024-01-24T15:13:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.009+0000 7f06d2cb8700 1 -- start start 2024-01-24T15:13:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.009+0000 7f06d2cb8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06cc0669b0 0x7f06cc134520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.009+0000 7f06d2cb8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06cc0c36c0 0x7f06cc136d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.009+0000 7f06d2cb8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06cc134ea0 0x7f06cc137220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.009+0000 7f06d2cb8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc137890 con 0x7f06cc134ea0 2024-01-24T15:13:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.009+0000 7f06d2cb8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc137b70 con 0x7f06cc0669b0 2024-01-24T15:13:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.009+0000 7f06d2cb8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc137e50 con 0x7f06cc0c36c0 2024-01-24T15:13:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06d1255700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06cc134ea0 0x7f06cc137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06d1255700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06cc134ea0 0x7f06cc137220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51648/0 (socket says 172.21.15.29:51648) 2024-01-24T15:13:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06d1255700 1 -- 172.21.15.29:0/2387503695 learned_addr learned my addr 172.21.15.29:0/2387503695 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06cbfff700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06cc0c36c0 0x7f06cc136d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06d0a54700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06cc0669b0 0x7f06cc134520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06d1255700 1 -- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06cc0c36c0 msgr2=0x7f06cc136d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06d1255700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06cc0c36c0 0x7f06cc136d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06d1255700 1 -- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06cc0669b0 msgr2=0x7f06cc134520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06d1255700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06cc0669b0 0x7f06cc134520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06d1255700 1 -- 172.21.15.29:0/2387503695 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0009370 con 0x7f06cc134ea0 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06d1255700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06cc134ea0 0x7f06cc137220 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f06c400c7c0 tx=0x7f06c4001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06c8ff9700 1 -- 172.21.15.29:0/2387503695 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4005c70 con 0x7f06cc134ea0 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06c8ff9700 1 -- 172.21.15.29:0/2387503695 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c4005dd0 con 0x7f06cc134ea0 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06c8ff9700 1 -- 172.21.15.29:0/2387503695 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c40072e0 con 0x7f06cc134ea0 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06d2cb8700 1 -- 172.21.15.29:0/2387503695 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06cc138130 con 0x7f06cc134ea0 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.010+0000 7f06d2cb8700 1 -- 172.21.15.29:0/2387503695 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc135770 con 0x7f06cc134ea0 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.011+0000 7f06d2cb8700 1 -- 172.21.15.29:0/2387503695 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06cc14cb10 con 0x7f06cc134ea0 2024-01-24T15:13:35.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.012+0000 7f06c8ff9700 1 -- 172.21.15.29:0/2387503695 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f06c40076d0 con 0x7f06cc134ea0 2024-01-24T15:13:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.012+0000 7f06c8ff9700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f06b005f270 0x7f06b0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.012+0000 7f06c8ff9700 1 -- 172.21.15.29:0/2387503695 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6334+0+0 (secure 0 0 0) 0x7f06c4079de0 con 0x7f06cc134ea0 2024-01-24T15:13:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.012+0000 7f06d0a54700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f06b005f270 0x7f06b0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.013+0000 7f06d0a54700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f06b005f270 0x7f06b0061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f06c0009130 tx=0x7f06c0017000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:35.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.017+0000 7f06c8ff9700 1 -- 172.21.15.29:0/2387503695 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f06c404ada0 con 0x7f06cc134ea0 2024-01-24T15:13:35.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.199+0000 7f06d2cb8700 1 -- 172.21.15.29:0/2387503695 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f06cc0475e0 con 0x7f06cc134ea0 2024-01-24T15:13:35.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.200+0000 7f06c8ff9700 1 -- 172.21.15.29:0/2387503695 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v174) v1 ==== 110+0+10 (secure 0 0 0) 0x7f06c4001630 con 0x7f06cc134ea0 2024-01-24T15:13:35.202 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T15:13:35.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.202+0000 7f06d2cb8700 1 -- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f06b005f270 msgr2=0x7f06b0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.202+0000 7f06d2cb8700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f06b005f270 0x7f06b0061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f06c0009130 tx=0x7f06c0017000).stop 2024-01-24T15:13:35.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.202+0000 7f06d2cb8700 1 -- 172.21.15.29:0/2387503695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06cc134ea0 msgr2=0x7f06cc137220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.202+0000 7f06d2cb8700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06cc134ea0 0x7f06cc137220 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f06c400c7c0 tx=0x7f06c4001510).stop 2024-01-24T15:13:35.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.202+0000 7f06d2cb8700 1 -- 172.21.15.29:0/2387503695 shutdown_connections 2024-01-24T15:13:35.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.202+0000 7f06d2cb8700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f06b005f270 0x7f06b0061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.202+0000 7f06d2cb8700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06cc134ea0 0x7f06cc137220 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.202+0000 7f06d2cb8700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06cc0669b0 0x7f06cc134520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.202+0000 7f06d2cb8700 1 --2- 172.21.15.29:0/2387503695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06cc0c36c0 0x7f06cc136d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.202+0000 7f06d2cb8700 1 -- 172.21.15.29:0/2387503695 >> 172.21.15.29:0/2387503695 conn(0x7f06cc0c8bd0 msgr2=0x7f06cc0bdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:35.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.202+0000 7f06d2cb8700 1 -- 172.21.15.29:0/2387503695 shutdown_connections 2024-01-24T15:13:35.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.202+0000 7f06d2cb8700 1 -- 172.21.15.29:0/2387503695 wait complete. 2024-01-24T15:13:35.218 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T15:13:35.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.348+0000 7ffa76a47700 1 Processor -- start 2024-01-24T15:13:35.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.348+0000 7ffa76a47700 1 -- start start 2024-01-24T15:13:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.349+0000 7ffa76a47700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa7013a5e0 0x7ffa700639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.349+0000 7ffa76a47700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa700679f0 con 0x7ffa70063eb0 2024-01-24T15:13:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.349+0000 7ffa76a47700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa70067cd0 con 0x7ffa70066bd0 2024-01-24T15:13:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.349+0000 7ffa76a47700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa70067fb0 con 0x7ffa7013a5e0 2024-01-24T15:13:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.350+0000 7ffa6ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa7013a5e0 0x7ffa700639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.350+0000 7ffa6ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffa7013a5e0 0x7ffa700639c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51662/0 (socket says 172.21.15.29:51662) 2024-01-24T15:13:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.350+0000 7ffa6ffff700 1 -- 172.21.15.29:0/1589518793 learned_addr learned my addr 172.21.15.29:0/1589518793 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.350+0000 7ffa6ffff700 1 -- 172.21.15.29:0/1589518793 >> v1:172.21.15.163:6789/0 conn(0x7ffa70066bd0 legacy=0x7ffa700bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.350+0000 7ffa6ffff700 1 -- 172.21.15.29:0/1589518793 >> v1:172.21.15.29:6789/0 conn(0x7ffa70063eb0 legacy=0x7ffa700664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.350+0000 7ffa6ffff700 1 -- 172.21.15.29:0/1589518793 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa70068290 con 0x7ffa7013a5e0 2024-01-24T15:13:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.350+0000 7ffa6ffff700 1 --2- 172.21.15.29:0/1589518793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa7013a5e0 0x7ffa700639c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7ffa60006fd0 tx=0x7ffa600090f0).ready entity=mon.0 client_cookie=4e25395ba31dbdb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.350+0000 7ffa6effd700 1 -- 172.21.15.29:0/1589518793 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000b600 con 0x7ffa7013a5e0 2024-01-24T15:13:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.350+0000 7ffa6effd700 1 -- 172.21.15.29:0/1589518793 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6000b760 con 0x7ffa7013a5e0 2024-01-24T15:13:35.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.350+0000 7ffa6effd700 1 -- 172.21.15.29:0/1589518793 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000bae0 con 0x7ffa7013a5e0 2024-01-24T15:13:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.351+0000 7ffa76a47700 1 -- 172.21.15.29:0/1589518793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa7013a5e0 msgr2=0x7ffa700639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.351+0000 7ffa76a47700 1 --2- 172.21.15.29:0/1589518793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa7013a5e0 0x7ffa700639c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7ffa60006fd0 tx=0x7ffa600090f0).stop 2024-01-24T15:13:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.351+0000 7ffa76a47700 1 -- 172.21.15.29:0/1589518793 shutdown_connections 2024-01-24T15:13:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.351+0000 7ffa76a47700 1 --2- 172.21.15.29:0/1589518793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa7013a5e0 0x7ffa700639c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.351+0000 7ffa76a47700 1 -- 172.21.15.29:0/1589518793 >> 172.21.15.29:0/1589518793 conn(0x7ffa700c8bd0 msgr2=0x7ffa700c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.351+0000 7ffa76a47700 1 -- 172.21.15.29:0/1589518793 shutdown_connections 2024-01-24T15:13:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.351+0000 7ffa76a47700 1 -- 172.21.15.29:0/1589518793 wait complete. 2024-01-24T15:13:35.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.352+0000 7ffa76a47700 1 Processor -- start 2024-01-24T15:13:35.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.353+0000 7ffa76a47700 1 -- start start 2024-01-24T15:13:35.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.353+0000 7ffa76a47700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa70063eb0 0x7ffa70164d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.353+0000 7ffa76a47700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa70066bd0 0x7ffa700a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.353+0000 7ffa76a47700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa7013a5e0 0x7ffa700a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.353+0000 7ffa76a47700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700a5260 con 0x7ffa7013a5e0 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.353+0000 7ffa76a47700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7013bde0 con 0x7ffa70063eb0 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.353+0000 7ffa76a47700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7013c0c0 con 0x7ffa70066bd0 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa74fe4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa7013a5e0 0x7ffa700a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa74fe4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa7013a5e0 0x7ffa700a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51674/0 (socket says 172.21.15.29:51674) 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa74fe4700 1 -- 172.21.15.29:0/828382144 learned_addr learned my addr 172.21.15.29:0/828382144 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa6ffff700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa70063eb0 0x7ffa70164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa6f7fe700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa70066bd0 0x7ffa700a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa74fe4700 1 -- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa70066bd0 msgr2=0x7ffa700a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa74fe4700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa70066bd0 0x7ffa700a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa74fe4700 1 -- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa70063eb0 msgr2=0x7ffa70164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa74fe4700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa70063eb0 0x7ffa70164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa74fe4700 1 -- 172.21.15.29:0/828382144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa60009370 con 0x7ffa7013a5e0 2024-01-24T15:13:35.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa74fe4700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa7013a5e0 0x7ffa700a4d70 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7ffa6800c810 tx=0x7ffa68002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:35.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa67fff700 1 -- 172.21.15.29:0/828382144 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68010510 con 0x7ffa7013a5e0 2024-01-24T15:13:35.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa67fff700 1 -- 172.21.15.29:0/828382144 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa680106b0 con 0x7ffa7013a5e0 2024-01-24T15:13:35.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa76a47700 1 -- 172.21.15.29:0/828382144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa700a7940 con 0x7ffa7013a5e0 2024-01-24T15:13:35.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa67fff700 1 -- 172.21.15.29:0/828382144 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68010a70 con 0x7ffa7013a5e0 2024-01-24T15:13:35.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.354+0000 7ffa76a47700 1 -- 172.21.15.29:0/828382144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa700a7f00 con 0x7ffa7013a5e0 2024-01-24T15:13:35.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.356+0000 7ffa76a47700 1 -- 172.21.15.29:0/828382144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7013c270 con 0x7ffa7013a5e0 2024-01-24T15:13:35.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.356+0000 7ffa67fff700 1 -- 172.21.15.29:0/828382144 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ffa68012440 con 0x7ffa7013a5e0 2024-01-24T15:13:35.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.356+0000 7ffa67fff700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffa5005f240 0x7ffa50061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.356+0000 7ffa67fff700 1 -- 172.21.15.29:0/828382144 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6334+0+0 (secure 0 0 0) 0x7ffa68079d30 con 0x7ffa7013a5e0 2024-01-24T15:13:35.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.357+0000 7ffa6ffff700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffa5005f240 0x7ffa50061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.357+0000 7ffa6ffff700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffa5005f240 0x7ffa50061700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7ffa60009130 tx=0x7ffa60017000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:35.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.361+0000 7ffa67fff700 1 -- 172.21.15.29:0/828382144 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ffa68048b00 con 0x7ffa7013a5e0 2024-01-24T15:13:35.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.540+0000 7ffa76a47700 1 -- 172.21.15.29:0/828382144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffa700475e0 con 0x7ffa7013a5e0 2024-01-24T15:13:35.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.540+0000 7ffa67fff700 1 -- 172.21.15.29:0/828382144 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v174) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffa7013c270 con 0x7ffa7013a5e0 2024-01-24T15:13:35.542 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T15:13:35.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.542+0000 7ffa76a47700 1 -- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffa5005f240 msgr2=0x7ffa50061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.542+0000 7ffa76a47700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffa5005f240 0x7ffa50061700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7ffa60009130 tx=0x7ffa60017000).stop 2024-01-24T15:13:35.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.542+0000 7ffa76a47700 1 -- 172.21.15.29:0/828382144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa7013a5e0 msgr2=0x7ffa700a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.542+0000 7ffa76a47700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa7013a5e0 0x7ffa700a4d70 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7ffa6800c810 tx=0x7ffa68002560).stop 2024-01-24T15:13:35.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.542+0000 7ffa76a47700 1 -- 172.21.15.29:0/828382144 shutdown_connections 2024-01-24T15:13:35.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.542+0000 7ffa76a47700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ffa5005f240 0x7ffa50061700 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.542+0000 7ffa76a47700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa7013a5e0 0x7ffa700a4d70 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.542+0000 7ffa76a47700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa70063eb0 0x7ffa70164d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.542+0000 7ffa76a47700 1 --2- 172.21.15.29:0/828382144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa70066bd0 0x7ffa700a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.542+0000 7ffa76a47700 1 -- 172.21.15.29:0/828382144 >> 172.21.15.29:0/828382144 conn(0x7ffa700c8bd0 msgr2=0x7ffa70156bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:35.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.542+0000 7ffa76a47700 1 -- 172.21.15.29:0/828382144 shutdown_connections 2024-01-24T15:13:35.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.543+0000 7ffa76a47700 1 -- 172.21.15.29:0/828382144 wait complete. 2024-01-24T15:13:35.558 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-24T15:13:35.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.687+0000 7f3cb7092700 1 Processor -- start 2024-01-24T15:13:35.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.687+0000 7f3cb7092700 1 -- start start 2024-01-24T15:13:35.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb7092700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3cb0150cb0 0x7f3cb0069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb7092700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cb0065e50 0x7f3cb0154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb7092700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3cb0154e40 0x7f3cb0151090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb7092700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb00c2c70 con 0x7f3cb0154e40 2024-01-24T15:13:35.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb7092700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb006a3a0 con 0x7f3cb0150cb0 2024-01-24T15:13:35.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb7092700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01552d0 con 0x7f3cb0065e50 2024-01-24T15:13:35.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb562f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3cb0154e40 0x7f3cb0151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb562f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3cb0154e40 0x7f3cb0151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51684/0 (socket says 172.21.15.29:51684) 2024-01-24T15:13:35.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb562f700 1 -- 172.21.15.29:0/3028215097 learned_addr learned my addr 172.21.15.29:0/3028215097 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:35.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3caffff700 1 --2- 172.21.15.29:0/3028215097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cb0065e50 0x7f3cb0154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb4e2e700 1 --2- 172.21.15.29:0/3028215097 >> v2:172.21.15.163:3300/0 conn(0x7f3cb0150cb0 0x7f3cb0069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb562f700 1 -- 172.21.15.29:0/3028215097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cb0065e50 msgr2=0x7f3cb0154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:35.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb562f700 1 --2- 172.21.15.29:0/3028215097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cb0065e50 0x7f3cb0154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb562f700 1 -- 172.21.15.29:0/3028215097 >> v2:172.21.15.163:3300/0 conn(0x7f3cb0150cb0 msgr2=0x7f3cb0069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb562f700 1 --2- 172.21.15.29:0/3028215097 >> v2:172.21.15.163:3300/0 conn(0x7f3cb0150cb0 0x7f3cb0069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb562f700 1 -- 172.21.15.29:0/3028215097 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb0069820 con 0x7f3cb0154e40 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb4e2e700 1 --2- 172.21.15.29:0/3028215097 >> v2:172.21.15.163:3300/0 conn(0x7f3cb0150cb0 0x7f3cb0069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3caffff700 1 --2- 172.21.15.29:0/3028215097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cb0065e50 0x7f3cb0154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.688+0000 7f3cb562f700 1 --2- 172.21.15.29:0/3028215097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cb0154e40 0x7f3cb0151090 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f3cb006a8e0 tx=0x7f3ca8009840).ready entity=mon.0 client_cookie=5f05ccab6f6b9c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.689+0000 7f3caf7fe700 1 -- 172.21.15.29:0/3028215097 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca800e070 con 0x7f3cb0154e40 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.689+0000 7f3caf7fe700 1 -- 172.21.15.29:0/3028215097 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca800ba50 con 0x7f3cb0154e40 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.689+0000 7f3caf7fe700 1 -- 172.21.15.29:0/3028215097 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca800bdd0 con 0x7f3cb0154e40 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.689+0000 7f3cb7092700 1 -- 172.21.15.29:0/3028215097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cb0154e40 msgr2=0x7f3cb0151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.689+0000 7f3cb7092700 1 --2- 172.21.15.29:0/3028215097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cb0154e40 0x7f3cb0151090 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f3cb006a8e0 tx=0x7f3ca8009840).stop 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.689+0000 7f3cb7092700 1 -- 172.21.15.29:0/3028215097 shutdown_connections 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.689+0000 7f3cb7092700 1 --2- 172.21.15.29:0/3028215097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cb0154e40 0x7f3cb0151090 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.689+0000 7f3cb7092700 1 --2- 172.21.15.29:0/3028215097 >> v2:172.21.15.163:3300/0 conn(0x7f3cb0150cb0 0x7f3cb0069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.689+0000 7f3cb7092700 1 --2- 172.21.15.29:0/3028215097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cb0065e50 0x7f3cb0154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.689+0000 7f3cb7092700 1 -- 172.21.15.29:0/3028215097 >> 172.21.15.29:0/3028215097 conn(0x7f3cb00c85c0 msgr2=0x7f3cb00c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:35.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.689+0000 7f3cb7092700 1 -- 172.21.15.29:0/3028215097 shutdown_connections 2024-01-24T15:13:35.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.689+0000 7f3cb7092700 1 -- 172.21.15.29:0/3028215097 wait complete. 2024-01-24T15:13:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.690+0000 7f3cb7092700 1 Processor -- start 2024-01-24T15:13:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.691+0000 7f3cb7092700 1 -- start start 2024-01-24T15:13:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.691+0000 7f3cb7092700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cb0065e50 0x7f3cb00c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.691+0000 7f3cb7092700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cb0150cb0 0x7f3cb00c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.691+0000 7f3cb7092700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cb0154e40 0x7f3cb0153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.691+0000 7f3cb7092700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00c6ad0 con 0x7f3cb0154e40 2024-01-24T15:13:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.691+0000 7f3cb7092700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00bef00 con 0x7f3cb0065e50 2024-01-24T15:13:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.691+0000 7f3cb7092700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00bf1e0 con 0x7f3cb0150cb0 2024-01-24T15:13:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb562f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cb0154e40 0x7f3cb0153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb562f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cb0154e40 0x7f3cb0153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51700/0 (socket says 172.21.15.29:51700) 2024-01-24T15:13:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb562f700 1 -- 172.21.15.29:0/1000886922 learned_addr learned my addr 172.21.15.29:0/1000886922 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3caffff700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cb0150cb0 0x7f3cb00c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb4e2e700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cb0065e50 0x7f3cb00c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb562f700 1 -- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cb0150cb0 msgr2=0x7f3cb00c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb562f700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cb0150cb0 0x7f3cb00c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb562f700 1 -- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cb0065e50 msgr2=0x7f3cb00c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb562f700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cb0065e50 0x7f3cb00c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb562f700 1 -- 172.21.15.29:0/1000886922 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca8009ae0 con 0x7f3cb0154e40 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb4e2e700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cb0065e50 0x7f3cb00c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb562f700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cb0154e40 0x7f3cb0153500 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f3ca8009fd0 tx=0x7f3ca8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cacff9700 1 -- 172.21.15.29:0/1000886922 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80032f0 con 0x7f3cb0154e40 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cacff9700 1 -- 172.21.15.29:0/1000886922 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca800bdd0 con 0x7f3cb0154e40 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb7092700 1 -- 172.21.15.29:0/1000886922 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb00bf4c0 con 0x7f3cb0154e40 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cacff9700 1 -- 172.21.15.29:0/1000886922 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca801d690 con 0x7f3cb0154e40 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.692+0000 7f3cb7092700 1 -- 172.21.15.29:0/1000886922 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb00bfa80 con 0x7f3cb0154e40 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.693+0000 7f3cb7092700 1 -- 172.21.15.29:0/1000886922 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb00bfdf0 con 0x7f3cb0154e40 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.694+0000 7f3cacff9700 1 -- 172.21.15.29:0/1000886922 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3ca8003040 con 0x7f3cb0154e40 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.694+0000 7f3cacff9700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3c9405f270 0x7f3c94061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:35.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.694+0000 7f3cacff9700 1 -- 172.21.15.29:0/1000886922 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6334+0+0 (secure 0 0 0) 0x7f3ca807b700 con 0x7f3cb0154e40 2024-01-24T15:13:35.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.695+0000 7f3cb4e2e700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3c9405f270 0x7f3c94061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:35.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.695+0000 7f3cb4e2e700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3c9405f270 0x7f3c94061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f3ca4006e00 tx=0x7f3ca4008040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:35.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.699+0000 7f3cacff9700 1 -- 172.21.15.29:0/1000886922 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3ca804c780 con 0x7f3cb0154e40 2024-01-24T15:13:35.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.880+0000 7f3cb7092700 1 -- 172.21.15.29:0/1000886922 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3cb00475e0 con 0x7f3cb0154e40 2024-01-24T15:13:35.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.880+0000 7f3cacff9700 1 -- 172.21.15.29:0/1000886922 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v174) v1 ==== 84+0+143 (secure 0 0 0) 0x7f3ca8022020 con 0x7f3cb0154e40 2024-01-24T15:13:35.882 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:35.882 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T15:13:35.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:35.883 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-01-24T15:13:35.883 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "device_health_metrics" 2024-01-24T15:13:35.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:13:35.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:35.883 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 2, 2024-01-24T15:13:35.883 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "rbd" 2024-01-24T15:13:35.883 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:35.883 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T15:13:35.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.882+0000 7f3cb7092700 1 -- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3c9405f270 msgr2=0x7f3c94061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.882+0000 7f3cb7092700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3c9405f270 0x7f3c94061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f3ca4006e00 tx=0x7f3ca4008040).stop 2024-01-24T15:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.882+0000 7f3cb7092700 1 -- 172.21.15.29:0/1000886922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cb0154e40 msgr2=0x7f3cb0153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.882+0000 7f3cb7092700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cb0154e40 0x7f3cb0153500 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f3ca8009fd0 tx=0x7f3ca8009840).stop 2024-01-24T15:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.882+0000 7f3cb7092700 1 -- 172.21.15.29:0/1000886922 shutdown_connections 2024-01-24T15:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.882+0000 7f3cb7092700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3c9405f270 0x7f3c94061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.882+0000 7f3cb7092700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cb0154e40 0x7f3cb0153500 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.882+0000 7f3cb7092700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cb0065e50 0x7f3cb00c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.882+0000 7f3cb7092700 1 --2- 172.21.15.29:0/1000886922 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cb0150cb0 0x7f3cb00c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.882+0000 7f3cb7092700 1 -- 172.21.15.29:0/1000886922 >> 172.21.15.29:0/1000886922 conn(0x7f3cb00c85c0 msgr2=0x7f3cb00c2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.882+0000 7f3cb7092700 1 -- 172.21.15.29:0/1000886922 shutdown_connections 2024-01-24T15:13:35.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:35.882+0000 7f3cb7092700 1 -- 172.21.15.29:0/1000886922 wait complete. 2024-01-24T15:13:35.898 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-24T15:13:35.898 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2024-01-24T15:13:36.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.031+0000 7fd1d1a8c700 1 Processor -- start 2024-01-24T15:13:36.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.031+0000 7fd1d1a8c700 1 -- start start 2024-01-24T15:13:36.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.032+0000 7fd1d1a8c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc150970 0x7fd1cc150d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:36.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.032+0000 7fd1d1a8c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd1cc069ee0 0x7fd1cc154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:36.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.032+0000 7fd1d1a8c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc069a50 con 0x7fd1cc06a340 2024-01-24T15:13:36.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.032+0000 7fd1d1a8c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1cc0c2b20 con 0x7fd1cc069ee0 2024-01-24T15:13:36.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.032+0000 7fd1d1a8c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc151240 con 0x7fd1cc150970 2024-01-24T15:13:36.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1c2ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd1cc069ee0 0x7fd1cc154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:36.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1cb7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc150970 0x7fd1cc150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:36.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1c2ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd1cc069ee0 0x7fd1cc154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:51590/0 (socket says 172.21.15.29:51590) 2024-01-24T15:13:36.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1cb7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc150970 0x7fd1cc150d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:35214/0 (socket says 172.21.15.29:35214) 2024-01-24T15:13:36.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1cbfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd1cc06a340 0x7fd1cc0c4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:53306/0 (socket says 172.21.15.29:53306) 2024-01-24T15:13:36.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1c2ffd700 1 -- 172.21.15.29:0/1819128322 learned_addr learned my addr 172.21.15.29:0/1819128322 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:36.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1c2ffd700 1 -- 172.21.15.29:0/1819128322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc150970 msgr2=0x7fd1cc150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:36.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1c2ffd700 1 --2- 172.21.15.29:0/1819128322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc150970 0x7fd1cc150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:36.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1c2ffd700 1 -- 172.21.15.29:0/1819128322 >> v1:172.21.15.163:6789/0 conn(0x7fd1cc06a340 legacy=0x7fd1cc0c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:36.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1caffd700 1 -- 172.21.15.29:0/1819128322 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003911257 0 0) 0x7fd1cc069a50 con 0x7fd1cc06a340 2024-01-24T15:13:36.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1c2ffd700 1 -- 172.21.15.29:0/1819128322 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc155510 con 0x7fd1cc069ee0 2024-01-24T15:13:36.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1cb7fe700 1 --2- 172.21.15.29:0/1819128322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc150970 0x7fd1cc150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:36.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1c2ffd700 1 --2- 172.21.15.29:0/1819128322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1cc069ee0 0x7fd1cc154950 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fd1b4006fd0 tx=0x7fd1b400b3b0).ready entity=mon.1 client_cookie=8feebc8fc58290cf server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:36.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1caffd700 1 -- 172.21.15.29:0/1819128322 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b400da70 con 0x7fd1cc069ee0 2024-01-24T15:13:36.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1caffd700 1 -- 172.21.15.29:0/1819128322 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b400dbd0 con 0x7fd1cc069ee0 2024-01-24T15:13:36.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.033+0000 7fd1caffd700 1 -- 172.21.15.29:0/1819128322 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4010470 con 0x7fd1cc069ee0 2024-01-24T15:13:36.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.034+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/1819128322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1cc069ee0 msgr2=0x7fd1cc154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:36.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.034+0000 7fd1d1a8c700 1 --2- 172.21.15.29:0/1819128322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1cc069ee0 0x7fd1cc154950 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fd1b4006fd0 tx=0x7fd1b400b3b0).stop 2024-01-24T15:13:36.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.034+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/1819128322 shutdown_connections 2024-01-24T15:13:36.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.034+0000 7fd1d1a8c700 1 --2- 172.21.15.29:0/1819128322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc150970 0x7fd1cc150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:36.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.034+0000 7fd1d1a8c700 1 --2- 172.21.15.29:0/1819128322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1cc069ee0 0x7fd1cc154950 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:36.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.034+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/1819128322 >> 172.21.15.29:0/1819128322 conn(0x7fd1cc067b20 msgr2=0x7fd1cc0638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:36.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.034+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/1819128322 shutdown_connections 2024-01-24T15:13:36.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.034+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/1819128322 wait complete. 2024-01-24T15:13:36.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.036+0000 7fd1d1a8c700 1 Processor -- start 2024-01-24T15:13:36.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.036+0000 7fd1d1a8c700 1 -- start start 2024-01-24T15:13:36.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1d1a8c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc069ee0 0x7fd1cc167db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:36.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1d1a8c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1cc06a340 0x7fd1cc0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:36.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1d1a8c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1cc150970 0x7fd1cc163f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1d1a8c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc1359d0 con 0x7fd1cc150970 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1d1a8c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc168ca0 con 0x7fd1cc06a340 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1d1a8c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc168400 con 0x7fd1cc069ee0 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1cbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1cc150970 0x7fd1cc163f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1cbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1cc150970 0x7fd1cc163f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51702/0 (socket says 172.21.15.29:51702) 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1cbfff700 1 -- 172.21.15.29:0/2321201204 learned_addr learned my addr 172.21.15.29:0/2321201204 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1c2ffd700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1cc06a340 0x7fd1cc0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1cb7fe700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc069ee0 0x7fd1cc167db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1cbfff700 1 -- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc069ee0 msgr2=0x7fd1cc167db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1cbfff700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc069ee0 0x7fd1cc167db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1cbfff700 1 -- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1cc06a340 msgr2=0x7fd1cc0a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1cbfff700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1cc06a340 0x7fd1cc0a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1cbfff700 1 -- 172.21.15.29:0/2321201204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc0092a0 con 0x7fd1cc150970 2024-01-24T15:13:36.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1cb7fe700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc069ee0 0x7fd1cc167db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.037+0000 7fd1cbfff700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1cc150970 0x7fd1cc163f80 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7fd1c400c810 tx=0x7fd1c4007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.038+0000 7fd1c3fff700 1 -- 172.21.15.29:0/2321201204 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4005c70 con 0x7fd1cc150970 2024-01-24T15:13:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.038+0000 7fd1c3fff700 1 -- 172.21.15.29:0/2321201204 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c4005dd0 con 0x7fd1cc150970 2024-01-24T15:13:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.038+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/2321201204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b400b610 con 0x7fd1cc150970 2024-01-24T15:13:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.038+0000 7fd1c3fff700 1 -- 172.21.15.29:0/2321201204 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4007340 con 0x7fd1cc150970 2024-01-24T15:13:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.038+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/2321201204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc1689c0 con 0x7fd1cc150970 2024-01-24T15:13:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.039+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/2321201204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc0475e0 con 0x7fd1cc150970 2024-01-24T15:13:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.039+0000 7fd1c3fff700 1 -- 172.21.15.29:0/2321201204 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd1c40076b0 con 0x7fd1cc150970 2024-01-24T15:13:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.039+0000 7fd1c3fff700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd1ac05f270 0x7fd1ac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.040+0000 7fd1c3fff700 1 -- 172.21.15.29:0/2321201204 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6334+0+0 (secure 0 0 0) 0x7fd1c407a490 con 0x7fd1cc150970 2024-01-24T15:13:36.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.040+0000 7fd1cb7fe700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd1ac05f270 0x7fd1ac061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:36.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.040+0000 7fd1cb7fe700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd1ac05f270 0x7fd1ac061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fd1bc009270 tx=0x7fd1bc008040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:36.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.044+0000 7fd1c3fff700 1 -- 172.21.15.29:0/2321201204 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd1c404b510 con 0x7fd1cc150970 2024-01-24T15:13:36.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:36.228+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/2321201204 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fd1cc1557f0 con 0x7fd1cc150970 2024-01-24T15:13:37.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.094+0000 7fd1c3fff700 1 -- 172.21.15.29:0/2321201204 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v175) v1 ==== 152+0+0 (secure 0 0 0) 0x7fd1c404eb30 con 0x7fd1cc150970 2024-01-24T15:13:37.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.099+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd1ac05f270 msgr2=0x7fd1ac061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:37.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.099+0000 7fd1d1a8c700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd1ac05f270 0x7fd1ac061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fd1bc009270 tx=0x7fd1bc008040).stop 2024-01-24T15:13:37.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.099+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/2321201204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1cc150970 msgr2=0x7fd1cc163f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:37.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.099+0000 7fd1d1a8c700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1cc150970 0x7fd1cc163f80 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7fd1c400c810 tx=0x7fd1c4007f30).stop 2024-01-24T15:13:37.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.100+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/2321201204 shutdown_connections 2024-01-24T15:13:37.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.101+0000 7fd1d1a8c700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd1ac05f270 0x7fd1ac061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:37.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.101+0000 7fd1d1a8c700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1cc150970 0x7fd1cc163f80 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:37.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.101+0000 7fd1d1a8c700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1cc069ee0 0x7fd1cc167db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:37.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.101+0000 7fd1d1a8c700 1 --2- 172.21.15.29:0/2321201204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1cc06a340 0x7fd1cc0a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:37.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.101+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/2321201204 >> 172.21.15.29:0/2321201204 conn(0x7fd1cc067b20 msgr2=0x7fd1cc143f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:37.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.105+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/2321201204 shutdown_connections 2024-01-24T15:13:37.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.107+0000 7fd1d1a8c700 1 -- 172.21.15.29:0/2321201204 wait complete. 2024-01-24T15:13:37.110 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-01-24T15:13:37.125 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-24T15:13:37.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.264+0000 7f44116f3700 1 Processor -- start 2024-01-24T15:13:37.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.264+0000 7f44116f3700 1 -- start start 2024-01-24T15:13:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.265+0000 7f44116f3700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f440c153fe0 0x7f440c1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.265+0000 7f44116f3700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440c0c90d0 con 0x7f440c0c5ef0 2024-01-24T15:13:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.265+0000 7f44116f3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440c0c93b0 con 0x7f440c0c63a0 2024-01-24T15:13:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.265+0000 7f44116f3700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440c0c9690 con 0x7f440c153fe0 2024-01-24T15:13:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.265+0000 7f440affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f440c153fe0 0x7f440c1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.265+0000 7f440affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f440c153fe0 0x7f440c1543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51706/0 (socket says 172.21.15.29:51706) 2024-01-24T15:13:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.265+0000 7f440affd700 1 -- 172.21.15.29:0/1013399015 learned_addr learned my addr 172.21.15.29:0/1013399015 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.266+0000 7f440affd700 1 -- 172.21.15.29:0/1013399015 >> v1:172.21.15.163:6789/0 conn(0x7f440c0c63a0 legacy=0x7f440c0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.266+0000 7f440affd700 1 -- 172.21.15.29:0/1013399015 >> v1:172.21.15.29:6789/0 conn(0x7f440c0c5ef0 legacy=0x7f440c0ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.266+0000 7f440affd700 1 -- 172.21.15.29:0/1013399015 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c0c9970 con 0x7f440c153fe0 2024-01-24T15:13:37.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.266+0000 7f440affd700 1 --2- 172.21.15.29:0/1013399015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f440c153fe0 0x7f440c1543c0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f43fc006fd0 tx=0x7f43fc0090f0).ready entity=mon.0 client_cookie=7947cde5cd1ffb22 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:37.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.266+0000 7f4409ffb700 1 -- 172.21.15.29:0/1013399015 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc00b600 con 0x7f440c153fe0 2024-01-24T15:13:37.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.266+0000 7f4409ffb700 1 -- 172.21.15.29:0/1013399015 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc00b760 con 0x7f440c153fe0 2024-01-24T15:13:37.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.266+0000 7f4409ffb700 1 -- 172.21.15.29:0/1013399015 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc00bae0 con 0x7f440c153fe0 2024-01-24T15:13:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.266+0000 7f44116f3700 1 -- 172.21.15.29:0/1013399015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f440c153fe0 msgr2=0x7f440c1543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.266+0000 7f44116f3700 1 --2- 172.21.15.29:0/1013399015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f440c153fe0 0x7f440c1543c0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f43fc006fd0 tx=0x7f43fc0090f0).stop 2024-01-24T15:13:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.267+0000 7f44116f3700 1 -- 172.21.15.29:0/1013399015 shutdown_connections 2024-01-24T15:13:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.267+0000 7f44116f3700 1 --2- 172.21.15.29:0/1013399015 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f440c153fe0 0x7f440c1543c0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.267+0000 7f44116f3700 1 -- 172.21.15.29:0/1013399015 >> 172.21.15.29:0/1013399015 conn(0x7f440c05e1f0 msgr2=0x7f440c05e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.267+0000 7f44116f3700 1 -- 172.21.15.29:0/1013399015 shutdown_connections 2024-01-24T15:13:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.267+0000 7f44116f3700 1 -- 172.21.15.29:0/1013399015 wait complete. 2024-01-24T15:13:37.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.268+0000 7f44116f3700 1 Processor -- start 2024-01-24T15:13:37.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.269+0000 7f44116f3700 1 -- start start 2024-01-24T15:13:37.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f44116f3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f440c0c5ef0 0x7f440c152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:37.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f44116f3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f440c0c63a0 0x7f440c0b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:37.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f44116f3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f440c153fe0 0x7f440c14da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:37.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f44116f3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c152fd0 con 0x7f440c0c63a0 2024-01-24T15:13:37.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f44116f3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0b0e90 con 0x7f440c0c5ef0 2024-01-24T15:13:37.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f44116f3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c150470 con 0x7f440c153fe0 2024-01-24T15:13:37.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f440a7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f440c0c63a0 0x7f440c0b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:37.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f440a7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f440c0c63a0 0x7f440c0b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51714/0 (socket says 172.21.15.29:51714) 2024-01-24T15:13:37.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f440a7fc700 1 -- 172.21.15.29:0/4055292611 learned_addr learned my addr 172.21.15.29:0/4055292611 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:37.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f440affd700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f440c0c5ef0 0x7f440c152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f440b7fe700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f440c153fe0 0x7f440c14da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f440a7fc700 1 -- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f440c153fe0 msgr2=0x7f440c14da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f440a7fc700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f440c153fe0 0x7f440c14da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f440a7fc700 1 -- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f440c0c5ef0 msgr2=0x7f440c152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f440a7fc700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f440c0c5ef0 0x7f440c152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f440a7fc700 1 -- 172.21.15.29:0/4055292611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc009370 con 0x7f440c0c63a0 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f440affd700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f440c0c5ef0 0x7f440c152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f440a7fc700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f440c0c63a0 0x7f440c0b09a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7f43f400c4f0 tx=0x7f43f4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f4402ffd700 1 -- 172.21.15.29:0/4055292611 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4007020 con 0x7f440c0c63a0 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f4402ffd700 1 -- 172.21.15.29:0/4055292611 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f4001e90 con 0x7f440c0c63a0 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f44116f3700 1 -- 172.21.15.29:0/4055292611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c1534c0 con 0x7f440c0c63a0 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.270+0000 7f4402ffd700 1 -- 172.21.15.29:0/4055292611 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40073b0 con 0x7f440c0c63a0 2024-01-24T15:13:37.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.271+0000 7f44116f3700 1 -- 172.21.15.29:0/4055292611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c14fea0 con 0x7f440c0c63a0 2024-01-24T15:13:37.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.272+0000 7f44116f3700 1 -- 172.21.15.29:0/4055292611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440c150220 con 0x7f440c0c63a0 2024-01-24T15:13:37.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.272+0000 7f4402ffd700 1 -- 172.21.15.29:0/4055292611 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f43f40068b0 con 0x7f440c0c63a0 2024-01-24T15:13:37.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.272+0000 7f4402ffd700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43ec05f270 0x7f43ec061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:37.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.272+0000 7f4402ffd700 1 -- 172.21.15.29:0/4055292611 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f43f4079f90 con 0x7f440c0c63a0 2024-01-24T15:13:37.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.273+0000 7f440affd700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43ec05f270 0x7f43ec061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:37.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.273+0000 7f440affd700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43ec05f270 0x7f43ec061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f43fc009130 tx=0x7f43fc017000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:37.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.277+0000 7f4402ffd700 1 -- 172.21.15.29:0/4055292611 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f43f404aea0 con 0x7f440c0c63a0 2024-01-24T15:13:37.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:37.460+0000 7f44116f3700 1 -- 172.21.15.29:0/4055292611 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f440c0475e0 con 0x7f440c0c63a0 2024-01-24T15:13:38.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.103+0000 7f4402ffd700 1 -- 172.21.15.29:0/4055292611 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v176) v1 ==== 170+0+0 (secure 0 0 0) 0x7f43f404e4c0 con 0x7f440c0c63a0 2024-01-24T15:13:38.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.117+0000 7f44116f3700 1 -- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43ec05f270 msgr2=0x7f43ec061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.117+0000 7f44116f3700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43ec05f270 0x7f43ec061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f43fc009130 tx=0x7f43fc017000).stop 2024-01-24T15:13:38.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.117+0000 7f44116f3700 1 -- 172.21.15.29:0/4055292611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f440c0c63a0 msgr2=0x7f440c0b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.117+0000 7f44116f3700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f440c0c63a0 0x7f440c0b09a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7f43f400c4f0 tx=0x7f43f4007e00).stop 2024-01-24T15:13:38.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.118+0000 7f44116f3700 1 -- 172.21.15.29:0/4055292611 shutdown_connections 2024-01-24T15:13:38.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.118+0000 7f44116f3700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f440c153fe0 0x7f440c14da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.118+0000 7f44116f3700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f440c0c5ef0 0x7f440c152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.118+0000 7f44116f3700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f43ec05f270 0x7f43ec061730 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0x7f43fc009130 tx=0x7f43fc017000).stop 2024-01-24T15:13:38.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.118+0000 7f44116f3700 1 --2- 172.21.15.29:0/4055292611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f440c0c63a0 0x7f440c0b09a0 secure :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0x7f43f400c4f0 tx=0x7f43f4007e00).stop 2024-01-24T15:13:38.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.118+0000 7f44116f3700 1 -- 172.21.15.29:0/4055292611 >> 172.21.15.29:0/4055292611 conn(0x7f440c05e1f0 msgr2=0x7f440c0c1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:38.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.122+0000 7f44116f3700 1 -- 172.21.15.29:0/4055292611 shutdown_connections 2024-01-24T15:13:38.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.122+0000 7f44116f3700 1 -- 172.21.15.29:0/4055292611 wait complete. 2024-01-24T15:13:38.131 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-01-24T15:13:38.148 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-24T15:13:38.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.283+0000 7f027bb95700 1 Processor -- start 2024-01-24T15:13:38.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.284+0000 7f027bb95700 1 -- start start 2024-01-24T15:13:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.284+0000 7f027bb95700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f027413a5e0 0x7f02740639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.284+0000 7f027bb95700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0274063eb0 0x7f02740664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.284+0000 7f027bb95700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f02740669b0 0x7f0274157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.284+0000 7f027bb95700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0274066e20 con 0x7f027413a5e0 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.284+0000 7f027bb95700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02740679f0 con 0x7f02740669b0 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.284+0000 7f027bb95700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0274067cd0 con 0x7f0274063eb0 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.285+0000 7f0279931700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f027413a5e0 0x7f02740639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.285+0000 7f0279931700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f027413a5e0 0x7f02740639c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51718/0 (socket says 172.21.15.29:51718) 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.285+0000 7f0279931700 1 -- 172.21.15.29:0/1537718880 learned_addr learned my addr 172.21.15.29:0/1537718880 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.285+0000 7f0279130700 1 --2- 172.21.15.29:0/1537718880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0274063eb0 0x7f02740664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.285+0000 7f027a132700 1 --2- 172.21.15.29:0/1537718880 >> v2:172.21.15.163:3300/0 conn(0x7f02740669b0 0x7f0274157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.285+0000 7f0279931700 1 -- 172.21.15.29:0/1537718880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0274063eb0 msgr2=0x7f02740664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.285+0000 7f0279931700 1 --2- 172.21.15.29:0/1537718880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0274063eb0 0x7f02740664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.285+0000 7f0279931700 1 -- 172.21.15.29:0/1537718880 >> v2:172.21.15.163:3300/0 conn(0x7f02740669b0 msgr2=0x7f0274157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:38.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.285+0000 7f0279931700 1 --2- 172.21.15.29:0/1537718880 >> v2:172.21.15.163:3300/0 conn(0x7f02740669b0 0x7f0274157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.285+0000 7f0279931700 1 -- 172.21.15.29:0/1537718880 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0274067fb0 con 0x7f027413a5e0 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.285+0000 7f0279931700 1 --2- 172.21.15.29:0/1537718880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f027413a5e0 0x7f02740639c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7f0274158d00 tx=0x7f02700090f0).ready entity=mon.0 client_cookie=531887839270d258 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.285+0000 7f027892f700 1 -- 172.21.15.29:0/1537718880 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027000f510 con 0x7f027413a5e0 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.286+0000 7f027892f700 1 -- 172.21.15.29:0/1537718880 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027000f670 con 0x7f027413a5e0 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.286+0000 7f027892f700 1 -- 172.21.15.29:0/1537718880 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027000f9f0 con 0x7f027413a5e0 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.286+0000 7f027bb95700 1 -- 172.21.15.29:0/1537718880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f027413a5e0 msgr2=0x7f02740639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.286+0000 7f027bb95700 1 --2- 172.21.15.29:0/1537718880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f027413a5e0 0x7f02740639c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7f0274158d00 tx=0x7f02700090f0).stop 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.286+0000 7f027bb95700 1 -- 172.21.15.29:0/1537718880 shutdown_connections 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.286+0000 7f027bb95700 1 --2- 172.21.15.29:0/1537718880 >> v2:172.21.15.163:3300/0 conn(0x7f02740669b0 0x7f0274157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.286+0000 7f027bb95700 1 --2- 172.21.15.29:0/1537718880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f027413a5e0 0x7f02740639c0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.286+0000 7f027bb95700 1 --2- 172.21.15.29:0/1537718880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0274063eb0 0x7f02740664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.286+0000 7f027bb95700 1 -- 172.21.15.29:0/1537718880 >> 172.21.15.29:0/1537718880 conn(0x7f02740c8bd0 msgr2=0x7f02740c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:38.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.286+0000 7f027bb95700 1 -- 172.21.15.29:0/1537718880 shutdown_connections 2024-01-24T15:13:38.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.286+0000 7f027bb95700 1 -- 172.21.15.29:0/1537718880 wait complete. 2024-01-24T15:13:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.287+0000 7f027bb95700 1 Processor -- start 2024-01-24T15:13:38.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.287+0000 7f027bb95700 1 -- start start 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.288+0000 7f027bb95700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0274063eb0 0x7f02740ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.288+0000 7f027bb95700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02740669b0 0x7f0274138120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.288+0000 7f027bb95700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f027413a5e0 0x7f0274155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.288+0000 7f027bb95700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02740ad0a0 con 0x7f02740669b0 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.288+0000 7f027bb95700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02740ab6e0 con 0x7f027413a5e0 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.288+0000 7f027bb95700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0274167ae0 con 0x7f0274063eb0 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.288+0000 7f0279130700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02740669b0 0x7f0274138120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.288+0000 7f0279130700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02740669b0 0x7f0274138120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51726/0 (socket says 172.21.15.29:51726) 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.288+0000 7f0279130700 1 -- 172.21.15.29:0/979525476 learned_addr learned my addr 172.21.15.29:0/979525476 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f0279931700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0274063eb0 0x7f02740ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f027a132700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f027413a5e0 0x7f0274155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f0279130700 1 -- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0274063eb0 msgr2=0x7f02740ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f0279130700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0274063eb0 0x7f02740ab1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f0279130700 1 -- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f027413a5e0 msgr2=0x7f0274155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f0279130700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f027413a5e0 0x7f0274155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f0279130700 1 -- 172.21.15.29:0/979525476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0270009370 con 0x7f02740669b0 2024-01-24T15:13:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f0279130700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02740669b0 0x7f0274138120 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f026800e900 tx=0x7f0268009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f0265ffb700 1 -- 172.21.15.29:0/979525476 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026800c640 con 0x7f02740669b0 2024-01-24T15:13:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f0265ffb700 1 -- 172.21.15.29:0/979525476 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026800c7a0 con 0x7f02740669b0 2024-01-24T15:13:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f027bb95700 1 -- 172.21.15.29:0/979525476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0274167dc0 con 0x7f02740669b0 2024-01-24T15:13:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f0265ffb700 1 -- 172.21.15.29:0/979525476 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026800cb20 con 0x7f02740669b0 2024-01-24T15:13:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.289+0000 7f027bb95700 1 -- 172.21.15.29:0/979525476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0274168380 con 0x7f02740669b0 2024-01-24T15:13:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.291+0000 7f027bb95700 1 -- 172.21.15.29:0/979525476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02740475e0 con 0x7f02740669b0 2024-01-24T15:13:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.291+0000 7f0265ffb700 1 -- 172.21.15.29:0/979525476 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f026801b400 con 0x7f02740669b0 2024-01-24T15:13:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.291+0000 7f0265ffb700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f025805f270 0x7f0258061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.292+0000 7f0265ffb700 1 -- 172.21.15.29:0/979525476 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 7086+0+0 (secure 0 0 0) 0x7f026807aba0 con 0x7f02740669b0 2024-01-24T15:13:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.292+0000 7f0279931700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f025805f270 0x7f0258061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.292+0000 7f0279931700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f025805f270 0x7f0258061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0270009130 tx=0x7f0270017000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:38.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.296+0000 7f0265ffb700 1 -- 172.21.15.29:0/979525476 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f026804b8d0 con 0x7f02740669b0 2024-01-24T15:13:38.524 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.522+0000 7f027bb95700 1 -- 172.21.15.29:0/979525476 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0274068290 con 0x7f02740669b0 2024-01-24T15:13:38.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.531+0000 7f0265ffb700 1 -- 172.21.15.29:0/979525476 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v70) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0268016070 con 0x7f02740669b0 2024-01-24T15:13:38.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.538+0000 7f027bb95700 1 -- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f025805f270 msgr2=0x7f0258061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.538+0000 7f027bb95700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f025805f270 0x7f0258061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0270009130 tx=0x7f0270017000).stop 2024-01-24T15:13:38.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.538+0000 7f027bb95700 1 -- 172.21.15.29:0/979525476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02740669b0 msgr2=0x7f0274138120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.538+0000 7f027bb95700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02740669b0 0x7f0274138120 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f026800e900 tx=0x7f0268009060).stop 2024-01-24T15:13:38.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.542+0000 7f027bb95700 1 -- 172.21.15.29:0/979525476 shutdown_connections 2024-01-24T15:13:38.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.542+0000 7f027bb95700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f027413a5e0 0x7f0274155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.542+0000 7f027bb95700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0274063eb0 0x7f02740ab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.542+0000 7f027bb95700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f025805f270 0x7f0258061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.542+0000 7f027bb95700 1 --2- 172.21.15.29:0/979525476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02740669b0 0x7f0274138120 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.542+0000 7f027bb95700 1 -- 172.21.15.29:0/979525476 >> 172.21.15.29:0/979525476 conn(0x7f02740c8bd0 msgr2=0x7f0274139710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:38.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.543+0000 7f027bb95700 1 -- 172.21.15.29:0/979525476 shutdown_connections 2024-01-24T15:13:38.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.543+0000 7f027bb95700 1 -- 172.21.15.29:0/979525476 wait complete. 2024-01-24T15:13:38.548 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 20 and data pool 21 2024-01-24T15:13:38.568 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:13:38.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.696+0000 7fd7a38e8700 1 Processor -- start 2024-01-24T15:13:38.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.696+0000 7fd7a38e8700 1 -- start start 2024-01-24T15:13:38.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.697+0000 7fd7a38e8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd79c155040 0x7fd79c069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.697+0000 7fd7a38e8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79c0c2ce0 con 0x7fd79c065ec0 2024-01-24T15:13:38.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.697+0000 7fd7a38e8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79c069890 con 0x7fd79c150d40 2024-01-24T15:13:38.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.697+0000 7fd7a38e8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79c069b70 con 0x7fd79c155040 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.697+0000 7fd7a1e85700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd79c065ec0 0x7fd79c0c3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41138/0 (socket says 172.21.15.29:41138) 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.697+0000 7fd7a1e85700 1 -- 172.21.15.29:0/2051603112 learned_addr learned my addr 172.21.15.29:0/2051603112 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.697+0000 7fd7a0e83700 1 --2- 172.21.15.29:0/2051603112 >> v2:172.21.15.163:3300/0 conn(0x7fd79c155040 0x7fd79c069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.697+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517684634 0 0) 0x7fd79c0c2ce0 con 0x7fd79c065ec0 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.697+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7880033f0 con 0x7fd79c065ec0 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.697+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1676996832 0 0) 0x7fd7880033f0 con 0x7fd79c065ec0 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.697+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 >> v2:172.21.15.163:3300/0 conn(0x7fd79c155040 msgr2=0x7fd79c069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.697+0000 7fd793fff700 1 --2- 172.21.15.29:0/2051603112 >> v2:172.21.15.163:3300/0 conn(0x7fd79c155040 0x7fd79c069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 >> v1:172.21.15.163:6789/0 conn(0x7fd79c150d40 legacy=0x7fd79c154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c069e50 con 0x7fd79c065ec0 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fd7940017f0 con 0x7fd79c065ec0 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 >> v1:172.21.15.29:6789/0 conn(0x7fd79c065ec0 legacy=0x7fd79c0c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd793fff700 1 --2- 172.21.15.29:0/2051603112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7880038e0 0x7fd788003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd793fff700 1 --2- 172.21.15.29:0/2051603112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 0x7fd788004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd793fff700 1 --2- 172.21.15.29:0/2051603112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd788005000 0x7fd78800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c069b70 con 0x7fd788004690 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78800be00 con 0x7fd7880038e0 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78800bfc0 con 0x7fd788005000 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd7a0e83700 1 --2- 172.21.15.29:0/2051603112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 0x7fd788004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd7a1684700 1 --2- 172.21.15.29:0/2051603112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7880038e0 0x7fd788003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd7a1e85700 1 --2- 172.21.15.29:0/2051603112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd788005000 0x7fd78800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd7a0e83700 1 -- 172.21.15.29:0/2051603112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd788005000 msgr2=0x7fd78800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd7a0e83700 1 --2- 172.21.15.29:0/2051603112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd788005000 0x7fd78800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd7a0e83700 1 -- 172.21.15.29:0/2051603112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7880038e0 msgr2=0x7fd788003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd7a0e83700 1 --2- 172.21.15.29:0/2051603112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7880038e0 0x7fd788003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd7a0e83700 1 -- 172.21.15.29:0/2051603112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7880033f0 con 0x7fd788004690 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd7a1e85700 1 -- 172.21.15.29:0/2051603112 reap_dead start 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd7a0e83700 1 --2- 172.21.15.29:0/2051603112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 0x7fd788004b10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7fd79c155440 tx=0x7fd78c009060).ready entity=mon.0 client_cookie=4f9b464877dac2b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.698+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c009e90 con 0x7fd788004690 2024-01-24T15:13:38.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.699+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c009430 con 0x7fd788004690 2024-01-24T15:13:38.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.699+0000 7fd793fff700 1 -- 172.21.15.29:0/2051603112 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c00e850 con 0x7fd788004690 2024-01-24T15:13:38.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.699+0000 7fd7a38e8700 1 -- 172.21.15.29:0/2051603112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 msgr2=0x7fd788004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.699+0000 7fd7a38e8700 1 --2- 172.21.15.29:0/2051603112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 0x7fd788004b10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7fd79c155440 tx=0x7fd78c009060).stop 2024-01-24T15:13:38.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.699+0000 7fd7a38e8700 1 -- 172.21.15.29:0/2051603112 shutdown_connections 2024-01-24T15:13:38.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.699+0000 7fd7a38e8700 1 --2- 172.21.15.29:0/2051603112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 0x7fd788004b10 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.699+0000 7fd7a38e8700 1 -- 172.21.15.29:0/2051603112 >> 172.21.15.29:0/2051603112 conn(0x7fd79c0c8580 msgr2=0x7fd79c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:38.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.699+0000 7fd7a38e8700 1 -- 172.21.15.29:0/2051603112 shutdown_connections 2024-01-24T15:13:38.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.699+0000 7fd7a38e8700 1 -- 172.21.15.29:0/2051603112 wait complete. 2024-01-24T15:13:38.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.700+0000 7fd7a38e8700 1 Processor -- start 2024-01-24T15:13:38.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.700+0000 7fd7a38e8700 1 -- start start 2024-01-24T15:13:38.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.701+0000 7fd7a38e8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 0x7fd79c148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.701+0000 7fd7a38e8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c0ad7a0 0x7fd79c149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.701+0000 7fd7a38e8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd79c15fd40 0x7fd79c1601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.701+0000 7fd7a38e8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c0c0a00 con 0x7fd788004690 2024-01-24T15:13:38.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.701+0000 7fd7a38e8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c0bfdf0 con 0x7fd79c0ad7a0 2024-01-24T15:13:38.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.701+0000 7fd7a38e8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c0c00d0 con 0x7fd79c15fd40 2024-01-24T15:13:38.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a1684700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 0x7fd79c148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a1684700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 0x7fd79c148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51744/0 (socket says 172.21.15.29:51744) 2024-01-24T15:13:38.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a1684700 1 -- 172.21.15.29:0/1939262827 learned_addr learned my addr 172.21.15.29:0/1939262827 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:38.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a1e85700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd79c15fd40 0x7fd79c1601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a0e83700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c0ad7a0 0x7fd79c149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a1684700 1 -- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd79c15fd40 msgr2=0x7fd79c1601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a1684700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd79c15fd40 0x7fd79c1601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a1684700 1 -- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c0ad7a0 msgr2=0x7fd79c149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a1684700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c0ad7a0 0x7fd79c149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a1684700 1 -- 172.21.15.29:0/1939262827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c0c03b0 con 0x7fd788004690 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a1684700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 0x7fd79c148ef0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7fd798002a50 tx=0x7fd798007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7917fa700 1 -- 172.21.15.29:0/1939262827 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798002e10 con 0x7fd788004690 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7917fa700 1 -- 172.21.15.29:0/1939262827 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79800f8d0 con 0x7fd788004690 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7917fa700 1 -- 172.21.15.29:0/1939262827 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79800fc50 con 0x7fd788004690 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a38e8700 1 -- 172.21.15.29:0/1939262827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79c0c0690 con 0x7fd788004690 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.702+0000 7fd7a38e8700 1 -- 172.21.15.29:0/1939262827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79c149b40 con 0x7fd788004690 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.703+0000 7fd7a38e8700 1 -- 172.21.15.29:0/1939262827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79c0475e0 con 0x7fd788004690 2024-01-24T15:13:38.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.704+0000 7fd7917fa700 1 -- 172.21.15.29:0/1939262827 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd798014450 con 0x7fd788004690 2024-01-24T15:13:38.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.704+0000 7fd7917fa700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd78005f270 0x7fd780061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:38.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.704+0000 7fd7917fa700 1 -- 172.21.15.29:0/1939262827 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fd79807a550 con 0x7fd788004690 2024-01-24T15:13:38.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.704+0000 7fd7a0e83700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd78005f270 0x7fd780061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:38.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.705+0000 7fd7a0e83700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd78005f270 0x7fd780061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fd78c0095a0 tx=0x7fd78c01a000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:38.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.708+0000 7fd7917fa700 1 -- 172.21.15.29:0/1939262827 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd79804ab00 con 0x7fd788004690 2024-01-24T15:13:38.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.889+0000 7fd7a38e8700 1 -- 172.21.15.29:0/1939262827 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd79c0c51a0 con 0x7fd788004690 2024-01-24T15:13:38.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.890+0000 7fd7917fa700 1 -- 172.21.15.29:0/1939262827 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+18270 (secure 0 0 0) 0x7fd79804b2c0 con 0x7fd788004690 2024-01-24T15:13:38.892 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:38.893 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":177,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:13:38.524275+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:13:36.272483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-24T15:13:37.504363+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:13:38.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.892+0000 7fd7a38e8700 1 -- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd78005f270 msgr2=0x7fd780061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.892+0000 7fd7a38e8700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd78005f270 0x7fd780061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fd78c0095a0 tx=0x7fd78c01a000).stop 2024-01-24T15:13:38.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.892+0000 7fd7a38e8700 1 -- 172.21.15.29:0/1939262827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 msgr2=0x7fd79c148ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:38.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.892+0000 7fd7a38e8700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 0x7fd79c148ef0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7fd798002a50 tx=0x7fd798007bb0).stop 2024-01-24T15:13:38.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.893+0000 7fd7a38e8700 1 -- 172.21.15.29:0/1939262827 shutdown_connections 2024-01-24T15:13:38.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.893+0000 7fd7a38e8700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd79c15fd40 0x7fd79c1601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.893+0000 7fd7a38e8700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd78005f270 0x7fd780061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.893+0000 7fd7a38e8700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd788004690 0x7fd79c148ef0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.893+0000 7fd7a38e8700 1 --2- 172.21.15.29:0/1939262827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c0ad7a0 0x7fd79c149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:38.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.893+0000 7fd7a38e8700 1 -- 172.21.15.29:0/1939262827 >> 172.21.15.29:0/1939262827 conn(0x7fd79c0c8580 msgr2=0x7fd79c154510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:38.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.893+0000 7fd7a38e8700 1 -- 172.21.15.29:0/1939262827 shutdown_connections 2024-01-24T15:13:38.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:38.893+0000 7fd7a38e8700 1 -- 172.21.15.29:0/1939262827 wait complete. 2024-01-24T15:13:38.909 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:13:39.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.037+0000 7f1bea900700 1 Processor -- start 2024-01-24T15:13:39.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.037+0000 7f1bea900700 1 -- start start 2024-01-24T15:13:39.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1bea900700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1be4069240 0x7f1be4154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:39.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1bea900700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be406a3a0 con 0x7f1be4154e40 2024-01-24T15:13:39.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1bea900700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be4155290 con 0x7f1be4150cb0 2024-01-24T15:13:39.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1bea900700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be40c2c70 con 0x7f1be4069240 2024-01-24T15:13:39.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1be37fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1be4069240 0x7f1be4154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:39.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1be37fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1be4069240 0x7f1be4154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51748/0 (socket says 172.21.15.29:51748) 2024-01-24T15:13:39.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1be37fe700 1 -- 172.21.15.29:0/3481202336 learned_addr learned my addr 172.21.15.29:0/3481202336 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:39.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1be37fe700 1 -- 172.21.15.29:0/3481202336 >> v1:172.21.15.163:6789/0 conn(0x7f1be4150cb0 legacy=0x7f1be4065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:39.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1be37fe700 1 -- 172.21.15.29:0/3481202336 >> v1:172.21.15.29:6789/0 conn(0x7f1be4154e40 legacy=0x7f1be40bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:39.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1be37fe700 1 -- 172.21.15.29:0/3481202336 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be4069820 con 0x7f1be4069240 2024-01-24T15:13:39.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1be37fe700 1 --2- 172.21.15.29:0/3481202336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be4069240 0x7f1be4154950 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f1bd8006fd0 tx=0x7f1bd80090f0).ready entity=mon.0 client_cookie=841f3f192e545b4b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:39.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1be2ffd700 1 -- 172.21.15.29:0/3481202336 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd800b600 con 0x7f1be4069240 2024-01-24T15:13:39.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1be2ffd700 1 -- 172.21.15.29:0/3481202336 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd800b760 con 0x7f1be4069240 2024-01-24T15:13:39.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.038+0000 7f1be2ffd700 1 -- 172.21.15.29:0/3481202336 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd800bae0 con 0x7f1be4069240 2024-01-24T15:13:39.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.039+0000 7f1bea900700 1 -- 172.21.15.29:0/3481202336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be4069240 msgr2=0x7f1be4154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:39.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.039+0000 7f1bea900700 1 --2- 172.21.15.29:0/3481202336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be4069240 0x7f1be4154950 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f1bd8006fd0 tx=0x7f1bd80090f0).stop 2024-01-24T15:13:39.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.039+0000 7f1bea900700 1 -- 172.21.15.29:0/3481202336 shutdown_connections 2024-01-24T15:13:39.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.039+0000 7f1bea900700 1 --2- 172.21.15.29:0/3481202336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be4069240 0x7f1be4154950 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:39.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.039+0000 7f1bea900700 1 -- 172.21.15.29:0/3481202336 >> 172.21.15.29:0/3481202336 conn(0x7f1be40c85c0 msgr2=0x7f1be40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:39.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.039+0000 7f1bea900700 1 -- 172.21.15.29:0/3481202336 shutdown_connections 2024-01-24T15:13:39.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.039+0000 7f1bea900700 1 -- 172.21.15.29:0/3481202336 wait complete. 2024-01-24T15:13:39.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.040+0000 7f1bea900700 1 Processor -- start 2024-01-24T15:13:39.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.040+0000 7f1bea900700 1 -- start start 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.041+0000 7f1bea900700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1be4069240 0x7f1be40c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.041+0000 7f1bea900700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1be4150cb0 0x7f1be4155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.041+0000 7f1bea900700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be4154e40 0x7f1be41464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.041+0000 7f1bea900700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be40c11c0 con 0x7f1be4154e40 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.041+0000 7f1bea900700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4155910 con 0x7f1be4069240 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.041+0000 7f1bea900700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4155b70 con 0x7f1be4150cb0 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.041+0000 7f1be8e9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be4154e40 0x7f1be41464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.041+0000 7f1be8e9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be4154e40 0x7f1be41464e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51764/0 (socket says 172.21.15.29:51764) 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.041+0000 7f1be8e9d700 1 -- 172.21.15.29:0/1772121494 learned_addr learned my addr 172.21.15.29:0/1772121494 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1be3fff700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1be4069240 0x7f1be40c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1be37fe700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1be4150cb0 0x7f1be4155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1be8e9d700 1 -- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1be4150cb0 msgr2=0x7f1be4155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:39.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1be8e9d700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1be4150cb0 0x7f1be4155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1be8e9d700 1 -- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1be4069240 msgr2=0x7f1be40c0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1be8e9d700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1be4069240 0x7f1be40c0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1be8e9d700 1 -- 172.21.15.29:0/1772121494 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd8009370 con 0x7f1be4154e40 2024-01-24T15:13:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1be8e9d700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be4154e40 0x7f1be41464e0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7f1bdc00c7c0 tx=0x7f1bdc001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1bcbfff700 1 -- 172.21.15.29:0/1772121494 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc010510 con 0x7f1be4154e40 2024-01-24T15:13:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1bcbfff700 1 -- 172.21.15.29:0/1772121494 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bdc010670 con 0x7f1be4154e40 2024-01-24T15:13:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1bea900700 1 -- 172.21.15.29:0/1772121494 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be4155e50 con 0x7f1be4154e40 2024-01-24T15:13:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1bcbfff700 1 -- 172.21.15.29:0/1772121494 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0109f0 con 0x7f1be4154e40 2024-01-24T15:13:39.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.042+0000 7f1bea900700 1 -- 172.21.15.29:0/1772121494 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be4156410 con 0x7f1be4154e40 2024-01-24T15:13:39.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.043+0000 7f1bea900700 1 -- 172.21.15.29:0/1772121494 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be40475e0 con 0x7f1be4154e40 2024-01-24T15:13:39.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.044+0000 7f1bcbfff700 1 -- 172.21.15.29:0/1772121494 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1bdc01b400 con 0x7f1be4154e40 2024-01-24T15:13:39.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.044+0000 7f1bcbfff700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1bd005f240 0x7f1bd0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:39.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.044+0000 7f1bcbfff700 1 -- 172.21.15.29:0/1772121494 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f1bdc07a5f0 con 0x7f1be4154e40 2024-01-24T15:13:39.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.044+0000 7f1be3fff700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1bd005f240 0x7f1bd0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:39.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.045+0000 7f1be3fff700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1bd005f240 0x7f1bd0061700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f1bd40036e0 tx=0x7f1bd4009040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:39.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.048+0000 7f1bcbfff700 1 -- 172.21.15.29:0/1772121494 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1bdc04b2b0 con 0x7f1be4154e40 2024-01-24T15:13:39.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.229+0000 7f1bea900700 1 -- 172.21.15.29:0/1772121494 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1be4069b00 con 0x7f1be4154e40 2024-01-24T15:13:39.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.230+0000 7f1bcbfff700 1 -- 172.21.15.29:0/1772121494 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+18270 (secure 0 0 0) 0x7f1bdc04e8d0 con 0x7f1be4154e40 2024-01-24T15:13:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:39.233 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":177,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:13:38.524275+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:13:36.272483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-24T15:13:37.504363+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:13:39.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.232+0000 7f1bea900700 1 -- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1bd005f240 msgr2=0x7f1bd0061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:39.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.232+0000 7f1bea900700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1bd005f240 0x7f1bd0061700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f1bd40036e0 tx=0x7f1bd4009040).stop 2024-01-24T15:13:39.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.232+0000 7f1bea900700 1 -- 172.21.15.29:0/1772121494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be4154e40 msgr2=0x7f1be41464e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:39.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.232+0000 7f1bea900700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be4154e40 0x7f1be41464e0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7f1bdc00c7c0 tx=0x7f1bdc001510).stop 2024-01-24T15:13:39.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.232+0000 7f1bea900700 1 -- 172.21.15.29:0/1772121494 shutdown_connections 2024-01-24T15:13:39.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.232+0000 7f1bea900700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1bd005f240 0x7f1bd0061700 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:39.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.232+0000 7f1bea900700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1be4154e40 0x7f1be41464e0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:39.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.232+0000 7f1bea900700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1be4069240 0x7f1be40c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:39.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.232+0000 7f1bea900700 1 --2- 172.21.15.29:0/1772121494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1be4150cb0 0x7f1be4155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:39.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.232+0000 7f1bea900700 1 -- 172.21.15.29:0/1772121494 >> 172.21.15.29:0/1772121494 conn(0x7f1be40c85c0 msgr2=0x7f1be40c3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:39.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.233+0000 7f1bea900700 1 -- 172.21.15.29:0/1772121494 shutdown_connections 2024-01-24T15:13:39.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.233+0000 7f1bea900700 1 -- 172.21.15.29:0/1772121494 wait complete. 2024-01-24T15:13:39.250 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-24T15:13:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.382+0000 7f57bce63700 1 Processor -- start 2024-01-24T15:13:39.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.382+0000 7f57bce63700 1 -- start start 2024-01-24T15:13:39.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57bce63700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57b80664c0 0x7f57b8066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:39.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57bce63700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57b80bdb30 0x7f57b80bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57bce63700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b80686a0 con 0x7f57b8134ea0 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57bce63700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b8066e30 con 0x7f57b80664c0 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57bce63700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80c38e0 con 0x7f57b80bdb30 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57b5d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57b80664c0 0x7f57b8066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57b5d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57b80664c0 0x7f57b8066940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51778/0 (socket says 172.21.15.29:51778) 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57b659c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f57b8134ea0 0x7f57b80c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41164/0 (socket says 172.21.15.29:41164) 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57b5d9b700 1 -- 172.21.15.29:0/1980546673 learned_addr learned my addr 172.21.15.29:0/1980546673 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57b6d9d700 1 --2- 172.21.15.29:0/1980546673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57b80bdb30 0x7f57b80bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57b5d9b700 1 -- 172.21.15.29:0/1980546673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57b80bdb30 msgr2=0x7f57b80bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57b5d9b700 1 --2- 172.21.15.29:0/1980546673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57b80bdb30 0x7f57b80bdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57b5d9b700 1 -- 172.21.15.29:0/1980546673 >> v1:172.21.15.29:6789/0 conn(0x7f57b8134ea0 legacy=0x7f57b80c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57b5d9b700 1 -- 172.21.15.29:0/1980546673 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b8068b30 con 0x7f57b80664c0 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.383+0000 7f57b6d9d700 1 --2- 172.21.15.29:0/1980546673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57b80bdb30 0x7f57b80bdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.384+0000 7f57b5d9b700 1 --2- 172.21.15.29:0/1980546673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57b80664c0 0x7f57b8066940 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f57ac006fd0 tx=0x7f57ac0090f0).ready entity=mon.0 client_cookie=86a757b40e42cbb2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:39.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.384+0000 7f57b559a700 1 -- 172.21.15.29:0/1980546673 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac00b600 con 0x7f57b80664c0 2024-01-24T15:13:39.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.384+0000 7f57b559a700 1 -- 172.21.15.29:0/1980546673 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac00b760 con 0x7f57b80664c0 2024-01-24T15:13:39.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.384+0000 7f57b559a700 1 -- 172.21.15.29:0/1980546673 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac00bae0 con 0x7f57b80664c0 2024-01-24T15:13:39.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.384+0000 7f57bce63700 1 -- 172.21.15.29:0/1980546673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57b80664c0 msgr2=0x7f57b8066940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:39.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.384+0000 7f57bce63700 1 --2- 172.21.15.29:0/1980546673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57b80664c0 0x7f57b8066940 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f57ac006fd0 tx=0x7f57ac0090f0).stop 2024-01-24T15:13:39.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.384+0000 7f57bce63700 1 -- 172.21.15.29:0/1980546673 shutdown_connections 2024-01-24T15:13:39.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.384+0000 7f57bce63700 1 --2- 172.21.15.29:0/1980546673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57b80bdb30 0x7f57b80bdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:39.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.384+0000 7f57bce63700 1 --2- 172.21.15.29:0/1980546673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57b80664c0 0x7f57b8066940 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:39.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.384+0000 7f57bce63700 1 -- 172.21.15.29:0/1980546673 >> 172.21.15.29:0/1980546673 conn(0x7f57b80c8bd0 msgr2=0x7f57b80c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:39.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.384+0000 7f57bce63700 1 -- 172.21.15.29:0/1980546673 shutdown_connections 2024-01-24T15:13:39.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.384+0000 7f57bce63700 1 -- 172.21.15.29:0/1980546673 wait complete. 2024-01-24T15:13:39.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.386+0000 7f57bce63700 1 Processor -- start 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.386+0000 7f57bce63700 1 -- start start 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.387+0000 7f57bce63700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b80664c0 0x7f57b8163fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.387+0000 7f57bce63700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57b80bdb30 0x7f57b8064da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.387+0000 7f57bce63700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57b8134ea0 0x7f57b8065290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.387+0000 7f57bce63700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80648d0 con 0x7f57b8134ea0 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.387+0000 7f57bce63700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8164490 con 0x7f57b80664c0 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.387+0000 7f57bce63700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8065780 con 0x7f57b80bdb30 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.387+0000 7f57b6d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57b8134ea0 0x7f57b8065290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.387+0000 7f57b6d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57b8134ea0 0x7f57b8065290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51784/0 (socket says 172.21.15.29:51784) 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.387+0000 7f57b6d9d700 1 -- 172.21.15.29:0/3959733623 learned_addr learned my addr 172.21.15.29:0/3959733623 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.387+0000 7f57b5d9b700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57b80bdb30 0x7f57b8064da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.387+0000 7f57b659c700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b80664c0 0x7f57b8163fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.388+0000 7f57b6d9d700 1 -- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57b80bdb30 msgr2=0x7f57b8064da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:39.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.388+0000 7f57b6d9d700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57b80bdb30 0x7f57b8064da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:39.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.388+0000 7f57b6d9d700 1 -- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b80664c0 msgr2=0x7f57b8163fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:39.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.388+0000 7f57b6d9d700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b80664c0 0x7f57b8163fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:39.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.388+0000 7f57b6d9d700 1 -- 172.21.15.29:0/3959733623 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac009370 con 0x7f57b8134ea0 2024-01-24T15:13:39.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.388+0000 7f57b6d9d700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57b8134ea0 0x7f57b8065290 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f57b0009fc0 tx=0x7f57b00099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:39.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.388+0000 7f57a67fc700 1 -- 172.21.15.29:0/3959733623 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000dbe0 con 0x7f57b8134ea0 2024-01-24T15:13:39.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.388+0000 7f57a67fc700 1 -- 172.21.15.29:0/3959733623 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b000f040 con 0x7f57b8134ea0 2024-01-24T15:13:39.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.388+0000 7f57bce63700 1 -- 172.21.15.29:0/3959733623 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b8135e20 con 0x7f57b8134ea0 2024-01-24T15:13:39.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.388+0000 7f57a67fc700 1 -- 172.21.15.29:0/3959733623 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b00133f0 con 0x7f57b8134ea0 2024-01-24T15:13:39.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.388+0000 7f57bce63700 1 -- 172.21.15.29:0/3959733623 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b8136170 con 0x7f57b8134ea0 2024-01-24T15:13:39.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.389+0000 7f57bce63700 1 -- 172.21.15.29:0/3959733623 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b80475e0 con 0x7f57b8134ea0 2024-01-24T15:13:39.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.389+0000 7f57a67fc700 1 -- 172.21.15.29:0/3959733623 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f57b00137e0 con 0x7f57b8134ea0 2024-01-24T15:13:39.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.390+0000 7f57a67fc700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f579805f270 0x7f5798061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:39.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.390+0000 7f57a67fc700 1 -- 172.21.15.29:0/3959733623 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f57b007ad60 con 0x7f57b8134ea0 2024-01-24T15:13:39.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.390+0000 7f57b659c700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f579805f270 0x7f5798061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:39.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.390+0000 7f57b659c700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f579805f270 0x7f5798061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f57a8007fd0 tx=0x7f57a8007d30).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:39.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.394+0000 7f57a67fc700 1 -- 172.21.15.29:0/3959733623 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f57b004ba80 con 0x7f57b8134ea0 2024-01-24T15:13:39.612 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:13:39.614 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:13:39.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:39.626+0000 7f57bce63700 1 -- 172.21.15.29:0/3959733623 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f57b8067a90 con 0x7f57b8134ea0 2024-01-24T15:13:40.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.539+0000 7f57a67fc700 1 -- 172.21.15.29:0/3959733623 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) v1 ==== 118+0+0 (secure 0 0 0) 0x7f57b004f0a0 con 0x7f57b8134ea0 2024-01-24T15:13:40.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.541+0000 7f57bce63700 1 -- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f579805f270 msgr2=0x7f5798061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:40.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.541+0000 7f57bce63700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f579805f270 0x7f5798061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f57a8007fd0 tx=0x7f57a8007d30).stop 2024-01-24T15:13:40.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.541+0000 7f57bce63700 1 -- 172.21.15.29:0/3959733623 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57b8134ea0 msgr2=0x7f57b8065290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:40.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.541+0000 7f57bce63700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57b8134ea0 0x7f57b8065290 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f57b0009fc0 tx=0x7f57b00099b0).stop 2024-01-24T15:13:40.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.541+0000 7f57bce63700 1 -- 172.21.15.29:0/3959733623 shutdown_connections 2024-01-24T15:13:40.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.541+0000 7f57bce63700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f579805f270 0x7f5798061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:40.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.542+0000 7f57bce63700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57b8134ea0 0x7f57b8065290 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:40.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.542+0000 7f57bce63700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b80664c0 0x7f57b8163fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:40.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.542+0000 7f57bce63700 1 --2- 172.21.15.29:0/3959733623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57b80bdb30 0x7f57b8064da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:40.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.542+0000 7f57bce63700 1 -- 172.21.15.29:0/3959733623 >> 172.21.15.29:0/3959733623 conn(0x7f57b80c8bd0 msgr2=0x7f57b80c7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:40.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.542+0000 7f57bce63700 1 -- 172.21.15.29:0/3959733623 shutdown_connections 2024-01-24T15:13:40.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.542+0000 7f57bce63700 1 -- 172.21.15.29:0/3959733623 wait complete. 2024-01-24T15:13:40.559 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-24T15:13:40.559 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-24T15:13:40.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.694+0000 7f18593c0700 1 Processor -- start 2024-01-24T15:13:40.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.694+0000 7f18593c0700 1 -- start start 2024-01-24T15:13:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.695+0000 7f18593c0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1854134ea0 0x7f18540c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.695+0000 7f18593c0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18540c36c0 0x7f18540664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.695+0000 7f18593c0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18540686a0 con 0x7f18540669b0 2024-01-24T15:13:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.695+0000 7f18593c0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1854066e30 con 0x7f1854134ea0 2024-01-24T15:13:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.695+0000 7f18593c0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1854068b30 con 0x7f18540c36c0 2024-01-24T15:13:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.696+0000 7f1852ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1854134ea0 0x7f18540c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.696+0000 7f1852ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1854134ea0 0x7f18540c31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51792/0 (socket says 172.21.15.29:51792) 2024-01-24T15:13:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.696+0000 7f1852ffd700 1 -- 172.21.15.29:0/3606882773 learned_addr learned my addr 172.21.15.29:0/3606882773 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:40.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.696+0000 7f18527fc700 1 --2- 172.21.15.29:0/3606882773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18540c36c0 0x7f18540664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.696+0000 7f1852ffd700 1 -- 172.21.15.29:0/3606882773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18540c36c0 msgr2=0x7f18540664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.696+0000 7f1852ffd700 1 --2- 172.21.15.29:0/3606882773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18540c36c0 0x7f18540664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.696+0000 7f1852ffd700 1 -- 172.21.15.29:0/3606882773 >> v1:172.21.15.163:6789/0 conn(0x7f18540669b0 legacy=0x7f18541559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.696+0000 7f1852ffd700 1 -- 172.21.15.29:0/3606882773 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1854067a90 con 0x7f1854134ea0 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.696+0000 7f1852ffd700 1 --2- 172.21.15.29:0/3606882773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1854134ea0 0x7f18540c31d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f1844006fd0 tx=0x7f18440090f0).ready entity=mon.0 client_cookie=8c81254333686a08 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.696+0000 7f1851ffb700 1 -- 172.21.15.29:0/3606882773 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184400b520 con 0x7f1854134ea0 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.696+0000 7f1851ffb700 1 -- 172.21.15.29:0/3606882773 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184400b680 con 0x7f1854134ea0 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.696+0000 7f1851ffb700 1 -- 172.21.15.29:0/3606882773 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184400ba00 con 0x7f1854134ea0 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.697+0000 7f18593c0700 1 -- 172.21.15.29:0/3606882773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1854134ea0 msgr2=0x7f18540c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.697+0000 7f18593c0700 1 --2- 172.21.15.29:0/3606882773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1854134ea0 0x7f18540c31d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f1844006fd0 tx=0x7f18440090f0).stop 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.697+0000 7f18593c0700 1 -- 172.21.15.29:0/3606882773 shutdown_connections 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.697+0000 7f18593c0700 1 --2- 172.21.15.29:0/3606882773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1854134ea0 0x7f18540c31d0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:40.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.697+0000 7f18593c0700 1 --2- 172.21.15.29:0/3606882773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18540c36c0 0x7f18540664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:40.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.697+0000 7f18593c0700 1 -- 172.21.15.29:0/3606882773 >> 172.21.15.29:0/3606882773 conn(0x7f18540c8bd0 msgr2=0x7f18540c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:40.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.697+0000 7f18593c0700 1 -- 172.21.15.29:0/3606882773 shutdown_connections 2024-01-24T15:13:40.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.697+0000 7f18593c0700 1 -- 172.21.15.29:0/3606882773 wait complete. 2024-01-24T15:13:40.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.698+0000 7f18593c0700 1 Processor -- start 2024-01-24T15:13:40.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.698+0000 7f18593c0700 1 -- start start 2024-01-24T15:13:40.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.699+0000 7f18593c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18540669b0 0x7f18540ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:40.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.699+0000 7f18593c0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18540c36c0 0x7f1854137b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:40.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.699+0000 7f18593c0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1854134ea0 0x7f185414e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:40.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.699+0000 7f18593c0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18540c0c40 con 0x7f1854134ea0 2024-01-24T15:13:40.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.699+0000 7f18593c0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18540c17c0 con 0x7f18540669b0 2024-01-24T15:13:40.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.699+0000 7f18593c0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18540c0e90 con 0x7f18540c36c0 2024-01-24T15:13:40.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.699+0000 7f18537fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1854134ea0 0x7f185414e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.699+0000 7f18537fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1854134ea0 0x7f185414e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51798/0 (socket says 172.21.15.29:51798) 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.699+0000 7f18537fe700 1 -- 172.21.15.29:0/3222774570 learned_addr learned my addr 172.21.15.29:0/3222774570 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.699+0000 7f1852ffd700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18540669b0 0x7f18540ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.700+0000 7f18527fc700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18540c36c0 0x7f1854137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.700+0000 7f18537fe700 1 -- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18540c36c0 msgr2=0x7f1854137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.700+0000 7f18537fe700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18540c36c0 0x7f1854137b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.700+0000 7f18537fe700 1 -- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18540669b0 msgr2=0x7f18540ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.700+0000 7f18537fe700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18540669b0 0x7f18540ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.700+0000 7f18537fe700 1 -- 172.21.15.29:0/3222774570 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1844009370 con 0x7f1854134ea0 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.700+0000 7f18537fe700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1854134ea0 0x7f185414e220 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f184c00b810 tx=0x7f184c009270).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.700+0000 7f184affd700 1 -- 172.21.15.29:0/3222774570 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c00d4a0 con 0x7f1854134ea0 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.700+0000 7f184affd700 1 -- 172.21.15.29:0/3222774570 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c00d640 con 0x7f1854134ea0 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.700+0000 7f18593c0700 1 -- 172.21.15.29:0/3222774570 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18540c1170 con 0x7f1854134ea0 2024-01-24T15:13:40.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.700+0000 7f184affd700 1 -- 172.21.15.29:0/3222774570 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c00da00 con 0x7f1854134ea0 2024-01-24T15:13:40.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.700+0000 7f18593c0700 1 -- 172.21.15.29:0/3222774570 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1854138000 con 0x7f1854134ea0 2024-01-24T15:13:40.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.701+0000 7f18593c0700 1 -- 172.21.15.29:0/3222774570 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18540c1930 con 0x7f1854134ea0 2024-01-24T15:13:40.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.702+0000 7f184affd700 1 -- 172.21.15.29:0/3222774570 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f184c011400 con 0x7f1854134ea0 2024-01-24T15:13:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.702+0000 7f184affd700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f183405f270 0x7f1834061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.702+0000 7f184affd700 1 -- 172.21.15.29:0/3222774570 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f184c079f80 con 0x7f1854134ea0 2024-01-24T15:13:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.703+0000 7f1852ffd700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f183405f270 0x7f1834061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:40.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.703+0000 7f1852ffd700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f183405f270 0x7f1834061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1844009130 tx=0x7f1844015000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:40.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.706+0000 7f184affd700 1 -- 172.21.15.29:0/3222774570 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f184c047b00 con 0x7f1854134ea0 2024-01-24T15:13:40.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:40.933+0000 7f18593c0700 1 -- 172.21.15.29:0/3222774570 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f18540475e0 con 0x7f1854134ea0 2024-01-24T15:13:41.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.544+0000 7f184affd700 1 -- 172.21.15.29:0/3222774570 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) v1 ==== 122+0+0 (secure 0 0 0) 0x7f184c04e430 con 0x7f1854134ea0 2024-01-24T15:13:41.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.546+0000 7f18593c0700 1 -- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f183405f270 msgr2=0x7f1834061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:41.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.546+0000 7f18593c0700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f183405f270 0x7f1834061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1844009130 tx=0x7f1844015000).stop 2024-01-24T15:13:41.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.546+0000 7f18593c0700 1 -- 172.21.15.29:0/3222774570 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1854134ea0 msgr2=0x7f185414e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:41.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.546+0000 7f18593c0700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1854134ea0 0x7f185414e220 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f184c00b810 tx=0x7f184c009270).stop 2024-01-24T15:13:41.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.546+0000 7f18593c0700 1 -- 172.21.15.29:0/3222774570 shutdown_connections 2024-01-24T15:13:41.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.546+0000 7f18593c0700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f183405f270 0x7f1834061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.546+0000 7f18593c0700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1854134ea0 0x7f185414e220 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.546+0000 7f18593c0700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18540669b0 0x7f18540ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.546+0000 7f18593c0700 1 --2- 172.21.15.29:0/3222774570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18540c36c0 0x7f1854137b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.546+0000 7f18593c0700 1 -- 172.21.15.29:0/3222774570 >> 172.21.15.29:0/3222774570 conn(0x7f18540c8bd0 msgr2=0x7f1854154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:41.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.547+0000 7f18593c0700 1 -- 172.21.15.29:0/3222774570 shutdown_connections 2024-01-24T15:13:41.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.547+0000 7f18593c0700 1 -- 172.21.15.29:0/3222774570 wait complete. 2024-01-24T15:13:41.563 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-24T15:13:41.563 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:13:41.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.696+0000 7efd31b0d700 1 Processor -- start 2024-01-24T15:13:41.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.696+0000 7efd31b0d700 1 -- start start 2024-01-24T15:13:41.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd31b0d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efd2c138330 0x7efd2c138710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:41.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd31b0d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c0bca50 0x7efd2c0be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:41.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd31b0d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2c065770 con 0x7efd2c0627b0 2024-01-24T15:13:41.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd31b0d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd2c069cf0 con 0x7efd2c138330 2024-01-24T15:13:41.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd31b0d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c069f60 con 0x7efd2c0bca50 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2bfff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7efd2c0627b0 0x7efd2c155ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41166/0 (socket says 172.21.15.29:41166) 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2bfff700 1 -- 172.21.15.29:0/31362573 learned_addr learned my addr 172.21.15.29:0/31362573 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2b7fe700 1 --2- 172.21.15.29:0/31362573 >> v2:172.21.15.163:3300/0 conn(0x7efd2c138330 0x7efd2c138710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2affd700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c0bca50 0x7efd2c0be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248709891 0 0) 0x7efd2c065770 con 0x7efd2c0627b0 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd140033f0 con 0x7efd2c0627b0 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1592743363 0 0) 0x7efd140033f0 con 0x7efd2c0627b0 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c0bca50 msgr2=0x7efd2c0be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c0bca50 0x7efd2c0be3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 >> v2:172.21.15.163:3300/0 conn(0x7efd2c138330 msgr2=0x7efd2c138710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 --2- 172.21.15.29:0/31362573 >> v2:172.21.15.163:3300/0 conn(0x7efd2c138330 0x7efd2c138710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c06a240 con 0x7efd2c0627b0 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7efd240017f0 con 0x7efd2c0627b0 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2b7fe700 1 --2- 172.21.15.29:0/31362573 >> v2:172.21.15.163:3300/0 conn(0x7efd2c138330 0x7efd2c138710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:41.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 >> v1:172.21.15.29:6789/0 conn(0x7efd2c0627b0 legacy=0x7efd2c155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd140038e0 0x7efd14003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd14004690 0x7efd14004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd14005000 0x7efd1400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c069cf0 con 0x7efd14004690 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c069f60 con 0x7efd140038e0 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.697+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1400bdd0 con 0x7efd14005000 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2affd700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c0bca50 0x7efd2c0be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2bfff700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd14005000 0x7efd1400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2b7fe700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd140038e0 0x7efd14003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2affd700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd14004690 0x7efd14004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2affd700 1 -- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd14005000 msgr2=0x7efd1400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2affd700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd14005000 0x7efd1400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2affd700 1 -- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd140038e0 msgr2=0x7efd14003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:41.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2affd700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd140038e0 0x7efd14003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2affd700 1 -- 172.21.15.29:0/31362573 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd140033f0 con 0x7efd14004690 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2b7fe700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd140038e0 0x7efd14003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2bfff700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd14005000 0x7efd1400b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2affd700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd14004690 0x7efd14004b10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7efd20009ad0 tx=0x7efd20009060).ready entity=mon.0 client_cookie=ecd20e5ff2170cce server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd200094d0 con 0x7efd14004690 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2000e640 con 0x7efd14004690 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.698+0000 7efd2a7fc700 1 -- 172.21.15.29:0/31362573 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2000ea00 con 0x7efd14004690 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.699+0000 7efd31b0d700 1 -- 172.21.15.29:0/31362573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd14004690 msgr2=0x7efd14004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.699+0000 7efd31b0d700 1 --2- 172.21.15.29:0/31362573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd14004690 0x7efd14004b10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7efd20009ad0 tx=0x7efd20009060).stop 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.699+0000 7efd2bfff700 1 -- 172.21.15.29:0/31362573 reap_dead start 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.699+0000 7efd31b0d700 1 -- 172.21.15.29:0/31362573 shutdown_connections 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.699+0000 7efd31b0d700 1 -- 172.21.15.29:0/31362573 >> 172.21.15.29:0/31362573 conn(0x7efd2c0c8580 msgr2=0x7efd2c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.699+0000 7efd31b0d700 1 -- 172.21.15.29:0/31362573 shutdown_connections 2024-01-24T15:13:41.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.699+0000 7efd31b0d700 1 -- 172.21.15.29:0/31362573 wait complete. 2024-01-24T15:13:41.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.700+0000 7efd31b0d700 1 Processor -- start 2024-01-24T15:13:41.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.701+0000 7efd31b0d700 1 -- start start 2024-01-24T15:13:41.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.701+0000 7efd31b0d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c137480 0x7efd2c0ad440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:41.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.701+0000 7efd31b0d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd2c0ad930 0x7efd2c156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:41.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.701+0000 7efd31b0d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd2c137860 0x7efd2c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:41.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.701+0000 7efd31b0d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c156fe0 con 0x7efd2c137860 2024-01-24T15:13:41.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.701+0000 7efd31b0d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c0ac0c0 con 0x7efd2c0ad930 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.701+0000 7efd31b0d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c0ab4b0 con 0x7efd2c137480 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.701+0000 7efd2bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd2c137860 0x7efd2c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd2bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd2c137860 0x7efd2c156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51818/0 (socket says 172.21.15.29:51818) 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd2bfff700 1 -- 172.21.15.29:0/544732994 learned_addr learned my addr 172.21.15.29:0/544732994 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd2b7fe700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c137480 0x7efd2c0ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd2affd700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd2c0ad930 0x7efd2c156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd2bfff700 1 -- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c137480 msgr2=0x7efd2c0ad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd2bfff700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c137480 0x7efd2c0ad440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd2bfff700 1 -- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd2c0ad930 msgr2=0x7efd2c156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd2bfff700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd2c0ad930 0x7efd2c156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd2bfff700 1 -- 172.21.15.29:0/544732994 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c0ab790 con 0x7efd2c137860 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd2bfff700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd2c137860 0x7efd2c156af0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7efd2c06aa40 tx=0x7efd24002210).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd137fe700 1 -- 172.21.15.29:0/544732994 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24001f50 con 0x7efd2c137860 2024-01-24T15:13:41.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd137fe700 1 -- 172.21.15.29:0/544732994 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd24007de0 con 0x7efd2c137860 2024-01-24T15:13:41.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd137fe700 1 -- 172.21.15.29:0/544732994 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2400ce90 con 0x7efd2c137860 2024-01-24T15:13:41.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd31b0d700 1 -- 172.21.15.29:0/544732994 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2c0aba70 con 0x7efd2c137860 2024-01-24T15:13:41.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.702+0000 7efd31b0d700 1 -- 172.21.15.29:0/544732994 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2c155ab0 con 0x7efd2c137860 2024-01-24T15:13:41.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.703+0000 7efd31b0d700 1 -- 172.21.15.29:0/544732994 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2c0ac250 con 0x7efd2c137860 2024-01-24T15:13:41.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.704+0000 7efd137fe700 1 -- 172.21.15.29:0/544732994 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7efd24015620 con 0x7efd2c137860 2024-01-24T15:13:41.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.704+0000 7efd137fe700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efd1805f270 0x7efd18061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:41.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.704+0000 7efd137fe700 1 -- 172.21.15.29:0/544732994 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7efd2407a4b0 con 0x7efd2c137860 2024-01-24T15:13:41.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.704+0000 7efd2b7fe700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efd1805f270 0x7efd18061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:41.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.705+0000 7efd2b7fe700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efd1805f270 0x7efd18061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7efd2c069890 tx=0x7efd1c009060).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:41.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.709+0000 7efd137fe700 1 -- 172.21.15.29:0/544732994 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7efd2404b1d0 con 0x7efd2c137860 2024-01-24T15:13:41.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.936+0000 7efd31b0d700 1 -- 172.21.15.29:0/544732994 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd2c065400 con 0x7efd2c137860 2024-01-24T15:13:41.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.936+0000 7efd137fe700 1 -- 172.21.15.29:0/544732994 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4497 (secure 0 0 0) 0x7efd2401d020 con 0x7efd2c137860 2024-01-24T15:13:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:41.939 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7319,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3000519646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3000519646},{"type":"v1","addr":"172.21.15.163:6836","nonce":3000519646}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3497543927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3497543927},{"type":"v1","addr":"172.21.15.29:6835","nonce":3497543927}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2353694052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2353694052},{"type":"v1","addr":"172.21.15.163:6837","nonce":2353694052}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:13:38.524646+0000","modified":"2024-01-24T15:13:41.539080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/3595147603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3595147603},{"type":"v1","addr":"172.21.15.29:6834","nonce":3595147603}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-24T15:13:41.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.939+0000 7efd31b0d700 1 -- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efd1805f270 msgr2=0x7efd18061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:41.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.939+0000 7efd31b0d700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efd1805f270 0x7efd18061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7efd2c069890 tx=0x7efd1c009060).stop 2024-01-24T15:13:41.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.939+0000 7efd31b0d700 1 -- 172.21.15.29:0/544732994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd2c137860 msgr2=0x7efd2c156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:41.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.939+0000 7efd31b0d700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd2c137860 0x7efd2c156af0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7efd2c06aa40 tx=0x7efd24002210).stop 2024-01-24T15:13:41.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.939+0000 7efd31b0d700 1 -- 172.21.15.29:0/544732994 shutdown_connections 2024-01-24T15:13:41.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.939+0000 7efd31b0d700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efd1805f270 0x7efd18061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.939+0000 7efd31b0d700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd2c137860 0x7efd2c156af0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.939+0000 7efd31b0d700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c137480 0x7efd2c0ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.939+0000 7efd31b0d700 1 --2- 172.21.15.29:0/544732994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd2c0ad930 0x7efd2c156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:41.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.939+0000 7efd31b0d700 1 -- 172.21.15.29:0/544732994 >> 172.21.15.29:0/544732994 conn(0x7efd2c0c8580 msgr2=0x7efd2c151810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:41.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.939+0000 7efd31b0d700 1 -- 172.21.15.29:0/544732994 shutdown_connections 2024-01-24T15:13:41.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:41.939+0000 7efd31b0d700 1 -- 172.21.15.29:0/544732994 wait complete. 2024-01-24T15:13:41.943 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 74 2024-01-24T15:13:41.955 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:13:42.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.083+0000 7f45cae7b700 1 Processor -- start 2024-01-24T15:13:42.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.084+0000 7f45cae7b700 1 -- start start 2024-01-24T15:13:42.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.085+0000 7f45cae7b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45c40638e0 0x7f45c40aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.085+0000 7f45cae7b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c40aa6c0 con 0x7f45c40c5410 2024-01-24T15:13:42.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.085+0000 7f45cae7b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c4064240 con 0x7f45c4154370 2024-01-24T15:13:42.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.085+0000 7f45cae7b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c40cb5c0 con 0x7f45c40638e0 2024-01-24T15:13:42.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.085+0000 7f45c9418700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45c40638e0 0x7f45c40aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.085+0000 7f45c9418700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45c40638e0 0x7f45c40aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51824/0 (socket says 172.21.15.29:51824) 2024-01-24T15:13:42.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.085+0000 7f45c9418700 1 -- 172.21.15.29:0/3258925979 learned_addr learned my addr 172.21.15.29:0/3258925979 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:42.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.085+0000 7f45c9418700 1 -- 172.21.15.29:0/3258925979 >> v1:172.21.15.163:6789/0 conn(0x7f45c4154370 legacy=0x7f45c40ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.085+0000 7f45c9418700 1 -- 172.21.15.29:0/3258925979 >> v1:172.21.15.29:6789/0 conn(0x7f45c40c5410 legacy=0x7f45c40631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.085+0000 7f45c9418700 1 -- 172.21.15.29:0/3258925979 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c40cb8a0 con 0x7f45c40638e0 2024-01-24T15:13:42.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.086+0000 7f45c9418700 1 --2- 172.21.15.29:0/3258925979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c40638e0 0x7f45c40aa0e0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f45bc009f20 tx=0x7f45bc009840).ready entity=mon.0 client_cookie=a015cbffedb367c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:42.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.086+0000 7f45c37fe700 1 -- 172.21.15.29:0/3258925979 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00e070 con 0x7f45c40638e0 2024-01-24T15:13:42.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.086+0000 7f45c37fe700 1 -- 172.21.15.29:0/3258925979 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc00ba80 con 0x7f45c40638e0 2024-01-24T15:13:42.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.086+0000 7f45c37fe700 1 -- 172.21.15.29:0/3258925979 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00be00 con 0x7f45c40638e0 2024-01-24T15:13:42.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.086+0000 7f45cae7b700 1 -- 172.21.15.29:0/3258925979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c40638e0 msgr2=0x7f45c40aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.086+0000 7f45cae7b700 1 --2- 172.21.15.29:0/3258925979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c40638e0 0x7f45c40aa0e0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f45bc009f20 tx=0x7f45bc009840).stop 2024-01-24T15:13:42.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.086+0000 7f45cae7b700 1 -- 172.21.15.29:0/3258925979 shutdown_connections 2024-01-24T15:13:42.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.086+0000 7f45cae7b700 1 --2- 172.21.15.29:0/3258925979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c40638e0 0x7f45c40aa0e0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.086+0000 7f45cae7b700 1 -- 172.21.15.29:0/3258925979 >> 172.21.15.29:0/3258925979 conn(0x7f45c405ec80 msgr2=0x7f45c40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:42.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.086+0000 7f45cae7b700 1 -- 172.21.15.29:0/3258925979 shutdown_connections 2024-01-24T15:13:42.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.086+0000 7f45cae7b700 1 -- 172.21.15.29:0/3258925979 wait complete. 2024-01-24T15:13:42.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.087+0000 7f45cae7b700 1 Processor -- start 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.088+0000 7f45cae7b700 1 -- start start 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.088+0000 7f45cae7b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c40638e0 0x7f45c40c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.088+0000 7f45cae7b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c40c5410 0x7f45c41646c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.088+0000 7f45cae7b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4154370 0x7f45c41684a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45cae7b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c40c3cb0 con 0x7f45c40c5410 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45cae7b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c4168010 con 0x7f45c40638e0 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45cae7b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c4164fb0 con 0x7f45c4154370 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c40c5410 0x7f45c41646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c40c5410 0x7f45c41646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51836/0 (socket says 172.21.15.29:51836) 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c3fff700 1 -- 172.21.15.29:0/1744486059 learned_addr learned my addr 172.21.15.29:0/1744486059 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c9418700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4154370 0x7f45c41684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c8c17700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c40638e0 0x7f45c40c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c3fff700 1 -- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4154370 msgr2=0x7f45c41684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c3fff700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4154370 0x7f45c41684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c3fff700 1 -- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c40638e0 msgr2=0x7f45c40c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c3fff700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c40638e0 0x7f45c40c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c3fff700 1 -- 172.21.15.29:0/1744486059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc009ae0 con 0x7f45c40c5410 2024-01-24T15:13:42.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c3fff700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c40c5410 0x7f45c41646c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7f45b000c4c0 tx=0x7f45b0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:42.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c0ff9700 1 -- 172.21.15.29:0/1744486059 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0007020 con 0x7f45c40c5410 2024-01-24T15:13:42.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c0ff9700 1 -- 172.21.15.29:0/1744486059 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b0001e90 con 0x7f45c40c5410 2024-01-24T15:13:42.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45c0ff9700 1 -- 172.21.15.29:0/1744486059 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b00073b0 con 0x7f45c40c5410 2024-01-24T15:13:42.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45cae7b700 1 -- 172.21.15.29:0/1744486059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c4165290 con 0x7f45c40c5410 2024-01-24T15:13:42.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.089+0000 7f45cae7b700 1 -- 172.21.15.29:0/1744486059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c4167ca0 con 0x7f45c40c5410 2024-01-24T15:13:42.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.091+0000 7f45cae7b700 1 -- 172.21.15.29:0/1744486059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c4157f90 con 0x7f45c40c5410 2024-01-24T15:13:42.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.091+0000 7f45c0ff9700 1 -- 172.21.15.29:0/1744486059 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f45b00068b0 con 0x7f45c40c5410 2024-01-24T15:13:42.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.091+0000 7f45c0ff9700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f45a805f270 0x7f45a8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.092+0000 7f45c0ff9700 1 -- 172.21.15.29:0/1744486059 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f45b007a190 con 0x7f45c40c5410 2024-01-24T15:13:42.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.092+0000 7f45c8c17700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f45a805f270 0x7f45a8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.092+0000 7f45c8c17700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f45a805f270 0x7f45a8061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f45b8007fd0 tx=0x7f45b8009040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:42.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.096+0000 7f45c0ff9700 1 -- 172.21.15.29:0/1744486059 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f45b004aeb0 con 0x7f45c40c5410 2024-01-24T15:13:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.278+0000 7f45cae7b700 1 -- 172.21.15.29:0/1744486059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45c40475e0 con 0x7f45c40c5410 2024-01-24T15:13:42.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.279+0000 7f45c0ff9700 1 -- 172.21.15.29:0/1744486059 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+18257 (secure 0 0 0) 0x7f45b004e4d0 con 0x7f45c40c5410 2024-01-24T15:13:42.281 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:42.281 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":178,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:13:39.526495+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:13:36.272483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-24T15:13:37.504363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:13:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.280+0000 7f45cae7b700 1 -- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f45a805f270 msgr2=0x7f45a8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.280+0000 7f45cae7b700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f45a805f270 0x7f45a8061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f45b8007fd0 tx=0x7f45b8009040).stop 2024-01-24T15:13:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.281+0000 7f45cae7b700 1 -- 172.21.15.29:0/1744486059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c40c5410 msgr2=0x7f45c41646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.281+0000 7f45cae7b700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c40c5410 0x7f45c41646c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7f45b000c4c0 tx=0x7f45b0007e00).stop 2024-01-24T15:13:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.281+0000 7f45cae7b700 1 -- 172.21.15.29:0/1744486059 shutdown_connections 2024-01-24T15:13:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.281+0000 7f45cae7b700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45c4154370 0x7f45c41684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.281+0000 7f45cae7b700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c40638e0 0x7f45c40c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.281+0000 7f45cae7b700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f45a805f270 0x7f45a8061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.281+0000 7f45cae7b700 1 --2- 172.21.15.29:0/1744486059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45c40c5410 0x7f45c41646c0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.281+0000 7f45cae7b700 1 -- 172.21.15.29:0/1744486059 >> 172.21.15.29:0/1744486059 conn(0x7f45c405ec80 msgr2=0x7f45c414fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:42.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.281+0000 7f45cae7b700 1 -- 172.21.15.29:0/1744486059 shutdown_connections 2024-01-24T15:13:42.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.281+0000 7f45cae7b700 1 -- 172.21.15.29:0/1744486059 wait complete. 2024-01-24T15:13:42.297 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-24T15:13:42.297 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:13:42.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.425+0000 7fc32b8be700 1 Processor -- start 2024-01-24T15:13:42.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.425+0000 7fc32b8be700 1 -- start start 2024-01-24T15:13:42.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.426+0000 7fc32b8be700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc324062970 0x7fc324062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.426+0000 7fc32b8be700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc324158600 0x7fc324154d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.426+0000 7fc32b8be700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3240bcf60 con 0x7fc324066b40 2024-01-24T15:13:42.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.426+0000 7fc32b8be700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3240bdba0 con 0x7fc324062970 2024-01-24T15:13:42.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.426+0000 7fc32b8be700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3240a7480 con 0x7fc324158600 2024-01-24T15:13:42.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.426+0000 7fc32965a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc324062970 0x7fc324062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.426+0000 7fc32965a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc324062970 0x7fc324062d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51844/0 (socket says 172.21.15.29:51844) 2024-01-24T15:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.426+0000 7fc32965a700 1 -- 172.21.15.29:0/1798814620 learned_addr learned my addr 172.21.15.29:0/1798814620 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.427+0000 7fc32965a700 1 -- 172.21.15.29:0/1798814620 >> v2:172.21.15.163:3300/0 conn(0x7fc324158600 msgr2=0x7fc324154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.427+0000 7fc329e5b700 1 --2- 172.21.15.29:0/1798814620 >> v2:172.21.15.163:3300/0 conn(0x7fc324158600 0x7fc324154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.427+0000 7fc32965a700 1 --2- 172.21.15.29:0/1798814620 >> v2:172.21.15.163:3300/0 conn(0x7fc324158600 0x7fc324154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.427+0000 7fc32965a700 1 -- 172.21.15.29:0/1798814620 >> v1:172.21.15.29:6789/0 conn(0x7fc324066b40 legacy=0x7fc3240c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.427+0000 7fc32965a700 1 -- 172.21.15.29:0/1798814620 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3240a6ec0 con 0x7fc324062970 2024-01-24T15:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.427+0000 7fc329e5b700 1 --2- 172.21.15.29:0/1798814620 >> v2:172.21.15.163:3300/0 conn(0x7fc324158600 0x7fc324154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.427+0000 7fc32965a700 1 --2- 172.21.15.29:0/1798814620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc324062970 0x7fc324062d50 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7fc32405f670 tx=0x7fc3140090f0).ready entity=mon.0 client_cookie=b885becfc7506297 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.427+0000 7fc31bfff700 1 -- 172.21.15.29:0/1798814620 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31400b600 con 0x7fc324062970 2024-01-24T15:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.427+0000 7fc31bfff700 1 -- 172.21.15.29:0/1798814620 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31400b760 con 0x7fc324062970 2024-01-24T15:13:42.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.427+0000 7fc31bfff700 1 -- 172.21.15.29:0/1798814620 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31400bae0 con 0x7fc324062970 2024-01-24T15:13:42.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.428+0000 7fc32b8be700 1 -- 172.21.15.29:0/1798814620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc324062970 msgr2=0x7fc324062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.428+0000 7fc32b8be700 1 --2- 172.21.15.29:0/1798814620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc324062970 0x7fc324062d50 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7fc32405f670 tx=0x7fc3140090f0).stop 2024-01-24T15:13:42.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.428+0000 7fc32b8be700 1 -- 172.21.15.29:0/1798814620 shutdown_connections 2024-01-24T15:13:42.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.428+0000 7fc32b8be700 1 --2- 172.21.15.29:0/1798814620 >> v2:172.21.15.163:3300/0 conn(0x7fc324158600 0x7fc324154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.428+0000 7fc32b8be700 1 --2- 172.21.15.29:0/1798814620 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc324062970 0x7fc324062d50 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.428+0000 7fc32b8be700 1 -- 172.21.15.29:0/1798814620 >> 172.21.15.29:0/1798814620 conn(0x7fc3240c8580 msgr2=0x7fc3240c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:42.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.428+0000 7fc32b8be700 1 -- 172.21.15.29:0/1798814620 shutdown_connections 2024-01-24T15:13:42.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.428+0000 7fc32b8be700 1 -- 172.21.15.29:0/1798814620 wait complete. 2024-01-24T15:13:42.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.429+0000 7fc32b8be700 1 Processor -- start 2024-01-24T15:13:42.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.429+0000 7fc32b8be700 1 -- start start 2024-01-24T15:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.430+0000 7fc32b8be700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc324062970 0x7fc3240aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.430+0000 7fc32b8be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc324066b40 0x7fc3240ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.430+0000 7fc32b8be700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc324158600 0x7fc3240ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.430+0000 7fc32b8be700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3240a89e0 con 0x7fc324062970 2024-01-24T15:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.430+0000 7fc32b8be700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3240ab950 con 0x7fc324066b40 2024-01-24T15:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.430+0000 7fc32b8be700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3240a8420 con 0x7fc324158600 2024-01-24T15:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.430+0000 7fc32965a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc324062970 0x7fc3240aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.430+0000 7fc32965a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc324062970 0x7fc3240aaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51854/0 (socket says 172.21.15.29:51854) 2024-01-24T15:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.430+0000 7fc32965a700 1 -- 172.21.15.29:0/1484699484 learned_addr learned my addr 172.21.15.29:0/1484699484 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.430+0000 7fc328e59700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc324066b40 0x7fc3240ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.430+0000 7fc329e5b700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc324158600 0x7fc3240ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.431+0000 7fc32965a700 1 -- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc324158600 msgr2=0x7fc3240ad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.431+0000 7fc32965a700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc324158600 0x7fc3240ad500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.431+0000 7fc32965a700 1 -- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc324066b40 msgr2=0x7fc3240ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.431+0000 7fc32965a700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc324066b40 0x7fc3240ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.431+0000 7fc32965a700 1 -- 172.21.15.29:0/1484699484 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc314009370 con 0x7fc324062970 2024-01-24T15:13:42.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.431+0000 7fc329e5b700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc324158600 0x7fc3240ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:42.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.431+0000 7fc32965a700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc324062970 0x7fc3240aaaa0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7fc314009130 tx=0x7fc314017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:42.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.431+0000 7fc3197fa700 1 -- 172.21.15.29:0/1484699484 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31400b8d0 con 0x7fc324062970 2024-01-24T15:13:42.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.431+0000 7fc3197fa700 1 -- 172.21.15.29:0/1484699484 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31400ba70 con 0x7fc324062970 2024-01-24T15:13:42.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.431+0000 7fc3197fa700 1 -- 172.21.15.29:0/1484699484 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31401b660 con 0x7fc324062970 2024-01-24T15:13:42.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.431+0000 7fc32b8be700 1 -- 172.21.15.29:0/1484699484 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3240a8700 con 0x7fc324062970 2024-01-24T15:13:42.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.431+0000 7fc32b8be700 1 -- 172.21.15.29:0/1484699484 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3240aaf90 con 0x7fc324062970 2024-01-24T15:13:42.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.432+0000 7fc32b8be700 1 -- 172.21.15.29:0/1484699484 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3240475e0 con 0x7fc324062970 2024-01-24T15:13:42.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.433+0000 7fc3197fa700 1 -- 172.21.15.29:0/1484699484 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc31401b9d0 con 0x7fc324062970 2024-01-24T15:13:42.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.433+0000 7fc3197fa700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc30805f270 0x7fc308061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.433+0000 7fc3197fa700 1 -- 172.21.15.29:0/1484699484 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fc31407bcc0 con 0x7fc324062970 2024-01-24T15:13:42.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.433+0000 7fc328e59700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc30805f270 0x7fc308061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.434+0000 7fc328e59700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc30805f270 0x7fc308061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc324167e20 tx=0x7fc320007d40).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:42.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.438+0000 7fc3197fa700 1 -- 172.21.15.29:0/1484699484 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc31404c9e0 con 0x7fc324062970 2024-01-24T15:13:42.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.616+0000 7fc32b8be700 1 -- 172.21.15.29:0/1484699484 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc3240c5700 con 0x7fc30805f270 2024-01-24T15:13:42.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.627+0000 7fc3197fa700 1 -- 172.21.15.29:0/1484699484 <== mgr.8056 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+354365 (secure 0 0 0) 0x7fc3240c5700 con 0x7fc30805f270 2024-01-24T15:13:42.633 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:42.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.631+0000 7fc32b8be700 1 -- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc30805f270 msgr2=0x7fc308061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.631+0000 7fc32b8be700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc30805f270 0x7fc308061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc324167e20 tx=0x7fc320007d40).stop 2024-01-24T15:13:42.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.631+0000 7fc32b8be700 1 -- 172.21.15.29:0/1484699484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc324062970 msgr2=0x7fc3240aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.631+0000 7fc32b8be700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc324062970 0x7fc3240aaaa0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7fc314009130 tx=0x7fc314017000).stop 2024-01-24T15:13:42.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.631+0000 7fc32b8be700 1 -- 172.21.15.29:0/1484699484 shutdown_connections 2024-01-24T15:13:42.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.631+0000 7fc32b8be700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc324158600 0x7fc3240ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.631+0000 7fc32b8be700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc30805f270 0x7fc308061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.631+0000 7fc32b8be700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc324062970 0x7fc3240aaaa0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.631+0000 7fc32b8be700 1 --2- 172.21.15.29:0/1484699484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc324066b40 0x7fc3240ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.631+0000 7fc32b8be700 1 -- 172.21.15.29:0/1484699484 >> 172.21.15.29:0/1484699484 conn(0x7fc3240c8580 msgr2=0x7fc3240c4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:42.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.631+0000 7fc32b8be700 1 -- 172.21.15.29:0/1484699484 shutdown_connections 2024-01-24T15:13:42.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.631+0000 7fc32b8be700 1 -- 172.21.15.29:0/1484699484 wait complete. 2024-01-24T15:13:42.637 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:13:42.646 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T15:13:41.038118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2090,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":106,"acting":106,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2451220,"kb_used_data":17108,"kb_used_omap":0,"kb_used_meta":2434048,"kb_avail":752523500,"statfs":{"total":773094113280,"available":770584064000,"internally_reserved":0,"allocated":17518592,"data_stored":10014904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2492465152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5037063,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-522,"num_write_kb":-5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.997696"},"pg_stats":[{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550222+0000","last_change":"2024-01-24T15:13:38.543675+0000","last_active":"2024-01-24T15:13:38.550222+0000","last_peered":"2024-01-24T15:13:38.550222+0000","last_clean":"2024-01-24T15:13:38.550222+0000","last_became_active":"2024-01-24T15:13:38.543012+0000","last_became_peered":"2024-01-24T15:13:38.543012+0000","last_unstale":"2024-01-24T15:13:38.550222+0000","last_undegraded":"2024-01-24T15:13:38.550222+0000","last_fullsized":"2024-01-24T15:13:38.550222+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3f","version":"177'3","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545770+0000","last_change":"2024-01-24T15:13:38.128076+0000","last_active":"2024-01-24T15:13:38.545770+0000","last_peered":"2024-01-24T15:13:38.545770+0000","last_clean":"2024-01-24T15:13:38.545770+0000","last_became_active":"2024-01-24T15:13:38.127791+0000","last_became_peered":"2024-01-24T15:13:38.127791+0000","last_unstale":"2024-01-24T15:13:38.545770+0000","last_undegraded":"2024-01-24T15:13:38.545770+0000","last_fullsized":"2024-01-24T15:13:38.545770+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557709+0000","last_change":"2024-01-24T15:13:38.122839+0000","last_active":"2024-01-24T15:13:38.557709+0000","last_peered":"2024-01-24T15:13:38.557709+0000","last_clean":"2024-01-24T15:13:38.557709+0000","last_became_active":"2024-01-24T15:13:38.121646+0000","last_became_peered":"2024-01-24T15:13:38.121646+0000","last_unstale":"2024-01-24T15:13:38.557709+0000","last_undegraded":"2024-01-24T15:13:38.557709+0000","last_fullsized":"2024-01-24T15:13:38.557709+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559220+0000","last_change":"2024-01-24T15:13:38.556825+0000","last_active":"2024-01-24T15:13:38.559220+0000","last_peered":"2024-01-24T15:13:38.559220+0000","last_clean":"2024-01-24T15:13:38.559220+0000","last_became_active":"2024-01-24T15:13:38.556530+0000","last_became_peered":"2024-01-24T15:13:38.556530+0000","last_unstale":"2024-01-24T15:13:38.559220+0000","last_undegraded":"2024-01-24T15:13:38.559220+0000","last_fullsized":"2024-01-24T15:13:38.559220+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557777+0000","last_change":"2024-01-24T15:13:38.555214+0000","last_active":"2024-01-24T15:13:38.557777+0000","last_peered":"2024-01-24T15:13:38.557777+0000","last_clean":"2024-01-24T15:13:38.557777+0000","last_became_active":"2024-01-24T15:13:38.554915+0000","last_became_peered":"2024-01-24T15:13:38.554915+0000","last_unstale":"2024-01-24T15:13:38.557777+0000","last_undegraded":"2024-01-24T15:13:38.557777+0000","last_fullsized":"2024-01-24T15:13:38.557777+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.539484+0000","last_change":"2024-01-24T15:13:38.127556+0000","last_active":"2024-01-24T15:13:38.539484+0000","last_peered":"2024-01-24T15:13:38.539484+0000","last_clean":"2024-01-24T15:13:38.539484+0000","last_became_active":"2024-01-24T15:13:38.127215+0000","last_became_peered":"2024-01-24T15:13:38.127215+0000","last_unstale":"2024-01-24T15:13:38.539484+0000","last_undegraded":"2024-01-24T15:13:38.539484+0000","last_fullsized":"2024-01-24T15:13:38.539484+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.565384+0000","last_change":"2024-01-24T15:13:38.123112+0000","last_active":"2024-01-24T15:13:38.565384+0000","last_peered":"2024-01-24T15:13:38.565384+0000","last_clean":"2024-01-24T15:13:38.565384+0000","last_became_active":"2024-01-24T15:13:38.122820+0000","last_became_peered":"2024-01-24T15:13:38.122820+0000","last_unstale":"2024-01-24T15:13:38.565384+0000","last_undegraded":"2024-01-24T15:13:38.565384+0000","last_fullsized":"2024-01-24T15:13:38.565384+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528656+0000","last_change":"2024-01-24T15:13:38.125651+0000","last_active":"2024-01-24T15:13:38.528656+0000","last_peered":"2024-01-24T15:13:38.528656+0000","last_clean":"2024-01-24T15:13:38.528656+0000","last_became_active":"2024-01-24T15:13:38.125334+0000","last_became_peered":"2024-01-24T15:13:38.125334+0000","last_unstale":"2024-01-24T15:13:38.528656+0000","last_undegraded":"2024-01-24T15:13:38.528656+0000","last_fullsized":"2024-01-24T15:13:38.528656+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545374+0000","last_change":"2024-01-24T15:13:38.543804+0000","last_active":"2024-01-24T15:13:38.545374+0000","last_peered":"2024-01-24T15:13:38.545374+0000","last_clean":"2024-01-24T15:13:38.545374+0000","last_became_active":"2024-01-24T15:13:38.543259+0000","last_became_peered":"2024-01-24T15:13:38.543259+0000","last_unstale":"2024-01-24T15:13:38.545374+0000","last_undegraded":"2024-01-24T15:13:38.545374+0000","last_fullsized":"2024-01-24T15:13:38.545374+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528152+0000","last_change":"2024-01-24T15:13:38.125234+0000","last_active":"2024-01-24T15:13:38.528152+0000","last_peered":"2024-01-24T15:13:38.528152+0000","last_clean":"2024-01-24T15:13:38.528152+0000","last_became_active":"2024-01-24T15:13:38.124520+0000","last_became_peered":"2024-01-24T15:13:38.124520+0000","last_unstale":"2024-01-24T15:13:38.528152+0000","last_undegraded":"2024-01-24T15:13:38.528152+0000","last_fullsized":"2024-01-24T15:13:38.528152+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532043+0000","last_change":"2024-01-24T15:13:38.122045+0000","last_active":"2024-01-24T15:13:38.532043+0000","last_peered":"2024-01-24T15:13:38.532043+0000","last_clean":"2024-01-24T15:13:38.532043+0000","last_became_active":"2024-01-24T15:13:38.121766+0000","last_became_peered":"2024-01-24T15:13:38.121766+0000","last_unstale":"2024-01-24T15:13:38.532043+0000","last_undegraded":"2024-01-24T15:13:38.532043+0000","last_fullsized":"2024-01-24T15:13:38.532043+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545141+0000","last_change":"2024-01-24T15:13:38.541723+0000","last_active":"2024-01-24T15:13:38.545141+0000","last_peered":"2024-01-24T15:13:38.545141+0000","last_clean":"2024-01-24T15:13:38.545141+0000","last_became_active":"2024-01-24T15:13:38.540083+0000","last_became_peered":"2024-01-24T15:13:38.540083+0000","last_unstale":"2024-01-24T15:13:38.545141+0000","last_undegraded":"2024-01-24T15:13:38.545141+0000","last_fullsized":"2024-01-24T15:13:38.545141+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532179+0000","last_change":"2024-01-24T15:13:38.128558+0000","last_active":"2024-01-24T15:13:38.532179+0000","last_peered":"2024-01-24T15:13:38.532179+0000","last_clean":"2024-01-24T15:13:38.532179+0000","last_became_active":"2024-01-24T15:13:38.128075+0000","last_became_peered":"2024-01-24T15:13:38.128075+0000","last_unstale":"2024-01-24T15:13:38.532179+0000","last_undegraded":"2024-01-24T15:13:38.532179+0000","last_fullsized":"2024-01-24T15:13:38.532179+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528767+0000","last_change":"2024-01-24T15:13:38.119386+0000","last_active":"2024-01-24T15:13:38.528767+0000","last_peered":"2024-01-24T15:13:38.528767+0000","last_clean":"2024-01-24T15:13:38.528767+0000","last_became_active":"2024-01-24T15:13:38.118881+0000","last_became_peered":"2024-01-24T15:13:38.118881+0000","last_unstale":"2024-01-24T15:13:38.528767+0000","last_undegraded":"2024-01-24T15:13:38.528767+0000","last_fullsized":"2024-01-24T15:13:38.528767+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558719+0000","last_change":"2024-01-24T15:13:38.555417+0000","last_active":"2024-01-24T15:13:38.558719+0000","last_peered":"2024-01-24T15:13:38.558719+0000","last_clean":"2024-01-24T15:13:38.558719+0000","last_became_active":"2024-01-24T15:13:38.555118+0000","last_became_peered":"2024-01-24T15:13:38.555118+0000","last_unstale":"2024-01-24T15:13:38.558719+0000","last_undegraded":"2024-01-24T15:13:38.558719+0000","last_fullsized":"2024-01-24T15:13:38.558719+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562480+0000","last_change":"2024-01-24T15:13:38.561900+0000","last_active":"2024-01-24T15:13:38.562480+0000","last_peered":"2024-01-24T15:13:38.562480+0000","last_clean":"2024-01-24T15:13:38.562480+0000","last_became_active":"2024-01-24T15:13:38.561655+0000","last_became_peered":"2024-01-24T15:13:38.561655+0000","last_unstale":"2024-01-24T15:13:38.562480+0000","last_undegraded":"2024-01-24T15:13:38.562480+0000","last_fullsized":"2024-01-24T15:13:38.562480+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.548146+0000","last_change":"2024-01-24T15:13:38.546025+0000","last_active":"2024-01-24T15:13:38.548146+0000","last_peered":"2024-01-24T15:13:38.548146+0000","last_clean":"2024-01-24T15:13:38.548146+0000","last_became_active":"2024-01-24T15:13:38.545702+0000","last_became_peered":"2024-01-24T15:13:38.545702+0000","last_unstale":"2024-01-24T15:13:38.548146+0000","last_undegraded":"2024-01-24T15:13:38.548146+0000","last_fullsized":"2024-01-24T15:13:38.548146+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.546793+0000","last_change":"2024-01-24T15:13:38.128387+0000","last_active":"2024-01-24T15:13:38.546793+0000","last_peered":"2024-01-24T15:13:38.546793+0000","last_clean":"2024-01-24T15:13:38.546793+0000","last_became_active":"2024-01-24T15:13:38.128088+0000","last_became_peered":"2024-01-24T15:13:38.128088+0000","last_unstale":"2024-01-24T15:13:38.546793+0000","last_undegraded":"2024-01-24T15:13:38.546793+0000","last_fullsized":"2024-01-24T15:13:38.546793+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545355+0000","last_change":"2024-01-24T15:13:38.543615+0000","last_active":"2024-01-24T15:13:38.545355+0000","last_peered":"2024-01-24T15:13:38.545355+0000","last_clean":"2024-01-24T15:13:38.545355+0000","last_became_active":"2024-01-24T15:13:38.541511+0000","last_became_peered":"2024-01-24T15:13:38.541511+0000","last_unstale":"2024-01-24T15:13:38.545355+0000","last_undegraded":"2024-01-24T15:13:38.545355+0000","last_fullsized":"2024-01-24T15:13:38.545355+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550198+0000","last_change":"2024-01-24T15:13:38.546422+0000","last_active":"2024-01-24T15:13:38.550198+0000","last_peered":"2024-01-24T15:13:38.550198+0000","last_clean":"2024-01-24T15:13:38.550198+0000","last_became_active":"2024-01-24T15:13:38.546168+0000","last_became_peered":"2024-01-24T15:13:38.546168+0000","last_unstale":"2024-01-24T15:13:38.550198+0000","last_undegraded":"2024-01-24T15:13:38.550198+0000","last_fullsized":"2024-01-24T15:13:38.550198+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.538459+0000","last_change":"2024-01-24T15:13:38.126860+0000","last_active":"2024-01-24T15:13:38.538459+0000","last_peered":"2024-01-24T15:13:38.538459+0000","last_clean":"2024-01-24T15:13:38.538459+0000","last_became_active":"2024-01-24T15:13:38.126528+0000","last_became_peered":"2024-01-24T15:13:38.126528+0000","last_unstale":"2024-01-24T15:13:38.538459+0000","last_undegraded":"2024-01-24T15:13:38.538459+0000","last_fullsized":"2024-01-24T15:13:38.538459+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.547990+0000","last_change":"2024-01-24T15:13:38.542686+0000","last_active":"2024-01-24T15:13:38.547990+0000","last_peered":"2024-01-24T15:13:38.547990+0000","last_clean":"2024-01-24T15:13:38.547990+0000","last_became_active":"2024-01-24T15:13:38.542392+0000","last_became_peered":"2024-01-24T15:13:38.542392+0000","last_unstale":"2024-01-24T15:13:38.547990+0000","last_undegraded":"2024-01-24T15:13:38.547990+0000","last_fullsized":"2024-01-24T15:13:38.547990+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.548937+0000","last_change":"2024-01-24T15:13:38.125904+0000","last_active":"2024-01-24T15:13:38.548937+0000","last_peered":"2024-01-24T15:13:38.548937+0000","last_clean":"2024-01-24T15:13:38.548937+0000","last_became_active":"2024-01-24T15:13:38.125609+0000","last_became_peered":"2024-01-24T15:13:38.125609+0000","last_unstale":"2024-01-24T15:13:38.548937+0000","last_undegraded":"2024-01-24T15:13:38.548937+0000","last_fullsized":"2024-01-24T15:13:38.548937+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550294+0000","last_change":"2024-01-24T15:13:38.547021+0000","last_active":"2024-01-24T15:13:38.550294+0000","last_peered":"2024-01-24T15:13:38.550294+0000","last_clean":"2024-01-24T15:13:38.550294+0000","last_became_active":"2024-01-24T15:13:38.546231+0000","last_became_peered":"2024-01-24T15:13:38.546231+0000","last_unstale":"2024-01-24T15:13:38.550294+0000","last_undegraded":"2024-01-24T15:13:38.550294+0000","last_fullsized":"2024-01-24T15:13:38.550294+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528666+0000","last_change":"2024-01-24T15:13:38.124808+0000","last_active":"2024-01-24T15:13:38.528666+0000","last_peered":"2024-01-24T15:13:38.528666+0000","last_clean":"2024-01-24T15:13:38.528666+0000","last_became_active":"2024-01-24T15:13:38.124504+0000","last_became_peered":"2024-01-24T15:13:38.124504+0000","last_unstale":"2024-01-24T15:13:38.528666+0000","last_undegraded":"2024-01-24T15:13:38.528666+0000","last_fullsized":"2024-01-24T15:13:38.528666+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542376+0000","last_change":"2024-01-24T15:13:38.540674+0000","last_active":"2024-01-24T15:13:38.542376+0000","last_peered":"2024-01-24T15:13:38.542376+0000","last_clean":"2024-01-24T15:13:38.542376+0000","last_became_active":"2024-01-24T15:13:38.540381+0000","last_became_peered":"2024-01-24T15:13:38.540381+0000","last_unstale":"2024-01-24T15:13:38.542376+0000","last_undegraded":"2024-01-24T15:13:38.542376+0000","last_fullsized":"2024-01-24T15:13:38.542376+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528151+0000","last_change":"2024-01-24T15:13:38.119586+0000","last_active":"2024-01-24T15:13:38.528151+0000","last_peered":"2024-01-24T15:13:38.528151+0000","last_clean":"2024-01-24T15:13:38.528151+0000","last_became_active":"2024-01-24T15:13:38.119285+0000","last_became_peered":"2024-01-24T15:13:38.119285+0000","last_unstale":"2024-01-24T15:13:38.528151+0000","last_undegraded":"2024-01-24T15:13:38.528151+0000","last_fullsized":"2024-01-24T15:13:38.528151+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557360+0000","last_change":"2024-01-24T15:13:38.549783+0000","last_active":"2024-01-24T15:13:38.557360+0000","last_peered":"2024-01-24T15:13:38.557360+0000","last_clean":"2024-01-24T15:13:38.557360+0000","last_became_active":"2024-01-24T15:13:38.549374+0000","last_became_peered":"2024-01-24T15:13:38.549374+0000","last_unstale":"2024-01-24T15:13:38.557360+0000","last_undegraded":"2024-01-24T15:13:38.557360+0000","last_fullsized":"2024-01-24T15:13:38.557360+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557933+0000","last_change":"2024-01-24T15:13:38.121798+0000","last_active":"2024-01-24T15:13:38.557933+0000","last_peered":"2024-01-24T15:13:38.557933+0000","last_clean":"2024-01-24T15:13:38.557933+0000","last_became_active":"2024-01-24T15:13:38.121333+0000","last_became_peered":"2024-01-24T15:13:38.121333+0000","last_unstale":"2024-01-24T15:13:38.557933+0000","last_undegraded":"2024-01-24T15:13:38.557933+0000","last_fullsized":"2024-01-24T15:13:38.557933+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.529610+0000","last_change":"2024-01-24T15:13:38.125875+0000","last_active":"2024-01-24T15:13:38.529610+0000","last_peered":"2024-01-24T15:13:38.529610+0000","last_clean":"2024-01-24T15:13:38.529610+0000","last_became_active":"2024-01-24T15:13:38.125536+0000","last_became_peered":"2024-01-24T15:13:38.125536+0000","last_unstale":"2024-01-24T15:13:38.529610+0000","last_undegraded":"2024-01-24T15:13:38.529610+0000","last_fullsized":"2024-01-24T15:13:38.529610+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532219+0000","last_change":"2024-01-24T15:13:38.122494+0000","last_active":"2024-01-24T15:13:38.532219+0000","last_peered":"2024-01-24T15:13:38.532219+0000","last_clean":"2024-01-24T15:13:38.532219+0000","last_became_active":"2024-01-24T15:13:38.122120+0000","last_became_peered":"2024-01-24T15:13:38.122120+0000","last_unstale":"2024-01-24T15:13:38.532219+0000","last_undegraded":"2024-01-24T15:13:38.532219+0000","last_fullsized":"2024-01-24T15:13:38.532219+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561628+0000","last_change":"2024-01-24T15:13:38.129725+0000","last_active":"2024-01-24T15:13:38.561628+0000","last_peered":"2024-01-24T15:13:38.561628+0000","last_clean":"2024-01-24T15:13:38.561628+0000","last_became_active":"2024-01-24T15:13:38.129199+0000","last_became_peered":"2024-01-24T15:13:38.129199+0000","last_unstale":"2024-01-24T15:13:38.561628+0000","last_undegraded":"2024-01-24T15:13:38.561628+0000","last_fullsized":"2024-01-24T15:13:38.561628+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556320+0000","last_change":"2024-01-24T15:13:38.550473+0000","last_active":"2024-01-24T15:13:38.556320+0000","last_peered":"2024-01-24T15:13:38.556320+0000","last_clean":"2024-01-24T15:13:38.556320+0000","last_became_active":"2024-01-24T15:13:38.550227+0000","last_became_peered":"2024-01-24T15:13:38.550227+0000","last_unstale":"2024-01-24T15:13:38.556320+0000","last_undegraded":"2024-01-24T15:13:38.556320+0000","last_fullsized":"2024-01-24T15:13:38.556320+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556378+0000","last_change":"2024-01-24T15:13:38.552294+0000","last_active":"2024-01-24T15:13:38.556378+0000","last_peered":"2024-01-24T15:13:38.556378+0000","last_clean":"2024-01-24T15:13:38.556378+0000","last_became_active":"2024-01-24T15:13:38.551919+0000","last_became_peered":"2024-01-24T15:13:38.551919+0000","last_unstale":"2024-01-24T15:13:38.556378+0000","last_undegraded":"2024-01-24T15:13:38.556378+0000","last_fullsized":"2024-01-24T15:13:38.556378+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561397+0000","last_change":"2024-01-24T15:13:38.557841+0000","last_active":"2024-01-24T15:13:38.561397+0000","last_peered":"2024-01-24T15:13:38.561397+0000","last_clean":"2024-01-24T15:13:38.561397+0000","last_became_active":"2024-01-24T15:13:38.557543+0000","last_became_peered":"2024-01-24T15:13:38.557543+0000","last_unstale":"2024-01-24T15:13:38.561397+0000","last_undegraded":"2024-01-24T15:13:38.561397+0000","last_fullsized":"2024-01-24T15:13:38.561397+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558444+0000","last_change":"2024-01-24T15:13:38.552923+0000","last_active":"2024-01-24T15:13:38.558444+0000","last_peered":"2024-01-24T15:13:38.558444+0000","last_clean":"2024-01-24T15:13:38.558444+0000","last_became_active":"2024-01-24T15:13:38.552646+0000","last_became_peered":"2024-01-24T15:13:38.552646+0000","last_unstale":"2024-01-24T15:13:38.558444+0000","last_undegraded":"2024-01-24T15:13:38.558444+0000","last_fullsized":"2024-01-24T15:13:38.558444+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.539981+0000","last_change":"2024-01-24T15:13:38.129534+0000","last_active":"2024-01-24T15:13:38.539981+0000","last_peered":"2024-01-24T15:13:38.539981+0000","last_clean":"2024-01-24T15:13:38.539981+0000","last_became_active":"2024-01-24T15:13:38.128874+0000","last_became_peered":"2024-01-24T15:13:38.128874+0000","last_unstale":"2024-01-24T15:13:38.539981+0000","last_undegraded":"2024-01-24T15:13:38.539981+0000","last_fullsized":"2024-01-24T15:13:38.539981+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.531693+0000","last_change":"2024-01-24T15:13:38.124495+0000","last_active":"2024-01-24T15:13:38.531693+0000","last_peered":"2024-01-24T15:13:38.531693+0000","last_clean":"2024-01-24T15:13:38.531693+0000","last_became_active":"2024-01-24T15:13:38.124175+0000","last_became_peered":"2024-01-24T15:13:38.124175+0000","last_unstale":"2024-01-24T15:13:38.531693+0000","last_undegraded":"2024-01-24T15:13:38.531693+0000","last_fullsized":"2024-01-24T15:13:38.531693+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559906+0000","last_change":"2024-01-24T15:13:38.557558+0000","last_active":"2024-01-24T15:13:38.559906+0000","last_peered":"2024-01-24T15:13:38.559906+0000","last_clean":"2024-01-24T15:13:38.559906+0000","last_became_active":"2024-01-24T15:13:38.557290+0000","last_became_peered":"2024-01-24T15:13:38.557290+0000","last_unstale":"2024-01-24T15:13:38.559906+0000","last_undegraded":"2024-01-24T15:13:38.559906+0000","last_fullsized":"2024-01-24T15:13:38.559906+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.058886+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:13:33.058886+0000","last_peered":"2024-01-24T15:13:33.058886+0000","last_clean":"2024-01-24T15:13:33.058886+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:13:33.058886+0000","last_undegraded":"2024-01-24T15:13:33.058886+0000","last_fullsized":"2024-01-24T15:13:33.058886+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558463+0000","last_change":"2024-01-24T15:13:38.552222+0000","last_active":"2024-01-24T15:13:38.558463+0000","last_peered":"2024-01-24T15:13:38.558463+0000","last_clean":"2024-01-24T15:13:38.558463+0000","last_became_active":"2024-01-24T15:13:38.551916+0000","last_became_peered":"2024-01-24T15:13:38.551916+0000","last_unstale":"2024-01-24T15:13:38.558463+0000","last_undegraded":"2024-01-24T15:13:38.558463+0000","last_fullsized":"2024-01-24T15:13:38.558463+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561816+0000","last_change":"2024-01-24T15:13:38.128196+0000","last_active":"2024-01-24T15:13:38.561816+0000","last_peered":"2024-01-24T15:13:38.561816+0000","last_clean":"2024-01-24T15:13:38.561816+0000","last_became_active":"2024-01-24T15:13:38.127782+0000","last_became_peered":"2024-01-24T15:13:38.127782+0000","last_unstale":"2024-01-24T15:13:38.561816+0000","last_undegraded":"2024-01-24T15:13:38.561816+0000","last_fullsized":"2024-01-24T15:13:38.561816+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":158,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.082678+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:13:33.082678+0000","last_peered":"2024-01-24T15:13:33.082678+0000","last_clean":"2024-01-24T15:13:33.082678+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:13:33.082678+0000","last_undegraded":"2024-01-24T15:13:33.082678+0000","last_fullsized":"2024-01-24T15:13:33.082678+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532992+0000","last_change":"2024-01-24T15:13:38.125400+0000","last_active":"2024-01-24T15:13:38.532992+0000","last_peered":"2024-01-24T15:13:38.532992+0000","last_clean":"2024-01-24T15:13:38.532992+0000","last_became_active":"2024-01-24T15:13:38.124725+0000","last_became_peered":"2024-01-24T15:13:38.124725+0000","last_unstale":"2024-01-24T15:13:38.532992+0000","last_undegraded":"2024-01-24T15:13:38.532992+0000","last_fullsized":"2024-01-24T15:13:38.532992+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528231+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:13:38.528231+0000","last_peered":"2024-01-24T15:13:38.528231+0000","last_clean":"2024-01-24T15:13:38.528231+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:13:38.528231+0000","last_undegraded":"2024-01-24T15:13:38.528231+0000","last_fullsized":"2024-01-24T15:13:38.528231+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561686+0000","last_change":"2024-01-24T15:13:38.554780+0000","last_active":"2024-01-24T15:13:38.561686+0000","last_peered":"2024-01-24T15:13:38.561686+0000","last_clean":"2024-01-24T15:13:38.561686+0000","last_became_active":"2024-01-24T15:13:38.554497+0000","last_became_peered":"2024-01-24T15:13:38.554497+0000","last_unstale":"2024-01-24T15:13:38.561686+0000","last_undegraded":"2024-01-24T15:13:38.561686+0000","last_fullsized":"2024-01-24T15:13:38.561686+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":158,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.082682+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:13:33.082682+0000","last_peered":"2024-01-24T15:13:33.082682+0000","last_clean":"2024-01-24T15:13:33.082682+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:13:33.082682+0000","last_undegraded":"2024-01-24T15:13:33.082682+0000","last_fullsized":"2024-01-24T15:13:33.082682+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":143,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.933666+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:13:33.933666+0000","last_peered":"2024-01-24T15:13:33.933666+0000","last_clean":"2024-01-24T15:13:33.933666+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:13:33.933666+0000","last_undegraded":"2024-01-24T15:13:33.933666+0000","last_fullsized":"2024-01-24T15:13:33.933666+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.933467+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:13:33.933467+0000","last_peered":"2024-01-24T15:13:33.933467+0000","last_clean":"2024-01-24T15:13:33.933467+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:13:33.933467+0000","last_undegraded":"2024-01-24T15:13:33.933467+0000","last_fullsized":"2024-01-24T15:13:33.933467+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.933453+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:13:33.933453+0000","last_peered":"2024-01-24T15:13:33.933453+0000","last_clean":"2024-01-24T15:13:33.933453+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:13:33.933453+0000","last_undegraded":"2024-01-24T15:13:33.933453+0000","last_fullsized":"2024-01-24T15:13:33.933453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.549726+0000","last_change":"2024-01-24T15:13:38.121778+0000","last_active":"2024-01-24T15:13:38.549726+0000","last_peered":"2024-01-24T15:13:38.549726+0000","last_clean":"2024-01-24T15:13:38.549726+0000","last_became_active":"2024-01-24T15:13:38.121489+0000","last_became_peered":"2024-01-24T15:13:38.121489+0000","last_unstale":"2024-01-24T15:13:38.549726+0000","last_undegraded":"2024-01-24T15:13:38.549726+0000","last_fullsized":"2024-01-24T15:13:38.549726+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":140,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.058947+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:13:33.058947+0000","last_peered":"2024-01-24T15:13:33.058947+0000","last_clean":"2024-01-24T15:13:33.058947+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:13:33.058947+0000","last_undegraded":"2024-01-24T15:13:33.058947+0000","last_fullsized":"2024-01-24T15:13:33.058947+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":138,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-01-24T15:13:29.696776+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:13:29.696776+0000","last_peered":"2024-01-24T15:13:29.696776+0000","last_clean":"2024-01-24T15:13:29.696776+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:13:29.696776+0000","last_undegraded":"2024-01-24T15:13:29.696776+0000","last_fullsized":"2024-01-24T15:13:29.696776+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":3},{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":2090,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":46,"acting":46,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575750,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":2166784,"data_stored":1239804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.59899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.28000000000000003},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.628,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.28899999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.1080000000000001},"last":0.64900000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.33300000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.505,"5min":0.40000000000000002,"15min":0.34799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53300000000000003}]},{"osd":2,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0329999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.38},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0880000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.41199999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.33200000000000002},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.31900000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.67600000000000005}]},{"osd":5,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.39600000000000002,"5min":0.38,"15min":0.34799999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.439},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.36199999999999999},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.31900000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.46400000000000002,"5min":0.40899999999999997,"15min":0.40500000000000003},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.68899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575751,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":2166784,"data_stored":1239804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.86599999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.42699999999999999,"15min":0.37},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64500000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.84199999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77100000000000002,"5min":0.81200000000000006,"15min":1.258},"last":0.51600000000000001}]},{"osd":2,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.40100000000000002},"max":{"1min":0.82299999999999995,"5min":0.85899999999999999,"15min":1.2430000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.47199999999999998,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.498},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.39000000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.317},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32400000000000001,"15min":0.30499999999999999},"max":{"1min":0.79300000000000004,"5min":0.80100000000000005,"15min":1.002},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.32500000000000001},"max":{"1min":0.76000000000000001,"5min":0.78100000000000003,"15min":0.83899999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":0.749,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.40899999999999997,"5min":0.372,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.90100000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.38},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":1.0429999999999999},"last":0.65800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575750,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307404,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064436,"statfs":{"total":96636764160,"available":96321982464,"internally_reserved":0,"allocated":2166784,"data_stored":1239804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.56399999999999995,"5min":0.754,"15min":0.754},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.443,"15min":0.435},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.26800000000000002},"max":{"1min":0.55800000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.40200000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.33700000000000002,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.57599999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44600000000000001,"15min":0.44700000000000001},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.28199999999999997},"max":{"1min":0.56200000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.435}]},{"osd":3,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"min":{"1min":0.32400000000000001,"5min":0.314,"15min":0.309},"max":{"1min":0.58399999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.35799999999999998,"15min":0.313},"max":{"1min":0.59899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.51800000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.86099999999999999,"5min":1.044,"15min":1.0649999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.51400000000000001,"5min":0.47399999999999998,"15min":0.41699999999999998},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63500000000000001}]},{"osd":5,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.49199999999999999,"15min":0.46300000000000002},"max":{"1min":0.83999999999999997,"5min":1.014,"15min":1.0780000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.439},"max":{"1min":0.86699999999999999,"5min":1.105,"15min":1.105},"last":0.85099999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.874,"5min":1.0029999999999999,"15min":1.069},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.56599999999999995,"5min":0.499,"15min":0.48199999999999998},"max":{"1min":0.92500000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.871}]},{"osd":7,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.83799999999999997,"5min":1.127,"15min":1.127},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.77500000000000002,"15min":0.76700000000000002},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.42799999999999999},"max":{"1min":0.88900000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.84399999999999997}]}]},{"osd":0,"up_from":14,"seq":60129543049,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307464,"kb_used_data":2176,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064376,"statfs":{"total":96636764160,"available":96321921024,"internally_reserved":0,"allocated":2228224,"data_stored":1271118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.28100000000000003,"5min":0.27500000000000002,"15min":0.247},"max":{"1min":0.60199999999999998,"5min":0.63400000000000001,"15min":0.66400000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.27700000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.54500000000000004}]},{"osd":2,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.29399999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.81999999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.84399999999999997},"last":0.496}]},{"osd":3,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.44600000000000001},"min":{"1min":0.317,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.65500000000000003,"15min":0.67000000000000004},"last":0.626},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.52000000000000002},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.90000000000000002},"last":0.66300000000000003}]},{"osd":4,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.433},"max":{"1min":0.99399999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.38700000000000001},"max":{"1min":0.995,"5min":0.995,"15min":1.0249999999999999},"last":0.78100000000000003}]},{"osd":5,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.70699999999999996},"min":{"1min":0.54900000000000004,"5min":0.498,"15min":0.42099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.94199999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.46899999999999997},"max":{"1min":1.022,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0820000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81100000000000005}]},{"osd":7,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0469999999999999,"5min":1.143,"15min":1.3260000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.375},"last":0.87}]}]},{"osd":2,"up_from":13,"seq":55834575751,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311564,"kb_used_data":2180,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060276,"statfs":{"total":96636764160,"available":96317722624,"internally_reserved":0,"allocated":2232320,"data_stored":1272937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.27200000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.60099999999999998,"5min":0.65000000000000002,"15min":0.66800000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.45100000000000001,"15min":0.45200000000000001},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.70399999999999996},"last":0.29099999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.25600000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.73599999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.46300000000000002},"min":{"1min":0.32500000000000001,"5min":0.29699999999999999,"15min":0.29599999999999999},"max":{"1min":0.622,"5min":0.69699999999999995,"15min":0.70099999999999996},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.73299999999999998},"last":0.34999999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96199999999999997,"5min":1.196,"15min":1.196},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":1,"15min":1.089},"last":0.38100000000000001}]},{"osd":5,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54400000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.111},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.46899999999999997,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.147,"15min":1.1830000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.64700000000000002,"5min":0.47699999999999998,"15min":0.45200000000000001},"max":{"1min":0.99399999999999999,"5min":0.998,"15min":1.129},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.58899999999999997,"5min":0.47299999999999998,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.98999999999999999,"15min":1.1950000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.753,"15min":0.746},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.497},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.248},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.65000000000000002,"5min":0.51500000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.24,"15min":1.24},"last":0.63100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307460,"kb_used_data":2172,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064380,"statfs":{"total":96636764160,"available":96321925120,"internally_reserved":0,"allocated":2224128,"data_stored":1271829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39800000000000002,"15min":0.39200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.247},"max":{"1min":0.56899999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.70199999999999996},"last":0.36199999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41899999999999998,"15min":0.41399999999999998},"min":{"1min":0.311,"5min":0.28100000000000003,"15min":0.27400000000000002},"max":{"1min":0.59999999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.34799999999999998,"5min":0.316,"15min":0.30499999999999999},"max":{"1min":0.67900000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.433}]},{"osd":2,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.497,"15min":0.502},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51300000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.41099999999999998},"max":{"1min":0.83299999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95999999999999996,"15min":1.5580000000000001},"last":0.58799999999999997}]},{"osd":5,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.1879999999999999,"15min":1.6240000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.647},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.45700000000000002,"15min":0.42699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.522},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.073,"5min":1.079,"15min":1.079},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.44800000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1499999999999999,"5min":1.246,"15min":1.246},"last":0.64400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307404,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064436,"statfs":{"total":96636764160,"available":96321982464,"internally_reserved":0,"allocated":2166784,"data_stored":1239804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.46700000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.83699999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76500000000000001,"5min":0.78400000000000003,"15min":0.84199999999999997},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.45500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.72499999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82999999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.61299999999999999}]},{"osd":2,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.84699999999999998,"15min":0.88300000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.626,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.372},"max":{"1min":0.79800000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.629}]},{"osd":3,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.38400000000000001},"max":{"1min":0.79900000000000004,"5min":0.80400000000000005,"15min":0.89500000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.42399999999999999},"max":{"1min":0.71899999999999997,"5min":0.82199999999999995,"15min":0.92300000000000004},"last":0.64000000000000001}]},{"osd":5,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":0.61299999999999999,"5min":0.79200000000000004,"15min":0.81799999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66400000000000003,"5min":0.72499999999999998,"15min":0.85499999999999998},"last":0.65800000000000003}]},{"osd":6,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.68300000000000005,"5min":0.78500000000000003,"15min":0.85499999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.38200000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.86799999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.33600000000000002},"max":{"1min":0.65200000000000002,"5min":0.77500000000000002,"15min":0.82799999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.72599999999999998,"5min":0.81999999999999995,"15min":0.96599999999999997},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575751,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":2166784,"data_stored":1239804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.38600000000000001,"15min":0.35099999999999998},"max":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":1.4470000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.81599999999999995},"last":0.53300000000000003}]},{"osd":1,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.82499999999999996},"last":0.435},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":1.4930000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.625,"15min":0.627},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.79300000000000004,"5min":0.80900000000000005,"15min":0.86499999999999999},"last":0.503},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":1.53},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.80800000000000005,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41899999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.65400000000000003,"5min":0.74399999999999999,"15min":0.84699999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":0.77200000000000002,"15min":0.80100000000000005},"last":0.38900000000000001}]},{"osd":6,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.5,"15min":0.504},"min":{"1min":0.374,"5min":0.35499999999999998,"15min":0.33400000000000002},"max":{"1min":0.65100000000000002,"5min":0.79400000000000004,"15min":0.81499999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.78500000000000003,"15min":0.81299999999999994},"last":0.44400000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.54900000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:13:42.652 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:13:42.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.817+0000 7f749291e700 1 Processor -- start 2024-01-24T15:13:42.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.817+0000 7f749291e700 1 -- start start 2024-01-24T15:13:42.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.818+0000 7f749291e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c062970 0x7f748c062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.818+0000 7f749291e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c0bcf60 con 0x7f748c066b40 2024-01-24T15:13:42.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.818+0000 7f749291e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c0a6ec0 con 0x7f748c158600 2024-01-24T15:13:42.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.818+0000 7f749291e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c0bdba0 con 0x7f748c062970 2024-01-24T15:13:42.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.818+0000 7f748b7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f748c066b40 0x7f748c0c4da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41180/0 (socket says 172.21.15.29:41180) 2024-01-24T15:13:42.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.818+0000 7f748b7fe700 1 -- 172.21.15.29:0/1052183918 learned_addr learned my addr 172.21.15.29:0/1052183918 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.818+0000 7f748bfff700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c062970 0x7f748c062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.818+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625867121 0 0) 0x7f748c0bcf60 con 0x7f748c066b40 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74740033f0 con 0x7f748c066b40 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3804791399 0 0) 0x7f74740033f0 con 0x7f748c066b40 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c062970 msgr2=0x7f748c062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c062970 0x7f748c062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 >> v1:172.21.15.163:6789/0 conn(0x7f748c158600 legacy=0x7f748c154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0a71a0 con 0x7f748c066b40 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f7480001470 con 0x7f748c066b40 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 >> v1:172.21.15.29:6789/0 conn(0x7f748c066b40 legacy=0x7f748c0c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7474003910 0x7f7474003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c062970 0x7f7474004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7474004b80 0x7f7474005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c0bdba0 con 0x7f7474003910 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747400b8e0 con 0x7f7474004b80 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747400ba50 con 0x7f748c062970 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748bfff700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7474003910 0x7f7474003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748b7fe700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c062970 0x7f7474004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f7490ebb700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7474004b80 0x7f7474005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748bfff700 1 -- 172.21.15.29:0/1052183918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c062970 msgr2=0x7f7474004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748bfff700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c062970 0x7f7474004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748bfff700 1 -- 172.21.15.29:0/1052183918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7474004b80 msgr2=0x7f7474005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748bfff700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7474004b80 0x7f7474005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f748bfff700 1 -- 172.21.15.29:0/1052183918 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74740033f0 con 0x7f7474003910 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.819+0000 7f7490ebb700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7474004b80 0x7f7474005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.820+0000 7f748bfff700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7474003910 0x7f7474003cf0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f747c009a60 tx=0x7f747c009060).ready entity=mon.0 client_cookie=772cd6a064cbee62 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:42.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.820+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c009ea0 con 0x7f7474003910 2024-01-24T15:13:42.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.820+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c009430 con 0x7f7474003910 2024-01-24T15:13:42.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.820+0000 7f748affd700 1 -- 172.21.15.29:0/1052183918 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00e810 con 0x7f7474003910 2024-01-24T15:13:42.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.820+0000 7f749291e700 1 -- 172.21.15.29:0/1052183918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7474003910 msgr2=0x7f7474003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.820+0000 7f749291e700 1 --2- 172.21.15.29:0/1052183918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7474003910 0x7f7474003cf0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f747c009a60 tx=0x7f747c009060).stop 2024-01-24T15:13:42.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.820+0000 7f7490ebb700 1 -- 172.21.15.29:0/1052183918 reap_dead start 2024-01-24T15:13:42.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.820+0000 7f749291e700 1 -- 172.21.15.29:0/1052183918 shutdown_connections 2024-01-24T15:13:42.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.820+0000 7f749291e700 1 -- 172.21.15.29:0/1052183918 >> 172.21.15.29:0/1052183918 conn(0x7f748c0c8580 msgr2=0x7f748c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:42.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.820+0000 7f749291e700 1 -- 172.21.15.29:0/1052183918 shutdown_connections 2024-01-24T15:13:42.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.820+0000 7f749291e700 1 -- 172.21.15.29:0/1052183918 wait complete. 2024-01-24T15:13:42.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.822+0000 7f749291e700 1 Processor -- start 2024-01-24T15:13:42.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.822+0000 7f749291e700 1 -- start start 2024-01-24T15:13:42.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.823+0000 7f749291e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7474003910 0x7f748c155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.823+0000 7f749291e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c153ee0 0x7f748c154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.823+0000 7f749291e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c1572c0 0x7f748c157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.823+0000 7f749291e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c0a7480 con 0x7f7474003910 2024-01-24T15:13:42.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.823+0000 7f749291e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c0c2fc0 con 0x7f748c153ee0 2024-01-24T15:13:42.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.823+0000 7f749291e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c155d00 con 0x7f748c1572c0 2024-01-24T15:13:42.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.823+0000 7f748bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7474003910 0x7f748c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f748bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7474003910 0x7f748c155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51872/0 (socket says 172.21.15.29:51872) 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f748bfff700 1 -- 172.21.15.29:0/122758502 learned_addr learned my addr 172.21.15.29:0/122758502 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f748b7fe700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c153ee0 0x7f748c154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f7490ebb700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c1572c0 0x7f748c157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f748bfff700 1 -- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c1572c0 msgr2=0x7f748c157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f748bfff700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c1572c0 0x7f748c157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f748bfff700 1 -- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c153ee0 msgr2=0x7f748c154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f748bfff700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c153ee0 0x7f748c154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f748bfff700 1 -- 172.21.15.29:0/122758502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0c2440 con 0x7f7474003910 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f748b7fe700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c153ee0 0x7f748c154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f748bfff700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7474003910 0x7f748c155810 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f747c00e4b0 tx=0x7f747c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f7473fff700 1 -- 172.21.15.29:0/122758502 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c020070 con 0x7f7474003910 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f7473fff700 1 -- 172.21.15.29:0/122758502 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c01ba10 con 0x7f7474003910 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f749291e700 1 -- 172.21.15.29:0/122758502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c0c2720 con 0x7f7474003910 2024-01-24T15:13:42.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f7473fff700 1 -- 172.21.15.29:0/122758502 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c01bdd0 con 0x7f7474003910 2024-01-24T15:13:42.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.824+0000 7f749291e700 1 -- 172.21.15.29:0/122758502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c0c2ce0 con 0x7f7474003910 2024-01-24T15:13:42.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.826+0000 7f749291e700 1 -- 172.21.15.29:0/122758502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c0475e0 con 0x7f7474003910 2024-01-24T15:13:42.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.826+0000 7f7473fff700 1 -- 172.21.15.29:0/122758502 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f747c025670 con 0x7f7474003910 2024-01-24T15:13:42.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.826+0000 7f7473fff700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f747805f270 0x7f7478061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:42.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.826+0000 7f7473fff700 1 -- 172.21.15.29:0/122758502 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f747c07bd10 con 0x7f7474003910 2024-01-24T15:13:42.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.827+0000 7f748b7fe700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f747805f270 0x7f7478061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:42.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.827+0000 7f748b7fe700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f747805f270 0x7f7478061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f74800015a0 tx=0x7f7480009040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:42.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:42.831+0000 7f7473fff700 1 -- 172.21.15.29:0/122758502 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f747c04ca30 con 0x7f7474003910 2024-01-24T15:13:43.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.006+0000 7f749291e700 1 -- 172.21.15.29:0/122758502 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f748c0c5700 con 0x7f747805f270 2024-01-24T15:13:43.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.016+0000 7f7473fff700 1 -- 172.21.15.29:0/122758502 <== mgr.8056 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+354365 (secure 0 0 0) 0x7f748c0c5700 con 0x7f747805f270 2024-01-24T15:13:43.023 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:43.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.020+0000 7f749291e700 1 -- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f747805f270 msgr2=0x7f7478061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.020+0000 7f749291e700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f747805f270 0x7f7478061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f74800015a0 tx=0x7f7480009040).stop 2024-01-24T15:13:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.020+0000 7f749291e700 1 -- 172.21.15.29:0/122758502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7474003910 msgr2=0x7f748c155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.020+0000 7f749291e700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7474003910 0x7f748c155810 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f747c00e4b0 tx=0x7f747c01a000).stop 2024-01-24T15:13:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.020+0000 7f749291e700 1 -- 172.21.15.29:0/122758502 shutdown_connections 2024-01-24T15:13:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.020+0000 7f749291e700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f748c1572c0 0x7f748c157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.020+0000 7f749291e700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f747805f270 0x7f7478061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.020+0000 7f749291e700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7474003910 0x7f748c155810 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.020+0000 7f749291e700 1 --2- 172.21.15.29:0/122758502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c153ee0 0x7f748c154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.020+0000 7f749291e700 1 -- 172.21.15.29:0/122758502 >> 172.21.15.29:0/122758502 conn(0x7f748c0c8580 msgr2=0x7f748c0c4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.020+0000 7f749291e700 1 -- 172.21.15.29:0/122758502 shutdown_connections 2024-01-24T15:13:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.020+0000 7f749291e700 1 -- 172.21.15.29:0/122758502 wait complete. 2024-01-24T15:13:43.027 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:13:43.035 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T15:13:41.038118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2090,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":106,"acting":106,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2451220,"kb_used_data":17108,"kb_used_omap":0,"kb_used_meta":2434048,"kb_avail":752523500,"statfs":{"total":773094113280,"available":770584064000,"internally_reserved":0,"allocated":17518592,"data_stored":10014904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2492465152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5037063,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-522,"num_write_kb":-5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.997696"},"pg_stats":[{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550222+0000","last_change":"2024-01-24T15:13:38.543675+0000","last_active":"2024-01-24T15:13:38.550222+0000","last_peered":"2024-01-24T15:13:38.550222+0000","last_clean":"2024-01-24T15:13:38.550222+0000","last_became_active":"2024-01-24T15:13:38.543012+0000","last_became_peered":"2024-01-24T15:13:38.543012+0000","last_unstale":"2024-01-24T15:13:38.550222+0000","last_undegraded":"2024-01-24T15:13:38.550222+0000","last_fullsized":"2024-01-24T15:13:38.550222+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3f","version":"177'3","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545770+0000","last_change":"2024-01-24T15:13:38.128076+0000","last_active":"2024-01-24T15:13:38.545770+0000","last_peered":"2024-01-24T15:13:38.545770+0000","last_clean":"2024-01-24T15:13:38.545770+0000","last_became_active":"2024-01-24T15:13:38.127791+0000","last_became_peered":"2024-01-24T15:13:38.127791+0000","last_unstale":"2024-01-24T15:13:38.545770+0000","last_undegraded":"2024-01-24T15:13:38.545770+0000","last_fullsized":"2024-01-24T15:13:38.545770+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557709+0000","last_change":"2024-01-24T15:13:38.122839+0000","last_active":"2024-01-24T15:13:38.557709+0000","last_peered":"2024-01-24T15:13:38.557709+0000","last_clean":"2024-01-24T15:13:38.557709+0000","last_became_active":"2024-01-24T15:13:38.121646+0000","last_became_peered":"2024-01-24T15:13:38.121646+0000","last_unstale":"2024-01-24T15:13:38.557709+0000","last_undegraded":"2024-01-24T15:13:38.557709+0000","last_fullsized":"2024-01-24T15:13:38.557709+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559220+0000","last_change":"2024-01-24T15:13:38.556825+0000","last_active":"2024-01-24T15:13:38.559220+0000","last_peered":"2024-01-24T15:13:38.559220+0000","last_clean":"2024-01-24T15:13:38.559220+0000","last_became_active":"2024-01-24T15:13:38.556530+0000","last_became_peered":"2024-01-24T15:13:38.556530+0000","last_unstale":"2024-01-24T15:13:38.559220+0000","last_undegraded":"2024-01-24T15:13:38.559220+0000","last_fullsized":"2024-01-24T15:13:38.559220+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557777+0000","last_change":"2024-01-24T15:13:38.555214+0000","last_active":"2024-01-24T15:13:38.557777+0000","last_peered":"2024-01-24T15:13:38.557777+0000","last_clean":"2024-01-24T15:13:38.557777+0000","last_became_active":"2024-01-24T15:13:38.554915+0000","last_became_peered":"2024-01-24T15:13:38.554915+0000","last_unstale":"2024-01-24T15:13:38.557777+0000","last_undegraded":"2024-01-24T15:13:38.557777+0000","last_fullsized":"2024-01-24T15:13:38.557777+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.539484+0000","last_change":"2024-01-24T15:13:38.127556+0000","last_active":"2024-01-24T15:13:38.539484+0000","last_peered":"2024-01-24T15:13:38.539484+0000","last_clean":"2024-01-24T15:13:38.539484+0000","last_became_active":"2024-01-24T15:13:38.127215+0000","last_became_peered":"2024-01-24T15:13:38.127215+0000","last_unstale":"2024-01-24T15:13:38.539484+0000","last_undegraded":"2024-01-24T15:13:38.539484+0000","last_fullsized":"2024-01-24T15:13:38.539484+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.565384+0000","last_change":"2024-01-24T15:13:38.123112+0000","last_active":"2024-01-24T15:13:38.565384+0000","last_peered":"2024-01-24T15:13:38.565384+0000","last_clean":"2024-01-24T15:13:38.565384+0000","last_became_active":"2024-01-24T15:13:38.122820+0000","last_became_peered":"2024-01-24T15:13:38.122820+0000","last_unstale":"2024-01-24T15:13:38.565384+0000","last_undegraded":"2024-01-24T15:13:38.565384+0000","last_fullsized":"2024-01-24T15:13:38.565384+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528656+0000","last_change":"2024-01-24T15:13:38.125651+0000","last_active":"2024-01-24T15:13:38.528656+0000","last_peered":"2024-01-24T15:13:38.528656+0000","last_clean":"2024-01-24T15:13:38.528656+0000","last_became_active":"2024-01-24T15:13:38.125334+0000","last_became_peered":"2024-01-24T15:13:38.125334+0000","last_unstale":"2024-01-24T15:13:38.528656+0000","last_undegraded":"2024-01-24T15:13:38.528656+0000","last_fullsized":"2024-01-24T15:13:38.528656+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545374+0000","last_change":"2024-01-24T15:13:38.543804+0000","last_active":"2024-01-24T15:13:38.545374+0000","last_peered":"2024-01-24T15:13:38.545374+0000","last_clean":"2024-01-24T15:13:38.545374+0000","last_became_active":"2024-01-24T15:13:38.543259+0000","last_became_peered":"2024-01-24T15:13:38.543259+0000","last_unstale":"2024-01-24T15:13:38.545374+0000","last_undegraded":"2024-01-24T15:13:38.545374+0000","last_fullsized":"2024-01-24T15:13:38.545374+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528152+0000","last_change":"2024-01-24T15:13:38.125234+0000","last_active":"2024-01-24T15:13:38.528152+0000","last_peered":"2024-01-24T15:13:38.528152+0000","last_clean":"2024-01-24T15:13:38.528152+0000","last_became_active":"2024-01-24T15:13:38.124520+0000","last_became_peered":"2024-01-24T15:13:38.124520+0000","last_unstale":"2024-01-24T15:13:38.528152+0000","last_undegraded":"2024-01-24T15:13:38.528152+0000","last_fullsized":"2024-01-24T15:13:38.528152+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532043+0000","last_change":"2024-01-24T15:13:38.122045+0000","last_active":"2024-01-24T15:13:38.532043+0000","last_peered":"2024-01-24T15:13:38.532043+0000","last_clean":"2024-01-24T15:13:38.532043+0000","last_became_active":"2024-01-24T15:13:38.121766+0000","last_became_peered":"2024-01-24T15:13:38.121766+0000","last_unstale":"2024-01-24T15:13:38.532043+0000","last_undegraded":"2024-01-24T15:13:38.532043+0000","last_fullsized":"2024-01-24T15:13:38.532043+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545141+0000","last_change":"2024-01-24T15:13:38.541723+0000","last_active":"2024-01-24T15:13:38.545141+0000","last_peered":"2024-01-24T15:13:38.545141+0000","last_clean":"2024-01-24T15:13:38.545141+0000","last_became_active":"2024-01-24T15:13:38.540083+0000","last_became_peered":"2024-01-24T15:13:38.540083+0000","last_unstale":"2024-01-24T15:13:38.545141+0000","last_undegraded":"2024-01-24T15:13:38.545141+0000","last_fullsized":"2024-01-24T15:13:38.545141+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532179+0000","last_change":"2024-01-24T15:13:38.128558+0000","last_active":"2024-01-24T15:13:38.532179+0000","last_peered":"2024-01-24T15:13:38.532179+0000","last_clean":"2024-01-24T15:13:38.532179+0000","last_became_active":"2024-01-24T15:13:38.128075+0000","last_became_peered":"2024-01-24T15:13:38.128075+0000","last_unstale":"2024-01-24T15:13:38.532179+0000","last_undegraded":"2024-01-24T15:13:38.532179+0000","last_fullsized":"2024-01-24T15:13:38.532179+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528767+0000","last_change":"2024-01-24T15:13:38.119386+0000","last_active":"2024-01-24T15:13:38.528767+0000","last_peered":"2024-01-24T15:13:38.528767+0000","last_clean":"2024-01-24T15:13:38.528767+0000","last_became_active":"2024-01-24T15:13:38.118881+0000","last_became_peered":"2024-01-24T15:13:38.118881+0000","last_unstale":"2024-01-24T15:13:38.528767+0000","last_undegraded":"2024-01-24T15:13:38.528767+0000","last_fullsized":"2024-01-24T15:13:38.528767+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558719+0000","last_change":"2024-01-24T15:13:38.555417+0000","last_active":"2024-01-24T15:13:38.558719+0000","last_peered":"2024-01-24T15:13:38.558719+0000","last_clean":"2024-01-24T15:13:38.558719+0000","last_became_active":"2024-01-24T15:13:38.555118+0000","last_became_peered":"2024-01-24T15:13:38.555118+0000","last_unstale":"2024-01-24T15:13:38.558719+0000","last_undegraded":"2024-01-24T15:13:38.558719+0000","last_fullsized":"2024-01-24T15:13:38.558719+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562480+0000","last_change":"2024-01-24T15:13:38.561900+0000","last_active":"2024-01-24T15:13:38.562480+0000","last_peered":"2024-01-24T15:13:38.562480+0000","last_clean":"2024-01-24T15:13:38.562480+0000","last_became_active":"2024-01-24T15:13:38.561655+0000","last_became_peered":"2024-01-24T15:13:38.561655+0000","last_unstale":"2024-01-24T15:13:38.562480+0000","last_undegraded":"2024-01-24T15:13:38.562480+0000","last_fullsized":"2024-01-24T15:13:38.562480+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.548146+0000","last_change":"2024-01-24T15:13:38.546025+0000","last_active":"2024-01-24T15:13:38.548146+0000","last_peered":"2024-01-24T15:13:38.548146+0000","last_clean":"2024-01-24T15:13:38.548146+0000","last_became_active":"2024-01-24T15:13:38.545702+0000","last_became_peered":"2024-01-24T15:13:38.545702+0000","last_unstale":"2024-01-24T15:13:38.548146+0000","last_undegraded":"2024-01-24T15:13:38.548146+0000","last_fullsized":"2024-01-24T15:13:38.548146+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.546793+0000","last_change":"2024-01-24T15:13:38.128387+0000","last_active":"2024-01-24T15:13:38.546793+0000","last_peered":"2024-01-24T15:13:38.546793+0000","last_clean":"2024-01-24T15:13:38.546793+0000","last_became_active":"2024-01-24T15:13:38.128088+0000","last_became_peered":"2024-01-24T15:13:38.128088+0000","last_unstale":"2024-01-24T15:13:38.546793+0000","last_undegraded":"2024-01-24T15:13:38.546793+0000","last_fullsized":"2024-01-24T15:13:38.546793+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545355+0000","last_change":"2024-01-24T15:13:38.543615+0000","last_active":"2024-01-24T15:13:38.545355+0000","last_peered":"2024-01-24T15:13:38.545355+0000","last_clean":"2024-01-24T15:13:38.545355+0000","last_became_active":"2024-01-24T15:13:38.541511+0000","last_became_peered":"2024-01-24T15:13:38.541511+0000","last_unstale":"2024-01-24T15:13:38.545355+0000","last_undegraded":"2024-01-24T15:13:38.545355+0000","last_fullsized":"2024-01-24T15:13:38.545355+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550198+0000","last_change":"2024-01-24T15:13:38.546422+0000","last_active":"2024-01-24T15:13:38.550198+0000","last_peered":"2024-01-24T15:13:38.550198+0000","last_clean":"2024-01-24T15:13:38.550198+0000","last_became_active":"2024-01-24T15:13:38.546168+0000","last_became_peered":"2024-01-24T15:13:38.546168+0000","last_unstale":"2024-01-24T15:13:38.550198+0000","last_undegraded":"2024-01-24T15:13:38.550198+0000","last_fullsized":"2024-01-24T15:13:38.550198+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.538459+0000","last_change":"2024-01-24T15:13:38.126860+0000","last_active":"2024-01-24T15:13:38.538459+0000","last_peered":"2024-01-24T15:13:38.538459+0000","last_clean":"2024-01-24T15:13:38.538459+0000","last_became_active":"2024-01-24T15:13:38.126528+0000","last_became_peered":"2024-01-24T15:13:38.126528+0000","last_unstale":"2024-01-24T15:13:38.538459+0000","last_undegraded":"2024-01-24T15:13:38.538459+0000","last_fullsized":"2024-01-24T15:13:38.538459+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.547990+0000","last_change":"2024-01-24T15:13:38.542686+0000","last_active":"2024-01-24T15:13:38.547990+0000","last_peered":"2024-01-24T15:13:38.547990+0000","last_clean":"2024-01-24T15:13:38.547990+0000","last_became_active":"2024-01-24T15:13:38.542392+0000","last_became_peered":"2024-01-24T15:13:38.542392+0000","last_unstale":"2024-01-24T15:13:38.547990+0000","last_undegraded":"2024-01-24T15:13:38.547990+0000","last_fullsized":"2024-01-24T15:13:38.547990+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.548937+0000","last_change":"2024-01-24T15:13:38.125904+0000","last_active":"2024-01-24T15:13:38.548937+0000","last_peered":"2024-01-24T15:13:38.548937+0000","last_clean":"2024-01-24T15:13:38.548937+0000","last_became_active":"2024-01-24T15:13:38.125609+0000","last_became_peered":"2024-01-24T15:13:38.125609+0000","last_unstale":"2024-01-24T15:13:38.548937+0000","last_undegraded":"2024-01-24T15:13:38.548937+0000","last_fullsized":"2024-01-24T15:13:38.548937+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550294+0000","last_change":"2024-01-24T15:13:38.547021+0000","last_active":"2024-01-24T15:13:38.550294+0000","last_peered":"2024-01-24T15:13:38.550294+0000","last_clean":"2024-01-24T15:13:38.550294+0000","last_became_active":"2024-01-24T15:13:38.546231+0000","last_became_peered":"2024-01-24T15:13:38.546231+0000","last_unstale":"2024-01-24T15:13:38.550294+0000","last_undegraded":"2024-01-24T15:13:38.550294+0000","last_fullsized":"2024-01-24T15:13:38.550294+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528666+0000","last_change":"2024-01-24T15:13:38.124808+0000","last_active":"2024-01-24T15:13:38.528666+0000","last_peered":"2024-01-24T15:13:38.528666+0000","last_clean":"2024-01-24T15:13:38.528666+0000","last_became_active":"2024-01-24T15:13:38.124504+0000","last_became_peered":"2024-01-24T15:13:38.124504+0000","last_unstale":"2024-01-24T15:13:38.528666+0000","last_undegraded":"2024-01-24T15:13:38.528666+0000","last_fullsized":"2024-01-24T15:13:38.528666+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542376+0000","last_change":"2024-01-24T15:13:38.540674+0000","last_active":"2024-01-24T15:13:38.542376+0000","last_peered":"2024-01-24T15:13:38.542376+0000","last_clean":"2024-01-24T15:13:38.542376+0000","last_became_active":"2024-01-24T15:13:38.540381+0000","last_became_peered":"2024-01-24T15:13:38.540381+0000","last_unstale":"2024-01-24T15:13:38.542376+0000","last_undegraded":"2024-01-24T15:13:38.542376+0000","last_fullsized":"2024-01-24T15:13:38.542376+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528151+0000","last_change":"2024-01-24T15:13:38.119586+0000","last_active":"2024-01-24T15:13:38.528151+0000","last_peered":"2024-01-24T15:13:38.528151+0000","last_clean":"2024-01-24T15:13:38.528151+0000","last_became_active":"2024-01-24T15:13:38.119285+0000","last_became_peered":"2024-01-24T15:13:38.119285+0000","last_unstale":"2024-01-24T15:13:38.528151+0000","last_undegraded":"2024-01-24T15:13:38.528151+0000","last_fullsized":"2024-01-24T15:13:38.528151+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557360+0000","last_change":"2024-01-24T15:13:38.549783+0000","last_active":"2024-01-24T15:13:38.557360+0000","last_peered":"2024-01-24T15:13:38.557360+0000","last_clean":"2024-01-24T15:13:38.557360+0000","last_became_active":"2024-01-24T15:13:38.549374+0000","last_became_peered":"2024-01-24T15:13:38.549374+0000","last_unstale":"2024-01-24T15:13:38.557360+0000","last_undegraded":"2024-01-24T15:13:38.557360+0000","last_fullsized":"2024-01-24T15:13:38.557360+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557933+0000","last_change":"2024-01-24T15:13:38.121798+0000","last_active":"2024-01-24T15:13:38.557933+0000","last_peered":"2024-01-24T15:13:38.557933+0000","last_clean":"2024-01-24T15:13:38.557933+0000","last_became_active":"2024-01-24T15:13:38.121333+0000","last_became_peered":"2024-01-24T15:13:38.121333+0000","last_unstale":"2024-01-24T15:13:38.557933+0000","last_undegraded":"2024-01-24T15:13:38.557933+0000","last_fullsized":"2024-01-24T15:13:38.557933+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.529610+0000","last_change":"2024-01-24T15:13:38.125875+0000","last_active":"2024-01-24T15:13:38.529610+0000","last_peered":"2024-01-24T15:13:38.529610+0000","last_clean":"2024-01-24T15:13:38.529610+0000","last_became_active":"2024-01-24T15:13:38.125536+0000","last_became_peered":"2024-01-24T15:13:38.125536+0000","last_unstale":"2024-01-24T15:13:38.529610+0000","last_undegraded":"2024-01-24T15:13:38.529610+0000","last_fullsized":"2024-01-24T15:13:38.529610+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532219+0000","last_change":"2024-01-24T15:13:38.122494+0000","last_active":"2024-01-24T15:13:38.532219+0000","last_peered":"2024-01-24T15:13:38.532219+0000","last_clean":"2024-01-24T15:13:38.532219+0000","last_became_active":"2024-01-24T15:13:38.122120+0000","last_became_peered":"2024-01-24T15:13:38.122120+0000","last_unstale":"2024-01-24T15:13:38.532219+0000","last_undegraded":"2024-01-24T15:13:38.532219+0000","last_fullsized":"2024-01-24T15:13:38.532219+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561628+0000","last_change":"2024-01-24T15:13:38.129725+0000","last_active":"2024-01-24T15:13:38.561628+0000","last_peered":"2024-01-24T15:13:38.561628+0000","last_clean":"2024-01-24T15:13:38.561628+0000","last_became_active":"2024-01-24T15:13:38.129199+0000","last_became_peered":"2024-01-24T15:13:38.129199+0000","last_unstale":"2024-01-24T15:13:38.561628+0000","last_undegraded":"2024-01-24T15:13:38.561628+0000","last_fullsized":"2024-01-24T15:13:38.561628+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556320+0000","last_change":"2024-01-24T15:13:38.550473+0000","last_active":"2024-01-24T15:13:38.556320+0000","last_peered":"2024-01-24T15:13:38.556320+0000","last_clean":"2024-01-24T15:13:38.556320+0000","last_became_active":"2024-01-24T15:13:38.550227+0000","last_became_peered":"2024-01-24T15:13:38.550227+0000","last_unstale":"2024-01-24T15:13:38.556320+0000","last_undegraded":"2024-01-24T15:13:38.556320+0000","last_fullsized":"2024-01-24T15:13:38.556320+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556378+0000","last_change":"2024-01-24T15:13:38.552294+0000","last_active":"2024-01-24T15:13:38.556378+0000","last_peered":"2024-01-24T15:13:38.556378+0000","last_clean":"2024-01-24T15:13:38.556378+0000","last_became_active":"2024-01-24T15:13:38.551919+0000","last_became_peered":"2024-01-24T15:13:38.551919+0000","last_unstale":"2024-01-24T15:13:38.556378+0000","last_undegraded":"2024-01-24T15:13:38.556378+0000","last_fullsized":"2024-01-24T15:13:38.556378+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561397+0000","last_change":"2024-01-24T15:13:38.557841+0000","last_active":"2024-01-24T15:13:38.561397+0000","last_peered":"2024-01-24T15:13:38.561397+0000","last_clean":"2024-01-24T15:13:38.561397+0000","last_became_active":"2024-01-24T15:13:38.557543+0000","last_became_peered":"2024-01-24T15:13:38.557543+0000","last_unstale":"2024-01-24T15:13:38.561397+0000","last_undegraded":"2024-01-24T15:13:38.561397+0000","last_fullsized":"2024-01-24T15:13:38.561397+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558444+0000","last_change":"2024-01-24T15:13:38.552923+0000","last_active":"2024-01-24T15:13:38.558444+0000","last_peered":"2024-01-24T15:13:38.558444+0000","last_clean":"2024-01-24T15:13:38.558444+0000","last_became_active":"2024-01-24T15:13:38.552646+0000","last_became_peered":"2024-01-24T15:13:38.552646+0000","last_unstale":"2024-01-24T15:13:38.558444+0000","last_undegraded":"2024-01-24T15:13:38.558444+0000","last_fullsized":"2024-01-24T15:13:38.558444+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.539981+0000","last_change":"2024-01-24T15:13:38.129534+0000","last_active":"2024-01-24T15:13:38.539981+0000","last_peered":"2024-01-24T15:13:38.539981+0000","last_clean":"2024-01-24T15:13:38.539981+0000","last_became_active":"2024-01-24T15:13:38.128874+0000","last_became_peered":"2024-01-24T15:13:38.128874+0000","last_unstale":"2024-01-24T15:13:38.539981+0000","last_undegraded":"2024-01-24T15:13:38.539981+0000","last_fullsized":"2024-01-24T15:13:38.539981+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.531693+0000","last_change":"2024-01-24T15:13:38.124495+0000","last_active":"2024-01-24T15:13:38.531693+0000","last_peered":"2024-01-24T15:13:38.531693+0000","last_clean":"2024-01-24T15:13:38.531693+0000","last_became_active":"2024-01-24T15:13:38.124175+0000","last_became_peered":"2024-01-24T15:13:38.124175+0000","last_unstale":"2024-01-24T15:13:38.531693+0000","last_undegraded":"2024-01-24T15:13:38.531693+0000","last_fullsized":"2024-01-24T15:13:38.531693+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559906+0000","last_change":"2024-01-24T15:13:38.557558+0000","last_active":"2024-01-24T15:13:38.559906+0000","last_peered":"2024-01-24T15:13:38.559906+0000","last_clean":"2024-01-24T15:13:38.559906+0000","last_became_active":"2024-01-24T15:13:38.557290+0000","last_became_peered":"2024-01-24T15:13:38.557290+0000","last_unstale":"2024-01-24T15:13:38.559906+0000","last_undegraded":"2024-01-24T15:13:38.559906+0000","last_fullsized":"2024-01-24T15:13:38.559906+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.058886+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:13:33.058886+0000","last_peered":"2024-01-24T15:13:33.058886+0000","last_clean":"2024-01-24T15:13:33.058886+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:13:33.058886+0000","last_undegraded":"2024-01-24T15:13:33.058886+0000","last_fullsized":"2024-01-24T15:13:33.058886+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558463+0000","last_change":"2024-01-24T15:13:38.552222+0000","last_active":"2024-01-24T15:13:38.558463+0000","last_peered":"2024-01-24T15:13:38.558463+0000","last_clean":"2024-01-24T15:13:38.558463+0000","last_became_active":"2024-01-24T15:13:38.551916+0000","last_became_peered":"2024-01-24T15:13:38.551916+0000","last_unstale":"2024-01-24T15:13:38.558463+0000","last_undegraded":"2024-01-24T15:13:38.558463+0000","last_fullsized":"2024-01-24T15:13:38.558463+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561816+0000","last_change":"2024-01-24T15:13:38.128196+0000","last_active":"2024-01-24T15:13:38.561816+0000","last_peered":"2024-01-24T15:13:38.561816+0000","last_clean":"2024-01-24T15:13:38.561816+0000","last_became_active":"2024-01-24T15:13:38.127782+0000","last_became_peered":"2024-01-24T15:13:38.127782+0000","last_unstale":"2024-01-24T15:13:38.561816+0000","last_undegraded":"2024-01-24T15:13:38.561816+0000","last_fullsized":"2024-01-24T15:13:38.561816+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":158,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.082678+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:13:33.082678+0000","last_peered":"2024-01-24T15:13:33.082678+0000","last_clean":"2024-01-24T15:13:33.082678+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:13:33.082678+0000","last_undegraded":"2024-01-24T15:13:33.082678+0000","last_fullsized":"2024-01-24T15:13:33.082678+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532992+0000","last_change":"2024-01-24T15:13:38.125400+0000","last_active":"2024-01-24T15:13:38.532992+0000","last_peered":"2024-01-24T15:13:38.532992+0000","last_clean":"2024-01-24T15:13:38.532992+0000","last_became_active":"2024-01-24T15:13:38.124725+0000","last_became_peered":"2024-01-24T15:13:38.124725+0000","last_unstale":"2024-01-24T15:13:38.532992+0000","last_undegraded":"2024-01-24T15:13:38.532992+0000","last_fullsized":"2024-01-24T15:13:38.532992+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528231+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:13:38.528231+0000","last_peered":"2024-01-24T15:13:38.528231+0000","last_clean":"2024-01-24T15:13:38.528231+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:13:38.528231+0000","last_undegraded":"2024-01-24T15:13:38.528231+0000","last_fullsized":"2024-01-24T15:13:38.528231+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561686+0000","last_change":"2024-01-24T15:13:38.554780+0000","last_active":"2024-01-24T15:13:38.561686+0000","last_peered":"2024-01-24T15:13:38.561686+0000","last_clean":"2024-01-24T15:13:38.561686+0000","last_became_active":"2024-01-24T15:13:38.554497+0000","last_became_peered":"2024-01-24T15:13:38.554497+0000","last_unstale":"2024-01-24T15:13:38.561686+0000","last_undegraded":"2024-01-24T15:13:38.561686+0000","last_fullsized":"2024-01-24T15:13:38.561686+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":158,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.082682+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:13:33.082682+0000","last_peered":"2024-01-24T15:13:33.082682+0000","last_clean":"2024-01-24T15:13:33.082682+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:13:33.082682+0000","last_undegraded":"2024-01-24T15:13:33.082682+0000","last_fullsized":"2024-01-24T15:13:33.082682+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":143,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.933666+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:13:33.933666+0000","last_peered":"2024-01-24T15:13:33.933666+0000","last_clean":"2024-01-24T15:13:33.933666+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:13:33.933666+0000","last_undegraded":"2024-01-24T15:13:33.933666+0000","last_fullsized":"2024-01-24T15:13:33.933666+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.933467+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:13:33.933467+0000","last_peered":"2024-01-24T15:13:33.933467+0000","last_clean":"2024-01-24T15:13:33.933467+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:13:33.933467+0000","last_undegraded":"2024-01-24T15:13:33.933467+0000","last_fullsized":"2024-01-24T15:13:33.933467+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.933453+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:13:33.933453+0000","last_peered":"2024-01-24T15:13:33.933453+0000","last_clean":"2024-01-24T15:13:33.933453+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:13:33.933453+0000","last_undegraded":"2024-01-24T15:13:33.933453+0000","last_fullsized":"2024-01-24T15:13:33.933453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.549726+0000","last_change":"2024-01-24T15:13:38.121778+0000","last_active":"2024-01-24T15:13:38.549726+0000","last_peered":"2024-01-24T15:13:38.549726+0000","last_clean":"2024-01-24T15:13:38.549726+0000","last_became_active":"2024-01-24T15:13:38.121489+0000","last_became_peered":"2024-01-24T15:13:38.121489+0000","last_unstale":"2024-01-24T15:13:38.549726+0000","last_undegraded":"2024-01-24T15:13:38.549726+0000","last_fullsized":"2024-01-24T15:13:38.549726+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":140,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-01-24T15:13:33.058947+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:13:33.058947+0000","last_peered":"2024-01-24T15:13:33.058947+0000","last_clean":"2024-01-24T15:13:33.058947+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:13:33.058947+0000","last_undegraded":"2024-01-24T15:13:33.058947+0000","last_fullsized":"2024-01-24T15:13:33.058947+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:38.093739+0000","last_change":"2024-01-24T15:13:38.093739+0000","last_active":"2024-01-24T15:13:38.093739+0000","last_peered":"2024-01-24T15:13:38.093739+0000","last_clean":"2024-01-24T15:13:38.093739+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:38.093739+0000","last_undegraded":"2024-01-24T15:13:38.093739+0000","last_fullsized":"2024-01-24T15:13:38.093739+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:13:37.088603+0000","last_change":"2024-01-24T15:13:37.088603+0000","last_active":"2024-01-24T15:13:37.088603+0000","last_peered":"2024-01-24T15:13:37.088603+0000","last_clean":"2024-01-24T15:13:37.088603+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:13:37.088603+0000","last_undegraded":"2024-01-24T15:13:37.088603+0000","last_fullsized":"2024-01-24T15:13:37.088603+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":138,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-01-24T15:13:29.696776+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:13:29.696776+0000","last_peered":"2024-01-24T15:13:29.696776+0000","last_clean":"2024-01-24T15:13:29.696776+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:13:29.696776+0000","last_undegraded":"2024-01-24T15:13:29.696776+0000","last_fullsized":"2024-01-24T15:13:29.696776+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":3},{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":2090,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":46,"acting":46,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575750,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":2166784,"data_stored":1239804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.59899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.28000000000000003},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.628,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.28899999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.1080000000000001},"last":0.64900000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.33300000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.505,"5min":0.40000000000000002,"15min":0.34799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53300000000000003}]},{"osd":2,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0329999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.38},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0880000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.41199999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.33200000000000002},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.31900000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.67600000000000005}]},{"osd":5,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.39600000000000002,"5min":0.38,"15min":0.34799999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.439},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.36199999999999999},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.31900000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.46400000000000002,"5min":0.40899999999999997,"15min":0.40500000000000003},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.68899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575751,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":2166784,"data_stored":1239804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.86599999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.42699999999999999,"15min":0.37},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64500000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.84199999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77100000000000002,"5min":0.81200000000000006,"15min":1.258},"last":0.51600000000000001}]},{"osd":2,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.40100000000000002},"max":{"1min":0.82299999999999995,"5min":0.85899999999999999,"15min":1.2430000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.47199999999999998,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.498},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.39000000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.317},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32400000000000001,"15min":0.30499999999999999},"max":{"1min":0.79300000000000004,"5min":0.80100000000000005,"15min":1.002},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.32500000000000001},"max":{"1min":0.76000000000000001,"5min":0.78100000000000003,"15min":0.83899999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":0.749,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.40899999999999997,"5min":0.372,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.90100000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.38},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":1.0429999999999999},"last":0.65800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575750,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307404,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064436,"statfs":{"total":96636764160,"available":96321982464,"internally_reserved":0,"allocated":2166784,"data_stored":1239804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.56399999999999995,"5min":0.754,"15min":0.754},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.443,"15min":0.435},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.26800000000000002},"max":{"1min":0.55800000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.40200000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.33700000000000002,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.57599999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44600000000000001,"15min":0.44700000000000001},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.28199999999999997},"max":{"1min":0.56200000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.435}]},{"osd":3,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"min":{"1min":0.32400000000000001,"5min":0.314,"15min":0.309},"max":{"1min":0.58399999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.35799999999999998,"15min":0.313},"max":{"1min":0.59899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.51800000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.86099999999999999,"5min":1.044,"15min":1.0649999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.51400000000000001,"5min":0.47399999999999998,"15min":0.41699999999999998},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63500000000000001}]},{"osd":5,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.49199999999999999,"15min":0.46300000000000002},"max":{"1min":0.83999999999999997,"5min":1.014,"15min":1.0780000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.439},"max":{"1min":0.86699999999999999,"5min":1.105,"15min":1.105},"last":0.85099999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.874,"5min":1.0029999999999999,"15min":1.069},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.56599999999999995,"5min":0.499,"15min":0.48199999999999998},"max":{"1min":0.92500000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.871}]},{"osd":7,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.83799999999999997,"5min":1.127,"15min":1.127},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.77500000000000002,"15min":0.76700000000000002},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.42799999999999999},"max":{"1min":0.88900000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.84399999999999997}]}]},{"osd":0,"up_from":14,"seq":60129543049,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307464,"kb_used_data":2176,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064376,"statfs":{"total":96636764160,"available":96321921024,"internally_reserved":0,"allocated":2228224,"data_stored":1271118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.28100000000000003,"5min":0.27500000000000002,"15min":0.247},"max":{"1min":0.60199999999999998,"5min":0.63400000000000001,"15min":0.66400000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.27700000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.54500000000000004}]},{"osd":2,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.29399999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.81999999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.84399999999999997},"last":0.496}]},{"osd":3,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.44600000000000001},"min":{"1min":0.317,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.65500000000000003,"15min":0.67000000000000004},"last":0.626},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.52000000000000002},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.90000000000000002},"last":0.66300000000000003}]},{"osd":4,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.433},"max":{"1min":0.99399999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.38700000000000001},"max":{"1min":0.995,"5min":0.995,"15min":1.0249999999999999},"last":0.78100000000000003}]},{"osd":5,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.70699999999999996},"min":{"1min":0.54900000000000004,"5min":0.498,"15min":0.42099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.94199999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.46899999999999997},"max":{"1min":1.022,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0820000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81100000000000005}]},{"osd":7,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0469999999999999,"5min":1.143,"15min":1.3260000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.375},"last":0.87}]}]},{"osd":2,"up_from":13,"seq":55834575751,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311564,"kb_used_data":2180,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060276,"statfs":{"total":96636764160,"available":96317722624,"internally_reserved":0,"allocated":2232320,"data_stored":1272937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.27200000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.60099999999999998,"5min":0.65000000000000002,"15min":0.66800000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.45100000000000001,"15min":0.45200000000000001},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.70399999999999996},"last":0.29099999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.25600000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.73599999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.46300000000000002},"min":{"1min":0.32500000000000001,"5min":0.29699999999999999,"15min":0.29599999999999999},"max":{"1min":0.622,"5min":0.69699999999999995,"15min":0.70099999999999996},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.73299999999999998},"last":0.34999999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96199999999999997,"5min":1.196,"15min":1.196},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":1,"15min":1.089},"last":0.38100000000000001}]},{"osd":5,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54400000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.111},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.46899999999999997,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.147,"15min":1.1830000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.64700000000000002,"5min":0.47699999999999998,"15min":0.45200000000000001},"max":{"1min":0.99399999999999999,"5min":0.998,"15min":1.129},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.58899999999999997,"5min":0.47299999999999998,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.98999999999999999,"15min":1.1950000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.753,"15min":0.746},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.497},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.248},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.65000000000000002,"5min":0.51500000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.24,"15min":1.24},"last":0.63100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307460,"kb_used_data":2172,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064380,"statfs":{"total":96636764160,"available":96321925120,"internally_reserved":0,"allocated":2224128,"data_stored":1271829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39800000000000002,"15min":0.39200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.247},"max":{"1min":0.56899999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.70199999999999996},"last":0.36199999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41899999999999998,"15min":0.41399999999999998},"min":{"1min":0.311,"5min":0.28100000000000003,"15min":0.27400000000000002},"max":{"1min":0.59999999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.34799999999999998,"5min":0.316,"15min":0.30499999999999999},"max":{"1min":0.67900000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.433}]},{"osd":2,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.497,"15min":0.502},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51300000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.41099999999999998},"max":{"1min":0.83299999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95999999999999996,"15min":1.5580000000000001},"last":0.58799999999999997}]},{"osd":5,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.1879999999999999,"15min":1.6240000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.647},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.45700000000000002,"15min":0.42699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.522},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.073,"5min":1.079,"15min":1.079},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.44800000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1499999999999999,"5min":1.246,"15min":1.246},"last":0.64400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307404,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064436,"statfs":{"total":96636764160,"available":96321982464,"internally_reserved":0,"allocated":2166784,"data_stored":1239804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.46700000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.83699999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76500000000000001,"5min":0.78400000000000003,"15min":0.84199999999999997},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.45500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.72499999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82999999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.61299999999999999}]},{"osd":2,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.84699999999999998,"15min":0.88300000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.626,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.372},"max":{"1min":0.79800000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.629}]},{"osd":3,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.38400000000000001},"max":{"1min":0.79900000000000004,"5min":0.80400000000000005,"15min":0.89500000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.42399999999999999},"max":{"1min":0.71899999999999997,"5min":0.82199999999999995,"15min":0.92300000000000004},"last":0.64000000000000001}]},{"osd":5,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":0.61299999999999999,"5min":0.79200000000000004,"15min":0.81799999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66400000000000003,"5min":0.72499999999999998,"15min":0.85499999999999998},"last":0.65800000000000003}]},{"osd":6,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.68300000000000005,"5min":0.78500000000000003,"15min":0.85499999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.38200000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.86799999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.33600000000000002},"max":{"1min":0.65200000000000002,"5min":0.77500000000000002,"15min":0.82799999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.72599999999999998,"5min":0.81999999999999995,"15min":0.96599999999999997},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575751,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":2166784,"data_stored":1239804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.38600000000000001,"15min":0.35099999999999998},"max":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":1.4470000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.81599999999999995},"last":0.53300000000000003}]},{"osd":1,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.82499999999999996},"last":0.435},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":1.4930000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.625,"15min":0.627},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.79300000000000004,"5min":0.80900000000000005,"15min":0.86499999999999999},"last":0.503},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":1.53},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.80800000000000005,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41899999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.65400000000000003,"5min":0.74399999999999999,"15min":0.84699999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":0.77200000000000002,"15min":0.80100000000000005},"last":0.38900000000000001}]},{"osd":6,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.5,"15min":0.504},"min":{"1min":0.374,"5min":0.35499999999999998,"15min":0.33400000000000002},"max":{"1min":0.65100000000000002,"5min":0.79400000000000004,"15min":0.81499999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.78500000000000003,"15min":0.81299999999999994},"last":0.44400000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.54900000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:13:43.041 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:13:43.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.207+0000 7f8471323700 1 Processor -- start 2024-01-24T15:13:43.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.207+0000 7f8471323700 1 -- start start 2024-01-24T15:13:43.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.208+0000 7f8471323700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c1544a0 0x7f846c0c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:43.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.208+0000 7f8471323700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f846c0a9ff0 0x7f846c0631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:43.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.208+0000 7f8471323700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846c063b70 con 0x7f846c0636c0 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.208+0000 7f8471323700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846c0cbcf0 con 0x7f846c0a9ff0 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.208+0000 7f8471323700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846c0cb520 con 0x7f846c1544a0 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.208+0000 7f846a7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f846c0a9ff0 0x7f846c0631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.208+0000 7f846a7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f846c0a9ff0 0x7f846c0631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51876/0 (socket says 172.21.15.29:51876) 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.208+0000 7f846a7fc700 1 -- 172.21.15.29:0/80686647 learned_addr learned my addr 172.21.15.29:0/80686647 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.208+0000 7f846affd700 1 --2- 172.21.15.29:0/80686647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c1544a0 0x7f846c0c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.209+0000 7f846a7fc700 1 -- 172.21.15.29:0/80686647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c1544a0 msgr2=0x7f846c0c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.209+0000 7f846a7fc700 1 --2- 172.21.15.29:0/80686647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c1544a0 0x7f846c0c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.209+0000 7f846a7fc700 1 -- 172.21.15.29:0/80686647 >> v1:172.21.15.163:6789/0 conn(0x7f846c0636c0 legacy=0x7f846c158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.209+0000 7f846a7fc700 1 -- 172.21.15.29:0/80686647 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c0cb730 con 0x7f846c0a9ff0 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.209+0000 7f846a7fc700 1 --2- 172.21.15.29:0/80686647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f846c0a9ff0 0x7f846c0631d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f845c009830 tx=0x7f845c0090f0).ready entity=mon.0 client_cookie=ab250c846e20288b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.209+0000 7f8469ffb700 1 -- 172.21.15.29:0/80686647 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c00b600 con 0x7f846c0a9ff0 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.209+0000 7f8469ffb700 1 -- 172.21.15.29:0/80686647 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c00b760 con 0x7f846c0a9ff0 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.209+0000 7f8469ffb700 1 -- 172.21.15.29:0/80686647 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c00bb00 con 0x7f846c0a9ff0 2024-01-24T15:13:43.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.209+0000 7f8471323700 1 -- 172.21.15.29:0/80686647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f846c0a9ff0 msgr2=0x7f846c0631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.209+0000 7f8471323700 1 --2- 172.21.15.29:0/80686647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f846c0a9ff0 0x7f846c0631d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f845c009830 tx=0x7f845c0090f0).stop 2024-01-24T15:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.209+0000 7f8471323700 1 -- 172.21.15.29:0/80686647 shutdown_connections 2024-01-24T15:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.210+0000 7f8471323700 1 --2- 172.21.15.29:0/80686647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c1544a0 0x7f846c0c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.210+0000 7f8471323700 1 --2- 172.21.15.29:0/80686647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f846c0a9ff0 0x7f846c0631d0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.210+0000 7f8471323700 1 -- 172.21.15.29:0/80686647 >> 172.21.15.29:0/80686647 conn(0x7f846c05ec80 msgr2=0x7f846c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.210+0000 7f8471323700 1 -- 172.21.15.29:0/80686647 shutdown_connections 2024-01-24T15:13:43.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.210+0000 7f8471323700 1 -- 172.21.15.29:0/80686647 wait complete. 2024-01-24T15:13:43.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.211+0000 7f8471323700 1 Processor -- start 2024-01-24T15:13:43.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.211+0000 7f8471323700 1 -- start start 2024-01-24T15:13:43.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f8471323700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f846c0636c0 0x7f846c152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:43.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f8471323700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c0a9ff0 0x7f846c1685a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f8471323700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f846c1544a0 0x7f846c13fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f8471323700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c153620 con 0x7f846c1544a0 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f8471323700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c14ff10 con 0x7f846c0636c0 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f8471323700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846c153340 con 0x7f846c0a9ff0 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f846b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f846c1544a0 0x7f846c13fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f846b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f846c1544a0 0x7f846c13fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:51880/0 (socket says 172.21.15.29:51880) 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f846b7fe700 1 -- 172.21.15.29:0/793795822 learned_addr learned my addr 172.21.15.29:0/793795822 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f846affd700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f846c0636c0 0x7f846c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f846a7fc700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c0a9ff0 0x7f846c1685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f846b7fe700 1 -- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c0a9ff0 msgr2=0x7f846c1685a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f846b7fe700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c0a9ff0 0x7f846c1685a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f846b7fe700 1 -- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f846c0636c0 msgr2=0x7f846c152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f846b7fe700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f846c0636c0 0x7f846c152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f846b7fe700 1 -- 172.21.15.29:0/793795822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c009370 con 0x7f846c1544a0 2024-01-24T15:13:43.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f846affd700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f846c0636c0 0x7f846c152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:43.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.212+0000 7f846b7fe700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f846c1544a0 0x7f846c13fa00 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f846400c7c0 tx=0x7f8464001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:43.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.213+0000 7f8462ffd700 1 -- 172.21.15.29:0/793795822 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464010510 con 0x7f846c1544a0 2024-01-24T15:13:43.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.213+0000 7f8462ffd700 1 -- 172.21.15.29:0/793795822 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8464010670 con 0x7f846c1544a0 2024-01-24T15:13:43.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.213+0000 7f8462ffd700 1 -- 172.21.15.29:0/793795822 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84640109f0 con 0x7f846c1544a0 2024-01-24T15:13:43.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.213+0000 7f8471323700 1 -- 172.21.15.29:0/793795822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846c14f8c0 con 0x7f846c1544a0 2024-01-24T15:13:43.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.213+0000 7f8471323700 1 -- 172.21.15.29:0/793795822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846c168a90 con 0x7f846c1544a0 2024-01-24T15:13:43.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.214+0000 7f8471323700 1 -- 172.21.15.29:0/793795822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846c0475e0 con 0x7f846c1544a0 2024-01-24T15:13:43.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.214+0000 7f8462ffd700 1 -- 172.21.15.29:0/793795822 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f846401b400 con 0x7f846c1544a0 2024-01-24T15:13:43.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.215+0000 7f8462ffd700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f844c05f270 0x7f844c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:43.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.215+0000 7f8462ffd700 1 -- 172.21.15.29:0/793795822 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f846407a3b0 con 0x7f846c1544a0 2024-01-24T15:13:43.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.215+0000 7f846affd700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f844c05f270 0x7f844c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:43.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.215+0000 7f846affd700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f844c05f270 0x7f844c061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f8454006fd0 tx=0x7f8454008040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:43.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.219+0000 7f8462ffd700 1 -- 172.21.15.29:0/793795822 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f846404b070 con 0x7f846c1544a0 2024-01-24T15:13:43.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.396+0000 7f8471323700 1 -- 172.21.15.29:0/793795822 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f846c14f740 con 0x7f844c05f270 2024-01-24T15:13:43.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.406+0000 7f8462ffd700 1 -- 172.21.15.29:0/793795822 <== mgr.8056 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361966 (secure 0 0 0) 0x7f845400f7d0 con 0x7f844c05f270 2024-01-24T15:13:43.413 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:43.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.410+0000 7f8471323700 1 -- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f844c05f270 msgr2=0x7f844c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.410+0000 7f8471323700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f844c05f270 0x7f844c061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f8454006fd0 tx=0x7f8454008040).stop 2024-01-24T15:13:43.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.410+0000 7f8471323700 1 -- 172.21.15.29:0/793795822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f846c1544a0 msgr2=0x7f846c13fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.410+0000 7f8471323700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f846c1544a0 0x7f846c13fa00 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f846400c7c0 tx=0x7f8464001510).stop 2024-01-24T15:13:43.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.410+0000 7f8471323700 1 -- 172.21.15.29:0/793795822 shutdown_connections 2024-01-24T15:13:43.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.410+0000 7f8471323700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f844c05f270 0x7f844c061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.410+0000 7f8471323700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f846c1544a0 0x7f846c13fa00 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.410+0000 7f8471323700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f846c0636c0 0x7f846c152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.410+0000 7f8471323700 1 --2- 172.21.15.29:0/793795822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c0a9ff0 0x7f846c1685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.410+0000 7f8471323700 1 -- 172.21.15.29:0/793795822 >> 172.21.15.29:0/793795822 conn(0x7f846c05ec80 msgr2=0x7f846c0ad160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:43.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.410+0000 7f8471323700 1 -- 172.21.15.29:0/793795822 shutdown_connections 2024-01-24T15:13:43.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.410+0000 7f8471323700 1 -- 172.21.15.29:0/793795822 wait complete. 2024-01-24T15:13:43.416 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:13:43.426 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-01-24T15:13:43.038672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":222,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2459676,"kb_used_data":17372,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752515044,"statfs":{"total":773094113280,"available":770575405056,"internally_reserved":0,"allocated":17788928,"data_stored":10170427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5036847,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-498,"num_write_kb":-5303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.998251"},"pg_stats":[{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550222+0000","last_change":"2024-01-24T15:13:38.543675+0000","last_active":"2024-01-24T15:13:38.550222+0000","last_peered":"2024-01-24T15:13:38.550222+0000","last_clean":"2024-01-24T15:13:38.550222+0000","last_became_active":"2024-01-24T15:13:38.543012+0000","last_became_peered":"2024-01-24T15:13:38.543012+0000","last_unstale":"2024-01-24T15:13:38.550222+0000","last_undegraded":"2024-01-24T15:13:38.550222+0000","last_fullsized":"2024-01-24T15:13:38.550222+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3f","version":"177'3","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545770+0000","last_change":"2024-01-24T15:13:38.128076+0000","last_active":"2024-01-24T15:13:38.545770+0000","last_peered":"2024-01-24T15:13:38.545770+0000","last_clean":"2024-01-24T15:13:38.545770+0000","last_became_active":"2024-01-24T15:13:38.127791+0000","last_became_peered":"2024-01-24T15:13:38.127791+0000","last_unstale":"2024-01-24T15:13:38.545770+0000","last_undegraded":"2024-01-24T15:13:38.545770+0000","last_fullsized":"2024-01-24T15:13:38.545770+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557217+0000","last_change":"2024-01-24T15:13:38.554545+0000","last_active":"2024-01-24T15:13:38.557217+0000","last_peered":"2024-01-24T15:13:38.557217+0000","last_clean":"2024-01-24T15:13:38.557217+0000","last_became_active":"2024-01-24T15:13:38.554226+0000","last_became_peered":"2024-01-24T15:13:38.554226+0000","last_unstale":"2024-01-24T15:13:38.557217+0000","last_undegraded":"2024-01-24T15:13:38.557217+0000","last_fullsized":"2024-01-24T15:13:38.557217+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533665+0000","last_change":"2024-01-24T15:13:38.118261+0000","last_active":"2024-01-24T15:13:38.533665+0000","last_peered":"2024-01-24T15:13:38.533665+0000","last_clean":"2024-01-24T15:13:38.533665+0000","last_became_active":"2024-01-24T15:13:38.117923+0000","last_became_peered":"2024-01-24T15:13:38.117923+0000","last_unstale":"2024-01-24T15:13:38.533665+0000","last_undegraded":"2024-01-24T15:13:38.533665+0000","last_fullsized":"2024-01-24T15:13:38.533665+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562338+0000","last_change":"2024-01-24T15:13:38.559794+0000","last_active":"2024-01-24T15:13:38.562338+0000","last_peered":"2024-01-24T15:13:38.562338+0000","last_clean":"2024-01-24T15:13:38.562338+0000","last_became_active":"2024-01-24T15:13:38.559480+0000","last_became_peered":"2024-01-24T15:13:38.559480+0000","last_unstale":"2024-01-24T15:13:38.562338+0000","last_undegraded":"2024-01-24T15:13:38.562338+0000","last_fullsized":"2024-01-24T15:13:38.562338+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537224+0000","last_change":"2024-01-24T15:13:38.132008+0000","last_active":"2024-01-24T15:13:38.537224+0000","last_peered":"2024-01-24T15:13:38.537224+0000","last_clean":"2024-01-24T15:13:38.537224+0000","last_became_active":"2024-01-24T15:13:38.131711+0000","last_became_peered":"2024-01-24T15:13:38.131711+0000","last_unstale":"2024-01-24T15:13:38.537224+0000","last_undegraded":"2024-01-24T15:13:38.537224+0000","last_fullsized":"2024-01-24T15:13:38.537224+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558646+0000","last_change":"2024-01-24T15:13:38.548443+0000","last_active":"2024-01-24T15:13:38.558646+0000","last_peered":"2024-01-24T15:13:38.558646+0000","last_clean":"2024-01-24T15:13:38.558646+0000","last_became_active":"2024-01-24T15:13:38.546973+0000","last_became_peered":"2024-01-24T15:13:38.546973+0000","last_unstale":"2024-01-24T15:13:38.558646+0000","last_undegraded":"2024-01-24T15:13:38.558646+0000","last_fullsized":"2024-01-24T15:13:38.558646+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558098+0000","last_change":"2024-01-24T15:13:38.129340+0000","last_active":"2024-01-24T15:13:38.558098+0000","last_peered":"2024-01-24T15:13:38.558098+0000","last_clean":"2024-01-24T15:13:38.558098+0000","last_became_active":"2024-01-24T15:13:38.129057+0000","last_became_peered":"2024-01-24T15:13:38.129057+0000","last_unstale":"2024-01-24T15:13:38.558098+0000","last_undegraded":"2024-01-24T15:13:38.558098+0000","last_fullsized":"2024-01-24T15:13:38.558098+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.555919+0000","last_change":"2024-01-24T15:13:38.551741+0000","last_active":"2024-01-24T15:13:38.555919+0000","last_peered":"2024-01-24T15:13:38.555919+0000","last_clean":"2024-01-24T15:13:38.555919+0000","last_became_active":"2024-01-24T15:13:38.551165+0000","last_became_peered":"2024-01-24T15:13:38.551165+0000","last_unstale":"2024-01-24T15:13:38.555919+0000","last_undegraded":"2024-01-24T15:13:38.555919+0000","last_fullsized":"2024-01-24T15:13:38.555919+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557709+0000","last_change":"2024-01-24T15:13:38.122839+0000","last_active":"2024-01-24T15:13:38.557709+0000","last_peered":"2024-01-24T15:13:38.557709+0000","last_clean":"2024-01-24T15:13:38.557709+0000","last_became_active":"2024-01-24T15:13:38.121646+0000","last_became_peered":"2024-01-24T15:13:38.121646+0000","last_unstale":"2024-01-24T15:13:38.557709+0000","last_undegraded":"2024-01-24T15:13:38.557709+0000","last_fullsized":"2024-01-24T15:13:38.557709+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551349+0000","last_change":"2024-01-24T15:13:38.548421+0000","last_active":"2024-01-24T15:13:38.551349+0000","last_peered":"2024-01-24T15:13:38.551349+0000","last_clean":"2024-01-24T15:13:38.551349+0000","last_became_active":"2024-01-24T15:13:38.548168+0000","last_became_peered":"2024-01-24T15:13:38.548168+0000","last_unstale":"2024-01-24T15:13:38.551349+0000","last_undegraded":"2024-01-24T15:13:38.551349+0000","last_fullsized":"2024-01-24T15:13:38.551349+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3a","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550926+0000","last_change":"2024-01-24T15:13:38.131347+0000","last_active":"2024-01-24T15:13:38.550926+0000","last_peered":"2024-01-24T15:13:38.550926+0000","last_clean":"2024-01-24T15:13:38.550926+0000","last_became_active":"2024-01-24T15:13:38.127049+0000","last_became_peered":"2024-01-24T15:13:38.127049+0000","last_unstale":"2024-01-24T15:13:38.550926+0000","last_undegraded":"2024-01-24T15:13:38.550926+0000","last_fullsized":"2024-01-24T15:13:38.550926+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557161+0000","last_change":"2024-01-24T15:13:38.553689+0000","last_active":"2024-01-24T15:13:38.557161+0000","last_peered":"2024-01-24T15:13:38.557161+0000","last_clean":"2024-01-24T15:13:38.557161+0000","last_became_active":"2024-01-24T15:13:38.553388+0000","last_became_peered":"2024-01-24T15:13:38.553388+0000","last_unstale":"2024-01-24T15:13:38.557161+0000","last_undegraded":"2024-01-24T15:13:38.557161+0000","last_fullsized":"2024-01-24T15:13:38.557161+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.540830+0000","last_change":"2024-01-24T15:13:38.133854+0000","last_active":"2024-01-24T15:13:38.540830+0000","last_peered":"2024-01-24T15:13:38.540830+0000","last_clean":"2024-01-24T15:13:38.540830+0000","last_became_active":"2024-01-24T15:13:38.133583+0000","last_became_peered":"2024-01-24T15:13:38.133583+0000","last_unstale":"2024-01-24T15:13:38.540830+0000","last_undegraded":"2024-01-24T15:13:38.540830+0000","last_fullsized":"2024-01-24T15:13:38.540830+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559220+0000","last_change":"2024-01-24T15:13:38.556825+0000","last_active":"2024-01-24T15:13:38.559220+0000","last_peered":"2024-01-24T15:13:38.559220+0000","last_clean":"2024-01-24T15:13:38.559220+0000","last_became_active":"2024-01-24T15:13:38.556530+0000","last_became_peered":"2024-01-24T15:13:38.556530+0000","last_unstale":"2024-01-24T15:13:38.559220+0000","last_undegraded":"2024-01-24T15:13:38.559220+0000","last_fullsized":"2024-01-24T15:13:38.559220+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.544558+0000","last_change":"2024-01-24T15:13:38.129183+0000","last_active":"2024-01-24T15:13:38.544558+0000","last_peered":"2024-01-24T15:13:38.544558+0000","last_clean":"2024-01-24T15:13:38.544558+0000","last_became_active":"2024-01-24T15:13:38.128889+0000","last_became_peered":"2024-01-24T15:13:38.128889+0000","last_unstale":"2024-01-24T15:13:38.544558+0000","last_undegraded":"2024-01-24T15:13:38.544558+0000","last_fullsized":"2024-01-24T15:13:38.544558+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558035+0000","last_change":"2024-01-24T15:13:38.552886+0000","last_active":"2024-01-24T15:13:38.558035+0000","last_peered":"2024-01-24T15:13:38.558035+0000","last_clean":"2024-01-24T15:13:38.558035+0000","last_became_active":"2024-01-24T15:13:38.552602+0000","last_became_peered":"2024-01-24T15:13:38.552602+0000","last_unstale":"2024-01-24T15:13:38.558035+0000","last_undegraded":"2024-01-24T15:13:38.558035+0000","last_fullsized":"2024-01-24T15:13:38.558035+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537881+0000","last_change":"2024-01-24T15:13:38.122116+0000","last_active":"2024-01-24T15:13:38.537881+0000","last_peered":"2024-01-24T15:13:38.537881+0000","last_clean":"2024-01-24T15:13:38.537881+0000","last_became_active":"2024-01-24T15:13:38.121183+0000","last_became_peered":"2024-01-24T15:13:38.121183+0000","last_unstale":"2024-01-24T15:13:38.537881+0000","last_undegraded":"2024-01-24T15:13:38.537881+0000","last_fullsized":"2024-01-24T15:13:38.537881+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557777+0000","last_change":"2024-01-24T15:13:38.555214+0000","last_active":"2024-01-24T15:13:38.557777+0000","last_peered":"2024-01-24T15:13:38.557777+0000","last_clean":"2024-01-24T15:13:38.557777+0000","last_became_active":"2024-01-24T15:13:38.554915+0000","last_became_peered":"2024-01-24T15:13:38.554915+0000","last_unstale":"2024-01-24T15:13:38.557777+0000","last_undegraded":"2024-01-24T15:13:38.557777+0000","last_fullsized":"2024-01-24T15:13:38.557777+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.539484+0000","last_change":"2024-01-24T15:13:38.127556+0000","last_active":"2024-01-24T15:13:38.539484+0000","last_peered":"2024-01-24T15:13:38.539484+0000","last_clean":"2024-01-24T15:13:38.539484+0000","last_became_active":"2024-01-24T15:13:38.127215+0000","last_became_peered":"2024-01-24T15:13:38.127215+0000","last_unstale":"2024-01-24T15:13:38.539484+0000","last_undegraded":"2024-01-24T15:13:38.539484+0000","last_fullsized":"2024-01-24T15:13:38.539484+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557185+0000","last_change":"2024-01-24T15:13:38.551757+0000","last_active":"2024-01-24T15:13:38.557185+0000","last_peered":"2024-01-24T15:13:38.557185+0000","last_clean":"2024-01-24T15:13:38.557185+0000","last_became_active":"2024-01-24T15:13:38.551453+0000","last_became_peered":"2024-01-24T15:13:38.551453+0000","last_unstale":"2024-01-24T15:13:38.557185+0000","last_undegraded":"2024-01-24T15:13:38.557185+0000","last_fullsized":"2024-01-24T15:13:38.557185+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533408+0000","last_change":"2024-01-24T15:13:38.121049+0000","last_active":"2024-01-24T15:13:38.533408+0000","last_peered":"2024-01-24T15:13:38.533408+0000","last_clean":"2024-01-24T15:13:38.533408+0000","last_became_active":"2024-01-24T15:13:38.120804+0000","last_became_peered":"2024-01-24T15:13:38.120804+0000","last_unstale":"2024-01-24T15:13:38.533408+0000","last_undegraded":"2024-01-24T15:13:38.533408+0000","last_fullsized":"2024-01-24T15:13:38.533408+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551655+0000","last_change":"2024-01-24T15:13:38.547007+0000","last_active":"2024-01-24T15:13:38.551655+0000","last_peered":"2024-01-24T15:13:38.551655+0000","last_clean":"2024-01-24T15:13:38.551655+0000","last_became_active":"2024-01-24T15:13:38.546766+0000","last_became_peered":"2024-01-24T15:13:38.546766+0000","last_unstale":"2024-01-24T15:13:38.551655+0000","last_undegraded":"2024-01-24T15:13:38.551655+0000","last_fullsized":"2024-01-24T15:13:38.551655+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.34","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.565384+0000","last_change":"2024-01-24T15:13:38.123112+0000","last_active":"2024-01-24T15:13:38.565384+0000","last_peered":"2024-01-24T15:13:38.565384+0000","last_clean":"2024-01-24T15:13:38.565384+0000","last_became_active":"2024-01-24T15:13:38.122820+0000","last_became_peered":"2024-01-24T15:13:38.122820+0000","last_unstale":"2024-01-24T15:13:38.565384+0000","last_undegraded":"2024-01-24T15:13:38.565384+0000","last_fullsized":"2024-01-24T15:13:38.565384+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556831+0000","last_change":"2024-01-24T15:13:38.552223+0000","last_active":"2024-01-24T15:13:38.556831+0000","last_peered":"2024-01-24T15:13:38.556831+0000","last_clean":"2024-01-24T15:13:38.556831+0000","last_became_active":"2024-01-24T15:13:38.551704+0000","last_became_peered":"2024-01-24T15:13:38.551704+0000","last_unstale":"2024-01-24T15:13:38.556831+0000","last_undegraded":"2024-01-24T15:13:38.556831+0000","last_fullsized":"2024-01-24T15:13:38.556831+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.33","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.552767+0000","last_change":"2024-01-24T15:13:38.120932+0000","last_active":"2024-01-24T15:13:38.552767+0000","last_peered":"2024-01-24T15:13:38.552767+0000","last_clean":"2024-01-24T15:13:38.552767+0000","last_became_active":"2024-01-24T15:13:38.120622+0000","last_became_peered":"2024-01-24T15:13:38.120622+0000","last_unstale":"2024-01-24T15:13:38.552767+0000","last_undegraded":"2024-01-24T15:13:38.552767+0000","last_fullsized":"2024-01-24T15:13:38.552767+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551328+0000","last_change":"2024-01-24T15:13:38.547976+0000","last_active":"2024-01-24T15:13:38.551328+0000","last_peered":"2024-01-24T15:13:38.551328+0000","last_clean":"2024-01-24T15:13:38.551328+0000","last_became_active":"2024-01-24T15:13:38.547686+0000","last_became_peered":"2024-01-24T15:13:38.547686+0000","last_unstale":"2024-01-24T15:13:38.551328+0000","last_undegraded":"2024-01-24T15:13:38.551328+0000","last_fullsized":"2024-01-24T15:13:38.551328+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528656+0000","last_change":"2024-01-24T15:13:38.125651+0000","last_active":"2024-01-24T15:13:38.528656+0000","last_peered":"2024-01-24T15:13:38.528656+0000","last_clean":"2024-01-24T15:13:38.528656+0000","last_became_active":"2024-01-24T15:13:38.125334+0000","last_became_peered":"2024-01-24T15:13:38.125334+0000","last_unstale":"2024-01-24T15:13:38.528656+0000","last_undegraded":"2024-01-24T15:13:38.528656+0000","last_fullsized":"2024-01-24T15:13:38.528656+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545374+0000","last_change":"2024-01-24T15:13:38.543804+0000","last_active":"2024-01-24T15:13:38.545374+0000","last_peered":"2024-01-24T15:13:38.545374+0000","last_clean":"2024-01-24T15:13:38.545374+0000","last_became_active":"2024-01-24T15:13:38.543259+0000","last_became_peered":"2024-01-24T15:13:38.543259+0000","last_unstale":"2024-01-24T15:13:38.545374+0000","last_undegraded":"2024-01-24T15:13:38.545374+0000","last_fullsized":"2024-01-24T15:13:38.545374+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528152+0000","last_change":"2024-01-24T15:13:38.125234+0000","last_active":"2024-01-24T15:13:38.528152+0000","last_peered":"2024-01-24T15:13:38.528152+0000","last_clean":"2024-01-24T15:13:38.528152+0000","last_became_active":"2024-01-24T15:13:38.124520+0000","last_became_peered":"2024-01-24T15:13:38.124520+0000","last_unstale":"2024-01-24T15:13:38.528152+0000","last_undegraded":"2024-01-24T15:13:38.528152+0000","last_fullsized":"2024-01-24T15:13:38.528152+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557010+0000","last_change":"2024-01-24T15:13:38.548644+0000","last_active":"2024-01-24T15:13:38.557010+0000","last_peered":"2024-01-24T15:13:38.557010+0000","last_clean":"2024-01-24T15:13:38.557010+0000","last_became_active":"2024-01-24T15:13:38.548355+0000","last_became_peered":"2024-01-24T15:13:38.548355+0000","last_unstale":"2024-01-24T15:13:38.557010+0000","last_undegraded":"2024-01-24T15:13:38.557010+0000","last_fullsized":"2024-01-24T15:13:38.557010+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.30","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.552473+0000","last_change":"2024-01-24T15:13:38.123443+0000","last_active":"2024-01-24T15:13:38.552473+0000","last_peered":"2024-01-24T15:13:38.552473+0000","last_clean":"2024-01-24T15:13:38.552473+0000","last_became_active":"2024-01-24T15:13:38.123066+0000","last_became_peered":"2024-01-24T15:13:38.123066+0000","last_unstale":"2024-01-24T15:13:38.552473+0000","last_undegraded":"2024-01-24T15:13:38.552473+0000","last_fullsized":"2024-01-24T15:13:38.552473+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557443+0000","last_change":"2024-01-24T15:13:38.546142+0000","last_active":"2024-01-24T15:13:38.557443+0000","last_peered":"2024-01-24T15:13:38.557443+0000","last_clean":"2024-01-24T15:13:38.557443+0000","last_became_active":"2024-01-24T15:13:38.545868+0000","last_became_peered":"2024-01-24T15:13:38.545868+0000","last_unstale":"2024-01-24T15:13:38.557443+0000","last_undegraded":"2024-01-24T15:13:38.557443+0000","last_fullsized":"2024-01-24T15:13:38.557443+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537981+0000","last_change":"2024-01-24T15:13:38.121742+0000","last_active":"2024-01-24T15:13:38.537981+0000","last_peered":"2024-01-24T15:13:38.537981+0000","last_clean":"2024-01-24T15:13:38.537981+0000","last_became_active":"2024-01-24T15:13:38.121386+0000","last_became_peered":"2024-01-24T15:13:38.121386+0000","last_unstale":"2024-01-24T15:13:38.537981+0000","last_undegraded":"2024-01-24T15:13:38.537981+0000","last_fullsized":"2024-01-24T15:13:38.537981+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562294+0000","last_change":"2024-01-24T15:13:38.558575+0000","last_active":"2024-01-24T15:13:38.562294+0000","last_peered":"2024-01-24T15:13:38.562294+0000","last_clean":"2024-01-24T15:13:38.562294+0000","last_became_active":"2024-01-24T15:13:38.558314+0000","last_became_peered":"2024-01-24T15:13:38.558314+0000","last_unstale":"2024-01-24T15:13:38.562294+0000","last_undegraded":"2024-01-24T15:13:38.562294+0000","last_fullsized":"2024-01-24T15:13:38.562294+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532043+0000","last_change":"2024-01-24T15:13:38.122045+0000","last_active":"2024-01-24T15:13:38.532043+0000","last_peered":"2024-01-24T15:13:38.532043+0000","last_clean":"2024-01-24T15:13:38.532043+0000","last_became_active":"2024-01-24T15:13:38.121766+0000","last_became_peered":"2024-01-24T15:13:38.121766+0000","last_unstale":"2024-01-24T15:13:38.532043+0000","last_undegraded":"2024-01-24T15:13:38.532043+0000","last_fullsized":"2024-01-24T15:13:38.532043+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545141+0000","last_change":"2024-01-24T15:13:38.541723+0000","last_active":"2024-01-24T15:13:38.545141+0000","last_peered":"2024-01-24T15:13:38.545141+0000","last_clean":"2024-01-24T15:13:38.545141+0000","last_became_active":"2024-01-24T15:13:38.540083+0000","last_became_peered":"2024-01-24T15:13:38.540083+0000","last_unstale":"2024-01-24T15:13:38.545141+0000","last_undegraded":"2024-01-24T15:13:38.545141+0000","last_fullsized":"2024-01-24T15:13:38.545141+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2d","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558223+0000","last_change":"2024-01-24T15:13:38.130633+0000","last_active":"2024-01-24T15:13:38.558223+0000","last_peered":"2024-01-24T15:13:38.558223+0000","last_clean":"2024-01-24T15:13:38.558223+0000","last_became_active":"2024-01-24T15:13:38.130234+0000","last_became_peered":"2024-01-24T15:13:38.130234+0000","last_unstale":"2024-01-24T15:13:38.558223+0000","last_undegraded":"2024-01-24T15:13:38.558223+0000","last_fullsized":"2024-01-24T15:13:38.558223+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.555856+0000","last_change":"2024-01-24T15:13:38.551556+0000","last_active":"2024-01-24T15:13:38.555856+0000","last_peered":"2024-01-24T15:13:38.555856+0000","last_clean":"2024-01-24T15:13:38.555856+0000","last_became_active":"2024-01-24T15:13:38.551182+0000","last_became_peered":"2024-01-24T15:13:38.551182+0000","last_unstale":"2024-01-24T15:13:38.555856+0000","last_undegraded":"2024-01-24T15:13:38.555856+0000","last_fullsized":"2024-01-24T15:13:38.555856+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532179+0000","last_change":"2024-01-24T15:13:38.128558+0000","last_active":"2024-01-24T15:13:38.532179+0000","last_peered":"2024-01-24T15:13:38.532179+0000","last_clean":"2024-01-24T15:13:38.532179+0000","last_became_active":"2024-01-24T15:13:38.128075+0000","last_became_peered":"2024-01-24T15:13:38.128075+0000","last_unstale":"2024-01-24T15:13:38.532179+0000","last_undegraded":"2024-01-24T15:13:38.532179+0000","last_fullsized":"2024-01-24T15:13:38.532179+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556852+0000","last_change":"2024-01-24T15:13:38.552974+0000","last_active":"2024-01-24T15:13:38.556852+0000","last_peered":"2024-01-24T15:13:38.556852+0000","last_clean":"2024-01-24T15:13:38.556852+0000","last_became_active":"2024-01-24T15:13:38.552706+0000","last_became_peered":"2024-01-24T15:13:38.552706+0000","last_unstale":"2024-01-24T15:13:38.556852+0000","last_undegraded":"2024-01-24T15:13:38.556852+0000","last_fullsized":"2024-01-24T15:13:38.556852+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.553360+0000","last_change":"2024-01-24T15:13:38.127821+0000","last_active":"2024-01-24T15:13:38.553360+0000","last_peered":"2024-01-24T15:13:38.553360+0000","last_clean":"2024-01-24T15:13:38.553360+0000","last_became_active":"2024-01-24T15:13:38.127478+0000","last_became_peered":"2024-01-24T15:13:38.127478+0000","last_unstale":"2024-01-24T15:13:38.553360+0000","last_undegraded":"2024-01-24T15:13:38.553360+0000","last_fullsized":"2024-01-24T15:13:38.553360+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558279+0000","last_change":"2024-01-24T15:13:38.555643+0000","last_active":"2024-01-24T15:13:38.558279+0000","last_peered":"2024-01-24T15:13:38.558279+0000","last_clean":"2024-01-24T15:13:38.558279+0000","last_became_active":"2024-01-24T15:13:38.555338+0000","last_became_peered":"2024-01-24T15:13:38.555338+0000","last_unstale":"2024-01-24T15:13:38.558279+0000","last_undegraded":"2024-01-24T15:13:38.558279+0000","last_fullsized":"2024-01-24T15:13:38.558279+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528767+0000","last_change":"2024-01-24T15:13:38.119386+0000","last_active":"2024-01-24T15:13:38.528767+0000","last_peered":"2024-01-24T15:13:38.528767+0000","last_clean":"2024-01-24T15:13:38.528767+0000","last_became_active":"2024-01-24T15:13:38.118881+0000","last_became_peered":"2024-01-24T15:13:38.118881+0000","last_unstale":"2024-01-24T15:13:38.528767+0000","last_undegraded":"2024-01-24T15:13:38.528767+0000","last_fullsized":"2024-01-24T15:13:38.528767+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557245+0000","last_change":"2024-01-24T15:13:38.552282+0000","last_active":"2024-01-24T15:13:38.557245+0000","last_peered":"2024-01-24T15:13:38.557245+0000","last_clean":"2024-01-24T15:13:38.557245+0000","last_became_active":"2024-01-24T15:13:38.552007+0000","last_became_peered":"2024-01-24T15:13:38.552007+0000","last_unstale":"2024-01-24T15:13:38.557245+0000","last_undegraded":"2024-01-24T15:13:38.557245+0000","last_fullsized":"2024-01-24T15:13:38.557245+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533440+0000","last_change":"2024-01-24T15:13:38.126343+0000","last_active":"2024-01-24T15:13:38.533440+0000","last_peered":"2024-01-24T15:13:38.533440+0000","last_clean":"2024-01-24T15:13:38.533440+0000","last_became_active":"2024-01-24T15:13:38.126099+0000","last_became_peered":"2024-01-24T15:13:38.126099+0000","last_unstale":"2024-01-24T15:13:38.533440+0000","last_undegraded":"2024-01-24T15:13:38.533440+0000","last_fullsized":"2024-01-24T15:13:38.533440+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562224+0000","last_change":"2024-01-24T15:13:38.557923+0000","last_active":"2024-01-24T15:13:38.562224+0000","last_peered":"2024-01-24T15:13:38.562224+0000","last_clean":"2024-01-24T15:13:38.562224+0000","last_became_active":"2024-01-24T15:13:38.557635+0000","last_became_peered":"2024-01-24T15:13:38.557635+0000","last_unstale":"2024-01-24T15:13:38.562224+0000","last_undegraded":"2024-01-24T15:13:38.562224+0000","last_fullsized":"2024-01-24T15:13:38.562224+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535538+0000","last_change":"2024-01-24T15:13:38.117376+0000","last_active":"2024-01-24T15:13:38.535538+0000","last_peered":"2024-01-24T15:13:38.535538+0000","last_clean":"2024-01-24T15:13:38.535538+0000","last_became_active":"2024-01-24T15:13:38.117052+0000","last_became_peered":"2024-01-24T15:13:38.117052+0000","last_unstale":"2024-01-24T15:13:38.535538+0000","last_undegraded":"2024-01-24T15:13:38.535538+0000","last_fullsized":"2024-01-24T15:13:38.535538+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.554021+0000","last_change":"2024-01-24T15:13:38.546194+0000","last_active":"2024-01-24T15:13:38.554021+0000","last_peered":"2024-01-24T15:13:38.554021+0000","last_clean":"2024-01-24T15:13:38.554021+0000","last_became_active":"2024-01-24T15:13:38.545907+0000","last_became_peered":"2024-01-24T15:13:38.545907+0000","last_unstale":"2024-01-24T15:13:38.554021+0000","last_undegraded":"2024-01-24T15:13:38.554021+0000","last_fullsized":"2024-01-24T15:13:38.554021+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537470+0000","last_change":"2024-01-24T15:13:38.120448+0000","last_active":"2024-01-24T15:13:38.537470+0000","last_peered":"2024-01-24T15:13:38.537470+0000","last_clean":"2024-01-24T15:13:38.537470+0000","last_became_active":"2024-01-24T15:13:38.120165+0000","last_became_peered":"2024-01-24T15:13:38.120165+0000","last_unstale":"2024-01-24T15:13:38.537470+0000","last_undegraded":"2024-01-24T15:13:38.537470+0000","last_fullsized":"2024-01-24T15:13:38.537470+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558719+0000","last_change":"2024-01-24T15:13:38.555417+0000","last_active":"2024-01-24T15:13:38.558719+0000","last_peered":"2024-01-24T15:13:38.558719+0000","last_clean":"2024-01-24T15:13:38.558719+0000","last_became_active":"2024-01-24T15:13:38.555118+0000","last_became_peered":"2024-01-24T15:13:38.555118+0000","last_unstale":"2024-01-24T15:13:38.558719+0000","last_undegraded":"2024-01-24T15:13:38.558719+0000","last_fullsized":"2024-01-24T15:13:38.558719+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537325+0000","last_change":"2024-01-24T15:13:38.128489+0000","last_active":"2024-01-24T15:13:38.537325+0000","last_peered":"2024-01-24T15:13:38.537325+0000","last_clean":"2024-01-24T15:13:38.537325+0000","last_became_active":"2024-01-24T15:13:38.128202+0000","last_became_peered":"2024-01-24T15:13:38.128202+0000","last_unstale":"2024-01-24T15:13:38.537325+0000","last_undegraded":"2024-01-24T15:13:38.537325+0000","last_fullsized":"2024-01-24T15:13:38.537325+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562480+0000","last_change":"2024-01-24T15:13:38.561900+0000","last_active":"2024-01-24T15:13:38.562480+0000","last_peered":"2024-01-24T15:13:38.562480+0000","last_clean":"2024-01-24T15:13:38.562480+0000","last_became_active":"2024-01-24T15:13:38.561655+0000","last_became_peered":"2024-01-24T15:13:38.561655+0000","last_unstale":"2024-01-24T15:13:38.562480+0000","last_undegraded":"2024-01-24T15:13:38.562480+0000","last_fullsized":"2024-01-24T15:13:38.562480+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542822+0000","last_change":"2024-01-24T15:13:38.130503+0000","last_active":"2024-01-24T15:13:38.542822+0000","last_peered":"2024-01-24T15:13:38.542822+0000","last_clean":"2024-01-24T15:13:38.542822+0000","last_became_active":"2024-01-24T15:13:38.129933+0000","last_became_peered":"2024-01-24T15:13:38.129933+0000","last_unstale":"2024-01-24T15:13:38.542822+0000","last_undegraded":"2024-01-24T15:13:38.542822+0000","last_fullsized":"2024-01-24T15:13:38.542822+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.548146+0000","last_change":"2024-01-24T15:13:38.546025+0000","last_active":"2024-01-24T15:13:38.548146+0000","last_peered":"2024-01-24T15:13:38.548146+0000","last_clean":"2024-01-24T15:13:38.548146+0000","last_became_active":"2024-01-24T15:13:38.545702+0000","last_became_peered":"2024-01-24T15:13:38.545702+0000","last_unstale":"2024-01-24T15:13:38.548146+0000","last_undegraded":"2024-01-24T15:13:38.548146+0000","last_fullsized":"2024-01-24T15:13:38.548146+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.546793+0000","last_change":"2024-01-24T15:13:38.128387+0000","last_active":"2024-01-24T15:13:38.546793+0000","last_peered":"2024-01-24T15:13:38.546793+0000","last_clean":"2024-01-24T15:13:38.546793+0000","last_became_active":"2024-01-24T15:13:38.128088+0000","last_became_peered":"2024-01-24T15:13:38.128088+0000","last_unstale":"2024-01-24T15:13:38.546793+0000","last_undegraded":"2024-01-24T15:13:38.546793+0000","last_fullsized":"2024-01-24T15:13:38.546793+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.560725+0000","last_change":"2024-01-24T15:13:38.554569+0000","last_active":"2024-01-24T15:13:38.560725+0000","last_peered":"2024-01-24T15:13:38.560725+0000","last_clean":"2024-01-24T15:13:38.560725+0000","last_became_active":"2024-01-24T15:13:38.554110+0000","last_became_peered":"2024-01-24T15:13:38.554110+0000","last_unstale":"2024-01-24T15:13:38.560725+0000","last_undegraded":"2024-01-24T15:13:38.560725+0000","last_fullsized":"2024-01-24T15:13:38.560725+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535857+0000","last_change":"2024-01-24T15:13:38.127508+0000","last_active":"2024-01-24T15:13:38.535857+0000","last_peered":"2024-01-24T15:13:38.535857+0000","last_clean":"2024-01-24T15:13:38.535857+0000","last_became_active":"2024-01-24T15:13:38.127242+0000","last_became_peered":"2024-01-24T15:13:38.127242+0000","last_unstale":"2024-01-24T15:13:38.535857+0000","last_undegraded":"2024-01-24T15:13:38.535857+0000","last_fullsized":"2024-01-24T15:13:38.535857+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557111+0000","last_change":"2024-01-24T15:13:38.552603+0000","last_active":"2024-01-24T15:13:38.557111+0000","last_peered":"2024-01-24T15:13:38.557111+0000","last_clean":"2024-01-24T15:13:38.557111+0000","last_became_active":"2024-01-24T15:13:38.550713+0000","last_became_peered":"2024-01-24T15:13:38.550713+0000","last_unstale":"2024-01-24T15:13:38.557111+0000","last_undegraded":"2024-01-24T15:13:38.557111+0000","last_fullsized":"2024-01-24T15:13:38.557111+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535420+0000","last_change":"2024-01-24T15:13:38.121734+0000","last_active":"2024-01-24T15:13:38.535420+0000","last_peered":"2024-01-24T15:13:38.535420+0000","last_clean":"2024-01-24T15:13:38.535420+0000","last_became_active":"2024-01-24T15:13:38.121067+0000","last_became_peered":"2024-01-24T15:13:38.121067+0000","last_unstale":"2024-01-24T15:13:38.535420+0000","last_undegraded":"2024-01-24T15:13:38.535420+0000","last_fullsized":"2024-01-24T15:13:38.535420+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545355+0000","last_change":"2024-01-24T15:13:38.543615+0000","last_active":"2024-01-24T15:13:38.545355+0000","last_peered":"2024-01-24T15:13:38.545355+0000","last_clean":"2024-01-24T15:13:38.545355+0000","last_became_active":"2024-01-24T15:13:38.541511+0000","last_became_peered":"2024-01-24T15:13:38.541511+0000","last_unstale":"2024-01-24T15:13:38.545355+0000","last_undegraded":"2024-01-24T15:13:38.545355+0000","last_fullsized":"2024-01-24T15:13:38.545355+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.544708+0000","last_change":"2024-01-24T15:13:38.122176+0000","last_active":"2024-01-24T15:13:38.544708+0000","last_peered":"2024-01-24T15:13:38.544708+0000","last_clean":"2024-01-24T15:13:38.544708+0000","last_became_active":"2024-01-24T15:13:38.121900+0000","last_became_peered":"2024-01-24T15:13:38.121900+0000","last_unstale":"2024-01-24T15:13:38.544708+0000","last_undegraded":"2024-01-24T15:13:38.544708+0000","last_fullsized":"2024-01-24T15:13:38.544708+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550198+0000","last_change":"2024-01-24T15:13:38.546422+0000","last_active":"2024-01-24T15:13:38.550198+0000","last_peered":"2024-01-24T15:13:38.550198+0000","last_clean":"2024-01-24T15:13:38.550198+0000","last_became_active":"2024-01-24T15:13:38.546168+0000","last_became_peered":"2024-01-24T15:13:38.546168+0000","last_unstale":"2024-01-24T15:13:38.550198+0000","last_undegraded":"2024-01-24T15:13:38.550198+0000","last_fullsized":"2024-01-24T15:13:38.550198+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.538459+0000","last_change":"2024-01-24T15:13:38.126860+0000","last_active":"2024-01-24T15:13:38.538459+0000","last_peered":"2024-01-24T15:13:38.538459+0000","last_clean":"2024-01-24T15:13:38.538459+0000","last_became_active":"2024-01-24T15:13:38.126528+0000","last_became_peered":"2024-01-24T15:13:38.126528+0000","last_unstale":"2024-01-24T15:13:38.538459+0000","last_undegraded":"2024-01-24T15:13:38.538459+0000","last_fullsized":"2024-01-24T15:13:38.538459+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558013+0000","last_change":"2024-01-24T15:13:38.551200+0000","last_active":"2024-01-24T15:13:38.558013+0000","last_peered":"2024-01-24T15:13:38.558013+0000","last_clean":"2024-01-24T15:13:38.558013+0000","last_became_active":"2024-01-24T15:13:38.550887+0000","last_became_peered":"2024-01-24T15:13:38.550887+0000","last_unstale":"2024-01-24T15:13:38.558013+0000","last_undegraded":"2024-01-24T15:13:38.558013+0000","last_fullsized":"2024-01-24T15:13:38.558013+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559181+0000","last_change":"2024-01-24T15:13:38.135912+0000","last_active":"2024-01-24T15:13:38.559181+0000","last_peered":"2024-01-24T15:13:38.559181+0000","last_clean":"2024-01-24T15:13:38.559181+0000","last_became_active":"2024-01-24T15:13:38.135637+0000","last_became_peered":"2024-01-24T15:13:38.135637+0000","last_unstale":"2024-01-24T15:13:38.559181+0000","last_undegraded":"2024-01-24T15:13:38.559181+0000","last_fullsized":"2024-01-24T15:13:38.559181+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.547990+0000","last_change":"2024-01-24T15:13:38.542686+0000","last_active":"2024-01-24T15:13:38.547990+0000","last_peered":"2024-01-24T15:13:38.547990+0000","last_clean":"2024-01-24T15:13:38.547990+0000","last_became_active":"2024-01-24T15:13:38.542392+0000","last_became_peered":"2024-01-24T15:13:38.542392+0000","last_unstale":"2024-01-24T15:13:38.547990+0000","last_undegraded":"2024-01-24T15:13:38.547990+0000","last_fullsized":"2024-01-24T15:13:38.547990+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.548937+0000","last_change":"2024-01-24T15:13:38.125904+0000","last_active":"2024-01-24T15:13:38.548937+0000","last_peered":"2024-01-24T15:13:38.548937+0000","last_clean":"2024-01-24T15:13:38.548937+0000","last_became_active":"2024-01-24T15:13:38.125609+0000","last_became_peered":"2024-01-24T15:13:38.125609+0000","last_unstale":"2024-01-24T15:13:38.548937+0000","last_undegraded":"2024-01-24T15:13:38.548937+0000","last_fullsized":"2024-01-24T15:13:38.548937+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557762+0000","last_change":"2024-01-24T15:13:38.553959+0000","last_active":"2024-01-24T15:13:38.557762+0000","last_peered":"2024-01-24T15:13:38.557762+0000","last_clean":"2024-01-24T15:13:38.557762+0000","last_became_active":"2024-01-24T15:13:38.553651+0000","last_became_peered":"2024-01-24T15:13:38.553651+0000","last_unstale":"2024-01-24T15:13:38.557762+0000","last_undegraded":"2024-01-24T15:13:38.557762+0000","last_fullsized":"2024-01-24T15:13:38.557762+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557845+0000","last_change":"2024-01-24T15:13:38.129150+0000","last_active":"2024-01-24T15:13:38.557845+0000","last_peered":"2024-01-24T15:13:38.557845+0000","last_clean":"2024-01-24T15:13:38.557845+0000","last_became_active":"2024-01-24T15:13:38.128848+0000","last_became_peered":"2024-01-24T15:13:38.128848+0000","last_unstale":"2024-01-24T15:13:38.557845+0000","last_undegraded":"2024-01-24T15:13:38.557845+0000","last_fullsized":"2024-01-24T15:13:38.557845+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558317+0000","last_change":"2024-01-24T15:13:38.555479+0000","last_active":"2024-01-24T15:13:38.558317+0000","last_peered":"2024-01-24T15:13:38.558317+0000","last_clean":"2024-01-24T15:13:38.558317+0000","last_became_active":"2024-01-24T15:13:38.555204+0000","last_became_peered":"2024-01-24T15:13:38.555204+0000","last_unstale":"2024-01-24T15:13:38.558317+0000","last_undegraded":"2024-01-24T15:13:38.558317+0000","last_fullsized":"2024-01-24T15:13:38.558317+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545284+0000","last_change":"2024-01-24T15:13:38.122342+0000","last_active":"2024-01-24T15:13:38.545284+0000","last_peered":"2024-01-24T15:13:38.545284+0000","last_clean":"2024-01-24T15:13:38.545284+0000","last_became_active":"2024-01-24T15:13:38.122043+0000","last_became_peered":"2024-01-24T15:13:38.122043+0000","last_unstale":"2024-01-24T15:13:38.545284+0000","last_undegraded":"2024-01-24T15:13:38.545284+0000","last_fullsized":"2024-01-24T15:13:38.545284+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550294+0000","last_change":"2024-01-24T15:13:38.547021+0000","last_active":"2024-01-24T15:13:38.550294+0000","last_peered":"2024-01-24T15:13:38.550294+0000","last_clean":"2024-01-24T15:13:38.550294+0000","last_became_active":"2024-01-24T15:13:38.546231+0000","last_became_peered":"2024-01-24T15:13:38.546231+0000","last_unstale":"2024-01-24T15:13:38.550294+0000","last_undegraded":"2024-01-24T15:13:38.550294+0000","last_fullsized":"2024-01-24T15:13:38.550294+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528666+0000","last_change":"2024-01-24T15:13:38.124808+0000","last_active":"2024-01-24T15:13:38.528666+0000","last_peered":"2024-01-24T15:13:38.528666+0000","last_clean":"2024-01-24T15:13:38.528666+0000","last_became_active":"2024-01-24T15:13:38.124504+0000","last_became_peered":"2024-01-24T15:13:38.124504+0000","last_unstale":"2024-01-24T15:13:38.528666+0000","last_undegraded":"2024-01-24T15:13:38.528666+0000","last_fullsized":"2024-01-24T15:13:38.528666+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562306+0000","last_change":"2024-01-24T15:13:38.558977+0000","last_active":"2024-01-24T15:13:38.562306+0000","last_peered":"2024-01-24T15:13:38.562306+0000","last_clean":"2024-01-24T15:13:38.562306+0000","last_became_active":"2024-01-24T15:13:38.558713+0000","last_became_peered":"2024-01-24T15:13:38.558713+0000","last_unstale":"2024-01-24T15:13:38.562306+0000","last_undegraded":"2024-01-24T15:13:38.562306+0000","last_fullsized":"2024-01-24T15:13:38.562306+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533932+0000","last_change":"2024-01-24T15:13:38.125055+0000","last_active":"2024-01-24T15:13:38.533932+0000","last_peered":"2024-01-24T15:13:38.533932+0000","last_clean":"2024-01-24T15:13:38.533932+0000","last_became_active":"2024-01-24T15:13:38.124694+0000","last_became_peered":"2024-01-24T15:13:38.124694+0000","last_unstale":"2024-01-24T15:13:38.533932+0000","last_undegraded":"2024-01-24T15:13:38.533932+0000","last_fullsized":"2024-01-24T15:13:38.533932+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542376+0000","last_change":"2024-01-24T15:13:38.540674+0000","last_active":"2024-01-24T15:13:38.542376+0000","last_peered":"2024-01-24T15:13:38.542376+0000","last_clean":"2024-01-24T15:13:38.542376+0000","last_became_active":"2024-01-24T15:13:38.540381+0000","last_became_peered":"2024-01-24T15:13:38.540381+0000","last_unstale":"2024-01-24T15:13:38.542376+0000","last_undegraded":"2024-01-24T15:13:38.542376+0000","last_fullsized":"2024-01-24T15:13:38.542376+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528151+0000","last_change":"2024-01-24T15:13:38.119586+0000","last_active":"2024-01-24T15:13:38.528151+0000","last_peered":"2024-01-24T15:13:38.528151+0000","last_clean":"2024-01-24T15:13:38.528151+0000","last_became_active":"2024-01-24T15:13:38.119285+0000","last_became_peered":"2024-01-24T15:13:38.119285+0000","last_unstale":"2024-01-24T15:13:38.528151+0000","last_undegraded":"2024-01-24T15:13:38.528151+0000","last_fullsized":"2024-01-24T15:13:38.528151+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551159+0000","last_change":"2024-01-24T15:13:38.544043+0000","last_active":"2024-01-24T15:13:38.551159+0000","last_peered":"2024-01-24T15:13:38.551159+0000","last_clean":"2024-01-24T15:13:38.551159+0000","last_became_active":"2024-01-24T15:13:38.543741+0000","last_became_peered":"2024-01-24T15:13:38.543741+0000","last_unstale":"2024-01-24T15:13:38.551159+0000","last_undegraded":"2024-01-24T15:13:38.551159+0000","last_fullsized":"2024-01-24T15:13:38.551159+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.544453+0000","last_change":"2024-01-24T15:13:38.124704+0000","last_active":"2024-01-24T15:13:38.544453+0000","last_peered":"2024-01-24T15:13:38.544453+0000","last_clean":"2024-01-24T15:13:38.544453+0000","last_became_active":"2024-01-24T15:13:38.124427+0000","last_became_peered":"2024-01-24T15:13:38.124427+0000","last_unstale":"2024-01-24T15:13:38.544453+0000","last_undegraded":"2024-01-24T15:13:38.544453+0000","last_fullsized":"2024-01-24T15:13:38.544453+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559264+0000","last_change":"2024-01-24T15:13:38.555038+0000","last_active":"2024-01-24T15:13:38.559264+0000","last_peered":"2024-01-24T15:13:38.559264+0000","last_clean":"2024-01-24T15:13:38.559264+0000","last_became_active":"2024-01-24T15:13:38.554763+0000","last_became_peered":"2024-01-24T15:13:38.554763+0000","last_unstale":"2024-01-24T15:13:38.559264+0000","last_undegraded":"2024-01-24T15:13:38.559264+0000","last_fullsized":"2024-01-24T15:13:38.559264+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559251+0000","last_change":"2024-01-24T15:13:38.122389+0000","last_active":"2024-01-24T15:13:38.559251+0000","last_peered":"2024-01-24T15:13:38.559251+0000","last_clean":"2024-01-24T15:13:38.559251+0000","last_became_active":"2024-01-24T15:13:38.121569+0000","last_became_peered":"2024-01-24T15:13:38.121569+0000","last_unstale":"2024-01-24T15:13:38.559251+0000","last_undegraded":"2024-01-24T15:13:38.559251+0000","last_fullsized":"2024-01-24T15:13:38.559251+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557360+0000","last_change":"2024-01-24T15:13:38.549783+0000","last_active":"2024-01-24T15:13:38.557360+0000","last_peered":"2024-01-24T15:13:38.557360+0000","last_clean":"2024-01-24T15:13:38.557360+0000","last_became_active":"2024-01-24T15:13:38.549374+0000","last_became_peered":"2024-01-24T15:13:38.549374+0000","last_unstale":"2024-01-24T15:13:38.557360+0000","last_undegraded":"2024-01-24T15:13:38.557360+0000","last_fullsized":"2024-01-24T15:13:38.557360+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557933+0000","last_change":"2024-01-24T15:13:38.121798+0000","last_active":"2024-01-24T15:13:38.557933+0000","last_peered":"2024-01-24T15:13:38.557933+0000","last_clean":"2024-01-24T15:13:38.557933+0000","last_became_active":"2024-01-24T15:13:38.121333+0000","last_became_peered":"2024-01-24T15:13:38.121333+0000","last_unstale":"2024-01-24T15:13:38.557933+0000","last_undegraded":"2024-01-24T15:13:38.557933+0000","last_fullsized":"2024-01-24T15:13:38.557933+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558881+0000","last_change":"2024-01-24T15:13:38.556149+0000","last_active":"2024-01-24T15:13:38.558881+0000","last_peered":"2024-01-24T15:13:38.558881+0000","last_clean":"2024-01-24T15:13:38.558881+0000","last_became_active":"2024-01-24T15:13:38.553256+0000","last_became_peered":"2024-01-24T15:13:38.553256+0000","last_unstale":"2024-01-24T15:13:38.558881+0000","last_undegraded":"2024-01-24T15:13:38.558881+0000","last_fullsized":"2024-01-24T15:13:38.558881+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.529610+0000","last_change":"2024-01-24T15:13:38.125875+0000","last_active":"2024-01-24T15:13:38.529610+0000","last_peered":"2024-01-24T15:13:38.529610+0000","last_clean":"2024-01-24T15:13:38.529610+0000","last_became_active":"2024-01-24T15:13:38.125536+0000","last_became_peered":"2024-01-24T15:13:38.125536+0000","last_unstale":"2024-01-24T15:13:38.529610+0000","last_undegraded":"2024-01-24T15:13:38.529610+0000","last_fullsized":"2024-01-24T15:13:38.529610+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.555881+0000","last_change":"2024-01-24T15:13:38.551671+0000","last_active":"2024-01-24T15:13:38.555881+0000","last_peered":"2024-01-24T15:13:38.555881+0000","last_clean":"2024-01-24T15:13:38.555881+0000","last_became_active":"2024-01-24T15:13:38.550779+0000","last_became_peered":"2024-01-24T15:13:38.550779+0000","last_unstale":"2024-01-24T15:13:38.555881+0000","last_undegraded":"2024-01-24T15:13:38.555881+0000","last_fullsized":"2024-01-24T15:13:38.555881+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532219+0000","last_change":"2024-01-24T15:13:38.122494+0000","last_active":"2024-01-24T15:13:38.532219+0000","last_peered":"2024-01-24T15:13:38.532219+0000","last_clean":"2024-01-24T15:13:38.532219+0000","last_became_active":"2024-01-24T15:13:38.122120+0000","last_became_peered":"2024-01-24T15:13:38.122120+0000","last_unstale":"2024-01-24T15:13:38.532219+0000","last_undegraded":"2024-01-24T15:13:38.532219+0000","last_fullsized":"2024-01-24T15:13:38.532219+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.560704+0000","last_change":"2024-01-24T15:13:38.554397+0000","last_active":"2024-01-24T15:13:38.560704+0000","last_peered":"2024-01-24T15:13:38.560704+0000","last_clean":"2024-01-24T15:13:38.560704+0000","last_became_active":"2024-01-24T15:13:38.553773+0000","last_became_peered":"2024-01-24T15:13:38.553773+0000","last_unstale":"2024-01-24T15:13:38.560704+0000","last_undegraded":"2024-01-24T15:13:38.560704+0000","last_fullsized":"2024-01-24T15:13:38.560704+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561628+0000","last_change":"2024-01-24T15:13:38.129725+0000","last_active":"2024-01-24T15:13:38.561628+0000","last_peered":"2024-01-24T15:13:38.561628+0000","last_clean":"2024-01-24T15:13:38.561628+0000","last_became_active":"2024-01-24T15:13:38.129199+0000","last_became_peered":"2024-01-24T15:13:38.129199+0000","last_unstale":"2024-01-24T15:13:38.561628+0000","last_undegraded":"2024-01-24T15:13:38.561628+0000","last_fullsized":"2024-01-24T15:13:38.561628+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556320+0000","last_change":"2024-01-24T15:13:38.550473+0000","last_active":"2024-01-24T15:13:38.556320+0000","last_peered":"2024-01-24T15:13:38.556320+0000","last_clean":"2024-01-24T15:13:38.556320+0000","last_became_active":"2024-01-24T15:13:38.550227+0000","last_became_peered":"2024-01-24T15:13:38.550227+0000","last_unstale":"2024-01-24T15:13:38.556320+0000","last_undegraded":"2024-01-24T15:13:38.556320+0000","last_fullsized":"2024-01-24T15:13:38.556320+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542174+0000","last_change":"2024-01-24T15:13:38.130818+0000","last_active":"2024-01-24T15:13:38.542174+0000","last_peered":"2024-01-24T15:13:38.542174+0000","last_clean":"2024-01-24T15:13:38.542174+0000","last_became_active":"2024-01-24T15:13:38.130326+0000","last_became_peered":"2024-01-24T15:13:38.130326+0000","last_unstale":"2024-01-24T15:13:38.542174+0000","last_undegraded":"2024-01-24T15:13:38.542174+0000","last_fullsized":"2024-01-24T15:13:38.542174+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562244+0000","last_change":"2024-01-24T15:13:38.557459+0000","last_active":"2024-01-24T15:13:38.562244+0000","last_peered":"2024-01-24T15:13:38.562244+0000","last_clean":"2024-01-24T15:13:38.562244+0000","last_became_active":"2024-01-24T15:13:38.557186+0000","last_became_peered":"2024-01-24T15:13:38.557186+0000","last_unstale":"2024-01-24T15:13:38.562244+0000","last_undegraded":"2024-01-24T15:13:38.562244+0000","last_fullsized":"2024-01-24T15:13:38.562244+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.541879+0000","last_change":"2024-01-24T15:13:38.134878+0000","last_active":"2024-01-24T15:13:38.541879+0000","last_peered":"2024-01-24T15:13:38.541879+0000","last_clean":"2024-01-24T15:13:38.541879+0000","last_became_active":"2024-01-24T15:13:38.134606+0000","last_became_peered":"2024-01-24T15:13:38.134606+0000","last_unstale":"2024-01-24T15:13:38.541879+0000","last_undegraded":"2024-01-24T15:13:38.541879+0000","last_fullsized":"2024-01-24T15:13:38.541879+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556378+0000","last_change":"2024-01-24T15:13:38.552294+0000","last_active":"2024-01-24T15:13:38.556378+0000","last_peered":"2024-01-24T15:13:38.556378+0000","last_clean":"2024-01-24T15:13:38.556378+0000","last_became_active":"2024-01-24T15:13:38.551919+0000","last_became_peered":"2024-01-24T15:13:38.551919+0000","last_unstale":"2024-01-24T15:13:38.556378+0000","last_undegraded":"2024-01-24T15:13:38.556378+0000","last_fullsized":"2024-01-24T15:13:38.556378+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535737+0000","last_change":"2024-01-24T15:13:38.132176+0000","last_active":"2024-01-24T15:13:38.535737+0000","last_peered":"2024-01-24T15:13:38.535737+0000","last_clean":"2024-01-24T15:13:38.535737+0000","last_became_active":"2024-01-24T15:13:38.131843+0000","last_became_peered":"2024-01-24T15:13:38.131843+0000","last_unstale":"2024-01-24T15:13:38.535737+0000","last_undegraded":"2024-01-24T15:13:38.535737+0000","last_fullsized":"2024-01-24T15:13:38.535737+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551132+0000","last_change":"2024-01-24T15:13:38.547885+0000","last_active":"2024-01-24T15:13:38.551132+0000","last_peered":"2024-01-24T15:13:38.551132+0000","last_clean":"2024-01-24T15:13:38.551132+0000","last_became_active":"2024-01-24T15:13:38.547529+0000","last_became_peered":"2024-01-24T15:13:38.547529+0000","last_unstale":"2024-01-24T15:13:38.551132+0000","last_undegraded":"2024-01-24T15:13:38.551132+0000","last_fullsized":"2024-01-24T15:13:38.551132+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559494+0000","last_change":"2024-01-24T15:13:38.134325+0000","last_active":"2024-01-24T15:13:38.559494+0000","last_peered":"2024-01-24T15:13:38.559494+0000","last_clean":"2024-01-24T15:13:38.559494+0000","last_became_active":"2024-01-24T15:13:38.134045+0000","last_became_peered":"2024-01-24T15:13:38.134045+0000","last_unstale":"2024-01-24T15:13:38.559494+0000","last_undegraded":"2024-01-24T15:13:38.559494+0000","last_fullsized":"2024-01-24T15:13:38.559494+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558867+0000","last_change":"2024-01-24T15:13:38.554889+0000","last_active":"2024-01-24T15:13:38.558867+0000","last_peered":"2024-01-24T15:13:38.558867+0000","last_clean":"2024-01-24T15:13:38.558867+0000","last_became_active":"2024-01-24T15:13:38.554624+0000","last_became_peered":"2024-01-24T15:13:38.554624+0000","last_unstale":"2024-01-24T15:13:38.558867+0000","last_undegraded":"2024-01-24T15:13:38.558867+0000","last_fullsized":"2024-01-24T15:13:38.558867+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537614+0000","last_change":"2024-01-24T15:13:38.126218+0000","last_active":"2024-01-24T15:13:38.537614+0000","last_peered":"2024-01-24T15:13:38.537614+0000","last_clean":"2024-01-24T15:13:38.537614+0000","last_became_active":"2024-01-24T15:13:38.125899+0000","last_became_peered":"2024-01-24T15:13:38.125899+0000","last_unstale":"2024-01-24T15:13:38.537614+0000","last_undegraded":"2024-01-24T15:13:38.537614+0000","last_fullsized":"2024-01-24T15:13:38.537614+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561397+0000","last_change":"2024-01-24T15:13:38.557841+0000","last_active":"2024-01-24T15:13:38.561397+0000","last_peered":"2024-01-24T15:13:38.561397+0000","last_clean":"2024-01-24T15:13:38.561397+0000","last_became_active":"2024-01-24T15:13:38.557543+0000","last_became_peered":"2024-01-24T15:13:38.557543+0000","last_unstale":"2024-01-24T15:13:38.561397+0000","last_undegraded":"2024-01-24T15:13:38.561397+0000","last_fullsized":"2024-01-24T15:13:38.561397+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.538204+0000","last_change":"2024-01-24T15:13:38.122661+0000","last_active":"2024-01-24T15:13:38.538204+0000","last_peered":"2024-01-24T15:13:38.538204+0000","last_clean":"2024-01-24T15:13:38.538204+0000","last_became_active":"2024-01-24T15:13:38.122384+0000","last_became_peered":"2024-01-24T15:13:38.122384+0000","last_unstale":"2024-01-24T15:13:38.538204+0000","last_undegraded":"2024-01-24T15:13:38.538204+0000","last_fullsized":"2024-01-24T15:13:38.538204+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557058+0000","last_change":"2024-01-24T15:13:38.553790+0000","last_active":"2024-01-24T15:13:38.557058+0000","last_peered":"2024-01-24T15:13:38.557058+0000","last_clean":"2024-01-24T15:13:38.557058+0000","last_became_active":"2024-01-24T15:13:38.553498+0000","last_became_peered":"2024-01-24T15:13:38.553498+0000","last_unstale":"2024-01-24T15:13:38.557058+0000","last_undegraded":"2024-01-24T15:13:38.557058+0000","last_fullsized":"2024-01-24T15:13:38.557058+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535549+0000","last_change":"2024-01-24T15:13:38.120324+0000","last_active":"2024-01-24T15:13:38.535549+0000","last_peered":"2024-01-24T15:13:38.535549+0000","last_clean":"2024-01-24T15:13:38.535549+0000","last_became_active":"2024-01-24T15:13:38.120007+0000","last_became_peered":"2024-01-24T15:13:38.120007+0000","last_unstale":"2024-01-24T15:13:38.535549+0000","last_undegraded":"2024-01-24T15:13:38.535549+0000","last_fullsized":"2024-01-24T15:13:38.535549+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558444+0000","last_change":"2024-01-24T15:13:38.552923+0000","last_active":"2024-01-24T15:13:38.558444+0000","last_peered":"2024-01-24T15:13:38.558444+0000","last_clean":"2024-01-24T15:13:38.558444+0000","last_became_active":"2024-01-24T15:13:38.552646+0000","last_became_peered":"2024-01-24T15:13:38.552646+0000","last_unstale":"2024-01-24T15:13:38.558444+0000","last_undegraded":"2024-01-24T15:13:38.558444+0000","last_fullsized":"2024-01-24T15:13:38.558444+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.539981+0000","last_change":"2024-01-24T15:13:38.129534+0000","last_active":"2024-01-24T15:13:38.539981+0000","last_peered":"2024-01-24T15:13:38.539981+0000","last_clean":"2024-01-24T15:13:38.539981+0000","last_became_active":"2024-01-24T15:13:38.128874+0000","last_became_peered":"2024-01-24T15:13:38.128874+0000","last_unstale":"2024-01-24T15:13:38.539981+0000","last_undegraded":"2024-01-24T15:13:38.539981+0000","last_fullsized":"2024-01-24T15:13:38.539981+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551365+0000","last_change":"2024-01-24T15:13:38.549194+0000","last_active":"2024-01-24T15:13:38.551365+0000","last_peered":"2024-01-24T15:13:38.551365+0000","last_clean":"2024-01-24T15:13:38.551365+0000","last_became_active":"2024-01-24T15:13:38.548929+0000","last_became_peered":"2024-01-24T15:13:38.548929+0000","last_unstale":"2024-01-24T15:13:38.551365+0000","last_undegraded":"2024-01-24T15:13:38.551365+0000","last_fullsized":"2024-01-24T15:13:38.551365+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542276+0000","last_change":"2024-01-24T15:13:38.130595+0000","last_active":"2024-01-24T15:13:38.542276+0000","last_peered":"2024-01-24T15:13:38.542276+0000","last_clean":"2024-01-24T15:13:38.542276+0000","last_became_active":"2024-01-24T15:13:38.130025+0000","last_became_peered":"2024-01-24T15:13:38.130025+0000","last_unstale":"2024-01-24T15:13:38.542276+0000","last_undegraded":"2024-01-24T15:13:38.542276+0000","last_fullsized":"2024-01-24T15:13:38.542276+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557093+0000","last_change":"2024-01-24T15:13:38.548613+0000","last_active":"2024-01-24T15:13:38.557093+0000","last_peered":"2024-01-24T15:13:38.557093+0000","last_clean":"2024-01-24T15:13:38.557093+0000","last_became_active":"2024-01-24T15:13:38.548288+0000","last_became_peered":"2024-01-24T15:13:38.548288+0000","last_unstale":"2024-01-24T15:13:38.557093+0000","last_undegraded":"2024-01-24T15:13:38.557093+0000","last_fullsized":"2024-01-24T15:13:38.557093+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535292+0000","last_change":"2024-01-24T15:13:38.131770+0000","last_active":"2024-01-24T15:13:38.535292+0000","last_peered":"2024-01-24T15:13:38.535292+0000","last_clean":"2024-01-24T15:13:38.535292+0000","last_became_active":"2024-01-24T15:13:38.131497+0000","last_became_peered":"2024-01-24T15:13:38.131497+0000","last_unstale":"2024-01-24T15:13:38.535292+0000","last_undegraded":"2024-01-24T15:13:38.535292+0000","last_fullsized":"2024-01-24T15:13:38.535292+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.546235+0000","last_change":"2024-01-24T15:13:38.538155+0000","last_active":"2024-01-24T15:13:38.546235+0000","last_peered":"2024-01-24T15:13:38.546235+0000","last_clean":"2024-01-24T15:13:38.546235+0000","last_became_active":"2024-01-24T15:13:38.537877+0000","last_became_peered":"2024-01-24T15:13:38.537877+0000","last_unstale":"2024-01-24T15:13:38.546235+0000","last_undegraded":"2024-01-24T15:13:38.546235+0000","last_fullsized":"2024-01-24T15:13:38.546235+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.531693+0000","last_change":"2024-01-24T15:13:38.124495+0000","last_active":"2024-01-24T15:13:38.531693+0000","last_peered":"2024-01-24T15:13:38.531693+0000","last_clean":"2024-01-24T15:13:38.531693+0000","last_became_active":"2024-01-24T15:13:38.124175+0000","last_became_peered":"2024-01-24T15:13:38.124175+0000","last_unstale":"2024-01-24T15:13:38.531693+0000","last_undegraded":"2024-01-24T15:13:38.531693+0000","last_fullsized":"2024-01-24T15:13:38.531693+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559906+0000","last_change":"2024-01-24T15:13:38.557558+0000","last_active":"2024-01-24T15:13:38.559906+0000","last_peered":"2024-01-24T15:13:38.559906+0000","last_clean":"2024-01-24T15:13:38.559906+0000","last_became_active":"2024-01-24T15:13:38.557290+0000","last_became_peered":"2024-01-24T15:13:38.557290+0000","last_unstale":"2024-01-24T15:13:38.559906+0000","last_undegraded":"2024-01-24T15:13:38.559906+0000","last_fullsized":"2024-01-24T15:13:38.559906+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535494+0000","last_change":"2024-01-24T15:13:38.121901+0000","last_active":"2024-01-24T15:13:38.535494+0000","last_peered":"2024-01-24T15:13:38.535494+0000","last_clean":"2024-01-24T15:13:38.535494+0000","last_became_active":"2024-01-24T15:13:38.121422+0000","last_became_peered":"2024-01-24T15:13:38.121422+0000","last_unstale":"2024-01-24T15:13:38.535494+0000","last_undegraded":"2024-01-24T15:13:38.535494+0000","last_fullsized":"2024-01-24T15:13:38.535494+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":144,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535646+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:13:38.535646+0000","last_peered":"2024-01-24T15:13:38.535646+0000","last_clean":"2024-01-24T15:13:38.535646+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:13:38.535646+0000","last_undegraded":"2024-01-24T15:13:38.535646+0000","last_fullsized":"2024-01-24T15:13:38.535646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558463+0000","last_change":"2024-01-24T15:13:38.552222+0000","last_active":"2024-01-24T15:13:38.558463+0000","last_peered":"2024-01-24T15:13:38.558463+0000","last_clean":"2024-01-24T15:13:38.558463+0000","last_became_active":"2024-01-24T15:13:38.551916+0000","last_became_peered":"2024-01-24T15:13:38.551916+0000","last_unstale":"2024-01-24T15:13:38.558463+0000","last_undegraded":"2024-01-24T15:13:38.558463+0000","last_fullsized":"2024-01-24T15:13:38.558463+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561816+0000","last_change":"2024-01-24T15:13:38.128196+0000","last_active":"2024-01-24T15:13:38.561816+0000","last_peered":"2024-01-24T15:13:38.561816+0000","last_clean":"2024-01-24T15:13:38.561816+0000","last_became_active":"2024-01-24T15:13:38.127782+0000","last_became_peered":"2024-01-24T15:13:38.127782+0000","last_unstale":"2024-01-24T15:13:38.561816+0000","last_undegraded":"2024-01-24T15:13:38.561816+0000","last_fullsized":"2024-01-24T15:13:38.561816+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":162,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.534014+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:13:38.534014+0000","last_peered":"2024-01-24T15:13:38.534014+0000","last_clean":"2024-01-24T15:13:38.534014+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:13:38.534014+0000","last_undegraded":"2024-01-24T15:13:38.534014+0000","last_fullsized":"2024-01-24T15:13:38.534014+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.547053+0000","last_change":"2024-01-24T15:13:38.538061+0000","last_active":"2024-01-24T15:13:38.547053+0000","last_peered":"2024-01-24T15:13:38.547053+0000","last_clean":"2024-01-24T15:13:38.547053+0000","last_became_active":"2024-01-24T15:13:38.537769+0000","last_became_peered":"2024-01-24T15:13:38.537769+0000","last_unstale":"2024-01-24T15:13:38.547053+0000","last_undegraded":"2024-01-24T15:13:38.547053+0000","last_fullsized":"2024-01-24T15:13:38.547053+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532992+0000","last_change":"2024-01-24T15:13:38.125400+0000","last_active":"2024-01-24T15:13:38.532992+0000","last_peered":"2024-01-24T15:13:38.532992+0000","last_clean":"2024-01-24T15:13:38.532992+0000","last_became_active":"2024-01-24T15:13:38.124725+0000","last_became_peered":"2024-01-24T15:13:38.124725+0000","last_unstale":"2024-01-24T15:13:38.532992+0000","last_undegraded":"2024-01-24T15:13:38.532992+0000","last_fullsized":"2024-01-24T15:13:38.532992+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528231+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:13:38.528231+0000","last_peered":"2024-01-24T15:13:38.528231+0000","last_clean":"2024-01-24T15:13:38.528231+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:13:38.528231+0000","last_undegraded":"2024-01-24T15:13:38.528231+0000","last_fullsized":"2024-01-24T15:13:38.528231+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561686+0000","last_change":"2024-01-24T15:13:38.554780+0000","last_active":"2024-01-24T15:13:38.561686+0000","last_peered":"2024-01-24T15:13:38.561686+0000","last_clean":"2024-01-24T15:13:38.561686+0000","last_became_active":"2024-01-24T15:13:38.554497+0000","last_became_peered":"2024-01-24T15:13:38.554497+0000","last_unstale":"2024-01-24T15:13:38.561686+0000","last_undegraded":"2024-01-24T15:13:38.561686+0000","last_fullsized":"2024-01-24T15:13:38.561686+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"177'3","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558878+0000","last_change":"2024-01-24T15:13:38.131986+0000","last_active":"2024-01-24T15:13:38.558878+0000","last_peered":"2024-01-24T15:13:38.558878+0000","last_clean":"2024-01-24T15:13:38.558878+0000","last_became_active":"2024-01-24T15:13:38.131663+0000","last_became_peered":"2024-01-24T15:13:38.131663+0000","last_unstale":"2024-01-24T15:13:38.558878+0000","last_undegraded":"2024-01-24T15:13:38.558878+0000","last_fullsized":"2024-01-24T15:13:38.558878+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":162,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533441+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:13:38.533441+0000","last_peered":"2024-01-24T15:13:38.533441+0000","last_clean":"2024-01-24T15:13:38.533441+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:13:38.533441+0000","last_undegraded":"2024-01-24T15:13:38.533441+0000","last_fullsized":"2024-01-24T15:13:38.533441+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":147,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.540654+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:13:38.540654+0000","last_peered":"2024-01-24T15:13:38.540654+0000","last_clean":"2024-01-24T15:13:38.540654+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:13:38.540654+0000","last_undegraded":"2024-01-24T15:13:38.540654+0000","last_fullsized":"2024-01-24T15:13:38.540654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562314+0000","last_change":"2024-01-24T15:13:38.559088+0000","last_active":"2024-01-24T15:13:38.562314+0000","last_peered":"2024-01-24T15:13:38.562314+0000","last_clean":"2024-01-24T15:13:38.562314+0000","last_became_active":"2024-01-24T15:13:38.558845+0000","last_became_peered":"2024-01-24T15:13:38.558845+0000","last_unstale":"2024-01-24T15:13:38.562314+0000","last_undegraded":"2024-01-24T15:13:38.562314+0000","last_fullsized":"2024-01-24T15:13:38.562314+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.541272+0000","last_change":"2024-01-24T15:13:38.134030+0000","last_active":"2024-01-24T15:13:38.541272+0000","last_peered":"2024-01-24T15:13:38.541272+0000","last_clean":"2024-01-24T15:13:38.541272+0000","last_became_active":"2024-01-24T15:13:38.133632+0000","last_became_peered":"2024-01-24T15:13:38.133632+0000","last_unstale":"2024-01-24T15:13:38.541272+0000","last_undegraded":"2024-01-24T15:13:38.541272+0000","last_fullsized":"2024-01-24T15:13:38.541272+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":145,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.540557+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:13:38.540557+0000","last_peered":"2024-01-24T15:13:38.540557+0000","last_clean":"2024-01-24T15:13:38.540557+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:13:38.540557+0000","last_undegraded":"2024-01-24T15:13:38.540557+0000","last_fullsized":"2024-01-24T15:13:38.540557+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558848+0000","last_change":"2024-01-24T15:13:38.553941+0000","last_active":"2024-01-24T15:13:38.558848+0000","last_peered":"2024-01-24T15:13:38.558848+0000","last_clean":"2024-01-24T15:13:38.558848+0000","last_became_active":"2024-01-24T15:13:38.552844+0000","last_became_peered":"2024-01-24T15:13:38.552844+0000","last_unstale":"2024-01-24T15:13:38.558848+0000","last_undegraded":"2024-01-24T15:13:38.558848+0000","last_fullsized":"2024-01-24T15:13:38.558848+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559449+0000","last_change":"2024-01-24T15:13:38.134046+0000","last_active":"2024-01-24T15:13:38.559449+0000","last_peered":"2024-01-24T15:13:38.559449+0000","last_clean":"2024-01-24T15:13:38.559449+0000","last_became_active":"2024-01-24T15:13:38.133216+0000","last_became_peered":"2024-01-24T15:13:38.133216+0000","last_unstale":"2024-01-24T15:13:38.559449+0000","last_undegraded":"2024-01-24T15:13:38.559449+0000","last_fullsized":"2024-01-24T15:13:38.559449+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":145,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.543025+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:13:38.543025+0000","last_peered":"2024-01-24T15:13:38.543025+0000","last_clean":"2024-01-24T15:13:38.543025+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:13:38.543025+0000","last_undegraded":"2024-01-24T15:13:38.543025+0000","last_fullsized":"2024-01-24T15:13:38.543025+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550309+0000","last_change":"2024-01-24T15:13:38.542717+0000","last_active":"2024-01-24T15:13:38.550309+0000","last_peered":"2024-01-24T15:13:38.550309+0000","last_clean":"2024-01-24T15:13:38.550309+0000","last_became_active":"2024-01-24T15:13:38.542424+0000","last_became_peered":"2024-01-24T15:13:38.542424+0000","last_unstale":"2024-01-24T15:13:38.550309+0000","last_undegraded":"2024-01-24T15:13:38.550309+0000","last_fullsized":"2024-01-24T15:13:38.550309+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.549726+0000","last_change":"2024-01-24T15:13:38.121778+0000","last_active":"2024-01-24T15:13:38.549726+0000","last_peered":"2024-01-24T15:13:38.549726+0000","last_clean":"2024-01-24T15:13:38.549726+0000","last_became_active":"2024-01-24T15:13:38.121489+0000","last_became_peered":"2024-01-24T15:13:38.121489+0000","last_unstale":"2024-01-24T15:13:38.549726+0000","last_undegraded":"2024-01-24T15:13:38.549726+0000","last_fullsized":"2024-01-24T15:13:38.549726+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537697+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:13:38.537697+0000","last_peered":"2024-01-24T15:13:38.537697+0000","last_clean":"2024-01-24T15:13:38.537697+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:13:38.537697+0000","last_undegraded":"2024-01-24T15:13:38.537697+0000","last_fullsized":"2024-01-24T15:13:38.537697+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558185+0000","last_change":"2024-01-24T15:13:38.550725+0000","last_active":"2024-01-24T15:13:38.558185+0000","last_peered":"2024-01-24T15:13:38.558185+0000","last_clean":"2024-01-24T15:13:38.558185+0000","last_became_active":"2024-01-24T15:13:38.550442+0000","last_became_peered":"2024-01-24T15:13:38.550442+0000","last_unstale":"2024-01-24T15:13:38.558185+0000","last_undegraded":"2024-01-24T15:13:38.558185+0000","last_fullsized":"2024-01-24T15:13:38.558185+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535563+0000","last_change":"2024-01-24T15:13:38.131236+0000","last_active":"2024-01-24T15:13:38.535563+0000","last_peered":"2024-01-24T15:13:38.535563+0000","last_clean":"2024-01-24T15:13:38.535563+0000","last_became_active":"2024-01-24T15:13:38.130890+0000","last_became_peered":"2024-01-24T15:13:38.130890+0000","last_unstale":"2024-01-24T15:13:38.535563+0000","last_undegraded":"2024-01-24T15:13:38.535563+0000","last_fullsized":"2024-01-24T15:13:38.535563+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537474+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:13:38.537474+0000","last_peered":"2024-01-24T15:13:38.537474+0000","last_clean":"2024-01-24T15:13:38.537474+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:13:38.537474+0000","last_undegraded":"2024-01-24T15:13:38.537474+0000","last_fullsized":"2024-01-24T15:13:38.537474+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575751,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303360,"kb_used_data":2168,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068480,"statfs":{"total":96636764160,"available":96326123520,"internally_reserved":0,"allocated":2220032,"data_stored":1270917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.59899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.28000000000000003},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.628,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.28899999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.1080000000000001},"last":0.442}]},{"osd":1,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.33300000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.505,"5min":0.40000000000000002,"15min":0.34799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.625}]},{"osd":2,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0329999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.38},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0880000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.41199999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.60899999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.33200000000000002},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.31900000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.69399999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.39600000000000002,"5min":0.38,"15min":0.34799999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.36199999999999999},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.31900000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.46400000000000002,"5min":0.40899999999999997,"15min":0.40500000000000003},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.81100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575752,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307464,"kb_used_data":2176,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064376,"statfs":{"total":96636764160,"available":96321921024,"internally_reserved":0,"allocated":2228224,"data_stored":1270993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.86599999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.42699999999999999,"15min":0.37},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57199999999999995}]},{"osd":1,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.84199999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77100000000000002,"5min":0.81200000000000006,"15min":1.258},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.40100000000000002},"max":{"1min":0.82299999999999995,"5min":0.85899999999999999,"15min":1.2430000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61599999999999999}]},{"osd":3,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.47199999999999998,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.39000000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.48699999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.317},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.628},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32400000000000001,"15min":0.30499999999999999},"max":{"1min":0.79300000000000004,"5min":0.80100000000000005,"15min":1.002},"last":0.64300000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.32500000000000001},"max":{"1min":0.76000000000000001,"5min":0.78100000000000003,"15min":0.83899999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":0.749,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.69399999999999995}]},{"osd":7,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.40899999999999997,"5min":0.372,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.90100000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.38},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":1.0429999999999999},"last":0.59199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575751,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307452,"kb_used_data":2164,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064388,"statfs":{"total":96636764160,"available":96321933312,"internally_reserved":0,"allocated":2215936,"data_stored":1270847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.56399999999999995,"5min":0.754,"15min":0.754},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.443,"15min":0.435},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.26800000000000002},"max":{"1min":0.55800000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.433}]},{"osd":2,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.33700000000000002,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.57599999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44600000000000001,"15min":0.44700000000000001},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.28199999999999997},"max":{"1min":0.56200000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.32800000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"min":{"1min":0.32400000000000001,"5min":0.314,"15min":0.309},"max":{"1min":0.58399999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.372},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.35799999999999998,"15min":0.313},"max":{"1min":0.59899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.51800000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.86099999999999999,"5min":1.044,"15min":1.0649999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.51400000000000001,"5min":0.47399999999999998,"15min":0.41699999999999998},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.58099999999999996}]},{"osd":5,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.49199999999999999,"15min":0.46300000000000002},"max":{"1min":0.83999999999999997,"5min":1.014,"15min":1.0780000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.439},"max":{"1min":0.86699999999999999,"5min":1.105,"15min":1.105},"last":0.60299999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.874,"5min":1.0029999999999999,"15min":1.069},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.56599999999999995,"5min":0.499,"15min":0.48199999999999998},"max":{"1min":0.92500000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.84299999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.83799999999999997,"5min":1.127,"15min":1.127},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.77500000000000002,"15min":0.76700000000000002},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.42799999999999999},"max":{"1min":0.88900000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.80900000000000005}]}]},{"osd":0,"up_from":14,"seq":60129543049,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307464,"kb_used_data":2176,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064376,"statfs":{"total":96636764160,"available":96321921024,"internally_reserved":0,"allocated":2228224,"data_stored":1271118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.28100000000000003,"5min":0.27500000000000002,"15min":0.247},"max":{"1min":0.60199999999999998,"5min":0.63400000000000001,"15min":0.66400000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.27700000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.54500000000000004}]},{"osd":2,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.29399999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.81999999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.84399999999999997},"last":0.496}]},{"osd":3,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.44600000000000001},"min":{"1min":0.317,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.65500000000000003,"15min":0.67000000000000004},"last":0.626},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.52000000000000002},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.90000000000000002},"last":0.66300000000000003}]},{"osd":4,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.433},"max":{"1min":0.99399999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.38700000000000001},"max":{"1min":0.995,"5min":0.995,"15min":1.0249999999999999},"last":0.78100000000000003}]},{"osd":5,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.70699999999999996},"min":{"1min":0.54900000000000004,"5min":0.498,"15min":0.42099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.94199999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.46899999999999997},"max":{"1min":1.022,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0820000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81100000000000005}]},{"osd":7,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0469999999999999,"5min":1.143,"15min":1.3260000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.375},"last":0.87}]}]},{"osd":2,"up_from":13,"seq":55834575751,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311564,"kb_used_data":2180,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060276,"statfs":{"total":96636764160,"available":96317722624,"internally_reserved":0,"allocated":2232320,"data_stored":1272937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.27200000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.60099999999999998,"5min":0.65000000000000002,"15min":0.66800000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.45100000000000001,"15min":0.45200000000000001},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.70399999999999996},"last":0.29099999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.25600000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.73599999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.46300000000000002},"min":{"1min":0.32500000000000001,"5min":0.29699999999999999,"15min":0.29599999999999999},"max":{"1min":0.622,"5min":0.69699999999999995,"15min":0.70099999999999996},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.73299999999999998},"last":0.34999999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96199999999999997,"5min":1.196,"15min":1.196},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":1,"15min":1.089},"last":0.38100000000000001}]},{"osd":5,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54400000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.111},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.46899999999999997,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.147,"15min":1.1830000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.64700000000000002,"5min":0.47699999999999998,"15min":0.45200000000000001},"max":{"1min":0.99399999999999999,"5min":0.998,"15min":1.129},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.58899999999999997,"5min":0.47299999999999998,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.98999999999999999,"15min":1.1950000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.753,"15min":0.746},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.497},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.248},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.65000000000000002,"5min":0.51500000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.24,"15min":1.24},"last":0.63100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307460,"kb_used_data":2172,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064380,"statfs":{"total":96636764160,"available":96321925120,"internally_reserved":0,"allocated":2224128,"data_stored":1271829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39800000000000002,"15min":0.39200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.247},"max":{"1min":0.56899999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.70199999999999996},"last":0.36199999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41899999999999998,"15min":0.41399999999999998},"min":{"1min":0.311,"5min":0.28100000000000003,"15min":0.27400000000000002},"max":{"1min":0.59999999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.34799999999999998,"5min":0.316,"15min":0.30499999999999999},"max":{"1min":0.67900000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.433}]},{"osd":2,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.497,"15min":0.502},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51300000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.41099999999999998},"max":{"1min":0.83299999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95999999999999996,"15min":1.5580000000000001},"last":0.58799999999999997}]},{"osd":5,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.1879999999999999,"15min":1.6240000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.647},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.45700000000000002,"15min":0.42699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.522},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.073,"5min":1.079,"15min":1.079},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.44800000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1499999999999999,"5min":1.246,"15min":1.246},"last":0.64400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575752,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307456,"kb_used_data":2168,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064384,"statfs":{"total":96636764160,"available":96321929216,"internally_reserved":0,"allocated":2220032,"data_stored":1270917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.46700000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.83699999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76500000000000001,"5min":0.78400000000000003,"15min":0.84199999999999997},"last":0.57099999999999995}]},{"osd":1,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.45500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.72499999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82999999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.60699999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.84699999999999998,"15min":0.88300000000000001},"last":0.496},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.626,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.372},"max":{"1min":0.79800000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52800000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.38400000000000001},"max":{"1min":0.79900000000000004,"5min":0.80400000000000005,"15min":0.89500000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.42399999999999999},"max":{"1min":0.71899999999999997,"5min":0.82199999999999995,"15min":0.92300000000000004},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":0.61299999999999999,"5min":0.79200000000000004,"15min":0.81799999999999995},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66400000000000003,"5min":0.72499999999999998,"15min":0.85499999999999998},"last":0.46000000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.68300000000000005,"5min":0.78500000000000003,"15min":0.85499999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.38200000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.86799999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.33600000000000002},"max":{"1min":0.65200000000000002,"5min":0.77500000000000002,"15min":0.82799999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.72599999999999998,"5min":0.81999999999999995,"15min":0.96599999999999997},"last":0.63200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575752,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307456,"kb_used_data":2168,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064384,"statfs":{"total":96636764160,"available":96321929216,"internally_reserved":0,"allocated":2220032,"data_stored":1270869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.38600000000000001,"15min":0.35099999999999998},"max":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":1.4470000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.81599999999999995},"last":0.53300000000000003}]},{"osd":1,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.82499999999999996},"last":0.435},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":1.4930000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.625,"15min":0.627},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.79300000000000004,"5min":0.80900000000000005,"15min":0.86499999999999999},"last":0.503},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":1.53},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.80800000000000005,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41899999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.65400000000000003,"5min":0.74399999999999999,"15min":0.84699999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":0.77200000000000002,"15min":0.80100000000000005},"last":0.38900000000000001}]},{"osd":6,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.5,"15min":0.504},"min":{"1min":0.374,"5min":0.35499999999999998,"15min":0.33400000000000002},"max":{"1min":0.65100000000000002,"5min":0.79400000000000004,"15min":0.81499999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.78500000000000003,"15min":0.81299999999999994},"last":0.44400000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.54900000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:13:43.432 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-24T15:13:43.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.597+0000 7f3e8c385700 1 Processor -- start 2024-01-24T15:13:43.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.597+0000 7f3e8c385700 1 -- start start 2024-01-24T15:13:43.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.598+0000 7f3e8c385700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e840bdc70 0x7f3e840bca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:43.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.598+0000 7f3e8c385700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3e8413a660 0x7f3e840c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:43.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.598+0000 7f3e8c385700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8413a4b0 con 0x7f3e8413a0d0 2024-01-24T15:13:43.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.598+0000 7f3e8c385700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e841548f0 con 0x7f3e8413a660 2024-01-24T15:13:43.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.598+0000 7f3e8c385700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e84155750 con 0x7f3e840bdc70 2024-01-24T15:13:43.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e8a922700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3e8413a660 0x7f3e840c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:43.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e8a922700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3e8413a660 0x7f3e840c2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40524/0 (socket says 172.21.15.29:40524) 2024-01-24T15:13:43.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e8a922700 1 -- 172.21.15.29:0/3923025546 learned_addr learned my addr 172.21.15.29:0/3923025546 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:43.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e89920700 1 --2- 172.21.15.29:0/3923025546 >> v2:172.21.15.163:3300/0 conn(0x7f3e840bdc70 0x7f3e840bca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:43.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e8a922700 1 -- 172.21.15.29:0/3923025546 >> v2:172.21.15.163:3300/0 conn(0x7f3e840bdc70 msgr2=0x7f3e840bca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:43.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e8a922700 1 --2- 172.21.15.29:0/3923025546 >> v2:172.21.15.163:3300/0 conn(0x7f3e840bdc70 0x7f3e840bca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e8a922700 1 -- 172.21.15.29:0/3923025546 >> v1:172.21.15.29:6789/0 conn(0x7f3e8413a0d0 legacy=0x7f3e840bd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e8a922700 1 -- 172.21.15.29:0/3923025546 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e840c59d0 con 0x7f3e8413a660 2024-01-24T15:13:43.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e89920700 1 --2- 172.21.15.29:0/3923025546 >> v2:172.21.15.163:3300/0 conn(0x7f3e840bdc70 0x7f3e840bca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:13:43.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e8a922700 1 --2- 172.21.15.29:0/3923025546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e8413a660 0x7f3e840c2ea0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f3e8406ae40 tx=0x7f3e7c009840).ready entity=mon.0 client_cookie=1e1a7e864267c6e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:43.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e8911f700 1 -- 172.21.15.29:0/3923025546 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c00e070 con 0x7f3e8413a660 2024-01-24T15:13:43.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e8911f700 1 -- 172.21.15.29:0/3923025546 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c00ba50 con 0x7f3e8413a660 2024-01-24T15:13:43.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.599+0000 7f3e8911f700 1 -- 172.21.15.29:0/3923025546 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c00bdd0 con 0x7f3e8413a660 2024-01-24T15:13:43.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.600+0000 7f3e8c385700 1 -- 172.21.15.29:0/3923025546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e8413a660 msgr2=0x7f3e840c2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.600+0000 7f3e8c385700 1 --2- 172.21.15.29:0/3923025546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e8413a660 0x7f3e840c2ea0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f3e8406ae40 tx=0x7f3e7c009840).stop 2024-01-24T15:13:43.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.600+0000 7f3e8c385700 1 -- 172.21.15.29:0/3923025546 shutdown_connections 2024-01-24T15:13:43.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.600+0000 7f3e8c385700 1 --2- 172.21.15.29:0/3923025546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e8413a660 0x7f3e840c2ea0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.600+0000 7f3e8c385700 1 --2- 172.21.15.29:0/3923025546 >> v2:172.21.15.163:3300/0 conn(0x7f3e840bdc70 0x7f3e840bca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.612 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.600+0000 7f3e8c385700 1 -- 172.21.15.29:0/3923025546 >> 172.21.15.29:0/3923025546 conn(0x7f3e840c4570 msgr2=0x7f3e840c4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:43.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.600+0000 7f3e8c385700 1 -- 172.21.15.29:0/3923025546 shutdown_connections 2024-01-24T15:13:43.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.600+0000 7f3e8c385700 1 -- 172.21.15.29:0/3923025546 wait complete. 2024-01-24T15:13:43.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.601+0000 7f3e8c385700 1 Processor -- start 2024-01-24T15:13:43.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.601+0000 7f3e8c385700 1 -- start start 2024-01-24T15:13:43.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8c385700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e840bdc70 0x7f3e840abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:43.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8c385700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e8413a0d0 0x7f3e840c17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:43.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8c385700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e8413a660 0x7f3e84150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:43.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8c385700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840c1ce0 con 0x7f3e840bdc70 2024-01-24T15:13:43.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8c385700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840acfb0 con 0x7f3e8413a660 2024-01-24T15:13:43.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8c385700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840ac3a0 con 0x7f3e8413a0d0 2024-01-24T15:13:43.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8a121700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e840bdc70 0x7f3e840abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:43.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8a121700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e840bdc70 0x7f3e840abd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40528/0 (socket says 172.21.15.29:40528) 2024-01-24T15:13:43.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8a121700 1 -- 172.21.15.29:0/3110153805 learned_addr learned my addr 172.21.15.29:0/3110153805 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:43.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8a922700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e8413a660 0x7f3e84150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:43.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e89920700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e8413a0d0 0x7f3e840c17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:43.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8a121700 1 -- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e8413a0d0 msgr2=0x7f3e840c17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8a121700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e8413a0d0 0x7f3e840c17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8a121700 1 -- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e8413a660 msgr2=0x7f3e84150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8a121700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e8413a660 0x7f3e84150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8a121700 1 -- 172.21.15.29:0/3110153805 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c009ae0 con 0x7f3e840bdc70 2024-01-24T15:13:43.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e89920700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e8413a0d0 0x7f3e840c17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:13:43.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.602+0000 7f3e8a922700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e8413a660 0x7f3e84150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:13:43.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.603+0000 7f3e8a121700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e840bdc70 0x7f3e840abd10 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f3e8000c4c0 tx=0x7f3e80007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:43.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.603+0000 7f3e767fc700 1 -- 172.21.15.29:0/3110153805 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80007020 con 0x7f3e840bdc70 2024-01-24T15:13:43.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.603+0000 7f3e767fc700 1 -- 172.21.15.29:0/3110153805 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e80001e90 con 0x7f3e840bdc70 2024-01-24T15:13:43.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.603+0000 7f3e8c385700 1 -- 172.21.15.29:0/3110153805 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e840ac680 con 0x7f3e840bdc70 2024-01-24T15:13:43.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.603+0000 7f3e767fc700 1 -- 172.21.15.29:0/3110153805 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e800073b0 con 0x7f3e840bdc70 2024-01-24T15:13:43.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.603+0000 7f3e8c385700 1 -- 172.21.15.29:0/3110153805 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e840acc40 con 0x7f3e840bdc70 2024-01-24T15:13:43.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.604+0000 7f3e8c385700 1 -- 172.21.15.29:0/3110153805 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e840475e0 con 0x7f3e840bdc70 2024-01-24T15:13:43.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.604+0000 7f3e767fc700 1 -- 172.21.15.29:0/3110153805 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3e800068b0 con 0x7f3e840bdc70 2024-01-24T15:13:43.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.605+0000 7f3e767fc700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3e6805f270 0x7f3e68061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:43.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.605+0000 7f3e767fc700 1 -- 172.21.15.29:0/3110153805 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f3e8007a180 con 0x7f3e840bdc70 2024-01-24T15:13:43.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.605+0000 7f3e89920700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3e6805f270 0x7f3e68061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:43.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.606+0000 7f3e89920700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3e6805f270 0x7f3e68061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f3e840bed70 tx=0x7f3e78008040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:43.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.609+0000 7f3e767fc700 1 -- 172.21.15.29:0/3110153805 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3e8004aea0 con 0x7f3e840bdc70 2024-01-24T15:13:43.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.860+0000 7f3e8c385700 1 -- 172.21.15.29:0/3110153805 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3e84154b40 con 0x7f3e840bdc70 2024-01-24T15:13:43.862 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.861+0000 7f3e767fc700 1 -- 172.21.15.29:0/3110153805 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+955 (secure 0 0 0) 0x7f3e8004e4c0 con 0x7f3e840bdc70 2024-01-24T15:13:43.863 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:43.863 INFO:teuthology.orchestra.run.smithi029.stdout:{"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":4542,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":178,"num_osds":8,"num_up_osds":8,"osd_up_since":1706104690,"num_in_osds":8,"osd_in_since":1706104683,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":137}],"num_pgs":137,"num_pools":4,"num_objects":21,"data_bytes":2306,"bytes_used":2518708224,"bytes_avail":770575405056,"bytes_total":773094113280},"fsmap":{"epoch":74,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":8242}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-01-24T15:03:35.457968+0000","services":{}},"progress_events":{}} 2024-01-24T15:13:43.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.862+0000 7f3e8c385700 1 -- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3e6805f270 msgr2=0x7f3e68061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.862+0000 7f3e8c385700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3e6805f270 0x7f3e68061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f3e840bed70 tx=0x7f3e78008040).stop 2024-01-24T15:13:43.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.862+0000 7f3e8c385700 1 -- 172.21.15.29:0/3110153805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e840bdc70 msgr2=0x7f3e840abd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:43.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.862+0000 7f3e8c385700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e840bdc70 0x7f3e840abd10 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f3e8000c4c0 tx=0x7f3e80007e00).stop 2024-01-24T15:13:43.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.862+0000 7f3e8c385700 1 -- 172.21.15.29:0/3110153805 shutdown_connections 2024-01-24T15:13:43.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.862+0000 7f3e8c385700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e8413a660 0x7f3e84150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.862+0000 7f3e8c385700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3e6805f270 0x7f3e68061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.862+0000 7f3e8c385700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e840bdc70 0x7f3e840abd10 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.862+0000 7f3e8c385700 1 --2- 172.21.15.29:0/3110153805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e8413a0d0 0x7f3e840c17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:43.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.863+0000 7f3e8c385700 1 -- 172.21.15.29:0/3110153805 >> 172.21.15.29:0/3110153805 conn(0x7f3e840c4570 msgr2=0x7f3e8415acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:43.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.863+0000 7f3e8c385700 1 -- 172.21.15.29:0/3110153805 shutdown_connections 2024-01-24T15:13:43.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:43.863+0000 7f3e8c385700 1 -- 172.21.15.29:0/3110153805 wait complete. 2024-01-24T15:13:43.878 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-24T15:13:43.878 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:13:44.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.007+0000 7f98fa9ca700 1 Processor -- start 2024-01-24T15:13:44.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.007+0000 7f98fa9ca700 1 -- start start 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.007+0000 7f98fa9ca700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f4153fe0 0x7f98f41543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98fa9ca700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f98f40bd340 0x7f98f40c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98fa9ca700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f405e850 con 0x7f98f40a4880 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98fa9ca700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f40631f0 con 0x7f98f40bd340 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98fa9ca700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f40bd7e0 con 0x7f98f4153fe0 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98f8f67700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f98f40bd340 0x7f98f40c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98f8f67700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f98f40bd340 0x7f98f40c1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:34048/0 (socket says 172.21.15.29:34048) 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98f3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f4153fe0 0x7f98f41543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98f8f67700 1 -- 172.21.15.29:0/3535899090 learned_addr learned my addr 172.21.15.29:0/3535899090 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98f37fe700 1 --1- 172.21.15.29:0/3535899090 >> v1:172.21.15.163:6789/0 conn(0x7f98f40a4880 0x7f98f40bcc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:48280/0 (socket says 172.21.15.29:48280) 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98f8f67700 1 -- 172.21.15.29:0/3535899090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f4153fe0 msgr2=0x7f98f41543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98f8f67700 1 --2- 172.21.15.29:0/3535899090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f4153fe0 0x7f98f41543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98f8f67700 1 -- 172.21.15.29:0/3535899090 >> v1:172.21.15.163:6789/0 conn(0x7f98f40a4880 legacy=0x7f98f40bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:44.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98f8f67700 1 -- 172.21.15.29:0/3535899090 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40628c0 con 0x7f98f40bd340 2024-01-24T15:13:44.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98f2ffd700 1 -- 172.21.15.29:0/3535899090 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194201854 0 0) 0x7f98f405e850 con 0x7f98f40a4880 2024-01-24T15:13:44.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.008+0000 7f98f3fff700 1 --2- 172.21.15.29:0/3535899090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f4153fe0 0x7f98f41543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:44.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.009+0000 7f98f8f67700 1 --2- 172.21.15.29:0/3535899090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40bd340 0x7f98f40c1870 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f98ec009f50 tx=0x7f98ec009840).ready entity=mon.1 client_cookie=74b864f518f84dca server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:44.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.009+0000 7f98f2ffd700 1 -- 172.21.15.29:0/3535899090 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00e070 con 0x7f98f40bd340 2024-01-24T15:13:44.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.009+0000 7f98f2ffd700 1 -- 172.21.15.29:0/3535899090 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec00b9c0 con 0x7f98f40bd340 2024-01-24T15:13:44.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.009+0000 7f98f2ffd700 1 -- 172.21.15.29:0/3535899090 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00bd40 con 0x7f98f40bd340 2024-01-24T15:13:44.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.009+0000 7f98fa9ca700 1 -- 172.21.15.29:0/3535899090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40bd340 msgr2=0x7f98f40c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:44.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.009+0000 7f98fa9ca700 1 --2- 172.21.15.29:0/3535899090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40bd340 0x7f98f40c1870 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f98ec009f50 tx=0x7f98ec009840).stop 2024-01-24T15:13:44.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.009+0000 7f98fa9ca700 1 -- 172.21.15.29:0/3535899090 shutdown_connections 2024-01-24T15:13:44.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.009+0000 7f98fa9ca700 1 --2- 172.21.15.29:0/3535899090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f4153fe0 0x7f98f41543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:44.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.009+0000 7f98fa9ca700 1 --2- 172.21.15.29:0/3535899090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40bd340 0x7f98f40c1870 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:44.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.009+0000 7f98fa9ca700 1 -- 172.21.15.29:0/3535899090 >> 172.21.15.29:0/3535899090 conn(0x7f98f4064aa0 msgr2=0x7f98f4064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:44.013 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.009+0000 7f98fa9ca700 1 -- 172.21.15.29:0/3535899090 shutdown_connections 2024-01-24T15:13:44.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.009+0000 7f98fa9ca700 1 -- 172.21.15.29:0/3535899090 wait complete. 2024-01-24T15:13:44.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.011+0000 7f98fa9ca700 1 Processor -- start 2024-01-24T15:13:44.014 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.011+0000 7f98fa9ca700 1 -- start start 2024-01-24T15:13:44.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98fa9ca700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40a4880 0x7f98f4154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:44.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98fa9ca700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f40bd340 0x7f98f40689a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:44.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98fa9ca700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98f4153fe0 0x7f98f414a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:44.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98fa9ca700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4069a90 con 0x7f98f4153fe0 2024-01-24T15:13:44.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98fa9ca700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f41564a0 con 0x7f98f40a4880 2024-01-24T15:13:44.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98fa9ca700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f40691f0 con 0x7f98f40bd340 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98f8f67700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98f4153fe0 0x7f98f414a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98f8f67700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98f4153fe0 0x7f98f414a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40532/0 (socket says 172.21.15.29:40532) 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98f8f67700 1 -- 172.21.15.29:0/4041046314 learned_addr learned my addr 172.21.15.29:0/4041046314 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98f37fe700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f40bd340 0x7f98f40689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98f3fff700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40a4880 0x7f98f4154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98f8f67700 1 -- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f40bd340 msgr2=0x7f98f40689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98f8f67700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f40bd340 0x7f98f40689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98f8f67700 1 -- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40a4880 msgr2=0x7f98f4154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98f8f67700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40a4880 0x7f98f4154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98f8f67700 1 -- 172.21.15.29:0/4041046314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e40092a0 con 0x7f98f4153fe0 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.012+0000 7f98f8f67700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98f4153fe0 0x7f98f414a270 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f98f4064330 tx=0x7f98ec009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.013+0000 7f98dbfff700 1 -- 172.21.15.29:0/4041046314 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0092e0 con 0x7f98f4153fe0 2024-01-24T15:13:44.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.013+0000 7f98dbfff700 1 -- 172.21.15.29:0/4041046314 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec01e470 con 0x7f98f4153fe0 2024-01-24T15:13:44.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.013+0000 7f98fa9ca700 1 -- 172.21.15.29:0/4041046314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ec009ae0 con 0x7f98f4153fe0 2024-01-24T15:13:44.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.013+0000 7f98dbfff700 1 -- 172.21.15.29:0/4041046314 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec01e830 con 0x7f98f4153fe0 2024-01-24T15:13:44.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.013+0000 7f98fa9ca700 1 -- 172.21.15.29:0/4041046314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f40697b0 con 0x7f98f4153fe0 2024-01-24T15:13:44.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.014+0000 7f98fa9ca700 1 -- 172.21.15.29:0/4041046314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f40475e0 con 0x7f98f4153fe0 2024-01-24T15:13:44.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.014+0000 7f98dbfff700 1 -- 172.21.15.29:0/4041046314 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f98ec003040 con 0x7f98f4153fe0 2024-01-24T15:13:44.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.014+0000 7f98dbfff700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f98e005f240 0x7f98e0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:44.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.015+0000 7f98dbfff700 1 -- 172.21.15.29:0/4041046314 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f98ec07bd20 con 0x7f98f4153fe0 2024-01-24T15:13:44.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.015+0000 7f98f3fff700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f98e005f240 0x7f98e0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:44.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.015+0000 7f98f3fff700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f98e005f240 0x7f98e0061700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f98e4009dc0 tx=0x7f98e4009060).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:44.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.019+0000 7f98dbfff700 1 -- 172.21.15.29:0/4041046314 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f98ec04ca30 con 0x7f98f4153fe0 2024-01-24T15:13:44.060 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-01-24T15:13:44.059+0000 7f9c98bd0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T15:13:44.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.199+0000 7f98fa9ca700 1 -- 172.21.15.29:0/4041046314 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98f405e330 con 0x7f98e005f240 2024-01-24T15:13:44.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.209+0000 7f98dbfff700 1 -- 172.21.15.29:0/4041046314 <== mgr.8056 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361966 (secure 0 0 0) 0x7f98f405e330 con 0x7f98e005f240 2024-01-24T15:13:44.216 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.214+0000 7f98fa9ca700 1 -- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f98e005f240 msgr2=0x7f98e0061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.214+0000 7f98fa9ca700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f98e005f240 0x7f98e0061700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f98e4009dc0 tx=0x7f98e4009060).stop 2024-01-24T15:13:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.214+0000 7f98fa9ca700 1 -- 172.21.15.29:0/4041046314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98f4153fe0 msgr2=0x7f98f414a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.214+0000 7f98fa9ca700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98f4153fe0 0x7f98f414a270 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f98f4064330 tx=0x7f98ec009840).stop 2024-01-24T15:13:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.214+0000 7f98fa9ca700 1 -- 172.21.15.29:0/4041046314 shutdown_connections 2024-01-24T15:13:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.214+0000 7f98fa9ca700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f98e005f240 0x7f98e0061700 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.214+0000 7f98fa9ca700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f98f4153fe0 0x7f98f414a270 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.214+0000 7f98fa9ca700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40a4880 0x7f98f4154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.214+0000 7f98fa9ca700 1 --2- 172.21.15.29:0/4041046314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f40bd340 0x7f98f40689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.214+0000 7f98fa9ca700 1 -- 172.21.15.29:0/4041046314 >> 172.21.15.29:0/4041046314 conn(0x7f98f4064aa0 msgr2=0x7f98f40a5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.214+0000 7f98fa9ca700 1 -- 172.21.15.29:0/4041046314 shutdown_connections 2024-01-24T15:13:44.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:44.214+0000 7f98fa9ca700 1 -- 172.21.15.29:0/4041046314 wait complete. 2024-01-24T15:13:44.220 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:13:44.229 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-01-24T15:13:43.038672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":222,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2459676,"kb_used_data":17372,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752515044,"statfs":{"total":773094113280,"available":770575405056,"internally_reserved":0,"allocated":17788928,"data_stored":10170427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5036847,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-498,"num_write_kb":-5303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.998251"},"pg_stats":[{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550222+0000","last_change":"2024-01-24T15:13:38.543675+0000","last_active":"2024-01-24T15:13:38.550222+0000","last_peered":"2024-01-24T15:13:38.550222+0000","last_clean":"2024-01-24T15:13:38.550222+0000","last_became_active":"2024-01-24T15:13:38.543012+0000","last_became_peered":"2024-01-24T15:13:38.543012+0000","last_unstale":"2024-01-24T15:13:38.550222+0000","last_undegraded":"2024-01-24T15:13:38.550222+0000","last_fullsized":"2024-01-24T15:13:38.550222+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3f","version":"177'3","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545770+0000","last_change":"2024-01-24T15:13:38.128076+0000","last_active":"2024-01-24T15:13:38.545770+0000","last_peered":"2024-01-24T15:13:38.545770+0000","last_clean":"2024-01-24T15:13:38.545770+0000","last_became_active":"2024-01-24T15:13:38.127791+0000","last_became_peered":"2024-01-24T15:13:38.127791+0000","last_unstale":"2024-01-24T15:13:38.545770+0000","last_undegraded":"2024-01-24T15:13:38.545770+0000","last_fullsized":"2024-01-24T15:13:38.545770+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557217+0000","last_change":"2024-01-24T15:13:38.554545+0000","last_active":"2024-01-24T15:13:38.557217+0000","last_peered":"2024-01-24T15:13:38.557217+0000","last_clean":"2024-01-24T15:13:38.557217+0000","last_became_active":"2024-01-24T15:13:38.554226+0000","last_became_peered":"2024-01-24T15:13:38.554226+0000","last_unstale":"2024-01-24T15:13:38.557217+0000","last_undegraded":"2024-01-24T15:13:38.557217+0000","last_fullsized":"2024-01-24T15:13:38.557217+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533665+0000","last_change":"2024-01-24T15:13:38.118261+0000","last_active":"2024-01-24T15:13:38.533665+0000","last_peered":"2024-01-24T15:13:38.533665+0000","last_clean":"2024-01-24T15:13:38.533665+0000","last_became_active":"2024-01-24T15:13:38.117923+0000","last_became_peered":"2024-01-24T15:13:38.117923+0000","last_unstale":"2024-01-24T15:13:38.533665+0000","last_undegraded":"2024-01-24T15:13:38.533665+0000","last_fullsized":"2024-01-24T15:13:38.533665+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562338+0000","last_change":"2024-01-24T15:13:38.559794+0000","last_active":"2024-01-24T15:13:38.562338+0000","last_peered":"2024-01-24T15:13:38.562338+0000","last_clean":"2024-01-24T15:13:38.562338+0000","last_became_active":"2024-01-24T15:13:38.559480+0000","last_became_peered":"2024-01-24T15:13:38.559480+0000","last_unstale":"2024-01-24T15:13:38.562338+0000","last_undegraded":"2024-01-24T15:13:38.562338+0000","last_fullsized":"2024-01-24T15:13:38.562338+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537224+0000","last_change":"2024-01-24T15:13:38.132008+0000","last_active":"2024-01-24T15:13:38.537224+0000","last_peered":"2024-01-24T15:13:38.537224+0000","last_clean":"2024-01-24T15:13:38.537224+0000","last_became_active":"2024-01-24T15:13:38.131711+0000","last_became_peered":"2024-01-24T15:13:38.131711+0000","last_unstale":"2024-01-24T15:13:38.537224+0000","last_undegraded":"2024-01-24T15:13:38.537224+0000","last_fullsized":"2024-01-24T15:13:38.537224+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558646+0000","last_change":"2024-01-24T15:13:38.548443+0000","last_active":"2024-01-24T15:13:38.558646+0000","last_peered":"2024-01-24T15:13:38.558646+0000","last_clean":"2024-01-24T15:13:38.558646+0000","last_became_active":"2024-01-24T15:13:38.546973+0000","last_became_peered":"2024-01-24T15:13:38.546973+0000","last_unstale":"2024-01-24T15:13:38.558646+0000","last_undegraded":"2024-01-24T15:13:38.558646+0000","last_fullsized":"2024-01-24T15:13:38.558646+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558098+0000","last_change":"2024-01-24T15:13:38.129340+0000","last_active":"2024-01-24T15:13:38.558098+0000","last_peered":"2024-01-24T15:13:38.558098+0000","last_clean":"2024-01-24T15:13:38.558098+0000","last_became_active":"2024-01-24T15:13:38.129057+0000","last_became_peered":"2024-01-24T15:13:38.129057+0000","last_unstale":"2024-01-24T15:13:38.558098+0000","last_undegraded":"2024-01-24T15:13:38.558098+0000","last_fullsized":"2024-01-24T15:13:38.558098+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.555919+0000","last_change":"2024-01-24T15:13:38.551741+0000","last_active":"2024-01-24T15:13:38.555919+0000","last_peered":"2024-01-24T15:13:38.555919+0000","last_clean":"2024-01-24T15:13:38.555919+0000","last_became_active":"2024-01-24T15:13:38.551165+0000","last_became_peered":"2024-01-24T15:13:38.551165+0000","last_unstale":"2024-01-24T15:13:38.555919+0000","last_undegraded":"2024-01-24T15:13:38.555919+0000","last_fullsized":"2024-01-24T15:13:38.555919+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557709+0000","last_change":"2024-01-24T15:13:38.122839+0000","last_active":"2024-01-24T15:13:38.557709+0000","last_peered":"2024-01-24T15:13:38.557709+0000","last_clean":"2024-01-24T15:13:38.557709+0000","last_became_active":"2024-01-24T15:13:38.121646+0000","last_became_peered":"2024-01-24T15:13:38.121646+0000","last_unstale":"2024-01-24T15:13:38.557709+0000","last_undegraded":"2024-01-24T15:13:38.557709+0000","last_fullsized":"2024-01-24T15:13:38.557709+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551349+0000","last_change":"2024-01-24T15:13:38.548421+0000","last_active":"2024-01-24T15:13:38.551349+0000","last_peered":"2024-01-24T15:13:38.551349+0000","last_clean":"2024-01-24T15:13:38.551349+0000","last_became_active":"2024-01-24T15:13:38.548168+0000","last_became_peered":"2024-01-24T15:13:38.548168+0000","last_unstale":"2024-01-24T15:13:38.551349+0000","last_undegraded":"2024-01-24T15:13:38.551349+0000","last_fullsized":"2024-01-24T15:13:38.551349+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3a","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550926+0000","last_change":"2024-01-24T15:13:38.131347+0000","last_active":"2024-01-24T15:13:38.550926+0000","last_peered":"2024-01-24T15:13:38.550926+0000","last_clean":"2024-01-24T15:13:38.550926+0000","last_became_active":"2024-01-24T15:13:38.127049+0000","last_became_peered":"2024-01-24T15:13:38.127049+0000","last_unstale":"2024-01-24T15:13:38.550926+0000","last_undegraded":"2024-01-24T15:13:38.550926+0000","last_fullsized":"2024-01-24T15:13:38.550926+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557161+0000","last_change":"2024-01-24T15:13:38.553689+0000","last_active":"2024-01-24T15:13:38.557161+0000","last_peered":"2024-01-24T15:13:38.557161+0000","last_clean":"2024-01-24T15:13:38.557161+0000","last_became_active":"2024-01-24T15:13:38.553388+0000","last_became_peered":"2024-01-24T15:13:38.553388+0000","last_unstale":"2024-01-24T15:13:38.557161+0000","last_undegraded":"2024-01-24T15:13:38.557161+0000","last_fullsized":"2024-01-24T15:13:38.557161+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.540830+0000","last_change":"2024-01-24T15:13:38.133854+0000","last_active":"2024-01-24T15:13:38.540830+0000","last_peered":"2024-01-24T15:13:38.540830+0000","last_clean":"2024-01-24T15:13:38.540830+0000","last_became_active":"2024-01-24T15:13:38.133583+0000","last_became_peered":"2024-01-24T15:13:38.133583+0000","last_unstale":"2024-01-24T15:13:38.540830+0000","last_undegraded":"2024-01-24T15:13:38.540830+0000","last_fullsized":"2024-01-24T15:13:38.540830+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559220+0000","last_change":"2024-01-24T15:13:38.556825+0000","last_active":"2024-01-24T15:13:38.559220+0000","last_peered":"2024-01-24T15:13:38.559220+0000","last_clean":"2024-01-24T15:13:38.559220+0000","last_became_active":"2024-01-24T15:13:38.556530+0000","last_became_peered":"2024-01-24T15:13:38.556530+0000","last_unstale":"2024-01-24T15:13:38.559220+0000","last_undegraded":"2024-01-24T15:13:38.559220+0000","last_fullsized":"2024-01-24T15:13:38.559220+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.544558+0000","last_change":"2024-01-24T15:13:38.129183+0000","last_active":"2024-01-24T15:13:38.544558+0000","last_peered":"2024-01-24T15:13:38.544558+0000","last_clean":"2024-01-24T15:13:38.544558+0000","last_became_active":"2024-01-24T15:13:38.128889+0000","last_became_peered":"2024-01-24T15:13:38.128889+0000","last_unstale":"2024-01-24T15:13:38.544558+0000","last_undegraded":"2024-01-24T15:13:38.544558+0000","last_fullsized":"2024-01-24T15:13:38.544558+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558035+0000","last_change":"2024-01-24T15:13:38.552886+0000","last_active":"2024-01-24T15:13:38.558035+0000","last_peered":"2024-01-24T15:13:38.558035+0000","last_clean":"2024-01-24T15:13:38.558035+0000","last_became_active":"2024-01-24T15:13:38.552602+0000","last_became_peered":"2024-01-24T15:13:38.552602+0000","last_unstale":"2024-01-24T15:13:38.558035+0000","last_undegraded":"2024-01-24T15:13:38.558035+0000","last_fullsized":"2024-01-24T15:13:38.558035+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537881+0000","last_change":"2024-01-24T15:13:38.122116+0000","last_active":"2024-01-24T15:13:38.537881+0000","last_peered":"2024-01-24T15:13:38.537881+0000","last_clean":"2024-01-24T15:13:38.537881+0000","last_became_active":"2024-01-24T15:13:38.121183+0000","last_became_peered":"2024-01-24T15:13:38.121183+0000","last_unstale":"2024-01-24T15:13:38.537881+0000","last_undegraded":"2024-01-24T15:13:38.537881+0000","last_fullsized":"2024-01-24T15:13:38.537881+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557777+0000","last_change":"2024-01-24T15:13:38.555214+0000","last_active":"2024-01-24T15:13:38.557777+0000","last_peered":"2024-01-24T15:13:38.557777+0000","last_clean":"2024-01-24T15:13:38.557777+0000","last_became_active":"2024-01-24T15:13:38.554915+0000","last_became_peered":"2024-01-24T15:13:38.554915+0000","last_unstale":"2024-01-24T15:13:38.557777+0000","last_undegraded":"2024-01-24T15:13:38.557777+0000","last_fullsized":"2024-01-24T15:13:38.557777+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.539484+0000","last_change":"2024-01-24T15:13:38.127556+0000","last_active":"2024-01-24T15:13:38.539484+0000","last_peered":"2024-01-24T15:13:38.539484+0000","last_clean":"2024-01-24T15:13:38.539484+0000","last_became_active":"2024-01-24T15:13:38.127215+0000","last_became_peered":"2024-01-24T15:13:38.127215+0000","last_unstale":"2024-01-24T15:13:38.539484+0000","last_undegraded":"2024-01-24T15:13:38.539484+0000","last_fullsized":"2024-01-24T15:13:38.539484+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557185+0000","last_change":"2024-01-24T15:13:38.551757+0000","last_active":"2024-01-24T15:13:38.557185+0000","last_peered":"2024-01-24T15:13:38.557185+0000","last_clean":"2024-01-24T15:13:38.557185+0000","last_became_active":"2024-01-24T15:13:38.551453+0000","last_became_peered":"2024-01-24T15:13:38.551453+0000","last_unstale":"2024-01-24T15:13:38.557185+0000","last_undegraded":"2024-01-24T15:13:38.557185+0000","last_fullsized":"2024-01-24T15:13:38.557185+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533408+0000","last_change":"2024-01-24T15:13:38.121049+0000","last_active":"2024-01-24T15:13:38.533408+0000","last_peered":"2024-01-24T15:13:38.533408+0000","last_clean":"2024-01-24T15:13:38.533408+0000","last_became_active":"2024-01-24T15:13:38.120804+0000","last_became_peered":"2024-01-24T15:13:38.120804+0000","last_unstale":"2024-01-24T15:13:38.533408+0000","last_undegraded":"2024-01-24T15:13:38.533408+0000","last_fullsized":"2024-01-24T15:13:38.533408+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551655+0000","last_change":"2024-01-24T15:13:38.547007+0000","last_active":"2024-01-24T15:13:38.551655+0000","last_peered":"2024-01-24T15:13:38.551655+0000","last_clean":"2024-01-24T15:13:38.551655+0000","last_became_active":"2024-01-24T15:13:38.546766+0000","last_became_peered":"2024-01-24T15:13:38.546766+0000","last_unstale":"2024-01-24T15:13:38.551655+0000","last_undegraded":"2024-01-24T15:13:38.551655+0000","last_fullsized":"2024-01-24T15:13:38.551655+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.34","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.565384+0000","last_change":"2024-01-24T15:13:38.123112+0000","last_active":"2024-01-24T15:13:38.565384+0000","last_peered":"2024-01-24T15:13:38.565384+0000","last_clean":"2024-01-24T15:13:38.565384+0000","last_became_active":"2024-01-24T15:13:38.122820+0000","last_became_peered":"2024-01-24T15:13:38.122820+0000","last_unstale":"2024-01-24T15:13:38.565384+0000","last_undegraded":"2024-01-24T15:13:38.565384+0000","last_fullsized":"2024-01-24T15:13:38.565384+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556831+0000","last_change":"2024-01-24T15:13:38.552223+0000","last_active":"2024-01-24T15:13:38.556831+0000","last_peered":"2024-01-24T15:13:38.556831+0000","last_clean":"2024-01-24T15:13:38.556831+0000","last_became_active":"2024-01-24T15:13:38.551704+0000","last_became_peered":"2024-01-24T15:13:38.551704+0000","last_unstale":"2024-01-24T15:13:38.556831+0000","last_undegraded":"2024-01-24T15:13:38.556831+0000","last_fullsized":"2024-01-24T15:13:38.556831+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.33","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.552767+0000","last_change":"2024-01-24T15:13:38.120932+0000","last_active":"2024-01-24T15:13:38.552767+0000","last_peered":"2024-01-24T15:13:38.552767+0000","last_clean":"2024-01-24T15:13:38.552767+0000","last_became_active":"2024-01-24T15:13:38.120622+0000","last_became_peered":"2024-01-24T15:13:38.120622+0000","last_unstale":"2024-01-24T15:13:38.552767+0000","last_undegraded":"2024-01-24T15:13:38.552767+0000","last_fullsized":"2024-01-24T15:13:38.552767+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551328+0000","last_change":"2024-01-24T15:13:38.547976+0000","last_active":"2024-01-24T15:13:38.551328+0000","last_peered":"2024-01-24T15:13:38.551328+0000","last_clean":"2024-01-24T15:13:38.551328+0000","last_became_active":"2024-01-24T15:13:38.547686+0000","last_became_peered":"2024-01-24T15:13:38.547686+0000","last_unstale":"2024-01-24T15:13:38.551328+0000","last_undegraded":"2024-01-24T15:13:38.551328+0000","last_fullsized":"2024-01-24T15:13:38.551328+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528656+0000","last_change":"2024-01-24T15:13:38.125651+0000","last_active":"2024-01-24T15:13:38.528656+0000","last_peered":"2024-01-24T15:13:38.528656+0000","last_clean":"2024-01-24T15:13:38.528656+0000","last_became_active":"2024-01-24T15:13:38.125334+0000","last_became_peered":"2024-01-24T15:13:38.125334+0000","last_unstale":"2024-01-24T15:13:38.528656+0000","last_undegraded":"2024-01-24T15:13:38.528656+0000","last_fullsized":"2024-01-24T15:13:38.528656+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545374+0000","last_change":"2024-01-24T15:13:38.543804+0000","last_active":"2024-01-24T15:13:38.545374+0000","last_peered":"2024-01-24T15:13:38.545374+0000","last_clean":"2024-01-24T15:13:38.545374+0000","last_became_active":"2024-01-24T15:13:38.543259+0000","last_became_peered":"2024-01-24T15:13:38.543259+0000","last_unstale":"2024-01-24T15:13:38.545374+0000","last_undegraded":"2024-01-24T15:13:38.545374+0000","last_fullsized":"2024-01-24T15:13:38.545374+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528152+0000","last_change":"2024-01-24T15:13:38.125234+0000","last_active":"2024-01-24T15:13:38.528152+0000","last_peered":"2024-01-24T15:13:38.528152+0000","last_clean":"2024-01-24T15:13:38.528152+0000","last_became_active":"2024-01-24T15:13:38.124520+0000","last_became_peered":"2024-01-24T15:13:38.124520+0000","last_unstale":"2024-01-24T15:13:38.528152+0000","last_undegraded":"2024-01-24T15:13:38.528152+0000","last_fullsized":"2024-01-24T15:13:38.528152+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557010+0000","last_change":"2024-01-24T15:13:38.548644+0000","last_active":"2024-01-24T15:13:38.557010+0000","last_peered":"2024-01-24T15:13:38.557010+0000","last_clean":"2024-01-24T15:13:38.557010+0000","last_became_active":"2024-01-24T15:13:38.548355+0000","last_became_peered":"2024-01-24T15:13:38.548355+0000","last_unstale":"2024-01-24T15:13:38.557010+0000","last_undegraded":"2024-01-24T15:13:38.557010+0000","last_fullsized":"2024-01-24T15:13:38.557010+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.30","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.552473+0000","last_change":"2024-01-24T15:13:38.123443+0000","last_active":"2024-01-24T15:13:38.552473+0000","last_peered":"2024-01-24T15:13:38.552473+0000","last_clean":"2024-01-24T15:13:38.552473+0000","last_became_active":"2024-01-24T15:13:38.123066+0000","last_became_peered":"2024-01-24T15:13:38.123066+0000","last_unstale":"2024-01-24T15:13:38.552473+0000","last_undegraded":"2024-01-24T15:13:38.552473+0000","last_fullsized":"2024-01-24T15:13:38.552473+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557443+0000","last_change":"2024-01-24T15:13:38.546142+0000","last_active":"2024-01-24T15:13:38.557443+0000","last_peered":"2024-01-24T15:13:38.557443+0000","last_clean":"2024-01-24T15:13:38.557443+0000","last_became_active":"2024-01-24T15:13:38.545868+0000","last_became_peered":"2024-01-24T15:13:38.545868+0000","last_unstale":"2024-01-24T15:13:38.557443+0000","last_undegraded":"2024-01-24T15:13:38.557443+0000","last_fullsized":"2024-01-24T15:13:38.557443+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537981+0000","last_change":"2024-01-24T15:13:38.121742+0000","last_active":"2024-01-24T15:13:38.537981+0000","last_peered":"2024-01-24T15:13:38.537981+0000","last_clean":"2024-01-24T15:13:38.537981+0000","last_became_active":"2024-01-24T15:13:38.121386+0000","last_became_peered":"2024-01-24T15:13:38.121386+0000","last_unstale":"2024-01-24T15:13:38.537981+0000","last_undegraded":"2024-01-24T15:13:38.537981+0000","last_fullsized":"2024-01-24T15:13:38.537981+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562294+0000","last_change":"2024-01-24T15:13:38.558575+0000","last_active":"2024-01-24T15:13:38.562294+0000","last_peered":"2024-01-24T15:13:38.562294+0000","last_clean":"2024-01-24T15:13:38.562294+0000","last_became_active":"2024-01-24T15:13:38.558314+0000","last_became_peered":"2024-01-24T15:13:38.558314+0000","last_unstale":"2024-01-24T15:13:38.562294+0000","last_undegraded":"2024-01-24T15:13:38.562294+0000","last_fullsized":"2024-01-24T15:13:38.562294+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532043+0000","last_change":"2024-01-24T15:13:38.122045+0000","last_active":"2024-01-24T15:13:38.532043+0000","last_peered":"2024-01-24T15:13:38.532043+0000","last_clean":"2024-01-24T15:13:38.532043+0000","last_became_active":"2024-01-24T15:13:38.121766+0000","last_became_peered":"2024-01-24T15:13:38.121766+0000","last_unstale":"2024-01-24T15:13:38.532043+0000","last_undegraded":"2024-01-24T15:13:38.532043+0000","last_fullsized":"2024-01-24T15:13:38.532043+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545141+0000","last_change":"2024-01-24T15:13:38.541723+0000","last_active":"2024-01-24T15:13:38.545141+0000","last_peered":"2024-01-24T15:13:38.545141+0000","last_clean":"2024-01-24T15:13:38.545141+0000","last_became_active":"2024-01-24T15:13:38.540083+0000","last_became_peered":"2024-01-24T15:13:38.540083+0000","last_unstale":"2024-01-24T15:13:38.545141+0000","last_undegraded":"2024-01-24T15:13:38.545141+0000","last_fullsized":"2024-01-24T15:13:38.545141+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2d","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558223+0000","last_change":"2024-01-24T15:13:38.130633+0000","last_active":"2024-01-24T15:13:38.558223+0000","last_peered":"2024-01-24T15:13:38.558223+0000","last_clean":"2024-01-24T15:13:38.558223+0000","last_became_active":"2024-01-24T15:13:38.130234+0000","last_became_peered":"2024-01-24T15:13:38.130234+0000","last_unstale":"2024-01-24T15:13:38.558223+0000","last_undegraded":"2024-01-24T15:13:38.558223+0000","last_fullsized":"2024-01-24T15:13:38.558223+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.555856+0000","last_change":"2024-01-24T15:13:38.551556+0000","last_active":"2024-01-24T15:13:38.555856+0000","last_peered":"2024-01-24T15:13:38.555856+0000","last_clean":"2024-01-24T15:13:38.555856+0000","last_became_active":"2024-01-24T15:13:38.551182+0000","last_became_peered":"2024-01-24T15:13:38.551182+0000","last_unstale":"2024-01-24T15:13:38.555856+0000","last_undegraded":"2024-01-24T15:13:38.555856+0000","last_fullsized":"2024-01-24T15:13:38.555856+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532179+0000","last_change":"2024-01-24T15:13:38.128558+0000","last_active":"2024-01-24T15:13:38.532179+0000","last_peered":"2024-01-24T15:13:38.532179+0000","last_clean":"2024-01-24T15:13:38.532179+0000","last_became_active":"2024-01-24T15:13:38.128075+0000","last_became_peered":"2024-01-24T15:13:38.128075+0000","last_unstale":"2024-01-24T15:13:38.532179+0000","last_undegraded":"2024-01-24T15:13:38.532179+0000","last_fullsized":"2024-01-24T15:13:38.532179+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556852+0000","last_change":"2024-01-24T15:13:38.552974+0000","last_active":"2024-01-24T15:13:38.556852+0000","last_peered":"2024-01-24T15:13:38.556852+0000","last_clean":"2024-01-24T15:13:38.556852+0000","last_became_active":"2024-01-24T15:13:38.552706+0000","last_became_peered":"2024-01-24T15:13:38.552706+0000","last_unstale":"2024-01-24T15:13:38.556852+0000","last_undegraded":"2024-01-24T15:13:38.556852+0000","last_fullsized":"2024-01-24T15:13:38.556852+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.553360+0000","last_change":"2024-01-24T15:13:38.127821+0000","last_active":"2024-01-24T15:13:38.553360+0000","last_peered":"2024-01-24T15:13:38.553360+0000","last_clean":"2024-01-24T15:13:38.553360+0000","last_became_active":"2024-01-24T15:13:38.127478+0000","last_became_peered":"2024-01-24T15:13:38.127478+0000","last_unstale":"2024-01-24T15:13:38.553360+0000","last_undegraded":"2024-01-24T15:13:38.553360+0000","last_fullsized":"2024-01-24T15:13:38.553360+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558279+0000","last_change":"2024-01-24T15:13:38.555643+0000","last_active":"2024-01-24T15:13:38.558279+0000","last_peered":"2024-01-24T15:13:38.558279+0000","last_clean":"2024-01-24T15:13:38.558279+0000","last_became_active":"2024-01-24T15:13:38.555338+0000","last_became_peered":"2024-01-24T15:13:38.555338+0000","last_unstale":"2024-01-24T15:13:38.558279+0000","last_undegraded":"2024-01-24T15:13:38.558279+0000","last_fullsized":"2024-01-24T15:13:38.558279+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528767+0000","last_change":"2024-01-24T15:13:38.119386+0000","last_active":"2024-01-24T15:13:38.528767+0000","last_peered":"2024-01-24T15:13:38.528767+0000","last_clean":"2024-01-24T15:13:38.528767+0000","last_became_active":"2024-01-24T15:13:38.118881+0000","last_became_peered":"2024-01-24T15:13:38.118881+0000","last_unstale":"2024-01-24T15:13:38.528767+0000","last_undegraded":"2024-01-24T15:13:38.528767+0000","last_fullsized":"2024-01-24T15:13:38.528767+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557245+0000","last_change":"2024-01-24T15:13:38.552282+0000","last_active":"2024-01-24T15:13:38.557245+0000","last_peered":"2024-01-24T15:13:38.557245+0000","last_clean":"2024-01-24T15:13:38.557245+0000","last_became_active":"2024-01-24T15:13:38.552007+0000","last_became_peered":"2024-01-24T15:13:38.552007+0000","last_unstale":"2024-01-24T15:13:38.557245+0000","last_undegraded":"2024-01-24T15:13:38.557245+0000","last_fullsized":"2024-01-24T15:13:38.557245+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533440+0000","last_change":"2024-01-24T15:13:38.126343+0000","last_active":"2024-01-24T15:13:38.533440+0000","last_peered":"2024-01-24T15:13:38.533440+0000","last_clean":"2024-01-24T15:13:38.533440+0000","last_became_active":"2024-01-24T15:13:38.126099+0000","last_became_peered":"2024-01-24T15:13:38.126099+0000","last_unstale":"2024-01-24T15:13:38.533440+0000","last_undegraded":"2024-01-24T15:13:38.533440+0000","last_fullsized":"2024-01-24T15:13:38.533440+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562224+0000","last_change":"2024-01-24T15:13:38.557923+0000","last_active":"2024-01-24T15:13:38.562224+0000","last_peered":"2024-01-24T15:13:38.562224+0000","last_clean":"2024-01-24T15:13:38.562224+0000","last_became_active":"2024-01-24T15:13:38.557635+0000","last_became_peered":"2024-01-24T15:13:38.557635+0000","last_unstale":"2024-01-24T15:13:38.562224+0000","last_undegraded":"2024-01-24T15:13:38.562224+0000","last_fullsized":"2024-01-24T15:13:38.562224+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535538+0000","last_change":"2024-01-24T15:13:38.117376+0000","last_active":"2024-01-24T15:13:38.535538+0000","last_peered":"2024-01-24T15:13:38.535538+0000","last_clean":"2024-01-24T15:13:38.535538+0000","last_became_active":"2024-01-24T15:13:38.117052+0000","last_became_peered":"2024-01-24T15:13:38.117052+0000","last_unstale":"2024-01-24T15:13:38.535538+0000","last_undegraded":"2024-01-24T15:13:38.535538+0000","last_fullsized":"2024-01-24T15:13:38.535538+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.554021+0000","last_change":"2024-01-24T15:13:38.546194+0000","last_active":"2024-01-24T15:13:38.554021+0000","last_peered":"2024-01-24T15:13:38.554021+0000","last_clean":"2024-01-24T15:13:38.554021+0000","last_became_active":"2024-01-24T15:13:38.545907+0000","last_became_peered":"2024-01-24T15:13:38.545907+0000","last_unstale":"2024-01-24T15:13:38.554021+0000","last_undegraded":"2024-01-24T15:13:38.554021+0000","last_fullsized":"2024-01-24T15:13:38.554021+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537470+0000","last_change":"2024-01-24T15:13:38.120448+0000","last_active":"2024-01-24T15:13:38.537470+0000","last_peered":"2024-01-24T15:13:38.537470+0000","last_clean":"2024-01-24T15:13:38.537470+0000","last_became_active":"2024-01-24T15:13:38.120165+0000","last_became_peered":"2024-01-24T15:13:38.120165+0000","last_unstale":"2024-01-24T15:13:38.537470+0000","last_undegraded":"2024-01-24T15:13:38.537470+0000","last_fullsized":"2024-01-24T15:13:38.537470+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558719+0000","last_change":"2024-01-24T15:13:38.555417+0000","last_active":"2024-01-24T15:13:38.558719+0000","last_peered":"2024-01-24T15:13:38.558719+0000","last_clean":"2024-01-24T15:13:38.558719+0000","last_became_active":"2024-01-24T15:13:38.555118+0000","last_became_peered":"2024-01-24T15:13:38.555118+0000","last_unstale":"2024-01-24T15:13:38.558719+0000","last_undegraded":"2024-01-24T15:13:38.558719+0000","last_fullsized":"2024-01-24T15:13:38.558719+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537325+0000","last_change":"2024-01-24T15:13:38.128489+0000","last_active":"2024-01-24T15:13:38.537325+0000","last_peered":"2024-01-24T15:13:38.537325+0000","last_clean":"2024-01-24T15:13:38.537325+0000","last_became_active":"2024-01-24T15:13:38.128202+0000","last_became_peered":"2024-01-24T15:13:38.128202+0000","last_unstale":"2024-01-24T15:13:38.537325+0000","last_undegraded":"2024-01-24T15:13:38.537325+0000","last_fullsized":"2024-01-24T15:13:38.537325+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562480+0000","last_change":"2024-01-24T15:13:38.561900+0000","last_active":"2024-01-24T15:13:38.562480+0000","last_peered":"2024-01-24T15:13:38.562480+0000","last_clean":"2024-01-24T15:13:38.562480+0000","last_became_active":"2024-01-24T15:13:38.561655+0000","last_became_peered":"2024-01-24T15:13:38.561655+0000","last_unstale":"2024-01-24T15:13:38.562480+0000","last_undegraded":"2024-01-24T15:13:38.562480+0000","last_fullsized":"2024-01-24T15:13:38.562480+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542822+0000","last_change":"2024-01-24T15:13:38.130503+0000","last_active":"2024-01-24T15:13:38.542822+0000","last_peered":"2024-01-24T15:13:38.542822+0000","last_clean":"2024-01-24T15:13:38.542822+0000","last_became_active":"2024-01-24T15:13:38.129933+0000","last_became_peered":"2024-01-24T15:13:38.129933+0000","last_unstale":"2024-01-24T15:13:38.542822+0000","last_undegraded":"2024-01-24T15:13:38.542822+0000","last_fullsized":"2024-01-24T15:13:38.542822+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.548146+0000","last_change":"2024-01-24T15:13:38.546025+0000","last_active":"2024-01-24T15:13:38.548146+0000","last_peered":"2024-01-24T15:13:38.548146+0000","last_clean":"2024-01-24T15:13:38.548146+0000","last_became_active":"2024-01-24T15:13:38.545702+0000","last_became_peered":"2024-01-24T15:13:38.545702+0000","last_unstale":"2024-01-24T15:13:38.548146+0000","last_undegraded":"2024-01-24T15:13:38.548146+0000","last_fullsized":"2024-01-24T15:13:38.548146+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.546793+0000","last_change":"2024-01-24T15:13:38.128387+0000","last_active":"2024-01-24T15:13:38.546793+0000","last_peered":"2024-01-24T15:13:38.546793+0000","last_clean":"2024-01-24T15:13:38.546793+0000","last_became_active":"2024-01-24T15:13:38.128088+0000","last_became_peered":"2024-01-24T15:13:38.128088+0000","last_unstale":"2024-01-24T15:13:38.546793+0000","last_undegraded":"2024-01-24T15:13:38.546793+0000","last_fullsized":"2024-01-24T15:13:38.546793+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.560725+0000","last_change":"2024-01-24T15:13:38.554569+0000","last_active":"2024-01-24T15:13:38.560725+0000","last_peered":"2024-01-24T15:13:38.560725+0000","last_clean":"2024-01-24T15:13:38.560725+0000","last_became_active":"2024-01-24T15:13:38.554110+0000","last_became_peered":"2024-01-24T15:13:38.554110+0000","last_unstale":"2024-01-24T15:13:38.560725+0000","last_undegraded":"2024-01-24T15:13:38.560725+0000","last_fullsized":"2024-01-24T15:13:38.560725+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535857+0000","last_change":"2024-01-24T15:13:38.127508+0000","last_active":"2024-01-24T15:13:38.535857+0000","last_peered":"2024-01-24T15:13:38.535857+0000","last_clean":"2024-01-24T15:13:38.535857+0000","last_became_active":"2024-01-24T15:13:38.127242+0000","last_became_peered":"2024-01-24T15:13:38.127242+0000","last_unstale":"2024-01-24T15:13:38.535857+0000","last_undegraded":"2024-01-24T15:13:38.535857+0000","last_fullsized":"2024-01-24T15:13:38.535857+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557111+0000","last_change":"2024-01-24T15:13:38.552603+0000","last_active":"2024-01-24T15:13:38.557111+0000","last_peered":"2024-01-24T15:13:38.557111+0000","last_clean":"2024-01-24T15:13:38.557111+0000","last_became_active":"2024-01-24T15:13:38.550713+0000","last_became_peered":"2024-01-24T15:13:38.550713+0000","last_unstale":"2024-01-24T15:13:38.557111+0000","last_undegraded":"2024-01-24T15:13:38.557111+0000","last_fullsized":"2024-01-24T15:13:38.557111+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535420+0000","last_change":"2024-01-24T15:13:38.121734+0000","last_active":"2024-01-24T15:13:38.535420+0000","last_peered":"2024-01-24T15:13:38.535420+0000","last_clean":"2024-01-24T15:13:38.535420+0000","last_became_active":"2024-01-24T15:13:38.121067+0000","last_became_peered":"2024-01-24T15:13:38.121067+0000","last_unstale":"2024-01-24T15:13:38.535420+0000","last_undegraded":"2024-01-24T15:13:38.535420+0000","last_fullsized":"2024-01-24T15:13:38.535420+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545355+0000","last_change":"2024-01-24T15:13:38.543615+0000","last_active":"2024-01-24T15:13:38.545355+0000","last_peered":"2024-01-24T15:13:38.545355+0000","last_clean":"2024-01-24T15:13:38.545355+0000","last_became_active":"2024-01-24T15:13:38.541511+0000","last_became_peered":"2024-01-24T15:13:38.541511+0000","last_unstale":"2024-01-24T15:13:38.545355+0000","last_undegraded":"2024-01-24T15:13:38.545355+0000","last_fullsized":"2024-01-24T15:13:38.545355+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.544708+0000","last_change":"2024-01-24T15:13:38.122176+0000","last_active":"2024-01-24T15:13:38.544708+0000","last_peered":"2024-01-24T15:13:38.544708+0000","last_clean":"2024-01-24T15:13:38.544708+0000","last_became_active":"2024-01-24T15:13:38.121900+0000","last_became_peered":"2024-01-24T15:13:38.121900+0000","last_unstale":"2024-01-24T15:13:38.544708+0000","last_undegraded":"2024-01-24T15:13:38.544708+0000","last_fullsized":"2024-01-24T15:13:38.544708+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550198+0000","last_change":"2024-01-24T15:13:38.546422+0000","last_active":"2024-01-24T15:13:38.550198+0000","last_peered":"2024-01-24T15:13:38.550198+0000","last_clean":"2024-01-24T15:13:38.550198+0000","last_became_active":"2024-01-24T15:13:38.546168+0000","last_became_peered":"2024-01-24T15:13:38.546168+0000","last_unstale":"2024-01-24T15:13:38.550198+0000","last_undegraded":"2024-01-24T15:13:38.550198+0000","last_fullsized":"2024-01-24T15:13:38.550198+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.538459+0000","last_change":"2024-01-24T15:13:38.126860+0000","last_active":"2024-01-24T15:13:38.538459+0000","last_peered":"2024-01-24T15:13:38.538459+0000","last_clean":"2024-01-24T15:13:38.538459+0000","last_became_active":"2024-01-24T15:13:38.126528+0000","last_became_peered":"2024-01-24T15:13:38.126528+0000","last_unstale":"2024-01-24T15:13:38.538459+0000","last_undegraded":"2024-01-24T15:13:38.538459+0000","last_fullsized":"2024-01-24T15:13:38.538459+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558013+0000","last_change":"2024-01-24T15:13:38.551200+0000","last_active":"2024-01-24T15:13:38.558013+0000","last_peered":"2024-01-24T15:13:38.558013+0000","last_clean":"2024-01-24T15:13:38.558013+0000","last_became_active":"2024-01-24T15:13:38.550887+0000","last_became_peered":"2024-01-24T15:13:38.550887+0000","last_unstale":"2024-01-24T15:13:38.558013+0000","last_undegraded":"2024-01-24T15:13:38.558013+0000","last_fullsized":"2024-01-24T15:13:38.558013+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559181+0000","last_change":"2024-01-24T15:13:38.135912+0000","last_active":"2024-01-24T15:13:38.559181+0000","last_peered":"2024-01-24T15:13:38.559181+0000","last_clean":"2024-01-24T15:13:38.559181+0000","last_became_active":"2024-01-24T15:13:38.135637+0000","last_became_peered":"2024-01-24T15:13:38.135637+0000","last_unstale":"2024-01-24T15:13:38.559181+0000","last_undegraded":"2024-01-24T15:13:38.559181+0000","last_fullsized":"2024-01-24T15:13:38.559181+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.547990+0000","last_change":"2024-01-24T15:13:38.542686+0000","last_active":"2024-01-24T15:13:38.547990+0000","last_peered":"2024-01-24T15:13:38.547990+0000","last_clean":"2024-01-24T15:13:38.547990+0000","last_became_active":"2024-01-24T15:13:38.542392+0000","last_became_peered":"2024-01-24T15:13:38.542392+0000","last_unstale":"2024-01-24T15:13:38.547990+0000","last_undegraded":"2024-01-24T15:13:38.547990+0000","last_fullsized":"2024-01-24T15:13:38.547990+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.548937+0000","last_change":"2024-01-24T15:13:38.125904+0000","last_active":"2024-01-24T15:13:38.548937+0000","last_peered":"2024-01-24T15:13:38.548937+0000","last_clean":"2024-01-24T15:13:38.548937+0000","last_became_active":"2024-01-24T15:13:38.125609+0000","last_became_peered":"2024-01-24T15:13:38.125609+0000","last_unstale":"2024-01-24T15:13:38.548937+0000","last_undegraded":"2024-01-24T15:13:38.548937+0000","last_fullsized":"2024-01-24T15:13:38.548937+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557762+0000","last_change":"2024-01-24T15:13:38.553959+0000","last_active":"2024-01-24T15:13:38.557762+0000","last_peered":"2024-01-24T15:13:38.557762+0000","last_clean":"2024-01-24T15:13:38.557762+0000","last_became_active":"2024-01-24T15:13:38.553651+0000","last_became_peered":"2024-01-24T15:13:38.553651+0000","last_unstale":"2024-01-24T15:13:38.557762+0000","last_undegraded":"2024-01-24T15:13:38.557762+0000","last_fullsized":"2024-01-24T15:13:38.557762+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557845+0000","last_change":"2024-01-24T15:13:38.129150+0000","last_active":"2024-01-24T15:13:38.557845+0000","last_peered":"2024-01-24T15:13:38.557845+0000","last_clean":"2024-01-24T15:13:38.557845+0000","last_became_active":"2024-01-24T15:13:38.128848+0000","last_became_peered":"2024-01-24T15:13:38.128848+0000","last_unstale":"2024-01-24T15:13:38.557845+0000","last_undegraded":"2024-01-24T15:13:38.557845+0000","last_fullsized":"2024-01-24T15:13:38.557845+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558317+0000","last_change":"2024-01-24T15:13:38.555479+0000","last_active":"2024-01-24T15:13:38.558317+0000","last_peered":"2024-01-24T15:13:38.558317+0000","last_clean":"2024-01-24T15:13:38.558317+0000","last_became_active":"2024-01-24T15:13:38.555204+0000","last_became_peered":"2024-01-24T15:13:38.555204+0000","last_unstale":"2024-01-24T15:13:38.558317+0000","last_undegraded":"2024-01-24T15:13:38.558317+0000","last_fullsized":"2024-01-24T15:13:38.558317+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545284+0000","last_change":"2024-01-24T15:13:38.122342+0000","last_active":"2024-01-24T15:13:38.545284+0000","last_peered":"2024-01-24T15:13:38.545284+0000","last_clean":"2024-01-24T15:13:38.545284+0000","last_became_active":"2024-01-24T15:13:38.122043+0000","last_became_peered":"2024-01-24T15:13:38.122043+0000","last_unstale":"2024-01-24T15:13:38.545284+0000","last_undegraded":"2024-01-24T15:13:38.545284+0000","last_fullsized":"2024-01-24T15:13:38.545284+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550294+0000","last_change":"2024-01-24T15:13:38.547021+0000","last_active":"2024-01-24T15:13:38.550294+0000","last_peered":"2024-01-24T15:13:38.550294+0000","last_clean":"2024-01-24T15:13:38.550294+0000","last_became_active":"2024-01-24T15:13:38.546231+0000","last_became_peered":"2024-01-24T15:13:38.546231+0000","last_unstale":"2024-01-24T15:13:38.550294+0000","last_undegraded":"2024-01-24T15:13:38.550294+0000","last_fullsized":"2024-01-24T15:13:38.550294+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528666+0000","last_change":"2024-01-24T15:13:38.124808+0000","last_active":"2024-01-24T15:13:38.528666+0000","last_peered":"2024-01-24T15:13:38.528666+0000","last_clean":"2024-01-24T15:13:38.528666+0000","last_became_active":"2024-01-24T15:13:38.124504+0000","last_became_peered":"2024-01-24T15:13:38.124504+0000","last_unstale":"2024-01-24T15:13:38.528666+0000","last_undegraded":"2024-01-24T15:13:38.528666+0000","last_fullsized":"2024-01-24T15:13:38.528666+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562306+0000","last_change":"2024-01-24T15:13:38.558977+0000","last_active":"2024-01-24T15:13:38.562306+0000","last_peered":"2024-01-24T15:13:38.562306+0000","last_clean":"2024-01-24T15:13:38.562306+0000","last_became_active":"2024-01-24T15:13:38.558713+0000","last_became_peered":"2024-01-24T15:13:38.558713+0000","last_unstale":"2024-01-24T15:13:38.562306+0000","last_undegraded":"2024-01-24T15:13:38.562306+0000","last_fullsized":"2024-01-24T15:13:38.562306+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533932+0000","last_change":"2024-01-24T15:13:38.125055+0000","last_active":"2024-01-24T15:13:38.533932+0000","last_peered":"2024-01-24T15:13:38.533932+0000","last_clean":"2024-01-24T15:13:38.533932+0000","last_became_active":"2024-01-24T15:13:38.124694+0000","last_became_peered":"2024-01-24T15:13:38.124694+0000","last_unstale":"2024-01-24T15:13:38.533932+0000","last_undegraded":"2024-01-24T15:13:38.533932+0000","last_fullsized":"2024-01-24T15:13:38.533932+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542376+0000","last_change":"2024-01-24T15:13:38.540674+0000","last_active":"2024-01-24T15:13:38.542376+0000","last_peered":"2024-01-24T15:13:38.542376+0000","last_clean":"2024-01-24T15:13:38.542376+0000","last_became_active":"2024-01-24T15:13:38.540381+0000","last_became_peered":"2024-01-24T15:13:38.540381+0000","last_unstale":"2024-01-24T15:13:38.542376+0000","last_undegraded":"2024-01-24T15:13:38.542376+0000","last_fullsized":"2024-01-24T15:13:38.542376+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528151+0000","last_change":"2024-01-24T15:13:38.119586+0000","last_active":"2024-01-24T15:13:38.528151+0000","last_peered":"2024-01-24T15:13:38.528151+0000","last_clean":"2024-01-24T15:13:38.528151+0000","last_became_active":"2024-01-24T15:13:38.119285+0000","last_became_peered":"2024-01-24T15:13:38.119285+0000","last_unstale":"2024-01-24T15:13:38.528151+0000","last_undegraded":"2024-01-24T15:13:38.528151+0000","last_fullsized":"2024-01-24T15:13:38.528151+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551159+0000","last_change":"2024-01-24T15:13:38.544043+0000","last_active":"2024-01-24T15:13:38.551159+0000","last_peered":"2024-01-24T15:13:38.551159+0000","last_clean":"2024-01-24T15:13:38.551159+0000","last_became_active":"2024-01-24T15:13:38.543741+0000","last_became_peered":"2024-01-24T15:13:38.543741+0000","last_unstale":"2024-01-24T15:13:38.551159+0000","last_undegraded":"2024-01-24T15:13:38.551159+0000","last_fullsized":"2024-01-24T15:13:38.551159+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.544453+0000","last_change":"2024-01-24T15:13:38.124704+0000","last_active":"2024-01-24T15:13:38.544453+0000","last_peered":"2024-01-24T15:13:38.544453+0000","last_clean":"2024-01-24T15:13:38.544453+0000","last_became_active":"2024-01-24T15:13:38.124427+0000","last_became_peered":"2024-01-24T15:13:38.124427+0000","last_unstale":"2024-01-24T15:13:38.544453+0000","last_undegraded":"2024-01-24T15:13:38.544453+0000","last_fullsized":"2024-01-24T15:13:38.544453+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559264+0000","last_change":"2024-01-24T15:13:38.555038+0000","last_active":"2024-01-24T15:13:38.559264+0000","last_peered":"2024-01-24T15:13:38.559264+0000","last_clean":"2024-01-24T15:13:38.559264+0000","last_became_active":"2024-01-24T15:13:38.554763+0000","last_became_peered":"2024-01-24T15:13:38.554763+0000","last_unstale":"2024-01-24T15:13:38.559264+0000","last_undegraded":"2024-01-24T15:13:38.559264+0000","last_fullsized":"2024-01-24T15:13:38.559264+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559251+0000","last_change":"2024-01-24T15:13:38.122389+0000","last_active":"2024-01-24T15:13:38.559251+0000","last_peered":"2024-01-24T15:13:38.559251+0000","last_clean":"2024-01-24T15:13:38.559251+0000","last_became_active":"2024-01-24T15:13:38.121569+0000","last_became_peered":"2024-01-24T15:13:38.121569+0000","last_unstale":"2024-01-24T15:13:38.559251+0000","last_undegraded":"2024-01-24T15:13:38.559251+0000","last_fullsized":"2024-01-24T15:13:38.559251+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557360+0000","last_change":"2024-01-24T15:13:38.549783+0000","last_active":"2024-01-24T15:13:38.557360+0000","last_peered":"2024-01-24T15:13:38.557360+0000","last_clean":"2024-01-24T15:13:38.557360+0000","last_became_active":"2024-01-24T15:13:38.549374+0000","last_became_peered":"2024-01-24T15:13:38.549374+0000","last_unstale":"2024-01-24T15:13:38.557360+0000","last_undegraded":"2024-01-24T15:13:38.557360+0000","last_fullsized":"2024-01-24T15:13:38.557360+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557933+0000","last_change":"2024-01-24T15:13:38.121798+0000","last_active":"2024-01-24T15:13:38.557933+0000","last_peered":"2024-01-24T15:13:38.557933+0000","last_clean":"2024-01-24T15:13:38.557933+0000","last_became_active":"2024-01-24T15:13:38.121333+0000","last_became_peered":"2024-01-24T15:13:38.121333+0000","last_unstale":"2024-01-24T15:13:38.557933+0000","last_undegraded":"2024-01-24T15:13:38.557933+0000","last_fullsized":"2024-01-24T15:13:38.557933+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558881+0000","last_change":"2024-01-24T15:13:38.556149+0000","last_active":"2024-01-24T15:13:38.558881+0000","last_peered":"2024-01-24T15:13:38.558881+0000","last_clean":"2024-01-24T15:13:38.558881+0000","last_became_active":"2024-01-24T15:13:38.553256+0000","last_became_peered":"2024-01-24T15:13:38.553256+0000","last_unstale":"2024-01-24T15:13:38.558881+0000","last_undegraded":"2024-01-24T15:13:38.558881+0000","last_fullsized":"2024-01-24T15:13:38.558881+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.529610+0000","last_change":"2024-01-24T15:13:38.125875+0000","last_active":"2024-01-24T15:13:38.529610+0000","last_peered":"2024-01-24T15:13:38.529610+0000","last_clean":"2024-01-24T15:13:38.529610+0000","last_became_active":"2024-01-24T15:13:38.125536+0000","last_became_peered":"2024-01-24T15:13:38.125536+0000","last_unstale":"2024-01-24T15:13:38.529610+0000","last_undegraded":"2024-01-24T15:13:38.529610+0000","last_fullsized":"2024-01-24T15:13:38.529610+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.555881+0000","last_change":"2024-01-24T15:13:38.551671+0000","last_active":"2024-01-24T15:13:38.555881+0000","last_peered":"2024-01-24T15:13:38.555881+0000","last_clean":"2024-01-24T15:13:38.555881+0000","last_became_active":"2024-01-24T15:13:38.550779+0000","last_became_peered":"2024-01-24T15:13:38.550779+0000","last_unstale":"2024-01-24T15:13:38.555881+0000","last_undegraded":"2024-01-24T15:13:38.555881+0000","last_fullsized":"2024-01-24T15:13:38.555881+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532219+0000","last_change":"2024-01-24T15:13:38.122494+0000","last_active":"2024-01-24T15:13:38.532219+0000","last_peered":"2024-01-24T15:13:38.532219+0000","last_clean":"2024-01-24T15:13:38.532219+0000","last_became_active":"2024-01-24T15:13:38.122120+0000","last_became_peered":"2024-01-24T15:13:38.122120+0000","last_unstale":"2024-01-24T15:13:38.532219+0000","last_undegraded":"2024-01-24T15:13:38.532219+0000","last_fullsized":"2024-01-24T15:13:38.532219+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.560704+0000","last_change":"2024-01-24T15:13:38.554397+0000","last_active":"2024-01-24T15:13:38.560704+0000","last_peered":"2024-01-24T15:13:38.560704+0000","last_clean":"2024-01-24T15:13:38.560704+0000","last_became_active":"2024-01-24T15:13:38.553773+0000","last_became_peered":"2024-01-24T15:13:38.553773+0000","last_unstale":"2024-01-24T15:13:38.560704+0000","last_undegraded":"2024-01-24T15:13:38.560704+0000","last_fullsized":"2024-01-24T15:13:38.560704+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561628+0000","last_change":"2024-01-24T15:13:38.129725+0000","last_active":"2024-01-24T15:13:38.561628+0000","last_peered":"2024-01-24T15:13:38.561628+0000","last_clean":"2024-01-24T15:13:38.561628+0000","last_became_active":"2024-01-24T15:13:38.129199+0000","last_became_peered":"2024-01-24T15:13:38.129199+0000","last_unstale":"2024-01-24T15:13:38.561628+0000","last_undegraded":"2024-01-24T15:13:38.561628+0000","last_fullsized":"2024-01-24T15:13:38.561628+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556320+0000","last_change":"2024-01-24T15:13:38.550473+0000","last_active":"2024-01-24T15:13:38.556320+0000","last_peered":"2024-01-24T15:13:38.556320+0000","last_clean":"2024-01-24T15:13:38.556320+0000","last_became_active":"2024-01-24T15:13:38.550227+0000","last_became_peered":"2024-01-24T15:13:38.550227+0000","last_unstale":"2024-01-24T15:13:38.556320+0000","last_undegraded":"2024-01-24T15:13:38.556320+0000","last_fullsized":"2024-01-24T15:13:38.556320+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542174+0000","last_change":"2024-01-24T15:13:38.130818+0000","last_active":"2024-01-24T15:13:38.542174+0000","last_peered":"2024-01-24T15:13:38.542174+0000","last_clean":"2024-01-24T15:13:38.542174+0000","last_became_active":"2024-01-24T15:13:38.130326+0000","last_became_peered":"2024-01-24T15:13:38.130326+0000","last_unstale":"2024-01-24T15:13:38.542174+0000","last_undegraded":"2024-01-24T15:13:38.542174+0000","last_fullsized":"2024-01-24T15:13:38.542174+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562244+0000","last_change":"2024-01-24T15:13:38.557459+0000","last_active":"2024-01-24T15:13:38.562244+0000","last_peered":"2024-01-24T15:13:38.562244+0000","last_clean":"2024-01-24T15:13:38.562244+0000","last_became_active":"2024-01-24T15:13:38.557186+0000","last_became_peered":"2024-01-24T15:13:38.557186+0000","last_unstale":"2024-01-24T15:13:38.562244+0000","last_undegraded":"2024-01-24T15:13:38.562244+0000","last_fullsized":"2024-01-24T15:13:38.562244+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.541879+0000","last_change":"2024-01-24T15:13:38.134878+0000","last_active":"2024-01-24T15:13:38.541879+0000","last_peered":"2024-01-24T15:13:38.541879+0000","last_clean":"2024-01-24T15:13:38.541879+0000","last_became_active":"2024-01-24T15:13:38.134606+0000","last_became_peered":"2024-01-24T15:13:38.134606+0000","last_unstale":"2024-01-24T15:13:38.541879+0000","last_undegraded":"2024-01-24T15:13:38.541879+0000","last_fullsized":"2024-01-24T15:13:38.541879+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556378+0000","last_change":"2024-01-24T15:13:38.552294+0000","last_active":"2024-01-24T15:13:38.556378+0000","last_peered":"2024-01-24T15:13:38.556378+0000","last_clean":"2024-01-24T15:13:38.556378+0000","last_became_active":"2024-01-24T15:13:38.551919+0000","last_became_peered":"2024-01-24T15:13:38.551919+0000","last_unstale":"2024-01-24T15:13:38.556378+0000","last_undegraded":"2024-01-24T15:13:38.556378+0000","last_fullsized":"2024-01-24T15:13:38.556378+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535737+0000","last_change":"2024-01-24T15:13:38.132176+0000","last_active":"2024-01-24T15:13:38.535737+0000","last_peered":"2024-01-24T15:13:38.535737+0000","last_clean":"2024-01-24T15:13:38.535737+0000","last_became_active":"2024-01-24T15:13:38.131843+0000","last_became_peered":"2024-01-24T15:13:38.131843+0000","last_unstale":"2024-01-24T15:13:38.535737+0000","last_undegraded":"2024-01-24T15:13:38.535737+0000","last_fullsized":"2024-01-24T15:13:38.535737+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551132+0000","last_change":"2024-01-24T15:13:38.547885+0000","last_active":"2024-01-24T15:13:38.551132+0000","last_peered":"2024-01-24T15:13:38.551132+0000","last_clean":"2024-01-24T15:13:38.551132+0000","last_became_active":"2024-01-24T15:13:38.547529+0000","last_became_peered":"2024-01-24T15:13:38.547529+0000","last_unstale":"2024-01-24T15:13:38.551132+0000","last_undegraded":"2024-01-24T15:13:38.551132+0000","last_fullsized":"2024-01-24T15:13:38.551132+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559494+0000","last_change":"2024-01-24T15:13:38.134325+0000","last_active":"2024-01-24T15:13:38.559494+0000","last_peered":"2024-01-24T15:13:38.559494+0000","last_clean":"2024-01-24T15:13:38.559494+0000","last_became_active":"2024-01-24T15:13:38.134045+0000","last_became_peered":"2024-01-24T15:13:38.134045+0000","last_unstale":"2024-01-24T15:13:38.559494+0000","last_undegraded":"2024-01-24T15:13:38.559494+0000","last_fullsized":"2024-01-24T15:13:38.559494+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558867+0000","last_change":"2024-01-24T15:13:38.554889+0000","last_active":"2024-01-24T15:13:38.558867+0000","last_peered":"2024-01-24T15:13:38.558867+0000","last_clean":"2024-01-24T15:13:38.558867+0000","last_became_active":"2024-01-24T15:13:38.554624+0000","last_became_peered":"2024-01-24T15:13:38.554624+0000","last_unstale":"2024-01-24T15:13:38.558867+0000","last_undegraded":"2024-01-24T15:13:38.558867+0000","last_fullsized":"2024-01-24T15:13:38.558867+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537614+0000","last_change":"2024-01-24T15:13:38.126218+0000","last_active":"2024-01-24T15:13:38.537614+0000","last_peered":"2024-01-24T15:13:38.537614+0000","last_clean":"2024-01-24T15:13:38.537614+0000","last_became_active":"2024-01-24T15:13:38.125899+0000","last_became_peered":"2024-01-24T15:13:38.125899+0000","last_unstale":"2024-01-24T15:13:38.537614+0000","last_undegraded":"2024-01-24T15:13:38.537614+0000","last_fullsized":"2024-01-24T15:13:38.537614+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561397+0000","last_change":"2024-01-24T15:13:38.557841+0000","last_active":"2024-01-24T15:13:38.561397+0000","last_peered":"2024-01-24T15:13:38.561397+0000","last_clean":"2024-01-24T15:13:38.561397+0000","last_became_active":"2024-01-24T15:13:38.557543+0000","last_became_peered":"2024-01-24T15:13:38.557543+0000","last_unstale":"2024-01-24T15:13:38.561397+0000","last_undegraded":"2024-01-24T15:13:38.561397+0000","last_fullsized":"2024-01-24T15:13:38.561397+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.538204+0000","last_change":"2024-01-24T15:13:38.122661+0000","last_active":"2024-01-24T15:13:38.538204+0000","last_peered":"2024-01-24T15:13:38.538204+0000","last_clean":"2024-01-24T15:13:38.538204+0000","last_became_active":"2024-01-24T15:13:38.122384+0000","last_became_peered":"2024-01-24T15:13:38.122384+0000","last_unstale":"2024-01-24T15:13:38.538204+0000","last_undegraded":"2024-01-24T15:13:38.538204+0000","last_fullsized":"2024-01-24T15:13:38.538204+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557058+0000","last_change":"2024-01-24T15:13:38.553790+0000","last_active":"2024-01-24T15:13:38.557058+0000","last_peered":"2024-01-24T15:13:38.557058+0000","last_clean":"2024-01-24T15:13:38.557058+0000","last_became_active":"2024-01-24T15:13:38.553498+0000","last_became_peered":"2024-01-24T15:13:38.553498+0000","last_unstale":"2024-01-24T15:13:38.557058+0000","last_undegraded":"2024-01-24T15:13:38.557058+0000","last_fullsized":"2024-01-24T15:13:38.557058+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535549+0000","last_change":"2024-01-24T15:13:38.120324+0000","last_active":"2024-01-24T15:13:38.535549+0000","last_peered":"2024-01-24T15:13:38.535549+0000","last_clean":"2024-01-24T15:13:38.535549+0000","last_became_active":"2024-01-24T15:13:38.120007+0000","last_became_peered":"2024-01-24T15:13:38.120007+0000","last_unstale":"2024-01-24T15:13:38.535549+0000","last_undegraded":"2024-01-24T15:13:38.535549+0000","last_fullsized":"2024-01-24T15:13:38.535549+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558444+0000","last_change":"2024-01-24T15:13:38.552923+0000","last_active":"2024-01-24T15:13:38.558444+0000","last_peered":"2024-01-24T15:13:38.558444+0000","last_clean":"2024-01-24T15:13:38.558444+0000","last_became_active":"2024-01-24T15:13:38.552646+0000","last_became_peered":"2024-01-24T15:13:38.552646+0000","last_unstale":"2024-01-24T15:13:38.558444+0000","last_undegraded":"2024-01-24T15:13:38.558444+0000","last_fullsized":"2024-01-24T15:13:38.558444+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.539981+0000","last_change":"2024-01-24T15:13:38.129534+0000","last_active":"2024-01-24T15:13:38.539981+0000","last_peered":"2024-01-24T15:13:38.539981+0000","last_clean":"2024-01-24T15:13:38.539981+0000","last_became_active":"2024-01-24T15:13:38.128874+0000","last_became_peered":"2024-01-24T15:13:38.128874+0000","last_unstale":"2024-01-24T15:13:38.539981+0000","last_undegraded":"2024-01-24T15:13:38.539981+0000","last_fullsized":"2024-01-24T15:13:38.539981+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551365+0000","last_change":"2024-01-24T15:13:38.549194+0000","last_active":"2024-01-24T15:13:38.551365+0000","last_peered":"2024-01-24T15:13:38.551365+0000","last_clean":"2024-01-24T15:13:38.551365+0000","last_became_active":"2024-01-24T15:13:38.548929+0000","last_became_peered":"2024-01-24T15:13:38.548929+0000","last_unstale":"2024-01-24T15:13:38.551365+0000","last_undegraded":"2024-01-24T15:13:38.551365+0000","last_fullsized":"2024-01-24T15:13:38.551365+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542276+0000","last_change":"2024-01-24T15:13:38.130595+0000","last_active":"2024-01-24T15:13:38.542276+0000","last_peered":"2024-01-24T15:13:38.542276+0000","last_clean":"2024-01-24T15:13:38.542276+0000","last_became_active":"2024-01-24T15:13:38.130025+0000","last_became_peered":"2024-01-24T15:13:38.130025+0000","last_unstale":"2024-01-24T15:13:38.542276+0000","last_undegraded":"2024-01-24T15:13:38.542276+0000","last_fullsized":"2024-01-24T15:13:38.542276+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557093+0000","last_change":"2024-01-24T15:13:38.548613+0000","last_active":"2024-01-24T15:13:38.557093+0000","last_peered":"2024-01-24T15:13:38.557093+0000","last_clean":"2024-01-24T15:13:38.557093+0000","last_became_active":"2024-01-24T15:13:38.548288+0000","last_became_peered":"2024-01-24T15:13:38.548288+0000","last_unstale":"2024-01-24T15:13:38.557093+0000","last_undegraded":"2024-01-24T15:13:38.557093+0000","last_fullsized":"2024-01-24T15:13:38.557093+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535292+0000","last_change":"2024-01-24T15:13:38.131770+0000","last_active":"2024-01-24T15:13:38.535292+0000","last_peered":"2024-01-24T15:13:38.535292+0000","last_clean":"2024-01-24T15:13:38.535292+0000","last_became_active":"2024-01-24T15:13:38.131497+0000","last_became_peered":"2024-01-24T15:13:38.131497+0000","last_unstale":"2024-01-24T15:13:38.535292+0000","last_undegraded":"2024-01-24T15:13:38.535292+0000","last_fullsized":"2024-01-24T15:13:38.535292+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.546235+0000","last_change":"2024-01-24T15:13:38.538155+0000","last_active":"2024-01-24T15:13:38.546235+0000","last_peered":"2024-01-24T15:13:38.546235+0000","last_clean":"2024-01-24T15:13:38.546235+0000","last_became_active":"2024-01-24T15:13:38.537877+0000","last_became_peered":"2024-01-24T15:13:38.537877+0000","last_unstale":"2024-01-24T15:13:38.546235+0000","last_undegraded":"2024-01-24T15:13:38.546235+0000","last_fullsized":"2024-01-24T15:13:38.546235+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.531693+0000","last_change":"2024-01-24T15:13:38.124495+0000","last_active":"2024-01-24T15:13:38.531693+0000","last_peered":"2024-01-24T15:13:38.531693+0000","last_clean":"2024-01-24T15:13:38.531693+0000","last_became_active":"2024-01-24T15:13:38.124175+0000","last_became_peered":"2024-01-24T15:13:38.124175+0000","last_unstale":"2024-01-24T15:13:38.531693+0000","last_undegraded":"2024-01-24T15:13:38.531693+0000","last_fullsized":"2024-01-24T15:13:38.531693+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559906+0000","last_change":"2024-01-24T15:13:38.557558+0000","last_active":"2024-01-24T15:13:38.559906+0000","last_peered":"2024-01-24T15:13:38.559906+0000","last_clean":"2024-01-24T15:13:38.559906+0000","last_became_active":"2024-01-24T15:13:38.557290+0000","last_became_peered":"2024-01-24T15:13:38.557290+0000","last_unstale":"2024-01-24T15:13:38.559906+0000","last_undegraded":"2024-01-24T15:13:38.559906+0000","last_fullsized":"2024-01-24T15:13:38.559906+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535494+0000","last_change":"2024-01-24T15:13:38.121901+0000","last_active":"2024-01-24T15:13:38.535494+0000","last_peered":"2024-01-24T15:13:38.535494+0000","last_clean":"2024-01-24T15:13:38.535494+0000","last_became_active":"2024-01-24T15:13:38.121422+0000","last_became_peered":"2024-01-24T15:13:38.121422+0000","last_unstale":"2024-01-24T15:13:38.535494+0000","last_undegraded":"2024-01-24T15:13:38.535494+0000","last_fullsized":"2024-01-24T15:13:38.535494+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":144,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535646+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:13:38.535646+0000","last_peered":"2024-01-24T15:13:38.535646+0000","last_clean":"2024-01-24T15:13:38.535646+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:13:38.535646+0000","last_undegraded":"2024-01-24T15:13:38.535646+0000","last_fullsized":"2024-01-24T15:13:38.535646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558463+0000","last_change":"2024-01-24T15:13:38.552222+0000","last_active":"2024-01-24T15:13:38.558463+0000","last_peered":"2024-01-24T15:13:38.558463+0000","last_clean":"2024-01-24T15:13:38.558463+0000","last_became_active":"2024-01-24T15:13:38.551916+0000","last_became_peered":"2024-01-24T15:13:38.551916+0000","last_unstale":"2024-01-24T15:13:38.558463+0000","last_undegraded":"2024-01-24T15:13:38.558463+0000","last_fullsized":"2024-01-24T15:13:38.558463+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561816+0000","last_change":"2024-01-24T15:13:38.128196+0000","last_active":"2024-01-24T15:13:38.561816+0000","last_peered":"2024-01-24T15:13:38.561816+0000","last_clean":"2024-01-24T15:13:38.561816+0000","last_became_active":"2024-01-24T15:13:38.127782+0000","last_became_peered":"2024-01-24T15:13:38.127782+0000","last_unstale":"2024-01-24T15:13:38.561816+0000","last_undegraded":"2024-01-24T15:13:38.561816+0000","last_fullsized":"2024-01-24T15:13:38.561816+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":162,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.534014+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:13:38.534014+0000","last_peered":"2024-01-24T15:13:38.534014+0000","last_clean":"2024-01-24T15:13:38.534014+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:13:38.534014+0000","last_undegraded":"2024-01-24T15:13:38.534014+0000","last_fullsized":"2024-01-24T15:13:38.534014+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.547053+0000","last_change":"2024-01-24T15:13:38.538061+0000","last_active":"2024-01-24T15:13:38.547053+0000","last_peered":"2024-01-24T15:13:38.547053+0000","last_clean":"2024-01-24T15:13:38.547053+0000","last_became_active":"2024-01-24T15:13:38.537769+0000","last_became_peered":"2024-01-24T15:13:38.537769+0000","last_unstale":"2024-01-24T15:13:38.547053+0000","last_undegraded":"2024-01-24T15:13:38.547053+0000","last_fullsized":"2024-01-24T15:13:38.547053+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532992+0000","last_change":"2024-01-24T15:13:38.125400+0000","last_active":"2024-01-24T15:13:38.532992+0000","last_peered":"2024-01-24T15:13:38.532992+0000","last_clean":"2024-01-24T15:13:38.532992+0000","last_became_active":"2024-01-24T15:13:38.124725+0000","last_became_peered":"2024-01-24T15:13:38.124725+0000","last_unstale":"2024-01-24T15:13:38.532992+0000","last_undegraded":"2024-01-24T15:13:38.532992+0000","last_fullsized":"2024-01-24T15:13:38.532992+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528231+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:13:38.528231+0000","last_peered":"2024-01-24T15:13:38.528231+0000","last_clean":"2024-01-24T15:13:38.528231+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:13:38.528231+0000","last_undegraded":"2024-01-24T15:13:38.528231+0000","last_fullsized":"2024-01-24T15:13:38.528231+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561686+0000","last_change":"2024-01-24T15:13:38.554780+0000","last_active":"2024-01-24T15:13:38.561686+0000","last_peered":"2024-01-24T15:13:38.561686+0000","last_clean":"2024-01-24T15:13:38.561686+0000","last_became_active":"2024-01-24T15:13:38.554497+0000","last_became_peered":"2024-01-24T15:13:38.554497+0000","last_unstale":"2024-01-24T15:13:38.561686+0000","last_undegraded":"2024-01-24T15:13:38.561686+0000","last_fullsized":"2024-01-24T15:13:38.561686+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"177'3","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558878+0000","last_change":"2024-01-24T15:13:38.131986+0000","last_active":"2024-01-24T15:13:38.558878+0000","last_peered":"2024-01-24T15:13:38.558878+0000","last_clean":"2024-01-24T15:13:38.558878+0000","last_became_active":"2024-01-24T15:13:38.131663+0000","last_became_peered":"2024-01-24T15:13:38.131663+0000","last_unstale":"2024-01-24T15:13:38.558878+0000","last_undegraded":"2024-01-24T15:13:38.558878+0000","last_fullsized":"2024-01-24T15:13:38.558878+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":162,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533441+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:13:38.533441+0000","last_peered":"2024-01-24T15:13:38.533441+0000","last_clean":"2024-01-24T15:13:38.533441+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:13:38.533441+0000","last_undegraded":"2024-01-24T15:13:38.533441+0000","last_fullsized":"2024-01-24T15:13:38.533441+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":147,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.540654+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:13:38.540654+0000","last_peered":"2024-01-24T15:13:38.540654+0000","last_clean":"2024-01-24T15:13:38.540654+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:13:38.540654+0000","last_undegraded":"2024-01-24T15:13:38.540654+0000","last_fullsized":"2024-01-24T15:13:38.540654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562314+0000","last_change":"2024-01-24T15:13:38.559088+0000","last_active":"2024-01-24T15:13:38.562314+0000","last_peered":"2024-01-24T15:13:38.562314+0000","last_clean":"2024-01-24T15:13:38.562314+0000","last_became_active":"2024-01-24T15:13:38.558845+0000","last_became_peered":"2024-01-24T15:13:38.558845+0000","last_unstale":"2024-01-24T15:13:38.562314+0000","last_undegraded":"2024-01-24T15:13:38.562314+0000","last_fullsized":"2024-01-24T15:13:38.562314+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.541272+0000","last_change":"2024-01-24T15:13:38.134030+0000","last_active":"2024-01-24T15:13:38.541272+0000","last_peered":"2024-01-24T15:13:38.541272+0000","last_clean":"2024-01-24T15:13:38.541272+0000","last_became_active":"2024-01-24T15:13:38.133632+0000","last_became_peered":"2024-01-24T15:13:38.133632+0000","last_unstale":"2024-01-24T15:13:38.541272+0000","last_undegraded":"2024-01-24T15:13:38.541272+0000","last_fullsized":"2024-01-24T15:13:38.541272+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":145,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.540557+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:13:38.540557+0000","last_peered":"2024-01-24T15:13:38.540557+0000","last_clean":"2024-01-24T15:13:38.540557+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:13:38.540557+0000","last_undegraded":"2024-01-24T15:13:38.540557+0000","last_fullsized":"2024-01-24T15:13:38.540557+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558848+0000","last_change":"2024-01-24T15:13:38.553941+0000","last_active":"2024-01-24T15:13:38.558848+0000","last_peered":"2024-01-24T15:13:38.558848+0000","last_clean":"2024-01-24T15:13:38.558848+0000","last_became_active":"2024-01-24T15:13:38.552844+0000","last_became_peered":"2024-01-24T15:13:38.552844+0000","last_unstale":"2024-01-24T15:13:38.558848+0000","last_undegraded":"2024-01-24T15:13:38.558848+0000","last_fullsized":"2024-01-24T15:13:38.558848+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559449+0000","last_change":"2024-01-24T15:13:38.134046+0000","last_active":"2024-01-24T15:13:38.559449+0000","last_peered":"2024-01-24T15:13:38.559449+0000","last_clean":"2024-01-24T15:13:38.559449+0000","last_became_active":"2024-01-24T15:13:38.133216+0000","last_became_peered":"2024-01-24T15:13:38.133216+0000","last_unstale":"2024-01-24T15:13:38.559449+0000","last_undegraded":"2024-01-24T15:13:38.559449+0000","last_fullsized":"2024-01-24T15:13:38.559449+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":145,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.543025+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:13:38.543025+0000","last_peered":"2024-01-24T15:13:38.543025+0000","last_clean":"2024-01-24T15:13:38.543025+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:13:38.543025+0000","last_undegraded":"2024-01-24T15:13:38.543025+0000","last_fullsized":"2024-01-24T15:13:38.543025+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550309+0000","last_change":"2024-01-24T15:13:38.542717+0000","last_active":"2024-01-24T15:13:38.550309+0000","last_peered":"2024-01-24T15:13:38.550309+0000","last_clean":"2024-01-24T15:13:38.550309+0000","last_became_active":"2024-01-24T15:13:38.542424+0000","last_became_peered":"2024-01-24T15:13:38.542424+0000","last_unstale":"2024-01-24T15:13:38.550309+0000","last_undegraded":"2024-01-24T15:13:38.550309+0000","last_fullsized":"2024-01-24T15:13:38.550309+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.549726+0000","last_change":"2024-01-24T15:13:38.121778+0000","last_active":"2024-01-24T15:13:38.549726+0000","last_peered":"2024-01-24T15:13:38.549726+0000","last_clean":"2024-01-24T15:13:38.549726+0000","last_became_active":"2024-01-24T15:13:38.121489+0000","last_became_peered":"2024-01-24T15:13:38.121489+0000","last_unstale":"2024-01-24T15:13:38.549726+0000","last_undegraded":"2024-01-24T15:13:38.549726+0000","last_fullsized":"2024-01-24T15:13:38.549726+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537697+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:13:38.537697+0000","last_peered":"2024-01-24T15:13:38.537697+0000","last_clean":"2024-01-24T15:13:38.537697+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:13:38.537697+0000","last_undegraded":"2024-01-24T15:13:38.537697+0000","last_fullsized":"2024-01-24T15:13:38.537697+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558185+0000","last_change":"2024-01-24T15:13:38.550725+0000","last_active":"2024-01-24T15:13:38.558185+0000","last_peered":"2024-01-24T15:13:38.558185+0000","last_clean":"2024-01-24T15:13:38.558185+0000","last_became_active":"2024-01-24T15:13:38.550442+0000","last_became_peered":"2024-01-24T15:13:38.550442+0000","last_unstale":"2024-01-24T15:13:38.558185+0000","last_undegraded":"2024-01-24T15:13:38.558185+0000","last_fullsized":"2024-01-24T15:13:38.558185+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535563+0000","last_change":"2024-01-24T15:13:38.131236+0000","last_active":"2024-01-24T15:13:38.535563+0000","last_peered":"2024-01-24T15:13:38.535563+0000","last_clean":"2024-01-24T15:13:38.535563+0000","last_became_active":"2024-01-24T15:13:38.130890+0000","last_became_peered":"2024-01-24T15:13:38.130890+0000","last_unstale":"2024-01-24T15:13:38.535563+0000","last_undegraded":"2024-01-24T15:13:38.535563+0000","last_fullsized":"2024-01-24T15:13:38.535563+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537474+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:13:38.537474+0000","last_peered":"2024-01-24T15:13:38.537474+0000","last_clean":"2024-01-24T15:13:38.537474+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:13:38.537474+0000","last_undegraded":"2024-01-24T15:13:38.537474+0000","last_fullsized":"2024-01-24T15:13:38.537474+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575751,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303360,"kb_used_data":2168,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068480,"statfs":{"total":96636764160,"available":96326123520,"internally_reserved":0,"allocated":2220032,"data_stored":1270917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.59899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.28000000000000003},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.628,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.28899999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.1080000000000001},"last":0.442}]},{"osd":1,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.33300000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.505,"5min":0.40000000000000002,"15min":0.34799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.625}]},{"osd":2,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0329999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.38},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0880000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.41199999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.60899999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.33200000000000002},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.31900000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.69399999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.39600000000000002,"5min":0.38,"15min":0.34799999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.36199999999999999},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.31900000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.46400000000000002,"5min":0.40899999999999997,"15min":0.40500000000000003},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.81100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575752,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307464,"kb_used_data":2176,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064376,"statfs":{"total":96636764160,"available":96321921024,"internally_reserved":0,"allocated":2228224,"data_stored":1270993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.86599999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.42699999999999999,"15min":0.37},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57199999999999995}]},{"osd":1,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.84199999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77100000000000002,"5min":0.81200000000000006,"15min":1.258},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.40100000000000002},"max":{"1min":0.82299999999999995,"5min":0.85899999999999999,"15min":1.2430000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61599999999999999}]},{"osd":3,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.47199999999999998,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.39000000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.48699999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.317},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.628},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32400000000000001,"15min":0.30499999999999999},"max":{"1min":0.79300000000000004,"5min":0.80100000000000005,"15min":1.002},"last":0.64300000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.32500000000000001},"max":{"1min":0.76000000000000001,"5min":0.78100000000000003,"15min":0.83899999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":0.749,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.69399999999999995}]},{"osd":7,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.40899999999999997,"5min":0.372,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.90100000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.38},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":1.0429999999999999},"last":0.59199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575751,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307452,"kb_used_data":2164,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064388,"statfs":{"total":96636764160,"available":96321933312,"internally_reserved":0,"allocated":2215936,"data_stored":1270847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.56399999999999995,"5min":0.754,"15min":0.754},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.443,"15min":0.435},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.26800000000000002},"max":{"1min":0.55800000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.433}]},{"osd":2,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.33700000000000002,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.57599999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44600000000000001,"15min":0.44700000000000001},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.28199999999999997},"max":{"1min":0.56200000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.32800000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"min":{"1min":0.32400000000000001,"5min":0.314,"15min":0.309},"max":{"1min":0.58399999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.372},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.35799999999999998,"15min":0.313},"max":{"1min":0.59899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.51800000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.86099999999999999,"5min":1.044,"15min":1.0649999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.51400000000000001,"5min":0.47399999999999998,"15min":0.41699999999999998},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.58099999999999996}]},{"osd":5,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.49199999999999999,"15min":0.46300000000000002},"max":{"1min":0.83999999999999997,"5min":1.014,"15min":1.0780000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.439},"max":{"1min":0.86699999999999999,"5min":1.105,"15min":1.105},"last":0.60299999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.874,"5min":1.0029999999999999,"15min":1.069},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.56599999999999995,"5min":0.499,"15min":0.48199999999999998},"max":{"1min":0.92500000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.84299999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.83799999999999997,"5min":1.127,"15min":1.127},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.77500000000000002,"15min":0.76700000000000002},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.42799999999999999},"max":{"1min":0.88900000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.80900000000000005}]}]},{"osd":0,"up_from":14,"seq":60129543049,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307464,"kb_used_data":2176,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064376,"statfs":{"total":96636764160,"available":96321921024,"internally_reserved":0,"allocated":2228224,"data_stored":1271118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.28100000000000003,"5min":0.27500000000000002,"15min":0.247},"max":{"1min":0.60199999999999998,"5min":0.63400000000000001,"15min":0.66400000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.27700000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.54500000000000004}]},{"osd":2,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.29399999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.81999999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.84399999999999997},"last":0.496}]},{"osd":3,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.44600000000000001},"min":{"1min":0.317,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.65500000000000003,"15min":0.67000000000000004},"last":0.626},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.52000000000000002},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.90000000000000002},"last":0.66300000000000003}]},{"osd":4,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.433},"max":{"1min":0.99399999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.38700000000000001},"max":{"1min":0.995,"5min":0.995,"15min":1.0249999999999999},"last":0.78100000000000003}]},{"osd":5,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.70699999999999996},"min":{"1min":0.54900000000000004,"5min":0.498,"15min":0.42099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.94199999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.46899999999999997},"max":{"1min":1.022,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0820000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81100000000000005}]},{"osd":7,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0469999999999999,"5min":1.143,"15min":1.3260000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.375},"last":0.87}]}]},{"osd":2,"up_from":13,"seq":55834575751,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311564,"kb_used_data":2180,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060276,"statfs":{"total":96636764160,"available":96317722624,"internally_reserved":0,"allocated":2232320,"data_stored":1272937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.27200000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.60099999999999998,"5min":0.65000000000000002,"15min":0.66800000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.45100000000000001,"15min":0.45200000000000001},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.70399999999999996},"last":0.29099999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.25600000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.73599999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.46300000000000002},"min":{"1min":0.32500000000000001,"5min":0.29699999999999999,"15min":0.29599999999999999},"max":{"1min":0.622,"5min":0.69699999999999995,"15min":0.70099999999999996},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.73299999999999998},"last":0.34999999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96199999999999997,"5min":1.196,"15min":1.196},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":1,"15min":1.089},"last":0.38100000000000001}]},{"osd":5,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54400000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.111},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.46899999999999997,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.147,"15min":1.1830000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.64700000000000002,"5min":0.47699999999999998,"15min":0.45200000000000001},"max":{"1min":0.99399999999999999,"5min":0.998,"15min":1.129},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.58899999999999997,"5min":0.47299999999999998,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.98999999999999999,"15min":1.1950000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.753,"15min":0.746},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.497},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.248},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.65000000000000002,"5min":0.51500000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.24,"15min":1.24},"last":0.63100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307460,"kb_used_data":2172,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064380,"statfs":{"total":96636764160,"available":96321925120,"internally_reserved":0,"allocated":2224128,"data_stored":1271829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39800000000000002,"15min":0.39200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.247},"max":{"1min":0.56899999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.70199999999999996},"last":0.36199999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41899999999999998,"15min":0.41399999999999998},"min":{"1min":0.311,"5min":0.28100000000000003,"15min":0.27400000000000002},"max":{"1min":0.59999999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.34799999999999998,"5min":0.316,"15min":0.30499999999999999},"max":{"1min":0.67900000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.433}]},{"osd":2,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.497,"15min":0.502},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51300000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.41099999999999998},"max":{"1min":0.83299999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95999999999999996,"15min":1.5580000000000001},"last":0.58799999999999997}]},{"osd":5,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.1879999999999999,"15min":1.6240000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.647},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.45700000000000002,"15min":0.42699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.522},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.073,"5min":1.079,"15min":1.079},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.44800000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1499999999999999,"5min":1.246,"15min":1.246},"last":0.64400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575752,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307456,"kb_used_data":2168,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064384,"statfs":{"total":96636764160,"available":96321929216,"internally_reserved":0,"allocated":2220032,"data_stored":1270917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.46700000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.83699999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76500000000000001,"5min":0.78400000000000003,"15min":0.84199999999999997},"last":0.57099999999999995}]},{"osd":1,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.45500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.72499999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82999999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.60699999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.84699999999999998,"15min":0.88300000000000001},"last":0.496},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.626,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.372},"max":{"1min":0.79800000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52800000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.38400000000000001},"max":{"1min":0.79900000000000004,"5min":0.80400000000000005,"15min":0.89500000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.42399999999999999},"max":{"1min":0.71899999999999997,"5min":0.82199999999999995,"15min":0.92300000000000004},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":0.61299999999999999,"5min":0.79200000000000004,"15min":0.81799999999999995},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66400000000000003,"5min":0.72499999999999998,"15min":0.85499999999999998},"last":0.46000000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.68300000000000005,"5min":0.78500000000000003,"15min":0.85499999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.38200000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.86799999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Wed Jan 24 15:12:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.33600000000000002},"max":{"1min":0.65200000000000002,"5min":0.77500000000000002,"15min":0.82799999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.72599999999999998,"5min":0.81999999999999995,"15min":0.96599999999999997},"last":0.63200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575752,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307456,"kb_used_data":2168,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064384,"statfs":{"total":96636764160,"available":96321929216,"internally_reserved":0,"allocated":2220032,"data_stored":1270869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.38600000000000001,"15min":0.35099999999999998},"max":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":1.4470000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.81599999999999995},"last":0.53300000000000003}]},{"osd":1,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.82499999999999996},"last":0.435},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":1.4930000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.625,"15min":0.627},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.79300000000000004,"5min":0.80900000000000005,"15min":0.86499999999999999},"last":0.503},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":1.53},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.80800000000000005,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41899999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.65400000000000003,"5min":0.74399999999999999,"15min":0.84699999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":0.77200000000000002,"15min":0.80100000000000005},"last":0.38900000000000001}]},{"osd":6,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.5,"15min":0.504},"min":{"1min":0.374,"5min":0.35499999999999998,"15min":0.33400000000000002},"max":{"1min":0.65100000000000002,"5min":0.79400000000000004,"15min":0.81499999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.78500000000000003,"15min":0.81299999999999994},"last":0.44400000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.54900000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:13:47.237 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:13:47.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.376+0000 7f75abb12700 1 Processor -- start 2024-01-24T15:13:47.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.376+0000 7f75abb12700 1 -- start start 2024-01-24T15:13:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75abb12700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f75a40bf6a0 0x7f75a4159250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75abb12700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a4158dc0 con 0x7f75a4067e20 2024-01-24T15:13:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75abb12700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a4067380 con 0x7f75a4159740 2024-01-24T15:13:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75abb12700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a4158a60 con 0x7f75a40bf6a0 2024-01-24T15:13:47.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75a98ae700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f75a4067e20 0x7f75a4068200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34716/0 (socket says 172.21.15.29:34716) 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75a98ae700 1 -- 172.21.15.29:0/4043297417 learned_addr learned my addr 172.21.15.29:0/4043297417 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75a90ad700 1 --2- 172.21.15.29:0/4043297417 >> v2:172.21.15.163:3300/0 conn(0x7f75a40bf6a0 0x7f75a4159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274168224 0 0) 0x7f75a4158dc0 con 0x7f75a4067e20 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75900033f0 con 0x7f75a4067e20 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 324387972 0 0) 0x7f75900033f0 con 0x7f75a4067e20 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 >> v2:172.21.15.163:3300/0 conn(0x7f75a40bf6a0 msgr2=0x7f75a4159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75a88ac700 1 --2- 172.21.15.29:0/4043297417 >> v2:172.21.15.163:3300/0 conn(0x7f75a40bf6a0 0x7f75a4159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.377+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 >> v1:172.21.15.163:6789/0 conn(0x7f75a4159740 legacy=0x7f75a40664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a40638c0 con 0x7f75a4067e20 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f75a0001470 con 0x7f75a4067e20 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 >> v1:172.21.15.29:6789/0 conn(0x7f75a4067e20 legacy=0x7f75a4068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a88ac700 1 --2- 172.21.15.29:0/4043297417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7590003930 0x7f7590003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a88ac700 1 --2- 172.21.15.29:0/4043297417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7590004690 0x7f7590004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a88ac700 1 --2- 172.21.15.29:0/4043297417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7590005000 0x7f759000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a4158a60 con 0x7f7590004690 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759000bf30 con 0x7f7590003930 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759000c0f0 con 0x7f7590005000 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a90ad700 1 --2- 172.21.15.29:0/4043297417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7590004690 0x7f7590004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a98ae700 1 --2- 172.21.15.29:0/4043297417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7590003930 0x7f7590003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75aa0af700 1 --2- 172.21.15.29:0/4043297417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7590005000 0x7f759000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a90ad700 1 -- 172.21.15.29:0/4043297417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7590005000 msgr2=0x7f759000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a90ad700 1 --2- 172.21.15.29:0/4043297417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7590005000 0x7f759000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a90ad700 1 -- 172.21.15.29:0/4043297417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7590003930 msgr2=0x7f7590003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a90ad700 1 --2- 172.21.15.29:0/4043297417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7590003930 0x7f7590003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a90ad700 1 -- 172.21.15.29:0/4043297417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75900033f0 con 0x7f7590004690 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75aa0af700 1 -- 172.21.15.29:0/4043297417 reap_dead start 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a90ad700 1 --2- 172.21.15.29:0/4043297417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7590004690 0x7f7590004b10 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7f7594009a80 tx=0x7f7594009060).ready entity=mon.0 client_cookie=b4cba535f86b2a40 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594009430 con 0x7f7590004690 2024-01-24T15:13:47.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759400e640 con 0x7f7590004690 2024-01-24T15:13:47.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.378+0000 7f75a88ac700 1 -- 172.21.15.29:0/4043297417 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759400e9c0 con 0x7f7590004690 2024-01-24T15:13:47.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.379+0000 7f75abb12700 1 -- 172.21.15.29:0/4043297417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7590004690 msgr2=0x7f7590004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:47.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.379+0000 7f75abb12700 1 --2- 172.21.15.29:0/4043297417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7590004690 0x7f7590004b10 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7f7594009a80 tx=0x7f7594009060).stop 2024-01-24T15:13:47.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.379+0000 7f75abb12700 1 -- 172.21.15.29:0/4043297417 shutdown_connections 2024-01-24T15:13:47.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.379+0000 7f75abb12700 1 --2- 172.21.15.29:0/4043297417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7590004690 0x7f7590004b10 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.379+0000 7f75abb12700 1 -- 172.21.15.29:0/4043297417 >> 172.21.15.29:0/4043297417 conn(0x7f75a40c8bb0 msgr2=0x7f75a40c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:47.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.379+0000 7f75abb12700 1 -- 172.21.15.29:0/4043297417 shutdown_connections 2024-01-24T15:13:47.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.379+0000 7f75abb12700 1 -- 172.21.15.29:0/4043297417 wait complete. 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.380+0000 7f75abb12700 1 Processor -- start 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.380+0000 7f75abb12700 1 -- start start 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.381+0000 7f75abb12700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7590004690 0x7f75a4140db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.381+0000 7f75abb12700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75a416b860 0x7f75a414f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.381+0000 7f75abb12700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75a415ad70 0x7f75a40c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.381+0000 7f75abb12700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40aa640 con 0x7f75a416b860 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.381+0000 7f75abb12700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40aa7d0 con 0x7f75a415ad70 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.381+0000 7f75abb12700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a416bdd0 con 0x7f7590004690 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.381+0000 7f75a90ad700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75a416b860 0x7f75a414f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.381+0000 7f75a90ad700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75a416b860 0x7f75a414f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40542/0 (socket says 172.21.15.29:40542) 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.381+0000 7f75a90ad700 1 -- 172.21.15.29:0/3216163276 learned_addr learned my addr 172.21.15.29:0/3216163276 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.381+0000 7f75a98ae700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7590004690 0x7f75a4140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.381+0000 7f75aa0af700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75a415ad70 0x7f75a40c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.381+0000 7f75a90ad700 1 -- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7590004690 msgr2=0x7f75a4140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.382+0000 7f75a90ad700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7590004690 0x7f75a4140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.382+0000 7f75a90ad700 1 -- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75a415ad70 msgr2=0x7f75a40c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.382+0000 7f75a90ad700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75a415ad70 0x7f75a40c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.382+0000 7f75a90ad700 1 -- 172.21.15.29:0/3216163276 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a416c0b0 con 0x7f75a416b860 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.382+0000 7f75a90ad700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75a416b860 0x7f75a414f430 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f7594006d50 tx=0x7f759401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.382+0000 7f7599ffb700 1 -- 172.21.15.29:0/3216163276 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594009ec0 con 0x7f75a416b860 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.382+0000 7f7599ffb700 1 -- 172.21.15.29:0/3216163276 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759400ed70 con 0x7f75a416b860 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.382+0000 7f75abb12700 1 -- 172.21.15.29:0/3216163276 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a416c390 con 0x7f75a416b860 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.382+0000 7f7599ffb700 1 -- 172.21.15.29:0/3216163276 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759400eee0 con 0x7f75a416b860 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.382+0000 7f75abb12700 1 -- 172.21.15.29:0/3216163276 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a41412a0 con 0x7f75a416b860 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.383+0000 7f75abb12700 1 -- 172.21.15.29:0/3216163276 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a40475e0 con 0x7f75a416b860 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.384+0000 7f7599ffb700 1 -- 172.21.15.29:0/3216163276 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f759402e440 con 0x7f75a416b860 2024-01-24T15:13:47.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.384+0000 7f7599ffb700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f758805f270 0x7f7588061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.384+0000 7f7599ffb700 1 -- 172.21.15.29:0/3216163276 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f759407bfc0 con 0x7f75a416b860 2024-01-24T15:13:47.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.384+0000 7f75a98ae700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f758805f270 0x7f7588061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.385+0000 7f75a98ae700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f758805f270 0x7f7588061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f75a0000e50 tx=0x7f75a0001d80).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:47.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.389+0000 7f7599ffb700 1 -- 172.21.15.29:0/3216163276 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f759404cc80 con 0x7f75a416b860 2024-01-24T15:13:47.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.564+0000 7f75abb12700 1 -- 172.21.15.29:0/3216163276 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75a4063ba0 con 0x7f758805f270 2024-01-24T15:13:47.577 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.575+0000 7f7599ffb700 1 -- 172.21.15.29:0/3216163276 <== mgr.8056 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361928 (secure 0 0 0) 0x7f75a4063ba0 con 0x7f758805f270 2024-01-24T15:13:47.582 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:47.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.578+0000 7f75abb12700 1 -- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f758805f270 msgr2=0x7f7588061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:47.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.578+0000 7f75abb12700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f758805f270 0x7f7588061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f75a0000e50 tx=0x7f75a0001d80).stop 2024-01-24T15:13:47.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.578+0000 7f75abb12700 1 -- 172.21.15.29:0/3216163276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75a416b860 msgr2=0x7f75a414f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:47.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.578+0000 7f75abb12700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75a416b860 0x7f75a414f430 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f7594006d50 tx=0x7f759401a000).stop 2024-01-24T15:13:47.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.578+0000 7f75abb12700 1 -- 172.21.15.29:0/3216163276 shutdown_connections 2024-01-24T15:13:47.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.578+0000 7f75abb12700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75a415ad70 0x7f75a40c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.578+0000 7f75abb12700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7590004690 0x7f75a4140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.578+0000 7f75abb12700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f758805f270 0x7f7588061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.578+0000 7f75abb12700 1 --2- 172.21.15.29:0/3216163276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75a416b860 0x7f75a414f430 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.578+0000 7f75abb12700 1 -- 172.21.15.29:0/3216163276 >> 172.21.15.29:0/3216163276 conn(0x7f75a40c8bb0 msgr2=0x7f75a40adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:47.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.578+0000 7f75abb12700 1 -- 172.21.15.29:0/3216163276 shutdown_connections 2024-01-24T15:13:47.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.579+0000 7f75abb12700 1 -- 172.21.15.29:0/3216163276 wait complete. 2024-01-24T15:13:47.586 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:13:47.594 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-01-24T15:13:47.039881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2459676,"kb_used_data":17372,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752515044,"statfs":{"total":773094113280,"available":770575405056,"internally_reserved":0,"allocated":17788928,"data_stored":10170427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5036847,"num_objects":-12,"num_object_clones":0,"num_object_copies":-24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-497,"num_write_kb":-5303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.999459"},"pg_stats":[{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550222+0000","last_change":"2024-01-24T15:13:38.543675+0000","last_active":"2024-01-24T15:13:38.550222+0000","last_peered":"2024-01-24T15:13:38.550222+0000","last_clean":"2024-01-24T15:13:38.550222+0000","last_became_active":"2024-01-24T15:13:38.543012+0000","last_became_peered":"2024-01-24T15:13:38.543012+0000","last_unstale":"2024-01-24T15:13:38.550222+0000","last_undegraded":"2024-01-24T15:13:38.550222+0000","last_fullsized":"2024-01-24T15:13:38.550222+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3f","version":"177'3","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545770+0000","last_change":"2024-01-24T15:13:38.128076+0000","last_active":"2024-01-24T15:13:38.545770+0000","last_peered":"2024-01-24T15:13:38.545770+0000","last_clean":"2024-01-24T15:13:38.545770+0000","last_became_active":"2024-01-24T15:13:38.127791+0000","last_became_peered":"2024-01-24T15:13:38.127791+0000","last_unstale":"2024-01-24T15:13:38.545770+0000","last_undegraded":"2024-01-24T15:13:38.545770+0000","last_fullsized":"2024-01-24T15:13:38.545770+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557217+0000","last_change":"2024-01-24T15:13:38.554545+0000","last_active":"2024-01-24T15:13:38.557217+0000","last_peered":"2024-01-24T15:13:38.557217+0000","last_clean":"2024-01-24T15:13:38.557217+0000","last_became_active":"2024-01-24T15:13:38.554226+0000","last_became_peered":"2024-01-24T15:13:38.554226+0000","last_unstale":"2024-01-24T15:13:38.557217+0000","last_undegraded":"2024-01-24T15:13:38.557217+0000","last_fullsized":"2024-01-24T15:13:38.557217+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533665+0000","last_change":"2024-01-24T15:13:38.118261+0000","last_active":"2024-01-24T15:13:38.533665+0000","last_peered":"2024-01-24T15:13:38.533665+0000","last_clean":"2024-01-24T15:13:38.533665+0000","last_became_active":"2024-01-24T15:13:38.117923+0000","last_became_peered":"2024-01-24T15:13:38.117923+0000","last_unstale":"2024-01-24T15:13:38.533665+0000","last_undegraded":"2024-01-24T15:13:38.533665+0000","last_fullsized":"2024-01-24T15:13:38.533665+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562338+0000","last_change":"2024-01-24T15:13:38.559794+0000","last_active":"2024-01-24T15:13:38.562338+0000","last_peered":"2024-01-24T15:13:38.562338+0000","last_clean":"2024-01-24T15:13:38.562338+0000","last_became_active":"2024-01-24T15:13:38.559480+0000","last_became_peered":"2024-01-24T15:13:38.559480+0000","last_unstale":"2024-01-24T15:13:38.562338+0000","last_undegraded":"2024-01-24T15:13:38.562338+0000","last_fullsized":"2024-01-24T15:13:38.562338+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537224+0000","last_change":"2024-01-24T15:13:38.132008+0000","last_active":"2024-01-24T15:13:38.537224+0000","last_peered":"2024-01-24T15:13:38.537224+0000","last_clean":"2024-01-24T15:13:38.537224+0000","last_became_active":"2024-01-24T15:13:38.131711+0000","last_became_peered":"2024-01-24T15:13:38.131711+0000","last_unstale":"2024-01-24T15:13:38.537224+0000","last_undegraded":"2024-01-24T15:13:38.537224+0000","last_fullsized":"2024-01-24T15:13:38.537224+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558646+0000","last_change":"2024-01-24T15:13:38.548443+0000","last_active":"2024-01-24T15:13:38.558646+0000","last_peered":"2024-01-24T15:13:38.558646+0000","last_clean":"2024-01-24T15:13:38.558646+0000","last_became_active":"2024-01-24T15:13:38.546973+0000","last_became_peered":"2024-01-24T15:13:38.546973+0000","last_unstale":"2024-01-24T15:13:38.558646+0000","last_undegraded":"2024-01-24T15:13:38.558646+0000","last_fullsized":"2024-01-24T15:13:38.558646+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558098+0000","last_change":"2024-01-24T15:13:38.129340+0000","last_active":"2024-01-24T15:13:38.558098+0000","last_peered":"2024-01-24T15:13:38.558098+0000","last_clean":"2024-01-24T15:13:38.558098+0000","last_became_active":"2024-01-24T15:13:38.129057+0000","last_became_peered":"2024-01-24T15:13:38.129057+0000","last_unstale":"2024-01-24T15:13:38.558098+0000","last_undegraded":"2024-01-24T15:13:38.558098+0000","last_fullsized":"2024-01-24T15:13:38.558098+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.555919+0000","last_change":"2024-01-24T15:13:38.551741+0000","last_active":"2024-01-24T15:13:38.555919+0000","last_peered":"2024-01-24T15:13:38.555919+0000","last_clean":"2024-01-24T15:13:38.555919+0000","last_became_active":"2024-01-24T15:13:38.551165+0000","last_became_peered":"2024-01-24T15:13:38.551165+0000","last_unstale":"2024-01-24T15:13:38.555919+0000","last_undegraded":"2024-01-24T15:13:38.555919+0000","last_fullsized":"2024-01-24T15:13:38.555919+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557709+0000","last_change":"2024-01-24T15:13:38.122839+0000","last_active":"2024-01-24T15:13:38.557709+0000","last_peered":"2024-01-24T15:13:38.557709+0000","last_clean":"2024-01-24T15:13:38.557709+0000","last_became_active":"2024-01-24T15:13:38.121646+0000","last_became_peered":"2024-01-24T15:13:38.121646+0000","last_unstale":"2024-01-24T15:13:38.557709+0000","last_undegraded":"2024-01-24T15:13:38.557709+0000","last_fullsized":"2024-01-24T15:13:38.557709+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551349+0000","last_change":"2024-01-24T15:13:38.548421+0000","last_active":"2024-01-24T15:13:38.551349+0000","last_peered":"2024-01-24T15:13:38.551349+0000","last_clean":"2024-01-24T15:13:38.551349+0000","last_became_active":"2024-01-24T15:13:38.548168+0000","last_became_peered":"2024-01-24T15:13:38.548168+0000","last_unstale":"2024-01-24T15:13:38.551349+0000","last_undegraded":"2024-01-24T15:13:38.551349+0000","last_fullsized":"2024-01-24T15:13:38.551349+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3a","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550926+0000","last_change":"2024-01-24T15:13:38.131347+0000","last_active":"2024-01-24T15:13:38.550926+0000","last_peered":"2024-01-24T15:13:38.550926+0000","last_clean":"2024-01-24T15:13:38.550926+0000","last_became_active":"2024-01-24T15:13:38.127049+0000","last_became_peered":"2024-01-24T15:13:38.127049+0000","last_unstale":"2024-01-24T15:13:38.550926+0000","last_undegraded":"2024-01-24T15:13:38.550926+0000","last_fullsized":"2024-01-24T15:13:38.550926+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557161+0000","last_change":"2024-01-24T15:13:38.553689+0000","last_active":"2024-01-24T15:13:38.557161+0000","last_peered":"2024-01-24T15:13:38.557161+0000","last_clean":"2024-01-24T15:13:38.557161+0000","last_became_active":"2024-01-24T15:13:38.553388+0000","last_became_peered":"2024-01-24T15:13:38.553388+0000","last_unstale":"2024-01-24T15:13:38.557161+0000","last_undegraded":"2024-01-24T15:13:38.557161+0000","last_fullsized":"2024-01-24T15:13:38.557161+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.540830+0000","last_change":"2024-01-24T15:13:38.133854+0000","last_active":"2024-01-24T15:13:38.540830+0000","last_peered":"2024-01-24T15:13:38.540830+0000","last_clean":"2024-01-24T15:13:38.540830+0000","last_became_active":"2024-01-24T15:13:38.133583+0000","last_became_peered":"2024-01-24T15:13:38.133583+0000","last_unstale":"2024-01-24T15:13:38.540830+0000","last_undegraded":"2024-01-24T15:13:38.540830+0000","last_fullsized":"2024-01-24T15:13:38.540830+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559220+0000","last_change":"2024-01-24T15:13:38.556825+0000","last_active":"2024-01-24T15:13:38.559220+0000","last_peered":"2024-01-24T15:13:38.559220+0000","last_clean":"2024-01-24T15:13:38.559220+0000","last_became_active":"2024-01-24T15:13:38.556530+0000","last_became_peered":"2024-01-24T15:13:38.556530+0000","last_unstale":"2024-01-24T15:13:38.559220+0000","last_undegraded":"2024-01-24T15:13:38.559220+0000","last_fullsized":"2024-01-24T15:13:38.559220+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.544558+0000","last_change":"2024-01-24T15:13:38.129183+0000","last_active":"2024-01-24T15:13:38.544558+0000","last_peered":"2024-01-24T15:13:38.544558+0000","last_clean":"2024-01-24T15:13:38.544558+0000","last_became_active":"2024-01-24T15:13:38.128889+0000","last_became_peered":"2024-01-24T15:13:38.128889+0000","last_unstale":"2024-01-24T15:13:38.544558+0000","last_undegraded":"2024-01-24T15:13:38.544558+0000","last_fullsized":"2024-01-24T15:13:38.544558+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558035+0000","last_change":"2024-01-24T15:13:38.552886+0000","last_active":"2024-01-24T15:13:38.558035+0000","last_peered":"2024-01-24T15:13:38.558035+0000","last_clean":"2024-01-24T15:13:38.558035+0000","last_became_active":"2024-01-24T15:13:38.552602+0000","last_became_peered":"2024-01-24T15:13:38.552602+0000","last_unstale":"2024-01-24T15:13:38.558035+0000","last_undegraded":"2024-01-24T15:13:38.558035+0000","last_fullsized":"2024-01-24T15:13:38.558035+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537881+0000","last_change":"2024-01-24T15:13:38.122116+0000","last_active":"2024-01-24T15:13:38.537881+0000","last_peered":"2024-01-24T15:13:38.537881+0000","last_clean":"2024-01-24T15:13:38.537881+0000","last_became_active":"2024-01-24T15:13:38.121183+0000","last_became_peered":"2024-01-24T15:13:38.121183+0000","last_unstale":"2024-01-24T15:13:38.537881+0000","last_undegraded":"2024-01-24T15:13:38.537881+0000","last_fullsized":"2024-01-24T15:13:38.537881+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557777+0000","last_change":"2024-01-24T15:13:38.555214+0000","last_active":"2024-01-24T15:13:38.557777+0000","last_peered":"2024-01-24T15:13:38.557777+0000","last_clean":"2024-01-24T15:13:38.557777+0000","last_became_active":"2024-01-24T15:13:38.554915+0000","last_became_peered":"2024-01-24T15:13:38.554915+0000","last_unstale":"2024-01-24T15:13:38.557777+0000","last_undegraded":"2024-01-24T15:13:38.557777+0000","last_fullsized":"2024-01-24T15:13:38.557777+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.539484+0000","last_change":"2024-01-24T15:13:38.127556+0000","last_active":"2024-01-24T15:13:38.539484+0000","last_peered":"2024-01-24T15:13:38.539484+0000","last_clean":"2024-01-24T15:13:38.539484+0000","last_became_active":"2024-01-24T15:13:38.127215+0000","last_became_peered":"2024-01-24T15:13:38.127215+0000","last_unstale":"2024-01-24T15:13:38.539484+0000","last_undegraded":"2024-01-24T15:13:38.539484+0000","last_fullsized":"2024-01-24T15:13:38.539484+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557185+0000","last_change":"2024-01-24T15:13:38.551757+0000","last_active":"2024-01-24T15:13:38.557185+0000","last_peered":"2024-01-24T15:13:38.557185+0000","last_clean":"2024-01-24T15:13:38.557185+0000","last_became_active":"2024-01-24T15:13:38.551453+0000","last_became_peered":"2024-01-24T15:13:38.551453+0000","last_unstale":"2024-01-24T15:13:38.557185+0000","last_undegraded":"2024-01-24T15:13:38.557185+0000","last_fullsized":"2024-01-24T15:13:38.557185+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533408+0000","last_change":"2024-01-24T15:13:38.121049+0000","last_active":"2024-01-24T15:13:38.533408+0000","last_peered":"2024-01-24T15:13:38.533408+0000","last_clean":"2024-01-24T15:13:38.533408+0000","last_became_active":"2024-01-24T15:13:38.120804+0000","last_became_peered":"2024-01-24T15:13:38.120804+0000","last_unstale":"2024-01-24T15:13:38.533408+0000","last_undegraded":"2024-01-24T15:13:38.533408+0000","last_fullsized":"2024-01-24T15:13:38.533408+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551655+0000","last_change":"2024-01-24T15:13:38.547007+0000","last_active":"2024-01-24T15:13:38.551655+0000","last_peered":"2024-01-24T15:13:38.551655+0000","last_clean":"2024-01-24T15:13:38.551655+0000","last_became_active":"2024-01-24T15:13:38.546766+0000","last_became_peered":"2024-01-24T15:13:38.546766+0000","last_unstale":"2024-01-24T15:13:38.551655+0000","last_undegraded":"2024-01-24T15:13:38.551655+0000","last_fullsized":"2024-01-24T15:13:38.551655+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.34","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.565384+0000","last_change":"2024-01-24T15:13:38.123112+0000","last_active":"2024-01-24T15:13:38.565384+0000","last_peered":"2024-01-24T15:13:38.565384+0000","last_clean":"2024-01-24T15:13:38.565384+0000","last_became_active":"2024-01-24T15:13:38.122820+0000","last_became_peered":"2024-01-24T15:13:38.122820+0000","last_unstale":"2024-01-24T15:13:38.565384+0000","last_undegraded":"2024-01-24T15:13:38.565384+0000","last_fullsized":"2024-01-24T15:13:38.565384+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556831+0000","last_change":"2024-01-24T15:13:38.552223+0000","last_active":"2024-01-24T15:13:38.556831+0000","last_peered":"2024-01-24T15:13:38.556831+0000","last_clean":"2024-01-24T15:13:38.556831+0000","last_became_active":"2024-01-24T15:13:38.551704+0000","last_became_peered":"2024-01-24T15:13:38.551704+0000","last_unstale":"2024-01-24T15:13:38.556831+0000","last_undegraded":"2024-01-24T15:13:38.556831+0000","last_fullsized":"2024-01-24T15:13:38.556831+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.33","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.552767+0000","last_change":"2024-01-24T15:13:38.120932+0000","last_active":"2024-01-24T15:13:38.552767+0000","last_peered":"2024-01-24T15:13:38.552767+0000","last_clean":"2024-01-24T15:13:38.552767+0000","last_became_active":"2024-01-24T15:13:38.120622+0000","last_became_peered":"2024-01-24T15:13:38.120622+0000","last_unstale":"2024-01-24T15:13:38.552767+0000","last_undegraded":"2024-01-24T15:13:38.552767+0000","last_fullsized":"2024-01-24T15:13:38.552767+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551328+0000","last_change":"2024-01-24T15:13:38.547976+0000","last_active":"2024-01-24T15:13:38.551328+0000","last_peered":"2024-01-24T15:13:38.551328+0000","last_clean":"2024-01-24T15:13:38.551328+0000","last_became_active":"2024-01-24T15:13:38.547686+0000","last_became_peered":"2024-01-24T15:13:38.547686+0000","last_unstale":"2024-01-24T15:13:38.551328+0000","last_undegraded":"2024-01-24T15:13:38.551328+0000","last_fullsized":"2024-01-24T15:13:38.551328+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528656+0000","last_change":"2024-01-24T15:13:38.125651+0000","last_active":"2024-01-24T15:13:38.528656+0000","last_peered":"2024-01-24T15:13:38.528656+0000","last_clean":"2024-01-24T15:13:38.528656+0000","last_became_active":"2024-01-24T15:13:38.125334+0000","last_became_peered":"2024-01-24T15:13:38.125334+0000","last_unstale":"2024-01-24T15:13:38.528656+0000","last_undegraded":"2024-01-24T15:13:38.528656+0000","last_fullsized":"2024-01-24T15:13:38.528656+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545374+0000","last_change":"2024-01-24T15:13:38.543804+0000","last_active":"2024-01-24T15:13:38.545374+0000","last_peered":"2024-01-24T15:13:38.545374+0000","last_clean":"2024-01-24T15:13:38.545374+0000","last_became_active":"2024-01-24T15:13:38.543259+0000","last_became_peered":"2024-01-24T15:13:38.543259+0000","last_unstale":"2024-01-24T15:13:38.545374+0000","last_undegraded":"2024-01-24T15:13:38.545374+0000","last_fullsized":"2024-01-24T15:13:38.545374+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528152+0000","last_change":"2024-01-24T15:13:38.125234+0000","last_active":"2024-01-24T15:13:38.528152+0000","last_peered":"2024-01-24T15:13:38.528152+0000","last_clean":"2024-01-24T15:13:38.528152+0000","last_became_active":"2024-01-24T15:13:38.124520+0000","last_became_peered":"2024-01-24T15:13:38.124520+0000","last_unstale":"2024-01-24T15:13:38.528152+0000","last_undegraded":"2024-01-24T15:13:38.528152+0000","last_fullsized":"2024-01-24T15:13:38.528152+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557010+0000","last_change":"2024-01-24T15:13:38.548644+0000","last_active":"2024-01-24T15:13:38.557010+0000","last_peered":"2024-01-24T15:13:38.557010+0000","last_clean":"2024-01-24T15:13:38.557010+0000","last_became_active":"2024-01-24T15:13:38.548355+0000","last_became_peered":"2024-01-24T15:13:38.548355+0000","last_unstale":"2024-01-24T15:13:38.557010+0000","last_undegraded":"2024-01-24T15:13:38.557010+0000","last_fullsized":"2024-01-24T15:13:38.557010+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.30","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.552473+0000","last_change":"2024-01-24T15:13:38.123443+0000","last_active":"2024-01-24T15:13:38.552473+0000","last_peered":"2024-01-24T15:13:38.552473+0000","last_clean":"2024-01-24T15:13:38.552473+0000","last_became_active":"2024-01-24T15:13:38.123066+0000","last_became_peered":"2024-01-24T15:13:38.123066+0000","last_unstale":"2024-01-24T15:13:38.552473+0000","last_undegraded":"2024-01-24T15:13:38.552473+0000","last_fullsized":"2024-01-24T15:13:38.552473+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557443+0000","last_change":"2024-01-24T15:13:38.546142+0000","last_active":"2024-01-24T15:13:38.557443+0000","last_peered":"2024-01-24T15:13:38.557443+0000","last_clean":"2024-01-24T15:13:38.557443+0000","last_became_active":"2024-01-24T15:13:38.545868+0000","last_became_peered":"2024-01-24T15:13:38.545868+0000","last_unstale":"2024-01-24T15:13:38.557443+0000","last_undegraded":"2024-01-24T15:13:38.557443+0000","last_fullsized":"2024-01-24T15:13:38.557443+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537981+0000","last_change":"2024-01-24T15:13:38.121742+0000","last_active":"2024-01-24T15:13:38.537981+0000","last_peered":"2024-01-24T15:13:38.537981+0000","last_clean":"2024-01-24T15:13:38.537981+0000","last_became_active":"2024-01-24T15:13:38.121386+0000","last_became_peered":"2024-01-24T15:13:38.121386+0000","last_unstale":"2024-01-24T15:13:38.537981+0000","last_undegraded":"2024-01-24T15:13:38.537981+0000","last_fullsized":"2024-01-24T15:13:38.537981+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562294+0000","last_change":"2024-01-24T15:13:38.558575+0000","last_active":"2024-01-24T15:13:38.562294+0000","last_peered":"2024-01-24T15:13:38.562294+0000","last_clean":"2024-01-24T15:13:38.562294+0000","last_became_active":"2024-01-24T15:13:38.558314+0000","last_became_peered":"2024-01-24T15:13:38.558314+0000","last_unstale":"2024-01-24T15:13:38.562294+0000","last_undegraded":"2024-01-24T15:13:38.562294+0000","last_fullsized":"2024-01-24T15:13:38.562294+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532043+0000","last_change":"2024-01-24T15:13:38.122045+0000","last_active":"2024-01-24T15:13:38.532043+0000","last_peered":"2024-01-24T15:13:38.532043+0000","last_clean":"2024-01-24T15:13:38.532043+0000","last_became_active":"2024-01-24T15:13:38.121766+0000","last_became_peered":"2024-01-24T15:13:38.121766+0000","last_unstale":"2024-01-24T15:13:38.532043+0000","last_undegraded":"2024-01-24T15:13:38.532043+0000","last_fullsized":"2024-01-24T15:13:38.532043+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545141+0000","last_change":"2024-01-24T15:13:38.541723+0000","last_active":"2024-01-24T15:13:38.545141+0000","last_peered":"2024-01-24T15:13:38.545141+0000","last_clean":"2024-01-24T15:13:38.545141+0000","last_became_active":"2024-01-24T15:13:38.540083+0000","last_became_peered":"2024-01-24T15:13:38.540083+0000","last_unstale":"2024-01-24T15:13:38.545141+0000","last_undegraded":"2024-01-24T15:13:38.545141+0000","last_fullsized":"2024-01-24T15:13:38.545141+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2d","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558223+0000","last_change":"2024-01-24T15:13:38.130633+0000","last_active":"2024-01-24T15:13:38.558223+0000","last_peered":"2024-01-24T15:13:38.558223+0000","last_clean":"2024-01-24T15:13:38.558223+0000","last_became_active":"2024-01-24T15:13:38.130234+0000","last_became_peered":"2024-01-24T15:13:38.130234+0000","last_unstale":"2024-01-24T15:13:38.558223+0000","last_undegraded":"2024-01-24T15:13:38.558223+0000","last_fullsized":"2024-01-24T15:13:38.558223+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.555856+0000","last_change":"2024-01-24T15:13:38.551556+0000","last_active":"2024-01-24T15:13:38.555856+0000","last_peered":"2024-01-24T15:13:38.555856+0000","last_clean":"2024-01-24T15:13:38.555856+0000","last_became_active":"2024-01-24T15:13:38.551182+0000","last_became_peered":"2024-01-24T15:13:38.551182+0000","last_unstale":"2024-01-24T15:13:38.555856+0000","last_undegraded":"2024-01-24T15:13:38.555856+0000","last_fullsized":"2024-01-24T15:13:38.555856+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532179+0000","last_change":"2024-01-24T15:13:38.128558+0000","last_active":"2024-01-24T15:13:38.532179+0000","last_peered":"2024-01-24T15:13:38.532179+0000","last_clean":"2024-01-24T15:13:38.532179+0000","last_became_active":"2024-01-24T15:13:38.128075+0000","last_became_peered":"2024-01-24T15:13:38.128075+0000","last_unstale":"2024-01-24T15:13:38.532179+0000","last_undegraded":"2024-01-24T15:13:38.532179+0000","last_fullsized":"2024-01-24T15:13:38.532179+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556852+0000","last_change":"2024-01-24T15:13:38.552974+0000","last_active":"2024-01-24T15:13:38.556852+0000","last_peered":"2024-01-24T15:13:38.556852+0000","last_clean":"2024-01-24T15:13:38.556852+0000","last_became_active":"2024-01-24T15:13:38.552706+0000","last_became_peered":"2024-01-24T15:13:38.552706+0000","last_unstale":"2024-01-24T15:13:38.556852+0000","last_undegraded":"2024-01-24T15:13:38.556852+0000","last_fullsized":"2024-01-24T15:13:38.556852+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.553360+0000","last_change":"2024-01-24T15:13:38.127821+0000","last_active":"2024-01-24T15:13:38.553360+0000","last_peered":"2024-01-24T15:13:38.553360+0000","last_clean":"2024-01-24T15:13:38.553360+0000","last_became_active":"2024-01-24T15:13:38.127478+0000","last_became_peered":"2024-01-24T15:13:38.127478+0000","last_unstale":"2024-01-24T15:13:38.553360+0000","last_undegraded":"2024-01-24T15:13:38.553360+0000","last_fullsized":"2024-01-24T15:13:38.553360+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558279+0000","last_change":"2024-01-24T15:13:38.555643+0000","last_active":"2024-01-24T15:13:38.558279+0000","last_peered":"2024-01-24T15:13:38.558279+0000","last_clean":"2024-01-24T15:13:38.558279+0000","last_became_active":"2024-01-24T15:13:38.555338+0000","last_became_peered":"2024-01-24T15:13:38.555338+0000","last_unstale":"2024-01-24T15:13:38.558279+0000","last_undegraded":"2024-01-24T15:13:38.558279+0000","last_fullsized":"2024-01-24T15:13:38.558279+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528767+0000","last_change":"2024-01-24T15:13:38.119386+0000","last_active":"2024-01-24T15:13:38.528767+0000","last_peered":"2024-01-24T15:13:38.528767+0000","last_clean":"2024-01-24T15:13:38.528767+0000","last_became_active":"2024-01-24T15:13:38.118881+0000","last_became_peered":"2024-01-24T15:13:38.118881+0000","last_unstale":"2024-01-24T15:13:38.528767+0000","last_undegraded":"2024-01-24T15:13:38.528767+0000","last_fullsized":"2024-01-24T15:13:38.528767+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557245+0000","last_change":"2024-01-24T15:13:38.552282+0000","last_active":"2024-01-24T15:13:38.557245+0000","last_peered":"2024-01-24T15:13:38.557245+0000","last_clean":"2024-01-24T15:13:38.557245+0000","last_became_active":"2024-01-24T15:13:38.552007+0000","last_became_peered":"2024-01-24T15:13:38.552007+0000","last_unstale":"2024-01-24T15:13:38.557245+0000","last_undegraded":"2024-01-24T15:13:38.557245+0000","last_fullsized":"2024-01-24T15:13:38.557245+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533440+0000","last_change":"2024-01-24T15:13:38.126343+0000","last_active":"2024-01-24T15:13:38.533440+0000","last_peered":"2024-01-24T15:13:38.533440+0000","last_clean":"2024-01-24T15:13:38.533440+0000","last_became_active":"2024-01-24T15:13:38.126099+0000","last_became_peered":"2024-01-24T15:13:38.126099+0000","last_unstale":"2024-01-24T15:13:38.533440+0000","last_undegraded":"2024-01-24T15:13:38.533440+0000","last_fullsized":"2024-01-24T15:13:38.533440+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562224+0000","last_change":"2024-01-24T15:13:38.557923+0000","last_active":"2024-01-24T15:13:38.562224+0000","last_peered":"2024-01-24T15:13:38.562224+0000","last_clean":"2024-01-24T15:13:38.562224+0000","last_became_active":"2024-01-24T15:13:38.557635+0000","last_became_peered":"2024-01-24T15:13:38.557635+0000","last_unstale":"2024-01-24T15:13:38.562224+0000","last_undegraded":"2024-01-24T15:13:38.562224+0000","last_fullsized":"2024-01-24T15:13:38.562224+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535538+0000","last_change":"2024-01-24T15:13:38.117376+0000","last_active":"2024-01-24T15:13:38.535538+0000","last_peered":"2024-01-24T15:13:38.535538+0000","last_clean":"2024-01-24T15:13:38.535538+0000","last_became_active":"2024-01-24T15:13:38.117052+0000","last_became_peered":"2024-01-24T15:13:38.117052+0000","last_unstale":"2024-01-24T15:13:38.535538+0000","last_undegraded":"2024-01-24T15:13:38.535538+0000","last_fullsized":"2024-01-24T15:13:38.535538+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.554021+0000","last_change":"2024-01-24T15:13:38.546194+0000","last_active":"2024-01-24T15:13:38.554021+0000","last_peered":"2024-01-24T15:13:38.554021+0000","last_clean":"2024-01-24T15:13:38.554021+0000","last_became_active":"2024-01-24T15:13:38.545907+0000","last_became_peered":"2024-01-24T15:13:38.545907+0000","last_unstale":"2024-01-24T15:13:38.554021+0000","last_undegraded":"2024-01-24T15:13:38.554021+0000","last_fullsized":"2024-01-24T15:13:38.554021+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537470+0000","last_change":"2024-01-24T15:13:38.120448+0000","last_active":"2024-01-24T15:13:38.537470+0000","last_peered":"2024-01-24T15:13:38.537470+0000","last_clean":"2024-01-24T15:13:38.537470+0000","last_became_active":"2024-01-24T15:13:38.120165+0000","last_became_peered":"2024-01-24T15:13:38.120165+0000","last_unstale":"2024-01-24T15:13:38.537470+0000","last_undegraded":"2024-01-24T15:13:38.537470+0000","last_fullsized":"2024-01-24T15:13:38.537470+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558719+0000","last_change":"2024-01-24T15:13:38.555417+0000","last_active":"2024-01-24T15:13:38.558719+0000","last_peered":"2024-01-24T15:13:38.558719+0000","last_clean":"2024-01-24T15:13:38.558719+0000","last_became_active":"2024-01-24T15:13:38.555118+0000","last_became_peered":"2024-01-24T15:13:38.555118+0000","last_unstale":"2024-01-24T15:13:38.558719+0000","last_undegraded":"2024-01-24T15:13:38.558719+0000","last_fullsized":"2024-01-24T15:13:38.558719+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.26","version":"178'1","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-01-24T15:13:39.660787+0000","last_change":"2024-01-24T15:13:38.128489+0000","last_active":"2024-01-24T15:13:39.660787+0000","last_peered":"2024-01-24T15:13:39.660787+0000","last_clean":"2024-01-24T15:13:39.660787+0000","last_became_active":"2024-01-24T15:13:38.128202+0000","last_became_peered":"2024-01-24T15:13:38.128202+0000","last_unstale":"2024-01-24T15:13:39.660787+0000","last_undegraded":"2024-01-24T15:13:39.660787+0000","last_fullsized":"2024-01-24T15:13:39.660787+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562480+0000","last_change":"2024-01-24T15:13:38.561900+0000","last_active":"2024-01-24T15:13:38.562480+0000","last_peered":"2024-01-24T15:13:38.562480+0000","last_clean":"2024-01-24T15:13:38.562480+0000","last_became_active":"2024-01-24T15:13:38.561655+0000","last_became_peered":"2024-01-24T15:13:38.561655+0000","last_unstale":"2024-01-24T15:13:38.562480+0000","last_undegraded":"2024-01-24T15:13:38.562480+0000","last_fullsized":"2024-01-24T15:13:38.562480+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542822+0000","last_change":"2024-01-24T15:13:38.130503+0000","last_active":"2024-01-24T15:13:38.542822+0000","last_peered":"2024-01-24T15:13:38.542822+0000","last_clean":"2024-01-24T15:13:38.542822+0000","last_became_active":"2024-01-24T15:13:38.129933+0000","last_became_peered":"2024-01-24T15:13:38.129933+0000","last_unstale":"2024-01-24T15:13:38.542822+0000","last_undegraded":"2024-01-24T15:13:38.542822+0000","last_fullsized":"2024-01-24T15:13:38.542822+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.548146+0000","last_change":"2024-01-24T15:13:38.546025+0000","last_active":"2024-01-24T15:13:38.548146+0000","last_peered":"2024-01-24T15:13:38.548146+0000","last_clean":"2024-01-24T15:13:38.548146+0000","last_became_active":"2024-01-24T15:13:38.545702+0000","last_became_peered":"2024-01-24T15:13:38.545702+0000","last_unstale":"2024-01-24T15:13:38.548146+0000","last_undegraded":"2024-01-24T15:13:38.548146+0000","last_fullsized":"2024-01-24T15:13:38.548146+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.546793+0000","last_change":"2024-01-24T15:13:38.128387+0000","last_active":"2024-01-24T15:13:38.546793+0000","last_peered":"2024-01-24T15:13:38.546793+0000","last_clean":"2024-01-24T15:13:38.546793+0000","last_became_active":"2024-01-24T15:13:38.128088+0000","last_became_peered":"2024-01-24T15:13:38.128088+0000","last_unstale":"2024-01-24T15:13:38.546793+0000","last_undegraded":"2024-01-24T15:13:38.546793+0000","last_fullsized":"2024-01-24T15:13:38.546793+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.560725+0000","last_change":"2024-01-24T15:13:38.554569+0000","last_active":"2024-01-24T15:13:38.560725+0000","last_peered":"2024-01-24T15:13:38.560725+0000","last_clean":"2024-01-24T15:13:38.560725+0000","last_became_active":"2024-01-24T15:13:38.554110+0000","last_became_peered":"2024-01-24T15:13:38.554110+0000","last_unstale":"2024-01-24T15:13:38.560725+0000","last_undegraded":"2024-01-24T15:13:38.560725+0000","last_fullsized":"2024-01-24T15:13:38.560725+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535857+0000","last_change":"2024-01-24T15:13:38.127508+0000","last_active":"2024-01-24T15:13:38.535857+0000","last_peered":"2024-01-24T15:13:38.535857+0000","last_clean":"2024-01-24T15:13:38.535857+0000","last_became_active":"2024-01-24T15:13:38.127242+0000","last_became_peered":"2024-01-24T15:13:38.127242+0000","last_unstale":"2024-01-24T15:13:38.535857+0000","last_undegraded":"2024-01-24T15:13:38.535857+0000","last_fullsized":"2024-01-24T15:13:38.535857+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557111+0000","last_change":"2024-01-24T15:13:38.552603+0000","last_active":"2024-01-24T15:13:38.557111+0000","last_peered":"2024-01-24T15:13:38.557111+0000","last_clean":"2024-01-24T15:13:38.557111+0000","last_became_active":"2024-01-24T15:13:38.550713+0000","last_became_peered":"2024-01-24T15:13:38.550713+0000","last_unstale":"2024-01-24T15:13:38.557111+0000","last_undegraded":"2024-01-24T15:13:38.557111+0000","last_fullsized":"2024-01-24T15:13:38.557111+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535420+0000","last_change":"2024-01-24T15:13:38.121734+0000","last_active":"2024-01-24T15:13:38.535420+0000","last_peered":"2024-01-24T15:13:38.535420+0000","last_clean":"2024-01-24T15:13:38.535420+0000","last_became_active":"2024-01-24T15:13:38.121067+0000","last_became_peered":"2024-01-24T15:13:38.121067+0000","last_unstale":"2024-01-24T15:13:38.535420+0000","last_undegraded":"2024-01-24T15:13:38.535420+0000","last_fullsized":"2024-01-24T15:13:38.535420+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545355+0000","last_change":"2024-01-24T15:13:38.543615+0000","last_active":"2024-01-24T15:13:38.545355+0000","last_peered":"2024-01-24T15:13:38.545355+0000","last_clean":"2024-01-24T15:13:38.545355+0000","last_became_active":"2024-01-24T15:13:38.541511+0000","last_became_peered":"2024-01-24T15:13:38.541511+0000","last_unstale":"2024-01-24T15:13:38.545355+0000","last_undegraded":"2024-01-24T15:13:38.545355+0000","last_fullsized":"2024-01-24T15:13:38.545355+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.544708+0000","last_change":"2024-01-24T15:13:38.122176+0000","last_active":"2024-01-24T15:13:38.544708+0000","last_peered":"2024-01-24T15:13:38.544708+0000","last_clean":"2024-01-24T15:13:38.544708+0000","last_became_active":"2024-01-24T15:13:38.121900+0000","last_became_peered":"2024-01-24T15:13:38.121900+0000","last_unstale":"2024-01-24T15:13:38.544708+0000","last_undegraded":"2024-01-24T15:13:38.544708+0000","last_fullsized":"2024-01-24T15:13:38.544708+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550198+0000","last_change":"2024-01-24T15:13:38.546422+0000","last_active":"2024-01-24T15:13:38.550198+0000","last_peered":"2024-01-24T15:13:38.550198+0000","last_clean":"2024-01-24T15:13:38.550198+0000","last_became_active":"2024-01-24T15:13:38.546168+0000","last_became_peered":"2024-01-24T15:13:38.546168+0000","last_unstale":"2024-01-24T15:13:38.550198+0000","last_undegraded":"2024-01-24T15:13:38.550198+0000","last_fullsized":"2024-01-24T15:13:38.550198+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.538459+0000","last_change":"2024-01-24T15:13:38.126860+0000","last_active":"2024-01-24T15:13:38.538459+0000","last_peered":"2024-01-24T15:13:38.538459+0000","last_clean":"2024-01-24T15:13:38.538459+0000","last_became_active":"2024-01-24T15:13:38.126528+0000","last_became_peered":"2024-01-24T15:13:38.126528+0000","last_unstale":"2024-01-24T15:13:38.538459+0000","last_undegraded":"2024-01-24T15:13:38.538459+0000","last_fullsized":"2024-01-24T15:13:38.538459+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558013+0000","last_change":"2024-01-24T15:13:38.551200+0000","last_active":"2024-01-24T15:13:38.558013+0000","last_peered":"2024-01-24T15:13:38.558013+0000","last_clean":"2024-01-24T15:13:38.558013+0000","last_became_active":"2024-01-24T15:13:38.550887+0000","last_became_peered":"2024-01-24T15:13:38.550887+0000","last_unstale":"2024-01-24T15:13:38.558013+0000","last_undegraded":"2024-01-24T15:13:38.558013+0000","last_fullsized":"2024-01-24T15:13:38.558013+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559181+0000","last_change":"2024-01-24T15:13:38.135912+0000","last_active":"2024-01-24T15:13:38.559181+0000","last_peered":"2024-01-24T15:13:38.559181+0000","last_clean":"2024-01-24T15:13:38.559181+0000","last_became_active":"2024-01-24T15:13:38.135637+0000","last_became_peered":"2024-01-24T15:13:38.135637+0000","last_unstale":"2024-01-24T15:13:38.559181+0000","last_undegraded":"2024-01-24T15:13:38.559181+0000","last_fullsized":"2024-01-24T15:13:38.559181+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.547990+0000","last_change":"2024-01-24T15:13:38.542686+0000","last_active":"2024-01-24T15:13:38.547990+0000","last_peered":"2024-01-24T15:13:38.547990+0000","last_clean":"2024-01-24T15:13:38.547990+0000","last_became_active":"2024-01-24T15:13:38.542392+0000","last_became_peered":"2024-01-24T15:13:38.542392+0000","last_unstale":"2024-01-24T15:13:38.547990+0000","last_undegraded":"2024-01-24T15:13:38.547990+0000","last_fullsized":"2024-01-24T15:13:38.547990+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.548937+0000","last_change":"2024-01-24T15:13:38.125904+0000","last_active":"2024-01-24T15:13:38.548937+0000","last_peered":"2024-01-24T15:13:38.548937+0000","last_clean":"2024-01-24T15:13:38.548937+0000","last_became_active":"2024-01-24T15:13:38.125609+0000","last_became_peered":"2024-01-24T15:13:38.125609+0000","last_unstale":"2024-01-24T15:13:38.548937+0000","last_undegraded":"2024-01-24T15:13:38.548937+0000","last_fullsized":"2024-01-24T15:13:38.548937+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557762+0000","last_change":"2024-01-24T15:13:38.553959+0000","last_active":"2024-01-24T15:13:38.557762+0000","last_peered":"2024-01-24T15:13:38.557762+0000","last_clean":"2024-01-24T15:13:38.557762+0000","last_became_active":"2024-01-24T15:13:38.553651+0000","last_became_peered":"2024-01-24T15:13:38.553651+0000","last_unstale":"2024-01-24T15:13:38.557762+0000","last_undegraded":"2024-01-24T15:13:38.557762+0000","last_fullsized":"2024-01-24T15:13:38.557762+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557845+0000","last_change":"2024-01-24T15:13:38.129150+0000","last_active":"2024-01-24T15:13:38.557845+0000","last_peered":"2024-01-24T15:13:38.557845+0000","last_clean":"2024-01-24T15:13:38.557845+0000","last_became_active":"2024-01-24T15:13:38.128848+0000","last_became_peered":"2024-01-24T15:13:38.128848+0000","last_unstale":"2024-01-24T15:13:38.557845+0000","last_undegraded":"2024-01-24T15:13:38.557845+0000","last_fullsized":"2024-01-24T15:13:38.557845+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558317+0000","last_change":"2024-01-24T15:13:38.555479+0000","last_active":"2024-01-24T15:13:38.558317+0000","last_peered":"2024-01-24T15:13:38.558317+0000","last_clean":"2024-01-24T15:13:38.558317+0000","last_became_active":"2024-01-24T15:13:38.555204+0000","last_became_peered":"2024-01-24T15:13:38.555204+0000","last_unstale":"2024-01-24T15:13:38.558317+0000","last_undegraded":"2024-01-24T15:13:38.558317+0000","last_fullsized":"2024-01-24T15:13:38.558317+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.545284+0000","last_change":"2024-01-24T15:13:38.122342+0000","last_active":"2024-01-24T15:13:38.545284+0000","last_peered":"2024-01-24T15:13:38.545284+0000","last_clean":"2024-01-24T15:13:38.545284+0000","last_became_active":"2024-01-24T15:13:38.122043+0000","last_became_peered":"2024-01-24T15:13:38.122043+0000","last_unstale":"2024-01-24T15:13:38.545284+0000","last_undegraded":"2024-01-24T15:13:38.545284+0000","last_fullsized":"2024-01-24T15:13:38.545284+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550294+0000","last_change":"2024-01-24T15:13:38.547021+0000","last_active":"2024-01-24T15:13:38.550294+0000","last_peered":"2024-01-24T15:13:38.550294+0000","last_clean":"2024-01-24T15:13:38.550294+0000","last_became_active":"2024-01-24T15:13:38.546231+0000","last_became_peered":"2024-01-24T15:13:38.546231+0000","last_unstale":"2024-01-24T15:13:38.550294+0000","last_undegraded":"2024-01-24T15:13:38.550294+0000","last_fullsized":"2024-01-24T15:13:38.550294+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528666+0000","last_change":"2024-01-24T15:13:38.124808+0000","last_active":"2024-01-24T15:13:38.528666+0000","last_peered":"2024-01-24T15:13:38.528666+0000","last_clean":"2024-01-24T15:13:38.528666+0000","last_became_active":"2024-01-24T15:13:38.124504+0000","last_became_peered":"2024-01-24T15:13:38.124504+0000","last_unstale":"2024-01-24T15:13:38.528666+0000","last_undegraded":"2024-01-24T15:13:38.528666+0000","last_fullsized":"2024-01-24T15:13:38.528666+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562306+0000","last_change":"2024-01-24T15:13:38.558977+0000","last_active":"2024-01-24T15:13:38.562306+0000","last_peered":"2024-01-24T15:13:38.562306+0000","last_clean":"2024-01-24T15:13:38.562306+0000","last_became_active":"2024-01-24T15:13:38.558713+0000","last_became_peered":"2024-01-24T15:13:38.558713+0000","last_unstale":"2024-01-24T15:13:38.562306+0000","last_undegraded":"2024-01-24T15:13:38.562306+0000","last_fullsized":"2024-01-24T15:13:38.562306+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533932+0000","last_change":"2024-01-24T15:13:38.125055+0000","last_active":"2024-01-24T15:13:38.533932+0000","last_peered":"2024-01-24T15:13:38.533932+0000","last_clean":"2024-01-24T15:13:38.533932+0000","last_became_active":"2024-01-24T15:13:38.124694+0000","last_became_peered":"2024-01-24T15:13:38.124694+0000","last_unstale":"2024-01-24T15:13:38.533932+0000","last_undegraded":"2024-01-24T15:13:38.533932+0000","last_fullsized":"2024-01-24T15:13:38.533932+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542376+0000","last_change":"2024-01-24T15:13:38.540674+0000","last_active":"2024-01-24T15:13:38.542376+0000","last_peered":"2024-01-24T15:13:38.542376+0000","last_clean":"2024-01-24T15:13:38.542376+0000","last_became_active":"2024-01-24T15:13:38.540381+0000","last_became_peered":"2024-01-24T15:13:38.540381+0000","last_unstale":"2024-01-24T15:13:38.542376+0000","last_undegraded":"2024-01-24T15:13:38.542376+0000","last_fullsized":"2024-01-24T15:13:38.542376+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528151+0000","last_change":"2024-01-24T15:13:38.119586+0000","last_active":"2024-01-24T15:13:38.528151+0000","last_peered":"2024-01-24T15:13:38.528151+0000","last_clean":"2024-01-24T15:13:38.528151+0000","last_became_active":"2024-01-24T15:13:38.119285+0000","last_became_peered":"2024-01-24T15:13:38.119285+0000","last_unstale":"2024-01-24T15:13:38.528151+0000","last_undegraded":"2024-01-24T15:13:38.528151+0000","last_fullsized":"2024-01-24T15:13:38.528151+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551159+0000","last_change":"2024-01-24T15:13:38.544043+0000","last_active":"2024-01-24T15:13:38.551159+0000","last_peered":"2024-01-24T15:13:38.551159+0000","last_clean":"2024-01-24T15:13:38.551159+0000","last_became_active":"2024-01-24T15:13:38.543741+0000","last_became_peered":"2024-01-24T15:13:38.543741+0000","last_unstale":"2024-01-24T15:13:38.551159+0000","last_undegraded":"2024-01-24T15:13:38.551159+0000","last_fullsized":"2024-01-24T15:13:38.551159+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.544453+0000","last_change":"2024-01-24T15:13:38.124704+0000","last_active":"2024-01-24T15:13:38.544453+0000","last_peered":"2024-01-24T15:13:38.544453+0000","last_clean":"2024-01-24T15:13:38.544453+0000","last_became_active":"2024-01-24T15:13:38.124427+0000","last_became_peered":"2024-01-24T15:13:38.124427+0000","last_unstale":"2024-01-24T15:13:38.544453+0000","last_undegraded":"2024-01-24T15:13:38.544453+0000","last_fullsized":"2024-01-24T15:13:38.544453+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559264+0000","last_change":"2024-01-24T15:13:38.555038+0000","last_active":"2024-01-24T15:13:38.559264+0000","last_peered":"2024-01-24T15:13:38.559264+0000","last_clean":"2024-01-24T15:13:38.559264+0000","last_became_active":"2024-01-24T15:13:38.554763+0000","last_became_peered":"2024-01-24T15:13:38.554763+0000","last_unstale":"2024-01-24T15:13:38.559264+0000","last_undegraded":"2024-01-24T15:13:38.559264+0000","last_fullsized":"2024-01-24T15:13:38.559264+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559251+0000","last_change":"2024-01-24T15:13:38.122389+0000","last_active":"2024-01-24T15:13:38.559251+0000","last_peered":"2024-01-24T15:13:38.559251+0000","last_clean":"2024-01-24T15:13:38.559251+0000","last_became_active":"2024-01-24T15:13:38.121569+0000","last_became_peered":"2024-01-24T15:13:38.121569+0000","last_unstale":"2024-01-24T15:13:38.559251+0000","last_undegraded":"2024-01-24T15:13:38.559251+0000","last_fullsized":"2024-01-24T15:13:38.559251+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557360+0000","last_change":"2024-01-24T15:13:38.549783+0000","last_active":"2024-01-24T15:13:38.557360+0000","last_peered":"2024-01-24T15:13:38.557360+0000","last_clean":"2024-01-24T15:13:38.557360+0000","last_became_active":"2024-01-24T15:13:38.549374+0000","last_became_peered":"2024-01-24T15:13:38.549374+0000","last_unstale":"2024-01-24T15:13:38.557360+0000","last_undegraded":"2024-01-24T15:13:38.557360+0000","last_fullsized":"2024-01-24T15:13:38.557360+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557933+0000","last_change":"2024-01-24T15:13:38.121798+0000","last_active":"2024-01-24T15:13:38.557933+0000","last_peered":"2024-01-24T15:13:38.557933+0000","last_clean":"2024-01-24T15:13:38.557933+0000","last_became_active":"2024-01-24T15:13:38.121333+0000","last_became_peered":"2024-01-24T15:13:38.121333+0000","last_unstale":"2024-01-24T15:13:38.557933+0000","last_undegraded":"2024-01-24T15:13:38.557933+0000","last_fullsized":"2024-01-24T15:13:38.557933+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558881+0000","last_change":"2024-01-24T15:13:38.556149+0000","last_active":"2024-01-24T15:13:38.558881+0000","last_peered":"2024-01-24T15:13:38.558881+0000","last_clean":"2024-01-24T15:13:38.558881+0000","last_became_active":"2024-01-24T15:13:38.553256+0000","last_became_peered":"2024-01-24T15:13:38.553256+0000","last_unstale":"2024-01-24T15:13:38.558881+0000","last_undegraded":"2024-01-24T15:13:38.558881+0000","last_fullsized":"2024-01-24T15:13:38.558881+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.529610+0000","last_change":"2024-01-24T15:13:38.125875+0000","last_active":"2024-01-24T15:13:38.529610+0000","last_peered":"2024-01-24T15:13:38.529610+0000","last_clean":"2024-01-24T15:13:38.529610+0000","last_became_active":"2024-01-24T15:13:38.125536+0000","last_became_peered":"2024-01-24T15:13:38.125536+0000","last_unstale":"2024-01-24T15:13:38.529610+0000","last_undegraded":"2024-01-24T15:13:38.529610+0000","last_fullsized":"2024-01-24T15:13:38.529610+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.555881+0000","last_change":"2024-01-24T15:13:38.551671+0000","last_active":"2024-01-24T15:13:38.555881+0000","last_peered":"2024-01-24T15:13:38.555881+0000","last_clean":"2024-01-24T15:13:38.555881+0000","last_became_active":"2024-01-24T15:13:38.550779+0000","last_became_peered":"2024-01-24T15:13:38.550779+0000","last_unstale":"2024-01-24T15:13:38.555881+0000","last_undegraded":"2024-01-24T15:13:38.555881+0000","last_fullsized":"2024-01-24T15:13:38.555881+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532219+0000","last_change":"2024-01-24T15:13:38.122494+0000","last_active":"2024-01-24T15:13:38.532219+0000","last_peered":"2024-01-24T15:13:38.532219+0000","last_clean":"2024-01-24T15:13:38.532219+0000","last_became_active":"2024-01-24T15:13:38.122120+0000","last_became_peered":"2024-01-24T15:13:38.122120+0000","last_unstale":"2024-01-24T15:13:38.532219+0000","last_undegraded":"2024-01-24T15:13:38.532219+0000","last_fullsized":"2024-01-24T15:13:38.532219+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.560704+0000","last_change":"2024-01-24T15:13:38.554397+0000","last_active":"2024-01-24T15:13:38.560704+0000","last_peered":"2024-01-24T15:13:38.560704+0000","last_clean":"2024-01-24T15:13:38.560704+0000","last_became_active":"2024-01-24T15:13:38.553773+0000","last_became_peered":"2024-01-24T15:13:38.553773+0000","last_unstale":"2024-01-24T15:13:38.560704+0000","last_undegraded":"2024-01-24T15:13:38.560704+0000","last_fullsized":"2024-01-24T15:13:38.560704+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561628+0000","last_change":"2024-01-24T15:13:38.129725+0000","last_active":"2024-01-24T15:13:38.561628+0000","last_peered":"2024-01-24T15:13:38.561628+0000","last_clean":"2024-01-24T15:13:38.561628+0000","last_became_active":"2024-01-24T15:13:38.129199+0000","last_became_peered":"2024-01-24T15:13:38.129199+0000","last_unstale":"2024-01-24T15:13:38.561628+0000","last_undegraded":"2024-01-24T15:13:38.561628+0000","last_fullsized":"2024-01-24T15:13:38.561628+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556320+0000","last_change":"2024-01-24T15:13:38.550473+0000","last_active":"2024-01-24T15:13:38.556320+0000","last_peered":"2024-01-24T15:13:38.556320+0000","last_clean":"2024-01-24T15:13:38.556320+0000","last_became_active":"2024-01-24T15:13:38.550227+0000","last_became_peered":"2024-01-24T15:13:38.550227+0000","last_unstale":"2024-01-24T15:13:38.556320+0000","last_undegraded":"2024-01-24T15:13:38.556320+0000","last_fullsized":"2024-01-24T15:13:38.556320+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542174+0000","last_change":"2024-01-24T15:13:38.130818+0000","last_active":"2024-01-24T15:13:38.542174+0000","last_peered":"2024-01-24T15:13:38.542174+0000","last_clean":"2024-01-24T15:13:38.542174+0000","last_became_active":"2024-01-24T15:13:38.130326+0000","last_became_peered":"2024-01-24T15:13:38.130326+0000","last_unstale":"2024-01-24T15:13:38.542174+0000","last_undegraded":"2024-01-24T15:13:38.542174+0000","last_fullsized":"2024-01-24T15:13:38.542174+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562244+0000","last_change":"2024-01-24T15:13:38.557459+0000","last_active":"2024-01-24T15:13:38.562244+0000","last_peered":"2024-01-24T15:13:38.562244+0000","last_clean":"2024-01-24T15:13:38.562244+0000","last_became_active":"2024-01-24T15:13:38.557186+0000","last_became_peered":"2024-01-24T15:13:38.557186+0000","last_unstale":"2024-01-24T15:13:38.562244+0000","last_undegraded":"2024-01-24T15:13:38.562244+0000","last_fullsized":"2024-01-24T15:13:38.562244+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.541879+0000","last_change":"2024-01-24T15:13:38.134878+0000","last_active":"2024-01-24T15:13:38.541879+0000","last_peered":"2024-01-24T15:13:38.541879+0000","last_clean":"2024-01-24T15:13:38.541879+0000","last_became_active":"2024-01-24T15:13:38.134606+0000","last_became_peered":"2024-01-24T15:13:38.134606+0000","last_unstale":"2024-01-24T15:13:38.541879+0000","last_undegraded":"2024-01-24T15:13:38.541879+0000","last_fullsized":"2024-01-24T15:13:38.541879+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.556378+0000","last_change":"2024-01-24T15:13:38.552294+0000","last_active":"2024-01-24T15:13:38.556378+0000","last_peered":"2024-01-24T15:13:38.556378+0000","last_clean":"2024-01-24T15:13:38.556378+0000","last_became_active":"2024-01-24T15:13:38.551919+0000","last_became_peered":"2024-01-24T15:13:38.551919+0000","last_unstale":"2024-01-24T15:13:38.556378+0000","last_undegraded":"2024-01-24T15:13:38.556378+0000","last_fullsized":"2024-01-24T15:13:38.556378+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535737+0000","last_change":"2024-01-24T15:13:38.132176+0000","last_active":"2024-01-24T15:13:38.535737+0000","last_peered":"2024-01-24T15:13:38.535737+0000","last_clean":"2024-01-24T15:13:38.535737+0000","last_became_active":"2024-01-24T15:13:38.131843+0000","last_became_peered":"2024-01-24T15:13:38.131843+0000","last_unstale":"2024-01-24T15:13:38.535737+0000","last_undegraded":"2024-01-24T15:13:38.535737+0000","last_fullsized":"2024-01-24T15:13:38.535737+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551132+0000","last_change":"2024-01-24T15:13:38.547885+0000","last_active":"2024-01-24T15:13:38.551132+0000","last_peered":"2024-01-24T15:13:38.551132+0000","last_clean":"2024-01-24T15:13:38.551132+0000","last_became_active":"2024-01-24T15:13:38.547529+0000","last_became_peered":"2024-01-24T15:13:38.547529+0000","last_unstale":"2024-01-24T15:13:38.551132+0000","last_undegraded":"2024-01-24T15:13:38.551132+0000","last_fullsized":"2024-01-24T15:13:38.551132+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559494+0000","last_change":"2024-01-24T15:13:38.134325+0000","last_active":"2024-01-24T15:13:38.559494+0000","last_peered":"2024-01-24T15:13:38.559494+0000","last_clean":"2024-01-24T15:13:38.559494+0000","last_became_active":"2024-01-24T15:13:38.134045+0000","last_became_peered":"2024-01-24T15:13:38.134045+0000","last_unstale":"2024-01-24T15:13:38.559494+0000","last_undegraded":"2024-01-24T15:13:38.559494+0000","last_fullsized":"2024-01-24T15:13:38.559494+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558867+0000","last_change":"2024-01-24T15:13:38.554889+0000","last_active":"2024-01-24T15:13:38.558867+0000","last_peered":"2024-01-24T15:13:38.558867+0000","last_clean":"2024-01-24T15:13:38.558867+0000","last_became_active":"2024-01-24T15:13:38.554624+0000","last_became_peered":"2024-01-24T15:13:38.554624+0000","last_unstale":"2024-01-24T15:13:38.558867+0000","last_undegraded":"2024-01-24T15:13:38.558867+0000","last_fullsized":"2024-01-24T15:13:38.558867+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537614+0000","last_change":"2024-01-24T15:13:38.126218+0000","last_active":"2024-01-24T15:13:38.537614+0000","last_peered":"2024-01-24T15:13:38.537614+0000","last_clean":"2024-01-24T15:13:38.537614+0000","last_became_active":"2024-01-24T15:13:38.125899+0000","last_became_peered":"2024-01-24T15:13:38.125899+0000","last_unstale":"2024-01-24T15:13:38.537614+0000","last_undegraded":"2024-01-24T15:13:38.537614+0000","last_fullsized":"2024-01-24T15:13:38.537614+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561397+0000","last_change":"2024-01-24T15:13:38.557841+0000","last_active":"2024-01-24T15:13:38.561397+0000","last_peered":"2024-01-24T15:13:38.561397+0000","last_clean":"2024-01-24T15:13:38.561397+0000","last_became_active":"2024-01-24T15:13:38.557543+0000","last_became_peered":"2024-01-24T15:13:38.557543+0000","last_unstale":"2024-01-24T15:13:38.561397+0000","last_undegraded":"2024-01-24T15:13:38.561397+0000","last_fullsized":"2024-01-24T15:13:38.561397+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.538204+0000","last_change":"2024-01-24T15:13:38.122661+0000","last_active":"2024-01-24T15:13:38.538204+0000","last_peered":"2024-01-24T15:13:38.538204+0000","last_clean":"2024-01-24T15:13:38.538204+0000","last_became_active":"2024-01-24T15:13:38.122384+0000","last_became_peered":"2024-01-24T15:13:38.122384+0000","last_unstale":"2024-01-24T15:13:38.538204+0000","last_undegraded":"2024-01-24T15:13:38.538204+0000","last_fullsized":"2024-01-24T15:13:38.538204+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557058+0000","last_change":"2024-01-24T15:13:38.553790+0000","last_active":"2024-01-24T15:13:38.557058+0000","last_peered":"2024-01-24T15:13:38.557058+0000","last_clean":"2024-01-24T15:13:38.557058+0000","last_became_active":"2024-01-24T15:13:38.553498+0000","last_became_peered":"2024-01-24T15:13:38.553498+0000","last_unstale":"2024-01-24T15:13:38.557058+0000","last_undegraded":"2024-01-24T15:13:38.557058+0000","last_fullsized":"2024-01-24T15:13:38.557058+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535549+0000","last_change":"2024-01-24T15:13:38.120324+0000","last_active":"2024-01-24T15:13:38.535549+0000","last_peered":"2024-01-24T15:13:38.535549+0000","last_clean":"2024-01-24T15:13:38.535549+0000","last_became_active":"2024-01-24T15:13:38.120007+0000","last_became_peered":"2024-01-24T15:13:38.120007+0000","last_unstale":"2024-01-24T15:13:38.535549+0000","last_undegraded":"2024-01-24T15:13:38.535549+0000","last_fullsized":"2024-01-24T15:13:38.535549+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558444+0000","last_change":"2024-01-24T15:13:38.552923+0000","last_active":"2024-01-24T15:13:38.558444+0000","last_peered":"2024-01-24T15:13:38.558444+0000","last_clean":"2024-01-24T15:13:38.558444+0000","last_became_active":"2024-01-24T15:13:38.552646+0000","last_became_peered":"2024-01-24T15:13:38.552646+0000","last_unstale":"2024-01-24T15:13:38.558444+0000","last_undegraded":"2024-01-24T15:13:38.558444+0000","last_fullsized":"2024-01-24T15:13:38.558444+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.539981+0000","last_change":"2024-01-24T15:13:38.129534+0000","last_active":"2024-01-24T15:13:38.539981+0000","last_peered":"2024-01-24T15:13:38.539981+0000","last_clean":"2024-01-24T15:13:38.539981+0000","last_became_active":"2024-01-24T15:13:38.128874+0000","last_became_peered":"2024-01-24T15:13:38.128874+0000","last_unstale":"2024-01-24T15:13:38.539981+0000","last_undegraded":"2024-01-24T15:13:38.539981+0000","last_fullsized":"2024-01-24T15:13:38.539981+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.551365+0000","last_change":"2024-01-24T15:13:38.549194+0000","last_active":"2024-01-24T15:13:38.551365+0000","last_peered":"2024-01-24T15:13:38.551365+0000","last_clean":"2024-01-24T15:13:38.551365+0000","last_became_active":"2024-01-24T15:13:38.548929+0000","last_became_peered":"2024-01-24T15:13:38.548929+0000","last_unstale":"2024-01-24T15:13:38.551365+0000","last_undegraded":"2024-01-24T15:13:38.551365+0000","last_fullsized":"2024-01-24T15:13:38.551365+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.542276+0000","last_change":"2024-01-24T15:13:38.130595+0000","last_active":"2024-01-24T15:13:38.542276+0000","last_peered":"2024-01-24T15:13:38.542276+0000","last_clean":"2024-01-24T15:13:38.542276+0000","last_became_active":"2024-01-24T15:13:38.130025+0000","last_became_peered":"2024-01-24T15:13:38.130025+0000","last_unstale":"2024-01-24T15:13:38.542276+0000","last_undegraded":"2024-01-24T15:13:38.542276+0000","last_fullsized":"2024-01-24T15:13:38.542276+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.557093+0000","last_change":"2024-01-24T15:13:38.548613+0000","last_active":"2024-01-24T15:13:38.557093+0000","last_peered":"2024-01-24T15:13:38.557093+0000","last_clean":"2024-01-24T15:13:38.557093+0000","last_became_active":"2024-01-24T15:13:38.548288+0000","last_became_peered":"2024-01-24T15:13:38.548288+0000","last_unstale":"2024-01-24T15:13:38.557093+0000","last_undegraded":"2024-01-24T15:13:38.557093+0000","last_fullsized":"2024-01-24T15:13:38.557093+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535292+0000","last_change":"2024-01-24T15:13:38.131770+0000","last_active":"2024-01-24T15:13:38.535292+0000","last_peered":"2024-01-24T15:13:38.535292+0000","last_clean":"2024-01-24T15:13:38.535292+0000","last_became_active":"2024-01-24T15:13:38.131497+0000","last_became_peered":"2024-01-24T15:13:38.131497+0000","last_unstale":"2024-01-24T15:13:38.535292+0000","last_undegraded":"2024-01-24T15:13:38.535292+0000","last_fullsized":"2024-01-24T15:13:38.535292+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.546235+0000","last_change":"2024-01-24T15:13:38.538155+0000","last_active":"2024-01-24T15:13:38.546235+0000","last_peered":"2024-01-24T15:13:38.546235+0000","last_clean":"2024-01-24T15:13:38.546235+0000","last_became_active":"2024-01-24T15:13:38.537877+0000","last_became_peered":"2024-01-24T15:13:38.537877+0000","last_unstale":"2024-01-24T15:13:38.546235+0000","last_undegraded":"2024-01-24T15:13:38.546235+0000","last_fullsized":"2024-01-24T15:13:38.546235+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.531693+0000","last_change":"2024-01-24T15:13:38.124495+0000","last_active":"2024-01-24T15:13:38.531693+0000","last_peered":"2024-01-24T15:13:38.531693+0000","last_clean":"2024-01-24T15:13:38.531693+0000","last_became_active":"2024-01-24T15:13:38.124175+0000","last_became_peered":"2024-01-24T15:13:38.124175+0000","last_unstale":"2024-01-24T15:13:38.531693+0000","last_undegraded":"2024-01-24T15:13:38.531693+0000","last_fullsized":"2024-01-24T15:13:38.531693+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559906+0000","last_change":"2024-01-24T15:13:38.557558+0000","last_active":"2024-01-24T15:13:38.559906+0000","last_peered":"2024-01-24T15:13:38.559906+0000","last_clean":"2024-01-24T15:13:38.559906+0000","last_became_active":"2024-01-24T15:13:38.557290+0000","last_became_peered":"2024-01-24T15:13:38.557290+0000","last_unstale":"2024-01-24T15:13:38.559906+0000","last_undegraded":"2024-01-24T15:13:38.559906+0000","last_fullsized":"2024-01-24T15:13:38.559906+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535494+0000","last_change":"2024-01-24T15:13:38.121901+0000","last_active":"2024-01-24T15:13:38.535494+0000","last_peered":"2024-01-24T15:13:38.535494+0000","last_clean":"2024-01-24T15:13:38.535494+0000","last_became_active":"2024-01-24T15:13:38.121422+0000","last_became_peered":"2024-01-24T15:13:38.121422+0000","last_unstale":"2024-01-24T15:13:38.535494+0000","last_undegraded":"2024-01-24T15:13:38.535494+0000","last_fullsized":"2024-01-24T15:13:38.535494+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":144,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535646+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:13:38.535646+0000","last_peered":"2024-01-24T15:13:38.535646+0000","last_clean":"2024-01-24T15:13:38.535646+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:13:38.535646+0000","last_undegraded":"2024-01-24T15:13:38.535646+0000","last_fullsized":"2024-01-24T15:13:38.535646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558463+0000","last_change":"2024-01-24T15:13:38.552222+0000","last_active":"2024-01-24T15:13:38.558463+0000","last_peered":"2024-01-24T15:13:38.558463+0000","last_clean":"2024-01-24T15:13:38.558463+0000","last_became_active":"2024-01-24T15:13:38.551916+0000","last_became_peered":"2024-01-24T15:13:38.551916+0000","last_unstale":"2024-01-24T15:13:38.558463+0000","last_undegraded":"2024-01-24T15:13:38.558463+0000","last_fullsized":"2024-01-24T15:13:38.558463+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561816+0000","last_change":"2024-01-24T15:13:38.128196+0000","last_active":"2024-01-24T15:13:38.561816+0000","last_peered":"2024-01-24T15:13:38.561816+0000","last_clean":"2024-01-24T15:13:38.561816+0000","last_became_active":"2024-01-24T15:13:38.127782+0000","last_became_peered":"2024-01-24T15:13:38.127782+0000","last_unstale":"2024-01-24T15:13:38.561816+0000","last_undegraded":"2024-01-24T15:13:38.561816+0000","last_fullsized":"2024-01-24T15:13:38.561816+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":162,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.534014+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:13:38.534014+0000","last_peered":"2024-01-24T15:13:38.534014+0000","last_clean":"2024-01-24T15:13:38.534014+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:13:38.534014+0000","last_undegraded":"2024-01-24T15:13:38.534014+0000","last_fullsized":"2024-01-24T15:13:38.534014+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.547053+0000","last_change":"2024-01-24T15:13:38.538061+0000","last_active":"2024-01-24T15:13:38.547053+0000","last_peered":"2024-01-24T15:13:38.547053+0000","last_clean":"2024-01-24T15:13:38.547053+0000","last_became_active":"2024-01-24T15:13:38.537769+0000","last_became_peered":"2024-01-24T15:13:38.537769+0000","last_unstale":"2024-01-24T15:13:38.547053+0000","last_undegraded":"2024-01-24T15:13:38.547053+0000","last_fullsized":"2024-01-24T15:13:38.547053+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.532992+0000","last_change":"2024-01-24T15:13:38.125400+0000","last_active":"2024-01-24T15:13:38.532992+0000","last_peered":"2024-01-24T15:13:38.532992+0000","last_clean":"2024-01-24T15:13:38.532992+0000","last_became_active":"2024-01-24T15:13:38.124725+0000","last_became_peered":"2024-01-24T15:13:38.124725+0000","last_unstale":"2024-01-24T15:13:38.532992+0000","last_undegraded":"2024-01-24T15:13:38.532992+0000","last_fullsized":"2024-01-24T15:13:38.532992+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.528231+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:13:38.528231+0000","last_peered":"2024-01-24T15:13:38.528231+0000","last_clean":"2024-01-24T15:13:38.528231+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:13:38.528231+0000","last_undegraded":"2024-01-24T15:13:38.528231+0000","last_fullsized":"2024-01-24T15:13:38.528231+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.561686+0000","last_change":"2024-01-24T15:13:38.554780+0000","last_active":"2024-01-24T15:13:38.561686+0000","last_peered":"2024-01-24T15:13:38.561686+0000","last_clean":"2024-01-24T15:13:38.561686+0000","last_became_active":"2024-01-24T15:13:38.554497+0000","last_became_peered":"2024-01-24T15:13:38.554497+0000","last_unstale":"2024-01-24T15:13:38.561686+0000","last_undegraded":"2024-01-24T15:13:38.561686+0000","last_fullsized":"2024-01-24T15:13:38.561686+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"177'3","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558878+0000","last_change":"2024-01-24T15:13:38.131986+0000","last_active":"2024-01-24T15:13:38.558878+0000","last_peered":"2024-01-24T15:13:38.558878+0000","last_clean":"2024-01-24T15:13:38.558878+0000","last_became_active":"2024-01-24T15:13:38.131663+0000","last_became_peered":"2024-01-24T15:13:38.131663+0000","last_unstale":"2024-01-24T15:13:38.558878+0000","last_undegraded":"2024-01-24T15:13:38.558878+0000","last_fullsized":"2024-01-24T15:13:38.558878+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":162,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.533441+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:13:38.533441+0000","last_peered":"2024-01-24T15:13:38.533441+0000","last_clean":"2024-01-24T15:13:38.533441+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:13:38.533441+0000","last_undegraded":"2024-01-24T15:13:38.533441+0000","last_fullsized":"2024-01-24T15:13:38.533441+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":147,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.540654+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:13:38.540654+0000","last_peered":"2024-01-24T15:13:38.540654+0000","last_clean":"2024-01-24T15:13:38.540654+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:13:38.540654+0000","last_undegraded":"2024-01-24T15:13:38.540654+0000","last_fullsized":"2024-01-24T15:13:38.540654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.562314+0000","last_change":"2024-01-24T15:13:38.559088+0000","last_active":"2024-01-24T15:13:38.562314+0000","last_peered":"2024-01-24T15:13:38.562314+0000","last_clean":"2024-01-24T15:13:38.562314+0000","last_became_active":"2024-01-24T15:13:38.558845+0000","last_became_peered":"2024-01-24T15:13:38.558845+0000","last_unstale":"2024-01-24T15:13:38.562314+0000","last_undegraded":"2024-01-24T15:13:38.562314+0000","last_fullsized":"2024-01-24T15:13:38.562314+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.541272+0000","last_change":"2024-01-24T15:13:38.134030+0000","last_active":"2024-01-24T15:13:38.541272+0000","last_peered":"2024-01-24T15:13:38.541272+0000","last_clean":"2024-01-24T15:13:38.541272+0000","last_became_active":"2024-01-24T15:13:38.133632+0000","last_became_peered":"2024-01-24T15:13:38.133632+0000","last_unstale":"2024-01-24T15:13:38.541272+0000","last_undegraded":"2024-01-24T15:13:38.541272+0000","last_fullsized":"2024-01-24T15:13:38.541272+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":145,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.540557+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:13:38.540557+0000","last_peered":"2024-01-24T15:13:38.540557+0000","last_clean":"2024-01-24T15:13:38.540557+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:13:38.540557+0000","last_undegraded":"2024-01-24T15:13:38.540557+0000","last_fullsized":"2024-01-24T15:13:38.540557+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558848+0000","last_change":"2024-01-24T15:13:38.553941+0000","last_active":"2024-01-24T15:13:38.558848+0000","last_peered":"2024-01-24T15:13:38.558848+0000","last_clean":"2024-01-24T15:13:38.558848+0000","last_became_active":"2024-01-24T15:13:38.552844+0000","last_became_peered":"2024-01-24T15:13:38.552844+0000","last_unstale":"2024-01-24T15:13:38.558848+0000","last_undegraded":"2024-01-24T15:13:38.558848+0000","last_fullsized":"2024-01-24T15:13:38.558848+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.559449+0000","last_change":"2024-01-24T15:13:38.134046+0000","last_active":"2024-01-24T15:13:38.559449+0000","last_peered":"2024-01-24T15:13:38.559449+0000","last_clean":"2024-01-24T15:13:38.559449+0000","last_became_active":"2024-01-24T15:13:38.133216+0000","last_became_peered":"2024-01-24T15:13:38.133216+0000","last_unstale":"2024-01-24T15:13:38.559449+0000","last_undegraded":"2024-01-24T15:13:38.559449+0000","last_fullsized":"2024-01-24T15:13:38.559449+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":145,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.543025+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:13:38.543025+0000","last_peered":"2024-01-24T15:13:38.543025+0000","last_clean":"2024-01-24T15:13:38.543025+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:13:38.543025+0000","last_undegraded":"2024-01-24T15:13:38.543025+0000","last_fullsized":"2024-01-24T15:13:38.543025+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.550309+0000","last_change":"2024-01-24T15:13:38.542717+0000","last_active":"2024-01-24T15:13:38.550309+0000","last_peered":"2024-01-24T15:13:38.550309+0000","last_clean":"2024-01-24T15:13:38.550309+0000","last_became_active":"2024-01-24T15:13:38.542424+0000","last_became_peered":"2024-01-24T15:13:38.542424+0000","last_unstale":"2024-01-24T15:13:38.550309+0000","last_undegraded":"2024-01-24T15:13:38.550309+0000","last_fullsized":"2024-01-24T15:13:38.550309+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.549726+0000","last_change":"2024-01-24T15:13:38.121778+0000","last_active":"2024-01-24T15:13:38.549726+0000","last_peered":"2024-01-24T15:13:38.549726+0000","last_clean":"2024-01-24T15:13:38.549726+0000","last_became_active":"2024-01-24T15:13:38.121489+0000","last_became_peered":"2024-01-24T15:13:38.121489+0000","last_unstale":"2024-01-24T15:13:38.549726+0000","last_undegraded":"2024-01-24T15:13:38.549726+0000","last_fullsized":"2024-01-24T15:13:38.549726+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537697+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:13:38.537697+0000","last_peered":"2024-01-24T15:13:38.537697+0000","last_clean":"2024-01-24T15:13:38.537697+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:13:38.537697+0000","last_undegraded":"2024-01-24T15:13:38.537697+0000","last_fullsized":"2024-01-24T15:13:38.537697+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.558185+0000","last_change":"2024-01-24T15:13:38.550725+0000","last_active":"2024-01-24T15:13:38.558185+0000","last_peered":"2024-01-24T15:13:38.558185+0000","last_clean":"2024-01-24T15:13:38.558185+0000","last_became_active":"2024-01-24T15:13:38.550442+0000","last_became_peered":"2024-01-24T15:13:38.550442+0000","last_unstale":"2024-01-24T15:13:38.558185+0000","last_undegraded":"2024-01-24T15:13:38.558185+0000","last_fullsized":"2024-01-24T15:13:38.558185+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:38.093739+0000","last_clean_scrub_stamp":"2024-01-24T15:13:38.093739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.535563+0000","last_change":"2024-01-24T15:13:38.131236+0000","last_active":"2024-01-24T15:13:38.535563+0000","last_peered":"2024-01-24T15:13:38.535563+0000","last_clean":"2024-01-24T15:13:38.535563+0000","last_became_active":"2024-01-24T15:13:38.130890+0000","last_became_peered":"2024-01-24T15:13:38.130890+0000","last_unstale":"2024-01-24T15:13:38.535563+0000","last_undegraded":"2024-01-24T15:13:38.535563+0000","last_fullsized":"2024-01-24T15:13:38.535563+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:13:37.088603+0000","last_clean_scrub_stamp":"2024-01-24T15:13:37.088603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-01-24T15:13:38.537474+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:13:38.537474+0000","last_peered":"2024-01-24T15:13:38.537474+0000","last_clean":"2024-01-24T15:13:38.537474+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:13:38.537474+0000","last_undegraded":"2024-01-24T15:13:38.537474+0000","last_fullsized":"2024-01-24T15:13:38.537474+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575752,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303360,"kb_used_data":2168,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068480,"statfs":{"total":96636764160,"available":96326123520,"internally_reserved":0,"allocated":2220032,"data_stored":1270917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.59899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.28000000000000003},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.628,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.28899999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.1080000000000001},"last":0.53900000000000003}]},{"osd":1,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.33300000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.505,"5min":0.40000000000000002,"15min":0.34799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61899999999999999}]},{"osd":2,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0329999999999999},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.38},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0880000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.41199999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.59799999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.33200000000000002},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.31900000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.71499999999999997}]},{"osd":5,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.39600000000000002,"5min":0.38,"15min":0.34799999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.36199999999999999},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.82199999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.31900000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.747},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.46400000000000002,"5min":0.40899999999999997,"15min":0.40500000000000003},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.86799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575753,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307464,"kb_used_data":2176,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064376,"statfs":{"total":96636764160,"available":96321921024,"internally_reserved":0,"allocated":2228224,"data_stored":1270993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.86599999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.42699999999999999,"15min":0.37},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57599999999999996}]},{"osd":1,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.84199999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77100000000000002,"5min":0.81200000000000006,"15min":1.258},"last":0.55900000000000005}]},{"osd":2,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.40100000000000002},"max":{"1min":0.82299999999999995,"5min":0.85899999999999999,"15min":1.2430000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.47199999999999998,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.39000000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.317},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32400000000000001,"15min":0.30499999999999999},"max":{"1min":0.79300000000000004,"5min":0.80100000000000005,"15min":1.002},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.32500000000000001},"max":{"1min":0.76000000000000001,"5min":0.78100000000000003,"15min":0.83899999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":0.749,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61899999999999999}]},{"osd":7,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.40899999999999997,"5min":0.372,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.90100000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.38},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":1.0429999999999999},"last":0.70799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575752,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307452,"kb_used_data":2164,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064388,"statfs":{"total":96636764160,"available":96321933312,"internally_reserved":0,"allocated":2215936,"data_stored":1270847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.56399999999999995,"5min":0.754,"15min":0.754},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.443,"15min":0.435},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.26800000000000002},"max":{"1min":0.55800000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34100000000000003}]},{"osd":2,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.33700000000000002,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.57599999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44600000000000001,"15min":0.44700000000000001},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.28199999999999997},"max":{"1min":0.56200000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"min":{"1min":0.32400000000000001,"5min":0.314,"15min":0.309},"max":{"1min":0.58399999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.35799999999999998,"15min":0.313},"max":{"1min":0.59899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.36599999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.51800000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.86099999999999999,"5min":1.044,"15min":1.0649999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.51400000000000001,"5min":0.47399999999999998,"15min":0.41699999999999998},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.49199999999999999,"15min":0.46300000000000002},"max":{"1min":0.83999999999999997,"5min":1.014,"15min":1.0780000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.439},"max":{"1min":0.86699999999999999,"5min":1.105,"15min":1.105},"last":0.55100000000000005}]},{"osd":6,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.874,"5min":1.0029999999999999,"15min":1.069},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.56599999999999995,"5min":0.499,"15min":0.48199999999999998},"max":{"1min":0.92500000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.56100000000000005}]},{"osd":7,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.83799999999999997,"5min":1.127,"15min":1.127},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.77500000000000002,"15min":0.76700000000000002},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.42799999999999999},"max":{"1min":0.88900000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.59899999999999998}]}]},{"osd":0,"up_from":14,"seq":60129543050,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307464,"kb_used_data":2176,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064376,"statfs":{"total":96636764160,"available":96321921024,"internally_reserved":0,"allocated":2228224,"data_stored":1271118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.28100000000000003,"5min":0.27500000000000002,"15min":0.247},"max":{"1min":0.60199999999999998,"5min":0.63400000000000001,"15min":0.66400000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.27700000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.29399999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.81999999999999995},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.84399999999999997},"last":0.52500000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.44600000000000001},"min":{"1min":0.317,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.65500000000000003,"15min":0.67000000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.52000000000000002},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.90000000000000002},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.433},"max":{"1min":0.99399999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.38700000000000001},"max":{"1min":0.995,"5min":0.995,"15min":1.0249999999999999},"last":0.81200000000000006}]},{"osd":5,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.70699999999999996},"min":{"1min":0.54900000000000004,"5min":0.498,"15min":0.42099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.46899999999999997},"max":{"1min":1.022,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0820000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.871}]},{"osd":7,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0469999999999999,"5min":1.143,"15min":1.3260000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.375},"last":0.78100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575752,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311564,"kb_used_data":2180,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060276,"statfs":{"total":96636764160,"available":96317722624,"internally_reserved":0,"allocated":2232320,"data_stored":1272937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.27200000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.60099999999999998,"5min":0.65000000000000002,"15min":0.66800000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.45100000000000001,"15min":0.45200000000000001},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.70399999999999996},"last":0.29099999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.25600000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.73599999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.46300000000000002},"min":{"1min":0.32500000000000001,"5min":0.29699999999999999,"15min":0.29599999999999999},"max":{"1min":0.622,"5min":0.69699999999999995,"15min":0.70099999999999996},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.73299999999999998},"last":0.34999999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96199999999999997,"5min":1.196,"15min":1.196},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":1,"15min":1.089},"last":0.38100000000000001}]},{"osd":5,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54400000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.111},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.46899999999999997,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.147,"15min":1.1830000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.64700000000000002,"5min":0.47699999999999998,"15min":0.45200000000000001},"max":{"1min":0.99399999999999999,"5min":0.998,"15min":1.129},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.58899999999999997,"5min":0.47299999999999998,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.98999999999999999,"15min":1.1950000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.753,"15min":0.746},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.497},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.248},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.65000000000000002,"5min":0.51500000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.24,"15min":1.24},"last":0.63100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307460,"kb_used_data":2172,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064380,"statfs":{"total":96636764160,"available":96321925120,"internally_reserved":0,"allocated":2224128,"data_stored":1271829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39800000000000002,"15min":0.39200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.247},"max":{"1min":0.56899999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.70199999999999996},"last":0.29499999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41899999999999998,"15min":0.41399999999999998},"min":{"1min":0.311,"5min":0.28100000000000003,"15min":0.27400000000000002},"max":{"1min":0.59999999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.371},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.34799999999999998,"5min":0.316,"15min":0.30499999999999999},"max":{"1min":0.67900000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.39200000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.497,"15min":0.502},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51300000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42299999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.41099999999999998},"max":{"1min":0.83299999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95999999999999996,"15min":1.5580000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.1879999999999999,"15min":1.6240000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.647},"last":0.68600000000000005}]},{"osd":6,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.45700000000000002,"15min":0.42699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.522},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.073,"5min":1.079,"15min":1.079},"last":0.61399999999999999}]},{"osd":7,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.44800000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1499999999999999,"5min":1.246,"15min":1.246},"last":0.71199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575753,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307456,"kb_used_data":2168,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064384,"statfs":{"total":96636764160,"available":96321929216,"internally_reserved":0,"allocated":2220032,"data_stored":1270917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.62},"min":{"1min":0.34899999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.84199999999999997},"last":0.624}]},{"osd":1,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.82499999999999996,"15min":0.82599999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.69799999999999995}]},{"osd":2,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.35799999999999998},"max":{"1min":0.82099999999999995,"5min":0.84699999999999998,"15min":0.88300000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.372},"max":{"1min":0.81499999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.38400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.89500000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.50900000000000001,"5min":0.46800000000000003,"15min":0.42399999999999999},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.92300000000000004},"last":0.68799999999999994}]},{"osd":5,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.36099999999999999,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.81799999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51500000000000001,"15min":0.51600000000000001},"min":{"1min":0.40899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65800000000000003,"5min":0.72499999999999998,"15min":0.85499999999999998},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.73699999999999999,"5min":0.78500000000000003,"15min":0.85499999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.86799999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55300000000000005,"15min":0.55200000000000005},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.33600000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.82799999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.96599999999999997},"last":0.63500000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575753,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307456,"kb_used_data":2168,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064384,"statfs":{"total":96636764160,"available":96321929216,"internally_reserved":0,"allocated":2220032,"data_stored":1270869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.38600000000000001,"15min":0.35099999999999998},"max":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":1.4470000000000001},"last":0.622},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.81599999999999995},"last":0.70199999999999996}]},{"osd":1,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.82499999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":1.4930000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.625,"15min":0.627},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.79300000000000004,"5min":0.80900000000000005,"15min":0.86499999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":1.53},"last":0.68400000000000005}]},{"osd":3,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.80800000000000005,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41899999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.63500000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.65400000000000003,"5min":0.74399999999999999,"15min":0.84699999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":0.77200000000000002,"15min":0.80100000000000005},"last":0.53200000000000003}]},{"osd":6,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.5,"15min":0.504},"min":{"1min":0.374,"5min":0.35499999999999998,"15min":0.33400000000000002},"max":{"1min":0.65100000000000002,"5min":0.79400000000000004,"15min":0.81499999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.78500000000000003,"15min":0.81299999999999994},"last":0.56899999999999995}]},{"osd":7,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.54900000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.497},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:13:47.601 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-24T15:13:47.601 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-24T15:13:47.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.772+0000 7fd6e22dd700 1 Processor -- start 2024-01-24T15:13:47.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.772+0000 7fd6e22dd700 1 -- start start 2024-01-24T15:13:47.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.773+0000 7fd6e22dd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6dc14e5f0 0x7fd6dc065f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.773+0000 7fd6e22dd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd6dc0c3e60 0x7fd6dc0c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.773+0000 7fd6e22dd700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6dc155230 con 0x7fd6dc0627b0 2024-01-24T15:13:47.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.773+0000 7fd6e22dd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6dc06a4b0 con 0x7fd6dc0c3e60 2024-01-24T15:13:47.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.773+0000 7fd6e22dd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc154640 con 0x7fd6dc14e5f0 2024-01-24T15:13:47.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.774+0000 7fd6db7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd6dc0c3e60 0x7fd6dc0c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.774+0000 7fd6dbfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6dc14e5f0 0x7fd6dc065f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.774+0000 7fd6db7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd6dc0c3e60 0x7fd6dc0c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:34106/0 (socket says 172.21.15.29:34106) 2024-01-24T15:13:47.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.774+0000 7fd6dbfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6dc14e5f0 0x7fd6dc065f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:46380/0 (socket says 172.21.15.29:46380) 2024-01-24T15:13:47.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.774+0000 7fd6db7fe700 1 -- 172.21.15.29:0/2176904922 learned_addr learned my addr 172.21.15.29:0/2176904922 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:47.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.774+0000 7fd6db7fe700 1 -- 172.21.15.29:0/2176904922 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6dc14e5f0 msgr2=0x7fd6dc065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:47.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.774+0000 7fd6db7fe700 1 --2- 172.21.15.29:0/2176904922 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6dc14e5f0 0x7fd6dc065f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.774+0000 7fd6db7fe700 1 -- 172.21.15.29:0/2176904922 >> v1:172.21.15.163:6789/0 conn(0x7fd6dc0627b0 legacy=0x7fd6dc0a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:47.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.774+0000 7fd6db7fe700 1 -- 172.21.15.29:0/2176904922 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc154900 con 0x7fd6dc0c3e60 2024-01-24T15:13:47.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.774+0000 7fd6daffd700 1 -- 172.21.15.29:0/2176904922 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003161221 0 0) 0x7fd6dc155230 con 0x7fd6dc0627b0 2024-01-24T15:13:47.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.774+0000 7fd6dbfff700 1 --2- 172.21.15.29:0/2176904922 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6dc14e5f0 0x7fd6dc065f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:47.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.774+0000 7fd6db7fe700 1 --2- 172.21.15.29:0/2176904922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6dc0c3e60 0x7fd6dc0c2260 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fd6c4006fd0 tx=0x7fd6c40090f0).ready entity=mon.1 client_cookie=a7f5e085def9f090 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.775+0000 7fd6daffd700 1 -- 172.21.15.29:0/2176904922 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400b580 con 0x7fd6dc0c3e60 2024-01-24T15:13:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.775+0000 7fd6daffd700 1 -- 172.21.15.29:0/2176904922 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c400b6e0 con 0x7fd6dc0c3e60 2024-01-24T15:13:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.775+0000 7fd6daffd700 1 -- 172.21.15.29:0/2176904922 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400ba60 con 0x7fd6dc0c3e60 2024-01-24T15:13:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.775+0000 7fd6e22dd700 1 -- 172.21.15.29:0/2176904922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6dc0c3e60 msgr2=0x7fd6dc0c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.775+0000 7fd6e22dd700 1 --2- 172.21.15.29:0/2176904922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6dc0c3e60 0x7fd6dc0c2260 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fd6c4006fd0 tx=0x7fd6c40090f0).stop 2024-01-24T15:13:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.775+0000 7fd6e22dd700 1 -- 172.21.15.29:0/2176904922 shutdown_connections 2024-01-24T15:13:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.775+0000 7fd6e22dd700 1 --2- 172.21.15.29:0/2176904922 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6dc14e5f0 0x7fd6dc065f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.775+0000 7fd6e22dd700 1 --2- 172.21.15.29:0/2176904922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6dc0c3e60 0x7fd6dc0c2260 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.775+0000 7fd6e22dd700 1 -- 172.21.15.29:0/2176904922 >> 172.21.15.29:0/2176904922 conn(0x7fd6dc0638c0 msgr2=0x7fd6dc063cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:47.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.775+0000 7fd6e22dd700 1 -- 172.21.15.29:0/2176904922 shutdown_connections 2024-01-24T15:13:47.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.775+0000 7fd6e22dd700 1 -- 172.21.15.29:0/2176904922 wait complete. 2024-01-24T15:13:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.777+0000 7fd6e22dd700 1 Processor -- start 2024-01-24T15:13:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.777+0000 7fd6e22dd700 1 -- start start 2024-01-24T15:13:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6e22dd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6dc0627b0 0x7fd6dc14c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6e22dd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6dc14e5f0 0x7fd6dc14cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6e22dd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6dc0a4900 0x7fd6dc0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6e22dd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0a5250 con 0x7fd6dc14e5f0 2024-01-24T15:13:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6e22dd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc13d0c0 con 0x7fd6dc0627b0 2024-01-24T15:13:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6e22dd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc13d3a0 con 0x7fd6dc0a4900 2024-01-24T15:13:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6db7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6dc14e5f0 0x7fd6dc14cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6db7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6dc14e5f0 0x7fd6dc14cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40558/0 (socket says 172.21.15.29:40558) 2024-01-24T15:13:47.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6db7fe700 1 -- 172.21.15.29:0/3000655637 learned_addr learned my addr 172.21.15.29:0/3000655637 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:47.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6e087a700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6dc0a4900 0x7fd6dc0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6dbfff700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6dc0627b0 0x7fd6dc14c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6db7fe700 1 -- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6dc0a4900 msgr2=0x7fd6dc0a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:47.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6db7fe700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6dc0a4900 0x7fd6dc0a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6db7fe700 1 -- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6dc0627b0 msgr2=0x7fd6dc14c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:47.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6db7fe700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6dc0627b0 0x7fd6dc14c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:47.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6db7fe700 1 -- 172.21.15.29:0/3000655637 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc0092a0 con 0x7fd6dc14e5f0 2024-01-24T15:13:47.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6db7fe700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6dc14e5f0 0x7fd6dc14cda0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7fd6c4009130 tx=0x7fd6c4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:47.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.778+0000 7fd6d3fff700 1 -- 172.21.15.29:0/3000655637 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4000bc0 con 0x7fd6dc14e5f0 2024-01-24T15:13:47.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.779+0000 7fd6d3fff700 1 -- 172.21.15.29:0/3000655637 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c400b850 con 0x7fd6dc14e5f0 2024-01-24T15:13:47.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.779+0000 7fd6e22dd700 1 -- 172.21.15.29:0/3000655637 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c4009370 con 0x7fd6dc14e5f0 2024-01-24T15:13:47.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.779+0000 7fd6d3fff700 1 -- 172.21.15.29:0/3000655637 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4000d20 con 0x7fd6dc14e5f0 2024-01-24T15:13:47.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.779+0000 7fd6e22dd700 1 -- 172.21.15.29:0/3000655637 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6dc13d960 con 0x7fd6dc14e5f0 2024-01-24T15:13:47.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.780+0000 7fd6e22dd700 1 -- 172.21.15.29:0/3000655637 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6dc0475e0 con 0x7fd6dc14e5f0 2024-01-24T15:13:47.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.780+0000 7fd6d3fff700 1 -- 172.21.15.29:0/3000655637 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd6c401cb10 con 0x7fd6dc14e5f0 2024-01-24T15:13:47.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.780+0000 7fd6d3fff700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd6bc05f240 0x7fd6bc061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:47.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.780+0000 7fd6d3fff700 1 -- 172.21.15.29:0/3000655637 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fd6c407c580 con 0x7fd6dc14e5f0 2024-01-24T15:13:47.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.781+0000 7fd6dbfff700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd6bc05f240 0x7fd6bc061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:47.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.781+0000 7fd6dbfff700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd6bc05f240 0x7fd6bc061700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fd6dc0adee0 tx=0x7fd6cc009060).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:47.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:47.785+0000 7fd6d3fff700 1 -- 172.21.15.29:0/3000655637 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd6c404d2a0 con 0x7fd6dc14e5f0 2024-01-24T15:13:48.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.039+0000 7fd6e22dd700 1 -- 172.21.15.29:0/3000655637 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd6dc155f60 con 0x7fd6dc14e5f0 2024-01-24T15:13:48.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.043+0000 7fd6d3fff700 1 -- 172.21.15.29:0/3000655637 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v44) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd6c402a020 con 0x7fd6dc14e5f0 2024-01-24T15:13:48.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.045+0000 7fd6e22dd700 1 -- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd6bc05f240 msgr2=0x7fd6bc061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:48.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.045+0000 7fd6e22dd700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd6bc05f240 0x7fd6bc061700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fd6dc0adee0 tx=0x7fd6cc009060).stop 2024-01-24T15:13:48.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.045+0000 7fd6e22dd700 1 -- 172.21.15.29:0/3000655637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6dc14e5f0 msgr2=0x7fd6dc14cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.045+0000 7fd6e22dd700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6dc14e5f0 0x7fd6dc14cda0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7fd6c4009130 tx=0x7fd6c4017000).stop 2024-01-24T15:13:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.045+0000 7fd6e22dd700 1 -- 172.21.15.29:0/3000655637 shutdown_connections 2024-01-24T15:13:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.045+0000 7fd6e22dd700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6dc0a4900 0x7fd6dc0a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.045+0000 7fd6e22dd700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6dc0627b0 0x7fd6dc14c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.045+0000 7fd6e22dd700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd6bc05f240 0x7fd6bc061700 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.045+0000 7fd6e22dd700 1 --2- 172.21.15.29:0/3000655637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6dc14e5f0 0x7fd6dc14cda0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.045+0000 7fd6e22dd700 1 -- 172.21.15.29:0/3000655637 >> 172.21.15.29:0/3000655637 conn(0x7fd6dc0638c0 msgr2=0x7fd6dc14d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.045+0000 7fd6e22dd700 1 -- 172.21.15.29:0/3000655637 shutdown_connections 2024-01-24T15:13:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.045+0000 7fd6e22dd700 1 -- 172.21.15.29:0/3000655637 wait complete. 2024-01-24T15:13:48.048 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-01-24T15:13:48.061 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:13:48.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.189+0000 7ff29927a700 1 Processor -- start 2024-01-24T15:13:48.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.189+0000 7ff29927a700 1 -- start start 2024-01-24T15:13:48.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.190+0000 7ff29927a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff294064570 0x7ff294064a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:48.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.190+0000 7ff29927a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2940c19e0 0x7ff2940c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:48.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.190+0000 7ff29927a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff294064f10 con 0x7ff294151d90 2024-01-24T15:13:48.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.190+0000 7ff29927a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff294134310 con 0x7ff294064570 2024-01-24T15:13:48.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.190+0000 7ff29927a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff294069d90 con 0x7ff2940c19e0 2024-01-24T15:13:48.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.190+0000 7ff292ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff294151d90 0x7ff2940692f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34720/0 (socket says 172.21.15.29:34720) 2024-01-24T15:13:48.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.190+0000 7ff292ffd700 1 -- 172.21.15.29:0/3188741141 learned_addr learned my addr 172.21.15.29:0/3188741141 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:48.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.190+0000 7ff2937fe700 1 --2- 172.21.15.29:0/3188741141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2940c19e0 0x7ff2940c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:48.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.190+0000 7ff2927fc700 1 --2- 172.21.15.29:0/3188741141 >> v2:172.21.15.163:3300/0 conn(0x7ff294064570 0x7ff294064a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:48.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.190+0000 7ff291ffb700 1 -- 172.21.15.29:0/3188741141 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167888277 0 0) 0x7ff284001110 con 0x7ff294151d90 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.191+0000 7ff291ffb700 1 -- 172.21.15.29:0/3188741141 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff27c0033f0 con 0x7ff294151d90 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.191+0000 7ff2927fc700 1 -- 172.21.15.29:0/3188741141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2940c19e0 msgr2=0x7ff2940c1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.191+0000 7ff2927fc700 1 --2- 172.21.15.29:0/3188741141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2940c19e0 0x7ff2940c1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.191+0000 7ff2927fc700 1 -- 172.21.15.29:0/3188741141 >> v1:172.21.15.29:6789/0 conn(0x7ff294151d90 legacy=0x7ff2940692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.191+0000 7ff2927fc700 1 -- 172.21.15.29:0/3188741141 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29406a070 con 0x7ff294064570 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.191+0000 7ff2927fc700 1 --2- 172.21.15.29:0/3188741141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff294064570 0x7ff294064a20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7ff288009760 tx=0x7ff288009060).ready entity=mon.1 client_cookie=4e682352f42f0604 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.191+0000 7ff291ffb700 1 -- 172.21.15.29:0/3188741141 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28800d4a0 con 0x7ff294064570 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.191+0000 7ff291ffb700 1 -- 172.21.15.29:0/3188741141 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28800d600 con 0x7ff294064570 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.191+0000 7ff291ffb700 1 -- 172.21.15.29:0/3188741141 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28800d980 con 0x7ff294064570 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.192+0000 7ff29927a700 1 -- 172.21.15.29:0/3188741141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff294064570 msgr2=0x7ff294064a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.192+0000 7ff29927a700 1 --2- 172.21.15.29:0/3188741141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff294064570 0x7ff294064a20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7ff288009760 tx=0x7ff288009060).stop 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.192+0000 7ff29927a700 1 -- 172.21.15.29:0/3188741141 shutdown_connections 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.192+0000 7ff29927a700 1 --2- 172.21.15.29:0/3188741141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2940c19e0 0x7ff2940c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.192+0000 7ff29927a700 1 --2- 172.21.15.29:0/3188741141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff294064570 0x7ff294064a20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.192+0000 7ff29927a700 1 -- 172.21.15.29:0/3188741141 >> 172.21.15.29:0/3188741141 conn(0x7ff294063280 msgr2=0x7ff2940c6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:48.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.192+0000 7ff29927a700 1 -- 172.21.15.29:0/3188741141 shutdown_connections 2024-01-24T15:13:48.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.192+0000 7ff29927a700 1 -- 172.21.15.29:0/3188741141 wait complete. 2024-01-24T15:13:48.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.193+0000 7ff29927a700 1 Processor -- start 2024-01-24T15:13:48.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.193+0000 7ff29927a700 1 -- start start 2024-01-24T15:13:48.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff29927a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff294064570 0x7ff2940c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:48.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff29927a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2940c19e0 0x7ff2941337e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:48.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff29927a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff294151d90 0x7ff2940aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:48.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff29927a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2940abc40 con 0x7ff294064570 2024-01-24T15:13:48.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff29927a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2940ab910 con 0x7ff2940c19e0 2024-01-24T15:13:48.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff29927a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff294135360 con 0x7ff294151d90 2024-01-24T15:13:48.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff292ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff294064570 0x7ff2940c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:48.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff292ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff294064570 0x7ff2940c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40568/0 (socket says 172.21.15.29:40568) 2024-01-24T15:13:48.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff292ffd700 1 -- 172.21.15.29:0/2045747948 learned_addr learned my addr 172.21.15.29:0/2045747948 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff2937fe700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff294151d90 0x7ff2940aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff2927fc700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2940c19e0 0x7ff2941337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff292ffd700 1 -- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff294151d90 msgr2=0x7ff2940aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.194+0000 7ff292ffd700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff294151d90 0x7ff2940aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.195+0000 7ff292ffd700 1 -- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2940c19e0 msgr2=0x7ff2941337e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.195+0000 7ff292ffd700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2940c19e0 0x7ff2941337e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.195+0000 7ff292ffd700 1 -- 172.21.15.29:0/2045747948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2880092a0 con 0x7ff294064570 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.195+0000 7ff292ffd700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff294064570 0x7ff2940c4c30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7ff28400c440 tx=0x7ff284007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.195+0000 7ff27affd700 1 -- 172.21.15.29:0/2045747948 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2840070d0 con 0x7ff294064570 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.195+0000 7ff27affd700 1 -- 172.21.15.29:0/2045747948 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff284007230 con 0x7ff294064570 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.195+0000 7ff29927a700 1 -- 172.21.15.29:0/2045747948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff294135640 con 0x7ff294064570 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.195+0000 7ff27affd700 1 -- 172.21.15.29:0/2045747948 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2840075f0 con 0x7ff294064570 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.195+0000 7ff29927a700 1 -- 172.21.15.29:0/2045747948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff294135920 con 0x7ff294064570 2024-01-24T15:13:48.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.196+0000 7ff29927a700 1 -- 172.21.15.29:0/2045747948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2940475e0 con 0x7ff294064570 2024-01-24T15:13:48.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.196+0000 7ff27affd700 1 -- 172.21.15.29:0/2045747948 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff2840059b0 con 0x7ff294064570 2024-01-24T15:13:48.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.197+0000 7ff27affd700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff28005f270 0x7ff280061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:48.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.197+0000 7ff27affd700 1 -- 172.21.15.29:0/2045747948 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7ff284079b30 con 0x7ff294064570 2024-01-24T15:13:48.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.197+0000 7ff2927fc700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff28005f270 0x7ff280061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:48.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.198+0000 7ff2927fc700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff28005f270 0x7ff280061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff28800d260 tx=0x7ff288016000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:48.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.201+0000 7ff27affd700 1 -- 172.21.15.29:0/2045747948 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff28404a850 con 0x7ff294064570 2024-01-24T15:13:48.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.429+0000 7ff29927a700 1 -- 172.21.15.29:0/2045747948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff294064f10 con 0x7ff294064570 2024-01-24T15:13:48.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.430+0000 7ff27affd700 1 -- 172.21.15.29:0/2045747948 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4497 (secure 0 0 0) 0x7ff294064f10 con 0x7ff294064570 2024-01-24T15:13:48.500 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:48.500 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7319,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3000519646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3000519646},{"type":"v1","addr":"172.21.15.163:6836","nonce":3000519646}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3497543927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3497543927},{"type":"v1","addr":"172.21.15.29:6835","nonce":3497543927}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2353694052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2353694052},{"type":"v1","addr":"172.21.15.163:6837","nonce":2353694052}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:13:38.524646+0000","modified":"2024-01-24T15:13:41.539080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/3595147603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3595147603},{"type":"v1","addr":"172.21.15.29:6834","nonce":3595147603}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-24T15:13:48.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.432+0000 7ff29927a700 1 -- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff28005f270 msgr2=0x7ff280061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:48.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.432+0000 7ff29927a700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff28005f270 0x7ff280061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff28800d260 tx=0x7ff288016000).stop 2024-01-24T15:13:48.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.432+0000 7ff29927a700 1 -- 172.21.15.29:0/2045747948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff294064570 msgr2=0x7ff2940c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:48.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.432+0000 7ff29927a700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff294064570 0x7ff2940c4c30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7ff28400c440 tx=0x7ff284007ab0).stop 2024-01-24T15:13:48.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.432+0000 7ff29927a700 1 -- 172.21.15.29:0/2045747948 shutdown_connections 2024-01-24T15:13:48.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.432+0000 7ff29927a700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff294151d90 0x7ff2940aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.432+0000 7ff29927a700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff28005f270 0x7ff280061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.432+0000 7ff29927a700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff294064570 0x7ff2940c4c30 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.432+0000 7ff29927a700 1 --2- 172.21.15.29:0/2045747948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2940c19e0 0x7ff2941337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:48.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.432+0000 7ff29927a700 1 -- 172.21.15.29:0/2045747948 >> 172.21.15.29:0/2045747948 conn(0x7ff294063280 msgr2=0x7ff2940bc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:48.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.432+0000 7ff29927a700 1 -- 172.21.15.29:0/2045747948 shutdown_connections 2024-01-24T15:13:48.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:48.432+0000 7ff29927a700 1 -- 172.21.15.29:0/2045747948 wait complete. 2024-01-24T15:13:48.502 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 74 2024-01-24T15:13:48.906 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T15:13:38.524646+0000', 'modified': '2024-01-24T15:13:41.539080+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8242': {'gid': 8242, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6834/3595147603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6832', 'nonce': 3595147603}, {'type': 'v1', 'addr': '172.21.15.29:6834', 'nonce': 3595147603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-24T15:13:48.906 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T15:13:48.906 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-24T15:13:49.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.044+0000 7f932327a700 1 Processor -- start 2024-01-24T15:13:49.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.044+0000 7f932327a700 1 -- start start 2024-01-24T15:13:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.045+0000 7f932327a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f931c066bb0 0x7f931c1525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.045+0000 7f932327a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f931c152ae0 0x7f931c0bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.045+0000 7f932327a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c156460 con 0x7f931c135350 2024-01-24T15:13:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.045+0000 7f932327a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c155c00 con 0x7f931c152ae0 2024-01-24T15:13:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.045+0000 7f932327a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c153390 con 0x7f931c066bb0 2024-01-24T15:13:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.045+0000 7f9321817700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f931c152ae0 0x7f931c0bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.045+0000 7f9321817700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f931c152ae0 0x7f931c0bdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40574/0 (socket says 172.21.15.29:40574) 2024-01-24T15:13:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.045+0000 7f9321817700 1 -- 172.21.15.29:0/3410381297 learned_addr learned my addr 172.21.15.29:0/3410381297 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.045+0000 7f9320815700 1 --2- 172.21.15.29:0/3410381297 >> v2:172.21.15.163:3300/0 conn(0x7f931c066bb0 0x7f931c1525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.046+0000 7f9321817700 1 -- 172.21.15.29:0/3410381297 >> v2:172.21.15.163:3300/0 conn(0x7f931c066bb0 msgr2=0x7f931c1525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.046+0000 7f9321817700 1 --2- 172.21.15.29:0/3410381297 >> v2:172.21.15.163:3300/0 conn(0x7f931c066bb0 0x7f931c1525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.046+0000 7f9321817700 1 -- 172.21.15.29:0/3410381297 >> v1:172.21.15.29:6789/0 conn(0x7f931c135350 legacy=0x7f931c0664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.046+0000 7f9321817700 1 -- 172.21.15.29:0/3410381297 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c155e10 con 0x7f931c152ae0 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.046+0000 7f9321817700 1 --2- 172.21.15.29:0/3410381297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f931c152ae0 0x7f931c0bdb00 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f931c05f220 tx=0x7f9314009840).ready entity=mon.0 client_cookie=20851b780388fd58 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.046+0000 7f9313fff700 1 -- 172.21.15.29:0/3410381297 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400e070 con 0x7f931c152ae0 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.046+0000 7f9313fff700 1 -- 172.21.15.29:0/3410381297 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931400ba50 con 0x7f931c152ae0 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.046+0000 7f9313fff700 1 -- 172.21.15.29:0/3410381297 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400bdd0 con 0x7f931c152ae0 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.046+0000 7f932327a700 1 -- 172.21.15.29:0/3410381297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f931c152ae0 msgr2=0x7f931c0bdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.046+0000 7f932327a700 1 --2- 172.21.15.29:0/3410381297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f931c152ae0 0x7f931c0bdb00 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f931c05f220 tx=0x7f9314009840).stop 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.046+0000 7f932327a700 1 -- 172.21.15.29:0/3410381297 shutdown_connections 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.047+0000 7f932327a700 1 --2- 172.21.15.29:0/3410381297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f931c152ae0 0x7f931c0bdb00 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.047+0000 7f932327a700 1 --2- 172.21.15.29:0/3410381297 >> v2:172.21.15.163:3300/0 conn(0x7f931c066bb0 0x7f931c1525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.047+0000 7f932327a700 1 -- 172.21.15.29:0/3410381297 >> 172.21.15.29:0/3410381297 conn(0x7f931c0c5640 msgr2=0x7f931c0c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.047+0000 7f932327a700 1 -- 172.21.15.29:0/3410381297 shutdown_connections 2024-01-24T15:13:49.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.047+0000 7f932327a700 1 -- 172.21.15.29:0/3410381297 wait complete. 2024-01-24T15:13:49.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.048+0000 7f932327a700 1 Processor -- start 2024-01-24T15:13:49.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.048+0000 7f932327a700 1 -- start start 2024-01-24T15:13:49.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f932327a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f931c066bb0 0x7f931c134870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f932327a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f931c135350 0x7f931c0a8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f932327a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f931c152ae0 0x7f931c0a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f932327a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0abee0 con 0x7f931c135350 2024-01-24T15:13:49.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f932327a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0a8fd0 con 0x7f931c066bb0 2024-01-24T15:13:49.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f932327a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0a92b0 con 0x7f931c152ae0 2024-01-24T15:13:49.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f9320815700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f931c135350 0x7f931c0a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f9320815700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f931c135350 0x7f931c0a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40582/0 (socket says 172.21.15.29:40582) 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f9320815700 1 -- 172.21.15.29:0/2992562438 learned_addr learned my addr 172.21.15.29:0/2992562438 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f9321817700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f931c152ae0 0x7f931c0a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f9321016700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f931c066bb0 0x7f931c134870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f9320815700 1 -- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f931c152ae0 msgr2=0x7f931c0a9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f9320815700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f931c152ae0 0x7f931c0a9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f9320815700 1 -- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f931c066bb0 msgr2=0x7f931c134870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f9320815700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f931c066bb0 0x7f931c134870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f9320815700 1 -- 172.21.15.29:0/2992562438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314009ae0 con 0x7f931c135350 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f9321016700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f931c066bb0 0x7f931c134870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f9320815700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f931c135350 0x7f931c0a8660 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f930c00d860 tx=0x7f930c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f93117fa700 1 -- 172.21.15.29:0/2992562438 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c0125f0 con 0x7f931c135350 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f93117fa700 1 -- 172.21.15.29:0/2992562438 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c012750 con 0x7f931c135350 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f932327a700 1 -- 172.21.15.29:0/2992562438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c0a9590 con 0x7f931c135350 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.050+0000 7f93117fa700 1 -- 172.21.15.29:0/2992562438 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c012ad0 con 0x7f931c135350 2024-01-24T15:13:49.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.051+0000 7f932327a700 1 -- 172.21.15.29:0/2992562438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c0a8b50 con 0x7f931c135350 2024-01-24T15:13:49.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.052+0000 7f93117fa700 1 -- 172.21.15.29:0/2992562438 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f930c013480 con 0x7f931c135350 2024-01-24T15:13:49.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.052+0000 7f93117fa700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f930005f270 0x7f9300061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.052+0000 7f93117fa700 1 -- 172.21.15.29:0/2992562438 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f930c07a500 con 0x7f931c135350 2024-01-24T15:13:49.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.053+0000 7f9321016700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f930005f270 0x7f9300061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.053+0000 7f9321016700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f930005f270 0x7f9300061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9318001730 tx=0x7f9318009040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.055+0000 7f9334c45b80 1 Processor -- start 2024-01-24T15:13:49.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.055+0000 7f9334c45b80 1 -- start start 2024-01-24T15:13:49.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.055+0000 7f9334c45b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f8620 0x5626399f8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.055+0000 7f9334c45b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 0x5626399f3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.055+0000 7f9334c45b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x5626399f4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.055+0000 7f9334c45b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263996f3d0 con 0x5626399f3df0 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.055+0000 7f9334c45b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562639934be0 con 0x5626399f8620 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.055+0000 7f9334c45b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56263994a260 con 0x562639a412c0 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f9321817700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x5626399f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f9321817700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x5626399f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40584/0 (socket says 172.21.15.29:40584) 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f9321817700 1 -- 172.21.15.29:0/1004102601 learned_addr learned my addr 172.21.15.29:0/1004102601 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f9320815700 1 --2- 172.21.15.29:0/1004102601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 0x5626399f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f9321016700 1 --2- 172.21.15.29:0/1004102601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f8620 0x5626399f8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f9321817700 1 -- 172.21.15.29:0/1004102601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 msgr2=0x5626399f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f9321817700 1 --2- 172.21.15.29:0/1004102601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 0x5626399f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f9321817700 1 -- 172.21.15.29:0/1004102601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f8620 msgr2=0x5626399f8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:49.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f9321817700 1 --2- 172.21.15.29:0/1004102601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f8620 0x5626399f8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f9321817700 1 -- 172.21.15.29:0/1004102601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626399432f0 con 0x5626399f3df0 2024-01-24T15:13:49.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f9321817700 1 --2- 172.21.15.29:0/1004102601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x5626399f4270 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f931400b9a0 tx=0x7f9314009840).ready entity=mon.0 client_cookie=2d39ed41b72a9d5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:49.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f93067fc700 1 -- 172.21.15.29:0/1004102601 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314022070 con 0x5626399f3df0 2024-01-24T15:13:49.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f93067fc700 1 -- 172.21.15.29:0/1004102601 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931400bbc0 con 0x5626399f3df0 2024-01-24T15:13:49.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.056+0000 7f93067fc700 1 -- 172.21.15.29:0/1004102601 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400bd30 con 0x5626399f3df0 2024-01-24T15:13:49.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.057+0000 7f9334c45b80 1 -- 172.21.15.29:0/1004102601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 msgr2=0x5626399f4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.057+0000 7f9334c45b80 1 --2- 172.21.15.29:0/1004102601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x5626399f4270 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f931400b9a0 tx=0x7f9314009840).stop 2024-01-24T15:13:49.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.057+0000 7f9334c45b80 1 -- 172.21.15.29:0/1004102601 shutdown_connections 2024-01-24T15:13:49.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.057+0000 7f9334c45b80 1 --2- 172.21.15.29:0/1004102601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x5626399f4270 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.057+0000 7f9334c45b80 1 --2- 172.21.15.29:0/1004102601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f8620 0x5626399f8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.057+0000 7f9334c45b80 1 --2- 172.21.15.29:0/1004102601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 0x5626399f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.057+0000 7f9334c45b80 1 -- 172.21.15.29:0/1004102601 >> 172.21.15.29:0/1004102601 conn(0x5626398612e0 msgr2=0x562639a41750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.057+0000 7f9334c45b80 1 -- 172.21.15.29:0/1004102601 shutdown_connections 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.057+0000 7f9334c45b80 1 -- 172.21.15.29:0/1004102601 wait complete. 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.058+0000 7f9334c45b80 1 -- start start 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.058+0000 7f9334c45b80 1 Processor -- start 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9334c45b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x562639a1fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9334c45b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5626399f8620 0x562639a21230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9334c45b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562639a412c0 0x562639a21b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9334c45b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263994a3a0 con 0x5626399f3df0 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9334c45b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263994a620 con 0x562639a412c0 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9334c45b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x562639915990 con 0x5626399f8620 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9334c45b80 10 client.0 ll_register_callbacks cb 0x5626399b8610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9321016700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x562639a1fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.067 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9321817700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562639a412c0 0x562639a21b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9321016700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x562639a1fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40592/0 (socket says 172.21.15.29:40592) 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9321016700 1 -- 172.21.15.29:0/1170438708 learned_addr learned my addr 172.21.15.29:0/1170438708 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9320815700 1 --2- 172.21.15.29:0/1170438708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5626399f8620 0x562639a21230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9321016700 1 -- 172.21.15.29:0/1170438708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5626399f8620 msgr2=0x562639a21230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9321016700 1 --2- 172.21.15.29:0/1170438708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5626399f8620 0x562639a21230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9321016700 1 -- 172.21.15.29:0/1170438708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562639a412c0 msgr2=0x562639a21b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9321016700 1 --2- 172.21.15.29:0/1170438708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562639a412c0 0x562639a21b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9321016700 1 -- 172.21.15.29:0/1170438708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c155e10 con 0x5626399f3df0 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f9321016700 1 --2- 172.21.15.29:0/1170438708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x562639a1fe70 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f9318018590 tx=0x7f931801a040).ready entity=mon.0 client_cookie=63c029452aabeb82 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.059+0000 7f92fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f92fb7fe700 1 -- 172.21.15.29:0/1170438708 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931801d040 con 0x5626399f3df0 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f92fb7fe700 1 -- 172.21.15.29:0/1170438708 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f9318016600 con 0x5626399f3df0 2024-01-24T15:13:49.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f9334c45b80 1 -- 172.21.15.29:0/1170438708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626399f2df0 con 0x5626399f3df0 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f92fb7fe700 1 -- 172.21.15.29:0/1170438708 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93180169d0 con 0x5626399f3df0 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f9334c45b80 10 client.8766 fetch_fsmap learned FSMap version 74 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f9334c45b80 1 -- 172.21.15.29:0/1170438708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x56263993e520 con 0x5626399f3df0 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f92fb7fe700 1 -- 172.21.15.29:0/1170438708 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626399f2df0 con 0x5626399f3df0 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f92fb7fe700 1 -- 172.21.15.29:0/1170438708 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 74) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9318016b30 con 0x5626399f3df0 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f9334c45b80 10 client.8766 fetch_fsmap finished waiting for FSMap version 74 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f9334c45b80 10 client.8766 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f9334c45b80 20 client.8766 populate_metadata read hostname 'smithi029' 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f9334c45b80 1 --2- 172.21.15.29:0/1170438708 >> [v2:172.21.15.29:6832/3595147603,v1:172.21.15.29:6834/3595147603] conn(0x562639a3a7d0 0x562639a3abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f9334c45b80 4 client.8766 mds_command: new command op to 8242 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f9334c45b80 1 -- 172.21.15.29:0/1170438708 --> [v2:172.21.15.29:6832/3595147603,v1:172.21.15.29:6834/3595147603] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56263993fd60 con 0x562639a3a7d0 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.060+0000 7f9320815700 1 --2- 172.21.15.29:0/1170438708 >> [v2:172.21.15.29:6832/3595147603,v1:172.21.15.29:6834/3595147603] conn(0x562639a3a7d0 0x562639a3abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.061+0000 7f92fb7fe700 10 client.8766 ms_handle_connect on v2:172.21.15.29:6832/3595147603 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.061+0000 7f9320815700 1 --2- 172.21.15.29:0/1170438708 >> [v2:172.21.15.29:6832/3595147603,v1:172.21.15.29:6834/3595147603] conn(0x562639a3a7d0 0x562639a3abb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7b0b402a53199c3 server_cookie=5992d0a0053390ba in_seq=0 out_seq=0 2024-01-24T15:13:49.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.061+0000 7f92fb7fe700 1 -- 172.21.15.29:0/1170438708 <== mds.0 v2:172.21.15.29:6832/3595147603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x56263993fd60 con 0x562639a3a7d0 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.061+0000 7f92fb7fe700 10 client.8766 handle_command_reply: tid=0 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.061+0000 7f9334c45b80 1 client.8766 shutdown 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.062+0000 7f9334c45b80 1 -- 172.21.15.29:0/1170438708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 msgr2=0x562639a1fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.062+0000 7f9334c45b80 1 --2- 172.21.15.29:0/1170438708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x562639a1fe70 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f9318018590 tx=0x7f931801a040).stop 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.062+0000 7f9334c45b80 1 -- 172.21.15.29:0/1170438708 shutdown_connections 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.062+0000 7f9334c45b80 1 --2- 172.21.15.29:0/1170438708 >> [v2:172.21.15.29:6832/3595147603,v1:172.21.15.29:6834/3595147603] conn(0x562639a3a7d0 0x562639a3abb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.062+0000 7f9334c45b80 1 --2- 172.21.15.29:0/1170438708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562639a412c0 0x562639a21b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.062+0000 7f9334c45b80 1 --2- 172.21.15.29:0/1170438708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x562639a1fe70 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.062+0000 7f9334c45b80 1 --2- 172.21.15.29:0/1170438708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5626399f8620 0x562639a21230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.062+0000 7f92fb7fe700 0 client.8766 ms_handle_reset on v2:172.21.15.29:6832/3595147603 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.062+0000 7f9334c45b80 1 -- 172.21.15.29:0/1170438708 >> 172.21.15.29:0/1170438708 conn(0x5626398612e0 msgr2=0x562639a3d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.062+0000 7f9334c45b80 1 -- 172.21.15.29:0/1170438708 shutdown_connections 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.062+0000 7f9334c45b80 1 -- 172.21.15.29:0/1170438708 wait complete. 2024-01-24T15:13:49.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.062+0000 7f9334c45b80 20 client.8766 trim_cache size 0 max 16384 2024-01-24T15:13:49.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.079+0000 7f9334c45b80 1 Processor -- start 2024-01-24T15:13:49.082 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.080+0000 7f9334c45b80 1 -- start start 2024-01-24T15:13:49.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.080+0000 7f9334c45b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f3df0 0x562639a20270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.080+0000 7f9334c45b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f8620 0x562639a388f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.080+0000 7f9334c45b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 0x562639a38de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.080+0000 7f9334c45b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626399f35f0 con 0x5626399f8620 2024-01-24T15:13:49.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.080+0000 7f9334c45b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562639a3b970 con 0x5626399f3df0 2024-01-24T15:13:49.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.080+0000 7f9334c45b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626399f41d0 con 0x562639a412c0 2024-01-24T15:13:49.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.080+0000 7f9320815700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f8620 0x562639a388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.080+0000 7f9320815700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f8620 0x562639a388f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40598/0 (socket says 172.21.15.29:40598) 2024-01-24T15:13:49.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.080+0000 7f9320815700 1 -- 172.21.15.29:0/3918782243 learned_addr learned my addr 172.21.15.29:0/3918782243 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:49.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.081+0000 7f9321016700 1 --2- 172.21.15.29:0/3918782243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f3df0 0x562639a20270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.081+0000 7f9321817700 1 --2- 172.21.15.29:0/3918782243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 0x562639a38de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.081+0000 7f9320815700 1 -- 172.21.15.29:0/3918782243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 msgr2=0x562639a38de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:49.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.081+0000 7f9320815700 1 --2- 172.21.15.29:0/3918782243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 0x562639a38de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.081+0000 7f9320815700 1 -- 172.21.15.29:0/3918782243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f3df0 msgr2=0x562639a20270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:49.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.081+0000 7f9320815700 1 --2- 172.21.15.29:0/3918782243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f3df0 0x562639a20270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.081+0000 7f9320815700 1 -- 172.21.15.29:0/3918782243 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56263993e270 con 0x5626399f8620 2024-01-24T15:13:49.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.081+0000 7f9321016700 1 --2- 172.21.15.29:0/3918782243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f3df0 0x562639a20270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:13:49.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.081+0000 7f9320815700 1 --2- 172.21.15.29:0/3918782243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f8620 0x562639a388f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f930c04ee60 tx=0x7f930c056040).ready entity=mon.0 client_cookie=658e31a49e833777 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:49.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.081+0000 7f9306ffd700 1 -- 172.21.15.29:0/3918782243 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c053040 con 0x5626399f8620 2024-01-24T15:13:49.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.081+0000 7f9306ffd700 1 -- 172.21.15.29:0/3918782243 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c054040 con 0x5626399f8620 2024-01-24T15:13:49.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.081+0000 7f9306ffd700 1 -- 172.21.15.29:0/3918782243 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c04d140 con 0x5626399f8620 2024-01-24T15:13:49.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.082+0000 7f9334c45b80 1 -- 172.21.15.29:0/3918782243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f8620 msgr2=0x562639a388f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.082+0000 7f9334c45b80 1 --2- 172.21.15.29:0/3918782243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f8620 0x562639a388f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f930c04ee60 tx=0x7f930c056040).stop 2024-01-24T15:13:49.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.082+0000 7f9334c45b80 1 -- 172.21.15.29:0/3918782243 shutdown_connections 2024-01-24T15:13:49.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.082+0000 7f9334c45b80 1 --2- 172.21.15.29:0/3918782243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 0x562639a38de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.082+0000 7f9334c45b80 1 --2- 172.21.15.29:0/3918782243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f3df0 0x562639a20270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.082+0000 7f9334c45b80 1 --2- 172.21.15.29:0/3918782243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f8620 0x562639a388f0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.082+0000 7f9334c45b80 1 -- 172.21.15.29:0/3918782243 >> 172.21.15.29:0/3918782243 conn(0x5626398612e0 msgr2=0x562639a41750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:49.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.082+0000 7f9334c45b80 1 -- 172.21.15.29:0/3918782243 shutdown_connections 2024-01-24T15:13:49.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.082+0000 7f9334c45b80 1 -- 172.21.15.29:0/3918782243 wait complete. 2024-01-24T15:13:49.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.082+0000 7f9334c45b80 1 -- start start 2024-01-24T15:13:49.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.083+0000 7f9334c45b80 1 Processor -- start 2024-01-24T15:13:49.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9334c45b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x562639aa30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9334c45b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f8620 0x562639a388f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9334c45b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 0x562639a38de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9334c45b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x562639a392d0 con 0x5626399f3df0 2024-01-24T15:13:49.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9334c45b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562639aa3620 con 0x5626399f8620 2024-01-24T15:13:49.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9334c45b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x562639a5e810 con 0x562639a412c0 2024-01-24T15:13:49.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9334c45b80 10 client.0 ll_register_callbacks cb 0x5626399b8610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T15:13:49.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9321016700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x562639aa30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9321016700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x562639aa30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40600/0 (socket says 172.21.15.29:40600) 2024-01-24T15:13:49.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9321016700 1 -- 172.21.15.29:0/96535283 learned_addr learned my addr 172.21.15.29:0/96535283 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:49.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9321817700 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 0x562639a38de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9320815700 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f8620 0x562639a388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9321016700 1 -- 172.21.15.29:0/96535283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 msgr2=0x562639a38de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:49.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9321016700 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 0x562639a38de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9321016700 1 -- 172.21.15.29:0/96535283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f8620 msgr2=0x562639a388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:49.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9321016700 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f8620 0x562639a388f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.084+0000 7f9321016700 1 -- 172.21.15.29:0/96535283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c0a8b50 con 0x5626399f3df0 2024-01-24T15:13:49.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f9321016700 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x562639aa30d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f9318016600 tx=0x7f9318013000).ready entity=mon.0 client_cookie=2fc991bf75d9d42c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:49.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f92fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T15:13:49.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f92fb7fe700 1 -- 172.21.15.29:0/96535283 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931801d030 con 0x5626399f3df0 2024-01-24T15:13:49.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f92fb7fe700 1 -- 172.21.15.29:0/96535283 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f9318016be0 con 0x5626399f3df0 2024-01-24T15:13:49.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f9334c45b80 1 -- 172.21.15.29:0/96535283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f931400bbc0 con 0x5626399f3df0 2024-01-24T15:13:49.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f92fb7fe700 1 -- 172.21.15.29:0/96535283 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9318031070 con 0x5626399f3df0 2024-01-24T15:13:49.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f9334c45b80 10 client.8772 fetch_fsmap learned FSMap version 74 2024-01-24T15:13:49.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f9334c45b80 1 -- 172.21.15.29:0/96535283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x5626397b7f80 con 0x5626399f3df0 2024-01-24T15:13:49.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f92fb7fe700 1 -- 172.21.15.29:0/96535283 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931400bbc0 con 0x5626399f3df0 2024-01-24T15:13:49.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f92fb7fe700 1 -- 172.21.15.29:0/96535283 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 74) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9318034020 con 0x5626399f3df0 2024-01-24T15:13:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f9334c45b80 10 client.8772 fetch_fsmap finished waiting for FSMap version 74 2024-01-24T15:13:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f9334c45b80 10 client.8772 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T15:13:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f9334c45b80 20 client.8772 populate_metadata read hostname 'smithi029' 2024-01-24T15:13:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f9334c45b80 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.29:6832/3595147603,v1:172.21.15.29:6834/3595147603] conn(0x562639a4c8d0 0x562639a554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f9334c45b80 4 client.8772 mds_command: new command op to 8242 tid=0[{"prefix": "status"}] 2024-01-24T15:13:49.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f9334c45b80 1 -- 172.21.15.29:0/96535283 --> [v2:172.21.15.29:6832/3595147603,v1:172.21.15.29:6834/3595147603] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5626399367d0 con 0x562639a4c8d0 2024-01-24T15:13:49.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.085+0000 7f9320815700 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.29:6832/3595147603,v1:172.21.15.29:6834/3595147603] conn(0x562639a4c8d0 0x562639a554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f92fb7fe700 10 client.8772 ms_handle_connect on v2:172.21.15.29:6832/3595147603 2024-01-24T15:13:49.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f9320815700 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.29:6832/3595147603,v1:172.21.15.29:6834/3595147603] conn(0x562639a4c8d0 0x562639a554f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=31858b92a70705f1 server_cookie=f91c56a00288bbe9 in_seq=0 out_seq=0 2024-01-24T15:13:49.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f92fb7fe700 1 -- 172.21.15.29:0/96535283 <== mds.0 v2:172.21.15.29:6832/3595147603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5626399367d0 con 0x562639a4c8d0 2024-01-24T15:13:49.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f92fb7fe700 10 client.8772 handle_command_reply: tid=0 2024-01-24T15:13:49.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f9334c45b80 1 client.8772 shutdown 2024-01-24T15:13:49.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f9334c45b80 1 -- 172.21.15.29:0/96535283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 msgr2=0x562639aa30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f9334c45b80 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x562639aa30d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f9318016600 tx=0x7f9318013000).stop 2024-01-24T15:13:49.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f9334c45b80 1 -- 172.21.15.29:0/96535283 shutdown_connections 2024-01-24T15:13:49.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f9334c45b80 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.29:6832/3595147603,v1:172.21.15.29:6834/3595147603] conn(0x562639a4c8d0 0x562639a554f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f9334c45b80 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562639a412c0 0x562639a38de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f9334c45b80 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5626399f3df0 0x562639aa30d0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f9334c45b80 1 --2- 172.21.15.29:0/96535283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626399f8620 0x562639a388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f92fb7fe700 0 client.8772 ms_handle_reset on v2:172.21.15.29:6832/3595147603 2024-01-24T15:13:49.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f9334c45b80 1 -- 172.21.15.29:0/96535283 >> 172.21.15.29:0/96535283 conn(0x5626398612e0 msgr2=0x5626399ef150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:49.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f9334c45b80 1 -- 172.21.15.29:0/96535283 shutdown_connections 2024-01-24T15:13:49.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.086+0000 7f9334c45b80 1 -- 172.21.15.29:0/96535283 wait complete. 2024-01-24T15:13:49.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f9334c45b80 20 client.8772 trim_cache size 0 max 16384 2024-01-24T15:13:49.107 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:13:49.107 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T15:13:49.107 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T15:13:49.107 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 8242, 2024-01-24T15:13:49.108 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T15:13:49.108 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T15:13:49.108 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T15:13:49.108 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 10.560421801, 2024-01-24T15:13:49.108 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 74, 2024-01-24T15:13:49.108 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 178, 2024-01-24T15:13:49.108 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 178, 2024-01-24T15:13:49.108 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 16.032368597000001 2024-01-24T15:13:49.108 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:13:49.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f932327a700 1 -- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f930005f270 msgr2=0x7f9300061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f932327a700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f930005f270 0x7f9300061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9318001730 tx=0x7f9318009040).stop 2024-01-24T15:13:49.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f932327a700 1 -- 172.21.15.29:0/2992562438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f931c135350 msgr2=0x7f931c0a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f932327a700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f931c135350 0x7f931c0a8660 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f930c00d860 tx=0x7f930c008040).stop 2024-01-24T15:13:49.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f932327a700 1 -- 172.21.15.29:0/2992562438 shutdown_connections 2024-01-24T15:13:49.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f932327a700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f931c152ae0 0x7f931c0a9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f932327a700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f931c066bb0 0x7f931c134870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f932327a700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f930005f270 0x7f9300061730 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f932327a700 1 --2- 172.21.15.29:0/2992562438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f931c135350 0x7f931c0a8660 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f932327a700 1 -- 172.21.15.29:0/2992562438 >> 172.21.15.29:0/2992562438 conn(0x7f931c0c5640 msgr2=0x7f931c0bea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:49.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f932327a700 1 -- 172.21.15.29:0/2992562438 shutdown_connections 2024-01-24T15:13:49.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.087+0000 7f932327a700 1 -- 172.21.15.29:0/2992562438 wait complete. 2024-01-24T15:13:49.113 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T15:13:49.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.227+0000 7fec635b1700 1 Processor -- start 2024-01-24T15:13:49.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.227+0000 7fec635b1700 1 -- start start 2024-01-24T15:13:49.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec635b1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fec5c13a4b0 0x7fec5c066740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec635b1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fec5c13a890 0x7fec5c066c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec635b1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec5c0694a0 0x7fec5c069920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec635b1700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec5c067ea0 con 0x7fec5c13a4b0 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec635b1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec5c069e10 con 0x7fec5c13a890 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec635b1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c067290 con 0x7fec5c0694a0 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec6134d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fec5c13a4b0 0x7fec5c066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec6134d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fec5c13a4b0 0x7fec5c066740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40610/0 (socket says 172.21.15.29:40610) 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec6134d700 1 -- 172.21.15.29:0/1644258686 learned_addr learned my addr 172.21.15.29:0/1644258686 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec61b4e700 1 --2- 172.21.15.29:0/1644258686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec5c0694a0 0x7fec5c069920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec60b4c700 1 --2- 172.21.15.29:0/1644258686 >> v2:172.21.15.163:3300/0 conn(0x7fec5c13a890 0x7fec5c066c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec6134d700 1 -- 172.21.15.29:0/1644258686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec5c0694a0 msgr2=0x7fec5c069920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec6134d700 1 --2- 172.21.15.29:0/1644258686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec5c0694a0 0x7fec5c069920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec6134d700 1 -- 172.21.15.29:0/1644258686 >> v2:172.21.15.163:3300/0 conn(0x7fec5c13a890 msgr2=0x7fec5c066c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:49.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec6134d700 1 --2- 172.21.15.29:0/1644258686 >> v2:172.21.15.163:3300/0 conn(0x7fec5c13a890 0x7fec5c066c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec6134d700 1 -- 172.21.15.29:0/1644258686 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c067570 con 0x7fec5c13a4b0 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.228+0000 7fec6134d700 1 --2- 172.21.15.29:0/1644258686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5c13a4b0 0x7fec5c066740 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7fec5c154830 tx=0x7fec580090f0).ready entity=mon.0 client_cookie=3293b389b391f96c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.229+0000 7fec53fff700 1 -- 172.21.15.29:0/1644258686 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5800b600 con 0x7fec5c13a4b0 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.229+0000 7fec53fff700 1 -- 172.21.15.29:0/1644258686 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5800b760 con 0x7fec5c13a4b0 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.229+0000 7fec53fff700 1 -- 172.21.15.29:0/1644258686 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5800bae0 con 0x7fec5c13a4b0 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.229+0000 7fec635b1700 1 -- 172.21.15.29:0/1644258686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5c13a4b0 msgr2=0x7fec5c066740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.229+0000 7fec635b1700 1 --2- 172.21.15.29:0/1644258686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5c13a4b0 0x7fec5c066740 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7fec5c154830 tx=0x7fec580090f0).stop 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.229+0000 7fec635b1700 1 -- 172.21.15.29:0/1644258686 shutdown_connections 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.229+0000 7fec635b1700 1 --2- 172.21.15.29:0/1644258686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec5c0694a0 0x7fec5c069920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.229+0000 7fec635b1700 1 --2- 172.21.15.29:0/1644258686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5c13a4b0 0x7fec5c066740 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.229+0000 7fec635b1700 1 --2- 172.21.15.29:0/1644258686 >> v2:172.21.15.163:3300/0 conn(0x7fec5c13a890 0x7fec5c066c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.229+0000 7fec635b1700 1 -- 172.21.15.29:0/1644258686 >> 172.21.15.29:0/1644258686 conn(0x7fec5c0c7b10 msgr2=0x7fec5c0c7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.229+0000 7fec635b1700 1 -- 172.21.15.29:0/1644258686 shutdown_connections 2024-01-24T15:13:49.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.229+0000 7fec635b1700 1 -- 172.21.15.29:0/1644258686 wait complete. 2024-01-24T15:13:49.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.230+0000 7fec635b1700 1 Processor -- start 2024-01-24T15:13:49.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.231+0000 7fec635b1700 1 -- start start 2024-01-24T15:13:49.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.231+0000 7fec635b1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec5c0694a0 0x7fec5c1317b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec635b1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5c13a4b0 0x7fec5c139420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec635b1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec5c13a890 0x7fec5c139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec635b1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c12fc50 con 0x7fec5c13a4b0 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec635b1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c151000 con 0x7fec5c13a890 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec635b1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c0c1920 con 0x7fec5c0694a0 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec60b4c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5c13a4b0 0x7fec5c139420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec60b4c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5c13a4b0 0x7fec5c139420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40616/0 (socket says 172.21.15.29:40616) 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec60b4c700 1 -- 172.21.15.29:0/1970906205 learned_addr learned my addr 172.21.15.29:0/1970906205 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec6134d700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec5c0694a0 0x7fec5c1317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec61b4e700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec5c13a890 0x7fec5c139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec60b4c700 1 -- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec5c0694a0 msgr2=0x7fec5c1317b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec60b4c700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec5c0694a0 0x7fec5c1317b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec60b4c700 1 -- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec5c13a890 msgr2=0x7fec5c139910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec60b4c700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec5c13a890 0x7fec5c139910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec60b4c700 1 -- 172.21.15.29:0/1970906205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec58009370 con 0x7fec5c13a4b0 2024-01-24T15:13:49.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec61b4e700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec5c13a890 0x7fec5c139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:49.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec60b4c700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5c13a4b0 0x7fec5c139420 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7fec4c00d890 tx=0x7fec4c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:49.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec517fa700 1 -- 172.21.15.29:0/1970906205 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c011590 con 0x7fec5c13a4b0 2024-01-24T15:13:49.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec517fa700 1 -- 172.21.15.29:0/1970906205 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4c0116f0 con 0x7fec5c13a4b0 2024-01-24T15:13:49.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec517fa700 1 -- 172.21.15.29:0/1970906205 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c011a70 con 0x7fec5c13a4b0 2024-01-24T15:13:49.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec635b1700 1 -- 172.21.15.29:0/1970906205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5c130cd0 con 0x7fec5c13a4b0 2024-01-24T15:13:49.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.232+0000 7fec635b1700 1 -- 172.21.15.29:0/1970906205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec5c130960 con 0x7fec5c13a4b0 2024-01-24T15:13:49.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.233+0000 7fec635b1700 1 -- 172.21.15.29:0/1970906205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5c130e60 con 0x7fec5c13a4b0 2024-01-24T15:13:49.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.234+0000 7fec517fa700 1 -- 172.21.15.29:0/1970906205 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fec4c012420 con 0x7fec5c13a4b0 2024-01-24T15:13:49.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.234+0000 7fec517fa700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fec4005f240 0x7fec40061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:49.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.234+0000 7fec517fa700 1 -- 172.21.15.29:0/1970906205 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fec4c07a1e0 con 0x7fec5c13a4b0 2024-01-24T15:13:49.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.234+0000 7fec6134d700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fec4005f240 0x7fec40061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:49.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.235+0000 7fec6134d700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fec4005f240 0x7fec40061700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fec58009130 tx=0x7fec58017000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:49.240 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.239+0000 7fec517fa700 1 -- 172.21.15.29:0/1970906205 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fec4c04aea0 con 0x7fec5c13a4b0 2024-01-24T15:13:49.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.464+0000 7fec635b1700 1 -- 172.21.15.29:0/1970906205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fec5c067850 con 0x7fec5c13a4b0 2024-01-24T15:13:49.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.465+0000 7fec517fa700 1 -- 172.21.15.29:0/1970906205 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) v1 ==== 53+0+73 (secure 0 0 0) 0x7fec4c011e60 con 0x7fec5c13a4b0 2024-01-24T15:13:49.467 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-24T15:13:49.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.467+0000 7fec635b1700 1 -- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fec4005f240 msgr2=0x7fec40061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.467+0000 7fec635b1700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fec4005f240 0x7fec40061700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fec58009130 tx=0x7fec58017000).stop 2024-01-24T15:13:49.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.467+0000 7fec635b1700 1 -- 172.21.15.29:0/1970906205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5c13a4b0 msgr2=0x7fec5c139420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:49.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.467+0000 7fec635b1700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5c13a4b0 0x7fec5c139420 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7fec4c00d890 tx=0x7fec4c008040).stop 2024-01-24T15:13:49.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.467+0000 7fec635b1700 1 -- 172.21.15.29:0/1970906205 shutdown_connections 2024-01-24T15:13:49.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.467+0000 7fec635b1700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec5c13a890 0x7fec5c139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.467+0000 7fec635b1700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec5c0694a0 0x7fec5c1317b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.467+0000 7fec635b1700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fec4005f240 0x7fec40061700 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.467+0000 7fec635b1700 1 --2- 172.21.15.29:0/1970906205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5c13a4b0 0x7fec5c139420 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:49.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.467+0000 7fec635b1700 1 -- 172.21.15.29:0/1970906205 >> 172.21.15.29:0/1970906205 conn(0x7fec5c0c7b10 msgr2=0x7fec5c0c4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:49.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.467+0000 7fec635b1700 1 -- 172.21.15.29:0/1970906205 shutdown_connections 2024-01-24T15:13:49.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:49.467+0000 7fec635b1700 1 -- 172.21.15.29:0/1970906205 wait complete. 2024-01-24T15:13:49.481 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-24T15:13:49.481 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T15:13:49.481 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T15:13:49.481 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-24T15:13:49.482 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-24T15:13:49.482 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-24T15:13:49.482 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-24T15:13:49.482 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T15:13:49.482 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T15:13:49.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:13:49.482 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T15:13:49.502 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T15:13:49.502 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T15:13:49.502 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T15:13:49.502 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:13:49.502 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 37821sec preferred_lft 37821sec 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T15:13:49.503 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::24bc:1cff:fe57:e704/64 scope link 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::2034:aaff:fef3:f12b/64 scope link 2024-01-24T15:13:49.504 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:13:49.505 INFO:teuthology.orchestra.run.smithi029.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T15:13:49.505 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether ba:ca:42:4a:9e:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T15:13:49.505 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::b8ca:42ff:fe4a:9e52/64 scope link 2024-01-24T15:13:49.505 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:13:49.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:13:49.505 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T15:13:49.573 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T15:13:49.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:13:49.575 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T15:13:49.642 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T15:13:49.642 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T15:13:49.642 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T15:13:49.642 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T15:13:49.643 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T15:13:49.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:13:49.643 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T15:13:49.643 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T15:13:49.644 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T15:13:49.644 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T15:13:49.644 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T15:13:49.644 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T15:13:49.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:13:49.765 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T15:13:49.785 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-24T15:13:49.786 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T15:13:49.786 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T15:13:49.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:13:49.786 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T15:13:49.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:13:49.861 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-01-24T15:13:49.923 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T15:13:49.923 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T15:13:50.017 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T15:13:50.017 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T15:13:50.039 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T15:13:50.039 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T15:13:50.042 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T15:13:50.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:13:50.042 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-24T15:13:50.074 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-24T15:13:50.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.253+0000 7f6dc8717700 1 Processor -- start 2024-01-24T15:13:50.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.253+0000 7f6dc8717700 1 -- start start 2024-01-24T15:13:50.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.254+0000 7f6dc8717700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6dc0064410 0x7f6dc01524d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:50.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.254+0000 7f6dc8717700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6dc0066160 0x7f6dc00647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:50.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.254+0000 7f6dc8717700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc00695f0 con 0x7f6dc0069a80 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.254+0000 7f6dc8717700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc0068550 con 0x7f6dc0066160 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.254+0000 7f6dc8717700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc0068740 con 0x7f6dc0064410 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc5cb2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6dc0066160 0x7f6dc00647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc6cb4700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6dc0069a80 0x7f6dc014f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34740/0 (socket says 172.21.15.29:34740) 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc6cb4700 1 -- 172.21.15.29:0/31155671 learned_addr learned my addr 172.21.15.29:0/31155671 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc64b3700 1 --2- 172.21.15.29:0/31155671 >> v2:172.21.15.163:3300/0 conn(0x7f6dc0064410 0x7f6dc01524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc5cb2700 1 -- 172.21.15.29:0/31155671 >> v2:172.21.15.163:3300/0 conn(0x7f6dc0064410 msgr2=0x7f6dc01524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc5cb2700 1 --2- 172.21.15.29:0/31155671 >> v2:172.21.15.163:3300/0 conn(0x7f6dc0064410 0x7f6dc01524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc5cb2700 1 -- 172.21.15.29:0/31155671 >> v1:172.21.15.29:6789/0 conn(0x7f6dc0069a80 legacy=0x7f6dc014f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc54b1700 1 -- 172.21.15.29:0/31155671 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583050170 0 0) 0x7f6dc00695f0 con 0x7f6dc0069a80 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc5cb2700 1 -- 172.21.15.29:0/31155671 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00689e0 con 0x7f6dc0066160 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc5cb2700 1 --2- 172.21.15.29:0/31155671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc0066160 0x7f6dc00647f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f6dc00c68c0 tx=0x7f6db0009060).ready entity=mon.0 client_cookie=c53b29e2c98c5dc5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc54b1700 1 -- 172.21.15.29:0/31155671 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db000b580 con 0x7f6dc0066160 2024-01-24T15:13:50.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc54b1700 1 -- 172.21.15.29:0/31155671 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db000b6e0 con 0x7f6dc0066160 2024-01-24T15:13:50.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.255+0000 7f6dc54b1700 1 -- 172.21.15.29:0/31155671 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db000ba60 con 0x7f6dc0066160 2024-01-24T15:13:50.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.256+0000 7f6dc8717700 1 -- 172.21.15.29:0/31155671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc0066160 msgr2=0x7f6dc00647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:50.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.256+0000 7f6dc8717700 1 --2- 172.21.15.29:0/31155671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc0066160 0x7f6dc00647f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f6dc00c68c0 tx=0x7f6db0009060).stop 2024-01-24T15:13:50.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.256+0000 7f6dc8717700 1 -- 172.21.15.29:0/31155671 shutdown_connections 2024-01-24T15:13:50.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.256+0000 7f6dc8717700 1 --2- 172.21.15.29:0/31155671 >> v2:172.21.15.163:3300/0 conn(0x7f6dc0064410 0x7f6dc01524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:50.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.256+0000 7f6dc8717700 1 --2- 172.21.15.29:0/31155671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc0066160 0x7f6dc00647f0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:50.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.256+0000 7f6dc8717700 1 -- 172.21.15.29:0/31155671 >> 172.21.15.29:0/31155671 conn(0x7f6dc00c7480 msgr2=0x7f6dc00c3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:50.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.256+0000 7f6dc8717700 1 -- 172.21.15.29:0/31155671 shutdown_connections 2024-01-24T15:13:50.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.256+0000 7f6dc8717700 1 -- 172.21.15.29:0/31155671 wait complete. 2024-01-24T15:13:50.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.258+0000 7f6dc8717700 1 Processor -- start 2024-01-24T15:13:50.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.258+0000 7f6dc8717700 1 -- start start 2024-01-24T15:13:50.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.258+0000 7f6dc8717700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dc0064410 0x7f6dc0156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:50.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.258+0000 7f6dc8717700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0066160 0x7f6dc00ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:50.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.258+0000 7f6dc8717700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc0069a80 0x7f6dc0135790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:50.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.258+0000 7f6dc8717700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc00abac0 con 0x7f6dc0069a80 2024-01-24T15:13:50.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.258+0000 7f6dc8717700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0156f90 con 0x7f6dc0066160 2024-01-24T15:13:50.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.258+0000 7f6dc8717700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0135c80 con 0x7f6dc0064410 2024-01-24T15:13:50.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc6cb4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc0069a80 0x7f6dc0135790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc6cb4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc0069a80 0x7f6dc0135790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40636/0 (socket says 172.21.15.29:40636) 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc6cb4700 1 -- 172.21.15.29:0/2294698242 learned_addr learned my addr 172.21.15.29:0/2294698242 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc64b3700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dc0064410 0x7f6dc0156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc5cb2700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0066160 0x7f6dc00ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc6cb4700 1 -- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dc0064410 msgr2=0x7f6dc0156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc6cb4700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dc0064410 0x7f6dc0156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc6cb4700 1 -- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0066160 msgr2=0x7f6dc00ab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc6cb4700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0066160 0x7f6dc00ab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc6cb4700 1 -- 172.21.15.29:0/2294698242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db00092a0 con 0x7f6dc0069a80 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc5cb2700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0066160 0x7f6dc00ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc6cb4700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc0069a80 0x7f6dc0135790 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f6db800c760 tx=0x7f6db8001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6db67fc700 1 -- 172.21.15.29:0/2294698242 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8005d30 con 0x7f6dc0069a80 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6db67fc700 1 -- 172.21.15.29:0/2294698242 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db8005e90 con 0x7f6dc0069a80 2024-01-24T15:13:50.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc8717700 1 -- 172.21.15.29:0/2294698242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc0135f00 con 0x7f6dc0069a80 2024-01-24T15:13:50.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6db67fc700 1 -- 172.21.15.29:0/2294698242 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8007340 con 0x7f6dc0069a80 2024-01-24T15:13:50.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.259+0000 7f6dc8717700 1 -- 172.21.15.29:0/2294698242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc0156940 con 0x7f6dc0069a80 2024-01-24T15:13:50.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.260+0000 7f6dc8717700 1 -- 172.21.15.29:0/2294698242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc0157100 con 0x7f6dc0069a80 2024-01-24T15:13:50.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.261+0000 7f6db67fc700 1 -- 172.21.15.29:0/2294698242 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6db80076f0 con 0x7f6dc0069a80 2024-01-24T15:13:50.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.261+0000 7f6db67fc700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6da405f270 0x7f6da4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:50.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.261+0000 7f6db67fc700 1 -- 172.21.15.29:0/2294698242 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f6db807a7c0 con 0x7f6dc0069a80 2024-01-24T15:13:50.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.261+0000 7f6dc64b3700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6da405f270 0x7f6da4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:50.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.262+0000 7f6dc64b3700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6da405f270 0x7f6da4061730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f6dbc00bb20 tx=0x7f6dbc008040).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:50.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.266+0000 7f6db67fc700 1 -- 172.21.15.29:0/2294698242 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f6db804b4e0 con 0x7f6dc0069a80 2024-01-24T15:13:50.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.476+0000 7f6dc8717700 1 -- 172.21.15.29:0/2294698242 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dc0068cc0 con 0x7f6da405f270 2024-01-24T15:13:50.480 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.478+0000 7f6db67fc700 1 -- 172.21.15.29:0/2294698242 <== mgr.8056 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6dc0068cc0 con 0x7f6da405f270 2024-01-24T15:13:50.480 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T15:13:50.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:13:50.481 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-01-24T15:13:50.481 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:13:50.481 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T15:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.480+0000 7f6dc8717700 1 -- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6da405f270 msgr2=0x7f6da4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.480+0000 7f6dc8717700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6da405f270 0x7f6da4061730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f6dbc00bb20 tx=0x7f6dbc008040).stop 2024-01-24T15:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.480+0000 7f6dc8717700 1 -- 172.21.15.29:0/2294698242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc0069a80 msgr2=0x7f6dc0135790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:50.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.480+0000 7f6dc8717700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc0069a80 0x7f6dc0135790 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f6db800c760 tx=0x7f6db8001520).stop 2024-01-24T15:13:50.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.480+0000 7f6dc8717700 1 -- 172.21.15.29:0/2294698242 shutdown_connections 2024-01-24T15:13:50.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.480+0000 7f6dc8717700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f6da405f270 0x7f6da4061730 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:50.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.480+0000 7f6dc8717700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc0069a80 0x7f6dc0135790 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:50.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.480+0000 7f6dc8717700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dc0064410 0x7f6dc0156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:50.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.480+0000 7f6dc8717700 1 --2- 172.21.15.29:0/2294698242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0066160 0x7f6dc00ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:50.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.480+0000 7f6dc8717700 1 -- 172.21.15.29:0/2294698242 >> 172.21.15.29:0/2294698242 conn(0x7f6dc00c7480 msgr2=0x7f6dc0154820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:50.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.480+0000 7f6dc8717700 1 -- 172.21.15.29:0/2294698242 shutdown_connections 2024-01-24T15:13:50.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.480+0000 7f6dc8717700 1 -- 172.21.15.29:0/2294698242 wait complete. 2024-01-24T15:13:50.496 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-01-24T15:13:50.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.630+0000 7f9ecf09b700 1 Processor -- start 2024-01-24T15:13:50.633 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.631+0000 7f9ecf09b700 1 -- start start 2024-01-24T15:13:50.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.632+0000 7f9ecf09b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec8134e70 0x7f9ec80c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:50.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.632+0000 7f9ecf09b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec8068470 con 0x7f9ec8068900 2024-01-24T15:13:50.635 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.632+0000 7f9ecf09b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec80c3860 con 0x7f9ec8066c30 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.632+0000 7f9ecf09b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec814c910 con 0x7f9ec8134e70 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.632+0000 7f9ec7fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9ec8068900 0x7f9ec8066520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34752/0 (socket says 172.21.15.29:34752) 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.632+0000 7f9ec7fff700 1 -- 172.21.15.29:0/3433452364 learned_addr learned my addr 172.21.15.29:0/3433452364 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.632+0000 7f9ecce37700 1 --2- 172.21.15.29:0/3433452364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec8134e70 0x7f9ec80c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.633+0000 7f9ec77fe700 1 -- 172.21.15.29:0/3433452364 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634189059 0 0) 0x7f9ec8068470 con 0x7f9ec8068900 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.633+0000 7f9ec77fe700 1 -- 172.21.15.29:0/3433452364 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb80033f0 con 0x7f9ec8068900 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.633+0000 7f9ec77fe700 1 -- 172.21.15.29:0/3433452364 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028719235 0 0) 0x7f9ec80c3860 con 0x7f9ec8066c30 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.633+0000 7f9ec77fe700 1 -- 172.21.15.29:0/3433452364 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec8068470 con 0x7f9ec8066c30 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.633+0000 7f9ec77fe700 1 -- 172.21.15.29:0/3433452364 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2134015405 0 0) 0x7f9eb80033f0 con 0x7f9ec8068900 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.633+0000 7f9ecce37700 1 -- 172.21.15.29:0/3433452364 >> v1:172.21.15.163:6789/0 conn(0x7f9ec8066c30 legacy=0x7f9ec80bdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.633+0000 7f9ecce37700 1 -- 172.21.15.29:0/3433452364 >> v1:172.21.15.29:6789/0 conn(0x7f9ec8068900 legacy=0x7f9ec8066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.633+0000 7f9ecce37700 1 -- 172.21.15.29:0/3433452364 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec8067b40 con 0x7f9ec8134e70 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.633+0000 7f9ecce37700 1 --2- 172.21.15.29:0/3433452364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec8134e70 0x7f9ec80c31a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f9ebc009790 tx=0x7f9ebc009060).ready entity=mon.2 client_cookie=ab3c4ea55bb0292d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:50.636 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.633+0000 7f9ec77fe700 1 -- 172.21.15.29:0/3433452364 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4001470 con 0x7f9ec8134e70 2024-01-24T15:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.633+0000 7f9ec77fe700 1 -- 172.21.15.29:0/3433452364 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc011400 con 0x7f9ec8134e70 2024-01-24T15:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.633+0000 7f9ec77fe700 1 -- 172.21.15.29:0/3433452364 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc011780 con 0x7f9ec8134e70 2024-01-24T15:13:50.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.634+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3433452364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec8134e70 msgr2=0x7f9ec80c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:50.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.634+0000 7f9ecf09b700 1 --2- 172.21.15.29:0/3433452364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec8134e70 0x7f9ec80c31a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f9ebc009790 tx=0x7f9ebc009060).stop 2024-01-24T15:13:50.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.634+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3433452364 shutdown_connections 2024-01-24T15:13:50.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.634+0000 7f9ecf09b700 1 --2- 172.21.15.29:0/3433452364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec8134e70 0x7f9ec80c31a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:50.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.634+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3433452364 >> 172.21.15.29:0/3433452364 conn(0x7f9ec80c8ba0 msgr2=0x7f9ec80c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:50.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.635+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3433452364 shutdown_connections 2024-01-24T15:13:50.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.635+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3433452364 wait complete. 2024-01-24T15:13:50.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.636+0000 7f9ecf09b700 1 Processor -- start 2024-01-24T15:13:50.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.637+0000 7f9ecf09b700 1 -- start start 2024-01-24T15:13:50.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.638+0000 7f9ecf09b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec8066c30 0x7f9ec80654b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:50.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.638+0000 7f9ecf09b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec8068900 0x7f9ec8149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:50.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.638+0000 7f9ecf09b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ec8134e70 0x7f9ec8164ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:50.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.638+0000 7f9ecf09b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec8065020 con 0x7f9ec8134e70 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.638+0000 7f9ecf09b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec8064410 con 0x7f9ec8066c30 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.638+0000 7f9ecf09b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80646f0 con 0x7f9ec8068900 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.638+0000 7f9ecd638700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ec8134e70 0x7f9ec8164ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.638+0000 7f9ecce37700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec8066c30 0x7f9ec80654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.638+0000 7f9ecd638700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ec8134e70 0x7f9ec8164ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40642/0 (socket says 172.21.15.29:40642) 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.638+0000 7f9ecce37700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec8066c30 0x7f9ec80654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:34180/0 (socket says 172.21.15.29:34180) 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.638+0000 7f9ecd638700 1 -- 172.21.15.29:0/3022451535 learned_addr learned my addr 172.21.15.29:0/3022451535 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.638+0000 7f9ec7fff700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec8068900 0x7f9ec8149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.639+0000 7f9ecd638700 1 -- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec8068900 msgr2=0x7f9ec8149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.639+0000 7f9ecd638700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec8068900 0x7f9ec8149450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.639+0000 7f9ecd638700 1 -- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec8066c30 msgr2=0x7f9ec80654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.639+0000 7f9ecd638700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec8066c30 0x7f9ec80654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.639+0000 7f9ecd638700 1 -- 172.21.15.29:0/3022451535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc0092a0 con 0x7f9ec8134e70 2024-01-24T15:13:50.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.639+0000 7f9ecd638700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ec8134e70 0x7f9ec8164ab0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f9ec000c7c0 tx=0x7f9ec0007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:50.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.639+0000 7f9ec4ff9700 1 -- 172.21.15.29:0/3022451535 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0010510 con 0x7f9ec8134e70 2024-01-24T15:13:50.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.639+0000 7f9ec4ff9700 1 -- 172.21.15.29:0/3022451535 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec0010670 con 0x7f9ec8134e70 2024-01-24T15:13:50.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.639+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3022451535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec80649d0 con 0x7f9ec8134e70 2024-01-24T15:13:50.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.639+0000 7f9ec4ff9700 1 -- 172.21.15.29:0/3022451535 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00109f0 con 0x7f9ec8134e70 2024-01-24T15:13:50.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.639+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3022451535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec81645f0 con 0x7f9ec8134e70 2024-01-24T15:13:50.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.640+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3022451535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec80651d0 con 0x7f9ec8134e70 2024-01-24T15:13:50.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.641+0000 7f9ec4ff9700 1 -- 172.21.15.29:0/3022451535 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 67) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9ec001b400 con 0x7f9ec8134e70 2024-01-24T15:13:50.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.641+0000 7f9ec4ff9700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9eac05f2a0 0x7f9eac061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:50.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.641+0000 7f9ec4ff9700 1 -- 172.21.15.29:0/3022451535 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f9ec007a5b0 con 0x7f9ec8134e70 2024-01-24T15:13:50.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.641+0000 7f9ecce37700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9eac05f2a0 0x7f9eac061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:50.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.642+0000 7f9ecce37700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9eac05f2a0 0x7f9eac061760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f9ebc0096e0 tx=0x7f9ebc016000).ready entity=mgr.8056 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:50.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.645+0000 7f9ec4ff9700 1 -- 172.21.15.29:0/3022451535 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9ec80651d0 con 0x7f9ec8134e70 2024-01-24T15:13:50.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:50.876+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3022451535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f9ec80659a0 con 0x7f9ec8134e70 2024-01-24T15:13:51.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.644+0000 7f9ec4ff9700 1 -- 172.21.15.29:0/3022451535 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 68) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f9ec0018d60 con 0x7f9ec8134e70 2024-01-24T15:13:51.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.645+0000 7f9ec4ff9700 1 -- 172.21.15.29:0/3022451535 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v68) v1 ==== 92+0+0 (secure 0 0 0) 0x7f9ec0073120 con 0x7f9ec8134e70 2024-01-24T15:13:51.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.647+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9eac05f2a0 msgr2=0x7f9eac061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:51.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.647+0000 7f9ecf09b700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9eac05f2a0 0x7f9eac061760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f9ebc0096e0 tx=0x7f9ebc016000).stop 2024-01-24T15:13:51.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.647+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3022451535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ec8134e70 msgr2=0x7f9ec8164ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:51.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.647+0000 7f9ecf09b700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ec8134e70 0x7f9ec8164ab0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f9ec000c7c0 tx=0x7f9ec0007f50).stop 2024-01-24T15:13:51.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.648+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3022451535 shutdown_connections 2024-01-24T15:13:51.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.648+0000 7f9ecf09b700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9eac05f2a0 0x7f9eac061760 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:51.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.648+0000 7f9ecf09b700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ec8134e70 0x7f9ec8164ab0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:51.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.648+0000 7f9ecf09b700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec8066c30 0x7f9ec80654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:51.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.648+0000 7f9ecf09b700 1 --2- 172.21.15.29:0/3022451535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec8068900 0x7f9ec8149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:51.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.648+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3022451535 >> 172.21.15.29:0/3022451535 conn(0x7f9ec80c8ba0 msgr2=0x7f9ec8134440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:51.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.648+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3022451535 shutdown_connections 2024-01-24T15:13:51.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.648+0000 7f9ecf09b700 1 -- 172.21.15.29:0/3022451535 wait complete. 2024-01-24T15:13:51.666 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-01-24T15:13:51.793 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:51.792+0000 7f83f161c680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:13:51.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.805+0000 7f61b2588700 1 Processor -- start 2024-01-24T15:13:51.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.805+0000 7f61b2588700 1 -- start start 2024-01-24T15:13:51.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.806+0000 7f61b2588700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f61ac064760 0x7f61ac064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:51.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.806+0000 7f61b2588700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61ac0c8490 0x7f61ac0a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:51.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.806+0000 7f61b2588700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac152240 con 0x7f61ac154700 2024-01-24T15:13:51.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.806+0000 7f61b2588700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ac1586c0 con 0x7f61ac064760 2024-01-24T15:13:51.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.806+0000 7f61b2588700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0c0d50 con 0x7f61ac0c8490 2024-01-24T15:13:51.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.806+0000 7f61abfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f61ac064760 0x7f61ac064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:51.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.806+0000 7f61abfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f61ac064760 0x7f61ac064b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40654/0 (socket says 172.21.15.29:40654) 2024-01-24T15:13:51.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.806+0000 7f61abfff700 1 -- 172.21.15.29:0/282185121 learned_addr learned my addr 172.21.15.29:0/282185121 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:51.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.806+0000 7f61ab7fe700 1 --2- 172.21.15.29:0/282185121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61ac0c8490 0x7f61ac0a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:51.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.807+0000 7f61abfff700 1 -- 172.21.15.29:0/282185121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61ac0c8490 msgr2=0x7f61ac0a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:51.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.807+0000 7f61abfff700 1 --2- 172.21.15.29:0/282185121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61ac0c8490 0x7f61ac0a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:51.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.807+0000 7f61abfff700 1 -- 172.21.15.29:0/282185121 >> v1:172.21.15.163:6789/0 conn(0x7f61ac154700 legacy=0x7f61ac154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:51.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.807+0000 7f61aaffd700 1 -- 172.21.15.29:0/282185121 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596352163 0 0) 0x7f61ac152240 con 0x7f61ac154700 2024-01-24T15:13:51.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.807+0000 7f61abfff700 1 -- 172.21.15.29:0/282185121 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac0aa670 con 0x7f61ac064760 2024-01-24T15:13:51.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.807+0000 7f61abfff700 1 --2- 172.21.15.29:0/282185121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61ac064760 0x7f61ac064b40 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f619c006fd0 tx=0x7f619c0090f0).ready entity=mon.0 client_cookie=b02275e30c4bfe7a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:51.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.807+0000 7f61aaffd700 1 -- 172.21.15.29:0/282185121 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c00b610 con 0x7f61ac064760 2024-01-24T15:13:51.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.807+0000 7f61aaffd700 1 -- 172.21.15.29:0/282185121 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c00b770 con 0x7f61ac064760 2024-01-24T15:13:51.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.807+0000 7f61aaffd700 1 -- 172.21.15.29:0/282185121 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c00baf0 con 0x7f61ac064760 2024-01-24T15:13:51.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.808+0000 7f61b2588700 1 -- 172.21.15.29:0/282185121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61ac064760 msgr2=0x7f61ac064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:51.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.808+0000 7f61b2588700 1 --2- 172.21.15.29:0/282185121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61ac064760 0x7f61ac064b40 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f619c006fd0 tx=0x7f619c0090f0).stop 2024-01-24T15:13:51.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.808+0000 7f61b2588700 1 -- 172.21.15.29:0/282185121 shutdown_connections 2024-01-24T15:13:51.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.808+0000 7f61b2588700 1 --2- 172.21.15.29:0/282185121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61ac064760 0x7f61ac064b40 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:51.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.808+0000 7f61b2588700 1 --2- 172.21.15.29:0/282185121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61ac0c8490 0x7f61ac0a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:51.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.808+0000 7f61b2588700 1 -- 172.21.15.29:0/282185121 >> 172.21.15.29:0/282185121 conn(0x7f61ac0cb8b0 msgr2=0x7f61ac0c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:51.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.808+0000 7f61b2588700 1 -- 172.21.15.29:0/282185121 shutdown_connections 2024-01-24T15:13:51.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.808+0000 7f61b2588700 1 -- 172.21.15.29:0/282185121 wait complete. 2024-01-24T15:13:51.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.809+0000 7f61b2588700 1 Processor -- start 2024-01-24T15:13:51.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.809+0000 7f61b2588700 1 -- start start 2024-01-24T15:13:51.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.810+0000 7f61b2588700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61ac064760 0x7f61ac0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:51.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.810+0000 7f61b2588700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61ac0c8490 0x7f61ac0c2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:51.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.810+0000 7f61b2588700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61ac154700 0x7f61ac0c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:51.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.810+0000 7f61b2588700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0ae2f0 con 0x7f61ac154700 2024-01-24T15:13:51.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.810+0000 7f61b2588700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0c3980 con 0x7f61ac064760 2024-01-24T15:13:51.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.810+0000 7f61b2588700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0c2d70 con 0x7f61ac0c8490 2024-01-24T15:13:51.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.810+0000 7f61b0b25700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61ac154700 0x7f61ac0c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:51.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.810+0000 7f61abfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61ac064760 0x7f61ac0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:51.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.810+0000 7f61ab7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61ac0c8490 0x7f61ac0c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:51.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.810+0000 7f61abfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61ac064760 0x7f61ac0ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:34218/0 (socket says 172.21.15.29:34218) 2024-01-24T15:13:51.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.810+0000 7f61b0b25700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61ac154700 0x7f61ac0c2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40656/0 (socket says 172.21.15.29:40656) 2024-01-24T15:13:51.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.810+0000 7f61abfff700 1 -- 172.21.15.29:0/3780943031 learned_addr learned my addr 172.21.15.29:0/3780943031 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:51.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f61abfff700 1 -- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61ac0c8490 msgr2=0x7f61ac0c2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:51.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f61abfff700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61ac0c8490 0x7f61ac0c2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:51.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f61abfff700 1 -- 172.21.15.29:0/3780943031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61ac154700 msgr2=0x7f61ac0c2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:51.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f61abfff700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61ac154700 0x7f61ac0c2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:51.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f61abfff700 1 -- 172.21.15.29:0/3780943031 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c009370 con 0x7f61ac064760 2024-01-24T15:13:51.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f61ab7fe700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61ac0c8490 0x7f61ac0c2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:51.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f61abfff700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61ac064760 0x7f61ac0ade00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f619c009130 tx=0x7f619c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:51.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f6193fff700 1 -- 172.21.15.29:0/3780943031 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c00beb0 con 0x7f61ac064760 2024-01-24T15:13:51.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f6193fff700 1 -- 172.21.15.29:0/3780943031 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c00b8e0 con 0x7f61ac064760 2024-01-24T15:13:51.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f61b2588700 1 -- 172.21.15.29:0/3780943031 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ac0c3050 con 0x7f61ac064760 2024-01-24T15:13:51.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f6193fff700 1 -- 172.21.15.29:0/3780943031 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c01c670 con 0x7f61ac064760 2024-01-24T15:13:51.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f61b2588700 1 -- 172.21.15.29:0/3780943031 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac0c3610 con 0x7f61ac064760 2024-01-24T15:13:51.817 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:51.811+0000 7f9869f01680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:13:51.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.813+0000 7f61b2588700 1 -- 172.21.15.29:0/3780943031 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ac0475e0 con 0x7f61ac064760 2024-01-24T15:13:51.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.813+0000 7f6193fff700 1 -- 172.21.15.29:0/3780943031 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 68) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f619c01ca20 con 0x7f61ac064760 2024-01-24T15:13:51.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.814+0000 7f6193fff700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f619805f2c0 0x7f6198061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:51.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.814+0000 7f6193fff700 1 -- 172.21.15.29:0/3780943031 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f619c07c420 con 0x7f61ac064760 2024-01-24T15:13:51.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.814+0000 7f61ab7fe700 1 -- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f619805f2c0 msgr2=0x7f6198061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:51.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.814+0000 7f61ab7fe700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f619805f2c0 0x7f6198061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:51.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:51.822+0000 7f6193fff700 1 -- 172.21.15.29:0/3780943031 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f619c04d140 con 0x7f61ac064760 2024-01-24T15:13:51.861 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:51.860+0000 7f83f161c680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:13:51.889 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:51.887+0000 7f9869f01680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:13:52.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.008+0000 7f61b2588700 1 -- 172.21.15.29:0/3780943031 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f61ac157d90 con 0x7f61ac064760 2024-01-24T15:13:52.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.014+0000 7f6193fff700 1 -- 172.21.15.29:0/3780943031 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f619c050760 con 0x7f61ac064760 2024-01-24T15:13:52.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.015+0000 7f61ab7fe700 1 -- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f619805f2c0 msgr2=0x7f6198061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:52.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.015+0000 7f61ab7fe700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f619805f2c0 0x7f6198061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:13:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.017+0000 7f61b2588700 1 -- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f619805f2c0 msgr2=0x7f6198061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.017+0000 7f61b2588700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f619805f2c0 0x7f6198061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.017+0000 7f61b2588700 1 -- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61ac064760 msgr2=0x7f61ac0ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.017+0000 7f61b2588700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61ac064760 0x7f61ac0ade00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f619c009130 tx=0x7f619c017000).stop 2024-01-24T15:13:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.017+0000 7f61b2588700 1 -- 172.21.15.29:0/3780943031 shutdown_connections 2024-01-24T15:13:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.017+0000 7f61b2588700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f619805f2c0 0x7f6198061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.017+0000 7f61b2588700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61ac154700 0x7f61ac0c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:52.019 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.017+0000 7f61b2588700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61ac064760 0x7f61ac0ade00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:52.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.017+0000 7f61b2588700 1 --2- 172.21.15.29:0/3780943031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61ac0c8490 0x7f61ac0c2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:52.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.017+0000 7f61b2588700 1 -- 172.21.15.29:0/3780943031 >> 172.21.15.29:0/3780943031 conn(0x7f61ac0cb8b0 msgr2=0x7f61ac1504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:52.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.017+0000 7f61b2588700 1 -- 172.21.15.29:0/3780943031 shutdown_connections 2024-01-24T15:13:52.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.017+0000 7f61b2588700 1 -- 172.21.15.29:0/3780943031 wait complete. 2024-01-24T15:13:52.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:52.034 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-01-24T15:13:52.061 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1h 2024-01-24T15:13:52.136 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:52.134+0000 7f83f161c680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:13:52.211 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:52.209+0000 7f9869f01680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:13:52.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.239+0000 7fabe2da6700 1 Processor -- start 2024-01-24T15:13:52.241 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.239+0000 7fabe2da6700 1 -- start start 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.240+0000 7fabe2da6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fabdc151e50 0x7fabdc0c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.240+0000 7fabe2da6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c7f70 0x7fabdc155420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.240+0000 7fabe2da6700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc154f90 con 0x7fabdc064780 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.240+0000 7fabe2da6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabdc155910 con 0x7fabdc151e50 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.240+0000 7fabe2da6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0a9f10 con 0x7fabdc0c7f70 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabe1343700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fabdc064780 0x7fabdc0bfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:34762/0 (socket says 172.21.15.29:34762) 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabe1343700 1 -- 172.21.15.29:0/1212931271 learned_addr learned my addr 172.21.15.29:0/1212931271 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabdbfff700 1 --2- 172.21.15.29:0/1212931271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c7f70 0x7fabdc155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabe0b42700 1 --2- 172.21.15.29:0/1212931271 >> v2:172.21.15.163:3300/0 conn(0x7fabdc151e50 0x7fabdc0c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabdb7fe700 1 -- 172.21.15.29:0/1212931271 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570324088 0 0) 0x7fabdc154f90 con 0x7fabdc064780 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabdb7fe700 1 -- 172.21.15.29:0/1212931271 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabcc0033f0 con 0x7fabdc064780 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabe0b42700 1 -- 172.21.15.29:0/1212931271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c7f70 msgr2=0x7fabdc155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:52.244 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabe0b42700 1 --2- 172.21.15.29:0/1212931271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c7f70 0x7fabdc155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:52.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabe0b42700 1 -- 172.21.15.29:0/1212931271 >> v1:172.21.15.29:6789/0 conn(0x7fabdc064780 legacy=0x7fabdc0bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:52.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabe0b42700 1 -- 172.21.15.29:0/1212931271 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc0aa1f0 con 0x7fabdc151e50 2024-01-24T15:13:52.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabdbfff700 1 --2- 172.21.15.29:0/1212931271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c7f70 0x7fabdc155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:52.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabe0b42700 1 --2- 172.21.15.29:0/1212931271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc151e50 0x7fabdc0c53c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fabd0009760 tx=0x7fabd0009060).ready entity=mon.1 client_cookie=ce39ecee40ffaad2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:52.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.241+0000 7fabdb7fe700 1 -- 172.21.15.29:0/1212931271 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd000d4a0 con 0x7fabdc151e50 2024-01-24T15:13:52.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.242+0000 7fabdb7fe700 1 -- 172.21.15.29:0/1212931271 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd000d600 con 0x7fabdc151e50 2024-01-24T15:13:52.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.242+0000 7fabdb7fe700 1 -- 172.21.15.29:0/1212931271 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd000d980 con 0x7fabdc151e50 2024-01-24T15:13:52.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.242+0000 7fabe2da6700 1 -- 172.21.15.29:0/1212931271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc151e50 msgr2=0x7fabdc0c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:52.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.242+0000 7fabe2da6700 1 --2- 172.21.15.29:0/1212931271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc151e50 0x7fabdc0c53c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fabd0009760 tx=0x7fabd0009060).stop 2024-01-24T15:13:52.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.243+0000 7fabe2da6700 1 -- 172.21.15.29:0/1212931271 shutdown_connections 2024-01-24T15:13:52.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.243+0000 7fabe2da6700 1 --2- 172.21.15.29:0/1212931271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc151e50 0x7fabdc0c53c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:52.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.243+0000 7fabe2da6700 1 --2- 172.21.15.29:0/1212931271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c7f70 0x7fabdc155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:52.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.243+0000 7fabe2da6700 1 -- 172.21.15.29:0/1212931271 >> 172.21.15.29:0/1212931271 conn(0x7fabdc0cb880 msgr2=0x7fabdc0c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:52.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.243+0000 7fabe2da6700 1 -- 172.21.15.29:0/1212931271 shutdown_connections 2024-01-24T15:13:52.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.243+0000 7fabe2da6700 1 -- 172.21.15.29:0/1212931271 wait complete. 2024-01-24T15:13:52.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.245+0000 7fabe2da6700 1 Processor -- start 2024-01-24T15:13:52.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.245+0000 7fabe2da6700 1 -- start start 2024-01-24T15:13:52.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.246+0000 7fabe2da6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc064780 0x7fabdc14f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:52.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.246+0000 7fabe2da6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c7f70 0x7fabdc0b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:52.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.246+0000 7fabe2da6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc151e50 0x7fabdc0b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.246+0000 7fabe2da6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc150d50 con 0x7fabdc151e50 2024-01-24T15:13:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.246+0000 7fabe2da6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc1508c0 con 0x7fabdc064780 2024-01-24T15:13:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.246+0000 7fabe2da6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0b0cd0 con 0x7fabdc0c7f70 2024-01-24T15:13:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.246+0000 7fabe0b42700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc064780 0x7fabdc14f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.246+0000 7fabdbfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c7f70 0x7fabdc0b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.246+0000 7fabe0b42700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc064780 0x7fabdc14f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:34230/0 (socket says 172.21.15.29:34230) 2024-01-24T15:13:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.246+0000 7fabe1343700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc151e50 0x7fabdc0b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.246+0000 7fabe0b42700 1 -- 172.21.15.29:0/51407028 learned_addr learned my addr 172.21.15.29:0/51407028 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.247+0000 7fabe0b42700 1 -- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c7f70 msgr2=0x7fabdc0b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.247+0000 7fabe0b42700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c7f70 0x7fabdc0b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.247+0000 7fabe0b42700 1 -- 172.21.15.29:0/51407028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc151e50 msgr2=0x7fabdc0b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.247+0000 7fabe0b42700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc151e50 0x7fabdc0b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.247+0000 7fabe0b42700 1 -- 172.21.15.29:0/51407028 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc80092a0 con 0x7fabdc064780 2024-01-24T15:13:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.247+0000 7fabe1343700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc151e50 0x7fabdc0b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.247+0000 7fabe0b42700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc064780 0x7fabdc14f3b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fabd0009d50 tx=0x7fabd0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.247+0000 7fabd8ff9700 1 -- 172.21.15.29:0/51407028 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd000dd30 con 0x7fabdc064780 2024-01-24T15:13:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.247+0000 7fabd8ff9700 1 -- 172.21.15.29:0/51407028 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd000de90 con 0x7fabdc064780 2024-01-24T15:13:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.247+0000 7fabd8ff9700 1 -- 172.21.15.29:0/51407028 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd000d980 con 0x7fabdc064780 2024-01-24T15:13:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.247+0000 7fabe2da6700 1 -- 172.21.15.29:0/51407028 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd00092a0 con 0x7fabdc064780 2024-01-24T15:13:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.247+0000 7fabe2da6700 1 -- 172.21.15.29:0/51407028 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc14ff90 con 0x7fabdc064780 2024-01-24T15:13:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.249+0000 7fabd8ff9700 1 -- 172.21.15.29:0/51407028 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 68) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fabd001d640 con 0x7fabdc064780 2024-01-24T15:13:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.249+0000 7fabe2da6700 1 -- 172.21.15.29:0/51407028 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc0aa710 con 0x7fabdc064780 2024-01-24T15:13:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.249+0000 7fabd8ff9700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 0x7fabc0061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.249+0000 7fabd8ff9700 1 -- 172.21.15.29:0/51407028 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fabd007bf10 con 0x7fabdc064780 2024-01-24T15:13:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.249+0000 7fabdbfff700 1 -- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 msgr2=0x7fabc0061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:52.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.249+0000 7fabdbfff700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 0x7fabc0061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:52.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.258+0000 7fabd8ff9700 1 -- 172.21.15.29:0/51407028 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fabd004cb70 con 0x7fabdc064780 2024-01-24T15:13:52.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.450+0000 7fabdbfff700 1 -- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 msgr2=0x7fabc0061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:52.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.450+0000 7fabdbfff700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 0x7fabc0061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:13:52.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.468+0000 7fabe2da6700 1 -- 172.21.15.29:0/51407028 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fabdc150a00 con 0x7fabc005f2c0 2024-01-24T15:13:52.755 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:52.753+0000 7f83f161c680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:13:52.826 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:52.825+0000 7f83f161c680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:13:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.851+0000 7fabdbfff700 1 -- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 msgr2=0x7fabc0061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:52.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:52.851+0000 7fabdbfff700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 0x7fabc0061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-01-24T15:13:52.940 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:52.938+0000 7f9869f01680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:13:53.013 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:53.011+0000 7f83f161c680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:13:53.014 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:53.013+0000 7f9869f01680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:13:53.229 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:53.227+0000 7f9869f01680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:13:53.434 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:53.433+0000 7f83f161c680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:13:53.538 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:53.536+0000 7f83f161c680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:13:53.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:53.652+0000 7fabdbfff700 1 -- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 msgr2=0x7fabc0061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:53.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:53.652+0000 7fabdbfff700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 0x7fabc0061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-01-24T15:13:53.691 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:53.689+0000 7f83f161c680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:13:53.710 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:53.705+0000 7f9869f01680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:13:53.769 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:53.764+0000 7f83f161c680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:13:53.825 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:53.822+0000 7f9869f01680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:13:53.833 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:53.831+0000 7f83f161c680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:13:54.001 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:53.998+0000 7f9869f01680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:13:54.085 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:54.083+0000 7f9869f01680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:13:54.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:54.087+0000 7f83f161c680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:13:54.164 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:54.161+0000 7f9869f01680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:13:54.274 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:54.273+0000 7f83f161c680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:13:54.455 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:54.453+0000 7f9869f01680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:13:54.668 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:54.665+0000 7f9869f01680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:13:54.929 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:54.928+0000 7f83f161c680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:13:54.999 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:54.997+0000 7f83f161c680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:13:55.074 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.073+0000 7f83f161c680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:13:55.221 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.219+0000 7f83f161c680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:13:55.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:55.254+0000 7fabdbfff700 1 -- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 msgr2=0x7fabc0061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:55.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:55.254+0000 7fabdbfff700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 0x7fabc0061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-01-24T15:13:55.291 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.290+0000 7f83f161c680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:13:55.405 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.403+0000 7f83f161c680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:13:55.417 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:55.415+0000 7f9869f01680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:13:55.495 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:55.493+0000 7f9869f01680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:13:55.511 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.510+0000 7f83f161c680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:13:55.580 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:55.578+0000 7f9869f01680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:13:55.745 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:55.743+0000 7f9869f01680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:13:55.803 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.801+0000 7f83f161c680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:13:55.824 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:55.822+0000 7f9869f01680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:13:55.873 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.872+0000 7f83f161c680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:13:55.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:55.892+0000 7fabd8ff9700 1 -- 172.21.15.29:0/51407028 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mgrmap(e 69) v1 ==== 75287+0+0 (secure 0 0 0) 0x7fabd0079670 con 0x7fabdc064780 2024-01-24T15:13:55.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:55.893+0000 7fabd8ff9700 1 -- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 msgr2=0x7fabc0061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:55.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:55.893+0000 7fabd8ff9700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 0x7fabc0061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:55.962 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:55.960+0000 7f9869f01680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:13:55.965 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.963+0000 7f8364390700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:55.965 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.963+0000 7f8364390700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:55.965 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.963+0000 7f8364390700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:55.965 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.963+0000 7f8364390700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:55.965 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.963+0000 7f8364390700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:55.965 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.964+0000 7f8368398700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:55.966 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.964+0000 7f8368398700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:55.966 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.964+0000 7f8368398700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:55.966 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.964+0000 7f8368398700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:55.966 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:55.964+0000 7f8368398700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:13:56.087 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:56.084+0000 7f9869f01680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:13:56.421 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:56.419+0000 7f9869f01680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:13:56.501 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:56.499+0000 7f9869f01680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:13:56.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.901+0000 7fabd8ff9700 1 -- 172.21.15.29:0/51407028 <== mon.1 v2:172.21.15.163:3300/0 8 ==== mgrmap(e 70) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fabd00799f0 con 0x7fabdc064780 2024-01-24T15:13:56.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.901+0000 7fabd8ff9700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 0x7fabc0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:56.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.901+0000 7fabd8ff9700 1 -- 172.21.15.29:0/51407028 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fabc0065e50 con 0x7fabc005f2c0 2024-01-24T15:13:56.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.902+0000 7fabdbfff700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 0x7fabc0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:56.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.902+0000 7fabdbfff700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 0x7fabc0061700 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fabdc1557f0 tx=0x7fabc8009060).ready entity=mgr.8407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:56.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.922+0000 7fabd8ff9700 1 -- 172.21.15.29:0/51407028 <== mgr.8407 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fabc0065e50 con 0x7fabc005f2c0 2024-01-24T15:13:56.924 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path snap_test_dir1 2024-01-24T15:13:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.924+0000 7fabe2da6700 1 -- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 msgr2=0x7fabc0061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.924+0000 7fabe2da6700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 0x7fabc0061700 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fabdc1557f0 tx=0x7fabc8009060).stop 2024-01-24T15:13:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.924+0000 7fabe2da6700 1 -- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc064780 msgr2=0x7fabdc14f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.924+0000 7fabe2da6700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc064780 0x7fabdc14f3b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fabd0009d50 tx=0x7fabd0016000).stop 2024-01-24T15:13:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.925+0000 7fabe2da6700 1 -- 172.21.15.29:0/51407028 shutdown_connections 2024-01-24T15:13:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.925+0000 7fabe2da6700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabc005f2c0 0x7fabc0061700 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:56.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.925+0000 7fabe2da6700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabdc151e50 0x7fabdc0b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:56.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.925+0000 7fabe2da6700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabdc064780 0x7fabdc14f3b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:56.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.925+0000 7fabe2da6700 1 --2- 172.21.15.29:0/51407028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabdc0c7f70 0x7fabdc0b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:56.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.925+0000 7fabe2da6700 1 -- 172.21.15.29:0/51407028 >> 172.21.15.29:0/51407028 conn(0x7fabdc0cb880 msgr2=0x7fabdc0c99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:56.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.925+0000 7fabe2da6700 1 -- 172.21.15.29:0/51407028 shutdown_connections 2024-01-24T15:13:56.928 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:56.925+0000 7fabe2da6700 1 -- 172.21.15.29:0/51407028 wait complete. 2024-01-24T15:13:56.943 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-01-24T15:13:56.943 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2024-01-24T15:13:57.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.079+0000 7fb60731a700 1 Processor -- start 2024-01-24T15:13:57.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.079+0000 7fb60731a700 1 -- start start 2024-01-24T15:13:57.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.080+0000 7fb60731a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb600064e90 0x7fb6000c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.080+0000 7fb60731a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6000c7f70 0x7fb600155420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.080+0000 7fb60731a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb600154f90 con 0x7fb600151e50 2024-01-24T15:13:57.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.080+0000 7fb60731a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb600155910 con 0x7fb600064e90 2024-01-24T15:13:57.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.080+0000 7fb60731a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000a9f10 con 0x7fb6000c7f70 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.080+0000 7fb6050b6700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb600151e50 0x7fb600064780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40162/0 (socket says 172.21.15.29:40162) 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.080+0000 7fb6050b6700 1 -- 172.21.15.29:0/3625732916 learned_addr learned my addr 172.21.15.29:0/3625732916 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb6048b5700 1 --2- 172.21.15.29:0/3625732916 >> v2:172.21.15.163:3300/0 conn(0x7fb600064e90 0x7fb6000c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb6058b7700 1 --2- 172.21.15.29:0/3625732916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6000c7f70 0x7fb600155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb5f7fff700 1 -- 172.21.15.29:0/3625732916 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451793589 0 0) 0x7fb600154f90 con 0x7fb600151e50 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb5f7fff700 1 -- 172.21.15.29:0/3625732916 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5ec0033f0 con 0x7fb600151e50 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb5f7fff700 1 -- 172.21.15.29:0/3625732916 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 161257631 0 0) 0x7fb5ec0033f0 con 0x7fb600151e50 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb6048b5700 1 -- 172.21.15.29:0/3625732916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6000c7f70 msgr2=0x7fb600155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb6048b5700 1 --2- 172.21.15.29:0/3625732916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6000c7f70 0x7fb600155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb6048b5700 1 -- 172.21.15.29:0/3625732916 >> v1:172.21.15.29:6789/0 conn(0x7fb600151e50 legacy=0x7fb600064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb6048b5700 1 -- 172.21.15.29:0/3625732916 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000aa1f0 con 0x7fb600064e90 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb6058b7700 1 --2- 172.21.15.29:0/3625732916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6000c7f70 0x7fb600155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:57.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb6048b5700 1 --2- 172.21.15.29:0/3625732916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb600064e90 0x7fb6000c53c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fb5fc009760 tx=0x7fb5fc009060).ready entity=mon.1 client_cookie=2931c007dda647de server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:57.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb5f7fff700 1 -- 172.21.15.29:0/3625732916 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0001470 con 0x7fb600064e90 2024-01-24T15:13:57.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb5f7fff700 1 -- 172.21.15.29:0/3625732916 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc011400 con 0x7fb600064e90 2024-01-24T15:13:57.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.081+0000 7fb5f7fff700 1 -- 172.21.15.29:0/3625732916 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc011780 con 0x7fb600064e90 2024-01-24T15:13:57.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.082+0000 7fb60731a700 1 -- 172.21.15.29:0/3625732916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb600064e90 msgr2=0x7fb6000c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.082+0000 7fb60731a700 1 --2- 172.21.15.29:0/3625732916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb600064e90 0x7fb6000c53c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fb5fc009760 tx=0x7fb5fc009060).stop 2024-01-24T15:13:57.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.082+0000 7fb60731a700 1 -- 172.21.15.29:0/3625732916 shutdown_connections 2024-01-24T15:13:57.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.082+0000 7fb60731a700 1 --2- 172.21.15.29:0/3625732916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6000c7f70 0x7fb600155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.082+0000 7fb60731a700 1 --2- 172.21.15.29:0/3625732916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb600064e90 0x7fb6000c53c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.082+0000 7fb60731a700 1 -- 172.21.15.29:0/3625732916 >> 172.21.15.29:0/3625732916 conn(0x7fb6000cb880 msgr2=0x7fb6000c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:57.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.082+0000 7fb60731a700 1 -- 172.21.15.29:0/3625732916 shutdown_connections 2024-01-24T15:13:57.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.082+0000 7fb60731a700 1 -- 172.21.15.29:0/3625732916 wait complete. 2024-01-24T15:13:57.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.083+0000 7fb60731a700 1 Processor -- start 2024-01-24T15:13:57.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.083+0000 7fb60731a700 1 -- start start 2024-01-24T15:13:57.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.084+0000 7fb60731a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb600064e90 0x7fb6001642c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.084+0000 7fb60731a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6000c7f70 0x7fb600143770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.084+0000 7fb60731a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb600151e50 0x7fb600156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.084+0000 7fb60731a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000add70 con 0x7fb6000c7f70 2024-01-24T15:13:57.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.084+0000 7fb60731a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000ae8f0 con 0x7fb600064e90 2024-01-24T15:13:57.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.084+0000 7fb60731a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000adfc0 con 0x7fb600151e50 2024-01-24T15:13:57.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.084+0000 7fb6048b5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6000c7f70 0x7fb600143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.084+0000 7fb6048b5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6000c7f70 0x7fb600143770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50342/0 (socket says 172.21.15.29:50342) 2024-01-24T15:13:57.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.084+0000 7fb6048b5700 1 -- 172.21.15.29:0/2453445887 learned_addr learned my addr 172.21.15.29:0/2453445887 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:57.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.084+0000 7fb6050b6700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb600064e90 0x7fb6001642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.085+0000 7fb6058b7700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb600151e50 0x7fb600156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.085+0000 7fb6048b5700 1 -- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb600151e50 msgr2=0x7fb600156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.085+0000 7fb6048b5700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb600151e50 0x7fb600156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.085+0000 7fb6048b5700 1 -- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb600064e90 msgr2=0x7fb6001642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.085+0000 7fb6048b5700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb600064e90 0x7fb6001642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.085+0000 7fb6048b5700 1 -- 172.21.15.29:0/2453445887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f8009a20 con 0x7fb6000c7f70 2024-01-24T15:13:57.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.085+0000 7fb6048b5700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6000c7f70 0x7fb600143770 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7fb600062f40 tx=0x7fb5fc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:57.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.085+0000 7fb5f57fa700 1 -- 172.21.15.29:0/2453445887 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc010740 con 0x7fb6000c7f70 2024-01-24T15:13:57.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.085+0000 7fb5f57fa700 1 -- 172.21.15.29:0/2453445887 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc0108a0 con 0x7fb6000c7f70 2024-01-24T15:13:57.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.085+0000 7fb60731a700 1 -- 172.21.15.29:0/2453445887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc0092a0 con 0x7fb6000c7f70 2024-01-24T15:13:57.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.085+0000 7fb5f57fa700 1 -- 172.21.15.29:0/2453445887 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc010c60 con 0x7fb6000c7f70 2024-01-24T15:13:57.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.085+0000 7fb60731a700 1 -- 172.21.15.29:0/2453445887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6000ae580 con 0x7fb6000c7f70 2024-01-24T15:13:57.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.086+0000 7fb60731a700 1 -- 172.21.15.29:0/2453445887 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6000aa710 con 0x7fb6000c7f70 2024-01-24T15:13:57.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.087+0000 7fb5f57fa700 1 -- 172.21.15.29:0/2453445887 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 70) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fb5fc0263b0 con 0x7fb6000c7f70 2024-01-24T15:13:57.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.087+0000 7fb5f57fa700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5e405f080 0x7fb5e4061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.087+0000 7fb5f57fa700 1 -- 172.21.15.29:0/2453445887 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7fb5fc07b000 con 0x7fb6000c7f70 2024-01-24T15:13:57.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.087+0000 7fb6050b6700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5e405f080 0x7fb5e4061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.088+0000 7fb6050b6700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5e405f080 0x7fb5e4061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb5f0000fb0 tx=0x7fb5f0001620).ready entity=mgr.8407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:57.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.092+0000 7fb5f57fa700 1 -- 172.21.15.29:0/2453445887 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb5fc04bfc0 con 0x7fb6000c7f70 2024-01-24T15:13:57.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.301+0000 7fb60731a700 1 -- 172.21.15.29:0/2453445887 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb600083310 con 0x7fb5e405f080 2024-01-24T15:13:57.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.305+0000 7fb5f57fa700 1 -- 172.21.15.29:0/2453445887 <== mgr.8407 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb600083310 con 0x7fb5e405f080 2024-01-24T15:13:57.308 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path snap_test_dir1 2024-01-24T15:13:57.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.307+0000 7fb60731a700 1 -- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5e405f080 msgr2=0x7fb5e4061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.307+0000 7fb60731a700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5e405f080 0x7fb5e4061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb5f0000fb0 tx=0x7fb5f0001620).stop 2024-01-24T15:13:57.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.307+0000 7fb60731a700 1 -- 172.21.15.29:0/2453445887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6000c7f70 msgr2=0x7fb600143770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.307+0000 7fb60731a700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6000c7f70 0x7fb600143770 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7fb600062f40 tx=0x7fb5fc016000).stop 2024-01-24T15:13:57.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.308+0000 7fb60731a700 1 -- 172.21.15.29:0/2453445887 shutdown_connections 2024-01-24T15:13:57.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.308+0000 7fb60731a700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb600151e50 0x7fb600156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.308+0000 7fb60731a700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb600064e90 0x7fb6001642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.308+0000 7fb60731a700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb5e405f080 0x7fb5e4061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.308+0000 7fb60731a700 1 --2- 172.21.15.29:0/2453445887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6000c7f70 0x7fb600143770 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.308+0000 7fb60731a700 1 -- 172.21.15.29:0/2453445887 >> 172.21.15.29:0/2453445887 conn(0x7fb6000cb880 msgr2=0x7fb6000ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:57.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.308+0000 7fb60731a700 1 -- 172.21.15.29:0/2453445887 shutdown_connections 2024-01-24T15:13:57.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.308+0000 7fb60731a700 1 -- 172.21.15.29:0/2453445887 wait complete. 2024-01-24T15:13:57.323 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-01-24T15:13:57.323 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2024-01-24T15:13:57.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.455+0000 7f003bb6d700 1 Processor -- start 2024-01-24T15:13:57.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.455+0000 7f003bb6d700 1 -- start start 2024-01-24T15:13:57.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.456+0000 7f003bb6d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0034150260 0x7f00340acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.456+0000 7f003bb6d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f00340c3010 0x7f00340c3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.456+0000 7f003bb6d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0034062e60 con 0x7f0034150640 2024-01-24T15:13:57.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.456+0000 7f003bb6d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0034062850 con 0x7f00340c3010 2024-01-24T15:13:57.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.456+0000 7f003bb6d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00340ad030 con 0x7f0034150260 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f0039909700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0034150260 0x7f00340acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f003a10a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f00340c3010 0x7f00340c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f003a10a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f00340c3010 0x7f00340c3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:34560/0 (socket says 172.21.15.29:34560) 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f003a10a700 1 -- 172.21.15.29:0/713757732 learned_addr learned my addr 172.21.15.29:0/713757732 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f0039909700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0034150260 0x7f00340acaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:47876/0 (socket says 172.21.15.29:47876) 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f0039108700 1 --1- 172.21.15.29:0/713757732 >> v1:172.21.15.163:6789/0 conn(0x7f0034150640 0x7f00340aaec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:57506/0 (socket says 172.21.15.29:57506) 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f003a10a700 1 -- 172.21.15.29:0/713757732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0034150260 msgr2=0x7f00340acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f003a10a700 1 --2- 172.21.15.29:0/713757732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0034150260 0x7f00340acaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f003a10a700 1 -- 172.21.15.29:0/713757732 >> v1:172.21.15.163:6789/0 conn(0x7f0034150640 legacy=0x7f00340aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f003a10a700 1 -- 172.21.15.29:0/713757732 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0034062af0 con 0x7f00340c3010 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f0038907700 1 -- 172.21.15.29:0/713757732 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119015438 0 0) 0x7f0034062e60 con 0x7f0034150640 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f003a10a700 1 --2- 172.21.15.29:0/713757732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00340c3010 0x7f00340c3490 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f002c00d1a0 tx=0x7f002c00ba50).ready entity=mon.1 client_cookie=e466ec48b0ad22e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f0038907700 1 -- 172.21.15.29:0/713757732 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c010070 con 0x7f00340c3010 2024-01-24T15:13:57.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f0038907700 1 -- 172.21.15.29:0/713757732 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c00de10 con 0x7f00340c3010 2024-01-24T15:13:57.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.457+0000 7f0038907700 1 -- 172.21.15.29:0/713757732 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c015680 con 0x7f00340c3010 2024-01-24T15:13:57.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.458+0000 7f003bb6d700 1 -- 172.21.15.29:0/713757732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00340c3010 msgr2=0x7f00340c3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.458+0000 7f003bb6d700 1 --2- 172.21.15.29:0/713757732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00340c3010 0x7f00340c3490 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f002c00d1a0 tx=0x7f002c00ba50).stop 2024-01-24T15:13:57.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.458+0000 7f003bb6d700 1 -- 172.21.15.29:0/713757732 shutdown_connections 2024-01-24T15:13:57.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.458+0000 7f003bb6d700 1 --2- 172.21.15.29:0/713757732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0034150260 0x7f00340acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.458+0000 7f003bb6d700 1 --2- 172.21.15.29:0/713757732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00340c3010 0x7f00340c3490 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.458+0000 7f003bb6d700 1 -- 172.21.15.29:0/713757732 >> 172.21.15.29:0/713757732 conn(0x7f00340c7280 msgr2=0x7f00340c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:57.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.458+0000 7f003bb6d700 1 -- 172.21.15.29:0/713757732 shutdown_connections 2024-01-24T15:13:57.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.458+0000 7f003bb6d700 1 -- 172.21.15.29:0/713757732 wait complete. 2024-01-24T15:13:57.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.459+0000 7f003bb6d700 1 Processor -- start 2024-01-24T15:13:57.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.459+0000 7f003bb6d700 1 -- start start 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.460+0000 7f003bb6d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00340c3010 0x7f0034152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.460+0000 7f003bb6d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0034150260 0x7f003414d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.460+0000 7f003bb6d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0034150640 0x7f00341479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.460+0000 7f003bb6d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00340c06b0 con 0x7f00340c3010 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.460+0000 7f003bb6d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00340c0c70 con 0x7f0034150640 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.460+0000 7f003bb6d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0034152ac0 con 0x7f0034150260 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.460+0000 7f0039909700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00340c3010 0x7f0034152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.460+0000 7f0039909700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00340c3010 0x7f0034152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50354/0 (socket says 172.21.15.29:50354) 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.460+0000 7f0039909700 1 -- 172.21.15.29:0/1172194368 learned_addr learned my addr 172.21.15.29:0/1172194368 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.460+0000 7f0039108700 1 --2- 172.21.15.29:0/1172194368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0034150260 0x7f003414d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.460+0000 7f003a10a700 1 --2- 172.21.15.29:0/1172194368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0034150640 0x7f00341479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.461+0000 7f0039909700 1 -- 172.21.15.29:0/1172194368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0034150260 msgr2=0x7f003414d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.461+0000 7f0039909700 1 --2- 172.21.15.29:0/1172194368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0034150260 0x7f003414d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.461+0000 7f0039909700 1 -- 172.21.15.29:0/1172194368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0034150640 msgr2=0x7f00341479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.461+0000 7f0039909700 1 --2- 172.21.15.29:0/1172194368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0034150640 0x7f00341479b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.461+0000 7f0039909700 1 -- 172.21.15.29:0/1172194368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c00bcf0 con 0x7f00340c3010 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.461+0000 7f0039909700 1 --2- 172.21.15.29:0/1172194368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00340c3010 0x7f0034152200 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f002800e930 tx=0x7f0028009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.461+0000 7f0025ffb700 1 -- 172.21.15.29:0/1172194368 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028012600 con 0x7f00340c3010 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.461+0000 7f0025ffb700 1 -- 172.21.15.29:0/1172194368 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0028012760 con 0x7f00340c3010 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.461+0000 7f0025ffb700 1 -- 172.21.15.29:0/1172194368 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028012ae0 con 0x7f00340c3010 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.461+0000 7f003bb6d700 1 -- 172.21.15.29:0/1172194368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0034152da0 con 0x7f00340c3010 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.461+0000 7f003bb6d700 1 -- 172.21.15.29:0/1172194368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00341526f0 con 0x7f00340c3010 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.463+0000 7f003bb6d700 1 -- 172.21.15.29:0/1172194368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00340475e0 con 0x7f00340c3010 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.463+0000 7f0025ffb700 1 -- 172.21.15.29:0/1172194368 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 70) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f002801b400 con 0x7f00340c3010 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.463+0000 7f0025ffb700 1 --2- 172.21.15.29:0/1172194368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f001805f080 0x7f0018061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.463+0000 7f0025ffb700 1 -- 172.21.15.29:0/1172194368 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f002807a920 con 0x7f00340c3010 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.463+0000 7f0039108700 1 --2- 172.21.15.29:0/1172194368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f001805f080 0x7f0018061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.464+0000 7f0039108700 1 --2- 172.21.15.29:0/1172194368 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f001805f080 0x7f0018061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f003414c1d0 tx=0x7f0030007d70).ready entity=mgr.8407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:57.469 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.468+0000 7f0025ffb700 1 -- 172.21.15.29:0/1172194368 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f002804b830 con 0x7f00340c3010 2024-01-24T15:13:57.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.676+0000 7f003bb6d700 1 -- 172.21.15.29:0/1172194368 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f00340c4aa0 con 0x7f001805f080 2024-01-24T15:13:57.679 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:57.678+0000 7f83703e8700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2024-01-24T15:13:57.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.678+0000 7f0025ffb700 1 -- 172.21.15.29:0/1172194368 <== mgr.8407 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f00340c4aa0 con 0x7f001805f080 2024-01-24T15:13:57.681 INFO:teuthology.orchestra.run.smithi029.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2024-01-24T15:13:57.684 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-01-24T15:13:57.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:57.685 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-01-24T15:13:57.706 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-01-24T15:13:57.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.880+0000 7ff4926c2700 1 Processor -- start 2024-01-24T15:13:57.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.880+0000 7ff4926c2700 1 -- start start 2024-01-24T15:13:57.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff4926c2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff48c066520 0x7ff48c066960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff4926c2700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c0be080 con 0x7ff48c157530 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff4926c2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c067380 con 0x7ff48c0be510 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff4926c2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48c065040 con 0x7ff48c066520 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48bfff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff48c157530 0x7ff48c068470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40164/0 (socket says 172.21.15.29:40164) 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48bfff700 1 -- 172.21.15.29:0/1299790695 learned_addr learned my addr 172.21.15.29:0/1299790695 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48b7fe700 1 --2- 172.21.15.29:0/1299790695 >> v2:172.21.15.163:3300/0 conn(0x7ff48c066520 0x7ff48c066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317310038 0 0) 0x7ff48c0be080 con 0x7ff48c157530 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4740033f0 con 0x7ff48c157530 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818791978 0 0) 0x7ff48c067380 con 0x7ff48c0be510 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c0be080 con 0x7ff48c0be510 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2724869484 0 0) 0x7ff4740033f0 con 0x7ff48c157530 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 >> v2:172.21.15.163:3300/0 conn(0x7ff48c066520 msgr2=0x7ff48c066960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 --2- 172.21.15.29:0/1299790695 >> v2:172.21.15.163:3300/0 conn(0x7ff48c066520 0x7ff48c066960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 >> v1:172.21.15.163:6789/0 conn(0x7ff48c0be510 legacy=0x7ff48c0be960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c068b80 con 0x7ff48c157530 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7ff47c001470 con 0x7ff48c157530 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 >> v1:172.21.15.29:6789/0 conn(0x7ff48c157530 legacy=0x7ff48c068470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 --2- 172.21.15.29:0/1299790695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff474003a70 0x7ff474003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 --2- 172.21.15.29:0/1299790695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff474004690 0x7ff474004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 --2- 172.21.15.29:0/1299790695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff474005000 0x7ff47400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c065040 con 0x7ff474004690 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47400bf60 con 0x7ff474003a70 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.881+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47400c0d0 con 0x7ff474005000 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff48b7fe700 1 --2- 172.21.15.29:0/1299790695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff474004690 0x7ff474004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff48bfff700 1 --2- 172.21.15.29:0/1299790695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff474003a70 0x7ff474003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff490c5f700 1 --2- 172.21.15.29:0/1299790695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff474005000 0x7ff47400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff48b7fe700 1 -- 172.21.15.29:0/1299790695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff474005000 msgr2=0x7ff47400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff48b7fe700 1 --2- 172.21.15.29:0/1299790695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff474005000 0x7ff47400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff48b7fe700 1 -- 172.21.15.29:0/1299790695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff474003a70 msgr2=0x7ff474003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff48b7fe700 1 --2- 172.21.15.29:0/1299790695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff474003a70 0x7ff474003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff48b7fe700 1 -- 172.21.15.29:0/1299790695 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740033f0 con 0x7ff474004690 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff490c5f700 1 --2- 172.21.15.29:0/1299790695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff474005000 0x7ff47400b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff490c5f700 1 -- 172.21.15.29:0/1299790695 reap_dead start 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff48b7fe700 1 --2- 172.21.15.29:0/1299790695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff474004690 0x7ff474004b10 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7ff480009a80 tx=0x7ff480009060).ready entity=mon.0 client_cookie=730b3104fb5378b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480009430 con 0x7ff474004690 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48000e640 con 0x7ff474004690 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.882+0000 7ff48affd700 1 -- 172.21.15.29:0/1299790695 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48000e9c0 con 0x7ff474004690 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.883+0000 7ff4926c2700 1 -- 172.21.15.29:0/1299790695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff474004690 msgr2=0x7ff474004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.883+0000 7ff4926c2700 1 --2- 172.21.15.29:0/1299790695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff474004690 0x7ff474004b10 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7ff480009a80 tx=0x7ff480009060).stop 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.883+0000 7ff4926c2700 1 -- 172.21.15.29:0/1299790695 shutdown_connections 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.883+0000 7ff4926c2700 1 --2- 172.21.15.29:0/1299790695 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff474004690 0x7ff474004b10 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.883+0000 7ff4926c2700 1 -- 172.21.15.29:0/1299790695 >> 172.21.15.29:0/1299790695 conn(0x7ff48c0c8ba0 msgr2=0x7ff48c0c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:57.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.883+0000 7ff4926c2700 1 -- 172.21.15.29:0/1299790695 shutdown_connections 2024-01-24T15:13:57.889 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.883+0000 7ff4926c2700 1 -- 172.21.15.29:0/1299790695 wait complete. 2024-01-24T15:13:57.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.884+0000 7ff4926c2700 1 Processor -- start 2024-01-24T15:13:57.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.884+0000 7ff4926c2700 1 -- start start 2024-01-24T15:13:57.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.885+0000 7ff4926c2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff474004690 0x7ff48c14adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.885+0000 7ff4926c2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48c157e20 0x7ff48c1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.885+0000 7ff4926c2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff48c0c0dd0 0x7ff48c0c1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.885+0000 7ff4926c2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0c08b0 con 0x7ff48c0c0dd0 2024-01-24T15:13:57.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.885+0000 7ff4926c2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c158790 con 0x7ff48c157e20 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.885+0000 7ff4926c2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0bfce0 con 0x7ff474004690 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.885+0000 7ff490c5f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff48c0c0dd0 0x7ff48c0c1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.885+0000 7ff490c5f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff48c0c0dd0 0x7ff48c0c1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50358/0 (socket says 172.21.15.29:50358) 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.885+0000 7ff490c5f700 1 -- 172.21.15.29:0/222381711 learned_addr learned my addr 172.21.15.29:0/222381711 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.885+0000 7ff48bfff700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff474004690 0x7ff48c14adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.885+0000 7ff48b7fe700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48c157e20 0x7ff48c1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.886+0000 7ff490c5f700 1 -- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff474004690 msgr2=0x7ff48c14adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.886+0000 7ff490c5f700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff474004690 0x7ff48c14adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.886+0000 7ff490c5f700 1 -- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48c157e20 msgr2=0x7ff48c1582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.886+0000 7ff490c5f700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48c157e20 0x7ff48c1582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.886+0000 7ff490c5f700 1 -- 172.21.15.29:0/222381711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c14b2e0 con 0x7ff48c0c0dd0 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.886+0000 7ff490c5f700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff48c0c0dd0 0x7ff48c0c1250 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7ff48c06a080 tx=0x7ff484009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.886+0000 7ff473fff700 1 -- 172.21.15.29:0/222381711 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484001930 con 0x7ff48c0c0dd0 2024-01-24T15:13:57.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.886+0000 7ff473fff700 1 -- 172.21.15.29:0/222381711 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff484001a90 con 0x7ff48c0c0dd0 2024-01-24T15:13:57.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.886+0000 7ff4926c2700 1 -- 172.21.15.29:0/222381711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c0bff80 con 0x7ff48c0c0dd0 2024-01-24T15:13:57.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.886+0000 7ff473fff700 1 -- 172.21.15.29:0/222381711 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484001e30 con 0x7ff48c0c0dd0 2024-01-24T15:13:57.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.886+0000 7ff4926c2700 1 -- 172.21.15.29:0/222381711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c0c0540 con 0x7ff48c0c0dd0 2024-01-24T15:13:57.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.887+0000 7ff4926c2700 1 -- 172.21.15.29:0/222381711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48c0475e0 con 0x7ff48c0c0dd0 2024-01-24T15:13:57.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.887+0000 7ff473fff700 1 -- 172.21.15.29:0/222381711 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 70) v1 ==== 75357+0+0 (secure 0 0 0) 0x7ff484003240 con 0x7ff48c0c0dd0 2024-01-24T15:13:57.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.888+0000 7ff473fff700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff47805f080 0x7ff478061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:57.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.888+0000 7ff473fff700 1 -- 172.21.15.29:0/222381711 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7ff484079a70 con 0x7ff48c0c0dd0 2024-01-24T15:13:57.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.888+0000 7ff48bfff700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff47805f080 0x7ff478061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:57.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.889+0000 7ff48bfff700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff47805f080 0x7ff478061540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff47c001200 tx=0x7ff47c001e10).ready entity=mgr.8407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:57.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:57.892+0000 7ff473fff700 1 -- 172.21.15.29:0/222381711 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff48404a9e0 con 0x7ff48c0c0dd0 2024-01-24T15:13:58.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.125+0000 7ff4926c2700 1 -- 172.21.15.29:0/222381711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7ff48c068df0 con 0x7ff48c0c0dd0 2024-01-24T15:13:58.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.902+0000 7ff473fff700 1 -- 172.21.15.29:0/222381711 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 71) v1 ==== 75605+0+0 (secure 0 0 0) 0x7ff4840774e0 con 0x7ff48c0c0dd0 2024-01-24T15:13:58.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.904+0000 7ff473fff700 1 -- 172.21.15.29:0/222381711 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v71) v1 ==== 93+0+0 (secure 0 0 0) 0x7ff484075dd0 con 0x7ff48c0c0dd0 2024-01-24T15:13:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.906+0000 7ff4926c2700 1 -- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff47805f080 msgr2=0x7ff478061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.906+0000 7ff4926c2700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff47805f080 0x7ff478061540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff47c001200 tx=0x7ff47c001e10).stop 2024-01-24T15:13:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.906+0000 7ff4926c2700 1 -- 172.21.15.29:0/222381711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff48c0c0dd0 msgr2=0x7ff48c0c1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.906+0000 7ff4926c2700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff48c0c0dd0 0x7ff48c0c1250 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7ff48c06a080 tx=0x7ff484009040).stop 2024-01-24T15:13:58.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.906+0000 7ff4926c2700 1 -- 172.21.15.29:0/222381711 shutdown_connections 2024-01-24T15:13:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.906+0000 7ff4926c2700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff47805f080 0x7ff478061540 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.906+0000 7ff4926c2700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff48c0c0dd0 0x7ff48c0c1250 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.906+0000 7ff4926c2700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff474004690 0x7ff48c14adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.906+0000 7ff4926c2700 1 --2- 172.21.15.29:0/222381711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48c157e20 0x7ff48c1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.906+0000 7ff4926c2700 1 -- 172.21.15.29:0/222381711 >> 172.21.15.29:0/222381711 conn(0x7ff48c0c8ba0 msgr2=0x7ff48c14b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.906+0000 7ff4926c2700 1 -- 172.21.15.29:0/222381711 shutdown_connections 2024-01-24T15:13:58.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:58.906+0000 7ff4926c2700 1 -- 172.21.15.29:0/222381711 wait complete. 2024-01-24T15:13:58.924 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T15:13:58.924 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:13:58.969 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:13:59.010 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T15:13:59.010 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-24T15:13:59.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:13:59.011 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-24T15:13:59.055 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:59.053+0000 7f51d2314680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:13:59.060 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T15:13:59.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:13:59.061 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-24T15:13:59.064 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:59.063+0000 7f88032b7680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:13:59.079 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:13:59.117 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:59.115+0000 7f51d2314680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:13:59.138 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:59.136+0000 7f88032b7680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:13:59.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.251+0000 7feac6235700 1 Processor -- start 2024-01-24T15:13:59.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.251+0000 7feac6235700 1 -- start start 2024-01-24T15:13:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.252+0000 7feac6235700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feac00be520 0x7feac00664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.252+0000 7feac6235700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feac00669e0 0x7feac00c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.252+0000 7feac6235700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac0064120 con 0x7feac013a5c0 2024-01-24T15:13:59.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.252+0000 7feac6235700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac00bf6a0 con 0x7feac00669e0 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.252+0000 7feac6235700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac0066e70 con 0x7feac00be520 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.252+0000 7feabf7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7feac013a5c0 0x7feac0063a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40192/0 (socket says 172.21.15.29:40192) 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.252+0000 7feabf7fe700 1 -- 172.21.15.29:0/2019745256 learned_addr learned my addr 172.21.15.29:0/2019745256 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabeffd700 1 --2- 172.21.15.29:0/2019745256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feac00be520 0x7feac00664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabffff700 1 --2- 172.21.15.29:0/2019745256 >> v2:172.21.15.163:3300/0 conn(0x7feac00669e0 0x7feac00c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabe7fc700 1 -- 172.21.15.29:0/2019745256 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263681722 0 0) 0x7feac0064120 con 0x7feac013a5c0 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabe7fc700 1 -- 172.21.15.29:0/2019745256 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa80033f0 con 0x7feac013a5c0 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabeffd700 1 -- 172.21.15.29:0/2019745256 >> v2:172.21.15.163:3300/0 conn(0x7feac00669e0 msgr2=0x7feac00c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabeffd700 1 --2- 172.21.15.29:0/2019745256 >> v2:172.21.15.163:3300/0 conn(0x7feac00669e0 0x7feac00c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabeffd700 1 -- 172.21.15.29:0/2019745256 >> v1:172.21.15.29:6789/0 conn(0x7feac013a5c0 legacy=0x7feac0063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabeffd700 1 -- 172.21.15.29:0/2019745256 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac00bea90 con 0x7feac00be520 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabffff700 1 --2- 172.21.15.29:0/2019745256 >> v2:172.21.15.163:3300/0 conn(0x7feac00669e0 0x7feac00c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabeffd700 1 --2- 172.21.15.29:0/2019745256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feac00be520 0x7feac00664f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7feab4009760 tx=0x7feab4009060).ready entity=mon.2 client_cookie=3aa23c5a77c8811a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:59.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabe7fc700 1 -- 172.21.15.29:0/2019745256 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab400d4a0 con 0x7feac00be520 2024-01-24T15:13:59.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabe7fc700 1 -- 172.21.15.29:0/2019745256 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab400d600 con 0x7feac00be520 2024-01-24T15:13:59.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.253+0000 7feabe7fc700 1 -- 172.21.15.29:0/2019745256 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab400d980 con 0x7feac00be520 2024-01-24T15:13:59.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.254+0000 7feac6235700 1 -- 172.21.15.29:0/2019745256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feac00be520 msgr2=0x7feac00664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:59.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.254+0000 7feac6235700 1 --2- 172.21.15.29:0/2019745256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feac00be520 0x7feac00664f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7feab4009760 tx=0x7feab4009060).stop 2024-01-24T15:13:59.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.254+0000 7feac6235700 1 -- 172.21.15.29:0/2019745256 shutdown_connections 2024-01-24T15:13:59.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.254+0000 7feac6235700 1 --2- 172.21.15.29:0/2019745256 >> v2:172.21.15.163:3300/0 conn(0x7feac00669e0 0x7feac00c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.254+0000 7feac6235700 1 --2- 172.21.15.29:0/2019745256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feac00be520 0x7feac00664f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.254+0000 7feac6235700 1 -- 172.21.15.29:0/2019745256 >> 172.21.15.29:0/2019745256 conn(0x7feac00c8bb0 msgr2=0x7feac00c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:59.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.254+0000 7feac6235700 1 -- 172.21.15.29:0/2019745256 shutdown_connections 2024-01-24T15:13:59.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.254+0000 7feac6235700 1 -- 172.21.15.29:0/2019745256 wait complete. 2024-01-24T15:13:59.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.256+0000 7feac6235700 1 Processor -- start 2024-01-24T15:13:59.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.256+0000 7feac6235700 1 -- start start 2024-01-24T15:13:59.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feac6235700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feac00669e0 0x7feac0164560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:59.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feac6235700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feac00be520 0x7feac01397c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:59.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feac6235700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feac013a5c0 0x7feac00a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:59.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feac6235700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac0139cb0 con 0x7feac00be520 2024-01-24T15:13:59.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feac6235700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac0164c10 con 0x7feac00669e0 2024-01-24T15:13:59.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feac6235700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac0164ef0 con 0x7feac013a5c0 2024-01-24T15:13:59.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feabeffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feac00be520 0x7feac01397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:59.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feabeffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feac00be520 0x7feac01397c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50362/0 (socket says 172.21.15.29:50362) 2024-01-24T15:13:59.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feabeffd700 1 -- 172.21.15.29:0/2745375709 learned_addr learned my addr 172.21.15.29:0/2745375709 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:59.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feabffff700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feac013a5c0 0x7feac00a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:59.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feabf7fe700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feac00669e0 0x7feac0164560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:59.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feabeffd700 1 -- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feac013a5c0 msgr2=0x7feac00a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:59.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feabeffd700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feac013a5c0 0x7feac00a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feabeffd700 1 -- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feac00669e0 msgr2=0x7feac0164560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:59.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feabeffd700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feac00669e0 0x7feac0164560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feabeffd700 1 -- 172.21.15.29:0/2745375709 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab8009a20 con 0x7feac00be520 2024-01-24T15:13:59.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feabeffd700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feac00be520 0x7feac01397c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7feab4009d50 tx=0x7feab4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:59.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.257+0000 7feaa77fe700 1 -- 172.21.15.29:0/2745375709 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4000c90 con 0x7feac00be520 2024-01-24T15:13:59.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.258+0000 7feaa77fe700 1 -- 172.21.15.29:0/2745375709 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab400d770 con 0x7feac00be520 2024-01-24T15:13:59.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.258+0000 7feac6235700 1 -- 172.21.15.29:0/2745375709 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab40092a0 con 0x7feac00be520 2024-01-24T15:13:59.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.258+0000 7feaa77fe700 1 -- 172.21.15.29:0/2745375709 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab401d430 con 0x7feac00be520 2024-01-24T15:13:59.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.258+0000 7feac6235700 1 -- 172.21.15.29:0/2745375709 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac01654b0 con 0x7feac00be520 2024-01-24T15:13:59.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.259+0000 7feac6235700 1 -- 172.21.15.29:0/2745375709 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac00475e0 con 0x7feac00be520 2024-01-24T15:13:59.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.259+0000 7feaa77fe700 1 -- 172.21.15.29:0/2745375709 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 71) v1 ==== 75605+0+0 (secure 0 0 0) 0x7feab401d820 con 0x7feac00be520 2024-01-24T15:13:59.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.260+0000 7feaa77fe700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feaac05f300 0x7feaac0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:59.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.260+0000 7feaa77fe700 1 -- 172.21.15.29:0/2745375709 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7feab407c2b0 con 0x7feac00be520 2024-01-24T15:13:59.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.260+0000 7feabf7fe700 1 -- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feaac05f300 msgr2=0x7feaac0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:59.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.260+0000 7feabf7fe700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feaac05f300 0x7feaac0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:59.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.264+0000 7feaa77fe700 1 -- 172.21.15.29:0/2745375709 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7feab404ce50 con 0x7feac00be520 2024-01-24T15:13:59.392 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:13:59.390+0000 7f51d2314680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:13:59.455 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:13:59.453+0000 7f88032b7680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:13:59.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.461+0000 7feabf7fe700 1 -- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feaac05f300 msgr2=0x7feaac0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:59.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.461+0000 7feabf7fe700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feaac05f300 0x7feaac0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:13:59.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.494+0000 7feac6235700 1 -- 172.21.15.29:0/2745375709 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feac00bed70 con 0x7feac00be520 2024-01-24T15:13:59.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.495+0000 7feaa77fe700 1 -- 172.21.15.29:0/2745375709 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4497 (secure 0 0 0) 0x7feab4050470 con 0x7feac00be520 2024-01-24T15:13:59.497 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:59.497 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7319,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3000519646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3000519646},{"type":"v1","addr":"172.21.15.163:6836","nonce":3000519646}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3497543927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3497543927},{"type":"v1","addr":"172.21.15.29:6835","nonce":3497543927}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2353694052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2353694052},{"type":"v1","addr":"172.21.15.163:6837","nonce":2353694052}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:13:38.524646+0000","modified":"2024-01-24T15:13:41.539080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/3595147603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3595147603},{"type":"v1","addr":"172.21.15.29:6834","nonce":3595147603}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-24T15:13:59.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.497+0000 7feac6235700 1 -- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feaac05f300 msgr2=0x7feaac0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:59.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.497+0000 7feac6235700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feaac05f300 0x7feaac0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:59.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.497+0000 7feac6235700 1 -- 172.21.15.29:0/2745375709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feac00be520 msgr2=0x7feac01397c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:59.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.497+0000 7feac6235700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feac00be520 0x7feac01397c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7feab4009d50 tx=0x7feab4016000).stop 2024-01-24T15:13:59.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.497+0000 7feac6235700 1 -- 172.21.15.29:0/2745375709 shutdown_connections 2024-01-24T15:13:59.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.497+0000 7feac6235700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feac013a5c0 0x7feac00a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.497+0000 7feac6235700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feac00669e0 0x7feac0164560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.497+0000 7feac6235700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7feaac05f300 0x7feaac0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:59.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.497+0000 7feac6235700 1 --2- 172.21.15.29:0/2745375709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feac00be520 0x7feac01397c0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.497+0000 7feac6235700 1 -- 172.21.15.29:0/2745375709 >> 172.21.15.29:0/2745375709 conn(0x7feac00c8bb0 msgr2=0x7feac0152ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:59.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.497+0000 7feac6235700 1 -- 172.21.15.29:0/2745375709 shutdown_connections 2024-01-24T15:13:59.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.497+0000 7feac6235700 1 -- 172.21.15.29:0/2745375709 wait complete. 2024-01-24T15:13:59.501 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 74 2024-01-24T15:13:59.514 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:13:59.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.682+0000 7f51402db700 1 Processor -- start 2024-01-24T15:13:59.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.682+0000 7f51402db700 1 -- start start 2024-01-24T15:13:59.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.683+0000 7f51402db700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5138154370 0x7f51380c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:59.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.683+0000 7f51402db700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51380aa0e0 0x7f5138154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:59.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.683+0000 7f51402db700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51380aa6c0 con 0x7f51380ab1f0 2024-01-24T15:13:59.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.683+0000 7f51402db700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5138064240 con 0x7f51380aa0e0 2024-01-24T15:13:59.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.683+0000 7f51402db700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51380cb5c0 con 0x7f5138154370 2024-01-24T15:13:59.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.683+0000 7f513d876700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51380aa0e0 0x7f5138154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:59.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.683+0000 7f513d876700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51380aa0e0 0x7f5138154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50366/0 (socket says 172.21.15.29:50366) 2024-01-24T15:13:59.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.683+0000 7f513d876700 1 -- 172.21.15.29:0/2574122957 learned_addr learned my addr 172.21.15.29:0/2574122957 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:59.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.683+0000 7f513e077700 1 --2- 172.21.15.29:0/2574122957 >> v2:172.21.15.163:3300/0 conn(0x7f5138154370 0x7f51380c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:59.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.684+0000 7f513d876700 1 -- 172.21.15.29:0/2574122957 >> v2:172.21.15.163:3300/0 conn(0x7f5138154370 msgr2=0x7f51380c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:13:59.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.684+0000 7f513d876700 1 --2- 172.21.15.29:0/2574122957 >> v2:172.21.15.163:3300/0 conn(0x7f5138154370 0x7f51380c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.684+0000 7f513d876700 1 -- 172.21.15.29:0/2574122957 >> v1:172.21.15.29:6789/0 conn(0x7f51380ab1f0 legacy=0x7f51380631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:59.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.684+0000 7f513d876700 1 -- 172.21.15.29:0/2574122957 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51380cb8a0 con 0x7f51380aa0e0 2024-01-24T15:13:59.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.684+0000 7f513d876700 1 --2- 172.21.15.29:0/2574122957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51380aa0e0 0x7f5138154750 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f5138084050 tx=0x7f51280090f0).ready entity=mon.0 client_cookie=233e8269acce5672 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:59.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.684+0000 7f513d075700 1 -- 172.21.15.29:0/2574122957 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800b520 con 0x7f51380aa0e0 2024-01-24T15:13:59.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.684+0000 7f513d075700 1 -- 172.21.15.29:0/2574122957 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512800b680 con 0x7f51380aa0e0 2024-01-24T15:13:59.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.684+0000 7f513d075700 1 -- 172.21.15.29:0/2574122957 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800ba00 con 0x7f51380aa0e0 2024-01-24T15:13:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.685+0000 7f51402db700 1 -- 172.21.15.29:0/2574122957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51380aa0e0 msgr2=0x7f5138154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.685+0000 7f51402db700 1 --2- 172.21.15.29:0/2574122957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51380aa0e0 0x7f5138154750 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f5138084050 tx=0x7f51280090f0).stop 2024-01-24T15:13:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.685+0000 7f51402db700 1 -- 172.21.15.29:0/2574122957 shutdown_connections 2024-01-24T15:13:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.685+0000 7f51402db700 1 --2- 172.21.15.29:0/2574122957 >> v2:172.21.15.163:3300/0 conn(0x7f5138154370 0x7f51380c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.685+0000 7f51402db700 1 --2- 172.21.15.29:0/2574122957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51380aa0e0 0x7f5138154750 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.685+0000 7f51402db700 1 -- 172.21.15.29:0/2574122957 >> 172.21.15.29:0/2574122957 conn(0x7f513805ec80 msgr2=0x7f51380cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.685+0000 7f51402db700 1 -- 172.21.15.29:0/2574122957 shutdown_connections 2024-01-24T15:13:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.685+0000 7f51402db700 1 -- 172.21.15.29:0/2574122957 wait complete. 2024-01-24T15:13:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.686+0000 7f51402db700 1 Processor -- start 2024-01-24T15:13:59.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.686+0000 7f51402db700 1 -- start start 2024-01-24T15:13:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.687+0000 7f51402db700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51380aa0e0 0x7f513814def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.687+0000 7f51402db700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51380ab1f0 0x7f5138157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.687+0000 7f51402db700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5138154370 0x7f51381577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.687+0000 7f51402db700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51380ac2b0 con 0x7f51380aa0e0 2024-01-24T15:13:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.687+0000 7f51402db700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51380ab6e0 con 0x7f51380ab1f0 2024-01-24T15:13:59.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.687+0000 7f51402db700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51380ab980 con 0x7f5138154370 2024-01-24T15:13:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.687+0000 7f513e077700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51380aa0e0 0x7f513814def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.687+0000 7f513e077700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51380aa0e0 0x7f513814def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50382/0 (socket says 172.21.15.29:50382) 2024-01-24T15:13:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.687+0000 7f513e077700 1 -- 172.21.15.29:0/2205037707 learned_addr learned my addr 172.21.15.29:0/2205037707 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:13:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.687+0000 7f513d876700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51380ab1f0 0x7f5138157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.687+0000 7f513e878700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5138154370 0x7f51381577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:13:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.688+0000 7f513e077700 1 -- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5138154370 msgr2=0x7f51381577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.688+0000 7f513e077700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5138154370 0x7f51381577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.688+0000 7f513e077700 1 -- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51380ab1f0 msgr2=0x7f5138157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.688+0000 7f513e077700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51380ab1f0 0x7f5138157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.688+0000 7f513e077700 1 -- 172.21.15.29:0/2205037707 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5128009370 con 0x7f51380aa0e0 2024-01-24T15:13:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.688+0000 7f513e077700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51380aa0e0 0x7f513814def0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7f513400e8d0 tx=0x7f5134006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:13:59.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.688+0000 7f512e7fc700 1 -- 172.21.15.29:0/2205037707 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134012600 con 0x7f51380aa0e0 2024-01-24T15:13:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.688+0000 7f512e7fc700 1 -- 172.21.15.29:0/2205037707 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5134012760 con 0x7f51380aa0e0 2024-01-24T15:13:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.688+0000 7f51402db700 1 -- 172.21.15.29:0/2205037707 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51380abc60 con 0x7f51380aa0e0 2024-01-24T15:13:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.688+0000 7f512e7fc700 1 -- 172.21.15.29:0/2205037707 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134012ae0 con 0x7f51380aa0e0 2024-01-24T15:13:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.688+0000 7f51402db700 1 -- 172.21.15.29:0/2205037707 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51380ad7a0 con 0x7f51380aa0e0 2024-01-24T15:13:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.689+0000 7f51402db700 1 -- 172.21.15.29:0/2205037707 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51380ac590 con 0x7f51380aa0e0 2024-01-24T15:13:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.690+0000 7f512e7fc700 1 -- 172.21.15.29:0/2205037707 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 71) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f513401b400 con 0x7f51380aa0e0 2024-01-24T15:13:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.690+0000 7f512e7fc700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f511c05f300 0x7f511c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:13:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.690+0000 7f512e7fc700 1 -- 172.21.15.29:0/2205037707 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f513407acd0 con 0x7f51380aa0e0 2024-01-24T15:13:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.690+0000 7f513d876700 1 -- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f511c05f300 msgr2=0x7f511c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:13:59.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.690+0000 7f513d876700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f511c05f300 0x7f511c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:13:59.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.694+0000 7f512e7fc700 1 -- 172.21.15.29:0/2205037707 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f513404b810 con 0x7f51380aa0e0 2024-01-24T15:13:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.877+0000 7f51402db700 1 -- 172.21.15.29:0/2205037707 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5138157f90 con 0x7f51380aa0e0 2024-01-24T15:13:59.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.879+0000 7f512e7fc700 1 -- 172.21.15.29:0/2205037707 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+18628 (secure 0 0 0) 0x7f5134054e60 con 0x7f51380aa0e0 2024-01-24T15:13:59.881 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:13:59.881 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":179,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:13:55.886363+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:13:36.272483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-24T15:13:37.504363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6833/43241":"2024-01-25T15:13:55.886328+0000","172.21.15.163:6832/43241":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/42051548":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/4250314662":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/4170284440":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/1833132326":"2024-01-25T15:13:55.886328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:13:59.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.881+0000 7f51402db700 1 -- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f511c05f300 msgr2=0x7f511c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:13:59.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.881+0000 7f51402db700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f511c05f300 0x7f511c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:59.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.881+0000 7f51402db700 1 -- 172.21.15.29:0/2205037707 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51380aa0e0 msgr2=0x7f513814def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:13:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.881+0000 7f51402db700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51380aa0e0 0x7f513814def0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7f513400e8d0 tx=0x7f5134006f40).stop 2024-01-24T15:13:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.881+0000 7f51402db700 1 -- 172.21.15.29:0/2205037707 shutdown_connections 2024-01-24T15:13:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.881+0000 7f51402db700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5138154370 0x7f51381577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.881+0000 7f51402db700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f511c05f300 0x7f511c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:13:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.881+0000 7f51402db700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51380aa0e0 0x7f513814def0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.881+0000 7f51402db700 1 --2- 172.21.15.29:0/2205037707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51380ab1f0 0x7f5138157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:13:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.881+0000 7f51402db700 1 -- 172.21.15.29:0/2205037707 >> 172.21.15.29:0/2205037707 conn(0x7f513805ec80 msgr2=0x7f51380c94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:13:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.881+0000 7f51402db700 1 -- 172.21.15.29:0/2205037707 shutdown_connections 2024-01-24T15:13:59.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:13:59.881+0000 7f51402db700 1 -- 172.21.15.29:0/2205037707 wait complete. 2024-01-24T15:13:59.897 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T15:13:36.272483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '177', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T15:13:37.504363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '178', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T15:13:59.898 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T15:14:00.009 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:00.007+0000 7f51d2314680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:14:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.032+0000 7faaa493a700 1 Processor -- start 2024-01-24T15:14:00.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.032+0000 7faaa493a700 1 -- start start 2024-01-24T15:14:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.033+0000 7faaa493a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faaa0064760 0x7faaa0064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.033+0000 7faaa493a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa0152240 con 0x7faaa00c8490 2024-01-24T15:14:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.033+0000 7faaa493a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa01586c0 con 0x7faaa00bfd40 2024-01-24T15:14:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.033+0000 7faaa493a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa00aa670 con 0x7faaa0064760 2024-01-24T15:14:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.033+0000 7faa97fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7faaa00c8490 0x7faaa00a9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40200/0 (socket says 172.21.15.29:40200) 2024-01-24T15:14:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.033+0000 7faa97fff700 1 -- 172.21.15.29:0/4193619973 learned_addr learned my addr 172.21.15.29:0/4193619973 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.033+0000 7faa9e59c700 1 --2- 172.21.15.29:0/4193619973 >> v2:172.21.15.163:3300/0 conn(0x7faaa0064760 0x7faaa0064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.034+0000 7faa9dd9b700 1 -- 172.21.15.29:0/4193619973 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177230750 0 0) 0x7faaa0152240 con 0x7faaa00c8490 2024-01-24T15:14:00.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.034+0000 7faa9dd9b700 1 -- 172.21.15.29:0/4193619973 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa8c0033f0 con 0x7faaa00c8490 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.034+0000 7faa9e59c700 1 -- 172.21.15.29:0/4193619973 >> v1:172.21.15.163:6789/0 conn(0x7faaa00bfd40 legacy=0x7faaa00c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.034+0000 7faa9dd9b700 1 -- 172.21.15.29:0/4193619973 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967005930 0 0) 0x7faaa01586c0 con 0x7faaa00bfd40 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.034+0000 7faa9e59c700 1 -- 172.21.15.29:0/4193619973 >> v1:172.21.15.29:6789/0 conn(0x7faaa00c8490 legacy=0x7faaa00a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.034+0000 7faa9e59c700 1 -- 172.21.15.29:0/4193619973 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa0157d90 con 0x7faaa0064760 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.034+0000 7faa9e59c700 1 --2- 172.21.15.29:0/4193619973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa0064760 0x7faaa0064b40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7faa90009760 tx=0x7faa90009060).ready entity=mon.1 client_cookie=d229f54d38aaf8f8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.034+0000 7faa9dd9b700 1 -- 172.21.15.29:0/4193619973 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9000b590 con 0x7faaa0064760 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.034+0000 7faa9dd9b700 1 -- 172.21.15.29:0/4193619973 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9000b6f0 con 0x7faaa0064760 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.034+0000 7faa9dd9b700 1 -- 172.21.15.29:0/4193619973 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9000ba70 con 0x7faaa0064760 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.035+0000 7faaa493a700 1 -- 172.21.15.29:0/4193619973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa0064760 msgr2=0x7faaa0064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.035+0000 7faaa493a700 1 --2- 172.21.15.29:0/4193619973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa0064760 0x7faaa0064b40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7faa90009760 tx=0x7faa90009060).stop 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.035+0000 7faaa493a700 1 -- 172.21.15.29:0/4193619973 shutdown_connections 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.035+0000 7faaa493a700 1 --2- 172.21.15.29:0/4193619973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa0064760 0x7faaa0064b40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.035+0000 7faaa493a700 1 -- 172.21.15.29:0/4193619973 >> 172.21.15.29:0/4193619973 conn(0x7faaa00cb8b0 msgr2=0x7faaa00c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:00.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.035+0000 7faaa493a700 1 -- 172.21.15.29:0/4193619973 shutdown_connections 2024-01-24T15:14:00.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.035+0000 7faaa493a700 1 -- 172.21.15.29:0/4193619973 wait complete. 2024-01-24T15:14:00.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.036+0000 7faaa493a700 1 Processor -- start 2024-01-24T15:14:00.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.036+0000 7faaa493a700 1 -- start start 2024-01-24T15:14:00.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.037+0000 7faaa493a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaa0064760 0x7faaa014ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.037+0000 7faaa493a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa00bfd40 0x7faaa00c1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.037+0000 7faaa493a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faaa00c8490 0x7faaa00c2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.037+0000 7faaa493a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa00c2f00 con 0x7faaa0064760 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.037+0000 7faaa493a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa014f320 con 0x7faaa00bfd40 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.037+0000 7faaa493a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa00ae2a0 con 0x7faaa00c8490 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.037+0000 7faa9e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaa0064760 0x7faaa014ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.037+0000 7faa9e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaa0064760 0x7faaa014ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50394/0 (socket says 172.21.15.29:50394) 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.037+0000 7faa9e59c700 1 -- 172.21.15.29:0/2960303705 learned_addr learned my addr 172.21.15.29:0/2960303705 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.037+0000 7faa97fff700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa00bfd40 0x7faaa00c1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.037+0000 7faa9ed9d700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faaa00c8490 0x7faaa00c2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.038+0000 7faa9e59c700 1 -- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faaa00c8490 msgr2=0x7faaa00c2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.038+0000 7faa9e59c700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faaa00c8490 0x7faaa00c2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.038+0000 7faa9e59c700 1 -- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa00bfd40 msgr2=0x7faaa00c1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.038+0000 7faa9e59c700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa00bfd40 0x7faaa00c1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.038+0000 7faa9e59c700 1 -- 172.21.15.29:0/2960303705 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa900092a0 con 0x7faaa0064760 2024-01-24T15:14:00.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.038+0000 7faa97fff700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa00bfd40 0x7faaa00c1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:00.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.038+0000 7faa9e59c700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaa0064760 0x7faaa014ee30 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7faa90009e00 tx=0x7faa90017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:00.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.038+0000 7faa967fc700 1 -- 172.21.15.29:0/2960303705 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9001a4d0 con 0x7faaa0064760 2024-01-24T15:14:00.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.038+0000 7faa967fc700 1 -- 172.21.15.29:0/2960303705 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9001e070 con 0x7faaa0064760 2024-01-24T15:14:00.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.038+0000 7faaa493a700 1 -- 172.21.15.29:0/2960303705 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa0155230 con 0x7faaa0064760 2024-01-24T15:14:00.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.038+0000 7faa967fc700 1 -- 172.21.15.29:0/2960303705 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9001a880 con 0x7faaa0064760 2024-01-24T15:14:00.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.038+0000 7faaa493a700 1 -- 172.21.15.29:0/2960303705 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa00c2c20 con 0x7faaa0064760 2024-01-24T15:14:00.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.039+0000 7faaa493a700 1 -- 172.21.15.29:0/2960303705 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa00cac20 con 0x7faaa0064760 2024-01-24T15:14:00.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.040+0000 7faa967fc700 1 -- 172.21.15.29:0/2960303705 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 71) v1 ==== 75605+0+0 (secure 0 0 0) 0x7faa90020030 con 0x7faaa0064760 2024-01-24T15:14:00.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.040+0000 7faa967fc700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa8005f300 0x7faa800617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.040+0000 7faa967fc700 1 -- 172.21.15.29:0/2960303705 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7faa9007ba60 con 0x7faaa0064760 2024-01-24T15:14:00.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.040+0000 7faa97fff700 1 -- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa8005f300 msgr2=0x7faa800617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:00.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.040+0000 7faa97fff700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa8005f300 0x7faa800617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:14:00.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.048+0000 7faa967fc700 1 -- 172.21.15.29:0/2960303705 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7faa9004c5a0 con 0x7faaa0064760 2024-01-24T15:14:00.083 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:00.080+0000 7f51d2314680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:14:00.192 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:00.188+0000 7f88032b7680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:14:00.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.241+0000 7faa97fff700 1 -- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa8005f300 msgr2=0x7faa800617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:00.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.241+0000 7faa97fff700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa8005f300 0x7faa800617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:14:00.266 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:00.263+0000 7f88032b7680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:14:00.270 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:00.268+0000 7f51d2314680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:14:00.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.277+0000 7faaa493a700 1 -- 172.21.15.29:0/2960303705 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7faaa00475e0 con 0x7faaa0064760 2024-01-24T15:14:00.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.277+0000 7faa967fc700 1 -- 172.21.15.29:0/2960303705 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v179) v1 ==== 110+0+10 (secure 0 0 0) 0x7faa9004fbc0 con 0x7faaa0064760 2024-01-24T15:14:00.280 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T15:14:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.280+0000 7faaa493a700 1 -- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa8005f300 msgr2=0x7faa800617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:14:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.280+0000 7faaa493a700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa8005f300 0x7faa800617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.280+0000 7faaa493a700 1 -- 172.21.15.29:0/2960303705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaa0064760 msgr2=0x7faaa014ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.280+0000 7faaa493a700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaa0064760 0x7faaa014ee30 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7faa90009e00 tx=0x7faa90017000).stop 2024-01-24T15:14:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.280+0000 7faaa493a700 1 -- 172.21.15.29:0/2960303705 shutdown_connections 2024-01-24T15:14:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.280+0000 7faaa493a700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faaa00c8490 0x7faaa00c2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.280+0000 7faaa493a700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7faa8005f300 0x7faa800617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:00.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.280+0000 7faaa493a700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaa0064760 0x7faaa014ee30 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.280+0000 7faaa493a700 1 --2- 172.21.15.29:0/2960303705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa00bfd40 0x7faaa00c1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.280+0000 7faaa493a700 1 -- 172.21.15.29:0/2960303705 >> 172.21.15.29:0/2960303705 conn(0x7faaa00cb8b0 msgr2=0x7faaa0150dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.280+0000 7faaa493a700 1 -- 172.21.15.29:0/2960303705 shutdown_connections 2024-01-24T15:14:00.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.280+0000 7faaa493a700 1 -- 172.21.15.29:0/2960303705 wait complete. 2024-01-24T15:14:00.296 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T15:14:00.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.429+0000 7f03f8e4f700 1 Processor -- start 2024-01-24T15:14:00.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.430+0000 7f03f8e4f700 1 -- start start 2024-01-24T15:14:00.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f8e4f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f03f4062ec0 0x7f03f406a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f8e4f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03f40c8270 0x7f03f41391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f8e4f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f40bcda0 con 0x7f03f41384f0 2024-01-24T15:14:00.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f8e4f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f4065f80 con 0x7f03f4062ec0 2024-01-24T15:14:00.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f8e4f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f4066260 con 0x7f03f40c8270 2024-01-24T15:14:00.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f259c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f03f41384f0 0x7f03f40627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40216/0 (socket says 172.21.15.29:40216) 2024-01-24T15:14:00.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f259c700 1 -- 172.21.15.29:0/1669111958 learned_addr learned my addr 172.21.15.29:0/1669111958 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:00.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f2d9d700 1 --2- 172.21.15.29:0/1669111958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03f40c8270 0x7f03f41391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f1d9b700 1 --2- 172.21.15.29:0/1669111958 >> v2:172.21.15.163:3300/0 conn(0x7f03f4062ec0 0x7f03f406a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f159a700 1 -- 172.21.15.29:0/1669111958 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994992381 0 0) 0x7f03f40bcda0 con 0x7f03f41384f0 2024-01-24T15:14:00.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f159a700 1 -- 172.21.15.29:0/1669111958 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03dc0033f0 con 0x7f03f41384f0 2024-01-24T15:14:00.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f1d9b700 1 -- 172.21.15.29:0/1669111958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03f40c8270 msgr2=0x7f03f41391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:00.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f1d9b700 1 --2- 172.21.15.29:0/1669111958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03f40c8270 0x7f03f41391d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f1d9b700 1 -- 172.21.15.29:0/1669111958 >> v1:172.21.15.29:6789/0 conn(0x7f03f41384f0 legacy=0x7f03f40627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f1d9b700 1 -- 172.21.15.29:0/1669111958 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f40bca50 con 0x7f03f4062ec0 2024-01-24T15:14:00.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.431+0000 7f03f2d9d700 1 --2- 172.21.15.29:0/1669111958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03f40c8270 0x7f03f41391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:00.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.432+0000 7f03f1d9b700 1 --2- 172.21.15.29:0/1669111958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03f4062ec0 0x7f03f406a460 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f03e8009760 tx=0x7f03e8009060).ready entity=mon.1 client_cookie=6dc3266c1714180d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:00.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.432+0000 7f03f159a700 1 -- 172.21.15.29:0/1669111958 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e80090d0 con 0x7f03f4062ec0 2024-01-24T15:14:00.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.432+0000 7f03f159a700 1 -- 172.21.15.29:0/1669111958 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e800d4a0 con 0x7f03f4062ec0 2024-01-24T15:14:00.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.432+0000 7f03f159a700 1 -- 172.21.15.29:0/1669111958 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800d820 con 0x7f03f4062ec0 2024-01-24T15:14:00.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.433+0000 7f03f8e4f700 1 -- 172.21.15.29:0/1669111958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03f4062ec0 msgr2=0x7f03f406a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.433+0000 7f03f8e4f700 1 --2- 172.21.15.29:0/1669111958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03f4062ec0 0x7f03f406a460 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f03e8009760 tx=0x7f03e8009060).stop 2024-01-24T15:14:00.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.433+0000 7f03f8e4f700 1 -- 172.21.15.29:0/1669111958 shutdown_connections 2024-01-24T15:14:00.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.433+0000 7f03f8e4f700 1 --2- 172.21.15.29:0/1669111958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03f40c8270 0x7f03f41391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.433+0000 7f03f8e4f700 1 --2- 172.21.15.29:0/1669111958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03f4062ec0 0x7f03f406a460 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.433+0000 7f03f8e4f700 1 -- 172.21.15.29:0/1669111958 >> 172.21.15.29:0/1669111958 conn(0x7f03f4064390 msgr2=0x7f03f40c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:00.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.433+0000 7f03f8e4f700 1 -- 172.21.15.29:0/1669111958 shutdown_connections 2024-01-24T15:14:00.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.433+0000 7f03f8e4f700 1 -- 172.21.15.29:0/1669111958 wait complete. 2024-01-24T15:14:00.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.435+0000 7f03f8e4f700 1 Processor -- start 2024-01-24T15:14:00.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.435+0000 7f03f8e4f700 1 -- start start 2024-01-24T15:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.436+0000 7f03f8e4f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03f4062ec0 0x7f03f41570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.436+0000 7f03f8e4f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03f40c8270 0x7f03f40beb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.436+0000 7f03f8e4f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03f41384f0 0x7f03f4134540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.436+0000 7f03f8e4f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f4134da0 con 0x7f03f4062ec0 2024-01-24T15:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.436+0000 7f03f8e4f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f40bf080 con 0x7f03f41384f0 2024-01-24T15:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.436+0000 7f03f8e4f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f40c1150 con 0x7f03f40c8270 2024-01-24T15:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.436+0000 7f03f259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03f4062ec0 0x7f03f41570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.436+0000 7f03f259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03f4062ec0 0x7f03f41570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50396/0 (socket says 172.21.15.29:50396) 2024-01-24T15:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.436+0000 7f03f259c700 1 -- 172.21.15.29:0/2870187704 learned_addr learned my addr 172.21.15.29:0/2870187704 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.436+0000 7f03f1d9b700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03f40c8270 0x7f03f40beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03f2d9d700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03f41384f0 0x7f03f4134540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03f259c700 1 -- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03f40c8270 msgr2=0x7f03f40beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03f259c700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03f40c8270 0x7f03f40beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03f259c700 1 -- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03f41384f0 msgr2=0x7f03f4134540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03f259c700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03f41384f0 0x7f03f4134540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03f259c700 1 -- 172.21.15.29:0/2870187704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec009a20 con 0x7f03f4062ec0 2024-01-24T15:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03f1d9b700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03f40c8270 0x7f03f40beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03f2d9d700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03f41384f0 0x7f03f4134540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03f259c700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03f4062ec0 0x7f03f41570d0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f03e400c450 tx=0x7f03e4007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03e27fc700 1 -- 172.21.15.29:0/2870187704 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e400ce70 con 0x7f03f4062ec0 2024-01-24T15:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03e27fc700 1 -- 172.21.15.29:0/2870187704 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e40072e0 con 0x7f03f4062ec0 2024-01-24T15:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03f8e4f700 1 -- 172.21.15.29:0/2870187704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e80092a0 con 0x7f03f4062ec0 2024-01-24T15:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03e27fc700 1 -- 172.21.15.29:0/2870187704 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e4007660 con 0x7f03f4062ec0 2024-01-24T15:14:00.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.437+0000 7f03f8e4f700 1 -- 172.21.15.29:0/2870187704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f40c16c0 con 0x7f03f4062ec0 2024-01-24T15:14:00.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.438+0000 7f03f8e4f700 1 -- 172.21.15.29:0/2870187704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f40475e0 con 0x7f03f4062ec0 2024-01-24T15:14:00.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.439+0000 7f03e27fc700 1 -- 172.21.15.29:0/2870187704 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 71) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f03e40059b0 con 0x7f03f4062ec0 2024-01-24T15:14:00.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.439+0000 7f03e27fc700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f03d405f330 0x7f03d40617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.440+0000 7f03f1d9b700 1 -- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f03d405f330 msgr2=0x7f03d40617f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:00.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.440+0000 7f03e27fc700 1 -- 172.21.15.29:0/2870187704 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f03e407a130 con 0x7f03f4062ec0 2024-01-24T15:14:00.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.440+0000 7f03f1d9b700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f03d405f330 0x7f03d40617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:14:00.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.444+0000 7f03e27fc700 1 -- 172.21.15.29:0/2870187704 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f03e404acd0 con 0x7f03f4062ec0 2024-01-24T15:14:00.486 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:00.484+0000 7f88032b7680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:14:00.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.626+0000 7f03f8e4f700 1 -- 172.21.15.29:0/2870187704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f03f40c8020 con 0x7f03f4062ec0 2024-01-24T15:14:00.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.627+0000 7f03e27fc700 1 -- 172.21.15.29:0/2870187704 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v179) v1 ==== 92+0+10 (secure 0 0 0) 0x7f03e404e2f0 con 0x7f03f4062ec0 2024-01-24T15:14:00.629 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T15:14:00.631 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.629+0000 7f03f8e4f700 1 -- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f03d405f330 msgr2=0x7f03d40617f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:14:00.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.629+0000 7f03f8e4f700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f03d405f330 0x7f03d40617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:00.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.629+0000 7f03f8e4f700 1 -- 172.21.15.29:0/2870187704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03f4062ec0 msgr2=0x7f03f41570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.629+0000 7f03f8e4f700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03f4062ec0 0x7f03f41570d0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f03e400c450 tx=0x7f03e4007c80).stop 2024-01-24T15:14:00.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.629+0000 7f03f8e4f700 1 -- 172.21.15.29:0/2870187704 shutdown_connections 2024-01-24T15:14:00.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.629+0000 7f03f8e4f700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03f41384f0 0x7f03f4134540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.629+0000 7f03f8e4f700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f03d405f330 0x7f03d40617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:00.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.629+0000 7f03f8e4f700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03f4062ec0 0x7f03f41570d0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.629+0000 7f03f8e4f700 1 --2- 172.21.15.29:0/2870187704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03f40c8270 0x7f03f40beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.629+0000 7f03f8e4f700 1 -- 172.21.15.29:0/2870187704 >> 172.21.15.29:0/2870187704 conn(0x7f03f4064390 msgr2=0x7f03f41373d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:00.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.629+0000 7f03f8e4f700 1 -- 172.21.15.29:0/2870187704 shutdown_connections 2024-01-24T15:14:00.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.629+0000 7f03f8e4f700 1 -- 172.21.15.29:0/2870187704 wait complete. 2024-01-24T15:14:00.645 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T15:14:00.691 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:00.688+0000 7f51d2314680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:14:00.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.779+0000 7f6448b4c700 1 Processor -- start 2024-01-24T15:14:00.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.779+0000 7f6448b4c700 1 -- start start 2024-01-24T15:14:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.780+0000 7f6448b4c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f644413a5e0 0x7f64440639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.780+0000 7f6448b4c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6444066bd0 0x7f64440bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.780+0000 7f6448b4c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64440679f0 con 0x7f6444063eb0 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f6448b4c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6444067cd0 con 0x7f6444066bd0 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f6448b4c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6444067fb0 con 0x7f644413a5e0 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f6442d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6444066bd0 0x7f64440bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f6442d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6444066bd0 0x7f64440bdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50404/0 (socket says 172.21.15.29:50404) 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f6442d9d700 1 -- 172.21.15.29:0/2070626689 learned_addr learned my addr 172.21.15.29:0/2070626689 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f644259c700 1 --2- 172.21.15.29:0/2070626689 >> v2:172.21.15.163:3300/0 conn(0x7f644413a5e0 0x7f64440639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f6442d9d700 1 -- 172.21.15.29:0/2070626689 >> v2:172.21.15.163:3300/0 conn(0x7f644413a5e0 msgr2=0x7f64440639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f6442d9d700 1 --2- 172.21.15.29:0/2070626689 >> v2:172.21.15.163:3300/0 conn(0x7f644413a5e0 0x7f64440639c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f6442d9d700 1 -- 172.21.15.29:0/2070626689 >> v1:172.21.15.29:6789/0 conn(0x7f6444063eb0 legacy=0x7f64440664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f6442d9d700 1 -- 172.21.15.29:0/2070626689 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6444068290 con 0x7f6444066bd0 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f6442d9d700 1 --2- 172.21.15.29:0/2070626689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6444066bd0 0x7f64440bdb30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f6444069ba0 tx=0x7f643c009840).ready entity=mon.0 client_cookie=d543fdbe70833fd0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f644159a700 1 -- 172.21.15.29:0/2070626689 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c00e070 con 0x7f6444066bd0 2024-01-24T15:14:00.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f644159a700 1 -- 172.21.15.29:0/2070626689 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643c00ba20 con 0x7f6444066bd0 2024-01-24T15:14:00.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.781+0000 7f644159a700 1 -- 172.21.15.29:0/2070626689 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c00bda0 con 0x7f6444066bd0 2024-01-24T15:14:00.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.782+0000 7f6448b4c700 1 -- 172.21.15.29:0/2070626689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6444066bd0 msgr2=0x7f64440bdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.782+0000 7f6448b4c700 1 --2- 172.21.15.29:0/2070626689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6444066bd0 0x7f64440bdb30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f6444069ba0 tx=0x7f643c009840).stop 2024-01-24T15:14:00.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.782+0000 7f6448b4c700 1 -- 172.21.15.29:0/2070626689 shutdown_connections 2024-01-24T15:14:00.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.782+0000 7f6448b4c700 1 --2- 172.21.15.29:0/2070626689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6444066bd0 0x7f64440bdb30 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.782+0000 7f6448b4c700 1 --2- 172.21.15.29:0/2070626689 >> v2:172.21.15.163:3300/0 conn(0x7f644413a5e0 0x7f64440639c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.782+0000 7f6448b4c700 1 -- 172.21.15.29:0/2070626689 >> 172.21.15.29:0/2070626689 conn(0x7f64440c8bd0 msgr2=0x7f64440c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:00.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.782+0000 7f6448b4c700 1 -- 172.21.15.29:0/2070626689 shutdown_connections 2024-01-24T15:14:00.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.782+0000 7f6448b4c700 1 -- 172.21.15.29:0/2070626689 wait complete. 2024-01-24T15:14:00.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.783+0000 7f6448b4c700 1 Processor -- start 2024-01-24T15:14:00.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.784+0000 7f6448b4c700 1 -- start start 2024-01-24T15:14:00.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.784+0000 7f6448b4c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6444063eb0 0x7f6444138c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.784+0000 7f6448b4c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6444066bd0 0x7f64440ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.784+0000 7f6448b4c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644413a5e0 0x7f64440a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.784+0000 7f6448b4c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64440ad4f0 con 0x7f644413a5e0 2024-01-24T15:14:00.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.784+0000 7f6448b4c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64440ac970 con 0x7f6444063eb0 2024-01-24T15:14:00.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.784+0000 7f6448b4c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64440acc50 con 0x7f6444066bd0 2024-01-24T15:14:00.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.784+0000 7f6442d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644413a5e0 0x7f64440a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.784+0000 7f6442d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644413a5e0 0x7f64440a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50412/0 (socket says 172.21.15.29:50412) 2024-01-24T15:14:00.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f644259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6444063eb0 0x7f6444138c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f6442d9d700 1 -- 172.21.15.29:0/3465680974 learned_addr learned my addr 172.21.15.29:0/3465680974 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f6441d9b700 1 --2- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6444066bd0 0x7f64440ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f6442d9d700 1 -- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6444066bd0 msgr2=0x7f64440ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f6442d9d700 1 --2- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6444066bd0 0x7f64440ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f6442d9d700 1 -- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6444063eb0 msgr2=0x7f6444138c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f6442d9d700 1 --2- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6444063eb0 0x7f6444138c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f6442d9d700 1 -- 172.21.15.29:0/3465680974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c009ae0 con 0x7f644413a5e0 2024-01-24T15:14:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f6442d9d700 1 --2- 172.21.15.29:0/3465680974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644413a5e0 0x7f64440a4880 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7f643c00b6c0 tx=0x7f643c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f64327fc700 1 -- 172.21.15.29:0/3465680974 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c0092e0 con 0x7f644413a5e0 2024-01-24T15:14:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f64327fc700 1 -- 172.21.15.29:0/3465680974 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643c01e470 con 0x7f644413a5e0 2024-01-24T15:14:00.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f64327fc700 1 -- 172.21.15.29:0/3465680974 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c01e830 con 0x7f644413a5e0 2024-01-24T15:14:00.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f6448b4c700 1 -- 172.21.15.29:0/3465680974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64440acf30 con 0x7f644413a5e0 2024-01-24T15:14:00.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.785+0000 7f6448b4c700 1 -- 172.21.15.29:0/3465680974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6444139180 con 0x7f644413a5e0 2024-01-24T15:14:00.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.786+0000 7f6448b4c700 1 -- 172.21.15.29:0/3465680974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64440475e0 con 0x7f644413a5e0 2024-01-24T15:14:00.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.787+0000 7f64327fc700 1 -- 172.21.15.29:0/3465680974 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 71) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f643c003040 con 0x7f644413a5e0 2024-01-24T15:14:00.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.787+0000 7f64327fc700 1 --2- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f642405f300 0x7f64240617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:00.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.787+0000 7f644259c700 1 -- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f642405f300 msgr2=0x7f64240617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:00.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.787+0000 7f64327fc700 1 -- 172.21.15.29:0/3465680974 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f643c07c040 con 0x7f644413a5e0 2024-01-24T15:14:00.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.787+0000 7f644259c700 1 --2- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f642405f300 0x7f64240617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:14:00.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.792+0000 7f64327fc700 1 -- 172.21.15.29:0/3465680974 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f643c04cc00 con 0x7f644413a5e0 2024-01-24T15:14:00.794 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:00.792+0000 7f51d2314680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:14:00.949 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:00.946+0000 7f51d2314680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:14:00.972 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:00.971+0000 7f88032b7680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:14:00.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.974+0000 7f6448b4c700 1 -- 172.21.15.29:0/3465680974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64440a8dd0 con 0x7f644413a5e0 2024-01-24T15:14:00.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.974+0000 7f64327fc700 1 -- 172.21.15.29:0/3465680974 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v179) v1 ==== 104+0+11 (secure 0 0 0) 0x7f643c023070 con 0x7f644413a5e0 2024-01-24T15:14:00.976 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T15:14:00.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.977+0000 7f6448b4c700 1 -- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f642405f300 msgr2=0x7f64240617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:14:00.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.977+0000 7f6448b4c700 1 --2- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f642405f300 0x7f64240617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:00.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.977+0000 7f6448b4c700 1 -- 172.21.15.29:0/3465680974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644413a5e0 msgr2=0x7f64440a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:00.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.977+0000 7f6448b4c700 1 --2- 172.21.15.29:0/3465680974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644413a5e0 0x7f64440a4880 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7f643c00b6c0 tx=0x7f643c009840).stop 2024-01-24T15:14:00.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.977+0000 7f6448b4c700 1 -- 172.21.15.29:0/3465680974 shutdown_connections 2024-01-24T15:14:00.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.977+0000 7f6448b4c700 1 --2- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f642405f300 0x7f64240617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:00.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.977+0000 7f6448b4c700 1 --2- 172.21.15.29:0/3465680974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644413a5e0 0x7f64440a4880 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.977+0000 7f6448b4c700 1 --2- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6444063eb0 0x7f6444138c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.977+0000 7f6448b4c700 1 --2- 172.21.15.29:0/3465680974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6444066bd0 0x7f64440ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:00.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.977+0000 7f6448b4c700 1 -- 172.21.15.29:0/3465680974 >> 172.21.15.29:0/3465680974 conn(0x7f64440c8bd0 msgr2=0x7f6444156b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:00.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.977+0000 7f6448b4c700 1 -- 172.21.15.29:0/3465680974 shutdown_connections 2024-01-24T15:14:00.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:00.977+0000 7f6448b4c700 1 -- 172.21.15.29:0/3465680974 wait complete. 2024-01-24T15:14:00.993 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T15:14:01.025 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:01.022+0000 7f51d2314680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:14:01.093 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:01.090+0000 7f51d2314680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:14:01.094 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:01.092+0000 7f88032b7680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:14:01.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.125+0000 7f954d244700 1 Processor -- start 2024-01-24T15:14:01.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.125+0000 7f954d244700 1 -- start start 2024-01-24T15:14:01.136 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f954d244700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9548064760 0x7f9548064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f954d244700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95480bfd40 0x7f95480c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f954d244700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9548152240 con 0x7f95480c8490 2024-01-24T15:14:01.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f954d244700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95481586c0 con 0x7f9548064760 2024-01-24T15:14:01.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f954d244700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95480aa670 con 0x7f95480bfd40 2024-01-24T15:14:01.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f954659c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f95480c8490 0x7f95480a9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40244/0 (socket says 172.21.15.29:40244) 2024-01-24T15:14:01.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f954659c700 1 -- 172.21.15.29:0/2800268243 learned_addr learned my addr 172.21.15.29:0/2800268243 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:01.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f954759e700 1 --2- 172.21.15.29:0/2800268243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95480bfd40 0x7f95480c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f9546d9d700 1 --2- 172.21.15.29:0/2800268243 >> v2:172.21.15.163:3300/0 conn(0x7f9548064760 0x7f9548064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f9545d9b700 1 -- 172.21.15.29:0/2800268243 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084001312 0 0) 0x7f9548152240 con 0x7f95480c8490 2024-01-24T15:14:01.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f9545d9b700 1 -- 172.21.15.29:0/2800268243 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95300033f0 con 0x7f95480c8490 2024-01-24T15:14:01.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f9546d9d700 1 -- 172.21.15.29:0/2800268243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95480bfd40 msgr2=0x7f95480c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:01.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f9546d9d700 1 --2- 172.21.15.29:0/2800268243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95480bfd40 0x7f95480c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f9546d9d700 1 -- 172.21.15.29:0/2800268243 >> v1:172.21.15.29:6789/0 conn(0x7f95480c8490 legacy=0x7f95480a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f9546d9d700 1 -- 172.21.15.29:0/2800268243 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9548157d90 con 0x7f9548064760 2024-01-24T15:14:01.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.126+0000 7f954759e700 1 --2- 172.21.15.29:0/2800268243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95480bfd40 0x7f95480c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:01.138 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.127+0000 7f9546d9d700 1 --2- 172.21.15.29:0/2800268243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9548064760 0x7f9548064b40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f9538009760 tx=0x7f9538009060).ready entity=mon.1 client_cookie=431b48645bef6456 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:01.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.127+0000 7f9545d9b700 1 -- 172.21.15.29:0/2800268243 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953800f510 con 0x7f9548064760 2024-01-24T15:14:01.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.127+0000 7f9545d9b700 1 -- 172.21.15.29:0/2800268243 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953800f670 con 0x7f9548064760 2024-01-24T15:14:01.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.127+0000 7f9545d9b700 1 -- 172.21.15.29:0/2800268243 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953800f9f0 con 0x7f9548064760 2024-01-24T15:14:01.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.127+0000 7f954d244700 1 -- 172.21.15.29:0/2800268243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9548064760 msgr2=0x7f9548064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.127+0000 7f954d244700 1 --2- 172.21.15.29:0/2800268243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9548064760 0x7f9548064b40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f9538009760 tx=0x7f9538009060).stop 2024-01-24T15:14:01.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.127+0000 7f954d244700 1 -- 172.21.15.29:0/2800268243 shutdown_connections 2024-01-24T15:14:01.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.127+0000 7f954d244700 1 --2- 172.21.15.29:0/2800268243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95480bfd40 0x7f95480c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.127+0000 7f954d244700 1 --2- 172.21.15.29:0/2800268243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9548064760 0x7f9548064b40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.127+0000 7f954d244700 1 -- 172.21.15.29:0/2800268243 >> 172.21.15.29:0/2800268243 conn(0x7f95480cb8b0 msgr2=0x7f95480c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:01.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.128+0000 7f954d244700 1 -- 172.21.15.29:0/2800268243 shutdown_connections 2024-01-24T15:14:01.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.128+0000 7f954d244700 1 -- 172.21.15.29:0/2800268243 wait complete. 2024-01-24T15:14:01.139 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.129+0000 7f954d244700 1 Processor -- start 2024-01-24T15:14:01.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.129+0000 7f954d244700 1 -- start start 2024-01-24T15:14:01.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954d244700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9548064760 0x7f954814efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954d244700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95480bfd40 0x7f9548153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954d244700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95480c8490 0x7f9548152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954d244700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9548152a30 con 0x7f95480bfd40 2024-01-24T15:14:01.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954d244700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9548154760 con 0x7f95480c8490 2024-01-24T15:14:01.140 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954d244700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9548154480 con 0x7f9548064760 2024-01-24T15:14:01.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95480bfd40 0x7f9548153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f9546d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9548064760 0x7f954814efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95480c8490 0x7f9548152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95480bfd40 0x7f9548153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50422/0 (socket says 172.21.15.29:50422) 2024-01-24T15:14:01.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954659c700 1 -- 172.21.15.29:0/3688772676 learned_addr learned my addr 172.21.15.29:0/3688772676 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:01.141 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954759e700 1 -- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9548064760 msgr2=0x7f954814efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954759e700 1 --2- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9548064760 0x7f954814efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954759e700 1 -- 172.21.15.29:0/3688772676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95480bfd40 msgr2=0x7f9548153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954759e700 1 --2- 172.21.15.29:0/3688772676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95480bfd40 0x7f9548153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.130+0000 7f954759e700 1 -- 172.21.15.29:0/3688772676 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540009a20 con 0x7f95480c8490 2024-01-24T15:14:01.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.131+0000 7f954759e700 1 --2- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95480c8490 0x7f9548152ec0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f9540009f10 tx=0x7f954000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:01.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.131+0000 7f952effd700 1 -- 172.21.15.29:0/3688772676 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954000cb70 con 0x7f95480c8490 2024-01-24T15:14:01.142 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.131+0000 7f952effd700 1 -- 172.21.15.29:0/3688772676 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954000f040 con 0x7f95480c8490 2024-01-24T15:14:01.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.131+0000 7f952effd700 1 -- 172.21.15.29:0/3688772676 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540013400 con 0x7f95480c8490 2024-01-24T15:14:01.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.131+0000 7f954d244700 1 -- 172.21.15.29:0/3688772676 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95380092a0 con 0x7f95480c8490 2024-01-24T15:14:01.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.131+0000 7f954d244700 1 -- 172.21.15.29:0/3688772676 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95481526c0 con 0x7f95480c8490 2024-01-24T15:14:01.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.132+0000 7f952effd700 1 -- 172.21.15.29:0/3688772676 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 71) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f95400137f0 con 0x7f95480c8490 2024-01-24T15:14:01.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.132+0000 7f954d244700 1 -- 172.21.15.29:0/3688772676 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9548152240 con 0x7f95480c8490 2024-01-24T15:14:01.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.132+0000 7f952effd700 1 --2- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f953405f300 0x7f95340617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.133+0000 7f952effd700 1 -- 172.21.15.29:0/3688772676 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f954007b4f0 con 0x7f95480c8490 2024-01-24T15:14:01.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.133+0000 7f9546d9d700 1 -- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f953405f300 msgr2=0x7f95340617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:01.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.133+0000 7f9546d9d700 1 --2- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f953405f300 0x7f95340617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:14:01.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.137+0000 7f952effd700 1 -- 172.21.15.29:0/3688772676 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9548152240 con 0x7f95480c8490 2024-01-24T15:14:01.293 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:01.291+0000 7f88032b7680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:14:01.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.334+0000 7f9546d9d700 1 -- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f953405f300 msgr2=0x7f95340617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:01.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.334+0000 7f9546d9d700 1 --2- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f953405f300 0x7f95340617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:14:01.351 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:01.348+0000 7f51d2314680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:14:01.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.355+0000 7f954d244700 1 -- 172.21.15.29:0/3688772676 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9548152240 con 0x7f95480c8490 2024-01-24T15:14:01.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.356+0000 7f952effd700 1 -- 172.21.15.29:0/3688772676 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v179) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9548152240 con 0x7f95480c8490 2024-01-24T15:14:01.358 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T15:14:01.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.358+0000 7f954d244700 1 -- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f953405f300 msgr2=0x7f95340617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:14:01.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.358+0000 7f954d244700 1 --2- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f953405f300 0x7f95340617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:01.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.358+0000 7f954d244700 1 -- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95480c8490 msgr2=0x7f9548152ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.358+0000 7f954d244700 1 --2- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95480c8490 0x7f9548152ec0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f9540009f10 tx=0x7f954000b040).stop 2024-01-24T15:14:01.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.358+0000 7f954d244700 1 -- 172.21.15.29:0/3688772676 shutdown_connections 2024-01-24T15:14:01.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.358+0000 7f954d244700 1 --2- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95480c8490 0x7f9548152ec0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.358+0000 7f954d244700 1 --2- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9548064760 0x7f954814efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.358+0000 7f954d244700 1 --2- 172.21.15.29:0/3688772676 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f953405f300 0x7f95340617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:01.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.358+0000 7f954d244700 1 --2- 172.21.15.29:0/3688772676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95480bfd40 0x7f9548153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.358+0000 7f954d244700 1 -- 172.21.15.29:0/3688772676 >> 172.21.15.29:0/3688772676 conn(0x7f95480cb8b0 msgr2=0x7f95481519b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:01.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.358+0000 7f954d244700 1 -- 172.21.15.29:0/3688772676 shutdown_connections 2024-01-24T15:14:01.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.358+0000 7f954d244700 1 -- 172.21.15.29:0/3688772676 wait complete. 2024-01-24T15:14:01.374 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:14:01.379 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:01.378+0000 7f88032b7680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:14:01.458 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:01.456+0000 7f88032b7680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:14:01.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.503+0000 7fb315e58700 1 Processor -- start 2024-01-24T15:14:01.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.503+0000 7fb315e58700 1 -- start start 2024-01-24T15:14:01.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.504+0000 7fb315e58700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb310154370 0x7fb3100c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.504+0000 7fb315e58700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3100aa0e0 0x7fb310154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.504+0000 7fb315e58700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3100aa6c0 con 0x7fb3100ab1f0 2024-01-24T15:14:01.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.504+0000 7fb315e58700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb310064240 con 0x7fb310154370 2024-01-24T15:14:01.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.504+0000 7fb315e58700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100cb5c0 con 0x7fb3100aa0e0 2024-01-24T15:14:01.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb30f7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb310154370 0x7fb3100c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb30f7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb310154370 0x7fb3100c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50434/0 (socket says 172.21.15.29:50434) 2024-01-24T15:14:01.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb30f7fe700 1 -- 172.21.15.29:0/3779737469 learned_addr learned my addr 172.21.15.29:0/3779737469 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:01.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb30ffff700 1 --1- 172.21.15.29:0/3779737469 >> v1:172.21.15.29:6789/0 conn(0x7fb3100ab1f0 0x7fb3100631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40254/0 (socket says 172.21.15.29:40254) 2024-01-24T15:14:01.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb306ffd700 1 --2- 172.21.15.29:0/3779737469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3100aa0e0 0x7fb310154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb30f7fe700 1 -- 172.21.15.29:0/3779737469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3100aa0e0 msgr2=0x7fb310154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:01.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb30f7fe700 1 --2- 172.21.15.29:0/3779737469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3100aa0e0 0x7fb310154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb30f7fe700 1 -- 172.21.15.29:0/3779737469 >> v1:172.21.15.29:6789/0 conn(0x7fb3100ab1f0 legacy=0x7fb3100631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb30f7fe700 1 -- 172.21.15.29:0/3779737469 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3100cb8a0 con 0x7fb310154370 2024-01-24T15:14:01.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb30f7fe700 1 --2- 172.21.15.29:0/3779737469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb310154370 0x7fb3100c5410 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7fb300009810 tx=0x7fb300009060).ready entity=mon.0 client_cookie=3a7a0917d9eea838 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:01.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb30effd700 1 -- 172.21.15.29:0/3779737469 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300005880 con 0x7fb310154370 2024-01-24T15:14:01.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb30effd700 1 -- 172.21.15.29:0/3779737469 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3000059e0 con 0x7fb310154370 2024-01-24T15:14:01.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.505+0000 7fb30effd700 1 -- 172.21.15.29:0/3779737469 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300005d60 con 0x7fb310154370 2024-01-24T15:14:01.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.506+0000 7fb315e58700 1 -- 172.21.15.29:0/3779737469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb310154370 msgr2=0x7fb3100c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.506+0000 7fb315e58700 1 --2- 172.21.15.29:0/3779737469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb310154370 0x7fb3100c5410 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7fb300009810 tx=0x7fb300009060).stop 2024-01-24T15:14:01.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.506+0000 7fb315e58700 1 -- 172.21.15.29:0/3779737469 shutdown_connections 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.506+0000 7fb315e58700 1 --2- 172.21.15.29:0/3779737469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb310154370 0x7fb3100c5410 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.506+0000 7fb315e58700 1 --2- 172.21.15.29:0/3779737469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3100aa0e0 0x7fb310154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.506+0000 7fb315e58700 1 -- 172.21.15.29:0/3779737469 >> 172.21.15.29:0/3779737469 conn(0x7fb31005ec80 msgr2=0x7fb3100cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.506+0000 7fb315e58700 1 -- 172.21.15.29:0/3779737469 shutdown_connections 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.507+0000 7fb315e58700 1 -- 172.21.15.29:0/3779737469 wait complete. 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.508+0000 7fb315e58700 1 Processor -- start 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.508+0000 7fb315e58700 1 -- start start 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb315e58700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3100aa0e0 0x7fb3100c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb315e58700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3100ab1f0 0x7fb310159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb315e58700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb310154370 0x7fb3100c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb315e58700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100c23a0 con 0x7fb3100aa0e0 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb315e58700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101648f0 con 0x7fb3100ab1f0 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb315e58700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb310164bd0 con 0x7fb310154370 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb30f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3100aa0e0 0x7fb3100c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb30f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3100aa0e0 0x7fb3100c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50440/0 (socket says 172.21.15.29:50440) 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb30f7fe700 1 -- 172.21.15.29:0/3886018665 learned_addr learned my addr 172.21.15.29:0/3886018665 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb30ffff700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb310154370 0x7fb3100c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb306ffd700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3100ab1f0 0x7fb310159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb30f7fe700 1 -- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb310154370 msgr2=0x7fb3100c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb30f7fe700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb310154370 0x7fb3100c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb30f7fe700 1 -- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3100ab1f0 msgr2=0x7fb310159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb30f7fe700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3100ab1f0 0x7fb310159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb30f7fe700 1 -- 172.21.15.29:0/3886018665 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb300009300 con 0x7fb3100aa0e0 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.509+0000 7fb30f7fe700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3100aa0e0 0x7fb3100c1eb0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7fb300009eb0 tx=0x7fb300016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.510+0000 7fb307fff700 1 -- 172.21.15.29:0/3886018665 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300006770 con 0x7fb3100aa0e0 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.510+0000 7fb307fff700 1 -- 172.21.15.29:0/3886018665 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30000b790 con 0x7fb3100aa0e0 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.510+0000 7fb307fff700 1 -- 172.21.15.29:0/3886018665 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30000bb10 con 0x7fb3100aa0e0 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.510+0000 7fb315e58700 1 -- 172.21.15.29:0/3886018665 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb310164eb0 con 0x7fb3100aa0e0 2024-01-24T15:14:01.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.510+0000 7fb315e58700 1 -- 172.21.15.29:0/3886018665 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31015a0b0 con 0x7fb3100aa0e0 2024-01-24T15:14:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.511+0000 7fb315e58700 1 -- 172.21.15.29:0/3886018665 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3100475e0 con 0x7fb3100aa0e0 2024-01-24T15:14:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.512+0000 7fb307fff700 1 -- 172.21.15.29:0/3886018665 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 71) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fb300018070 con 0x7fb3100aa0e0 2024-01-24T15:14:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.512+0000 7fb307fff700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2f005f300 0x7fb2f00617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.512+0000 7fb306ffd700 1 -- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2f005f300 msgr2=0x7fb2f00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:01.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.512+0000 7fb307fff700 1 -- 172.21.15.29:0/3886018665 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7fb30007b890 con 0x7fb3100aa0e0 2024-01-24T15:14:01.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.512+0000 7fb306ffd700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2f005f300 0x7fb2f00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:14:01.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.517+0000 7fb307fff700 1 -- 172.21.15.29:0/3886018665 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb30004c3d0 con 0x7fb3100aa0e0 2024-01-24T15:14:01.538 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:01.536+0000 7f51d2314680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:14:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.713+0000 7fb306ffd700 1 -- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2f005f300 msgr2=0x7fb2f00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:01.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.713+0000 7fb306ffd700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2f005f300 0x7fb2f00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:14:01.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.746+0000 7fb315e58700 1 -- 172.21.15.29:0/3886018665 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3100cbb80 con 0x7fb3100aa0e0 2024-01-24T15:14:01.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.747+0000 7fb307fff700 1 -- 172.21.15.29:0/3886018665 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fb300009160 con 0x7fb3100aa0e0 2024-01-24T15:14:01.749 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:01.749 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7319,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3000519646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3000519646},{"type":"v1","addr":"172.21.15.163:6836","nonce":3000519646}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3497543927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3497543927},{"type":"v1","addr":"172.21.15.29:6835","nonce":3497543927}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2353694052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2353694052},{"type":"v1","addr":"172.21.15.163:6837","nonce":2353694052}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:13:38.524646+0000","modified":"2024-01-24T15:13:41.539080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/3595147603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3595147603},{"type":"v1","addr":"172.21.15.29:6834","nonce":3595147603}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-24T15:14:01.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.749+0000 7fb315e58700 1 -- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2f005f300 msgr2=0x7fb2f00617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:14:01.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.749+0000 7fb315e58700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2f005f300 0x7fb2f00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:01.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.749+0000 7fb315e58700 1 -- 172.21.15.29:0/3886018665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3100aa0e0 msgr2=0x7fb3100c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.749+0000 7fb315e58700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3100aa0e0 0x7fb3100c1eb0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7fb300009eb0 tx=0x7fb300016000).stop 2024-01-24T15:14:01.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.749+0000 7fb315e58700 1 -- 172.21.15.29:0/3886018665 shutdown_connections 2024-01-24T15:14:01.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.749+0000 7fb315e58700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb310154370 0x7fb3100c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.749+0000 7fb315e58700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb2f005f300 0x7fb2f00617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:01.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.749+0000 7fb315e58700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3100aa0e0 0x7fb3100c1eb0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.749+0000 7fb315e58700 1 --2- 172.21.15.29:0/3886018665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3100ab1f0 0x7fb310159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.749+0000 7fb315e58700 1 -- 172.21.15.29:0/3886018665 >> 172.21.15.29:0/3886018665 conn(0x7fb31005ec80 msgr2=0x7fb3101507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:01.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.749+0000 7fb315e58700 1 -- 172.21.15.29:0/3886018665 shutdown_connections 2024-01-24T15:14:01.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.749+0000 7fb315e58700 1 -- 172.21.15.29:0/3886018665 wait complete. 2024-01-24T15:14:01.753 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 74 2024-01-24T15:14:01.759 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:01.757+0000 7f88032b7680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:14:01.765 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:14:01.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.939+0000 7f494b96a700 1 Processor -- start 2024-01-24T15:14:01.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.940+0000 7f494b96a700 1 -- start start 2024-01-24T15:14:01.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f494b96a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4944154370 0x7f49440c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f494b96a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4944154750 0x7f49440631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f494b96a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49440aa6c0 con 0x7f49440aa0e0 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f494b96a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4944064240 con 0x7f4944154370 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f494b96a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49440cb5c0 con 0x7f4944154750 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f4949706700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4944154370 0x7f49440c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f4949706700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4944154370 0x7f49440c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50452/0 (socket says 172.21.15.29:50452) 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f4949706700 1 -- 172.21.15.29:0/511469382 learned_addr learned my addr 172.21.15.29:0/511469382 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f4949f07700 1 --2- 172.21.15.29:0/511469382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4944154750 0x7f49440631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f4949706700 1 -- 172.21.15.29:0/511469382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4944154750 msgr2=0x7f49440631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f4949706700 1 --2- 172.21.15.29:0/511469382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4944154750 0x7f49440631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f4949706700 1 -- 172.21.15.29:0/511469382 >> v1:172.21.15.163:6789/0 conn(0x7f49440aa0e0 legacy=0x7f49440ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f493bfff700 1 -- 172.21.15.29:0/511469382 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412240809 0 0) 0x7f49440aa6c0 con 0x7f49440aa0e0 2024-01-24T15:14:01.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f4949706700 1 -- 172.21.15.29:0/511469382 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440cb8a0 con 0x7f4944154370 2024-01-24T15:14:01.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.941+0000 7f4949706700 1 --2- 172.21.15.29:0/511469382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4944154370 0x7f49440c5410 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f4940009810 tx=0x7f4940009060).ready entity=mon.0 client_cookie=1cb20c94e7e84128 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:01.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.942+0000 7f493bfff700 1 -- 172.21.15.29:0/511469382 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940005880 con 0x7f4944154370 2024-01-24T15:14:01.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.942+0000 7f493bfff700 1 -- 172.21.15.29:0/511469382 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49400059e0 con 0x7f4944154370 2024-01-24T15:14:01.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.942+0000 7f493bfff700 1 -- 172.21.15.29:0/511469382 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940005d60 con 0x7f4944154370 2024-01-24T15:14:01.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.942+0000 7f494b96a700 1 -- 172.21.15.29:0/511469382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4944154370 msgr2=0x7f49440c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.942+0000 7f494b96a700 1 --2- 172.21.15.29:0/511469382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4944154370 0x7f49440c5410 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f4940009810 tx=0x7f4940009060).stop 2024-01-24T15:14:01.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.942+0000 7f494b96a700 1 -- 172.21.15.29:0/511469382 shutdown_connections 2024-01-24T15:14:01.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.942+0000 7f494b96a700 1 --2- 172.21.15.29:0/511469382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4944154750 0x7f49440631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.942+0000 7f494b96a700 1 --2- 172.21.15.29:0/511469382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4944154370 0x7f49440c5410 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.942+0000 7f494b96a700 1 -- 172.21.15.29:0/511469382 >> 172.21.15.29:0/511469382 conn(0x7f494405ec80 msgr2=0x7f49440cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:01.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.942+0000 7f494b96a700 1 -- 172.21.15.29:0/511469382 shutdown_connections 2024-01-24T15:14:01.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.942+0000 7f494b96a700 1 -- 172.21.15.29:0/511469382 wait complete. 2024-01-24T15:14:01.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.944+0000 7f494b96a700 1 Processor -- start 2024-01-24T15:14:01.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.944+0000 7f494b96a700 1 -- start start 2024-01-24T15:14:01.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f494b96a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49440aa0e0 0x7f4944158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f494b96a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4944154370 0x7f49441598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f494b96a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4944154750 0x7f494414cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f494b96a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944167c60 con 0x7f49440aa0e0 2024-01-24T15:14:01.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f494b96a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944159db0 con 0x7f4944154750 2024-01-24T15:14:01.948 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f494b96a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4944158730 con 0x7f4944154370 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f4949f07700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4944154750 0x7f494414cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f4948f05700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4944154370 0x7f49441598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f4949f07700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4944154750 0x7f494414cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:34736/0 (socket says 172.21.15.29:34736) 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f4949f07700 1 -- 172.21.15.29:0/3533410745 learned_addr learned my addr 172.21.15.29:0/3533410745 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f4949706700 1 --2- 172.21.15.29:0/3533410745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49440aa0e0 0x7f4944158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f4948f05700 1 -- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4944154750 msgr2=0x7f494414cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f4948f05700 1 --2- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4944154750 0x7f494414cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f4948f05700 1 -- 172.21.15.29:0/3533410745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49440aa0e0 msgr2=0x7f4944158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f4948f05700 1 --2- 172.21.15.29:0/3533410745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49440aa0e0 0x7f4944158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f4948f05700 1 -- 172.21.15.29:0/3533410745 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940009300 con 0x7f4944154370 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.945+0000 7f4948f05700 1 --2- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4944154370 0x7f49441598c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f4934001eb0 tx=0x7f4934007bb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.946+0000 7f49397fa700 1 -- 172.21.15.29:0/3533410745 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934012400 con 0x7f4944154370 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.946+0000 7f49397fa700 1 -- 172.21.15.29:0/3533410745 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4934012580 con 0x7f4944154370 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.946+0000 7f494b96a700 1 -- 172.21.15.29:0/3533410745 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4944158a10 con 0x7f4944154370 2024-01-24T15:14:01.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.946+0000 7f49397fa700 1 -- 172.21.15.29:0/3533410745 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934012940 con 0x7f4944154370 2024-01-24T15:14:01.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.946+0000 7f494b96a700 1 -- 172.21.15.29:0/3533410745 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4944158fd0 con 0x7f4944154370 2024-01-24T15:14:01.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.947+0000 7f494b96a700 1 -- 172.21.15.29:0/3533410745 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4944167dd0 con 0x7f4944154370 2024-01-24T15:14:01.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.947+0000 7f49397fa700 1 -- 172.21.15.29:0/3533410745 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 71) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f4934012cb0 con 0x7f4944154370 2024-01-24T15:14:01.950 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.948+0000 7f49397fa700 1 --2- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f492805f300 0x7f49280617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:01.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.948+0000 7f4949706700 1 -- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f492805f300 msgr2=0x7f49280617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:01.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.948+0000 7f49397fa700 1 -- 172.21.15.29:0/3533410745 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f4934079df0 con 0x7f4944154370 2024-01-24T15:14:01.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.948+0000 7f4949706700 1 --2- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f492805f300 0x7f49280617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:14:01.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:01.954+0000 7f49397fa700 1 -- 172.21.15.29:0/3533410745 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f493404a990 con 0x7f4944154370 2024-01-24T15:14:01.976 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:01.974+0000 7f88032b7680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:14:02.143 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.141+0000 7f494b96a700 1 -- 172.21.15.29:0/3533410745 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49440475e0 con 0x7f4944154370 2024-01-24T15:14:02.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.142+0000 7f49397fa700 1 -- 172.21.15.29:0/3533410745 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+18628 (secure 0 0 0) 0x7f493404dfb0 con 0x7f4944154370 2024-01-24T15:14:02.145 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:02.145 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":179,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:13:55.886363+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:13:36.272483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-24T15:13:37.504363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6833/43241":"2024-01-25T15:13:55.886328+0000","172.21.15.163:6832/43241":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/42051548":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/4250314662":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/4170284440":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/1833132326":"2024-01-25T15:13:55.886328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:14:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.144+0000 7f494b96a700 1 -- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f492805f300 msgr2=0x7f49280617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:14:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.144+0000 7f494b96a700 1 --2- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f492805f300 0x7f49280617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.144+0000 7f494b96a700 1 -- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4944154370 msgr2=0x7f49441598c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.144+0000 7f494b96a700 1 --2- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4944154370 0x7f49441598c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f4934001eb0 tx=0x7f4934007bb0).stop 2024-01-24T15:14:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.144+0000 7f494b96a700 1 -- 172.21.15.29:0/3533410745 shutdown_connections 2024-01-24T15:14:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.144+0000 7f494b96a700 1 --2- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4944154750 0x7f494414cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.144+0000 7f494b96a700 1 --2- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f492805f300 0x7f49280617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:02.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.144+0000 7f494b96a700 1 --2- 172.21.15.29:0/3533410745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49440aa0e0 0x7f4944158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.144+0000 7f494b96a700 1 --2- 172.21.15.29:0/3533410745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4944154370 0x7f49441598c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.144+0000 7f494b96a700 1 -- 172.21.15.29:0/3533410745 >> 172.21.15.29:0/3533410745 conn(0x7f494405ec80 msgr2=0x7f49440ca4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:02.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.144+0000 7f494b96a700 1 -- 172.21.15.29:0/3533410745 shutdown_connections 2024-01-24T15:14:02.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.145+0000 7f494b96a700 1 -- 172.21.15.29:0/3533410745 wait complete. 2024-01-24T15:14:02.161 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T15:14:02.161 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:14:02.194 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:02.191+0000 7f51d2314680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:14:02.263 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:02.261+0000 7f51d2314680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:14:02.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.291+0000 7fe3f5eb6700 1 Processor -- start 2024-01-24T15:14:02.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.291+0000 7fe3f5eb6700 1 -- start start 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.292+0000 7fe3f5eb6700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe3f0062970 0x7fe3f0062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.292+0000 7fe3f5eb6700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f00bcf60 con 0x7fe3f0066b40 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.292+0000 7fe3f5eb6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f00a6ec0 con 0x7fe3f0158600 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.292+0000 7fe3f5eb6700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f00bdba0 con 0x7fe3f0062970 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.292+0000 7fe3ef7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe3f0062970 0x7fe3f0062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.292+0000 7fe3ef7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe3f0062970 0x7fe3f0062d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50464/0 (socket says 172.21.15.29:50464) 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.292+0000 7fe3ef7fe700 1 -- 172.21.15.29:0/3392418073 learned_addr learned my addr 172.21.15.29:0/3392418073 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.292+0000 7fe3ef7fe700 1 -- 172.21.15.29:0/3392418073 >> v1:172.21.15.163:6789/0 conn(0x7fe3f0158600 legacy=0x7fe3f0154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.292+0000 7fe3ef7fe700 1 -- 172.21.15.29:0/3392418073 >> v1:172.21.15.29:6789/0 conn(0x7fe3f0066b40 legacy=0x7fe3f00c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.292+0000 7fe3ef7fe700 1 -- 172.21.15.29:0/3392418073 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f00a71a0 con 0x7fe3f0062970 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.293+0000 7fe3ef7fe700 1 --2- 172.21.15.29:0/3392418073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f0062970 0x7fe3f0062d50 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7fe3e0006fd0 tx=0x7fe3e00090f0).ready entity=mon.0 client_cookie=fc524a880bf0f844 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.293+0000 7fe3ee7fc700 1 -- 172.21.15.29:0/3392418073 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000b600 con 0x7fe3f0062970 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.293+0000 7fe3ee7fc700 1 -- 172.21.15.29:0/3392418073 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e000b760 con 0x7fe3f0062970 2024-01-24T15:14:02.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.293+0000 7fe3ee7fc700 1 -- 172.21.15.29:0/3392418073 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000bae0 con 0x7fe3f0062970 2024-01-24T15:14:02.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.294+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/3392418073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f0062970 msgr2=0x7fe3f0062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:02.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.294+0000 7fe3f5eb6700 1 --2- 172.21.15.29:0/3392418073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f0062970 0x7fe3f0062d50 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7fe3e0006fd0 tx=0x7fe3e00090f0).stop 2024-01-24T15:14:02.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.294+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/3392418073 shutdown_connections 2024-01-24T15:14:02.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.294+0000 7fe3f5eb6700 1 --2- 172.21.15.29:0/3392418073 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f0062970 0x7fe3f0062d50 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.294+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/3392418073 >> 172.21.15.29:0/3392418073 conn(0x7fe3f00c8580 msgr2=0x7fe3f00c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:02.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.294+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/3392418073 shutdown_connections 2024-01-24T15:14:02.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.294+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/3392418073 wait complete. 2024-01-24T15:14:02.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.296+0000 7fe3f5eb6700 1 Processor -- start 2024-01-24T15:14:02.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.296+0000 7fe3f5eb6700 1 -- start start 2024-01-24T15:14:02.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.297+0000 7fe3f5eb6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f0062970 0x7fe3f0156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:02.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.297+0000 7fe3f5eb6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f0066b40 0x7fe3f0153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:02.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.297+0000 7fe3f5eb6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3f0158600 0x7fe3f00bf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:02.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.297+0000 7fe3f5eb6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f00a7480 con 0x7fe3f0062970 2024-01-24T15:14:02.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.297+0000 7fe3f5eb6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f01573e0 con 0x7fe3f0066b40 2024-01-24T15:14:02.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.297+0000 7fe3f5eb6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f0153680 con 0x7fe3f0158600 2024-01-24T15:14:02.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3ef7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f0062970 0x7fe3f0156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:02.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3ef7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f0062970 0x7fe3f0156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50480/0 (socket says 172.21.15.29:50480) 2024-01-24T15:14:02.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3ef7fe700 1 -- 172.21.15.29:0/4251445526 learned_addr learned my addr 172.21.15.29:0/4251445526 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:02.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3eeffd700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f0066b40 0x7fe3f0153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:02.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3effff700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3f0158600 0x7fe3f00bf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:02.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3ef7fe700 1 -- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3f0158600 msgr2=0x7fe3f00bf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:02.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3ef7fe700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3f0158600 0x7fe3f00bf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3ef7fe700 1 -- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f0066b40 msgr2=0x7fe3f0153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:02.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3ef7fe700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f0066b40 0x7fe3f0153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3ef7fe700 1 -- 172.21.15.29:0/4251445526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e0009370 con 0x7fe3f0062970 2024-01-24T15:14:02.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3ef7fe700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f0062970 0x7fe3f0156ef0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7fe3e0009f50 tx=0x7fe3e0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:02.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3d77fe700 1 -- 172.21.15.29:0/4251445526 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000b8d0 con 0x7fe3f0062970 2024-01-24T15:14:02.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3d77fe700 1 -- 172.21.15.29:0/4251445526 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e000ba30 con 0x7fe3f0062970 2024-01-24T15:14:02.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/4251445526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f00bebf0 con 0x7fe3f0062970 2024-01-24T15:14:02.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3d77fe700 1 -- 172.21.15.29:0/4251445526 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e001b640 con 0x7fe3f0062970 2024-01-24T15:14:02.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.298+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/4251445526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f0157bb0 con 0x7fe3f0062970 2024-01-24T15:14:02.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.299+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/4251445526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f00475e0 con 0x7fe3f0062970 2024-01-24T15:14:02.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.301+0000 7fe3d77fe700 1 -- 172.21.15.29:0/4251445526 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 71) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fe3e001b9b0 con 0x7fe3f0062970 2024-01-24T15:14:02.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.301+0000 7fe3d77fe700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3dc05f300 0x7fe3dc0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:02.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.301+0000 7fe3eeffd700 1 -- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3dc05f300 msgr2=0x7fe3dc0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:02.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.301+0000 7fe3d77fe700 1 -- 172.21.15.29:0/4251445526 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7fe3e007c6a0 con 0x7fe3f0062970 2024-01-24T15:14:02.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.301+0000 7fe3eeffd700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3dc05f300 0x7fe3dc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:14:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.305+0000 7fe3d77fe700 1 -- 172.21.15.29:0/4251445526 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe3e004d2c0 con 0x7fe3f0062970 2024-01-24T15:14:02.340 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:02.337+0000 7f51d2314680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:14:02.488 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:02.485+0000 7f51d2314680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:14:02.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.502+0000 7fe3eeffd700 1 -- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3dc05f300 msgr2=0x7fe3dc0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:02.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.502+0000 7fe3eeffd700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3dc05f300 0x7fe3dc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:14:02.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.533+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/4251445526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3f00bebf0 con 0x7fe3f0062970 2024-01-24T15:14:02.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.534+0000 7fe3d77fe700 1 -- 172.21.15.29:0/4251445526 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fe3f00bebf0 con 0x7fe3f0062970 2024-01-24T15:14:02.536 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:02.536 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7319,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3000519646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3000519646},{"type":"v1","addr":"172.21.15.163:6836","nonce":3000519646}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3497543927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3497543927},{"type":"v1","addr":"172.21.15.29:6835","nonce":3497543927}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2353694052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2353694052},{"type":"v1","addr":"172.21.15.163:6837","nonce":2353694052}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:13:38.524646+0000","modified":"2024-01-24T15:13:41.539080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/3595147603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3595147603},{"type":"v1","addr":"172.21.15.29:6834","nonce":3595147603}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-24T15:14:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.536+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3dc05f300 msgr2=0x7fe3dc0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:14:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.536+0000 7fe3f5eb6700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3dc05f300 0x7fe3dc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.536+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/4251445526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f0062970 msgr2=0x7fe3f0156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:02.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.536+0000 7fe3f5eb6700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f0062970 0x7fe3f0156ef0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7fe3e0009f50 tx=0x7fe3e0017000).stop 2024-01-24T15:14:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.536+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/4251445526 shutdown_connections 2024-01-24T15:14:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.536+0000 7fe3f5eb6700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3f0158600 0x7fe3f00bf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.536+0000 7fe3f5eb6700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe3dc05f300 0x7fe3dc0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.536+0000 7fe3f5eb6700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f0062970 0x7fe3f0156ef0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.536+0000 7fe3f5eb6700 1 --2- 172.21.15.29:0/4251445526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f0066b40 0x7fe3f0153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.536+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/4251445526 >> 172.21.15.29:0/4251445526 conn(0x7fe3f00c8580 msgr2=0x7fe3f0065360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.536+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/4251445526 shutdown_connections 2024-01-24T15:14:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.536+0000 7fe3f5eb6700 1 -- 172.21.15.29:0/4251445526 wait complete. 2024-01-24T15:14:02.539 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 74 2024-01-24T15:14:02.554 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:14:02.560 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:02.558+0000 7f51d2314680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:14:02.676 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:02.674+0000 7f51d2314680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:14:02.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.689+0000 7f1725915700 1 Processor -- start 2024-01-24T15:14:02.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.689+0000 7f1725915700 1 -- start start 2024-01-24T15:14:02.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f1725915700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f172014aac0 0x7f17200654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f1725915700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1720065990 0x7f17200c1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f1725915700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17200c22d0 con 0x7f17200c1e20 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f1725915700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1720150b00 con 0x7f172014aac0 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f1725915700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1720151920 con 0x7f1720065990 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f171f7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f17200c1e20 0x7f17201540d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40280/0 (socket says 172.21.15.29:40280) 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f171f7fe700 1 -- 172.21.15.29:0/4191223167 learned_addr learned my addr 172.21.15.29:0/4191223167 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f171e7fc700 1 --2- 172.21.15.29:0/4191223167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1720065990 0x7f17200c1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f171effd700 1 --2- 172.21.15.29:0/4191223167 >> v2:172.21.15.163:3300/0 conn(0x7f172014aac0 0x7f17200654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f171dffb700 1 -- 172.21.15.29:0/4191223167 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634337075 0 0) 0x7f1718001490 con 0x7f17200c1e20 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f171dffb700 1 -- 172.21.15.29:0/4191223167 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17080033f0 con 0x7f17200c1e20 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f171e7fc700 1 -- 172.21.15.29:0/4191223167 >> v2:172.21.15.163:3300/0 conn(0x7f172014aac0 msgr2=0x7f17200654a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f171e7fc700 1 --2- 172.21.15.29:0/4191223167 >> v2:172.21.15.163:3300/0 conn(0x7f172014aac0 0x7f17200654a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f171e7fc700 1 -- 172.21.15.29:0/4191223167 >> v1:172.21.15.29:6789/0 conn(0x7f17200c1e20 legacy=0x7f17201540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:02.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f171e7fc700 1 -- 172.21.15.29:0/4191223167 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1720150d10 con 0x7f1720065990 2024-01-24T15:14:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.690+0000 7f171e7fc700 1 --2- 172.21.15.29:0/4191223167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1720065990 0x7f17200c1930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f171400a900 tx=0x7f171400a200).ready entity=mon.2 client_cookie=f7f298054fee928f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.691+0000 7f171dffb700 1 -- 172.21.15.29:0/4191223167 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171400c6b0 con 0x7f1720065990 2024-01-24T15:14:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.691+0000 7f171dffb700 1 -- 172.21.15.29:0/4191223167 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171400c810 con 0x7f1720065990 2024-01-24T15:14:02.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.691+0000 7f171dffb700 1 -- 172.21.15.29:0/4191223167 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171400cb90 con 0x7f1720065990 2024-01-24T15:14:02.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.691+0000 7f1725915700 1 -- 172.21.15.29:0/4191223167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1720065990 msgr2=0x7f17200c1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:02.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.691+0000 7f1725915700 1 --2- 172.21.15.29:0/4191223167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1720065990 0x7f17200c1930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f171400a900 tx=0x7f171400a200).stop 2024-01-24T15:14:02.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.691+0000 7f1725915700 1 -- 172.21.15.29:0/4191223167 shutdown_connections 2024-01-24T15:14:02.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.691+0000 7f1725915700 1 --2- 172.21.15.29:0/4191223167 >> v2:172.21.15.163:3300/0 conn(0x7f172014aac0 0x7f17200654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.691+0000 7f1725915700 1 --2- 172.21.15.29:0/4191223167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1720065990 0x7f17200c1930 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.691+0000 7f1725915700 1 -- 172.21.15.29:0/4191223167 >> 172.21.15.29:0/4191223167 conn(0x7f17200c69f0 msgr2=0x7f17200c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:02.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.691+0000 7f1725915700 1 -- 172.21.15.29:0/4191223167 shutdown_connections 2024-01-24T15:14:02.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.691+0000 7f1725915700 1 -- 172.21.15.29:0/4191223167 wait complete. 2024-01-24T15:14:02.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.694+0000 7f1725915700 1 Processor -- start 2024-01-24T15:14:02.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.694+0000 7f1725915700 1 -- start start 2024-01-24T15:14:02.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.695+0000 7f1725915700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17200c1e20 0x7f17200bda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:02.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.695+0000 7f1725915700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172014aac0 0x7f17200abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:02.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.695+0000 7f1725915700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1720162430 0x7f1720145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:02.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.695+0000 7f1725915700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17200befe0 con 0x7f17200c1e20 2024-01-24T15:14:02.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.695+0000 7f1725915700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17200be370 con 0x7f1720162430 2024-01-24T15:14:02.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.695+0000 7f1725915700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17200be650 con 0x7f172014aac0 2024-01-24T15:14:02.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.695+0000 7f171effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17200c1e20 0x7f17200bda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:02.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.695+0000 7f171effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17200c1e20 0x7f17200bda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50496/0 (socket says 172.21.15.29:50496) 2024-01-24T15:14:02.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.695+0000 7f171effd700 1 -- 172.21.15.29:0/912806596 learned_addr learned my addr 172.21.15.29:0/912806596 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:02.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.695+0000 7f171e7fc700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172014aac0 0x7f17200abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:02.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.695+0000 7f171f7fe700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1720162430 0x7f1720145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:02.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.696+0000 7f171effd700 1 -- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172014aac0 msgr2=0x7f17200abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:02.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.696+0000 7f171effd700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172014aac0 0x7f17200abd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.696+0000 7f171effd700 1 -- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1720162430 msgr2=0x7f1720145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:02.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.696+0000 7f171effd700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1720162430 0x7f1720145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.696+0000 7f171effd700 1 -- 172.21.15.29:0/912806596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171400a440 con 0x7f17200c1e20 2024-01-24T15:14:02.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.696+0000 7f171effd700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17200c1e20 0x7f17200bda90 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f171000e900 tx=0x7f1710009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:02.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.696+0000 7f1706ffd700 1 -- 172.21.15.29:0/912806596 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710012600 con 0x7f17200c1e20 2024-01-24T15:14:02.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.696+0000 7f1706ffd700 1 -- 172.21.15.29:0/912806596 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1710012760 con 0x7f17200c1e20 2024-01-24T15:14:02.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.696+0000 7f1725915700 1 -- 172.21.15.29:0/912806596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17200be930 con 0x7f17200c1e20 2024-01-24T15:14:02.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.696+0000 7f1706ffd700 1 -- 172.21.15.29:0/912806596 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710012ae0 con 0x7f17200c1e20 2024-01-24T15:14:02.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.696+0000 7f1725915700 1 -- 172.21.15.29:0/912806596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17200bdf80 con 0x7f17200c1e20 2024-01-24T15:14:02.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.697+0000 7f1725915700 1 -- 172.21.15.29:0/912806596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17200475e0 con 0x7f17200c1e20 2024-01-24T15:14:02.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.698+0000 7f1706ffd700 1 -- 172.21.15.29:0/912806596 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 71) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f171001b400 con 0x7f17200c1e20 2024-01-24T15:14:02.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.698+0000 7f1706ffd700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f170c05f300 0x7f170c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:02.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.699+0000 7f1706ffd700 1 -- 172.21.15.29:0/912806596 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f171007ad00 con 0x7f17200c1e20 2024-01-24T15:14:02.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.699+0000 7f171e7fc700 1 -- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f170c05f300 msgr2=0x7f170c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:02.701 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.699+0000 7f171e7fc700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f170c05f300 0x7f170c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:14:02.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.703+0000 7f1706ffd700 1 -- 172.21.15.29:0/912806596 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f171007e030 con 0x7f17200c1e20 2024-01-24T15:14:02.751 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:02.749+0000 7f88032b7680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:14:02.785 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:02.783+0000 7f51d2314680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:14:02.831 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:02.829+0000 7f88032b7680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:14:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.900+0000 7f171e7fc700 1 -- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f170c05f300 msgr2=0x7f170c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.900+0000 7f171e7fc700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f170c05f300 0x7f170c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:14:02.918 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:02.916+0000 7f88032b7680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:14:02.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.932+0000 7f1725915700 1 -- 172.21.15.29:0/912806596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1720150ff0 con 0x7f17200c1e20 2024-01-24T15:14:02.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.932+0000 7f1706ffd700 1 -- 172.21.15.29:0/912806596 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f171004b840 con 0x7f17200c1e20 2024-01-24T15:14:02.934 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:02.934 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7319,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3000519646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3000519646},{"type":"v1","addr":"172.21.15.163:6836","nonce":3000519646}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3497543927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3497543927},{"type":"v1","addr":"172.21.15.29:6835","nonce":3497543927}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2353694052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2353694052},{"type":"v1","addr":"172.21.15.163:6837","nonce":2353694052}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:13:38.524646+0000","modified":"2024-01-24T15:13:41.539080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/3595147603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3595147603},{"type":"v1","addr":"172.21.15.29:6834","nonce":3595147603}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-24T15:14:02.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.934+0000 7f1725915700 1 -- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f170c05f300 msgr2=0x7f170c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:14:02.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.934+0000 7f1725915700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f170c05f300 0x7f170c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:02.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.934+0000 7f1725915700 1 -- 172.21.15.29:0/912806596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17200c1e20 msgr2=0x7f17200bda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:02.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.934+0000 7f1725915700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17200c1e20 0x7f17200bda90 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f171000e900 tx=0x7f1710009060).stop 2024-01-24T15:14:02.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.935+0000 7f1725915700 1 -- 172.21.15.29:0/912806596 shutdown_connections 2024-01-24T15:14:02.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.935+0000 7f1725915700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1720162430 0x7f1720145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.935+0000 7f1725915700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f170c05f300 0x7f170c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:02.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.935+0000 7f1725915700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17200c1e20 0x7f17200bda90 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.935+0000 7f1725915700 1 --2- 172.21.15.29:0/912806596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172014aac0 0x7f17200abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:02.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.935+0000 7f1725915700 1 -- 172.21.15.29:0/912806596 >> 172.21.15.29:0/912806596 conn(0x7f17200c69f0 msgr2=0x7f17201525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:02.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.935+0000 7f1725915700 1 -- 172.21.15.29:0/912806596 shutdown_connections 2024-01-24T15:14:02.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:02.935+0000 7f1725915700 1 -- 172.21.15.29:0/912806596 wait complete. 2024-01-24T15:14:02.938 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 74 2024-01-24T15:14:02.951 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:14:03.079 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:03.077+0000 7f51d2314680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:14:03.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.081+0000 7f7990ce4700 1 Processor -- start 2024-01-24T15:14:03.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.081+0000 7f7990ce4700 1 -- start start 2024-01-24T15:14:03.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.082+0000 7f7990ce4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f798c154370 0x7f798c0c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:03.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.082+0000 7f7990ce4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c154750 0x7f798c0631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:03.086 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.082+0000 7f7990ce4700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f798c0aa6c0 con 0x7f798c0aa0e0 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.082+0000 7f7990ce4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798c064240 con 0x7f798c154370 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.082+0000 7f7990ce4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798c0cb5c0 con 0x7f798c154750 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798a59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f798c154370 0x7f798c0c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798a59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f798c154370 0x7f798c0c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50512/0 (socket says 172.21.15.29:50512) 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f7989d9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f798c0aa0e0 0x7f798c0ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40294/0 (socket says 172.21.15.29:40294) 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798a59c700 1 -- 172.21.15.29:0/3357678310 learned_addr learned my addr 172.21.15.29:0/3357678310 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798ad9d700 1 --2- 172.21.15.29:0/3357678310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c154750 0x7f798c0631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798ad9d700 1 -- 172.21.15.29:0/3357678310 >> v2:172.21.15.29:3300/0 conn(0x7f798c154370 msgr2=0x7f798c0c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798ad9d700 1 --2- 172.21.15.29:0/3357678310 >> v2:172.21.15.29:3300/0 conn(0x7f798c154370 0x7f798c0c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798ad9d700 1 -- 172.21.15.29:0/3357678310 >> v1:172.21.15.29:6789/0 conn(0x7f798c0aa0e0 legacy=0x7f798c0ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798ad9d700 1 -- 172.21.15.29:0/3357678310 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798c0cb8a0 con 0x7f798c154750 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798a59c700 1 --2- 172.21.15.29:0/3357678310 >> v2:172.21.15.29:3300/0 conn(0x7f798c154370 0x7f798c0c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798ad9d700 1 --2- 172.21.15.29:0/3357678310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c154750 0x7f798c0631d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f7984009e60 tx=0x7f7984009780).ready entity=mon.2 client_cookie=3bba0bd877d7f6a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:03.087 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798959a700 1 -- 172.21.15.29:0/3357678310 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798400e070 con 0x7f798c154750 2024-01-24T15:14:03.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798959a700 1 -- 172.21.15.29:0/3357678310 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798400bb60 con 0x7f798c154750 2024-01-24T15:14:03.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.083+0000 7f798959a700 1 -- 172.21.15.29:0/3357678310 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984012410 con 0x7f798c154750 2024-01-24T15:14:03.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.084+0000 7f7990ce4700 1 -- 172.21.15.29:0/3357678310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c154750 msgr2=0x7f798c0631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:03.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.084+0000 7f7990ce4700 1 --2- 172.21.15.29:0/3357678310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c154750 0x7f798c0631d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f7984009e60 tx=0x7f7984009780).stop 2024-01-24T15:14:03.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.084+0000 7f7990ce4700 1 -- 172.21.15.29:0/3357678310 shutdown_connections 2024-01-24T15:14:03.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.084+0000 7f7990ce4700 1 --2- 172.21.15.29:0/3357678310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c154750 0x7f798c0631d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.084+0000 7f7990ce4700 1 --2- 172.21.15.29:0/3357678310 >> v2:172.21.15.29:3300/0 conn(0x7f798c154370 0x7f798c0c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.084+0000 7f7990ce4700 1 -- 172.21.15.29:0/3357678310 >> 172.21.15.29:0/3357678310 conn(0x7f798c05ec80 msgr2=0x7f798c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:03.088 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.084+0000 7f7990ce4700 1 -- 172.21.15.29:0/3357678310 shutdown_connections 2024-01-24T15:14:03.089 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.084+0000 7f7990ce4700 1 -- 172.21.15.29:0/3357678310 wait complete. 2024-01-24T15:14:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.086+0000 7f7990ce4700 1 Processor -- start 2024-01-24T15:14:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.086+0000 7f7990ce4700 1 -- start start 2024-01-24T15:14:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7990ce4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0aa0e0 0x7f798c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7990ce4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f798c154370 0x7f798c1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7990ce4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f798c154750 0x7f798c14cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7990ce4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798c159340 con 0x7f798c154370 2024-01-24T15:14:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7990ce4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798c158730 con 0x7f798c154750 2024-01-24T15:14:03.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7990ce4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798c158a10 con 0x7f798c0aa0e0 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7989d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f798c154370 0x7f798c1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7989d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f798c154370 0x7f798c1597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:50516/0 (socket says 172.21.15.29:50516) 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7989d9b700 1 -- 172.21.15.29:0/4183649472 learned_addr learned my addr 172.21.15.29:0/4183649472 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f798a59c700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0aa0e0 0x7f798c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f798ad9d700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f798c154750 0x7f798c14cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7989d9b700 1 -- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0aa0e0 msgr2=0x7f798c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7989d9b700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0aa0e0 0x7f798c158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7989d9b700 1 -- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f798c154750 msgr2=0x7f798c14cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7989d9b700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f798c154750 0x7f798c14cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.087+0000 7f7989d9b700 1 -- 172.21.15.29:0/4183649472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c009300 con 0x7f798c154370 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.088+0000 7f7989d9b700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f798c154370 0x7f798c1597d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f7980007fd0 tx=0x7f7980007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.088+0000 7f797a7fc700 1 -- 172.21.15.29:0/4183649472 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798000ce10 con 0x7f798c154370 2024-01-24T15:14:03.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.088+0000 7f797a7fc700 1 -- 172.21.15.29:0/4183649472 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79800072e0 con 0x7f798c154370 2024-01-24T15:14:03.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.088+0000 7f797a7fc700 1 -- 172.21.15.29:0/4183649472 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79800056e0 con 0x7f798c154370 2024-01-24T15:14:03.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.088+0000 7f7990ce4700 1 -- 172.21.15.29:0/4183649472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7984009a20 con 0x7f798c154370 2024-01-24T15:14:03.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.088+0000 7f7990ce4700 1 -- 172.21.15.29:0/4183649472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798c158fd0 con 0x7f798c154370 2024-01-24T15:14:03.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.089+0000 7f7990ce4700 1 -- 172.21.15.29:0/4183649472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f798c0475e0 con 0x7f798c154370 2024-01-24T15:14:03.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.090+0000 7f797a7fc700 1 -- 172.21.15.29:0/4183649472 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 71) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f7980005ad0 con 0x7f798c154370 2024-01-24T15:14:03.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.090+0000 7f797a7fc700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f796c05f300 0x7f796c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:03.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.090+0000 7f798a59c700 1 -- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f796c05f300 msgr2=0x7f796c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:03.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.090+0000 7f797a7fc700 1 -- 172.21.15.29:0/4183649472 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f798007a750 con 0x7f798c154370 2024-01-24T15:14:03.094 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.090+0000 7f798a59c700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f796c05f300 0x7f796c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:14:03.095 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:03.094+0000 7f88032b7680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:14:03.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.094+0000 7f797a7fc700 1 -- 172.21.15.29:0/4183649472 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f798004b270 con 0x7f798c154370 2024-01-24T15:14:03.152 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:03.150+0000 7f51d2314680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:14:03.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.170+0000 7f797a7fc700 1 -- 172.21.15.29:0/4183649472 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 72) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f7980077d70 con 0x7f798c154370 2024-01-24T15:14:03.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.170+0000 7f797a7fc700 1 -- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f796c05f300 msgr2=0x7f796c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:14:03.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.170+0000 7f797a7fc700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f796c05f300 0x7f796c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:03.186 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:03.184+0000 7f88032b7680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:14:03.237 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:03.234+0000 7f514f09c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:03.238 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:03.234+0000 7f514f09c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:03.238 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:03.234+0000 7f514f09c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:03.238 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:03.234+0000 7f514f09c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:03.238 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:03.234+0000 7f514f09c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:03.238 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:03.234+0000 7f514b895700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:03.238 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:03.234+0000 7f514b895700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:03.239 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:03.234+0000 7f514b895700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:03.239 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:03.234+0000 7f514b895700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:03.239 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:03.234+0000 7f514b895700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:03.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.288+0000 7f7990ce4700 1 -- 172.21.15.29:0/4183649472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f798c0cbb80 con 0x7f798c154370 2024-01-24T15:14:03.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.289+0000 7f797a7fc700 1 -- 172.21.15.29:0/4183649472 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+18940 (secure 0 0 0) 0x7f7980072ae0 con 0x7f798c154370 2024-01-24T15:14:03.292 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:03.292 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":180,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:14:03.165614+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:13:36.272483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-24T15:13:37.504363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/2489367159":"2024-01-25T15:14:03.165551+0000","172.21.15.163:0/2871742763":"2024-01-25T15:14:03.165551+0000","172.21.15.163:0/617019536":"2024-01-25T15:14:03.165551+0000","172.21.15.163:0/145348358":"2024-01-25T15:14:03.165551+0000","172.21.15.163:6833/43241":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/486866805":"2024-01-25T15:14:03.165551+0000","172.21.15.163:6832/43241":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/42051548":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/4250314662":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/4170284440":"2024-01-25T15:13:55.886328+0000","172.21.15.163:0/1833132326":"2024-01-25T15:13:55.886328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:14:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.292+0000 7f7990ce4700 1 -- 172.21.15.29:0/4183649472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f798c154370 msgr2=0x7f798c1597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.292+0000 7f7990ce4700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f798c154370 0x7f798c1597d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f7980007fd0 tx=0x7f7980007df0).stop 2024-01-24T15:14:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.292+0000 7f7990ce4700 1 -- 172.21.15.29:0/4183649472 shutdown_connections 2024-01-24T15:14:03.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.292+0000 7f7990ce4700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f798c154750 0x7f798c14cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.292+0000 7f7990ce4700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0aa0e0 0x7f798c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.292+0000 7f7990ce4700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f796c05f300 0x7f796c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:03.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.292+0000 7f7990ce4700 1 --2- 172.21.15.29:0/4183649472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f798c154370 0x7f798c1597d0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.292+0000 7f7990ce4700 1 -- 172.21.15.29:0/4183649472 >> 172.21.15.29:0/4183649472 conn(0x7f798c05ec80 msgr2=0x7f798c153260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:03.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.292+0000 7f7990ce4700 1 -- 172.21.15.29:0/4183649472 shutdown_connections 2024-01-24T15:14:03.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.292+0000 7f7990ce4700 1 -- 172.21.15.29:0/4183649472 wait complete. 2024-01-24T15:14:03.308 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-24T15:14:03.324 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:03.323+0000 7f88032b7680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:14:03.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.439+0000 7f7cb4d71700 1 Processor -- start 2024-01-24T15:14:03.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.439+0000 7f7cb4d71700 1 -- start start 2024-01-24T15:14:03.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.441+0000 7f7cb4d71700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb0135350 0x7f7cb00664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:03.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.441+0000 7f7cb4d71700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7cb0066990 0x7f7cb01525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:03.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.441+0000 7f7cb4d71700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb0156460 con 0x7f7cb0152ae0 2024-01-24T15:14:03.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.441+0000 7f7cb4d71700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb0155c00 con 0x7f7cb0066990 2024-01-24T15:14:03.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.441+0000 7f7cb4d71700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb0066e10 con 0x7f7cb0135350 2024-01-24T15:14:03.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.441+0000 7f7cadd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7cb0066990 0x7f7cb01525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:03.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.441+0000 7f7cae59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb0135350 0x7f7cb00664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:03.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.441+0000 7f7caed9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7cb0152ae0 0x7f7cb00679a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:41458/0 (socket says 172.21.15.29:41458) 2024-01-24T15:14:03.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.441+0000 7f7caed9d700 1 -- 172.21.15.29:0/3340386099 learned_addr learned my addr 172.21.15.29:0/3340386099 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:03.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.442+0000 7f7cadd9b700 1 -- 172.21.15.29:0/3340386099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb0135350 msgr2=0x7f7cb00664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:03.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.442+0000 7f7cadd9b700 1 --2- 172.21.15.29:0/3340386099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb0135350 0x7f7cb00664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.442+0000 7f7cadd9b700 1 -- 172.21.15.29:0/3340386099 >> v1:172.21.15.163:6789/0 conn(0x7f7cb0152ae0 legacy=0x7f7cb00679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.442+0000 7f7cadd9b700 1 -- 172.21.15.29:0/3340386099 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb0153390 con 0x7f7cb0066990 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.442+0000 7f7cad59a700 1 -- 172.21.15.29:0/3340386099 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220282799 0 0) 0x7f7cb0156460 con 0x7f7cb0152ae0 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.442+0000 7f7cadd9b700 1 --2- 172.21.15.29:0/3340386099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb0066990 0x7f7cb01525f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f7ca4009760 tx=0x7f7ca4009060).ready entity=mon.1 client_cookie=465226bf5d943ef0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.442+0000 7f7cad59a700 1 -- 172.21.15.29:0/3340386099 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400f510 con 0x7f7cb0066990 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.442+0000 7f7cad59a700 1 -- 172.21.15.29:0/3340386099 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca400f670 con 0x7f7cb0066990 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.442+0000 7f7cad59a700 1 -- 172.21.15.29:0/3340386099 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400f9f0 con 0x7f7cb0066990 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.443+0000 7f7cb4d71700 1 -- 172.21.15.29:0/3340386099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb0066990 msgr2=0x7f7cb01525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.443+0000 7f7cb4d71700 1 --2- 172.21.15.29:0/3340386099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb0066990 0x7f7cb01525f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f7ca4009760 tx=0x7f7ca4009060).stop 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.443+0000 7f7cb4d71700 1 -- 172.21.15.29:0/3340386099 shutdown_connections 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.443+0000 7f7cb4d71700 1 --2- 172.21.15.29:0/3340386099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb0135350 0x7f7cb00664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.443+0000 7f7cb4d71700 1 --2- 172.21.15.29:0/3340386099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb0066990 0x7f7cb01525f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.443+0000 7f7cb4d71700 1 -- 172.21.15.29:0/3340386099 >> 172.21.15.29:0/3340386099 conn(0x7f7cb00c5640 msgr2=0x7f7cb00c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.443+0000 7f7cb4d71700 1 -- 172.21.15.29:0/3340386099 shutdown_connections 2024-01-24T15:14:03.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.443+0000 7f7cb4d71700 1 -- 172.21.15.29:0/3340386099 wait complete. 2024-01-24T15:14:03.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.444+0000 7f7cb4d71700 1 Processor -- start 2024-01-24T15:14:03.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.444+0000 7f7cb4d71700 1 -- start start 2024-01-24T15:14:03.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.445+0000 7f7cb4d71700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb0066990 0x7f7cb00ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:03.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.445+0000 7f7cb4d71700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb0135350 0x7f7cb00ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:03.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.445+0000 7f7cb4d71700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb0152ae0 0x7f7cb00a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:03.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.445+0000 7f7cb4d71700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb0136290 con 0x7f7cb0152ae0 2024-01-24T15:14:03.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.445+0000 7f7cb4d71700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00ae250 con 0x7f7cb0066990 2024-01-24T15:14:03.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.445+0000 7f7cb4d71700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00ab6c0 con 0x7f7cb0135350 2024-01-24T15:14:03.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.445+0000 7f7caed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb0152ae0 0x7f7cb00a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.445+0000 7f7cae59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb0066990 0x7f7cb00ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.445+0000 7f7caed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb0152ae0 0x7f7cb00a9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41952/0 (socket says 172.21.15.29:41952) 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.445+0000 7f7caed9d700 1 -- 172.21.15.29:0/599248045 learned_addr learned my addr 172.21.15.29:0/599248045 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.445+0000 7f7cadd9b700 1 --2- 172.21.15.29:0/599248045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb0135350 0x7f7cb00ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.446+0000 7f7cae59c700 1 -- 172.21.15.29:0/599248045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb0135350 msgr2=0x7f7cb00ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.446+0000 7f7cae59c700 1 --2- 172.21.15.29:0/599248045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb0135350 0x7f7cb00ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.446+0000 7f7cae59c700 1 -- 172.21.15.29:0/599248045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb0152ae0 msgr2=0x7f7cb00a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.446+0000 7f7cae59c700 1 --2- 172.21.15.29:0/599248045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb0152ae0 0x7f7cb00a9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.446+0000 7f7cae59c700 1 -- 172.21.15.29:0/599248045 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca40092a0 con 0x7f7cb0066990 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.446+0000 7f7caed9d700 1 --2- 172.21.15.29:0/599248045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb0152ae0 0x7f7cb00a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.446+0000 7f7cae59c700 1 --2- 172.21.15.29:0/599248045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb0066990 0x7f7cb00ab1d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f7ca000e930 tx=0x7f7ca0009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.446+0000 7f7c9e7fc700 1 -- 172.21.15.29:0/599248045 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0009450 con 0x7f7cb0066990 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.446+0000 7f7c9e7fc700 1 -- 172.21.15.29:0/599248045 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0012600 con 0x7f7cb0066990 2024-01-24T15:14:03.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.446+0000 7f7cb4d71700 1 -- 172.21.15.29:0/599248045 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb01358f0 con 0x7f7cb0066990 2024-01-24T15:14:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.446+0000 7f7c9e7fc700 1 -- 172.21.15.29:0/599248045 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0012980 con 0x7f7cb0066990 2024-01-24T15:14:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.446+0000 7f7cb4d71700 1 -- 172.21.15.29:0/599248045 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb0135eb0 con 0x7f7cb0066990 2024-01-24T15:14:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.447+0000 7f7cb4d71700 1 -- 172.21.15.29:0/599248045 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb0136500 con 0x7f7cb0066990 2024-01-24T15:14:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.447+0000 7f7c9e7fc700 1 -- 172.21.15.29:0/599248045 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 72) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f7ca001b400 con 0x7f7cb0066990 2024-01-24T15:14:03.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.448+0000 7f7c9e7fc700 1 -- 172.21.15.29:0/599248045 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 7659+0+0 (secure 0 0 0) 0x7f7ca007a8b0 con 0x7f7cb0066990 2024-01-24T15:14:03.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.453+0000 7f7c9e7fc700 1 -- 172.21.15.29:0/599248045 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7ca004b6f0 con 0x7f7cb0066990 2024-01-24T15:14:03.455 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:03.453+0000 7f88032b7680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:14:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:03.680+0000 7f7cb4d71700 1 -- 172.21.15.29:0/599248045 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7cb00475e0 con 0x7f7cb0066990 2024-01-24T15:14:03.793 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:03.791+0000 7f88032b7680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:14:03.875 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:03.873+0000 7f88032b7680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:14:04.169 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-01-24T15:14:04.168+0000 7fe09bfc3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-24T15:14:04.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.171+0000 7f7c9e7fc700 1 -- 172.21.15.29:0/599248045 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7ca004ed10 con 0x7f7cb0066990 2024-01-24T15:14:04.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.174+0000 7f7cb4d71700 1 -- 172.21.15.29:0/599248045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb0066990 msgr2=0x7f7cb00ab1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:04.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.174+0000 7f7cb4d71700 1 --2- 172.21.15.29:0/599248045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb0066990 0x7f7cb00ab1d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f7ca000e930 tx=0x7f7ca0009060).stop 2024-01-24T15:14:04.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.174+0000 7f7cb4d71700 1 -- 172.21.15.29:0/599248045 shutdown_connections 2024-01-24T15:14:04.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.174+0000 7f7cb4d71700 1 --2- 172.21.15.29:0/599248045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cb0152ae0 0x7f7cb00a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:04.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.174+0000 7f7cb4d71700 1 --2- 172.21.15.29:0/599248045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cb0066990 0x7f7cb00ab1d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:04.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.174+0000 7f7cb4d71700 1 --2- 172.21.15.29:0/599248045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cb0135350 0x7f7cb00ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:04.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.174+0000 7f7cb4d71700 1 -- 172.21.15.29:0/599248045 >> 172.21.15.29:0/599248045 conn(0x7f7cb00c5640 msgr2=0x7f7cb01535b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:04.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.174+0000 7f7cb4d71700 1 -- 172.21.15.29:0/599248045 shutdown_connections 2024-01-24T15:14:04.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.174+0000 7f7cb4d71700 1 -- 172.21.15.29:0/599248045 wait complete. 2024-01-24T15:14:04.177 INFO:tasks.ceph.mds.c.smithi029.stderr: -2772> 2024-01-24T15:13:44.059+0000 7f9c98bd0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T15:14:04.178 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-24T15:14:04.200 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-24T15:14:04.212 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T15:14:04.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.346+0000 7f27bae5d700 1 Processor -- start 2024-01-24T15:14:04.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.346+0000 7f27bae5d700 1 -- start start 2024-01-24T15:14:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27bae5d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27b4064740 0x7f27b4064b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27bae5d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27b40c5af0 0x7f27b40c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27bae5d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b40c6820 con 0x7f27b4157cb0 2024-01-24T15:14:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27bae5d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b41556b0 con 0x7f27b4064740 2024-01-24T15:14:04.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27bae5d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b4155220 con 0x7f27b40c5af0 2024-01-24T15:14:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27b8bf9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27b4064740 0x7f27b4064b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27b8bf9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27b4064740 0x7f27b4064b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41972/0 (socket says 172.21.15.29:41972) 2024-01-24T15:14:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27b8bf9700 1 -- 172.21.15.29:0/3301177664 learned_addr learned my addr 172.21.15.29:0/3301177664 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27b3fff700 1 --2- 172.21.15.29:0/3301177664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27b40c5af0 0x7f27b40c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27b8bf9700 1 -- 172.21.15.29:0/3301177664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27b40c5af0 msgr2=0x7f27b40c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27b8bf9700 1 --2- 172.21.15.29:0/3301177664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27b40c5af0 0x7f27b40c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27b8bf9700 1 -- 172.21.15.29:0/3301177664 >> v1:172.21.15.163:6789/0 conn(0x7f27b4157cb0 legacy=0x7f27b4158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27b8bf9700 1 -- 172.21.15.29:0/3301177664 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b4154680 con 0x7f27b4064740 2024-01-24T15:14:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27b8bf9700 1 --2- 172.21.15.29:0/3301177664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b4064740 0x7f27b4064b20 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f27a8006fd0 tx=0x7f27a80090f0).ready entity=mon.0 client_cookie=3f496805db73e7f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27b37fe700 1 -- 172.21.15.29:0/3301177664 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a800b580 con 0x7f27b4064740 2024-01-24T15:14:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27b37fe700 1 -- 172.21.15.29:0/3301177664 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a800b6e0 con 0x7f27b4064740 2024-01-24T15:14:04.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.347+0000 7f27b37fe700 1 -- 172.21.15.29:0/3301177664 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a800ba60 con 0x7f27b4064740 2024-01-24T15:14:04.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.348+0000 7f27bae5d700 1 -- 172.21.15.29:0/3301177664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b4064740 msgr2=0x7f27b4064b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:04.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.348+0000 7f27bae5d700 1 --2- 172.21.15.29:0/3301177664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b4064740 0x7f27b4064b20 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f27a8006fd0 tx=0x7f27a80090f0).stop 2024-01-24T15:14:04.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.348+0000 7f27bae5d700 1 -- 172.21.15.29:0/3301177664 shutdown_connections 2024-01-24T15:14:04.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.348+0000 7f27bae5d700 1 --2- 172.21.15.29:0/3301177664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b4064740 0x7f27b4064b20 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:04.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.348+0000 7f27bae5d700 1 --2- 172.21.15.29:0/3301177664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27b40c5af0 0x7f27b40c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:04.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.348+0000 7f27bae5d700 1 -- 172.21.15.29:0/3301177664 >> 172.21.15.29:0/3301177664 conn(0x7f27b40cb890 msgr2=0x7f27b40c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:04.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.348+0000 7f27bae5d700 1 -- 172.21.15.29:0/3301177664 shutdown_connections 2024-01-24T15:14:04.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.348+0000 7f27bae5d700 1 -- 172.21.15.29:0/3301177664 wait complete. 2024-01-24T15:14:04.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.349+0000 7f27bae5d700 1 Processor -- start 2024-01-24T15:14:04.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.349+0000 7f27bae5d700 1 -- start start 2024-01-24T15:14:04.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27bae5d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27b4064740 0x7f27b40c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:04.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27bae5d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27b40c5af0 0x7f27b4150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:04.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27bae5d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b4157cb0 0x7f27b40abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:04.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27bae5d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b40c31e0 con 0x7f27b4157cb0 2024-01-24T15:14:04.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27bae5d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b40c2350 con 0x7f27b40c5af0 2024-01-24T15:14:04.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27bae5d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b40c25d0 con 0x7f27b4064740 2024-01-24T15:14:04.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27b93fa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b4157cb0 0x7f27b40abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:04.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27b93fa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b4157cb0 0x7f27b40abd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41974/0 (socket says 172.21.15.29:41974) 2024-01-24T15:14:04.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27b93fa700 1 -- 172.21.15.29:0/3022252207 learned_addr learned my addr 172.21.15.29:0/3022252207 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:04.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27b8bf9700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27b4064740 0x7f27b40c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:04.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27b3fff700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27b40c5af0 0x7f27b4150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:04.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27b93fa700 1 -- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27b4064740 msgr2=0x7f27b40c1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:04.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27b93fa700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27b4064740 0x7f27b40c1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:04.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27b93fa700 1 -- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27b40c5af0 msgr2=0x7f27b4150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:04.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27b93fa700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27b40c5af0 0x7f27b4150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:04.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.350+0000 7f27b93fa700 1 -- 172.21.15.29:0/3022252207 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a8009370 con 0x7f27b4157cb0 2024-01-24T15:14:04.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.351+0000 7f27b93fa700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b4157cb0 0x7f27b40abd90 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f27ac00c7c0 tx=0x7f27ac001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:04.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.351+0000 7f27b0ff9700 1 -- 172.21.15.29:0/3022252207 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00e4a0 con 0x7f27b4157cb0 2024-01-24T15:14:04.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.351+0000 7f27b0ff9700 1 -- 172.21.15.29:0/3022252207 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac00e600 con 0x7f27b4157cb0 2024-01-24T15:14:04.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.351+0000 7f27bae5d700 1 -- 172.21.15.29:0/3022252207 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b40c28b0 con 0x7f27b4157cb0 2024-01-24T15:14:04.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.351+0000 7f27b0ff9700 1 -- 172.21.15.29:0/3022252207 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00e980 con 0x7f27b4157cb0 2024-01-24T15:14:04.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.351+0000 7f27bae5d700 1 -- 172.21.15.29:0/3022252207 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b40c2e70 con 0x7f27b4157cb0 2024-01-24T15:14:04.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.352+0000 7f27bae5d700 1 -- 172.21.15.29:0/3022252207 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b40475e0 con 0x7f27b4157cb0 2024-01-24T15:14:04.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.352+0000 7f27b0ff9700 1 -- 172.21.15.29:0/3022252207 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 73) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f27ac0123f0 con 0x7f27b4157cb0 2024-01-24T15:14:04.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.353+0000 7f27b0ff9700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f279805f030 0x7f27980614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:04.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.353+0000 7f27b0ff9700 1 -- 172.21.15.29:0/3022252207 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 7745+0+0 (secure 0 0 0) 0x7f27ac079c60 con 0x7f27b4157cb0 2024-01-24T15:14:04.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.353+0000 7f27b8bf9700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f279805f030 0x7f27980614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:04.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.353+0000 7f27b8bf9700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f279805f030 0x7f27980614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f27a8009fd0 tx=0x7f27a8017000).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:04.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.358+0000 7f27b0ff9700 1 -- 172.21.15.29:0/3022252207 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f27ac047b00 con 0x7f27b4157cb0 2024-01-24T15:14:04.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:04.586+0000 7f27bae5d700 1 -- 172.21.15.29:0/3022252207 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f27b40c6820 con 0x7f27b4157cb0 2024-01-24T15:14:05.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.186+0000 7f27b0ff9700 1 -- 172.21.15.29:0/3022252207 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7f27b40c6820 con 0x7f27b4157cb0 2024-01-24T15:14:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.188+0000 7f27bae5d700 1 -- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f279805f030 msgr2=0x7f27980614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.188+0000 7f27bae5d700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f279805f030 0x7f27980614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f27a8009fd0 tx=0x7f27a8017000).stop 2024-01-24T15:14:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.188+0000 7f27bae5d700 1 -- 172.21.15.29:0/3022252207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b4157cb0 msgr2=0x7f27b40abd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.188+0000 7f27bae5d700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b4157cb0 0x7f27b40abd90 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f27ac00c7c0 tx=0x7f27ac001510).stop 2024-01-24T15:14:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.188+0000 7f27bae5d700 1 -- 172.21.15.29:0/3022252207 shutdown_connections 2024-01-24T15:14:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.188+0000 7f27bae5d700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f279805f030 0x7f27980614f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.188+0000 7f27bae5d700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27b4157cb0 0x7f27b40abd90 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.188+0000 7f27bae5d700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f27b4064740 0x7f27b40c1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.188+0000 7f27bae5d700 1 --2- 172.21.15.29:0/3022252207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27b40c5af0 0x7f27b4150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:05.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.188+0000 7f27bae5d700 1 -- 172.21.15.29:0/3022252207 >> 172.21.15.29:0/3022252207 conn(0x7f27b40cb890 msgr2=0x7f27b40ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:05.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.189+0000 7f27bae5d700 1 -- 172.21.15.29:0/3022252207 shutdown_connections 2024-01-24T15:14:05.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.189+0000 7f27bae5d700 1 -- 172.21.15.29:0/3022252207 wait complete. 2024-01-24T15:14:05.206 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-24T15:14:05.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.343+0000 7f3c9ed6f700 1 Processor -- start 2024-01-24T15:14:05.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.343+0000 7f3c9ed6f700 1 -- start start 2024-01-24T15:14:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.344+0000 7f3c9ed6f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3c98069970 0x7f3c98067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.344+0000 7f3c9ed6f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c980bd2b0 con 0x7f3c98151eb0 2024-01-24T15:14:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.344+0000 7f3c9ed6f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c98152330 con 0x7f3c980c3d80 2024-01-24T15:14:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.344+0000 7f3c9ed6f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c980bca50 con 0x7f3c98069970 2024-01-24T15:14:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.345+0000 7f3c9d30c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3c98069970 0x7f3c98067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:05.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.345+0000 7f3c97fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3c98151eb0 0x7f3c98069260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53480/0 (socket says 172.21.15.29:53480) 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.345+0000 7f3c97fff700 1 -- 172.21.15.29:0/2668345553 learned_addr learned my addr 172.21.15.29:0/2668345553 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.345+0000 7f3c9d30c700 1 -- 172.21.15.29:0/2668345553 >> v1:172.21.15.163:6789/0 conn(0x7f3c980c3d80 legacy=0x7f3c980c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.345+0000 7f3c9d30c700 1 -- 172.21.15.29:0/2668345553 >> v1:172.21.15.29:6789/0 conn(0x7f3c98151eb0 legacy=0x7f3c98069260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.345+0000 7f3c9d30c700 1 -- 172.21.15.29:0/2668345553 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c98067630 con 0x7f3c98069970 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.345+0000 7f3c9d30c700 1 --2- 172.21.15.29:0/2668345553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c98069970 0x7f3c98067140 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f3c980bcb90 tx=0x7f3c90009780).ready entity=mon.0 client_cookie=5d036e8b2490bcd5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.345+0000 7f3c977fe700 1 -- 172.21.15.29:0/2668345553 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9000e070 con 0x7f3c98069970 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.345+0000 7f3c977fe700 1 -- 172.21.15.29:0/2668345553 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9000bc30 con 0x7f3c98069970 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.345+0000 7f3c977fe700 1 -- 172.21.15.29:0/2668345553 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90012410 con 0x7f3c98069970 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.346+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/2668345553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c98069970 msgr2=0x7f3c98067140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.346+0000 7f3c9ed6f700 1 --2- 172.21.15.29:0/2668345553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c98069970 0x7f3c98067140 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f3c980bcb90 tx=0x7f3c90009780).stop 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.346+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/2668345553 shutdown_connections 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.346+0000 7f3c9ed6f700 1 --2- 172.21.15.29:0/2668345553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c98069970 0x7f3c98067140 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.346+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/2668345553 >> 172.21.15.29:0/2668345553 conn(0x7f3c980c7af0 msgr2=0x7f3c980c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:05.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.346+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/2668345553 shutdown_connections 2024-01-24T15:14:05.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.346+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/2668345553 wait complete. 2024-01-24T15:14:05.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.347+0000 7f3c9ed6f700 1 Processor -- start 2024-01-24T15:14:05.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.347+0000 7f3c9ed6f700 1 -- start start 2024-01-24T15:14:05.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9ed6f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c98069970 0x7f3c98138e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:05.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9ed6f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c980c3d80 0x7f3c980c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:05.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9ed6f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c98151eb0 0x7f3c980c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:05.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9ed6f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980c5100 con 0x7f3c98151eb0 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9ed6f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c98166410 con 0x7f3c98069970 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9ed6f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9814c280 con 0x7f3c980c3d80 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9d30c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c98151eb0 0x7f3c980c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9d30c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c98151eb0 0x7f3c980c4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:41998/0 (socket says 172.21.15.29:41998) 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9d30c700 1 -- 172.21.15.29:0/504626979 learned_addr learned my addr 172.21.15.29:0/504626979 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9cb0b700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c98069970 0x7f3c98138e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c97fff700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c980c3d80 0x7f3c980c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9d30c700 1 -- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c980c3d80 msgr2=0x7f3c980c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9d30c700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c980c3d80 0x7f3c980c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9d30c700 1 -- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c98069970 msgr2=0x7f3c98138e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9d30c700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c98069970 0x7f3c98138e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c9d30c700 1 -- 172.21.15.29:0/504626979 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c90009a20 con 0x7f3c98151eb0 2024-01-24T15:14:05.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.348+0000 7f3c97fff700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c980c3d80 0x7f3c980c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:14:05.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.349+0000 7f3c9d30c700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c98151eb0 0x7f3c980c4c10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f3c90011000 tx=0x7f3c98151360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:05.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.349+0000 7f3c94ff9700 1 -- 172.21.15.29:0/504626979 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90011750 con 0x7f3c98151eb0 2024-01-24T15:14:05.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.349+0000 7f3c94ff9700 1 -- 172.21.15.29:0/504626979 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c900118b0 con 0x7f3c98151eb0 2024-01-24T15:14:05.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.349+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/504626979 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9814eb20 con 0x7f3c98151eb0 2024-01-24T15:14:05.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.349+0000 7f3c94ff9700 1 -- 172.21.15.29:0/504626979 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90011c70 con 0x7f3c98151eb0 2024-01-24T15:14:05.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.349+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/504626979 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9814bc30 con 0x7f3c98151eb0 2024-01-24T15:14:05.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.350+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/504626979 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c980475e0 con 0x7f3c98151eb0 2024-01-24T15:14:05.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.350+0000 7f3c94ff9700 1 -- 172.21.15.29:0/504626979 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 73) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f3c90003040 con 0x7f3c98151eb0 2024-01-24T15:14:05.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.351+0000 7f3c94ff9700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3c7c05f030 0x7f3c7c0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:05.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.351+0000 7f3c94ff9700 1 -- 172.21.15.29:0/504626979 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 7745+0+0 (secure 0 0 0) 0x7f3c9007c5e0 con 0x7f3c98151eb0 2024-01-24T15:14:05.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.351+0000 7f3c9cb0b700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3c7c05f030 0x7f3c7c0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:05.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.352+0000 7f3c9cb0b700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3c7c05f030 0x7f3c7c0614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f3c8c0036d0 tx=0x7f3c8c009040).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:05.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.355+0000 7f3c94ff9700 1 -- 172.21.15.29:0/504626979 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3c9004d420 con 0x7f3c98151eb0 2024-01-24T15:14:05.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:05.535+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/504626979 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3c980678c0 con 0x7f3c98151eb0 2024-01-24T15:14:06.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.183+0000 7f3c94ff9700 1 -- 172.21.15.29:0/504626979 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3c90079f20 con 0x7f3c98151eb0 2024-01-24T15:14:06.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.198+0000 7f3c94ff9700 1 -- 172.21.15.29:0/504626979 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v183) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3c90043950 con 0x7f3c98151eb0 2024-01-24T15:14:06.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.202+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3c7c05f030 msgr2=0x7f3c7c0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:06.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.202+0000 7f3c9ed6f700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3c7c05f030 0x7f3c7c0614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f3c8c0036d0 tx=0x7f3c8c009040).stop 2024-01-24T15:14:06.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.202+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/504626979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c98151eb0 msgr2=0x7f3c980c4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:06.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.202+0000 7f3c9ed6f700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c98151eb0 0x7f3c980c4c10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f3c90011000 tx=0x7f3c98151360).stop 2024-01-24T15:14:06.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.202+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/504626979 shutdown_connections 2024-01-24T15:14:06.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.202+0000 7f3c9ed6f700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3c7c05f030 0x7f3c7c0614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:06.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.202+0000 7f3c9ed6f700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c98151eb0 0x7f3c980c4c10 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:06.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.202+0000 7f3c9ed6f700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c98069970 0x7f3c98138e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:06.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.202+0000 7f3c9ed6f700 1 --2- 172.21.15.29:0/504626979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c980c3d80 0x7f3c980c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:06.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.202+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/504626979 >> 172.21.15.29:0/504626979 conn(0x7f3c980c7af0 msgr2=0x7f3c98062e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:06.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.202+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/504626979 shutdown_connections 2024-01-24T15:14:06.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.202+0000 7f3c9ed6f700 1 -- 172.21.15.29:0/504626979 wait complete. 2024-01-24T15:14:06.204 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-01-24T15:14:06.219 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-24T15:14:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.357+0000 7f7a9d444700 1 Processor -- start 2024-01-24T15:14:06.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.357+0000 7f7a9d444700 1 -- start start 2024-01-24T15:14:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.358+0000 7f7a9d444700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a981560e0 0x7f7a980bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.358+0000 7f7a9d444700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980c38c0 con 0x7f7a98134e80 2024-01-24T15:14:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.358+0000 7f7a9d444700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980c5b40 con 0x7f7a980c5fd0 2024-01-24T15:14:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.358+0000 7f7a9d444700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980ab3a0 con 0x7f7a981560e0 2024-01-24T15:14:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.358+0000 7f7a96ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7a98134e80 0x7f7a980c31b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53488/0 (socket says 172.21.15.29:53488) 2024-01-24T15:14:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.358+0000 7f7a96ffd700 1 -- 172.21.15.29:0/2541995070 learned_addr learned my addr 172.21.15.29:0/2541995070 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.358+0000 7f7a977fe700 1 --2- 172.21.15.29:0/2541995070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a981560e0 0x7f7a980bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.358+0000 7f7a95ffb700 1 -- 172.21.15.29:0/2541995070 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210224738 0 0) 0x7f7a88001110 con 0x7f7a98134e80 2024-01-24T15:14:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.358+0000 7f7a95ffb700 1 -- 172.21.15.29:0/2541995070 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a840033f0 con 0x7f7a98134e80 2024-01-24T15:14:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.359+0000 7f7a95ffb700 1 -- 172.21.15.29:0/2541995070 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585359493 0 0) 0x7f7a980c5b40 con 0x7f7a980c5fd0 2024-01-24T15:14:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.359+0000 7f7a95ffb700 1 -- 172.21.15.29:0/2541995070 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a88001110 con 0x7f7a980c5fd0 2024-01-24T15:14:06.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.359+0000 7f7a95ffb700 1 -- 172.21.15.29:0/2541995070 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 729982680 0 0) 0x7f7a840033f0 con 0x7f7a98134e80 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.359+0000 7f7a977fe700 1 -- 172.21.15.29:0/2541995070 >> v1:172.21.15.163:6789/0 conn(0x7f7a980c5fd0 legacy=0x7f7a981559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.359+0000 7f7a977fe700 1 -- 172.21.15.29:0/2541995070 >> v1:172.21.15.29:6789/0 conn(0x7f7a98134e80 legacy=0x7f7a980c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.359+0000 7f7a977fe700 1 -- 172.21.15.29:0/2541995070 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980c4f30 con 0x7f7a981560e0 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.359+0000 7f7a977fe700 1 --2- 172.21.15.29:0/2541995070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a981560e0 0x7f7a980bdb10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f7a90009e90 tx=0x7f7a90009780).ready entity=mon.2 client_cookie=a1faa7db165c49a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.359+0000 7f7a95ffb700 1 -- 172.21.15.29:0/2541995070 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9000e070 con 0x7f7a981560e0 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.359+0000 7f7a95ffb700 1 -- 172.21.15.29:0/2541995070 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9000b7a0 con 0x7f7a981560e0 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.359+0000 7f7a95ffb700 1 -- 172.21.15.29:0/2541995070 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88001600 con 0x7f7a981560e0 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.360+0000 7f7a9d444700 1 -- 172.21.15.29:0/2541995070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a981560e0 msgr2=0x7f7a980bdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.360+0000 7f7a9d444700 1 --2- 172.21.15.29:0/2541995070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a981560e0 0x7f7a980bdb10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f7a90009e90 tx=0x7f7a90009780).stop 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.360+0000 7f7a9d444700 1 -- 172.21.15.29:0/2541995070 shutdown_connections 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.360+0000 7f7a9d444700 1 --2- 172.21.15.29:0/2541995070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a981560e0 0x7f7a980bdb10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.360+0000 7f7a9d444700 1 -- 172.21.15.29:0/2541995070 >> 172.21.15.29:0/2541995070 conn(0x7f7a9805e1f0 msgr2=0x7f7a9805e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.360+0000 7f7a9d444700 1 -- 172.21.15.29:0/2541995070 shutdown_connections 2024-01-24T15:14:06.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.360+0000 7f7a9d444700 1 -- 172.21.15.29:0/2541995070 wait complete. 2024-01-24T15:14:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.362+0000 7f7a9d444700 1 Processor -- start 2024-01-24T15:14:06.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.362+0000 7f7a9d444700 1 -- start start 2024-01-24T15:14:06.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a9d444700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a980c5fd0 0x7f7a98137e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:06.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a9d444700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a98134e80 0x7f7a980abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:06.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a9d444700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a981560e0 0x7f7a980ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:06.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a9d444700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980c1270 con 0x7f7a981560e0 2024-01-24T15:14:06.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a9d444700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980c0940 con 0x7f7a980c5fd0 2024-01-24T15:14:06.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a9d444700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980c0c20 con 0x7f7a98134e80 2024-01-24T15:14:06.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a977fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a981560e0 0x7f7a980ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:06.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a967fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a98134e80 0x7f7a980abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:06.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a96ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a980c5fd0 0x7f7a98137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:06.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a977fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a981560e0 0x7f7a980ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42002/0 (socket says 172.21.15.29:42002) 2024-01-24T15:14:06.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a977fe700 1 -- 172.21.15.29:0/3952859129 learned_addr learned my addr 172.21.15.29:0/3952859129 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:06.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a977fe700 1 -- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a98134e80 msgr2=0x7f7a980abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:06.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a977fe700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a98134e80 0x7f7a980abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:06.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a977fe700 1 -- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a980c5fd0 msgr2=0x7f7a98137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:06.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a977fe700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a980c5fd0 0x7f7a98137e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:06.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a977fe700 1 -- 172.21.15.29:0/3952859129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90009a20 con 0x7f7a981560e0 2024-01-24T15:14:06.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a967fc700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a98134e80 0x7f7a980abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:06.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.363+0000 7f7a977fe700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a981560e0 0x7f7a980ac050 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f7a900097c0 tx=0x7f7a980c5ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:06.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.364+0000 7f7a8effd700 1 -- 172.21.15.29:0/3952859129 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90012e10 con 0x7f7a981560e0 2024-01-24T15:14:06.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.364+0000 7f7a8effd700 1 -- 172.21.15.29:0/3952859129 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9001e440 con 0x7f7a981560e0 2024-01-24T15:14:06.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.364+0000 7f7a9d444700 1 -- 172.21.15.29:0/3952859129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a980c0f00 con 0x7f7a981560e0 2024-01-24T15:14:06.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.364+0000 7f7a8effd700 1 -- 172.21.15.29:0/3952859129 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9001e7e0 con 0x7f7a981560e0 2024-01-24T15:14:06.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.364+0000 7f7a9d444700 1 -- 172.21.15.29:0/3952859129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a98138380 con 0x7f7a981560e0 2024-01-24T15:14:06.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.365+0000 7f7a9d444700 1 -- 172.21.15.29:0/3952859129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a980475e0 con 0x7f7a981560e0 2024-01-24T15:14:06.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.365+0000 7f7a8effd700 1 -- 172.21.15.29:0/3952859129 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7a90003040 con 0x7f7a981560e0 2024-01-24T15:14:06.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.366+0000 7f7a8effd700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7a7805f270 0x7f7a78061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:06.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.366+0000 7f7a8effd700 1 -- 172.21.15.29:0/3952859129 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 7307+0+0 (secure 0 0 0) 0x7f7a9007bf80 con 0x7f7a981560e0 2024-01-24T15:14:06.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.366+0000 7f7a96ffd700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7a7805f270 0x7f7a78061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:06.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.366+0000 7f7a96ffd700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7a7805f270 0x7f7a78061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7a880010e0 tx=0x7f7a88001450).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:06.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.370+0000 7f7a8effd700 1 -- 172.21.15.29:0/3952859129 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7a900aab00 con 0x7f7a981560e0 2024-01-24T15:14:06.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:06.550+0000 7f7a9d444700 1 -- 172.21.15.29:0/3952859129 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7a980c5210 con 0x7f7a981560e0 2024-01-24T15:14:07.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.200+0000 7f7a8effd700 1 -- 172.21.15.29:0/3952859129 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v184) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7a90023070 con 0x7f7a981560e0 2024-01-24T15:14:07.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.204+0000 7f7a9d444700 1 -- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7a7805f270 msgr2=0x7f7a78061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.204+0000 7f7a9d444700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7a7805f270 0x7f7a78061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7a880010e0 tx=0x7f7a88001450).stop 2024-01-24T15:14:07.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.204+0000 7f7a9d444700 1 -- 172.21.15.29:0/3952859129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a981560e0 msgr2=0x7f7a980ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.204+0000 7f7a9d444700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a981560e0 0x7f7a980ac050 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f7a900097c0 tx=0x7f7a980c5ab0).stop 2024-01-24T15:14:07.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.204+0000 7f7a9d444700 1 -- 172.21.15.29:0/3952859129 shutdown_connections 2024-01-24T15:14:07.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.204+0000 7f7a9d444700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f7a7805f270 0x7f7a78061730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.204+0000 7f7a9d444700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a981560e0 0x7f7a980ac050 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.204+0000 7f7a9d444700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a980c5fd0 0x7f7a98137e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.204+0000 7f7a9d444700 1 --2- 172.21.15.29:0/3952859129 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a98134e80 0x7f7a980abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.204+0000 7f7a9d444700 1 -- 172.21.15.29:0/3952859129 >> 172.21.15.29:0/3952859129 conn(0x7f7a9805e1f0 msgr2=0x7f7a98066bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:07.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.205+0000 7f7a9d444700 1 -- 172.21.15.29:0/3952859129 shutdown_connections 2024-01-24T15:14:07.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.205+0000 7f7a9d444700 1 -- 172.21.15.29:0/3952859129 wait complete. 2024-01-24T15:14:07.207 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-01-24T15:14:07.221 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-24T15:14:07.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.356+0000 7f9f91a1f700 1 Processor -- start 2024-01-24T15:14:07.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.356+0000 7f9f91a1f700 1 -- start start 2024-01-24T15:14:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.357+0000 7f9f91a1f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9f8c064740 0x7f9f8c064b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.357+0000 7f9f91a1f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f8c0c5af0 0x7f9f8c0c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.357+0000 7f9f91a1f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c0c6820 con 0x7f9f8c157cb0 2024-01-24T15:14:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.357+0000 7f9f91a1f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f8c1556b0 con 0x7f9f8c064740 2024-01-24T15:14:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.357+0000 7f9f91a1f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c155220 con 0x7f9f8c0c5af0 2024-01-24T15:14:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.357+0000 7f9f8affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9f8c064740 0x7f9f8c064b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.357+0000 7f9f8affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9f8c064740 0x7f9f8c064b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42004/0 (socket says 172.21.15.29:42004) 2024-01-24T15:14:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.357+0000 7f9f8affd700 1 -- 172.21.15.29:0/1762673562 learned_addr learned my addr 172.21.15.29:0/1762673562 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.357+0000 7f9f8a7fc700 1 --2- 172.21.15.29:0/1762673562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f8c0c5af0 0x7f9f8c0c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f8affd700 1 -- 172.21.15.29:0/1762673562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f8c0c5af0 msgr2=0x7f9f8c0c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:07.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f8affd700 1 --2- 172.21.15.29:0/1762673562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f8c0c5af0 0x7f9f8c0c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f8affd700 1 -- 172.21.15.29:0/1762673562 >> v1:172.21.15.163:6789/0 conn(0x7f9f8c157cb0 legacy=0x7f9f8c158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f8affd700 1 -- 172.21.15.29:0/1762673562 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c154680 con 0x7f9f8c064740 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f8affd700 1 --2- 172.21.15.29:0/1762673562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8c064740 0x7f9f8c064b20 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f9f7c006fd0 tx=0x7f9f7c0090f0).ready entity=mon.0 client_cookie=bfb143f79f085f82 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f89ffb700 1 -- 172.21.15.29:0/1762673562 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c00b580 con 0x7f9f8c064740 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f89ffb700 1 -- 172.21.15.29:0/1762673562 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c00b6e0 con 0x7f9f8c064740 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f89ffb700 1 -- 172.21.15.29:0/1762673562 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c00ba60 con 0x7f9f8c064740 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f91a1f700 1 -- 172.21.15.29:0/1762673562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8c064740 msgr2=0x7f9f8c064b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f91a1f700 1 --2- 172.21.15.29:0/1762673562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8c064740 0x7f9f8c064b20 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f9f7c006fd0 tx=0x7f9f7c0090f0).stop 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f91a1f700 1 -- 172.21.15.29:0/1762673562 shutdown_connections 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f91a1f700 1 --2- 172.21.15.29:0/1762673562 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8c064740 0x7f9f8c064b20 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.358+0000 7f9f91a1f700 1 --2- 172.21.15.29:0/1762673562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f8c0c5af0 0x7f9f8c0c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.359+0000 7f9f91a1f700 1 -- 172.21.15.29:0/1762673562 >> 172.21.15.29:0/1762673562 conn(0x7f9f8c0cb890 msgr2=0x7f9f8c0c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.359+0000 7f9f91a1f700 1 -- 172.21.15.29:0/1762673562 shutdown_connections 2024-01-24T15:14:07.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.359+0000 7f9f91a1f700 1 -- 172.21.15.29:0/1762673562 wait complete. 2024-01-24T15:14:07.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.360+0000 7f9f91a1f700 1 Processor -- start 2024-01-24T15:14:07.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.360+0000 7f9f91a1f700 1 -- start start 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f91a1f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8c064740 0x7f9f8c1689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f91a1f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f8c0c5af0 0x7f9f8c0a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f91a1f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f8c157cb0 0x7f9f8c0aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f91a1f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c168550 con 0x7f9f8c064740 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f91a1f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c167f40 con 0x7f9f8c157cb0 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f91a1f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0b0aa0 con 0x7f9f8c0c5af0 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f8affd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8c064740 0x7f9f8c1689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f8affd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8c064740 0x7f9f8c1689e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42006/0 (socket says 172.21.15.29:42006) 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f8affd700 1 -- 172.21.15.29:0/442478153 learned_addr learned my addr 172.21.15.29:0/442478153 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f8a7fc700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f8c0c5af0 0x7f9f8c0a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f8b7fe700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f8c157cb0 0x7f9f8c0aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f8affd700 1 -- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f8c0c5af0 msgr2=0x7f9f8c0a91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f8affd700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f8c0c5af0 0x7f9f8c0a91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f8affd700 1 -- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f8c157cb0 msgr2=0x7f9f8c0aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f8affd700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f8c157cb0 0x7f9f8c0aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.361+0000 7f9f8affd700 1 -- 172.21.15.29:0/442478153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c009370 con 0x7f9f8c064740 2024-01-24T15:14:07.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.362+0000 7f9f8affd700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8c064740 0x7f9f8c1689e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f9f8c062f00 tx=0x7f9f7c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.362+0000 7f9f82ffd700 1 -- 172.21.15.29:0/442478153 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c000bc0 con 0x7f9f8c064740 2024-01-24T15:14:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.362+0000 7f9f82ffd700 1 -- 172.21.15.29:0/442478153 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c00b850 con 0x7f9f8c064740 2024-01-24T15:14:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.362+0000 7f9f91a1f700 1 -- 172.21.15.29:0/442478153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8c1681e0 con 0x7f9f8c064740 2024-01-24T15:14:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.362+0000 7f9f82ffd700 1 -- 172.21.15.29:0/442478153 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c000d20 con 0x7f9f8c064740 2024-01-24T15:14:07.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.362+0000 7f9f91a1f700 1 -- 172.21.15.29:0/442478153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c0c2880 con 0x7f9f8c064740 2024-01-24T15:14:07.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.363+0000 7f9f91a1f700 1 -- 172.21.15.29:0/442478153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8c0475e0 con 0x7f9f8c064740 2024-01-24T15:14:07.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.363+0000 7f9f82ffd700 1 -- 172.21.15.29:0/442478153 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9f7c01cad0 con 0x7f9f8c064740 2024-01-24T15:14:07.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.364+0000 7f9f82ffd700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6c05f270 0x7f9f6c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:07.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.364+0000 7f9f82ffd700 1 -- 172.21.15.29:0/442478153 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6893+0+0 (secure 0 0 0) 0x7f9f7c07c600 con 0x7f9f8c064740 2024-01-24T15:14:07.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.364+0000 7f9f8a7fc700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6c05f270 0x7f9f6c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:07.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.364+0000 7f9f8a7fc700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6c05f270 0x7f9f6c061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f9f8c168fb0 tx=0x7f9f74009060).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:07.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.368+0000 7f9f82ffd700 1 -- 172.21.15.29:0/442478153 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9f7c04d450 con 0x7f9f8c064740 2024-01-24T15:14:07.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.549+0000 7f9f91a1f700 1 -- 172.21.15.29:0/442478153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9f8c065720 con 0x7f9f8c064740 2024-01-24T15:14:07.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.553+0000 7f9f82ffd700 1 -- 172.21.15.29:0/442478153 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9f7c0290a0 con 0x7f9f8c064740 2024-01-24T15:14:07.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.555+0000 7f9f91a1f700 1 -- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6c05f270 msgr2=0x7f9f6c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.555+0000 7f9f91a1f700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6c05f270 0x7f9f6c061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f9f8c168fb0 tx=0x7f9f74009060).stop 2024-01-24T15:14:07.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.555+0000 7f9f91a1f700 1 -- 172.21.15.29:0/442478153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8c064740 msgr2=0x7f9f8c1689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.555+0000 7f9f91a1f700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8c064740 0x7f9f8c1689e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f9f8c062f00 tx=0x7f9f7c017000).stop 2024-01-24T15:14:07.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.555+0000 7f9f91a1f700 1 -- 172.21.15.29:0/442478153 shutdown_connections 2024-01-24T15:14:07.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.555+0000 7f9f91a1f700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f8c157cb0 0x7f9f8c0aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.555+0000 7f9f91a1f700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f9f6c05f270 0x7f9f6c061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.555+0000 7f9f91a1f700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f8c064740 0x7f9f8c1689e0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.555+0000 7f9f91a1f700 1 --2- 172.21.15.29:0/442478153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f8c0c5af0 0x7f9f8c0a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.555+0000 7f9f91a1f700 1 -- 172.21.15.29:0/442478153 >> 172.21.15.29:0/442478153 conn(0x7f9f8c0cb890 msgr2=0x7f9f8c0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:07.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.555+0000 7f9f91a1f700 1 -- 172.21.15.29:0/442478153 shutdown_connections 2024-01-24T15:14:07.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.555+0000 7f9f91a1f700 1 -- 172.21.15.29:0/442478153 wait complete. 2024-01-24T15:14:07.572 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-01-24T15:14:07.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.702+0000 7f782c792700 1 Processor -- start 2024-01-24T15:14:07.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.702+0000 7f782c792700 1 -- start start 2024-01-24T15:14:07.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.703+0000 7f782c792700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7824157530 0x7f7824068470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:07.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.703+0000 7f782c792700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7824066c30 0x7f78240be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.703+0000 7f782c792700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78240be080 con 0x7f7824068960 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.703+0000 7f782c792700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7824065040 con 0x7f7824066c30 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.703+0000 7f782c792700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7824064140 con 0x7f7824157530 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f7829d2d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7824068960 0x7f7824066520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53500/0 (socket says 172.21.15.29:53500) 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f7829d2d700 1 -- 172.21.15.29:0/2898564309 learned_addr learned my addr 172.21.15.29:0/2898564309 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f782ad2f700 1 --2- 172.21.15.29:0/2898564309 >> v2:172.21.15.163:3300/0 conn(0x7f7824066c30 0x7f78240be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f782a52e700 1 --2- 172.21.15.29:0/2898564309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7824157530 0x7f7824068470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f782952c700 1 -- 172.21.15.29:0/2898564309 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205245568 0 0) 0x7f78240be080 con 0x7f7824068960 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f782952c700 1 -- 172.21.15.29:0/2898564309 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78100033f0 con 0x7f7824068960 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f782a52e700 1 -- 172.21.15.29:0/2898564309 >> v2:172.21.15.163:3300/0 conn(0x7f7824066c30 msgr2=0x7f78240be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f782952c700 1 -- 172.21.15.29:0/2898564309 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3576616237 0 0) 0x7f78100033f0 con 0x7f7824068960 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f782a52e700 1 --2- 172.21.15.29:0/2898564309 >> v2:172.21.15.163:3300/0 conn(0x7f7824066c30 0x7f78240be510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f782a52e700 1 -- 172.21.15.29:0/2898564309 >> v1:172.21.15.29:6789/0 conn(0x7f7824068960 legacy=0x7f7824066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f782a52e700 1 -- 172.21.15.29:0/2898564309 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7824067380 con 0x7f7824157530 2024-01-24T15:14:07.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f782ad2f700 1 --2- 172.21.15.29:0/2898564309 >> v2:172.21.15.163:3300/0 conn(0x7f7824066c30 0x7f78240be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:07.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.704+0000 7f782a52e700 1 --2- 172.21.15.29:0/2898564309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7824157530 0x7f7824068470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f7818009760 tx=0x7f7818009060).ready entity=mon.2 client_cookie=ef5dd392b466e783 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:07.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.705+0000 7f782952c700 1 -- 172.21.15.29:0/2898564309 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820001470 con 0x7f7824157530 2024-01-24T15:14:07.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.705+0000 7f782952c700 1 -- 172.21.15.29:0/2898564309 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7818011400 con 0x7f7824157530 2024-01-24T15:14:07.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.705+0000 7f782952c700 1 -- 172.21.15.29:0/2898564309 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818011780 con 0x7f7824157530 2024-01-24T15:14:07.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.705+0000 7f782c792700 1 -- 172.21.15.29:0/2898564309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7824157530 msgr2=0x7f7824068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.705+0000 7f782c792700 1 --2- 172.21.15.29:0/2898564309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7824157530 0x7f7824068470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f7818009760 tx=0x7f7818009060).stop 2024-01-24T15:14:07.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.705+0000 7f782c792700 1 -- 172.21.15.29:0/2898564309 shutdown_connections 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.705+0000 7f782c792700 1 --2- 172.21.15.29:0/2898564309 >> v2:172.21.15.163:3300/0 conn(0x7f7824066c30 0x7f78240be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.705+0000 7f782c792700 1 --2- 172.21.15.29:0/2898564309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7824157530 0x7f7824068470 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.705+0000 7f782c792700 1 -- 172.21.15.29:0/2898564309 >> 172.21.15.29:0/2898564309 conn(0x7f78240c8ba0 msgr2=0x7f78240c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.705+0000 7f782c792700 1 -- 172.21.15.29:0/2898564309 shutdown_connections 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.705+0000 7f782c792700 1 -- 172.21.15.29:0/2898564309 wait complete. 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.707+0000 7f782c792700 1 Processor -- start 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.707+0000 7f782c792700 1 -- start start 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782c792700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7824066c30 0x7f78240a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782c792700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7824068960 0x7f7824142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782c792700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7824157530 0x7f78240aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782c792700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78240a6a90 con 0x7f7824157530 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782c792700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78240a5e80 con 0x7f7824068960 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782c792700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78240a6160 con 0x7f7824066c30 2024-01-24T15:14:07.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782ad2f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7824157530 0x7f78240aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782ad2f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7824157530 0x7f78240aaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42016/0 (socket says 172.21.15.29:42016) 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782ad2f700 1 -- 172.21.15.29:0/3302196311 learned_addr learned my addr 172.21.15.29:0/3302196311 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782a52e700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7824066c30 0x7f78240a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f7829d2d700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7824068960 0x7f7824142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782ad2f700 1 -- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7824066c30 msgr2=0x7f78240a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782ad2f700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7824066c30 0x7f78240a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.708+0000 7f782ad2f700 1 -- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7824068960 msgr2=0x7f7824142590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.709+0000 7f782ad2f700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7824068960 0x7f7824142590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.709+0000 7f782ad2f700 1 -- 172.21.15.29:0/3302196311 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c009a20 con 0x7f7824157530 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.709+0000 7f782ad2f700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7824157530 0x7f78240aaf00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f781c009f60 tx=0x7f781c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.709+0000 7f78167fc700 1 -- 172.21.15.29:0/3302196311 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c00cb70 con 0x7f7824157530 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.709+0000 7f78167fc700 1 -- 172.21.15.29:0/3302196311 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c00f040 con 0x7f7824157530 2024-01-24T15:14:07.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.709+0000 7f78167fc700 1 -- 172.21.15.29:0/3302196311 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c013400 con 0x7f7824157530 2024-01-24T15:14:07.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.709+0000 7f782c792700 1 -- 172.21.15.29:0/3302196311 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78180092a0 con 0x7f7824157530 2024-01-24T15:14:07.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.709+0000 7f782c792700 1 -- 172.21.15.29:0/3302196311 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78240a6720 con 0x7f7824157530 2024-01-24T15:14:07.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.710+0000 7f782c792700 1 -- 172.21.15.29:0/3302196311 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78240a6c40 con 0x7f7824157530 2024-01-24T15:14:07.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.711+0000 7f78167fc700 1 -- 172.21.15.29:0/3302196311 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f781c0137b0 con 0x7f7824157530 2024-01-24T15:14:07.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.711+0000 7f78167fc700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f780805f270 0x7f7808061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:07.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.711+0000 7f78167fc700 1 -- 172.21.15.29:0/3302196311 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6893+0+0 (secure 0 0 0) 0x7f781c07aec0 con 0x7f7824157530 2024-01-24T15:14:07.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.711+0000 7f782a52e700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f780805f270 0x7f7808061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:07.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.712+0000 7f782a52e700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f780805f270 0x7f7808061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f78240c6200 tx=0x7f7818016000).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:07.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.716+0000 7f78167fc700 1 -- 172.21.15.29:0/3302196311 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f78240a6c40 con 0x7f7824157530 2024-01-24T15:14:07.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.897+0000 7f782c792700 1 -- 172.21.15.29:0/3302196311 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f78240475e0 con 0x7f7824157530 2024-01-24T15:14:07.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.901+0000 7f78167fc700 1 -- 172.21.15.29:0/3302196311 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v33) v1 ==== 120+0+0 (secure 0 0 0) 0x7f781c04bd10 con 0x7f7824157530 2024-01-24T15:14:07.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.903+0000 7f782c792700 1 -- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f780805f270 msgr2=0x7f7808061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.903+0000 7f782c792700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f780805f270 0x7f7808061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f78240c6200 tx=0x7f7818016000).stop 2024-01-24T15:14:07.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.903+0000 7f782c792700 1 -- 172.21.15.29:0/3302196311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7824157530 msgr2=0x7f78240aaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:07.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.903+0000 7f782c792700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7824157530 0x7f78240aaf00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f781c009f60 tx=0x7f781c00b040).stop 2024-01-24T15:14:07.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.903+0000 7f782c792700 1 -- 172.21.15.29:0/3302196311 shutdown_connections 2024-01-24T15:14:07.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.903+0000 7f782c792700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f780805f270 0x7f7808061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.903+0000 7f782c792700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7824157530 0x7f78240aaf00 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.903+0000 7f782c792700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7824066c30 0x7f78240a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.903+0000 7f782c792700 1 --2- 172.21.15.29:0/3302196311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7824068960 0x7f7824142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:07.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.903+0000 7f782c792700 1 -- 172.21.15.29:0/3302196311 >> 172.21.15.29:0/3302196311 conn(0x7f78240c8ba0 msgr2=0x7f78240be510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:07.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.903+0000 7f782c792700 1 -- 172.21.15.29:0/3302196311 shutdown_connections 2024-01-24T15:14:07.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:07.903+0000 7f782c792700 1 -- 172.21.15.29:0/3302196311 wait complete. 2024-01-24T15:14:07.918 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2024-01-24T15:14:08.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.048+0000 7f813cc6d700 1 Processor -- start 2024-01-24T15:14:08.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.048+0000 7f813cc6d700 1 -- start start 2024-01-24T15:14:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.049+0000 7f813cc6d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8138153fe0 0x7f81381543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.049+0000 7f813cc6d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813805e850 con 0x7f81380bd340 2024-01-24T15:14:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.049+0000 7f813cc6d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81380631f0 con 0x7f81380a4880 2024-01-24T15:14:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.049+0000 7f813cc6d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81380bd7e0 con 0x7f8138153fe0 2024-01-24T15:14:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.049+0000 7f8136d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f81380bd340 0x7f81380c1870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53508/0 (socket says 172.21.15.29:53508) 2024-01-24T15:14:08.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.049+0000 7f8136d9d700 1 -- 172.21.15.29:0/1942567385 learned_addr learned my addr 172.21.15.29:0/1942567385 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:08.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.049+0000 7f813659c700 1 --2- 172.21.15.29:0/1942567385 >> v2:172.21.15.163:3300/0 conn(0x7f8138153fe0 0x7f81381543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:08.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.049+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186938134 0 0) 0x7f813805e850 con 0x7f81380bd340 2024-01-24T15:14:08.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.049+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81200033f0 con 0x7f81380bd340 2024-01-24T15:14:08.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726524636 0 0) 0x7f81200033f0 con 0x7f81380bd340 2024-01-24T15:14:08.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 >> v2:172.21.15.163:3300/0 conn(0x7f8138153fe0 msgr2=0x7f81381543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:08.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 --2- 172.21.15.29:0/1942567385 >> v2:172.21.15.163:3300/0 conn(0x7f8138153fe0 0x7f81381543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 >> v1:172.21.15.163:6789/0 conn(0x7f81380a4880 legacy=0x7f81380bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81380628c0 con 0x7f81380bd340 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f81300017f0 con 0x7f81380bd340 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 >> v1:172.21.15.29:6789/0 conn(0x7f81380bd340 legacy=0x7f81380c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 --2- 172.21.15.29:0/1942567385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 0x7f8120003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 --2- 172.21.15.29:0/1942567385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8120004690 0x7f8120004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 --2- 172.21.15.29:0/1942567385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8120005000 0x7f812000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81380bd7e0 con 0x7f81200038e0 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812000beb0 con 0x7f8120005000 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812000c070 con 0x7f8120004690 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813659c700 1 --2- 172.21.15.29:0/1942567385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 0x7f8120003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f8136d9d700 1 --2- 172.21.15.29:0/1942567385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8120005000 0x7f812000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f8135d9b700 1 --2- 172.21.15.29:0/1942567385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8120004690 0x7f8120004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813659c700 1 -- 172.21.15.29:0/1942567385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8120004690 msgr2=0x7f8120004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:08.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813659c700 1 --2- 172.21.15.29:0/1942567385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8120004690 0x7f8120004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813659c700 1 -- 172.21.15.29:0/1942567385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8120005000 msgr2=0x7f812000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813659c700 1 --2- 172.21.15.29:0/1942567385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8120005000 0x7f812000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813659c700 1 -- 172.21.15.29:0/1942567385 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81200033f0 con 0x7f81200038e0 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f8136d9d700 1 -- 172.21.15.29:0/1942567385 reap_dead start 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.050+0000 7f813659c700 1 --2- 172.21.15.29:0/1942567385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 0x7f8120003d00 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7f8128009a80 tx=0x7f8128009060).ready entity=mon.0 client_cookie=e24c2548611a5912 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.051+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128009430 con 0x7f81200038e0 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.051+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812800e640 con 0x7f81200038e0 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.051+0000 7f813559a700 1 -- 172.21.15.29:0/1942567385 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812800e9c0 con 0x7f81200038e0 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.051+0000 7f813cc6d700 1 -- 172.21.15.29:0/1942567385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 msgr2=0x7f8120003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.051+0000 7f813cc6d700 1 --2- 172.21.15.29:0/1942567385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 0x7f8120003d00 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7f8128009a80 tx=0x7f8128009060).stop 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.051+0000 7f813cc6d700 1 -- 172.21.15.29:0/1942567385 shutdown_connections 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.051+0000 7f813cc6d700 1 --2- 172.21.15.29:0/1942567385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 0x7f8120003d00 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.051+0000 7f813cc6d700 1 -- 172.21.15.29:0/1942567385 >> 172.21.15.29:0/1942567385 conn(0x7f8138064aa0 msgr2=0x7f8138064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.051+0000 7f813cc6d700 1 -- 172.21.15.29:0/1942567385 shutdown_connections 2024-01-24T15:14:08.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.051+0000 7f813cc6d700 1 -- 172.21.15.29:0/1942567385 wait complete. 2024-01-24T15:14:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.052+0000 7f813cc6d700 1 Processor -- start 2024-01-24T15:14:08.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.053+0000 7f813cc6d700 1 -- start start 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.053+0000 7f813cc6d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 0x7f8138156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.053+0000 7f813cc6d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81380a6c10 0x7f8138146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.053+0000 7f813cc6d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8138063580 0x7f8138063a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.053+0000 7f813cc6d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8138063ef0 con 0x7f81200038e0 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.053+0000 7f813cc6d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81380bed50 con 0x7f8138063580 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.053+0000 7f813cc6d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81380a7380 con 0x7f81380a6c10 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.053+0000 7f813659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 0x7f8138156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.053+0000 7f813659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 0x7f8138156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42034/0 (socket says 172.21.15.29:42034) 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.053+0000 7f813659c700 1 -- 172.21.15.29:0/2684383877 learned_addr learned my addr 172.21.15.29:0/2684383877 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f8136d9d700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8138063580 0x7f8138063a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f8135d9b700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81380a6c10 0x7f8138146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f813659c700 1 -- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81380a6c10 msgr2=0x7f8138146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:08.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f813659c700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81380a6c10 0x7f8138146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f813659c700 1 -- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8138063580 msgr2=0x7f8138063a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f813659c700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8138063580 0x7f8138063a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f813659c700 1 -- 172.21.15.29:0/2684383877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81380a7660 con 0x7f81200038e0 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f8136d9d700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8138063580 0x7f8138063a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f8135d9b700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81380a6c10 0x7f8138146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f813659c700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 0x7f8138156f00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7f8128006d50 tx=0x7f812801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f81267fc700 1 -- 172.21.15.29:0/2684383877 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128000cb0 con 0x7f81200038e0 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f81267fc700 1 -- 172.21.15.29:0/2684383877 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812800e7b0 con 0x7f81200038e0 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f81267fc700 1 -- 172.21.15.29:0/2684383877 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81280243f0 con 0x7f81200038e0 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f813cc6d700 1 -- 172.21.15.29:0/2684383877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81380a7940 con 0x7f81200038e0 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.054+0000 7f813cc6d700 1 -- 172.21.15.29:0/2684383877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81380a7090 con 0x7f81200038e0 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.055+0000 7f813cc6d700 1 -- 172.21.15.29:0/2684383877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81380bf060 con 0x7f81200038e0 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.056+0000 7f81267fc700 1 -- 172.21.15.29:0/2684383877 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f81280123f0 con 0x7f81200038e0 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.056+0000 7f81267fc700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f811805f270 0x7f8118061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:08.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.056+0000 7f81267fc700 1 -- 172.21.15.29:0/2684383877 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6893+0+0 (secure 0 0 0) 0x7f812807bb70 con 0x7f81200038e0 2024-01-24T15:14:08.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.056+0000 7f8135d9b700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f811805f270 0x7f8118061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:08.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.057+0000 7f8135d9b700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f811805f270 0x7f8118061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f812c000f60 tx=0x7f812c009040).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:08.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.060+0000 7f81267fc700 1 -- 172.21.15.29:0/2684383877 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f812804c960 con 0x7f81200038e0 2024-01-24T15:14:08.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:08.321+0000 7f813cc6d700 1 -- 172.21.15.29:0/2684383877 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f81380475e0 con 0x7f81200038e0 2024-01-24T15:14:09.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.208+0000 7f81267fc700 1 -- 172.21.15.29:0/2684383877 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2455) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8128029020 con 0x7f81200038e0 2024-01-24T15:14:09.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.210+0000 7f813cc6d700 1 -- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f811805f270 msgr2=0x7f8118061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:09.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.210+0000 7f813cc6d700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f811805f270 0x7f8118061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f812c000f60 tx=0x7f812c009040).stop 2024-01-24T15:14:09.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.210+0000 7f813cc6d700 1 -- 172.21.15.29:0/2684383877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 msgr2=0x7f8138156f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:09.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.210+0000 7f813cc6d700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 0x7f8138156f00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7f8128006d50 tx=0x7f812801a000).stop 2024-01-24T15:14:09.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.210+0000 7f813cc6d700 1 -- 172.21.15.29:0/2684383877 shutdown_connections 2024-01-24T15:14:09.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.210+0000 7f813cc6d700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8138063580 0x7f8138063a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:09.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.210+0000 7f813cc6d700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f811805f270 0x7f8118061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:09.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.210+0000 7f813cc6d700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81200038e0 0x7f8138156f00 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:09.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.210+0000 7f813cc6d700 1 --2- 172.21.15.29:0/2684383877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81380a6c10 0x7f8138146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:09.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.210+0000 7f813cc6d700 1 -- 172.21.15.29:0/2684383877 >> 172.21.15.29:0/2684383877 conn(0x7f8138064aa0 msgr2=0x7f81381521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:09.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.210+0000 7f813cc6d700 1 -- 172.21.15.29:0/2684383877 shutdown_connections 2024-01-24T15:14:09.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.210+0000 7f813cc6d700 1 -- 172.21.15.29:0/2684383877 wait complete. 2024-01-24T15:14:09.227 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2024-01-24T15:14:09.228 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2024-01-24T15:14:09.228 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-01-24T15:14:09.228 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2024-01-24T15:14:09.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.385+0000 7f0f99449700 1 Processor -- start 2024-01-24T15:14:09.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.386+0000 7f0f99449700 1 -- start start 2024-01-24T15:14:09.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.386+0000 7f0f99449700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0f94153fe0 0x7f0f941543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:09.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.386+0000 7f0f99449700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0f940bd340 0x7f0f940c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:09.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.386+0000 7f0f99449700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f9405e850 con 0x7f0f940a4880 2024-01-24T15:14:09.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.386+0000 7f0f99449700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f940631f0 con 0x7f0f940bd340 2024-01-24T15:14:09.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.386+0000 7f0f99449700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f940bd7e0 con 0x7f0f94153fe0 2024-01-24T15:14:09.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f937fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0f940bd340 0x7f0f940c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:09.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f927fc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0f940a4880 0x7f0f940bcc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53510/0 (socket says 172.21.15.29:53510) 2024-01-24T15:14:09.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f927fc700 1 -- 172.21.15.29:0/1026285550 learned_addr learned my addr 172.21.15.29:0/1026285550 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:09.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f92ffd700 1 --2- 172.21.15.29:0/1026285550 >> v2:172.21.15.163:3300/0 conn(0x7f0f94153fe0 0x7f0f941543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:09.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f91ffb700 1 -- 172.21.15.29:0/1026285550 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376308869 0 0) 0x7f0f9405e850 con 0x7f0f940a4880 2024-01-24T15:14:09.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f91ffb700 1 -- 172.21.15.29:0/1026285550 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f7c0033f0 con 0x7f0f940a4880 2024-01-24T15:14:09.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f937fe700 1 -- 172.21.15.29:0/1026285550 >> v2:172.21.15.163:3300/0 conn(0x7f0f94153fe0 msgr2=0x7f0f941543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:09.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f937fe700 1 --2- 172.21.15.29:0/1026285550 >> v2:172.21.15.163:3300/0 conn(0x7f0f94153fe0 0x7f0f941543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:09.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f937fe700 1 -- 172.21.15.29:0/1026285550 >> v1:172.21.15.29:6789/0 conn(0x7f0f940a4880 legacy=0x7f0f940bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:09.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f937fe700 1 -- 172.21.15.29:0/1026285550 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f940628c0 con 0x7f0f940bd340 2024-01-24T15:14:09.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f937fe700 1 --2- 172.21.15.29:0/1026285550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f940bd340 0x7f0f940c1870 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f0f940c71d0 tx=0x7f0f8c009780).ready entity=mon.0 client_cookie=b86a01f48da77887 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:09.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f91ffb700 1 -- 172.21.15.29:0/1026285550 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00e070 con 0x7f0f940bd340 2024-01-24T15:14:09.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f91ffb700 1 -- 172.21.15.29:0/1026285550 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c00bab0 con 0x7f0f940bd340 2024-01-24T15:14:09.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.387+0000 7f0f91ffb700 1 -- 172.21.15.29:0/1026285550 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00be30 con 0x7f0f940bd340 2024-01-24T15:14:09.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.388+0000 7f0f99449700 1 -- 172.21.15.29:0/1026285550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f940bd340 msgr2=0x7f0f940c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:09.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.388+0000 7f0f99449700 1 --2- 172.21.15.29:0/1026285550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f940bd340 0x7f0f940c1870 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f0f940c71d0 tx=0x7f0f8c009780).stop 2024-01-24T15:14:09.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.388+0000 7f0f99449700 1 -- 172.21.15.29:0/1026285550 shutdown_connections 2024-01-24T15:14:09.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.388+0000 7f0f99449700 1 --2- 172.21.15.29:0/1026285550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f940bd340 0x7f0f940c1870 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:09.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.388+0000 7f0f99449700 1 --2- 172.21.15.29:0/1026285550 >> v2:172.21.15.163:3300/0 conn(0x7f0f94153fe0 0x7f0f941543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:09.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.388+0000 7f0f99449700 1 -- 172.21.15.29:0/1026285550 >> 172.21.15.29:0/1026285550 conn(0x7f0f94064aa0 msgr2=0x7f0f94064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:09.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.388+0000 7f0f99449700 1 -- 172.21.15.29:0/1026285550 shutdown_connections 2024-01-24T15:14:09.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.388+0000 7f0f99449700 1 -- 172.21.15.29:0/1026285550 wait complete. 2024-01-24T15:14:09.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.390+0000 7f0f99449700 1 Processor -- start 2024-01-24T15:14:09.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.390+0000 7f0f99449700 1 -- start start 2024-01-24T15:14:09.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.391+0000 7f0f99449700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f940a4880 0x7f0f94151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:09.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.391+0000 7f0f99449700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f940bd340 0x7f0f94155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:09.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.391+0000 7f0f99449700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f94153fe0 0x7f0f94155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:09.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.391+0000 7f0f99449700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f941576b0 con 0x7f0f940bd340 2024-01-24T15:14:09.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.391+0000 7f0f99449700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f94156570 con 0x7f0f94153fe0 2024-01-24T15:14:09.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.391+0000 7f0f99449700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f94156850 con 0x7f0f940a4880 2024-01-24T15:14:09.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.391+0000 7f0f927fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f940bd340 0x7f0f94155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.391+0000 7f0f927fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f940bd340 0x7f0f94155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42050/0 (socket says 172.21.15.29:42050) 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.391+0000 7f0f927fc700 1 -- 172.21.15.29:0/773520053 learned_addr learned my addr 172.21.15.29:0/773520053 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.391+0000 7f0f92ffd700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f940a4880 0x7f0f94151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.391+0000 7f0f937fe700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f94153fe0 0x7f0f94155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.392+0000 7f0f927fc700 1 -- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f940a4880 msgr2=0x7f0f94151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.392+0000 7f0f927fc700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f940a4880 0x7f0f94151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.392+0000 7f0f927fc700 1 -- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f94153fe0 msgr2=0x7f0f94155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.392+0000 7f0f927fc700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f94153fe0 0x7f0f94155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.392+0000 7f0f927fc700 1 -- 172.21.15.29:0/773520053 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c009a20 con 0x7f0f940bd340 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.392+0000 7f0f927fc700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f940bd340 0x7f0f94155a00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f0f8800c590 tx=0x7f0f88007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.392+0000 7f0f7affd700 1 -- 172.21.15.29:0/773520053 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880072e0 con 0x7f0f940bd340 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.392+0000 7f0f7affd700 1 -- 172.21.15.29:0/773520053 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f88007440 con 0x7f0f940bd340 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.392+0000 7f0f99449700 1 -- 172.21.15.29:0/773520053 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f94156b30 con 0x7f0f940bd340 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.392+0000 7f0f7affd700 1 -- 172.21.15.29:0/773520053 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880057a0 con 0x7f0f940bd340 2024-01-24T15:14:09.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.392+0000 7f0f99449700 1 -- 172.21.15.29:0/773520053 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f940a7180 con 0x7f0f940bd340 2024-01-24T15:14:09.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.393+0000 7f0f99449700 1 -- 172.21.15.29:0/773520053 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f940475e0 con 0x7f0f940bd340 2024-01-24T15:14:09.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.393+0000 7f0f7affd700 1 -- 172.21.15.29:0/773520053 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0f88005bd0 con 0x7f0f940bd340 2024-01-24T15:14:09.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.394+0000 7f0f7affd700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0f8005f240 0x7f0f80061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:09.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.394+0000 7f0f7affd700 1 -- 172.21.15.29:0/773520053 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6893+0+0 (secure 0 0 0) 0x7f0f88079df0 con 0x7f0f940bd340 2024-01-24T15:14:09.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.394+0000 7f0f92ffd700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0f8005f240 0x7f0f80061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:09.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.395+0000 7f0f92ffd700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0f8005f240 0x7f0f80061700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f0f84009b90 tx=0x7f0f84009060).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:09.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.398+0000 7f0f7affd700 1 -- 172.21.15.29:0/773520053 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0f8804ac40 con 0x7f0f940bd340 2024-01-24T15:14:09.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:09.649+0000 7f0f99449700 1 -- 172.21.15.29:0/773520053 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f0f940a9dd0 con 0x7f0f940bd340 2024-01-24T15:14:09.700 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:14:09.703 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:14:10.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.209+0000 7f0f7affd700 1 -- 172.21.15.29:0/773520053 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2456) v1 ==== 168+0+0 (secure 0 0 0) 0x7f0f8804e260 con 0x7f0f940bd340 2024-01-24T15:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.212+0000 7f0f99449700 1 -- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0f8005f240 msgr2=0x7f0f80061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.212+0000 7f0f99449700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0f8005f240 0x7f0f80061700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f0f84009b90 tx=0x7f0f84009060).stop 2024-01-24T15:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.212+0000 7f0f99449700 1 -- 172.21.15.29:0/773520053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f940bd340 msgr2=0x7f0f94155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.212+0000 7f0f99449700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f940bd340 0x7f0f94155a00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f0f8800c590 tx=0x7f0f88007c80).stop 2024-01-24T15:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.212+0000 7f0f99449700 1 -- 172.21.15.29:0/773520053 shutdown_connections 2024-01-24T15:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.212+0000 7f0f99449700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f94153fe0 0x7f0f94155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.212+0000 7f0f99449700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f940a4880 0x7f0f94151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.212+0000 7f0f99449700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f0f8005f240 0x7f0f80061700 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.212+0000 7f0f99449700 1 --2- 172.21.15.29:0/773520053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f940bd340 0x7f0f94155a00 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.212+0000 7f0f99449700 1 -- 172.21.15.29:0/773520053 >> 172.21.15.29:0/773520053 conn(0x7f0f94064aa0 msgr2=0x7f0f94064e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.212+0000 7f0f99449700 1 -- 172.21.15.29:0/773520053 shutdown_connections 2024-01-24T15:14:10.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.212+0000 7f0f99449700 1 -- 172.21.15.29:0/773520053 wait complete. 2024-01-24T15:14:10.229 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-24T15:14:10.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.364+0000 7fe0f4939700 1 Processor -- start 2024-01-24T15:14:10.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.364+0000 7fe0f4939700 1 -- start start 2024-01-24T15:14:10.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0f4939700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0f0134ea0 0x7fe0f00c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0f4939700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0f00c36c0 0x7fe0f00664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0f4939700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe0f00669b0 0x7fe0f01559f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0f4939700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f0066e60 con 0x7fe0f00c36c0 2024-01-24T15:14:10.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0f4939700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f00686a0 con 0x7fe0f00669b0 2024-01-24T15:14:10.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0f4939700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f0068b30 con 0x7fe0f0134ea0 2024-01-24T15:14:10.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0edd9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0f00c36c0 0x7fe0f00664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0edd9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0f00c36c0 0x7fe0f00664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42054/0 (socket says 172.21.15.29:42054) 2024-01-24T15:14:10.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0edd9b700 1 -- 172.21.15.29:0/2336210528 learned_addr learned my addr 172.21.15.29:0/2336210528 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:10.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0ee59c700 1 --2- 172.21.15.29:0/2336210528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0f0134ea0 0x7fe0f00c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0eed9d700 1 --2- 172.21.15.29:0/2336210528 >> v2:172.21.15.163:3300/0 conn(0x7fe0f00669b0 0x7fe0f01559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0edd9b700 1 -- 172.21.15.29:0/2336210528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0f0134ea0 msgr2=0x7fe0f00c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:10.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0edd9b700 1 --2- 172.21.15.29:0/2336210528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0f0134ea0 0x7fe0f00c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0edd9b700 1 -- 172.21.15.29:0/2336210528 >> v2:172.21.15.163:3300/0 conn(0x7fe0f00669b0 msgr2=0x7fe0f01559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:10.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0edd9b700 1 --2- 172.21.15.29:0/2336210528 >> v2:172.21.15.163:3300/0 conn(0x7fe0f00669b0 0x7fe0f01559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0edd9b700 1 -- 172.21.15.29:0/2336210528 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f0067a90 con 0x7fe0f00c36c0 2024-01-24T15:14:10.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.365+0000 7fe0edd9b700 1 --2- 172.21.15.29:0/2336210528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0f00c36c0 0x7fe0f00664c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7fe0f0150020 tx=0x7fe0e40090f0).ready entity=mon.0 client_cookie=2b2ba22b4e999102 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:10.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.366+0000 7fe0ed59a700 1 -- 172.21.15.29:0/2336210528 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e400f510 con 0x7fe0f00c36c0 2024-01-24T15:14:10.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.366+0000 7fe0ed59a700 1 -- 172.21.15.29:0/2336210528 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e400f670 con 0x7fe0f00c36c0 2024-01-24T15:14:10.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.366+0000 7fe0ed59a700 1 -- 172.21.15.29:0/2336210528 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e400f9f0 con 0x7fe0f00c36c0 2024-01-24T15:14:10.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.366+0000 7fe0f4939700 1 -- 172.21.15.29:0/2336210528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0f00c36c0 msgr2=0x7fe0f00664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:10.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.366+0000 7fe0f4939700 1 --2- 172.21.15.29:0/2336210528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0f00c36c0 0x7fe0f00664c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7fe0f0150020 tx=0x7fe0e40090f0).stop 2024-01-24T15:14:10.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.366+0000 7fe0f4939700 1 -- 172.21.15.29:0/2336210528 shutdown_connections 2024-01-24T15:14:10.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.366+0000 7fe0f4939700 1 --2- 172.21.15.29:0/2336210528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0f0134ea0 0x7fe0f00c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.366+0000 7fe0f4939700 1 --2- 172.21.15.29:0/2336210528 >> v2:172.21.15.163:3300/0 conn(0x7fe0f00669b0 0x7fe0f01559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.366+0000 7fe0f4939700 1 --2- 172.21.15.29:0/2336210528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0f00c36c0 0x7fe0f00664c0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.366+0000 7fe0f4939700 1 -- 172.21.15.29:0/2336210528 >> 172.21.15.29:0/2336210528 conn(0x7fe0f00c8bd0 msgr2=0x7fe0f00c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:10.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.366+0000 7fe0f4939700 1 -- 172.21.15.29:0/2336210528 shutdown_connections 2024-01-24T15:14:10.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.366+0000 7fe0f4939700 1 -- 172.21.15.29:0/2336210528 wait complete. 2024-01-24T15:14:10.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.368+0000 7fe0f4939700 1 Processor -- start 2024-01-24T15:14:10.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.368+0000 7fe0f4939700 1 -- start start 2024-01-24T15:14:10.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.368+0000 7fe0f4939700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0f00669b0 0x7fe0f00accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.368+0000 7fe0f4939700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0f00c36c0 0x7fe0f014a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.368+0000 7fe0f4939700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0f0134ea0 0x7fe0f014cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.368+0000 7fe0f4939700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f00aec60 con 0x7fe0f0134ea0 2024-01-24T15:14:10.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0f4939700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f0064330 con 0x7fe0f00669b0 2024-01-24T15:14:10.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0f4939700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f00ae980 con 0x7fe0f00c36c0 2024-01-24T15:14:10.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0eed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0f0134ea0 0x7fe0f014cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0eed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0f0134ea0 0x7fe0f014cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42056/0 (socket says 172.21.15.29:42056) 2024-01-24T15:14:10.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0eed9d700 1 -- 172.21.15.29:0/362528469 learned_addr learned my addr 172.21.15.29:0/362528469 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:10.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0edd9b700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0f00c36c0 0x7fe0f014a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0ee59c700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0f00669b0 0x7fe0f00accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0eed9d700 1 -- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0f00c36c0 msgr2=0x7fe0f014a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:10.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0eed9d700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0f00c36c0 0x7fe0f014a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0eed9d700 1 -- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0f00669b0 msgr2=0x7fe0f00accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:10.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0eed9d700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0f00669b0 0x7fe0f00accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0eed9d700 1 -- 172.21.15.29:0/362528469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e4009370 con 0x7fe0f0134ea0 2024-01-24T15:14:10.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0eed9d700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0f0134ea0 0x7fe0f014cb30 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7fe0e8009df0 tx=0x7fe0e800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:10.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0de7fc700 1 -- 172.21.15.29:0/362528469 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e800caa0 con 0x7fe0f0134ea0 2024-01-24T15:14:10.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0de7fc700 1 -- 172.21.15.29:0/362528469 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e800f040 con 0x7fe0f0134ea0 2024-01-24T15:14:10.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0f4939700 1 -- 172.21.15.29:0/362528469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f0063ce0 con 0x7fe0f0134ea0 2024-01-24T15:14:10.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0de7fc700 1 -- 172.21.15.29:0/362528469 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e800ce10 con 0x7fe0f0134ea0 2024-01-24T15:14:10.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.369+0000 7fe0f4939700 1 -- 172.21.15.29:0/362528469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f014ae60 con 0x7fe0f0134ea0 2024-01-24T15:14:10.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.371+0000 7fe0f4939700 1 -- 172.21.15.29:0/362528469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f00644a0 con 0x7fe0f0134ea0 2024-01-24T15:14:10.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.371+0000 7fe0de7fc700 1 -- 172.21.15.29:0/362528469 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe0e8013610 con 0x7fe0f0134ea0 2024-01-24T15:14:10.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.371+0000 7fe0de7fc700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe0d005f270 0x7fe0d0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.371+0000 7fe0de7fc700 1 -- 172.21.15.29:0/362528469 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6893+0+0 (secure 0 0 0) 0x7fe0e807ad80 con 0x7fe0f0134ea0 2024-01-24T15:14:10.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.371+0000 7fe0ee59c700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe0d005f270 0x7fe0d0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.372+0000 7fe0ee59c700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe0d005f270 0x7fe0d0061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe0f00ab4f0 tx=0x7fe0e0009060).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:10.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.376+0000 7fe0de7fc700 1 -- 172.21.15.29:0/362528469 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe0e804bbd0 con 0x7fe0f0134ea0 2024-01-24T15:14:10.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.561+0000 7fe0f4939700 1 -- 172.21.15.29:0/362528469 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe0f00475e0 con 0x7fe0f0134ea0 2024-01-24T15:14:10.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.565+0000 7fe0de7fc700 1 -- 172.21.15.29:0/362528469 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe0e804f1f0 con 0x7fe0f0134ea0 2024-01-24T15:14:10.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.567+0000 7fe0f4939700 1 -- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe0d005f270 msgr2=0x7fe0d0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:10.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.567+0000 7fe0f4939700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe0d005f270 0x7fe0d0061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe0f00ab4f0 tx=0x7fe0e0009060).stop 2024-01-24T15:14:10.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.567+0000 7fe0f4939700 1 -- 172.21.15.29:0/362528469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0f0134ea0 msgr2=0x7fe0f014cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:10.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.567+0000 7fe0f4939700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0f0134ea0 0x7fe0f014cb30 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7fe0e8009df0 tx=0x7fe0e800b040).stop 2024-01-24T15:14:10.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.567+0000 7fe0f4939700 1 -- 172.21.15.29:0/362528469 shutdown_connections 2024-01-24T15:14:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.567+0000 7fe0f4939700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fe0d005f270 0x7fe0d0061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.567+0000 7fe0f4939700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0f0134ea0 0x7fe0f014cb30 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.567+0000 7fe0f4939700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0f00669b0 0x7fe0f00accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.567+0000 7fe0f4939700 1 --2- 172.21.15.29:0/362528469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0f00c36c0 0x7fe0f014a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.567+0000 7fe0f4939700 1 -- 172.21.15.29:0/362528469 >> 172.21.15.29:0/362528469 conn(0x7fe0f00c8bd0 msgr2=0x7fe0f00c4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.567+0000 7fe0f4939700 1 -- 172.21.15.29:0/362528469 shutdown_connections 2024-01-24T15:14:10.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.567+0000 7fe0f4939700 1 -- 172.21.15.29:0/362528469 wait complete. 2024-01-24T15:14:10.584 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T15:14:10.584 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:14:10.636 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:14:10.682 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T15:14:10.682 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-24T15:14:10.682 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:14:10.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.814+0000 7fb139bc8700 1 Processor -- start 2024-01-24T15:14:10.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.814+0000 7fb139bc8700 1 -- start start 2024-01-24T15:14:10.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb139bc8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb134066c00 0x7fb1340bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb139bc8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb1340670a0 0x7fb134158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb139bc8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1341582b0 con 0x7fb1340c4a10 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb139bc8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb134157ca0 con 0x7fb134066c00 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb139bc8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb134063c60 con 0x7fb1340670a0 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb132ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb134066c00 0x7fb1340bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb1337fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb1340c4a10 0x7fb1340664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53512/0 (socket says 172.21.15.29:53512) 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb1337fe700 1 -- 172.21.15.29:0/3976062442 learned_addr learned my addr 172.21.15.29:0/3976062442 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb133fff700 1 --2- 172.21.15.29:0/3976062442 >> v2:172.21.15.163:3300/0 conn(0x7fb1340670a0 0x7fb134158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb132ffd700 1 -- 172.21.15.29:0/3976062442 >> v2:172.21.15.163:3300/0 conn(0x7fb1340670a0 msgr2=0x7fb134158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb132ffd700 1 --2- 172.21.15.29:0/3976062442 >> v2:172.21.15.163:3300/0 conn(0x7fb1340670a0 0x7fb134158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb132ffd700 1 -- 172.21.15.29:0/3976062442 >> v1:172.21.15.29:6789/0 conn(0x7fb1340c4a10 legacy=0x7fb1340664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb132ffd700 1 -- 172.21.15.29:0/3976062442 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134157f40 con 0x7fb134066c00 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb1327fc700 1 -- 172.21.15.29:0/3976062442 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068015701 0 0) 0x7fb1341582b0 con 0x7fb1340c4a10 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb132ffd700 1 --2- 172.21.15.29:0/3976062442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb134066c00 0x7fb1340bf6d0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7fb134065a00 tx=0x7fb128009060).ready entity=mon.0 client_cookie=6f0f64ee7b1af29b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:10.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb1327fc700 1 -- 172.21.15.29:0/3976062442 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12800b580 con 0x7fb134066c00 2024-01-24T15:14:10.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb1327fc700 1 -- 172.21.15.29:0/3976062442 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12800b6e0 con 0x7fb134066c00 2024-01-24T15:14:10.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.815+0000 7fb1327fc700 1 -- 172.21.15.29:0/3976062442 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12800ba60 con 0x7fb134066c00 2024-01-24T15:14:10.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.816+0000 7fb139bc8700 1 -- 172.21.15.29:0/3976062442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb134066c00 msgr2=0x7fb1340bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:10.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.816+0000 7fb139bc8700 1 --2- 172.21.15.29:0/3976062442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb134066c00 0x7fb1340bf6d0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7fb134065a00 tx=0x7fb128009060).stop 2024-01-24T15:14:10.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.816+0000 7fb139bc8700 1 -- 172.21.15.29:0/3976062442 shutdown_connections 2024-01-24T15:14:10.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.816+0000 7fb139bc8700 1 --2- 172.21.15.29:0/3976062442 >> v2:172.21.15.163:3300/0 conn(0x7fb1340670a0 0x7fb134158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.816+0000 7fb139bc8700 1 --2- 172.21.15.29:0/3976062442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb134066c00 0x7fb1340bf6d0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.816+0000 7fb139bc8700 1 -- 172.21.15.29:0/3976062442 >> 172.21.15.29:0/3976062442 conn(0x7fb1340c8bb0 msgr2=0x7fb1340c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:10.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.816+0000 7fb139bc8700 1 -- 172.21.15.29:0/3976062442 shutdown_connections 2024-01-24T15:14:10.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.816+0000 7fb139bc8700 1 -- 172.21.15.29:0/3976062442 wait complete. 2024-01-24T15:14:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.818+0000 7fb139bc8700 1 Processor -- start 2024-01-24T15:14:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.818+0000 7fb139bc8700 1 -- start start 2024-01-24T15:14:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.818+0000 7fb139bc8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb134066c00 0x7fb1340a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.818+0000 7fb139bc8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1340670a0 0x7fb1340a6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.818+0000 7fb139bc8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340c4a10 0x7fb1340adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.818+0000 7fb139bc8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134141630 con 0x7fb1340670a0 2024-01-24T15:14:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.818+0000 7fb139bc8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340a7080 con 0x7fb134066c00 2024-01-24T15:14:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.818+0000 7fb139bc8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340a7360 con 0x7fb1340c4a10 2024-01-24T15:14:10.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb132ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1340670a0 0x7fb1340a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb132ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1340670a0 0x7fb1340a6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42070/0 (socket says 172.21.15.29:42070) 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb132ffd700 1 -- 172.21.15.29:0/3291146102 learned_addr learned my addr 172.21.15.29:0/3291146102 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb1337fe700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb134066c00 0x7fb1340a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb133fff700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340c4a10 0x7fb1340adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb132ffd700 1 -- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340c4a10 msgr2=0x7fb1340adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb132ffd700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340c4a10 0x7fb1340adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb132ffd700 1 -- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb134066c00 msgr2=0x7fb1340a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb132ffd700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb134066c00 0x7fb1340a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb132ffd700 1 -- 172.21.15.29:0/3291146102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1280092a0 con 0x7fb1340670a0 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb132ffd700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1340670a0 0x7fb1340a6610 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7fb128009fd0 tx=0x7fb128017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb11b7fe700 1 -- 172.21.15.29:0/3291146102 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128000bc0 con 0x7fb1340670a0 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb11b7fe700 1 -- 172.21.15.29:0/3291146102 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12800b850 con 0x7fb1340670a0 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb139bc8700 1 -- 172.21.15.29:0/3291146102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1340a7640 con 0x7fb1340670a0 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb11b7fe700 1 -- 172.21.15.29:0/3291146102 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128000d20 con 0x7fb1340670a0 2024-01-24T15:14:10.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.819+0000 7fb139bc8700 1 -- 172.21.15.29:0/3291146102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1341529d0 con 0x7fb1340670a0 2024-01-24T15:14:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.820+0000 7fb139bc8700 1 -- 172.21.15.29:0/3291146102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1341417e0 con 0x7fb1340670a0 2024-01-24T15:14:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.821+0000 7fb11b7fe700 1 -- 172.21.15.29:0/3291146102 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb12801b8f0 con 0x7fb1340670a0 2024-01-24T15:14:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.821+0000 7fb11b7fe700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb12005f240 0x7fb120061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.821+0000 7fb11b7fe700 1 -- 172.21.15.29:0/3291146102 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6893+0+0 (secure 0 0 0) 0x7fb12807c0e0 con 0x7fb1340670a0 2024-01-24T15:14:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.822+0000 7fb1337fe700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb12005f240 0x7fb120061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:10.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.822+0000 7fb1337fe700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb12005f240 0x7fb120061700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fb124007fd0 tx=0x7fb124007dd0).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:10.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:10.826+0000 7fb11b7fe700 1 -- 172.21.15.29:0/3291146102 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb1341417e0 con 0x7fb1340670a0 2024-01-24T15:14:11.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.054+0000 7fb139bc8700 1 -- 172.21.15.29:0/3291146102 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1340475e0 con 0x7fb1340670a0 2024-01-24T15:14:11.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.055+0000 7fb11b7fe700 1 -- 172.21.15.29:0/3291146102 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fb12801f090 con 0x7fb1340670a0 2024-01-24T15:14:11.057 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:11.058 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7319,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3000519646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3000519646},{"type":"v1","addr":"172.21.15.163:6836","nonce":3000519646}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3497543927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3497543927},{"type":"v1","addr":"172.21.15.29:6835","nonce":3497543927}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2353694052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2353694052},{"type":"v1","addr":"172.21.15.163:6837","nonce":2353694052}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/3279168110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3279168110},{"type":"v1","addr":"172.21.15.29:6834","nonce":3279168110}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[]} 2024-01-24T15:14:11.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.057+0000 7fb139bc8700 1 -- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb12005f240 msgr2=0x7fb120061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:11.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.057+0000 7fb139bc8700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb12005f240 0x7fb120061700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fb124007fd0 tx=0x7fb124007dd0).stop 2024-01-24T15:14:11.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.057+0000 7fb139bc8700 1 -- 172.21.15.29:0/3291146102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1340670a0 msgr2=0x7fb1340a6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:11.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.057+0000 7fb139bc8700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1340670a0 0x7fb1340a6610 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7fb128009fd0 tx=0x7fb128017000).stop 2024-01-24T15:14:11.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.057+0000 7fb139bc8700 1 -- 172.21.15.29:0/3291146102 shutdown_connections 2024-01-24T15:14:11.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.057+0000 7fb139bc8700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340c4a10 0x7fb1340adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:11.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.057+0000 7fb139bc8700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb134066c00 0x7fb1340a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:11.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.057+0000 7fb139bc8700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb12005f240 0x7fb120061700 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:11.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.057+0000 7fb139bc8700 1 --2- 172.21.15.29:0/3291146102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1340670a0 0x7fb1340a6610 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:11.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.057+0000 7fb139bc8700 1 -- 172.21.15.29:0/3291146102 >> 172.21.15.29:0/3291146102 conn(0x7fb1340c8bb0 msgr2=0x7fb1340c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:11.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.057+0000 7fb139bc8700 1 -- 172.21.15.29:0/3291146102 shutdown_connections 2024-01-24T15:14:11.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.057+0000 7fb139bc8700 1 -- 172.21.15.29:0/3291146102 wait complete. 2024-01-24T15:14:11.061 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 76 2024-01-24T15:14:11.073 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-24T15:14:11.073 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-24T15:14:11.073 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-24T15:14:11.073 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:14:11.073 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-24T15:14:11.073 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-24T15:14:11.074 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-24T15:14:11.074 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:14:11.074 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-24T15:14:11.074 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-24T15:14:11.074 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-24T15:14:11.074 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:14:11.074 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-24T15:14:11.074 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-24T15:14:11.075 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-24T15:14:11.075 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:14:11.082 INFO:tasks.ceph.mds.a:Stopped 2024-01-24T15:14:11.083 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-24T15:14:11.083 INFO:tasks.ceph.mds.c:Stopped 2024-01-24T15:14:11.084 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-24T15:14:11.085 INFO:tasks.ceph.mds.b:Stopped 2024-01-24T15:14:11.085 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-24T15:14:11.086 INFO:tasks.ceph.mds.d:Stopped 2024-01-24T15:14:11.086 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-24T15:14:11.125 INFO:tasks.ceph.mds.a:Started 2024-01-24T15:14:11.126 DEBUG:teuthology.parallel:result is None 2024-01-24T15:14:11.127 INFO:tasks.ceph.mds.c:Started 2024-01-24T15:14:11.127 DEBUG:teuthology.parallel:result is None 2024-01-24T15:14:11.128 INFO:tasks.ceph.mds.b:Started 2024-01-24T15:14:11.128 DEBUG:teuthology.parallel:result is None 2024-01-24T15:14:11.129 INFO:tasks.ceph.mds.d:Started 2024-01-24T15:14:11.129 DEBUG:teuthology.parallel:result is None 2024-01-24T15:14:11.129 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-24T15:14:11.204 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T15:14:11.204 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-01-24T15:14:11.215 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T15:14:11.217 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T15:14:11.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.269+0000 7ff3b6590700 1 Processor -- start 2024-01-24T15:14:11.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.270+0000 7ff3b6590700 1 -- start start 2024-01-24T15:14:11.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.270+0000 7ff3b6590700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff3b0062ec0 0x7ff3b00aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:11.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.270+0000 7ff3b6590700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3b00c4c60 0x7ff3b00bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:11.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.270+0000 7ff3b6590700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b0069c10 con 0x7ff3b00a9c40 2024-01-24T15:14:11.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.270+0000 7ff3b6590700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b006a280 con 0x7ff3b0062ec0 2024-01-24T15:14:11.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.270+0000 7ff3b6590700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00be540 con 0x7ff3b00c4c60 2024-01-24T15:14:11.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3affff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff3b00a9c40 0x7ff3b00627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53530/0 (socket says 172.21.15.29:53530) 2024-01-24T15:14:11.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3affff700 1 -- 172.21.15.29:0/275836389 learned_addr learned my addr 172.21.15.29:0/275836389 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:11.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3af7fe700 1 --2- 172.21.15.29:0/275836389 >> v2:172.21.15.29:3300/0 conn(0x7ff3b0062ec0 0x7ff3b00aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:11.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3b4b2d700 1 --2- 172.21.15.29:0/275836389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3b00c4c60 0x7ff3b00bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:11.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3aeffd700 1 -- 172.21.15.29:0/275836389 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494210679 0 0) 0x7ff3b0069c10 con 0x7ff3b00a9c40 2024-01-24T15:14:11.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3aeffd700 1 -- 172.21.15.29:0/275836389 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3980033f0 con 0x7ff3b00a9c40 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3af7fe700 1 -- 172.21.15.29:0/275836389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3b00c4c60 msgr2=0x7ff3b00bca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3af7fe700 1 --2- 172.21.15.29:0/275836389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3b00c4c60 0x7ff3b00bca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3af7fe700 1 -- 172.21.15.29:0/275836389 >> v1:172.21.15.29:6789/0 conn(0x7ff3b00a9c40 legacy=0x7ff3b00627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3af7fe700 1 -- 172.21.15.29:0/275836389 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b00be820 con 0x7ff3b0062ec0 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3af7fe700 1 --2- 172.21.15.29:0/275836389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3b0062ec0 0x7ff3b00aa020 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7ff3a4009760 tx=0x7ff3a4009060).ready entity=mon.0 client_cookie=56cb6fac30f4ecfe server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3aeffd700 1 -- 172.21.15.29:0/275836389 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400d4a0 con 0x7ff3b0062ec0 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3aeffd700 1 -- 172.21.15.29:0/275836389 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a400d600 con 0x7ff3b0062ec0 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.271+0000 7ff3aeffd700 1 -- 172.21.15.29:0/275836389 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400d980 con 0x7ff3b0062ec0 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.272+0000 7ff3b6590700 1 -- 172.21.15.29:0/275836389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3b0062ec0 msgr2=0x7ff3b00aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.272+0000 7ff3b6590700 1 --2- 172.21.15.29:0/275836389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3b0062ec0 0x7ff3b00aa020 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7ff3a4009760 tx=0x7ff3a4009060).stop 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.272+0000 7ff3b6590700 1 -- 172.21.15.29:0/275836389 shutdown_connections 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.272+0000 7ff3b6590700 1 --2- 172.21.15.29:0/275836389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3b00c4c60 0x7ff3b00bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:11.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.272+0000 7ff3b6590700 1 --2- 172.21.15.29:0/275836389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3b0062ec0 0x7ff3b00aa020 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.272+0000 7ff3b6590700 1 -- 172.21.15.29:0/275836389 >> 172.21.15.29:0/275836389 conn(0x7ff3b00c8580 msgr2=0x7ff3b00c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.272+0000 7ff3b6590700 1 -- 172.21.15.29:0/275836389 shutdown_connections 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.272+0000 7ff3b6590700 1 -- 172.21.15.29:0/275836389 wait complete. 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.274+0000 7ff3b6590700 1 Processor -- start 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.274+0000 7ff3b6590700 1 -- start start 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b6590700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3b0062ec0 0x7ff3b00ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b6590700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3b00a9c40 0x7ff3b00c0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b6590700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3b00c4c60 0x7ff3b00c1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b6590700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00c1680 con 0x7ff3b00c4c60 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b6590700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00c2c80 con 0x7ff3b0062ec0 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b6590700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00c20b0 con 0x7ff3b00a9c40 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b4b2d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3b00c4c60 0x7ff3b00c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b4b2d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3b00c4c60 0x7ff3b00c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42104/0 (socket says 172.21.15.29:42104) 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b4b2d700 1 -- 172.21.15.29:0/155153291 learned_addr learned my addr 172.21.15.29:0/155153291 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3af7fe700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3b00a9c40 0x7ff3b00c0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:11.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3affff700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3b0062ec0 0x7ff3b00ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b4b2d700 1 -- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3b00a9c40 msgr2=0x7ff3b00c0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b4b2d700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3b00a9c40 0x7ff3b00c0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b4b2d700 1 -- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3b0062ec0 msgr2=0x7ff3b00ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b4b2d700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3b0062ec0 0x7ff3b00ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.275+0000 7ff3b4b2d700 1 -- 172.21.15.29:0/155153291 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40092a0 con 0x7ff3b00c4c60 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.276+0000 7ff3b4b2d700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3b00c4c60 0x7ff3b00c1190 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7ff3a8009e90 tx=0x7ff3a800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.276+0000 7ff397fff700 1 -- 172.21.15.29:0/155153291 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a800caa0 con 0x7ff3b00c4c60 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.276+0000 7ff397fff700 1 -- 172.21.15.29:0/155153291 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a800f040 con 0x7ff3b00c4c60 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.276+0000 7ff3b6590700 1 -- 172.21.15.29:0/155153291 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b00c2350 con 0x7ff3b00c4c60 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.276+0000 7ff397fff700 1 -- 172.21.15.29:0/155153291 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a800ce10 con 0x7ff3b00c4c60 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.276+0000 7ff3b6590700 1 -- 172.21.15.29:0/155153291 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b00c2910 con 0x7ff3b00c4c60 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.277+0000 7ff3b6590700 1 -- 172.21.15.29:0/155153291 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b00c2df0 con 0x7ff3b00c4c60 2024-01-24T15:14:11.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.277+0000 7ff397fff700 1 -- 172.21.15.29:0/155153291 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff3a8013610 con 0x7ff3b00c4c60 2024-01-24T15:14:11.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.278+0000 7ff397fff700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff39c05f2a0 0x7ff39c061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:11.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.278+0000 7ff397fff700 1 -- 172.21.15.29:0/155153291 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6893+0+0 (secure 0 0 0) 0x7ff3a807aea0 con 0x7ff3b00c4c60 2024-01-24T15:14:11.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.278+0000 7ff3affff700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff39c05f2a0 0x7ff39c061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:11.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.279+0000 7ff3affff700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff39c05f2a0 0x7ff39c061760 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ff3a0005750 tx=0x7ff3a0009040).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:11.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.283+0000 7ff397fff700 1 -- 172.21.15.29:0/155153291 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff3a804bcf0 con 0x7ff3b00c4c60 2024-01-24T15:14:11.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:11.481+0000 7ff3b6590700 1 -- 172.21.15.29:0/155153291 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff3b00475e0 con 0x7ff3b00c4c60 2024-01-24T15:14:12.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.240+0000 7ff397fff700 1 -- 172.21.15.29:0/155153291 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v189) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff3a804f310 con 0x7ff3b00c4c60 2024-01-24T15:14:12.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.242+0000 7ff3b6590700 1 -- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff39c05f2a0 msgr2=0x7ff39c061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.242+0000 7ff3b6590700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff39c05f2a0 0x7ff39c061760 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ff3a0005750 tx=0x7ff3a0009040).stop 2024-01-24T15:14:12.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.242+0000 7ff3b6590700 1 -- 172.21.15.29:0/155153291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3b00c4c60 msgr2=0x7ff3b00c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.242+0000 7ff3b6590700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3b00c4c60 0x7ff3b00c1190 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7ff3a8009e90 tx=0x7ff3a800b040).stop 2024-01-24T15:14:12.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.242+0000 7ff3b6590700 1 -- 172.21.15.29:0/155153291 shutdown_connections 2024-01-24T15:14:12.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.242+0000 7ff3b6590700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff39c05f2a0 0x7ff39c061760 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.242+0000 7ff3b6590700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3b00c4c60 0x7ff3b00c1190 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.242+0000 7ff3b6590700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3b0062ec0 0x7ff3b00ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.242+0000 7ff3b6590700 1 --2- 172.21.15.29:0/155153291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3b00a9c40 0x7ff3b00c0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.242+0000 7ff3b6590700 1 -- 172.21.15.29:0/155153291 >> 172.21.15.29:0/155153291 conn(0x7ff3b00c8580 msgr2=0x7ff3b013a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:12.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.243+0000 7ff3b6590700 1 -- 172.21.15.29:0/155153291 shutdown_connections 2024-01-24T15:14:12.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.243+0000 7ff3b6590700 1 -- 172.21.15.29:0/155153291 wait complete. 2024-01-24T15:14:12.246 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-01-24T15:14:12.259 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-24T15:14:12.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.392+0000 7f085a9dc700 1 Processor -- start 2024-01-24T15:14:12.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.392+0000 7f085a9dc700 1 -- start start 2024-01-24T15:14:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.393+0000 7f085a9dc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f08540690b0 0x7f08540627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:12.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.393+0000 7f085a9dc700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08540639a0 con 0x7f085414c4b0 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.393+0000 7f085a9dc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0854062d90 con 0x7f085414c890 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.393+0000 7f085a9dc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0854063070 con 0x7f08540690b0 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.393+0000 7f0853fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f085414c4b0 0x7f0854153060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53532/0 (socket says 172.21.15.29:53532) 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.393+0000 7f0853fff700 1 -- 172.21.15.29:0/2219347496 learned_addr learned my addr 172.21.15.29:0/2219347496 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0858f79700 1 --2- 172.21.15.29:0/2219347496 >> v2:172.21.15.163:3300/0 conn(0x7f08540690b0 0x7f08540627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0852ffd700 1 -- 172.21.15.29:0/2219347496 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265206994 0 0) 0x7f08540639a0 con 0x7f085414c4b0 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0852ffd700 1 -- 172.21.15.29:0/2219347496 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f083c0033f0 con 0x7f085414c4b0 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0852ffd700 1 -- 172.21.15.29:0/2219347496 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811351233 0 0) 0x7f0854062d90 con 0x7f085414c890 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0852ffd700 1 -- 172.21.15.29:0/2219347496 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08540639a0 con 0x7f085414c890 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0852ffd700 1 -- 172.21.15.29:0/2219347496 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4103876404 0 0) 0x7f083c0033f0 con 0x7f085414c4b0 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0858f79700 1 -- 172.21.15.29:0/2219347496 >> v1:172.21.15.163:6789/0 conn(0x7f085414c890 legacy=0x7f08540689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0858f79700 1 -- 172.21.15.29:0/2219347496 >> v1:172.21.15.29:6789/0 conn(0x7f085414c4b0 legacy=0x7f0854153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0858f79700 1 -- 172.21.15.29:0/2219347496 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0854063350 con 0x7f08540690b0 2024-01-24T15:14:12.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0858f79700 1 --2- 172.21.15.29:0/2219347496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08540690b0 0x7f08540627b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f084c009e60 tx=0x7f084c009780).ready entity=mon.1 client_cookie=940a044eeb1c0f3b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0852ffd700 1 -- 172.21.15.29:0/2219347496 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c00e070 con 0x7f08540690b0 2024-01-24T15:14:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0852ffd700 1 -- 172.21.15.29:0/2219347496 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c00b7a0 con 0x7f08540690b0 2024-01-24T15:14:12.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.394+0000 7f0852ffd700 1 -- 172.21.15.29:0/2219347496 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0844001470 con 0x7f08540690b0 2024-01-24T15:14:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.395+0000 7f085a9dc700 1 -- 172.21.15.29:0/2219347496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08540690b0 msgr2=0x7f08540627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.395+0000 7f085a9dc700 1 --2- 172.21.15.29:0/2219347496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08540690b0 0x7f08540627b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f084c009e60 tx=0x7f084c009780).stop 2024-01-24T15:14:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.395+0000 7f085a9dc700 1 -- 172.21.15.29:0/2219347496 shutdown_connections 2024-01-24T15:14:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.395+0000 7f085a9dc700 1 --2- 172.21.15.29:0/2219347496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08540690b0 0x7f08540627b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.395+0000 7f085a9dc700 1 -- 172.21.15.29:0/2219347496 >> 172.21.15.29:0/2219347496 conn(0x7f0854064aa0 msgr2=0x7f0854064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.395+0000 7f085a9dc700 1 -- 172.21.15.29:0/2219347496 shutdown_connections 2024-01-24T15:14:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.395+0000 7f085a9dc700 1 -- 172.21.15.29:0/2219347496 wait complete. 2024-01-24T15:14:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.396+0000 7f085a9dc700 1 Processor -- start 2024-01-24T15:14:12.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.397+0000 7f085a9dc700 1 -- start start 2024-01-24T15:14:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.397+0000 7f085a9dc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08540690b0 0x7f0854133f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.397+0000 7f085a9dc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f085414c4b0 0x7f08540c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:12.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.397+0000 7f085a9dc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f085414c890 0x7f08540c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.397+0000 7f085a9dc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08540c5790 con 0x7f08540690b0 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.397+0000 7f085a9dc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0854158e90 con 0x7f085414c4b0 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.397+0000 7f085a9dc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0854159170 con 0x7f085414c890 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.397+0000 7f0853fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08540690b0 0x7f0854133f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f0853fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08540690b0 0x7f0854133f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42112/0 (socket says 172.21.15.29:42112) 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f0853fff700 1 -- 172.21.15.29:0/3148558223 learned_addr learned my addr 172.21.15.29:0/3148558223 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f0858f79700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f085414c890 0x7f08540c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f08537fe700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f085414c4b0 0x7f08540c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f0853fff700 1 -- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f085414c890 msgr2=0x7f08540c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f0853fff700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f085414c890 0x7f08540c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f0853fff700 1 -- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f085414c4b0 msgr2=0x7f08540c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f0853fff700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f085414c4b0 0x7f08540c4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f0853fff700 1 -- 172.21.15.29:0/3148558223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c009a20 con 0x7f08540690b0 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f0853fff700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08540690b0 0x7f0854133f60 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f084400c800 tx=0x7f0844001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:12.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f083bfff700 1 -- 172.21.15.29:0/3148558223 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0844007170 con 0x7f08540690b0 2024-01-24T15:14:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f083bfff700 1 -- 172.21.15.29:0/3148558223 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08440072d0 con 0x7f08540690b0 2024-01-24T15:14:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f083bfff700 1 -- 172.21.15.29:0/3148558223 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08440048c0 con 0x7f08540690b0 2024-01-24T15:14:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f085a9dc700 1 -- 172.21.15.29:0/3148558223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0854159450 con 0x7f08540690b0 2024-01-24T15:14:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.398+0000 7f085a9dc700 1 -- 172.21.15.29:0/3148558223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08540aad20 con 0x7f08540690b0 2024-01-24T15:14:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.399+0000 7f085a9dc700 1 -- 172.21.15.29:0/3148558223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08540aaf40 con 0x7f08540690b0 2024-01-24T15:14:12.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.400+0000 7f083bfff700 1 -- 172.21.15.29:0/3148558223 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0844004c30 con 0x7f08540690b0 2024-01-24T15:14:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.400+0000 7f083bfff700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f084005f270 0x7f0840061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.400+0000 7f083bfff700 1 -- 172.21.15.29:0/3148558223 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6334+0+0 (secure 0 0 0) 0x7f0844079df0 con 0x7f08540690b0 2024-01-24T15:14:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.400+0000 7f08537fe700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f084005f270 0x7f0840061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:12.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.401+0000 7f08537fe700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f084005f270 0x7f0840061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f08540aacb0 tx=0x7f0848007bd0).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:12.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.405+0000 7f083bfff700 1 -- 172.21.15.29:0/3148558223 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f084404ae70 con 0x7f08540690b0 2024-01-24T15:14:12.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.660+0000 7f085a9dc700 1 -- 172.21.15.29:0/3148558223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f08540475e0 con 0x7f08540690b0 2024-01-24T15:14:12.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.660+0000 7f083bfff700 1 -- 172.21.15.29:0/3148558223 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v45) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0844019020 con 0x7f08540690b0 2024-01-24T15:14:12.663 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:12.663 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:14:12.663 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-01-24T15:14:12.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.663 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-01-24T15:14:12.663 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew==", 2024-01-24T15:14:12.663 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.663 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:14:12.663 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:14:12.663 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:14:12.663 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ==", 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.664 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw==", 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ==", 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:14:12.665 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ==", 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.666 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA==", 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw==", 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:14:12.667 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ==", 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-01-24T15:14:12.668 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw==", 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ==", 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A==", 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ==", 2024-01-24T15:14:12.670 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw==", 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-01-24T15:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA==", 2024-01-24T15:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T15:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T15:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T15:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-01-24T15:14:12.675 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g==", 2024-01-24T15:14:12.675 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.675 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ==", 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T15:14:12.676 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlxK1bFxAAZpfJn+Tcpmq7R9Yq29FvVg==", 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-01-24T15:14:12.677 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw==", 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA==", 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:12.679 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-01-24T15:14:12.679 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:14:12.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.662+0000 7f085a9dc700 1 -- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f084005f270 msgr2=0x7f0840061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.662+0000 7f085a9dc700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f084005f270 0x7f0840061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f08540aacb0 tx=0x7f0848007bd0).stop 2024-01-24T15:14:12.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.662+0000 7f085a9dc700 1 -- 172.21.15.29:0/3148558223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08540690b0 msgr2=0x7f0854133f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.662+0000 7f085a9dc700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08540690b0 0x7f0854133f60 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f084400c800 tx=0x7f0844001b50).stop 2024-01-24T15:14:12.680 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.663+0000 7f085a9dc700 1 -- 172.21.15.29:0/3148558223 shutdown_connections 2024-01-24T15:14:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.663+0000 7f085a9dc700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f085414c890 0x7f08540c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.663+0000 7f085a9dc700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f084005f270 0x7f0840061730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.663+0000 7f085a9dc700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08540690b0 0x7f0854133f60 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.663+0000 7f085a9dc700 1 --2- 172.21.15.29:0/3148558223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f085414c4b0 0x7f08540c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.663+0000 7f085a9dc700 1 -- 172.21.15.29:0/3148558223 >> 172.21.15.29:0/3148558223 conn(0x7f0854064aa0 msgr2=0x7f0854154d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.663+0000 7f085a9dc700 1 -- 172.21.15.29:0/3148558223 shutdown_connections 2024-01-24T15:14:12.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.663+0000 7f085a9dc700 1 -- 172.21.15.29:0/3148558223 wait complete. 2024-01-24T15:14:12.682 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:14:12.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.809+0000 7f553d082700 1 Processor -- start 2024-01-24T15:14:12.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.809+0000 7f553d082700 1 -- start start 2024-01-24T15:14:12.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.810+0000 7f553d082700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5538062970 0x7f55380633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:12.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f553d082700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5538066ab0 0x7f5538062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:12.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f553d082700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55380bce60 con 0x7f55380be540 2024-01-24T15:14:12.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f553d082700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5538069c10 con 0x7f5538062970 2024-01-24T15:14:12.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f553d082700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55380be9c0 con 0x7f5538066ab0 2024-01-24T15:14:12.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f5536d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5538062970 0x7f55380633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f553659c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f55380be540 0x7f5538065360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53546/0 (socket says 172.21.15.29:53546) 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f553659c700 1 -- 172.21.15.29:0/91663801 learned_addr learned my addr 172.21.15.29:0/91663801 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f553759e700 1 --2- 172.21.15.29:0/91663801 >> v2:172.21.15.163:3300/0 conn(0x7f5538066ab0 0x7f5538062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f5536d9d700 1 -- 172.21.15.29:0/91663801 >> v2:172.21.15.163:3300/0 conn(0x7f5538066ab0 msgr2=0x7f5538062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f5536d9d700 1 --2- 172.21.15.29:0/91663801 >> v2:172.21.15.163:3300/0 conn(0x7f5538066ab0 0x7f5538062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f5536d9d700 1 -- 172.21.15.29:0/91663801 >> v1:172.21.15.29:6789/0 conn(0x7f55380be540 legacy=0x7f5538065360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f5535d9b700 1 -- 172.21.15.29:0/91663801 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156085357 0 0) 0x7f55380bce60 con 0x7f55380be540 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f5536d9d700 1 -- 172.21.15.29:0/91663801 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5538069df0 con 0x7f5538062970 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f5536d9d700 1 --2- 172.21.15.29:0/91663801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5538062970 0x7f55380633c0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f55380bddd0 tx=0x7f5528009060).ready entity=mon.0 client_cookie=bd063cfad504e0a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.811+0000 7f5535d9b700 1 -- 172.21.15.29:0/91663801 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552800f510 con 0x7f5538062970 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.812+0000 7f5535d9b700 1 -- 172.21.15.29:0/91663801 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552800f670 con 0x7f5538062970 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.812+0000 7f5535d9b700 1 -- 172.21.15.29:0/91663801 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552800f9f0 con 0x7f5538062970 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.812+0000 7f553d082700 1 -- 172.21.15.29:0/91663801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5538062970 msgr2=0x7f55380633c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.812+0000 7f553d082700 1 --2- 172.21.15.29:0/91663801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5538062970 0x7f55380633c0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f55380bddd0 tx=0x7f5528009060).stop 2024-01-24T15:14:12.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.812+0000 7f553d082700 1 -- 172.21.15.29:0/91663801 shutdown_connections 2024-01-24T15:14:12.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.812+0000 7f553d082700 1 --2- 172.21.15.29:0/91663801 >> v2:172.21.15.163:3300/0 conn(0x7f5538066ab0 0x7f5538062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.812+0000 7f553d082700 1 --2- 172.21.15.29:0/91663801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5538062970 0x7f55380633c0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.812+0000 7f553d082700 1 -- 172.21.15.29:0/91663801 >> 172.21.15.29:0/91663801 conn(0x7f55380c85c0 msgr2=0x7f55380c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:12.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.812+0000 7f553d082700 1 -- 172.21.15.29:0/91663801 shutdown_connections 2024-01-24T15:14:12.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.812+0000 7f553d082700 1 -- 172.21.15.29:0/91663801 wait complete. 2024-01-24T15:14:12.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.813+0000 7f553d082700 1 Processor -- start 2024-01-24T15:14:12.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.814+0000 7f553d082700 1 -- start start 2024-01-24T15:14:12.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.814+0000 7f553d082700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5538062970 0x7f55380aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.814+0000 7f553d082700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5538066ab0 0x7f55380ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.814+0000 7f553d082700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55380be540 0x7f55380ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.814+0000 7f553d082700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380ace60 con 0x7f5538066ab0 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.814+0000 7f553d082700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380bed50 con 0x7f5538062970 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.814+0000 7f553d082700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55380ad060 con 0x7f55380be540 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.814+0000 7f553659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5538066ab0 0x7f55380ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f553659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5538066ab0 0x7f55380ac480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42130/0 (socket says 172.21.15.29:42130) 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f553659c700 1 -- 172.21.15.29:0/2860820144 learned_addr learned my addr 172.21.15.29:0/2860820144 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f5536d9d700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5538062970 0x7f55380aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f553759e700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55380be540 0x7f55380ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f553659c700 1 -- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55380be540 msgr2=0x7f55380ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f553659c700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55380be540 0x7f55380ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f553659c700 1 -- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5538062970 msgr2=0x7f55380aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f553659c700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5538062970 0x7f55380aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:12.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f553659c700 1 -- 172.21.15.29:0/2860820144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55280092a0 con 0x7f5538066ab0 2024-01-24T15:14:12.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f553659c700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5538066ab0 0x7f55380ac480 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f552000c490 tx=0x7f5520007e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:12.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f552effd700 1 -- 172.21.15.29:0/2860820144 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55200070d0 con 0x7f5538066ab0 2024-01-24T15:14:12.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f553d082700 1 -- 172.21.15.29:0/2860820144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55380beff0 con 0x7f5538066ab0 2024-01-24T15:14:12.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f552effd700 1 -- 172.21.15.29:0/2860820144 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5520007230 con 0x7f5538066ab0 2024-01-24T15:14:12.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f552effd700 1 -- 172.21.15.29:0/2860820144 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55200075d0 con 0x7f5538066ab0 2024-01-24T15:14:12.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.815+0000 7f553d082700 1 -- 172.21.15.29:0/2860820144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55380ad340 con 0x7f5538066ab0 2024-01-24T15:14:12.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.816+0000 7f553d082700 1 -- 172.21.15.29:0/2860820144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55380475e0 con 0x7f5538066ab0 2024-01-24T15:14:12.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.817+0000 7f552effd700 1 -- 172.21.15.29:0/2860820144 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5520005bc0 con 0x7f5538066ab0 2024-01-24T15:14:12.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.817+0000 7f552effd700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f551805f270 0x7f5518061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:12.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.817+0000 7f552effd700 1 -- 172.21.15.29:0/2860820144 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6334+0+0 (secure 0 0 0) 0x7f5520079d60 con 0x7f5538066ab0 2024-01-24T15:14:12.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.817+0000 7f5536d9d700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f551805f270 0x7f5518061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:12.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.818+0000 7f5536d9d700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f551805f270 0x7f5518061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f55380bddd0 tx=0x7f5528017000).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:12.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:12.822+0000 7f552effd700 1 -- 172.21.15.29:0/2860820144 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f552004ade0 con 0x7f5538066ab0 2024-01-24T15:14:13.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.002+0000 7f553d082700 1 -- 172.21.15.29:0/2860820144 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f553806a0d0 con 0x7f5538066ab0 2024-01-24T15:14:13.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.003+0000 7f552effd700 1 -- 172.21.15.29:0/2860820144 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+15037 (secure 0 0 0) 0x7f552004e400 con 0x7f5538066ab0 2024-01-24T15:14:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:13.005 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":189,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:14:12.237507+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:14:13.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.005+0000 7f553d082700 1 -- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f551805f270 msgr2=0x7f5518061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.005+0000 7f553d082700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f551805f270 0x7f5518061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f55380bddd0 tx=0x7f5528017000).stop 2024-01-24T15:14:13.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.005+0000 7f553d082700 1 -- 172.21.15.29:0/2860820144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5538066ab0 msgr2=0x7f55380ac480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.005+0000 7f553d082700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5538066ab0 0x7f55380ac480 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f552000c490 tx=0x7f5520007e60).stop 2024-01-24T15:14:13.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.005+0000 7f553d082700 1 -- 172.21.15.29:0/2860820144 shutdown_connections 2024-01-24T15:14:13.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.005+0000 7f553d082700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55380be540 0x7f55380ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.005+0000 7f553d082700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5538062970 0x7f55380aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.005+0000 7f553d082700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f551805f270 0x7f5518061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.005+0000 7f553d082700 1 --2- 172.21.15.29:0/2860820144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5538066ab0 0x7f55380ac480 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.005+0000 7f553d082700 1 -- 172.21.15.29:0/2860820144 >> 172.21.15.29:0/2860820144 conn(0x7f55380c85c0 msgr2=0x7f5538153af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:13.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.005+0000 7f553d082700 1 -- 172.21.15.29:0/2860820144 shutdown_connections 2024-01-24T15:14:13.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.005+0000 7f553d082700 1 -- 172.21.15.29:0/2860820144 wait complete. 2024-01-24T15:14:13.020 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T15:14:13.021 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T15:14:13.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.151+0000 7f73dc9f9700 1 Processor -- start 2024-01-24T15:14:13.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.151+0000 7f73dc9f9700 1 -- start start 2024-01-24T15:14:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.152+0000 7f73dc9f9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f73d8134ea0 0x7f73d80c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.156 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73dc9f9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d80c36c0 0x7f73d80664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73dc9f9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d80686a0 con 0x7f73d80669b0 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73dc9f9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d8066e30 con 0x7f73d8134ea0 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73dc9f9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d8068b30 con 0x7f73d80c36c0 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73d5d9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d80c36c0 0x7f73d80664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73d659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f73d8134ea0 0x7f73d80c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73d6d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f73d80669b0 0x7f73d81559f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:41534/0 (socket says 172.21.15.29:41534) 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73d5d9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d80c36c0 0x7f73d80664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:57344/0 (socket says 172.21.15.29:57344) 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73d6d9d700 1 -- 172.21.15.29:0/1765808937 learned_addr learned my addr 172.21.15.29:0/1765808937 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73d559a700 1 -- 172.21.15.29:0/1765808937 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720344202 0 0) 0x7f73d80686a0 con 0x7f73d80669b0 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73d559a700 1 -- 172.21.15.29:0/1765808937 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73c00033f0 con 0x7f73d80669b0 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73d659c700 1 -- 172.21.15.29:0/1765808937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d80c36c0 msgr2=0x7f73d80664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73d659c700 1 --2- 172.21.15.29:0/1765808937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d80c36c0 0x7f73d80664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73d659c700 1 -- 172.21.15.29:0/1765808937 >> v1:172.21.15.163:6789/0 conn(0x7f73d80669b0 legacy=0x7f73d81559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73d659c700 1 -- 172.21.15.29:0/1765808937 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d8067a90 con 0x7f73d8134ea0 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.153+0000 7f73d5d9b700 1 --2- 172.21.15.29:0/1765808937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d80c36c0 0x7f73d80664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.154+0000 7f73d659c700 1 --2- 172.21.15.29:0/1765808937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d8134ea0 0x7f73d80c31d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f73cc009760 tx=0x7f73cc009060).ready entity=mon.1 client_cookie=1cbbb6f2dce19a5e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.154+0000 7f73d559a700 1 -- 172.21.15.29:0/1765808937 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00d4a0 con 0x7f73d8134ea0 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.154+0000 7f73d559a700 1 -- 172.21.15.29:0/1765808937 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc00d600 con 0x7f73d8134ea0 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.154+0000 7f73d559a700 1 -- 172.21.15.29:0/1765808937 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00d980 con 0x7f73d8134ea0 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.154+0000 7f73dc9f9700 1 -- 172.21.15.29:0/1765808937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d8134ea0 msgr2=0x7f73d80c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.154+0000 7f73dc9f9700 1 --2- 172.21.15.29:0/1765808937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d8134ea0 0x7f73d80c31d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f73cc009760 tx=0x7f73cc009060).stop 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.154+0000 7f73dc9f9700 1 -- 172.21.15.29:0/1765808937 shutdown_connections 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.154+0000 7f73dc9f9700 1 --2- 172.21.15.29:0/1765808937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d8134ea0 0x7f73d80c31d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.154+0000 7f73dc9f9700 1 --2- 172.21.15.29:0/1765808937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d80c36c0 0x7f73d80664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.154+0000 7f73dc9f9700 1 -- 172.21.15.29:0/1765808937 >> 172.21.15.29:0/1765808937 conn(0x7f73d80c8bd0 msgr2=0x7f73d80c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.154+0000 7f73dc9f9700 1 -- 172.21.15.29:0/1765808937 shutdown_connections 2024-01-24T15:14:13.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.154+0000 7f73dc9f9700 1 -- 172.21.15.29:0/1765808937 wait complete. 2024-01-24T15:14:13.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.156+0000 7f73dc9f9700 1 Processor -- start 2024-01-24T15:14:13.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.156+0000 7f73dc9f9700 1 -- start start 2024-01-24T15:14:13.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.156+0000 7f73dc9f9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d80669b0 0x7f73d80ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.156+0000 7f73dc9f9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73d80c36c0 0x7f73d80c0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.156+0000 7f73dc9f9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d8134ea0 0x7f73d8064890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.156+0000 7f73dc9f9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80c2170 con 0x7f73d80c36c0 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.156+0000 7f73dc9f9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80c1560 con 0x7f73d8134ea0 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.156+0000 7f73dc9f9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80c1840 con 0x7f73d80669b0 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73d5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73d80c36c0 0x7f73d80c0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73d5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73d80c36c0 0x7f73d80c0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42138/0 (socket says 172.21.15.29:42138) 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73d5d9b700 1 -- 172.21.15.29:0/3660316822 learned_addr learned my addr 172.21.15.29:0/3660316822 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73d659c700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d80669b0 0x7f73d80ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73d6d9d700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d8134ea0 0x7f73d8064890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73d5d9b700 1 -- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d80669b0 msgr2=0x7f73d80ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73d5d9b700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d80669b0 0x7f73d80ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73d5d9b700 1 -- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d8134ea0 msgr2=0x7f73d8064890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73d5d9b700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d8134ea0 0x7f73d8064890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73d5d9b700 1 -- 172.21.15.29:0/3660316822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c80092a0 con 0x7f73d80c36c0 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73d5d9b700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73d80c36c0 0x7f73d80c0d10 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f73c800ea50 tx=0x7f73c8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:13.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73c67fc700 1 -- 172.21.15.29:0/3660316822 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c800c640 con 0x7f73d80c36c0 2024-01-24T15:14:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73c67fc700 1 -- 172.21.15.29:0/3660316822 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c800c7a0 con 0x7f73d80c36c0 2024-01-24T15:14:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73dc9f9700 1 -- 172.21.15.29:0/3660316822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73cc0092a0 con 0x7f73d80c36c0 2024-01-24T15:14:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.157+0000 7f73c67fc700 1 -- 172.21.15.29:0/3660316822 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c800cb20 con 0x7f73d80c36c0 2024-01-24T15:14:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.158+0000 7f73dc9f9700 1 -- 172.21.15.29:0/3660316822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d80c1e00 con 0x7f73d80c36c0 2024-01-24T15:14:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.159+0000 7f73dc9f9700 1 -- 172.21.15.29:0/3660316822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d80475e0 con 0x7f73d80c36c0 2024-01-24T15:14:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.159+0000 7f73c67fc700 1 -- 172.21.15.29:0/3660316822 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f73c801b400 con 0x7f73d80c36c0 2024-01-24T15:14:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.159+0000 7f73c67fc700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f73b805f270 0x7f73b8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.159+0000 7f73c67fc700 1 -- 172.21.15.29:0/3660316822 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6334+0+0 (secure 0 0 0) 0x7f73c807a9b0 con 0x7f73d80c36c0 2024-01-24T15:14:13.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.160+0000 7f73d659c700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f73b805f270 0x7f73b8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.160+0000 7f73d659c700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f73b805f270 0x7f73b8061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f73cc00d260 tx=0x7f73cc016000).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:13.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.164+0000 7f73c67fc700 1 -- 172.21.15.29:0/3660316822 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f73c804b9d0 con 0x7f73d80c36c0 2024-01-24T15:14:13.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.344+0000 7f73dc9f9700 1 -- 172.21.15.29:0/3660316822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f73d80c22b0 con 0x7f73d80c36c0 2024-01-24T15:14:13.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.345+0000 7f73c67fc700 1 -- 172.21.15.29:0/3660316822 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v189) v1 ==== 110+0+10 (secure 0 0 0) 0x7f73c804eff0 con 0x7f73d80c36c0 2024-01-24T15:14:13.347 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T15:14:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.347+0000 7f73dc9f9700 1 -- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f73b805f270 msgr2=0x7f73b8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.347+0000 7f73dc9f9700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f73b805f270 0x7f73b8061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f73cc00d260 tx=0x7f73cc016000).stop 2024-01-24T15:14:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.347+0000 7f73dc9f9700 1 -- 172.21.15.29:0/3660316822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73d80c36c0 msgr2=0x7f73d80c0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.347+0000 7f73dc9f9700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73d80c36c0 0x7f73d80c0d10 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f73c800ea50 tx=0x7f73c8009060).stop 2024-01-24T15:14:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.347+0000 7f73dc9f9700 1 -- 172.21.15.29:0/3660316822 shutdown_connections 2024-01-24T15:14:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.347+0000 7f73dc9f9700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d8134ea0 0x7f73d8064890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.347+0000 7f73dc9f9700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d80669b0 0x7f73d80ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.347+0000 7f73dc9f9700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f73b805f270 0x7f73b8061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.347+0000 7f73dc9f9700 1 --2- 172.21.15.29:0/3660316822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73d80c36c0 0x7f73d80c0d10 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.347+0000 7f73dc9f9700 1 -- 172.21.15.29:0/3660316822 >> 172.21.15.29:0/3660316822 conn(0x7f73d80c8bd0 msgr2=0x7f73d8150bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:13.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.347+0000 7f73dc9f9700 1 -- 172.21.15.29:0/3660316822 shutdown_connections 2024-01-24T15:14:13.350 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.347+0000 7f73dc9f9700 1 -- 172.21.15.29:0/3660316822 wait complete. 2024-01-24T15:14:13.362 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T15:14:13.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.493+0000 7fdd08bd8700 1 Processor -- start 2024-01-24T15:14:13.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.493+0000 7fdd08bd8700 1 -- start start 2024-01-24T15:14:13.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.494+0000 7fdd08bd8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd04153fe0 0x7fdd040c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.494+0000 7fdd08bd8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdd040c4920 0x7fdd040aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.494+0000 7fdd08bd8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd040637a0 con 0x7fdd04063c30 2024-01-24T15:14:13.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.494+0000 7fdd08bd8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd04062b90 con 0x7fdd040c4920 2024-01-24T15:14:13.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.494+0000 7fdd08bd8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04062e70 con 0x7fdd04153fe0 2024-01-24T15:14:13.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.494+0000 7fdd0259c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd04153fe0 0x7fdd040c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.494+0000 7fdcf9d9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdd040c4920 0x7fdd040aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.494+0000 7fdd0259c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd04153fe0 0x7fdd040c4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:54858/0 (socket says 172.21.15.29:54858) 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.494+0000 7fdcf9d9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdd040c4920 0x7fdd040aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:59964/0 (socket says 172.21.15.29:59964) 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.494+0000 7fdd02d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fdd04063c30 0x7fdd040ac400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:39564/0 (socket says 172.21.15.29:39564) 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.494+0000 7fdd0259c700 1 -- 172.21.15.29:0/3004542380 learned_addr learned my addr 172.21.15.29:0/3004542380 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.495+0000 7fdd0259c700 1 -- 172.21.15.29:0/3004542380 >> v2:172.21.15.163:3300/0 conn(0x7fdd040c4920 msgr2=0x7fdd040aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.495+0000 7fdd0259c700 1 --2- 172.21.15.29:0/3004542380 >> v2:172.21.15.163:3300/0 conn(0x7fdd040c4920 0x7fdd040aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.495+0000 7fdd0259c700 1 -- 172.21.15.29:0/3004542380 >> v1:172.21.15.163:6789/0 conn(0x7fdd04063c30 legacy=0x7fdd040ac400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.495+0000 7fdd0259c700 1 -- 172.21.15.29:0/3004542380 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd04063150 con 0x7fdd04153fe0 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.495+0000 7fdd01d9b700 1 -- 172.21.15.29:0/3004542380 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253190885 0 0) 0x7fdd040637a0 con 0x7fdd04063c30 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.495+0000 7fdcf9d9b700 1 --2- 172.21.15.29:0/3004542380 >> v2:172.21.15.163:3300/0 conn(0x7fdd040c4920 0x7fdd040aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.495+0000 7fdd0259c700 1 --2- 172.21.15.29:0/3004542380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd04153fe0 0x7fdd040c4430 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fdcf4006fd0 tx=0x7fdcf40090f0).ready entity=mon.2 client_cookie=4e1cf4fe65be63ca server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.495+0000 7fdd01d9b700 1 -- 172.21.15.29:0/3004542380 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf400b580 con 0x7fdd04153fe0 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.495+0000 7fdd01d9b700 1 -- 172.21.15.29:0/3004542380 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf400b6e0 con 0x7fdd04153fe0 2024-01-24T15:14:13.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.495+0000 7fdd01d9b700 1 -- 172.21.15.29:0/3004542380 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf400ba60 con 0x7fdd04153fe0 2024-01-24T15:14:13.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.496+0000 7fdd08bd8700 1 -- 172.21.15.29:0/3004542380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd04153fe0 msgr2=0x7fdd040c4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.496+0000 7fdd08bd8700 1 --2- 172.21.15.29:0/3004542380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd04153fe0 0x7fdd040c4430 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fdcf4006fd0 tx=0x7fdcf40090f0).stop 2024-01-24T15:14:13.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.496+0000 7fdd08bd8700 1 -- 172.21.15.29:0/3004542380 shutdown_connections 2024-01-24T15:14:13.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.496+0000 7fdd08bd8700 1 --2- 172.21.15.29:0/3004542380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd04153fe0 0x7fdd040c4430 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.496+0000 7fdd08bd8700 1 --2- 172.21.15.29:0/3004542380 >> v2:172.21.15.163:3300/0 conn(0x7fdd040c4920 0x7fdd040aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.496+0000 7fdd08bd8700 1 -- 172.21.15.29:0/3004542380 >> 172.21.15.29:0/3004542380 conn(0x7fdd040c7d50 msgr2=0x7fdd040c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.496+0000 7fdd08bd8700 1 -- 172.21.15.29:0/3004542380 shutdown_connections 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.496+0000 7fdd08bd8700 1 -- 172.21.15.29:0/3004542380 wait complete. 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.497+0000 7fdd08bd8700 1 Processor -- start 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.497+0000 7fdd08bd8700 1 -- start start 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.498+0000 7fdd08bd8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd04063c30 0x7fdd0413bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.498+0000 7fdd08bd8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd040c4920 0x7fdd0415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.498+0000 7fdd08bd8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd04153fe0 0x7fdd0414df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.498+0000 7fdd08bd8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04157990 con 0x7fdd04153fe0 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.498+0000 7fdd08bd8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04157c70 con 0x7fdd04063c30 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.498+0000 7fdd08bd8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04157f50 con 0x7fdd040c4920 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.498+0000 7fdd02d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd04153fe0 0x7fdd0414df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.498+0000 7fdd02d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd04153fe0 0x7fdd0414df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39124/0 (socket says 172.21.15.29:39124) 2024-01-24T15:14:13.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.498+0000 7fdd02d9d700 1 -- 172.21.15.29:0/1095340031 learned_addr learned my addr 172.21.15.29:0/1095340031 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.498+0000 7fdcf9d9b700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd040c4920 0x7fdd0415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.498+0000 7fdd0259c700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd04063c30 0x7fdd0413bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.499+0000 7fdd02d9d700 1 -- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd040c4920 msgr2=0x7fdd0415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.499+0000 7fdd02d9d700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd040c4920 0x7fdd0415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.499+0000 7fdd02d9d700 1 -- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd04063c30 msgr2=0x7fdd0413bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.499+0000 7fdd02d9d700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd04063c30 0x7fdd0413bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.499+0000 7fdd02d9d700 1 -- 172.21.15.29:0/1095340031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec0092a0 con 0x7fdd04153fe0 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.499+0000 7fdcf9d9b700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd040c4920 0x7fdd0415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.499+0000 7fdd02d9d700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd04153fe0 0x7fdd0414df10 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7fdcfc00c810 tx=0x7fdcfc0014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.499+0000 7fdcfaffd700 1 -- 172.21.15.29:0/1095340031 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc005c70 con 0x7fdd04153fe0 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.499+0000 7fdcfaffd700 1 -- 172.21.15.29:0/1095340031 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc005dd0 con 0x7fdd04153fe0 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.499+0000 7fdcfaffd700 1 -- 172.21.15.29:0/1095340031 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc007320 con 0x7fdd04153fe0 2024-01-24T15:14:13.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.499+0000 7fdd08bd8700 1 -- 172.21.15.29:0/1095340031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf4009370 con 0x7fdd04153fe0 2024-01-24T15:14:13.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.499+0000 7fdd08bd8700 1 -- 172.21.15.29:0/1095340031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd04158510 con 0x7fdd04153fe0 2024-01-24T15:14:13.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.500+0000 7fdd08bd8700 1 -- 172.21.15.29:0/1095340031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd04158720 con 0x7fdd04153fe0 2024-01-24T15:14:13.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.501+0000 7fdcfaffd700 1 -- 172.21.15.29:0/1095340031 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdcfc007710 con 0x7fdd04153fe0 2024-01-24T15:14:13.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.501+0000 7fdcfaffd700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdce405f270 0x7fdce4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.501+0000 7fdcfaffd700 1 -- 172.21.15.29:0/1095340031 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6334+0+0 (secure 0 0 0) 0x7fdcfc079d70 con 0x7fdd04153fe0 2024-01-24T15:14:13.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.501+0000 7fdd0259c700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdce405f270 0x7fdce4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.502+0000 7fdd0259c700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdce405f270 0x7fdce4061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fdcf4009130 tx=0x7fdcf4017000).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:13.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.506+0000 7fdcfaffd700 1 -- 172.21.15.29:0/1095340031 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdcfc04adf0 con 0x7fdd04153fe0 2024-01-24T15:14:13.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.685+0000 7fdd08bd8700 1 -- 172.21.15.29:0/1095340031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdd04063430 con 0x7fdd04153fe0 2024-01-24T15:14:13.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.686+0000 7fdcfaffd700 1 -- 172.21.15.29:0/1095340031 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v189) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdcfc04e410 con 0x7fdd04153fe0 2024-01-24T15:14:13.688 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T15:14:13.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.688+0000 7fdd08bd8700 1 -- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdce405f270 msgr2=0x7fdce4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.688+0000 7fdd08bd8700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdce405f270 0x7fdce4061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fdcf4009130 tx=0x7fdcf4017000).stop 2024-01-24T15:14:13.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.688+0000 7fdd08bd8700 1 -- 172.21.15.29:0/1095340031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd04153fe0 msgr2=0x7fdd0414df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.688+0000 7fdd08bd8700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd04153fe0 0x7fdd0414df10 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7fdcfc00c810 tx=0x7fdcfc0014a0).stop 2024-01-24T15:14:13.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.688+0000 7fdd08bd8700 1 -- 172.21.15.29:0/1095340031 shutdown_connections 2024-01-24T15:14:13.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.688+0000 7fdd08bd8700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fdce405f270 0x7fdce4061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.688+0000 7fdd08bd8700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd04153fe0 0x7fdd0414df10 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.688+0000 7fdd08bd8700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd04063c30 0x7fdd0413bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.688+0000 7fdd08bd8700 1 --2- 172.21.15.29:0/1095340031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd040c4920 0x7fdd0415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.688+0000 7fdd08bd8700 1 -- 172.21.15.29:0/1095340031 >> 172.21.15.29:0/1095340031 conn(0x7fdd040c7d50 msgr2=0x7fdd040c0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:13.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.688+0000 7fdd08bd8700 1 -- 172.21.15.29:0/1095340031 shutdown_connections 2024-01-24T15:14:13.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.688+0000 7fdd08bd8700 1 -- 172.21.15.29:0/1095340031 wait complete. 2024-01-24T15:14:13.704 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-24T15:14:13.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.833+0000 7f5f71601700 1 Processor -- start 2024-01-24T15:14:13.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.833+0000 7f5f71601700 1 -- start start 2024-01-24T15:14:13.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f71601700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f6c062970 0x7f5f6c062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f71601700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5f6c154190 0x7f5f6c0a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f71601700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f6c0bd190 con 0x7f5f6c069dd0 2024-01-24T15:14:13.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f71601700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6c154610 con 0x7f5f6c154190 2024-01-24T15:14:13.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f71601700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0c4e60 con 0x7f5f6c062970 2024-01-24T15:14:13.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f6affd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f6c062970 0x7f5f6c062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f6b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5f6c154190 0x7f5f6c0a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f6a7fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5f6c069dd0 0x7f5f6c153a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:39576/0 (socket says 172.21.15.29:39576) 2024-01-24T15:14:13.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f6a7fc700 1 -- 172.21.15.29:0/4224783915 learned_addr learned my addr 172.21.15.29:0/4224783915 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:13.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f6affd700 1 -- 172.21.15.29:0/4224783915 >> v2:172.21.15.163:3300/0 conn(0x7f5f6c154190 msgr2=0x7f5f6c0a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:13.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f6affd700 1 --2- 172.21.15.29:0/4224783915 >> v2:172.21.15.163:3300/0 conn(0x7f5f6c154190 0x7f5f6c0a6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f6affd700 1 -- 172.21.15.29:0/4224783915 >> v1:172.21.15.163:6789/0 conn(0x7f5f6c069dd0 legacy=0x7f5f6c153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f6affd700 1 -- 172.21.15.29:0/4224783915 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c066370 con 0x7f5f6c062970 2024-01-24T15:14:13.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.834+0000 7f5f6b7fe700 1 --2- 172.21.15.29:0/4224783915 >> v2:172.21.15.163:3300/0 conn(0x7f5f6c154190 0x7f5f6c0a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:13.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.835+0000 7f5f6affd700 1 --2- 172.21.15.29:0/4224783915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f6c062970 0x7f5f6c062d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f5f5c009760 tx=0x7f5f5c009060).ready entity=mon.2 client_cookie=416964bed705e442 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.835+0000 7f5f69ffb700 1 -- 172.21.15.29:0/4224783915 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c0090d0 con 0x7f5f6c062970 2024-01-24T15:14:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.835+0000 7f5f69ffb700 1 -- 172.21.15.29:0/4224783915 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5c00d4a0 con 0x7f5f6c062970 2024-01-24T15:14:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.835+0000 7f5f69ffb700 1 -- 172.21.15.29:0/4224783915 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c00d820 con 0x7f5f6c062970 2024-01-24T15:14:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.835+0000 7f5f71601700 1 -- 172.21.15.29:0/4224783915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f6c062970 msgr2=0x7f5f6c062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.835+0000 7f5f71601700 1 --2- 172.21.15.29:0/4224783915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f6c062970 0x7f5f6c062d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f5f5c009760 tx=0x7f5f5c009060).stop 2024-01-24T15:14:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.835+0000 7f5f71601700 1 -- 172.21.15.29:0/4224783915 shutdown_connections 2024-01-24T15:14:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.835+0000 7f5f71601700 1 --2- 172.21.15.29:0/4224783915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f6c062970 0x7f5f6c062d50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.835+0000 7f5f71601700 1 --2- 172.21.15.29:0/4224783915 >> v2:172.21.15.163:3300/0 conn(0x7f5f6c154190 0x7f5f6c0a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.835+0000 7f5f71601700 1 -- 172.21.15.29:0/4224783915 >> 172.21.15.29:0/4224783915 conn(0x7f5f6c0c85c0 msgr2=0x7f5f6c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.835+0000 7f5f71601700 1 -- 172.21.15.29:0/4224783915 shutdown_connections 2024-01-24T15:14:13.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.835+0000 7f5f71601700 1 -- 172.21.15.29:0/4224783915 wait complete. 2024-01-24T15:14:13.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.837+0000 7f5f71601700 1 Processor -- start 2024-01-24T15:14:13.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.837+0000 7f5f71601700 1 -- start start 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.837+0000 7f5f71601700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f6c062970 0x7f5f6c14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f71601700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f6c069dd0 0x7f5f6c1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f71601700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f6c154190 0x7f5f6c0c3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f71601700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0a7af0 con 0x7f5f6c062970 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f71601700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c151ef0 con 0x7f5f6c069dd0 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f71601700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0a7d90 con 0x7f5f6c154190 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f6affd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f6c062970 0x7f5f6c14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f6affd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f6c062970 0x7f5f6c14d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39138/0 (socket says 172.21.15.29:39138) 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f6affd700 1 -- 172.21.15.29:0/1708926362 learned_addr learned my addr 172.21.15.29:0/1708926362 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f6b7fe700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f6c154190 0x7f5f6c0c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f6a7fc700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f6c069dd0 0x7f5f6c1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f6affd700 1 -- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f6c154190 msgr2=0x7f5f6c0c3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f6affd700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f6c154190 0x7f5f6c0c3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f6affd700 1 -- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f6c069dd0 msgr2=0x7f5f6c1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f6affd700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f6c069dd0 0x7f5f6c1515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f6affd700 1 -- 172.21.15.29:0/1708926362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f64009a20 con 0x7f5f6c062970 2024-01-24T15:14:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f6a7fc700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f6c069dd0 0x7f5f6c1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:14:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f6affd700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f6c062970 0x7f5f6c14d920 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f5f6c0c4fa0 tx=0x7f5f5c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f52ffd700 1 -- 172.21.15.29:0/1708926362 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c012a50 con 0x7f5f6c062970 2024-01-24T15:14:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.838+0000 7f5f52ffd700 1 -- 172.21.15.29:0/1708926362 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5c012bb0 con 0x7f5f6c062970 2024-01-24T15:14:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.839+0000 7f5f71601700 1 -- 172.21.15.29:0/1708926362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5c0092a0 con 0x7f5f6c062970 2024-01-24T15:14:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.839+0000 7f5f52ffd700 1 -- 172.21.15.29:0/1708926362 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c01c420 con 0x7f5f6c062970 2024-01-24T15:14:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.839+0000 7f5f71601700 1 -- 172.21.15.29:0/1708926362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c0a8350 con 0x7f5f6c062970 2024-01-24T15:14:13.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.840+0000 7f5f71601700 1 -- 172.21.15.29:0/1708926362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f6c0475e0 con 0x7f5f6c062970 2024-01-24T15:14:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.840+0000 7f5f52ffd700 1 -- 172.21.15.29:0/1708926362 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5f5c01c850 con 0x7f5f6c062970 2024-01-24T15:14:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.841+0000 7f5f52ffd700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f5805f270 0x7f5f58061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.841+0000 7f5f52ffd700 1 -- 172.21.15.29:0/1708926362 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6334+0+0 (secure 0 0 0) 0x7f5f5c07bc30 con 0x7f5f6c062970 2024-01-24T15:14:13.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.841+0000 7f5f6a7fc700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f5805f270 0x7f5f58061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:13.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.841+0000 7f5f6a7fc700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f5805f270 0x7f5f58061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f5f6c0aa0a0 tx=0x7f5f60009040).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:13.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:13.845+0000 7f5f52ffd700 1 -- 172.21.15.29:0/1708926362 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5f5c04cbf0 con 0x7f5f6c062970 2024-01-24T15:14:14.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.029+0000 7f5f71601700 1 -- 172.21.15.29:0/1708926362 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5f6c063380 con 0x7f5f6c062970 2024-01-24T15:14:14.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.029+0000 7f5f52ffd700 1 -- 172.21.15.29:0/1708926362 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v189) v1 ==== 84+0+143 (secure 0 0 0) 0x7f5f5c016030 con 0x7f5f6c062970 2024-01-24T15:14:14.032 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:14.032 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T15:14:14.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:14.032 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-01-24T15:14:14.032 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "device_health_metrics" 2024-01-24T15:14:14.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:14:14.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:14.032 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 2, 2024-01-24T15:14:14.032 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "rbd" 2024-01-24T15:14:14.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:14.032 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T15:14:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.031+0000 7f5f71601700 1 -- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f5805f270 msgr2=0x7f5f58061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.031+0000 7f5f71601700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f5805f270 0x7f5f58061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f5f6c0aa0a0 tx=0x7f5f60009040).stop 2024-01-24T15:14:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.031+0000 7f5f71601700 1 -- 172.21.15.29:0/1708926362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f6c062970 msgr2=0x7f5f6c14d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.031+0000 7f5f71601700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f6c062970 0x7f5f6c14d920 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f5f6c0c4fa0 tx=0x7f5f5c014000).stop 2024-01-24T15:14:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.031+0000 7f5f71601700 1 -- 172.21.15.29:0/1708926362 shutdown_connections 2024-01-24T15:14:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.031+0000 7f5f71601700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f6c154190 0x7f5f6c0c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.031+0000 7f5f71601700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f5f5805f270 0x7f5f58061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.031+0000 7f5f71601700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5f6c062970 0x7f5f6c14d920 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.031+0000 7f5f71601700 1 --2- 172.21.15.29:0/1708926362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f6c069dd0 0x7f5f6c1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.031+0000 7f5f71601700 1 -- 172.21.15.29:0/1708926362 >> 172.21.15.29:0/1708926362 conn(0x7f5f6c0c85c0 msgr2=0x7f5f6c14f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.031+0000 7f5f71601700 1 -- 172.21.15.29:0/1708926362 shutdown_connections 2024-01-24T15:14:14.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.032+0000 7f5f71601700 1 -- 172.21.15.29:0/1708926362 wait complete. 2024-01-24T15:14:14.047 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-24T15:14:14.047 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2024-01-24T15:14:14.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.181+0000 7fd1f054b700 1 Processor -- start 2024-01-24T15:14:14.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.181+0000 7fd1f054b700 1 -- start start 2024-01-24T15:14:14.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.182+0000 7fd1f054b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e8069220 0x7fd1e815f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:14.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.182+0000 7fd1f054b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e80627b0 con 0x7fd1e8153f50 2024-01-24T15:14:14.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.182+0000 7fd1f054b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e8158230 con 0x7fd1e8154330 2024-01-24T15:14:14.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.182+0000 7fd1f054b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e8158510 con 0x7fd1e8069220 2024-01-24T15:14:14.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.182+0000 7fd1ee2e7700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd1e8153f50 0x7fd1e80c7ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35008/0 (socket says 172.21.15.29:35008) 2024-01-24T15:14:14.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.182+0000 7fd1ee2e7700 1 -- 172.21.15.29:0/3287239283 learned_addr learned my addr 172.21.15.29:0/3287239283 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:14.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.182+0000 7fd1eeae8700 1 --2- 172.21.15.29:0/3287239283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e8069220 0x7fd1e815f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:14.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.183+0000 7fd1ed2e5700 1 -- 172.21.15.29:0/3287239283 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956616488 0 0) 0x7fd1e80627b0 con 0x7fd1e8153f50 2024-01-24T15:14:14.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.183+0000 7fd1ed2e5700 1 -- 172.21.15.29:0/3287239283 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1d40033f0 con 0x7fd1e8153f50 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.183+0000 7fd1ed2e5700 1 -- 172.21.15.29:0/3287239283 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117107792 0 0) 0x7fd1e8158230 con 0x7fd1e8154330 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.183+0000 7fd1ed2e5700 1 -- 172.21.15.29:0/3287239283 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1e80627b0 con 0x7fd1e8154330 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.183+0000 7fd1eeae8700 1 -- 172.21.15.29:0/3287239283 >> v1:172.21.15.163:6789/0 conn(0x7fd1e8154330 legacy=0x7fd1e8068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.183+0000 7fd1eeae8700 1 -- 172.21.15.29:0/3287239283 >> v1:172.21.15.29:6789/0 conn(0x7fd1e8153f50 legacy=0x7fd1e80c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.183+0000 7fd1eeae8700 1 -- 172.21.15.29:0/3287239283 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e81587f0 con 0x7fd1e8069220 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.183+0000 7fd1eeae8700 1 --2- 172.21.15.29:0/3287239283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e8069220 0x7fd1e815f060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fd1e0009e90 tx=0x7fd1e0009780).ready entity=mon.2 client_cookie=e86eed48f380b2a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.183+0000 7fd1ed2e5700 1 -- 172.21.15.29:0/3287239283 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e000e070 con 0x7fd1e8069220 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.183+0000 7fd1ed2e5700 1 -- 172.21.15.29:0/3287239283 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e000bab0 con 0x7fd1e8069220 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.183+0000 7fd1ed2e5700 1 -- 172.21.15.29:0/3287239283 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e000be30 con 0x7fd1e8069220 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.184+0000 7fd1f054b700 1 -- 172.21.15.29:0/3287239283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e8069220 msgr2=0x7fd1e815f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.184+0000 7fd1f054b700 1 --2- 172.21.15.29:0/3287239283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e8069220 0x7fd1e815f060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fd1e0009e90 tx=0x7fd1e0009780).stop 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.184+0000 7fd1f054b700 1 -- 172.21.15.29:0/3287239283 shutdown_connections 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.184+0000 7fd1f054b700 1 --2- 172.21.15.29:0/3287239283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e8069220 0x7fd1e815f060 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:14.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.184+0000 7fd1f054b700 1 -- 172.21.15.29:0/3287239283 >> 172.21.15.29:0/3287239283 conn(0x7fd1e8065600 msgr2=0x7fd1e80c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:14.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.184+0000 7fd1f054b700 1 -- 172.21.15.29:0/3287239283 shutdown_connections 2024-01-24T15:14:14.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.184+0000 7fd1f054b700 1 -- 172.21.15.29:0/3287239283 wait complete. 2024-01-24T15:14:14.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.185+0000 7fd1f054b700 1 Processor -- start 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.185+0000 7fd1f054b700 1 -- start start 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1f054b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e8069220 0x7fd1e814ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1f054b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1e8153f50 0x7fd1e8164190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1f054b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e8154330 0x7fd1e80c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1f054b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80c5ba0 con 0x7fd1e8154330 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1f054b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80c2130 con 0x7fd1e8153f50 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1f054b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e8150430 con 0x7fd1e8069220 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1eeae8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e8154330 0x7fd1e80c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1eeae8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e8154330 0x7fd1e80c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39140/0 (socket says 172.21.15.29:39140) 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1eeae8700 1 -- 172.21.15.29:0/2259527262 learned_addr learned my addr 172.21.15.29:0/2259527262 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1edae6700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1e8153f50 0x7fd1e8164190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1ee2e7700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e8069220 0x7fd1e814ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1eeae8700 1 -- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e8069220 msgr2=0x7fd1e814ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:14.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1eeae8700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e8069220 0x7fd1e814ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:14.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1eeae8700 1 -- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1e8153f50 msgr2=0x7fd1e8164190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:14.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1eeae8700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1e8153f50 0x7fd1e8164190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:14.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.186+0000 7fd1eeae8700 1 -- 172.21.15.29:0/2259527262 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e0009a20 con 0x7fd1e8154330 2024-01-24T15:14:14.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.187+0000 7fd1eeae8700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e8154330 0x7fd1e80c5090 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7fd1e8068520 tx=0x7fd1e0009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:14.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.187+0000 7fd1de7fc700 1 -- 172.21.15.29:0/2259527262 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e001d5a0 con 0x7fd1e8154330 2024-01-24T15:14:14.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.187+0000 7fd1de7fc700 1 -- 172.21.15.29:0/2259527262 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e001d720 con 0x7fd1e8154330 2024-01-24T15:14:14.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.187+0000 7fd1f054b700 1 -- 172.21.15.29:0/2259527262 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e8164680 con 0x7fd1e8154330 2024-01-24T15:14:14.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.187+0000 7fd1de7fc700 1 -- 172.21.15.29:0/2259527262 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e001dae0 con 0x7fd1e8154330 2024-01-24T15:14:14.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.187+0000 7fd1f054b700 1 -- 172.21.15.29:0/2259527262 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e80c1ae0 con 0x7fd1e8154330 2024-01-24T15:14:14.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.188+0000 7fd1f054b700 1 -- 172.21.15.29:0/2259527262 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e80c1940 con 0x7fd1e8154330 2024-01-24T15:14:14.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.189+0000 7fd1de7fc700 1 -- 172.21.15.29:0/2259527262 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd1e0003040 con 0x7fd1e8154330 2024-01-24T15:14:14.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.189+0000 7fd1de7fc700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd1cc05f270 0x7fd1cc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:14.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.189+0000 7fd1de7fc700 1 -- 172.21.15.29:0/2259527262 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6334+0+0 (secure 0 0 0) 0x7fd1e007d0d0 con 0x7fd1e8154330 2024-01-24T15:14:14.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.190+0000 7fd1ee2e7700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd1cc05f270 0x7fd1cc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:14.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.190+0000 7fd1ee2e7700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd1cc05f270 0x7fd1cc061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fd1e4005750 tx=0x7fd1e4009040).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:14.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.193+0000 7fd1de7fc700 1 -- 172.21.15.29:0/2259527262 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd1e004e190 con 0x7fd1e8154330 2024-01-24T15:14:14.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:14.374+0000 7fd1f054b700 1 -- 172.21.15.29:0/2259527262 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fd1e80c1940 con 0x7fd1e8154330 2024-01-24T15:14:15.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.251+0000 7fd1de7fc700 1 -- 172.21.15.29:0/2259527262 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v190) v1 ==== 152+0+0 (secure 0 0 0) 0x7fd1e80c1940 con 0x7fd1e8154330 2024-01-24T15:14:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.255+0000 7fd1f054b700 1 -- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd1cc05f270 msgr2=0x7fd1cc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.255+0000 7fd1f054b700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd1cc05f270 0x7fd1cc061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fd1e4005750 tx=0x7fd1e4009040).stop 2024-01-24T15:14:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.255+0000 7fd1f054b700 1 -- 172.21.15.29:0/2259527262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e8154330 msgr2=0x7fd1e80c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.255+0000 7fd1f054b700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e8154330 0x7fd1e80c5090 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7fd1e8068520 tx=0x7fd1e0009780).stop 2024-01-24T15:14:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.255+0000 7fd1f054b700 1 -- 172.21.15.29:0/2259527262 shutdown_connections 2024-01-24T15:14:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.255+0000 7fd1f054b700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd1cc05f270 0x7fd1cc061730 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0x7fd1e4005750 tx=0x7fd1e4009040).stop 2024-01-24T15:14:15.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.255+0000 7fd1f054b700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e8154330 0x7fd1e80c5090 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.255+0000 7fd1f054b700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e8069220 0x7fd1e814ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.255+0000 7fd1f054b700 1 --2- 172.21.15.29:0/2259527262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1e8153f50 0x7fd1e8164190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.255+0000 7fd1f054b700 1 -- 172.21.15.29:0/2259527262 >> 172.21.15.29:0/2259527262 conn(0x7fd1e8065600 msgr2=0x7fd1e8157070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:15.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.256+0000 7fd1f054b700 1 -- 172.21.15.29:0/2259527262 shutdown_connections 2024-01-24T15:14:15.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.258+0000 7fd1f054b700 1 -- 172.21.15.29:0/2259527262 wait complete. 2024-01-24T15:14:15.263 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-01-24T15:14:15.278 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-24T15:14:15.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.417+0000 7f3f63441700 1 Processor -- start 2024-01-24T15:14:15.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.417+0000 7f3f63441700 1 -- start start 2024-01-24T15:14:15.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.418+0000 7f3f63441700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f5c0ba5a0 0x7f3f5c064c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:15.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.418+0000 7f3f63441700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3f5c0ba980 0x7f3f5c159570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:15.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.418+0000 7f3f63441700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5c159a60 con 0x7f3f5c065a80 2024-01-24T15:14:15.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.418+0000 7f3f63441700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c158ab0 con 0x7f3f5c0ba980 2024-01-24T15:14:15.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.418+0000 7f3f63441700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c065220 con 0x7f3f5c0ba5a0 2024-01-24T15:14:15.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f611dd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f5c0ba5a0 0x7f3f5c064c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:15.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f609dc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3f5c0ba980 0x7f3f5c159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:15.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f619de700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3f5c065a80 0x7f3f5c0c19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:39590/0 (socket says 172.21.15.29:39590) 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f619de700 1 -- 172.21.15.29:0/3823554024 learned_addr learned my addr 172.21.15.29:0/3823554024 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f609dc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3f5c0ba980 0x7f3f5c159570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:60032/0 (socket says 172.21.15.29:60032) 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f611dd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f5c0ba5a0 0x7f3f5c064c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:54906/0 (socket says 172.21.15.29:54906) 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f53fff700 1 -- 172.21.15.29:0/3823554024 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029705186 0 0) 0x7f3f5c159a60 con 0x7f3f5c065a80 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f53fff700 1 -- 172.21.15.29:0/3823554024 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f480033f0 con 0x7f3f5c065a80 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f611dd700 1 -- 172.21.15.29:0/3823554024 >> v2:172.21.15.163:3300/0 conn(0x7f3f5c0ba980 msgr2=0x7f3f5c159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f611dd700 1 --2- 172.21.15.29:0/3823554024 >> v2:172.21.15.163:3300/0 conn(0x7f3f5c0ba980 0x7f3f5c159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f611dd700 1 -- 172.21.15.29:0/3823554024 >> v1:172.21.15.163:6789/0 conn(0x7f3f5c065a80 legacy=0x7f3f5c0c19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f611dd700 1 -- 172.21.15.29:0/3823554024 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c065500 con 0x7f3f5c0ba5a0 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f609dc700 1 --2- 172.21.15.29:0/3823554024 >> v2:172.21.15.163:3300/0 conn(0x7f3f5c0ba980 0x7f3f5c159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f611dd700 1 --2- 172.21.15.29:0/3823554024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f5c0ba5a0 0x7f3f5c064c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f3f58006fd0 tx=0x7f3f58006f40).ready entity=mon.2 client_cookie=d33da25599e8e89 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f53fff700 1 -- 172.21.15.29:0/3823554024 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58005880 con 0x7f3f5c0ba5a0 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f53fff700 1 -- 172.21.15.29:0/3823554024 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f580059e0 con 0x7f3f5c0ba5a0 2024-01-24T15:14:15.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.419+0000 7f3f53fff700 1 -- 172.21.15.29:0/3823554024 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58005d60 con 0x7f3f5c0ba5a0 2024-01-24T15:14:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.420+0000 7f3f63441700 1 -- 172.21.15.29:0/3823554024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f5c0ba5a0 msgr2=0x7f3f5c064c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.420+0000 7f3f63441700 1 --2- 172.21.15.29:0/3823554024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f5c0ba5a0 0x7f3f5c064c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f3f58006fd0 tx=0x7f3f58006f40).stop 2024-01-24T15:14:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.420+0000 7f3f63441700 1 -- 172.21.15.29:0/3823554024 shutdown_connections 2024-01-24T15:14:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.420+0000 7f3f63441700 1 --2- 172.21.15.29:0/3823554024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f5c0ba5a0 0x7f3f5c064c40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.420+0000 7f3f63441700 1 --2- 172.21.15.29:0/3823554024 >> v2:172.21.15.163:3300/0 conn(0x7f3f5c0ba980 0x7f3f5c159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.420+0000 7f3f63441700 1 -- 172.21.15.29:0/3823554024 >> 172.21.15.29:0/3823554024 conn(0x7f3f5c0c7330 msgr2=0x7f3f5c0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.420+0000 7f3f63441700 1 -- 172.21.15.29:0/3823554024 shutdown_connections 2024-01-24T15:14:15.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.420+0000 7f3f63441700 1 -- 172.21.15.29:0/3823554024 wait complete. 2024-01-24T15:14:15.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.421+0000 7f3f63441700 1 Processor -- start 2024-01-24T15:14:15.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.421+0000 7f3f63441700 1 -- start start 2024-01-24T15:14:15.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.422+0000 7f3f63441700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f5c065a80 0x7f3f5c1693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:15.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.422+0000 7f3f63441700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f5c0ba5a0 0x7f3f5c153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:15.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.422+0000 7f3f63441700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f5c0ba980 0x7f3f5c0b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.422+0000 7f3f63441700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c1698a0 con 0x7f3f5c065a80 2024-01-24T15:14:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.422+0000 7f3f63441700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c153670 con 0x7f3f5c0ba980 2024-01-24T15:14:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.422+0000 7f3f63441700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0b60a0 con 0x7f3f5c0ba5a0 2024-01-24T15:14:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f611dd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f5c065a80 0x7f3f5c1693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f611dd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f5c065a80 0x7f3f5c1693b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39146/0 (socket says 172.21.15.29:39146) 2024-01-24T15:14:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f611dd700 1 -- 172.21.15.29:0/2078377290 learned_addr learned my addr 172.21.15.29:0/2078377290 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f609dc700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f5c0ba5a0 0x7f3f5c153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f619de700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f5c0ba980 0x7f3f5c0b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:15.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f611dd700 1 -- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f5c0ba5a0 msgr2=0x7f3f5c153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f611dd700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f5c0ba5a0 0x7f3f5c153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f611dd700 1 -- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f5c0ba980 msgr2=0x7f3f5c0b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f611dd700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f5c0ba980 0x7f3f5c0b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f611dd700 1 -- 172.21.15.29:0/2078377290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c0092a0 con 0x7f3f5c065a80 2024-01-24T15:14:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f619de700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f5c0ba980 0x7f3f5c0b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:14:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f611dd700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f5c065a80 0x7f3f5c1693b0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7f3f5c065360 tx=0x7f3f58015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f517fa700 1 -- 172.21.15.29:0/2078377290 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58006bd0 con 0x7f3f5c065a80 2024-01-24T15:14:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f517fa700 1 -- 172.21.15.29:0/2078377290 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5800d6b0 con 0x7f3f5c065a80 2024-01-24T15:14:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f63441700 1 -- 172.21.15.29:0/2078377290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f58009250 con 0x7f3f5c065a80 2024-01-24T15:14:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f517fa700 1 -- 172.21.15.29:0/2078377290 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800da70 con 0x7f3f5c065a80 2024-01-24T15:14:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.423+0000 7f3f63441700 1 -- 172.21.15.29:0/2078377290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5c1538d0 con 0x7f3f5c065a80 2024-01-24T15:14:15.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.424+0000 7f3f63441700 1 -- 172.21.15.29:0/2078377290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5c0b63f0 con 0x7f3f5c065a80 2024-01-24T15:14:15.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.425+0000 7f3f517fa700 1 -- 172.21.15.29:0/2078377290 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3f5801e460 con 0x7f3f5c065a80 2024-01-24T15:14:15.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.425+0000 7f3f517fa700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f4005f270 0x7f3f40061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:15.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.425+0000 7f3f517fa700 1 -- 172.21.15.29:0/2078377290 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f3f5807b970 con 0x7f3f5c065a80 2024-01-24T15:14:15.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.426+0000 7f3f609dc700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f4005f270 0x7f3f40061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:15.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.426+0000 7f3f609dc700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f4005f270 0x7f3f40061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f3f5c0ba030 tx=0x7f3f4c009060).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:15.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.430+0000 7f3f517fa700 1 -- 172.21.15.29:0/2078377290 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3f5804c8d0 con 0x7f3f5c065a80 2024-01-24T15:14:15.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:15.616+0000 7f3f63441700 1 -- 172.21.15.29:0/2078377290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3f5c158d70 con 0x7f3f5c065a80 2024-01-24T15:14:16.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.256+0000 7f3f517fa700 1 -- 172.21.15.29:0/2078377290 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v191) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3f58023020 con 0x7f3f5c065a80 2024-01-24T15:14:16.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.266+0000 7f3f63441700 1 -- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f4005f270 msgr2=0x7f3f40061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:16.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.266+0000 7f3f63441700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f4005f270 0x7f3f40061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f3f5c0ba030 tx=0x7f3f4c009060).stop 2024-01-24T15:14:16.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.266+0000 7f3f63441700 1 -- 172.21.15.29:0/2078377290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f5c065a80 msgr2=0x7f3f5c1693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:16.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.266+0000 7f3f63441700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f5c065a80 0x7f3f5c1693b0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7f3f5c065360 tx=0x7f3f58015000).stop 2024-01-24T15:14:16.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.267+0000 7f3f63441700 1 -- 172.21.15.29:0/2078377290 shutdown_connections 2024-01-24T15:14:16.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.268+0000 7f3f63441700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f5c0ba980 0x7f3f5c0b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.268+0000 7f3f63441700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f3f4005f270 0x7f3f40061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.268+0000 7f3f63441700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f5c065a80 0x7f3f5c1693b0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.268+0000 7f3f63441700 1 --2- 172.21.15.29:0/2078377290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f5c0ba5a0 0x7f3f5c153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.268+0000 7f3f63441700 1 -- 172.21.15.29:0/2078377290 >> 172.21.15.29:0/2078377290 conn(0x7f3f5c0c7330 msgr2=0x7f3f5c0c5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:16.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.270+0000 7f3f63441700 1 -- 172.21.15.29:0/2078377290 shutdown_connections 2024-01-24T15:14:16.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.270+0000 7f3f63441700 1 -- 172.21.15.29:0/2078377290 wait complete. 2024-01-24T15:14:16.285 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-01-24T15:14:16.298 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-24T15:14:16.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.431+0000 7f2fd22db700 1 Processor -- start 2024-01-24T15:14:16.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.431+0000 7f2fd22db700 1 -- start start 2024-01-24T15:14:16.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.432+0000 7f2fd22db700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2fcc0692a0 0x7f2fcc065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:16.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.432+0000 7f2fd22db700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fcc066370 0x7f2fcc069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:16.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fd22db700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fcc0c3e70 con 0x7f2fcc069b70 2024-01-24T15:14:16.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fd22db700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fcc0c4300 con 0x7f2fcc0692a0 2024-01-24T15:14:16.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fd22db700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc0c3260 con 0x7f2fcc066370 2024-01-24T15:14:16.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fcbfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2fcc0692a0 0x7f2fcc065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:16.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fd0878700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2fcc069b70 0x7f2fcc0bcd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35022/0 (socket says 172.21.15.29:35022) 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fd0878700 1 -- 172.21.15.29:0/3217224961 learned_addr learned my addr 172.21.15.29:0/3217224961 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fcb7fe700 1 --2- 172.21.15.29:0/3217224961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fcc066370 0x7f2fcc069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fcbfff700 1 -- 172.21.15.29:0/3217224961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fcc066370 msgr2=0x7f2fcc069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fcbfff700 1 --2- 172.21.15.29:0/3217224961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fcc066370 0x7f2fcc069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fcbfff700 1 -- 172.21.15.29:0/3217224961 >> v1:172.21.15.29:6789/0 conn(0x7f2fcc069b70 legacy=0x7f2fcc0bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fcbfff700 1 -- 172.21.15.29:0/3217224961 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc0c3540 con 0x7f2fcc0692a0 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fcaffd700 1 -- 172.21.15.29:0/3217224961 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301396135 0 0) 0x7f2fcc0c3e70 con 0x7f2fcc069b70 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fcb7fe700 1 --2- 172.21.15.29:0/3217224961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fcc066370 0x7f2fcc069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fcbfff700 1 --2- 172.21.15.29:0/3217224961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fcc0692a0 0x7f2fcc065e80 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f2fbc009760 tx=0x7f2fbc009060).ready entity=mon.0 client_cookie=62d636b0afe9ce25 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fcaffd700 1 -- 172.21.15.29:0/3217224961 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc00b580 con 0x7f2fcc0692a0 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fcaffd700 1 -- 172.21.15.29:0/3217224961 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc00b6e0 con 0x7f2fcc0692a0 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.433+0000 7f2fcaffd700 1 -- 172.21.15.29:0/3217224961 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc00ba60 con 0x7f2fcc0692a0 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.434+0000 7f2fd22db700 1 -- 172.21.15.29:0/3217224961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fcc0692a0 msgr2=0x7f2fcc065e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:16.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.434+0000 7f2fd22db700 1 --2- 172.21.15.29:0/3217224961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fcc0692a0 0x7f2fcc065e80 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f2fbc009760 tx=0x7f2fbc009060).stop 2024-01-24T15:14:16.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.434+0000 7f2fd22db700 1 -- 172.21.15.29:0/3217224961 shutdown_connections 2024-01-24T15:14:16.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.434+0000 7f2fd22db700 1 --2- 172.21.15.29:0/3217224961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fcc0692a0 0x7f2fcc065e80 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.434+0000 7f2fd22db700 1 --2- 172.21.15.29:0/3217224961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fcc066370 0x7f2fcc069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.434+0000 7f2fd22db700 1 -- 172.21.15.29:0/3217224961 >> 172.21.15.29:0/3217224961 conn(0x7f2fcc064390 msgr2=0x7f2fcc0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:16.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.434+0000 7f2fd22db700 1 -- 172.21.15.29:0/3217224961 shutdown_connections 2024-01-24T15:14:16.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.434+0000 7f2fd22db700 1 -- 172.21.15.29:0/3217224961 wait complete. 2024-01-24T15:14:16.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.435+0000 7f2fd22db700 1 Processor -- start 2024-01-24T15:14:16.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.436+0000 7f2fd22db700 1 -- start start 2024-01-24T15:14:16.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.436+0000 7f2fd22db700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fcc066370 0x7f2fcc14c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:16.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.436+0000 7f2fd22db700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fcc0692a0 0x7f2fcc153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:16.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.436+0000 7f2fd22db700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fcc069b70 0x7f2fcc161db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:16.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.436+0000 7f2fd22db700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc154c60 con 0x7f2fcc0692a0 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.436+0000 7f2fd22db700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc1522d0 con 0x7f2fcc066370 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.436+0000 7f2fd22db700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc1513e0 con 0x7f2fcc069b70 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.436+0000 7f2fcb7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fcc0692a0 0x7f2fcc153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.436+0000 7f2fcb7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fcc0692a0 0x7f2fcc153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39162/0 (socket says 172.21.15.29:39162) 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.436+0000 7f2fcb7fe700 1 -- 172.21.15.29:0/1018712380 learned_addr learned my addr 172.21.15.29:0/1018712380 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fd0878700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fcc069b70 0x7f2fcc161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fcbfff700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fcc066370 0x7f2fcc14c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fcb7fe700 1 -- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fcc069b70 msgr2=0x7f2fcc161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fcb7fe700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fcc069b70 0x7f2fcc161db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fcb7fe700 1 -- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fcc066370 msgr2=0x7f2fcc14c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fcb7fe700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fcc066370 0x7f2fcc14c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fcb7fe700 1 -- 172.21.15.29:0/1018712380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc0092a0 con 0x7f2fcc0692a0 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fcb7fe700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fcc0692a0 0x7f2fcc153bc0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f2fb400d890 tx=0x7f2fb4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fc3fff700 1 -- 172.21.15.29:0/1018712380 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb40125f0 con 0x7f2fcc0692a0 2024-01-24T15:14:16.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fc3fff700 1 -- 172.21.15.29:0/1018712380 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb4012750 con 0x7f2fcc0692a0 2024-01-24T15:14:16.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fc3fff700 1 -- 172.21.15.29:0/1018712380 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4012ad0 con 0x7f2fcc0692a0 2024-01-24T15:14:16.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fd22db700 1 -- 172.21.15.29:0/1018712380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fcc1516c0 con 0x7f2fcc0692a0 2024-01-24T15:14:16.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.437+0000 7f2fd22db700 1 -- 172.21.15.29:0/1018712380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fcc151c80 con 0x7f2fcc0692a0 2024-01-24T15:14:16.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.438+0000 7f2fd22db700 1 -- 172.21.15.29:0/1018712380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fcc0475e0 con 0x7f2fcc0692a0 2024-01-24T15:14:16.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.439+0000 7f2fc3fff700 1 -- 172.21.15.29:0/1018712380 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2fb4013480 con 0x7f2fcc0692a0 2024-01-24T15:14:16.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.439+0000 7f2fc3fff700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2fac05f270 0x7f2fac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:16.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.439+0000 7f2fc3fff700 1 -- 172.21.15.29:0/1018712380 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 7086+0+0 (secure 0 0 0) 0x7f2fb407a640 con 0x7f2fcc0692a0 2024-01-24T15:14:16.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.439+0000 7f2fcbfff700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2fac05f270 0x7f2fac061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:16.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.440+0000 7f2fcbfff700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2fac05f270 0x7f2fac061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2fcc165260 tx=0x7f2fbc017000).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:16.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.444+0000 7f2fc3fff700 1 -- 172.21.15.29:0/1018712380 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2fb404b2b0 con 0x7f2fcc0692a0 2024-01-24T15:14:16.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.671+0000 7f2fd22db700 1 -- 172.21.15.29:0/1018712380 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2fcc0c2190 con 0x7f2fcc0692a0 2024-01-24T15:14:16.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.679+0000 7f2fc3fff700 1 -- 172.21.15.29:0/1018712380 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v78) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2fb404e8d0 con 0x7f2fcc0692a0 2024-01-24T15:14:16.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.683+0000 7f2fd22db700 1 -- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2fac05f270 msgr2=0x7f2fac061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:16.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.683+0000 7f2fd22db700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2fac05f270 0x7f2fac061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2fcc165260 tx=0x7f2fbc017000).stop 2024-01-24T15:14:16.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.683+0000 7f2fd22db700 1 -- 172.21.15.29:0/1018712380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fcc0692a0 msgr2=0x7f2fcc153bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:16.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.683+0000 7f2fd22db700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fcc0692a0 0x7f2fcc153bc0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f2fb400d890 tx=0x7f2fb4008040).stop 2024-01-24T15:14:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.685+0000 7f2fd22db700 1 -- 172.21.15.29:0/1018712380 shutdown_connections 2024-01-24T15:14:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.685+0000 7f2fd22db700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fcc069b70 0x7f2fcc161db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.685+0000 7f2fd22db700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fcc066370 0x7f2fcc14c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.685+0000 7f2fd22db700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f2fac05f270 0x7f2fac061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.685+0000 7f2fd22db700 1 --2- 172.21.15.29:0/1018712380 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fcc0692a0 0x7f2fcc153bc0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.685+0000 7f2fd22db700 1 -- 172.21.15.29:0/1018712380 >> 172.21.15.29:0/1018712380 conn(0x7f2fcc064390 msgr2=0x7f2fcc0ab550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:16.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.686+0000 7f2fd22db700 1 -- 172.21.15.29:0/1018712380 shutdown_connections 2024-01-24T15:14:16.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.686+0000 7f2fd22db700 1 -- 172.21.15.29:0/1018712380 wait complete. 2024-01-24T15:14:16.698 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 22 and data pool 23 2024-01-24T15:14:16.715 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:14:16.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.843+0000 7f84bd14f700 1 Processor -- start 2024-01-24T15:14:16.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.843+0000 7f84bd14f700 1 -- start start 2024-01-24T15:14:16.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84bd14f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b8066c00 0x7f84b80bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84bd14f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84b80670a0 0x7f84b8158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84bd14f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b81582b0 con 0x7f84b80c4a10 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84bd14f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b8157ca0 con 0x7f84b80670a0 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84bd14f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b8063c60 con 0x7f84b8066c00 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84b759e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84b80670a0 0x7f84b8158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84b759e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f84b80670a0 0x7f84b8158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39166/0 (socket says 172.21.15.29:39166) 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84b759e700 1 -- 172.21.15.29:0/2791344997 learned_addr learned my addr 172.21.15.29:0/2791344997 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84b659c700 1 --2- 172.21.15.29:0/2791344997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b8066c00 0x7f84b80bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84b759e700 1 -- 172.21.15.29:0/2791344997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b8066c00 msgr2=0x7f84b80bf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84b759e700 1 --2- 172.21.15.29:0/2791344997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b8066c00 0x7f84b80bf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84b759e700 1 -- 172.21.15.29:0/2791344997 >> v1:172.21.15.163:6789/0 conn(0x7f84b80c4a10 legacy=0x7f84b80664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84b759e700 1 -- 172.21.15.29:0/2791344997 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b8157f40 con 0x7f84b80670a0 2024-01-24T15:14:16.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84b659c700 1 --2- 172.21.15.29:0/2791344997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b8066c00 0x7f84b80bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:14:16.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.844+0000 7f84b759e700 1 --2- 172.21.15.29:0/2791344997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b80670a0 0x7f84b8158740 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7f84b0009f20 tx=0x7f84b0009840).ready entity=mon.0 client_cookie=3eb00a483426d60f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:16.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.845+0000 7f84b5d9b700 1 -- 172.21.15.29:0/2791344997 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000e070 con 0x7f84b80670a0 2024-01-24T15:14:16.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.845+0000 7f84b5d9b700 1 -- 172.21.15.29:0/2791344997 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b000ba40 con 0x7f84b80670a0 2024-01-24T15:14:16.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.845+0000 7f84b5d9b700 1 -- 172.21.15.29:0/2791344997 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000bdc0 con 0x7f84b80670a0 2024-01-24T15:14:16.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.845+0000 7f84bd14f700 1 -- 172.21.15.29:0/2791344997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b80670a0 msgr2=0x7f84b8158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:16.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.845+0000 7f84bd14f700 1 --2- 172.21.15.29:0/2791344997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b80670a0 0x7f84b8158740 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7f84b0009f20 tx=0x7f84b0009840).stop 2024-01-24T15:14:16.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.845+0000 7f84bd14f700 1 -- 172.21.15.29:0/2791344997 shutdown_connections 2024-01-24T15:14:16.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.845+0000 7f84bd14f700 1 --2- 172.21.15.29:0/2791344997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b80670a0 0x7f84b8158740 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.845+0000 7f84bd14f700 1 --2- 172.21.15.29:0/2791344997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b8066c00 0x7f84b80bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.845+0000 7f84bd14f700 1 -- 172.21.15.29:0/2791344997 >> 172.21.15.29:0/2791344997 conn(0x7f84b80c8bb0 msgr2=0x7f84b80c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:16.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.845+0000 7f84bd14f700 1 -- 172.21.15.29:0/2791344997 shutdown_connections 2024-01-24T15:14:16.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.845+0000 7f84bd14f700 1 -- 172.21.15.29:0/2791344997 wait complete. 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.847+0000 7f84bd14f700 1 Processor -- start 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.847+0000 7f84bd14f700 1 -- start start 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.847+0000 7f84bd14f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b8066c00 0x7f84b8155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.847+0000 7f84bd14f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84b80670a0 0x7f84b80c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.847+0000 7f84bd14f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b80c4a10 0x7f84b8163a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.847+0000 7f84bd14f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b8154b80 con 0x7f84b80c4a10 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.847+0000 7f84bd14f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b8164a60 con 0x7f84b80670a0 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.847+0000 7f84bd14f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b81644a0 con 0x7f84b8066c00 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b759e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b80c4a10 0x7f84b8163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b759e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b80c4a10 0x7f84b8163a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39178/0 (socket says 172.21.15.29:39178) 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b759e700 1 -- 172.21.15.29:0/3229859462 learned_addr learned my addr 172.21.15.29:0/3229859462 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b659c700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84b80670a0 0x7f84b80c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:16.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b6d9d700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b8066c00 0x7f84b8155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b759e700 1 -- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b8066c00 msgr2=0x7f84b8155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b759e700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b8066c00 0x7f84b8155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b759e700 1 -- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84b80670a0 msgr2=0x7f84b80c0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b759e700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84b80670a0 0x7f84b80c0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b759e700 1 -- 172.21.15.29:0/3229859462 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b0009ae0 con 0x7f84b80c4a10 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b659c700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84b80670a0 0x7f84b80c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b6d9d700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b8066c00 0x7f84b8155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84b759e700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b80c4a10 0x7f84b8163a50 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f84b0009880 tx=0x7f84b8066140).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f849effd700 1 -- 172.21.15.29:0/3229859462 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b001d410 con 0x7f84b80c4a10 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f849effd700 1 -- 172.21.15.29:0/3229859462 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b001d570 con 0x7f84b80c4a10 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84bd14f700 1 -- 172.21.15.29:0/3229859462 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b8164780 con 0x7f84b80c4a10 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f849effd700 1 -- 172.21.15.29:0/3229859462 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b001d930 con 0x7f84b80c4a10 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.848+0000 7f84bd14f700 1 -- 172.21.15.29:0/3229859462 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b8155500 con 0x7f84b80c4a10 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.849+0000 7f84bd14f700 1 -- 172.21.15.29:0/3229859462 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b8154670 con 0x7f84b80c4a10 2024-01-24T15:14:16.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.850+0000 7f849effd700 1 -- 172.21.15.29:0/3229859462 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f84b001dce0 con 0x7f84b80c4a10 2024-01-24T15:14:16.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.850+0000 7f849effd700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f84a405f270 0x7f84a4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:16.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.850+0000 7f849effd700 1 -- 172.21.15.29:0/3229859462 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f84b007c460 con 0x7f84b80c4a10 2024-01-24T15:14:16.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.850+0000 7f84b6d9d700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f84a405f270 0x7f84a4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:16.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.851+0000 7f84b6d9d700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f84a405f270 0x7f84a4061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f84b80c4100 tx=0x7f84a8008040).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:16.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:16.855+0000 7f849effd700 1 -- 172.21.15.29:0/3229859462 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f84b8154670 con 0x7f84b80c4a10 2024-01-24T15:14:17.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.042+0000 7f84bd14f700 1 -- 172.21.15.29:0/3229859462 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84b8154d30 con 0x7f84b80c4a10 2024-01-24T15:14:17.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.043+0000 7f849effd700 1 -- 172.21.15.29:0/3229859462 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+18270 (secure 0 0 0) 0x7f84b8154d30 con 0x7f84b80c4a10 2024-01-24T15:14:17.045 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:17.045 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":192,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:14:16.673237+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:14:14.419558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-24T15:14:15.659901+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:14:17.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.045+0000 7f84bd14f700 1 -- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f84a405f270 msgr2=0x7f84a4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.045+0000 7f84bd14f700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f84a405f270 0x7f84a4061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f84b80c4100 tx=0x7f84a8008040).stop 2024-01-24T15:14:17.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.045+0000 7f84bd14f700 1 -- 172.21.15.29:0/3229859462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b80c4a10 msgr2=0x7f84b8163a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.045+0000 7f84bd14f700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b80c4a10 0x7f84b8163a50 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f84b0009880 tx=0x7f84b8066140).stop 2024-01-24T15:14:17.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.045+0000 7f84bd14f700 1 -- 172.21.15.29:0/3229859462 shutdown_connections 2024-01-24T15:14:17.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.045+0000 7f84bd14f700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f84a405f270 0x7f84a4061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.045+0000 7f84bd14f700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84b80c4a10 0x7f84b8163a50 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.045+0000 7f84bd14f700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84b8066c00 0x7f84b8155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.045+0000 7f84bd14f700 1 --2- 172.21.15.29:0/3229859462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84b80670a0 0x7f84b80c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.045+0000 7f84bd14f700 1 -- 172.21.15.29:0/3229859462 >> 172.21.15.29:0/3229859462 conn(0x7f84b80c8bb0 msgr2=0x7f84b80c4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:17.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.045+0000 7f84bd14f700 1 -- 172.21.15.29:0/3229859462 shutdown_connections 2024-01-24T15:14:17.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.045+0000 7f84bd14f700 1 -- 172.21.15.29:0/3229859462 wait complete. 2024-01-24T15:14:17.061 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:14:17.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.191+0000 7f965343d700 1 Processor -- start 2024-01-24T15:14:17.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.191+0000 7f965343d700 1 -- start start 2024-01-24T15:14:17.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f965343d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f964c14b6e0 0x7f964c0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f965343d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f964c062ca0 0x7f964c0c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f965343d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964c06a1c0 con 0x7f964c14b300 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f965343d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964c0630d0 con 0x7f964c14b6e0 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f965343d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964c069920 con 0x7f964c062ca0 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f96509d8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f964c14b6e0 0x7f964c0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f96511d9700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f964c14b300 0x7f964c068dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35026/0 (socket says 172.21.15.29:35026) 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f96511d9700 1 -- 172.21.15.29:0/486093155 learned_addr learned my addr 172.21.15.29:0/486093155 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f96519da700 1 --2- 172.21.15.29:0/486093155 >> v2:172.21.15.163:3300/0 conn(0x7f964c062ca0 0x7f964c0c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f96509d8700 1 -- 172.21.15.29:0/486093155 >> v2:172.21.15.163:3300/0 conn(0x7f964c062ca0 msgr2=0x7f964c0c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f96509d8700 1 --2- 172.21.15.29:0/486093155 >> v2:172.21.15.163:3300/0 conn(0x7f964c062ca0 0x7f964c0c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f96509d8700 1 -- 172.21.15.29:0/486093155 >> v1:172.21.15.29:6789/0 conn(0x7f964c14b300 legacy=0x7f964c068dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f963ffff700 1 -- 172.21.15.29:0/486093155 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990733391 0 0) 0x7f964c06a1c0 con 0x7f964c14b300 2024-01-24T15:14:17.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f96509d8700 1 -- 172.21.15.29:0/486093155 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c069c00 con 0x7f964c14b6e0 2024-01-24T15:14:17.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.192+0000 7f96509d8700 1 --2- 172.21.15.29:0/486093155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f964c14b6e0 0x7f964c0627b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f964c14efc0 tx=0x7f9648009060).ready entity=mon.0 client_cookie=72f20664933d04b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:17.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.193+0000 7f963ffff700 1 -- 172.21.15.29:0/486093155 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96480090d0 con 0x7f964c14b6e0 2024-01-24T15:14:17.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.193+0000 7f963ffff700 1 -- 172.21.15.29:0/486093155 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964800b580 con 0x7f964c14b6e0 2024-01-24T15:14:17.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.193+0000 7f963ffff700 1 -- 172.21.15.29:0/486093155 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964800b900 con 0x7f964c14b6e0 2024-01-24T15:14:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.193+0000 7f965343d700 1 -- 172.21.15.29:0/486093155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f964c14b6e0 msgr2=0x7f964c0627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.193+0000 7f965343d700 1 --2- 172.21.15.29:0/486093155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f964c14b6e0 0x7f964c0627b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f964c14efc0 tx=0x7f9648009060).stop 2024-01-24T15:14:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.193+0000 7f965343d700 1 -- 172.21.15.29:0/486093155 shutdown_connections 2024-01-24T15:14:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.193+0000 7f965343d700 1 --2- 172.21.15.29:0/486093155 >> v2:172.21.15.163:3300/0 conn(0x7f964c062ca0 0x7f964c0c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.193+0000 7f965343d700 1 --2- 172.21.15.29:0/486093155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f964c14b6e0 0x7f964c0627b0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.193+0000 7f965343d700 1 -- 172.21.15.29:0/486093155 >> 172.21.15.29:0/486093155 conn(0x7f964c0c69f0 msgr2=0x7f964c0c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.193+0000 7f965343d700 1 -- 172.21.15.29:0/486093155 shutdown_connections 2024-01-24T15:14:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.193+0000 7f965343d700 1 -- 172.21.15.29:0/486093155 wait complete. 2024-01-24T15:14:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.195+0000 7f965343d700 1 Processor -- start 2024-01-24T15:14:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.195+0000 7f965343d700 1 -- start start 2024-01-24T15:14:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f965343d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f964c062ca0 0x7f964c14a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f965343d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f964c14b300 0x7f964c13b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f965343d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f964c14a710 0x7f964c13b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f965343d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c13ba90 con 0x7f964c062ca0 2024-01-24T15:14:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f965343d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c13f3a0 con 0x7f964c14a710 2024-01-24T15:14:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f965343d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c150b70 con 0x7f964c14b300 2024-01-24T15:14:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f96511d9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f964c062ca0 0x7f964c14a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f96511d9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f964c062ca0 0x7f964c14a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39202/0 (socket says 172.21.15.29:39202) 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f96511d9700 1 -- 172.21.15.29:0/477919526 learned_addr learned my addr 172.21.15.29:0/477919526 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f96509d8700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f964c14b300 0x7f964c13b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f96519da700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f964c14a710 0x7f964c13b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f96511d9700 1 -- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f964c14b300 msgr2=0x7f964c13b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f96511d9700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f964c14b300 0x7f964c13b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f96511d9700 1 -- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f964c14a710 msgr2=0x7f964c13b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f96511d9700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f964c14a710 0x7f964c13b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f96511d9700 1 -- 172.21.15.29:0/477919526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96480092a0 con 0x7f964c062ca0 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f96511d9700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f964c062ca0 0x7f964c14a220 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f964000c4e0 tx=0x7f9640007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f963d7fa700 1 -- 172.21.15.29:0/477919526 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96400072e0 con 0x7f964c062ca0 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f963d7fa700 1 -- 172.21.15.29:0/477919526 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9640007440 con 0x7f964c062ca0 2024-01-24T15:14:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.196+0000 7f963d7fa700 1 -- 172.21.15.29:0/477919526 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96400057a0 con 0x7f964c062ca0 2024-01-24T15:14:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.197+0000 7f965343d700 1 -- 172.21.15.29:0/477919526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c150e50 con 0x7f964c062ca0 2024-01-24T15:14:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.197+0000 7f965343d700 1 -- 172.21.15.29:0/477919526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c14fe20 con 0x7f964c062ca0 2024-01-24T15:14:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.198+0000 7f965343d700 1 -- 172.21.15.29:0/477919526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964c0475e0 con 0x7f964c062ca0 2024-01-24T15:14:17.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.198+0000 7f963d7fa700 1 -- 172.21.15.29:0/477919526 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9640005b90 con 0x7f964c062ca0 2024-01-24T15:14:17.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.198+0000 7f963d7fa700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f963005f240 0x7f9630061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:17.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.199+0000 7f963d7fa700 1 -- 172.21.15.29:0/477919526 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f964007a100 con 0x7f964c062ca0 2024-01-24T15:14:17.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.199+0000 7f96509d8700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f963005f240 0x7f9630061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:17.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.199+0000 7f96509d8700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f963005f240 0x7f9630061700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f964c143520 tx=0x7f9648017000).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:17.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.203+0000 7f963d7fa700 1 -- 172.21.15.29:0/477919526 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f964004ae20 con 0x7f964c062ca0 2024-01-24T15:14:17.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.387+0000 7f965343d700 1 -- 172.21.15.29:0/477919526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f964c069ee0 con 0x7f964c062ca0 2024-01-24T15:14:17.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.388+0000 7f963d7fa700 1 -- 172.21.15.29:0/477919526 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+18270 (secure 0 0 0) 0x7f964004e440 con 0x7f964c062ca0 2024-01-24T15:14:17.390 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:17.390 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":192,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:14:16.673237+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:14:14.419558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-24T15:14:15.659901+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:14:17.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.390+0000 7f965343d700 1 -- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f963005f240 msgr2=0x7f9630061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.390+0000 7f965343d700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f963005f240 0x7f9630061700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f964c143520 tx=0x7f9648017000).stop 2024-01-24T15:14:17.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.390+0000 7f965343d700 1 -- 172.21.15.29:0/477919526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f964c062ca0 msgr2=0x7f964c14a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.390+0000 7f965343d700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f964c062ca0 0x7f964c14a220 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f964000c4e0 tx=0x7f9640007c70).stop 2024-01-24T15:14:17.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.390+0000 7f965343d700 1 -- 172.21.15.29:0/477919526 shutdown_connections 2024-01-24T15:14:17.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.390+0000 7f965343d700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f964c14a710 0x7f964c13b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.390+0000 7f965343d700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f963005f240 0x7f9630061700 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.390+0000 7f965343d700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f964c062ca0 0x7f964c14a220 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.390+0000 7f965343d700 1 --2- 172.21.15.29:0/477919526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f964c14b300 0x7f964c13b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.390+0000 7f965343d700 1 -- 172.21.15.29:0/477919526 >> 172.21.15.29:0/477919526 conn(0x7f964c0c69f0 msgr2=0x7f964c0c5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:17.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.390+0000 7f965343d700 1 -- 172.21.15.29:0/477919526 shutdown_connections 2024-01-24T15:14:17.393 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.390+0000 7f965343d700 1 -- 172.21.15.29:0/477919526 wait complete. 2024-01-24T15:14:17.406 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-24T15:14:17.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.537+0000 7fa2e5d49700 1 Processor -- start 2024-01-24T15:14:17.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.538+0000 7fa2e5d49700 1 -- start start 2024-01-24T15:14:17.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.538+0000 7fa2e5d49700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e01544e0 0x7fa2e015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:17.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.538+0000 7fa2e5d49700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e006a110 con 0x7fa2e0069440 2024-01-24T15:14:17.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.538+0000 7fa2e5d49700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e00c3d40 con 0x7fa2e0069820 2024-01-24T15:14:17.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.538+0000 7fa2e5d49700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0066540 con 0x7fa2e01544e0 2024-01-24T15:14:17.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.539+0000 7fa2df7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa2e0069440 0x7fa2e0065e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35036/0 (socket says 172.21.15.29:35036) 2024-01-24T15:14:17.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.539+0000 7fa2df7fe700 1 -- 172.21.15.29:0/905949492 learned_addr learned my addr 172.21.15.29:0/905949492 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:17.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.539+0000 7fa2dffff700 1 --2- 172.21.15.29:0/905949492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e01544e0 0x7fa2e015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.539+0000 7fa2de7fc700 1 -- 172.21.15.29:0/905949492 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740409765 0 0) 0x7fa2e006a110 con 0x7fa2e0069440 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.539+0000 7fa2de7fc700 1 -- 172.21.15.29:0/905949492 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2cc0033f0 con 0x7fa2e0069440 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.539+0000 7fa2de7fc700 1 -- 172.21.15.29:0/905949492 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878805254 0 0) 0x7fa2e00c3d40 con 0x7fa2e0069820 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.539+0000 7fa2de7fc700 1 -- 172.21.15.29:0/905949492 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e006a110 con 0x7fa2e0069820 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.539+0000 7fa2dffff700 1 -- 172.21.15.29:0/905949492 >> v1:172.21.15.163:6789/0 conn(0x7fa2e0069820 legacy=0x7fa2e0153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.539+0000 7fa2dffff700 1 -- 172.21.15.29:0/905949492 >> v1:172.21.15.29:6789/0 conn(0x7fa2e0069440 legacy=0x7fa2e0065e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.539+0000 7fa2dffff700 1 -- 172.21.15.29:0/905949492 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00c3130 con 0x7fa2e01544e0 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.539+0000 7fa2dffff700 1 --2- 172.21.15.29:0/905949492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e01544e0 0x7fa2e015bd70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fa2d8009e90 tx=0x7fa2d8009780).ready entity=mon.2 client_cookie=460435b0e4b224ef server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.539+0000 7fa2de7fc700 1 -- 172.21.15.29:0/905949492 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800e070 con 0x7fa2e01544e0 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.540+0000 7fa2de7fc700 1 -- 172.21.15.29:0/905949492 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d800bab0 con 0x7fa2e01544e0 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.540+0000 7fa2de7fc700 1 -- 172.21.15.29:0/905949492 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800be30 con 0x7fa2e01544e0 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.540+0000 7fa2e5d49700 1 -- 172.21.15.29:0/905949492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e01544e0 msgr2=0x7fa2e015bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.540+0000 7fa2e5d49700 1 --2- 172.21.15.29:0/905949492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e01544e0 0x7fa2e015bd70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fa2d8009e90 tx=0x7fa2d8009780).stop 2024-01-24T15:14:17.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.540+0000 7fa2e5d49700 1 -- 172.21.15.29:0/905949492 shutdown_connections 2024-01-24T15:14:17.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.540+0000 7fa2e5d49700 1 --2- 172.21.15.29:0/905949492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e01544e0 0x7fa2e015bd70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.540+0000 7fa2e5d49700 1 -- 172.21.15.29:0/905949492 >> 172.21.15.29:0/905949492 conn(0x7fa2e0064390 msgr2=0x7fa2e00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:17.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.540+0000 7fa2e5d49700 1 -- 172.21.15.29:0/905949492 shutdown_connections 2024-01-24T15:14:17.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.540+0000 7fa2e5d49700 1 -- 172.21.15.29:0/905949492 wait complete. 2024-01-24T15:14:17.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.542+0000 7fa2e5d49700 1 Processor -- start 2024-01-24T15:14:17.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.542+0000 7fa2e5d49700 1 -- start start 2024-01-24T15:14:17.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.542+0000 7fa2e5d49700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e0069440 0x7fa2e014cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.542+0000 7fa2e5d49700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e0069820 0x7fa2e00bce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.542+0000 7fa2e5d49700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e01544e0 0x7fa2e00bd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.542+0000 7fa2e5d49700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0152670 con 0x7fa2e0069440 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.542+0000 7fa2e5d49700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00bd810 con 0x7fa2e01544e0 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.542+0000 7fa2e5d49700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0164730 con 0x7fa2e0069820 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.542+0000 7fa2df7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e0069440 0x7fa2e014cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2df7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e0069440 0x7fa2e014cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39218/0 (socket says 172.21.15.29:39218) 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2df7fe700 1 -- 172.21.15.29:0/3233787019 learned_addr learned my addr 172.21.15.29:0/3233787019 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2deffd700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e0069820 0x7fa2e00bce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2dffff700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e01544e0 0x7fa2e00bd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2df7fe700 1 -- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e0069820 msgr2=0x7fa2e00bce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2df7fe700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e0069820 0x7fa2e00bce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2df7fe700 1 -- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e01544e0 msgr2=0x7fa2e00bd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:17.547 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2df7fe700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e01544e0 0x7fa2e00bd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:17.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2df7fe700 1 -- 172.21.15.29:0/3233787019 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d8009a20 con 0x7fa2e0069440 2024-01-24T15:14:17.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2dffff700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e01544e0 0x7fa2e00bd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:14:17.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2df7fe700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e0069440 0x7fa2e014cd90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7fa2d000c560 tx=0x7fa2d0007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:17.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2d77fe700 1 -- 172.21.15.29:0/3233787019 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00072e0 con 0x7fa2e0069440 2024-01-24T15:14:17.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2d77fe700 1 -- 172.21.15.29:0/3233787019 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d0007440 con 0x7fa2e0069440 2024-01-24T15:14:17.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2e5d49700 1 -- 172.21.15.29:0/3233787019 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e0164420 con 0x7fa2e0069440 2024-01-24T15:14:17.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2d77fe700 1 -- 172.21.15.29:0/3233787019 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00057a0 con 0x7fa2e0069440 2024-01-24T15:14:17.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.543+0000 7fa2e5d49700 1 -- 172.21.15.29:0/3233787019 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e0152390 con 0x7fa2e0069440 2024-01-24T15:14:17.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.544+0000 7fa2e5d49700 1 -- 172.21.15.29:0/3233787019 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e00bdc70 con 0x7fa2e0069440 2024-01-24T15:14:17.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.545+0000 7fa2d77fe700 1 -- 172.21.15.29:0/3233787019 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa2d0005b90 con 0x7fa2e0069440 2024-01-24T15:14:17.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.545+0000 7fa2d77fe700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa2c005f270 0x7fa2c0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:17.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.545+0000 7fa2d77fe700 1 -- 172.21.15.29:0/3233787019 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fa2d0079e40 con 0x7fa2e0069440 2024-01-24T15:14:17.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.545+0000 7fa2deffd700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa2c005f270 0x7fa2c0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:17.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.546+0000 7fa2deffd700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa2c005f270 0x7fa2c0061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fa2e00ad190 tx=0x7fa2c8007bd0).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:17.551 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.550+0000 7fa2d77fe700 1 -- 172.21.15.29:0/3233787019 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa2d004ab60 con 0x7fa2e0069440 2024-01-24T15:14:17.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:17.779+0000 7fa2e5d49700 1 -- 172.21.15.29:0/3233787019 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa2e00475e0 con 0x7fa2e0069440 2024-01-24T15:14:18.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.687+0000 7fa2d77fe700 1 -- 172.21.15.29:0/3233787019 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa2d004e180 con 0x7fa2e0069440 2024-01-24T15:14:18.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.690+0000 7fa2e5d49700 1 -- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa2c005f270 msgr2=0x7fa2c0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:18.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.690+0000 7fa2e5d49700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa2c005f270 0x7fa2c0061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fa2e00ad190 tx=0x7fa2c8007bd0).stop 2024-01-24T15:14:18.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.690+0000 7fa2e5d49700 1 -- 172.21.15.29:0/3233787019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e0069440 msgr2=0x7fa2e014cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:18.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.690+0000 7fa2e5d49700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e0069440 0x7fa2e014cd90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7fa2d000c560 tx=0x7fa2d0007d20).stop 2024-01-24T15:14:18.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.690+0000 7fa2e5d49700 1 -- 172.21.15.29:0/3233787019 shutdown_connections 2024-01-24T15:14:18.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.690+0000 7fa2e5d49700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e01544e0 0x7fa2e00bd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:18.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.690+0000 7fa2e5d49700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fa2c005f270 0x7fa2c0061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:18.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.690+0000 7fa2e5d49700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e0069440 0x7fa2e014cd90 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:18.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.690+0000 7fa2e5d49700 1 --2- 172.21.15.29:0/3233787019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e0069820 0x7fa2e00bce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:18.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.690+0000 7fa2e5d49700 1 -- 172.21.15.29:0/3233787019 >> 172.21.15.29:0/3233787019 conn(0x7fa2e0064390 msgr2=0x7fa2e014ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:18.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.690+0000 7fa2e5d49700 1 -- 172.21.15.29:0/3233787019 shutdown_connections 2024-01-24T15:14:18.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.690+0000 7fa2e5d49700 1 -- 172.21.15.29:0/3233787019 wait complete. 2024-01-24T15:14:18.708 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-24T15:14:18.708 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-24T15:14:18.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.843+0000 7ff39f0ea700 1 Processor -- start 2024-01-24T15:14:18.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.843+0000 7ff39f0ea700 1 -- start start 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.844+0000 7ff39f0ea700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff398153f40 0x7ff3980a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.844+0000 7ff39f0ea700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff398062ec0 0x7ff398065440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.844+0000 7ff39f0ea700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff39806a360 con 0x7ff3980c8290 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.844+0000 7ff39f0ea700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3980bcef0 con 0x7ff398153f40 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.844+0000 7ff39f0ea700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff398069de0 con 0x7ff398062ec0 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.844+0000 7ff39ce86700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff398153f40 0x7ff3980a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.844+0000 7ff39ce86700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff398153f40 0x7ff3980a6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39230/0 (socket says 172.21.15.29:39230) 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.844+0000 7ff39ce86700 1 -- 172.21.15.29:0/4294127050 learned_addr learned my addr 172.21.15.29:0/4294127050 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.844+0000 7ff39d687700 1 --2- 172.21.15.29:0/4294127050 >> v2:172.21.15.163:3300/0 conn(0x7ff398062ec0 0x7ff398065440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.845+0000 7ff39ce86700 1 -- 172.21.15.29:0/4294127050 >> v2:172.21.15.163:3300/0 conn(0x7ff398062ec0 msgr2=0x7ff398065440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.845+0000 7ff39ce86700 1 --2- 172.21.15.29:0/4294127050 >> v2:172.21.15.163:3300/0 conn(0x7ff398062ec0 0x7ff398065440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.845+0000 7ff39ce86700 1 -- 172.21.15.29:0/4294127050 >> v1:172.21.15.163:6789/0 conn(0x7ff3980c8290 legacy=0x7ff3980627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.845+0000 7ff39ce86700 1 -- 172.21.15.29:0/4294127050 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39806a080 con 0x7ff398153f40 2024-01-24T15:14:18.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.845+0000 7ff39ce86700 1 --2- 172.21.15.29:0/4294127050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff398153f40 0x7ff3980a6db0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7ff3980bce60 tx=0x7ff38c0090f0).ready entity=mon.0 client_cookie=160c6515b74fd2a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:18.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.845+0000 7ff3977fe700 1 -- 172.21.15.29:0/4294127050 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00b520 con 0x7ff398153f40 2024-01-24T15:14:18.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.845+0000 7ff3977fe700 1 -- 172.21.15.29:0/4294127050 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38c00b680 con 0x7ff398153f40 2024-01-24T15:14:18.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.845+0000 7ff3977fe700 1 -- 172.21.15.29:0/4294127050 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00ba20 con 0x7ff398153f40 2024-01-24T15:14:18.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.845+0000 7ff39f0ea700 1 -- 172.21.15.29:0/4294127050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff398153f40 msgr2=0x7ff3980a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:18.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.845+0000 7ff39f0ea700 1 --2- 172.21.15.29:0/4294127050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff398153f40 0x7ff3980a6db0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7ff3980bce60 tx=0x7ff38c0090f0).stop 2024-01-24T15:14:18.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.846+0000 7ff39f0ea700 1 -- 172.21.15.29:0/4294127050 shutdown_connections 2024-01-24T15:14:18.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.846+0000 7ff39f0ea700 1 --2- 172.21.15.29:0/4294127050 >> v2:172.21.15.163:3300/0 conn(0x7ff398062ec0 0x7ff398065440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:18.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.846+0000 7ff39f0ea700 1 --2- 172.21.15.29:0/4294127050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff398153f40 0x7ff3980a6db0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:18.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.846+0000 7ff39f0ea700 1 -- 172.21.15.29:0/4294127050 >> 172.21.15.29:0/4294127050 conn(0x7ff398064390 msgr2=0x7ff3980c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:18.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.846+0000 7ff39f0ea700 1 -- 172.21.15.29:0/4294127050 shutdown_connections 2024-01-24T15:14:18.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.846+0000 7ff39f0ea700 1 -- 172.21.15.29:0/4294127050 wait complete. 2024-01-24T15:14:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.847+0000 7ff39f0ea700 1 Processor -- start 2024-01-24T15:14:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.847+0000 7ff39f0ea700 1 -- start start 2024-01-24T15:14:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff39f0ea700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff398062ec0 0x7ff398150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff39f0ea700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3980c8290 0x7ff3980ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff39f0ea700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff398153f40 0x7ff398151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff39f0ea700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff398151a40 con 0x7ff3980c8290 2024-01-24T15:14:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff39f0ea700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff398150e30 con 0x7ff398153f40 2024-01-24T15:14:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff39f0ea700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff398151110 con 0x7ff398062ec0 2024-01-24T15:14:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff397fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3980c8290 0x7ff3980ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff397fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3980c8290 0x7ff3980ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39240/0 (socket says 172.21.15.29:39240) 2024-01-24T15:14:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff397fff700 1 -- 172.21.15.29:0/1825358985 learned_addr learned my addr 172.21.15.29:0/1825358985 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:18.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff39ce86700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff398062ec0 0x7ff398150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff39d687700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff398153f40 0x7ff398151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff397fff700 1 -- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff398062ec0 msgr2=0x7ff398150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff397fff700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff398062ec0 0x7ff398150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff397fff700 1 -- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff398153f40 msgr2=0x7ff398151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff397fff700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff398153f40 0x7ff398151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.848+0000 7ff397fff700 1 -- 172.21.15.29:0/1825358985 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c009370 con 0x7ff3980c8290 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.849+0000 7ff397fff700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3980c8290 0x7ff3980ad230 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7ff38400c410 tx=0x7ff384007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.849+0000 7ff394ff9700 1 -- 172.21.15.29:0/1825358985 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38400ce70 con 0x7ff3980c8290 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.849+0000 7ff394ff9700 1 -- 172.21.15.29:0/1825358985 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38400f470 con 0x7ff3980c8290 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.849+0000 7ff394ff9700 1 -- 172.21.15.29:0/1825358985 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38400f810 con 0x7ff3980c8290 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.849+0000 7ff39f0ea700 1 -- 172.21.15.29:0/1825358985 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3981513f0 con 0x7ff3980c8290 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.849+0000 7ff39f0ea700 1 -- 172.21.15.29:0/1825358985 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff398150940 con 0x7ff3980c8290 2024-01-24T15:14:18.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.850+0000 7ff39f0ea700 1 -- 172.21.15.29:0/1825358985 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff398151bf0 con 0x7ff3980c8290 2024-01-24T15:14:18.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.850+0000 7ff394ff9700 1 -- 172.21.15.29:0/1825358985 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff38400fc00 con 0x7ff3980c8290 2024-01-24T15:14:18.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.851+0000 7ff394ff9700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff37c05f240 0x7ff37c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:18.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.851+0000 7ff394ff9700 1 -- 172.21.15.29:0/1825358985 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7ff384079bd0 con 0x7ff3980c8290 2024-01-24T15:14:18.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.851+0000 7ff39ce86700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff37c05f240 0x7ff37c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:18.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.851+0000 7ff39ce86700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff37c05f240 0x7ff37c061700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ff38c00b2e0 tx=0x7ff38c017000).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:18.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:18.855+0000 7ff394ff9700 1 -- 172.21.15.29:0/1825358985 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff384048b00 con 0x7ff3980c8290 2024-01-24T15:14:19.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.082+0000 7ff39f0ea700 1 -- 172.21.15.29:0/1825358985 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff3981515e0 con 0x7ff3980c8290 2024-01-24T15:14:19.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.692+0000 7ff394ff9700 1 -- 172.21.15.29:0/1825358985 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff384017070 con 0x7ff3980c8290 2024-01-24T15:14:19.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.695+0000 7ff39f0ea700 1 -- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff37c05f240 msgr2=0x7ff37c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:19.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.695+0000 7ff39f0ea700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff37c05f240 0x7ff37c061700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ff38c00b2e0 tx=0x7ff38c017000).stop 2024-01-24T15:14:19.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.695+0000 7ff39f0ea700 1 -- 172.21.15.29:0/1825358985 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3980c8290 msgr2=0x7ff3980ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:19.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.695+0000 7ff39f0ea700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3980c8290 0x7ff3980ad230 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7ff38400c410 tx=0x7ff384007d90).stop 2024-01-24T15:14:19.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.695+0000 7ff39f0ea700 1 -- 172.21.15.29:0/1825358985 shutdown_connections 2024-01-24T15:14:19.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.695+0000 7ff39f0ea700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff398153f40 0x7ff398151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:19.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.695+0000 7ff39f0ea700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff398062ec0 0x7ff398150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:19.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.695+0000 7ff39f0ea700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7ff37c05f240 0x7ff37c061700 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:19.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.695+0000 7ff39f0ea700 1 --2- 172.21.15.29:0/1825358985 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3980c8290 0x7ff3980ad230 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:19.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.695+0000 7ff39f0ea700 1 -- 172.21.15.29:0/1825358985 >> 172.21.15.29:0/1825358985 conn(0x7ff398064390 msgr2=0x7ff398065350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:19.777 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.695+0000 7ff39f0ea700 1 -- 172.21.15.29:0/1825358985 shutdown_connections 2024-01-24T15:14:19.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.695+0000 7ff39f0ea700 1 -- 172.21.15.29:0/1825358985 wait complete. 2024-01-24T15:14:19.778 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-24T15:14:19.778 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:14:19.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.919+0000 7efe01ff3700 1 Processor -- start 2024-01-24T15:14:19.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.919+0000 7efe01ff3700 1 -- start start 2024-01-24T15:14:19.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.920+0000 7efe01ff3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdfc062ec0 0x7efdfc0c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:19.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.920+0000 7efe01ff3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efdfc1550f0 0x7efdfc069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:19.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.920+0000 7efe01ff3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdfc065360 con 0x7efdfc1540a0 2024-01-24T15:14:19.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.920+0000 7efe01ff3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc0a74f0 con 0x7efdfc1550f0 2024-01-24T15:14:19.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.920+0000 7efe01ff3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0a6ce0 con 0x7efdfc062ec0 2024-01-24T15:14:19.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.920+0000 7efdfaffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdfc062ec0 0x7efdfc0c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:19.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.920+0000 7efdfbfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efdfc1550f0 0x7efdfc069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:19.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.920+0000 7efdfaffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdfc062ec0 0x7efdfc0c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:54982/0 (socket says 172.21.15.29:54982) 2024-01-24T15:14:19.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.920+0000 7efdfb7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7efdfc1540a0 0x7efdfc0627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:39622/0 (socket says 172.21.15.29:39622) 2024-01-24T15:14:19.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.920+0000 7efdfaffd700 1 -- 172.21.15.29:0/463952230 learned_addr learned my addr 172.21.15.29:0/463952230 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:19.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.921+0000 7efdfbfff700 1 -- 172.21.15.29:0/463952230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdfc062ec0 msgr2=0x7efdfc0c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:19.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.921+0000 7efdfbfff700 1 --2- 172.21.15.29:0/463952230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdfc062ec0 0x7efdfc0c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:19.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.921+0000 7efdfbfff700 1 -- 172.21.15.29:0/463952230 >> v1:172.21.15.163:6789/0 conn(0x7efdfc1540a0 legacy=0x7efdfc0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:19.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.921+0000 7efdfbfff700 1 -- 172.21.15.29:0/463952230 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0636f0 con 0x7efdfc1550f0 2024-01-24T15:14:19.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.921+0000 7efdfaffd700 1 --2- 172.21.15.29:0/463952230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdfc062ec0 0x7efdfc0c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:19.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.921+0000 7efdfbfff700 1 --2- 172.21.15.29:0/463952230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc1550f0 0x7efdfc069240 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7efdf4009e60 tx=0x7efdf4009780).ready entity=mon.1 client_cookie=12fdf351f0eba0d6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:19.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.921+0000 7efdfa7fc700 1 -- 172.21.15.29:0/463952230 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400e070 con 0x7efdfc1550f0 2024-01-24T15:14:19.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.921+0000 7efdfa7fc700 1 -- 172.21.15.29:0/463952230 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf400bb60 con 0x7efdfc1550f0 2024-01-24T15:14:19.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.921+0000 7efdfa7fc700 1 -- 172.21.15.29:0/463952230 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4012410 con 0x7efdfc1550f0 2024-01-24T15:14:19.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.921+0000 7efe01ff3700 1 -- 172.21.15.29:0/463952230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc1550f0 msgr2=0x7efdfc069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:19.933 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.921+0000 7efe01ff3700 1 --2- 172.21.15.29:0/463952230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc1550f0 0x7efdfc069240 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7efdf4009e60 tx=0x7efdf4009780).stop 2024-01-24T15:14:19.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.922+0000 7efe01ff3700 1 -- 172.21.15.29:0/463952230 shutdown_connections 2024-01-24T15:14:19.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.922+0000 7efe01ff3700 1 --2- 172.21.15.29:0/463952230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc1550f0 0x7efdfc069240 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:19.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.922+0000 7efe01ff3700 1 --2- 172.21.15.29:0/463952230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdfc062ec0 0x7efdfc0c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:19.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.922+0000 7efe01ff3700 1 -- 172.21.15.29:0/463952230 >> 172.21.15.29:0/463952230 conn(0x7efdfc0c85c0 msgr2=0x7efdfc0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:19.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.922+0000 7efe01ff3700 1 -- 172.21.15.29:0/463952230 shutdown_connections 2024-01-24T15:14:19.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.922+0000 7efe01ff3700 1 -- 172.21.15.29:0/463952230 wait complete. 2024-01-24T15:14:19.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.923+0000 7efe01ff3700 1 Processor -- start 2024-01-24T15:14:19.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.923+0000 7efe01ff3700 1 -- start start 2024-01-24T15:14:19.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efe01ff3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdfc062ec0 0x7efdfc0c1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:19.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efe01ff3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdfc1540a0 0x7efdfc0c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:19.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efe01ff3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc1550f0 0x7efdfc14c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:19.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efe01ff3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0c0f80 con 0x7efdfc062ec0 2024-01-24T15:14:19.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efe01ff3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0c3590 con 0x7efdfc1550f0 2024-01-24T15:14:19.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efe01ff3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0bef70 con 0x7efdfc1540a0 2024-01-24T15:14:19.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdfc062ec0 0x7efdfc0c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:19.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdfc062ec0 0x7efdfc0c1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39256/0 (socket says 172.21.15.29:39256) 2024-01-24T15:14:19.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efdfb7fe700 1 -- 172.21.15.29:0/1161021940 learned_addr learned my addr 172.21.15.29:0/1161021940 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:19.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efdfbfff700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc1550f0 0x7efdfc14c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:19.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efdfaffd700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdfc1540a0 0x7efdfc0c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:19.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efdfb7fe700 1 -- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdfc1540a0 msgr2=0x7efdfc0c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:19.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efdfb7fe700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdfc1540a0 0x7efdfc0c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:19.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efdfb7fe700 1 -- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc1550f0 msgr2=0x7efdfc14c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:19.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efdfb7fe700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc1550f0 0x7efdfc14c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:19.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.924+0000 7efdfb7fe700 1 -- 172.21.15.29:0/1161021940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf0009370 con 0x7efdfc062ec0 2024-01-24T15:14:19.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.925+0000 7efdfb7fe700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdfc062ec0 0x7efdfc0c1410 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7efdec007fd0 tx=0x7efdec007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:19.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.925+0000 7efde37fe700 1 -- 172.21.15.29:0/1161021940 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0072e0 con 0x7efdfc062ec0 2024-01-24T15:14:19.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.925+0000 7efde37fe700 1 -- 172.21.15.29:0/1161021940 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec007440 con 0x7efdfc062ec0 2024-01-24T15:14:19.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.925+0000 7efe01ff3700 1 -- 172.21.15.29:0/1161021940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf4009a20 con 0x7efdfc062ec0 2024-01-24T15:14:19.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.925+0000 7efde37fe700 1 -- 172.21.15.29:0/1161021940 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0058f0 con 0x7efdfc062ec0 2024-01-24T15:14:19.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.925+0000 7efe01ff3700 1 -- 172.21.15.29:0/1161021940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc0c0c10 con 0x7efdfc062ec0 2024-01-24T15:14:19.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.926+0000 7efe01ff3700 1 -- 172.21.15.29:0/1161021940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc0475e0 con 0x7efdfc062ec0 2024-01-24T15:14:19.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.926+0000 7efde37fe700 1 -- 172.21.15.29:0/1161021940 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7efdec005ce0 con 0x7efdfc062ec0 2024-01-24T15:14:19.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.927+0000 7efde37fe700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efde805f270 0x7efde8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:19.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.927+0000 7efde37fe700 1 -- 172.21.15.29:0/1161021940 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7efdec079fa0 con 0x7efdfc062ec0 2024-01-24T15:14:19.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.927+0000 7efdfaffd700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efde805f270 0x7efde8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:19.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.927+0000 7efdfaffd700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efde805f270 0x7efde8061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7efdf0009340 tx=0x7efdf0008040).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:19.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:19.931+0000 7efde37fe700 1 -- 172.21.15.29:0/1161021940 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7efdec04acc0 con 0x7efdfc062ec0 2024-01-24T15:14:20.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.157+0000 7efe01ff3700 1 -- 172.21.15.29:0/1161021940 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdfc062c70 con 0x7efdfc062ec0 2024-01-24T15:14:20.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.158+0000 7efde37fe700 1 -- 172.21.15.29:0/1161021940 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4496 (secure 0 0 0) 0x7efdec04e2e0 con 0x7efdfc062ec0 2024-01-24T15:14:20.160 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:20.160 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1350237946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1350237946},{"type":"v1","addr":"172.21.15.163:6836","nonce":1350237946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8656,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/250387328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":250387328},{"type":"v1","addr":"172.21.15.163:6837","nonce":250387328}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":9069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3867710046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3867710046},{"type":"v1","addr":"172.21.15.29:6833","nonce":3867710046}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:14:16.673720+0000","modified":"2024-01-24T15:14:19.686795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9072":{"gid":9072,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1929837915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1929837915},{"type":"v1","addr":"172.21.15.29:6835","nonce":1929837915}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-24T15:14:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.160+0000 7efe01ff3700 1 -- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efde805f270 msgr2=0x7efde8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.160+0000 7efe01ff3700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efde805f270 0x7efde8061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7efdf0009340 tx=0x7efdf0008040).stop 2024-01-24T15:14:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.160+0000 7efe01ff3700 1 -- 172.21.15.29:0/1161021940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdfc062ec0 msgr2=0x7efdfc0c1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.160+0000 7efe01ff3700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdfc062ec0 0x7efdfc0c1410 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7efdec007fd0 tx=0x7efdec007df0).stop 2024-01-24T15:14:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.160+0000 7efe01ff3700 1 -- 172.21.15.29:0/1161021940 shutdown_connections 2024-01-24T15:14:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.160+0000 7efe01ff3700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc1550f0 0x7efdfc14c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.160+0000 7efe01ff3700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7efde805f270 0x7efde8061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.160+0000 7efe01ff3700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdfc062ec0 0x7efdfc0c1410 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.160+0000 7efe01ff3700 1 --2- 172.21.15.29:0/1161021940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdfc1540a0 0x7efdfc0c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.160+0000 7efe01ff3700 1 -- 172.21.15.29:0/1161021940 >> 172.21.15.29:0/1161021940 conn(0x7efdfc0c85c0 msgr2=0x7efdfc0c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:20.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.160+0000 7efe01ff3700 1 -- 172.21.15.29:0/1161021940 shutdown_connections 2024-01-24T15:14:20.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.160+0000 7efe01ff3700 1 -- 172.21.15.29:0/1161021940 wait complete. 2024-01-24T15:14:20.163 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 82 2024-01-24T15:14:20.175 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:14:20.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.302+0000 7f719072f700 1 Processor -- start 2024-01-24T15:14:20.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.303+0000 7f719072f700 1 -- start start 2024-01-24T15:14:20.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f719072f700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7188150d40 0x7f7188069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:20.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f719072f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7188155040 0x7f71880c3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:20.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f719072f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71880c2ce0 con 0x7f7188065ec0 2024-01-24T15:14:20.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f719072f700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7188069890 con 0x7f7188150d40 2024-01-24T15:14:20.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f719072f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7188069b70 con 0x7f7188155040 2024-01-24T15:14:20.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f718e4cb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7188150d40 0x7f7188069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:20.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f718e4cb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7188150d40 0x7f7188069240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39260/0 (socket says 172.21.15.29:39260) 2024-01-24T15:14:20.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f718e4cb700 1 -- 172.21.15.29:0/3216589865 learned_addr learned my addr 172.21.15.29:0/3216589865 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:20.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f718eccc700 1 --2- 172.21.15.29:0/3216589865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7188155040 0x7f71880c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:20.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f718e4cb700 1 -- 172.21.15.29:0/3216589865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7188155040 msgr2=0x7f71880c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:20.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f718e4cb700 1 --2- 172.21.15.29:0/3216589865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7188155040 0x7f71880c3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f718e4cb700 1 -- 172.21.15.29:0/3216589865 >> v1:172.21.15.163:6789/0 conn(0x7f7188065ec0 legacy=0x7f7188154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f718e4cb700 1 -- 172.21.15.29:0/3216589865 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7188069e50 con 0x7f7188150d40 2024-01-24T15:14:20.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.304+0000 7f718e4cb700 1 --2- 172.21.15.29:0/3216589865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7188150d40 0x7f7188069240 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f7178006fd0 tx=0x7f717800b3b0).ready entity=mon.0 client_cookie=7b56e7e59fcb813f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:20.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.305+0000 7f718d4c9700 1 -- 172.21.15.29:0/3216589865 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800da70 con 0x7f7188150d40 2024-01-24T15:14:20.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.305+0000 7f718d4c9700 1 -- 172.21.15.29:0/3216589865 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717800dbd0 con 0x7f7188150d40 2024-01-24T15:14:20.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.305+0000 7f718d4c9700 1 -- 172.21.15.29:0/3216589865 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178010470 con 0x7f7188150d40 2024-01-24T15:14:20.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.305+0000 7f719072f700 1 -- 172.21.15.29:0/3216589865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7188150d40 msgr2=0x7f7188069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.305+0000 7f719072f700 1 --2- 172.21.15.29:0/3216589865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7188150d40 0x7f7188069240 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f7178006fd0 tx=0x7f717800b3b0).stop 2024-01-24T15:14:20.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.305+0000 7f719072f700 1 -- 172.21.15.29:0/3216589865 shutdown_connections 2024-01-24T15:14:20.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.305+0000 7f719072f700 1 --2- 172.21.15.29:0/3216589865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7188155040 0x7f71880c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.305+0000 7f719072f700 1 --2- 172.21.15.29:0/3216589865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7188150d40 0x7f7188069240 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.305+0000 7f719072f700 1 -- 172.21.15.29:0/3216589865 >> 172.21.15.29:0/3216589865 conn(0x7f71880c8580 msgr2=0x7f71880c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:20.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.305+0000 7f719072f700 1 -- 172.21.15.29:0/3216589865 shutdown_connections 2024-01-24T15:14:20.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.305+0000 7f719072f700 1 -- 172.21.15.29:0/3216589865 wait complete. 2024-01-24T15:14:20.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.307+0000 7f719072f700 1 Processor -- start 2024-01-24T15:14:20.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.307+0000 7f719072f700 1 -- start start 2024-01-24T15:14:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.307+0000 7f719072f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7188065ec0 0x7f71880aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.307+0000 7f719072f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7188150d40 0x7f718814d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.307+0000 7f719072f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7188155040 0x7f71881494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.307+0000 7f719072f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71880ab350 con 0x7f7188155040 2024-01-24T15:14:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.307+0000 7f719072f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71880ab5a0 con 0x7f7188065ec0 2024-01-24T15:14:20.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.307+0000 7f719072f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71880ab880 con 0x7f7188150d40 2024-01-24T15:14:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f718eccc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7188155040 0x7f71881494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f718eccc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7188155040 0x7f71881494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39264/0 (socket says 172.21.15.29:39264) 2024-01-24T15:14:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f718eccc700 1 -- 172.21.15.29:0/2245811486 learned_addr learned my addr 172.21.15.29:0/2245811486 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f718dcca700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7188150d40 0x7f718814d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f718e4cb700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7188065ec0 0x7f71880aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f718eccc700 1 -- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7188150d40 msgr2=0x7f718814d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f718eccc700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7188150d40 0x7f718814d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f718eccc700 1 -- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7188065ec0 msgr2=0x7f71880aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f718eccc700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7188065ec0 0x7f71880aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f718eccc700 1 -- 172.21.15.29:0/2245811486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717800b610 con 0x7f7188155040 2024-01-24T15:14:20.321 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f718eccc700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7188155040 0x7f71881494a0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f7180009d70 tx=0x7f718000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f717e7fc700 1 -- 172.21.15.29:0/2245811486 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718000caa0 con 0x7f7188155040 2024-01-24T15:14:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f717e7fc700 1 -- 172.21.15.29:0/2245811486 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718000f040 con 0x7f7188155040 2024-01-24T15:14:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f719072f700 1 -- 172.21.15.29:0/2245811486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718814c880 con 0x7f7188155040 2024-01-24T15:14:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f717e7fc700 1 -- 172.21.15.29:0/2245811486 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718000ce10 con 0x7f7188155040 2024-01-24T15:14:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.308+0000 7f719072f700 1 -- 172.21.15.29:0/2245811486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718814ce40 con 0x7f7188155040 2024-01-24T15:14:20.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.309+0000 7f719072f700 1 -- 172.21.15.29:0/2245811486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71880475e0 con 0x7f7188155040 2024-01-24T15:14:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.310+0000 7f717e7fc700 1 -- 172.21.15.29:0/2245811486 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7180013610 con 0x7f7188155040 2024-01-24T15:14:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.310+0000 7f717e7fc700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f716c05f270 0x7f716c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.310+0000 7f717e7fc700 1 -- 172.21.15.29:0/2245811486 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f718007aea0 con 0x7f7188155040 2024-01-24T15:14:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.311+0000 7f718e4cb700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f716c05f270 0x7f716c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.311+0000 7f718e4cb700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f716c05f270 0x7f716c061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f717800bfd0 tx=0x7f717800b3b0).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.315+0000 7f717e7fc700 1 -- 172.21.15.29:0/2245811486 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f718004bbc0 con 0x7f7188155040 2024-01-24T15:14:20.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.496+0000 7f719072f700 1 -- 172.21.15.29:0/2245811486 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71880c51a0 con 0x7f7188155040 2024-01-24T15:14:20.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.497+0000 7f717e7fc700 1 -- 172.21.15.29:0/2245811486 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+18257 (secure 0 0 0) 0x7f718004f1e0 con 0x7f7188155040 2024-01-24T15:14:20.500 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:20.500 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":193,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:14:17.675645+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:14:14.419558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-24T15:14:15.659901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:14:20.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.499+0000 7f719072f700 1 -- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f716c05f270 msgr2=0x7f716c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.499+0000 7f719072f700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f716c05f270 0x7f716c061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f717800bfd0 tx=0x7f717800b3b0).stop 2024-01-24T15:14:20.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.499+0000 7f719072f700 1 -- 172.21.15.29:0/2245811486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7188155040 msgr2=0x7f71881494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.499+0000 7f719072f700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7188155040 0x7f71881494a0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f7180009d70 tx=0x7f718000b040).stop 2024-01-24T15:14:20.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.499+0000 7f719072f700 1 -- 172.21.15.29:0/2245811486 shutdown_connections 2024-01-24T15:14:20.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.499+0000 7f719072f700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f716c05f270 0x7f716c061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.499+0000 7f719072f700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7188155040 0x7f71881494a0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.499+0000 7f719072f700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7188065ec0 0x7f71880aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.499+0000 7f719072f700 1 --2- 172.21.15.29:0/2245811486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7188150d40 0x7f718814d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.499+0000 7f719072f700 1 -- 172.21.15.29:0/2245811486 >> 172.21.15.29:0/2245811486 conn(0x7f71880c8580 msgr2=0x7f718814ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:20.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.499+0000 7f719072f700 1 -- 172.21.15.29:0/2245811486 shutdown_connections 2024-01-24T15:14:20.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.499+0000 7f719072f700 1 -- 172.21.15.29:0/2245811486 wait complete. 2024-01-24T15:14:20.515 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-24T15:14:20.515 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:14:20.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.645+0000 7f90a0a3e700 1 Processor -- start 2024-01-24T15:14:20.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.645+0000 7f90a0a3e700 1 -- start start 2024-01-24T15:14:20.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.646+0000 7f90a0a3e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f909c069240 0x7f909c065ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:20.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.646+0000 7f90a0a3e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c0c2ce0 con 0x7f909c155040 2024-01-24T15:14:20.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f90a0a3e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c0663b0 con 0x7f909c150d40 2024-01-24T15:14:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f90a0a3e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c069890 con 0x7f909c069240 2024-01-24T15:14:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f909ad9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f909c069240 0x7f909c065ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f9099d9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f909c155040 0x7f909c0c3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35040/0 (socket says 172.21.15.29:35040) 2024-01-24T15:14:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f9099d9b700 1 -- 172.21.15.29:0/429308684 learned_addr learned my addr 172.21.15.29:0/429308684 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f909ad9d700 1 -- 172.21.15.29:0/429308684 >> v1:172.21.15.163:6789/0 conn(0x7f909c150d40 legacy=0x7f909c154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f909ad9d700 1 -- 172.21.15.29:0/429308684 >> v1:172.21.15.29:6789/0 conn(0x7f909c155040 legacy=0x7f909c0c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f909ad9d700 1 -- 172.21.15.29:0/429308684 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c069b70 con 0x7f909c069240 2024-01-24T15:14:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f909959a700 1 -- 172.21.15.29:0/429308684 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531408769 0 0) 0x7f909c0c2ce0 con 0x7f909c155040 2024-01-24T15:14:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f909ad9d700 1 --2- 172.21.15.29:0/429308684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c069240 0x7f909c065ec0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f9094009e60 tx=0x7f9094009780).ready entity=mon.0 client_cookie=de3d1c0183025162 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f909959a700 1 -- 172.21.15.29:0/429308684 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400e070 con 0x7f909c069240 2024-01-24T15:14:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f909959a700 1 -- 172.21.15.29:0/429308684 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909400bbf0 con 0x7f909c069240 2024-01-24T15:14:20.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.647+0000 7f909959a700 1 -- 172.21.15.29:0/429308684 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094012410 con 0x7f909c069240 2024-01-24T15:14:20.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.648+0000 7f90a0a3e700 1 -- 172.21.15.29:0/429308684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c069240 msgr2=0x7f909c065ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.648+0000 7f90a0a3e700 1 --2- 172.21.15.29:0/429308684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c069240 0x7f909c065ec0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f9094009e60 tx=0x7f9094009780).stop 2024-01-24T15:14:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.648+0000 7f90a0a3e700 1 -- 172.21.15.29:0/429308684 shutdown_connections 2024-01-24T15:14:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.648+0000 7f90a0a3e700 1 --2- 172.21.15.29:0/429308684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c069240 0x7f909c065ec0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.648+0000 7f90a0a3e700 1 -- 172.21.15.29:0/429308684 >> 172.21.15.29:0/429308684 conn(0x7f909c0c8580 msgr2=0x7f909c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.648+0000 7f90a0a3e700 1 -- 172.21.15.29:0/429308684 shutdown_connections 2024-01-24T15:14:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.648+0000 7f90a0a3e700 1 -- 172.21.15.29:0/429308684 wait complete. 2024-01-24T15:14:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.649+0000 7f90a0a3e700 1 Processor -- start 2024-01-24T15:14:20.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.649+0000 7f90a0a3e700 1 -- start start 2024-01-24T15:14:20.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.650+0000 7f90a0a3e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f909c069240 0x7f909c0ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:20.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.650+0000 7f90a0a3e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f909c150d40 0x7f909c14e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:20.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.650+0000 7f90a0a3e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c155040 0x7f909c0c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.650+0000 7f90a0a3e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c14fb80 con 0x7f909c155040 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.650+0000 7f90a0a3e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c14ef70 con 0x7f909c150d40 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.650+0000 7f90a0a3e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c14f250 con 0x7f909c069240 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.650+0000 7f909ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c155040 0x7f909c0c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.650+0000 7f909ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c155040 0x7f909c0c0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39278/0 (socket says 172.21.15.29:39278) 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.650+0000 7f909ad9d700 1 -- 172.21.15.29:0/3687055690 learned_addr learned my addr 172.21.15.29:0/3687055690 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f9099d9b700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f909c150d40 0x7f909c14e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f909a59c700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f909c069240 0x7f909c0ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f909ad9d700 1 -- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f909c069240 msgr2=0x7f909c0ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f909ad9d700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f909c069240 0x7f909c0ad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f909ad9d700 1 -- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f909c150d40 msgr2=0x7f909c14e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f909ad9d700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f909c150d40 0x7f909c14e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f909ad9d700 1 -- 172.21.15.29:0/3687055690 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094009a20 con 0x7f909c155040 2024-01-24T15:14:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f9099d9b700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f909c150d40 0x7f909c14e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:14:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f909ad9d700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c155040 0x7f909c0c0da0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7f909c069030 tx=0x7f9094009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f908a7fc700 1 -- 172.21.15.29:0/3687055690 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90940117d0 con 0x7f909c155040 2024-01-24T15:14:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f908a7fc700 1 -- 172.21.15.29:0/3687055690 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9094011930 con 0x7f909c155040 2024-01-24T15:14:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f908a7fc700 1 -- 172.21.15.29:0/3687055690 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094011cf0 con 0x7f909c155040 2024-01-24T15:14:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f90a0a3e700 1 -- 172.21.15.29:0/3687055690 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909c14f530 con 0x7f909c155040 2024-01-24T15:14:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.651+0000 7f90a0a3e700 1 -- 172.21.15.29:0/3687055690 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c0ada30 con 0x7f909c155040 2024-01-24T15:14:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.652+0000 7f90a0a3e700 1 -- 172.21.15.29:0/3687055690 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909c14fcf0 con 0x7f909c155040 2024-01-24T15:14:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.653+0000 7f908a7fc700 1 -- 172.21.15.29:0/3687055690 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f909401d630 con 0x7f909c155040 2024-01-24T15:14:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.653+0000 7f908a7fc700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f907c05f270 0x7f907c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.653+0000 7f908a7fc700 1 -- 172.21.15.29:0/3687055690 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f909407ca30 con 0x7f909c155040 2024-01-24T15:14:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.653+0000 7f909a59c700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f907c05f270 0x7f907c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.654+0000 7f909a59c700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f907c05f270 0x7f907c061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f908c0036e0 tx=0x7f908c009040).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.658+0000 7f908a7fc700 1 -- 172.21.15.29:0/3687055690 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f909407ee60 con 0x7f909c155040 2024-01-24T15:14:20.838 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.836+0000 7f90a0a3e700 1 -- 172.21.15.29:0/3687055690 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f909c0c46f0 con 0x7f907c05f270 2024-01-24T15:14:20.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.847+0000 7f908a7fc700 1 -- 172.21.15.29:0/3687055690 <== mgr.7535 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+354939 (secure 0 0 0) 0x7f909c0c46f0 con 0x7f907c05f270 2024-01-24T15:14:20.854 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:20.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.850+0000 7f90a0a3e700 1 -- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f907c05f270 msgr2=0x7f907c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.850+0000 7f90a0a3e700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f907c05f270 0x7f907c061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f908c0036e0 tx=0x7f908c009040).stop 2024-01-24T15:14:20.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.850+0000 7f90a0a3e700 1 -- 172.21.15.29:0/3687055690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c155040 msgr2=0x7f909c0c0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:20.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.850+0000 7f90a0a3e700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c155040 0x7f909c0c0da0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7f909c069030 tx=0x7f9094009780).stop 2024-01-24T15:14:20.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.850+0000 7f90a0a3e700 1 -- 172.21.15.29:0/3687055690 shutdown_connections 2024-01-24T15:14:20.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.850+0000 7f90a0a3e700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f907c05f270 0x7f907c061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.850+0000 7f90a0a3e700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f909c155040 0x7f909c0c0da0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.850+0000 7f90a0a3e700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f909c069240 0x7f909c0ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.850+0000 7f90a0a3e700 1 --2- 172.21.15.29:0/3687055690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f909c150d40 0x7f909c14e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:20.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.850+0000 7f90a0a3e700 1 -- 172.21.15.29:0/3687055690 >> 172.21.15.29:0/3687055690 conn(0x7f909c0c8580 msgr2=0x7f909c0638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:20.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.850+0000 7f90a0a3e700 1 -- 172.21.15.29:0/3687055690 shutdown_connections 2024-01-24T15:14:20.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:20.850+0000 7f90a0a3e700 1 -- 172.21.15.29:0/3687055690 wait complete. 2024-01-24T15:14:20.858 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:14:20.867 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T15:14:19.187219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":734,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":124,"acting":124,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2460868,"kb_used_data":18564,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752513852,"statfs":{"total":773094113280,"available":770574184448,"internally_reserved":0,"allocated":19009536,"data_stored":10904793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13569,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.010984"},"pg_stats":[{"pgid":"23.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.712181+0000","last_change":"2024-01-24T15:14:16.709796+0000","last_active":"2024-01-24T15:14:16.712181+0000","last_peered":"2024-01-24T15:14:16.712181+0000","last_clean":"2024-01-24T15:14:16.712181+0000","last_became_active":"2024-01-24T15:14:16.709471+0000","last_became_peered":"2024-01-24T15:14:16.709471+0000","last_unstale":"2024-01-24T15:14:16.712181+0000","last_undegraded":"2024-01-24T15:14:16.712181+0000","last_fullsized":"2024-01-24T15:14:16.712181+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.3b","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701962+0000","last_change":"2024-01-24T15:14:16.283517+0000","last_active":"2024-01-24T15:14:16.701962+0000","last_peered":"2024-01-24T15:14:16.701962+0000","last_clean":"2024-01-24T15:14:16.701962+0000","last_became_active":"2024-01-24T15:14:16.282222+0000","last_became_peered":"2024-01-24T15:14:16.282222+0000","last_unstale":"2024-01-24T15:14:16.701962+0000","last_undegraded":"2024-01-24T15:14:16.701962+0000","last_fullsized":"2024-01-24T15:14:16.701962+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3a","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701654+0000","last_change":"2024-01-24T15:14:16.284621+0000","last_active":"2024-01-24T15:14:16.701654+0000","last_peered":"2024-01-24T15:14:16.701654+0000","last_clean":"2024-01-24T15:14:16.701654+0000","last_became_active":"2024-01-24T15:14:16.284254+0000","last_became_peered":"2024-01-24T15:14:16.284254+0000","last_unstale":"2024-01-24T15:14:16.701654+0000","last_undegraded":"2024-01-24T15:14:16.701654+0000","last_fullsized":"2024-01-24T15:14:16.701654+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.683842+0000","last_change":"2024-01-24T15:14:16.274503+0000","last_active":"2024-01-24T15:14:16.683842+0000","last_peered":"2024-01-24T15:14:16.683842+0000","last_clean":"2024-01-24T15:14:16.683842+0000","last_became_active":"2024-01-24T15:14:16.271720+0000","last_became_peered":"2024-01-24T15:14:16.271720+0000","last_unstale":"2024-01-24T15:14:16.683842+0000","last_undegraded":"2024-01-24T15:14:16.683842+0000","last_fullsized":"2024-01-24T15:14:16.683842+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.683853+0000","last_change":"2024-01-24T15:14:16.279818+0000","last_active":"2024-01-24T15:14:16.683853+0000","last_peered":"2024-01-24T15:14:16.683853+0000","last_clean":"2024-01-24T15:14:16.683853+0000","last_became_active":"2024-01-24T15:14:16.279516+0000","last_became_peered":"2024-01-24T15:14:16.279516+0000","last_unstale":"2024-01-24T15:14:16.683853+0000","last_undegraded":"2024-01-24T15:14:16.683853+0000","last_fullsized":"2024-01-24T15:14:16.683853+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.686533+0000","last_change":"2024-01-24T15:14:16.279986+0000","last_active":"2024-01-24T15:14:16.686533+0000","last_peered":"2024-01-24T15:14:16.686533+0000","last_clean":"2024-01-24T15:14:16.686533+0000","last_became_active":"2024-01-24T15:14:16.279707+0000","last_became_peered":"2024-01-24T15:14:16.279707+0000","last_unstale":"2024-01-24T15:14:16.686533+0000","last_undegraded":"2024-01-24T15:14:16.686533+0000","last_fullsized":"2024-01-24T15:14:16.686533+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.678432+0000","last_change":"2024-01-24T15:14:16.277496+0000","last_active":"2024-01-24T15:14:16.678432+0000","last_peered":"2024-01-24T15:14:16.678432+0000","last_clean":"2024-01-24T15:14:16.678432+0000","last_became_active":"2024-01-24T15:14:16.276796+0000","last_became_peered":"2024-01-24T15:14:16.276796+0000","last_unstale":"2024-01-24T15:14:16.678432+0000","last_undegraded":"2024-01-24T15:14:16.678432+0000","last_fullsized":"2024-01-24T15:14:16.678432+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.33","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702361+0000","last_change":"2024-01-24T15:14:16.283115+0000","last_active":"2024-01-24T15:14:16.702361+0000","last_peered":"2024-01-24T15:14:16.702361+0000","last_clean":"2024-01-24T15:14:16.702361+0000","last_became_active":"2024-01-24T15:14:16.281456+0000","last_became_peered":"2024-01-24T15:14:16.281456+0000","last_unstale":"2024-01-24T15:14:16.702361+0000","last_undegraded":"2024-01-24T15:14:16.702361+0000","last_fullsized":"2024-01-24T15:14:16.702361+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.30","version":"192'4","reported_seq":14,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.707118+0000","last_change":"2024-01-24T15:14:16.277935+0000","last_active":"2024-01-24T15:14:16.707118+0000","last_peered":"2024-01-24T15:14:16.707118+0000","last_clean":"2024-01-24T15:14:16.707118+0000","last_became_active":"2024-01-24T15:14:16.277594+0000","last_became_peered":"2024-01-24T15:14:16.277594+0000","last_unstale":"2024-01-24T15:14:16.707118+0000","last_undegraded":"2024-01-24T15:14:16.707118+0000","last_fullsized":"2024-01-24T15:14:16.707118+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.707215+0000","last_change":"2024-01-24T15:14:16.703112+0000","last_active":"2024-01-24T15:14:16.707215+0000","last_peered":"2024-01-24T15:14:16.707215+0000","last_clean":"2024-01-24T15:14:16.707215+0000","last_became_active":"2024-01-24T15:14:16.702682+0000","last_became_peered":"2024-01-24T15:14:16.702682+0000","last_unstale":"2024-01-24T15:14:16.707215+0000","last_undegraded":"2024-01-24T15:14:16.707215+0000","last_fullsized":"2024-01-24T15:14:16.707215+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.693115+0000","last_change":"2024-01-24T15:14:16.688384+0000","last_active":"2024-01-24T15:14:16.693115+0000","last_peered":"2024-01-24T15:14:16.693115+0000","last_clean":"2024-01-24T15:14:16.693115+0000","last_became_active":"2024-01-24T15:14:16.686041+0000","last_became_peered":"2024-01-24T15:14:16.686041+0000","last_unstale":"2024-01-24T15:14:16.693115+0000","last_undegraded":"2024-01-24T15:14:16.693115+0000","last_fullsized":"2024-01-24T15:14:16.693115+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.677801+0000","last_change":"2024-01-24T15:14:16.270443+0000","last_active":"2024-01-24T15:14:16.677801+0000","last_peered":"2024-01-24T15:14:16.677801+0000","last_clean":"2024-01-24T15:14:16.677801+0000","last_became_active":"2024-01-24T15:14:16.270104+0000","last_became_peered":"2024-01-24T15:14:16.270104+0000","last_unstale":"2024-01-24T15:14:16.677801+0000","last_undegraded":"2024-01-24T15:14:16.677801+0000","last_fullsized":"2024-01-24T15:14:16.677801+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.690691+0000","last_change":"2024-01-24T15:14:16.686134+0000","last_active":"2024-01-24T15:14:16.690691+0000","last_peered":"2024-01-24T15:14:16.690691+0000","last_clean":"2024-01-24T15:14:16.690691+0000","last_became_active":"2024-01-24T15:14:16.685894+0000","last_became_peered":"2024-01-24T15:14:16.685894+0000","last_unstale":"2024-01-24T15:14:16.690691+0000","last_undegraded":"2024-01-24T15:14:16.690691+0000","last_fullsized":"2024-01-24T15:14:16.690691+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2b","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702248+0000","last_change":"2024-01-24T15:14:16.283313+0000","last_active":"2024-01-24T15:14:16.702248+0000","last_peered":"2024-01-24T15:14:16.702248+0000","last_clean":"2024-01-24T15:14:16.702248+0000","last_became_active":"2024-01-24T15:14:16.281802+0000","last_became_peered":"2024-01-24T15:14:16.281802+0000","last_unstale":"2024-01-24T15:14:16.702248+0000","last_undegraded":"2024-01-24T15:14:16.702248+0000","last_fullsized":"2024-01-24T15:14:16.702248+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.687649+0000","last_change":"2024-01-24T15:14:16.685690+0000","last_active":"2024-01-24T15:14:16.687649+0000","last_peered":"2024-01-24T15:14:16.687649+0000","last_clean":"2024-01-24T15:14:16.687649+0000","last_became_active":"2024-01-24T15:14:16.685430+0000","last_became_peered":"2024-01-24T15:14:16.685430+0000","last_unstale":"2024-01-24T15:14:16.687649+0000","last_undegraded":"2024-01-24T15:14:16.687649+0000","last_fullsized":"2024-01-24T15:14:16.687649+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.683170+0000","last_change":"2024-01-24T15:14:16.267120+0000","last_active":"2024-01-24T15:14:16.683170+0000","last_peered":"2024-01-24T15:14:16.683170+0000","last_clean":"2024-01-24T15:14:16.683170+0000","last_became_active":"2024-01-24T15:14:16.266784+0000","last_became_peered":"2024-01-24T15:14:16.266784+0000","last_unstale":"2024-01-24T15:14:16.683170+0000","last_undegraded":"2024-01-24T15:14:16.683170+0000","last_fullsized":"2024-01-24T15:14:16.683170+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.706771+0000","last_change":"2024-01-24T15:14:16.705525+0000","last_active":"2024-01-24T15:14:16.706771+0000","last_peered":"2024-01-24T15:14:16.706771+0000","last_clean":"2024-01-24T15:14:16.706771+0000","last_became_active":"2024-01-24T15:14:16.705230+0000","last_became_peered":"2024-01-24T15:14:16.705230+0000","last_unstale":"2024-01-24T15:14:16.706771+0000","last_undegraded":"2024-01-24T15:14:16.706771+0000","last_fullsized":"2024-01-24T15:14:16.706771+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.678684+0000","last_change":"2024-01-24T15:14:16.271054+0000","last_active":"2024-01-24T15:14:16.678684+0000","last_peered":"2024-01-24T15:14:16.678684+0000","last_clean":"2024-01-24T15:14:16.678684+0000","last_became_active":"2024-01-24T15:14:16.270755+0000","last_became_peered":"2024-01-24T15:14:16.270755+0000","last_unstale":"2024-01-24T15:14:16.678684+0000","last_undegraded":"2024-01-24T15:14:16.678684+0000","last_fullsized":"2024-01-24T15:14:16.678684+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.682654+0000","last_change":"2024-01-24T15:14:16.278084+0000","last_active":"2024-01-24T15:14:16.682654+0000","last_peered":"2024-01-24T15:14:16.682654+0000","last_clean":"2024-01-24T15:14:16.682654+0000","last_became_active":"2024-01-24T15:14:16.277795+0000","last_became_peered":"2024-01-24T15:14:16.277795+0000","last_unstale":"2024-01-24T15:14:16.682654+0000","last_undegraded":"2024-01-24T15:14:16.682654+0000","last_fullsized":"2024-01-24T15:14:16.682654+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.678070+0000","last_change":"2024-01-24T15:14:16.278694+0000","last_active":"2024-01-24T15:14:16.678070+0000","last_peered":"2024-01-24T15:14:16.678070+0000","last_clean":"2024-01-24T15:14:16.678070+0000","last_became_active":"2024-01-24T15:14:16.278405+0000","last_became_peered":"2024-01-24T15:14:16.278405+0000","last_unstale":"2024-01-24T15:14:16.678070+0000","last_undegraded":"2024-01-24T15:14:16.678070+0000","last_fullsized":"2024-01-24T15:14:16.678070+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.703206+0000","last_change":"2024-01-24T15:14:16.693978+0000","last_active":"2024-01-24T15:14:16.703206+0000","last_peered":"2024-01-24T15:14:16.703206+0000","last_clean":"2024-01-24T15:14:16.703206+0000","last_became_active":"2024-01-24T15:14:16.693681+0000","last_became_peered":"2024-01-24T15:14:16.693681+0000","last_unstale":"2024-01-24T15:14:16.703206+0000","last_undegraded":"2024-01-24T15:14:16.703206+0000","last_fullsized":"2024-01-24T15:14:16.703206+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.684283+0000","last_change":"2024-01-24T15:14:16.286055+0000","last_active":"2024-01-24T15:14:16.684283+0000","last_peered":"2024-01-24T15:14:16.684283+0000","last_clean":"2024-01-24T15:14:16.684283+0000","last_became_active":"2024-01-24T15:14:16.285785+0000","last_became_peered":"2024-01-24T15:14:16.285785+0000","last_unstale":"2024-01-24T15:14:16.684283+0000","last_undegraded":"2024-01-24T15:14:16.684283+0000","last_fullsized":"2024-01-24T15:14:16.684283+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.678798+0000","last_change":"2024-01-24T15:14:16.281188+0000","last_active":"2024-01-24T15:14:16.678798+0000","last_peered":"2024-01-24T15:14:16.678798+0000","last_clean":"2024-01-24T15:14:16.678798+0000","last_became_active":"2024-01-24T15:14:16.280878+0000","last_became_peered":"2024-01-24T15:14:16.280878+0000","last_unstale":"2024-01-24T15:14:16.678798+0000","last_undegraded":"2024-01-24T15:14:16.678798+0000","last_fullsized":"2024-01-24T15:14:16.678798+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.703314+0000","last_change":"2024-01-24T15:14:16.691985+0000","last_active":"2024-01-24T15:14:16.703314+0000","last_peered":"2024-01-24T15:14:16.703314+0000","last_clean":"2024-01-24T15:14:16.703314+0000","last_became_active":"2024-01-24T15:14:16.691708+0000","last_became_peered":"2024-01-24T15:14:16.691708+0000","last_unstale":"2024-01-24T15:14:16.703314+0000","last_undegraded":"2024-01-24T15:14:16.703314+0000","last_fullsized":"2024-01-24T15:14:16.703314+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.682608+0000","last_change":"2024-01-24T15:14:16.275498+0000","last_active":"2024-01-24T15:14:16.682608+0000","last_peered":"2024-01-24T15:14:16.682608+0000","last_clean":"2024-01-24T15:14:16.682608+0000","last_became_active":"2024-01-24T15:14:16.275112+0000","last_became_peered":"2024-01-24T15:14:16.275112+0000","last_unstale":"2024-01-24T15:14:16.682608+0000","last_undegraded":"2024-01-24T15:14:16.682608+0000","last_fullsized":"2024-01-24T15:14:16.682608+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.711484+0000","last_change":"2024-01-24T15:14:16.710358+0000","last_active":"2024-01-24T15:14:16.711484+0000","last_peered":"2024-01-24T15:14:16.711484+0000","last_clean":"2024-01-24T15:14:16.711484+0000","last_became_active":"2024-01-24T15:14:16.710087+0000","last_became_peered":"2024-01-24T15:14:16.710087+0000","last_unstale":"2024-01-24T15:14:16.711484+0000","last_undegraded":"2024-01-24T15:14:16.711484+0000","last_fullsized":"2024-01-24T15:14:16.711484+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.683729+0000","last_change":"2024-01-24T15:14:16.272629+0000","last_active":"2024-01-24T15:14:16.683729+0000","last_peered":"2024-01-24T15:14:16.683729+0000","last_clean":"2024-01-24T15:14:16.683729+0000","last_became_active":"2024-01-24T15:14:16.272067+0000","last_became_peered":"2024-01-24T15:14:16.272067+0000","last_unstale":"2024-01-24T15:14:16.683729+0000","last_undegraded":"2024-01-24T15:14:16.683729+0000","last_fullsized":"2024-01-24T15:14:16.683729+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.706182+0000","last_change":"2024-01-24T15:14:16.704248+0000","last_active":"2024-01-24T15:14:16.706182+0000","last_peered":"2024-01-24T15:14:16.706182+0000","last_clean":"2024-01-24T15:14:16.706182+0000","last_became_active":"2024-01-24T15:14:16.703837+0000","last_became_peered":"2024-01-24T15:14:16.703837+0000","last_unstale":"2024-01-24T15:14:16.706182+0000","last_undegraded":"2024-01-24T15:14:16.706182+0000","last_fullsized":"2024-01-24T15:14:16.706182+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.703478+0000","last_change":"2024-01-24T15:14:16.282326+0000","last_active":"2024-01-24T15:14:16.703478+0000","last_peered":"2024-01-24T15:14:16.703478+0000","last_clean":"2024-01-24T15:14:16.703478+0000","last_became_active":"2024-01-24T15:14:16.281995+0000","last_became_peered":"2024-01-24T15:14:16.281995+0000","last_unstale":"2024-01-24T15:14:16.703478+0000","last_undegraded":"2024-01-24T15:14:16.703478+0000","last_fullsized":"2024-01-24T15:14:16.703478+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.695576+0000","last_change":"2024-01-24T15:14:16.693864+0000","last_active":"2024-01-24T15:14:16.695576+0000","last_peered":"2024-01-24T15:14:16.695576+0000","last_clean":"2024-01-24T15:14:16.695576+0000","last_became_active":"2024-01-24T15:14:16.693552+0000","last_became_peered":"2024-01-24T15:14:16.693552+0000","last_unstale":"2024-01-24T15:14:16.695576+0000","last_undegraded":"2024-01-24T15:14:16.695576+0000","last_fullsized":"2024-01-24T15:14:16.695576+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.712687+0000","last_change":"2024-01-24T15:14:16.709855+0000","last_active":"2024-01-24T15:14:16.712687+0000","last_peered":"2024-01-24T15:14:16.712687+0000","last_clean":"2024-01-24T15:14:16.712687+0000","last_became_active":"2024-01-24T15:14:16.709553+0000","last_became_peered":"2024-01-24T15:14:16.709553+0000","last_unstale":"2024-01-24T15:14:16.712687+0000","last_undegraded":"2024-01-24T15:14:16.712687+0000","last_fullsized":"2024-01-24T15:14:16.712687+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702053+0000","last_change":"2024-01-24T15:14:16.693517+0000","last_active":"2024-01-24T15:14:16.702053+0000","last_peered":"2024-01-24T15:14:16.702053+0000","last_clean":"2024-01-24T15:14:16.702053+0000","last_became_active":"2024-01-24T15:14:16.693259+0000","last_became_peered":"2024-01-24T15:14:16.693259+0000","last_unstale":"2024-01-24T15:14:16.702053+0000","last_undegraded":"2024-01-24T15:14:16.702053+0000","last_fullsized":"2024-01-24T15:14:16.702053+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.706794+0000","last_change":"2024-01-24T15:14:16.705535+0000","last_active":"2024-01-24T15:14:16.706794+0000","last_peered":"2024-01-24T15:14:16.706794+0000","last_clean":"2024-01-24T15:14:16.706794+0000","last_became_active":"2024-01-24T15:14:16.705216+0000","last_became_peered":"2024-01-24T15:14:16.705216+0000","last_unstale":"2024-01-24T15:14:16.706794+0000","last_undegraded":"2024-01-24T15:14:16.706794+0000","last_fullsized":"2024-01-24T15:14:16.706794+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.707194+0000","last_change":"2024-01-24T15:14:16.702939+0000","last_active":"2024-01-24T15:14:16.707194+0000","last_peered":"2024-01-24T15:14:16.707194+0000","last_clean":"2024-01-24T15:14:16.707194+0000","last_became_active":"2024-01-24T15:14:16.702378+0000","last_became_peered":"2024-01-24T15:14:16.702378+0000","last_unstale":"2024-01-24T15:14:16.707194+0000","last_undegraded":"2024-01-24T15:14:16.707194+0000","last_fullsized":"2024-01-24T15:14:16.707194+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701981+0000","last_change":"2024-01-24T15:14:16.284287+0000","last_active":"2024-01-24T15:14:16.701981+0000","last_peered":"2024-01-24T15:14:16.701981+0000","last_clean":"2024-01-24T15:14:16.701981+0000","last_became_active":"2024-01-24T15:14:16.283984+0000","last_became_peered":"2024-01-24T15:14:16.283984+0000","last_unstale":"2024-01-24T15:14:16.701981+0000","last_undegraded":"2024-01-24T15:14:16.701981+0000","last_fullsized":"2024-01-24T15:14:16.701981+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.699221+0000","last_change":"2024-01-24T15:14:16.697555+0000","last_active":"2024-01-24T15:14:16.699221+0000","last_peered":"2024-01-24T15:14:16.699221+0000","last_clean":"2024-01-24T15:14:16.699221+0000","last_became_active":"2024-01-24T15:14:16.697248+0000","last_became_peered":"2024-01-24T15:14:16.697248+0000","last_unstale":"2024-01-24T15:14:16.699221+0000","last_undegraded":"2024-01-24T15:14:16.699221+0000","last_fullsized":"2024-01-24T15:14:16.699221+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702035+0000","last_change":"2024-01-24T15:14:16.694777+0000","last_active":"2024-01-24T15:14:16.702035+0000","last_peered":"2024-01-24T15:14:16.702035+0000","last_clean":"2024-01-24T15:14:16.702035+0000","last_became_active":"2024-01-24T15:14:16.694478+0000","last_became_peered":"2024-01-24T15:14:16.694478+0000","last_unstale":"2024-01-24T15:14:16.702035+0000","last_undegraded":"2024-01-24T15:14:16.702035+0000","last_fullsized":"2024-01-24T15:14:16.702035+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.685807+0000","last_change":"2024-01-24T15:14:16.281779+0000","last_active":"2024-01-24T15:14:16.685807+0000","last_peered":"2024-01-24T15:14:16.685807+0000","last_clean":"2024-01-24T15:14:16.685807+0000","last_became_active":"2024-01-24T15:14:16.281503+0000","last_became_peered":"2024-01-24T15:14:16.281503+0000","last_unstale":"2024-01-24T15:14:16.685807+0000","last_undegraded":"2024-01-24T15:14:16.685807+0000","last_fullsized":"2024-01-24T15:14:16.685807+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701963+0000","last_change":"2024-01-24T15:14:16.691909+0000","last_active":"2024-01-24T15:14:16.701963+0000","last_peered":"2024-01-24T15:14:16.701963+0000","last_clean":"2024-01-24T15:14:16.701963+0000","last_became_active":"2024-01-24T15:14:16.691625+0000","last_became_peered":"2024-01-24T15:14:16.691625+0000","last_unstale":"2024-01-24T15:14:16.701963+0000","last_undegraded":"2024-01-24T15:14:16.701963+0000","last_fullsized":"2024-01-24T15:14:16.701963+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.684389+0000","last_change":"2024-01-24T15:14:16.282171+0000","last_active":"2024-01-24T15:14:16.684389+0000","last_peered":"2024-01-24T15:14:16.684389+0000","last_clean":"2024-01-24T15:14:16.684389+0000","last_became_active":"2024-01-24T15:14:16.281888+0000","last_became_peered":"2024-01-24T15:14:16.281888+0000","last_unstale":"2024-01-24T15:14:16.684389+0000","last_undegraded":"2024-01-24T15:14:16.684389+0000","last_fullsized":"2024-01-24T15:14:16.684389+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.705155+0000","last_change":"2024-01-24T15:14:16.700552+0000","last_active":"2024-01-24T15:14:16.705155+0000","last_peered":"2024-01-24T15:14:16.705155+0000","last_clean":"2024-01-24T15:14:16.705155+0000","last_became_active":"2024-01-24T15:14:16.700172+0000","last_became_peered":"2024-01-24T15:14:16.700172+0000","last_unstale":"2024-01-24T15:14:16.705155+0000","last_undegraded":"2024-01-24T15:14:16.705155+0000","last_fullsized":"2024-01-24T15:14:16.705155+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.683343+0000","last_change":"2024-01-24T15:14:16.281660+0000","last_active":"2024-01-24T15:14:16.683343+0000","last_peered":"2024-01-24T15:14:16.683343+0000","last_clean":"2024-01-24T15:14:16.683343+0000","last_became_active":"2024-01-24T15:14:16.281382+0000","last_became_peered":"2024-01-24T15:14:16.281382+0000","last_unstale":"2024-01-24T15:14:16.683343+0000","last_undegraded":"2024-01-24T15:14:16.683343+0000","last_fullsized":"2024-01-24T15:14:16.683343+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.10","version":"192'4","reported_seq":14,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702479+0000","last_change":"2024-01-24T15:14:16.267755+0000","last_active":"2024-01-24T15:14:16.702479+0000","last_peered":"2024-01-24T15:14:16.702479+0000","last_clean":"2024-01-24T15:14:16.702479+0000","last_became_active":"2024-01-24T15:14:16.267489+0000","last_became_peered":"2024-01-24T15:14:16.267489+0000","last_unstale":"2024-01-24T15:14:16.702479+0000","last_undegraded":"2024-01-24T15:14:16.702479+0000","last_fullsized":"2024-01-24T15:14:16.702479+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-01-24T15:14:12.101333+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:14:12.101333+0000","last_peered":"2024-01-24T15:14:12.101333+0000","last_clean":"2024-01-24T15:14:12.101333+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:14:12.101333+0000","last_undegraded":"2024-01-24T15:14:12.101333+0000","last_fullsized":"2024-01-24T15:14:12.101333+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.716522+0000","last_change":"2024-01-24T15:14:16.715837+0000","last_active":"2024-01-24T15:14:16.716522+0000","last_peered":"2024-01-24T15:14:16.716522+0000","last_clean":"2024-01-24T15:14:16.716522+0000","last_became_active":"2024-01-24T15:14:16.712087+0000","last_became_peered":"2024-01-24T15:14:16.712087+0000","last_unstale":"2024-01-24T15:14:16.716522+0000","last_undegraded":"2024-01-24T15:14:16.716522+0000","last_fullsized":"2024-01-24T15:14:16.716522+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.677909+0000","last_change":"2024-01-24T15:14:16.268328+0000","last_active":"2024-01-24T15:14:16.677909+0000","last_peered":"2024-01-24T15:14:16.677909+0000","last_clean":"2024-01-24T15:14:16.677909+0000","last_became_active":"2024-01-24T15:14:16.267970+0000","last_became_peered":"2024-01-24T15:14:16.267970+0000","last_unstale":"2024-01-24T15:14:16.677909+0000","last_undegraded":"2024-01-24T15:14:16.677909+0000","last_fullsized":"2024-01-24T15:14:16.677909+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-01-24T15:14:11.236582+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:14:11.236582+0000","last_peered":"2024-01-24T15:14:11.236582+0000","last_clean":"2024-01-24T15:14:11.236582+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:14:11.236582+0000","last_undegraded":"2024-01-24T15:14:11.236582+0000","last_fullsized":"2024-01-24T15:14:11.236582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.706701+0000","last_change":"2024-01-24T15:14:16.702686+0000","last_active":"2024-01-24T15:14:16.706701+0000","last_peered":"2024-01-24T15:14:16.706701+0000","last_clean":"2024-01-24T15:14:16.706701+0000","last_became_active":"2024-01-24T15:14:16.702391+0000","last_became_peered":"2024-01-24T15:14:16.702391+0000","last_unstale":"2024-01-24T15:14:16.706701+0000","last_undegraded":"2024-01-24T15:14:16.706701+0000","last_fullsized":"2024-01-24T15:14:16.706701+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"192'3","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701311+0000","last_change":"2024-01-24T15:14:16.277740+0000","last_active":"2024-01-24T15:14:16.701311+0000","last_peered":"2024-01-24T15:14:16.701311+0000","last_clean":"2024-01-24T15:14:16.701311+0000","last_became_active":"2024-01-24T15:14:16.277193+0000","last_became_peered":"2024-01-24T15:14:16.277193+0000","last_unstale":"2024-01-24T15:14:16.701311+0000","last_undegraded":"2024-01-24T15:14:16.701311+0000","last_fullsized":"2024-01-24T15:14:16.701311+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.678379+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:14:16.678379+0000","last_peered":"2024-01-24T15:14:16.678379+0000","last_clean":"2024-01-24T15:14:16.678379+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:14:16.678379+0000","last_undegraded":"2024-01-24T15:14:16.678379+0000","last_fullsized":"2024-01-24T15:14:16.678379+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.716186+0000","last_change":"2024-01-24T15:14:16.713849+0000","last_active":"2024-01-24T15:14:16.716186+0000","last_peered":"2024-01-24T15:14:16.716186+0000","last_clean":"2024-01-24T15:14:16.716186+0000","last_became_active":"2024-01-24T15:14:16.712035+0000","last_became_peered":"2024-01-24T15:14:16.712035+0000","last_unstale":"2024-01-24T15:14:16.716186+0000","last_undegraded":"2024-01-24T15:14:16.716186+0000","last_fullsized":"2024-01-24T15:14:16.716186+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.684209+0000","last_change":"2024-01-24T15:14:16.274086+0000","last_active":"2024-01-24T15:14:16.684209+0000","last_peered":"2024-01-24T15:14:16.684209+0000","last_clean":"2024-01-24T15:14:16.684209+0000","last_became_active":"2024-01-24T15:14:16.273819+0000","last_became_peered":"2024-01-24T15:14:16.273819+0000","last_unstale":"2024-01-24T15:14:16.684209+0000","last_undegraded":"2024-01-24T15:14:16.684209+0000","last_fullsized":"2024-01-24T15:14:16.684209+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":173,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-01-24T15:14:11.236604+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:14:11.236604+0000","last_peered":"2024-01-24T15:14:11.236604+0000","last_clean":"2024-01-24T15:14:11.236604+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:14:11.236604+0000","last_undegraded":"2024-01-24T15:14:11.236604+0000","last_fullsized":"2024-01-24T15:14:11.236604+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":155,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-01-24T15:14:12.099721+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:14:12.099721+0000","last_peered":"2024-01-24T15:14:12.099721+0000","last_clean":"2024-01-24T15:14:12.099721+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:14:12.099721+0000","last_undegraded":"2024-01-24T15:14:12.099721+0000","last_fullsized":"2024-01-24T15:14:12.099721+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.703202+0000","last_change":"2024-01-24T15:14:16.693968+0000","last_active":"2024-01-24T15:14:16.703202+0000","last_peered":"2024-01-24T15:14:16.703202+0000","last_clean":"2024-01-24T15:14:16.703202+0000","last_became_active":"2024-01-24T15:14:16.693735+0000","last_became_peered":"2024-01-24T15:14:16.693735+0000","last_unstale":"2024-01-24T15:14:16.703202+0000","last_undegraded":"2024-01-24T15:14:16.703202+0000","last_fullsized":"2024-01-24T15:14:16.703202+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702171+0000","last_change":"2024-01-24T15:14:16.273322+0000","last_active":"2024-01-24T15:14:16.702171+0000","last_peered":"2024-01-24T15:14:16.702171+0000","last_clean":"2024-01-24T15:14:16.702171+0000","last_became_active":"2024-01-24T15:14:16.273048+0000","last_became_peered":"2024-01-24T15:14:16.273048+0000","last_unstale":"2024-01-24T15:14:16.702171+0000","last_undegraded":"2024-01-24T15:14:16.702171+0000","last_fullsized":"2024-01-24T15:14:16.702171+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-01-24T15:14:12.099690+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:14:12.099690+0000","last_peered":"2024-01-24T15:14:12.099690+0000","last_clean":"2024-01-24T15:14:12.099690+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:14:12.099690+0000","last_undegraded":"2024-01-24T15:14:12.099690+0000","last_fullsized":"2024-01-24T15:14:12.099690+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-01-24T15:14:12.099704+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:14:12.099704+0000","last_peered":"2024-01-24T15:14:12.099704+0000","last_clean":"2024-01-24T15:14:12.099704+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:14:12.099704+0000","last_undegraded":"2024-01-24T15:14:12.099704+0000","last_fullsized":"2024-01-24T15:14:12.099704+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:16.250162+0000","last_change":"2024-01-24T15:14:16.250162+0000","last_active":"2024-01-24T15:14:16.250162+0000","last_peered":"2024-01-24T15:14:16.250162+0000","last_clean":"2024-01-24T15:14:16.250162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:16.250162+0000","last_undegraded":"2024-01-24T15:14:16.250162+0000","last_fullsized":"2024-01-24T15:14:16.250162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.683606+0000","last_change":"2024-01-24T15:14:16.284535+0000","last_active":"2024-01-24T15:14:16.683606+0000","last_peered":"2024-01-24T15:14:16.683606+0000","last_clean":"2024-01-24T15:14:16.683606+0000","last_became_active":"2024-01-24T15:14:16.283799+0000","last_became_peered":"2024-01-24T15:14:16.283799+0000","last_unstale":"2024-01-24T15:14:16.683606+0000","last_undegraded":"2024-01-24T15:14:16.683606+0000","last_fullsized":"2024-01-24T15:14:16.683606+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-01-24T15:14:12.101288+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:14:12.101288+0000","last_peered":"2024-01-24T15:14:12.101288+0000","last_clean":"2024-01-24T15:14:12.101288+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:14:12.101288+0000","last_undegraded":"2024-01-24T15:14:12.101288+0000","last_fullsized":"2024-01-24T15:14:12.101288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.709023+0000","last_change":"2024-01-24T15:14:16.706451+0000","last_active":"2024-01-24T15:14:16.709023+0000","last_peered":"2024-01-24T15:14:16.709023+0000","last_clean":"2024-01-24T15:14:16.709023+0000","last_became_active":"2024-01-24T15:14:16.706147+0000","last_became_peered":"2024-01-24T15:14:16.706147+0000","last_unstale":"2024-01-24T15:14:16.709023+0000","last_undegraded":"2024-01-24T15:14:16.709023+0000","last_fullsized":"2024-01-24T15:14:16.709023+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-24T15:14:15.245157+0000","last_change":"2024-01-24T15:14:15.245157+0000","last_active":"2024-01-24T15:14:15.245157+0000","last_peered":"2024-01-24T15:14:15.245157+0000","last_clean":"2024-01-24T15:14:15.245157+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-24T15:14:15.245157+0000","last_undegraded":"2024-01-24T15:14:15.245157+0000","last_fullsized":"2024-01-24T15:14:15.245157+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":150,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-01-24T15:14:12.100238+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:14:12.100238+0000","last_peered":"2024-01-24T15:14:12.100238+0000","last_clean":"2024-01-24T15:14:12.100238+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:14:12.100238+0000","last_undegraded":"2024-01-24T15:14:12.100238+0000","last_fullsized":"2024-01-24T15:14:12.100238+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":734,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":58,"acting":58,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575757,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303484,"kb_used_data":2292,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068356,"statfs":{"total":96636764160,"available":96325996544,"internally_reserved":0,"allocated":2347008,"data_stored":1347028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.59899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.28000000000000003},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.628,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.28899999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.1080000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.33300000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.505,"5min":0.40000000000000002,"15min":0.34799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.50600000000000001}]},{"osd":2,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0329999999999999},"last":0.39800000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.38},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0880000000000001},"last":0.439},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.41199999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.33200000000000002},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.31900000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.46600000000000003}]},{"osd":5,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.39600000000000002,"5min":0.38,"15min":0.34799999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.36199999999999999},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.498}]},{"osd":6,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.31900000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.46400000000000002,"5min":0.40899999999999997,"15min":0.40500000000000003},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.53900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575758,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307592,"kb_used_data":2304,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064248,"statfs":{"total":96636764160,"available":96321789952,"internally_reserved":0,"allocated":2359296,"data_stored":1354009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.86599999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.42699999999999999,"15min":0.37},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52800000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.84199999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77100000000000002,"5min":0.81200000000000006,"15min":1.258},"last":0.55300000000000005}]},{"osd":2,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.40100000000000002},"max":{"1min":0.82299999999999995,"5min":0.85899999999999999,"15min":1.2430000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.66800000000000004}]},{"osd":3,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.47199999999999998,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.39000000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.75900000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.317},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32400000000000001,"15min":0.30499999999999999},"max":{"1min":0.79300000000000004,"5min":0.80100000000000005,"15min":1.002},"last":0.80300000000000005}]},{"osd":5,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.32500000000000001},"max":{"1min":0.76000000000000001,"5min":0.78100000000000003,"15min":0.83899999999999997},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":0.749,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.83599999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.40899999999999997,"5min":0.372,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.90100000000000002},"last":0.878},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.38},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":1.0429999999999999},"last":1.0369999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575757,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307592,"kb_used_data":2304,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064248,"statfs":{"total":96636764160,"available":96321789952,"internally_reserved":0,"allocated":2359296,"data_stored":1354009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.56399999999999995,"5min":0.754,"15min":0.754},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.443,"15min":0.435},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.26800000000000002},"max":{"1min":0.55800000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.40000000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.33700000000000002,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.57599999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44600000000000001,"15min":0.44700000000000001},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.28199999999999997},"max":{"1min":0.56200000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.38700000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"min":{"1min":0.32400000000000001,"5min":0.314,"15min":0.309},"max":{"1min":0.58399999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.35799999999999998,"15min":0.313},"max":{"1min":0.59899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.34300000000000003}]},{"osd":4,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.51800000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.86099999999999999,"5min":1.044,"15min":1.0649999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.51400000000000001,"5min":0.47399999999999998,"15min":0.41699999999999998},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.49199999999999999,"15min":0.46300000000000002},"max":{"1min":0.83999999999999997,"5min":1.014,"15min":1.0780000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.439},"max":{"1min":0.86699999999999999,"5min":1.105,"15min":1.105},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.874,"5min":1.0029999999999999,"15min":1.069},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.56599999999999995,"5min":0.499,"15min":0.48199999999999998},"max":{"1min":0.92500000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.56399999999999995}]},{"osd":7,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.83799999999999997,"5min":1.127,"15min":1.127},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.77500000000000002,"15min":0.76700000000000002},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.42799999999999999},"max":{"1min":0.88900000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.73499999999999999}]}]},{"osd":0,"up_from":14,"seq":60129543055,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307576,"kb_used_data":2288,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064264,"statfs":{"total":96636764160,"available":96321806336,"internally_reserved":0,"allocated":2342912,"data_stored":1346067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.28100000000000003,"5min":0.27500000000000002,"15min":0.247},"max":{"1min":0.60199999999999998,"5min":0.63400000000000001,"15min":0.66400000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.27700000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.437}]},{"osd":2,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.29399999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.81999999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.84399999999999997},"last":0.47599999999999998}]},{"osd":3,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.44600000000000001},"min":{"1min":0.317,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.65500000000000003,"15min":0.67000000000000004},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.52000000000000002},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.90000000000000002},"last":0.42399999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.433},"max":{"1min":0.99399999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.38700000000000001},"max":{"1min":0.995,"5min":0.995,"15min":1.0249999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.70699999999999996},"min":{"1min":0.54900000000000004,"5min":0.498,"15min":0.42099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.64400000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.46899999999999997},"max":{"1min":1.022,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0820000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0469999999999999,"5min":1.143,"15min":1.3260000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.375},"last":0.63600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575758,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311728,"kb_used_data":2344,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060112,"statfs":{"total":96636764160,"available":96317554688,"internally_reserved":0,"allocated":2400256,"data_stored":1377373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.27200000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.60099999999999998,"5min":0.65000000000000002,"15min":0.66800000000000004},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.45100000000000001,"15min":0.45200000000000001},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.70399999999999996},"last":0.43099999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.25600000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.73599999999999999},"last":0.46300000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.46300000000000002},"min":{"1min":0.32500000000000001,"5min":0.29699999999999999,"15min":0.29599999999999999},"max":{"1min":0.622,"5min":0.69699999999999995,"15min":0.70099999999999996},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.73299999999999998},"last":0.39500000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96199999999999997,"5min":1.196,"15min":1.196},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":1,"15min":1.089},"last":0.627}]},{"osd":5,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54400000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.111},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.46899999999999997,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.147,"15min":1.1830000000000001},"last":0.55700000000000005}]},{"osd":6,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.64700000000000002,"5min":0.47699999999999998,"15min":0.45200000000000001},"max":{"1min":0.99399999999999999,"5min":0.998,"15min":1.129},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.58899999999999997,"5min":0.47299999999999998,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.98999999999999999,"15min":1.1950000000000001},"last":0.61499999999999999}]},{"osd":7,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.753,"15min":0.746},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.497},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.248},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.65000000000000002,"5min":0.51500000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.24,"15min":1.24},"last":0.71699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575758,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307648,"kb_used_data":2360,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064192,"statfs":{"total":96636764160,"available":96321732608,"internally_reserved":0,"allocated":2416640,"data_stored":1385978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39800000000000002,"15min":0.39200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.247},"max":{"1min":0.56899999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.70199999999999996},"last":0.33300000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41899999999999998,"15min":0.41399999999999998},"min":{"1min":0.311,"5min":0.28100000000000003,"15min":0.27400000000000002},"max":{"1min":0.59999999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.377},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.34799999999999998,"5min":0.316,"15min":0.30499999999999999},"max":{"1min":0.67900000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.441}]},{"osd":2,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.497,"15min":0.502},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51300000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42999999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.41099999999999998},"max":{"1min":0.83299999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95999999999999996,"15min":1.5580000000000001},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.1879999999999999,"15min":1.6240000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.647},"last":0.67600000000000005}]},{"osd":6,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.45700000000000002,"15min":0.42699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.522},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.073,"5min":1.079,"15min":1.079},"last":0.75700000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.44800000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1499999999999999,"5min":1.246,"15min":1.246},"last":0.66000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307656,"kb_used_data":2368,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064184,"statfs":{"total":96636764160,"available":96321724416,"internally_reserved":0,"allocated":2424832,"data_stored":1386320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.62},"min":{"1min":0.34899999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.84199999999999997},"last":0.66700000000000004}]},{"osd":1,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.82499999999999996,"15min":0.82599999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63300000000000001}]},{"osd":2,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.35799999999999998},"max":{"1min":0.82099999999999995,"5min":0.84699999999999998,"15min":0.88300000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.372},"max":{"1min":0.81499999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.62}]},{"osd":3,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.38400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.89500000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.50900000000000001,"5min":0.46800000000000003,"15min":0.42399999999999999},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.92300000000000004},"last":0.64500000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.36099999999999999,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.81799999999999995},"last":0.374},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51500000000000001,"15min":0.51600000000000001},"min":{"1min":0.40899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65800000000000003,"5min":0.72499999999999998,"15min":0.85499999999999998},"last":0.41799999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.73699999999999999,"5min":0.78500000000000003,"15min":0.85499999999999998},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.86799999999999999},"last":0.438}]},{"osd":7,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55300000000000005,"15min":0.55200000000000005},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.33600000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.82799999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.96599999999999997},"last":0.45200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575758,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307592,"kb_used_data":2304,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064248,"statfs":{"total":96636764160,"available":96321789952,"internally_reserved":0,"allocated":2359296,"data_stored":1354009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.38600000000000001,"15min":0.35099999999999998},"max":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":1.4470000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.81599999999999995},"last":0.94299999999999995}]},{"osd":1,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.82499999999999996},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":1.4930000000000001},"last":0.86099999999999999}]},{"osd":2,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.625,"15min":0.627},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.79300000000000004,"5min":0.80900000000000005,"15min":0.86499999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":1.53},"last":0.93200000000000005}]},{"osd":3,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.80800000000000005,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41899999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.90100000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.65400000000000003,"5min":0.74399999999999999,"15min":0.84699999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":0.77200000000000002,"15min":0.80100000000000005},"last":0.71199999999999997}]},{"osd":6,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.5,"15min":0.504},"min":{"1min":0.374,"5min":0.35499999999999998,"15min":0.33400000000000002},"max":{"1min":0.65100000000000002,"5min":0.79400000000000004,"15min":0.81499999999999995},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.78500000000000003,"15min":0.81299999999999994},"last":0.73699999999999999}]},{"osd":7,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.54900000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.95399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:14:20.878 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:14:21.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.037+0000 7fb84a58a700 1 Processor -- start 2024-01-24T15:14:21.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.038+0000 7fb84a58a700 1 -- start start 2024-01-24T15:14:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.038+0000 7fb84a58a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8441540a0 0x7fb8440c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.038+0000 7fb84a58a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb8441550f0 0x7fb8440627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.038+0000 7fb84a58a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb844065360 con 0x7fb844062ca0 2024-01-24T15:14:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.038+0000 7fb84a58a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8440a74f0 con 0x7fb8441550f0 2024-01-24T15:14:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.038+0000 7fb84a58a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440a6ce0 con 0x7fb8441540a0 2024-01-24T15:14:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.039+0000 7fb8437fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb8441550f0 0x7fb8440627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.039+0000 7fb8437fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb8441550f0 0x7fb8440627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39280/0 (socket says 172.21.15.29:39280) 2024-01-24T15:14:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.039+0000 7fb8437fe700 1 -- 172.21.15.29:0/3038374243 learned_addr learned my addr 172.21.15.29:0/3038374243 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.039+0000 7fb843fff700 1 --2- 172.21.15.29:0/3038374243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8441540a0 0x7fb8440c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.039+0000 7fb8437fe700 1 -- 172.21.15.29:0/3038374243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8441540a0 msgr2=0x7fb8440c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.039+0000 7fb8437fe700 1 --2- 172.21.15.29:0/3038374243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8441540a0 0x7fb8440c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.039+0000 7fb8437fe700 1 -- 172.21.15.29:0/3038374243 >> v1:172.21.15.163:6789/0 conn(0x7fb844062ca0 legacy=0x7fb844157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.039+0000 7fb8437fe700 1 -- 172.21.15.29:0/3038374243 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb844063130 con 0x7fb8441550f0 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.039+0000 7fb8437fe700 1 --2- 172.21.15.29:0/3038374243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8441550f0 0x7fb8440627b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fb838006fd0 tx=0x7fb8380090f0).ready entity=mon.0 client_cookie=130ec88912e467cd server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.039+0000 7fb842ffd700 1 -- 172.21.15.29:0/3038374243 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83800b600 con 0x7fb8441550f0 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.039+0000 7fb842ffd700 1 -- 172.21.15.29:0/3038374243 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83800b760 con 0x7fb8441550f0 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.039+0000 7fb842ffd700 1 -- 172.21.15.29:0/3038374243 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83800bae0 con 0x7fb8441550f0 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.040+0000 7fb84a58a700 1 -- 172.21.15.29:0/3038374243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8441550f0 msgr2=0x7fb8440627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.040+0000 7fb84a58a700 1 --2- 172.21.15.29:0/3038374243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8441550f0 0x7fb8440627b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fb838006fd0 tx=0x7fb8380090f0).stop 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.040+0000 7fb84a58a700 1 -- 172.21.15.29:0/3038374243 shutdown_connections 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.040+0000 7fb84a58a700 1 --2- 172.21.15.29:0/3038374243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8441540a0 0x7fb8440c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.040+0000 7fb84a58a700 1 --2- 172.21.15.29:0/3038374243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8441550f0 0x7fb8440627b0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.040+0000 7fb84a58a700 1 -- 172.21.15.29:0/3038374243 >> 172.21.15.29:0/3038374243 conn(0x7fb8440c85c0 msgr2=0x7fb8440c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.040+0000 7fb84a58a700 1 -- 172.21.15.29:0/3038374243 shutdown_connections 2024-01-24T15:14:21.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.040+0000 7fb84a58a700 1 -- 172.21.15.29:0/3038374243 wait complete. 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.041+0000 7fb84a58a700 1 Processor -- start 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.042+0000 7fb84a58a700 1 -- start start 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.042+0000 7fb84a58a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb844062ca0 0x7fb84414a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.042+0000 7fb84a58a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8441540a0 0x7fb8440aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.042+0000 7fb84a58a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8441550f0 0x7fb8440aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.042+0000 7fb84a58a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440a7980 con 0x7fb8441540a0 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.042+0000 7fb84a58a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440a7630 con 0x7fb8441550f0 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.042+0000 7fb84a58a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440ab5a0 con 0x7fb844062ca0 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.042+0000 7fb8437fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8441540a0 0x7fb8440aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.042+0000 7fb8437fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8441540a0 0x7fb8440aaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39288/0 (socket says 172.21.15.29:39288) 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.042+0000 7fb8437fe700 1 -- 172.21.15.29:0/1736635438 learned_addr learned my addr 172.21.15.29:0/1736635438 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb843fff700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb844062ca0 0x7fb84414a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb848b27700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8441550f0 0x7fb8440aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb8437fe700 1 -- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb844062ca0 msgr2=0x7fb84414a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb8437fe700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb844062ca0 0x7fb84414a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb8437fe700 1 -- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8441550f0 msgr2=0x7fb8440aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb8437fe700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8441550f0 0x7fb8440aaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb8437fe700 1 -- 172.21.15.29:0/1736635438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb838009370 con 0x7fb8441540a0 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb8437fe700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8441540a0 0x7fb8440aaaa0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7fb838011000 tx=0x7fb838017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb82bfff700 1 -- 172.21.15.29:0/1736635438 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83800b8d0 con 0x7fb8441540a0 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb82bfff700 1 -- 172.21.15.29:0/1736635438 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83800ba30 con 0x7fb8441540a0 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb84a58a700 1 -- 172.21.15.29:0/1736635438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8440ab880 con 0x7fb8441540a0 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb82bfff700 1 -- 172.21.15.29:0/1736635438 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83801c670 con 0x7fb8441540a0 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.043+0000 7fb84a58a700 1 -- 172.21.15.29:0/1736635438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8440a9b20 con 0x7fb8441540a0 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.044+0000 7fb84a58a700 1 -- 172.21.15.29:0/1736635438 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8440bd010 con 0x7fb8441540a0 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.045+0000 7fb82bfff700 1 -- 172.21.15.29:0/1736635438 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb83801ca20 con 0x7fb8441540a0 2024-01-24T15:14:21.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.045+0000 7fb82bfff700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb83005f240 0x7fb830061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.045+0000 7fb82bfff700 1 -- 172.21.15.29:0/1736635438 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fb83807c0f0 con 0x7fb8441540a0 2024-01-24T15:14:21.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.045+0000 7fb843fff700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb83005f240 0x7fb830061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.046+0000 7fb843fff700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb83005f240 0x7fb830061700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb83400bb20 tx=0x7fb834008040).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:21.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.049+0000 7fb82bfff700 1 -- 172.21.15.29:0/1736635438 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb83804ce10 con 0x7fb8441540a0 2024-01-24T15:14:21.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.225+0000 7fb84a58a700 1 -- 172.21.15.29:0/1736635438 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8440a7ac0 con 0x7fb83005f240 2024-01-24T15:14:21.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.236+0000 7fb82bfff700 1 -- 172.21.15.29:0/1736635438 <== mgr.7535 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361931 (secure 0 0 0) 0x7fb8440a7ac0 con 0x7fb83005f240 2024-01-24T15:14:21.242 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:21.245 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.240+0000 7fb84a58a700 1 -- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb83005f240 msgr2=0x7fb830061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.240+0000 7fb84a58a700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb83005f240 0x7fb830061700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb83400bb20 tx=0x7fb834008040).stop 2024-01-24T15:14:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.240+0000 7fb84a58a700 1 -- 172.21.15.29:0/1736635438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8441540a0 msgr2=0x7fb8440aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.240+0000 7fb84a58a700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8441540a0 0x7fb8440aaaa0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7fb838011000 tx=0x7fb838017000).stop 2024-01-24T15:14:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.240+0000 7fb84a58a700 1 -- 172.21.15.29:0/1736635438 shutdown_connections 2024-01-24T15:14:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.240+0000 7fb84a58a700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8441550f0 0x7fb8440aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.240+0000 7fb84a58a700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb844062ca0 0x7fb84414a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.240+0000 7fb84a58a700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fb83005f240 0x7fb830061700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.240+0000 7fb84a58a700 1 --2- 172.21.15.29:0/1736635438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8441540a0 0x7fb8440aaaa0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.240+0000 7fb84a58a700 1 -- 172.21.15.29:0/1736635438 >> 172.21.15.29:0/1736635438 conn(0x7fb8440c85c0 msgr2=0x7fb8440c6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.240+0000 7fb84a58a700 1 -- 172.21.15.29:0/1736635438 shutdown_connections 2024-01-24T15:14:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.240+0000 7fb84a58a700 1 -- 172.21.15.29:0/1736635438 wait complete. 2024-01-24T15:14:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:14:21.255 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-01-24T15:14:21.187725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2461156,"kb_used_data":18852,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752513564,"statfs":{"total":773094113280,"available":770573889536,"internally_reserved":0,"allocated":19304448,"data_stored":11068529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11997,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-46,"num_write_kb":-32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.011490"},"pg_stats":[{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.708714+0000","last_change":"2024-01-24T15:14:16.705752+0000","last_active":"2024-01-24T15:14:16.708714+0000","last_peered":"2024-01-24T15:14:16.708714+0000","last_clean":"2024-01-24T15:14:16.708714+0000","last_became_active":"2024-01-24T15:14:16.705423+0000","last_became_peered":"2024-01-24T15:14:16.705423+0000","last_unstale":"2024-01-24T15:14:16.708714+0000","last_undegraded":"2024-01-24T15:14:16.708714+0000","last_fullsized":"2024-01-24T15:14:16.708714+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3f","version":"192'3","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.707034+0000","last_change":"2024-01-24T15:14:16.276298+0000","last_active":"2024-01-24T15:14:16.707034+0000","last_peered":"2024-01-24T15:14:16.707034+0000","last_clean":"2024-01-24T15:14:16.707034+0000","last_became_active":"2024-01-24T15:14:16.275997+0000","last_became_peered":"2024-01-24T15:14:16.275997+0000","last_unstale":"2024-01-24T15:14:16.707034+0000","last_undegraded":"2024-01-24T15:14:16.707034+0000","last_fullsized":"2024-01-24T15:14:16.707034+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.700849+0000","last_change":"2024-01-24T15:14:16.696738+0000","last_active":"2024-01-24T15:14:16.700849+0000","last_peered":"2024-01-24T15:14:16.700849+0000","last_clean":"2024-01-24T15:14:16.700849+0000","last_became_active":"2024-01-24T15:14:16.696312+0000","last_became_peered":"2024-01-24T15:14:16.696312+0000","last_unstale":"2024-01-24T15:14:16.700849+0000","last_undegraded":"2024-01-24T15:14:16.700849+0000","last_fullsized":"2024-01-24T15:14:16.700849+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.700501+0000","last_change":"2024-01-24T15:14:16.283141+0000","last_active":"2024-01-24T15:14:16.700501+0000","last_peered":"2024-01-24T15:14:16.700501+0000","last_clean":"2024-01-24T15:14:16.700501+0000","last_became_active":"2024-01-24T15:14:16.282851+0000","last_became_peered":"2024-01-24T15:14:16.282851+0000","last_unstale":"2024-01-24T15:14:16.700501+0000","last_undegraded":"2024-01-24T15:14:16.700501+0000","last_fullsized":"2024-01-24T15:14:16.700501+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.707574+0000","last_change":"2024-01-24T15:14:16.702913+0000","last_active":"2024-01-24T15:14:16.707574+0000","last_peered":"2024-01-24T15:14:16.707574+0000","last_clean":"2024-01-24T15:14:16.707574+0000","last_became_active":"2024-01-24T15:14:16.702684+0000","last_became_peered":"2024-01-24T15:14:16.702684+0000","last_unstale":"2024-01-24T15:14:16.707574+0000","last_undegraded":"2024-01-24T15:14:16.707574+0000","last_fullsized":"2024-01-24T15:14:16.707574+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.696163+0000","last_change":"2024-01-24T15:14:16.275496+0000","last_active":"2024-01-24T15:14:16.696163+0000","last_peered":"2024-01-24T15:14:16.696163+0000","last_clean":"2024-01-24T15:14:16.696163+0000","last_became_active":"2024-01-24T15:14:16.275186+0000","last_became_peered":"2024-01-24T15:14:16.275186+0000","last_unstale":"2024-01-24T15:14:16.696163+0000","last_undegraded":"2024-01-24T15:14:16.696163+0000","last_fullsized":"2024-01-24T15:14:16.696163+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.692935+0000","last_change":"2024-01-24T15:14:16.688711+0000","last_active":"2024-01-24T15:14:16.692935+0000","last_peered":"2024-01-24T15:14:16.692935+0000","last_clean":"2024-01-24T15:14:16.692935+0000","last_became_active":"2024-01-24T15:14:16.688393+0000","last_became_peered":"2024-01-24T15:14:16.688393+0000","last_unstale":"2024-01-24T15:14:16.692935+0000","last_undegraded":"2024-01-24T15:14:16.692935+0000","last_fullsized":"2024-01-24T15:14:16.692935+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3c","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.708731+0000","last_change":"2024-01-24T15:14:16.281800+0000","last_active":"2024-01-24T15:14:16.708731+0000","last_peered":"2024-01-24T15:14:16.708731+0000","last_clean":"2024-01-24T15:14:16.708731+0000","last_became_active":"2024-01-24T15:14:16.281589+0000","last_became_peered":"2024-01-24T15:14:16.281589+0000","last_unstale":"2024-01-24T15:14:16.708731+0000","last_undegraded":"2024-01-24T15:14:16.708731+0000","last_fullsized":"2024-01-24T15:14:16.708731+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.712181+0000","last_change":"2024-01-24T15:14:16.709796+0000","last_active":"2024-01-24T15:14:16.712181+0000","last_peered":"2024-01-24T15:14:16.712181+0000","last_clean":"2024-01-24T15:14:16.712181+0000","last_became_active":"2024-01-24T15:14:16.709471+0000","last_became_peered":"2024-01-24T15:14:16.709471+0000","last_unstale":"2024-01-24T15:14:16.712181+0000","last_undegraded":"2024-01-24T15:14:16.712181+0000","last_fullsized":"2024-01-24T15:14:16.712181+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.3b","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701962+0000","last_change":"2024-01-24T15:14:16.283517+0000","last_active":"2024-01-24T15:14:16.701962+0000","last_peered":"2024-01-24T15:14:16.701962+0000","last_clean":"2024-01-24T15:14:16.701962+0000","last_became_active":"2024-01-24T15:14:16.282222+0000","last_became_peered":"2024-01-24T15:14:16.282222+0000","last_unstale":"2024-01-24T15:14:16.701962+0000","last_undegraded":"2024-01-24T15:14:16.701962+0000","last_fullsized":"2024-01-24T15:14:16.701962+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.709106+0000","last_change":"2024-01-24T15:14:16.708164+0000","last_active":"2024-01-24T15:14:16.709106+0000","last_peered":"2024-01-24T15:14:16.709106+0000","last_clean":"2024-01-24T15:14:16.709106+0000","last_became_active":"2024-01-24T15:14:16.707817+0000","last_became_peered":"2024-01-24T15:14:16.707817+0000","last_unstale":"2024-01-24T15:14:16.709106+0000","last_undegraded":"2024-01-24T15:14:16.709106+0000","last_fullsized":"2024-01-24T15:14:16.709106+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3a","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701654+0000","last_change":"2024-01-24T15:14:16.284621+0000","last_active":"2024-01-24T15:14:16.701654+0000","last_peered":"2024-01-24T15:14:16.701654+0000","last_clean":"2024-01-24T15:14:16.701654+0000","last_became_active":"2024-01-24T15:14:16.284254+0000","last_became_peered":"2024-01-24T15:14:16.284254+0000","last_unstale":"2024-01-24T15:14:16.701654+0000","last_undegraded":"2024-01-24T15:14:16.701654+0000","last_fullsized":"2024-01-24T15:14:16.701654+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.710584+0000","last_change":"2024-01-24T15:14:16.709055+0000","last_active":"2024-01-24T15:14:16.710584+0000","last_peered":"2024-01-24T15:14:16.710584+0000","last_clean":"2024-01-24T15:14:16.710584+0000","last_became_active":"2024-01-24T15:14:16.708772+0000","last_became_peered":"2024-01-24T15:14:16.708772+0000","last_unstale":"2024-01-24T15:14:16.710584+0000","last_undegraded":"2024-01-24T15:14:16.710584+0000","last_fullsized":"2024-01-24T15:14:16.710584+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.684424+0000","last_change":"2024-01-24T15:14:16.272376+0000","last_active":"2024-01-24T15:14:16.684424+0000","last_peered":"2024-01-24T15:14:16.684424+0000","last_clean":"2024-01-24T15:14:16.684424+0000","last_became_active":"2024-01-24T15:14:16.272047+0000","last_became_peered":"2024-01-24T15:14:16.272047+0000","last_unstale":"2024-01-24T15:14:16.684424+0000","last_undegraded":"2024-01-24T15:14:16.684424+0000","last_fullsized":"2024-01-24T15:14:16.684424+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.711732+0000","last_change":"2024-01-24T15:14:16.704612+0000","last_active":"2024-01-24T15:14:16.711732+0000","last_peered":"2024-01-24T15:14:16.711732+0000","last_clean":"2024-01-24T15:14:16.711732+0000","last_became_active":"2024-01-24T15:14:16.704295+0000","last_became_peered":"2024-01-24T15:14:16.704295+0000","last_unstale":"2024-01-24T15:14:16.711732+0000","last_undegraded":"2024-01-24T15:14:16.711732+0000","last_fullsized":"2024-01-24T15:14:16.711732+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.683842+0000","last_change":"2024-01-24T15:14:16.274503+0000","last_active":"2024-01-24T15:14:16.683842+0000","last_peered":"2024-01-24T15:14:16.683842+0000","last_clean":"2024-01-24T15:14:16.683842+0000","last_became_active":"2024-01-24T15:14:16.271720+0000","last_became_peered":"2024-01-24T15:14:16.271720+0000","last_unstale":"2024-01-24T15:14:16.683842+0000","last_undegraded":"2024-01-24T15:14:16.683842+0000","last_fullsized":"2024-01-24T15:14:16.683842+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.710605+0000","last_change":"2024-01-24T15:14:16.709524+0000","last_active":"2024-01-24T15:14:16.710605+0000","last_peered":"2024-01-24T15:14:16.710605+0000","last_clean":"2024-01-24T15:14:16.710605+0000","last_became_active":"2024-01-24T15:14:16.709298+0000","last_became_peered":"2024-01-24T15:14:16.709298+0000","last_unstale":"2024-01-24T15:14:16.710605+0000","last_undegraded":"2024-01-24T15:14:16.710605+0000","last_fullsized":"2024-01-24T15:14:16.710605+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.683853+0000","last_change":"2024-01-24T15:14:16.279818+0000","last_active":"2024-01-24T15:14:16.683853+0000","last_peered":"2024-01-24T15:14:16.683853+0000","last_clean":"2024-01-24T15:14:16.683853+0000","last_became_active":"2024-01-24T15:14:16.279516+0000","last_became_peered":"2024-01-24T15:14:16.279516+0000","last_unstale":"2024-01-24T15:14:16.683853+0000","last_undegraded":"2024-01-24T15:14:16.683853+0000","last_fullsized":"2024-01-24T15:14:16.683853+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.709483+0000","last_change":"2024-01-24T15:14:16.704696+0000","last_active":"2024-01-24T15:14:16.709483+0000","last_peered":"2024-01-24T15:14:16.709483+0000","last_clean":"2024-01-24T15:14:16.709483+0000","last_became_active":"2024-01-24T15:14:16.704392+0000","last_became_peered":"2024-01-24T15:14:16.704392+0000","last_unstale":"2024-01-24T15:14:16.709483+0000","last_undegraded":"2024-01-24T15:14:16.709483+0000","last_fullsized":"2024-01-24T15:14:16.709483+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.686533+0000","last_change":"2024-01-24T15:14:16.279986+0000","last_active":"2024-01-24T15:14:16.686533+0000","last_peered":"2024-01-24T15:14:16.686533+0000","last_clean":"2024-01-24T15:14:16.686533+0000","last_became_active":"2024-01-24T15:14:16.279707+0000","last_became_peered":"2024-01-24T15:14:16.279707+0000","last_unstale":"2024-01-24T15:14:16.686533+0000","last_undegraded":"2024-01-24T15:14:16.686533+0000","last_fullsized":"2024-01-24T15:14:16.686533+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.708810+0000","last_change":"2024-01-24T15:14:16.707168+0000","last_active":"2024-01-24T15:14:16.708810+0000","last_peered":"2024-01-24T15:14:16.708810+0000","last_clean":"2024-01-24T15:14:16.708810+0000","last_became_active":"2024-01-24T15:14:16.706912+0000","last_became_peered":"2024-01-24T15:14:16.706912+0000","last_unstale":"2024-01-24T15:14:16.708810+0000","last_undegraded":"2024-01-24T15:14:16.708810+0000","last_fullsized":"2024-01-24T15:14:16.708810+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.678432+0000","last_change":"2024-01-24T15:14:16.277496+0000","last_active":"2024-01-24T15:14:16.678432+0000","last_peered":"2024-01-24T15:14:16.678432+0000","last_clean":"2024-01-24T15:14:16.678432+0000","last_became_active":"2024-01-24T15:14:16.276796+0000","last_became_peered":"2024-01-24T15:14:16.276796+0000","last_unstale":"2024-01-24T15:14:16.678432+0000","last_undegraded":"2024-01-24T15:14:16.678432+0000","last_fullsized":"2024-01-24T15:14:16.678432+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.708724+0000","last_change":"2024-01-24T15:14:16.705977+0000","last_active":"2024-01-24T15:14:16.708724+0000","last_peered":"2024-01-24T15:14:16.708724+0000","last_clean":"2024-01-24T15:14:16.708724+0000","last_became_active":"2024-01-24T15:14:16.705700+0000","last_became_peered":"2024-01-24T15:14:16.705700+0000","last_unstale":"2024-01-24T15:14:16.708724+0000","last_undegraded":"2024-01-24T15:14:16.708724+0000","last_fullsized":"2024-01-24T15:14:16.708724+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.34","version":"192'4","reported_seq":14,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.716660+0000","last_change":"2024-01-24T15:14:16.284083+0000","last_active":"2024-01-24T15:14:16.716660+0000","last_peered":"2024-01-24T15:14:16.716660+0000","last_clean":"2024-01-24T15:14:16.716660+0000","last_became_active":"2024-01-24T15:14:16.283814+0000","last_became_peered":"2024-01-24T15:14:16.283814+0000","last_unstale":"2024-01-24T15:14:16.716660+0000","last_undegraded":"2024-01-24T15:14:16.716660+0000","last_fullsized":"2024-01-24T15:14:16.716660+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.697133+0000","last_change":"2024-01-24T15:14:16.694969+0000","last_active":"2024-01-24T15:14:16.697133+0000","last_peered":"2024-01-24T15:14:16.697133+0000","last_clean":"2024-01-24T15:14:16.697133+0000","last_became_active":"2024-01-24T15:14:16.694286+0000","last_became_peered":"2024-01-24T15:14:16.694286+0000","last_unstale":"2024-01-24T15:14:16.697133+0000","last_undegraded":"2024-01-24T15:14:16.697133+0000","last_fullsized":"2024-01-24T15:14:16.697133+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.33","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702361+0000","last_change":"2024-01-24T15:14:16.283115+0000","last_active":"2024-01-24T15:14:16.702361+0000","last_peered":"2024-01-24T15:14:16.702361+0000","last_clean":"2024-01-24T15:14:16.702361+0000","last_became_active":"2024-01-24T15:14:16.281456+0000","last_became_peered":"2024-01-24T15:14:16.281456+0000","last_unstale":"2024-01-24T15:14:16.702361+0000","last_undegraded":"2024-01-24T15:14:16.702361+0000","last_fullsized":"2024-01-24T15:14:16.702361+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.696881+0000","last_change":"2024-01-24T15:14:16.694638+0000","last_active":"2024-01-24T15:14:16.696881+0000","last_peered":"2024-01-24T15:14:16.696881+0000","last_clean":"2024-01-24T15:14:16.696881+0000","last_became_active":"2024-01-24T15:14:16.694358+0000","last_became_peered":"2024-01-24T15:14:16.694358+0000","last_unstale":"2024-01-24T15:14:16.696881+0000","last_undegraded":"2024-01-24T15:14:16.696881+0000","last_fullsized":"2024-01-24T15:14:16.696881+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701262+0000","last_change":"2024-01-24T15:14:16.288376+0000","last_active":"2024-01-24T15:14:16.701262+0000","last_peered":"2024-01-24T15:14:16.701262+0000","last_clean":"2024-01-24T15:14:16.701262+0000","last_became_active":"2024-01-24T15:14:16.288113+0000","last_became_peered":"2024-01-24T15:14:16.288113+0000","last_unstale":"2024-01-24T15:14:16.701262+0000","last_undegraded":"2024-01-24T15:14:16.701262+0000","last_fullsized":"2024-01-24T15:14:16.701262+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.709378+0000","last_change":"2024-01-24T15:14:16.701448+0000","last_active":"2024-01-24T15:14:16.709378+0000","last_peered":"2024-01-24T15:14:16.709378+0000","last_clean":"2024-01-24T15:14:16.709378+0000","last_became_active":"2024-01-24T15:14:16.701158+0000","last_became_peered":"2024-01-24T15:14:16.701158+0000","last_unstale":"2024-01-24T15:14:16.709378+0000","last_undegraded":"2024-01-24T15:14:16.709378+0000","last_fullsized":"2024-01-24T15:14:16.709378+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.693394+0000","last_change":"2024-01-24T15:14:16.283606+0000","last_active":"2024-01-24T15:14:16.693394+0000","last_peered":"2024-01-24T15:14:16.693394+0000","last_clean":"2024-01-24T15:14:16.693394+0000","last_became_active":"2024-01-24T15:14:16.283112+0000","last_became_peered":"2024-01-24T15:14:16.283112+0000","last_unstale":"2024-01-24T15:14:16.693394+0000","last_undegraded":"2024-01-24T15:14:16.693394+0000","last_fullsized":"2024-01-24T15:14:16.693394+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.709292+0000","last_change":"2024-01-24T15:14:16.707853+0000","last_active":"2024-01-24T15:14:16.709292+0000","last_peered":"2024-01-24T15:14:16.709292+0000","last_clean":"2024-01-24T15:14:16.709292+0000","last_became_active":"2024-01-24T15:14:16.707560+0000","last_became_peered":"2024-01-24T15:14:16.707560+0000","last_unstale":"2024-01-24T15:14:16.709292+0000","last_undegraded":"2024-01-24T15:14:16.709292+0000","last_fullsized":"2024-01-24T15:14:16.709292+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.30","version":"192'4","reported_seq":14,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.707118+0000","last_change":"2024-01-24T15:14:16.277935+0000","last_active":"2024-01-24T15:14:16.707118+0000","last_peered":"2024-01-24T15:14:16.707118+0000","last_clean":"2024-01-24T15:14:16.707118+0000","last_became_active":"2024-01-24T15:14:16.277594+0000","last_became_peered":"2024-01-24T15:14:16.277594+0000","last_unstale":"2024-01-24T15:14:16.707118+0000","last_undegraded":"2024-01-24T15:14:16.707118+0000","last_fullsized":"2024-01-24T15:14:16.707118+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.707215+0000","last_change":"2024-01-24T15:14:16.703112+0000","last_active":"2024-01-24T15:14:16.707215+0000","last_peered":"2024-01-24T15:14:16.707215+0000","last_clean":"2024-01-24T15:14:16.707215+0000","last_became_active":"2024-01-24T15:14:16.702682+0000","last_became_peered":"2024-01-24T15:14:16.702682+0000","last_unstale":"2024-01-24T15:14:16.707215+0000","last_undegraded":"2024-01-24T15:14:16.707215+0000","last_fullsized":"2024-01-24T15:14:16.707215+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.704965+0000","last_change":"2024-01-24T15:14:16.284874+0000","last_active":"2024-01-24T15:14:16.704965+0000","last_peered":"2024-01-24T15:14:16.704965+0000","last_clean":"2024-01-24T15:14:16.704965+0000","last_became_active":"2024-01-24T15:14:16.284577+0000","last_became_peered":"2024-01-24T15:14:16.284577+0000","last_unstale":"2024-01-24T15:14:16.704965+0000","last_undegraded":"2024-01-24T15:14:16.704965+0000","last_fullsized":"2024-01-24T15:14:16.704965+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.693115+0000","last_change":"2024-01-24T15:14:16.688384+0000","last_active":"2024-01-24T15:14:16.693115+0000","last_peered":"2024-01-24T15:14:16.693115+0000","last_clean":"2024-01-24T15:14:16.693115+0000","last_became_active":"2024-01-24T15:14:16.686041+0000","last_became_peered":"2024-01-24T15:14:16.686041+0000","last_unstale":"2024-01-24T15:14:16.693115+0000","last_undegraded":"2024-01-24T15:14:16.693115+0000","last_fullsized":"2024-01-24T15:14:16.693115+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.677801+0000","last_change":"2024-01-24T15:14:16.270443+0000","last_active":"2024-01-24T15:14:16.677801+0000","last_peered":"2024-01-24T15:14:16.677801+0000","last_clean":"2024-01-24T15:14:16.677801+0000","last_became_active":"2024-01-24T15:14:16.270104+0000","last_became_peered":"2024-01-24T15:14:16.270104+0000","last_unstale":"2024-01-24T15:14:16.677801+0000","last_undegraded":"2024-01-24T15:14:16.677801+0000","last_fullsized":"2024-01-24T15:14:16.677801+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.696910+0000","last_change":"2024-01-24T15:14:16.693988+0000","last_active":"2024-01-24T15:14:16.696910+0000","last_peered":"2024-01-24T15:14:16.696910+0000","last_clean":"2024-01-24T15:14:16.696910+0000","last_became_active":"2024-01-24T15:14:16.693707+0000","last_became_peered":"2024-01-24T15:14:16.693707+0000","last_unstale":"2024-01-24T15:14:16.696910+0000","last_undegraded":"2024-01-24T15:14:16.696910+0000","last_fullsized":"2024-01-24T15:14:16.696910+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2d","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.705712+0000","last_change":"2024-01-24T15:14:16.274139+0000","last_active":"2024-01-24T15:14:16.705712+0000","last_peered":"2024-01-24T15:14:16.705712+0000","last_clean":"2024-01-24T15:14:16.705712+0000","last_became_active":"2024-01-24T15:14:16.273456+0000","last_became_peered":"2024-01-24T15:14:16.273456+0000","last_unstale":"2024-01-24T15:14:16.705712+0000","last_undegraded":"2024-01-24T15:14:16.705712+0000","last_fullsized":"2024-01-24T15:14:16.705712+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.690691+0000","last_change":"2024-01-24T15:14:16.686134+0000","last_active":"2024-01-24T15:14:16.690691+0000","last_peered":"2024-01-24T15:14:16.690691+0000","last_clean":"2024-01-24T15:14:16.690691+0000","last_became_active":"2024-01-24T15:14:16.685894+0000","last_became_peered":"2024-01-24T15:14:16.685894+0000","last_unstale":"2024-01-24T15:14:16.690691+0000","last_undegraded":"2024-01-24T15:14:16.690691+0000","last_fullsized":"2024-01-24T15:14:16.690691+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.694477+0000","last_change":"2024-01-24T15:14:16.276270+0000","last_active":"2024-01-24T15:14:16.694477+0000","last_peered":"2024-01-24T15:14:16.694477+0000","last_clean":"2024-01-24T15:14:16.694477+0000","last_became_active":"2024-01-24T15:14:16.275940+0000","last_became_peered":"2024-01-24T15:14:16.275940+0000","last_unstale":"2024-01-24T15:14:16.694477+0000","last_undegraded":"2024-01-24T15:14:16.694477+0000","last_fullsized":"2024-01-24T15:14:16.694477+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.712819+0000","last_change":"2024-01-24T15:14:16.712259+0000","last_active":"2024-01-24T15:14:16.712819+0000","last_peered":"2024-01-24T15:14:16.712819+0000","last_clean":"2024-01-24T15:14:16.712819+0000","last_became_active":"2024-01-24T15:14:16.711918+0000","last_became_peered":"2024-01-24T15:14:16.711918+0000","last_unstale":"2024-01-24T15:14:16.712819+0000","last_undegraded":"2024-01-24T15:14:16.712819+0000","last_fullsized":"2024-01-24T15:14:16.712819+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2b","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702248+0000","last_change":"2024-01-24T15:14:16.283313+0000","last_active":"2024-01-24T15:14:16.702248+0000","last_peered":"2024-01-24T15:14:16.702248+0000","last_clean":"2024-01-24T15:14:16.702248+0000","last_became_active":"2024-01-24T15:14:16.281802+0000","last_became_peered":"2024-01-24T15:14:16.281802+0000","last_unstale":"2024-01-24T15:14:16.702248+0000","last_undegraded":"2024-01-24T15:14:16.702248+0000","last_fullsized":"2024-01-24T15:14:16.702248+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.687649+0000","last_change":"2024-01-24T15:14:16.685690+0000","last_active":"2024-01-24T15:14:16.687649+0000","last_peered":"2024-01-24T15:14:16.687649+0000","last_clean":"2024-01-24T15:14:16.687649+0000","last_became_active":"2024-01-24T15:14:16.685430+0000","last_became_peered":"2024-01-24T15:14:16.685430+0000","last_unstale":"2024-01-24T15:14:16.687649+0000","last_undegraded":"2024-01-24T15:14:16.687649+0000","last_fullsized":"2024-01-24T15:14:16.687649+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.685265+0000","last_change":"2024-01-24T15:14:16.269315+0000","last_active":"2024-01-24T15:14:16.685265+0000","last_peered":"2024-01-24T15:14:16.685265+0000","last_clean":"2024-01-24T15:14:16.685265+0000","last_became_active":"2024-01-24T15:14:16.269009+0000","last_became_peered":"2024-01-24T15:14:16.269009+0000","last_unstale":"2024-01-24T15:14:16.685265+0000","last_undegraded":"2024-01-24T15:14:16.685265+0000","last_fullsized":"2024-01-24T15:14:16.685265+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.708957+0000","last_change":"2024-01-24T15:14:16.706632+0000","last_active":"2024-01-24T15:14:16.708957+0000","last_peered":"2024-01-24T15:14:16.708957+0000","last_clean":"2024-01-24T15:14:16.708957+0000","last_became_active":"2024-01-24T15:14:16.705497+0000","last_became_peered":"2024-01-24T15:14:16.705497+0000","last_unstale":"2024-01-24T15:14:16.708957+0000","last_undegraded":"2024-01-24T15:14:16.708957+0000","last_fullsized":"2024-01-24T15:14:16.708957+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.683170+0000","last_change":"2024-01-24T15:14:16.267120+0000","last_active":"2024-01-24T15:14:16.683170+0000","last_peered":"2024-01-24T15:14:16.683170+0000","last_clean":"2024-01-24T15:14:16.683170+0000","last_became_active":"2024-01-24T15:14:16.266784+0000","last_became_peered":"2024-01-24T15:14:16.266784+0000","last_unstale":"2024-01-24T15:14:16.683170+0000","last_undegraded":"2024-01-24T15:14:16.683170+0000","last_fullsized":"2024-01-24T15:14:16.683170+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.706771+0000","last_change":"2024-01-24T15:14:16.705525+0000","last_active":"2024-01-24T15:14:16.706771+0000","last_peered":"2024-01-24T15:14:16.706771+0000","last_clean":"2024-01-24T15:14:16.706771+0000","last_became_active":"2024-01-24T15:14:16.705230+0000","last_became_peered":"2024-01-24T15:14:16.705230+0000","last_unstale":"2024-01-24T15:14:16.706771+0000","last_undegraded":"2024-01-24T15:14:16.706771+0000","last_fullsized":"2024-01-24T15:14:16.706771+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.678684+0000","last_change":"2024-01-24T15:14:16.271054+0000","last_active":"2024-01-24T15:14:16.678684+0000","last_peered":"2024-01-24T15:14:16.678684+0000","last_clean":"2024-01-24T15:14:16.678684+0000","last_became_active":"2024-01-24T15:14:16.270755+0000","last_became_peered":"2024-01-24T15:14:16.270755+0000","last_unstale":"2024-01-24T15:14:16.678684+0000","last_undegraded":"2024-01-24T15:14:16.678684+0000","last_fullsized":"2024-01-24T15:14:16.678684+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.711751+0000","last_change":"2024-01-24T15:14:16.707388+0000","last_active":"2024-01-24T15:14:16.711751+0000","last_peered":"2024-01-24T15:14:16.711751+0000","last_clean":"2024-01-24T15:14:16.711751+0000","last_became_active":"2024-01-24T15:14:16.707083+0000","last_became_peered":"2024-01-24T15:14:16.707083+0000","last_unstale":"2024-01-24T15:14:16.711751+0000","last_undegraded":"2024-01-24T15:14:16.711751+0000","last_fullsized":"2024-01-24T15:14:16.711751+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.686899+0000","last_change":"2024-01-24T15:14:16.275667+0000","last_active":"2024-01-24T15:14:16.686899+0000","last_peered":"2024-01-24T15:14:16.686899+0000","last_clean":"2024-01-24T15:14:16.686899+0000","last_became_active":"2024-01-24T15:14:16.275338+0000","last_became_peered":"2024-01-24T15:14:16.275338+0000","last_unstale":"2024-01-24T15:14:16.686899+0000","last_undegraded":"2024-01-24T15:14:16.686899+0000","last_fullsized":"2024-01-24T15:14:16.686899+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.692696+0000","last_change":"2024-01-24T15:14:16.687779+0000","last_active":"2024-01-24T15:14:16.692696+0000","last_peered":"2024-01-24T15:14:16.692696+0000","last_clean":"2024-01-24T15:14:16.692696+0000","last_became_active":"2024-01-24T15:14:16.687487+0000","last_became_peered":"2024-01-24T15:14:16.687487+0000","last_unstale":"2024-01-24T15:14:16.692696+0000","last_undegraded":"2024-01-24T15:14:16.692696+0000","last_fullsized":"2024-01-24T15:14:16.692696+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.686818+0000","last_change":"2024-01-24T15:14:16.271309+0000","last_active":"2024-01-24T15:14:16.686818+0000","last_peered":"2024-01-24T15:14:16.686818+0000","last_clean":"2024-01-24T15:14:16.686818+0000","last_became_active":"2024-01-24T15:14:16.270771+0000","last_became_peered":"2024-01-24T15:14:16.270771+0000","last_unstale":"2024-01-24T15:14:16.686818+0000","last_undegraded":"2024-01-24T15:14:16.686818+0000","last_fullsized":"2024-01-24T15:14:16.686818+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.712631+0000","last_change":"2024-01-24T15:14:16.711595+0000","last_active":"2024-01-24T15:14:16.712631+0000","last_peered":"2024-01-24T15:14:16.712631+0000","last_clean":"2024-01-24T15:14:16.712631+0000","last_became_active":"2024-01-24T15:14:16.711031+0000","last_became_peered":"2024-01-24T15:14:16.711031+0000","last_unstale":"2024-01-24T15:14:16.712631+0000","last_undegraded":"2024-01-24T15:14:16.712631+0000","last_fullsized":"2024-01-24T15:14:16.712631+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.682654+0000","last_change":"2024-01-24T15:14:16.278084+0000","last_active":"2024-01-24T15:14:16.682654+0000","last_peered":"2024-01-24T15:14:16.682654+0000","last_clean":"2024-01-24T15:14:16.682654+0000","last_became_active":"2024-01-24T15:14:16.277795+0000","last_became_peered":"2024-01-24T15:14:16.277795+0000","last_unstale":"2024-01-24T15:14:16.682654+0000","last_undegraded":"2024-01-24T15:14:16.682654+0000","last_fullsized":"2024-01-24T15:14:16.682654+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.705517+0000","last_change":"2024-01-24T15:14:16.698621+0000","last_active":"2024-01-24T15:14:16.705517+0000","last_peered":"2024-01-24T15:14:16.705517+0000","last_clean":"2024-01-24T15:14:16.705517+0000","last_became_active":"2024-01-24T15:14:16.698318+0000","last_became_peered":"2024-01-24T15:14:16.698318+0000","last_unstale":"2024-01-24T15:14:16.705517+0000","last_undegraded":"2024-01-24T15:14:16.705517+0000","last_fullsized":"2024-01-24T15:14:16.705517+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.678070+0000","last_change":"2024-01-24T15:14:16.278694+0000","last_active":"2024-01-24T15:14:16.678070+0000","last_peered":"2024-01-24T15:14:16.678070+0000","last_clean":"2024-01-24T15:14:16.678070+0000","last_became_active":"2024-01-24T15:14:16.278405+0000","last_became_peered":"2024-01-24T15:14:16.278405+0000","last_unstale":"2024-01-24T15:14:16.678070+0000","last_undegraded":"2024-01-24T15:14:16.678070+0000","last_fullsized":"2024-01-24T15:14:16.678070+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.703206+0000","last_change":"2024-01-24T15:14:16.693978+0000","last_active":"2024-01-24T15:14:16.703206+0000","last_peered":"2024-01-24T15:14:16.703206+0000","last_clean":"2024-01-24T15:14:16.703206+0000","last_became_active":"2024-01-24T15:14:16.693681+0000","last_became_peered":"2024-01-24T15:14:16.693681+0000","last_unstale":"2024-01-24T15:14:16.703206+0000","last_undegraded":"2024-01-24T15:14:16.703206+0000","last_fullsized":"2024-01-24T15:14:16.703206+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.684283+0000","last_change":"2024-01-24T15:14:16.286055+0000","last_active":"2024-01-24T15:14:16.684283+0000","last_peered":"2024-01-24T15:14:16.684283+0000","last_clean":"2024-01-24T15:14:16.684283+0000","last_became_active":"2024-01-24T15:14:16.285785+0000","last_became_peered":"2024-01-24T15:14:16.285785+0000","last_unstale":"2024-01-24T15:14:16.684283+0000","last_undegraded":"2024-01-24T15:14:16.684283+0000","last_fullsized":"2024-01-24T15:14:16.684283+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702424+0000","last_change":"2024-01-24T15:14:16.697992+0000","last_active":"2024-01-24T15:14:16.702424+0000","last_peered":"2024-01-24T15:14:16.702424+0000","last_clean":"2024-01-24T15:14:16.702424+0000","last_became_active":"2024-01-24T15:14:16.697716+0000","last_became_peered":"2024-01-24T15:14:16.697716+0000","last_unstale":"2024-01-24T15:14:16.702424+0000","last_undegraded":"2024-01-24T15:14:16.702424+0000","last_fullsized":"2024-01-24T15:14:16.702424+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.678798+0000","last_change":"2024-01-24T15:14:16.281188+0000","last_active":"2024-01-24T15:14:16.678798+0000","last_peered":"2024-01-24T15:14:16.678798+0000","last_clean":"2024-01-24T15:14:16.678798+0000","last_became_active":"2024-01-24T15:14:16.280878+0000","last_became_peered":"2024-01-24T15:14:16.280878+0000","last_unstale":"2024-01-24T15:14:16.678798+0000","last_undegraded":"2024-01-24T15:14:16.678798+0000","last_fullsized":"2024-01-24T15:14:16.678798+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.703314+0000","last_change":"2024-01-24T15:14:16.691985+0000","last_active":"2024-01-24T15:14:16.703314+0000","last_peered":"2024-01-24T15:14:16.703314+0000","last_clean":"2024-01-24T15:14:16.703314+0000","last_became_active":"2024-01-24T15:14:16.691708+0000","last_became_peered":"2024-01-24T15:14:16.691708+0000","last_unstale":"2024-01-24T15:14:16.703314+0000","last_undegraded":"2024-01-24T15:14:16.703314+0000","last_fullsized":"2024-01-24T15:14:16.703314+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.682608+0000","last_change":"2024-01-24T15:14:16.275498+0000","last_active":"2024-01-24T15:14:16.682608+0000","last_peered":"2024-01-24T15:14:16.682608+0000","last_clean":"2024-01-24T15:14:16.682608+0000","last_became_active":"2024-01-24T15:14:16.275112+0000","last_became_peered":"2024-01-24T15:14:16.275112+0000","last_unstale":"2024-01-24T15:14:16.682608+0000","last_undegraded":"2024-01-24T15:14:16.682608+0000","last_fullsized":"2024-01-24T15:14:16.682608+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.711484+0000","last_change":"2024-01-24T15:14:16.710358+0000","last_active":"2024-01-24T15:14:16.711484+0000","last_peered":"2024-01-24T15:14:16.711484+0000","last_clean":"2024-01-24T15:14:16.711484+0000","last_became_active":"2024-01-24T15:14:16.710087+0000","last_became_peered":"2024-01-24T15:14:16.710087+0000","last_unstale":"2024-01-24T15:14:16.711484+0000","last_undegraded":"2024-01-24T15:14:16.711484+0000","last_fullsized":"2024-01-24T15:14:16.711484+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.683729+0000","last_change":"2024-01-24T15:14:16.272629+0000","last_active":"2024-01-24T15:14:16.683729+0000","last_peered":"2024-01-24T15:14:16.683729+0000","last_clean":"2024-01-24T15:14:16.683729+0000","last_became_active":"2024-01-24T15:14:16.272067+0000","last_became_peered":"2024-01-24T15:14:16.272067+0000","last_unstale":"2024-01-24T15:14:16.683729+0000","last_undegraded":"2024-01-24T15:14:16.683729+0000","last_fullsized":"2024-01-24T15:14:16.683729+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.711773+0000","last_change":"2024-01-24T15:14:16.708378+0000","last_active":"2024-01-24T15:14:16.711773+0000","last_peered":"2024-01-24T15:14:16.711773+0000","last_clean":"2024-01-24T15:14:16.711773+0000","last_became_active":"2024-01-24T15:14:16.708041+0000","last_became_peered":"2024-01-24T15:14:16.708041+0000","last_unstale":"2024-01-24T15:14:16.711773+0000","last_undegraded":"2024-01-24T15:14:16.711773+0000","last_fullsized":"2024-01-24T15:14:16.711773+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.696866+0000","last_change":"2024-01-24T15:14:16.277965+0000","last_active":"2024-01-24T15:14:16.696866+0000","last_peered":"2024-01-24T15:14:16.696866+0000","last_clean":"2024-01-24T15:14:16.696866+0000","last_became_active":"2024-01-24T15:14:16.277626+0000","last_became_peered":"2024-01-24T15:14:16.277626+0000","last_unstale":"2024-01-24T15:14:16.696866+0000","last_undegraded":"2024-01-24T15:14:16.696866+0000","last_fullsized":"2024-01-24T15:14:16.696866+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.706182+0000","last_change":"2024-01-24T15:14:16.704248+0000","last_active":"2024-01-24T15:14:16.706182+0000","last_peered":"2024-01-24T15:14:16.706182+0000","last_clean":"2024-01-24T15:14:16.706182+0000","last_became_active":"2024-01-24T15:14:16.703837+0000","last_became_peered":"2024-01-24T15:14:16.703837+0000","last_unstale":"2024-01-24T15:14:16.706182+0000","last_undegraded":"2024-01-24T15:14:16.706182+0000","last_fullsized":"2024-01-24T15:14:16.706182+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.703478+0000","last_change":"2024-01-24T15:14:16.282326+0000","last_active":"2024-01-24T15:14:16.703478+0000","last_peered":"2024-01-24T15:14:16.703478+0000","last_clean":"2024-01-24T15:14:16.703478+0000","last_became_active":"2024-01-24T15:14:16.281995+0000","last_became_peered":"2024-01-24T15:14:16.281995+0000","last_unstale":"2024-01-24T15:14:16.703478+0000","last_undegraded":"2024-01-24T15:14:16.703478+0000","last_fullsized":"2024-01-24T15:14:16.703478+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.695576+0000","last_change":"2024-01-24T15:14:16.693864+0000","last_active":"2024-01-24T15:14:16.695576+0000","last_peered":"2024-01-24T15:14:16.695576+0000","last_clean":"2024-01-24T15:14:16.695576+0000","last_became_active":"2024-01-24T15:14:16.693552+0000","last_became_peered":"2024-01-24T15:14:16.693552+0000","last_unstale":"2024-01-24T15:14:16.695576+0000","last_undegraded":"2024-01-24T15:14:16.695576+0000","last_fullsized":"2024-01-24T15:14:16.695576+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.706737+0000","last_change":"2024-01-24T15:14:16.281027+0000","last_active":"2024-01-24T15:14:16.706737+0000","last_peered":"2024-01-24T15:14:16.706737+0000","last_clean":"2024-01-24T15:14:16.706737+0000","last_became_active":"2024-01-24T15:14:16.280679+0000","last_became_peered":"2024-01-24T15:14:16.280679+0000","last_unstale":"2024-01-24T15:14:16.706737+0000","last_undegraded":"2024-01-24T15:14:16.706737+0000","last_fullsized":"2024-01-24T15:14:16.706737+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.696449+0000","last_change":"2024-01-24T15:14:16.692788+0000","last_active":"2024-01-24T15:14:16.696449+0000","last_peered":"2024-01-24T15:14:16.696449+0000","last_clean":"2024-01-24T15:14:16.696449+0000","last_became_active":"2024-01-24T15:14:16.692044+0000","last_became_peered":"2024-01-24T15:14:16.692044+0000","last_unstale":"2024-01-24T15:14:16.696449+0000","last_undegraded":"2024-01-24T15:14:16.696449+0000","last_fullsized":"2024-01-24T15:14:16.696449+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.696402+0000","last_change":"2024-01-24T15:14:16.277778+0000","last_active":"2024-01-24T15:14:16.696402+0000","last_peered":"2024-01-24T15:14:16.696402+0000","last_clean":"2024-01-24T15:14:16.696402+0000","last_became_active":"2024-01-24T15:14:16.277502+0000","last_became_peered":"2024-01-24T15:14:16.277502+0000","last_unstale":"2024-01-24T15:14:16.696402+0000","last_undegraded":"2024-01-24T15:14:16.696402+0000","last_fullsized":"2024-01-24T15:14:16.696402+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.710506+0000","last_change":"2024-01-24T15:14:16.703933+0000","last_active":"2024-01-24T15:14:16.710506+0000","last_peered":"2024-01-24T15:14:16.710506+0000","last_clean":"2024-01-24T15:14:16.710506+0000","last_became_active":"2024-01-24T15:14:16.703615+0000","last_became_peered":"2024-01-24T15:14:16.703615+0000","last_unstale":"2024-01-24T15:14:16.710506+0000","last_undegraded":"2024-01-24T15:14:16.710506+0000","last_fullsized":"2024-01-24T15:14:16.710506+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.686245+0000","last_change":"2024-01-24T15:14:16.268794+0000","last_active":"2024-01-24T15:14:16.686245+0000","last_peered":"2024-01-24T15:14:16.686245+0000","last_clean":"2024-01-24T15:14:16.686245+0000","last_became_active":"2024-01-24T15:14:16.268498+0000","last_became_peered":"2024-01-24T15:14:16.268498+0000","last_unstale":"2024-01-24T15:14:16.686245+0000","last_undegraded":"2024-01-24T15:14:16.686245+0000","last_fullsized":"2024-01-24T15:14:16.686245+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.712140+0000","last_change":"2024-01-24T15:14:16.709803+0000","last_active":"2024-01-24T15:14:16.712140+0000","last_peered":"2024-01-24T15:14:16.712140+0000","last_clean":"2024-01-24T15:14:16.712140+0000","last_became_active":"2024-01-24T15:14:16.709507+0000","last_became_peered":"2024-01-24T15:14:16.709507+0000","last_unstale":"2024-01-24T15:14:16.712140+0000","last_undegraded":"2024-01-24T15:14:16.712140+0000","last_fullsized":"2024-01-24T15:14:16.712140+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.693254+0000","last_change":"2024-01-24T15:14:16.278597+0000","last_active":"2024-01-24T15:14:16.693254+0000","last_peered":"2024-01-24T15:14:16.693254+0000","last_clean":"2024-01-24T15:14:16.693254+0000","last_became_active":"2024-01-24T15:14:16.278243+0000","last_became_peered":"2024-01-24T15:14:16.278243+0000","last_unstale":"2024-01-24T15:14:16.693254+0000","last_undegraded":"2024-01-24T15:14:16.693254+0000","last_fullsized":"2024-01-24T15:14:16.693254+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.706687+0000","last_change":"2024-01-24T15:14:16.697080+0000","last_active":"2024-01-24T15:14:16.706687+0000","last_peered":"2024-01-24T15:14:16.706687+0000","last_clean":"2024-01-24T15:14:16.706687+0000","last_became_active":"2024-01-24T15:14:16.696775+0000","last_became_peered":"2024-01-24T15:14:16.696775+0000","last_unstale":"2024-01-24T15:14:16.706687+0000","last_undegraded":"2024-01-24T15:14:16.706687+0000","last_fullsized":"2024-01-24T15:14:16.706687+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701571+0000","last_change":"2024-01-24T15:14:16.283263+0000","last_active":"2024-01-24T15:14:16.701571+0000","last_peered":"2024-01-24T15:14:16.701571+0000","last_clean":"2024-01-24T15:14:16.701571+0000","last_became_active":"2024-01-24T15:14:16.283026+0000","last_became_peered":"2024-01-24T15:14:16.283026+0000","last_unstale":"2024-01-24T15:14:16.701571+0000","last_undegraded":"2024-01-24T15:14:16.701571+0000","last_fullsized":"2024-01-24T15:14:16.701571+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.712687+0000","last_change":"2024-01-24T15:14:16.709855+0000","last_active":"2024-01-24T15:14:16.712687+0000","last_peered":"2024-01-24T15:14:16.712687+0000","last_clean":"2024-01-24T15:14:16.712687+0000","last_became_active":"2024-01-24T15:14:16.709553+0000","last_became_peered":"2024-01-24T15:14:16.709553+0000","last_unstale":"2024-01-24T15:14:16.712687+0000","last_undegraded":"2024-01-24T15:14:16.712687+0000","last_fullsized":"2024-01-24T15:14:16.712687+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.698815+0000","last_change":"2024-01-24T15:14:16.273046+0000","last_active":"2024-01-24T15:14:16.698815+0000","last_peered":"2024-01-24T15:14:16.698815+0000","last_clean":"2024-01-24T15:14:16.698815+0000","last_became_active":"2024-01-24T15:14:16.272746+0000","last_became_peered":"2024-01-24T15:14:16.272746+0000","last_unstale":"2024-01-24T15:14:16.698815+0000","last_undegraded":"2024-01-24T15:14:16.698815+0000","last_fullsized":"2024-01-24T15:14:16.698815+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.707542+0000","last_change":"2024-01-24T15:14:16.701514+0000","last_active":"2024-01-24T15:14:16.707542+0000","last_peered":"2024-01-24T15:14:16.707542+0000","last_clean":"2024-01-24T15:14:16.707542+0000","last_became_active":"2024-01-24T15:14:16.701234+0000","last_became_peered":"2024-01-24T15:14:16.701234+0000","last_unstale":"2024-01-24T15:14:16.707542+0000","last_undegraded":"2024-01-24T15:14:16.707542+0000","last_fullsized":"2024-01-24T15:14:16.707542+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.697225+0000","last_change":"2024-01-24T15:14:16.280562+0000","last_active":"2024-01-24T15:14:16.697225+0000","last_peered":"2024-01-24T15:14:16.697225+0000","last_clean":"2024-01-24T15:14:16.697225+0000","last_became_active":"2024-01-24T15:14:16.280288+0000","last_became_peered":"2024-01-24T15:14:16.280288+0000","last_unstale":"2024-01-24T15:14:16.697225+0000","last_undegraded":"2024-01-24T15:14:16.697225+0000","last_fullsized":"2024-01-24T15:14:16.697225+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702053+0000","last_change":"2024-01-24T15:14:16.693517+0000","last_active":"2024-01-24T15:14:16.702053+0000","last_peered":"2024-01-24T15:14:16.702053+0000","last_clean":"2024-01-24T15:14:16.702053+0000","last_became_active":"2024-01-24T15:14:16.693259+0000","last_became_peered":"2024-01-24T15:14:16.693259+0000","last_unstale":"2024-01-24T15:14:16.702053+0000","last_undegraded":"2024-01-24T15:14:16.702053+0000","last_fullsized":"2024-01-24T15:14:16.702053+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.703767+0000","last_change":"2024-01-24T15:14:16.286040+0000","last_active":"2024-01-24T15:14:16.703767+0000","last_peered":"2024-01-24T15:14:16.703767+0000","last_clean":"2024-01-24T15:14:16.703767+0000","last_became_active":"2024-01-24T15:14:16.285743+0000","last_became_peered":"2024-01-24T15:14:16.285743+0000","last_unstale":"2024-01-24T15:14:16.703767+0000","last_undegraded":"2024-01-24T15:14:16.703767+0000","last_fullsized":"2024-01-24T15:14:16.703767+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.707471+0000","last_change":"2024-01-24T15:14:16.700525+0000","last_active":"2024-01-24T15:14:16.707471+0000","last_peered":"2024-01-24T15:14:16.707471+0000","last_clean":"2024-01-24T15:14:16.707471+0000","last_became_active":"2024-01-24T15:14:16.700230+0000","last_became_peered":"2024-01-24T15:14:16.700230+0000","last_unstale":"2024-01-24T15:14:16.707471+0000","last_undegraded":"2024-01-24T15:14:16.707471+0000","last_fullsized":"2024-01-24T15:14:16.707471+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.691362+0000","last_change":"2024-01-24T15:14:16.274304+0000","last_active":"2024-01-24T15:14:16.691362+0000","last_peered":"2024-01-24T15:14:16.691362+0000","last_clean":"2024-01-24T15:14:16.691362+0000","last_became_active":"2024-01-24T15:14:16.273842+0000","last_became_peered":"2024-01-24T15:14:16.273842+0000","last_unstale":"2024-01-24T15:14:16.691362+0000","last_undegraded":"2024-01-24T15:14:16.691362+0000","last_fullsized":"2024-01-24T15:14:16.691362+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.711336+0000","last_change":"2024-01-24T15:14:16.704771+0000","last_active":"2024-01-24T15:14:16.711336+0000","last_peered":"2024-01-24T15:14:16.711336+0000","last_clean":"2024-01-24T15:14:16.711336+0000","last_became_active":"2024-01-24T15:14:16.704397+0000","last_became_peered":"2024-01-24T15:14:16.704397+0000","last_unstale":"2024-01-24T15:14:16.711336+0000","last_undegraded":"2024-01-24T15:14:16.711336+0000","last_fullsized":"2024-01-24T15:14:16.711336+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.691500+0000","last_change":"2024-01-24T15:14:16.284208+0000","last_active":"2024-01-24T15:14:16.691500+0000","last_peered":"2024-01-24T15:14:16.691500+0000","last_clean":"2024-01-24T15:14:16.691500+0000","last_became_active":"2024-01-24T15:14:16.283892+0000","last_became_peered":"2024-01-24T15:14:16.283892+0000","last_unstale":"2024-01-24T15:14:16.691500+0000","last_undegraded":"2024-01-24T15:14:16.691500+0000","last_fullsized":"2024-01-24T15:14:16.691500+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.697157+0000","last_change":"2024-01-24T15:14:16.695153+0000","last_active":"2024-01-24T15:14:16.697157+0000","last_peered":"2024-01-24T15:14:16.697157+0000","last_clean":"2024-01-24T15:14:16.697157+0000","last_became_active":"2024-01-24T15:14:16.694684+0000","last_became_peered":"2024-01-24T15:14:16.694684+0000","last_unstale":"2024-01-24T15:14:16.697157+0000","last_undegraded":"2024-01-24T15:14:16.697157+0000","last_fullsized":"2024-01-24T15:14:16.697157+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.704822+0000","last_change":"2024-01-24T15:14:16.281057+0000","last_active":"2024-01-24T15:14:16.704822+0000","last_peered":"2024-01-24T15:14:16.704822+0000","last_clean":"2024-01-24T15:14:16.704822+0000","last_became_active":"2024-01-24T15:14:16.280746+0000","last_became_peered":"2024-01-24T15:14:16.280746+0000","last_unstale":"2024-01-24T15:14:16.704822+0000","last_undegraded":"2024-01-24T15:14:16.704822+0000","last_fullsized":"2024-01-24T15:14:16.704822+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.711675+0000","last_change":"2024-01-24T15:14:16.704921+0000","last_active":"2024-01-24T15:14:16.711675+0000","last_peered":"2024-01-24T15:14:16.711675+0000","last_clean":"2024-01-24T15:14:16.711675+0000","last_became_active":"2024-01-24T15:14:16.704664+0000","last_became_peered":"2024-01-24T15:14:16.704664+0000","last_unstale":"2024-01-24T15:14:16.711675+0000","last_undegraded":"2024-01-24T15:14:16.711675+0000","last_fullsized":"2024-01-24T15:14:16.711675+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.690578+0000","last_change":"2024-01-24T15:14:16.274564+0000","last_active":"2024-01-24T15:14:16.690578+0000","last_peered":"2024-01-24T15:14:16.690578+0000","last_clean":"2024-01-24T15:14:16.690578+0000","last_became_active":"2024-01-24T15:14:16.274284+0000","last_became_peered":"2024-01-24T15:14:16.274284+0000","last_unstale":"2024-01-24T15:14:16.690578+0000","last_undegraded":"2024-01-24T15:14:16.690578+0000","last_fullsized":"2024-01-24T15:14:16.690578+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.706964+0000","last_change":"2024-01-24T15:14:16.704614+0000","last_active":"2024-01-24T15:14:16.706964+0000","last_peered":"2024-01-24T15:14:16.706964+0000","last_clean":"2024-01-24T15:14:16.706964+0000","last_became_active":"2024-01-24T15:14:16.704329+0000","last_became_peered":"2024-01-24T15:14:16.704329+0000","last_unstale":"2024-01-24T15:14:16.706964+0000","last_undegraded":"2024-01-24T15:14:16.706964+0000","last_fullsized":"2024-01-24T15:14:16.706964+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.693601+0000","last_change":"2024-01-24T15:14:16.272251+0000","last_active":"2024-01-24T15:14:16.693601+0000","last_peered":"2024-01-24T15:14:16.693601+0000","last_clean":"2024-01-24T15:14:16.693601+0000","last_became_active":"2024-01-24T15:14:16.271944+0000","last_became_peered":"2024-01-24T15:14:16.271944+0000","last_unstale":"2024-01-24T15:14:16.693601+0000","last_undegraded":"2024-01-24T15:14:16.693601+0000","last_fullsized":"2024-01-24T15:14:16.693601+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.706794+0000","last_change":"2024-01-24T15:14:16.705535+0000","last_active":"2024-01-24T15:14:16.706794+0000","last_peered":"2024-01-24T15:14:16.706794+0000","last_clean":"2024-01-24T15:14:16.706794+0000","last_became_active":"2024-01-24T15:14:16.705216+0000","last_became_peered":"2024-01-24T15:14:16.705216+0000","last_unstale":"2024-01-24T15:14:16.706794+0000","last_undegraded":"2024-01-24T15:14:16.706794+0000","last_fullsized":"2024-01-24T15:14:16.706794+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.691132+0000","last_change":"2024-01-24T15:14:16.274161+0000","last_active":"2024-01-24T15:14:16.691132+0000","last_peered":"2024-01-24T15:14:16.691132+0000","last_clean":"2024-01-24T15:14:16.691132+0000","last_became_active":"2024-01-24T15:14:16.273868+0000","last_became_peered":"2024-01-24T15:14:16.273868+0000","last_unstale":"2024-01-24T15:14:16.691132+0000","last_undegraded":"2024-01-24T15:14:16.691132+0000","last_fullsized":"2024-01-24T15:14:16.691132+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.707194+0000","last_change":"2024-01-24T15:14:16.702939+0000","last_active":"2024-01-24T15:14:16.707194+0000","last_peered":"2024-01-24T15:14:16.707194+0000","last_clean":"2024-01-24T15:14:16.707194+0000","last_became_active":"2024-01-24T15:14:16.702378+0000","last_became_peered":"2024-01-24T15:14:16.702378+0000","last_unstale":"2024-01-24T15:14:16.707194+0000","last_undegraded":"2024-01-24T15:14:16.707194+0000","last_fullsized":"2024-01-24T15:14:16.707194+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701981+0000","last_change":"2024-01-24T15:14:16.284287+0000","last_active":"2024-01-24T15:14:16.701981+0000","last_peered":"2024-01-24T15:14:16.701981+0000","last_clean":"2024-01-24T15:14:16.701981+0000","last_became_active":"2024-01-24T15:14:16.283984+0000","last_became_peered":"2024-01-24T15:14:16.283984+0000","last_unstale":"2024-01-24T15:14:16.701981+0000","last_undegraded":"2024-01-24T15:14:16.701981+0000","last_fullsized":"2024-01-24T15:14:16.701981+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.711316+0000","last_change":"2024-01-24T15:14:16.706468+0000","last_active":"2024-01-24T15:14:16.711316+0000","last_peered":"2024-01-24T15:14:16.711316+0000","last_clean":"2024-01-24T15:14:16.711316+0000","last_became_active":"2024-01-24T15:14:16.706169+0000","last_became_peered":"2024-01-24T15:14:16.706169+0000","last_unstale":"2024-01-24T15:14:16.711316+0000","last_undegraded":"2024-01-24T15:14:16.711316+0000","last_fullsized":"2024-01-24T15:14:16.711316+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.689074+0000","last_change":"2024-01-24T15:14:16.276447+0000","last_active":"2024-01-24T15:14:16.689074+0000","last_peered":"2024-01-24T15:14:16.689074+0000","last_clean":"2024-01-24T15:14:16.689074+0000","last_became_active":"2024-01-24T15:14:16.276159+0000","last_became_peered":"2024-01-24T15:14:16.276159+0000","last_unstale":"2024-01-24T15:14:16.689074+0000","last_undegraded":"2024-01-24T15:14:16.689074+0000","last_fullsized":"2024-01-24T15:14:16.689074+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.699221+0000","last_change":"2024-01-24T15:14:16.697555+0000","last_active":"2024-01-24T15:14:16.699221+0000","last_peered":"2024-01-24T15:14:16.699221+0000","last_clean":"2024-01-24T15:14:16.699221+0000","last_became_active":"2024-01-24T15:14:16.697248+0000","last_became_peered":"2024-01-24T15:14:16.697248+0000","last_unstale":"2024-01-24T15:14:16.699221+0000","last_undegraded":"2024-01-24T15:14:16.699221+0000","last_fullsized":"2024-01-24T15:14:16.699221+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701608+0000","last_change":"2024-01-24T15:14:16.287349+0000","last_active":"2024-01-24T15:14:16.701608+0000","last_peered":"2024-01-24T15:14:16.701608+0000","last_clean":"2024-01-24T15:14:16.701608+0000","last_became_active":"2024-01-24T15:14:16.287125+0000","last_became_peered":"2024-01-24T15:14:16.287125+0000","last_unstale":"2024-01-24T15:14:16.701608+0000","last_undegraded":"2024-01-24T15:14:16.701608+0000","last_fullsized":"2024-01-24T15:14:16.701608+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702035+0000","last_change":"2024-01-24T15:14:16.694777+0000","last_active":"2024-01-24T15:14:16.702035+0000","last_peered":"2024-01-24T15:14:16.702035+0000","last_clean":"2024-01-24T15:14:16.702035+0000","last_became_active":"2024-01-24T15:14:16.694478+0000","last_became_peered":"2024-01-24T15:14:16.694478+0000","last_unstale":"2024-01-24T15:14:16.702035+0000","last_undegraded":"2024-01-24T15:14:16.702035+0000","last_fullsized":"2024-01-24T15:14:16.702035+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.685807+0000","last_change":"2024-01-24T15:14:16.281779+0000","last_active":"2024-01-24T15:14:16.685807+0000","last_peered":"2024-01-24T15:14:16.685807+0000","last_clean":"2024-01-24T15:14:16.685807+0000","last_became_active":"2024-01-24T15:14:16.281503+0000","last_became_peered":"2024-01-24T15:14:16.281503+0000","last_unstale":"2024-01-24T15:14:16.685807+0000","last_undegraded":"2024-01-24T15:14:16.685807+0000","last_fullsized":"2024-01-24T15:14:16.685807+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701963+0000","last_change":"2024-01-24T15:14:16.691909+0000","last_active":"2024-01-24T15:14:16.701963+0000","last_peered":"2024-01-24T15:14:16.701963+0000","last_clean":"2024-01-24T15:14:16.701963+0000","last_became_active":"2024-01-24T15:14:16.691625+0000","last_became_peered":"2024-01-24T15:14:16.691625+0000","last_unstale":"2024-01-24T15:14:16.701963+0000","last_undegraded":"2024-01-24T15:14:16.701963+0000","last_fullsized":"2024-01-24T15:14:16.701963+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.684389+0000","last_change":"2024-01-24T15:14:16.282171+0000","last_active":"2024-01-24T15:14:16.684389+0000","last_peered":"2024-01-24T15:14:16.684389+0000","last_clean":"2024-01-24T15:14:16.684389+0000","last_became_active":"2024-01-24T15:14:16.281888+0000","last_became_peered":"2024-01-24T15:14:16.281888+0000","last_unstale":"2024-01-24T15:14:16.684389+0000","last_undegraded":"2024-01-24T15:14:16.684389+0000","last_fullsized":"2024-01-24T15:14:16.684389+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.705155+0000","last_change":"2024-01-24T15:14:16.700552+0000","last_active":"2024-01-24T15:14:16.705155+0000","last_peered":"2024-01-24T15:14:16.705155+0000","last_clean":"2024-01-24T15:14:16.705155+0000","last_became_active":"2024-01-24T15:14:16.700172+0000","last_became_peered":"2024-01-24T15:14:16.700172+0000","last_unstale":"2024-01-24T15:14:16.705155+0000","last_undegraded":"2024-01-24T15:14:16.705155+0000","last_fullsized":"2024-01-24T15:14:16.705155+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.693365+0000","last_change":"2024-01-24T15:14:16.273154+0000","last_active":"2024-01-24T15:14:16.693365+0000","last_peered":"2024-01-24T15:14:16.693365+0000","last_clean":"2024-01-24T15:14:16.693365+0000","last_became_active":"2024-01-24T15:14:16.272839+0000","last_became_peered":"2024-01-24T15:14:16.272839+0000","last_unstale":"2024-01-24T15:14:16.693365+0000","last_undegraded":"2024-01-24T15:14:16.693365+0000","last_fullsized":"2024-01-24T15:14:16.693365+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.711704+0000","last_change":"2024-01-24T15:14:16.709784+0000","last_active":"2024-01-24T15:14:16.711704+0000","last_peered":"2024-01-24T15:14:16.711704+0000","last_clean":"2024-01-24T15:14:16.711704+0000","last_became_active":"2024-01-24T15:14:16.709535+0000","last_became_peered":"2024-01-24T15:14:16.709535+0000","last_unstale":"2024-01-24T15:14:16.711704+0000","last_undegraded":"2024-01-24T15:14:16.711704+0000","last_fullsized":"2024-01-24T15:14:16.711704+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701688+0000","last_change":"2024-01-24T15:14:16.289711+0000","last_active":"2024-01-24T15:14:16.701688+0000","last_peered":"2024-01-24T15:14:16.701688+0000","last_clean":"2024-01-24T15:14:16.701688+0000","last_became_active":"2024-01-24T15:14:16.289432+0000","last_became_peered":"2024-01-24T15:14:16.289432+0000","last_unstale":"2024-01-24T15:14:16.701688+0000","last_undegraded":"2024-01-24T15:14:16.701688+0000","last_fullsized":"2024-01-24T15:14:16.701688+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.698813+0000","last_change":"2024-01-24T15:14:16.696123+0000","last_active":"2024-01-24T15:14:16.698813+0000","last_peered":"2024-01-24T15:14:16.698813+0000","last_clean":"2024-01-24T15:14:16.698813+0000","last_became_active":"2024-01-24T15:14:16.695859+0000","last_became_peered":"2024-01-24T15:14:16.695859+0000","last_unstale":"2024-01-24T15:14:16.698813+0000","last_undegraded":"2024-01-24T15:14:16.698813+0000","last_fullsized":"2024-01-24T15:14:16.698813+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.683343+0000","last_change":"2024-01-24T15:14:16.281660+0000","last_active":"2024-01-24T15:14:16.683343+0000","last_peered":"2024-01-24T15:14:16.683343+0000","last_clean":"2024-01-24T15:14:16.683343+0000","last_became_active":"2024-01-24T15:14:16.281382+0000","last_became_peered":"2024-01-24T15:14:16.281382+0000","last_unstale":"2024-01-24T15:14:16.683343+0000","last_undegraded":"2024-01-24T15:14:16.683343+0000","last_fullsized":"2024-01-24T15:14:16.683343+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.706066+0000","last_change":"2024-01-24T15:14:16.700917+0000","last_active":"2024-01-24T15:14:16.706066+0000","last_peered":"2024-01-24T15:14:16.706066+0000","last_clean":"2024-01-24T15:14:16.706066+0000","last_became_active":"2024-01-24T15:14:16.700614+0000","last_became_peered":"2024-01-24T15:14:16.700614+0000","last_unstale":"2024-01-24T15:14:16.706066+0000","last_undegraded":"2024-01-24T15:14:16.706066+0000","last_fullsized":"2024-01-24T15:14:16.706066+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"192'4","reported_seq":14,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702479+0000","last_change":"2024-01-24T15:14:16.267755+0000","last_active":"2024-01-24T15:14:16.702479+0000","last_peered":"2024-01-24T15:14:16.702479+0000","last_clean":"2024-01-24T15:14:16.702479+0000","last_became_active":"2024-01-24T15:14:16.267489+0000","last_became_peered":"2024-01-24T15:14:16.267489+0000","last_unstale":"2024-01-24T15:14:16.702479+0000","last_undegraded":"2024-01-24T15:14:16.702479+0000","last_fullsized":"2024-01-24T15:14:16.702479+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":155,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.696449+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:14:16.696449+0000","last_peered":"2024-01-24T15:14:16.696449+0000","last_clean":"2024-01-24T15:14:16.696449+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:14:16.696449+0000","last_undegraded":"2024-01-24T15:14:16.696449+0000","last_fullsized":"2024-01-24T15:14:16.696449+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.716522+0000","last_change":"2024-01-24T15:14:16.715837+0000","last_active":"2024-01-24T15:14:16.716522+0000","last_peered":"2024-01-24T15:14:16.716522+0000","last_clean":"2024-01-24T15:14:16.716522+0000","last_became_active":"2024-01-24T15:14:16.712087+0000","last_became_peered":"2024-01-24T15:14:16.712087+0000","last_unstale":"2024-01-24T15:14:16.716522+0000","last_undegraded":"2024-01-24T15:14:16.716522+0000","last_fullsized":"2024-01-24T15:14:16.716522+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.677909+0000","last_change":"2024-01-24T15:14:16.268328+0000","last_active":"2024-01-24T15:14:16.677909+0000","last_peered":"2024-01-24T15:14:16.677909+0000","last_clean":"2024-01-24T15:14:16.677909+0000","last_became_active":"2024-01-24T15:14:16.267970+0000","last_became_peered":"2024-01-24T15:14:16.267970+0000","last_unstale":"2024-01-24T15:14:16.677909+0000","last_undegraded":"2024-01-24T15:14:16.677909+0000","last_fullsized":"2024-01-24T15:14:16.677909+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":177,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.685268+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:14:16.685268+0000","last_peered":"2024-01-24T15:14:16.685268+0000","last_clean":"2024-01-24T15:14:16.685268+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:14:16.685268+0000","last_undegraded":"2024-01-24T15:14:16.685268+0000","last_fullsized":"2024-01-24T15:14:16.685268+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.706701+0000","last_change":"2024-01-24T15:14:16.702686+0000","last_active":"2024-01-24T15:14:16.706701+0000","last_peered":"2024-01-24T15:14:16.706701+0000","last_clean":"2024-01-24T15:14:16.706701+0000","last_became_active":"2024-01-24T15:14:16.702391+0000","last_became_peered":"2024-01-24T15:14:16.702391+0000","last_unstale":"2024-01-24T15:14:16.706701+0000","last_undegraded":"2024-01-24T15:14:16.706701+0000","last_fullsized":"2024-01-24T15:14:16.706701+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"192'3","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.701311+0000","last_change":"2024-01-24T15:14:16.277740+0000","last_active":"2024-01-24T15:14:16.701311+0000","last_peered":"2024-01-24T15:14:16.701311+0000","last_clean":"2024-01-24T15:14:16.701311+0000","last_became_active":"2024-01-24T15:14:16.277193+0000","last_became_peered":"2024-01-24T15:14:16.277193+0000","last_unstale":"2024-01-24T15:14:16.701311+0000","last_undegraded":"2024-01-24T15:14:16.701311+0000","last_fullsized":"2024-01-24T15:14:16.701311+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.678379+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:14:16.678379+0000","last_peered":"2024-01-24T15:14:16.678379+0000","last_clean":"2024-01-24T15:14:16.678379+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:14:16.678379+0000","last_undegraded":"2024-01-24T15:14:16.678379+0000","last_fullsized":"2024-01-24T15:14:16.678379+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.716186+0000","last_change":"2024-01-24T15:14:16.713849+0000","last_active":"2024-01-24T15:14:16.716186+0000","last_peered":"2024-01-24T15:14:16.716186+0000","last_clean":"2024-01-24T15:14:16.716186+0000","last_became_active":"2024-01-24T15:14:16.712035+0000","last_became_peered":"2024-01-24T15:14:16.712035+0000","last_unstale":"2024-01-24T15:14:16.716186+0000","last_undegraded":"2024-01-24T15:14:16.716186+0000","last_fullsized":"2024-01-24T15:14:16.716186+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.684209+0000","last_change":"2024-01-24T15:14:16.274086+0000","last_active":"2024-01-24T15:14:16.684209+0000","last_peered":"2024-01-24T15:14:16.684209+0000","last_clean":"2024-01-24T15:14:16.684209+0000","last_became_active":"2024-01-24T15:14:16.273819+0000","last_became_peered":"2024-01-24T15:14:16.273819+0000","last_unstale":"2024-01-24T15:14:16.684209+0000","last_undegraded":"2024-01-24T15:14:16.684209+0000","last_fullsized":"2024-01-24T15:14:16.684209+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":177,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.686046+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:14:16.686046+0000","last_peered":"2024-01-24T15:14:16.686046+0000","last_clean":"2024-01-24T15:14:16.686046+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:14:16.686046+0000","last_undegraded":"2024-01-24T15:14:16.686046+0000","last_fullsized":"2024-01-24T15:14:16.686046+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":159,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.692348+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:14:16.692348+0000","last_peered":"2024-01-24T15:14:16.692348+0000","last_clean":"2024-01-24T15:14:16.692348+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:14:16.692348+0000","last_undegraded":"2024-01-24T15:14:16.692348+0000","last_fullsized":"2024-01-24T15:14:16.692348+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.703202+0000","last_change":"2024-01-24T15:14:16.693968+0000","last_active":"2024-01-24T15:14:16.703202+0000","last_peered":"2024-01-24T15:14:16.703202+0000","last_clean":"2024-01-24T15:14:16.703202+0000","last_became_active":"2024-01-24T15:14:16.693735+0000","last_became_peered":"2024-01-24T15:14:16.693735+0000","last_unstale":"2024-01-24T15:14:16.703202+0000","last_undegraded":"2024-01-24T15:14:16.703202+0000","last_fullsized":"2024-01-24T15:14:16.703202+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.702171+0000","last_change":"2024-01-24T15:14:16.273322+0000","last_active":"2024-01-24T15:14:16.702171+0000","last_peered":"2024-01-24T15:14:16.702171+0000","last_clean":"2024-01-24T15:14:16.702171+0000","last_became_active":"2024-01-24T15:14:16.273048+0000","last_became_peered":"2024-01-24T15:14:16.273048+0000","last_unstale":"2024-01-24T15:14:16.702171+0000","last_undegraded":"2024-01-24T15:14:16.702171+0000","last_fullsized":"2024-01-24T15:14:16.702171+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":157,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.691215+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:14:16.691215+0000","last_peered":"2024-01-24T15:14:16.691215+0000","last_clean":"2024-01-24T15:14:16.691215+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:14:16.691215+0000","last_undegraded":"2024-01-24T15:14:16.691215+0000","last_fullsized":"2024-01-24T15:14:16.691215+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.704001+0000","last_change":"2024-01-24T15:14:16.695074+0000","last_active":"2024-01-24T15:14:16.704001+0000","last_peered":"2024-01-24T15:14:16.704001+0000","last_clean":"2024-01-24T15:14:16.704001+0000","last_became_active":"2024-01-24T15:14:16.694779+0000","last_became_peered":"2024-01-24T15:14:16.694779+0000","last_unstale":"2024-01-24T15:14:16.704001+0000","last_undegraded":"2024-01-24T15:14:16.704001+0000","last_fullsized":"2024-01-24T15:14:16.704001+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.693497+0000","last_change":"2024-01-24T15:14:16.283416+0000","last_active":"2024-01-24T15:14:16.693497+0000","last_peered":"2024-01-24T15:14:16.693497+0000","last_clean":"2024-01-24T15:14:16.693497+0000","last_became_active":"2024-01-24T15:14:16.282710+0000","last_became_peered":"2024-01-24T15:14:16.282710+0000","last_unstale":"2024-01-24T15:14:16.693497+0000","last_undegraded":"2024-01-24T15:14:16.693497+0000","last_fullsized":"2024-01-24T15:14:16.693497+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":157,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.691480+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:14:16.691480+0000","last_peered":"2024-01-24T15:14:16.691480+0000","last_clean":"2024-01-24T15:14:16.691480+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:14:16.691480+0000","last_undegraded":"2024-01-24T15:14:16.691480+0000","last_fullsized":"2024-01-24T15:14:16.691480+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.709074+0000","last_change":"2024-01-24T15:14:16.699999+0000","last_active":"2024-01-24T15:14:16.709074+0000","last_peered":"2024-01-24T15:14:16.709074+0000","last_clean":"2024-01-24T15:14:16.709074+0000","last_became_active":"2024-01-24T15:14:16.699672+0000","last_became_peered":"2024-01-24T15:14:16.699672+0000","last_unstale":"2024-01-24T15:14:16.709074+0000","last_undegraded":"2024-01-24T15:14:16.709074+0000","last_fullsized":"2024-01-24T15:14:16.709074+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.683606+0000","last_change":"2024-01-24T15:14:16.284535+0000","last_active":"2024-01-24T15:14:16.683606+0000","last_peered":"2024-01-24T15:14:16.683606+0000","last_clean":"2024-01-24T15:14:16.683606+0000","last_became_active":"2024-01-24T15:14:16.283799+0000","last_became_peered":"2024-01-24T15:14:16.283799+0000","last_unstale":"2024-01-24T15:14:16.683606+0000","last_undegraded":"2024-01-24T15:14:16.683606+0000","last_fullsized":"2024-01-24T15:14:16.683606+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":155,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.700561+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:14:16.700561+0000","last_peered":"2024-01-24T15:14:16.700561+0000","last_clean":"2024-01-24T15:14:16.700561+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:14:16.700561+0000","last_undegraded":"2024-01-24T15:14:16.700561+0000","last_fullsized":"2024-01-24T15:14:16.700561+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.709023+0000","last_change":"2024-01-24T15:14:16.706451+0000","last_active":"2024-01-24T15:14:16.709023+0000","last_peered":"2024-01-24T15:14:16.709023+0000","last_clean":"2024-01-24T15:14:16.709023+0000","last_became_active":"2024-01-24T15:14:16.706147+0000","last_became_peered":"2024-01-24T15:14:16.706147+0000","last_unstale":"2024-01-24T15:14:16.709023+0000","last_undegraded":"2024-01-24T15:14:16.709023+0000","last_fullsized":"2024-01-24T15:14:16.709023+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:16.250162+0000","last_clean_scrub_stamp":"2024-01-24T15:14:16.250162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.710713+0000","last_change":"2024-01-24T15:14:16.286220+0000","last_active":"2024-01-24T15:14:16.710713+0000","last_peered":"2024-01-24T15:14:16.710713+0000","last_clean":"2024-01-24T15:14:16.710713+0000","last_became_active":"2024-01-24T15:14:16.285890+0000","last_became_peered":"2024-01-24T15:14:16.285890+0000","last_unstale":"2024-01-24T15:14:16.710713+0000","last_undegraded":"2024-01-24T15:14:16.710713+0000","last_fullsized":"2024-01-24T15:14:16.710713+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:14:15.245157+0000","last_clean_scrub_stamp":"2024-01-24T15:14:15.245157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":154,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-01-24T15:14:16.705063+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:14:16.705063+0000","last_peered":"2024-01-24T15:14:16.705063+0000","last_clean":"2024-01-24T15:14:16.705063+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:14:16.705063+0000","last_undegraded":"2024-01-24T15:14:16.705063+0000","last_fullsized":"2024-01-24T15:14:16.705063+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575758,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303560,"kb_used_data":2368,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068280,"statfs":{"total":96636764160,"available":96325918720,"internally_reserved":0,"allocated":2424832,"data_stored":1385856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.59899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.28000000000000003},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.628,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.28899999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.1080000000000001},"last":0.63500000000000001}]},{"osd":1,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.33300000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.505,"5min":0.40000000000000002,"15min":0.34799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.43099999999999999}]},{"osd":2,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0329999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.38},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0880000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.41199999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.46800000000000003}]},{"osd":4,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.36299999999999999,"5min":0.34699999999999998,"15min":0.33200000000000002},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.31900000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.57399999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.39600000000000002,"5min":0.38,"15min":0.34799999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.505},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.36199999999999999},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.31900000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.46400000000000002,"5min":0.40899999999999997,"15min":0.40500000000000003},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.67700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575759,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307648,"kb_used_data":2360,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064192,"statfs":{"total":96636764160,"available":96321732608,"internally_reserved":0,"allocated":2416640,"data_stored":1385966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.86599999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.42699999999999999,"15min":0.37},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.69599999999999995}]},{"osd":1,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.84199999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77100000000000002,"5min":0.81200000000000006,"15min":1.258},"last":0.59799999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.40100000000000002},"max":{"1min":0.82299999999999995,"5min":0.85899999999999999,"15min":1.2430000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.75}]},{"osd":3,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.47199999999999998,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.39000000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66600000000000004}]},{"osd":4,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.317},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32400000000000001,"15min":0.30499999999999999},"max":{"1min":0.79300000000000004,"5min":0.80100000000000005,"15min":1.002},"last":0.76500000000000001}]},{"osd":5,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.32500000000000001},"max":{"1min":0.76000000000000001,"5min":0.78100000000000003,"15min":0.83899999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.32400000000000001},"max":{"1min":0.749,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.78900000000000003}]},{"osd":7,"last update":"Wed Jan 24 15:13:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.40899999999999997,"5min":0.372,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.90100000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.38},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":1.0429999999999999},"last":0.84699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575758,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307644,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064196,"statfs":{"total":96636764160,"available":96321736704,"internally_reserved":0,"allocated":2412544,"data_stored":1385086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.56399999999999995,"5min":0.754,"15min":0.754},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.443,"15min":0.435},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.26800000000000002},"max":{"1min":0.55800000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.627}]},{"osd":2,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.33700000000000002,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.57599999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44600000000000001,"15min":0.44700000000000001},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.28199999999999997},"max":{"1min":0.56200000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.56399999999999995}]},{"osd":3,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"min":{"1min":0.32400000000000001,"5min":0.314,"15min":0.309},"max":{"1min":0.58399999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.35799999999999998,"15min":0.313},"max":{"1min":0.59899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.63900000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.51800000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.86099999999999999,"5min":1.044,"15min":1.0649999999999999},"last":0.751},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.51400000000000001,"5min":0.47399999999999998,"15min":0.41699999999999998},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.49199999999999999,"15min":0.46300000000000002},"max":{"1min":0.83999999999999997,"5min":1.014,"15min":1.0780000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.439},"max":{"1min":0.86699999999999999,"5min":1.105,"15min":1.105},"last":0.78700000000000003}]},{"osd":6,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.874,"5min":1.0029999999999999,"15min":1.069},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.56599999999999995,"5min":0.499,"15min":0.48199999999999998},"max":{"1min":0.92500000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.879}]},{"osd":7,"last update":"Wed Jan 24 15:13:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.83799999999999997,"5min":1.127,"15min":1.127},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.77500000000000002,"15min":0.76700000000000002},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.42799999999999999},"max":{"1min":0.88900000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.84699999999999998}]}]},{"osd":0,"up_from":14,"seq":60129543056,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307624,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064216,"statfs":{"total":96636764160,"available":96321757184,"internally_reserved":0,"allocated":2392064,"data_stored":1376322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.28100000000000003,"5min":0.27500000000000002,"15min":0.247},"max":{"1min":0.60199999999999998,"5min":0.63400000000000001,"15min":0.66400000000000003},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.27700000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.82299999999999995}]},{"osd":2,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.29399999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.81999999999999995},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.84399999999999997},"last":0.75700000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.44600000000000001},"min":{"1min":0.317,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.65500000000000003,"15min":0.67000000000000004},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.52000000000000002},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.90000000000000002},"last":0.73799999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.433},"max":{"1min":0.99399999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.38700000000000001},"max":{"1min":0.995,"5min":0.995,"15min":1.0249999999999999},"last":0.86699999999999999}]},{"osd":5,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.70699999999999996},"min":{"1min":0.54900000000000004,"5min":0.498,"15min":0.42099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.874}]},{"osd":6,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.46899999999999997},"max":{"1min":1.022,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0820000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.88600000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:13:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0469999999999999,"5min":1.143,"15min":1.3260000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.375},"last":0.90300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575758,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311728,"kb_used_data":2344,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060112,"statfs":{"total":96636764160,"available":96317554688,"internally_reserved":0,"allocated":2400256,"data_stored":1377373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.27200000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.60099999999999998,"5min":0.65000000000000002,"15min":0.66800000000000004},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.45100000000000001,"15min":0.45200000000000001},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.70399999999999996},"last":0.43099999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.25600000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.73599999999999999},"last":0.46300000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.46300000000000002},"min":{"1min":0.32500000000000001,"5min":0.29699999999999999,"15min":0.29599999999999999},"max":{"1min":0.622,"5min":0.69699999999999995,"15min":0.70099999999999996},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.73299999999999998},"last":0.39500000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96199999999999997,"5min":1.196,"15min":1.196},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":1,"15min":1.089},"last":0.627}]},{"osd":5,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54400000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.111},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.46899999999999997,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.147,"15min":1.1830000000000001},"last":0.55700000000000005}]},{"osd":6,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.64700000000000002,"5min":0.47699999999999998,"15min":0.45200000000000001},"max":{"1min":0.99399999999999999,"5min":0.998,"15min":1.129},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.58899999999999997,"5min":0.47299999999999998,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.98999999999999999,"15min":1.1950000000000001},"last":0.61499999999999999}]},{"osd":7,"last update":"Wed Jan 24 15:13:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.753,"15min":0.746},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.497},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.248},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.65000000000000002,"5min":0.51500000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.24,"15min":1.24},"last":0.71699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575758,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307648,"kb_used_data":2360,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064192,"statfs":{"total":96636764160,"available":96321732608,"internally_reserved":0,"allocated":2416640,"data_stored":1385978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39800000000000002,"15min":0.39200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.247},"max":{"1min":0.56899999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.70199999999999996},"last":0.33300000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41899999999999998,"15min":0.41399999999999998},"min":{"1min":0.311,"5min":0.28100000000000003,"15min":0.27400000000000002},"max":{"1min":0.59999999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.377},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.34799999999999998,"5min":0.316,"15min":0.30499999999999999},"max":{"1min":0.67900000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.441}]},{"osd":2,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.497,"15min":0.502},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51300000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42999999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.41099999999999998},"max":{"1min":0.83299999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95999999999999996,"15min":1.5580000000000001},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.1879999999999999,"15min":1.6240000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.647},"last":0.67600000000000005}]},{"osd":6,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.45700000000000002,"15min":0.42699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.522},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.073,"5min":1.079,"15min":1.079},"last":0.75700000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:13:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.44800000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1499999999999999,"5min":1.246,"15min":1.246},"last":0.66000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307656,"kb_used_data":2368,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064184,"statfs":{"total":96636764160,"available":96321724416,"internally_reserved":0,"allocated":2424832,"data_stored":1386320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.62},"min":{"1min":0.34899999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.84199999999999997},"last":0.66700000000000004}]},{"osd":1,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.82499999999999996,"15min":0.82599999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63300000000000001}]},{"osd":2,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.35799999999999998},"max":{"1min":0.82099999999999995,"5min":0.84699999999999998,"15min":0.88300000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.372},"max":{"1min":0.81499999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.62}]},{"osd":3,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.38400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.89500000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.50900000000000001,"5min":0.46800000000000003,"15min":0.42399999999999999},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.92300000000000004},"last":0.64500000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.36099999999999999,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.81799999999999995},"last":0.374},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51500000000000001,"15min":0.51600000000000001},"min":{"1min":0.40899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65800000000000003,"5min":0.72499999999999998,"15min":0.85499999999999998},"last":0.41799999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.73699999999999999,"5min":0.78500000000000003,"15min":0.85499999999999998},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.86799999999999999},"last":0.438}]},{"osd":7,"last update":"Wed Jan 24 15:13:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55300000000000005,"15min":0.55200000000000005},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.33600000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.82799999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.34699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.96599999999999997},"last":0.45200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575759,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307648,"kb_used_data":2360,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064192,"statfs":{"total":96636764160,"available":96321732608,"internally_reserved":0,"allocated":2416640,"data_stored":1385628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.38600000000000001,"15min":0.35099999999999998},"max":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":1.4470000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.81599999999999995},"last":0.82799999999999996}]},{"osd":1,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.82499999999999996},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":1.4930000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.625,"15min":0.627},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.79300000000000004,"5min":0.80900000000000005,"15min":0.86499999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":1.53},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.80800000000000005,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41899999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.92600000000000005}]},{"osd":4,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.65400000000000003,"5min":0.74399999999999999,"15min":0.84699999999999998},"last":0.753},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":0.77200000000000002,"15min":0.80100000000000005},"last":0.54500000000000004}]},{"osd":6,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.5,"15min":0.504},"min":{"1min":0.374,"5min":0.35499999999999998,"15min":0.33400000000000002},"max":{"1min":0.65100000000000002,"5min":0.79400000000000004,"15min":0.81499999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.78500000000000003,"15min":0.81299999999999994},"last":0.58699999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:13:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.54900000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:14:21.261 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-24T15:14:21.261 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-24T15:14:21.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.434+0000 7fac1497a700 1 Processor -- start 2024-01-24T15:14:21.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.434+0000 7fac1497a700 1 -- start start 2024-01-24T15:14:21.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.435+0000 7fac1497a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fac10150fe0 0x7fac10154920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.435+0000 7fac1497a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac10154e10 0x7fac100c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.435+0000 7fac1497a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac100c3b60 con 0x7fac10150c00 2024-01-24T15:14:21.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.435+0000 7fac1497a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac10066960 con 0x7fac10150fe0 2024-01-24T15:14:21.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.435+0000 7fac1497a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac101558a0 con 0x7fac10154e10 2024-01-24T15:14:21.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.435+0000 7fac0e59c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fac10150c00 0x7fac100c3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35052/0 (socket says 172.21.15.29:35052) 2024-01-24T15:14:21.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.435+0000 7fac0e59c700 1 -- 172.21.15.29:0/1314966179 learned_addr learned my addr 172.21.15.29:0/1314966179 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:21.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.435+0000 7fac0ed9d700 1 --2- 172.21.15.29:0/1314966179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac10154e10 0x7fac100c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.435+0000 7fac0dd9b700 1 --2- 172.21.15.29:0/1314966179 >> v2:172.21.15.163:3300/0 conn(0x7fac10150fe0 0x7fac10154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.435+0000 7fac0d59a700 1 -- 172.21.15.29:0/1314966179 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894555357 0 0) 0x7fac100c3b60 con 0x7fac10150c00 2024-01-24T15:14:21.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.435+0000 7fac0d59a700 1 -- 172.21.15.29:0/1314966179 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabf80033f0 con 0x7fac10150c00 2024-01-24T15:14:21.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.436+0000 7fac0ed9d700 1 -- 172.21.15.29:0/1314966179 >> v2:172.21.15.163:3300/0 conn(0x7fac10150fe0 msgr2=0x7fac10154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:21.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.436+0000 7fac0ed9d700 1 --2- 172.21.15.29:0/1314966179 >> v2:172.21.15.163:3300/0 conn(0x7fac10150fe0 0x7fac10154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.436+0000 7fac0ed9d700 1 -- 172.21.15.29:0/1314966179 >> v1:172.21.15.29:6789/0 conn(0x7fac10150c00 legacy=0x7fac100c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.436+0000 7fac0ed9d700 1 -- 172.21.15.29:0/1314966179 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac100c2fc0 con 0x7fac10154e10 2024-01-24T15:14:21.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.436+0000 7fac0dd9b700 1 --2- 172.21.15.29:0/1314966179 >> v2:172.21.15.163:3300/0 conn(0x7fac10150fe0 0x7fac10154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:21.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.436+0000 7fac0ed9d700 1 --2- 172.21.15.29:0/1314966179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac10154e10 0x7fac100c44c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fac08009e60 tx=0x7fac08009780).ready entity=mon.2 client_cookie=84e25e053f8e104b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:21.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.436+0000 7fac0d59a700 1 -- 172.21.15.29:0/1314966179 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0800bab0 con 0x7fac10154e10 2024-01-24T15:14:21.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.436+0000 7fac0d59a700 1 -- 172.21.15.29:0/1314966179 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0800bc10 con 0x7fac10154e10 2024-01-24T15:14:21.439 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.436+0000 7fac0d59a700 1 -- 172.21.15.29:0/1314966179 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08011480 con 0x7fac10154e10 2024-01-24T15:14:21.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.437+0000 7fac1497a700 1 -- 172.21.15.29:0/1314966179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac10154e10 msgr2=0x7fac100c44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.437+0000 7fac1497a700 1 --2- 172.21.15.29:0/1314966179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac10154e10 0x7fac100c44c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fac08009e60 tx=0x7fac08009780).stop 2024-01-24T15:14:21.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.437+0000 7fac1497a700 1 -- 172.21.15.29:0/1314966179 shutdown_connections 2024-01-24T15:14:21.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.437+0000 7fac1497a700 1 --2- 172.21.15.29:0/1314966179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac10154e10 0x7fac100c44c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.437+0000 7fac1497a700 1 --2- 172.21.15.29:0/1314966179 >> v2:172.21.15.163:3300/0 conn(0x7fac10150fe0 0x7fac10154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.437+0000 7fac1497a700 1 -- 172.21.15.29:0/1314966179 >> 172.21.15.29:0/1314966179 conn(0x7fac100638c0 msgr2=0x7fac10063cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.437+0000 7fac1497a700 1 -- 172.21.15.29:0/1314966179 shutdown_connections 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.437+0000 7fac1497a700 1 -- 172.21.15.29:0/1314966179 wait complete. 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.438+0000 7fac1497a700 1 Processor -- start 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.438+0000 7fac1497a700 1 -- start start 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac1497a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac10150c00 0x7fac100bcb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac1497a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac10150fe0 0x7fac10151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac1497a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac10154e10 0x7fac100627b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac1497a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac10153bb0 con 0x7fac10150fe0 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac1497a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac10153280 con 0x7fac10150c00 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac1497a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac10153560 con 0x7fac10154e10 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac0dd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac10150fe0 0x7fac10151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac0dd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac10150fe0 0x7fac10151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39300/0 (socket says 172.21.15.29:39300) 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac0dd9b700 1 -- 172.21.15.29:0/2248862473 learned_addr learned my addr 172.21.15.29:0/2248862473 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac0ed9d700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac10154e10 0x7fac100627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac0e59c700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac10150c00 0x7fac100bcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac0dd9b700 1 -- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac10154e10 msgr2=0x7fac100627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac0dd9b700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac10154e10 0x7fac100627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac0dd9b700 1 -- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac10150c00 msgr2=0x7fac100bcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac0dd9b700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac10150c00 0x7fac100bcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.439+0000 7fac0dd9b700 1 -- 172.21.15.29:0/2248862473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac040092a0 con 0x7fac10150fe0 2024-01-24T15:14:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.440+0000 7fac0dd9b700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac10150fe0 0x7fac10151d90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7fac0400ea20 tx=0x7fac04009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.440+0000 7fabfe7fc700 1 -- 172.21.15.29:0/2248862473 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0400c850 con 0x7fac10150fe0 2024-01-24T15:14:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.440+0000 7fabfe7fc700 1 -- 172.21.15.29:0/2248862473 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0400c9b0 con 0x7fac10150fe0 2024-01-24T15:14:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.440+0000 7fabfe7fc700 1 -- 172.21.15.29:0/2248862473 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0400cd30 con 0x7fac10150fe0 2024-01-24T15:14:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.440+0000 7fac1497a700 1 -- 172.21.15.29:0/2248862473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac08009a20 con 0x7fac10150fe0 2024-01-24T15:14:21.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.440+0000 7fac1497a700 1 -- 172.21.15.29:0/2248862473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac10152280 con 0x7fac10150fe0 2024-01-24T15:14:21.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.441+0000 7fac1497a700 1 -- 172.21.15.29:0/2248862473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac100475e0 con 0x7fac10150fe0 2024-01-24T15:14:21.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.441+0000 7fabfe7fc700 1 -- 172.21.15.29:0/2248862473 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fac04012600 con 0x7fac10150fe0 2024-01-24T15:14:21.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.442+0000 7fabfe7fc700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabf005f270 0x7fabf0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.442+0000 7fabfe7fc700 1 -- 172.21.15.29:0/2248862473 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fac0407ae30 con 0x7fac10150fe0 2024-01-24T15:14:21.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.442+0000 7fac0e59c700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabf005f270 0x7fabf0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.442+0000 7fac0e59c700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabf005f270 0x7fabf0061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fac00005750 tx=0x7fac00009040).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:21.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.446+0000 7fabfe7fc700 1 -- 172.21.15.29:0/2248862473 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fac0404bb50 con 0x7fac10150fe0 2024-01-24T15:14:21.578 INFO:tasks.ceph.mds.c.smithi029.stderr:2024-01-24T15:14:21.576+0000 7f163508d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T15:14:21.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.701+0000 7fac1497a700 1 -- 172.21.15.29:0/2248862473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fac10153a00 con 0x7fac10150fe0 2024-01-24T15:14:21.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.703+0000 7fabfe7fc700 1 -- 172.21.15.29:0/2248862473 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v46) v1 ==== 190+0+0 (secure 0 0 0) 0x7fac0404f170 con 0x7fac10150fe0 2024-01-24T15:14:21.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.706+0000 7fac1497a700 1 -- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabf005f270 msgr2=0x7fabf0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.706+0000 7fac1497a700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabf005f270 0x7fabf0061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fac00005750 tx=0x7fac00009040).stop 2024-01-24T15:14:21.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.706+0000 7fac1497a700 1 -- 172.21.15.29:0/2248862473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac10150fe0 msgr2=0x7fac10151d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.706+0000 7fac1497a700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac10150fe0 0x7fac10151d90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7fac0400ea20 tx=0x7fac04009060).stop 2024-01-24T15:14:21.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.706+0000 7fac1497a700 1 -- 172.21.15.29:0/2248862473 shutdown_connections 2024-01-24T15:14:21.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.706+0000 7fac1497a700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac10154e10 0x7fac100627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.706+0000 7fac1497a700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac10150c00 0x7fac100bcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.706+0000 7fac1497a700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fabf005f270 0x7fabf0061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.706+0000 7fac1497a700 1 --2- 172.21.15.29:0/2248862473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac10150fe0 0x7fac10151d90 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.706+0000 7fac1497a700 1 -- 172.21.15.29:0/2248862473 >> 172.21.15.29:0/2248862473 conn(0x7fac100638c0 msgr2=0x7fac100aac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:21.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.706+0000 7fac1497a700 1 -- 172.21.15.29:0/2248862473 shutdown_connections 2024-01-24T15:14:21.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.706+0000 7fac1497a700 1 -- 172.21.15.29:0/2248862473 wait complete. 2024-01-24T15:14:21.709 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-01-24T15:14:21.721 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:14:21.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.849+0000 7f021e975700 1 Processor -- start 2024-01-24T15:14:21.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.850+0000 7f021e975700 1 -- start start 2024-01-24T15:14:21.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.850+0000 7f021e975700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0218155b00 0x7f02180664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.850+0000 7f021e975700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f02180669e0 0x7f0218155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.850+0000 7f021e975700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0218066e60 con 0x7f02181563d0 2024-01-24T15:14:21.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.850+0000 7f021e975700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02180c5500 con 0x7f02180669e0 2024-01-24T15:14:21.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.850+0000 7f021e975700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02180c3910 con 0x7f0218155b00 2024-01-24T15:14:21.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f02177fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f02180669e0 0x7f0218155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f02177fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f02180669e0 0x7f0218155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39314/0 (socket says 172.21.15.29:39314) 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f02177fe700 1 -- 172.21.15.29:0/324312598 learned_addr learned my addr 172.21.15.29:0/324312598 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f0217fff700 1 --2- 172.21.15.29:0/324312598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0218155b00 0x7f02180664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f02177fe700 1 -- 172.21.15.29:0/324312598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0218155b00 msgr2=0x7f02180664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f02177fe700 1 --2- 172.21.15.29:0/324312598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0218155b00 0x7f02180664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f02177fe700 1 -- 172.21.15.29:0/324312598 >> v1:172.21.15.163:6789/0 conn(0x7f02181563d0 legacy=0x7f0218067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f02177fe700 1 -- 172.21.15.29:0/324312598 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02180c5220 con 0x7f02180669e0 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f0216ffd700 1 -- 172.21.15.29:0/324312598 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578499088 0 0) 0x7f0218066e60 con 0x7f02181563d0 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f02177fe700 1 --2- 172.21.15.29:0/324312598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02180669e0 0x7f0218155ee0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f020c006fd0 tx=0x7f020c0090f0).ready entity=mon.0 client_cookie=fe4a56938bce99c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f0216ffd700 1 -- 172.21.15.29:0/324312598 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c00b580 con 0x7f02180669e0 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f0216ffd700 1 -- 172.21.15.29:0/324312598 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c00b6e0 con 0x7f02180669e0 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.851+0000 7f0216ffd700 1 -- 172.21.15.29:0/324312598 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c00ba60 con 0x7f02180669e0 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.852+0000 7f021e975700 1 -- 172.21.15.29:0/324312598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02180669e0 msgr2=0x7f0218155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.852+0000 7f021e975700 1 --2- 172.21.15.29:0/324312598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02180669e0 0x7f0218155ee0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f020c006fd0 tx=0x7f020c0090f0).stop 2024-01-24T15:14:21.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.852+0000 7f021e975700 1 -- 172.21.15.29:0/324312598 shutdown_connections 2024-01-24T15:14:21.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.852+0000 7f021e975700 1 --2- 172.21.15.29:0/324312598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0218155b00 0x7f02180664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.852+0000 7f021e975700 1 --2- 172.21.15.29:0/324312598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02180669e0 0x7f0218155ee0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.852+0000 7f021e975700 1 -- 172.21.15.29:0/324312598 >> 172.21.15.29:0/324312598 conn(0x7f02180c8bb0 msgr2=0x7f02180c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:21.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.852+0000 7f021e975700 1 -- 172.21.15.29:0/324312598 shutdown_connections 2024-01-24T15:14:21.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.852+0000 7f021e975700 1 -- 172.21.15.29:0/324312598 wait complete. 2024-01-24T15:14:21.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.853+0000 7f021e975700 1 Processor -- start 2024-01-24T15:14:21.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.853+0000 7f021e975700 1 -- start start 2024-01-24T15:14:21.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.854+0000 7f021e975700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02180669e0 0x7f0218139f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.854+0000 7f021e975700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0218155b00 0x7f0218151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.854+0000 7f021e975700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02181563d0 0x7f021813b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.854+0000 7f021e975700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021813a910 con 0x7f0218155b00 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.854+0000 7f021e975700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021813a430 con 0x7f02181563d0 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.854+0000 7f021e975700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02180c13c0 con 0x7f02180669e0 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.854+0000 7f02177fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0218155b00 0x7f0218151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.854+0000 7f02177fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0218155b00 0x7f0218151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39326/0 (socket says 172.21.15.29:39326) 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.854+0000 7f02177fe700 1 -- 172.21.15.29:0/1272357533 learned_addr learned my addr 172.21.15.29:0/1272357533 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.854+0000 7f0217fff700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02180669e0 0x7f0218139f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.854+0000 7f021cf12700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02181563d0 0x7f021813b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.855+0000 7f02177fe700 1 -- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02180669e0 msgr2=0x7f0218139f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.855+0000 7f02177fe700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02180669e0 0x7f0218139f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.855+0000 7f02177fe700 1 -- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02181563d0 msgr2=0x7f021813b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.855+0000 7f02177fe700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02181563d0 0x7f021813b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:21.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.855+0000 7f02177fe700 1 -- 172.21.15.29:0/1272357533 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c009370 con 0x7f0218155b00 2024-01-24T15:14:21.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.855+0000 7f02177fe700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0218155b00 0x7f0218151600 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f020c009130 tx=0x7f020c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:21.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.855+0000 7f01fffff700 1 -- 172.21.15.29:0/1272357533 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c000bc0 con 0x7f0218155b00 2024-01-24T15:14:21.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.855+0000 7f01fffff700 1 -- 172.21.15.29:0/1272357533 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c00b850 con 0x7f0218155b00 2024-01-24T15:14:21.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.855+0000 7f021e975700 1 -- 172.21.15.29:0/1272357533 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02180c16a0 con 0x7f0218155b00 2024-01-24T15:14:21.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.855+0000 7f01fffff700 1 -- 172.21.15.29:0/1272357533 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c000d20 con 0x7f0218155b00 2024-01-24T15:14:21.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.855+0000 7f021e975700 1 -- 172.21.15.29:0/1272357533 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02180c1c60 con 0x7f0218155b00 2024-01-24T15:14:21.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.856+0000 7f021e975700 1 -- 172.21.15.29:0/1272357533 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021813ad90 con 0x7f0218155b00 2024-01-24T15:14:21.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.857+0000 7f01fffff700 1 -- 172.21.15.29:0/1272357533 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f020c01c900 con 0x7f0218155b00 2024-01-24T15:14:21.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.857+0000 7f01fffff700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f020405f270 0x7f0204061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:21.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.857+0000 7f01fffff700 1 -- 172.21.15.29:0/1272357533 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f020c07c3b0 con 0x7f0218155b00 2024-01-24T15:14:21.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.857+0000 7f0217fff700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f020405f270 0x7f0204061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:21.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.858+0000 7f0217fff700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f020405f270 0x7f0204061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0208009c30 tx=0x7f0208009060).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:21.863 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:21.862+0000 7f01fffff700 1 -- 172.21.15.29:0/1272357533 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f020c04d0d0 con 0x7f0218155b00 2024-01-24T15:14:22.090 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.088+0000 7f021e975700 1 -- 172.21.15.29:0/1272357533 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02180475e0 con 0x7f0218155b00 2024-01-24T15:14:22.091 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.088+0000 7f01fffff700 1 -- 172.21.15.29:0/1272357533 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f020c02a020 con 0x7f0218155b00 2024-01-24T15:14:22.091 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:14:22.091 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1350237946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1350237946},{"type":"v1","addr":"172.21.15.163:6836","nonce":1350237946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8656,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/250387328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":250387328},{"type":"v1","addr":"172.21.15.163:6837","nonce":250387328}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":9069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3867710046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3867710046},{"type":"v1","addr":"172.21.15.29:6833","nonce":3867710046}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:14:16.673720+0000","modified":"2024-01-24T15:14:19.686795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9072":{"gid":9072,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1929837915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1929837915},{"type":"v1","addr":"172.21.15.29:6835","nonce":1929837915}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-24T15:14:22.092 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.090+0000 7f021e975700 1 -- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f020405f270 msgr2=0x7f0204061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.090+0000 7f021e975700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f020405f270 0x7f0204061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0208009c30 tx=0x7f0208009060).stop 2024-01-24T15:14:22.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.090+0000 7f021e975700 1 -- 172.21.15.29:0/1272357533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0218155b00 msgr2=0x7f0218151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.090+0000 7f021e975700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0218155b00 0x7f0218151600 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f020c009130 tx=0x7f020c017000).stop 2024-01-24T15:14:22.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.090+0000 7f021e975700 1 -- 172.21.15.29:0/1272357533 shutdown_connections 2024-01-24T15:14:22.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.090+0000 7f021e975700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02181563d0 0x7f021813b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.090+0000 7f021e975700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02180669e0 0x7f0218139f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.090+0000 7f021e975700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f020405f270 0x7f0204061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.090+0000 7f021e975700 1 --2- 172.21.15.29:0/1272357533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0218155b00 0x7f0218151600 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.090+0000 7f021e975700 1 -- 172.21.15.29:0/1272357533 >> 172.21.15.29:0/1272357533 conn(0x7f02180c8bb0 msgr2=0x7f02180be850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:22.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.090+0000 7f021e975700 1 -- 172.21.15.29:0/1272357533 shutdown_connections 2024-01-24T15:14:22.093 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.090+0000 7f021e975700 1 -- 172.21.15.29:0/1272357533 wait complete. 2024-01-24T15:14:22.094 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 82 2024-01-24T15:14:22.508 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T15:14:16.673720+0000', 'modified': '2024-01-24T15:14:19.686795+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9072}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9072': {'gid': 9072, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/1929837915', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 1929837915}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 1929837915}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-24T15:14:22.508 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T15:14:22.509 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-24T15:14:22.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.649+0000 7fc4a2b24700 1 Processor -- start 2024-01-24T15:14:22.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.649+0000 7fc4a2b24700 1 -- start start 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a2b24700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc49c151de0 0x7fc49c155470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a2b24700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc49c0638c0 0x7fc49c0664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a2b24700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49c066990 0x7fc49c13d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a2b24700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49c158800 con 0x7fc49c151de0 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a2b24700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49c066e20 con 0x7fc49c0638c0 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a2b24700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c157ed0 con 0x7fc49c066990 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a08c0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc49c151de0 0x7fc49c155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a08c0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc49c151de0 0x7fc49c155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39336/0 (socket says 172.21.15.29:39336) 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a08c0700 1 -- 172.21.15.29:0/2872462377 learned_addr learned my addr 172.21.15.29:0/2872462377 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a10c1700 1 --2- 172.21.15.29:0/2872462377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49c066990 0x7fc49c13d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc49bfff700 1 --2- 172.21.15.29:0/2872462377 >> v2:172.21.15.163:3300/0 conn(0x7fc49c0638c0 0x7fc49c0664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a08c0700 1 -- 172.21.15.29:0/2872462377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49c066990 msgr2=0x7fc49c13d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/2872462377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49c066990 0x7fc49c13d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a08c0700 1 -- 172.21.15.29:0/2872462377 >> v2:172.21.15.163:3300/0 conn(0x7fc49c0638c0 msgr2=0x7fc49c0664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.650+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/2872462377 >> v2:172.21.15.163:3300/0 conn(0x7fc49c0638c0 0x7fc49c0664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc4a08c0700 1 -- 172.21.15.29:0/2872462377 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c1581b0 con 0x7fc49c151de0 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/2872462377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c151de0 0x7fc49c155470 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7fc49c0c69a0 tx=0x7fc4900090f0).ready entity=mon.0 client_cookie=676784fa5cfa1093 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc49b7fe700 1 -- 172.21.15.29:0/2872462377 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49000b600 con 0x7fc49c151de0 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc49b7fe700 1 -- 172.21.15.29:0/2872462377 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49000b760 con 0x7fc49c151de0 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc49b7fe700 1 -- 172.21.15.29:0/2872462377 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49000bae0 con 0x7fc49c151de0 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc4a2b24700 1 -- 172.21.15.29:0/2872462377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c151de0 msgr2=0x7fc49c155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc4a2b24700 1 --2- 172.21.15.29:0/2872462377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c151de0 0x7fc49c155470 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7fc49c0c69a0 tx=0x7fc4900090f0).stop 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc4a2b24700 1 -- 172.21.15.29:0/2872462377 shutdown_connections 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc4a2b24700 1 --2- 172.21.15.29:0/2872462377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49c066990 0x7fc49c13d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc4a2b24700 1 --2- 172.21.15.29:0/2872462377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c151de0 0x7fc49c155470 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc4a2b24700 1 --2- 172.21.15.29:0/2872462377 >> v2:172.21.15.163:3300/0 conn(0x7fc49c0638c0 0x7fc49c0664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc4a2b24700 1 -- 172.21.15.29:0/2872462377 >> 172.21.15.29:0/2872462377 conn(0x7fc49c0c5640 msgr2=0x7fc49c0c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.651+0000 7fc4a2b24700 1 -- 172.21.15.29:0/2872462377 shutdown_connections 2024-01-24T15:14:22.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.652+0000 7fc4a2b24700 1 -- 172.21.15.29:0/2872462377 wait complete. 2024-01-24T15:14:22.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.653+0000 7fc4a2b24700 1 Processor -- start 2024-01-24T15:14:22.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.653+0000 7fc4a2b24700 1 -- start start 2024-01-24T15:14:22.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc4a2b24700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49c0638c0 0x7fc49c168be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc4a2b24700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c066990 0x7fc49c13d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc4a2b24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc49c151de0 0x7fc49c0a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc4a2b24700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0c29e0 con 0x7fc49c066990 2024-01-24T15:14:22.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc4a2b24700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0a84d0 con 0x7fc49c151de0 2024-01-24T15:14:22.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc4a2b24700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c1690d0 con 0x7fc49c0638c0 2024-01-24T15:14:22.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc49bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c066990 0x7fc49c13d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc49bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c066990 0x7fc49c13d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39348/0 (socket says 172.21.15.29:39348) 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc49bfff700 1 -- 172.21.15.29:0/1239009487 learned_addr learned my addr 172.21.15.29:0/1239009487 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc4a10c1700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc49c151de0 0x7fc49c0a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49c0638c0 0x7fc49c168be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc49bfff700 1 -- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49c0638c0 msgr2=0x7fc49c168be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc49bfff700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49c0638c0 0x7fc49c168be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc49bfff700 1 -- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc49c151de0 msgr2=0x7fc49c0a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc49bfff700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc49c151de0 0x7fc49c0a9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc49bfff700 1 -- 172.21.15.29:0/1239009487 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc490009370 con 0x7fc49c066990 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc4a10c1700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc49c151de0 0x7fc49c0a9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.654+0000 7fc49bfff700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c066990 0x7fc49c13d7f0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7fc48800d890 tx=0x7fc488008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.655+0000 7fc498ff9700 1 -- 172.21.15.29:0/1239009487 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4880125f0 con 0x7fc49c066990 2024-01-24T15:14:22.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.655+0000 7fc498ff9700 1 -- 172.21.15.29:0/1239009487 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc488012750 con 0x7fc49c066990 2024-01-24T15:14:22.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.655+0000 7fc4a2b24700 1 -- 172.21.15.29:0/1239009487 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c13dce0 con 0x7fc49c066990 2024-01-24T15:14:22.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.655+0000 7fc498ff9700 1 -- 172.21.15.29:0/1239009487 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488012ad0 con 0x7fc49c066990 2024-01-24T15:14:22.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.655+0000 7fc4a2b24700 1 -- 172.21.15.29:0/1239009487 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c1654b0 con 0x7fc49c066990 2024-01-24T15:14:22.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.656+0000 7fc498ff9700 1 -- 172.21.15.29:0/1239009487 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc488013480 con 0x7fc49c066990 2024-01-24T15:14:22.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.656+0000 7fc498ff9700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc48005f270 0x7fc480061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.657+0000 7fc498ff9700 1 -- 172.21.15.29:0/1239009487 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fc48807a550 con 0x7fc49c066990 2024-01-24T15:14:22.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.657+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc48005f270 0x7fc480061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.657+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc48005f270 0x7fc480061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fc490009fd0 tx=0x7fc490017000).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:22.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.659+0000 7fc4b44efb80 1 Processor -- start 2024-01-24T15:14:22.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.659+0000 7fc4b44efb80 1 -- start start 2024-01-24T15:14:22.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.659+0000 7fc4b44efb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 0x55fdbc146a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.659+0000 7fc4b44efb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc18f2c0 0x55fdbc141900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.659+0000 7fc4b44efb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc141df0 0x55fdbc142270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.659+0000 7fc4b44efb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdbc0bd3d0 con 0x55fdbc146620 2024-01-24T15:14:22.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.659+0000 7fc4b44efb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdbc082be0 con 0x55fdbc141df0 2024-01-24T15:14:22.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.659+0000 7fc4b44efb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdbc098260 con 0x55fdbc18f2c0 2024-01-24T15:14:22.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc4a08c0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 0x55fdbc146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc4a08c0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 0x55fdbc146a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39350/0 (socket says 172.21.15.29:39350) 2024-01-24T15:14:22.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc4a08c0700 1 -- 172.21.15.29:0/712111958 learned_addr learned my addr 172.21.15.29:0/712111958 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:22.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc49bfff700 1 --2- 172.21.15.29:0/712111958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc18f2c0 0x55fdbc141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc4a10c1700 1 --2- 172.21.15.29:0/712111958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc141df0 0x55fdbc142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc4a08c0700 1 -- 172.21.15.29:0/712111958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc18f2c0 msgr2=0x55fdbc141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:22.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/712111958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc18f2c0 0x55fdbc141900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc4a08c0700 1 -- 172.21.15.29:0/712111958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc141df0 msgr2=0x55fdbc142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:22.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/712111958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc141df0 0x55fdbc142270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc4a08c0700 1 -- 172.21.15.29:0/712111958 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdbc0912f0 con 0x55fdbc146620 2024-01-24T15:14:22.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc49bfff700 1 --2- 172.21.15.29:0/712111958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc18f2c0 0x55fdbc141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:14:22.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/712111958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 0x55fdbc146a00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7fc490000cc0 tx=0x7fc49001e070).ready entity=mon.0 client_cookie=2ae18f35d17d3a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:22.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc485ffb700 1 -- 172.21.15.29:0/712111958 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49001c5f0 con 0x55fdbc146620 2024-01-24T15:14:22.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc485ffb700 1 -- 172.21.15.29:0/712111958 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49001c750 con 0x55fdbc146620 2024-01-24T15:14:22.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.660+0000 7fc485ffb700 1 -- 172.21.15.29:0/712111958 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49001cad0 con 0x55fdbc146620 2024-01-24T15:14:22.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.661+0000 7fc4b44efb80 1 -- 172.21.15.29:0/712111958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 msgr2=0x55fdbc146a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.661+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/712111958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 0x55fdbc146a00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7fc490000cc0 tx=0x7fc49001e070).stop 2024-01-24T15:14:22.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.661+0000 7fc4b44efb80 1 -- 172.21.15.29:0/712111958 shutdown_connections 2024-01-24T15:14:22.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.661+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/712111958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc141df0 0x55fdbc142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.661+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/712111958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 0x55fdbc146a00 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.661+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/712111958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc18f2c0 0x55fdbc141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.661+0000 7fc4b44efb80 1 -- 172.21.15.29:0/712111958 >> 172.21.15.29:0/712111958 conn(0x55fdbbfaf2e0 msgr2=0x55fdbc18f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:22.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.661+0000 7fc4b44efb80 1 -- 172.21.15.29:0/712111958 shutdown_connections 2024-01-24T15:14:22.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.661+0000 7fc4b44efb80 1 -- 172.21.15.29:0/712111958 wait complete. 2024-01-24T15:14:22.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.662+0000 7fc4b44efb80 1 -- start start 2024-01-24T15:14:22.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.662+0000 7fc4b44efb80 1 Processor -- start 2024-01-24T15:14:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4b44efb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc141df0 0x55fdbc170cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4b44efb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc146620 0x55fdbc1711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4b44efb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc18f2c0 0x55fdbc142270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4b44efb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdbc0983a0 con 0x55fdbc141df0 2024-01-24T15:14:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4b44efb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdbc098620 con 0x55fdbc18f2c0 2024-01-24T15:14:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4b44efb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdbc063990 con 0x55fdbc146620 2024-01-24T15:14:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4b44efb80 10 client.0 ll_register_callbacks cb 0x55fdbc106610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T15:14:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4a08c0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc141df0 0x55fdbc170cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4a08c0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc141df0 0x55fdbc170cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39352/0 (socket says 172.21.15.29:39352) 2024-01-24T15:14:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4a08c0700 1 -- 172.21.15.29:0/4192565250 learned_addr learned my addr 172.21.15.29:0/4192565250 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc49bfff700 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc146620 0x55fdbc1711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4a10c1700 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc18f2c0 0x55fdbc142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4a08c0700 1 -- 172.21.15.29:0/4192565250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc146620 msgr2=0x55fdbc1711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc146620 0x55fdbc1711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4a08c0700 1 -- 172.21.15.29:0/4192565250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc18f2c0 msgr2=0x55fdbc142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc18f2c0 0x55fdbc142270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4a08c0700 1 -- 172.21.15.29:0/4192565250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c1581b0 con 0x55fdbc141df0 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc141df0 0x55fdbc170cf0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7fc490006e80 tx=0x7fc490027000).ready entity=mon.0 client_cookie=118e136e47037d1a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.663+0000 7fc47affd700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc47affd700 1 -- 172.21.15.29:0/4192565250 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49002f730 con 0x55fdbc141df0 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc47affd700 1 -- 172.21.15.29:0/4192565250 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fc49002f890 con 0x55fdbc141df0 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc4b44efb80 1 -- 172.21.15.29:0/4192565250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fdbc140df0 con 0x55fdbc141df0 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc47affd700 1 -- 172.21.15.29:0/4192565250 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc49002fc60 con 0x55fdbc141df0 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc4b44efb80 10 client.9198 fetch_fsmap learned FSMap version 82 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc4b44efb80 1 -- 172.21.15.29:0/4192565250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55fdbc08c520 con 0x55fdbc141df0 2024-01-24T15:14:22.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc47affd700 1 -- 172.21.15.29:0/4192565250 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fdbc140df0 con 0x55fdbc141df0 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc47affd700 1 -- 172.21.15.29:0/4192565250 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 82) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc490026400 con 0x55fdbc141df0 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc4b44efb80 10 client.9198 fetch_fsmap finished waiting for FSMap version 82 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc4b44efb80 10 client.9198 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc4b44efb80 20 client.9198 populate_metadata read hostname 'smithi029' 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.29:6834/1929837915,v1:172.21.15.29:6835/1929837915] conn(0x55fdbc16f590 0x55fdbc16f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc4b44efb80 4 client.9198 mds_command: new command op to 9072 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc4b44efb80 1 -- 172.21.15.29:0/4192565250 --> [v2:172.21.15.29:6834/1929837915,v1:172.21.15.29:6835/1929837915] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdbc08dd60 con 0x55fdbc16f590 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.664+0000 7fc49bfff700 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.29:6834/1929837915,v1:172.21.15.29:6835/1929837915] conn(0x55fdbc16f590 0x55fdbc16f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.665+0000 7fc49bfff700 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.29:6834/1929837915,v1:172.21.15.29:6835/1929837915] conn(0x55fdbc16f590 0x55fdbc16f970 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=df8a8a9d02fe1fd1 server_cookie=31c923305126b3e3 in_seq=0 out_seq=0 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.665+0000 7fc47affd700 10 client.9198 ms_handle_connect on v2:172.21.15.29:6834/1929837915 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.665+0000 7fc47affd700 1 -- 172.21.15.29:0/4192565250 <== mds.0 v2:172.21.15.29:6834/1929837915 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x55fdbc08dd60 con 0x55fdbc16f590 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.665+0000 7fc47affd700 10 client.9198 handle_command_reply: tid=0 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.665+0000 7fc4b44efb80 1 client.9198 shutdown 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.665+0000 7fc4b44efb80 1 -- 172.21.15.29:0/4192565250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc141df0 msgr2=0x55fdbc170cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.665+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc141df0 0x55fdbc170cf0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7fc490006e80 tx=0x7fc490027000).stop 2024-01-24T15:14:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.665+0000 7fc4b44efb80 1 -- 172.21.15.29:0/4192565250 shutdown_connections 2024-01-24T15:14:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.665+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.29:6834/1929837915,v1:172.21.15.29:6835/1929837915] conn(0x55fdbc16f590 0x55fdbc16f970 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.666+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc18f2c0 0x55fdbc142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.666+0000 7fc47affd700 0 client.9198 ms_handle_reset on v2:172.21.15.29:6834/1929837915 2024-01-24T15:14:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.666+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc141df0 0x55fdbc170cf0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.666+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/4192565250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc146620 0x55fdbc1711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.666+0000 7fc4b44efb80 1 -- 172.21.15.29:0/4192565250 >> 172.21.15.29:0/4192565250 conn(0x55fdbbfaf2e0 msgr2=0x55fdbc144690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.666+0000 7fc4b44efb80 1 -- 172.21.15.29:0/4192565250 shutdown_connections 2024-01-24T15:14:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.666+0000 7fc4b44efb80 1 -- 172.21.15.29:0/4192565250 wait complete. 2024-01-24T15:14:22.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.666+0000 7fc4b44efb80 20 client.9198 trim_cache size 0 max 16384 2024-01-24T15:14:22.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.683+0000 7fc4b44efb80 1 Processor -- start 2024-01-24T15:14:22.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.683+0000 7fc4b44efb80 1 -- start start 2024-01-24T15:14:22.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.684+0000 7fc4b44efb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc141df0 0x55fdbc1708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.684+0000 7fc4b44efb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 0x55fdbc16ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.684+0000 7fc4b44efb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc18f2c0 0x55fdbc148410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.684+0000 7fc4b44efb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdbc174a80 con 0x55fdbc146620 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.684+0000 7fc4b44efb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdbc147570 con 0x55fdbc18f2c0 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.684+0000 7fc4b44efb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdbc13b0e0 con 0x55fdbc141df0 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.684+0000 7fc49bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 0x55fdbc16ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.684+0000 7fc49bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 0x55fdbc16ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39364/0 (socket says 172.21.15.29:39364) 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.684+0000 7fc49bfff700 1 -- 172.21.15.29:0/988411301 learned_addr learned my addr 172.21.15.29:0/988411301 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.684+0000 7fc4a10c1700 1 --2- 172.21.15.29:0/988411301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc18f2c0 0x55fdbc148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.684+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/988411301 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc141df0 0x55fdbc1708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc49bfff700 1 -- 172.21.15.29:0/988411301 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc141df0 msgr2=0x55fdbc1708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc49bfff700 1 --2- 172.21.15.29:0/988411301 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc141df0 0x55fdbc1708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc49bfff700 1 -- 172.21.15.29:0/988411301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc18f2c0 msgr2=0x55fdbc148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc49bfff700 1 --2- 172.21.15.29:0/988411301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc18f2c0 0x55fdbc148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc49bfff700 1 -- 172.21.15.29:0/988411301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdbc08c270 con 0x55fdbc146620 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc49bfff700 1 --2- 172.21.15.29:0/988411301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 0x55fdbc16ffe0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7fc48804ee00 tx=0x7fc48807a040).ready entity=mon.0 client_cookie=1c72d82ffdd36cb5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc4867fc700 1 -- 172.21.15.29:0/988411301 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48804d0e0 con 0x55fdbc146620 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc4867fc700 1 -- 172.21.15.29:0/988411301 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48804d270 con 0x55fdbc146620 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc4867fc700 1 -- 172.21.15.29:0/988411301 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4880780f0 con 0x55fdbc146620 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc4b44efb80 1 -- 172.21.15.29:0/988411301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 msgr2=0x55fdbc16ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/988411301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 0x55fdbc16ffe0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7fc48804ee00 tx=0x7fc48807a040).stop 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc4b44efb80 1 -- 172.21.15.29:0/988411301 shutdown_connections 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/988411301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc18f2c0 0x55fdbc148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/988411301 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc141df0 0x55fdbc1708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/988411301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc146620 0x55fdbc16ffe0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc4b44efb80 1 -- 172.21.15.29:0/988411301 >> 172.21.15.29:0/988411301 conn(0x55fdbbfaf2e0 msgr2=0x55fdbc1438b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.685+0000 7fc4b44efb80 1 -- 172.21.15.29:0/988411301 shutdown_connections 2024-01-24T15:14:22.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.686+0000 7fc4b44efb80 1 -- 172.21.15.29:0/988411301 wait complete. 2024-01-24T15:14:22.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.686+0000 7fc4b44efb80 1 -- start start 2024-01-24T15:14:22.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.686+0000 7fc4b44efb80 1 Processor -- start 2024-01-24T15:14:22.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.687+0000 7fc4b44efb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc141df0 0x55fdbc1a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.687+0000 7fc4b44efb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc146620 0x55fdbc1a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.687+0000 7fc4b44efb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc18f2c0 0x55fdbc187fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.687+0000 7fc4b44efb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdbc191500 con 0x55fdbc18f2c0 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.687+0000 7fc4b44efb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdbc1a5350 con 0x55fdbc141df0 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.687+0000 7fc4b44efb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdbc1f1fb0 con 0x55fdbc146620 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.687+0000 7fc4b44efb80 10 client.0 ll_register_callbacks cb 0x55fdbc106610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.687+0000 7fc4a10c1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc18f2c0 0x55fdbc187fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.687+0000 7fc4a10c1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc18f2c0 0x55fdbc187fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39380/0 (socket says 172.21.15.29:39380) 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.687+0000 7fc4a10c1700 1 -- 172.21.15.29:0/2714590517 learned_addr learned my addr 172.21.15.29:0/2714590517 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.687+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc141df0 0x55fdbc1a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.687+0000 7fc49bfff700 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc146620 0x55fdbc1a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc4a10c1700 1 -- 172.21.15.29:0/2714590517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc146620 msgr2=0x55fdbc1a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc4a10c1700 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc146620 0x55fdbc1a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc4a10c1700 1 -- 172.21.15.29:0/2714590517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc141df0 msgr2=0x55fdbc1a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc4a10c1700 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc141df0 0x55fdbc1a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc4a10c1700 1 -- 172.21.15.29:0/2714590517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c1654b0 con 0x55fdbc18f2c0 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc4a10c1700 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc18f2c0 0x55fdbc187fb0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7fc49c0c8320 tx=0x7fc49400b040).ready entity=mon.0 client_cookie=52c58177fe593030 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc47affd700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc47affd700 1 -- 172.21.15.29:0/2714590517 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494009dc0 con 0x55fdbc18f2c0 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc4b44efb80 1 -- 172.21.15.29:0/2714590517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc49001c750 con 0x55fdbc18f2c0 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc47affd700 1 -- 172.21.15.29:0/2714590517 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fc4940079e0 con 0x55fdbc18f2c0 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc47affd700 1 -- 172.21.15.29:0/2714590517 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc494007ba0 con 0x55fdbc18f2c0 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc4b44efb80 10 client.9204 fetch_fsmap learned FSMap version 82 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc4b44efb80 1 -- 172.21.15.29:0/2714590517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55fdbbf05f80 con 0x55fdbc18f2c0 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc47affd700 1 -- 172.21.15.29:0/2714590517 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49001c750 con 0x55fdbc18f2c0 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc47affd700 1 -- 172.21.15.29:0/2714590517 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 82) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc494007d00 con 0x55fdbc18f2c0 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.688+0000 7fc4b44efb80 10 client.9204 fetch_fsmap finished waiting for FSMap version 82 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 10 client.9204 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 20 client.9204 populate_metadata read hostname 'smithi029' 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.29:6834/1929837915,v1:172.21.15.29:6835/1929837915] conn(0x55fdbc1884a0 0x55fdbc188880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 4 client.9204 mds_command: new command op to 9072 tid=0[{"prefix": "status"}] 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 1 -- 172.21.15.29:0/2714590517 --> [v2:172.21.15.29:6834/1929837915,v1:172.21.15.29:6835/1929837915] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fdbc0847d0 con 0x55fdbc1884a0 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.29:6834/1929837915,v1:172.21.15.29:6835/1929837915] conn(0x55fdbc1884a0 0x55fdbc188880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc47affd700 10 client.9204 ms_handle_connect on v2:172.21.15.29:6834/1929837915 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4a08c0700 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.29:6834/1929837915,v1:172.21.15.29:6835/1929837915] conn(0x55fdbc1884a0 0x55fdbc188880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a1dc0db964a455b1 server_cookie=d4bf2b4b526d4f7 in_seq=0 out_seq=0 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc47affd700 1 -- 172.21.15.29:0/2714590517 <== mds.0 v2:172.21.15.29:6834/1929837915 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55fdbc0847d0 con 0x55fdbc1884a0 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc47affd700 10 client.9204 handle_command_reply: tid=0 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 1 client.9204 shutdown 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 1 -- 172.21.15.29:0/2714590517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc18f2c0 msgr2=0x55fdbc187fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc18f2c0 0x55fdbc187fb0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7fc49c0c8320 tx=0x7fc49400b040).stop 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 1 -- 172.21.15.29:0/2714590517 shutdown_connections 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.29:6834/1929837915,v1:172.21.15.29:6835/1929837915] conn(0x55fdbc1884a0 0x55fdbc188880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdbc18f2c0 0x55fdbc187fb0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdbc141df0 0x55fdbc1a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc47affd700 0 client.9204 ms_handle_reset on v2:172.21.15.29:6834/1929837915 2024-01-24T15:14:22.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 1 --2- 172.21.15.29:0/2714590517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55fdbc146620 0x55fdbc1a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.689+0000 7fc4b44efb80 1 -- 172.21.15.29:0/2714590517 >> 172.21.15.29:0/2714590517 conn(0x55fdbbfaf2e0 msgr2=0x55fdbc16d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:22.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4b44efb80 1 -- 172.21.15.29:0/2714590517 shutdown_connections 2024-01-24T15:14:22.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4b44efb80 1 -- 172.21.15.29:0/2714590517 wait complete. 2024-01-24T15:14:22.697 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4b44efb80 20 client.9204 trim_cache size 0 max 16384 2024-01-24T15:14:22.697 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:14:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T15:14:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T15:14:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 9072, 2024-01-24T15:14:22.697 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T15:14:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T15:14:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T15:14:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 6.0150301590000002, 2024-01-24T15:14:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 82, 2024-01-24T15:14:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 193, 2024-01-24T15:14:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 193, 2024-01-24T15:14:22.698 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 11.470951815999999 2024-01-24T15:14:22.698 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:14:22.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4a2b24700 1 -- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc48005f270 msgr2=0x7fc480061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4a2b24700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc48005f270 0x7fc480061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fc490009fd0 tx=0x7fc490017000).stop 2024-01-24T15:14:22.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4a2b24700 1 -- 172.21.15.29:0/1239009487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c066990 msgr2=0x7fc49c13d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4a2b24700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c066990 0x7fc49c13d7f0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7fc48800d890 tx=0x7fc488008040).stop 2024-01-24T15:14:22.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4a2b24700 1 -- 172.21.15.29:0/1239009487 shutdown_connections 2024-01-24T15:14:22.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4a2b24700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc49c151de0 0x7fc49c0a9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4a2b24700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49c0638c0 0x7fc49c168be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4a2b24700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc48005f270 0x7fc480061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4a2b24700 1 --2- 172.21.15.29:0/1239009487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49c066990 0x7fc49c13d7f0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4a2b24700 1 -- 172.21.15.29:0/1239009487 >> 172.21.15.29:0/1239009487 conn(0x7fc49c0c5640 msgr2=0x7fc49c0c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:22.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4a2b24700 1 -- 172.21.15.29:0/1239009487 shutdown_connections 2024-01-24T15:14:22.700 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.690+0000 7fc4a2b24700 1 -- 172.21.15.29:0/1239009487 wait complete. 2024-01-24T15:14:22.706 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T15:14:22.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.819+0000 7f1f86db9700 1 Processor -- start 2024-01-24T15:14:22.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.819+0000 7f1f86db9700 1 -- start start 2024-01-24T15:14:22.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.820+0000 7f1f86db9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1f80157480 0x7f1f80065230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.820+0000 7f1f86db9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1f800c6b50 0x7f1f800a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.820+0000 7f1f86db9700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f800c6fe0 con 0x7f1f80157860 2024-01-24T15:14:22.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.820+0000 7f1f86db9700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f800c72c0 con 0x7f1f800c6b50 2024-01-24T15:14:22.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.820+0000 7f1f86db9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f800c75a0 con 0x7f1f80157480 2024-01-24T15:14:22.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.820+0000 7f1f7ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1f800c6b50 0x7f1f800a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.820+0000 7f1f85356700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1f80157860 0x7f1f800be490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:35066/0 (socket says 172.21.15.29:35066) 2024-01-24T15:14:22.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.820+0000 7f1f7ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1f800c6b50 0x7f1f800a5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39390/0 (socket says 172.21.15.29:39390) 2024-01-24T15:14:22.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.820+0000 7f1f85356700 1 -- 172.21.15.29:0/1713116609 learned_addr learned my addr 172.21.15.29:0/1713116609 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:22.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.820+0000 7f1f84b55700 1 --2- 172.21.15.29:0/1713116609 >> v2:172.21.15.163:3300/0 conn(0x7f1f80157480 0x7f1f80065230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.821+0000 7f1f7ffff700 1 -- 172.21.15.29:0/1713116609 >> v2:172.21.15.163:3300/0 conn(0x7f1f80157480 msgr2=0x7f1f80065230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:22.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.821+0000 7f1f7ffff700 1 --2- 172.21.15.29:0/1713116609 >> v2:172.21.15.163:3300/0 conn(0x7f1f80157480 0x7f1f80065230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.821+0000 7f1f7ffff700 1 -- 172.21.15.29:0/1713116609 >> v1:172.21.15.29:6789/0 conn(0x7f1f80157860 legacy=0x7f1f800be490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.821+0000 7f1f7ffff700 1 -- 172.21.15.29:0/1713116609 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f800c7880 con 0x7f1f800c6b50 2024-01-24T15:14:22.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.821+0000 7f1f7f7fe700 1 -- 172.21.15.29:0/1713116609 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154463121 0 0) 0x7f1f800c6fe0 con 0x7f1f80157860 2024-01-24T15:14:22.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.821+0000 7f1f7ffff700 1 --2- 172.21.15.29:0/1713116609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f800c6b50 0x7f1f800a5a20 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7f1f800691d0 tx=0x7f1f68009060).ready entity=mon.0 client_cookie=8880bad6d4d41d10 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:22.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.821+0000 7f1f7f7fe700 1 -- 172.21.15.29:0/1713116609 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6800f510 con 0x7f1f800c6b50 2024-01-24T15:14:22.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.821+0000 7f1f7f7fe700 1 -- 172.21.15.29:0/1713116609 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6800f670 con 0x7f1f800c6b50 2024-01-24T15:14:22.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.821+0000 7f1f7f7fe700 1 -- 172.21.15.29:0/1713116609 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6800f9f0 con 0x7f1f800c6b50 2024-01-24T15:14:22.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.821+0000 7f1f86db9700 1 -- 172.21.15.29:0/1713116609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f800c6b50 msgr2=0x7f1f800a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.821+0000 7f1f86db9700 1 --2- 172.21.15.29:0/1713116609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f800c6b50 0x7f1f800a5a20 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7f1f800691d0 tx=0x7f1f68009060).stop 2024-01-24T15:14:22.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.821+0000 7f1f86db9700 1 -- 172.21.15.29:0/1713116609 shutdown_connections 2024-01-24T15:14:22.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.822+0000 7f1f86db9700 1 --2- 172.21.15.29:0/1713116609 >> v2:172.21.15.163:3300/0 conn(0x7f1f80157480 0x7f1f80065230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.822+0000 7f1f86db9700 1 --2- 172.21.15.29:0/1713116609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f800c6b50 0x7f1f800a5a20 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.822+0000 7f1f86db9700 1 -- 172.21.15.29:0/1713116609 >> 172.21.15.29:0/1713116609 conn(0x7f1f800922f0 msgr2=0x7f1f80062f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:22.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.822+0000 7f1f86db9700 1 -- 172.21.15.29:0/1713116609 shutdown_connections 2024-01-24T15:14:22.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.822+0000 7f1f86db9700 1 -- 172.21.15.29:0/1713116609 wait complete. 2024-01-24T15:14:22.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.823+0000 7f1f86db9700 1 Processor -- start 2024-01-24T15:14:22.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.823+0000 7f1f86db9700 1 -- start start 2024-01-24T15:14:22.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f86db9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80157480 0x7f1f8014ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f86db9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f80157860 0x7f1f800a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f86db9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f800aa190 0x7f1f801515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f86db9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f80152e90 con 0x7f1f80157480 2024-01-24T15:14:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f86db9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f800aa610 con 0x7f1f80157860 2024-01-24T15:14:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f86db9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f80150420 con 0x7f1f800aa190 2024-01-24T15:14:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f84b55700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80157480 0x7f1f8014ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f84b55700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80157480 0x7f1f8014ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39398/0 (socket says 172.21.15.29:39398) 2024-01-24T15:14:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f84b55700 1 -- 172.21.15.29:0/3843774791 learned_addr learned my addr 172.21.15.29:0/3843774791 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f7ffff700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f80157860 0x7f1f800a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f85356700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f800aa190 0x7f1f801515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f84b55700 1 -- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f800aa190 msgr2=0x7f1f801515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f84b55700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f800aa190 0x7f1f801515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f84b55700 1 -- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f80157860 msgr2=0x7f1f800a9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f84b55700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f80157860 0x7f1f800a9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f84b55700 1 -- 172.21.15.29:0/3843774791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f680092a0 con 0x7f1f80157480 2024-01-24T15:14:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f85356700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f800aa190 0x7f1f801515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:14:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.824+0000 7f1f84b55700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80157480 0x7f1f8014ff30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f1f7000e8d0 tx=0x7f1f70006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.825+0000 7f1f7cff9700 1 -- 172.21.15.29:0/3843774791 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7000c640 con 0x7f1f80157480 2024-01-24T15:14:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.825+0000 7f1f7cff9700 1 -- 172.21.15.29:0/3843774791 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7000c7a0 con 0x7f1f80157480 2024-01-24T15:14:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.825+0000 7f1f86db9700 1 -- 172.21.15.29:0/3843774791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f80152230 con 0x7f1f80157480 2024-01-24T15:14:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.825+0000 7f1f7cff9700 1 -- 172.21.15.29:0/3843774791 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7000cb20 con 0x7f1f80157480 2024-01-24T15:14:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.825+0000 7f1f86db9700 1 -- 172.21.15.29:0/3843774791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f80152840 con 0x7f1f80157480 2024-01-24T15:14:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.826+0000 7f1f86db9700 1 -- 172.21.15.29:0/3843774791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f80153170 con 0x7f1f80157480 2024-01-24T15:14:22.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.827+0000 7f1f7cff9700 1 -- 172.21.15.29:0/3843774791 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1f7001b400 con 0x7f1f80157480 2024-01-24T15:14:22.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.827+0000 7f1f7cff9700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1f6005f270 0x7f1f60061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:22.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.827+0000 7f1f7cff9700 1 -- 172.21.15.29:0/3843774791 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f1f7007b1d0 con 0x7f1f80157480 2024-01-24T15:14:22.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.828+0000 7f1f7ffff700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1f6005f270 0x7f1f60061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:22.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.828+0000 7f1f7ffff700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1f6005f270 0x7f1f60061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1f800691d0 tx=0x7f1f68017000).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:22.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:22.831+0000 7f1f7cff9700 1 -- 172.21.15.29:0/3843774791 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1f7004bf50 con 0x7f1f80157480 2024-01-24T15:14:23.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.056+0000 7f1f86db9700 1 -- 172.21.15.29:0/3843774791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1f8005f400 con 0x7f1f80157480 2024-01-24T15:14:23.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.056+0000 7f1f7cff9700 1 -- 172.21.15.29:0/3843774791 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1f7004f570 con 0x7f1f80157480 2024-01-24T15:14:23.058 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-24T15:14:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.058+0000 7f1f86db9700 1 -- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1f6005f270 msgr2=0x7f1f60061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.058+0000 7f1f86db9700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1f6005f270 0x7f1f60061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1f800691d0 tx=0x7f1f68017000).stop 2024-01-24T15:14:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.058+0000 7f1f86db9700 1 -- 172.21.15.29:0/3843774791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80157480 msgr2=0x7f1f8014ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.058+0000 7f1f86db9700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80157480 0x7f1f8014ff30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f1f7000e8d0 tx=0x7f1f70006f40).stop 2024-01-24T15:14:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.058+0000 7f1f86db9700 1 -- 172.21.15.29:0/3843774791 shutdown_connections 2024-01-24T15:14:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.058+0000 7f1f86db9700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f800aa190 0x7f1f801515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.058+0000 7f1f86db9700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f1f6005f270 0x7f1f60061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.058+0000 7f1f86db9700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80157480 0x7f1f8014ff30 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.058+0000 7f1f86db9700 1 --2- 172.21.15.29:0/3843774791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f80157860 0x7f1f800a9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.058+0000 7f1f86db9700 1 -- 172.21.15.29:0/3843774791 >> 172.21.15.29:0/3843774791 conn(0x7f1f800922f0 msgr2=0x7f1f800a7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.059+0000 7f1f86db9700 1 -- 172.21.15.29:0/3843774791 shutdown_connections 2024-01-24T15:14:23.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.059+0000 7f1f86db9700 1 -- 172.21.15.29:0/3843774791 wait complete. 2024-01-24T15:14:23.073 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-24T15:14:23.073 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T15:14:23.073 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T15:14:23.073 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-24T15:14:23.073 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-24T15:14:23.073 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-24T15:14:23.073 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-24T15:14:23.073 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T15:14:23.073 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T15:14:23.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:14:23.074 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T15:14:23.094 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T15:14:23.094 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T15:14:23.094 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T15:14:23.094 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:14:23.094 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T15:14:23.094 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:14:23.094 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T15:14:23.094 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T15:14:23.094 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T15:14:23.094 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T15:14:23.094 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 37787sec preferred_lft 37787sec 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T15:14:23.095 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff 2024-01-24T15:14:23.096 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T15:14:23.096 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:14:23.096 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::24bc:1cff:fe57:e704/64 scope link 2024-01-24T15:14:23.096 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:14:23.096 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T15:14:23.096 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T15:14:23.096 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::2034:aaff:fef3:f12b/64 scope link 2024-01-24T15:14:23.096 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:14:23.096 INFO:teuthology.orchestra.run.smithi029.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T15:14:23.096 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether ba:ca:42:4a:9e:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T15:14:23.096 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::b8ca:42ff:fe4a:9e52/64 scope link 2024-01-24T15:14:23.096 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:14:23.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:14:23.097 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T15:14:23.165 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T15:14:23.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:14:23.167 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T15:14:23.232 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T15:14:23.232 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T15:14:23.232 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T15:14:23.232 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T15:14:23.232 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T15:14:23.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:14:23.232 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T15:14:23.233 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T15:14:23.233 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T15:14:23.233 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T15:14:23.233 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T15:14:23.233 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T15:14:23.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:14:23.356 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T15:14:23.377 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-24T15:14:23.377 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T15:14:23.377 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T15:14:23.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:14:23.377 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T15:14:23.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:14:23.452 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-01-24T15:14:23.515 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T15:14:23.515 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T15:14:23.607 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T15:14:23.608 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T15:14:23.631 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T15:14:23.631 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T15:14:23.634 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T15:14:23.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:14:23.634 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-24T15:14:23.667 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-24T15:14:23.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.843+0000 7fd8b7c65700 1 Processor -- start 2024-01-24T15:14:23.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.843+0000 7fd8b7c65700 1 -- start start 2024-01-24T15:14:23.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.844+0000 7fd8b7c65700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd8b0069240 0x7fd8b0150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:23.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.844+0000 7fd8b7c65700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b006a350 con 0x7fd8b0150bd0 2024-01-24T15:14:23.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.844+0000 7fd8b7c65700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b0069740 con 0x7fd8b01514a0 2024-01-24T15:14:23.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.844+0000 7fd8b7c65700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b0069a20 con 0x7fd8b0069240 2024-01-24T15:14:23.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.844+0000 7fd8b5200700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd8b0069240 0x7fd8b0150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:23.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.844+0000 7fd8b5200700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd8b0069240 0x7fd8b0150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58878/0 (socket says 172.21.15.29:58878) 2024-01-24T15:14:23.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.845+0000 7fd8b5200700 1 -- 172.21.15.29:0/3801210316 learned_addr learned my addr 172.21.15.29:0/3801210316 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:23.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.845+0000 7fd8b5200700 1 -- 172.21.15.29:0/3801210316 >> v1:172.21.15.163:6789/0 conn(0x7fd8b01514a0 legacy=0x7fd8b00bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:23.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.845+0000 7fd8b5200700 1 -- 172.21.15.29:0/3801210316 >> v1:172.21.15.29:6789/0 conn(0x7fd8b0150bd0 legacy=0x7fd8b0065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:23.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.845+0000 7fd8b49ff700 1 -- 172.21.15.29:0/3801210316 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222158264 0 0) 0x7fd8b006a350 con 0x7fd8b0150bd0 2024-01-24T15:14:23.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.845+0000 7fd8b5200700 1 -- 172.21.15.29:0/3801210316 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0069d00 con 0x7fd8b0069240 2024-01-24T15:14:23.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.845+0000 7fd8b5200700 1 --2- 172.21.15.29:0/3801210316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b0069240 0x7fd8b0150fb0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7fd8a0006fd0 tx=0x7fd8a00090f0).ready entity=mon.0 client_cookie=9cbc7a1440f633be server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:23.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.845+0000 7fd8b49ff700 1 -- 172.21.15.29:0/3801210316 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000b600 con 0x7fd8b0069240 2024-01-24T15:14:23.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.845+0000 7fd8b49ff700 1 -- 172.21.15.29:0/3801210316 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a000b760 con 0x7fd8b0069240 2024-01-24T15:14:23.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.845+0000 7fd8b49ff700 1 -- 172.21.15.29:0/3801210316 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000bae0 con 0x7fd8b0069240 2024-01-24T15:14:23.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.846+0000 7fd8b7c65700 1 -- 172.21.15.29:0/3801210316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b0069240 msgr2=0x7fd8b0150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:23.849 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.846+0000 7fd8b7c65700 1 --2- 172.21.15.29:0/3801210316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b0069240 0x7fd8b0150fb0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7fd8a0006fd0 tx=0x7fd8a00090f0).stop 2024-01-24T15:14:23.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.846+0000 7fd8b7c65700 1 -- 172.21.15.29:0/3801210316 shutdown_connections 2024-01-24T15:14:23.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.846+0000 7fd8b7c65700 1 --2- 172.21.15.29:0/3801210316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b0069240 0x7fd8b0150fb0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:23.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.846+0000 7fd8b7c65700 1 -- 172.21.15.29:0/3801210316 >> 172.21.15.29:0/3801210316 conn(0x7fd8b00c8580 msgr2=0x7fd8b00c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:23.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.846+0000 7fd8b7c65700 1 -- 172.21.15.29:0/3801210316 shutdown_connections 2024-01-24T15:14:23.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.846+0000 7fd8b7c65700 1 -- 172.21.15.29:0/3801210316 wait complete. 2024-01-24T15:14:23.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.847+0000 7fd8b7c65700 1 Processor -- start 2024-01-24T15:14:23.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.848+0000 7fd8b7c65700 1 -- start start 2024-01-24T15:14:23.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.848+0000 7fd8b7c65700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8b0069240 0x7fd8b0148430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:23.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.848+0000 7fd8b7c65700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0150bd0 0x7fd8b014e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:23.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.848+0000 7fd8b7c65700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b01514a0 0x7fd8b0062d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:23.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.848+0000 7fd8b7c65700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0165ae0 con 0x7fd8b01514a0 2024-01-24T15:14:23.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.848+0000 7fd8b7c65700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b014ea60 con 0x7fd8b0069240 2024-01-24T15:14:23.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.848+0000 7fd8b7c65700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0165d30 con 0x7fd8b0150bd0 2024-01-24T15:14:23.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.848+0000 7fd8b6202700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b01514a0 0x7fd8b0062d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:23.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b6202700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b01514a0 0x7fd8b0062d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58882/0 (socket says 172.21.15.29:58882) 2024-01-24T15:14:23.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b6202700 1 -- 172.21.15.29:0/2760478781 learned_addr learned my addr 172.21.15.29:0/2760478781 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:23.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b5200700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0150bd0 0x7fd8b014e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:23.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b5a01700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8b0069240 0x7fd8b0148430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:23.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b6202700 1 -- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0150bd0 msgr2=0x7fd8b014e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:23.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b6202700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0150bd0 0x7fd8b014e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:23.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b6202700 1 -- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8b0069240 msgr2=0x7fd8b0148430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:23.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b6202700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8b0069240 0x7fd8b0148430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:23.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b6202700 1 -- 172.21.15.29:0/2760478781 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a0009370 con 0x7fd8b01514a0 2024-01-24T15:14:23.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b5a01700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8b0069240 0x7fd8b0148430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:14:23.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b6202700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b01514a0 0x7fd8b0062d00 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7fd8a800c7c0 tx=0x7fd8a8001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:23.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8a5ffb700 1 -- 172.21.15.29:0/2760478781 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8010510 con 0x7fd8b01514a0 2024-01-24T15:14:23.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8a5ffb700 1 -- 172.21.15.29:0/2760478781 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8010670 con 0x7fd8b01514a0 2024-01-24T15:14:23.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b7c65700 1 -- 172.21.15.29:0/2760478781 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b0166010 con 0x7fd8b01514a0 2024-01-24T15:14:23.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8a5ffb700 1 -- 172.21.15.29:0/2760478781 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80109f0 con 0x7fd8b01514a0 2024-01-24T15:14:23.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.849+0000 7fd8b7c65700 1 -- 172.21.15.29:0/2760478781 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b00ace40 con 0x7fd8b01514a0 2024-01-24T15:14:23.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.850+0000 7fd8b7c65700 1 -- 172.21.15.29:0/2760478781 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b00475e0 con 0x7fd8b01514a0 2024-01-24T15:14:23.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.851+0000 7fd8a5ffb700 1 -- 172.21.15.29:0/2760478781 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd8a801b400 con 0x7fd8b01514a0 2024-01-24T15:14:23.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.851+0000 7fd8a5ffb700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd89405f270 0x7fd894061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:23.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.851+0000 7fd8a5ffb700 1 -- 172.21.15.29:0/2760478781 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fd8a807a520 con 0x7fd8b01514a0 2024-01-24T15:14:23.855 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.851+0000 7fd8b5a01700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd89405f270 0x7fd894061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:23.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.852+0000 7fd8b5a01700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd89405f270 0x7fd894061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd8b014c0e0 tx=0x7fd8ac009040).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:23.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:23.855+0000 7fd8a5ffb700 1 -- 172.21.15.29:0/2760478781 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd8a804b1e0 con 0x7fd8b01514a0 2024-01-24T15:14:24.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.066+0000 7fd8b7c65700 1 -- 172.21.15.29:0/2760478781 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8b00c1f00 con 0x7fd89405f270 2024-01-24T15:14:24.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.068+0000 7fd8a5ffb700 1 -- 172.21.15.29:0/2760478781 <== mgr.7535 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd8b00c1f00 con 0x7fd89405f270 2024-01-24T15:14:24.070 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T15:14:24.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:14:24.070 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-01-24T15:14:24.071 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:14:24.071 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T15:14:24.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.070+0000 7fd8b7c65700 1 -- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd89405f270 msgr2=0x7fd894061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:24.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.070+0000 7fd8b7c65700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd89405f270 0x7fd894061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd8b014c0e0 tx=0x7fd8ac009040).stop 2024-01-24T15:14:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.070+0000 7fd8b7c65700 1 -- 172.21.15.29:0/2760478781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b01514a0 msgr2=0x7fd8b0062d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.070+0000 7fd8b7c65700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b01514a0 0x7fd8b0062d00 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7fd8a800c7c0 tx=0x7fd8a8001510).stop 2024-01-24T15:14:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.070+0000 7fd8b7c65700 1 -- 172.21.15.29:0/2760478781 shutdown_connections 2024-01-24T15:14:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.070+0000 7fd8b7c65700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd89405f270 0x7fd894061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.070+0000 7fd8b7c65700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8b01514a0 0x7fd8b0062d00 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.070+0000 7fd8b7c65700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8b0069240 0x7fd8b0148430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.070+0000 7fd8b7c65700 1 --2- 172.21.15.29:0/2760478781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8b0150bd0 0x7fd8b014e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.070+0000 7fd8b7c65700 1 -- 172.21.15.29:0/2760478781 >> 172.21.15.29:0/2760478781 conn(0x7fd8b00c8580 msgr2=0x7fd8b014b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.070+0000 7fd8b7c65700 1 -- 172.21.15.29:0/2760478781 shutdown_connections 2024-01-24T15:14:24.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.070+0000 7fd8b7c65700 1 -- 172.21.15.29:0/2760478781 wait complete. 2024-01-24T15:14:24.086 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-01-24T15:14:24.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.216+0000 7fd7b26ab700 1 Processor -- start 2024-01-24T15:14:24.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.216+0000 7fd7b26ab700 1 -- start start 2024-01-24T15:14:24.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.216+0000 7fd7b26ab700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7ac155160 0x7fd7ac0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:24.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.216+0000 7fd7b26ab700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ac154cd0 con 0x7fd7ac062ca0 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7b26ab700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ac1555c0 con 0x7fd7ac14b140 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7b26ab700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ac063110 con 0x7fd7ac155160 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7ab7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7ac155160 0x7fd7ac0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7b0c48700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd7ac062ca0 0x7fd7ac069280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54050/0 (socket says 172.21.15.29:54050) 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7b0c48700 1 -- 172.21.15.29:0/2266785926 learned_addr learned my addr 172.21.15.29:0/2266785926 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7ab7fe700 1 -- 172.21.15.29:0/2266785926 >> v1:172.21.15.163:6789/0 conn(0x7fd7ac14b140 legacy=0x7fd7ac0bca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7ab7fe700 1 -- 172.21.15.29:0/2266785926 >> v1:172.21.15.29:6789/0 conn(0x7fd7ac062ca0 legacy=0x7fd7ac069280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7ab7fe700 1 -- 172.21.15.29:0/2266785926 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac1546a0 con 0x7fd7ac155160 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7aaffd700 1 -- 172.21.15.29:0/2266785926 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476914693 0 0) 0x7fd7ac154cd0 con 0x7fd7ac062ca0 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7ab7fe700 1 --2- 172.21.15.29:0/2266785926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7ac155160 0x7fd7ac0627b0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7fd7a0009760 tx=0x7fd7a0009060).ready entity=mon.0 client_cookie=f998dcf6df46168f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7aaffd700 1 -- 172.21.15.29:0/2266785926 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a000b580 con 0x7fd7ac155160 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7aaffd700 1 -- 172.21.15.29:0/2266785926 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a000b6e0 con 0x7fd7ac155160 2024-01-24T15:14:24.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.217+0000 7fd7aaffd700 1 -- 172.21.15.29:0/2266785926 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a000ba60 con 0x7fd7ac155160 2024-01-24T15:14:24.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.218+0000 7fd7b26ab700 1 -- 172.21.15.29:0/2266785926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7ac155160 msgr2=0x7fd7ac0627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:24.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.218+0000 7fd7b26ab700 1 --2- 172.21.15.29:0/2266785926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7ac155160 0x7fd7ac0627b0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7fd7a0009760 tx=0x7fd7a0009060).stop 2024-01-24T15:14:24.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.218+0000 7fd7b26ab700 1 -- 172.21.15.29:0/2266785926 shutdown_connections 2024-01-24T15:14:24.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.218+0000 7fd7b26ab700 1 --2- 172.21.15.29:0/2266785926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7ac155160 0x7fd7ac0627b0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.218+0000 7fd7b26ab700 1 -- 172.21.15.29:0/2266785926 >> 172.21.15.29:0/2266785926 conn(0x7fd7ac064390 msgr2=0x7fd7ac0c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:24.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.218+0000 7fd7b26ab700 1 -- 172.21.15.29:0/2266785926 shutdown_connections 2024-01-24T15:14:24.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.218+0000 7fd7b26ab700 1 -- 172.21.15.29:0/2266785926 wait complete. 2024-01-24T15:14:24.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.219+0000 7fd7b26ab700 1 Processor -- start 2024-01-24T15:14:24.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.220+0000 7fd7b26ab700 1 -- start start 2024-01-24T15:14:24.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.220+0000 7fd7b26ab700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7ac062ca0 0x7fd7ac14a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:24.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.220+0000 7fd7b26ab700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7ac14b140 0x7fd7ac14ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:24.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.220+0000 7fd7b26ab700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7ac155160 0x7fd7ac150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:24.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.220+0000 7fd7b26ab700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac1502e0 con 0x7fd7ac14b140 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.220+0000 7fd7b26ab700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac14add0 con 0x7fd7ac062ca0 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.220+0000 7fd7b26ab700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac14f6d0 con 0x7fd7ac155160 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.220+0000 7fd7ab7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7ac14b140 0x7fd7ac14ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7ab7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7ac14b140 0x7fd7ac14ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58910/0 (socket says 172.21.15.29:58910) 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7ab7fe700 1 -- 172.21.15.29:0/968914781 learned_addr learned my addr 172.21.15.29:0/968914781 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7abfff700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7ac062ca0 0x7fd7ac14a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7b0c48700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7ac155160 0x7fd7ac150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7ab7fe700 1 -- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7ac155160 msgr2=0x7fd7ac150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7ab7fe700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7ac155160 0x7fd7ac150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7ab7fe700 1 -- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7ac062ca0 msgr2=0x7fd7ac14a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7ab7fe700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7ac062ca0 0x7fd7ac14a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7ab7fe700 1 -- 172.21.15.29:0/968914781 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a00092a0 con 0x7fd7ac14b140 2024-01-24T15:14:24.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7b0c48700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7ac155160 0x7fd7ac150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:14:24.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7ab7fe700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7ac14b140 0x7fd7ac14ed10 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7fd7a0009eb0 tx=0x7fd7a0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:24.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd793fff700 1 -- 172.21.15.29:0/968914781 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a000b850 con 0x7fd7ac14b140 2024-01-24T15:14:24.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd793fff700 1 -- 172.21.15.29:0/968914781 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a000b9b0 con 0x7fd7ac14b140 2024-01-24T15:14:24.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7b26ab700 1 -- 172.21.15.29:0/968914781 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ac14f9b0 con 0x7fd7ac14b140 2024-01-24T15:14:24.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd793fff700 1 -- 172.21.15.29:0/968914781 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a001b620 con 0x7fd7ac14b140 2024-01-24T15:14:24.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.221+0000 7fd7b26ab700 1 -- 172.21.15.29:0/968914781 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac14ff70 con 0x7fd7ac14b140 2024-01-24T15:14:24.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.222+0000 7fd7b26ab700 1 -- 172.21.15.29:0/968914781 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ac150490 con 0x7fd7ac14b140 2024-01-24T15:14:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.223+0000 7fd793fff700 1 -- 172.21.15.29:0/968914781 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 74) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd7a001ba50 con 0x7fd7ac14b140 2024-01-24T15:14:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.223+0000 7fd793fff700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd79805f270 0x7fd798061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.223+0000 7fd793fff700 1 -- 172.21.15.29:0/968914781 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fd7a007c230 con 0x7fd7ac14b140 2024-01-24T15:14:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.223+0000 7fd7abfff700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd79805f270 0x7fd798061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:24.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.224+0000 7fd7abfff700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd79805f270 0x7fd798061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd79c003900 tx=0x7fd79c007d40).ready entity=mgr.7535 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:24.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.228+0000 7fd793fff700 1 -- 172.21.15.29:0/968914781 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd7ac150490 con 0x7fd7ac14b140 2024-01-24T15:14:24.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.461+0000 7fd7b26ab700 1 -- 172.21.15.29:0/968914781 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fd7ac150490 con 0x7fd7ac14b140 2024-01-24T15:14:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.713+0000 7fd793fff700 1 -- 172.21.15.29:0/968914781 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 75) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fd7a001bd00 con 0x7fd7ac14b140 2024-01-24T15:14:24.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.715+0000 7fd793fff700 1 -- 172.21.15.29:0/968914781 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v75) v1 ==== 92+0+0 (secure 0 0 0) 0x7fd7a0041b50 con 0x7fd7ac14b140 2024-01-24T15:14:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.718+0000 7fd7b26ab700 1 -- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd79805f270 msgr2=0x7fd798061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.718+0000 7fd7b26ab700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd79805f270 0x7fd798061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd79c003900 tx=0x7fd79c007d40).stop 2024-01-24T15:14:24.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.718+0000 7fd7b26ab700 1 -- 172.21.15.29:0/968914781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7ac14b140 msgr2=0x7fd7ac14ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:24.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.718+0000 7fd7b26ab700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7ac14b140 0x7fd7ac14ed10 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7fd7a0009eb0 tx=0x7fd7a0017000).stop 2024-01-24T15:14:24.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.718+0000 7fd7b26ab700 1 -- 172.21.15.29:0/968914781 shutdown_connections 2024-01-24T15:14:24.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.718+0000 7fd7b26ab700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7ac155160 0x7fd7ac150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.718+0000 7fd7b26ab700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7ac062ca0 0x7fd7ac14a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.718+0000 7fd7b26ab700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fd79805f270 0x7fd798061730 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.718+0000 7fd7b26ab700 1 --2- 172.21.15.29:0/968914781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7ac14b140 0x7fd7ac14ed10 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.718+0000 7fd7b26ab700 1 -- 172.21.15.29:0/968914781 >> 172.21.15.29:0/968914781 conn(0x7fd7ac064390 msgr2=0x7fd7ac0a6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:24.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.718+0000 7fd7b26ab700 1 -- 172.21.15.29:0/968914781 shutdown_connections 2024-01-24T15:14:24.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.718+0000 7fd7b26ab700 1 -- 172.21.15.29:0/968914781 wait complete. 2024-01-24T15:14:24.737 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-01-24T15:14:24.863 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:24.862+0000 7f116f151680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:14:24.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.872+0000 7f5985166700 1 Processor -- start 2024-01-24T15:14:24.875 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.873+0000 7f5985166700 1 -- start start 2024-01-24T15:14:24.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.873+0000 7f5985166700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5980064760 0x7f5980064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:24.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.873+0000 7f5985166700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f59800c8490 0x7f59800a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:24.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.873+0000 7f5985166700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5980152240 con 0x7f5980154700 2024-01-24T15:14:24.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.873+0000 7f5985166700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59801586c0 con 0x7f59800c8490 2024-01-24T15:14:24.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.873+0000 7f5985166700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59800c0d50 con 0x7f5980064760 2024-01-24T15:14:24.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597e59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f59800c8490 0x7f59800a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:24.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597f59e700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5980154700 0x7f5980154b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54054/0 (socket says 172.21.15.29:54054) 2024-01-24T15:14:24.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597f59e700 1 -- 172.21.15.29:0/1373387057 learned_addr learned my addr 172.21.15.29:0/1373387057 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:24.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597ed9d700 1 --2- 172.21.15.29:0/1373387057 >> v2:172.21.15.163:3300/0 conn(0x7f5980064760 0x7f5980064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597e59c700 1 -- 172.21.15.29:0/1373387057 >> v2:172.21.15.163:3300/0 conn(0x7f5980064760 msgr2=0x7f5980064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597e59c700 1 --2- 172.21.15.29:0/1373387057 >> v2:172.21.15.163:3300/0 conn(0x7f5980064760 0x7f5980064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597e59c700 1 -- 172.21.15.29:0/1373387057 >> v1:172.21.15.29:6789/0 conn(0x7f5980154700 legacy=0x7f5980154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597e59c700 1 -- 172.21.15.29:0/1373387057 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59800aa670 con 0x7f59800c8490 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597dd9b700 1 -- 172.21.15.29:0/1373387057 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290549285 0 0) 0x7f5980152240 con 0x7f5980154700 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597e59c700 1 --2- 172.21.15.29:0/1373387057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59800c8490 0x7f59800a9d90 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f5980084450 tx=0x7f5974009060).ready entity=mon.0 client_cookie=aa781b383ca8423c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597dd9b700 1 -- 172.21.15.29:0/1373387057 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597400f510 con 0x7f59800c8490 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597dd9b700 1 -- 172.21.15.29:0/1373387057 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597400f670 con 0x7f59800c8490 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.874+0000 7f597dd9b700 1 -- 172.21.15.29:0/1373387057 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597400f9f0 con 0x7f59800c8490 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.875+0000 7f5985166700 1 -- 172.21.15.29:0/1373387057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59800c8490 msgr2=0x7f59800a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.875+0000 7f5985166700 1 --2- 172.21.15.29:0/1373387057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59800c8490 0x7f59800a9d90 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f5980084450 tx=0x7f5974009060).stop 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.875+0000 7f5985166700 1 -- 172.21.15.29:0/1373387057 shutdown_connections 2024-01-24T15:14:24.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.875+0000 7f5985166700 1 --2- 172.21.15.29:0/1373387057 >> v2:172.21.15.163:3300/0 conn(0x7f5980064760 0x7f5980064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.875+0000 7f5985166700 1 --2- 172.21.15.29:0/1373387057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59800c8490 0x7f59800a9d90 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.875+0000 7f5985166700 1 -- 172.21.15.29:0/1373387057 >> 172.21.15.29:0/1373387057 conn(0x7f59800cb8b0 msgr2=0x7f59800c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:24.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.875+0000 7f5985166700 1 -- 172.21.15.29:0/1373387057 shutdown_connections 2024-01-24T15:14:24.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.875+0000 7f5985166700 1 -- 172.21.15.29:0/1373387057 wait complete. 2024-01-24T15:14:24.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.877+0000 7f5985166700 1 Processor -- start 2024-01-24T15:14:24.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.877+0000 7f5985166700 1 -- start start 2024-01-24T15:14:24.882 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:24.878+0000 7fb97db37680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:14:24.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.878+0000 7f5985166700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5980064760 0x7f59800abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:24.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.878+0000 7f5985166700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59800c8490 0x7f59800ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:24.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.878+0000 7f5985166700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5980154700 0x7f59801564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:24.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.878+0000 7f5985166700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59800acc70 con 0x7f59800c8490 2024-01-24T15:14:24.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.878+0000 7f5985166700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59800ac990 con 0x7f5980154700 2024-01-24T15:14:24.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.878+0000 7f5985166700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59800c1f90 con 0x7f5980064760 2024-01-24T15:14:24.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.878+0000 7f597e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59800c8490 0x7f59800ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:24.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.878+0000 7f597f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5980154700 0x7f59801564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:24.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.878+0000 7f597ed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5980064760 0x7f59800abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.878+0000 7f597e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59800c8490 0x7f59800ac0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58942/0 (socket says 172.21.15.29:58942) 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.878+0000 7f597e59c700 1 -- 172.21.15.29:0/1411364204 learned_addr learned my addr 172.21.15.29:0/1411364204 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.879+0000 7f597ed9d700 1 -- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5980154700 msgr2=0x7f59801564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.879+0000 7f597ed9d700 1 --2- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5980154700 0x7f59801564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.879+0000 7f597ed9d700 1 -- 172.21.15.29:0/1411364204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59800c8490 msgr2=0x7f59800ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.879+0000 7f597ed9d700 1 --2- 172.21.15.29:0/1411364204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59800c8490 0x7f59800ac0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.879+0000 7f597ed9d700 1 -- 172.21.15.29:0/1411364204 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59740092a0 con 0x7f5980064760 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.879+0000 7f597e59c700 1 --2- 172.21.15.29:0/1411364204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59800c8490 0x7f59800ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.879+0000 7f597ed9d700 1 --2- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5980064760 0x7f59800abbf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f597000e900 tx=0x7f5970009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.879+0000 7f5966ffd700 1 -- 172.21.15.29:0/1411364204 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970012600 con 0x7f5980064760 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.879+0000 7f5966ffd700 1 -- 172.21.15.29:0/1411364204 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5970012760 con 0x7f5980064760 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.879+0000 7f5985166700 1 -- 172.21.15.29:0/1411364204 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59800c2270 con 0x7f5980064760 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.879+0000 7f5966ffd700 1 -- 172.21.15.29:0/1411364204 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970012ae0 con 0x7f5980064760 2024-01-24T15:14:24.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.879+0000 7f5985166700 1 -- 172.21.15.29:0/1411364204 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59800ac650 con 0x7f5980064760 2024-01-24T15:14:24.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.880+0000 7f5985166700 1 -- 172.21.15.29:0/1411364204 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59800475e0 con 0x7f5980064760 2024-01-24T15:14:24.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.881+0000 7f5966ffd700 1 -- 172.21.15.29:0/1411364204 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 75) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f597001b400 con 0x7f5980064760 2024-01-24T15:14:24.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.881+0000 7f5966ffd700 1 --2- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f596c05f2c0 0x7f596c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:24.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.881+0000 7f5966ffd700 1 -- 172.21.15.29:0/1411364204 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f597007ac70 con 0x7f5980064760 2024-01-24T15:14:24.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.881+0000 7f597e59c700 1 -- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f596c05f2c0 msgr2=0x7f596c061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:24.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.881+0000 7f597e59c700 1 --2- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f596c05f2c0 0x7f596c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:14:24.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:24.886+0000 7f5966ffd700 1 -- 172.21.15.29:0/1411364204 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f597004b990 con 0x7f5980064760 2024-01-24T15:14:24.938 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:24.926+0000 7f116f151680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:14:24.953 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:24.951+0000 7fb97db37680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:14:25.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.070+0000 7f5985166700 1 -- 172.21.15.29:0/1411364204 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f5980157d90 con 0x7f5980064760 2024-01-24T15:14:25.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.075+0000 7f5966ffd700 1 -- 172.21.15.29:0/1411364204 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f597004efb0 con 0x7f5980064760 2024-01-24T15:14:25.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.077+0000 7f5985166700 1 -- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f596c05f2c0 msgr2=0x7f596c061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:14:25.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.077+0000 7f5985166700 1 --2- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f596c05f2c0 0x7f596c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:25.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.077+0000 7f5985166700 1 -- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5980064760 msgr2=0x7f59800abbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:25.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.077+0000 7f5985166700 1 --2- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5980064760 0x7f59800abbf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f597000e900 tx=0x7f5970009060).stop 2024-01-24T15:14:25.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.077+0000 7f5985166700 1 -- 172.21.15.29:0/1411364204 shutdown_connections 2024-01-24T15:14:25.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.077+0000 7f5985166700 1 --2- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5980154700 0x7f59801564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:25.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.077+0000 7f5985166700 1 --2- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5980064760 0x7f59800abbf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:25.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.077+0000 7f5985166700 1 --2- 172.21.15.29:0/1411364204 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f596c05f2c0 0x7f596c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:25.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.077+0000 7f5985166700 1 --2- 172.21.15.29:0/1411364204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59800c8490 0x7f59800ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:25.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.077+0000 7f5985166700 1 -- 172.21.15.29:0/1411364204 >> 172.21.15.29:0/1411364204 conn(0x7f59800cb8b0 msgr2=0x7f5980153c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:25.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.077+0000 7f5985166700 1 -- 172.21.15.29:0/1411364204 shutdown_connections 2024-01-24T15:14:25.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.077+0000 7f5985166700 1 -- 172.21.15.29:0/1411364204 wait complete. 2024-01-24T15:14:25.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:14:25.093 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-01-24T15:14:25.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:14:25.120 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2024-01-24T15:14:25.183 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/test_restart 1M 2024-01-24T15:14:25.206 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:25.203+0000 7f116f151680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:14:25.272 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:25.270+0000 7fb97db37680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:14:25.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.360+0000 7fc7aabf3700 1 Processor -- start 2024-01-24T15:14:25.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.360+0000 7fc7aabf3700 1 -- start start 2024-01-24T15:14:25.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.361+0000 7fc7aabf3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc7a4066450 0x7fc7a4066830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.361+0000 7fc7aabf3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7a4159da0 0x7fc7a415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.361+0000 7fc7aabf3700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a41530b0 con 0x7fc7a4152540 2024-01-24T15:14:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.361+0000 7fc7aabf3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a4153300 con 0x7fc7a4066450 2024-01-24T15:14:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.361+0000 7fc7aabf3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a4067b10 con 0x7fc7a4159da0 2024-01-24T15:14:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.361+0000 7fc7a3fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc7a4152540 0x7fc7a41529a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54068/0 (socket says 172.21.15.29:54068) 2024-01-24T15:14:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.361+0000 7fc7a3fff700 1 -- 172.21.15.29:0/2867800653 learned_addr learned my addr 172.21.15.29:0/2867800653 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.361+0000 7fc7a9190700 1 --2- 172.21.15.29:0/2867800653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7a4159da0 0x7fc7a415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.361+0000 7fc7a898f700 1 --2- 172.21.15.29:0/2867800653 >> v2:172.21.15.163:3300/0 conn(0x7fc7a4066450 0x7fc7a4066830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.361+0000 7fc7a37fe700 1 -- 172.21.15.29:0/2867800653 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2440359884 0 0) 0x7fc7a41530b0 con 0x7fc7a4152540 2024-01-24T15:14:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.361+0000 7fc7a37fe700 1 -- 172.21.15.29:0/2867800653 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7940033f0 con 0x7fc7a4152540 2024-01-24T15:14:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.362+0000 7fc7a9190700 1 -- 172.21.15.29:0/2867800653 >> v2:172.21.15.163:3300/0 conn(0x7fc7a4066450 msgr2=0x7fc7a4066830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.362+0000 7fc7a9190700 1 --2- 172.21.15.29:0/2867800653 >> v2:172.21.15.163:3300/0 conn(0x7fc7a4066450 0x7fc7a4066830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.362+0000 7fc7a9190700 1 -- 172.21.15.29:0/2867800653 >> v1:172.21.15.29:6789/0 conn(0x7fc7a4152540 legacy=0x7fc7a41529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.362+0000 7fc7a9190700 1 -- 172.21.15.29:0/2867800653 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4067df0 con 0x7fc7a4159da0 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.362+0000 7fc7a898f700 1 --2- 172.21.15.29:0/2867800653 >> v2:172.21.15.163:3300/0 conn(0x7fc7a4066450 0x7fc7a4066830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.362+0000 7fc7a9190700 1 --2- 172.21.15.29:0/2867800653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7a4159da0 0x7fc7a415c190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fc79c009e60 tx=0x7fc79c009780).ready entity=mon.2 client_cookie=fc07c375d347b6c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.362+0000 7fc7a37fe700 1 -- 172.21.15.29:0/2867800653 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00e070 con 0x7fc7a4159da0 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.362+0000 7fc7a37fe700 1 -- 172.21.15.29:0/2867800653 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c00bab0 con 0x7fc7a4159da0 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.362+0000 7fc7a37fe700 1 -- 172.21.15.29:0/2867800653 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00be30 con 0x7fc7a4159da0 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.363+0000 7fc7aabf3700 1 -- 172.21.15.29:0/2867800653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7a4159da0 msgr2=0x7fc7a415c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.363+0000 7fc7aabf3700 1 --2- 172.21.15.29:0/2867800653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7a4159da0 0x7fc7a415c190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fc79c009e60 tx=0x7fc79c009780).stop 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.363+0000 7fc7aabf3700 1 -- 172.21.15.29:0/2867800653 shutdown_connections 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.363+0000 7fc7aabf3700 1 --2- 172.21.15.29:0/2867800653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7a4159da0 0x7fc7a415c190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.363+0000 7fc7aabf3700 1 --2- 172.21.15.29:0/2867800653 >> v2:172.21.15.163:3300/0 conn(0x7fc7a4066450 0x7fc7a4066830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.363+0000 7fc7aabf3700 1 -- 172.21.15.29:0/2867800653 >> 172.21.15.29:0/2867800653 conn(0x7fc7a40c8bd0 msgr2=0x7fc7a40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:25.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.363+0000 7fc7aabf3700 1 -- 172.21.15.29:0/2867800653 shutdown_connections 2024-01-24T15:14:25.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.363+0000 7fc7aabf3700 1 -- 172.21.15.29:0/2867800653 wait complete. 2024-01-24T15:14:25.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.364+0000 7fc7aabf3700 1 Processor -- start 2024-01-24T15:14:25.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.364+0000 7fc7aabf3700 1 -- start start 2024-01-24T15:14:25.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.365+0000 7fc7aabf3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a4066450 0x7fc7a4137f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:25.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.365+0000 7fc7aabf3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7a4152540 0x7fc7a413b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:25.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.365+0000 7fc7aabf3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7a4159da0 0x7fc7a4157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:25.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.365+0000 7fc7aabf3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a4064020 con 0x7fc7a4159da0 2024-01-24T15:14:25.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.365+0000 7fc7aabf3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a4138f40 con 0x7fc7a4066450 2024-01-24T15:14:25.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.365+0000 7fc7aabf3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a40639d0 con 0x7fc7a4152540 2024-01-24T15:14:25.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.365+0000 7fc7a9190700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7a4159da0 0x7fc7a4157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:25.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.365+0000 7fc7a3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7a4152540 0x7fc7a413b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:25.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.365+0000 7fc7a9190700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7a4159da0 0x7fc7a4157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58958/0 (socket says 172.21.15.29:58958) 2024-01-24T15:14:25.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.365+0000 7fc7a9190700 1 -- 172.21.15.29:0/764754060 learned_addr learned my addr 172.21.15.29:0/764754060 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:25.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.365+0000 7fc7a898f700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a4066450 0x7fc7a4137f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:25.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.366+0000 7fc7a3fff700 1 -- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a4066450 msgr2=0x7fc7a4137f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:25.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.366+0000 7fc7a3fff700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a4066450 0x7fc7a4137f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:25.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.366+0000 7fc7a3fff700 1 -- 172.21.15.29:0/764754060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7a4159da0 msgr2=0x7fc7a4157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:25.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.366+0000 7fc7a3fff700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7a4159da0 0x7fc7a4157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:25.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.366+0000 7fc7a3fff700 1 -- 172.21.15.29:0/764754060 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980092a0 con 0x7fc7a4152540 2024-01-24T15:14:25.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.366+0000 7fc7a9190700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7a4159da0 0x7fc7a4157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:14:25.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.366+0000 7fc7a3fff700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7a4152540 0x7fc7a413b290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fc790001ed0 tx=0x7fc790007bd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:25.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.366+0000 7fc7a0ff9700 1 -- 172.21.15.29:0/764754060 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7900070d0 con 0x7fc7a4152540 2024-01-24T15:14:25.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.366+0000 7fc7a0ff9700 1 -- 172.21.15.29:0/764754060 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc790007230 con 0x7fc7a4152540 2024-01-24T15:14:25.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.366+0000 7fc7aabf3700 1 -- 172.21.15.29:0/764754060 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79c009a20 con 0x7fc7a4152540 2024-01-24T15:14:25.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.366+0000 7fc7a0ff9700 1 -- 172.21.15.29:0/764754060 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7900075f0 con 0x7fc7a4152540 2024-01-24T15:14:25.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.366+0000 7fc7aabf3700 1 -- 172.21.15.29:0/764754060 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a4139220 con 0x7fc7a4152540 2024-01-24T15:14:25.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.367+0000 7fc7aabf3700 1 -- 172.21.15.29:0/764754060 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a40475e0 con 0x7fc7a4152540 2024-01-24T15:14:25.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.368+0000 7fc7a0ff9700 1 -- 172.21.15.29:0/764754060 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 75) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fc7900059b0 con 0x7fc7a4152540 2024-01-24T15:14:25.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.368+0000 7fc7a0ff9700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 0x7fc788061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:25.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.368+0000 7fc7a0ff9700 1 -- 172.21.15.29:0/764754060 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fc790079f20 con 0x7fc7a4152540 2024-01-24T15:14:25.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.368+0000 7fc7a898f700 1 -- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 msgr2=0x7fc788061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:25.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.368+0000 7fc7a898f700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 0x7fc788061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:14:25.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.373+0000 7fc7a0ff9700 1 -- 172.21.15.29:0/764754060 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc79004ac40 con 0x7fc7a4152540 2024-01-24T15:14:25.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.569+0000 7fc7a898f700 1 -- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 msgr2=0x7fc788061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:25.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.569+0000 7fc7a898f700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 0x7fc788061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:14:25.580 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.578+0000 7fc7aabf3700 1 -- 172.21.15.29:0/764754060 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7a4066f90 con 0x7fc78805f2c0 2024-01-24T15:14:25.823 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:25.822+0000 7f116f151680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:14:25.896 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:25.894+0000 7f116f151680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:14:25.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.970+0000 7fc7a898f700 1 -- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 msgr2=0x7fc788061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:25.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:25.970+0000 7fc7a898f700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 0x7fc788061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-01-24T15:14:25.995 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:25.993+0000 7fb97db37680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:14:26.070 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:26.068+0000 7fb97db37680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:14:26.084 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:26.082+0000 7f116f151680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:14:26.285 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:26.283+0000 7fb97db37680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:14:26.508 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:26.507+0000 7f116f151680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:14:26.612 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:26.611+0000 7f116f151680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:14:26.764 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:26.762+0000 7fb97db37680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:14:26.767 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:26.765+0000 7f116f151680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:14:26.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:26.771+0000 7fc7a898f700 1 -- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 msgr2=0x7fc788061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:26.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:26.771+0000 7fc7a898f700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 0x7fc788061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-01-24T15:14:26.843 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:26.842+0000 7f116f151680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:14:26.882 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:26.880+0000 7fb97db37680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:14:26.911 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:26.910+0000 7f116f151680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:14:27.059 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:27.057+0000 7fb97db37680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:14:27.144 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:27.142+0000 7fb97db37680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:14:27.168 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:27.167+0000 7f116f151680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:14:27.221 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:27.219+0000 7fb97db37680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:14:27.352 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:27.351+0000 7f116f151680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:14:27.514 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:27.512+0000 7fb97db37680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:14:27.728 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:27.726+0000 7fb97db37680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:14:28.006 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:28.005+0000 7f116f151680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:14:28.076 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:28.075+0000 7f116f151680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:14:28.151 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:28.150+0000 7f116f151680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:14:28.298 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:28.297+0000 7f116f151680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:14:28.370 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:28.368+0000 7f116f151680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:14:28.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:28.373+0000 7fc7a898f700 1 -- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 msgr2=0x7fc788061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/43241 2024-01-24T15:14:28.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:28.373+0000 7fc7a898f700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 0x7fc788061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-01-24T15:14:28.482 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:28.480+0000 7fb97db37680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:14:28.487 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:28.485+0000 7f116f151680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:14:28.561 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:28.559+0000 7fb97db37680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:14:28.595 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:28.594+0000 7f116f151680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:14:28.646 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:28.644+0000 7fb97db37680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:14:28.814 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:28.812+0000 7fb97db37680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:14:28.886 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:28.885+0000 7f116f151680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:14:28.894 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:28.892+0000 7fb97db37680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:14:28.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:28.957+0000 7f116f151680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:14:28.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:28.978+0000 7fc7a0ff9700 1 -- 172.21.15.29:0/764754060 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mgrmap(e 76) v1 ==== 75287+0+0 (secure 0 0 0) 0x7fc790077740 con 0x7fc7a4152540 2024-01-24T15:14:28.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:28.979+0000 7fc7a0ff9700 1 -- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 msgr2=0x7fc788061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:14:28.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:28.979+0000 7fc7a0ff9700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 0x7fc788061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:14:29.032 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:29.030+0000 7fb97db37680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:14:29.047 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:29.045+0000 7f10e2ec7700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:29.047 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:29.045+0000 7f10e2ec7700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:29.047 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:29.045+0000 7f10e2ec7700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:29.047 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:29.045+0000 7f10e2ec7700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:29.048 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:29.045+0000 7f10e2ec7700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:29.048 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:29.045+0000 7f10e56cc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:29.048 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:29.045+0000 7f10e56cc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:29.048 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:29.045+0000 7f10e56cc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:29.048 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:29.045+0000 7f10e56cc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:29.048 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:14:29.045+0000 7f10e56cc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:14:29.156 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:29.154+0000 7fb97db37680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:14:29.491 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:29.489+0000 7fb97db37680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:14:29.573 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:14:29.571+0000 7fb97db37680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:14:29.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.982+0000 7fc7a0ff9700 1 -- 172.21.15.29:0/764754060 <== mon.2 v2:172.21.15.163:3301/0 8 ==== mgrmap(e 77) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fc790077ac0 con 0x7fc7a4152540 2024-01-24T15:14:29.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.982+0000 7fc7a0ff9700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 0x7fc788061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:29.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.982+0000 7fc7a0ff9700 1 -- 172.21.15.29:0/764754060 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7a4066f90 con 0x7fc78805f2c0 2024-01-24T15:14:29.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.985+0000 7fc7a898f700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 0x7fc788061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:29.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.986+0000 7fc7a898f700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 0x7fc788061700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fc7980096f0 tx=0x7fc798009060).ready entity=mgr.8803 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:29.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.997+0000 7fc7a0ff9700 1 -- 172.21.15.29:0/764754060 <== mgr.8803 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc7a4066f90 con 0x7fc78805f2c0 2024-01-24T15:14:29.999 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path /snap_test_dir1/test_restart 2024-01-24T15:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.999+0000 7fc7aabf3700 1 -- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 msgr2=0x7fc788061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.999+0000 7fc7aabf3700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 0x7fc788061700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fc7980096f0 tx=0x7fc798009060).stop 2024-01-24T15:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.999+0000 7fc7aabf3700 1 -- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7a4152540 msgr2=0x7fc7a413b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.999+0000 7fc7aabf3700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7a4152540 0x7fc7a413b290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fc790001ed0 tx=0x7fc790007bd0).stop 2024-01-24T15:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.999+0000 7fc7aabf3700 1 -- 172.21.15.29:0/764754060 shutdown_connections 2024-01-24T15:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.999+0000 7fc7aabf3700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc78805f2c0 0x7fc788061700 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.999+0000 7fc7aabf3700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7a4159da0 0x7fc7a4157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.999+0000 7fc7aabf3700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a4066450 0x7fc7a4137f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.999+0000 7fc7aabf3700 1 --2- 172.21.15.29:0/764754060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7a4152540 0x7fc7a413b290 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.999+0000 7fc7aabf3700 1 -- 172.21.15.29:0/764754060 >> 172.21.15.29:0/764754060 conn(0x7fc7a40c8bd0 msgr2=0x7fc7a4134430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.999+0000 7fc7aabf3700 1 -- 172.21.15.29:0/764754060 shutdown_connections 2024-01-24T15:14:30.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:29.999+0000 7fc7aabf3700 1 -- 172.21.15.29:0/764754060 wait complete. 2024-01-24T15:14:30.017 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2024-01-24T15:14:30.017 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/test_restart 1M 2024-01-24T15:14:30.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.155+0000 7f480f8eb700 1 Processor -- start 2024-01-24T15:14:30.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.155+0000 7f480f8eb700 1 -- start start 2024-01-24T15:14:30.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.156+0000 7f480f8eb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f48081350b0 0x7f48080664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:30.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.156+0000 7f480f8eb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48080c38c0 0x7f48080669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:30.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.156+0000 7f480f8eb700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4808068450 con 0x7f4808153240 2024-01-24T15:14:30.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.156+0000 7f480f8eb700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4808066f70 con 0x7f48081350b0 2024-01-24T15:14:30.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.156+0000 7f480f8eb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4808067b20 con 0x7f48080c38c0 2024-01-24T15:14:30.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.156+0000 7f480d687700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f48081350b0 0x7f48080664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:30.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.156+0000 7f480ce86700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4808153240 0x7f48080c31b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54074/0 (socket says 172.21.15.29:54074) 2024-01-24T15:14:30.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.156+0000 7f480ce86700 1 -- 172.21.15.29:0/1077330684 learned_addr learned my addr 172.21.15.29:0/1077330684 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:14:30.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.156+0000 7f480de88700 1 --2- 172.21.15.29:0/1077330684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48080c38c0 0x7f48080669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:30.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f480d687700 1 -- 172.21.15.29:0/1077330684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48080c38c0 msgr2=0x7f48080669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:14:30.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f480d687700 1 --2- 172.21.15.29:0/1077330684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48080c38c0 0x7f48080669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f480d687700 1 -- 172.21.15.29:0/1077330684 >> v1:172.21.15.29:6789/0 conn(0x7f4808153240 legacy=0x7f48080c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:30.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f480d687700 1 -- 172.21.15.29:0/1077330684 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4808067e00 con 0x7f48081350b0 2024-01-24T15:14:30.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f47fffff700 1 -- 172.21.15.29:0/1077330684 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762958138 0 0) 0x7f4808068450 con 0x7f4808153240 2024-01-24T15:14:30.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f480d687700 1 --2- 172.21.15.29:0/1077330684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48081350b0 0x7f48080664b0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f47f8009760 tx=0x7f47f8009060).ready entity=mon.0 client_cookie=c593cdfd3f255d49 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:30.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f47fffff700 1 -- 172.21.15.29:0/1077330684 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f800f510 con 0x7f48081350b0 2024-01-24T15:14:30.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f47fffff700 1 -- 172.21.15.29:0/1077330684 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f800f670 con 0x7f48081350b0 2024-01-24T15:14:30.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f47fffff700 1 -- 172.21.15.29:0/1077330684 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f800f9f0 con 0x7f48081350b0 2024-01-24T15:14:30.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f480f8eb700 1 -- 172.21.15.29:0/1077330684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48081350b0 msgr2=0x7f48080664b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:30.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f480f8eb700 1 --2- 172.21.15.29:0/1077330684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48081350b0 0x7f48080664b0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f47f8009760 tx=0x7f47f8009060).stop 2024-01-24T15:14:30.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f480f8eb700 1 -- 172.21.15.29:0/1077330684 shutdown_connections 2024-01-24T15:14:30.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f480f8eb700 1 --2- 172.21.15.29:0/1077330684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48080c38c0 0x7f48080669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f480f8eb700 1 --2- 172.21.15.29:0/1077330684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48081350b0 0x7f48080664b0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.157+0000 7f480f8eb700 1 -- 172.21.15.29:0/1077330684 >> 172.21.15.29:0/1077330684 conn(0x7f48080c8bb0 msgr2=0x7f48080c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:30.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.158+0000 7f480f8eb700 1 -- 172.21.15.29:0/1077330684 shutdown_connections 2024-01-24T15:14:30.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.158+0000 7f480f8eb700 1 -- 172.21.15.29:0/1077330684 wait complete. 2024-01-24T15:14:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.159+0000 7f480f8eb700 1 Processor -- start 2024-01-24T15:14:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.159+0000 7f480f8eb700 1 -- start start 2024-01-24T15:14:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480f8eb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48080c38c0 0x7f48081346b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:30.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480f8eb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48081350b0 0x7f48080acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480f8eb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4808153240 0x7f4808133af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480f8eb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48080c1fd0 con 0x7f4808153240 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480f8eb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48080c16e0 con 0x7f48081350b0 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480f8eb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48080c1980 con 0x7f48080c38c0 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480de88700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4808153240 0x7f4808133af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480de88700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4808153240 0x7f4808133af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:58988/0 (socket says 172.21.15.29:58988) 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480de88700 1 -- 172.21.15.29:0/1656883158 learned_addr learned my addr 172.21.15.29:0/1656883158 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480d687700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48080c38c0 0x7f48081346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480ce86700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48081350b0 0x7f48080acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480de88700 1 -- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48080c38c0 msgr2=0x7f48081346b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480de88700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48080c38c0 0x7f48081346b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480de88700 1 -- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48081350b0 msgr2=0x7f48080acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480de88700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48081350b0 0x7f48080acb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.160+0000 7f480de88700 1 -- 172.21.15.29:0/1656883158 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f80092a0 con 0x7f4808153240 2024-01-24T15:14:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.161+0000 7f480de88700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4808153240 0x7f4808133af0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f4800009e90 tx=0x7f480000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.161+0000 7f47fd7fa700 1 -- 172.21.15.29:0/1656883158 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480000cc80 con 0x7f4808153240 2024-01-24T15:14:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.161+0000 7f47fd7fa700 1 -- 172.21.15.29:0/1656883158 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480000f040 con 0x7f4808153240 2024-01-24T15:14:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.161+0000 7f480f8eb700 1 -- 172.21.15.29:0/1656883158 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48080c1c60 con 0x7f4808153240 2024-01-24T15:14:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.161+0000 7f47fd7fa700 1 -- 172.21.15.29:0/1656883158 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800013400 con 0x7f4808153240 2024-01-24T15:14:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.161+0000 7f480f8eb700 1 -- 172.21.15.29:0/1656883158 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48080ad020 con 0x7f4808153240 2024-01-24T15:14:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.162+0000 7f480f8eb700 1 -- 172.21.15.29:0/1656883158 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48080c2180 con 0x7f4808153240 2024-01-24T15:14:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.162+0000 7f47fd7fa700 1 -- 172.21.15.29:0/1656883158 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 77) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f48000137b0 con 0x7f4808153240 2024-01-24T15:14:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.163+0000 7f47fd7fa700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f47ec05f080 0x7f47ec061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:14:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.163+0000 7f47fd7fa700 1 -- 172.21.15.29:0/1656883158 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f480007aeb0 con 0x7f4808153240 2024-01-24T15:14:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.163+0000 7f480d687700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f47ec05f080 0x7f47ec061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:14:30.167 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.163+0000 7f480d687700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f47ec05f080 0x7f47ec061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f47f8009760 tx=0x7f47f8017000).ready entity=mgr.8803 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:14:30.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.167+0000 7f47fd7fa700 1 -- 172.21.15.29:0/1656883158 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f48080c2180 con 0x7f4808153240 2024-01-24T15:14:30.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.374+0000 7f480f8eb700 1 -- 172.21.15.29:0/1656883158 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4808152940 con 0x7f47ec05f080 2024-01-24T15:14:30.382 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.380+0000 7f47fd7fa700 1 -- 172.21.15.29:0/1656883158 <== mgr.8803 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f4808152940 con 0x7f47ec05f080 2024-01-24T15:14:30.382 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule activated for path /snap_test_dir1/test_restart 2024-01-24T15:14:30.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.382+0000 7f480f8eb700 1 -- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f47ec05f080 msgr2=0x7f47ec061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:30.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.382+0000 7f480f8eb700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f47ec05f080 0x7f47ec061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f47f8009760 tx=0x7f47f8017000).stop 2024-01-24T15:14:30.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.382+0000 7f480f8eb700 1 -- 172.21.15.29:0/1656883158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4808153240 msgr2=0x7f4808133af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:14:30.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.382+0000 7f480f8eb700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4808153240 0x7f4808133af0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f4800009e90 tx=0x7f480000b040).stop 2024-01-24T15:14:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.382+0000 7f480f8eb700 1 -- 172.21.15.29:0/1656883158 shutdown_connections 2024-01-24T15:14:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.382+0000 7f480f8eb700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7f47ec05f080 0x7f47ec061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.382+0000 7f480f8eb700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4808153240 0x7f4808133af0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.382+0000 7f480f8eb700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48080c38c0 0x7f48081346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.382+0000 7f480f8eb700 1 --2- 172.21.15.29:0/1656883158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48081350b0 0x7f48080acb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:14:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.382+0000 7f480f8eb700 1 -- 172.21.15.29:0/1656883158 >> 172.21.15.29:0/1656883158 conn(0x7f48080c8bb0 msgr2=0x7f480813b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:14:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.382+0000 7f480f8eb700 1 -- 172.21.15.29:0/1656883158 shutdown_connections 2024-01-24T15:14:30.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:14:30.382+0000 7f480f8eb700 1 -- 172.21.15.29:0/1656883158 wait complete. 2024-01-24T15:14:30.398 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2024-01-24T15:14:39.763 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:14:39.767 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:15:00.387 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:15:00.385+0000 7f10d4eab700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:15:00.387 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:15:00.385+0000 7f10d4eab700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:15:00.387 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:15:00.385+0000 7f10d4eab700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:15:00.388 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:15:00.385+0000 7f10d4eab700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:15:00.388 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:15:00.385+0000 7f10d4eab700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:15:09.835 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:15:09.839 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:15:40.056 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:15:40.059 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:16:10.137 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:16:10.140 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:16:40.208 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:16:40.211 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:17:10.280 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:17:10.283 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:17:40.351 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:17:40.354 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:18:10.423 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:18:10.427 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:18:40.512 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:18:40.515 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:19:10.600 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:19:10.603 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:19:40.705 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:19:40.708 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:20:10.776 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:20:10.780 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:20:40.848 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:20:40.851 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:21:10.928 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:21:10.932 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:21:40.993 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:21:40.996 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:22:11.062 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:22:11.066 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:22:41.151 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:22:41.154 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:23:11.216 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:23:11.219 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:23:41.343 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:23:41.346 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:24:11.439 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:24:11.442 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:24:41.515 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:24:41.519 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:25:11.589 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:25:11.593 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:25:15.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:25:15.399 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2024-01-24T15:25:15.424 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_15_00_UTC 2024-01-24T15:25:15.424 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_16_00_UTC 2024-01-24T15:25:15.424 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_17_00_UTC 2024-01-24T15:25:15.424 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_18_00_UTC 2024-01-24T15:25:15.424 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_19_00_UTC 2024-01-24T15:25:15.424 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_20_00_UTC 2024-01-24T15:25:15.424 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_21_00_UTC 2024-01-24T15:25:15.425 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_22_00_UTC 2024-01-24T15:25:15.425 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_23_00_UTC 2024-01-24T15:25:15.425 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_24_00_UTC 2024-01-24T15:25:15.425 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_25_00_UTC 2024-01-24T15:25:15.425 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-15_15_00_UTC', 'scheduled-2024-01-24-15_16_00_UTC', 'scheduled-2024-01-24-15_17_00_UTC', 'scheduled-2024-01-24-15_18_00_UTC', 'scheduled-2024-01-24-15_19_00_UTC', 'scheduled-2024-01-24-15_20_00_UTC', 'scheduled-2024-01-24-15_21_00_UTC', 'scheduled-2024-01-24-15_22_00_UTC', 'scheduled-2024-01-24-15_23_00_UTC', 'scheduled-2024-01-24-15_24_00_UTC', 'scheduled-2024-01-24-15_25_00_UTC'] 2024-01-24T15:25:15.425 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2024-01-24T15:25:15.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.604+0000 7fab4d570700 1 Processor -- start 2024-01-24T15:25:15.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.604+0000 7fab4d570700 1 -- start start 2024-01-24T15:25:15.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.605+0000 7fab4d570700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab480bf870 0x7fab48068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:25:15.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.605+0000 7fab4d570700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab480643f0 con 0x7fab4813a6f0 2024-01-24T15:25:15.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.605+0000 7fab4d570700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab480bdb30 con 0x7fab48066450 2024-01-24T15:25:15.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.605+0000 7fab4d570700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab480be3d0 con 0x7fab480bf870 2024-01-24T15:25:15.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.605+0000 7fab467fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab480bf870 0x7fab48068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:25:15.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.605+0000 7fab46ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fab4813a6f0 0x7fab480638c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:51290/0 (socket says 172.21.15.29:51290) 2024-01-24T15:25:15.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.605+0000 7fab46ffd700 1 -- 172.21.15.29:0/1410391376 learned_addr learned my addr 172.21.15.29:0/1410391376 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:25:15.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.606+0000 7fab467fc700 1 -- 172.21.15.29:0/1410391376 >> v1:172.21.15.163:6789/0 conn(0x7fab48066450 legacy=0x7fab4815de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:15.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.606+0000 7fab467fc700 1 -- 172.21.15.29:0/1410391376 >> v1:172.21.15.29:6789/0 conn(0x7fab4813a6f0 legacy=0x7fab480638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:15.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.606+0000 7fab467fc700 1 -- 172.21.15.29:0/1410391376 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab48064110 con 0x7fab480bf870 2024-01-24T15:25:15.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.606+0000 7fab45ffb700 1 -- 172.21.15.29:0/1410391376 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074895460 0 0) 0x7fab480643f0 con 0x7fab4813a6f0 2024-01-24T15:25:15.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.606+0000 7fab467fc700 1 --2- 172.21.15.29:0/1410391376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab480bf870 0x7fab48068660 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fab38009760 tx=0x7fab38009060).ready entity=mon.0 client_cookie=94092fc9fe1e91c8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:25:15.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.606+0000 7fab45ffb700 1 -- 172.21.15.29:0/1410391376 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800b580 con 0x7fab480bf870 2024-01-24T15:25:15.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.606+0000 7fab45ffb700 1 -- 172.21.15.29:0/1410391376 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3800b6e0 con 0x7fab480bf870 2024-01-24T15:25:15.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.606+0000 7fab45ffb700 1 -- 172.21.15.29:0/1410391376 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800ba60 con 0x7fab480bf870 2024-01-24T15:25:15.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.607+0000 7fab4d570700 1 -- 172.21.15.29:0/1410391376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab480bf870 msgr2=0x7fab48068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:15.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.607+0000 7fab4d570700 1 --2- 172.21.15.29:0/1410391376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab480bf870 0x7fab48068660 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fab38009760 tx=0x7fab38009060).stop 2024-01-24T15:25:15.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.607+0000 7fab4d570700 1 -- 172.21.15.29:0/1410391376 shutdown_connections 2024-01-24T15:25:15.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.607+0000 7fab4d570700 1 --2- 172.21.15.29:0/1410391376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab480bf870 0x7fab48068660 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:15.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.607+0000 7fab4d570700 1 -- 172.21.15.29:0/1410391376 >> 172.21.15.29:0/1410391376 conn(0x7fab480c8bd0 msgr2=0x7fab480c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:25:15.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.607+0000 7fab4d570700 1 -- 172.21.15.29:0/1410391376 shutdown_connections 2024-01-24T15:25:15.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.607+0000 7fab4d570700 1 -- 172.21.15.29:0/1410391376 wait complete. 2024-01-24T15:25:15.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.608+0000 7fab4d570700 1 Processor -- start 2024-01-24T15:25:15.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.608+0000 7fab4d570700 1 -- start start 2024-01-24T15:25:15.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab4d570700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab48066450 0x7fab480abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:25:15.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab4d570700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab480bf870 0x7fab48162e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:25:15.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab4d570700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab4813a6f0 0x7fab48163350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:25:15.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab4d570700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48125d60 con 0x7fab48066450 2024-01-24T15:25:15.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab4d570700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48125fd0 con 0x7fab480bf870 2024-01-24T15:25:15.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab4d570700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab481262b0 con 0x7fab4813a6f0 2024-01-24T15:25:15.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab46ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab48066450 0x7fab480abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:25:15.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab46ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab48066450 0x7fab480abd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35072/0 (socket says 172.21.15.29:35072) 2024-01-24T15:25:15.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab46ffd700 1 -- 172.21.15.29:0/1294345739 learned_addr learned my addr 172.21.15.29:0/1294345739 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab467fc700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab480bf870 0x7fab48162e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab477fe700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab4813a6f0 0x7fab48163350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab46ffd700 1 -- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab4813a6f0 msgr2=0x7fab48163350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab46ffd700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab4813a6f0 0x7fab48163350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab46ffd700 1 -- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab480bf870 msgr2=0x7fab48162e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab46ffd700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab480bf870 0x7fab48162e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.609+0000 7fab46ffd700 1 -- 172.21.15.29:0/1294345739 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab380092a0 con 0x7fab48066450 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.610+0000 7fab46ffd700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab48066450 0x7fab480abd20 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7fab3c00c490 tx=0x7fab3c007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.610+0000 7fab2effd700 1 -- 172.21.15.29:0/1294345739 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c0072e0 con 0x7fab48066450 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.610+0000 7fab2effd700 1 -- 172.21.15.29:0/1294345739 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c007440 con 0x7fab48066450 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.610+0000 7fab4d570700 1 -- 172.21.15.29:0/1294345739 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab48126590 con 0x7fab48066450 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.610+0000 7fab2effd700 1 -- 172.21.15.29:0/1294345739 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c0057a0 con 0x7fab48066450 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.610+0000 7fab4d570700 1 -- 172.21.15.29:0/1294345739 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab480ac210 con 0x7fab48066450 2024-01-24T15:25:15.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.611+0000 7fab4d570700 1 -- 172.21.15.29:0/1294345739 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab480ac4f0 con 0x7fab48066450 2024-01-24T15:25:15.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.611+0000 7fab2effd700 1 -- 172.21.15.29:0/1294345739 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 80) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fab3c005b90 con 0x7fab48066450 2024-01-24T15:25:15.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.612+0000 7fab2effd700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3405f2c0 0x7fab34061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:25:15.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.612+0000 7fab2effd700 1 -- 172.21.15.29:0/1294345739 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 7502+0+0 (secure 0 0 0) 0x7fab3c07a120 con 0x7fab48066450 2024-01-24T15:25:15.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.612+0000 7fab467fc700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3405f2c0 0x7fab34061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:25:15.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.612+0000 7fab467fc700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3405f2c0 0x7fab34061780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fab480ab8a0 tx=0x7fab38017000).ready entity=mgr.8803 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:25:15.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.616+0000 7fab2effd700 1 -- 172.21.15.29:0/1294345739 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fab3c04ad10 con 0x7fab48066450 2024-01-24T15:25:15.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.822+0000 7fab4d570700 1 -- 172.21.15.29:0/1294345739 --> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab48163c60 con 0x7fab3405f2c0 2024-01-24T15:25:15.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.825+0000 7fab2effd700 1 -- 172.21.15.29:0/1294345739 <== mgr.8803 v2:172.21.15.163:6832/43241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fab48163c60 con 0x7fab3405f2c0 2024-01-24T15:25:15.827 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:25:15.827 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T15:14:29", "first": "2024-01-24T15:15:00", "last": "2024-01-24T15:25:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2024-01-24T15:25:15.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.827+0000 7fab4d570700 1 -- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3405f2c0 msgr2=0x7fab34061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:15.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.827+0000 7fab4d570700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3405f2c0 0x7fab34061780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fab480ab8a0 tx=0x7fab38017000).stop 2024-01-24T15:25:15.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.827+0000 7fab4d570700 1 -- 172.21.15.29:0/1294345739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab48066450 msgr2=0x7fab480abd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:15.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.827+0000 7fab4d570700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab48066450 0x7fab480abd20 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7fab3c00c490 tx=0x7fab3c007e40).stop 2024-01-24T15:25:15.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.827+0000 7fab4d570700 1 -- 172.21.15.29:0/1294345739 shutdown_connections 2024-01-24T15:25:15.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.827+0000 7fab4d570700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab4813a6f0 0x7fab48163350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:15.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.827+0000 7fab4d570700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fab3405f2c0 0x7fab34061780 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:15.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.827+0000 7fab4d570700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab48066450 0x7fab480abd20 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:15.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.827+0000 7fab4d570700 1 --2- 172.21.15.29:0/1294345739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab480bf870 0x7fab48162e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:15.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.827+0000 7fab4d570700 1 -- 172.21.15.29:0/1294345739 >> 172.21.15.29:0/1294345739 conn(0x7fab480c8bd0 msgr2=0x7fab480c7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:25:15.830 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.827+0000 7fab4d570700 1 -- 172.21.15.29:0/1294345739 shutdown_connections 2024-01-24T15:25:15.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.827+0000 7fab4d570700 1 -- 172.21.15.29:0/1294345739 wait complete. 2024-01-24T15:25:15.842 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T15:14:29", "first": "2024-01-24T15:15:00", "last": "2024-01-24T15:25:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2024-01-24T15:25:15.842 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T15:14:29', 'first': '2024-01-24T15:15:00', 'last': '2024-01-24T15:25:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2024-01-24T15:25:15.843 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-01-24T15:25:15.843 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-01-24T15:25:15.843 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-24T15:25:15.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.971+0000 7fc9499e9700 1 Processor -- start 2024-01-24T15:25:15.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.971+0000 7fc9499e9700 1 -- start start 2024-01-24T15:25:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.972+0000 7fc9499e9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc944066c00 0x7fc944155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:25:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.972+0000 7fc9499e9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9441563d0 0x7fc9440bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:25:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.972+0000 7fc9499e9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9440c3910 con 0x7fc944155b00 2024-01-24T15:25:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.972+0000 7fc9499e9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9440c5500 con 0x7fc944066c00 2024-01-24T15:25:15.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.972+0000 7fc9499e9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440c5220 con 0x7fc9441563d0 2024-01-24T15:25:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.972+0000 7fc9437fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9441563d0 0x7fc9440bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:25:15.976 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.972+0000 7fc9427fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc944066c00 0x7fc944155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:25:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.972+0000 7fc9437fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9441563d0 0x7fc9440bdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:53492/0 (socket says 172.21.15.29:53492) 2024-01-24T15:25:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.972+0000 7fc9427fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc944066c00 0x7fc944155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:38008/0 (socket says 172.21.15.29:38008) 2024-01-24T15:25:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.972+0000 7fc9437fe700 1 -- 172.21.15.29:0/142487183 learned_addr learned my addr 172.21.15.29:0/142487183 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:25:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.972+0000 7fc942ffd700 1 --1- 172.21.15.29:0/142487183 >> v1:172.21.15.163:6789/0 conn(0x7fc944155b00 0x7fc9440664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:43792/0 (socket says 172.21.15.29:43792) 2024-01-24T15:25:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.973+0000 7fc9437fe700 1 -- 172.21.15.29:0/142487183 >> v2:172.21.15.163:3300/0 conn(0x7fc944066c00 msgr2=0x7fc944155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:25:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.973+0000 7fc9437fe700 1 --2- 172.21.15.29:0/142487183 >> v2:172.21.15.163:3300/0 conn(0x7fc944066c00 0x7fc944155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.973+0000 7fc9437fe700 1 -- 172.21.15.29:0/142487183 >> v1:172.21.15.163:6789/0 conn(0x7fc944155b00 legacy=0x7fc9440664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.973+0000 7fc9437fe700 1 -- 172.21.15.29:0/142487183 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9440c3350 con 0x7fc9441563d0 2024-01-24T15:25:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.973+0000 7fc941ffb700 1 -- 172.21.15.29:0/142487183 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021321088 0 0) 0x7fc9440c3910 con 0x7fc944155b00 2024-01-24T15:25:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.973+0000 7fc9427fc700 1 --2- 172.21.15.29:0/142487183 >> v2:172.21.15.163:3300/0 conn(0x7fc944066c00 0x7fc944155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:25:15.977 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.973+0000 7fc9437fe700 1 --2- 172.21.15.29:0/142487183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9441563d0 0x7fc9440bdeb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fc93c009f20 tx=0x7fc93c009840).ready entity=mon.2 client_cookie=c6868643b99ab623 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:25:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.973+0000 7fc941ffb700 1 -- 172.21.15.29:0/142487183 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00e070 con 0x7fc9441563d0 2024-01-24T15:25:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.973+0000 7fc941ffb700 1 -- 172.21.15.29:0/142487183 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c00b9d0 con 0x7fc9441563d0 2024-01-24T15:25:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.973+0000 7fc941ffb700 1 -- 172.21.15.29:0/142487183 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00bd50 con 0x7fc9441563d0 2024-01-24T15:25:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.973+0000 7fc9499e9700 1 -- 172.21.15.29:0/142487183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9441563d0 msgr2=0x7fc9440bdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.973+0000 7fc9499e9700 1 --2- 172.21.15.29:0/142487183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9441563d0 0x7fc9440bdeb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fc93c009f20 tx=0x7fc93c009840).stop 2024-01-24T15:25:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.974+0000 7fc9499e9700 1 -- 172.21.15.29:0/142487183 shutdown_connections 2024-01-24T15:25:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.974+0000 7fc9499e9700 1 --2- 172.21.15.29:0/142487183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9441563d0 0x7fc9440bdeb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.974+0000 7fc9499e9700 1 --2- 172.21.15.29:0/142487183 >> v2:172.21.15.163:3300/0 conn(0x7fc944066c00 0x7fc944155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.974+0000 7fc9499e9700 1 -- 172.21.15.29:0/142487183 >> 172.21.15.29:0/142487183 conn(0x7fc9440c8bb0 msgr2=0x7fc9440c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:25:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.974+0000 7fc9499e9700 1 -- 172.21.15.29:0/142487183 shutdown_connections 2024-01-24T15:25:15.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.974+0000 7fc9499e9700 1 -- 172.21.15.29:0/142487183 wait complete. 2024-01-24T15:25:15.981 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.975+0000 7fc9499e9700 1 Processor -- start 2024-01-24T15:25:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.975+0000 7fc9499e9700 1 -- start start 2024-01-24T15:25:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9499e9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc944066c00 0x7fc94414ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:25:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9499e9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc944155b00 0x7fc9440a8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:25:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9499e9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9441563d0 0x7fc9440a8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:25:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9499e9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440a93e0 con 0x7fc9441563d0 2024-01-24T15:25:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9499e9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc944154140 con 0x7fc944066c00 2024-01-24T15:25:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9499e9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc944154420 con 0x7fc944155b00 2024-01-24T15:25:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9437fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9441563d0 0x7fc9440a8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:25:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9437fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9441563d0 0x7fc9440a8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35084/0 (socket says 172.21.15.29:35084) 2024-01-24T15:25:15.982 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9437fe700 1 -- 172.21.15.29:0/2836364117 learned_addr learned my addr 172.21.15.29:0/2836364117 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:25:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc942ffd700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc944066c00 0x7fc94414ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:25:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9427fc700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc944155b00 0x7fc9440a8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:25:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9437fe700 1 -- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc944155b00 msgr2=0x7fc9440a8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9437fe700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc944155b00 0x7fc9440a8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9437fe700 1 -- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc944066c00 msgr2=0x7fc94414ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9437fe700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc944066c00 0x7fc94414ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9437fe700 1 -- 172.21.15.29:0/2836364117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9380092a0 con 0x7fc9441563d0 2024-01-24T15:25:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc942ffd700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc944066c00 0x7fc94414ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:25:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.976+0000 7fc9427fc700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc944155b00 0x7fc9440a8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:25:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.977+0000 7fc9437fe700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9441563d0 0x7fc9440a8ef0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7fc93c00b610 tx=0x7fc93c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:25:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.977+0000 7fc92affd700 1 -- 172.21.15.29:0/2836364117 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00b640 con 0x7fc9441563d0 2024-01-24T15:25:15.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.977+0000 7fc92affd700 1 -- 172.21.15.29:0/2836364117 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c007610 con 0x7fc9441563d0 2024-01-24T15:25:15.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.977+0000 7fc9499e9700 1 -- 172.21.15.29:0/2836364117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93c009ae0 con 0x7fc9441563d0 2024-01-24T15:25:15.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.977+0000 7fc92affd700 1 -- 172.21.15.29:0/2836364117 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0079b0 con 0x7fc9441563d0 2024-01-24T15:25:15.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.977+0000 7fc9499e9700 1 -- 172.21.15.29:0/2836364117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9441549e0 con 0x7fc9441563d0 2024-01-24T15:25:15.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.978+0000 7fc9499e9700 1 -- 172.21.15.29:0/2836364117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9440475e0 con 0x7fc9441563d0 2024-01-24T15:25:15.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.978+0000 7fc92affd700 1 -- 172.21.15.29:0/2836364117 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 80) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fc93c003040 con 0x7fc9441563d0 2024-01-24T15:25:15.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.979+0000 7fc92affd700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc93005f2f0 0x7fc9300617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:25:15.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.979+0000 7fc92affd700 1 -- 172.21.15.29:0/2836364117 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 7502+0+0 (secure 0 0 0) 0x7fc93c07c3d0 con 0x7fc9441563d0 2024-01-24T15:25:15.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.979+0000 7fc942ffd700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc93005f2f0 0x7fc9300617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:25:15.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.980+0000 7fc942ffd700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc93005f2f0 0x7fc9300617b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fc9340011e0 tx=0x7fc934007b10).ready entity=mgr.8803 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:25:15.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:15.983+0000 7fc92affd700 1 -- 172.21.15.29:0/2836364117 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc93c04cfc0 con 0x7fc9441563d0 2024-01-24T15:25:16.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.211+0000 7fc9499e9700 1 -- 172.21.15.29:0/2836364117 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc9440c3630 con 0x7fc9441563d0 2024-01-24T15:25:16.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.214+0000 7fc92affd700 1 -- 172.21.15.29:0/2836364117 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v80) v1 ==== 74+0+149164 (secure 0 0 0) 0x7fc93c023020 con 0x7fc9441563d0 2024-01-24T15:25:16.219 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:25:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.216+0000 7fc9499e9700 1 -- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc93005f2f0 msgr2=0x7fc9300617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.216+0000 7fc9499e9700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc93005f2f0 0x7fc9300617b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fc9340011e0 tx=0x7fc934007b10).stop 2024-01-24T15:25:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.216+0000 7fc9499e9700 1 -- 172.21.15.29:0/2836364117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9441563d0 msgr2=0x7fc9440a8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:25:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.216+0000 7fc9499e9700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9441563d0 0x7fc9440a8ef0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7fc93c00b610 tx=0x7fc93c009840).stop 2024-01-24T15:25:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.216+0000 7fc9499e9700 1 -- 172.21.15.29:0/2836364117 shutdown_connections 2024-01-24T15:25:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.216+0000 7fc9499e9700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:6832/43241,v1:172.21.15.163:6833/43241] conn(0x7fc93005f2f0 0x7fc9300617b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:16.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.216+0000 7fc9499e9700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9441563d0 0x7fc9440a8ef0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:16.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.216+0000 7fc9499e9700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc944066c00 0x7fc94414ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:16.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.216+0000 7fc9499e9700 1 --2- 172.21.15.29:0/2836364117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc944155b00 0x7fc9440a8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:25:16.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.216+0000 7fc9499e9700 1 -- 172.21.15.29:0/2836364117 >> 172.21.15.29:0/2836364117 conn(0x7fc9440c8bb0 msgr2=0x7fc9441465a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:25:16.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.217+0000 7fc9499e9700 1 -- 172.21.15.29:0/2836364117 shutdown_connections 2024-01-24T15:25:16.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:25:16.217+0000 7fc9499e9700 1 -- 172.21.15.29:0/2836364117 wait complete. 2024-01-24T15:25:16.232 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":80,"active_gid":8803,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":43241},{"type":"v1","addr":"172.21.15.163:6833","nonce":43241}]},"active_addr":"172.21.15.163:6833/43241","active_change":"2024-01-24T15:14:28.971640+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":7808,"name":"y","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":194,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":3305307869}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":2002665960}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":4000114899}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":3544734349}]}]} 2024-01-24T15:25:16.235 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: x 2024-01-24T15:25:16.235 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-24T15:25:16.235 INFO:tasks.ceph.mgr.x:Stopping old one... 2024-01-24T15:25:16.236 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-01-24T15:25:16.236 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:25:16.265 INFO:tasks.ceph.mgr.x:Stopped 2024-01-24T15:25:16.265 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-01-24T15:25:16.309 INFO:tasks.ceph.mgr.x:Started 2024-01-24T15:25:16.500 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:16.499+0000 7f8369692680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:25:16.564 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:16.562+0000 7f8369692680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:25:16.841 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:16.839+0000 7f8369692680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:25:17.463 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:17.462+0000 7f8369692680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:25:17.536 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:17.534+0000 7f8369692680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:25:17.726 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:17.724+0000 7f8369692680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:25:18.150 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:18.148+0000 7f8369692680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:25:18.254 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:18.253+0000 7f8369692680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:25:18.410 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:18.408+0000 7f8369692680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:25:18.485 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:18.484+0000 7f8369692680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:25:18.553 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:18.552+0000 7f8369692680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:25:18.812 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:18.811+0000 7f8369692680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:25:18.997 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:18.995+0000 7f8369692680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:25:19.657 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:19.655+0000 7f8369692680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:25:19.729 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:19.727+0000 7f8369692680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:25:19.806 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:19.804+0000 7f8369692680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:25:19.954 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:19.953+0000 7f8369692680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:25:20.027 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.025+0000 7f8369692680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:25:20.143 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.142+0000 7f8369692680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:25:20.253 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.252+0000 7f8369692680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:25:20.548 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.547+0000 7f8369692680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:25:20.621 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.620+0000 7f8369692680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:25:20.714 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.712+0000 7f82dcc07700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:25:20.714 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.712+0000 7f82dcc07700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:25:20.714 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.712+0000 7f82dcc07700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:25:20.714 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.712+0000 7f82dcc07700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:25:20.714 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.712+0000 7f82dcc07700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:25:20.715 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.713+0000 7f82e040e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:25:20.715 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.713+0000 7f82e040e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:25:20.715 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.713+0000 7f82e040e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:25:20.715 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.713+0000 7f82e040e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:25:20.715 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:25:20.713+0000 7f82e040e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:25:41.682 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:25:41.688 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:26:00.706 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:26:00.704+0000 7f82e8c1f700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:26:00.706 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:26:00.704+0000 7f82e8c1f700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:26:00.707 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:26:00.704+0000 7f82e8c1f700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:26:00.707 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:26:00.704+0000 7f82e8c1f700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:26:00.707 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:26:00.704+0000 7f82e8c1f700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:26:11.751 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:26:11.754 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:26:41.823 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:26:41.826 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:27:11.880 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:27:11.884 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:27:41.970 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:27:41.974 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:28:12.053 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:28:12.056 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:28:42.128 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:28:42.132 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:29:12.213 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:29:12.217 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:29:42.293 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:29:42.297 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:30:12.365 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:30:12.368 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:30:16.310 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/test_restart 1M 2024-01-24T15:30:16.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.451+0000 7f380c496700 1 Processor -- start 2024-01-24T15:30:16.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.451+0000 7f380c496700 1 -- start start 2024-01-24T15:30:16.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.452+0000 7f380c496700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f380414e920 0x7f38041586d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:16.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.452+0000 7f380c496700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3804154d30 0x7f3804066100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:16.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.452+0000 7f380c496700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3804067150 con 0x7f380414e540 2024-01-24T15:30:16.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.452+0000 7f380c496700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3804158240 con 0x7f380414e920 2024-01-24T15:30:16.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.452+0000 7f380c496700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3804155200 con 0x7f3804154d30 2024-01-24T15:30:16.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.452+0000 7f3809a31700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f380414e920 0x7f38041586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:16.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.452+0000 7f3809a31700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f380414e920 0x7f38041586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39880/0 (socket says 172.21.15.29:39880) 2024-01-24T15:30:16.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.452+0000 7f3809a31700 1 -- 172.21.15.29:0/1845515773 learned_addr learned my addr 172.21.15.29:0/1845515773 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:16.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.452+0000 7f380aa33700 1 --2- 172.21.15.29:0/1845515773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3804154d30 0x7f3804066100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:16.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.453+0000 7f3809a31700 1 -- 172.21.15.29:0/1845515773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3804154d30 msgr2=0x7f3804066100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:16.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.453+0000 7f3809a31700 1 --2- 172.21.15.29:0/1845515773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3804154d30 0x7f3804066100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.453+0000 7f3809a31700 1 -- 172.21.15.29:0/1845515773 >> v1:172.21.15.163:6789/0 conn(0x7f380414e540 legacy=0x7f3804154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:16.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.453+0000 7f3809a31700 1 -- 172.21.15.29:0/1845515773 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38041554e0 con 0x7f380414e920 2024-01-24T15:30:16.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.453+0000 7f3809a31700 1 --2- 172.21.15.29:0/1845515773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380414e920 0x7f38041586d0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f3800006fd0 tx=0x7f38000090f0).ready entity=mon.0 client_cookie=7e8bbee2971520ab server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:16.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.453+0000 7f3809230700 1 -- 172.21.15.29:0/1845515773 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380000b580 con 0x7f380414e920 2024-01-24T15:30:16.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.453+0000 7f3809230700 1 -- 172.21.15.29:0/1845515773 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380000b6e0 con 0x7f380414e920 2024-01-24T15:30:16.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.453+0000 7f3809230700 1 -- 172.21.15.29:0/1845515773 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380000ba60 con 0x7f380414e920 2024-01-24T15:30:16.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.453+0000 7f380c496700 1 -- 172.21.15.29:0/1845515773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380414e920 msgr2=0x7f38041586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:16.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.453+0000 7f380c496700 1 --2- 172.21.15.29:0/1845515773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380414e920 0x7f38041586d0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f3800006fd0 tx=0x7f38000090f0).stop 2024-01-24T15:30:16.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.454+0000 7f380c496700 1 -- 172.21.15.29:0/1845515773 shutdown_connections 2024-01-24T15:30:16.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.454+0000 7f380c496700 1 --2- 172.21.15.29:0/1845515773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3804154d30 0x7f3804066100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.454+0000 7f380c496700 1 --2- 172.21.15.29:0/1845515773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380414e920 0x7f38041586d0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.454+0000 7f380c496700 1 -- 172.21.15.29:0/1845515773 >> 172.21.15.29:0/1845515773 conn(0x7f38040c3d20 msgr2=0x7f38040c4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:16.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.454+0000 7f380c496700 1 -- 172.21.15.29:0/1845515773 shutdown_connections 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.454+0000 7f380c496700 1 -- 172.21.15.29:0/1845515773 wait complete. 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.455+0000 7f380c496700 1 Processor -- start 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.455+0000 7f380c496700 1 -- start start 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380c496700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380414e540 0x7f38040a7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380c496700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f380414e920 0x7f38040a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380c496700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3804154d30 0x7f380414ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380c496700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3804164510 con 0x7f380414e540 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380c496700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38040a7f80 con 0x7f3804154d30 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380c496700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38040a4a20 con 0x7f380414e920 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380a232700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380414e540 0x7f38040a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380a232700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380414e540 0x7f38040a7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39896/0 (socket says 172.21.15.29:39896) 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380a232700 1 -- 172.21.15.29:0/2065927076 learned_addr learned my addr 172.21.15.29:0/2065927076 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380aa33700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3804154d30 0x7f380414ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:16.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f3809a31700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f380414e920 0x7f38040a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:16.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380a232700 1 -- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f380414e920 msgr2=0x7f38040a6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:16.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380a232700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f380414e920 0x7f38040a6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380a232700 1 -- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3804154d30 msgr2=0x7f380414ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:16.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380a232700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3804154d30 0x7f380414ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380a232700 1 -- 172.21.15.29:0/2065927076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3800009370 con 0x7f380414e540 2024-01-24T15:30:16.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f3809a31700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f380414e920 0x7f38040a6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:16.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.456+0000 7f380a232700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380414e540 0x7f38040a7660 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f37f400c440 tx=0x7f37f4007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:16.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.457+0000 7f37fa7fc700 1 -- 172.21.15.29:0/2065927076 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f400ce40 con 0x7f380414e540 2024-01-24T15:30:16.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.457+0000 7f37fa7fc700 1 -- 172.21.15.29:0/2065927076 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f4007020 con 0x7f380414e540 2024-01-24T15:30:16.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.457+0000 7f380c496700 1 -- 172.21.15.29:0/2065927076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38040a4d00 con 0x7f380414e540 2024-01-24T15:30:16.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.457+0000 7f37fa7fc700 1 -- 172.21.15.29:0/2065927076 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f40073c0 con 0x7f380414e540 2024-01-24T15:30:16.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.457+0000 7f380c496700 1 -- 172.21.15.29:0/2065927076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38040a7b50 con 0x7f380414e540 2024-01-24T15:30:16.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.458+0000 7f380c496700 1 -- 172.21.15.29:0/2065927076 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38040a4880 con 0x7f380414e540 2024-01-24T15:30:16.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.458+0000 7f37fa7fc700 1 -- 172.21.15.29:0/2065927076 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 85) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f37f40068b0 con 0x7f380414e540 2024-01-24T15:30:16.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.459+0000 7f37fa7fc700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f37e805f2c0 0x7f37e8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:16.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.459+0000 7f37fa7fc700 1 -- 172.21.15.29:0/2065927076 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7674+0+0 (secure 0 0 0) 0x7f37f407a430 con 0x7f380414e540 2024-01-24T15:30:16.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.459+0000 7f3809a31700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f37e805f2c0 0x7f37e8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:16.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.459+0000 7f3809a31700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f37e805f2c0 0x7f37e8061780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f380414ed00 tx=0x7f3800017000).ready entity=mgr.8881 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:16.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.463+0000 7f37fa7fc700 1 -- 172.21.15.29:0/2065927076 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f38040a4880 con 0x7f380414e540 2024-01-24T15:30:16.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.676+0000 7f380c496700 1 -- 172.21.15.29:0/2065927076 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f38040a4880 con 0x7f37e805f2c0 2024-01-24T15:30:16.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.682+0000 7f37fa7fc700 1 -- 172.21.15.29:0/2065927076 <== mgr.8881 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f38040a4880 con 0x7f37e805f2c0 2024-01-24T15:30:16.684 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2024-01-24T15:30:16.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.684+0000 7f380c496700 1 -- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f37e805f2c0 msgr2=0x7f37e8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:16.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.684+0000 7f380c496700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f37e805f2c0 0x7f37e8061780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f380414ed00 tx=0x7f3800017000).stop 2024-01-24T15:30:16.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.684+0000 7f380c496700 1 -- 172.21.15.29:0/2065927076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380414e540 msgr2=0x7f38040a7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.684+0000 7f380c496700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380414e540 0x7f38040a7660 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f37f400c440 tx=0x7f37f4007dd0).stop 2024-01-24T15:30:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.684+0000 7f380c496700 1 -- 172.21.15.29:0/2065927076 shutdown_connections 2024-01-24T15:30:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.684+0000 7f380c496700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3804154d30 0x7f380414ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.684+0000 7f380c496700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380414e540 0x7f38040a7660 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.684+0000 7f380c496700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f37e805f2c0 0x7f37e8061780 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.684+0000 7f380c496700 1 --2- 172.21.15.29:0/2065927076 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f380414e920 0x7f38040a6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.684+0000 7f380c496700 1 -- 172.21.15.29:0/2065927076 >> 172.21.15.29:0/2065927076 conn(0x7f38040c3d20 msgr2=0x7f3804069340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.684+0000 7f380c496700 1 -- 172.21.15.29:0/2065927076 shutdown_connections 2024-01-24T15:30:16.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.684+0000 7f380c496700 1 -- 172.21.15.29:0/2065927076 wait complete. 2024-01-24T15:30:16.699 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2024-01-24T15:30:16.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:16.699 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2024-01-24T15:30:16.719 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_15_00_UTC 2024-01-24T15:30:16.719 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_16_00_UTC 2024-01-24T15:30:16.719 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_17_00_UTC 2024-01-24T15:30:16.719 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_18_00_UTC 2024-01-24T15:30:16.719 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_19_00_UTC 2024-01-24T15:30:16.720 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_20_00_UTC 2024-01-24T15:30:16.720 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_21_00_UTC 2024-01-24T15:30:16.720 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_22_00_UTC 2024-01-24T15:30:16.720 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_23_00_UTC 2024-01-24T15:30:16.720 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_24_00_UTC 2024-01-24T15:30:16.720 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_25_00_UTC 2024-01-24T15:30:16.720 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_26_00_UTC 2024-01-24T15:30:16.720 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_27_00_UTC 2024-01-24T15:30:16.720 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_28_00_UTC 2024-01-24T15:30:16.720 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_29_00_UTC 2024-01-24T15:30:16.720 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_30_00_UTC 2024-01-24T15:30:16.721 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-01-24-15_15_00_UTC', 'scheduled-2024-01-24-15_16_00_UTC', 'scheduled-2024-01-24-15_17_00_UTC', 'scheduled-2024-01-24-15_18_00_UTC', 'scheduled-2024-01-24-15_19_00_UTC', 'scheduled-2024-01-24-15_20_00_UTC', 'scheduled-2024-01-24-15_21_00_UTC', 'scheduled-2024-01-24-15_22_00_UTC', 'scheduled-2024-01-24-15_23_00_UTC', 'scheduled-2024-01-24-15_24_00_UTC', 'scheduled-2024-01-24-15_25_00_UTC', 'scheduled-2024-01-24-15_26_00_UTC', 'scheduled-2024-01-24-15_27_00_UTC', 'scheduled-2024-01-24-15_28_00_UTC', 'scheduled-2024-01-24-15_29_00_UTC', 'scheduled-2024-01-24-15_30_00_UTC'] 2024-01-24T15:30:16.721 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2024-01-24T15:30:16.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.897+0000 7fe3f5890700 1 Processor -- start 2024-01-24T15:30:16.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.897+0000 7fe3f5890700 1 -- start start 2024-01-24T15:30:16.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.897+0000 7fe3f5890700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe3f00bc300 0x7fe3f00c3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:16.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3f5890700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe3f00bc6e0 0x7fe3f0154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:16.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3f5890700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f00bcb60 con 0x7fe3f0158470 2024-01-24T15:30:16.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3f5890700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f0154d80 con 0x7fe3f00bc6e0 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3f5890700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f0157f70 con 0x7fe3f00bc300 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3ee7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe3f00bc6e0 0x7fe3f0154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3ee7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe3f00bc6e0 0x7fe3f0154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39900/0 (socket says 172.21.15.29:39900) 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3ee7fc700 1 -- 172.21.15.29:0/1083722893 learned_addr learned my addr 172.21.15.29:0/1083722893 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3eeffd700 1 --2- 172.21.15.29:0/1083722893 >> v2:172.21.15.163:3300/0 conn(0x7fe3f00bc300 0x7fe3f00c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3ee7fc700 1 -- 172.21.15.29:0/1083722893 >> v2:172.21.15.163:3300/0 conn(0x7fe3f00bc300 msgr2=0x7fe3f00c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3ee7fc700 1 --2- 172.21.15.29:0/1083722893 >> v2:172.21.15.163:3300/0 conn(0x7fe3f00bc300 0x7fe3f00c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3ee7fc700 1 -- 172.21.15.29:0/1083722893 >> v1:172.21.15.163:6789/0 conn(0x7fe3f0158470 legacy=0x7fe3f0064550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3ee7fc700 1 -- 172.21.15.29:0/1083722893 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f0155010 con 0x7fe3f00bc6e0 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3ee7fc700 1 --2- 172.21.15.29:0/1083722893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f00bc6e0 0x7fe3f0154890 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7fe3f00c6d10 tx=0x7fe3e00090f0).ready entity=mon.0 client_cookie=e2304bee18d476f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3edffb700 1 -- 172.21.15.29:0/1083722893 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000b5a0 con 0x7fe3f00bc6e0 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3edffb700 1 -- 172.21.15.29:0/1083722893 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e000b700 con 0x7fe3f00bc6e0 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.898+0000 7fe3edffb700 1 -- 172.21.15.29:0/1083722893 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000ba80 con 0x7fe3f00bc6e0 2024-01-24T15:30:16.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.899+0000 7fe3f5890700 1 -- 172.21.15.29:0/1083722893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f00bc6e0 msgr2=0x7fe3f0154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:16.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.899+0000 7fe3f5890700 1 --2- 172.21.15.29:0/1083722893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f00bc6e0 0x7fe3f0154890 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7fe3f00c6d10 tx=0x7fe3e00090f0).stop 2024-01-24T15:30:16.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.899+0000 7fe3f5890700 1 -- 172.21.15.29:0/1083722893 shutdown_connections 2024-01-24T15:30:16.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.899+0000 7fe3f5890700 1 --2- 172.21.15.29:0/1083722893 >> v2:172.21.15.163:3300/0 conn(0x7fe3f00bc300 0x7fe3f00c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.899+0000 7fe3f5890700 1 --2- 172.21.15.29:0/1083722893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f00bc6e0 0x7fe3f0154890 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.899+0000 7fe3f5890700 1 -- 172.21.15.29:0/1083722893 >> 172.21.15.29:0/1083722893 conn(0x7fe3f0063280 msgr2=0x7fe3f00c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:16.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.899+0000 7fe3f5890700 1 -- 172.21.15.29:0/1083722893 shutdown_connections 2024-01-24T15:30:16.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.899+0000 7fe3f5890700 1 -- 172.21.15.29:0/1083722893 wait complete. 2024-01-24T15:30:16.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.900+0000 7fe3f5890700 1 Processor -- start 2024-01-24T15:30:16.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.901+0000 7fe3f5890700 1 -- start start 2024-01-24T15:30:16.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.901+0000 7fe3f5890700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f00bc300 0x7fe3f0156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:16.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.901+0000 7fe3f5890700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3f00bc6e0 0x7fe3f00a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:16.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.901+0000 7fe3f5890700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f0158470 0x7fe3f01427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:16.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.901+0000 7fe3f5890700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f00a6e20 con 0x7fe3f00bc300 2024-01-24T15:30:16.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.901+0000 7fe3f5890700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f00a7100 con 0x7fe3f0158470 2024-01-24T15:30:16.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.901+0000 7fe3f5890700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f00a73e0 con 0x7fe3f00bc6e0 2024-01-24T15:30:16.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.901+0000 7fe3eeffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f00bc300 0x7fe3f0156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:16.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.901+0000 7fe3eeffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f00bc300 0x7fe3f0156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39916/0 (socket says 172.21.15.29:39916) 2024-01-24T15:30:16.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.901+0000 7fe3eeffd700 1 -- 172.21.15.29:0/3153757028 learned_addr learned my addr 172.21.15.29:0/3153757028 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3ef7fe700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f0158470 0x7fe3f01427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3ee7fc700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3f00bc6e0 0x7fe3f00a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3eeffd700 1 -- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3f00bc6e0 msgr2=0x7fe3f00a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3eeffd700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3f00bc6e0 0x7fe3f00a6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3eeffd700 1 -- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f0158470 msgr2=0x7fe3f01427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3eeffd700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f0158470 0x7fe3f01427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3eeffd700 1 -- 172.21.15.29:0/3153757028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e0009370 con 0x7fe3f00bc300 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3ef7fe700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f0158470 0x7fe3f01427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3eeffd700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f00bc300 0x7fe3f0156d80 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7fe3d800d890 tx=0x7fe3d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3e6ffd700 1 -- 172.21.15.29:0/3153757028 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80125f0 con 0x7fe3f00bc300 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3e6ffd700 1 -- 172.21.15.29:0/3153757028 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d8012750 con 0x7fe3f00bc300 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3f5890700 1 -- 172.21.15.29:0/3153757028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f00a76c0 con 0x7fe3f00bc300 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3e6ffd700 1 -- 172.21.15.29:0/3153757028 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8012ad0 con 0x7fe3f00bc300 2024-01-24T15:30:16.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.902+0000 7fe3f5890700 1 -- 172.21.15.29:0/3153757028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f0159310 con 0x7fe3f00bc300 2024-01-24T15:30:16.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.903+0000 7fe3f5890700 1 -- 172.21.15.29:0/3153757028 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f00475e0 con 0x7fe3f00bc300 2024-01-24T15:30:16.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.904+0000 7fe3e6ffd700 1 -- 172.21.15.29:0/3153757028 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 85) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fe3d8013480 con 0x7fe3f00bc300 2024-01-24T15:30:16.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.904+0000 7fe3e6ffd700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe3d005f2c0 0x7fe3d0061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:16.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.904+0000 7fe3e6ffd700 1 -- 172.21.15.29:0/3153757028 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7674+0+0 (secure 0 0 0) 0x7fe3d807a850 con 0x7fe3f00bc300 2024-01-24T15:30:16.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.904+0000 7fe3ee7fc700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe3d005f2c0 0x7fe3d0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:16.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.905+0000 7fe3ee7fc700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe3d005f2c0 0x7fe3d0061780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fe3f01504c0 tx=0x7fe3e0015000).ready entity=mgr.8881 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:16.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:16.909+0000 7fe3e6ffd700 1 -- 172.21.15.29:0/3153757028 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe3d804b330 con 0x7fe3f00bc300 2024-01-24T15:30:17.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.116+0000 7fe3f5890700 1 -- 172.21.15.29:0/3153757028 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3f00c28b0 con 0x7fe3d005f2c0 2024-01-24T15:30:17.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.119+0000 7fe3e6ffd700 1 -- 172.21.15.29:0/3153757028 <== mgr.8881 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7fe3f00c28b0 con 0x7fe3d005f2c0 2024-01-24T15:30:17.121 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:17.121 INFO:teuthology.orchestra.run.smithi029.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T15:14:29", "first": "2024-01-24T15:15:00", "last": "2024-01-24T15:30:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2024-01-24T15:30:17.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.121+0000 7fe3f5890700 1 -- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe3d005f2c0 msgr2=0x7fe3d0061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:17.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.121+0000 7fe3f5890700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe3d005f2c0 0x7fe3d0061780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fe3f01504c0 tx=0x7fe3e0015000).stop 2024-01-24T15:30:17.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.121+0000 7fe3f5890700 1 -- 172.21.15.29:0/3153757028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f00bc300 msgr2=0x7fe3f0156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:17.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.121+0000 7fe3f5890700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f00bc300 0x7fe3f0156d80 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7fe3d800d890 tx=0x7fe3d8008040).stop 2024-01-24T15:30:17.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.121+0000 7fe3f5890700 1 -- 172.21.15.29:0/3153757028 shutdown_connections 2024-01-24T15:30:17.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.121+0000 7fe3f5890700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f0158470 0x7fe3f01427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.121+0000 7fe3f5890700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3f00bc300 0x7fe3f0156d80 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.121+0000 7fe3f5890700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe3d005f2c0 0x7fe3d0061780 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.121+0000 7fe3f5890700 1 --2- 172.21.15.29:0/3153757028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3f00bc6e0 0x7fe3f00a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.121+0000 7fe3f5890700 1 -- 172.21.15.29:0/3153757028 >> 172.21.15.29:0/3153757028 conn(0x7fe3f0063280 msgr2=0x7fe3f01513d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:17.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.121+0000 7fe3f5890700 1 -- 172.21.15.29:0/3153757028 shutdown_connections 2024-01-24T15:30:17.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.121+0000 7fe3f5890700 1 -- 172.21.15.29:0/3153757028 wait complete. 2024-01-24T15:30:17.137 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2024-01-24T00:00:00", "created": "2024-01-24T15:14:29", "first": "2024-01-24T15:15:00", "last": "2024-01-24T15:30:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2024-01-24T15:30:17.138 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2024-01-24T00:00:00', 'created': '2024-01-24T15:14:29', 'first': '2024-01-24T15:15:00', 'last': '2024-01-24T15:30:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2024-01-24T15:30:17.138 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2024-01-24T15:30:17.138 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2024-01-24T15:30:17.138 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/test_restart 1M 2024-01-24T15:30:17.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.272+0000 7f407be92700 1 Processor -- start 2024-01-24T15:30:17.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.272+0000 7f407be92700 1 -- start start 2024-01-24T15:30:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.273+0000 7f407be92700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f407414b0a0 0x7f407414b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.273+0000 7f407be92700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40740c3660 0x7f40740c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.273+0000 7f407be92700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40741557e0 con 0x7f40740c48e0 2024-01-24T15:30:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.273+0000 7f407be92700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40740c28a0 con 0x7f407414b0a0 2024-01-24T15:30:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.273+0000 7f407be92700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40740c2b80 con 0x7f40740c3660 2024-01-24T15:30:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.273+0000 7f4079c2e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f407414b0a0 0x7f407414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.273+0000 7f407a42f700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f40740c48e0 0x7f4074154a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38006/0 (socket says 172.21.15.29:38006) 2024-01-24T15:30:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.273+0000 7f407a42f700 1 -- 172.21.15.29:0/163129200 learned_addr learned my addr 172.21.15.29:0/163129200 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.273+0000 7f407942d700 1 --2- 172.21.15.29:0/163129200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40740c3660 0x7f40740c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.274+0000 7f4079c2e700 1 -- 172.21.15.29:0/163129200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40740c3660 msgr2=0x7f40740c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.274+0000 7f4079c2e700 1 --2- 172.21.15.29:0/163129200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40740c3660 0x7f40740c43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.274+0000 7f4079c2e700 1 -- 172.21.15.29:0/163129200 >> v1:172.21.15.29:6789/0 conn(0x7f40740c48e0 legacy=0x7f4074154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.274+0000 7f4078c2c700 1 -- 172.21.15.29:0/163129200 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974417704 0 0) 0x7f40741557e0 con 0x7f40740c48e0 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.274+0000 7f4079c2e700 1 -- 172.21.15.29:0/163129200 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40740c2e60 con 0x7f407414b0a0 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.274+0000 7f4079c2e700 1 --2- 172.21.15.29:0/163129200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f407414b0a0 0x7f407414b480 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f4070009760 tx=0x7f4070009060).ready entity=mon.0 client_cookie=d0712141a4bb87bb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.274+0000 7f4078c2c700 1 -- 172.21.15.29:0/163129200 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407000b580 con 0x7f407414b0a0 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.274+0000 7f4078c2c700 1 -- 172.21.15.29:0/163129200 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407000b6e0 con 0x7f407414b0a0 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.274+0000 7f4078c2c700 1 -- 172.21.15.29:0/163129200 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407000ba60 con 0x7f407414b0a0 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.274+0000 7f407be92700 1 -- 172.21.15.29:0/163129200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f407414b0a0 msgr2=0x7f407414b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.274+0000 7f407be92700 1 --2- 172.21.15.29:0/163129200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f407414b0a0 0x7f407414b480 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f4070009760 tx=0x7f4070009060).stop 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.275+0000 7f407be92700 1 -- 172.21.15.29:0/163129200 shutdown_connections 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.275+0000 7f407be92700 1 --2- 172.21.15.29:0/163129200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f407414b0a0 0x7f407414b480 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.275+0000 7f407be92700 1 --2- 172.21.15.29:0/163129200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40740c3660 0x7f40740c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.275+0000 7f407be92700 1 -- 172.21.15.29:0/163129200 >> 172.21.15.29:0/163129200 conn(0x7f4074067b20 msgr2=0x7f40740638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.275+0000 7f407be92700 1 -- 172.21.15.29:0/163129200 shutdown_connections 2024-01-24T15:30:17.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.275+0000 7f407be92700 1 -- 172.21.15.29:0/163129200 wait complete. 2024-01-24T15:30:17.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.276+0000 7f407be92700 1 Processor -- start 2024-01-24T15:30:17.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.276+0000 7f407be92700 1 -- start start 2024-01-24T15:30:17.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f407be92700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40740c3660 0x7f40740a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:17.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f407be92700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40740c48e0 0x7f4074167c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:17.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f407be92700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f407414b0a0 0x7f4074168100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f407be92700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40740bf980 con 0x7f40740c3660 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f407be92700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40740a5460 con 0x7f40740c48e0 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f407be92700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40740a58f0 con 0x7f407414b0a0 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f4079c2e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40740c3660 0x7f40740a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f4079c2e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40740c3660 0x7f40740a9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39934/0 (socket says 172.21.15.29:39934) 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f4079c2e700 1 -- 172.21.15.29:0/3793564039 learned_addr learned my addr 172.21.15.29:0/3793564039 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f407942d700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40740c48e0 0x7f4074167c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f407a42f700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f407414b0a0 0x7f4074168100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f4079c2e700 1 -- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f407414b0a0 msgr2=0x7f4074168100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f4079c2e700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f407414b0a0 0x7f4074168100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f4079c2e700 1 -- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40740c48e0 msgr2=0x7f4074167c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f4079c2e700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40740c48e0 0x7f4074167c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f4079c2e700 1 -- 172.21.15.29:0/3793564039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40700092a0 con 0x7f40740c3660 2024-01-24T15:30:17.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f407942d700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40740c48e0 0x7f4074167c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:17.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f407a42f700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f407414b0a0 0x7f4074168100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:17.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.277+0000 7f4079c2e700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40740c3660 0x7f40740a9f40 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f4070009eb0 tx=0x7f4070017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:17.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.278+0000 7f4069ffb700 1 -- 172.21.15.29:0/3793564039 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407000b850 con 0x7f40740c3660 2024-01-24T15:30:17.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.278+0000 7f4069ffb700 1 -- 172.21.15.29:0/3793564039 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407000b9b0 con 0x7f40740c3660 2024-01-24T15:30:17.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.278+0000 7f407be92700 1 -- 172.21.15.29:0/3793564039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40740a4890 con 0x7f40740c3660 2024-01-24T15:30:17.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.278+0000 7f4069ffb700 1 -- 172.21.15.29:0/3793564039 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407001b660 con 0x7f40740c3660 2024-01-24T15:30:17.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.278+0000 7f407be92700 1 -- 172.21.15.29:0/3793564039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40740a4e10 con 0x7f40740c3660 2024-01-24T15:30:17.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.279+0000 7f407be92700 1 -- 172.21.15.29:0/3793564039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40740a5740 con 0x7f40740c3660 2024-01-24T15:30:17.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.279+0000 7f4069ffb700 1 -- 172.21.15.29:0/3793564039 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 85) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f407001ba90 con 0x7f40740c3660 2024-01-24T15:30:17.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.280+0000 7f4069ffb700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f405805f2f0 0x7f40580617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:17.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.280+0000 7f4069ffb700 1 -- 172.21.15.29:0/3793564039 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7674+0+0 (secure 0 0 0) 0x7f407007c510 con 0x7f40740c3660 2024-01-24T15:30:17.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.280+0000 7f407942d700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f405805f2f0 0x7f40580617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:17.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.281+0000 7f407942d700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f405805f2f0 0x7f40580617b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f407414b6f0 tx=0x7f4064008040).ready entity=mgr.8881 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:17.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.284+0000 7f4069ffb700 1 -- 172.21.15.29:0/3793564039 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f407004d050 con 0x7f40740c3660 2024-01-24T15:30:17.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.493+0000 7f407be92700 1 -- 172.21.15.29:0/3793564039 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f40740475e0 con 0x7f405805f2f0 2024-01-24T15:30:17.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.498+0000 7f4069ffb700 1 -- 172.21.15.29:0/3793564039 <== mgr.8881 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f40740475e0 con 0x7f405805f2f0 2024-01-24T15:30:17.500 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path /snap_test_dir1/test_restart 2024-01-24T15:30:17.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.500+0000 7f407be92700 1 -- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f405805f2f0 msgr2=0x7f40580617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:17.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.500+0000 7f407be92700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f405805f2f0 0x7f40580617b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f407414b6f0 tx=0x7f4064008040).stop 2024-01-24T15:30:17.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.500+0000 7f407be92700 1 -- 172.21.15.29:0/3793564039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40740c3660 msgr2=0x7f40740a9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:17.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.500+0000 7f407be92700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40740c3660 0x7f40740a9f40 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f4070009eb0 tx=0x7f4070017000).stop 2024-01-24T15:30:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.500+0000 7f407be92700 1 -- 172.21.15.29:0/3793564039 shutdown_connections 2024-01-24T15:30:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.500+0000 7f407be92700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f405805f2f0 0x7f40580617b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.500+0000 7f407be92700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f407414b0a0 0x7f4074168100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.500+0000 7f407be92700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40740c3660 0x7f40740a9f40 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.500+0000 7f407be92700 1 --2- 172.21.15.29:0/3793564039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40740c48e0 0x7f4074167c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.500+0000 7f407be92700 1 -- 172.21.15.29:0/3793564039 >> 172.21.15.29:0/3793564039 conn(0x7f4074067b20 msgr2=0x7f40740c5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.500+0000 7f407be92700 1 -- 172.21.15.29:0/3793564039 shutdown_connections 2024-01-24T15:30:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:17.500+0000 7f407be92700 1 -- 172.21.15.29:0/3793564039 wait complete. 2024-01-24T15:30:17.515 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2024-01-24T15:30:17.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:17.515 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2024-01-24T15:30:17.535 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_15_00_UTC 2024-01-24T15:30:17.536 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_16_00_UTC 2024-01-24T15:30:17.536 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_17_00_UTC 2024-01-24T15:30:17.536 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_18_00_UTC 2024-01-24T15:30:17.536 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_19_00_UTC 2024-01-24T15:30:17.536 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_20_00_UTC 2024-01-24T15:30:17.536 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_21_00_UTC 2024-01-24T15:30:17.536 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_22_00_UTC 2024-01-24T15:30:17.536 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_23_00_UTC 2024-01-24T15:30:17.536 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_24_00_UTC 2024-01-24T15:30:17.537 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_25_00_UTC 2024-01-24T15:30:17.537 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_26_00_UTC 2024-01-24T15:30:17.537 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_27_00_UTC 2024-01-24T15:30:17.537 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_28_00_UTC 2024-01-24T15:30:17.537 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_29_00_UTC 2024-01-24T15:30:17.537 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_30_00_UTC 2024-01-24T15:30:17.537 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_15_00_UTC 2024-01-24T15:30:17.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:17.538 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_15_00_UTC) 2024-01-24T15:30:17.604 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_16_00_UTC 2024-01-24T15:30:17.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:17.604 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_16_00_UTC) 2024-01-24T15:30:17.675 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_17_00_UTC 2024-01-24T15:30:17.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:17.675 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_17_00_UTC) 2024-01-24T15:30:17.744 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_18_00_UTC 2024-01-24T15:30:17.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:17.744 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_18_00_UTC) 2024-01-24T15:30:17.813 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_19_00_UTC 2024-01-24T15:30:17.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:17.813 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_19_00_UTC) 2024-01-24T15:30:17.882 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_20_00_UTC 2024-01-24T15:30:17.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:17.883 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_20_00_UTC) 2024-01-24T15:30:17.951 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_21_00_UTC 2024-01-24T15:30:17.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:17.952 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_21_00_UTC) 2024-01-24T15:30:18.020 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_22_00_UTC 2024-01-24T15:30:18.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:18.021 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_22_00_UTC) 2024-01-24T15:30:18.090 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_23_00_UTC 2024-01-24T15:30:18.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:18.090 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_23_00_UTC) 2024-01-24T15:30:18.159 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_24_00_UTC 2024-01-24T15:30:18.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:18.159 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_24_00_UTC) 2024-01-24T15:30:18.226 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_25_00_UTC 2024-01-24T15:30:18.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:18.227 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_25_00_UTC) 2024-01-24T15:30:18.296 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_26_00_UTC 2024-01-24T15:30:18.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:18.296 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_26_00_UTC) 2024-01-24T15:30:18.365 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_27_00_UTC 2024-01-24T15:30:18.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:18.365 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_27_00_UTC) 2024-01-24T15:30:18.433 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_28_00_UTC 2024-01-24T15:30:18.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:18.433 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_28_00_UTC) 2024-01-24T15:30:18.503 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_29_00_UTC 2024-01-24T15:30:18.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:18.504 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_29_00_UTC) 2024-01-24T15:30:18.575 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_30_00_UTC 2024-01-24T15:30:18.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:18.576 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-01-24-15_30_00_UTC) 2024-01-24T15:30:18.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:18.645 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2024-01-24T15:30:18.712 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-01-24T15:30:18.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.890+0000 7f4208992700 1 Processor -- start 2024-01-24T15:30:18.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.890+0000 7f4208992700 1 -- start start 2024-01-24T15:30:18.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.891+0000 7f4208992700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4204151b10 0x7f4204155260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:18.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.891+0000 7f4208992700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4204151ef0 0x7f4204069490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:18.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.891+0000 7f4208992700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4204069980 con 0x7f42040c0710 2024-01-24T15:30:18.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.891+0000 7f4208992700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42041547c0 con 0x7f4204151ef0 2024-01-24T15:30:18.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.891+0000 7f4208992700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f420405e540 con 0x7f4204151b10 2024-01-24T15:30:18.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.891+0000 7f4201d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4204151ef0 0x7f4204069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:18.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.891+0000 7f4202d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f42040c0710 0x7f4204062cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38010/0 (socket says 172.21.15.29:38010) 2024-01-24T15:30:18.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.892+0000 7f4202d9d700 1 -- 172.21.15.29:0/196930193 learned_addr learned my addr 172.21.15.29:0/196930193 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:18.899 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.891+0000 7f4201d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4204151ef0 0x7f4204069490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39940/0 (socket says 172.21.15.29:39940) 2024-01-24T15:30:18.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.892+0000 7f420259c700 1 --2- 172.21.15.29:0/196930193 >> v2:172.21.15.163:3300/0 conn(0x7f4204151b10 0x7f4204155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:18.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.892+0000 7f4201d9b700 1 -- 172.21.15.29:0/196930193 >> v2:172.21.15.163:3300/0 conn(0x7f4204151b10 msgr2=0x7f4204155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:18.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.892+0000 7f4201d9b700 1 --2- 172.21.15.29:0/196930193 >> v2:172.21.15.163:3300/0 conn(0x7f4204151b10 0x7f4204155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:18.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.892+0000 7f4201d9b700 1 -- 172.21.15.29:0/196930193 >> v1:172.21.15.29:6789/0 conn(0x7f42040c0710 legacy=0x7f4204062cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:18.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.892+0000 7f4201d9b700 1 -- 172.21.15.29:0/196930193 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4204154a60 con 0x7f4204151ef0 2024-01-24T15:30:18.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.892+0000 7f420159a700 1 -- 172.21.15.29:0/196930193 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084166817 0 0) 0x7f4204069980 con 0x7f42040c0710 2024-01-24T15:30:18.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.892+0000 7f4201d9b700 1 --2- 172.21.15.29:0/196930193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4204151ef0 0x7f4204069490 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7f42040c71d0 tx=0x7f41f4009060).ready entity=mon.0 client_cookie=892d12d2b7ab8099 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:18.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.892+0000 7f420159a700 1 -- 172.21.15.29:0/196930193 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f400f510 con 0x7f4204151ef0 2024-01-24T15:30:18.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.892+0000 7f420159a700 1 -- 172.21.15.29:0/196930193 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f400f670 con 0x7f4204151ef0 2024-01-24T15:30:18.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.892+0000 7f420159a700 1 -- 172.21.15.29:0/196930193 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f400f9f0 con 0x7f4204151ef0 2024-01-24T15:30:18.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.893+0000 7f4208992700 1 -- 172.21.15.29:0/196930193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4204151ef0 msgr2=0x7f4204069490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:18.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.893+0000 7f4208992700 1 --2- 172.21.15.29:0/196930193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4204151ef0 0x7f4204069490 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7f42040c71d0 tx=0x7f41f4009060).stop 2024-01-24T15:30:18.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.893+0000 7f4208992700 1 -- 172.21.15.29:0/196930193 shutdown_connections 2024-01-24T15:30:18.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.893+0000 7f4208992700 1 --2- 172.21.15.29:0/196930193 >> v2:172.21.15.163:3300/0 conn(0x7f4204151b10 0x7f4204155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:18.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.893+0000 7f4208992700 1 --2- 172.21.15.29:0/196930193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4204151ef0 0x7f4204069490 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:18.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.893+0000 7f4208992700 1 -- 172.21.15.29:0/196930193 >> 172.21.15.29:0/196930193 conn(0x7f42040c7d40 msgr2=0x7f42040c3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:18.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.893+0000 7f4208992700 1 -- 172.21.15.29:0/196930193 shutdown_connections 2024-01-24T15:30:18.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.893+0000 7f4208992700 1 -- 172.21.15.29:0/196930193 wait complete. 2024-01-24T15:30:18.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.894+0000 7f4208992700 1 Processor -- start 2024-01-24T15:30:18.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.894+0000 7f4208992700 1 -- start start 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f4208992700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42040c0710 0x7f420414f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f4208992700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4204151b10 0x7f42041583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f4208992700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4204151ef0 0x7f42040c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f4208992700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42040a0610 con 0x7f42040c0710 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f4208992700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4204159990 con 0x7f4204151b10 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f4208992700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420414f980 con 0x7f4204151ef0 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f420259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42040c0710 0x7f420414f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f420259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42040c0710 0x7f420414f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39944/0 (socket says 172.21.15.29:39944) 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f420259c700 1 -- 172.21.15.29:0/233701825 learned_addr learned my addr 172.21.15.29:0/233701825 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f4201d9b700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4204151b10 0x7f42041583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f4202d9d700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4204151ef0 0x7f42040c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f420259c700 1 -- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4204151ef0 msgr2=0x7f42040c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f420259c700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4204151ef0 0x7f42040c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:18.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f420259c700 1 -- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4204151b10 msgr2=0x7f42041583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:18.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f420259c700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4204151b10 0x7f42041583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:18.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f420259c700 1 -- 172.21.15.29:0/233701825 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f40092a0 con 0x7f42040c0710 2024-01-24T15:30:18.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f4201d9b700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4204151b10 0x7f42041583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:18.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f4202d9d700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4204151ef0 0x7f42040c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:18.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.895+0000 7f420259c700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42040c0710 0x7f420414f490 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f41ec00e930 tx=0x7f41ec009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:18.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.896+0000 7f41fa7fc700 1 -- 172.21.15.29:0/233701825 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec00eed0 con 0x7f42040c0710 2024-01-24T15:30:18.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.896+0000 7f41fa7fc700 1 -- 172.21.15.29:0/233701825 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec00c850 con 0x7f42040c0710 2024-01-24T15:30:18.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.896+0000 7f4208992700 1 -- 172.21.15.29:0/233701825 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4204158d80 con 0x7f42040c0710 2024-01-24T15:30:18.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.896+0000 7f41fa7fc700 1 -- 172.21.15.29:0/233701825 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec00cbd0 con 0x7f42040c0710 2024-01-24T15:30:18.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.896+0000 7f4208992700 1 -- 172.21.15.29:0/233701825 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4204159340 con 0x7f42040c0710 2024-01-24T15:30:18.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.897+0000 7f4208992700 1 -- 172.21.15.29:0/233701825 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42040475e0 con 0x7f42040c0710 2024-01-24T15:30:18.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.897+0000 7f41fa7fc700 1 -- 172.21.15.29:0/233701825 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 85) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f41ec01b400 con 0x7f42040c0710 2024-01-24T15:30:18.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.898+0000 7f41fa7fc700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f41e405f2c0 0x7f41e4061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:18.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.898+0000 7f41fa7fc700 1 -- 172.21.15.29:0/233701825 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7674+0+0 (secure 0 0 0) 0x7f41ec07b000 con 0x7f42040c0710 2024-01-24T15:30:18.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.898+0000 7f4201d9b700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f41e405f2c0 0x7f41e4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:18.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.898+0000 7f4201d9b700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f41e405f2c0 0x7f41e4061780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f42041522d0 tx=0x7f41f4017000).ready entity=mgr.8881 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:18.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:18.902+0000 7f41fa7fc700 1 -- 172.21.15.29:0/233701825 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f41ec04bae0 con 0x7f42040c0710 2024-01-24T15:30:19.135 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.133+0000 7f4208992700 1 -- 172.21.15.29:0/233701825 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f4204154d40 con 0x7f42040c0710 2024-01-24T15:30:19.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.295+0000 7f41fa7fc700 1 -- 172.21.15.29:0/233701825 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 86) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f41ec078640 con 0x7f42040c0710 2024-01-24T15:30:19.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.296+0000 7f41fa7fc700 1 -- 172.21.15.29:0/233701825 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v86) v1 ==== 93+0+0 (secure 0 0 0) 0x7f41ec071470 con 0x7f42040c0710 2024-01-24T15:30:19.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.299+0000 7f4208992700 1 -- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f41e405f2c0 msgr2=0x7f41e4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:19.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.299+0000 7f4208992700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f41e405f2c0 0x7f41e4061780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f42041522d0 tx=0x7f41f4017000).stop 2024-01-24T15:30:19.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.299+0000 7f4208992700 1 -- 172.21.15.29:0/233701825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42040c0710 msgr2=0x7f420414f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:19.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.299+0000 7f4208992700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42040c0710 0x7f420414f490 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f41ec00e930 tx=0x7f41ec009060).stop 2024-01-24T15:30:19.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.299+0000 7f4208992700 1 -- 172.21.15.29:0/233701825 shutdown_connections 2024-01-24T15:30:19.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.299+0000 7f4208992700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f41e405f2c0 0x7f41e4061780 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:19.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.299+0000 7f4208992700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4204151ef0 0x7f42040c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:19.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.299+0000 7f4208992700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42040c0710 0x7f420414f490 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:19.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.299+0000 7f4208992700 1 --2- 172.21.15.29:0/233701825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4204151b10 0x7f42041583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:19.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.299+0000 7f4208992700 1 -- 172.21.15.29:0/233701825 >> 172.21.15.29:0/233701825 conn(0x7f42040c7d40 msgr2=0x7f4204152300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:19.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.299+0000 7f4208992700 1 -- 172.21.15.29:0/233701825 shutdown_connections 2024-01-24T15:30:19.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.299+0000 7f4208992700 1 -- 172.21.15.29:0/233701825 wait complete. 2024-01-24T15:30:19.316 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T15:30:19.317 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:30:19.363 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:30:19.413 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T15:30:19.413 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-24T15:30:19.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:19.414 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-24T15:30:19.449 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:19.448+0000 7f5fb946e680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:30:19.460 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:19.458+0000 7fae01b96680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:30:19.475 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T15:30:19.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:30:19.475 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-24T15:30:19.493 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:30:19.514 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:19.512+0000 7f5fb946e680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:30:19.535 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:19.533+0000 7fae01b96680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:30:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.664+0000 7f61cabcc700 1 Processor -- start 2024-01-24T15:30:19.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.664+0000 7f61cabcc700 1 -- start start 2024-01-24T15:30:19.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61cabcc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61c4154370 0x7f61c40c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:19.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61cabcc700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c40aa6c0 con 0x7f61c40aa0e0 2024-01-24T15:30:19.668 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61cabcc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c4064240 con 0x7f61c4154750 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61cabcc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40cb5c0 con 0x7f61c4154370 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61c3fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f61c40aa0e0 0x7f61c40ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38014/0 (socket says 172.21.15.29:38014) 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61c3fff700 1 -- 172.21.15.29:0/4163574069 learned_addr learned my addr 172.21.15.29:0/4163574069 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61c8968700 1 --2- 172.21.15.29:0/4163574069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61c4154370 0x7f61c40c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61c37fe700 1 -- 172.21.15.29:0/4163574069 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20415515 0 0) 0x7f61c40aa6c0 con 0x7f61c40aa0e0 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61c37fe700 1 -- 172.21.15.29:0/4163574069 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b40033f0 con 0x7f61c40aa0e0 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61c37fe700 1 -- 172.21.15.29:0/4163574069 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986944734 0 0) 0x7f61c4064240 con 0x7f61c4154750 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61c37fe700 1 -- 172.21.15.29:0/4163574069 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c40aa6c0 con 0x7f61c4154750 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61c8968700 1 -- 172.21.15.29:0/4163574069 >> v1:172.21.15.163:6789/0 conn(0x7f61c4154750 legacy=0x7f61c40631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61c8968700 1 -- 172.21.15.29:0/4163574069 >> v1:172.21.15.29:6789/0 conn(0x7f61c40aa0e0 legacy=0x7f61c40ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.665+0000 7f61c8968700 1 -- 172.21.15.29:0/4163574069 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c40cb8a0 con 0x7f61c4154370 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.666+0000 7f61c8968700 1 --2- 172.21.15.29:0/4163574069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61c4154370 0x7f61c40c5410 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f61b8009760 tx=0x7f61b8006f40).ready entity=mon.2 client_cookie=fd41cf84623a9fb7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:19.669 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.666+0000 7f61c37fe700 1 -- 172.21.15.29:0/4163574069 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8005880 con 0x7f61c4154370 2024-01-24T15:30:19.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.666+0000 7f61c37fe700 1 -- 172.21.15.29:0/4163574069 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b80059e0 con 0x7f61c4154370 2024-01-24T15:30:19.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.666+0000 7f61c37fe700 1 -- 172.21.15.29:0/4163574069 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8005d60 con 0x7f61c4154370 2024-01-24T15:30:19.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.666+0000 7f61cabcc700 1 -- 172.21.15.29:0/4163574069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61c4154370 msgr2=0x7f61c40c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:19.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.666+0000 7f61cabcc700 1 --2- 172.21.15.29:0/4163574069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61c4154370 0x7f61c40c5410 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f61b8009760 tx=0x7f61b8006f40).stop 2024-01-24T15:30:19.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.666+0000 7f61cabcc700 1 -- 172.21.15.29:0/4163574069 shutdown_connections 2024-01-24T15:30:19.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.666+0000 7f61cabcc700 1 --2- 172.21.15.29:0/4163574069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61c4154370 0x7f61c40c5410 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:19.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.666+0000 7f61cabcc700 1 -- 172.21.15.29:0/4163574069 >> 172.21.15.29:0/4163574069 conn(0x7f61c405ec80 msgr2=0x7f61c40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:19.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.666+0000 7f61cabcc700 1 -- 172.21.15.29:0/4163574069 shutdown_connections 2024-01-24T15:30:19.670 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.666+0000 7f61cabcc700 1 -- 172.21.15.29:0/4163574069 wait complete. 2024-01-24T15:30:19.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.668+0000 7f61cabcc700 1 Processor -- start 2024-01-24T15:30:19.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.668+0000 7f61cabcc700 1 -- start start 2024-01-24T15:30:19.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61cabcc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61c40aa0e0 0x7f61c40c2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:19.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61cabcc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61c4154370 0x7f61c40c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:19.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61cabcc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61c4154750 0x7f61c40c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:19.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61cabcc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c414e450 con 0x7f61c40aa0e0 2024-01-24T15:30:19.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61cabcc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c414e730 con 0x7f61c4154750 2024-01-24T15:30:19.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61cabcc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c414ea10 con 0x7f61c4154370 2024-01-24T15:30:19.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c8968700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61c40aa0e0 0x7f61c40c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:19.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c8968700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61c40aa0e0 0x7f61c40c2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39966/0 (socket says 172.21.15.29:39966) 2024-01-24T15:30:19.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c8968700 1 -- 172.21.15.29:0/515259782 learned_addr learned my addr 172.21.15.29:0/515259782 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:19.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c3fff700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61c4154370 0x7f61c40c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c9169700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61c4154750 0x7f61c40c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c3fff700 1 -- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61c4154750 msgr2=0x7f61c40c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c3fff700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61c4154750 0x7f61c40c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c3fff700 1 -- 172.21.15.29:0/515259782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61c40aa0e0 msgr2=0x7f61c40c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c3fff700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61c40aa0e0 0x7f61c40c2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c3fff700 1 -- 172.21.15.29:0/515259782 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b8009250 con 0x7f61c4154370 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c8968700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61c40aa0e0 0x7f61c40c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c3fff700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61c4154370 0x7f61c40c2810 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f61c406c490 tx=0x7f61b0007bf0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c0ff9700 1 -- 172.21.15.29:0/515259782 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00070d0 con 0x7f61c4154370 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.669+0000 7f61c0ff9700 1 -- 172.21.15.29:0/515259782 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b0007230 con 0x7f61c4154370 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.670+0000 7f61cabcc700 1 -- 172.21.15.29:0/515259782 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c414ecf0 con 0x7f61c4154370 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.670+0000 7f61c0ff9700 1 -- 172.21.15.29:0/515259782 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00075d0 con 0x7f61c4154370 2024-01-24T15:30:19.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.670+0000 7f61cabcc700 1 -- 172.21.15.29:0/515259782 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c40c48b0 con 0x7f61c4154370 2024-01-24T15:30:19.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.671+0000 7f61cabcc700 1 -- 172.21.15.29:0/515259782 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c40475e0 con 0x7f61c4154370 2024-01-24T15:30:19.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.671+0000 7f61c0ff9700 1 -- 172.21.15.29:0/515259782 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 86) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f61b00059b0 con 0x7f61c4154370 2024-01-24T15:30:19.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.671+0000 7f61c0ff9700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61a805f270 0x7f61a8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:19.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.671+0000 7f61c8968700 1 -- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61a805f270 msgr2=0x7f61a8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:19.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.671+0000 7f61c0ff9700 1 -- 172.21.15.29:0/515259782 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7674+0+0 (secure 0 0 0) 0x7f61b007a000 con 0x7f61c4154370 2024-01-24T15:30:19.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.671+0000 7f61c8968700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61a805f270 0x7f61a8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:19.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.676+0000 7f61c0ff9700 1 -- 172.21.15.29:0/515259782 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f61b004ab40 con 0x7f61c4154370 2024-01-24T15:30:19.789 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:19.788+0000 7f5fb946e680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:30:19.855 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:19.853+0000 7fae01b96680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:30:19.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.872+0000 7f61c8968700 1 -- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61a805f270 msgr2=0x7f61a8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:19.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.872+0000 7f61c8968700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61a805f270 0x7f61a8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:30:19.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.905+0000 7f61cabcc700 1 -- 172.21.15.29:0/515259782 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61c40cbb80 con 0x7f61c4154370 2024-01-24T15:30:19.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.905+0000 7f61c0ff9700 1 -- 172.21.15.29:0/515259782 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f61b004e160 con 0x7f61c4154370 2024-01-24T15:30:19.908 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:19.908 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1350237946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1350237946},{"type":"v1","addr":"172.21.15.163:6836","nonce":1350237946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8656,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/250387328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":250387328},{"type":"v1","addr":"172.21.15.163:6837","nonce":250387328}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":9069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3867710046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3867710046},{"type":"v1","addr":"172.21.15.29:6833","nonce":3867710046}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:14:16.673720+0000","modified":"2024-01-24T15:14:19.686795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9072":{"gid":9072,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1929837915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1929837915},{"type":"v1","addr":"172.21.15.29:6835","nonce":1929837915}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-24T15:30:19.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.907+0000 7f61cabcc700 1 -- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61a805f270 msgr2=0x7f61a8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:19.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.907+0000 7f61cabcc700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61a805f270 0x7f61a8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:19.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.907+0000 7f61cabcc700 1 -- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61c4154370 msgr2=0x7f61c40c2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:19.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.907+0000 7f61cabcc700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61c4154370 0x7f61c40c2810 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f61c406c490 tx=0x7f61b0007bf0).stop 2024-01-24T15:30:19.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.907+0000 7f61cabcc700 1 -- 172.21.15.29:0/515259782 shutdown_connections 2024-01-24T15:30:19.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.907+0000 7f61cabcc700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61c4154750 0x7f61c40c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:19.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.907+0000 7f61cabcc700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61c40aa0e0 0x7f61c40c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:19.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.907+0000 7f61cabcc700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61a805f270 0x7f61a8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:19.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.908+0000 7f61cabcc700 1 --2- 172.21.15.29:0/515259782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61c4154370 0x7f61c40c2810 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:19.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.908+0000 7f61cabcc700 1 -- 172.21.15.29:0/515259782 >> 172.21.15.29:0/515259782 conn(0x7f61c405ec80 msgr2=0x7f61c40c12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:19.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.908+0000 7f61cabcc700 1 -- 172.21.15.29:0/515259782 shutdown_connections 2024-01-24T15:30:19.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:19.908+0000 7f61cabcc700 1 -- 172.21.15.29:0/515259782 wait complete. 2024-01-24T15:30:19.912 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 82 2024-01-24T15:30:19.925 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:30:20.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.055+0000 7ff24b7a4700 1 Processor -- start 2024-01-24T15:30:20.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.055+0000 7ff24b7a4700 1 -- start start 2024-01-24T15:30:20.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.056+0000 7ff24b7a4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2440638e0 0x7ff2440aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.056+0000 7ff24b7a4700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2440aa6c0 con 0x7ff2440c5410 2024-01-24T15:30:20.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.056+0000 7ff24b7a4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff244064240 con 0x7ff244154370 2024-01-24T15:30:20.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.056+0000 7ff24b7a4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2440cb5c0 con 0x7ff2440638e0 2024-01-24T15:30:20.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.056+0000 7ff249d41700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff2440638e0 0x7ff2440aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.056+0000 7ff248d3f700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff2440c5410 0x7ff2440631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38028/0 (socket says 172.21.15.29:38028) 2024-01-24T15:30:20.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.056+0000 7ff248d3f700 1 -- 172.21.15.29:0/2406109131 learned_addr learned my addr 172.21.15.29:0/2406109131 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:20.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.056+0000 7ff23bfff700 1 -- 172.21.15.29:0/2406109131 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199119652 0 0) 0x7ff244064240 con 0x7ff244154370 2024-01-24T15:30:20.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.057+0000 7ff249d41700 1 -- 172.21.15.29:0/2406109131 >> v1:172.21.15.163:6789/0 conn(0x7ff244154370 legacy=0x7ff2440ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.057+0000 7ff249d41700 1 -- 172.21.15.29:0/2406109131 >> v1:172.21.15.29:6789/0 conn(0x7ff2440c5410 legacy=0x7ff2440631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.057+0000 7ff249d41700 1 -- 172.21.15.29:0/2406109131 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2440cb8a0 con 0x7ff2440638e0 2024-01-24T15:30:20.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.057+0000 7ff249d41700 1 --2- 172.21.15.29:0/2406109131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2440638e0 0x7ff2440aa0e0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7ff23c009e60 tx=0x7ff23c009780).ready entity=mon.0 client_cookie=54ed9853a1a22059 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:20.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.057+0000 7ff23bfff700 1 -- 172.21.15.29:0/2406109131 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c00e070 con 0x7ff2440638e0 2024-01-24T15:30:20.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.057+0000 7ff23bfff700 1 -- 172.21.15.29:0/2406109131 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c00bb30 con 0x7ff2440638e0 2024-01-24T15:30:20.060 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.057+0000 7ff23bfff700 1 -- 172.21.15.29:0/2406109131 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c00beb0 con 0x7ff2440638e0 2024-01-24T15:30:20.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.057+0000 7ff24b7a4700 1 -- 172.21.15.29:0/2406109131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2440638e0 msgr2=0x7ff2440aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.057+0000 7ff24b7a4700 1 --2- 172.21.15.29:0/2406109131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2440638e0 0x7ff2440aa0e0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7ff23c009e60 tx=0x7ff23c009780).stop 2024-01-24T15:30:20.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.057+0000 7ff24b7a4700 1 -- 172.21.15.29:0/2406109131 shutdown_connections 2024-01-24T15:30:20.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.058+0000 7ff24b7a4700 1 --2- 172.21.15.29:0/2406109131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2440638e0 0x7ff2440aa0e0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.058+0000 7ff24b7a4700 1 -- 172.21.15.29:0/2406109131 >> 172.21.15.29:0/2406109131 conn(0x7ff24405ec80 msgr2=0x7ff2440cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:20.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.058+0000 7ff24b7a4700 1 -- 172.21.15.29:0/2406109131 shutdown_connections 2024-01-24T15:30:20.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.058+0000 7ff24b7a4700 1 -- 172.21.15.29:0/2406109131 wait complete. 2024-01-24T15:30:20.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.059+0000 7ff24b7a4700 1 Processor -- start 2024-01-24T15:30:20.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.059+0000 7ff24b7a4700 1 -- start start 2024-01-24T15:30:20.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff24b7a4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2440638e0 0x7ff2440adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff24b7a4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2440c5410 0x7ff2440ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff24b7a4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff244154370 0x7ff244141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff24b7a4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440af420 con 0x7ff244154370 2024-01-24T15:30:20.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff24b7a4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440ae380 con 0x7ff2440638e0 2024-01-24T15:30:20.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff24b7a4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440ae660 con 0x7ff2440c5410 2024-01-24T15:30:20.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff249d41700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff244154370 0x7ff244141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff249d41700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff244154370 0x7ff244141a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39984/0 (socket says 172.21.15.29:39984) 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff249d41700 1 -- 172.21.15.29:0/1882117288 learned_addr learned my addr 172.21.15.29:0/1882117288 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff249540700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2440638e0 0x7ff2440adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff248d3f700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2440c5410 0x7ff2440ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff249d41700 1 -- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2440c5410 msgr2=0x7ff2440ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff249d41700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2440c5410 0x7ff2440ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff249d41700 1 -- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2440638e0 msgr2=0x7ff2440adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff249d41700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2440638e0 0x7ff2440adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.060+0000 7ff249d41700 1 -- 172.21.15.29:0/1882117288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c009a20 con 0x7ff244154370 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.061+0000 7ff249d41700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff244154370 0x7ff244141a20 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7ff23c011000 tx=0x7ff23c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.061+0000 7ff2397fa700 1 -- 172.21.15.29:0/1882117288 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c021070 con 0x7ff244154370 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.061+0000 7ff2397fa700 1 -- 172.21.15.29:0/1882117288 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c01d630 con 0x7ff244154370 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.061+0000 7ff24b7a4700 1 -- 172.21.15.29:0/1882117288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2440ae940 con 0x7ff244154370 2024-01-24T15:30:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.061+0000 7ff2397fa700 1 -- 172.21.15.29:0/1882117288 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c01d9f0 con 0x7ff244154370 2024-01-24T15:30:20.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.061+0000 7ff24b7a4700 1 -- 172.21.15.29:0/1882117288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2440aef00 con 0x7ff244154370 2024-01-24T15:30:20.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.062+0000 7ff24b7a4700 1 -- 172.21.15.29:0/1882117288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2440475e0 con 0x7ff244154370 2024-01-24T15:30:20.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.062+0000 7ff2397fa700 1 -- 172.21.15.29:0/1882117288 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 86) v1 ==== 75560+0+0 (secure 0 0 0) 0x7ff23c003040 con 0x7ff244154370 2024-01-24T15:30:20.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.063+0000 7ff2397fa700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff22805f270 0x7ff228061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.063+0000 7ff2397fa700 1 -- 172.21.15.29:0/1882117288 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7674+0+0 (secure 0 0 0) 0x7ff23c07c0d0 con 0x7ff244154370 2024-01-24T15:30:20.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.063+0000 7ff249540700 1 -- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff22805f270 msgr2=0x7ff228061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:20.066 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.063+0000 7ff249540700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff22805f270 0x7ff228061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:20.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.068+0000 7ff2397fa700 1 -- 172.21.15.29:0/1882117288 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff23c04cc10 con 0x7ff244154370 2024-01-24T15:30:20.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.251+0000 7ff24b7a4700 1 -- 172.21.15.29:0/1882117288 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2440cbb80 con 0x7ff244154370 2024-01-24T15:30:20.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.252+0000 7ff2397fa700 1 -- 172.21.15.29:0/1882117288 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+18979 (secure 0 0 0) 0x7ff23c030a40 con 0x7ff244154370 2024-01-24T15:30:20.254 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:20.254 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":197,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:25:20.634328+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:14:14.419558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-24T15:14:15.659901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.5","mappings":[{"from":3,"to":0}]},{"pgid":"22.13","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/2002665960":"2024-01-25T15:25:20.634238+0000","172.21.15.163:0/3305307869":"2024-01-25T15:25:20.634238+0000","172.21.15.163:0/4000114899":"2024-01-25T15:25:20.634238+0000","172.21.15.163:6832/43241":"2024-01-25T15:14:28.971452+0000","172.21.15.163:0/2434196663":"2024-01-25T15:14:28.971452+0000","172.21.15.163:6833/43241":"2024-01-25T15:14:28.971452+0000","172.21.15.163:0/4284075186":"2024-01-25T15:14:28.971452+0000","172.21.15.163:0/3544734349":"2024-01-25T15:25:20.634238+0000","172.21.15.163:0/2471445585":"2024-01-25T15:14:28.971452+0000","172.21.15.163:0/2024052538":"2024-01-25T15:14:28.971452+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:30:20.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.254+0000 7ff24b7a4700 1 -- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff22805f270 msgr2=0x7ff228061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:20.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.254+0000 7ff24b7a4700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff22805f270 0x7ff228061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:20.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.254+0000 7ff24b7a4700 1 -- 172.21.15.29:0/1882117288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff244154370 msgr2=0x7ff244141a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.254+0000 7ff24b7a4700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff244154370 0x7ff244141a20 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7ff23c011000 tx=0x7ff23c009780).stop 2024-01-24T15:30:20.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.254+0000 7ff24b7a4700 1 -- 172.21.15.29:0/1882117288 shutdown_connections 2024-01-24T15:30:20.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.254+0000 7ff24b7a4700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff244154370 0x7ff244141a20 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.254+0000 7ff24b7a4700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2440638e0 0x7ff2440adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.254+0000 7ff24b7a4700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff22805f270 0x7ff228061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:20.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.254+0000 7ff24b7a4700 1 --2- 172.21.15.29:0/1882117288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2440c5410 0x7ff2440ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.254+0000 7ff24b7a4700 1 -- 172.21.15.29:0/1882117288 >> 172.21.15.29:0/1882117288 conn(0x7ff24405ec80 msgr2=0x7ff2440c5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:20.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.254+0000 7ff24b7a4700 1 -- 172.21.15.29:0/1882117288 shutdown_connections 2024-01-24T15:30:20.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.254+0000 7ff24b7a4700 1 -- 172.21.15.29:0/1882117288 wait complete. 2024-01-24T15:30:20.270 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '196', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T15:14:14.419558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '192', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T15:14:15.659901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '193', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T15:30:20.271 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T15:30:20.404 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:20.403+0000 7f5fb946e680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:30:20.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.406+0000 7fef937d8700 1 Processor -- start 2024-01-24T15:30:20.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.407+0000 7fef937d8700 1 -- start start 2024-01-24T15:30:20.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.407+0000 7fef937d8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fef8c0692a0 0x7fef8c065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.407+0000 7fef937d8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c069d90 0x7fef8c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef937d8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8c0c3e70 con 0x7fef8c066370 2024-01-24T15:30:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef937d8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef8c0c4300 con 0x7fef8c0692a0 2024-01-24T15:30:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef937d8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0c3260 con 0x7fef8c069d90 2024-01-24T15:30:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef90d73700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fef8c066370 0x7fef8c069680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38038/0 (socket says 172.21.15.29:38038) 2024-01-24T15:30:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef90d73700 1 -- 172.21.15.29:0/1426558774 learned_addr learned my addr 172.21.15.29:0/1426558774 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef91d75700 1 --2- 172.21.15.29:0/1426558774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c069d90 0x7fef8c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef91574700 1 --2- 172.21.15.29:0/1426558774 >> v2:172.21.15.163:3300/0 conn(0x7fef8c0692a0 0x7fef8c065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef83fff700 1 -- 172.21.15.29:0/1426558774 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4248760412 0 0) 0x7fef8c0c3e70 con 0x7fef8c066370 2024-01-24T15:30:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef83fff700 1 -- 172.21.15.29:0/1426558774 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef780033f0 con 0x7fef8c066370 2024-01-24T15:30:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef91574700 1 -- 172.21.15.29:0/1426558774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c069d90 msgr2=0x7fef8c15a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:20.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef91574700 1 --2- 172.21.15.29:0/1426558774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c069d90 0x7fef8c15a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef91574700 1 -- 172.21.15.29:0/1426558774 >> v1:172.21.15.29:6789/0 conn(0x7fef8c066370 legacy=0x7fef8c069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef91574700 1 -- 172.21.15.29:0/1426558774 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0c3540 con 0x7fef8c0692a0 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef91d75700 1 --2- 172.21.15.29:0/1426558774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c069d90 0x7fef8c15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.408+0000 7fef91574700 1 --2- 172.21.15.29:0/1426558774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef8c0692a0 0x7fef8c065e80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fef88009760 tx=0x7fef88009060).ready entity=mon.1 client_cookie=3a40a9da6d9ca7b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.409+0000 7fef83fff700 1 -- 172.21.15.29:0/1426558774 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800d4a0 con 0x7fef8c0692a0 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.409+0000 7fef83fff700 1 -- 172.21.15.29:0/1426558774 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8800d600 con 0x7fef8c0692a0 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.409+0000 7fef83fff700 1 -- 172.21.15.29:0/1426558774 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800d980 con 0x7fef8c0692a0 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.409+0000 7fef937d8700 1 -- 172.21.15.29:0/1426558774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef8c0692a0 msgr2=0x7fef8c065e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.409+0000 7fef937d8700 1 --2- 172.21.15.29:0/1426558774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef8c0692a0 0x7fef8c065e80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fef88009760 tx=0x7fef88009060).stop 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.409+0000 7fef937d8700 1 -- 172.21.15.29:0/1426558774 shutdown_connections 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.409+0000 7fef937d8700 1 --2- 172.21.15.29:0/1426558774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c069d90 0x7fef8c15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.409+0000 7fef937d8700 1 --2- 172.21.15.29:0/1426558774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef8c0692a0 0x7fef8c065e80 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.409+0000 7fef937d8700 1 -- 172.21.15.29:0/1426558774 >> 172.21.15.29:0/1426558774 conn(0x7fef8c064390 msgr2=0x7fef8c0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:20.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.409+0000 7fef937d8700 1 -- 172.21.15.29:0/1426558774 shutdown_connections 2024-01-24T15:30:20.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.409+0000 7fef937d8700 1 -- 172.21.15.29:0/1426558774 wait complete. 2024-01-24T15:30:20.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.411+0000 7fef937d8700 1 Processor -- start 2024-01-24T15:30:20.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.411+0000 7fef937d8700 1 -- start start 2024-01-24T15:30:20.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef937d8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c066370 0x7fef8c062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef937d8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef8c0692a0 0x7fef8c163ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef937d8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef8c069d90 0x7fef8c163fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef937d8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0ac630 con 0x7fef8c069d90 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef937d8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c151c50 con 0x7fef8c0692a0 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef937d8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c151ef0 con 0x7fef8c066370 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef91d75700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef8c069d90 0x7fef8c163fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef91574700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c066370 0x7fef8c062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef91d75700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef8c069d90 0x7fef8c163fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39988/0 (socket says 172.21.15.29:39988) 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef91574700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c066370 0x7fef8c062970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:52206/0 (socket says 172.21.15.29:52206) 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef91d75700 1 -- 172.21.15.29:0/3702995728 learned_addr learned my addr 172.21.15.29:0/3702995728 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef90d73700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef8c0692a0 0x7fef8c163ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef91574700 1 -- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef8c0692a0 msgr2=0x7fef8c163ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef91574700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef8c0692a0 0x7fef8c163ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef91574700 1 -- 172.21.15.29:0/3702995728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef8c069d90 msgr2=0x7fef8c163fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef91574700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef8c069d90 0x7fef8c163fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef91574700 1 -- 172.21.15.29:0/3702995728 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef84009a20 con 0x7fef8c066370 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef91d75700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef8c069d90 0x7fef8c163fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.412+0000 7fef91574700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c066370 0x7fef8c062970 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fef8c0c8b30 tx=0x7fef88016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.413+0000 7fef817fa700 1 -- 172.21.15.29:0/3702995728 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800dd30 con 0x7fef8c066370 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.413+0000 7fef817fa700 1 -- 172.21.15.29:0/3702995728 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8800de90 con 0x7fef8c066370 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.413+0000 7fef817fa700 1 -- 172.21.15.29:0/3702995728 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800d980 con 0x7fef8c066370 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.413+0000 7fef937d8700 1 -- 172.21.15.29:0/3702995728 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef880092a0 con 0x7fef8c066370 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.413+0000 7fef937d8700 1 -- 172.21.15.29:0/3702995728 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c1644f0 con 0x7fef8c066370 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.414+0000 7fef937d8700 1 -- 172.21.15.29:0/3702995728 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8c0475e0 con 0x7fef8c066370 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.414+0000 7fef817fa700 1 -- 172.21.15.29:0/3702995728 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 86) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fef8801d640 con 0x7fef8c066370 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.415+0000 7fef817fa700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fef7005f270 0x7fef70061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.415+0000 7fef90d73700 1 -- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fef7005f270 msgr2=0x7fef70061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:20.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.415+0000 7fef817fa700 1 -- 172.21.15.29:0/3702995728 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7674+0+0 (secure 0 0 0) 0x7fef8807bd90 con 0x7fef8c066370 2024-01-24T15:30:20.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.415+0000 7fef90d73700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fef7005f270 0x7fef70061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:20.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.422+0000 7fef817fa700 1 -- 172.21.15.29:0/3702995728 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fef8804c8d0 con 0x7fef8c066370 2024-01-24T15:30:20.477 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:20.475+0000 7f5fb946e680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:30:20.595 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:20.593+0000 7fae01b96680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:30:20.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.616+0000 7fef90d73700 1 -- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fef7005f270 msgr2=0x7fef70061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:20.618 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.616+0000 7fef90d73700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fef7005f270 0x7fef70061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:30:20.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.651+0000 7fef937d8700 1 -- 172.21.15.29:0/3702995728 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fef8c1644f0 con 0x7fef8c066370 2024-01-24T15:30:20.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.652+0000 7fef817fa700 1 -- 172.21.15.29:0/3702995728 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v197) v1 ==== 110+0+10 (secure 0 0 0) 0x7fef8c1644f0 con 0x7fef8c066370 2024-01-24T15:30:20.654 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T15:30:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.654+0000 7fef937d8700 1 -- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fef7005f270 msgr2=0x7fef70061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.654+0000 7fef937d8700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fef7005f270 0x7fef70061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.654+0000 7fef937d8700 1 -- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c066370 msgr2=0x7fef8c062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.654+0000 7fef937d8700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c066370 0x7fef8c062970 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fef8c0c8b30 tx=0x7fef88016000).stop 2024-01-24T15:30:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.654+0000 7fef937d8700 1 -- 172.21.15.29:0/3702995728 shutdown_connections 2024-01-24T15:30:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.654+0000 7fef937d8700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef8c069d90 0x7fef8c163fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.654+0000 7fef937d8700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fef7005f270 0x7fef70061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.654+0000 7fef937d8700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef8c066370 0x7fef8c062970 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.654+0000 7fef937d8700 1 --2- 172.21.15.29:0/3702995728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef8c0692a0 0x7fef8c163ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.654+0000 7fef937d8700 1 -- 172.21.15.29:0/3702995728 >> 172.21.15.29:0/3702995728 conn(0x7fef8c064390 msgr2=0x7fef8c156120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.654+0000 7fef937d8700 1 -- 172.21.15.29:0/3702995728 shutdown_connections 2024-01-24T15:30:20.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.654+0000 7fef937d8700 1 -- 172.21.15.29:0/3702995728 wait complete. 2024-01-24T15:30:20.665 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:20.662+0000 7f5fb946e680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:30:20.670 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T15:30:20.671 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:20.670+0000 7fae01b96680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:30:20.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.806+0000 7fbfa6c8b700 1 Processor -- start 2024-01-24T15:30:20.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.806+0000 7fbfa6c8b700 1 -- start start 2024-01-24T15:30:20.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbfa6c8b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbfa00a7210 0x7fbfa00c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbfa6c8b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfa0062ec0 0x7fbfa0065420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbfa6c8b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa00a6d80 con 0x7fbfa0154100 2024-01-24T15:30:20.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbfa6c8b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa00bcaf0 con 0x7fbfa00a7210 2024-01-24T15:30:20.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbfa6c8b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa00bcd90 con 0x7fbfa0062ec0 2024-01-24T15:30:20.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbf9ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbfa00a7210 0x7fbfa00c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbf9ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbfa00a7210 0x7fbfa00c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:39996/0 (socket says 172.21.15.29:39996) 2024-01-24T15:30:20.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbf9ffff700 1 -- 172.21.15.29:0/3460093385 learned_addr learned my addr 172.21.15.29:0/3460093385 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:20.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbfa5228700 1 --2- 172.21.15.29:0/3460093385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfa0062ec0 0x7fbfa0065420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbf9ffff700 1 -- 172.21.15.29:0/3460093385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfa0062ec0 msgr2=0x7fbfa0065420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbf9ffff700 1 --2- 172.21.15.29:0/3460093385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfa0062ec0 0x7fbfa0065420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbf9ffff700 1 -- 172.21.15.29:0/3460093385 >> v1:172.21.15.163:6789/0 conn(0x7fbfa0154100 legacy=0x7fbfa00627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbf9ffff700 1 -- 172.21.15.29:0/3460093385 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa00bd070 con 0x7fbfa00a7210 2024-01-24T15:30:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbf9f7fe700 1 -- 172.21.15.29:0/3460093385 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599741116 0 0) 0x7fbfa00a6d80 con 0x7fbfa0154100 2024-01-24T15:30:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.807+0000 7fbf9ffff700 1 --2- 172.21.15.29:0/3460093385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa00a7210 0x7fbfa00c8200 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7fbf8c006fd0 tx=0x7fbf8c0090f0).ready entity=mon.0 client_cookie=6cb2ceaeeb7858d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.808+0000 7fbf9f7fe700 1 -- 172.21.15.29:0/3460093385 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c00b580 con 0x7fbfa00a7210 2024-01-24T15:30:20.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.808+0000 7fbf9f7fe700 1 -- 172.21.15.29:0/3460093385 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8c00b6e0 con 0x7fbfa00a7210 2024-01-24T15:30:20.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.808+0000 7fbf9f7fe700 1 -- 172.21.15.29:0/3460093385 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c00ba60 con 0x7fbfa00a7210 2024-01-24T15:30:20.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.808+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3460093385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa00a7210 msgr2=0x7fbfa00c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.808+0000 7fbfa6c8b700 1 --2- 172.21.15.29:0/3460093385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa00a7210 0x7fbfa00c8200 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7fbf8c006fd0 tx=0x7fbf8c0090f0).stop 2024-01-24T15:30:20.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.808+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3460093385 shutdown_connections 2024-01-24T15:30:20.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.808+0000 7fbfa6c8b700 1 --2- 172.21.15.29:0/3460093385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfa0062ec0 0x7fbfa0065420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.808+0000 7fbfa6c8b700 1 --2- 172.21.15.29:0/3460093385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa00a7210 0x7fbfa00c8200 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.808+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3460093385 >> 172.21.15.29:0/3460093385 conn(0x7fbfa0064390 msgr2=0x7fbfa00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:20.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.808+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3460093385 shutdown_connections 2024-01-24T15:30:20.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.808+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3460093385 wait complete. 2024-01-24T15:30:20.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.810+0000 7fbfa6c8b700 1 Processor -- start 2024-01-24T15:30:20.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.810+0000 7fbfa6c8b700 1 -- start start 2024-01-24T15:30:20.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa6c8b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa0062ec0 0x7fbfa01426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa6c8b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfa00a7210 0x7fbfa00a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa6c8b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa0154100 0x7fbfa00a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa6c8b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa0142b90 con 0x7fbfa0062ec0 2024-01-24T15:30:20.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa6c8b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa00a5260 con 0x7fbfa0154100 2024-01-24T15:30:20.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa6c8b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa00bfb60 con 0x7fbfa00a7210 2024-01-24T15:30:20.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa4a27700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa0062ec0 0x7fbfa01426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa4a27700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa0062ec0 0x7fbfa01426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40002/0 (socket says 172.21.15.29:40002) 2024-01-24T15:30:20.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa4a27700 1 -- 172.21.15.29:0/3669171016 learned_addr learned my addr 172.21.15.29:0/3669171016 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:20.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbf9ffff700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfa00a7210 0x7fbfa00a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa5228700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa0154100 0x7fbfa00a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:20.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa4a27700 1 -- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfa00a7210 msgr2=0x7fbfa00a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa4a27700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfa00a7210 0x7fbfa00a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa4a27700 1 -- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa0154100 msgr2=0x7fbfa00a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:20.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa4a27700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa0154100 0x7fbfa00a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:20.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa4a27700 1 -- 172.21.15.29:0/3669171016 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c009370 con 0x7fbfa0062ec0 2024-01-24T15:30:20.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbfa4a27700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa0062ec0 0x7fbfa01426a0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7fbf9400c410 tx=0x7fbf94007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:20.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.811+0000 7fbf9cff9700 1 -- 172.21.15.29:0/3669171016 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94007020 con 0x7fbfa0062ec0 2024-01-24T15:30:20.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.812+0000 7fbf9cff9700 1 -- 172.21.15.29:0/3669171016 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf940071a0 con 0x7fbfa0062ec0 2024-01-24T15:30:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.812+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3669171016 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa00c06e0 con 0x7fbfa0062ec0 2024-01-24T15:30:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.812+0000 7fbf9cff9700 1 -- 172.21.15.29:0/3669171016 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94007560 con 0x7fbfa0062ec0 2024-01-24T15:30:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.812+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3669171016 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa00c0090 con 0x7fbfa0062ec0 2024-01-24T15:30:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.813+0000 7fbf9cff9700 1 -- 172.21.15.29:0/3669171016 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 86) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fbf940068b0 con 0x7fbfa0062ec0 2024-01-24T15:30:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.813+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3669171016 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa00475e0 con 0x7fbfa0062ec0 2024-01-24T15:30:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.813+0000 7fbf9cff9700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf8405f270 0x7fbf84061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.814+0000 7fbf9ffff700 1 -- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf8405f270 msgr2=0x7fbf84061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.814+0000 7fbf9cff9700 1 -- 172.21.15.29:0/3669171016 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7674+0+0 (secure 0 0 0) 0x7fbf94079d70 con 0x7fbfa0062ec0 2024-01-24T15:30:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.814+0000 7fbf9ffff700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf8405f270 0x7fbf84061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:20.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:20.818+0000 7fbf9cff9700 1 -- 172.21.15.29:0/3669171016 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbf94047b00 con 0x7fbfa0062ec0 2024-01-24T15:30:20.918 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:20.916+0000 7fae01b96680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:30:21.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.015+0000 7fbf9ffff700 1 -- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf8405f270 msgr2=0x7fbf84061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:21.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.015+0000 7fbf9ffff700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf8405f270 0x7fbf84061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:30:21.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.031+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3669171016 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbfa006a1e0 con 0x7fbfa0062ec0 2024-01-24T15:30:21.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.031+0000 7fbf9cff9700 1 -- 172.21.15.29:0/3669171016 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v197) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbf9404a7f0 con 0x7fbfa0062ec0 2024-01-24T15:30:21.033 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T15:30:21.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.034+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf8405f270 msgr2=0x7fbf84061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:21.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.034+0000 7fbfa6c8b700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf8405f270 0x7fbf84061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:21.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.034+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3669171016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa0062ec0 msgr2=0x7fbfa01426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.034+0000 7fbfa6c8b700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa0062ec0 0x7fbfa01426a0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7fbf9400c410 tx=0x7fbf94007e00).stop 2024-01-24T15:30:21.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.034+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3669171016 shutdown_connections 2024-01-24T15:30:21.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.034+0000 7fbfa6c8b700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa0154100 0x7fbfa00a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.034+0000 7fbfa6c8b700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfa0062ec0 0x7fbfa01426a0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.034+0000 7fbfa6c8b700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf8405f270 0x7fbf84061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:21.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.034+0000 7fbfa6c8b700 1 --2- 172.21.15.29:0/3669171016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfa00a7210 0x7fbfa00a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.034+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3669171016 >> 172.21.15.29:0/3669171016 conn(0x7fbfa0064390 msgr2=0x7fbfa00c5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:21.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.034+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3669171016 shutdown_connections 2024-01-24T15:30:21.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.034+0000 7fbfa6c8b700 1 -- 172.21.15.29:0/3669171016 wait complete. 2024-01-24T15:30:21.050 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T15:30:21.087 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:21.084+0000 7f5fb946e680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:30:21.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.184+0000 7f2e478ec700 1 Processor -- start 2024-01-24T15:30:21.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.184+0000 7f2e478ec700 1 -- start start 2024-01-24T15:30:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.185+0000 7f2e478ec700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2e4013a5e0 0x7f2e400639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.185+0000 7f2e478ec700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2e40066bd0 0x7f2e400bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.185+0000 7f2e478ec700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e400679f0 con 0x7f2e40063eb0 2024-01-24T15:30:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.185+0000 7f2e478ec700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e40067cd0 con 0x7f2e4013a5e0 2024-01-24T15:30:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.185+0000 7f2e478ec700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e40067fb0 con 0x7f2e40066bd0 2024-01-24T15:30:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.185+0000 7f2e45688700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2e4013a5e0 0x7f2e400639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.185+0000 7f2e45688700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2e4013a5e0 0x7f2e400639c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40008/0 (socket says 172.21.15.29:40008) 2024-01-24T15:30:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.185+0000 7f2e45688700 1 -- 172.21.15.29:0/3869733875 learned_addr learned my addr 172.21.15.29:0/3869733875 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.185+0000 7f2e45e89700 1 --2- 172.21.15.29:0/3869733875 >> v2:172.21.15.163:3300/0 conn(0x7f2e40066bd0 0x7f2e400bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.186+0000 7f2e45688700 1 -- 172.21.15.29:0/3869733875 >> v2:172.21.15.163:3300/0 conn(0x7f2e40066bd0 msgr2=0x7f2e400bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.186+0000 7f2e45688700 1 --2- 172.21.15.29:0/3869733875 >> v2:172.21.15.163:3300/0 conn(0x7f2e40066bd0 0x7f2e400bdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.186+0000 7f2e45688700 1 -- 172.21.15.29:0/3869733875 >> v1:172.21.15.163:6789/0 conn(0x7f2e40063eb0 legacy=0x7f2e400664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.186+0000 7f2e45688700 1 -- 172.21.15.29:0/3869733875 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e40068290 con 0x7f2e4013a5e0 2024-01-24T15:30:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.186+0000 7f2e45688700 1 --2- 172.21.15.29:0/3869733875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e4013a5e0 0x7f2e400639c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f2e4006a250 tx=0x7f2e380090f0).ready entity=mon.0 client_cookie=8da50f93e8b1999e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.186+0000 7f2e37fff700 1 -- 172.21.15.29:0/3869733875 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3800b520 con 0x7f2e4013a5e0 2024-01-24T15:30:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.186+0000 7f2e37fff700 1 -- 172.21.15.29:0/3869733875 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3800b680 con 0x7f2e4013a5e0 2024-01-24T15:30:21.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.186+0000 7f2e37fff700 1 -- 172.21.15.29:0/3869733875 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3800ba00 con 0x7f2e4013a5e0 2024-01-24T15:30:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.187+0000 7f2e478ec700 1 -- 172.21.15.29:0/3869733875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e4013a5e0 msgr2=0x7f2e400639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.187+0000 7f2e478ec700 1 --2- 172.21.15.29:0/3869733875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e4013a5e0 0x7f2e400639c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f2e4006a250 tx=0x7f2e380090f0).stop 2024-01-24T15:30:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.187+0000 7f2e478ec700 1 -- 172.21.15.29:0/3869733875 shutdown_connections 2024-01-24T15:30:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.187+0000 7f2e478ec700 1 --2- 172.21.15.29:0/3869733875 >> v2:172.21.15.163:3300/0 conn(0x7f2e40066bd0 0x7f2e400bdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.187+0000 7f2e478ec700 1 --2- 172.21.15.29:0/3869733875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e4013a5e0 0x7f2e400639c0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.187+0000 7f2e478ec700 1 -- 172.21.15.29:0/3869733875 >> 172.21.15.29:0/3869733875 conn(0x7f2e400c8bd0 msgr2=0x7f2e400c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.187+0000 7f2e478ec700 1 -- 172.21.15.29:0/3869733875 shutdown_connections 2024-01-24T15:30:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.187+0000 7f2e478ec700 1 -- 172.21.15.29:0/3869733875 wait complete. 2024-01-24T15:30:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.188+0000 7f2e478ec700 1 Processor -- start 2024-01-24T15:30:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.188+0000 7f2e478ec700 1 -- start start 2024-01-24T15:30:21.192 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:21.189+0000 7f5fb946e680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:30:21.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.189+0000 7f2e478ec700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e40063eb0 0x7f2e400ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.189+0000 7f2e478ec700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e40066bd0 0x7f2e400ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.189+0000 7f2e478ec700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4013a5e0 0x7f2e400c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.189+0000 7f2e478ec700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4013b950 con 0x7f2e40063eb0 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.189+0000 7f2e478ec700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4013ad40 con 0x7f2e4013a5e0 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.189+0000 7f2e478ec700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4013b020 con 0x7f2e40066bd0 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.189+0000 7f2e45688700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e40063eb0 0x7f2e400ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.189+0000 7f2e45688700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e40063eb0 0x7f2e400ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40014/0 (socket says 172.21.15.29:40014) 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.189+0000 7f2e45e89700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4013a5e0 0x7f2e400c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.189+0000 7f2e45688700 1 -- 172.21.15.29:0/347019339 learned_addr learned my addr 172.21.15.29:0/347019339 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.189+0000 7f2e44e87700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e40066bd0 0x7f2e400ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.190+0000 7f2e45e89700 1 -- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e40066bd0 msgr2=0x7f2e400ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.190+0000 7f2e45e89700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e40066bd0 0x7f2e400ac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.190+0000 7f2e45e89700 1 -- 172.21.15.29:0/347019339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e40063eb0 msgr2=0x7f2e400ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.190+0000 7f2e45e89700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e40063eb0 0x7f2e400ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.190+0000 7f2e45e89700 1 -- 172.21.15.29:0/347019339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38009370 con 0x7f2e4013a5e0 2024-01-24T15:30:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.190+0000 7f2e45688700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e40063eb0 0x7f2e400ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.190+0000 7f2e45e89700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4013a5e0 0x7f2e400c79d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f2e3c009e50 tx=0x7f2e3c00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.190+0000 7f2e357fa700 1 -- 172.21.15.29:0/347019339 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00caa0 con 0x7f2e4013a5e0 2024-01-24T15:30:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.190+0000 7f2e357fa700 1 -- 172.21.15.29:0/347019339 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c00f040 con 0x7f2e4013a5e0 2024-01-24T15:30:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.190+0000 7f2e478ec700 1 -- 172.21.15.29:0/347019339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4013b300 con 0x7f2e4013a5e0 2024-01-24T15:30:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.190+0000 7f2e478ec700 1 -- 172.21.15.29:0/347019339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e400ae9f0 con 0x7f2e4013a5e0 2024-01-24T15:30:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.190+0000 7f2e357fa700 1 -- 172.21.15.29:0/347019339 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00ce10 con 0x7f2e4013a5e0 2024-01-24T15:30:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.192+0000 7f2e478ec700 1 -- 172.21.15.29:0/347019339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4013bac0 con 0x7f2e4013a5e0 2024-01-24T15:30:21.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.192+0000 7f2e357fa700 1 -- 172.21.15.29:0/347019339 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 86) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f2e3c013400 con 0x7f2e4013a5e0 2024-01-24T15:30:21.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.193+0000 7f2e357fa700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2e2405f270 0x7f2e24061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.193+0000 7f2e357fa700 1 -- 172.21.15.29:0/347019339 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7674+0+0 (secure 0 0 0) 0x7f2e3c07b2a0 con 0x7f2e4013a5e0 2024-01-24T15:30:21.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.193+0000 7f2e45688700 1 -- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2e2405f270 msgr2=0x7f2e24061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:21.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.193+0000 7f2e45688700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2e2405f270 0x7f2e24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:21.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.197+0000 7f2e357fa700 1 -- 172.21.15.29:0/347019339 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f2e3c04bd80 con 0x7f2e4013a5e0 2024-01-24T15:30:21.355 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:21.352+0000 7f5fb946e680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:30:21.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.394+0000 7f2e45688700 1 -- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2e2405f270 msgr2=0x7f2e24061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:21.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.394+0000 7f2e45688700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2e2405f270 0x7f2e24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:30:21.439 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:21.436+0000 7f5fb946e680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:30:21.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.472+0000 7f2e478ec700 1 -- 172.21.15.29:0/347019339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2e400475e0 con 0x7f2e4013a5e0 2024-01-24T15:30:21.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.473+0000 7f2e357fa700 1 -- 172.21.15.29:0/347019339 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v198) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2e3c04f3a0 con 0x7f2e4013a5e0 2024-01-24T15:30:21.475 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T15:30:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.475+0000 7f2e478ec700 1 -- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2e2405f270 msgr2=0x7f2e24061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.475+0000 7f2e478ec700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2e2405f270 0x7f2e24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.475+0000 7f2e478ec700 1 -- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4013a5e0 msgr2=0x7f2e400c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.475+0000 7f2e478ec700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4013a5e0 0x7f2e400c79d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f2e3c009e50 tx=0x7f2e3c00b040).stop 2024-01-24T15:30:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.475+0000 7f2e478ec700 1 -- 172.21.15.29:0/347019339 shutdown_connections 2024-01-24T15:30:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.475+0000 7f2e478ec700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4013a5e0 0x7f2e400c79d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.475+0000 7f2e478ec700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e40063eb0 0x7f2e400ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.475+0000 7f2e478ec700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2e2405f270 0x7f2e24061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.475+0000 7f2e478ec700 1 --2- 172.21.15.29:0/347019339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e40066bd0 0x7f2e400ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.475+0000 7f2e478ec700 1 -- 172.21.15.29:0/347019339 >> 172.21.15.29:0/347019339 conn(0x7f2e400c8bd0 msgr2=0x7f2e40139820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.475+0000 7f2e478ec700 1 -- 172.21.15.29:0/347019339 shutdown_connections 2024-01-24T15:30:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.475+0000 7f2e478ec700 1 -- 172.21.15.29:0/347019339 wait complete. 2024-01-24T15:30:21.491 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T15:30:21.491 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:21.489+0000 7fae01b96680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:30:21.507 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:21.505+0000 7f5fb946e680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:30:21.626 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:21.625+0000 7fae01b96680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:30:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.637+0000 7f5b08a29700 1 Processor -- start 2024-01-24T15:30:21.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.637+0000 7f5b08a29700 1 -- start start 2024-01-24T15:30:21.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.638+0000 7f5b08a29700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5b04066950 0x7f5b040be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.638+0000 7f5b08a29700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b040640d0 con 0x7f5b040664c0 2024-01-24T15:30:21.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.638+0000 7f5b08a29700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b040679f0 con 0x7f5b0413a5e0 2024-01-24T15:30:21.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.638+0000 7f5b08a29700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b04067cd0 con 0x7f5b04066950 2024-01-24T15:30:21.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.638+0000 7f5b01d9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5b040664c0 0x7f5b040bdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38048/0 (socket says 172.21.15.29:38048) 2024-01-24T15:30:21.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.638+0000 7f5b01d9b700 1 -- 172.21.15.29:0/901979262 learned_addr learned my addr 172.21.15.29:0/901979262 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:21.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.638+0000 7f5b02d9d700 1 --2- 172.21.15.29:0/901979262 >> v2:172.21.15.163:3300/0 conn(0x7f5b04066950 0x7f5b040be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.639+0000 7f5b0159a700 1 -- 172.21.15.29:0/901979262 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998664688 0 0) 0x7f5b040640d0 con 0x7f5b040664c0 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.639+0000 7f5b0159a700 1 -- 172.21.15.29:0/901979262 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5aec0033f0 con 0x7f5b040664c0 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.639+0000 7f5b0159a700 1 -- 172.21.15.29:0/901979262 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130171397 0 0) 0x7f5b040679f0 con 0x7f5b0413a5e0 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.639+0000 7f5b0159a700 1 -- 172.21.15.29:0/901979262 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b040640d0 con 0x7f5b0413a5e0 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.639+0000 7f5b02d9d700 1 -- 172.21.15.29:0/901979262 >> v1:172.21.15.163:6789/0 conn(0x7f5b0413a5e0 legacy=0x7f5b040639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.639+0000 7f5b02d9d700 1 -- 172.21.15.29:0/901979262 >> v1:172.21.15.29:6789/0 conn(0x7f5b040664c0 legacy=0x7f5b040bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.639+0000 7f5b02d9d700 1 -- 172.21.15.29:0/901979262 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b04067fb0 con 0x7f5b04066950 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.639+0000 7f5b02d9d700 1 --2- 172.21.15.29:0/901979262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b04066950 0x7f5b040be240 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f5afc009e60 tx=0x7f5afc009780).ready entity=mon.1 client_cookie=88da0b6222dcc83f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.639+0000 7f5b0159a700 1 -- 172.21.15.29:0/901979262 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00e070 con 0x7f5b04066950 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.639+0000 7f5b0159a700 1 -- 172.21.15.29:0/901979262 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc00bab0 con 0x7f5b04066950 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.639+0000 7f5b0159a700 1 -- 172.21.15.29:0/901979262 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00be30 con 0x7f5b04066950 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.640+0000 7f5b08a29700 1 -- 172.21.15.29:0/901979262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b04066950 msgr2=0x7f5b040be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.640+0000 7f5b08a29700 1 --2- 172.21.15.29:0/901979262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b04066950 0x7f5b040be240 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f5afc009e60 tx=0x7f5afc009780).stop 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.640+0000 7f5b08a29700 1 -- 172.21.15.29:0/901979262 shutdown_connections 2024-01-24T15:30:21.643 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.640+0000 7f5b08a29700 1 --2- 172.21.15.29:0/901979262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b04066950 0x7f5b040be240 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.640+0000 7f5b08a29700 1 -- 172.21.15.29:0/901979262 >> 172.21.15.29:0/901979262 conn(0x7f5b040c8bd0 msgr2=0x7f5b040c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:21.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.640+0000 7f5b08a29700 1 -- 172.21.15.29:0/901979262 shutdown_connections 2024-01-24T15:30:21.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.640+0000 7f5b08a29700 1 -- 172.21.15.29:0/901979262 wait complete. 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.641+0000 7f5b08a29700 1 Processor -- start 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.641+0000 7f5b08a29700 1 -- start start 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.642+0000 7f5b08a29700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b040664c0 0x7f5b04166d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.642+0000 7f5b08a29700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b04066950 0x7f5b040a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.642+0000 7f5b08a29700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b0413a5e0 0x7f5b040a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.642+0000 7f5b08a29700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040a73b0 con 0x7f5b04066950 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.642+0000 7f5b08a29700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0413b950 con 0x7f5b0413a5e0 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.642+0000 7f5b08a29700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0413ad40 con 0x7f5b040664c0 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.642+0000 7f5b01d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b04066950 0x7f5b040a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.642+0000 7f5b01d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b04066950 0x7f5b040a69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40030/0 (socket says 172.21.15.29:40030) 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.642+0000 7f5b01d9b700 1 -- 172.21.15.29:0/260152553 learned_addr learned my addr 172.21.15.29:0/260152553 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.642+0000 7f5b0259c700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b040664c0 0x7f5b04166d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.642+0000 7f5b02d9d700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b0413a5e0 0x7f5b040a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.643+0000 7f5b01d9b700 1 -- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b040664c0 msgr2=0x7f5b04166d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.643+0000 7f5b01d9b700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b040664c0 0x7f5b04166d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.643+0000 7f5b01d9b700 1 -- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b0413a5e0 msgr2=0x7f5b040a6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.643+0000 7f5b01d9b700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b0413a5e0 0x7f5b040a6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.643+0000 7f5b01d9b700 1 -- 172.21.15.29:0/260152553 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc009a20 con 0x7f5b04066950 2024-01-24T15:30:21.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.643+0000 7f5b01d9b700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b04066950 0x7f5b040a69d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f5af800c560 tx=0x7f5af8007d00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:21.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.643+0000 7f5af27fc700 1 -- 172.21.15.29:0/260152553 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af800ce70 con 0x7f5b04066950 2024-01-24T15:30:21.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.643+0000 7f5af27fc700 1 -- 172.21.15.29:0/260152553 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af80072e0 con 0x7f5b04066950 2024-01-24T15:30:21.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.643+0000 7f5b08a29700 1 -- 172.21.15.29:0/260152553 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0413b020 con 0x7f5b04066950 2024-01-24T15:30:21.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.643+0000 7f5af27fc700 1 -- 172.21.15.29:0/260152553 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af8007660 con 0x7f5b04066950 2024-01-24T15:30:21.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.643+0000 7f5b08a29700 1 -- 172.21.15.29:0/260152553 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0413b5e0 con 0x7f5b04066950 2024-01-24T15:30:21.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.644+0000 7f5b08a29700 1 -- 172.21.15.29:0/260152553 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b040475e0 con 0x7f5b04066950 2024-01-24T15:30:21.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.645+0000 7f5af27fc700 1 -- 172.21.15.29:0/260152553 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 86) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f5af80059b0 con 0x7f5b04066950 2024-01-24T15:30:21.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.645+0000 7f5af27fc700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5ae405f270 0x7f5ae4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.645+0000 7f5af27fc700 1 -- 172.21.15.29:0/260152553 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 7730+0+0 (secure 0 0 0) 0x7f5af807a110 con 0x7f5b04066950 2024-01-24T15:30:21.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.645+0000 7f5b0259c700 1 -- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5ae405f270 msgr2=0x7f5ae4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:21.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.645+0000 7f5b0259c700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5ae405f270 0x7f5ae4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:21.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.650+0000 7f5af27fc700 1 -- 172.21.15.29:0/260152553 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5af804ac10 con 0x7f5b04066950 2024-01-24T15:30:21.785 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:21.783+0000 7f5fb946e680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:30:21.811 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:21.809+0000 7fae01b96680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:30:21.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.834+0000 7f5b08a29700 1 -- 172.21.15.29:0/260152553 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b0413ba90 con 0x7f5b04066950 2024-01-24T15:30:21.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.834+0000 7f5af27fc700 1 -- 172.21.15.29:0/260152553 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v198) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5af8018070 con 0x7f5b04066950 2024-01-24T15:30:21.837 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T15:30:21.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.836+0000 7f5b08a29700 1 -- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5ae405f270 msgr2=0x7f5ae4061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:21.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.836+0000 7f5b08a29700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5ae405f270 0x7f5ae4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.836+0000 7f5b08a29700 1 -- 172.21.15.29:0/260152553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b04066950 msgr2=0x7f5b040a69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.836+0000 7f5b08a29700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b04066950 0x7f5b040a69d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f5af800c560 tx=0x7f5af8007d00).stop 2024-01-24T15:30:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.836+0000 7f5b08a29700 1 -- 172.21.15.29:0/260152553 shutdown_connections 2024-01-24T15:30:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.836+0000 7f5b08a29700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b0413a5e0 0x7f5b040a6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.836+0000 7f5b08a29700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5ae405f270 0x7f5ae4061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:21.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.836+0000 7f5b08a29700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b040664c0 0x7f5b04166d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.836+0000 7f5b08a29700 1 --2- 172.21.15.29:0/260152553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b04066950 0x7f5b040a69d0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.836+0000 7f5b08a29700 1 -- 172.21.15.29:0/260152553 >> 172.21.15.29:0/260152553 conn(0x7f5b040c8bd0 msgr2=0x7f5b04139480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:21.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.836+0000 7f5b08a29700 1 -- 172.21.15.29:0/260152553 shutdown_connections 2024-01-24T15:30:21.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.836+0000 7f5b08a29700 1 -- 172.21.15.29:0/260152553 wait complete. 2024-01-24T15:30:21.853 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:30:21.902 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:21.900+0000 7fae01b96680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:30:21.978 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:21.976+0000 7f5fb946e680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:30:21.982 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:21.980+0000 7fae01b96680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:30:21.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.984+0000 7f88c9c08700 1 Processor -- start 2024-01-24T15:30:21.986 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.984+0000 7f88c9c08700 1 -- start start 2024-01-24T15:30:21.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.985+0000 7f88c9c08700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f88c4155b00 0x7f88c40664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.985+0000 7f88c9c08700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c4066e90 con 0x7f88c40669e0 2024-01-24T15:30:21.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.985+0000 7f88c9c08700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c40c3910 con 0x7f88c41565f0 2024-01-24T15:30:21.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.985+0000 7f88c9c08700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c40c5500 con 0x7f88c4155b00 2024-01-24T15:30:21.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.985+0000 7f88c37fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f88c4155b00 0x7f88c40664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.985+0000 7f88bbfff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f88c40669e0 0x7f88c4155ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38060/0 (socket says 172.21.15.29:38060) 2024-01-24T15:30:21.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.985+0000 7f88bbfff700 1 -- 172.21.15.29:0/1683614899 learned_addr learned my addr 172.21.15.29:0/1683614899 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:21.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.985+0000 7f88c37fe700 1 -- 172.21.15.29:0/1683614899 >> v1:172.21.15.163:6789/0 conn(0x7f88c41565f0 legacy=0x7f88c40bdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.985+0000 7f88c37fe700 1 -- 172.21.15.29:0/1683614899 >> v1:172.21.15.29:6789/0 conn(0x7f88c40669e0 legacy=0x7f88c4155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.985+0000 7f88c2ffd700 1 -- 172.21.15.29:0/1683614899 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2907428299 0 0) 0x7f88c40c3910 con 0x7f88c41565f0 2024-01-24T15:30:21.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.985+0000 7f88c37fe700 1 -- 172.21.15.29:0/1683614899 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c40c5220 con 0x7f88c4155b00 2024-01-24T15:30:21.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.985+0000 7f88c37fe700 1 --2- 172.21.15.29:0/1683614899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88c4155b00 0x7f88c40664f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f88b4009760 tx=0x7f88b4009060).ready entity=mon.0 client_cookie=ce3c5d7d014d612b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:21.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.986+0000 7f88c2ffd700 1 -- 172.21.15.29:0/1683614899 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400f510 con 0x7f88c4155b00 2024-01-24T15:30:21.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.986+0000 7f88c2ffd700 1 -- 172.21.15.29:0/1683614899 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b400f670 con 0x7f88c4155b00 2024-01-24T15:30:21.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.986+0000 7f88c2ffd700 1 -- 172.21.15.29:0/1683614899 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400f9f0 con 0x7f88c4155b00 2024-01-24T15:30:21.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.986+0000 7f88c9c08700 1 -- 172.21.15.29:0/1683614899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88c4155b00 msgr2=0x7f88c40664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.986+0000 7f88c9c08700 1 --2- 172.21.15.29:0/1683614899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88c4155b00 0x7f88c40664f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f88b4009760 tx=0x7f88b4009060).stop 2024-01-24T15:30:21.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.986+0000 7f88c9c08700 1 -- 172.21.15.29:0/1683614899 shutdown_connections 2024-01-24T15:30:21.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.986+0000 7f88c9c08700 1 --2- 172.21.15.29:0/1683614899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88c4155b00 0x7f88c40664f0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.986+0000 7f88c9c08700 1 -- 172.21.15.29:0/1683614899 >> 172.21.15.29:0/1683614899 conn(0x7f88c40c8bb0 msgr2=0x7f88c40c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:21.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.986+0000 7f88c9c08700 1 -- 172.21.15.29:0/1683614899 shutdown_connections 2024-01-24T15:30:21.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.986+0000 7f88c9c08700 1 -- 172.21.15.29:0/1683614899 wait complete. 2024-01-24T15:30:21.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.988+0000 7f88c9c08700 1 Processor -- start 2024-01-24T15:30:21.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.988+0000 7f88c9c08700 1 -- start start 2024-01-24T15:30:21.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.988+0000 7f88c9c08700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88c40669e0 0x7f88c40c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.988+0000 7f88c9c08700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c4155b00 0x7f88c414b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c9c08700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c41565f0 0x7f88c40c2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c9c08700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c40c1620 con 0x7f88c40669e0 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c9c08700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c40c0a10 con 0x7f88c41565f0 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c9c08700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c40c0cf0 con 0x7f88c4155b00 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88c40669e0 0x7f88c40c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88c40669e0 0x7f88c40c1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40062/0 (socket says 172.21.15.29:40062) 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c37fe700 1 -- 172.21.15.29:0/1407031517 learned_addr learned my addr 172.21.15.29:0/1407031517 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c3fff700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c41565f0 0x7f88c40c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88bbfff700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c4155b00 0x7f88c414b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c37fe700 1 -- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c4155b00 msgr2=0x7f88c414b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c37fe700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c4155b00 0x7f88c414b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c37fe700 1 -- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c41565f0 msgr2=0x7f88c40c2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c37fe700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c41565f0 0x7f88c40c2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c37fe700 1 -- 172.21.15.29:0/1407031517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b40092a0 con 0x7f88c40669e0 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88bbfff700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c4155b00 0x7f88c414b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c3fff700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c41565f0 0x7f88c40c2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:21.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c37fe700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88c40669e0 0x7f88c40c1ab0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7f88b4009eb0 tx=0x7f88b4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:21.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88bb7fe700 1 -- 172.21.15.29:0/1407031517 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400f9f0 con 0x7f88c40669e0 2024-01-24T15:30:21.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88bb7fe700 1 -- 172.21.15.29:0/1407031517 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b4000c90 con 0x7f88c40669e0 2024-01-24T15:30:21.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c9c08700 1 -- 172.21.15.29:0/1407031517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c40c0fd0 con 0x7f88c40669e0 2024-01-24T15:30:21.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88bb7fe700 1 -- 172.21.15.29:0/1407031517 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40246a0 con 0x7f88c40669e0 2024-01-24T15:30:21.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.989+0000 7f88c9c08700 1 -- 172.21.15.29:0/1407031517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c40c1fa0 con 0x7f88c40669e0 2024-01-24T15:30:21.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.991+0000 7f88c9c08700 1 -- 172.21.15.29:0/1407031517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c40c1900 con 0x7f88c40669e0 2024-01-24T15:30:21.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.991+0000 7f88bb7fe700 1 -- 172.21.15.29:0/1407031517 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 86) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f88b4021070 con 0x7f88c40669e0 2024-01-24T15:30:21.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.991+0000 7f88bb7fe700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f88a405f270 0x7f88a4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:21.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.992+0000 7f88bb7fe700 1 -- 172.21.15.29:0/1407031517 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 7730+0+0 (secure 0 0 0) 0x7f88b407c660 con 0x7f88c40669e0 2024-01-24T15:30:21.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.992+0000 7f88bbfff700 1 -- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f88a405f270 msgr2=0x7f88a4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:21.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.992+0000 7f88bbfff700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f88a405f270 0x7f88a4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:21.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:21.996+0000 7f88bb7fe700 1 -- 172.21.15.29:0/1407031517 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f88b404d160 con 0x7f88c40669e0 2024-01-24T15:30:22.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.193+0000 7f88bbfff700 1 -- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f88a405f270 msgr2=0x7f88a4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:22.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.193+0000 7f88bbfff700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f88a405f270 0x7f88a4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:30:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.257+0000 7f88c9c08700 1 -- 172.21.15.29:0/1407031517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88c40475e0 con 0x7f88c40669e0 2024-01-24T15:30:22.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.257+0000 7f88bb7fe700 1 -- 172.21.15.29:0/1407031517 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f88b4050780 con 0x7f88c40669e0 2024-01-24T15:30:22.260 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:22.260 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1350237946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1350237946},{"type":"v1","addr":"172.21.15.163:6836","nonce":1350237946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8656,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/250387328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":250387328},{"type":"v1","addr":"172.21.15.163:6837","nonce":250387328}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":9069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3867710046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3867710046},{"type":"v1","addr":"172.21.15.29:6833","nonce":3867710046}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:14:16.673720+0000","modified":"2024-01-24T15:14:19.686795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9072":{"gid":9072,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1929837915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1929837915},{"type":"v1","addr":"172.21.15.29:6835","nonce":1929837915}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-24T15:30:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.259+0000 7f88c9c08700 1 -- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f88a405f270 msgr2=0x7f88a4061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.259+0000 7f88c9c08700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f88a405f270 0x7f88a4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.259+0000 7f88c9c08700 1 -- 172.21.15.29:0/1407031517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88c40669e0 msgr2=0x7f88c40c1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.259+0000 7f88c9c08700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88c40669e0 0x7f88c40c1ab0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7f88b4009eb0 tx=0x7f88b4017000).stop 2024-01-24T15:30:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.259+0000 7f88c9c08700 1 -- 172.21.15.29:0/1407031517 shutdown_connections 2024-01-24T15:30:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.259+0000 7f88c9c08700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c41565f0 0x7f88c40c2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.259+0000 7f88c9c08700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88c40669e0 0x7f88c40c1ab0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.259+0000 7f88c9c08700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f88a405f270 0x7f88a4061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.259+0000 7f88c9c08700 1 --2- 172.21.15.29:0/1407031517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c4155b00 0x7f88c414b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.259+0000 7f88c9c08700 1 -- 172.21.15.29:0/1407031517 >> 172.21.15.29:0/1407031517 conn(0x7f88c40c8bb0 msgr2=0x7f88c40638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.259+0000 7f88c9c08700 1 -- 172.21.15.29:0/1407031517 shutdown_connections 2024-01-24T15:30:22.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.259+0000 7f88c9c08700 1 -- 172.21.15.29:0/1407031517 wait complete. 2024-01-24T15:30:22.263 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 82 2024-01-24T15:30:22.276 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:30:22.322 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:22.320+0000 7fae01b96680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:30:22.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.405+0000 7f26f9def700 1 Processor -- start 2024-01-24T15:30:22.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.405+0000 7f26f9def700 1 -- start start 2024-01-24T15:30:22.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.406+0000 7f26f9def700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26f4154370 0x7f26f40c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:22.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.406+0000 7f26f9def700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f26f40aa0e0 0x7f26f4154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:22.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.406+0000 7f26f9def700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f40aa6c0 con 0x7f26f40ab1f0 2024-01-24T15:30:22.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.406+0000 7f26f9def700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f4064240 con 0x7f26f40aa0e0 2024-01-24T15:30:22.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.406+0000 7f26f9def700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40cb5c0 con 0x7f26f4154370 2024-01-24T15:30:22.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.407+0000 7f26f2ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f26f40aa0e0 0x7f26f4154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:22.410 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.407+0000 7f26f2ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f26f40aa0e0 0x7f26f4154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40074/0 (socket says 172.21.15.29:40074) 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.407+0000 7f26f2ffd700 1 -- 172.21.15.29:0/2244283235 learned_addr learned my addr 172.21.15.29:0/2244283235 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.407+0000 7f26f37fe700 1 --2- 172.21.15.29:0/2244283235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26f4154370 0x7f26f40c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.407+0000 7f26f2ffd700 1 -- 172.21.15.29:0/2244283235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26f4154370 msgr2=0x7f26f40c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.407+0000 7f26f2ffd700 1 --2- 172.21.15.29:0/2244283235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26f4154370 0x7f26f40c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.407+0000 7f26f2ffd700 1 -- 172.21.15.29:0/2244283235 >> v1:172.21.15.163:6789/0 conn(0x7f26f40ab1f0 legacy=0x7f26f40631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.407+0000 7f26f2ffd700 1 -- 172.21.15.29:0/2244283235 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f40cb8a0 con 0x7f26f40aa0e0 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.407+0000 7f26f2ffd700 1 --2- 172.21.15.29:0/2244283235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26f40aa0e0 0x7f26f4154750 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7f26dc006fd0 tx=0x7f26dc0090f0).ready entity=mon.0 client_cookie=9036f4004ec1291e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.407+0000 7f26f27fc700 1 -- 172.21.15.29:0/2244283235 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc00b580 con 0x7f26f40aa0e0 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.407+0000 7f26f27fc700 1 -- 172.21.15.29:0/2244283235 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc00b6e0 con 0x7f26f40aa0e0 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.407+0000 7f26f27fc700 1 -- 172.21.15.29:0/2244283235 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc00ba60 con 0x7f26f40aa0e0 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.408+0000 7f26f9def700 1 -- 172.21.15.29:0/2244283235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26f40aa0e0 msgr2=0x7f26f4154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.408+0000 7f26f9def700 1 --2- 172.21.15.29:0/2244283235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26f40aa0e0 0x7f26f4154750 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7f26dc006fd0 tx=0x7f26dc0090f0).stop 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.408+0000 7f26f9def700 1 -- 172.21.15.29:0/2244283235 shutdown_connections 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.408+0000 7f26f9def700 1 --2- 172.21.15.29:0/2244283235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26f4154370 0x7f26f40c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.408+0000 7f26f9def700 1 --2- 172.21.15.29:0/2244283235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26f40aa0e0 0x7f26f4154750 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.408+0000 7f26f9def700 1 -- 172.21.15.29:0/2244283235 >> 172.21.15.29:0/2244283235 conn(0x7f26f405ec80 msgr2=0x7f26f40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:22.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.408+0000 7f26f9def700 1 -- 172.21.15.29:0/2244283235 shutdown_connections 2024-01-24T15:30:22.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.408+0000 7f26f9def700 1 -- 172.21.15.29:0/2244283235 wait complete. 2024-01-24T15:30:22.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.409+0000 7f26f9def700 1 Processor -- start 2024-01-24T15:30:22.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.410+0000 7f26f9def700 1 -- start start 2024-01-24T15:30:22.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.410+0000 7f26f9def700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26f40aa0e0 0x7f26f4159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:22.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.410+0000 7f26f9def700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26f40ab1f0 0x7f26f4168630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:22.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.410+0000 7f26f9def700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26f4154370 0x7f26f4152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:22.414 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.410+0000 7f26f9def700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4165020 con 0x7f26f4154370 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.410+0000 7f26f9def700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4150450 con 0x7f26f40ab1f0 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.410+0000 7f26f9def700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4165180 con 0x7f26f40aa0e0 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.410+0000 7f26f3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26f4154370 0x7f26f4152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26f4154370 0x7f26f4152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40078/0 (socket says 172.21.15.29:40078) 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f3fff700 1 -- 172.21.15.29:0/3842710082 learned_addr learned my addr 172.21.15.29:0/3842710082 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f37fe700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26f40aa0e0 0x7f26f4159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f2ffd700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26f40ab1f0 0x7f26f4168630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f3fff700 1 -- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26f40aa0e0 msgr2=0x7f26f4159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f3fff700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26f40aa0e0 0x7f26f4159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f3fff700 1 -- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26f40ab1f0 msgr2=0x7f26f4168630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f3fff700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26f40ab1f0 0x7f26f4168630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f3fff700 1 -- 172.21.15.29:0/3842710082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc009370 con 0x7f26f4154370 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f37fe700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26f40aa0e0 0x7f26f4159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:22.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f3fff700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26f4154370 0x7f26f4152e90 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7f26ec00c440 tx=0x7f26ec007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:22.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26eb7fe700 1 -- 172.21.15.29:0/3842710082 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0123f0 con 0x7f26f4154370 2024-01-24T15:30:22.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26eb7fe700 1 -- 172.21.15.29:0/3842710082 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec001e90 con 0x7f26f4154370 2024-01-24T15:30:22.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f9def700 1 -- 172.21.15.29:0/3842710082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f41571c0 con 0x7f26f4154370 2024-01-24T15:30:22.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26eb7fe700 1 -- 172.21.15.29:0/3842710082 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec012760 con 0x7f26f4154370 2024-01-24T15:30:22.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.411+0000 7f26f9def700 1 -- 172.21.15.29:0/3842710082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f4157780 con 0x7f26f4154370 2024-01-24T15:30:22.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.413+0000 7f26f9def700 1 -- 172.21.15.29:0/3842710082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f40caad0 con 0x7f26f4154370 2024-01-24T15:30:22.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.413+0000 7f26eb7fe700 1 -- 172.21.15.29:0/3842710082 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 86) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f26ec012b50 con 0x7f26f4154370 2024-01-24T15:30:22.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.413+0000 7f26eb7fe700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f26d405f240 0x7f26d4061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:22.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.413+0000 7f26eb7fe700 1 -- 172.21.15.29:0/3842710082 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 7730+0+0 (secure 0 0 0) 0x7f26ec079d00 con 0x7f26f4154370 2024-01-24T15:30:22.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.413+0000 7f26f37fe700 1 -- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f26d405f240 msgr2=0x7f26d4061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:22.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.413+0000 7f26f37fe700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f26d405f240 0x7f26d4061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:22.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.418+0000 7f26eb7fe700 1 -- 172.21.15.29:0/3842710082 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f26ec047b00 con 0x7f26f4154370 2024-01-24T15:30:22.539 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:22.537+0000 7fae01b96680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:30:22.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.602+0000 7f26f9def700 1 -- 172.21.15.29:0/3842710082 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26f40475e0 con 0x7f26f4154370 2024-01-24T15:30:22.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.603+0000 7f26eb7fe700 1 -- 172.21.15.29:0/3842710082 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+19094 (secure 0 0 0) 0x7f26ec04a800 con 0x7f26f4154370 2024-01-24T15:30:22.605 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:22.605 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":198,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:30:21.293990+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:14:14.419558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-24T15:14:15.659901+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":198,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.5","mappings":[{"from":3,"to":0}]},{"pgid":"22.13","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/2002665960":"2024-01-25T15:25:20.634238+0000","172.21.15.163:0/3305307869":"2024-01-25T15:25:20.634238+0000","172.21.15.163:0/4000114899":"2024-01-25T15:25:20.634238+0000","172.21.15.163:6832/43241":"2024-01-25T15:14:28.971452+0000","172.21.15.163:0/2434196663":"2024-01-25T15:14:28.971452+0000","172.21.15.163:6833/43241":"2024-01-25T15:14:28.971452+0000","172.21.15.163:0/4284075186":"2024-01-25T15:14:28.971452+0000","172.21.15.163:0/3544734349":"2024-01-25T15:25:20.634238+0000","172.21.15.163:0/2471445585":"2024-01-25T15:14:28.971452+0000","172.21.15.163:0/2024052538":"2024-01-25T15:14:28.971452+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:30:22.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.605+0000 7f26f9def700 1 -- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f26d405f240 msgr2=0x7f26d4061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:22.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.605+0000 7f26f9def700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f26d405f240 0x7f26d4061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:22.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.605+0000 7f26f9def700 1 -- 172.21.15.29:0/3842710082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26f4154370 msgr2=0x7f26f4152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:22.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.605+0000 7f26f9def700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26f4154370 0x7f26f4152e90 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7f26ec00c440 tx=0x7f26ec007d70).stop 2024-01-24T15:30:22.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.605+0000 7f26f9def700 1 -- 172.21.15.29:0/3842710082 shutdown_connections 2024-01-24T15:30:22.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.605+0000 7f26f9def700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26f4154370 0x7f26f4152e90 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.605+0000 7f26f9def700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f26d405f240 0x7f26d4061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:22.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.605+0000 7f26f9def700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26f40aa0e0 0x7f26f4159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.605+0000 7f26f9def700 1 --2- 172.21.15.29:0/3842710082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26f40ab1f0 0x7f26f4168630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.605+0000 7f26f9def700 1 -- 172.21.15.29:0/3842710082 >> 172.21.15.29:0/3842710082 conn(0x7f26f405ec80 msgr2=0x7f26f4151890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:22.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.605+0000 7f26f9def700 1 -- 172.21.15.29:0/3842710082 shutdown_connections 2024-01-24T15:30:22.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.605+0000 7f26f9def700 1 -- 172.21.15.29:0/3842710082 wait complete. 2024-01-24T15:30:22.621 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T15:30:22.622 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:30:22.657 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:22.655+0000 7f5fb946e680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:30:22.726 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:22.723+0000 7f5fb946e680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:30:22.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.751+0000 7f7652468700 1 Processor -- start 2024-01-24T15:30:22.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.751+0000 7f7652468700 1 -- start start 2024-01-24T15:30:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.752+0000 7f7652468700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f764c155040 0x7f764c069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.752+0000 7f7652468700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c0c2ce0 con 0x7f764c065ec0 2024-01-24T15:30:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.752+0000 7f7652468700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c069890 con 0x7f764c150d40 2024-01-24T15:30:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.752+0000 7f7652468700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764c069b70 con 0x7f764c155040 2024-01-24T15:30:22.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.752+0000 7f764b7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f764c155040 0x7f764c069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.752+0000 7f7650a05700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f764c065ec0 0x7f764c0c3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:38062/0 (socket says 172.21.15.29:38062) 2024-01-24T15:30:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.752+0000 7f7650a05700 1 -- 172.21.15.29:0/2711126319 learned_addr learned my addr 172.21.15.29:0/2711126319 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.753+0000 7f764b7fe700 1 -- 172.21.15.29:0/2711126319 >> v1:172.21.15.163:6789/0 conn(0x7f764c150d40 legacy=0x7f764c154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.753+0000 7f764affd700 1 -- 172.21.15.29:0/2711126319 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417021097 0 0) 0x7f764c069890 con 0x7f764c150d40 2024-01-24T15:30:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.753+0000 7f764b7fe700 1 -- 172.21.15.29:0/2711126319 >> v1:172.21.15.29:6789/0 conn(0x7f764c065ec0 legacy=0x7f764c0c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.753+0000 7f764b7fe700 1 -- 172.21.15.29:0/2711126319 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c069e50 con 0x7f764c155040 2024-01-24T15:30:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.753+0000 7f764b7fe700 1 --2- 172.21.15.29:0/2711126319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f764c155040 0x7f764c069240 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f7640009760 tx=0x7f7640009060).ready entity=mon.0 client_cookie=556fb9c25e5b97f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.753+0000 7f764affd700 1 -- 172.21.15.29:0/2711126319 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764000f510 con 0x7f764c155040 2024-01-24T15:30:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.753+0000 7f764affd700 1 -- 172.21.15.29:0/2711126319 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764000f670 con 0x7f764c155040 2024-01-24T15:30:22.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.753+0000 7f764affd700 1 -- 172.21.15.29:0/2711126319 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764000f9f0 con 0x7f764c155040 2024-01-24T15:30:22.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.754+0000 7f7652468700 1 -- 172.21.15.29:0/2711126319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f764c155040 msgr2=0x7f764c069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:22.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.754+0000 7f7652468700 1 --2- 172.21.15.29:0/2711126319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f764c155040 0x7f764c069240 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f7640009760 tx=0x7f7640009060).stop 2024-01-24T15:30:22.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.754+0000 7f7652468700 1 -- 172.21.15.29:0/2711126319 shutdown_connections 2024-01-24T15:30:22.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.754+0000 7f7652468700 1 --2- 172.21.15.29:0/2711126319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f764c155040 0x7f764c069240 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.754+0000 7f7652468700 1 -- 172.21.15.29:0/2711126319 >> 172.21.15.29:0/2711126319 conn(0x7f764c0c8580 msgr2=0x7f764c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:22.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.754+0000 7f7652468700 1 -- 172.21.15.29:0/2711126319 shutdown_connections 2024-01-24T15:30:22.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.754+0000 7f7652468700 1 -- 172.21.15.29:0/2711126319 wait complete. 2024-01-24T15:30:22.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.755+0000 7f7652468700 1 Processor -- start 2024-01-24T15:30:22.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.755+0000 7f7652468700 1 -- start start 2024-01-24T15:30:22.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.756+0000 7f7652468700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f764c065ec0 0x7f764c14f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:22.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.756+0000 7f7652468700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f764c150d40 0x7f764c14a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:22.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.756+0000 7f7652468700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c155040 0x7f764c14c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:22.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.756+0000 7f7652468700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c14ac90 con 0x7f764c150d40 2024-01-24T15:30:22.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.756+0000 7f7652468700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0bff30 con 0x7f764c155040 2024-01-24T15:30:22.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.756+0000 7f7652468700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0bfc00 con 0x7f764c065ec0 2024-01-24T15:30:22.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.756+0000 7f764b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f764c150d40 0x7f764c14a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:22.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.756+0000 7f764b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f764c150d40 0x7f764c14a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40098/0 (socket says 172.21.15.29:40098) 2024-01-24T15:30:22.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.756+0000 7f764b7fe700 1 -- 172.21.15.29:0/2723479112 learned_addr learned my addr 172.21.15.29:0/2723479112 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:22.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.756+0000 7f7650a05700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c155040 0x7f764c14c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:22.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.756+0000 7f764bfff700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f764c065ec0 0x7f764c14f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:22.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.757+0000 7f764b7fe700 1 -- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f764c065ec0 msgr2=0x7f764c14f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.757+0000 7f764b7fe700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f764c065ec0 0x7f764c14f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.757+0000 7f764b7fe700 1 -- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c155040 msgr2=0x7f764c14c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.757+0000 7f764b7fe700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c155040 0x7f764c14c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.757+0000 7f764b7fe700 1 -- 172.21.15.29:0/2723479112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400092a0 con 0x7f764c150d40 2024-01-24T15:30:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.757+0000 7f764b7fe700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f764c150d40 0x7f764c14a7a0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f764c069030 tx=0x7f7640017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.757+0000 7f7633fff700 1 -- 172.21.15.29:0/2723479112 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764000f7e0 con 0x7f764c150d40 2024-01-24T15:30:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.757+0000 7f7633fff700 1 -- 172.21.15.29:0/2723479112 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764000f940 con 0x7f764c150d40 2024-01-24T15:30:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.757+0000 7f7652468700 1 -- 172.21.15.29:0/2723479112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c14cee0 con 0x7f764c150d40 2024-01-24T15:30:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.757+0000 7f7633fff700 1 -- 172.21.15.29:0/2723479112 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764001b640 con 0x7f764c150d40 2024-01-24T15:30:22.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.757+0000 7f7652468700 1 -- 172.21.15.29:0/2723479112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c14d490 con 0x7f764c150d40 2024-01-24T15:30:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.758+0000 7f7652468700 1 -- 172.21.15.29:0/2723479112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c0c00c0 con 0x7f764c150d40 2024-01-24T15:30:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.759+0000 7f7633fff700 1 -- 172.21.15.29:0/2723479112 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 86) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f764001b9b0 con 0x7f764c150d40 2024-01-24T15:30:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.759+0000 7f7633fff700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f763805f270 0x7f7638061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.759+0000 7f764bfff700 1 -- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f763805f270 msgr2=0x7f7638061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.759+0000 7f7633fff700 1 -- 172.21.15.29:0/2723479112 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 7730+0+0 (secure 0 0 0) 0x7f764007c240 con 0x7f764c150d40 2024-01-24T15:30:22.762 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.759+0000 7f764bfff700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f763805f270 0x7f7638061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:22.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.764+0000 7f7633fff700 1 -- 172.21.15.29:0/2723479112 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f764004ccc0 con 0x7f764c150d40 2024-01-24T15:30:22.802 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:22.799+0000 7f5fb946e680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:30:22.949 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:22.947+0000 7f5fb946e680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:30:22.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.960+0000 7f764bfff700 1 -- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f763805f270 msgr2=0x7f7638061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:22.962 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.960+0000 7f764bfff700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f763805f270 0x7f7638061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:30:22.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.992+0000 7f7652468700 1 -- 172.21.15.29:0/2723479112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f764c0475e0 con 0x7f764c150d40 2024-01-24T15:30:22.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.992+0000 7f7633fff700 1 -- 172.21.15.29:0/2723479112 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f7640029090 con 0x7f764c150d40 2024-01-24T15:30:22.995 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:22.995 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1350237946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1350237946},{"type":"v1","addr":"172.21.15.163:6836","nonce":1350237946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8656,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/250387328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":250387328},{"type":"v1","addr":"172.21.15.163:6837","nonce":250387328}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":9069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3867710046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3867710046},{"type":"v1","addr":"172.21.15.29:6833","nonce":3867710046}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:14:16.673720+0000","modified":"2024-01-24T15:14:19.686795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9072":{"gid":9072,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1929837915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1929837915},{"type":"v1","addr":"172.21.15.29:6835","nonce":1929837915}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-24T15:30:22.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.995+0000 7f7652468700 1 -- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f763805f270 msgr2=0x7f7638061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:22.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.995+0000 7f7652468700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f763805f270 0x7f7638061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:22.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.995+0000 7f7652468700 1 -- 172.21.15.29:0/2723479112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f764c150d40 msgr2=0x7f764c14a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:22.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.995+0000 7f7652468700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f764c150d40 0x7f764c14a7a0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f764c069030 tx=0x7f7640017000).stop 2024-01-24T15:30:22.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.995+0000 7f7652468700 1 -- 172.21.15.29:0/2723479112 shutdown_connections 2024-01-24T15:30:22.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.995+0000 7f7652468700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c155040 0x7f764c14c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.995+0000 7f7652468700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f763805f270 0x7f7638061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:22.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.995+0000 7f7652468700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f764c065ec0 0x7f764c14f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.995+0000 7f7652468700 1 --2- 172.21.15.29:0/2723479112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f764c150d40 0x7f764c14a7a0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:22.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.995+0000 7f7652468700 1 -- 172.21.15.29:0/2723479112 >> 172.21.15.29:0/2723479112 conn(0x7f764c0c8580 msgr2=0x7f764c0c6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:22.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.995+0000 7f7652468700 1 -- 172.21.15.29:0/2723479112 shutdown_connections 2024-01-24T15:30:22.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:22.995+0000 7f7652468700 1 -- 172.21.15.29:0/2723479112 wait complete. 2024-01-24T15:30:22.998 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 82 2024-01-24T15:30:23.011 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:30:23.020 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.018+0000 7f5fb946e680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:30:23.135 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.133+0000 7f5fb946e680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:30:23.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.142+0000 7f57ec336700 1 Processor -- start 2024-01-24T15:30:23.144 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.142+0000 7f57ec336700 1 -- start start 2024-01-24T15:30:23.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.143+0000 7f57ec336700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57e4069240 0x7f57e4154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.143+0000 7f57ec336700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e4154e40 0x7f57e40bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.143+0000 7f57ec336700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e406a3a0 con 0x7f57e4150cb0 2024-01-24T15:30:23.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.143+0000 7f57ec336700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e40c2c70 con 0x7f57e4069240 2024-01-24T15:30:23.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.143+0000 7f57ec336700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e41552d0 con 0x7f57e4154e40 2024-01-24T15:30:23.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57e98d1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57e4069240 0x7f57e4154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57e98d1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57e4069240 0x7f57e4154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40102/0 (socket says 172.21.15.29:40102) 2024-01-24T15:30:23.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57e98d1700 1 -- 172.21.15.29:0/2783944951 learned_addr learned my addr 172.21.15.29:0/2783944951 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:23.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57ea8d3700 1 --2- 172.21.15.29:0/2783944951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e4154e40 0x7f57e40bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57e98d1700 1 -- 172.21.15.29:0/2783944951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e4154e40 msgr2=0x7f57e40bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:23.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57e98d1700 1 --2- 172.21.15.29:0/2783944951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e4154e40 0x7f57e40bcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57e90d0700 1 -- 172.21.15.29:0/2783944951 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332578358 0 0) 0x7f57e406a3a0 con 0x7f57e4150cb0 2024-01-24T15:30:23.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57e98d1700 1 -- 172.21.15.29:0/2783944951 >> v1:172.21.15.163:6789/0 conn(0x7f57e4150cb0 legacy=0x7f57e4065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57e98d1700 1 -- 172.21.15.29:0/2783944951 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e4069820 con 0x7f57e4069240 2024-01-24T15:30:23.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57ea8d3700 1 --2- 172.21.15.29:0/2783944951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e4154e40 0x7f57e40bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:23.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57e98d1700 1 --2- 172.21.15.29:0/2783944951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e4069240 0x7f57e4154950 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7f57d4006fd0 tx=0x7f57d40090f0).ready entity=mon.0 client_cookie=66b002fa40687bd7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:23.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57e90d0700 1 -- 172.21.15.29:0/2783944951 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d400b580 con 0x7f57e4069240 2024-01-24T15:30:23.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.144+0000 7f57e90d0700 1 -- 172.21.15.29:0/2783944951 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d400b6e0 con 0x7f57e4069240 2024-01-24T15:30:23.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.145+0000 7f57e90d0700 1 -- 172.21.15.29:0/2783944951 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d400ba60 con 0x7f57e4069240 2024-01-24T15:30:23.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.145+0000 7f57ec336700 1 -- 172.21.15.29:0/2783944951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e4069240 msgr2=0x7f57e4154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.145+0000 7f57ec336700 1 --2- 172.21.15.29:0/2783944951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e4069240 0x7f57e4154950 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7f57d4006fd0 tx=0x7f57d40090f0).stop 2024-01-24T15:30:23.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.145+0000 7f57ec336700 1 -- 172.21.15.29:0/2783944951 shutdown_connections 2024-01-24T15:30:23.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.145+0000 7f57ec336700 1 --2- 172.21.15.29:0/2783944951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e4154e40 0x7f57e40bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.145+0000 7f57ec336700 1 --2- 172.21.15.29:0/2783944951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e4069240 0x7f57e4154950 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.145+0000 7f57ec336700 1 -- 172.21.15.29:0/2783944951 >> 172.21.15.29:0/2783944951 conn(0x7f57e40c85c0 msgr2=0x7f57e40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:23.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.145+0000 7f57ec336700 1 -- 172.21.15.29:0/2783944951 shutdown_connections 2024-01-24T15:30:23.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.145+0000 7f57ec336700 1 -- 172.21.15.29:0/2783944951 wait complete. 2024-01-24T15:30:23.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.147+0000 7f57ec336700 1 Processor -- start 2024-01-24T15:30:23.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.147+0000 7f57ec336700 1 -- start start 2024-01-24T15:30:23.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.147+0000 7f57ec336700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e4069240 0x7f57e4063af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.147+0000 7f57ec336700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e4150cb0 0x7f57e414c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57ec336700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e4154e40 0x7f57e414c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57ec336700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e4152530 con 0x7f57e4150cb0 2024-01-24T15:30:23.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57ec336700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e4153320 con 0x7f57e4069240 2024-01-24T15:30:23.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57ec336700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e414caf0 con 0x7f57e4154e40 2024-01-24T15:30:23.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57e98d1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e4150cb0 0x7f57e414c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57e98d1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e4150cb0 0x7f57e414c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:40118/0 (socket says 172.21.15.29:40118) 2024-01-24T15:30:23.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57e98d1700 1 -- 172.21.15.29:0/1418022638 learned_addr learned my addr 172.21.15.29:0/1418022638 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:23.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57ea8d3700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e4154e40 0x7f57e414c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57ea0d2700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e4069240 0x7f57e4063af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57e98d1700 1 -- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e4154e40 msgr2=0x7f57e414c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57e98d1700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e4154e40 0x7f57e414c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57e98d1700 1 -- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e4069240 msgr2=0x7f57e4063af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57e98d1700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e4069240 0x7f57e4063af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57e98d1700 1 -- 172.21.15.29:0/1418022638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc009a20 con 0x7f57e4150cb0 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57ea8d3700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e4154e40 0x7f57e414c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57e98d1700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e4150cb0 0x7f57e414c110 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7f57d4009130 tx=0x7f57d4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57da7fc700 1 -- 172.21.15.29:0/1418022638 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4000bc0 con 0x7f57e4150cb0 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.148+0000 7f57da7fc700 1 -- 172.21.15.29:0/1418022638 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d400b850 con 0x7f57e4150cb0 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.149+0000 7f57ec336700 1 -- 172.21.15.29:0/1418022638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d4009370 con 0x7f57e4150cb0 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.149+0000 7f57da7fc700 1 -- 172.21.15.29:0/1418022638 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4000d20 con 0x7f57e4150cb0 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.149+0000 7f57ec336700 1 -- 172.21.15.29:0/1418022638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e41529f0 con 0x7f57e4150cb0 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.150+0000 7f57ec336700 1 -- 172.21.15.29:0/1418022638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e40475e0 con 0x7f57e4150cb0 2024-01-24T15:30:23.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.150+0000 7f57da7fc700 1 -- 172.21.15.29:0/1418022638 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 86) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f57d401b890 con 0x7f57e4150cb0 2024-01-24T15:30:23.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.150+0000 7f57da7fc700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f57c805f240 0x7f57c8061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.151+0000 7f57da7fc700 1 -- 172.21.15.29:0/1418022638 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 7730+0+0 (secure 0 0 0) 0x7f57d407c2a0 con 0x7f57e4150cb0 2024-01-24T15:30:23.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.151+0000 7f57ea0d2700 1 -- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f57c805f240 msgr2=0x7f57c8061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:23.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.151+0000 7f57ea0d2700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f57c805f240 0x7f57c8061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:23.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.155+0000 7f57da7fc700 1 -- 172.21.15.29:0/1418022638 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f57d404cda0 con 0x7f57e4150cb0 2024-01-24T15:30:23.244 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.242+0000 7f5fb946e680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:30:23.314 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:23.312+0000 7fae01b96680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:30:23.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.352+0000 7f57ea0d2700 1 -- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f57c805f240 msgr2=0x7f57c8061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:23.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.352+0000 7f57ea0d2700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f57c805f240 0x7f57c8061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:30:23.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.383+0000 7f57ec336700 1 -- 172.21.15.29:0/1418022638 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57e406a5f0 con 0x7f57e4150cb0 2024-01-24T15:30:23.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.384+0000 7f57da7fc700 1 -- 172.21.15.29:0/1418022638 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f57d4006b30 con 0x7f57e4150cb0 2024-01-24T15:30:23.386 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:23.386 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1350237946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1350237946},{"type":"v1","addr":"172.21.15.163:6836","nonce":1350237946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8656,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/250387328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":250387328},{"type":"v1","addr":"172.21.15.163:6837","nonce":250387328}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":9069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3867710046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3867710046},{"type":"v1","addr":"172.21.15.29:6833","nonce":3867710046}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:14:16.673720+0000","modified":"2024-01-24T15:14:19.686795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9072":{"gid":9072,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1929837915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1929837915},{"type":"v1","addr":"172.21.15.29:6835","nonce":1929837915}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-24T15:30:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.386+0000 7f57ec336700 1 -- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f57c805f240 msgr2=0x7f57c8061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.386+0000 7f57ec336700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f57c805f240 0x7f57c8061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.386+0000 7f57ec336700 1 -- 172.21.15.29:0/1418022638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e4150cb0 msgr2=0x7f57e414c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.386+0000 7f57ec336700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e4150cb0 0x7f57e414c110 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7f57d4009130 tx=0x7f57d4017000).stop 2024-01-24T15:30:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.386+0000 7f57ec336700 1 -- 172.21.15.29:0/1418022638 shutdown_connections 2024-01-24T15:30:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.386+0000 7f57ec336700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f57c805f240 0x7f57c8061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.386+0000 7f57ec336700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e4154e40 0x7f57e414c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.386+0000 7f57ec336700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e4069240 0x7f57e4063af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.386+0000 7f57ec336700 1 --2- 172.21.15.29:0/1418022638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57e4150cb0 0x7f57e414c110 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.386+0000 7f57ec336700 1 -- 172.21.15.29:0/1418022638 >> 172.21.15.29:0/1418022638 conn(0x7f57e40c85c0 msgr2=0x7f57e40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.386+0000 7f57ec336700 1 -- 172.21.15.29:0/1418022638 shutdown_connections 2024-01-24T15:30:23.389 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.386+0000 7f57ec336700 1 -- 172.21.15.29:0/1418022638 wait complete. 2024-01-24T15:30:23.390 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 82 2024-01-24T15:30:23.395 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:23.393+0000 7fae01b96680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:30:23.403 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:30:23.485 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:23.483+0000 7fae01b96680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:30:23.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.531+0000 7f1f84e02700 1 Processor -- start 2024-01-24T15:30:23.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.531+0000 7f1f84e02700 1 -- start start 2024-01-24T15:30:23.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.533+0000 7f1f84e02700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1f80154370 0x7f1f800c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.533+0000 7f1f84e02700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f800aa6c0 con 0x7f1f800aa0e0 2024-01-24T15:30:23.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.533+0000 7f1f84e02700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f80064240 con 0x7f1f80154750 2024-01-24T15:30:23.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.533+0000 7f1f84e02700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f80154ba0 con 0x7f1f80154370 2024-01-24T15:30:23.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.533+0000 7f1f7e59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1f80154370 0x7f1f800c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.533+0000 7f1f7dd9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1f800aa0e0 0x7f1f800ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46172/0 (socket says 172.21.15.29:46172) 2024-01-24T15:30:23.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.533+0000 7f1f7dd9b700 1 -- 172.21.15.29:0/3019343386 learned_addr learned my addr 172.21.15.29:0/3019343386 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:23.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.533+0000 7f1f7e59c700 1 -- 172.21.15.29:0/3019343386 >> v1:172.21.15.163:6789/0 conn(0x7f1f80154750 legacy=0x7f1f800631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.533+0000 7f1f7e59c700 1 -- 172.21.15.29:0/3019343386 >> v1:172.21.15.29:6789/0 conn(0x7f1f800aa0e0 legacy=0x7f1f800ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.533+0000 7f1f7e59c700 1 -- 172.21.15.29:0/3019343386 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f800cb5c0 con 0x7f1f80154370 2024-01-24T15:30:23.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.533+0000 7f1f7d59a700 1 -- 172.21.15.29:0/3019343386 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589632891 0 0) 0x7f1f800aa6c0 con 0x7f1f800aa0e0 2024-01-24T15:30:23.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.534+0000 7f1f7e59c700 1 --2- 172.21.15.29:0/3019343386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80154370 0x7f1f800c5410 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7f1f70006fd0 tx=0x7f1f70006f40).ready entity=mon.0 client_cookie=8380f3a0fff5ed9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:23.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.534+0000 7f1f7d59a700 1 -- 172.21.15.29:0/3019343386 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70005880 con 0x7f1f80154370 2024-01-24T15:30:23.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.534+0000 7f1f7d59a700 1 -- 172.21.15.29:0/3019343386 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f700059e0 con 0x7f1f80154370 2024-01-24T15:30:23.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.534+0000 7f1f7d59a700 1 -- 172.21.15.29:0/3019343386 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70005d60 con 0x7f1f80154370 2024-01-24T15:30:23.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.534+0000 7f1f84e02700 1 -- 172.21.15.29:0/3019343386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80154370 msgr2=0x7f1f800c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.534+0000 7f1f84e02700 1 --2- 172.21.15.29:0/3019343386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80154370 0x7f1f800c5410 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7f1f70006fd0 tx=0x7f1f70006f40).stop 2024-01-24T15:30:23.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.535+0000 7f1f84e02700 1 -- 172.21.15.29:0/3019343386 shutdown_connections 2024-01-24T15:30:23.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.535+0000 7f1f84e02700 1 --2- 172.21.15.29:0/3019343386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80154370 0x7f1f800c5410 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.535+0000 7f1f84e02700 1 -- 172.21.15.29:0/3019343386 >> 172.21.15.29:0/3019343386 conn(0x7f1f8005ec80 msgr2=0x7f1f800cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:23.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.535+0000 7f1f84e02700 1 -- 172.21.15.29:0/3019343386 shutdown_connections 2024-01-24T15:30:23.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.535+0000 7f1f84e02700 1 -- 172.21.15.29:0/3019343386 wait complete. 2024-01-24T15:30:23.539 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.533+0000 7f5fb946e680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:30:23.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.536+0000 7f1f84e02700 1 Processor -- start 2024-01-24T15:30:23.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.536+0000 7f1f84e02700 1 -- start start 2024-01-24T15:30:23.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f84e02700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f800aa0e0 0x7f1f801681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f84e02700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80154370 0x7f1f800c2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f84e02700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f80154750 0x7f1f800c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f84e02700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f80169110 con 0x7f1f80154370 2024-01-24T15:30:23.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f84e02700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f80168870 con 0x7f1f800aa0e0 2024-01-24T15:30:23.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f84e02700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f80168b50 con 0x7f1f80154750 2024-01-24T15:30:23.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f7dd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80154370 0x7f1f800c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f7dd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80154370 0x7f1f800c2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52568/0 (socket says 172.21.15.29:52568) 2024-01-24T15:30:23.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f7dd9b700 1 -- 172.21.15.29:0/3926790310 learned_addr learned my addr 172.21.15.29:0/3926790310 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f7e59c700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f800aa0e0 0x7f1f801681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f7ed9d700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f80154750 0x7f1f800c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f7dd9b700 1 -- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f80154750 msgr2=0x7f1f800c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f7dd9b700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f80154750 0x7f1f800c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f7dd9b700 1 -- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f800aa0e0 msgr2=0x7f1f801681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f7dd9b700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f800aa0e0 0x7f1f801681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f7dd9b700 1 -- 172.21.15.29:0/3926790310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f70009250 con 0x7f1f80154370 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f7dd9b700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80154370 0x7f1f800c2830 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7f1f7400c4c0 tx=0x7f1f74007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.537+0000 7f1f6e7fc700 1 -- 172.21.15.29:0/3926790310 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400ce30 con 0x7f1f80154370 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.538+0000 7f1f6e7fc700 1 -- 172.21.15.29:0/3926790310 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f740072e0 con 0x7f1f80154370 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.538+0000 7f1f6e7fc700 1 -- 172.21.15.29:0/3926790310 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f740057a0 con 0x7f1f80154370 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.538+0000 7f1f84e02700 1 -- 172.21.15.29:0/3926790310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f80168e30 con 0x7f1f80154370 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.538+0000 7f1f84e02700 1 -- 172.21.15.29:0/3926790310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f800c9720 con 0x7f1f80154370 2024-01-24T15:30:23.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.539+0000 7f1f84e02700 1 -- 172.21.15.29:0/3926790310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f800ae7f0 con 0x7f1f80154370 2024-01-24T15:30:23.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.539+0000 7f1f6e7fc700 1 -- 172.21.15.29:0/3926790310 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 86) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f1f74005bd0 con 0x7f1f80154370 2024-01-24T15:30:23.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.540+0000 7f1f6e7fc700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f1f6005f240 0x7f1f60061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.540+0000 7f1f6e7fc700 1 -- 172.21.15.29:0/3926790310 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 7730+0+0 (secure 0 0 0) 0x7f1f7407a080 con 0x7f1f80154370 2024-01-24T15:30:23.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.540+0000 7f1f7e59c700 1 -- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f1f6005f240 msgr2=0x7f1f60061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:23.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.540+0000 7f1f7e59c700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f1f6005f240 0x7f1f60061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:23.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.544+0000 7f1f6e7fc700 1 -- 172.21.15.29:0/3926790310 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1f800ae7f0 con 0x7f1f80154370 2024-01-24T15:30:23.608 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.605+0000 7f5fb946e680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:30:23.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.627+0000 7f1f6e7fc700 1 -- 172.21.15.29:0/3926790310 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 87) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f1f74077680 con 0x7f1f80154370 2024-01-24T15:30:23.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.627+0000 7f1f6e7fc700 1 -- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f1f6005f240 msgr2=0x7f1f60061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:23.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.627+0000 7f1f6e7fc700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f1f6005f240 0x7f1f60061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:23.665 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:23.663+0000 7fae01b96680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:30:23.680 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.678+0000 7f5f329ef700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:23.680 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.678+0000 7f5f329ef700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:23.680 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.678+0000 7f5f329ef700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:23.680 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.678+0000 7f5f329ef700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:23.680 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.678+0000 7f5f329ef700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:23.681 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.678+0000 7f5f361f6700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:23.681 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.678+0000 7f5f361f6700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:23.681 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.678+0000 7f5f361f6700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:23.681 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.678+0000 7f5f361f6700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:23.681 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:23.678+0000 7f5f361f6700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:23.738 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.736+0000 7f1f84e02700 1 -- 172.21.15.29:0/3926790310 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f800ae7f0 con 0x7f1f80154370 2024-01-24T15:30:23.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.737+0000 7f1f6e7fc700 1 -- 172.21.15.29:0/3926790310 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+19422 (secure 0 0 0) 0x7f1f74059bf0 con 0x7f1f80154370 2024-01-24T15:30:23.739 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:23.739 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":199,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:30:23.621475+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:14:14.419558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-24T15:14:15.659901+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":198,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.5","mappings":[{"from":3,"to":0}]},{"pgid":"22.13","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/616281728":"2024-01-25T15:30:23.621415+0000","172.21.15.163:0/2497469811":"2024-01-25T15:30:23.621415+0000","172.21.15.163:0/1858174817":"2024-01-25T15:30:23.621415+0000","172.21.15.163:0/2475631050":"2024-01-25T15:30:23.621415+0000","172.21.15.163:6833/57365":"2024-01-25T15:30:23.621415+0000","172.21.15.163:6832/57365":"2024-01-25T15:30:23.621415+0000","172.21.15.163:0/2002665960":"2024-01-25T15:25:20.634238+0000","172.21.15.163:0/3305307869":"2024-01-25T15:25:20.634238+0000","172.21.15.163:0/4000114899":"2024-01-25T15:25:20.634238+0000","172.21.15.163:6832/43241":"2024-01-25T15:14:28.971452+0000","172.21.15.163:0/2434196663":"2024-01-25T15:14:28.971452+0000","172.21.15.163:6833/43241":"2024-01-25T15:14:28.971452+0000","172.21.15.163:0/4284075186":"2024-01-25T15:14:28.971452+0000","172.21.15.163:0/3544734349":"2024-01-25T15:25:20.634238+0000","172.21.15.163:0/2471445585":"2024-01-25T15:14:28.971452+0000","172.21.15.163:0/2024052538":"2024-01-25T15:14:28.971452+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:30:23.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.739+0000 7f1f84e02700 1 -- 172.21.15.29:0/3926790310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80154370 msgr2=0x7f1f800c2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.739+0000 7f1f84e02700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80154370 0x7f1f800c2830 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7f1f7400c4c0 tx=0x7f1f74007c10).stop 2024-01-24T15:30:23.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.739+0000 7f1f84e02700 1 -- 172.21.15.29:0/3926790310 shutdown_connections 2024-01-24T15:30:23.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.739+0000 7f1f84e02700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f1f6005f240 0x7f1f60061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:23.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.739+0000 7f1f84e02700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f80154750 0x7f1f800c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.739+0000 7f1f84e02700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f800aa0e0 0x7f1f801681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.739+0000 7f1f84e02700 1 --2- 172.21.15.29:0/3926790310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f80154370 0x7f1f800c2830 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.739+0000 7f1f84e02700 1 -- 172.21.15.29:0/3926790310 >> 172.21.15.29:0/3926790310 conn(0x7f1f8005ec80 msgr2=0x7f1f800c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:23.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.739+0000 7f1f84e02700 1 -- 172.21.15.29:0/3926790310 shutdown_connections 2024-01-24T15:30:23.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.739+0000 7f1f84e02700 1 -- 172.21.15.29:0/3926790310 wait complete. 2024-01-24T15:30:23.750 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:23.748+0000 7fae01b96680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:30:23.755 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-24T15:30:23.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.885+0000 7f185c9bf700 1 Processor -- start 2024-01-24T15:30:23.888 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.885+0000 7f185c9bf700 1 -- start start 2024-01-24T15:30:23.888 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:23.886+0000 7fae01b96680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:30:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f185c9bf700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1858151aa0 0x7f18580a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f185c9bf700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1858151e80 0x7f18580627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f185c9bf700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1858065360 con 0x7f18580a71d0 2024-01-24T15:30:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f185c9bf700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18580bcab0 con 0x7f1858151e80 2024-01-24T15:30:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f185c9bf700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18580bcd90 con 0x7f1858151aa0 2024-01-24T15:30:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f1855d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1858151e80 0x7f18580627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f1856d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f18580a71d0 0x7f1858157c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46182/0 (socket says 172.21.15.29:46182) 2024-01-24T15:30:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f1856d9d700 1 -- 172.21.15.29:0/989281964 learned_addr learned my addr 172.21.15.29:0/989281964 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f185659c700 1 --2- 172.21.15.29:0/989281964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1858151aa0 0x7f18580a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f1855d9b700 1 -- 172.21.15.29:0/989281964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1858151aa0 msgr2=0x7f18580a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:23.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f1855d9b700 1 --2- 172.21.15.29:0/989281964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1858151aa0 0x7f18580a6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f1855d9b700 1 -- 172.21.15.29:0/989281964 >> v1:172.21.15.29:6789/0 conn(0x7f18580a71d0 legacy=0x7f1858157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f1855d9b700 1 -- 172.21.15.29:0/989281964 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18580bd070 con 0x7f1858151e80 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f185559a700 1 -- 172.21.15.29:0/989281964 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693626984 0 0) 0x7f1858065360 con 0x7f18580a71d0 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.887+0000 7f1855d9b700 1 --2- 172.21.15.29:0/989281964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1858151e80 0x7f18580627b0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f184c009760 tx=0x7f184c009060).ready entity=mon.0 client_cookie=fa785f32ff8670cc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.888+0000 7f185559a700 1 -- 172.21.15.29:0/989281964 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c00b580 con 0x7f1858151e80 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.888+0000 7f185559a700 1 -- 172.21.15.29:0/989281964 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c00b6e0 con 0x7f1858151e80 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.888+0000 7f185559a700 1 -- 172.21.15.29:0/989281964 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c00ba60 con 0x7f1858151e80 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.888+0000 7f185c9bf700 1 -- 172.21.15.29:0/989281964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1858151e80 msgr2=0x7f18580627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.888+0000 7f185c9bf700 1 --2- 172.21.15.29:0/989281964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1858151e80 0x7f18580627b0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f184c009760 tx=0x7f184c009060).stop 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.888+0000 7f185c9bf700 1 -- 172.21.15.29:0/989281964 shutdown_connections 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.888+0000 7f185c9bf700 1 --2- 172.21.15.29:0/989281964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1858151aa0 0x7f18580a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.888+0000 7f185c9bf700 1 --2- 172.21.15.29:0/989281964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1858151e80 0x7f18580627b0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.888+0000 7f185c9bf700 1 -- 172.21.15.29:0/989281964 >> 172.21.15.29:0/989281964 conn(0x7f18580c8580 msgr2=0x7f18580c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:23.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.888+0000 7f185c9bf700 1 -- 172.21.15.29:0/989281964 shutdown_connections 2024-01-24T15:30:23.893 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.888+0000 7f185c9bf700 1 -- 172.21.15.29:0/989281964 wait complete. 2024-01-24T15:30:23.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.890+0000 7f185c9bf700 1 Processor -- start 2024-01-24T15:30:23.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.890+0000 7f185c9bf700 1 -- start start 2024-01-24T15:30:23.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.890+0000 7f185c9bf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18580a71d0 0x7f18580a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.890+0000 7f185c9bf700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1858151aa0 0x7f18580a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.890+0000 7f185c9bf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1858151e80 0x7f18580a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.890+0000 7f185c9bf700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18580a9bc0 con 0x7f1858151e80 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.890+0000 7f185c9bf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185814cef0 con 0x7f18580a71d0 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.890+0000 7f185c9bf700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18580a54b0 con 0x7f1858151aa0 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f1856d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1858151e80 0x7f18580a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f1856d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1858151e80 0x7f18580a5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52590/0 (socket says 172.21.15.29:52590) 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f1856d9d700 1 -- 172.21.15.29:0/2849895645 learned_addr learned my addr 172.21.15.29:0/2849895645 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f1855d9b700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1858151aa0 0x7f18580a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f185659c700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18580a71d0 0x7f18580a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f1856d9d700 1 -- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1858151aa0 msgr2=0x7f18580a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f1856d9d700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1858151aa0 0x7f18580a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f1856d9d700 1 -- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18580a71d0 msgr2=0x7f18580a7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:23.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f1856d9d700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18580a71d0 0x7f18580a7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:23.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f1856d9d700 1 -- 172.21.15.29:0/2849895645 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c0092a0 con 0x7f1858151e80 2024-01-24T15:30:23.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f1856d9d700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1858151e80 0x7f18580a5b70 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f185000c810 tx=0x7f18500014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:23.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f18467fc700 1 -- 172.21.15.29:0/2849895645 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850005d30 con 0x7f1858151e80 2024-01-24T15:30:23.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f18467fc700 1 -- 172.21.15.29:0/2849895645 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1850005e90 con 0x7f1858151e80 2024-01-24T15:30:23.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f185c9bf700 1 -- 172.21.15.29:0/2849895645 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18580aa160 con 0x7f1858151e80 2024-01-24T15:30:23.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f18467fc700 1 -- 172.21.15.29:0/2849895645 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850007300 con 0x7f1858151e80 2024-01-24T15:30:23.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.891+0000 7f185c9bf700 1 -- 172.21.15.29:0/2849895645 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18580a6970 con 0x7f1858151e80 2024-01-24T15:30:23.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.893+0000 7f185c9bf700 1 -- 172.21.15.29:0/2849895645 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18580a4f50 con 0x7f1858151e80 2024-01-24T15:30:23.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.893+0000 7f18467fc700 1 -- 172.21.15.29:0/2849895645 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 87) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f18500076b0 con 0x7f1858151e80 2024-01-24T15:30:23.897 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.894+0000 7f18467fc700 1 -- 172.21.15.29:0/2849895645 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 7960+0+0 (secure 0 0 0) 0x7f185007a080 con 0x7f1858151e80 2024-01-24T15:30:23.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:23.899+0000 7f18467fc700 1 -- 172.21.15.29:0/2849895645 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f185004ace0 con 0x7f1858151e80 2024-01-24T15:30:24.015 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:24.013+0000 7fae01b96680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:30:24.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.128+0000 7f185c9bf700 1 -- 172.21.15.29:0/2849895645 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f18580c7850 con 0x7f1858151e80 2024-01-24T15:30:24.356 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:24.354+0000 7fae01b96680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:30:24.438 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:24.436+0000 7fae01b96680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:30:24.626 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-01-24T15:30:24.624+0000 7fe09bfc3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-24T15:30:24.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.630+0000 7f18467fc700 1 -- 172.21.15.29:0/2849895645 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 88) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f18500777e0 con 0x7f1858151e80 2024-01-24T15:30:24.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.630+0000 7f18467fc700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f18380302f0 0x7f1838068cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:24.632 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.631+0000 7f185659c700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f18380302f0 0x7f1838068cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:24.634 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.632+0000 7f185659c700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f18380302f0 0x7f1838068cb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f184800bbc0 tx=0x7f1848008040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:24.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.640+0000 7f18467fc700 1 -- 172.21.15.29:0/2849895645 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) v1 ==== 161+0+0 (secure 0 0 0) 0x7f185006eaa0 con 0x7f1858151e80 2024-01-24T15:30:24.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.642+0000 7f185c9bf700 1 -- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f18380302f0 msgr2=0x7f1838068cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:24.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.642+0000 7f185c9bf700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f18380302f0 0x7f1838068cb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f184800bbc0 tx=0x7f1848008040).stop 2024-01-24T15:30:24.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.642+0000 7f185c9bf700 1 -- 172.21.15.29:0/2849895645 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1858151e80 msgr2=0x7f18580a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:24.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.642+0000 7f185c9bf700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1858151e80 0x7f18580a5b70 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f185000c810 tx=0x7f18500014a0).stop 2024-01-24T15:30:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.643+0000 7f185c9bf700 1 -- 172.21.15.29:0/2849895645 shutdown_connections 2024-01-24T15:30:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.643+0000 7f185c9bf700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1858151e80 0x7f18580a5b70 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.643+0000 7f185c9bf700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18580a71d0 0x7f18580a7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.643+0000 7f185c9bf700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f18380302f0 0x7f1838068cb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.643+0000 7f185c9bf700 1 --2- 172.21.15.29:0/2849895645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1858151aa0 0x7f18580a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.643+0000 7f185c9bf700 1 -- 172.21.15.29:0/2849895645 >> 172.21.15.29:0/2849895645 conn(0x7f18580c8580 msgr2=0x7f18580c61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.643+0000 7f185c9bf700 1 -- 172.21.15.29:0/2849895645 shutdown_connections 2024-01-24T15:30:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.643+0000 7f185c9bf700 1 -- 172.21.15.29:0/2849895645 wait complete. 2024-01-24T15:30:24.646 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-24T15:30:24.660 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-24T15:30:24.683 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T15:30:24.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.797+0000 7f6203cd8700 1 Processor -- start 2024-01-24T15:30:24.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.797+0000 7f6203cd8700 1 -- start start 2024-01-24T15:30:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.798+0000 7f6203cd8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61fc069280 0x7f61fc069700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.798+0000 7f6203cd8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f61fc0c2e00 0x7f61fc0c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.798+0000 7f6203cd8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61fc069bf0 con 0x7f61fc151db0 2024-01-24T15:30:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.798+0000 7f6203cd8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61fc065420 con 0x7f61fc0c2e00 2024-01-24T15:30:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.798+0000 7f6203cd8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc065630 con 0x7f61fc069280 2024-01-24T15:30:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6202275700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f61fc0c2e00 0x7f61fc0c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:24.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6201273700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61fc069280 0x7f61fc069700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6201a74700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f61fc151db0 0x7f61fc069fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:47436/0 (socket says 172.21.15.29:47436) 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6202275700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f61fc0c2e00 0x7f61fc0c3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:37902/0 (socket says 172.21.15.29:37902) 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6201a74700 1 -- 172.21.15.29:0/3424198454 learned_addr learned my addr 172.21.15.29:0/3424198454 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6202275700 1 -- 172.21.15.29:0/3424198454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61fc069280 msgr2=0x7f61fc069700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6202275700 1 --2- 172.21.15.29:0/3424198454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61fc069280 0x7f61fc069700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6202275700 1 -- 172.21.15.29:0/3424198454 >> v1:172.21.15.163:6789/0 conn(0x7f61fc151db0 legacy=0x7f61fc069fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6202275700 1 -- 172.21.15.29:0/3424198454 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc065910 con 0x7f61fc0c2e00 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6200a72700 1 -- 172.21.15.29:0/3424198454 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471572383 0 0) 0x7f61fc069bf0 con 0x7f61fc151db0 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6201273700 1 --2- 172.21.15.29:0/3424198454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61fc069280 0x7f61fc069700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6202275700 1 --2- 172.21.15.29:0/3424198454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61fc0c2e00 0x7f61fc0c3250 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f61f4009e60 tx=0x7f61f4009780).ready entity=mon.1 client_cookie=bf0014dac0b4e96a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6200a72700 1 -- 172.21.15.29:0/3424198454 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f400e070 con 0x7f61fc0c2e00 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6200a72700 1 -- 172.21.15.29:0/3424198454 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f400bb60 con 0x7f61fc0c2e00 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.799+0000 7f6200a72700 1 -- 172.21.15.29:0/3424198454 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4012410 con 0x7f61fc0c2e00 2024-01-24T15:30:24.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.800+0000 7f6203cd8700 1 -- 172.21.15.29:0/3424198454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61fc0c2e00 msgr2=0x7f61fc0c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:24.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.800+0000 7f6203cd8700 1 --2- 172.21.15.29:0/3424198454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61fc0c2e00 0x7f61fc0c3250 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f61f4009e60 tx=0x7f61f4009780).stop 2024-01-24T15:30:24.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.800+0000 7f6203cd8700 1 -- 172.21.15.29:0/3424198454 shutdown_connections 2024-01-24T15:30:24.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.800+0000 7f6203cd8700 1 --2- 172.21.15.29:0/3424198454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61fc0c2e00 0x7f61fc0c3250 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:24.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.800+0000 7f6203cd8700 1 --2- 172.21.15.29:0/3424198454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61fc069280 0x7f61fc069700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:24.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.800+0000 7f6203cd8700 1 -- 172.21.15.29:0/3424198454 >> 172.21.15.29:0/3424198454 conn(0x7f61fc064390 msgr2=0x7f61fc0c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:24.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.800+0000 7f6203cd8700 1 -- 172.21.15.29:0/3424198454 shutdown_connections 2024-01-24T15:30:24.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.800+0000 7f6203cd8700 1 -- 172.21.15.29:0/3424198454 wait complete. 2024-01-24T15:30:24.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.801+0000 7f6203cd8700 1 Processor -- start 2024-01-24T15:30:24.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.801+0000 7f6203cd8700 1 -- start start 2024-01-24T15:30:24.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.802+0000 7f6203cd8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61fc069280 0x7f61fc1457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.802+0000 7f6203cd8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61fc0c2e00 0x7f61fc142f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.802+0000 7f6203cd8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61fc151db0 0x7f61fc163890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.802+0000 7f6203cd8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc146a80 con 0x7f61fc069280 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.802+0000 7f6203cd8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc1527e0 con 0x7f61fc0c2e00 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.802+0000 7f6203cd8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc145e70 con 0x7f61fc151db0 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.802+0000 7f6201a74700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61fc069280 0x7f61fc1457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.802+0000 7f6201273700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61fc0c2e00 0x7f61fc142f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.802+0000 7f6201a74700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61fc069280 0x7f61fc1457e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52612/0 (socket says 172.21.15.29:52612) 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.802+0000 7f6201a74700 1 -- 172.21.15.29:0/3828418148 learned_addr learned my addr 172.21.15.29:0/3828418148 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.802+0000 7f6202275700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61fc151db0 0x7f61fc163890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.803+0000 7f6201273700 1 -- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61fc151db0 msgr2=0x7f61fc163890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.803+0000 7f6201273700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61fc151db0 0x7f61fc163890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:24.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.803+0000 7f6201273700 1 -- 172.21.15.29:0/3828418148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61fc069280 msgr2=0x7f61fc1457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.803+0000 7f6201273700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61fc069280 0x7f61fc1457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.803+0000 7f6201273700 1 -- 172.21.15.29:0/3828418148 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec0092a0 con 0x7f61fc0c2e00 2024-01-24T15:30:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.803+0000 7f6201273700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61fc0c2e00 0x7f61fc142f00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f61ec00e880 tx=0x7f61ec009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.803+0000 7f61f1ffb700 1 -- 172.21.15.29:0/3828418148 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec012600 con 0x7f61fc0c2e00 2024-01-24T15:30:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.803+0000 7f61f1ffb700 1 -- 172.21.15.29:0/3828418148 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec012760 con 0x7f61fc0c2e00 2024-01-24T15:30:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.803+0000 7f61f1ffb700 1 -- 172.21.15.29:0/3828418148 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec012ae0 con 0x7f61fc0c2e00 2024-01-24T15:30:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.803+0000 7f6203cd8700 1 -- 172.21.15.29:0/3828418148 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f4009a20 con 0x7f61fc0c2e00 2024-01-24T15:30:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.803+0000 7f6203cd8700 1 -- 172.21.15.29:0/3828418148 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc146430 con 0x7f61fc0c2e00 2024-01-24T15:30:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.804+0000 7f6203cd8700 1 -- 172.21.15.29:0/3828418148 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61fc0475e0 con 0x7f61fc0c2e00 2024-01-24T15:30:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.804+0000 7f61f1ffb700 1 -- 172.21.15.29:0/3828418148 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 88) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f61ec01b400 con 0x7f61fc0c2e00 2024-01-24T15:30:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.805+0000 7f61f1ffb700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61e005f030 0x7f61e00614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.805+0000 7f61f1ffb700 1 -- 172.21.15.29:0/3828418148 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 8046+0+0 (secure 0 0 0) 0x7f61ec07aae0 con 0x7f61fc0c2e00 2024-01-24T15:30:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.805+0000 7f6201a74700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61e005f030 0x7f61e00614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.806+0000 7f6201a74700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61e005f030 0x7f61e00614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f61f8007fd0 tx=0x7f61f8007e90).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:24.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:24.810+0000 7f61f1ffb700 1 -- 172.21.15.29:0/3828418148 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f61ec04b7a0 con 0x7f61fc0c2e00 2024-01-24T15:30:25.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.041+0000 7f6203cd8700 1 -- 172.21.15.29:0/3828418148 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f61fc146bc0 con 0x7f61fc0c2e00 2024-01-24T15:30:25.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.642+0000 7f61f1ffb700 1 -- 172.21.15.29:0/3828418148 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) v1 ==== 104+0+0 (secure 0 0 0) 0x7f61ec016070 con 0x7f61fc0c2e00 2024-01-24T15:30:25.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.645+0000 7f6203cd8700 1 -- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61e005f030 msgr2=0x7f61e00614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:25.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.645+0000 7f6203cd8700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61e005f030 0x7f61e00614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f61f8007fd0 tx=0x7f61f8007e90).stop 2024-01-24T15:30:25.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.645+0000 7f6203cd8700 1 -- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61fc0c2e00 msgr2=0x7f61fc142f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:25.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.645+0000 7f6203cd8700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61fc0c2e00 0x7f61fc142f00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f61ec00e880 tx=0x7f61ec009060).stop 2024-01-24T15:30:25.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.645+0000 7f6203cd8700 1 -- 172.21.15.29:0/3828418148 shutdown_connections 2024-01-24T15:30:25.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.645+0000 7f6203cd8700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f61e005f030 0x7f61e00614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:25.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.645+0000 7f6203cd8700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61fc151db0 0x7f61fc163890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:25.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.645+0000 7f6203cd8700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61fc069280 0x7f61fc1457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:25.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.645+0000 7f6203cd8700 1 --2- 172.21.15.29:0/3828418148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61fc0c2e00 0x7f61fc142f00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:25.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.645+0000 7f6203cd8700 1 -- 172.21.15.29:0/3828418148 >> 172.21.15.29:0/3828418148 conn(0x7f61fc064390 msgr2=0x7f61fc144e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:25.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.645+0000 7f6203cd8700 1 -- 172.21.15.29:0/3828418148 shutdown_connections 2024-01-24T15:30:25.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.646+0000 7f6203cd8700 1 -- 172.21.15.29:0/3828418148 wait complete. 2024-01-24T15:30:25.662 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-24T15:30:25.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.837+0000 7fcf7903a700 1 Processor -- start 2024-01-24T15:30:25.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.837+0000 7fcf7903a700 1 -- start start 2024-01-24T15:30:25.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.838+0000 7fcf7903a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcf7414e610 0x7fcf741545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:25.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.838+0000 7fcf7903a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7405e4b0 0x7fcf740627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:25.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.838+0000 7fcf7903a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf74154b90 con 0x7fcf74068b10 2024-01-24T15:30:25.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.838+0000 7fcf7903a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf74062ca0 con 0x7fcf7414e610 2024-01-24T15:30:25.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.838+0000 7fcf7903a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74154e70 con 0x7fcf7405e4b0 2024-01-24T15:30:25.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.838+0000 7fcf7259c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7405e4b0 0x7fcf740627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.838+0000 7fcf72d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcf7414e610 0x7fcf741545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.838+0000 7fcf7359e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fcf74068b10 0x7fcf740ac7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:47438/0 (socket says 172.21.15.29:47438) 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.838+0000 7fcf7359e700 1 -- 172.21.15.29:0/1829339555 learned_addr learned my addr 172.21.15.29:0/1829339555 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.839+0000 7fcf72d9d700 1 -- 172.21.15.29:0/1829339555 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7405e4b0 msgr2=0x7fcf740627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.839+0000 7fcf72d9d700 1 --2- 172.21.15.29:0/1829339555 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7405e4b0 0x7fcf740627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.839+0000 7fcf72d9d700 1 -- 172.21.15.29:0/1829339555 >> v1:172.21.15.163:6789/0 conn(0x7fcf74068b10 legacy=0x7fcf740ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.839+0000 7fcf72d9d700 1 -- 172.21.15.29:0/1829339555 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf74155150 con 0x7fcf7414e610 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.839+0000 7fcf7259c700 1 --2- 172.21.15.29:0/1829339555 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7405e4b0 0x7fcf740627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.839+0000 7fcf71d9b700 1 -- 172.21.15.29:0/1829339555 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162990336 0 0) 0x7fcf74154b90 con 0x7fcf74068b10 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.839+0000 7fcf72d9d700 1 --2- 172.21.15.29:0/1829339555 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7414e610 0x7fcf741545f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fcf68009760 tx=0x7fcf68009060).ready entity=mon.1 client_cookie=d1ef028e7d691085 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.839+0000 7fcf71d9b700 1 -- 172.21.15.29:0/1829339555 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800f510 con 0x7fcf7414e610 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.839+0000 7fcf71d9b700 1 -- 172.21.15.29:0/1829339555 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6800f670 con 0x7fcf7414e610 2024-01-24T15:30:25.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.839+0000 7fcf71d9b700 1 -- 172.21.15.29:0/1829339555 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800f9f0 con 0x7fcf7414e610 2024-01-24T15:30:25.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.840+0000 7fcf7903a700 1 -- 172.21.15.29:0/1829339555 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7414e610 msgr2=0x7fcf741545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:25.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.840+0000 7fcf7903a700 1 --2- 172.21.15.29:0/1829339555 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7414e610 0x7fcf741545f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fcf68009760 tx=0x7fcf68009060).stop 2024-01-24T15:30:25.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.840+0000 7fcf7903a700 1 -- 172.21.15.29:0/1829339555 shutdown_connections 2024-01-24T15:30:25.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.840+0000 7fcf7903a700 1 --2- 172.21.15.29:0/1829339555 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7414e610 0x7fcf741545f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:25.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.840+0000 7fcf7903a700 1 --2- 172.21.15.29:0/1829339555 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7405e4b0 0x7fcf740627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:25.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.840+0000 7fcf7903a700 1 -- 172.21.15.29:0/1829339555 >> 172.21.15.29:0/1829339555 conn(0x7fcf740c7280 msgr2=0x7fcf740c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:25.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.840+0000 7fcf7903a700 1 -- 172.21.15.29:0/1829339555 shutdown_connections 2024-01-24T15:30:25.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.840+0000 7fcf7903a700 1 -- 172.21.15.29:0/1829339555 wait complete. 2024-01-24T15:30:25.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.841+0000 7fcf7903a700 1 Processor -- start 2024-01-24T15:30:25.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.841+0000 7fcf7903a700 1 -- start start 2024-01-24T15:30:25.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.842+0000 7fcf7903a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7405e4b0 0x7fcf740c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:25.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.842+0000 7fcf7903a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74068b10 0x7fcf74147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:25.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.842+0000 7fcf7903a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7414e610 0x7fcf7413abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:25.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.842+0000 7fcf7903a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740c3d40 con 0x7fcf74068b10 2024-01-24T15:30:25.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.842+0000 7fcf7903a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740c3130 con 0x7fcf7405e4b0 2024-01-24T15:30:25.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.842+0000 7fcf7903a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740c3410 con 0x7fcf7414e610 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.842+0000 7fcf7259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74068b10 0x7fcf74147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.842+0000 7fcf72d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7405e4b0 0x7fcf740c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.842+0000 7fcf7259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74068b10 0x7fcf74147340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52618/0 (socket says 172.21.15.29:52618) 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.842+0000 7fcf7259c700 1 -- 172.21.15.29:0/604343124 learned_addr learned my addr 172.21.15.29:0/604343124 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.842+0000 7fcf7359e700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7414e610 0x7fcf7413abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.843+0000 7fcf7259c700 1 -- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7414e610 msgr2=0x7fcf7413abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.843+0000 7fcf7259c700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7414e610 0x7fcf7413abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.843+0000 7fcf7259c700 1 -- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7405e4b0 msgr2=0x7fcf740c17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.843+0000 7fcf7259c700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7405e4b0 0x7fcf740c17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.843+0000 7fcf7259c700 1 -- 172.21.15.29:0/604343124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf640092a0 con 0x7fcf74068b10 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.843+0000 7fcf72d9d700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7405e4b0 0x7fcf740c17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.843+0000 7fcf7259c700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74068b10 0x7fcf74147340 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7fcf6400ea20 tx=0x7fcf64009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.843+0000 7fcf5affd700 1 -- 172.21.15.29:0/604343124 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400ee60 con 0x7fcf74068b10 2024-01-24T15:30:25.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.843+0000 7fcf7903a700 1 -- 172.21.15.29:0/604343124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf680092a0 con 0x7fcf74068b10 2024-01-24T15:30:25.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.843+0000 7fcf7903a700 1 -- 172.21.15.29:0/604343124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf740c39d0 con 0x7fcf74068b10 2024-01-24T15:30:25.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.843+0000 7fcf5affd700 1 -- 172.21.15.29:0/604343124 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6400ca60 con 0x7fcf74068b10 2024-01-24T15:30:25.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.843+0000 7fcf5affd700 1 -- 172.21.15.29:0/604343124 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400cbd0 con 0x7fcf74068b10 2024-01-24T15:30:25.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.844+0000 7fcf7903a700 1 -- 172.21.15.29:0/604343124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf54005120 con 0x7fcf74068b10 2024-01-24T15:30:25.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.844+0000 7fcf5affd700 1 -- 172.21.15.29:0/604343124 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 88) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fcf6401b400 con 0x7fcf74068b10 2024-01-24T15:30:25.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.845+0000 7fcf5affd700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcf6005f030 0x7fcf600614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:25.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.845+0000 7fcf5affd700 1 -- 172.21.15.29:0/604343124 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 8046+0+0 (secure 0 0 0) 0x7fcf6407ac00 con 0x7fcf74068b10 2024-01-24T15:30:25.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.845+0000 7fcf72d9d700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcf6005f030 0x7fcf600614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:25.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.846+0000 7fcf72d9d700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcf6005f030 0x7fcf600614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fcf68009fd0 tx=0x7fcf68017000).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:25.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:25.850+0000 7fcf5affd700 1 -- 172.21.15.29:0/604343124 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcf64080020 con 0x7fcf74068b10 2024-01-24T15:30:26.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.033+0000 7fcf7903a700 1 -- 172.21.15.29:0/604343124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcf540057f0 con 0x7fcf74068b10 2024-01-24T15:30:26.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.324+0000 7fcf5affd700 1 -- 172.21.15.29:0/604343124 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcf640784a0 con 0x7fcf74068b10 2024-01-24T15:30:26.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.657+0000 7fcf5affd700 1 -- 172.21.15.29:0/604343124 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v202) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcf64070fb0 con 0x7fcf74068b10 2024-01-24T15:30:26.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.661+0000 7fcf7903a700 1 -- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcf6005f030 msgr2=0x7fcf600614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:26.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.661+0000 7fcf7903a700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcf6005f030 0x7fcf600614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fcf68009fd0 tx=0x7fcf68017000).stop 2024-01-24T15:30:26.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.661+0000 7fcf7903a700 1 -- 172.21.15.29:0/604343124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74068b10 msgr2=0x7fcf74147340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:26.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.661+0000 7fcf7903a700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74068b10 0x7fcf74147340 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7fcf6400ea20 tx=0x7fcf64009060).stop 2024-01-24T15:30:26.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.662+0000 7fcf7903a700 1 -- 172.21.15.29:0/604343124 shutdown_connections 2024-01-24T15:30:26.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.662+0000 7fcf7903a700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcf6005f030 0x7fcf600614f0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0x7fcf68009fd0 tx=0x7fcf68017000).stop 2024-01-24T15:30:26.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.662+0000 7fcf7903a700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf7414e610 0x7fcf7413abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:26.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.662+0000 7fcf7903a700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf7405e4b0 0x7fcf740c17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:26.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.662+0000 7fcf7903a700 1 --2- 172.21.15.29:0/604343124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf74068b10 0x7fcf74147340 secure :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 rx=0x7fcf6400ea20 tx=0x7fcf64009060).stop 2024-01-24T15:30:26.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.662+0000 7fcf7903a700 1 -- 172.21.15.29:0/604343124 >> 172.21.15.29:0/604343124 conn(0x7fcf740c7280 msgr2=0x7fcf7414d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:26.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.662+0000 7fcf7903a700 1 -- 172.21.15.29:0/604343124 shutdown_connections 2024-01-24T15:30:26.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.662+0000 7fcf7903a700 1 -- 172.21.15.29:0/604343124 wait complete. 2024-01-24T15:30:26.664 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-01-24T15:30:26.679 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-24T15:30:26.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.815+0000 7fb32f299700 1 Processor -- start 2024-01-24T15:30:26.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.816+0000 7fb32f299700 1 -- start start 2024-01-24T15:30:26.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.817+0000 7fb32f299700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb32814f8f0 0x7fb328152490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.817+0000 7fb32f299700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb3280a77d0 0x7fb328065410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.817+0000 7fb32f299700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32806a690 con 0x7fb32814fcd0 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.817+0000 7fb32f299700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb328155a60 con 0x7fb32814f8f0 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.817+0000 7fb32f299700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb328155d00 con 0x7fb3280a77d0 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.817+0000 7fb32d035700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb32814f8f0 0x7fb328152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.817+0000 7fb32d035700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb32814f8f0 0x7fb328152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52628/0 (socket says 172.21.15.29:52628) 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.817+0000 7fb32d035700 1 -- 172.21.15.29:0/2169217644 learned_addr learned my addr 172.21.15.29:0/2169217644 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.817+0000 7fb32d836700 1 --2- 172.21.15.29:0/2169217644 >> v2:172.21.15.163:3300/0 conn(0x7fb3280a77d0 0x7fb328065410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.818+0000 7fb32d035700 1 -- 172.21.15.29:0/2169217644 >> v2:172.21.15.163:3300/0 conn(0x7fb3280a77d0 msgr2=0x7fb328065410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.818+0000 7fb32d035700 1 --2- 172.21.15.29:0/2169217644 >> v2:172.21.15.163:3300/0 conn(0x7fb3280a77d0 0x7fb328065410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.818+0000 7fb32d035700 1 -- 172.21.15.29:0/2169217644 >> v1:172.21.15.163:6789/0 conn(0x7fb32814fcd0 legacy=0x7fb328156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.818+0000 7fb32d035700 1 -- 172.21.15.29:0/2169217644 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328152cf0 con 0x7fb32814f8f0 2024-01-24T15:30:26.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.818+0000 7fb32d035700 1 --2- 172.21.15.29:0/2169217644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb32814f8f0 0x7fb328152490 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7fb328068fd0 tx=0x7fb3240090f0).ready entity=mon.0 client_cookie=c8f01cd0742a645f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:26.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.818+0000 7fb31bfff700 1 -- 172.21.15.29:0/2169217644 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32400b520 con 0x7fb32814f8f0 2024-01-24T15:30:26.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.818+0000 7fb31bfff700 1 -- 172.21.15.29:0/2169217644 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32400b680 con 0x7fb32814f8f0 2024-01-24T15:30:26.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.818+0000 7fb31bfff700 1 -- 172.21.15.29:0/2169217644 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32400ba00 con 0x7fb32814f8f0 2024-01-24T15:30:26.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.818+0000 7fb32f299700 1 -- 172.21.15.29:0/2169217644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb32814f8f0 msgr2=0x7fb328152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:26.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.818+0000 7fb32f299700 1 --2- 172.21.15.29:0/2169217644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb32814f8f0 0x7fb328152490 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7fb328068fd0 tx=0x7fb3240090f0).stop 2024-01-24T15:30:26.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.819+0000 7fb32f299700 1 -- 172.21.15.29:0/2169217644 shutdown_connections 2024-01-24T15:30:26.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.819+0000 7fb32f299700 1 --2- 172.21.15.29:0/2169217644 >> v2:172.21.15.163:3300/0 conn(0x7fb3280a77d0 0x7fb328065410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:26.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.819+0000 7fb32f299700 1 --2- 172.21.15.29:0/2169217644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb32814f8f0 0x7fb328152490 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:26.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.819+0000 7fb32f299700 1 -- 172.21.15.29:0/2169217644 >> 172.21.15.29:0/2169217644 conn(0x7fb328067b20 msgr2=0x7fb3280638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:26.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.819+0000 7fb32f299700 1 -- 172.21.15.29:0/2169217644 shutdown_connections 2024-01-24T15:30:26.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.819+0000 7fb32f299700 1 -- 172.21.15.29:0/2169217644 wait complete. 2024-01-24T15:30:26.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.820+0000 7fb32f299700 1 Processor -- start 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.820+0000 7fb32f299700 1 -- start start 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32f299700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3280a77d0 0x7fb328143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32f299700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb32814f8f0 0x7fb3281440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32f299700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb32814fcd0 0x7fb3281445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32f299700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3281431e0 con 0x7fb32814f8f0 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32f299700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb328143b60 con 0x7fb32814fcd0 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32f299700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3281425d0 con 0x7fb3280a77d0 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32c834700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb32814f8f0 0x7fb3281440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32c834700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb32814f8f0 0x7fb3281440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52634/0 (socket says 172.21.15.29:52634) 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32c834700 1 -- 172.21.15.29:0/2616407069 learned_addr learned my addr 172.21.15.29:0/2616407069 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32d836700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb32814fcd0 0x7fb3281445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32d035700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3280a77d0 0x7fb328143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:26.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32c834700 1 -- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3280a77d0 msgr2=0x7fb328143670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:26.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32c834700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3280a77d0 0x7fb328143670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:26.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32c834700 1 -- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb32814fcd0 msgr2=0x7fb3281445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:26.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32c834700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb32814fcd0 0x7fb3281445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:26.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.821+0000 7fb32c834700 1 -- 172.21.15.29:0/2616407069 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb324009370 con 0x7fb32814f8f0 2024-01-24T15:30:26.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.822+0000 7fb32c834700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb32814f8f0 0x7fb3281440f0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7fb31c00c440 tx=0x7fb31c007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:26.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.822+0000 7fb3197fa700 1 -- 172.21.15.29:0/2616407069 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c007020 con 0x7fb32814f8f0 2024-01-24T15:30:26.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.822+0000 7fb3197fa700 1 -- 172.21.15.29:0/2616407069 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31c007180 con 0x7fb32814f8f0 2024-01-24T15:30:26.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.822+0000 7fb32f299700 1 -- 172.21.15.29:0/2616407069 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3281428b0 con 0x7fb32814f8f0 2024-01-24T15:30:26.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.822+0000 7fb3197fa700 1 -- 172.21.15.29:0/2616407069 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c007520 con 0x7fb32814f8f0 2024-01-24T15:30:26.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.822+0000 7fb32f299700 1 -- 172.21.15.29:0/2616407069 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb328142e70 con 0x7fb32814f8f0 2024-01-24T15:30:26.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.823+0000 7fb32f299700 1 -- 172.21.15.29:0/2616407069 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3280475e0 con 0x7fb32814f8f0 2024-01-24T15:30:26.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.823+0000 7fb3197fa700 1 -- 172.21.15.29:0/2616407069 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb31c0068b0 con 0x7fb32814f8f0 2024-01-24T15:30:26.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.824+0000 7fb3197fa700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fb30c05f270 0x7fb30c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:26.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.824+0000 7fb3197fa700 1 -- 172.21.15.29:0/2616407069 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 7550+0+0 (secure 0 0 0) 0x7fb31c07a070 con 0x7fb32814f8f0 2024-01-24T15:30:26.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.824+0000 7fb32d035700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fb30c05f270 0x7fb30c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:26.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.824+0000 7fb32d035700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fb30c05f270 0x7fb30c061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fb324009130 tx=0x7fb324017000).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:26.829 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:26.828+0000 7fb3197fa700 1 -- 172.21.15.29:0/2616407069 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb31c04ac30 con 0x7fb32814f8f0 2024-01-24T15:30:27.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.023+0000 7fb32f299700 1 -- 172.21.15.29:0/2616407069 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb32806a690 con 0x7fb32814f8f0 2024-01-24T15:30:27.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.871+0000 7fb3197fa700 1 -- 172.21.15.29:0/2616407069 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v204) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb32806a690 con 0x7fb32814f8f0 2024-01-24T15:30:27.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.875+0000 7fb32f299700 1 -- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fb30c05f270 msgr2=0x7fb30c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:27.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.875+0000 7fb32f299700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fb30c05f270 0x7fb30c061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fb324009130 tx=0x7fb324017000).stop 2024-01-24T15:30:27.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.875+0000 7fb32f299700 1 -- 172.21.15.29:0/2616407069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb32814f8f0 msgr2=0x7fb3281440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:27.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.875+0000 7fb32f299700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb32814f8f0 0x7fb3281440f0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7fb31c00c440 tx=0x7fb31c007da0).stop 2024-01-24T15:30:27.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.876+0000 7fb32f299700 1 -- 172.21.15.29:0/2616407069 shutdown_connections 2024-01-24T15:30:27.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.876+0000 7fb32f299700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb32814fcd0 0x7fb3281445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.876+0000 7fb32f299700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fb30c05f270 0x7fb30c061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.876+0000 7fb32f299700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3280a77d0 0x7fb328143670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.876+0000 7fb32f299700 1 --2- 172.21.15.29:0/2616407069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb32814f8f0 0x7fb3281440f0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.876+0000 7fb32f299700 1 -- 172.21.15.29:0/2616407069 >> 172.21.15.29:0/2616407069 conn(0x7fb328067b20 msgr2=0x7fb3280c59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.876+0000 7fb32f299700 1 -- 172.21.15.29:0/2616407069 shutdown_connections 2024-01-24T15:30:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:27.876+0000 7fb32f299700 1 -- 172.21.15.29:0/2616407069 wait complete. 2024-01-24T15:30:27.878 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-01-24T15:30:27.893 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-24T15:30:28.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.029+0000 7effa6b00700 1 Processor -- start 2024-01-24T15:30:28.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.029+0000 7effa6b00700 1 -- start start 2024-01-24T15:30:28.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.030+0000 7effa6b00700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7effa0157530 0x7effa0068470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.030+0000 7effa6b00700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa0068960 0x7effa0066520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.030+0000 7effa6b00700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa0066ec0 con 0x7effa0066a10 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.030+0000 7effa6b00700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa0065040 con 0x7effa0157530 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.030+0000 7effa6b00700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa0064140 con 0x7effa0068960 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.030+0000 7effa509d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7effa0066a10 0x7effa0157c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46202/0 (socket says 172.21.15.29:46202) 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.030+0000 7effa509d700 1 -- 172.21.15.29:0/1965626543 learned_addr learned my addr 172.21.15.29:0/1965626543 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.030+0000 7effa489c700 1 --2- 172.21.15.29:0/1965626543 >> v2:172.21.15.163:3300/0 conn(0x7effa0157530 0x7effa0068470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.030+0000 7eff9ffff700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa0068960 0x7effa0066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.030+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1776466048 0 0) 0x7eff98001490 con 0x7effa0066a10 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.030+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff900033f0 con 0x7effa0066a10 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2985949262 0 0) 0x7eff900033f0 con 0x7effa0066a10 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa0068960 msgr2=0x7effa0066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa0068960 0x7effa0066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 >> v2:172.21.15.163:3300/0 conn(0x7effa0157530 msgr2=0x7effa0068470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 --2- 172.21.15.29:0/1965626543 >> v2:172.21.15.163:3300/0 conn(0x7effa0157530 0x7effa0068470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa00be080 con 0x7effa0066a10 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7eff98001980 con 0x7effa0066a10 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7effa489c700 1 --2- 172.21.15.29:0/1965626543 >> v2:172.21.15.163:3300/0 conn(0x7effa0157530 0x7effa0068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 >> v1:172.21.15.29:6789/0 conn(0x7effa0066a10 legacy=0x7effa0157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff900038e0 0x7eff90003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa0068960 0x7eff90004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff90004b80 0x7eff90005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa0065040 con 0x7eff90004b80 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa0064140 con 0x7eff900038e0 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9000b8e0 con 0x7effa0068960 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7effa509d700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff90004b80 0x7eff90005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7effa489c700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff900038e0 0x7eff90003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7eff9ffff700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa0068960 0x7eff90004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7effa509d700 1 -- 172.21.15.29:0/1965626543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa0068960 msgr2=0x7eff90004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7effa509d700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa0068960 0x7eff90004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7effa509d700 1 -- 172.21.15.29:0/1965626543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff900038e0 msgr2=0x7eff90003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7effa509d700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff900038e0 0x7eff90003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7effa509d700 1 -- 172.21.15.29:0/1965626543 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff900033f0 con 0x7eff90004b80 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7effa489c700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff900038e0 0x7eff90003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.031+0000 7effa509d700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff90004b80 0x7eff90005000 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7eff9800c6d0 tx=0x7eff98007c80).ready entity=mon.0 client_cookie=8eba406410cc26da server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.032+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98005b20 con 0x7eff90004b80 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.032+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff98005c80 con 0x7eff90004b80 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.032+0000 7eff9f7fe700 1 -- 172.21.15.29:0/1965626543 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98007110 con 0x7eff90004b80 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.032+0000 7effa6b00700 1 -- 172.21.15.29:0/1965626543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff90004b80 msgr2=0x7eff90005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.032+0000 7effa6b00700 1 --2- 172.21.15.29:0/1965626543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7eff90004b80 0x7eff90005000 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7eff9800c6d0 tx=0x7eff98007c80).stop 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.032+0000 7effa509d700 1 -- 172.21.15.29:0/1965626543 reap_dead start 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.032+0000 7effa6b00700 1 -- 172.21.15.29:0/1965626543 shutdown_connections 2024-01-24T15:30:28.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.032+0000 7effa6b00700 1 -- 172.21.15.29:0/1965626543 >> 172.21.15.29:0/1965626543 conn(0x7effa00c8ba0 msgr2=0x7effa00c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:28.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.032+0000 7effa6b00700 1 -- 172.21.15.29:0/1965626543 shutdown_connections 2024-01-24T15:30:28.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.032+0000 7effa6b00700 1 -- 172.21.15.29:0/1965626543 wait complete. 2024-01-24T15:30:28.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.033+0000 7effa6b00700 1 Processor -- start 2024-01-24T15:30:28.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.034+0000 7effa6b00700 1 -- start start 2024-01-24T15:30:28.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.034+0000 7effa6b00700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa014cff0 0x7effa00c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.034+0000 7effa6b00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa014d690 0x7effa0159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.034+0000 7effa6b00700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa00c3920 0x7effa00c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.034+0000 7effa6b00700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa00c1950 con 0x7effa014cff0 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.034+0000 7effa6b00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa014d3d0 con 0x7effa014d690 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.034+0000 7effa6b00700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa00c4290 con 0x7effa00c3920 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa489c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa014cff0 0x7effa00c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa489c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa014cff0 0x7effa00c1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52654/0 (socket says 172.21.15.29:52654) 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa489c700 1 -- 172.21.15.29:0/2253541605 learned_addr learned my addr 172.21.15.29:0/2253541605 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7eff9ffff700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa014d690 0x7effa0159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa509d700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa00c3920 0x7effa00c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa489c700 1 -- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa00c3920 msgr2=0x7effa00c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa489c700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa00c3920 0x7effa00c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa489c700 1 -- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa014d690 msgr2=0x7effa0159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa489c700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa014d690 0x7effa0159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa489c700 1 -- 172.21.15.29:0/2253541605 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa0159b80 con 0x7effa014cff0 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa509d700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa00c3920 0x7effa00c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7eff9ffff700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa014d690 0x7effa0159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa489c700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa014cff0 0x7effa00c1de0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7eff94011d70 tx=0x7eff94009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7eff9cff9700 1 -- 172.21.15.29:0/2253541605 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9400fcd0 con 0x7effa014cff0 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7eff9cff9700 1 -- 172.21.15.29:0/2253541605 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9400fe30 con 0x7effa014cff0 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa6b00700 1 -- 172.21.15.29:0/2253541605 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa00c0d40 con 0x7effa014cff0 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7eff9cff9700 1 -- 172.21.15.29:0/2253541605 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94014650 con 0x7effa014cff0 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.035+0000 7effa6b00700 1 -- 172.21.15.29:0/2253541605 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa00c1300 con 0x7effa014cff0 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.036+0000 7effa6b00700 1 -- 172.21.15.29:0/2253541605 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa00475e0 con 0x7effa014cff0 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.037+0000 7eff9cff9700 1 -- 172.21.15.29:0/2253541605 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7eff94014a80 con 0x7effa014cff0 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.037+0000 7eff9cff9700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7eff8405f270 0x7eff84061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.037+0000 7eff9cff9700 1 -- 172.21.15.29:0/2253541605 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 7108+0+0 (secure 0 0 0) 0x7eff9407b480 con 0x7effa014cff0 2024-01-24T15:30:28.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.037+0000 7eff9ffff700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7eff8405f270 0x7eff84061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.038+0000 7eff9ffff700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7eff8405f270 0x7eff84061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7eff8c009270 tx=0x7eff8c008040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:28.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.042+0000 7eff9cff9700 1 -- 172.21.15.29:0/2253541605 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7eff9404c130 con 0x7effa014cff0 2024-01-24T15:30:28.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.222+0000 7effa6b00700 1 -- 172.21.15.29:0/2253541605 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7effa015a720 con 0x7effa014cff0 2024-01-24T15:30:28.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.226+0000 7eff9cff9700 1 -- 172.21.15.29:0/2253541605 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7eff9404f750 con 0x7effa014cff0 2024-01-24T15:30:28.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.228+0000 7effa6b00700 1 -- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7eff8405f270 msgr2=0x7eff84061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.228+0000 7effa6b00700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7eff8405f270 0x7eff84061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7eff8c009270 tx=0x7eff8c008040).stop 2024-01-24T15:30:28.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.228+0000 7effa6b00700 1 -- 172.21.15.29:0/2253541605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa014cff0 msgr2=0x7effa00c1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.228+0000 7effa6b00700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa014cff0 0x7effa00c1de0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7eff94011d70 tx=0x7eff94009060).stop 2024-01-24T15:30:28.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.228+0000 7effa6b00700 1 -- 172.21.15.29:0/2253541605 shutdown_connections 2024-01-24T15:30:28.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.228+0000 7effa6b00700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7eff8405f270 0x7eff84061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.228+0000 7effa6b00700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effa00c3920 0x7effa00c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.228+0000 7effa6b00700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa014cff0 0x7effa00c1de0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.228+0000 7effa6b00700 1 --2- 172.21.15.29:0/2253541605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa014d690 0x7effa0159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.228+0000 7effa6b00700 1 -- 172.21.15.29:0/2253541605 >> 172.21.15.29:0/2253541605 conn(0x7effa00c8ba0 msgr2=0x7effa00c5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:28.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.228+0000 7effa6b00700 1 -- 172.21.15.29:0/2253541605 shutdown_connections 2024-01-24T15:30:28.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.228+0000 7effa6b00700 1 -- 172.21.15.29:0/2253541605 wait complete. 2024-01-24T15:30:28.243 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-01-24T15:30:28.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.375+0000 7f749044d700 1 Processor -- start 2024-01-24T15:30:28.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.375+0000 7f749044d700 1 -- start start 2024-01-24T15:30:28.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.376+0000 7f749044d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7488157530 0x7f7488068470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.376+0000 7f749044d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7488068960 0x7f7488066520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.376+0000 7f749044d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74880be080 con 0x7f7488066a10 2024-01-24T15:30:28.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.376+0000 7f749044d700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7488065040 con 0x7f7488068960 2024-01-24T15:30:28.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.376+0000 7f749044d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7488064140 con 0x7f7488157530 2024-01-24T15:30:28.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.376+0000 7f748d9e8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7488068960 0x7f7488066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.376+0000 7f748d9e8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7488068960 0x7f7488066520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52666/0 (socket says 172.21.15.29:52666) 2024-01-24T15:30:28.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.376+0000 7f748d9e8700 1 -- 172.21.15.29:0/549306584 learned_addr learned my addr 172.21.15.29:0/549306584 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:28.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.377+0000 7f748e1e9700 1 --2- 172.21.15.29:0/549306584 >> v2:172.21.15.163:3300/0 conn(0x7f7488157530 0x7f7488068470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.377+0000 7f748d9e8700 1 -- 172.21.15.29:0/549306584 >> v2:172.21.15.163:3300/0 conn(0x7f7488157530 msgr2=0x7f7488068470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.377+0000 7f748d9e8700 1 --2- 172.21.15.29:0/549306584 >> v2:172.21.15.163:3300/0 conn(0x7f7488157530 0x7f7488068470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.377+0000 7f748d9e8700 1 -- 172.21.15.29:0/549306584 >> v1:172.21.15.163:6789/0 conn(0x7f7488066a10 legacy=0x7f7488157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.377+0000 7f748d9e8700 1 -- 172.21.15.29:0/549306584 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7488067380 con 0x7f7488068960 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.377+0000 7f748d9e8700 1 --2- 172.21.15.29:0/549306584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488068960 0x7f7488066520 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7f74880c66b0 tx=0x7f74840090f0).ready entity=mon.0 client_cookie=66fb9d610645d2eb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.377+0000 7f748d1e7700 1 -- 172.21.15.29:0/549306584 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400b520 con 0x7f7488068960 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.377+0000 7f748d1e7700 1 -- 172.21.15.29:0/549306584 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748400b680 con 0x7f7488068960 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.377+0000 7f748d1e7700 1 -- 172.21.15.29:0/549306584 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400ba00 con 0x7f7488068960 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.378+0000 7f749044d700 1 -- 172.21.15.29:0/549306584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488068960 msgr2=0x7f7488066520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.378+0000 7f749044d700 1 --2- 172.21.15.29:0/549306584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488068960 0x7f7488066520 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7f74880c66b0 tx=0x7f74840090f0).stop 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.378+0000 7f749044d700 1 -- 172.21.15.29:0/549306584 shutdown_connections 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.378+0000 7f749044d700 1 --2- 172.21.15.29:0/549306584 >> v2:172.21.15.163:3300/0 conn(0x7f7488157530 0x7f7488068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.378+0000 7f749044d700 1 --2- 172.21.15.29:0/549306584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488068960 0x7f7488066520 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.378+0000 7f749044d700 1 -- 172.21.15.29:0/549306584 >> 172.21.15.29:0/549306584 conn(0x7f74880c8ba0 msgr2=0x7f74880c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:28.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.378+0000 7f749044d700 1 -- 172.21.15.29:0/549306584 shutdown_connections 2024-01-24T15:30:28.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.378+0000 7f749044d700 1 -- 172.21.15.29:0/549306584 wait complete. 2024-01-24T15:30:28.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.379+0000 7f749044d700 1 Processor -- start 2024-01-24T15:30:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.379+0000 7f749044d700 1 -- start start 2024-01-24T15:30:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f749044d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7488066a10 0x7f74880ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f749044d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7488068960 0x7f74880a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f749044d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488157530 0x7f74880a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f749044d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880a9060 con 0x7f7488157530 2024-01-24T15:30:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f749044d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880c0580 con 0x7f7488066a10 2024-01-24T15:30:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f749044d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74880c46a0 con 0x7f7488068960 2024-01-24T15:30:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f748e9ea700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488157530 0x7f74880a8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f748e9ea700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488157530 0x7f74880a8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52682/0 (socket says 172.21.15.29:52682) 2024-01-24T15:30:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f748e9ea700 1 -- 172.21.15.29:0/3455615739 learned_addr learned my addr 172.21.15.29:0/3455615739 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:28.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f748e1e9700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7488066a10 0x7f74880ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f748d9e8700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7488068960 0x7f74880a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f748e9ea700 1 -- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7488068960 msgr2=0x7f74880a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.380+0000 7f748e9ea700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7488068960 0x7f74880a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.381+0000 7f748e9ea700 1 -- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7488066a10 msgr2=0x7f74880ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.381+0000 7f748e9ea700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7488066a10 0x7f74880ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.381+0000 7f748e9ea700 1 -- 172.21.15.29:0/3455615739 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484009370 con 0x7f7488157530 2024-01-24T15:30:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.381+0000 7f748e9ea700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488157530 0x7f74880a8b70 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f748000c7c0 tx=0x7f7480001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.381+0000 7f747e7fc700 1 -- 172.21.15.29:0/3455615739 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748000e4a0 con 0x7f7488157530 2024-01-24T15:30:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.381+0000 7f747e7fc700 1 -- 172.21.15.29:0/3455615739 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748000e600 con 0x7f7488157530 2024-01-24T15:30:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.381+0000 7f749044d700 1 -- 172.21.15.29:0/3455615739 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74880c4980 con 0x7f7488157530 2024-01-24T15:30:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.381+0000 7f747e7fc700 1 -- 172.21.15.29:0/3455615739 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748000e980 con 0x7f7488157530 2024-01-24T15:30:28.385 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.381+0000 7f749044d700 1 -- 172.21.15.29:0/3455615739 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74880c4f40 con 0x7f7488157530 2024-01-24T15:30:28.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.382+0000 7f749044d700 1 -- 172.21.15.29:0/3455615739 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74880c0710 con 0x7f7488157530 2024-01-24T15:30:28.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.382+0000 7f747e7fc700 1 -- 172.21.15.29:0/3455615739 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f74800123f0 con 0x7f7488157530 2024-01-24T15:30:28.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.383+0000 7f747e7fc700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f746c05f270 0x7f746c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.383+0000 7f747e7fc700 1 -- 172.21.15.29:0/3455615739 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 7108+0+0 (secure 0 0 0) 0x7f7480079fa0 con 0x7f7488157530 2024-01-24T15:30:28.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.383+0000 7f748e1e9700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f746c05f270 0x7f746c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.386 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.384+0000 7f748e1e9700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f746c05f270 0x7f746c061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f7478009c30 tx=0x7f7478009060).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:28.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.387+0000 7f747e7fc700 1 -- 172.21.15.29:0/3455615739 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7480047b00 con 0x7f7488157530 2024-01-24T15:30:28.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.580+0000 7f749044d700 1 -- 172.21.15.29:0/3455615739 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f74880475e0 con 0x7f7488157530 2024-01-24T15:30:28.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.583+0000 7f747e7fc700 1 -- 172.21.15.29:0/3455615739 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v37) v1 ==== 120+0+0 (secure 0 0 0) 0x7f748004acb0 con 0x7f7488157530 2024-01-24T15:30:28.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.585+0000 7f749044d700 1 -- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f746c05f270 msgr2=0x7f746c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.585+0000 7f749044d700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f746c05f270 0x7f746c061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f7478009c30 tx=0x7f7478009060).stop 2024-01-24T15:30:28.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.585+0000 7f749044d700 1 -- 172.21.15.29:0/3455615739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488157530 msgr2=0x7f74880a8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.585+0000 7f749044d700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488157530 0x7f74880a8b70 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f748000c7c0 tx=0x7f7480001510).stop 2024-01-24T15:30:28.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.585+0000 7f749044d700 1 -- 172.21.15.29:0/3455615739 shutdown_connections 2024-01-24T15:30:28.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.585+0000 7f749044d700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7488157530 0x7f74880a8b70 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.585+0000 7f749044d700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7488066a10 0x7f74880ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.585+0000 7f749044d700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f746c05f270 0x7f746c061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.585+0000 7f749044d700 1 --2- 172.21.15.29:0/3455615739 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7488068960 0x7f74880a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.585+0000 7f749044d700 1 -- 172.21.15.29:0/3455615739 >> 172.21.15.29:0/3455615739 conn(0x7f74880c8ba0 msgr2=0x7f7488151010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:28.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.586+0000 7f749044d700 1 -- 172.21.15.29:0/3455615739 shutdown_connections 2024-01-24T15:30:28.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.586+0000 7f749044d700 1 -- 172.21.15.29:0/3455615739 wait complete. 2024-01-24T15:30:28.600 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2024-01-24T15:30:28.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.730+0000 7fc864f3b700 1 Processor -- start 2024-01-24T15:30:28.732 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.730+0000 7fc864f3b700 1 -- start start 2024-01-24T15:30:28.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.731+0000 7fc864f3b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86014e840 0x7fc8600c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.731+0000 7fc864f3b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc860062ec0 0x7fc8600bc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.731+0000 7fc864f3b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8600c21e0 con 0x7fc8600c1d60 2024-01-24T15:30:28.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.731+0000 7fc864f3b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8600640f0 con 0x7fc860062ec0 2024-01-24T15:30:28.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.731+0000 7fc864f3b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8600634e0 con 0x7fc86014e840 2024-01-24T15:30:28.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.731+0000 7fc85ed9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc860062ec0 0x7fc8600bc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.731+0000 7fc85dd9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc8600c1d60 0x7fc8600627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46206/0 (socket says 172.21.15.29:46206) 2024-01-24T15:30:28.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.732+0000 7fc85dd9b700 1 -- 172.21.15.29:0/2765169257 learned_addr learned my addr 172.21.15.29:0/2765169257 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:28.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.732+0000 7fc85e59c700 1 --2- 172.21.15.29:0/2765169257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86014e840 0x7fc8600c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.732+0000 7fc85ed9d700 1 -- 172.21.15.29:0/2765169257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86014e840 msgr2=0x7fc8600c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:28.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.732+0000 7fc85ed9d700 1 --2- 172.21.15.29:0/2765169257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86014e840 0x7fc8600c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.732+0000 7fc85ed9d700 1 -- 172.21.15.29:0/2765169257 >> v1:172.21.15.29:6789/0 conn(0x7fc8600c1d60 legacy=0x7fc8600627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.732+0000 7fc85d59a700 1 -- 172.21.15.29:0/2765169257 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271716451 0 0) 0x7fc8600c21e0 con 0x7fc8600c1d60 2024-01-24T15:30:28.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.732+0000 7fc85ed9d700 1 -- 172.21.15.29:0/2765169257 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8600637c0 con 0x7fc860062ec0 2024-01-24T15:30:28.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.732+0000 7fc85e59c700 1 --2- 172.21.15.29:0/2765169257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86014e840 0x7fc8600c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:28.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.732+0000 7fc85ed9d700 1 --2- 172.21.15.29:0/2765169257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc860062ec0 0x7fc8600bc4b0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7fc858009e60 tx=0x7fc858009780).ready entity=mon.0 client_cookie=c6a3bf60b6470fa3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:28.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.732+0000 7fc85d59a700 1 -- 172.21.15.29:0/2765169257 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85800e070 con 0x7fc860062ec0 2024-01-24T15:30:28.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.732+0000 7fc85d59a700 1 -- 172.21.15.29:0/2765169257 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85800bc30 con 0x7fc860062ec0 2024-01-24T15:30:28.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.732+0000 7fc85d59a700 1 -- 172.21.15.29:0/2765169257 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858012410 con 0x7fc860062ec0 2024-01-24T15:30:28.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.733+0000 7fc864f3b700 1 -- 172.21.15.29:0/2765169257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc860062ec0 msgr2=0x7fc8600bc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.733+0000 7fc864f3b700 1 --2- 172.21.15.29:0/2765169257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc860062ec0 0x7fc8600bc4b0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7fc858009e60 tx=0x7fc858009780).stop 2024-01-24T15:30:28.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.733+0000 7fc864f3b700 1 -- 172.21.15.29:0/2765169257 shutdown_connections 2024-01-24T15:30:28.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.733+0000 7fc864f3b700 1 --2- 172.21.15.29:0/2765169257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc860062ec0 0x7fc8600bc4b0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.733+0000 7fc864f3b700 1 --2- 172.21.15.29:0/2765169257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86014e840 0x7fc8600c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.733+0000 7fc864f3b700 1 -- 172.21.15.29:0/2765169257 >> 172.21.15.29:0/2765169257 conn(0x7fc860064aa0 msgr2=0x7fc860064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:28.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.733+0000 7fc864f3b700 1 -- 172.21.15.29:0/2765169257 shutdown_connections 2024-01-24T15:30:28.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.733+0000 7fc864f3b700 1 -- 172.21.15.29:0/2765169257 wait complete. 2024-01-24T15:30:28.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.734+0000 7fc864f3b700 1 Processor -- start 2024-01-24T15:30:28.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.734+0000 7fc864f3b700 1 -- start start 2024-01-24T15:30:28.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc864f3b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc860062ec0 0x7fc860136f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc864f3b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8600c1d60 0x7fc8600c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc864f3b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc86014e840 0x7fc8600a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc864f3b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc860137470 con 0x7fc86014e840 2024-01-24T15:30:28.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc864f3b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8600c5c80 con 0x7fc8600c1d60 2024-01-24T15:30:28.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc864f3b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8600c6250 con 0x7fc860062ec0 2024-01-24T15:30:28.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc85ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc86014e840 0x7fc8600a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc85ed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc86014e840 0x7fc8600a5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52702/0 (socket says 172.21.15.29:52702) 2024-01-24T15:30:28.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc85ed9d700 1 -- 172.21.15.29:0/54129060 learned_addr learned my addr 172.21.15.29:0/54129060 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:28.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc85dd9b700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8600c1d60 0x7fc8600c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc85e59c700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc860062ec0 0x7fc860136f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc85ed9d700 1 -- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc860062ec0 msgr2=0x7fc860136f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc85ed9d700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc860062ec0 0x7fc860136f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc85ed9d700 1 -- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8600c1d60 msgr2=0x7fc8600c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:28.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc85ed9d700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8600c1d60 0x7fc8600c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:28.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc85ed9d700 1 -- 172.21.15.29:0/54129060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc858009a20 con 0x7fc86014e840 2024-01-24T15:30:28.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.735+0000 7fc85ed9d700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc86014e840 0x7fc8600a5850 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7fc85800b6c0 tx=0x7fc858009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:28.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.736+0000 7fc8567fc700 1 -- 172.21.15.29:0/54129060 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858011750 con 0x7fc86014e840 2024-01-24T15:30:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.736+0000 7fc8567fc700 1 -- 172.21.15.29:0/54129060 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8580118b0 con 0x7fc86014e840 2024-01-24T15:30:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.736+0000 7fc864f3b700 1 -- 172.21.15.29:0/54129060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8600c6530 con 0x7fc86014e840 2024-01-24T15:30:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.736+0000 7fc8567fc700 1 -- 172.21.15.29:0/54129060 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858011c70 con 0x7fc86014e840 2024-01-24T15:30:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.736+0000 7fc864f3b700 1 -- 172.21.15.29:0/54129060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8600c6af0 con 0x7fc86014e840 2024-01-24T15:30:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.737+0000 7fc864f3b700 1 -- 172.21.15.29:0/54129060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8600c6e20 con 0x7fc86014e840 2024-01-24T15:30:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.737+0000 7fc8567fc700 1 -- 172.21.15.29:0/54129060 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc858003040 con 0x7fc86014e840 2024-01-24T15:30:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.738+0000 7fc8567fc700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc84005f270 0x7fc840061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.738+0000 7fc8567fc700 1 -- 172.21.15.29:0/54129060 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 7108+0+0 (secure 0 0 0) 0x7fc85807c7b0 con 0x7fc86014e840 2024-01-24T15:30:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.738+0000 7fc85e59c700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc84005f270 0x7fc840061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:28.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.738+0000 7fc85e59c700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc84005f270 0x7fc840061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc8480088c0 tx=0x7fc848008040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:28.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.742+0000 7fc8567fc700 1 -- 172.21.15.29:0/54129060 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc85804d520 con 0x7fc86014e840 2024-01-24T15:30:28.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:28.997+0000 7fc864f3b700 1 -- 172.21.15.29:0/54129060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7fc860063aa0 con 0x7fc86014e840 2024-01-24T15:30:29.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.877+0000 7fc8567fc700 1 -- 172.21.15.29:0/54129060 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2963) v1 ==== 165+0+0 (secure 0 0 0) 0x7fc858050b40 con 0x7fc86014e840 2024-01-24T15:30:29.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.879+0000 7fc864f3b700 1 -- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc84005f270 msgr2=0x7fc840061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:29.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.879+0000 7fc864f3b700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc84005f270 0x7fc840061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc8480088c0 tx=0x7fc848008040).stop 2024-01-24T15:30:29.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.879+0000 7fc864f3b700 1 -- 172.21.15.29:0/54129060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc86014e840 msgr2=0x7fc8600a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:29.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.879+0000 7fc864f3b700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc86014e840 0x7fc8600a5850 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7fc85800b6c0 tx=0x7fc858009780).stop 2024-01-24T15:30:29.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.879+0000 7fc864f3b700 1 -- 172.21.15.29:0/54129060 shutdown_connections 2024-01-24T15:30:29.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.879+0000 7fc864f3b700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc86014e840 0x7fc8600a5850 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:29.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.879+0000 7fc864f3b700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc84005f270 0x7fc840061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:29.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.879+0000 7fc864f3b700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc860062ec0 0x7fc860136f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:29.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.879+0000 7fc864f3b700 1 --2- 172.21.15.29:0/54129060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8600c1d60 0x7fc8600c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:29.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.879+0000 7fc864f3b700 1 -- 172.21.15.29:0/54129060 >> 172.21.15.29:0/54129060 conn(0x7fc860064aa0 msgr2=0x7fc860153b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:29.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.880+0000 7fc864f3b700 1 -- 172.21.15.29:0/54129060 shutdown_connections 2024-01-24T15:30:29.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:29.880+0000 7fc864f3b700 1 -- 172.21.15.29:0/54129060 wait complete. 2024-01-24T15:30:29.896 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2024-01-24T15:30:29.896 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2024-01-24T15:30:29.896 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-01-24T15:30:29.897 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2024-01-24T15:30:30.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.030+0000 7f294ea60700 1 Processor -- start 2024-01-24T15:30:30.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.030+0000 7f294ea60700 1 -- start start 2024-01-24T15:30:30.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.031+0000 7f294ea60700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2948154370 0x7f29480c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:30.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.031+0000 7f294ea60700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2948154750 0x7f29480631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:30.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.031+0000 7f294ea60700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29480aa6c0 con 0x7f29480aa0e0 2024-01-24T15:30:30.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.031+0000 7f294ea60700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2948064240 con 0x7f2948154370 2024-01-24T15:30:30.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.031+0000 7f294ea60700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29480cb5c0 con 0x7f2948154750 2024-01-24T15:30:30.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.031+0000 7f2947fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2948154370 0x7f29480c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:30.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.031+0000 7f2947fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2948154370 0x7f29480c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52716/0 (socket says 172.21.15.29:52716) 2024-01-24T15:30:30.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.031+0000 7f29477fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f29480aa0e0 0x7f29480ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46220/0 (socket says 172.21.15.29:46220) 2024-01-24T15:30:30.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.031+0000 7f2947fff700 1 -- 172.21.15.29:0/4086279462 learned_addr learned my addr 172.21.15.29:0/4086279462 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:30.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.031+0000 7f294cffd700 1 --2- 172.21.15.29:0/4086279462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2948154750 0x7f29480631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:30.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.032+0000 7f2947fff700 1 -- 172.21.15.29:0/4086279462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2948154750 msgr2=0x7f29480631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:30.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.032+0000 7f2947fff700 1 --2- 172.21.15.29:0/4086279462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2948154750 0x7f29480631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:30.034 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.032+0000 7f2947fff700 1 -- 172.21.15.29:0/4086279462 >> v1:172.21.15.29:6789/0 conn(0x7f29480aa0e0 legacy=0x7f29480ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.032+0000 7f2947fff700 1 -- 172.21.15.29:0/4086279462 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29480cb8a0 con 0x7f2948154370 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.032+0000 7f2946ffd700 1 -- 172.21.15.29:0/4086279462 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601948501 0 0) 0x7f29480aa6c0 con 0x7f29480aa0e0 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.032+0000 7f294cffd700 1 --2- 172.21.15.29:0/4086279462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2948154750 0x7f29480631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.032+0000 7f2947fff700 1 --2- 172.21.15.29:0/4086279462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2948154370 0x7f29480c5410 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f2938009810 tx=0x7f2938009060).ready entity=mon.0 client_cookie=ca8354243e081e9f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.032+0000 7f2946ffd700 1 -- 172.21.15.29:0/4086279462 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938005880 con 0x7f2948154370 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.032+0000 7f2946ffd700 1 -- 172.21.15.29:0/4086279462 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29380059e0 con 0x7f2948154370 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.032+0000 7f2946ffd700 1 -- 172.21.15.29:0/4086279462 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938005d60 con 0x7f2948154370 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.032+0000 7f294ea60700 1 -- 172.21.15.29:0/4086279462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2948154370 msgr2=0x7f29480c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.032+0000 7f294ea60700 1 --2- 172.21.15.29:0/4086279462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2948154370 0x7f29480c5410 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f2938009810 tx=0x7f2938009060).stop 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.033+0000 7f294ea60700 1 -- 172.21.15.29:0/4086279462 shutdown_connections 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.033+0000 7f294ea60700 1 --2- 172.21.15.29:0/4086279462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2948154750 0x7f29480631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.033+0000 7f294ea60700 1 --2- 172.21.15.29:0/4086279462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2948154370 0x7f29480c5410 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:30.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.033+0000 7f294ea60700 1 -- 172.21.15.29:0/4086279462 >> 172.21.15.29:0/4086279462 conn(0x7f294805ec80 msgr2=0x7f29480cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.033+0000 7f294ea60700 1 -- 172.21.15.29:0/4086279462 shutdown_connections 2024-01-24T15:30:30.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.033+0000 7f294ea60700 1 -- 172.21.15.29:0/4086279462 wait complete. 2024-01-24T15:30:30.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.034+0000 7f294ea60700 1 Processor -- start 2024-01-24T15:30:30.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.034+0000 7f294ea60700 1 -- start start 2024-01-24T15:30:30.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294ea60700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29480aa0e0 0x7f2948144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:30.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294ea60700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2948154370 0x7f29480c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:30.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294ea60700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2948154750 0x7f29480c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:30.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294ea60700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29480c85f0 con 0x7f2948154750 2024-01-24T15:30:30.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294ea60700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29481590a0 con 0x7f2948154370 2024-01-24T15:30:30.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294ea60700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2948159c20 con 0x7f29480aa0e0 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294cffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2948154750 0x7f29480c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294cffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2948154750 0x7f29480c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52728/0 (socket says 172.21.15.29:52728) 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294cffd700 1 -- 172.21.15.29:0/3295215848 learned_addr learned my addr 172.21.15.29:0/3295215848 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f2947fff700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29480aa0e0 0x7f2948144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f29477fe700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2948154370 0x7f29480c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294cffd700 1 -- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29480aa0e0 msgr2=0x7f2948144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294cffd700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29480aa0e0 0x7f2948144c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294cffd700 1 -- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2948154370 msgr2=0x7f29480c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294cffd700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2948154370 0x7f29480c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f294cffd700 1 -- 172.21.15.29:0/3295215848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938009300 con 0x7f2948154750 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.035+0000 7f29477fe700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2948154370 0x7f29480c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.036+0000 7f294cffd700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2948154750 0x7f29480c8100 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7f294000d100 tx=0x7f29400099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.036+0000 7f292ffff700 1 -- 172.21.15.29:0/3295215848 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294000dcf0 con 0x7f2948154750 2024-01-24T15:30:30.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.036+0000 7f292ffff700 1 -- 172.21.15.29:0/3295215848 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294000f040 con 0x7f2948154750 2024-01-24T15:30:30.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.036+0000 7f294ea60700 1 -- 172.21.15.29:0/3295215848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29481592f0 con 0x7f2948154750 2024-01-24T15:30:30.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.036+0000 7f292ffff700 1 -- 172.21.15.29:0/3295215848 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940013600 con 0x7f2948154750 2024-01-24T15:30:30.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.036+0000 7f294ea60700 1 -- 172.21.15.29:0/3295215848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29481598b0 con 0x7f2948154750 2024-01-24T15:30:30.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.037+0000 7f294ea60700 1 -- 172.21.15.29:0/3295215848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29480475e0 con 0x7f2948154750 2024-01-24T15:30:30.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.037+0000 7f292ffff700 1 -- 172.21.15.29:0/3295215848 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f29400139b0 con 0x7f2948154750 2024-01-24T15:30:30.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.038+0000 7f292ffff700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f293405f270 0x7f2934061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:30.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.038+0000 7f292ffff700 1 -- 172.21.15.29:0/3295215848 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 7108+0+0 (secure 0 0 0) 0x7f294007adc0 con 0x7f2948154750 2024-01-24T15:30:30.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.038+0000 7f2947fff700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f293405f270 0x7f2934061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:30.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.038+0000 7f2947fff700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f293405f270 0x7f2934061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f293800b400 tx=0x7f2938016000).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:30.043 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.042+0000 7f292ffff700 1 -- 172.21.15.29:0/3295215848 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f294004bb30 con 0x7f2948154750 2024-01-24T15:30:30.293 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.291+0000 7f294ea60700 1 -- 172.21.15.29:0/3295215848 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f2948159d60 con 0x7f2948154750 2024-01-24T15:30:30.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.879+0000 7f292ffff700 1 -- 172.21.15.29:0/3295215848 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v2964) v1 ==== 166+0+0 (secure 0 0 0) 0x7f294007c070 con 0x7f2948154750 2024-01-24T15:30:30.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.881+0000 7f294ea60700 1 -- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f293405f270 msgr2=0x7f2934061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:30.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.881+0000 7f294ea60700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f293405f270 0x7f2934061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f293800b400 tx=0x7f2938016000).stop 2024-01-24T15:30:30.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.881+0000 7f294ea60700 1 -- 172.21.15.29:0/3295215848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2948154750 msgr2=0x7f29480c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:30.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.881+0000 7f294ea60700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2948154750 0x7f29480c8100 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7f294000d100 tx=0x7f29400099b0).stop 2024-01-24T15:30:30.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.881+0000 7f294ea60700 1 -- 172.21.15.29:0/3295215848 shutdown_connections 2024-01-24T15:30:30.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.881+0000 7f294ea60700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2948154750 0x7f29480c8100 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:30.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.881+0000 7f294ea60700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f293405f270 0x7f2934061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:30.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.881+0000 7f294ea60700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29480aa0e0 0x7f2948144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:30.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.881+0000 7f294ea60700 1 --2- 172.21.15.29:0/3295215848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2948154370 0x7f29480c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:30.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.881+0000 7f294ea60700 1 -- 172.21.15.29:0/3295215848 >> 172.21.15.29:0/3295215848 conn(0x7f294805ec80 msgr2=0x7f29481534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:30.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.882+0000 7f294ea60700 1 -- 172.21.15.29:0/3295215848 shutdown_connections 2024-01-24T15:30:30.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:30.882+0000 7f294ea60700 1 -- 172.21.15.29:0/3295215848 wait complete. 2024-01-24T15:30:30.898 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-24T15:30:31.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.034+0000 7f310c8fb700 1 Processor -- start 2024-01-24T15:30:31.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.034+0000 7f310c8fb700 1 -- start start 2024-01-24T15:30:31.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.035+0000 7f310c8fb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3108134ea0 0x7f31080c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.035+0000 7f310c8fb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3108066bd0 0x7f31080bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.035+0000 7f310c8fb700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31080686a0 con 0x7f31080c36c0 2024-01-24T15:30:31.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.035+0000 7f310c8fb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3108068b30 con 0x7f3108134ea0 2024-01-24T15:30:31.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.035+0000 7f310c8fb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3108067a90 con 0x7f3108066bd0 2024-01-24T15:30:31.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.035+0000 7f3105d9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f31080c36c0 0x7f31080664c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46230/0 (socket says 172.21.15.29:46230) 2024-01-24T15:30:31.039 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.035+0000 7f3105d9b700 1 -- 172.21.15.29:0/500604688 learned_addr learned my addr 172.21.15.29:0/500604688 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:31.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.035+0000 7f3106d9d700 1 --2- 172.21.15.29:0/500604688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3108066bd0 0x7f31080bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.035+0000 7f310659c700 1 --2- 172.21.15.29:0/500604688 >> v2:172.21.15.163:3300/0 conn(0x7f3108134ea0 0x7f31080c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.035+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488882131 0 0) 0x7f31080686a0 con 0x7f31080c36c0 2024-01-24T15:30:31.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30f00033f0 con 0x7f31080c36c0 2024-01-24T15:30:31.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2217140657 0 0) 0x7f30f00033f0 con 0x7f31080c36c0 2024-01-24T15:30:31.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3108066bd0 msgr2=0x7f31080bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:31.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 --2- 172.21.15.29:0/500604688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3108066bd0 0x7f31080bdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 >> v2:172.21.15.163:3300/0 conn(0x7f3108134ea0 msgr2=0x7f31080c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:31.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 --2- 172.21.15.29:0/500604688 >> v2:172.21.15.163:3300/0 conn(0x7f3108134ea0 0x7f31080c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3108067d70 con 0x7f31080c36c0 2024-01-24T15:30:31.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f30fc001470 con 0x7f31080c36c0 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 >> v1:172.21.15.29:6789/0 conn(0x7f31080c36c0 legacy=0x7f31080664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 --2- 172.21.15.29:0/500604688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3108066bd0 0x7f30f00038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 --2- 172.21.15.29:0/500604688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30f0003e10 0x7f30f0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 --2- 172.21.15.29:0/500604688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f0004b80 0x7f30f0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3108068b30 con 0x7f30f0003e10 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3108067a90 con 0x7f30f0004b80 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f000b8e0 con 0x7f3108066bd0 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f3105d9b700 1 --2- 172.21.15.29:0/500604688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30f0003e10 0x7f30f0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f310659c700 1 --2- 172.21.15.29:0/500604688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3108066bd0 0x7f30f00038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f3106d9d700 1 --2- 172.21.15.29:0/500604688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f0004b80 0x7f30f0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f3105d9b700 1 -- 172.21.15.29:0/500604688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3108066bd0 msgr2=0x7f30f00038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f3105d9b700 1 --2- 172.21.15.29:0/500604688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3108066bd0 0x7f30f00038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f3105d9b700 1 -- 172.21.15.29:0/500604688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f0004b80 msgr2=0x7f30f0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f3105d9b700 1 --2- 172.21.15.29:0/500604688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f0004b80 0x7f30f0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.036+0000 7f3105d9b700 1 -- 172.21.15.29:0/500604688 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f00033f0 con 0x7f30f0003e10 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.037+0000 7f3105d9b700 1 --2- 172.21.15.29:0/500604688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30f0003e10 0x7f30f0004690 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f30fc00c210 tx=0x7f30fc001e90).ready entity=mon.0 client_cookie=277f8ed230f56a84 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.037+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc007a60 con 0x7f30f0003e10 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.037+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30fc00caa0 con 0x7f30f0003e10 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.037+0000 7f310559a700 1 -- 172.21.15.29:0/500604688 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc00ce40 con 0x7f30f0003e10 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.037+0000 7f310c8fb700 1 -- 172.21.15.29:0/500604688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30f0003e10 msgr2=0x7f30f0004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.037+0000 7f310c8fb700 1 --2- 172.21.15.29:0/500604688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30f0003e10 0x7f30f0004690 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f30fc00c210 tx=0x7f30fc001e90).stop 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.037+0000 7f3106d9d700 1 -- 172.21.15.29:0/500604688 reap_dead start 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.037+0000 7f310c8fb700 1 -- 172.21.15.29:0/500604688 shutdown_connections 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.037+0000 7f310c8fb700 1 -- 172.21.15.29:0/500604688 >> 172.21.15.29:0/500604688 conn(0x7f31080c8bd0 msgr2=0x7f31080c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.037+0000 7f310c8fb700 1 -- 172.21.15.29:0/500604688 shutdown_connections 2024-01-24T15:30:31.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.037+0000 7f310c8fb700 1 -- 172.21.15.29:0/500604688 wait complete. 2024-01-24T15:30:31.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.039+0000 7f310c8fb700 1 Processor -- start 2024-01-24T15:30:31.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.039+0000 7f310c8fb700 1 -- start start 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.039+0000 7f310c8fb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f0003e10 0x7f3108151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.039+0000 7f310c8fb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31081515f0 0x7f31080ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.039+0000 7f310c8fb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31080add10 0x7f3108134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.039+0000 7f310c8fb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31080646e0 con 0x7f31081515f0 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.039+0000 7f310c8fb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3108063960 con 0x7f31080add10 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.039+0000 7f310c8fb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3108063c00 con 0x7f30f0003e10 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f3105d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31081515f0 0x7f31080ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f3105d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31081515f0 0x7f31080ad820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52744/0 (socket says 172.21.15.29:52744) 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f3105d9b700 1 -- 172.21.15.29:0/2270692177 learned_addr learned my addr 172.21.15.29:0/2270692177 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f3106d9d700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31080add10 0x7f3108134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f310659c700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f0003e10 0x7f3108151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f3105d9b700 1 -- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f0003e10 msgr2=0x7f3108151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f3105d9b700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f0003e10 0x7f3108151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f3105d9b700 1 -- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31080add10 msgr2=0x7f3108134110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f3105d9b700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31080add10 0x7f3108134110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f3105d9b700 1 -- 172.21.15.29:0/2270692177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3108063ee0 con 0x7f31081515f0 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f3106d9d700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31080add10 0x7f3108134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f310659c700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f0003e10 0x7f3108151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f3105d9b700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31081515f0 0x7f31080ad820 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f30f00045a0 tx=0x7f30fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f30f67fc700 1 -- 172.21.15.29:0/2270692177 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc007640 con 0x7f31081515f0 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f30f67fc700 1 -- 172.21.15.29:0/2270692177 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30fc0046b0 con 0x7f31081515f0 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f310c8fb700 1 -- 172.21.15.29:0/2270692177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31080641c0 con 0x7f31081515f0 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f30f67fc700 1 -- 172.21.15.29:0/2270692177 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc004a70 con 0x7f31081515f0 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.040+0000 7f310c8fb700 1 -- 172.21.15.29:0/2270692177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31080acf80 con 0x7f31081515f0 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.041+0000 7f310c8fb700 1 -- 172.21.15.29:0/2270692177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3108064400 con 0x7f31081515f0 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.042+0000 7f30f67fc700 1 -- 172.21.15.29:0/2270692177 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f30fc021460 con 0x7f31081515f0 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.042+0000 7f30f67fc700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f30e805f270 0x7f30e8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.042+0000 7f30f67fc700 1 -- 172.21.15.29:0/2270692177 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 7108+0+0 (secure 0 0 0) 0x7f30fc019020 con 0x7f31081515f0 2024-01-24T15:30:31.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.042+0000 7f310659c700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f30e805f270 0x7f30e8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.043+0000 7f310659c700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f30e805f270 0x7f30e8061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f30f8006d80 tx=0x7f30f8008040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:31.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.047+0000 7f30f67fc700 1 -- 172.21.15.29:0/2270692177 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3108064400 con 0x7f31081515f0 2024-01-24T15:30:31.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.228+0000 7f310c8fb700 1 -- 172.21.15.29:0/2270692177 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f31080646e0 con 0x7f31081515f0 2024-01-24T15:30:31.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.231+0000 7f30f67fc700 1 -- 172.21.15.29:0/2270692177 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) v1 ==== 122+0+0 (secure 0 0 0) 0x7f30fc04c640 con 0x7f31081515f0 2024-01-24T15:30:31.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.233+0000 7f310c8fb700 1 -- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f30e805f270 msgr2=0x7f30e8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.233+0000 7f310c8fb700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f30e805f270 0x7f30e8061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f30f8006d80 tx=0x7f30f8008040).stop 2024-01-24T15:30:31.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.233+0000 7f310c8fb700 1 -- 172.21.15.29:0/2270692177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31081515f0 msgr2=0x7f31080ad820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.233+0000 7f310c8fb700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31081515f0 0x7f31080ad820 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f30f00045a0 tx=0x7f30fc015000).stop 2024-01-24T15:30:31.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.233+0000 7f310c8fb700 1 -- 172.21.15.29:0/2270692177 shutdown_connections 2024-01-24T15:30:31.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.233+0000 7f310c8fb700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31080add10 0x7f3108134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.233+0000 7f310c8fb700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f30e805f270 0x7f30e8061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.233+0000 7f310c8fb700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f0003e10 0x7f3108151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.233+0000 7f310c8fb700 1 --2- 172.21.15.29:0/2270692177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31081515f0 0x7f31080ad820 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.233+0000 7f310c8fb700 1 -- 172.21.15.29:0/2270692177 >> 172.21.15.29:0/2270692177 conn(0x7f31080c8bd0 msgr2=0x7f310814b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:31.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.233+0000 7f310c8fb700 1 -- 172.21.15.29:0/2270692177 shutdown_connections 2024-01-24T15:30:31.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.233+0000 7f310c8fb700 1 -- 172.21.15.29:0/2270692177 wait complete. 2024-01-24T15:30:31.248 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T15:30:31.248 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:30:31.295 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:30:31.339 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T15:30:31.339 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-24T15:30:31.339 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:30:31.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.474+0000 7f7d3e0eb700 1 Processor -- start 2024-01-24T15:30:31.475 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.474+0000 7f7d3e0eb700 1 -- start start 2024-01-24T15:30:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.474+0000 7f7d3e0eb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d38138330 0x7f7d38138710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.474+0000 7f7d3e0eb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7d380627b0 0x7f7d38062c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.474+0000 7f7d3e0eb700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d38065770 con 0x7f7d380bca50 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.474+0000 7f7d3e0eb700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d38069cf0 con 0x7f7d38138330 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.474+0000 7f7d3e0eb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d380630f0 con 0x7f7d380627b0 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.475+0000 7f7d377fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d38138330 0x7f7d38138710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.475+0000 7f7d377fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d38138330 0x7f7d38138710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52756/0 (socket says 172.21.15.29:52756) 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.475+0000 7f7d377fe700 1 -- 172.21.15.29:0/1377465452 learned_addr learned my addr 172.21.15.29:0/1377465452 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.475+0000 7f7d37fff700 1 --2- 172.21.15.29:0/1377465452 >> v2:172.21.15.163:3300/0 conn(0x7f7d380627b0 0x7f7d38062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.475+0000 7f7d377fe700 1 -- 172.21.15.29:0/1377465452 >> v2:172.21.15.163:3300/0 conn(0x7f7d380627b0 msgr2=0x7f7d38062c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.475+0000 7f7d377fe700 1 --2- 172.21.15.29:0/1377465452 >> v2:172.21.15.163:3300/0 conn(0x7f7d380627b0 0x7f7d38062c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.475+0000 7f7d377fe700 1 -- 172.21.15.29:0/1377465452 >> v1:172.21.15.163:6789/0 conn(0x7f7d380bca50 legacy=0x7f7d380be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.475+0000 7f7d377fe700 1 -- 172.21.15.29:0/1377465452 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d38069f60 con 0x7f7d38138330 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.475+0000 7f7d377fe700 1 --2- 172.21.15.29:0/1377465452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d38138330 0x7f7d38138710 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7f7d380c5370 tx=0x7f7d280090f0).ready entity=mon.0 client_cookie=c25509d9b0a3423 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:31.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.475+0000 7f7d367fc700 1 -- 172.21.15.29:0/1377465452 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800b600 con 0x7f7d38138330 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.475+0000 7f7d367fc700 1 -- 172.21.15.29:0/1377465452 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2800b760 con 0x7f7d38138330 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.475+0000 7f7d367fc700 1 -- 172.21.15.29:0/1377465452 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800bae0 con 0x7f7d38138330 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.476+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/1377465452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d38138330 msgr2=0x7f7d38138710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.476+0000 7f7d3e0eb700 1 --2- 172.21.15.29:0/1377465452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d38138330 0x7f7d38138710 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7f7d380c5370 tx=0x7f7d280090f0).stop 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.476+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/1377465452 shutdown_connections 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.476+0000 7f7d3e0eb700 1 --2- 172.21.15.29:0/1377465452 >> v2:172.21.15.163:3300/0 conn(0x7f7d380627b0 0x7f7d38062c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.476+0000 7f7d3e0eb700 1 --2- 172.21.15.29:0/1377465452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d38138330 0x7f7d38138710 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.476+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/1377465452 >> 172.21.15.29:0/1377465452 conn(0x7f7d380c85c0 msgr2=0x7f7d380c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.476+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/1377465452 shutdown_connections 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.476+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/1377465452 wait complete. 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.477+0000 7f7d3e0eb700 1 Processor -- start 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.477+0000 7f7d3e0eb700 1 -- start start 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.478+0000 7f7d3e0eb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d380627b0 0x7f7d380a8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.483 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.478+0000 7f7d3e0eb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d380bca50 0x7f7d380a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.478+0000 7f7d3e0eb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d38138330 0x7f7d380a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.478+0000 7f7d3e0eb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d380a9210 con 0x7f7d380627b0 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.478+0000 7f7d3e0eb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d380c2da0 con 0x7f7d380bca50 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.478+0000 7f7d3e0eb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d380c2190 con 0x7f7d38138330 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.478+0000 7f7d377fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d380627b0 0x7f7d380a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.478+0000 7f7d377fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d380627b0 0x7f7d380a8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52768/0 (socket says 172.21.15.29:52768) 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.478+0000 7f7d377fe700 1 -- 172.21.15.29:0/3223880683 learned_addr learned my addr 172.21.15.29:0/3223880683 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d37fff700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d38138330 0x7f7d380a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d36ffd700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d380bca50 0x7f7d380a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d377fe700 1 -- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d38138330 msgr2=0x7f7d380a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d377fe700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d38138330 0x7f7d380a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d377fe700 1 -- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d380bca50 msgr2=0x7f7d380a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d377fe700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d380bca50 0x7f7d380a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d377fe700 1 -- 172.21.15.29:0/3223880683 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d28009370 con 0x7f7d380627b0 2024-01-24T15:30:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d377fe700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d380627b0 0x7f7d380a8830 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7f7d28009130 tx=0x7f7d28017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d1f7fe700 1 -- 172.21.15.29:0/3223880683 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800b8d0 con 0x7f7d380627b0 2024-01-24T15:30:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d1f7fe700 1 -- 172.21.15.29:0/3223880683 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2800ba30 con 0x7f7d380627b0 2024-01-24T15:30:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/3223880683 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d380c2470 con 0x7f7d380627b0 2024-01-24T15:30:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d1f7fe700 1 -- 172.21.15.29:0/3223880683 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2801b640 con 0x7f7d380627b0 2024-01-24T15:30:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.479+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/3223880683 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d380c2a30 con 0x7f7d380627b0 2024-01-24T15:30:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.480+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/3223880683 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d380475e0 con 0x7f7d380627b0 2024-01-24T15:30:31.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.481+0000 7f7d1f7fe700 1 -- 172.21.15.29:0/3223880683 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7d2801b9f0 con 0x7f7d380627b0 2024-01-24T15:30:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.481+0000 7f7d1f7fe700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7d2405f270 0x7f7d24061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.481+0000 7f7d1f7fe700 1 -- 172.21.15.29:0/3223880683 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 7108+0+0 (secure 0 0 0) 0x7f7d2807c220 con 0x7f7d380627b0 2024-01-24T15:30:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.481+0000 7f7d36ffd700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7d2405f270 0x7f7d24061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.482+0000 7f7d36ffd700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7d2405f270 0x7f7d24061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f7d380adb10 tx=0x7f7d2c009040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:31.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.485+0000 7f7d1f7fe700 1 -- 172.21.15.29:0/3223880683 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7d2804cf90 con 0x7f7d380627b0 2024-01-24T15:30:31.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.711+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/3223880683 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d38137bf0 con 0x7f7d380627b0 2024-01-24T15:30:31.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.711+0000 7f7d1f7fe700 1 -- 172.21.15.29:0/3223880683 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f7d2801fde0 con 0x7f7d380627b0 2024-01-24T15:30:31.713 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:31.713 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":84,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1350237946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1350237946},{"type":"v1","addr":"172.21.15.163:6836","nonce":1350237946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":8656,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/250387328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":250387328},{"type":"v1","addr":"172.21.15.163:6837","nonce":250387328}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":9069,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/3867710046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3867710046},{"type":"v1","addr":"172.21.15.29:6833","nonce":3867710046}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":9495,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3351112285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3351112285},{"type":"v1","addr":"172.21.15.29:6835","nonce":3351112285}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[]} 2024-01-24T15:30:31.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.713+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7d2405f270 msgr2=0x7f7d24061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.714+0000 7f7d3e0eb700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7d2405f270 0x7f7d24061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f7d380adb10 tx=0x7f7d2c009040).stop 2024-01-24T15:30:31.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.714+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/3223880683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d380627b0 msgr2=0x7f7d380a8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.714+0000 7f7d3e0eb700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d380627b0 0x7f7d380a8830 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7f7d28009130 tx=0x7f7d28017000).stop 2024-01-24T15:30:31.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.714+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/3223880683 shutdown_connections 2024-01-24T15:30:31.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.714+0000 7f7d3e0eb700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7d2405f270 0x7f7d24061730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.714+0000 7f7d3e0eb700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d38138330 0x7f7d380a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.714+0000 7f7d3e0eb700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d380627b0 0x7f7d380a8830 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.714+0000 7f7d3e0eb700 1 --2- 172.21.15.29:0/3223880683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d380bca50 0x7f7d380a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.714+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/3223880683 >> 172.21.15.29:0/3223880683 conn(0x7f7d380c85c0 msgr2=0x7f7d3814f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:31.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.714+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/3223880683 shutdown_connections 2024-01-24T15:30:31.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.714+0000 7f7d3e0eb700 1 -- 172.21.15.29:0/3223880683 wait complete. 2024-01-24T15:30:31.716 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 84 2024-01-24T15:30:31.729 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-24T15:30:31.729 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-24T15:30:31.729 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-24T15:30:31.729 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:30:31.729 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-24T15:30:31.729 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-24T15:30:31.730 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-24T15:30:31.730 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:30:31.730 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-24T15:30:31.730 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-24T15:30:31.730 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-24T15:30:31.730 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:30:31.730 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-24T15:30:31.731 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-24T15:30:31.731 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-24T15:30:31.731 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:30:31.739 INFO:tasks.ceph.mds.a:Stopped 2024-01-24T15:30:31.739 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-24T15:30:31.740 INFO:tasks.ceph.mds.c:Stopped 2024-01-24T15:30:31.740 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-24T15:30:31.741 INFO:tasks.ceph.mds.d:Stopped 2024-01-24T15:30:31.742 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-24T15:30:31.742 INFO:tasks.ceph.mds.b:Stopped 2024-01-24T15:30:31.742 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-24T15:30:31.782 INFO:tasks.ceph.mds.a:Started 2024-01-24T15:30:31.782 DEBUG:teuthology.parallel:result is None 2024-01-24T15:30:31.783 INFO:tasks.ceph.mds.c:Started 2024-01-24T15:30:31.783 DEBUG:teuthology.parallel:result is None 2024-01-24T15:30:31.784 INFO:tasks.ceph.mds.d:Started 2024-01-24T15:30:31.784 DEBUG:teuthology.parallel:result is None 2024-01-24T15:30:31.785 INFO:tasks.ceph.mds.b:Started 2024-01-24T15:30:31.785 DEBUG:teuthology.parallel:result is None 2024-01-24T15:30:31.785 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-24T15:30:31.868 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T15:30:31.868 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-01-24T15:30:31.869 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T15:30:31.870 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T15:30:31.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.924+0000 7facf3644700 1 Processor -- start 2024-01-24T15:30:31.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.925+0000 7facf3644700 1 -- start start 2024-01-24T15:30:31.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf3644700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7facec154240 0x7facec064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf3644700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7facec0a9d20 0x7facec0aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf3644700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec1550e0 con 0x7facec157cf0 2024-01-24T15:30:31.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf3644700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec0c0d10 con 0x7facec0a9d20 2024-01-24T15:30:31.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf3644700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec0aa670 con 0x7facec154240 2024-01-24T15:30:31.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf0bdf700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7facec0a9d20 0x7facec0aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf0bdf700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7facec0a9d20 0x7facec0aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52806/0 (socket says 172.21.15.29:52806) 2024-01-24T15:30:31.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf0bdf700 1 -- 172.21.15.29:0/2244942383 learned_addr learned my addr 172.21.15.29:0/2244942383 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:31.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf13e0700 1 --2- 172.21.15.29:0/2244942383 >> v2:172.21.15.163:3300/0 conn(0x7facec154240 0x7facec064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf0bdf700 1 -- 172.21.15.29:0/2244942383 >> v2:172.21.15.163:3300/0 conn(0x7facec154240 msgr2=0x7facec064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:31.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf0bdf700 1 --2- 172.21.15.29:0/2244942383 >> v2:172.21.15.163:3300/0 conn(0x7facec154240 0x7facec064780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf0bdf700 1 -- 172.21.15.29:0/2244942383 >> v1:172.21.15.163:6789/0 conn(0x7facec157cf0 legacy=0x7facec158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7face3fff700 1 -- 172.21.15.29:0/2244942383 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435238885 0 0) 0x7facec1550e0 con 0x7facec157cf0 2024-01-24T15:30:31.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.926+0000 7facf0bdf700 1 -- 172.21.15.29:0/2244942383 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec0650d0 con 0x7facec0a9d20 2024-01-24T15:30:31.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.927+0000 7facf0bdf700 1 --2- 172.21.15.29:0/2244942383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facec0a9d20 0x7facec0aa180 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7facec083d10 tx=0x7facdc0090f0).ready entity=mon.0 client_cookie=cebaef376a75f116 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:31.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.927+0000 7face3fff700 1 -- 172.21.15.29:0/2244942383 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc00b600 con 0x7facec0a9d20 2024-01-24T15:30:31.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.927+0000 7face3fff700 1 -- 172.21.15.29:0/2244942383 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc00b760 con 0x7facec0a9d20 2024-01-24T15:30:31.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.927+0000 7face3fff700 1 -- 172.21.15.29:0/2244942383 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc00bae0 con 0x7facec0a9d20 2024-01-24T15:30:31.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.927+0000 7facf3644700 1 -- 172.21.15.29:0/2244942383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facec0a9d20 msgr2=0x7facec0aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.927+0000 7facf3644700 1 --2- 172.21.15.29:0/2244942383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facec0a9d20 0x7facec0aa180 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7facec083d10 tx=0x7facdc0090f0).stop 2024-01-24T15:30:31.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.927+0000 7facf3644700 1 -- 172.21.15.29:0/2244942383 shutdown_connections 2024-01-24T15:30:31.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.927+0000 7facf3644700 1 --2- 172.21.15.29:0/2244942383 >> v2:172.21.15.163:3300/0 conn(0x7facec154240 0x7facec064780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.927+0000 7facf3644700 1 --2- 172.21.15.29:0/2244942383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facec0a9d20 0x7facec0aa180 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.927+0000 7facf3644700 1 -- 172.21.15.29:0/2244942383 >> 172.21.15.29:0/2244942383 conn(0x7facec0c8310 msgr2=0x7facec05e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:31.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.927+0000 7facf3644700 1 -- 172.21.15.29:0/2244942383 shutdown_connections 2024-01-24T15:30:31.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.927+0000 7facf3644700 1 -- 172.21.15.29:0/2244942383 wait complete. 2024-01-24T15:30:31.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.929+0000 7facf3644700 1 Processor -- start 2024-01-24T15:30:31.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.929+0000 7facf3644700 1 -- start start 2024-01-24T15:30:31.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf3644700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec0a9d20 0x7facec13c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf3644700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facec154240 0x7facec0add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf3644700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec157cf0 0x7facec0ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf3644700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec0ae880 con 0x7facec154240 2024-01-24T15:30:31.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf3644700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec164e80 con 0x7facec157cf0 2024-01-24T15:30:31.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf3644700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec0afd60 con 0x7facec0a9d20 2024-01-24T15:30:31.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf0bdf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facec154240 0x7facec0add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf0bdf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facec154240 0x7facec0add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52816/0 (socket says 172.21.15.29:52816) 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf0bdf700 1 -- 172.21.15.29:0/2228070501 learned_addr learned my addr 172.21.15.29:0/2228070501 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf13e0700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec0a9d20 0x7facec13c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf1be1700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec157cf0 0x7facec0ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf0bdf700 1 -- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec0a9d20 msgr2=0x7facec13c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf0bdf700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec0a9d20 0x7facec13c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf0bdf700 1 -- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec157cf0 msgr2=0x7facec0ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf0bdf700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec157cf0 0x7facec0ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf0bdf700 1 -- 172.21.15.29:0/2228070501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc009370 con 0x7facec154240 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7facf0bdf700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facec154240 0x7facec0add30 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7facdc009130 tx=0x7facdc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.930+0000 7face17fa700 1 -- 172.21.15.29:0/2228070501 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc00bea0 con 0x7facec154240 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.931+0000 7face17fa700 1 -- 172.21.15.29:0/2228070501 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc00b8d0 con 0x7facec154240 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.931+0000 7facf3644700 1 -- 172.21.15.29:0/2228070501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec0b08c0 con 0x7facec154240 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.931+0000 7face17fa700 1 -- 172.21.15.29:0/2228070501 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc01c670 con 0x7facec154240 2024-01-24T15:30:31.938 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.931+0000 7facf3644700 1 -- 172.21.15.29:0/2228070501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec0b0d50 con 0x7facec154240 2024-01-24T15:30:31.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.932+0000 7facf3644700 1 -- 172.21.15.29:0/2228070501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facec0b0a50 con 0x7facec154240 2024-01-24T15:30:31.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.932+0000 7face17fa700 1 -- 172.21.15.29:0/2228070501 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7facdc01c9e0 con 0x7facec154240 2024-01-24T15:30:31.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.932+0000 7face17fa700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7facd005f240 0x7facd0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:31.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.932+0000 7face17fa700 1 -- 172.21.15.29:0/2228070501 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 7108+0+0 (secure 0 0 0) 0x7facdc07bfb0 con 0x7facec154240 2024-01-24T15:30:31.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.933+0000 7facf13e0700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7facd005f240 0x7facd0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:31.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.933+0000 7facf13e0700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7facd005f240 0x7facd0061700 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7facec0ad370 tx=0x7face8008040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:31.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:31.937+0000 7face17fa700 1 -- 172.21.15.29:0/2228070501 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7facdc04cc60 con 0x7facec154240 2024-01-24T15:30:31.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-01-24T15:30:31.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-01-24T15:30:32.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.132+0000 7facf3644700 1 -- 172.21.15.29:0/2228070501 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7facec0475e0 con 0x7facec154240 2024-01-24T15:30:32.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.968+0000 7face17fa700 1 -- 172.21.15.29:0/2228070501 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v209) v1 ==== 97+0+0 (secure 0 0 0) 0x7facdc020020 con 0x7facec154240 2024-01-24T15:30:32.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.971+0000 7facf3644700 1 -- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7facd005f240 msgr2=0x7facd0061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:32.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.971+0000 7facf3644700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7facd005f240 0x7facd0061700 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7facec0ad370 tx=0x7face8008040).stop 2024-01-24T15:30:32.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.971+0000 7facf3644700 1 -- 172.21.15.29:0/2228070501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facec154240 msgr2=0x7facec0add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:32.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.971+0000 7facf3644700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facec154240 0x7facec0add30 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7facdc009130 tx=0x7facdc017000).stop 2024-01-24T15:30:32.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.971+0000 7facf3644700 1 -- 172.21.15.29:0/2228070501 shutdown_connections 2024-01-24T15:30:32.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.971+0000 7facf3644700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec157cf0 0x7facec0ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:32.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.971+0000 7facf3644700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7facd005f240 0x7facd0061700 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:32.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.971+0000 7facf3644700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec0a9d20 0x7facec13c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:32.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.971+0000 7facf3644700 1 --2- 172.21.15.29:0/2228070501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facec154240 0x7facec0add30 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:32.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.971+0000 7facf3644700 1 -- 172.21.15.29:0/2228070501 >> 172.21.15.29:0/2228070501 conn(0x7facec0c8310 msgr2=0x7facec0c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:32.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.971+0000 7facf3644700 1 -- 172.21.15.29:0/2228070501 shutdown_connections 2024-01-24T15:30:32.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:32.971+0000 7facf3644700 1 -- 172.21.15.29:0/2228070501 wait complete. 2024-01-24T15:30:32.974 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-01-24T15:30:32.988 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-24T15:30:33.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.118+0000 7f4c973d6700 1 Processor -- start 2024-01-24T15:30:33.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.119+0000 7f4c973d6700 1 -- start start 2024-01-24T15:30:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.119+0000 7f4c973d6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4c900c4a10 0x7f4c900bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.119+0000 7f4c973d6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c90066c00 0x7f4c90158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.119+0000 7f4c973d6700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c901582b0 con 0x7f4c900670a0 2024-01-24T15:30:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c973d6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c90157ca0 con 0x7f4c900c4a10 2024-01-24T15:30:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c973d6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90063c60 con 0x7f4c90066c00 2024-01-24T15:30:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c94971700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4c900670a0 0x7f4c900664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:46256/0 (socket says 172.21.15.29:46256) 2024-01-24T15:30:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c94971700 1 -- 172.21.15.29:0/876811423 learned_addr learned my addr 172.21.15.29:0/876811423 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c95172700 1 --2- 172.21.15.29:0/876811423 >> v2:172.21.15.163:3300/0 conn(0x7f4c900c4a10 0x7f4c900bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c95973700 1 --2- 172.21.15.29:0/876811423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c90066c00 0x7f4c90158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c87fff700 1 -- 172.21.15.29:0/876811423 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557181376 0 0) 0x7f4c901582b0 con 0x7f4c900670a0 2024-01-24T15:30:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c87fff700 1 -- 172.21.15.29:0/876811423 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c7c0033f0 con 0x7f4c900670a0 2024-01-24T15:30:33.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c95172700 1 -- 172.21.15.29:0/876811423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c90066c00 msgr2=0x7f4c90158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:33.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c95172700 1 --2- 172.21.15.29:0/876811423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c90066c00 0x7f4c90158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c95172700 1 -- 172.21.15.29:0/876811423 >> v1:172.21.15.29:6789/0 conn(0x7f4c900670a0 legacy=0x7f4c900664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c95172700 1 -- 172.21.15.29:0/876811423 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c90157f40 con 0x7f4c900c4a10 2024-01-24T15:30:33.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c95973700 1 --2- 172.21.15.29:0/876811423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c90066c00 0x7f4c90158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:33.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.120+0000 7f4c95172700 1 --2- 172.21.15.29:0/876811423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c900c4a10 0x7f4c900bf6d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f4c8c009760 tx=0x7f4c8c009060).ready entity=mon.1 client_cookie=4e996e143baf6500 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:33.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.121+0000 7f4c87fff700 1 -- 172.21.15.29:0/876811423 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00d4a0 con 0x7f4c900c4a10 2024-01-24T15:30:33.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.121+0000 7f4c87fff700 1 -- 172.21.15.29:0/876811423 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00d600 con 0x7f4c900c4a10 2024-01-24T15:30:33.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.121+0000 7f4c87fff700 1 -- 172.21.15.29:0/876811423 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00d980 con 0x7f4c900c4a10 2024-01-24T15:30:33.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.121+0000 7f4c973d6700 1 -- 172.21.15.29:0/876811423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c900c4a10 msgr2=0x7f4c900bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.121+0000 7f4c973d6700 1 --2- 172.21.15.29:0/876811423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c900c4a10 0x7f4c900bf6d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f4c8c009760 tx=0x7f4c8c009060).stop 2024-01-24T15:30:33.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.121+0000 7f4c973d6700 1 -- 172.21.15.29:0/876811423 shutdown_connections 2024-01-24T15:30:33.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.121+0000 7f4c973d6700 1 --2- 172.21.15.29:0/876811423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c90066c00 0x7f4c90158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.121+0000 7f4c973d6700 1 --2- 172.21.15.29:0/876811423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c900c4a10 0x7f4c900bf6d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.121+0000 7f4c973d6700 1 -- 172.21.15.29:0/876811423 >> 172.21.15.29:0/876811423 conn(0x7f4c900c8bb0 msgr2=0x7f4c900c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:33.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.121+0000 7f4c973d6700 1 -- 172.21.15.29:0/876811423 shutdown_connections 2024-01-24T15:30:33.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.121+0000 7f4c973d6700 1 -- 172.21.15.29:0/876811423 wait complete. 2024-01-24T15:30:33.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.122+0000 7f4c973d6700 1 Processor -- start 2024-01-24T15:30:33.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.123+0000 7f4c973d6700 1 -- start start 2024-01-24T15:30:33.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.123+0000 7f4c973d6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c90066c00 0x7f4c901530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.123+0000 7f4c973d6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c900670a0 0x7f4c901644b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c973d6700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c900c4a10 0x7f4c901649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c973d6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900c25f0 con 0x7f4c90066c00 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c973d6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90154670 con 0x7f4c900670a0 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c973d6700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c901551b0 con 0x7f4c900c4a10 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c95172700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c90066c00 0x7f4c901530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c95172700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c90066c00 0x7f4c901530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52824/0 (socket says 172.21.15.29:52824) 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c95172700 1 -- 172.21.15.29:0/4668538 learned_addr learned my addr 172.21.15.29:0/4668538 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c95973700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c900c4a10 0x7f4c901649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c94971700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c900670a0 0x7f4c901644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c95172700 1 -- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c900c4a10 msgr2=0x7f4c901649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c95172700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c900c4a10 0x7f4c901649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c95172700 1 -- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c900670a0 msgr2=0x7f4c901644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c95172700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c900670a0 0x7f4c901644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c95172700 1 -- 172.21.15.29:0/4668538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c88009a20 con 0x7f4c90066c00 2024-01-24T15:30:33.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c95172700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c90066c00 0x7f4c901530d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7f4c8c009d50 tx=0x7f4c8c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:33.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c857fa700 1 -- 172.21.15.29:0/4668538 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00dd30 con 0x7f4c90066c00 2024-01-24T15:30:33.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c857fa700 1 -- 172.21.15.29:0/4668538 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00de90 con 0x7f4c90066c00 2024-01-24T15:30:33.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c973d6700 1 -- 172.21.15.29:0/4668538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8c0092a0 con 0x7f4c90066c00 2024-01-24T15:30:33.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c857fa700 1 -- 172.21.15.29:0/4668538 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00d980 con 0x7f4c90066c00 2024-01-24T15:30:33.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.124+0000 7f4c973d6700 1 -- 172.21.15.29:0/4668538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c90154b60 con 0x7f4c90066c00 2024-01-24T15:30:33.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.125+0000 7f4c973d6700 1 -- 172.21.15.29:0/4668538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c900688c0 con 0x7f4c90066c00 2024-01-24T15:30:33.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.126+0000 7f4c857fa700 1 -- 172.21.15.29:0/4668538 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4c8c01d640 con 0x7f4c90066c00 2024-01-24T15:30:33.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.126+0000 7f4c857fa700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4c7405f270 0x7f4c74061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.126+0000 7f4c857fa700 1 -- 172.21.15.29:0/4668538 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6334+0+0 (secure 0 0 0) 0x7f4c8c07bcd0 con 0x7f4c90066c00 2024-01-24T15:30:33.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.127+0000 7f4c94971700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4c7405f270 0x7f4c74061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.127+0000 7f4c94971700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4c7405f270 0x7f4c74061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4c900c37d0 tx=0x7f4c80009040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:33.132 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.131+0000 7f4c857fa700 1 -- 172.21.15.29:0/4668538 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4c8c04cd50 con 0x7f4c90066c00 2024-01-24T15:30:33.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.381+0000 7f4c973d6700 1 -- 172.21.15.29:0/4668538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4c900475e0 con 0x7f4c90066c00 2024-01-24T15:30:33.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.382+0000 7f4c857fa700 1 -- 172.21.15.29:0/4668538 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v52) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4c8c050370 con 0x7f4c90066c00 2024-01-24T15:30:33.384 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:33.384 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:30:33.384 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-01-24T15:30:33.384 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.384 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-01-24T15:30:33.384 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew==", 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ==", 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:30:33.385 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw==", 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.386 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ==", 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ==", 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:30:33.387 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA==", 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.388 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw==", 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ==", 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:30:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw==", 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ==", 2024-01-24T15:30:33.390 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A==", 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ==", 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw==", 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:30:33.392 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA==", 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.393 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-01-24T15:30:33.394 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g==", 2024-01-24T15:30:33.394 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.394 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:30:33.394 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T15:30:33.394 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T15:30:33.394 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T15:30:33.394 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.394 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-01-24T15:30:33.394 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ==", 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlxK1bFxAAZpfJn+Tcpmq7R9Yq29FvVg==", 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:30:33.395 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw==", 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-01-24T15:30:33.396 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA==", 2024-01-24T15:30:33.397 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:30:33.397 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T15:30:33.397 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T15:30:33.397 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:30:33.397 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.397 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:33.397 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-01-24T15:30:33.397 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:30:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.383+0000 7f4c973d6700 1 -- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4c7405f270 msgr2=0x7f4c74061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.383+0000 7f4c973d6700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4c7405f270 0x7f4c74061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4c900c37d0 tx=0x7f4c80009040).stop 2024-01-24T15:30:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.383+0000 7f4c973d6700 1 -- 172.21.15.29:0/4668538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c90066c00 msgr2=0x7f4c901530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.383+0000 7f4c973d6700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c90066c00 0x7f4c901530d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7f4c8c009d50 tx=0x7f4c8c016000).stop 2024-01-24T15:30:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.384+0000 7f4c973d6700 1 -- 172.21.15.29:0/4668538 shutdown_connections 2024-01-24T15:30:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.384+0000 7f4c973d6700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4c7405f270 0x7f4c74061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.384+0000 7f4c973d6700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c900c4a10 0x7f4c901649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.384+0000 7f4c973d6700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c90066c00 0x7f4c901530d0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.384+0000 7f4c973d6700 1 --2- 172.21.15.29:0/4668538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c900670a0 0x7f4c901644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.384+0000 7f4c973d6700 1 -- 172.21.15.29:0/4668538 >> 172.21.15.29:0/4668538 conn(0x7f4c900c8bb0 msgr2=0x7f4c900c7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.384+0000 7f4c973d6700 1 -- 172.21.15.29:0/4668538 shutdown_connections 2024-01-24T15:30:33.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.384+0000 7f4c973d6700 1 -- 172.21.15.29:0/4668538 wait complete. 2024-01-24T15:30:33.400 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:30:33.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.529+0000 7fcca3ed0700 1 Processor -- start 2024-01-24T15:30:33.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.529+0000 7fcca3ed0700 1 -- start start 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca3ed0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc9c13a5c0 0x7fcc9c063a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca3ed0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc9c066c00 0x7fcc9c0c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca3ed0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc9c0bf6a0 con 0x7fcc9c0be520 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca3ed0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc9c0bea90 con 0x7fcc9c066c00 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca3ed0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c0bed70 con 0x7fcc9c13a5c0 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca246d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc9c066c00 0x7fcc9c0c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca246d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcc9c066c00 0x7fcc9c0c31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37156/0 (socket says 172.21.15.29:37156) 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca246d700 1 -- 172.21.15.29:0/1602121918 learned_addr learned my addr 172.21.15.29:0/1602121918 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca1c6c700 1 --2- 172.21.15.29:0/1602121918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc9c13a5c0 0x7fcc9c063a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca246d700 1 -- 172.21.15.29:0/1602121918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc9c13a5c0 msgr2=0x7fcc9c063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca246d700 1 --2- 172.21.15.29:0/1602121918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc9c13a5c0 0x7fcc9c063a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca246d700 1 -- 172.21.15.29:0/1602121918 >> v1:172.21.15.163:6789/0 conn(0x7fcc9c0be520 legacy=0x7fcc9c0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca246d700 1 -- 172.21.15.29:0/1602121918 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9c0bf050 con 0x7fcc9c066c00 2024-01-24T15:30:33.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.530+0000 7fcca246d700 1 --2- 172.21.15.29:0/1602121918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9c066c00 0x7fcc9c0c31b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7fcc94009f20 tx=0x7fcc94009840).ready entity=mon.0 client_cookie=37629453f4669864 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:33.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.531+0000 7fcca0c6a700 1 -- 172.21.15.29:0/1602121918 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400e070 con 0x7fcc9c066c00 2024-01-24T15:30:33.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.531+0000 7fcca0c6a700 1 -- 172.21.15.29:0/1602121918 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9400ba80 con 0x7fcc9c066c00 2024-01-24T15:30:33.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.531+0000 7fcca0c6a700 1 -- 172.21.15.29:0/1602121918 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400be00 con 0x7fcc9c066c00 2024-01-24T15:30:33.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.531+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1602121918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9c066c00 msgr2=0x7fcc9c0c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.531+0000 7fcca3ed0700 1 --2- 172.21.15.29:0/1602121918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9c066c00 0x7fcc9c0c31b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7fcc94009f20 tx=0x7fcc94009840).stop 2024-01-24T15:30:33.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.531+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1602121918 shutdown_connections 2024-01-24T15:30:33.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.531+0000 7fcca3ed0700 1 --2- 172.21.15.29:0/1602121918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc9c13a5c0 0x7fcc9c063a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.531+0000 7fcca3ed0700 1 --2- 172.21.15.29:0/1602121918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9c066c00 0x7fcc9c0c31b0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.531+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1602121918 >> 172.21.15.29:0/1602121918 conn(0x7fcc9c0c8bb0 msgr2=0x7fcc9c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:33.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.531+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1602121918 shutdown_connections 2024-01-24T15:30:33.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.531+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1602121918 wait complete. 2024-01-24T15:30:33.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.533+0000 7fcca3ed0700 1 Processor -- start 2024-01-24T15:30:33.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.533+0000 7fcca3ed0700 1 -- start start 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.533+0000 7fcca3ed0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc9c066c00 0x7fcc9c13d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.533+0000 7fcca3ed0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9c0be520 0x7fcc9c137880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.533+0000 7fcca3ed0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc9c13a5c0 0x7fcc9c138290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.533+0000 7fcca3ed0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c0a84e0 con 0x7fcc9c0be520 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.533+0000 7fcca3ed0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c0a9340 con 0x7fcc9c13a5c0 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.533+0000 7fcca3ed0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c165b30 con 0x7fcc9c066c00 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca146b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9c0be520 0x7fcc9c137880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca146b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9c0be520 0x7fcc9c137880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37162/0 (socket says 172.21.15.29:37162) 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca146b700 1 -- 172.21.15.29:0/1100548170 learned_addr learned my addr 172.21.15.29:0/1100548170 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca246d700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc9c13a5c0 0x7fcc9c138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca1c6c700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc9c066c00 0x7fcc9c13d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca146b700 1 -- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc9c066c00 msgr2=0x7fcc9c13d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca146b700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc9c066c00 0x7fcc9c13d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca146b700 1 -- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc9c13a5c0 msgr2=0x7fcc9c138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca146b700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc9c13a5c0 0x7fcc9c138290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca146b700 1 -- 172.21.15.29:0/1100548170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc94009ae0 con 0x7fcc9c0be520 2024-01-24T15:30:33.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca1c6c700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc9c066c00 0x7fcc9c13d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:33.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca146b700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9c0be520 0x7fcc9c137880 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7fcc8c00c4c0 tx=0x7fcc8c007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:33.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcc91ffb700 1 -- 172.21.15.29:0/1100548170 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c007020 con 0x7fcc9c0be520 2024-01-24T15:30:33.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcc91ffb700 1 -- 172.21.15.29:0/1100548170 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c007180 con 0x7fcc9c0be520 2024-01-24T15:30:33.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1100548170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9c0a8730 con 0x7fcc9c0be520 2024-01-24T15:30:33.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.534+0000 7fcc91ffb700 1 -- 172.21.15.29:0/1100548170 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c007520 con 0x7fcc9c0be520 2024-01-24T15:30:33.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.535+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1100548170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9c0a8cf0 con 0x7fcc9c0be520 2024-01-24T15:30:33.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.536+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1100548170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9c0a94b0 con 0x7fcc9c0be520 2024-01-24T15:30:33.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.536+0000 7fcc91ffb700 1 -- 172.21.15.29:0/1100548170 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcc8c0068b0 con 0x7fcc9c0be520 2024-01-24T15:30:33.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.536+0000 7fcc91ffb700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc8005f270 0x7fcc80061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.536+0000 7fcc91ffb700 1 -- 172.21.15.29:0/1100548170 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6334+0+0 (secure 0 0 0) 0x7fcc8c079800 con 0x7fcc9c0be520 2024-01-24T15:30:33.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.536+0000 7fcca1c6c700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc8005f270 0x7fcc80061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.537+0000 7fcca1c6c700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc8005f270 0x7fcc80061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcc9800bb60 tx=0x7fcc98008040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:33.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.541+0000 7fcc91ffb700 1 -- 172.21.15.29:0/1100548170 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcc8c047b00 con 0x7fcc9c0be520 2024-01-24T15:30:33.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.723+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1100548170 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc9c0475e0 con 0x7fcc9c0be520 2024-01-24T15:30:33.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.724+0000 7fcc91ffb700 1 -- 172.21.15.29:0/1100548170 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v209) v1 ==== 74+0+15037 (secure 0 0 0) 0x7fcc8c04a880 con 0x7fcc9c0be520 2024-01-24T15:30:33.726 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:33.727 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":209,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:30:32.919959+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:30:33.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.726+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc8005f270 msgr2=0x7fcc80061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.726+0000 7fcca3ed0700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc8005f270 0x7fcc80061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcc9800bb60 tx=0x7fcc98008040).stop 2024-01-24T15:30:33.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.726+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1100548170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9c0be520 msgr2=0x7fcc9c137880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.726+0000 7fcca3ed0700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9c0be520 0x7fcc9c137880 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7fcc8c00c4c0 tx=0x7fcc8c007dd0).stop 2024-01-24T15:30:33.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.726+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1100548170 shutdown_connections 2024-01-24T15:30:33.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.726+0000 7fcca3ed0700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc9c13a5c0 0x7fcc9c138290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.726+0000 7fcca3ed0700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc8005f270 0x7fcc80061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.726+0000 7fcca3ed0700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc9c066c00 0x7fcc9c13d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.726+0000 7fcca3ed0700 1 --2- 172.21.15.29:0/1100548170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc9c0be520 0x7fcc9c137880 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.726+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1100548170 >> 172.21.15.29:0/1100548170 conn(0x7fcc9c0c8bb0 msgr2=0x7fcc9c0ac3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:33.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.726+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1100548170 shutdown_connections 2024-01-24T15:30:33.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.726+0000 7fcca3ed0700 1 -- 172.21.15.29:0/1100548170 wait complete. 2024-01-24T15:30:33.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '201', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T15:30:33.743 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T15:30:33.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.874+0000 7f3be2f4d700 1 Processor -- start 2024-01-24T15:30:33.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.874+0000 7f3be2f4d700 1 -- start start 2024-01-24T15:30:33.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.874+0000 7f3be2f4d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3bdc0a7210 0x7f3bdc0c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.874+0000 7f3be2f4d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bdc062ec0 0x7f3bdc065420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.874+0000 7f3be2f4d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bdc0a6d80 con 0x7f3bdc154100 2024-01-24T15:30:33.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.874+0000 7f3be2f4d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bdc0bcaf0 con 0x7f3bdc0a7210 2024-01-24T15:30:33.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.874+0000 7f3be2f4d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0bcd90 con 0x7f3bdc062ec0 2024-01-24T15:30:33.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3be0ce9700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3bdc154100 0x7f3bdc0627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41902/0 (socket says 172.21.15.29:41902) 2024-01-24T15:30:33.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3be0ce9700 1 -- 172.21.15.29:0/674763765 learned_addr learned my addr 172.21.15.29:0/674763765 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:33.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3be14ea700 1 --2- 172.21.15.29:0/674763765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bdc062ec0 0x7f3bdc065420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3bdbfff700 1 --2- 172.21.15.29:0/674763765 >> v2:172.21.15.163:3300/0 conn(0x7f3bdc0a7210 0x7f3bdc0c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3bdb7fe700 1 -- 172.21.15.29:0/674763765 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043532769 0 0) 0x7f3bdc0a6d80 con 0x7f3bdc154100 2024-01-24T15:30:33.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3bdb7fe700 1 -- 172.21.15.29:0/674763765 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bcc0033f0 con 0x7f3bdc154100 2024-01-24T15:30:33.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3bdb7fe700 1 -- 172.21.15.29:0/674763765 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2331601736 0 0) 0x7f3bcc0033f0 con 0x7f3bdc154100 2024-01-24T15:30:33.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3bdbfff700 1 -- 172.21.15.29:0/674763765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bdc062ec0 msgr2=0x7f3bdc065420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:33.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3bdbfff700 1 --2- 172.21.15.29:0/674763765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bdc062ec0 0x7f3bdc065420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3bdbfff700 1 -- 172.21.15.29:0/674763765 >> v1:172.21.15.29:6789/0 conn(0x7f3bdc154100 legacy=0x7f3bdc0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3bdbfff700 1 -- 172.21.15.29:0/674763765 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc0bd070 con 0x7f3bdc0a7210 2024-01-24T15:30:33.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3be14ea700 1 --2- 172.21.15.29:0/674763765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bdc062ec0 0x7f3bdc065420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:33.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.875+0000 7f3bdbfff700 1 --2- 172.21.15.29:0/674763765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bdc0a7210 0x7f3bdc0c8200 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f3bc8009760 tx=0x7f3bc8009060).ready entity=mon.1 client_cookie=8a8e83edfab74677 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:33.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.876+0000 7f3bdb7fe700 1 -- 172.21.15.29:0/674763765 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0001470 con 0x7f3bdc0a7210 2024-01-24T15:30:33.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.876+0000 7f3bdb7fe700 1 -- 172.21.15.29:0/674763765 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc8011400 con 0x7f3bdc0a7210 2024-01-24T15:30:33.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.876+0000 7f3bdb7fe700 1 -- 172.21.15.29:0/674763765 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8011780 con 0x7f3bdc0a7210 2024-01-24T15:30:33.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.876+0000 7f3be2f4d700 1 -- 172.21.15.29:0/674763765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bdc0a7210 msgr2=0x7f3bdc0c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.876+0000 7f3be2f4d700 1 --2- 172.21.15.29:0/674763765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bdc0a7210 0x7f3bdc0c8200 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f3bc8009760 tx=0x7f3bc8009060).stop 2024-01-24T15:30:33.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.876+0000 7f3be2f4d700 1 -- 172.21.15.29:0/674763765 shutdown_connections 2024-01-24T15:30:33.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.876+0000 7f3be2f4d700 1 --2- 172.21.15.29:0/674763765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bdc062ec0 0x7f3bdc065420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.876+0000 7f3be2f4d700 1 --2- 172.21.15.29:0/674763765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bdc0a7210 0x7f3bdc0c8200 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.876+0000 7f3be2f4d700 1 -- 172.21.15.29:0/674763765 >> 172.21.15.29:0/674763765 conn(0x7f3bdc064390 msgr2=0x7f3bdc0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:33.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.876+0000 7f3be2f4d700 1 -- 172.21.15.29:0/674763765 shutdown_connections 2024-01-24T15:30:33.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.876+0000 7f3be2f4d700 1 -- 172.21.15.29:0/674763765 wait complete. 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.877+0000 7f3be2f4d700 1 Processor -- start 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.878+0000 7f3be2f4d700 1 -- start start 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.878+0000 7f3be2f4d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bdc062ec0 0x7f3bdc154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.878+0000 7f3be2f4d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bdc0a7210 0x7f3bdc14c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.878+0000 7f3be2f4d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bdc154100 0x7f3bdc0c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.878+0000 7f3be2f4d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc14f000 con 0x7f3bdc062ec0 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.878+0000 7f3be2f4d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0c2740 con 0x7f3bdc0a7210 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.878+0000 7f3be2f4d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0c2a20 con 0x7f3bdc154100 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be0ce9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bdc062ec0 0x7f3bdc154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be0ce9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bdc062ec0 0x7f3bdc154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37164/0 (socket says 172.21.15.29:37164) 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be0ce9700 1 -- 172.21.15.29:0/970913708 learned_addr learned my addr 172.21.15.29:0/970913708 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3bdbfff700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bdc0a7210 0x7f3bdc14c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be14ea700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bdc154100 0x7f3bdc0c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be0ce9700 1 -- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bdc154100 msgr2=0x7f3bdc0c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be0ce9700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bdc154100 0x7f3bdc0c20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be0ce9700 1 -- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bdc0a7210 msgr2=0x7f3bdc14c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:33.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be0ce9700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bdc0a7210 0x7f3bdc14c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:33.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be0ce9700 1 -- 172.21.15.29:0/970913708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd4009a20 con 0x7f3bdc062ec0 2024-01-24T15:30:33.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be14ea700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bdc154100 0x7f3bdc0c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:33.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be0ce9700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bdc062ec0 0x7f3bdc154eb0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7f3bd000c810 tx=0x7f3bd0001ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:33.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3bd8ff9700 1 -- 172.21.15.29:0/970913708 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0007170 con 0x7f3bdc062ec0 2024-01-24T15:30:33.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3bd8ff9700 1 -- 172.21.15.29:0/970913708 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd00072d0 con 0x7f3bdc062ec0 2024-01-24T15:30:33.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3bd8ff9700 1 -- 172.21.15.29:0/970913708 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00048c0 con 0x7f3bdc062ec0 2024-01-24T15:30:33.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be2f4d700 1 -- 172.21.15.29:0/970913708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc80092a0 con 0x7f3bdc062ec0 2024-01-24T15:30:33.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.879+0000 7f3be2f4d700 1 -- 172.21.15.29:0/970913708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bdc0c2fe0 con 0x7f3bdc062ec0 2024-01-24T15:30:33.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.880+0000 7f3be2f4d700 1 -- 172.21.15.29:0/970913708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bdc0475e0 con 0x7f3bdc062ec0 2024-01-24T15:30:33.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.881+0000 7f3bd8ff9700 1 -- 172.21.15.29:0/970913708 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3bd0004c70 con 0x7f3bdc062ec0 2024-01-24T15:30:33.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.881+0000 7f3bd8ff9700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3bc005f270 0x7f3bc0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:33.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.881+0000 7f3bd8ff9700 1 -- 172.21.15.29:0/970913708 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6334+0+0 (secure 0 0 0) 0x7f3bd007a060 con 0x7f3bdc062ec0 2024-01-24T15:30:33.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.882+0000 7f3bdbfff700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3bc005f270 0x7f3bc0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:33.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.882+0000 7f3bdbfff700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3bc005f270 0x7f3bc0061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f3bdc153850 tx=0x7f3bc8016000).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:33.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:33.886+0000 7f3bd8ff9700 1 -- 172.21.15.29:0/970913708 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3bd004b0e0 con 0x7f3bdc062ec0 2024-01-24T15:30:34.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.068+0000 7f3be2f4d700 1 -- 172.21.15.29:0/970913708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3bdc06a1e0 con 0x7f3bdc062ec0 2024-01-24T15:30:34.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.068+0000 7f3bd8ff9700 1 -- 172.21.15.29:0/970913708 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v209) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3bd004e700 con 0x7f3bdc062ec0 2024-01-24T15:30:34.071 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T15:30:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.070+0000 7f3be2f4d700 1 -- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3bc005f270 msgr2=0x7f3bc0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.070+0000 7f3be2f4d700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3bc005f270 0x7f3bc0061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f3bdc153850 tx=0x7f3bc8016000).stop 2024-01-24T15:30:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.070+0000 7f3be2f4d700 1 -- 172.21.15.29:0/970913708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bdc062ec0 msgr2=0x7f3bdc154eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.070+0000 7f3be2f4d700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bdc062ec0 0x7f3bdc154eb0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7f3bd000c810 tx=0x7f3bd0001ab0).stop 2024-01-24T15:30:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.070+0000 7f3be2f4d700 1 -- 172.21.15.29:0/970913708 shutdown_connections 2024-01-24T15:30:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.070+0000 7f3be2f4d700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3bc005f270 0x7f3bc0061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.070+0000 7f3be2f4d700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bdc154100 0x7f3bdc0c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.070+0000 7f3be2f4d700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3bdc062ec0 0x7f3bdc154eb0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.070+0000 7f3be2f4d700 1 --2- 172.21.15.29:0/970913708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bdc0a7210 0x7f3bdc14c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.070+0000 7f3be2f4d700 1 -- 172.21.15.29:0/970913708 >> 172.21.15.29:0/970913708 conn(0x7f3bdc064390 msgr2=0x7f3bdc14f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.070+0000 7f3be2f4d700 1 -- 172.21.15.29:0/970913708 shutdown_connections 2024-01-24T15:30:34.073 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.070+0000 7f3be2f4d700 1 -- 172.21.15.29:0/970913708 wait complete. 2024-01-24T15:30:34.086 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T15:30:34.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.217+0000 7f372ca52700 1 Processor -- start 2024-01-24T15:30:34.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.217+0000 7f372ca52700 1 -- start start 2024-01-24T15:30:34.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.218+0000 7f372ca52700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3728062970 0x7f3728062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.218+0000 7f372ca52700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3728069c50 0x7f37280bca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.218+0000 7f372ca52700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3728065750 con 0x7f37281380a0 2024-01-24T15:30:34.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.218+0000 7f372ca52700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37280c8550 con 0x7f3728062970 2024-01-24T15:30:34.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.218+0000 7f372ca52700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37280bcf40 con 0x7f3728069c50 2024-01-24T15:30:34.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.218+0000 7f372659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3728062970 0x7f3728062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.218+0000 7f372659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3728062970 0x7f3728062d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37172/0 (socket says 172.21.15.29:37172) 2024-01-24T15:30:34.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.218+0000 7f372659c700 1 -- 172.21.15.29:0/2028865229 learned_addr learned my addr 172.21.15.29:0/2028865229 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:34.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.218+0000 7f3725d9b700 1 --2- 172.21.15.29:0/2028865229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3728069c50 0x7f37280bca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.219+0000 7f372659c700 1 -- 172.21.15.29:0/2028865229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3728069c50 msgr2=0x7f37280bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:34.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.219+0000 7f372659c700 1 --2- 172.21.15.29:0/2028865229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3728069c50 0x7f37280bca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.219+0000 7f372659c700 1 -- 172.21.15.29:0/2028865229 >> v1:172.21.15.163:6789/0 conn(0x7f37281380a0 legacy=0x7f3728155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.219+0000 7f372659c700 1 -- 172.21.15.29:0/2028865229 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3728065420 con 0x7f3728062970 2024-01-24T15:30:34.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.219+0000 7f372659c700 1 --2- 172.21.15.29:0/2028865229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3728062970 0x7f3728062d50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7f371c006fd0 tx=0x7f371c0090f0).ready entity=mon.0 client_cookie=6ab9a406d89ab4be server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:34.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.219+0000 7f372559a700 1 -- 172.21.15.29:0/2028865229 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c00b580 con 0x7f3728062970 2024-01-24T15:30:34.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.219+0000 7f372559a700 1 -- 172.21.15.29:0/2028865229 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c00b6e0 con 0x7f3728062970 2024-01-24T15:30:34.222 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.219+0000 7f372559a700 1 -- 172.21.15.29:0/2028865229 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c00ba60 con 0x7f3728062970 2024-01-24T15:30:34.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.219+0000 7f372ca52700 1 -- 172.21.15.29:0/2028865229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3728062970 msgr2=0x7f3728062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.219+0000 7f372ca52700 1 --2- 172.21.15.29:0/2028865229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3728062970 0x7f3728062d50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7f371c006fd0 tx=0x7f371c0090f0).stop 2024-01-24T15:30:34.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.220+0000 7f372ca52700 1 -- 172.21.15.29:0/2028865229 shutdown_connections 2024-01-24T15:30:34.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.220+0000 7f372ca52700 1 --2- 172.21.15.29:0/2028865229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3728062970 0x7f3728062d50 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.220+0000 7f372ca52700 1 --2- 172.21.15.29:0/2028865229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3728069c50 0x7f37280bca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.220+0000 7f372ca52700 1 -- 172.21.15.29:0/2028865229 >> 172.21.15.29:0/2028865229 conn(0x7f3728064390 msgr2=0x7f37280c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:34.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.220+0000 7f372ca52700 1 -- 172.21.15.29:0/2028865229 shutdown_connections 2024-01-24T15:30:34.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.220+0000 7f372ca52700 1 -- 172.21.15.29:0/2028865229 wait complete. 2024-01-24T15:30:34.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.221+0000 7f372ca52700 1 Processor -- start 2024-01-24T15:30:34.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.221+0000 7f372ca52700 1 -- start start 2024-01-24T15:30:34.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372ca52700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3728062970 0x7f3728164760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372ca52700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3728069c50 0x7f3728155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372ca52700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37281380a0 0x7f3728165150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372ca52700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37281571e0 con 0x7f3728062970 2024-01-24T15:30:34.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372ca52700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37280bee00 con 0x7f37281380a0 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372ca52700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3728156330 con 0x7f3728069c50 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3728062970 0x7f3728164760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3728062970 0x7f3728164760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37180/0 (socket says 172.21.15.29:37180) 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372659c700 1 -- 172.21.15.29:0/3157596319 learned_addr learned my addr 172.21.15.29:0/3157596319 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f3726d9d700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37281380a0 0x7f3728165150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372659c700 1 -- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3728069c50 msgr2=0x7f3728155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372659c700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3728069c50 0x7f3728155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372659c700 1 -- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37281380a0 msgr2=0x7f3728165150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372659c700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37281380a0 0x7f3728165150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372659c700 1 -- 172.21.15.29:0/3157596319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c009370 con 0x7f3728062970 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372659c700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3728062970 0x7f3728164760 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7f371c009130 tx=0x7f371c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f37167fc700 1 -- 172.21.15.29:0/3157596319 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c00b850 con 0x7f3728062970 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f37167fc700 1 -- 172.21.15.29:0/3157596319 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c00b9b0 con 0x7f3728062970 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372ca52700 1 -- 172.21.15.29:0/3157596319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37281565d0 con 0x7f3728062970 2024-01-24T15:30:34.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f37167fc700 1 -- 172.21.15.29:0/3157596319 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c01c670 con 0x7f3728062970 2024-01-24T15:30:34.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.222+0000 7f372ca52700 1 -- 172.21.15.29:0/3157596319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3728156b90 con 0x7f3728062970 2024-01-24T15:30:34.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.223+0000 7f372ca52700 1 -- 172.21.15.29:0/3157596319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3728157550 con 0x7f3728062970 2024-01-24T15:30:34.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.224+0000 7f37167fc700 1 -- 172.21.15.29:0/3157596319 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f371c01ca60 con 0x7f3728062970 2024-01-24T15:30:34.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.224+0000 7f37167fc700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f370805f270 0x7f3708061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.224+0000 7f37167fc700 1 -- 172.21.15.29:0/3157596319 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6334+0+0 (secure 0 0 0) 0x7f371c07c090 con 0x7f3728062970 2024-01-24T15:30:34.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.224+0000 7f3725d9b700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f370805f270 0x7f3708061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.225+0000 7f3725d9b700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f370805f270 0x7f3708061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f37281578d0 tx=0x7f3718009060).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:34.230 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.229+0000 7f37167fc700 1 -- 172.21.15.29:0/3157596319 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f371c04d110 con 0x7f3728062970 2024-01-24T15:30:34.412 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.410+0000 7f372ca52700 1 -- 172.21.15.29:0/3157596319 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f37280475e0 con 0x7f3728062970 2024-01-24T15:30:34.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.411+0000 7f37167fc700 1 -- 172.21.15.29:0/3157596319 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v209) v1 ==== 92+0+10 (secure 0 0 0) 0x7f371c050730 con 0x7f3728062970 2024-01-24T15:30:34.413 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T15:30:34.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.413+0000 7f372ca52700 1 -- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f370805f270 msgr2=0x7f3708061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.413+0000 7f372ca52700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f370805f270 0x7f3708061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f37281578d0 tx=0x7f3718009060).stop 2024-01-24T15:30:34.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.413+0000 7f372ca52700 1 -- 172.21.15.29:0/3157596319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3728062970 msgr2=0x7f3728164760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.413+0000 7f372ca52700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3728062970 0x7f3728164760 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7f371c009130 tx=0x7f371c017000).stop 2024-01-24T15:30:34.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.413+0000 7f372ca52700 1 -- 172.21.15.29:0/3157596319 shutdown_connections 2024-01-24T15:30:34.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.413+0000 7f372ca52700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37281380a0 0x7f3728165150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.413+0000 7f372ca52700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3728062970 0x7f3728164760 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.413+0000 7f372ca52700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f370805f270 0x7f3708061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.413+0000 7f372ca52700 1 --2- 172.21.15.29:0/3157596319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3728069c50 0x7f3728155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:34.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.413+0000 7f372ca52700 1 -- 172.21.15.29:0/3157596319 >> 172.21.15.29:0/3157596319 conn(0x7f3728064390 msgr2=0x7f3728065be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:34.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.413+0000 7f372ca52700 1 -- 172.21.15.29:0/3157596319 shutdown_connections 2024-01-24T15:30:34.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.413+0000 7f372ca52700 1 -- 172.21.15.29:0/3157596319 wait complete. 2024-01-24T15:30:34.429 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-24T15:30:34.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.557+0000 7f49b275b700 1 Processor -- start 2024-01-24T15:30:34.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.557+0000 7f49b275b700 1 -- start start 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b275b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f49ac066c00 0x7f49ac155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b275b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f49ac1563d0 0x7f49ac0bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b275b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ac0c3910 con 0x7f49ac155b00 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b275b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac0c5500 con 0x7f49ac1563d0 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b275b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac0c5220 con 0x7f49ac066c00 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b0cf8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f49ac1563d0 0x7f49ac0bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b0cf8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f49ac1563d0 0x7f49ac0bdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37192/0 (socket says 172.21.15.29:37192) 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b0cf8700 1 -- 172.21.15.29:0/466606646 learned_addr learned my addr 172.21.15.29:0/466606646 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49ab7fe700 1 --2- 172.21.15.29:0/466606646 >> v2:172.21.15.163:3300/0 conn(0x7f49ac066c00 0x7f49ac155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b0cf8700 1 -- 172.21.15.29:0/466606646 >> v2:172.21.15.163:3300/0 conn(0x7f49ac066c00 msgr2=0x7f49ac155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b0cf8700 1 --2- 172.21.15.29:0/466606646 >> v2:172.21.15.163:3300/0 conn(0x7f49ac066c00 0x7f49ac155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b0cf8700 1 -- 172.21.15.29:0/466606646 >> v1:172.21.15.163:6789/0 conn(0x7f49ac155b00 legacy=0x7f49ac0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:34.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b0cf8700 1 -- 172.21.15.29:0/466606646 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac0c3350 con 0x7f49ac1563d0 2024-01-24T15:30:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.558+0000 7f49b0cf8700 1 --2- 172.21.15.29:0/466606646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49ac1563d0 0x7f49ac0bdeb0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 rx=0x7f49a4009f20 tx=0x7f49a4009840).ready entity=mon.0 client_cookie=d65afb6784f60ec8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.559+0000 7f49aaffd700 1 -- 172.21.15.29:0/466606646 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a400e070 con 0x7f49ac1563d0 2024-01-24T15:30:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.559+0000 7f49aaffd700 1 -- 172.21.15.29:0/466606646 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a400ba00 con 0x7f49ac1563d0 2024-01-24T15:30:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.559+0000 7f49aaffd700 1 -- 172.21.15.29:0/466606646 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a400bd80 con 0x7f49ac1563d0 2024-01-24T15:30:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.559+0000 7f49b275b700 1 -- 172.21.15.29:0/466606646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49ac1563d0 msgr2=0x7f49ac0bdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.559+0000 7f49b275b700 1 --2- 172.21.15.29:0/466606646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49ac1563d0 0x7f49ac0bdeb0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 rx=0x7f49a4009f20 tx=0x7f49a4009840).stop 2024-01-24T15:30:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.559+0000 7f49b275b700 1 -- 172.21.15.29:0/466606646 shutdown_connections 2024-01-24T15:30:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.559+0000 7f49b275b700 1 --2- 172.21.15.29:0/466606646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49ac1563d0 0x7f49ac0bdeb0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.559+0000 7f49b275b700 1 --2- 172.21.15.29:0/466606646 >> v2:172.21.15.163:3300/0 conn(0x7f49ac066c00 0x7f49ac155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.559+0000 7f49b275b700 1 -- 172.21.15.29:0/466606646 >> 172.21.15.29:0/466606646 conn(0x7f49ac0c8bb0 msgr2=0x7f49ac0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.559+0000 7f49b275b700 1 -- 172.21.15.29:0/466606646 shutdown_connections 2024-01-24T15:30:34.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.559+0000 7f49b275b700 1 -- 172.21.15.29:0/466606646 wait complete. 2024-01-24T15:30:34.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.561+0000 7f49b275b700 1 Processor -- start 2024-01-24T15:30:34.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.561+0000 7f49b275b700 1 -- start start 2024-01-24T15:30:34.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49b275b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac066c00 0x7f49ac064820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49b275b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49ac155b00 0x7f49ac1674c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49b275b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49ac1563d0 0x7f49ac0652f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49b275b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac14c700 con 0x7f49ac155b00 2024-01-24T15:30:34.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49b275b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac1679b0 con 0x7f49ac066c00 2024-01-24T15:30:34.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49b275b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac167c60 con 0x7f49ac1563d0 2024-01-24T15:30:34.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49ab7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49ac155b00 0x7f49ac1674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49ab7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49ac155b00 0x7f49ac1674c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37194/0 (socket says 172.21.15.29:37194) 2024-01-24T15:30:34.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49ab7fe700 1 -- 172.21.15.29:0/668183528 learned_addr learned my addr 172.21.15.29:0/668183528 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:34.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49ab7fe700 1 -- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49ac1563d0 msgr2=0x7f49ac0652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49abfff700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac066c00 0x7f49ac064820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49b0cf8700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49ac1563d0 0x7f49ac0652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49ab7fe700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49ac1563d0 0x7f49ac0652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49ab7fe700 1 -- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac066c00 msgr2=0x7f49ac064820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49ab7fe700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac066c00 0x7f49ac064820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49ab7fe700 1 -- 172.21.15.29:0/668183528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4009ae0 con 0x7f49ac155b00 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49abfff700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac066c00 0x7f49ac064820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49b0cf8700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49ac1563d0 0x7f49ac0652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49ab7fe700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49ac155b00 0x7f49ac1674c0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 rx=0x7f499400d890 tx=0x7f4994008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.562+0000 7f49a3fff700 1 -- 172.21.15.29:0/668183528 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49940125f0 con 0x7f49ac155b00 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.563+0000 7f49a3fff700 1 -- 172.21.15.29:0/668183528 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4994012750 con 0x7f49ac155b00 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.563+0000 7f49b275b700 1 -- 172.21.15.29:0/668183528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ac167f40 con 0x7f49ac155b00 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.563+0000 7f49a3fff700 1 -- 172.21.15.29:0/668183528 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994012ad0 con 0x7f49ac155b00 2024-01-24T15:30:34.567 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.563+0000 7f49b275b700 1 -- 172.21.15.29:0/668183528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ac14c9a0 con 0x7f49ac155b00 2024-01-24T15:30:34.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.564+0000 7f49b275b700 1 -- 172.21.15.29:0/668183528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ac0475e0 con 0x7f49ac155b00 2024-01-24T15:30:34.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.564+0000 7f49a3fff700 1 -- 172.21.15.29:0/668183528 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4994013480 con 0x7f49ac155b00 2024-01-24T15:30:34.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.564+0000 7f49a3fff700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f498c05f240 0x7f498c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.565+0000 7f49a3fff700 1 -- 172.21.15.29:0/668183528 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6334+0+0 (secure 0 0 0) 0x7f499407a2a0 con 0x7f49ac155b00 2024-01-24T15:30:34.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.565+0000 7f49abfff700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f498c05f240 0x7f498c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.565+0000 7f49abfff700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f498c05f240 0x7f498c061700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f499c0026f0 tx=0x7f499c009040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:34.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.569+0000 7f49a3fff700 1 -- 172.21.15.29:0/668183528 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f499404b2c0 con 0x7f49ac155b00 2024-01-24T15:30:34.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.755+0000 7f49b275b700 1 -- 172.21.15.29:0/668183528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f49ac0c3630 con 0x7f49ac155b00 2024-01-24T15:30:34.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.755+0000 7f49a3fff700 1 -- 172.21.15.29:0/668183528 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v209) v1 ==== 84+0+143 (secure 0 0 0) 0x7f499404e8e0 con 0x7f49ac155b00 2024-01-24T15:30:34.758 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:34.758 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T15:30:34.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:34.759 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-01-24T15:30:34.759 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "device_health_metrics" 2024-01-24T15:30:34.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:30:34.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:34.759 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 2, 2024-01-24T15:30:34.759 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "rbd" 2024-01-24T15:30:34.759 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:34.759 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T15:30:34.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.757+0000 7f49b275b700 1 -- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f498c05f240 msgr2=0x7f498c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.757+0000 7f49b275b700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f498c05f240 0x7f498c061700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f499c0026f0 tx=0x7f499c009040).stop 2024-01-24T15:30:34.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.757+0000 7f49b275b700 1 -- 172.21.15.29:0/668183528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49ac155b00 msgr2=0x7f49ac1674c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.757+0000 7f49b275b700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49ac155b00 0x7f49ac1674c0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 rx=0x7f499400d890 tx=0x7f4994008040).stop 2024-01-24T15:30:34.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.757+0000 7f49b275b700 1 -- 172.21.15.29:0/668183528 shutdown_connections 2024-01-24T15:30:34.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.757+0000 7f49b275b700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f498c05f240 0x7f498c061700 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.757+0000 7f49b275b700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49ac1563d0 0x7f49ac0652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.757+0000 7f49b275b700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac066c00 0x7f49ac064820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.757+0000 7f49b275b700 1 --2- 172.21.15.29:0/668183528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49ac155b00 0x7f49ac1674c0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.757+0000 7f49b275b700 1 -- 172.21.15.29:0/668183528 >> 172.21.15.29:0/668183528 conn(0x7f49ac0c8bb0 msgr2=0x7f49ac146f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:34.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.757+0000 7f49b275b700 1 -- 172.21.15.29:0/668183528 shutdown_connections 2024-01-24T15:30:34.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.757+0000 7f49b275b700 1 -- 172.21.15.29:0/668183528 wait complete. 2024-01-24T15:30:34.773 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-24T15:30:34.773 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2024-01-24T15:30:34.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.907+0000 7ff46779b700 1 Processor -- start 2024-01-24T15:30:34.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.907+0000 7ff46779b700 1 -- start start 2024-01-24T15:30:34.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff46779b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff460150970 0x7ff460150d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff46779b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff460155060 0x7ff4600c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff46779b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff460069a50 con 0x7ff460069ee0 2024-01-24T15:30:34.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff46779b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4600c2b20 con 0x7ff460150970 2024-01-24T15:30:34.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff46779b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff460151240 con 0x7ff460155060 2024-01-24T15:30:34.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff465537700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff460150970 0x7ff460150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff465537700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff460150970 0x7ff460150d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37198/0 (socket says 172.21.15.29:37198) 2024-01-24T15:30:34.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff465537700 1 -- 172.21.15.29:0/115183050 learned_addr learned my addr 172.21.15.29:0/115183050 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:34.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff465d38700 1 --2- 172.21.15.29:0/115183050 >> v2:172.21.15.163:3300/0 conn(0x7ff460155060 0x7ff4600c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff465537700 1 -- 172.21.15.29:0/115183050 >> v2:172.21.15.163:3300/0 conn(0x7ff460155060 msgr2=0x7ff4600c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff465537700 1 --2- 172.21.15.29:0/115183050 >> v2:172.21.15.163:3300/0 conn(0x7ff460155060 0x7ff4600c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff465537700 1 -- 172.21.15.29:0/115183050 >> v1:172.21.15.163:6789/0 conn(0x7ff460069ee0 legacy=0x7ff460154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.908+0000 7ff465537700 1 -- 172.21.15.29:0/115183050 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff460155510 con 0x7ff460150970 2024-01-24T15:30:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.909+0000 7ff465537700 1 --2- 172.21.15.29:0/115183050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff460150970 0x7ff460150d50 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7ff4600a7d60 tx=0x7ff45c0090f0).ready entity=mon.0 client_cookie=f7f5a91ae28f8349 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.909+0000 7ff457fff700 1 -- 172.21.15.29:0/115183050 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c00b520 con 0x7ff460150970 2024-01-24T15:30:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.909+0000 7ff457fff700 1 -- 172.21.15.29:0/115183050 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c00b680 con 0x7ff460150970 2024-01-24T15:30:34.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.909+0000 7ff457fff700 1 -- 172.21.15.29:0/115183050 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c00ba00 con 0x7ff460150970 2024-01-24T15:30:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.909+0000 7ff46779b700 1 -- 172.21.15.29:0/115183050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff460150970 msgr2=0x7ff460150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.909+0000 7ff46779b700 1 --2- 172.21.15.29:0/115183050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff460150970 0x7ff460150d50 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7ff4600a7d60 tx=0x7ff45c0090f0).stop 2024-01-24T15:30:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.909+0000 7ff46779b700 1 -- 172.21.15.29:0/115183050 shutdown_connections 2024-01-24T15:30:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.909+0000 7ff46779b700 1 --2- 172.21.15.29:0/115183050 >> v2:172.21.15.163:3300/0 conn(0x7ff460155060 0x7ff4600c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.909+0000 7ff46779b700 1 --2- 172.21.15.29:0/115183050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff460150970 0x7ff460150d50 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.909+0000 7ff46779b700 1 -- 172.21.15.29:0/115183050 >> 172.21.15.29:0/115183050 conn(0x7ff460067b20 msgr2=0x7ff4600638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.910+0000 7ff46779b700 1 -- 172.21.15.29:0/115183050 shutdown_connections 2024-01-24T15:30:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.910+0000 7ff46779b700 1 -- 172.21.15.29:0/115183050 wait complete. 2024-01-24T15:30:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.911+0000 7ff46779b700 1 Processor -- start 2024-01-24T15:30:34.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.911+0000 7ff46779b700 1 -- start start 2024-01-24T15:30:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff46779b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff460069ee0 0x7ff46014bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff46779b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff460150970 0x7ff4601496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff46779b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff460155060 0x7ff460153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff46779b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46014b8a0 con 0x7ff460069ee0 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff46779b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff460149bc0 con 0x7ff460150970 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff46779b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46014ac90 con 0x7ff460155060 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff465537700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff460069ee0 0x7ff46014bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff465537700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff460069ee0 0x7ff46014bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37214/0 (socket says 172.21.15.29:37214) 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff465537700 1 -- 172.21.15.29:0/3878803710 learned_addr learned my addr 172.21.15.29:0/3878803710 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff464d36700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff460150970 0x7ff4601496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff465d38700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff460155060 0x7ff460153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff465537700 1 -- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff460155060 msgr2=0x7ff460153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff465537700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff460155060 0x7ff460153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff465537700 1 -- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff460150970 msgr2=0x7ff4601496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff465537700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff460150970 0x7ff4601496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:34.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff465537700 1 -- 172.21.15.29:0/3878803710 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c009370 con 0x7ff460069ee0 2024-01-24T15:30:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.912+0000 7ff465537700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff460069ee0 0x7ff46014bd30 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 rx=0x7ff45c009130 tx=0x7ff45c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.913+0000 7ff4557fa700 1 -- 172.21.15.29:0/3878803710 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c000bc0 con 0x7ff460069ee0 2024-01-24T15:30:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.913+0000 7ff4557fa700 1 -- 172.21.15.29:0/3878803710 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c00b7f0 con 0x7ff460069ee0 2024-01-24T15:30:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.913+0000 7ff46779b700 1 -- 172.21.15.29:0/3878803710 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46014af70 con 0x7ff460069ee0 2024-01-24T15:30:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.913+0000 7ff4557fa700 1 -- 172.21.15.29:0/3878803710 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c000d20 con 0x7ff460069ee0 2024-01-24T15:30:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.913+0000 7ff46779b700 1 -- 172.21.15.29:0/3878803710 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46014b530 con 0x7ff460069ee0 2024-01-24T15:30:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.914+0000 7ff46779b700 1 -- 172.21.15.29:0/3878803710 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46014bb80 con 0x7ff460069ee0 2024-01-24T15:30:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.914+0000 7ff4557fa700 1 -- 172.21.15.29:0/3878803710 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff45c01c8c0 con 0x7ff460069ee0 2024-01-24T15:30:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.914+0000 7ff4557fa700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff44405f270 0x7ff444061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.915+0000 7ff4557fa700 1 -- 172.21.15.29:0/3878803710 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6334+0+0 (secure 0 0 0) 0x7ff45c07c130 con 0x7ff460069ee0 2024-01-24T15:30:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.915+0000 7ff464d36700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff44405f270 0x7ff444061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:34.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.915+0000 7ff464d36700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff44405f270 0x7ff444061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff46014c4d0 tx=0x7ff450007d40).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:34.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:34.919+0000 7ff4557fa700 1 -- 172.21.15.29:0/3878803710 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff45c04d1b0 con 0x7ff460069ee0 2024-01-24T15:30:35.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.100+0000 7ff46779b700 1 -- 172.21.15.29:0/3878803710 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7ff4600475e0 con 0x7ff460069ee0 2024-01-24T15:30:35.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.982+0000 7ff4557fa700 1 -- 172.21.15.29:0/3878803710 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v210) v1 ==== 152+0+0 (secure 0 0 0) 0x7ff45c02a020 con 0x7ff460069ee0 2024-01-24T15:30:35.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.986+0000 7ff46779b700 1 -- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff44405f270 msgr2=0x7ff444061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:35.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.986+0000 7ff46779b700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff44405f270 0x7ff444061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff46014c4d0 tx=0x7ff450007d40).stop 2024-01-24T15:30:35.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.986+0000 7ff46779b700 1 -- 172.21.15.29:0/3878803710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff460069ee0 msgr2=0x7ff46014bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:35.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.986+0000 7ff46779b700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff460069ee0 0x7ff46014bd30 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 rx=0x7ff45c009130 tx=0x7ff45c017000).stop 2024-01-24T15:30:35.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.987+0000 7ff46779b700 1 -- 172.21.15.29:0/3878803710 shutdown_connections 2024-01-24T15:30:35.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.987+0000 7ff46779b700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff44405f270 0x7ff444061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:35.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.987+0000 7ff46779b700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff460155060 0x7ff460153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:35.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.987+0000 7ff46779b700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff460069ee0 0x7ff46014bd30 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:35.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.987+0000 7ff46779b700 1 --2- 172.21.15.29:0/3878803710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff460150970 0x7ff4601496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:35.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.987+0000 7ff46779b700 1 -- 172.21.15.29:0/3878803710 >> 172.21.15.29:0/3878803710 conn(0x7ff460067b20 msgr2=0x7ff4600acec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:35.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.987+0000 7ff46779b700 1 -- 172.21.15.29:0/3878803710 shutdown_connections 2024-01-24T15:30:35.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:35.987+0000 7ff46779b700 1 -- 172.21.15.29:0/3878803710 wait complete. 2024-01-24T15:30:35.997 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-01-24T15:30:36.011 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-24T15:30:36.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.148+0000 7f6992c56700 1 Processor -- start 2024-01-24T15:30:36.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.149+0000 7f6992c56700 1 -- start start 2024-01-24T15:30:36.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.149+0000 7f6992c56700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f698c0a96e0 0x7f698c064650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:36.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.149+0000 7f6992c56700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698c0bbe20 con 0x7f698c151f90 2024-01-24T15:30:36.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.149+0000 7f6992c56700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698c0c76b0 con 0x7f698c152370 2024-01-24T15:30:36.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.149+0000 7f6992c56700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698c064c30 con 0x7f698c0a96e0 2024-01-24T15:30:36.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.150+0000 7f69909f2700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f698c151f90 0x7f698c0a8fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41910/0 (socket says 172.21.15.29:41910) 2024-01-24T15:30:36.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.150+0000 7f69909f2700 1 -- 172.21.15.29:0/2028416494 learned_addr learned my addr 172.21.15.29:0/2028416494 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:36.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.150+0000 7f698bfff700 1 --2- 172.21.15.29:0/2028416494 >> v2:172.21.15.163:3300/0 conn(0x7f698c0a96e0 0x7f698c064650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:36.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.150+0000 7f698b7fe700 1 -- 172.21.15.29:0/2028416494 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643724953 0 0) 0x7f698c0bbe20 con 0x7f698c151f90 2024-01-24T15:30:36.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.150+0000 7f698b7fe700 1 -- 172.21.15.29:0/2028416494 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f697c0033f0 con 0x7f698c151f90 2024-01-24T15:30:36.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.150+0000 7f698bfff700 1 -- 172.21.15.29:0/2028416494 >> v1:172.21.15.163:6789/0 conn(0x7f698c152370 legacy=0x7f698c0bc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:36.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.150+0000 7f698b7fe700 1 -- 172.21.15.29:0/2028416494 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4278597191 0 0) 0x7f697c0033f0 con 0x7f698c151f90 2024-01-24T15:30:36.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.150+0000 7f698bfff700 1 -- 172.21.15.29:0/2028416494 >> v1:172.21.15.29:6789/0 conn(0x7f698c151f90 legacy=0x7f698c0a8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:36.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.150+0000 7f698bfff700 1 -- 172.21.15.29:0/2028416494 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c064f10 con 0x7f698c0a96e0 2024-01-24T15:30:36.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.150+0000 7f698bfff700 1 --2- 172.21.15.29:0/2028416494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c0a96e0 0x7f698c064650 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f6978009760 tx=0x7f6978009060).ready entity=mon.1 client_cookie=9455010360197935 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:36.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.151+0000 7f698b7fe700 1 -- 172.21.15.29:0/2028416494 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6980001470 con 0x7f698c0a96e0 2024-01-24T15:30:36.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.151+0000 7f698b7fe700 1 -- 172.21.15.29:0/2028416494 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6978009130 con 0x7f698c0a96e0 2024-01-24T15:30:36.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.151+0000 7f698b7fe700 1 -- 172.21.15.29:0/2028416494 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978012620 con 0x7f698c0a96e0 2024-01-24T15:30:36.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.151+0000 7f6992c56700 1 -- 172.21.15.29:0/2028416494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c0a96e0 msgr2=0x7f698c064650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:36.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.151+0000 7f6992c56700 1 --2- 172.21.15.29:0/2028416494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c0a96e0 0x7f698c064650 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f6978009760 tx=0x7f6978009060).stop 2024-01-24T15:30:36.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.151+0000 7f6992c56700 1 -- 172.21.15.29:0/2028416494 shutdown_connections 2024-01-24T15:30:36.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.151+0000 7f6992c56700 1 --2- 172.21.15.29:0/2028416494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c0a96e0 0x7f698c064650 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:36.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.151+0000 7f6992c56700 1 -- 172.21.15.29:0/2028416494 >> 172.21.15.29:0/2028416494 conn(0x7f698c063240 msgr2=0x7f698c0c6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:36.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.151+0000 7f6992c56700 1 -- 172.21.15.29:0/2028416494 shutdown_connections 2024-01-24T15:30:36.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.151+0000 7f6992c56700 1 -- 172.21.15.29:0/2028416494 wait complete. 2024-01-24T15:30:36.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.153+0000 7f6992c56700 1 Processor -- start 2024-01-24T15:30:36.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.153+0000 7f6992c56700 1 -- start start 2024-01-24T15:30:36.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.153+0000 7f6992c56700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0a96e0 0x7f698c14b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:36.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f6992c56700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c151f90 0x7f698c1510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:36.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f6992c56700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c152370 0x7f698c0a7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:36.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f6992c56700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c14b8e0 con 0x7f698c0a96e0 2024-01-24T15:30:36.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f6992c56700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c0ab1f0 con 0x7f698c152370 2024-01-24T15:30:36.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f6992c56700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698c0aad60 con 0x7f698c151f90 2024-01-24T15:30:36.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f69909f2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0a96e0 0x7f698c14b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:36.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f69909f2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0a96e0 0x7f698c14b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37230/0 (socket says 172.21.15.29:37230) 2024-01-24T15:30:36.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f69909f2700 1 -- 172.21.15.29:0/4211124711 learned_addr learned my addr 172.21.15.29:0/4211124711 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:36.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f698bfff700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c151f90 0x7f698c1510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:36.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f69911f3700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c152370 0x7f698c0a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:36.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f69909f2700 1 -- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c151f90 msgr2=0x7f698c1510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:36.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f69909f2700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c151f90 0x7f698c1510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:36.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f69909f2700 1 -- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c152370 msgr2=0x7f698c0a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:36.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f69909f2700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c152370 0x7f698c0a7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:36.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f69909f2700 1 -- 172.21.15.29:0/4211124711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69780092a0 con 0x7f698c0a96e0 2024-01-24T15:30:36.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f698bfff700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c151f90 0x7f698c1510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:36.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f69911f3700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c152370 0x7f698c0a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:36.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f69909f2700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0a96e0 0x7f698c14b3f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f698000c730 tx=0x7f6980001aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:36.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f6988ff9700 1 -- 172.21.15.29:0/4211124711 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6980007170 con 0x7f698c0a96e0 2024-01-24T15:30:36.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f6988ff9700 1 -- 172.21.15.29:0/4211124711 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69800072f0 con 0x7f698c0a96e0 2024-01-24T15:30:36.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f6988ff9700 1 -- 172.21.15.29:0/4211124711 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69800048c0 con 0x7f698c0a96e0 2024-01-24T15:30:36.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.154+0000 7f6992c56700 1 -- 172.21.15.29:0/4211124711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698c0a9e70 con 0x7f698c0a96e0 2024-01-24T15:30:36.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.155+0000 7f6992c56700 1 -- 172.21.15.29:0/4211124711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698c0aa430 con 0x7f698c0a96e0 2024-01-24T15:30:36.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.156+0000 7f6992c56700 1 -- 172.21.15.29:0/4211124711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698c0ab040 con 0x7f698c0a96e0 2024-01-24T15:30:36.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.156+0000 7f6988ff9700 1 -- 172.21.15.29:0/4211124711 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6980004c70 con 0x7f698c0a96e0 2024-01-24T15:30:36.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.156+0000 7f6988ff9700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f697005f270 0x7f6970061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:36.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.157+0000 7f6988ff9700 1 -- 172.21.15.29:0/4211124711 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f698007a170 con 0x7f698c0a96e0 2024-01-24T15:30:36.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.157+0000 7f698bfff700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f697005f270 0x7f6970061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:36.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.157+0000 7f698bfff700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f697005f270 0x7f6970061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f698c156d10 tx=0x7f6978015000).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:36.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.161+0000 7f6988ff9700 1 -- 172.21.15.29:0/4211124711 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f698004b080 con 0x7f698c0a96e0 2024-01-24T15:30:36.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.344+0000 7f6992c56700 1 -- 172.21.15.29:0/4211124711 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f698c0475e0 con 0x7f698c0a96e0 2024-01-24T15:30:36.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.988+0000 7f6988ff9700 1 -- 172.21.15.29:0/4211124711 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v211) v1 ==== 170+0+0 (secure 0 0 0) 0x7f698004e6a0 con 0x7f698c0a96e0 2024-01-24T15:30:36.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.995+0000 7f6992c56700 1 -- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f697005f270 msgr2=0x7f6970061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:36.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.995+0000 7f6992c56700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f697005f270 0x7f6970061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f698c156d10 tx=0x7f6978015000).stop 2024-01-24T15:30:36.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.995+0000 7f6992c56700 1 -- 172.21.15.29:0/4211124711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0a96e0 msgr2=0x7f698c14b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:36.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.995+0000 7f6992c56700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0a96e0 0x7f698c14b3f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f698000c730 tx=0x7f6980001aa0).stop 2024-01-24T15:30:36.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.997+0000 7f6992c56700 1 -- 172.21.15.29:0/4211124711 shutdown_connections 2024-01-24T15:30:36.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.997+0000 7f6992c56700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f698c152370 0x7f698c0a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:36.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.997+0000 7f6992c56700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f698c0a96e0 0x7f698c14b3f0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:36.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.997+0000 7f6992c56700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f697005f270 0x7f6970061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:36.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.997+0000 7f6992c56700 1 --2- 172.21.15.29:0/4211124711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f698c151f90 0x7f698c1510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:36.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:36.997+0000 7f6992c56700 1 -- 172.21.15.29:0/4211124711 >> 172.21.15.29:0/4211124711 conn(0x7f698c063240 msgr2=0x7f698c0c4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:37.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.010+0000 7f6992c56700 1 -- 172.21.15.29:0/4211124711 shutdown_connections 2024-01-24T15:30:37.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.013+0000 7f6992c56700 1 -- 172.21.15.29:0/4211124711 wait complete. 2024-01-24T15:30:37.019 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-01-24T15:30:37.036 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-24T15:30:37.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.170+0000 7ff97e633700 1 Processor -- start 2024-01-24T15:30:37.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.170+0000 7ff97e633700 1 -- start start 2024-01-24T15:30:37.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.170+0000 7ff97e633700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff9780692a0 0x7ff978065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.170+0000 7ff97e633700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff978069d90 0x7ff97815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff97e633700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9780c3e70 con 0x7ff978066370 2024-01-24T15:30:37.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff97e633700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9780c4300 con 0x7ff978069d90 2024-01-24T15:30:37.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff97e633700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9780c3260 con 0x7ff9780692a0 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff97cbd0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff978069d90 0x7ff97815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff97cbd0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff978069d90 0x7ff97815a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37244/0 (socket says 172.21.15.29:37244) 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff97cbd0700 1 -- 172.21.15.29:0/1714939650 learned_addr learned my addr 172.21.15.29:0/1714939650 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff9777fe700 1 --1- 172.21.15.29:0/1714939650 >> v1:172.21.15.29:6789/0 conn(0x7ff978066370 0x7ff978069680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41924/0 (socket says 172.21.15.29:41924) 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff977fff700 1 --2- 172.21.15.29:0/1714939650 >> v2:172.21.15.163:3300/0 conn(0x7ff9780692a0 0x7ff978065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff97cbd0700 1 -- 172.21.15.29:0/1714939650 >> v2:172.21.15.163:3300/0 conn(0x7ff9780692a0 msgr2=0x7ff978065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff97cbd0700 1 --2- 172.21.15.29:0/1714939650 >> v2:172.21.15.163:3300/0 conn(0x7ff9780692a0 0x7ff978065e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff97cbd0700 1 -- 172.21.15.29:0/1714939650 >> v1:172.21.15.29:6789/0 conn(0x7ff978066370 legacy=0x7ff978069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff97cbd0700 1 -- 172.21.15.29:0/1714939650 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9780c3540 con 0x7ff978069d90 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff976ffd700 1 -- 172.21.15.29:0/1714939650 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217603737 0 0) 0x7ff9780c3e70 con 0x7ff978066370 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff97cbd0700 1 --2- 172.21.15.29:0/1714939650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff978069d90 0x7ff97815a9f0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7ff97805ed00 tx=0x7ff970009840).ready entity=mon.0 client_cookie=b72746690a64de69 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff976ffd700 1 -- 172.21.15.29:0/1714939650 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97000e070 con 0x7ff978069d90 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.171+0000 7ff976ffd700 1 -- 172.21.15.29:0/1714939650 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97000ba50 con 0x7ff978069d90 2024-01-24T15:30:37.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.172+0000 7ff976ffd700 1 -- 172.21.15.29:0/1714939650 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97000bdd0 con 0x7ff978069d90 2024-01-24T15:30:37.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.172+0000 7ff97e633700 1 -- 172.21.15.29:0/1714939650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff978069d90 msgr2=0x7ff97815a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.172+0000 7ff97e633700 1 --2- 172.21.15.29:0/1714939650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff978069d90 0x7ff97815a9f0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7ff97805ed00 tx=0x7ff970009840).stop 2024-01-24T15:30:37.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.172+0000 7ff97e633700 1 -- 172.21.15.29:0/1714939650 shutdown_connections 2024-01-24T15:30:37.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.172+0000 7ff97e633700 1 --2- 172.21.15.29:0/1714939650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff978069d90 0x7ff97815a9f0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.172+0000 7ff97e633700 1 --2- 172.21.15.29:0/1714939650 >> v2:172.21.15.163:3300/0 conn(0x7ff9780692a0 0x7ff978065e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.172+0000 7ff97e633700 1 -- 172.21.15.29:0/1714939650 >> 172.21.15.29:0/1714939650 conn(0x7ff978064390 msgr2=0x7ff9780c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:37.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.172+0000 7ff97e633700 1 -- 172.21.15.29:0/1714939650 shutdown_connections 2024-01-24T15:30:37.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.172+0000 7ff97e633700 1 -- 172.21.15.29:0/1714939650 wait complete. 2024-01-24T15:30:37.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.173+0000 7ff97e633700 1 Processor -- start 2024-01-24T15:30:37.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.174+0000 7ff97e633700 1 -- start start 2024-01-24T15:30:37.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.174+0000 7ff97e633700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff978066370 0x7ff97814a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.174+0000 7ff97e633700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9780692a0 0x7ff978149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.174+0000 7ff97e633700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff978069d90 0x7ff97814faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.174+0000 7ff97e633700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97814dfd0 con 0x7ff9780692a0 2024-01-24T15:30:37.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.174+0000 7ff97e633700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97814e590 con 0x7ff978069d90 2024-01-24T15:30:37.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.174+0000 7ff97e633700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97814d660 con 0x7ff978066370 2024-01-24T15:30:37.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.174+0000 7ff9777fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9780692a0 0x7ff978149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff9777fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9780692a0 0x7ff978149720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37258/0 (socket says 172.21.15.29:37258) 2024-01-24T15:30:37.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff9777fe700 1 -- 172.21.15.29:0/1060964867 learned_addr learned my addr 172.21.15.29:0/1060964867 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:37.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff97cbd0700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff978069d90 0x7ff97814faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff977fff700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff978066370 0x7ff97814a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff9777fe700 1 -- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff978066370 msgr2=0x7ff97814a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff9777fe700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff978066370 0x7ff97814a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff9777fe700 1 -- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff978069d90 msgr2=0x7ff97814faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff9777fe700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff978069d90 0x7ff97814faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff9777fe700 1 -- 172.21.15.29:0/1060964867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff970009ae0 con 0x7ff9780692a0 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff97cbd0700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff978069d90 0x7ff97814faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff9777fe700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9780692a0 0x7ff978149720 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7ff96c00c440 tx=0x7ff96c007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff95ffff700 1 -- 172.21.15.29:0/1060964867 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c0072e0 con 0x7ff9780692a0 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff95ffff700 1 -- 172.21.15.29:0/1060964867 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96c007460 con 0x7ff9780692a0 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff95ffff700 1 -- 172.21.15.29:0/1060964867 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c005930 con 0x7ff9780692a0 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff97e633700 1 -- 172.21.15.29:0/1060964867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff97814d940 con 0x7ff9780692a0 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.175+0000 7ff97e633700 1 -- 172.21.15.29:0/1060964867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff978154140 con 0x7ff9780692a0 2024-01-24T15:30:37.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.176+0000 7ff97e633700 1 -- 172.21.15.29:0/1060964867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9781510d0 con 0x7ff9780692a0 2024-01-24T15:30:37.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.177+0000 7ff95ffff700 1 -- 172.21.15.29:0/1060964867 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff96c005d20 con 0x7ff9780692a0 2024-01-24T15:30:37.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.177+0000 7ff95ffff700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff96405f270 0x7ff964061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.177+0000 7ff95ffff700 1 -- 172.21.15.29:0/1060964867 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 7086+0+0 (secure 0 0 0) 0x7ff96c079da0 con 0x7ff9780692a0 2024-01-24T15:30:37.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.177+0000 7ff977fff700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff96405f270 0x7ff964061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.178+0000 7ff977fff700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff96405f270 0x7ff964061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7ff9780abaf0 tx=0x7ff968008040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:37.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.182+0000 7ff95ffff700 1 -- 172.21.15.29:0/1060964867 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff9781510d0 con 0x7ff9780692a0 2024-01-24T15:30:37.411 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.409+0000 7ff97e633700 1 -- 172.21.15.29:0/1060964867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff9781510d0 con 0x7ff9780692a0 2024-01-24T15:30:37.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.415+0000 7ff95ffff700 1 -- 172.21.15.29:0/1060964867 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v86) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff9781510d0 con 0x7ff9780692a0 2024-01-24T15:30:37.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.426+0000 7ff97e633700 1 -- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff96405f270 msgr2=0x7ff964061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.426+0000 7ff97e633700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff96405f270 0x7ff964061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7ff9780abaf0 tx=0x7ff968008040).stop 2024-01-24T15:30:37.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.426+0000 7ff97e633700 1 -- 172.21.15.29:0/1060964867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9780692a0 msgr2=0x7ff978149720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.426+0000 7ff97e633700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9780692a0 0x7ff978149720 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7ff96c00c440 tx=0x7ff96c007e30).stop 2024-01-24T15:30:37.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.430+0000 7ff97e633700 1 -- 172.21.15.29:0/1060964867 shutdown_connections 2024-01-24T15:30:37.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.430+0000 7ff97e633700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff978069d90 0x7ff97814faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.430+0000 7ff97e633700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff96405f270 0x7ff964061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.430+0000 7ff97e633700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff978066370 0x7ff97814a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.430+0000 7ff97e633700 1 --2- 172.21.15.29:0/1060964867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9780692a0 0x7ff978149720 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.430+0000 7ff97e633700 1 -- 172.21.15.29:0/1060964867 >> 172.21.15.29:0/1060964867 conn(0x7ff978064390 msgr2=0x7ff9780aaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:37.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.434+0000 7ff97e633700 1 -- 172.21.15.29:0/1060964867 shutdown_connections 2024-01-24T15:30:37.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.435+0000 7ff97e633700 1 -- 172.21.15.29:0/1060964867 wait complete. 2024-01-24T15:30:37.443 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 24 and data pool 25 2024-01-24T15:30:37.459 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:30:37.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.586+0000 7f8e5b14e700 1 Processor -- start 2024-01-24T15:30:37.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.586+0000 7f8e5b14e700 1 -- start start 2024-01-24T15:30:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.587+0000 7f8e5b14e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8e54066c00 0x7f8e540bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.587+0000 7f8e5b14e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e540670a0 0x7f8e54158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.587+0000 7f8e5b14e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e541582b0 con 0x7f8e540c4a10 2024-01-24T15:30:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.587+0000 7f8e5b14e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e54157ca0 con 0x7f8e540670a0 2024-01-24T15:30:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.587+0000 7f8e5b14e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e54063c60 con 0x7f8e54066c00 2024-01-24T15:30:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.587+0000 7f8e596eb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8e540670a0 0x7f8e54158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.587+0000 7f8e58eea700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8e540c4a10 0x7f8e540664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41934/0 (socket says 172.21.15.29:41934) 2024-01-24T15:30:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.587+0000 7f8e58eea700 1 -- 172.21.15.29:0/4233975882 learned_addr learned my addr 172.21.15.29:0/4233975882 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.588+0000 7f8e53fff700 1 --2- 172.21.15.29:0/4233975882 >> v2:172.21.15.163:3300/0 conn(0x7f8e54066c00 0x7f8e540bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.588+0000 7f8e537fe700 1 -- 172.21.15.29:0/4233975882 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443557982 0 0) 0x7f8e541582b0 con 0x7f8e540c4a10 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.588+0000 7f8e537fe700 1 -- 172.21.15.29:0/4233975882 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e440033f0 con 0x7f8e540c4a10 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.588+0000 7f8e596eb700 1 -- 172.21.15.29:0/4233975882 >> v2:172.21.15.163:3300/0 conn(0x7f8e54066c00 msgr2=0x7f8e540bf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.588+0000 7f8e596eb700 1 --2- 172.21.15.29:0/4233975882 >> v2:172.21.15.163:3300/0 conn(0x7f8e54066c00 0x7f8e540bf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.588+0000 7f8e596eb700 1 -- 172.21.15.29:0/4233975882 >> v1:172.21.15.29:6789/0 conn(0x7f8e540c4a10 legacy=0x7f8e540664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.588+0000 7f8e596eb700 1 -- 172.21.15.29:0/4233975882 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e54157f40 con 0x7f8e540670a0 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.588+0000 7f8e53fff700 1 --2- 172.21.15.29:0/4233975882 >> v2:172.21.15.163:3300/0 conn(0x7f8e54066c00 0x7f8e540bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.588+0000 7f8e596eb700 1 --2- 172.21.15.29:0/4233975882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e540670a0 0x7f8e54158740 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7f8e54068cb0 tx=0x7f8e4c009780).ready entity=mon.0 client_cookie=95d4c9e332d42b7f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.588+0000 7f8e537fe700 1 -- 172.21.15.29:0/4233975882 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00e070 con 0x7f8e540670a0 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.588+0000 7f8e537fe700 1 -- 172.21.15.29:0/4233975882 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c00bb60 con 0x7f8e540670a0 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.588+0000 7f8e537fe700 1 -- 172.21.15.29:0/4233975882 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c012410 con 0x7f8e540670a0 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.589+0000 7f8e5b14e700 1 -- 172.21.15.29:0/4233975882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e540670a0 msgr2=0x7f8e54158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.589+0000 7f8e5b14e700 1 --2- 172.21.15.29:0/4233975882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e540670a0 0x7f8e54158740 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7f8e54068cb0 tx=0x7f8e4c009780).stop 2024-01-24T15:30:37.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.589+0000 7f8e5b14e700 1 -- 172.21.15.29:0/4233975882 shutdown_connections 2024-01-24T15:30:37.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.589+0000 7f8e5b14e700 1 --2- 172.21.15.29:0/4233975882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e540670a0 0x7f8e54158740 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.589+0000 7f8e5b14e700 1 --2- 172.21.15.29:0/4233975882 >> v2:172.21.15.163:3300/0 conn(0x7f8e54066c00 0x7f8e540bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.589+0000 7f8e5b14e700 1 -- 172.21.15.29:0/4233975882 >> 172.21.15.29:0/4233975882 conn(0x7f8e540c8bb0 msgr2=0x7f8e540c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:37.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.589+0000 7f8e5b14e700 1 -- 172.21.15.29:0/4233975882 shutdown_connections 2024-01-24T15:30:37.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.589+0000 7f8e5b14e700 1 -- 172.21.15.29:0/4233975882 wait complete. 2024-01-24T15:30:37.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.590+0000 7f8e5b14e700 1 Processor -- start 2024-01-24T15:30:37.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.590+0000 7f8e5b14e700 1 -- start start 2024-01-24T15:30:37.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e5b14e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e54066c00 0x7f8e540aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e5b14e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e540670a0 0x7f8e54164b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e5b14e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e540c4a10 0x7f8e541516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e5b14e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e540ac5d0 con 0x7f8e540670a0 2024-01-24T15:30:37.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e5b14e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e54165050 con 0x7f8e540c4a10 2024-01-24T15:30:37.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e5b14e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e540ae770 con 0x7f8e54066c00 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e53fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e540670a0 0x7f8e54164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e53fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e540670a0 0x7f8e54164b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37282/0 (socket says 172.21.15.29:37282) 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e53fff700 1 -- 172.21.15.29:0/371410398 learned_addr learned my addr 172.21.15.29:0/371410398 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e596eb700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e540c4a10 0x7f8e541516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e58eea700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e54066c00 0x7f8e540aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e53fff700 1 -- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e54066c00 msgr2=0x7f8e540aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e53fff700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e54066c00 0x7f8e540aca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e53fff700 1 -- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e540c4a10 msgr2=0x7f8e541516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.591+0000 7f8e53fff700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e540c4a10 0x7f8e541516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.592+0000 7f8e53fff700 1 -- 172.21.15.29:0/371410398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c009a20 con 0x7f8e540670a0 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.592+0000 7f8e53fff700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e540670a0 0x7f8e54164b60 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 rx=0x7f8e4000d860 tx=0x7f8e40008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.592+0000 7f8e50ff9700 1 -- 172.21.15.29:0/371410398 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000f520 con 0x7f8e540670a0 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.592+0000 7f8e50ff9700 1 -- 172.21.15.29:0/371410398 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4000f680 con 0x7f8e540670a0 2024-01-24T15:30:37.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.592+0000 7f8e50ff9700 1 -- 172.21.15.29:0/371410398 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000fa00 con 0x7f8e540670a0 2024-01-24T15:30:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.592+0000 7f8e5b14e700 1 -- 172.21.15.29:0/371410398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e540aea50 con 0x7f8e540670a0 2024-01-24T15:30:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.592+0000 7f8e5b14e700 1 -- 172.21.15.29:0/371410398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e540ac260 con 0x7f8e540670a0 2024-01-24T15:30:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.593+0000 7f8e5b14e700 1 -- 172.21.15.29:0/371410398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e540475e0 con 0x7f8e540670a0 2024-01-24T15:30:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.593+0000 7f8e50ff9700 1 -- 172.21.15.29:0/371410398 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8e400123e0 con 0x7f8e540670a0 2024-01-24T15:30:37.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.594+0000 7f8e50ff9700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f8e3805f270 0x7f8e38061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.594+0000 7f8e50ff9700 1 -- 172.21.15.29:0/371410398 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f8e4007a3c0 con 0x7f8e540670a0 2024-01-24T15:30:37.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.594+0000 7f8e58eea700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f8e3805f270 0x7f8e38061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.594+0000 7f8e58eea700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f8e3805f270 0x7f8e38061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8e48005750 tx=0x7f8e48009040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:37.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.598+0000 7f8e50ff9700 1 -- 172.21.15.29:0/371410398 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f8e4004b080 con 0x7f8e540670a0 2024-01-24T15:30:37.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.778+0000 7f8e5b14e700 1 -- 172.21.15.29:0/371410398 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e540ac710 con 0x7f8e540670a0 2024-01-24T15:30:37.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.779+0000 7f8e50ff9700 1 -- 172.21.15.29:0/371410398 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+18270 (secure 0 0 0) 0x7f8e4004e6a0 con 0x7f8e540670a0 2024-01-24T15:30:37.781 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:37.781 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":212,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:30:37.410964+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:30:35.147040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-01-24T15:30:36.390723+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:30:37.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.781+0000 7f8e5b14e700 1 -- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f8e3805f270 msgr2=0x7f8e38061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.781+0000 7f8e5b14e700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f8e3805f270 0x7f8e38061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8e48005750 tx=0x7f8e48009040).stop 2024-01-24T15:30:37.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.781+0000 7f8e5b14e700 1 -- 172.21.15.29:0/371410398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e540670a0 msgr2=0x7f8e54164b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.781+0000 7f8e5b14e700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e540670a0 0x7f8e54164b60 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 rx=0x7f8e4000d860 tx=0x7f8e40008040).stop 2024-01-24T15:30:37.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.781+0000 7f8e5b14e700 1 -- 172.21.15.29:0/371410398 shutdown_connections 2024-01-24T15:30:37.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.781+0000 7f8e5b14e700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e540c4a10 0x7f8e541516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.781+0000 7f8e5b14e700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f8e3805f270 0x7f8e38061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.781+0000 7f8e5b14e700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e54066c00 0x7f8e540aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.781+0000 7f8e5b14e700 1 --2- 172.21.15.29:0/371410398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e540670a0 0x7f8e54164b60 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.781+0000 7f8e5b14e700 1 -- 172.21.15.29:0/371410398 >> 172.21.15.29:0/371410398 conn(0x7f8e540c8bb0 msgr2=0x7f8e540664f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:37.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.781+0000 7f8e5b14e700 1 -- 172.21.15.29:0/371410398 shutdown_connections 2024-01-24T15:30:37.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.781+0000 7f8e5b14e700 1 -- 172.21.15.29:0/371410398 wait complete. 2024-01-24T15:30:37.797 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:30:37.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.925+0000 7f55b6b42700 1 Processor -- start 2024-01-24T15:30:37.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.925+0000 7f55b6b42700 1 -- start start 2024-01-24T15:30:37.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.926+0000 7f55b6b42700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f55b014e5a0 0x7f55b00a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.926+0000 7f55b6b42700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b0068440 0x7f55b00a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.926+0000 7f55b6b42700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b01545c0 con 0x7f55b00638c0 2024-01-24T15:30:37.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.926+0000 7f55b6b42700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b0068920 con 0x7f55b014e5a0 2024-01-24T15:30:37.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.926+0000 7f55b6b42700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b01551c0 con 0x7f55b0068440 2024-01-24T15:30:37.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.926+0000 7f55b48de700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f55b014e5a0 0x7f55b00a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.926+0000 7f55b48de700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f55b014e5a0 0x7f55b00a7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37292/0 (socket says 172.21.15.29:37292) 2024-01-24T15:30:37.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.926+0000 7f55b48de700 1 -- 172.21.15.29:0/1363940777 learned_addr learned my addr 172.21.15.29:0/1363940777 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:37.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.926+0000 7f55affff700 1 --2- 172.21.15.29:0/1363940777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b0068440 0x7f55b00a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55b48de700 1 -- 172.21.15.29:0/1363940777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b0068440 msgr2=0x7f55b00a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:37.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55b48de700 1 --2- 172.21.15.29:0/1363940777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b0068440 0x7f55b00a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55b48de700 1 -- 172.21.15.29:0/1363940777 >> v1:172.21.15.163:6789/0 conn(0x7f55b00638c0 legacy=0x7f55b00664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55b48de700 1 -- 172.21.15.29:0/1363940777 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b01555e0 con 0x7f55b014e5a0 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55b48de700 1 --2- 172.21.15.29:0/1363940777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b014e5a0 0x7f55b00a7ac0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f55a4006fd0 tx=0x7f55a40090f0).ready entity=mon.0 client_cookie=92f70156611808cc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55af7fe700 1 -- 172.21.15.29:0/1363940777 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400b600 con 0x7f55b014e5a0 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55af7fe700 1 -- 172.21.15.29:0/1363940777 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a400b760 con 0x7f55b014e5a0 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55af7fe700 1 -- 172.21.15.29:0/1363940777 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400bae0 con 0x7f55b014e5a0 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55b6b42700 1 -- 172.21.15.29:0/1363940777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b014e5a0 msgr2=0x7f55b00a7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55b6b42700 1 --2- 172.21.15.29:0/1363940777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b014e5a0 0x7f55b00a7ac0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f55a4006fd0 tx=0x7f55a40090f0).stop 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55b6b42700 1 -- 172.21.15.29:0/1363940777 shutdown_connections 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55b6b42700 1 --2- 172.21.15.29:0/1363940777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b014e5a0 0x7f55b00a7ac0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55b6b42700 1 --2- 172.21.15.29:0/1363940777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b0068440 0x7f55b00a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.927+0000 7f55b6b42700 1 -- 172.21.15.29:0/1363940777 >> 172.21.15.29:0/1363940777 conn(0x7f55b00c8bb0 msgr2=0x7f55b00c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.928+0000 7f55b6b42700 1 -- 172.21.15.29:0/1363940777 shutdown_connections 2024-01-24T15:30:37.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.928+0000 7f55b6b42700 1 -- 172.21.15.29:0/1363940777 wait complete. 2024-01-24T15:30:37.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.929+0000 7f55b6b42700 1 Processor -- start 2024-01-24T15:30:37.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.929+0000 7f55b6b42700 1 -- start start 2024-01-24T15:30:37.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b6b42700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b00638c0 0x7f55b0155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b6b42700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b0068440 0x7f55b014b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b6b42700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b014e5a0 0x7f55b00ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b6b42700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b00ab0e0 con 0x7f55b014e5a0 2024-01-24T15:30:37.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b6b42700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b00a8680 con 0x7f55b0068440 2024-01-24T15:30:37.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b6b42700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b00a8960 con 0x7f55b00638c0 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b50df700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b014e5a0 0x7f55b00ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b50df700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b014e5a0 0x7f55b00ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37300/0 (socket says 172.21.15.29:37300) 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b50df700 1 -- 172.21.15.29:0/1641942378 learned_addr learned my addr 172.21.15.29:0/1641942378 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b48de700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b00638c0 0x7f55b0155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55affff700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b0068440 0x7f55b014b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b50df700 1 -- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b00638c0 msgr2=0x7f55b0155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b50df700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b00638c0 0x7f55b0155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b50df700 1 -- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b0068440 msgr2=0x7f55b014b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b50df700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b0068440 0x7f55b014b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.930+0000 7f55b50df700 1 -- 172.21.15.29:0/1641942378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4009370 con 0x7f55b014e5a0 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.931+0000 7f55b50df700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b014e5a0 0x7f55b00ad080 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7f55a8002230 tx=0x7f55a80015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.931+0000 7f55acff9700 1 -- 172.21.15.29:0/1641942378 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a800e4a0 con 0x7f55b014e5a0 2024-01-24T15:30:37.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.931+0000 7f55acff9700 1 -- 172.21.15.29:0/1641942378 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a800e600 con 0x7f55b014e5a0 2024-01-24T15:30:37.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.931+0000 7f55acff9700 1 -- 172.21.15.29:0/1641942378 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a800e980 con 0x7f55b014e5a0 2024-01-24T15:30:37.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.931+0000 7f55b6b42700 1 -- 172.21.15.29:0/1641942378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b00a8c40 con 0x7f55b014e5a0 2024-01-24T15:30:37.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.931+0000 7f55b6b42700 1 -- 172.21.15.29:0/1641942378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b014bb40 con 0x7f55b014e5a0 2024-01-24T15:30:37.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.932+0000 7f55b6b42700 1 -- 172.21.15.29:0/1641942378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b00a8e30 con 0x7f55b014e5a0 2024-01-24T15:30:37.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.932+0000 7f55acff9700 1 -- 172.21.15.29:0/1641942378 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f55a801b3b0 con 0x7f55b014e5a0 2024-01-24T15:30:37.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.933+0000 7f55acff9700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f559405f270 0x7f5594061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:37.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.933+0000 7f55acff9700 1 -- 172.21.15.29:0/1641942378 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f55a8079e00 con 0x7f55b014e5a0 2024-01-24T15:30:37.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.933+0000 7f55b48de700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f559405f270 0x7f5594061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:37.937 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.933+0000 7f55b48de700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f559405f270 0x7f5594061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f55a4009130 tx=0x7f55a4017000).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:37.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:37.937+0000 7f55acff9700 1 -- 172.21.15.29:0/1641942378 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f55a8048b00 con 0x7f55b014e5a0 2024-01-24T15:30:38.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.117+0000 7f55b6b42700 1 -- 172.21.15.29:0/1641942378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55b00475e0 con 0x7f55b014e5a0 2024-01-24T15:30:38.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.118+0000 7f55acff9700 1 -- 172.21.15.29:0/1641942378 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+18270 (secure 0 0 0) 0x7f55a8079020 con 0x7f55b014e5a0 2024-01-24T15:30:38.121 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:38.121 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":212,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:30:37.410964+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:30:35.147040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-01-24T15:30:36.390723+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:30:38.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.120+0000 7f55b6b42700 1 -- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f559405f270 msgr2=0x7f5594061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:38.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.120+0000 7f55b6b42700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f559405f270 0x7f5594061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f55a4009130 tx=0x7f55a4017000).stop 2024-01-24T15:30:38.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.120+0000 7f55b6b42700 1 -- 172.21.15.29:0/1641942378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b014e5a0 msgr2=0x7f55b00ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:38.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.120+0000 7f55b6b42700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b014e5a0 0x7f55b00ad080 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7f55a8002230 tx=0x7f55a80015e0).stop 2024-01-24T15:30:38.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.120+0000 7f55b6b42700 1 -- 172.21.15.29:0/1641942378 shutdown_connections 2024-01-24T15:30:38.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.120+0000 7f55b6b42700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55b014e5a0 0x7f55b00ad080 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:38.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.120+0000 7f55b6b42700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f559405f270 0x7f5594061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:38.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.120+0000 7f55b6b42700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55b00638c0 0x7f55b0155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:38.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.120+0000 7f55b6b42700 1 --2- 172.21.15.29:0/1641942378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55b0068440 0x7f55b014b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:38.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.120+0000 7f55b6b42700 1 -- 172.21.15.29:0/1641942378 >> 172.21.15.29:0/1641942378 conn(0x7f55b00c8bb0 msgr2=0x7f55b00c79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:38.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.120+0000 7f55b6b42700 1 -- 172.21.15.29:0/1641942378 shutdown_connections 2024-01-24T15:30:38.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.120+0000 7f55b6b42700 1 -- 172.21.15.29:0/1641942378 wait complete. 2024-01-24T15:30:38.136 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-24T15:30:38.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.268+0000 7f73194f8700 1 Processor -- start 2024-01-24T15:30:38.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.268+0000 7f73194f8700 1 -- start start 2024-01-24T15:30:38.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.268+0000 7f73194f8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7314064760 0x7f7314064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:38.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f73194f8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f73140c8490 0x7f73140a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:38.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f73194f8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7314152240 con 0x7f7314154700 2024-01-24T15:30:38.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f73194f8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73141586c0 con 0x7f7314064760 2024-01-24T15:30:38.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f73194f8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73140c0d50 con 0x7f73140c8490 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f7312ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7314064760 0x7f7314064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f73137fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7314154700 0x7f7314154b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41950/0 (socket says 172.21.15.29:41950) 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f73137fe700 1 -- 172.21.15.29:0/35915029 learned_addr learned my addr 172.21.15.29:0/35915029 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f73127fc700 1 --2- 172.21.15.29:0/35915029 >> v2:172.21.15.163:3300/0 conn(0x7f73140c8490 0x7f73140a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f7312ffd700 1 -- 172.21.15.29:0/35915029 >> v2:172.21.15.163:3300/0 conn(0x7f73140c8490 msgr2=0x7f73140a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f7312ffd700 1 --2- 172.21.15.29:0/35915029 >> v2:172.21.15.163:3300/0 conn(0x7f73140c8490 0x7f73140a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f7311ffb700 1 -- 172.21.15.29:0/35915029 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047161227 0 0) 0x7f7314152240 con 0x7f7314154700 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f7312ffd700 1 -- 172.21.15.29:0/35915029 >> v1:172.21.15.29:6789/0 conn(0x7f7314154700 legacy=0x7f7314154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f7312ffd700 1 -- 172.21.15.29:0/35915029 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73140aa670 con 0x7f7314064760 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f7312ffd700 1 --2- 172.21.15.29:0/35915029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7314064760 0x7f7314064b40 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f73140c83d0 tx=0x7f7304009060).ready entity=mon.0 client_cookie=a08953e5ada362b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f7311ffb700 1 -- 172.21.15.29:0/35915029 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730400f510 con 0x7f7314064760 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.269+0000 7f7311ffb700 1 -- 172.21.15.29:0/35915029 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730400f670 con 0x7f7314064760 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.270+0000 7f7311ffb700 1 -- 172.21.15.29:0/35915029 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730400f9f0 con 0x7f7314064760 2024-01-24T15:30:38.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.270+0000 7f73194f8700 1 -- 172.21.15.29:0/35915029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7314064760 msgr2=0x7f7314064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:38.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.270+0000 7f73194f8700 1 --2- 172.21.15.29:0/35915029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7314064760 0x7f7314064b40 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f73140c83d0 tx=0x7f7304009060).stop 2024-01-24T15:30:38.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.270+0000 7f73194f8700 1 -- 172.21.15.29:0/35915029 shutdown_connections 2024-01-24T15:30:38.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.270+0000 7f73194f8700 1 --2- 172.21.15.29:0/35915029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7314064760 0x7f7314064b40 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:38.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.270+0000 7f73194f8700 1 --2- 172.21.15.29:0/35915029 >> v2:172.21.15.163:3300/0 conn(0x7f73140c8490 0x7f73140a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:38.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.270+0000 7f73194f8700 1 -- 172.21.15.29:0/35915029 >> 172.21.15.29:0/35915029 conn(0x7f73140cb8b0 msgr2=0x7f73140c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:38.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.270+0000 7f73194f8700 1 -- 172.21.15.29:0/35915029 shutdown_connections 2024-01-24T15:30:38.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.270+0000 7f73194f8700 1 -- 172.21.15.29:0/35915029 wait complete. 2024-01-24T15:30:38.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.271+0000 7f73194f8700 1 Processor -- start 2024-01-24T15:30:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.272+0000 7f73194f8700 1 -- start start 2024-01-24T15:30:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.272+0000 7f73194f8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7314064760 0x7f7314164800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.272+0000 7f73194f8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73140c8490 0x7f73140ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.272+0000 7f73194f8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7314154700 0x7f73140c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.272+0000 7f73194f8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73140c3360 con 0x7f7314064760 2024-01-24T15:30:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.272+0000 7f73194f8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73140c2750 con 0x7f7314154700 2024-01-24T15:30:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.272+0000 7f73194f8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73140c2a30 con 0x7f73140c8490 2024-01-24T15:30:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.272+0000 7f7312ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7314064760 0x7f7314164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.272+0000 7f7312ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7314064760 0x7f7314164800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37326/0 (socket says 172.21.15.29:37326) 2024-01-24T15:30:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.272+0000 7f7312ffd700 1 -- 172.21.15.29:0/2751275202 learned_addr learned my addr 172.21.15.29:0/2751275202 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f73137fe700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7314154700 0x7f73140c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:38.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f73127fc700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73140c8490 0x7f73140ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f7312ffd700 1 -- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73140c8490 msgr2=0x7f73140ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f7312ffd700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73140c8490 0x7f73140ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f7312ffd700 1 -- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7314154700 msgr2=0x7f73140c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f7312ffd700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7314154700 0x7f73140c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f7312ffd700 1 -- 172.21.15.29:0/2751275202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73040092a0 con 0x7f7314064760 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f73127fc700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73140c8490 0x7f73140ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f7312ffd700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7314064760 0x7f7314164800 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7f7304009dd0 tx=0x7f7304017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f72faffd700 1 -- 172.21.15.29:0/2751275202 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730401a4d0 con 0x7f7314064760 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f72faffd700 1 -- 172.21.15.29:0/2751275202 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730401e070 con 0x7f7314064760 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f73194f8700 1 -- 172.21.15.29:0/2751275202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73140c2d10 con 0x7f7314064760 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f72faffd700 1 -- 172.21.15.29:0/2751275202 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730401a880 con 0x7f7314064760 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.273+0000 7f73194f8700 1 -- 172.21.15.29:0/2751275202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73140c2390 con 0x7f7314064760 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.274+0000 7f73194f8700 1 -- 172.21.15.29:0/2751275202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7314155f30 con 0x7f7314064760 2024-01-24T15:30:38.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.275+0000 7f72faffd700 1 -- 172.21.15.29:0/2751275202 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7304020030 con 0x7f7314064760 2024-01-24T15:30:38.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.275+0000 7f72faffd700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f730005f240 0x7f7300061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:38.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.275+0000 7f72faffd700 1 -- 172.21.15.29:0/2751275202 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f730407b410 con 0x7f7314064760 2024-01-24T15:30:38.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.276+0000 7f73127fc700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f730005f240 0x7f7300061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:38.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.276+0000 7f73127fc700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f730005f240 0x7f7300061700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f73140caa20 tx=0x7f7308008040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:38.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.280+0000 7f72faffd700 1 -- 172.21.15.29:0/2751275202 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f730404c0d0 con 0x7f7314064760 2024-01-24T15:30:38.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:38.506+0000 7f73194f8700 1 -- 172.21.15.29:0/2751275202 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f73140475e0 con 0x7f7314064760 2024-01-24T15:30:38.658 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-01-24T15:30:38.658 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-01-24T15:30:39.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.428+0000 7f72faffd700 1 -- 172.21.15.29:0/2751275202 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) v1 ==== 118+0+0 (secure 0 0 0) 0x7f730404f6f0 con 0x7f7314064760 2024-01-24T15:30:39.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.431+0000 7f73194f8700 1 -- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f730005f240 msgr2=0x7f7300061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:39.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.431+0000 7f73194f8700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f730005f240 0x7f7300061700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f73140caa20 tx=0x7f7308008040).stop 2024-01-24T15:30:39.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.431+0000 7f73194f8700 1 -- 172.21.15.29:0/2751275202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7314064760 msgr2=0x7f7314164800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:39.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.431+0000 7f73194f8700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7314064760 0x7f7314164800 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7f7304009dd0 tx=0x7f7304017000).stop 2024-01-24T15:30:39.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.431+0000 7f73194f8700 1 -- 172.21.15.29:0/2751275202 shutdown_connections 2024-01-24T15:30:39.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.431+0000 7f73194f8700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7314154700 0x7f73140c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:39.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.431+0000 7f73194f8700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7314064760 0x7f7314164800 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:39.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.431+0000 7f73194f8700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f730005f240 0x7f7300061700 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:39.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.431+0000 7f73194f8700 1 --2- 172.21.15.29:0/2751275202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73140c8490 0x7f73140ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:39.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.431+0000 7f73194f8700 1 -- 172.21.15.29:0/2751275202 >> 172.21.15.29:0/2751275202 conn(0x7f73140cb8b0 msgr2=0x7f73140bfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:39.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.431+0000 7f73194f8700 1 -- 172.21.15.29:0/2751275202 shutdown_connections 2024-01-24T15:30:39.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.431+0000 7f73194f8700 1 -- 172.21.15.29:0/2751275202 wait complete. 2024-01-24T15:30:39.449 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-24T15:30:39.449 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-24T15:30:39.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.583+0000 7f3c6c7be700 1 Processor -- start 2024-01-24T15:30:39.586 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.583+0000 7f3c6c7be700 1 -- start start 2024-01-24T15:30:39.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.584+0000 7f3c6c7be700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c64156240 0x7f3c6415ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:39.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.584+0000 7f3c6c7be700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c64063fc0 con 0x7f3c64155290 2024-01-24T15:30:39.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.584+0000 7f3c6c7be700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c64156920 con 0x7f3c64155670 2024-01-24T15:30:39.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.584+0000 7f3c6c7be700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c64065530 con 0x7f3c64156240 2024-01-24T15:30:39.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.584+0000 7f3c6a55a700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3c64155290 0x7f3c640bc2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41962/0 (socket says 172.21.15.29:41962) 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.584+0000 7f3c6a55a700 1 -- 172.21.15.29:0/2049203917 learned_addr learned my addr 172.21.15.29:0/2049203917 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.585+0000 7f3c6ad5b700 1 --2- 172.21.15.29:0/2049203917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c64156240 0x7f3c6415ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.585+0000 7f3c69558700 1 -- 172.21.15.29:0/2049203917 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464957870 0 0) 0x7f3c64063fc0 con 0x7f3c64155290 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.585+0000 7f3c69558700 1 -- 172.21.15.29:0/2049203917 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c500033f0 con 0x7f3c64155290 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.585+0000 7f3c69558700 1 -- 172.21.15.29:0/2049203917 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005230130 0 0) 0x7f3c64156920 con 0x7f3c64155670 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.585+0000 7f3c69558700 1 -- 172.21.15.29:0/2049203917 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c64063fc0 con 0x7f3c64155670 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.585+0000 7f3c69558700 1 -- 172.21.15.29:0/2049203917 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3726079928 0 0) 0x7f3c500033f0 con 0x7f3c64155290 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.585+0000 7f3c6ad5b700 1 -- 172.21.15.29:0/2049203917 >> v1:172.21.15.163:6789/0 conn(0x7f3c64155670 legacy=0x7f3c64155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.585+0000 7f3c6ad5b700 1 -- 172.21.15.29:0/2049203917 >> v1:172.21.15.29:6789/0 conn(0x7f3c64155290 legacy=0x7f3c640bc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.585+0000 7f3c6ad5b700 1 -- 172.21.15.29:0/2049203917 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c64065710 con 0x7f3c64156240 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.585+0000 7f3c6ad5b700 1 --2- 172.21.15.29:0/2049203917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c64156240 0x7f3c6415ce70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f3c5c009e90 tx=0x7f3c5c009780).ready entity=mon.2 client_cookie=f3684031b4fff044 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.585+0000 7f3c69558700 1 -- 172.21.15.29:0/2049203917 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00e070 con 0x7f3c64156240 2024-01-24T15:30:39.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.586+0000 7f3c69558700 1 -- 172.21.15.29:0/2049203917 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c00b7a0 con 0x7f3c64156240 2024-01-24T15:30:39.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.586+0000 7f3c69558700 1 -- 172.21.15.29:0/2049203917 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60001470 con 0x7f3c64156240 2024-01-24T15:30:39.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.586+0000 7f3c6c7be700 1 -- 172.21.15.29:0/2049203917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c64156240 msgr2=0x7f3c6415ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:39.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.586+0000 7f3c6c7be700 1 --2- 172.21.15.29:0/2049203917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c64156240 0x7f3c6415ce70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f3c5c009e90 tx=0x7f3c5c009780).stop 2024-01-24T15:30:39.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.586+0000 7f3c6c7be700 1 -- 172.21.15.29:0/2049203917 shutdown_connections 2024-01-24T15:30:39.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.586+0000 7f3c6c7be700 1 --2- 172.21.15.29:0/2049203917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c64156240 0x7f3c6415ce70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:39.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.586+0000 7f3c6c7be700 1 -- 172.21.15.29:0/2049203917 >> 172.21.15.29:0/2049203917 conn(0x7f3c640c7260 msgr2=0x7f3c640c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:39.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.586+0000 7f3c6c7be700 1 -- 172.21.15.29:0/2049203917 shutdown_connections 2024-01-24T15:30:39.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.586+0000 7f3c6c7be700 1 -- 172.21.15.29:0/2049203917 wait complete. 2024-01-24T15:30:39.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.587+0000 7f3c6c7be700 1 Processor -- start 2024-01-24T15:30:39.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.587+0000 7f3c6c7be700 1 -- start start 2024-01-24T15:30:39.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.588+0000 7f3c6c7be700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c64155290 0x7f3c640ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:39.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.588+0000 7f3c6c7be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c64155670 0x7f3c640abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:39.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.588+0000 7f3c6c7be700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c64156240 0x7f3c64147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:39.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.588+0000 7f3c6c7be700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640befc0 con 0x7f3c64156240 2024-01-24T15:30:39.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.588+0000 7f3c6c7be700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640beb30 con 0x7f3c64155670 2024-01-24T15:30:39.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.588+0000 7f3c6c7be700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640ac1d0 con 0x7f3c64155290 2024-01-24T15:30:39.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.588+0000 7f3c6ad5b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c64156240 0x7f3c64147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.588+0000 7f3c6ad5b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c64156240 0x7f3c64147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37332/0 (socket says 172.21.15.29:37332) 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.588+0000 7f3c6ad5b700 1 -- 172.21.15.29:0/1414916583 learned_addr learned my addr 172.21.15.29:0/1414916583 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.588+0000 7f3c6a55a700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c64155290 0x7f3c640ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.588+0000 7f3c69d59700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c64155670 0x7f3c640abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.589+0000 7f3c6ad5b700 1 -- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c64155290 msgr2=0x7f3c640ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.589+0000 7f3c6ad5b700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c64155290 0x7f3c640ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.589+0000 7f3c6ad5b700 1 -- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c64155670 msgr2=0x7f3c640abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.589+0000 7f3c6ad5b700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c64155670 0x7f3c640abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.589+0000 7f3c6ad5b700 1 -- 172.21.15.29:0/1414916583 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c009a20 con 0x7f3c64156240 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.589+0000 7f3c6ad5b700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c64156240 0x7f3c64147c80 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7f3c64068550 tx=0x7f3c5c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.589+0000 7f3c5a7fc700 1 -- 172.21.15.29:0/1414916583 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c007040 con 0x7f3c64156240 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.589+0000 7f3c5a7fc700 1 -- 172.21.15.29:0/1414916583 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c00bd80 con 0x7f3c64156240 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.589+0000 7f3c6c7be700 1 -- 172.21.15.29:0/1414916583 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c640be200 con 0x7f3c64156240 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.589+0000 7f3c5a7fc700 1 -- 172.21.15.29:0/1414916583 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c01e690 con 0x7f3c64156240 2024-01-24T15:30:39.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.589+0000 7f3c6c7be700 1 -- 172.21.15.29:0/1414916583 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c640be7c0 con 0x7f3c64156240 2024-01-24T15:30:39.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.590+0000 7f3c6c7be700 1 -- 172.21.15.29:0/1414916583 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c640becc0 con 0x7f3c64156240 2024-01-24T15:30:39.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.590+0000 7f3c5a7fc700 1 -- 172.21.15.29:0/1414916583 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3c5c003040 con 0x7f3c64156240 2024-01-24T15:30:39.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.591+0000 7f3c5a7fc700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3c4805f270 0x7f3c48061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:39.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.591+0000 7f3c5a7fc700 1 -- 172.21.15.29:0/1414916583 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f3c5c07c040 con 0x7f3c64156240 2024-01-24T15:30:39.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.591+0000 7f3c6a55a700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3c4805f270 0x7f3c48061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:39.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.592+0000 7f3c6a55a700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3c4805f270 0x7f3c48061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f3c600015a0 tx=0x7f3c60000bc0).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:39.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.595+0000 7f3c5a7fc700 1 -- 172.21.15.29:0/1414916583 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3c5c04ce20 con 0x7f3c64156240 2024-01-24T15:30:39.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:39.823+0000 7f3c6c7be700 1 -- 172.21.15.29:0/1414916583 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3c640475e0 con 0x7f3c64156240 2024-01-24T15:30:40.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.435+0000 7f3c5a7fc700 1 -- 172.21.15.29:0/1414916583 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3c5c050440 con 0x7f3c64156240 2024-01-24T15:30:40.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.438+0000 7f3c6c7be700 1 -- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3c4805f270 msgr2=0x7f3c48061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:40.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.438+0000 7f3c6c7be700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3c4805f270 0x7f3c48061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f3c600015a0 tx=0x7f3c60000bc0).stop 2024-01-24T15:30:40.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.438+0000 7f3c6c7be700 1 -- 172.21.15.29:0/1414916583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c64156240 msgr2=0x7f3c64147c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:40.440 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.438+0000 7f3c6c7be700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c64156240 0x7f3c64147c80 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7f3c64068550 tx=0x7f3c5c009780).stop 2024-01-24T15:30:40.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.438+0000 7f3c6c7be700 1 -- 172.21.15.29:0/1414916583 shutdown_connections 2024-01-24T15:30:40.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.438+0000 7f3c6c7be700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c64156240 0x7f3c64147c80 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.438+0000 7f3c6c7be700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3c4805f270 0x7f3c48061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.438+0000 7f3c6c7be700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c64155290 0x7f3c640ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.438+0000 7f3c6c7be700 1 --2- 172.21.15.29:0/1414916583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c64155670 0x7f3c640abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.438+0000 7f3c6c7be700 1 -- 172.21.15.29:0/1414916583 >> 172.21.15.29:0/1414916583 conn(0x7f3c640c7260 msgr2=0x7f3c64151f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:40.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.438+0000 7f3c6c7be700 1 -- 172.21.15.29:0/1414916583 shutdown_connections 2024-01-24T15:30:40.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.438+0000 7f3c6c7be700 1 -- 172.21.15.29:0/1414916583 wait complete. 2024-01-24T15:30:40.455 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-24T15:30:40.455 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:30:40.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.588+0000 7f0d042a2700 1 Processor -- start 2024-01-24T15:30:40.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.588+0000 7f0d042a2700 1 -- start start 2024-01-24T15:30:40.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.589+0000 7f0d042a2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0cfc155b00 0x7f0cfc0664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:40.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.589+0000 7f0d042a2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0cfc0669e0 0x7f0cfc155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.589+0000 7f0d042a2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc1563d0 0x7f0cfc067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.589+0000 7f0d042a2700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cfc0c5500 con 0x7f0cfc155b00 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.589+0000 7f0d042a2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cfc066e90 con 0x7f0cfc0669e0 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.589+0000 7f0d042a2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc0c3910 con 0x7f0cfc1563d0 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.589+0000 7f0d0203e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0cfc155b00 0x7f0cfc0664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.589+0000 7f0d0203e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0cfc155b00 0x7f0cfc0664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37334/0 (socket says 172.21.15.29:37334) 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.589+0000 7f0d0203e700 1 -- 172.21.15.29:0/1932624708 learned_addr learned my addr 172.21.15.29:0/1932624708 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.590+0000 7f0d0183d700 1 --2- 172.21.15.29:0/1932624708 >> v2:172.21.15.163:3300/0 conn(0x7f0cfc0669e0 0x7f0cfc155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.590+0000 7f0d0283f700 1 --2- 172.21.15.29:0/1932624708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc1563d0 0x7f0cfc067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.590+0000 7f0d0203e700 1 -- 172.21.15.29:0/1932624708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc1563d0 msgr2=0x7f0cfc067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.590+0000 7f0d0203e700 1 --2- 172.21.15.29:0/1932624708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc1563d0 0x7f0cfc067a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.590+0000 7f0d0203e700 1 -- 172.21.15.29:0/1932624708 >> v2:172.21.15.163:3300/0 conn(0x7f0cfc0669e0 msgr2=0x7f0cfc155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:40.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.590+0000 7f0d0203e700 1 --2- 172.21.15.29:0/1932624708 >> v2:172.21.15.163:3300/0 conn(0x7f0cfc0669e0 0x7f0cfc155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.590+0000 7f0d0203e700 1 -- 172.21.15.29:0/1932624708 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc0c5220 con 0x7f0cfc155b00 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.590+0000 7f0d0203e700 1 --2- 172.21.15.29:0/1932624708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cfc155b00 0x7f0cfc0664f0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7f0cfc0c6960 tx=0x7f0cf80090f0).ready entity=mon.0 client_cookie=cb1229d1fea9ed7e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.590+0000 7f0d0103c700 1 -- 172.21.15.29:0/1932624708 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800f510 con 0x7f0cfc155b00 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.590+0000 7f0d0103c700 1 -- 172.21.15.29:0/1932624708 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf800f670 con 0x7f0cfc155b00 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.590+0000 7f0d0103c700 1 -- 172.21.15.29:0/1932624708 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800f9f0 con 0x7f0cfc155b00 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.591+0000 7f0d042a2700 1 -- 172.21.15.29:0/1932624708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cfc155b00 msgr2=0x7f0cfc0664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.591+0000 7f0d042a2700 1 --2- 172.21.15.29:0/1932624708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cfc155b00 0x7f0cfc0664f0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7f0cfc0c6960 tx=0x7f0cf80090f0).stop 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.591+0000 7f0d042a2700 1 -- 172.21.15.29:0/1932624708 shutdown_connections 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.591+0000 7f0d042a2700 1 --2- 172.21.15.29:0/1932624708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc1563d0 0x7f0cfc067a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.591+0000 7f0d042a2700 1 --2- 172.21.15.29:0/1932624708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cfc155b00 0x7f0cfc0664f0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.591+0000 7f0d042a2700 1 --2- 172.21.15.29:0/1932624708 >> v2:172.21.15.163:3300/0 conn(0x7f0cfc0669e0 0x7f0cfc155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.591+0000 7f0d042a2700 1 -- 172.21.15.29:0/1932624708 >> 172.21.15.29:0/1932624708 conn(0x7f0cfc0c8bb0 msgr2=0x7f0cfc0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.591+0000 7f0d042a2700 1 -- 172.21.15.29:0/1932624708 shutdown_connections 2024-01-24T15:30:40.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.591+0000 7f0d042a2700 1 -- 172.21.15.29:0/1932624708 wait complete. 2024-01-24T15:30:40.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.592+0000 7f0d042a2700 1 Processor -- start 2024-01-24T15:30:40.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.592+0000 7f0d042a2700 1 -- start start 2024-01-24T15:30:40.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d042a2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cfc0669e0 0x7f0cfc0aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:40.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d042a2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cfc155b00 0x7f0cfc0651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:40.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d042a2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc1563d0 0x7f0cfc150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d042a2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc164fa0 con 0x7f0cfc155b00 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d042a2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc164cc0 con 0x7f0cfc0669e0 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d042a2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc0ac3d0 con 0x7f0cfc1563d0 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d0183d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cfc155b00 0x7f0cfc0651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d0183d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cfc155b00 0x7f0cfc0651f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37350/0 (socket says 172.21.15.29:37350) 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d0183d700 1 -- 172.21.15.29:0/3943514799 learned_addr learned my addr 172.21.15.29:0/3943514799 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d0203e700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cfc0669e0 0x7f0cfc0aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d0283f700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc1563d0 0x7f0cfc150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d0183d700 1 -- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc1563d0 msgr2=0x7f0cfc150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d0183d700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc1563d0 0x7f0cfc150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d0183d700 1 -- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cfc0669e0 msgr2=0x7f0cfc0aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d0183d700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cfc0669e0 0x7f0cfc0aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.593+0000 7f0d0183d700 1 -- 172.21.15.29:0/3943514799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8009370 con 0x7f0cfc155b00 2024-01-24T15:30:40.599 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.594+0000 7f0d0183d700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cfc155b00 0x7f0cfc0651f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7f0cec00e930 tx=0x7f0cec009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:40.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.594+0000 7f0cf27fc700 1 -- 172.21.15.29:0/3943514799 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec009510 con 0x7f0cfc155b00 2024-01-24T15:30:40.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.594+0000 7f0cf27fc700 1 -- 172.21.15.29:0/3943514799 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec012600 con 0x7f0cfc155b00 2024-01-24T15:30:40.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.594+0000 7f0d042a2700 1 -- 172.21.15.29:0/3943514799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cfc0ac6b0 con 0x7f0cfc155b00 2024-01-24T15:30:40.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.594+0000 7f0cf27fc700 1 -- 172.21.15.29:0/3943514799 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec012980 con 0x7f0cfc155b00 2024-01-24T15:30:40.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.594+0000 7f0d042a2700 1 -- 172.21.15.29:0/3943514799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cfc0ad3a0 con 0x7f0cfc155b00 2024-01-24T15:30:40.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.595+0000 7f0d042a2700 1 -- 172.21.15.29:0/3943514799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cfc0acbd0 con 0x7f0cfc155b00 2024-01-24T15:30:40.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.595+0000 7f0cf27fc700 1 -- 172.21.15.29:0/3943514799 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0cec01b400 con 0x7f0cfc155b00 2024-01-24T15:30:40.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.596+0000 7f0cf27fc700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f0ce005f270 0x7f0ce0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:40.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.596+0000 7f0cf27fc700 1 -- 172.21.15.29:0/3943514799 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f0cec07ac60 con 0x7f0cfc155b00 2024-01-24T15:30:40.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.596+0000 7f0d0203e700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f0ce005f270 0x7f0ce0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:40.600 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.596+0000 7f0d0203e700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f0ce005f270 0x7f0ce0061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f0cf8009130 tx=0x7f0cf8017000).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:40.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.600+0000 7f0cf27fc700 1 -- 172.21.15.29:0/3943514799 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0cfc0acbd0 con 0x7f0cfc155b00 2024-01-24T15:30:40.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.826+0000 7f0d042a2700 1 -- 172.21.15.29:0/3943514799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cfc0acbd0 con 0x7f0cfc155b00 2024-01-24T15:30:40.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.826+0000 7f0cf27fc700 1 -- 172.21.15.29:0/3943514799 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f0cfc0acbd0 con 0x7f0cfc155b00 2024-01-24T15:30:40.828 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:40.829 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8060,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3418669722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3418669722},{"type":"v1","addr":"172.21.15.163:6836","nonce":3418669722}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8063,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/407488941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":407488941},{"type":"v1","addr":"172.21.15.163:6837","nonce":407488941}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4018429461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4018429461},{"type":"v1","addr":"172.21.15.29:6834","nonce":4018429461}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:30:37.411326+0000","modified":"2024-01-24T15:30:40.428239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2886498894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2886498894},{"type":"v1","addr":"172.21.15.29:6835","nonce":2886498894}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-24T15:30:40.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.828+0000 7f0d042a2700 1 -- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f0ce005f270 msgr2=0x7f0ce0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:40.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.828+0000 7f0d042a2700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f0ce005f270 0x7f0ce0061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f0cf8009130 tx=0x7f0cf8017000).stop 2024-01-24T15:30:40.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.828+0000 7f0d042a2700 1 -- 172.21.15.29:0/3943514799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cfc155b00 msgr2=0x7f0cfc0651f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:40.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.828+0000 7f0d042a2700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cfc155b00 0x7f0cfc0651f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7f0cec00e930 tx=0x7f0cec009060).stop 2024-01-24T15:30:40.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.828+0000 7f0d042a2700 1 -- 172.21.15.29:0/3943514799 shutdown_connections 2024-01-24T15:30:40.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.828+0000 7f0d042a2700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f0ce005f270 0x7f0ce0061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.828+0000 7f0d042a2700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc1563d0 0x7f0cfc150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.828+0000 7f0d042a2700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cfc0669e0 0x7f0cfc0aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.828+0000 7f0d042a2700 1 --2- 172.21.15.29:0/3943514799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cfc155b00 0x7f0cfc0651f0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.831 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.828+0000 7f0d042a2700 1 -- 172.21.15.29:0/3943514799 >> 172.21.15.29:0/3943514799 conn(0x7f0cfc0c8bb0 msgr2=0x7f0cfc13a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:40.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.829+0000 7f0d042a2700 1 -- 172.21.15.29:0/3943514799 shutdown_connections 2024-01-24T15:30:40.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.829+0000 7f0d042a2700 1 -- 172.21.15.29:0/3943514799 wait complete. 2024-01-24T15:30:40.832 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 90 2024-01-24T15:30:40.844 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:30:40.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.972+0000 7fcc8ffe7700 1 Processor -- start 2024-01-24T15:30:40.974 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.972+0000 7fcc8ffe7700 1 -- start start 2024-01-24T15:30:40.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.973+0000 7fcc8ffe7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc88151b10 0x7fcc88066210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:40.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.973+0000 7fcc8ffe7700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8806a5a0 con 0x7fcc88069240 2024-01-24T15:30:40.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.973+0000 7fcc8ffe7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc880696d0 con 0x7fcc8814eb80 2024-01-24T15:30:40.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.973+0000 7fcc8ffe7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc88069990 con 0x7fcc88151b10 2024-01-24T15:30:40.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.973+0000 7fcc8e584700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcc88069240 0x7fcc880c30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41978/0 (socket says 172.21.15.29:41978) 2024-01-24T15:30:40.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.973+0000 7fcc8e584700 1 -- 172.21.15.29:0/2423344704 learned_addr learned my addr 172.21.15.29:0/2423344704 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:40.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.974+0000 7fcc8d582700 1 --2- 172.21.15.29:0/2423344704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc88151b10 0x7fcc88066210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:40.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.974+0000 7fcc8cd81700 1 -- 172.21.15.29:0/2423344704 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1313410244 0 0) 0x7fcc8806a5a0 con 0x7fcc88069240 2024-01-24T15:30:40.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.974+0000 7fcc8cd81700 1 -- 172.21.15.29:0/2423344704 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc740033f0 con 0x7fcc88069240 2024-01-24T15:30:40.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.974+0000 7fcc8cd81700 1 -- 172.21.15.29:0/2423344704 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382694853 0 0) 0x7fcc880696d0 con 0x7fcc8814eb80 2024-01-24T15:30:40.978 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.974+0000 7fcc8cd81700 1 -- 172.21.15.29:0/2423344704 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc8806a5a0 con 0x7fcc8814eb80 2024-01-24T15:30:40.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.974+0000 7fcc8d582700 1 -- 172.21.15.29:0/2423344704 >> v1:172.21.15.163:6789/0 conn(0x7fcc8814eb80 legacy=0x7fcc88151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:40.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.974+0000 7fcc8d582700 1 -- 172.21.15.29:0/2423344704 >> v1:172.21.15.29:6789/0 conn(0x7fcc88069240 legacy=0x7fcc880c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:40.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.974+0000 7fcc8d582700 1 -- 172.21.15.29:0/2423344704 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc88069c70 con 0x7fcc88151b10 2024-01-24T15:30:40.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.974+0000 7fcc8d582700 1 --2- 172.21.15.29:0/2423344704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc88151b10 0x7fcc88066210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fcc78009790 tx=0x7fcc78009060).ready entity=mon.2 client_cookie=43e61fbbe2c36af1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:40.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.974+0000 7fcc8cd81700 1 -- 172.21.15.29:0/2423344704 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7800f510 con 0x7fcc88151b10 2024-01-24T15:30:40.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.974+0000 7fcc8cd81700 1 -- 172.21.15.29:0/2423344704 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7800f670 con 0x7fcc88151b10 2024-01-24T15:30:40.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.974+0000 7fcc8cd81700 1 -- 172.21.15.29:0/2423344704 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7800f9f0 con 0x7fcc88151b10 2024-01-24T15:30:40.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.975+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2423344704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc88151b10 msgr2=0x7fcc88066210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:40.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.975+0000 7fcc8ffe7700 1 --2- 172.21.15.29:0/2423344704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc88151b10 0x7fcc88066210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fcc78009790 tx=0x7fcc78009060).stop 2024-01-24T15:30:40.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.975+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2423344704 shutdown_connections 2024-01-24T15:30:40.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.975+0000 7fcc8ffe7700 1 --2- 172.21.15.29:0/2423344704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc88151b10 0x7fcc88066210 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.979 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.975+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2423344704 >> 172.21.15.29:0/2423344704 conn(0x7fcc880c85c0 msgr2=0x7fcc880c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:40.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.975+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2423344704 shutdown_connections 2024-01-24T15:30:40.980 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.975+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2423344704 wait complete. 2024-01-24T15:30:40.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.976+0000 7fcc8ffe7700 1 Processor -- start 2024-01-24T15:30:40.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.976+0000 7fcc8ffe7700 1 -- start start 2024-01-24T15:30:40.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.977+0000 7fcc8ffe7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc88069240 0x7fcc880a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:40.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.977+0000 7fcc8ffe7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc8814eb80 0x7fcc880ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:40.983 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.977+0000 7fcc8ffe7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc88151b10 0x7fcc880ac770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:40.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.977+0000 7fcc8ffe7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880a7630 con 0x7fcc88069240 2024-01-24T15:30:40.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.977+0000 7fcc8ffe7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880a9fd0 con 0x7fcc8814eb80 2024-01-24T15:30:40.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.977+0000 7fcc8ffe7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880a7880 con 0x7fcc88151b10 2024-01-24T15:30:40.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.977+0000 7fcc8dd83700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc88069240 0x7fcc880a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:40.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.977+0000 7fcc8dd83700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc88069240 0x7fcc880a9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37352/0 (socket says 172.21.15.29:37352) 2024-01-24T15:30:40.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.977+0000 7fcc8dd83700 1 -- 172.21.15.29:0/2838791910 learned_addr learned my addr 172.21.15.29:0/2838791910 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:40.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.977+0000 7fcc8d582700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc8814eb80 0x7fcc880ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:40.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.977+0000 7fcc8e584700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc88151b10 0x7fcc880ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:40.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.978+0000 7fcc8dd83700 1 -- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc88151b10 msgr2=0x7fcc880ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:40.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.978+0000 7fcc8dd83700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc88151b10 0x7fcc880ac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.978+0000 7fcc8dd83700 1 -- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc8814eb80 msgr2=0x7fcc880ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:40.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.978+0000 7fcc8dd83700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc8814eb80 0x7fcc880ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:40.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.978+0000 7fcc8dd83700 1 -- 172.21.15.29:0/2838791910 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc780092a0 con 0x7fcc88069240 2024-01-24T15:30:40.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.978+0000 7fcc8d582700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc8814eb80 0x7fcc880ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:40.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.978+0000 7fcc8dd83700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc88069240 0x7fcc880a9ae0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7fcc8400c3a0 tx=0x7fcc84007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:40.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.978+0000 7fcc7dffb700 1 -- 172.21.15.29:0/2838791910 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc840123f0 con 0x7fcc88069240 2024-01-24T15:30:40.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.978+0000 7fcc7dffb700 1 -- 172.21.15.29:0/2838791910 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc84012550 con 0x7fcc88069240 2024-01-24T15:30:40.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.978+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2838791910 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc880a7b60 con 0x7fcc88069240 2024-01-24T15:30:40.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.978+0000 7fcc7dffb700 1 -- 172.21.15.29:0/2838791910 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84012700 con 0x7fcc88069240 2024-01-24T15:30:40.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.978+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2838791910 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc880a8120 con 0x7fcc88069240 2024-01-24T15:30:40.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.979+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2838791910 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc880c4d80 con 0x7fcc88069240 2024-01-24T15:30:40.987 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.980+0000 7fcc7dffb700 1 -- 172.21.15.29:0/2838791910 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcc84012af0 con 0x7fcc88069240 2024-01-24T15:30:40.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.980+0000 7fcc7dffb700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc6c05f270 0x7fcc6c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:40.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.980+0000 7fcc7dffb700 1 -- 172.21.15.29:0/2838791910 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fcc84079b40 con 0x7fcc88069240 2024-01-24T15:30:40.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.980+0000 7fcc8d582700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc6c05f270 0x7fcc6c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:40.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.981+0000 7fcc8d582700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc6c05f270 0x7fcc6c061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fcc78009fd0 tx=0x7fcc78017000).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:40.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:40.984+0000 7fcc7dffb700 1 -- 172.21.15.29:0/2838791910 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcc84047b00 con 0x7fcc88069240 2024-01-24T15:30:41.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.166+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2838791910 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc880a8760 con 0x7fcc88069240 2024-01-24T15:30:41.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.167+0000 7fcc7dffb700 1 -- 172.21.15.29:0/2838791910 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) v1 ==== 74+0+18257 (secure 0 0 0) 0x7fcc880a8760 con 0x7fcc88069240 2024-01-24T15:30:41.170 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:41.170 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":213,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:30:38.413131+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:30:35.147040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-01-24T15:30:36.390723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:30:41.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.169+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc6c05f270 msgr2=0x7fcc6c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.169+0000 7fcc8ffe7700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc6c05f270 0x7fcc6c061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fcc78009fd0 tx=0x7fcc78017000).stop 2024-01-24T15:30:41.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.169+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2838791910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc88069240 msgr2=0x7fcc880a9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.169+0000 7fcc8ffe7700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc88069240 0x7fcc880a9ae0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7fcc8400c3a0 tx=0x7fcc84007c40).stop 2024-01-24T15:30:41.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.169+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2838791910 shutdown_connections 2024-01-24T15:30:41.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.169+0000 7fcc8ffe7700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc6c05f270 0x7fcc6c061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.169+0000 7fcc8ffe7700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc88151b10 0x7fcc880ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.169+0000 7fcc8ffe7700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc88069240 0x7fcc880a9ae0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.169+0000 7fcc8ffe7700 1 --2- 172.21.15.29:0/2838791910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc8814eb80 0x7fcc880ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.172 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.169+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2838791910 >> 172.21.15.29:0/2838791910 conn(0x7fcc880c85c0 msgr2=0x7fcc88063550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:41.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.169+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2838791910 shutdown_connections 2024-01-24T15:30:41.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.169+0000 7fcc8ffe7700 1 -- 172.21.15.29:0/2838791910 wait complete. 2024-01-24T15:30:41.185 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-24T15:30:41.186 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:30:41.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.314+0000 7fd491f02700 1 Processor -- start 2024-01-24T15:30:41.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.314+0000 7fd491f02700 1 -- start start 2024-01-24T15:30:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.315+0000 7fd491f02700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd48c069240 0x7fd48c065ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.315+0000 7fd491f02700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48c0c2ce0 con 0x7fd48c150d40 2024-01-24T15:30:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.315+0000 7fd491f02700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48c0663b0 con 0x7fd48c155040 2024-01-24T15:30:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.315+0000 7fd491f02700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c069890 con 0x7fd48c069240 2024-01-24T15:30:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.315+0000 7fd48b7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd48c150d40 0x7fd48c154930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:41994/0 (socket says 172.21.15.29:41994) 2024-01-24T15:30:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.315+0000 7fd48b7fe700 1 -- 172.21.15.29:0/3777126250 learned_addr learned my addr 172.21.15.29:0/3777126250 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:41.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.315+0000 7fd48bfff700 1 --2- 172.21.15.29:0/3777126250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd48c069240 0x7fd48c065ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.315+0000 7fd48a7fc700 1 -- 172.21.15.29:0/3777126250 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953781633 0 0) 0x7fd48c0c2ce0 con 0x7fd48c150d40 2024-01-24T15:30:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.315+0000 7fd48a7fc700 1 -- 172.21.15.29:0/3777126250 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4740033f0 con 0x7fd48c150d40 2024-01-24T15:30:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.316+0000 7fd48a7fc700 1 -- 172.21.15.29:0/3777126250 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341458691 0 0) 0x7fd48c0663b0 con 0x7fd48c155040 2024-01-24T15:30:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.316+0000 7fd48a7fc700 1 -- 172.21.15.29:0/3777126250 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd48c0c2ce0 con 0x7fd48c155040 2024-01-24T15:30:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.316+0000 7fd48bfff700 1 -- 172.21.15.29:0/3777126250 >> v1:172.21.15.163:6789/0 conn(0x7fd48c155040 legacy=0x7fd48c0c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.316+0000 7fd48bfff700 1 -- 172.21.15.29:0/3777126250 >> v1:172.21.15.29:6789/0 conn(0x7fd48c150d40 legacy=0x7fd48c154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.316+0000 7fd48bfff700 1 -- 172.21.15.29:0/3777126250 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c069b70 con 0x7fd48c069240 2024-01-24T15:30:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.316+0000 7fd48bfff700 1 --2- 172.21.15.29:0/3777126250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd48c069240 0x7fd48c065ec0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fd484009e90 tx=0x7fd484009780).ready entity=mon.2 client_cookie=1d362c2faa45b57e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.316+0000 7fd48a7fc700 1 -- 172.21.15.29:0/3777126250 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48400e070 con 0x7fd48c069240 2024-01-24T15:30:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.316+0000 7fd48a7fc700 1 -- 172.21.15.29:0/3777126250 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48400bab0 con 0x7fd48c069240 2024-01-24T15:30:41.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.316+0000 7fd48a7fc700 1 -- 172.21.15.29:0/3777126250 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48400be30 con 0x7fd48c069240 2024-01-24T15:30:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.317+0000 7fd491f02700 1 -- 172.21.15.29:0/3777126250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd48c069240 msgr2=0x7fd48c065ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.317+0000 7fd491f02700 1 --2- 172.21.15.29:0/3777126250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd48c069240 0x7fd48c065ec0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fd484009e90 tx=0x7fd484009780).stop 2024-01-24T15:30:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.317+0000 7fd491f02700 1 -- 172.21.15.29:0/3777126250 shutdown_connections 2024-01-24T15:30:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.317+0000 7fd491f02700 1 --2- 172.21.15.29:0/3777126250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd48c069240 0x7fd48c065ec0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.317+0000 7fd491f02700 1 -- 172.21.15.29:0/3777126250 >> 172.21.15.29:0/3777126250 conn(0x7fd48c0c8580 msgr2=0x7fd48c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.317+0000 7fd491f02700 1 -- 172.21.15.29:0/3777126250 shutdown_connections 2024-01-24T15:30:41.322 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.317+0000 7fd491f02700 1 -- 172.21.15.29:0/3777126250 wait complete. 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.318+0000 7fd491f02700 1 Processor -- start 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.318+0000 7fd491f02700 1 -- start start 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd491f02700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd48c069240 0x7fd48c0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd491f02700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd48c150d40 0x7fd48c062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd491f02700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd48c155040 0x7fd48c152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd491f02700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c063190 con 0x7fd48c150d40 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd491f02700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0aae70 con 0x7fd48c155040 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd491f02700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0aab40 con 0x7fd48c069240 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd48affd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd48c150d40 0x7fd48c062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd48affd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd48c150d40 0x7fd48c062ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37354/0 (socket says 172.21.15.29:37354) 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd48affd700 1 -- 172.21.15.29:0/123963599 learned_addr learned my addr 172.21.15.29:0/123963599 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd48bfff700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd48c155040 0x7fd48c152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd48b7fe700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd48c069240 0x7fd48c0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd48affd700 1 -- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd48c069240 msgr2=0x7fd48c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.323 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd48affd700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd48c069240 0x7fd48c0627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd48affd700 1 -- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd48c155040 msgr2=0x7fd48c152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd48affd700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd48c155040 0x7fd48c152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.319+0000 7fd48affd700 1 -- 172.21.15.29:0/123963599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd484009a20 con 0x7fd48c150d40 2024-01-24T15:30:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.320+0000 7fd48affd700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd48c150d40 0x7fd48c062ca0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7fd48000c420 tx=0x7fd480007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.320+0000 7fd4737fe700 1 -- 172.21.15.29:0/123963599 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480012400 con 0x7fd48c150d40 2024-01-24T15:30:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.320+0000 7fd4737fe700 1 -- 172.21.15.29:0/123963599 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd480012560 con 0x7fd48c150d40 2024-01-24T15:30:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.320+0000 7fd491f02700 1 -- 172.21.15.29:0/123963599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48c0c16b0 con 0x7fd48c150d40 2024-01-24T15:30:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.320+0000 7fd4737fe700 1 -- 172.21.15.29:0/123963599 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480012920 con 0x7fd48c150d40 2024-01-24T15:30:41.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.320+0000 7fd491f02700 1 -- 172.21.15.29:0/123963599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c0c1c70 con 0x7fd48c150d40 2024-01-24T15:30:41.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.321+0000 7fd491f02700 1 -- 172.21.15.29:0/123963599 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48c0ab000 con 0x7fd48c150d40 2024-01-24T15:30:41.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.321+0000 7fd4737fe700 1 -- 172.21.15.29:0/123963599 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd48001b3b0 con 0x7fd48c150d40 2024-01-24T15:30:41.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.322+0000 7fd4737fe700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd47805f270 0x7fd478061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.322+0000 7fd4737fe700 1 -- 172.21.15.29:0/123963599 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fd480079f40 con 0x7fd48c150d40 2024-01-24T15:30:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.322+0000 7fd48b7fe700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd47805f270 0x7fd478061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.326 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.322+0000 7fd48b7fe700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd47805f270 0x7fd478061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fd47c005750 tx=0x7fd47c009040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:41.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.326+0000 7fd4737fe700 1 -- 172.21.15.29:0/123963599 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd48004ac00 con 0x7fd48c150d40 2024-01-24T15:30:41.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.507+0000 7fd491f02700 1 -- 172.21.15.29:0/123963599 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd48c0c51a0 con 0x7fd47805f270 2024-01-24T15:30:41.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.519+0000 7fd4737fe700 1 -- 172.21.15.29:0/123963599 <== mgr.8938 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361949 (secure 0 0 0) 0x7fd48c0c51a0 con 0x7fd47805f270 2024-01-24T15:30:41.526 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:41.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.523+0000 7fd491f02700 1 -- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd47805f270 msgr2=0x7fd478061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.523+0000 7fd491f02700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd47805f270 0x7fd478061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fd47c005750 tx=0x7fd47c009040).stop 2024-01-24T15:30:41.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.523+0000 7fd491f02700 1 -- 172.21.15.29:0/123963599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd48c150d40 msgr2=0x7fd48c062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.523+0000 7fd491f02700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd48c150d40 0x7fd48c062ca0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7fd48000c420 tx=0x7fd480007c60).stop 2024-01-24T15:30:41.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.523+0000 7fd491f02700 1 -- 172.21.15.29:0/123963599 shutdown_connections 2024-01-24T15:30:41.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.523+0000 7fd491f02700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd48c155040 0x7fd48c152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.523+0000 7fd491f02700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd47805f270 0x7fd478061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.523+0000 7fd491f02700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd48c069240 0x7fd48c0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.523+0000 7fd491f02700 1 --2- 172.21.15.29:0/123963599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd48c150d40 0x7fd48c062ca0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.523+0000 7fd491f02700 1 -- 172.21.15.29:0/123963599 >> 172.21.15.29:0/123963599 conn(0x7fd48c0c8580 msgr2=0x7fd48c14b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:41.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.523+0000 7fd491f02700 1 -- 172.21.15.29:0/123963599 shutdown_connections 2024-01-24T15:30:41.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.523+0000 7fd491f02700 1 -- 172.21.15.29:0/123963599 wait complete. 2024-01-24T15:30:41.530 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:30:41.539 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-01-24T15:30:39.642660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2463140,"kb_used_data":20836,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752511580,"statfs":{"total":773094113280,"available":770571857920,"internally_reserved":0,"allocated":21336064,"data_stored":12321905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-602802,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":-209,"num_write_kb":-728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-34,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.016880"},"pg_stats":[{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446734+0000","last_change":"2024-01-24T15:30:37.444275+0000","last_active":"2024-01-24T15:30:37.446734+0000","last_peered":"2024-01-24T15:30:37.446734+0000","last_clean":"2024-01-24T15:30:37.446734+0000","last_became_active":"2024-01-24T15:30:37.444036+0000","last_became_peered":"2024-01-24T15:30:37.444036+0000","last_unstale":"2024-01-24T15:30:37.446734+0000","last_undegraded":"2024-01-24T15:30:37.446734+0000","last_fullsized":"2024-01-24T15:30:37.446734+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3f","version":"212'3","reported_seq":13,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444091+0000","last_change":"2024-01-24T15:30:37.010273+0000","last_active":"2024-01-24T15:30:37.444091+0000","last_peered":"2024-01-24T15:30:37.444091+0000","last_clean":"2024-01-24T15:30:37.444091+0000","last_became_active":"2024-01-24T15:30:37.009556+0000","last_became_peered":"2024-01-24T15:30:37.009556+0000","last_unstale":"2024-01-24T15:30:37.444091+0000","last_undegraded":"2024-01-24T15:30:37.444091+0000","last_fullsized":"2024-01-24T15:30:37.444091+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439203+0000","last_change":"2024-01-24T15:30:37.437053+0000","last_active":"2024-01-24T15:30:37.439203+0000","last_peered":"2024-01-24T15:30:37.439203+0000","last_clean":"2024-01-24T15:30:37.439203+0000","last_became_active":"2024-01-24T15:30:37.436635+0000","last_became_peered":"2024-01-24T15:30:37.436635+0000","last_unstale":"2024-01-24T15:30:37.439203+0000","last_undegraded":"2024-01-24T15:30:37.439203+0000","last_fullsized":"2024-01-24T15:30:37.439203+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.427440+0000","last_change":"2024-01-24T15:30:37.006735+0000","last_active":"2024-01-24T15:30:37.427440+0000","last_peered":"2024-01-24T15:30:37.427440+0000","last_clean":"2024-01-24T15:30:37.427440+0000","last_became_active":"2024-01-24T15:30:37.006397+0000","last_became_peered":"2024-01-24T15:30:37.006397+0000","last_unstale":"2024-01-24T15:30:37.427440+0000","last_undegraded":"2024-01-24T15:30:37.427440+0000","last_fullsized":"2024-01-24T15:30:37.427440+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446750+0000","last_change":"2024-01-24T15:30:37.444312+0000","last_active":"2024-01-24T15:30:37.446750+0000","last_peered":"2024-01-24T15:30:37.446750+0000","last_clean":"2024-01-24T15:30:37.446750+0000","last_became_active":"2024-01-24T15:30:37.444022+0000","last_became_peered":"2024-01-24T15:30:37.444022+0000","last_unstale":"2024-01-24T15:30:37.446750+0000","last_undegraded":"2024-01-24T15:30:37.446750+0000","last_fullsized":"2024-01-24T15:30:37.446750+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420562+0000","last_change":"2024-01-24T15:30:37.016546+0000","last_active":"2024-01-24T15:30:37.420562+0000","last_peered":"2024-01-24T15:30:37.420562+0000","last_clean":"2024-01-24T15:30:37.420562+0000","last_became_active":"2024-01-24T15:30:37.016269+0000","last_became_peered":"2024-01-24T15:30:37.016269+0000","last_unstale":"2024-01-24T15:30:37.420562+0000","last_undegraded":"2024-01-24T15:30:37.420562+0000","last_fullsized":"2024-01-24T15:30:37.420562+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.436232+0000","last_change":"2024-01-24T15:30:37.431127+0000","last_active":"2024-01-24T15:30:37.436232+0000","last_peered":"2024-01-24T15:30:37.436232+0000","last_clean":"2024-01-24T15:30:37.436232+0000","last_became_active":"2024-01-24T15:30:37.430816+0000","last_became_peered":"2024-01-24T15:30:37.430816+0000","last_unstale":"2024-01-24T15:30:37.436232+0000","last_undegraded":"2024-01-24T15:30:37.436232+0000","last_fullsized":"2024-01-24T15:30:37.436232+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3c","version":"212'2","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448521+0000","last_change":"2024-01-24T15:30:37.011274+0000","last_active":"2024-01-24T15:30:37.448521+0000","last_peered":"2024-01-24T15:30:37.448521+0000","last_clean":"2024-01-24T15:30:37.448521+0000","last_became_active":"2024-01-24T15:30:37.010993+0000","last_became_peered":"2024-01-24T15:30:37.010993+0000","last_unstale":"2024-01-24T15:30:37.448521+0000","last_undegraded":"2024-01-24T15:30:37.448521+0000","last_fullsized":"2024-01-24T15:30:37.448521+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.437961+0000","last_change":"2024-01-24T15:30:37.432762+0000","last_active":"2024-01-24T15:30:37.437961+0000","last_peered":"2024-01-24T15:30:37.437961+0000","last_clean":"2024-01-24T15:30:37.437961+0000","last_became_active":"2024-01-24T15:30:37.432457+0000","last_became_peered":"2024-01-24T15:30:37.432457+0000","last_unstale":"2024-01-24T15:30:37.437961+0000","last_undegraded":"2024-01-24T15:30:37.437961+0000","last_fullsized":"2024-01-24T15:30:37.437961+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3b","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.441932+0000","last_change":"2024-01-24T15:30:37.013704+0000","last_active":"2024-01-24T15:30:37.441932+0000","last_peered":"2024-01-24T15:30:37.441932+0000","last_clean":"2024-01-24T15:30:37.441932+0000","last_became_active":"2024-01-24T15:30:37.013363+0000","last_became_peered":"2024-01-24T15:30:37.013363+0000","last_unstale":"2024-01-24T15:30:37.441932+0000","last_undegraded":"2024-01-24T15:30:37.441932+0000","last_fullsized":"2024-01-24T15:30:37.441932+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.433763+0000","last_change":"2024-01-24T15:30:37.422209+0000","last_active":"2024-01-24T15:30:37.433763+0000","last_peered":"2024-01-24T15:30:37.433763+0000","last_clean":"2024-01-24T15:30:37.433763+0000","last_became_active":"2024-01-24T15:30:37.421873+0000","last_became_peered":"2024-01-24T15:30:37.421873+0000","last_unstale":"2024-01-24T15:30:37.433763+0000","last_undegraded":"2024-01-24T15:30:37.433763+0000","last_fullsized":"2024-01-24T15:30:37.433763+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3a","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444208+0000","last_change":"2024-01-24T15:30:37.009252+0000","last_active":"2024-01-24T15:30:37.444208+0000","last_peered":"2024-01-24T15:30:37.444208+0000","last_clean":"2024-01-24T15:30:37.444208+0000","last_became_active":"2024-01-24T15:30:37.008966+0000","last_became_peered":"2024-01-24T15:30:37.008966+0000","last_unstale":"2024-01-24T15:30:37.444208+0000","last_undegraded":"2024-01-24T15:30:37.444208+0000","last_fullsized":"2024-01-24T15:30:37.444208+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446295+0000","last_change":"2024-01-24T15:30:37.426413+0000","last_active":"2024-01-24T15:30:37.446295+0000","last_peered":"2024-01-24T15:30:37.446295+0000","last_clean":"2024-01-24T15:30:37.446295+0000","last_became_active":"2024-01-24T15:30:37.426107+0000","last_became_peered":"2024-01-24T15:30:37.426107+0000","last_unstale":"2024-01-24T15:30:37.446295+0000","last_undegraded":"2024-01-24T15:30:37.446295+0000","last_fullsized":"2024-01-24T15:30:37.446295+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425127+0000","last_change":"2024-01-24T15:30:37.020917+0000","last_active":"2024-01-24T15:30:37.425127+0000","last_peered":"2024-01-24T15:30:37.425127+0000","last_clean":"2024-01-24T15:30:37.425127+0000","last_became_active":"2024-01-24T15:30:37.020610+0000","last_became_peered":"2024-01-24T15:30:37.020610+0000","last_unstale":"2024-01-24T15:30:37.425127+0000","last_undegraded":"2024-01-24T15:30:37.425127+0000","last_fullsized":"2024-01-24T15:30:37.425127+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.443942+0000","last_change":"2024-01-24T15:30:37.439074+0000","last_active":"2024-01-24T15:30:37.443942+0000","last_peered":"2024-01-24T15:30:37.443942+0000","last_clean":"2024-01-24T15:30:37.443942+0000","last_became_active":"2024-01-24T15:30:37.438736+0000","last_became_peered":"2024-01-24T15:30:37.438736+0000","last_unstale":"2024-01-24T15:30:37.443942+0000","last_undegraded":"2024-01-24T15:30:37.443942+0000","last_fullsized":"2024-01-24T15:30:37.443942+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425289+0000","last_change":"2024-01-24T15:30:37.005419+0000","last_active":"2024-01-24T15:30:37.425289+0000","last_peered":"2024-01-24T15:30:37.425289+0000","last_clean":"2024-01-24T15:30:37.425289+0000","last_became_active":"2024-01-24T15:30:37.004989+0000","last_became_peered":"2024-01-24T15:30:37.004989+0000","last_unstale":"2024-01-24T15:30:37.425289+0000","last_undegraded":"2024-01-24T15:30:37.425289+0000","last_fullsized":"2024-01-24T15:30:37.425289+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439558+0000","last_change":"2024-01-24T15:30:37.435202+0000","last_active":"2024-01-24T15:30:37.439558+0000","last_peered":"2024-01-24T15:30:37.439558+0000","last_clean":"2024-01-24T15:30:37.439558+0000","last_became_active":"2024-01-24T15:30:37.434876+0000","last_became_peered":"2024-01-24T15:30:37.434876+0000","last_unstale":"2024-01-24T15:30:37.439558+0000","last_undegraded":"2024-01-24T15:30:37.439558+0000","last_fullsized":"2024-01-24T15:30:37.439558+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423953+0000","last_change":"2024-01-24T15:30:37.010943+0000","last_active":"2024-01-24T15:30:37.423953+0000","last_peered":"2024-01-24T15:30:37.423953+0000","last_clean":"2024-01-24T15:30:37.423953+0000","last_became_active":"2024-01-24T15:30:37.010232+0000","last_became_peered":"2024-01-24T15:30:37.010232+0000","last_unstale":"2024-01-24T15:30:37.423953+0000","last_undegraded":"2024-01-24T15:30:37.423953+0000","last_fullsized":"2024-01-24T15:30:37.423953+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439290+0000","last_change":"2024-01-24T15:30:37.429110+0000","last_active":"2024-01-24T15:30:37.439290+0000","last_peered":"2024-01-24T15:30:37.439290+0000","last_clean":"2024-01-24T15:30:37.439290+0000","last_became_active":"2024-01-24T15:30:37.428605+0000","last_became_peered":"2024-01-24T15:30:37.428605+0000","last_unstale":"2024-01-24T15:30:37.439290+0000","last_undegraded":"2024-01-24T15:30:37.439290+0000","last_fullsized":"2024-01-24T15:30:37.439290+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423834+0000","last_change":"2024-01-24T15:30:37.013549+0000","last_active":"2024-01-24T15:30:37.423834+0000","last_peered":"2024-01-24T15:30:37.423834+0000","last_clean":"2024-01-24T15:30:37.423834+0000","last_became_active":"2024-01-24T15:30:37.013199+0000","last_became_peered":"2024-01-24T15:30:37.013199+0000","last_unstale":"2024-01-24T15:30:37.423834+0000","last_undegraded":"2024-01-24T15:30:37.423834+0000","last_fullsized":"2024-01-24T15:30:37.423834+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.438145+0000","last_change":"2024-01-24T15:30:37.431975+0000","last_active":"2024-01-24T15:30:37.438145+0000","last_peered":"2024-01-24T15:30:37.438145+0000","last_clean":"2024-01-24T15:30:37.438145+0000","last_became_active":"2024-01-24T15:30:37.431687+0000","last_became_peered":"2024-01-24T15:30:37.431687+0000","last_unstale":"2024-01-24T15:30:37.438145+0000","last_undegraded":"2024-01-24T15:30:37.438145+0000","last_fullsized":"2024-01-24T15:30:37.438145+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420586+0000","last_change":"2024-01-24T15:30:37.007787+0000","last_active":"2024-01-24T15:30:37.420586+0000","last_peered":"2024-01-24T15:30:37.420586+0000","last_clean":"2024-01-24T15:30:37.420586+0000","last_became_active":"2024-01-24T15:30:37.007488+0000","last_became_peered":"2024-01-24T15:30:37.007488+0000","last_unstale":"2024-01-24T15:30:37.420586+0000","last_undegraded":"2024-01-24T15:30:37.420586+0000","last_fullsized":"2024-01-24T15:30:37.420586+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.443785+0000","last_change":"2024-01-24T15:30:37.438455+0000","last_active":"2024-01-24T15:30:37.443785+0000","last_peered":"2024-01-24T15:30:37.443785+0000","last_clean":"2024-01-24T15:30:37.443785+0000","last_became_active":"2024-01-24T15:30:37.432327+0000","last_became_peered":"2024-01-24T15:30:37.432327+0000","last_unstale":"2024-01-24T15:30:37.443785+0000","last_undegraded":"2024-01-24T15:30:37.443785+0000","last_fullsized":"2024-01-24T15:30:37.443785+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.34","version":"212'4","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.449549+0000","last_change":"2024-01-24T15:30:37.008132+0000","last_active":"2024-01-24T15:30:37.449549+0000","last_peered":"2024-01-24T15:30:37.449549+0000","last_clean":"2024-01-24T15:30:37.449549+0000","last_became_active":"2024-01-24T15:30:37.007824+0000","last_became_peered":"2024-01-24T15:30:37.007824+0000","last_unstale":"2024-01-24T15:30:37.449549+0000","last_undegraded":"2024-01-24T15:30:37.449549+0000","last_fullsized":"2024-01-24T15:30:37.449549+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.440194+0000","last_change":"2024-01-24T15:30:37.437420+0000","last_active":"2024-01-24T15:30:37.440194+0000","last_peered":"2024-01-24T15:30:37.440194+0000","last_clean":"2024-01-24T15:30:37.440194+0000","last_became_active":"2024-01-24T15:30:37.437129+0000","last_became_peered":"2024-01-24T15:30:37.437129+0000","last_unstale":"2024-01-24T15:30:37.440194+0000","last_undegraded":"2024-01-24T15:30:37.440194+0000","last_fullsized":"2024-01-24T15:30:37.440194+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.33","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448433+0000","last_change":"2024-01-24T15:30:37.010090+0000","last_active":"2024-01-24T15:30:37.448433+0000","last_peered":"2024-01-24T15:30:37.448433+0000","last_clean":"2024-01-24T15:30:37.448433+0000","last_became_active":"2024-01-24T15:30:37.009819+0000","last_became_peered":"2024-01-24T15:30:37.009819+0000","last_unstale":"2024-01-24T15:30:37.448433+0000","last_undegraded":"2024-01-24T15:30:37.448433+0000","last_fullsized":"2024-01-24T15:30:37.448433+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446313+0000","last_change":"2024-01-24T15:30:37.442227+0000","last_active":"2024-01-24T15:30:37.446313+0000","last_peered":"2024-01-24T15:30:37.446313+0000","last_clean":"2024-01-24T15:30:37.446313+0000","last_became_active":"2024-01-24T15:30:37.441942+0000","last_became_peered":"2024-01-24T15:30:37.441942+0000","last_unstale":"2024-01-24T15:30:37.446313+0000","last_undegraded":"2024-01-24T15:30:37.446313+0000","last_fullsized":"2024-01-24T15:30:37.446313+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.426637+0000","last_change":"2024-01-24T15:30:37.023501+0000","last_active":"2024-01-24T15:30:37.426637+0000","last_peered":"2024-01-24T15:30:37.426637+0000","last_clean":"2024-01-24T15:30:37.426637+0000","last_became_active":"2024-01-24T15:30:37.023194+0000","last_became_peered":"2024-01-24T15:30:37.023194+0000","last_unstale":"2024-01-24T15:30:37.426637+0000","last_undegraded":"2024-01-24T15:30:37.426637+0000","last_fullsized":"2024-01-24T15:30:37.426637+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446326+0000","last_change":"2024-01-24T15:30:37.427357+0000","last_active":"2024-01-24T15:30:37.446326+0000","last_peered":"2024-01-24T15:30:37.446326+0000","last_clean":"2024-01-24T15:30:37.446326+0000","last_became_active":"2024-01-24T15:30:37.426441+0000","last_became_peered":"2024-01-24T15:30:37.426441+0000","last_unstale":"2024-01-24T15:30:37.446326+0000","last_undegraded":"2024-01-24T15:30:37.446326+0000","last_fullsized":"2024-01-24T15:30:37.446326+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419347+0000","last_change":"2024-01-24T15:30:37.013635+0000","last_active":"2024-01-24T15:30:37.419347+0000","last_peered":"2024-01-24T15:30:37.419347+0000","last_clean":"2024-01-24T15:30:37.419347+0000","last_became_active":"2024-01-24T15:30:37.013207+0000","last_became_peered":"2024-01-24T15:30:37.013207+0000","last_unstale":"2024-01-24T15:30:37.419347+0000","last_undegraded":"2024-01-24T15:30:37.419347+0000","last_fullsized":"2024-01-24T15:30:37.419347+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439504+0000","last_change":"2024-01-24T15:30:37.436058+0000","last_active":"2024-01-24T15:30:37.439504+0000","last_peered":"2024-01-24T15:30:37.439504+0000","last_clean":"2024-01-24T15:30:37.439504+0000","last_became_active":"2024-01-24T15:30:37.435693+0000","last_became_peered":"2024-01-24T15:30:37.435693+0000","last_unstale":"2024-01-24T15:30:37.439504+0000","last_undegraded":"2024-01-24T15:30:37.439504+0000","last_fullsized":"2024-01-24T15:30:37.439504+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.30","version":"212'4","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.449754+0000","last_change":"2024-01-24T15:30:37.012073+0000","last_active":"2024-01-24T15:30:37.449754+0000","last_peered":"2024-01-24T15:30:37.449754+0000","last_clean":"2024-01-24T15:30:37.449754+0000","last_became_active":"2024-01-24T15:30:37.011589+0000","last_became_peered":"2024-01-24T15:30:37.011589+0000","last_unstale":"2024-01-24T15:30:37.449754+0000","last_undegraded":"2024-01-24T15:30:37.449754+0000","last_fullsized":"2024-01-24T15:30:37.449754+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.443656+0000","last_change":"2024-01-24T15:30:37.440683+0000","last_active":"2024-01-24T15:30:37.443656+0000","last_peered":"2024-01-24T15:30:37.443656+0000","last_clean":"2024-01-24T15:30:37.443656+0000","last_became_active":"2024-01-24T15:30:37.440366+0000","last_became_peered":"2024-01-24T15:30:37.440366+0000","last_unstale":"2024-01-24T15:30:37.443656+0000","last_undegraded":"2024-01-24T15:30:37.443656+0000","last_fullsized":"2024-01-24T15:30:37.443656+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.424060+0000","last_change":"2024-01-24T15:30:37.011161+0000","last_active":"2024-01-24T15:30:37.424060+0000","last_peered":"2024-01-24T15:30:37.424060+0000","last_clean":"2024-01-24T15:30:37.424060+0000","last_became_active":"2024-01-24T15:30:37.010595+0000","last_became_peered":"2024-01-24T15:30:37.010595+0000","last_unstale":"2024-01-24T15:30:37.424060+0000","last_undegraded":"2024-01-24T15:30:37.424060+0000","last_fullsized":"2024-01-24T15:30:37.424060+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444316+0000","last_change":"2024-01-24T15:30:37.434050+0000","last_active":"2024-01-24T15:30:37.444316+0000","last_peered":"2024-01-24T15:30:37.444316+0000","last_clean":"2024-01-24T15:30:37.444316+0000","last_became_active":"2024-01-24T15:30:37.433763+0000","last_became_peered":"2024-01-24T15:30:37.433763+0000","last_unstale":"2024-01-24T15:30:37.444316+0000","last_undegraded":"2024-01-24T15:30:37.444316+0000","last_fullsized":"2024-01-24T15:30:37.444316+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423220+0000","last_change":"2024-01-24T15:30:37.014740+0000","last_active":"2024-01-24T15:30:37.423220+0000","last_peered":"2024-01-24T15:30:37.423220+0000","last_clean":"2024-01-24T15:30:37.423220+0000","last_became_active":"2024-01-24T15:30:37.014450+0000","last_became_peered":"2024-01-24T15:30:37.014450+0000","last_unstale":"2024-01-24T15:30:37.423220+0000","last_undegraded":"2024-01-24T15:30:37.423220+0000","last_fullsized":"2024-01-24T15:30:37.423220+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444322+0000","last_change":"2024-01-24T15:30:37.433732+0000","last_active":"2024-01-24T15:30:37.444322+0000","last_peered":"2024-01-24T15:30:37.444322+0000","last_clean":"2024-01-24T15:30:37.444322+0000","last_became_active":"2024-01-24T15:30:37.433457+0000","last_became_peered":"2024-01-24T15:30:37.433457+0000","last_unstale":"2024-01-24T15:30:37.444322+0000","last_undegraded":"2024-01-24T15:30:37.444322+0000","last_fullsized":"2024-01-24T15:30:37.444322+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2d","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.440198+0000","last_change":"2024-01-24T15:30:37.011775+0000","last_active":"2024-01-24T15:30:37.440198+0000","last_peered":"2024-01-24T15:30:37.440198+0000","last_clean":"2024-01-24T15:30:37.440198+0000","last_became_active":"2024-01-24T15:30:37.011456+0000","last_became_peered":"2024-01-24T15:30:37.011456+0000","last_unstale":"2024-01-24T15:30:37.440198+0000","last_undegraded":"2024-01-24T15:30:37.440198+0000","last_fullsized":"2024-01-24T15:30:37.440198+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.441166+0000","last_change":"2024-01-24T15:30:37.438868+0000","last_active":"2024-01-24T15:30:37.441166+0000","last_peered":"2024-01-24T15:30:37.441166+0000","last_clean":"2024-01-24T15:30:37.441166+0000","last_became_active":"2024-01-24T15:30:37.438588+0000","last_became_peered":"2024-01-24T15:30:37.438588+0000","last_unstale":"2024-01-24T15:30:37.441166+0000","last_undegraded":"2024-01-24T15:30:37.441166+0000","last_fullsized":"2024-01-24T15:30:37.441166+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.416885+0000","last_change":"2024-01-24T15:30:37.006475+0000","last_active":"2024-01-24T15:30:37.416885+0000","last_peered":"2024-01-24T15:30:37.416885+0000","last_clean":"2024-01-24T15:30:37.416885+0000","last_became_active":"2024-01-24T15:30:37.006131+0000","last_became_peered":"2024-01-24T15:30:37.006131+0000","last_unstale":"2024-01-24T15:30:37.416885+0000","last_undegraded":"2024-01-24T15:30:37.416885+0000","last_fullsized":"2024-01-24T15:30:37.416885+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448917+0000","last_change":"2024-01-24T15:30:37.447503+0000","last_active":"2024-01-24T15:30:37.448917+0000","last_peered":"2024-01-24T15:30:37.448917+0000","last_clean":"2024-01-24T15:30:37.448917+0000","last_became_active":"2024-01-24T15:30:37.447214+0000","last_became_peered":"2024-01-24T15:30:37.447214+0000","last_unstale":"2024-01-24T15:30:37.448917+0000","last_undegraded":"2024-01-24T15:30:37.448917+0000","last_fullsized":"2024-01-24T15:30:37.448917+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.2b","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448502+0000","last_change":"2024-01-24T15:30:37.007302+0000","last_active":"2024-01-24T15:30:37.448502+0000","last_peered":"2024-01-24T15:30:37.448502+0000","last_clean":"2024-01-24T15:30:37.448502+0000","last_became_active":"2024-01-24T15:30:37.007009+0000","last_became_peered":"2024-01-24T15:30:37.007009+0000","last_unstale":"2024-01-24T15:30:37.448502+0000","last_undegraded":"2024-01-24T15:30:37.448502+0000","last_fullsized":"2024-01-24T15:30:37.448502+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444349+0000","last_change":"2024-01-24T15:30:37.433667+0000","last_active":"2024-01-24T15:30:37.444349+0000","last_peered":"2024-01-24T15:30:37.444349+0000","last_clean":"2024-01-24T15:30:37.444349+0000","last_became_active":"2024-01-24T15:30:37.433352+0000","last_became_peered":"2024-01-24T15:30:37.433352+0000","last_unstale":"2024-01-24T15:30:37.444349+0000","last_undegraded":"2024-01-24T15:30:37.444349+0000","last_fullsized":"2024-01-24T15:30:37.444349+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.424357+0000","last_change":"2024-01-24T15:30:37.013357+0000","last_active":"2024-01-24T15:30:37.424357+0000","last_peered":"2024-01-24T15:30:37.424357+0000","last_clean":"2024-01-24T15:30:37.424357+0000","last_became_active":"2024-01-24T15:30:37.013084+0000","last_became_peered":"2024-01-24T15:30:37.013084+0000","last_unstale":"2024-01-24T15:30:37.424357+0000","last_undegraded":"2024-01-24T15:30:37.424357+0000","last_fullsized":"2024-01-24T15:30:37.424357+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448720+0000","last_change":"2024-01-24T15:30:37.431843+0000","last_active":"2024-01-24T15:30:37.448720+0000","last_peered":"2024-01-24T15:30:37.448720+0000","last_clean":"2024-01-24T15:30:37.448720+0000","last_became_active":"2024-01-24T15:30:37.431536+0000","last_became_peered":"2024-01-24T15:30:37.431536+0000","last_unstale":"2024-01-24T15:30:37.448720+0000","last_undegraded":"2024-01-24T15:30:37.448720+0000","last_fullsized":"2024-01-24T15:30:37.448720+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425112+0000","last_change":"2024-01-24T15:30:37.014302+0000","last_active":"2024-01-24T15:30:37.425112+0000","last_peered":"2024-01-24T15:30:37.425112+0000","last_clean":"2024-01-24T15:30:37.425112+0000","last_became_active":"2024-01-24T15:30:37.013985+0000","last_became_peered":"2024-01-24T15:30:37.013985+0000","last_unstale":"2024-01-24T15:30:37.425112+0000","last_undegraded":"2024-01-24T15:30:37.425112+0000","last_fullsized":"2024-01-24T15:30:37.425112+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.437434+0000","last_change":"2024-01-24T15:30:37.436637+0000","last_active":"2024-01-24T15:30:37.437434+0000","last_peered":"2024-01-24T15:30:37.437434+0000","last_clean":"2024-01-24T15:30:37.437434+0000","last_became_active":"2024-01-24T15:30:37.436352+0000","last_became_peered":"2024-01-24T15:30:37.436352+0000","last_unstale":"2024-01-24T15:30:37.437434+0000","last_undegraded":"2024-01-24T15:30:37.437434+0000","last_fullsized":"2024-01-24T15:30:37.437434+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.417111+0000","last_change":"2024-01-24T15:30:37.011770+0000","last_active":"2024-01-24T15:30:37.417111+0000","last_peered":"2024-01-24T15:30:37.417111+0000","last_clean":"2024-01-24T15:30:37.417111+0000","last_became_active":"2024-01-24T15:30:37.011249+0000","last_became_peered":"2024-01-24T15:30:37.011249+0000","last_unstale":"2024-01-24T15:30:37.417111+0000","last_undegraded":"2024-01-24T15:30:37.417111+0000","last_fullsized":"2024-01-24T15:30:37.417111+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.443836+0000","last_change":"2024-01-24T15:30:37.439059+0000","last_active":"2024-01-24T15:30:37.443836+0000","last_peered":"2024-01-24T15:30:37.443836+0000","last_clean":"2024-01-24T15:30:37.443836+0000","last_became_active":"2024-01-24T15:30:37.438774+0000","last_became_peered":"2024-01-24T15:30:37.438774+0000","last_unstale":"2024-01-24T15:30:37.443836+0000","last_undegraded":"2024-01-24T15:30:37.443836+0000","last_fullsized":"2024-01-24T15:30:37.443836+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420163+0000","last_change":"2024-01-24T15:30:37.008070+0000","last_active":"2024-01-24T15:30:37.420163+0000","last_peered":"2024-01-24T15:30:37.420163+0000","last_clean":"2024-01-24T15:30:37.420163+0000","last_became_active":"2024-01-24T15:30:37.007541+0000","last_became_peered":"2024-01-24T15:30:37.007541+0000","last_unstale":"2024-01-24T15:30:37.420163+0000","last_undegraded":"2024-01-24T15:30:37.420163+0000","last_fullsized":"2024-01-24T15:30:37.420163+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.441336+0000","last_change":"2024-01-24T15:30:37.437781+0000","last_active":"2024-01-24T15:30:37.441336+0000","last_peered":"2024-01-24T15:30:37.441336+0000","last_clean":"2024-01-24T15:30:37.441336+0000","last_became_active":"2024-01-24T15:30:37.437475+0000","last_became_peered":"2024-01-24T15:30:37.437475+0000","last_unstale":"2024-01-24T15:30:37.441336+0000","last_undegraded":"2024-01-24T15:30:37.441336+0000","last_fullsized":"2024-01-24T15:30:37.441336+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.417035+0000","last_change":"2024-01-24T15:30:37.002395+0000","last_active":"2024-01-24T15:30:37.417035+0000","last_peered":"2024-01-24T15:30:37.417035+0000","last_clean":"2024-01-24T15:30:37.417035+0000","last_became_active":"2024-01-24T15:30:37.002056+0000","last_became_peered":"2024-01-24T15:30:37.002056+0000","last_unstale":"2024-01-24T15:30:37.417035+0000","last_undegraded":"2024-01-24T15:30:37.417035+0000","last_fullsized":"2024-01-24T15:30:37.417035+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444355+0000","last_change":"2024-01-24T15:30:37.435232+0000","last_active":"2024-01-24T15:30:37.444355+0000","last_peered":"2024-01-24T15:30:37.444355+0000","last_clean":"2024-01-24T15:30:37.444355+0000","last_became_active":"2024-01-24T15:30:37.434904+0000","last_became_peered":"2024-01-24T15:30:37.434904+0000","last_unstale":"2024-01-24T15:30:37.444355+0000","last_undegraded":"2024-01-24T15:30:37.444355+0000","last_fullsized":"2024-01-24T15:30:37.444355+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.426737+0000","last_change":"2024-01-24T15:30:37.017404+0000","last_active":"2024-01-24T15:30:37.426737+0000","last_peered":"2024-01-24T15:30:37.426737+0000","last_clean":"2024-01-24T15:30:37.426737+0000","last_became_active":"2024-01-24T15:30:37.017086+0000","last_became_peered":"2024-01-24T15:30:37.017086+0000","last_unstale":"2024-01-24T15:30:37.426737+0000","last_undegraded":"2024-01-24T15:30:37.426737+0000","last_fullsized":"2024-01-24T15:30:37.426737+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.436983+0000","last_change":"2024-01-24T15:30:37.433013+0000","last_active":"2024-01-24T15:30:37.436983+0000","last_peered":"2024-01-24T15:30:37.436983+0000","last_clean":"2024-01-24T15:30:37.436983+0000","last_became_active":"2024-01-24T15:30:37.432700+0000","last_became_peered":"2024-01-24T15:30:37.432700+0000","last_unstale":"2024-01-24T15:30:37.436983+0000","last_undegraded":"2024-01-24T15:30:37.436983+0000","last_fullsized":"2024-01-24T15:30:37.436983+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419797+0000","last_change":"2024-01-24T15:30:37.011691+0000","last_active":"2024-01-24T15:30:37.419797+0000","last_peered":"2024-01-24T15:30:37.419797+0000","last_clean":"2024-01-24T15:30:37.419797+0000","last_became_active":"2024-01-24T15:30:37.011242+0000","last_became_peered":"2024-01-24T15:30:37.011242+0000","last_unstale":"2024-01-24T15:30:37.419797+0000","last_undegraded":"2024-01-24T15:30:37.419797+0000","last_fullsized":"2024-01-24T15:30:37.419797+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439731+0000","last_change":"2024-01-24T15:30:37.431586+0000","last_active":"2024-01-24T15:30:37.439731+0000","last_peered":"2024-01-24T15:30:37.439731+0000","last_clean":"2024-01-24T15:30:37.439731+0000","last_became_active":"2024-01-24T15:30:37.431292+0000","last_became_peered":"2024-01-24T15:30:37.431292+0000","last_unstale":"2024-01-24T15:30:37.439731+0000","last_undegraded":"2024-01-24T15:30:37.439731+0000","last_fullsized":"2024-01-24T15:30:37.439731+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.427681+0000","last_change":"2024-01-24T15:30:37.015085+0000","last_active":"2024-01-24T15:30:37.427681+0000","last_peered":"2024-01-24T15:30:37.427681+0000","last_clean":"2024-01-24T15:30:37.427681+0000","last_became_active":"2024-01-24T15:30:37.013725+0000","last_became_peered":"2024-01-24T15:30:37.013725+0000","last_unstale":"2024-01-24T15:30:37.427681+0000","last_undegraded":"2024-01-24T15:30:37.427681+0000","last_fullsized":"2024-01-24T15:30:37.427681+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439773+0000","last_change":"2024-01-24T15:30:37.436446+0000","last_active":"2024-01-24T15:30:37.439773+0000","last_peered":"2024-01-24T15:30:37.439773+0000","last_clean":"2024-01-24T15:30:37.439773+0000","last_became_active":"2024-01-24T15:30:37.435849+0000","last_became_peered":"2024-01-24T15:30:37.435849+0000","last_unstale":"2024-01-24T15:30:37.439773+0000","last_undegraded":"2024-01-24T15:30:37.439773+0000","last_fullsized":"2024-01-24T15:30:37.439773+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.422248+0000","last_change":"2024-01-24T15:30:37.017515+0000","last_active":"2024-01-24T15:30:37.422248+0000","last_peered":"2024-01-24T15:30:37.422248+0000","last_clean":"2024-01-24T15:30:37.422248+0000","last_became_active":"2024-01-24T15:30:37.017340+0000","last_became_peered":"2024-01-24T15:30:37.017340+0000","last_unstale":"2024-01-24T15:30:37.422248+0000","last_undegraded":"2024-01-24T15:30:37.422248+0000","last_fullsized":"2024-01-24T15:30:37.422248+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.434262+0000","last_change":"2024-01-24T15:30:37.425592+0000","last_active":"2024-01-24T15:30:37.434262+0000","last_peered":"2024-01-24T15:30:37.434262+0000","last_clean":"2024-01-24T15:30:37.434262+0000","last_became_active":"2024-01-24T15:30:37.425302+0000","last_became_peered":"2024-01-24T15:30:37.425302+0000","last_unstale":"2024-01-24T15:30:37.434262+0000","last_undegraded":"2024-01-24T15:30:37.434262+0000","last_fullsized":"2024-01-24T15:30:37.434262+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419477+0000","last_change":"2024-01-24T15:30:37.013451+0000","last_active":"2024-01-24T15:30:37.419477+0000","last_peered":"2024-01-24T15:30:37.419477+0000","last_clean":"2024-01-24T15:30:37.419477+0000","last_became_active":"2024-01-24T15:30:37.012903+0000","last_became_peered":"2024-01-24T15:30:37.012903+0000","last_unstale":"2024-01-24T15:30:37.419477+0000","last_undegraded":"2024-01-24T15:30:37.419477+0000","last_fullsized":"2024-01-24T15:30:37.419477+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.435385+0000","last_change":"2024-01-24T15:30:37.430607+0000","last_active":"2024-01-24T15:30:37.435385+0000","last_peered":"2024-01-24T15:30:37.435385+0000","last_clean":"2024-01-24T15:30:37.435385+0000","last_became_active":"2024-01-24T15:30:37.430270+0000","last_became_peered":"2024-01-24T15:30:37.430270+0000","last_unstale":"2024-01-24T15:30:37.435385+0000","last_undegraded":"2024-01-24T15:30:37.435385+0000","last_fullsized":"2024-01-24T15:30:37.435385+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.422030+0000","last_change":"2024-01-24T15:30:37.004925+0000","last_active":"2024-01-24T15:30:37.422030+0000","last_peered":"2024-01-24T15:30:37.422030+0000","last_clean":"2024-01-24T15:30:37.422030+0000","last_became_active":"2024-01-24T15:30:37.004638+0000","last_became_peered":"2024-01-24T15:30:37.004638+0000","last_unstale":"2024-01-24T15:30:37.422030+0000","last_undegraded":"2024-01-24T15:30:37.422030+0000","last_fullsized":"2024-01-24T15:30:37.422030+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.452067+0000","last_change":"2024-01-24T15:30:37.447427+0000","last_active":"2024-01-24T15:30:37.452067+0000","last_peered":"2024-01-24T15:30:37.452067+0000","last_clean":"2024-01-24T15:30:37.452067+0000","last_became_active":"2024-01-24T15:30:37.447142+0000","last_became_peered":"2024-01-24T15:30:37.447142+0000","last_unstale":"2024-01-24T15:30:37.452067+0000","last_undegraded":"2024-01-24T15:30:37.452067+0000","last_fullsized":"2024-01-24T15:30:37.452067+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420378+0000","last_change":"2024-01-24T15:30:37.005241+0000","last_active":"2024-01-24T15:30:37.420378+0000","last_peered":"2024-01-24T15:30:37.420378+0000","last_clean":"2024-01-24T15:30:37.420378+0000","last_became_active":"2024-01-24T15:30:37.004783+0000","last_became_peered":"2024-01-24T15:30:37.004783+0000","last_unstale":"2024-01-24T15:30:37.420378+0000","last_undegraded":"2024-01-24T15:30:37.420378+0000","last_fullsized":"2024-01-24T15:30:37.420378+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.449454+0000","last_change":"2024-01-24T15:30:37.441027+0000","last_active":"2024-01-24T15:30:37.449454+0000","last_peered":"2024-01-24T15:30:37.449454+0000","last_clean":"2024-01-24T15:30:37.449454+0000","last_became_active":"2024-01-24T15:30:37.440698+0000","last_became_peered":"2024-01-24T15:30:37.440698+0000","last_unstale":"2024-01-24T15:30:37.449454+0000","last_undegraded":"2024-01-24T15:30:37.449454+0000","last_fullsized":"2024-01-24T15:30:37.449454+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.417176+0000","last_change":"2024-01-24T15:30:37.013248+0000","last_active":"2024-01-24T15:30:37.417176+0000","last_peered":"2024-01-24T15:30:37.417176+0000","last_clean":"2024-01-24T15:30:37.417176+0000","last_became_active":"2024-01-24T15:30:37.012908+0000","last_became_peered":"2024-01-24T15:30:37.012908+0000","last_unstale":"2024-01-24T15:30:37.417176+0000","last_undegraded":"2024-01-24T15:30:37.417176+0000","last_fullsized":"2024-01-24T15:30:37.417176+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.430541+0000","last_change":"2024-01-24T15:30:37.425489+0000","last_active":"2024-01-24T15:30:37.430541+0000","last_peered":"2024-01-24T15:30:37.430541+0000","last_clean":"2024-01-24T15:30:37.430541+0000","last_became_active":"2024-01-24T15:30:37.425212+0000","last_became_peered":"2024-01-24T15:30:37.425212+0000","last_unstale":"2024-01-24T15:30:37.430541+0000","last_undegraded":"2024-01-24T15:30:37.430541+0000","last_fullsized":"2024-01-24T15:30:37.430541+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"212'3","reported_seq":13,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448965+0000","last_change":"2024-01-24T15:30:36.999392+0000","last_active":"2024-01-24T15:30:37.448965+0000","last_peered":"2024-01-24T15:30:37.448965+0000","last_clean":"2024-01-24T15:30:37.448965+0000","last_became_active":"2024-01-24T15:30:36.999152+0000","last_became_peered":"2024-01-24T15:30:36.999152+0000","last_unstale":"2024-01-24T15:30:37.448965+0000","last_undegraded":"2024-01-24T15:30:37.448965+0000","last_fullsized":"2024-01-24T15:30:37.448965+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.450210+0000","last_change":"2024-01-24T15:30:37.449016+0000","last_active":"2024-01-24T15:30:37.450210+0000","last_peered":"2024-01-24T15:30:37.450210+0000","last_clean":"2024-01-24T15:30:37.450210+0000","last_became_active":"2024-01-24T15:30:37.448727+0000","last_became_peered":"2024-01-24T15:30:37.448727+0000","last_unstale":"2024-01-24T15:30:37.450210+0000","last_undegraded":"2024-01-24T15:30:37.450210+0000","last_fullsized":"2024-01-24T15:30:37.450210+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"212'2","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.445026+0000","last_change":"2024-01-24T15:30:37.009875+0000","last_active":"2024-01-24T15:30:37.445026+0000","last_peered":"2024-01-24T15:30:37.445026+0000","last_clean":"2024-01-24T15:30:37.445026+0000","last_became_active":"2024-01-24T15:30:37.009538+0000","last_became_peered":"2024-01-24T15:30:37.009538+0000","last_unstale":"2024-01-24T15:30:37.445026+0000","last_undegraded":"2024-01-24T15:30:37.445026+0000","last_fullsized":"2024-01-24T15:30:37.445026+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439676+0000","last_change":"2024-01-24T15:30:37.437804+0000","last_active":"2024-01-24T15:30:37.439676+0000","last_peered":"2024-01-24T15:30:37.439676+0000","last_clean":"2024-01-24T15:30:37.439676+0000","last_became_active":"2024-01-24T15:30:37.437531+0000","last_became_peered":"2024-01-24T15:30:37.437531+0000","last_unstale":"2024-01-24T15:30:37.439676+0000","last_undegraded":"2024-01-24T15:30:37.439676+0000","last_fullsized":"2024-01-24T15:30:37.439676+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.449259+0000","last_change":"2024-01-24T15:30:37.010627+0000","last_active":"2024-01-24T15:30:37.449259+0000","last_peered":"2024-01-24T15:30:37.449259+0000","last_clean":"2024-01-24T15:30:37.449259+0000","last_became_active":"2024-01-24T15:30:37.010267+0000","last_became_peered":"2024-01-24T15:30:37.010267+0000","last_unstale":"2024-01-24T15:30:37.449259+0000","last_undegraded":"2024-01-24T15:30:37.449259+0000","last_fullsized":"2024-01-24T15:30:37.449259+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.441633+0000","last_change":"2024-01-24T15:30:37.436591+0000","last_active":"2024-01-24T15:30:37.441633+0000","last_peered":"2024-01-24T15:30:37.441633+0000","last_clean":"2024-01-24T15:30:37.441633+0000","last_became_active":"2024-01-24T15:30:37.436324+0000","last_became_peered":"2024-01-24T15:30:37.436324+0000","last_unstale":"2024-01-24T15:30:37.441633+0000","last_undegraded":"2024-01-24T15:30:37.441633+0000","last_fullsized":"2024-01-24T15:30:37.441633+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425787+0000","last_change":"2024-01-24T15:30:37.023045+0000","last_active":"2024-01-24T15:30:37.425787+0000","last_peered":"2024-01-24T15:30:37.425787+0000","last_clean":"2024-01-24T15:30:37.425787+0000","last_became_active":"2024-01-24T15:30:37.022693+0000","last_became_peered":"2024-01-24T15:30:37.022693+0000","last_unstale":"2024-01-24T15:30:37.425787+0000","last_undegraded":"2024-01-24T15:30:37.425787+0000","last_fullsized":"2024-01-24T15:30:37.425787+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.440121+0000","last_change":"2024-01-24T15:30:37.433914+0000","last_active":"2024-01-24T15:30:37.440121+0000","last_peered":"2024-01-24T15:30:37.440121+0000","last_clean":"2024-01-24T15:30:37.440121+0000","last_became_active":"2024-01-24T15:30:37.433647+0000","last_became_peered":"2024-01-24T15:30:37.433647+0000","last_unstale":"2024-01-24T15:30:37.440121+0000","last_undegraded":"2024-01-24T15:30:37.440121+0000","last_fullsized":"2024-01-24T15:30:37.440121+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.418880+0000","last_change":"2024-01-24T15:30:37.010985+0000","last_active":"2024-01-24T15:30:37.418880+0000","last_peered":"2024-01-24T15:30:37.418880+0000","last_clean":"2024-01-24T15:30:37.418880+0000","last_became_active":"2024-01-24T15:30:37.010679+0000","last_became_peered":"2024-01-24T15:30:37.010679+0000","last_unstale":"2024-01-24T15:30:37.418880+0000","last_undegraded":"2024-01-24T15:30:37.418880+0000","last_fullsized":"2024-01-24T15:30:37.418880+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.441848+0000","last_change":"2024-01-24T15:30:37.441271+0000","last_active":"2024-01-24T15:30:37.441848+0000","last_peered":"2024-01-24T15:30:37.441848+0000","last_clean":"2024-01-24T15:30:37.441848+0000","last_became_active":"2024-01-24T15:30:37.440093+0000","last_became_peered":"2024-01-24T15:30:37.440093+0000","last_unstale":"2024-01-24T15:30:37.441848+0000","last_undegraded":"2024-01-24T15:30:37.441848+0000","last_fullsized":"2024-01-24T15:30:37.441848+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"212'4","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.450180+0000","last_change":"2024-01-24T15:30:37.011917+0000","last_active":"2024-01-24T15:30:37.450180+0000","last_peered":"2024-01-24T15:30:37.450180+0000","last_clean":"2024-01-24T15:30:37.450180+0000","last_became_active":"2024-01-24T15:30:37.011638+0000","last_became_peered":"2024-01-24T15:30:37.011638+0000","last_unstale":"2024-01-24T15:30:37.450180+0000","last_undegraded":"2024-01-24T15:30:37.450180+0000","last_fullsized":"2024-01-24T15:30:37.450180+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.441484+0000","last_change":"2024-01-24T15:30:37.439809+0000","last_active":"2024-01-24T15:30:37.441484+0000","last_peered":"2024-01-24T15:30:37.441484+0000","last_clean":"2024-01-24T15:30:37.441484+0000","last_became_active":"2024-01-24T15:30:37.439397+0000","last_became_peered":"2024-01-24T15:30:37.439397+0000","last_unstale":"2024-01-24T15:30:37.441484+0000","last_undegraded":"2024-01-24T15:30:37.441484+0000","last_fullsized":"2024-01-24T15:30:37.441484+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444386+0000","last_change":"2024-01-24T15:30:37.007051+0000","last_active":"2024-01-24T15:30:37.444386+0000","last_peered":"2024-01-24T15:30:37.444386+0000","last_clean":"2024-01-24T15:30:37.444386+0000","last_became_active":"2024-01-24T15:30:37.006421+0000","last_became_peered":"2024-01-24T15:30:37.006421+0000","last_unstale":"2024-01-24T15:30:37.444386+0000","last_undegraded":"2024-01-24T15:30:37.444386+0000","last_fullsized":"2024-01-24T15:30:37.444386+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.440140+0000","last_change":"2024-01-24T15:30:37.435862+0000","last_active":"2024-01-24T15:30:37.440140+0000","last_peered":"2024-01-24T15:30:37.440140+0000","last_clean":"2024-01-24T15:30:37.440140+0000","last_became_active":"2024-01-24T15:30:37.435556+0000","last_became_peered":"2024-01-24T15:30:37.435556+0000","last_unstale":"2024-01-24T15:30:37.440140+0000","last_undegraded":"2024-01-24T15:30:37.440140+0000","last_fullsized":"2024-01-24T15:30:37.440140+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"212'2","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.449367+0000","last_change":"2024-01-24T15:30:37.008278+0000","last_active":"2024-01-24T15:30:37.449367+0000","last_peered":"2024-01-24T15:30:37.449367+0000","last_clean":"2024-01-24T15:30:37.449367+0000","last_became_active":"2024-01-24T15:30:37.008023+0000","last_became_peered":"2024-01-24T15:30:37.008023+0000","last_unstale":"2024-01-24T15:30:37.449367+0000","last_undegraded":"2024-01-24T15:30:37.449367+0000","last_fullsized":"2024-01-24T15:30:37.449367+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439812+0000","last_change":"2024-01-24T15:30:37.432885+0000","last_active":"2024-01-24T15:30:37.439812+0000","last_peered":"2024-01-24T15:30:37.439812+0000","last_clean":"2024-01-24T15:30:37.439812+0000","last_became_active":"2024-01-24T15:30:37.432563+0000","last_became_peered":"2024-01-24T15:30:37.432563+0000","last_unstale":"2024-01-24T15:30:37.439812+0000","last_undegraded":"2024-01-24T15:30:37.439812+0000","last_fullsized":"2024-01-24T15:30:37.439812+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.426843+0000","last_change":"2024-01-24T15:30:37.012298+0000","last_active":"2024-01-24T15:30:37.426843+0000","last_peered":"2024-01-24T15:30:37.426843+0000","last_clean":"2024-01-24T15:30:37.426843+0000","last_became_active":"2024-01-24T15:30:37.012015+0000","last_became_peered":"2024-01-24T15:30:37.012015+0000","last_unstale":"2024-01-24T15:30:37.426843+0000","last_undegraded":"2024-01-24T15:30:37.426843+0000","last_fullsized":"2024-01-24T15:30:37.426843+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446774+0000","last_change":"2024-01-24T15:30:37.445997+0000","last_active":"2024-01-24T15:30:37.446774+0000","last_peered":"2024-01-24T15:30:37.446774+0000","last_clean":"2024-01-24T15:30:37.446774+0000","last_became_active":"2024-01-24T15:30:37.445713+0000","last_became_peered":"2024-01-24T15:30:37.445713+0000","last_unstale":"2024-01-24T15:30:37.446774+0000","last_undegraded":"2024-01-24T15:30:37.446774+0000","last_fullsized":"2024-01-24T15:30:37.446774+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.421185+0000","last_change":"2024-01-24T15:30:37.008994+0000","last_active":"2024-01-24T15:30:37.421185+0000","last_peered":"2024-01-24T15:30:37.421185+0000","last_clean":"2024-01-24T15:30:37.421185+0000","last_became_active":"2024-01-24T15:30:37.008656+0000","last_became_peered":"2024-01-24T15:30:37.008656+0000","last_unstale":"2024-01-24T15:30:37.421185+0000","last_undegraded":"2024-01-24T15:30:37.421185+0000","last_fullsized":"2024-01-24T15:30:37.421185+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.447986+0000","last_change":"2024-01-24T15:30:37.446107+0000","last_active":"2024-01-24T15:30:37.447986+0000","last_peered":"2024-01-24T15:30:37.447986+0000","last_clean":"2024-01-24T15:30:37.447986+0000","last_became_active":"2024-01-24T15:30:37.445877+0000","last_became_peered":"2024-01-24T15:30:37.445877+0000","last_unstale":"2024-01-24T15:30:37.447986+0000","last_undegraded":"2024-01-24T15:30:37.447986+0000","last_fullsized":"2024-01-24T15:30:37.447986+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444715+0000","last_change":"2024-01-24T15:30:37.023023+0000","last_active":"2024-01-24T15:30:37.444715+0000","last_peered":"2024-01-24T15:30:37.444715+0000","last_clean":"2024-01-24T15:30:37.444715+0000","last_became_active":"2024-01-24T15:30:37.022717+0000","last_became_peered":"2024-01-24T15:30:37.022717+0000","last_unstale":"2024-01-24T15:30:37.444715+0000","last_undegraded":"2024-01-24T15:30:37.444715+0000","last_fullsized":"2024-01-24T15:30:37.444715+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.428146+0000","last_change":"2024-01-24T15:30:37.424872+0000","last_active":"2024-01-24T15:30:37.428146+0000","last_peered":"2024-01-24T15:30:37.428146+0000","last_clean":"2024-01-24T15:30:37.428146+0000","last_became_active":"2024-01-24T15:30:37.424550+0000","last_became_peered":"2024-01-24T15:30:37.424550+0000","last_unstale":"2024-01-24T15:30:37.428146+0000","last_undegraded":"2024-01-24T15:30:37.428146+0000","last_fullsized":"2024-01-24T15:30:37.428146+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.415400+0000","last_change":"2024-01-24T15:30:37.010684+0000","last_active":"2024-01-24T15:30:37.415400+0000","last_peered":"2024-01-24T15:30:37.415400+0000","last_clean":"2024-01-24T15:30:37.415400+0000","last_became_active":"2024-01-24T15:30:37.010372+0000","last_became_peered":"2024-01-24T15:30:37.010372+0000","last_unstale":"2024-01-24T15:30:37.415400+0000","last_undegraded":"2024-01-24T15:30:37.415400+0000","last_fullsized":"2024-01-24T15:30:37.415400+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.438100+0000","last_change":"2024-01-24T15:30:37.432163+0000","last_active":"2024-01-24T15:30:37.438100+0000","last_peered":"2024-01-24T15:30:37.438100+0000","last_clean":"2024-01-24T15:30:37.438100+0000","last_became_active":"2024-01-24T15:30:37.431851+0000","last_became_peered":"2024-01-24T15:30:37.431851+0000","last_unstale":"2024-01-24T15:30:37.438100+0000","last_undegraded":"2024-01-24T15:30:37.438100+0000","last_fullsized":"2024-01-24T15:30:37.438100+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425230+0000","last_change":"2024-01-24T15:30:37.020665+0000","last_active":"2024-01-24T15:30:37.425230+0000","last_peered":"2024-01-24T15:30:37.425230+0000","last_clean":"2024-01-24T15:30:37.425230+0000","last_became_active":"2024-01-24T15:30:37.019763+0000","last_became_peered":"2024-01-24T15:30:37.019763+0000","last_unstale":"2024-01-24T15:30:37.425230+0000","last_undegraded":"2024-01-24T15:30:37.425230+0000","last_fullsized":"2024-01-24T15:30:37.425230+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.434399+0000","last_change":"2024-01-24T15:30:37.429153+0000","last_active":"2024-01-24T15:30:37.434399+0000","last_peered":"2024-01-24T15:30:37.434399+0000","last_clean":"2024-01-24T15:30:37.434399+0000","last_became_active":"2024-01-24T15:30:37.428657+0000","last_became_peered":"2024-01-24T15:30:37.428657+0000","last_unstale":"2024-01-24T15:30:37.434399+0000","last_undegraded":"2024-01-24T15:30:37.434399+0000","last_fullsized":"2024-01-24T15:30:37.434399+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419309+0000","last_change":"2024-01-24T15:30:37.010593+0000","last_active":"2024-01-24T15:30:37.419309+0000","last_peered":"2024-01-24T15:30:37.419309+0000","last_clean":"2024-01-24T15:30:37.419309+0000","last_became_active":"2024-01-24T15:30:37.010306+0000","last_became_peered":"2024-01-24T15:30:37.010306+0000","last_unstale":"2024-01-24T15:30:37.419309+0000","last_undegraded":"2024-01-24T15:30:37.419309+0000","last_fullsized":"2024-01-24T15:30:37.419309+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.434255+0000","last_change":"2024-01-24T15:30:37.428903+0000","last_active":"2024-01-24T15:30:37.434255+0000","last_peered":"2024-01-24T15:30:37.434255+0000","last_clean":"2024-01-24T15:30:37.434255+0000","last_became_active":"2024-01-24T15:30:37.428533+0000","last_became_peered":"2024-01-24T15:30:37.428533+0000","last_unstale":"2024-01-24T15:30:37.434255+0000","last_undegraded":"2024-01-24T15:30:37.434255+0000","last_fullsized":"2024-01-24T15:30:37.434255+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"212'2","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.445039+0000","last_change":"2024-01-24T15:30:37.020234+0000","last_active":"2024-01-24T15:30:37.445039+0000","last_peered":"2024-01-24T15:30:37.445039+0000","last_clean":"2024-01-24T15:30:37.445039+0000","last_became_active":"2024-01-24T15:30:37.019961+0000","last_became_peered":"2024-01-24T15:30:37.019961+0000","last_unstale":"2024-01-24T15:30:37.445039+0000","last_undegraded":"2024-01-24T15:30:37.445039+0000","last_fullsized":"2024-01-24T15:30:37.445039+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439389+0000","last_change":"2024-01-24T15:30:37.436321+0000","last_active":"2024-01-24T15:30:37.439389+0000","last_peered":"2024-01-24T15:30:37.439389+0000","last_clean":"2024-01-24T15:30:37.439389+0000","last_became_active":"2024-01-24T15:30:37.435999+0000","last_became_peered":"2024-01-24T15:30:37.435999+0000","last_unstale":"2024-01-24T15:30:37.439389+0000","last_undegraded":"2024-01-24T15:30:37.439389+0000","last_fullsized":"2024-01-24T15:30:37.439389+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.434740+0000","last_change":"2024-01-24T15:30:37.020412+0000","last_active":"2024-01-24T15:30:37.434740+0000","last_peered":"2024-01-24T15:30:37.434740+0000","last_clean":"2024-01-24T15:30:37.434740+0000","last_became_active":"2024-01-24T15:30:37.020172+0000","last_became_peered":"2024-01-24T15:30:37.020172+0000","last_unstale":"2024-01-24T15:30:37.434740+0000","last_undegraded":"2024-01-24T15:30:37.434740+0000","last_fullsized":"2024-01-24T15:30:37.434740+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448311+0000","last_change":"2024-01-24T15:30:37.431673+0000","last_active":"2024-01-24T15:30:37.448311+0000","last_peered":"2024-01-24T15:30:37.448311+0000","last_clean":"2024-01-24T15:30:37.448311+0000","last_became_active":"2024-01-24T15:30:37.431342+0000","last_became_peered":"2024-01-24T15:30:37.431342+0000","last_unstale":"2024-01-24T15:30:37.448311+0000","last_undegraded":"2024-01-24T15:30:37.448311+0000","last_fullsized":"2024-01-24T15:30:37.448311+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.427225+0000","last_change":"2024-01-24T15:30:37.019758+0000","last_active":"2024-01-24T15:30:37.427225+0000","last_peered":"2024-01-24T15:30:37.427225+0000","last_clean":"2024-01-24T15:30:37.427225+0000","last_became_active":"2024-01-24T15:30:37.019451+0000","last_became_peered":"2024-01-24T15:30:37.019451+0000","last_unstale":"2024-01-24T15:30:37.427225+0000","last_undegraded":"2024-01-24T15:30:37.427225+0000","last_fullsized":"2024-01-24T15:30:37.427225+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448339+0000","last_change":"2024-01-24T15:30:37.439015+0000","last_active":"2024-01-24T15:30:37.448339+0000","last_peered":"2024-01-24T15:30:37.448339+0000","last_clean":"2024-01-24T15:30:37.448339+0000","last_became_active":"2024-01-24T15:30:37.438713+0000","last_became_peered":"2024-01-24T15:30:37.438713+0000","last_unstale":"2024-01-24T15:30:37.448339+0000","last_undegraded":"2024-01-24T15:30:37.448339+0000","last_fullsized":"2024-01-24T15:30:37.448339+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419898+0000","last_change":"2024-01-24T15:30:37.011516+0000","last_active":"2024-01-24T15:30:37.419898+0000","last_peered":"2024-01-24T15:30:37.419898+0000","last_clean":"2024-01-24T15:30:37.419898+0000","last_became_active":"2024-01-24T15:30:37.010843+0000","last_became_peered":"2024-01-24T15:30:37.010843+0000","last_unstale":"2024-01-24T15:30:37.419898+0000","last_undegraded":"2024-01-24T15:30:37.419898+0000","last_fullsized":"2024-01-24T15:30:37.419898+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.449445+0000","last_change":"2024-01-24T15:30:37.447387+0000","last_active":"2024-01-24T15:30:37.449445+0000","last_peered":"2024-01-24T15:30:37.449445+0000","last_clean":"2024-01-24T15:30:37.449445+0000","last_became_active":"2024-01-24T15:30:37.447106+0000","last_became_peered":"2024-01-24T15:30:37.447106+0000","last_unstale":"2024-01-24T15:30:37.449445+0000","last_undegraded":"2024-01-24T15:30:37.449445+0000","last_fullsized":"2024-01-24T15:30:37.449445+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420642+0000","last_change":"2024-01-24T15:30:37.006558+0000","last_active":"2024-01-24T15:30:37.420642+0000","last_peered":"2024-01-24T15:30:37.420642+0000","last_clean":"2024-01-24T15:30:37.420642+0000","last_became_active":"2024-01-24T15:30:37.006275+0000","last_became_peered":"2024-01-24T15:30:37.006275+0000","last_unstale":"2024-01-24T15:30:37.420642+0000","last_undegraded":"2024-01-24T15:30:37.420642+0000","last_fullsized":"2024-01-24T15:30:37.420642+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446588+0000","last_change":"2024-01-24T15:30:37.444460+0000","last_active":"2024-01-24T15:30:37.446588+0000","last_peered":"2024-01-24T15:30:37.446588+0000","last_clean":"2024-01-24T15:30:37.446588+0000","last_became_active":"2024-01-24T15:30:37.444196+0000","last_became_peered":"2024-01-24T15:30:37.444196+0000","last_unstale":"2024-01-24T15:30:37.446588+0000","last_undegraded":"2024-01-24T15:30:37.446588+0000","last_fullsized":"2024-01-24T15:30:37.446588+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419850+0000","last_change":"2024-01-24T15:30:37.002116+0000","last_active":"2024-01-24T15:30:37.419850+0000","last_peered":"2024-01-24T15:30:37.419850+0000","last_clean":"2024-01-24T15:30:37.419850+0000","last_became_active":"2024-01-24T15:30:37.001841+0000","last_became_peered":"2024-01-24T15:30:37.001841+0000","last_unstale":"2024-01-24T15:30:37.419850+0000","last_undegraded":"2024-01-24T15:30:37.419850+0000","last_fullsized":"2024-01-24T15:30:37.419850+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.438080+0000","last_change":"2024-01-24T15:30:37.431537+0000","last_active":"2024-01-24T15:30:37.438080+0000","last_peered":"2024-01-24T15:30:37.438080+0000","last_clean":"2024-01-24T15:30:37.438080+0000","last_became_active":"2024-01-24T15:30:37.431229+0000","last_became_peered":"2024-01-24T15:30:37.431229+0000","last_unstale":"2024-01-24T15:30:37.438080+0000","last_undegraded":"2024-01-24T15:30:37.438080+0000","last_fullsized":"2024-01-24T15:30:37.438080+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423792+0000","last_change":"2024-01-24T15:30:37.009420+0000","last_active":"2024-01-24T15:30:37.423792+0000","last_peered":"2024-01-24T15:30:37.423792+0000","last_clean":"2024-01-24T15:30:37.423792+0000","last_became_active":"2024-01-24T15:30:37.009141+0000","last_became_peered":"2024-01-24T15:30:37.009141+0000","last_unstale":"2024-01-24T15:30:37.423792+0000","last_undegraded":"2024-01-24T15:30:37.423792+0000","last_fullsized":"2024-01-24T15:30:37.423792+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444250+0000","last_change":"2024-01-24T15:30:37.434072+0000","last_active":"2024-01-24T15:30:37.444250+0000","last_peered":"2024-01-24T15:30:37.444250+0000","last_clean":"2024-01-24T15:30:37.444250+0000","last_became_active":"2024-01-24T15:30:37.433708+0000","last_became_peered":"2024-01-24T15:30:37.433708+0000","last_unstale":"2024-01-24T15:30:37.444250+0000","last_undegraded":"2024-01-24T15:30:37.444250+0000","last_fullsized":"2024-01-24T15:30:37.444250+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425395+0000","last_change":"2024-01-24T15:30:37.020172+0000","last_active":"2024-01-24T15:30:37.425395+0000","last_peered":"2024-01-24T15:30:37.425395+0000","last_clean":"2024-01-24T15:30:37.425395+0000","last_became_active":"2024-01-24T15:30:37.019841+0000","last_became_peered":"2024-01-24T15:30:37.019841+0000","last_unstale":"2024-01-24T15:30:37.425395+0000","last_undegraded":"2024-01-24T15:30:37.425395+0000","last_fullsized":"2024-01-24T15:30:37.425395+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.445457+0000","last_change":"2024-01-24T15:30:37.444454+0000","last_active":"2024-01-24T15:30:37.445457+0000","last_peered":"2024-01-24T15:30:37.445457+0000","last_clean":"2024-01-24T15:30:37.445457+0000","last_became_active":"2024-01-24T15:30:37.444142+0000","last_became_peered":"2024-01-24T15:30:37.444142+0000","last_unstale":"2024-01-24T15:30:37.445457+0000","last_undegraded":"2024-01-24T15:30:37.445457+0000","last_fullsized":"2024-01-24T15:30:37.445457+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439691+0000","last_change":"2024-01-24T15:30:37.005757+0000","last_active":"2024-01-24T15:30:37.439691+0000","last_peered":"2024-01-24T15:30:37.439691+0000","last_clean":"2024-01-24T15:30:37.439691+0000","last_became_active":"2024-01-24T15:30:37.005509+0000","last_became_peered":"2024-01-24T15:30:37.005509+0000","last_unstale":"2024-01-24T15:30:37.439691+0000","last_undegraded":"2024-01-24T15:30:37.439691+0000","last_fullsized":"2024-01-24T15:30:37.439691+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423817+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:30:37.423817+0000","last_peered":"2024-01-24T15:30:37.423817+0000","last_clean":"2024-01-24T15:30:37.423817+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:30:37.423817+0000","last_undegraded":"2024-01-24T15:30:37.423817+0000","last_fullsized":"2024-01-24T15:30:37.423817+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.443700+0000","last_change":"2024-01-24T15:30:37.438090+0000","last_active":"2024-01-24T15:30:37.443700+0000","last_peered":"2024-01-24T15:30:37.443700+0000","last_clean":"2024-01-24T15:30:37.443700+0000","last_became_active":"2024-01-24T15:30:37.437816+0000","last_became_peered":"2024-01-24T15:30:37.437816+0000","last_unstale":"2024-01-24T15:30:37.443700+0000","last_undegraded":"2024-01-24T15:30:37.443700+0000","last_fullsized":"2024-01-24T15:30:37.443700+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425229+0000","last_change":"2024-01-24T15:30:37.011798+0000","last_active":"2024-01-24T15:30:37.425229+0000","last_peered":"2024-01-24T15:30:37.425229+0000","last_clean":"2024-01-24T15:30:37.425229+0000","last_became_active":"2024-01-24T15:30:37.010303+0000","last_became_peered":"2024-01-24T15:30:37.010303+0000","last_unstale":"2024-01-24T15:30:37.425229+0000","last_undegraded":"2024-01-24T15:30:37.425229+0000","last_fullsized":"2024-01-24T15:30:37.425229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":196,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420380+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:30:37.420380+0000","last_peered":"2024-01-24T15:30:37.420380+0000","last_clean":"2024-01-24T15:30:37.420380+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:30:37.420380+0000","last_undegraded":"2024-01-24T15:30:37.420380+0000","last_fullsized":"2024-01-24T15:30:37.420380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448897+0000","last_change":"2024-01-24T15:30:37.447169+0000","last_active":"2024-01-24T15:30:37.448897+0000","last_peered":"2024-01-24T15:30:37.448897+0000","last_clean":"2024-01-24T15:30:37.448897+0000","last_became_active":"2024-01-24T15:30:37.446858+0000","last_became_peered":"2024-01-24T15:30:37.446858+0000","last_unstale":"2024-01-24T15:30:37.448897+0000","last_undegraded":"2024-01-24T15:30:37.448897+0000","last_fullsized":"2024-01-24T15:30:37.448897+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.415807+0000","last_change":"2024-01-24T15:30:37.007210+0000","last_active":"2024-01-24T15:30:37.415807+0000","last_peered":"2024-01-24T15:30:37.415807+0000","last_clean":"2024-01-24T15:30:37.415807+0000","last_became_active":"2024-01-24T15:30:37.006907+0000","last_became_peered":"2024-01-24T15:30:37.006907+0000","last_unstale":"2024-01-24T15:30:37.415807+0000","last_undegraded":"2024-01-24T15:30:37.415807+0000","last_fullsized":"2024-01-24T15:30:37.415807+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":190,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.415645+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:30:37.415645+0000","last_peered":"2024-01-24T15:30:37.415645+0000","last_clean":"2024-01-24T15:30:37.415645+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:30:37.415645+0000","last_undegraded":"2024-01-24T15:30:37.415645+0000","last_fullsized":"2024-01-24T15:30:37.415645+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.435324+0000","last_change":"2024-01-24T15:30:37.431803+0000","last_active":"2024-01-24T15:30:37.435324+0000","last_peered":"2024-01-24T15:30:37.435324+0000","last_clean":"2024-01-24T15:30:37.435324+0000","last_became_active":"2024-01-24T15:30:37.431639+0000","last_became_peered":"2024-01-24T15:30:37.431639+0000","last_unstale":"2024-01-24T15:30:37.435324+0000","last_undegraded":"2024-01-24T15:30:37.435324+0000","last_fullsized":"2024-01-24T15:30:37.435324+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423196+0000","last_change":"2024-01-24T15:30:37.016795+0000","last_active":"2024-01-24T15:30:37.423196+0000","last_peered":"2024-01-24T15:30:37.423196+0000","last_clean":"2024-01-24T15:30:37.423196+0000","last_became_active":"2024-01-24T15:30:37.013889+0000","last_became_peered":"2024-01-24T15:30:37.013889+0000","last_unstale":"2024-01-24T15:30:37.423196+0000","last_undegraded":"2024-01-24T15:30:37.423196+0000","last_fullsized":"2024-01-24T15:30:37.423196+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":196,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420780+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:30:37.420780+0000","last_peered":"2024-01-24T15:30:37.420780+0000","last_clean":"2024-01-24T15:30:37.420780+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:30:37.420780+0000","last_undegraded":"2024-01-24T15:30:37.420780+0000","last_fullsized":"2024-01-24T15:30:37.420780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":175,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425505+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:30:37.425505+0000","last_peered":"2024-01-24T15:30:37.425505+0000","last_clean":"2024-01-24T15:30:37.425505+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:30:37.425505+0000","last_undegraded":"2024-01-24T15:30:37.425505+0000","last_fullsized":"2024-01-24T15:30:37.425505+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446726+0000","last_change":"2024-01-24T15:30:37.429923+0000","last_active":"2024-01-24T15:30:37.446726+0000","last_peered":"2024-01-24T15:30:37.446726+0000","last_clean":"2024-01-24T15:30:37.446726+0000","last_became_active":"2024-01-24T15:30:37.426365+0000","last_became_peered":"2024-01-24T15:30:37.426365+0000","last_unstale":"2024-01-24T15:30:37.446726+0000","last_undegraded":"2024-01-24T15:30:37.446726+0000","last_fullsized":"2024-01-24T15:30:37.446726+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419043+0000","last_change":"2024-01-24T15:30:37.014881+0000","last_active":"2024-01-24T15:30:37.419043+0000","last_peered":"2024-01-24T15:30:37.419043+0000","last_clean":"2024-01-24T15:30:37.419043+0000","last_became_active":"2024-01-24T15:30:37.014589+0000","last_became_peered":"2024-01-24T15:30:37.014589+0000","last_unstale":"2024-01-24T15:30:37.419043+0000","last_undegraded":"2024-01-24T15:30:37.419043+0000","last_fullsized":"2024-01-24T15:30:37.419043+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":173,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425424+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:30:37.425424+0000","last_peered":"2024-01-24T15:30:37.425424+0000","last_clean":"2024-01-24T15:30:37.425424+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:30:37.425424+0000","last_undegraded":"2024-01-24T15:30:37.425424+0000","last_fullsized":"2024-01-24T15:30:37.425424+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446260+0000","last_change":"2024-01-24T15:30:37.425910+0000","last_active":"2024-01-24T15:30:37.446260+0000","last_peered":"2024-01-24T15:30:37.446260+0000","last_clean":"2024-01-24T15:30:37.446260+0000","last_became_active":"2024-01-24T15:30:37.425624+0000","last_became_peered":"2024-01-24T15:30:37.425624+0000","last_unstale":"2024-01-24T15:30:37.446260+0000","last_undegraded":"2024-01-24T15:30:37.446260+0000","last_fullsized":"2024-01-24T15:30:37.446260+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.443429+0000","last_change":"2024-01-24T15:30:37.002291+0000","last_active":"2024-01-24T15:30:37.443429+0000","last_peered":"2024-01-24T15:30:37.443429+0000","last_clean":"2024-01-24T15:30:37.443429+0000","last_became_active":"2024-01-24T15:30:37.001934+0000","last_became_peered":"2024-01-24T15:30:37.001934+0000","last_unstale":"2024-01-24T15:30:37.443429+0000","last_undegraded":"2024-01-24T15:30:37.443429+0000","last_fullsized":"2024-01-24T15:30:37.443429+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":173,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.426939+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:30:37.426939+0000","last_peered":"2024-01-24T15:30:37.426939+0000","last_clean":"2024-01-24T15:30:37.426939+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:30:37.426939+0000","last_undegraded":"2024-01-24T15:30:37.426939+0000","last_fullsized":"2024-01-24T15:30:37.426939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.430435+0000","last_change":"2024-01-24T15:30:37.425987+0000","last_active":"2024-01-24T15:30:37.430435+0000","last_peered":"2024-01-24T15:30:37.430435+0000","last_clean":"2024-01-24T15:30:37.430435+0000","last_became_active":"2024-01-24T15:30:37.425720+0000","last_became_peered":"2024-01-24T15:30:37.425720+0000","last_unstale":"2024-01-24T15:30:37.430435+0000","last_undegraded":"2024-01-24T15:30:37.430435+0000","last_fullsized":"2024-01-24T15:30:37.430435+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423469+0000","last_change":"2024-01-24T15:30:37.008862+0000","last_active":"2024-01-24T15:30:37.423469+0000","last_peered":"2024-01-24T15:30:37.423469+0000","last_clean":"2024-01-24T15:30:37.423469+0000","last_became_active":"2024-01-24T15:30:37.008557+0000","last_became_peered":"2024-01-24T15:30:37.008557+0000","last_unstale":"2024-01-24T15:30:37.423469+0000","last_undegraded":"2024-01-24T15:30:37.423469+0000","last_fullsized":"2024-01-24T15:30:37.423469+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":172,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423914+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:30:37.423914+0000","last_peered":"2024-01-24T15:30:37.423914+0000","last_clean":"2024-01-24T15:30:37.423914+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:30:37.423914+0000","last_undegraded":"2024-01-24T15:30:37.423914+0000","last_fullsized":"2024-01-24T15:30:37.423914+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.428810+0000","last_change":"2024-01-24T15:30:37.424775+0000","last_active":"2024-01-24T15:30:37.428810+0000","last_peered":"2024-01-24T15:30:37.428810+0000","last_clean":"2024-01-24T15:30:37.428810+0000","last_became_active":"2024-01-24T15:30:37.424447+0000","last_became_peered":"2024-01-24T15:30:37.424447+0000","last_unstale":"2024-01-24T15:30:37.428810+0000","last_undegraded":"2024-01-24T15:30:37.428810+0000","last_fullsized":"2024-01-24T15:30:37.428810+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"212'2","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.440081+0000","last_change":"2024-01-24T15:30:37.017107+0000","last_active":"2024-01-24T15:30:37.440081+0000","last_peered":"2024-01-24T15:30:37.440081+0000","last_clean":"2024-01-24T15:30:37.440081+0000","last_became_active":"2024-01-24T15:30:37.016820+0000","last_became_peered":"2024-01-24T15:30:37.016820+0000","last_unstale":"2024-01-24T15:30:37.440081+0000","last_undegraded":"2024-01-24T15:30:37.440081+0000","last_fullsized":"2024-01-24T15:30:37.440081+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":171,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.424181+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:30:37.424181+0000","last_peered":"2024-01-24T15:30:37.424181+0000","last_clean":"2024-01-24T15:30:37.424181+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:30:37.424181+0000","last_undegraded":"2024-01-24T15:30:37.424181+0000","last_fullsized":"2024-01-24T15:30:37.424181+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575953,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303788,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068052,"statfs":{"total":96636764160,"available":96325685248,"internally_reserved":0,"allocated":2658304,"data_stored":1534881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.48699999999999999,"5min":0.38800000000000001,"15min":0.32000000000000001},"max":{"1min":0.80300000000000005,"5min":0.80500000000000005,"15min":0.83199999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.63300000000000001,"15min":0.63200000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81100000000000005,"5min":0.84799999999999998,"15min":0.85499999999999998},"last":0.68300000000000005}]},{"osd":1,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.64400000000000002},"min":{"1min":0.497,"5min":0.46000000000000002,"15min":0.36699999999999999},"max":{"1min":0.84499999999999997,"5min":0.878,"15min":0.878},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73699999999999999,"5min":0.80600000000000005,"15min":0.89600000000000002},"last":0.63600000000000001}]},{"osd":2,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.77400000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":0.83299999999999996,"15min":0.88800000000000001},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.42399999999999999},"max":{"1min":0.71899999999999997,"5min":0.83399999999999996,"15min":0.92600000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45200000000000001,"5min":0.42799999999999999,"15min":0.42099999999999999},"max":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.88},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.36199999999999999,"5min":0.35599999999999998,"15min":0.30599999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79900000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.83299999999999996},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.50600000000000001},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.32300000000000001},"max":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.76100000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.42299999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.83399999999999996},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.41299999999999998,"5min":0.377,"15min":0.34999999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.81899999999999995},"last":0.626},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.35099999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.92700000000000005},"last":0.71699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575954,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307892,"kb_used_data":2604,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063948,"statfs":{"total":96636764160,"available":96321482752,"internally_reserved":0,"allocated":2666496,"data_stored":1542278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.504,"5min":0.42299999999999999,"15min":0.31900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.63200000000000001,"15min":0.61299999999999999},"min":{"1min":0.46999999999999997,"5min":0.44700000000000001,"15min":0.32800000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.95599999999999996},"last":0.53000000000000003}]},{"osd":1,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.34399999999999997},"max":{"1min":0.72999999999999998,"5min":0.86399999999999999,"15min":1.089},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.47099999999999997,"5min":0.42699999999999999,"15min":0.372},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.94299999999999995},"last":0.51800000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.629,"15min":0.625},"min":{"1min":0.439,"5min":0.439,"15min":0.36599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63700000000000001,"15min":0.62},"min":{"1min":0.48799999999999999,"5min":0.45100000000000001,"15min":0.40600000000000003},"max":{"1min":0.81000000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64500000000000002,"15min":0.63600000000000001},"min":{"1min":0.56000000000000005,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.0980000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.47199999999999998,"5min":0.42399999999999999,"15min":0.40999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":1.169},"last":0.48899999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.38500000000000001,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.71399999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":0.81399999999999995,"15min":0.83999999999999997},"last":0.56599999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55300000000000005,"15min":0.54400000000000004},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36399999999999999},"max":{"1min":0.78800000000000003,"5min":0.84199999999999997,"15min":0.86499999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54800000000000004,"15min":0.53300000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33900000000000002},"max":{"1min":0.68700000000000006,"5min":0.80100000000000005,"15min":0.82799999999999996},"last":0.36299999999999999}]},{"osd":7,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83699999999999997,"5min":0.88,"15min":0.88},"last":0.378},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.50900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575953,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307900,"kb_used_data":2612,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063940,"statfs":{"total":96636764160,"available":96321474560,"internally_reserved":0,"allocated":2674688,"data_stored":1542438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.28399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58099999999999996,"5min":0.66000000000000003,"15min":0.68500000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.44800000000000001,"15min":0.443},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.26100000000000001},"max":{"1min":0.51600000000000001,"5min":0.745,"15min":0.745},"last":0.33400000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.308,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56299999999999994,"5min":0.72999999999999998,"15min":0.76100000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.45100000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.626,"5min":0.68100000000000005,"15min":0.71099999999999997},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46999999999999997,"15min":0.46700000000000003},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.60099999999999998,"5min":0.71399999999999997,"15min":0.77000000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.502,"15min":0.499},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.32600000000000001},"max":{"1min":0.68300000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57999999999999996}]},{"osd":4,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73699999999999999,"15min":0.70099999999999996},"min":{"1min":0.51800000000000002,"5min":0.48399999999999999,"15min":0.41599999999999998},"max":{"1min":0.89000000000000001,"5min":5.3630000000000004,"15min":5.3630000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70599999999999996,"15min":0.68899999999999995},"min":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":1.677,"15min":1.677},"last":0.80600000000000005}]},{"osd":5,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.48499999999999999},"max":{"1min":0.92700000000000005,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.54900000000000004,"5min":0.5,"15min":0.46100000000000002},"max":{"1min":0.89600000000000002,"5min":1.1399999999999999,"15min":1.2989999999999999},"last":0.85599999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.52300000000000002,"5min":0.47899999999999998,"15min":0.46400000000000002},"max":{"1min":0.85699999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76300000000000001,"15min":0.753},"min":{"1min":0.63800000000000001,"5min":0.52000000000000002,"15min":0.47799999999999998},"max":{"1min":0.91200000000000003,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.84199999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.52900000000000003,"15min":0.40799999999999997},"max":{"1min":0.82399999999999995,"5min":1.704,"15min":1.704},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.82099999999999995,"15min":0.78900000000000003},"min":{"1min":0.61699999999999999,"5min":0.53800000000000003,"15min":0.47999999999999998},"max":{"1min":0.96699999999999997,"5min":5.29,"15min":5.29},"last":0.79400000000000004}]}]},{"osd":0,"up_from":14,"seq":60129543251,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307876,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063964,"statfs":{"total":96636764160,"available":96321499136,"internally_reserved":0,"allocated":2650112,"data_stored":1533016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41099999999999998,"15min":0.40600000000000003},"min":{"1min":0.27300000000000002,"5min":0.26500000000000001,"15min":0.26000000000000001},"max":{"1min":0.58199999999999996,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.27300000000000002},"max":{"1min":0.59199999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.34200000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.60999999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.625},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47499999999999998,"15min":0.46800000000000003},"min":{"1min":0.36699999999999999,"5min":0.33600000000000002,"15min":0.317},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.72999999999999998},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.441,"15min":0.441},"min":{"1min":0.33100000000000002,"5min":0.307,"15min":0.28000000000000003},"max":{"1min":0.61699999999999999,"5min":0.67400000000000004,"15min":0.72799999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.36399999999999999,"15min":0.35999999999999999},"max":{"1min":0.69399999999999995,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.81200000000000006}]},{"osd":4,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.496,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":0.82899999999999996,"5min":0.88600000000000001,"15min":0.98699999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.67100000000000004},"min":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.41199999999999998},"max":{"1min":0.877,"5min":0.998,"15min":1.0189999999999999},"last":0.755}]},{"osd":5,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.40699999999999997},"max":{"1min":0.95699999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.71499999999999997},"min":{"1min":0.59999999999999998,"5min":0.51400000000000001,"15min":0.46500000000000002},"max":{"1min":0.99199999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.94599999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73799999999999999,"15min":0.71899999999999997},"min":{"1min":0.57499999999999996,"5min":0.503,"15min":0.49299999999999999},"max":{"1min":0.98399999999999999,"5min":1.27,"15min":1.27},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72399999999999998,"15min":0.71499999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.71799999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.745,"15min":0.73299999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.72799999999999998},"min":{"1min":0.56299999999999994,"5min":0.5,"15min":0.46600000000000003},"max":{"1min":1.0189999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575953,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311992,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059848,"statfs":{"total":96636764160,"available":96317284352,"internally_reserved":0,"allocated":2670592,"data_stored":1541836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:30:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.41299999999999998,"15min":0.40500000000000003},"min":{"1min":0.30099999999999999,"5min":0.26800000000000002,"15min":0.252},"max":{"1min":0.59799999999999998,"5min":0.627,"15min":0.69699999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.45300000000000001},"min":{"1min":0.317,"5min":0.30399999999999999,"15min":0.28499999999999998},"max":{"1min":0.63,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.45500000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.29199999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.629,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.47999999999999998},"min":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.27400000000000002},"max":{"1min":0.69799999999999995,"5min":0.70299999999999996,"15min":0.80900000000000005},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Jan 24 15:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48099999999999998,"15min":0.47199999999999998},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.29099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.39800000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68500000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.503}]},{"osd":4,"last update":"Wed Jan 24 15:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68999999999999995,"15min":0.66700000000000004},"min":{"1min":0.53900000000000003,"5min":0.51600000000000001,"15min":0.38300000000000001},"max":{"1min":0.95999999999999996,"5min":0.96399999999999997,"15min":0.97999999999999998},"last":0.745},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70999999999999996,"15min":0.68899999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.376},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.056},"last":0.80900000000000005}]},{"osd":5,"last update":"Wed Jan 24 15:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.71599999999999997},"min":{"1min":0.54100000000000004,"5min":0.53600000000000003,"15min":0.439},"max":{"1min":0.996,"5min":1.0589999999999999,"15min":1.0660000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70899999999999996,"15min":0.69599999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.45700000000000002},"max":{"1min":1.024,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.59299999999999997}]},{"osd":6,"last update":"Wed Jan 24 15:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.748,"15min":0.73099999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.39700000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.0740000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76300000000000001,"15min":0.74399999999999999},"min":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.49399999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.107},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76300000000000001,"15min":0.749},"min":{"1min":0.60799999999999998,"5min":0.52300000000000002,"15min":0.47199999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.628},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78900000000000003,"15min":0.76200000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.495},"max":{"1min":1.02,"5min":1.0429999999999999,"15min":1.123},"last":0.84399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575953,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307896,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063944,"statfs":{"total":96636764160,"available":96321478656,"internally_reserved":0,"allocated":2670592,"data_stored":1541848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40899999999999997,"15min":0.40699999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51800000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.436,"15min":0.436},"min":{"1min":0.33500000000000002,"5min":0.28299999999999997,"15min":0.27900000000000003},"max":{"1min":0.70299999999999996,"5min":0.76100000000000001,"15min":0.77100000000000002},"last":0.60699999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42299999999999999,"15min":0.42799999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":0.57799999999999996,"5min":0.625,"15min":0.66900000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48199999999999998,"15min":0.47999999999999998},"min":{"1min":0.33100000000000002,"5min":0.312,"15min":0.29099999999999998},"max":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.752},"last":0.63100000000000001}]},{"osd":2,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.64400000000000002,"5min":0.71899999999999997,"15min":0.80800000000000005},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.433,"5min":0.34899999999999998,"15min":0.33400000000000002},"max":{"1min":0.68500000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.47699999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.79300000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.44700000000000001},"max":{"1min":0.88800000000000001,"5min":1.026,"15min":1.026},"last":1.026}]},{"osd":5,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.50600000000000001,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.97699999999999998,"5min":1.05,"15min":1.05},"last":1.056},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.47199999999999998},"max":{"1min":0.92300000000000004,"5min":0.997,"15min":0.997},"last":1.095}]},{"osd":6,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.52500000000000002,"5min":0.499,"15min":0.47999999999999998},"max":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":1.1499999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73299999999999998,"15min":0.72199999999999998},"min":{"1min":0.53600000000000003,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.90400000000000003,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.88700000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58199999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93300000000000005,"5min":1.002,"15min":1.083},"last":1.071},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77500000000000002,"15min":0.76900000000000002},"min":{"1min":0.55700000000000005,"5min":0.55500000000000005,"15min":0.52600000000000002},"max":{"1min":0.96899999999999997,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.91300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575954,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307904,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063936,"statfs":{"total":96636764160,"available":96321470464,"internally_reserved":0,"allocated":2678784,"data_stored":1543814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.375},"max":{"1min":0.80100000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.443},{"interface":"front","average":{"1min":0.627,"5min":0.621,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.44700000000000001,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.876,"15min":0.876},"last":0.51500000000000001}]},{"osd":1,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.45900000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.83899999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.46100000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78400000000000003,"5min":0.81499999999999995,"15min":0.85199999999999998},"last":0.35299999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60999999999999999,"15min":0.61899999999999999},"min":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.71599999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.379},{"interface":"front","average":{"1min":0.627,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.48199999999999998,"5min":0.42699999999999999,"15min":0.38500000000000001},"max":{"1min":0.84399999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42599999999999999}]},{"osd":3,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72499999999999998,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.752,"5min":0.86799999999999999,"15min":0.87},"last":0.56999999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.308},"max":{"1min":0.73599999999999999,"5min":0.77900000000000003,"15min":0.78200000000000003},"last":0.502}]},{"osd":6,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51400000000000001,"15min":0.50900000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.35199999999999998},"max":{"1min":0.83399999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.34899999999999998},"max":{"1min":0.69999999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.86099999999999999,"15min":0.92700000000000005},"last":0.60499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575954,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307892,"kb_used_data":2604,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063948,"statfs":{"total":96636764160,"available":96321482752,"internally_reserved":0,"allocated":2666496,"data_stored":1541794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61299999999999999},"min":{"1min":0.46899999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72799999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.625},{"interface":"front","average":{"1min":0.621,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.49099999999999999,"5min":0.371,"15min":0.32100000000000001},"max":{"1min":0.75800000000000001,"5min":1.149,"15min":1.149},"last":0.42899999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.34200000000000003},"max":{"1min":0.76600000000000001,"5min":1.125,"15min":1.125},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.45300000000000001,"5min":0.38400000000000001,"15min":0.35799999999999998},"max":{"1min":0.76300000000000001,"5min":0.83499999999999996,"15min":0.85299999999999998},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.63600000000000001},"min":{"1min":0.53100000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76800000000000002,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60599999999999998,"15min":0.61499999999999999},"min":{"1min":0.46899999999999997,"5min":0.41199999999999998,"15min":0.379},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":1.252},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.42599999999999999},"max":{"1min":0.80200000000000005,"5min":0.83599999999999997,"15min":0.95499999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47099999999999997,"5min":0.46899999999999997,"15min":0.46000000000000002},"max":{"1min":0.81599999999999995,"5min":1.196,"15min":1.196},"last":0.65800000000000003}]},{"osd":4,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.90500000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51100000000000001,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.35299999999999998,"15min":0.32500000000000001},"max":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.84899999999999998},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.497,"15min":0.50600000000000001},"min":{"1min":0.376,"5min":0.33400000000000002,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.82199999999999995,"15min":1.0209999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.40500000000000003,"5min":0.38800000000000001,"15min":0.36699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78900000000000003,"15min":0.98399999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.54700000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.33600000000000002},"max":{"1min":0.751,"5min":0.77900000000000003,"15min":0.86399999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.91700000000000004},"last":0.51700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:30:41.546 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:30:41.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.710+0000 7f7117eac700 1 Processor -- start 2024-01-24T15:30:41.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.710+0000 7f7117eac700 1 -- start start 2024-01-24T15:30:41.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.711+0000 7f7117eac700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7110069240 0x7f7110154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.711+0000 7f7117eac700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711006a3a0 con 0x7f7110150cb0 2024-01-24T15:30:41.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.711+0000 7f7117eac700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7110155290 con 0x7f7110154e40 2024-01-24T15:30:41.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.711+0000 7f7117eac700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71100c2c70 con 0x7f7110069240 2024-01-24T15:30:41.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.711+0000 7f7115c48700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7110150cb0 0x7f7110065e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42008/0 (socket says 172.21.15.29:42008) 2024-01-24T15:30:41.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.711+0000 7f7115c48700 1 -- 172.21.15.29:0/2771890223 learned_addr learned my addr 172.21.15.29:0/2771890223 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:41.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.711+0000 7f7115447700 1 --2- 172.21.15.29:0/2771890223 >> v2:172.21.15.163:3300/0 conn(0x7f7110069240 0x7f7110154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463521553 0 0) 0x7f711006a3a0 con 0x7f7110150cb0 2024-01-24T15:30:41.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70fc0033f0 con 0x7f7110150cb0 2024-01-24T15:30:41.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2776806447 0 0) 0x7f70fc0033f0 con 0x7f7110150cb0 2024-01-24T15:30:41.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 >> v2:172.21.15.163:3300/0 conn(0x7f7110069240 msgr2=0x7f7110154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:41.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 --2- 172.21.15.29:0/2771890223 >> v2:172.21.15.163:3300/0 conn(0x7f7110069240 0x7f7110154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 >> v1:172.21.15.163:6789/0 conn(0x7f7110154e40 legacy=0x7f71100bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7110069820 con 0x7f7110150cb0 2024-01-24T15:30:41.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f710c001470 con 0x7f7110150cb0 2024-01-24T15:30:41.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 >> v1:172.21.15.29:6789/0 conn(0x7f7110150cb0 legacy=0x7f7110065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70fc0038e0 0x7f70fc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70fc004690 0x7f70fc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70fc005000 0x7f70fc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71100c2c70 con 0x7f70fc0038e0 2024-01-24T15:30:41.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc00beb0 con 0x7f70fc004690 2024-01-24T15:30:41.721 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc00c070 con 0x7f70fc005000 2024-01-24T15:30:41.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7115c48700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70fc0038e0 0x7f70fc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7115447700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70fc004690 0x7f70fc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7116449700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70fc005000 0x7f70fc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7115c48700 1 -- 172.21.15.29:0/2771890223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70fc005000 msgr2=0x7f70fc00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:41.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7115c48700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70fc005000 0x7f70fc00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7115c48700 1 -- 172.21.15.29:0/2771890223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70fc004690 msgr2=0x7f70fc004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:41.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7115c48700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70fc004690 0x7f70fc004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7115c48700 1 -- 172.21.15.29:0/2771890223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc0033f0 con 0x7f70fc0038e0 2024-01-24T15:30:41.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7116449700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70fc005000 0x7f70fc00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:41.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7115447700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70fc004690 0x7f70fc004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:41.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.712+0000 7f7116449700 1 -- 172.21.15.29:0/2771890223 reap_dead start 2024-01-24T15:30:41.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.713+0000 7f7115c48700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70fc0038e0 0x7f70fc003d00 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 rx=0x7f710c00c190 tx=0x7f710c001e50).ready entity=mon.0 client_cookie=412abdfab593635a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:41.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.713+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c007a20 con 0x7f70fc0038e0 2024-01-24T15:30:41.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.713+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c00cac0 con 0x7f70fc0038e0 2024-01-24T15:30:41.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.713+0000 7f7114c46700 1 -- 172.21.15.29:0/2771890223 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c00ce40 con 0x7f70fc0038e0 2024-01-24T15:30:41.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.713+0000 7f7117eac700 1 -- 172.21.15.29:0/2771890223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70fc0038e0 msgr2=0x7f70fc003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.713+0000 7f7117eac700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70fc0038e0 0x7f70fc003d00 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 rx=0x7f710c00c190 tx=0x7f710c001e50).stop 2024-01-24T15:30:41.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.713+0000 7f7117eac700 1 -- 172.21.15.29:0/2771890223 shutdown_connections 2024-01-24T15:30:41.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.713+0000 7f7117eac700 1 --2- 172.21.15.29:0/2771890223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70fc0038e0 0x7f70fc003d00 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.713+0000 7f7117eac700 1 -- 172.21.15.29:0/2771890223 >> 172.21.15.29:0/2771890223 conn(0x7f71100c85c0 msgr2=0x7f71100c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:41.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.713+0000 7f7117eac700 1 -- 172.21.15.29:0/2771890223 shutdown_connections 2024-01-24T15:30:41.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.713+0000 7f7117eac700 1 -- 172.21.15.29:0/2771890223 wait complete. 2024-01-24T15:30:41.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.715+0000 7f7117eac700 1 Processor -- start 2024-01-24T15:30:41.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.715+0000 7f7117eac700 1 -- start start 2024-01-24T15:30:41.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.715+0000 7f7117eac700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70fc0038e0 0x7f7110166590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.715+0000 7f7117eac700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f711013e600 0x7f71100c1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.715+0000 7f7117eac700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71100c1800 0x7f7110151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.715+0000 7f7117eac700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71101623f0 con 0x7f71100c1800 2024-01-24T15:30:41.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.715+0000 7f7117eac700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71100c0b00 con 0x7f70fc0038e0 2024-01-24T15:30:41.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.715+0000 7f7117eac700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711013ea80 con 0x7f711013e600 2024-01-24T15:30:41.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.715+0000 7f7116449700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71100c1800 0x7f7110151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7116449700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71100c1800 0x7f7110151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37382/0 (socket says 172.21.15.29:37382) 2024-01-24T15:30:41.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7116449700 1 -- 172.21.15.29:0/3757968077 learned_addr learned my addr 172.21.15.29:0/3757968077 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:41.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7115c48700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70fc0038e0 0x7f7110166590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7115447700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f711013e600 0x7f71100c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7116449700 1 -- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f711013e600 msgr2=0x7f71100c1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7116449700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f711013e600 0x7f71100c1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7116449700 1 -- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70fc0038e0 msgr2=0x7f7110166590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7116449700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70fc0038e0 0x7f7110166590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7116449700 1 -- 172.21.15.29:0/3757968077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71100c0d50 con 0x7f71100c1800 2024-01-24T15:30:41.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7116449700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71100c1800 0x7f7110151090 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7f7110155220 tx=0x7f7108009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:41.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7105ffb700 1 -- 172.21.15.29:0/3757968077 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108002490 con 0x7f71100c1800 2024-01-24T15:30:41.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7105ffb700 1 -- 172.21.15.29:0/3757968077 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71080025f0 con 0x7f71100c1800 2024-01-24T15:30:41.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7117eac700 1 -- 172.21.15.29:0/3757968077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71100c1030 con 0x7f71100c1800 2024-01-24T15:30:41.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.716+0000 7f7105ffb700 1 -- 172.21.15.29:0/3757968077 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108002970 con 0x7f71100c1800 2024-01-24T15:30:41.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.717+0000 7f7117eac700 1 -- 172.21.15.29:0/3757968077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7110151580 con 0x7f71100c1800 2024-01-24T15:30:41.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.717+0000 7f7117eac700 1 -- 172.21.15.29:0/3757968077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71100475e0 con 0x7f71100c1800 2024-01-24T15:30:41.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.718+0000 7f7105ffb700 1 -- 172.21.15.29:0/3757968077 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7108003070 con 0x7f71100c1800 2024-01-24T15:30:41.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.718+0000 7f7105ffb700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f70f405f270 0x7f70f4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:41.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.718+0000 7f7105ffb700 1 -- 172.21.15.29:0/3757968077 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f710807a080 con 0x7f71100c1800 2024-01-24T15:30:41.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.719+0000 7f7115c48700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f70f405f270 0x7f70f4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:41.727 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.719+0000 7f7115c48700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f70f405f270 0x7f70f4061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f71100be230 tx=0x7f710c015000).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:41.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.723+0000 7f7105ffb700 1 -- 172.21.15.29:0/3757968077 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7108049b00 con 0x7f71100c1800 2024-01-24T15:30:41.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.901+0000 7f7117eac700 1 -- 172.21.15.29:0/3757968077 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f711006a5f0 con 0x7f70f405f270 2024-01-24T15:30:41.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.912+0000 7f7105ffb700 1 -- 172.21.15.29:0/3757968077 <== mgr.8938 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361949 (secure 0 0 0) 0x7f711006a5f0 con 0x7f70f405f270 2024-01-24T15:30:41.918 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:41.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.915+0000 7f7117eac700 1 -- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f70f405f270 msgr2=0x7f70f4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.915+0000 7f7117eac700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f70f405f270 0x7f70f4061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f71100be230 tx=0x7f710c015000).stop 2024-01-24T15:30:41.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.915+0000 7f7117eac700 1 -- 172.21.15.29:0/3757968077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71100c1800 msgr2=0x7f7110151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:41.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.915+0000 7f7117eac700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71100c1800 0x7f7110151090 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7f7110155220 tx=0x7f7108009040).stop 2024-01-24T15:30:41.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.915+0000 7f7117eac700 1 -- 172.21.15.29:0/3757968077 shutdown_connections 2024-01-24T15:30:41.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.915+0000 7f7117eac700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71100c1800 0x7f7110151090 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.915+0000 7f7117eac700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70fc0038e0 0x7f7110166590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.915+0000 7f7117eac700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f70f405f270 0x7f70f4061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.915+0000 7f7117eac700 1 --2- 172.21.15.29:0/3757968077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f711013e600 0x7f71100c1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:41.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.915+0000 7f7117eac700 1 -- 172.21.15.29:0/3757968077 >> 172.21.15.29:0/3757968077 conn(0x7f71100c85c0 msgr2=0x7f71100c6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:41.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.915+0000 7f7117eac700 1 -- 172.21.15.29:0/3757968077 shutdown_connections 2024-01-24T15:30:41.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:41.915+0000 7f7117eac700 1 -- 172.21.15.29:0/3757968077 wait complete. 2024-01-24T15:30:41.922 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:30:41.931 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-01-24T15:30:41.643316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2463140,"kb_used_data":20836,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752511580,"statfs":{"total":773094113280,"available":770571857920,"internally_reserved":0,"allocated":21336064,"data_stored":12321905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-602802,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":-209,"num_write_kb":-728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-34,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017536"},"pg_stats":[{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446734+0000","last_change":"2024-01-24T15:30:37.444275+0000","last_active":"2024-01-24T15:30:37.446734+0000","last_peered":"2024-01-24T15:30:37.446734+0000","last_clean":"2024-01-24T15:30:37.446734+0000","last_became_active":"2024-01-24T15:30:37.444036+0000","last_became_peered":"2024-01-24T15:30:37.444036+0000","last_unstale":"2024-01-24T15:30:37.446734+0000","last_undegraded":"2024-01-24T15:30:37.446734+0000","last_fullsized":"2024-01-24T15:30:37.446734+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3f","version":"212'3","reported_seq":13,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444091+0000","last_change":"2024-01-24T15:30:37.010273+0000","last_active":"2024-01-24T15:30:37.444091+0000","last_peered":"2024-01-24T15:30:37.444091+0000","last_clean":"2024-01-24T15:30:37.444091+0000","last_became_active":"2024-01-24T15:30:37.009556+0000","last_became_peered":"2024-01-24T15:30:37.009556+0000","last_unstale":"2024-01-24T15:30:37.444091+0000","last_undegraded":"2024-01-24T15:30:37.444091+0000","last_fullsized":"2024-01-24T15:30:37.444091+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439203+0000","last_change":"2024-01-24T15:30:37.437053+0000","last_active":"2024-01-24T15:30:37.439203+0000","last_peered":"2024-01-24T15:30:37.439203+0000","last_clean":"2024-01-24T15:30:37.439203+0000","last_became_active":"2024-01-24T15:30:37.436635+0000","last_became_peered":"2024-01-24T15:30:37.436635+0000","last_unstale":"2024-01-24T15:30:37.439203+0000","last_undegraded":"2024-01-24T15:30:37.439203+0000","last_fullsized":"2024-01-24T15:30:37.439203+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.427440+0000","last_change":"2024-01-24T15:30:37.006735+0000","last_active":"2024-01-24T15:30:37.427440+0000","last_peered":"2024-01-24T15:30:37.427440+0000","last_clean":"2024-01-24T15:30:37.427440+0000","last_became_active":"2024-01-24T15:30:37.006397+0000","last_became_peered":"2024-01-24T15:30:37.006397+0000","last_unstale":"2024-01-24T15:30:37.427440+0000","last_undegraded":"2024-01-24T15:30:37.427440+0000","last_fullsized":"2024-01-24T15:30:37.427440+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446750+0000","last_change":"2024-01-24T15:30:37.444312+0000","last_active":"2024-01-24T15:30:37.446750+0000","last_peered":"2024-01-24T15:30:37.446750+0000","last_clean":"2024-01-24T15:30:37.446750+0000","last_became_active":"2024-01-24T15:30:37.444022+0000","last_became_peered":"2024-01-24T15:30:37.444022+0000","last_unstale":"2024-01-24T15:30:37.446750+0000","last_undegraded":"2024-01-24T15:30:37.446750+0000","last_fullsized":"2024-01-24T15:30:37.446750+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420562+0000","last_change":"2024-01-24T15:30:37.016546+0000","last_active":"2024-01-24T15:30:37.420562+0000","last_peered":"2024-01-24T15:30:37.420562+0000","last_clean":"2024-01-24T15:30:37.420562+0000","last_became_active":"2024-01-24T15:30:37.016269+0000","last_became_peered":"2024-01-24T15:30:37.016269+0000","last_unstale":"2024-01-24T15:30:37.420562+0000","last_undegraded":"2024-01-24T15:30:37.420562+0000","last_fullsized":"2024-01-24T15:30:37.420562+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.436232+0000","last_change":"2024-01-24T15:30:37.431127+0000","last_active":"2024-01-24T15:30:37.436232+0000","last_peered":"2024-01-24T15:30:37.436232+0000","last_clean":"2024-01-24T15:30:37.436232+0000","last_became_active":"2024-01-24T15:30:37.430816+0000","last_became_peered":"2024-01-24T15:30:37.430816+0000","last_unstale":"2024-01-24T15:30:37.436232+0000","last_undegraded":"2024-01-24T15:30:37.436232+0000","last_fullsized":"2024-01-24T15:30:37.436232+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3c","version":"212'2","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448521+0000","last_change":"2024-01-24T15:30:37.011274+0000","last_active":"2024-01-24T15:30:37.448521+0000","last_peered":"2024-01-24T15:30:37.448521+0000","last_clean":"2024-01-24T15:30:37.448521+0000","last_became_active":"2024-01-24T15:30:37.010993+0000","last_became_peered":"2024-01-24T15:30:37.010993+0000","last_unstale":"2024-01-24T15:30:37.448521+0000","last_undegraded":"2024-01-24T15:30:37.448521+0000","last_fullsized":"2024-01-24T15:30:37.448521+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.437961+0000","last_change":"2024-01-24T15:30:37.432762+0000","last_active":"2024-01-24T15:30:37.437961+0000","last_peered":"2024-01-24T15:30:37.437961+0000","last_clean":"2024-01-24T15:30:37.437961+0000","last_became_active":"2024-01-24T15:30:37.432457+0000","last_became_peered":"2024-01-24T15:30:37.432457+0000","last_unstale":"2024-01-24T15:30:37.437961+0000","last_undegraded":"2024-01-24T15:30:37.437961+0000","last_fullsized":"2024-01-24T15:30:37.437961+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3b","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.441932+0000","last_change":"2024-01-24T15:30:37.013704+0000","last_active":"2024-01-24T15:30:37.441932+0000","last_peered":"2024-01-24T15:30:37.441932+0000","last_clean":"2024-01-24T15:30:37.441932+0000","last_became_active":"2024-01-24T15:30:37.013363+0000","last_became_peered":"2024-01-24T15:30:37.013363+0000","last_unstale":"2024-01-24T15:30:37.441932+0000","last_undegraded":"2024-01-24T15:30:37.441932+0000","last_fullsized":"2024-01-24T15:30:37.441932+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.433763+0000","last_change":"2024-01-24T15:30:37.422209+0000","last_active":"2024-01-24T15:30:37.433763+0000","last_peered":"2024-01-24T15:30:37.433763+0000","last_clean":"2024-01-24T15:30:37.433763+0000","last_became_active":"2024-01-24T15:30:37.421873+0000","last_became_peered":"2024-01-24T15:30:37.421873+0000","last_unstale":"2024-01-24T15:30:37.433763+0000","last_undegraded":"2024-01-24T15:30:37.433763+0000","last_fullsized":"2024-01-24T15:30:37.433763+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3a","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444208+0000","last_change":"2024-01-24T15:30:37.009252+0000","last_active":"2024-01-24T15:30:37.444208+0000","last_peered":"2024-01-24T15:30:37.444208+0000","last_clean":"2024-01-24T15:30:37.444208+0000","last_became_active":"2024-01-24T15:30:37.008966+0000","last_became_peered":"2024-01-24T15:30:37.008966+0000","last_unstale":"2024-01-24T15:30:37.444208+0000","last_undegraded":"2024-01-24T15:30:37.444208+0000","last_fullsized":"2024-01-24T15:30:37.444208+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446295+0000","last_change":"2024-01-24T15:30:37.426413+0000","last_active":"2024-01-24T15:30:37.446295+0000","last_peered":"2024-01-24T15:30:37.446295+0000","last_clean":"2024-01-24T15:30:37.446295+0000","last_became_active":"2024-01-24T15:30:37.426107+0000","last_became_peered":"2024-01-24T15:30:37.426107+0000","last_unstale":"2024-01-24T15:30:37.446295+0000","last_undegraded":"2024-01-24T15:30:37.446295+0000","last_fullsized":"2024-01-24T15:30:37.446295+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425127+0000","last_change":"2024-01-24T15:30:37.020917+0000","last_active":"2024-01-24T15:30:37.425127+0000","last_peered":"2024-01-24T15:30:37.425127+0000","last_clean":"2024-01-24T15:30:37.425127+0000","last_became_active":"2024-01-24T15:30:37.020610+0000","last_became_peered":"2024-01-24T15:30:37.020610+0000","last_unstale":"2024-01-24T15:30:37.425127+0000","last_undegraded":"2024-01-24T15:30:37.425127+0000","last_fullsized":"2024-01-24T15:30:37.425127+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.443942+0000","last_change":"2024-01-24T15:30:37.439074+0000","last_active":"2024-01-24T15:30:37.443942+0000","last_peered":"2024-01-24T15:30:37.443942+0000","last_clean":"2024-01-24T15:30:37.443942+0000","last_became_active":"2024-01-24T15:30:37.438736+0000","last_became_peered":"2024-01-24T15:30:37.438736+0000","last_unstale":"2024-01-24T15:30:37.443942+0000","last_undegraded":"2024-01-24T15:30:37.443942+0000","last_fullsized":"2024-01-24T15:30:37.443942+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425289+0000","last_change":"2024-01-24T15:30:37.005419+0000","last_active":"2024-01-24T15:30:37.425289+0000","last_peered":"2024-01-24T15:30:37.425289+0000","last_clean":"2024-01-24T15:30:37.425289+0000","last_became_active":"2024-01-24T15:30:37.004989+0000","last_became_peered":"2024-01-24T15:30:37.004989+0000","last_unstale":"2024-01-24T15:30:37.425289+0000","last_undegraded":"2024-01-24T15:30:37.425289+0000","last_fullsized":"2024-01-24T15:30:37.425289+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439558+0000","last_change":"2024-01-24T15:30:37.435202+0000","last_active":"2024-01-24T15:30:37.439558+0000","last_peered":"2024-01-24T15:30:37.439558+0000","last_clean":"2024-01-24T15:30:37.439558+0000","last_became_active":"2024-01-24T15:30:37.434876+0000","last_became_peered":"2024-01-24T15:30:37.434876+0000","last_unstale":"2024-01-24T15:30:37.439558+0000","last_undegraded":"2024-01-24T15:30:37.439558+0000","last_fullsized":"2024-01-24T15:30:37.439558+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423953+0000","last_change":"2024-01-24T15:30:37.010943+0000","last_active":"2024-01-24T15:30:37.423953+0000","last_peered":"2024-01-24T15:30:37.423953+0000","last_clean":"2024-01-24T15:30:37.423953+0000","last_became_active":"2024-01-24T15:30:37.010232+0000","last_became_peered":"2024-01-24T15:30:37.010232+0000","last_unstale":"2024-01-24T15:30:37.423953+0000","last_undegraded":"2024-01-24T15:30:37.423953+0000","last_fullsized":"2024-01-24T15:30:37.423953+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439290+0000","last_change":"2024-01-24T15:30:37.429110+0000","last_active":"2024-01-24T15:30:37.439290+0000","last_peered":"2024-01-24T15:30:37.439290+0000","last_clean":"2024-01-24T15:30:37.439290+0000","last_became_active":"2024-01-24T15:30:37.428605+0000","last_became_peered":"2024-01-24T15:30:37.428605+0000","last_unstale":"2024-01-24T15:30:37.439290+0000","last_undegraded":"2024-01-24T15:30:37.439290+0000","last_fullsized":"2024-01-24T15:30:37.439290+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423834+0000","last_change":"2024-01-24T15:30:37.013549+0000","last_active":"2024-01-24T15:30:37.423834+0000","last_peered":"2024-01-24T15:30:37.423834+0000","last_clean":"2024-01-24T15:30:37.423834+0000","last_became_active":"2024-01-24T15:30:37.013199+0000","last_became_peered":"2024-01-24T15:30:37.013199+0000","last_unstale":"2024-01-24T15:30:37.423834+0000","last_undegraded":"2024-01-24T15:30:37.423834+0000","last_fullsized":"2024-01-24T15:30:37.423834+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.438145+0000","last_change":"2024-01-24T15:30:37.431975+0000","last_active":"2024-01-24T15:30:37.438145+0000","last_peered":"2024-01-24T15:30:37.438145+0000","last_clean":"2024-01-24T15:30:37.438145+0000","last_became_active":"2024-01-24T15:30:37.431687+0000","last_became_peered":"2024-01-24T15:30:37.431687+0000","last_unstale":"2024-01-24T15:30:37.438145+0000","last_undegraded":"2024-01-24T15:30:37.438145+0000","last_fullsized":"2024-01-24T15:30:37.438145+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420586+0000","last_change":"2024-01-24T15:30:37.007787+0000","last_active":"2024-01-24T15:30:37.420586+0000","last_peered":"2024-01-24T15:30:37.420586+0000","last_clean":"2024-01-24T15:30:37.420586+0000","last_became_active":"2024-01-24T15:30:37.007488+0000","last_became_peered":"2024-01-24T15:30:37.007488+0000","last_unstale":"2024-01-24T15:30:37.420586+0000","last_undegraded":"2024-01-24T15:30:37.420586+0000","last_fullsized":"2024-01-24T15:30:37.420586+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.443785+0000","last_change":"2024-01-24T15:30:37.438455+0000","last_active":"2024-01-24T15:30:37.443785+0000","last_peered":"2024-01-24T15:30:37.443785+0000","last_clean":"2024-01-24T15:30:37.443785+0000","last_became_active":"2024-01-24T15:30:37.432327+0000","last_became_peered":"2024-01-24T15:30:37.432327+0000","last_unstale":"2024-01-24T15:30:37.443785+0000","last_undegraded":"2024-01-24T15:30:37.443785+0000","last_fullsized":"2024-01-24T15:30:37.443785+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.34","version":"212'4","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.449549+0000","last_change":"2024-01-24T15:30:37.008132+0000","last_active":"2024-01-24T15:30:37.449549+0000","last_peered":"2024-01-24T15:30:37.449549+0000","last_clean":"2024-01-24T15:30:37.449549+0000","last_became_active":"2024-01-24T15:30:37.007824+0000","last_became_peered":"2024-01-24T15:30:37.007824+0000","last_unstale":"2024-01-24T15:30:37.449549+0000","last_undegraded":"2024-01-24T15:30:37.449549+0000","last_fullsized":"2024-01-24T15:30:37.449549+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.440194+0000","last_change":"2024-01-24T15:30:37.437420+0000","last_active":"2024-01-24T15:30:37.440194+0000","last_peered":"2024-01-24T15:30:37.440194+0000","last_clean":"2024-01-24T15:30:37.440194+0000","last_became_active":"2024-01-24T15:30:37.437129+0000","last_became_peered":"2024-01-24T15:30:37.437129+0000","last_unstale":"2024-01-24T15:30:37.440194+0000","last_undegraded":"2024-01-24T15:30:37.440194+0000","last_fullsized":"2024-01-24T15:30:37.440194+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.33","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448433+0000","last_change":"2024-01-24T15:30:37.010090+0000","last_active":"2024-01-24T15:30:37.448433+0000","last_peered":"2024-01-24T15:30:37.448433+0000","last_clean":"2024-01-24T15:30:37.448433+0000","last_became_active":"2024-01-24T15:30:37.009819+0000","last_became_peered":"2024-01-24T15:30:37.009819+0000","last_unstale":"2024-01-24T15:30:37.448433+0000","last_undegraded":"2024-01-24T15:30:37.448433+0000","last_fullsized":"2024-01-24T15:30:37.448433+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446313+0000","last_change":"2024-01-24T15:30:37.442227+0000","last_active":"2024-01-24T15:30:37.446313+0000","last_peered":"2024-01-24T15:30:37.446313+0000","last_clean":"2024-01-24T15:30:37.446313+0000","last_became_active":"2024-01-24T15:30:37.441942+0000","last_became_peered":"2024-01-24T15:30:37.441942+0000","last_unstale":"2024-01-24T15:30:37.446313+0000","last_undegraded":"2024-01-24T15:30:37.446313+0000","last_fullsized":"2024-01-24T15:30:37.446313+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.426637+0000","last_change":"2024-01-24T15:30:37.023501+0000","last_active":"2024-01-24T15:30:37.426637+0000","last_peered":"2024-01-24T15:30:37.426637+0000","last_clean":"2024-01-24T15:30:37.426637+0000","last_became_active":"2024-01-24T15:30:37.023194+0000","last_became_peered":"2024-01-24T15:30:37.023194+0000","last_unstale":"2024-01-24T15:30:37.426637+0000","last_undegraded":"2024-01-24T15:30:37.426637+0000","last_fullsized":"2024-01-24T15:30:37.426637+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446326+0000","last_change":"2024-01-24T15:30:37.427357+0000","last_active":"2024-01-24T15:30:37.446326+0000","last_peered":"2024-01-24T15:30:37.446326+0000","last_clean":"2024-01-24T15:30:37.446326+0000","last_became_active":"2024-01-24T15:30:37.426441+0000","last_became_peered":"2024-01-24T15:30:37.426441+0000","last_unstale":"2024-01-24T15:30:37.446326+0000","last_undegraded":"2024-01-24T15:30:37.446326+0000","last_fullsized":"2024-01-24T15:30:37.446326+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419347+0000","last_change":"2024-01-24T15:30:37.013635+0000","last_active":"2024-01-24T15:30:37.419347+0000","last_peered":"2024-01-24T15:30:37.419347+0000","last_clean":"2024-01-24T15:30:37.419347+0000","last_became_active":"2024-01-24T15:30:37.013207+0000","last_became_peered":"2024-01-24T15:30:37.013207+0000","last_unstale":"2024-01-24T15:30:37.419347+0000","last_undegraded":"2024-01-24T15:30:37.419347+0000","last_fullsized":"2024-01-24T15:30:37.419347+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439504+0000","last_change":"2024-01-24T15:30:37.436058+0000","last_active":"2024-01-24T15:30:37.439504+0000","last_peered":"2024-01-24T15:30:37.439504+0000","last_clean":"2024-01-24T15:30:37.439504+0000","last_became_active":"2024-01-24T15:30:37.435693+0000","last_became_peered":"2024-01-24T15:30:37.435693+0000","last_unstale":"2024-01-24T15:30:37.439504+0000","last_undegraded":"2024-01-24T15:30:37.439504+0000","last_fullsized":"2024-01-24T15:30:37.439504+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.30","version":"212'4","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.449754+0000","last_change":"2024-01-24T15:30:37.012073+0000","last_active":"2024-01-24T15:30:37.449754+0000","last_peered":"2024-01-24T15:30:37.449754+0000","last_clean":"2024-01-24T15:30:37.449754+0000","last_became_active":"2024-01-24T15:30:37.011589+0000","last_became_peered":"2024-01-24T15:30:37.011589+0000","last_unstale":"2024-01-24T15:30:37.449754+0000","last_undegraded":"2024-01-24T15:30:37.449754+0000","last_fullsized":"2024-01-24T15:30:37.449754+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.443656+0000","last_change":"2024-01-24T15:30:37.440683+0000","last_active":"2024-01-24T15:30:37.443656+0000","last_peered":"2024-01-24T15:30:37.443656+0000","last_clean":"2024-01-24T15:30:37.443656+0000","last_became_active":"2024-01-24T15:30:37.440366+0000","last_became_peered":"2024-01-24T15:30:37.440366+0000","last_unstale":"2024-01-24T15:30:37.443656+0000","last_undegraded":"2024-01-24T15:30:37.443656+0000","last_fullsized":"2024-01-24T15:30:37.443656+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.424060+0000","last_change":"2024-01-24T15:30:37.011161+0000","last_active":"2024-01-24T15:30:37.424060+0000","last_peered":"2024-01-24T15:30:37.424060+0000","last_clean":"2024-01-24T15:30:37.424060+0000","last_became_active":"2024-01-24T15:30:37.010595+0000","last_became_peered":"2024-01-24T15:30:37.010595+0000","last_unstale":"2024-01-24T15:30:37.424060+0000","last_undegraded":"2024-01-24T15:30:37.424060+0000","last_fullsized":"2024-01-24T15:30:37.424060+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444316+0000","last_change":"2024-01-24T15:30:37.434050+0000","last_active":"2024-01-24T15:30:37.444316+0000","last_peered":"2024-01-24T15:30:37.444316+0000","last_clean":"2024-01-24T15:30:37.444316+0000","last_became_active":"2024-01-24T15:30:37.433763+0000","last_became_peered":"2024-01-24T15:30:37.433763+0000","last_unstale":"2024-01-24T15:30:37.444316+0000","last_undegraded":"2024-01-24T15:30:37.444316+0000","last_fullsized":"2024-01-24T15:30:37.444316+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423220+0000","last_change":"2024-01-24T15:30:37.014740+0000","last_active":"2024-01-24T15:30:37.423220+0000","last_peered":"2024-01-24T15:30:37.423220+0000","last_clean":"2024-01-24T15:30:37.423220+0000","last_became_active":"2024-01-24T15:30:37.014450+0000","last_became_peered":"2024-01-24T15:30:37.014450+0000","last_unstale":"2024-01-24T15:30:37.423220+0000","last_undegraded":"2024-01-24T15:30:37.423220+0000","last_fullsized":"2024-01-24T15:30:37.423220+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444322+0000","last_change":"2024-01-24T15:30:37.433732+0000","last_active":"2024-01-24T15:30:37.444322+0000","last_peered":"2024-01-24T15:30:37.444322+0000","last_clean":"2024-01-24T15:30:37.444322+0000","last_became_active":"2024-01-24T15:30:37.433457+0000","last_became_peered":"2024-01-24T15:30:37.433457+0000","last_unstale":"2024-01-24T15:30:37.444322+0000","last_undegraded":"2024-01-24T15:30:37.444322+0000","last_fullsized":"2024-01-24T15:30:37.444322+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2d","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.440198+0000","last_change":"2024-01-24T15:30:37.011775+0000","last_active":"2024-01-24T15:30:37.440198+0000","last_peered":"2024-01-24T15:30:37.440198+0000","last_clean":"2024-01-24T15:30:37.440198+0000","last_became_active":"2024-01-24T15:30:37.011456+0000","last_became_peered":"2024-01-24T15:30:37.011456+0000","last_unstale":"2024-01-24T15:30:37.440198+0000","last_undegraded":"2024-01-24T15:30:37.440198+0000","last_fullsized":"2024-01-24T15:30:37.440198+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.441166+0000","last_change":"2024-01-24T15:30:37.438868+0000","last_active":"2024-01-24T15:30:37.441166+0000","last_peered":"2024-01-24T15:30:37.441166+0000","last_clean":"2024-01-24T15:30:37.441166+0000","last_became_active":"2024-01-24T15:30:37.438588+0000","last_became_peered":"2024-01-24T15:30:37.438588+0000","last_unstale":"2024-01-24T15:30:37.441166+0000","last_undegraded":"2024-01-24T15:30:37.441166+0000","last_fullsized":"2024-01-24T15:30:37.441166+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.416885+0000","last_change":"2024-01-24T15:30:37.006475+0000","last_active":"2024-01-24T15:30:37.416885+0000","last_peered":"2024-01-24T15:30:37.416885+0000","last_clean":"2024-01-24T15:30:37.416885+0000","last_became_active":"2024-01-24T15:30:37.006131+0000","last_became_peered":"2024-01-24T15:30:37.006131+0000","last_unstale":"2024-01-24T15:30:37.416885+0000","last_undegraded":"2024-01-24T15:30:37.416885+0000","last_fullsized":"2024-01-24T15:30:37.416885+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448917+0000","last_change":"2024-01-24T15:30:37.447503+0000","last_active":"2024-01-24T15:30:37.448917+0000","last_peered":"2024-01-24T15:30:37.448917+0000","last_clean":"2024-01-24T15:30:37.448917+0000","last_became_active":"2024-01-24T15:30:37.447214+0000","last_became_peered":"2024-01-24T15:30:37.447214+0000","last_unstale":"2024-01-24T15:30:37.448917+0000","last_undegraded":"2024-01-24T15:30:37.448917+0000","last_fullsized":"2024-01-24T15:30:37.448917+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.2b","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448502+0000","last_change":"2024-01-24T15:30:37.007302+0000","last_active":"2024-01-24T15:30:37.448502+0000","last_peered":"2024-01-24T15:30:37.448502+0000","last_clean":"2024-01-24T15:30:37.448502+0000","last_became_active":"2024-01-24T15:30:37.007009+0000","last_became_peered":"2024-01-24T15:30:37.007009+0000","last_unstale":"2024-01-24T15:30:37.448502+0000","last_undegraded":"2024-01-24T15:30:37.448502+0000","last_fullsized":"2024-01-24T15:30:37.448502+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444349+0000","last_change":"2024-01-24T15:30:37.433667+0000","last_active":"2024-01-24T15:30:37.444349+0000","last_peered":"2024-01-24T15:30:37.444349+0000","last_clean":"2024-01-24T15:30:37.444349+0000","last_became_active":"2024-01-24T15:30:37.433352+0000","last_became_peered":"2024-01-24T15:30:37.433352+0000","last_unstale":"2024-01-24T15:30:37.444349+0000","last_undegraded":"2024-01-24T15:30:37.444349+0000","last_fullsized":"2024-01-24T15:30:37.444349+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.424357+0000","last_change":"2024-01-24T15:30:37.013357+0000","last_active":"2024-01-24T15:30:37.424357+0000","last_peered":"2024-01-24T15:30:37.424357+0000","last_clean":"2024-01-24T15:30:37.424357+0000","last_became_active":"2024-01-24T15:30:37.013084+0000","last_became_peered":"2024-01-24T15:30:37.013084+0000","last_unstale":"2024-01-24T15:30:37.424357+0000","last_undegraded":"2024-01-24T15:30:37.424357+0000","last_fullsized":"2024-01-24T15:30:37.424357+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448720+0000","last_change":"2024-01-24T15:30:37.431843+0000","last_active":"2024-01-24T15:30:37.448720+0000","last_peered":"2024-01-24T15:30:37.448720+0000","last_clean":"2024-01-24T15:30:37.448720+0000","last_became_active":"2024-01-24T15:30:37.431536+0000","last_became_peered":"2024-01-24T15:30:37.431536+0000","last_unstale":"2024-01-24T15:30:37.448720+0000","last_undegraded":"2024-01-24T15:30:37.448720+0000","last_fullsized":"2024-01-24T15:30:37.448720+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425112+0000","last_change":"2024-01-24T15:30:37.014302+0000","last_active":"2024-01-24T15:30:37.425112+0000","last_peered":"2024-01-24T15:30:37.425112+0000","last_clean":"2024-01-24T15:30:37.425112+0000","last_became_active":"2024-01-24T15:30:37.013985+0000","last_became_peered":"2024-01-24T15:30:37.013985+0000","last_unstale":"2024-01-24T15:30:37.425112+0000","last_undegraded":"2024-01-24T15:30:37.425112+0000","last_fullsized":"2024-01-24T15:30:37.425112+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.437434+0000","last_change":"2024-01-24T15:30:37.436637+0000","last_active":"2024-01-24T15:30:37.437434+0000","last_peered":"2024-01-24T15:30:37.437434+0000","last_clean":"2024-01-24T15:30:37.437434+0000","last_became_active":"2024-01-24T15:30:37.436352+0000","last_became_peered":"2024-01-24T15:30:37.436352+0000","last_unstale":"2024-01-24T15:30:37.437434+0000","last_undegraded":"2024-01-24T15:30:37.437434+0000","last_fullsized":"2024-01-24T15:30:37.437434+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.417111+0000","last_change":"2024-01-24T15:30:37.011770+0000","last_active":"2024-01-24T15:30:37.417111+0000","last_peered":"2024-01-24T15:30:37.417111+0000","last_clean":"2024-01-24T15:30:37.417111+0000","last_became_active":"2024-01-24T15:30:37.011249+0000","last_became_peered":"2024-01-24T15:30:37.011249+0000","last_unstale":"2024-01-24T15:30:37.417111+0000","last_undegraded":"2024-01-24T15:30:37.417111+0000","last_fullsized":"2024-01-24T15:30:37.417111+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.443836+0000","last_change":"2024-01-24T15:30:37.439059+0000","last_active":"2024-01-24T15:30:37.443836+0000","last_peered":"2024-01-24T15:30:37.443836+0000","last_clean":"2024-01-24T15:30:37.443836+0000","last_became_active":"2024-01-24T15:30:37.438774+0000","last_became_peered":"2024-01-24T15:30:37.438774+0000","last_unstale":"2024-01-24T15:30:37.443836+0000","last_undegraded":"2024-01-24T15:30:37.443836+0000","last_fullsized":"2024-01-24T15:30:37.443836+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420163+0000","last_change":"2024-01-24T15:30:37.008070+0000","last_active":"2024-01-24T15:30:37.420163+0000","last_peered":"2024-01-24T15:30:37.420163+0000","last_clean":"2024-01-24T15:30:37.420163+0000","last_became_active":"2024-01-24T15:30:37.007541+0000","last_became_peered":"2024-01-24T15:30:37.007541+0000","last_unstale":"2024-01-24T15:30:37.420163+0000","last_undegraded":"2024-01-24T15:30:37.420163+0000","last_fullsized":"2024-01-24T15:30:37.420163+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.441336+0000","last_change":"2024-01-24T15:30:37.437781+0000","last_active":"2024-01-24T15:30:37.441336+0000","last_peered":"2024-01-24T15:30:37.441336+0000","last_clean":"2024-01-24T15:30:37.441336+0000","last_became_active":"2024-01-24T15:30:37.437475+0000","last_became_peered":"2024-01-24T15:30:37.437475+0000","last_unstale":"2024-01-24T15:30:37.441336+0000","last_undegraded":"2024-01-24T15:30:37.441336+0000","last_fullsized":"2024-01-24T15:30:37.441336+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.417035+0000","last_change":"2024-01-24T15:30:37.002395+0000","last_active":"2024-01-24T15:30:37.417035+0000","last_peered":"2024-01-24T15:30:37.417035+0000","last_clean":"2024-01-24T15:30:37.417035+0000","last_became_active":"2024-01-24T15:30:37.002056+0000","last_became_peered":"2024-01-24T15:30:37.002056+0000","last_unstale":"2024-01-24T15:30:37.417035+0000","last_undegraded":"2024-01-24T15:30:37.417035+0000","last_fullsized":"2024-01-24T15:30:37.417035+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444355+0000","last_change":"2024-01-24T15:30:37.435232+0000","last_active":"2024-01-24T15:30:37.444355+0000","last_peered":"2024-01-24T15:30:37.444355+0000","last_clean":"2024-01-24T15:30:37.444355+0000","last_became_active":"2024-01-24T15:30:37.434904+0000","last_became_peered":"2024-01-24T15:30:37.434904+0000","last_unstale":"2024-01-24T15:30:37.444355+0000","last_undegraded":"2024-01-24T15:30:37.444355+0000","last_fullsized":"2024-01-24T15:30:37.444355+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.426737+0000","last_change":"2024-01-24T15:30:37.017404+0000","last_active":"2024-01-24T15:30:37.426737+0000","last_peered":"2024-01-24T15:30:37.426737+0000","last_clean":"2024-01-24T15:30:37.426737+0000","last_became_active":"2024-01-24T15:30:37.017086+0000","last_became_peered":"2024-01-24T15:30:37.017086+0000","last_unstale":"2024-01-24T15:30:37.426737+0000","last_undegraded":"2024-01-24T15:30:37.426737+0000","last_fullsized":"2024-01-24T15:30:37.426737+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.436983+0000","last_change":"2024-01-24T15:30:37.433013+0000","last_active":"2024-01-24T15:30:37.436983+0000","last_peered":"2024-01-24T15:30:37.436983+0000","last_clean":"2024-01-24T15:30:37.436983+0000","last_became_active":"2024-01-24T15:30:37.432700+0000","last_became_peered":"2024-01-24T15:30:37.432700+0000","last_unstale":"2024-01-24T15:30:37.436983+0000","last_undegraded":"2024-01-24T15:30:37.436983+0000","last_fullsized":"2024-01-24T15:30:37.436983+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419797+0000","last_change":"2024-01-24T15:30:37.011691+0000","last_active":"2024-01-24T15:30:37.419797+0000","last_peered":"2024-01-24T15:30:37.419797+0000","last_clean":"2024-01-24T15:30:37.419797+0000","last_became_active":"2024-01-24T15:30:37.011242+0000","last_became_peered":"2024-01-24T15:30:37.011242+0000","last_unstale":"2024-01-24T15:30:37.419797+0000","last_undegraded":"2024-01-24T15:30:37.419797+0000","last_fullsized":"2024-01-24T15:30:37.419797+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439731+0000","last_change":"2024-01-24T15:30:37.431586+0000","last_active":"2024-01-24T15:30:37.439731+0000","last_peered":"2024-01-24T15:30:37.439731+0000","last_clean":"2024-01-24T15:30:37.439731+0000","last_became_active":"2024-01-24T15:30:37.431292+0000","last_became_peered":"2024-01-24T15:30:37.431292+0000","last_unstale":"2024-01-24T15:30:37.439731+0000","last_undegraded":"2024-01-24T15:30:37.439731+0000","last_fullsized":"2024-01-24T15:30:37.439731+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.427681+0000","last_change":"2024-01-24T15:30:37.015085+0000","last_active":"2024-01-24T15:30:37.427681+0000","last_peered":"2024-01-24T15:30:37.427681+0000","last_clean":"2024-01-24T15:30:37.427681+0000","last_became_active":"2024-01-24T15:30:37.013725+0000","last_became_peered":"2024-01-24T15:30:37.013725+0000","last_unstale":"2024-01-24T15:30:37.427681+0000","last_undegraded":"2024-01-24T15:30:37.427681+0000","last_fullsized":"2024-01-24T15:30:37.427681+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439773+0000","last_change":"2024-01-24T15:30:37.436446+0000","last_active":"2024-01-24T15:30:37.439773+0000","last_peered":"2024-01-24T15:30:37.439773+0000","last_clean":"2024-01-24T15:30:37.439773+0000","last_became_active":"2024-01-24T15:30:37.435849+0000","last_became_peered":"2024-01-24T15:30:37.435849+0000","last_unstale":"2024-01-24T15:30:37.439773+0000","last_undegraded":"2024-01-24T15:30:37.439773+0000","last_fullsized":"2024-01-24T15:30:37.439773+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.422248+0000","last_change":"2024-01-24T15:30:37.017515+0000","last_active":"2024-01-24T15:30:37.422248+0000","last_peered":"2024-01-24T15:30:37.422248+0000","last_clean":"2024-01-24T15:30:37.422248+0000","last_became_active":"2024-01-24T15:30:37.017340+0000","last_became_peered":"2024-01-24T15:30:37.017340+0000","last_unstale":"2024-01-24T15:30:37.422248+0000","last_undegraded":"2024-01-24T15:30:37.422248+0000","last_fullsized":"2024-01-24T15:30:37.422248+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.434262+0000","last_change":"2024-01-24T15:30:37.425592+0000","last_active":"2024-01-24T15:30:37.434262+0000","last_peered":"2024-01-24T15:30:37.434262+0000","last_clean":"2024-01-24T15:30:37.434262+0000","last_became_active":"2024-01-24T15:30:37.425302+0000","last_became_peered":"2024-01-24T15:30:37.425302+0000","last_unstale":"2024-01-24T15:30:37.434262+0000","last_undegraded":"2024-01-24T15:30:37.434262+0000","last_fullsized":"2024-01-24T15:30:37.434262+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419477+0000","last_change":"2024-01-24T15:30:37.013451+0000","last_active":"2024-01-24T15:30:37.419477+0000","last_peered":"2024-01-24T15:30:37.419477+0000","last_clean":"2024-01-24T15:30:37.419477+0000","last_became_active":"2024-01-24T15:30:37.012903+0000","last_became_peered":"2024-01-24T15:30:37.012903+0000","last_unstale":"2024-01-24T15:30:37.419477+0000","last_undegraded":"2024-01-24T15:30:37.419477+0000","last_fullsized":"2024-01-24T15:30:37.419477+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.435385+0000","last_change":"2024-01-24T15:30:37.430607+0000","last_active":"2024-01-24T15:30:37.435385+0000","last_peered":"2024-01-24T15:30:37.435385+0000","last_clean":"2024-01-24T15:30:37.435385+0000","last_became_active":"2024-01-24T15:30:37.430270+0000","last_became_peered":"2024-01-24T15:30:37.430270+0000","last_unstale":"2024-01-24T15:30:37.435385+0000","last_undegraded":"2024-01-24T15:30:37.435385+0000","last_fullsized":"2024-01-24T15:30:37.435385+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.422030+0000","last_change":"2024-01-24T15:30:37.004925+0000","last_active":"2024-01-24T15:30:37.422030+0000","last_peered":"2024-01-24T15:30:37.422030+0000","last_clean":"2024-01-24T15:30:37.422030+0000","last_became_active":"2024-01-24T15:30:37.004638+0000","last_became_peered":"2024-01-24T15:30:37.004638+0000","last_unstale":"2024-01-24T15:30:37.422030+0000","last_undegraded":"2024-01-24T15:30:37.422030+0000","last_fullsized":"2024-01-24T15:30:37.422030+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.452067+0000","last_change":"2024-01-24T15:30:37.447427+0000","last_active":"2024-01-24T15:30:37.452067+0000","last_peered":"2024-01-24T15:30:37.452067+0000","last_clean":"2024-01-24T15:30:37.452067+0000","last_became_active":"2024-01-24T15:30:37.447142+0000","last_became_peered":"2024-01-24T15:30:37.447142+0000","last_unstale":"2024-01-24T15:30:37.452067+0000","last_undegraded":"2024-01-24T15:30:37.452067+0000","last_fullsized":"2024-01-24T15:30:37.452067+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420378+0000","last_change":"2024-01-24T15:30:37.005241+0000","last_active":"2024-01-24T15:30:37.420378+0000","last_peered":"2024-01-24T15:30:37.420378+0000","last_clean":"2024-01-24T15:30:37.420378+0000","last_became_active":"2024-01-24T15:30:37.004783+0000","last_became_peered":"2024-01-24T15:30:37.004783+0000","last_unstale":"2024-01-24T15:30:37.420378+0000","last_undegraded":"2024-01-24T15:30:37.420378+0000","last_fullsized":"2024-01-24T15:30:37.420378+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.449454+0000","last_change":"2024-01-24T15:30:37.441027+0000","last_active":"2024-01-24T15:30:37.449454+0000","last_peered":"2024-01-24T15:30:37.449454+0000","last_clean":"2024-01-24T15:30:37.449454+0000","last_became_active":"2024-01-24T15:30:37.440698+0000","last_became_peered":"2024-01-24T15:30:37.440698+0000","last_unstale":"2024-01-24T15:30:37.449454+0000","last_undegraded":"2024-01-24T15:30:37.449454+0000","last_fullsized":"2024-01-24T15:30:37.449454+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.417176+0000","last_change":"2024-01-24T15:30:37.013248+0000","last_active":"2024-01-24T15:30:37.417176+0000","last_peered":"2024-01-24T15:30:37.417176+0000","last_clean":"2024-01-24T15:30:37.417176+0000","last_became_active":"2024-01-24T15:30:37.012908+0000","last_became_peered":"2024-01-24T15:30:37.012908+0000","last_unstale":"2024-01-24T15:30:37.417176+0000","last_undegraded":"2024-01-24T15:30:37.417176+0000","last_fullsized":"2024-01-24T15:30:37.417176+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.430541+0000","last_change":"2024-01-24T15:30:37.425489+0000","last_active":"2024-01-24T15:30:37.430541+0000","last_peered":"2024-01-24T15:30:37.430541+0000","last_clean":"2024-01-24T15:30:37.430541+0000","last_became_active":"2024-01-24T15:30:37.425212+0000","last_became_peered":"2024-01-24T15:30:37.425212+0000","last_unstale":"2024-01-24T15:30:37.430541+0000","last_undegraded":"2024-01-24T15:30:37.430541+0000","last_fullsized":"2024-01-24T15:30:37.430541+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"212'3","reported_seq":13,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448965+0000","last_change":"2024-01-24T15:30:36.999392+0000","last_active":"2024-01-24T15:30:37.448965+0000","last_peered":"2024-01-24T15:30:37.448965+0000","last_clean":"2024-01-24T15:30:37.448965+0000","last_became_active":"2024-01-24T15:30:36.999152+0000","last_became_peered":"2024-01-24T15:30:36.999152+0000","last_unstale":"2024-01-24T15:30:37.448965+0000","last_undegraded":"2024-01-24T15:30:37.448965+0000","last_fullsized":"2024-01-24T15:30:37.448965+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.450210+0000","last_change":"2024-01-24T15:30:37.449016+0000","last_active":"2024-01-24T15:30:37.450210+0000","last_peered":"2024-01-24T15:30:37.450210+0000","last_clean":"2024-01-24T15:30:37.450210+0000","last_became_active":"2024-01-24T15:30:37.448727+0000","last_became_peered":"2024-01-24T15:30:37.448727+0000","last_unstale":"2024-01-24T15:30:37.450210+0000","last_undegraded":"2024-01-24T15:30:37.450210+0000","last_fullsized":"2024-01-24T15:30:37.450210+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"212'2","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.445026+0000","last_change":"2024-01-24T15:30:37.009875+0000","last_active":"2024-01-24T15:30:37.445026+0000","last_peered":"2024-01-24T15:30:37.445026+0000","last_clean":"2024-01-24T15:30:37.445026+0000","last_became_active":"2024-01-24T15:30:37.009538+0000","last_became_peered":"2024-01-24T15:30:37.009538+0000","last_unstale":"2024-01-24T15:30:37.445026+0000","last_undegraded":"2024-01-24T15:30:37.445026+0000","last_fullsized":"2024-01-24T15:30:37.445026+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439676+0000","last_change":"2024-01-24T15:30:37.437804+0000","last_active":"2024-01-24T15:30:37.439676+0000","last_peered":"2024-01-24T15:30:37.439676+0000","last_clean":"2024-01-24T15:30:37.439676+0000","last_became_active":"2024-01-24T15:30:37.437531+0000","last_became_peered":"2024-01-24T15:30:37.437531+0000","last_unstale":"2024-01-24T15:30:37.439676+0000","last_undegraded":"2024-01-24T15:30:37.439676+0000","last_fullsized":"2024-01-24T15:30:37.439676+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.449259+0000","last_change":"2024-01-24T15:30:37.010627+0000","last_active":"2024-01-24T15:30:37.449259+0000","last_peered":"2024-01-24T15:30:37.449259+0000","last_clean":"2024-01-24T15:30:37.449259+0000","last_became_active":"2024-01-24T15:30:37.010267+0000","last_became_peered":"2024-01-24T15:30:37.010267+0000","last_unstale":"2024-01-24T15:30:37.449259+0000","last_undegraded":"2024-01-24T15:30:37.449259+0000","last_fullsized":"2024-01-24T15:30:37.449259+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.441633+0000","last_change":"2024-01-24T15:30:37.436591+0000","last_active":"2024-01-24T15:30:37.441633+0000","last_peered":"2024-01-24T15:30:37.441633+0000","last_clean":"2024-01-24T15:30:37.441633+0000","last_became_active":"2024-01-24T15:30:37.436324+0000","last_became_peered":"2024-01-24T15:30:37.436324+0000","last_unstale":"2024-01-24T15:30:37.441633+0000","last_undegraded":"2024-01-24T15:30:37.441633+0000","last_fullsized":"2024-01-24T15:30:37.441633+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425787+0000","last_change":"2024-01-24T15:30:37.023045+0000","last_active":"2024-01-24T15:30:37.425787+0000","last_peered":"2024-01-24T15:30:37.425787+0000","last_clean":"2024-01-24T15:30:37.425787+0000","last_became_active":"2024-01-24T15:30:37.022693+0000","last_became_peered":"2024-01-24T15:30:37.022693+0000","last_unstale":"2024-01-24T15:30:37.425787+0000","last_undegraded":"2024-01-24T15:30:37.425787+0000","last_fullsized":"2024-01-24T15:30:37.425787+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.440121+0000","last_change":"2024-01-24T15:30:37.433914+0000","last_active":"2024-01-24T15:30:37.440121+0000","last_peered":"2024-01-24T15:30:37.440121+0000","last_clean":"2024-01-24T15:30:37.440121+0000","last_became_active":"2024-01-24T15:30:37.433647+0000","last_became_peered":"2024-01-24T15:30:37.433647+0000","last_unstale":"2024-01-24T15:30:37.440121+0000","last_undegraded":"2024-01-24T15:30:37.440121+0000","last_fullsized":"2024-01-24T15:30:37.440121+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.418880+0000","last_change":"2024-01-24T15:30:37.010985+0000","last_active":"2024-01-24T15:30:37.418880+0000","last_peered":"2024-01-24T15:30:37.418880+0000","last_clean":"2024-01-24T15:30:37.418880+0000","last_became_active":"2024-01-24T15:30:37.010679+0000","last_became_peered":"2024-01-24T15:30:37.010679+0000","last_unstale":"2024-01-24T15:30:37.418880+0000","last_undegraded":"2024-01-24T15:30:37.418880+0000","last_fullsized":"2024-01-24T15:30:37.418880+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.441848+0000","last_change":"2024-01-24T15:30:37.441271+0000","last_active":"2024-01-24T15:30:37.441848+0000","last_peered":"2024-01-24T15:30:37.441848+0000","last_clean":"2024-01-24T15:30:37.441848+0000","last_became_active":"2024-01-24T15:30:37.440093+0000","last_became_peered":"2024-01-24T15:30:37.440093+0000","last_unstale":"2024-01-24T15:30:37.441848+0000","last_undegraded":"2024-01-24T15:30:37.441848+0000","last_fullsized":"2024-01-24T15:30:37.441848+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"212'4","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.450180+0000","last_change":"2024-01-24T15:30:37.011917+0000","last_active":"2024-01-24T15:30:37.450180+0000","last_peered":"2024-01-24T15:30:37.450180+0000","last_clean":"2024-01-24T15:30:37.450180+0000","last_became_active":"2024-01-24T15:30:37.011638+0000","last_became_peered":"2024-01-24T15:30:37.011638+0000","last_unstale":"2024-01-24T15:30:37.450180+0000","last_undegraded":"2024-01-24T15:30:37.450180+0000","last_fullsized":"2024-01-24T15:30:37.450180+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.441484+0000","last_change":"2024-01-24T15:30:37.439809+0000","last_active":"2024-01-24T15:30:37.441484+0000","last_peered":"2024-01-24T15:30:37.441484+0000","last_clean":"2024-01-24T15:30:37.441484+0000","last_became_active":"2024-01-24T15:30:37.439397+0000","last_became_peered":"2024-01-24T15:30:37.439397+0000","last_unstale":"2024-01-24T15:30:37.441484+0000","last_undegraded":"2024-01-24T15:30:37.441484+0000","last_fullsized":"2024-01-24T15:30:37.441484+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444386+0000","last_change":"2024-01-24T15:30:37.007051+0000","last_active":"2024-01-24T15:30:37.444386+0000","last_peered":"2024-01-24T15:30:37.444386+0000","last_clean":"2024-01-24T15:30:37.444386+0000","last_became_active":"2024-01-24T15:30:37.006421+0000","last_became_peered":"2024-01-24T15:30:37.006421+0000","last_unstale":"2024-01-24T15:30:37.444386+0000","last_undegraded":"2024-01-24T15:30:37.444386+0000","last_fullsized":"2024-01-24T15:30:37.444386+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.440140+0000","last_change":"2024-01-24T15:30:37.435862+0000","last_active":"2024-01-24T15:30:37.440140+0000","last_peered":"2024-01-24T15:30:37.440140+0000","last_clean":"2024-01-24T15:30:37.440140+0000","last_became_active":"2024-01-24T15:30:37.435556+0000","last_became_peered":"2024-01-24T15:30:37.435556+0000","last_unstale":"2024-01-24T15:30:37.440140+0000","last_undegraded":"2024-01-24T15:30:37.440140+0000","last_fullsized":"2024-01-24T15:30:37.440140+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"212'2","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.449367+0000","last_change":"2024-01-24T15:30:37.008278+0000","last_active":"2024-01-24T15:30:37.449367+0000","last_peered":"2024-01-24T15:30:37.449367+0000","last_clean":"2024-01-24T15:30:37.449367+0000","last_became_active":"2024-01-24T15:30:37.008023+0000","last_became_peered":"2024-01-24T15:30:37.008023+0000","last_unstale":"2024-01-24T15:30:37.449367+0000","last_undegraded":"2024-01-24T15:30:37.449367+0000","last_fullsized":"2024-01-24T15:30:37.449367+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439812+0000","last_change":"2024-01-24T15:30:37.432885+0000","last_active":"2024-01-24T15:30:37.439812+0000","last_peered":"2024-01-24T15:30:37.439812+0000","last_clean":"2024-01-24T15:30:37.439812+0000","last_became_active":"2024-01-24T15:30:37.432563+0000","last_became_peered":"2024-01-24T15:30:37.432563+0000","last_unstale":"2024-01-24T15:30:37.439812+0000","last_undegraded":"2024-01-24T15:30:37.439812+0000","last_fullsized":"2024-01-24T15:30:37.439812+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.426843+0000","last_change":"2024-01-24T15:30:37.012298+0000","last_active":"2024-01-24T15:30:37.426843+0000","last_peered":"2024-01-24T15:30:37.426843+0000","last_clean":"2024-01-24T15:30:37.426843+0000","last_became_active":"2024-01-24T15:30:37.012015+0000","last_became_peered":"2024-01-24T15:30:37.012015+0000","last_unstale":"2024-01-24T15:30:37.426843+0000","last_undegraded":"2024-01-24T15:30:37.426843+0000","last_fullsized":"2024-01-24T15:30:37.426843+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446774+0000","last_change":"2024-01-24T15:30:37.445997+0000","last_active":"2024-01-24T15:30:37.446774+0000","last_peered":"2024-01-24T15:30:37.446774+0000","last_clean":"2024-01-24T15:30:37.446774+0000","last_became_active":"2024-01-24T15:30:37.445713+0000","last_became_peered":"2024-01-24T15:30:37.445713+0000","last_unstale":"2024-01-24T15:30:37.446774+0000","last_undegraded":"2024-01-24T15:30:37.446774+0000","last_fullsized":"2024-01-24T15:30:37.446774+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.421185+0000","last_change":"2024-01-24T15:30:37.008994+0000","last_active":"2024-01-24T15:30:37.421185+0000","last_peered":"2024-01-24T15:30:37.421185+0000","last_clean":"2024-01-24T15:30:37.421185+0000","last_became_active":"2024-01-24T15:30:37.008656+0000","last_became_peered":"2024-01-24T15:30:37.008656+0000","last_unstale":"2024-01-24T15:30:37.421185+0000","last_undegraded":"2024-01-24T15:30:37.421185+0000","last_fullsized":"2024-01-24T15:30:37.421185+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.447986+0000","last_change":"2024-01-24T15:30:37.446107+0000","last_active":"2024-01-24T15:30:37.447986+0000","last_peered":"2024-01-24T15:30:37.447986+0000","last_clean":"2024-01-24T15:30:37.447986+0000","last_became_active":"2024-01-24T15:30:37.445877+0000","last_became_peered":"2024-01-24T15:30:37.445877+0000","last_unstale":"2024-01-24T15:30:37.447986+0000","last_undegraded":"2024-01-24T15:30:37.447986+0000","last_fullsized":"2024-01-24T15:30:37.447986+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444715+0000","last_change":"2024-01-24T15:30:37.023023+0000","last_active":"2024-01-24T15:30:37.444715+0000","last_peered":"2024-01-24T15:30:37.444715+0000","last_clean":"2024-01-24T15:30:37.444715+0000","last_became_active":"2024-01-24T15:30:37.022717+0000","last_became_peered":"2024-01-24T15:30:37.022717+0000","last_unstale":"2024-01-24T15:30:37.444715+0000","last_undegraded":"2024-01-24T15:30:37.444715+0000","last_fullsized":"2024-01-24T15:30:37.444715+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.428146+0000","last_change":"2024-01-24T15:30:37.424872+0000","last_active":"2024-01-24T15:30:37.428146+0000","last_peered":"2024-01-24T15:30:37.428146+0000","last_clean":"2024-01-24T15:30:37.428146+0000","last_became_active":"2024-01-24T15:30:37.424550+0000","last_became_peered":"2024-01-24T15:30:37.424550+0000","last_unstale":"2024-01-24T15:30:37.428146+0000","last_undegraded":"2024-01-24T15:30:37.428146+0000","last_fullsized":"2024-01-24T15:30:37.428146+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.415400+0000","last_change":"2024-01-24T15:30:37.010684+0000","last_active":"2024-01-24T15:30:37.415400+0000","last_peered":"2024-01-24T15:30:37.415400+0000","last_clean":"2024-01-24T15:30:37.415400+0000","last_became_active":"2024-01-24T15:30:37.010372+0000","last_became_peered":"2024-01-24T15:30:37.010372+0000","last_unstale":"2024-01-24T15:30:37.415400+0000","last_undegraded":"2024-01-24T15:30:37.415400+0000","last_fullsized":"2024-01-24T15:30:37.415400+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.438100+0000","last_change":"2024-01-24T15:30:37.432163+0000","last_active":"2024-01-24T15:30:37.438100+0000","last_peered":"2024-01-24T15:30:37.438100+0000","last_clean":"2024-01-24T15:30:37.438100+0000","last_became_active":"2024-01-24T15:30:37.431851+0000","last_became_peered":"2024-01-24T15:30:37.431851+0000","last_unstale":"2024-01-24T15:30:37.438100+0000","last_undegraded":"2024-01-24T15:30:37.438100+0000","last_fullsized":"2024-01-24T15:30:37.438100+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425230+0000","last_change":"2024-01-24T15:30:37.020665+0000","last_active":"2024-01-24T15:30:37.425230+0000","last_peered":"2024-01-24T15:30:37.425230+0000","last_clean":"2024-01-24T15:30:37.425230+0000","last_became_active":"2024-01-24T15:30:37.019763+0000","last_became_peered":"2024-01-24T15:30:37.019763+0000","last_unstale":"2024-01-24T15:30:37.425230+0000","last_undegraded":"2024-01-24T15:30:37.425230+0000","last_fullsized":"2024-01-24T15:30:37.425230+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.434399+0000","last_change":"2024-01-24T15:30:37.429153+0000","last_active":"2024-01-24T15:30:37.434399+0000","last_peered":"2024-01-24T15:30:37.434399+0000","last_clean":"2024-01-24T15:30:37.434399+0000","last_became_active":"2024-01-24T15:30:37.428657+0000","last_became_peered":"2024-01-24T15:30:37.428657+0000","last_unstale":"2024-01-24T15:30:37.434399+0000","last_undegraded":"2024-01-24T15:30:37.434399+0000","last_fullsized":"2024-01-24T15:30:37.434399+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419309+0000","last_change":"2024-01-24T15:30:37.010593+0000","last_active":"2024-01-24T15:30:37.419309+0000","last_peered":"2024-01-24T15:30:37.419309+0000","last_clean":"2024-01-24T15:30:37.419309+0000","last_became_active":"2024-01-24T15:30:37.010306+0000","last_became_peered":"2024-01-24T15:30:37.010306+0000","last_unstale":"2024-01-24T15:30:37.419309+0000","last_undegraded":"2024-01-24T15:30:37.419309+0000","last_fullsized":"2024-01-24T15:30:37.419309+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.434255+0000","last_change":"2024-01-24T15:30:37.428903+0000","last_active":"2024-01-24T15:30:37.434255+0000","last_peered":"2024-01-24T15:30:37.434255+0000","last_clean":"2024-01-24T15:30:37.434255+0000","last_became_active":"2024-01-24T15:30:37.428533+0000","last_became_peered":"2024-01-24T15:30:37.428533+0000","last_unstale":"2024-01-24T15:30:37.434255+0000","last_undegraded":"2024-01-24T15:30:37.434255+0000","last_fullsized":"2024-01-24T15:30:37.434255+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"212'2","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.445039+0000","last_change":"2024-01-24T15:30:37.020234+0000","last_active":"2024-01-24T15:30:37.445039+0000","last_peered":"2024-01-24T15:30:37.445039+0000","last_clean":"2024-01-24T15:30:37.445039+0000","last_became_active":"2024-01-24T15:30:37.019961+0000","last_became_peered":"2024-01-24T15:30:37.019961+0000","last_unstale":"2024-01-24T15:30:37.445039+0000","last_undegraded":"2024-01-24T15:30:37.445039+0000","last_fullsized":"2024-01-24T15:30:37.445039+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439389+0000","last_change":"2024-01-24T15:30:37.436321+0000","last_active":"2024-01-24T15:30:37.439389+0000","last_peered":"2024-01-24T15:30:37.439389+0000","last_clean":"2024-01-24T15:30:37.439389+0000","last_became_active":"2024-01-24T15:30:37.435999+0000","last_became_peered":"2024-01-24T15:30:37.435999+0000","last_unstale":"2024-01-24T15:30:37.439389+0000","last_undegraded":"2024-01-24T15:30:37.439389+0000","last_fullsized":"2024-01-24T15:30:37.439389+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.434740+0000","last_change":"2024-01-24T15:30:37.020412+0000","last_active":"2024-01-24T15:30:37.434740+0000","last_peered":"2024-01-24T15:30:37.434740+0000","last_clean":"2024-01-24T15:30:37.434740+0000","last_became_active":"2024-01-24T15:30:37.020172+0000","last_became_peered":"2024-01-24T15:30:37.020172+0000","last_unstale":"2024-01-24T15:30:37.434740+0000","last_undegraded":"2024-01-24T15:30:37.434740+0000","last_fullsized":"2024-01-24T15:30:37.434740+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448311+0000","last_change":"2024-01-24T15:30:37.431673+0000","last_active":"2024-01-24T15:30:37.448311+0000","last_peered":"2024-01-24T15:30:37.448311+0000","last_clean":"2024-01-24T15:30:37.448311+0000","last_became_active":"2024-01-24T15:30:37.431342+0000","last_became_peered":"2024-01-24T15:30:37.431342+0000","last_unstale":"2024-01-24T15:30:37.448311+0000","last_undegraded":"2024-01-24T15:30:37.448311+0000","last_fullsized":"2024-01-24T15:30:37.448311+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.427225+0000","last_change":"2024-01-24T15:30:37.019758+0000","last_active":"2024-01-24T15:30:37.427225+0000","last_peered":"2024-01-24T15:30:37.427225+0000","last_clean":"2024-01-24T15:30:37.427225+0000","last_became_active":"2024-01-24T15:30:37.019451+0000","last_became_peered":"2024-01-24T15:30:37.019451+0000","last_unstale":"2024-01-24T15:30:37.427225+0000","last_undegraded":"2024-01-24T15:30:37.427225+0000","last_fullsized":"2024-01-24T15:30:37.427225+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448339+0000","last_change":"2024-01-24T15:30:37.439015+0000","last_active":"2024-01-24T15:30:37.448339+0000","last_peered":"2024-01-24T15:30:37.448339+0000","last_clean":"2024-01-24T15:30:37.448339+0000","last_became_active":"2024-01-24T15:30:37.438713+0000","last_became_peered":"2024-01-24T15:30:37.438713+0000","last_unstale":"2024-01-24T15:30:37.448339+0000","last_undegraded":"2024-01-24T15:30:37.448339+0000","last_fullsized":"2024-01-24T15:30:37.448339+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419898+0000","last_change":"2024-01-24T15:30:37.011516+0000","last_active":"2024-01-24T15:30:37.419898+0000","last_peered":"2024-01-24T15:30:37.419898+0000","last_clean":"2024-01-24T15:30:37.419898+0000","last_became_active":"2024-01-24T15:30:37.010843+0000","last_became_peered":"2024-01-24T15:30:37.010843+0000","last_unstale":"2024-01-24T15:30:37.419898+0000","last_undegraded":"2024-01-24T15:30:37.419898+0000","last_fullsized":"2024-01-24T15:30:37.419898+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.449445+0000","last_change":"2024-01-24T15:30:37.447387+0000","last_active":"2024-01-24T15:30:37.449445+0000","last_peered":"2024-01-24T15:30:37.449445+0000","last_clean":"2024-01-24T15:30:37.449445+0000","last_became_active":"2024-01-24T15:30:37.447106+0000","last_became_peered":"2024-01-24T15:30:37.447106+0000","last_unstale":"2024-01-24T15:30:37.449445+0000","last_undegraded":"2024-01-24T15:30:37.449445+0000","last_fullsized":"2024-01-24T15:30:37.449445+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420642+0000","last_change":"2024-01-24T15:30:37.006558+0000","last_active":"2024-01-24T15:30:37.420642+0000","last_peered":"2024-01-24T15:30:37.420642+0000","last_clean":"2024-01-24T15:30:37.420642+0000","last_became_active":"2024-01-24T15:30:37.006275+0000","last_became_peered":"2024-01-24T15:30:37.006275+0000","last_unstale":"2024-01-24T15:30:37.420642+0000","last_undegraded":"2024-01-24T15:30:37.420642+0000","last_fullsized":"2024-01-24T15:30:37.420642+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446588+0000","last_change":"2024-01-24T15:30:37.444460+0000","last_active":"2024-01-24T15:30:37.446588+0000","last_peered":"2024-01-24T15:30:37.446588+0000","last_clean":"2024-01-24T15:30:37.446588+0000","last_became_active":"2024-01-24T15:30:37.444196+0000","last_became_peered":"2024-01-24T15:30:37.444196+0000","last_unstale":"2024-01-24T15:30:37.446588+0000","last_undegraded":"2024-01-24T15:30:37.446588+0000","last_fullsized":"2024-01-24T15:30:37.446588+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419850+0000","last_change":"2024-01-24T15:30:37.002116+0000","last_active":"2024-01-24T15:30:37.419850+0000","last_peered":"2024-01-24T15:30:37.419850+0000","last_clean":"2024-01-24T15:30:37.419850+0000","last_became_active":"2024-01-24T15:30:37.001841+0000","last_became_peered":"2024-01-24T15:30:37.001841+0000","last_unstale":"2024-01-24T15:30:37.419850+0000","last_undegraded":"2024-01-24T15:30:37.419850+0000","last_fullsized":"2024-01-24T15:30:37.419850+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.438080+0000","last_change":"2024-01-24T15:30:37.431537+0000","last_active":"2024-01-24T15:30:37.438080+0000","last_peered":"2024-01-24T15:30:37.438080+0000","last_clean":"2024-01-24T15:30:37.438080+0000","last_became_active":"2024-01-24T15:30:37.431229+0000","last_became_peered":"2024-01-24T15:30:37.431229+0000","last_unstale":"2024-01-24T15:30:37.438080+0000","last_undegraded":"2024-01-24T15:30:37.438080+0000","last_fullsized":"2024-01-24T15:30:37.438080+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423792+0000","last_change":"2024-01-24T15:30:37.009420+0000","last_active":"2024-01-24T15:30:37.423792+0000","last_peered":"2024-01-24T15:30:37.423792+0000","last_clean":"2024-01-24T15:30:37.423792+0000","last_became_active":"2024-01-24T15:30:37.009141+0000","last_became_peered":"2024-01-24T15:30:37.009141+0000","last_unstale":"2024-01-24T15:30:37.423792+0000","last_undegraded":"2024-01-24T15:30:37.423792+0000","last_fullsized":"2024-01-24T15:30:37.423792+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.444250+0000","last_change":"2024-01-24T15:30:37.434072+0000","last_active":"2024-01-24T15:30:37.444250+0000","last_peered":"2024-01-24T15:30:37.444250+0000","last_clean":"2024-01-24T15:30:37.444250+0000","last_became_active":"2024-01-24T15:30:37.433708+0000","last_became_peered":"2024-01-24T15:30:37.433708+0000","last_unstale":"2024-01-24T15:30:37.444250+0000","last_undegraded":"2024-01-24T15:30:37.444250+0000","last_fullsized":"2024-01-24T15:30:37.444250+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425395+0000","last_change":"2024-01-24T15:30:37.020172+0000","last_active":"2024-01-24T15:30:37.425395+0000","last_peered":"2024-01-24T15:30:37.425395+0000","last_clean":"2024-01-24T15:30:37.425395+0000","last_became_active":"2024-01-24T15:30:37.019841+0000","last_became_peered":"2024-01-24T15:30:37.019841+0000","last_unstale":"2024-01-24T15:30:37.425395+0000","last_undegraded":"2024-01-24T15:30:37.425395+0000","last_fullsized":"2024-01-24T15:30:37.425395+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.445457+0000","last_change":"2024-01-24T15:30:37.444454+0000","last_active":"2024-01-24T15:30:37.445457+0000","last_peered":"2024-01-24T15:30:37.445457+0000","last_clean":"2024-01-24T15:30:37.445457+0000","last_became_active":"2024-01-24T15:30:37.444142+0000","last_became_peered":"2024-01-24T15:30:37.444142+0000","last_unstale":"2024-01-24T15:30:37.445457+0000","last_undegraded":"2024-01-24T15:30:37.445457+0000","last_fullsized":"2024-01-24T15:30:37.445457+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.439691+0000","last_change":"2024-01-24T15:30:37.005757+0000","last_active":"2024-01-24T15:30:37.439691+0000","last_peered":"2024-01-24T15:30:37.439691+0000","last_clean":"2024-01-24T15:30:37.439691+0000","last_became_active":"2024-01-24T15:30:37.005509+0000","last_became_peered":"2024-01-24T15:30:37.005509+0000","last_unstale":"2024-01-24T15:30:37.439691+0000","last_undegraded":"2024-01-24T15:30:37.439691+0000","last_fullsized":"2024-01-24T15:30:37.439691+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423817+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:30:37.423817+0000","last_peered":"2024-01-24T15:30:37.423817+0000","last_clean":"2024-01-24T15:30:37.423817+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:30:37.423817+0000","last_undegraded":"2024-01-24T15:30:37.423817+0000","last_fullsized":"2024-01-24T15:30:37.423817+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.443700+0000","last_change":"2024-01-24T15:30:37.438090+0000","last_active":"2024-01-24T15:30:37.443700+0000","last_peered":"2024-01-24T15:30:37.443700+0000","last_clean":"2024-01-24T15:30:37.443700+0000","last_became_active":"2024-01-24T15:30:37.437816+0000","last_became_peered":"2024-01-24T15:30:37.437816+0000","last_unstale":"2024-01-24T15:30:37.443700+0000","last_undegraded":"2024-01-24T15:30:37.443700+0000","last_fullsized":"2024-01-24T15:30:37.443700+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425229+0000","last_change":"2024-01-24T15:30:37.011798+0000","last_active":"2024-01-24T15:30:37.425229+0000","last_peered":"2024-01-24T15:30:37.425229+0000","last_clean":"2024-01-24T15:30:37.425229+0000","last_became_active":"2024-01-24T15:30:37.010303+0000","last_became_peered":"2024-01-24T15:30:37.010303+0000","last_unstale":"2024-01-24T15:30:37.425229+0000","last_undegraded":"2024-01-24T15:30:37.425229+0000","last_fullsized":"2024-01-24T15:30:37.425229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":196,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420380+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:30:37.420380+0000","last_peered":"2024-01-24T15:30:37.420380+0000","last_clean":"2024-01-24T15:30:37.420380+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:30:37.420380+0000","last_undegraded":"2024-01-24T15:30:37.420380+0000","last_fullsized":"2024-01-24T15:30:37.420380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.448897+0000","last_change":"2024-01-24T15:30:37.447169+0000","last_active":"2024-01-24T15:30:37.448897+0000","last_peered":"2024-01-24T15:30:37.448897+0000","last_clean":"2024-01-24T15:30:37.448897+0000","last_became_active":"2024-01-24T15:30:37.446858+0000","last_became_peered":"2024-01-24T15:30:37.446858+0000","last_unstale":"2024-01-24T15:30:37.448897+0000","last_undegraded":"2024-01-24T15:30:37.448897+0000","last_fullsized":"2024-01-24T15:30:37.448897+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.415807+0000","last_change":"2024-01-24T15:30:37.007210+0000","last_active":"2024-01-24T15:30:37.415807+0000","last_peered":"2024-01-24T15:30:37.415807+0000","last_clean":"2024-01-24T15:30:37.415807+0000","last_became_active":"2024-01-24T15:30:37.006907+0000","last_became_peered":"2024-01-24T15:30:37.006907+0000","last_unstale":"2024-01-24T15:30:37.415807+0000","last_undegraded":"2024-01-24T15:30:37.415807+0000","last_fullsized":"2024-01-24T15:30:37.415807+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":190,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.415645+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:30:37.415645+0000","last_peered":"2024-01-24T15:30:37.415645+0000","last_clean":"2024-01-24T15:30:37.415645+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:30:37.415645+0000","last_undegraded":"2024-01-24T15:30:37.415645+0000","last_fullsized":"2024-01-24T15:30:37.415645+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.435324+0000","last_change":"2024-01-24T15:30:37.431803+0000","last_active":"2024-01-24T15:30:37.435324+0000","last_peered":"2024-01-24T15:30:37.435324+0000","last_clean":"2024-01-24T15:30:37.435324+0000","last_became_active":"2024-01-24T15:30:37.431639+0000","last_became_peered":"2024-01-24T15:30:37.431639+0000","last_unstale":"2024-01-24T15:30:37.435324+0000","last_undegraded":"2024-01-24T15:30:37.435324+0000","last_fullsized":"2024-01-24T15:30:37.435324+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423196+0000","last_change":"2024-01-24T15:30:37.016795+0000","last_active":"2024-01-24T15:30:37.423196+0000","last_peered":"2024-01-24T15:30:37.423196+0000","last_clean":"2024-01-24T15:30:37.423196+0000","last_became_active":"2024-01-24T15:30:37.013889+0000","last_became_peered":"2024-01-24T15:30:37.013889+0000","last_unstale":"2024-01-24T15:30:37.423196+0000","last_undegraded":"2024-01-24T15:30:37.423196+0000","last_fullsized":"2024-01-24T15:30:37.423196+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":196,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.420780+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:30:37.420780+0000","last_peered":"2024-01-24T15:30:37.420780+0000","last_clean":"2024-01-24T15:30:37.420780+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:30:37.420780+0000","last_undegraded":"2024-01-24T15:30:37.420780+0000","last_fullsized":"2024-01-24T15:30:37.420780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":175,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425505+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:30:37.425505+0000","last_peered":"2024-01-24T15:30:37.425505+0000","last_clean":"2024-01-24T15:30:37.425505+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:30:37.425505+0000","last_undegraded":"2024-01-24T15:30:37.425505+0000","last_fullsized":"2024-01-24T15:30:37.425505+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446726+0000","last_change":"2024-01-24T15:30:37.429923+0000","last_active":"2024-01-24T15:30:37.446726+0000","last_peered":"2024-01-24T15:30:37.446726+0000","last_clean":"2024-01-24T15:30:37.446726+0000","last_became_active":"2024-01-24T15:30:37.426365+0000","last_became_peered":"2024-01-24T15:30:37.426365+0000","last_unstale":"2024-01-24T15:30:37.446726+0000","last_undegraded":"2024-01-24T15:30:37.446726+0000","last_fullsized":"2024-01-24T15:30:37.446726+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.419043+0000","last_change":"2024-01-24T15:30:37.014881+0000","last_active":"2024-01-24T15:30:37.419043+0000","last_peered":"2024-01-24T15:30:37.419043+0000","last_clean":"2024-01-24T15:30:37.419043+0000","last_became_active":"2024-01-24T15:30:37.014589+0000","last_became_peered":"2024-01-24T15:30:37.014589+0000","last_unstale":"2024-01-24T15:30:37.419043+0000","last_undegraded":"2024-01-24T15:30:37.419043+0000","last_fullsized":"2024-01-24T15:30:37.419043+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":173,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.425424+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:30:37.425424+0000","last_peered":"2024-01-24T15:30:37.425424+0000","last_clean":"2024-01-24T15:30:37.425424+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:30:37.425424+0000","last_undegraded":"2024-01-24T15:30:37.425424+0000","last_fullsized":"2024-01-24T15:30:37.425424+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.446260+0000","last_change":"2024-01-24T15:30:37.425910+0000","last_active":"2024-01-24T15:30:37.446260+0000","last_peered":"2024-01-24T15:30:37.446260+0000","last_clean":"2024-01-24T15:30:37.446260+0000","last_became_active":"2024-01-24T15:30:37.425624+0000","last_became_peered":"2024-01-24T15:30:37.425624+0000","last_unstale":"2024-01-24T15:30:37.446260+0000","last_undegraded":"2024-01-24T15:30:37.446260+0000","last_fullsized":"2024-01-24T15:30:37.446260+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"212'1","reported_seq":11,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.443429+0000","last_change":"2024-01-24T15:30:37.002291+0000","last_active":"2024-01-24T15:30:37.443429+0000","last_peered":"2024-01-24T15:30:37.443429+0000","last_clean":"2024-01-24T15:30:37.443429+0000","last_became_active":"2024-01-24T15:30:37.001934+0000","last_became_peered":"2024-01-24T15:30:37.001934+0000","last_unstale":"2024-01-24T15:30:37.443429+0000","last_undegraded":"2024-01-24T15:30:37.443429+0000","last_fullsized":"2024-01-24T15:30:37.443429+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":173,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.426939+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:30:37.426939+0000","last_peered":"2024-01-24T15:30:37.426939+0000","last_clean":"2024-01-24T15:30:37.426939+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:30:37.426939+0000","last_undegraded":"2024-01-24T15:30:37.426939+0000","last_fullsized":"2024-01-24T15:30:37.426939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.430435+0000","last_change":"2024-01-24T15:30:37.425987+0000","last_active":"2024-01-24T15:30:37.430435+0000","last_peered":"2024-01-24T15:30:37.430435+0000","last_clean":"2024-01-24T15:30:37.430435+0000","last_became_active":"2024-01-24T15:30:37.425720+0000","last_became_peered":"2024-01-24T15:30:37.425720+0000","last_unstale":"2024-01-24T15:30:37.430435+0000","last_undegraded":"2024-01-24T15:30:37.430435+0000","last_fullsized":"2024-01-24T15:30:37.430435+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":10,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423469+0000","last_change":"2024-01-24T15:30:37.008862+0000","last_active":"2024-01-24T15:30:37.423469+0000","last_peered":"2024-01-24T15:30:37.423469+0000","last_clean":"2024-01-24T15:30:37.423469+0000","last_became_active":"2024-01-24T15:30:37.008557+0000","last_became_peered":"2024-01-24T15:30:37.008557+0000","last_unstale":"2024-01-24T15:30:37.423469+0000","last_undegraded":"2024-01-24T15:30:37.423469+0000","last_fullsized":"2024-01-24T15:30:37.423469+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":172,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.423914+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:30:37.423914+0000","last_peered":"2024-01-24T15:30:37.423914+0000","last_clean":"2024-01-24T15:30:37.423914+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:30:37.423914+0000","last_undegraded":"2024-01-24T15:30:37.423914+0000","last_fullsized":"2024-01-24T15:30:37.423914+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.428810+0000","last_change":"2024-01-24T15:30:37.424775+0000","last_active":"2024-01-24T15:30:37.428810+0000","last_peered":"2024-01-24T15:30:37.428810+0000","last_clean":"2024-01-24T15:30:37.428810+0000","last_became_active":"2024-01-24T15:30:37.424447+0000","last_became_peered":"2024-01-24T15:30:37.424447+0000","last_unstale":"2024-01-24T15:30:37.428810+0000","last_undegraded":"2024-01-24T15:30:37.428810+0000","last_fullsized":"2024-01-24T15:30:37.428810+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:36.982084+0000","last_clean_scrub_stamp":"2024-01-24T15:30:36.982084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"212'2","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.440081+0000","last_change":"2024-01-24T15:30:37.017107+0000","last_active":"2024-01-24T15:30:37.440081+0000","last_peered":"2024-01-24T15:30:37.440081+0000","last_clean":"2024-01-24T15:30:37.440081+0000","last_became_active":"2024-01-24T15:30:37.016820+0000","last_became_peered":"2024-01-24T15:30:37.016820+0000","last_unstale":"2024-01-24T15:30:37.440081+0000","last_undegraded":"2024-01-24T15:30:37.440081+0000","last_fullsized":"2024-01-24T15:30:37.440081+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:30:35.974079+0000","last_clean_scrub_stamp":"2024-01-24T15:30:35.974079+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":171,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-01-24T15:30:37.424181+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:30:37.424181+0000","last_peered":"2024-01-24T15:30:37.424181+0000","last_clean":"2024-01-24T15:30:37.424181+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:30:37.424181+0000","last_undegraded":"2024-01-24T15:30:37.424181+0000","last_fullsized":"2024-01-24T15:30:37.424181+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575953,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303788,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068052,"statfs":{"total":96636764160,"available":96325685248,"internally_reserved":0,"allocated":2658304,"data_stored":1534881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.48699999999999999,"5min":0.38800000000000001,"15min":0.32000000000000001},"max":{"1min":0.80300000000000005,"5min":0.80500000000000005,"15min":0.83199999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.63300000000000001,"15min":0.63200000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81100000000000005,"5min":0.84799999999999998,"15min":0.85499999999999998},"last":0.68300000000000005}]},{"osd":1,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.64400000000000002},"min":{"1min":0.497,"5min":0.46000000000000002,"15min":0.36699999999999999},"max":{"1min":0.84499999999999997,"5min":0.878,"15min":0.878},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73699999999999999,"5min":0.80600000000000005,"15min":0.89600000000000002},"last":0.63600000000000001}]},{"osd":2,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.77400000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":0.83299999999999996,"15min":0.88800000000000001},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.42399999999999999},"max":{"1min":0.71899999999999997,"5min":0.83399999999999996,"15min":0.92600000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45200000000000001,"5min":0.42799999999999999,"15min":0.42099999999999999},"max":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.88},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.36199999999999999,"5min":0.35599999999999998,"15min":0.30599999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79900000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.83299999999999996},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.50600000000000001},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.32300000000000001},"max":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.76100000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.42299999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.83399999999999996},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.41299999999999998,"5min":0.377,"15min":0.34999999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.81899999999999995},"last":0.626},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.35099999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.92700000000000005},"last":0.71699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575954,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307892,"kb_used_data":2604,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063948,"statfs":{"total":96636764160,"available":96321482752,"internally_reserved":0,"allocated":2666496,"data_stored":1542278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.504,"5min":0.42299999999999999,"15min":0.31900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.63200000000000001,"15min":0.61299999999999999},"min":{"1min":0.46999999999999997,"5min":0.44700000000000001,"15min":0.32800000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.95599999999999996},"last":0.53000000000000003}]},{"osd":1,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.34399999999999997},"max":{"1min":0.72999999999999998,"5min":0.86399999999999999,"15min":1.089},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.47099999999999997,"5min":0.42699999999999999,"15min":0.372},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.94299999999999995},"last":0.51800000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.629,"15min":0.625},"min":{"1min":0.439,"5min":0.439,"15min":0.36599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63700000000000001,"15min":0.62},"min":{"1min":0.48799999999999999,"5min":0.45100000000000001,"15min":0.40600000000000003},"max":{"1min":0.81000000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64500000000000002,"15min":0.63600000000000001},"min":{"1min":0.56000000000000005,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.0980000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.47199999999999998,"5min":0.42399999999999999,"15min":0.40999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":1.169},"last":0.48899999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.38500000000000001,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.71399999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":0.81399999999999995,"15min":0.83999999999999997},"last":0.56599999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55300000000000005,"15min":0.54400000000000004},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36399999999999999},"max":{"1min":0.78800000000000003,"5min":0.84199999999999997,"15min":0.86499999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54800000000000004,"15min":0.53300000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33900000000000002},"max":{"1min":0.68700000000000006,"5min":0.80100000000000005,"15min":0.82799999999999996},"last":0.36299999999999999}]},{"osd":7,"last update":"Wed Jan 24 15:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83699999999999997,"5min":0.88,"15min":0.88},"last":0.378},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.50900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575953,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307900,"kb_used_data":2612,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063940,"statfs":{"total":96636764160,"available":96321474560,"internally_reserved":0,"allocated":2674688,"data_stored":1542438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.28399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58099999999999996,"5min":0.66000000000000003,"15min":0.68500000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.44800000000000001,"15min":0.443},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.26100000000000001},"max":{"1min":0.51600000000000001,"5min":0.745,"15min":0.745},"last":0.33400000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.308,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56299999999999994,"5min":0.72999999999999998,"15min":0.76100000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.45100000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.626,"5min":0.68100000000000005,"15min":0.71099999999999997},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46999999999999997,"15min":0.46700000000000003},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.60099999999999998,"5min":0.71399999999999997,"15min":0.77000000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.502,"15min":0.499},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.32600000000000001},"max":{"1min":0.68300000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57999999999999996}]},{"osd":4,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73699999999999999,"15min":0.70099999999999996},"min":{"1min":0.51800000000000002,"5min":0.48399999999999999,"15min":0.41599999999999998},"max":{"1min":0.89000000000000001,"5min":5.3630000000000004,"15min":5.3630000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70599999999999996,"15min":0.68899999999999995},"min":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":1.677,"15min":1.677},"last":0.80600000000000005}]},{"osd":5,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.48499999999999999},"max":{"1min":0.92700000000000005,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.54900000000000004,"5min":0.5,"15min":0.46100000000000002},"max":{"1min":0.89600000000000002,"5min":1.1399999999999999,"15min":1.2989999999999999},"last":0.85599999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.52300000000000002,"5min":0.47899999999999998,"15min":0.46400000000000002},"max":{"1min":0.85699999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76300000000000001,"15min":0.753},"min":{"1min":0.63800000000000001,"5min":0.52000000000000002,"15min":0.47799999999999998},"max":{"1min":0.91200000000000003,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.84199999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.52900000000000003,"15min":0.40799999999999997},"max":{"1min":0.82399999999999995,"5min":1.704,"15min":1.704},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.82099999999999995,"15min":0.78900000000000003},"min":{"1min":0.61699999999999999,"5min":0.53800000000000003,"15min":0.47999999999999998},"max":{"1min":0.96699999999999997,"5min":5.29,"15min":5.29},"last":0.79400000000000004}]}]},{"osd":0,"up_from":14,"seq":60129543251,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307876,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063964,"statfs":{"total":96636764160,"available":96321499136,"internally_reserved":0,"allocated":2650112,"data_stored":1533016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41099999999999998,"15min":0.40600000000000003},"min":{"1min":0.27300000000000002,"5min":0.26500000000000001,"15min":0.26000000000000001},"max":{"1min":0.58199999999999996,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.27300000000000002},"max":{"1min":0.59199999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.34200000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.60999999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.625},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47499999999999998,"15min":0.46800000000000003},"min":{"1min":0.36699999999999999,"5min":0.33600000000000002,"15min":0.317},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.72999999999999998},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.441,"15min":0.441},"min":{"1min":0.33100000000000002,"5min":0.307,"15min":0.28000000000000003},"max":{"1min":0.61699999999999999,"5min":0.67400000000000004,"15min":0.72799999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.36399999999999999,"15min":0.35999999999999999},"max":{"1min":0.69399999999999995,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.81200000000000006}]},{"osd":4,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.496,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":0.82899999999999996,"5min":0.88600000000000001,"15min":0.98699999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.67100000000000004},"min":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.41199999999999998},"max":{"1min":0.877,"5min":0.998,"15min":1.0189999999999999},"last":0.755}]},{"osd":5,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.40699999999999997},"max":{"1min":0.95699999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.71499999999999997},"min":{"1min":0.59999999999999998,"5min":0.51400000000000001,"15min":0.46500000000000002},"max":{"1min":0.99199999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.94599999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73799999999999999,"15min":0.71899999999999997},"min":{"1min":0.57499999999999996,"5min":0.503,"15min":0.49299999999999999},"max":{"1min":0.98399999999999999,"5min":1.27,"15min":1.27},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72399999999999998,"15min":0.71499999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.71799999999999997}]},{"osd":7,"last update":"Wed Jan 24 15:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.745,"15min":0.73299999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.72799999999999998},"min":{"1min":0.56299999999999994,"5min":0.5,"15min":0.46600000000000003},"max":{"1min":1.0189999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575953,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311992,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059848,"statfs":{"total":96636764160,"available":96317284352,"internally_reserved":0,"allocated":2670592,"data_stored":1541836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:30:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.41299999999999998,"15min":0.40500000000000003},"min":{"1min":0.30099999999999999,"5min":0.26800000000000002,"15min":0.252},"max":{"1min":0.59799999999999998,"5min":0.627,"15min":0.69699999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.45300000000000001},"min":{"1min":0.317,"5min":0.30399999999999999,"15min":0.28499999999999998},"max":{"1min":0.63,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.45500000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.29199999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.629,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.47999999999999998},"min":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.27400000000000002},"max":{"1min":0.69799999999999995,"5min":0.70299999999999996,"15min":0.80900000000000005},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Jan 24 15:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48099999999999998,"15min":0.47199999999999998},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.29099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.39800000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68500000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.503}]},{"osd":4,"last update":"Wed Jan 24 15:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68999999999999995,"15min":0.66700000000000004},"min":{"1min":0.53900000000000003,"5min":0.51600000000000001,"15min":0.38300000000000001},"max":{"1min":0.95999999999999996,"5min":0.96399999999999997,"15min":0.97999999999999998},"last":0.745},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70999999999999996,"15min":0.68899999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.376},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.056},"last":0.80900000000000005}]},{"osd":5,"last update":"Wed Jan 24 15:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.71599999999999997},"min":{"1min":0.54100000000000004,"5min":0.53600000000000003,"15min":0.439},"max":{"1min":0.996,"5min":1.0589999999999999,"15min":1.0660000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70899999999999996,"15min":0.69599999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.45700000000000002},"max":{"1min":1.024,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.59299999999999997}]},{"osd":6,"last update":"Wed Jan 24 15:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.748,"15min":0.73099999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.39700000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.0740000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76300000000000001,"15min":0.74399999999999999},"min":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.49399999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.107},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76300000000000001,"15min":0.749},"min":{"1min":0.60799999999999998,"5min":0.52300000000000002,"15min":0.47199999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.628},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78900000000000003,"15min":0.76200000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.495},"max":{"1min":1.02,"5min":1.0429999999999999,"15min":1.123},"last":0.84399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575953,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307896,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063944,"statfs":{"total":96636764160,"available":96321478656,"internally_reserved":0,"allocated":2670592,"data_stored":1541848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40899999999999997,"15min":0.40699999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51800000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.436,"15min":0.436},"min":{"1min":0.33500000000000002,"5min":0.28299999999999997,"15min":0.27900000000000003},"max":{"1min":0.70299999999999996,"5min":0.76100000000000001,"15min":0.77100000000000002},"last":0.60699999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42299999999999999,"15min":0.42799999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":0.57799999999999996,"5min":0.625,"15min":0.66900000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48199999999999998,"15min":0.47999999999999998},"min":{"1min":0.33100000000000002,"5min":0.312,"15min":0.29099999999999998},"max":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.752},"last":0.63100000000000001}]},{"osd":2,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.64400000000000002,"5min":0.71899999999999997,"15min":0.80800000000000005},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.433,"5min":0.34899999999999998,"15min":0.33400000000000002},"max":{"1min":0.68500000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.47699999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.79300000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.44700000000000001},"max":{"1min":0.88800000000000001,"5min":1.026,"15min":1.026},"last":1.026}]},{"osd":5,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.50600000000000001,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.97699999999999998,"5min":1.05,"15min":1.05},"last":1.056},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.47199999999999998},"max":{"1min":0.92300000000000004,"5min":0.997,"15min":0.997},"last":1.095}]},{"osd":6,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.52500000000000002,"5min":0.499,"15min":0.47999999999999998},"max":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":1.1499999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73299999999999998,"15min":0.72199999999999998},"min":{"1min":0.53600000000000003,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.90400000000000003,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.88700000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58199999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93300000000000005,"5min":1.002,"15min":1.083},"last":1.071},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77500000000000002,"15min":0.76900000000000002},"min":{"1min":0.55700000000000005,"5min":0.55500000000000005,"15min":0.52600000000000002},"max":{"1min":0.96899999999999997,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.91300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575954,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307904,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063936,"statfs":{"total":96636764160,"available":96321470464,"internally_reserved":0,"allocated":2678784,"data_stored":1543814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.375},"max":{"1min":0.80100000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.443},{"interface":"front","average":{"1min":0.627,"5min":0.621,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.44700000000000001,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.876,"15min":0.876},"last":0.51500000000000001}]},{"osd":1,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.45900000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.83899999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.46100000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78400000000000003,"5min":0.81499999999999995,"15min":0.85199999999999998},"last":0.35299999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60999999999999999,"15min":0.61899999999999999},"min":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.71599999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.379},{"interface":"front","average":{"1min":0.627,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.48199999999999998,"5min":0.42699999999999999,"15min":0.38500000000000001},"max":{"1min":0.84399999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42599999999999999}]},{"osd":3,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72499999999999998,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.752,"5min":0.86799999999999999,"15min":0.87},"last":0.56999999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.308},"max":{"1min":0.73599999999999999,"5min":0.77900000000000003,"15min":0.78200000000000003},"last":0.502}]},{"osd":6,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51400000000000001,"15min":0.50900000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.35199999999999998},"max":{"1min":0.83399999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Jan 24 15:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.34899999999999998},"max":{"1min":0.69999999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.86099999999999999,"15min":0.92700000000000005},"last":0.60499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575954,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307892,"kb_used_data":2604,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063948,"statfs":{"total":96636764160,"available":96321482752,"internally_reserved":0,"allocated":2666496,"data_stored":1541794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61299999999999999},"min":{"1min":0.46899999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72799999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.625},{"interface":"front","average":{"1min":0.621,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.49099999999999999,"5min":0.371,"15min":0.32100000000000001},"max":{"1min":0.75800000000000001,"5min":1.149,"15min":1.149},"last":0.42899999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.34200000000000003},"max":{"1min":0.76600000000000001,"5min":1.125,"15min":1.125},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.45300000000000001,"5min":0.38400000000000001,"15min":0.35799999999999998},"max":{"1min":0.76300000000000001,"5min":0.83499999999999996,"15min":0.85299999999999998},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.63600000000000001},"min":{"1min":0.53100000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76800000000000002,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60599999999999998,"15min":0.61499999999999999},"min":{"1min":0.46899999999999997,"5min":0.41199999999999998,"15min":0.379},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":1.252},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.42599999999999999},"max":{"1min":0.80200000000000005,"5min":0.83599999999999997,"15min":0.95499999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47099999999999997,"5min":0.46899999999999997,"15min":0.46000000000000002},"max":{"1min":0.81599999999999995,"5min":1.196,"15min":1.196},"last":0.65800000000000003}]},{"osd":4,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.90500000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51100000000000001,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.35299999999999998,"15min":0.32500000000000001},"max":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.84899999999999998},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.497,"15min":0.50600000000000001},"min":{"1min":0.376,"5min":0.33400000000000002,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.82199999999999995,"15min":1.0209999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.40500000000000003,"5min":0.38800000000000001,"15min":0.36699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78900000000000003,"15min":0.98399999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.54700000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.33600000000000002},"max":{"1min":0.751,"5min":0.77900000000000003,"15min":0.86399999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.91700000000000004},"last":0.51700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:30:41.937 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-24T15:30:41.938 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-24T15:30:42.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.110+0000 7fdb40071700 1 Processor -- start 2024-01-24T15:30:42.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.111+0000 7fdb40071700 1 -- start start 2024-01-24T15:30:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.111+0000 7fdb40071700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb3814e4f0 0x7fdb3814e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.111+0000 7fdb40071700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdb38157c80 0x7fdb38154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.111+0000 7fdb40071700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb38158220 con 0x7fdb38065a80 2024-01-24T15:30:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.111+0000 7fdb40071700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb38154850 con 0x7fdb38157c80 2024-01-24T15:30:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.111+0000 7fdb40071700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38158500 con 0x7fdb3814e4f0 2024-01-24T15:30:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.112+0000 7fdb3e60e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdb38157c80 0x7fdb38154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.112+0000 7fdb3e60e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdb38157c80 0x7fdb38154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37392/0 (socket says 172.21.15.29:37392) 2024-01-24T15:30:42.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.112+0000 7fdb3e60e700 1 -- 172.21.15.29:0/1754419664 learned_addr learned my addr 172.21.15.29:0/1754419664 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.112+0000 7fdb3de0d700 1 --2- 172.21.15.29:0/1754419664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb3814e4f0 0x7fdb3814e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.112+0000 7fdb3e60e700 1 -- 172.21.15.29:0/1754419664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb3814e4f0 msgr2=0x7fdb3814e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.112+0000 7fdb3e60e700 1 --2- 172.21.15.29:0/1754419664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb3814e4f0 0x7fdb3814e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.112+0000 7fdb3e60e700 1 -- 172.21.15.29:0/1754419664 >> v1:172.21.15.163:6789/0 conn(0x7fdb38065a80 legacy=0x7fdb380627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.112+0000 7fdb3e60e700 1 -- 172.21.15.29:0/1754419664 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb381587e0 con 0x7fdb38157c80 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.112+0000 7fdb3e60e700 1 --2- 172.21.15.29:0/1754419664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb38157c80 0x7fdb38154ca0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 rx=0x7fdb30009f20 tx=0x7fdb30009840).ready entity=mon.0 client_cookie=fb59ac2f0b0ba68b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.112+0000 7fdb3ce0b700 1 -- 172.21.15.29:0/1754419664 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000e070 con 0x7fdb38157c80 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.112+0000 7fdb3ce0b700 1 -- 172.21.15.29:0/1754419664 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3000b9d0 con 0x7fdb38157c80 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.112+0000 7fdb3ce0b700 1 -- 172.21.15.29:0/1754419664 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000bd50 con 0x7fdb38157c80 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.113+0000 7fdb40071700 1 -- 172.21.15.29:0/1754419664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb38157c80 msgr2=0x7fdb38154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.113+0000 7fdb40071700 1 --2- 172.21.15.29:0/1754419664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb38157c80 0x7fdb38154ca0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 rx=0x7fdb30009f20 tx=0x7fdb30009840).stop 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.113+0000 7fdb40071700 1 -- 172.21.15.29:0/1754419664 shutdown_connections 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.113+0000 7fdb40071700 1 --2- 172.21.15.29:0/1754419664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb3814e4f0 0x7fdb3814e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.116 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.113+0000 7fdb40071700 1 --2- 172.21.15.29:0/1754419664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb38157c80 0x7fdb38154ca0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.113+0000 7fdb40071700 1 -- 172.21.15.29:0/1754419664 >> 172.21.15.29:0/1754419664 conn(0x7fdb380c7330 msgr2=0x7fdb380c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:42.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.113+0000 7fdb40071700 1 -- 172.21.15.29:0/1754419664 shutdown_connections 2024-01-24T15:30:42.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.113+0000 7fdb40071700 1 -- 172.21.15.29:0/1754419664 wait complete. 2024-01-24T15:30:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.114+0000 7fdb40071700 1 Processor -- start 2024-01-24T15:30:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.114+0000 7fdb40071700 1 -- start start 2024-01-24T15:30:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.115+0000 7fdb40071700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb38065a80 0x7fdb380ad150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.115+0000 7fdb40071700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb3814e4f0 0x7fdb381455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.115+0000 7fdb40071700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb38157c80 0x7fdb38156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.115+0000 7fdb40071700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38146c40 con 0x7fdb38157c80 2024-01-24T15:30:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.115+0000 7fdb40071700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380acf20 con 0x7fdb3814e4f0 2024-01-24T15:30:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.115+0000 7fdb40071700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38147410 con 0x7fdb38065a80 2024-01-24T15:30:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.115+0000 7fdb3e60e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb38157c80 0x7fdb38156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.115+0000 7fdb3e60e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb38157c80 0x7fdb38156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37400/0 (socket says 172.21.15.29:37400) 2024-01-24T15:30:42.119 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.115+0000 7fdb3e60e700 1 -- 172.21.15.29:0/1415959650 learned_addr learned my addr 172.21.15.29:0/1415959650 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.115+0000 7fdb3de0d700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb38065a80 0x7fdb380ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.115+0000 7fdb3d60c700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb3814e4f0 0x7fdb381455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.116+0000 7fdb3e60e700 1 -- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb38065a80 msgr2=0x7fdb380ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.116+0000 7fdb3e60e700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb38065a80 0x7fdb380ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.116+0000 7fdb3e60e700 1 -- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb3814e4f0 msgr2=0x7fdb381455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.116+0000 7fdb3e60e700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb3814e4f0 0x7fdb381455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.116+0000 7fdb3e60e700 1 -- 172.21.15.29:0/1415959650 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb30009ae0 con 0x7fdb38157c80 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.116+0000 7fdb3e60e700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb38157c80 0x7fdb38156760 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 rx=0x7fdb3000b610 tx=0x7fdb30009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.116+0000 7fdb2dffb700 1 -- 172.21.15.29:0/1415959650 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb300092e0 con 0x7fdb38157c80 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.116+0000 7fdb2dffb700 1 -- 172.21.15.29:0/1415959650 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb30007610 con 0x7fdb38157c80 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.116+0000 7fdb40071700 1 -- 172.21.15.29:0/1415959650 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb38149530 con 0x7fdb38157c80 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.116+0000 7fdb2dffb700 1 -- 172.21.15.29:0/1415959650 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb300079d0 con 0x7fdb38157c80 2024-01-24T15:30:42.120 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.116+0000 7fdb40071700 1 -- 172.21.15.29:0/1415959650 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb38147130 con 0x7fdb38157c80 2024-01-24T15:30:42.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.117+0000 7fdb40071700 1 -- 172.21.15.29:0/1415959650 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb380475e0 con 0x7fdb38157c80 2024-01-24T15:30:42.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.118+0000 7fdb2dffb700 1 -- 172.21.15.29:0/1415959650 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdb30003040 con 0x7fdb38157c80 2024-01-24T15:30:42.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.118+0000 7fdb2dffb700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdb1c05f270 0x7fdb1c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.118+0000 7fdb2dffb700 1 -- 172.21.15.29:0/1415959650 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fdb3007bf50 con 0x7fdb38157c80 2024-01-24T15:30:42.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.118+0000 7fdb3de0d700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdb1c05f270 0x7fdb1c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.119+0000 7fdb3de0d700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdb1c05f270 0x7fdb1c061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fdb34006fd0 tx=0x7fdb34006f40).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:42.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.122+0000 7fdb2dffb700 1 -- 172.21.15.29:0/1415959650 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdb3004cd20 con 0x7fdb38157c80 2024-01-24T15:30:42.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.375+0000 7fdb40071700 1 -- 172.21.15.29:0/1415959650 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdb38062ec0 con 0x7fdb38157c80 2024-01-24T15:30:42.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.378+0000 7fdb2dffb700 1 -- 172.21.15.29:0/1415959650 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v53) v1 ==== 190+0+0 (secure 0 0 0) 0x7fdb30023030 con 0x7fdb38157c80 2024-01-24T15:30:42.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.380+0000 7fdb40071700 1 -- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdb1c05f270 msgr2=0x7fdb1c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:42.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.380+0000 7fdb40071700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdb1c05f270 0x7fdb1c061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fdb34006fd0 tx=0x7fdb34006f40).stop 2024-01-24T15:30:42.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.380+0000 7fdb40071700 1 -- 172.21.15.29:0/1415959650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb38157c80 msgr2=0x7fdb38156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:42.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.380+0000 7fdb40071700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb38157c80 0x7fdb38156760 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 rx=0x7fdb3000b610 tx=0x7fdb30009840).stop 2024-01-24T15:30:42.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.380+0000 7fdb40071700 1 -- 172.21.15.29:0/1415959650 shutdown_connections 2024-01-24T15:30:42.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.380+0000 7fdb40071700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb38157c80 0x7fdb38156760 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.380+0000 7fdb40071700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdb1c05f270 0x7fdb1c061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.380+0000 7fdb40071700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb38065a80 0x7fdb380ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.380+0000 7fdb40071700 1 --2- 172.21.15.29:0/1415959650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb3814e4f0 0x7fdb381455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.380+0000 7fdb40071700 1 -- 172.21.15.29:0/1415959650 >> 172.21.15.29:0/1415959650 conn(0x7fdb380c7330 msgr2=0x7fdb38154c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:42.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.380+0000 7fdb40071700 1 -- 172.21.15.29:0/1415959650 shutdown_connections 2024-01-24T15:30:42.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.381+0000 7fdb40071700 1 -- 172.21.15.29:0/1415959650 wait complete. 2024-01-24T15:30:42.384 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-01-24T15:30:42.400 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:30:42.451 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:30:42.453 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:30:42.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.530+0000 7f0882e93700 1 Processor -- start 2024-01-24T15:30:42.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.531+0000 7f0882e93700 1 -- start start 2024-01-24T15:30:42.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.531+0000 7f0882e93700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f087c134cc0 0x7f087c0c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.531+0000 7f0882e93700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f087c1350a0 0x7f087c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.531+0000 7f0882e93700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c0664f0 0x7f087c0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.531+0000 7f0882e93700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c153360 con 0x7f087c1350a0 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.531+0000 7f0882e93700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c066e90 con 0x7f087c134cc0 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.531+0000 7f0882e93700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c068360 con 0x7f087c0664f0 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f087bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f087c1350a0 0x7f087c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f087bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f087c1350a0 0x7f087c152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37416/0 (socket says 172.21.15.29:37416) 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f087bfff700 1 -- 172.21.15.29:0/2344043661 learned_addr learned my addr 172.21.15.29:0/2344043661 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f0880c2f700 1 --2- 172.21.15.29:0/2344043661 >> v2:172.21.15.163:3300/0 conn(0x7f087c134cc0 0x7f087c0c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f0881430700 1 --2- 172.21.15.29:0/2344043661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c0664f0 0x7f087c0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f087bfff700 1 -- 172.21.15.29:0/2344043661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c0664f0 msgr2=0x7f087c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f087bfff700 1 --2- 172.21.15.29:0/2344043661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c0664f0 0x7f087c0669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f087bfff700 1 -- 172.21.15.29:0/2344043661 >> v2:172.21.15.163:3300/0 conn(0x7f087c134cc0 msgr2=0x7f087c0c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f087bfff700 1 --2- 172.21.15.29:0/2344043661 >> v2:172.21.15.163:3300/0 conn(0x7f087c134cc0 0x7f087c0c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f087bfff700 1 -- 172.21.15.29:0/2344043661 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c0c3820 con 0x7f087c1350a0 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f0880c2f700 1 --2- 172.21.15.29:0/2344043661 >> v2:172.21.15.163:3300/0 conn(0x7f087c134cc0 0x7f087c0c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f0881430700 1 --2- 172.21.15.29:0/2344043661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c0664f0 0x7f087c0669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f087bfff700 1 --2- 172.21.15.29:0/2344043661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1350a0 0x7f087c152e70 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f087c0aee60 tx=0x7f08680090f0).ready entity=mon.0 client_cookie=e10d41aed67500ec server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f087b7fe700 1 -- 172.21.15.29:0/2344043661 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086800b600 con 0x7f087c1350a0 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f087b7fe700 1 -- 172.21.15.29:0/2344043661 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086800b760 con 0x7f087c1350a0 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.532+0000 7f087b7fe700 1 -- 172.21.15.29:0/2344043661 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086800bae0 con 0x7f087c1350a0 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.533+0000 7f0882e93700 1 -- 172.21.15.29:0/2344043661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1350a0 msgr2=0x7f087c152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.533+0000 7f0882e93700 1 --2- 172.21.15.29:0/2344043661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1350a0 0x7f087c152e70 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f087c0aee60 tx=0x7f08680090f0).stop 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.533+0000 7f0882e93700 1 -- 172.21.15.29:0/2344043661 shutdown_connections 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.533+0000 7f0882e93700 1 --2- 172.21.15.29:0/2344043661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c0664f0 0x7f087c0669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.533+0000 7f0882e93700 1 --2- 172.21.15.29:0/2344043661 >> v2:172.21.15.163:3300/0 conn(0x7f087c134cc0 0x7f087c0c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.533+0000 7f0882e93700 1 --2- 172.21.15.29:0/2344043661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1350a0 0x7f087c152e70 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.533+0000 7f0882e93700 1 -- 172.21.15.29:0/2344043661 >> 172.21.15.29:0/2344043661 conn(0x7f087c0c8bb0 msgr2=0x7f087c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:42.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.533+0000 7f0882e93700 1 -- 172.21.15.29:0/2344043661 shutdown_connections 2024-01-24T15:30:42.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.533+0000 7f0882e93700 1 -- 172.21.15.29:0/2344043661 wait complete. 2024-01-24T15:30:42.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.534+0000 7f0882e93700 1 Processor -- start 2024-01-24T15:30:42.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.535+0000 7f0882e93700 1 -- start start 2024-01-24T15:30:42.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.535+0000 7f0882e93700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f087c0664f0 0x7f087c14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.535+0000 7f0882e93700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c134cc0 0x7f087c0ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.535+0000 7f0882e93700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1350a0 0x7f087c0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.535+0000 7f0882e93700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0ab7c0 con 0x7f087c1350a0 2024-01-24T15:30:42.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.535+0000 7f0882e93700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0ae170 con 0x7f087c0664f0 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.535+0000 7f0882e93700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c14ba70 con 0x7f087c134cc0 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.535+0000 7f0881430700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1350a0 0x7f087c0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0881430700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1350a0 0x7f087c0adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37430/0 (socket says 172.21.15.29:37430) 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0881430700 1 -- 172.21.15.29:0/3049196985 learned_addr learned my addr 172.21.15.29:0/3049196985 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f087bfff700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c134cc0 0x7f087c0ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0880c2f700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f087c0664f0 0x7f087c14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0881430700 1 -- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c134cc0 msgr2=0x7f087c0ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0881430700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c134cc0 0x7f087c0ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0881430700 1 -- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f087c0664f0 msgr2=0x7f087c14b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0881430700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f087c0664f0 0x7f087c14b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0881430700 1 -- 172.21.15.29:0/3049196985 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868009370 con 0x7f087c1350a0 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f087bfff700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c134cc0 0x7f087c0ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:42.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0880c2f700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f087c0664f0 0x7f087c14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:42.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0881430700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1350a0 0x7f087c0adc80 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7f087400a380 tx=0x7f0874007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:42.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0878ff9700 1 -- 172.21.15.29:0/3049196985 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087400ae10 con 0x7f087c1350a0 2024-01-24T15:30:42.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0878ff9700 1 -- 172.21.15.29:0/3049196985 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087400d650 con 0x7f087c1350a0 2024-01-24T15:30:42.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0882e93700 1 -- 172.21.15.29:0/3049196985 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c0ab200 con 0x7f087c1350a0 2024-01-24T15:30:42.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0878ff9700 1 -- 172.21.15.29:0/3049196985 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087400d9d0 con 0x7f087c1350a0 2024-01-24T15:30:42.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.536+0000 7f0882e93700 1 -- 172.21.15.29:0/3049196985 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c0c0dc0 con 0x7f087c1350a0 2024-01-24T15:30:42.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.538+0000 7f0882e93700 1 -- 172.21.15.29:0/3049196985 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c0c0c20 con 0x7f087c1350a0 2024-01-24T15:30:42.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.538+0000 7f0878ff9700 1 -- 172.21.15.29:0/3049196985 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0874013070 con 0x7f087c1350a0 2024-01-24T15:30:42.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.538+0000 7f0878ff9700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f086005f270 0x7f0860061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:42.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.538+0000 7f0878ff9700 1 -- 172.21.15.29:0/3049196985 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f087407af40 con 0x7f087c1350a0 2024-01-24T15:30:42.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.538+0000 7f0880c2f700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f086005f270 0x7f0860061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:42.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.539+0000 7f0880c2f700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f086005f270 0x7f0860061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f087000aa90 tx=0x7f0870008040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:42.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.542+0000 7f0878ff9700 1 -- 172.21.15.29:0/3049196985 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f087c0c0c20 con 0x7f087c1350a0 2024-01-24T15:30:42.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.770+0000 7f0882e93700 1 -- 172.21.15.29:0/3049196985 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f087c0c0c20 con 0x7f087c1350a0 2024-01-24T15:30:42.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.771+0000 7f0878ff9700 1 -- 172.21.15.29:0/3049196985 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f087c0c0c20 con 0x7f087c1350a0 2024-01-24T15:30:42.773 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:42.773 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8060,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3418669722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3418669722},{"type":"v1","addr":"172.21.15.163:6836","nonce":3418669722}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8063,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/407488941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":407488941},{"type":"v1","addr":"172.21.15.163:6837","nonce":407488941}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4018429461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4018429461},{"type":"v1","addr":"172.21.15.29:6834","nonce":4018429461}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:30:37.411326+0000","modified":"2024-01-24T15:30:40.428239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2886498894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2886498894},{"type":"v1","addr":"172.21.15.29:6835","nonce":2886498894}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-24T15:30:42.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.773+0000 7f0882e93700 1 -- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f086005f270 msgr2=0x7f0860061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:42.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.773+0000 7f0882e93700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f086005f270 0x7f0860061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f087000aa90 tx=0x7f0870008040).stop 2024-01-24T15:30:42.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.773+0000 7f0882e93700 1 -- 172.21.15.29:0/3049196985 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1350a0 msgr2=0x7f087c0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:42.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.773+0000 7f0882e93700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1350a0 0x7f087c0adc80 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7f087400a380 tx=0x7f0874007900).stop 2024-01-24T15:30:42.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.773+0000 7f0882e93700 1 -- 172.21.15.29:0/3049196985 shutdown_connections 2024-01-24T15:30:42.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.773+0000 7f0882e93700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1350a0 0x7f087c0adc80 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.773+0000 7f0882e93700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f087c0664f0 0x7f087c14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.773+0000 7f0882e93700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f086005f270 0x7f0860061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.773+0000 7f0882e93700 1 --2- 172.21.15.29:0/3049196985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c134cc0 0x7f087c0ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:42.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.773+0000 7f0882e93700 1 -- 172.21.15.29:0/3049196985 >> 172.21.15.29:0/3049196985 conn(0x7f087c0c8bb0 msgr2=0x7f087c0bf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:42.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.773+0000 7f0882e93700 1 -- 172.21.15.29:0/3049196985 shutdown_connections 2024-01-24T15:30:42.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:42.773+0000 7f0882e93700 1 -- 172.21.15.29:0/3049196985 wait complete. 2024-01-24T15:30:42.776 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 90 2024-01-24T15:30:42.872 INFO:tasks.ceph.mds.a.smithi029.stderr:2024-01-24T15:30:42.870+0000 7f422e68c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T15:30:43.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T15:30:37.411326+0000', 'modified': '2024-01-24T15:30:40.428239+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9124}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9124': {'gid': 9124, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/2886498894', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6832', 'nonce': 2886498894}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 2886498894}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-24T15:30:43.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T15:30:43.192 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-24T15:30:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.331+0000 7f2d4b275700 1 Processor -- start 2024-01-24T15:30:43.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.332+0000 7f2d4b275700 1 -- start start 2024-01-24T15:30:43.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.333+0000 7f2d4b275700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2d44069240 0x7f2d44150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.333+0000 7f2d4b275700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2d441514a0 0x7f2d44151950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.333+0000 7f2d4b275700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4406a350 con 0x7f2d44150bd0 2024-01-24T15:30:43.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.333+0000 7f2d4b275700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d44069740 con 0x7f2d44069240 2024-01-24T15:30:43.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.333+0000 7f2d4b275700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d44069a20 con 0x7f2d441514a0 2024-01-24T15:30:43.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.333+0000 7f2d48810700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2d44069240 0x7f2d44150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.333+0000 7f2d48810700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2d44069240 0x7f2d44150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37432/0 (socket says 172.21.15.29:37432) 2024-01-24T15:30:43.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.333+0000 7f2d48810700 1 -- 172.21.15.29:0/1905515041 learned_addr learned my addr 172.21.15.29:0/1905515041 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:43.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.333+0000 7f2d49011700 1 --1- 172.21.15.29:0/1905515041 >> v1:172.21.15.29:6789/0 conn(0x7f2d44150bd0 0x7f2d44065e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:42016/0 (socket says 172.21.15.29:42016) 2024-01-24T15:30:43.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.333+0000 7f2d49812700 1 --2- 172.21.15.29:0/1905515041 >> v2:172.21.15.163:3300/0 conn(0x7f2d441514a0 0x7f2d44151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d48810700 1 -- 172.21.15.29:0/1905515041 >> v2:172.21.15.163:3300/0 conn(0x7f2d441514a0 msgr2=0x7f2d44151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d48810700 1 --2- 172.21.15.29:0/1905515041 >> v2:172.21.15.163:3300/0 conn(0x7f2d441514a0 0x7f2d44151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d48810700 1 -- 172.21.15.29:0/1905515041 >> v1:172.21.15.29:6789/0 conn(0x7f2d44150bd0 legacy=0x7f2d44065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d3bfff700 1 -- 172.21.15.29:0/1905515041 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560732719 0 0) 0x7f2d4406a350 con 0x7f2d44150bd0 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d48810700 1 -- 172.21.15.29:0/1905515041 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d44069d00 con 0x7f2d44069240 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d49812700 1 --2- 172.21.15.29:0/1905515041 >> v2:172.21.15.163:3300/0 conn(0x7f2d441514a0 0x7f2d44151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d48810700 1 --2- 172.21.15.29:0/1905515041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d44069240 0x7f2d44150fb0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7f2d440be950 tx=0x7f2d340090f0).ready entity=mon.0 client_cookie=6e94a58107aa93f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d3bfff700 1 -- 172.21.15.29:0/1905515041 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3400b600 con 0x7f2d44069240 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d3bfff700 1 -- 172.21.15.29:0/1905515041 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3400b760 con 0x7f2d44069240 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d3bfff700 1 -- 172.21.15.29:0/1905515041 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3400bae0 con 0x7f2d44069240 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d4b275700 1 -- 172.21.15.29:0/1905515041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d44069240 msgr2=0x7f2d44150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d4b275700 1 --2- 172.21.15.29:0/1905515041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d44069240 0x7f2d44150fb0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7f2d440be950 tx=0x7f2d340090f0).stop 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d4b275700 1 -- 172.21.15.29:0/1905515041 shutdown_connections 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d4b275700 1 --2- 172.21.15.29:0/1905515041 >> v2:172.21.15.163:3300/0 conn(0x7f2d441514a0 0x7f2d44151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d4b275700 1 --2- 172.21.15.29:0/1905515041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d44069240 0x7f2d44150fb0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.338 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.334+0000 7f2d4b275700 1 -- 172.21.15.29:0/1905515041 >> 172.21.15.29:0/1905515041 conn(0x7f2d440c8580 msgr2=0x7f2d440c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:43.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.335+0000 7f2d4b275700 1 -- 172.21.15.29:0/1905515041 shutdown_connections 2024-01-24T15:30:43.339 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.335+0000 7f2d4b275700 1 -- 172.21.15.29:0/1905515041 wait complete. 2024-01-24T15:30:43.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.336+0000 7f2d4b275700 1 Processor -- start 2024-01-24T15:30:43.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.336+0000 7f2d4b275700 1 -- start start 2024-01-24T15:30:43.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d4b275700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d44069240 0x7f2d440aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d4b275700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d44150bd0 0x7f2d44136ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d4b275700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d441514a0 0x7f2d441373b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d4b275700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d44153290 con 0x7f2d44069240 2024-01-24T15:30:43.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d4b275700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d441378a0 con 0x7f2d441514a0 2024-01-24T15:30:43.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d4b275700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d44152680 con 0x7f2d44150bd0 2024-01-24T15:30:43.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d49011700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d44069240 0x7f2d440aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d49011700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d44069240 0x7f2d440aac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37434/0 (socket says 172.21.15.29:37434) 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d49011700 1 -- 172.21.15.29:0/4275803813 learned_addr learned my addr 172.21.15.29:0/4275803813 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d49812700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d441514a0 0x7f2d441373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d48810700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d44150bd0 0x7f2d44136ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d49011700 1 -- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d44150bd0 msgr2=0x7f2d44136ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d49011700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d44150bd0 0x7f2d44136ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d49011700 1 -- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d441514a0 msgr2=0x7f2d441373b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d49011700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d441514a0 0x7f2d441373b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d49011700 1 -- 172.21.15.29:0/4275803813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d34009370 con 0x7f2d44069240 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d48810700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d44150bd0 0x7f2d44136ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d49812700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d441514a0 0x7f2d441373b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.337+0000 7f2d49011700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d44069240 0x7f2d440aac60 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 rx=0x7f2d4000c560 tx=0x7f2d40007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.338+0000 7f2d397fa700 1 -- 172.21.15.29:0/4275803813 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d400072e0 con 0x7f2d44069240 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.338+0000 7f2d397fa700 1 -- 172.21.15.29:0/4275803813 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d40007440 con 0x7f2d44069240 2024-01-24T15:30:43.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.338+0000 7f2d4b275700 1 -- 172.21.15.29:0/4275803813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d44152960 con 0x7f2d44069240 2024-01-24T15:30:43.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.338+0000 7f2d397fa700 1 -- 172.21.15.29:0/4275803813 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d400058f0 con 0x7f2d44069240 2024-01-24T15:30:43.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.338+0000 7f2d4b275700 1 -- 172.21.15.29:0/4275803813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d44152f20 con 0x7f2d44069240 2024-01-24T15:30:43.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.339+0000 7f2d397fa700 1 -- 172.21.15.29:0/4275803813 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2d40005ce0 con 0x7f2d44069240 2024-01-24T15:30:43.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.340+0000 7f2d397fa700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2d2805f270 0x7f2d28061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.340+0000 7f2d397fa700 1 -- 172.21.15.29:0/4275803813 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f2d4007a100 con 0x7f2d44069240 2024-01-24T15:30:43.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.340+0000 7f2d48810700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2d2805f270 0x7f2d28061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.340+0000 7f2d48810700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2d2805f270 0x7f2d28061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f2d440be950 tx=0x7f2d34017000).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:43.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.342+0000 7f2d5cc40b80 1 Processor -- start 2024-01-24T15:30:43.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.342+0000 7f2d5cc40b80 1 -- start start 2024-01-24T15:30:43.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d5cc40b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f65a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d5cc40b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 0x555c78f60900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d5cc40b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f61270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d5cc40b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c78edc3d0 con 0x555c78f60df0 2024-01-24T15:30:43.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d5cc40b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c78ea1be0 con 0x555c78f65620 2024-01-24T15:30:43.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d5cc40b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c78eb7260 con 0x555c78fae2c0 2024-01-24T15:30:43.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d49812700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f61270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d49812700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f61270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37438/0 (socket says 172.21.15.29:37438) 2024-01-24T15:30:43.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d49812700 1 -- 172.21.15.29:0/1428320272 learned_addr learned my addr 172.21.15.29:0/1428320272 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:43.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d49011700 1 --2- 172.21.15.29:0/1428320272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f65a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d48810700 1 --2- 172.21.15.29:0/1428320272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 0x555c78f60900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d49812700 1 -- 172.21.15.29:0/1428320272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 msgr2=0x555c78f60900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d49812700 1 --2- 172.21.15.29:0/1428320272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 0x555c78f60900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d49812700 1 -- 172.21.15.29:0/1428320272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 msgr2=0x555c78f65a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d49812700 1 --2- 172.21.15.29:0/1428320272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f65a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d49812700 1 -- 172.21.15.29:0/1428320272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c78eb02f0 con 0x555c78f60df0 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d49011700 1 --2- 172.21.15.29:0/1428320272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f65a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d49812700 1 --2- 172.21.15.29:0/1428320272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f61270 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 rx=0x7f2d3c00d380 tx=0x7f2d3c0079a0).ready entity=mon.0 client_cookie=19e83ee2d550b2d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.343+0000 7f2d2e7fc700 1 -- 172.21.15.29:0/1428320272 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c017420 con 0x555c78f60df0 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.344+0000 7f2d2e7fc700 1 -- 172.21.15.29:0/1428320272 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c017580 con 0x555c78f60df0 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.344+0000 7f2d2e7fc700 1 -- 172.21.15.29:0/1428320272 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c0176f0 con 0x555c78f60df0 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.344+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/1428320272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 msgr2=0x555c78f61270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.344+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/1428320272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f61270 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 rx=0x7f2d3c00d380 tx=0x7f2d3c0079a0).stop 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.344+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/1428320272 shutdown_connections 2024-01-24T15:30:43.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.344+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/1428320272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f61270 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.344+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/1428320272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f65a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.344+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/1428320272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 0x555c78f60900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.344+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/1428320272 >> 172.21.15.29:0/1428320272 conn(0x555c78dce2e0 msgr2=0x555c78fae750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:43.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.344+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/1428320272 shutdown_connections 2024-01-24T15:30:43.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.344+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/1428320272 wait complete. 2024-01-24T15:30:43.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.345+0000 7f2d5cc40b80 1 -- start start 2024-01-24T15:30:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.345+0000 7f2d5cc40b80 1 Processor -- start 2024-01-24T15:30:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d5cc40b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f8ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d5cc40b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f8d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d5cc40b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 0x555c78f903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d5cc40b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c78eb73a0 con 0x555c78f60df0 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d5cc40b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c78eb7620 con 0x555c78f65620 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d5cc40b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c78e82990 con 0x555c78fae2c0 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d5cc40b80 10 client.0 ll_register_callbacks cb 0x555c78f25610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d49011700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f8ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d49011700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f8ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37452/0 (socket says 172.21.15.29:37452) 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d49011700 1 -- 172.21.15.29:0/119059791 learned_addr learned my addr 172.21.15.29:0/119059791 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d48810700 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f8d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d49812700 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 0x555c78f903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d49011700 1 -- 172.21.15.29:0/119059791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 msgr2=0x555c78f903e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d49011700 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 0x555c78f903e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d49011700 1 -- 172.21.15.29:0/119059791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 msgr2=0x555c78f8d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.346+0000 7f2d49011700 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f8d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d49011700 1 -- 172.21.15.29:0/119059791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3c007c80 con 0x555c78f60df0 2024-01-24T15:30:43.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d49011700 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f8ed20 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 rx=0x7f2d4004e720 tx=0x7f2d40050040).ready entity=mon.0 client_cookie=ad1b6e9d138b0f23 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d237fe700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d237fe700 1 -- 172.21.15.29:0/119059791 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4004ce90 con 0x555c78f60df0 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d237fe700 1 -- 172.21.15.29:0/119059791 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f2d4004df10 con 0x555c78f60df0 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/119059791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c78f5fdf0 con 0x555c78f60df0 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d237fe700 1 -- 172.21.15.29:0/119059791 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d4004a810 con 0x555c78f60df0 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d5cc40b80 10 client.9711 fetch_fsmap learned FSMap version 90 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/119059791 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x555c78eab520 con 0x555c78f60df0 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d237fe700 1 -- 172.21.15.29:0/119059791 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c78f5fdf0 con 0x555c78f60df0 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d237fe700 1 -- 172.21.15.29:0/119059791 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2d4004a970 con 0x555c78f60df0 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d5cc40b80 10 client.9711 fetch_fsmap finished waiting for FSMap version 90 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d5cc40b80 10 client.9711 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d5cc40b80 20 client.9711 populate_metadata read hostname 'smithi029' 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.29:6832/2886498894,v1:172.21.15.29:6835/2886498894] conn(0x555c78fbdea0 0x555c78fbe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d5cc40b80 4 client.9711 mds_command: new command op to 9124 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.347+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/119059791 --> [v2:172.21.15.29:6832/2886498894,v1:172.21.15.29:6835/2886498894] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c78eacd60 con 0x555c78fbdea0 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.348+0000 7f2d48810700 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.29:6832/2886498894,v1:172.21.15.29:6835/2886498894] conn(0x555c78fbdea0 0x555c78fbe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.348+0000 7f2d237fe700 10 client.9711 ms_handle_connect on v2:172.21.15.29:6832/2886498894 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.348+0000 7f2d48810700 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.29:6832/2886498894,v1:172.21.15.29:6835/2886498894] conn(0x555c78fbdea0 0x555c78fbe280 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=145eb9e9dc84d230 server_cookie=bb79923fe6fdd3fe in_seq=0 out_seq=0 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.348+0000 7f2d237fe700 1 -- 172.21.15.29:0/119059791 <== mds.0 v2:172.21.15.29:6832/2886498894 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x555c78eacd60 con 0x555c78fbdea0 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.348+0000 7f2d237fe700 10 client.9711 handle_command_reply: tid=0 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d5cc40b80 1 client.9711 shutdown 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/119059791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 msgr2=0x555c78f8ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f8ed20 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 rx=0x7f2d4004e720 tx=0x7f2d40050040).stop 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/119059791 shutdown_connections 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.29:6832/2886498894,v1:172.21.15.29:6835/2886498894] conn(0x555c78fbdea0 0x555c78fbe280 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 0x555c78f903e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f8ed20 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/119059791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f8d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d237fe700 0 client.9711 ms_handle_reset on v2:172.21.15.29:6832/2886498894 2024-01-24T15:30:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/119059791 >> 172.21.15.29:0/119059791 conn(0x555c78dce2e0 msgr2=0x555c78fa9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/119059791 shutdown_connections 2024-01-24T15:30:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/119059791 wait complete. 2024-01-24T15:30:43.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.349+0000 7f2d5cc40b80 20 client.9711 trim_cache size 0 max 16384 2024-01-24T15:30:43.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.366+0000 7f2d5cc40b80 1 Processor -- start 2024-01-24T15:30:43.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.366+0000 7f2d5cc40b80 1 -- start start 2024-01-24T15:30:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d5cc40b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f61970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d5cc40b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f5c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d5cc40b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 0x555c78f8cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d5cc40b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c78faaa60 con 0x555c78f60df0 2024-01-24T15:30:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d5cc40b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c78f662f0 con 0x555c78f65620 2024-01-24T15:30:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d5cc40b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c78f8ff60 con 0x555c78fae2c0 2024-01-24T15:30:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d49011700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f61970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d49011700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f61970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37466/0 (socket says 172.21.15.29:37466) 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d49011700 1 -- 172.21.15.29:0/1967436756 learned_addr learned my addr 172.21.15.29:0/1967436756 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d49812700 1 --2- 172.21.15.29:0/1967436756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 0x555c78f8cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d48810700 1 --2- 172.21.15.29:0/1967436756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f5c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d49011700 1 -- 172.21.15.29:0/1967436756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 msgr2=0x555c78f8cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d49011700 1 --2- 172.21.15.29:0/1967436756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 0x555c78f8cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d49011700 1 -- 172.21.15.29:0/1967436756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 msgr2=0x555c78f5c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d49011700 1 --2- 172.21.15.29:0/1967436756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f5c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.367+0000 7f2d49011700 1 -- 172.21.15.29:0/1967436756 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c78eab270 con 0x555c78f60df0 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d49011700 1 --2- 172.21.15.29:0/1967436756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f61970 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 rx=0x7f2d4004e950 tx=0x7f2d40063000).ready entity=mon.0 client_cookie=7350cd22ae55b200 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d2effd700 1 -- 172.21.15.29:0/1967436756 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4006d070 con 0x555c78f60df0 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d2effd700 1 -- 172.21.15.29:0/1967436756 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4000e040 con 0x555c78f60df0 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d2effd700 1 -- 172.21.15.29:0/1967436756 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4004ce90 con 0x555c78f60df0 2024-01-24T15:30:43.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/1967436756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 msgr2=0x555c78f61970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/1967436756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f61970 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 rx=0x7f2d4004e950 tx=0x7f2d40063000).stop 2024-01-24T15:30:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/1967436756 shutdown_connections 2024-01-24T15:30:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/1967436756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78fae2c0 0x555c78f8cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/1967436756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f61970 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/1967436756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78f65620 0x555c78f5c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/1967436756 >> 172.21.15.29:0/1967436756 conn(0x555c78dce2e0 msgr2=0x555c78fae750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/1967436756 shutdown_connections 2024-01-24T15:30:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.368+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/1967436756 wait complete. 2024-01-24T15:30:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.369+0000 7f2d5cc40b80 1 -- start start 2024-01-24T15:30:43.373 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.369+0000 7f2d5cc40b80 1 Processor -- start 2024-01-24T15:30:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.370+0000 7f2d5cc40b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f8f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.370+0000 7f2d5cc40b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78f65620 0x555c78f8cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.370+0000 7f2d5cc40b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78fae2c0 0x555c78f8d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.370+0000 7f2d5cc40b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c78f8fb50 con 0x555c78f60df0 2024-01-24T15:30:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.370+0000 7f2d5cc40b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c78f8d8d0 con 0x555c78fae2c0 2024-01-24T15:30:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.370+0000 7f2d5cc40b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c78f93180 con 0x555c78f65620 2024-01-24T15:30:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.370+0000 7f2d5cc40b80 10 client.0 ll_register_callbacks cb 0x555c78f25610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T15:30:43.377 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.370+0000 7f2d49011700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f8f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.370+0000 7f2d49011700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f8f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:37470/0 (socket says 172.21.15.29:37470) 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.370+0000 7f2d49011700 1 -- 172.21.15.29:0/247568526 learned_addr learned my addr 172.21.15.29:0/247568526 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.370+0000 7f2d49812700 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78fae2c0 0x555c78f8d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.370+0000 7f2d48810700 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78f65620 0x555c78f8cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d49011700 1 -- 172.21.15.29:0/247568526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78f65620 msgr2=0x555c78f8cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d49011700 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78f65620 0x555c78f8cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d49011700 1 -- 172.21.15.29:0/247568526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78fae2c0 msgr2=0x555c78f8d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d49011700 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78fae2c0 0x555c78f8d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d49011700 1 -- 172.21.15.29:0/247568526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c78eab520 con 0x555c78f60df0 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d49011700 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f8f660 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7f2d4004e720 tx=0x7f2d4006d000).ready entity=mon.0 client_cookie=3fc379197579d293 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d237fe700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d237fe700 1 -- 172.21.15.29:0/247568526 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4004aa20 con 0x555c78f60df0 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d237fe700 1 -- 172.21.15.29:0/247568526 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f2d4004cdc0 con 0x555c78f60df0 2024-01-24T15:30:43.378 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/247568526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c78ea37d0 con 0x555c78f60df0 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d237fe700 1 -- 172.21.15.29:0/247568526 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d40077080 con 0x555c78f60df0 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d5cc40b80 10 client.9717 fetch_fsmap learned FSMap version 90 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/247568526 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x555c78d24f80 con 0x555c78f60df0 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d237fe700 1 -- 172.21.15.29:0/247568526 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c78ea37d0 con 0x555c78f60df0 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d237fe700 1 -- 172.21.15.29:0/247568526 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2d400771e0 con 0x555c78f60df0 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d5cc40b80 10 client.9717 fetch_fsmap finished waiting for FSMap version 90 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d5cc40b80 10 client.9717 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d5cc40b80 20 client.9717 populate_metadata read hostname 'smithi029' 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.29:6832/2886498894,v1:172.21.15.29:6835/2886498894] conn(0x555c78f917d0 0x555c78f91bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d5cc40b80 4 client.9717 mds_command: new command op to 9124 tid=0[{"prefix": "status"}] 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.371+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/247568526 --> [v2:172.21.15.29:6832/2886498894,v1:172.21.15.29:6835/2886498894] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555c78d1fca0 con 0x555c78f917d0 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d48810700 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.29:6832/2886498894,v1:172.21.15.29:6835/2886498894] conn(0x555c78f917d0 0x555c78f91bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d237fe700 10 client.9717 ms_handle_connect on v2:172.21.15.29:6832/2886498894 2024-01-24T15:30:43.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d48810700 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.29:6832/2886498894,v1:172.21.15.29:6835/2886498894] conn(0x555c78f917d0 0x555c78f91bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ae1e4d1ed63be7c1 server_cookie=b496de4ae75f691c in_seq=0 out_seq=0 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d237fe700 1 -- 172.21.15.29:0/247568526 <== mds.0 v2:172.21.15.29:6832/2886498894 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x555c78d1fca0 con 0x555c78f917d0 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d237fe700 10 client.9717 handle_command_reply: tid=0 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d5cc40b80 1 client.9717 shutdown 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/247568526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 msgr2=0x555c78f8f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f8f660 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7f2d4004e720 tx=0x7f2d4006d000).stop 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/247568526 shutdown_connections 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.29:6832/2886498894,v1:172.21.15.29:6835/2886498894] conn(0x555c78f917d0 0x555c78f91bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c78fae2c0 0x555c78f8d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555c78f60df0 0x555c78f8f660 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d237fe700 0 client.9717 ms_handle_reset on v2:172.21.15.29:6832/2886498894 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d5cc40b80 1 --2- 172.21.15.29:0/247568526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555c78f65620 0x555c78f8cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/247568526 >> 172.21.15.29:0/247568526 conn(0x555c78dce2e0 msgr2=0x555c78f900b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/247568526 shutdown_connections 2024-01-24T15:30:43.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.372+0000 7f2d5cc40b80 1 -- 172.21.15.29:0/247568526 wait complete. 2024-01-24T15:30:43.381 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:30:43.381 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T15:30:43.381 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T15:30:43.381 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 9124, 2024-01-24T15:30:43.381 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T15:30:43.381 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T15:30:43.381 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T15:30:43.381 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 5.9594683570000004, 2024-01-24T15:30:43.381 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 90, 2024-01-24T15:30:43.381 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 213, 2024-01-24T15:30:43.381 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 213, 2024-01-24T15:30:43.382 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 11.502633272000001 2024-01-24T15:30:43.382 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:30:43.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.373+0000 7f2d5cc40b80 20 client.9717 trim_cache size 0 max 16384 2024-01-24T15:30:43.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.373+0000 7f2d4b275700 1 -- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2d2805f270 msgr2=0x7f2d28061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.373+0000 7f2d4b275700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2d2805f270 0x7f2d28061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f2d440be950 tx=0x7f2d34017000).stop 2024-01-24T15:30:43.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.373+0000 7f2d4b275700 1 -- 172.21.15.29:0/4275803813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d44069240 msgr2=0x7f2d440aac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.373+0000 7f2d4b275700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d44069240 0x7f2d440aac60 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 rx=0x7f2d4000c560 tx=0x7f2d40007c90).stop 2024-01-24T15:30:43.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.373+0000 7f2d4b275700 1 -- 172.21.15.29:0/4275803813 shutdown_connections 2024-01-24T15:30:43.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.373+0000 7f2d4b275700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d441514a0 0x7f2d441373b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.373+0000 7f2d4b275700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d44069240 0x7f2d440aac60 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.373+0000 7f2d4b275700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2d2805f270 0x7f2d28061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.373+0000 7f2d4b275700 1 --2- 172.21.15.29:0/4275803813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d44150bd0 0x7f2d44136ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.373+0000 7f2d4b275700 1 -- 172.21.15.29:0/4275803813 >> 172.21.15.29:0/4275803813 conn(0x7f2d440c8580 msgr2=0x7f2d44153f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:43.383 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.373+0000 7f2d4b275700 1 -- 172.21.15.29:0/4275803813 shutdown_connections 2024-01-24T15:30:43.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.374+0000 7f2d4b275700 1 -- 172.21.15.29:0/4275803813 wait complete. 2024-01-24T15:30:43.389 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T15:30:43.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.503+0000 7f566c5e3700 1 Processor -- start 2024-01-24T15:30:43.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.503+0000 7f566c5e3700 1 -- start start 2024-01-24T15:30:43.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f566c5e3700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f56640c0d60 0x7f5664154430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f566c5e3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5664154920 0x7f5664154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.507 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f566c5e3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566406aa10 con 0x7f5664154050 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f566c5e3700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56640a82f0 con 0x7f56640c0d60 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f566c5e3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5664155360 con 0x7f5664154920 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f5669b7e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f56640c0d60 0x7f5664154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f5669b7e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f56640c0d60 0x7f5664154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56986/0 (socket says 172.21.15.29:56986) 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f5669b7e700 1 -- 172.21.15.29:0/1535692956 learned_addr learned my addr 172.21.15.29:0/1535692956 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f566ab80700 1 --2- 172.21.15.29:0/1535692956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5664154920 0x7f5664154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f5669b7e700 1 -- 172.21.15.29:0/1535692956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5664154920 msgr2=0x7f5664154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f5669b7e700 1 --2- 172.21.15.29:0/1535692956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5664154920 0x7f5664154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f5669b7e700 1 -- 172.21.15.29:0/1535692956 >> v1:172.21.15.163:6789/0 conn(0x7f5664154050 legacy=0x7f56640c0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f5669b7e700 1 -- 172.21.15.29:0/1535692956 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5664155640 con 0x7f56640c0d60 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.504+0000 7f5669b7e700 1 --2- 172.21.15.29:0/1535692956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56640c0d60 0x7f5664154430 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7f5654006fd0 tx=0x7f56540090f0).ready entity=mon.0 client_cookie=23af934ba920de3a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.505+0000 7f566937d700 1 -- 172.21.15.29:0/1535692956 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565400b580 con 0x7f56640c0d60 2024-01-24T15:30:43.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.505+0000 7f566937d700 1 -- 172.21.15.29:0/1535692956 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565400b6e0 con 0x7f56640c0d60 2024-01-24T15:30:43.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.505+0000 7f566937d700 1 -- 172.21.15.29:0/1535692956 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565400ba60 con 0x7f56640c0d60 2024-01-24T15:30:43.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.505+0000 7f566c5e3700 1 -- 172.21.15.29:0/1535692956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56640c0d60 msgr2=0x7f5664154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.505+0000 7f566c5e3700 1 --2- 172.21.15.29:0/1535692956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56640c0d60 0x7f5664154430 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7f5654006fd0 tx=0x7f56540090f0).stop 2024-01-24T15:30:43.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.505+0000 7f566c5e3700 1 -- 172.21.15.29:0/1535692956 shutdown_connections 2024-01-24T15:30:43.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.505+0000 7f566c5e3700 1 --2- 172.21.15.29:0/1535692956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5664154920 0x7f5664154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.505+0000 7f566c5e3700 1 --2- 172.21.15.29:0/1535692956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56640c0d60 0x7f5664154430 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.505+0000 7f566c5e3700 1 -- 172.21.15.29:0/1535692956 >> 172.21.15.29:0/1535692956 conn(0x7f5664084bb0 msgr2=0x7f56640944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:43.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.505+0000 7f566c5e3700 1 -- 172.21.15.29:0/1535692956 shutdown_connections 2024-01-24T15:30:43.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.505+0000 7f566c5e3700 1 -- 172.21.15.29:0/1535692956 wait complete. 2024-01-24T15:30:43.511 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.507+0000 7f566c5e3700 1 Processor -- start 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.507+0000 7f566c5e3700 1 -- start start 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.507+0000 7f566c5e3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56640c0d60 0x7f56640aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.507+0000 7f566c5e3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5664154050 0x7f56640b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.507+0000 7f566c5e3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5664154920 0x7f56640c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.507+0000 7f566c5e3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5664150c00 con 0x7f5664154920 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.507+0000 7f566c5e3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640ae820 con 0x7f5664154050 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.507+0000 7f566c5e3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56640adef0 con 0x7f56640c0d60 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566ab80700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5664154920 0x7f56640c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566ab80700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5664154920 0x7f56640c30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56994/0 (socket says 172.21.15.29:56994) 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566ab80700 1 -- 172.21.15.29:0/3316159336 learned_addr learned my addr 172.21.15.29:0/3316159336 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566a37f700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56640c0d60 0x7f56640aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f5669b7e700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5664154050 0x7f56640b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566ab80700 1 -- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56640c0d60 msgr2=0x7f56640aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566ab80700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56640c0d60 0x7f56640aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566ab80700 1 -- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5664154050 msgr2=0x7f56640b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566ab80700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5664154050 0x7f56640b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566ab80700 1 -- 172.21.15.29:0/3316159336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5654009370 con 0x7f5664154920 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566a37f700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56640c0d60 0x7f56640aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f5669b7e700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5664154050 0x7f56640b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566ab80700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5664154920 0x7f56640c30f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7f565c009df0 tx=0x7f565c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f565a7fc700 1 -- 172.21.15.29:0/3316159336 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c009b50 con 0x7f5664154920 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f565a7fc700 1 -- 172.21.15.29:0/3316159336 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c00f040 con 0x7f5664154920 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566c5e3700 1 -- 172.21.15.29:0/3316159336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56640ae1d0 con 0x7f5664154920 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f565a7fc700 1 -- 172.21.15.29:0/3316159336 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c00cd60 con 0x7f5664154920 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.508+0000 7f566c5e3700 1 -- 172.21.15.29:0/3316159336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56640b0b60 con 0x7f5664154920 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.509+0000 7f566c5e3700 1 -- 172.21.15.29:0/3316159336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56640ae9b0 con 0x7f5664154920 2024-01-24T15:30:43.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.510+0000 7f565a7fc700 1 -- 172.21.15.29:0/3316159336 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f565c013610 con 0x7f5664154920 2024-01-24T15:30:43.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.510+0000 7f565a7fc700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f564805f270 0x7f5648061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:43.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.510+0000 7f565a7fc700 1 -- 172.21.15.29:0/3316159336 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f565c07afb0 con 0x7f5664154920 2024-01-24T15:30:43.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.510+0000 7f566a37f700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f564805f270 0x7f5648061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:43.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.511+0000 7f566a37f700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f564805f270 0x7f5648061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f56600016a0 tx=0x7f5660009040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:43.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.514+0000 7f565a7fc700 1 -- 172.21.15.29:0/3316159336 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f565c0a9b00 con 0x7f5664154920 2024-01-24T15:30:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.744+0000 7f566c5e3700 1 -- 172.21.15.29:0/3316159336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f56640a7ca0 con 0x7f5664154920 2024-01-24T15:30:43.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.744+0000 7f565a7fc700 1 -- 172.21.15.29:0/3316159336 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7f565c04bcd0 con 0x7f5664154920 2024-01-24T15:30:43.746 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-24T15:30:43.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.746+0000 7f566c5e3700 1 -- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f564805f270 msgr2=0x7f5648061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.746+0000 7f566c5e3700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f564805f270 0x7f5648061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f56600016a0 tx=0x7f5660009040).stop 2024-01-24T15:30:43.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.746+0000 7f566c5e3700 1 -- 172.21.15.29:0/3316159336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5664154920 msgr2=0x7f56640c30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:43.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.746+0000 7f566c5e3700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5664154920 0x7f56640c30f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7f565c009df0 tx=0x7f565c00b040).stop 2024-01-24T15:30:43.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.746+0000 7f566c5e3700 1 -- 172.21.15.29:0/3316159336 shutdown_connections 2024-01-24T15:30:43.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.746+0000 7f566c5e3700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5664154920 0x7f56640c30f0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.746+0000 7f566c5e3700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f564805f270 0x7f5648061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.746+0000 7f566c5e3700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56640c0d60 0x7f56640aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.746+0000 7f566c5e3700 1 --2- 172.21.15.29:0/3316159336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5664154050 0x7f56640b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:43.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.746+0000 7f566c5e3700 1 -- 172.21.15.29:0/3316159336 >> 172.21.15.29:0/3316159336 conn(0x7f5664084bb0 msgr2=0x7f5664064180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:43.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.747+0000 7f566c5e3700 1 -- 172.21.15.29:0/3316159336 shutdown_connections 2024-01-24T15:30:43.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:43.747+0000 7f566c5e3700 1 -- 172.21.15.29:0/3316159336 wait complete. 2024-01-24T15:30:43.761 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-24T15:30:43.761 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T15:30:43.761 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T15:30:43.761 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-24T15:30:43.761 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-24T15:30:43.761 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-24T15:30:43.761 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-24T15:30:43.761 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T15:30:43.761 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T15:30:43.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:30:43.762 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T15:30:43.781 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T15:30:43.781 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 36807sec preferred_lft 36807sec 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T15:30:43.782 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::24bc:1cff:fe57:e704/64 scope link 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T15:30:43.783 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T15:30:43.784 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::2034:aaff:fef3:f12b/64 scope link 2024-01-24T15:30:43.784 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:30:43.784 INFO:teuthology.orchestra.run.smithi029.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T15:30:43.784 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether ba:ca:42:4a:9e:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T15:30:43.784 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::b8ca:42ff:fe4a:9e52/64 scope link 2024-01-24T15:30:43.784 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:30:43.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:30:43.784 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T15:30:43.853 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T15:30:43.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:30:43.854 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T15:30:43.918 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T15:30:43.918 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T15:30:43.918 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T15:30:43.918 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T15:30:43.918 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T15:30:43.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:30:43.918 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T15:30:43.918 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T15:30:43.919 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T15:30:43.919 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T15:30:43.919 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T15:30:43.919 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T15:30:44.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:30:44.042 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T15:30:44.063 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-24T15:30:44.063 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T15:30:44.063 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T15:30:44.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:30:44.063 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T15:30:44.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:30:44.139 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-01-24T15:30:44.201 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T15:30:44.201 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T15:30:44.293 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T15:30:44.293 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T15:30:44.317 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T15:30:44.317 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T15:30:44.320 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T15:30:44.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:30:44.320 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-24T15:30:44.352 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-24T15:30:44.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.530+0000 7f6365fda700 1 Processor -- start 2024-01-24T15:30:44.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.530+0000 7f6365fda700 1 -- start start 2024-01-24T15:30:44.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f6365fda700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6360156480 0x7f636015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f6365fda700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6360156b40 con 0x7f6360135080 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f6365fda700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6360153110 con 0x7f6360066bb0 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f6365fda700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f636014cb90 con 0x7f6360156480 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f635ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6360156480 0x7f636015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f635ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6360156480 0x7f636015cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57010/0 (socket says 172.21.15.29:57010) 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f635ffff700 1 -- 172.21.15.29:0/1688992006 learned_addr learned my addr 172.21.15.29:0/1688992006 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f635f7fe700 1 --1- 172.21.15.29:0/1688992006 >> v1:172.21.15.29:6789/0 conn(0x7f6360135080 0x7f63600664a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52656/0 (socket says 172.21.15.29:52656) 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f635ffff700 1 -- 172.21.15.29:0/1688992006 >> v1:172.21.15.163:6789/0 conn(0x7f6360066bb0 legacy=0x7f6360155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f635ffff700 1 -- 172.21.15.29:0/1688992006 >> v1:172.21.15.29:6789/0 conn(0x7f6360135080 legacy=0x7f63600664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f635e7fc700 1 -- 172.21.15.29:0/1688992006 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282900933 0 0) 0x7f6360156b40 con 0x7f6360135080 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f635ffff700 1 -- 172.21.15.29:0/1688992006 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63601568d0 con 0x7f6360156480 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f635ffff700 1 --2- 172.21.15.29:0/1688992006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6360156480 0x7f636015cfc0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f6358009f20 tx=0x7f6358009840).ready entity=mon.0 client_cookie=61006115bbcbbc75 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f635e7fc700 1 -- 172.21.15.29:0/1688992006 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800e070 con 0x7f6360156480 2024-01-24T15:30:44.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f635e7fc700 1 -- 172.21.15.29:0/1688992006 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635800ba80 con 0x7f6360156480 2024-01-24T15:30:44.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.531+0000 7f635e7fc700 1 -- 172.21.15.29:0/1688992006 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800be00 con 0x7f6360156480 2024-01-24T15:30:44.535 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.532+0000 7f6365fda700 1 -- 172.21.15.29:0/1688992006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6360156480 msgr2=0x7f636015cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:44.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.532+0000 7f6365fda700 1 --2- 172.21.15.29:0/1688992006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6360156480 0x7f636015cfc0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f6358009f20 tx=0x7f6358009840).stop 2024-01-24T15:30:44.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.532+0000 7f6365fda700 1 -- 172.21.15.29:0/1688992006 shutdown_connections 2024-01-24T15:30:44.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.532+0000 7f6365fda700 1 --2- 172.21.15.29:0/1688992006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6360156480 0x7f636015cfc0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:44.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.532+0000 7f6365fda700 1 -- 172.21.15.29:0/1688992006 >> 172.21.15.29:0/1688992006 conn(0x7f63600c5640 msgr2=0x7f63600c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:44.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.532+0000 7f6365fda700 1 -- 172.21.15.29:0/1688992006 shutdown_connections 2024-01-24T15:30:44.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.532+0000 7f6365fda700 1 -- 172.21.15.29:0/1688992006 wait complete. 2024-01-24T15:30:44.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.534+0000 7f6365fda700 1 Processor -- start 2024-01-24T15:30:44.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.534+0000 7f6365fda700 1 -- start start 2024-01-24T15:30:44.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.534+0000 7f6365fda700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6360066bb0 0x7f636014a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:44.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.534+0000 7f6365fda700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6360135080 0x7f6360165700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:44.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.534+0000 7f6365fda700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6360156480 0x7f6360165bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:44.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.534+0000 7f6365fda700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6360135dd0 con 0x7f6360066bb0 2024-01-24T15:30:44.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.534+0000 7f6365fda700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63600c2810 con 0x7f6360156480 2024-01-24T15:30:44.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.534+0000 7f6365fda700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6360166250 con 0x7f6360135080 2024-01-24T15:30:44.538 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f635f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6360066bb0 0x7f636014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f635f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6360066bb0 0x7f636014a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57018/0 (socket says 172.21.15.29:57018) 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f635f7fe700 1 -- 172.21.15.29:0/263840219 learned_addr learned my addr 172.21.15.29:0/263840219 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f635ffff700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6360156480 0x7f6360165bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f635effd700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6360135080 0x7f6360165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f635f7fe700 1 -- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6360135080 msgr2=0x7f6360165700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f635f7fe700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6360135080 0x7f6360165700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f635f7fe700 1 -- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6360156480 msgr2=0x7f6360165bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f635f7fe700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6360156480 0x7f6360165bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f635f7fe700 1 -- 172.21.15.29:0/263840219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358009ae0 con 0x7f6360066bb0 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f635f7fe700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6360066bb0 0x7f636014a770 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7f635000c4c0 tx=0x7f6350007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f63477fe700 1 -- 172.21.15.29:0/263840219 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635000ce30 con 0x7f6360066bb0 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f63477fe700 1 -- 172.21.15.29:0/263840219 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63500072e0 con 0x7f6360066bb0 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f6365fda700 1 -- 172.21.15.29:0/263840219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6360166530 con 0x7f6360066bb0 2024-01-24T15:30:44.539 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f63477fe700 1 -- 172.21.15.29:0/263840219 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63500056e0 con 0x7f6360066bb0 2024-01-24T15:30:44.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.535+0000 7f6365fda700 1 -- 172.21.15.29:0/263840219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6360135a60 con 0x7f6360066bb0 2024-01-24T15:30:44.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.537+0000 7f6365fda700 1 -- 172.21.15.29:0/263840219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63600475e0 con 0x7f6360066bb0 2024-01-24T15:30:44.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.537+0000 7f63477fe700 1 -- 172.21.15.29:0/263840219 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6350005a90 con 0x7f6360066bb0 2024-01-24T15:30:44.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.537+0000 7f63477fe700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f634c05f270 0x7f634c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:44.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.537+0000 7f63477fe700 1 -- 172.21.15.29:0/263840219 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f6350079e60 con 0x7f6360066bb0 2024-01-24T15:30:44.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.537+0000 7f635effd700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f634c05f270 0x7f634c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:44.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.538+0000 7f635effd700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f634c05f270 0x7f634c061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f63600bf7e0 tx=0x7f6354007d40).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:44.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.542+0000 7f63477fe700 1 -- 172.21.15.29:0/263840219 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f635004ab80 con 0x7f6360066bb0 2024-01-24T15:30:44.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.751+0000 7f6365fda700 1 -- 172.21.15.29:0/263840219 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6360135f10 con 0x7f634c05f270 2024-01-24T15:30:44.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.753+0000 7f63477fe700 1 -- 172.21.15.29:0/263840219 <== mgr.8938 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6354003590 con 0x7f634c05f270 2024-01-24T15:30:44.755 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T15:30:44.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:30:44.755 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-01-24T15:30:44.755 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:30:44.755 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T15:30:44.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.754+0000 7f6365fda700 1 -- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f634c05f270 msgr2=0x7f634c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:44.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.754+0000 7f6365fda700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f634c05f270 0x7f634c061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f63600bf7e0 tx=0x7f6354007d40).stop 2024-01-24T15:30:44.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.754+0000 7f6365fda700 1 -- 172.21.15.29:0/263840219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6360066bb0 msgr2=0x7f636014a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:44.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.754+0000 7f6365fda700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6360066bb0 0x7f636014a770 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7f635000c4c0 tx=0x7f6350007c10).stop 2024-01-24T15:30:44.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.754+0000 7f6365fda700 1 -- 172.21.15.29:0/263840219 shutdown_connections 2024-01-24T15:30:44.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.754+0000 7f6365fda700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6360156480 0x7f6360165bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:44.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.754+0000 7f6365fda700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6360066bb0 0x7f636014a770 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:44.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.754+0000 7f6365fda700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f634c05f270 0x7f634c061730 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:44.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.754+0000 7f6365fda700 1 --2- 172.21.15.29:0/263840219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6360135080 0x7f6360165700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:44.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.755+0000 7f6365fda700 1 -- 172.21.15.29:0/263840219 >> 172.21.15.29:0/263840219 conn(0x7f63600c5640 msgr2=0x7f636014bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:44.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.755+0000 7f6365fda700 1 -- 172.21.15.29:0/263840219 shutdown_connections 2024-01-24T15:30:44.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.755+0000 7f6365fda700 1 -- 172.21.15.29:0/263840219 wait complete. 2024-01-24T15:30:44.770 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-01-24T15:30:44.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.900+0000 7f17c7426700 1 Processor -- start 2024-01-24T15:30:44.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.901+0000 7f17c7426700 1 -- start start 2024-01-24T15:30:44.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.901+0000 7f17c7426700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f17c0134e70 0x7f17c00c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:44.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.901+0000 7f17c7426700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17c0066c30 0x7f17c0152e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.901+0000 7f17c7426700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c0068470 con 0x7f17c0068900 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.901+0000 7f17c7426700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c014c910 con 0x7f17c0066c30 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.901+0000 7f17c7426700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c0153340 con 0x7f17c0134e70 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17c59c3700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17c0066c30 0x7f17c0152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17c49c1700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f17c0068900 0x7f17c0066520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52658/0 (socket says 172.21.15.29:52658) 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17c49c1700 1 -- 172.21.15.29:0/2618227188 learned_addr learned my addr 172.21.15.29:0/2618227188 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17c51c2700 1 --2- 172.21.15.29:0/2618227188 >> v2:172.21.15.163:3300/0 conn(0x7f17c0134e70 0x7f17c00c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17c59c3700 1 -- 172.21.15.29:0/2618227188 >> v2:172.21.15.163:3300/0 conn(0x7f17c0134e70 msgr2=0x7f17c00c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17c59c3700 1 --2- 172.21.15.29:0/2618227188 >> v2:172.21.15.163:3300/0 conn(0x7f17c0134e70 0x7f17c00c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17c59c3700 1 -- 172.21.15.29:0/2618227188 >> v1:172.21.15.29:6789/0 conn(0x7f17c0068900 legacy=0x7f17c0066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17c59c3700 1 -- 172.21.15.29:0/2618227188 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c00c3860 con 0x7f17c0066c30 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17b7fff700 1 -- 172.21.15.29:0/2618227188 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072379270 0 0) 0x7f17c0068470 con 0x7f17c0068900 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17c59c3700 1 --2- 172.21.15.29:0/2618227188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17c0066c30 0x7f17c0152e50 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f17c00c66b0 tx=0x7f17b8009780).ready entity=mon.0 client_cookie=5478ce39bc74a803 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:44.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17b7fff700 1 -- 172.21.15.29:0/2618227188 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b800e070 con 0x7f17c0066c30 2024-01-24T15:30:44.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17b7fff700 1 -- 172.21.15.29:0/2618227188 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b800bb60 con 0x7f17c0066c30 2024-01-24T15:30:44.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.902+0000 7f17b7fff700 1 -- 172.21.15.29:0/2618227188 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8012410 con 0x7f17c0066c30 2024-01-24T15:30:44.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.903+0000 7f17c7426700 1 -- 172.21.15.29:0/2618227188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17c0066c30 msgr2=0x7f17c0152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:44.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.903+0000 7f17c7426700 1 --2- 172.21.15.29:0/2618227188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17c0066c30 0x7f17c0152e50 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f17c00c66b0 tx=0x7f17b8009780).stop 2024-01-24T15:30:44.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.903+0000 7f17c7426700 1 -- 172.21.15.29:0/2618227188 shutdown_connections 2024-01-24T15:30:44.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.903+0000 7f17c7426700 1 --2- 172.21.15.29:0/2618227188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17c0066c30 0x7f17c0152e50 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:44.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.903+0000 7f17c7426700 1 --2- 172.21.15.29:0/2618227188 >> v2:172.21.15.163:3300/0 conn(0x7f17c0134e70 0x7f17c00c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:44.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.903+0000 7f17c7426700 1 -- 172.21.15.29:0/2618227188 >> 172.21.15.29:0/2618227188 conn(0x7f17c00c8ba0 msgr2=0x7f17c00c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:44.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.903+0000 7f17c7426700 1 -- 172.21.15.29:0/2618227188 shutdown_connections 2024-01-24T15:30:44.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.903+0000 7f17c7426700 1 -- 172.21.15.29:0/2618227188 wait complete. 2024-01-24T15:30:44.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.904+0000 7f17c7426700 1 Processor -- start 2024-01-24T15:30:44.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.904+0000 7f17c7426700 1 -- start start 2024-01-24T15:30:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.905+0000 7f17c7426700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17c0066c30 0x7f17c00a8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.905+0000 7f17c7426700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17c0068900 0x7f17c0134660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.905+0000 7f17c7426700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c0134e70 0x7f17c0064430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.905+0000 7f17c7426700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0134b50 con 0x7f17c0066c30 2024-01-24T15:30:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.905+0000 7f17c7426700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0064920 con 0x7f17c0134e70 2024-01-24T15:30:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.905+0000 7f17c7426700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00a8c10 con 0x7f17c0068900 2024-01-24T15:30:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c51c2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17c0066c30 0x7f17c00a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c51c2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17c0066c30 0x7f17c00a8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57038/0 (socket says 172.21.15.29:57038) 2024-01-24T15:30:44.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c51c2700 1 -- 172.21.15.29:0/4179249168 learned_addr learned my addr 172.21.15.29:0/4179249168 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c51c2700 1 -- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17c0068900 msgr2=0x7f17c0134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c49c1700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17c0068900 0x7f17c0134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c59c3700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c0134e70 0x7f17c0064430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c51c2700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17c0068900 0x7f17c0134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c51c2700 1 -- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c0134e70 msgr2=0x7f17c0064430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c51c2700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c0134e70 0x7f17c0064430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c51c2700 1 -- 172.21.15.29:0/4179249168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b8009a20 con 0x7f17c0066c30 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c51c2700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17c0066c30 0x7f17c00a8580 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7f17b000e930 tx=0x7f17b0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17b57fa700 1 -- 172.21.15.29:0/4179249168 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000eed0 con 0x7f17c0066c30 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17b57fa700 1 -- 172.21.15.29:0/4179249168 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b000c850 con 0x7f17c0066c30 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17b57fa700 1 -- 172.21.15.29:0/4179249168 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000cbd0 con 0x7f17c0066c30 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c7426700 1 -- 172.21.15.29:0/4179249168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c00a8ef0 con 0x7f17c0066c30 2024-01-24T15:30:44.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.906+0000 7f17c7426700 1 -- 172.21.15.29:0/4179249168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c00a94b0 con 0x7f17c0066c30 2024-01-24T15:30:44.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.908+0000 7f17c7426700 1 -- 172.21.15.29:0/4179249168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c00a9b00 con 0x7f17c0066c30 2024-01-24T15:30:44.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.908+0000 7f17b57fa700 1 -- 172.21.15.29:0/4179249168 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 89) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f17b001b400 con 0x7f17c0066c30 2024-01-24T15:30:44.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.908+0000 7f17b57fa700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f17a405f270 0x7f17a4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:44.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.908+0000 7f17b57fa700 1 -- 172.21.15.29:0/4179249168 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f17b007ad70 con 0x7f17c0066c30 2024-01-24T15:30:44.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.909+0000 7f17c49c1700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f17a405f270 0x7f17a4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:44.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.909+0000 7f17c49c1700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f17a405f270 0x7f17a4061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f17bc007fd0 tx=0x7f17bc009040).ready entity=mgr.8938 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:44.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:44.913+0000 7f17b57fa700 1 -- 172.21.15.29:0/4179249168 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f17b004bae0 con 0x7f17c0066c30 2024-01-24T15:30:45.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.146+0000 7f17c7426700 1 -- 172.21.15.29:0/4179249168 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f17c00475e0 con 0x7f17c0066c30 2024-01-24T15:30:45.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.876+0000 7f17b57fa700 1 -- 172.21.15.29:0/4179249168 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 90) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f17b0078590 con 0x7f17c0066c30 2024-01-24T15:30:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.877+0000 7f17b57fa700 1 -- 172.21.15.29:0/4179249168 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v90) v1 ==== 92+0+0 (secure 0 0 0) 0x7f17b00712c0 con 0x7f17c0066c30 2024-01-24T15:30:45.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.880+0000 7f17c7426700 1 -- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f17a405f270 msgr2=0x7f17a4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:45.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.880+0000 7f17c7426700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f17a405f270 0x7f17a4061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f17bc007fd0 tx=0x7f17bc009040).stop 2024-01-24T15:30:45.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.880+0000 7f17c7426700 1 -- 172.21.15.29:0/4179249168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17c0066c30 msgr2=0x7f17c00a8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:45.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.880+0000 7f17c7426700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17c0066c30 0x7f17c00a8580 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7f17b000e930 tx=0x7f17b0009060).stop 2024-01-24T15:30:45.882 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.880+0000 7f17c7426700 1 -- 172.21.15.29:0/4179249168 shutdown_connections 2024-01-24T15:30:45.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.880+0000 7f17c7426700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c0134e70 0x7f17c0064430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:45.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.880+0000 7f17c7426700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17c0066c30 0x7f17c00a8580 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:45.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.880+0000 7f17c7426700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f17a405f270 0x7f17a4061730 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:45.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.880+0000 7f17c7426700 1 --2- 172.21.15.29:0/4179249168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17c0068900 0x7f17c0134660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:45.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.880+0000 7f17c7426700 1 -- 172.21.15.29:0/4179249168 >> 172.21.15.29:0/4179249168 conn(0x7f17c00c8ba0 msgr2=0x7f17c00bec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:45.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.880+0000 7f17c7426700 1 -- 172.21.15.29:0/4179249168 shutdown_connections 2024-01-24T15:30:45.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:45.880+0000 7f17c7426700 1 -- 172.21.15.29:0/4179249168 wait complete. 2024-01-24T15:30:45.899 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-01-24T15:30:46.026 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:46.025+0000 7f62da106680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:30:46.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.034+0000 7f24b01a9700 1 Processor -- start 2024-01-24T15:30:46.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.035+0000 7f24b01a9700 1 -- start start 2024-01-24T15:30:46.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.035+0000 7f24b01a9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f24a8064760 0x7f24a8064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:46.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24b01a9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f24a80c8490 0x7f24a80a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:46.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24b01a9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a8152240 con 0x7f24a8154700 2024-01-24T15:30:46.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24b01a9700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a81586c0 con 0x7f24a80c8490 2024-01-24T15:30:46.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24b01a9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a80c0d50 con 0x7f24a8064760 2024-01-24T15:30:46.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24ad744700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f24a80c8490 0x7f24a80a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:46.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24ad744700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f24a80c8490 0x7f24a80a9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57056/0 (socket says 172.21.15.29:57056) 2024-01-24T15:30:46.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24ad744700 1 -- 172.21.15.29:0/2309568725 learned_addr learned my addr 172.21.15.29:0/2309568725 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:46.040 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24adf45700 1 --2- 172.21.15.29:0/2309568725 >> v2:172.21.15.163:3300/0 conn(0x7f24a8064760 0x7f24a8064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24ad744700 1 -- 172.21.15.29:0/2309568725 >> v2:172.21.15.163:3300/0 conn(0x7f24a8064760 msgr2=0x7f24a8064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24ad744700 1 --2- 172.21.15.29:0/2309568725 >> v2:172.21.15.163:3300/0 conn(0x7f24a8064760 0x7f24a8064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24ad744700 1 -- 172.21.15.29:0/2309568725 >> v1:172.21.15.163:6789/0 conn(0x7f24a8154700 legacy=0x7f24a8154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24ad744700 1 -- 172.21.15.29:0/2309568725 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a80aa670 con 0x7f24a80c8490 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24adf45700 1 --2- 172.21.15.29:0/2309568725 >> v2:172.21.15.163:3300/0 conn(0x7f24a8064760 0x7f24a8064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.036+0000 7f24ad744700 1 --2- 172.21.15.29:0/2309568725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24a80c8490 0x7f24a80a9d90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 rx=0x7f24a8084450 tx=0x7f24980090f0).ready entity=mon.0 client_cookie=3cff3fa03f5847c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.037+0000 7f24acf43700 1 -- 172.21.15.29:0/2309568725 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249800b5b0 con 0x7f24a80c8490 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.037+0000 7f24acf43700 1 -- 172.21.15.29:0/2309568725 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249800b710 con 0x7f24a80c8490 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.037+0000 7f24acf43700 1 -- 172.21.15.29:0/2309568725 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249800ba90 con 0x7f24a80c8490 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.037+0000 7f24b01a9700 1 -- 172.21.15.29:0/2309568725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24a80c8490 msgr2=0x7f24a80a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.037+0000 7f24b01a9700 1 --2- 172.21.15.29:0/2309568725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24a80c8490 0x7f24a80a9d90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 rx=0x7f24a8084450 tx=0x7f24980090f0).stop 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.037+0000 7f24b01a9700 1 -- 172.21.15.29:0/2309568725 shutdown_connections 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.037+0000 7f24b01a9700 1 --2- 172.21.15.29:0/2309568725 >> v2:172.21.15.163:3300/0 conn(0x7f24a8064760 0x7f24a8064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.037+0000 7f24b01a9700 1 --2- 172.21.15.29:0/2309568725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24a80c8490 0x7f24a80a9d90 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.037+0000 7f24b01a9700 1 -- 172.21.15.29:0/2309568725 >> 172.21.15.29:0/2309568725 conn(0x7f24a80cb8b0 msgr2=0x7f24a80c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:46.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.037+0000 7f24b01a9700 1 -- 172.21.15.29:0/2309568725 shutdown_connections 2024-01-24T15:30:46.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.037+0000 7f24b01a9700 1 -- 172.21.15.29:0/2309568725 wait complete. 2024-01-24T15:30:46.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.039+0000 7f24b01a9700 1 Processor -- start 2024-01-24T15:30:46.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.039+0000 7f24b01a9700 1 -- start start 2024-01-24T15:30:46.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.039+0000 7f24b01a9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a8064760 0x7f24a8155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:46.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.039+0000 7f24b01a9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24a80c8490 0x7f24a814fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:46.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.039+0000 7f24b01a9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a8154700 0x7f24a81575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:46.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.039+0000 7f24b01a9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a8155d80 con 0x7f24a80c8490 2024-01-24T15:30:46.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.039+0000 7f24b01a9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80c3af0 con 0x7f24a8064760 2024-01-24T15:30:46.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.039+0000 7f24b01a9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a814ffb0 con 0x7f24a8154700 2024-01-24T15:30:46.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f24ad744700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24a80c8490 0x7f24a814fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f24ad744700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24a80c8490 0x7f24a814fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57062/0 (socket says 172.21.15.29:57062) 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f24ad744700 1 -- 172.21.15.29:0/158884692 learned_addr learned my addr 172.21.15.29:0/158884692 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f24adf45700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a8064760 0x7f24a8155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f24ae746700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a8154700 0x7f24a81575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f24ad744700 1 -- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a8154700 msgr2=0x7f24a81575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f24ad744700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a8154700 0x7f24a81575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f24ad744700 1 -- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a8064760 msgr2=0x7f24a8155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f24ad744700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a8064760 0x7f24a8155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f24ad744700 1 -- 172.21.15.29:0/158884692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a40092a0 con 0x7f24a80c8490 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f24ad744700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24a80c8490 0x7f24a814fac0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 rx=0x7f2498006fd0 tx=0x7f2498017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f249dffb700 1 -- 172.21.15.29:0/158884692 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249800be50 con 0x7f24a80c8490 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f249dffb700 1 -- 172.21.15.29:0/158884692 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249800b880 con 0x7f24a80c8490 2024-01-24T15:30:46.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f24b01a9700 1 -- 172.21.15.29:0/158884692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2498009370 con 0x7f24a80c8490 2024-01-24T15:30:46.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.040+0000 7f249dffb700 1 -- 172.21.15.29:0/158884692 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249801c670 con 0x7f24a80c8490 2024-01-24T15:30:46.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.041+0000 7f24b01a9700 1 -- 172.21.15.29:0/158884692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a8155f60 con 0x7f24a80c8490 2024-01-24T15:30:46.047 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:46.041+0000 7f0fd0a61680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:30:46.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.042+0000 7f24b01a9700 1 -- 172.21.15.29:0/158884692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a80c3ca0 con 0x7f24a80c8490 2024-01-24T15:30:46.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.042+0000 7f249dffb700 1 -- 172.21.15.29:0/158884692 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 90) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f249801c9e0 con 0x7f24a80c8490 2024-01-24T15:30:46.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.043+0000 7f249dffb700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f248c05f2c0 0x7f248c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:46.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.043+0000 7f24adf45700 1 -- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f248c05f2c0 msgr2=0x7f248c061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:46.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.043+0000 7f249dffb700 1 -- 172.21.15.29:0/158884692 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f249807c2a0 con 0x7f24a80c8490 2024-01-24T15:30:46.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.043+0000 7f24adf45700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f248c05f2c0 0x7f248c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:46.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.047+0000 7f249dffb700 1 -- 172.21.15.29:0/158884692 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f24a80c3ca0 con 0x7f24a80c8490 2024-01-24T15:30:46.090 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:46.088+0000 7f62da106680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:30:46.117 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:46.115+0000 7f0fd0a61680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:30:46.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.230+0000 7f24b01a9700 1 -- 172.21.15.29:0/158884692 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f24a80c3ca0 con 0x7f24a80c8490 2024-01-24T15:30:46.235 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.234+0000 7f249dffb700 1 -- 172.21.15.29:0/158884692 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v39)=0 v39) v1 ==== 138+0+0 (secure 0 0 0) 0x7f24a80c3ca0 con 0x7f24a80c8490 2024-01-24T15:30:46.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.236+0000 7f24b01a9700 1 -- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f248c05f2c0 msgr2=0x7f248c061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:46.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.236+0000 7f24b01a9700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f248c05f2c0 0x7f248c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:46.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.236+0000 7f24b01a9700 1 -- 172.21.15.29:0/158884692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24a80c8490 msgr2=0x7f24a814fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:46.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.236+0000 7f24b01a9700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24a80c8490 0x7f24a814fac0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 rx=0x7f2498006fd0 tx=0x7f2498017000).stop 2024-01-24T15:30:46.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.236+0000 7f24b01a9700 1 -- 172.21.15.29:0/158884692 shutdown_connections 2024-01-24T15:30:46.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.236+0000 7f24b01a9700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f248c05f2c0 0x7f248c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:46.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.236+0000 7f24b01a9700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a8154700 0x7f24a81575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.236+0000 7f24b01a9700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a8064760 0x7f24a8155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.236+0000 7f24b01a9700 1 --2- 172.21.15.29:0/158884692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24a80c8490 0x7f24a814fac0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.236+0000 7f24b01a9700 1 -- 172.21.15.29:0/158884692 >> 172.21.15.29:0/158884692 conn(0x7f24a80cb8b0 msgr2=0x7f24a81512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:46.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.236+0000 7f24b01a9700 1 -- 172.21.15.29:0/158884692 shutdown_connections 2024-01-24T15:30:46.239 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.236+0000 7f24b01a9700 1 -- 172.21.15.29:0/158884692 wait complete. 2024-01-24T15:30:46.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:46.253 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-01-24T15:30:46.276 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2024-01-24T15:30:46.364 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:46.363+0000 7f62da106680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:30:46.436 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:46.432+0000 7f0fd0a61680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:30:46.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.451+0000 7f416969a700 1 Processor -- start 2024-01-24T15:30:46.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.451+0000 7f416969a700 1 -- start start 2024-01-24T15:30:46.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f416969a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f416414eb10 0x7f41640c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:46.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f416969a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f41640c7f70 0x7f41640a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:46.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f416969a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4164154dc0 con 0x7f416414e730 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f416969a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41640c6300 con 0x7f416414eb10 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f416969a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4164154830 con 0x7f41640c7f70 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f41627fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f416414eb10 0x7f41640c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f4162ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f416414e730 0x7f4164155250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52684/0 (socket says 172.21.15.29:52684) 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f4162ffd700 1 -- 172.21.15.29:0/657512049 learned_addr learned my addr 172.21.15.29:0/657512049 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f41627fc700 1 --2- 172.21.15.29:0/657512049 >> v2:172.21.15.29:3300/0 conn(0x7f416414eb10 0x7f41640c5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57068/0 (socket says 172.21.15.29:57068) 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f41637fe700 1 --2- 172.21.15.29:0/657512049 >> v2:172.21.15.163:3300/0 conn(0x7f41640c7f70 0x7f41640a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f41627fc700 1 -- 172.21.15.29:0/657512049 >> v2:172.21.15.163:3300/0 conn(0x7f41640c7f70 msgr2=0x7f41640a9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f41627fc700 1 --2- 172.21.15.29:0/657512049 >> v2:172.21.15.163:3300/0 conn(0x7f41640c7f70 0x7f41640a9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f41627fc700 1 -- 172.21.15.29:0/657512049 >> v1:172.21.15.29:6789/0 conn(0x7f416414e730 legacy=0x7f4164155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f41627fc700 1 -- 172.21.15.29:0/657512049 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41640c5470 con 0x7f416414eb10 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f41637fe700 1 --2- 172.21.15.29:0/657512049 >> v2:172.21.15.163:3300/0 conn(0x7f41640c7f70 0x7f41640a9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:30:46.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.452+0000 7f4161ffb700 1 -- 172.21.15.29:0/657512049 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824414432 0 0) 0x7f4164154dc0 con 0x7f416414e730 2024-01-24T15:30:46.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.453+0000 7f41627fc700 1 --2- 172.21.15.29:0/657512049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f416414eb10 0x7f41640c5e10 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 rx=0x7f4164065870 tx=0x7f4158009060).ready entity=mon.0 client_cookie=26f8c8ca92aa554f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:46.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.453+0000 7f4161ffb700 1 -- 172.21.15.29:0/657512049 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415800b580 con 0x7f416414eb10 2024-01-24T15:30:46.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.453+0000 7f4161ffb700 1 -- 172.21.15.29:0/657512049 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415800b6e0 con 0x7f416414eb10 2024-01-24T15:30:46.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.453+0000 7f4161ffb700 1 -- 172.21.15.29:0/657512049 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415800ba60 con 0x7f416414eb10 2024-01-24T15:30:46.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.453+0000 7f416969a700 1 -- 172.21.15.29:0/657512049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f416414eb10 msgr2=0x7f41640c5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:46.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.453+0000 7f416969a700 1 --2- 172.21.15.29:0/657512049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f416414eb10 0x7f41640c5e10 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 rx=0x7f4164065870 tx=0x7f4158009060).stop 2024-01-24T15:30:46.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.453+0000 7f416969a700 1 -- 172.21.15.29:0/657512049 shutdown_connections 2024-01-24T15:30:46.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.453+0000 7f416969a700 1 --2- 172.21.15.29:0/657512049 >> v2:172.21.15.163:3300/0 conn(0x7f41640c7f70 0x7f41640a9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.453+0000 7f416969a700 1 --2- 172.21.15.29:0/657512049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f416414eb10 0x7f41640c5e10 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.453+0000 7f416969a700 1 -- 172.21.15.29:0/657512049 >> 172.21.15.29:0/657512049 conn(0x7f41640cb880 msgr2=0x7f41640c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:46.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.454+0000 7f416969a700 1 -- 172.21.15.29:0/657512049 shutdown_connections 2024-01-24T15:30:46.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.454+0000 7f416969a700 1 -- 172.21.15.29:0/657512049 wait complete. 2024-01-24T15:30:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.455+0000 7f416969a700 1 Processor -- start 2024-01-24T15:30:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.455+0000 7f416969a700 1 -- start start 2024-01-24T15:30:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f416969a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41640c7f70 0x7f4164157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f416969a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f416414e730 0x7f4164157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f416969a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f416414eb10 0x7f4164157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f416969a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416414f660 con 0x7f416414e730 2024-01-24T15:30:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f416969a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41641581f0 con 0x7f416414eb10 2024-01-24T15:30:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f416969a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41641584d0 con 0x7f41640c7f70 2024-01-24T15:30:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f41627fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f416414e730 0x7f4164157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f41627fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f416414e730 0x7f4164157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57082/0 (socket says 172.21.15.29:57082) 2024-01-24T15:30:46.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f41627fc700 1 -- 172.21.15.29:0/2261916537 learned_addr learned my addr 172.21.15.29:0/2261916537 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:46.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f4162ffd700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41640c7f70 0x7f4164157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:46.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f41637fe700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f416414eb10 0x7f4164157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:46.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f41627fc700 1 -- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41640c7f70 msgr2=0x7f4164157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:46.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f41627fc700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41640c7f70 0x7f4164157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f41627fc700 1 -- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f416414eb10 msgr2=0x7f4164157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:46.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f41627fc700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f416414eb10 0x7f4164157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:46.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f41627fc700 1 -- 172.21.15.29:0/2261916537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580092a0 con 0x7f416414e730 2024-01-24T15:30:46.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.456+0000 7f41627fc700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f416414e730 0x7f4164157660 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 rx=0x7f4158009fd0 tx=0x7f4158017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:46.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.457+0000 7f414affd700 1 -- 172.21.15.29:0/2261916537 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158000bc0 con 0x7f416414e730 2024-01-24T15:30:46.464 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.457+0000 7f414affd700 1 -- 172.21.15.29:0/2261916537 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415800b850 con 0x7f416414e730 2024-01-24T15:30:46.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.457+0000 7f416969a700 1 -- 172.21.15.29:0/2261916537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416414f010 con 0x7f416414e730 2024-01-24T15:30:46.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.457+0000 7f414affd700 1 -- 172.21.15.29:0/2261916537 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158000d20 con 0x7f416414e730 2024-01-24T15:30:46.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.457+0000 7f416969a700 1 -- 172.21.15.29:0/2261916537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41640af660 con 0x7f416414e730 2024-01-24T15:30:46.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.458+0000 7f416969a700 1 -- 172.21.15.29:0/2261916537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416414f810 con 0x7f416414e730 2024-01-24T15:30:46.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.458+0000 7f414affd700 1 -- 172.21.15.29:0/2261916537 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 90) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f415801c900 con 0x7f416414e730 2024-01-24T15:30:46.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.458+0000 7f414affd700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 0x7f4150061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:46.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.459+0000 7f414affd700 1 -- 172.21.15.29:0/2261916537 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7186+0+0 (secure 0 0 0) 0x7f415807c400 con 0x7f416414e730 2024-01-24T15:30:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.459+0000 7f4162ffd700 1 -- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 msgr2=0x7f4150061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.459+0000 7f4162ffd700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 0x7f4150061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:30:46.466 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.463+0000 7f414affd700 1 -- 172.21.15.29:0/2261916537 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f416414f810 con 0x7f416414e730 2024-01-24T15:30:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.660+0000 7f4162ffd700 1 -- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 msgr2=0x7f4150061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:46.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.660+0000 7f4162ffd700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 0x7f4150061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:30:46.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:46.673+0000 7f416969a700 1 -- 172.21.15.29:0/2261916537 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4164083310 con 0x7f415005f2c0 2024-01-24T15:30:46.978 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:46.977+0000 7f62da106680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:30:47.050 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:47.049+0000 7f62da106680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:30:47.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:47.061+0000 7f4162ffd700 1 -- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 msgr2=0x7f4150061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:47.063 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:47.061+0000 7f4162ffd700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 0x7f4150061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-01-24T15:30:47.157 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:47.155+0000 7f0fd0a61680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:30:47.232 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:47.230+0000 7f0fd0a61680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:30:47.237 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:47.235+0000 7f62da106680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:30:47.445 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:47.443+0000 7f0fd0a61680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:30:47.659 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:47.657+0000 7f62da106680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:30:47.762 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:47.760+0000 7f62da106680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:30:47.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:47.862+0000 7f4162ffd700 1 -- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 msgr2=0x7f4150061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:47.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:47.862+0000 7f4162ffd700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 0x7f4150061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-01-24T15:30:47.916 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:47.914+0000 7f62da106680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:30:47.923 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:47.922+0000 7f0fd0a61680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:30:47.992 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:47.990+0000 7f62da106680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:30:48.041 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:48.039+0000 7f0fd0a61680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:30:48.059 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:48.057+0000 7f62da106680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:30:48.219 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:48.217+0000 7f0fd0a61680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:30:48.303 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:48.302+0000 7f0fd0a61680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:30:48.313 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:48.312+0000 7f62da106680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:30:48.381 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:48.379+0000 7f0fd0a61680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:30:48.499 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:48.498+0000 7f62da106680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:30:48.676 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:48.673+0000 7f0fd0a61680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:30:48.887 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:48.885+0000 7f0fd0a61680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:30:49.150 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:49.148+0000 7f62da106680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:30:49.219 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:49.217+0000 7f62da106680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:30:49.294 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:49.293+0000 7f62da106680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:30:49.442 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:49.441+0000 7f62da106680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:30:49.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:49.463+0000 7f4162ffd700 1 -- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 msgr2=0x7f4150061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:30:49.465 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:49.463+0000 7f4162ffd700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 0x7f4150061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-01-24T15:30:49.513 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:49.512+0000 7f62da106680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:30:49.629 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:49.627+0000 7f62da106680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:30:49.639 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:49.637+0000 7f0fd0a61680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:30:49.717 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:49.715+0000 7f0fd0a61680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:30:49.737 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:49.736+0000 7f62da106680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:30:49.803 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:49.801+0000 7f0fd0a61680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:30:49.970 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:49.968+0000 7f0fd0a61680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:30:50.028 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:50.026+0000 7f62da106680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:30:50.050 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:50.048+0000 7f0fd0a61680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:30:50.098 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:50.097+0000 7f62da106680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:30:50.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:50.115+0000 7f414affd700 1 -- 172.21.15.29:0/2261916537 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 91) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f4158079c20 con 0x7f416414e730 2024-01-24T15:30:50.117 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:50.116+0000 7f414affd700 1 -- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 msgr2=0x7f4150061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:30:50.118 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:50.116+0000 7f414affd700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 0x7f4150061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:30:50.190 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:50.188+0000 7f0fd0a61680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:30:50.192 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:50.191+0000 7f6250681700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:50.192 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:50.191+0000 7f6250681700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:50.192 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:50.191+0000 7f6250681700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:50.192 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:50.191+0000 7f6250681700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:50.192 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:50.191+0000 7f6250681700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:50.193 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:50.191+0000 7f624c679700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:50.193 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:50.191+0000 7f624c679700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:50.193 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:50.191+0000 7f624c679700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:50.193 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:50.191+0000 7f624c679700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:50.193 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:30:50.191+0000 7f624c679700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:30:50.315 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:50.313+0000 7f0fd0a61680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:30:50.652 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:50.650+0000 7f0fd0a61680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:30:50.732 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:30:50.730+0000 7f0fd0a61680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:30:51.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.119+0000 7f414affd700 1 -- 172.21.15.29:0/2261916537 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mgrmap(e 92) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f4158079fa0 con 0x7f416414e730 2024-01-24T15:30:51.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.120+0000 7f414affd700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 0x7f4150061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.120+0000 7f414affd700 1 -- 172.21.15.29:0/2261916537 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4164083310 con 0x7f415005f2c0 2024-01-24T15:30:51.123 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.122+0000 7f4162ffd700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 0x7f4150061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.126+0000 7f4162ffd700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 0x7f4150061700 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f41640aa2c0 tx=0x7f4154007bb0).ready entity=mgr.8174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:51.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.144+0000 7f414affd700 1 -- 172.21.15.29:0/2261916537 <== mgr.8174 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4164083310 con 0x7f415005f2c0 2024-01-24T15:30:51.146 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path snap_test_dir1 2024-01-24T15:30:51.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.146+0000 7f416969a700 1 -- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 msgr2=0x7f4150061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.146+0000 7f416969a700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 0x7f4150061700 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f41640aa2c0 tx=0x7f4154007bb0).stop 2024-01-24T15:30:51.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.146+0000 7f416969a700 1 -- 172.21.15.29:0/2261916537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f416414e730 msgr2=0x7f4164157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.146+0000 7f416969a700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f416414e730 0x7f4164157660 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 rx=0x7f4158009fd0 tx=0x7f4158017000).stop 2024-01-24T15:30:51.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.146+0000 7f416969a700 1 -- 172.21.15.29:0/2261916537 shutdown_connections 2024-01-24T15:30:51.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.146+0000 7f416969a700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f416414eb10 0x7f4164157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.146+0000 7f416969a700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f415005f2c0 0x7f4150061700 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.146+0000 7f416969a700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41640c7f70 0x7f4164157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.146+0000 7f416969a700 1 --2- 172.21.15.29:0/2261916537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f416414e730 0x7f4164157660 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.146+0000 7f416969a700 1 -- 172.21.15.29:0/2261916537 >> 172.21.15.29:0/2261916537 conn(0x7f41640cb880 msgr2=0x7f41640ad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:51.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.146+0000 7f416969a700 1 -- 172.21.15.29:0/2261916537 shutdown_connections 2024-01-24T15:30:51.149 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.146+0000 7f416969a700 1 -- 172.21.15.29:0/2261916537 wait complete. 2024-01-24T15:30:51.164 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-01-24T15:30:51.164 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2024-01-24T15:30:51.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.302+0000 7f1834023700 1 Processor -- start 2024-01-24T15:30:51.305 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.302+0000 7f1834023700 1 -- start start 2024-01-24T15:30:51.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.303+0000 7f1834023700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c066940 0x7f182c0be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.303+0000 7f1834023700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c0686a0 con 0x7f182c134ea0 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.303+0000 7f1834023700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c0c38e0 con 0x7f182c0664c0 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.303+0000 7f1834023700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c068b30 con 0x7f182c066940 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1831dbf700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f182c134ea0 0x7f182c0c31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52694/0 (socket says 172.21.15.29:52694) 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1831dbf700 1 -- 172.21.15.29:0/2599700603 learned_addr learned my addr 172.21.15.29:0/2599700603 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f18325c0700 1 --2- 172.21.15.29:0/2599700603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c066940 0x7f182c0be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921354741 0 0) 0x7f182c0686a0 con 0x7f182c134ea0 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18180033f0 con 0x7f182c134ea0 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2977173795 0 0) 0x7f182c0c38e0 con 0x7f182c0664c0 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f182c0686a0 con 0x7f182c0664c0 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2053126163 0 0) 0x7f18180033f0 con 0x7f182c134ea0 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c066940 msgr2=0x7f182c0be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 --2- 172.21.15.29:0/2599700603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c066940 0x7f182c0be240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.310 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 >> v1:172.21.15.163:6789/0 conn(0x7f182c0664c0 legacy=0x7f182c0bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c066dd0 con 0x7f182c134ea0 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f1820001470 con 0x7f182c134ea0 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 >> v1:172.21.15.29:6789/0 conn(0x7f182c134ea0 legacy=0x7f182c0c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 --2- 172.21.15.29:0/2599700603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1818003aa0 0x7f1818003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 --2- 172.21.15.29:0/2599700603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1818004690 0x7f1818004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 --2- 172.21.15.29:0/2599700603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c066940 0x7f1818005020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c068b30 con 0x7f1818004690 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181800b990 con 0x7f1818003aa0 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181800bb50 con 0x7f182c066940 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.304+0000 7f18315be700 1 --2- 172.21.15.29:0/2599700603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1818004690 0x7f1818004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.305+0000 7f1831dbf700 1 --2- 172.21.15.29:0/2599700603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1818003aa0 0x7f1818003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.305+0000 7f18325c0700 1 --2- 172.21.15.29:0/2599700603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c066940 0x7f1818005020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.305+0000 7f18315be700 1 -- 172.21.15.29:0/2599700603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c066940 msgr2=0x7f1818005020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.305+0000 7f18315be700 1 --2- 172.21.15.29:0/2599700603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c066940 0x7f1818005020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.305+0000 7f18315be700 1 -- 172.21.15.29:0/2599700603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1818003aa0 msgr2=0x7f1818003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.305+0000 7f18315be700 1 --2- 172.21.15.29:0/2599700603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1818003aa0 0x7f1818003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.305+0000 7f18315be700 1 -- 172.21.15.29:0/2599700603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18180033f0 con 0x7f1818004690 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.305+0000 7f18315be700 1 --2- 172.21.15.29:0/2599700603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1818004690 0x7f1818004b10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 rx=0x7f18280018d0 tx=0x7f1828007bb0).ready entity=mon.0 client_cookie=eb114ab04c3b6a21 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.305+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828007020 con 0x7f1818004690 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.305+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1828007180 con 0x7f1818004690 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.305+0000 7f1830dbd700 1 -- 172.21.15.29:0/2599700603 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828007500 con 0x7f1818004690 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.306+0000 7f1834023700 1 -- 172.21.15.29:0/2599700603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1818004690 msgr2=0x7f1818004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.306+0000 7f1834023700 1 --2- 172.21.15.29:0/2599700603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1818004690 0x7f1818004b10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 rx=0x7f18280018d0 tx=0x7f1828007bb0).stop 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.306+0000 7f18325c0700 1 -- 172.21.15.29:0/2599700603 reap_dead start 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.306+0000 7f1834023700 1 -- 172.21.15.29:0/2599700603 shutdown_connections 2024-01-24T15:30:51.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.306+0000 7f1834023700 1 -- 172.21.15.29:0/2599700603 >> 172.21.15.29:0/2599700603 conn(0x7f182c0c8bd0 msgr2=0x7f182c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:51.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.306+0000 7f1834023700 1 -- 172.21.15.29:0/2599700603 shutdown_connections 2024-01-24T15:30:51.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.306+0000 7f1834023700 1 -- 172.21.15.29:0/2599700603 wait complete. 2024-01-24T15:30:51.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.307+0000 7f1834023700 1 Processor -- start 2024-01-24T15:30:51.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.307+0000 7f1834023700 1 -- start start 2024-01-24T15:30:51.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.308+0000 7f1834023700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1818004690 0x7f182c0a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.308+0000 7f1834023700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f182c0a7ae0 0x7f182c1379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.308+0000 7f1834023700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c064260 0x7f182c0aab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.308+0000 7f1834023700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c065910 con 0x7f182c0a7ae0 2024-01-24T15:30:51.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.308+0000 7f1834023700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0a8f10 con 0x7f1818004690 2024-01-24T15:30:51.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.308+0000 7f1834023700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c063dd0 con 0x7f182c064260 2024-01-24T15:30:51.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.308+0000 7f18315be700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f182c0a7ae0 0x7f182c1379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.308+0000 7f18315be700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f182c0a7ae0 0x7f182c1379b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57098/0 (socket says 172.21.15.29:57098) 2024-01-24T15:30:51.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.308+0000 7f18315be700 1 -- 172.21.15.29:0/789311743 learned_addr learned my addr 172.21.15.29:0/789311743 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:51.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.308+0000 7f1831dbf700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1818004690 0x7f182c0a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.308+0000 7f18325c0700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c064260 0x7f182c0aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.309+0000 7f18315be700 1 -- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c064260 msgr2=0x7f182c0aab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.309+0000 7f18315be700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c064260 0x7f182c0aab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.309+0000 7f18315be700 1 -- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1818004690 msgr2=0x7f182c0a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.309+0000 7f18315be700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1818004690 0x7f182c0a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.309+0000 7f18315be700 1 -- 172.21.15.29:0/789311743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0a9720 con 0x7f182c0a7ae0 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.309+0000 7f18315be700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f182c0a7ae0 0x7f182c1379b0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7f1828006e30 tx=0x7f1828016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.309+0000 7f181dffb700 1 -- 172.21.15.29:0/789311743 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828007020 con 0x7f182c0a7ae0 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.309+0000 7f181dffb700 1 -- 172.21.15.29:0/789311743 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1828012610 con 0x7f182c0a7ae0 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.309+0000 7f1834023700 1 -- 172.21.15.29:0/789311743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c0a9160 con 0x7f182c0a7ae0 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.309+0000 7f181dffb700 1 -- 172.21.15.29:0/789311743 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828012990 con 0x7f182c0a7ae0 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.309+0000 7f1834023700 1 -- 172.21.15.29:0/789311743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c063a60 con 0x7f182c0a7ae0 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.310+0000 7f1834023700 1 -- 172.21.15.29:0/789311743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c0638c0 con 0x7f182c0a7ae0 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.311+0000 7f181dffb700 1 -- 172.21.15.29:0/789311743 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 92) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f1828012d40 con 0x7f182c0a7ae0 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.312+0000 7f181dffb700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f181005f080 0x7f1810061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.312+0000 7f181dffb700 1 -- 172.21.15.29:0/789311743 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f182807b640 con 0x7f182c0a7ae0 2024-01-24T15:30:51.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.312+0000 7f1831dbf700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f181005f080 0x7f1810061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.312+0000 7f1831dbf700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f181005f080 0x7f1810061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f182c152000 tx=0x7f1820001d20).ready entity=mgr.8174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:51.318 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.316+0000 7f181dffb700 1 -- 172.21.15.29:0/789311743 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f182804c5d0 con 0x7f182c0a7ae0 2024-01-24T15:30:51.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.521+0000 7f1834023700 1 -- 172.21.15.29:0/789311743 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f182c0638c0 con 0x7f181005f080 2024-01-24T15:30:51.528 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.527+0000 7f181dffb700 1 -- 172.21.15.29:0/789311743 <== mgr.8174 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f182c0638c0 con 0x7f181005f080 2024-01-24T15:30:51.529 INFO:teuthology.orchestra.run.smithi029.stdout:Retention added to path snap_test_dir1 2024-01-24T15:30:51.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.528+0000 7f1834023700 1 -- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f181005f080 msgr2=0x7f1810061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.528+0000 7f1834023700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f181005f080 0x7f1810061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f182c152000 tx=0x7f1820001d20).stop 2024-01-24T15:30:51.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.528+0000 7f1834023700 1 -- 172.21.15.29:0/789311743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f182c0a7ae0 msgr2=0x7f182c1379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.528+0000 7f1834023700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f182c0a7ae0 0x7f182c1379b0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7f1828006e30 tx=0x7f1828016000).stop 2024-01-24T15:30:51.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.528+0000 7f1834023700 1 -- 172.21.15.29:0/789311743 shutdown_connections 2024-01-24T15:30:51.531 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.528+0000 7f1834023700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f181005f080 0x7f1810061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.528+0000 7f1834023700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f182c064260 0x7f182c0aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.528+0000 7f1834023700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1818004690 0x7f182c0a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.528+0000 7f1834023700 1 --2- 172.21.15.29:0/789311743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f182c0a7ae0 0x7f182c1379b0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.528+0000 7f1834023700 1 -- 172.21.15.29:0/789311743 >> 172.21.15.29:0/789311743 conn(0x7f182c0c8bd0 msgr2=0x7f182c0c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:51.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.528+0000 7f1834023700 1 -- 172.21.15.29:0/789311743 shutdown_connections 2024-01-24T15:30:51.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.529+0000 7f1834023700 1 -- 172.21.15.29:0/789311743 wait complete. 2024-01-24T15:30:51.544 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2024-01-24T15:30:51.545 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2024-01-24-15_31_00 in ~9s... 2024-01-24T15:30:51.545 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2024-01-24T15:30:51.545 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2024-01-24T15:30:51.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.676+0000 7f3819fb5700 1 Processor -- start 2024-01-24T15:30:51.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.677+0000 7f3819fb5700 1 -- start start 2024-01-24T15:30:51.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.677+0000 7f3819fb5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38140c3660 0x7f381414b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.677+0000 7f3819fb5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38140c4b00 0x7f3814154a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.677+0000 7f3819fb5700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38141557e0 con 0x7f381414b0a0 2024-01-24T15:30:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.677+0000 7f3819fb5700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38140c28a0 con 0x7f38140c4b00 2024-01-24T15:30:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.677+0000 7f3819fb5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38140c2b80 con 0x7f38140c3660 2024-01-24T15:30:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f3813fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38140c4b00 0x7f3814154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f3813fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38140c4b00 0x7f3814154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57104/0 (socket says 172.21.15.29:57104) 2024-01-24T15:30:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f3813fff700 1 -- 172.21.15.29:0/41427154 learned_addr learned my addr 172.21.15.29:0/41427154 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f38137fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f381414b0a0 0x7f38140c43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:52710/0 (socket says 172.21.15.29:52710) 2024-01-24T15:30:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f3812ffd700 1 --2- 172.21.15.29:0/41427154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38140c3660 0x7f381414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f3813fff700 1 -- 172.21.15.29:0/41427154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38140c3660 msgr2=0x7f381414b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:30:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f3813fff700 1 --2- 172.21.15.29:0/41427154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38140c3660 0x7f381414b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f3813fff700 1 -- 172.21.15.29:0/41427154 >> v1:172.21.15.29:6789/0 conn(0x7f381414b0a0 legacy=0x7f38140c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f3813fff700 1 -- 172.21.15.29:0/41427154 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140c2e60 con 0x7f38140c4b00 2024-01-24T15:30:51.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f38127fc700 1 -- 172.21.15.29:0/41427154 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363904853 0 0) 0x7f38141557e0 con 0x7f381414b0a0 2024-01-24T15:30:51.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f3813fff700 1 --2- 172.21.15.29:0/41427154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38140c4b00 0x7f3814154a70 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f380c009f20 tx=0x7f380c009840).ready entity=mon.0 client_cookie=bf572226e5aca5bb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:51.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f38127fc700 1 -- 172.21.15.29:0/41427154 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c00e070 con 0x7f38140c4b00 2024-01-24T15:30:51.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f38127fc700 1 -- 172.21.15.29:0/41427154 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c00ba80 con 0x7f38140c4b00 2024-01-24T15:30:51.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.678+0000 7f38127fc700 1 -- 172.21.15.29:0/41427154 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c00be00 con 0x7f38140c4b00 2024-01-24T15:30:51.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.679+0000 7f3819fb5700 1 -- 172.21.15.29:0/41427154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38140c4b00 msgr2=0x7f3814154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.679+0000 7f3819fb5700 1 --2- 172.21.15.29:0/41427154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38140c4b00 0x7f3814154a70 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f380c009f20 tx=0x7f380c009840).stop 2024-01-24T15:30:51.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.679+0000 7f3819fb5700 1 -- 172.21.15.29:0/41427154 shutdown_connections 2024-01-24T15:30:51.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.679+0000 7f3819fb5700 1 --2- 172.21.15.29:0/41427154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38140c4b00 0x7f3814154a70 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.679+0000 7f3819fb5700 1 --2- 172.21.15.29:0/41427154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38140c3660 0x7f381414b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.679+0000 7f3819fb5700 1 -- 172.21.15.29:0/41427154 >> 172.21.15.29:0/41427154 conn(0x7f3814067b20 msgr2=0x7f38140638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:51.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.679+0000 7f3819fb5700 1 -- 172.21.15.29:0/41427154 shutdown_connections 2024-01-24T15:30:51.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.679+0000 7f3819fb5700 1 -- 172.21.15.29:0/41427154 wait complete. 2024-01-24T15:30:51.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.680+0000 7f3819fb5700 1 Processor -- start 2024-01-24T15:30:51.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.681+0000 7f3819fb5700 1 -- start start 2024-01-24T15:30:51.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.681+0000 7f3819fb5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38140c3660 0x7f38140a9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.681+0000 7f3819fb5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38140c4b00 0x7f38140c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.681+0000 7f3819fb5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f381414b0a0 0x7f3814151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.681+0000 7f3819fb5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3814149170 con 0x7f38140c4b00 2024-01-24T15:30:51.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.681+0000 7f3819fb5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3814149450 con 0x7f381414b0a0 2024-01-24T15:30:51.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.681+0000 7f3819fb5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381414fe10 con 0x7f38140c3660 2024-01-24T15:30:51.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.681+0000 7f3812ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38140c4b00 0x7f38140c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.681+0000 7f3812ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38140c4b00 0x7f38140c10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:57120/0 (socket says 172.21.15.29:57120) 2024-01-24T15:30:51.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f3812ffd700 1 -- 172.21.15.29:0/3247341736 learned_addr learned my addr 172.21.15.29:0/3247341736 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:30:51.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f3813fff700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f381414b0a0 0x7f3814151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f38137fe700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38140c3660 0x7f38140a9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f3812ffd700 1 -- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38140c3660 msgr2=0x7f38140a9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f3812ffd700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38140c3660 0x7f38140a9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f3812ffd700 1 -- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f381414b0a0 msgr2=0x7f3814151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f3812ffd700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f381414b0a0 0x7f3814151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f3812ffd700 1 -- 172.21.15.29:0/3247341736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c009ae0 con 0x7f38140c4b00 2024-01-24T15:30:51.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f3812ffd700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38140c4b00 0x7f38140c10e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 rx=0x7f380800d890 tx=0x7f3808008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:51.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f37fb7fe700 1 -- 172.21.15.29:0/3247341736 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38080125f0 con 0x7f38140c4b00 2024-01-24T15:30:51.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f37fb7fe700 1 -- 172.21.15.29:0/3247341736 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3808012750 con 0x7f38140c4b00 2024-01-24T15:30:51.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f3819fb5700 1 -- 172.21.15.29:0/3247341736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38141500f0 con 0x7f38140c4b00 2024-01-24T15:30:51.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f37fb7fe700 1 -- 172.21.15.29:0/3247341736 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808012ad0 con 0x7f38140c4b00 2024-01-24T15:30:51.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.682+0000 7f3819fb5700 1 -- 172.21.15.29:0/3247341736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38140c15d0 con 0x7f38140c4b00 2024-01-24T15:30:51.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.683+0000 7f3819fb5700 1 -- 172.21.15.29:0/3247341736 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3814150610 con 0x7f38140c4b00 2024-01-24T15:30:51.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.684+0000 7f37fb7fe700 1 -- 172.21.15.29:0/3247341736 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 92) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f3808013480 con 0x7f38140c4b00 2024-01-24T15:30:51.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.684+0000 7f37fb7fe700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f380005f080 0x7f3800061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:30:51.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.684+0000 7f37fb7fe700 1 -- 172.21.15.29:0/3247341736 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f380807a2d0 con 0x7f38140c4b00 2024-01-24T15:30:51.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.684+0000 7f38137fe700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f380005f080 0x7f3800061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:30:51.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.685+0000 7f38137fe700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f380005f080 0x7f3800061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f381406a370 tx=0x7f3804007c70).ready entity=mgr.8174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:30:51.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.689+0000 7f37fb7fe700 1 -- 172.21.15.29:0/3247341736 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f380804b1e0 con 0x7f38140c4b00 2024-01-24T15:30:51.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.896+0000 7f3819fb5700 1 -- 172.21.15.29:0/3247341736 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38141495e0 con 0x7f380005f080 2024-01-24T15:30:51.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.899+0000 7f37fb7fe700 1 -- 172.21.15.29:0/3247341736 <== mgr.8174 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f38141495e0 con 0x7f380005f080 2024-01-24T15:30:51.901 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:30:51.902 INFO:teuthology.orchestra.run.smithi029.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2024-01-24T15:30:51.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.901+0000 7f3819fb5700 1 -- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f380005f080 msgr2=0x7f3800061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.901+0000 7f3819fb5700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f380005f080 0x7f3800061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f381406a370 tx=0x7f3804007c70).stop 2024-01-24T15:30:51.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.901+0000 7f3819fb5700 1 -- 172.21.15.29:0/3247341736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38140c4b00 msgr2=0x7f38140c10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:30:51.904 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.901+0000 7f3819fb5700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38140c4b00 0x7f38140c10e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 rx=0x7f380800d890 tx=0x7f3808008040).stop 2024-01-24T15:30:51.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.901+0000 7f3819fb5700 1 -- 172.21.15.29:0/3247341736 shutdown_connections 2024-01-24T15:30:51.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.901+0000 7f3819fb5700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f381414b0a0 0x7f3814151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.901+0000 7f3819fb5700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f380005f080 0x7f3800061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.901+0000 7f3819fb5700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38140c3660 0x7f38140a9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.901+0000 7f3819fb5700 1 --2- 172.21.15.29:0/3247341736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38140c4b00 0x7f38140c10e0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:30:51.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.901+0000 7f3819fb5700 1 -- 172.21.15.29:0/3247341736 >> 172.21.15.29:0/3247341736 conn(0x7f3814067b20 msgr2=0x7f38141473c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:30:51.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.901+0000 7f3819fb5700 1 -- 172.21.15.29:0/3247341736 shutdown_connections 2024-01-24T15:30:51.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:30:51.901+0000 7f3819fb5700 1 -- 172.21.15.29:0/3247341736 wait complete. 2024-01-24T15:30:51.917 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2024-01-24T15:30:51.917 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2024-01-24T15:30:51.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:51.918 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:30:52.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:52.939 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:30:53.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:53.964 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:30:54.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:54.988 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:30:56.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:56.014 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:30:57.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:57.038 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:30:58.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:58.063 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:30:59.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:30:59.087 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:00.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:00.111 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:00.533 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:31:00.531+0000 7f623fe60700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:31:00.533 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:31:00.531+0000 7f623fe60700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:31:00.533 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:31:00.531+0000 7f623fe60700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:31:00.533 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:31:00.531+0000 7f623fe60700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:31:00.533 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:31:00.531+0000 7f623fe60700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:31:01.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:01.135 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:01.157 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:01.158 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2024-01-24-15_31_00_UTC'] 2024-01-24T15:31:01.158 DEBUG:tasks.cephfs.test_snap_schedules:exec=1706110251.5449693, now = 1706110261.1585884, timo = 9 2024-01-24T15:31:02.159 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2024-01-24-15_31_00 in ~69s... 2024-01-24T15:31:02.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:02.160 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:02.183 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:03.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:03.186 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:03.210 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:04.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:04.211 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:04.233 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:05.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:05.236 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:05.259 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:06.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:06.260 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:06.284 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:07.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:07.285 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:07.307 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:08.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:08.309 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:08.331 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:09.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:09.334 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:09.358 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:10.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:10.359 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:10.382 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:11.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:11.384 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:11.407 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:12.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:12.408 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:12.431 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:12.766 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:31:12.768 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:31:13.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:13.433 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:13.455 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:14.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:14.457 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:14.480 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:15.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:15.482 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:15.506 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:16.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:16.507 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:16.529 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:17.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:17.531 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:17.553 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:18.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:18.556 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:18.578 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:19.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:19.580 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:19.603 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:20.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:20.604 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:20.627 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:21.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:21.629 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:21.652 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:22.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:22.653 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:22.676 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:23.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:23.678 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:23.702 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:24.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:24.703 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:24.726 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:25.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:25.728 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:25.750 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:26.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:26.752 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:26.776 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:27.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:27.777 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:27.799 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:28.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:28.802 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:28.824 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:29.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:29.826 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:29.849 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:30.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:30.851 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:30.874 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:31.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:31.875 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:31.899 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:32.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:32.900 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:32.923 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:33.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:33.924 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:33.947 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:34.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:34.949 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:34.972 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:35.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:35.974 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:35.998 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:36.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:36.999 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:37.021 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:38.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:38.023 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:38.045 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:39.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:39.046 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:39.069 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:40.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:40.070 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:40.093 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:41.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:41.095 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:41.118 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:42.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:42.119 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:42.142 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:42.847 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:31:42.850 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:31:43.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:43.144 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:43.166 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:44.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:44.168 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:44.190 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:45.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:45.193 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:45.216 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:46.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:46.218 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:46.240 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:47.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:47.242 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:47.265 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:48.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:48.267 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:48.289 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:49.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:49.291 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:49.314 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:50.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:50.315 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:50.339 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:51.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:51.340 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:51.362 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:52.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:52.364 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:52.386 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:53.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:53.388 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:53.411 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:54.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:54.412 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:54.440 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:55.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:55.441 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:55.464 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:56.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:56.466 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:56.490 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:57.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:57.491 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:57.514 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:58.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:58.515 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:58.538 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:31:59.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:31:59.540 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:31:59.563 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_31_00_UTC 2024-01-24T15:32:00.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:32:00.565 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:32:00.588 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_32_00_UTC 2024-01-24T15:32:00.588 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2024-01-24-15_31_00_UTC'] 2024-01-24T15:32:00.588 DEBUG:tasks.cephfs.test_snap_schedules:exec=1706110251.5449693, now = 1706110320.588602, timo = 69 2024-01-24T15:32:01.589 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2024-01-24T15:32:01.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.731+0000 7f4f6dcbd700 1 Processor -- start 2024-01-24T15:32:01.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.731+0000 7f4f6dcbd700 1 -- start start 2024-01-24T15:32:01.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f6dcbd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4f681552a0 0x7f4f68066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:01.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f6dcbd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f68066940 0x7f4f68151350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:01.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f6dcbd700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f68068540 con 0x7f4f68154ec0 2024-01-24T15:32:01.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f6dcbd700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f680641c0 con 0x7f4f681552a0 2024-01-24T15:32:01.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f6dcbd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68063fd0 con 0x7f4f68066940 2024-01-24T15:32:01.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f66ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4f681552a0 0x7f4f68066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:01.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f677fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4f68154ec0 0x7f4f680638c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37622/0 (socket says 172.21.15.29:37622) 2024-01-24T15:32:01.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f66ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4f681552a0 0x7f4f68066450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42928/0 (socket says 172.21.15.29:42928) 2024-01-24T15:32:01.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f677fe700 1 -- 172.21.15.29:0/2488604852 learned_addr learned my addr 172.21.15.29:0/2488604852 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:01.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f67fff700 1 --2- 172.21.15.29:0/2488604852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f68066940 0x7f4f68151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:01.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f66ffd700 1 -- 172.21.15.29:0/2488604852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f68066940 msgr2=0x7f4f68151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:01.771 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f66ffd700 1 --2- 172.21.15.29:0/2488604852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f68066940 0x7f4f68151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f66ffd700 1 -- 172.21.15.29:0/2488604852 >> v1:172.21.15.29:6789/0 conn(0x7f4f68154ec0 legacy=0x7f4f680638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f66ffd700 1 -- 172.21.15.29:0/2488604852 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f680689d0 con 0x7f4f681552a0 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f667fc700 1 -- 172.21.15.29:0/2488604852 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012451343 0 0) 0x7f4f68068540 con 0x7f4f68154ec0 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f67fff700 1 --2- 172.21.15.29:0/2488604852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f68066940 0x7f4f68151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f66ffd700 1 --2- 172.21.15.29:0/2488604852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f681552a0 0x7f4f68066450 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7f4f5c009760 tx=0x7f4f5c009060).ready entity=mon.0 client_cookie=5400edf0420835c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.732+0000 7f4f667fc700 1 -- 172.21.15.29:0/2488604852 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c00b580 con 0x7f4f681552a0 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.733+0000 7f4f667fc700 1 -- 172.21.15.29:0/2488604852 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c00b6e0 con 0x7f4f681552a0 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.733+0000 7f4f667fc700 1 -- 172.21.15.29:0/2488604852 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c00ba60 con 0x7f4f681552a0 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.733+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/2488604852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f681552a0 msgr2=0x7f4f68066450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.733+0000 7f4f6dcbd700 1 --2- 172.21.15.29:0/2488604852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f681552a0 0x7f4f68066450 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7f4f5c009760 tx=0x7f4f5c009060).stop 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.733+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/2488604852 shutdown_connections 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.733+0000 7f4f6dcbd700 1 --2- 172.21.15.29:0/2488604852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f68066940 0x7f4f68151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.733+0000 7f4f6dcbd700 1 --2- 172.21.15.29:0/2488604852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f681552a0 0x7f4f68066450 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:01.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.733+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/2488604852 >> 172.21.15.29:0/2488604852 conn(0x7f4f680c8bd0 msgr2=0x7f4f680c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.733+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/2488604852 shutdown_connections 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.733+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/2488604852 wait complete. 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.735+0000 7f4f6dcbd700 1 Processor -- start 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.735+0000 7f4f6dcbd700 1 -- start start 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.735+0000 7f4f6dcbd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f68066940 0x7f4f680a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.735+0000 7f4f6dcbd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f68154ec0 0x7f4f680ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.735+0000 7f4f6dcbd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f681552a0 0x7f4f68152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.735+0000 7f4f6dcbd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f680aa200 con 0x7f4f681552a0 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.735+0000 7f4f6dcbd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f680aa4e0 con 0x7f4f68154ec0 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.735+0000 7f4f6dcbd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f680aa7c0 con 0x7f4f68066940 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f67fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f681552a0 0x7f4f68152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f67fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f681552a0 0x7f4f68152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42934/0 (socket says 172.21.15.29:42934) 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f67fff700 1 -- 172.21.15.29:0/1074541417 learned_addr learned my addr 172.21.15.29:0/1074541417 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f677fe700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f68066940 0x7f4f680a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f66ffd700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f68154ec0 0x7f4f680ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f67fff700 1 -- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f68066940 msgr2=0x7f4f680a9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f67fff700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f68066940 0x7f4f680a9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f67fff700 1 -- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f68154ec0 msgr2=0x7f4f680ab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f67fff700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f68154ec0 0x7f4f680ab2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f67fff700 1 -- 172.21.15.29:0/1074541417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c0092a0 con 0x7f4f681552a0 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f66ffd700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f68154ec0 0x7f4f680ab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f67fff700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f681552a0 0x7f4f68152810 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 rx=0x7f4f6000cef0 tx=0x7f4f60007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f4f7fe700 1 -- 172.21.15.29:0/1074541417 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000add0 con 0x7f4f681552a0 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f4f7fe700 1 -- 172.21.15.29:0/1074541417 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f60016400 con 0x7f4f681552a0 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/1074541417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f680aaaa0 con 0x7f4f681552a0 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f4f7fe700 1 -- 172.21.15.29:0/1074541417 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60016780 con 0x7f4f681552a0 2024-01-24T15:32:01.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.736+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/1074541417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f68152d00 con 0x7f4f681552a0 2024-01-24T15:32:01.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.737+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/1074541417 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f680aafc0 con 0x7f4f681552a0 2024-01-24T15:32:01.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.738+0000 7f4f4f7fe700 1 -- 172.21.15.29:0/1074541417 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 94) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f4f6001f3f0 con 0x7f4f681552a0 2024-01-24T15:32:01.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.738+0000 7f4f4f7fe700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4f5405f3e0 0x7f4f540618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:01.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.738+0000 7f4f4f7fe700 1 -- 172.21.15.29:0/1074541417 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7502+0+0 (secure 0 0 0) 0x7f4f6007b390 con 0x7f4f681552a0 2024-01-24T15:32:01.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.738+0000 7f4f677fe700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4f5405f3e0 0x7f4f540618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:01.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.739+0000 7f4f677fe700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4f5405f3e0 0x7f4f540618a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4f580011e0 tx=0x7f4f58007bb0).ready entity=mgr.8174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:01.776 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.743+0000 7f4f4f7fe700 1 -- 172.21.15.29:0/1074541417 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4f6004bc90 con 0x7f4f681552a0 2024-01-24T15:32:01.953 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.951+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/1074541417 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f680475e0 con 0x7f4f5405f3e0 2024-01-24T15:32:01.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.956+0000 7f4f4f7fe700 1 -- 172.21.15.29:0/1074541417 <== mgr.8174 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4f680475e0 con 0x7f4f5405f3e0 2024-01-24T15:32:01.957 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path snap_test_dir1 2024-01-24T15:32:01.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.957+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4f5405f3e0 msgr2=0x7f4f540618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:01.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.957+0000 7f4f6dcbd700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4f5405f3e0 0x7f4f540618a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4f580011e0 tx=0x7f4f58007bb0).stop 2024-01-24T15:32:01.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.957+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/1074541417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f681552a0 msgr2=0x7f4f68152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:01.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.957+0000 7f4f6dcbd700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f681552a0 0x7f4f68152810 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 rx=0x7f4f6000cef0 tx=0x7f4f60007860).stop 2024-01-24T15:32:01.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.957+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/1074541417 shutdown_connections 2024-01-24T15:32:01.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.957+0000 7f4f6dcbd700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f681552a0 0x7f4f68152810 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:01.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.957+0000 7f4f6dcbd700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4f5405f3e0 0x7f4f540618a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:01.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.957+0000 7f4f6dcbd700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f68066940 0x7f4f680a9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:01.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.957+0000 7f4f6dcbd700 1 --2- 172.21.15.29:0/1074541417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f68154ec0 0x7f4f680ab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:01.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.957+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/1074541417 >> 172.21.15.29:0/1074541417 conn(0x7f4f680c8bd0 msgr2=0x7f4f680c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:01.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.957+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/1074541417 shutdown_connections 2024-01-24T15:32:01.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:01.957+0000 7f4f6dcbd700 1 -- 172.21.15.29:0/1074541417 wait complete. 2024-01-24T15:32:01.973 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-01-24T15:32:01.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:32:01.973 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:32:01.992 INFO:teuthology.orchestra.run.smithi029.stdout:scheduled-2024-01-24-15_32_00_UTC 2024-01-24T15:32:01.993 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-01-24-15_32_00_UTC 2024-01-24T15:32:01.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:32:01.993 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-01-24-15_32_00_UTC) 2024-01-24T15:32:02.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:32:02.058 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-01-24T15:32:02.124 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-01-24T15:32:02.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.301+0000 7fa376a50700 1 Processor -- start 2024-01-24T15:32:02.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.301+0000 7fa376a50700 1 -- start start 2024-01-24T15:32:02.306 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.302+0000 7fa376a50700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa370157530 0x7fa370068470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.302+0000 7fa376a50700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa370066c30 0x7fa3700be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.302+0000 7fa376a50700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3700be080 con 0x7fa370068960 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.302+0000 7fa376a50700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa370065040 con 0x7fa370157530 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.302+0000 7fa376a50700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa370064140 con 0x7fa370066c30 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.302+0000 7fa36ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa370157530 0x7fa370068470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.303+0000 7fa36ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa370157530 0x7fa370068470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42940/0 (socket says 172.21.15.29:42940) 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.303+0000 7fa36ffff700 1 -- 172.21.15.29:0/1428553383 learned_addr learned my addr 172.21.15.29:0/1428553383 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.303+0000 7fa374fed700 1 --2- 172.21.15.29:0/1428553383 >> v2:172.21.15.163:3300/0 conn(0x7fa370066c30 0x7fa3700be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.303+0000 7fa36ffff700 1 -- 172.21.15.29:0/1428553383 >> v2:172.21.15.163:3300/0 conn(0x7fa370066c30 msgr2=0x7fa3700be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.303+0000 7fa36ffff700 1 --2- 172.21.15.29:0/1428553383 >> v2:172.21.15.163:3300/0 conn(0x7fa370066c30 0x7fa3700be510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.303+0000 7fa36ffff700 1 -- 172.21.15.29:0/1428553383 >> v1:172.21.15.163:6789/0 conn(0x7fa370068960 legacy=0x7fa370066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.303+0000 7fa36ffff700 1 -- 172.21.15.29:0/1428553383 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa370067380 con 0x7fa370157530 2024-01-24T15:32:02.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.303+0000 7fa36ffff700 1 --2- 172.21.15.29:0/1428553383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa370157530 0x7fa370068470 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7fa3700c9870 tx=0x7fa3600090f0).ready entity=mon.0 client_cookie=531066b5674decc4 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:02.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.303+0000 7fa36effd700 1 -- 172.21.15.29:0/1428553383 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36000b5a0 con 0x7fa370157530 2024-01-24T15:32:02.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.303+0000 7fa36effd700 1 -- 172.21.15.29:0/1428553383 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36000b700 con 0x7fa370157530 2024-01-24T15:32:02.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.303+0000 7fa36effd700 1 -- 172.21.15.29:0/1428553383 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36000ba80 con 0x7fa370157530 2024-01-24T15:32:02.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.304+0000 7fa376a50700 1 -- 172.21.15.29:0/1428553383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa370157530 msgr2=0x7fa370068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:02.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.304+0000 7fa376a50700 1 --2- 172.21.15.29:0/1428553383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa370157530 0x7fa370068470 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7fa3700c9870 tx=0x7fa3600090f0).stop 2024-01-24T15:32:02.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.304+0000 7fa376a50700 1 -- 172.21.15.29:0/1428553383 shutdown_connections 2024-01-24T15:32:02.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.304+0000 7fa376a50700 1 --2- 172.21.15.29:0/1428553383 >> v2:172.21.15.163:3300/0 conn(0x7fa370066c30 0x7fa3700be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:02.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.304+0000 7fa376a50700 1 --2- 172.21.15.29:0/1428553383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa370157530 0x7fa370068470 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:02.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.304+0000 7fa376a50700 1 -- 172.21.15.29:0/1428553383 >> 172.21.15.29:0/1428553383 conn(0x7fa3700c8ba0 msgr2=0x7fa3700c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:02.308 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.304+0000 7fa376a50700 1 -- 172.21.15.29:0/1428553383 shutdown_connections 2024-01-24T15:32:02.309 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.304+0000 7fa376a50700 1 -- 172.21.15.29:0/1428553383 wait complete. 2024-01-24T15:32:02.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.305+0000 7fa376a50700 1 Processor -- start 2024-01-24T15:32:02.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.305+0000 7fa376a50700 1 -- start start 2024-01-24T15:32:02.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.306+0000 7fa376a50700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa370066c30 0x7fa3700c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:02.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.306+0000 7fa376a50700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa370068960 0x7fa3700c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:02.311 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.306+0000 7fa376a50700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa370157530 0x7fa3700c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.306+0000 7fa376a50700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700bfc40 con 0x7fa370157530 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.306+0000 7fa376a50700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700c3f00 con 0x7fa370068960 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.306+0000 7fa376a50700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700a9fc0 con 0x7fa370066c30 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.306+0000 7fa374fed700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa370157530 0x7fa3700c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.306+0000 7fa374fed700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa370157530 0x7fa3700c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42952/0 (socket says 172.21.15.29:42952) 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.306+0000 7fa374fed700 1 -- 172.21.15.29:0/1408846328 learned_addr learned my addr 172.21.15.29:0/1408846328 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.306+0000 7fa36f7fe700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa370068960 0x7fa3700c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.306+0000 7fa36ffff700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa370066c30 0x7fa3700c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.307+0000 7fa374fed700 1 -- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa370066c30 msgr2=0x7fa3700c27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.307+0000 7fa374fed700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa370066c30 0x7fa3700c27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.307+0000 7fa374fed700 1 -- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa370068960 msgr2=0x7fa3700c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.307+0000 7fa374fed700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa370068960 0x7fa3700c3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.307+0000 7fa374fed700 1 -- 172.21.15.29:0/1408846328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa360009370 con 0x7fa370157530 2024-01-24T15:32:02.312 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.307+0000 7fa374fed700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa370157530 0x7fa3700c3a10 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x7fa368009fa0 tx=0x7fa368009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:02.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.307+0000 7fa357fff700 1 -- 172.21.15.29:0/1408846328 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36800ebc0 con 0x7fa370157530 2024-01-24T15:32:02.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.307+0000 7fa357fff700 1 -- 172.21.15.29:0/1408846328 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36800ed20 con 0x7fa370157530 2024-01-24T15:32:02.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.307+0000 7fa376a50700 1 -- 172.21.15.29:0/1408846328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3700a93b0 con 0x7fa370157530 2024-01-24T15:32:02.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.307+0000 7fa357fff700 1 -- 172.21.15.29:0/1408846328 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368014610 con 0x7fa370157530 2024-01-24T15:32:02.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.307+0000 7fa376a50700 1 -- 172.21.15.29:0/1408846328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3700a9970 con 0x7fa370157530 2024-01-24T15:32:02.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.308+0000 7fa376a50700 1 -- 172.21.15.29:0/1408846328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3700475e0 con 0x7fa370157530 2024-01-24T15:32:02.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.309+0000 7fa357fff700 1 -- 172.21.15.29:0/1408846328 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 94) v1 ==== 75679+0+0 (secure 0 0 0) 0x7fa36801d3b0 con 0x7fa370157530 2024-01-24T15:32:02.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.309+0000 7fa357fff700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa35c05f3e0 0x7fa35c0618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:02.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.309+0000 7fa357fff700 1 -- 172.21.15.29:0/1408846328 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7502+0+0 (secure 0 0 0) 0x7fa36807b030 con 0x7fa370157530 2024-01-24T15:32:02.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.309+0000 7fa36ffff700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa35c05f3e0 0x7fa35c0618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:02.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.310+0000 7fa36ffff700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa35c05f3e0 0x7fa35c0618a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa360009130 tx=0x7fa360015000).ready entity=mgr.8174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:02.315 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.314+0000 7fa357fff700 1 -- 172.21.15.29:0/1408846328 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa36804ba40 con 0x7fa370157530 2024-01-24T15:32:02.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.542+0000 7fa376a50700 1 -- 172.21.15.29:0/1408846328 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fa3700aa100 con 0x7fa370157530 2024-01-24T15:32:02.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.900+0000 7fa357fff700 1 -- 172.21.15.29:0/1408846328 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 95) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fa3680785a0 con 0x7fa370157530 2024-01-24T15:32:02.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.901+0000 7fa357fff700 1 -- 172.21.15.29:0/1408846328 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v95) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa368072eb0 con 0x7fa370157530 2024-01-24T15:32:02.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.904+0000 7fa376a50700 1 -- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa35c05f3e0 msgr2=0x7fa35c0618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:02.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.904+0000 7fa376a50700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa35c05f3e0 0x7fa35c0618a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa360009130 tx=0x7fa360015000).stop 2024-01-24T15:32:02.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.904+0000 7fa376a50700 1 -- 172.21.15.29:0/1408846328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa370157530 msgr2=0x7fa3700c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:02.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.904+0000 7fa376a50700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa370157530 0x7fa3700c3a10 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x7fa368009fa0 tx=0x7fa368009900).stop 2024-01-24T15:32:02.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.904+0000 7fa376a50700 1 -- 172.21.15.29:0/1408846328 shutdown_connections 2024-01-24T15:32:02.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.904+0000 7fa376a50700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa370157530 0x7fa3700c3a10 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:02.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.904+0000 7fa376a50700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa35c05f3e0 0x7fa35c0618a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:02.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.904+0000 7fa376a50700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa370066c30 0x7fa3700c27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:02.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.904+0000 7fa376a50700 1 --2- 172.21.15.29:0/1408846328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa370068960 0x7fa3700c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:02.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.904+0000 7fa376a50700 1 -- 172.21.15.29:0/1408846328 >> 172.21.15.29:0/1408846328 conn(0x7fa3700c8ba0 msgr2=0x7fa370151000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:02.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.904+0000 7fa376a50700 1 -- 172.21.15.29:0/1408846328 shutdown_connections 2024-01-24T15:32:02.907 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:02.904+0000 7fa376a50700 1 -- 172.21.15.29:0/1408846328 wait complete. 2024-01-24T15:32:02.921 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T15:32:02.921 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:32:02.966 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:32:03.010 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T15:32:03.010 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-24T15:32:03.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:32:03.010 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-24T15:32:03.054 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:03.053+0000 7f26c5437680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:32:03.056 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T15:32:03.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:03.057 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-24T15:32:03.060 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:03.058+0000 7fc8f7399680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:32:03.075 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:32:03.117 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:03.116+0000 7f26c5437680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:32:03.134 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:03.132+0000 7fc8f7399680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:32:03.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.246+0000 7f38bb746700 1 Processor -- start 2024-01-24T15:32:03.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.247+0000 7f38bb746700 1 -- start start 2024-01-24T15:32:03.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.248+0000 7f38bb746700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38b4154370 0x7f38b40c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:03.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.248+0000 7f38bb746700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b40aa6c0 con 0x7f38b40aa0e0 2024-01-24T15:32:03.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.248+0000 7f38bb746700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b4064240 con 0x7f38b4154750 2024-01-24T15:32:03.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.248+0000 7f38bb746700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b4154ba0 con 0x7f38b4154370 2024-01-24T15:32:03.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.248+0000 7f38b94e2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38b4154370 0x7f38b40c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:03.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.249+0000 7f38b94e2700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38b4154370 0x7f38b40c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42970/0 (socket says 172.21.15.29:42970) 2024-01-24T15:32:03.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.248+0000 7f38b8ce1700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f38b40aa0e0 0x7f38b40ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37646/0 (socket says 172.21.15.29:37646) 2024-01-24T15:32:03.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.249+0000 7f38b94e2700 1 -- 172.21.15.29:0/3799576244 learned_addr learned my addr 172.21.15.29:0/3799576244 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:03.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.249+0000 7f38b94e2700 1 -- 172.21.15.29:0/3799576244 >> v1:172.21.15.163:6789/0 conn(0x7f38b4154750 legacy=0x7f38b40631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:03.252 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.249+0000 7f38b94e2700 1 -- 172.21.15.29:0/3799576244 >> v1:172.21.15.29:6789/0 conn(0x7f38b40aa0e0 legacy=0x7f38b40ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.249+0000 7f38abfff700 1 -- 172.21.15.29:0/3799576244 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2169277199 0 0) 0x7f38b40aa6c0 con 0x7f38b40aa0e0 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.249+0000 7f38b94e2700 1 -- 172.21.15.29:0/3799576244 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b40cb5c0 con 0x7f38b4154370 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.249+0000 7f38b94e2700 1 --2- 172.21.15.29:0/3799576244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38b4154370 0x7f38b40c5410 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f38b0009810 tx=0x7f38b0009060).ready entity=mon.0 client_cookie=3ce27cefff67af20 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.249+0000 7f38abfff700 1 -- 172.21.15.29:0/3799576244 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b0005880 con 0x7f38b4154370 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.249+0000 7f38abfff700 1 -- 172.21.15.29:0/3799576244 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b00059e0 con 0x7f38b4154370 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.249+0000 7f38abfff700 1 -- 172.21.15.29:0/3799576244 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b0005d60 con 0x7f38b4154370 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.250+0000 7f38bb746700 1 -- 172.21.15.29:0/3799576244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38b4154370 msgr2=0x7f38b40c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.250+0000 7f38bb746700 1 --2- 172.21.15.29:0/3799576244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38b4154370 0x7f38b40c5410 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f38b0009810 tx=0x7f38b0009060).stop 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.250+0000 7f38bb746700 1 -- 172.21.15.29:0/3799576244 shutdown_connections 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.250+0000 7f38bb746700 1 --2- 172.21.15.29:0/3799576244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38b4154370 0x7f38b40c5410 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.250+0000 7f38bb746700 1 -- 172.21.15.29:0/3799576244 >> 172.21.15.29:0/3799576244 conn(0x7f38b405ec80 msgr2=0x7f38b40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.250+0000 7f38bb746700 1 -- 172.21.15.29:0/3799576244 shutdown_connections 2024-01-24T15:32:03.253 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.250+0000 7f38bb746700 1 -- 172.21.15.29:0/3799576244 wait complete. 2024-01-24T15:32:03.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.251+0000 7f38bb746700 1 Processor -- start 2024-01-24T15:32:03.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.251+0000 7f38bb746700 1 -- start start 2024-01-24T15:32:03.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.252+0000 7f38bb746700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38b40aa0e0 0x7f38b41681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:03.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.252+0000 7f38bb746700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38b4154370 0x7f38b40c2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:03.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.252+0000 7f38bb746700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38b4154750 0x7f38b40c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:03.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.252+0000 7f38bb746700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b4169110 con 0x7f38b4154750 2024-01-24T15:32:03.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.252+0000 7f38bb746700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b4168870 con 0x7f38b4154370 2024-01-24T15:32:03.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.252+0000 7f38bb746700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b4168b50 con 0x7f38b40aa0e0 2024-01-24T15:32:03.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.252+0000 7f38b9ce3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38b4154750 0x7f38b40c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:03.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.252+0000 7f38b8ce1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38b4154370 0x7f38b40c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.252+0000 7f38b9ce3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38b4154750 0x7f38b40c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:42984/0 (socket says 172.21.15.29:42984) 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.252+0000 7f38b9ce3700 1 -- 172.21.15.29:0/1542181783 learned_addr learned my addr 172.21.15.29:0/1542181783 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.252+0000 7f38b94e2700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38b40aa0e0 0x7f38b41681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.253+0000 7f38b9ce3700 1 -- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38b40aa0e0 msgr2=0x7f38b41681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.253+0000 7f38b9ce3700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38b40aa0e0 0x7f38b41681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.253+0000 7f38b9ce3700 1 -- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38b4154370 msgr2=0x7f38b40c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.253+0000 7f38b9ce3700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38b4154370 0x7f38b40c2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.253+0000 7f38b9ce3700 1 -- 172.21.15.29:0/1542181783 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b0009300 con 0x7f38b4154750 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.253+0000 7f38b9ce3700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38b4154750 0x7f38b40c9230 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 rx=0x7f38ac00c7c0 tx=0x7f38ac001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.253+0000 7f38a97fa700 1 -- 172.21.15.29:0/1542181783 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00e4a0 con 0x7f38b4154750 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.253+0000 7f38a97fa700 1 -- 172.21.15.29:0/1542181783 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac00e600 con 0x7f38b4154750 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.253+0000 7f38bb746700 1 -- 172.21.15.29:0/1542181783 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b4168e30 con 0x7f38b4154750 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.253+0000 7f38a97fa700 1 -- 172.21.15.29:0/1542181783 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00e980 con 0x7f38b4154750 2024-01-24T15:32:03.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.253+0000 7f38bb746700 1 -- 172.21.15.29:0/1542181783 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b40c9720 con 0x7f38b4154750 2024-01-24T15:32:03.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.254+0000 7f38bb746700 1 -- 172.21.15.29:0/1542181783 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b40ae7f0 con 0x7f38b4154750 2024-01-24T15:32:03.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.255+0000 7f38a97fa700 1 -- 172.21.15.29:0/1542181783 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 95) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f38ac0123f0 con 0x7f38b4154750 2024-01-24T15:32:03.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.255+0000 7f38a97fa700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f389805f360 0x7f3898061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:03.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.255+0000 7f38a97fa700 1 -- 172.21.15.29:0/1542181783 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7502+0+0 (secure 0 0 0) 0x7f38ac07a2e0 con 0x7f38b4154750 2024-01-24T15:32:03.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.255+0000 7f38b94e2700 1 -- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f389805f360 msgr2=0x7f3898061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:03.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.255+0000 7f38b94e2700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f389805f360 0x7f3898061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:03.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.260+0000 7f38a97fa700 1 -- 172.21.15.29:0/1542181783 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f38ac047b00 con 0x7f38b4154750 2024-01-24T15:32:03.393 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:03.392+0000 7f26c5437680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:32:03.455 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:03.453+0000 7fc8f7399680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:32:03.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.456+0000 7f38b94e2700 1 -- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f389805f360 msgr2=0x7f3898061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:03.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.456+0000 7f38b94e2700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f389805f360 0x7f3898061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:32:03.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.487+0000 7f38bb746700 1 -- 172.21.15.29:0/1542181783 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38b40475e0 con 0x7f38b4154750 2024-01-24T15:32:03.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.488+0000 7f38a97fa700 1 -- 172.21.15.29:0/1542181783 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f38b40ae7f0 con 0x7f38b4154750 2024-01-24T15:32:03.490 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:03.490 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8060,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3418669722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3418669722},{"type":"v1","addr":"172.21.15.163:6836","nonce":3418669722}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8063,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/407488941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":407488941},{"type":"v1","addr":"172.21.15.163:6837","nonce":407488941}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4018429461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4018429461},{"type":"v1","addr":"172.21.15.29:6834","nonce":4018429461}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:30:37.411326+0000","modified":"2024-01-24T15:30:40.428239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2886498894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2886498894},{"type":"v1","addr":"172.21.15.29:6835","nonce":2886498894}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-24T15:32:03.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.490+0000 7f38bb746700 1 -- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f389805f360 msgr2=0x7f3898061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:03.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.490+0000 7f38bb746700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f389805f360 0x7f3898061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:03.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.490+0000 7f38bb746700 1 -- 172.21.15.29:0/1542181783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38b4154750 msgr2=0x7f38b40c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:03.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.490+0000 7f38bb746700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38b4154750 0x7f38b40c9230 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 rx=0x7f38ac00c7c0 tx=0x7f38ac001510).stop 2024-01-24T15:32:03.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.490+0000 7f38bb746700 1 -- 172.21.15.29:0/1542181783 shutdown_connections 2024-01-24T15:32:03.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.490+0000 7f38bb746700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38b4154750 0x7f38b40c9230 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.490+0000 7f38bb746700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f389805f360 0x7f3898061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:03.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.490+0000 7f38bb746700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38b40aa0e0 0x7f38b41681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.490+0000 7f38bb746700 1 --2- 172.21.15.29:0/1542181783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38b4154370 0x7f38b40c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.490+0000 7f38bb746700 1 -- 172.21.15.29:0/1542181783 >> 172.21.15.29:0/1542181783 conn(0x7f38b405ec80 msgr2=0x7f38b40c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:03.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.490+0000 7f38bb746700 1 -- 172.21.15.29:0/1542181783 shutdown_connections 2024-01-24T15:32:03.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.490+0000 7f38bb746700 1 -- 172.21.15.29:0/1542181783 wait complete. 2024-01-24T15:32:03.494 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 90 2024-01-24T15:32:03.507 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:32:03.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.676+0000 7fa877acd700 1 Processor -- start 2024-01-24T15:32:03.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.676+0000 7fa877acd700 1 -- start start 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa877acd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa870154370 0x7fa8700c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa877acd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa8700aa0e0 0x7fa870154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa877acd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8700ab1f0 0x7fa8700ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa877acd700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700aa6c0 con 0x7fa8700aa0e0 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa877acd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa870064240 con 0x7fa8700ab1f0 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa877acd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700cb5c0 con 0x7fa870154370 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa875068700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa8700aa0e0 0x7fa870154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa875068700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa8700aa0e0 0x7fa870154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34694/0 (socket says 172.21.15.29:34694) 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa875068700 1 -- 172.21.15.29:0/1441340421 learned_addr learned my addr 172.21.15.29:0/1441340421 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa87606a700 1 --2- 172.21.15.29:0/1441340421 >> v2:172.21.15.163:3300/0 conn(0x7fa8700ab1f0 0x7fa8700ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa875869700 1 --2- 172.21.15.29:0/1441340421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa870154370 0x7fa8700c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa875068700 1 -- 172.21.15.29:0/1441340421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa870154370 msgr2=0x7fa8700c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:03.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa875068700 1 --2- 172.21.15.29:0/1441340421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa870154370 0x7fa8700c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa875068700 1 -- 172.21.15.29:0/1441340421 >> v2:172.21.15.163:3300/0 conn(0x7fa8700ab1f0 msgr2=0x7fa8700ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa875068700 1 --2- 172.21.15.29:0/1441340421 >> v2:172.21.15.163:3300/0 conn(0x7fa8700ab1f0 0x7fa8700ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.677+0000 7fa875068700 1 -- 172.21.15.29:0/1441340421 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700cb8a0 con 0x7fa8700aa0e0 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.678+0000 7fa875068700 1 --2- 172.21.15.29:0/1441340421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8700aa0e0 0x7fa870154750 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 rx=0x7fa870063f90 tx=0x7fa8600090f0).ready entity=mon.0 client_cookie=cc2b79db43efac38 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.678+0000 7fa874867700 1 -- 172.21.15.29:0/1441340421 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000f510 con 0x7fa8700aa0e0 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.678+0000 7fa874867700 1 -- 172.21.15.29:0/1441340421 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86000f670 con 0x7fa8700aa0e0 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.678+0000 7fa874867700 1 -- 172.21.15.29:0/1441340421 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000f9f0 con 0x7fa8700aa0e0 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.678+0000 7fa877acd700 1 -- 172.21.15.29:0/1441340421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8700aa0e0 msgr2=0x7fa870154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.678+0000 7fa877acd700 1 --2- 172.21.15.29:0/1441340421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8700aa0e0 0x7fa870154750 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 rx=0x7fa870063f90 tx=0x7fa8600090f0).stop 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.678+0000 7fa877acd700 1 -- 172.21.15.29:0/1441340421 shutdown_connections 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.678+0000 7fa877acd700 1 --2- 172.21.15.29:0/1441340421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa870154370 0x7fa8700c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.678+0000 7fa877acd700 1 --2- 172.21.15.29:0/1441340421 >> v2:172.21.15.163:3300/0 conn(0x7fa8700ab1f0 0x7fa8700ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.678+0000 7fa877acd700 1 --2- 172.21.15.29:0/1441340421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8700aa0e0 0x7fa870154750 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.678+0000 7fa877acd700 1 -- 172.21.15.29:0/1441340421 >> 172.21.15.29:0/1441340421 conn(0x7fa87005ec80 msgr2=0x7fa8700cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.679+0000 7fa877acd700 1 -- 172.21.15.29:0/1441340421 shutdown_connections 2024-01-24T15:32:03.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.679+0000 7fa877acd700 1 -- 172.21.15.29:0/1441340421 wait complete. 2024-01-24T15:32:03.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.680+0000 7fa877acd700 1 Processor -- start 2024-01-24T15:32:03.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.680+0000 7fa877acd700 1 -- start start 2024-01-24T15:32:03.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa877acd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8700aa0e0 0x7fa870168a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:03.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa877acd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700ab1f0 0x7fa87013f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:03.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa877acd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa870154370 0x7fa8700630e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:03.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa877acd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700c1eb0 con 0x7fa870154370 2024-01-24T15:32:03.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa877acd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700c26c0 con 0x7fa8700aa0e0 2024-01-24T15:32:03.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa877acd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700c3b20 con 0x7fa8700ab1f0 2024-01-24T15:32:03.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa87606a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa870154370 0x7fa8700630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:03.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa87606a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa870154370 0x7fa8700630e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34696/0 (socket says 172.21.15.29:34696) 2024-01-24T15:32:03.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa87606a700 1 -- 172.21.15.29:0/2738373181 learned_addr learned my addr 172.21.15.29:0/2738373181 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:03.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa875068700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700ab1f0 0x7fa87013f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa875869700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8700aa0e0 0x7fa870168a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa87606a700 1 -- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700ab1f0 msgr2=0x7fa87013f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa87606a700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700ab1f0 0x7fa87013f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa87606a700 1 -- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8700aa0e0 msgr2=0x7fa870168a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa87606a700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8700aa0e0 0x7fa870168a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa87606a700 1 -- 172.21.15.29:0/2738373181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860009370 con 0x7fa870154370 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa875068700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700ab1f0 0x7fa87013f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.681+0000 7fa87606a700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa870154370 0x7fa8700630e0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7fa868009ec0 tx=0x7fa86800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.682+0000 7fa865ffb700 1 -- 172.21.15.29:0/2738373181 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86800cb70 con 0x7fa870154370 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.682+0000 7fa865ffb700 1 -- 172.21.15.29:0/2738373181 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86800f040 con 0x7fa870154370 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.682+0000 7fa865ffb700 1 -- 172.21.15.29:0/2738373181 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86800cee0 con 0x7fa870154370 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.682+0000 7fa877acd700 1 -- 172.21.15.29:0/2738373181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87015af90 con 0x7fa870154370 2024-01-24T15:32:03.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.682+0000 7fa877acd700 1 -- 172.21.15.29:0/2738373181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8700c23e0 con 0x7fa870154370 2024-01-24T15:32:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.683+0000 7fa877acd700 1 -- 172.21.15.29:0/2738373181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8700475e0 con 0x7fa870154370 2024-01-24T15:32:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.683+0000 7fa865ffb700 1 -- 172.21.15.29:0/2738373181 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 95) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fa868013610 con 0x7fa870154370 2024-01-24T15:32:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.684+0000 7fa865ffb700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa85405f390 0x7fa854061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:03.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.684+0000 7fa865ffb700 1 -- 172.21.15.29:0/2738373181 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7502+0+0 (secure 0 0 0) 0x7fa86807b3e0 con 0x7fa870154370 2024-01-24T15:32:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.684+0000 7fa875869700 1 -- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa85405f390 msgr2=0x7fa854061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:03.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.684+0000 7fa875869700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa85405f390 0x7fa854061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:03.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.688+0000 7fa865ffb700 1 -- 172.21.15.29:0/2738373181 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa86804bd40 con 0x7fa870154370 2024-01-24T15:32:03.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.871+0000 7fa877acd700 1 -- 172.21.15.29:0/2738373181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8700cbb80 con 0x7fa870154370 2024-01-24T15:32:03.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.872+0000 7fa865ffb700 1 -- 172.21.15.29:0/2738373181 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) v1 ==== 74+0+18725 (secure 0 0 0) 0x7fa86804f360 con 0x7fa870154370 2024-01-24T15:32:03.875 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:03.875 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":216,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:31:51.918853+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:30:35.147040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-01-24T15:30:36.390723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.12","mappings":[{"from":0,"to":2}]},{"pgid":"25.32","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/35271322":"2024-01-25T15:30:50.111384+0000","172.21.15.163:0/3751389554":"2024-01-25T15:30:50.111384+0000","172.21.15.163:0/310856783":"2024-01-25T15:30:50.111384+0000","172.21.15.163:0/1051569194":"2024-01-25T15:30:50.111384+0000","172.21.15.163:6833/57365":"2024-01-25T15:30:50.111384+0000","172.21.15.163:6832/57365":"2024-01-25T15:30:50.111384+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:32:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.874+0000 7fa877acd700 1 -- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa85405f390 msgr2=0x7fa854061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.874+0000 7fa877acd700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa85405f390 0x7fa854061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.874+0000 7fa877acd700 1 -- 172.21.15.29:0/2738373181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa870154370 msgr2=0x7fa8700630e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.874+0000 7fa877acd700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa870154370 0x7fa8700630e0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7fa868009ec0 tx=0x7fa86800b040).stop 2024-01-24T15:32:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.874+0000 7fa877acd700 1 -- 172.21.15.29:0/2738373181 shutdown_connections 2024-01-24T15:32:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.874+0000 7fa877acd700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa870154370 0x7fa8700630e0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.874+0000 7fa877acd700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8700aa0e0 0x7fa870168a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.874+0000 7fa877acd700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa85405f390 0x7fa854061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.874+0000 7fa877acd700 1 --2- 172.21.15.29:0/2738373181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8700ab1f0 0x7fa87013f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.874+0000 7fa877acd700 1 -- 172.21.15.29:0/2738373181 >> 172.21.15.29:0/2738373181 conn(0x7fa87005ec80 msgr2=0x7fa8700c75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:03.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.875+0000 7fa877acd700 1 -- 172.21.15.29:0/2738373181 shutdown_connections 2024-01-24T15:32:03.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:03.875+0000 7fa877acd700 1 -- 172.21.15.29:0/2738373181 wait complete. 2024-01-24T15:32:03.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '216', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T15:30:35.147040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T15:30:36.390723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '213', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T15:32:03.891 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T15:32:04.008 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:04.007+0000 7f26c5437680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:32:04.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.025+0000 7f5b4b5d2700 1 Processor -- start 2024-01-24T15:32:04.028 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.026+0000 7f5b4b5d2700 1 -- start start 2024-01-24T15:32:04.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b4b5d2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5b440692a0 0x7f5b44065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b4b5d2700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b440c3e70 con 0x7f5b44066370 2024-01-24T15:32:04.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b4b5d2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b440c4300 con 0x7f5b44069d90 2024-01-24T15:32:04.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b4b5d2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b440c3260 con 0x7f5b440692a0 2024-01-24T15:32:04.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b48b6d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5b44066370 0x7f5b44069680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57982/0 (socket says 172.21.15.29:57982) 2024-01-24T15:32:04.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b48b6d700 1 -- 172.21.15.29:0/1895114232 learned_addr learned my addr 172.21.15.29:0/1895114232 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:04.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b4936e700 1 --2- 172.21.15.29:0/1895114232 >> v2:172.21.15.163:3300/0 conn(0x7f5b440692a0 0x7f5b44065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b3bfff700 1 -- 172.21.15.29:0/1895114232 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006333961 0 0) 0x7f5b440c3e70 con 0x7f5b44066370 2024-01-24T15:32:04.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b3bfff700 1 -- 172.21.15.29:0/1895114232 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b300033f0 con 0x7f5b44066370 2024-01-24T15:32:04.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b4936e700 1 -- 172.21.15.29:0/1895114232 >> v1:172.21.15.163:6789/0 conn(0x7f5b44069d90 legacy=0x7f5b4415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b3bfff700 1 -- 172.21.15.29:0/1895114232 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764076895 0 0) 0x7f5b440c4300 con 0x7f5b44069d90 2024-01-24T15:32:04.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b4936e700 1 -- 172.21.15.29:0/1895114232 >> v1:172.21.15.29:6789/0 conn(0x7f5b44066370 legacy=0x7f5b44069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b4936e700 1 -- 172.21.15.29:0/1895114232 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b440c3540 con 0x7f5b440692a0 2024-01-24T15:32:04.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.027+0000 7f5b4936e700 1 --2- 172.21.15.29:0/1895114232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b440692a0 0x7f5b44065e80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f5b40009760 tx=0x7f5b40009060).ready entity=mon.1 client_cookie=b9b8a9ba2e3e4b1b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:04.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.028+0000 7f5b3bfff700 1 -- 172.21.15.29:0/1895114232 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000f510 con 0x7f5b440692a0 2024-01-24T15:32:04.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.028+0000 7f5b3bfff700 1 -- 172.21.15.29:0/1895114232 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4000f670 con 0x7f5b440692a0 2024-01-24T15:32:04.031 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.028+0000 7f5b3bfff700 1 -- 172.21.15.29:0/1895114232 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000f9f0 con 0x7f5b440692a0 2024-01-24T15:32:04.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.028+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1895114232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b440692a0 msgr2=0x7f5b44065e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.028+0000 7f5b4b5d2700 1 --2- 172.21.15.29:0/1895114232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b440692a0 0x7f5b44065e80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f5b40009760 tx=0x7f5b40009060).stop 2024-01-24T15:32:04.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.028+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1895114232 shutdown_connections 2024-01-24T15:32:04.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.028+0000 7f5b4b5d2700 1 --2- 172.21.15.29:0/1895114232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b440692a0 0x7f5b44065e80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.028+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1895114232 >> 172.21.15.29:0/1895114232 conn(0x7f5b44064390 msgr2=0x7f5b440c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:04.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.028+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1895114232 shutdown_connections 2024-01-24T15:32:04.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.028+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1895114232 wait complete. 2024-01-24T15:32:04.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.030+0000 7f5b4b5d2700 1 Processor -- start 2024-01-24T15:32:04.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.030+0000 7f5b4b5d2700 1 -- start start 2024-01-24T15:32:04.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.031+0000 7f5b4b5d2700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b44066370 0x7f5b4414b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.031+0000 7f5b4b5d2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b440692a0 0x7f5b44160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.031+0000 7f5b4b5d2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b44069d90 0x7f5b44161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.031+0000 7f5b4b5d2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4414b500 con 0x7f5b440692a0 2024-01-24T15:32:04.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.031+0000 7f5b4b5d2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4414e990 con 0x7f5b44069d90 2024-01-24T15:32:04.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.031+0000 7f5b4b5d2700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4414ec70 con 0x7f5b44066370 2024-01-24T15:32:04.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b48b6d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b440692a0 0x7f5b44160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b48b6d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b440692a0 0x7f5b44160e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34704/0 (socket says 172.21.15.29:34704) 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b48b6d700 1 -- 172.21.15.29:0/1240205509 learned_addr learned my addr 172.21.15.29:0/1240205509 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b49b6f700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b44069d90 0x7f5b44161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b4936e700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b44066370 0x7f5b4414b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b48b6d700 1 -- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b44066370 msgr2=0x7f5b4414b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b48b6d700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b44066370 0x7f5b4414b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b48b6d700 1 -- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b44069d90 msgr2=0x7f5b44161300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b48b6d700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b44069d90 0x7f5b44161300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b48b6d700 1 -- 172.21.15.29:0/1240205509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b400092a0 con 0x7f5b440692a0 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b49b6f700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b44069d90 0x7f5b44161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b48b6d700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b440692a0 0x7f5b44160e10 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 rx=0x7f5b3400e6a0 tx=0x7f5b34007d00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b397fa700 1 -- 172.21.15.29:0/1240205509 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b340072e0 con 0x7f5b440692a0 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b397fa700 1 -- 172.21.15.29:0/1240205509 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b34007440 con 0x7f5b440692a0 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1240205509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4414ef50 con 0x7f5b440692a0 2024-01-24T15:32:04.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b397fa700 1 -- 172.21.15.29:0/1240205509 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b340057a0 con 0x7f5b440692a0 2024-01-24T15:32:04.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.032+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1240205509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4414f510 con 0x7f5b440692a0 2024-01-24T15:32:04.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.034+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1240205509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b440475e0 con 0x7f5b440692a0 2024-01-24T15:32:04.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.034+0000 7f5b397fa700 1 -- 172.21.15.29:0/1240205509 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 95) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f5b34005b50 con 0x7f5b440692a0 2024-01-24T15:32:04.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.035+0000 7f5b397fa700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5b2805f390 0x7f5b28061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.035+0000 7f5b4936e700 1 -- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5b2805f390 msgr2=0x7f5b28061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:04.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.035+0000 7f5b397fa700 1 -- 172.21.15.29:0/1240205509 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7502+0+0 (secure 0 0 0) 0x7f5b3407cce0 con 0x7f5b440692a0 2024-01-24T15:32:04.038 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.035+0000 7f5b4936e700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5b2805f390 0x7f5b28061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:04.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.039+0000 7f5b397fa700 1 -- 172.21.15.29:0/1240205509 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5b3404d580 con 0x7f5b440692a0 2024-01-24T15:32:04.082 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:04.081+0000 7f26c5437680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:32:04.189 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:04.186+0000 7fc8f7399680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:32:04.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.236+0000 7f5b4936e700 1 -- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5b2805f390 msgr2=0x7f5b28061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:04.238 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.236+0000 7f5b4936e700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5b2805f390 0x7f5b28061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:32:04.264 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:04.262+0000 7fc8f7399680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.265+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1240205509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5b440c2470 con 0x7f5b440692a0 2024-01-24T15:32:04.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.266+0000 7f5b397fa700 1 -- 172.21.15.29:0/1240205509 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v216) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5b3407ae60 con 0x7f5b440692a0 2024-01-24T15:32:04.268 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T15:32:04.268 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:04.267+0000 7f26c5437680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:32:04.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.268+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5b2805f390 msgr2=0x7f5b28061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:04.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.268+0000 7f5b4b5d2700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5b2805f390 0x7f5b28061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.268+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1240205509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b440692a0 msgr2=0x7f5b44160e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.268+0000 7f5b4b5d2700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b440692a0 0x7f5b44160e10 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 rx=0x7f5b3400e6a0 tx=0x7f5b34007d00).stop 2024-01-24T15:32:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.268+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1240205509 shutdown_connections 2024-01-24T15:32:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.268+0000 7f5b4b5d2700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b44069d90 0x7f5b44161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.268+0000 7f5b4b5d2700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5b2805f390 0x7f5b28061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.268+0000 7f5b4b5d2700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b44066370 0x7f5b4414b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.268+0000 7f5b4b5d2700 1 --2- 172.21.15.29:0/1240205509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b440692a0 0x7f5b44160e10 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.268+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1240205509 >> 172.21.15.29:0/1240205509 conn(0x7f5b44064390 msgr2=0x7f5b440c7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.268+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1240205509 shutdown_connections 2024-01-24T15:32:04.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.268+0000 7f5b4b5d2700 1 -- 172.21.15.29:0/1240205509 wait complete. 2024-01-24T15:32:04.284 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T15:32:04.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.419+0000 7ff9e6d80700 1 Processor -- start 2024-01-24T15:32:04.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.419+0000 7ff9e6d80700 1 -- start start 2024-01-24T15:32:04.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.420+0000 7ff9e6d80700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff9e00692a0 0x7ff9e0065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.420+0000 7ff9e6d80700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff9e0069d90 0x7ff9e015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.420+0000 7ff9e6d80700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e00c3e70 con 0x7ff9e0066370 2024-01-24T15:32:04.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.420+0000 7ff9e6d80700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e00c4300 con 0x7ff9e00692a0 2024-01-24T15:32:04.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.420+0000 7ff9e6d80700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e00c3260 con 0x7ff9e0069d90 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.420+0000 7ff9e4b1c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff9e00692a0 0x7ff9e0065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.420+0000 7ff9e4b1c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff9e00692a0 0x7ff9e0065e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34710/0 (socket says 172.21.15.29:34710) 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.420+0000 7ff9dffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff9e0066370 0x7ff9e0069680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57996/0 (socket says 172.21.15.29:57996) 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.420+0000 7ff9e4b1c700 1 -- 172.21.15.29:0/1688600940 learned_addr learned my addr 172.21.15.29:0/1688600940 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.420+0000 7ff9e531d700 1 --2- 172.21.15.29:0/1688600940 >> v2:172.21.15.163:3300/0 conn(0x7ff9e0069d90 0x7ff9e015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.421+0000 7ff9e4b1c700 1 -- 172.21.15.29:0/1688600940 >> v2:172.21.15.163:3300/0 conn(0x7ff9e0069d90 msgr2=0x7ff9e015a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.421+0000 7ff9e4b1c700 1 --2- 172.21.15.29:0/1688600940 >> v2:172.21.15.163:3300/0 conn(0x7ff9e0069d90 0x7ff9e015a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.421+0000 7ff9e4b1c700 1 -- 172.21.15.29:0/1688600940 >> v1:172.21.15.29:6789/0 conn(0x7ff9e0066370 legacy=0x7ff9e0069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.421+0000 7ff9e4b1c700 1 -- 172.21.15.29:0/1688600940 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e00c3540 con 0x7ff9e00692a0 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.421+0000 7ff9df7fe700 1 -- 172.21.15.29:0/1688600940 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308992925 0 0) 0x7ff9e00c3e70 con 0x7ff9e0066370 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.421+0000 7ff9e4b1c700 1 --2- 172.21.15.29:0/1688600940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e00692a0 0x7ff9e0065e80 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7ff9e0046ef0 tx=0x7ff9d400b3b0).ready entity=mon.0 client_cookie=1f1d2cceec5d4267 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.421+0000 7ff9df7fe700 1 -- 172.21.15.29:0/1688600940 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400da40 con 0x7ff9e00692a0 2024-01-24T15:32:04.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.421+0000 7ff9df7fe700 1 -- 172.21.15.29:0/1688600940 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d400dba0 con 0x7ff9e00692a0 2024-01-24T15:32:04.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.421+0000 7ff9df7fe700 1 -- 172.21.15.29:0/1688600940 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4010470 con 0x7ff9e00692a0 2024-01-24T15:32:04.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.422+0000 7ff9e6d80700 1 -- 172.21.15.29:0/1688600940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e00692a0 msgr2=0x7ff9e0065e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.422+0000 7ff9e6d80700 1 --2- 172.21.15.29:0/1688600940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e00692a0 0x7ff9e0065e80 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7ff9e0046ef0 tx=0x7ff9d400b3b0).stop 2024-01-24T15:32:04.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.422+0000 7ff9e6d80700 1 -- 172.21.15.29:0/1688600940 shutdown_connections 2024-01-24T15:32:04.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.422+0000 7ff9e6d80700 1 --2- 172.21.15.29:0/1688600940 >> v2:172.21.15.163:3300/0 conn(0x7ff9e0069d90 0x7ff9e015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.422+0000 7ff9e6d80700 1 --2- 172.21.15.29:0/1688600940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e00692a0 0x7ff9e0065e80 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.422+0000 7ff9e6d80700 1 -- 172.21.15.29:0/1688600940 >> 172.21.15.29:0/1688600940 conn(0x7ff9e0064390 msgr2=0x7ff9e00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:04.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.422+0000 7ff9e6d80700 1 -- 172.21.15.29:0/1688600940 shutdown_connections 2024-01-24T15:32:04.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.422+0000 7ff9e6d80700 1 -- 172.21.15.29:0/1688600940 wait complete. 2024-01-24T15:32:04.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.423+0000 7ff9e6d80700 1 Processor -- start 2024-01-24T15:32:04.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.424+0000 7ff9e6d80700 1 -- start start 2024-01-24T15:32:04.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e6d80700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9e0066370 0x7ff9e00aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e6d80700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e00692a0 0x7ff9e00beb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e6d80700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e0069d90 0x7ff9e0163330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e6d80700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0152900 con 0x7ff9e0069d90 2024-01-24T15:32:04.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e6d80700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c15a0 con 0x7ff9e0066370 2024-01-24T15:32:04.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e6d80700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c1880 con 0x7ff9e00692a0 2024-01-24T15:32:04.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e531d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e0069d90 0x7ff9e0163330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e531d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e0069d90 0x7ff9e0163330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34714/0 (socket says 172.21.15.29:34714) 2024-01-24T15:32:04.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e531d700 1 -- 172.21.15.29:0/82245870 learned_addr learned my addr 172.21.15.29:0/82245870 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:04.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e4b1c700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9e0066370 0x7ff9e00aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9dffff700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e00692a0 0x7ff9e00beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e531d700 1 -- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e00692a0 msgr2=0x7ff9e00beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e531d700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e00692a0 0x7ff9e00beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e531d700 1 -- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9e0066370 msgr2=0x7ff9e00aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e531d700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9e0066370 0x7ff9e00aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.431 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e531d700 1 -- 172.21.15.29:0/82245870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d400b610 con 0x7ff9e0069d90 2024-01-24T15:32:04.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.425+0000 7ff9e531d700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e0069d90 0x7ff9e0163330 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7ff9d8009e90 tx=0x7ff9d800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:04.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.426+0000 7ff9dcff9700 1 -- 172.21.15.29:0/82245870 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800caa0 con 0x7ff9e0069d90 2024-01-24T15:32:04.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.426+0000 7ff9dcff9700 1 -- 172.21.15.29:0/82245870 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d800f040 con 0x7ff9e0069d90 2024-01-24T15:32:04.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.426+0000 7ff9e6d80700 1 -- 172.21.15.29:0/82245870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e0152b10 con 0x7ff9e0069d90 2024-01-24T15:32:04.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.426+0000 7ff9dcff9700 1 -- 172.21.15.29:0/82245870 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800ce10 con 0x7ff9e0069d90 2024-01-24T15:32:04.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.426+0000 7ff9e6d80700 1 -- 172.21.15.29:0/82245870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e00bf080 con 0x7ff9e0069d90 2024-01-24T15:32:04.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.427+0000 7ff9e6d80700 1 -- 172.21.15.29:0/82245870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e00c1aa0 con 0x7ff9e0069d90 2024-01-24T15:32:04.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.427+0000 7ff9dcff9700 1 -- 172.21.15.29:0/82245870 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 95) v1 ==== 75662+0+0 (secure 0 0 0) 0x7ff9d8013610 con 0x7ff9e0069d90 2024-01-24T15:32:04.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.428+0000 7ff9dcff9700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff9c405f390 0x7ff9c4061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.428+0000 7ff9dcff9700 1 -- 172.21.15.29:0/82245870 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7502+0+0 (secure 0 0 0) 0x7ff9d807b410 con 0x7ff9e0069d90 2024-01-24T15:32:04.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.428+0000 7ff9e4b1c700 1 -- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff9c405f390 msgr2=0x7ff9c4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:04.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.428+0000 7ff9e4b1c700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff9c405f390 0x7ff9c4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:04.434 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.432+0000 7ff9dcff9700 1 -- 172.21.15.29:0/82245870 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff9d804bd70 con 0x7ff9e0069d90 2024-01-24T15:32:04.487 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:04.485+0000 7fc8f7399680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:32:04.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.614+0000 7ff9e6d80700 1 -- 172.21.15.29:0/82245870 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff9e00475e0 con 0x7ff9e0069d90 2024-01-24T15:32:04.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.614+0000 7ff9dcff9700 1 -- 172.21.15.29:0/82245870 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v216) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff9d804f390 con 0x7ff9e0069d90 2024-01-24T15:32:04.617 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T15:32:04.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.617+0000 7ff9e6d80700 1 -- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff9c405f390 msgr2=0x7ff9c4061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:04.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.617+0000 7ff9e6d80700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff9c405f390 0x7ff9c4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:04.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.617+0000 7ff9e6d80700 1 -- 172.21.15.29:0/82245870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e0069d90 msgr2=0x7ff9e0163330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.617+0000 7ff9e6d80700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e0069d90 0x7ff9e0163330 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7ff9d8009e90 tx=0x7ff9d800b040).stop 2024-01-24T15:32:04.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.617+0000 7ff9e6d80700 1 -- 172.21.15.29:0/82245870 shutdown_connections 2024-01-24T15:32:04.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.617+0000 7ff9e6d80700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e0069d90 0x7ff9e0163330 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.617+0000 7ff9e6d80700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9e0066370 0x7ff9e00aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.617+0000 7ff9e6d80700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff9c405f390 0x7ff9c4061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:04.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.617+0000 7ff9e6d80700 1 --2- 172.21.15.29:0/82245870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e00692a0 0x7ff9e00beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.617+0000 7ff9e6d80700 1 -- 172.21.15.29:0/82245870 >> 172.21.15.29:0/82245870 conn(0x7ff9e0064390 msgr2=0x7ff9e00c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:04.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.617+0000 7ff9e6d80700 1 -- 172.21.15.29:0/82245870 shutdown_connections 2024-01-24T15:32:04.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.617+0000 7ff9e6d80700 1 -- 172.21.15.29:0/82245870 wait complete. 2024-01-24T15:32:04.633 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T15:32:04.688 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:04.687+0000 7f26c5437680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:32:04.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.768+0000 7f37b5ab5700 1 Processor -- start 2024-01-24T15:32:04.770 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.768+0000 7f37b5ab5700 1 -- start start 2024-01-24T15:32:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37b5ab5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37b0064760 0x7f37b0064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37b5ab5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f37b00bfd40 0x7f37b00c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37b5ab5700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b0152240 con 0x7f37b00c8490 2024-01-24T15:32:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37b5ab5700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b01586c0 con 0x7f37b00bfd40 2024-01-24T15:32:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37b5ab5700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b00aa670 con 0x7f37b0064760 2024-01-24T15:32:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37affff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f37b00bfd40 0x7f37b00c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.772 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37affff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f37b00bfd40 0x7f37b00c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34728/0 (socket says 172.21.15.29:34728) 2024-01-24T15:32:04.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37aeffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f37b00c8490 0x7f37b00a9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58002/0 (socket says 172.21.15.29:58002) 2024-01-24T15:32:04.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37affff700 1 -- 172.21.15.29:0/1238865687 learned_addr learned my addr 172.21.15.29:0/1238865687 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:04.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37af7fe700 1 --2- 172.21.15.29:0/1238865687 >> v2:172.21.15.163:3300/0 conn(0x7f37b0064760 0x7f37b0064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37affff700 1 -- 172.21.15.29:0/1238865687 >> v2:172.21.15.163:3300/0 conn(0x7f37b0064760 msgr2=0x7f37b0064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:04.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37affff700 1 --2- 172.21.15.29:0/1238865687 >> v2:172.21.15.163:3300/0 conn(0x7f37b0064760 0x7f37b0064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37affff700 1 -- 172.21.15.29:0/1238865687 >> v1:172.21.15.29:6789/0 conn(0x7f37b00c8490 legacy=0x7f37b00a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37ae7fc700 1 -- 172.21.15.29:0/1238865687 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494664236 0 0) 0x7f37b0152240 con 0x7f37b00c8490 2024-01-24T15:32:04.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.769+0000 7f37affff700 1 -- 172.21.15.29:0/1238865687 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b0157d90 con 0x7f37b00bfd40 2024-01-24T15:32:04.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.770+0000 7f37affff700 1 --2- 172.21.15.29:0/1238865687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37b00bfd40 0x7f37b00c01d0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7f37b0084450 tx=0x7f37a8009840).ready entity=mon.0 client_cookie=627508f5d96be4b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:04.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.770+0000 7f37ae7fc700 1 -- 172.21.15.29:0/1238865687 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a800e070 con 0x7f37b00bfd40 2024-01-24T15:32:04.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.770+0000 7f37ae7fc700 1 -- 172.21.15.29:0/1238865687 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a800bab0 con 0x7f37b00bfd40 2024-01-24T15:32:04.773 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.770+0000 7f37ae7fc700 1 -- 172.21.15.29:0/1238865687 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a800be30 con 0x7f37b00bfd40 2024-01-24T15:32:04.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.771+0000 7f37b5ab5700 1 -- 172.21.15.29:0/1238865687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37b00bfd40 msgr2=0x7f37b00c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.771+0000 7f37b5ab5700 1 --2- 172.21.15.29:0/1238865687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37b00bfd40 0x7f37b00c01d0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7f37b0084450 tx=0x7f37a8009840).stop 2024-01-24T15:32:04.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.771+0000 7f37b5ab5700 1 -- 172.21.15.29:0/1238865687 shutdown_connections 2024-01-24T15:32:04.774 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.771+0000 7f37b5ab5700 1 --2- 172.21.15.29:0/1238865687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37b00bfd40 0x7f37b00c01d0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.771+0000 7f37b5ab5700 1 --2- 172.21.15.29:0/1238865687 >> v2:172.21.15.163:3300/0 conn(0x7f37b0064760 0x7f37b0064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.771+0000 7f37b5ab5700 1 -- 172.21.15.29:0/1238865687 >> 172.21.15.29:0/1238865687 conn(0x7f37b00cb8b0 msgr2=0x7f37b00c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:04.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.771+0000 7f37b5ab5700 1 -- 172.21.15.29:0/1238865687 shutdown_connections 2024-01-24T15:32:04.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.771+0000 7f37b5ab5700 1 -- 172.21.15.29:0/1238865687 wait complete. 2024-01-24T15:32:04.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.773+0000 7f37b5ab5700 1 Processor -- start 2024-01-24T15:32:04.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.773+0000 7f37b5ab5700 1 -- start start 2024-01-24T15:32:04.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37b5ab5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37b0064760 0x7f37b00add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37b5ab5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37b00bfd40 0x7f37b014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37b5ab5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b00c8490 0x7f37b00af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37b5ab5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00ae670 con 0x7f37b00bfd40 2024-01-24T15:32:04.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37b5ab5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00ae950 con 0x7f37b00c8490 2024-01-24T15:32:04.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37b5ab5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00aec30 con 0x7f37b0064760 2024-01-24T15:32:04.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37aeffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37b00bfd40 0x7f37b014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37aeffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37b00bfd40 0x7f37b014f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34734/0 (socket says 172.21.15.29:34734) 2024-01-24T15:32:04.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37aeffd700 1 -- 172.21.15.29:0/118008283 learned_addr learned my addr 172.21.15.29:0/118008283 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:04.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37affff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b00c8490 0x7f37b00af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37af7fe700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37b0064760 0x7f37b00add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:04.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37aeffd700 1 -- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37b0064760 msgr2=0x7f37b00add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37aeffd700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37b0064760 0x7f37b00add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37aeffd700 1 -- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b00c8490 msgr2=0x7f37b00af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37aeffd700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b00c8490 0x7f37b00af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37aeffd700 1 -- 172.21.15.29:0/118008283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a8009ae0 con 0x7f37b00bfd40 2024-01-24T15:32:04.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.774+0000 7f37affff700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b00c8490 0x7f37b00af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:04.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.775+0000 7f37aeffd700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37b00bfd40 0x7f37b014f3e0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7f37a400c440 tx=0x7f37a4007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:04.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.775+0000 7f37977fe700 1 -- 172.21.15.29:0/118008283 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a40070d0 con 0x7f37b00bfd40 2024-01-24T15:32:04.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.775+0000 7f37977fe700 1 -- 172.21.15.29:0/118008283 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a4007230 con 0x7f37b00bfd40 2024-01-24T15:32:04.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.775+0000 7f37b5ab5700 1 -- 172.21.15.29:0/118008283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b00aef10 con 0x7f37b00bfd40 2024-01-24T15:32:04.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.775+0000 7f37977fe700 1 -- 172.21.15.29:0/118008283 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a40056e0 con 0x7f37b00bfd40 2024-01-24T15:32:04.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.775+0000 7f37b5ab5700 1 -- 172.21.15.29:0/118008283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b00ae280 con 0x7f37b00bfd40 2024-01-24T15:32:04.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.776+0000 7f37b5ab5700 1 -- 172.21.15.29:0/118008283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b00af430 con 0x7f37b00bfd40 2024-01-24T15:32:04.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.776+0000 7f37977fe700 1 -- 172.21.15.29:0/118008283 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 95) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f37a4005ad0 con 0x7f37b00bfd40 2024-01-24T15:32:04.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.777+0000 7f37977fe700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f379c05f360 0x7f379c061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:04.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.777+0000 7f37977fe700 1 -- 172.21.15.29:0/118008283 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7502+0+0 (secure 0 0 0) 0x7f37a407a100 con 0x7f37b00bfd40 2024-01-24T15:32:04.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.777+0000 7f37af7fe700 1 -- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f379c05f360 msgr2=0x7f379c061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:04.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.777+0000 7f37af7fe700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f379c05f360 0x7f379c061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:04.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.781+0000 7f37977fe700 1 -- 172.21.15.29:0/118008283 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f37b00af430 con 0x7f37b00bfd40 2024-01-24T15:32:04.794 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:04.792+0000 7f26c5437680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:32:04.948 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:04.946+0000 7f26c5437680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:32:04.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.965+0000 7f37b5ab5700 1 -- 172.21.15.29:0/118008283 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37b00af430 con 0x7f37b00bfd40 2024-01-24T15:32:04.968 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.965+0000 7f37977fe700 1 -- 172.21.15.29:0/118008283 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v216) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37b00af430 con 0x7f37b00bfd40 2024-01-24T15:32:04.968 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T15:32:04.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.968+0000 7f37b5ab5700 1 -- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f379c05f360 msgr2=0x7f379c061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:04.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.968+0000 7f37b5ab5700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f379c05f360 0x7f379c061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:04.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.968+0000 7f37b5ab5700 1 -- 172.21.15.29:0/118008283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37b00bfd40 msgr2=0x7f37b014f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:04.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.968+0000 7f37b5ab5700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37b00bfd40 0x7f37b014f3e0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7f37a400c440 tx=0x7f37a4007e30).stop 2024-01-24T15:32:04.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.968+0000 7f37b5ab5700 1 -- 172.21.15.29:0/118008283 shutdown_connections 2024-01-24T15:32:04.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.968+0000 7f37b5ab5700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b00c8490 0x7f37b00af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.968+0000 7f37b5ab5700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f379c05f360 0x7f379c061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:04.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.968+0000 7f37b5ab5700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37b0064760 0x7f37b00add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.968+0000 7f37b5ab5700 1 --2- 172.21.15.29:0/118008283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37b00bfd40 0x7f37b014f3e0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:04.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.968+0000 7f37b5ab5700 1 -- 172.21.15.29:0/118008283 >> 172.21.15.29:0/118008283 conn(0x7f37b00cb8b0 msgr2=0x7f37b00c5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:04.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.968+0000 7f37b5ab5700 1 -- 172.21.15.29:0/118008283 shutdown_connections 2024-01-24T15:32:04.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:04.968+0000 7f37b5ab5700 1 -- 172.21.15.29:0/118008283 wait complete. 2024-01-24T15:32:04.978 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:04.976+0000 7fc8f7399680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:32:04.984 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T15:32:05.025 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:05.023+0000 7f26c5437680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:32:05.092 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:05.090+0000 7f26c5437680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:32:05.103 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:05.101+0000 7fc8f7399680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:32:05.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.119+0000 7f51e864b700 1 Processor -- start 2024-01-24T15:32:05.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.119+0000 7f51e864b700 1 -- start start 2024-01-24T15:32:05.122 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.120+0000 7f51e864b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f51e0064760 0x7f51e0064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.120+0000 7f51e864b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51e00bfd40 0x7f51e00c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.120+0000 7f51e864b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e0152240 con 0x7f51e00c8490 2024-01-24T15:32:05.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.120+0000 7f51e864b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e01586c0 con 0x7f51e00bfd40 2024-01-24T15:32:05.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.120+0000 7f51e864b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e00aa670 con 0x7f51e0064760 2024-01-24T15:32:05.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.120+0000 7f51e6be8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51e00bfd40 0x7f51e00c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.120+0000 7f51e63e7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f51e0064760 0x7f51e0064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.120+0000 7f51e6be8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51e00bfd40 0x7f51e00c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34738/0 (socket says 172.21.15.29:34738) 2024-01-24T15:32:05.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.120+0000 7f51e6be8700 1 -- 172.21.15.29:0/2255673691 learned_addr learned my addr 172.21.15.29:0/2255673691 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:05.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.120+0000 7f51e5be6700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f51e00c8490 0x7f51e00a9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:36312/0 (socket says 172.21.15.29:36312) 2024-01-24T15:32:05.124 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.121+0000 7f51e63e7700 1 -- 172.21.15.29:0/2255673691 >> v2:172.21.15.29:3300/0 conn(0x7f51e00bfd40 msgr2=0x7f51e00c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.121+0000 7f51e63e7700 1 --2- 172.21.15.29:0/2255673691 >> v2:172.21.15.29:3300/0 conn(0x7f51e00bfd40 0x7f51e00c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.121+0000 7f51e63e7700 1 -- 172.21.15.29:0/2255673691 >> v1:172.21.15.163:6789/0 conn(0x7f51e00c8490 legacy=0x7f51e00a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.121+0000 7f51e63e7700 1 -- 172.21.15.29:0/2255673691 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e0157d90 con 0x7f51e0064760 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.121+0000 7f51e63e7700 1 --2- 172.21.15.29:0/2255673691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e0064760 0x7f51e0064b40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f51e0084450 tx=0x7f51dc009060).ready entity=mon.1 client_cookie=d1834023b9c61c28 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.121+0000 7f51e53e5700 1 -- 172.21.15.29:0/2255673691 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc00f510 con 0x7f51e0064760 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.121+0000 7f51e53e5700 1 -- 172.21.15.29:0/2255673691 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc00f670 con 0x7f51e0064760 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.121+0000 7f51e53e5700 1 -- 172.21.15.29:0/2255673691 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc00f9f0 con 0x7f51e0064760 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.122+0000 7f51e864b700 1 -- 172.21.15.29:0/2255673691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e0064760 msgr2=0x7f51e0064b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.122+0000 7f51e864b700 1 --2- 172.21.15.29:0/2255673691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e0064760 0x7f51e0064b40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f51e0084450 tx=0x7f51dc009060).stop 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.122+0000 7f51e864b700 1 -- 172.21.15.29:0/2255673691 shutdown_connections 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.122+0000 7f51e864b700 1 --2- 172.21.15.29:0/2255673691 >> v2:172.21.15.29:3300/0 conn(0x7f51e00bfd40 0x7f51e00c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.122+0000 7f51e864b700 1 --2- 172.21.15.29:0/2255673691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e0064760 0x7f51e0064b40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.122+0000 7f51e864b700 1 -- 172.21.15.29:0/2255673691 >> 172.21.15.29:0/2255673691 conn(0x7f51e00cb8b0 msgr2=0x7f51e00c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:05.125 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.122+0000 7f51e864b700 1 -- 172.21.15.29:0/2255673691 shutdown_connections 2024-01-24T15:32:05.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.122+0000 7f51e864b700 1 -- 172.21.15.29:0/2255673691 wait complete. 2024-01-24T15:32:05.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.123+0000 7f51e864b700 1 Processor -- start 2024-01-24T15:32:05.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.123+0000 7f51e864b700 1 -- start start 2024-01-24T15:32:05.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.124+0000 7f51e864b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e0064760 0x7f51e00c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.124+0000 7f51e864b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e00bfd40 0x7f51e0154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e864b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51e00c8490 0x7f51e00c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e864b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e0167f20 con 0x7f51e00bfd40 2024-01-24T15:32:05.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e864b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00c33b0 con 0x7f51e0064760 2024-01-24T15:32:05.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e864b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e0168170 con 0x7f51e00c8490 2024-01-24T15:32:05.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e5be6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e00bfd40 0x7f51e0154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.128 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e5be6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e00bfd40 0x7f51e0154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34740/0 (socket says 172.21.15.29:34740) 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e5be6700 1 -- 172.21.15.29:0/274900183 learned_addr learned my addr 172.21.15.29:0/274900183 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e63e7700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e0064760 0x7f51e00c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e6be8700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51e00c8490 0x7f51e00c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e5be6700 1 -- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51e00c8490 msgr2=0x7f51e00c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e5be6700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51e00c8490 0x7f51e00c3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e5be6700 1 -- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e0064760 msgr2=0x7f51e00c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e5be6700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e0064760 0x7f51e00c2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e5be6700 1 -- 172.21.15.29:0/274900183 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0092a0 con 0x7f51e00bfd40 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e5be6700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e00bfd40 0x7f51e0154bb0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7f51d0001eb0 tx=0x7f51d0007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51d67fc700 1 -- 172.21.15.29:0/274900183 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00070d0 con 0x7f51e00bfd40 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51d67fc700 1 -- 172.21.15.29:0/274900183 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d0007230 con 0x7f51e00bfd40 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51d67fc700 1 -- 172.21.15.29:0/274900183 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00075b0 con 0x7f51e00bfd40 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.125+0000 7f51e864b700 1 -- 172.21.15.29:0/274900183 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e0168450 con 0x7f51e00bfd40 2024-01-24T15:32:05.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.126+0000 7f51e864b700 1 -- 172.21.15.29:0/274900183 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e0168a10 con 0x7f51e00bfd40 2024-01-24T15:32:05.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.127+0000 7f51d67fc700 1 -- 172.21.15.29:0/274900183 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 95) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f51d00058f0 con 0x7f51e00bfd40 2024-01-24T15:32:05.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.127+0000 7f51e864b700 1 -- 172.21.15.29:0/274900183 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e0164970 con 0x7f51e00bfd40 2024-01-24T15:32:05.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.127+0000 7f51d67fc700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f51c405f360 0x7f51c4061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.127+0000 7f51d67fc700 1 -- 172.21.15.29:0/274900183 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7502+0+0 (secure 0 0 0) 0x7f51d007a0e0 con 0x7f51e00bfd40 2024-01-24T15:32:05.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.127+0000 7f51e63e7700 1 -- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f51c405f360 msgr2=0x7f51c4061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:05.131 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.128+0000 7f51e63e7700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f51c405f360 0x7f51c4061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:05.134 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.132+0000 7f51d67fc700 1 -- 172.21.15.29:0/274900183 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f51d004ab50 con 0x7f51e00bfd40 2024-01-24T15:32:05.304 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:05.302+0000 7fc8f7399680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:32:05.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.329+0000 7f51e63e7700 1 -- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f51c405f360 msgr2=0x7f51c4061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:05.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.329+0000 7f51e63e7700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f51c405f360 0x7f51c4061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:32:05.349 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:05.348+0000 7f26c5437680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:32:05.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.351+0000 7f51e864b700 1 -- 172.21.15.29:0/274900183 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51e00475e0 con 0x7f51e00bfd40 2024-01-24T15:32:05.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.352+0000 7f51d67fc700 1 -- 172.21.15.29:0/274900183 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v216) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51d004e170 con 0x7f51e00bfd40 2024-01-24T15:32:05.354 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T15:32:05.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.354+0000 7f51e864b700 1 -- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f51c405f360 msgr2=0x7f51c4061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:05.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.354+0000 7f51e864b700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f51c405f360 0x7f51c4061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:05.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.354+0000 7f51e864b700 1 -- 172.21.15.29:0/274900183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e00bfd40 msgr2=0x7f51e0154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.354+0000 7f51e864b700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e00bfd40 0x7f51e0154bb0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7f51d0001eb0 tx=0x7f51d0007df0).stop 2024-01-24T15:32:05.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.354+0000 7f51e864b700 1 -- 172.21.15.29:0/274900183 shutdown_connections 2024-01-24T15:32:05.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.354+0000 7f51e864b700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f51c405f360 0x7f51c4061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:05.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.354+0000 7f51e864b700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51e00c8490 0x7f51e00c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.354+0000 7f51e864b700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e0064760 0x7f51e00c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.354+0000 7f51e864b700 1 --2- 172.21.15.29:0/274900183 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51e00bfd40 0x7f51e0154bb0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.354+0000 7f51e864b700 1 -- 172.21.15.29:0/274900183 >> 172.21.15.29:0/274900183 conn(0x7f51e00cb8b0 msgr2=0x7f51e0156700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:05.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.354+0000 7f51e864b700 1 -- 172.21.15.29:0/274900183 shutdown_connections 2024-01-24T15:32:05.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.354+0000 7f51e864b700 1 -- 172.21.15.29:0/274900183 wait complete. 2024-01-24T15:32:05.371 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:32:05.393 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:05.391+0000 7fc8f7399680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:32:05.475 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:05.473+0000 7fc8f7399680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:32:05.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.503+0000 7fbae9445700 1 Processor -- start 2024-01-24T15:32:05.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.503+0000 7fbae9445700 1 -- start start 2024-01-24T15:32:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.504+0000 7fbae9445700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbae4154370 0x7fbae40c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.504+0000 7fbae9445700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbae4154750 0x7fbae40631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.504+0000 7fbae9445700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae40aa6c0 con 0x7fbae40aa0e0 2024-01-24T15:32:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.504+0000 7fbae9445700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae4154ba0 con 0x7fbae4154370 2024-01-24T15:32:05.514 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.504+0000 7fbae9445700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae4064240 con 0x7fbae4154750 2024-01-24T15:32:05.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae2ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbae4154370 0x7fbae40c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae2ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbae4154370 0x7fbae40c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34746/0 (socket says 172.21.15.29:34746) 2024-01-24T15:32:05.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae2ffd700 1 -- 172.21.15.29:0/1335700455 learned_addr learned my addr 172.21.15.29:0/1335700455 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:05.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae37fe700 1 --2- 172.21.15.29:0/1335700455 >> v2:172.21.15.163:3300/0 conn(0x7fbae4154750 0x7fbae40631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae1ffb700 1 -- 172.21.15.29:0/1335700455 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1074402075 0 0) 0x7fbae40aa6c0 con 0x7fbae40aa0e0 2024-01-24T15:32:05.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae2ffd700 1 -- 172.21.15.29:0/1335700455 >> v2:172.21.15.163:3300/0 conn(0x7fbae4154750 msgr2=0x7fbae40631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:05.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae2ffd700 1 --2- 172.21.15.29:0/1335700455 >> v2:172.21.15.163:3300/0 conn(0x7fbae4154750 0x7fbae40631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae2ffd700 1 -- 172.21.15.29:0/1335700455 >> v1:172.21.15.163:6789/0 conn(0x7fbae40aa0e0 legacy=0x7fbae40ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae2ffd700 1 -- 172.21.15.29:0/1335700455 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae40cb5c0 con 0x7fbae4154370 2024-01-24T15:32:05.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae2ffd700 1 --2- 172.21.15.29:0/1335700455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae4154370 0x7fbae40c5410 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 rx=0x7fbae405f190 tx=0x7fbad4009060).ready entity=mon.0 client_cookie=82e88f31d8820aa1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:05.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae1ffb700 1 -- 172.21.15.29:0/1335700455 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad4005880 con 0x7fbae4154370 2024-01-24T15:32:05.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae1ffb700 1 -- 172.21.15.29:0/1335700455 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad40059e0 con 0x7fbae4154370 2024-01-24T15:32:05.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.505+0000 7fbae1ffb700 1 -- 172.21.15.29:0/1335700455 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad4005d60 con 0x7fbae4154370 2024-01-24T15:32:05.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.506+0000 7fbae9445700 1 -- 172.21.15.29:0/1335700455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae4154370 msgr2=0x7fbae40c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.506+0000 7fbae9445700 1 --2- 172.21.15.29:0/1335700455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae4154370 0x7fbae40c5410 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 rx=0x7fbae405f190 tx=0x7fbad4009060).stop 2024-01-24T15:32:05.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.506+0000 7fbae9445700 1 -- 172.21.15.29:0/1335700455 shutdown_connections 2024-01-24T15:32:05.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.506+0000 7fbae9445700 1 --2- 172.21.15.29:0/1335700455 >> v2:172.21.15.163:3300/0 conn(0x7fbae4154750 0x7fbae40631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.506+0000 7fbae9445700 1 --2- 172.21.15.29:0/1335700455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae4154370 0x7fbae40c5410 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.506+0000 7fbae9445700 1 -- 172.21.15.29:0/1335700455 >> 172.21.15.29:0/1335700455 conn(0x7fbae405ec80 msgr2=0x7fbae40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:05.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.506+0000 7fbae9445700 1 -- 172.21.15.29:0/1335700455 shutdown_connections 2024-01-24T15:32:05.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.506+0000 7fbae9445700 1 -- 172.21.15.29:0/1335700455 wait complete. 2024-01-24T15:32:05.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.508+0000 7fbae9445700 1 Processor -- start 2024-01-24T15:32:05.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.508+0000 7fbae9445700 1 -- start start 2024-01-24T15:32:05.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.508+0000 7fbae9445700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae40aa0e0 0x7fbae414dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.508+0000 7fbae9445700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbae4154370 0x7fbae40afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae9445700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbae4154750 0x7fbae40b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae9445700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40b0780 con 0x7fbae40aa0e0 2024-01-24T15:32:05.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae9445700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40c3fe0 con 0x7fbae4154750 2024-01-24T15:32:05.518 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae9445700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40c2f60 con 0x7fbae4154370 2024-01-24T15:32:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae2ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae40aa0e0 0x7fbae414dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae2ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae40aa0e0 0x7fbae414dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34756/0 (socket says 172.21.15.29:34756) 2024-01-24T15:32:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae2ffd700 1 -- 172.21.15.29:0/167187984 learned_addr learned my addr 172.21.15.29:0/167187984 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae27fc700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbae4154370 0x7fbae40afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae37fe700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbae4154750 0x7fbae40b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae2ffd700 1 -- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbae4154370 msgr2=0x7fbae40afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae2ffd700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbae4154370 0x7fbae40afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae2ffd700 1 -- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbae4154750 msgr2=0x7fbae40b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae2ffd700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbae4154750 0x7fbae40b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae2ffd700 1 -- 172.21.15.29:0/167187984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad4009300 con 0x7fbae40aa0e0 2024-01-24T15:32:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae37fe700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbae4154750 0x7fbae40b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae2ffd700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae40aa0e0 0x7fbae414dbf0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7fbae40c8920 tx=0x7fbad4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbacaffd700 1 -- 172.21.15.29:0/167187984 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad400b580 con 0x7fbae40aa0e0 2024-01-24T15:32:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbacaffd700 1 -- 172.21.15.29:0/167187984 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad400b6e0 con 0x7fbae40aa0e0 2024-01-24T15:32:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.509+0000 7fbae9445700 1 -- 172.21.15.29:0/167187984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae40c3240 con 0x7fbae40aa0e0 2024-01-24T15:32:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.510+0000 7fbacaffd700 1 -- 172.21.15.29:0/167187984 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad400baa0 con 0x7fbae40aa0e0 2024-01-24T15:32:05.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.510+0000 7fbae9445700 1 -- 172.21.15.29:0/167187984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae40c3880 con 0x7fbae40aa0e0 2024-01-24T15:32:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.511+0000 7fbae9445700 1 -- 172.21.15.29:0/167187984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae40c3e50 con 0x7fbae40aa0e0 2024-01-24T15:32:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.511+0000 7fbacaffd700 1 -- 172.21.15.29:0/167187984 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 95) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fbad4018070 con 0x7fbae40aa0e0 2024-01-24T15:32:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.511+0000 7fbacaffd700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbad005f390 0x7fbad0061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.511+0000 7fbacaffd700 1 -- 172.21.15.29:0/167187984 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7502+0+0 (secure 0 0 0) 0x7fbad407bae0 con 0x7fbae40aa0e0 2024-01-24T15:32:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.511+0000 7fbae27fc700 1 -- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbad005f390 msgr2=0x7fbad0061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:05.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.512+0000 7fbae27fc700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbad005f390 0x7fbad0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:05.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.516+0000 7fbacaffd700 1 -- 172.21.15.29:0/167187984 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbad404c550 con 0x7fbae40aa0e0 2024-01-24T15:32:05.537 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:05.535+0000 7f26c5437680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:32:05.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.713+0000 7fbae27fc700 1 -- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbad005f390 msgr2=0x7fbad0061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:05.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.713+0000 7fbae27fc700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbad005f390 0x7fbad0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:32:05.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.742+0000 7fbae9445700 1 -- 172.21.15.29:0/167187984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbae40c33d0 con 0x7fbae40aa0e0 2024-01-24T15:32:05.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.743+0000 7fbacaffd700 1 -- 172.21.15.29:0/167187984 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4496 (secure 0 0 0) 0x7fbad404fb70 con 0x7fbae40aa0e0 2024-01-24T15:32:05.746 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:05.746 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8060,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3418669722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3418669722},{"type":"v1","addr":"172.21.15.163:6836","nonce":3418669722}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8063,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/407488941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":407488941},{"type":"v1","addr":"172.21.15.163:6837","nonce":407488941}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4018429461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4018429461},{"type":"v1","addr":"172.21.15.29:6834","nonce":4018429461}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:30:37.411326+0000","modified":"2024-01-24T15:30:40.428239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2886498894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2886498894},{"type":"v1","addr":"172.21.15.29:6835","nonce":2886498894}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-24T15:32:05.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.745+0000 7fbae9445700 1 -- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbad005f390 msgr2=0x7fbad0061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:05.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.745+0000 7fbae9445700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbad005f390 0x7fbad0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:05.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.745+0000 7fbae9445700 1 -- 172.21.15.29:0/167187984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae40aa0e0 msgr2=0x7fbae414dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.745+0000 7fbae9445700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae40aa0e0 0x7fbae414dbf0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7fbae40c8920 tx=0x7fbad4016000).stop 2024-01-24T15:32:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.745+0000 7fbae9445700 1 -- 172.21.15.29:0/167187984 shutdown_connections 2024-01-24T15:32:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.745+0000 7fbae9445700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbae4154750 0x7fbae40b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.745+0000 7fbae9445700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbae40aa0e0 0x7fbae414dbf0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.745+0000 7fbae9445700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbad005f390 0x7fbad0061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.745+0000 7fbae9445700 1 --2- 172.21.15.29:0/167187984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbae4154370 0x7fbae40afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.745+0000 7fbae9445700 1 -- 172.21.15.29:0/167187984 >> 172.21.15.29:0/167187984 conn(0x7fbae405ec80 msgr2=0x7fbae4151bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.745+0000 7fbae9445700 1 -- 172.21.15.29:0/167187984 shutdown_connections 2024-01-24T15:32:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.745+0000 7fbae9445700 1 -- 172.21.15.29:0/167187984 wait complete. 2024-01-24T15:32:05.748 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 90 2024-01-24T15:32:05.761 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:32:05.776 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:05.774+0000 7fc8f7399680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:32:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.907+0000 7f46b093c700 1 Processor -- start 2024-01-24T15:32:05.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.907+0000 7f46b093c700 1 -- start start 2024-01-24T15:32:05.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46b093c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46ac154370 0x7f46ac0c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46b093c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f46ac0aa0e0 0x7f46ac154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46b093c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46ac0ab1f0 0x7f46ac0ab680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46b093c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46ac0aa6c0 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46b093c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46ac064240 con 0x7f46ac154370 2024-01-24T15:32:05.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46b093c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0cb5c0 con 0x7f46ac0ab1f0 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46a9d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f46ac0aa0e0 0x7f46ac154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46a9d9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f46ac0aa0e0 0x7f46ac154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34772/0 (socket says 172.21.15.29:34772) 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46a9d9b700 1 -- 172.21.15.29:0/3164745840 learned_addr learned my addr 172.21.15.29:0/3164745840 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46aad9d700 1 --2- 172.21.15.29:0/3164745840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46ac0ab1f0 0x7f46ac0ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46aa59c700 1 --2- 172.21.15.29:0/3164745840 >> v2:172.21.15.163:3300/0 conn(0x7f46ac154370 0x7f46ac0c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46a9d9b700 1 -- 172.21.15.29:0/3164745840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46ac0ab1f0 msgr2=0x7f46ac0ab680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46a9d9b700 1 --2- 172.21.15.29:0/3164745840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46ac0ab1f0 0x7f46ac0ab680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46a9d9b700 1 -- 172.21.15.29:0/3164745840 >> v2:172.21.15.163:3300/0 conn(0x7f46ac154370 msgr2=0x7f46ac0c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46a9d9b700 1 --2- 172.21.15.29:0/3164745840 >> v2:172.21.15.163:3300/0 conn(0x7f46ac154370 0x7f46ac0c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46a9d9b700 1 -- 172.21.15.29:0/3164745840 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac0cb8a0 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.909+0000 7f46a9d9b700 1 --2- 172.21.15.29:0/3164745840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46ac0aa0e0 0x7f46ac154750 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 rx=0x7f46ac084050 tx=0x7f46a00090f0).ready entity=mon.0 client_cookie=8aa0d125dd8540ba server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.910+0000 7f46a959a700 1 -- 172.21.15.29:0/3164745840 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a000f510 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.910+0000 7f46a959a700 1 -- 172.21.15.29:0/3164745840 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a000f670 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.910+0000 7f46a959a700 1 -- 172.21.15.29:0/3164745840 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a000f9f0 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.910+0000 7f46b093c700 1 -- 172.21.15.29:0/3164745840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46ac0aa0e0 msgr2=0x7f46ac154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.910+0000 7f46b093c700 1 --2- 172.21.15.29:0/3164745840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46ac0aa0e0 0x7f46ac154750 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 rx=0x7f46ac084050 tx=0x7f46a00090f0).stop 2024-01-24T15:32:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.911+0000 7f46b093c700 1 -- 172.21.15.29:0/3164745840 shutdown_connections 2024-01-24T15:32:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.911+0000 7f46b093c700 1 --2- 172.21.15.29:0/3164745840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46ac0ab1f0 0x7f46ac0ab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.911+0000 7f46b093c700 1 --2- 172.21.15.29:0/3164745840 >> v2:172.21.15.163:3300/0 conn(0x7f46ac154370 0x7f46ac0c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.911+0000 7f46b093c700 1 --2- 172.21.15.29:0/3164745840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46ac0aa0e0 0x7f46ac154750 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.911+0000 7f46b093c700 1 -- 172.21.15.29:0/3164745840 >> 172.21.15.29:0/3164745840 conn(0x7f46ac05ec80 msgr2=0x7f46ac0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:05.916 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.911+0000 7f46b093c700 1 -- 172.21.15.29:0/3164745840 shutdown_connections 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.911+0000 7f46b093c700 1 -- 172.21.15.29:0/3164745840 wait complete. 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.912+0000 7f46b093c700 1 Processor -- start 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.912+0000 7f46b093c700 1 -- start start 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46b093c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46ac0aa0e0 0x7f46ac168360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46b093c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46ac0ab1f0 0x7f46ac153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46b093c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46ac154370 0x7f46ac1642b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46b093c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac167ed0 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46b093c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac1678c0 con 0x7f46ac0ab1f0 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46b093c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0b0a70 con 0x7f46ac154370 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46aa59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46ac0aa0e0 0x7f46ac168360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46aa59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46ac0aa0e0 0x7f46ac168360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34776/0 (socket says 172.21.15.29:34776) 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46aa59c700 1 -- 172.21.15.29:0/1213240528 learned_addr learned my addr 172.21.15.29:0/1213240528 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46aad9d700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46ac154370 0x7f46ac1642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46a9d9b700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46ac0ab1f0 0x7f46ac153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:05.917 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46aa59c700 1 -- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46ac154370 msgr2=0x7f46ac1642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46aa59c700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46ac154370 0x7f46ac1642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46aa59c700 1 -- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46ac0ab1f0 msgr2=0x7f46ac153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46aa59c700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46ac0ab1f0 0x7f46ac153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.913+0000 7f46aa59c700 1 -- 172.21.15.29:0/1213240528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a0009370 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.914+0000 7f46aa59c700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46ac0aa0e0 0x7f46ac168360 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7f469c00e8d0 tx=0x7f469c006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.914+0000 7f469a7fc700 1 -- 172.21.15.29:0/1213240528 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c009450 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.914+0000 7f469a7fc700 1 -- 172.21.15.29:0/1213240528 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469c012600 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.914+0000 7f46b093c700 1 -- 172.21.15.29:0/1213240528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ac164e00 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.914+0000 7f469a7fc700 1 -- 172.21.15.29:0/1213240528 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c012980 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.914+0000 7f46b093c700 1 -- 172.21.15.29:0/1213240528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ac167b60 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.915+0000 7f46b093c700 1 -- 172.21.15.29:0/1213240528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ac1681b0 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.915+0000 7f469a7fc700 1 -- 172.21.15.29:0/1213240528 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 95) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f469c012cf0 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.916+0000 7f469a7fc700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f468c05f390 0x7f468c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.916+0000 7f46a9d9b700 1 -- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f468c05f390 msgr2=0x7f468c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.916+0000 7f469a7fc700 1 -- 172.21.15.29:0/1213240528 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7502+0+0 (secure 0 0 0) 0x7f469c07af90 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.919 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.916+0000 7f46a9d9b700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f468c05f390 0x7f468c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:05.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:05.920+0000 7f469a7fc700 1 -- 172.21.15.29:0/1213240528 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f469c0a9b00 con 0x7f46ac0aa0e0 2024-01-24T15:32:05.997 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:05.995+0000 7fc8f7399680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:32:06.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.107+0000 7f46b093c700 1 -- 172.21.15.29:0/1213240528 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46ac164bc0 con 0x7f46ac0aa0e0 2024-01-24T15:32:06.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.108+0000 7f469a7fc700 1 -- 172.21.15.29:0/1213240528 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) v1 ==== 74+0+18725 (secure 0 0 0) 0x7f46ac164bc0 con 0x7f46ac0aa0e0 2024-01-24T15:32:06.110 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:06.111 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":216,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:31:51.918853+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:30:35.147040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-01-24T15:30:36.390723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.12","mappings":[{"from":0,"to":2}]},{"pgid":"25.32","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/35271322":"2024-01-25T15:30:50.111384+0000","172.21.15.163:0/3751389554":"2024-01-25T15:30:50.111384+0000","172.21.15.163:0/310856783":"2024-01-25T15:30:50.111384+0000","172.21.15.163:0/1051569194":"2024-01-25T15:30:50.111384+0000","172.21.15.163:6833/57365":"2024-01-25T15:30:50.111384+0000","172.21.15.163:6832/57365":"2024-01-25T15:30:50.111384+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:32:06.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.110+0000 7f46b093c700 1 -- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f468c05f390 msgr2=0x7f468c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.110+0000 7f46b093c700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f468c05f390 0x7f468c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.110+0000 7f46b093c700 1 -- 172.21.15.29:0/1213240528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46ac0aa0e0 msgr2=0x7f46ac168360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.110+0000 7f46b093c700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46ac0aa0e0 0x7f46ac168360 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7f469c00e8d0 tx=0x7f469c006f40).stop 2024-01-24T15:32:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.110+0000 7f46b093c700 1 -- 172.21.15.29:0/1213240528 shutdown_connections 2024-01-24T15:32:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.110+0000 7f46b093c700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f468c05f390 0x7f468c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.110+0000 7f46b093c700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46ac154370 0x7f46ac1642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.110+0000 7f46b093c700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46ac0aa0e0 0x7f46ac168360 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.110+0000 7f46b093c700 1 --2- 172.21.15.29:0/1213240528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46ac0ab1f0 0x7f46ac153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.110+0000 7f46b093c700 1 -- 172.21.15.29:0/1213240528 >> 172.21.15.29:0/1213240528 conn(0x7f46ac05ec80 msgr2=0x7f46ac0c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.110+0000 7f46b093c700 1 -- 172.21.15.29:0/1213240528 shutdown_connections 2024-01-24T15:32:06.113 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.110+0000 7f46b093c700 1 -- 172.21.15.29:0/1213240528 wait complete. 2024-01-24T15:32:06.127 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T15:32:06.127 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:32:06.191 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:06.189+0000 7f26c5437680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:32:06.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.256+0000 7f79a80ca700 1 Processor -- start 2024-01-24T15:32:06.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.256+0000 7f79a80ca700 1 -- start start 2024-01-24T15:32:06.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.257+0000 7f79a80ca700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f79a0141e00 0x7f79a01595b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:06.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.257+0000 7f79a80ca700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79a0065360 0x7f79a00657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:06.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.257+0000 7f79a80ca700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a0159aa0 con 0x7f79a00c4ca0 2024-01-24T15:32:06.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.257+0000 7f79a80ca700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a00bd2f0 con 0x7f79a0141e00 2024-01-24T15:32:06.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.257+0000 7f79a80ca700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a0069240 con 0x7f79a0065360 2024-01-24T15:32:06.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.257+0000 7f79a5665700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f79a00c4ca0 0x7f79a0069970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58010/0 (socket says 172.21.15.29:58010) 2024-01-24T15:32:06.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.257+0000 7f79a5665700 1 -- 172.21.15.29:0/1822078226 learned_addr learned my addr 172.21.15.29:0/1822078226 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:06.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.257+0000 7f79a5e66700 1 --2- 172.21.15.29:0/1822078226 >> v2:172.21.15.163:3300/0 conn(0x7f79a0141e00 0x7f79a01595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:06.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.257+0000 7f79a6667700 1 --2- 172.21.15.29:0/1822078226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79a0065360 0x7f79a00657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:06.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.257+0000 7f79a4e64700 1 -- 172.21.15.29:0/1822078226 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010948208 0 0) 0x7f79a0159aa0 con 0x7f79a00c4ca0 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.258+0000 7f79a4e64700 1 -- 172.21.15.29:0/1822078226 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f798c0033f0 con 0x7f79a00c4ca0 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.258+0000 7f79a6667700 1 -- 172.21.15.29:0/1822078226 >> v2:172.21.15.163:3300/0 conn(0x7f79a0141e00 msgr2=0x7f79a01595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.258+0000 7f79a6667700 1 --2- 172.21.15.29:0/1822078226 >> v2:172.21.15.163:3300/0 conn(0x7f79a0141e00 0x7f79a01595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.258+0000 7f79a6667700 1 -- 172.21.15.29:0/1822078226 >> v1:172.21.15.29:6789/0 conn(0x7f79a00c4ca0 legacy=0x7f79a0069970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.258+0000 7f79a6667700 1 -- 172.21.15.29:0/1822078226 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a0065ca0 con 0x7f79a0065360 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.258+0000 7f79a5e66700 1 --2- 172.21.15.29:0/1822078226 >> v2:172.21.15.163:3300/0 conn(0x7f79a0141e00 0x7f79a01595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.258+0000 7f79a6667700 1 --2- 172.21.15.29:0/1822078226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79a0065360 0x7f79a00657b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f7998009e60 tx=0x7f7998009780).ready entity=mon.2 client_cookie=3a582954faa330b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.258+0000 7f79a4e64700 1 -- 172.21.15.29:0/1822078226 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799800e070 con 0x7f79a0065360 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.258+0000 7f79a4e64700 1 -- 172.21.15.29:0/1822078226 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799800bab0 con 0x7f79a0065360 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.258+0000 7f79a4e64700 1 -- 172.21.15.29:0/1822078226 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799800be30 con 0x7f79a0065360 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.259+0000 7f79a80ca700 1 -- 172.21.15.29:0/1822078226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79a0065360 msgr2=0x7f79a00657b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.259+0000 7f79a80ca700 1 --2- 172.21.15.29:0/1822078226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79a0065360 0x7f79a00657b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f7998009e60 tx=0x7f7998009780).stop 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.259+0000 7f79a80ca700 1 -- 172.21.15.29:0/1822078226 shutdown_connections 2024-01-24T15:32:06.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.259+0000 7f79a80ca700 1 --2- 172.21.15.29:0/1822078226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79a0065360 0x7f79a00657b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.259+0000 7f79a80ca700 1 --2- 172.21.15.29:0/1822078226 >> v2:172.21.15.163:3300/0 conn(0x7f79a0141e00 0x7f79a01595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.259+0000 7f79a80ca700 1 -- 172.21.15.29:0/1822078226 >> 172.21.15.29:0/1822078226 conn(0x7f79a00c85c0 msgr2=0x7f79a00c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:06.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.259+0000 7f79a80ca700 1 -- 172.21.15.29:0/1822078226 shutdown_connections 2024-01-24T15:32:06.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.259+0000 7f79a80ca700 1 -- 172.21.15.29:0/1822078226 wait complete. 2024-01-24T15:32:06.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.260+0000 7f79a80ca700 1 Processor -- start 2024-01-24T15:32:06.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.260+0000 7f79a80ca700 1 -- start start 2024-01-24T15:32:06.264 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:06.258+0000 7f26c5437680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:32:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a80ca700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79a0065360 0x7f79a013fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a80ca700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a00c4ca0 0x7f79a00c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a80ca700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a0141e00 0x7f79a00c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a80ca700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a00c7670 con 0x7f79a00c4ca0 2024-01-24T15:32:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a80ca700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a00c1990 con 0x7f79a0141e00 2024-01-24T15:32:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a80ca700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a00c21f0 con 0x7f79a0065360 2024-01-24T15:32:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a5665700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a00c4ca0 0x7f79a00c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a5665700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a00c4ca0 0x7f79a00c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34788/0 (socket says 172.21.15.29:34788) 2024-01-24T15:32:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a5665700 1 -- 172.21.15.29:0/999076724 learned_addr learned my addr 172.21.15.29:0/999076724 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a5e66700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79a0065360 0x7f79a013fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a6667700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a0141e00 0x7f79a00c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:06.267 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a5665700 1 -- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79a0065360 msgr2=0x7f79a013fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a5665700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79a0065360 0x7f79a013fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a5665700 1 -- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a0141e00 msgr2=0x7f79a00c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a5665700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a0141e00 0x7f79a00c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a5665700 1 -- 172.21.15.29:0/999076724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c0092a0 con 0x7f79a00c4ca0 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.261+0000 7f79a5665700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a00c4ca0 0x7f79a00c6c90 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 rx=0x7f799000c450 tx=0x7f7990007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.262+0000 7f7995ffb700 1 -- 172.21.15.29:0/999076724 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79900070d0 con 0x7f79a00c4ca0 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.262+0000 7f7995ffb700 1 -- 172.21.15.29:0/999076724 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7990007230 con 0x7f79a00c4ca0 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.262+0000 7f79a80ca700 1 -- 172.21.15.29:0/999076724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7998009a20 con 0x7f79a00c4ca0 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.262+0000 7f7995ffb700 1 -- 172.21.15.29:0/999076724 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79900075b0 con 0x7f79a00c4ca0 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.262+0000 7f79a80ca700 1 -- 172.21.15.29:0/999076724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a00c1f10 con 0x7f79a00c4ca0 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.263+0000 7f79a80ca700 1 -- 172.21.15.29:0/999076724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a00c1d20 con 0x7f79a00c4ca0 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.263+0000 7f7995ffb700 1 -- 172.21.15.29:0/999076724 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 95) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f79900059b0 con 0x7f79a00c4ca0 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.264+0000 7f7995ffb700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f798405f390 0x7f7984061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:06.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.264+0000 7f79a5e66700 1 -- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f798405f390 msgr2=0x7f7984061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.264+0000 7f7995ffb700 1 -- 172.21.15.29:0/999076724 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7502+0+0 (secure 0 0 0) 0x7f799007a180 con 0x7f79a00c4ca0 2024-01-24T15:32:06.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.264+0000 7f79a5e66700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f798405f390 0x7f7984061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:06.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.268+0000 7f7995ffb700 1 -- 172.21.15.29:0/999076724 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f79a00c1d20 con 0x7f79a00c4ca0 2024-01-24T15:32:06.336 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:06.335+0000 7f26c5437680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:32:06.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.465+0000 7f79a5e66700 1 -- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f798405f390 msgr2=0x7f7984061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:06.467 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.465+0000 7f79a5e66700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f798405f390 0x7f7984061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:32:06.483 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:06.482+0000 7f26c5437680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:32:06.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.496+0000 7f79a80ca700 1 -- 172.21.15.29:0/999076724 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79a00c1d20 con 0x7f79a00c4ca0 2024-01-24T15:32:06.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.496+0000 7f7995ffb700 1 -- 172.21.15.29:0/999076724 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f79a00c1d20 con 0x7f79a00c4ca0 2024-01-24T15:32:06.498 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:06.499 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8060,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3418669722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3418669722},{"type":"v1","addr":"172.21.15.163:6836","nonce":3418669722}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8063,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/407488941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":407488941},{"type":"v1","addr":"172.21.15.163:6837","nonce":407488941}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4018429461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4018429461},{"type":"v1","addr":"172.21.15.29:6834","nonce":4018429461}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:30:37.411326+0000","modified":"2024-01-24T15:30:40.428239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2886498894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2886498894},{"type":"v1","addr":"172.21.15.29:6835","nonce":2886498894}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-24T15:32:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.498+0000 7f79a80ca700 1 -- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f798405f390 msgr2=0x7f7984061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.498+0000 7f79a80ca700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f798405f390 0x7f7984061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.498+0000 7f79a80ca700 1 -- 172.21.15.29:0/999076724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a00c4ca0 msgr2=0x7f79a00c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.498+0000 7f79a80ca700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a00c4ca0 0x7f79a00c6c90 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 rx=0x7f799000c450 tx=0x7f7990007c80).stop 2024-01-24T15:32:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.498+0000 7f79a80ca700 1 -- 172.21.15.29:0/999076724 shutdown_connections 2024-01-24T15:32:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.498+0000 7f79a80ca700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a0141e00 0x7f79a00c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.498+0000 7f79a80ca700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f798405f390 0x7f7984061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.498+0000 7f79a80ca700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79a0065360 0x7f79a013fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.498+0000 7f79a80ca700 1 --2- 172.21.15.29:0/999076724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a00c4ca0 0x7f79a00c6c90 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.498+0000 7f79a80ca700 1 -- 172.21.15.29:0/999076724 >> 172.21.15.29:0/999076724 conn(0x7f79a00c85c0 msgr2=0x7f79a0140c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.498+0000 7f79a80ca700 1 -- 172.21.15.29:0/999076724 shutdown_connections 2024-01-24T15:32:06.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.498+0000 7f79a80ca700 1 -- 172.21.15.29:0/999076724 wait complete. 2024-01-24T15:32:06.502 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 90 2024-01-24T15:32:06.514 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:32:06.554 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:06.553+0000 7f26c5437680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:32:06.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.647+0000 7f7badd5b700 1 Processor -- start 2024-01-24T15:32:06.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.647+0000 7f7badd5b700 1 -- start start 2024-01-24T15:32:06.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7badd5b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7ba8150d40 0x7f7ba8069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:06.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7badd5b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ba8155040 0x7f7ba80c3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:06.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7badd5b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba80c2ce0 con 0x7f7ba8065ec0 2024-01-24T15:32:06.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7badd5b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba8069890 con 0x7f7ba8155040 2024-01-24T15:32:06.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7badd5b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba8069b70 con 0x7f7ba8150d40 2024-01-24T15:32:06.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7ba7fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ba8155040 0x7f7ba80c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:06.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7ba7fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ba8155040 0x7f7ba80c3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34802/0 (socket says 172.21.15.29:34802) 2024-01-24T15:32:06.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7ba7fff700 1 -- 172.21.15.29:0/2127922523 learned_addr learned my addr 172.21.15.29:0/2127922523 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:06.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7ba77fe700 1 --2- 172.21.15.29:0/2127922523 >> v2:172.21.15.163:3300/0 conn(0x7f7ba8150d40 0x7f7ba8069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:06.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7ba7fff700 1 -- 172.21.15.29:0/2127922523 >> v2:172.21.15.163:3300/0 conn(0x7f7ba8150d40 msgr2=0x7f7ba8069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:06.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7ba7fff700 1 --2- 172.21.15.29:0/2127922523 >> v2:172.21.15.163:3300/0 conn(0x7f7ba8150d40 0x7f7ba8069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7ba7fff700 1 -- 172.21.15.29:0/2127922523 >> v1:172.21.15.163:6789/0 conn(0x7f7ba8065ec0 legacy=0x7f7ba8154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:06.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7ba7fff700 1 -- 172.21.15.29:0/2127922523 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba8069e50 con 0x7f7ba8155040 2024-01-24T15:32:06.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7ba7fff700 1 --2- 172.21.15.29:0/2127922523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ba8155040 0x7f7ba80c3150 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 rx=0x7f7ba80648d0 tx=0x7f7ba0009840).ready entity=mon.0 client_cookie=cb45821253aa71d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:06.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.648+0000 7f7ba6ffd700 1 -- 172.21.15.29:0/2127922523 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba000e070 con 0x7f7ba8155040 2024-01-24T15:32:06.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.649+0000 7f7ba6ffd700 1 -- 172.21.15.29:0/2127922523 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba000ba20 con 0x7f7ba8155040 2024-01-24T15:32:06.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.649+0000 7f7ba6ffd700 1 -- 172.21.15.29:0/2127922523 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba000bda0 con 0x7f7ba8155040 2024-01-24T15:32:06.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.649+0000 7f7badd5b700 1 -- 172.21.15.29:0/2127922523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ba8155040 msgr2=0x7f7ba80c3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:06.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.649+0000 7f7badd5b700 1 --2- 172.21.15.29:0/2127922523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ba8155040 0x7f7ba80c3150 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 rx=0x7f7ba80648d0 tx=0x7f7ba0009840).stop 2024-01-24T15:32:06.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.649+0000 7f7badd5b700 1 -- 172.21.15.29:0/2127922523 shutdown_connections 2024-01-24T15:32:06.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.649+0000 7f7badd5b700 1 --2- 172.21.15.29:0/2127922523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ba8155040 0x7f7ba80c3150 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.649+0000 7f7badd5b700 1 --2- 172.21.15.29:0/2127922523 >> v2:172.21.15.163:3300/0 conn(0x7f7ba8150d40 0x7f7ba8069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.649+0000 7f7badd5b700 1 -- 172.21.15.29:0/2127922523 >> 172.21.15.29:0/2127922523 conn(0x7f7ba80c8580 msgr2=0x7f7ba80c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:06.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.649+0000 7f7badd5b700 1 -- 172.21.15.29:0/2127922523 shutdown_connections 2024-01-24T15:32:06.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.649+0000 7f7badd5b700 1 -- 172.21.15.29:0/2127922523 wait complete. 2024-01-24T15:32:06.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.651+0000 7f7badd5b700 1 Processor -- start 2024-01-24T15:32:06.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.651+0000 7f7badd5b700 1 -- start start 2024-01-24T15:32:06.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.651+0000 7f7badd5b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ba8065ec0 0x7f7ba8154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:06.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.651+0000 7f7badd5b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ba8150d40 0x7f7ba80beb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:06.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.651+0000 7f7badd5b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ba8155040 0x7f7ba80633c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:06.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.651+0000 7f7badd5b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80bf4e0 con 0x7f7ba8065ec0 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.651+0000 7f7badd5b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80bf7c0 con 0x7f7ba8150d40 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.651+0000 7f7badd5b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80bfaa0 con 0x7f7ba8155040 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7ba77fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ba8065ec0 0x7f7ba8154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7ba77fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ba8065ec0 0x7f7ba8154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34812/0 (socket says 172.21.15.29:34812) 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7ba7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ba8155040 0x7f7ba80633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7ba77fe700 1 -- 172.21.15.29:0/710102066 learned_addr learned my addr 172.21.15.29:0/710102066 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7b9ffff700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ba8150d40 0x7f7ba80beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7ba77fe700 1 -- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ba8155040 msgr2=0x7f7ba80633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7ba77fe700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ba8155040 0x7f7ba80633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7ba77fe700 1 -- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ba8150d40 msgr2=0x7f7ba80beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7ba77fe700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ba8150d40 0x7f7ba80beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7ba77fe700 1 -- 172.21.15.29:0/710102066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba0009ae0 con 0x7f7ba8065ec0 2024-01-24T15:32:06.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7b9ffff700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ba8150d40 0x7f7ba80beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:06.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7ba77fe700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ba8065ec0 0x7f7ba8154130 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 rx=0x7f7b9800ebb0 tx=0x7f7b9800b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:06.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7b9f7fe700 1 -- 172.21.15.29:0/710102066 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800da50 con 0x7f7ba8065ec0 2024-01-24T15:32:06.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7b9f7fe700 1 -- 172.21.15.29:0/710102066 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9800dbb0 con 0x7f7ba8065ec0 2024-01-24T15:32:06.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7b9f7fe700 1 -- 172.21.15.29:0/710102066 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98010440 con 0x7f7ba8065ec0 2024-01-24T15:32:06.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.652+0000 7f7badd5b700 1 -- 172.21.15.29:0/710102066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba80bfd80 con 0x7f7ba8065ec0 2024-01-24T15:32:06.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.653+0000 7f7badd5b700 1 -- 172.21.15.29:0/710102066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba8154620 con 0x7f7ba8065ec0 2024-01-24T15:32:06.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.654+0000 7f7badd5b700 1 -- 172.21.15.29:0/710102066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba80475e0 con 0x7f7ba8065ec0 2024-01-24T15:32:06.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.654+0000 7f7b9f7fe700 1 -- 172.21.15.29:0/710102066 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 95) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f7b98010830 con 0x7f7ba8065ec0 2024-01-24T15:32:06.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.654+0000 7f7b9f7fe700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7b8805f390 0x7f7b88061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:06.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.655+0000 7f7b9f7fe700 1 -- 172.21.15.29:0/710102066 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7502+0+0 (secure 0 0 0) 0x7f7b9807b340 con 0x7f7ba8065ec0 2024-01-24T15:32:06.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.655+0000 7f7b9ffff700 1 -- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7b8805f390 msgr2=0x7f7b88061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:06.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.655+0000 7f7b9ffff700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7b8805f390 0x7f7b88061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:06.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.659+0000 7f7b9f7fe700 1 -- 172.21.15.29:0/710102066 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7b9804bdb0 con 0x7f7ba8065ec0 2024-01-24T15:32:06.671 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:06.669+0000 7f26c5437680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:32:06.773 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:06.771+0000 7fc8f7399680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:32:06.779 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:06.777+0000 7f26c5437680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:32:06.853 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:06.850+0000 7fc8f7399680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:32:06.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.856+0000 7f7b9ffff700 1 -- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7b8805f390 msgr2=0x7f7b88061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:06.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.856+0000 7f7b9ffff700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7b8805f390 0x7f7b88061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:32:06.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.890+0000 7f7badd5b700 1 -- 172.21.15.29:0/710102066 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ba80c51a0 con 0x7f7ba8065ec0 2024-01-24T15:32:06.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.890+0000 7f7b9f7fe700 1 -- 172.21.15.29:0/710102066 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f7b9804f3d0 con 0x7f7ba8065ec0 2024-01-24T15:32:06.892 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:06.893 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8060,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3418669722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3418669722},{"type":"v1","addr":"172.21.15.163:6836","nonce":3418669722}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8063,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/407488941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":407488941},{"type":"v1","addr":"172.21.15.163:6837","nonce":407488941}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4018429461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4018429461},{"type":"v1","addr":"172.21.15.29:6834","nonce":4018429461}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:30:37.411326+0000","modified":"2024-01-24T15:30:40.428239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2886498894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2886498894},{"type":"v1","addr":"172.21.15.29:6835","nonce":2886498894}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-24T15:32:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.892+0000 7f7badd5b700 1 -- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7b8805f390 msgr2=0x7f7b88061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.892+0000 7f7badd5b700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7b8805f390 0x7f7b88061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.892+0000 7f7badd5b700 1 -- 172.21.15.29:0/710102066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ba8065ec0 msgr2=0x7f7ba8154130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:06.895 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.892+0000 7f7badd5b700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ba8065ec0 0x7f7ba8154130 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 rx=0x7f7b9800ebb0 tx=0x7f7b9800b300).stop 2024-01-24T15:32:06.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.892+0000 7f7badd5b700 1 -- 172.21.15.29:0/710102066 shutdown_connections 2024-01-24T15:32:06.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.892+0000 7f7badd5b700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7b8805f390 0x7f7b88061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:06.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.892+0000 7f7badd5b700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ba8155040 0x7f7ba80633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.892+0000 7f7badd5b700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ba8065ec0 0x7f7ba8154130 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.892+0000 7f7badd5b700 1 --2- 172.21.15.29:0/710102066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ba8150d40 0x7f7ba80beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:06.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.892+0000 7f7badd5b700 1 -- 172.21.15.29:0/710102066 >> 172.21.15.29:0/710102066 conn(0x7f7ba80c8580 msgr2=0x7f7ba81494b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:06.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.892+0000 7f7badd5b700 1 -- 172.21.15.29:0/710102066 shutdown_connections 2024-01-24T15:32:06.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:06.892+0000 7f7badd5b700 1 -- 172.21.15.29:0/710102066 wait complete. 2024-01-24T15:32:06.896 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 90 2024-01-24T15:32:06.909 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:32:06.949 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:06.947+0000 7fc8f7399680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:32:07.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.046+0000 7f44e94cd700 1 Processor -- start 2024-01-24T15:32:07.048 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.046+0000 7f44e94cd700 1 -- start start 2024-01-24T15:32:07.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e94cd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f44e4062ec0 0x7f44e41527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:07.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e94cd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f44e4152cd0 0x7f44e415be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:07.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e94cd700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e4064d90 con 0x7f44e4134f10 2024-01-24T15:32:07.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e94cd700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e4064250 con 0x7f44e4152cd0 2024-01-24T15:32:07.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e94cd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e4065220 con 0x7f44e4062ec0 2024-01-24T15:32:07.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e37fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f44e4152cd0 0x7f44e415be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:07.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e2ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f44e4134f10 0x7f44e40627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58024/0 (socket says 172.21.15.29:58024) 2024-01-24T15:32:07.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e2ffd700 1 -- 172.21.15.29:0/3208664836 learned_addr learned my addr 172.21.15.29:0/3208664836 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:07.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e27fc700 1 --2- 172.21.15.29:0/3208664836 >> v2:172.21.15.163:3300/0 conn(0x7f44e4062ec0 0x7f44e41527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:07.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e37fe700 1 -- 172.21.15.29:0/3208664836 >> v2:172.21.15.163:3300/0 conn(0x7f44e4062ec0 msgr2=0x7f44e41527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:07.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e37fe700 1 --2- 172.21.15.29:0/3208664836 >> v2:172.21.15.163:3300/0 conn(0x7f44e4062ec0 0x7f44e41527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e37fe700 1 -- 172.21.15.29:0/3208664836 >> v1:172.21.15.29:6789/0 conn(0x7f44e4134f10 legacy=0x7f44e40627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:07.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e37fe700 1 -- 172.21.15.29:0/3208664836 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4064460 con 0x7f44e4152cd0 2024-01-24T15:32:07.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e1ffb700 1 -- 172.21.15.29:0/3208664836 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333611935 0 0) 0x7f44e4064d90 con 0x7f44e4134f10 2024-01-24T15:32:07.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.047+0000 7f44e37fe700 1 --2- 172.21.15.29:0/3208664836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e4152cd0 0x7f44e415be40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7f44e4069d20 tx=0x7f44dc009780).ready entity=mon.0 client_cookie=1faa44eb487cd0b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:07.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.048+0000 7f44e1ffb700 1 -- 172.21.15.29:0/3208664836 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc00e070 con 0x7f44e4152cd0 2024-01-24T15:32:07.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.048+0000 7f44e1ffb700 1 -- 172.21.15.29:0/3208664836 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44dc00bb60 con 0x7f44e4152cd0 2024-01-24T15:32:07.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.048+0000 7f44e1ffb700 1 -- 172.21.15.29:0/3208664836 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc012410 con 0x7f44e4152cd0 2024-01-24T15:32:07.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.048+0000 7f44e94cd700 1 -- 172.21.15.29:0/3208664836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e4152cd0 msgr2=0x7f44e415be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:07.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.048+0000 7f44e94cd700 1 --2- 172.21.15.29:0/3208664836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e4152cd0 0x7f44e415be40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7f44e4069d20 tx=0x7f44dc009780).stop 2024-01-24T15:32:07.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.048+0000 7f44e94cd700 1 -- 172.21.15.29:0/3208664836 shutdown_connections 2024-01-24T15:32:07.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.048+0000 7f44e94cd700 1 --2- 172.21.15.29:0/3208664836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e4152cd0 0x7f44e415be40 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.048+0000 7f44e94cd700 1 --2- 172.21.15.29:0/3208664836 >> v2:172.21.15.163:3300/0 conn(0x7f44e4062ec0 0x7f44e41527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.048+0000 7f44e94cd700 1 -- 172.21.15.29:0/3208664836 >> 172.21.15.29:0/3208664836 conn(0x7f44e40c69f0 msgr2=0x7f44e40c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:07.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.048+0000 7f44e94cd700 1 -- 172.21.15.29:0/3208664836 shutdown_connections 2024-01-24T15:32:07.053 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.048+0000 7f44e94cd700 1 -- 172.21.15.29:0/3208664836 wait complete. 2024-01-24T15:32:07.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.050+0000 7f44e94cd700 1 Processor -- start 2024-01-24T15:32:07.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.050+0000 7f44e94cd700 1 -- start start 2024-01-24T15:32:07.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.051+0000 7f44e94cd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e4062ec0 0x7f44e4135f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:07.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.051+0000 7f44e94cd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44e4134f10 0x7f44e4151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.051+0000 7f44e94cd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44e4152cd0 0x7f44e40bea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.051+0000 7f44e94cd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40abbf0 con 0x7f44e4062ec0 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.051+0000 7f44e94cd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e4133cb0 con 0x7f44e4152cd0 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.051+0000 7f44e94cd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e41364f0 con 0x7f44e4134f10 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.051+0000 7f44e2ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e4062ec0 0x7f44e4135f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.051+0000 7f44e2ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e4062ec0 0x7f44e4135f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34830/0 (socket says 172.21.15.29:34830) 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.051+0000 7f44e2ffd700 1 -- 172.21.15.29:0/379698463 learned_addr learned my addr 172.21.15.29:0/379698463 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.051+0000 7f44e27fc700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44e4134f10 0x7f44e4151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.052+0000 7f44e37fe700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44e4152cd0 0x7f44e40bea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.052+0000 7f44e2ffd700 1 -- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44e4134f10 msgr2=0x7f44e4151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.052+0000 7f44e2ffd700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44e4134f10 0x7f44e4151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.052+0000 7f44e2ffd700 1 -- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44e4152cd0 msgr2=0x7f44e40bea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:07.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.052+0000 7f44e2ffd700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44e4152cd0 0x7f44e40bea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.052+0000 7f44e2ffd700 1 -- 172.21.15.29:0/379698463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc009a20 con 0x7f44e4062ec0 2024-01-24T15:32:07.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.052+0000 7f44e2ffd700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e4062ec0 0x7f44e4135f10 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7f44d400c4e0 tx=0x7f44d4007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:07.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.052+0000 7f44caffd700 1 -- 172.21.15.29:0/379698463 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d400ce90 con 0x7f44e4062ec0 2024-01-24T15:32:07.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.052+0000 7f44caffd700 1 -- 172.21.15.29:0/379698463 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d40072e0 con 0x7f44e4062ec0 2024-01-24T15:32:07.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.052+0000 7f44e94cd700 1 -- 172.21.15.29:0/379698463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e40aa850 con 0x7f44e4062ec0 2024-01-24T15:32:07.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.052+0000 7f44caffd700 1 -- 172.21.15.29:0/379698463 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4007660 con 0x7f44e4062ec0 2024-01-24T15:32:07.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.052+0000 7f44e94cd700 1 -- 172.21.15.29:0/379698463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e40aae10 con 0x7f44e4062ec0 2024-01-24T15:32:07.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.053+0000 7f44e94cd700 1 -- 172.21.15.29:0/379698463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e40abd60 con 0x7f44e4062ec0 2024-01-24T15:32:07.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.054+0000 7f44caffd700 1 -- 172.21.15.29:0/379698463 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 95) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f44d40059b0 con 0x7f44e4062ec0 2024-01-24T15:32:07.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.055+0000 7f44caffd700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f44d005f390 0x7f44d0061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:07.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.055+0000 7f44caffd700 1 -- 172.21.15.29:0/379698463 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7558+0+0 (secure 0 0 0) 0x7f44d407a870 con 0x7f44e4062ec0 2024-01-24T15:32:07.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.055+0000 7f44e27fc700 1 -- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f44d005f390 msgr2=0x7f44d0061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:07.059 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.055+0000 7f44e27fc700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f44d005f390 0x7f44d0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:07.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.059+0000 7f44caffd700 1 -- 172.21.15.29:0/379698463 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f44d404b2e0 con 0x7f44e4062ec0 2024-01-24T15:32:07.077 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:07.075+0000 7f26c5437680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:32:07.128 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:07.126+0000 7fc8f7399680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:32:07.151 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:07.150+0000 7f26c5437680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:32:07.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.171+0000 7f44caffd700 1 -- 172.21.15.29:0/379698463 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 96) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f44d4077de0 con 0x7f44e4062ec0 2024-01-24T15:32:07.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.172+0000 7f44caffd700 1 -- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f44d005f390 msgr2=0x7f44d0061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:07.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.172+0000 7f44caffd700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f44d005f390 0x7f44d0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:07.216 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:07.213+0000 7fc8f7399680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:32:07.237 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:07.235+0000 7f263e9b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:07.237 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:07.235+0000 7f263e9b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:07.237 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:07.235+0000 7f263e9b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:07.237 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:07.235+0000 7f263e9b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:07.237 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:07.235+0000 7f263e9b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:07.238 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:07.237+0000 7f26409bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:07.238 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:07.237+0000 7f26409bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:07.238 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:07.237+0000 7f26409bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:07.238 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:07.237+0000 7f26409bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:07.238 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:07.237+0000 7f26409bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:07.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.257+0000 7f44e94cd700 1 -- 172.21.15.29:0/379698463 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44e40abd30 con 0x7f44e4062ec0 2024-01-24T15:32:07.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.259+0000 7f44caffd700 1 -- 172.21.15.29:0/379698463 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v218) v1 ==== 74+0+19169 (secure 0 0 0) 0x7f44d4018070 con 0x7f44e4062ec0 2024-01-24T15:32:07.261 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:07.261 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":218,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:32:07.164750+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:30:35.147040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-01-24T15:30:36.390723+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":217,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.12","mappings":[{"from":0,"to":2}]},{"pgid":"25.32","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/548999321":"2024-01-25T15:32:07.164681+0000","172.21.15.163:0/1908697403":"2024-01-25T15:32:07.164681+0000","172.21.15.163:0/1490553657":"2024-01-25T15:32:07.164681+0000","172.21.15.163:0/1486638326":"2024-01-25T15:32:07.164681+0000","172.21.15.163:0/35271322":"2024-01-25T15:30:50.111384+0000","172.21.15.163:0/821122947":"2024-01-25T15:32:07.164681+0000","172.21.15.163:0/3751389554":"2024-01-25T15:30:50.111384+0000","172.21.15.163:0/310856783":"2024-01-25T15:30:50.111384+0000","172.21.15.163:0/3227444295":"2024-01-25T15:32:07.164681+0000","172.21.15.163:0/1051569194":"2024-01-25T15:30:50.111384+0000","172.21.15.163:6833/57365":"2024-01-25T15:30:50.111384+0000","172.21.15.163:6832/57365":"2024-01-25T15:30:50.111384+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:32:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.261+0000 7f44e94cd700 1 -- 172.21.15.29:0/379698463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e4062ec0 msgr2=0x7f44e4135f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.261+0000 7f44e94cd700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e4062ec0 0x7f44e4135f10 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7f44d400c4e0 tx=0x7f44d4007c70).stop 2024-01-24T15:32:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.261+0000 7f44e94cd700 1 -- 172.21.15.29:0/379698463 shutdown_connections 2024-01-24T15:32:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.261+0000 7f44e94cd700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44e4152cd0 0x7f44e40bea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.261+0000 7f44e94cd700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e4062ec0 0x7f44e4135f10 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.261+0000 7f44e94cd700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f44d005f390 0x7f44d0061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:07.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.261+0000 7f44e94cd700 1 --2- 172.21.15.29:0/379698463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44e4134f10 0x7f44e4151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.261+0000 7f44e94cd700 1 -- 172.21.15.29:0/379698463 >> 172.21.15.29:0/379698463 conn(0x7f44e40c69f0 msgr2=0x7f44e41344a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:07.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.261+0000 7f44e94cd700 1 -- 172.21.15.29:0/379698463 shutdown_connections 2024-01-24T15:32:07.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.261+0000 7f44e94cd700 1 -- 172.21.15.29:0/379698463 wait complete. 2024-01-24T15:32:07.277 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-24T15:32:07.359 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:07.357+0000 7fc8f7399680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:32:07.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.449+0000 7f19fc03b700 1 Processor -- start 2024-01-24T15:32:07.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.449+0000 7f19fc03b700 1 -- start start 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.450+0000 7f19fc03b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19f4154240 0x7f19f4064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19fc03b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19f40a9d20 0x7f19f40aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19fc03b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f41550e0 con 0x7f19f4157cf0 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19fc03b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f40c0d10 con 0x7f19f4154240 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19fc03b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40aa670 con 0x7f19f40a9d20 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19f9dd7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19f4154240 0x7f19f4064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19f9dd7700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19f4154240 0x7f19f4064780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34846/0 (socket says 172.21.15.29:34846) 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19f9dd7700 1 -- 172.21.15.29:0/2883081658 learned_addr learned my addr 172.21.15.29:0/2883081658 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19f95d6700 1 --2- 172.21.15.29:0/2883081658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19f40a9d20 0x7f19f40aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19f9dd7700 1 -- 172.21.15.29:0/2883081658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19f40a9d20 msgr2=0x7f19f40aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19f9dd7700 1 --2- 172.21.15.29:0/2883081658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19f40a9d20 0x7f19f40aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19f9dd7700 1 -- 172.21.15.29:0/2883081658 >> v1:172.21.15.163:6789/0 conn(0x7f19f4157cf0 legacy=0x7f19f4158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19f9dd7700 1 -- 172.21.15.29:0/2883081658 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40650d0 con 0x7f19f4154240 2024-01-24T15:32:07.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19f8dd5700 1 -- 172.21.15.29:0/2883081658 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166965628 0 0) 0x7f19f41550e0 con 0x7f19f4157cf0 2024-01-24T15:32:07.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19f9dd7700 1 --2- 172.21.15.29:0/2883081658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19f4154240 0x7f19f4064780 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 rx=0x7f19f0006fd0 tx=0x7f19f00090f0).ready entity=mon.0 client_cookie=31577b4d0762c12b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:07.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.451+0000 7f19f8dd5700 1 -- 172.21.15.29:0/2883081658 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f000b580 con 0x7f19f4154240 2024-01-24T15:32:07.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.452+0000 7f19f8dd5700 1 -- 172.21.15.29:0/2883081658 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f000b6e0 con 0x7f19f4154240 2024-01-24T15:32:07.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.452+0000 7f19f8dd5700 1 -- 172.21.15.29:0/2883081658 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f000ba60 con 0x7f19f4154240 2024-01-24T15:32:07.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.452+0000 7f19fc03b700 1 -- 172.21.15.29:0/2883081658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19f4154240 msgr2=0x7f19f4064780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:07.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.452+0000 7f19fc03b700 1 --2- 172.21.15.29:0/2883081658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19f4154240 0x7f19f4064780 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 rx=0x7f19f0006fd0 tx=0x7f19f00090f0).stop 2024-01-24T15:32:07.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.452+0000 7f19fc03b700 1 -- 172.21.15.29:0/2883081658 shutdown_connections 2024-01-24T15:32:07.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.452+0000 7f19fc03b700 1 --2- 172.21.15.29:0/2883081658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19f4154240 0x7f19f4064780 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.452+0000 7f19fc03b700 1 --2- 172.21.15.29:0/2883081658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19f40a9d20 0x7f19f40aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.452+0000 7f19fc03b700 1 -- 172.21.15.29:0/2883081658 >> 172.21.15.29:0/2883081658 conn(0x7f19f40c8310 msgr2=0x7f19f405e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:07.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.452+0000 7f19fc03b700 1 -- 172.21.15.29:0/2883081658 shutdown_connections 2024-01-24T15:32:07.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.452+0000 7f19fc03b700 1 -- 172.21.15.29:0/2883081658 wait complete. 2024-01-24T15:32:07.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.454+0000 7f19fc03b700 1 Processor -- start 2024-01-24T15:32:07.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.454+0000 7f19fc03b700 1 -- start start 2024-01-24T15:32:07.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fc03b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19f40a9d20 0x7f19f40c4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fc03b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19f4154240 0x7f19f4167db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fc03b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19f4157cf0 0x7f19f41682a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fc03b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40aefe0 con 0x7f19f4157cf0 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fc03b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40ae700 con 0x7f19f40a9d20 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fc03b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40b0ba0 con 0x7f19f4154240 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fa5d8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19f4157cf0 0x7f19f41682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fa5d8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19f4157cf0 0x7f19f41682a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34854/0 (socket says 172.21.15.29:34854) 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fa5d8700 1 -- 172.21.15.29:0/1184610772 learned_addr learned my addr 172.21.15.29:0/1184610772 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19f95d6700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19f4154240 0x7f19f4167db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19f9dd7700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19f40a9d20 0x7f19f40c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fa5d8700 1 -- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19f4154240 msgr2=0x7f19f4167db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fa5d8700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19f4154240 0x7f19f4167db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fa5d8700 1 -- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19f40a9d20 msgr2=0x7f19f40c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fa5d8700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19f40a9d20 0x7f19f40c4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:07.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fa5d8700 1 -- 172.21.15.29:0/1184610772 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f0009370 con 0x7f19f4157cf0 2024-01-24T15:32:07.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19f9dd7700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19f40a9d20 0x7f19f40c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:07.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fa5d8700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19f4157cf0 0x7f19f41682a0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7f19ec00c810 tx=0x7f19ec001500).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:07.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19e9ffb700 1 -- 172.21.15.29:0/1184610772 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec010510 con 0x7f19f4157cf0 2024-01-24T15:32:07.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19e9ffb700 1 -- 172.21.15.29:0/1184610772 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec010670 con 0x7f19f4157cf0 2024-01-24T15:32:07.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19e9ffb700 1 -- 172.21.15.29:0/1184610772 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec010a10 con 0x7f19f4157cf0 2024-01-24T15:32:07.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.455+0000 7f19fc03b700 1 -- 172.21.15.29:0/1184610772 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f40ae990 con 0x7f19f4157cf0 2024-01-24T15:32:07.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.456+0000 7f19fc03b700 1 -- 172.21.15.29:0/1184610772 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f40b0730 con 0x7f19f4157cf0 2024-01-24T15:32:07.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.457+0000 7f19fc03b700 1 -- 172.21.15.29:0/1184610772 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f40af190 con 0x7f19f4157cf0 2024-01-24T15:32:07.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.457+0000 7f19e9ffb700 1 -- 172.21.15.29:0/1184610772 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 96) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f19ec012440 con 0x7f19f4157cf0 2024-01-24T15:32:07.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.457+0000 7f19e9ffb700 1 -- 172.21.15.29:0/1184610772 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 7788+0+0 (secure 0 0 0) 0x7f19ec079f20 con 0x7f19f4157cf0 2024-01-24T15:32:07.468 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.466+0000 7f19e9ffb700 1 -- 172.21.15.29:0/1184610772 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f19ec048b00 con 0x7f19f4157cf0 2024-01-24T15:32:07.490 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:07.488+0000 7fc8f7399680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:32:07.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:07.702+0000 7f19fc03b700 1 -- 172.21.15.29:0/1184610772 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f19f40c75b0 con 0x7f19f4157cf0 2024-01-24T15:32:07.833 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:07.831+0000 7fc8f7399680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:32:07.915 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:07.913+0000 7fc8f7399680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:32:08.170 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-01-24T15:32:08.168+0000 7fe09bfc3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-24T15:32:08.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.173+0000 7f19e9ffb700 1 -- 172.21.15.29:0/1184610772 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 97) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f19ec077840 con 0x7f19f4157cf0 2024-01-24T15:32:08.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.174+0000 7f19e9ffb700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f19d80302f0 0x7f19d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:08.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.174+0000 7f19f9dd7700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f19d80302f0 0x7f19d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:08.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.176+0000 7f19f9dd7700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f19d80302f0 0x7f19d8068bf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f19f0009fd0 tx=0x7f19f0017000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:08.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.181+0000 7f19e9ffb700 1 -- 172.21.15.29:0/1184610772 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) v1 ==== 161+0+0 (secure 0 0 0) 0x7f19ec0705b0 con 0x7f19f4157cf0 2024-01-24T15:32:08.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.184+0000 7f19fc03b700 1 -- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f19d80302f0 msgr2=0x7f19d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:08.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.184+0000 7f19fc03b700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f19d80302f0 0x7f19d8068bf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f19f0009fd0 tx=0x7f19f0017000).stop 2024-01-24T15:32:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.184+0000 7f19fc03b700 1 -- 172.21.15.29:0/1184610772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19f4157cf0 msgr2=0x7f19f41682a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.184+0000 7f19fc03b700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19f4157cf0 0x7f19f41682a0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7f19ec00c810 tx=0x7f19ec001500).stop 2024-01-24T15:32:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.184+0000 7f19fc03b700 1 -- 172.21.15.29:0/1184610772 shutdown_connections 2024-01-24T15:32:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.184+0000 7f19fc03b700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19f4157cf0 0x7f19f41682a0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.184+0000 7f19fc03b700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19f40a9d20 0x7f19f40c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.184+0000 7f19fc03b700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f19d80302f0 0x7f19d8068bf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.184+0000 7f19fc03b700 1 --2- 172.21.15.29:0/1184610772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19f4154240 0x7f19f4167db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.184+0000 7f19fc03b700 1 -- 172.21.15.29:0/1184610772 >> 172.21.15.29:0/1184610772 conn(0x7f19f40c8310 msgr2=0x7f19f4153750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.184+0000 7f19fc03b700 1 -- 172.21.15.29:0/1184610772 shutdown_connections 2024-01-24T15:32:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.184+0000 7f19fc03b700 1 -- 172.21.15.29:0/1184610772 wait complete. 2024-01-24T15:32:08.187 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-24T15:32:08.202 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-24T15:32:08.223 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T15:32:08.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.353+0000 7f386b03a700 1 Processor -- start 2024-01-24T15:32:08.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.354+0000 7f386b03a700 1 -- start start 2024-01-24T15:32:08.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.354+0000 7f386b03a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3864157530 0x7f3864068470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:08.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.354+0000 7f386b03a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38640be080 con 0x7f3864066c30 2024-01-24T15:32:08.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.354+0000 7f386b03a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3864067380 con 0x7f3864068960 2024-01-24T15:32:08.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.354+0000 7f386b03a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3864065040 con 0x7f3864157530 2024-01-24T15:32:08.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.355+0000 7f38695d7700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3864066c30 0x7f38640be510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58036/0 (socket says 172.21.15.29:58036) 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.355+0000 7f38695d7700 1 -- 172.21.15.29:0/3605865559 learned_addr learned my addr 172.21.15.29:0/3605865559 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.355+0000 7f3868dd6700 1 --2- 172.21.15.29:0/3605865559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3864157530 0x7f3864068470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.355+0000 7f38637fe700 1 -- 172.21.15.29:0/3605865559 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876836652 0 0) 0x7f38640be080 con 0x7f3864066c30 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.355+0000 7f38637fe700 1 -- 172.21.15.29:0/3605865559 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38540033f0 con 0x7f3864066c30 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.355+0000 7f38637fe700 1 -- 172.21.15.29:0/3605865559 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714840138 0 0) 0x7f3864067380 con 0x7f3864068960 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.355+0000 7f38637fe700 1 -- 172.21.15.29:0/3605865559 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38640be080 con 0x7f3864068960 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.355+0000 7f38637fe700 1 -- 172.21.15.29:0/3605865559 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 903977163 0 0) 0x7f38540033f0 con 0x7f3864066c30 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.355+0000 7f3868dd6700 1 -- 172.21.15.29:0/3605865559 >> v1:172.21.15.163:6789/0 conn(0x7f3864068960 legacy=0x7f3864066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.355+0000 7f3868dd6700 1 -- 172.21.15.29:0/3605865559 >> v1:172.21.15.29:6789/0 conn(0x7f3864066c30 legacy=0x7f38640be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.355+0000 7f3868dd6700 1 -- 172.21.15.29:0/3605865559 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38640638c0 con 0x7f3864157530 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.355+0000 7f3868dd6700 1 --2- 172.21.15.29:0/3605865559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3864157530 0x7f3864068470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f3858009790 tx=0x7f3858009060).ready entity=mon.2 client_cookie=647f3b66a82e4e75 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.356+0000 7f38637fe700 1 -- 172.21.15.29:0/3605865559 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c0017f0 con 0x7f3864157530 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.356+0000 7f38637fe700 1 -- 172.21.15.29:0/3605865559 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3858011400 con 0x7f3864157530 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.356+0000 7f38637fe700 1 -- 172.21.15.29:0/3605865559 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858011780 con 0x7f3864157530 2024-01-24T15:32:08.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.356+0000 7f386b03a700 1 -- 172.21.15.29:0/3605865559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3864157530 msgr2=0x7f3864068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:08.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.356+0000 7f386b03a700 1 --2- 172.21.15.29:0/3605865559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3864157530 0x7f3864068470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f3858009790 tx=0x7f3858009060).stop 2024-01-24T15:32:08.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.356+0000 7f386b03a700 1 -- 172.21.15.29:0/3605865559 shutdown_connections 2024-01-24T15:32:08.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.356+0000 7f386b03a700 1 --2- 172.21.15.29:0/3605865559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3864157530 0x7f3864068470 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:08.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.356+0000 7f386b03a700 1 -- 172.21.15.29:0/3605865559 >> 172.21.15.29:0/3605865559 conn(0x7f38640c8ba0 msgr2=0x7f38640c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:08.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.356+0000 7f386b03a700 1 -- 172.21.15.29:0/3605865559 shutdown_connections 2024-01-24T15:32:08.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.356+0000 7f386b03a700 1 -- 172.21.15.29:0/3605865559 wait complete. 2024-01-24T15:32:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.357+0000 7f386b03a700 1 Processor -- start 2024-01-24T15:32:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.358+0000 7f386b03a700 1 -- start start 2024-01-24T15:32:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.358+0000 7f386b03a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3864066c30 0x7f38640c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.358+0000 7f386b03a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3864068960 0x7f3864156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.358+0000 7f386b03a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3864157530 0x7f3864150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:08.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.358+0000 7f386b03a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38640ac480 con 0x7f3864157530 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.358+0000 7f386b03a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3864156a70 con 0x7f3864068960 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.358+0000 7f386b03a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38640abb50 con 0x7f3864066c30 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.358+0000 7f38695d7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3864157530 0x7f3864150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f38695d7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3864157530 0x7f3864150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34882/0 (socket says 172.21.15.29:34882) 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f38695d7700 1 -- 172.21.15.29:0/2613591402 learned_addr learned my addr 172.21.15.29:0/2613591402 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f3868dd6700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3864066c30 0x7f38640c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f3863fff700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3864068960 0x7f3864156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f38695d7700 1 -- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3864066c30 msgr2=0x7f38640c27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f38695d7700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3864066c30 0x7f38640c27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f38695d7700 1 -- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3864068960 msgr2=0x7f3864156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f38695d7700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3864068960 0x7f3864156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f38695d7700 1 -- 172.21.15.29:0/2613591402 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38580092a0 con 0x7f3864157530 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f38695d7700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3864157530 0x7f3864150820 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 rx=0x7f385c00cb60 tx=0x7f385c001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f3860ff9700 1 -- 172.21.15.29:0/2613591402 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c004c40 con 0x7f3864157530 2024-01-24T15:32:08.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f3860ff9700 1 -- 172.21.15.29:0/2613591402 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c004da0 con 0x7f3864157530 2024-01-24T15:32:08.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f386b03a700 1 -- 172.21.15.29:0/2613591402 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38640abe30 con 0x7f3864157530 2024-01-24T15:32:08.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f3860ff9700 1 -- 172.21.15.29:0/2613591402 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c0073a0 con 0x7f3864157530 2024-01-24T15:32:08.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.359+0000 7f386b03a700 1 -- 172.21.15.29:0/2613591402 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3864156780 con 0x7f3864157530 2024-01-24T15:32:08.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.360+0000 7f386b03a700 1 -- 172.21.15.29:0/2613591402 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38640ac630 con 0x7f3864157530 2024-01-24T15:32:08.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.361+0000 7f3860ff9700 1 -- 172.21.15.29:0/2613591402 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 97) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f385c007710 con 0x7f3864157530 2024-01-24T15:32:08.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.361+0000 7f3860ff9700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f384805f000 0x7f38480614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:08.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.361+0000 7f3860ff9700 1 -- 172.21.15.29:0/2613591402 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 7874+0+0 (secure 0 0 0) 0x7f385c07a400 con 0x7f3864157530 2024-01-24T15:32:08.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.361+0000 7f3868dd6700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f384805f000 0x7f38480614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:08.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.362+0000 7f3868dd6700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f384805f000 0x7f38480614c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f38580096e0 tx=0x7f3858016000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:08.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.365+0000 7f3860ff9700 1 -- 172.21.15.29:0/2613591402 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f38640ac630 con 0x7f3864157530 2024-01-24T15:32:08.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:08.592+0000 7f386b03a700 1 -- 172.21.15.29:0/2613591402 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f38640475e0 con 0x7f3864157530 2024-01-24T15:32:09.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.182+0000 7f3860ff9700 1 -- 172.21.15.29:0/2613591402 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) v1 ==== 104+0+0 (secure 0 0 0) 0x7f385c04b160 con 0x7f3864157530 2024-01-24T15:32:09.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.185+0000 7f386b03a700 1 -- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f384805f000 msgr2=0x7f38480614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:09.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.185+0000 7f386b03a700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f384805f000 0x7f38480614c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f38580096e0 tx=0x7f3858016000).stop 2024-01-24T15:32:09.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.185+0000 7f386b03a700 1 -- 172.21.15.29:0/2613591402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3864157530 msgr2=0x7f3864150820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:09.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.185+0000 7f386b03a700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3864157530 0x7f3864150820 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 rx=0x7f385c00cb60 tx=0x7f385c001e50).stop 2024-01-24T15:32:09.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.186+0000 7f386b03a700 1 -- 172.21.15.29:0/2613591402 shutdown_connections 2024-01-24T15:32:09.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.186+0000 7f386b03a700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3864157530 0x7f3864150820 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:09.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.186+0000 7f386b03a700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f384805f000 0x7f38480614c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:09.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.186+0000 7f386b03a700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3864066c30 0x7f38640c27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:09.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.186+0000 7f386b03a700 1 --2- 172.21.15.29:0/2613591402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3864068960 0x7f3864156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:09.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.186+0000 7f386b03a700 1 -- 172.21.15.29:0/2613591402 >> 172.21.15.29:0/2613591402 conn(0x7f38640c8ba0 msgr2=0x7f38640be510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:09.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.186+0000 7f386b03a700 1 -- 172.21.15.29:0/2613591402 shutdown_connections 2024-01-24T15:32:09.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.186+0000 7f386b03a700 1 -- 172.21.15.29:0/2613591402 wait complete. 2024-01-24T15:32:09.203 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-24T15:32:09.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.339+0000 7f1c11431700 1 Processor -- start 2024-01-24T15:32:09.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.339+0000 7f1c11431700 1 -- start start 2024-01-24T15:32:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.340+0000 7f1c11431700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1c0c151e50 0x7f1c0c0c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.340+0000 7f1c11431700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c0c0c7f70 0x7f1c0c155420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.340+0000 7f1c11431700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1c0c064780 0x7f1c0c064c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.340+0000 7f1c11431700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c0c155910 con 0x7f1c0c151e50 2024-01-24T15:32:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.340+0000 7f1c11431700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c0c154f90 con 0x7f1c0c064780 2024-01-24T15:32:09.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.340+0000 7f1c11431700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c065100 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.340+0000 7f1c0affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1c0c151e50 0x7f1c0c0c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.340+0000 7f1c0affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1c0c151e50 0x7f1c0c0c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34890/0 (socket says 172.21.15.29:34890) 2024-01-24T15:32:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.340+0000 7f1c0affd700 1 -- 172.21.15.29:0/3233991553 learned_addr learned my addr 172.21.15.29:0/3233991553 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.341+0000 7f1c0b7fe700 1 --2- 172.21.15.29:0/3233991553 >> v2:172.21.15.163:3300/0 conn(0x7f1c0c064780 0x7f1c0c064c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.341+0000 7f1c0a7fc700 1 --2- 172.21.15.29:0/3233991553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c0c0c7f70 0x7f1c0c155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.341+0000 7f1c0affd700 1 -- 172.21.15.29:0/3233991553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c0c0c7f70 msgr2=0x7f1c0c155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.341+0000 7f1c0affd700 1 --2- 172.21.15.29:0/3233991553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c0c0c7f70 0x7f1c0c155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.341+0000 7f1c0affd700 1 -- 172.21.15.29:0/3233991553 >> v2:172.21.15.163:3300/0 conn(0x7f1c0c064780 msgr2=0x7f1c0c064c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:09.346 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.341+0000 7f1c0affd700 1 --2- 172.21.15.29:0/3233991553 >> v2:172.21.15.163:3300/0 conn(0x7f1c0c064780 0x7f1c0c064c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:09.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.341+0000 7f1c0affd700 1 -- 172.21.15.29:0/3233991553 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c0a9f10 con 0x7f1c0c151e50 2024-01-24T15:32:09.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.341+0000 7f1c0b7fe700 1 --2- 172.21.15.29:0/3233991553 >> v2:172.21.15.163:3300/0 conn(0x7f1c0c064780 0x7f1c0c064c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:09.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.341+0000 7f1c0affd700 1 --2- 172.21.15.29:0/3233991553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0c151e50 0x7f1c0c0c53c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f1c0c083240 tx=0x7f1bfc0090f0).ready entity=mon.0 client_cookie=aaf2bede21d982b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:09.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.341+0000 7f1c09ffb700 1 -- 172.21.15.29:0/3233991553 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00b600 con 0x7f1c0c151e50 2024-01-24T15:32:09.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.341+0000 7f1c09ffb700 1 -- 172.21.15.29:0/3233991553 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc00b760 con 0x7f1c0c151e50 2024-01-24T15:32:09.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.341+0000 7f1c09ffb700 1 -- 172.21.15.29:0/3233991553 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00bae0 con 0x7f1c0c151e50 2024-01-24T15:32:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.342+0000 7f1c11431700 1 -- 172.21.15.29:0/3233991553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0c151e50 msgr2=0x7f1c0c0c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.342+0000 7f1c11431700 1 --2- 172.21.15.29:0/3233991553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0c151e50 0x7f1c0c0c53c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f1c0c083240 tx=0x7f1bfc0090f0).stop 2024-01-24T15:32:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.342+0000 7f1c11431700 1 -- 172.21.15.29:0/3233991553 shutdown_connections 2024-01-24T15:32:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.342+0000 7f1c11431700 1 --2- 172.21.15.29:0/3233991553 >> v2:172.21.15.163:3300/0 conn(0x7f1c0c064780 0x7f1c0c064c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.342+0000 7f1c11431700 1 --2- 172.21.15.29:0/3233991553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0c151e50 0x7f1c0c0c53c0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.342+0000 7f1c11431700 1 --2- 172.21.15.29:0/3233991553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c0c0c7f70 0x7f1c0c155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.342+0000 7f1c11431700 1 -- 172.21.15.29:0/3233991553 >> 172.21.15.29:0/3233991553 conn(0x7f1c0c0cb880 msgr2=0x7f1c0c0c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.342+0000 7f1c11431700 1 -- 172.21.15.29:0/3233991553 shutdown_connections 2024-01-24T15:32:09.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.342+0000 7f1c11431700 1 -- 172.21.15.29:0/3233991553 wait complete. 2024-01-24T15:32:09.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.343+0000 7f1c11431700 1 Processor -- start 2024-01-24T15:32:09.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.343+0000 7f1c11431700 1 -- start start 2024-01-24T15:32:09.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c11431700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c0c064780 0x7f1c0c14d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:09.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c11431700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0c0c7f70 0x7f1c0c14f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:09.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c11431700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c0c151e50 0x7f1c0c0afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:09.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c11431700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c0aed40 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c11431700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c0af020 con 0x7f1c0c064780 2024-01-24T15:32:09.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c11431700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c0af300 con 0x7f1c0c151e50 2024-01-24T15:32:09.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c0a7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0c0c7f70 0x7f1c0c14f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:09.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c0a7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0c0c7f70 0x7f1c0c14f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34898/0 (socket says 172.21.15.29:34898) 2024-01-24T15:32:09.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c0a7fc700 1 -- 172.21.15.29:0/148009588 learned_addr learned my addr 172.21.15.29:0/148009588 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:09.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c0affd700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c0c064780 0x7f1c0c14d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:09.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c0b7fe700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c0c151e50 0x7f1c0c0afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:09.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c0a7fc700 1 -- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c0c151e50 msgr2=0x7f1c0c0afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:09.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c0a7fc700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c0c151e50 0x7f1c0c0afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:09.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c0a7fc700 1 -- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c0c064780 msgr2=0x7f1c0c14d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:09.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c0a7fc700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c0c064780 0x7f1c0c14d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:09.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.344+0000 7f1c0a7fc700 1 -- 172.21.15.29:0/148009588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc009370 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.345+0000 7f1c0a7fc700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0c0c7f70 0x7f1c0c14f8e0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 rx=0x7f1c0000d890 tx=0x7f1c00008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:09.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.345+0000 7f1bf2ffd700 1 -- 172.21.15.29:0/148009588 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00011590 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.345+0000 7f1bf2ffd700 1 -- 172.21.15.29:0/148009588 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c000116f0 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.345+0000 7f1c11431700 1 -- 172.21.15.29:0/148009588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0c0af5e0 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.345+0000 7f1bf2ffd700 1 -- 172.21.15.29:0/148009588 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00011a70 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.345+0000 7f1c11431700 1 -- 172.21.15.29:0/148009588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0c14dee0 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.346+0000 7f1c11431700 1 -- 172.21.15.29:0/148009588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0c0aa710 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.346+0000 7f1bf2ffd700 1 -- 172.21.15.29:0/148009588 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 97) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f1c00012420 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.347+0000 7f1bf2ffd700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f1bf805f030 0x7f1bf80614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:09.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.347+0000 7f1bf2ffd700 1 -- 172.21.15.29:0/148009588 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7874+0+0 (secure 0 0 0) 0x7f1c0007a040 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.347+0000 7f1c0affd700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f1bf805f030 0x7f1bf80614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:09.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.348+0000 7f1c0affd700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f1bf805f030 0x7f1bf80614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f1bfc009130 tx=0x7f1bfc017000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:09.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.351+0000 7f1bf2ffd700 1 -- 172.21.15.29:0/148009588 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f1c0004ada0 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.535+0000 7f1c11431700 1 -- 172.21.15.29:0/148009588 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1c0c0afb00 con 0x7f1c0c0c7f70 2024-01-24T15:32:09.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:09.922+0000 7f1bf2ffd700 1 -- 172.21.15.29:0/148009588 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1c00077900 con 0x7f1c0c0c7f70 2024-01-24T15:32:10.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.190+0000 7f1bf2ffd700 1 -- 172.21.15.29:0/148009588 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v221) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1c000726c0 con 0x7f1c0c0c7f70 2024-01-24T15:32:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.194+0000 7f1c11431700 1 -- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f1bf805f030 msgr2=0x7f1bf80614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.194+0000 7f1c11431700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f1bf805f030 0x7f1bf80614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f1bfc009130 tx=0x7f1bfc017000).stop 2024-01-24T15:32:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.194+0000 7f1c11431700 1 -- 172.21.15.29:0/148009588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0c0c7f70 msgr2=0x7f1c0c14f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.194+0000 7f1c11431700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0c0c7f70 0x7f1c0c14f8e0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 rx=0x7f1c0000d890 tx=0x7f1c00008040).stop 2024-01-24T15:32:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.194+0000 7f1c11431700 1 -- 172.21.15.29:0/148009588 shutdown_connections 2024-01-24T15:32:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.194+0000 7f1c11431700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f1bf805f030 0x7f1bf80614f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.194+0000 7f1c11431700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c0c151e50 0x7f1c0c0afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.194+0000 7f1c11431700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c0c064780 0x7f1c0c14d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.194+0000 7f1c11431700 1 --2- 172.21.15.29:0/148009588 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c0c0c7f70 0x7f1c0c14f8e0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.194+0000 7f1c11431700 1 -- 172.21.15.29:0/148009588 >> 172.21.15.29:0/148009588 conn(0x7f1c0c0cb880 msgr2=0x7f1c0c0ca600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:10.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.195+0000 7f1c11431700 1 -- 172.21.15.29:0/148009588 shutdown_connections 2024-01-24T15:32:10.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.195+0000 7f1c11431700 1 -- 172.21.15.29:0/148009588 wait complete. 2024-01-24T15:32:10.199 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-01-24T15:32:10.214 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-24T15:32:10.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.350+0000 7fbb065c9700 1 Processor -- start 2024-01-24T15:32:10.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.350+0000 7fbb065c9700 1 -- start start 2024-01-24T15:32:10.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.351+0000 7fbb065c9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb000aa810 0x7fbb001546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:10.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.351+0000 7fbb065c9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb0005e1f0 0x7fbb000c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:10.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.351+0000 7fbb065c9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb0014b640 con 0x7fbb000aabf0 2024-01-24T15:32:10.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.351+0000 7fbb065c9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb00157fa0 con 0x7fbb0005e1f0 2024-01-24T15:32:10.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.351+0000 7fbb065c9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb00158b40 con 0x7fbb000aa810 2024-01-24T15:32:10.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.352+0000 7fbb04b66700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb0005e1f0 0x7fbb000c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:10.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.352+0000 7fbafffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb000aa810 0x7fbb001546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:10.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.352+0000 7fbb04b66700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb0005e1f0 0x7fbb000c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:34876/0 (socket says 172.21.15.29:34876) 2024-01-24T15:32:10.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.352+0000 7fbafffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb000aa810 0x7fbb001546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:37850/0 (socket says 172.21.15.29:37850) 2024-01-24T15:32:10.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.352+0000 7fbb04b66700 1 -- 172.21.15.29:0/4156429711 learned_addr learned my addr 172.21.15.29:0/4156429711 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:10.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.352+0000 7fbafffff700 1 -- 172.21.15.29:0/4156429711 >> v2:172.21.15.163:3300/0 conn(0x7fbb0005e1f0 msgr2=0x7fbb000c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:10.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.352+0000 7fbafffff700 1 --2- 172.21.15.29:0/4156429711 >> v2:172.21.15.163:3300/0 conn(0x7fbb0005e1f0 0x7fbb000c3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:10.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.352+0000 7fbafffff700 1 -- 172.21.15.29:0/4156429711 >> v1:172.21.15.163:6789/0 conn(0x7fbb000aabf0 legacy=0x7fbb00158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:10.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.352+0000 7fbafffff700 1 -- 172.21.15.29:0/4156429711 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb00154f20 con 0x7fbb000aa810 2024-01-24T15:32:10.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.352+0000 7fbb04b66700 1 --2- 172.21.15.29:0/4156429711 >> v2:172.21.15.163:3300/0 conn(0x7fbb0005e1f0 0x7fbb000c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:10.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.352+0000 7fbafffff700 1 --2- 172.21.15.29:0/4156429711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb000aa810 0x7fbb001546a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fbaf0009760 tx=0x7fbaf0009060).ready entity=mon.2 client_cookie=bebe3b117b39d867 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:10.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.353+0000 7fbafeffd700 1 -- 172.21.15.29:0/4156429711 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf000f510 con 0x7fbb000aa810 2024-01-24T15:32:10.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.353+0000 7fbafeffd700 1 -- 172.21.15.29:0/4156429711 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf000f670 con 0x7fbb000aa810 2024-01-24T15:32:10.356 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.353+0000 7fbafeffd700 1 -- 172.21.15.29:0/4156429711 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf000f9f0 con 0x7fbb000aa810 2024-01-24T15:32:10.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.353+0000 7fbb065c9700 1 -- 172.21.15.29:0/4156429711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb000aa810 msgr2=0x7fbb001546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.353+0000 7fbb065c9700 1 --2- 172.21.15.29:0/4156429711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb000aa810 0x7fbb001546a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fbaf0009760 tx=0x7fbaf0009060).stop 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.353+0000 7fbb065c9700 1 -- 172.21.15.29:0/4156429711 shutdown_connections 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.353+0000 7fbb065c9700 1 --2- 172.21.15.29:0/4156429711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb000aa810 0x7fbb001546a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.353+0000 7fbb065c9700 1 --2- 172.21.15.29:0/4156429711 >> v2:172.21.15.163:3300/0 conn(0x7fbb0005e1f0 0x7fbb000c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.353+0000 7fbb065c9700 1 -- 172.21.15.29:0/4156429711 >> 172.21.15.29:0/4156429711 conn(0x7fbb000c7280 msgr2=0x7fbb000c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.353+0000 7fbb065c9700 1 -- 172.21.15.29:0/4156429711 shutdown_connections 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.353+0000 7fbb065c9700 1 -- 172.21.15.29:0/4156429711 wait complete. 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.354+0000 7fbb065c9700 1 Processor -- start 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.355+0000 7fbb065c9700 1 -- start start 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.355+0000 7fbb065c9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb0005e1f0 0x7fbb00164ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.355+0000 7fbb065c9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb000aa810 0x7fbb000c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.355+0000 7fbb065c9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb000aabf0 0x7fbb000c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:10.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.355+0000 7fbb065c9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0014c750 con 0x7fbb000aa810 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.355+0000 7fbb065c9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb00165390 con 0x7fbb000aabf0 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.355+0000 7fbb065c9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0014bb40 con 0x7fbb0005e1f0 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbaff7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb000aa810 0x7fbb000c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbaff7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb000aa810 0x7fbb000c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34904/0 (socket says 172.21.15.29:34904) 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbaff7fe700 1 -- 172.21.15.29:0/1346652669 learned_addr learned my addr 172.21.15.29:0/1346652669 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbafffff700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb0005e1f0 0x7fbb00164ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbb04b66700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb000aabf0 0x7fbb000c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbaff7fe700 1 -- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb0005e1f0 msgr2=0x7fbb00164ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbaff7fe700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb0005e1f0 0x7fbb00164ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbaff7fe700 1 -- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb000aabf0 msgr2=0x7fbb000c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbaff7fe700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb000aabf0 0x7fbb000c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbaff7fe700 1 -- 172.21.15.29:0/1346652669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf8009ae0 con 0x7fbb000aa810 2024-01-24T15:32:10.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbaff7fe700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb000aa810 0x7fbb000c1c10 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 rx=0x7fbaf400c440 tx=0x7fbaf4007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:10.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbae7fff700 1 -- 172.21.15.29:0/1346652669 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4007020 con 0x7fbb000aa810 2024-01-24T15:32:10.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbae7fff700 1 -- 172.21.15.29:0/1346652669 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf4001e90 con 0x7fbb000aa810 2024-01-24T15:32:10.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbae7fff700 1 -- 172.21.15.29:0/1346652669 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf40073b0 con 0x7fbb000aa810 2024-01-24T15:32:10.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbb065c9700 1 -- 172.21.15.29:0/1346652669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf00092a0 con 0x7fbb000aa810 2024-01-24T15:32:10.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.356+0000 7fbb065c9700 1 -- 172.21.15.29:0/1346652669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0014c100 con 0x7fbb000aa810 2024-01-24T15:32:10.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.357+0000 7fbb065c9700 1 -- 172.21.15.29:0/1346652669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb000475e0 con 0x7fbb000aa810 2024-01-24T15:32:10.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.358+0000 7fbae7fff700 1 -- 172.21.15.29:0/1346652669 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbaf40068b0 con 0x7fbb000aa810 2024-01-24T15:32:10.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.358+0000 7fbae7fff700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbaec05f270 0x7fbaec061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:10.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.358+0000 7fbae7fff700 1 -- 172.21.15.29:0/1346652669 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7436+0+0 (secure 0 0 0) 0x7fbaf407a0a0 con 0x7fbb000aa810 2024-01-24T15:32:10.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.358+0000 7fbafffff700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbaec05f270 0x7fbaec061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:10.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.359+0000 7fbafffff700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbaec05f270 0x7fbaec061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fbaf0009fd0 tx=0x7fbaf0017000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:10.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.363+0000 7fbae7fff700 1 -- 172.21.15.29:0/1346652669 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbaf404acd0 con 0x7fbb000aa810 2024-01-24T15:32:10.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:10.543+0000 7fbb065c9700 1 -- 172.21.15.29:0/1346652669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbb000c4aa0 con 0x7fbb000aa810 2024-01-24T15:32:11.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.194+0000 7fbae7fff700 1 -- 172.21.15.29:0/1346652669 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v222) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbaf404e2f0 con 0x7fbb000aa810 2024-01-24T15:32:11.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.199+0000 7fbb065c9700 1 -- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbaec05f270 msgr2=0x7fbaec061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.199+0000 7fbb065c9700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbaec05f270 0x7fbaec061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fbaf0009fd0 tx=0x7fbaf0017000).stop 2024-01-24T15:32:11.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.199+0000 7fbb065c9700 1 -- 172.21.15.29:0/1346652669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb000aa810 msgr2=0x7fbb000c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.199+0000 7fbb065c9700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb000aa810 0x7fbb000c1c10 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 rx=0x7fbaf400c440 tx=0x7fbaf4007d70).stop 2024-01-24T15:32:11.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.199+0000 7fbb065c9700 1 -- 172.21.15.29:0/1346652669 shutdown_connections 2024-01-24T15:32:11.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.199+0000 7fbb065c9700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb000aabf0 0x7fbb000c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.199+0000 7fbb065c9700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbaec05f270 0x7fbaec061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.199+0000 7fbb065c9700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb0005e1f0 0x7fbb00164ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.199+0000 7fbb065c9700 1 --2- 172.21.15.29:0/1346652669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb000aa810 0x7fbb000c1c10 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.199+0000 7fbb065c9700 1 -- 172.21.15.29:0/1346652669 >> 172.21.15.29:0/1346652669 conn(0x7fbb000c7280 msgr2=0x7fbb00148de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:11.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.200+0000 7fbb065c9700 1 -- 172.21.15.29:0/1346652669 shutdown_connections 2024-01-24T15:32:11.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.200+0000 7fbb065c9700 1 -- 172.21.15.29:0/1346652669 wait complete. 2024-01-24T15:32:11.204 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-01-24T15:32:11.220 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-24T15:32:11.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.353+0000 7fbf45e16700 1 Processor -- start 2024-01-24T15:32:11.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.353+0000 7fbf45e16700 1 -- start start 2024-01-24T15:32:11.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf45e16700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf40066520 0x7fbf40066960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:11.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf45e16700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbf400be510 0x7fbf400be960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:11.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf45e16700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf400be080 con 0x7fbf40157530 2024-01-24T15:32:11.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf45e16700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf40065040 con 0x7fbf40066520 2024-01-24T15:32:11.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf45e16700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf40064140 con 0x7fbf400be510 2024-01-24T15:32:11.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf3effd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf40066520 0x7fbf40066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:11.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf3effd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf40066520 0x7fbf40066960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34908/0 (socket says 172.21.15.29:34908) 2024-01-24T15:32:11.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf3effd700 1 -- 172.21.15.29:0/112327525 learned_addr learned my addr 172.21.15.29:0/112327525 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:11.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf3ffff700 1 --2- 172.21.15.29:0/112327525 >> v2:172.21.15.163:3300/0 conn(0x7fbf400be510 0x7fbf400be960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:11.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf3effd700 1 -- 172.21.15.29:0/112327525 >> v2:172.21.15.163:3300/0 conn(0x7fbf400be510 msgr2=0x7fbf400be960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:11.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf3effd700 1 --2- 172.21.15.29:0/112327525 >> v2:172.21.15.163:3300/0 conn(0x7fbf400be510 0x7fbf400be960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf3effd700 1 -- 172.21.15.29:0/112327525 >> v1:172.21.15.163:6789/0 conn(0x7fbf40157530 legacy=0x7fbf40068470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:11.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf3effd700 1 -- 172.21.15.29:0/112327525 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf40067380 con 0x7fbf40066520 2024-01-24T15:32:11.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.354+0000 7fbf3effd700 1 --2- 172.21.15.29:0/112327525 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf40066520 0x7fbf40066960 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7fbf400c9870 tx=0x7fbf340090f0).ready entity=mon.0 client_cookie=e532d8db45ae3bf server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:11.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.355+0000 7fbf3e7fc700 1 -- 172.21.15.29:0/112327525 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3400b5a0 con 0x7fbf40066520 2024-01-24T15:32:11.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.355+0000 7fbf3e7fc700 1 -- 172.21.15.29:0/112327525 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3400b700 con 0x7fbf40066520 2024-01-24T15:32:11.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.355+0000 7fbf3e7fc700 1 -- 172.21.15.29:0/112327525 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3400ba80 con 0x7fbf40066520 2024-01-24T15:32:11.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.355+0000 7fbf45e16700 1 -- 172.21.15.29:0/112327525 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf40066520 msgr2=0x7fbf40066960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.355+0000 7fbf45e16700 1 --2- 172.21.15.29:0/112327525 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf40066520 0x7fbf40066960 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7fbf400c9870 tx=0x7fbf340090f0).stop 2024-01-24T15:32:11.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.355+0000 7fbf45e16700 1 -- 172.21.15.29:0/112327525 shutdown_connections 2024-01-24T15:32:11.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.355+0000 7fbf45e16700 1 --2- 172.21.15.29:0/112327525 >> v2:172.21.15.163:3300/0 conn(0x7fbf400be510 0x7fbf400be960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.355+0000 7fbf45e16700 1 --2- 172.21.15.29:0/112327525 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf40066520 0x7fbf40066960 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.355+0000 7fbf45e16700 1 -- 172.21.15.29:0/112327525 >> 172.21.15.29:0/112327525 conn(0x7fbf400c8ba0 msgr2=0x7fbf400c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:11.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.355+0000 7fbf45e16700 1 -- 172.21.15.29:0/112327525 shutdown_connections 2024-01-24T15:32:11.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.355+0000 7fbf45e16700 1 -- 172.21.15.29:0/112327525 wait complete. 2024-01-24T15:32:11.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.357+0000 7fbf45e16700 1 Processor -- start 2024-01-24T15:32:11.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.357+0000 7fbf45e16700 1 -- start start 2024-01-24T15:32:11.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.358+0000 7fbf45e16700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf40066520 0x7fbf400bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:11.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.358+0000 7fbf45e16700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf400be510 0x7fbf400c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:11.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.358+0000 7fbf45e16700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf40157530 0x7fbf400c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:11.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.358+0000 7fbf45e16700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf400c0700 con 0x7fbf40157530 2024-01-24T15:32:11.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.358+0000 7fbf45e16700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf400c09e0 con 0x7fbf400be510 2024-01-24T15:32:11.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.358+0000 7fbf45e16700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf400c0cc0 con 0x7fbf40066520 2024-01-24T15:32:11.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.358+0000 7fbf3ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf40157530 0x7fbf400c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:11.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.358+0000 7fbf3ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf40157530 0x7fbf400c17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34918/0 (socket says 172.21.15.29:34918) 2024-01-24T15:32:11.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf3ffff700 1 -- 172.21.15.29:0/446214902 learned_addr learned my addr 172.21.15.29:0/446214902 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:11.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf3effd700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf400be510 0x7fbf400c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:11.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf3f7fe700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf40066520 0x7fbf400bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:11.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf3ffff700 1 -- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf40066520 msgr2=0x7fbf400bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf3ffff700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf40066520 0x7fbf400bfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf3ffff700 1 -- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf400be510 msgr2=0x7fbf400c0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf3ffff700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf400be510 0x7fbf400c0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf3ffff700 1 -- 172.21.15.29:0/446214902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf34009370 con 0x7fbf40157530 2024-01-24T15:32:11.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf3ffff700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf40157530 0x7fbf400c17a0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 rx=0x7fbf38009fa0 tx=0x7fbf38009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:11.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf277fe700 1 -- 172.21.15.29:0/446214902 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800ebc0 con 0x7fbf40157530 2024-01-24T15:32:11.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf277fe700 1 -- 172.21.15.29:0/446214902 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3800ed20 con 0x7fbf40157530 2024-01-24T15:32:11.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf45e16700 1 -- 172.21.15.29:0/446214902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf400c0fa0 con 0x7fbf40157530 2024-01-24T15:32:11.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf277fe700 1 -- 172.21.15.29:0/446214902 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38014610 con 0x7fbf40157530 2024-01-24T15:32:11.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.359+0000 7fbf45e16700 1 -- 172.21.15.29:0/446214902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf400a7430 con 0x7fbf40157530 2024-01-24T15:32:11.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.360+0000 7fbf45e16700 1 -- 172.21.15.29:0/446214902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf40066030 con 0x7fbf40157530 2024-01-24T15:32:11.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.361+0000 7fbf277fe700 1 -- 172.21.15.29:0/446214902 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbf3801d3b0 con 0x7fbf40157530 2024-01-24T15:32:11.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.361+0000 7fbf277fe700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf2c05f270 0x7fbf2c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:11.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.361+0000 7fbf277fe700 1 -- 172.21.15.29:0/446214902 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6964+0+0 (secure 0 0 0) 0x7fbf3807aba0 con 0x7fbf40157530 2024-01-24T15:32:11.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.361+0000 7fbf3f7fe700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf2c05f270 0x7fbf2c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:11.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.362+0000 7fbf3f7fe700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf2c05f270 0x7fbf2c061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fbf3000b9b0 tx=0x7fbf30008040).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:11.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.366+0000 7fbf277fe700 1 -- 172.21.15.29:0/446214902 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbf3804b940 con 0x7fbf40157530 2024-01-24T15:32:11.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.547+0000 7fbf45e16700 1 -- 172.21.15.29:0/446214902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbf400475e0 con 0x7fbf40157530 2024-01-24T15:32:11.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.551+0000 7fbf277fe700 1 -- 172.21.15.29:0/446214902 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbf3804ef60 con 0x7fbf40157530 2024-01-24T15:32:11.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.553+0000 7fbf45e16700 1 -- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf2c05f270 msgr2=0x7fbf2c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.553+0000 7fbf45e16700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf2c05f270 0x7fbf2c061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fbf3000b9b0 tx=0x7fbf30008040).stop 2024-01-24T15:32:11.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.553+0000 7fbf45e16700 1 -- 172.21.15.29:0/446214902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf40157530 msgr2=0x7fbf400c17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.553+0000 7fbf45e16700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf40157530 0x7fbf400c17a0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 rx=0x7fbf38009fa0 tx=0x7fbf38009900).stop 2024-01-24T15:32:11.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.553+0000 7fbf45e16700 1 -- 172.21.15.29:0/446214902 shutdown_connections 2024-01-24T15:32:11.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.553+0000 7fbf45e16700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf40157530 0x7fbf400c17a0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.553+0000 7fbf45e16700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbf2c05f270 0x7fbf2c061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.553+0000 7fbf45e16700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf40066520 0x7fbf400bfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.553+0000 7fbf45e16700 1 --2- 172.21.15.29:0/446214902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf400be510 0x7fbf400c0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.553+0000 7fbf45e16700 1 -- 172.21.15.29:0/446214902 >> 172.21.15.29:0/446214902 conn(0x7fbf400c8ba0 msgr2=0x7fbf400c7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:11.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.553+0000 7fbf45e16700 1 -- 172.21.15.29:0/446214902 shutdown_connections 2024-01-24T15:32:11.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.553+0000 7fbf45e16700 1 -- 172.21.15.29:0/446214902 wait complete. 2024-01-24T15:32:11.569 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-01-24T15:32:11.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.699+0000 7fd013569700 1 Processor -- start 2024-01-24T15:32:11.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.700+0000 7fd013569700 1 -- start start 2024-01-24T15:32:11.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd013569700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd00c0663b0 0x7fd00c152f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:11.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd013569700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd00c0be4b0 0x7fd00c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:11.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd013569700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c069690 con 0x7fd00c06a010 2024-01-24T15:32:11.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd013569700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00c0be930 con 0x7fd00c0be4b0 2024-01-24T15:32:11.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd013569700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0bca30 con 0x7fd00c0663b0 2024-01-24T15:32:11.703 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd010b04700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd00c0be4b0 0x7fd00c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:11.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd010b04700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd00c0be4b0 0x7fd00c069b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:34918/0 (socket says 172.21.15.29:34918) 2024-01-24T15:32:11.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd011305700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd00c0663b0 0x7fd00c152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:11.704 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd010b04700 1 -- 172.21.15.29:0/2163016387 learned_addr learned my addr 172.21.15.29:0/2163016387 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:11.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd010b04700 1 -- 172.21.15.29:0/2163016387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd00c0663b0 msgr2=0x7fd00c152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:11.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd010b04700 1 --2- 172.21.15.29:0/2163016387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd00c0663b0 0x7fd00c152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd010b04700 1 -- 172.21.15.29:0/2163016387 >> v1:172.21.15.163:6789/0 conn(0x7fd00c06a010 legacy=0x7fd00c065420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.701+0000 7fd010b04700 1 -- 172.21.15.29:0/2163016387 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c0bcc40 con 0x7fd00c0be4b0 2024-01-24T15:32:11.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.702+0000 7fd011305700 1 --2- 172.21.15.29:0/2163016387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd00c0663b0 0x7fd00c152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:11.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.702+0000 7fd010b04700 1 --2- 172.21.15.29:0/2163016387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd00c0be4b0 0x7fd00c069b20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7fcffc006fd0 tx=0x7fcffc0090f0).ready entity=mon.1 client_cookie=57fe8cd231a5cd35 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:11.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.702+0000 7fd003fff700 1 -- 172.21.15.29:0/2163016387 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc00b580 con 0x7fd00c0be4b0 2024-01-24T15:32:11.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.702+0000 7fd003fff700 1 -- 172.21.15.29:0/2163016387 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc00b6e0 con 0x7fd00c0be4b0 2024-01-24T15:32:11.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.702+0000 7fd003fff700 1 -- 172.21.15.29:0/2163016387 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc00ba80 con 0x7fd00c0be4b0 2024-01-24T15:32:11.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.702+0000 7fd013569700 1 -- 172.21.15.29:0/2163016387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd00c0be4b0 msgr2=0x7fd00c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.702+0000 7fd013569700 1 --2- 172.21.15.29:0/2163016387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd00c0be4b0 0x7fd00c069b20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7fcffc006fd0 tx=0x7fcffc0090f0).stop 2024-01-24T15:32:11.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.702+0000 7fd013569700 1 -- 172.21.15.29:0/2163016387 shutdown_connections 2024-01-24T15:32:11.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.702+0000 7fd013569700 1 --2- 172.21.15.29:0/2163016387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd00c0663b0 0x7fd00c152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.702+0000 7fd013569700 1 --2- 172.21.15.29:0/2163016387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd00c0be4b0 0x7fd00c069b20 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.703+0000 7fd013569700 1 -- 172.21.15.29:0/2163016387 >> 172.21.15.29:0/2163016387 conn(0x7fd00c064390 msgr2=0x7fd00c0c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:11.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.703+0000 7fd013569700 1 -- 172.21.15.29:0/2163016387 shutdown_connections 2024-01-24T15:32:11.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.703+0000 7fd013569700 1 -- 172.21.15.29:0/2163016387 wait complete. 2024-01-24T15:32:11.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.704+0000 7fd013569700 1 Processor -- start 2024-01-24T15:32:11.707 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.704+0000 7fd013569700 1 -- start start 2024-01-24T15:32:11.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd013569700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd00c0663b0 0x7fd00c136c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:11.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd013569700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd00c06a010 0x7fd00c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:11.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd013569700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd00c0be4b0 0x7fd00c0bfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd013569700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0c0b80 con 0x7fd00c0663b0 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd013569700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c13b900 con 0x7fd00c0be4b0 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd013569700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0abfc0 con 0x7fd00c06a010 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd011305700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd00c0663b0 0x7fd00c136c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd011305700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd00c0663b0 0x7fd00c136c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34934/0 (socket says 172.21.15.29:34934) 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd011305700 1 -- 172.21.15.29:0/1399234039 learned_addr learned my addr 172.21.15.29:0/1399234039 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd010b04700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd00c06a010 0x7fd00c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd011b06700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd00c0be4b0 0x7fd00c0bfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd011305700 1 -- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd00c06a010 msgr2=0x7fd00c138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd011305700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd00c06a010 0x7fd00c138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd011305700 1 -- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd00c0be4b0 msgr2=0x7fd00c0bfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.705+0000 7fd011305700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd00c0be4b0 0x7fd00c0bfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.706+0000 7fd011305700 1 -- 172.21.15.29:0/1399234039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0080092a0 con 0x7fd00c0663b0 2024-01-24T15:32:11.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.706+0000 7fd011305700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd00c0663b0 0x7fd00c136c10 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 rx=0x7fd00800ea20 tx=0x7fd008009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:11.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.706+0000 7fd0017fa700 1 -- 172.21.15.29:0/1399234039 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00800c850 con 0x7fd00c0663b0 2024-01-24T15:32:11.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.706+0000 7fd0017fa700 1 -- 172.21.15.29:0/1399234039 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00800c9b0 con 0x7fd00c0663b0 2024-01-24T15:32:11.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.706+0000 7fd013569700 1 -- 172.21.15.29:0/1399234039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc009370 con 0x7fd00c0663b0 2024-01-24T15:32:11.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.706+0000 7fd0017fa700 1 -- 172.21.15.29:0/1399234039 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00800cd30 con 0x7fd00c0663b0 2024-01-24T15:32:11.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.706+0000 7fd013569700 1 -- 172.21.15.29:0/1399234039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c0c08a0 con 0x7fd00c0663b0 2024-01-24T15:32:11.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.707+0000 7fd013569700 1 -- 172.21.15.29:0/1399234039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00c0475e0 con 0x7fd00c0663b0 2024-01-24T15:32:11.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.707+0000 7fd0017fa700 1 -- 172.21.15.29:0/1399234039 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd008012600 con 0x7fd00c0663b0 2024-01-24T15:32:11.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.708+0000 7fd0017fa700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcff005f270 0x7fcff0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:11.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.708+0000 7fd0017fa700 1 -- 172.21.15.29:0/1399234039 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6964+0+0 (secure 0 0 0) 0x7fd00807ad60 con 0x7fd00c0663b0 2024-01-24T15:32:11.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.708+0000 7fd010b04700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcff005f270 0x7fcff0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:11.713 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.709+0000 7fd010b04700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcff005f270 0x7fcff0061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fcffc006fd0 tx=0x7fcffc017000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:11.714 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.712+0000 7fd0017fa700 1 -- 172.21.15.29:0/1399234039 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd00804bb60 con 0x7fd00c0663b0 2024-01-24T15:32:11.896 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.894+0000 7fd013569700 1 -- 172.21.15.29:0/1399234039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fd00c0bcf20 con 0x7fd00c0663b0 2024-01-24T15:32:11.898 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.897+0000 7fd0017fa700 1 -- 172.21.15.29:0/1399234039 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v41) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd00804f180 con 0x7fd00c0663b0 2024-01-24T15:32:11.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.898+0000 7fd013569700 1 -- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcff005f270 msgr2=0x7fcff0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.898+0000 7fd013569700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcff005f270 0x7fcff0061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fcffc006fd0 tx=0x7fcffc017000).stop 2024-01-24T15:32:11.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.898+0000 7fd013569700 1 -- 172.21.15.29:0/1399234039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd00c0663b0 msgr2=0x7fd00c136c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:11.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.898+0000 7fd013569700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd00c0663b0 0x7fd00c136c10 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 rx=0x7fd00800ea20 tx=0x7fd008009060).stop 2024-01-24T15:32:11.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.899+0000 7fd013569700 1 -- 172.21.15.29:0/1399234039 shutdown_connections 2024-01-24T15:32:11.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.899+0000 7fd013569700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd00c0be4b0 0x7fd00c0bfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.899+0000 7fd013569700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd00c0663b0 0x7fd00c136c10 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.901 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.899+0000 7fd013569700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcff005f270 0x7fcff0061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.899+0000 7fd013569700 1 --2- 172.21.15.29:0/1399234039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd00c06a010 0x7fd00c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:11.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.899+0000 7fd013569700 1 -- 172.21.15.29:0/1399234039 >> 172.21.15.29:0/1399234039 conn(0x7fd00c064390 msgr2=0x7fd00c138180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:11.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.899+0000 7fd013569700 1 -- 172.21.15.29:0/1399234039 shutdown_connections 2024-01-24T15:32:11.902 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:11.899+0000 7fd013569700 1 -- 172.21.15.29:0/1399234039 wait complete. 2024-01-24T15:32:11.914 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2024-01-24T15:32:12.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.042+0000 7fedb96ba700 1 Processor -- start 2024-01-24T15:32:12.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.042+0000 7fedb96ba700 1 -- start start 2024-01-24T15:32:12.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.043+0000 7fedb96ba700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb4150cb0 0x7fedb4069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:12.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.043+0000 7fedb96ba700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb406a3a0 con 0x7fedb4065e50 2024-01-24T15:32:12.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.043+0000 7fedb96ba700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb4069820 con 0x7fedb4155060 2024-01-24T15:32:12.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.043+0000 7fedb96ba700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40c2c70 con 0x7fedb4150cb0 2024-01-24T15:32:12.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.043+0000 7fedb27fc700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fedb4065e50 0x7fedb4154950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58044/0 (socket says 172.21.15.29:58044) 2024-01-24T15:32:12.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.043+0000 7fedb27fc700 1 -- 172.21.15.29:0/472528138 learned_addr learned my addr 172.21.15.29:0/472528138 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:12.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb2ffd700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb4150cb0 0x7fedb4069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:12.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369595632 0 0) 0x7fedb406a3a0 con 0x7fedb4065e50 2024-01-24T15:32:12.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feda00033f0 con 0x7fedb4065e50 2024-01-24T15:32:12.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196198168 0 0) 0x7fedb4069820 con 0x7fedb4155060 2024-01-24T15:32:12.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb406a3a0 con 0x7fedb4155060 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2492350137 0 0) 0x7feda00033f0 con 0x7fedb4065e50 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb4150cb0 msgr2=0x7fedb4069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb4150cb0 0x7fedb4069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 >> v1:172.21.15.163:6789/0 conn(0x7fedb4155060 legacy=0x7fedb40bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb4069b00 con 0x7fedb4065e50 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7feda4001470 con 0x7fedb4065e50 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb2ffd700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb4150cb0 0x7fedb4069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 >> v1:172.21.15.29:6789/0 conn(0x7fedb4065e50 legacy=0x7fedb4154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda0003aa0 0x7feda0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feda0004690 0x7feda0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb4150cb0 0x7feda0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40c2c70 con 0x7feda0003aa0 2024-01-24T15:32:12.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda000b8e0 con 0x7feda0004690 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda000ba70 con 0x7fedb4150cb0 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb2ffd700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda0003aa0 0x7feda0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.044+0000 7fedb27fc700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feda0004690 0x7feda0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.045+0000 7fedb37fe700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb4150cb0 0x7feda0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.045+0000 7fedb2ffd700 1 -- 172.21.15.29:0/472528138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb4150cb0 msgr2=0x7feda0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.045+0000 7fedb2ffd700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb4150cb0 0x7feda0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.045+0000 7fedb2ffd700 1 -- 172.21.15.29:0/472528138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feda0004690 msgr2=0x7feda0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.045+0000 7fedb2ffd700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feda0004690 0x7feda0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.045+0000 7fedb2ffd700 1 -- 172.21.15.29:0/472528138 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c0092a0 con 0x7feda0003aa0 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.045+0000 7fedb2ffd700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda0003aa0 0x7feda0003ea0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 rx=0x7fed9c009c70 tx=0x7fed9c009060).ready entity=mon.0 client_cookie=9afbc1e6a101ac4b server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.045+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0095c0 con 0x7feda0003aa0 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.045+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c00e690 con 0x7feda0003aa0 2024-01-24T15:32:12.051 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.045+0000 7fedb1ffb700 1 -- 172.21.15.29:0/472528138 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00ea10 con 0x7feda0003aa0 2024-01-24T15:32:12.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.046+0000 7fedb96ba700 1 -- 172.21.15.29:0/472528138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda0003aa0 msgr2=0x7feda0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:12.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.046+0000 7fedb96ba700 1 --2- 172.21.15.29:0/472528138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda0003aa0 0x7feda0003ea0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 rx=0x7fed9c009c70 tx=0x7fed9c009060).stop 2024-01-24T15:32:12.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.046+0000 7fedb37fe700 1 -- 172.21.15.29:0/472528138 reap_dead start 2024-01-24T15:32:12.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.046+0000 7fedb96ba700 1 -- 172.21.15.29:0/472528138 shutdown_connections 2024-01-24T15:32:12.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.046+0000 7fedb96ba700 1 -- 172.21.15.29:0/472528138 >> 172.21.15.29:0/472528138 conn(0x7fedb40c85c0 msgr2=0x7fedb40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:12.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.046+0000 7fedb96ba700 1 -- 172.21.15.29:0/472528138 shutdown_connections 2024-01-24T15:32:12.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.046+0000 7fedb96ba700 1 -- 172.21.15.29:0/472528138 wait complete. 2024-01-24T15:32:12.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.048+0000 7fedb96ba700 1 Processor -- start 2024-01-24T15:32:12.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.048+0000 7fedb96ba700 1 -- start start 2024-01-24T15:32:12.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb96ba700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda0003aa0 0x7fedb40631c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:12.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb96ba700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedb40aaaa0 0x7fedb4161c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:12.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb96ba700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fedb4162170 0x7fedb41661b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:12.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb96ba700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40aba40 con 0x7fedb4162170 2024-01-24T15:32:12.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb96ba700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb4165d20 con 0x7fedb40aaaa0 2024-01-24T15:32:12.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb96ba700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40aaf20 con 0x7feda0003aa0 2024-01-24T15:32:12.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fedb4162170 0x7fedb41661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:12.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fedb4162170 0x7fedb41661b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34952/0 (socket says 172.21.15.29:34952) 2024-01-24T15:32:12.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb37fe700 1 -- 172.21.15.29:0/914993809 learned_addr learned my addr 172.21.15.29:0/914993809 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:12.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb2ffd700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda0003aa0 0x7fedb40631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb27fc700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedb40aaaa0 0x7fedb4161c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb37fe700 1 -- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda0003aa0 msgr2=0x7fedb40631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb37fe700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda0003aa0 0x7fedb40631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb37fe700 1 -- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedb40aaaa0 msgr2=0x7fedb4161c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb37fe700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedb40aaaa0 0x7fedb4161c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb37fe700 1 -- 172.21.15.29:0/914993809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb40ab1a0 con 0x7fedb4162170 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb27fc700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedb40aaaa0 0x7fedb4161c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedb37fe700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fedb4162170 0x7fedb41661b0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7fedb406aa40 tx=0x7fedac007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.049+0000 7fedaaffd700 1 -- 172.21.15.29:0/914993809 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac002ea0 con 0x7fedb4162170 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.050+0000 7fedaaffd700 1 -- 172.21.15.29:0/914993809 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac00eea0 con 0x7fedb4162170 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.050+0000 7fedaaffd700 1 -- 172.21.15.29:0/914993809 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac00c9e0 con 0x7fedb4162170 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.050+0000 7fedb96ba700 1 -- 172.21.15.29:0/914993809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb40ab480 con 0x7fedb4162170 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.050+0000 7fedb96ba700 1 -- 172.21.15.29:0/914993809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb41666a0 con 0x7fedb4162170 2024-01-24T15:32:12.056 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.051+0000 7fedb96ba700 1 -- 172.21.15.29:0/914993809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb4165af0 con 0x7fedb4162170 2024-01-24T15:32:12.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.051+0000 7fedaaffd700 1 -- 172.21.15.29:0/914993809 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fedac0133e0 con 0x7fedb4162170 2024-01-24T15:32:12.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.051+0000 7fedaaffd700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fed9405f270 0x7fed94061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:12.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.051+0000 7fedaaffd700 1 -- 172.21.15.29:0/914993809 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6964+0+0 (secure 0 0 0) 0x7fedac07a100 con 0x7fedb4162170 2024-01-24T15:32:12.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.052+0000 7fedb2ffd700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fed9405f270 0x7fed94061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:12.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.052+0000 7fedb2ffd700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fed9405f270 0x7fed94061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fed9c006d50 tx=0x7fed9c01a000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:12.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.056+0000 7fedaaffd700 1 -- 172.21.15.29:0/914993809 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fedb4165af0 con 0x7fedb4162170 2024-01-24T15:32:12.319 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:12.317+0000 7fedb96ba700 1 -- 172.21.15.29:0/914993809 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7fedb4165eb0 con 0x7fedb4162170 2024-01-24T15:32:12.939 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:32:12.942 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:32:13.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.199+0000 7fedaaffd700 1 -- 172.21.15.29:0/914993809 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3031) v1 ==== 163+0+0 (secure 0 0 0) 0x7fedac019020 con 0x7fedb4162170 2024-01-24T15:32:13.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.202+0000 7fedb96ba700 1 -- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fed9405f270 msgr2=0x7fed94061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:13.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.202+0000 7fedb96ba700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fed9405f270 0x7fed94061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fed9c006d50 tx=0x7fed9c01a000).stop 2024-01-24T15:32:13.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.202+0000 7fedb96ba700 1 -- 172.21.15.29:0/914993809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fedb4162170 msgr2=0x7fedb41661b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:13.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.202+0000 7fedb96ba700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fedb4162170 0x7fedb41661b0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7fedb406aa40 tx=0x7fedac007f30).stop 2024-01-24T15:32:13.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.202+0000 7fedb96ba700 1 -- 172.21.15.29:0/914993809 shutdown_connections 2024-01-24T15:32:13.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.202+0000 7fedb96ba700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fedb4162170 0x7fedb41661b0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:13.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.202+0000 7fedb96ba700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fed9405f270 0x7fed94061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:13.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.202+0000 7fedb96ba700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda0003aa0 0x7fedb40631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:13.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.202+0000 7fedb96ba700 1 --2- 172.21.15.29:0/914993809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedb40aaaa0 0x7fedb4161c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:13.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.202+0000 7fedb96ba700 1 -- 172.21.15.29:0/914993809 >> 172.21.15.29:0/914993809 conn(0x7fedb40c85c0 msgr2=0x7fedb4151090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:13.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.202+0000 7fedb96ba700 1 -- 172.21.15.29:0/914993809 shutdown_connections 2024-01-24T15:32:13.205 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.202+0000 7fedb96ba700 1 -- 172.21.15.29:0/914993809 wait complete. 2024-01-24T15:32:13.220 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2024-01-24T15:32:13.220 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2024-01-24T15:32:13.220 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2024-01-24T15:32:13.220 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2024-01-24T15:32:13.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.353+0000 7fd27640a700 1 Processor -- start 2024-01-24T15:32:13.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.353+0000 7fd27640a700 1 -- start start 2024-01-24T15:32:13.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd27640a700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd270062ec0 0x7fd2700c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:13.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd27640a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2701550f0 0x7fd270069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:13.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd27640a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd270065360 con 0x7fd2701540a0 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd27640a700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2700a74f0 con 0x7fd270062ec0 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd27640a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2700a6ce0 con 0x7fd2701550f0 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd26f7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd270062ec0 0x7fd2700c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd26ffff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd2701540a0 0x7fd2700627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:58056/0 (socket says 172.21.15.29:58056) 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd26ffff700 1 -- 172.21.15.29:0/2552225803 learned_addr learned my addr 172.21.15.29:0/2552225803 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd2749a7700 1 --2- 172.21.15.29:0/2552225803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2701550f0 0x7fd270069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd26f7fe700 1 -- 172.21.15.29:0/2552225803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2701550f0 msgr2=0x7fd270069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd26f7fe700 1 --2- 172.21.15.29:0/2552225803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2701550f0 0x7fd270069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd26f7fe700 1 -- 172.21.15.29:0/2552225803 >> v1:172.21.15.29:6789/0 conn(0x7fd2701540a0 legacy=0x7fd2700627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd26effd700 1 -- 172.21.15.29:0/2552225803 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700827231 0 0) 0x7fd270065360 con 0x7fd2701540a0 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.354+0000 7fd26f7fe700 1 -- 172.21.15.29:0/2552225803 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2700636f0 con 0x7fd270062ec0 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.355+0000 7fd26f7fe700 1 --2- 172.21.15.29:0/2552225803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd270062ec0 0x7fd2700c4ca0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7fd264009760 tx=0x7fd264009060).ready entity=mon.0 client_cookie=1198f88b5d678e87 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.355+0000 7fd26effd700 1 -- 172.21.15.29:0/2552225803 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26400b580 con 0x7fd270062ec0 2024-01-24T15:32:13.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.355+0000 7fd26effd700 1 -- 172.21.15.29:0/2552225803 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26400b6e0 con 0x7fd270062ec0 2024-01-24T15:32:13.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.355+0000 7fd26effd700 1 -- 172.21.15.29:0/2552225803 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26400ba60 con 0x7fd270062ec0 2024-01-24T15:32:13.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.355+0000 7fd27640a700 1 -- 172.21.15.29:0/2552225803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd270062ec0 msgr2=0x7fd2700c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:13.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.355+0000 7fd27640a700 1 --2- 172.21.15.29:0/2552225803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd270062ec0 0x7fd2700c4ca0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7fd264009760 tx=0x7fd264009060).stop 2024-01-24T15:32:13.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.355+0000 7fd27640a700 1 -- 172.21.15.29:0/2552225803 shutdown_connections 2024-01-24T15:32:13.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.355+0000 7fd27640a700 1 --2- 172.21.15.29:0/2552225803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2701550f0 0x7fd270069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:13.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.355+0000 7fd27640a700 1 --2- 172.21.15.29:0/2552225803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd270062ec0 0x7fd2700c4ca0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:13.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.355+0000 7fd27640a700 1 -- 172.21.15.29:0/2552225803 >> 172.21.15.29:0/2552225803 conn(0x7fd2700c85c0 msgr2=0x7fd2700c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:13.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.355+0000 7fd27640a700 1 -- 172.21.15.29:0/2552225803 shutdown_connections 2024-01-24T15:32:13.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.355+0000 7fd27640a700 1 -- 172.21.15.29:0/2552225803 wait complete. 2024-01-24T15:32:13.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.357+0000 7fd27640a700 1 Processor -- start 2024-01-24T15:32:13.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.357+0000 7fd27640a700 1 -- start start 2024-01-24T15:32:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.358+0000 7fd27640a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd270062ec0 0x7fd2700a7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.358+0000 7fd27640a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2701540a0 0x7fd2700c1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.358+0000 7fd27640a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2701550f0 0x7fd27014cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.358+0000 7fd27640a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd270168090 con 0x7fd2701540a0 2024-01-24T15:32:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.358+0000 7fd27640a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2701682e0 con 0x7fd270062ec0 2024-01-24T15:32:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.358+0000 7fd27640a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2701685c0 con 0x7fd2701550f0 2024-01-24T15:32:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd26f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2701540a0 0x7fd2700c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd26f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2701540a0 0x7fd2700c1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34974/0 (socket says 172.21.15.29:34974) 2024-01-24T15:32:13.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd26f7fe700 1 -- 172.21.15.29:0/1838702502 learned_addr learned my addr 172.21.15.29:0/1838702502 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd2749a7700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2701550f0 0x7fd27014cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd26ffff700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd270062ec0 0x7fd2700a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd26f7fe700 1 -- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2701550f0 msgr2=0x7fd27014cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd26f7fe700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2701550f0 0x7fd27014cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd26f7fe700 1 -- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd270062ec0 msgr2=0x7fd2700a7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd26f7fe700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd270062ec0 0x7fd2700a7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd26f7fe700 1 -- 172.21.15.29:0/1838702502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2640092a0 con 0x7fd2701540a0 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd26f7fe700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2701540a0 0x7fd2700c1330 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 rx=0x7fd270069030 tx=0x7fd264017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd257fff700 1 -- 172.21.15.29:0/1838702502 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26400b850 con 0x7fd2701540a0 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd257fff700 1 -- 172.21.15.29:0/1838702502 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26400b9b0 con 0x7fd2701540a0 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd27640a700 1 -- 172.21.15.29:0/1838702502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2701688a0 con 0x7fd2701540a0 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd257fff700 1 -- 172.21.15.29:0/1838702502 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26401b660 con 0x7fd2701540a0 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.359+0000 7fd27640a700 1 -- 172.21.15.29:0/1838702502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd270168e60 con 0x7fd2701540a0 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.360+0000 7fd27640a700 1 -- 172.21.15.29:0/1838702502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2700aaaa0 con 0x7fd2701540a0 2024-01-24T15:32:13.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.361+0000 7fd257fff700 1 -- 172.21.15.29:0/1838702502 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd26401ba90 con 0x7fd2701540a0 2024-01-24T15:32:13.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.361+0000 7fd257fff700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd25c05f270 0x7fd25c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:13.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.361+0000 7fd257fff700 1 -- 172.21.15.29:0/1838702502 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6964+0+0 (secure 0 0 0) 0x7fd26407c1c0 con 0x7fd2701540a0 2024-01-24T15:32:13.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.362+0000 7fd26ffff700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd25c05f270 0x7fd25c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:13.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.362+0000 7fd26ffff700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd25c05f270 0x7fd25c061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fd2600011e0 tx=0x7fd260009040).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:13.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.366+0000 7fd257fff700 1 -- 172.21.15.29:0/1838702502 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd26404cfc0 con 0x7fd2701540a0 2024-01-24T15:32:13.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:13.619+0000 7fd27640a700 1 -- 172.21.15.29:0/1838702502 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7fd2700475e0 con 0x7fd2701540a0 2024-01-24T15:32:14.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.201+0000 7fd257fff700 1 -- 172.21.15.29:0/1838702502 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3032) v1 ==== 158+0+0 (secure 0 0 0) 0x7fd2640290e0 con 0x7fd2701540a0 2024-01-24T15:32:14.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.204+0000 7fd27640a700 1 -- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd25c05f270 msgr2=0x7fd25c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:14.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.204+0000 7fd27640a700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd25c05f270 0x7fd25c061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fd2600011e0 tx=0x7fd260009040).stop 2024-01-24T15:32:14.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.204+0000 7fd27640a700 1 -- 172.21.15.29:0/1838702502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2701540a0 msgr2=0x7fd2700c1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:14.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.204+0000 7fd27640a700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2701540a0 0x7fd2700c1330 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 rx=0x7fd270069030 tx=0x7fd264017000).stop 2024-01-24T15:32:14.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.204+0000 7fd27640a700 1 -- 172.21.15.29:0/1838702502 shutdown_connections 2024-01-24T15:32:14.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.204+0000 7fd27640a700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd25c05f270 0x7fd25c061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.204+0000 7fd27640a700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2701550f0 0x7fd27014cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.204+0000 7fd27640a700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd270062ec0 0x7fd2700a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.204+0000 7fd27640a700 1 --2- 172.21.15.29:0/1838702502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2701540a0 0x7fd2700c1330 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.204+0000 7fd27640a700 1 -- 172.21.15.29:0/1838702502 >> 172.21.15.29:0/1838702502 conn(0x7fd2700c85c0 msgr2=0x7fd270065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:14.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.205+0000 7fd27640a700 1 -- 172.21.15.29:0/1838702502 shutdown_connections 2024-01-24T15:32:14.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.205+0000 7fd27640a700 1 -- 172.21.15.29:0/1838702502 wait complete. 2024-01-24T15:32:14.222 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-24T15:32:14.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.358+0000 7fab7d8be700 1 Processor -- start 2024-01-24T15:32:14.360 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.358+0000 7fab7d8be700 1 -- start start 2024-01-24T15:32:14.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab7d8be700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab780c8480 0x7fab781553b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab7d8be700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab78064760 0x7fab78064c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab7d8be700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab78065100 con 0x7fab7814e980 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab7d8be700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab780c63d0 con 0x7fab78064760 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab7d8be700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab780c68a0 con 0x7fab780c8480 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab777fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab78064760 0x7fab78064c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab76ffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fab7814e980 0x7fab780c5cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56760/0 (socket says 172.21.15.29:56760) 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab76ffd700 1 -- 172.21.15.29:0/3960015145 learned_addr learned my addr 172.21.15.29:0/3960015145 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab767fc700 1 --2- 172.21.15.29:0/3960015145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab780c8480 0x7fab781553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab777fe700 1 -- 172.21.15.29:0/3960015145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab780c8480 msgr2=0x7fab781553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab777fe700 1 --2- 172.21.15.29:0/3960015145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab780c8480 0x7fab781553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab777fe700 1 -- 172.21.15.29:0/3960015145 >> v1:172.21.15.29:6789/0 conn(0x7fab7814e980 legacy=0x7fab780c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab777fe700 1 -- 172.21.15.29:0/3960015145 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab780c65c0 con 0x7fab78064760 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab75ffb700 1 -- 172.21.15.29:0/3960015145 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354820537 0 0) 0x7fab68001110 con 0x7fab7814e980 2024-01-24T15:32:14.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.359+0000 7fab767fc700 1 --2- 172.21.15.29:0/3960015145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab780c8480 0x7fab781553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.360+0000 7fab777fe700 1 --2- 172.21.15.29:0/3960015145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab78064760 0x7fab78064c10 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7fab70009e60 tx=0x7fab70009780).ready entity=mon.0 client_cookie=9ead500f0e13814a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.360+0000 7fab75ffb700 1 -- 172.21.15.29:0/3960015145 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7000e070 con 0x7fab78064760 2024-01-24T15:32:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.360+0000 7fab75ffb700 1 -- 172.21.15.29:0/3960015145 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7000bc30 con 0x7fab78064760 2024-01-24T15:32:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.360+0000 7fab75ffb700 1 -- 172.21.15.29:0/3960015145 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70012410 con 0x7fab78064760 2024-01-24T15:32:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.360+0000 7fab7d8be700 1 -- 172.21.15.29:0/3960015145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab78064760 msgr2=0x7fab78064c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.360+0000 7fab7d8be700 1 --2- 172.21.15.29:0/3960015145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab78064760 0x7fab78064c10 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7fab70009e60 tx=0x7fab70009780).stop 2024-01-24T15:32:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.360+0000 7fab7d8be700 1 -- 172.21.15.29:0/3960015145 shutdown_connections 2024-01-24T15:32:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.360+0000 7fab7d8be700 1 --2- 172.21.15.29:0/3960015145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab78064760 0x7fab78064c10 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.360+0000 7fab7d8be700 1 --2- 172.21.15.29:0/3960015145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab780c8480 0x7fab781553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.360+0000 7fab7d8be700 1 -- 172.21.15.29:0/3960015145 >> 172.21.15.29:0/3960015145 conn(0x7fab780cb870 msgr2=0x7fab780c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.360+0000 7fab7d8be700 1 -- 172.21.15.29:0/3960015145 shutdown_connections 2024-01-24T15:32:14.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.360+0000 7fab7d8be700 1 -- 172.21.15.29:0/3960015145 wait complete. 2024-01-24T15:32:14.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.362+0000 7fab7d8be700 1 Processor -- start 2024-01-24T15:32:14.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.362+0000 7fab7d8be700 1 -- start start 2024-01-24T15:32:14.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab7d8be700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab78064760 0x7fab780b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:14.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab7d8be700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab780c8480 0x7fab78141750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab7d8be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7814e980 0x7fab780c0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab7d8be700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab780c2c10 con 0x7fab780c8480 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab7d8be700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab780afa60 con 0x7fab7814e980 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab7d8be700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab780afd40 con 0x7fab78064760 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab767fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab780c8480 0x7fab78141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab767fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab780c8480 0x7fab78141750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53286/0 (socket says 172.21.15.29:53286) 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab767fc700 1 -- 172.21.15.29:0/392195072 learned_addr learned my addr 172.21.15.29:0/392195072 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab76ffd700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab78064760 0x7fab780b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab777fe700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7814e980 0x7fab780c0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab767fc700 1 -- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab78064760 msgr2=0x7fab780b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab767fc700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab78064760 0x7fab780b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab767fc700 1 -- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7814e980 msgr2=0x7fab780c0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab767fc700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7814e980 0x7fab780c0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab767fc700 1 -- 172.21.15.29:0/392195072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab70009a20 con 0x7fab780c8480 2024-01-24T15:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab777fe700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7814e980 0x7fab780c0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab767fc700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab780c8480 0x7fab78141750 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 rx=0x7fab6000d860 tx=0x7fab60008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.363+0000 7fab6effd700 1 -- 172.21.15.29:0/392195072 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab60011590 con 0x7fab780c8480 2024-01-24T15:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.364+0000 7fab6effd700 1 -- 172.21.15.29:0/392195072 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab600116f0 con 0x7fab780c8480 2024-01-24T15:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.364+0000 7fab7d8be700 1 -- 172.21.15.29:0/392195072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab780b0020 con 0x7fab780c8480 2024-01-24T15:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.364+0000 7fab6effd700 1 -- 172.21.15.29:0/392195072 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab60011a70 con 0x7fab780c8480 2024-01-24T15:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.364+0000 7fab7d8be700 1 -- 172.21.15.29:0/392195072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab780b05e0 con 0x7fab780c8480 2024-01-24T15:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.365+0000 7fab7d8be700 1 -- 172.21.15.29:0/392195072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab780475e0 con 0x7fab780c8480 2024-01-24T15:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.365+0000 7fab6effd700 1 -- 172.21.15.29:0/392195072 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fab6001c460 con 0x7fab780c8480 2024-01-24T15:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.365+0000 7fab6effd700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fab5805f270 0x7fab58061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.365+0000 7fab6effd700 1 -- 172.21.15.29:0/392195072 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6964+0+0 (secure 0 0 0) 0x7fab6007a5f0 con 0x7fab780c8480 2024-01-24T15:32:14.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.366+0000 7fab76ffd700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fab5805f270 0x7fab58061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:14.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.366+0000 7fab76ffd700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fab5805f270 0x7fab58061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fab68001720 tx=0x7fab68009040).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:14.371 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.370+0000 7fab6effd700 1 -- 172.21.15.29:0/392195072 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fab6004b390 con 0x7fab780c8480 2024-01-24T15:32:14.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.551+0000 7fab7d8be700 1 -- 172.21.15.29:0/392195072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fab78065780 con 0x7fab780c8480 2024-01-24T15:32:14.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.555+0000 7fab6effd700 1 -- 172.21.15.29:0/392195072 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) v1 ==== 122+0+0 (secure 0 0 0) 0x7fab6004e9b0 con 0x7fab780c8480 2024-01-24T15:32:14.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.557+0000 7fab7d8be700 1 -- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fab5805f270 msgr2=0x7fab58061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:14.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.557+0000 7fab7d8be700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fab5805f270 0x7fab58061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fab68001720 tx=0x7fab68009040).stop 2024-01-24T15:32:14.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.557+0000 7fab7d8be700 1 -- 172.21.15.29:0/392195072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab780c8480 msgr2=0x7fab78141750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:14.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.557+0000 7fab7d8be700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab780c8480 0x7fab78141750 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 rx=0x7fab6000d860 tx=0x7fab60008040).stop 2024-01-24T15:32:14.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.557+0000 7fab7d8be700 1 -- 172.21.15.29:0/392195072 shutdown_connections 2024-01-24T15:32:14.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.557+0000 7fab7d8be700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab7814e980 0x7fab780c0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.557+0000 7fab7d8be700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fab5805f270 0x7fab58061730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.557+0000 7fab7d8be700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab78064760 0x7fab780b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.557+0000 7fab7d8be700 1 --2- 172.21.15.29:0/392195072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab780c8480 0x7fab78141750 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.557+0000 7fab7d8be700 1 -- 172.21.15.29:0/392195072 >> 172.21.15.29:0/392195072 conn(0x7fab780cb870 msgr2=0x7fab780c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:14.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.557+0000 7fab7d8be700 1 -- 172.21.15.29:0/392195072 shutdown_connections 2024-01-24T15:32:14.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.557+0000 7fab7d8be700 1 -- 172.21.15.29:0/392195072 wait complete. 2024-01-24T15:32:14.573 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T15:32:14.573 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:32:14.617 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:32:14.668 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T15:32:14.668 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-24T15:32:14.669 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:32:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.802+0000 7f220959e700 1 Processor -- start 2024-01-24T15:32:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.802+0000 7f220959e700 1 -- start start 2024-01-24T15:32:14.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f220959e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f22041519c0 0x7f22040687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:14.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f220959e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f22040c62e0 0x7f22040a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f220959e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22040a6db0 con 0x7f2204068ce0 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f220959e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22040691d0 con 0x7f22040c62e0 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f220959e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22040694b0 con 0x7f22041519c0 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f2208d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f22040c62e0 0x7f22040a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f22037fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f2204068ce0 0x7f22040c5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56772/0 (socket says 172.21.15.29:56772) 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f2208d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f22040c62e0 0x7f22040a41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53290/0 (socket says 172.21.15.29:53290) 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f22037fe700 1 -- 172.21.15.29:0/2184918680 learned_addr learned my addr 172.21.15.29:0/2184918680 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f2203fff700 1 --2- 172.21.15.29:0/2184918680 >> v2:172.21.15.163:3300/0 conn(0x7f22041519c0 0x7f22040687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f2208d9d700 1 -- 172.21.15.29:0/2184918680 >> v2:172.21.15.163:3300/0 conn(0x7f22041519c0 msgr2=0x7f22040687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f2208d9d700 1 --2- 172.21.15.29:0/2184918680 >> v2:172.21.15.163:3300/0 conn(0x7f22041519c0 0x7f22040687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f2208d9d700 1 -- 172.21.15.29:0/2184918680 >> v1:172.21.15.29:6789/0 conn(0x7f2204068ce0 legacy=0x7f22040c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f2208d9d700 1 -- 172.21.15.29:0/2184918680 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2204069790 con 0x7f22040c62e0 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f2202ffd700 1 -- 172.21.15.29:0/2184918680 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374484543 0 0) 0x7f22040a6db0 con 0x7f2204068ce0 2024-01-24T15:32:14.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f2203fff700 1 --2- 172.21.15.29:0/2184918680 >> v2:172.21.15.163:3300/0 conn(0x7f22041519c0 0x7f22040687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.803+0000 7f2208d9d700 1 --2- 172.21.15.29:0/2184918680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22040c62e0 0x7f22040a41f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x7f2204067780 tx=0x7f21fc009780).ready entity=mon.0 client_cookie=b2ce4f75c24f1eb6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.804+0000 7f2202ffd700 1 -- 172.21.15.29:0/2184918680 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc00e070 con 0x7f22040c62e0 2024-01-24T15:32:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.804+0000 7f2202ffd700 1 -- 172.21.15.29:0/2184918680 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc00bcc0 con 0x7f22040c62e0 2024-01-24T15:32:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.804+0000 7f2202ffd700 1 -- 172.21.15.29:0/2184918680 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc012620 con 0x7f22040c62e0 2024-01-24T15:32:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.804+0000 7f220959e700 1 -- 172.21.15.29:0/2184918680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22040c62e0 msgr2=0x7f22040a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.804+0000 7f220959e700 1 --2- 172.21.15.29:0/2184918680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22040c62e0 0x7f22040a41f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x7f2204067780 tx=0x7f21fc009780).stop 2024-01-24T15:32:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.804+0000 7f220959e700 1 -- 172.21.15.29:0/2184918680 shutdown_connections 2024-01-24T15:32:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.804+0000 7f220959e700 1 --2- 172.21.15.29:0/2184918680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22040c62e0 0x7f22040a41f0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.804+0000 7f220959e700 1 --2- 172.21.15.29:0/2184918680 >> v2:172.21.15.163:3300/0 conn(0x7f22041519c0 0x7f22040687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.804+0000 7f220959e700 1 -- 172.21.15.29:0/2184918680 >> 172.21.15.29:0/2184918680 conn(0x7f22040c3b00 msgr2=0x7f22040c3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.804+0000 7f220959e700 1 -- 172.21.15.29:0/2184918680 shutdown_connections 2024-01-24T15:32:14.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.804+0000 7f220959e700 1 -- 172.21.15.29:0/2184918680 wait complete. 2024-01-24T15:32:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.805+0000 7f220959e700 1 Processor -- start 2024-01-24T15:32:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.806+0000 7f220959e700 1 -- start start 2024-01-24T15:32:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.806+0000 7f220959e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2204068ce0 0x7f22041500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.806+0000 7f220959e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22040c62e0 0x7f2204165ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.806+0000 7f220959e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22041519c0 0x7f22040ac130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f220959e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2204165650 con 0x7f2204068ce0 2024-01-24T15:32:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f220959e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2204165020 con 0x7f22041519c0 2024-01-24T15:32:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f220959e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22040acba0 con 0x7f22040c62e0 2024-01-24T15:32:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f2203fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2204068ce0 0x7f22041500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f2203fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2204068ce0 0x7f22041500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53292/0 (socket says 172.21.15.29:53292) 2024-01-24T15:32:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f2203fff700 1 -- 172.21.15.29:0/3993649917 learned_addr learned my addr 172.21.15.29:0/3993649917 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f2208d9d700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22041519c0 0x7f22040ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f22037fe700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22040c62e0 0x7f2204165ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f2203fff700 1 -- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22040c62e0 msgr2=0x7f2204165ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f2203fff700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22040c62e0 0x7f2204165ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f2203fff700 1 -- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22041519c0 msgr2=0x7f22040ac130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f2203fff700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22041519c0 0x7f22040ac130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f2203fff700 1 -- 172.21.15.29:0/3993649917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc009a20 con 0x7f2204068ce0 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f2203fff700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2204068ce0 0x7f22041500e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7f21f400d890 tx=0x7f21f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f21ebfff700 1 -- 172.21.15.29:0/3993649917 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f40125f0 con 0x7f2204068ce0 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f21ebfff700 1 -- 172.21.15.29:0/3993649917 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f4012750 con 0x7f2204068ce0 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f220959e700 1 -- 172.21.15.29:0/3993649917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22040ace80 con 0x7f2204068ce0 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f21ebfff700 1 -- 172.21.15.29:0/3993649917 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4012ad0 con 0x7f2204068ce0 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.807+0000 7f220959e700 1 -- 172.21.15.29:0/3993649917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22040be960 con 0x7f2204068ce0 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.808+0000 7f220959e700 1 -- 172.21.15.29:0/3993649917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22040680f0 con 0x7f2204068ce0 2024-01-24T15:32:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.809+0000 7f21ebfff700 1 -- 172.21.15.29:0/3993649917 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f21f4013480 con 0x7f2204068ce0 2024-01-24T15:32:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.809+0000 7f21ebfff700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f21f005f270 0x7f21f0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.809+0000 7f21ebfff700 1 -- 172.21.15.29:0/3993649917 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6964+0+0 (secure 0 0 0) 0x7f21f407a490 con 0x7f2204068ce0 2024-01-24T15:32:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.810+0000 7f22037fe700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f21f005f270 0x7f21f0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.810+0000 7f22037fe700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f21f005f270 0x7f21f0061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f2204157750 tx=0x7f21f8009040).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:14.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:14.814+0000 7f21ebfff700 1 -- 172.21.15.29:0/3993649917 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f21f404b230 con 0x7f2204068ce0 2024-01-24T15:32:15.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.039+0000 7f220959e700 1 -- 172.21.15.29:0/3993649917 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2204069a70 con 0x7f2204068ce0 2024-01-24T15:32:15.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.040+0000 7f21ebfff700 1 -- 172.21.15.29:0/3993649917 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f21f404e850 con 0x7f2204068ce0 2024-01-24T15:32:15.042 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:15.042 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8060,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3418669722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3418669722},{"type":"v1","addr":"172.21.15.163:6836","nonce":3418669722}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8063,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/407488941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":407488941},{"type":"v1","addr":"172.21.15.163:6837","nonce":407488941}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":8066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/4018429461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":4018429461},{"type":"v1","addr":"172.21.15.29:6834","nonce":4018429461}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":9957,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3319144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":3319144675},{"type":"v1","addr":"172.21.15.29:6835","nonce":3319144675}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92}],"filesystems":[]} 2024-01-24T15:32:15.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.042+0000 7f220959e700 1 -- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f21f005f270 msgr2=0x7f21f0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:15.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.042+0000 7f220959e700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f21f005f270 0x7f21f0061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f2204157750 tx=0x7f21f8009040).stop 2024-01-24T15:32:15.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.042+0000 7f220959e700 1 -- 172.21.15.29:0/3993649917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2204068ce0 msgr2=0x7f22041500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:15.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.042+0000 7f220959e700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2204068ce0 0x7f22041500e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7f21f400d890 tx=0x7f21f4008040).stop 2024-01-24T15:32:15.044 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.042+0000 7f220959e700 1 -- 172.21.15.29:0/3993649917 shutdown_connections 2024-01-24T15:32:15.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.042+0000 7f220959e700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22041519c0 0x7f22040ac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:15.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.042+0000 7f220959e700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2204068ce0 0x7f22041500e0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:15.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.042+0000 7f220959e700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f21f005f270 0x7f21f0061730 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:15.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.042+0000 7f220959e700 1 --2- 172.21.15.29:0/3993649917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22040c62e0 0x7f2204165ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:15.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.042+0000 7f220959e700 1 -- 172.21.15.29:0/3993649917 >> 172.21.15.29:0/3993649917 conn(0x7f22040c3b00 msgr2=0x7f22040bd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:15.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.042+0000 7f220959e700 1 -- 172.21.15.29:0/3993649917 shutdown_connections 2024-01-24T15:32:15.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.042+0000 7f220959e700 1 -- 172.21.15.29:0/3993649917 wait complete. 2024-01-24T15:32:15.045 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 92 2024-01-24T15:32:15.058 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-24T15:32:15.058 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-24T15:32:15.058 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-24T15:32:15.058 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:32:15.058 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-24T15:32:15.058 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-24T15:32:15.059 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-24T15:32:15.059 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:32:15.059 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-24T15:32:15.059 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-24T15:32:15.059 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-24T15:32:15.059 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:32:15.059 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-24T15:32:15.060 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-24T15:32:15.060 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-24T15:32:15.060 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:32:15.068 INFO:tasks.ceph.mds.a:Stopped 2024-01-24T15:32:15.068 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-24T15:32:15.069 INFO:tasks.ceph.mds.c:Stopped 2024-01-24T15:32:15.069 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-24T15:32:15.071 INFO:tasks.ceph.mds.d:Stopped 2024-01-24T15:32:15.071 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-24T15:32:15.071 INFO:tasks.ceph.mds.b:Stopped 2024-01-24T15:32:15.072 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-24T15:32:15.111 INFO:tasks.ceph.mds.a:Started 2024-01-24T15:32:15.111 DEBUG:teuthology.parallel:result is None 2024-01-24T15:32:15.112 INFO:tasks.ceph.mds.c:Started 2024-01-24T15:32:15.112 DEBUG:teuthology.parallel:result is None 2024-01-24T15:32:15.115 INFO:tasks.ceph.mds.d:Started 2024-01-24T15:32:15.115 DEBUG:teuthology.parallel:result is None 2024-01-24T15:32:15.116 INFO:tasks.ceph.mds.b:Started 2024-01-24T15:32:15.116 DEBUG:teuthology.parallel:result is None 2024-01-24T15:32:15.116 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-24T15:32:15.197 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-01-24T15:32:15.197 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T15:32:15.198 INFO:tasks.ceph.mds.a.smithi029.stdout:starting mds.a at 2024-01-24T15:32:15.199 INFO:tasks.ceph.mds.c.smithi029.stdout:starting mds.c at 2024-01-24T15:32:15.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.259+0000 7fccf1384700 1 Processor -- start 2024-01-24T15:32:15.261 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.259+0000 7fccf1384700 1 -- start start 2024-01-24T15:32:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fccf1384700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fccec154240 0x7fccec064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fccf1384700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fccec0a9d20 0x7fccec0aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fccf1384700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccec1550e0 con 0x7fccec157cf0 2024-01-24T15:32:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fccf1384700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccec0c0d10 con 0x7fccec0a9d20 2024-01-24T15:32:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fccf1384700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccec0aa670 con 0x7fccec154240 2024-01-24T15:32:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fccea7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fccec0a9d20 0x7fccec0aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fcceb7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fccec157cf0 0x7fccec158100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56790/0 (socket says 172.21.15.29:56790) 2024-01-24T15:32:15.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fccea7fc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fccec0a9d20 0x7fccec0aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53356/0 (socket says 172.21.15.29:53356) 2024-01-24T15:32:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fcceb7fe700 1 -- 172.21.15.29:0/2966668812 learned_addr learned my addr 172.21.15.29:0/2966668812 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fcceaffd700 1 --2- 172.21.15.29:0/2966668812 >> v2:172.21.15.163:3300/0 conn(0x7fccec154240 0x7fccec064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fccea7fc700 1 -- 172.21.15.29:0/2966668812 >> v2:172.21.15.163:3300/0 conn(0x7fccec154240 msgr2=0x7fccec064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fccea7fc700 1 --2- 172.21.15.29:0/2966668812 >> v2:172.21.15.163:3300/0 conn(0x7fccec154240 0x7fccec064780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fccea7fc700 1 -- 172.21.15.29:0/2966668812 >> v1:172.21.15.29:6789/0 conn(0x7fccec157cf0 legacy=0x7fccec158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fccea7fc700 1 -- 172.21.15.29:0/2966668812 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0650d0 con 0x7fccec0a9d20 2024-01-24T15:32:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fcce9ffb700 1 -- 172.21.15.29:0/2966668812 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163525351 0 0) 0x7fccec1550e0 con 0x7fccec157cf0 2024-01-24T15:32:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.260+0000 7fccea7fc700 1 --2- 172.21.15.29:0/2966668812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec0a9d20 0x7fccec0aa180 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 rx=0x7fccec083d10 tx=0x7fcce0009060).ready entity=mon.0 client_cookie=73822a34ff3b110c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.261+0000 7fcce9ffb700 1 -- 172.21.15.29:0/2966668812 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce000b580 con 0x7fccec0a9d20 2024-01-24T15:32:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.261+0000 7fcce9ffb700 1 -- 172.21.15.29:0/2966668812 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce000b6e0 con 0x7fccec0a9d20 2024-01-24T15:32:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.261+0000 7fcce9ffb700 1 -- 172.21.15.29:0/2966668812 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce000ba60 con 0x7fccec0a9d20 2024-01-24T15:32:15.265 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.261+0000 7fccf1384700 1 -- 172.21.15.29:0/2966668812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec0a9d20 msgr2=0x7fccec0aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.261+0000 7fccf1384700 1 --2- 172.21.15.29:0/2966668812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec0a9d20 0x7fccec0aa180 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 rx=0x7fccec083d10 tx=0x7fcce0009060).stop 2024-01-24T15:32:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.261+0000 7fccf1384700 1 -- 172.21.15.29:0/2966668812 shutdown_connections 2024-01-24T15:32:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.261+0000 7fccf1384700 1 --2- 172.21.15.29:0/2966668812 >> v2:172.21.15.163:3300/0 conn(0x7fccec154240 0x7fccec064780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.261+0000 7fccf1384700 1 --2- 172.21.15.29:0/2966668812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec0a9d20 0x7fccec0aa180 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.261+0000 7fccf1384700 1 -- 172.21.15.29:0/2966668812 >> 172.21.15.29:0/2966668812 conn(0x7fccec0c8310 msgr2=0x7fccec05e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.261+0000 7fccf1384700 1 -- 172.21.15.29:0/2966668812 shutdown_connections 2024-01-24T15:32:15.266 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.261+0000 7fccf1384700 1 -- 172.21.15.29:0/2966668812 wait complete. 2024-01-24T15:32:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.263+0000 7fccf1384700 1 Processor -- start 2024-01-24T15:32:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.263+0000 7fccf1384700 1 -- start start 2024-01-24T15:32:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.263+0000 7fccf1384700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccec0a9d20 0x7fccec0c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.263+0000 7fccf1384700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccec154240 0x7fccec167be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.263+0000 7fccf1384700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec157cf0 0x7fccec0ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.263+0000 7fccf1384700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec0af420 con 0x7fccec157cf0 2024-01-24T15:32:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.263+0000 7fccf1384700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec0af120 con 0x7fccec0a9d20 2024-01-24T15:32:15.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.263+0000 7fccf1384700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec168430 con 0x7fccec154240 2024-01-24T15:32:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fcceb7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec157cf0 0x7fccec0ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fcceb7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec157cf0 0x7fccec0ac1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53358/0 (socket says 172.21.15.29:53358) 2024-01-24T15:32:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fcceb7fe700 1 -- 172.21.15.29:0/2025746312 learned_addr learned my addr 172.21.15.29:0/2025746312 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fccea7fc700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccec154240 0x7fccec167be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fcceaffd700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccec0a9d20 0x7fccec0c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fcceb7fe700 1 -- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccec154240 msgr2=0x7fccec167be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fcceb7fe700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccec154240 0x7fccec167be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fcceb7fe700 1 -- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccec0a9d20 msgr2=0x7fccec0c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fcceb7fe700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccec0a9d20 0x7fccec0c4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fcceb7fe700 1 -- 172.21.15.29:0/2025746312 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce00092a0 con 0x7fccec157cf0 2024-01-24T15:32:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fcceb7fe700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec157cf0 0x7fccec0ac1a0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 rx=0x7fcce400c730 tx=0x7fcce4001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:15.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fccd2ffd700 1 -- 172.21.15.29:0/2025746312 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce4005d30 con 0x7fccec157cf0 2024-01-24T15:32:15.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fccd2ffd700 1 -- 172.21.15.29:0/2025746312 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce4005e90 con 0x7fccec157cf0 2024-01-24T15:32:15.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fccf1384700 1 -- 172.21.15.29:0/2025746312 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec168710 con 0x7fccec157cf0 2024-01-24T15:32:15.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fccd2ffd700 1 -- 172.21.15.29:0/2025746312 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce4007300 con 0x7fccec157cf0 2024-01-24T15:32:15.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.264+0000 7fccf1384700 1 -- 172.21.15.29:0/2025746312 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec1680d0 con 0x7fccec157cf0 2024-01-24T15:32:15.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.266+0000 7fccf1384700 1 -- 172.21.15.29:0/2025746312 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccec0af590 con 0x7fccec157cf0 2024-01-24T15:32:15.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.266+0000 7fccd2ffd700 1 -- 172.21.15.29:0/2025746312 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcce40076b0 con 0x7fccec157cf0 2024-01-24T15:32:15.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.266+0000 7fccd2ffd700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fccd805f240 0x7fccd8061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:15.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.267+0000 7fccd2ffd700 1 -- 172.21.15.29:0/2025746312 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6964+0+0 (secure 0 0 0) 0x7fcce407a180 con 0x7fccec157cf0 2024-01-24T15:32:15.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.267+0000 7fcceaffd700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fccd805f240 0x7fccd8061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:15.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.267+0000 7fcceaffd700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fccd805f240 0x7fccd8061700 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fccec0acd20 tx=0x7fccdc008040).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:15.273 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.271+0000 7fccd2ffd700 1 -- 172.21.15.29:0/2025746312 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcce404af80 con 0x7fccec157cf0 2024-01-24T15:32:15.470 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:15.468+0000 7fccf1384700 1 -- 172.21.15.29:0/2025746312 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fccec0475e0 con 0x7fccec157cf0 2024-01-24T15:32:16.242 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.240+0000 7fccd2ffd700 1 -- 172.21.15.29:0/2025746312 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v227) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcce4018020 con 0x7fccec157cf0 2024-01-24T15:32:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.243+0000 7fccf1384700 1 -- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fccd805f240 msgr2=0x7fccd8061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.243+0000 7fccf1384700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fccd805f240 0x7fccd8061700 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fccec0acd20 tx=0x7fccdc008040).stop 2024-01-24T15:32:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.243+0000 7fccf1384700 1 -- 172.21.15.29:0/2025746312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec157cf0 msgr2=0x7fccec0ac1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.243+0000 7fccf1384700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec157cf0 0x7fccec0ac1a0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 rx=0x7fcce400c730 tx=0x7fcce4001520).stop 2024-01-24T15:32:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.243+0000 7fccf1384700 1 -- 172.21.15.29:0/2025746312 shutdown_connections 2024-01-24T15:32:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.243+0000 7fccf1384700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccec157cf0 0x7fccec0ac1a0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.243+0000 7fccf1384700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccec0a9d20 0x7fccec0c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.243+0000 7fccf1384700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fccd805f240 0x7fccd8061700 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.243+0000 7fccf1384700 1 --2- 172.21.15.29:0/2025746312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccec154240 0x7fccec167be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.243+0000 7fccf1384700 1 -- 172.21.15.29:0/2025746312 >> 172.21.15.29:0/2025746312 conn(0x7fccec0c8310 msgr2=0x7fccec155600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.243+0000 7fccf1384700 1 -- 172.21.15.29:0/2025746312 shutdown_connections 2024-01-24T15:32:16.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.243+0000 7fccf1384700 1 -- 172.21.15.29:0/2025746312 wait complete. 2024-01-24T15:32:16.247 INFO:teuthology.orchestra.run.smithi029.stderr: removed all blocklist entries 2024-01-24T15:32:16.261 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-24T15:32:16.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.394+0000 7f89a92c0700 1 Processor -- start 2024-01-24T15:32:16.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.394+0000 7f89a92c0700 1 -- start start 2024-01-24T15:32:16.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.394+0000 7f89a92c0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f89a414aac0 0x7f89a40654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:16.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a92c0700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a4151920 con 0x7f89a40c2040 2024-01-24T15:32:16.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a92c0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a4150d10 con 0x7f89a4065990 2024-01-24T15:32:16.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a92c0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a4150b00 con 0x7f89a414aac0 2024-01-24T15:32:16.398 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a2ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f89a414aac0 0x7f89a40654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a37fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f89a40c2040 0x7f89a4068860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56794/0 (socket says 172.21.15.29:56794) 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a37fe700 1 -- 172.21.15.29:0/2847958470 learned_addr learned my addr 172.21.15.29:0/2847958470 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a2ffd700 1 -- 172.21.15.29:0/2847958470 >> v1:172.21.15.163:6789/0 conn(0x7f89a4065990 legacy=0x7f89a40c1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a2ffd700 1 -- 172.21.15.29:0/2847958470 >> v1:172.21.15.29:6789/0 conn(0x7f89a40c2040 legacy=0x7f89a4068860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a2ffd700 1 -- 172.21.15.29:0/2847958470 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a4150ff0 con 0x7f89a414aac0 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a1ffb700 1 -- 172.21.15.29:0/2847958470 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705681779 0 0) 0x7f89a4151920 con 0x7f89a40c2040 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a2ffd700 1 --2- 172.21.15.29:0/2847958470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89a414aac0 0x7f89a40654a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 rx=0x7f899400ba00 tx=0x7f899400b300).ready entity=mon.0 client_cookie=2de8f3c47a14b6b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a1ffb700 1 -- 172.21.15.29:0/2847958470 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899400da70 con 0x7f89a414aac0 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a1ffb700 1 -- 172.21.15.29:0/2847958470 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899400dbd0 con 0x7f89a414aac0 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.395+0000 7f89a1ffb700 1 -- 172.21.15.29:0/2847958470 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994010470 con 0x7f89a414aac0 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.396+0000 7f89a92c0700 1 -- 172.21.15.29:0/2847958470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89a414aac0 msgr2=0x7f89a40654a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.396+0000 7f89a92c0700 1 --2- 172.21.15.29:0/2847958470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89a414aac0 0x7f89a40654a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 rx=0x7f899400ba00 tx=0x7f899400b300).stop 2024-01-24T15:32:16.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.396+0000 7f89a92c0700 1 -- 172.21.15.29:0/2847958470 shutdown_connections 2024-01-24T15:32:16.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.396+0000 7f89a92c0700 1 --2- 172.21.15.29:0/2847958470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89a414aac0 0x7f89a40654a0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.396+0000 7f89a92c0700 1 -- 172.21.15.29:0/2847958470 >> 172.21.15.29:0/2847958470 conn(0x7f89a40c69f0 msgr2=0x7f89a40c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:16.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.396+0000 7f89a92c0700 1 -- 172.21.15.29:0/2847958470 shutdown_connections 2024-01-24T15:32:16.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.396+0000 7f89a92c0700 1 -- 172.21.15.29:0/2847958470 wait complete. 2024-01-24T15:32:16.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.397+0000 7f89a92c0700 1 Processor -- start 2024-01-24T15:32:16.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.398+0000 7f89a92c0700 1 -- start start 2024-01-24T15:32:16.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.398+0000 7f89a92c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89a40c2040 0x7f89a41491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:16.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.398+0000 7f89a92c0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89a414aac0 0x7f89a41521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:16.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.398+0000 7f89a92c0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89a4147c30 0x7f89a41526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.398+0000 7f89a92c0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a4152ba0 con 0x7f89a414aac0 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.398+0000 7f89a92c0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40ab480 con 0x7f89a40c2040 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.398+0000 7f89a92c0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40aab50 con 0x7f89a4147c30 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.398+0000 7f89a27fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89a414aac0 0x7f89a41521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f89a27fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89a414aac0 0x7f89a41521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53362/0 (socket says 172.21.15.29:53362) 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f89a27fc700 1 -- 172.21.15.29:0/4269936314 learned_addr learned my addr 172.21.15.29:0/4269936314 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f89a2ffd700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89a40c2040 0x7f89a41491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f89a37fe700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89a4147c30 0x7f89a41526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f89a27fc700 1 -- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89a4147c30 msgr2=0x7f89a41526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f89a27fc700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89a4147c30 0x7f89a41526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f89a27fc700 1 -- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89a40c2040 msgr2=0x7f89a41491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f89a27fc700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89a40c2040 0x7f89a41491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f89a27fc700 1 -- 172.21.15.29:0/4269936314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899400b540 con 0x7f89a414aac0 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f89a27fc700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89a414aac0 0x7f89a41521c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 rx=0x7f899800c4c0 tx=0x7f8998007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:16.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f898affd700 1 -- 172.21.15.29:0/4269936314 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899800ce70 con 0x7f89a414aac0 2024-01-24T15:32:16.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f898affd700 1 -- 172.21.15.29:0/4269936314 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8998001e90 con 0x7f89a414aac0 2024-01-24T15:32:16.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f89a92c0700 1 -- 172.21.15.29:0/4269936314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a40aae30 con 0x7f89a414aac0 2024-01-24T15:32:16.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f898affd700 1 -- 172.21.15.29:0/4269936314 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998007250 con 0x7f89a414aac0 2024-01-24T15:32:16.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.399+0000 7f89a92c0700 1 -- 172.21.15.29:0/4269936314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a41469b0 con 0x7f89a414aac0 2024-01-24T15:32:16.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.400+0000 7f89a92c0700 1 -- 172.21.15.29:0/4269936314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a40ab610 con 0x7f89a414aac0 2024-01-24T15:32:16.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.401+0000 7f898affd700 1 -- 172.21.15.29:0/4269936314 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f89980066a0 con 0x7f89a414aac0 2024-01-24T15:32:16.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.401+0000 7f898affd700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f899005f270 0x7f8990061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:16.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.401+0000 7f898affd700 1 -- 172.21.15.29:0/4269936314 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 6362+0+0 (secure 0 0 0) 0x7f8998079d30 con 0x7f89a414aac0 2024-01-24T15:32:16.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.402+0000 7f89a2ffd700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f899005f270 0x7f8990061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:16.405 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.402+0000 7f89a2ffd700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f899005f270 0x7f8990061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f89940105a0 tx=0x7f899400b300).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:16.408 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.406+0000 7f898affd700 1 -- 172.21.15.29:0/4269936314 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f899804ad90 con 0x7f89a414aac0 2024-01-24T15:32:16.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.655+0000 7f89a92c0700 1 -- 172.21.15.29:0/4269936314 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f89a41512d0 con 0x7f89a414aac0 2024-01-24T15:32:16.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.656+0000 7f898affd700 1 -- 172.21.15.29:0/4269936314 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v53) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f899804e3b0 con 0x7f89a414aac0 2024-01-24T15:32:16.658 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:16.658 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:32:16.658 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_dump": [ 2024-01-24T15:32:16.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.658 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.a", 2024-01-24T15:32:16.658 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fla6ydAxAA6iWiyZDvC8oA/ACSno51Ew==", 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.b", 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl/sAdERAA9lCJxUFfSMY4TuPGmCKndQ==", 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:32:16.659 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.c", 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FliCrCCxAAC/Va7UzT59huFNjo/SRWNw==", 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mds.d", 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlyJZrGBAAU2LOvjVM8N9QMZv88hJPyQ==", 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.0", 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBLF7FlADzBABAA70GXCSJTffEe/urId76AKQ==", 2024-01-24T15:32:16.661 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.1", 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBNF7Flfel7OhAASaKRQHw7eH9kQM5hEBaWdA==", 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.2", 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBQF7Flk00rORAA3rKR6ndTkJPbyteV7hvjyw==", 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.3", 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBTF7FlHwPoNxAAah69kOdpkGsfLUif/wNdgQ==", 2024-01-24T15:32:16.663 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.4", 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBZF7FlY2SJOhAAxOWfPBO1iOYjznsTedlHLw==", 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.664 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.5", 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBcF7FlVRxfOBAADEN0vHLU1hDQc+XitD9vUQ==", 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.6", 2024-01-24T15:32:16.665 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBfF7FlFPt6NhAAVoSi7O0FdAooJDR5ns446A==", 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "osd.7", 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBiF7Fl4jrGMxAAHrHnkeXkCczyiq9FS23rZQ==", 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow profile osd", 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile osd", 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.0", 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FlTR0zHRAAPY9fu5fzucXWTmOemllanw==", 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow r", 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.1", 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FleobpJBAA2E1mUoSBMLy9i0SmuNXjtA==", 2024-01-24T15:32:16.667 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.2", 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7Fl7gXYIBAAqZHybP7+GxyljviKaYkc3g==", 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T15:32:16.668 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.3", 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBJF7FllRZnKhAACCNCjyz4Wbuf+yCPbS+4xQ==", 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow", 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow r", 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow rw", 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow rwx" 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "client.admin", 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlxK1bFxAAZpfJn+Tcpmq7R9Yq29FvVg==", 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": "allow *", 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow *", 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.x", 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlRpdCOxAAFUk7jsOd9y4FdsXAfkiamw==", 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T15:32:16.670 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: "entity": "mgr.y", 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "AQBIF7FlSn0kNxAA4oZPRORJZ0VGtOzn/zy9yA==", 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": { 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": "allow *", 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: "mon": "allow profile mgr", 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": "allow *" 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-01-24T15:32:16.671 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:32:16.672 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.658+0000 7f89a92c0700 1 -- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f899005f270 msgr2=0x7f8990061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.658+0000 7f89a92c0700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f899005f270 0x7f8990061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f89940105a0 tx=0x7f899400b300).stop 2024-01-24T15:32:16.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.658+0000 7f89a92c0700 1 -- 172.21.15.29:0/4269936314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89a414aac0 msgr2=0x7f89a41521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.658+0000 7f89a92c0700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89a414aac0 0x7f89a41521c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 rx=0x7f899800c4c0 tx=0x7f8998007e00).stop 2024-01-24T15:32:16.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.658+0000 7f89a92c0700 1 -- 172.21.15.29:0/4269936314 shutdown_connections 2024-01-24T15:32:16.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.658+0000 7f89a92c0700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f899005f270 0x7f8990061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.658+0000 7f89a92c0700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89a4147c30 0x7f89a41526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.658+0000 7f89a92c0700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89a40c2040 0x7f89a41491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.658+0000 7f89a92c0700 1 --2- 172.21.15.29:0/4269936314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89a414aac0 0x7f89a41521c0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.658+0000 7f89a92c0700 1 -- 172.21.15.29:0/4269936314 >> 172.21.15.29:0/4269936314 conn(0x7f89a40c69f0 msgr2=0x7f89a40c0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:16.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.658+0000 7f89a92c0700 1 -- 172.21.15.29:0/4269936314 shutdown_connections 2024-01-24T15:32:16.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.658+0000 7f89a92c0700 1 -- 172.21.15.29:0/4269936314 wait complete. 2024-01-24T15:32:16.674 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:32:16.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.803+0000 7f3321d4c700 1 Processor -- start 2024-01-24T15:32:16.805 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.803+0000 7f3321d4c700 1 -- start start 2024-01-24T15:32:16.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f3321d4c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f331c13b640 0x7f331c152f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:16.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f3321d4c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f331c0c4db0 0x7f331c0664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:16.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f3321d4c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331c155dc0 con 0x7f331c13ba20 2024-01-24T15:32:16.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f3321d4c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331c155a90 con 0x7f331c0c4db0 2024-01-24T15:32:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f3321d4c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331c0c3250 con 0x7f331c13b640 2024-01-24T15:32:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f331c0c4db0 0x7f331c0664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331affd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f331c13ba20 0x7f331c156250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56798/0 (socket says 172.21.15.29:56798) 2024-01-24T15:32:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331affd700 1 -- 172.21.15.29:0/2566146389 learned_addr learned my addr 172.21.15.29:0/2566146389 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331b7fe700 1 --2- 172.21.15.29:0/2566146389 >> v2:172.21.15.163:3300/0 conn(0x7f331c13b640 0x7f331c152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331bfff700 1 -- 172.21.15.29:0/2566146389 >> v2:172.21.15.163:3300/0 conn(0x7f331c13b640 msgr2=0x7f331c152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331bfff700 1 --2- 172.21.15.29:0/2566146389 >> v2:172.21.15.163:3300/0 conn(0x7f331c13b640 0x7f331c152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331bfff700 1 -- 172.21.15.29:0/2566146389 >> v1:172.21.15.29:6789/0 conn(0x7f331c13ba20 legacy=0x7f331c156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331bfff700 1 -- 172.21.15.29:0/2566146389 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c0c3530 con 0x7f331c0c4db0 2024-01-24T15:32:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331a7fc700 1 -- 172.21.15.29:0/2566146389 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801582099 0 0) 0x7f331c155dc0 con 0x7f331c13ba20 2024-01-24T15:32:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331bfff700 1 --2- 172.21.15.29:0/2566146389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f331c0c4db0 0x7f331c0664f0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 rx=0x7f331c0c8ac0 tx=0x7f3314009780).ready entity=mon.0 client_cookie=11b60f02c84f33b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:16.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331a7fc700 1 -- 172.21.15.29:0/2566146389 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331400e070 con 0x7f331c0c4db0 2024-01-24T15:32:16.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331a7fc700 1 -- 172.21.15.29:0/2566146389 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331400bc30 con 0x7f331c0c4db0 2024-01-24T15:32:16.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.804+0000 7f331a7fc700 1 -- 172.21.15.29:0/2566146389 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3314012410 con 0x7f331c0c4db0 2024-01-24T15:32:16.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.805+0000 7f3321d4c700 1 -- 172.21.15.29:0/2566146389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f331c0c4db0 msgr2=0x7f331c0664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.805+0000 7f3321d4c700 1 --2- 172.21.15.29:0/2566146389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f331c0c4db0 0x7f331c0664f0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 rx=0x7f331c0c8ac0 tx=0x7f3314009780).stop 2024-01-24T15:32:16.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.805+0000 7f3321d4c700 1 -- 172.21.15.29:0/2566146389 shutdown_connections 2024-01-24T15:32:16.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.805+0000 7f3321d4c700 1 --2- 172.21.15.29:0/2566146389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f331c0c4db0 0x7f331c0664f0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.805+0000 7f3321d4c700 1 --2- 172.21.15.29:0/2566146389 >> v2:172.21.15.163:3300/0 conn(0x7f331c13b640 0x7f331c152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.805+0000 7f3321d4c700 1 -- 172.21.15.29:0/2566146389 >> 172.21.15.29:0/2566146389 conn(0x7f331c0c8bb0 msgr2=0x7f331c0c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:16.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.805+0000 7f3321d4c700 1 -- 172.21.15.29:0/2566146389 shutdown_connections 2024-01-24T15:32:16.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.805+0000 7f3321d4c700 1 -- 172.21.15.29:0/2566146389 wait complete. 2024-01-24T15:32:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.807+0000 7f3321d4c700 1 Processor -- start 2024-01-24T15:32:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.807+0000 7f3321d4c700 1 -- start start 2024-01-24T15:32:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.808+0000 7f3321d4c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f331c0c4db0 0x7f331c0ab310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.808+0000 7f3321d4c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f331c13b640 0x7f331c0ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.808+0000 7f3321d4c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f331c0adf40 0x7f331c138b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.808+0000 7f3321d4c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c0a9090 con 0x7f331c0c4db0 2024-01-24T15:32:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.808+0000 7f3321d4c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c0ae3c0 con 0x7f331c0adf40 2024-01-24T15:32:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.808+0000 7f3321d4c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331c0ae6a0 con 0x7f331c13b640 2024-01-24T15:32:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.808+0000 7f331b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f331c0c4db0 0x7f331c0ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.808+0000 7f331b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f331c0c4db0 0x7f331c0ab310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53388/0 (socket says 172.21.15.29:53388) 2024-01-24T15:32:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.808+0000 7f331b7fe700 1 -- 172.21.15.29:0/3181478903 learned_addr learned my addr 172.21.15.29:0/3181478903 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:16.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.808+0000 7f331affd700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f331c13b640 0x7f331c0ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.808+0000 7f331bfff700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f331c0adf40 0x7f331c138b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.809+0000 7f331b7fe700 1 -- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f331c13b640 msgr2=0x7f331c0ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.809+0000 7f331b7fe700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f331c13b640 0x7f331c0ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.809+0000 7f331b7fe700 1 -- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f331c0adf40 msgr2=0x7f331c138b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.809+0000 7f331b7fe700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f331c0adf40 0x7f331c138b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.809+0000 7f331b7fe700 1 -- 172.21.15.29:0/3181478903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3314009a20 con 0x7f331c0c4db0 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.809+0000 7f331bfff700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f331c0adf40 0x7f331c138b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.809+0000 7f331b7fe700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f331c0c4db0 0x7f331c0ab310 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 rx=0x7f330c00d890 tx=0x7f330c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.809+0000 7f33037fe700 1 -- 172.21.15.29:0/3181478903 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c0125f0 con 0x7f331c0c4db0 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.809+0000 7f33037fe700 1 -- 172.21.15.29:0/3181478903 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330c012750 con 0x7f331c0c4db0 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.809+0000 7f33037fe700 1 -- 172.21.15.29:0/3181478903 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c012ad0 con 0x7f331c0c4db0 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.809+0000 7f3321d4c700 1 -- 172.21.15.29:0/3181478903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331c0ae980 con 0x7f331c0c4db0 2024-01-24T15:32:16.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.809+0000 7f3321d4c700 1 -- 172.21.15.29:0/3181478903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331c0a8770 con 0x7f331c0c4db0 2024-01-24T15:32:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.810+0000 7f3321d4c700 1 -- 172.21.15.29:0/3181478903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331c0a9370 con 0x7f331c0c4db0 2024-01-24T15:32:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.811+0000 7f33037fe700 1 -- 172.21.15.29:0/3181478903 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f330c013480 con 0x7f331c0c4db0 2024-01-24T15:32:16.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.811+0000 7f33037fe700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f330805f270 0x7f3308061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.811+0000 7f33037fe700 1 -- 172.21.15.29:0/3181478903 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 6362+0+0 (secure 0 0 0) 0x7f330c07a260 con 0x7f331c0c4db0 2024-01-24T15:32:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.811+0000 7f331affd700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f330805f270 0x7f3308061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:16.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.812+0000 7f331affd700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f330805f270 0x7f3308061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f331c0acd30 tx=0x7f3310009040).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:16.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.815+0000 7f33037fe700 1 -- 172.21.15.29:0/3181478903 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f330c04b260 con 0x7f331c0c4db0 2024-01-24T15:32:17.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.998+0000 7f3321d4c700 1 -- 172.21.15.29:0/3181478903 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f331c0a8ee0 con 0x7f331c0c4db0 2024-01-24T15:32:17.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:16.999+0000 7f33037fe700 1 -- 172.21.15.29:0/3181478903 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v227) v1 ==== 74+0+15082 (secure 0 0 0) 0x7f330c04e880 con 0x7f331c0c4db0 2024-01-24T15:32:17.001 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:17.001 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":227,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:32:16.238244+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:32:17.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.001+0000 7f3321d4c700 1 -- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f330805f270 msgr2=0x7f3308061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.001+0000 7f3321d4c700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f330805f270 0x7f3308061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f331c0acd30 tx=0x7f3310009040).stop 2024-01-24T15:32:17.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.001+0000 7f3321d4c700 1 -- 172.21.15.29:0/3181478903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f331c0c4db0 msgr2=0x7f331c0ab310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.001+0000 7f3321d4c700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f331c0c4db0 0x7f331c0ab310 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 rx=0x7f330c00d890 tx=0x7f330c008040).stop 2024-01-24T15:32:17.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.001+0000 7f3321d4c700 1 -- 172.21.15.29:0/3181478903 shutdown_connections 2024-01-24T15:32:17.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.001+0000 7f3321d4c700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f331c0adf40 0x7f331c138b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.001+0000 7f3321d4c700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f331c0c4db0 0x7f331c0ab310 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.001+0000 7f3321d4c700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f330805f270 0x7f3308061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.001+0000 7f3321d4c700 1 --2- 172.21.15.29:0/3181478903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f331c13b640 0x7f331c0ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.001+0000 7f3321d4c700 1 -- 172.21.15.29:0/3181478903 >> 172.21.15.29:0/3181478903 conn(0x7f331c0c8bb0 msgr2=0x7f331c0be520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:17.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.001+0000 7f3321d4c700 1 -- 172.21.15.29:0/3181478903 shutdown_connections 2024-01-24T15:32:17.004 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.001+0000 7f3321d4c700 1 -- 172.21.15.29:0/3181478903 wait complete. 2024-01-24T15:32:17.017 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-01-24T15:32:17.017 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T15:32:17.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.148+0000 7ff9e74ad700 1 Processor -- start 2024-01-24T15:32:17.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.148+0000 7ff9e74ad700 1 -- start start 2024-01-24T15:32:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.149+0000 7ff9e74ad700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e015c830 0x7ff9e00627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.149+0000 7ff9e74ad700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e0063fe0 con 0x7ff9e0156110 2024-01-24T15:32:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.149+0000 7ff9e74ad700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e015c3a0 con 0x7ff9e015ccb0 2024-01-24T15:32:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.149+0000 7ff9e74ad700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e01564f0 con 0x7ff9e015c830 2024-01-24T15:32:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.149+0000 7ff9e5249700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff9e0156110 0x7ff9e0158a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56806/0 (socket says 172.21.15.29:56806) 2024-01-24T15:32:17.153 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.149+0000 7ff9e5249700 1 -- 172.21.15.29:0/2548495631 learned_addr learned my addr 172.21.15.29:0/2548495631 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9e4a48700 1 --2- 172.21.15.29:0/2548495631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e015c830 0x7ff9e00627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9d3fff700 1 -- 172.21.15.29:0/2548495631 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004480186 0 0) 0x7ff9e0063fe0 con 0x7ff9e0156110 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9d3fff700 1 -- 172.21.15.29:0/2548495631 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9cc0033f0 con 0x7ff9e0156110 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9d3fff700 1 -- 172.21.15.29:0/2548495631 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252853388 0 0) 0x7ff9e015c3a0 con 0x7ff9e015ccb0 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9d3fff700 1 -- 172.21.15.29:0/2548495631 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e0063fe0 con 0x7ff9e015ccb0 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9d3fff700 1 -- 172.21.15.29:0/2548495631 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2119718361 0 0) 0x7ff9cc0033f0 con 0x7ff9e0156110 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9e4a48700 1 -- 172.21.15.29:0/2548495631 >> v1:172.21.15.163:6789/0 conn(0x7ff9e015ccb0 legacy=0x7ff9e0064b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9e4a48700 1 -- 172.21.15.29:0/2548495631 >> v1:172.21.15.29:6789/0 conn(0x7ff9e0156110 legacy=0x7ff9e0158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9e4a48700 1 -- 172.21.15.29:0/2548495631 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0159320 con 0x7ff9e015c830 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9e4a48700 1 --2- 172.21.15.29:0/2548495631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e015c830 0x7ff9e00627b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7ff9dc009790 tx=0x7ff9dc009060).ready entity=mon.2 client_cookie=2f20fff68285d465 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9d3fff700 1 -- 172.21.15.29:0/2548495631 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4001470 con 0x7ff9e015c830 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9d3fff700 1 -- 172.21.15.29:0/2548495631 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc011400 con 0x7ff9e015c830 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.150+0000 7ff9d3fff700 1 -- 172.21.15.29:0/2548495631 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc011780 con 0x7ff9e015c830 2024-01-24T15:32:17.154 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.151+0000 7ff9e74ad700 1 -- 172.21.15.29:0/2548495631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e015c830 msgr2=0x7ff9e00627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.151+0000 7ff9e74ad700 1 --2- 172.21.15.29:0/2548495631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e015c830 0x7ff9e00627b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7ff9dc009790 tx=0x7ff9dc009060).stop 2024-01-24T15:32:17.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.151+0000 7ff9e74ad700 1 -- 172.21.15.29:0/2548495631 shutdown_connections 2024-01-24T15:32:17.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.151+0000 7ff9e74ad700 1 --2- 172.21.15.29:0/2548495631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e015c830 0x7ff9e00627b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.151+0000 7ff9e74ad700 1 -- 172.21.15.29:0/2548495631 >> 172.21.15.29:0/2548495631 conn(0x7ff9e00c7260 msgr2=0x7ff9e00c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:17.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.151+0000 7ff9e74ad700 1 -- 172.21.15.29:0/2548495631 shutdown_connections 2024-01-24T15:32:17.155 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.151+0000 7ff9e74ad700 1 -- 172.21.15.29:0/2548495631 wait complete. 2024-01-24T15:32:17.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.152+0000 7ff9e74ad700 1 Processor -- start 2024-01-24T15:32:17.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.152+0000 7ff9e74ad700 1 -- start start 2024-01-24T15:32:17.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.153+0000 7ff9e74ad700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9e0156110 0x7ff9e01526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.153+0000 7ff9e74ad700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e015c830 0x7ff9e0153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.153+0000 7ff9e74ad700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e015ccb0 0x7ff9e00c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.157 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.153+0000 7ff9e74ad700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00bf060 con 0x7ff9e015ccb0 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.153+0000 7ff9e74ad700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c6340 con 0x7ff9e0156110 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.153+0000 7ff9e74ad700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00bf2b0 con 0x7ff9e015c830 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.153+0000 7ff9e5a4a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e015ccb0 0x7ff9e00c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.153+0000 7ff9e5a4a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e015ccb0 0x7ff9e00c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53390/0 (socket says 172.21.15.29:53390) 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.153+0000 7ff9e5a4a700 1 -- 172.21.15.29:0/915067878 learned_addr learned my addr 172.21.15.29:0/915067878 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9e4a48700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e015c830 0x7ff9e0153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9e5249700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9e0156110 0x7ff9e01526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9e5a4a700 1 -- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e015c830 msgr2=0x7ff9e0153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9e5a4a700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e015c830 0x7ff9e0153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9e5a4a700 1 -- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9e0156110 msgr2=0x7ff9e01526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9e5a4a700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9e0156110 0x7ff9e01526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9e5a4a700 1 -- 172.21.15.29:0/915067878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc0092a0 con 0x7ff9e015ccb0 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9e5a4a700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e015ccb0 0x7ff9e00c67d0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 rx=0x7ff9d800c7c0 tx=0x7ff9d800cad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:17.158 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9d17fa700 1 -- 172.21.15.29:0/915067878 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8002020 con 0x7ff9e015ccb0 2024-01-24T15:32:17.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9d17fa700 1 -- 172.21.15.29:0/915067878 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d8002180 con 0x7ff9e015ccb0 2024-01-24T15:32:17.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9d17fa700 1 -- 172.21.15.29:0/915067878 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8001590 con 0x7ff9e015ccb0 2024-01-24T15:32:17.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9e74ad700 1 -- 172.21.15.29:0/915067878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e00bf590 con 0x7ff9e015ccb0 2024-01-24T15:32:17.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.154+0000 7ff9e74ad700 1 -- 172.21.15.29:0/915067878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e00bfb50 con 0x7ff9e015ccb0 2024-01-24T15:32:17.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.155+0000 7ff9e74ad700 1 -- 172.21.15.29:0/915067878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e00c64b0 con 0x7ff9e015ccb0 2024-01-24T15:32:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.156+0000 7ff9d17fa700 1 -- 172.21.15.29:0/915067878 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff9d8001980 con 0x7ff9e015ccb0 2024-01-24T15:32:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.156+0000 7ff9d17fa700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff9c405f270 0x7ff9c4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.156+0000 7ff9d17fa700 1 -- 172.21.15.29:0/915067878 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 6362+0+0 (secure 0 0 0) 0x7ff9d8079ad0 con 0x7ff9e015ccb0 2024-01-24T15:32:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.156+0000 7ff9e5249700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff9c405f270 0x7ff9c4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.160 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.157+0000 7ff9e5249700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff9c405f270 0x7ff9c4061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff9d40011e0 tx=0x7ff9d4000bc0).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:17.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.161+0000 7ff9d17fa700 1 -- 172.21.15.29:0/915067878 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff9d8047b00 con 0x7ff9e015ccb0 2024-01-24T15:32:17.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.349+0000 7ff9e74ad700 1 -- 172.21.15.29:0/915067878 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff9e00475e0 con 0x7ff9e015ccb0 2024-01-24T15:32:17.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.350+0000 7ff9d17fa700 1 -- 172.21.15.29:0/915067878 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v227) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff9d804aad0 con 0x7ff9e015ccb0 2024-01-24T15:32:17.353 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T15:32:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.352+0000 7ff9e74ad700 1 -- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff9c405f270 msgr2=0x7ff9c4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.352+0000 7ff9e74ad700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff9c405f270 0x7ff9c4061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff9d40011e0 tx=0x7ff9d4000bc0).stop 2024-01-24T15:32:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.352+0000 7ff9e74ad700 1 -- 172.21.15.29:0/915067878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e015ccb0 msgr2=0x7ff9e00c67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.352+0000 7ff9e74ad700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e015ccb0 0x7ff9e00c67d0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 rx=0x7ff9d800c7c0 tx=0x7ff9d800cad0).stop 2024-01-24T15:32:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.352+0000 7ff9e74ad700 1 -- 172.21.15.29:0/915067878 shutdown_connections 2024-01-24T15:32:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.352+0000 7ff9e74ad700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9e015ccb0 0x7ff9e00c67d0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.352+0000 7ff9e74ad700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9e0156110 0x7ff9e01526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.352+0000 7ff9e74ad700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff9c405f270 0x7ff9c4061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.352+0000 7ff9e74ad700 1 --2- 172.21.15.29:0/915067878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9e015c830 0x7ff9e0153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.352+0000 7ff9e74ad700 1 -- 172.21.15.29:0/915067878 >> 172.21.15.29:0/915067878 conn(0x7ff9e00c7260 msgr2=0x7ff9e015a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:17.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.352+0000 7ff9e74ad700 1 -- 172.21.15.29:0/915067878 shutdown_connections 2024-01-24T15:32:17.355 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.352+0000 7ff9e74ad700 1 -- 172.21.15.29:0/915067878 wait complete. 2024-01-24T15:32:17.367 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T15:32:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.499+0000 7f5412de5700 1 Processor -- start 2024-01-24T15:32:17.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.499+0000 7f5412de5700 1 -- start start 2024-01-24T15:32:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.500+0000 7f5412de5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f540c0c8270 0x7f540c06a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.500+0000 7f5412de5700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540c0bcda0 con 0x7f540c062ec0 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.500+0000 7f5412de5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540c065f80 con 0x7f540c1384f0 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.500+0000 7f5412de5700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c066260 con 0x7f540c0c8270 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.500+0000 7f540bfff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f540c062ec0 0x7f540c1391d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56810/0 (socket says 172.21.15.29:56810) 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.500+0000 7f540bfff700 1 -- 172.21.15.29:0/495488578 learned_addr learned my addr 172.21.15.29:0/495488578 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.501+0000 7f5411382700 1 --2- 172.21.15.29:0/495488578 >> v2:172.21.15.163:3300/0 conn(0x7f540c0c8270 0x7f540c06a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.501+0000 7f540b7fe700 1 -- 172.21.15.29:0/495488578 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043680049 0 0) 0x7f540c0bcda0 con 0x7f540c062ec0 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.501+0000 7f540b7fe700 1 -- 172.21.15.29:0/495488578 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53fc0033f0 con 0x7f540c062ec0 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.501+0000 7f5411382700 1 -- 172.21.15.29:0/495488578 >> v1:172.21.15.163:6789/0 conn(0x7f540c1384f0 legacy=0x7f540c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.501+0000 7f5411382700 1 -- 172.21.15.29:0/495488578 >> v1:172.21.15.29:6789/0 conn(0x7f540c062ec0 legacy=0x7f540c1391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.501+0000 7f540b7fe700 1 -- 172.21.15.29:0/495488578 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3947468121 0 0) 0x7f53fc0033f0 con 0x7f540c062ec0 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.501+0000 7f5411382700 1 -- 172.21.15.29:0/495488578 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c0bca50 con 0x7f540c0c8270 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.501+0000 7f5411382700 1 --2- 172.21.15.29:0/495488578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f540c0c8270 0x7f540c06a460 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f5404009e60 tx=0x7f5404009780).ready entity=mon.1 client_cookie=592c482befbcc2c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.501+0000 7f540b7fe700 1 -- 172.21.15.29:0/495488578 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540400e070 con 0x7f540c0c8270 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.501+0000 7f540b7fe700 1 -- 172.21.15.29:0/495488578 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540400b850 con 0x7f540c0c8270 2024-01-24T15:32:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.501+0000 7f540b7fe700 1 -- 172.21.15.29:0/495488578 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8001470 con 0x7f540c0c8270 2024-01-24T15:32:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.502+0000 7f5412de5700 1 -- 172.21.15.29:0/495488578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f540c0c8270 msgr2=0x7f540c06a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.502+0000 7f5412de5700 1 --2- 172.21.15.29:0/495488578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f540c0c8270 0x7f540c06a460 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f5404009e60 tx=0x7f5404009780).stop 2024-01-24T15:32:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.502+0000 7f5412de5700 1 -- 172.21.15.29:0/495488578 shutdown_connections 2024-01-24T15:32:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.502+0000 7f5412de5700 1 --2- 172.21.15.29:0/495488578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f540c0c8270 0x7f540c06a460 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.502+0000 7f5412de5700 1 -- 172.21.15.29:0/495488578 >> 172.21.15.29:0/495488578 conn(0x7f540c064390 msgr2=0x7f540c0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.502+0000 7f5412de5700 1 -- 172.21.15.29:0/495488578 shutdown_connections 2024-01-24T15:32:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.502+0000 7f5412de5700 1 -- 172.21.15.29:0/495488578 wait complete. 2024-01-24T15:32:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.503+0000 7f5412de5700 1 Processor -- start 2024-01-24T15:32:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.504+0000 7f5412de5700 1 -- start start 2024-01-24T15:32:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.504+0000 7f5412de5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f540c062ec0 0x7f540c156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.504+0000 7f5412de5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f540c0c8270 0x7f540c1674a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.504+0000 7f5412de5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f540c1384f0 0x7f540c167990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.504+0000 7f5412de5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c167e80 con 0x7f540c062ec0 2024-01-24T15:32:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.504+0000 7f5412de5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c0c2bf0 con 0x7f540c1384f0 2024-01-24T15:32:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.504+0000 7f5412de5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c0c25e0 con 0x7f540c0c8270 2024-01-24T15:32:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.504+0000 7f5410b81700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f540c062ec0 0x7f540c156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.504+0000 7f5410b81700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f540c062ec0 0x7f540c156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53400/0 (socket says 172.21.15.29:53400) 2024-01-24T15:32:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.504+0000 7f5410b81700 1 -- 172.21.15.29:0/1767810594 learned_addr learned my addr 172.21.15.29:0/1767810594 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f540bfff700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f540c0c8270 0x7f540c1674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5411382700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f540c1384f0 0x7f540c167990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.508 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5410b81700 1 -- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f540c0c8270 msgr2=0x7f540c1674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5410b81700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f540c0c8270 0x7f540c1674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5410b81700 1 -- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f540c1384f0 msgr2=0x7f540c167990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5410b81700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f540c1384f0 0x7f540c167990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5410b81700 1 -- 172.21.15.29:0/1767810594 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5404009a20 con 0x7f540c062ec0 2024-01-24T15:32:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5411382700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f540c1384f0 0x7f540c167990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5410b81700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f540c062ec0 0x7f540c156c40 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 rx=0x7f540000c440 tx=0x7f5400007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5408ff9700 1 -- 172.21.15.29:0/1767810594 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540000ce40 con 0x7f540c062ec0 2024-01-24T15:32:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5408ff9700 1 -- 172.21.15.29:0/1767810594 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5400007020 con 0x7f540c062ec0 2024-01-24T15:32:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5412de5700 1 -- 172.21.15.29:0/1767810594 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540c168080 con 0x7f540c062ec0 2024-01-24T15:32:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5408ff9700 1 -- 172.21.15.29:0/1767810594 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54000073c0 con 0x7f540c062ec0 2024-01-24T15:32:17.509 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.505+0000 7f5412de5700 1 -- 172.21.15.29:0/1767810594 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540c168360 con 0x7f540c062ec0 2024-01-24T15:32:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.506+0000 7f5412de5700 1 -- 172.21.15.29:0/1767810594 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540c0475e0 con 0x7f540c062ec0 2024-01-24T15:32:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.507+0000 7f5408ff9700 1 -- 172.21.15.29:0/1767810594 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f54000068b0 con 0x7f540c062ec0 2024-01-24T15:32:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.507+0000 7f5408ff9700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f53f005f270 0x7f53f0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.507+0000 7f5408ff9700 1 -- 172.21.15.29:0/1767810594 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 6362+0+0 (secure 0 0 0) 0x7f5400079840 con 0x7f540c062ec0 2024-01-24T15:32:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.507+0000 7f540bfff700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f53f005f270 0x7f53f0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.510 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.508+0000 7f540bfff700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f53f005f270 0x7f53f0061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f53f8001ed0 tx=0x7f53f8000bc0).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:17.513 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.512+0000 7f5408ff9700 1 -- 172.21.15.29:0/1767810594 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5400047b00 con 0x7f540c062ec0 2024-01-24T15:32:17.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.691+0000 7f5412de5700 1 -- 172.21.15.29:0/1767810594 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f540c0c5f60 con 0x7f540c062ec0 2024-01-24T15:32:17.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.691+0000 7f5408ff9700 1 -- 172.21.15.29:0/1767810594 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v227) v1 ==== 92+0+10 (secure 0 0 0) 0x7f540004a7e0 con 0x7f540c062ec0 2024-01-24T15:32:17.693 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T15:32:17.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.693+0000 7f5412de5700 1 -- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f53f005f270 msgr2=0x7f53f0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.693+0000 7f5412de5700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f53f005f270 0x7f53f0061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f53f8001ed0 tx=0x7f53f8000bc0).stop 2024-01-24T15:32:17.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.693+0000 7f5412de5700 1 -- 172.21.15.29:0/1767810594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f540c062ec0 msgr2=0x7f540c156c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.693+0000 7f5412de5700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f540c062ec0 0x7f540c156c40 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 rx=0x7f540000c440 tx=0x7f5400007e00).stop 2024-01-24T15:32:17.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.693+0000 7f5412de5700 1 -- 172.21.15.29:0/1767810594 shutdown_connections 2024-01-24T15:32:17.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.693+0000 7f5412de5700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f540c1384f0 0x7f540c167990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.693+0000 7f5412de5700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f540c062ec0 0x7f540c156c40 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.693+0000 7f5412de5700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f53f005f270 0x7f53f0061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.693+0000 7f5412de5700 1 --2- 172.21.15.29:0/1767810594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f540c0c8270 0x7f540c1674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.693+0000 7f5412de5700 1 -- 172.21.15.29:0/1767810594 >> 172.21.15.29:0/1767810594 conn(0x7f540c064390 msgr2=0x7f540c137540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:17.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.693+0000 7f5412de5700 1 -- 172.21.15.29:0/1767810594 shutdown_connections 2024-01-24T15:32:17.696 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.693+0000 7f5412de5700 1 -- 172.21.15.29:0/1767810594 wait complete. 2024-01-24T15:32:17.709 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-24T15:32:17.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.837+0000 7f0bda7db700 1 Processor -- start 2024-01-24T15:32:17.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.837+0000 7f0bda7db700 1 -- start start 2024-01-24T15:32:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.837+0000 7f0bda7db700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bd4062970 0x7f0bd4062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.837+0000 7f0bda7db700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0bd4069dd0 0x7f0bd40a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bda7db700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd40bd190 con 0x7f0bd4153a80 2024-01-24T15:32:17.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bda7db700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd4154610 con 0x7f0bd4069dd0 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bda7db700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40c4e60 con 0x7f0bd4062970 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd8d78700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f0bd4153a80 0x7f0bd4153f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56822/0 (socket says 172.21.15.29:56822) 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd8d78700 1 -- 172.21.15.29:0/2011167295 learned_addr learned my addr 172.21.15.29:0/2011167295 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd3fff700 1 --2- 172.21.15.29:0/2011167295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bd4062970 0x7f0bd4062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd37fe700 1 --2- 172.21.15.29:0/2011167295 >> v2:172.21.15.163:3300/0 conn(0x7f0bd4069dd0 0x7f0bd40a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd2ffd700 1 -- 172.21.15.29:0/2011167295 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397866947 0 0) 0x7f0bd40bd190 con 0x7f0bd4153a80 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd2ffd700 1 -- 172.21.15.29:0/2011167295 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bbc0033f0 con 0x7f0bd4153a80 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd2ffd700 1 -- 172.21.15.29:0/2011167295 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 967232133 0 0) 0x7f0bbc0033f0 con 0x7f0bd4153a80 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd3fff700 1 -- 172.21.15.29:0/2011167295 >> v2:172.21.15.163:3300/0 conn(0x7f0bd4069dd0 msgr2=0x7f0bd40a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd3fff700 1 --2- 172.21.15.29:0/2011167295 >> v2:172.21.15.163:3300/0 conn(0x7f0bd4069dd0 0x7f0bd40a6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd3fff700 1 -- 172.21.15.29:0/2011167295 >> v1:172.21.15.29:6789/0 conn(0x7f0bd4153a80 legacy=0x7f0bd4153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd3fff700 1 -- 172.21.15.29:0/2011167295 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd4066370 con 0x7f0bd4062970 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd37fe700 1 --2- 172.21.15.29:0/2011167295 >> v2:172.21.15.163:3300/0 conn(0x7f0bd4069dd0 0x7f0bd40a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.838+0000 7f0bd3fff700 1 --2- 172.21.15.29:0/2011167295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bd4062970 0x7f0bd4062d50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f0bc4009760 tx=0x7f0bc4009060).ready entity=mon.2 client_cookie=f89d5be8e3889d81 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.839+0000 7f0bd2ffd700 1 -- 172.21.15.29:0/2011167295 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0017f0 con 0x7f0bd4062970 2024-01-24T15:32:17.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.839+0000 7f0bd2ffd700 1 -- 172.21.15.29:0/2011167295 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc40090d0 con 0x7f0bd4062970 2024-01-24T15:32:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.839+0000 7f0bd2ffd700 1 -- 172.21.15.29:0/2011167295 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4011610 con 0x7f0bd4062970 2024-01-24T15:32:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.839+0000 7f0bda7db700 1 -- 172.21.15.29:0/2011167295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bd4062970 msgr2=0x7f0bd4062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.839+0000 7f0bda7db700 1 --2- 172.21.15.29:0/2011167295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bd4062970 0x7f0bd4062d50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f0bc4009760 tx=0x7f0bc4009060).stop 2024-01-24T15:32:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.839+0000 7f0bda7db700 1 -- 172.21.15.29:0/2011167295 shutdown_connections 2024-01-24T15:32:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.839+0000 7f0bda7db700 1 --2- 172.21.15.29:0/2011167295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bd4062970 0x7f0bd4062d50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.839+0000 7f0bda7db700 1 --2- 172.21.15.29:0/2011167295 >> v2:172.21.15.163:3300/0 conn(0x7f0bd4069dd0 0x7f0bd40a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.839+0000 7f0bda7db700 1 -- 172.21.15.29:0/2011167295 >> 172.21.15.29:0/2011167295 conn(0x7f0bd40c85c0 msgr2=0x7f0bd40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.839+0000 7f0bda7db700 1 -- 172.21.15.29:0/2011167295 shutdown_connections 2024-01-24T15:32:17.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.839+0000 7f0bda7db700 1 -- 172.21.15.29:0/2011167295 wait complete. 2024-01-24T15:32:17.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.840+0000 7f0bda7db700 1 Processor -- start 2024-01-24T15:32:17.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.841+0000 7f0bda7db700 1 -- start start 2024-01-24T15:32:17.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.841+0000 7f0bda7db700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0bd4062970 0x7f0bd40a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bda7db700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bd4069dd0 0x7f0bd40a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bda7db700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bd4153a80 0x7f0bd40c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bda7db700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40aabd0 con 0x7f0bd4153a80 2024-01-24T15:32:17.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bda7db700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40aa070 con 0x7f0bd4062970 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bda7db700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40a8540 con 0x7f0bd4069dd0 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bd8d78700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bd4153a80 0x7f0bd40c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bd8d78700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bd4153a80 0x7f0bd40c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53416/0 (socket says 172.21.15.29:53416) 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bd8d78700 1 -- 172.21.15.29:0/3419886203 learned_addr learned my addr 172.21.15.29:0/3419886203 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bd37fe700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bd4069dd0 0x7f0bd40a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bd3fff700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0bd4062970 0x7f0bd40a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bd8d78700 1 -- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bd4069dd0 msgr2=0x7f0bd40a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bd8d78700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bd4069dd0 0x7f0bd40a9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bd8d78700 1 -- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0bd4062970 msgr2=0x7f0bd40a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bd8d78700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0bd4062970 0x7f0bd40a7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bd8d78700 1 -- 172.21.15.29:0/3419886203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc80092a0 con 0x7f0bd4153a80 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bd8d78700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bd4153a80 0x7f0bd40c6d20 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 rx=0x7f0bcc001560 tx=0x7f0bcc007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bbbfff700 1 -- 172.21.15.29:0/3419886203 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00ce40 con 0x7f0bd4153a80 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bbbfff700 1 -- 172.21.15.29:0/3419886203 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc004c40 con 0x7f0bd4153a80 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bda7db700 1 -- 172.21.15.29:0/3419886203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc40092a0 con 0x7f0bd4153a80 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.842+0000 7f0bbbfff700 1 -- 172.21.15.29:0/3419886203 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc007170 con 0x7f0bd4153a80 2024-01-24T15:32:17.846 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.843+0000 7f0bda7db700 1 -- 172.21.15.29:0/3419886203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd40a7f80 con 0x7f0bd4153a80 2024-01-24T15:32:17.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.843+0000 7f0bda7db700 1 -- 172.21.15.29:0/3419886203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd40aad40 con 0x7f0bd4153a80 2024-01-24T15:32:17.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.844+0000 7f0bbbfff700 1 -- 172.21.15.29:0/3419886203 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0bcc0074e0 con 0x7f0bd4153a80 2024-01-24T15:32:17.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.845+0000 7f0bbbfff700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f0bc005f270 0x7f0bc0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:17.847 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.845+0000 7f0bbbfff700 1 -- 172.21.15.29:0/3419886203 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 6362+0+0 (secure 0 0 0) 0x7f0bcc07a0b0 con 0x7f0bd4153a80 2024-01-24T15:32:17.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.845+0000 7f0bd3fff700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f0bc005f270 0x7f0bc0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:17.848 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.845+0000 7f0bd3fff700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f0bc005f270 0x7f0bc0061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0bc4009760 tx=0x7f0bc4016000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:17.851 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:17.849+0000 7f0bbbfff700 1 -- 172.21.15.29:0/3419886203 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0bcc04b110 con 0x7f0bd4153a80 2024-01-24T15:32:18.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.031+0000 7f0bda7db700 1 -- 172.21.15.29:0/3419886203 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0bd40a6680 con 0x7f0bd4153a80 2024-01-24T15:32:18.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.031+0000 7f0bbbfff700 1 -- 172.21.15.29:0/3419886203 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v227) v1 ==== 84+0+143 (secure 0 0 0) 0x7f0bcc04e730 con 0x7f0bd4153a80 2024-01-24T15:32:18.033 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:18.034 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T15:32:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 1, 2024-01-24T15:32:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "device_health_metrics" 2024-01-24T15:32:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-01-24T15:32:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: "poolnum": 2, 2024-01-24T15:32:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: "poolname": "rbd" 2024-01-24T15:32:18.034 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:18.034 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T15:32:18.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.033+0000 7f0bda7db700 1 -- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f0bc005f270 msgr2=0x7f0bc0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:18.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.033+0000 7f0bda7db700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f0bc005f270 0x7f0bc0061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0bc4009760 tx=0x7f0bc4016000).stop 2024-01-24T15:32:18.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.033+0000 7f0bda7db700 1 -- 172.21.15.29:0/3419886203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bd4153a80 msgr2=0x7f0bd40c6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:18.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.033+0000 7f0bda7db700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bd4153a80 0x7f0bd40c6d20 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 rx=0x7f0bcc001560 tx=0x7f0bcc007ba0).stop 2024-01-24T15:32:18.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.033+0000 7f0bda7db700 1 -- 172.21.15.29:0/3419886203 shutdown_connections 2024-01-24T15:32:18.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.033+0000 7f0bda7db700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0bd4153a80 0x7f0bd40c6d20 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:18.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.033+0000 7f0bda7db700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0bd4062970 0x7f0bd40a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:18.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.033+0000 7f0bda7db700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f0bc005f270 0x7f0bc0061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:18.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.033+0000 7f0bda7db700 1 --2- 172.21.15.29:0/3419886203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0bd4069dd0 0x7f0bd40a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:18.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.033+0000 7f0bda7db700 1 -- 172.21.15.29:0/3419886203 >> 172.21.15.29:0/3419886203 conn(0x7f0bd40c85c0 msgr2=0x7f0bd40a6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:18.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.033+0000 7f0bda7db700 1 -- 172.21.15.29:0/3419886203 shutdown_connections 2024-01-24T15:32:18.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.033+0000 7f0bda7db700 1 -- 172.21.15.29:0/3419886203 wait complete. 2024-01-24T15:32:18.049 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-24T15:32:18.049 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2024-01-24T15:32:18.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.182+0000 7fe632c5d700 1 Processor -- start 2024-01-24T15:32:18.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.183+0000 7fe632c5d700 1 -- start start 2024-01-24T15:32:18.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.183+0000 7fe632c5d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe62c151e50 0x7fe62c0c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:18.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.183+0000 7fe632c5d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe62c064e90 0x7fe62c155420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:18.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.183+0000 7fe632c5d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe62c154f90 con 0x7fe62c0c7f70 2024-01-24T15:32:18.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.183+0000 7fe632c5d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62c155910 con 0x7fe62c151e50 2024-01-24T15:32:18.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.183+0000 7fe632c5d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c0a9f10 con 0x7fe62c064e90 2024-01-24T15:32:18.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.184+0000 7fe6309f9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe62c151e50 0x7fe62c0c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:18.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.184+0000 7fe6311fa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe62c064e90 0x7fe62c155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:18.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.184+0000 7fe6311fa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe62c064e90 0x7fe62c155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:48376/0 (socket says 172.21.15.29:48376) 2024-01-24T15:32:18.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.184+0000 7fe6309f9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe62c151e50 0x7fe62c0c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:48586/0 (socket says 172.21.15.29:48586) 2024-01-24T15:32:18.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.184+0000 7fe6311fa700 1 -- 172.21.15.29:0/2492652613 learned_addr learned my addr 172.21.15.29:0/2492652613 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:18.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.184+0000 7fe6309f9700 1 -- 172.21.15.29:0/2492652613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe62c064e90 msgr2=0x7fe62c155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:18.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.184+0000 7fe6309f9700 1 --2- 172.21.15.29:0/2492652613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe62c064e90 0x7fe62c155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:18.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.184+0000 7fe6309f9700 1 -- 172.21.15.29:0/2492652613 >> v1:172.21.15.163:6789/0 conn(0x7fe62c0c7f70 legacy=0x7fe62c064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:18.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.184+0000 7fe6309f9700 1 -- 172.21.15.29:0/2492652613 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c0aa1f0 con 0x7fe62c151e50 2024-01-24T15:32:18.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.184+0000 7fe6311fa700 1 --2- 172.21.15.29:0/2492652613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe62c064e90 0x7fe62c155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:18.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.185+0000 7fe6309f9700 1 --2- 172.21.15.29:0/2492652613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe62c151e50 0x7fe62c0c53c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fe620009760 tx=0x7fe620009060).ready entity=mon.1 client_cookie=ce6f8026bad471c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:18.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.185+0000 7fe62b7fe700 1 -- 172.21.15.29:0/2492652613 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62000f510 con 0x7fe62c151e50 2024-01-24T15:32:18.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.185+0000 7fe62b7fe700 1 -- 172.21.15.29:0/2492652613 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62000f670 con 0x7fe62c151e50 2024-01-24T15:32:18.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.185+0000 7fe62b7fe700 1 -- 172.21.15.29:0/2492652613 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62000f9f0 con 0x7fe62c151e50 2024-01-24T15:32:18.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.185+0000 7fe632c5d700 1 -- 172.21.15.29:0/2492652613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe62c151e50 msgr2=0x7fe62c0c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:18.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.185+0000 7fe632c5d700 1 --2- 172.21.15.29:0/2492652613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe62c151e50 0x7fe62c0c53c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fe620009760 tx=0x7fe620009060).stop 2024-01-24T15:32:18.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.185+0000 7fe632c5d700 1 -- 172.21.15.29:0/2492652613 shutdown_connections 2024-01-24T15:32:18.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.185+0000 7fe632c5d700 1 --2- 172.21.15.29:0/2492652613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe62c064e90 0x7fe62c155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:18.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.185+0000 7fe632c5d700 1 --2- 172.21.15.29:0/2492652613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe62c151e50 0x7fe62c0c53c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:18.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.185+0000 7fe632c5d700 1 -- 172.21.15.29:0/2492652613 >> 172.21.15.29:0/2492652613 conn(0x7fe62c0cb880 msgr2=0x7fe62c0c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:18.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.185+0000 7fe632c5d700 1 -- 172.21.15.29:0/2492652613 shutdown_connections 2024-01-24T15:32:18.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.185+0000 7fe632c5d700 1 -- 172.21.15.29:0/2492652613 wait complete. 2024-01-24T15:32:18.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.187+0000 7fe632c5d700 1 Processor -- start 2024-01-24T15:32:18.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.187+0000 7fe632c5d700 1 -- start start 2024-01-24T15:32:18.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe632c5d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe62c064e90 0x7fe62c0c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:18.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe632c5d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe62c0c7f70 0x7fe62c14c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:18.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe632c5d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c151e50 0x7fe62c14f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:18.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe632c5d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c14e150 con 0x7fe62c151e50 2024-01-24T15:32:18.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe632c5d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c0c3d10 con 0x7fe62c0c7f70 2024-01-24T15:32:18.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe632c5d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c0aede0 con 0x7fe62c064e90 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe6311fa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c151e50 0x7fe62c14f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe6311fa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c151e50 0x7fe62c14f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53430/0 (socket says 172.21.15.29:53430) 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe6311fa700 1 -- 172.21.15.29:0/2986506569 learned_addr learned my addr 172.21.15.29:0/2986506569 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe6309f9700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe62c064e90 0x7fe62c0c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe62bfff700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe62c0c7f70 0x7fe62c14c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe6311fa700 1 -- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe62c064e90 msgr2=0x7fe62c0c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe6311fa700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe62c064e90 0x7fe62c0c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe6311fa700 1 -- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe62c0c7f70 msgr2=0x7fe62c14c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe6311fa700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe62c0c7f70 0x7fe62c14c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe6311fa700 1 -- 172.21.15.29:0/2986506569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe624009ae0 con 0x7fe62c151e50 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe6311fa700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c151e50 0x7fe62c14f550 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 rx=0x7fe624009f50 tx=0x7fe62400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.188+0000 7fe628ff9700 1 -- 172.21.15.29:0/2986506569 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62400c9e0 con 0x7fe62c151e50 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.189+0000 7fe628ff9700 1 -- 172.21.15.29:0/2986506569 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62400f040 con 0x7fe62c151e50 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.189+0000 7fe632c5d700 1 -- 172.21.15.29:0/2986506569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6200092a0 con 0x7fe62c151e50 2024-01-24T15:32:18.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.189+0000 7fe628ff9700 1 -- 172.21.15.29:0/2986506569 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62400cd50 con 0x7fe62c151e50 2024-01-24T15:32:18.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.189+0000 7fe632c5d700 1 -- 172.21.15.29:0/2986506569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62c0af680 con 0x7fe62c151e50 2024-01-24T15:32:18.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.190+0000 7fe632c5d700 1 -- 172.21.15.29:0/2986506569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62c0aa710 con 0x7fe62c151e50 2024-01-24T15:32:18.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.190+0000 7fe628ff9700 1 -- 172.21.15.29:0/2986506569 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe624014620 con 0x7fe62c151e50 2024-01-24T15:32:18.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.190+0000 7fe628ff9700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe61005f270 0x7fe610061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:18.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.191+0000 7fe628ff9700 1 -- 172.21.15.29:0/2986506569 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 6362+0+0 (secure 0 0 0) 0x7fe62407a740 con 0x7fe62c151e50 2024-01-24T15:32:18.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.191+0000 7fe6309f9700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe61005f270 0x7fe610061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:18.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.191+0000 7fe6309f9700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe61005f270 0x7fe610061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe620009fd0 tx=0x7fe620017000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:18.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.195+0000 7fe628ff9700 1 -- 172.21.15.29:0/2986506569 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe624049b00 con 0x7fe62c151e50 2024-01-24T15:32:18.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:18.379+0000 7fe632c5d700 1 -- 172.21.15.29:0/2986506569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fe62c083310 con 0x7fe62c151e50 2024-01-24T15:32:19.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.249+0000 7fe628ff9700 1 -- 172.21.15.29:0/2986506569 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v228) v1 ==== 152+0+0 (secure 0 0 0) 0x7fe62404b6e0 con 0x7fe62c151e50 2024-01-24T15:32:19.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.254+0000 7fe632c5d700 1 -- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe61005f270 msgr2=0x7fe610061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:19.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.254+0000 7fe632c5d700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe61005f270 0x7fe610061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe620009fd0 tx=0x7fe620017000).stop 2024-01-24T15:32:19.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.254+0000 7fe632c5d700 1 -- 172.21.15.29:0/2986506569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c151e50 msgr2=0x7fe62c14f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:19.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.254+0000 7fe632c5d700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c151e50 0x7fe62c14f550 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 rx=0x7fe624009f50 tx=0x7fe62400b040).stop 2024-01-24T15:32:19.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.254+0000 7fe632c5d700 1 -- 172.21.15.29:0/2986506569 shutdown_connections 2024-01-24T15:32:19.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.254+0000 7fe632c5d700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62c151e50 0x7fe62c14f550 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:19.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.254+0000 7fe632c5d700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe61005f270 0x7fe610061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:19.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.254+0000 7fe632c5d700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe62c064e90 0x7fe62c0c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:19.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.254+0000 7fe632c5d700 1 --2- 172.21.15.29:0/2986506569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe62c0c7f70 0x7fe62c14c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:19.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.254+0000 7fe632c5d700 1 -- 172.21.15.29:0/2986506569 >> 172.21.15.29:0/2986506569 conn(0x7fe62c0cb880 msgr2=0x7fe62c152600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:19.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.255+0000 7fe632c5d700 1 -- 172.21.15.29:0/2986506569 shutdown_connections 2024-01-24T15:32:19.258 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.256+0000 7fe632c5d700 1 -- 172.21.15.29:0/2986506569 wait complete. 2024-01-24T15:32:19.262 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' created 2024-01-24T15:32:19.278 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-24T15:32:19.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.416+0000 7fd1e2ca8700 1 Processor -- start 2024-01-24T15:32:19.418 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.416+0000 7fd1e2ca8700 1 -- start start 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.417+0000 7fd1e2ca8700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd1dc064c40 0x7fd1dc065080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.417+0000 7fd1e2ca8700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1dc05e790 con 0x7fd1dc0aa4f0 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.417+0000 7fd1e2ca8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1dc0ac0f0 con 0x7fd1dc152740 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.417+0000 7fd1e2ca8700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1dc0aac60 con 0x7fd1dc064c40 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.417+0000 7fd1e1245700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd1dc064c40 0x7fd1dc065080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.417+0000 7fd1dbfff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd1dc0aa4f0 0x7fd1dc152b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56834/0 (socket says 172.21.15.29:56834) 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.417+0000 7fd1dbfff700 1 -- 172.21.15.29:0/3735782656 learned_addr learned my addr 172.21.15.29:0/3735782656 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.417+0000 7fd1e1245700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd1dc064c40 0x7fd1dc065080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53442/0 (socket says 172.21.15.29:53442) 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1e1245700 1 -- 172.21.15.29:0/3735782656 >> v1:172.21.15.163:6789/0 conn(0x7fd1dc152740 legacy=0x7fd1dc0ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1e1245700 1 -- 172.21.15.29:0/3735782656 >> v1:172.21.15.29:6789/0 conn(0x7fd1dc0aa4f0 legacy=0x7fd1dc152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1e1245700 1 -- 172.21.15.29:0/3735782656 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc065570 con 0x7fd1dc064c40 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1db7fe700 1 -- 172.21.15.29:0/3735782656 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031342265 0 0) 0x7fd1dc05e790 con 0x7fd1dc0aa4f0 2024-01-24T15:32:19.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1e1245700 1 --2- 172.21.15.29:0/3735782656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1dc064c40 0x7fd1dc065080 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 rx=0x7fd1d4009e60 tx=0x7fd1d4009780).ready entity=mon.0 client_cookie=e92ab7d9c27132ad server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:19.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1db7fe700 1 -- 172.21.15.29:0/3735782656 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400e070 con 0x7fd1dc064c40 2024-01-24T15:32:19.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1db7fe700 1 -- 172.21.15.29:0/3735782656 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d400bc30 con 0x7fd1dc064c40 2024-01-24T15:32:19.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1db7fe700 1 -- 172.21.15.29:0/3735782656 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4012410 con 0x7fd1dc064c40 2024-01-24T15:32:19.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/3735782656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1dc064c40 msgr2=0x7fd1dc065080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:19.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1e2ca8700 1 --2- 172.21.15.29:0/3735782656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1dc064c40 0x7fd1dc065080 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 rx=0x7fd1d4009e60 tx=0x7fd1d4009780).stop 2024-01-24T15:32:19.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/3735782656 shutdown_connections 2024-01-24T15:32:19.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1e2ca8700 1 --2- 172.21.15.29:0/3735782656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1dc064c40 0x7fd1dc065080 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:19.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.418+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/3735782656 >> 172.21.15.29:0/3735782656 conn(0x7fd1dc0c7330 msgr2=0x7fd1dc0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:19.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.419+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/3735782656 shutdown_connections 2024-01-24T15:32:19.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.419+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/3735782656 wait complete. 2024-01-24T15:32:19.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.420+0000 7fd1e2ca8700 1 Processor -- start 2024-01-24T15:32:19.423 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.421+0000 7fd1e2ca8700 1 -- start start 2024-01-24T15:32:19.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.421+0000 7fd1e2ca8700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc064c40 0x7fd1dc14d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:19.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.421+0000 7fd1e2ca8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc152740 0x7fd1dc0bee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:19.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.421+0000 7fd1e2ca8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1dc0bf350 0x7fd1dc0c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:19.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.421+0000 7fd1e2ca8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0bf890 con 0x7fd1dc0bf350 2024-01-24T15:32:19.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.421+0000 7fd1e2ca8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0bfb70 con 0x7fd1dc152740 2024-01-24T15:32:19.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.421+0000 7fd1e2ca8700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0bfe50 con 0x7fd1dc064c40 2024-01-24T15:32:19.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e1245700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1dc0bf350 0x7fd1dc0c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:19.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e1245700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1dc0bf350 0x7fd1dc0c7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53452/0 (socket says 172.21.15.29:53452) 2024-01-24T15:32:19.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e1245700 1 -- 172.21.15.29:0/4088519060 learned_addr learned my addr 172.21.15.29:0/4088519060 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:19.425 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1dbfff700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc152740 0x7fd1dc0bee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e0a44700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc064c40 0x7fd1dc14d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e1245700 1 -- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc064c40 msgr2=0x7fd1dc14d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e1245700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc064c40 0x7fd1dc14d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e1245700 1 -- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc152740 msgr2=0x7fd1dc0bee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e1245700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc152740 0x7fd1dc0bee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e1245700 1 -- 172.21.15.29:0/4088519060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d4009a20 con 0x7fd1dc0bf350 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e0a44700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc064c40 0x7fd1dc14d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e1245700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1dc0bf350 0x7fd1dc0c7710 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 rx=0x7fd1d400b6c0 tx=0x7fd1d4009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1d8ff9700 1 -- 172.21.15.29:0/4088519060 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d40064a0 con 0x7fd1dc0bf350 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1d8ff9700 1 -- 172.21.15.29:0/4088519060 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d4006600 con 0x7fd1dc0bf350 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/4088519060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc0c0130 con 0x7fd1dc0bf350 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1d8ff9700 1 -- 172.21.15.29:0/4088519060 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d40069a0 con 0x7fd1dc0bf350 2024-01-24T15:32:19.426 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.422+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/4088519060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1dc14dc70 con 0x7fd1dc0bf350 2024-01-24T15:32:19.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.423+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/4088519060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1dc0c0650 con 0x7fd1dc0bf350 2024-01-24T15:32:19.427 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.424+0000 7fd1d8ff9700 1 -- 172.21.15.29:0/4088519060 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd1d401d630 con 0x7fd1dc0bf350 2024-01-24T15:32:19.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.425+0000 7fd1d8ff9700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd1c805f270 0x7fd1c8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:19.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.425+0000 7fd1d8ff9700 1 -- 172.21.15.29:0/4088519060 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6732+0+0 (secure 0 0 0) 0x7fd1d407d380 con 0x7fd1dc0bf350 2024-01-24T15:32:19.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.425+0000 7fd1e0a44700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd1c805f270 0x7fd1c8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:19.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.426+0000 7fd1e0a44700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd1c805f270 0x7fd1c8061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fd1cc001780 tx=0x7fd1cc009040).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:19.430 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.429+0000 7fd1d8ff9700 1 -- 172.21.15.29:0/4088519060 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd1d404e330 con 0x7fd1dc0bf350 2024-01-24T15:32:19.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:19.609+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/4088519060 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd1dc0475e0 con 0x7fd1dc0bf350 2024-01-24T15:32:20.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.258+0000 7fd1d8ff9700 1 -- 172.21.15.29:0/4088519060 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v229) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd1d4051950 con 0x7fd1dc0bf350 2024-01-24T15:32:20.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.286+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd1c805f270 msgr2=0x7fd1c8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:20.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.286+0000 7fd1e2ca8700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd1c805f270 0x7fd1c8061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fd1cc001780 tx=0x7fd1cc009040).stop 2024-01-24T15:32:20.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.286+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/4088519060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1dc0bf350 msgr2=0x7fd1dc0c7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:20.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.286+0000 7fd1e2ca8700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1dc0bf350 0x7fd1dc0c7710 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 rx=0x7fd1d400b6c0 tx=0x7fd1d4009780).stop 2024-01-24T15:32:20.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.287+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/4088519060 shutdown_connections 2024-01-24T15:32:20.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.287+0000 7fd1e2ca8700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1dc0bf350 0x7fd1dc0c7710 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.287+0000 7fd1e2ca8700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd1c805f270 0x7fd1c8061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.287+0000 7fd1e2ca8700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc064c40 0x7fd1dc14d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.287+0000 7fd1e2ca8700 1 --2- 172.21.15.29:0/4088519060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc152740 0x7fd1dc0bee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.287+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/4088519060 >> 172.21.15.29:0/4088519060 conn(0x7fd1dc0c7330 msgr2=0x7fd1dc0c5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:20.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.288+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/4088519060 shutdown_connections 2024-01-24T15:32:20.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.288+0000 7fd1e2ca8700 1 -- 172.21.15.29:0/4088519060 wait complete. 2024-01-24T15:32:20.291 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' created 2024-01-24T15:32:20.305 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-24T15:32:20.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.440+0000 7fdb64808700 1 Processor -- start 2024-01-24T15:32:20.442 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.440+0000 7fdb64808700 1 -- start start 2024-01-24T15:32:20.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.441+0000 7fdb64808700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdb60157380 0x7fdb60063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:20.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.441+0000 7fdb64808700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 0x7fdb60157c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:20.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.441+0000 7fdb64808700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb600627b0 con 0x7fdb60158110 2024-01-24T15:32:20.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.441+0000 7fdb64808700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb60062d70 con 0x7fdb60157380 2024-01-24T15:32:20.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.441+0000 7fdb64808700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601586e0 con 0x7fdb60157760 2024-01-24T15:32:20.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.441+0000 7fdb5dd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 0x7fdb60157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:20.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.441+0000 7fdb5dd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 0x7fdb60157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:48390/0 (socket says 172.21.15.29:48390) 2024-01-24T15:32:20.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.441+0000 7fdb5dd9b700 1 -- 172.21.15.29:0/2624996519 learned_addr learned my addr 172.21.15.29:0/2624996519 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:20.444 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.441+0000 7fdb5e59c700 1 --2- 172.21.15.29:0/2624996519 >> v2:172.21.15.163:3300/0 conn(0x7fdb60157380 0x7fdb60063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:20.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.442+0000 7fdb5e59c700 1 -- 172.21.15.29:0/2624996519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 msgr2=0x7fdb60157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:20.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.442+0000 7fdb5e59c700 1 --2- 172.21.15.29:0/2624996519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 0x7fdb60157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.442+0000 7fdb5e59c700 1 -- 172.21.15.29:0/2624996519 >> v1:172.21.15.163:6789/0 conn(0x7fdb60158110 legacy=0x7fdb60064b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.442+0000 7fdb5e59c700 1 -- 172.21.15.29:0/2624996519 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb601589c0 con 0x7fdb60157380 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.442+0000 7fdb5dd9b700 1 --2- 172.21.15.29:0/2624996519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 0x7fdb60157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.442+0000 7fdb5e59c700 1 --2- 172.21.15.29:0/2624996519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb60157380 0x7fdb60063200 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7fdb54009760 tx=0x7fdb54009060).ready entity=mon.1 client_cookie=b72323766b50accc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.442+0000 7fdb5d59a700 1 -- 172.21.15.29:0/2624996519 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb540090d0 con 0x7fdb60157380 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.442+0000 7fdb5d59a700 1 -- 172.21.15.29:0/2624996519 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5400f510 con 0x7fdb60157380 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.442+0000 7fdb5d59a700 1 -- 172.21.15.29:0/2624996519 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5400f890 con 0x7fdb60157380 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.443+0000 7fdb64808700 1 -- 172.21.15.29:0/2624996519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb60157380 msgr2=0x7fdb60063200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.443+0000 7fdb64808700 1 --2- 172.21.15.29:0/2624996519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb60157380 0x7fdb60063200 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7fdb54009760 tx=0x7fdb54009060).stop 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.443+0000 7fdb64808700 1 -- 172.21.15.29:0/2624996519 shutdown_connections 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.443+0000 7fdb64808700 1 --2- 172.21.15.29:0/2624996519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb60157380 0x7fdb60063200 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.443+0000 7fdb64808700 1 --2- 172.21.15.29:0/2624996519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 0x7fdb60157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.443+0000 7fdb64808700 1 -- 172.21.15.29:0/2624996519 >> 172.21.15.29:0/2624996519 conn(0x7fdb600c7260 msgr2=0x7fdb600c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:20.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.443+0000 7fdb64808700 1 -- 172.21.15.29:0/2624996519 shutdown_connections 2024-01-24T15:32:20.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.443+0000 7fdb64808700 1 -- 172.21.15.29:0/2624996519 wait complete. 2024-01-24T15:32:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.444+0000 7fdb64808700 1 Processor -- start 2024-01-24T15:32:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.444+0000 7fdb64808700 1 -- start start 2024-01-24T15:32:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.445+0000 7fdb64808700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb60157380 0x7fdb600c0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.445+0000 7fdb64808700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 0x7fdb600c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.445+0000 7fdb64808700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb60158110 0x7fdb60147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.445+0000 7fdb64808700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600c2090 con 0x7fdb60157380 2024-01-24T15:32:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.445+0000 7fdb64808700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600c2370 con 0x7fdb60158110 2024-01-24T15:32:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.445+0000 7fdb64808700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600c2650 con 0x7fdb60157760 2024-01-24T15:32:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.445+0000 7fdb5e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb60157380 0x7fdb600c0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:20.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.445+0000 7fdb5dd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 0x7fdb600c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.445+0000 7fdb5e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb60157380 0x7fdb600c0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53462/0 (socket says 172.21.15.29:53462) 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.445+0000 7fdb5e59c700 1 -- 172.21.15.29:0/355957942 learned_addr learned my addr 172.21.15.29:0/355957942 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.445+0000 7fdb5dd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 0x7fdb600c18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:48392/0 (socket says 172.21.15.29:48392) 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.445+0000 7fdb5ed9d700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb60158110 0x7fdb60147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.446+0000 7fdb5ed9d700 1 -- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 msgr2=0x7fdb600c18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.446+0000 7fdb5ed9d700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 0x7fdb600c18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.446+0000 7fdb5ed9d700 1 -- 172.21.15.29:0/355957942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb60157380 msgr2=0x7fdb600c0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.446+0000 7fdb5ed9d700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb60157380 0x7fdb600c0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.446+0000 7fdb5ed9d700 1 -- 172.21.15.29:0/355957942 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb50009370 con 0x7fdb60158110 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.446+0000 7fdb5ed9d700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb60158110 0x7fdb60147670 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7fdb5800c790 tx=0x7fdb58001460).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.446+0000 7fdb4e7fc700 1 -- 172.21.15.29:0/355957942 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800f470 con 0x7fdb60158110 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.446+0000 7fdb4e7fc700 1 -- 172.21.15.29:0/355957942 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5800f5d0 con 0x7fdb60158110 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.446+0000 7fdb4e7fc700 1 -- 172.21.15.29:0/355957942 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800f950 con 0x7fdb60158110 2024-01-24T15:32:20.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.446+0000 7fdb64808700 1 -- 172.21.15.29:0/355957942 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb540092a0 con 0x7fdb60158110 2024-01-24T15:32:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.446+0000 7fdb64808700 1 -- 172.21.15.29:0/355957942 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb600c2c10 con 0x7fdb60158110 2024-01-24T15:32:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.447+0000 7fdb64808700 1 -- 172.21.15.29:0/355957942 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb600c2e20 con 0x7fdb60158110 2024-01-24T15:32:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.448+0000 7fdb4e7fc700 1 -- 172.21.15.29:0/355957942 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdb5800fd40 con 0x7fdb60158110 2024-01-24T15:32:20.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.448+0000 7fdb4e7fc700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdb4005f270 0x7fdb40061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:20.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.448+0000 7fdb4e7fc700 1 -- 172.21.15.29:0/355957942 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fdb58079c60 con 0x7fdb60158110 2024-01-24T15:32:20.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.448+0000 7fdb5e59c700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdb4005f270 0x7fdb40061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:20.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.448+0000 7fdb5e59c700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdb4005f270 0x7fdb40061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fdb54009270 tx=0x7fdb54017000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:20.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.453+0000 7fdb4e7fc700 1 -- 172.21.15.29:0/355957942 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdb58048b00 con 0x7fdb60158110 2024-01-24T15:32:20.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.682+0000 7fdb64808700 1 -- 172.21.15.29:0/355957942 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdb60062a00 con 0x7fdb60158110 2024-01-24T15:32:20.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.700+0000 7fdb4e7fc700 1 -- 172.21.15.29:0/355957942 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v94) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdb5804aa80 con 0x7fdb60158110 2024-01-24T15:32:20.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.706+0000 7fdb64808700 1 -- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdb4005f270 msgr2=0x7fdb40061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:20.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.706+0000 7fdb64808700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdb4005f270 0x7fdb40061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fdb54009270 tx=0x7fdb54017000).stop 2024-01-24T15:32:20.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.706+0000 7fdb64808700 1 -- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb60158110 msgr2=0x7fdb60147670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:20.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.706+0000 7fdb64808700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb60158110 0x7fdb60147670 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7fdb5800c790 tx=0x7fdb58001460).stop 2024-01-24T15:32:20.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.706+0000 7fdb64808700 1 -- 172.21.15.29:0/355957942 shutdown_connections 2024-01-24T15:32:20.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.706+0000 7fdb64808700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb60158110 0x7fdb60147670 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.706+0000 7fdb64808700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb60157380 0x7fdb600c0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.706+0000 7fdb64808700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdb4005f270 0x7fdb40061730 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0x7fdb54009270 tx=0x7fdb54017000).stop 2024-01-24T15:32:20.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.706+0000 7fdb64808700 1 --2- 172.21.15.29:0/355957942 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb60157760 0x7fdb600c18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.706+0000 7fdb64808700 1 -- 172.21.15.29:0/355957942 >> 172.21.15.29:0/355957942 conn(0x7fdb600c7260 msgr2=0x7fdb600aad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:20.709 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.706+0000 7fdb64808700 1 -- 172.21.15.29:0/355957942 shutdown_connections 2024-01-24T15:32:20.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.709+0000 7fdb64808700 1 -- 172.21.15.29:0/355957942 wait complete. 2024-01-24T15:32:20.722 INFO:teuthology.orchestra.run.smithi029.stderr:new fs with metadata pool 26 and data pool 27 2024-01-24T15:32:20.735 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:32:20.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.863+0000 7f283f248700 1 Processor -- start 2024-01-24T15:32:20.865 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.863+0000 7f283f248700 1 -- start start 2024-01-24T15:32:20.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283f248700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2838150e50 0x7f2838069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:20.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283f248700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2838063fd0 0x7f28380c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:20.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283f248700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28380bd550 con 0x7f283806a610 2024-01-24T15:32:20.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283f248700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283806aa70 con 0x7f2838150e50 2024-01-24T15:32:20.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283f248700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28380bd0c0 con 0x7f2838063fd0 2024-01-24T15:32:20.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283cfe4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2838150e50 0x7f2838069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:20.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283cfe4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2838150e50 0x7f2838069640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53476/0 (socket says 172.21.15.29:53476) 2024-01-24T15:32:20.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283cfe4700 1 -- 172.21.15.29:0/1374686974 learned_addr learned my addr 172.21.15.29:0/1374686974 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:20.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283d7e5700 1 --2- 172.21.15.29:0/1374686974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2838063fd0 0x7f28380c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:20.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283cfe4700 1 -- 172.21.15.29:0/1374686974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2838063fd0 msgr2=0x7f28380c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:20.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283cfe4700 1 --2- 172.21.15.29:0/1374686974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2838063fd0 0x7f28380c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283cfe4700 1 -- 172.21.15.29:0/1374686974 >> v1:172.21.15.163:6789/0 conn(0x7f283806a610 legacy=0x7f28380638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:20.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.864+0000 7f283cfe4700 1 -- 172.21.15.29:0/1374686974 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28380bc1d0 con 0x7f2838150e50 2024-01-24T15:32:20.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.865+0000 7f283cfe4700 1 --2- 172.21.15.29:0/1374686974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838150e50 0x7f2838069640 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 rx=0x7f282c006fd0 tx=0x7f282c0090f0).ready entity=mon.0 client_cookie=ac77dfcb322e4fc7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:20.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.865+0000 7f28377fe700 1 -- 172.21.15.29:0/1374686974 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c00b600 con 0x7f2838150e50 2024-01-24T15:32:20.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.865+0000 7f28377fe700 1 -- 172.21.15.29:0/1374686974 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c00b760 con 0x7f2838150e50 2024-01-24T15:32:20.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.865+0000 7f28377fe700 1 -- 172.21.15.29:0/1374686974 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c00bae0 con 0x7f2838150e50 2024-01-24T15:32:20.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.865+0000 7f283f248700 1 -- 172.21.15.29:0/1374686974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838150e50 msgr2=0x7f2838069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:20.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.865+0000 7f283f248700 1 --2- 172.21.15.29:0/1374686974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838150e50 0x7f2838069640 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 rx=0x7f282c006fd0 tx=0x7f282c0090f0).stop 2024-01-24T15:32:20.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.865+0000 7f283f248700 1 -- 172.21.15.29:0/1374686974 shutdown_connections 2024-01-24T15:32:20.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.865+0000 7f283f248700 1 --2- 172.21.15.29:0/1374686974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2838063fd0 0x7f28380c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.865+0000 7f283f248700 1 --2- 172.21.15.29:0/1374686974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838150e50 0x7f2838069640 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.865+0000 7f283f248700 1 -- 172.21.15.29:0/1374686974 >> 172.21.15.29:0/1374686974 conn(0x7f2838064aa0 msgr2=0x7f2838064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:20.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.865+0000 7f283f248700 1 -- 172.21.15.29:0/1374686974 shutdown_connections 2024-01-24T15:32:20.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.865+0000 7f283f248700 1 -- 172.21.15.29:0/1374686974 wait complete. 2024-01-24T15:32:20.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.867+0000 7f283f248700 1 Processor -- start 2024-01-24T15:32:20.870 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.867+0000 7f283f248700 1 -- start start 2024-01-24T15:32:20.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283f248700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838063fd0 0x7f28380c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283f248700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f283806a610 0x7f28380c1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283f248700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2838150e50 0x7f28380c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283f248700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28380c1960 con 0x7f2838063fd0 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283f248700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28380c1b50 con 0x7f283806a610 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283f248700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28380c1e30 con 0x7f2838150e50 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283cfe4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838063fd0 0x7f28380c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283cfe4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838063fd0 0x7f28380c0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53480/0 (socket says 172.21.15.29:53480) 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283cfe4700 1 -- 172.21.15.29:0/1366738290 learned_addr learned my addr 172.21.15.29:0/1366738290 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f2837fff700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f283806a610 0x7f28380c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283d7e5700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2838150e50 0x7f28380c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283cfe4700 1 -- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2838150e50 msgr2=0x7f28380c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283cfe4700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2838150e50 0x7f28380c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283cfe4700 1 -- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f283806a610 msgr2=0x7f28380c1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:20.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283cfe4700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f283806a610 0x7f28380c1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:20.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283cfe4700 1 -- 172.21.15.29:0/1366738290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c009370 con 0x7f2838063fd0 2024-01-24T15:32:20.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f283cfe4700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838063fd0 0x7f28380c0c60 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 rx=0x7f2838068790 tx=0x7f282c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:20.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.868+0000 7f2834ff9700 1 -- 172.21.15.29:0/1366738290 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c00b8d0 con 0x7f2838063fd0 2024-01-24T15:32:20.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.869+0000 7f2834ff9700 1 -- 172.21.15.29:0/1366738290 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c00ba30 con 0x7f2838063fd0 2024-01-24T15:32:20.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.869+0000 7f283f248700 1 -- 172.21.15.29:0/1366738290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28380c2110 con 0x7f2838063fd0 2024-01-24T15:32:20.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.869+0000 7f2834ff9700 1 -- 172.21.15.29:0/1366738290 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c01b660 con 0x7f2838063fd0 2024-01-24T15:32:20.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.869+0000 7f283f248700 1 -- 172.21.15.29:0/1366738290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28380c26d0 con 0x7f2838063fd0 2024-01-24T15:32:20.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.870+0000 7f283f248700 1 -- 172.21.15.29:0/1366738290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28380475e0 con 0x7f2838063fd0 2024-01-24T15:32:20.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.870+0000 7f2834ff9700 1 -- 172.21.15.29:0/1366738290 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f282c01ba50 con 0x7f2838063fd0 2024-01-24T15:32:20.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.871+0000 7f2834ff9700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f281c05f270 0x7f281c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:20.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.871+0000 7f2834ff9700 1 -- 172.21.15.29:0/1366738290 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f282c07c480 con 0x7f2838063fd0 2024-01-24T15:32:20.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.871+0000 7f2837fff700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f281c05f270 0x7f281c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:20.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.871+0000 7f2837fff700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f281c05f270 0x7f281c061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f2824003900 tx=0x7f2824007d40).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:20.877 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:20.875+0000 7f2834ff9700 1 -- 172.21.15.29:0/1366738290 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f282c04d250 con 0x7f2838063fd0 2024-01-24T15:32:21.061 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.059+0000 7f283f248700 1 -- 172.21.15.29:0/1366738290 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28380bc4b0 con 0x7f2838063fd0 2024-01-24T15:32:21.062 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.060+0000 7f2834ff9700 1 -- 172.21.15.29:0/1366738290 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v230) v1 ==== 74+0+18315 (secure 0 0 0) 0x7f282c029070 con 0x7f2838063fd0 2024-01-24T15:32:21.062 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:21.062 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":230,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:32:20.685171+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:32:18.425709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-01-24T15:32:19.656025+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:32:21.064 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.062+0000 7f283f248700 1 -- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f281c05f270 msgr2=0x7f281c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.062+0000 7f283f248700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f281c05f270 0x7f281c061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f2824003900 tx=0x7f2824007d40).stop 2024-01-24T15:32:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.062+0000 7f283f248700 1 -- 172.21.15.29:0/1366738290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838063fd0 msgr2=0x7f28380c0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.062+0000 7f283f248700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838063fd0 0x7f28380c0c60 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 rx=0x7f2838068790 tx=0x7f282c017000).stop 2024-01-24T15:32:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.062+0000 7f283f248700 1 -- 172.21.15.29:0/1366738290 shutdown_connections 2024-01-24T15:32:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.062+0000 7f283f248700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f281c05f270 0x7f281c061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.062+0000 7f283f248700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2838150e50 0x7f28380c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.062+0000 7f283f248700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838063fd0 0x7f28380c0c60 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.062+0000 7f283f248700 1 --2- 172.21.15.29:0/1366738290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f283806a610 0x7f28380c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.062+0000 7f283f248700 1 -- 172.21.15.29:0/1366738290 >> 172.21.15.29:0/1366738290 conn(0x7f2838064aa0 msgr2=0x7f2838066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.062+0000 7f283f248700 1 -- 172.21.15.29:0/1366738290 shutdown_connections 2024-01-24T15:32:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.062+0000 7f283f248700 1 -- 172.21.15.29:0/1366738290 wait complete. 2024-01-24T15:32:21.078 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:32:21.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.207+0000 7f704a59c700 1 Processor -- start 2024-01-24T15:32:21.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.207+0000 7f704a59c700 1 -- start start 2024-01-24T15:32:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.207+0000 7f704a59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70440c4a10 0x7f70440bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.207+0000 7f704a59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70440670a0 0x7f7044158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.207+0000 7f704a59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70440664f0 0x7f70440669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.207+0000 7f704a59c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7044066e90 con 0x7f70440664f0 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.207+0000 7f704a59c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70441582b0 con 0x7f70440c4a10 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.207+0000 7f704a59c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7044157ca0 con 0x7f70440670a0 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7048b39700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70440664f0 0x7f70440669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7048b39700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f70440664f0 0x7f70440669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53486/0 (socket says 172.21.15.29:53486) 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7048b39700 1 -- 172.21.15.29:0/578366777 learned_addr learned my addr 172.21.15.29:0/578366777 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7043fff700 1 --2- 172.21.15.29:0/578366777 >> v2:172.21.15.163:3300/0 conn(0x7f70440c4a10 0x7f70440bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f70437fe700 1 --2- 172.21.15.29:0/578366777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70440670a0 0x7f7044158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7048b39700 1 -- 172.21.15.29:0/578366777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70440670a0 msgr2=0x7f7044158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7048b39700 1 --2- 172.21.15.29:0/578366777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70440670a0 0x7f7044158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7048b39700 1 -- 172.21.15.29:0/578366777 >> v2:172.21.15.163:3300/0 conn(0x7f70440c4a10 msgr2=0x7f70440bf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7048b39700 1 --2- 172.21.15.29:0/578366777 >> v2:172.21.15.163:3300/0 conn(0x7f70440c4a10 0x7f70440bf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7048b39700 1 -- 172.21.15.29:0/578366777 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7044063c60 con 0x7f70440664f0 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7048b39700 1 --2- 172.21.15.29:0/578366777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70440664f0 0x7f70440669a0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 rx=0x7f7044069ca0 tx=0x7f703c009840).ready entity=mon.0 client_cookie=d2c0644ddef45137 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7042ffd700 1 -- 172.21.15.29:0/578366777 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c00e070 con 0x7f70440664f0 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7042ffd700 1 -- 172.21.15.29:0/578366777 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c00b970 con 0x7f70440664f0 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.208+0000 7f7042ffd700 1 -- 172.21.15.29:0/578366777 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c00bcf0 con 0x7f70440664f0 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.209+0000 7f704a59c700 1 -- 172.21.15.29:0/578366777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70440664f0 msgr2=0x7f70440669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.209+0000 7f704a59c700 1 --2- 172.21.15.29:0/578366777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70440664f0 0x7f70440669a0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 rx=0x7f7044069ca0 tx=0x7f703c009840).stop 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.209+0000 7f704a59c700 1 -- 172.21.15.29:0/578366777 shutdown_connections 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.209+0000 7f704a59c700 1 --2- 172.21.15.29:0/578366777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70440664f0 0x7f70440669a0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.209+0000 7f704a59c700 1 --2- 172.21.15.29:0/578366777 >> v2:172.21.15.163:3300/0 conn(0x7f70440c4a10 0x7f70440bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.209+0000 7f704a59c700 1 --2- 172.21.15.29:0/578366777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70440670a0 0x7f7044158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.209+0000 7f704a59c700 1 -- 172.21.15.29:0/578366777 >> 172.21.15.29:0/578366777 conn(0x7f70440c8bb0 msgr2=0x7f70440c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.209+0000 7f704a59c700 1 -- 172.21.15.29:0/578366777 shutdown_connections 2024-01-24T15:32:21.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.209+0000 7f704a59c700 1 -- 172.21.15.29:0/578366777 wait complete. 2024-01-24T15:32:21.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.211+0000 7f704a59c700 1 Processor -- start 2024-01-24T15:32:21.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.211+0000 7f704a59c700 1 -- start start 2024-01-24T15:32:21.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.212+0000 7f704a59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70440664f0 0x7f70440ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.212+0000 7f704a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70440670a0 0x7f70440add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.212+0000 7f704a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70440c4a10 0x7f7044151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.212+0000 7f704a59c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70440ad3f0 con 0x7f70440670a0 2024-01-24T15:32:21.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.212+0000 7f704a59c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70440ac7e0 con 0x7f70440c4a10 2024-01-24T15:32:21.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.212+0000 7f704a59c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70440acac0 con 0x7f70440664f0 2024-01-24T15:32:21.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.212+0000 7f70437fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70440670a0 0x7f70440add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.212+0000 7f70437fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70440670a0 0x7f70440add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53494/0 (socket says 172.21.15.29:53494) 2024-01-24T15:32:21.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.212+0000 7f70437fe700 1 -- 172.21.15.29:0/2847423124 learned_addr learned my addr 172.21.15.29:0/2847423124 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.212+0000 7f7043fff700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70440664f0 0x7f70440ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.212+0000 7f7048b39700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70440c4a10 0x7f7044151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.213+0000 7f70437fe700 1 -- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70440664f0 msgr2=0x7f70440ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.213+0000 7f70437fe700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70440664f0 0x7f70440ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.213+0000 7f70437fe700 1 -- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70440c4a10 msgr2=0x7f7044151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.213+0000 7f70437fe700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70440c4a10 0x7f7044151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.213+0000 7f70437fe700 1 -- 172.21.15.29:0/2847423124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c009ae0 con 0x7f70440670a0 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.213+0000 7f70437fe700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70440670a0 0x7f70440add70 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 rx=0x7f703800e900 tx=0x7f7038009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.213+0000 7f702bfff700 1 -- 172.21.15.29:0/2847423124 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038012600 con 0x7f70440670a0 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.213+0000 7f702bfff700 1 -- 172.21.15.29:0/2847423124 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7038012760 con 0x7f70440670a0 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.213+0000 7f702bfff700 1 -- 172.21.15.29:0/2847423124 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038012ae0 con 0x7f70440670a0 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.213+0000 7f704a59c700 1 -- 172.21.15.29:0/2847423124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70440acda0 con 0x7f70440670a0 2024-01-24T15:32:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.213+0000 7f704a59c700 1 -- 172.21.15.29:0/2847423124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7044151d20 con 0x7f70440670a0 2024-01-24T15:32:21.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.214+0000 7f704a59c700 1 -- 172.21.15.29:0/2847423124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70440475e0 con 0x7f70440670a0 2024-01-24T15:32:21.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.214+0000 7f702bfff700 1 -- 172.21.15.29:0/2847423124 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f703801b400 con 0x7f70440670a0 2024-01-24T15:32:21.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.215+0000 7f702bfff700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f703005f270 0x7f7030061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.215+0000 7f702bfff700 1 -- 172.21.15.29:0/2847423124 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f703807acd0 con 0x7f70440670a0 2024-01-24T15:32:21.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.215+0000 7f7043fff700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f703005f270 0x7f7030061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.216+0000 7f7043fff700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f703005f270 0x7f7030061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7034009c30 tx=0x7f7034009060).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:21.221 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.220+0000 7f702bfff700 1 -- 172.21.15.29:0/2847423124 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f703804b980 con 0x7f70440670a0 2024-01-24T15:32:21.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.400+0000 7f704a59c700 1 -- 172.21.15.29:0/2847423124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70440ad530 con 0x7f70440670a0 2024-01-24T15:32:21.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.401+0000 7f702bfff700 1 -- 172.21.15.29:0/2847423124 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v230) v1 ==== 74+0+18315 (secure 0 0 0) 0x7f703804efa0 con 0x7f70440670a0 2024-01-24T15:32:21.404 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:21.404 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":230,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:32:20.685171+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:32:18.425709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-01-24T15:32:19.656025+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:32:21.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.403+0000 7f704a59c700 1 -- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f703005f270 msgr2=0x7f7030061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:21.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.403+0000 7f704a59c700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f703005f270 0x7f7030061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7034009c30 tx=0x7f7034009060).stop 2024-01-24T15:32:21.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.403+0000 7f704a59c700 1 -- 172.21.15.29:0/2847423124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70440670a0 msgr2=0x7f70440add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:21.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.403+0000 7f704a59c700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70440670a0 0x7f70440add70 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 rx=0x7f703800e900 tx=0x7f7038009060).stop 2024-01-24T15:32:21.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.403+0000 7f704a59c700 1 -- 172.21.15.29:0/2847423124 shutdown_connections 2024-01-24T15:32:21.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.403+0000 7f704a59c700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70440c4a10 0x7f7044151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.403+0000 7f704a59c700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f703005f270 0x7f7030061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.403+0000 7f704a59c700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70440664f0 0x7f70440ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.403+0000 7f704a59c700 1 --2- 172.21.15.29:0/2847423124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f70440670a0 0x7f70440add70 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.403+0000 7f704a59c700 1 -- 172.21.15.29:0/2847423124 >> 172.21.15.29:0/2847423124 conn(0x7f70440c8bb0 msgr2=0x7f70440be9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:21.406 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.404+0000 7f704a59c700 1 -- 172.21.15.29:0/2847423124 shutdown_connections 2024-01-24T15:32:21.407 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.404+0000 7f704a59c700 1 -- 172.21.15.29:0/2847423124 wait complete. 2024-01-24T15:32:21.419 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-24T15:32:21.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b827e700 1 Processor -- start 2024-01-24T15:32:21.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b827e700 1 -- start start 2024-01-24T15:32:21.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b827e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f59b00692a0 0x7f59b0065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b827e700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f59b0066370 0x7f59b0069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b827e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b00c3e70 con 0x7f59b0069b70 2024-01-24T15:32:21.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b827e700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b00c4300 con 0x7f59b0066370 2024-01-24T15:32:21.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b827e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b00c3260 con 0x7f59b00692a0 2024-01-24T15:32:21.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b5819700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f59b0066370 0x7f59b0069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b681b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f59b0069b70 0x7f59b00bcd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56850/0 (socket says 172.21.15.29:56850) 2024-01-24T15:32:21.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b681b700 1 -- 172.21.15.29:0/2949369134 learned_addr learned my addr 172.21.15.29:0/2949369134 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:21.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b601a700 1 --2- 172.21.15.29:0/2949369134 >> v2:172.21.15.163:3300/0 conn(0x7f59b00692a0 0x7f59b0065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b5819700 1 -- 172.21.15.29:0/2949369134 >> v2:172.21.15.163:3300/0 conn(0x7f59b00692a0 msgr2=0x7f59b0065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:21.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b5819700 1 --2- 172.21.15.29:0/2949369134 >> v2:172.21.15.163:3300/0 conn(0x7f59b00692a0 0x7f59b0065e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.552+0000 7f59b5819700 1 -- 172.21.15.29:0/2949369134 >> v1:172.21.15.29:6789/0 conn(0x7f59b0069b70 legacy=0x7f59b00bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.553+0000 7f59b5819700 1 -- 172.21.15.29:0/2949369134 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b00c3540 con 0x7f59b0066370 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.553+0000 7f59b5018700 1 -- 172.21.15.29:0/2949369134 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99332180 0 0) 0x7f59b00c3e70 con 0x7f59b0069b70 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.553+0000 7f59b5819700 1 --2- 172.21.15.29:0/2949369134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59b0066370 0x7f59b0069680 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 rx=0x7f59b005ed00 tx=0x7f59a0009060).ready entity=mon.0 client_cookie=e9e9a64c2da6b80a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.553+0000 7f59b5018700 1 -- 172.21.15.29:0/2949369134 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a000b580 con 0x7f59b0066370 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.553+0000 7f59b5018700 1 -- 172.21.15.29:0/2949369134 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a000b6e0 con 0x7f59b0066370 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.553+0000 7f59b5018700 1 -- 172.21.15.29:0/2949369134 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a000ba60 con 0x7f59b0066370 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.553+0000 7f59b827e700 1 -- 172.21.15.29:0/2949369134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59b0066370 msgr2=0x7f59b0069680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.553+0000 7f59b827e700 1 --2- 172.21.15.29:0/2949369134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59b0066370 0x7f59b0069680 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 rx=0x7f59b005ed00 tx=0x7f59a0009060).stop 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.553+0000 7f59b827e700 1 -- 172.21.15.29:0/2949369134 shutdown_connections 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.553+0000 7f59b827e700 1 --2- 172.21.15.29:0/2949369134 >> v2:172.21.15.163:3300/0 conn(0x7f59b00692a0 0x7f59b0065e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.554+0000 7f59b827e700 1 --2- 172.21.15.29:0/2949369134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59b0066370 0x7f59b0069680 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.554+0000 7f59b827e700 1 -- 172.21.15.29:0/2949369134 >> 172.21.15.29:0/2949369134 conn(0x7f59b0064390 msgr2=0x7f59b00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.554+0000 7f59b827e700 1 -- 172.21.15.29:0/2949369134 shutdown_connections 2024-01-24T15:32:21.557 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.554+0000 7f59b827e700 1 -- 172.21.15.29:0/2949369134 wait complete. 2024-01-24T15:32:21.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.555+0000 7f59b827e700 1 Processor -- start 2024-01-24T15:32:21.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.555+0000 7f59b827e700 1 -- start start 2024-01-24T15:32:21.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b827e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59b0066370 0x7f59b0152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b827e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b00692a0 0x7f59b0163c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b827e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59b0069b70 0x7f59b01510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b827e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b01526c0 con 0x7f59b0069b70 2024-01-24T15:32:21.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b827e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0164120 con 0x7f59b00692a0 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b827e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0151ab0 con 0x7f59b0066370 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b681b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59b0069b70 0x7f59b01510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b681b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59b0069b70 0x7f59b01510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53506/0 (socket says 172.21.15.29:53506) 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b681b700 1 -- 172.21.15.29:0/551786238 learned_addr learned my addr 172.21.15.29:0/551786238 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b5819700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b00692a0 0x7f59b0163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b601a700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59b0066370 0x7f59b0152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b681b700 1 -- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59b0066370 msgr2=0x7f59b0152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b681b700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59b0066370 0x7f59b0152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b681b700 1 -- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b00692a0 msgr2=0x7f59b0163c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b681b700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b00692a0 0x7f59b0163c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b681b700 1 -- 172.21.15.29:0/551786238 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a00092a0 con 0x7f59b0069b70 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b681b700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59b0069b70 0x7f59b01510d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 rx=0x7f59a800c7e0 tx=0x7f59a8007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59a67fc700 1 -- 172.21.15.29:0/551786238 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8005d30 con 0x7f59b0069b70 2024-01-24T15:32:21.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59a67fc700 1 -- 172.21.15.29:0/551786238 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a8005e90 con 0x7f59b0069b70 2024-01-24T15:32:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b827e700 1 -- 172.21.15.29:0/551786238 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b0151d90 con 0x7f59b0069b70 2024-01-24T15:32:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59a67fc700 1 -- 172.21.15.29:0/551786238 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8007320 con 0x7f59b0069b70 2024-01-24T15:32:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.556+0000 7f59b827e700 1 -- 172.21.15.29:0/551786238 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b0152350 con 0x7f59b0069b70 2024-01-24T15:32:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.558+0000 7f59b827e700 1 -- 172.21.15.29:0/551786238 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b0152830 con 0x7f59b0069b70 2024-01-24T15:32:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.558+0000 7f59a67fc700 1 -- 172.21.15.29:0/551786238 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f59a8007710 con 0x7f59b0069b70 2024-01-24T15:32:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.558+0000 7f59a67fc700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f599405f240 0x7f5994061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.558+0000 7f59a67fc700 1 -- 172.21.15.29:0/551786238 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f59a807a960 con 0x7f59b0069b70 2024-01-24T15:32:21.562 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.558+0000 7f59b601a700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f599405f240 0x7f5994061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:21.563 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.559+0000 7f59b601a700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f599405f240 0x7f5994061700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f59ac00bb20 tx=0x7f59ac008040).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:21.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.564+0000 7f59a67fc700 1 -- 172.21.15.29:0/551786238 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f59a804b5b0 con 0x7f59b0069b70 2024-01-24T15:32:21.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:21.791+0000 7f59b827e700 1 -- 172.21.15.29:0/551786238 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f59b00475e0 con 0x7f59b0069b70 2024-01-24T15:32:22.708 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.707+0000 7f59a67fc700 1 -- 172.21.15.29:0/551786238 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v97) v1 ==== 118+0+0 (secure 0 0 0) 0x7f59a804ebd0 con 0x7f59b0069b70 2024-01-24T15:32:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.709+0000 7f59b827e700 1 -- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f599405f240 msgr2=0x7f5994061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.709+0000 7f59b827e700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f599405f240 0x7f5994061700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f59ac00bb20 tx=0x7f59ac008040).stop 2024-01-24T15:32:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.709+0000 7f59b827e700 1 -- 172.21.15.29:0/551786238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59b0069b70 msgr2=0x7f59b01510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.709+0000 7f59b827e700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59b0069b70 0x7f59b01510d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 rx=0x7f59a800c7e0 tx=0x7f59a8007f90).stop 2024-01-24T15:32:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.710+0000 7f59b827e700 1 -- 172.21.15.29:0/551786238 shutdown_connections 2024-01-24T15:32:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.710+0000 7f59b827e700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59b0069b70 0x7f59b01510d0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:22.711 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.710+0000 7f59b827e700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f599405f240 0x7f5994061700 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:22.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.710+0000 7f59b827e700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59b0066370 0x7f59b0152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:22.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.710+0000 7f59b827e700 1 --2- 172.21.15.29:0/551786238 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b00692a0 0x7f59b0163c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:22.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.710+0000 7f59b827e700 1 -- 172.21.15.29:0/551786238 >> 172.21.15.29:0/551786238 conn(0x7f59b0064390 msgr2=0x7f59b00c5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:22.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.710+0000 7f59b827e700 1 -- 172.21.15.29:0/551786238 shutdown_connections 2024-01-24T15:32:22.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.710+0000 7f59b827e700 1 -- 172.21.15.29:0/551786238 wait complete. 2024-01-24T15:32:22.726 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-24T15:32:22.726 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-24T15:32:22.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.863+0000 7fdaba615700 1 Processor -- start 2024-01-24T15:32:22.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.863+0000 7fdaba615700 1 -- start start 2024-01-24T15:32:22.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.864+0000 7fdaba615700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdab41599f0 0x7fdab415bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:22.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.864+0000 7fdaba615700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab40c3e70 con 0x7fdab40692a0 2024-01-24T15:32:22.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.864+0000 7fdaba615700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab40c4300 con 0x7fdab4069680 2024-01-24T15:32:22.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.864+0000 7fdaba615700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab40c3260 con 0x7fdab41599f0 2024-01-24T15:32:22.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.864+0000 7fdab3fff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdab40692a0 0x7fdab4065e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:56860/0 (socket says 172.21.15.29:56860) 2024-01-24T15:32:22.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.864+0000 7fdab3fff700 1 -- 172.21.15.29:0/2115868281 learned_addr learned my addr 172.21.15.29:0/2115868281 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:22.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab8bb2700 1 --2- 172.21.15.29:0/2115868281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdab41599f0 0x7fdab415bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:22.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab2ffd700 1 -- 172.21.15.29:0/2115868281 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948230588 0 0) 0x7fdab40c3e70 con 0x7fdab40692a0 2024-01-24T15:32:22.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab2ffd700 1 -- 172.21.15.29:0/2115868281 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda9c0033f0 con 0x7fdab40692a0 2024-01-24T15:32:22.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab2ffd700 1 -- 172.21.15.29:0/2115868281 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775291407 0 0) 0x7fdab40c4300 con 0x7fdab4069680 2024-01-24T15:32:22.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab2ffd700 1 -- 172.21.15.29:0/2115868281 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab40c3e70 con 0x7fdab4069680 2024-01-24T15:32:22.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab2ffd700 1 -- 172.21.15.29:0/2115868281 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1831433351 0 0) 0x7fda9c0033f0 con 0x7fdab40692a0 2024-01-24T15:32:22.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab8bb2700 1 -- 172.21.15.29:0/2115868281 >> v1:172.21.15.163:6789/0 conn(0x7fdab4069680 legacy=0x7fdab4069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:22.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab8bb2700 1 -- 172.21.15.29:0/2115868281 >> v1:172.21.15.29:6789/0 conn(0x7fdab40692a0 legacy=0x7fdab4065e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:22.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab8bb2700 1 -- 172.21.15.29:0/2115868281 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab40c3540 con 0x7fdab41599f0 2024-01-24T15:32:22.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab8bb2700 1 --2- 172.21.15.29:0/2115868281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdab41599f0 0x7fdab415bde0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fdaac009e90 tx=0x7fdaac009780).ready entity=mon.2 client_cookie=a69873182af94503 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:22.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab2ffd700 1 -- 172.21.15.29:0/2115868281 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac00e070 con 0x7fdab41599f0 2024-01-24T15:32:22.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab2ffd700 1 -- 172.21.15.29:0/2115868281 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaac00b7a0 con 0x7fdab41599f0 2024-01-24T15:32:22.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.865+0000 7fdab2ffd700 1 -- 172.21.15.29:0/2115868281 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4001470 con 0x7fdab41599f0 2024-01-24T15:32:22.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.866+0000 7fdaba615700 1 -- 172.21.15.29:0/2115868281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdab41599f0 msgr2=0x7fdab415bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:22.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.866+0000 7fdaba615700 1 --2- 172.21.15.29:0/2115868281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdab41599f0 0x7fdab415bde0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fdaac009e90 tx=0x7fdaac009780).stop 2024-01-24T15:32:22.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.866+0000 7fdaba615700 1 -- 172.21.15.29:0/2115868281 shutdown_connections 2024-01-24T15:32:22.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.866+0000 7fdaba615700 1 --2- 172.21.15.29:0/2115868281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdab41599f0 0x7fdab415bde0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:22.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.866+0000 7fdaba615700 1 -- 172.21.15.29:0/2115868281 >> 172.21.15.29:0/2115868281 conn(0x7fdab4064390 msgr2=0x7fdab40c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:22.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.866+0000 7fdaba615700 1 -- 172.21.15.29:0/2115868281 shutdown_connections 2024-01-24T15:32:22.869 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.866+0000 7fdaba615700 1 -- 172.21.15.29:0/2115868281 wait complete. 2024-01-24T15:32:22.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.867+0000 7fdaba615700 1 Processor -- start 2024-01-24T15:32:22.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.867+0000 7fdaba615700 1 -- start start 2024-01-24T15:32:22.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.868+0000 7fdaba615700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab40692a0 0x7fdab4148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.868+0000 7fdaba615700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdab4069680 0x7fdab40bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.868+0000 7fdaba615700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdab41599f0 0x7fdab40bd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.868+0000 7fdaba615700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab4146ed0 con 0x7fdab4069680 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.868+0000 7fdaba615700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab40bdba0 con 0x7fdab40692a0 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.868+0000 7fdaba615700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab4147fe0 con 0x7fdab41599f0 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.868+0000 7fdab37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdab4069680 0x7fdab40bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.868+0000 7fdab37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdab4069680 0x7fdab40bd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:53516/0 (socket says 172.21.15.29:53516) 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.868+0000 7fdab37fe700 1 -- 172.21.15.29:0/4027815090 learned_addr learned my addr 172.21.15.29:0/4027815090 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fdab8bb2700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdab41599f0 0x7fdab40bd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fdab3fff700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab40692a0 0x7fdab4148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fdab37fe700 1 -- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdab41599f0 msgr2=0x7fdab40bd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fdab37fe700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdab41599f0 0x7fdab40bd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fdab37fe700 1 -- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab40692a0 msgr2=0x7fdab4148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:22.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fdab37fe700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab40692a0 0x7fdab4148470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:22.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fdab37fe700 1 -- 172.21.15.29:0/4027815090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaac009a20 con 0x7fdab4069680 2024-01-24T15:32:22.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fdab37fe700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdab4069680 0x7fdab40bd1c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7fdaa800c4a0 tx=0x7fdaa8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:22.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fda9bfff700 1 -- 172.21.15.29:0/4027815090 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa800f470 con 0x7fdab4069680 2024-01-24T15:32:22.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fda9bfff700 1 -- 172.21.15.29:0/4027815090 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa800f5d0 con 0x7fdab4069680 2024-01-24T15:32:22.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fdaba615700 1 -- 172.21.15.29:0/4027815090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab41473d0 con 0x7fdab4069680 2024-01-24T15:32:22.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fda9bfff700 1 -- 172.21.15.29:0/4027815090 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa800f970 con 0x7fdab4069680 2024-01-24T15:32:22.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.869+0000 7fdaba615700 1 -- 172.21.15.29:0/4027815090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab4147990 con 0x7fdab4069680 2024-01-24T15:32:22.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.870+0000 7fdaba615700 1 -- 172.21.15.29:0/4027815090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab40475e0 con 0x7fdab4069680 2024-01-24T15:32:22.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.870+0000 7fda9bfff700 1 -- 172.21.15.29:0/4027815090 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdaa800fce0 con 0x7fdab4069680 2024-01-24T15:32:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.871+0000 7fda9bfff700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdaa005f270 0x7fdaa0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.871+0000 7fda9bfff700 1 -- 172.21.15.29:0/4027815090 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7fdaa8079ec0 con 0x7fdab4069680 2024-01-24T15:32:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.871+0000 7fdab3fff700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdaa005f270 0x7fdaa0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:22.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.872+0000 7fdab3fff700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdaa005f270 0x7fdaa0061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fdaa4001ed0 tx=0x7fdaa4000bc0).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:22.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:22.875+0000 7fda9bfff700 1 -- 172.21.15.29:0/4027815090 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdaa804ab70 con 0x7fdab4069680 2024-01-24T15:32:23.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.102+0000 7fdaba615700 1 -- 172.21.15.29:0/4027815090 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdab4148120 con 0x7fdab4069680 2024-01-24T15:32:23.712 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.711+0000 7fda9bfff700 1 -- 172.21.15.29:0/4027815090 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v98) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdaa804e190 con 0x7fdab4069680 2024-01-24T15:32:23.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.714+0000 7fdaba615700 1 -- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdaa005f270 msgr2=0x7fdaa0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:23.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.714+0000 7fdaba615700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdaa005f270 0x7fdaa0061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fdaa4001ed0 tx=0x7fdaa4000bc0).stop 2024-01-24T15:32:23.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.714+0000 7fdaba615700 1 -- 172.21.15.29:0/4027815090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdab4069680 msgr2=0x7fdab40bd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:23.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.714+0000 7fdaba615700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdab4069680 0x7fdab40bd1c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7fdaa800c4a0 tx=0x7fdaa8007c50).stop 2024-01-24T15:32:23.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.714+0000 7fdaba615700 1 -- 172.21.15.29:0/4027815090 shutdown_connections 2024-01-24T15:32:23.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.714+0000 7fdaba615700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdaa005f270 0x7fdaa0061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:23.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.714+0000 7fdaba615700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdab41599f0 0x7fdab40bd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:23.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.714+0000 7fdaba615700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab40692a0 0x7fdab4148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:23.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.714+0000 7fdaba615700 1 --2- 172.21.15.29:0/4027815090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdab4069680 0x7fdab40bd1c0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:23.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.714+0000 7fdaba615700 1 -- 172.21.15.29:0/4027815090 >> 172.21.15.29:0/4027815090 conn(0x7fdab4064390 msgr2=0x7fdab40c69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:23.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.714+0000 7fdaba615700 1 -- 172.21.15.29:0/4027815090 shutdown_connections 2024-01-24T15:32:23.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.714+0000 7fdaba615700 1 -- 172.21.15.29:0/4027815090 wait complete. 2024-01-24T15:32:23.731 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-24T15:32:23.731 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.863+0000 7f6db8dbb700 1 Processor -- start 2024-01-24T15:32:23.864 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.863+0000 7f6db8dbb700 1 -- start start 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.864+0000 7f6db8dbb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6db414f710 0x7f6db414faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.864+0000 7f6db8dbb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db4068400 0x7f6db41559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.864+0000 7f6db8dbb700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db4066720 con 0x7f6db40c5000 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.864+0000 7f6db8dbb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db4068b20 con 0x7f6db414f710 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.864+0000 7f6db8dbb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40c5480 con 0x7f6db4068400 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.864+0000 7f6db1d9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f6db40c5000 0x7f6db40bdb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54206/0 (socket says 172.21.15.29:54206) 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.864+0000 7f6db1d9b700 1 -- 172.21.15.29:0/2265571182 learned_addr learned my addr 172.21.15.29:0/2265571182 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.865+0000 7f6db2d9d700 1 --2- 172.21.15.29:0/2265571182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db4068400 0x7f6db41559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.865+0000 7f6db259c700 1 --2- 172.21.15.29:0/2265571182 >> v2:172.21.15.163:3300/0 conn(0x7f6db414f710 0x7f6db414faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.865+0000 7f6db159a700 1 -- 172.21.15.29:0/2265571182 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34468780 0 0) 0x7f6db4066720 con 0x7f6db40c5000 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.865+0000 7f6db159a700 1 -- 172.21.15.29:0/2265571182 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6da00033f0 con 0x7f6db40c5000 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.865+0000 7f6db2d9d700 1 -- 172.21.15.29:0/2265571182 >> v2:172.21.15.163:3300/0 conn(0x7f6db414f710 msgr2=0x7f6db414faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.865+0000 7f6db2d9d700 1 --2- 172.21.15.29:0/2265571182 >> v2:172.21.15.163:3300/0 conn(0x7f6db414f710 0x7f6db414faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:23.867 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.865+0000 7f6db159a700 1 -- 172.21.15.29:0/2265571182 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 70005769 0 0) 0x7f6da00033f0 con 0x7f6db40c5000 2024-01-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.865+0000 7f6db2d9d700 1 -- 172.21.15.29:0/2265571182 >> v1:172.21.15.29:6789/0 conn(0x7f6db40c5000 legacy=0x7f6db40bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.865+0000 7f6db2d9d700 1 -- 172.21.15.29:0/2265571182 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db40be220 con 0x7f6db4068400 2024-01-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.865+0000 7f6db259c700 1 --2- 172.21.15.29:0/2265571182 >> v2:172.21.15.163:3300/0 conn(0x7f6db414f710 0x7f6db414faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.865+0000 7f6db2d9d700 1 --2- 172.21.15.29:0/2265571182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db4068400 0x7f6db41559b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f6dac009e60 tx=0x7f6dac009780).ready entity=mon.2 client_cookie=481f4119d75a2e72 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.865+0000 7f6db159a700 1 -- 172.21.15.29:0/2265571182 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00e070 con 0x7f6db4068400 2024-01-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.866+0000 7f6db159a700 1 -- 172.21.15.29:0/2265571182 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac00b7a0 con 0x7f6db4068400 2024-01-24T15:32:23.868 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.866+0000 7f6db159a700 1 -- 172.21.15.29:0/2265571182 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4001470 con 0x7f6db4068400 2024-01-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.866+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2265571182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db4068400 msgr2=0x7f6db41559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.866+0000 7f6db8dbb700 1 --2- 172.21.15.29:0/2265571182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db4068400 0x7f6db41559b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f6dac009e60 tx=0x7f6dac009780).stop 2024-01-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.866+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2265571182 shutdown_connections 2024-01-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.866+0000 7f6db8dbb700 1 --2- 172.21.15.29:0/2265571182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db4068400 0x7f6db41559b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.866+0000 7f6db8dbb700 1 --2- 172.21.15.29:0/2265571182 >> v2:172.21.15.163:3300/0 conn(0x7f6db414f710 0x7f6db414faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.866+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2265571182 >> 172.21.15.29:0/2265571182 conn(0x7f6db40c8bb0 msgr2=0x7f6db40c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.866+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2265571182 shutdown_connections 2024-01-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.866+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2265571182 wait complete. 2024-01-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.867+0000 7f6db8dbb700 1 Processor -- start 2024-01-24T15:32:23.871 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.867+0000 7f6db8dbb700 1 -- start start 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.868+0000 7f6db8dbb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db4068400 0x7f6db4165260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.868+0000 7f6db8dbb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db40c5000 0x7f6db40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.868+0000 7f6db8dbb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db414f710 0x7f6db4167660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.868+0000 7f6db8dbb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40a6d90 con 0x7f6db414f710 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.868+0000 7f6db8dbb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4150820 con 0x7f6db40c5000 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.868+0000 7f6db8dbb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40a6ab0 con 0x7f6db4068400 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.868+0000 7f6db2d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db414f710 0x7f6db4167660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.868+0000 7f6db2d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db414f710 0x7f6db4167660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55164/0 (socket says 172.21.15.29:55164) 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.868+0000 7f6db2d9d700 1 -- 172.21.15.29:0/2006443861 learned_addr learned my addr 172.21.15.29:0/2006443861 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6db259c700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db4068400 0x7f6db4165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6db1d9b700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db40c5000 0x7f6db40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6db2d9d700 1 -- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db4068400 msgr2=0x7f6db4165260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6db2d9d700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db4068400 0x7f6db4165260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:23.872 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6db2d9d700 1 -- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db40c5000 msgr2=0x7f6db40a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6db2d9d700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db40c5000 0x7f6db40a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6db2d9d700 1 -- 172.21.15.29:0/2006443861 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c0092a0 con 0x7f6db414f710 2024-01-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6db1d9b700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db40c5000 0x7f6db40a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6db2d9d700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db414f710 0x7f6db4167660 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7f6dac00b610 tx=0x7f6dac009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6daa7fc700 1 -- 172.21.15.29:0/2006443861 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac01d420 con 0x7f6db414f710 2024-01-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6daa7fc700 1 -- 172.21.15.29:0/2006443861 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac01d580 con 0x7f6db414f710 2024-01-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2006443861 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dac009a20 con 0x7f6db414f710 2024-01-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6daa7fc700 1 -- 172.21.15.29:0/2006443861 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac01d940 con 0x7f6db414f710 2024-01-24T15:32:23.873 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.869+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2006443861 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db41504b0 con 0x7f6db414f710 2024-01-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.870+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2006443861 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db4150030 con 0x7f6db414f710 2024-01-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.871+0000 7f6daa7fc700 1 -- 172.21.15.29:0/2006443861 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6dac003040 con 0x7f6db414f710 2024-01-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.871+0000 7f6daa7fc700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f6d9405f270 0x7f6d94061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.871+0000 7f6daa7fc700 1 -- 172.21.15.29:0/2006443861 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f6dac07c360 con 0x7f6db414f710 2024-01-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.871+0000 7f6db259c700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f6d9405f270 0x7f6d94061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:23.874 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.872+0000 7f6db259c700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f6d9405f270 0x7f6d94061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f6db414e3c0 tx=0x7f6d9c009060).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:23.876 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:23.876+0000 7f6daa7fc700 1 -- 172.21.15.29:0/2006443861 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f6db4150030 con 0x7f6db414f710 2024-01-24T15:32:24.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.102+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2006443861 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6db4150030 con 0x7f6db414f710 2024-01-24T15:32:24.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.103+0000 7f6daa7fc700 1 -- 172.21.15.29:0/2006443861 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4499 (secure 0 0 0) 0x7f6db4150030 con 0x7f6db414f710 2024-01-24T15:32:24.104 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:24.104 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1883755467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1883755467},{"type":"v1","addr":"172.21.15.163:6836","nonce":1883755467}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1773668659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1773668659},{"type":"v1","addr":"172.21.15.29:6834","nonce":1773668659}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3438722509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3438722509},{"type":"v1","addr":"172.21.15.29:6835","nonce":3438722509}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:32:20.685536+0000","modified":"2024-01-24T15:32:23.705717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10029":{"gid":10029,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/532321390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":532321390},{"type":"v1","addr":"172.21.15.163:6837","nonce":532321390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2024-01-24T15:32:24.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.104+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f6d9405f270 msgr2=0x7f6d94061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.104+0000 7f6db8dbb700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f6d9405f270 0x7f6d94061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f6db414e3c0 tx=0x7f6d9c009060).stop 2024-01-24T15:32:24.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.104+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2006443861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db414f710 msgr2=0x7f6db4167660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.104+0000 7f6db8dbb700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db414f710 0x7f6db4167660 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7f6dac00b610 tx=0x7f6dac009780).stop 2024-01-24T15:32:24.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.104+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2006443861 shutdown_connections 2024-01-24T15:32:24.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.104+0000 7f6db8dbb700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6db414f710 0x7f6db4167660 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.105+0000 7f6db8dbb700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f6d9405f270 0x7f6d94061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.105+0000 7f6db8dbb700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db4068400 0x7f6db4165260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.105+0000 7f6db8dbb700 1 --2- 172.21.15.29:0/2006443861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db40c5000 0x7f6db40a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.105+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2006443861 >> 172.21.15.29:0/2006443861 conn(0x7f6db40c8bb0 msgr2=0x7f6db4066bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:24.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.105+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2006443861 shutdown_connections 2024-01-24T15:32:24.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.105+0000 7f6db8dbb700 1 -- 172.21.15.29:0/2006443861 wait complete. 2024-01-24T15:32:24.107 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 98 2024-01-24T15:32:24.119 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:32:24.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.248+0000 7f9e10425700 1 Processor -- start 2024-01-24T15:32:24.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.248+0000 7f9e10425700 1 -- start start 2024-01-24T15:32:24.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.249+0000 7f9e10425700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08153f40 0x7f9e08069ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.249+0000 7f9e10425700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e080c4f40 con 0x7f9e08154320 2024-01-24T15:32:24.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.249+0000 7f9e10425700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e080c53d0 con 0x7f9e080627b0 2024-01-24T15:32:24.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.249+0000 7f9e10425700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e080bd2f0 con 0x7f9e08153f40 2024-01-24T15:32:24.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d9c0700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9e08154320 0x7f9e080a6db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54210/0 (socket says 172.21.15.29:54210) 2024-01-24T15:32:24.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d9c0700 1 -- 172.21.15.29:0/1796353228 learned_addr learned my addr 172.21.15.29:0/1796353228 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:24.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0e1c1700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08153f40 0x7f9e08069ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918412943 0 0) 0x7f9e080c4f40 con 0x7f9e08154320 2024-01-24T15:32:24.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9df40033f0 con 0x7f9e08154320 2024-01-24T15:32:24.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2909092250 0 0) 0x7f9df40033f0 con 0x7f9e08154320 2024-01-24T15:32:24.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08153f40 msgr2=0x7f9e08069ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:24.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08153f40 0x7f9e08069ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 >> v1:172.21.15.163:6789/0 conn(0x7f9e080627b0 legacy=0x7f9e08065360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e080bca50 con 0x7f9e08154320 2024-01-24T15:32:24.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f9e04001470 con 0x7f9e08154320 2024-01-24T15:32:24.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 >> v1:172.21.15.29:6789/0 conn(0x7f9e08154320 legacy=0x7f9e080a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9df4003910 0x7f9df4003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4004690 0x7f9df4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08153f40 0x7f9df4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df400b900 con 0x7f9df4003910 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df400ba70 con 0x7f9df4004690 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df400bc30 con 0x7f9e08153f40 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.250+0000 7f9e0e1c1700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9df4003910 0x7f9df4003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.251+0000 7f9e0d9c0700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4004690 0x7f9df4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.251+0000 7f9e0e9c2700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08153f40 0x7f9df4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.251+0000 7f9e0e1c1700 1 -- 172.21.15.29:0/1796353228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08153f40 msgr2=0x7f9df4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.251+0000 7f9e0e1c1700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08153f40 0x7f9df4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.251+0000 7f9e0e1c1700 1 -- 172.21.15.29:0/1796353228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4004690 msgr2=0x7f9df4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.251+0000 7f9e0e1c1700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4004690 0x7f9df4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.251+0000 7f9e0e1c1700 1 -- 172.21.15.29:0/1796353228 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df40033f0 con 0x7f9df4003910 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.251+0000 7f9e0e9c2700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08153f40 0x7f9df4005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:24.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.251+0000 7f9e0e1c1700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9df4003910 0x7f9df4003cf0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 rx=0x7f9dfc009a80 tx=0x7f9dfc009060).ready entity=mon.0 client_cookie=1bf4afa64a9d57f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.251+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc009430 con 0x7f9df4003910 2024-01-24T15:32:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.251+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dfc00e640 con 0x7f9df4003910 2024-01-24T15:32:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.251+0000 7f9e0d1bf700 1 -- 172.21.15.29:0/1796353228 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc00e9c0 con 0x7f9df4003910 2024-01-24T15:32:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.252+0000 7f9e10425700 1 -- 172.21.15.29:0/1796353228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9df4003910 msgr2=0x7f9df4003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.252+0000 7f9e10425700 1 --2- 172.21.15.29:0/1796353228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9df4003910 0x7f9df4003cf0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 rx=0x7f9dfc009a80 tx=0x7f9dfc009060).stop 2024-01-24T15:32:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.252+0000 7f9e0e9c2700 1 -- 172.21.15.29:0/1796353228 reap_dead start 2024-01-24T15:32:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.252+0000 7f9e10425700 1 -- 172.21.15.29:0/1796353228 shutdown_connections 2024-01-24T15:32:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.252+0000 7f9e10425700 1 -- 172.21.15.29:0/1796353228 >> 172.21.15.29:0/1796353228 conn(0x7f9e080c85c0 msgr2=0x7f9e080c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.252+0000 7f9e10425700 1 -- 172.21.15.29:0/1796353228 shutdown_connections 2024-01-24T15:32:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.252+0000 7f9e10425700 1 -- 172.21.15.29:0/1796353228 wait complete. 2024-01-24T15:32:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.253+0000 7f9e10425700 1 Processor -- start 2024-01-24T15:32:24.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.253+0000 7f9e10425700 1 -- start start 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e10425700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4003910 0x7f9e08165180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e10425700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e080ac230 0x7f9e080ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e10425700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08065360 0x7f9e080657e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e10425700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e08168750 con 0x7f9e080ac230 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e10425700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e08065cd0 con 0x7f9df4003910 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e10425700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e080accf0 con 0x7f9e08065360 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e0d9c0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e080ac230 0x7f9e080ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e0d9c0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e080ac230 0x7f9e080ac6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55174/0 (socket says 172.21.15.29:55174) 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e0d9c0700 1 -- 172.21.15.29:0/2807554493 learned_addr learned my addr 172.21.15.29:0/2807554493 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e0e1c1700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4003910 0x7f9e08165180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e0e9c2700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08065360 0x7f9e080657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e0d9c0700 1 -- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08065360 msgr2=0x7f9e080657e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e0d9c0700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08065360 0x7f9e080657e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e0d9c0700 1 -- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4003910 msgr2=0x7f9e08165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.259 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e0d9c0700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4003910 0x7f9e08165180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.254+0000 7f9e0d9c0700 1 -- 172.21.15.29:0/2807554493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e08167e20 con 0x7f9e080ac230 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.255+0000 7f9e0d9c0700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e080ac230 0x7f9e080ac6b0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 rx=0x7f9e040049e0 tx=0x7f9e04001e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.255+0000 7f9dfa7fc700 1 -- 172.21.15.29:0/2807554493 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04001470 con 0x7f9e080ac230 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.255+0000 7f9dfa7fc700 1 -- 172.21.15.29:0/2807554493 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e04000f70 con 0x7f9e080ac230 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.255+0000 7f9e10425700 1 -- 172.21.15.29:0/2807554493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e08168100 con 0x7f9e080ac230 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.255+0000 7f9dfa7fc700 1 -- 172.21.15.29:0/2807554493 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0400faa0 con 0x7f9e080ac230 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.255+0000 7f9e10425700 1 -- 172.21.15.29:0/2807554493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e08168be0 con 0x7f9e080ac230 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.256+0000 7f9e10425700 1 -- 172.21.15.29:0/2807554493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e080475e0 con 0x7f9e080ac230 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.256+0000 7f9dfa7fc700 1 -- 172.21.15.29:0/2807554493 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9e0401d3b0 con 0x7f9e080ac230 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.256+0000 7f9dfa7fc700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9dec05f270 0x7f9dec061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.257+0000 7f9dfa7fc700 1 -- 172.21.15.29:0/2807554493 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f9e0407a2d0 con 0x7f9e080ac230 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.257+0000 7f9e0e1c1700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9dec05f270 0x7f9dec061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.260 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.257+0000 7f9e0e1c1700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9dec05f270 0x7f9dec061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9dfc006d50 tx=0x7f9dfc01a000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:24.262 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.261+0000 7f9dfa7fc700 1 -- 172.21.15.29:0/2807554493 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9e0404e9b0 con 0x7f9e080ac230 2024-01-24T15:32:24.447 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.446+0000 7f9e10425700 1 -- 172.21.15.29:0/2807554493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e08069ca0 con 0x7f9e080ac230 2024-01-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.447+0000 7f9dfa7fc700 1 -- 172.21.15.29:0/2807554493 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v231) v1 ==== 74+0+18302 (secure 0 0 0) 0x7f9e04019070 con 0x7f9e080ac230 2024-01-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:24.448 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":231,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:32:21.687225+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:32:18.425709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-01-24T15:32:19.656025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"231","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.449+0000 7f9e10425700 1 -- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9dec05f270 msgr2=0x7f9dec061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.449+0000 7f9e10425700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9dec05f270 0x7f9dec061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9dfc006d50 tx=0x7f9dfc01a000).stop 2024-01-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.449+0000 7f9e10425700 1 -- 172.21.15.29:0/2807554493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e080ac230 msgr2=0x7f9e080ac6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.449+0000 7f9e10425700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e080ac230 0x7f9e080ac6b0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 rx=0x7f9e040049e0 tx=0x7f9e04001e70).stop 2024-01-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.449+0000 7f9e10425700 1 -- 172.21.15.29:0/2807554493 shutdown_connections 2024-01-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.449+0000 7f9e10425700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9dec05f270 0x7f9dec061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.449+0000 7f9e10425700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e08065360 0x7f9e080657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.449+0000 7f9e10425700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4003910 0x7f9e08165180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.449+0000 7f9e10425700 1 --2- 172.21.15.29:0/2807554493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e080ac230 0x7f9e080ac6b0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.449+0000 7f9e10425700 1 -- 172.21.15.29:0/2807554493 >> 172.21.15.29:0/2807554493 conn(0x7f9e080c85c0 msgr2=0x7f9e0814ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.449+0000 7f9e10425700 1 -- 172.21.15.29:0/2807554493 shutdown_connections 2024-01-24T15:32:24.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.449+0000 7f9e10425700 1 -- 172.21.15.29:0/2807554493 wait complete. 2024-01-24T15:32:24.464 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-24T15:32:24.464 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:32:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.592+0000 7faac8b07700 1 Processor -- start 2024-01-24T15:32:24.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.593+0000 7faac8b07700 1 -- start start 2024-01-24T15:32:24.596 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac8b07700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac4150d40 0x7faac4069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac8b07700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac40c2ce0 con 0x7faac4065ec0 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac8b07700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac4069890 con 0x7faac4155040 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac8b07700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac4069b70 con 0x7faac4150d40 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac1d9b700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7faac4065ec0 0x7faac4154930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54216/0 (socket says 172.21.15.29:54216) 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac1d9b700 1 -- 172.21.15.29:0/659862506 learned_addr learned my addr 172.21.15.29:0/659862506 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac259c700 1 --2- 172.21.15.29:0/659862506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac4150d40 0x7faac4069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac159a700 1 -- 172.21.15.29:0/659862506 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908682201 0 0) 0x7faac40c2ce0 con 0x7faac4065ec0 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac159a700 1 -- 172.21.15.29:0/659862506 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faab00033f0 con 0x7faac4065ec0 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac159a700 1 -- 172.21.15.29:0/659862506 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803483072 0 0) 0x7faac4069890 con 0x7faac4155040 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac159a700 1 -- 172.21.15.29:0/659862506 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faac40c2ce0 con 0x7faac4155040 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac159a700 1 -- 172.21.15.29:0/659862506 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3485099293 0 0) 0x7faab00033f0 con 0x7faac4065ec0 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac259c700 1 -- 172.21.15.29:0/659862506 >> v1:172.21.15.163:6789/0 conn(0x7faac4155040 legacy=0x7faac40c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.597 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac259c700 1 -- 172.21.15.29:0/659862506 >> v1:172.21.15.29:6789/0 conn(0x7faac4065ec0 legacy=0x7faac4154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.594+0000 7faac259c700 1 -- 172.21.15.29:0/659862506 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac4069e50 con 0x7faac4150d40 2024-01-24T15:32:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.595+0000 7faac259c700 1 --2- 172.21.15.29:0/659862506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac4150d40 0x7faac4069240 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7faab4009790 tx=0x7faab4009060).ready entity=mon.2 client_cookie=e67e71c1d4c6a4ff server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.595+0000 7faac159a700 1 -- 172.21.15.29:0/659862506 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac001470 con 0x7faac4150d40 2024-01-24T15:32:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.595+0000 7faac159a700 1 -- 172.21.15.29:0/659862506 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab4011400 con 0x7faac4150d40 2024-01-24T15:32:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.595+0000 7faac159a700 1 -- 172.21.15.29:0/659862506 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4011780 con 0x7faac4150d40 2024-01-24T15:32:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.595+0000 7faac8b07700 1 -- 172.21.15.29:0/659862506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac4150d40 msgr2=0x7faac4069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.595+0000 7faac8b07700 1 --2- 172.21.15.29:0/659862506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac4150d40 0x7faac4069240 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7faab4009790 tx=0x7faab4009060).stop 2024-01-24T15:32:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.595+0000 7faac8b07700 1 -- 172.21.15.29:0/659862506 shutdown_connections 2024-01-24T15:32:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.595+0000 7faac8b07700 1 --2- 172.21.15.29:0/659862506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac4150d40 0x7faac4069240 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.595+0000 7faac8b07700 1 -- 172.21.15.29:0/659862506 >> 172.21.15.29:0/659862506 conn(0x7faac40c8580 msgr2=0x7faac40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.595+0000 7faac8b07700 1 -- 172.21.15.29:0/659862506 shutdown_connections 2024-01-24T15:32:24.598 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.596+0000 7faac8b07700 1 -- 172.21.15.29:0/659862506 wait complete. 2024-01-24T15:32:24.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.597+0000 7faac8b07700 1 Processor -- start 2024-01-24T15:32:24.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.597+0000 7faac8b07700 1 -- start start 2024-01-24T15:32:24.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.597+0000 7faac8b07700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac4065ec0 0x7faac40beb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.602 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.597+0000 7faac8b07700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac4150d40 0x7faac4166980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac8b07700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac4155040 0x7faac413e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.603 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac8b07700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4166e70 con 0x7faac4150d40 2024-01-24T15:32:24.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac8b07700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac41512c0 con 0x7faac4155040 2024-01-24T15:32:24.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac8b07700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac414d250 con 0x7faac4065ec0 2024-01-24T15:32:24.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac1d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac4150d40 0x7faac4166980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac1d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac4150d40 0x7faac4166980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55184/0 (socket says 172.21.15.29:55184) 2024-01-24T15:32:24.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac1d9b700 1 -- 172.21.15.29:0/298271530 learned_addr learned my addr 172.21.15.29:0/298271530 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:24.604 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac259c700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac4065ec0 0x7faac40beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac2d9d700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac4155040 0x7faac413e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac1d9b700 1 -- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac4065ec0 msgr2=0x7faac40beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac1d9b700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac4065ec0 0x7faac40beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.605 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac1d9b700 1 -- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac4155040 msgr2=0x7faac413e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac1d9b700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac4155040 0x7faac413e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac1d9b700 1 -- 172.21.15.29:0/298271530 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab40092a0 con 0x7faac4150d40 2024-01-24T15:32:24.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac259c700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac4065ec0 0x7faac40beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:24.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac1d9b700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac4150d40 0x7faac4166980 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 rx=0x7faaac00c810 tx=0x7faaac001960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:24.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faaba7fc700 1 -- 172.21.15.29:0/298271530 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac007320 con 0x7faac4150d40 2024-01-24T15:32:24.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faaba7fc700 1 -- 172.21.15.29:0/298271530 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaac007480 con 0x7faac4150d40 2024-01-24T15:32:24.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac8b07700 1 -- 172.21.15.29:0/298271530 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac40bf200 con 0x7faac4150d40 2024-01-24T15:32:24.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faaba7fc700 1 -- 172.21.15.29:0/298271530 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac0059b0 con 0x7faac4150d40 2024-01-24T15:32:24.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.598+0000 7faac8b07700 1 -- 172.21.15.29:0/298271530 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac40bf7c0 con 0x7faac4150d40 2024-01-24T15:32:24.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.599+0000 7faac8b07700 1 -- 172.21.15.29:0/298271530 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac4064860 con 0x7faac4150d40 2024-01-24T15:32:24.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.600+0000 7faaba7fc700 1 -- 172.21.15.29:0/298271530 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7faaac013410 con 0x7faac4150d40 2024-01-24T15:32:24.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.600+0000 7faaba7fc700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7faaa405f240 0x7faaa4061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.600+0000 7faaba7fc700 1 -- 172.21.15.29:0/298271530 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7faaac07a850 con 0x7faac4150d40 2024-01-24T15:32:24.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.600+0000 7faac259c700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7faaa405f240 0x7faaa4061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.601+0000 7faac259c700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7faaa405f240 0x7faaa4061700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7faab400a010 tx=0x7faab4016000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:24.608 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.604+0000 7faaba7fc700 1 -- 172.21.15.29:0/298271530 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7faaac04b560 con 0x7faac4150d40 2024-01-24T15:32:24.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.787+0000 7faac8b07700 1 -- 172.21.15.29:0/298271530 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faac40475e0 con 0x7faaa405f240 2024-01-24T15:32:24.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.798+0000 7faaba7fc700 1 -- 172.21.15.29:0/298271530 <== mgr.9352 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361599 (secure 0 0 0) 0x7faac40475e0 con 0x7faaa405f240 2024-01-24T15:32:24.805 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.802+0000 7faac8b07700 1 -- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7faaa405f240 msgr2=0x7faaa4061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.802+0000 7faac8b07700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7faaa405f240 0x7faaa4061700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7faab400a010 tx=0x7faab4016000).stop 2024-01-24T15:32:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.802+0000 7faac8b07700 1 -- 172.21.15.29:0/298271530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac4150d40 msgr2=0x7faac4166980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.802+0000 7faac8b07700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac4150d40 0x7faac4166980 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 rx=0x7faaac00c810 tx=0x7faaac001960).stop 2024-01-24T15:32:24.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.802+0000 7faac8b07700 1 -- 172.21.15.29:0/298271530 shutdown_connections 2024-01-24T15:32:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.802+0000 7faac8b07700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac4155040 0x7faac413e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.802+0000 7faac8b07700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7faaa405f240 0x7faaa4061700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.802+0000 7faac8b07700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faac4065ec0 0x7faac40beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.802+0000 7faac8b07700 1 --2- 172.21.15.29:0/298271530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faac4150d40 0x7faac4166980 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.802+0000 7faac8b07700 1 -- 172.21.15.29:0/298271530 >> 172.21.15.29:0/298271530 conn(0x7faac40c8580 msgr2=0x7faac40bd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.802+0000 7faac8b07700 1 -- 172.21.15.29:0/298271530 shutdown_connections 2024-01-24T15:32:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.802+0000 7faac8b07700 1 -- 172.21.15.29:0/298271530 wait complete. 2024-01-24T15:32:24.809 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:32:24.817 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T15:32:23.186668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2473100,"kb_used_data":22604,"kb_used_omap":0,"kb_used_meta":2450432,"kb_avail":752501620,"statfs":{"total":773094113280,"available":770561658880,"internally_reserved":0,"allocated":23146496,"data_stored":13420991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2509242368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28801,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-81,"num_write_kb":-63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.014086"},"pg_stats":[{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721584+0000","last_change":"2024-01-24T15:32:20.716703+0000","last_active":"2024-01-24T15:32:20.721584+0000","last_peered":"2024-01-24T15:32:20.721584+0000","last_clean":"2024-01-24T15:32:20.721584+0000","last_became_active":"2024-01-24T15:32:20.716088+0000","last_became_peered":"2024-01-24T15:32:20.716088+0000","last_unstale":"2024-01-24T15:32:20.721584+0000","last_undegraded":"2024-01-24T15:32:20.721584+0000","last_fullsized":"2024-01-24T15:32:20.721584+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3f","version":"230'3","reported_seq":13,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717200+0000","last_change":"2024-01-24T15:32:20.282862+0000","last_active":"2024-01-24T15:32:20.717200+0000","last_peered":"2024-01-24T15:32:20.717200+0000","last_clean":"2024-01-24T15:32:20.717200+0000","last_became_active":"2024-01-24T15:32:20.282210+0000","last_became_peered":"2024-01-24T15:32:20.282210+0000","last_unstale":"2024-01-24T15:32:20.717200+0000","last_undegraded":"2024-01-24T15:32:20.717200+0000","last_fullsized":"2024-01-24T15:32:20.717200+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.718080+0000","last_change":"2024-01-24T15:32:20.715833+0000","last_active":"2024-01-24T15:32:20.718080+0000","last_peered":"2024-01-24T15:32:20.718080+0000","last_clean":"2024-01-24T15:32:20.718080+0000","last_became_active":"2024-01-24T15:32:20.714863+0000","last_became_peered":"2024-01-24T15:32:20.714863+0000","last_unstale":"2024-01-24T15:32:20.718080+0000","last_undegraded":"2024-01-24T15:32:20.718080+0000","last_fullsized":"2024-01-24T15:32:20.718080+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703777+0000","last_change":"2024-01-24T15:32:20.278260+0000","last_active":"2024-01-24T15:32:20.703777+0000","last_peered":"2024-01-24T15:32:20.703777+0000","last_clean":"2024-01-24T15:32:20.703777+0000","last_became_active":"2024-01-24T15:32:20.277911+0000","last_became_peered":"2024-01-24T15:32:20.277911+0000","last_unstale":"2024-01-24T15:32:20.703777+0000","last_undegraded":"2024-01-24T15:32:20.703777+0000","last_fullsized":"2024-01-24T15:32:20.703777+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.713435+0000","last_change":"2024-01-24T15:32:20.706244+0000","last_active":"2024-01-24T15:32:20.713435+0000","last_peered":"2024-01-24T15:32:20.713435+0000","last_clean":"2024-01-24T15:32:20.713435+0000","last_became_active":"2024-01-24T15:32:20.705940+0000","last_became_peered":"2024-01-24T15:32:20.705940+0000","last_unstale":"2024-01-24T15:32:20.713435+0000","last_undegraded":"2024-01-24T15:32:20.713435+0000","last_fullsized":"2024-01-24T15:32:20.713435+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.694717+0000","last_change":"2024-01-24T15:32:20.275633+0000","last_active":"2024-01-24T15:32:20.694717+0000","last_peered":"2024-01-24T15:32:20.694717+0000","last_clean":"2024-01-24T15:32:20.694717+0000","last_became_active":"2024-01-24T15:32:20.275324+0000","last_became_peered":"2024-01-24T15:32:20.275324+0000","last_unstale":"2024-01-24T15:32:20.694717+0000","last_undegraded":"2024-01-24T15:32:20.694717+0000","last_fullsized":"2024-01-24T15:32:20.694717+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.706306+0000","last_change":"2024-01-24T15:32:20.702709+0000","last_active":"2024-01-24T15:32:20.706306+0000","last_peered":"2024-01-24T15:32:20.706306+0000","last_clean":"2024-01-24T15:32:20.706306+0000","last_became_active":"2024-01-24T15:32:20.702362+0000","last_became_peered":"2024-01-24T15:32:20.702362+0000","last_unstale":"2024-01-24T15:32:20.706306+0000","last_undegraded":"2024-01-24T15:32:20.706306+0000","last_fullsized":"2024-01-24T15:32:20.706306+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3c","version":"230'2","reported_seq":12,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.715604+0000","last_change":"2024-01-24T15:32:20.286812+0000","last_active":"2024-01-24T15:32:20.715604+0000","last_peered":"2024-01-24T15:32:20.715604+0000","last_clean":"2024-01-24T15:32:20.715604+0000","last_became_active":"2024-01-24T15:32:20.286524+0000","last_became_peered":"2024-01-24T15:32:20.286524+0000","last_unstale":"2024-01-24T15:32:20.715604+0000","last_undegraded":"2024-01-24T15:32:20.715604+0000","last_fullsized":"2024-01-24T15:32:20.715604+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.720561+0000","last_change":"2024-01-24T15:32:20.719477+0000","last_active":"2024-01-24T15:32:20.720561+0000","last_peered":"2024-01-24T15:32:20.720561+0000","last_clean":"2024-01-24T15:32:20.720561+0000","last_became_active":"2024-01-24T15:32:20.718955+0000","last_became_peered":"2024-01-24T15:32:20.718955+0000","last_unstale":"2024-01-24T15:32:20.720561+0000","last_undegraded":"2024-01-24T15:32:20.720561+0000","last_fullsized":"2024-01-24T15:32:20.720561+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3b","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.709068+0000","last_change":"2024-01-24T15:32:20.277736+0000","last_active":"2024-01-24T15:32:20.709068+0000","last_peered":"2024-01-24T15:32:20.709068+0000","last_clean":"2024-01-24T15:32:20.709068+0000","last_became_active":"2024-01-24T15:32:20.277388+0000","last_became_peered":"2024-01-24T15:32:20.277388+0000","last_unstale":"2024-01-24T15:32:20.709068+0000","last_undegraded":"2024-01-24T15:32:20.709068+0000","last_fullsized":"2024-01-24T15:32:20.709068+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.719971+0000","last_change":"2024-01-24T15:32:20.716481+0000","last_active":"2024-01-24T15:32:20.719971+0000","last_peered":"2024-01-24T15:32:20.719971+0000","last_clean":"2024-01-24T15:32:20.719971+0000","last_became_active":"2024-01-24T15:32:20.716151+0000","last_became_peered":"2024-01-24T15:32:20.716151+0000","last_unstale":"2024-01-24T15:32:20.719971+0000","last_undegraded":"2024-01-24T15:32:20.719971+0000","last_fullsized":"2024-01-24T15:32:20.719971+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3a","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711035+0000","last_change":"2024-01-24T15:32:20.281678+0000","last_active":"2024-01-24T15:32:20.711035+0000","last_peered":"2024-01-24T15:32:20.711035+0000","last_clean":"2024-01-24T15:32:20.711035+0000","last_became_active":"2024-01-24T15:32:20.281365+0000","last_became_peered":"2024-01-24T15:32:20.281365+0000","last_unstale":"2024-01-24T15:32:20.711035+0000","last_undegraded":"2024-01-24T15:32:20.711035+0000","last_fullsized":"2024-01-24T15:32:20.711035+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.716944+0000","last_change":"2024-01-24T15:32:20.709907+0000","last_active":"2024-01-24T15:32:20.716944+0000","last_peered":"2024-01-24T15:32:20.716944+0000","last_clean":"2024-01-24T15:32:20.716944+0000","last_became_active":"2024-01-24T15:32:20.709598+0000","last_became_peered":"2024-01-24T15:32:20.709598+0000","last_unstale":"2024-01-24T15:32:20.716944+0000","last_undegraded":"2024-01-24T15:32:20.716944+0000","last_fullsized":"2024-01-24T15:32:20.716944+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689070+0000","last_change":"2024-01-24T15:32:20.277935+0000","last_active":"2024-01-24T15:32:20.689070+0000","last_peered":"2024-01-24T15:32:20.689070+0000","last_clean":"2024-01-24T15:32:20.689070+0000","last_became_active":"2024-01-24T15:32:20.277643+0000","last_became_peered":"2024-01-24T15:32:20.277643+0000","last_unstale":"2024-01-24T15:32:20.689070+0000","last_undegraded":"2024-01-24T15:32:20.689070+0000","last_fullsized":"2024-01-24T15:32:20.689070+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717085+0000","last_change":"2024-01-24T15:32:20.714495+0000","last_active":"2024-01-24T15:32:20.717085+0000","last_peered":"2024-01-24T15:32:20.717085+0000","last_clean":"2024-01-24T15:32:20.717085+0000","last_became_active":"2024-01-24T15:32:20.714180+0000","last_became_peered":"2024-01-24T15:32:20.714180+0000","last_unstale":"2024-01-24T15:32:20.717085+0000","last_undegraded":"2024-01-24T15:32:20.717085+0000","last_fullsized":"2024-01-24T15:32:20.717085+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699747+0000","last_change":"2024-01-24T15:32:20.277287+0000","last_active":"2024-01-24T15:32:20.699747+0000","last_peered":"2024-01-24T15:32:20.699747+0000","last_clean":"2024-01-24T15:32:20.699747+0000","last_became_active":"2024-01-24T15:32:20.277027+0000","last_became_peered":"2024-01-24T15:32:20.277027+0000","last_unstale":"2024-01-24T15:32:20.699747+0000","last_undegraded":"2024-01-24T15:32:20.699747+0000","last_fullsized":"2024-01-24T15:32:20.699747+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.710984+0000","last_change":"2024-01-24T15:32:20.704751+0000","last_active":"2024-01-24T15:32:20.710984+0000","last_peered":"2024-01-24T15:32:20.710984+0000","last_clean":"2024-01-24T15:32:20.710984+0000","last_became_active":"2024-01-24T15:32:20.704441+0000","last_became_peered":"2024-01-24T15:32:20.704441+0000","last_unstale":"2024-01-24T15:32:20.710984+0000","last_undegraded":"2024-01-24T15:32:20.710984+0000","last_fullsized":"2024-01-24T15:32:20.710984+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.705165+0000","last_change":"2024-01-24T15:32:20.278896+0000","last_active":"2024-01-24T15:32:20.705165+0000","last_peered":"2024-01-24T15:32:20.705165+0000","last_clean":"2024-01-24T15:32:20.705165+0000","last_became_active":"2024-01-24T15:32:20.278591+0000","last_became_peered":"2024-01-24T15:32:20.278591+0000","last_unstale":"2024-01-24T15:32:20.705165+0000","last_undegraded":"2024-01-24T15:32:20.705165+0000","last_fullsized":"2024-01-24T15:32:20.705165+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721434+0000","last_change":"2024-01-24T15:32:20.716038+0000","last_active":"2024-01-24T15:32:20.721434+0000","last_peered":"2024-01-24T15:32:20.721434+0000","last_clean":"2024-01-24T15:32:20.721434+0000","last_became_active":"2024-01-24T15:32:20.715639+0000","last_became_peered":"2024-01-24T15:32:20.715639+0000","last_unstale":"2024-01-24T15:32:20.721434+0000","last_undegraded":"2024-01-24T15:32:20.721434+0000","last_fullsized":"2024-01-24T15:32:20.721434+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.693184+0000","last_change":"2024-01-24T15:32:20.280691+0000","last_active":"2024-01-24T15:32:20.693184+0000","last_peered":"2024-01-24T15:32:20.693184+0000","last_clean":"2024-01-24T15:32:20.693184+0000","last_became_active":"2024-01-24T15:32:20.280347+0000","last_became_peered":"2024-01-24T15:32:20.280347+0000","last_unstale":"2024-01-24T15:32:20.693184+0000","last_undegraded":"2024-01-24T15:32:20.693184+0000","last_fullsized":"2024-01-24T15:32:20.693184+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717801+0000","last_change":"2024-01-24T15:32:20.714425+0000","last_active":"2024-01-24T15:32:20.717801+0000","last_peered":"2024-01-24T15:32:20.717801+0000","last_clean":"2024-01-24T15:32:20.717801+0000","last_became_active":"2024-01-24T15:32:20.714159+0000","last_became_peered":"2024-01-24T15:32:20.714159+0000","last_unstale":"2024-01-24T15:32:20.717801+0000","last_undegraded":"2024-01-24T15:32:20.717801+0000","last_fullsized":"2024-01-24T15:32:20.717801+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.707140+0000","last_change":"2024-01-24T15:32:20.276025+0000","last_active":"2024-01-24T15:32:20.707140+0000","last_peered":"2024-01-24T15:32:20.707140+0000","last_clean":"2024-01-24T15:32:20.707140+0000","last_became_active":"2024-01-24T15:32:20.275736+0000","last_became_peered":"2024-01-24T15:32:20.275736+0000","last_unstale":"2024-01-24T15:32:20.707140+0000","last_undegraded":"2024-01-24T15:32:20.707140+0000","last_fullsized":"2024-01-24T15:32:20.707140+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.705134+0000","last_change":"2024-01-24T15:32:20.699568+0000","last_active":"2024-01-24T15:32:20.705134+0000","last_peered":"2024-01-24T15:32:20.705134+0000","last_clean":"2024-01-24T15:32:20.705134+0000","last_became_active":"2024-01-24T15:32:20.698341+0000","last_became_peered":"2024-01-24T15:32:20.698341+0000","last_unstale":"2024-01-24T15:32:20.705134+0000","last_undegraded":"2024-01-24T15:32:20.705134+0000","last_fullsized":"2024-01-24T15:32:20.705134+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.34","version":"230'4","reported_seq":14,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.723557+0000","last_change":"2024-01-24T15:32:20.284601+0000","last_active":"2024-01-24T15:32:20.723557+0000","last_peered":"2024-01-24T15:32:20.723557+0000","last_clean":"2024-01-24T15:32:20.723557+0000","last_became_active":"2024-01-24T15:32:20.284292+0000","last_became_peered":"2024-01-24T15:32:20.284292+0000","last_unstale":"2024-01-24T15:32:20.723557+0000","last_undegraded":"2024-01-24T15:32:20.723557+0000","last_fullsized":"2024-01-24T15:32:20.723557+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.719688+0000","last_change":"2024-01-24T15:32:20.716461+0000","last_active":"2024-01-24T15:32:20.719688+0000","last_peered":"2024-01-24T15:32:20.719688+0000","last_clean":"2024-01-24T15:32:20.719688+0000","last_became_active":"2024-01-24T15:32:20.714787+0000","last_became_peered":"2024-01-24T15:32:20.714787+0000","last_unstale":"2024-01-24T15:32:20.719688+0000","last_undegraded":"2024-01-24T15:32:20.719688+0000","last_fullsized":"2024-01-24T15:32:20.719688+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.709661+0000","last_change":"2024-01-24T15:32:20.278404+0000","last_active":"2024-01-24T15:32:20.709661+0000","last_peered":"2024-01-24T15:32:20.709661+0000","last_clean":"2024-01-24T15:32:20.709661+0000","last_became_active":"2024-01-24T15:32:20.277732+0000","last_became_peered":"2024-01-24T15:32:20.277732+0000","last_unstale":"2024-01-24T15:32:20.709661+0000","last_undegraded":"2024-01-24T15:32:20.709661+0000","last_fullsized":"2024-01-24T15:32:20.709661+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711512+0000","last_change":"2024-01-24T15:32:20.706675+0000","last_active":"2024-01-24T15:32:20.711512+0000","last_peered":"2024-01-24T15:32:20.711512+0000","last_clean":"2024-01-24T15:32:20.711512+0000","last_became_active":"2024-01-24T15:32:20.706349+0000","last_became_peered":"2024-01-24T15:32:20.706349+0000","last_unstale":"2024-01-24T15:32:20.711512+0000","last_undegraded":"2024-01-24T15:32:20.711512+0000","last_fullsized":"2024-01-24T15:32:20.711512+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689155+0000","last_change":"2024-01-24T15:32:20.283554+0000","last_active":"2024-01-24T15:32:20.689155+0000","last_peered":"2024-01-24T15:32:20.689155+0000","last_clean":"2024-01-24T15:32:20.689155+0000","last_became_active":"2024-01-24T15:32:20.283234+0000","last_became_peered":"2024-01-24T15:32:20.283234+0000","last_unstale":"2024-01-24T15:32:20.689155+0000","last_undegraded":"2024-01-24T15:32:20.689155+0000","last_fullsized":"2024-01-24T15:32:20.689155+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711895+0000","last_change":"2024-01-24T15:32:20.706837+0000","last_active":"2024-01-24T15:32:20.711895+0000","last_peered":"2024-01-24T15:32:20.711895+0000","last_clean":"2024-01-24T15:32:20.711895+0000","last_became_active":"2024-01-24T15:32:20.706517+0000","last_became_peered":"2024-01-24T15:32:20.706517+0000","last_unstale":"2024-01-24T15:32:20.711895+0000","last_undegraded":"2024-01-24T15:32:20.711895+0000","last_fullsized":"2024-01-24T15:32:20.711895+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703639+0000","last_change":"2024-01-24T15:32:20.282841+0000","last_active":"2024-01-24T15:32:20.703639+0000","last_peered":"2024-01-24T15:32:20.703639+0000","last_clean":"2024-01-24T15:32:20.703639+0000","last_became_active":"2024-01-24T15:32:20.282029+0000","last_became_peered":"2024-01-24T15:32:20.282029+0000","last_unstale":"2024-01-24T15:32:20.703639+0000","last_undegraded":"2024-01-24T15:32:20.703639+0000","last_fullsized":"2024-01-24T15:32:20.703639+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.715887+0000","last_change":"2024-01-24T15:32:20.705283+0000","last_active":"2024-01-24T15:32:20.715887+0000","last_peered":"2024-01-24T15:32:20.715887+0000","last_clean":"2024-01-24T15:32:20.715887+0000","last_became_active":"2024-01-24T15:32:20.704976+0000","last_became_peered":"2024-01-24T15:32:20.704976+0000","last_unstale":"2024-01-24T15:32:20.715887+0000","last_undegraded":"2024-01-24T15:32:20.715887+0000","last_fullsized":"2024-01-24T15:32:20.715887+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.30","version":"230'4","reported_seq":14,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.707456+0000","last_change":"2024-01-24T15:32:20.278699+0000","last_active":"2024-01-24T15:32:20.707456+0000","last_peered":"2024-01-24T15:32:20.707456+0000","last_clean":"2024-01-24T15:32:20.707456+0000","last_became_active":"2024-01-24T15:32:20.278196+0000","last_became_peered":"2024-01-24T15:32:20.278196+0000","last_unstale":"2024-01-24T15:32:20.707456+0000","last_undegraded":"2024-01-24T15:32:20.707456+0000","last_fullsized":"2024-01-24T15:32:20.707456+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711999+0000","last_change":"2024-01-24T15:32:20.707454+0000","last_active":"2024-01-24T15:32:20.711999+0000","last_peered":"2024-01-24T15:32:20.711999+0000","last_clean":"2024-01-24T15:32:20.711999+0000","last_became_active":"2024-01-24T15:32:20.707186+0000","last_became_peered":"2024-01-24T15:32:20.707186+0000","last_unstale":"2024-01-24T15:32:20.711999+0000","last_undegraded":"2024-01-24T15:32:20.711999+0000","last_fullsized":"2024-01-24T15:32:20.711999+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.692664+0000","last_change":"2024-01-24T15:32:20.271993+0000","last_active":"2024-01-24T15:32:20.692664+0000","last_peered":"2024-01-24T15:32:20.692664+0000","last_clean":"2024-01-24T15:32:20.692664+0000","last_became_active":"2024-01-24T15:32:20.271685+0000","last_became_peered":"2024-01-24T15:32:20.271685+0000","last_unstale":"2024-01-24T15:32:20.692664+0000","last_undegraded":"2024-01-24T15:32:20.692664+0000","last_fullsized":"2024-01-24T15:32:20.692664+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.718055+0000","last_change":"2024-01-24T15:32:20.715615+0000","last_active":"2024-01-24T15:32:20.718055+0000","last_peered":"2024-01-24T15:32:20.718055+0000","last_clean":"2024-01-24T15:32:20.718055+0000","last_became_active":"2024-01-24T15:32:20.714761+0000","last_became_peered":"2024-01-24T15:32:20.714761+0000","last_unstale":"2024-01-24T15:32:20.718055+0000","last_undegraded":"2024-01-24T15:32:20.718055+0000","last_fullsized":"2024-01-24T15:32:20.718055+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.698794+0000","last_change":"2024-01-24T15:32:20.279177+0000","last_active":"2024-01-24T15:32:20.698794+0000","last_peered":"2024-01-24T15:32:20.698794+0000","last_clean":"2024-01-24T15:32:20.698794+0000","last_became_active":"2024-01-24T15:32:20.278881+0000","last_became_peered":"2024-01-24T15:32:20.278881+0000","last_unstale":"2024-01-24T15:32:20.698794+0000","last_undegraded":"2024-01-24T15:32:20.698794+0000","last_fullsized":"2024-01-24T15:32:20.698794+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.719883+0000","last_change":"2024-01-24T15:32:20.716486+0000","last_active":"2024-01-24T15:32:20.719883+0000","last_peered":"2024-01-24T15:32:20.719883+0000","last_clean":"2024-01-24T15:32:20.719883+0000","last_became_active":"2024-01-24T15:32:20.716183+0000","last_became_peered":"2024-01-24T15:32:20.716183+0000","last_unstale":"2024-01-24T15:32:20.719883+0000","last_undegraded":"2024-01-24T15:32:20.719883+0000","last_fullsized":"2024-01-24T15:32:20.719883+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2d","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.708205+0000","last_change":"2024-01-24T15:32:20.283867+0000","last_active":"2024-01-24T15:32:20.708205+0000","last_peered":"2024-01-24T15:32:20.708205+0000","last_clean":"2024-01-24T15:32:20.708205+0000","last_became_active":"2024-01-24T15:32:20.283405+0000","last_became_peered":"2024-01-24T15:32:20.283405+0000","last_unstale":"2024-01-24T15:32:20.708205+0000","last_undegraded":"2024-01-24T15:32:20.708205+0000","last_fullsized":"2024-01-24T15:32:20.708205+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.710832+0000","last_change":"2024-01-24T15:32:20.704821+0000","last_active":"2024-01-24T15:32:20.710832+0000","last_peered":"2024-01-24T15:32:20.710832+0000","last_clean":"2024-01-24T15:32:20.710832+0000","last_became_active":"2024-01-24T15:32:20.704520+0000","last_became_peered":"2024-01-24T15:32:20.704520+0000","last_unstale":"2024-01-24T15:32:20.710832+0000","last_undegraded":"2024-01-24T15:32:20.710832+0000","last_fullsized":"2024-01-24T15:32:20.710832+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.692306+0000","last_change":"2024-01-24T15:32:20.272305+0000","last_active":"2024-01-24T15:32:20.692306+0000","last_peered":"2024-01-24T15:32:20.692306+0000","last_clean":"2024-01-24T15:32:20.692306+0000","last_became_active":"2024-01-24T15:32:20.271955+0000","last_became_peered":"2024-01-24T15:32:20.271955+0000","last_unstale":"2024-01-24T15:32:20.692306+0000","last_undegraded":"2024-01-24T15:32:20.692306+0000","last_fullsized":"2024-01-24T15:32:20.692306+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.713224+0000","last_change":"2024-01-24T15:32:20.705360+0000","last_active":"2024-01-24T15:32:20.713224+0000","last_peered":"2024-01-24T15:32:20.713224+0000","last_clean":"2024-01-24T15:32:20.713224+0000","last_became_active":"2024-01-24T15:32:20.704394+0000","last_became_peered":"2024-01-24T15:32:20.704394+0000","last_unstale":"2024-01-24T15:32:20.713224+0000","last_undegraded":"2024-01-24T15:32:20.713224+0000","last_fullsized":"2024-01-24T15:32:20.713224+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2b","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714478+0000","last_change":"2024-01-24T15:32:20.279023+0000","last_active":"2024-01-24T15:32:20.714478+0000","last_peered":"2024-01-24T15:32:20.714478+0000","last_clean":"2024-01-24T15:32:20.714478+0000","last_became_active":"2024-01-24T15:32:20.278611+0000","last_became_peered":"2024-01-24T15:32:20.278611+0000","last_unstale":"2024-01-24T15:32:20.714478+0000","last_undegraded":"2024-01-24T15:32:20.714478+0000","last_fullsized":"2024-01-24T15:32:20.714478+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714381+0000","last_change":"2024-01-24T15:32:20.712354+0000","last_active":"2024-01-24T15:32:20.714381+0000","last_peered":"2024-01-24T15:32:20.714381+0000","last_clean":"2024-01-24T15:32:20.714381+0000","last_became_active":"2024-01-24T15:32:20.712116+0000","last_became_peered":"2024-01-24T15:32:20.712116+0000","last_unstale":"2024-01-24T15:32:20.714381+0000","last_undegraded":"2024-01-24T15:32:20.714381+0000","last_fullsized":"2024-01-24T15:32:20.714381+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.704665+0000","last_change":"2024-01-24T15:32:20.281147+0000","last_active":"2024-01-24T15:32:20.704665+0000","last_peered":"2024-01-24T15:32:20.704665+0000","last_clean":"2024-01-24T15:32:20.704665+0000","last_became_active":"2024-01-24T15:32:20.280772+0000","last_became_peered":"2024-01-24T15:32:20.280772+0000","last_unstale":"2024-01-24T15:32:20.704665+0000","last_undegraded":"2024-01-24T15:32:20.704665+0000","last_fullsized":"2024-01-24T15:32:20.704665+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714804+0000","last_change":"2024-01-24T15:32:20.711895+0000","last_active":"2024-01-24T15:32:20.714804+0000","last_peered":"2024-01-24T15:32:20.714804+0000","last_clean":"2024-01-24T15:32:20.714804+0000","last_became_active":"2024-01-24T15:32:20.711612+0000","last_became_peered":"2024-01-24T15:32:20.711612+0000","last_unstale":"2024-01-24T15:32:20.714804+0000","last_undegraded":"2024-01-24T15:32:20.714804+0000","last_fullsized":"2024-01-24T15:32:20.714804+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.698649+0000","last_change":"2024-01-24T15:32:20.284966+0000","last_active":"2024-01-24T15:32:20.698649+0000","last_peered":"2024-01-24T15:32:20.698649+0000","last_clean":"2024-01-24T15:32:20.698649+0000","last_became_active":"2024-01-24T15:32:20.284662+0000","last_became_peered":"2024-01-24T15:32:20.284662+0000","last_unstale":"2024-01-24T15:32:20.698649+0000","last_undegraded":"2024-01-24T15:32:20.698649+0000","last_fullsized":"2024-01-24T15:32:20.698649+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721848+0000","last_change":"2024-01-24T15:32:20.717940+0000","last_active":"2024-01-24T15:32:20.721848+0000","last_peered":"2024-01-24T15:32:20.721848+0000","last_clean":"2024-01-24T15:32:20.721848+0000","last_became_active":"2024-01-24T15:32:20.717078+0000","last_became_peered":"2024-01-24T15:32:20.717078+0000","last_unstale":"2024-01-24T15:32:20.721848+0000","last_undegraded":"2024-01-24T15:32:20.721848+0000","last_fullsized":"2024-01-24T15:32:20.721848+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703319+0000","last_change":"2024-01-24T15:32:20.276295+0000","last_active":"2024-01-24T15:32:20.703319+0000","last_peered":"2024-01-24T15:32:20.703319+0000","last_clean":"2024-01-24T15:32:20.703319+0000","last_became_active":"2024-01-24T15:32:20.275936+0000","last_became_peered":"2024-01-24T15:32:20.275936+0000","last_unstale":"2024-01-24T15:32:20.703319+0000","last_undegraded":"2024-01-24T15:32:20.703319+0000","last_fullsized":"2024-01-24T15:32:20.703319+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.718102+0000","last_change":"2024-01-24T15:32:20.714991+0000","last_active":"2024-01-24T15:32:20.718102+0000","last_peered":"2024-01-24T15:32:20.718102+0000","last_clean":"2024-01-24T15:32:20.718102+0000","last_became_active":"2024-01-24T15:32:20.714659+0000","last_became_peered":"2024-01-24T15:32:20.714659+0000","last_unstale":"2024-01-24T15:32:20.718102+0000","last_undegraded":"2024-01-24T15:32:20.718102+0000","last_fullsized":"2024-01-24T15:32:20.718102+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689450+0000","last_change":"2024-01-24T15:32:20.283489+0000","last_active":"2024-01-24T15:32:20.689450+0000","last_peered":"2024-01-24T15:32:20.689450+0000","last_clean":"2024-01-24T15:32:20.689450+0000","last_became_active":"2024-01-24T15:32:20.283183+0000","last_became_peered":"2024-01-24T15:32:20.283183+0000","last_unstale":"2024-01-24T15:32:20.689450+0000","last_undegraded":"2024-01-24T15:32:20.689450+0000","last_fullsized":"2024-01-24T15:32:20.689450+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.720219+0000","last_change":"2024-01-24T15:32:20.713009+0000","last_active":"2024-01-24T15:32:20.720219+0000","last_peered":"2024-01-24T15:32:20.720219+0000","last_clean":"2024-01-24T15:32:20.720219+0000","last_became_active":"2024-01-24T15:32:20.712705+0000","last_became_peered":"2024-01-24T15:32:20.712705+0000","last_unstale":"2024-01-24T15:32:20.720219+0000","last_undegraded":"2024-01-24T15:32:20.720219+0000","last_fullsized":"2024-01-24T15:32:20.720219+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.709683+0000","last_change":"2024-01-24T15:32:20.282888+0000","last_active":"2024-01-24T15:32:20.709683+0000","last_peered":"2024-01-24T15:32:20.709683+0000","last_clean":"2024-01-24T15:32:20.709683+0000","last_became_active":"2024-01-24T15:32:20.282421+0000","last_became_peered":"2024-01-24T15:32:20.282421+0000","last_unstale":"2024-01-24T15:32:20.709683+0000","last_undegraded":"2024-01-24T15:32:20.709683+0000","last_fullsized":"2024-01-24T15:32:20.709683+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721250+0000","last_change":"2024-01-24T15:32:20.712004+0000","last_active":"2024-01-24T15:32:20.721250+0000","last_peered":"2024-01-24T15:32:20.721250+0000","last_clean":"2024-01-24T15:32:20.721250+0000","last_became_active":"2024-01-24T15:32:20.711681+0000","last_became_peered":"2024-01-24T15:32:20.711681+0000","last_unstale":"2024-01-24T15:32:20.721250+0000","last_undegraded":"2024-01-24T15:32:20.721250+0000","last_fullsized":"2024-01-24T15:32:20.721250+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699109+0000","last_change":"2024-01-24T15:32:20.275343+0000","last_active":"2024-01-24T15:32:20.699109+0000","last_peered":"2024-01-24T15:32:20.699109+0000","last_clean":"2024-01-24T15:32:20.699109+0000","last_became_active":"2024-01-24T15:32:20.275046+0000","last_became_peered":"2024-01-24T15:32:20.275046+0000","last_unstale":"2024-01-24T15:32:20.699109+0000","last_undegraded":"2024-01-24T15:32:20.699109+0000","last_fullsized":"2024-01-24T15:32:20.699109+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.715026+0000","last_change":"2024-01-24T15:32:20.711194+0000","last_active":"2024-01-24T15:32:20.715026+0000","last_peered":"2024-01-24T15:32:20.715026+0000","last_clean":"2024-01-24T15:32:20.715026+0000","last_became_active":"2024-01-24T15:32:20.710933+0000","last_became_peered":"2024-01-24T15:32:20.710933+0000","last_unstale":"2024-01-24T15:32:20.715026+0000","last_undegraded":"2024-01-24T15:32:20.715026+0000","last_fullsized":"2024-01-24T15:32:20.715026+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.692466+0000","last_change":"2024-01-24T15:32:20.276754+0000","last_active":"2024-01-24T15:32:20.692466+0000","last_peered":"2024-01-24T15:32:20.692466+0000","last_clean":"2024-01-24T15:32:20.692466+0000","last_became_active":"2024-01-24T15:32:20.276459+0000","last_became_peered":"2024-01-24T15:32:20.276459+0000","last_unstale":"2024-01-24T15:32:20.692466+0000","last_undegraded":"2024-01-24T15:32:20.692466+0000","last_fullsized":"2024-01-24T15:32:20.692466+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721747+0000","last_change":"2024-01-24T15:32:20.720881+0000","last_active":"2024-01-24T15:32:20.721747+0000","last_peered":"2024-01-24T15:32:20.721747+0000","last_clean":"2024-01-24T15:32:20.721747+0000","last_became_active":"2024-01-24T15:32:20.720537+0000","last_became_peered":"2024-01-24T15:32:20.720537+0000","last_unstale":"2024-01-24T15:32:20.721747+0000","last_undegraded":"2024-01-24T15:32:20.721747+0000","last_fullsized":"2024-01-24T15:32:20.721747+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.705510+0000","last_change":"2024-01-24T15:32:20.280328+0000","last_active":"2024-01-24T15:32:20.705510+0000","last_peered":"2024-01-24T15:32:20.705510+0000","last_clean":"2024-01-24T15:32:20.705510+0000","last_became_active":"2024-01-24T15:32:20.280071+0000","last_became_peered":"2024-01-24T15:32:20.280071+0000","last_unstale":"2024-01-24T15:32:20.705510+0000","last_undegraded":"2024-01-24T15:32:20.705510+0000","last_fullsized":"2024-01-24T15:32:20.705510+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.715071+0000","last_change":"2024-01-24T15:32:20.711414+0000","last_active":"2024-01-24T15:32:20.715071+0000","last_peered":"2024-01-24T15:32:20.715071+0000","last_clean":"2024-01-24T15:32:20.715071+0000","last_became_active":"2024-01-24T15:32:20.710834+0000","last_became_peered":"2024-01-24T15:32:20.710834+0000","last_unstale":"2024-01-24T15:32:20.715071+0000","last_undegraded":"2024-01-24T15:32:20.715071+0000","last_fullsized":"2024-01-24T15:32:20.715071+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.705432+0000","last_change":"2024-01-24T15:32:20.289152+0000","last_active":"2024-01-24T15:32:20.705432+0000","last_peered":"2024-01-24T15:32:20.705432+0000","last_clean":"2024-01-24T15:32:20.705432+0000","last_became_active":"2024-01-24T15:32:20.288964+0000","last_became_peered":"2024-01-24T15:32:20.288964+0000","last_unstale":"2024-01-24T15:32:20.705432+0000","last_undegraded":"2024-01-24T15:32:20.705432+0000","last_fullsized":"2024-01-24T15:32:20.705432+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.716561+0000","last_change":"2024-01-24T15:32:20.712729+0000","last_active":"2024-01-24T15:32:20.716561+0000","last_peered":"2024-01-24T15:32:20.716561+0000","last_clean":"2024-01-24T15:32:20.716561+0000","last_became_active":"2024-01-24T15:32:20.712428+0000","last_became_peered":"2024-01-24T15:32:20.712428+0000","last_unstale":"2024-01-24T15:32:20.716561+0000","last_undegraded":"2024-01-24T15:32:20.716561+0000","last_fullsized":"2024-01-24T15:32:20.716561+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689133+0000","last_change":"2024-01-24T15:32:20.285176+0000","last_active":"2024-01-24T15:32:20.689133+0000","last_peered":"2024-01-24T15:32:20.689133+0000","last_clean":"2024-01-24T15:32:20.689133+0000","last_became_active":"2024-01-24T15:32:20.284865+0000","last_became_peered":"2024-01-24T15:32:20.284865+0000","last_unstale":"2024-01-24T15:32:20.689133+0000","last_undegraded":"2024-01-24T15:32:20.689133+0000","last_fullsized":"2024-01-24T15:32:20.689133+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711957+0000","last_change":"2024-01-24T15:32:20.707547+0000","last_active":"2024-01-24T15:32:20.711957+0000","last_peered":"2024-01-24T15:32:20.711957+0000","last_clean":"2024-01-24T15:32:20.711957+0000","last_became_active":"2024-01-24T15:32:20.707227+0000","last_became_peered":"2024-01-24T15:32:20.707227+0000","last_unstale":"2024-01-24T15:32:20.711957+0000","last_undegraded":"2024-01-24T15:32:20.711957+0000","last_fullsized":"2024-01-24T15:32:20.711957+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.693316+0000","last_change":"2024-01-24T15:32:20.279162+0000","last_active":"2024-01-24T15:32:20.693316+0000","last_peered":"2024-01-24T15:32:20.693316+0000","last_clean":"2024-01-24T15:32:20.693316+0000","last_became_active":"2024-01-24T15:32:20.278862+0000","last_became_peered":"2024-01-24T15:32:20.278862+0000","last_unstale":"2024-01-24T15:32:20.693316+0000","last_undegraded":"2024-01-24T15:32:20.693316+0000","last_fullsized":"2024-01-24T15:32:20.693316+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.720836+0000","last_change":"2024-01-24T15:32:20.714570+0000","last_active":"2024-01-24T15:32:20.720836+0000","last_peered":"2024-01-24T15:32:20.720836+0000","last_clean":"2024-01-24T15:32:20.720836+0000","last_became_active":"2024-01-24T15:32:20.714285+0000","last_became_peered":"2024-01-24T15:32:20.714285+0000","last_unstale":"2024-01-24T15:32:20.720836+0000","last_undegraded":"2024-01-24T15:32:20.720836+0000","last_fullsized":"2024-01-24T15:32:20.720836+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.705266+0000","last_change":"2024-01-24T15:32:20.282984+0000","last_active":"2024-01-24T15:32:20.705266+0000","last_peered":"2024-01-24T15:32:20.705266+0000","last_clean":"2024-01-24T15:32:20.705266+0000","last_became_active":"2024-01-24T15:32:20.282644+0000","last_became_peered":"2024-01-24T15:32:20.282644+0000","last_unstale":"2024-01-24T15:32:20.705266+0000","last_undegraded":"2024-01-24T15:32:20.705266+0000","last_fullsized":"2024-01-24T15:32:20.705266+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717988+0000","last_change":"2024-01-24T15:32:20.708731+0000","last_active":"2024-01-24T15:32:20.717988+0000","last_peered":"2024-01-24T15:32:20.717988+0000","last_clean":"2024-01-24T15:32:20.717988+0000","last_became_active":"2024-01-24T15:32:20.708433+0000","last_became_peered":"2024-01-24T15:32:20.708433+0000","last_unstale":"2024-01-24T15:32:20.717988+0000","last_undegraded":"2024-01-24T15:32:20.717988+0000","last_fullsized":"2024-01-24T15:32:20.717988+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.698898+0000","last_change":"2024-01-24T15:32:20.276650+0000","last_active":"2024-01-24T15:32:20.698898+0000","last_peered":"2024-01-24T15:32:20.698898+0000","last_clean":"2024-01-24T15:32:20.698898+0000","last_became_active":"2024-01-24T15:32:20.276437+0000","last_became_peered":"2024-01-24T15:32:20.276437+0000","last_unstale":"2024-01-24T15:32:20.698898+0000","last_undegraded":"2024-01-24T15:32:20.698898+0000","last_fullsized":"2024-01-24T15:32:20.698898+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714313+0000","last_change":"2024-01-24T15:32:20.712042+0000","last_active":"2024-01-24T15:32:20.714313+0000","last_peered":"2024-01-24T15:32:20.714313+0000","last_clean":"2024-01-24T15:32:20.714313+0000","last_became_active":"2024-01-24T15:32:20.707486+0000","last_became_peered":"2024-01-24T15:32:20.707486+0000","last_unstale":"2024-01-24T15:32:20.714313+0000","last_undegraded":"2024-01-24T15:32:20.714313+0000","last_fullsized":"2024-01-24T15:32:20.714313+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"230'3","reported_seq":13,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.719452+0000","last_change":"2024-01-24T15:32:20.289013+0000","last_active":"2024-01-24T15:32:20.719452+0000","last_peered":"2024-01-24T15:32:20.719452+0000","last_clean":"2024-01-24T15:32:20.719452+0000","last_became_active":"2024-01-24T15:32:20.288764+0000","last_became_peered":"2024-01-24T15:32:20.288764+0000","last_unstale":"2024-01-24T15:32:20.719452+0000","last_undegraded":"2024-01-24T15:32:20.719452+0000","last_fullsized":"2024-01-24T15:32:20.719452+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.716822+0000","last_change":"2024-01-24T15:32:20.705865+0000","last_active":"2024-01-24T15:32:20.716822+0000","last_peered":"2024-01-24T15:32:20.716822+0000","last_clean":"2024-01-24T15:32:20.716822+0000","last_became_active":"2024-01-24T15:32:20.705170+0000","last_became_peered":"2024-01-24T15:32:20.705170+0000","last_unstale":"2024-01-24T15:32:20.716822+0000","last_undegraded":"2024-01-24T15:32:20.716822+0000","last_fullsized":"2024-01-24T15:32:20.716822+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"230'2","reported_seq":12,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.713900+0000","last_change":"2024-01-24T15:32:20.282155+0000","last_active":"2024-01-24T15:32:20.713900+0000","last_peered":"2024-01-24T15:32:20.713900+0000","last_clean":"2024-01-24T15:32:20.713900+0000","last_became_active":"2024-01-24T15:32:20.281877+0000","last_became_peered":"2024-01-24T15:32:20.281877+0000","last_unstale":"2024-01-24T15:32:20.713900+0000","last_undegraded":"2024-01-24T15:32:20.713900+0000","last_fullsized":"2024-01-24T15:32:20.713900+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.720540+0000","last_change":"2024-01-24T15:32:20.719293+0000","last_active":"2024-01-24T15:32:20.720540+0000","last_peered":"2024-01-24T15:32:20.720540+0000","last_clean":"2024-01-24T15:32:20.720540+0000","last_became_active":"2024-01-24T15:32:20.718598+0000","last_became_peered":"2024-01-24T15:32:20.718598+0000","last_unstale":"2024-01-24T15:32:20.720540+0000","last_undegraded":"2024-01-24T15:32:20.720540+0000","last_fullsized":"2024-01-24T15:32:20.720540+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.709707+0000","last_change":"2024-01-24T15:32:20.278608+0000","last_active":"2024-01-24T15:32:20.709707+0000","last_peered":"2024-01-24T15:32:20.709707+0000","last_clean":"2024-01-24T15:32:20.709707+0000","last_became_active":"2024-01-24T15:32:20.278124+0000","last_became_peered":"2024-01-24T15:32:20.278124+0000","last_unstale":"2024-01-24T15:32:20.709707+0000","last_undegraded":"2024-01-24T15:32:20.709707+0000","last_fullsized":"2024-01-24T15:32:20.709707+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717675+0000","last_change":"2024-01-24T15:32:20.713089+0000","last_active":"2024-01-24T15:32:20.717675+0000","last_peered":"2024-01-24T15:32:20.717675+0000","last_clean":"2024-01-24T15:32:20.717675+0000","last_became_active":"2024-01-24T15:32:20.712818+0000","last_became_peered":"2024-01-24T15:32:20.712818+0000","last_unstale":"2024-01-24T15:32:20.717675+0000","last_undegraded":"2024-01-24T15:32:20.717675+0000","last_fullsized":"2024-01-24T15:32:20.717675+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.697089+0000","last_change":"2024-01-24T15:32:20.287919+0000","last_active":"2024-01-24T15:32:20.697089+0000","last_peered":"2024-01-24T15:32:20.697089+0000","last_clean":"2024-01-24T15:32:20.697089+0000","last_became_active":"2024-01-24T15:32:20.287555+0000","last_became_peered":"2024-01-24T15:32:20.287555+0000","last_unstale":"2024-01-24T15:32:20.697089+0000","last_undegraded":"2024-01-24T15:32:20.697089+0000","last_fullsized":"2024-01-24T15:32:20.697089+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.718063+0000","last_change":"2024-01-24T15:32:20.708923+0000","last_active":"2024-01-24T15:32:20.718063+0000","last_peered":"2024-01-24T15:32:20.718063+0000","last_clean":"2024-01-24T15:32:20.718063+0000","last_became_active":"2024-01-24T15:32:20.708439+0000","last_became_peered":"2024-01-24T15:32:20.708439+0000","last_unstale":"2024-01-24T15:32:20.718063+0000","last_undegraded":"2024-01-24T15:32:20.718063+0000","last_fullsized":"2024-01-24T15:32:20.718063+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.692569+0000","last_change":"2024-01-24T15:32:20.276616+0000","last_active":"2024-01-24T15:32:20.692569+0000","last_peered":"2024-01-24T15:32:20.692569+0000","last_clean":"2024-01-24T15:32:20.692569+0000","last_became_active":"2024-01-24T15:32:20.276312+0000","last_became_peered":"2024-01-24T15:32:20.276312+0000","last_unstale":"2024-01-24T15:32:20.692569+0000","last_undegraded":"2024-01-24T15:32:20.692569+0000","last_fullsized":"2024-01-24T15:32:20.692569+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717063+0000","last_change":"2024-01-24T15:32:20.713385+0000","last_active":"2024-01-24T15:32:20.717063+0000","last_peered":"2024-01-24T15:32:20.717063+0000","last_clean":"2024-01-24T15:32:20.717063+0000","last_became_active":"2024-01-24T15:32:20.713106+0000","last_became_peered":"2024-01-24T15:32:20.713106+0000","last_unstale":"2024-01-24T15:32:20.717063+0000","last_undegraded":"2024-01-24T15:32:20.717063+0000","last_fullsized":"2024-01-24T15:32:20.717063+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"230'4","reported_seq":14,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.712431+0000","last_change":"2024-01-24T15:32:20.277127+0000","last_active":"2024-01-24T15:32:20.712431+0000","last_peered":"2024-01-24T15:32:20.712431+0000","last_clean":"2024-01-24T15:32:20.712431+0000","last_became_active":"2024-01-24T15:32:20.275139+0000","last_became_peered":"2024-01-24T15:32:20.275139+0000","last_unstale":"2024-01-24T15:32:20.712431+0000","last_undegraded":"2024-01-24T15:32:20.712431+0000","last_fullsized":"2024-01-24T15:32:20.712431+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721595+0000","last_change":"2024-01-24T15:32:20.718169+0000","last_active":"2024-01-24T15:32:20.721595+0000","last_peered":"2024-01-24T15:32:20.721595+0000","last_clean":"2024-01-24T15:32:20.721595+0000","last_became_active":"2024-01-24T15:32:20.716914+0000","last_became_peered":"2024-01-24T15:32:20.716914+0000","last_unstale":"2024-01-24T15:32:20.721595+0000","last_undegraded":"2024-01-24T15:32:20.721595+0000","last_fullsized":"2024-01-24T15:32:20.721595+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717108+0000","last_change":"2024-01-24T15:32:20.277339+0000","last_active":"2024-01-24T15:32:20.717108+0000","last_peered":"2024-01-24T15:32:20.717108+0000","last_clean":"2024-01-24T15:32:20.717108+0000","last_became_active":"2024-01-24T15:32:20.277018+0000","last_became_peered":"2024-01-24T15:32:20.277018+0000","last_unstale":"2024-01-24T15:32:20.717108+0000","last_undegraded":"2024-01-24T15:32:20.717108+0000","last_fullsized":"2024-01-24T15:32:20.717108+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721582+0000","last_change":"2024-01-24T15:32:20.716601+0000","last_active":"2024-01-24T15:32:20.721582+0000","last_peered":"2024-01-24T15:32:20.721582+0000","last_clean":"2024-01-24T15:32:20.721582+0000","last_became_active":"2024-01-24T15:32:20.716311+0000","last_became_peered":"2024-01-24T15:32:20.716311+0000","last_unstale":"2024-01-24T15:32:20.721582+0000","last_undegraded":"2024-01-24T15:32:20.721582+0000","last_fullsized":"2024-01-24T15:32:20.721582+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"230'2","reported_seq":12,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721533+0000","last_change":"2024-01-24T15:32:20.282723+0000","last_active":"2024-01-24T15:32:20.721533+0000","last_peered":"2024-01-24T15:32:20.721533+0000","last_clean":"2024-01-24T15:32:20.721533+0000","last_became_active":"2024-01-24T15:32:20.282380+0000","last_became_peered":"2024-01-24T15:32:20.282380+0000","last_unstale":"2024-01-24T15:32:20.721533+0000","last_undegraded":"2024-01-24T15:32:20.721533+0000","last_fullsized":"2024-01-24T15:32:20.721533+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721274+0000","last_change":"2024-01-24T15:32:20.712626+0000","last_active":"2024-01-24T15:32:20.721274+0000","last_peered":"2024-01-24T15:32:20.721274+0000","last_clean":"2024-01-24T15:32:20.721274+0000","last_became_active":"2024-01-24T15:32:20.712245+0000","last_became_peered":"2024-01-24T15:32:20.712245+0000","last_unstale":"2024-01-24T15:32:20.721274+0000","last_undegraded":"2024-01-24T15:32:20.721274+0000","last_fullsized":"2024-01-24T15:32:20.721274+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.694279+0000","last_change":"2024-01-24T15:32:20.279875+0000","last_active":"2024-01-24T15:32:20.694279+0000","last_peered":"2024-01-24T15:32:20.694279+0000","last_clean":"2024-01-24T15:32:20.694279+0000","last_became_active":"2024-01-24T15:32:20.279568+0000","last_became_peered":"2024-01-24T15:32:20.279568+0000","last_unstale":"2024-01-24T15:32:20.694279+0000","last_undegraded":"2024-01-24T15:32:20.694279+0000","last_fullsized":"2024-01-24T15:32:20.694279+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711798+0000","last_change":"2024-01-24T15:32:20.707837+0000","last_active":"2024-01-24T15:32:20.711798+0000","last_peered":"2024-01-24T15:32:20.711798+0000","last_clean":"2024-01-24T15:32:20.711798+0000","last_became_active":"2024-01-24T15:32:20.707522+0000","last_became_peered":"2024-01-24T15:32:20.707522+0000","last_unstale":"2024-01-24T15:32:20.711798+0000","last_undegraded":"2024-01-24T15:32:20.711798+0000","last_fullsized":"2024-01-24T15:32:20.711798+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689619+0000","last_change":"2024-01-24T15:32:20.285248+0000","last_active":"2024-01-24T15:32:20.689619+0000","last_peered":"2024-01-24T15:32:20.689619+0000","last_clean":"2024-01-24T15:32:20.689619+0000","last_became_active":"2024-01-24T15:32:20.284886+0000","last_became_peered":"2024-01-24T15:32:20.284886+0000","last_unstale":"2024-01-24T15:32:20.689619+0000","last_undegraded":"2024-01-24T15:32:20.689619+0000","last_fullsized":"2024-01-24T15:32:20.689619+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714275+0000","last_change":"2024-01-24T15:32:20.711880+0000","last_active":"2024-01-24T15:32:20.714275+0000","last_peered":"2024-01-24T15:32:20.714275+0000","last_clean":"2024-01-24T15:32:20.714275+0000","last_became_active":"2024-01-24T15:32:20.711632+0000","last_became_peered":"2024-01-24T15:32:20.711632+0000","last_unstale":"2024-01-24T15:32:20.714275+0000","last_undegraded":"2024-01-24T15:32:20.714275+0000","last_fullsized":"2024-01-24T15:32:20.714275+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.709216+0000","last_change":"2024-01-24T15:32:20.273923+0000","last_active":"2024-01-24T15:32:20.709216+0000","last_peered":"2024-01-24T15:32:20.709216+0000","last_clean":"2024-01-24T15:32:20.709216+0000","last_became_active":"2024-01-24T15:32:20.273725+0000","last_became_peered":"2024-01-24T15:32:20.273725+0000","last_unstale":"2024-01-24T15:32:20.709216+0000","last_undegraded":"2024-01-24T15:32:20.709216+0000","last_fullsized":"2024-01-24T15:32:20.709216+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721234+0000","last_change":"2024-01-24T15:32:20.710548+0000","last_active":"2024-01-24T15:32:20.721234+0000","last_peered":"2024-01-24T15:32:20.721234+0000","last_clean":"2024-01-24T15:32:20.721234+0000","last_became_active":"2024-01-24T15:32:20.710252+0000","last_became_peered":"2024-01-24T15:32:20.710252+0000","last_unstale":"2024-01-24T15:32:20.721234+0000","last_undegraded":"2024-01-24T15:32:20.721234+0000","last_fullsized":"2024-01-24T15:32:20.721234+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.696722+0000","last_change":"2024-01-24T15:32:20.286609+0000","last_active":"2024-01-24T15:32:20.696722+0000","last_peered":"2024-01-24T15:32:20.696722+0000","last_clean":"2024-01-24T15:32:20.696722+0000","last_became_active":"2024-01-24T15:32:20.286254+0000","last_became_peered":"2024-01-24T15:32:20.286254+0000","last_unstale":"2024-01-24T15:32:20.696722+0000","last_undegraded":"2024-01-24T15:32:20.696722+0000","last_fullsized":"2024-01-24T15:32:20.696722+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721642+0000","last_change":"2024-01-24T15:32:20.718322+0000","last_active":"2024-01-24T15:32:20.721642+0000","last_peered":"2024-01-24T15:32:20.721642+0000","last_clean":"2024-01-24T15:32:20.721642+0000","last_became_active":"2024-01-24T15:32:20.716991+0000","last_became_peered":"2024-01-24T15:32:20.716991+0000","last_unstale":"2024-01-24T15:32:20.721642+0000","last_undegraded":"2024-01-24T15:32:20.721642+0000","last_fullsized":"2024-01-24T15:32:20.721642+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.695960+0000","last_change":"2024-01-24T15:32:20.287451+0000","last_active":"2024-01-24T15:32:20.695960+0000","last_peered":"2024-01-24T15:32:20.695960+0000","last_clean":"2024-01-24T15:32:20.695960+0000","last_became_active":"2024-01-24T15:32:20.287139+0000","last_became_peered":"2024-01-24T15:32:20.287139+0000","last_unstale":"2024-01-24T15:32:20.695960+0000","last_undegraded":"2024-01-24T15:32:20.695960+0000","last_fullsized":"2024-01-24T15:32:20.695960+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714185+0000","last_change":"2024-01-24T15:32:20.709005+0000","last_active":"2024-01-24T15:32:20.714185+0000","last_peered":"2024-01-24T15:32:20.714185+0000","last_clean":"2024-01-24T15:32:20.714185+0000","last_became_active":"2024-01-24T15:32:20.708732+0000","last_became_peered":"2024-01-24T15:32:20.708732+0000","last_unstale":"2024-01-24T15:32:20.714185+0000","last_undegraded":"2024-01-24T15:32:20.714185+0000","last_fullsized":"2024-01-24T15:32:20.714185+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703453+0000","last_change":"2024-01-24T15:32:20.274724+0000","last_active":"2024-01-24T15:32:20.703453+0000","last_peered":"2024-01-24T15:32:20.703453+0000","last_clean":"2024-01-24T15:32:20.703453+0000","last_became_active":"2024-01-24T15:32:20.274398+0000","last_became_peered":"2024-01-24T15:32:20.274398+0000","last_unstale":"2024-01-24T15:32:20.703453+0000","last_undegraded":"2024-01-24T15:32:20.703453+0000","last_fullsized":"2024-01-24T15:32:20.703453+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.718113+0000","last_change":"2024-01-24T15:32:20.714659+0000","last_active":"2024-01-24T15:32:20.718113+0000","last_peered":"2024-01-24T15:32:20.718113+0000","last_clean":"2024-01-24T15:32:20.718113+0000","last_became_active":"2024-01-24T15:32:20.714372+0000","last_became_peered":"2024-01-24T15:32:20.714372+0000","last_unstale":"2024-01-24T15:32:20.718113+0000","last_undegraded":"2024-01-24T15:32:20.718113+0000","last_fullsized":"2024-01-24T15:32:20.718113+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"230'2","reported_seq":12,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.709086+0000","last_change":"2024-01-24T15:32:20.280405+0000","last_active":"2024-01-24T15:32:20.709086+0000","last_peered":"2024-01-24T15:32:20.709086+0000","last_clean":"2024-01-24T15:32:20.709086+0000","last_became_active":"2024-01-24T15:32:20.280088+0000","last_became_peered":"2024-01-24T15:32:20.280088+0000","last_unstale":"2024-01-24T15:32:20.709086+0000","last_undegraded":"2024-01-24T15:32:20.709086+0000","last_fullsized":"2024-01-24T15:32:20.709086+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.719681+0000","last_change":"2024-01-24T15:32:20.716121+0000","last_active":"2024-01-24T15:32:20.719681+0000","last_peered":"2024-01-24T15:32:20.719681+0000","last_clean":"2024-01-24T15:32:20.719681+0000","last_became_active":"2024-01-24T15:32:20.714444+0000","last_became_peered":"2024-01-24T15:32:20.714444+0000","last_unstale":"2024-01-24T15:32:20.719681+0000","last_undegraded":"2024-01-24T15:32:20.719681+0000","last_fullsized":"2024-01-24T15:32:20.719681+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699223+0000","last_change":"2024-01-24T15:32:20.276747+0000","last_active":"2024-01-24T15:32:20.699223+0000","last_peered":"2024-01-24T15:32:20.699223+0000","last_clean":"2024-01-24T15:32:20.699223+0000","last_became_active":"2024-01-24T15:32:20.276453+0000","last_became_peered":"2024-01-24T15:32:20.276453+0000","last_unstale":"2024-01-24T15:32:20.699223+0000","last_undegraded":"2024-01-24T15:32:20.699223+0000","last_fullsized":"2024-01-24T15:32:20.699223+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721667+0000","last_change":"2024-01-24T15:32:20.718687+0000","last_active":"2024-01-24T15:32:20.721667+0000","last_peered":"2024-01-24T15:32:20.721667+0000","last_clean":"2024-01-24T15:32:20.721667+0000","last_became_active":"2024-01-24T15:32:20.718208+0000","last_became_peered":"2024-01-24T15:32:20.718208+0000","last_unstale":"2024-01-24T15:32:20.721667+0000","last_undegraded":"2024-01-24T15:32:20.721667+0000","last_fullsized":"2024-01-24T15:32:20.721667+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703742+0000","last_change":"2024-01-24T15:32:20.287469+0000","last_active":"2024-01-24T15:32:20.703742+0000","last_peered":"2024-01-24T15:32:20.703742+0000","last_clean":"2024-01-24T15:32:20.703742+0000","last_became_active":"2024-01-24T15:32:20.287145+0000","last_became_peered":"2024-01-24T15:32:20.287145+0000","last_unstale":"2024-01-24T15:32:20.703742+0000","last_undegraded":"2024-01-24T15:32:20.703742+0000","last_fullsized":"2024-01-24T15:32:20.703742+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714010+0000","last_change":"2024-01-24T15:32:20.707884+0000","last_active":"2024-01-24T15:32:20.714010+0000","last_peered":"2024-01-24T15:32:20.714010+0000","last_clean":"2024-01-24T15:32:20.714010+0000","last_became_active":"2024-01-24T15:32:20.706589+0000","last_became_peered":"2024-01-24T15:32:20.706589+0000","last_unstale":"2024-01-24T15:32:20.714010+0000","last_undegraded":"2024-01-24T15:32:20.714010+0000","last_fullsized":"2024-01-24T15:32:20.714010+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689896+0000","last_change":"2024-01-24T15:32:20.280860+0000","last_active":"2024-01-24T15:32:20.689896+0000","last_peered":"2024-01-24T15:32:20.689896+0000","last_clean":"2024-01-24T15:32:20.689896+0000","last_became_active":"2024-01-24T15:32:20.280544+0000","last_became_peered":"2024-01-24T15:32:20.280544+0000","last_unstale":"2024-01-24T15:32:20.689896+0000","last_undegraded":"2024-01-24T15:32:20.689896+0000","last_fullsized":"2024-01-24T15:32:20.689896+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721488+0000","last_change":"2024-01-24T15:32:20.719006+0000","last_active":"2024-01-24T15:32:20.721488+0000","last_peered":"2024-01-24T15:32:20.721488+0000","last_clean":"2024-01-24T15:32:20.721488+0000","last_became_active":"2024-01-24T15:32:20.716857+0000","last_became_peered":"2024-01-24T15:32:20.716857+0000","last_unstale":"2024-01-24T15:32:20.721488+0000","last_undegraded":"2024-01-24T15:32:20.721488+0000","last_fullsized":"2024-01-24T15:32:20.721488+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699341+0000","last_change":"2024-01-24T15:32:20.284768+0000","last_active":"2024-01-24T15:32:20.699341+0000","last_peered":"2024-01-24T15:32:20.699341+0000","last_clean":"2024-01-24T15:32:20.699341+0000","last_became_active":"2024-01-24T15:32:20.284447+0000","last_became_peered":"2024-01-24T15:32:20.284447+0000","last_unstale":"2024-01-24T15:32:20.699341+0000","last_undegraded":"2024-01-24T15:32:20.699341+0000","last_fullsized":"2024-01-24T15:32:20.699341+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.706537+0000","last_change":"2024-01-24T15:32:20.701039+0000","last_active":"2024-01-24T15:32:20.706537+0000","last_peered":"2024-01-24T15:32:20.706537+0000","last_clean":"2024-01-24T15:32:20.706537+0000","last_became_active":"2024-01-24T15:32:20.700696+0000","last_became_peered":"2024-01-24T15:32:20.700696+0000","last_unstale":"2024-01-24T15:32:20.706537+0000","last_undegraded":"2024-01-24T15:32:20.706537+0000","last_fullsized":"2024-01-24T15:32:20.706537+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.704768+0000","last_change":"2024-01-24T15:32:20.282425+0000","last_active":"2024-01-24T15:32:20.704768+0000","last_peered":"2024-01-24T15:32:20.704768+0000","last_clean":"2024-01-24T15:32:20.704768+0000","last_became_active":"2024-01-24T15:32:20.281461+0000","last_became_peered":"2024-01-24T15:32:20.281461+0000","last_unstale":"2024-01-24T15:32:20.704768+0000","last_undegraded":"2024-01-24T15:32:20.704768+0000","last_fullsized":"2024-01-24T15:32:20.704768+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.700488+0000","last_change":"2024-01-24T15:32:20.695888+0000","last_active":"2024-01-24T15:32:20.700488+0000","last_peered":"2024-01-24T15:32:20.700488+0000","last_clean":"2024-01-24T15:32:20.700488+0000","last_became_active":"2024-01-24T15:32:20.695583+0000","last_became_peered":"2024-01-24T15:32:20.695583+0000","last_unstale":"2024-01-24T15:32:20.700488+0000","last_undegraded":"2024-01-24T15:32:20.700488+0000","last_fullsized":"2024-01-24T15:32:20.700488+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699069+0000","last_change":"2024-01-24T15:32:20.271749+0000","last_active":"2024-01-24T15:32:20.699069+0000","last_peered":"2024-01-24T15:32:20.699069+0000","last_clean":"2024-01-24T15:32:20.699069+0000","last_became_active":"2024-01-24T15:32:20.271536+0000","last_became_peered":"2024-01-24T15:32:20.271536+0000","last_unstale":"2024-01-24T15:32:20.699069+0000","last_undegraded":"2024-01-24T15:32:20.699069+0000","last_fullsized":"2024-01-24T15:32:20.699069+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.720279+0000","last_change":"2024-01-24T15:32:20.712200+0000","last_active":"2024-01-24T15:32:20.720279+0000","last_peered":"2024-01-24T15:32:20.720279+0000","last_clean":"2024-01-24T15:32:20.720279+0000","last_became_active":"2024-01-24T15:32:20.711833+0000","last_became_peered":"2024-01-24T15:32:20.711833+0000","last_unstale":"2024-01-24T15:32:20.720279+0000","last_undegraded":"2024-01-24T15:32:20.720279+0000","last_fullsized":"2024-01-24T15:32:20.720279+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.693885+0000","last_change":"2024-01-24T15:32:20.287927+0000","last_active":"2024-01-24T15:32:20.693885+0000","last_peered":"2024-01-24T15:32:20.693885+0000","last_clean":"2024-01-24T15:32:20.693885+0000","last_became_active":"2024-01-24T15:32:20.287600+0000","last_became_peered":"2024-01-24T15:32:20.287600+0000","last_unstale":"2024-01-24T15:32:20.693885+0000","last_undegraded":"2024-01-24T15:32:20.693885+0000","last_fullsized":"2024-01-24T15:32:20.693885+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711821+0000","last_change":"2024-01-24T15:32:20.707934+0000","last_active":"2024-01-24T15:32:20.711821+0000","last_peered":"2024-01-24T15:32:20.711821+0000","last_clean":"2024-01-24T15:32:20.711821+0000","last_became_active":"2024-01-24T15:32:20.707715+0000","last_became_peered":"2024-01-24T15:32:20.707715+0000","last_unstale":"2024-01-24T15:32:20.711821+0000","last_undegraded":"2024-01-24T15:32:20.711821+0000","last_fullsized":"2024-01-24T15:32:20.711821+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699558+0000","last_change":"2024-01-24T15:32:20.287254+0000","last_active":"2024-01-24T15:32:20.699558+0000","last_peered":"2024-01-24T15:32:20.699558+0000","last_clean":"2024-01-24T15:32:20.699558+0000","last_became_active":"2024-01-24T15:32:20.286952+0000","last_became_peered":"2024-01-24T15:32:20.286952+0000","last_unstale":"2024-01-24T15:32:20.699558+0000","last_undegraded":"2024-01-24T15:32:20.699558+0000","last_fullsized":"2024-01-24T15:32:20.699558+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":187,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703800+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:32:20.703800+0000","last_peered":"2024-01-24T15:32:20.703800+0000","last_clean":"2024-01-24T15:32:20.703800+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:32:20.703800+0000","last_undegraded":"2024-01-24T15:32:20.703800+0000","last_fullsized":"2024-01-24T15:32:20.703800+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714191+0000","last_change":"2024-01-24T15:32:20.708196+0000","last_active":"2024-01-24T15:32:20.714191+0000","last_peered":"2024-01-24T15:32:20.714191+0000","last_clean":"2024-01-24T15:32:20.714191+0000","last_became_active":"2024-01-24T15:32:20.707375+0000","last_became_peered":"2024-01-24T15:32:20.707375+0000","last_unstale":"2024-01-24T15:32:20.714191+0000","last_undegraded":"2024-01-24T15:32:20.714191+0000","last_fullsized":"2024-01-24T15:32:20.714191+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.704868+0000","last_change":"2024-01-24T15:32:20.285721+0000","last_active":"2024-01-24T15:32:20.704868+0000","last_peered":"2024-01-24T15:32:20.704868+0000","last_clean":"2024-01-24T15:32:20.704868+0000","last_became_active":"2024-01-24T15:32:20.281894+0000","last_became_peered":"2024-01-24T15:32:20.281894+0000","last_unstale":"2024-01-24T15:32:20.704868+0000","last_undegraded":"2024-01-24T15:32:20.704868+0000","last_fullsized":"2024-01-24T15:32:20.704868+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":213,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.705777+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:32:20.705777+0000","last_peered":"2024-01-24T15:32:20.705777+0000","last_clean":"2024-01-24T15:32:20.705777+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:32:20.705777+0000","last_undegraded":"2024-01-24T15:32:20.705777+0000","last_fullsized":"2024-01-24T15:32:20.705777+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.718212+0000","last_change":"2024-01-24T15:32:20.715254+0000","last_active":"2024-01-24T15:32:20.718212+0000","last_peered":"2024-01-24T15:32:20.718212+0000","last_clean":"2024-01-24T15:32:20.718212+0000","last_became_active":"2024-01-24T15:32:20.714892+0000","last_became_peered":"2024-01-24T15:32:20.714892+0000","last_unstale":"2024-01-24T15:32:20.718212+0000","last_undegraded":"2024-01-24T15:32:20.718212+0000","last_fullsized":"2024-01-24T15:32:20.718212+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703779+0000","last_change":"2024-01-24T15:32:20.284194+0000","last_active":"2024-01-24T15:32:20.703779+0000","last_peered":"2024-01-24T15:32:20.703779+0000","last_clean":"2024-01-24T15:32:20.703779+0000","last_became_active":"2024-01-24T15:32:20.283879+0000","last_became_peered":"2024-01-24T15:32:20.283879+0000","last_unstale":"2024-01-24T15:32:20.703779+0000","last_undegraded":"2024-01-24T15:32:20.703779+0000","last_fullsized":"2024-01-24T15:32:20.703779+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":208,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689230+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:32:20.689230+0000","last_peered":"2024-01-24T15:32:20.689230+0000","last_clean":"2024-01-24T15:32:20.689230+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:32:20.689230+0000","last_undegraded":"2024-01-24T15:32:20.689230+0000","last_fullsized":"2024-01-24T15:32:20.689230+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717102+0000","last_change":"2024-01-24T15:32:20.705419+0000","last_active":"2024-01-24T15:32:20.717102+0000","last_peered":"2024-01-24T15:32:20.717102+0000","last_clean":"2024-01-24T15:32:20.717102+0000","last_became_active":"2024-01-24T15:32:20.705138+0000","last_became_peered":"2024-01-24T15:32:20.705138+0000","last_unstale":"2024-01-24T15:32:20.717102+0000","last_undegraded":"2024-01-24T15:32:20.717102+0000","last_fullsized":"2024-01-24T15:32:20.717102+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.694974+0000","last_change":"2024-01-24T15:32:20.281846+0000","last_active":"2024-01-24T15:32:20.694974+0000","last_peered":"2024-01-24T15:32:20.694974+0000","last_clean":"2024-01-24T15:32:20.694974+0000","last_became_active":"2024-01-24T15:32:20.281577+0000","last_became_peered":"2024-01-24T15:32:20.281577+0000","last_unstale":"2024-01-24T15:32:20.694974+0000","last_undegraded":"2024-01-24T15:32:20.694974+0000","last_fullsized":"2024-01-24T15:32:20.694974+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":213,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.707255+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:32:20.707255+0000","last_peered":"2024-01-24T15:32:20.707255+0000","last_clean":"2024-01-24T15:32:20.707255+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:32:20.707255+0000","last_undegraded":"2024-01-24T15:32:20.707255+0000","last_fullsized":"2024-01-24T15:32:20.707255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":190,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699920+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:32:20.699920+0000","last_peered":"2024-01-24T15:32:20.699920+0000","last_clean":"2024-01-24T15:32:20.699920+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:32:20.699920+0000","last_undegraded":"2024-01-24T15:32:20.699920+0000","last_fullsized":"2024-01-24T15:32:20.699920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714413+0000","last_change":"2024-01-24T15:32:20.710149+0000","last_active":"2024-01-24T15:32:20.714413+0000","last_peered":"2024-01-24T15:32:20.714413+0000","last_clean":"2024-01-24T15:32:20.714413+0000","last_became_active":"2024-01-24T15:32:20.709851+0000","last_became_peered":"2024-01-24T15:32:20.709851+0000","last_unstale":"2024-01-24T15:32:20.714413+0000","last_undegraded":"2024-01-24T15:32:20.714413+0000","last_fullsized":"2024-01-24T15:32:20.714413+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.696473+0000","last_change":"2024-01-24T15:32:20.277807+0000","last_active":"2024-01-24T15:32:20.696473+0000","last_peered":"2024-01-24T15:32:20.696473+0000","last_clean":"2024-01-24T15:32:20.696473+0000","last_became_active":"2024-01-24T15:32:20.277507+0000","last_became_peered":"2024-01-24T15:32:20.277507+0000","last_unstale":"2024-01-24T15:32:20.696473+0000","last_undegraded":"2024-01-24T15:32:20.696473+0000","last_fullsized":"2024-01-24T15:32:20.696473+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":188,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699831+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:32:20.699831+0000","last_peered":"2024-01-24T15:32:20.699831+0000","last_clean":"2024-01-24T15:32:20.699831+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:32:20.699831+0000","last_undegraded":"2024-01-24T15:32:20.699831+0000","last_fullsized":"2024-01-24T15:32:20.699831+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721388+0000","last_change":"2024-01-24T15:32:20.718896+0000","last_active":"2024-01-24T15:32:20.721388+0000","last_peered":"2024-01-24T15:32:20.721388+0000","last_clean":"2024-01-24T15:32:20.721388+0000","last_became_active":"2024-01-24T15:32:20.716591+0000","last_became_peered":"2024-01-24T15:32:20.716591+0000","last_unstale":"2024-01-24T15:32:20.721388+0000","last_undegraded":"2024-01-24T15:32:20.721388+0000","last_fullsized":"2024-01-24T15:32:20.721388+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"230'2","reported_seq":12,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711777+0000","last_change":"2024-01-24T15:32:20.279372+0000","last_active":"2024-01-24T15:32:20.711777+0000","last_peered":"2024-01-24T15:32:20.711777+0000","last_clean":"2024-01-24T15:32:20.711777+0000","last_became_active":"2024-01-24T15:32:20.279083+0000","last_became_peered":"2024-01-24T15:32:20.279083+0000","last_unstale":"2024-01-24T15:32:20.711777+0000","last_undegraded":"2024-01-24T15:32:20.711777+0000","last_fullsized":"2024-01-24T15:32:20.711777+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":188,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699277+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:32:20.699277+0000","last_peered":"2024-01-24T15:32:20.699277+0000","last_clean":"2024-01-24T15:32:20.699277+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:32:20.699277+0000","last_undegraded":"2024-01-24T15:32:20.699277+0000","last_fullsized":"2024-01-24T15:32:20.699277+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.713979+0000","last_change":"2024-01-24T15:32:20.708663+0000","last_active":"2024-01-24T15:32:20.713979+0000","last_peered":"2024-01-24T15:32:20.713979+0000","last_clean":"2024-01-24T15:32:20.713979+0000","last_became_active":"2024-01-24T15:32:20.708364+0000","last_became_peered":"2024-01-24T15:32:20.708364+0000","last_unstale":"2024-01-24T15:32:20.713979+0000","last_undegraded":"2024-01-24T15:32:20.713979+0000","last_fullsized":"2024-01-24T15:32:20.713979+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721234+0000","last_change":"2024-01-24T15:32:20.282694+0000","last_active":"2024-01-24T15:32:20.721234+0000","last_peered":"2024-01-24T15:32:20.721234+0000","last_clean":"2024-01-24T15:32:20.721234+0000","last_became_active":"2024-01-24T15:32:20.282008+0000","last_became_peered":"2024-01-24T15:32:20.282008+0000","last_unstale":"2024-01-24T15:32:20.721234+0000","last_undegraded":"2024-01-24T15:32:20.721234+0000","last_fullsized":"2024-01-24T15:32:20.721234+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":187,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703902+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:32:20.703902+0000","last_peered":"2024-01-24T15:32:20.703902+0000","last_clean":"2024-01-24T15:32:20.703902+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:32:20.703902+0000","last_undegraded":"2024-01-24T15:32:20.703902+0000","last_fullsized":"2024-01-24T15:32:20.703902+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721696+0000","last_change":"2024-01-24T15:32:20.718975+0000","last_active":"2024-01-24T15:32:20.721696+0000","last_peered":"2024-01-24T15:32:20.721696+0000","last_clean":"2024-01-24T15:32:20.721696+0000","last_became_active":"2024-01-24T15:32:20.718720+0000","last_became_peered":"2024-01-24T15:32:20.718720+0000","last_unstale":"2024-01-24T15:32:20.721696+0000","last_undegraded":"2024-01-24T15:32:20.721696+0000","last_fullsized":"2024-01-24T15:32:20.721696+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.707048+0000","last_change":"2024-01-24T15:32:20.276859+0000","last_active":"2024-01-24T15:32:20.707048+0000","last_peered":"2024-01-24T15:32:20.707048+0000","last_clean":"2024-01-24T15:32:20.707048+0000","last_became_active":"2024-01-24T15:32:20.276557+0000","last_became_peered":"2024-01-24T15:32:20.276557+0000","last_unstale":"2024-01-24T15:32:20.707048+0000","last_undegraded":"2024-01-24T15:32:20.707048+0000","last_fullsized":"2024-01-24T15:32:20.707048+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":186,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.695447+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:32:20.695447+0000","last_peered":"2024-01-24T15:32:20.695447+0000","last_clean":"2024-01-24T15:32:20.695447+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:32:20.695447+0000","last_undegraded":"2024-01-24T15:32:20.695447+0000","last_fullsized":"2024-01-24T15:32:20.695447+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575973,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304032,"kb_used_data":2840,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94067808,"statfs":{"total":96636764160,"available":96325435392,"internally_reserved":0,"allocated":2908160,"data_stored":1683156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.47499999999999998,"5min":0.39300000000000002,"15min":0.32000000000000001},"max":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.83199999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.75700000000000001,"5min":0.84799999999999998,"15min":0.85499999999999998},"last":0.621}]},{"osd":1,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.61299999999999999,"5min":0.374,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":0.878,"15min":0.878},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.89600000000000002},"last":0.63400000000000001}]},{"osd":2,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.55900000000000005,"5min":0.374,"15min":0.374},"max":{"1min":0.82899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80800000000000005,"5min":0.83299999999999996,"15min":0.85099999999999998},"last":0.58699999999999997}]},{"osd":3,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":0.82399999999999995,"5min":0.83199999999999996,"15min":0.92600000000000005},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.50700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.88},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51400000000000001,"15min":0.49399999999999999},"min":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.30599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.5}]},{"osd":5,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.439,"5min":0.35899999999999999,"15min":0.32300000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.76100000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.42399999999999999,"5min":0.39000000000000001,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81899999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.623,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.40999999999999998,"15min":0.35099999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.92700000000000005},"last":0.67200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575974,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308100,"kb_used_data":2812,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063740,"statfs":{"total":96636764160,"available":96321269760,"internally_reserved":0,"allocated":2879488,"data_stored":1673269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.442,"5min":0.377,"15min":0.31900000000000001},"max":{"1min":0.83699999999999997,"5min":0.998,"15min":0.998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.61699999999999999},"min":{"1min":0.48599999999999999,"5min":0.44700000000000001,"15min":0.32800000000000001},"max":{"1min":0.82399999999999995,"5min":0.82699999999999996,"15min":0.95599999999999996},"last":0.36599999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":1.089},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.45000000000000001,"5min":0.41899999999999998,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.85599999999999998,"15min":0.94299999999999995},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.627},"min":{"1min":0.379,"5min":0.379,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.443},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.90900000000000003},"last":0.79000000000000004}]},{"osd":3,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64200000000000002,"15min":0.63900000000000001},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.371},"max":{"1min":0.879,"5min":0.96299999999999997,"15min":1.0980000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.42299999999999999},"max":{"1min":0.78300000000000003,"5min":0.83899999999999997,"15min":1.169},"last":0.50700000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":0.73599999999999999,"5min":0.753,"15min":0.753},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.33100000000000002,"15min":0.312},"max":{"1min":0.79100000000000004,"5min":0.81399999999999995,"15min":0.83999999999999997},"last":0.66400000000000003}]},{"osd":5,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":0.82499999999999996,"15min":0.86499999999999999},"last":0.628},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69599999999999995,"5min":0.81699999999999995,"15min":0.82799999999999996},"last":0.86699999999999999}]},{"osd":7,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71099999999999997,"5min":0.83699999999999997,"15min":0.88},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.83599999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575973,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308112,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063728,"statfs":{"total":96636764160,"available":96321257472,"internally_reserved":0,"allocated":2891776,"data_stored":1673879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.41799999999999998},"min":{"1min":0.28999999999999998,"5min":0.26600000000000001,"15min":0.24299999999999999},"max":{"1min":0.91200000000000003,"5min":3.254,"15min":3.254},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.26100000000000001},"max":{"1min":0.82299999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.28100000000000003}]},{"osd":2,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.34200000000000003,"5min":0.30299999999999999,"15min":0.28499999999999998},"max":{"1min":0.86499999999999999,"5min":1.306,"15min":1.306},"last":0.318},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.45200000000000001},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.46600000000000003},"min":{"1min":0.31900000000000001,"5min":0.312,"15min":0.308},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.36899999999999999,"5min":0.34100000000000003,"15min":0.32600000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.33400000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73799999999999999,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47299999999999998},"max":{"1min":1.1080000000000001,"5min":5.3630000000000004,"15min":5.3630000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.55500000000000005,"5min":0.46300000000000002,"15min":0.40400000000000003},"max":{"1min":1.244,"5min":1.677,"15min":1.677},"last":0.69899999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.71699999999999997},"min":{"1min":0.53900000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.232,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.56100000000000005,"5min":0.47399999999999998,"15min":0.46100000000000002},"max":{"1min":1.081,"5min":1.117,"15min":1.2989999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.54100000000000004,"5min":0.47899999999999998,"15min":0.46800000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.57599999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1859999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.72499999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.58099999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1759999999999999,"5min":1.704,"15min":1.704},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81100000000000005,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1599999999999999,"5min":5.29,"15min":5.29},"last":0.67100000000000004}]}]},{"osd":0,"up_from":14,"seq":60129543271,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312200,"kb_used_data":2816,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059640,"statfs":{"total":96636764160,"available":96317071360,"internally_reserved":0,"allocated":2883584,"data_stored":1673290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.40799999999999997},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.26000000000000001},"max":{"1min":0.65600000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.27300000000000002},"max":{"1min":0.66100000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47899999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.318,"15min":0.29599999999999999},"max":{"1min":0.73399999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.46899999999999997,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":0.64600000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42899999999999999}]},{"osd":3,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.44500000000000001,"15min":0.442},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59299999999999997,"5min":0.67400000000000004,"15min":0.72799999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76600000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.98699999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.51900000000000002,"5min":0.45100000000000001,"15min":0.41199999999999998},"max":{"1min":0.92700000000000005,"5min":0.998,"15min":0.998},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67900000000000005,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.40699999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.0029999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.55800000000000005,"5min":0.46700000000000003,"15min":0.46500000000000002},"max":{"1min":1.0960000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.53500000000000003}]},{"osd":6,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97799999999999998,"5min":1.27,"15min":1.27},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.61799999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.746}]},{"osd":7,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72799999999999998,"15min":0.73099999999999998},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98699999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.55600000000000005,"5min":0.51000000000000001,"15min":0.46600000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.64000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575973,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312200,"kb_used_data":2816,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059640,"statfs":{"total":96636764160,"available":96317071360,"internally_reserved":0,"allocated":2883584,"data_stored":1673339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.69699999999999995},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.60899999999999999,"5min":0.66600000000000004,"15min":0.70599999999999996},"last":0.54200000000000004}]},{"osd":1,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.432,"15min":0.432},"min":{"1min":0.32200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.40200000000000002,"5min":0.32600000000000001,"15min":0.27400000000000002},"max":{"1min":0.66400000000000003,"5min":0.69799999999999995,"15min":0.80900000000000005},"last":0.51200000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.29099999999999998},"max":{"1min":0.623,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.57799999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78500000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.753},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.55100000000000005,"5min":0.45000000000000001,"15min":0.376},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.69199999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":0.996,"15min":1.0589999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.45700000000000002},"max":{"1min":0.95999999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.39700000000000002},"max":{"1min":0.91800000000000004,"5min":1.006,"15min":1.0740000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.51700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.89200000000000002,"5min":1.0229999999999999,"15min":1.083},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96599999999999997,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.75800000000000001},"min":{"1min":0.61299999999999999,"5min":0.52000000000000002,"15min":0.51100000000000001},"max":{"1min":0.92000000000000004,"5min":1.0429999999999999,"15min":1.123},"last":0.82799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575973,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308120,"kb_used_data":2832,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063720,"statfs":{"total":96636764160,"available":96321249280,"internally_reserved":0,"allocated":2899968,"data_stored":1681178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.318,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.438},"min":{"1min":0.35299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.76100000000000001},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42799999999999999},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.65000000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.752},"last":0.432}]},{"osd":2,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.80800000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.81899999999999995},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66000000000000003,"15min":0.66100000000000003},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.437},"max":{"1min":0.80100000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.498},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.93899999999999995,"5min":1.026,"15min":1.026},"last":0.47099999999999997}]},{"osd":5,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.056,"15min":1.056},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.499,"15min":0.47199999999999998},"max":{"1min":0.93700000000000006,"5min":1.095,"15min":1.095},"last":0.57099999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":1.1499999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72099999999999997,"15min":0.72199999999999998},"min":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.0940000000000001},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.55000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.018,"5min":1.071,"15min":1.083},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.53600000000000003},"max":{"1min":0.95399999999999996,"5min":1.081,"15min":1.1359999999999999},"last":0.56000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575974,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312224,"kb_used_data":2840,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059616,"statfs":{"total":96636764160,"available":96317046784,"internally_reserved":0,"allocated":2908160,"data_stored":1681814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.46700000000000003,"5min":0.39400000000000002,"15min":0.375},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.503},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.80100000000000005,"5min":0.876,"15min":0.876},"last":0.52500000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.622},"min":{"1min":0.496,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.45900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.79100000000000004,"5min":0.80400000000000005,"15min":0.82199999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.40500000000000003,"15min":0.38500000000000001},"max":{"1min":0.82899999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.61699999999999999}]},{"osd":3,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82299999999999995,"5min":0.86799999999999999,"15min":0.87},"last":0.69699999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.69699999999999995,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.33800000000000002,"5min":0.31900000000000001,"15min":0.308},"max":{"1min":0.66200000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.93600000000000005}]},{"osd":6,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.33600000000000002},"max":{"1min":0.61199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.37},"max":{"1min":0.72599999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.34899999999999998},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.375,"15min":0.35799999999999998},"max":{"1min":0.78100000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.83199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308112,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063728,"statfs":{"total":96636764160,"available":96321257472,"internally_reserved":0,"allocated":2891776,"data_stored":1681066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.34799999999999998},"max":{"1min":0.88300000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.624},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.48399999999999999,"5min":0.41999999999999998,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":1.149,"15min":1.149},"last":0.61099999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.52800000000000002,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":1.125,"15min":1.125},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.38400000000000001,"15min":0.38},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54200000000000004}]},{"osd":2,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.432,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77900000000000003,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.61599999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.41199999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":1.252},"last":0.55500000000000005}]},{"osd":3,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.51100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.95499999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.64400000000000002},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":1.196,"15min":1.196},"last":0.65000000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.58699999999999997,"5min":0.78700000000000003,"15min":0.90500000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.81299999999999994},"last":0.52400000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.50600000000000001},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68400000000000005,"5min":0.82199999999999995,"15min":1.0209999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.54200000000000004},"min":{"1min":0.41099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.98399999999999999},"last":0.56499999999999995}]},{"osd":7,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.54200000000000004},"min":{"1min":0.39700000000000002,"5min":0.371,"15min":0.33600000000000002},"max":{"1min":0.629,"5min":0.80100000000000005,"15min":0.86399999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.57799999999999996},"min":{"1min":0.372,"5min":0.371,"15min":0.371},"max":{"1min":0.80900000000000005,"5min":0.83099999999999996,"15min":0.91700000000000004},"last":0.66400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:32:24.829 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:32:24.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.989+0000 7f65aa495700 1 Processor -- start 2024-01-24T15:32:24.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.989+0000 7f65aa495700 1 -- start start 2024-01-24T15:32:24.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.990+0000 7f65aa495700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65a41540a0 0x7f65a40c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.990+0000 7f65aa495700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f65a41550f0 0x7f65a40627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.990+0000 7f65aa495700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a4065360 con 0x7f65a4062ca0 2024-01-24T15:32:24.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.990+0000 7f65aa495700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a40a74f0 con 0x7f65a41550f0 2024-01-24T15:32:24.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.990+0000 7f65aa495700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a40a6ce0 con 0x7f65a41540a0 2024-01-24T15:32:24.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.990+0000 7f65a3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65a41540a0 0x7f65a40c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.990+0000 7f65a37fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f65a41550f0 0x7f65a40627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.990+0000 7f65a8a32700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f65a4062ca0 0x7f65a4157c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:51960/0 (socket says 172.21.15.29:51960) 2024-01-24T15:32:24.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.990+0000 7f65a8a32700 1 -- 172.21.15.29:0/1525309096 learned_addr learned my addr 172.21.15.29:0/1525309096 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:24.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.990+0000 7f65a2ffd700 1 -- 172.21.15.29:0/1525309096 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134787174 0 0) 0x7f65a4065360 con 0x7f65a4062ca0 2024-01-24T15:32:24.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.991+0000 7f65a2ffd700 1 -- 172.21.15.29:0/1525309096 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f658c0033f0 con 0x7f65a4062ca0 2024-01-24T15:32:24.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.991+0000 7f65a3fff700 1 -- 172.21.15.29:0/1525309096 >> v2:172.21.15.163:3300/0 conn(0x7f65a41550f0 msgr2=0x7f65a40627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:24.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.991+0000 7f65a3fff700 1 --2- 172.21.15.29:0/1525309096 >> v2:172.21.15.163:3300/0 conn(0x7f65a41550f0 0x7f65a40627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.991+0000 7f65a3fff700 1 -- 172.21.15.29:0/1525309096 >> v1:172.21.15.163:6789/0 conn(0x7f65a4062ca0 legacy=0x7f65a4157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.991+0000 7f65a3fff700 1 -- 172.21.15.29:0/1525309096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a4063130 con 0x7f65a41540a0 2024-01-24T15:32:24.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.991+0000 7f65a3fff700 1 --2- 172.21.15.29:0/1525309096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65a41540a0 0x7f65a40c4ca0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f6594009760 tx=0x7f6594009060).ready entity=mon.2 client_cookie=bc1543653fa88f7 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:24.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.991+0000 7f65a2ffd700 1 -- 172.21.15.29:0/1525309096 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659400d4a0 con 0x7f65a41540a0 2024-01-24T15:32:24.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.991+0000 7f65a2ffd700 1 -- 172.21.15.29:0/1525309096 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659400d600 con 0x7f65a41540a0 2024-01-24T15:32:24.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.991+0000 7f65a2ffd700 1 -- 172.21.15.29:0/1525309096 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659400d980 con 0x7f65a41540a0 2024-01-24T15:32:24.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.991+0000 7f65aa495700 1 -- 172.21.15.29:0/1525309096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65a41540a0 msgr2=0x7f65a40c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.991+0000 7f65aa495700 1 --2- 172.21.15.29:0/1525309096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65a41540a0 0x7f65a40c4ca0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f6594009760 tx=0x7f6594009060).stop 2024-01-24T15:32:24.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.992+0000 7f65aa495700 1 -- 172.21.15.29:0/1525309096 shutdown_connections 2024-01-24T15:32:24.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.992+0000 7f65aa495700 1 --2- 172.21.15.29:0/1525309096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65a41540a0 0x7f65a40c4ca0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.992+0000 7f65aa495700 1 --2- 172.21.15.29:0/1525309096 >> v2:172.21.15.163:3300/0 conn(0x7f65a41550f0 0x7f65a40627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.992+0000 7f65aa495700 1 -- 172.21.15.29:0/1525309096 >> 172.21.15.29:0/1525309096 conn(0x7f65a40c85c0 msgr2=0x7f65a40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:24.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.992+0000 7f65aa495700 1 -- 172.21.15.29:0/1525309096 shutdown_connections 2024-01-24T15:32:24.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.992+0000 7f65aa495700 1 -- 172.21.15.29:0/1525309096 wait complete. 2024-01-24T15:32:24.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.993+0000 7f65aa495700 1 Processor -- start 2024-01-24T15:32:24.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.993+0000 7f65aa495700 1 -- start start 2024-01-24T15:32:24.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65aa495700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65a4062ca0 0x7f65a40ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65aa495700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a41540a0 0x7f65a40aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65aa495700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65a41550f0 0x7f65a4152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:24.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65aa495700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a40abf00 con 0x7f65a41540a0 2024-01-24T15:32:24.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65aa495700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a40ab2f0 con 0x7f65a4062ca0 2024-01-24T15:32:24.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65aa495700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a40ab5d0 con 0x7f65a41550f0 2024-01-24T15:32:24.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65a37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a41540a0 0x7f65a40aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65a37fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a41540a0 0x7f65a40aaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55194/0 (socket says 172.21.15.29:55194) 2024-01-24T15:32:24.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65a37fe700 1 -- 172.21.15.29:0/1328081893 learned_addr learned my addr 172.21.15.29:0/1328081893 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:24.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65a3fff700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65a4062ca0 0x7f65a40ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65a8a32700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65a41550f0 0x7f65a4152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:24.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65a37fe700 1 -- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65a41550f0 msgr2=0x7f65a4152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65a37fe700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65a41550f0 0x7f65a4152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65a37fe700 1 -- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65a4062ca0 msgr2=0x7f65a40ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:24.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65a37fe700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65a4062ca0 0x7f65a40ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:24.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.994+0000 7f65a37fe700 1 -- 172.21.15.29:0/1328081893 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65940092a0 con 0x7f65a41540a0 2024-01-24T15:32:24.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.995+0000 7f65a37fe700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a41540a0 0x7f65a40aaaa0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 rx=0x7f659800e8d0 tx=0x7f6598009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:24.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.995+0000 7f658bfff700 1 -- 172.21.15.29:0/1328081893 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6598009390 con 0x7f65a41540a0 2024-01-24T15:32:24.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.995+0000 7f658bfff700 1 -- 172.21.15.29:0/1328081893 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6598012600 con 0x7f65a41540a0 2024-01-24T15:32:24.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.995+0000 7f65aa495700 1 -- 172.21.15.29:0/1328081893 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a40ab8b0 con 0x7f65a41540a0 2024-01-24T15:32:24.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.995+0000 7f658bfff700 1 -- 172.21.15.29:0/1328081893 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6598012980 con 0x7f65a41540a0 2024-01-24T15:32:24.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.995+0000 7f65aa495700 1 -- 172.21.15.29:0/1328081893 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a40aaf90 con 0x7f65a41540a0 2024-01-24T15:32:24.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.996+0000 7f65aa495700 1 -- 172.21.15.29:0/1328081893 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a40ac070 con 0x7f65a41540a0 2024-01-24T15:32:25.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.996+0000 7f658bfff700 1 -- 172.21.15.29:0/1328081893 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6598012d30 con 0x7f65a41540a0 2024-01-24T15:32:25.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.997+0000 7f658bfff700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f659005f270 0x7f6590061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.997+0000 7f658bfff700 1 -- 172.21.15.29:0/1328081893 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f659807aa80 con 0x7f65a41540a0 2024-01-24T15:32:25.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.997+0000 7f65a3fff700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f659005f270 0x7f6590061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:24.997+0000 7f65a3fff700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f659005f270 0x7f6590061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f65a4167430 tx=0x7f6594016000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:25.002 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.001+0000 7f658bfff700 1 -- 172.21.15.29:0/1328081893 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f659804b790 con 0x7f65a41540a0 2024-01-24T15:32:25.177 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.176+0000 7f65aa495700 1 -- 172.21.15.29:0/1328081893 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65a40abd50 con 0x7f659005f270 2024-01-24T15:32:25.188 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.187+0000 7f658bfff700 1 -- 172.21.15.29:0/1328081893 <== mgr.9352 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361599 (secure 0 0 0) 0x7f65a40abd50 con 0x7f659005f270 2024-01-24T15:32:25.196 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:25.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.191+0000 7f65aa495700 1 -- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f659005f270 msgr2=0x7f6590061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:25.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.191+0000 7f65aa495700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f659005f270 0x7f6590061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f65a4167430 tx=0x7f6594016000).stop 2024-01-24T15:32:25.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.191+0000 7f65aa495700 1 -- 172.21.15.29:0/1328081893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a41540a0 msgr2=0x7f65a40aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:25.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.191+0000 7f65aa495700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a41540a0 0x7f65a40aaaa0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 rx=0x7f659800e8d0 tx=0x7f6598009060).stop 2024-01-24T15:32:25.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.191+0000 7f65aa495700 1 -- 172.21.15.29:0/1328081893 shutdown_connections 2024-01-24T15:32:25.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.191+0000 7f65aa495700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f659005f270 0x7f6590061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.191+0000 7f65aa495700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65a41550f0 0x7f65a4152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.191+0000 7f65aa495700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65a4062ca0 0x7f65a40ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.191+0000 7f65aa495700 1 --2- 172.21.15.29:0/1328081893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65a41540a0 0x7f65a40aaaa0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.191+0000 7f65aa495700 1 -- 172.21.15.29:0/1328081893 >> 172.21.15.29:0/1328081893 conn(0x7f65a40c85c0 msgr2=0x7f65a4065b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:25.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.191+0000 7f65aa495700 1 -- 172.21.15.29:0/1328081893 shutdown_connections 2024-01-24T15:32:25.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.191+0000 7f65aa495700 1 -- 172.21.15.29:0/1328081893 wait complete. 2024-01-24T15:32:25.201 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:32:25.206 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T15:32:23.186668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2473100,"kb_used_data":22604,"kb_used_omap":0,"kb_used_meta":2450432,"kb_avail":752501620,"statfs":{"total":773094113280,"available":770561658880,"internally_reserved":0,"allocated":23146496,"data_stored":13420991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2509242368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28801,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-81,"num_write_kb":-63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.014086"},"pg_stats":[{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721584+0000","last_change":"2024-01-24T15:32:20.716703+0000","last_active":"2024-01-24T15:32:20.721584+0000","last_peered":"2024-01-24T15:32:20.721584+0000","last_clean":"2024-01-24T15:32:20.721584+0000","last_became_active":"2024-01-24T15:32:20.716088+0000","last_became_peered":"2024-01-24T15:32:20.716088+0000","last_unstale":"2024-01-24T15:32:20.721584+0000","last_undegraded":"2024-01-24T15:32:20.721584+0000","last_fullsized":"2024-01-24T15:32:20.721584+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3f","version":"230'3","reported_seq":13,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717200+0000","last_change":"2024-01-24T15:32:20.282862+0000","last_active":"2024-01-24T15:32:20.717200+0000","last_peered":"2024-01-24T15:32:20.717200+0000","last_clean":"2024-01-24T15:32:20.717200+0000","last_became_active":"2024-01-24T15:32:20.282210+0000","last_became_peered":"2024-01-24T15:32:20.282210+0000","last_unstale":"2024-01-24T15:32:20.717200+0000","last_undegraded":"2024-01-24T15:32:20.717200+0000","last_fullsized":"2024-01-24T15:32:20.717200+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.718080+0000","last_change":"2024-01-24T15:32:20.715833+0000","last_active":"2024-01-24T15:32:20.718080+0000","last_peered":"2024-01-24T15:32:20.718080+0000","last_clean":"2024-01-24T15:32:20.718080+0000","last_became_active":"2024-01-24T15:32:20.714863+0000","last_became_peered":"2024-01-24T15:32:20.714863+0000","last_unstale":"2024-01-24T15:32:20.718080+0000","last_undegraded":"2024-01-24T15:32:20.718080+0000","last_fullsized":"2024-01-24T15:32:20.718080+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703777+0000","last_change":"2024-01-24T15:32:20.278260+0000","last_active":"2024-01-24T15:32:20.703777+0000","last_peered":"2024-01-24T15:32:20.703777+0000","last_clean":"2024-01-24T15:32:20.703777+0000","last_became_active":"2024-01-24T15:32:20.277911+0000","last_became_peered":"2024-01-24T15:32:20.277911+0000","last_unstale":"2024-01-24T15:32:20.703777+0000","last_undegraded":"2024-01-24T15:32:20.703777+0000","last_fullsized":"2024-01-24T15:32:20.703777+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.713435+0000","last_change":"2024-01-24T15:32:20.706244+0000","last_active":"2024-01-24T15:32:20.713435+0000","last_peered":"2024-01-24T15:32:20.713435+0000","last_clean":"2024-01-24T15:32:20.713435+0000","last_became_active":"2024-01-24T15:32:20.705940+0000","last_became_peered":"2024-01-24T15:32:20.705940+0000","last_unstale":"2024-01-24T15:32:20.713435+0000","last_undegraded":"2024-01-24T15:32:20.713435+0000","last_fullsized":"2024-01-24T15:32:20.713435+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.694717+0000","last_change":"2024-01-24T15:32:20.275633+0000","last_active":"2024-01-24T15:32:20.694717+0000","last_peered":"2024-01-24T15:32:20.694717+0000","last_clean":"2024-01-24T15:32:20.694717+0000","last_became_active":"2024-01-24T15:32:20.275324+0000","last_became_peered":"2024-01-24T15:32:20.275324+0000","last_unstale":"2024-01-24T15:32:20.694717+0000","last_undegraded":"2024-01-24T15:32:20.694717+0000","last_fullsized":"2024-01-24T15:32:20.694717+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.706306+0000","last_change":"2024-01-24T15:32:20.702709+0000","last_active":"2024-01-24T15:32:20.706306+0000","last_peered":"2024-01-24T15:32:20.706306+0000","last_clean":"2024-01-24T15:32:20.706306+0000","last_became_active":"2024-01-24T15:32:20.702362+0000","last_became_peered":"2024-01-24T15:32:20.702362+0000","last_unstale":"2024-01-24T15:32:20.706306+0000","last_undegraded":"2024-01-24T15:32:20.706306+0000","last_fullsized":"2024-01-24T15:32:20.706306+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3c","version":"230'2","reported_seq":12,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.715604+0000","last_change":"2024-01-24T15:32:20.286812+0000","last_active":"2024-01-24T15:32:20.715604+0000","last_peered":"2024-01-24T15:32:20.715604+0000","last_clean":"2024-01-24T15:32:20.715604+0000","last_became_active":"2024-01-24T15:32:20.286524+0000","last_became_peered":"2024-01-24T15:32:20.286524+0000","last_unstale":"2024-01-24T15:32:20.715604+0000","last_undegraded":"2024-01-24T15:32:20.715604+0000","last_fullsized":"2024-01-24T15:32:20.715604+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.720561+0000","last_change":"2024-01-24T15:32:20.719477+0000","last_active":"2024-01-24T15:32:20.720561+0000","last_peered":"2024-01-24T15:32:20.720561+0000","last_clean":"2024-01-24T15:32:20.720561+0000","last_became_active":"2024-01-24T15:32:20.718955+0000","last_became_peered":"2024-01-24T15:32:20.718955+0000","last_unstale":"2024-01-24T15:32:20.720561+0000","last_undegraded":"2024-01-24T15:32:20.720561+0000","last_fullsized":"2024-01-24T15:32:20.720561+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3b","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.709068+0000","last_change":"2024-01-24T15:32:20.277736+0000","last_active":"2024-01-24T15:32:20.709068+0000","last_peered":"2024-01-24T15:32:20.709068+0000","last_clean":"2024-01-24T15:32:20.709068+0000","last_became_active":"2024-01-24T15:32:20.277388+0000","last_became_peered":"2024-01-24T15:32:20.277388+0000","last_unstale":"2024-01-24T15:32:20.709068+0000","last_undegraded":"2024-01-24T15:32:20.709068+0000","last_fullsized":"2024-01-24T15:32:20.709068+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.719971+0000","last_change":"2024-01-24T15:32:20.716481+0000","last_active":"2024-01-24T15:32:20.719971+0000","last_peered":"2024-01-24T15:32:20.719971+0000","last_clean":"2024-01-24T15:32:20.719971+0000","last_became_active":"2024-01-24T15:32:20.716151+0000","last_became_peered":"2024-01-24T15:32:20.716151+0000","last_unstale":"2024-01-24T15:32:20.719971+0000","last_undegraded":"2024-01-24T15:32:20.719971+0000","last_fullsized":"2024-01-24T15:32:20.719971+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3a","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711035+0000","last_change":"2024-01-24T15:32:20.281678+0000","last_active":"2024-01-24T15:32:20.711035+0000","last_peered":"2024-01-24T15:32:20.711035+0000","last_clean":"2024-01-24T15:32:20.711035+0000","last_became_active":"2024-01-24T15:32:20.281365+0000","last_became_peered":"2024-01-24T15:32:20.281365+0000","last_unstale":"2024-01-24T15:32:20.711035+0000","last_undegraded":"2024-01-24T15:32:20.711035+0000","last_fullsized":"2024-01-24T15:32:20.711035+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.716944+0000","last_change":"2024-01-24T15:32:20.709907+0000","last_active":"2024-01-24T15:32:20.716944+0000","last_peered":"2024-01-24T15:32:20.716944+0000","last_clean":"2024-01-24T15:32:20.716944+0000","last_became_active":"2024-01-24T15:32:20.709598+0000","last_became_peered":"2024-01-24T15:32:20.709598+0000","last_unstale":"2024-01-24T15:32:20.716944+0000","last_undegraded":"2024-01-24T15:32:20.716944+0000","last_fullsized":"2024-01-24T15:32:20.716944+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689070+0000","last_change":"2024-01-24T15:32:20.277935+0000","last_active":"2024-01-24T15:32:20.689070+0000","last_peered":"2024-01-24T15:32:20.689070+0000","last_clean":"2024-01-24T15:32:20.689070+0000","last_became_active":"2024-01-24T15:32:20.277643+0000","last_became_peered":"2024-01-24T15:32:20.277643+0000","last_unstale":"2024-01-24T15:32:20.689070+0000","last_undegraded":"2024-01-24T15:32:20.689070+0000","last_fullsized":"2024-01-24T15:32:20.689070+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717085+0000","last_change":"2024-01-24T15:32:20.714495+0000","last_active":"2024-01-24T15:32:20.717085+0000","last_peered":"2024-01-24T15:32:20.717085+0000","last_clean":"2024-01-24T15:32:20.717085+0000","last_became_active":"2024-01-24T15:32:20.714180+0000","last_became_peered":"2024-01-24T15:32:20.714180+0000","last_unstale":"2024-01-24T15:32:20.717085+0000","last_undegraded":"2024-01-24T15:32:20.717085+0000","last_fullsized":"2024-01-24T15:32:20.717085+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699747+0000","last_change":"2024-01-24T15:32:20.277287+0000","last_active":"2024-01-24T15:32:20.699747+0000","last_peered":"2024-01-24T15:32:20.699747+0000","last_clean":"2024-01-24T15:32:20.699747+0000","last_became_active":"2024-01-24T15:32:20.277027+0000","last_became_peered":"2024-01-24T15:32:20.277027+0000","last_unstale":"2024-01-24T15:32:20.699747+0000","last_undegraded":"2024-01-24T15:32:20.699747+0000","last_fullsized":"2024-01-24T15:32:20.699747+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.710984+0000","last_change":"2024-01-24T15:32:20.704751+0000","last_active":"2024-01-24T15:32:20.710984+0000","last_peered":"2024-01-24T15:32:20.710984+0000","last_clean":"2024-01-24T15:32:20.710984+0000","last_became_active":"2024-01-24T15:32:20.704441+0000","last_became_peered":"2024-01-24T15:32:20.704441+0000","last_unstale":"2024-01-24T15:32:20.710984+0000","last_undegraded":"2024-01-24T15:32:20.710984+0000","last_fullsized":"2024-01-24T15:32:20.710984+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.705165+0000","last_change":"2024-01-24T15:32:20.278896+0000","last_active":"2024-01-24T15:32:20.705165+0000","last_peered":"2024-01-24T15:32:20.705165+0000","last_clean":"2024-01-24T15:32:20.705165+0000","last_became_active":"2024-01-24T15:32:20.278591+0000","last_became_peered":"2024-01-24T15:32:20.278591+0000","last_unstale":"2024-01-24T15:32:20.705165+0000","last_undegraded":"2024-01-24T15:32:20.705165+0000","last_fullsized":"2024-01-24T15:32:20.705165+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721434+0000","last_change":"2024-01-24T15:32:20.716038+0000","last_active":"2024-01-24T15:32:20.721434+0000","last_peered":"2024-01-24T15:32:20.721434+0000","last_clean":"2024-01-24T15:32:20.721434+0000","last_became_active":"2024-01-24T15:32:20.715639+0000","last_became_peered":"2024-01-24T15:32:20.715639+0000","last_unstale":"2024-01-24T15:32:20.721434+0000","last_undegraded":"2024-01-24T15:32:20.721434+0000","last_fullsized":"2024-01-24T15:32:20.721434+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.693184+0000","last_change":"2024-01-24T15:32:20.280691+0000","last_active":"2024-01-24T15:32:20.693184+0000","last_peered":"2024-01-24T15:32:20.693184+0000","last_clean":"2024-01-24T15:32:20.693184+0000","last_became_active":"2024-01-24T15:32:20.280347+0000","last_became_peered":"2024-01-24T15:32:20.280347+0000","last_unstale":"2024-01-24T15:32:20.693184+0000","last_undegraded":"2024-01-24T15:32:20.693184+0000","last_fullsized":"2024-01-24T15:32:20.693184+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717801+0000","last_change":"2024-01-24T15:32:20.714425+0000","last_active":"2024-01-24T15:32:20.717801+0000","last_peered":"2024-01-24T15:32:20.717801+0000","last_clean":"2024-01-24T15:32:20.717801+0000","last_became_active":"2024-01-24T15:32:20.714159+0000","last_became_peered":"2024-01-24T15:32:20.714159+0000","last_unstale":"2024-01-24T15:32:20.717801+0000","last_undegraded":"2024-01-24T15:32:20.717801+0000","last_fullsized":"2024-01-24T15:32:20.717801+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.707140+0000","last_change":"2024-01-24T15:32:20.276025+0000","last_active":"2024-01-24T15:32:20.707140+0000","last_peered":"2024-01-24T15:32:20.707140+0000","last_clean":"2024-01-24T15:32:20.707140+0000","last_became_active":"2024-01-24T15:32:20.275736+0000","last_became_peered":"2024-01-24T15:32:20.275736+0000","last_unstale":"2024-01-24T15:32:20.707140+0000","last_undegraded":"2024-01-24T15:32:20.707140+0000","last_fullsized":"2024-01-24T15:32:20.707140+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.705134+0000","last_change":"2024-01-24T15:32:20.699568+0000","last_active":"2024-01-24T15:32:20.705134+0000","last_peered":"2024-01-24T15:32:20.705134+0000","last_clean":"2024-01-24T15:32:20.705134+0000","last_became_active":"2024-01-24T15:32:20.698341+0000","last_became_peered":"2024-01-24T15:32:20.698341+0000","last_unstale":"2024-01-24T15:32:20.705134+0000","last_undegraded":"2024-01-24T15:32:20.705134+0000","last_fullsized":"2024-01-24T15:32:20.705134+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.34","version":"230'4","reported_seq":14,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.723557+0000","last_change":"2024-01-24T15:32:20.284601+0000","last_active":"2024-01-24T15:32:20.723557+0000","last_peered":"2024-01-24T15:32:20.723557+0000","last_clean":"2024-01-24T15:32:20.723557+0000","last_became_active":"2024-01-24T15:32:20.284292+0000","last_became_peered":"2024-01-24T15:32:20.284292+0000","last_unstale":"2024-01-24T15:32:20.723557+0000","last_undegraded":"2024-01-24T15:32:20.723557+0000","last_fullsized":"2024-01-24T15:32:20.723557+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.719688+0000","last_change":"2024-01-24T15:32:20.716461+0000","last_active":"2024-01-24T15:32:20.719688+0000","last_peered":"2024-01-24T15:32:20.719688+0000","last_clean":"2024-01-24T15:32:20.719688+0000","last_became_active":"2024-01-24T15:32:20.714787+0000","last_became_peered":"2024-01-24T15:32:20.714787+0000","last_unstale":"2024-01-24T15:32:20.719688+0000","last_undegraded":"2024-01-24T15:32:20.719688+0000","last_fullsized":"2024-01-24T15:32:20.719688+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.709661+0000","last_change":"2024-01-24T15:32:20.278404+0000","last_active":"2024-01-24T15:32:20.709661+0000","last_peered":"2024-01-24T15:32:20.709661+0000","last_clean":"2024-01-24T15:32:20.709661+0000","last_became_active":"2024-01-24T15:32:20.277732+0000","last_became_peered":"2024-01-24T15:32:20.277732+0000","last_unstale":"2024-01-24T15:32:20.709661+0000","last_undegraded":"2024-01-24T15:32:20.709661+0000","last_fullsized":"2024-01-24T15:32:20.709661+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711512+0000","last_change":"2024-01-24T15:32:20.706675+0000","last_active":"2024-01-24T15:32:20.711512+0000","last_peered":"2024-01-24T15:32:20.711512+0000","last_clean":"2024-01-24T15:32:20.711512+0000","last_became_active":"2024-01-24T15:32:20.706349+0000","last_became_peered":"2024-01-24T15:32:20.706349+0000","last_unstale":"2024-01-24T15:32:20.711512+0000","last_undegraded":"2024-01-24T15:32:20.711512+0000","last_fullsized":"2024-01-24T15:32:20.711512+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689155+0000","last_change":"2024-01-24T15:32:20.283554+0000","last_active":"2024-01-24T15:32:20.689155+0000","last_peered":"2024-01-24T15:32:20.689155+0000","last_clean":"2024-01-24T15:32:20.689155+0000","last_became_active":"2024-01-24T15:32:20.283234+0000","last_became_peered":"2024-01-24T15:32:20.283234+0000","last_unstale":"2024-01-24T15:32:20.689155+0000","last_undegraded":"2024-01-24T15:32:20.689155+0000","last_fullsized":"2024-01-24T15:32:20.689155+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711895+0000","last_change":"2024-01-24T15:32:20.706837+0000","last_active":"2024-01-24T15:32:20.711895+0000","last_peered":"2024-01-24T15:32:20.711895+0000","last_clean":"2024-01-24T15:32:20.711895+0000","last_became_active":"2024-01-24T15:32:20.706517+0000","last_became_peered":"2024-01-24T15:32:20.706517+0000","last_unstale":"2024-01-24T15:32:20.711895+0000","last_undegraded":"2024-01-24T15:32:20.711895+0000","last_fullsized":"2024-01-24T15:32:20.711895+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703639+0000","last_change":"2024-01-24T15:32:20.282841+0000","last_active":"2024-01-24T15:32:20.703639+0000","last_peered":"2024-01-24T15:32:20.703639+0000","last_clean":"2024-01-24T15:32:20.703639+0000","last_became_active":"2024-01-24T15:32:20.282029+0000","last_became_peered":"2024-01-24T15:32:20.282029+0000","last_unstale":"2024-01-24T15:32:20.703639+0000","last_undegraded":"2024-01-24T15:32:20.703639+0000","last_fullsized":"2024-01-24T15:32:20.703639+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.715887+0000","last_change":"2024-01-24T15:32:20.705283+0000","last_active":"2024-01-24T15:32:20.715887+0000","last_peered":"2024-01-24T15:32:20.715887+0000","last_clean":"2024-01-24T15:32:20.715887+0000","last_became_active":"2024-01-24T15:32:20.704976+0000","last_became_peered":"2024-01-24T15:32:20.704976+0000","last_unstale":"2024-01-24T15:32:20.715887+0000","last_undegraded":"2024-01-24T15:32:20.715887+0000","last_fullsized":"2024-01-24T15:32:20.715887+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.30","version":"230'4","reported_seq":14,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.707456+0000","last_change":"2024-01-24T15:32:20.278699+0000","last_active":"2024-01-24T15:32:20.707456+0000","last_peered":"2024-01-24T15:32:20.707456+0000","last_clean":"2024-01-24T15:32:20.707456+0000","last_became_active":"2024-01-24T15:32:20.278196+0000","last_became_peered":"2024-01-24T15:32:20.278196+0000","last_unstale":"2024-01-24T15:32:20.707456+0000","last_undegraded":"2024-01-24T15:32:20.707456+0000","last_fullsized":"2024-01-24T15:32:20.707456+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711999+0000","last_change":"2024-01-24T15:32:20.707454+0000","last_active":"2024-01-24T15:32:20.711999+0000","last_peered":"2024-01-24T15:32:20.711999+0000","last_clean":"2024-01-24T15:32:20.711999+0000","last_became_active":"2024-01-24T15:32:20.707186+0000","last_became_peered":"2024-01-24T15:32:20.707186+0000","last_unstale":"2024-01-24T15:32:20.711999+0000","last_undegraded":"2024-01-24T15:32:20.711999+0000","last_fullsized":"2024-01-24T15:32:20.711999+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.692664+0000","last_change":"2024-01-24T15:32:20.271993+0000","last_active":"2024-01-24T15:32:20.692664+0000","last_peered":"2024-01-24T15:32:20.692664+0000","last_clean":"2024-01-24T15:32:20.692664+0000","last_became_active":"2024-01-24T15:32:20.271685+0000","last_became_peered":"2024-01-24T15:32:20.271685+0000","last_unstale":"2024-01-24T15:32:20.692664+0000","last_undegraded":"2024-01-24T15:32:20.692664+0000","last_fullsized":"2024-01-24T15:32:20.692664+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.718055+0000","last_change":"2024-01-24T15:32:20.715615+0000","last_active":"2024-01-24T15:32:20.718055+0000","last_peered":"2024-01-24T15:32:20.718055+0000","last_clean":"2024-01-24T15:32:20.718055+0000","last_became_active":"2024-01-24T15:32:20.714761+0000","last_became_peered":"2024-01-24T15:32:20.714761+0000","last_unstale":"2024-01-24T15:32:20.718055+0000","last_undegraded":"2024-01-24T15:32:20.718055+0000","last_fullsized":"2024-01-24T15:32:20.718055+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.698794+0000","last_change":"2024-01-24T15:32:20.279177+0000","last_active":"2024-01-24T15:32:20.698794+0000","last_peered":"2024-01-24T15:32:20.698794+0000","last_clean":"2024-01-24T15:32:20.698794+0000","last_became_active":"2024-01-24T15:32:20.278881+0000","last_became_peered":"2024-01-24T15:32:20.278881+0000","last_unstale":"2024-01-24T15:32:20.698794+0000","last_undegraded":"2024-01-24T15:32:20.698794+0000","last_fullsized":"2024-01-24T15:32:20.698794+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.719883+0000","last_change":"2024-01-24T15:32:20.716486+0000","last_active":"2024-01-24T15:32:20.719883+0000","last_peered":"2024-01-24T15:32:20.719883+0000","last_clean":"2024-01-24T15:32:20.719883+0000","last_became_active":"2024-01-24T15:32:20.716183+0000","last_became_peered":"2024-01-24T15:32:20.716183+0000","last_unstale":"2024-01-24T15:32:20.719883+0000","last_undegraded":"2024-01-24T15:32:20.719883+0000","last_fullsized":"2024-01-24T15:32:20.719883+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2d","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.708205+0000","last_change":"2024-01-24T15:32:20.283867+0000","last_active":"2024-01-24T15:32:20.708205+0000","last_peered":"2024-01-24T15:32:20.708205+0000","last_clean":"2024-01-24T15:32:20.708205+0000","last_became_active":"2024-01-24T15:32:20.283405+0000","last_became_peered":"2024-01-24T15:32:20.283405+0000","last_unstale":"2024-01-24T15:32:20.708205+0000","last_undegraded":"2024-01-24T15:32:20.708205+0000","last_fullsized":"2024-01-24T15:32:20.708205+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.710832+0000","last_change":"2024-01-24T15:32:20.704821+0000","last_active":"2024-01-24T15:32:20.710832+0000","last_peered":"2024-01-24T15:32:20.710832+0000","last_clean":"2024-01-24T15:32:20.710832+0000","last_became_active":"2024-01-24T15:32:20.704520+0000","last_became_peered":"2024-01-24T15:32:20.704520+0000","last_unstale":"2024-01-24T15:32:20.710832+0000","last_undegraded":"2024-01-24T15:32:20.710832+0000","last_fullsized":"2024-01-24T15:32:20.710832+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.692306+0000","last_change":"2024-01-24T15:32:20.272305+0000","last_active":"2024-01-24T15:32:20.692306+0000","last_peered":"2024-01-24T15:32:20.692306+0000","last_clean":"2024-01-24T15:32:20.692306+0000","last_became_active":"2024-01-24T15:32:20.271955+0000","last_became_peered":"2024-01-24T15:32:20.271955+0000","last_unstale":"2024-01-24T15:32:20.692306+0000","last_undegraded":"2024-01-24T15:32:20.692306+0000","last_fullsized":"2024-01-24T15:32:20.692306+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.713224+0000","last_change":"2024-01-24T15:32:20.705360+0000","last_active":"2024-01-24T15:32:20.713224+0000","last_peered":"2024-01-24T15:32:20.713224+0000","last_clean":"2024-01-24T15:32:20.713224+0000","last_became_active":"2024-01-24T15:32:20.704394+0000","last_became_peered":"2024-01-24T15:32:20.704394+0000","last_unstale":"2024-01-24T15:32:20.713224+0000","last_undegraded":"2024-01-24T15:32:20.713224+0000","last_fullsized":"2024-01-24T15:32:20.713224+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2b","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714478+0000","last_change":"2024-01-24T15:32:20.279023+0000","last_active":"2024-01-24T15:32:20.714478+0000","last_peered":"2024-01-24T15:32:20.714478+0000","last_clean":"2024-01-24T15:32:20.714478+0000","last_became_active":"2024-01-24T15:32:20.278611+0000","last_became_peered":"2024-01-24T15:32:20.278611+0000","last_unstale":"2024-01-24T15:32:20.714478+0000","last_undegraded":"2024-01-24T15:32:20.714478+0000","last_fullsized":"2024-01-24T15:32:20.714478+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714381+0000","last_change":"2024-01-24T15:32:20.712354+0000","last_active":"2024-01-24T15:32:20.714381+0000","last_peered":"2024-01-24T15:32:20.714381+0000","last_clean":"2024-01-24T15:32:20.714381+0000","last_became_active":"2024-01-24T15:32:20.712116+0000","last_became_peered":"2024-01-24T15:32:20.712116+0000","last_unstale":"2024-01-24T15:32:20.714381+0000","last_undegraded":"2024-01-24T15:32:20.714381+0000","last_fullsized":"2024-01-24T15:32:20.714381+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.704665+0000","last_change":"2024-01-24T15:32:20.281147+0000","last_active":"2024-01-24T15:32:20.704665+0000","last_peered":"2024-01-24T15:32:20.704665+0000","last_clean":"2024-01-24T15:32:20.704665+0000","last_became_active":"2024-01-24T15:32:20.280772+0000","last_became_peered":"2024-01-24T15:32:20.280772+0000","last_unstale":"2024-01-24T15:32:20.704665+0000","last_undegraded":"2024-01-24T15:32:20.704665+0000","last_fullsized":"2024-01-24T15:32:20.704665+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714804+0000","last_change":"2024-01-24T15:32:20.711895+0000","last_active":"2024-01-24T15:32:20.714804+0000","last_peered":"2024-01-24T15:32:20.714804+0000","last_clean":"2024-01-24T15:32:20.714804+0000","last_became_active":"2024-01-24T15:32:20.711612+0000","last_became_peered":"2024-01-24T15:32:20.711612+0000","last_unstale":"2024-01-24T15:32:20.714804+0000","last_undegraded":"2024-01-24T15:32:20.714804+0000","last_fullsized":"2024-01-24T15:32:20.714804+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.698649+0000","last_change":"2024-01-24T15:32:20.284966+0000","last_active":"2024-01-24T15:32:20.698649+0000","last_peered":"2024-01-24T15:32:20.698649+0000","last_clean":"2024-01-24T15:32:20.698649+0000","last_became_active":"2024-01-24T15:32:20.284662+0000","last_became_peered":"2024-01-24T15:32:20.284662+0000","last_unstale":"2024-01-24T15:32:20.698649+0000","last_undegraded":"2024-01-24T15:32:20.698649+0000","last_fullsized":"2024-01-24T15:32:20.698649+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721848+0000","last_change":"2024-01-24T15:32:20.717940+0000","last_active":"2024-01-24T15:32:20.721848+0000","last_peered":"2024-01-24T15:32:20.721848+0000","last_clean":"2024-01-24T15:32:20.721848+0000","last_became_active":"2024-01-24T15:32:20.717078+0000","last_became_peered":"2024-01-24T15:32:20.717078+0000","last_unstale":"2024-01-24T15:32:20.721848+0000","last_undegraded":"2024-01-24T15:32:20.721848+0000","last_fullsized":"2024-01-24T15:32:20.721848+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703319+0000","last_change":"2024-01-24T15:32:20.276295+0000","last_active":"2024-01-24T15:32:20.703319+0000","last_peered":"2024-01-24T15:32:20.703319+0000","last_clean":"2024-01-24T15:32:20.703319+0000","last_became_active":"2024-01-24T15:32:20.275936+0000","last_became_peered":"2024-01-24T15:32:20.275936+0000","last_unstale":"2024-01-24T15:32:20.703319+0000","last_undegraded":"2024-01-24T15:32:20.703319+0000","last_fullsized":"2024-01-24T15:32:20.703319+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.718102+0000","last_change":"2024-01-24T15:32:20.714991+0000","last_active":"2024-01-24T15:32:20.718102+0000","last_peered":"2024-01-24T15:32:20.718102+0000","last_clean":"2024-01-24T15:32:20.718102+0000","last_became_active":"2024-01-24T15:32:20.714659+0000","last_became_peered":"2024-01-24T15:32:20.714659+0000","last_unstale":"2024-01-24T15:32:20.718102+0000","last_undegraded":"2024-01-24T15:32:20.718102+0000","last_fullsized":"2024-01-24T15:32:20.718102+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689450+0000","last_change":"2024-01-24T15:32:20.283489+0000","last_active":"2024-01-24T15:32:20.689450+0000","last_peered":"2024-01-24T15:32:20.689450+0000","last_clean":"2024-01-24T15:32:20.689450+0000","last_became_active":"2024-01-24T15:32:20.283183+0000","last_became_peered":"2024-01-24T15:32:20.283183+0000","last_unstale":"2024-01-24T15:32:20.689450+0000","last_undegraded":"2024-01-24T15:32:20.689450+0000","last_fullsized":"2024-01-24T15:32:20.689450+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.720219+0000","last_change":"2024-01-24T15:32:20.713009+0000","last_active":"2024-01-24T15:32:20.720219+0000","last_peered":"2024-01-24T15:32:20.720219+0000","last_clean":"2024-01-24T15:32:20.720219+0000","last_became_active":"2024-01-24T15:32:20.712705+0000","last_became_peered":"2024-01-24T15:32:20.712705+0000","last_unstale":"2024-01-24T15:32:20.720219+0000","last_undegraded":"2024-01-24T15:32:20.720219+0000","last_fullsized":"2024-01-24T15:32:20.720219+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.709683+0000","last_change":"2024-01-24T15:32:20.282888+0000","last_active":"2024-01-24T15:32:20.709683+0000","last_peered":"2024-01-24T15:32:20.709683+0000","last_clean":"2024-01-24T15:32:20.709683+0000","last_became_active":"2024-01-24T15:32:20.282421+0000","last_became_peered":"2024-01-24T15:32:20.282421+0000","last_unstale":"2024-01-24T15:32:20.709683+0000","last_undegraded":"2024-01-24T15:32:20.709683+0000","last_fullsized":"2024-01-24T15:32:20.709683+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721250+0000","last_change":"2024-01-24T15:32:20.712004+0000","last_active":"2024-01-24T15:32:20.721250+0000","last_peered":"2024-01-24T15:32:20.721250+0000","last_clean":"2024-01-24T15:32:20.721250+0000","last_became_active":"2024-01-24T15:32:20.711681+0000","last_became_peered":"2024-01-24T15:32:20.711681+0000","last_unstale":"2024-01-24T15:32:20.721250+0000","last_undegraded":"2024-01-24T15:32:20.721250+0000","last_fullsized":"2024-01-24T15:32:20.721250+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699109+0000","last_change":"2024-01-24T15:32:20.275343+0000","last_active":"2024-01-24T15:32:20.699109+0000","last_peered":"2024-01-24T15:32:20.699109+0000","last_clean":"2024-01-24T15:32:20.699109+0000","last_became_active":"2024-01-24T15:32:20.275046+0000","last_became_peered":"2024-01-24T15:32:20.275046+0000","last_unstale":"2024-01-24T15:32:20.699109+0000","last_undegraded":"2024-01-24T15:32:20.699109+0000","last_fullsized":"2024-01-24T15:32:20.699109+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.715026+0000","last_change":"2024-01-24T15:32:20.711194+0000","last_active":"2024-01-24T15:32:20.715026+0000","last_peered":"2024-01-24T15:32:20.715026+0000","last_clean":"2024-01-24T15:32:20.715026+0000","last_became_active":"2024-01-24T15:32:20.710933+0000","last_became_peered":"2024-01-24T15:32:20.710933+0000","last_unstale":"2024-01-24T15:32:20.715026+0000","last_undegraded":"2024-01-24T15:32:20.715026+0000","last_fullsized":"2024-01-24T15:32:20.715026+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.692466+0000","last_change":"2024-01-24T15:32:20.276754+0000","last_active":"2024-01-24T15:32:20.692466+0000","last_peered":"2024-01-24T15:32:20.692466+0000","last_clean":"2024-01-24T15:32:20.692466+0000","last_became_active":"2024-01-24T15:32:20.276459+0000","last_became_peered":"2024-01-24T15:32:20.276459+0000","last_unstale":"2024-01-24T15:32:20.692466+0000","last_undegraded":"2024-01-24T15:32:20.692466+0000","last_fullsized":"2024-01-24T15:32:20.692466+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721747+0000","last_change":"2024-01-24T15:32:20.720881+0000","last_active":"2024-01-24T15:32:20.721747+0000","last_peered":"2024-01-24T15:32:20.721747+0000","last_clean":"2024-01-24T15:32:20.721747+0000","last_became_active":"2024-01-24T15:32:20.720537+0000","last_became_peered":"2024-01-24T15:32:20.720537+0000","last_unstale":"2024-01-24T15:32:20.721747+0000","last_undegraded":"2024-01-24T15:32:20.721747+0000","last_fullsized":"2024-01-24T15:32:20.721747+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.705510+0000","last_change":"2024-01-24T15:32:20.280328+0000","last_active":"2024-01-24T15:32:20.705510+0000","last_peered":"2024-01-24T15:32:20.705510+0000","last_clean":"2024-01-24T15:32:20.705510+0000","last_became_active":"2024-01-24T15:32:20.280071+0000","last_became_peered":"2024-01-24T15:32:20.280071+0000","last_unstale":"2024-01-24T15:32:20.705510+0000","last_undegraded":"2024-01-24T15:32:20.705510+0000","last_fullsized":"2024-01-24T15:32:20.705510+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.715071+0000","last_change":"2024-01-24T15:32:20.711414+0000","last_active":"2024-01-24T15:32:20.715071+0000","last_peered":"2024-01-24T15:32:20.715071+0000","last_clean":"2024-01-24T15:32:20.715071+0000","last_became_active":"2024-01-24T15:32:20.710834+0000","last_became_peered":"2024-01-24T15:32:20.710834+0000","last_unstale":"2024-01-24T15:32:20.715071+0000","last_undegraded":"2024-01-24T15:32:20.715071+0000","last_fullsized":"2024-01-24T15:32:20.715071+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.705432+0000","last_change":"2024-01-24T15:32:20.289152+0000","last_active":"2024-01-24T15:32:20.705432+0000","last_peered":"2024-01-24T15:32:20.705432+0000","last_clean":"2024-01-24T15:32:20.705432+0000","last_became_active":"2024-01-24T15:32:20.288964+0000","last_became_peered":"2024-01-24T15:32:20.288964+0000","last_unstale":"2024-01-24T15:32:20.705432+0000","last_undegraded":"2024-01-24T15:32:20.705432+0000","last_fullsized":"2024-01-24T15:32:20.705432+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.716561+0000","last_change":"2024-01-24T15:32:20.712729+0000","last_active":"2024-01-24T15:32:20.716561+0000","last_peered":"2024-01-24T15:32:20.716561+0000","last_clean":"2024-01-24T15:32:20.716561+0000","last_became_active":"2024-01-24T15:32:20.712428+0000","last_became_peered":"2024-01-24T15:32:20.712428+0000","last_unstale":"2024-01-24T15:32:20.716561+0000","last_undegraded":"2024-01-24T15:32:20.716561+0000","last_fullsized":"2024-01-24T15:32:20.716561+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689133+0000","last_change":"2024-01-24T15:32:20.285176+0000","last_active":"2024-01-24T15:32:20.689133+0000","last_peered":"2024-01-24T15:32:20.689133+0000","last_clean":"2024-01-24T15:32:20.689133+0000","last_became_active":"2024-01-24T15:32:20.284865+0000","last_became_peered":"2024-01-24T15:32:20.284865+0000","last_unstale":"2024-01-24T15:32:20.689133+0000","last_undegraded":"2024-01-24T15:32:20.689133+0000","last_fullsized":"2024-01-24T15:32:20.689133+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711957+0000","last_change":"2024-01-24T15:32:20.707547+0000","last_active":"2024-01-24T15:32:20.711957+0000","last_peered":"2024-01-24T15:32:20.711957+0000","last_clean":"2024-01-24T15:32:20.711957+0000","last_became_active":"2024-01-24T15:32:20.707227+0000","last_became_peered":"2024-01-24T15:32:20.707227+0000","last_unstale":"2024-01-24T15:32:20.711957+0000","last_undegraded":"2024-01-24T15:32:20.711957+0000","last_fullsized":"2024-01-24T15:32:20.711957+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.693316+0000","last_change":"2024-01-24T15:32:20.279162+0000","last_active":"2024-01-24T15:32:20.693316+0000","last_peered":"2024-01-24T15:32:20.693316+0000","last_clean":"2024-01-24T15:32:20.693316+0000","last_became_active":"2024-01-24T15:32:20.278862+0000","last_became_peered":"2024-01-24T15:32:20.278862+0000","last_unstale":"2024-01-24T15:32:20.693316+0000","last_undegraded":"2024-01-24T15:32:20.693316+0000","last_fullsized":"2024-01-24T15:32:20.693316+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.720836+0000","last_change":"2024-01-24T15:32:20.714570+0000","last_active":"2024-01-24T15:32:20.720836+0000","last_peered":"2024-01-24T15:32:20.720836+0000","last_clean":"2024-01-24T15:32:20.720836+0000","last_became_active":"2024-01-24T15:32:20.714285+0000","last_became_peered":"2024-01-24T15:32:20.714285+0000","last_unstale":"2024-01-24T15:32:20.720836+0000","last_undegraded":"2024-01-24T15:32:20.720836+0000","last_fullsized":"2024-01-24T15:32:20.720836+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.705266+0000","last_change":"2024-01-24T15:32:20.282984+0000","last_active":"2024-01-24T15:32:20.705266+0000","last_peered":"2024-01-24T15:32:20.705266+0000","last_clean":"2024-01-24T15:32:20.705266+0000","last_became_active":"2024-01-24T15:32:20.282644+0000","last_became_peered":"2024-01-24T15:32:20.282644+0000","last_unstale":"2024-01-24T15:32:20.705266+0000","last_undegraded":"2024-01-24T15:32:20.705266+0000","last_fullsized":"2024-01-24T15:32:20.705266+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717988+0000","last_change":"2024-01-24T15:32:20.708731+0000","last_active":"2024-01-24T15:32:20.717988+0000","last_peered":"2024-01-24T15:32:20.717988+0000","last_clean":"2024-01-24T15:32:20.717988+0000","last_became_active":"2024-01-24T15:32:20.708433+0000","last_became_peered":"2024-01-24T15:32:20.708433+0000","last_unstale":"2024-01-24T15:32:20.717988+0000","last_undegraded":"2024-01-24T15:32:20.717988+0000","last_fullsized":"2024-01-24T15:32:20.717988+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.698898+0000","last_change":"2024-01-24T15:32:20.276650+0000","last_active":"2024-01-24T15:32:20.698898+0000","last_peered":"2024-01-24T15:32:20.698898+0000","last_clean":"2024-01-24T15:32:20.698898+0000","last_became_active":"2024-01-24T15:32:20.276437+0000","last_became_peered":"2024-01-24T15:32:20.276437+0000","last_unstale":"2024-01-24T15:32:20.698898+0000","last_undegraded":"2024-01-24T15:32:20.698898+0000","last_fullsized":"2024-01-24T15:32:20.698898+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714313+0000","last_change":"2024-01-24T15:32:20.712042+0000","last_active":"2024-01-24T15:32:20.714313+0000","last_peered":"2024-01-24T15:32:20.714313+0000","last_clean":"2024-01-24T15:32:20.714313+0000","last_became_active":"2024-01-24T15:32:20.707486+0000","last_became_peered":"2024-01-24T15:32:20.707486+0000","last_unstale":"2024-01-24T15:32:20.714313+0000","last_undegraded":"2024-01-24T15:32:20.714313+0000","last_fullsized":"2024-01-24T15:32:20.714313+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"230'3","reported_seq":13,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.719452+0000","last_change":"2024-01-24T15:32:20.289013+0000","last_active":"2024-01-24T15:32:20.719452+0000","last_peered":"2024-01-24T15:32:20.719452+0000","last_clean":"2024-01-24T15:32:20.719452+0000","last_became_active":"2024-01-24T15:32:20.288764+0000","last_became_peered":"2024-01-24T15:32:20.288764+0000","last_unstale":"2024-01-24T15:32:20.719452+0000","last_undegraded":"2024-01-24T15:32:20.719452+0000","last_fullsized":"2024-01-24T15:32:20.719452+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.716822+0000","last_change":"2024-01-24T15:32:20.705865+0000","last_active":"2024-01-24T15:32:20.716822+0000","last_peered":"2024-01-24T15:32:20.716822+0000","last_clean":"2024-01-24T15:32:20.716822+0000","last_became_active":"2024-01-24T15:32:20.705170+0000","last_became_peered":"2024-01-24T15:32:20.705170+0000","last_unstale":"2024-01-24T15:32:20.716822+0000","last_undegraded":"2024-01-24T15:32:20.716822+0000","last_fullsized":"2024-01-24T15:32:20.716822+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"230'2","reported_seq":12,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.713900+0000","last_change":"2024-01-24T15:32:20.282155+0000","last_active":"2024-01-24T15:32:20.713900+0000","last_peered":"2024-01-24T15:32:20.713900+0000","last_clean":"2024-01-24T15:32:20.713900+0000","last_became_active":"2024-01-24T15:32:20.281877+0000","last_became_peered":"2024-01-24T15:32:20.281877+0000","last_unstale":"2024-01-24T15:32:20.713900+0000","last_undegraded":"2024-01-24T15:32:20.713900+0000","last_fullsized":"2024-01-24T15:32:20.713900+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.720540+0000","last_change":"2024-01-24T15:32:20.719293+0000","last_active":"2024-01-24T15:32:20.720540+0000","last_peered":"2024-01-24T15:32:20.720540+0000","last_clean":"2024-01-24T15:32:20.720540+0000","last_became_active":"2024-01-24T15:32:20.718598+0000","last_became_peered":"2024-01-24T15:32:20.718598+0000","last_unstale":"2024-01-24T15:32:20.720540+0000","last_undegraded":"2024-01-24T15:32:20.720540+0000","last_fullsized":"2024-01-24T15:32:20.720540+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.709707+0000","last_change":"2024-01-24T15:32:20.278608+0000","last_active":"2024-01-24T15:32:20.709707+0000","last_peered":"2024-01-24T15:32:20.709707+0000","last_clean":"2024-01-24T15:32:20.709707+0000","last_became_active":"2024-01-24T15:32:20.278124+0000","last_became_peered":"2024-01-24T15:32:20.278124+0000","last_unstale":"2024-01-24T15:32:20.709707+0000","last_undegraded":"2024-01-24T15:32:20.709707+0000","last_fullsized":"2024-01-24T15:32:20.709707+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717675+0000","last_change":"2024-01-24T15:32:20.713089+0000","last_active":"2024-01-24T15:32:20.717675+0000","last_peered":"2024-01-24T15:32:20.717675+0000","last_clean":"2024-01-24T15:32:20.717675+0000","last_became_active":"2024-01-24T15:32:20.712818+0000","last_became_peered":"2024-01-24T15:32:20.712818+0000","last_unstale":"2024-01-24T15:32:20.717675+0000","last_undegraded":"2024-01-24T15:32:20.717675+0000","last_fullsized":"2024-01-24T15:32:20.717675+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.697089+0000","last_change":"2024-01-24T15:32:20.287919+0000","last_active":"2024-01-24T15:32:20.697089+0000","last_peered":"2024-01-24T15:32:20.697089+0000","last_clean":"2024-01-24T15:32:20.697089+0000","last_became_active":"2024-01-24T15:32:20.287555+0000","last_became_peered":"2024-01-24T15:32:20.287555+0000","last_unstale":"2024-01-24T15:32:20.697089+0000","last_undegraded":"2024-01-24T15:32:20.697089+0000","last_fullsized":"2024-01-24T15:32:20.697089+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.718063+0000","last_change":"2024-01-24T15:32:20.708923+0000","last_active":"2024-01-24T15:32:20.718063+0000","last_peered":"2024-01-24T15:32:20.718063+0000","last_clean":"2024-01-24T15:32:20.718063+0000","last_became_active":"2024-01-24T15:32:20.708439+0000","last_became_peered":"2024-01-24T15:32:20.708439+0000","last_unstale":"2024-01-24T15:32:20.718063+0000","last_undegraded":"2024-01-24T15:32:20.718063+0000","last_fullsized":"2024-01-24T15:32:20.718063+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.692569+0000","last_change":"2024-01-24T15:32:20.276616+0000","last_active":"2024-01-24T15:32:20.692569+0000","last_peered":"2024-01-24T15:32:20.692569+0000","last_clean":"2024-01-24T15:32:20.692569+0000","last_became_active":"2024-01-24T15:32:20.276312+0000","last_became_peered":"2024-01-24T15:32:20.276312+0000","last_unstale":"2024-01-24T15:32:20.692569+0000","last_undegraded":"2024-01-24T15:32:20.692569+0000","last_fullsized":"2024-01-24T15:32:20.692569+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717063+0000","last_change":"2024-01-24T15:32:20.713385+0000","last_active":"2024-01-24T15:32:20.717063+0000","last_peered":"2024-01-24T15:32:20.717063+0000","last_clean":"2024-01-24T15:32:20.717063+0000","last_became_active":"2024-01-24T15:32:20.713106+0000","last_became_peered":"2024-01-24T15:32:20.713106+0000","last_unstale":"2024-01-24T15:32:20.717063+0000","last_undegraded":"2024-01-24T15:32:20.717063+0000","last_fullsized":"2024-01-24T15:32:20.717063+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"230'4","reported_seq":14,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.712431+0000","last_change":"2024-01-24T15:32:20.277127+0000","last_active":"2024-01-24T15:32:20.712431+0000","last_peered":"2024-01-24T15:32:20.712431+0000","last_clean":"2024-01-24T15:32:20.712431+0000","last_became_active":"2024-01-24T15:32:20.275139+0000","last_became_peered":"2024-01-24T15:32:20.275139+0000","last_unstale":"2024-01-24T15:32:20.712431+0000","last_undegraded":"2024-01-24T15:32:20.712431+0000","last_fullsized":"2024-01-24T15:32:20.712431+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721595+0000","last_change":"2024-01-24T15:32:20.718169+0000","last_active":"2024-01-24T15:32:20.721595+0000","last_peered":"2024-01-24T15:32:20.721595+0000","last_clean":"2024-01-24T15:32:20.721595+0000","last_became_active":"2024-01-24T15:32:20.716914+0000","last_became_peered":"2024-01-24T15:32:20.716914+0000","last_unstale":"2024-01-24T15:32:20.721595+0000","last_undegraded":"2024-01-24T15:32:20.721595+0000","last_fullsized":"2024-01-24T15:32:20.721595+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717108+0000","last_change":"2024-01-24T15:32:20.277339+0000","last_active":"2024-01-24T15:32:20.717108+0000","last_peered":"2024-01-24T15:32:20.717108+0000","last_clean":"2024-01-24T15:32:20.717108+0000","last_became_active":"2024-01-24T15:32:20.277018+0000","last_became_peered":"2024-01-24T15:32:20.277018+0000","last_unstale":"2024-01-24T15:32:20.717108+0000","last_undegraded":"2024-01-24T15:32:20.717108+0000","last_fullsized":"2024-01-24T15:32:20.717108+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721582+0000","last_change":"2024-01-24T15:32:20.716601+0000","last_active":"2024-01-24T15:32:20.721582+0000","last_peered":"2024-01-24T15:32:20.721582+0000","last_clean":"2024-01-24T15:32:20.721582+0000","last_became_active":"2024-01-24T15:32:20.716311+0000","last_became_peered":"2024-01-24T15:32:20.716311+0000","last_unstale":"2024-01-24T15:32:20.721582+0000","last_undegraded":"2024-01-24T15:32:20.721582+0000","last_fullsized":"2024-01-24T15:32:20.721582+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"230'2","reported_seq":12,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721533+0000","last_change":"2024-01-24T15:32:20.282723+0000","last_active":"2024-01-24T15:32:20.721533+0000","last_peered":"2024-01-24T15:32:20.721533+0000","last_clean":"2024-01-24T15:32:20.721533+0000","last_became_active":"2024-01-24T15:32:20.282380+0000","last_became_peered":"2024-01-24T15:32:20.282380+0000","last_unstale":"2024-01-24T15:32:20.721533+0000","last_undegraded":"2024-01-24T15:32:20.721533+0000","last_fullsized":"2024-01-24T15:32:20.721533+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721274+0000","last_change":"2024-01-24T15:32:20.712626+0000","last_active":"2024-01-24T15:32:20.721274+0000","last_peered":"2024-01-24T15:32:20.721274+0000","last_clean":"2024-01-24T15:32:20.721274+0000","last_became_active":"2024-01-24T15:32:20.712245+0000","last_became_peered":"2024-01-24T15:32:20.712245+0000","last_unstale":"2024-01-24T15:32:20.721274+0000","last_undegraded":"2024-01-24T15:32:20.721274+0000","last_fullsized":"2024-01-24T15:32:20.721274+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.694279+0000","last_change":"2024-01-24T15:32:20.279875+0000","last_active":"2024-01-24T15:32:20.694279+0000","last_peered":"2024-01-24T15:32:20.694279+0000","last_clean":"2024-01-24T15:32:20.694279+0000","last_became_active":"2024-01-24T15:32:20.279568+0000","last_became_peered":"2024-01-24T15:32:20.279568+0000","last_unstale":"2024-01-24T15:32:20.694279+0000","last_undegraded":"2024-01-24T15:32:20.694279+0000","last_fullsized":"2024-01-24T15:32:20.694279+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711798+0000","last_change":"2024-01-24T15:32:20.707837+0000","last_active":"2024-01-24T15:32:20.711798+0000","last_peered":"2024-01-24T15:32:20.711798+0000","last_clean":"2024-01-24T15:32:20.711798+0000","last_became_active":"2024-01-24T15:32:20.707522+0000","last_became_peered":"2024-01-24T15:32:20.707522+0000","last_unstale":"2024-01-24T15:32:20.711798+0000","last_undegraded":"2024-01-24T15:32:20.711798+0000","last_fullsized":"2024-01-24T15:32:20.711798+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689619+0000","last_change":"2024-01-24T15:32:20.285248+0000","last_active":"2024-01-24T15:32:20.689619+0000","last_peered":"2024-01-24T15:32:20.689619+0000","last_clean":"2024-01-24T15:32:20.689619+0000","last_became_active":"2024-01-24T15:32:20.284886+0000","last_became_peered":"2024-01-24T15:32:20.284886+0000","last_unstale":"2024-01-24T15:32:20.689619+0000","last_undegraded":"2024-01-24T15:32:20.689619+0000","last_fullsized":"2024-01-24T15:32:20.689619+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714275+0000","last_change":"2024-01-24T15:32:20.711880+0000","last_active":"2024-01-24T15:32:20.714275+0000","last_peered":"2024-01-24T15:32:20.714275+0000","last_clean":"2024-01-24T15:32:20.714275+0000","last_became_active":"2024-01-24T15:32:20.711632+0000","last_became_peered":"2024-01-24T15:32:20.711632+0000","last_unstale":"2024-01-24T15:32:20.714275+0000","last_undegraded":"2024-01-24T15:32:20.714275+0000","last_fullsized":"2024-01-24T15:32:20.714275+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.709216+0000","last_change":"2024-01-24T15:32:20.273923+0000","last_active":"2024-01-24T15:32:20.709216+0000","last_peered":"2024-01-24T15:32:20.709216+0000","last_clean":"2024-01-24T15:32:20.709216+0000","last_became_active":"2024-01-24T15:32:20.273725+0000","last_became_peered":"2024-01-24T15:32:20.273725+0000","last_unstale":"2024-01-24T15:32:20.709216+0000","last_undegraded":"2024-01-24T15:32:20.709216+0000","last_fullsized":"2024-01-24T15:32:20.709216+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721234+0000","last_change":"2024-01-24T15:32:20.710548+0000","last_active":"2024-01-24T15:32:20.721234+0000","last_peered":"2024-01-24T15:32:20.721234+0000","last_clean":"2024-01-24T15:32:20.721234+0000","last_became_active":"2024-01-24T15:32:20.710252+0000","last_became_peered":"2024-01-24T15:32:20.710252+0000","last_unstale":"2024-01-24T15:32:20.721234+0000","last_undegraded":"2024-01-24T15:32:20.721234+0000","last_fullsized":"2024-01-24T15:32:20.721234+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.696722+0000","last_change":"2024-01-24T15:32:20.286609+0000","last_active":"2024-01-24T15:32:20.696722+0000","last_peered":"2024-01-24T15:32:20.696722+0000","last_clean":"2024-01-24T15:32:20.696722+0000","last_became_active":"2024-01-24T15:32:20.286254+0000","last_became_peered":"2024-01-24T15:32:20.286254+0000","last_unstale":"2024-01-24T15:32:20.696722+0000","last_undegraded":"2024-01-24T15:32:20.696722+0000","last_fullsized":"2024-01-24T15:32:20.696722+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721642+0000","last_change":"2024-01-24T15:32:20.718322+0000","last_active":"2024-01-24T15:32:20.721642+0000","last_peered":"2024-01-24T15:32:20.721642+0000","last_clean":"2024-01-24T15:32:20.721642+0000","last_became_active":"2024-01-24T15:32:20.716991+0000","last_became_peered":"2024-01-24T15:32:20.716991+0000","last_unstale":"2024-01-24T15:32:20.721642+0000","last_undegraded":"2024-01-24T15:32:20.721642+0000","last_fullsized":"2024-01-24T15:32:20.721642+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.695960+0000","last_change":"2024-01-24T15:32:20.287451+0000","last_active":"2024-01-24T15:32:20.695960+0000","last_peered":"2024-01-24T15:32:20.695960+0000","last_clean":"2024-01-24T15:32:20.695960+0000","last_became_active":"2024-01-24T15:32:20.287139+0000","last_became_peered":"2024-01-24T15:32:20.287139+0000","last_unstale":"2024-01-24T15:32:20.695960+0000","last_undegraded":"2024-01-24T15:32:20.695960+0000","last_fullsized":"2024-01-24T15:32:20.695960+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714185+0000","last_change":"2024-01-24T15:32:20.709005+0000","last_active":"2024-01-24T15:32:20.714185+0000","last_peered":"2024-01-24T15:32:20.714185+0000","last_clean":"2024-01-24T15:32:20.714185+0000","last_became_active":"2024-01-24T15:32:20.708732+0000","last_became_peered":"2024-01-24T15:32:20.708732+0000","last_unstale":"2024-01-24T15:32:20.714185+0000","last_undegraded":"2024-01-24T15:32:20.714185+0000","last_fullsized":"2024-01-24T15:32:20.714185+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703453+0000","last_change":"2024-01-24T15:32:20.274724+0000","last_active":"2024-01-24T15:32:20.703453+0000","last_peered":"2024-01-24T15:32:20.703453+0000","last_clean":"2024-01-24T15:32:20.703453+0000","last_became_active":"2024-01-24T15:32:20.274398+0000","last_became_peered":"2024-01-24T15:32:20.274398+0000","last_unstale":"2024-01-24T15:32:20.703453+0000","last_undegraded":"2024-01-24T15:32:20.703453+0000","last_fullsized":"2024-01-24T15:32:20.703453+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.718113+0000","last_change":"2024-01-24T15:32:20.714659+0000","last_active":"2024-01-24T15:32:20.718113+0000","last_peered":"2024-01-24T15:32:20.718113+0000","last_clean":"2024-01-24T15:32:20.718113+0000","last_became_active":"2024-01-24T15:32:20.714372+0000","last_became_peered":"2024-01-24T15:32:20.714372+0000","last_unstale":"2024-01-24T15:32:20.718113+0000","last_undegraded":"2024-01-24T15:32:20.718113+0000","last_fullsized":"2024-01-24T15:32:20.718113+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"230'2","reported_seq":12,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.709086+0000","last_change":"2024-01-24T15:32:20.280405+0000","last_active":"2024-01-24T15:32:20.709086+0000","last_peered":"2024-01-24T15:32:20.709086+0000","last_clean":"2024-01-24T15:32:20.709086+0000","last_became_active":"2024-01-24T15:32:20.280088+0000","last_became_peered":"2024-01-24T15:32:20.280088+0000","last_unstale":"2024-01-24T15:32:20.709086+0000","last_undegraded":"2024-01-24T15:32:20.709086+0000","last_fullsized":"2024-01-24T15:32:20.709086+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.719681+0000","last_change":"2024-01-24T15:32:20.716121+0000","last_active":"2024-01-24T15:32:20.719681+0000","last_peered":"2024-01-24T15:32:20.719681+0000","last_clean":"2024-01-24T15:32:20.719681+0000","last_became_active":"2024-01-24T15:32:20.714444+0000","last_became_peered":"2024-01-24T15:32:20.714444+0000","last_unstale":"2024-01-24T15:32:20.719681+0000","last_undegraded":"2024-01-24T15:32:20.719681+0000","last_fullsized":"2024-01-24T15:32:20.719681+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699223+0000","last_change":"2024-01-24T15:32:20.276747+0000","last_active":"2024-01-24T15:32:20.699223+0000","last_peered":"2024-01-24T15:32:20.699223+0000","last_clean":"2024-01-24T15:32:20.699223+0000","last_became_active":"2024-01-24T15:32:20.276453+0000","last_became_peered":"2024-01-24T15:32:20.276453+0000","last_unstale":"2024-01-24T15:32:20.699223+0000","last_undegraded":"2024-01-24T15:32:20.699223+0000","last_fullsized":"2024-01-24T15:32:20.699223+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721667+0000","last_change":"2024-01-24T15:32:20.718687+0000","last_active":"2024-01-24T15:32:20.721667+0000","last_peered":"2024-01-24T15:32:20.721667+0000","last_clean":"2024-01-24T15:32:20.721667+0000","last_became_active":"2024-01-24T15:32:20.718208+0000","last_became_peered":"2024-01-24T15:32:20.718208+0000","last_unstale":"2024-01-24T15:32:20.721667+0000","last_undegraded":"2024-01-24T15:32:20.721667+0000","last_fullsized":"2024-01-24T15:32:20.721667+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703742+0000","last_change":"2024-01-24T15:32:20.287469+0000","last_active":"2024-01-24T15:32:20.703742+0000","last_peered":"2024-01-24T15:32:20.703742+0000","last_clean":"2024-01-24T15:32:20.703742+0000","last_became_active":"2024-01-24T15:32:20.287145+0000","last_became_peered":"2024-01-24T15:32:20.287145+0000","last_unstale":"2024-01-24T15:32:20.703742+0000","last_undegraded":"2024-01-24T15:32:20.703742+0000","last_fullsized":"2024-01-24T15:32:20.703742+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714010+0000","last_change":"2024-01-24T15:32:20.707884+0000","last_active":"2024-01-24T15:32:20.714010+0000","last_peered":"2024-01-24T15:32:20.714010+0000","last_clean":"2024-01-24T15:32:20.714010+0000","last_became_active":"2024-01-24T15:32:20.706589+0000","last_became_peered":"2024-01-24T15:32:20.706589+0000","last_unstale":"2024-01-24T15:32:20.714010+0000","last_undegraded":"2024-01-24T15:32:20.714010+0000","last_fullsized":"2024-01-24T15:32:20.714010+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689896+0000","last_change":"2024-01-24T15:32:20.280860+0000","last_active":"2024-01-24T15:32:20.689896+0000","last_peered":"2024-01-24T15:32:20.689896+0000","last_clean":"2024-01-24T15:32:20.689896+0000","last_became_active":"2024-01-24T15:32:20.280544+0000","last_became_peered":"2024-01-24T15:32:20.280544+0000","last_unstale":"2024-01-24T15:32:20.689896+0000","last_undegraded":"2024-01-24T15:32:20.689896+0000","last_fullsized":"2024-01-24T15:32:20.689896+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721488+0000","last_change":"2024-01-24T15:32:20.719006+0000","last_active":"2024-01-24T15:32:20.721488+0000","last_peered":"2024-01-24T15:32:20.721488+0000","last_clean":"2024-01-24T15:32:20.721488+0000","last_became_active":"2024-01-24T15:32:20.716857+0000","last_became_peered":"2024-01-24T15:32:20.716857+0000","last_unstale":"2024-01-24T15:32:20.721488+0000","last_undegraded":"2024-01-24T15:32:20.721488+0000","last_fullsized":"2024-01-24T15:32:20.721488+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699341+0000","last_change":"2024-01-24T15:32:20.284768+0000","last_active":"2024-01-24T15:32:20.699341+0000","last_peered":"2024-01-24T15:32:20.699341+0000","last_clean":"2024-01-24T15:32:20.699341+0000","last_became_active":"2024-01-24T15:32:20.284447+0000","last_became_peered":"2024-01-24T15:32:20.284447+0000","last_unstale":"2024-01-24T15:32:20.699341+0000","last_undegraded":"2024-01-24T15:32:20.699341+0000","last_fullsized":"2024-01-24T15:32:20.699341+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.706537+0000","last_change":"2024-01-24T15:32:20.701039+0000","last_active":"2024-01-24T15:32:20.706537+0000","last_peered":"2024-01-24T15:32:20.706537+0000","last_clean":"2024-01-24T15:32:20.706537+0000","last_became_active":"2024-01-24T15:32:20.700696+0000","last_became_peered":"2024-01-24T15:32:20.700696+0000","last_unstale":"2024-01-24T15:32:20.706537+0000","last_undegraded":"2024-01-24T15:32:20.706537+0000","last_fullsized":"2024-01-24T15:32:20.706537+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.704768+0000","last_change":"2024-01-24T15:32:20.282425+0000","last_active":"2024-01-24T15:32:20.704768+0000","last_peered":"2024-01-24T15:32:20.704768+0000","last_clean":"2024-01-24T15:32:20.704768+0000","last_became_active":"2024-01-24T15:32:20.281461+0000","last_became_peered":"2024-01-24T15:32:20.281461+0000","last_unstale":"2024-01-24T15:32:20.704768+0000","last_undegraded":"2024-01-24T15:32:20.704768+0000","last_fullsized":"2024-01-24T15:32:20.704768+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.700488+0000","last_change":"2024-01-24T15:32:20.695888+0000","last_active":"2024-01-24T15:32:20.700488+0000","last_peered":"2024-01-24T15:32:20.700488+0000","last_clean":"2024-01-24T15:32:20.700488+0000","last_became_active":"2024-01-24T15:32:20.695583+0000","last_became_peered":"2024-01-24T15:32:20.695583+0000","last_unstale":"2024-01-24T15:32:20.700488+0000","last_undegraded":"2024-01-24T15:32:20.700488+0000","last_fullsized":"2024-01-24T15:32:20.700488+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699069+0000","last_change":"2024-01-24T15:32:20.271749+0000","last_active":"2024-01-24T15:32:20.699069+0000","last_peered":"2024-01-24T15:32:20.699069+0000","last_clean":"2024-01-24T15:32:20.699069+0000","last_became_active":"2024-01-24T15:32:20.271536+0000","last_became_peered":"2024-01-24T15:32:20.271536+0000","last_unstale":"2024-01-24T15:32:20.699069+0000","last_undegraded":"2024-01-24T15:32:20.699069+0000","last_fullsized":"2024-01-24T15:32:20.699069+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.720279+0000","last_change":"2024-01-24T15:32:20.712200+0000","last_active":"2024-01-24T15:32:20.720279+0000","last_peered":"2024-01-24T15:32:20.720279+0000","last_clean":"2024-01-24T15:32:20.720279+0000","last_became_active":"2024-01-24T15:32:20.711833+0000","last_became_peered":"2024-01-24T15:32:20.711833+0000","last_unstale":"2024-01-24T15:32:20.720279+0000","last_undegraded":"2024-01-24T15:32:20.720279+0000","last_fullsized":"2024-01-24T15:32:20.720279+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.693885+0000","last_change":"2024-01-24T15:32:20.287927+0000","last_active":"2024-01-24T15:32:20.693885+0000","last_peered":"2024-01-24T15:32:20.693885+0000","last_clean":"2024-01-24T15:32:20.693885+0000","last_became_active":"2024-01-24T15:32:20.287600+0000","last_became_peered":"2024-01-24T15:32:20.287600+0000","last_unstale":"2024-01-24T15:32:20.693885+0000","last_undegraded":"2024-01-24T15:32:20.693885+0000","last_fullsized":"2024-01-24T15:32:20.693885+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711821+0000","last_change":"2024-01-24T15:32:20.707934+0000","last_active":"2024-01-24T15:32:20.711821+0000","last_peered":"2024-01-24T15:32:20.711821+0000","last_clean":"2024-01-24T15:32:20.711821+0000","last_became_active":"2024-01-24T15:32:20.707715+0000","last_became_peered":"2024-01-24T15:32:20.707715+0000","last_unstale":"2024-01-24T15:32:20.711821+0000","last_undegraded":"2024-01-24T15:32:20.711821+0000","last_fullsized":"2024-01-24T15:32:20.711821+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699558+0000","last_change":"2024-01-24T15:32:20.287254+0000","last_active":"2024-01-24T15:32:20.699558+0000","last_peered":"2024-01-24T15:32:20.699558+0000","last_clean":"2024-01-24T15:32:20.699558+0000","last_became_active":"2024-01-24T15:32:20.286952+0000","last_became_peered":"2024-01-24T15:32:20.286952+0000","last_unstale":"2024-01-24T15:32:20.699558+0000","last_undegraded":"2024-01-24T15:32:20.699558+0000","last_fullsized":"2024-01-24T15:32:20.699558+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":187,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703800+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:32:20.703800+0000","last_peered":"2024-01-24T15:32:20.703800+0000","last_clean":"2024-01-24T15:32:20.703800+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:32:20.703800+0000","last_undegraded":"2024-01-24T15:32:20.703800+0000","last_fullsized":"2024-01-24T15:32:20.703800+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714191+0000","last_change":"2024-01-24T15:32:20.708196+0000","last_active":"2024-01-24T15:32:20.714191+0000","last_peered":"2024-01-24T15:32:20.714191+0000","last_clean":"2024-01-24T15:32:20.714191+0000","last_became_active":"2024-01-24T15:32:20.707375+0000","last_became_peered":"2024-01-24T15:32:20.707375+0000","last_unstale":"2024-01-24T15:32:20.714191+0000","last_undegraded":"2024-01-24T15:32:20.714191+0000","last_fullsized":"2024-01-24T15:32:20.714191+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.704868+0000","last_change":"2024-01-24T15:32:20.285721+0000","last_active":"2024-01-24T15:32:20.704868+0000","last_peered":"2024-01-24T15:32:20.704868+0000","last_clean":"2024-01-24T15:32:20.704868+0000","last_became_active":"2024-01-24T15:32:20.281894+0000","last_became_peered":"2024-01-24T15:32:20.281894+0000","last_unstale":"2024-01-24T15:32:20.704868+0000","last_undegraded":"2024-01-24T15:32:20.704868+0000","last_fullsized":"2024-01-24T15:32:20.704868+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":213,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.705777+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:32:20.705777+0000","last_peered":"2024-01-24T15:32:20.705777+0000","last_clean":"2024-01-24T15:32:20.705777+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:32:20.705777+0000","last_undegraded":"2024-01-24T15:32:20.705777+0000","last_fullsized":"2024-01-24T15:32:20.705777+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.718212+0000","last_change":"2024-01-24T15:32:20.715254+0000","last_active":"2024-01-24T15:32:20.718212+0000","last_peered":"2024-01-24T15:32:20.718212+0000","last_clean":"2024-01-24T15:32:20.718212+0000","last_became_active":"2024-01-24T15:32:20.714892+0000","last_became_peered":"2024-01-24T15:32:20.714892+0000","last_unstale":"2024-01-24T15:32:20.718212+0000","last_undegraded":"2024-01-24T15:32:20.718212+0000","last_fullsized":"2024-01-24T15:32:20.718212+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703779+0000","last_change":"2024-01-24T15:32:20.284194+0000","last_active":"2024-01-24T15:32:20.703779+0000","last_peered":"2024-01-24T15:32:20.703779+0000","last_clean":"2024-01-24T15:32:20.703779+0000","last_became_active":"2024-01-24T15:32:20.283879+0000","last_became_peered":"2024-01-24T15:32:20.283879+0000","last_unstale":"2024-01-24T15:32:20.703779+0000","last_undegraded":"2024-01-24T15:32:20.703779+0000","last_fullsized":"2024-01-24T15:32:20.703779+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":208,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.689230+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:32:20.689230+0000","last_peered":"2024-01-24T15:32:20.689230+0000","last_clean":"2024-01-24T15:32:20.689230+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:32:20.689230+0000","last_undegraded":"2024-01-24T15:32:20.689230+0000","last_fullsized":"2024-01-24T15:32:20.689230+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.717102+0000","last_change":"2024-01-24T15:32:20.705419+0000","last_active":"2024-01-24T15:32:20.717102+0000","last_peered":"2024-01-24T15:32:20.717102+0000","last_clean":"2024-01-24T15:32:20.717102+0000","last_became_active":"2024-01-24T15:32:20.705138+0000","last_became_peered":"2024-01-24T15:32:20.705138+0000","last_unstale":"2024-01-24T15:32:20.717102+0000","last_undegraded":"2024-01-24T15:32:20.717102+0000","last_fullsized":"2024-01-24T15:32:20.717102+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.694974+0000","last_change":"2024-01-24T15:32:20.281846+0000","last_active":"2024-01-24T15:32:20.694974+0000","last_peered":"2024-01-24T15:32:20.694974+0000","last_clean":"2024-01-24T15:32:20.694974+0000","last_became_active":"2024-01-24T15:32:20.281577+0000","last_became_peered":"2024-01-24T15:32:20.281577+0000","last_unstale":"2024-01-24T15:32:20.694974+0000","last_undegraded":"2024-01-24T15:32:20.694974+0000","last_fullsized":"2024-01-24T15:32:20.694974+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":213,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.707255+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:32:20.707255+0000","last_peered":"2024-01-24T15:32:20.707255+0000","last_clean":"2024-01-24T15:32:20.707255+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:32:20.707255+0000","last_undegraded":"2024-01-24T15:32:20.707255+0000","last_fullsized":"2024-01-24T15:32:20.707255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":190,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699920+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:32:20.699920+0000","last_peered":"2024-01-24T15:32:20.699920+0000","last_clean":"2024-01-24T15:32:20.699920+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:32:20.699920+0000","last_undegraded":"2024-01-24T15:32:20.699920+0000","last_fullsized":"2024-01-24T15:32:20.699920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.714413+0000","last_change":"2024-01-24T15:32:20.710149+0000","last_active":"2024-01-24T15:32:20.714413+0000","last_peered":"2024-01-24T15:32:20.714413+0000","last_clean":"2024-01-24T15:32:20.714413+0000","last_became_active":"2024-01-24T15:32:20.709851+0000","last_became_peered":"2024-01-24T15:32:20.709851+0000","last_unstale":"2024-01-24T15:32:20.714413+0000","last_undegraded":"2024-01-24T15:32:20.714413+0000","last_fullsized":"2024-01-24T15:32:20.714413+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.696473+0000","last_change":"2024-01-24T15:32:20.277807+0000","last_active":"2024-01-24T15:32:20.696473+0000","last_peered":"2024-01-24T15:32:20.696473+0000","last_clean":"2024-01-24T15:32:20.696473+0000","last_became_active":"2024-01-24T15:32:20.277507+0000","last_became_peered":"2024-01-24T15:32:20.277507+0000","last_unstale":"2024-01-24T15:32:20.696473+0000","last_undegraded":"2024-01-24T15:32:20.696473+0000","last_fullsized":"2024-01-24T15:32:20.696473+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":188,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699831+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:32:20.699831+0000","last_peered":"2024-01-24T15:32:20.699831+0000","last_clean":"2024-01-24T15:32:20.699831+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:32:20.699831+0000","last_undegraded":"2024-01-24T15:32:20.699831+0000","last_fullsized":"2024-01-24T15:32:20.699831+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721388+0000","last_change":"2024-01-24T15:32:20.718896+0000","last_active":"2024-01-24T15:32:20.721388+0000","last_peered":"2024-01-24T15:32:20.721388+0000","last_clean":"2024-01-24T15:32:20.721388+0000","last_became_active":"2024-01-24T15:32:20.716591+0000","last_became_peered":"2024-01-24T15:32:20.716591+0000","last_unstale":"2024-01-24T15:32:20.721388+0000","last_undegraded":"2024-01-24T15:32:20.721388+0000","last_fullsized":"2024-01-24T15:32:20.721388+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"230'2","reported_seq":12,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.711777+0000","last_change":"2024-01-24T15:32:20.279372+0000","last_active":"2024-01-24T15:32:20.711777+0000","last_peered":"2024-01-24T15:32:20.711777+0000","last_clean":"2024-01-24T15:32:20.711777+0000","last_became_active":"2024-01-24T15:32:20.279083+0000","last_became_peered":"2024-01-24T15:32:20.279083+0000","last_unstale":"2024-01-24T15:32:20.711777+0000","last_undegraded":"2024-01-24T15:32:20.711777+0000","last_fullsized":"2024-01-24T15:32:20.711777+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":188,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.699277+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:32:20.699277+0000","last_peered":"2024-01-24T15:32:20.699277+0000","last_clean":"2024-01-24T15:32:20.699277+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:32:20.699277+0000","last_undegraded":"2024-01-24T15:32:20.699277+0000","last_fullsized":"2024-01-24T15:32:20.699277+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.713979+0000","last_change":"2024-01-24T15:32:20.708663+0000","last_active":"2024-01-24T15:32:20.713979+0000","last_peered":"2024-01-24T15:32:20.713979+0000","last_clean":"2024-01-24T15:32:20.713979+0000","last_became_active":"2024-01-24T15:32:20.708364+0000","last_became_peered":"2024-01-24T15:32:20.708364+0000","last_unstale":"2024-01-24T15:32:20.713979+0000","last_undegraded":"2024-01-24T15:32:20.713979+0000","last_fullsized":"2024-01-24T15:32:20.713979+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721234+0000","last_change":"2024-01-24T15:32:20.282694+0000","last_active":"2024-01-24T15:32:20.721234+0000","last_peered":"2024-01-24T15:32:20.721234+0000","last_clean":"2024-01-24T15:32:20.721234+0000","last_became_active":"2024-01-24T15:32:20.282008+0000","last_became_peered":"2024-01-24T15:32:20.282008+0000","last_unstale":"2024-01-24T15:32:20.721234+0000","last_undegraded":"2024-01-24T15:32:20.721234+0000","last_fullsized":"2024-01-24T15:32:20.721234+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":187,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.703902+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:32:20.703902+0000","last_peered":"2024-01-24T15:32:20.703902+0000","last_clean":"2024-01-24T15:32:20.703902+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:32:20.703902+0000","last_undegraded":"2024-01-24T15:32:20.703902+0000","last_fullsized":"2024-01-24T15:32:20.703902+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.721696+0000","last_change":"2024-01-24T15:32:20.718975+0000","last_active":"2024-01-24T15:32:20.721696+0000","last_peered":"2024-01-24T15:32:20.721696+0000","last_clean":"2024-01-24T15:32:20.721696+0000","last_became_active":"2024-01-24T15:32:20.718720+0000","last_became_peered":"2024-01-24T15:32:20.718720+0000","last_unstale":"2024-01-24T15:32:20.721696+0000","last_undegraded":"2024-01-24T15:32:20.721696+0000","last_fullsized":"2024-01-24T15:32:20.721696+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:20.248503+0000","last_clean_scrub_stamp":"2024-01-24T15:32:20.248503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"230'1","reported_seq":11,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.707048+0000","last_change":"2024-01-24T15:32:20.276859+0000","last_active":"2024-01-24T15:32:20.707048+0000","last_peered":"2024-01-24T15:32:20.707048+0000","last_clean":"2024-01-24T15:32:20.707048+0000","last_became_active":"2024-01-24T15:32:20.276557+0000","last_became_peered":"2024-01-24T15:32:20.276557+0000","last_unstale":"2024-01-24T15:32:20.707048+0000","last_undegraded":"2024-01-24T15:32:20.707048+0000","last_fullsized":"2024-01-24T15:32:20.707048+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:19.242980+0000","last_clean_scrub_stamp":"2024-01-24T15:32:19.242980+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":186,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-01-24T15:32:20.695447+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:32:20.695447+0000","last_peered":"2024-01-24T15:32:20.695447+0000","last_clean":"2024-01-24T15:32:20.695447+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:32:20.695447+0000","last_undegraded":"2024-01-24T15:32:20.695447+0000","last_fullsized":"2024-01-24T15:32:20.695447+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575973,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304032,"kb_used_data":2840,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94067808,"statfs":{"total":96636764160,"available":96325435392,"internally_reserved":0,"allocated":2908160,"data_stored":1683156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.47499999999999998,"5min":0.39300000000000002,"15min":0.32000000000000001},"max":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.83199999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.75700000000000001,"5min":0.84799999999999998,"15min":0.85499999999999998},"last":0.621}]},{"osd":1,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.61299999999999999,"5min":0.374,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":0.878,"15min":0.878},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.89600000000000002},"last":0.63400000000000001}]},{"osd":2,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.55900000000000005,"5min":0.374,"15min":0.374},"max":{"1min":0.82899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80800000000000005,"5min":0.83299999999999996,"15min":0.85099999999999998},"last":0.58699999999999997}]},{"osd":3,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":0.82399999999999995,"5min":0.83199999999999996,"15min":0.92600000000000005},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.50700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.88},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51400000000000001,"15min":0.49399999999999999},"min":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.30599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.5}]},{"osd":5,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.439,"5min":0.35899999999999999,"15min":0.32300000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.76100000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.42399999999999999,"5min":0.39000000000000001,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81899999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.623,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.40999999999999998,"15min":0.35099999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.92700000000000005},"last":0.67200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575974,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308100,"kb_used_data":2812,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063740,"statfs":{"total":96636764160,"available":96321269760,"internally_reserved":0,"allocated":2879488,"data_stored":1673269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.442,"5min":0.377,"15min":0.31900000000000001},"max":{"1min":0.83699999999999997,"5min":0.998,"15min":0.998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.61699999999999999},"min":{"1min":0.48599999999999999,"5min":0.44700000000000001,"15min":0.32800000000000001},"max":{"1min":0.82399999999999995,"5min":0.82699999999999996,"15min":0.95599999999999996},"last":0.36599999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":1.089},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.45000000000000001,"5min":0.41899999999999998,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.85599999999999998,"15min":0.94299999999999995},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.627},"min":{"1min":0.379,"5min":0.379,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.443},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.90900000000000003},"last":0.79000000000000004}]},{"osd":3,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64200000000000002,"15min":0.63900000000000001},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.371},"max":{"1min":0.879,"5min":0.96299999999999997,"15min":1.0980000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.42299999999999999},"max":{"1min":0.78300000000000003,"5min":0.83899999999999997,"15min":1.169},"last":0.50700000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":0.73599999999999999,"5min":0.753,"15min":0.753},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.33100000000000002,"15min":0.312},"max":{"1min":0.79100000000000004,"5min":0.81399999999999995,"15min":0.83999999999999997},"last":0.66400000000000003}]},{"osd":5,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":0.82499999999999996,"15min":0.86499999999999999},"last":0.628},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69599999999999995,"5min":0.81699999999999995,"15min":0.82799999999999996},"last":0.86699999999999999}]},{"osd":7,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71099999999999997,"5min":0.83699999999999997,"15min":0.88},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.83599999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575973,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308112,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063728,"statfs":{"total":96636764160,"available":96321257472,"internally_reserved":0,"allocated":2891776,"data_stored":1673879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.41799999999999998},"min":{"1min":0.28999999999999998,"5min":0.26600000000000001,"15min":0.24299999999999999},"max":{"1min":0.91200000000000003,"5min":3.254,"15min":3.254},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.26100000000000001},"max":{"1min":0.82299999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.28100000000000003}]},{"osd":2,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.34200000000000003,"5min":0.30299999999999999,"15min":0.28499999999999998},"max":{"1min":0.86499999999999999,"5min":1.306,"15min":1.306},"last":0.318},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.45200000000000001},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.46600000000000003},"min":{"1min":0.31900000000000001,"5min":0.312,"15min":0.308},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.36899999999999999,"5min":0.34100000000000003,"15min":0.32600000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.33400000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73799999999999999,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47299999999999998},"max":{"1min":1.1080000000000001,"5min":5.3630000000000004,"15min":5.3630000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.55500000000000005,"5min":0.46300000000000002,"15min":0.40400000000000003},"max":{"1min":1.244,"5min":1.677,"15min":1.677},"last":0.69899999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.71699999999999997},"min":{"1min":0.53900000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.232,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.56100000000000005,"5min":0.47399999999999998,"15min":0.46100000000000002},"max":{"1min":1.081,"5min":1.117,"15min":1.2989999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.54100000000000004,"5min":0.47899999999999998,"15min":0.46800000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.57599999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1859999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.72499999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.58099999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1759999999999999,"5min":1.704,"15min":1.704},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81100000000000005,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1599999999999999,"5min":5.29,"15min":5.29},"last":0.67100000000000004}]}]},{"osd":0,"up_from":14,"seq":60129543271,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312200,"kb_used_data":2816,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059640,"statfs":{"total":96636764160,"available":96317071360,"internally_reserved":0,"allocated":2883584,"data_stored":1673290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.40799999999999997},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.26000000000000001},"max":{"1min":0.65600000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.27300000000000002},"max":{"1min":0.66100000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47899999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.318,"15min":0.29599999999999999},"max":{"1min":0.73399999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.46899999999999997,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":0.64600000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42899999999999999}]},{"osd":3,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.44500000000000001,"15min":0.442},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59299999999999997,"5min":0.67400000000000004,"15min":0.72799999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76600000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.98699999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.51900000000000002,"5min":0.45100000000000001,"15min":0.41199999999999998},"max":{"1min":0.92700000000000005,"5min":0.998,"15min":0.998},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67900000000000005,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.40699999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.0029999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.55800000000000005,"5min":0.46700000000000003,"15min":0.46500000000000002},"max":{"1min":1.0960000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.53500000000000003}]},{"osd":6,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97799999999999998,"5min":1.27,"15min":1.27},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.61799999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.746}]},{"osd":7,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72799999999999998,"15min":0.73099999999999998},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98699999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.55600000000000005,"5min":0.51000000000000001,"15min":0.46600000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.64000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575973,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312200,"kb_used_data":2816,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059640,"statfs":{"total":96636764160,"available":96317071360,"internally_reserved":0,"allocated":2883584,"data_stored":1673339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.69699999999999995},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.60899999999999999,"5min":0.66600000000000004,"15min":0.70599999999999996},"last":0.54200000000000004}]},{"osd":1,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.432,"15min":0.432},"min":{"1min":0.32200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.40200000000000002,"5min":0.32600000000000001,"15min":0.27400000000000002},"max":{"1min":0.66400000000000003,"5min":0.69799999999999995,"15min":0.80900000000000005},"last":0.51200000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.29099999999999998},"max":{"1min":0.623,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.57799999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78500000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.753},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.55100000000000005,"5min":0.45000000000000001,"15min":0.376},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.69199999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":0.996,"15min":1.0589999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.45700000000000002},"max":{"1min":0.95999999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.39700000000000002},"max":{"1min":0.91800000000000004,"5min":1.006,"15min":1.0740000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.51700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.89200000000000002,"5min":1.0229999999999999,"15min":1.083},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96599999999999997,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.75800000000000001},"min":{"1min":0.61299999999999999,"5min":0.52000000000000002,"15min":0.51100000000000001},"max":{"1min":0.92000000000000004,"5min":1.0429999999999999,"15min":1.123},"last":0.82799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575973,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308120,"kb_used_data":2832,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063720,"statfs":{"total":96636764160,"available":96321249280,"internally_reserved":0,"allocated":2899968,"data_stored":1681178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.318,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.438},"min":{"1min":0.35299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.76100000000000001},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42799999999999999},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.65000000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.752},"last":0.432}]},{"osd":2,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.80800000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.81899999999999995},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66000000000000003,"15min":0.66100000000000003},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.437},"max":{"1min":0.80100000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.498},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.93899999999999995,"5min":1.026,"15min":1.026},"last":0.47099999999999997}]},{"osd":5,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.056,"15min":1.056},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.499,"15min":0.47199999999999998},"max":{"1min":0.93700000000000006,"5min":1.095,"15min":1.095},"last":0.57099999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":1.1499999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72099999999999997,"15min":0.72199999999999998},"min":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.0940000000000001},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.55000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.018,"5min":1.071,"15min":1.083},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.53600000000000003},"max":{"1min":0.95399999999999996,"5min":1.081,"15min":1.1359999999999999},"last":0.56000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575974,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312224,"kb_used_data":2840,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059616,"statfs":{"total":96636764160,"available":96317046784,"internally_reserved":0,"allocated":2908160,"data_stored":1681814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.46700000000000003,"5min":0.39400000000000002,"15min":0.375},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.503},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.80100000000000005,"5min":0.876,"15min":0.876},"last":0.52500000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.622},"min":{"1min":0.496,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.45900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.79100000000000004,"5min":0.80400000000000005,"15min":0.82199999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.40500000000000003,"15min":0.38500000000000001},"max":{"1min":0.82899999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.61699999999999999}]},{"osd":3,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82299999999999995,"5min":0.86799999999999999,"15min":0.87},"last":0.69699999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.69699999999999995,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.33800000000000002,"5min":0.31900000000000001,"15min":0.308},"max":{"1min":0.66200000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.93600000000000005}]},{"osd":6,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.33600000000000002},"max":{"1min":0.61199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.37},"max":{"1min":0.72599999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.34899999999999998},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.375,"15min":0.35799999999999998},"max":{"1min":0.78100000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.83199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308112,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063728,"statfs":{"total":96636764160,"available":96321257472,"internally_reserved":0,"allocated":2891776,"data_stored":1681066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.34799999999999998},"max":{"1min":0.88300000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.624},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.48399999999999999,"5min":0.41999999999999998,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":1.149,"15min":1.149},"last":0.61099999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.52800000000000002,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":1.125,"15min":1.125},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.38400000000000001,"15min":0.38},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54200000000000004}]},{"osd":2,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.432,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77900000000000003,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.61599999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.41199999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":1.252},"last":0.55500000000000005}]},{"osd":3,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.51100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.95499999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.64400000000000002},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":1.196,"15min":1.196},"last":0.65000000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.58699999999999997,"5min":0.78700000000000003,"15min":0.90500000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.81299999999999994},"last":0.52400000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.50600000000000001},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68400000000000005,"5min":0.82199999999999995,"15min":1.0209999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.54200000000000004},"min":{"1min":0.41099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.98399999999999999},"last":0.56499999999999995}]},{"osd":7,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.54200000000000004},"min":{"1min":0.39700000000000002,"5min":0.371,"15min":0.33600000000000002},"max":{"1min":0.629,"5min":0.80100000000000005,"15min":0.86399999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.57799999999999996},"min":{"1min":0.372,"5min":0.371,"15min":0.371},"max":{"1min":0.80900000000000005,"5min":0.83099999999999996,"15min":0.91700000000000004},"last":0.66400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:32:25.215 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-24T15:32:25.216 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-24T15:32:25.387 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.386+0000 7f617ad61700 1 Processor -- start 2024-01-24T15:32:25.388 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.386+0000 7f617ad61700 1 -- start start 2024-01-24T15:32:25.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.387+0000 7f617ad61700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f617414e5f0 0x7f6174065f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.387+0000 7f617ad61700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f61740c3e60 0x7f61740c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.387+0000 7f617ad61700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6174155230 con 0x7f61740627b0 2024-01-24T15:32:25.390 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.387+0000 7f617ad61700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f617406a4b0 con 0x7f61740c3e60 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.387+0000 7f617ad61700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6174154640 con 0x7f617414e5f0 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.387+0000 7f6173fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f61740c3e60 0x7f61740c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.387+0000 7f6173fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f61740c3e60 0x7f61740c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55196/0 (socket says 172.21.15.29:55196) 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.387+0000 7f6173fff700 1 -- 172.21.15.29:0/3831295865 learned_addr learned my addr 172.21.15.29:0/3831295865 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.388+0000 7f6178afd700 1 --2- 172.21.15.29:0/3831295865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f617414e5f0 0x7f6174065f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.388+0000 7f6173fff700 1 -- 172.21.15.29:0/3831295865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f617414e5f0 msgr2=0x7f6174065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.388+0000 7f6173fff700 1 --2- 172.21.15.29:0/3831295865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f617414e5f0 0x7f6174065f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.388+0000 7f6173fff700 1 -- 172.21.15.29:0/3831295865 >> v1:172.21.15.29:6789/0 conn(0x7f61740627b0 legacy=0x7f61740a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.388+0000 7f6173fff700 1 -- 172.21.15.29:0/3831295865 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6174154900 con 0x7f61740c3e60 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.388+0000 7f61737fe700 1 -- 172.21.15.29:0/3831295865 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610147069 0 0) 0x7f6174155230 con 0x7f61740627b0 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.388+0000 7f6178afd700 1 --2- 172.21.15.29:0/3831295865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f617414e5f0 0x7f6174065f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.388+0000 7f6173fff700 1 --2- 172.21.15.29:0/3831295865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61740c3e60 0x7f61740c2260 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 rx=0x7f6168006fd0 tx=0x7f61680090f0).ready entity=mon.0 client_cookie=935bcd029274f39 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.388+0000 7f61737fe700 1 -- 172.21.15.29:0/3831295865 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800b600 con 0x7f61740c3e60 2024-01-24T15:32:25.391 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.388+0000 7f61737fe700 1 -- 172.21.15.29:0/3831295865 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616800b760 con 0x7f61740c3e60 2024-01-24T15:32:25.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.388+0000 7f61737fe700 1 -- 172.21.15.29:0/3831295865 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800bae0 con 0x7f61740c3e60 2024-01-24T15:32:25.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.389+0000 7f617ad61700 1 -- 172.21.15.29:0/3831295865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61740c3e60 msgr2=0x7f61740c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:25.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.389+0000 7f617ad61700 1 --2- 172.21.15.29:0/3831295865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61740c3e60 0x7f61740c2260 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 rx=0x7f6168006fd0 tx=0x7f61680090f0).stop 2024-01-24T15:32:25.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.389+0000 7f617ad61700 1 -- 172.21.15.29:0/3831295865 shutdown_connections 2024-01-24T15:32:25.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.389+0000 7f617ad61700 1 --2- 172.21.15.29:0/3831295865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f617414e5f0 0x7f6174065f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.389+0000 7f617ad61700 1 --2- 172.21.15.29:0/3831295865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61740c3e60 0x7f61740c2260 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.389+0000 7f617ad61700 1 -- 172.21.15.29:0/3831295865 >> 172.21.15.29:0/3831295865 conn(0x7f61740638c0 msgr2=0x7f6174063cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:25.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.389+0000 7f617ad61700 1 -- 172.21.15.29:0/3831295865 shutdown_connections 2024-01-24T15:32:25.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.389+0000 7f617ad61700 1 -- 172.21.15.29:0/3831295865 wait complete. 2024-01-24T15:32:25.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.390+0000 7f617ad61700 1 Processor -- start 2024-01-24T15:32:25.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.390+0000 7f617ad61700 1 -- start start 2024-01-24T15:32:25.394 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.391+0000 7f617ad61700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61740627b0 0x7f617414d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.391+0000 7f617ad61700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f617414e5f0 0x7f617413bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.391+0000 7f617ad61700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f617414db00 0x7f6174063ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.391+0000 7f617ad61700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61740c7860 con 0x7f61740627b0 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.391+0000 7f617ad61700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61740c7b40 con 0x7f617414db00 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.391+0000 7f617ad61700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61740c7e20 con 0x7f617414e5f0 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.391+0000 7f6178afd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61740627b0 0x7f617414d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.391+0000 7f6178afd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61740627b0 0x7f617414d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55208/0 (socket says 172.21.15.29:55208) 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.391+0000 7f6178afd700 1 -- 172.21.15.29:0/2643568608 learned_addr learned my addr 172.21.15.29:0/2643568608 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.391+0000 7f6173fff700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f617414e5f0 0x7f617413bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.391+0000 7f61792fe700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f617414db00 0x7f6174063ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.392+0000 7f6178afd700 1 -- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f617414e5f0 msgr2=0x7f617413bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.392+0000 7f6178afd700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f617414e5f0 0x7f617413bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.392+0000 7f6178afd700 1 -- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f617414db00 msgr2=0x7f6174063ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:25.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.392+0000 7f6178afd700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f617414db00 0x7f6174063ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.392+0000 7f6178afd700 1 -- 172.21.15.29:0/2643568608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6168009370 con 0x7f61740627b0 2024-01-24T15:32:25.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.392+0000 7f6178afd700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61740627b0 0x7f617414d610 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 rx=0x7f616400d890 tx=0x7f6164008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:25.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.392+0000 7f6170ff9700 1 -- 172.21.15.29:0/2643568608 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164011590 con 0x7f61740627b0 2024-01-24T15:32:25.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.392+0000 7f6170ff9700 1 -- 172.21.15.29:0/2643568608 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61640116f0 con 0x7f61740627b0 2024-01-24T15:32:25.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.392+0000 7f617ad61700 1 -- 172.21.15.29:0/2643568608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61740c8100 con 0x7f61740627b0 2024-01-24T15:32:25.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.392+0000 7f6170ff9700 1 -- 172.21.15.29:0/2643568608 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164011a70 con 0x7f61740627b0 2024-01-24T15:32:25.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.392+0000 7f617ad61700 1 -- 172.21.15.29:0/2643568608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61740c73f0 con 0x7f61740627b0 2024-01-24T15:32:25.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.393+0000 7f6170ff9700 1 -- 172.21.15.29:0/2643568608 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6164012420 con 0x7f61740627b0 2024-01-24T15:32:25.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.394+0000 7f617ad61700 1 -- 172.21.15.29:0/2643568608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61740c75e0 con 0x7f61740627b0 2024-01-24T15:32:25.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.394+0000 7f6170ff9700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f616005f270 0x7f6160061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.394+0000 7f6170ff9700 1 -- 172.21.15.29:0/2643568608 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f616407a110 con 0x7f61740627b0 2024-01-24T15:32:25.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.394+0000 7f6173fff700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f616005f270 0x7f6160061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.395+0000 7f6173fff700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f616005f270 0x7f6160061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f61740c3c50 tx=0x7f6168017000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:25.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.398+0000 7f6170ff9700 1 -- 172.21.15.29:0/2643568608 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f616404adc0 con 0x7f61740627b0 2024-01-24T15:32:25.565 INFO:tasks.ceph.mds.d.smithi163.stderr:2024-01-24T15:32:25.562+0000 7f8b8aa13700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T15:32:25.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.653+0000 7f617ad61700 1 -- 172.21.15.29:0/2643568608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f61740475e0 con 0x7f61740627b0 2024-01-24T15:32:25.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.657+0000 7f6170ff9700 1 -- 172.21.15.29:0/2643568608 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v54) v1 ==== 190+0+0 (secure 0 0 0) 0x7f616404e3e0 con 0x7f61740627b0 2024-01-24T15:32:25.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.659+0000 7f617ad61700 1 -- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f616005f270 msgr2=0x7f6160061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:25.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.659+0000 7f617ad61700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f616005f270 0x7f6160061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f61740c3c50 tx=0x7f6168017000).stop 2024-01-24T15:32:25.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.659+0000 7f617ad61700 1 -- 172.21.15.29:0/2643568608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61740627b0 msgr2=0x7f617414d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:25.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.659+0000 7f617ad61700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61740627b0 0x7f617414d610 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 rx=0x7f616400d890 tx=0x7f6164008040).stop 2024-01-24T15:32:25.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.659+0000 7f617ad61700 1 -- 172.21.15.29:0/2643568608 shutdown_connections 2024-01-24T15:32:25.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.659+0000 7f617ad61700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f617414db00 0x7f6174063ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.659+0000 7f617ad61700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61740627b0 0x7f617414d610 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.659+0000 7f617ad61700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f616005f270 0x7f6160061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.659+0000 7f617ad61700 1 --2- 172.21.15.29:0/2643568608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f617414e5f0 0x7f617413bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.659+0000 7f617ad61700 1 -- 172.21.15.29:0/2643568608 >> 172.21.15.29:0/2643568608 conn(0x7f61740638c0 msgr2=0x7f61740bdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:25.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.659+0000 7f617ad61700 1 -- 172.21.15.29:0/2643568608 shutdown_connections 2024-01-24T15:32:25.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.659+0000 7f617ad61700 1 -- 172.21.15.29:0/2643568608 wait complete. 2024-01-24T15:32:25.662 INFO:teuthology.orchestra.run.smithi029.stderr:updated caps for client.0 2024-01-24T15:32:25.675 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:32:25.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.805+0000 7f074bb1b700 1 Processor -- start 2024-01-24T15:32:25.806 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.805+0000 7f074bb1b700 1 -- start start 2024-01-24T15:32:25.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.806+0000 7f074bb1b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0744155b00 0x7f07440664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.807 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.806+0000 7f074bb1b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07441565f0 0x7f07440bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.806+0000 7f074bb1b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0744066e60 con 0x7f07440669e0 2024-01-24T15:32:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.806+0000 7f074bb1b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07440c5500 con 0x7f07441565f0 2024-01-24T15:32:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.806+0000 7f074bb1b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07440c3910 con 0x7f0744155b00 2024-01-24T15:32:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f07498b7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0744155b00 0x7f07440664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f074a0b8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07441565f0 0x7f07440bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f074a0b8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07441565f0 0x7f07440bdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:43978/0 (socket says 172.21.15.29:43978) 2024-01-24T15:32:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f074a0b8700 1 -- 172.21.15.29:0/1528306194 learned_addr learned my addr 172.21.15.29:0/1528306194 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:25.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f07498b7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0744155b00 0x7f07440664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:34094/0 (socket says 172.21.15.29:34094) 2024-01-24T15:32:25.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f074a0b8700 1 -- 172.21.15.29:0/1528306194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0744155b00 msgr2=0x7f07440664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:25.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f074a0b8700 1 --2- 172.21.15.29:0/1528306194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0744155b00 0x7f07440664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.810 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f074a0b8700 1 -- 172.21.15.29:0/1528306194 >> v1:172.21.15.163:6789/0 conn(0x7f07440669e0 legacy=0x7f0744155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f074a0b8700 1 -- 172.21.15.29:0/1528306194 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07440c5220 con 0x7f07441565f0 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f07498b7700 1 --2- 172.21.15.29:0/1528306194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0744155b00 0x7f07440664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f074a0b8700 1 --2- 172.21.15.29:0/1528306194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07441565f0 0x7f07440bdeb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f073c009f20 tx=0x7f073c009840).ready entity=mon.1 client_cookie=7c66d2b4138d0d87 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f07488b5700 1 -- 172.21.15.29:0/1528306194 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c00e070 con 0x7f07441565f0 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f07488b5700 1 -- 172.21.15.29:0/1528306194 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073c00b9d0 con 0x7f07441565f0 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.807+0000 7f07488b5700 1 -- 172.21.15.29:0/1528306194 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c00bd50 con 0x7f07441565f0 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.808+0000 7f074bb1b700 1 -- 172.21.15.29:0/1528306194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07441565f0 msgr2=0x7f07440bdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.808+0000 7f074bb1b700 1 --2- 172.21.15.29:0/1528306194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07441565f0 0x7f07440bdeb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f073c009f20 tx=0x7f073c009840).stop 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.808+0000 7f074bb1b700 1 -- 172.21.15.29:0/1528306194 shutdown_connections 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.808+0000 7f074bb1b700 1 --2- 172.21.15.29:0/1528306194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0744155b00 0x7f07440664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.808+0000 7f074bb1b700 1 --2- 172.21.15.29:0/1528306194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07441565f0 0x7f07440bdeb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.808+0000 7f074bb1b700 1 -- 172.21.15.29:0/1528306194 >> 172.21.15.29:0/1528306194 conn(0x7f07440c8bb0 msgr2=0x7f07440c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.808+0000 7f074bb1b700 1 -- 172.21.15.29:0/1528306194 shutdown_connections 2024-01-24T15:32:25.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.808+0000 7f074bb1b700 1 -- 172.21.15.29:0/1528306194 wait complete. 2024-01-24T15:32:25.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.809+0000 7f074bb1b700 1 Processor -- start 2024-01-24T15:32:25.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.809+0000 7f074bb1b700 1 -- start start 2024-01-24T15:32:25.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.810+0000 7f074bb1b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07440669e0 0x7f0744166fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.810+0000 7f074bb1b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0744155b00 0x7f074413a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.810+0000 7f074bb1b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07441565f0 0x7f074413a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.810+0000 7f074bb1b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07440638c0 con 0x7f07441565f0 2024-01-24T15:32:25.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.810+0000 7f074bb1b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0744064440 con 0x7f0744155b00 2024-01-24T15:32:25.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.810+0000 7f074bb1b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074413aac0 con 0x7f07440669e0 2024-01-24T15:32:25.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.810+0000 7f074a0b8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07441565f0 0x7f074413a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.810+0000 7f074a0b8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07441565f0 0x7f074413a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55224/0 (socket says 172.21.15.29:55224) 2024-01-24T15:32:25.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.810+0000 7f074a0b8700 1 -- 172.21.15.29:0/3428249391 learned_addr learned my addr 172.21.15.29:0/3428249391 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:25.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f07498b7700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07440669e0 0x7f0744166fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f07490b6700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0744155b00 0x7f074413a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f074a0b8700 1 -- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07440669e0 msgr2=0x7f0744166fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:25.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f074a0b8700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07440669e0 0x7f0744166fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f074a0b8700 1 -- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0744155b00 msgr2=0x7f074413a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:25.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f074a0b8700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0744155b00 0x7f074413a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:25.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f074a0b8700 1 -- 172.21.15.29:0/3428249391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07340092a0 con 0x7f07441565f0 2024-01-24T15:32:25.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f074a0b8700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07441565f0 0x7f074413a5d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 rx=0x7f073c00b610 tx=0x7f07440c6780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:25.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f0739ffb700 1 -- 172.21.15.29:0/3428249391 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c0092e0 con 0x7f07441565f0 2024-01-24T15:32:25.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f074bb1b700 1 -- 172.21.15.29:0/3428249391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c009ae0 con 0x7f07441565f0 2024-01-24T15:32:25.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f0739ffb700 1 -- 172.21.15.29:0/3428249391 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073c007610 con 0x7f07441565f0 2024-01-24T15:32:25.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f0739ffb700 1 -- 172.21.15.29:0/3428249391 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c0079d0 con 0x7f07441565f0 2024-01-24T15:32:25.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.811+0000 7f074bb1b700 1 -- 172.21.15.29:0/3428249391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0744063df0 con 0x7f07441565f0 2024-01-24T15:32:25.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.812+0000 7f074bb1b700 1 -- 172.21.15.29:0/3428249391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074413bbe0 con 0x7f07441565f0 2024-01-24T15:32:25.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.813+0000 7f0739ffb700 1 -- 172.21.15.29:0/3428249391 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f073c003040 con 0x7f07441565f0 2024-01-24T15:32:25.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.813+0000 7f0739ffb700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f072805f270 0x7f0728061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:25.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.813+0000 7f0739ffb700 1 -- 172.21.15.29:0/3428249391 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f073c07c0d0 con 0x7f07441565f0 2024-01-24T15:32:25.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.814+0000 7f07498b7700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f072805f270 0x7f0728061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:25.816 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.814+0000 7f07498b7700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f072805f270 0x7f0728061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f07440aed80 tx=0x7f0734009060).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:25.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:25.818+0000 7f0739ffb700 1 -- 172.21.15.29:0/3428249391 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f073c04cf50 con 0x7f07441565f0 2024-01-24T15:32:26.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.044+0000 7f074bb1b700 1 -- 172.21.15.29:0/3428249391 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07440645d0 con 0x7f07441565f0 2024-01-24T15:32:26.046 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.045+0000 7f0739ffb700 1 -- 172.21.15.29:0/3428249391 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4499 (secure 0 0 0) 0x7f073c023030 con 0x7f07441565f0 2024-01-24T15:32:26.047 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:26.047 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1883755467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1883755467},{"type":"v1","addr":"172.21.15.163:6836","nonce":1883755467}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1773668659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1773668659},{"type":"v1","addr":"172.21.15.29:6834","nonce":1773668659}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3438722509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3438722509},{"type":"v1","addr":"172.21.15.29:6835","nonce":3438722509}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:32:20.685536+0000","modified":"2024-01-24T15:32:23.705717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10029":{"gid":10029,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/532321390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":532321390},{"type":"v1","addr":"172.21.15.163:6837","nonce":532321390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2024-01-24T15:32:26.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.047+0000 7f074bb1b700 1 -- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f072805f270 msgr2=0x7f0728061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.047+0000 7f074bb1b700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f072805f270 0x7f0728061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f07440aed80 tx=0x7f0734009060).stop 2024-01-24T15:32:26.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.047+0000 7f074bb1b700 1 -- 172.21.15.29:0/3428249391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07441565f0 msgr2=0x7f074413a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.047+0000 7f074bb1b700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07441565f0 0x7f074413a5d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 rx=0x7f073c00b610 tx=0x7f07440c6780).stop 2024-01-24T15:32:26.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.047+0000 7f074bb1b700 1 -- 172.21.15.29:0/3428249391 shutdown_connections 2024-01-24T15:32:26.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.047+0000 7f074bb1b700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07441565f0 0x7f074413a5d0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.047+0000 7f074bb1b700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f072805f270 0x7f0728061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.047+0000 7f074bb1b700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07440669e0 0x7f0744166fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.047+0000 7f074bb1b700 1 --2- 172.21.15.29:0/3428249391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0744155b00 0x7f074413a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.049 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.047+0000 7f074bb1b700 1 -- 172.21.15.29:0/3428249391 >> 172.21.15.29:0/3428249391 conn(0x7f07440c8bb0 msgr2=0x7f074413b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:26.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.047+0000 7f074bb1b700 1 -- 172.21.15.29:0/3428249391 shutdown_connections 2024-01-24T15:32:26.050 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.047+0000 7f074bb1b700 1 -- 172.21.15.29:0/3428249391 wait complete. 2024-01-24T15:32:26.050 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 98 2024-01-24T15:32:26.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-24T15:32:20.685536+0000', 'modified': '2024-01-24T15:32:23.705717+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10029}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10029': {'gid': 10029, 'name': 'd', 'rank': 0, 'incarnation': 94, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/532321390', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 532321390}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 532321390}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-24T15:32:26.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-24T15:32:26.467 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-01-24T15:32:26.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.606+0000 7f67b64eb700 1 Processor -- start 2024-01-24T15:32:26.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.606+0000 7f67b64eb700 1 -- start start 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.607+0000 7f67b64eb700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f67b013e220 0x7f67b00639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.607+0000 7f67b64eb700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b00bd770 con 0x7f67b013de40 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.607+0000 7f67b64eb700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b0063540 con 0x7f67b0063ec0 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.607+0000 7f67b64eb700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b0062930 con 0x7f67b013e220 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.607+0000 7f67af7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f67b013e220 0x7f67b00639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.607+0000 7f67affff700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f67b013de40 0x7f67b00bd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54232/0 (socket says 172.21.15.29:54232) 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.607+0000 7f67affff700 1 -- 172.21.15.29:0/2947979220 learned_addr learned my addr 172.21.15.29:0/2947979220 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.607+0000 7f67af7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f67b013e220 0x7f67b00639d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55228/0 (socket says 172.21.15.29:55228) 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.608+0000 7f67af7fe700 1 -- 172.21.15.29:0/2947979220 >> v1:172.21.15.163:6789/0 conn(0x7f67b0063ec0 legacy=0x7f67b00c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.608+0000 7f67af7fe700 1 -- 172.21.15.29:0/2947979220 >> v1:172.21.15.29:6789/0 conn(0x7f67b013de40 legacy=0x7f67b00bd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.608+0000 7f67af7fe700 1 -- 172.21.15.29:0/2947979220 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b0062c10 con 0x7f67b013e220 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.608+0000 7f67aeffd700 1 -- 172.21.15.29:0/2947979220 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167226318 0 0) 0x7f67b00bd770 con 0x7f67b013de40 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.608+0000 7f67af7fe700 1 --2- 172.21.15.29:0/2947979220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67b013e220 0x7f67b00639d0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 rx=0x7f67a4009760 tx=0x7f67a4009060).ready entity=mon.0 client_cookie=dd7ae01ae6360c7a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:26.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.608+0000 7f67aeffd700 1 -- 172.21.15.29:0/2947979220 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a400b520 con 0x7f67b013e220 2024-01-24T15:32:26.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.608+0000 7f67aeffd700 1 -- 172.21.15.29:0/2947979220 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a400b680 con 0x7f67b013e220 2024-01-24T15:32:26.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.608+0000 7f67aeffd700 1 -- 172.21.15.29:0/2947979220 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a400ba00 con 0x7f67b013e220 2024-01-24T15:32:26.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.608+0000 7f67b64eb700 1 -- 172.21.15.29:0/2947979220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67b013e220 msgr2=0x7f67b00639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.608+0000 7f67b64eb700 1 --2- 172.21.15.29:0/2947979220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67b013e220 0x7f67b00639d0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 rx=0x7f67a4009760 tx=0x7f67a4009060).stop 2024-01-24T15:32:26.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.608+0000 7f67b64eb700 1 -- 172.21.15.29:0/2947979220 shutdown_connections 2024-01-24T15:32:26.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.608+0000 7f67b64eb700 1 --2- 172.21.15.29:0/2947979220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67b013e220 0x7f67b00639d0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.609+0000 7f67b64eb700 1 -- 172.21.15.29:0/2947979220 >> 172.21.15.29:0/2947979220 conn(0x7f67b00c3cd0 msgr2=0x7f67b00c40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:26.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.609+0000 7f67b64eb700 1 -- 172.21.15.29:0/2947979220 shutdown_connections 2024-01-24T15:32:26.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.609+0000 7f67b64eb700 1 -- 172.21.15.29:0/2947979220 wait complete. 2024-01-24T15:32:26.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.610+0000 7f67b64eb700 1 Processor -- start 2024-01-24T15:32:26.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.610+0000 7f67b64eb700 1 -- start start 2024-01-24T15:32:26.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67b64eb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67b0063ec0 0x7f67b00aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.613 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67b64eb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67b013de40 0x7f67b00c1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67b64eb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67b013e220 0x7f67b01489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67b64eb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00a8e70 con 0x7f67b0063ec0 2024-01-24T15:32:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67b64eb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00a7f10 con 0x7f67b013e220 2024-01-24T15:32:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67b64eb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00a81f0 con 0x7f67b013de40 2024-01-24T15:32:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67affff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67b0063ec0 0x7f67b00aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67affff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67b0063ec0 0x7f67b00aa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55238/0 (socket says 172.21.15.29:55238) 2024-01-24T15:32:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67affff700 1 -- 172.21.15.29:0/2234895260 learned_addr learned my addr 172.21.15.29:0/2234895260 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67af7fe700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67b013de40 0x7f67b00c1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.614 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67affff700 1 -- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67b013de40 msgr2=0x7f67b00c1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67b4a88700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67b013e220 0x7f67b01489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67affff700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67b013de40 0x7f67b00c1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67affff700 1 -- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67b013e220 msgr2=0x7f67b01489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67affff700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67b013e220 0x7f67b01489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67affff700 1 -- 172.21.15.29:0/2234895260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a40092a0 con 0x7f67b0063ec0 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67af7fe700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67b013de40 0x7f67b00c1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67b4a88700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67b013e220 0x7f67b01489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.611+0000 7f67affff700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67b0063ec0 0x7f67b00aa1e0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 rx=0x7f67a000c4e0 tx=0x7f67a0007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.612+0000 7f6797fff700 1 -- 172.21.15.29:0/2234895260 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a00072e0 con 0x7f67b0063ec0 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.612+0000 7f6797fff700 1 -- 172.21.15.29:0/2234895260 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a0007440 con 0x7f67b0063ec0 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.612+0000 7f6797fff700 1 -- 172.21.15.29:0/2234895260 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a00057a0 con 0x7f67b0063ec0 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.612+0000 7f67b64eb700 1 -- 172.21.15.29:0/2234895260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b00a84d0 con 0x7f67b0063ec0 2024-01-24T15:32:26.615 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.612+0000 7f67b64eb700 1 -- 172.21.15.29:0/2234895260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b00a8a90 con 0x7f67b0063ec0 2024-01-24T15:32:26.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.613+0000 7f6797fff700 1 -- 172.21.15.29:0/2234895260 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f67a0005bd0 con 0x7f67b0063ec0 2024-01-24T15:32:26.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.613+0000 7f6797fff700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f679c05f240 0x7f679c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.616 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.614+0000 7f6797fff700 1 -- 172.21.15.29:0/2234895260 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f67a007a150 con 0x7f67b0063ec0 2024-01-24T15:32:26.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.614+0000 7f67af7fe700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f679c05f240 0x7f679c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.614+0000 7f67af7fe700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f679c05f240 0x7f679c061700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f67b006aea0 tx=0x7f67a4017000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:26.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.616+0000 7f67c7eb6b80 1 Processor -- start 2024-01-24T15:32:26.617 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.616+0000 7f67c7eb6b80 1 -- start start 2024-01-24T15:32:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.617+0000 7f67c7eb6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bd13a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.617+0000 7f67c7eb6b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd5c2c0 0x562e5bd0e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.617+0000 7f67c7eb6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd0edf0 0x562e5bd0f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.617+0000 7f67c7eb6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5bc8a3d0 con 0x562e5bd13620 2024-01-24T15:32:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.617+0000 7f67c7eb6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5bc4fbe0 con 0x562e5bd0edf0 2024-01-24T15:32:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.617+0000 7f67c7eb6b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e5bc65260 con 0x562e5bd5c2c0 2024-01-24T15:32:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.617+0000 7f67affff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bd13a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.617+0000 7f67affff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bd13a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55242/0 (socket says 172.21.15.29:55242) 2024-01-24T15:32:26.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.617+0000 7f67affff700 1 -- 172.21.15.29:0/3976883454 learned_addr learned my addr 172.21.15.29:0/3976883454 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.618+0000 7f67af7fe700 1 --2- 172.21.15.29:0/3976883454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd5c2c0 0x562e5bd0e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.618+0000 7f67b4a88700 1 --2- 172.21.15.29:0/3976883454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd0edf0 0x562e5bd0f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.618+0000 7f67affff700 1 -- 172.21.15.29:0/3976883454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd5c2c0 msgr2=0x562e5bd0e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.618+0000 7f67affff700 1 --2- 172.21.15.29:0/3976883454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd5c2c0 0x562e5bd0e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.618+0000 7f67affff700 1 -- 172.21.15.29:0/3976883454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd0edf0 msgr2=0x562e5bd0f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.618+0000 7f67affff700 1 --2- 172.21.15.29:0/3976883454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd0edf0 0x562e5bd0f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.618+0000 7f67affff700 1 -- 172.21.15.29:0/3976883454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e5bc5e2f0 con 0x562e5bd13620 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.618+0000 7f67affff700 1 --2- 172.21.15.29:0/3976883454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bd13a00 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 rx=0x7f67a00780c0 tx=0x7f67a000f000).ready entity=mon.0 client_cookie=c94f65a951fc3cd5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.618+0000 7f67957fa700 1 -- 172.21.15.29:0/3976883454 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0058030 con 0x562e5bd13620 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.618+0000 7f67957fa700 1 -- 172.21.15.29:0/3976883454 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a004cc60 con 0x562e5bd13620 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.618+0000 7f67957fa700 1 -- 172.21.15.29:0/3976883454 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0054150 con 0x562e5bd13620 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.619+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/3976883454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 msgr2=0x562e5bd13a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.619+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/3976883454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bd13a00 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 rx=0x7f67a00780c0 tx=0x7f67a000f000).stop 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.619+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/3976883454 shutdown_connections 2024-01-24T15:32:26.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.619+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/3976883454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd0edf0 0x562e5bd0f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.619+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/3976883454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bd13a00 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.619+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/3976883454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd5c2c0 0x562e5bd0e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.619+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/3976883454 >> 172.21.15.29:0/3976883454 conn(0x562e5bb7c2e0 msgr2=0x562e5bd5c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:26.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.619+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/3976883454 shutdown_connections 2024-01-24T15:32:26.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.619+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/3976883454 wait complete. 2024-01-24T15:32:26.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.619+0000 7f67c7eb6b80 1 -- start start 2024-01-24T15:32:26.626 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.620+0000 7f67c7eb6b80 1 Processor -- start 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.620+0000 7f67c7eb6b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd0edf0 0x562e5bd3b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.620+0000 7f67c7eb6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bd3e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.620+0000 7f67c7eb6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bd57fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.620+0000 7f67c7eb6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5bc653a0 con 0x562e5bd13620 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.620+0000 7f67c7eb6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5bc65620 con 0x562e5bd5c2c0 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.620+0000 7f67c7eb6b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e5bc30990 con 0x562e5bd0edf0 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.620+0000 7f67c7eb6b80 10 client.0 ll_register_callbacks cb 0x562e5bcd3610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67af7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bd3e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67af7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bd3e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55252/0 (socket says 172.21.15.29:55252) 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67af7fe700 1 -- 172.21.15.29:0/2797160475 learned_addr learned my addr 172.21.15.29:0/2797160475 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67affff700 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd0edf0 0x562e5bd3b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67b4a88700 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bd57fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67af7fe700 1 -- 172.21.15.29:0/2797160475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd0edf0 msgr2=0x562e5bd3b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:26.627 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67af7fe700 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd0edf0 0x562e5bd3b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67af7fe700 1 -- 172.21.15.29:0/2797160475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 msgr2=0x562e5bd57fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67af7fe700 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bd57fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67af7fe700 1 -- 172.21.15.29:0/2797160475 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b00a8a90 con 0x562e5bd13620 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67af7fe700 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bd3e3b0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 rx=0x7f67b005f410 tx=0x7f67a401c000).ready entity=mon.0 client_cookie=dc19165275b01d9a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f678e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f678e7fc700 1 -- 172.21.15.29:0/2797160475 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a400bee0 con 0x562e5bd13620 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f678e7fc700 1 -- 172.21.15.29:0/2797160475 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f67a4011610 con 0x562e5bd13620 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/2797160475 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e5bd0ddf0 con 0x562e5bd13620 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f678e7fc700 1 -- 172.21.15.29:0/2797160475 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67a40119e0 con 0x562e5bd13620 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67c7eb6b80 10 client.10164 fetch_fsmap learned FSMap version 98 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.621+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/2797160475 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) v3 -- 0x562e5bc59520 con 0x562e5bd13620 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.622+0000 7f678e7fc700 1 -- 172.21.15.29:0/2797160475 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e5bd0ddf0 con 0x562e5bd13620 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.622+0000 7f678e7fc700 1 -- 172.21.15.29:0/2797160475 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 98) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f67a4011b40 con 0x562e5bd13620 2024-01-24T15:32:26.628 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.622+0000 7f67c7eb6b80 10 client.10164 fetch_fsmap finished waiting for FSMap version 98 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.622+0000 7f67c7eb6b80 10 client.10164 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.622+0000 7f67c7eb6b80 20 client.10164 populate_metadata read hostname 'smithi029' 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.622+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.163:6835/532321390,v1:172.21.15.163:6837/532321390] conn(0x562e5bd5ef90 0x562e5bd5f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.622+0000 7f67c7eb6b80 4 client.10164 mds_command: new command op to 10029 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.622+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/2797160475 --> [v2:172.21.15.163:6835/532321390,v1:172.21.15.163:6837/532321390] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e5bc5ad60 con 0x562e5bd5ef90 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.622+0000 7f67affff700 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.163:6835/532321390,v1:172.21.15.163:6837/532321390] conn(0x562e5bd5ef90 0x562e5bd5f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.622+0000 7f678e7fc700 10 client.10164 ms_handle_connect on v2:172.21.15.163:6835/532321390 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.622+0000 7f67affff700 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.163:6835/532321390,v1:172.21.15.163:6837/532321390] conn(0x562e5bd5ef90 0x562e5bd5f370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c1a125c56eef24e server_cookie=f152d3dc245e9b44 in_seq=0 out_seq=0 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.623+0000 7f678e7fc700 1 -- 172.21.15.29:0/2797160475 <== mds.0 v2:172.21.15.163:6835/532321390 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x562e5bc5ad60 con 0x562e5bd5ef90 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.623+0000 7f678e7fc700 10 client.10164 handle_command_reply: tid=0 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.623+0000 7f67c7eb6b80 1 client.10164 shutdown 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.624+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/2797160475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 msgr2=0x562e5bd3e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.624+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bd3e3b0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 rx=0x7f67b005f410 tx=0x7f67a401c000).stop 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.624+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/2797160475 shutdown_connections 2024-01-24T15:32:26.629 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.624+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.163:6835/532321390,v1:172.21.15.163:6837/532321390] conn(0x562e5bd5ef90 0x562e5bd5f370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.624+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bd57fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.624+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd0edf0 0x562e5bd3b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.624+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/2797160475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bd3e3b0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.624+0000 7f678e7fc700 0 client.10164 ms_handle_reset on v2:172.21.15.163:6835/532321390 2024-01-24T15:32:26.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.624+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/2797160475 >> 172.21.15.29:0/2797160475 conn(0x562e5bb7c2e0 msgr2=0x562e5bd13ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:26.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.624+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/2797160475 shutdown_connections 2024-01-24T15:32:26.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.624+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/2797160475 wait complete. 2024-01-24T15:32:26.630 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.624+0000 7f67c7eb6b80 20 client.10164 trim_cache size 0 max 16384 2024-01-24T15:32:26.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.641+0000 7f67c7eb6b80 1 Processor -- start 2024-01-24T15:32:26.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.641+0000 7f67c7eb6b80 1 -- start start 2024-01-24T15:32:26.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.642+0000 7f67c7eb6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd0edf0 0x562e5bd11510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.642+0000 7f67c7eb6b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd13620 0x562e5bd57f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.642+0000 7f67c7eb6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bd41d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.642+0000 7f67c7eb6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5bd3ed90 con 0x562e5bd0edf0 2024-01-24T15:32:26.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.642+0000 7f67c7eb6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5bd132b0 con 0x562e5bd5c2c0 2024-01-24T15:32:26.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.642+0000 7f67c7eb6b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e5bd41aa0 con 0x562e5bd13620 2024-01-24T15:32:26.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.642+0000 7f67affff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd0edf0 0x562e5bd11510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.642+0000 7f67affff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd0edf0 0x562e5bd11510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55256/0 (socket says 172.21.15.29:55256) 2024-01-24T15:32:26.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.642+0000 7f67affff700 1 -- 172.21.15.29:0/571143295 learned_addr learned my addr 172.21.15.29:0/571143295 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.643+0000 7f67b4a88700 1 --2- 172.21.15.29:0/571143295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bd41d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.643+0000 7f67af7fe700 1 --2- 172.21.15.29:0/571143295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd13620 0x562e5bd57f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.643+0000 7f67affff700 1 -- 172.21.15.29:0/571143295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd13620 msgr2=0x562e5bd57f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.643+0000 7f67affff700 1 --2- 172.21.15.29:0/571143295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd13620 0x562e5bd57f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.643+0000 7f67affff700 1 -- 172.21.15.29:0/571143295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 msgr2=0x562e5bd41d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.643+0000 7f67affff700 1 --2- 172.21.15.29:0/571143295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bd41d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.643+0000 7f67affff700 1 -- 172.21.15.29:0/571143295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e5bd10490 con 0x562e5bd0edf0 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.643+0000 7f67b4a88700 1 --2- 172.21.15.29:0/571143295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bd41d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.643+0000 7f67affff700 1 --2- 172.21.15.29:0/571143295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd0edf0 0x562e5bd11510 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 rx=0x7f67a000cc30 tx=0x7f67a0051000).ready entity=mon.0 client_cookie=d604b8d447f596f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.643+0000 7f6795ffb700 1 -- 172.21.15.29:0/571143295 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0063030 con 0x562e5bd0edf0 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.643+0000 7f6795ffb700 1 -- 172.21.15.29:0/571143295 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a0052040 con 0x562e5bd0edf0 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.643+0000 7f6795ffb700 1 -- 172.21.15.29:0/571143295 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0058040 con 0x562e5bd0edf0 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.644+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/571143295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd0edf0 msgr2=0x562e5bd11510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.644+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/571143295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd0edf0 0x562e5bd11510 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 rx=0x7f67a000cc30 tx=0x7f67a0051000).stop 2024-01-24T15:32:26.646 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.644+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/571143295 shutdown_connections 2024-01-24T15:32:26.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.644+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/571143295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bd41d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.644+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/571143295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd0edf0 0x562e5bd11510 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.644+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/571143295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd13620 0x562e5bd57f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.644+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/571143295 >> 172.21.15.29:0/571143295 conn(0x562e5bb7c2e0 msgr2=0x562e5bd5d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:26.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.644+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/571143295 shutdown_connections 2024-01-24T15:32:26.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.644+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/571143295 wait complete. 2024-01-24T15:32:26.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.644+0000 7f67c7eb6b80 1 -- start start 2024-01-24T15:32:26.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.644+0000 7f67c7eb6b80 1 Processor -- start 2024-01-24T15:32:26.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.645+0000 7f67c7eb6b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd0edf0 0x562e5bd59780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.645+0000 7f67c7eb6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bdbce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.645+0000 7f67c7eb6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bdbd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.645+0000 7f67c7eb6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5bd6a420 con 0x562e5bd13620 2024-01-24T15:32:26.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.645+0000 7f67c7eb6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5bd14470 con 0x562e5bd5c2c0 2024-01-24T15:32:26.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.645+0000 7f67c7eb6b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e5bdc1e70 con 0x562e5bd0edf0 2024-01-24T15:32:26.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.645+0000 7f67c7eb6b80 10 client.0 ll_register_callbacks cb 0x562e5bcd3610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.645+0000 7f67af7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bdbce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.645+0000 7f67af7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bdbce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55270/0 (socket says 172.21.15.29:55270) 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f67af7fe700 1 -- 172.21.15.29:0/1845833933 learned_addr learned my addr 172.21.15.29:0/1845833933 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f67b4a88700 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bdbd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f67affff700 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd0edf0 0x562e5bd59780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f67af7fe700 1 -- 172.21.15.29:0/1845833933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd0edf0 msgr2=0x562e5bd59780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f67af7fe700 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd0edf0 0x562e5bd59780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f67af7fe700 1 -- 172.21.15.29:0/1845833933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 msgr2=0x562e5bdbd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f67af7fe700 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bdbd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f67af7fe700 1 -- 172.21.15.29:0/1845833933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e5bc5e2f0 con 0x562e5bd13620 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f67af7fe700 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bdbce80 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 rx=0x7f67a4011fd0 tx=0x7f67a4027070).ready entity=mon.0 client_cookie=d8de3d9869c64686 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f678e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f678e7fc700 1 -- 172.21.15.29:0/1845833933 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a401b590 con 0x562e5bd13620 2024-01-24T15:32:26.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f678e7fc700 1 -- 172.21.15.29:0/1845833933 <== mon.0 v2:172.21.15.29:3300/0 2 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f67a402e470 con 0x562e5bd13620 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/1845833933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67a004cc60 con 0x562e5bd13620 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f678e7fc700 1 -- 172.21.15.29:0/1845833933 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67a401b6f0 con 0x562e5bd13620 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f67c7eb6b80 10 client.10170 fetch_fsmap learned FSMap version 98 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/1845833933 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) v3 -- 0x562e5bc59270 con 0x562e5bd13620 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.646+0000 7f678e7fc700 1 -- 172.21.15.29:0/1845833933 <== mon.0 v2:172.21.15.29:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a004cc60 con 0x562e5bd13620 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f678e7fc700 1 -- 172.21.15.29:0/1845833933 <== mon.0 v2:172.21.15.29:3300/0 5 ==== fsmap(e 98) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f67a402e840 con 0x562e5bd13620 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f67c7eb6b80 10 client.10170 fetch_fsmap finished waiting for FSMap version 98 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f67c7eb6b80 10 client.10170 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f67c7eb6b80 20 client.10170 populate_metadata read hostname 'smithi029' 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.163:6835/532321390,v1:172.21.15.163:6837/532321390] conn(0x562e5bd3dc10 0x562e5bd3dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f67c7eb6b80 4 client.10170 mds_command: new command op to 10029 tid=0[{"prefix": "status"}] 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/1845833933 --> [v2:172.21.15.163:6835/532321390,v1:172.21.15.163:6837/532321390] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562e5bc517d0 con 0x562e5bd3dc10 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f67affff700 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.163:6835/532321390,v1:172.21.15.163:6837/532321390] conn(0x562e5bd3dc10 0x562e5bd3dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f67affff700 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.163:6835/532321390,v1:172.21.15.163:6837/532321390] conn(0x562e5bd3dc10 0x562e5bd3dff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=2b63f8c23331dde2 server_cookie=321c0559d2d750b7 in_seq=0 out_seq=0 2024-01-24T15:32:26.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f678e7fc700 10 client.10170 ms_handle_connect on v2:172.21.15.163:6835/532321390 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f678e7fc700 1 -- 172.21.15.29:0/1845833933 <== mds.0 v2:172.21.15.163:6835/532321390 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x562e5bc517d0 con 0x562e5bd3dc10 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f678e7fc700 10 client.10170 handle_command_reply: tid=0 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.647+0000 7f67c7eb6b80 1 client.10170 shutdown 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/1845833933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 msgr2=0x562e5bdbce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bdbce80 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 rx=0x7f67a4011fd0 tx=0x7f67a4027070).stop 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/1845833933 shutdown_connections 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.163:6835/532321390,v1:172.21.15.163:6837/532321390] conn(0x562e5bd3dc10 0x562e5bd3dff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e5bd5c2c0 0x562e5bdbd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f678e7fc700 0 client.10170 ms_handle_reset on v2:172.21.15.163:6835/532321390 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562e5bd0edf0 0x562e5bd59780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67c7eb6b80 1 --2- 172.21.15.29:0/1845833933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e5bd13620 0x562e5bdbce80 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/1845833933 >> 172.21.15.29:0/1845833933 conn(0x562e5bb7c2e0 msgr2=0x562e5bd0a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/1845833933 shutdown_connections 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67c7eb6b80 1 -- 172.21.15.29:0/1845833933 wait complete. 2024-01-24T15:32:26.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67c7eb6b80 20 client.10170 trim_cache size 0 max 16384 2024-01-24T15:32:26.655 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:32:26.655 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_fsid": "b1c88d29-1963-443e-8a67-63a822ae75dd", 2024-01-24T15:32:26.655 INFO:teuthology.orchestra.run.smithi029.stdout: "whoami": 0, 2024-01-24T15:32:26.655 INFO:teuthology.orchestra.run.smithi029.stdout: "id": 10029, 2024-01-24T15:32:26.655 INFO:teuthology.orchestra.run.smithi029.stdout: "want_state": "up:active", 2024-01-24T15:32:26.655 INFO:teuthology.orchestra.run.smithi029.stdout: "state": "up:active", 2024-01-24T15:32:26.655 INFO:teuthology.orchestra.run.smithi029.stdout: "fs_name": "cephfs", 2024-01-24T15:32:26.655 INFO:teuthology.orchestra.run.smithi029.stdout: "rank_uptime": 5.9605844399999999, 2024-01-24T15:32:26.655 INFO:teuthology.orchestra.run.smithi029.stdout: "mdsmap_epoch": 98, 2024-01-24T15:32:26.655 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch": 231, 2024-01-24T15:32:26.655 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_epoch_barrier": 231, 2024-01-24T15:32:26.656 INFO:teuthology.orchestra.run.smithi029.stdout: "uptime": 11.450082333999999 2024-01-24T15:32:26.656 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:32:26.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67b64eb700 1 -- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f679c05f240 msgr2=0x7f679c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67b64eb700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f679c05f240 0x7f679c061700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f67b006aea0 tx=0x7f67a4017000).stop 2024-01-24T15:32:26.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67b64eb700 1 -- 172.21.15.29:0/2234895260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67b0063ec0 msgr2=0x7f67b00aa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67b64eb700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67b0063ec0 0x7f67b00aa1e0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 rx=0x7f67a000c4e0 tx=0x7f67a0007c70).stop 2024-01-24T15:32:26.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67b64eb700 1 -- 172.21.15.29:0/2234895260 shutdown_connections 2024-01-24T15:32:26.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67b64eb700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67b013e220 0x7f67b01489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67b64eb700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67b0063ec0 0x7f67b00aa1e0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67b64eb700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f679c05f240 0x7f679c061700 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67b64eb700 1 --2- 172.21.15.29:0/2234895260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67b013de40 0x7f67b00c1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.648+0000 7f67b64eb700 1 -- 172.21.15.29:0/2234895260 >> 172.21.15.29:0/2234895260 conn(0x7f67b00c3cd0 msgr2=0x7f67b013a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:26.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.649+0000 7f67b64eb700 1 -- 172.21.15.29:0/2234895260 shutdown_connections 2024-01-24T15:32:26.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.649+0000 7f67b64eb700 1 -- 172.21.15.29:0/2234895260 wait complete. 2024-01-24T15:32:26.663 DEBUG:teuthology.orchestra.run.smithi029:> ceph fs ls 2024-01-24T15:32:26.778 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.777+0000 7f5ac0970700 1 Processor -- start 2024-01-24T15:32:26.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.777+0000 7f5ac0970700 1 -- start start 2024-01-24T15:32:26.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.778+0000 7f5ac0970700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5abc063980 0x7f5abc063d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.778+0000 7f5ac0970700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5abc066ec0 0x7f5abc067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.778+0000 7f5ac0970700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5abc0c1ad0 con 0x7f5abc0c1f60 2024-01-24T15:32:26.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.778+0000 7f5ac0970700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5abc069fa0 con 0x7f5abc063980 2024-01-24T15:32:26.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.778+0000 7f5ac0970700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc069700 con 0x7f5abc066ec0 2024-01-24T15:32:26.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.779+0000 7f5aba59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5abc063980 0x7f5abc063d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.779+0000 7f5abad9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5abc0c1f60 0x7f5abc0a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54234/0 (socket says 172.21.15.29:54234) 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.779+0000 7f5abad9d700 1 -- 172.21.15.29:0/1912495908 learned_addr learned my addr 172.21.15.29:0/1912495908 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.779+0000 7f5ab9d9b700 1 --2- 172.21.15.29:0/1912495908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5abc066ec0 0x7f5abc067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.779+0000 7f5aba59c700 1 -- 172.21.15.29:0/1912495908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5abc066ec0 msgr2=0x7f5abc067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.779+0000 7f5aba59c700 1 --2- 172.21.15.29:0/1912495908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5abc066ec0 0x7f5abc067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.779+0000 7f5aba59c700 1 -- 172.21.15.29:0/1912495908 >> v1:172.21.15.29:6789/0 conn(0x7f5abc0c1f60 legacy=0x7f5abc0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.779+0000 7f5aba59c700 1 -- 172.21.15.29:0/1912495908 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0699e0 con 0x7f5abc063980 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.779+0000 7f5aba59c700 1 --2- 172.21.15.29:0/1912495908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5abc063980 0x7f5abc063d60 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 rx=0x7f5aa4009760 tx=0x7f5aa4009060).ready entity=mon.0 client_cookie=543084042995921c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.779+0000 7f5ab959a700 1 -- 172.21.15.29:0/1912495908 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400f510 con 0x7f5abc063980 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.779+0000 7f5ab959a700 1 -- 172.21.15.29:0/1912495908 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa400f670 con 0x7f5abc063980 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.779+0000 7f5ab959a700 1 -- 172.21.15.29:0/1912495908 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400f9f0 con 0x7f5abc063980 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.780+0000 7f5ac0970700 1 -- 172.21.15.29:0/1912495908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5abc063980 msgr2=0x7f5abc063d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.780+0000 7f5ac0970700 1 --2- 172.21.15.29:0/1912495908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5abc063980 0x7f5abc063d60 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 rx=0x7f5aa4009760 tx=0x7f5aa4009060).stop 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.780+0000 7f5ac0970700 1 -- 172.21.15.29:0/1912495908 shutdown_connections 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.780+0000 7f5ac0970700 1 --2- 172.21.15.29:0/1912495908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5abc063980 0x7f5abc063d60 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.780+0000 7f5ac0970700 1 --2- 172.21.15.29:0/1912495908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5abc066ec0 0x7f5abc067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.780+0000 7f5ac0970700 1 -- 172.21.15.29:0/1912495908 >> 172.21.15.29:0/1912495908 conn(0x7f5abc0c4810 msgr2=0x7f5abc068c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:26.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.780+0000 7f5ac0970700 1 -- 172.21.15.29:0/1912495908 shutdown_connections 2024-01-24T15:32:26.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.780+0000 7f5ac0970700 1 -- 172.21.15.29:0/1912495908 wait complete. 2024-01-24T15:32:26.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.781+0000 7f5ac0970700 1 Processor -- start 2024-01-24T15:32:26.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.781+0000 7f5ac0970700 1 -- start start 2024-01-24T15:32:26.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.782+0000 7f5ac0970700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5abc063980 0x7f5abc168700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.782+0000 7f5ac0970700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5abc066ec0 0x7f5abc14e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.782+0000 7f5ac0970700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5abc0c1f60 0x7f5abc0c0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.782+0000 7f5ac0970700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0a6610 con 0x7f5abc0c1f60 2024-01-24T15:32:26.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.782+0000 7f5ac0970700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc153f20 con 0x7f5abc063980 2024-01-24T15:32:26.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.782+0000 7f5ac0970700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc154200 con 0x7f5abc066ec0 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.782+0000 7f5abad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5abc0c1f60 0x7f5abc0c0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.782+0000 7f5abad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5abc0c1f60 0x7f5abc0c0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55280/0 (socket says 172.21.15.29:55280) 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.782+0000 7f5abad9d700 1 -- 172.21.15.29:0/2539502743 learned_addr learned my addr 172.21.15.29:0/2539502743 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.782+0000 7f5ab9d9b700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5abc066ec0 0x7f5abc14e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.782+0000 7f5aba59c700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5abc063980 0x7f5abc168700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.783+0000 7f5abad9d700 1 -- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5abc066ec0 msgr2=0x7f5abc14e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.783+0000 7f5abad9d700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5abc066ec0 0x7f5abc14e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.783+0000 7f5abad9d700 1 -- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5abc063980 msgr2=0x7f5abc168700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.783+0000 7f5abad9d700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5abc063980 0x7f5abc168700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.783+0000 7f5abad9d700 1 -- 172.21.15.29:0/2539502743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa40092a0 con 0x7f5abc0c1f60 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.783+0000 7f5abad9d700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5abc0c1f60 0x7f5abc0c0740 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 rx=0x7f5ab400c7c0 tx=0x7f5ab4001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.783+0000 7f5ab27fc700 1 -- 172.21.15.29:0/2539502743 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4005d30 con 0x7f5abc0c1f60 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.783+0000 7f5ab27fc700 1 -- 172.21.15.29:0/2539502743 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab4005e90 con 0x7f5abc0c1f60 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.783+0000 7f5ac0970700 1 -- 172.21.15.29:0/2539502743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc1544e0 con 0x7f5abc0c1f60 2024-01-24T15:32:26.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.783+0000 7f5ab27fc700 1 -- 172.21.15.29:0/2539502743 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40072e0 con 0x7f5abc0c1f60 2024-01-24T15:32:26.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.783+0000 7f5ac0970700 1 -- 172.21.15.29:0/2539502743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc168bf0 con 0x7f5abc0c1f60 2024-01-24T15:32:26.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.784+0000 7f5ac0970700 1 -- 172.21.15.29:0/2539502743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc154a00 con 0x7f5abc0c1f60 2024-01-24T15:32:26.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.784+0000 7f5ab27fc700 1 -- 172.21.15.29:0/2539502743 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5ab4007750 con 0x7f5abc0c1f60 2024-01-24T15:32:26.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.785+0000 7f5ab27fc700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5a9c05f270 0x7f5a9c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:26.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.785+0000 7f5ab27fc700 1 -- 172.21.15.29:0/2539502743 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f5ab407a190 con 0x7f5abc0c1f60 2024-01-24T15:32:26.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.785+0000 7f5aba59c700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5a9c05f270 0x7f5a9c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:26.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.786+0000 7f5aba59c700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5a9c05f270 0x7f5a9c061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f5aa4009fd0 tx=0x7f5aa4017000).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:26.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:26.789+0000 7f5ab27fc700 1 -- 172.21.15.29:0/2539502743 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f5abc154a00 con 0x7f5abc0c1f60 2024-01-24T15:32:27.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.019+0000 7f5ac0970700 1 -- 172.21.15.29:0/2539502743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5abc154a00 con 0x7f5abc0c1f60 2024-01-24T15:32:27.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.020+0000 7f5ab27fc700 1 -- 172.21.15.29:0/2539502743 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5abc154a00 con 0x7f5abc0c1f60 2024-01-24T15:32:27.021 INFO:teuthology.orchestra.run.smithi029.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-24T15:32:27.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.022+0000 7f5ac0970700 1 -- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5a9c05f270 msgr2=0x7f5a9c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:27.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.022+0000 7f5ac0970700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5a9c05f270 0x7f5a9c061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f5aa4009fd0 tx=0x7f5aa4017000).stop 2024-01-24T15:32:27.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.022+0000 7f5ac0970700 1 -- 172.21.15.29:0/2539502743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5abc0c1f60 msgr2=0x7f5abc0c0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:27.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.022+0000 7f5ac0970700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5abc0c1f60 0x7f5abc0c0740 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 rx=0x7f5ab400c7c0 tx=0x7f5ab4001510).stop 2024-01-24T15:32:27.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.022+0000 7f5ac0970700 1 -- 172.21.15.29:0/2539502743 shutdown_connections 2024-01-24T15:32:27.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.022+0000 7f5ac0970700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5abc0c1f60 0x7f5abc0c0740 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:27.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.022+0000 7f5ac0970700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5abc063980 0x7f5abc168700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:27.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.022+0000 7f5ac0970700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f5a9c05f270 0x7f5a9c061730 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:27.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.022+0000 7f5ac0970700 1 --2- 172.21.15.29:0/2539502743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5abc066ec0 0x7f5abc14e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:27.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.022+0000 7f5ac0970700 1 -- 172.21.15.29:0/2539502743 >> 172.21.15.29:0/2539502743 conn(0x7f5abc0c4810 msgr2=0x7f5abc14edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:27.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.022+0000 7f5ac0970700 1 -- 172.21.15.29:0/2539502743 shutdown_connections 2024-01-24T15:32:27.024 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.022+0000 7f5ac0970700 1 -- 172.21.15.29:0/2539502743 wait complete. 2024-01-24T15:32:27.035 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-24T15:32:27.035 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-24T15:32:27.036 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-01-24T15:32:27.036 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-24T15:32:27.036 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-24T15:32:27.036 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-24T15:32:27.036 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-24T15:32:27.036 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-24T15:32:27.036 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T15:32:27.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:27.036 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-01-24T15:32:27.056 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-24T15:32:27.056 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-24T15:32:27.056 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-01-24T15:32:27.056 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:32:27.056 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-01-24T15:32:27.056 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:32:27.056 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T15:32:27.056 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-01-24T15:32:27.056 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-01-24T15:32:27.056 INFO:teuthology.orchestra.run.smithi029.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 36704sec preferred_lft 36704sec 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff 2024-01-24T15:32:27.057 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-24T15:32:27.058 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:32:27.058 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::24bc:1cff:fe57:e704/64 scope link 2024-01-24T15:32:27.058 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:32:27.058 INFO:teuthology.orchestra.run.smithi029.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T15:32:27.058 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 22:34:aa:f3:f1:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T15:32:27.058 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::2034:aaff:fef3:f12b/64 scope link 2024-01-24T15:32:27.058 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:32:27.058 INFO:teuthology.orchestra.run.smithi029.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-24T15:32:27.058 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether ba:ca:42:4a:9e:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T15:32:27.058 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::b8ca:42ff:fe4a:9e52/64 scope link 2024-01-24T15:32:27.058 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-01-24T15:32:27.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:27.058 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-24T15:32:27.126 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-01-24T15:32:27.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:27.128 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-01-24T15:32:27.193 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-01-24T15:32:27.193 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-24T15:32:27.193 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-24T15:32:27.193 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-24T15:32:27.193 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-24T15:32:27.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:27.194 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec bash -c ' 2024-01-24T15:32:27.194 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-01-24T15:32:27.194 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-24T15:32:27.194 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-24T15:32:27.194 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-24T15:32:27.194 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-01-24T15:32:27.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:27.318 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T15:32:27.338 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-24T15:32:27.338 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T15:32:27.338 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-24T15:32:27.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:27.339 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-01-24T15:32:27.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:27.415 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-01-24T15:32:27.477 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-24T15:32:27.477 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T15:32:27.569 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T15:32:27.569 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T15:32:27.592 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-01-24T15:32:27.592 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-01-24T15:32:27.595 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-24T15:32:27.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:27.595 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-24T15:32:27.628 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-24T15:32:27.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.805+0000 7f0881ebf700 1 Processor -- start 2024-01-24T15:32:27.808 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.806+0000 7f0881ebf700 1 -- start start 2024-01-24T15:32:27.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.806+0000 7f0881ebf700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f087c069240 0x7f087c150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:27.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.806+0000 7f0881ebf700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f087c1514a0 0x7f087c151950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:27.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.806+0000 7f0881ebf700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c06a350 con 0x7f087c150bd0 2024-01-24T15:32:27.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.806+0000 7f0881ebf700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c069740 con 0x7f087c069240 2024-01-24T15:32:27.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.806+0000 7f0881ebf700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c069a20 con 0x7f087c1514a0 2024-01-24T15:32:27.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f087c069240 0x7f087c150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:27.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f087c069240 0x7f087c150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55286/0 (socket says 172.21.15.29:55286) 2024-01-24T15:32:27.819 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087affd700 1 -- 172.21.15.29:0/3181605051 learned_addr learned my addr 172.21.15.29:0/3181605051 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:27.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087b7fe700 1 --1- 172.21.15.29:0/3181605051 >> v1:172.21.15.29:6789/0 conn(0x7f087c150bd0 0x7f087c065e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54244/0 (socket says 172.21.15.29:54244) 2024-01-24T15:32:27.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087bfff700 1 --2- 172.21.15.29:0/3181605051 >> v2:172.21.15.163:3300/0 conn(0x7f087c1514a0 0x7f087c151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:27.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087affd700 1 -- 172.21.15.29:0/3181605051 >> v2:172.21.15.163:3300/0 conn(0x7f087c1514a0 msgr2=0x7f087c151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:27.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087affd700 1 --2- 172.21.15.29:0/3181605051 >> v2:172.21.15.163:3300/0 conn(0x7f087c1514a0 0x7f087c151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:27.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087affd700 1 -- 172.21.15.29:0/3181605051 >> v1:172.21.15.29:6789/0 conn(0x7f087c150bd0 legacy=0x7f087c065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:27.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087affd700 1 -- 172.21.15.29:0/3181605051 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c069d00 con 0x7f087c069240 2024-01-24T15:32:27.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087a7fc700 1 -- 172.21.15.29:0/3181605051 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969075343 0 0) 0x7f087c06a350 con 0x7f087c150bd0 2024-01-24T15:32:27.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087affd700 1 --2- 172.21.15.29:0/3181605051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c069240 0x7f087c150fb0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 rx=0x7f087c0be950 tx=0x7f08700090f0).ready entity=mon.0 client_cookie=dbf0db79d3f2515c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:27.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087a7fc700 1 -- 172.21.15.29:0/3181605051 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000b600 con 0x7f087c069240 2024-01-24T15:32:27.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087a7fc700 1 -- 172.21.15.29:0/3181605051 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000b760 con 0x7f087c069240 2024-01-24T15:32:27.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.807+0000 7f087a7fc700 1 -- 172.21.15.29:0/3181605051 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000bae0 con 0x7f087c069240 2024-01-24T15:32:27.821 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.808+0000 7f0881ebf700 1 -- 172.21.15.29:0/3181605051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c069240 msgr2=0x7f087c150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:27.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.808+0000 7f0881ebf700 1 --2- 172.21.15.29:0/3181605051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c069240 0x7f087c150fb0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 rx=0x7f087c0be950 tx=0x7f08700090f0).stop 2024-01-24T15:32:27.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.808+0000 7f0881ebf700 1 -- 172.21.15.29:0/3181605051 shutdown_connections 2024-01-24T15:32:27.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.808+0000 7f0881ebf700 1 --2- 172.21.15.29:0/3181605051 >> v2:172.21.15.163:3300/0 conn(0x7f087c1514a0 0x7f087c151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:27.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.808+0000 7f0881ebf700 1 --2- 172.21.15.29:0/3181605051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c069240 0x7f087c150fb0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:27.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.808+0000 7f0881ebf700 1 -- 172.21.15.29:0/3181605051 >> 172.21.15.29:0/3181605051 conn(0x7f087c0c8580 msgr2=0x7f087c0c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:27.822 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.808+0000 7f0881ebf700 1 -- 172.21.15.29:0/3181605051 shutdown_connections 2024-01-24T15:32:27.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.808+0000 7f0881ebf700 1 -- 172.21.15.29:0/3181605051 wait complete. 2024-01-24T15:32:27.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.809+0000 7f0881ebf700 1 Processor -- start 2024-01-24T15:32:27.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.809+0000 7f0881ebf700 1 -- start start 2024-01-24T15:32:27.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.810+0000 7f0881ebf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f087c069240 0x7f087c153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:27.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.810+0000 7f0881ebf700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c150bd0 0x7f087c0ab470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:27.823 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.810+0000 7f0881ebf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1514a0 0x7f087c0bced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:27.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.810+0000 7f0881ebf700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c165240 con 0x7f087c1514a0 2024-01-24T15:32:27.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.810+0000 7f0881ebf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c164910 con 0x7f087c069240 2024-01-24T15:32:27.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.810+0000 7f0881ebf700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c164bf0 con 0x7f087c150bd0 2024-01-24T15:32:27.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.810+0000 7f087bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1514a0 0x7f087c0bced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:27.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.810+0000 7f087bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1514a0 0x7f087c0bced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55292/0 (socket says 172.21.15.29:55292) 2024-01-24T15:32:27.824 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.810+0000 7f087bfff700 1 -- 172.21.15.29:0/171209450 learned_addr learned my addr 172.21.15.29:0/171209450 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:27.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.810+0000 7f087affd700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c150bd0 0x7f087c0ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:27.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.810+0000 7f087b7fe700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f087c069240 0x7f087c153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:27.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.811+0000 7f087bfff700 1 -- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c150bd0 msgr2=0x7f087c0ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:27.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.811+0000 7f087bfff700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c150bd0 0x7f087c0ab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:27.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.811+0000 7f087bfff700 1 -- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f087c069240 msgr2=0x7f087c153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:27.825 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.811+0000 7f087bfff700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f087c069240 0x7f087c153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:27.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.811+0000 7f087bfff700 1 -- 172.21.15.29:0/171209450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0870009370 con 0x7f087c1514a0 2024-01-24T15:32:27.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.811+0000 7f087bfff700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1514a0 0x7f087c0bced0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 rx=0x7f0874007f20 tx=0x7f0874009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:27.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.811+0000 7f08637fe700 1 -- 172.21.15.29:0/171209450 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087400eb90 con 0x7f087c1514a0 2024-01-24T15:32:27.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.811+0000 7f08637fe700 1 -- 172.21.15.29:0/171209450 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087400ecf0 con 0x7f087c1514a0 2024-01-24T15:32:27.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.811+0000 7f0881ebf700 1 -- 172.21.15.29:0/171209450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c164ed0 con 0x7f087c1514a0 2024-01-24T15:32:27.826 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.811+0000 7f08637fe700 1 -- 172.21.15.29:0/171209450 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874014610 con 0x7f087c1514a0 2024-01-24T15:32:27.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.811+0000 7f0881ebf700 1 -- 172.21.15.29:0/171209450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c153820 con 0x7f087c1514a0 2024-01-24T15:32:27.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.812+0000 7f0881ebf700 1 -- 172.21.15.29:0/171209450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c1653b0 con 0x7f087c1514a0 2024-01-24T15:32:27.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.812+0000 7f08637fe700 1 -- 172.21.15.29:0/171209450 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0874014980 con 0x7f087c1514a0 2024-01-24T15:32:27.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.813+0000 7f08637fe700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f086805f270 0x7f0868061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:27.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.813+0000 7f08637fe700 1 -- 172.21.15.29:0/171209450 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7f087407ac00 con 0x7f087c1514a0 2024-01-24T15:32:27.827 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.813+0000 7f087b7fe700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f086805f270 0x7f0868061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:27.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.813+0000 7f087b7fe700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f086805f270 0x7f0868061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f086c0011e0 tx=0x7f086c007bd0).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:27.828 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:27.817+0000 7f08637fe700 1 -- 172.21.15.29:0/171209450 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f087404b910 con 0x7f087c1514a0 2024-01-24T15:32:28.030 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.028+0000 7f0881ebf700 1 -- 172.21.15.29:0/171209450 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f087c0c4cd0 con 0x7f086805f270 2024-01-24T15:32:28.032 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.030+0000 7f08637fe700 1 -- 172.21.15.29:0/171209450 <== mgr.9352 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f087c0c4cd0 con 0x7f086805f270 2024-01-24T15:32:28.032 INFO:teuthology.orchestra.run.smithi029.stdout:[ 2024-01-24T15:32:28.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-01-24T15:32:28.033 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "cephfs" 2024-01-24T15:32:28.033 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-01-24T15:32:28.033 INFO:teuthology.orchestra.run.smithi029.stdout:] 2024-01-24T15:32:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.032+0000 7f0881ebf700 1 -- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f086805f270 msgr2=0x7f0868061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.032+0000 7f0881ebf700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f086805f270 0x7f0868061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f086c0011e0 tx=0x7f086c007bd0).stop 2024-01-24T15:32:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.032+0000 7f0881ebf700 1 -- 172.21.15.29:0/171209450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1514a0 msgr2=0x7f087c0bced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.032+0000 7f0881ebf700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1514a0 0x7f087c0bced0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 rx=0x7f0874007f20 tx=0x7f0874009860).stop 2024-01-24T15:32:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.032+0000 7f0881ebf700 1 -- 172.21.15.29:0/171209450 shutdown_connections 2024-01-24T15:32:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.032+0000 7f0881ebf700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f087c1514a0 0x7f087c0bced0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.032+0000 7f0881ebf700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f087c069240 0x7f087c153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.035 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.032+0000 7f0881ebf700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f086805f270 0x7f0868061730 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.032+0000 7f0881ebf700 1 --2- 172.21.15.29:0/171209450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f087c150bd0 0x7f087c0ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.032+0000 7f0881ebf700 1 -- 172.21.15.29:0/171209450 >> 172.21.15.29:0/171209450 conn(0x7f087c0c8580 msgr2=0x7f087c153b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.032+0000 7f0881ebf700 1 -- 172.21.15.29:0/171209450 shutdown_connections 2024-01-24T15:32:28.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.032+0000 7f0881ebf700 1 -- 172.21.15.29:0/171209450 wait complete. 2024-01-24T15:32:28.048 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-01-24T15:32:28.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.179+0000 7fd3769db700 1 Processor -- start 2024-01-24T15:32:28.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.179+0000 7fd3769db700 1 -- start start 2024-01-24T15:32:28.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd3769db700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd37014b330 0x7fd3701549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:28.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd3769db700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd3700c3360 0x7fd370154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:28.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd3769db700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd370155840 con 0x7fd37014b710 2024-01-24T15:32:28.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd3769db700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd370155390 con 0x7fd3700c3360 2024-01-24T15:32:28.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd3769db700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd370069280 con 0x7fd37014b330 2024-01-24T15:32:28.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd374f78700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd3700c3360 0x7fd370154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:28.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd36f7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd37014b710 0x7fd3700c2c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54254/0 (socket says 172.21.15.29:54254) 2024-01-24T15:32:28.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd36f7fe700 1 -- 172.21.15.29:0/2176618115 learned_addr learned my addr 172.21.15.29:0/2176618115 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:28.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd36ffff700 1 --2- 172.21.15.29:0/2176618115 >> v2:172.21.15.163:3300/0 conn(0x7fd37014b330 0x7fd3701549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:28.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd374f78700 1 -- 172.21.15.29:0/2176618115 >> v2:172.21.15.163:3300/0 conn(0x7fd37014b330 msgr2=0x7fd3701549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:28.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd374f78700 1 --2- 172.21.15.29:0/2176618115 >> v2:172.21.15.163:3300/0 conn(0x7fd37014b330 0x7fd3701549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd374f78700 1 -- 172.21.15.29:0/2176618115 >> v1:172.21.15.29:6789/0 conn(0x7fd37014b710 legacy=0x7fd3700c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd374f78700 1 -- 172.21.15.29:0/2176618115 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3700c3910 con 0x7fd3700c3360 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd36effd700 1 -- 172.21.15.29:0/2176618115 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206599846 0 0) 0x7fd370155840 con 0x7fd37014b710 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd374f78700 1 --2- 172.21.15.29:0/2176618115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3700c3360 0x7fd370154ea0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 rx=0x7fd37006c670 tx=0x7fd368009780).ready entity=mon.0 client_cookie=c3c484682692d75c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.180+0000 7fd36effd700 1 -- 172.21.15.29:0/2176618115 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36800e070 con 0x7fd3700c3360 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.181+0000 7fd36effd700 1 -- 172.21.15.29:0/2176618115 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36800bc30 con 0x7fd3700c3360 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.181+0000 7fd36effd700 1 -- 172.21.15.29:0/2176618115 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368012410 con 0x7fd3700c3360 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.181+0000 7fd3769db700 1 -- 172.21.15.29:0/2176618115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3700c3360 msgr2=0x7fd370154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.181+0000 7fd3769db700 1 --2- 172.21.15.29:0/2176618115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3700c3360 0x7fd370154ea0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 rx=0x7fd37006c670 tx=0x7fd368009780).stop 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.181+0000 7fd3769db700 1 -- 172.21.15.29:0/2176618115 shutdown_connections 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.181+0000 7fd3769db700 1 --2- 172.21.15.29:0/2176618115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3700c3360 0x7fd370154ea0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.181+0000 7fd3769db700 1 --2- 172.21.15.29:0/2176618115 >> v2:172.21.15.163:3300/0 conn(0x7fd37014b330 0x7fd3701549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.181+0000 7fd3769db700 1 -- 172.21.15.29:0/2176618115 >> 172.21.15.29:0/2176618115 conn(0x7fd370064390 msgr2=0x7fd3700c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.181+0000 7fd3769db700 1 -- 172.21.15.29:0/2176618115 shutdown_connections 2024-01-24T15:32:28.185 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.181+0000 7fd3769db700 1 -- 172.21.15.29:0/2176618115 wait complete. 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.183+0000 7fd3769db700 1 Processor -- start 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.183+0000 7fd3769db700 1 -- start start 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd3769db700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3700c3360 0x7fd3700c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd3769db700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd37014b330 0x7fd370148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd3769db700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd37014b710 0x7fd370150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd3769db700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700aad60 con 0x7fd37014b330 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd3769db700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700aaef0 con 0x7fd3700c3360 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd3769db700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700c6dc0 con 0x7fd37014b710 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd36f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd37014b330 0x7fd370148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd36f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd37014b330 0x7fd370148130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55306/0 (socket says 172.21.15.29:55306) 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd36f7fe700 1 -- 172.21.15.29:0/3797743961 learned_addr learned my addr 172.21.15.29:0/3797743961 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd36ffff700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3700c3360 0x7fd3700c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:28.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd374f78700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd37014b710 0x7fd370150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd36f7fe700 1 -- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd37014b710 msgr2=0x7fd370150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd36f7fe700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd37014b710 0x7fd370150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd36f7fe700 1 -- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3700c3360 msgr2=0x7fd3700c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd36f7fe700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3700c3360 0x7fd3700c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd36f7fe700 1 -- 172.21.15.29:0/3797743961 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd368009a20 con 0x7fd37014b330 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd36ffff700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3700c3360 0x7fd3700c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd374f78700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd37014b710 0x7fd370150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd36f7fe700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd37014b330 0x7fd370148130 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 rx=0x7fd36400c490 tx=0x7fd364007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd357fff700 1 -- 172.21.15.29:0/3797743961 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3640072e0 con 0x7fd37014b330 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd357fff700 1 -- 172.21.15.29:0/3797743961 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd364007440 con 0x7fd37014b330 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd3769db700 1 -- 172.21.15.29:0/3797743961 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3700c70a0 con 0x7fd37014b330 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd357fff700 1 -- 172.21.15.29:0/3797743961 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3640057a0 con 0x7fd37014b330 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.184+0000 7fd3769db700 1 -- 172.21.15.29:0/3797743961 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3700c7660 con 0x7fd37014b330 2024-01-24T15:32:28.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.186+0000 7fd3769db700 1 -- 172.21.15.29:0/3797743961 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3700ab0a0 con 0x7fd37014b330 2024-01-24T15:32:28.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.186+0000 7fd357fff700 1 -- 172.21.15.29:0/3797743961 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 98) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd364005bd0 con 0x7fd37014b330 2024-01-24T15:32:28.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.186+0000 7fd357fff700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd35c05f2a0 0x7fd35c061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:28.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.186+0000 7fd357fff700 1 -- 172.21.15.29:0/3797743961 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7fd364079e20 con 0x7fd37014b330 2024-01-24T15:32:28.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.187+0000 7fd36ffff700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd35c05f2a0 0x7fd35c061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:28.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.187+0000 7fd36ffff700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd35c05f2a0 0x7fd35c061760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fd36000bb60 tx=0x7fd360008040).ready entity=mgr.9352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:28.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.191+0000 7fd357fff700 1 -- 172.21.15.29:0/3797743961 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd3700ab0a0 con 0x7fd37014b330 2024-01-24T15:32:28.428 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.426+0000 7fd3769db700 1 -- 172.21.15.29:0/3797743961 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fd3700ab060 con 0x7fd37014b330 2024-01-24T15:32:28.905 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.903+0000 7fd357fff700 1 -- 172.21.15.29:0/3797743961 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 99) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fd364077630 con 0x7fd37014b330 2024-01-24T15:32:28.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.904+0000 7fd357fff700 1 -- 172.21.15.29:0/3797743961 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v99) v1 ==== 92+0+0 (secure 0 0 0) 0x7fd364077230 con 0x7fd37014b330 2024-01-24T15:32:28.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.908+0000 7fd3769db700 1 -- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd35c05f2a0 msgr2=0x7fd35c061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:28.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.908+0000 7fd3769db700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd35c05f2a0 0x7fd35c061760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fd36000bb60 tx=0x7fd360008040).stop 2024-01-24T15:32:28.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.908+0000 7fd3769db700 1 -- 172.21.15.29:0/3797743961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd37014b330 msgr2=0x7fd370148130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:28.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.908+0000 7fd3769db700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd37014b330 0x7fd370148130 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 rx=0x7fd36400c490 tx=0x7fd364007e30).stop 2024-01-24T15:32:28.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.908+0000 7fd3769db700 1 -- 172.21.15.29:0/3797743961 shutdown_connections 2024-01-24T15:32:28.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.908+0000 7fd3769db700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd35c05f2a0 0x7fd35c061760 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.908+0000 7fd3769db700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd37014b710 0x7fd370150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.908+0000 7fd3769db700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3700c3360 0x7fd3700c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.908+0000 7fd3769db700 1 --2- 172.21.15.29:0/3797743961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd37014b330 0x7fd370148130 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:28.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.908+0000 7fd3769db700 1 -- 172.21.15.29:0/3797743961 >> 172.21.15.29:0/3797743961 conn(0x7fd370064390 msgr2=0x7fd370149ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:28.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.908+0000 7fd3769db700 1 -- 172.21.15.29:0/3797743961 shutdown_connections 2024-01-24T15:32:28.911 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:28.908+0000 7fd3769db700 1 -- 172.21.15.29:0/3797743961 wait complete. 2024-01-24T15:32:28.929 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-01-24T15:32:29.051 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:29.048+0000 7fe6be2ee680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:32:29.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.063+0000 7fce495ca700 1 Processor -- start 2024-01-24T15:32:29.065 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.063+0000 7fce495ca700 1 -- start start 2024-01-24T15:32:29.066 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:29.065+0000 7f1b6316f680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:32:29.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.065+0000 7fce495ca700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce44064760 0x7fce44064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:29.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.065+0000 7fce495ca700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fce440c8490 0x7fce440a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:29.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.065+0000 7fce495ca700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce44152240 con 0x7fce44154700 2024-01-24T15:32:29.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.065+0000 7fce495ca700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce441586c0 con 0x7fce440c8490 2024-01-24T15:32:29.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.065+0000 7fce495ca700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce440c0d50 con 0x7fce44064760 2024-01-24T15:32:29.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.065+0000 7fce427fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fce440c8490 0x7fce440a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:29.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.065+0000 7fce42ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce44064760 0x7fce44064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:29.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.065+0000 7fce437fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fce44154700 0x7fce44154bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:51990/0 (socket says 172.21.15.29:51990) 2024-01-24T15:32:29.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.065+0000 7fce437fe700 1 -- 172.21.15.29:0/2622627260 learned_addr learned my addr 172.21.15.29:0/2622627260 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:29.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.066+0000 7fce427fc700 1 -- 172.21.15.29:0/2622627260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce44064760 msgr2=0x7fce44064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.066+0000 7fce427fc700 1 --2- 172.21.15.29:0/2622627260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce44064760 0x7fce44064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.066+0000 7fce427fc700 1 -- 172.21.15.29:0/2622627260 >> v1:172.21.15.163:6789/0 conn(0x7fce44154700 legacy=0x7fce44154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.066+0000 7fce427fc700 1 -- 172.21.15.29:0/2622627260 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce440aa670 con 0x7fce440c8490 2024-01-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.066+0000 7fce42ffd700 1 --2- 172.21.15.29:0/2622627260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce44064760 0x7fce44064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.066+0000 7fce41ffb700 1 -- 172.21.15.29:0/2622627260 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224043945 0 0) 0x7fce44152240 con 0x7fce44154700 2024-01-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.066+0000 7fce427fc700 1 --2- 172.21.15.29:0/2622627260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce440c8490 0x7fce440a9d90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7fce38009760 tx=0x7fce38009060).ready entity=mon.1 client_cookie=836650b356e13f11 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.066+0000 7fce41ffb700 1 -- 172.21.15.29:0/2622627260 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3800b590 con 0x7fce440c8490 2024-01-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.066+0000 7fce41ffb700 1 -- 172.21.15.29:0/2622627260 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3800b6f0 con 0x7fce440c8490 2024-01-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.066+0000 7fce41ffb700 1 -- 172.21.15.29:0/2622627260 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3800ba70 con 0x7fce440c8490 2024-01-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.067+0000 7fce495ca700 1 -- 172.21.15.29:0/2622627260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce440c8490 msgr2=0x7fce440a9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.067+0000 7fce495ca700 1 --2- 172.21.15.29:0/2622627260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce440c8490 0x7fce440a9d90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7fce38009760 tx=0x7fce38009060).stop 2024-01-24T15:32:29.070 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.067+0000 7fce495ca700 1 -- 172.21.15.29:0/2622627260 shutdown_connections 2024-01-24T15:32:29.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.067+0000 7fce495ca700 1 --2- 172.21.15.29:0/2622627260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce44064760 0x7fce44064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:29.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.067+0000 7fce495ca700 1 --2- 172.21.15.29:0/2622627260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce440c8490 0x7fce440a9d90 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:29.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.067+0000 7fce495ca700 1 -- 172.21.15.29:0/2622627260 >> 172.21.15.29:0/2622627260 conn(0x7fce440cb8b0 msgr2=0x7fce440c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:29.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.067+0000 7fce495ca700 1 -- 172.21.15.29:0/2622627260 shutdown_connections 2024-01-24T15:32:29.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.067+0000 7fce495ca700 1 -- 172.21.15.29:0/2622627260 wait complete. 2024-01-24T15:32:29.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.069+0000 7fce495ca700 1 Processor -- start 2024-01-24T15:32:29.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.069+0000 7fce495ca700 1 -- start start 2024-01-24T15:32:29.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.070+0000 7fce495ca700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce44064760 0x7fce4414f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:29.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.070+0000 7fce495ca700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c8490 0x7fce44154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:29.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.070+0000 7fce495ca700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce44154700 0x7fce440ac620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:29.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.070+0000 7fce495ca700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce440ae520 con 0x7fce44064760 2024-01-24T15:32:29.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.070+0000 7fce495ca700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce440ae1f0 con 0x7fce44154700 2024-01-24T15:32:29.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.070+0000 7fce495ca700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce44155570 con 0x7fce440c8490 2024-01-24T15:32:29.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.070+0000 7fce42ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce44064760 0x7fce4414f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:29.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce42ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce44064760 0x7fce4414f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55328/0 (socket says 172.21.15.29:55328) 2024-01-24T15:32:29.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce42ffd700 1 -- 172.21.15.29:0/1647884239 learned_addr learned my addr 172.21.15.29:0/1647884239 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:29.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce427fc700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c8490 0x7fce44154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:29.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce437fe700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce44154700 0x7fce440ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:29.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce42ffd700 1 -- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c8490 msgr2=0x7fce44154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:29.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce42ffd700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c8490 0x7fce44154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:29.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce42ffd700 1 -- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce44154700 msgr2=0x7fce440ac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:29.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce42ffd700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce44154700 0x7fce440ac620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:29.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce42ffd700 1 -- 172.21.15.29:0/1647884239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce340092a0 con 0x7fce44064760 2024-01-24T15:32:29.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce42ffd700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce44064760 0x7fce4414f280 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 rx=0x7fce3400ea20 tx=0x7fce34009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:29.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce2affd700 1 -- 172.21.15.29:0/1647884239 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400ee60 con 0x7fce44064760 2024-01-24T15:32:29.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce2affd700 1 -- 172.21.15.29:0/1647884239 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3400c850 con 0x7fce44064760 2024-01-24T15:32:29.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce495ca700 1 -- 172.21.15.29:0/1647884239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce380092a0 con 0x7fce44064760 2024-01-24T15:32:29.075 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce2affd700 1 -- 172.21.15.29:0/1647884239 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400cbd0 con 0x7fce44064760 2024-01-24T15:32:29.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.071+0000 7fce495ca700 1 -- 172.21.15.29:0/1647884239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce44155b30 con 0x7fce44064760 2024-01-24T15:32:29.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.072+0000 7fce495ca700 1 -- 172.21.15.29:0/1647884239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce440ae800 con 0x7fce44064760 2024-01-24T15:32:29.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.073+0000 7fce2affd700 1 -- 172.21.15.29:0/1647884239 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 99) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fce3401b400 con 0x7fce44064760 2024-01-24T15:32:29.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.074+0000 7fce2affd700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fce3005f2c0 0x7fce30061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:29.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.074+0000 7fce2affd700 1 -- 172.21.15.29:0/1647884239 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7fce3407b2f0 con 0x7fce44064760 2024-01-24T15:32:29.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.074+0000 7fce427fc700 1 -- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fce3005f2c0 msgr2=0x7fce30061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:29.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.074+0000 7fce427fc700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fce3005f2c0 0x7fce30061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:29.079 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.078+0000 7fce2affd700 1 -- 172.21.15.29:0/1647884239 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fce3404bfe0 con 0x7fce44064760 2024-01-24T15:32:29.115 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:29.113+0000 7fe6be2ee680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:32:29.142 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:29.140+0000 7f1b6316f680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:32:29.263 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.261+0000 7fce495ca700 1 -- 172.21.15.29:0/1647884239 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fce440ae050 con 0x7fce44064760 2024-01-24T15:32:29.268 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.266+0000 7fce2affd700 1 -- 172.21.15.29:0/1647884239 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v43)=0 v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7fce440ae050 con 0x7fce44064760 2024-01-24T15:32:29.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.268+0000 7fce495ca700 1 -- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fce3005f2c0 msgr2=0x7fce30061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:29.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.268+0000 7fce495ca700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fce3005f2c0 0x7fce30061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:29.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.268+0000 7fce495ca700 1 -- 172.21.15.29:0/1647884239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce44064760 msgr2=0x7fce4414f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:29.270 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.268+0000 7fce495ca700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce44064760 0x7fce4414f280 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 rx=0x7fce3400ea20 tx=0x7fce34009060).stop 2024-01-24T15:32:29.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.268+0000 7fce495ca700 1 -- 172.21.15.29:0/1647884239 shutdown_connections 2024-01-24T15:32:29.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.268+0000 7fce495ca700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce44154700 0x7fce440ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:29.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.268+0000 7fce495ca700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce44064760 0x7fce4414f280 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:29.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.268+0000 7fce495ca700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fce3005f2c0 0x7fce30061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:29.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.268+0000 7fce495ca700 1 --2- 172.21.15.29:0/1647884239 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c8490 0x7fce44154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:29.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.268+0000 7fce495ca700 1 -- 172.21.15.29:0/1647884239 >> 172.21.15.29:0/1647884239 conn(0x7fce440cb8b0 msgr2=0x7fce440c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:29.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.268+0000 7fce495ca700 1 -- 172.21.15.29:0/1647884239 shutdown_connections 2024-01-24T15:32:29.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.268+0000 7fce495ca700 1 -- 172.21.15.29:0/1647884239 wait complete. 2024-01-24T15:32:29.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:32:29.285 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-01-24T15:32:29.310 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2024-01-24T15:32:29.393 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:29.390+0000 7fe6be2ee680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:32:29.461 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:29.459+0000 7f1b6316f680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:32:29.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.484+0000 7fdbf5b73700 1 Processor -- start 2024-01-24T15:32:29.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.484+0000 7fdbf5b73700 1 -- start start 2024-01-24T15:32:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.486+0000 7fdbf5b73700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdbf00c7f70 0x7fdbf00c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.486+0000 7fdbf5b73700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf0154f90 con 0x7fdbf0151e50 2024-01-24T15:32:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.486+0000 7fdbf5b73700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf00a9f10 con 0x7fdbf0064e90 2024-01-24T15:32:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.486+0000 7fdbf5b73700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf00aa1f0 con 0x7fdbf00c7f70 2024-01-24T15:32:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.486+0000 7fdbef7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdbf0151e50 0x7fdbf0064780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54276/0 (socket says 172.21.15.29:54276) 2024-01-24T15:32:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.486+0000 7fdbef7fe700 1 -- 172.21.15.29:0/3276184228 learned_addr learned my addr 172.21.15.29:0/3276184228 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:29.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.486+0000 7fdbeffff700 1 --2- 172.21.15.29:0/3276184228 >> v2:172.21.15.163:3300/0 conn(0x7fdbf00c7f70 0x7fdbf00c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.486+0000 7fdbee7fc700 1 -- 172.21.15.29:0/3276184228 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916487089 0 0) 0x7fdbf0154f90 con 0x7fdbf0151e50 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.486+0000 7fdbee7fc700 1 -- 172.21.15.29:0/3276184228 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbd80033f0 con 0x7fdbf0151e50 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.487+0000 7fdbee7fc700 1 -- 172.21.15.29:0/3276184228 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815790210 0 0) 0x7fdbf00a9f10 con 0x7fdbf0064e90 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.487+0000 7fdbee7fc700 1 -- 172.21.15.29:0/3276184228 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbf0154f90 con 0x7fdbf0064e90 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.487+0000 7fdbeffff700 1 -- 172.21.15.29:0/3276184228 >> v1:172.21.15.163:6789/0 conn(0x7fdbf0064e90 legacy=0x7fdbf00bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.487+0000 7fdbeffff700 1 -- 172.21.15.29:0/3276184228 >> v1:172.21.15.29:6789/0 conn(0x7fdbf0151e50 legacy=0x7fdbf0064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.487+0000 7fdbeffff700 1 -- 172.21.15.29:0/3276184228 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf00aa4d0 con 0x7fdbf00c7f70 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.487+0000 7fdbeffff700 1 --2- 172.21.15.29:0/3276184228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbf00c7f70 0x7fdbf00c53c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7fdbe8009760 tx=0x7fdbe8009060).ready entity=mon.1 client_cookie=c9733f7d34d0b042 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.487+0000 7fdbee7fc700 1 -- 172.21.15.29:0/3276184228 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe800d4a0 con 0x7fdbf00c7f70 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.487+0000 7fdbee7fc700 1 -- 172.21.15.29:0/3276184228 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe800d600 con 0x7fdbf00c7f70 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.487+0000 7fdbee7fc700 1 -- 172.21.15.29:0/3276184228 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe800d980 con 0x7fdbf00c7f70 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.488+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3276184228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbf00c7f70 msgr2=0x7fdbf00c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.488+0000 7fdbf5b73700 1 --2- 172.21.15.29:0/3276184228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbf00c7f70 0x7fdbf00c53c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7fdbe8009760 tx=0x7fdbe8009060).stop 2024-01-24T15:32:29.491 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.488+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3276184228 shutdown_connections 2024-01-24T15:32:29.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.488+0000 7fdbf5b73700 1 --2- 172.21.15.29:0/3276184228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbf00c7f70 0x7fdbf00c53c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:29.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.488+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3276184228 >> 172.21.15.29:0/3276184228 conn(0x7fdbf00cb880 msgr2=0x7fdbf00c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:29.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.488+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3276184228 shutdown_connections 2024-01-24T15:32:29.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.488+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3276184228 wait complete. 2024-01-24T15:32:29.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.490+0000 7fdbf5b73700 1 Processor -- start 2024-01-24T15:32:29.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.490+0000 7fdbf5b73700 1 -- start start 2024-01-24T15:32:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.491+0000 7fdbf5b73700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf0064e90 0x7fdbf0144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.491+0000 7fdbf5b73700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbf00c7f70 0x7fdbf00c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.491+0000 7fdbf5b73700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbf0151e50 0x7fdbf01575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.491+0000 7fdbf5b73700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf00ae240 con 0x7fdbf0064e90 2024-01-24T15:32:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.491+0000 7fdbf5b73700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf0144e50 con 0x7fdbf0151e50 2024-01-24T15:32:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.491+0000 7fdbf5b73700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf0167b60 con 0x7fdbf00c7f70 2024-01-24T15:32:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.491+0000 7fdbef7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf0064e90 0x7fdbf0144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbef7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf0064e90 0x7fdbf0144960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:55338/0 (socket says 172.21.15.29:55338) 2024-01-24T15:32:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbef7fe700 1 -- 172.21.15.29:0/3510264470 learned_addr learned my addr 172.21.15.29:0/3510264470 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:29.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbeffff700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbf0151e50 0x7fdbf01575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbeeffd700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbf00c7f70 0x7fdbf00c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbef7fe700 1 -- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbf00c7f70 msgr2=0x7fdbf00c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbef7fe700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbf00c7f70 0x7fdbf00c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbef7fe700 1 -- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbf0151e50 msgr2=0x7fdbf01575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbef7fe700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbf0151e50 0x7fdbf01575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbef7fe700 1 -- 172.21.15.29:0/3510264470 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe80092a0 con 0x7fdbf0064e90 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbeeffd700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbf00c7f70 0x7fdbf00c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbeffff700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbf0151e50 0x7fdbf01575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbef7fe700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf0064e90 0x7fdbf0144960 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7fdbe0002250 tx=0x7fdbe0001640).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbd77fe700 1 -- 172.21.15.29:0/3510264470 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0005d30 con 0x7fdbf0064e90 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbd77fe700 1 -- 172.21.15.29:0/3510264470 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe0005e90 con 0x7fdbf0064e90 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3510264470 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf0167e40 con 0x7fdbf0064e90 2024-01-24T15:32:29.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbd77fe700 1 -- 172.21.15.29:0/3510264470 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0007320 con 0x7fdbf0064e90 2024-01-24T15:32:29.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.492+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3510264470 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf0168400 con 0x7fdbf0064e90 2024-01-24T15:32:29.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.494+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3510264470 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf00aa710 con 0x7fdbf0064e90 2024-01-24T15:32:29.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.494+0000 7fdbd77fe700 1 -- 172.21.15.29:0/3510264470 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 99) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fdbe00076d0 con 0x7fdbf0064e90 2024-01-24T15:32:29.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.495+0000 7fdbd77fe700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 0x7fdbdc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:29.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.495+0000 7fdbd77fe700 1 -- 172.21.15.29:0/3510264470 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7214+0+0 (secure 0 0 0) 0x7fdbe007a750 con 0x7fdbf0064e90 2024-01-24T15:32:29.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.495+0000 7fdbeeffd700 1 -- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 msgr2=0x7fdbdc061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:29.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.495+0000 7fdbeeffd700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 0x7fdbdc061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:29.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.499+0000 7fdbd77fe700 1 -- 172.21.15.29:0/3510264470 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdbe004b400 con 0x7fdbf0064e90 2024-01-24T15:32:29.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.696+0000 7fdbeeffd700 1 -- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 msgr2=0x7fdbdc061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:29.698 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.696+0000 7fdbeeffd700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 0x7fdbdc061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:32:29.710 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:29.708+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3510264470 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbf00832e0 con 0x7fdbdc05f2c0 2024-01-24T15:32:30.015 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:30.012+0000 7fe6be2ee680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:32:30.087 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:30.085+0000 7fe6be2ee680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:32:30.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:30.097+0000 7fdbeeffd700 1 -- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 msgr2=0x7fdbdc061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:30.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:30.097+0000 7fdbeeffd700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 0x7fdbdc061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-01-24T15:32:30.191 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:30.189+0000 7f1b6316f680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:32:30.267 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:30.265+0000 7f1b6316f680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:32:30.275 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:30.273+0000 7fe6be2ee680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:32:30.481 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:30.479+0000 7f1b6316f680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:32:30.699 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:30.697+0000 7fe6be2ee680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:32:30.803 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:30.800+0000 7fe6be2ee680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:32:30.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:30.898+0000 7fdbeeffd700 1 -- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 msgr2=0x7fdbdc061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:30.900 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:30.898+0000 7fdbeeffd700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 0x7fdbdc061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-01-24T15:32:30.958 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:30.955+0000 7fe6be2ee680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:32:30.963 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:30.961+0000 7f1b6316f680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:32:31.034 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:31.031+0000 7fe6be2ee680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:32:31.081 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:31.079+0000 7f1b6316f680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:32:31.101 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:31.099+0000 7fe6be2ee680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:32:31.258 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:31.256+0000 7f1b6316f680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:32:31.343 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:31.341+0000 7f1b6316f680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:32:31.362 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:31.359+0000 7fe6be2ee680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:32:31.421 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:31.419+0000 7f1b6316f680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:32:31.548 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:31.544+0000 7fe6be2ee680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:32:31.717 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:31.715+0000 7f1b6316f680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:32:31.929 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:31.927+0000 7f1b6316f680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:32:32.203 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:32.201+0000 7fe6be2ee680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:32:32.273 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:32.270+0000 7fe6be2ee680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:32:32.349 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:32.347+0000 7fe6be2ee680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:32:32.498 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:32.496+0000 7fe6be2ee680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:32:32.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:32.500+0000 7fdbeeffd700 1 -- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 msgr2=0x7fdbdc061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:32.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:32.500+0000 7fdbeeffd700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 0x7fdbdc061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-01-24T15:32:32.570 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:32.567+0000 7fe6be2ee680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:32:32.687 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:32.683+0000 7fe6be2ee680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:32:32.687 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:32.682+0000 7f1b6316f680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:32:32.763 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:32.761+0000 7f1b6316f680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:32:32.795 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:32.793+0000 7fe6be2ee680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:32:32.849 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:32.847+0000 7f1b6316f680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:32:33.018 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:33.016+0000 7f1b6316f680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:32:33.089 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:33.086+0000 7fe6be2ee680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:32:33.100 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:33.098+0000 7f1b6316f680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:32:33.161 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:33.158+0000 7fe6be2ee680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:32:33.181 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:33.179+0000 7fdbd77fe700 1 -- 172.21.15.29:0/3510264470 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 100) v1 ==== 75287+0+0 (secure 0 0 0) 0x7fdbe0077f60 con 0x7fdbf0064e90 2024-01-24T15:32:33.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:33.179+0000 7fdbd77fe700 1 -- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 msgr2=0x7fdbdc061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:33.182 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:33.179+0000 7fdbd77fe700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 0x7fdbdc061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:33.240 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:33.238+0000 7f1b6316f680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:32:33.249 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:33.246+0000 7fe634869700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:33.249 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:33.246+0000 7fe634869700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:33.249 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:33.246+0000 7fe634869700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:33.249 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:33.246+0000 7fe634869700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:33.249 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:33.246+0000 7fe634869700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:33.249 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:33.247+0000 7fe631062700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:33.249 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:33.247+0000 7fe631062700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:33.249 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:33.247+0000 7fe631062700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:33.250 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:33.247+0000 7fe631062700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:33.250 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:33.247+0000 7fe631062700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:33.365 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:33.363+0000 7f1b6316f680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:32:33.701 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:33.699+0000 7f1b6316f680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:32:33.783 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:33.781+0000 7f1b6316f680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:32:34.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.181+0000 7fdbd77fe700 1 -- 172.21.15.29:0/3510264470 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mgrmap(e 101) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fdbe00782e0 con 0x7fdbf0064e90 2024-01-24T15:32:34.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.182+0000 7fdbd77fe700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 0x7fdbdc061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.182+0000 7fdbd77fe700 1 -- 172.21.15.29:0/3510264470 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbf00832e0 con 0x7fdbdc05f2c0 2024-01-24T15:32:34.184 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.182+0000 7fdbeeffd700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 0x7fdbdc061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.184+0000 7fdbeeffd700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 0x7fdbdc061700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7fdbf00ae130 tx=0x7fdbe4007bd0).ready entity=mgr.9694 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:34.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.193+0000 7fdbd77fe700 1 -- 172.21.15.29:0/3510264470 <== mgr.9694 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdbf00832e0 con 0x7fdbdc05f2c0 2024-01-24T15:32:34.195 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule set for path snap_test_dir1 2024-01-24T15:32:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.195+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 msgr2=0x7fdbdc061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.195+0000 7fdbf5b73700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 0x7fdbdc061700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7fdbf00ae130 tx=0x7fdbe4007bd0).stop 2024-01-24T15:32:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.195+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3510264470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf0064e90 msgr2=0x7fdbf0144960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.195+0000 7fdbf5b73700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf0064e90 0x7fdbf0144960 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7fdbe0002250 tx=0x7fdbe0001640).stop 2024-01-24T15:32:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.195+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3510264470 shutdown_connections 2024-01-24T15:32:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.195+0000 7fdbf5b73700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbf0151e50 0x7fdbf01575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.195+0000 7fdbf5b73700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbf0064e90 0x7fdbf0144960 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.195+0000 7fdbf5b73700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdbdc05f2c0 0x7fdbdc061700 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.195+0000 7fdbf5b73700 1 --2- 172.21.15.29:0/3510264470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbf00c7f70 0x7fdbf00c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.196+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3510264470 >> 172.21.15.29:0/3510264470 conn(0x7fdbf00cb880 msgr2=0x7fdbf00c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.196+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3510264470 shutdown_connections 2024-01-24T15:32:34.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.196+0000 7fdbf5b73700 1 -- 172.21.15.29:0/3510264470 wait complete. 2024-01-24T15:32:34.214 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-01-24T15:32:34.214 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2024-01-24T15:32:34.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.352+0000 7fa277804700 1 Processor -- start 2024-01-24T15:32:34.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.352+0000 7fa277804700 1 -- start start 2024-01-24T15:32:34.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.353+0000 7fa277804700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa27013a5e0 0x7fa2700639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.353+0000 7fa277804700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa270063eb0 0x7fa2700664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.353+0000 7fa277804700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa270066e20 con 0x7fa2700669b0 2024-01-24T15:32:34.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.353+0000 7fa277804700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2700679f0 con 0x7fa27013a5e0 2024-01-24T15:32:34.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.353+0000 7fa277804700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa270067cd0 con 0x7fa270063eb0 2024-01-24T15:32:34.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.353+0000 7fa2755a0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa27013a5e0 0x7fa2700639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.357 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.353+0000 7fa2755a0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa27013a5e0 0x7fa2700639c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38434/0 (socket says 172.21.15.29:38434) 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.353+0000 7fa2755a0700 1 -- 172.21.15.29:0/4047607491 learned_addr learned my addr 172.21.15.29:0/4047607491 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.354+0000 7fa274d9f700 1 --2- 172.21.15.29:0/4047607491 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa270063eb0 0x7fa2700664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.354+0000 7fa2755a0700 1 -- 172.21.15.29:0/4047607491 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa270063eb0 msgr2=0x7fa2700664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.354+0000 7fa2755a0700 1 --2- 172.21.15.29:0/4047607491 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa270063eb0 0x7fa2700664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.354+0000 7fa2755a0700 1 -- 172.21.15.29:0/4047607491 >> v1:172.21.15.163:6789/0 conn(0x7fa2700669b0 legacy=0x7fa270157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.354+0000 7fa2755a0700 1 -- 172.21.15.29:0/4047607491 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa270067fb0 con 0x7fa27013a5e0 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.354+0000 7fa2755a0700 1 --2- 172.21.15.29:0/4047607491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa27013a5e0 0x7fa2700639c0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7fa26c006fd0 tx=0x7fa26c0090f0).ready entity=mon.0 client_cookie=5c9f2264b49b05cd server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.354+0000 7fa267fff700 1 -- 172.21.15.29:0/4047607491 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c00b600 con 0x7fa27013a5e0 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.354+0000 7fa267fff700 1 -- 172.21.15.29:0/4047607491 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c00b760 con 0x7fa27013a5e0 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.354+0000 7fa267fff700 1 -- 172.21.15.29:0/4047607491 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c00bae0 con 0x7fa27013a5e0 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.354+0000 7fa277804700 1 -- 172.21.15.29:0/4047607491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa27013a5e0 msgr2=0x7fa2700639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.354+0000 7fa277804700 1 --2- 172.21.15.29:0/4047607491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa27013a5e0 0x7fa2700639c0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7fa26c006fd0 tx=0x7fa26c0090f0).stop 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.355+0000 7fa277804700 1 -- 172.21.15.29:0/4047607491 shutdown_connections 2024-01-24T15:32:34.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.355+0000 7fa277804700 1 --2- 172.21.15.29:0/4047607491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa27013a5e0 0x7fa2700639c0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.355+0000 7fa277804700 1 --2- 172.21.15.29:0/4047607491 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa270063eb0 0x7fa2700664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.355+0000 7fa277804700 1 -- 172.21.15.29:0/4047607491 >> 172.21.15.29:0/4047607491 conn(0x7fa2700c8bd0 msgr2=0x7fa2700c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:34.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.355+0000 7fa277804700 1 -- 172.21.15.29:0/4047607491 shutdown_connections 2024-01-24T15:32:34.359 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.355+0000 7fa277804700 1 -- 172.21.15.29:0/4047607491 wait complete. 2024-01-24T15:32:34.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.356+0000 7fa277804700 1 Processor -- start 2024-01-24T15:32:34.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.356+0000 7fa277804700 1 -- start start 2024-01-24T15:32:34.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa277804700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa270063eb0 0x7fa2700a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa277804700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2700669b0 0x7fa2700ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa277804700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa27013a5e0 0x7fa27012bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa277804700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700aed20 con 0x7fa27013a5e0 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa277804700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700aaed0 con 0x7fa270063eb0 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa277804700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700aaa40 con 0x7fa2700669b0 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa275da1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa27013a5e0 0x7fa27012bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa275da1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa27013a5e0 0x7fa27012bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38442/0 (socket says 172.21.15.29:38442) 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa275da1700 1 -- 172.21.15.29:0/47255856 learned_addr learned my addr 172.21.15.29:0/47255856 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa274d9f700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2700669b0 0x7fa2700ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa2755a0700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa270063eb0 0x7fa2700a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa275da1700 1 -- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2700669b0 msgr2=0x7fa2700ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa275da1700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2700669b0 0x7fa2700ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa275da1700 1 -- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa270063eb0 msgr2=0x7fa2700a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa275da1700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa270063eb0 0x7fa2700a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.357+0000 7fa275da1700 1 -- 172.21.15.29:0/47255856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c009370 con 0x7fa27013a5e0 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.358+0000 7fa275da1700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa27013a5e0 0x7fa27012bc50 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 rx=0x7fa26800c840 tx=0x7fa268002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.358+0000 7fa2657fa700 1 -- 172.21.15.29:0/47255856 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800e4a0 con 0x7fa27013a5e0 2024-01-24T15:32:34.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.358+0000 7fa2657fa700 1 -- 172.21.15.29:0/47255856 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26800e640 con 0x7fa27013a5e0 2024-01-24T15:32:34.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.358+0000 7fa2657fa700 1 -- 172.21.15.29:0/47255856 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800ea00 con 0x7fa27013a5e0 2024-01-24T15:32:34.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.358+0000 7fa277804700 1 -- 172.21.15.29:0/47255856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2700a9e30 con 0x7fa27013a5e0 2024-01-24T15:32:34.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.358+0000 7fa277804700 1 -- 172.21.15.29:0/47255856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2700aa3f0 con 0x7fa27013a5e0 2024-01-24T15:32:34.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.359+0000 7fa277804700 1 -- 172.21.15.29:0/47255856 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2700475e0 con 0x7fa27013a5e0 2024-01-24T15:32:34.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.359+0000 7fa2657fa700 1 -- 172.21.15.29:0/47255856 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 101) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fa2680123f0 con 0x7fa27013a5e0 2024-01-24T15:32:34.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.360+0000 7fa2657fa700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa25405f050 0x7fa254061510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.360+0000 7fa2657fa700 1 -- 172.21.15.29:0/47255856 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7fa268079e30 con 0x7fa27013a5e0 2024-01-24T15:32:34.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.360+0000 7fa2755a0700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa25405f050 0x7fa254061510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.360+0000 7fa2755a0700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa25405f050 0x7fa254061510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa26c009130 tx=0x7fa26c017000).ready entity=mgr.9694 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:34.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.364+0000 7fa2657fa700 1 -- 172.21.15.29:0/47255856 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa268047b00 con 0x7fa27013a5e0 2024-01-24T15:32:34.576 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.573+0000 7fa277804700 1 -- 172.21.15.29:0/47255856 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2700aab80 con 0x7fa25405f050 2024-01-24T15:32:34.581 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.579+0000 7fa2657fa700 1 -- 172.21.15.29:0/47255856 <== mgr.9694 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7fa2700aab80 con 0x7fa25405f050 2024-01-24T15:32:34.581 INFO:teuthology.orchestra.run.smithi029.stdout:Retention added to path snap_test_dir1 2024-01-24T15:32:34.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.581+0000 7fa277804700 1 -- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa25405f050 msgr2=0x7fa254061510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.581+0000 7fa277804700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa25405f050 0x7fa254061510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa26c009130 tx=0x7fa26c017000).stop 2024-01-24T15:32:34.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.581+0000 7fa277804700 1 -- 172.21.15.29:0/47255856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa27013a5e0 msgr2=0x7fa27012bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.581+0000 7fa277804700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa27013a5e0 0x7fa27012bc50 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 rx=0x7fa26800c840 tx=0x7fa268002560).stop 2024-01-24T15:32:34.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.581+0000 7fa277804700 1 -- 172.21.15.29:0/47255856 shutdown_connections 2024-01-24T15:32:34.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.581+0000 7fa277804700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa27013a5e0 0x7fa27012bc50 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.581+0000 7fa277804700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa270063eb0 0x7fa2700a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.581+0000 7fa277804700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa25405f050 0x7fa254061510 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.584 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.581+0000 7fa277804700 1 --2- 172.21.15.29:0/47255856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2700669b0 0x7fa2700ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.581+0000 7fa277804700 1 -- 172.21.15.29:0/47255856 >> 172.21.15.29:0/47255856 conn(0x7fa2700c8bd0 msgr2=0x7fa270156cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:34.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.581+0000 7fa277804700 1 -- 172.21.15.29:0/47255856 shutdown_connections 2024-01-24T15:32:34.585 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.581+0000 7fa277804700 1 -- 172.21.15.29:0/47255856 wait complete. 2024-01-24T15:32:34.597 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2024-01-24T15:32:34.597 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2024-01-24-15_33_00 in ~26s... 2024-01-24T15:32:34.597 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2024-01-24T15:32:34.598 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2024-01-24T15:32:34.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.729+0000 7fb814069700 1 Processor -- start 2024-01-24T15:32:34.731 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.729+0000 7fb814069700 1 -- start start 2024-01-24T15:32:34.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.730+0000 7fb814069700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb80c13a6f0 0x7fb80c068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.734 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.730+0000 7fb814069700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb80c0638c0 0x7fb80c066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.730+0000 7fb814069700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80c0643f0 con 0x7fb80c066940 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.730+0000 7fb814069700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80c066df0 con 0x7fb80c0638c0 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.730+0000 7fb814069700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c0bdb30 con 0x7fb80c13a6f0 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.730+0000 7fb812606700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb80c066940 0x7fb80c0be860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37166/0 (socket says 172.21.15.29:37166) 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.730+0000 7fb812606700 1 -- 172.21.15.29:0/4066932285 learned_addr learned my addr 172.21.15.29:0/4066932285 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb811604700 1 --2- 172.21.15.29:0/4066932285 >> v2:172.21.15.29:3300/0 conn(0x7fb80c0638c0 0x7fb80c066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb811e05700 1 --2- 172.21.15.29:0/4066932285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb80c13a6f0 0x7fb80c068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb810e03700 1 -- 172.21.15.29:0/4066932285 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716767301 0 0) 0x7fb80c0643f0 con 0x7fb80c066940 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb810e03700 1 -- 172.21.15.29:0/4066932285 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f80033f0 con 0x7fb80c066940 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb811604700 1 -- 172.21.15.29:0/4066932285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb80c13a6f0 msgr2=0x7fb80c068660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb811604700 1 --2- 172.21.15.29:0/4066932285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb80c13a6f0 0x7fb80c068660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb811604700 1 -- 172.21.15.29:0/4066932285 >> v1:172.21.15.29:6789/0 conn(0x7fb80c066940 legacy=0x7fb80c0be860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.735 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb810e03700 1 -- 172.21.15.29:0/4066932285 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2221070101 0 0) 0x7fb7f80033f0 con 0x7fb80c066940 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb811604700 1 -- 172.21.15.29:0/4066932285 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c0be3d0 con 0x7fb80c0638c0 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb811604700 1 --2- 172.21.15.29:0/4066932285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb80c0638c0 0x7fb80c066450 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7fb808009760 tx=0x7fb808009060).ready entity=mon.0 client_cookie=355556c59612df98 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb810e03700 1 -- 172.21.15.29:0/4066932285 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8040017f0 con 0x7fb80c0638c0 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb810e03700 1 -- 172.21.15.29:0/4066932285 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb808011400 con 0x7fb80c0638c0 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.731+0000 7fb810e03700 1 -- 172.21.15.29:0/4066932285 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808011780 con 0x7fb80c0638c0 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.732+0000 7fb814069700 1 -- 172.21.15.29:0/4066932285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb80c0638c0 msgr2=0x7fb80c066450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.732+0000 7fb814069700 1 --2- 172.21.15.29:0/4066932285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb80c0638c0 0x7fb80c066450 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7fb808009760 tx=0x7fb808009060).stop 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.732+0000 7fb814069700 1 -- 172.21.15.29:0/4066932285 shutdown_connections 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.732+0000 7fb814069700 1 --2- 172.21.15.29:0/4066932285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb80c13a6f0 0x7fb80c068660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.732+0000 7fb814069700 1 --2- 172.21.15.29:0/4066932285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb80c0638c0 0x7fb80c066450 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.732+0000 7fb814069700 1 -- 172.21.15.29:0/4066932285 >> 172.21.15.29:0/4066932285 conn(0x7fb80c0c8bd0 msgr2=0x7fb80c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.732+0000 7fb814069700 1 -- 172.21.15.29:0/4066932285 shutdown_connections 2024-01-24T15:32:34.736 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.732+0000 7fb814069700 1 -- 172.21.15.29:0/4066932285 wait complete. 2024-01-24T15:32:34.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.733+0000 7fb814069700 1 Processor -- start 2024-01-24T15:32:34.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.733+0000 7fb814069700 1 -- start start 2024-01-24T15:32:34.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.734+0000 7fb814069700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb80c0638c0 0x7fb80c1396e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.734+0000 7fb814069700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb80c066940 0x7fb80c0aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.734+0000 7fb814069700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb80c13a6f0 0x7fb80c1551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.734+0000 7fb814069700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c139bd0 con 0x7fb80c066940 2024-01-24T15:32:34.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.734+0000 7fb814069700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c154d30 con 0x7fb80c0638c0 2024-01-24T15:32:34.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.734+0000 7fb814069700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c0acf20 con 0x7fb80c13a6f0 2024-01-24T15:32:34.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.734+0000 7fb811604700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb80c066940 0x7fb80c0aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.734+0000 7fb811604700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb80c066940 0x7fb80c0aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38456/0 (socket says 172.21.15.29:38456) 2024-01-24T15:32:34.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.734+0000 7fb811604700 1 -- 172.21.15.29:0/2617198167 learned_addr learned my addr 172.21.15.29:0/2617198167 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:34.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb811e05700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb80c0638c0 0x7fb80c1396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb812606700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb80c13a6f0 0x7fb80c1551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb811604700 1 -- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb80c13a6f0 msgr2=0x7fb80c1551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb811604700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb80c13a6f0 0x7fb80c1551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb811604700 1 -- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb80c0638c0 msgr2=0x7fb80c1396e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb811604700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb80c0638c0 0x7fb80c1396e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb811604700 1 -- 172.21.15.29:0/2617198167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8080092a0 con 0x7fb80c066940 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb811e05700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb80c0638c0 0x7fb80c1396e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb811604700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb80c066940 0x7fb80c0aca30 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 rx=0x7fb80c068ff0 tx=0x7fb808016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb7fdffb700 1 -- 172.21.15.29:0/2617198167 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808021070 con 0x7fb80c066940 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb7fdffb700 1 -- 172.21.15.29:0/2617198167 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8080108b0 con 0x7fb80c066940 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb814069700 1 -- 172.21.15.29:0/2617198167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c0ad1f0 con 0x7fb80c066940 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb7fdffb700 1 -- 172.21.15.29:0/2617198167 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808010c30 con 0x7fb80c066940 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.735+0000 7fb814069700 1 -- 172.21.15.29:0/2617198167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80c0ad7b0 con 0x7fb80c066940 2024-01-24T15:32:34.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.736+0000 7fb814069700 1 -- 172.21.15.29:0/2617198167 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80c06ae00 con 0x7fb80c066940 2024-01-24T15:32:34.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.737+0000 7fb7fdffb700 1 -- 172.21.15.29:0/2617198167 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 101) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fb80801d430 con 0x7fb80c066940 2024-01-24T15:32:34.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.737+0000 7fb7fdffb700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fb7f005f080 0x7fb7f0061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:34.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.737+0000 7fb7fdffb700 1 -- 172.21.15.29:0/2617198167 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7fb80807b7b0 con 0x7fb80c066940 2024-01-24T15:32:34.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.737+0000 7fb811e05700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fb7f005f080 0x7fb7f0061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:34.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.738+0000 7fb811e05700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fb7f005f080 0x7fb7f0061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fb800006d80 tx=0x7fb800008040).ready entity=mgr.9694 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:34.743 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.741+0000 7fb7fdffb700 1 -- 172.21.15.29:0/2617198167 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fb80804c6b0 con 0x7fb80c066940 2024-01-24T15:32:34.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.946+0000 7fb814069700 1 -- 172.21.15.29:0/2617198167 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb80c154ec0 con 0x7fb7f005f080 2024-01-24T15:32:34.951 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.949+0000 7fb7fdffb700 1 -- 172.21.15.29:0/2617198167 <== mgr.9694 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7fb80c154ec0 con 0x7fb7f005f080 2024-01-24T15:32:34.952 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:34.952 INFO:teuthology.orchestra.run.smithi029.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2024-01-24T15:32:34.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.952+0000 7fb814069700 1 -- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fb7f005f080 msgr2=0x7fb7f0061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.954 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.952+0000 7fb814069700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fb7f005f080 0x7fb7f0061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fb800006d80 tx=0x7fb800008040).stop 2024-01-24T15:32:34.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.952+0000 7fb814069700 1 -- 172.21.15.29:0/2617198167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb80c066940 msgr2=0x7fb80c0aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:34.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.952+0000 7fb814069700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb80c066940 0x7fb80c0aca30 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 rx=0x7fb80c068ff0 tx=0x7fb808016000).stop 2024-01-24T15:32:34.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.952+0000 7fb814069700 1 -- 172.21.15.29:0/2617198167 shutdown_connections 2024-01-24T15:32:34.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.952+0000 7fb814069700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fb7f005f080 0x7fb7f0061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.952+0000 7fb814069700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb80c13a6f0 0x7fb80c1551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.952+0000 7fb814069700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb80c0638c0 0x7fb80c1396e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.952+0000 7fb814069700 1 --2- 172.21.15.29:0/2617198167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb80c066940 0x7fb80c0aca30 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:34.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.952+0000 7fb814069700 1 -- 172.21.15.29:0/2617198167 >> 172.21.15.29:0/2617198167 conn(0x7fb80c0c8bd0 msgr2=0x7fb80c153880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:34.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.952+0000 7fb814069700 1 -- 172.21.15.29:0/2617198167 shutdown_connections 2024-01-24T15:32:34.955 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:34.952+0000 7fb814069700 1 -- 172.21.15.29:0/2617198167 wait complete. 2024-01-24T15:32:34.968 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2024-01-24T15:32:34.968 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2024-01-24T15:32:34.968 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2024-01-24T15:32:35.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.100+0000 7f367b56c700 1 Processor -- start 2024-01-24T15:32:35.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.101+0000 7f367b56c700 1 -- start start 2024-01-24T15:32:35.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.101+0000 7f367b56c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3674154170 0x7f36740627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.101+0000 7f367b56c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3674069360 con 0x7f3674153d90 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.101+0000 7f367b56c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f367406a1c0 con 0x7f3674062ca0 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.101+0000 7f367b56c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36741545f0 con 0x7f3674154170 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.102+0000 7f3679308700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3674153d90 0x7f36740c2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37168/0 (socket says 172.21.15.29:37168) 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.102+0000 7f3679308700 1 -- 172.21.15.29:0/99625113 learned_addr learned my addr 172.21.15.29:0/99625113 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.102+0000 7f3678b07700 1 --2- 172.21.15.29:0/99625113 >> v2:172.21.15.163:3300/0 conn(0x7f3674154170 0x7f36740627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.102+0000 7f366bfff700 1 -- 172.21.15.29:0/99625113 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285966433 0 0) 0x7f3674069360 con 0x7f3674153d90 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.102+0000 7f366bfff700 1 -- 172.21.15.29:0/99625113 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36600033f0 con 0x7f3674153d90 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.102+0000 7f3678b07700 1 -- 172.21.15.29:0/99625113 >> v1:172.21.15.163:6789/0 conn(0x7f3674062ca0 legacy=0x7f36740c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.102+0000 7f3678b07700 1 -- 172.21.15.29:0/99625113 >> v1:172.21.15.29:6789/0 conn(0x7f3674153d90 legacy=0x7f36740c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.102+0000 7f3678b07700 1 -- 172.21.15.29:0/99625113 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367406a650 con 0x7f3674154170 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.102+0000 7f3678b07700 1 --2- 172.21.15.29:0/99625113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3674154170 0x7f36740627b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f3664009760 tx=0x7f3664009060).ready entity=mon.1 client_cookie=38f684ab9d271b73 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.103+0000 7f366bfff700 1 -- 172.21.15.29:0/99625113 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366400f510 con 0x7f3674154170 2024-01-24T15:32:35.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.103+0000 7f366bfff700 1 -- 172.21.15.29:0/99625113 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366400f670 con 0x7f3674154170 2024-01-24T15:32:35.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.103+0000 7f366bfff700 1 -- 172.21.15.29:0/99625113 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366400f9f0 con 0x7f3674154170 2024-01-24T15:32:35.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.103+0000 7f367b56c700 1 -- 172.21.15.29:0/99625113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3674154170 msgr2=0x7f36740627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:35.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.103+0000 7f367b56c700 1 --2- 172.21.15.29:0/99625113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3674154170 0x7f36740627b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f3664009760 tx=0x7f3664009060).stop 2024-01-24T15:32:35.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.103+0000 7f367b56c700 1 -- 172.21.15.29:0/99625113 shutdown_connections 2024-01-24T15:32:35.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.103+0000 7f367b56c700 1 --2- 172.21.15.29:0/99625113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3674154170 0x7f36740627b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:35.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.103+0000 7f367b56c700 1 -- 172.21.15.29:0/99625113 >> 172.21.15.29:0/99625113 conn(0x7f3674067b20 msgr2=0x7f36740638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:35.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.103+0000 7f367b56c700 1 -- 172.21.15.29:0/99625113 shutdown_connections 2024-01-24T15:32:35.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.103+0000 7f367b56c700 1 -- 172.21.15.29:0/99625113 wait complete. 2024-01-24T15:32:35.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.104+0000 7f367b56c700 1 Processor -- start 2024-01-24T15:32:35.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.105+0000 7f367b56c700 1 -- start start 2024-01-24T15:32:35.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.105+0000 7f367b56c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3674062ca0 0x7f3674152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:35.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.105+0000 7f367b56c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3674153d90 0x7f36740bf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:35.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.105+0000 7f367b56c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3674154170 0x7f36740a9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:35.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.105+0000 7f367b56c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3674152770 con 0x7f3674154170 2024-01-24T15:32:35.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.105+0000 7f367b56c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36740a9f00 con 0x7f3674062ca0 2024-01-24T15:32:35.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.105+0000 7f367b56c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36740c58f0 con 0x7f3674153d90 2024-01-24T15:32:35.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f3679b09700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3674154170 0x7f36740a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:35.109 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f3679b09700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3674154170 0x7f36740a9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38460/0 (socket says 172.21.15.29:38460) 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f3679b09700 1 -- 172.21.15.29:0/205881301 learned_addr learned my addr 172.21.15.29:0/205881301 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f3678b07700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3674153d90 0x7f36740bf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f3679308700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3674062ca0 0x7f3674152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f3679b09700 1 -- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3674153d90 msgr2=0x7f36740bf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f3679b09700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3674153d90 0x7f36740bf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f3679b09700 1 -- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3674062ca0 msgr2=0x7f3674152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f3679b09700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3674062ca0 0x7f3674152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f3679b09700 1 -- 172.21.15.29:0/205881301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36640092a0 con 0x7f3674154170 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f3679b09700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3674154170 0x7f36740a9520 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 rx=0x7f366c002230 tx=0x7f366c0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f36697fa700 1 -- 172.21.15.29:0/205881301 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c010510 con 0x7f3674154170 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f36697fa700 1 -- 172.21.15.29:0/205881301 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c010670 con 0x7f3674154170 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f36697fa700 1 -- 172.21.15.29:0/205881301 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c0109f0 con 0x7f3674154170 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f367b56c700 1 -- 172.21.15.29:0/205881301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36740c5bd0 con 0x7f3674154170 2024-01-24T15:32:35.110 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.106+0000 7f367b56c700 1 -- 172.21.15.29:0/205881301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36740c6190 con 0x7f3674154170 2024-01-24T15:32:35.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.108+0000 7f367b56c700 1 -- 172.21.15.29:0/205881301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36740475e0 con 0x7f3674154170 2024-01-24T15:32:35.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.108+0000 7f36697fa700 1 -- 172.21.15.29:0/205881301 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 101) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f366c01b400 con 0x7f3674154170 2024-01-24T15:32:35.111 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.108+0000 7f36697fa700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f365805f080 0x7f3658061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:35.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.108+0000 7f36697fa700 1 -- 172.21.15.29:0/205881301 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7f366c07a3b0 con 0x7f3674154170 2024-01-24T15:32:35.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.109+0000 7f3679308700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f365805f080 0x7f3658061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:35.112 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.109+0000 7f3679308700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f365805f080 0x7f3658061540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f3670005750 tx=0x7f3670009040).ready entity=mgr.9694 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:35.115 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.113+0000 7f36697fa700 1 -- 172.21.15.29:0/205881301 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f366c04b2b0 con 0x7f3674154170 2024-01-24T15:32:35.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.323+0000 7f367b56c700 1 -- 172.21.15.29:0/205881301 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f36740646b0 con 0x7f365805f080 2024-01-24T15:32:35.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.328+0000 7f36697fa700 1 -- 172.21.15.29:0/205881301 <== mgr.9694 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f36740646b0 con 0x7f365805f080 2024-01-24T15:32:35.330 INFO:teuthology.orchestra.run.smithi029.stdout:Schedule removed for path snap_test_dir1 2024-01-24T15:32:35.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.330+0000 7f367b56c700 1 -- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f365805f080 msgr2=0x7f3658061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:35.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.330+0000 7f367b56c700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f365805f080 0x7f3658061540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f3670005750 tx=0x7f3670009040).stop 2024-01-24T15:32:35.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.330+0000 7f367b56c700 1 -- 172.21.15.29:0/205881301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3674154170 msgr2=0x7f36740a9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:35.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.330+0000 7f367b56c700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3674154170 0x7f36740a9520 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 rx=0x7f366c002230 tx=0x7f366c0015e0).stop 2024-01-24T15:32:35.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.331+0000 7f367b56c700 1 -- 172.21.15.29:0/205881301 shutdown_connections 2024-01-24T15:32:35.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.331+0000 7f367b56c700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3674154170 0x7f36740a9520 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:35.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.331+0000 7f367b56c700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3674062ca0 0x7f3674152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:35.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.331+0000 7f367b56c700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f365805f080 0x7f3658061540 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:35.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.331+0000 7f367b56c700 1 --2- 172.21.15.29:0/205881301 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3674153d90 0x7f36740bf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:35.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.331+0000 7f367b56c700 1 -- 172.21.15.29:0/205881301 >> 172.21.15.29:0/205881301 conn(0x7f3674067b20 msgr2=0x7f36740ac090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:35.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.331+0000 7f367b56c700 1 -- 172.21.15.29:0/205881301 shutdown_connections 2024-01-24T15:32:35.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.331+0000 7f367b56c700 1 -- 172.21.15.29:0/205881301 wait complete. 2024-01-24T15:32:35.346 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-01-24T15:32:35.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:32:35.347 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-01-24T15:32:35.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:32:35.367 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-01-24T15:32:35.431 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-01-24T15:32:35.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.608+0000 7fdecced0700 1 Processor -- start 2024-01-24T15:32:35.611 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.609+0000 7fdecced0700 1 -- start start 2024-01-24T15:32:35.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdecced0700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdec8157530 0x7fdec8068470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:35.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdecced0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec8068960 0x7fdec8066520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:35.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdecced0700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec8066ec0 con 0x7fdec8066a10 2024-01-24T15:32:35.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdecced0700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec8065040 con 0x7fdec8157530 2024-01-24T15:32:35.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdecced0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8064140 con 0x7fdec8068960 2024-01-24T15:32:35.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdec659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdec8157530 0x7fdec8068470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:35.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdec659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdec8157530 0x7fdec8068470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38468/0 (socket says 172.21.15.29:38468) 2024-01-24T15:32:35.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdec659c700 1 -- 172.21.15.29:0/1539053874 learned_addr learned my addr 172.21.15.29:0/1539053874 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:35.620 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdec5d9b700 1 --2- 172.21.15.29:0/1539053874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec8068960 0x7fdec8066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdec659c700 1 -- 172.21.15.29:0/1539053874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec8068960 msgr2=0x7fdec8066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdec659c700 1 --2- 172.21.15.29:0/1539053874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec8068960 0x7fdec8066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdec659c700 1 -- 172.21.15.29:0/1539053874 >> v1:172.21.15.29:6789/0 conn(0x7fdec8066a10 legacy=0x7fdec8157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdec659c700 1 -- 172.21.15.29:0/1539053874 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec80be080 con 0x7fdec8157530 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdec5d9b700 1 --2- 172.21.15.29:0/1539053874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec8068960 0x7fdec8066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.610+0000 7fdec659c700 1 --2- 172.21.15.29:0/1539053874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdec8157530 0x7fdec8068470 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 rx=0x7fdeb8006fd0 tx=0x7fdeb80090f0).ready entity=mon.0 client_cookie=ca515b729d53669d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.611+0000 7fdec559a700 1 -- 172.21.15.29:0/1539053874 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb800b600 con 0x7fdec8157530 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.611+0000 7fdec559a700 1 -- 172.21.15.29:0/1539053874 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb800b760 con 0x7fdec8157530 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.611+0000 7fdec559a700 1 -- 172.21.15.29:0/1539053874 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb800bae0 con 0x7fdec8157530 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.611+0000 7fdecced0700 1 -- 172.21.15.29:0/1539053874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdec8157530 msgr2=0x7fdec8068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.611+0000 7fdecced0700 1 --2- 172.21.15.29:0/1539053874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdec8157530 0x7fdec8068470 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 rx=0x7fdeb8006fd0 tx=0x7fdeb80090f0).stop 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.611+0000 7fdecced0700 1 -- 172.21.15.29:0/1539053874 shutdown_connections 2024-01-24T15:32:35.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.611+0000 7fdecced0700 1 --2- 172.21.15.29:0/1539053874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdec8157530 0x7fdec8068470 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.611+0000 7fdecced0700 1 --2- 172.21.15.29:0/1539053874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec8068960 0x7fdec8066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.611+0000 7fdecced0700 1 -- 172.21.15.29:0/1539053874 >> 172.21.15.29:0/1539053874 conn(0x7fdec80c8ba0 msgr2=0x7fdec80c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.611+0000 7fdecced0700 1 -- 172.21.15.29:0/1539053874 shutdown_connections 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.611+0000 7fdecced0700 1 -- 172.21.15.29:0/1539053874 wait complete. 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.613+0000 7fdecced0700 1 Processor -- start 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.613+0000 7fdecced0700 1 -- start start 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.613+0000 7fdecced0700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec8066a10 0x7fdec813c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.613+0000 7fdecced0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdec8068960 0x7fdec80a7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.613+0000 7fdecced0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec8157530 0x7fdec8157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.613+0000 7fdecced0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8168910 con 0x7fdec8068960 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.613+0000 7fdecced0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec80a6290 con 0x7fdec8157530 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.613+0000 7fdecced0700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec80a6570 con 0x7fdec8066a10 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdec8068960 0x7fdec80a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdec8068960 0x7fdec80a7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38472/0 (socket says 172.21.15.29:38472) 2024-01-24T15:32:35.622 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec5d9b700 1 -- 172.21.15.29:0/1730124799 learned_addr learned my addr 172.21.15.29:0/1730124799 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec659c700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec8066a10 0x7fdec813c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec5d9b700 1 -- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec8066a10 msgr2=0x7fdec813c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec6d9d700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec8157530 0x7fdec8157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec5d9b700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec8066a10 0x7fdec813c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec5d9b700 1 -- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec8157530 msgr2=0x7fdec8157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec5d9b700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec8157530 0x7fdec8157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec5d9b700 1 -- 172.21.15.29:0/1730124799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb8009370 con 0x7fdec8068960 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec659c700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec8066a10 0x7fdec813c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec6d9d700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec8157530 0x7fdec8157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdec5d9b700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdec8068960 0x7fdec80a7050 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 rx=0x7fdeb0006fd0 tx=0x7fdeb0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdebe7fc700 1 -- 172.21.15.29:0/1730124799 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0017070 con 0x7fdec8068960 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdebe7fc700 1 -- 172.21.15.29:0/1730124799 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb00125f0 con 0x7fdec8068960 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdecced0700 1 -- 172.21.15.29:0/1730124799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec80a6850 con 0x7fdec8068960 2024-01-24T15:32:35.623 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdebe7fc700 1 -- 172.21.15.29:0/1730124799 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0012970 con 0x7fdec8068960 2024-01-24T15:32:35.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.614+0000 7fdecced0700 1 -- 172.21.15.29:0/1730124799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec813c9d0 con 0x7fdec8068960 2024-01-24T15:32:35.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.616+0000 7fdecced0700 1 -- 172.21.15.29:0/1730124799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec80475e0 con 0x7fdec8068960 2024-01-24T15:32:35.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.616+0000 7fdebe7fc700 1 -- 172.21.15.29:0/1730124799 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 101) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fdeb0012d20 con 0x7fdec8068960 2024-01-24T15:32:35.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.616+0000 7fdebe7fc700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdea805f080 0x7fdea8061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:35.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.616+0000 7fdebe7fc700 1 -- 172.21.15.29:0/1730124799 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7fdeb007a2c0 con 0x7fdec8068960 2024-01-24T15:32:35.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.617+0000 7fdec659c700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdea805f080 0x7fdea8061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:35.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.617+0000 7fdec659c700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdea805f080 0x7fdea8061540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fdeb8009130 tx=0x7fdeb8017000).ready entity=mgr.9694 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:35.624 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.621+0000 7fdebe7fc700 1 -- 172.21.15.29:0/1730124799 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fdeb004b220 con 0x7fdec8068960 2024-01-24T15:32:35.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:35.850+0000 7fdecced0700 1 -- 172.21.15.29:0/1730124799 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fdec8156d80 con 0x7fdec8068960 2024-01-24T15:32:36.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.189+0000 7fdebe7fc700 1 -- 172.21.15.29:0/1730124799 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fdeb0077d20 con 0x7fdec8068960 2024-01-24T15:32:36.192 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.190+0000 7fdebe7fc700 1 -- 172.21.15.29:0/1730124799 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v102) v1 ==== 93+0+0 (secure 0 0 0) 0x7fdeb0072600 con 0x7fdec8068960 2024-01-24T15:32:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.193+0000 7fdecced0700 1 -- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdea805f080 msgr2=0x7fdea8061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.193+0000 7fdecced0700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdea805f080 0x7fdea8061540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fdeb8009130 tx=0x7fdeb8017000).stop 2024-01-24T15:32:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.193+0000 7fdecced0700 1 -- 172.21.15.29:0/1730124799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdec8068960 msgr2=0x7fdec80a7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.193+0000 7fdecced0700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdec8068960 0x7fdec80a7050 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 rx=0x7fdeb0006fd0 tx=0x7fdeb0008040).stop 2024-01-24T15:32:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.193+0000 7fdecced0700 1 -- 172.21.15.29:0/1730124799 shutdown_connections 2024-01-24T15:32:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.193+0000 7fdecced0700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec8157530 0x7fdec8157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.193+0000 7fdecced0700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fdea805f080 0x7fdea8061540 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.193+0000 7fdecced0700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec8066a10 0x7fdec813c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.193+0000 7fdecced0700 1 --2- 172.21.15.29:0/1730124799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdec8068960 0x7fdec80a7050 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.193+0000 7fdecced0700 1 -- 172.21.15.29:0/1730124799 >> 172.21.15.29:0/1730124799 conn(0x7fdec80c8ba0 msgr2=0x7fdec80c5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:36.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.193+0000 7fdecced0700 1 -- 172.21.15.29:0/1730124799 shutdown_connections 2024-01-24T15:32:36.196 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.193+0000 7fdecced0700 1 -- 172.21.15.29:0/1730124799 wait complete. 2024-01-24T15:32:36.211 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-24T15:32:36.211 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:32:36.268 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-24T15:32:36.313 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-24T15:32:36.313 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-24T15:32:36.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-24T15:32:36.314 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-24T15:32:36.348 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:36.345+0000 7f51ebb85680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:32:36.359 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:36.357+0000 7f18d2708680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T15:32:36.361 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T15:32:36.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:36.361 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-24T15:32:36.380 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:32:36.412 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:36.409+0000 7f51ebb85680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:32:36.432 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:36.431+0000 7f18d2708680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T15:32:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.549+0000 7f04232ca700 1 Processor -- start 2024-01-24T15:32:36.552 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.550+0000 7f04232ca700 1 -- start start 2024-01-24T15:32:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.551+0000 7f04232ca700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f041c0c5410 0x7f041c0aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.551+0000 7f04232ca700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c0aa6c0 con 0x7f041c154370 2024-01-24T15:32:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.551+0000 7f04232ca700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c064240 con 0x7f041c154750 2024-01-24T15:32:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.551+0000 7f04232ca700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c154ba0 con 0x7f041c0c5410 2024-01-24T15:32:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.551+0000 7f0420865700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f041c0c5410 0x7f041c0aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.551+0000 7f0421066700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f041c154370 0x7f041c0ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37184/0 (socket says 172.21.15.29:37184) 2024-01-24T15:32:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.551+0000 7f0420865700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f041c0c5410 0x7f041c0aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38498/0 (socket says 172.21.15.29:38498) 2024-01-24T15:32:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.551+0000 7f0421066700 1 -- 172.21.15.29:0/1181294767 learned_addr learned my addr 172.21.15.29:0/1181294767 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.551+0000 7f0420865700 1 -- 172.21.15.29:0/1181294767 >> v1:172.21.15.163:6789/0 conn(0x7f041c154750 legacy=0x7f041c0631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.551+0000 7f0420865700 1 -- 172.21.15.29:0/1181294767 >> v1:172.21.15.29:6789/0 conn(0x7f041c154370 legacy=0x7f041c0ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.551+0000 7f0420865700 1 -- 172.21.15.29:0/1181294767 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c0cb5c0 con 0x7f041c0c5410 2024-01-24T15:32:36.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.551+0000 7f0420865700 1 --2- 172.21.15.29:0/1181294767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f041c0c5410 0x7f041c0aa0e0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 rx=0x7f040c009760 tx=0x7f040c009060).ready entity=mon.0 client_cookie=5a03b43dea4f36da server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.552+0000 7f0413fff700 1 -- 172.21.15.29:0/1181294767 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c00b580 con 0x7f041c0c5410 2024-01-24T15:32:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.552+0000 7f0413fff700 1 -- 172.21.15.29:0/1181294767 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c00b6e0 con 0x7f041c0c5410 2024-01-24T15:32:36.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.552+0000 7f0413fff700 1 -- 172.21.15.29:0/1181294767 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c00ba60 con 0x7f041c0c5410 2024-01-24T15:32:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.552+0000 7f04232ca700 1 -- 172.21.15.29:0/1181294767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f041c0c5410 msgr2=0x7f041c0aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.552+0000 7f04232ca700 1 --2- 172.21.15.29:0/1181294767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f041c0c5410 0x7f041c0aa0e0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 rx=0x7f040c009760 tx=0x7f040c009060).stop 2024-01-24T15:32:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.552+0000 7f04232ca700 1 -- 172.21.15.29:0/1181294767 shutdown_connections 2024-01-24T15:32:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.552+0000 7f04232ca700 1 --2- 172.21.15.29:0/1181294767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f041c0c5410 0x7f041c0aa0e0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.552+0000 7f04232ca700 1 -- 172.21.15.29:0/1181294767 >> 172.21.15.29:0/1181294767 conn(0x7f041c05ec80 msgr2=0x7f041c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.552+0000 7f04232ca700 1 -- 172.21.15.29:0/1181294767 shutdown_connections 2024-01-24T15:32:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.552+0000 7f04232ca700 1 -- 172.21.15.29:0/1181294767 wait complete. 2024-01-24T15:32:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.554+0000 7f04232ca700 1 Processor -- start 2024-01-24T15:32:36.556 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.554+0000 7f04232ca700 1 -- start start 2024-01-24T15:32:36.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.555+0000 7f04232ca700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f041c0c5410 0x7f041c14cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.555+0000 7f04232ca700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f041c154370 0x7f041c165620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.555+0000 7f04232ca700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f041c154750 0x7f041c0c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.555+0000 7f04232ca700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c165b10 con 0x7f041c154750 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.555+0000 7f04232ca700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c165190 con 0x7f041c0c5410 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.555+0000 7f04232ca700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c0c4ad0 con 0x7f041c154370 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f0421867700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f041c154750 0x7f041c0c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f0421867700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f041c154750 0x7f041c0c3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38510/0 (socket says 172.21.15.29:38510) 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f0421867700 1 -- 172.21.15.29:0/2632586905 learned_addr learned my addr 172.21.15.29:0/2632586905 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f0421066700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f041c0c5410 0x7f041c14cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f0420865700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f041c154370 0x7f041c165620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f0421867700 1 -- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f041c154370 msgr2=0x7f041c165620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f0421867700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f041c154370 0x7f041c165620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.559 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f0421867700 1 -- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f041c0c5410 msgr2=0x7f041c14cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f0421867700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f041c0c5410 0x7f041c14cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f0421867700 1 -- 172.21.15.29:0/2632586905 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c0092a0 con 0x7f041c154750 2024-01-24T15:32:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f0421867700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f041c154750 0x7f041c0c3f80 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 rx=0x7f041400c7c0 tx=0x7f0414001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f04117fa700 1 -- 172.21.15.29:0/2632586905 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400e4a0 con 0x7f041c154750 2024-01-24T15:32:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f04117fa700 1 -- 172.21.15.29:0/2632586905 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041400e600 con 0x7f041c154750 2024-01-24T15:32:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f04232ca700 1 -- 172.21.15.29:0/2632586905 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c0c4db0 con 0x7f041c154750 2024-01-24T15:32:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f04117fa700 1 -- 172.21.15.29:0/2632586905 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400e980 con 0x7f041c154750 2024-01-24T15:32:36.560 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.556+0000 7f04232ca700 1 -- 172.21.15.29:0/2632586905 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c164ad0 con 0x7f041c154750 2024-01-24T15:32:36.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.558+0000 7f04232ca700 1 -- 172.21.15.29:0/2632586905 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c0475e0 con 0x7f041c154750 2024-01-24T15:32:36.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.558+0000 7f04117fa700 1 -- 172.21.15.29:0/2632586905 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f04140123f0 con 0x7f041c154750 2024-01-24T15:32:36.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.558+0000 7f04117fa700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f040005f2d0 0x7f0400061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:36.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.558+0000 7f0421066700 1 -- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f040005f2d0 msgr2=0x7f0400061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:36.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.558+0000 7f04117fa700 1 -- 172.21.15.29:0/2632586905 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7f041407a080 con 0x7f041c154750 2024-01-24T15:32:36.561 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.558+0000 7f0421066700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f040005f2d0 0x7f0400061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:36.564 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.562+0000 7f04117fa700 1 -- 172.21.15.29:0/2632586905 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f0414047b00 con 0x7f041c154750 2024-01-24T15:32:36.687 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:36.684+0000 7f51ebb85680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:32:36.754 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:36.752+0000 7f18d2708680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T15:32:36.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.759+0000 7f0421066700 1 -- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f040005f2d0 msgr2=0x7f0400061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:36.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.759+0000 7f0421066700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f040005f2d0 0x7f0400061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:32:36.799 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.797+0000 7f04232ca700 1 -- 172.21.15.29:0/2632586905 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f041c0c4890 con 0x7f041c154750 2024-01-24T15:32:36.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.798+0000 7f04117fa700 1 -- 172.21.15.29:0/2632586905 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4499 (secure 0 0 0) 0x7f041c0c4890 con 0x7f041c154750 2024-01-24T15:32:36.800 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:36.800 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1883755467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1883755467},{"type":"v1","addr":"172.21.15.163:6836","nonce":1883755467}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1773668659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1773668659},{"type":"v1","addr":"172.21.15.29:6834","nonce":1773668659}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3438722509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3438722509},{"type":"v1","addr":"172.21.15.29:6835","nonce":3438722509}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:32:20.685536+0000","modified":"2024-01-24T15:32:23.705717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10029":{"gid":10029,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/532321390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":532321390},{"type":"v1","addr":"172.21.15.163:6837","nonce":532321390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2024-01-24T15:32:36.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.800+0000 7f04232ca700 1 -- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f040005f2d0 msgr2=0x7f0400061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:36.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.800+0000 7f04232ca700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f040005f2d0 0x7f0400061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:36.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.800+0000 7f04232ca700 1 -- 172.21.15.29:0/2632586905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f041c154750 msgr2=0x7f041c0c3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.800+0000 7f04232ca700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f041c154750 0x7f041c0c3f80 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 rx=0x7f041400c7c0 tx=0x7f0414001510).stop 2024-01-24T15:32:36.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.800+0000 7f04232ca700 1 -- 172.21.15.29:0/2632586905 shutdown_connections 2024-01-24T15:32:36.802 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.800+0000 7f04232ca700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f041c154750 0x7f041c0c3f80 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.800+0000 7f04232ca700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f041c0c5410 0x7f041c14cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.800+0000 7f04232ca700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f040005f2d0 0x7f0400061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:36.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.800+0000 7f04232ca700 1 --2- 172.21.15.29:0/2632586905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f041c154370 0x7f041c165620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.800+0000 7f04232ca700 1 -- 172.21.15.29:0/2632586905 >> 172.21.15.29:0/2632586905 conn(0x7f041c05ec80 msgr2=0x7f041c0c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:36.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.800+0000 7f04232ca700 1 -- 172.21.15.29:0/2632586905 shutdown_connections 2024-01-24T15:32:36.803 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.800+0000 7f04232ca700 1 -- 172.21.15.29:0/2632586905 wait complete. 2024-01-24T15:32:36.803 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 98 2024-01-24T15:32:36.817 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:32:36.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.987+0000 7f066202e700 1 Processor -- start 2024-01-24T15:32:36.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.987+0000 7f066202e700 1 -- start start 2024-01-24T15:32:36.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.988+0000 7f066202e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f065c154370 0x7f065c0c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:36.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.988+0000 7f066202e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c0aa6c0 con 0x7f065c0aa0e0 2024-01-24T15:32:36.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.988+0000 7f066202e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c064240 con 0x7f065c154750 2024-01-24T15:32:36.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.988+0000 7f066202e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0cb5c0 con 0x7f065c154370 2024-01-24T15:32:36.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.988+0000 7f065affd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f065c0aa0e0 0x7f065c0ab1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37200/0 (socket says 172.21.15.29:37200) 2024-01-24T15:32:36.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.988+0000 7f065affd700 1 -- 172.21.15.29:0/800831823 learned_addr learned my addr 172.21.15.29:0/800831823 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:36.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.988+0000 7f065b7fe700 1 --2- 172.21.15.29:0/800831823 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f065c154370 0x7f065c0c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:36.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.989+0000 7f065a7fc700 1 -- 172.21.15.29:0/800831823 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691861943 0 0) 0x7f065c0aa6c0 con 0x7f065c0aa0e0 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.989+0000 7f065a7fc700 1 -- 172.21.15.29:0/800831823 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06440033f0 con 0x7f065c0aa0e0 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.989+0000 7f065a7fc700 1 -- 172.21.15.29:0/800831823 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115536259 0 0) 0x7f065c064240 con 0x7f065c154750 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.989+0000 7f065a7fc700 1 -- 172.21.15.29:0/800831823 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f065c0aa6c0 con 0x7f065c154750 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.989+0000 7f065b7fe700 1 -- 172.21.15.29:0/800831823 >> v1:172.21.15.163:6789/0 conn(0x7f065c154750 legacy=0x7f065c0631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.989+0000 7f065b7fe700 1 -- 172.21.15.29:0/800831823 >> v1:172.21.15.29:6789/0 conn(0x7f065c0aa0e0 legacy=0x7f065c0ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.989+0000 7f065b7fe700 1 -- 172.21.15.29:0/800831823 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c0cb8a0 con 0x7f065c154370 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.989+0000 7f065b7fe700 1 --2- 172.21.15.29:0/800831823 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f065c154370 0x7f065c0c5410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f064c009ee0 tx=0x7f064c009780).ready entity=mon.2 client_cookie=420ac92b9c10b36a server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.989+0000 7f065a7fc700 1 -- 172.21.15.29:0/800831823 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c00e070 con 0x7f065c154370 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.989+0000 7f065a7fc700 1 -- 172.21.15.29:0/800831823 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c00bb80 con 0x7f065c154370 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.989+0000 7f065a7fc700 1 -- 172.21.15.29:0/800831823 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c012410 con 0x7f065c154370 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.990+0000 7f066202e700 1 -- 172.21.15.29:0/800831823 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f065c154370 msgr2=0x7f065c0c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.990+0000 7f066202e700 1 --2- 172.21.15.29:0/800831823 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f065c154370 0x7f065c0c5410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f064c009ee0 tx=0x7f064c009780).stop 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.990+0000 7f066202e700 1 -- 172.21.15.29:0/800831823 shutdown_connections 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.990+0000 7f066202e700 1 --2- 172.21.15.29:0/800831823 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f065c154370 0x7f065c0c5410 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.993 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.990+0000 7f066202e700 1 -- 172.21.15.29:0/800831823 >> 172.21.15.29:0/800831823 conn(0x7f065c05ec80 msgr2=0x7f065c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:36.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.990+0000 7f066202e700 1 -- 172.21.15.29:0/800831823 shutdown_connections 2024-01-24T15:32:36.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.990+0000 7f066202e700 1 -- 172.21.15.29:0/800831823 wait complete. 2024-01-24T15:32:36.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.991+0000 7f066202e700 1 Processor -- start 2024-01-24T15:32:36.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.991+0000 7f066202e700 1 -- start start 2024-01-24T15:32:36.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.992+0000 7f066202e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f065c0aa0e0 0x7f065c164a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:36.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.992+0000 7f066202e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f065c154370 0x7f065c0ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:36.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.992+0000 7f066202e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f065c154750 0x7f065c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:36.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.992+0000 7f066202e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0af540 con 0x7f065c0aa0e0 2024-01-24T15:32:36.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.992+0000 7f066202e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0adff0 con 0x7f065c154750 2024-01-24T15:32:36.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.992+0000 7f066202e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0ae930 con 0x7f065c154370 2024-01-24T15:32:36.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.992+0000 7f065b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f065c0aa0e0 0x7f065c164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:36.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.992+0000 7f065b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f065c0aa0e0 0x7f065c164a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38518/0 (socket says 172.21.15.29:38518) 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.992+0000 7f065b7fe700 1 -- 172.21.15.29:0/519885648 learned_addr learned my addr 172.21.15.29:0/519885648 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.992+0000 7f065affd700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f065c154370 0x7f065c0ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.992+0000 7f065bfff700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f065c154750 0x7f065c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.993+0000 7f065b7fe700 1 -- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f065c154370 msgr2=0x7f065c0ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.993+0000 7f065b7fe700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f065c154370 0x7f065c0ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.993+0000 7f065b7fe700 1 -- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f065c154750 msgr2=0x7f065c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.993+0000 7f065b7fe700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f065c154750 0x7f065c0b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.993+0000 7f065b7fe700 1 -- 172.21.15.29:0/519885648 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c009a20 con 0x7f065c0aa0e0 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.993+0000 7f065b7fe700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f065c0aa0e0 0x7f065c164a40 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 rx=0x7f065c0c8920 tx=0x7f065c0643d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.993+0000 7f06437fe700 1 -- 172.21.15.29:0/519885648 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c01d420 con 0x7f065c0aa0e0 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.993+0000 7f06437fe700 1 -- 172.21.15.29:0/519885648 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c01d5a0 con 0x7f065c0aa0e0 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.993+0000 7f06437fe700 1 -- 172.21.15.29:0/519885648 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c01d960 con 0x7f065c0aa0e0 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.993+0000 7f066202e700 1 -- 172.21.15.29:0/519885648 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c0aec10 con 0x7f065c0aa0e0 2024-01-24T15:32:36.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.993+0000 7f066202e700 1 -- 172.21.15.29:0/519885648 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065c0af1d0 con 0x7f065c0aa0e0 2024-01-24T15:32:36.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.995+0000 7f06437fe700 1 -- 172.21.15.29:0/519885648 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f064c01dcd0 con 0x7f065c0aa0e0 2024-01-24T15:32:36.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.995+0000 7f066202e700 1 -- 172.21.15.29:0/519885648 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065c0c56c0 con 0x7f065c0aa0e0 2024-01-24T15:32:36.998 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.995+0000 7f06437fe700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f064805f2d0 0x7f0648061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:36.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.995+0000 7f06437fe700 1 -- 172.21.15.29:0/519885648 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7f064c07c710 con 0x7f065c0aa0e0 2024-01-24T15:32:36.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.995+0000 7f065affd700 1 -- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f064805f2d0 msgr2=0x7f0648061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:36.999 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:36.995+0000 7f065affd700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f064805f2d0 0x7f0648061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:37.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.004+0000 7f06437fe700 1 -- 172.21.15.29:0/519885648 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f064c04d230 con 0x7f065c0aa0e0 2024-01-24T15:32:37.189 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.187+0000 7f066202e700 1 -- 172.21.15.29:0/519885648 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f065c157f60 con 0x7f065c0aa0e0 2024-01-24T15:32:37.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.188+0000 7f06437fe700 1 -- 172.21.15.29:0/519885648 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v232) v1 ==== 74+0+18673 (secure 0 0 0) 0x7f064c050850 con 0x7f065c0aa0e0 2024-01-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:37.191 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":232,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:32:33.173608+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:32:18.425709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-01-24T15:32:19.656025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"231","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/343370470":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/3387006926":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/741220992":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/2316406386":"2024-01-25T15:32:33.173546+0000","172.21.15.163:6833/57365":"2024-01-25T15:32:33.173546+0000","172.21.15.163:6832/57365":"2024-01-25T15:32:33.173546+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.190+0000 7f066202e700 1 -- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f064805f2d0 msgr2=0x7f0648061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.190+0000 7f066202e700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f064805f2d0 0x7f0648061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.190+0000 7f066202e700 1 -- 172.21.15.29:0/519885648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f065c0aa0e0 msgr2=0x7f065c164a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.190+0000 7f066202e700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f065c0aa0e0 0x7f065c164a40 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 rx=0x7f065c0c8920 tx=0x7f065c0643d0).stop 2024-01-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.190+0000 7f066202e700 1 -- 172.21.15.29:0/519885648 shutdown_connections 2024-01-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.190+0000 7f066202e700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f065c154750 0x7f065c0b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.190+0000 7f066202e700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f065c0aa0e0 0x7f065c164a40 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.190+0000 7f066202e700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f064805f2d0 0x7f0648061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:37.193 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.190+0000 7f066202e700 1 --2- 172.21.15.29:0/519885648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f065c154370 0x7f065c0ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.190+0000 7f066202e700 1 -- 172.21.15.29:0/519885648 >> 172.21.15.29:0/519885648 conn(0x7f065c05ec80 msgr2=0x7f065c150ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.191+0000 7f066202e700 1 -- 172.21.15.29:0/519885648 shutdown_connections 2024-01-24T15:32:37.194 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.191+0000 7f066202e700 1 -- 172.21.15.29:0/519885648 wait complete. 2024-01-24T15:32:37.207 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-01-24T13:58:09.781701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-01-24T13:58:12.196460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-24T15:32:18.425709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '230', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-01-24T15:32:19.656025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '231', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-24T15:32:37.208 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-01-24T15:32:37.306 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:37.304+0000 7f51ebb85680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:32:37.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.342+0000 7fc2336f4700 1 Processor -- start 2024-01-24T15:32:37.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.343+0000 7fc2336f4700 1 -- start start 2024-01-24T15:32:37.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.343+0000 7fc2336f4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc22c064760 0x7fc22c064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc2336f4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc22c0bfd40 0x7fc22c0c01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.347 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc2336f4700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c152240 con 0x7fc22c0c8490 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc2336f4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22c1586c0 con 0x7fc22c0bfd40 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc2336f4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0aa670 con 0x7fc22c064760 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc231c91700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc22c0bfd40 0x7fc22c0c01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc231c91700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc22c0bfd40 0x7fc22c0c01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38526/0 (socket says 172.21.15.29:38526) 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc231c91700 1 -- 172.21.15.29:0/3783635934 learned_addr learned my addr 172.21.15.29:0/3783635934 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc230c8f700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc22c0c8490 0x7fc22c0a9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37202/0 (socket says 172.21.15.29:37202) 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc231490700 1 --2- 172.21.15.29:0/3783635934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc22c064760 0x7fc22c064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc231c91700 1 -- 172.21.15.29:0/3783635934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc22c064760 msgr2=0x7fc22c064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc231c91700 1 --2- 172.21.15.29:0/3783635934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc22c064760 0x7fc22c064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc231c91700 1 -- 172.21.15.29:0/3783635934 >> v1:172.21.15.29:6789/0 conn(0x7fc22c0c8490 legacy=0x7fc22c0a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc231c91700 1 -- 172.21.15.29:0/3783635934 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c157d90 con 0x7fc22c0bfd40 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc223fff700 1 -- 172.21.15.29:0/3783635934 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111614549 0 0) 0x7fc22c152240 con 0x7fc22c0c8490 2024-01-24T15:32:37.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc231c91700 1 --2- 172.21.15.29:0/3783635934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22c0bfd40 0x7fc22c0c01f0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 rx=0x7fc224009f20 tx=0x7fc224009840).ready entity=mon.0 client_cookie=513f4722ddb45d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:37.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc223fff700 1 -- 172.21.15.29:0/3783635934 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22400e070 con 0x7fc22c0bfd40 2024-01-24T15:32:37.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.344+0000 7fc223fff700 1 -- 172.21.15.29:0/3783635934 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22400ba60 con 0x7fc22c0bfd40 2024-01-24T15:32:37.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.345+0000 7fc223fff700 1 -- 172.21.15.29:0/3783635934 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22400bde0 con 0x7fc22c0bfd40 2024-01-24T15:32:37.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.345+0000 7fc2336f4700 1 -- 172.21.15.29:0/3783635934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22c0bfd40 msgr2=0x7fc22c0c01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:37.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.345+0000 7fc2336f4700 1 --2- 172.21.15.29:0/3783635934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22c0bfd40 0x7fc22c0c01f0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 rx=0x7fc224009f20 tx=0x7fc224009840).stop 2024-01-24T15:32:37.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.345+0000 7fc2336f4700 1 -- 172.21.15.29:0/3783635934 shutdown_connections 2024-01-24T15:32:37.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.345+0000 7fc2336f4700 1 --2- 172.21.15.29:0/3783635934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22c0bfd40 0x7fc22c0c01f0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.345+0000 7fc2336f4700 1 --2- 172.21.15.29:0/3783635934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc22c064760 0x7fc22c064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.345+0000 7fc2336f4700 1 -- 172.21.15.29:0/3783635934 >> 172.21.15.29:0/3783635934 conn(0x7fc22c0cb8b0 msgr2=0x7fc22c0c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:37.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.345+0000 7fc2336f4700 1 -- 172.21.15.29:0/3783635934 shutdown_connections 2024-01-24T15:32:37.349 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.345+0000 7fc2336f4700 1 -- 172.21.15.29:0/3783635934 wait complete. 2024-01-24T15:32:37.351 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.346+0000 7fc2336f4700 1 Processor -- start 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.347+0000 7fc2336f4700 1 -- start start 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.347+0000 7fc2336f4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22c064760 0x7fc22c0add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.347+0000 7fc2336f4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc22c0bfd40 0x7fc22c1653c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc2336f4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc22c0c8490 0x7fc22c0ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc2336f4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0af260 con 0x7fc22c064760 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc2336f4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c1658b0 con 0x7fc22c0c8490 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc2336f4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0afd50 con 0x7fc22c0bfd40 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc231490700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22c064760 0x7fc22c0add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc231490700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22c064760 0x7fc22c0add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38540/0 (socket says 172.21.15.29:38540) 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc231490700 1 -- 172.21.15.29:0/1957456275 learned_addr learned my addr 172.21.15.29:0/1957456275 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc231c91700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc22c0c8490 0x7fc22c0ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc230c8f700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc22c0bfd40 0x7fc22c1653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc231490700 1 -- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc22c0bfd40 msgr2=0x7fc22c1653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:37.352 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc231490700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc22c0bfd40 0x7fc22c1653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc231490700 1 -- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc22c0c8490 msgr2=0x7fc22c0ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:37.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc231490700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc22c0c8490 0x7fc22c0ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc231490700 1 -- 172.21.15.29:0/1957456275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc224009ae0 con 0x7fc22c064760 2024-01-24T15:32:37.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc231c91700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc22c0c8490 0x7fc22c0ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:37.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc231490700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22c064760 0x7fc22c0add90 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 rx=0x7fc22800e900 tx=0x7fc228009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:37.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc2217fa700 1 -- 172.21.15.29:0/1957456275 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228012600 con 0x7fc22c064760 2024-01-24T15:32:37.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc2217fa700 1 -- 172.21.15.29:0/1957456275 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc228012760 con 0x7fc22c064760 2024-01-24T15:32:37.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc2336f4700 1 -- 172.21.15.29:0/1957456275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c0b08d0 con 0x7fc22c064760 2024-01-24T15:32:37.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.348+0000 7fc2217fa700 1 -- 172.21.15.29:0/1957456275 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228012ae0 con 0x7fc22c064760 2024-01-24T15:32:37.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.349+0000 7fc2336f4700 1 -- 172.21.15.29:0/1957456275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c0b0020 con 0x7fc22c064760 2024-01-24T15:32:37.353 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.350+0000 7fc2336f4700 1 -- 172.21.15.29:0/1957456275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c0475e0 con 0x7fc22c064760 2024-01-24T15:32:37.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.350+0000 7fc2217fa700 1 -- 172.21.15.29:0/1957456275 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fc22801b400 con 0x7fc22c064760 2024-01-24T15:32:37.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.350+0000 7fc2217fa700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc21005f300 0x7fc2100617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.351+0000 7fc2217fa700 1 -- 172.21.15.29:0/1957456275 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7fc22807ad30 con 0x7fc22c064760 2024-01-24T15:32:37.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.351+0000 7fc230c8f700 1 -- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc21005f300 msgr2=0x7fc2100617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:37.354 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.351+0000 7fc230c8f700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc21005f300 0x7fc2100617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:37.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.359+0000 7fc2217fa700 1 -- 172.21.15.29:0/1957456275 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc22804b8e0 con 0x7fc22c064760 2024-01-24T15:32:37.380 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:37.377+0000 7f51ebb85680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:32:37.492 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:37.490+0000 7f18d2708680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T15:32:37.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.552+0000 7fc230c8f700 1 -- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc21005f300 msgr2=0x7fc2100617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:37.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.552+0000 7fc230c8f700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc21005f300 0x7fc2100617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:32:37.566 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:37.563+0000 7f51ebb85680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:32:37.566 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:37.564+0000 7f18d2708680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T15:32:37.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.585+0000 7fc2336f4700 1 -- 172.21.15.29:0/1957456275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc22c0657a0 con 0x7fc22c064760 2024-01-24T15:32:37.588 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.586+0000 7fc2217fa700 1 -- 172.21.15.29:0/1957456275 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v232) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc22804ef00 con 0x7fc22c064760 2024-01-24T15:32:37.588 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 1 2024-01-24T15:32:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.588+0000 7fc2336f4700 1 -- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc21005f300 msgr2=0x7fc2100617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.588+0000 7fc2336f4700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc21005f300 0x7fc2100617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.588+0000 7fc2336f4700 1 -- 172.21.15.29:0/1957456275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22c064760 msgr2=0x7fc22c0add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.588+0000 7fc2336f4700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22c064760 0x7fc22c0add90 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 rx=0x7fc22800e900 tx=0x7fc228009060).stop 2024-01-24T15:32:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.588+0000 7fc2336f4700 1 -- 172.21.15.29:0/1957456275 shutdown_connections 2024-01-24T15:32:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.588+0000 7fc2336f4700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc22c0c8490 0x7fc22c0ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.588+0000 7fc2336f4700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc22c064760 0x7fc22c0add90 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.588+0000 7fc2336f4700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc21005f300 0x7fc2100617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.588+0000 7fc2336f4700 1 --2- 172.21.15.29:0/1957456275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc22c0bfd40 0x7fc22c1653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.588+0000 7fc2336f4700 1 -- 172.21.15.29:0/1957456275 >> 172.21.15.29:0/1957456275 conn(0x7fc22c0cb8b0 msgr2=0x7fc22c0ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.588+0000 7fc2336f4700 1 -- 172.21.15.29:0/1957456275 shutdown_connections 2024-01-24T15:32:37.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.588+0000 7fc2336f4700 1 -- 172.21.15.29:0/1957456275 wait complete. 2024-01-24T15:32:37.604 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T15:32:37.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.738+0000 7f3018ca9700 1 Processor -- start 2024-01-24T15:32:37.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.738+0000 7f3018ca9700 1 -- start start 2024-01-24T15:32:37.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3018ca9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3014134ea0 0x7f30140c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3018ca9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140c36c0 0x7f30140664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3018ca9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30140669b0 0x7f30141559f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3018ca9700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30140686a0 con 0x7f30140669b0 2024-01-24T15:32:37.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3018ca9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3014066e30 con 0x7f3014134ea0 2024-01-24T15:32:37.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3018ca9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3014068b30 con 0x7f30140c36c0 2024-01-24T15:32:37.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3012d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30140669b0 0x7f30141559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:37.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3012d9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30140669b0 0x7f30141559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38544/0 (socket says 172.21.15.29:38544) 2024-01-24T15:32:37.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3012d9d700 1 -- 172.21.15.29:0/1111255017 learned_addr learned my addr 172.21.15.29:0/1111255017 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:37.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3011d9b700 1 --2- 172.21.15.29:0/1111255017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140c36c0 0x7f30140664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:37.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f301259c700 1 --2- 172.21.15.29:0/1111255017 >> v2:172.21.15.163:3300/0 conn(0x7f3014134ea0 0x7f30140c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:37.744 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3012d9d700 1 -- 172.21.15.29:0/1111255017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140c36c0 msgr2=0x7f30140664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3012d9d700 1 --2- 172.21.15.29:0/1111255017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140c36c0 0x7f30140664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3012d9d700 1 -- 172.21.15.29:0/1111255017 >> v2:172.21.15.163:3300/0 conn(0x7f3014134ea0 msgr2=0x7f30140c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3012d9d700 1 --2- 172.21.15.29:0/1111255017 >> v2:172.21.15.163:3300/0 conn(0x7f3014134ea0 0x7f30140c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3012d9d700 1 -- 172.21.15.29:0/1111255017 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3014067a90 con 0x7f30140669b0 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f3011d9b700 1 --2- 172.21.15.29:0/1111255017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140c36c0 0x7f30140664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.740+0000 7f301259c700 1 --2- 172.21.15.29:0/1111255017 >> v2:172.21.15.163:3300/0 conn(0x7f3014134ea0 0x7f30140c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.741+0000 7f3012d9d700 1 --2- 172.21.15.29:0/1111255017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140669b0 0x7f30141559f0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 rx=0x7f30140c66e0 tx=0x7f300c009840).ready entity=mon.0 client_cookie=ee21d2b4249cca6e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.741+0000 7f301159a700 1 -- 172.21.15.29:0/1111255017 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c00e070 con 0x7f30140669b0 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.741+0000 7f301159a700 1 -- 172.21.15.29:0/1111255017 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c00b970 con 0x7f30140669b0 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.741+0000 7f301159a700 1 -- 172.21.15.29:0/1111255017 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c00bcf0 con 0x7f30140669b0 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.741+0000 7f3018ca9700 1 -- 172.21.15.29:0/1111255017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140669b0 msgr2=0x7f30141559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.741+0000 7f3018ca9700 1 --2- 172.21.15.29:0/1111255017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140669b0 0x7f30141559f0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 rx=0x7f30140c66e0 tx=0x7f300c009840).stop 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.742+0000 7f3018ca9700 1 -- 172.21.15.29:0/1111255017 shutdown_connections 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.742+0000 7f3018ca9700 1 --2- 172.21.15.29:0/1111255017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30140669b0 0x7f30141559f0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.742+0000 7f3018ca9700 1 --2- 172.21.15.29:0/1111255017 >> v2:172.21.15.163:3300/0 conn(0x7f3014134ea0 0x7f30140c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.742+0000 7f3018ca9700 1 --2- 172.21.15.29:0/1111255017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140c36c0 0x7f30140664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.742+0000 7f3018ca9700 1 -- 172.21.15.29:0/1111255017 >> 172.21.15.29:0/1111255017 conn(0x7f30140c8bd0 msgr2=0x7f30140c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:37.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.742+0000 7f3018ca9700 1 -- 172.21.15.29:0/1111255017 shutdown_connections 2024-01-24T15:32:37.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.742+0000 7f3018ca9700 1 -- 172.21.15.29:0/1111255017 wait complete. 2024-01-24T15:32:37.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.743+0000 7f3018ca9700 1 Processor -- start 2024-01-24T15:32:37.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.743+0000 7f3018ca9700 1 -- start start 2024-01-24T15:32:37.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.744+0000 7f3018ca9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140669b0 0x7f301414f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.744+0000 7f3018ca9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30140c36c0 0x7f3014154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.744+0000 7f3018ca9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3014134ea0 0x7f30141529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.744+0000 7f3018ca9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30141553e0 con 0x7f3014134ea0 2024-01-24T15:32:37.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.744+0000 7f3018ca9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30140aec20 con 0x7f30140c36c0 2024-01-24T15:32:37.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.744+0000 7f3018ca9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301414e640 con 0x7f30140669b0 2024-01-24T15:32:37.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f3012d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3014134ea0 0x7f30141529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:37.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f3012d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3014134ea0 0x7f30141529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38548/0 (socket says 172.21.15.29:38548) 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f3012d9d700 1 -- 172.21.15.29:0/3971714285 learned_addr learned my addr 172.21.15.29:0/3971714285 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f301259c700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140669b0 0x7f301414f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f3011d9b700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30140c36c0 0x7f3014154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f3012d9d700 1 -- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140669b0 msgr2=0x7f301414f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f3012d9d700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140669b0 0x7f301414f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f3012d9d700 1 -- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30140c36c0 msgr2=0x7f3014154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f3012d9d700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30140c36c0 0x7f3014154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f3012d9d700 1 -- 172.21.15.29:0/3971714285 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30080092a0 con 0x7f3014134ea0 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f301259c700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140669b0 0x7f301414f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f3012d9d700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3014134ea0 0x7f30141529a0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 rx=0x7f300c00b610 tx=0x7f300c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f30027fc700 1 -- 172.21.15.29:0/3971714285 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c00bee0 con 0x7f3014134ea0 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f30027fc700 1 -- 172.21.15.29:0/3971714285 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c00bcf0 con 0x7f3014134ea0 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f3018ca9700 1 -- 172.21.15.29:0/3971714285 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30040092a0 con 0x7f3014134ea0 2024-01-24T15:32:37.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f30027fc700 1 -- 172.21.15.29:0/3971714285 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c01d6b0 con 0x7f3014134ea0 2024-01-24T15:32:37.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.745+0000 7f3018ca9700 1 -- 172.21.15.29:0/3971714285 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c009ae0 con 0x7f3014134ea0 2024-01-24T15:32:37.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.747+0000 7f3018ca9700 1 -- 172.21.15.29:0/3971714285 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30140475e0 con 0x7f3014134ea0 2024-01-24T15:32:37.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.747+0000 7f30027fc700 1 -- 172.21.15.29:0/3971714285 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f300c003040 con 0x7f3014134ea0 2024-01-24T15:32:37.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.748+0000 7f30027fc700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2ff405f2d0 0x7f2ff4061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:37.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.748+0000 7f30027fc700 1 -- 172.21.15.29:0/3971714285 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7f300c07baf0 con 0x7f3014134ea0 2024-01-24T15:32:37.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.748+0000 7f301259c700 1 -- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2ff405f2d0 msgr2=0x7f2ff4061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:37.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.748+0000 7f301259c700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2ff405f2d0 0x7f2ff4061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:37.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.756+0000 7f30027fc700 1 -- 172.21.15.29:0/3971714285 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f300c050070 con 0x7f3014134ea0 2024-01-24T15:32:37.793 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:37.791+0000 7f18d2708680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T15:32:37.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.940+0000 7f3018ca9700 1 -- 172.21.15.29:0/3971714285 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3014067d70 con 0x7f3014134ea0 2024-01-24T15:32:37.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.940+0000 7f30027fc700 1 -- 172.21.15.29:0/3971714285 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v232) v1 ==== 92+0+10 (secure 0 0 0) 0x7f300c022020 con 0x7f3014134ea0 2024-01-24T15:32:37.943 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 8 2024-01-24T15:32:37.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.942+0000 7f3018ca9700 1 -- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2ff405f2d0 msgr2=0x7f2ff4061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:37.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.942+0000 7f3018ca9700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2ff405f2d0 0x7f2ff4061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:37.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.942+0000 7f3018ca9700 1 -- 172.21.15.29:0/3971714285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3014134ea0 msgr2=0x7f30141529a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:37.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.942+0000 7f3018ca9700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3014134ea0 0x7f30141529a0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 rx=0x7f300c00b610 tx=0x7f300c009840).stop 2024-01-24T15:32:37.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.943+0000 7f3018ca9700 1 -- 172.21.15.29:0/3971714285 shutdown_connections 2024-01-24T15:32:37.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.943+0000 7f3018ca9700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3014134ea0 0x7f30141529a0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.943+0000 7f3018ca9700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f2ff405f2d0 0x7f2ff4061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:37.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.943+0000 7f3018ca9700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30140669b0 0x7f301414f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.943+0000 7f3018ca9700 1 --2- 172.21.15.29:0/3971714285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30140c36c0 0x7f3014154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.943+0000 7f3018ca9700 1 -- 172.21.15.29:0/3971714285 >> 172.21.15.29:0/3971714285 conn(0x7f30140c8bd0 msgr2=0x7f301414c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.943+0000 7f3018ca9700 1 -- 172.21.15.29:0/3971714285 shutdown_connections 2024-01-24T15:32:37.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:37.943+0000 7f3018ca9700 1 -- 172.21.15.29:0/3971714285 wait complete. 2024-01-24T15:32:37.959 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-24T15:32:37.989 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:37.986+0000 7f51ebb85680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:32:38.092 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:38.090+0000 7f51ebb85680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:32:38.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.093+0000 7f3896f32700 1 Processor -- start 2024-01-24T15:32:38.095 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.093+0000 7f3896f32700 1 -- start start 2024-01-24T15:32:38.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.094+0000 7f3896f32700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38900692a0 0x7f3890065e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.094+0000 7f3896f32700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3890066370 0x7f3890069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.094+0000 7f3896f32700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3890069b70 0x7f38900bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.094+0000 7f3896f32700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38900c3e70 con 0x7f38900692a0 2024-01-24T15:32:38.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.094+0000 7f3896f32700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38900c4300 con 0x7f3890069b70 2024-01-24T15:32:38.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.094+0000 7f3896f32700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38900c3260 con 0x7f3890066370 2024-01-24T15:32:38.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.094+0000 7f3894cce700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38900692a0 0x7f3890065e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.098 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.094+0000 7f3894cce700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38900692a0 0x7f3890065e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38560/0 (socket says 172.21.15.29:38560) 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.094+0000 7f3894cce700 1 -- 172.21.15.29:0/2493460387 learned_addr learned my addr 172.21.15.29:0/2493460387 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.094+0000 7f38954cf700 1 --2- 172.21.15.29:0/2493460387 >> v2:172.21.15.163:3300/0 conn(0x7f3890069b70 0x7f38900bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.094+0000 7f388ffff700 1 --2- 172.21.15.29:0/2493460387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3890066370 0x7f3890069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.094+0000 7f3894cce700 1 -- 172.21.15.29:0/2493460387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3890066370 msgr2=0x7f3890069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.095+0000 7f3894cce700 1 --2- 172.21.15.29:0/2493460387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3890066370 0x7f3890069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.095+0000 7f3894cce700 1 -- 172.21.15.29:0/2493460387 >> v2:172.21.15.163:3300/0 conn(0x7f3890069b70 msgr2=0x7f38900bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.095+0000 7f3894cce700 1 --2- 172.21.15.29:0/2493460387 >> v2:172.21.15.163:3300/0 conn(0x7f3890069b70 0x7f38900bcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.095+0000 7f3894cce700 1 -- 172.21.15.29:0/2493460387 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38900c3540 con 0x7f38900692a0 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.095+0000 7f38954cf700 1 --2- 172.21.15.29:0/2493460387 >> v2:172.21.15.163:3300/0 conn(0x7f3890069b70 0x7f38900bcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.095+0000 7f3894cce700 1 --2- 172.21.15.29:0/2493460387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38900692a0 0x7f3890065e80 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 rx=0x7f38901548f0 tx=0x7f38840090f0).ready entity=mon.0 client_cookie=6c26417d5693ddf5 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.095+0000 7f388f7fe700 1 -- 172.21.15.29:0/2493460387 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388400b520 con 0x7f38900692a0 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.095+0000 7f388f7fe700 1 -- 172.21.15.29:0/2493460387 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388400b680 con 0x7f38900692a0 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.095+0000 7f388f7fe700 1 -- 172.21.15.29:0/2493460387 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388400ba00 con 0x7f38900692a0 2024-01-24T15:32:38.099 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.095+0000 7f3896f32700 1 -- 172.21.15.29:0/2493460387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38900692a0 msgr2=0x7f3890065e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:38.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.095+0000 7f3896f32700 1 --2- 172.21.15.29:0/2493460387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38900692a0 0x7f3890065e80 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 rx=0x7f38901548f0 tx=0x7f38840090f0).stop 2024-01-24T15:32:38.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.096+0000 7f3896f32700 1 -- 172.21.15.29:0/2493460387 shutdown_connections 2024-01-24T15:32:38.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.096+0000 7f3896f32700 1 --2- 172.21.15.29:0/2493460387 >> v2:172.21.15.163:3300/0 conn(0x7f3890069b70 0x7f38900bcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.096+0000 7f3896f32700 1 --2- 172.21.15.29:0/2493460387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38900692a0 0x7f3890065e80 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.096+0000 7f3896f32700 1 --2- 172.21.15.29:0/2493460387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3890066370 0x7f3890069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.096+0000 7f3896f32700 1 -- 172.21.15.29:0/2493460387 >> 172.21.15.29:0/2493460387 conn(0x7f3890064390 msgr2=0x7f38900c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:38.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.096+0000 7f3896f32700 1 -- 172.21.15.29:0/2493460387 shutdown_connections 2024-01-24T15:32:38.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.096+0000 7f3896f32700 1 -- 172.21.15.29:0/2493460387 wait complete. 2024-01-24T15:32:38.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.097+0000 7f3896f32700 1 Processor -- start 2024-01-24T15:32:38.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.097+0000 7f3896f32700 1 -- start start 2024-01-24T15:32:38.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f3896f32700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3890066370 0x7f38901516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f3896f32700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38900692a0 0x7f38900ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f3896f32700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3890069b70 0x7f389013a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f3896f32700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389013ab80 con 0x7f38900692a0 2024-01-24T15:32:38.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f3896f32700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3890152920 con 0x7f3890069b70 2024-01-24T15:32:38.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f3896f32700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3890151d10 con 0x7f3890066370 2024-01-24T15:32:38.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f388ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38900692a0 0x7f38900ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f388ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38900692a0 0x7f38900ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38566/0 (socket says 172.21.15.29:38566) 2024-01-24T15:32:38.103 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f388ffff700 1 -- 172.21.15.29:0/3936061099 learned_addr learned my addr 172.21.15.29:0/3936061099 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f3894cce700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3890066370 0x7f38901516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f38954cf700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3890069b70 0x7f389013a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f388ffff700 1 -- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3890066370 msgr2=0x7f38901516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f388ffff700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3890066370 0x7f38901516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f388ffff700 1 -- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3890069b70 msgr2=0x7f389013a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f388ffff700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3890069b70 0x7f389013a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.098+0000 7f388ffff700 1 -- 172.21.15.29:0/3936061099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3884009370 con 0x7f38900692a0 2024-01-24T15:32:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.099+0000 7f388ffff700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38900692a0 0x7f38900ad680 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 rx=0x7f387c00e900 tx=0x7f387c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.099+0000 7f388cff9700 1 -- 172.21.15.29:0/3936061099 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c012600 con 0x7f38900692a0 2024-01-24T15:32:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.099+0000 7f388cff9700 1 -- 172.21.15.29:0/3936061099 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c012760 con 0x7f38900692a0 2024-01-24T15:32:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.099+0000 7f3896f32700 1 -- 172.21.15.29:0/3936061099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3890151ff0 con 0x7f38900692a0 2024-01-24T15:32:38.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.099+0000 7f388cff9700 1 -- 172.21.15.29:0/3936061099 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c012ae0 con 0x7f38900692a0 2024-01-24T15:32:38.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.099+0000 7f3896f32700 1 -- 172.21.15.29:0/3936061099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38901525b0 con 0x7f38900692a0 2024-01-24T15:32:38.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.100+0000 7f3896f32700 1 -- 172.21.15.29:0/3936061099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3890152ad0 con 0x7f38900692a0 2024-01-24T15:32:38.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.100+0000 7f388cff9700 1 -- 172.21.15.29:0/3936061099 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f387c01b400 con 0x7f38900692a0 2024-01-24T15:32:38.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.101+0000 7f388cff9700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f387405f300 0x7f38740617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.101+0000 7f3894cce700 1 -- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f387405f300 msgr2=0x7f38740617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:38.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.101+0000 7f388cff9700 1 -- 172.21.15.29:0/3936061099 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7f387c07abf0 con 0x7f38900692a0 2024-01-24T15:32:38.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.101+0000 7f3894cce700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f387405f300 0x7f38740617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:38.107 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.105+0000 7f388cff9700 1 -- 172.21.15.29:0/3936061099 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3890152ad0 con 0x7f38900692a0 2024-01-24T15:32:38.249 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:38.247+0000 7f51ebb85680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:32:38.286 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:38.284+0000 7f18d2708680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.289+0000 7f3896f32700 1 -- 172.21.15.29:0/3936061099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3890152a90 con 0x7f38900692a0 2024-01-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.290+0000 7f388cff9700 1 -- 172.21.15.29:0/3936061099 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v232) v1 ==== 104+0+11 (secure 0 0 0) 0x7f387c04b740 con 0x7f38900692a0 2024-01-24T15:32:38.292 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.292+0000 7f3896f32700 1 -- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f387405f300 msgr2=0x7f38740617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.292+0000 7f3896f32700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f387405f300 0x7f38740617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.292+0000 7f3896f32700 1 -- 172.21.15.29:0/3936061099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38900692a0 msgr2=0x7f38900ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.292+0000 7f3896f32700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38900692a0 0x7f38900ad680 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 rx=0x7f387c00e900 tx=0x7f387c009060).stop 2024-01-24T15:32:38.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.292+0000 7f3896f32700 1 -- 172.21.15.29:0/3936061099 shutdown_connections 2024-01-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.292+0000 7f3896f32700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3890069b70 0x7f389013a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.292+0000 7f3896f32700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f387405f300 0x7f38740617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.292+0000 7f3896f32700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3890066370 0x7f38901516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.292+0000 7f3896f32700 1 --2- 172.21.15.29:0/3936061099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38900692a0 0x7f38900ad680 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.292+0000 7f3896f32700 1 -- 172.21.15.29:0/3936061099 >> 172.21.15.29:0/3936061099 conn(0x7f3890064390 msgr2=0x7f38900c7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.292+0000 7f3896f32700 1 -- 172.21.15.29:0/3936061099 shutdown_connections 2024-01-24T15:32:38.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.292+0000 7f3896f32700 1 -- 172.21.15.29:0/3936061099 wait complete. 2024-01-24T15:32:38.308 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-24T15:32:38.324 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:38.321+0000 7f51ebb85680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:32:38.393 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:38.390+0000 7f51ebb85680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:32:38.411 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:38.409+0000 7f18d2708680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:32:38.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.443+0000 7f545972b700 1 Processor -- start 2024-01-24T15:32:38.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.443+0000 7f545972b700 1 -- start start 2024-01-24T15:32:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.444+0000 7f545972b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f54540644d0 0x7f54540648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.444+0000 7f545972b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5454153510 0x7f5454153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.444+0000 7f545972b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5454069fd0 con 0x7f5454154280 2024-01-24T15:32:38.454 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.444+0000 7f545972b700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54540bc970 con 0x7f54540644d0 2024-01-24T15:32:38.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.444+0000 7f545972b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54540693c0 con 0x7f5454153510 2024-01-24T15:32:38.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.444+0000 7f5452ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f54540644d0 0x7f54540648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.444+0000 7f5452ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f54540644d0 0x7f54540648b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38578/0 (socket says 172.21.15.29:38578) 2024-01-24T15:32:38.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.444+0000 7f5452ffd700 1 -- 172.21.15.29:0/1409598224 learned_addr learned my addr 172.21.15.29:0/1409598224 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:38.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.444+0000 7f54537fe700 1 --2- 172.21.15.29:0/1409598224 >> v2:172.21.15.163:3300/0 conn(0x7f5454153510 0x7f5454153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.444+0000 7f5452ffd700 1 -- 172.21.15.29:0/1409598224 >> v2:172.21.15.163:3300/0 conn(0x7f5454153510 msgr2=0x7f5454153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:38.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.444+0000 7f5452ffd700 1 --2- 172.21.15.29:0/1409598224 >> v2:172.21.15.163:3300/0 conn(0x7f5454153510 0x7f5454153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.445+0000 7f5452ffd700 1 -- 172.21.15.29:0/1409598224 >> v1:172.21.15.163:6789/0 conn(0x7f5454154280 legacy=0x7f54540689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:38.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.445+0000 7f5452ffd700 1 -- 172.21.15.29:0/1409598224 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54540696a0 con 0x7f54540644d0 2024-01-24T15:32:38.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.445+0000 7f5452ffd700 1 --2- 172.21.15.29:0/1409598224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54540644d0 0x7f54540648b0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 rx=0x7f54541552a0 tx=0x7f54440090f0).ready entity=mon.0 client_cookie=9b48d88c29e0e8f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:38.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.445+0000 7f5451ffb700 1 -- 172.21.15.29:0/1409598224 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544400b520 con 0x7f54540644d0 2024-01-24T15:32:38.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.445+0000 7f5451ffb700 1 -- 172.21.15.29:0/1409598224 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544400b680 con 0x7f54540644d0 2024-01-24T15:32:38.456 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.445+0000 7f5451ffb700 1 -- 172.21.15.29:0/1409598224 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544400ba20 con 0x7f54540644d0 2024-01-24T15:32:38.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.445+0000 7f545972b700 1 -- 172.21.15.29:0/1409598224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54540644d0 msgr2=0x7f54540648b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:38.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.445+0000 7f545972b700 1 --2- 172.21.15.29:0/1409598224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54540644d0 0x7f54540648b0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 rx=0x7f54541552a0 tx=0x7f54440090f0).stop 2024-01-24T15:32:38.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.446+0000 7f545972b700 1 -- 172.21.15.29:0/1409598224 shutdown_connections 2024-01-24T15:32:38.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.446+0000 7f545972b700 1 --2- 172.21.15.29:0/1409598224 >> v2:172.21.15.163:3300/0 conn(0x7f5454153510 0x7f5454153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.446+0000 7f545972b700 1 --2- 172.21.15.29:0/1409598224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54540644d0 0x7f54540648b0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.446+0000 7f545972b700 1 -- 172.21.15.29:0/1409598224 >> 172.21.15.29:0/1409598224 conn(0x7f54540627b0 msgr2=0x7f5454062bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:38.457 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.446+0000 7f545972b700 1 -- 172.21.15.29:0/1409598224 shutdown_connections 2024-01-24T15:32:38.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.446+0000 7f545972b700 1 -- 172.21.15.29:0/1409598224 wait complete. 2024-01-24T15:32:38.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.447+0000 7f545972b700 1 Processor -- start 2024-01-24T15:32:38.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.447+0000 7f545972b700 1 -- start start 2024-01-24T15:32:38.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f545972b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54540644d0 0x7f5454143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f545972b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5454153510 0x7f5454143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.458 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f545972b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5454154280 0x7f54540aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f545972b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54540c5e90 con 0x7f5454154280 2024-01-24T15:32:38.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f545972b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54540c5880 con 0x7f54540644d0 2024-01-24T15:32:38.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f545972b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5454143a20 con 0x7f5454153510 2024-01-24T15:32:38.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f54537fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5454154280 0x7f54540aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f5452ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54540644d0 0x7f5454143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.459 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f54537fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5454154280 0x7f54540aa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38582/0 (socket says 172.21.15.29:38582) 2024-01-24T15:32:38.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f54537fe700 1 -- 172.21.15.29:0/3727185512 learned_addr learned my addr 172.21.15.29:0/3727185512 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:38.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f5452ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54540644d0 0x7f5454143cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:41652/0 (socket says 172.21.15.29:41652) 2024-01-24T15:32:38.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f54527fc700 1 --2- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5454153510 0x7f5454143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f54537fe700 1 -- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5454153510 msgr2=0x7f5454143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:38.460 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f54537fe700 1 --2- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5454153510 0x7f5454143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f54537fe700 1 -- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54540644d0 msgr2=0x7f5454143cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:38.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f54537fe700 1 --2- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54540644d0 0x7f5454143cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.448+0000 7f54537fe700 1 -- 172.21.15.29:0/3727185512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5444009370 con 0x7f5454154280 2024-01-24T15:32:38.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.449+0000 7f54537fe700 1 --2- 172.21.15.29:0/3727185512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5454154280 0x7f54540aa2a0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 rx=0x7f544c0070c0 tx=0x7f544c00a780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:38.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.449+0000 7f543affd700 1 -- 172.21.15.29:0/3727185512 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c00abd0 con 0x7f5454154280 2024-01-24T15:32:38.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.449+0000 7f543affd700 1 -- 172.21.15.29:0/3727185512 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c010040 con 0x7f5454154280 2024-01-24T15:32:38.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.449+0000 7f545972b700 1 -- 172.21.15.29:0/3727185512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54540bf850 con 0x7f5454154280 2024-01-24T15:32:38.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.449+0000 7f543affd700 1 -- 172.21.15.29:0/3727185512 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c014640 con 0x7f5454154280 2024-01-24T15:32:38.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.449+0000 7f545972b700 1 -- 172.21.15.29:0/3727185512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54540bfb30 con 0x7f5454154280 2024-01-24T15:32:38.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.450+0000 7f545972b700 1 -- 172.21.15.29:0/3727185512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54540c6000 con 0x7f5454154280 2024-01-24T15:32:38.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.450+0000 7f543affd700 1 -- 172.21.15.29:0/3727185512 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f544c01d3b0 con 0x7f5454154280 2024-01-24T15:32:38.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.451+0000 7f543affd700 1 --2- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f544005f330 0x7f54400617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.451+0000 7f543affd700 1 -- 172.21.15.29:0/3727185512 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7f544c07adb0 con 0x7f5454154280 2024-01-24T15:32:38.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.451+0000 7f5452ffd700 1 -- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f544005f330 msgr2=0x7f54400617f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:38.462 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.451+0000 7f5452ffd700 1 --2- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f544005f330 0x7f54400617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:38.463 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.455+0000 7f543affd700 1 -- 172.21.15.29:0/3727185512 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f544c04b7f0 con 0x7f5454154280 2024-01-24T15:32:38.593 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:38.591+0000 7f18d2708680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T15:32:38.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.637+0000 7f545972b700 1 -- 172.21.15.29:0/3727185512 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54540475e0 con 0x7f5454154280 2024-01-24T15:32:38.639 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.637+0000 7f543affd700 1 -- 172.21.15.29:0/3727185512 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v232) v1 ==== 100+0+11 (secure 0 0 0) 0x7f544c04ee10 con 0x7f5454154280 2024-01-24T15:32:38.639 INFO:teuthology.orchestra.run.smithi029.stdout:pg_num: 64 2024-01-24T15:32:38.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.639+0000 7f545972b700 1 -- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f544005f330 msgr2=0x7f54400617f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:38.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.639+0000 7f545972b700 1 --2- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f544005f330 0x7f54400617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:38.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.639+0000 7f545972b700 1 -- 172.21.15.29:0/3727185512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5454154280 msgr2=0x7f54540aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:38.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.639+0000 7f545972b700 1 --2- 172.21.15.29:0/3727185512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5454154280 0x7f54540aa2a0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 rx=0x7f544c0070c0 tx=0x7f544c00a780).stop 2024-01-24T15:32:38.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.639+0000 7f545972b700 1 -- 172.21.15.29:0/3727185512 shutdown_connections 2024-01-24T15:32:38.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.639+0000 7f545972b700 1 --2- 172.21.15.29:0/3727185512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5454154280 0x7f54540aa2a0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.639+0000 7f545972b700 1 --2- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54540644d0 0x7f5454143cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.639+0000 7f545972b700 1 --2- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f544005f330 0x7f54400617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:38.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.639+0000 7f545972b700 1 --2- 172.21.15.29:0/3727185512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5454153510 0x7f5454143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.639+0000 7f545972b700 1 -- 172.21.15.29:0/3727185512 >> 172.21.15.29:0/3727185512 conn(0x7f54540627b0 msgr2=0x7f5454062b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:38.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.639+0000 7f545972b700 1 -- 172.21.15.29:0/3727185512 shutdown_connections 2024-01-24T15:32:38.642 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.639+0000 7f545972b700 1 -- 172.21.15.29:0/3727185512 wait complete. 2024-01-24T15:32:38.649 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:38.647+0000 7f51ebb85680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:32:38.656 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:32:38.682 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:38.680+0000 7f18d2708680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T15:32:38.762 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:38.760+0000 7f18d2708680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.785+0000 7efdfd7a4700 1 Processor -- start 2024-01-24T15:32:38.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.785+0000 7efdfd7a4700 1 -- start start 2024-01-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.786+0000 7efdfd7a4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efdf8134f10 0x7efdf80627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.786+0000 7efdfd7a4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efdf8062ca0 0x7efdf81527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.786+0000 7efdfd7a4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdf8152cd0 0x7efdf8155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.786+0000 7efdfd7a4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf8064250 con 0x7efdf8134f10 2024-01-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.786+0000 7efdfd7a4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf8063150 con 0x7efdf8062ca0 2024-01-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.786+0000 7efdfd7a4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf8064d90 con 0x7efdf8152cd0 2024-01-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.786+0000 7efdf6ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efdf8134f10 0x7efdf80627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.786+0000 7efdf6ffd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efdf8134f10 0x7efdf80627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38598/0 (socket says 172.21.15.29:38598) 2024-01-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.786+0000 7efdf6ffd700 1 -- 172.21.15.29:0/2416688601 learned_addr learned my addr 172.21.15.29:0/2416688601 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.786+0000 7efdf67fc700 1 --2- 172.21.15.29:0/2416688601 >> v2:172.21.15.163:3300/0 conn(0x7efdf8062ca0 0x7efdf81527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.786+0000 7efdf77fe700 1 --2- 172.21.15.29:0/2416688601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdf8152cd0 0x7efdf8155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.787+0000 7efdf6ffd700 1 -- 172.21.15.29:0/2416688601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdf8152cd0 msgr2=0x7efdf8155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.787+0000 7efdf6ffd700 1 --2- 172.21.15.29:0/2416688601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdf8152cd0 0x7efdf8155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.787+0000 7efdf6ffd700 1 -- 172.21.15.29:0/2416688601 >> v2:172.21.15.163:3300/0 conn(0x7efdf8062ca0 msgr2=0x7efdf81527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.787+0000 7efdf6ffd700 1 --2- 172.21.15.29:0/2416688601 >> v2:172.21.15.163:3300/0 conn(0x7efdf8062ca0 0x7efdf81527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.787+0000 7efdf6ffd700 1 -- 172.21.15.29:0/2416688601 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf8065220 con 0x7efdf8134f10 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.787+0000 7efdf6ffd700 1 --2- 172.21.15.29:0/2416688601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdf8134f10 0x7efdf80627b0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 rx=0x7efdf8069d20 tx=0x7efde80090f0).ready entity=mon.0 client_cookie=83e667ed07ca62c server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.787+0000 7efdf5ffb700 1 -- 172.21.15.29:0/2416688601 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde800b600 con 0x7efdf8134f10 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.787+0000 7efdf5ffb700 1 -- 172.21.15.29:0/2416688601 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde800b760 con 0x7efdf8134f10 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.787+0000 7efdf5ffb700 1 -- 172.21.15.29:0/2416688601 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde800bb00 con 0x7efdf8134f10 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.787+0000 7efdfd7a4700 1 -- 172.21.15.29:0/2416688601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdf8134f10 msgr2=0x7efdf80627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.787+0000 7efdfd7a4700 1 --2- 172.21.15.29:0/2416688601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdf8134f10 0x7efdf80627b0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 rx=0x7efdf8069d20 tx=0x7efde80090f0).stop 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.788+0000 7efdfd7a4700 1 -- 172.21.15.29:0/2416688601 shutdown_connections 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.788+0000 7efdfd7a4700 1 --2- 172.21.15.29:0/2416688601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdf8152cd0 0x7efdf8155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.788+0000 7efdfd7a4700 1 --2- 172.21.15.29:0/2416688601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdf8134f10 0x7efdf80627b0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.788+0000 7efdfd7a4700 1 --2- 172.21.15.29:0/2416688601 >> v2:172.21.15.163:3300/0 conn(0x7efdf8062ca0 0x7efdf81527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.788+0000 7efdfd7a4700 1 -- 172.21.15.29:0/2416688601 >> 172.21.15.29:0/2416688601 conn(0x7efdf80c69f0 msgr2=0x7efdf80c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:38.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.788+0000 7efdfd7a4700 1 -- 172.21.15.29:0/2416688601 shutdown_connections 2024-01-24T15:32:38.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.788+0000 7efdfd7a4700 1 -- 172.21.15.29:0/2416688601 wait complete. 2024-01-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.790+0000 7efdfd7a4700 1 Processor -- start 2024-01-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.790+0000 7efdfd7a4700 1 -- start start 2024-01-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.791+0000 7efdfd7a4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdf8062ca0 0x7efdf8138360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.791+0000 7efdfd7a4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdf8134f10 0x7efdf80bfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.791+0000 7efdfd7a4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdf8152cd0 0x7efdf81344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.791+0000 7efdfd7a4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf8138850 con 0x7efdf8152cd0 2024-01-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.791+0000 7efdfd7a4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf81356d0 con 0x7efdf8134f10 2024-01-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.791+0000 7efdfd7a4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf8135f70 con 0x7efdf8062ca0 2024-01-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.791+0000 7efdf77fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdf8152cd0 0x7efdf81344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.791+0000 7efdf77fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdf8152cd0 0x7efdf81344d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38612/0 (socket says 172.21.15.29:38612) 2024-01-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.791+0000 7efdf77fe700 1 -- 172.21.15.29:0/3729863706 learned_addr learned my addr 172.21.15.29:0/3729863706 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:38.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.791+0000 7efdf6ffd700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdf8062ca0 0x7efdf8138360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.791+0000 7efdf67fc700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdf8134f10 0x7efdf80bfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.792+0000 7efdf77fe700 1 -- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdf8062ca0 msgr2=0x7efdf8138360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.792+0000 7efdf77fe700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdf8062ca0 0x7efdf8138360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.792+0000 7efdf77fe700 1 -- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdf8134f10 msgr2=0x7efdf80bfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.792+0000 7efdf77fe700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdf8134f10 0x7efdf80bfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.792+0000 7efdf77fe700 1 -- 172.21.15.29:0/3729863706 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde8009370 con 0x7efdf8152cd0 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.792+0000 7efdf6ffd700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdf8062ca0 0x7efdf8138360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.792+0000 7efdf77fe700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdf8152cd0 0x7efdf81344d0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 rx=0x7efdf0009fd0 tx=0x7efdf0009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.792+0000 7efddeffd700 1 -- 172.21.15.29:0/3729863706 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000ebf0 con 0x7efdf8152cd0 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.792+0000 7efddeffd700 1 -- 172.21.15.29:0/3729863706 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf000ed50 con 0x7efdf8152cd0 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.792+0000 7efdfd7a4700 1 -- 172.21.15.29:0/3729863706 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf80c0090 con 0x7efdf8152cd0 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.792+0000 7efddeffd700 1 -- 172.21.15.29:0/3729863706 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0014610 con 0x7efdf8152cd0 2024-01-24T15:32:38.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.792+0000 7efdfd7a4700 1 -- 172.21.15.29:0/3729863706 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf80c02f0 con 0x7efdf8152cd0 2024-01-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.794+0000 7efdfd7a4700 1 -- 172.21.15.29:0/3729863706 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf80475e0 con 0x7efdf8152cd0 2024-01-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.794+0000 7efddeffd700 1 -- 172.21.15.29:0/3729863706 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7efdf0014980 con 0x7efdf8152cd0 2024-01-24T15:32:38.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.794+0000 7efddeffd700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efde405f300 0x7efde40617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.794+0000 7efdf6ffd700 1 -- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efde405f300 msgr2=0x7efde40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.794+0000 7efddeffd700 1 -- 172.21.15.29:0/3729863706 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7efdf007ad90 con 0x7efdf8152cd0 2024-01-24T15:32:38.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.794+0000 7efdf6ffd700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efde405f300 0x7efde40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:38.800 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.798+0000 7efddeffd700 1 -- 172.21.15.29:0/3729863706 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7efdf004b940 con 0x7efdf8152cd0 2024-01-24T15:32:38.835 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:38.832+0000 7f51ebb85680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:32:38.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.995+0000 7efdf6ffd700 1 -- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efde405f300 msgr2=0x7efde40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:38.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:38.995+0000 7efdf6ffd700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efde405f300 0x7efde40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:32:39.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.031+0000 7efdfd7a4700 1 -- 172.21.15.29:0/3729863706 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdf80656c0 con 0x7efdf8152cd0 2024-01-24T15:32:39.033 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.031+0000 7efddeffd700 1 -- 172.21.15.29:0/3729863706 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4499 (secure 0 0 0) 0x7efdf004ef60 con 0x7efdf8152cd0 2024-01-24T15:32:39.034 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:39.034 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1883755467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1883755467},{"type":"v1","addr":"172.21.15.163:6836","nonce":1883755467}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1773668659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1773668659},{"type":"v1","addr":"172.21.15.29:6834","nonce":1773668659}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3438722509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3438722509},{"type":"v1","addr":"172.21.15.29:6835","nonce":3438722509}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:32:20.685536+0000","modified":"2024-01-24T15:32:23.705717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10029":{"gid":10029,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/532321390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":532321390},{"type":"v1","addr":"172.21.15.163:6837","nonce":532321390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2024-01-24T15:32:39.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.033+0000 7efdfd7a4700 1 -- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efde405f300 msgr2=0x7efde40617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:39.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.033+0000 7efdfd7a4700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efde405f300 0x7efde40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:39.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.033+0000 7efdfd7a4700 1 -- 172.21.15.29:0/3729863706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdf8152cd0 msgr2=0x7efdf81344d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.033+0000 7efdfd7a4700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdf8152cd0 0x7efdf81344d0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 rx=0x7efdf0009fd0 tx=0x7efdf0009900).stop 2024-01-24T15:32:39.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.033+0000 7efdfd7a4700 1 -- 172.21.15.29:0/3729863706 shutdown_connections 2024-01-24T15:32:39.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.033+0000 7efdfd7a4700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdf8152cd0 0x7efdf81344d0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.033+0000 7efdfd7a4700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efde405f300 0x7efde40617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:39.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.033+0000 7efdfd7a4700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdf8062ca0 0x7efdf8138360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.033+0000 7efdfd7a4700 1 --2- 172.21.15.29:0/3729863706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdf8134f10 0x7efdf80bfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.036 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.033+0000 7efdfd7a4700 1 -- 172.21.15.29:0/3729863706 >> 172.21.15.29:0/3729863706 conn(0x7efdf80c69f0 msgr2=0x7efdf814eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:39.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.034+0000 7efdfd7a4700 1 -- 172.21.15.29:0/3729863706 shutdown_connections 2024-01-24T15:32:39.037 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.034+0000 7efdfd7a4700 1 -- 172.21.15.29:0/3729863706 wait complete. 2024-01-24T15:32:39.037 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 98 2024-01-24T15:32:39.049 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:32:39.062 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:39.060+0000 7f18d2708680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T15:32:39.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.192+0000 7f83084a1700 1 Processor -- start 2024-01-24T15:32:39.195 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.193+0000 7f83084a1700 1 -- start start 2024-01-24T15:32:39.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f83084a1700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8300134cc0 0x7f83000c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f83084a1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f83001350a0 0x7f8300152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f83084a1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83000664f0 0x7f83000669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f83084a1700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8300153360 con 0x7f8300134cc0 2024-01-24T15:32:39.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f83084a1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8300066e90 con 0x7f83001350a0 2024-01-24T15:32:39.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f83084a1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8300068360 con 0x7f83000664f0 2024-01-24T15:32:39.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f830623d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8300134cc0 0x7f83000c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f830623d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8300134cc0 0x7f83000c31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38614/0 (socket says 172.21.15.29:38614) 2024-01-24T15:32:39.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f830623d700 1 -- 172.21.15.29:0/1589852565 learned_addr learned my addr 172.21.15.29:0/1589852565 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f8306a3e700 1 --2- 172.21.15.29:0/1589852565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83000664f0 0x7f83000669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f8305a3c700 1 --2- 172.21.15.29:0/1589852565 >> v2:172.21.15.163:3300/0 conn(0x7f83001350a0 0x7f8300152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f830623d700 1 -- 172.21.15.29:0/1589852565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83000664f0 msgr2=0x7f83000669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f830623d700 1 --2- 172.21.15.29:0/1589852565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83000664f0 0x7f83000669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f830623d700 1 -- 172.21.15.29:0/1589852565 >> v2:172.21.15.163:3300/0 conn(0x7f83001350a0 msgr2=0x7f8300152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f830623d700 1 --2- 172.21.15.29:0/1589852565 >> v2:172.21.15.163:3300/0 conn(0x7f83001350a0 0x7f8300152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f830623d700 1 -- 172.21.15.29:0/1589852565 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83000c3820 con 0x7f8300134cc0 2024-01-24T15:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f8305a3c700 1 --2- 172.21.15.29:0/1589852565 >> v2:172.21.15.163:3300/0 conn(0x7f83001350a0 0x7f8300152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.194+0000 7f830623d700 1 --2- 172.21.15.29:0/1589852565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8300134cc0 0x7f83000c31b0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 rx=0x7f83000aed60 tx=0x7f82fc0090f0).ready entity=mon.0 client_cookie=815ff8b70205da26 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.195+0000 7f830523b700 1 -- 172.21.15.29:0/1589852565 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00b520 con 0x7f8300134cc0 2024-01-24T15:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.195+0000 7f830523b700 1 -- 172.21.15.29:0/1589852565 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82fc00b680 con 0x7f8300134cc0 2024-01-24T15:32:39.198 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.195+0000 7f830523b700 1 -- 172.21.15.29:0/1589852565 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00ba00 con 0x7f8300134cc0 2024-01-24T15:32:39.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.195+0000 7f83084a1700 1 -- 172.21.15.29:0/1589852565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8300134cc0 msgr2=0x7f83000c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.195+0000 7f83084a1700 1 --2- 172.21.15.29:0/1589852565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8300134cc0 0x7f83000c31b0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 rx=0x7f83000aed60 tx=0x7f82fc0090f0).stop 2024-01-24T15:32:39.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.195+0000 7f83084a1700 1 -- 172.21.15.29:0/1589852565 shutdown_connections 2024-01-24T15:32:39.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.195+0000 7f83084a1700 1 --2- 172.21.15.29:0/1589852565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83000664f0 0x7f83000669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.195+0000 7f83084a1700 1 --2- 172.21.15.29:0/1589852565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8300134cc0 0x7f83000c31b0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.195+0000 7f83084a1700 1 --2- 172.21.15.29:0/1589852565 >> v2:172.21.15.163:3300/0 conn(0x7f83001350a0 0x7f8300152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.195+0000 7f83084a1700 1 -- 172.21.15.29:0/1589852565 >> 172.21.15.29:0/1589852565 conn(0x7f83000c8bb0 msgr2=0x7f83000c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:39.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.196+0000 7f83084a1700 1 -- 172.21.15.29:0/1589852565 shutdown_connections 2024-01-24T15:32:39.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.196+0000 7f83084a1700 1 -- 172.21.15.29:0/1589852565 wait complete. 2024-01-24T15:32:39.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.197+0000 7f83084a1700 1 Processor -- start 2024-01-24T15:32:39.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.197+0000 7f83084a1700 1 -- start start 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f83084a1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83000664f0 0x7f830014a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f83084a1700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8300134cc0 0x7f8300133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f83084a1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83001350a0 0x7f83001344a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f83084a1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8300134990 con 0x7f83000664f0 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f83084a1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8300135ed0 con 0x7f83001350a0 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f83084a1700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000ab430 con 0x7f8300134cc0 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f830623d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83000664f0 0x7f830014a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f830623d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83000664f0 0x7f830014a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38622/0 (socket says 172.21.15.29:38622) 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f830623d700 1 -- 172.21.15.29:0/2235424174 learned_addr learned my addr 172.21.15.29:0/2235424174 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f8305a3c700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8300134cc0 0x7f8300133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f8306a3e700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83001350a0 0x7f83001344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f830623d700 1 -- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8300134cc0 msgr2=0x7f8300133fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f830623d700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8300134cc0 0x7f8300133fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f830623d700 1 -- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83001350a0 msgr2=0x7f83001344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f830623d700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83001350a0 0x7f83001344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f830623d700 1 -- 172.21.15.29:0/2235424174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc009370 con 0x7f83000664f0 2024-01-24T15:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.198+0000 7f830623d700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83000664f0 0x7f830014a980 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 rx=0x7f830006a130 tx=0x7f82fc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.199+0000 7f82f67fc700 1 -- 172.21.15.29:0/2235424174 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00be90 con 0x7f83000664f0 2024-01-24T15:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.199+0000 7f82f67fc700 1 -- 172.21.15.29:0/2235424174 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82fc00b7f0 con 0x7f83000664f0 2024-01-24T15:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.199+0000 7f82f67fc700 1 -- 172.21.15.29:0/2235424174 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc01c630 con 0x7f83000664f0 2024-01-24T15:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.199+0000 7f83084a1700 1 -- 172.21.15.29:0/2235424174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83001355a0 con 0x7f83000664f0 2024-01-24T15:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.199+0000 7f83084a1700 1 -- 172.21.15.29:0/2235424174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8300135930 con 0x7f83000664f0 2024-01-24T15:32:39.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.200+0000 7f83084a1700 1 -- 172.21.15.29:0/2235424174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830014af00 con 0x7f83000664f0 2024-01-24T15:32:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.200+0000 7f82f67fc700 1 -- 172.21.15.29:0/2235424174 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f82fc01ca20 con 0x7f83000664f0 2024-01-24T15:32:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.201+0000 7f82f67fc700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f82e405f300 0x7f82e40617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.201+0000 7f82f67fc700 1 -- 172.21.15.29:0/2235424174 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7f82fc07ba80 con 0x7f83000664f0 2024-01-24T15:32:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.201+0000 7f8305a3c700 1 -- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f82e405f300 msgr2=0x7f82e40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:39.204 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.201+0000 7f8305a3c700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f82e405f300 0x7f82e40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:39.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.205+0000 7f82f67fc700 1 -- 172.21.15.29:0/2235424174 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f82fc04c6e0 con 0x7f83000664f0 2024-01-24T15:32:39.281 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:39.279+0000 7f18d2708680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T15:32:39.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.393+0000 7f83084a1700 1 -- 172.21.15.29:0/2235424174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83000475e0 con 0x7f83000664f0 2024-01-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.394+0000 7f82f67fc700 1 -- 172.21.15.29:0/2235424174 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v232) v1 ==== 74+0+18673 (secure 0 0 0) 0x7f82fc030910 con 0x7f83000664f0 2024-01-24T15:32:39.396 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:39.397 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":232,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:32:33.173608+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:32:18.425709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-01-24T15:32:19.656025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"231","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/343370470":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/3387006926":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/741220992":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/2316406386":"2024-01-25T15:32:33.173546+0000","172.21.15.163:6833/57365":"2024-01-25T15:32:33.173546+0000","172.21.15.163:6832/57365":"2024-01-25T15:32:33.173546+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.396+0000 7f83084a1700 1 -- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f82e405f300 msgr2=0x7f82e40617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.396+0000 7f83084a1700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f82e405f300 0x7f82e40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.396+0000 7f83084a1700 1 -- 172.21.15.29:0/2235424174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83000664f0 msgr2=0x7f830014a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.396+0000 7f83084a1700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83000664f0 0x7f830014a980 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 rx=0x7f830006a130 tx=0x7f82fc017000).stop 2024-01-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.396+0000 7f83084a1700 1 -- 172.21.15.29:0/2235424174 shutdown_connections 2024-01-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.396+0000 7f83084a1700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83001350a0 0x7f83001344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.396+0000 7f83084a1700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83000664f0 0x7f830014a980 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.396+0000 7f83084a1700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f82e405f300 0x7f82e40617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.396+0000 7f83084a1700 1 --2- 172.21.15.29:0/2235424174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8300134cc0 0x7f8300133fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.396+0000 7f83084a1700 1 -- 172.21.15.29:0/2235424174 >> 172.21.15.29:0/2235424174 conn(0x7f83000c8bb0 msgr2=0x7f83000c7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.396+0000 7f83084a1700 1 -- 172.21.15.29:0/2235424174 shutdown_connections 2024-01-24T15:32:39.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.396+0000 7f83084a1700 1 -- 172.21.15.29:0/2235424174 wait complete. 2024-01-24T15:32:39.412 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T15:32:39.413 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:32:39.491 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:39.489+0000 7f51ebb85680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:32:39.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.543+0000 7f3baf811700 1 Processor -- start 2024-01-24T15:32:39.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.543+0000 7f3baf811700 1 -- start start 2024-01-24T15:32:39.548 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.544+0000 7f3baf811700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3ba80627b0 0x7f3ba8062c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.544+0000 7f3baf811700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba8065770 con 0x7f3ba8138710 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.544+0000 7f3baf811700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba8069cf0 con 0x7f3ba8138330 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.544+0000 7f3baf811700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba80630f0 con 0x7f3ba80627b0 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3bacdac700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f3ba8138710 0x7f3ba80be3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37208/0 (socket says 172.21.15.29:37208) 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3bacdac700 1 -- 172.21.15.29:0/2643561250 learned_addr learned my addr 172.21.15.29:0/2643561250 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3baddae700 1 --2- 172.21.15.29:0/2643561250 >> v2:172.21.15.163:3300/0 conn(0x7f3ba80627b0 0x7f3ba8062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3b9ffff700 1 -- 172.21.15.29:0/2643561250 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570433633 0 0) 0x7f3ba8065770 con 0x7f3ba8138710 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3b9ffff700 1 -- 172.21.15.29:0/2643561250 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b940033f0 con 0x7f3ba8138710 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3b9ffff700 1 -- 172.21.15.29:0/2643561250 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784595565 0 0) 0x7f3ba8069cf0 con 0x7f3ba8138330 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3b9ffff700 1 -- 172.21.15.29:0/2643561250 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ba8065770 con 0x7f3ba8138330 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3baddae700 1 -- 172.21.15.29:0/2643561250 >> v1:172.21.15.163:6789/0 conn(0x7f3ba8138330 legacy=0x7f3ba80bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3baddae700 1 -- 172.21.15.29:0/2643561250 >> v1:172.21.15.29:6789/0 conn(0x7f3ba8138710 legacy=0x7f3ba80be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3baddae700 1 -- 172.21.15.29:0/2643561250 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba8069f60 con 0x7f3ba80627b0 2024-01-24T15:32:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3baddae700 1 --2- 172.21.15.29:0/2643561250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ba80627b0 0x7f3ba8062c00 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f3ba0009e60 tx=0x7f3ba0009780).ready entity=mon.1 client_cookie=4220a1f7d051c278 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3b9ffff700 1 -- 172.21.15.29:0/2643561250 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000e070 con 0x7f3ba80627b0 2024-01-24T15:32:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3b9ffff700 1 -- 172.21.15.29:0/2643561250 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba000bab0 con 0x7f3ba80627b0 2024-01-24T15:32:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.545+0000 7f3b9ffff700 1 -- 172.21.15.29:0/2643561250 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000be30 con 0x7f3ba80627b0 2024-01-24T15:32:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.546+0000 7f3baf811700 1 -- 172.21.15.29:0/2643561250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ba80627b0 msgr2=0x7f3ba8062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.546+0000 7f3baf811700 1 --2- 172.21.15.29:0/2643561250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ba80627b0 0x7f3ba8062c00 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f3ba0009e60 tx=0x7f3ba0009780).stop 2024-01-24T15:32:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.546+0000 7f3baf811700 1 -- 172.21.15.29:0/2643561250 shutdown_connections 2024-01-24T15:32:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.546+0000 7f3baf811700 1 --2- 172.21.15.29:0/2643561250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ba80627b0 0x7f3ba8062c00 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.546+0000 7f3baf811700 1 -- 172.21.15.29:0/2643561250 >> 172.21.15.29:0/2643561250 conn(0x7f3ba80c85c0 msgr2=0x7f3ba80c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.546+0000 7f3baf811700 1 -- 172.21.15.29:0/2643561250 shutdown_connections 2024-01-24T15:32:39.550 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.546+0000 7f3baf811700 1 -- 172.21.15.29:0/2643561250 wait complete. 2024-01-24T15:32:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.548+0000 7f3baf811700 1 Processor -- start 2024-01-24T15:32:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.548+0000 7f3baf811700 1 -- start start 2024-01-24T15:32:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baf811700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ba80627b0 0x7f3ba8156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baf811700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba8138330 0x7f3ba8137360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baf811700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ba8138710 0x7f3ba8154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baf811700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba8163080 con 0x7f3ba8138330 2024-01-24T15:32:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baf811700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba8156520 con 0x7f3ba8138710 2024-01-24T15:32:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baf811700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba8162470 con 0x7f3ba80627b0 2024-01-24T15:32:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3bacdac700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba8138330 0x7f3ba8137360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.553 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baddae700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ba8138710 0x7f3ba8154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3bacdac700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba8138330 0x7f3ba8137360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38638/0 (socket says 172.21.15.29:38638) 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3bacdac700 1 -- 172.21.15.29:0/4206640506 learned_addr learned my addr 172.21.15.29:0/4206640506 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3bad5ad700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ba80627b0 0x7f3ba8156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baddae700 1 -- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ba80627b0 msgr2=0x7f3ba8156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baddae700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ba80627b0 0x7f3ba8156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baddae700 1 -- 172.21.15.29:0/4206640506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba8138330 msgr2=0x7f3ba8137360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baddae700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba8138330 0x7f3ba8137360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baddae700 1 -- 172.21.15.29:0/4206640506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba0009a20 con 0x7f3ba8138710 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3bacdac700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba8138330 0x7f3ba8137360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.549+0000 7f3baddae700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ba8138710 0x7f3ba8154040 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f3ba0014000 tx=0x7f3ba0009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.550+0000 7f3b9d7fa700 1 -- 172.21.15.29:0/4206640506 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0012e10 con 0x7f3ba8138710 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.550+0000 7f3b9d7fa700 1 -- 172.21.15.29:0/4206640506 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba001d630 con 0x7f3ba8138710 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.550+0000 7f3baf811700 1 -- 172.21.15.29:0/4206640506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba8162750 con 0x7f3ba8138710 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.550+0000 7f3b9d7fa700 1 -- 172.21.15.29:0/4206640506 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba001d9b0 con 0x7f3ba8138710 2024-01-24T15:32:39.554 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.550+0000 7f3baf811700 1 -- 172.21.15.29:0/4206640506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba8162d10 con 0x7f3ba8138710 2024-01-24T15:32:39.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.551+0000 7f3baf811700 1 -- 172.21.15.29:0/4206640506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba81525c0 con 0x7f3ba8138710 2024-01-24T15:32:39.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.551+0000 7f3b9d7fa700 1 -- 172.21.15.29:0/4206640506 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f3ba001dd20 con 0x7f3ba8138710 2024-01-24T15:32:39.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.551+0000 7f3b9d7fa700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3b8c05f2d0 0x7f3b8c061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.551+0000 7f3b9d7fa700 1 -- 172.21.15.29:0/4206640506 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7f3ba007c790 con 0x7f3ba8138710 2024-01-24T15:32:39.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.552+0000 7f3bad5ad700 1 -- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3b8c05f2d0 msgr2=0x7f3b8c061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:39.555 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.552+0000 7f3bad5ad700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3b8c05f2d0 0x7f3b8c061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:39.558 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.556+0000 7f3b9d7fa700 1 -- 172.21.15.29:0/4206640506 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3ba81525c0 con 0x7f3ba8138710 2024-01-24T15:32:39.560 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:39.559+0000 7f51ebb85680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:32:39.636 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:39.635+0000 7f51ebb85680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:32:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.753+0000 7f3bad5ad700 1 -- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3b8c05f2d0 msgr2=0x7f3b8c061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:39.755 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.753+0000 7f3bad5ad700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3b8c05f2d0 0x7f3b8c061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:32:39.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.782+0000 7f3baf811700 1 -- 172.21.15.29:0/4206640506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ba81525c0 con 0x7f3ba8138710 2024-01-24T15:32:39.784 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:39.783+0000 7f51ebb85680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:32:39.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.782+0000 7f3b9d7fa700 1 -- 172.21.15.29:0/4206640506 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4499 (secure 0 0 0) 0x7f3ba81525c0 con 0x7f3ba8138710 2024-01-24T15:32:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:39.785 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1883755467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1883755467},{"type":"v1","addr":"172.21.15.163:6836","nonce":1883755467}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1773668659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1773668659},{"type":"v1","addr":"172.21.15.29:6834","nonce":1773668659}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3438722509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3438722509},{"type":"v1","addr":"172.21.15.29:6835","nonce":3438722509}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:32:20.685536+0000","modified":"2024-01-24T15:32:23.705717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10029":{"gid":10029,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/532321390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":532321390},{"type":"v1","addr":"172.21.15.163:6837","nonce":532321390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2024-01-24T15:32:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.784+0000 7f3baf811700 1 -- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3b8c05f2d0 msgr2=0x7f3b8c061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.784+0000 7f3baf811700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3b8c05f2d0 0x7f3b8c061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.784+0000 7f3baf811700 1 -- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ba8138710 msgr2=0x7f3ba8154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.784+0000 7f3baf811700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ba8138710 0x7f3ba8154040 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f3ba0014000 tx=0x7f3ba0009780).stop 2024-01-24T15:32:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.785+0000 7f3baf811700 1 -- 172.21.15.29:0/4206640506 shutdown_connections 2024-01-24T15:32:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.785+0000 7f3baf811700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ba8138710 0x7f3ba8154040 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.785+0000 7f3baf811700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f3b8c05f2d0 0x7f3b8c061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:39.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.785+0000 7f3baf811700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ba80627b0 0x7f3ba8156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.785+0000 7f3baf811700 1 --2- 172.21.15.29:0/4206640506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba8138330 0x7f3ba8137360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.785+0000 7f3baf811700 1 -- 172.21.15.29:0/4206640506 >> 172.21.15.29:0/4206640506 conn(0x7f3ba80c85c0 msgr2=0x7f3ba81518c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:39.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.785+0000 7f3baf811700 1 -- 172.21.15.29:0/4206640506 shutdown_connections 2024-01-24T15:32:39.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.785+0000 7f3baf811700 1 -- 172.21.15.29:0/4206640506 wait complete. 2024-01-24T15:32:39.789 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 98 2024-01-24T15:32:39.801 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:32:39.856 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:39.854+0000 7f51ebb85680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.934+0000 7fa2e5bc3700 1 Processor -- start 2024-01-24T15:32:39.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.934+0000 7fa2e5bc3700 1 -- start start 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2e5bc3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa2e00638e0 0x7fa2e00aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2e5bc3700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e00aa6c0 con 0x7fa2e00c5410 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2e5bc3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e0064240 con 0x7fa2e0154370 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2e5bc3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e00cb5c0 con 0x7fa2e00638e0 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2deffd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa2e00c5410 0x7fa2e00631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37224/0 (socket says 172.21.15.29:37224) 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2deffd700 1 -- 172.21.15.29:0/3976087849 learned_addr learned my addr 172.21.15.29:0/3976087849 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2dffff700 1 --2- 172.21.15.29:0/3976087849 >> v2:172.21.15.163:3300/0 conn(0x7fa2e00638e0 0x7fa2e00aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2de7fc700 1 -- 172.21.15.29:0/3976087849 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350751188 0 0) 0x7fa2e00aa6c0 con 0x7fa2e00c5410 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2de7fc700 1 -- 172.21.15.29:0/3976087849 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c80033f0 con 0x7fa2e00c5410 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2de7fc700 1 -- 172.21.15.29:0/3976087849 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015656403 0 0) 0x7fa2e0064240 con 0x7fa2e0154370 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2de7fc700 1 -- 172.21.15.29:0/3976087849 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e00aa6c0 con 0x7fa2e0154370 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2dffff700 1 -- 172.21.15.29:0/3976087849 >> v1:172.21.15.163:6789/0 conn(0x7fa2e0154370 legacy=0x7fa2e00ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2de7fc700 1 -- 172.21.15.29:0/3976087849 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1938270216 0 0) 0x7fa2c80033f0 con 0x7fa2e00c5410 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2dffff700 1 -- 172.21.15.29:0/3976087849 >> v1:172.21.15.29:6789/0 conn(0x7fa2e00c5410 legacy=0x7fa2e00631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2dffff700 1 -- 172.21.15.29:0/3976087849 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00cb8a0 con 0x7fa2e00638e0 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.935+0000 7fa2dffff700 1 --2- 172.21.15.29:0/3976087849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e00638e0 0x7fa2e00aa0e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7fa2d8009760 tx=0x7fa2d8009060).ready entity=mon.1 client_cookie=b4a4128a2cecc05f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.936+0000 7fa2de7fc700 1 -- 172.21.15.29:0/3976087849 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4001470 con 0x7fa2e00638e0 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.936+0000 7fa2de7fc700 1 -- 172.21.15.29:0/3976087849 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d8011400 con 0x7fa2e00638e0 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.936+0000 7fa2de7fc700 1 -- 172.21.15.29:0/3976087849 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8011780 con 0x7fa2e00638e0 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.936+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/3976087849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e00638e0 msgr2=0x7fa2e00aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.936+0000 7fa2e5bc3700 1 --2- 172.21.15.29:0/3976087849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e00638e0 0x7fa2e00aa0e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7fa2d8009760 tx=0x7fa2d8009060).stop 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.936+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/3976087849 shutdown_connections 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.936+0000 7fa2e5bc3700 1 --2- 172.21.15.29:0/3976087849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e00638e0 0x7fa2e00aa0e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.936+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/3976087849 >> 172.21.15.29:0/3976087849 conn(0x7fa2e005ec80 msgr2=0x7fa2e00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.937+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/3976087849 shutdown_connections 2024-01-24T15:32:39.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.937+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/3976087849 wait complete. 2024-01-24T15:32:39.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.938+0000 7fa2e5bc3700 1 Processor -- start 2024-01-24T15:32:39.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.938+0000 7fa2e5bc3700 1 -- start start 2024-01-24T15:32:39.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.939+0000 7fa2e5bc3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e00638e0 0x7fa2e00ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.939+0000 7fa2e5bc3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e00c5410 0x7fa2e00ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.939+0000 7fa2e5bc3700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e0154370 0x7fa2e00c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.939+0000 7fa2e5bc3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00ac080 con 0x7fa2e00c5410 2024-01-24T15:32:39.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.939+0000 7fa2e5bc3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00ac360 con 0x7fa2e00638e0 2024-01-24T15:32:39.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.939+0000 7fa2e5bc3700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00ac640 con 0x7fa2e0154370 2024-01-24T15:32:39.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.939+0000 7fa2deffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e00c5410 0x7fa2e00ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.939+0000 7fa2df7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e00638e0 0x7fa2e00ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.943 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.939+0000 7fa2deffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e00c5410 0x7fa2e00ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38646/0 (socket says 172.21.15.29:38646) 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.939+0000 7fa2df7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e00638e0 0x7fa2e00ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:41716/0 (socket says 172.21.15.29:41716) 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.939+0000 7fa2deffd700 1 -- 172.21.15.29:0/1383120970 learned_addr learned my addr 172.21.15.29:0/1383120970 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.939+0000 7fa2dffff700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e0154370 0x7fa2e00c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2deffd700 1 -- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e0154370 msgr2=0x7fa2e00c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2deffd700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e0154370 0x7fa2e00c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2deffd700 1 -- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e00638e0 msgr2=0x7fa2e00ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2deffd700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e00638e0 0x7fa2e00ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2deffd700 1 -- 172.21.15.29:0/1383120970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d80092a0 con 0x7fa2e00c5410 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2dffff700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e0154370 0x7fa2e00c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2df7fe700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e00638e0 0x7fa2e00ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2deffd700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e00c5410 0x7fa2e00ad120 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 rx=0x7fa2d400c810 tx=0x7fa2d4001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2c77fe700 1 -- 172.21.15.29:0/1383120970 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4007170 con 0x7fa2e00c5410 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2c77fe700 1 -- 172.21.15.29:0/1383120970 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d40072f0 con 0x7fa2e00c5410 2024-01-24T15:32:39.944 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/1383120970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e00ac920 con 0x7fa2e00c5410 2024-01-24T15:32:39.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2c77fe700 1 -- 172.21.15.29:0/1383120970 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40048c0 con 0x7fa2e00c5410 2024-01-24T15:32:39.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.940+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/1383120970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e00abd90 con 0x7fa2e00c5410 2024-01-24T15:32:39.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.941+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/1383120970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e00475e0 con 0x7fa2e00c5410 2024-01-24T15:32:39.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.942+0000 7fa2c77fe700 1 -- 172.21.15.29:0/1383120970 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fa2d4004c70 con 0x7fa2e00c5410 2024-01-24T15:32:39.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.942+0000 7fa2c77fe700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa2cc05f300 0x7fa2cc0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:39.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.942+0000 7fa2c77fe700 1 -- 172.21.15.29:0/1383120970 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7fa2d407a370 con 0x7fa2e00c5410 2024-01-24T15:32:39.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.942+0000 7fa2df7fe700 1 -- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa2cc05f300 msgr2=0x7fa2cc0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:39.946 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.942+0000 7fa2df7fe700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa2cc05f300 0x7fa2cc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:39.949 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:39.947+0000 7fa2c77fe700 1 -- 172.21.15.29:0/1383120970 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa2d404e950 con 0x7fa2e00c5410 2024-01-24T15:32:39.973 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:39.972+0000 7f51ebb85680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:32:40.052 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:40.050+0000 7f18d2708680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-24T15:32:40.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.080+0000 7f51ebb85680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:32:40.132 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:40.130+0000 7f18d2708680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T15:32:40.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.143+0000 7fa2df7fe700 1 -- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa2cc05f300 msgr2=0x7fa2cc0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:40.145 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.143+0000 7fa2df7fe700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa2cc05f300 0x7fa2cc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-01-24T15:32:40.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.172+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/1383120970 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2e00cbb80 con 0x7fa2e00c5410 2024-01-24T15:32:40.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.173+0000 7fa2c77fe700 1 -- 172.21.15.29:0/1383120970 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4499 (secure 0 0 0) 0x7fa2d4019020 con 0x7fa2e00c5410 2024-01-24T15:32:40.175 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:40.175 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1883755467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1883755467},{"type":"v1","addr":"172.21.15.163:6836","nonce":1883755467}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1773668659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1773668659},{"type":"v1","addr":"172.21.15.29:6834","nonce":1773668659}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3438722509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3438722509},{"type":"v1","addr":"172.21.15.29:6835","nonce":3438722509}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-24T15:32:20.685536+0000","modified":"2024-01-24T15:32:23.705717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10029":{"gid":10029,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/532321390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":532321390},{"type":"v1","addr":"172.21.15.163:6837","nonce":532321390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2024-01-24T15:32:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.175+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa2cc05f300 msgr2=0x7fa2cc0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.175+0000 7fa2e5bc3700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa2cc05f300 0x7fa2cc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.175+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/1383120970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e00c5410 msgr2=0x7fa2e00ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.175+0000 7fa2e5bc3700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e00c5410 0x7fa2e00ad120 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 rx=0x7fa2d400c810 tx=0x7fa2d4001b50).stop 2024-01-24T15:32:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.175+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/1383120970 shutdown_connections 2024-01-24T15:32:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.175+0000 7fa2e5bc3700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa2cc05f300 0x7fa2cc0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.175+0000 7fa2e5bc3700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2e0154370 0x7fa2e00c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.175+0000 7fa2e5bc3700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e00638e0 0x7fa2e00ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.175+0000 7fa2e5bc3700 1 --2- 172.21.15.29:0/1383120970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2e00c5410 0x7fa2e00ad120 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.175+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/1383120970 >> 172.21.15.29:0/1383120970 conn(0x7fa2e005ec80 msgr2=0x7fa2e00ca430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.175+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/1383120970 shutdown_connections 2024-01-24T15:32:40.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.175+0000 7fa2e5bc3700 1 -- 172.21.15.29:0/1383120970 wait complete. 2024-01-24T15:32:40.179 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 98 2024-01-24T15:32:40.192 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:32:40.223 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:40.221+0000 7f18d2708680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T15:32:40.324 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.322+0000 7fccbbaf7700 1 Processor -- start 2024-01-24T15:32:40.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.323+0000 7fccbbaf7700 1 -- start start 2024-01-24T15:32:40.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccbbaf7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fccb40c41c0 0x7fccb415ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:40.327 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccbbaf7700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb4064120 con 0x7fccb40be520 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccbbaf7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb40bf6a0 con 0x7fccb413a5c0 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccbbaf7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb40bea90 con 0x7fccb40c41c0 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccb9092700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fccb40be520 0x7fccb40664f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37226/0 (socket says 172.21.15.29:37226) 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccb9092700 1 -- 172.21.15.29:0/1689603642 learned_addr learned my addr 172.21.15.29:0/1689603642 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccba094700 1 --2- 172.21.15.29:0/1689603642 >> v2:172.21.15.163:3300/0 conn(0x7fccb40c41c0 0x7fccb415ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccb8891700 1 -- 172.21.15.29:0/1689603642 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1234423506 0 0) 0x7fccb4064120 con 0x7fccb40be520 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccb8891700 1 -- 172.21.15.29:0/1689603642 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcca00033f0 con 0x7fccb40be520 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccba094700 1 -- 172.21.15.29:0/1689603642 >> v1:172.21.15.163:6789/0 conn(0x7fccb413a5c0 legacy=0x7fccb4063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccb8891700 1 -- 172.21.15.29:0/1689603642 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733996592 0 0) 0x7fccb40bf6a0 con 0x7fccb413a5c0 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccba094700 1 -- 172.21.15.29:0/1689603642 >> v1:172.21.15.29:6789/0 conn(0x7fccb40be520 legacy=0x7fccb40664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccba094700 1 -- 172.21.15.29:0/1689603642 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb40bed70 con 0x7fccb40c41c0 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.324+0000 7fccba094700 1 --2- 172.21.15.29:0/1689603642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccb40c41c0 0x7fccb415ec60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7fccac009e60 tx=0x7fccac009780).ready entity=mon.1 client_cookie=d7c32c75be2ea9eb server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:40.328 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.325+0000 7fccb8891700 1 -- 172.21.15.29:0/1689603642 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac00e070 con 0x7fccb40c41c0 2024-01-24T15:32:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.325+0000 7fccb8891700 1 -- 172.21.15.29:0/1689603642 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccac00bb60 con 0x7fccb40c41c0 2024-01-24T15:32:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.325+0000 7fccb8891700 1 -- 172.21.15.29:0/1689603642 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac00bee0 con 0x7fccb40c41c0 2024-01-24T15:32:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.325+0000 7fccbbaf7700 1 -- 172.21.15.29:0/1689603642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccb40c41c0 msgr2=0x7fccb415ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.325+0000 7fccbbaf7700 1 --2- 172.21.15.29:0/1689603642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccb40c41c0 0x7fccb415ec60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7fccac009e60 tx=0x7fccac009780).stop 2024-01-24T15:32:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.325+0000 7fccbbaf7700 1 -- 172.21.15.29:0/1689603642 shutdown_connections 2024-01-24T15:32:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.325+0000 7fccbbaf7700 1 --2- 172.21.15.29:0/1689603642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccb40c41c0 0x7fccb415ec60 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.325+0000 7fccbbaf7700 1 -- 172.21.15.29:0/1689603642 >> 172.21.15.29:0/1689603642 conn(0x7fccb40c8bb0 msgr2=0x7fccb40c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.325+0000 7fccbbaf7700 1 -- 172.21.15.29:0/1689603642 shutdown_connections 2024-01-24T15:32:40.329 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.325+0000 7fccbbaf7700 1 -- 172.21.15.29:0/1689603642 wait complete. 2024-01-24T15:32:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.327+0000 7fccbbaf7700 1 Processor -- start 2024-01-24T15:32:40.330 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.327+0000 7fccbbaf7700 1 -- start start 2024-01-24T15:32:40.331 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.328+0000 7fccbbaf7700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccb40be520 0x7fccb40a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.328+0000 7fccbbaf7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccb40c41c0 0x7fccb413c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.328+0000 7fccbbaf7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccb413a5c0 0x7fccb413bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.328+0000 7fccbbaf7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb4155710 con 0x7fccb40c41c0 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.328+0000 7fccbbaf7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb4154b00 con 0x7fccb413a5c0 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.328+0000 7fccbbaf7700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb4154de0 con 0x7fccb40be520 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.328+0000 7fccb9092700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccb40c41c0 0x7fccb413c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.328+0000 7fccb9092700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccb40c41c0 0x7fccb413c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38662/0 (socket says 172.21.15.29:38662) 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.328+0000 7fccb9092700 1 -- 172.21.15.29:0/3831327579 learned_addr learned my addr 172.21.15.29:0/3831327579 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.328+0000 7fccba094700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccb413a5c0 0x7fccb413bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.328+0000 7fccb9893700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccb40be520 0x7fccb40a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.329+0000 7fccb9092700 1 -- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccb40be520 msgr2=0x7fccb40a9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.329+0000 7fccb9092700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccb40be520 0x7fccb40a9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.329+0000 7fccb9092700 1 -- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccb413a5c0 msgr2=0x7fccb413bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:40.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.329+0000 7fccb9092700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccb413a5c0 0x7fccb413bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.329+0000 7fccb9092700 1 -- 172.21.15.29:0/3831327579 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac009a20 con 0x7fccb40c41c0 2024-01-24T15:32:40.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.329+0000 7fccb9092700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccb40c41c0 0x7fccb413c9d0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 rx=0x7fcca4001ed0 tx=0x7fcca4007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:40.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.329+0000 7fcca9ffb700 1 -- 172.21.15.29:0/3831327579 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca400ce10 con 0x7fccb40c41c0 2024-01-24T15:32:40.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.329+0000 7fcca9ffb700 1 -- 172.21.15.29:0/3831327579 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca40072e0 con 0x7fccb40c41c0 2024-01-24T15:32:40.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.329+0000 7fccbbaf7700 1 -- 172.21.15.29:0/3831327579 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb41550c0 con 0x7fccb40c41c0 2024-01-24T15:32:40.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.329+0000 7fcca9ffb700 1 -- 172.21.15.29:0/3831327579 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca40057a0 con 0x7fccb40c41c0 2024-01-24T15:32:40.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.329+0000 7fccbbaf7700 1 -- 172.21.15.29:0/3831327579 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb413cec0 con 0x7fccb40c41c0 2024-01-24T15:32:40.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.331+0000 7fccbbaf7700 1 -- 172.21.15.29:0/3831327579 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb40475e0 con 0x7fccb40c41c0 2024-01-24T15:32:40.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.331+0000 7fcca9ffb700 1 -- 172.21.15.29:0/3831327579 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fcca4005b50 con 0x7fccb40c41c0 2024-01-24T15:32:40.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.331+0000 7fcca9ffb700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc9805f300 0x7fcc980617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:40.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.331+0000 7fccb9893700 1 -- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc9805f300 msgr2=0x7fcc980617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/57365 2024-01-24T15:32:40.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.331+0000 7fcca9ffb700 1 -- 172.21.15.29:0/3831327579 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 7472+0+0 (secure 0 0 0) 0x7fcca4079fc0 con 0x7fccb40c41c0 2024-01-24T15:32:40.334 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.331+0000 7fccb9893700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc9805f300 0x7fcc980617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-01-24T15:32:40.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.336+0000 7fcca9ffb700 1 -- 172.21.15.29:0/3831327579 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fcca404ab70 con 0x7fccb40c41c0 2024-01-24T15:32:40.377 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.375+0000 7f51ebb85680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:32:40.400 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:40.398+0000 7f18d2708680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T15:32:40.448 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.447+0000 7f51ebb85680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:32:40.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.469+0000 7fcca9ffb700 1 -- 172.21.15.29:0/3831327579 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 103) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fcca4077670 con 0x7fccb40c41c0 2024-01-24T15:32:40.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.469+0000 7fcca9ffb700 1 -- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc9805f300 msgr2=0x7fcc980617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T15:32:40.471 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.469+0000 7fcca9ffb700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc9805f300 0x7fcc980617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:40.487 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:40.485+0000 7f18d2708680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T15:32:40.529 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.527+0000 7fccbbaf7700 1 -- 172.21.15.29:0/3831327579 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccb40bf050 con 0x7fccb40c41c0 2024-01-24T15:32:40.530 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.528+0000 7fcca9ffb700 1 -- 172.21.15.29:0/3831327579 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v233) v1 ==== 74+0+18988 (secure 0 0 0) 0x7fcca4073f60 con 0x7fccb40c41c0 2024-01-24T15:32:40.530 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:40.530 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":233,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:32:40.461478+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-01-24T15:32:18.425709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-01-24T15:32:19.656025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"231","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/2645482892":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/4093084652":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/3512593770":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/343370470":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/4238824203":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/3387006926":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/3908030498":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/741220992":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/2316406386":"2024-01-25T15:32:33.173546+0000","172.21.15.163:6833/57365":"2024-01-25T15:32:33.173546+0000","172.21.15.163:6832/57365":"2024-01-25T15:32:33.173546+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:32:40.531 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.529+0000 7f516810c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:40.531 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.529+0000 7f516810c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:40.531 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.529+0000 7f5163903700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:40.531 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.529+0000 7f516810c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:40.531 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.529+0000 7f5163903700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:40.531 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.529+0000 7f516810c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:40.531 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.529+0000 7f5163903700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:40.531 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.529+0000 7f516810c700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:40.531 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.529+0000 7f5163903700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:40.531 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-01-24T15:32:40.529+0000 7f5163903700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-24T15:32:40.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.530+0000 7fccbbaf7700 1 -- 172.21.15.29:0/3831327579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccb40c41c0 msgr2=0x7fccb413c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:40.532 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.530+0000 7fccbbaf7700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccb40c41c0 0x7fccb413c9d0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 rx=0x7fcca4001ed0 tx=0x7fcca4007c70).stop 2024-01-24T15:32:40.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.531+0000 7fccbbaf7700 1 -- 172.21.15.29:0/3831327579 shutdown_connections 2024-01-24T15:32:40.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.531+0000 7fccbbaf7700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccb413a5c0 0x7fccb413bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.531+0000 7fccbbaf7700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fcc9805f300 0x7fcc980617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:40.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.531+0000 7fccbbaf7700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccb40be520 0x7fccb40a9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.531+0000 7fccbbaf7700 1 --2- 172.21.15.29:0/3831327579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fccb40c41c0 0x7fccb413c9d0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.531+0000 7fccbbaf7700 1 -- 172.21.15.29:0/3831327579 >> 172.21.15.29:0/3831327579 conn(0x7fccb40c8bb0 msgr2=0x7fccb4063880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:40.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.531+0000 7fccbbaf7700 1 -- 172.21.15.29:0/3831327579 shutdown_connections 2024-01-24T15:32:40.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.531+0000 7fccbbaf7700 1 -- 172.21.15.29:0/3831327579 wait complete. 2024-01-24T15:32:40.548 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-24T15:32:40.626 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:40.624+0000 7f18d2708680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T15:32:40.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.718+0000 7f37278cf700 1 Processor -- start 2024-01-24T15:32:40.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.718+0000 7f37278cf700 1 -- start start 2024-01-24T15:32:40.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.719+0000 7f37278cf700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37200a9c40 0x7f37200aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:40.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f37278cf700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3720062ec0 0x7f37200bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:40.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f37278cf700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3720069c10 con 0x7f37200c4c60 2024-01-24T15:32:40.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f37278cf700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f372006a280 con 0x7f3720062ec0 2024-01-24T15:32:40.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f37278cf700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37200be540 con 0x7f37200a9c40 2024-01-24T15:32:40.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f3724e6a700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f37200c4c60 0x7f37200627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37242/0 (socket says 172.21.15.29:37242) 2024-01-24T15:32:40.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f3724e6a700 1 -- 172.21.15.29:0/2540821640 learned_addr learned my addr 172.21.15.29:0/2540821640 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:40.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f372566b700 1 --2- 172.21.15.29:0/2540821640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37200a9c40 0x7f37200aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:40.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f3725e6c700 1 --2- 172.21.15.29:0/2540821640 >> v2:172.21.15.163:3300/0 conn(0x7f3720062ec0 0x7f37200bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:40.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f3717fff700 1 -- 172.21.15.29:0/2540821640 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866195481 0 0) 0x7f3720069c10 con 0x7f37200c4c60 2024-01-24T15:32:40.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f3717fff700 1 -- 172.21.15.29:0/2540821640 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f370c0033f0 con 0x7f37200c4c60 2024-01-24T15:32:40.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f372566b700 1 -- 172.21.15.29:0/2540821640 >> v2:172.21.15.163:3300/0 conn(0x7f3720062ec0 msgr2=0x7f37200bca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:40.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f3717fff700 1 -- 172.21.15.29:0/2540821640 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3956659821 0 0) 0x7f370c0033f0 con 0x7f37200c4c60 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f372566b700 1 --2- 172.21.15.29:0/2540821640 >> v2:172.21.15.163:3300/0 conn(0x7f3720062ec0 0x7f37200bca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f372566b700 1 -- 172.21.15.29:0/2540821640 >> v1:172.21.15.29:6789/0 conn(0x7f37200c4c60 legacy=0x7f37200627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f372566b700 1 -- 172.21.15.29:0/2540821640 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37200be820 con 0x7f37200a9c40 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.720+0000 7f3725e6c700 1 --2- 172.21.15.29:0/2540821640 >> v2:172.21.15.163:3300/0 conn(0x7f3720062ec0 0x7f37200bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.721+0000 7f372566b700 1 --2- 172.21.15.29:0/2540821640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37200a9c40 0x7f37200aa020 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f371c009760 tx=0x7f371c009060).ready entity=mon.2 client_cookie=1437eb377587baef server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.721+0000 7f3717fff700 1 -- 172.21.15.29:0/2540821640 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710001470 con 0x7f37200a9c40 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.721+0000 7f3717fff700 1 -- 172.21.15.29:0/2540821640 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c011400 con 0x7f37200a9c40 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.721+0000 7f3717fff700 1 -- 172.21.15.29:0/2540821640 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c011780 con 0x7f37200a9c40 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.721+0000 7f37278cf700 1 -- 172.21.15.29:0/2540821640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37200a9c40 msgr2=0x7f37200aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.721+0000 7f37278cf700 1 --2- 172.21.15.29:0/2540821640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37200a9c40 0x7f37200aa020 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f371c009760 tx=0x7f371c009060).stop 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.721+0000 7f37278cf700 1 -- 172.21.15.29:0/2540821640 shutdown_connections 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.721+0000 7f37278cf700 1 --2- 172.21.15.29:0/2540821640 >> v2:172.21.15.163:3300/0 conn(0x7f3720062ec0 0x7f37200bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.721+0000 7f37278cf700 1 --2- 172.21.15.29:0/2540821640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37200a9c40 0x7f37200aa020 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.721+0000 7f37278cf700 1 -- 172.21.15.29:0/2540821640 >> 172.21.15.29:0/2540821640 conn(0x7f37200c8580 msgr2=0x7f37200c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:40.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.721+0000 7f37278cf700 1 -- 172.21.15.29:0/2540821640 shutdown_connections 2024-01-24T15:32:40.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.721+0000 7f37278cf700 1 -- 172.21.15.29:0/2540821640 wait complete. 2024-01-24T15:32:40.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.723+0000 7f37278cf700 1 Processor -- start 2024-01-24T15:32:40.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.723+0000 7f37278cf700 1 -- start start 2024-01-24T15:32:40.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.724+0000 7f37278cf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3720062ec0 0x7f37200a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:40.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.724+0000 7f37278cf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37200a9c40 0x7f37200bfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:40.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.724+0000 7f37278cf700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37200c4c60 0x7f37200c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:40.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.724+0000 7f37278cf700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37200c2610 con 0x7f3720062ec0 2024-01-24T15:32:40.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.724+0000 7f37278cf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37201650c0 con 0x7f37200a9c40 2024-01-24T15:32:40.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.724+0000 7f37278cf700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37201653a0 con 0x7f37200c4c60 2024-01-24T15:32:40.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f372566b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3720062ec0 0x7f37200a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:40.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f372566b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3720062ec0 0x7f37200a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38678/0 (socket says 172.21.15.29:38678) 2024-01-24T15:32:40.728 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f372566b700 1 -- 172.21.15.29:0/2513600610 learned_addr learned my addr 172.21.15.29:0/2513600610 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f3725e6c700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37200c4c60 0x7f37200c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f3724e6a700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37200a9c40 0x7f37200bfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f372566b700 1 -- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37200c4c60 msgr2=0x7f37200c01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f372566b700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37200c4c60 0x7f37200c01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f372566b700 1 -- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37200a9c40 msgr2=0x7f37200bfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f372566b700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37200a9c40 0x7f37200bfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f372566b700 1 -- 172.21.15.29:0/2513600610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3718009a20 con 0x7f3720062ec0 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f372566b700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3720062ec0 0x7f37200a4fd0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 rx=0x7f37200be680 tx=0x7f371c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f37157fa700 1 -- 172.21.15.29:0/2513600610 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c010740 con 0x7f3720062ec0 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f37157fa700 1 -- 172.21.15.29:0/2513600610 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c0108a0 con 0x7f3720062ec0 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f37278cf700 1 -- 172.21.15.29:0/2513600610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c0092a0 con 0x7f3720062ec0 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f37157fa700 1 -- 172.21.15.29:0/2513600610 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c010c40 con 0x7f3720062ec0 2024-01-24T15:32:40.729 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.725+0000 7f37278cf700 1 -- 172.21.15.29:0/2513600610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37200a54c0 con 0x7f3720062ec0 2024-01-24T15:32:40.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.727+0000 7f37278cf700 1 -- 172.21.15.29:0/2513600610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37200475e0 con 0x7f3720062ec0 2024-01-24T15:32:40.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.727+0000 7f37157fa700 1 -- 172.21.15.29:0/2513600610 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 103) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f371c01d640 con 0x7f3720062ec0 2024-01-24T15:32:40.730 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.727+0000 7f37157fa700 1 -- 172.21.15.29:0/2513600610 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 7687+0+0 (secure 0 0 0) 0x7f371c07ba80 con 0x7f3720062ec0 2024-01-24T15:32:40.733 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.732+0000 7f37157fa700 1 -- 172.21.15.29:0/2513600610 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f371c04c900 con 0x7f3720062ec0 2024-01-24T15:32:40.754 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:40.752+0000 7f18d2708680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T15:32:40.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:40.961+0000 7f37278cf700 1 -- 172.21.15.29:0/2513600610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f37201522b0 con 0x7f3720062ec0 2024-01-24T15:32:41.094 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:41.092+0000 7f18d2708680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T15:32:41.176 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-01-24T15:32:41.174+0000 7f18d2708680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T15:32:41.466 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-01-24T15:32:41.464+0000 7fe09bfc3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-24T15:32:41.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.472+0000 7f37157fa700 1 -- 172.21.15.29:0/2513600610 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 104) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f371c079400 con 0x7f3720062ec0 2024-01-24T15:32:41.474 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.472+0000 7f37157fa700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f37040302c0 0x7f3704068a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:41.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.474+0000 7f3724e6a700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f37040302c0 0x7f3704068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:41.476 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.475+0000 7f3724e6a700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f37040302c0 0x7f3704068a70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f37200649c0 tx=0x7f37100011a0).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:41.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.480+0000 7f37157fa700 1 -- 172.21.15.29:0/2513600610 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) v1 ==== 161+0+0 (secure 0 0 0) 0x7f371c054390 con 0x7f3720062ec0 2024-01-24T15:32:41.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.483+0000 7f37278cf700 1 -- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f37040302c0 msgr2=0x7f3704068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:41.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.483+0000 7f37278cf700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f37040302c0 0x7f3704068a70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f37200649c0 tx=0x7f37100011a0).stop 2024-01-24T15:32:41.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.483+0000 7f37278cf700 1 -- 172.21.15.29:0/2513600610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3720062ec0 msgr2=0x7f37200a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:41.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.483+0000 7f37278cf700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3720062ec0 0x7f37200a4fd0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 rx=0x7f37200be680 tx=0x7f371c016000).stop 2024-01-24T15:32:41.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.483+0000 7f37278cf700 1 -- 172.21.15.29:0/2513600610 shutdown_connections 2024-01-24T15:32:41.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.483+0000 7f37278cf700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f37040302c0 0x7f3704068a70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:41.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.483+0000 7f37278cf700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37200c4c60 0x7f37200c01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:41.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.483+0000 7f37278cf700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3720062ec0 0x7f37200a4fd0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:41.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.483+0000 7f37278cf700 1 --2- 172.21.15.29:0/2513600610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37200a9c40 0x7f37200bfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:41.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.483+0000 7f37278cf700 1 -- 172.21.15.29:0/2513600610 >> 172.21.15.29:0/2513600610 conn(0x7f37200c8580 msgr2=0x7f37200c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:41.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.483+0000 7f37278cf700 1 -- 172.21.15.29:0/2513600610 shutdown_connections 2024-01-24T15:32:41.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.483+0000 7f37278cf700 1 -- 172.21.15.29:0/2513600610 wait complete. 2024-01-24T15:32:41.487 INFO:tasks.ceph.mds.d.smithi163.stderr: -2712> 2024-01-24T15:32:25.562+0000 7f8b8aa13700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-24T15:32:41.487 INFO:teuthology.orchestra.run.smithi029.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-24T15:32:41.501 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-24T15:32:41.509 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-01-24T15:32:41.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.680+0000 7fbb4493a700 1 Processor -- start 2024-01-24T15:32:41.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.680+0000 7fbb4493a700 1 -- start start 2024-01-24T15:32:41.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.681+0000 7fbb4493a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb4014b710 0x7fbb400c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.681+0000 7fbb4493a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb40155840 con 0x7fbb4014b330 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.681+0000 7fbb4493a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb40069280 con 0x7fbb401550c0 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.681+0000 7fbb4493a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb400c3910 con 0x7fbb4014b710 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.681+0000 7fbb3e59c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbb4014b330 0x7fbb401549b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37250/0 (socket says 172.21.15.29:37250) 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.681+0000 7fbb3e59c700 1 -- 172.21.15.29:0/1188825389 learned_addr learned my addr 172.21.15.29:0/1188825389 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb37fff700 1 --2- 172.21.15.29:0/1188825389 >> v2:172.21.15.163:3300/0 conn(0x7fbb4014b710 0x7fbb400c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813030502 0 0) 0x7fbb40155840 con 0x7fbb4014b330 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb2c0033f0 con 0x7fbb4014b330 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256718413 0 0) 0x7fbb40069280 con 0x7fbb401550c0 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb40155840 con 0x7fbb401550c0 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2015235746 0 0) 0x7fbb2c0033f0 con 0x7fbb4014b330 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 >> v2:172.21.15.163:3300/0 conn(0x7fbb4014b710 msgr2=0x7fbb400c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:41.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 --2- 172.21.15.29:0/1188825389 >> v2:172.21.15.163:3300/0 conn(0x7fbb4014b710 0x7fbb400c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 >> v1:172.21.15.163:6789/0 conn(0x7fbb401550c0 legacy=0x7fbb400c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb400c3bf0 con 0x7fbb4014b330 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fbb30001410 con 0x7fbb4014b330 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb37fff700 1 --2- 172.21.15.29:0/1188825389 >> v2:172.21.15.163:3300/0 conn(0x7fbb4014b710 0x7fbb400c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 >> v1:172.21.15.29:6789/0 conn(0x7fbb4014b330 legacy=0x7fbb401549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 --2- 172.21.15.29:0/1188825389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb2c003ac0 0x7fbb2c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 --2- 172.21.15.29:0/1188825389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb2c004690 0x7fbb2c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 --2- 172.21.15.29:0/1188825389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb2c005000 0x7fbb2c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400c3910 con 0x7fbb2c003ac0 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c00bf60 con 0x7fbb2c004690 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c00c120 con 0x7fbb2c005000 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3e59c700 1 --2- 172.21.15.29:0/1188825389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb2c003ac0 0x7fbb2c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb37fff700 1 --2- 172.21.15.29:0/1188825389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb2c004690 0x7fbb2c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:41.688 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.682+0000 7fbb3ed9d700 1 --2- 172.21.15.29:0/1188825389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb2c005000 0x7fbb2c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb3e59c700 1 -- 172.21.15.29:0/1188825389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb2c005000 msgr2=0x7fbb2c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb3e59c700 1 --2- 172.21.15.29:0/1188825389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb2c005000 0x7fbb2c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb3e59c700 1 -- 172.21.15.29:0/1188825389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb2c004690 msgr2=0x7fbb2c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb3e59c700 1 --2- 172.21.15.29:0/1188825389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb2c004690 0x7fbb2c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb3e59c700 1 -- 172.21.15.29:0/1188825389 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c0033f0 con 0x7fbb2c003ac0 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb3ed9d700 1 -- 172.21.15.29:0/1188825389 reap_dead start 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb37fff700 1 --2- 172.21.15.29:0/1188825389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb2c004690 0x7fbb2c004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb3e59c700 1 --2- 172.21.15.29:0/1188825389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb2c003ac0 0x7fbb2c003ec0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 rx=0x7fbb30007ef0 tx=0x7fbb30001ce0).ready entity=mon.0 client_cookie=d34b1fac04ef94f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb30007170 con 0x7fbb2c003ac0 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb300072d0 con 0x7fbb2c003ac0 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb3dd9b700 1 -- 172.21.15.29:0/1188825389 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3000c980 con 0x7fbb2c003ac0 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb4493a700 1 -- 172.21.15.29:0/1188825389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb2c003ac0 msgr2=0x7fbb2c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.683+0000 7fbb4493a700 1 --2- 172.21.15.29:0/1188825389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb2c003ac0 0x7fbb2c003ec0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 rx=0x7fbb30007ef0 tx=0x7fbb30001ce0).stop 2024-01-24T15:32:41.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.684+0000 7fbb4493a700 1 -- 172.21.15.29:0/1188825389 shutdown_connections 2024-01-24T15:32:41.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.684+0000 7fbb4493a700 1 --2- 172.21.15.29:0/1188825389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb2c003ac0 0x7fbb2c003ec0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:41.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.684+0000 7fbb4493a700 1 -- 172.21.15.29:0/1188825389 >> 172.21.15.29:0/1188825389 conn(0x7fbb40064390 msgr2=0x7fbb400c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:41.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.684+0000 7fbb4493a700 1 -- 172.21.15.29:0/1188825389 shutdown_connections 2024-01-24T15:32:41.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.684+0000 7fbb4493a700 1 -- 172.21.15.29:0/1188825389 wait complete. 2024-01-24T15:32:41.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.685+0000 7fbb4493a700 1 Processor -- start 2024-01-24T15:32:41.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.685+0000 7fbb4493a700 1 -- start start 2024-01-24T15:32:41.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.686+0000 7fbb4493a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb2c003ac0 0x7fbb400bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:41.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.686+0000 7fbb4493a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb40147510 0x7fbb40144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:41.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.686+0000 7fbb4493a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb401451e0 0x7fbb4014a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:41.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.686+0000 7fbb4493a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400bfbc0 con 0x7fbb40147510 2024-01-24T15:32:41.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.686+0000 7fbb4493a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb40149d10 con 0x7fbb401451e0 2024-01-24T15:32:41.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.686+0000 7fbb4493a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb40147c80 con 0x7fbb2c003ac0 2024-01-24T15:32:41.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.686+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb40147510 0x7fbb40144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:41.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.686+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb40147510 0x7fbb40144cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38696/0 (socket says 172.21.15.29:38696) 2024-01-24T15:32:41.693 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb37fff700 1 -- 172.21.15.29:0/1144453072 learned_addr learned my addr 172.21.15.29:0/1144453072 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb3e59c700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb2c003ac0 0x7fbb400bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb3ed9d700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb401451e0 0x7fbb4014a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb37fff700 1 -- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb2c003ac0 msgr2=0x7fbb400bf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb37fff700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb2c003ac0 0x7fbb400bf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb37fff700 1 -- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb401451e0 msgr2=0x7fbb4014a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb37fff700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb401451e0 0x7fbb4014a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb37fff700 1 -- 172.21.15.29:0/1144453072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb40147f60 con 0x7fbb40147510 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb3ed9d700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb401451e0 0x7fbb4014a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb37fff700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb40147510 0x7fbb40144cf0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 rx=0x7fbb28011d00 tx=0x7fbb28009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb367fc700 1 -- 172.21.15.29:0/1144453072 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800fac0 con 0x7fbb40147510 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb367fc700 1 -- 172.21.15.29:0/1144453072 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2800fc20 con 0x7fbb40147510 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb4493a700 1 -- 172.21.15.29:0/1144453072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb401499a0 con 0x7fbb40147510 2024-01-24T15:32:41.694 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb367fc700 1 -- 172.21.15.29:0/1144453072 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb28014440 con 0x7fbb40147510 2024-01-24T15:32:41.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.687+0000 7fbb4493a700 1 -- 172.21.15.29:0/1144453072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb40149650 con 0x7fbb40147510 2024-01-24T15:32:41.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.688+0000 7fbb4493a700 1 -- 172.21.15.29:0/1144453072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb400475e0 con 0x7fbb40147510 2024-01-24T15:32:41.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.689+0000 7fbb367fc700 1 -- 172.21.15.29:0/1144453072 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 104) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fbb28014870 con 0x7fbb40147510 2024-01-24T15:32:41.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.689+0000 7fbb367fc700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbb2005f030 0x7fbb200614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:41.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.689+0000 7fbb367fc700 1 -- 172.21.15.29:0/1144453072 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 7773+0+0 (secure 0 0 0) 0x7fbb2807adc0 con 0x7fbb40147510 2024-01-24T15:32:41.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.689+0000 7fbb3e59c700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbb2005f030 0x7fbb200614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:41.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.690+0000 7fbb3e59c700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbb2005f030 0x7fbb200614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fbb30007d00 tx=0x7fbb30015000).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:41.695 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.693+0000 7fbb367fc700 1 -- 172.21.15.29:0/1144453072 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fbb2804bbf0 con 0x7fbb40147510 2024-01-24T15:32:41.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:41.922+0000 7fbb4493a700 1 -- 172.21.15.29:0/1144453072 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbb400c80e0 con 0x7fbb40147510 2024-01-24T15:32:42.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.488+0000 7fbb367fc700 1 -- 172.21.15.29:0/1144453072 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v100) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbb28018070 con 0x7fbb40147510 2024-01-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.491+0000 7fbb4493a700 1 -- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbb2005f030 msgr2=0x7fbb200614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.491+0000 7fbb4493a700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbb2005f030 0x7fbb200614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fbb30007d00 tx=0x7fbb30015000).stop 2024-01-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.491+0000 7fbb4493a700 1 -- 172.21.15.29:0/1144453072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb40147510 msgr2=0x7fbb40144cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:42.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.491+0000 7fbb4493a700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb40147510 0x7fbb40144cf0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 rx=0x7fbb28011d00 tx=0x7fbb28009060).stop 2024-01-24T15:32:42.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.491+0000 7fbb4493a700 1 -- 172.21.15.29:0/1144453072 shutdown_connections 2024-01-24T15:32:42.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.491+0000 7fbb4493a700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb401451e0 0x7fbb4014a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:42.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.491+0000 7fbb4493a700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fbb2005f030 0x7fbb200614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:42.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.491+0000 7fbb4493a700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb2c003ac0 0x7fbb400bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:42.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.491+0000 7fbb4493a700 1 --2- 172.21.15.29:0/1144453072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb40147510 0x7fbb40144cf0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:42.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.491+0000 7fbb4493a700 1 -- 172.21.15.29:0/1144453072 >> 172.21.15.29:0/1144453072 conn(0x7fbb40064390 msgr2=0x7fbb400c2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:42.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.491+0000 7fbb4493a700 1 -- 172.21.15.29:0/1144453072 shutdown_connections 2024-01-24T15:32:42.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.491+0000 7fbb4493a700 1 -- 172.21.15.29:0/1144453072 wait complete. 2024-01-24T15:32:42.509 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-24T15:32:42.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.644+0000 7f36d02cb700 1 Processor -- start 2024-01-24T15:32:42.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.644+0000 7f36d02cb700 1 -- start start 2024-01-24T15:32:42.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.645+0000 7f36d02cb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36c8156060 0x7f36c81594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.645+0000 7f36d02cb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f36c80be070 0x7f36c80be520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36d02cb700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c80bdbe0 con 0x7f36c80bca50 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36d02cb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c80bcfd0 con 0x7f36c80be070 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36d02cb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c80bd2b0 con 0x7f36c8156060 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36cd866700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f36c80bca50 0x7f36c8159c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:37260/0 (socket says 172.21.15.29:37260) 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36cd866700 1 -- 172.21.15.29:0/379520433 learned_addr learned my addr 172.21.15.29:0/379520433 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36ce067700 1 --2- 172.21.15.29:0/379520433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36c8156060 0x7f36c81594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36ce868700 1 --2- 172.21.15.29:0/379520433 >> v2:172.21.15.163:3300/0 conn(0x7f36c80be070 0x7f36c80be520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36cd065700 1 -- 172.21.15.29:0/379520433 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812515922 0 0) 0x7f36c80bdbe0 con 0x7f36c80bca50 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36cd065700 1 -- 172.21.15.29:0/379520433 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36b40033f0 con 0x7f36c80bca50 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36cd065700 1 -- 172.21.15.29:0/379520433 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2261010675 0 0) 0x7f36b40033f0 con 0x7f36c80bca50 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36ce868700 1 -- 172.21.15.29:0/379520433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36c8156060 msgr2=0x7f36c81594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:42.650 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36ce868700 1 --2- 172.21.15.29:0/379520433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36c8156060 0x7f36c81594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:42.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36ce868700 1 -- 172.21.15.29:0/379520433 >> v1:172.21.15.29:6789/0 conn(0x7f36c80bca50 legacy=0x7f36c8159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:42.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.646+0000 7f36ce868700 1 -- 172.21.15.29:0/379520433 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c80bd590 con 0x7f36c80be070 2024-01-24T15:32:42.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.647+0000 7f36ce868700 1 --2- 172.21.15.29:0/379520433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c80be070 0x7f36c80be520 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f36c0009e60 tx=0x7f36c0009780).ready entity=mon.1 client_cookie=660cf86ba2e0827f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:42.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.647+0000 7f36cd065700 1 -- 172.21.15.29:0/379520433 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c000e070 con 0x7f36c80be070 2024-01-24T15:32:42.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.647+0000 7f36cd065700 1 -- 172.21.15.29:0/379520433 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c000b830 con 0x7f36c80be070 2024-01-24T15:32:42.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.647+0000 7f36cd065700 1 -- 172.21.15.29:0/379520433 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4001470 con 0x7f36c80be070 2024-01-24T15:32:42.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.647+0000 7f36d02cb700 1 -- 172.21.15.29:0/379520433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c80be070 msgr2=0x7f36c80be520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:42.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.647+0000 7f36d02cb700 1 --2- 172.21.15.29:0/379520433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c80be070 0x7f36c80be520 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f36c0009e60 tx=0x7f36c0009780).stop 2024-01-24T15:32:42.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.647+0000 7f36d02cb700 1 -- 172.21.15.29:0/379520433 shutdown_connections 2024-01-24T15:32:42.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.647+0000 7f36d02cb700 1 --2- 172.21.15.29:0/379520433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c80be070 0x7f36c80be520 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.647+0000 7f36d02cb700 1 --2- 172.21.15.29:0/379520433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36c8156060 0x7f36c81594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.647+0000 7f36d02cb700 1 -- 172.21.15.29:0/379520433 >> 172.21.15.29:0/379520433 conn(0x7f36c80c7af0 msgr2=0x7f36c80c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.647+0000 7f36d02cb700 1 -- 172.21.15.29:0/379520433 shutdown_connections 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.647+0000 7f36d02cb700 1 -- 172.21.15.29:0/379520433 wait complete. 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.649+0000 7f36d02cb700 1 Processor -- start 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.649+0000 7f36d02cb700 1 -- start start 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.649+0000 7f36d02cb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36c80bca50 0x7f36c8147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.649+0000 7f36d02cb700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36c80be070 0x7f36c80a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.649+0000 7f36d02cb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c8156060 0x7f36c80a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.649+0000 7f36d02cb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c80a9bb0 con 0x7f36c80bca50 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.649+0000 7f36d02cb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c81555e0 con 0x7f36c8156060 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.649+0000 7f36d02cb700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c80bfcd0 con 0x7f36c80be070 2024-01-24T15:32:42.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36ce067700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36c80bca50 0x7f36c8147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36ce067700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36c80bca50 0x7f36c8147e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:38712/0 (socket says 172.21.15.29:38712) 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36ce067700 1 -- 172.21.15.29:0/646302816 learned_addr learned my addr 172.21.15.29:0/646302816 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36cd866700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36c80be070 0x7f36c80a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36ce868700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c8156060 0x7f36c80a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36ce067700 1 -- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36c80be070 msgr2=0x7f36c80a91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36ce067700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36c80be070 0x7f36c80a91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36ce067700 1 -- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c8156060 msgr2=0x7f36c80a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36ce067700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c8156060 0x7f36c80a96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36ce067700 1 -- 172.21.15.29:0/646302816 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c0009a20 con 0x7f36c80bca50 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36ce067700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36c80bca50 0x7f36c8147e80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 rx=0x7f36b800e960 tx=0x7f36b8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36be7fc700 1 -- 172.21.15.29:0/646302816 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8012600 con 0x7f36c80bca50 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36be7fc700 1 -- 172.21.15.29:0/646302816 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b8012760 con 0x7f36c80bca50 2024-01-24T15:32:42.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36d02cb700 1 -- 172.21.15.29:0/646302816 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c8148370 con 0x7f36c80bca50 2024-01-24T15:32:42.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36be7fc700 1 -- 172.21.15.29:0/646302816 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8012ae0 con 0x7f36c80bca50 2024-01-24T15:32:42.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.650+0000 7f36d02cb700 1 -- 172.21.15.29:0/646302816 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c815d8a0 con 0x7f36c80bca50 2024-01-24T15:32:42.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.652+0000 7f36d02cb700 1 -- 172.21.15.29:0/646302816 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c80475e0 con 0x7f36c80bca50 2024-01-24T15:32:42.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.652+0000 7f36be7fc700 1 -- 172.21.15.29:0/646302816 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 104) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f36b801b400 con 0x7f36c80bca50 2024-01-24T15:32:42.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.652+0000 7f36be7fc700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f36ac05f030 0x7f36ac0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:42.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.652+0000 7f36be7fc700 1 -- 172.21.15.29:0/646302816 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7773+0+0 (secure 0 0 0) 0x7f36b807a9b0 con 0x7f36c80bca50 2024-01-24T15:32:42.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.652+0000 7f36cd866700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f36ac05f030 0x7f36ac0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:42.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.653+0000 7f36cd866700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f36ac05f030 0x7f36ac0614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f36c4001ed0 tx=0x7f36c40011a0).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:42.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.657+0000 7f36be7fc700 1 -- 172.21.15.29:0/646302816 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f36b804b780 con 0x7f36c80bca50 2024-01-24T15:32:42.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:42.838+0000 7f36d02cb700 1 -- 172.21.15.29:0/646302816 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f36c80c66d0 con 0x7f36c80bca50 2024-01-24T15:32:43.011 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:32:43.013 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:32:43.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.248+0000 7f36be7fc700 1 -- 172.21.15.29:0/646302816 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f36b80782e0 con 0x7f36c80bca50 2024-01-24T15:32:43.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.491+0000 7f36be7fc700 1 -- 172.21.15.29:0/646302816 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v236) v1 ==== 181+0+0 (secure 0 0 0) 0x7f36b8071d40 con 0x7f36c80bca50 2024-01-24T15:32:43.498 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.496+0000 7f36d02cb700 1 -- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f36ac05f030 msgr2=0x7f36ac0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:43.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.496+0000 7f36d02cb700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f36ac05f030 0x7f36ac0614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f36c4001ed0 tx=0x7f36c40011a0).stop 2024-01-24T15:32:43.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.496+0000 7f36d02cb700 1 -- 172.21.15.29:0/646302816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36c80bca50 msgr2=0x7f36c8147e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:43.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.496+0000 7f36d02cb700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36c80bca50 0x7f36c8147e80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 rx=0x7f36b800e960 tx=0x7f36b8009060).stop 2024-01-24T15:32:43.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.496+0000 7f36d02cb700 1 -- 172.21.15.29:0/646302816 shutdown_connections 2024-01-24T15:32:43.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.496+0000 7f36d02cb700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c8156060 0x7f36c80a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:43.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.496+0000 7f36d02cb700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36c80bca50 0x7f36c8147e80 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:43.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.496+0000 7f36d02cb700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f36ac05f030 0x7f36ac0614f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:43.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.496+0000 7f36d02cb700 1 --2- 172.21.15.29:0/646302816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36c80be070 0x7f36c80a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:43.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.496+0000 7f36d02cb700 1 -- 172.21.15.29:0/646302816 >> 172.21.15.29:0/646302816 conn(0x7f36c80c7af0 msgr2=0x7f36c8156440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:43.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.496+0000 7f36d02cb700 1 -- 172.21.15.29:0/646302816 shutdown_connections 2024-01-24T15:32:43.499 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.496+0000 7f36d02cb700 1 -- 172.21.15.29:0/646302816 wait complete. 2024-01-24T15:32:43.500 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_metadata' removed 2024-01-24T15:32:43.514 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.651+0000 7feb967dc700 1 Processor -- start 2024-01-24T15:32:43.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.651+0000 7feb967dc700 1 -- start start 2024-01-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.651+0000 7feb967dc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feb9014e380 0x7feb900642d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.651+0000 7feb967dc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feb90151850 0x7feb900647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb967dc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb90150b30 con 0x7feb9014dfa0 2024-01-24T15:32:43.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb967dc700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb90064cb0 con 0x7feb90151850 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb967dc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb901513c0 con 0x7feb9014e380 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb94d79700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feb90151850 0x7feb900647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb94d79700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feb90151850 0x7feb900647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52356/0 (socket says 172.21.15.29:52356) 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb94d79700 1 -- 172.21.15.29:0/1408748842 learned_addr learned my addr 172.21.15.29:0/1408748842 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb8f7fe700 1 --2- 172.21.15.29:0/1408748842 >> v2:172.21.15.163:3300/0 conn(0x7feb9014e380 0x7feb900642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb94d79700 1 -- 172.21.15.29:0/1408748842 >> v2:172.21.15.163:3300/0 conn(0x7feb9014e380 msgr2=0x7feb900642d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb94d79700 1 --2- 172.21.15.29:0/1408748842 >> v2:172.21.15.163:3300/0 conn(0x7feb9014e380 0x7feb900642d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb94d79700 1 -- 172.21.15.29:0/1408748842 >> v1:172.21.15.163:6789/0 conn(0x7feb9014dfa0 legacy=0x7feb900c1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb94d79700 1 -- 172.21.15.29:0/1408748842 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb90065990 con 0x7feb90151850 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb94d79700 1 --2- 172.21.15.29:0/1408748842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb90151850 0x7feb900647c0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 rx=0x7feb90064f30 tx=0x7feb8800ba50).ready entity=mon.0 client_cookie=cc193abae85c0ce6 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb8effd700 1 -- 172.21.15.29:0/1408748842 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88010070 con 0x7feb90151850 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb8effd700 1 -- 172.21.15.29:0/1408748842 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8800de50 con 0x7feb90151850 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.652+0000 7feb8effd700 1 -- 172.21.15.29:0/1408748842 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8800b2e0 con 0x7feb90151850 2024-01-24T15:32:43.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.653+0000 7feb967dc700 1 -- 172.21.15.29:0/1408748842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb90151850 msgr2=0x7feb900647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.653+0000 7feb967dc700 1 --2- 172.21.15.29:0/1408748842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb90151850 0x7feb900647c0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 rx=0x7feb90064f30 tx=0x7feb8800ba50).stop 2024-01-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.653+0000 7feb967dc700 1 -- 172.21.15.29:0/1408748842 shutdown_connections 2024-01-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.653+0000 7feb967dc700 1 --2- 172.21.15.29:0/1408748842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb90151850 0x7feb900647c0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.653+0000 7feb967dc700 1 --2- 172.21.15.29:0/1408748842 >> v2:172.21.15.163:3300/0 conn(0x7feb9014e380 0x7feb900642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.653+0000 7feb967dc700 1 -- 172.21.15.29:0/1408748842 >> 172.21.15.29:0/1408748842 conn(0x7feb900627b0 msgr2=0x7feb90062bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.653+0000 7feb967dc700 1 -- 172.21.15.29:0/1408748842 shutdown_connections 2024-01-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.653+0000 7feb967dc700 1 -- 172.21.15.29:0/1408748842 wait complete. 2024-01-24T15:32:43.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.655+0000 7feb967dc700 1 Processor -- start 2024-01-24T15:32:43.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.655+0000 7feb967dc700 1 -- start start 2024-01-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb967dc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb9014dfa0 0x7feb900c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:43.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb967dc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb9014e380 0x7feb900bde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb967dc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb90151850 0x7feb900a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb967dc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb900c5530 con 0x7feb9014e380 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb967dc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9013f750 con 0x7feb90151850 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb967dc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb900c4920 con 0x7feb9014dfa0 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb8f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb9014e380 0x7feb900bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb8f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb9014e380 0x7feb900bde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52358/0 (socket says 172.21.15.29:52358) 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb8f7fe700 1 -- 172.21.15.29:0/2400152331 learned_addr learned my addr 172.21.15.29:0/2400152331 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb8ffff700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb9014dfa0 0x7feb900c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb94d79700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb90151850 0x7feb900a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb8f7fe700 1 -- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb9014dfa0 msgr2=0x7feb900c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.656+0000 7feb8f7fe700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb9014dfa0 0x7feb900c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.657+0000 7feb8f7fe700 1 -- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb90151850 msgr2=0x7feb900a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:43.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.657+0000 7feb8f7fe700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb90151850 0x7feb900a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.657+0000 7feb8f7fe700 1 -- 172.21.15.29:0/2400152331 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8800bcf0 con 0x7feb9014e380 2024-01-24T15:32:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.657+0000 7feb8f7fe700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb9014e380 0x7feb900bde50 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 rx=0x7feb7800d890 tx=0x7feb78008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.657+0000 7feb87fff700 1 -- 172.21.15.29:0/2400152331 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78011590 con 0x7feb9014e380 2024-01-24T15:32:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.657+0000 7feb87fff700 1 -- 172.21.15.29:0/2400152331 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb780116f0 con 0x7feb9014e380 2024-01-24T15:32:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.657+0000 7feb967dc700 1 -- 172.21.15.29:0/2400152331 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb900c4c00 con 0x7feb9014e380 2024-01-24T15:32:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.657+0000 7feb87fff700 1 -- 172.21.15.29:0/2400152331 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78011a70 con 0x7feb9014e380 2024-01-24T15:32:43.661 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.657+0000 7feb967dc700 1 -- 172.21.15.29:0/2400152331 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb900c51c0 con 0x7feb9014e380 2024-01-24T15:32:43.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.658+0000 7feb967dc700 1 -- 172.21.15.29:0/2400152331 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb900c56e0 con 0x7feb9014e380 2024-01-24T15:32:43.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.658+0000 7feb87fff700 1 -- 172.21.15.29:0/2400152331 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7feb7801c460 con 0x7feb9014e380 2024-01-24T15:32:43.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.659+0000 7feb87fff700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7feb7005f270 0x7feb70061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:43.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.659+0000 7feb87fff700 1 -- 172.21.15.29:0/2400152331 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7335+0+0 (secure 0 0 0) 0x7feb7807a6a0 con 0x7feb9014e380 2024-01-24T15:32:43.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.659+0000 7feb8ffff700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7feb7005f270 0x7feb70061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:43.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.660+0000 7feb8ffff700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7feb7005f270 0x7feb70061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7feb800036e0 tx=0x7feb80009040).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:43.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.663+0000 7feb87fff700 1 -- 172.21.15.29:0/2400152331 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7feb900c56e0 con 0x7feb9014e380 2024-01-24T15:32:43.845 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:43.843+0000 7feb967dc700 1 -- 172.21.15.29:0/2400152331 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feb900c56e0 con 0x7feb9014e380 2024-01-24T15:32:44.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.492+0000 7feb87fff700 1 -- 172.21.15.29:0/2400152331 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v237) v1 ==== 169+0+0 (secure 0 0 0) 0x7feb900c56e0 con 0x7feb9014e380 2024-01-24T15:32:44.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.498+0000 7feb967dc700 1 -- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7feb7005f270 msgr2=0x7feb70061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:44.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.498+0000 7feb967dc700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7feb7005f270 0x7feb70061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7feb800036e0 tx=0x7feb80009040).stop 2024-01-24T15:32:44.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.498+0000 7feb967dc700 1 -- 172.21.15.29:0/2400152331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb9014e380 msgr2=0x7feb900bde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:44.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.498+0000 7feb967dc700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb9014e380 0x7feb900bde50 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 rx=0x7feb7800d890 tx=0x7feb78008040).stop 2024-01-24T15:32:44.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.499+0000 7feb967dc700 1 -- 172.21.15.29:0/2400152331 shutdown_connections 2024-01-24T15:32:44.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.499+0000 7feb967dc700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb90151850 0x7feb900a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.499+0000 7feb967dc700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7feb7005f270 0x7feb70061730 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0x7feb800036e0 tx=0x7feb80009040).stop 2024-01-24T15:32:44.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.499+0000 7feb967dc700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb9014dfa0 0x7feb900c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.499+0000 7feb967dc700 1 --2- 172.21.15.29:0/2400152331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb9014e380 0x7feb900bde50 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.500+0000 7feb967dc700 1 -- 172.21.15.29:0/2400152331 >> 172.21.15.29:0/2400152331 conn(0x7feb900627b0 msgr2=0x7feb901529c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:44.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.501+0000 7feb967dc700 1 -- 172.21.15.29:0/2400152331 shutdown_connections 2024-01-24T15:32:44.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.501+0000 7feb967dc700 1 -- 172.21.15.29:0/2400152331 wait complete. 2024-01-24T15:32:44.504 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'cephfs_data' removed 2024-01-24T15:32:44.520 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-24T15:32:44.655 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.653+0000 7f9fec8de700 1 Processor -- start 2024-01-24T15:32:44.656 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.653+0000 7f9fec8de700 1 -- start start 2024-01-24T15:32:44.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.654+0000 7f9fec8de700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fe814b140 0x7f9fe8155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:44.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.654+0000 7f9fec8de700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9fe80bca30 0x7f9fe80bce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:44.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.654+0000 7f9fec8de700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9fe80627b0 0x7f9fe8062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:44.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.654+0000 7f9fec8de700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe8154cd0 con 0x7f9fe80bca30 2024-01-24T15:32:44.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.654+0000 7f9fec8de700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe8063150 con 0x7f9fe80627b0 2024-01-24T15:32:44.658 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.654+0000 7f9fec8de700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81546a0 con 0x7f9fe814b140 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.654+0000 7f9fddd9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9fe80bca30 0x7f9fe80bce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.654+0000 7f9fddd9b700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9fe80bca30 0x7f9fe80bce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52364/0 (socket says 172.21.15.29:52364) 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.654+0000 7f9fddd9b700 1 -- 172.21.15.29:0/3984984120 learned_addr learned my addr 172.21.15.29:0/3984984120 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.654+0000 7f9fe659c700 1 --2- 172.21.15.29:0/3984984120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fe814b140 0x7f9fe8155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.654+0000 7f9fe6d9d700 1 --2- 172.21.15.29:0/3984984120 >> v2:172.21.15.163:3300/0 conn(0x7f9fe80627b0 0x7f9fe8062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fddd9b700 1 -- 172.21.15.29:0/3984984120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fe814b140 msgr2=0x7f9fe8155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fddd9b700 1 --2- 172.21.15.29:0/3984984120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fe814b140 0x7f9fe8155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fddd9b700 1 -- 172.21.15.29:0/3984984120 >> v2:172.21.15.163:3300/0 conn(0x7f9fe80627b0 msgr2=0x7f9fe8062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fddd9b700 1 --2- 172.21.15.29:0/3984984120 >> v2:172.21.15.163:3300/0 conn(0x7f9fe80627b0 0x7f9fe8062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fddd9b700 1 -- 172.21.15.29:0/3984984120 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe80c81e0 con 0x7f9fe80bca30 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fddd9b700 1 --2- 172.21.15.29:0/3984984120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fe80bca30 0x7f9fe80bce90 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 rx=0x7f9fe806b5c0 tx=0x7f9fd00090f0).ready entity=mon.0 client_cookie=7669ecd96ed38514 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fe5d9b700 1 -- 172.21.15.29:0/3984984120 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd000b600 con 0x7f9fe80bca30 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fe5d9b700 1 -- 172.21.15.29:0/3984984120 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd000b760 con 0x7f9fe80bca30 2024-01-24T15:32:44.659 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fe5d9b700 1 -- 172.21.15.29:0/3984984120 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd000bae0 con 0x7f9fe80bca30 2024-01-24T15:32:44.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fec8de700 1 -- 172.21.15.29:0/3984984120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fe80bca30 msgr2=0x7f9fe80bce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:44.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fec8de700 1 --2- 172.21.15.29:0/3984984120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fe80bca30 0x7f9fe80bce90 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 rx=0x7f9fe806b5c0 tx=0x7f9fd00090f0).stop 2024-01-24T15:32:44.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fec8de700 1 -- 172.21.15.29:0/3984984120 shutdown_connections 2024-01-24T15:32:44.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fec8de700 1 --2- 172.21.15.29:0/3984984120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fe814b140 0x7f9fe8155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.655+0000 7f9fec8de700 1 --2- 172.21.15.29:0/3984984120 >> v2:172.21.15.163:3300/0 conn(0x7f9fe80627b0 0x7f9fe8062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.656+0000 7f9fec8de700 1 --2- 172.21.15.29:0/3984984120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fe80bca30 0x7f9fe80bce90 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.656+0000 7f9fec8de700 1 -- 172.21.15.29:0/3984984120 >> 172.21.15.29:0/3984984120 conn(0x7f9fe8064390 msgr2=0x7f9fe80c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:44.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.656+0000 7f9fec8de700 1 -- 172.21.15.29:0/3984984120 shutdown_connections 2024-01-24T15:32:44.660 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.656+0000 7f9fec8de700 1 -- 172.21.15.29:0/3984984120 wait complete. 2024-01-24T15:32:44.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.657+0000 7f9fec8de700 1 Processor -- start 2024-01-24T15:32:44.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.657+0000 7f9fec8de700 1 -- start start 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fec8de700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fe80627b0 0x7f9fe81581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fec8de700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fe80bca30 0x7f9fe80a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fec8de700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fe814b140 0x7f9fe8148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fec8de700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80a9250 con 0x7f9fe80627b0 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fec8de700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81575b0 con 0x7f9fe80bca30 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fec8de700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80a8c00 con 0x7f9fe814b140 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fe659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fe80627b0 0x7f9fe81581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fe659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fe80627b0 0x7f9fe81581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52380/0 (socket says 172.21.15.29:52380) 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fe659c700 1 -- 172.21.15.29:0/1434250622 learned_addr learned my addr 172.21.15.29:0/1434250622 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fe6d9d700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fe814b140 0x7f9fe8148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fddd9b700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fe80bca30 0x7f9fe80a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fe659c700 1 -- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fe814b140 msgr2=0x7f9fe8148a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:44.664 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fe659c700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fe814b140 0x7f9fe8148a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fe659c700 1 -- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fe80bca30 msgr2=0x7f9fe80a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:44.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fe659c700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fe80bca30 0x7f9fe80a96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fe659c700 1 -- 172.21.15.29:0/1434250622 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd0009370 con 0x7f9fe80627b0 2024-01-24T15:32:44.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fddd9b700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fe80bca30 0x7f9fe80a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:44.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.658+0000 7f9fe659c700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fe80627b0 0x7f9fe81581d0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 rx=0x7f9fd800d890 tx=0x7f9fd8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:44.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.659+0000 7f9fdeffd700 1 -- 172.21.15.29:0/1434250622 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd80125f0 con 0x7f9fe80627b0 2024-01-24T15:32:44.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.659+0000 7f9fdeffd700 1 -- 172.21.15.29:0/1434250622 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd8012750 con 0x7f9fe80627b0 2024-01-24T15:32:44.665 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.659+0000 7f9fec8de700 1 -- 172.21.15.29:0/1434250622 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe80a8ee0 con 0x7f9fe80627b0 2024-01-24T15:32:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.659+0000 7f9fdeffd700 1 -- 172.21.15.29:0/1434250622 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8012ad0 con 0x7f9fe80627b0 2024-01-24T15:32:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.659+0000 7f9fec8de700 1 -- 172.21.15.29:0/1434250622 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe80a9bd0 con 0x7f9fe80627b0 2024-01-24T15:32:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.660+0000 7f9fec8de700 1 -- 172.21.15.29:0/1434250622 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe80475e0 con 0x7f9fe80627b0 2024-01-24T15:32:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.660+0000 7f9fdeffd700 1 -- 172.21.15.29:0/1434250622 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9fd8013480 con 0x7f9fe80627b0 2024-01-24T15:32:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.661+0000 7f9fdeffd700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9fc805f240 0x7f9fc8061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.661+0000 7f9fdeffd700 1 -- 172.21.15.29:0/1434250622 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f9fd807a440 con 0x7f9fe80627b0 2024-01-24T15:32:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.661+0000 7f9fddd9b700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9fc805f240 0x7f9fc8061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:44.666 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.662+0000 7f9fddd9b700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9fc805f240 0x7f9fc8061700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f9fe806b5c0 tx=0x7f9fd0017000).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:44.667 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.665+0000 7f9fdeffd700 1 -- 172.21.15.29:0/1434250622 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9fd804b210 con 0x7f9fe80627b0 2024-01-24T15:32:44.852 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.850+0000 7f9fec8de700 1 -- 172.21.15.29:0/1434250622 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9fe806a440 con 0x7f9fe80627b0 2024-01-24T15:32:44.856 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.854+0000 7f9fdeffd700 1 -- 172.21.15.29:0/1434250622 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9fd804e830 con 0x7f9fe80627b0 2024-01-24T15:32:44.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.856+0000 7f9fec8de700 1 -- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9fc805f240 msgr2=0x7f9fc8061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:44.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.856+0000 7f9fec8de700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9fc805f240 0x7f9fc8061700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f9fe806b5c0 tx=0x7f9fd0017000).stop 2024-01-24T15:32:44.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.856+0000 7f9fec8de700 1 -- 172.21.15.29:0/1434250622 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fe80627b0 msgr2=0x7f9fe81581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:44.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.856+0000 7f9fec8de700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fe80627b0 0x7f9fe81581d0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 rx=0x7f9fd800d890 tx=0x7f9fd8008040).stop 2024-01-24T15:32:44.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.856+0000 7f9fec8de700 1 -- 172.21.15.29:0/1434250622 shutdown_connections 2024-01-24T15:32:44.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.856+0000 7f9fec8de700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9fc805f240 0x7f9fc8061700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.856+0000 7f9fec8de700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fe814b140 0x7f9fe8148a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.856+0000 7f9fec8de700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9fe80627b0 0x7f9fe81581d0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.856+0000 7f9fec8de700 1 --2- 172.21.15.29:0/1434250622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fe80bca30 0x7f9fe80a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:44.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.856+0000 7f9fec8de700 1 -- 172.21.15.29:0/1434250622 >> 172.21.15.29:0/1434250622 conn(0x7f9fe8064390 msgr2=0x7f9fe80c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:44.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.856+0000 7f9fec8de700 1 -- 172.21.15.29:0/1434250622 shutdown_connections 2024-01-24T15:32:44.860 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:44.856+0000 7f9fec8de700 1 -- 172.21.15.29:0/1434250622 wait complete. 2024-01-24T15:32:44.872 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-01-24T15:32:45.005 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.003+0000 7f7863300700 1 Processor -- start 2024-01-24T15:32:45.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.003+0000 7f7863300700 1 -- start start 2024-01-24T15:32:45.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.004+0000 7f7863300700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785c066c30 0x7f785c1633a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.004+0000 7f7863300700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f785c0bfa90 con 0x7f785c13b660 2024-01-24T15:32:45.009 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.004+0000 7f7863300700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f785c067aa0 con 0x7f785c067180 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.004+0000 7f7863300700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785c068900 con 0x7f785c066c30 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.004+0000 7f786109c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f785c13b660 0x7f785c063a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53934/0 (socket says 172.21.15.29:53934) 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f786109c700 1 -- 172.21.15.29:0/2860711595 learned_addr learned my addr 172.21.15.29:0/2860711595 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f786189d700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785c066c30 0x7f785c1633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924471586 0 0) 0x7f785c0bfa90 con 0x7f785c13b660 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78480033f0 con 0x7f785c13b660 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109712102 0 0) 0x7f785c067aa0 con 0x7f785c067180 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f785c0bfa90 con 0x7f785c067180 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2649232479 0 0) 0x7f78480033f0 con 0x7f785c13b660 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785c066c30 msgr2=0x7f785c1633a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785c066c30 0x7f785c1633a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 >> v1:172.21.15.163:6789/0 conn(0x7f785c067180 legacy=0x7f785c066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c13ba40 con 0x7f785c13b660 2024-01-24T15:32:45.010 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f784c001470 con 0x7f785c13b660 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f786189d700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785c066c30 0x7f785c1633a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 >> v1:172.21.15.29:6789/0 conn(0x7f785c13b660 legacy=0x7f785c063a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785c066c30 0x7f7848003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7848003f90 0x7f7848004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7848004b80 0x7f7848005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c068900 con 0x7f7848004b80 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784800b940 con 0x7f7848003f90 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784800bb00 con 0x7f785c066c30 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f786189d700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7848004b80 0x7f7848005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f786109c700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785c066c30 0x7f7848003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.005+0000 7f786089b700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7848003f90 0x7f7848004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.006+0000 7f786189d700 1 -- 172.21.15.29:0/2860711595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785c066c30 msgr2=0x7f7848003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:45.011 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.006+0000 7f786189d700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f785c066c30 0x7f7848003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.006+0000 7f786189d700 1 -- 172.21.15.29:0/2860711595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7848003f90 msgr2=0x7f7848004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.006+0000 7f786189d700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7848003f90 0x7f7848004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.006+0000 7f786189d700 1 -- 172.21.15.29:0/2860711595 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7854009a20 con 0x7f7848004b80 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.006+0000 7f786189d700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7848004b80 0x7f7848005000 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 rx=0x7f785c06a280 tx=0x7f785400b040).ready entity=mon.0 client_cookie=c43bc56b4f502bc server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.006+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854009c50 con 0x7f7848004b80 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.006+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785400f040 con 0x7f7848004b80 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.006+0000 7f7853fff700 1 -- 172.21.15.29:0/2860711595 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785400ee50 con 0x7f7848004b80 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.007+0000 7f7863300700 1 -- 172.21.15.29:0/2860711595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7848004b80 msgr2=0x7f7848005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.007+0000 7f7863300700 1 --2- 172.21.15.29:0/2860711595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7848004b80 0x7f7848005000 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 rx=0x7f785c06a280 tx=0x7f785400b040).stop 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.007+0000 7f786189d700 1 -- 172.21.15.29:0/2860711595 reap_dead start 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.007+0000 7f7863300700 1 -- 172.21.15.29:0/2860711595 shutdown_connections 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.007+0000 7f7863300700 1 -- 172.21.15.29:0/2860711595 >> 172.21.15.29:0/2860711595 conn(0x7f785c0c8ba0 msgr2=0x7f785c0c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.007+0000 7f7863300700 1 -- 172.21.15.29:0/2860711595 shutdown_connections 2024-01-24T15:32:45.012 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.007+0000 7f7863300700 1 -- 172.21.15.29:0/2860711595 wait complete. 2024-01-24T15:32:45.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.008+0000 7f7863300700 1 Processor -- start 2024-01-24T15:32:45.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.008+0000 7f7863300700 1 -- start start 2024-01-24T15:32:45.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.009+0000 7f7863300700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7848004b80 0x7f785c13bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.009+0000 7f7863300700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f785c13c2d0 0x7f785c0c22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.009+0000 7f7863300700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f785c0c27b0 0x7f785c16bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.015 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.009+0000 7f7863300700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c0abbf0 con 0x7f785c13c2d0 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.009+0000 7f7863300700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c0abed0 con 0x7f785c0c27b0 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.009+0000 7f7863300700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785c0ac1b0 con 0x7f7848004b80 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.009+0000 7f786089b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f785c13c2d0 0x7f785c0c22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.009+0000 7f786089b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f785c13c2d0 0x7f785c0c22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52402/0 (socket says 172.21.15.29:52402) 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.009+0000 7f786089b700 1 -- 172.21.15.29:0/2390068925 learned_addr learned my addr 172.21.15.29:0/2390068925 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.009+0000 7f786109c700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7848004b80 0x7f785c13bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.009+0000 7f786189d700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f785c0c27b0 0x7f785c16bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.010+0000 7f786089b700 1 -- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7848004b80 msgr2=0x7f785c13bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.010+0000 7f786089b700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7848004b80 0x7f785c13bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.010+0000 7f786089b700 1 -- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f785c0c27b0 msgr2=0x7f785c16bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.010+0000 7f786089b700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f785c0c27b0 0x7f785c16bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.010+0000 7f786089b700 1 -- 172.21.15.29:0/2390068925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c0ac490 con 0x7f785c13c2d0 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.010+0000 7f786089b700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f785c13c2d0 0x7f785c0c22c0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 rx=0x7f7858002a20 tx=0x7f7858007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:45.016 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.010+0000 7f78517fa700 1 -- 172.21.15.29:0/2390068925 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800f8d0 con 0x7f785c13c2d0 2024-01-24T15:32:45.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.010+0000 7f78517fa700 1 -- 172.21.15.29:0/2390068925 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785800fa30 con 0x7f785c13c2d0 2024-01-24T15:32:45.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.010+0000 7f78517fa700 1 -- 172.21.15.29:0/2390068925 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800fdd0 con 0x7f785c13c2d0 2024-01-24T15:32:45.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.010+0000 7f7863300700 1 -- 172.21.15.29:0/2390068925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c0ac770 con 0x7f785c13c2d0 2024-01-24T15:32:45.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.010+0000 7f7863300700 1 -- 172.21.15.29:0/2390068925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c0c2df0 con 0x7f785c13c2d0 2024-01-24T15:32:45.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.011+0000 7f7863300700 1 -- 172.21.15.29:0/2390068925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785c0475e0 con 0x7f785c13c2d0 2024-01-24T15:32:45.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.011+0000 7f78517fa700 1 -- 172.21.15.29:0/2390068925 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f785801d3b0 con 0x7f785c13c2d0 2024-01-24T15:32:45.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.012+0000 7f78517fa700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f784005f270 0x7f7840061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.012+0000 7f78517fa700 1 -- 172.21.15.29:0/2390068925 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f785807a710 con 0x7f785c13c2d0 2024-01-24T15:32:45.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.012+0000 7f786109c700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f784005f270 0x7f7840061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.017 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.013+0000 7f786109c700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f784005f270 0x7f7840061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f784c000e10 tx=0x7f784c009040).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:45.018 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.016+0000 7f78517fa700 1 -- 172.21.15.29:0/2390068925 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f785804b4e0 con 0x7f785c13c2d0 2024-01-24T15:32:45.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.204+0000 7f7863300700 1 -- 172.21.15.29:0/2390068925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f785c067cf0 con 0x7f785c13c2d0 2024-01-24T15:32:45.210 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.208+0000 7f78517fa700 1 -- 172.21.15.29:0/2390068925 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v45) v1 ==== 120+0+0 (secure 0 0 0) 0x7f785804eb00 con 0x7f785c13c2d0 2024-01-24T15:32:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.210+0000 7f7863300700 1 -- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f784005f270 msgr2=0x7f7840061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.210+0000 7f7863300700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f784005f270 0x7f7840061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f784c000e10 tx=0x7f784c009040).stop 2024-01-24T15:32:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.210+0000 7f7863300700 1 -- 172.21.15.29:0/2390068925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f785c13c2d0 msgr2=0x7f785c0c22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.210+0000 7f7863300700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f785c13c2d0 0x7f785c0c22c0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 rx=0x7f7858002a20 tx=0x7f7858007bb0).stop 2024-01-24T15:32:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.210+0000 7f7863300700 1 -- 172.21.15.29:0/2390068925 shutdown_connections 2024-01-24T15:32:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.210+0000 7f7863300700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f785c0c27b0 0x7f785c16bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.210+0000 7f7863300700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f784005f270 0x7f7840061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.210+0000 7f7863300700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7848004b80 0x7f785c13bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.210+0000 7f7863300700 1 --2- 172.21.15.29:0/2390068925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f785c13c2d0 0x7f785c0c22c0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.210+0000 7f7863300700 1 -- 172.21.15.29:0/2390068925 >> 172.21.15.29:0/2390068925 conn(0x7f785c0c8ba0 msgr2=0x7f785c150330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.210+0000 7f7863300700 1 -- 172.21.15.29:0/2390068925 shutdown_connections 2024-01-24T15:32:45.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.210+0000 7f7863300700 1 -- 172.21.15.29:0/2390068925 wait complete. 2024-01-24T15:32:45.226 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2024-01-24T15:32:45.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.356+0000 7fd0e5faa700 1 Processor -- start 2024-01-24T15:32:45.358 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.356+0000 7fd0e5faa700 1 -- start start 2024-01-24T15:32:45.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.357+0000 7fd0e5faa700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd0e0134cc0 0x7fd0e00c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.357+0000 7fd0e5faa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0e01350a0 0x7fd0e0152e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.357+0000 7fd0e5faa700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e0068360 con 0x7fd0e00664f0 2024-01-24T15:32:45.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.357+0000 7fd0e5faa700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e0153360 con 0x7fd0e0134cc0 2024-01-24T15:32:45.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.357+0000 7fd0e5faa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e00c3820 con 0x7fd0e01350a0 2024-01-24T15:32:45.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.357+0000 7fd0df7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd0e0134cc0 0x7fd0e00c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.357+0000 7fd0df7fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd0e0134cc0 0x7fd0e00c31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52416/0 (socket says 172.21.15.29:52416) 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.357+0000 7fd0df7fe700 1 -- 172.21.15.29:0/1426052049 learned_addr learned my addr 172.21.15.29:0/1426052049 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.357+0000 7fd0deffd700 1 --2- 172.21.15.29:0/1426052049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0e01350a0 0x7fd0e0152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.357+0000 7fd0df7fe700 1 -- 172.21.15.29:0/1426052049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0e01350a0 msgr2=0x7fd0e0152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.357+0000 7fd0df7fe700 1 --2- 172.21.15.29:0/1426052049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0e01350a0 0x7fd0e0152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0df7fe700 1 -- 172.21.15.29:0/1426052049 >> v1:172.21.15.163:6789/0 conn(0x7fd0e00664f0 legacy=0x7fd0e0155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0df7fe700 1 -- 172.21.15.29:0/1426052049 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e0067a30 con 0x7fd0e0134cc0 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0df7fe700 1 --2- 172.21.15.29:0/1426052049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0e0134cc0 0x7fd0e00c31b0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 rx=0x7fd0d0006fd0 tx=0x7fd0d00090f0).ready entity=mon.0 client_cookie=9059c3d0a928ab61 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0de7fc700 1 -- 172.21.15.29:0/1426052049 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000b600 con 0x7fd0e0134cc0 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0de7fc700 1 -- 172.21.15.29:0/1426052049 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d000b760 con 0x7fd0e0134cc0 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0de7fc700 1 -- 172.21.15.29:0/1426052049 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000bae0 con 0x7fd0e0134cc0 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0e5faa700 1 -- 172.21.15.29:0/1426052049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0e0134cc0 msgr2=0x7fd0e00c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0e5faa700 1 --2- 172.21.15.29:0/1426052049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0e0134cc0 0x7fd0e00c31b0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 rx=0x7fd0d0006fd0 tx=0x7fd0d00090f0).stop 2024-01-24T15:32:45.362 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0e5faa700 1 -- 172.21.15.29:0/1426052049 shutdown_connections 2024-01-24T15:32:45.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0e5faa700 1 --2- 172.21.15.29:0/1426052049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0e0134cc0 0x7fd0e00c31b0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0e5faa700 1 --2- 172.21.15.29:0/1426052049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0e01350a0 0x7fd0e0152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0e5faa700 1 -- 172.21.15.29:0/1426052049 >> 172.21.15.29:0/1426052049 conn(0x7fd0e00c8bb0 msgr2=0x7fd0e00c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:45.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.358+0000 7fd0e5faa700 1 -- 172.21.15.29:0/1426052049 shutdown_connections 2024-01-24T15:32:45.363 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.359+0000 7fd0e5faa700 1 -- 172.21.15.29:0/1426052049 wait complete. 2024-01-24T15:32:45.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.360+0000 7fd0e5faa700 1 Processor -- start 2024-01-24T15:32:45.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.360+0000 7fd0e5faa700 1 -- start start 2024-01-24T15:32:45.365 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0e5faa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0e00664f0 0x7fd0e00ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0e5faa700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0e0134cc0 0x7fd0e01343a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0e5faa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0e01350a0 0x7fd0e00bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0e5faa700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e01359d0 con 0x7fd0e01350a0 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0e5faa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e0135cb0 con 0x7fd0e00664f0 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0e5faa700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e0135f90 con 0x7fd0e0134cc0 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0dffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0e01350a0 0x7fd0e00bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0dffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0e01350a0 0x7fd0e00bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52426/0 (socket says 172.21.15.29:52426) 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0dffff700 1 -- 172.21.15.29:0/3569751452 learned_addr learned my addr 172.21.15.29:0/3569751452 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0df7fe700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0e00664f0 0x7fd0e00ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0deffd700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0e0134cc0 0x7fd0e01343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0dffff700 1 -- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0e0134cc0 msgr2=0x7fd0e01343a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0dffff700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0e0134cc0 0x7fd0e01343a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0dffff700 1 -- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0e00664f0 msgr2=0x7fd0e00ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:45.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0dffff700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0e00664f0 0x7fd0e00ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0dffff700 1 -- 172.21.15.29:0/3569751452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d0009370 con 0x7fd0e01350a0 2024-01-24T15:32:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0deffd700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0e0134cc0 0x7fd0e01343a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0df7fe700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0e00664f0 0x7fd0e00ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.361+0000 7fd0dffff700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0e01350a0 0x7fd0e00bfec0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 rx=0x7fd0d8002230 tx=0x7fd0d80015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.362+0000 7fd0d77fe700 1 -- 172.21.15.29:0/3569751452 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8010510 con 0x7fd0e01350a0 2024-01-24T15:32:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.362+0000 7fd0d77fe700 1 -- 172.21.15.29:0/3569751452 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d8010670 con 0x7fd0e01350a0 2024-01-24T15:32:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.362+0000 7fd0e5faa700 1 -- 172.21.15.29:0/3569751452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e0136270 con 0x7fd0e01350a0 2024-01-24T15:32:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.362+0000 7fd0d77fe700 1 -- 172.21.15.29:0/3569751452 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80109f0 con 0x7fd0e01350a0 2024-01-24T15:32:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.362+0000 7fd0e5faa700 1 -- 172.21.15.29:0/3569751452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e0135540 con 0x7fd0e01350a0 2024-01-24T15:32:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.363+0000 7fd0e5faa700 1 -- 172.21.15.29:0/3569751452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e0136790 con 0x7fd0e01350a0 2024-01-24T15:32:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.363+0000 7fd0d77fe700 1 -- 172.21.15.29:0/3569751452 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd0d801b400 con 0x7fd0e01350a0 2024-01-24T15:32:45.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.364+0000 7fd0d77fe700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd0c005f270 0x7fd0c0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:45.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.364+0000 7fd0d77fe700 1 -- 172.21.15.29:0/3569751452 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fd0d807a4e0 con 0x7fd0e01350a0 2024-01-24T15:32:45.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.364+0000 7fd0df7fe700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd0c005f270 0x7fd0c0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:45.368 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.364+0000 7fd0df7fe700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd0c005f270 0x7fd0c0061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fd0d0009130 tx=0x7fd0d0017000).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:45.370 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.368+0000 7fd0d77fe700 1 -- 172.21.15.29:0/3569751452 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd0e0136790 con 0x7fd0e01350a0 2024-01-24T15:32:45.621 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:45.619+0000 7fd0e5faa700 1 -- 172.21.15.29:0/3569751452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7fd0e0136790 con 0x7fd0e01350a0 2024-01-24T15:32:46.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.498+0000 7fd0d77fe700 1 -- 172.21.15.29:0/3569751452 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3061) v1 ==== 155+0+0 (secure 0 0 0) 0x7fd0e0136790 con 0x7fd0e01350a0 2024-01-24T15:32:46.503 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.501+0000 7fd0e5faa700 1 -- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd0c005f270 msgr2=0x7fd0c0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:46.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.501+0000 7fd0e5faa700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd0c005f270 0x7fd0c0061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fd0d0009130 tx=0x7fd0d0017000).stop 2024-01-24T15:32:46.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.501+0000 7fd0e5faa700 1 -- 172.21.15.29:0/3569751452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0e01350a0 msgr2=0x7fd0e00bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:46.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.501+0000 7fd0e5faa700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0e01350a0 0x7fd0e00bfec0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 rx=0x7fd0d8002230 tx=0x7fd0d80015e0).stop 2024-01-24T15:32:46.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.501+0000 7fd0e5faa700 1 -- 172.21.15.29:0/3569751452 shutdown_connections 2024-01-24T15:32:46.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.501+0000 7fd0e5faa700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0e01350a0 0x7fd0e00bfec0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:46.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.501+0000 7fd0e5faa700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0e00664f0 0x7fd0e00ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:46.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.501+0000 7fd0e5faa700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd0c005f270 0x7fd0c0061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:46.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.501+0000 7fd0e5faa700 1 --2- 172.21.15.29:0/3569751452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0e0134cc0 0x7fd0e01343a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:46.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.501+0000 7fd0e5faa700 1 -- 172.21.15.29:0/3569751452 >> 172.21.15.29:0/3569751452 conn(0x7fd0e00c8bb0 msgr2=0x7fd0e00c6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:46.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.501+0000 7fd0e5faa700 1 -- 172.21.15.29:0/3569751452 shutdown_connections 2024-01-24T15:32:46.504 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:46.501+0000 7fd0e5faa700 1 -- 172.21.15.29:0/3569751452 wait complete. 2024-01-24T15:32:46.519 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2024-01-24T15:32:46.519 INFO:tasks.cephfs_test_runner: 2024-01-24T15:32:46.519 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-24T15:32:46.519 INFO:tasks.cephfs_test_runner:Ran 11 tests in 5624.873s 2024-01-24T15:32:46.520 INFO:tasks.cephfs_test_runner: 2024-01-24T15:32:46.520 INFO:tasks.cephfs_test_runner:OK 2024-01-24T15:32:46.520 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-01-24T15:32:46.533 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-01-24T15:32:46.558 INFO:tasks.kclient:Unmounting kernel clients... 2024-01-24T15:32:46.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:46.559 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-24T15:32:46.581 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-24T15:32:46.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:46.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-24T15:32:46.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:46.670 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-24T15:32:46.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:46.739 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link delete ceph-brx 2024-01-24T15:32:46.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:46.830 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-01-24T15:32:46.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:46.852 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link delete ceph-brx 2024-01-24T15:32:46.933 INFO:teuthology.orchestra.run.smithi029.stderr:Cannot find device "ceph-brx" 2024-01-24T15:32:46.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T15:32:46.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:46.935 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-01-24T15:32:46.956 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-24T15:32:46.957 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-24T15:32:46.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:46.957 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-24T15:32:47.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:47.039 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-24T15:32:47.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:47.122 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-01-24T15:32:47.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:47.248 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-01-24T15:32:47.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-24T15:32:47.268 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-01-24T15:32:47.340 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2024-01-24T15:32:47.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T15:32:47.342 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-01-24T15:32:47.351 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-01-24T15:32:47.351 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:32:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.488+0000 7fe2a2cf4700 1 Processor -- start 2024-01-24T15:32:47.490 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.488+0000 7fe2a2cf4700 1 -- start start 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe2a2cf4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe29c14b300 0x7fe29c068dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe2a2cf4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe29c14b6e0 0x7fe29c0692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe2a2cf4700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29c06a1c0 con 0x7fe29c0627b0 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe2a2cf4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29c0c2a90 con 0x7fe29c14b6e0 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe2a2cf4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c069920 con 0x7fe29c14b300 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe2a1291700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe29c0627b0 0x7fe29c062be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53938/0 (socket says 172.21.15.29:53938) 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe2a1291700 1 -- 172.21.15.29:0/3841142074 learned_addr learned my addr 172.21.15.29:0/3841142074 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281398843 0 0) 0x7fe29c06a1c0 con 0x7fe29c0627b0 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe28c0033f0 con 0x7fe29c0627b0 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe29bfff700 1 --2- 172.21.15.29:0/3841142074 >> v2:172.21.15.163:3300/0 conn(0x7fe29c14b6e0 0x7fe29c0692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe2a0a90700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe29c14b300 0x7fe29c068dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2965907599 0 0) 0x7fe28c0033f0 con 0x7fe29c0627b0 2024-01-24T15:32:47.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe29c14b300 msgr2=0x7fe29c068dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe29b7fe700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe29c14b300 0x7fe29c068dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 >> v2:172.21.15.163:3300/0 conn(0x7fe29c14b6e0 msgr2=0x7fe29c0692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe29b7fe700 1 --2- 172.21.15.29:0/3841142074 >> v2:172.21.15.163:3300/0 conn(0x7fe29c14b6e0 0x7fe29c0692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c069c00 con 0x7fe29c0627b0 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7fe2940017f0 con 0x7fe29c0627b0 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.489+0000 7fe2a0a90700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe29c14b300 0x7fe29c068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29bfff700 1 --2- 172.21.15.29:0/3841142074 >> v2:172.21.15.163:3300/0 conn(0x7fe29c14b6e0 0x7fe29c0692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 >> v1:172.21.15.29:6789/0 conn(0x7fe29c0627b0 legacy=0x7fe29c062be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29b7fe700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe29c14b300 0x7fe28c003960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29b7fe700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe28c004030 0x7fe28c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29b7fe700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c004d80 0x7fe28c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c0c2a90 con 0x7fe28c004030 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c069920 con 0x7fe28c004d80 2024-01-24T15:32:47.495 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c005560 con 0x7fe29c14b300 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29bfff700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe28c004030 0x7fe28c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe2a0a90700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe29c14b300 0x7fe28c003960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe2a1291700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c004d80 0x7fe28c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29bfff700 1 -- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe29c14b300 msgr2=0x7fe28c003960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29bfff700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe29c14b300 0x7fe28c003960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29bfff700 1 -- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c004d80 msgr2=0x7fe28c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29bfff700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c004d80 0x7fe28c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29bfff700 1 -- 172.21.15.29:0/3841142074 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0033f0 con 0x7fe28c004030 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe2a1291700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe28c004d80 0x7fe28c00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29bfff700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe28c004030 0x7fe28c004690 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 rx=0x7fe29c0c44d0 tx=0x7fe288008040).ready entity=mon.0 client_cookie=abf0a869d9c1aa33 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800c420 con 0x7fe28c004030 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28800c580 con 0x7fe28c004030 2024-01-24T15:32:47.496 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.490+0000 7fe29b7fe700 1 -- 172.21.15.29:0/3841142074 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800c900 con 0x7fe28c004030 2024-01-24T15:32:47.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.491+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/3841142074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe28c004030 msgr2=0x7fe28c004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:47.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.491+0000 7fe2a2cf4700 1 --2- 172.21.15.29:0/3841142074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe28c004030 0x7fe28c004690 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 rx=0x7fe29c0c44d0 tx=0x7fe288008040).stop 2024-01-24T15:32:47.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.491+0000 7fe2a1291700 1 -- 172.21.15.29:0/3841142074 reap_dead start 2024-01-24T15:32:47.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.491+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/3841142074 shutdown_connections 2024-01-24T15:32:47.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.491+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/3841142074 >> 172.21.15.29:0/3841142074 conn(0x7fe29c0c69f0 msgr2=0x7fe29c0c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:47.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.491+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/3841142074 shutdown_connections 2024-01-24T15:32:47.497 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.491+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/3841142074 wait complete. 2024-01-24T15:32:47.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.492+0000 7fe2a2cf4700 1 Processor -- start 2024-01-24T15:32:47.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.493+0000 7fe2a2cf4700 1 -- start start 2024-01-24T15:32:47.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.493+0000 7fe2a2cf4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe28c004030 0x7fe29c0c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.493+0000 7fe2a2cf4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe29c1435d0 0x7fe29c143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.493+0000 7fe2a2cf4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29c147b00 0x7fe29c147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.493+0000 7fe2a2cf4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c0c09a0 con 0x7fe29c1435d0 2024-01-24T15:32:47.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.493+0000 7fe2a2cf4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c0be4f0 con 0x7fe29c147b00 2024-01-24T15:32:47.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.493+0000 7fe2a2cf4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c148470 con 0x7fe28c004030 2024-01-24T15:32:47.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.493+0000 7fe29bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe29c1435d0 0x7fe29c143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe29bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe29c1435d0 0x7fe29c143b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52442/0 (socket says 172.21.15.29:52442) 2024-01-24T15:32:47.500 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe29bfff700 1 -- 172.21.15.29:0/2335578163 learned_addr learned my addr 172.21.15.29:0/2335578163 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe2a0a90700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe28c004030 0x7fe29c0c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe2a1291700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29c147b00 0x7fe29c147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe29bfff700 1 -- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe28c004030 msgr2=0x7fe29c0c04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe29bfff700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe28c004030 0x7fe29c0c04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe29bfff700 1 -- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29c147b00 msgr2=0x7fe29c147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe29bfff700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29c147b00 0x7fe29c147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe29bfff700 1 -- 172.21.15.29:0/2335578163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c144460 con 0x7fe29c1435d0 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe2a1291700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29c147b00 0x7fe29c147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe2a0a90700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe28c004030 0x7fe29c0c04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe29bfff700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe29c1435d0 0x7fe29c143b80 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 rx=0x7fe28800cd70 tx=0x7fe288008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe298ff9700 1 -- 172.21.15.29:0/2335578163 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288011ab0 con 0x7fe29c1435d0 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe298ff9700 1 -- 172.21.15.29:0/2335578163 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe288011c10 con 0x7fe29c1435d0 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/2335578163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c144740 con 0x7fe29c1435d0 2024-01-24T15:32:47.501 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe298ff9700 1 -- 172.21.15.29:0/2335578163 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288011da0 con 0x7fe29c1435d0 2024-01-24T15:32:47.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.494+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/2335578163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c0be180 con 0x7fe29c1435d0 2024-01-24T15:32:47.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.495+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/2335578163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c0bdfe0 con 0x7fe29c1435d0 2024-01-24T15:32:47.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.496+0000 7fe298ff9700 1 -- 172.21.15.29:0/2335578163 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe288022020 con 0x7fe29c1435d0 2024-01-24T15:32:47.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.496+0000 7fe298ff9700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe28005f270 0x7fe280061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.496+0000 7fe298ff9700 1 -- 172.21.15.29:0/2335578163 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fe28807bea0 con 0x7fe29c1435d0 2024-01-24T15:32:47.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.496+0000 7fe2a0a90700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe28005f270 0x7fe280061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.497+0000 7fe2a0a90700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe28005f270 0x7fe280061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe2900069a0 tx=0x7fe290008040).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:47.502 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.501+0000 7fe298ff9700 1 -- 172.21.15.29:0/2335578163 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fe29c0bdfe0 con 0x7fe29c1435d0 2024-01-24T15:32:47.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.683+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/2335578163 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe29c0bdfe0 con 0x7fe29c1435d0 2024-01-24T15:32:47.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.684+0000 7fe298ff9700 1 -- 172.21.15.29:0/2335578163 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v237) v1 ==== 74+0+15898 (secure 0 0 0) 0x7fe29c0bdfe0 con 0x7fe29c1435d0 2024-01-24T15:32:47.686 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:47.687 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":237,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:32:44.489705+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/2645482892":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/4093084652":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/3512593770":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/343370470":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/4238824203":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/3387006926":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/3908030498":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/741220992":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/2316406386":"2024-01-25T15:32:33.173546+0000","172.21.15.163:6833/57365":"2024-01-25T15:32:33.173546+0000","172.21.15.163:6837/532321390":"2024-01-25T15:32:40.962950+0000","172.21.15.163:6835/532321390":"2024-01-25T15:32:40.962950+0000","172.21.15.163:6832/57365":"2024-01-25T15:32:33.173546+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:32:47.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.686+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe28005f270 msgr2=0x7fe280061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:47.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.686+0000 7fe2a2cf4700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe28005f270 0x7fe280061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe2900069a0 tx=0x7fe290008040).stop 2024-01-24T15:32:47.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.686+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/2335578163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe29c1435d0 msgr2=0x7fe29c143b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:47.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.686+0000 7fe2a2cf4700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe29c1435d0 0x7fe29c143b80 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 rx=0x7fe28800cd70 tx=0x7fe288008040).stop 2024-01-24T15:32:47.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.686+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/2335578163 shutdown_connections 2024-01-24T15:32:47.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.686+0000 7fe2a2cf4700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29c147b00 0x7fe29c147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.686+0000 7fe2a2cf4700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe28005f270 0x7fe280061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.686+0000 7fe2a2cf4700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe28c004030 0x7fe29c0c04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.686+0000 7fe2a2cf4700 1 --2- 172.21.15.29:0/2335578163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe29c1435d0 0x7fe29c143b80 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.686+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/2335578163 >> 172.21.15.29:0/2335578163 conn(0x7fe29c0c69f0 msgr2=0x7fe29c157bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:47.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.686+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/2335578163 shutdown_connections 2024-01-24T15:32:47.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.686+0000 7fe2a2cf4700 1 -- 172.21.15.29:0/2335578163 wait complete. 2024-01-24T15:32:47.702 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-24T15:32:47.702 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:32:47.832 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.830+0000 7f2556cf5700 1 Processor -- start 2024-01-24T15:32:47.833 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.830+0000 7f2556cf5700 1 -- start start 2024-01-24T15:32:47.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.831+0000 7f2556cf5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2550062970 0x7f2550062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.831+0000 7f2556cf5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2550066b40 0x7f2550158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.831+0000 7f2556cf5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25500bcf60 con 0x7f25500c4da0 2024-01-24T15:32:47.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.831+0000 7f2556cf5700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25500bdba0 con 0x7f2550066b40 2024-01-24T15:32:47.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.831+0000 7f2556cf5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25500a7480 con 0x7f2550062970 2024-01-24T15:32:47.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.832+0000 7f254ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2550066b40 0x7f2550158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.835 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.832+0000 7f254ffff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2550066b40 0x7f2550158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52450/0 (socket says 172.21.15.29:52450) 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.832+0000 7f254ffff700 1 -- 172.21.15.29:0/3467681383 learned_addr learned my addr 172.21.15.29:0/3467681383 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.832+0000 7f2554a91700 1 --2- 172.21.15.29:0/3467681383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2550062970 0x7f2550062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.832+0000 7f254ffff700 1 -- 172.21.15.29:0/3467681383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2550062970 msgr2=0x7f2550062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.832+0000 7f254ffff700 1 --2- 172.21.15.29:0/3467681383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2550062970 0x7f2550062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.832+0000 7f254ffff700 1 -- 172.21.15.29:0/3467681383 >> v1:172.21.15.163:6789/0 conn(0x7f25500c4da0 legacy=0x7f2550069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.832+0000 7f254ffff700 1 -- 172.21.15.29:0/3467681383 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2550158af0 con 0x7f2550066b40 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.832+0000 7f254ffff700 1 --2- 172.21.15.29:0/3467681383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2550066b40 0x7f2550158600 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 rx=0x7f253c006fd0 tx=0x7f253c0090f0).ready entity=mon.0 client_cookie=79b63c68ef98f671 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.832+0000 7f254f7fe700 1 -- 172.21.15.29:0/3467681383 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c00b580 con 0x7f2550066b40 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.832+0000 7f254f7fe700 1 -- 172.21.15.29:0/3467681383 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c00b6e0 con 0x7f2550066b40 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.832+0000 7f254f7fe700 1 -- 172.21.15.29:0/3467681383 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c00ba60 con 0x7f2550066b40 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.833+0000 7f2556cf5700 1 -- 172.21.15.29:0/3467681383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2550066b40 msgr2=0x7f2550158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.833+0000 7f2556cf5700 1 --2- 172.21.15.29:0/3467681383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2550066b40 0x7f2550158600 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 rx=0x7f253c006fd0 tx=0x7f253c0090f0).stop 2024-01-24T15:32:47.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.833+0000 7f2556cf5700 1 -- 172.21.15.29:0/3467681383 shutdown_connections 2024-01-24T15:32:47.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.833+0000 7f2556cf5700 1 --2- 172.21.15.29:0/3467681383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2550062970 0x7f2550062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.833+0000 7f2556cf5700 1 --2- 172.21.15.29:0/3467681383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2550066b40 0x7f2550158600 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.833+0000 7f2556cf5700 1 -- 172.21.15.29:0/3467681383 >> 172.21.15.29:0/3467681383 conn(0x7f25500c8580 msgr2=0x7f25500c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:47.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.833+0000 7f2556cf5700 1 -- 172.21.15.29:0/3467681383 shutdown_connections 2024-01-24T15:32:47.837 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.833+0000 7f2556cf5700 1 -- 172.21.15.29:0/3467681383 wait complete. 2024-01-24T15:32:47.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.834+0000 7f2556cf5700 1 Processor -- start 2024-01-24T15:32:47.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.834+0000 7f2556cf5700 1 -- start start 2024-01-24T15:32:47.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.835+0000 7f2556cf5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2550062970 0x7f25500a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.835+0000 7f2556cf5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2550066b40 0x7f25500a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.835+0000 7f2556cf5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25500c4da0 0x7f25500abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.835+0000 7f2556cf5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25500a75c0 con 0x7f2550062970 2024-01-24T15:32:47.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.835+0000 7f2556cf5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25500a7e60 con 0x7f2550066b40 2024-01-24T15:32:47.839 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.835+0000 7f2556cf5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25500acf20 con 0x7f25500c4da0 2024-01-24T15:32:47.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.835+0000 7f2554a91700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2550062970 0x7f25500a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.835+0000 7f2554a91700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2550062970 0x7f25500a99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52464/0 (socket says 172.21.15.29:52464) 2024-01-24T15:32:47.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.835+0000 7f2554a91700 1 -- 172.21.15.29:0/2968243669 learned_addr learned my addr 172.21.15.29:0/2968243669 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:47.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.835+0000 7f2555292700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25500c4da0 0x7f25500abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.835+0000 7f254ffff700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2550066b40 0x7f25500a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.836+0000 7f2554a91700 1 -- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25500c4da0 msgr2=0x7f25500abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:47.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.836+0000 7f2554a91700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25500c4da0 0x7f25500abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.836+0000 7f2554a91700 1 -- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2550066b40 msgr2=0x7f25500a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:47.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.836+0000 7f2554a91700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2550066b40 0x7f25500a9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:47.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.836+0000 7f2554a91700 1 -- 172.21.15.29:0/2968243669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c009370 con 0x7f2550062970 2024-01-24T15:32:47.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.836+0000 7f2554a91700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2550062970 0x7f25500a99f0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 rx=0x7f254400e950 tx=0x7f2544009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:47.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.836+0000 7f254cff9700 1 -- 172.21.15.29:0/2968243669 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544009530 con 0x7f2550062970 2024-01-24T15:32:47.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.836+0000 7f254cff9700 1 -- 172.21.15.29:0/2968243669 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254400c850 con 0x7f2550062970 2024-01-24T15:32:47.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.836+0000 7f254cff9700 1 -- 172.21.15.29:0/2968243669 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254400cbd0 con 0x7f2550062970 2024-01-24T15:32:47.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.836+0000 7f2556cf5700 1 -- 172.21.15.29:0/2968243669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25500a7810 con 0x7f2550062970 2024-01-24T15:32:47.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.836+0000 7f2556cf5700 1 -- 172.21.15.29:0/2968243669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25500aa3d0 con 0x7f2550062970 2024-01-24T15:32:47.841 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.837+0000 7f2556cf5700 1 -- 172.21.15.29:0/2968243669 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25500475e0 con 0x7f2550062970 2024-01-24T15:32:47.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.837+0000 7f254cff9700 1 -- 172.21.15.29:0/2968243669 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f254401b400 con 0x7f2550062970 2024-01-24T15:32:47.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.838+0000 7f254cff9700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f253405f270 0x7f2534061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:47.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.838+0000 7f254cff9700 1 -- 172.21.15.29:0/2968243669 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f254407ac50 con 0x7f2550062970 2024-01-24T15:32:47.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.838+0000 7f254ffff700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f253405f270 0x7f2534061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:47.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.839+0000 7f254ffff700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f253405f270 0x7f2534061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f25500648d0 tx=0x7f253c017000).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:47.844 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:47.842+0000 7f254cff9700 1 -- 172.21.15.29:0/2968243669 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f254404ba20 con 0x7f2550062970 2024-01-24T15:32:48.020 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.017+0000 7f2556cf5700 1 -- 172.21.15.29:0/2968243669 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f255006b350 con 0x7f253405f270 2024-01-24T15:32:48.023 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.021+0000 7f254cff9700 1 -- 172.21.15.29:0/2968243669 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70724 (secure 0 0 0) 0x7f255006b350 con 0x7f253405f270 2024-01-24T15:32:48.024 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:48.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.023+0000 7f2556cf5700 1 -- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f253405f270 msgr2=0x7f2534061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.023+0000 7f2556cf5700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f253405f270 0x7f2534061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f25500648d0 tx=0x7f253c017000).stop 2024-01-24T15:32:48.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.023+0000 7f2556cf5700 1 -- 172.21.15.29:0/2968243669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2550062970 msgr2=0x7f25500a99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.023+0000 7f2556cf5700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2550062970 0x7f25500a99f0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 rx=0x7f254400e950 tx=0x7f2544009060).stop 2024-01-24T15:32:48.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.023+0000 7f2556cf5700 1 -- 172.21.15.29:0/2968243669 shutdown_connections 2024-01-24T15:32:48.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.023+0000 7f2556cf5700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f253405f270 0x7f2534061730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.023+0000 7f2556cf5700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25500c4da0 0x7f25500abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.023+0000 7f2556cf5700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2550062970 0x7f25500a99f0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.023+0000 7f2556cf5700 1 --2- 172.21.15.29:0/2968243669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2550066b40 0x7f25500a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.023+0000 7f2556cf5700 1 -- 172.21.15.29:0/2968243669 >> 172.21.15.29:0/2968243669 conn(0x7f25500c8580 msgr2=0x7f255014d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:48.026 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.023+0000 7f2556cf5700 1 -- 172.21.15.29:0/2968243669 shutdown_connections 2024-01-24T15:32:48.027 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.023+0000 7f2556cf5700 1 -- 172.21.15.29:0/2968243669 wait complete. 2024-01-24T15:32:48.027 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:32:48.039 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-01-24T15:32:46.478472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":207,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2473504,"kb_used_data":23008,"kb_used_omap":0,"kb_used_meta":2450432,"kb_avail":752501216,"statfs":{"total":773094113280,"available":770561245184,"internally_reserved":0,"allocated":23560192,"data_stored":13690149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2509242368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13041,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.008062"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":190,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.522546+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:32:40.522546+0000","last_peered":"2024-01-24T15:32:40.522546+0000","last_clean":"2024-01-24T15:32:40.522546+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:32:40.522546+0000","last_undegraded":"2024-01-24T15:32:40.522546+0000","last_fullsized":"2024-01-24T15:32:40.522546+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":216,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.467607+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:32:40.467607+0000","last_peered":"2024-01-24T15:32:40.467607+0000","last_clean":"2024-01-24T15:32:40.467607+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:32:40.467607+0000","last_undegraded":"2024-01-24T15:32:40.467607+0000","last_fullsized":"2024-01-24T15:32:40.467607+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":214,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.493041+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:32:43.493041+0000","last_peered":"2024-01-24T15:32:43.493041+0000","last_clean":"2024-01-24T15:32:43.493041+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:32:43.493041+0000","last_undegraded":"2024-01-24T15:32:43.493041+0000","last_fullsized":"2024-01-24T15:32:43.493041+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":216,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.467565+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:32:40.467565+0000","last_peered":"2024-01-24T15:32:40.467565+0000","last_clean":"2024-01-24T15:32:40.467565+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:32:40.467565+0000","last_undegraded":"2024-01-24T15:32:40.467565+0000","last_fullsized":"2024-01-24T15:32:40.467565+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":193,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.476085+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:32:40.476085+0000","last_peered":"2024-01-24T15:32:40.476085+0000","last_clean":"2024-01-24T15:32:40.476085+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:32:40.476085+0000","last_undegraded":"2024-01-24T15:32:40.476085+0000","last_fullsized":"2024-01-24T15:32:40.476085+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":191,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.476001+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:32:40.476001+0000","last_peered":"2024-01-24T15:32:40.476001+0000","last_clean":"2024-01-24T15:32:40.476001+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:32:40.476001+0000","last_undegraded":"2024-01-24T15:32:40.476001+0000","last_fullsized":"2024-01-24T15:32:40.476001+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":191,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.476346+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:32:40.476346+0000","last_peered":"2024-01-24T15:32:40.476346+0000","last_clean":"2024-01-24T15:32:40.476346+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:32:40.476346+0000","last_undegraded":"2024-01-24T15:32:40.476346+0000","last_fullsized":"2024-01-24T15:32:40.476346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":190,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.522854+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:32:40.522854+0000","last_peered":"2024-01-24T15:32:40.522854+0000","last_clean":"2024-01-24T15:32:40.522854+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:32:40.522854+0000","last_undegraded":"2024-01-24T15:32:40.522854+0000","last_fullsized":"2024-01-24T15:32:40.522854+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":189,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.748676+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:32:40.748676+0000","last_peered":"2024-01-24T15:32:40.748676+0000","last_clean":"2024-01-24T15:32:40.748676+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:32:40.748676+0000","last_undegraded":"2024-01-24T15:32:40.748676+0000","last_fullsized":"2024-01-24T15:32:40.748676+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575976,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304064,"kb_used_data":2872,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94067776,"statfs":{"total":96636764160,"available":96325402624,"internally_reserved":0,"allocated":2940928,"data_stored":1709169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.47499999999999998,"5min":0.39300000000000002,"15min":0.32000000000000001},"max":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.83199999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.75700000000000001,"5min":0.84799999999999998,"15min":0.85499999999999998},"last":0.59599999999999997}]},{"osd":1,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.61299999999999999,"5min":0.374,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":0.878,"15min":0.878},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.89600000000000002},"last":0.437}]},{"osd":2,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.55900000000000005,"5min":0.374,"15min":0.374},"max":{"1min":0.82899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.378},{"interface":"front","average":{"1min":0.629,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80800000000000005,"5min":0.83299999999999996,"15min":0.85099999999999998},"last":0.498}]},{"osd":3,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":0.82399999999999995,"5min":0.83199999999999996,"15min":0.92600000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.50700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.88},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51400000000000001,"15min":0.49399999999999999},"min":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.30599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.39600000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.439,"5min":0.35899999999999999,"15min":0.32300000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.76100000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52800000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.42399999999999999,"5min":0.39000000000000001,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81899999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.623,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.40999999999999998,"15min":0.35099999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.92700000000000005},"last":0.42499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575977,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308148,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063692,"statfs":{"total":96636764160,"available":96321220608,"internally_reserved":0,"allocated":2928640,"data_stored":1705219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.442,"5min":0.377,"15min":0.31900000000000001},"max":{"1min":0.83699999999999997,"5min":0.998,"15min":0.998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.61699999999999999},"min":{"1min":0.48599999999999999,"5min":0.44700000000000001,"15min":0.32800000000000001},"max":{"1min":0.82399999999999995,"5min":0.82699999999999996,"15min":0.95599999999999996},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":1.089},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.45000000000000001,"5min":0.41899999999999998,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.85599999999999998,"15min":0.94299999999999995},"last":0.49199999999999999}]},{"osd":2,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.627},"min":{"1min":0.379,"5min":0.379,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.90900000000000003},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64200000000000002,"15min":0.63900000000000001},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.371},"max":{"1min":0.879,"5min":0.96299999999999997,"15min":1.0980000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.42299999999999999},"max":{"1min":0.78300000000000003,"5min":0.83899999999999997,"15min":1.169},"last":0.621}]},{"osd":4,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":0.73599999999999999,"5min":0.753,"15min":0.753},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.33100000000000002,"15min":0.312},"max":{"1min":0.79100000000000004,"5min":0.81399999999999995,"15min":0.83999999999999997},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":0.82499999999999996,"15min":0.86499999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69599999999999995,"5min":0.81699999999999995,"15min":0.82799999999999996},"last":0.69799999999999995}]},{"osd":7,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71099999999999997,"5min":0.83699999999999997,"15min":0.88},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.83599999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575976,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308156,"kb_used_data":2868,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063684,"statfs":{"total":96636764160,"available":96321212416,"internally_reserved":0,"allocated":2936832,"data_stored":1698602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.41799999999999998},"min":{"1min":0.28999999999999998,"5min":0.26600000000000001,"15min":0.24299999999999999},"max":{"1min":0.91200000000000003,"5min":3.254,"15min":3.254},"last":0.312},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.26100000000000001},"max":{"1min":0.82299999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.34999999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.34200000000000003,"5min":0.30299999999999999,"15min":0.28499999999999998},"max":{"1min":0.86499999999999999,"5min":1.306,"15min":1.306},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.45200000000000001},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39100000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.46600000000000003},"min":{"1min":0.31900000000000001,"5min":0.312,"15min":0.308},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.377},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.36899999999999999,"5min":0.34100000000000003,"15min":0.32600000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.48399999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73799999999999999,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47299999999999998},"max":{"1min":1.1080000000000001,"5min":5.3630000000000004,"15min":5.3630000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.55500000000000005,"5min":0.46300000000000002,"15min":0.40400000000000003},"max":{"1min":1.244,"5min":1.677,"15min":1.677},"last":0.622}]},{"osd":5,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.71699999999999997},"min":{"1min":0.53900000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.232,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.56100000000000005,"5min":0.47399999999999998,"15min":0.46100000000000002},"max":{"1min":1.081,"5min":1.117,"15min":1.2989999999999999},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.54100000000000004,"5min":0.47899999999999998,"15min":0.46800000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.57599999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1859999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.63500000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.58099999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1759999999999999,"5min":1.704,"15min":1.704},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81100000000000005,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1599999999999999,"5min":5.29,"15min":5.29},"last":0.69499999999999995}]}]},{"osd":0,"up_from":14,"seq":60129543274,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312244,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059596,"statfs":{"total":96636764160,"available":96317026304,"internally_reserved":0,"allocated":2928640,"data_stored":1707100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.40799999999999997},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.26000000000000001},"max":{"1min":0.65600000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.27300000000000002},"max":{"1min":0.66100000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.44}]},{"osd":2,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.318,"15min":0.29599999999999999},"max":{"1min":0.73399999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.46899999999999997,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":0.64600000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.44500000000000001,"15min":0.442},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59299999999999997,"5min":0.67400000000000004,"15min":0.72799999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76600000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.98699999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.51900000000000002,"5min":0.45100000000000001,"15min":0.41199999999999998},"max":{"1min":0.92700000000000005,"5min":0.998,"15min":0.998},"last":0.58499999999999996}]},{"osd":5,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67900000000000005,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.40699999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.0029999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.55800000000000005,"5min":0.46700000000000003,"15min":0.46500000000000002},"max":{"1min":1.0960000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.504}]},{"osd":6,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97799999999999998,"5min":1.27,"15min":1.27},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.61799999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72799999999999998,"15min":0.73099999999999998},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98699999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.55600000000000005,"5min":0.51000000000000001,"15min":0.46600000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.70999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575977,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312288,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059552,"statfs":{"total":96636764160,"available":96316981248,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.69699999999999995},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.60899999999999999,"5min":0.66600000000000004,"15min":0.70599999999999996},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.432,"15min":0.432},"min":{"1min":0.32200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.40200000000000002,"5min":0.32600000000000001,"15min":0.27400000000000002},"max":{"1min":0.66400000000000003,"5min":0.69799999999999995,"15min":0.80900000000000005},"last":0.46400000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.29099999999999998},"max":{"1min":0.623,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.441}]},{"osd":4,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.57799999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78500000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.55100000000000005,"5min":0.45000000000000001,"15min":0.376},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.72299999999999998}]},{"osd":5,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":0.996,"15min":1.0589999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.45700000000000002},"max":{"1min":0.95999999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.92300000000000004}]},{"osd":6,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.39700000000000002},"max":{"1min":0.91800000000000004,"5min":1.006,"15min":1.0740000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.51700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.89200000000000002,"5min":1.0229999999999999,"15min":1.083},"last":0.84699999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96599999999999997,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.75800000000000001},"min":{"1min":0.61299999999999999,"5min":0.52000000000000002,"15min":0.51100000000000001},"max":{"1min":0.92000000000000004,"5min":1.0429999999999999,"15min":1.123},"last":0.89900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575977,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.318,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.438},"min":{"1min":0.35299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.76100000000000001},"last":0.38}]},{"osd":1,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42799999999999999},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.65000000000000002},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.752},"last":0.40100000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.80800000000000005},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.81899999999999995},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66000000000000003,"15min":0.66100000000000003},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.437},"max":{"1min":0.80100000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.93899999999999995,"5min":1.026,"15min":1.026},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.056,"15min":1.056},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.499,"15min":0.47199999999999998},"max":{"1min":0.93700000000000006,"5min":1.095,"15min":1.095},"last":0.56899999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":1.1499999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72099999999999997,"15min":0.72199999999999998},"min":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.0940000000000001},"last":0.54300000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.55000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.018,"5min":1.071,"15min":1.083},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.53600000000000003},"max":{"1min":0.95399999999999996,"5min":1.081,"15min":1.1359999999999999},"last":0.627}]}]},{"osd":4,"up_from":13,"seq":55834575977,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312260,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059580,"statfs":{"total":96636764160,"available":96317009920,"internally_reserved":0,"allocated":2945024,"data_stored":1705967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.46700000000000003,"5min":0.39400000000000002,"15min":0.375},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.80100000000000005,"5min":0.876,"15min":0.876},"last":0.45000000000000001}]},{"osd":1,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.622},"min":{"1min":0.496,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.45900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.379}]},{"osd":2,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.79100000000000004,"5min":0.80400000000000005,"15min":0.82199999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.40500000000000003,"15min":0.38500000000000001},"max":{"1min":0.82899999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.442},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82299999999999995,"5min":0.86799999999999999,"15min":0.87},"last":0.54600000000000004}]},{"osd":5,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.69699999999999995,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.33800000000000002,"5min":0.31900000000000001,"15min":0.308},"max":{"1min":0.66200000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.41399999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.33600000000000002},"max":{"1min":0.61199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.502},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.37},"max":{"1min":0.72599999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51600000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.34899999999999998},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.375,"15min":0.35799999999999998},"max":{"1min":0.78100000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.52300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575977,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308152,"kb_used_data":2864,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063688,"statfs":{"total":96636764160,"available":96321216512,"internally_reserved":0,"allocated":2932736,"data_stored":1705942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.34799999999999998},"max":{"1min":0.88300000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.432},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.48399999999999999,"5min":0.41999999999999998,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":1.149,"15min":1.149},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.52800000000000002,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":1.125,"15min":1.125},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.38400000000000001,"15min":0.38},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.432,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77900000000000003,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.61599999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.41199999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":1.252},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.51100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.95499999999999996},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.64400000000000002},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":1.196,"15min":1.196},"last":0.52200000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.58699999999999997,"5min":0.78700000000000003,"15min":0.90500000000000003},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.81299999999999994},"last":0.30299999999999999}]},{"osd":6,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.50600000000000001},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68400000000000005,"5min":0.82199999999999995,"15min":1.0209999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.54200000000000004},"min":{"1min":0.41099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.98399999999999999},"last":0.40000000000000002}]},{"osd":7,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.54200000000000004},"min":{"1min":0.39700000000000002,"5min":0.371,"15min":0.33600000000000002},"max":{"1min":0.629,"5min":0.80100000000000005,"15min":0.86399999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.57799999999999996},"min":{"1min":0.372,"5min":0.371,"15min":0.371},"max":{"1min":0.80900000000000005,"5min":0.83099999999999996,"15min":0.91700000000000004},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:32:48.041 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:32:48.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.168+0000 7f9ab05ff700 1 Processor -- start 2024-01-24T15:32:48.171 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.168+0000 7f9ab05ff700 1 -- start start 2024-01-24T15:32:48.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.169+0000 7f9ab05ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9aa8150d40 0x7f9aa8069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9ab05ff700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa80c2ce0 con 0x7f9aa8065ec0 2024-01-24T15:32:48.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9ab05ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa8069890 con 0x7f9aa8155040 2024-01-24T15:32:48.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9ab05ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa8069b70 con 0x7f9aa8150d40 2024-01-24T15:32:48.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9aadb9a700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9aa8065ec0 0x7f9aa8154930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53944/0 (socket says 172.21.15.29:53944) 2024-01-24T15:32:48.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9aadb9a700 1 -- 172.21.15.29:0/3357932558 learned_addr learned my addr 172.21.15.29:0/3357932558 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:48.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9aae39b700 1 --2- 172.21.15.29:0/3357932558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9aa8150d40 0x7f9aa8069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9aad399700 1 -- 172.21.15.29:0/3357932558 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3735965230 0 0) 0x7f9aa80c2ce0 con 0x7f9aa8065ec0 2024-01-24T15:32:48.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9aad399700 1 -- 172.21.15.29:0/3357932558 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a940033f0 con 0x7f9aa8065ec0 2024-01-24T15:32:48.173 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9aad399700 1 -- 172.21.15.29:0/3357932558 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884168203 0 0) 0x7f9aa8069890 con 0x7f9aa8155040 2024-01-24T15:32:48.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9aad399700 1 -- 172.21.15.29:0/3357932558 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa80c2ce0 con 0x7f9aa8155040 2024-01-24T15:32:48.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9aae39b700 1 -- 172.21.15.29:0/3357932558 >> v1:172.21.15.163:6789/0 conn(0x7f9aa8155040 legacy=0x7f9aa80c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9aae39b700 1 -- 172.21.15.29:0/3357932558 >> v1:172.21.15.29:6789/0 conn(0x7f9aa8065ec0 legacy=0x7f9aa8154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.170+0000 7f9aae39b700 1 -- 172.21.15.29:0/3357932558 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa8069e50 con 0x7f9aa8150d40 2024-01-24T15:32:48.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.171+0000 7f9aae39b700 1 --2- 172.21.15.29:0/3357932558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9aa8150d40 0x7f9aa8069240 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f9aa4009790 tx=0x7f9aa4009060).ready entity=mon.2 client_cookie=6b051ee68e4c9826 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:48.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.171+0000 7f9aad399700 1 -- 172.21.15.29:0/3357932558 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400f510 con 0x7f9aa8150d40 2024-01-24T15:32:48.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.171+0000 7f9aad399700 1 -- 172.21.15.29:0/3357932558 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa400f670 con 0x7f9aa8150d40 2024-01-24T15:32:48.174 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.171+0000 7f9aad399700 1 -- 172.21.15.29:0/3357932558 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400f9f0 con 0x7f9aa8150d40 2024-01-24T15:32:48.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.171+0000 7f9ab05ff700 1 -- 172.21.15.29:0/3357932558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9aa8150d40 msgr2=0x7f9aa8069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.171+0000 7f9ab05ff700 1 --2- 172.21.15.29:0/3357932558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9aa8150d40 0x7f9aa8069240 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f9aa4009790 tx=0x7f9aa4009060).stop 2024-01-24T15:32:48.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.171+0000 7f9ab05ff700 1 -- 172.21.15.29:0/3357932558 shutdown_connections 2024-01-24T15:32:48.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.171+0000 7f9ab05ff700 1 --2- 172.21.15.29:0/3357932558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9aa8150d40 0x7f9aa8069240 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.171+0000 7f9ab05ff700 1 -- 172.21.15.29:0/3357932558 >> 172.21.15.29:0/3357932558 conn(0x7f9aa80c8580 msgr2=0x7f9aa80c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:48.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.171+0000 7f9ab05ff700 1 -- 172.21.15.29:0/3357932558 shutdown_connections 2024-01-24T15:32:48.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.171+0000 7f9ab05ff700 1 -- 172.21.15.29:0/3357932558 wait complete. 2024-01-24T15:32:48.175 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.173+0000 7f9ab05ff700 1 Processor -- start 2024-01-24T15:32:48.176 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.173+0000 7f9ab05ff700 1 -- start start 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.173+0000 7f9ab05ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9aa8065ec0 0x7f9aa814e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.173+0000 7f9ab05ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa8150d40 0x7f9aa8167ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9ab05ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9aa8155040 0x7f9aa81683e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9ab05ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa81688d0 con 0x7f9aa8150d40 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9ab05ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa80ab260 con 0x7f9aa8065ec0 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9ab05ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa814f400 con 0x7f9aa8155040 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9aadb9a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa8150d40 0x7f9aa8167ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9aadb9a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa8150d40 0x7f9aa8167ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52468/0 (socket says 172.21.15.29:52468) 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9aadb9a700 1 -- 172.21.15.29:0/230950808 learned_addr learned my addr 172.21.15.29:0/230950808 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9aae39b700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9aa8065ec0 0x7f9aa814e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9aaeb9c700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9aa8155040 0x7f9aa81683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9aadb9a700 1 -- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9aa8155040 msgr2=0x7f9aa81683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9aadb9a700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9aa8155040 0x7f9aa81683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9aadb9a700 1 -- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9aa8065ec0 msgr2=0x7f9aa814e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.178 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9aadb9a700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9aa8065ec0 0x7f9aa814e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9aadb9a700 1 -- 172.21.15.29:0/230950808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa40092a0 con 0x7f9aa8150d40 2024-01-24T15:32:48.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9aae39b700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9aa8065ec0 0x7f9aa814e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:48.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9aadb9a700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa8150d40 0x7f9aa8167ef0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 rx=0x7f9a9800c470 tx=0x7f9a98007cc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:48.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9a9e7fc700 1 -- 172.21.15.29:0/230950808 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a980072e0 con 0x7f9aa8150d40 2024-01-24T15:32:48.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9a9e7fc700 1 -- 172.21.15.29:0/230950808 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a98007440 con 0x7f9aa8150d40 2024-01-24T15:32:48.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9ab05ff700 1 -- 172.21.15.29:0/230950808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa814f120 con 0x7f9aa8150d40 2024-01-24T15:32:48.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9a9e7fc700 1 -- 172.21.15.29:0/230950808 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a980057a0 con 0x7f9aa8150d40 2024-01-24T15:32:48.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.174+0000 7f9ab05ff700 1 -- 172.21.15.29:0/230950808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa80aaef0 con 0x7f9aa8150d40 2024-01-24T15:32:48.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.176+0000 7f9a9e7fc700 1 -- 172.21.15.29:0/230950808 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9a98005bd0 con 0x7f9aa8150d40 2024-01-24T15:32:48.179 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.176+0000 7f9ab05ff700 1 -- 172.21.15.29:0/230950808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa80ab410 con 0x7f9aa8150d40 2024-01-24T15:32:48.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.176+0000 7f9a9e7fc700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9a8c05f270 0x7f9a8c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.176+0000 7f9a9e7fc700 1 -- 172.21.15.29:0/230950808 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f9a98079f40 con 0x7f9aa8150d40 2024-01-24T15:32:48.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.176+0000 7f9aae39b700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9a8c05f270 0x7f9a8c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.180 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.177+0000 7f9aae39b700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9a8c05f270 0x7f9a8c061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9aa8165aa0 tx=0x7f9aa4017000).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:48.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.181+0000 7f9a9e7fc700 1 -- 172.21.15.29:0/230950808 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9aa80ab410 con 0x7f9aa8150d40 2024-01-24T15:32:48.361 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.359+0000 7f9ab05ff700 1 -- 172.21.15.29:0/230950808 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9aa80ab3d0 con 0x7f9a8c05f270 2024-01-24T15:32:48.364 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.362+0000 7f9a9e7fc700 1 -- 172.21.15.29:0/230950808 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70724 (secure 0 0 0) 0x7f9aa80ab3d0 con 0x7f9a8c05f270 2024-01-24T15:32:48.365 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:48.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.364+0000 7f9ab05ff700 1 -- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9a8c05f270 msgr2=0x7f9a8c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.364+0000 7f9ab05ff700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9a8c05f270 0x7f9a8c061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9aa8165aa0 tx=0x7f9aa4017000).stop 2024-01-24T15:32:48.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.364+0000 7f9ab05ff700 1 -- 172.21.15.29:0/230950808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa8150d40 msgr2=0x7f9aa8167ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.366 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.364+0000 7f9ab05ff700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa8150d40 0x7f9aa8167ef0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 rx=0x7f9a9800c470 tx=0x7f9a98007cc0).stop 2024-01-24T15:32:48.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.364+0000 7f9ab05ff700 1 -- 172.21.15.29:0/230950808 shutdown_connections 2024-01-24T15:32:48.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.364+0000 7f9ab05ff700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f9a8c05f270 0x7f9a8c061730 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.364+0000 7f9ab05ff700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9aa8155040 0x7f9aa81683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.364+0000 7f9ab05ff700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9aa8065ec0 0x7f9aa814e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.364+0000 7f9ab05ff700 1 --2- 172.21.15.29:0/230950808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9aa8150d40 0x7f9aa8167ef0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.364+0000 7f9ab05ff700 1 -- 172.21.15.29:0/230950808 >> 172.21.15.29:0/230950808 conn(0x7f9aa80c8580 msgr2=0x7f9aa80c5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:48.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.364+0000 7f9ab05ff700 1 -- 172.21.15.29:0/230950808 shutdown_connections 2024-01-24T15:32:48.367 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.364+0000 7f9ab05ff700 1 -- 172.21.15.29:0/230950808 wait complete. 2024-01-24T15:32:48.368 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:32:48.379 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-01-24T15:32:46.478472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":207,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2473504,"kb_used_data":23008,"kb_used_omap":0,"kb_used_meta":2450432,"kb_avail":752501216,"statfs":{"total":773094113280,"available":770561245184,"internally_reserved":0,"allocated":23560192,"data_stored":13690149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2509242368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13041,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.008062"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":190,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.522546+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:32:40.522546+0000","last_peered":"2024-01-24T15:32:40.522546+0000","last_clean":"2024-01-24T15:32:40.522546+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:32:40.522546+0000","last_undegraded":"2024-01-24T15:32:40.522546+0000","last_fullsized":"2024-01-24T15:32:40.522546+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":216,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.467607+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:32:40.467607+0000","last_peered":"2024-01-24T15:32:40.467607+0000","last_clean":"2024-01-24T15:32:40.467607+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:32:40.467607+0000","last_undegraded":"2024-01-24T15:32:40.467607+0000","last_fullsized":"2024-01-24T15:32:40.467607+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":214,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.493041+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:32:43.493041+0000","last_peered":"2024-01-24T15:32:43.493041+0000","last_clean":"2024-01-24T15:32:43.493041+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:32:43.493041+0000","last_undegraded":"2024-01-24T15:32:43.493041+0000","last_fullsized":"2024-01-24T15:32:43.493041+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":216,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.467565+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:32:40.467565+0000","last_peered":"2024-01-24T15:32:40.467565+0000","last_clean":"2024-01-24T15:32:40.467565+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:32:40.467565+0000","last_undegraded":"2024-01-24T15:32:40.467565+0000","last_fullsized":"2024-01-24T15:32:40.467565+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":193,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.476085+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:32:40.476085+0000","last_peered":"2024-01-24T15:32:40.476085+0000","last_clean":"2024-01-24T15:32:40.476085+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:32:40.476085+0000","last_undegraded":"2024-01-24T15:32:40.476085+0000","last_fullsized":"2024-01-24T15:32:40.476085+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":191,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.476001+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:32:40.476001+0000","last_peered":"2024-01-24T15:32:40.476001+0000","last_clean":"2024-01-24T15:32:40.476001+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:32:40.476001+0000","last_undegraded":"2024-01-24T15:32:40.476001+0000","last_fullsized":"2024-01-24T15:32:40.476001+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":191,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.476346+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:32:40.476346+0000","last_peered":"2024-01-24T15:32:40.476346+0000","last_clean":"2024-01-24T15:32:40.476346+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:32:40.476346+0000","last_undegraded":"2024-01-24T15:32:40.476346+0000","last_fullsized":"2024-01-24T15:32:40.476346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":190,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.522854+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:32:40.522854+0000","last_peered":"2024-01-24T15:32:40.522854+0000","last_clean":"2024-01-24T15:32:40.522854+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:32:40.522854+0000","last_undegraded":"2024-01-24T15:32:40.522854+0000","last_fullsized":"2024-01-24T15:32:40.522854+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":189,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-01-24T15:32:40.748676+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:32:40.748676+0000","last_peered":"2024-01-24T15:32:40.748676+0000","last_clean":"2024-01-24T15:32:40.748676+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:32:40.748676+0000","last_undegraded":"2024-01-24T15:32:40.748676+0000","last_fullsized":"2024-01-24T15:32:40.748676+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575976,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304064,"kb_used_data":2872,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94067776,"statfs":{"total":96636764160,"available":96325402624,"internally_reserved":0,"allocated":2940928,"data_stored":1709169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.47499999999999998,"5min":0.39300000000000002,"15min":0.32000000000000001},"max":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.83199999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.75700000000000001,"5min":0.84799999999999998,"15min":0.85499999999999998},"last":0.59599999999999997}]},{"osd":1,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.61299999999999999,"5min":0.374,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":0.878,"15min":0.878},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.89600000000000002},"last":0.437}]},{"osd":2,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.55900000000000005,"5min":0.374,"15min":0.374},"max":{"1min":0.82899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.378},{"interface":"front","average":{"1min":0.629,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80800000000000005,"5min":0.83299999999999996,"15min":0.85099999999999998},"last":0.498}]},{"osd":3,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":0.82399999999999995,"5min":0.83199999999999996,"15min":0.92600000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.50700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.88},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51400000000000001,"15min":0.49399999999999999},"min":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.30599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.39600000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.439,"5min":0.35899999999999999,"15min":0.32300000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.76100000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52800000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.42399999999999999,"5min":0.39000000000000001,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81899999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.623,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.40999999999999998,"15min":0.35099999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.92700000000000005},"last":0.42499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575977,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308148,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063692,"statfs":{"total":96636764160,"available":96321220608,"internally_reserved":0,"allocated":2928640,"data_stored":1705219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.442,"5min":0.377,"15min":0.31900000000000001},"max":{"1min":0.83699999999999997,"5min":0.998,"15min":0.998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.61699999999999999},"min":{"1min":0.48599999999999999,"5min":0.44700000000000001,"15min":0.32800000000000001},"max":{"1min":0.82399999999999995,"5min":0.82699999999999996,"15min":0.95599999999999996},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":1.089},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.45000000000000001,"5min":0.41899999999999998,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.85599999999999998,"15min":0.94299999999999995},"last":0.49199999999999999}]},{"osd":2,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.627},"min":{"1min":0.379,"5min":0.379,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.90900000000000003},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64200000000000002,"15min":0.63900000000000001},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.371},"max":{"1min":0.879,"5min":0.96299999999999997,"15min":1.0980000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.42299999999999999},"max":{"1min":0.78300000000000003,"5min":0.83899999999999997,"15min":1.169},"last":0.621}]},{"osd":4,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":0.73599999999999999,"5min":0.753,"15min":0.753},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.33100000000000002,"15min":0.312},"max":{"1min":0.79100000000000004,"5min":0.81399999999999995,"15min":0.83999999999999997},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":0.82499999999999996,"15min":0.86499999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69599999999999995,"5min":0.81699999999999995,"15min":0.82799999999999996},"last":0.69799999999999995}]},{"osd":7,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71099999999999997,"5min":0.83699999999999997,"15min":0.88},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.83599999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575976,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308156,"kb_used_data":2868,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063684,"statfs":{"total":96636764160,"available":96321212416,"internally_reserved":0,"allocated":2936832,"data_stored":1698602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.41799999999999998},"min":{"1min":0.28999999999999998,"5min":0.26600000000000001,"15min":0.24299999999999999},"max":{"1min":0.91200000000000003,"5min":3.254,"15min":3.254},"last":0.312},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.26100000000000001},"max":{"1min":0.82299999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.34999999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.34200000000000003,"5min":0.30299999999999999,"15min":0.28499999999999998},"max":{"1min":0.86499999999999999,"5min":1.306,"15min":1.306},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.45200000000000001},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39100000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.46600000000000003},"min":{"1min":0.31900000000000001,"5min":0.312,"15min":0.308},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.377},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.36899999999999999,"5min":0.34100000000000003,"15min":0.32600000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.48399999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73799999999999999,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47299999999999998},"max":{"1min":1.1080000000000001,"5min":5.3630000000000004,"15min":5.3630000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.55500000000000005,"5min":0.46300000000000002,"15min":0.40400000000000003},"max":{"1min":1.244,"5min":1.677,"15min":1.677},"last":0.622}]},{"osd":5,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.71699999999999997},"min":{"1min":0.53900000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.232,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.56100000000000005,"5min":0.47399999999999998,"15min":0.46100000000000002},"max":{"1min":1.081,"5min":1.117,"15min":1.2989999999999999},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.54100000000000004,"5min":0.47899999999999998,"15min":0.46800000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.57599999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1859999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.63500000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.58099999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1759999999999999,"5min":1.704,"15min":1.704},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81100000000000005,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1599999999999999,"5min":5.29,"15min":5.29},"last":0.69499999999999995}]}]},{"osd":0,"up_from":14,"seq":60129543274,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312244,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059596,"statfs":{"total":96636764160,"available":96317026304,"internally_reserved":0,"allocated":2928640,"data_stored":1707100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.40799999999999997},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.26000000000000001},"max":{"1min":0.65600000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.27300000000000002},"max":{"1min":0.66100000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.44}]},{"osd":2,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.318,"15min":0.29599999999999999},"max":{"1min":0.73399999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.46899999999999997,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":0.64600000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.44500000000000001,"15min":0.442},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59299999999999997,"5min":0.67400000000000004,"15min":0.72799999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76600000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.98699999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.51900000000000002,"5min":0.45100000000000001,"15min":0.41199999999999998},"max":{"1min":0.92700000000000005,"5min":0.998,"15min":0.998},"last":0.58499999999999996}]},{"osd":5,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67900000000000005,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.40699999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.0029999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.55800000000000005,"5min":0.46700000000000003,"15min":0.46500000000000002},"max":{"1min":1.0960000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.504}]},{"osd":6,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97799999999999998,"5min":1.27,"15min":1.27},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.61799999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72799999999999998,"15min":0.73099999999999998},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98699999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.55600000000000005,"5min":0.51000000000000001,"15min":0.46600000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.70999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575977,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312288,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059552,"statfs":{"total":96636764160,"available":96316981248,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.69699999999999995},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.60899999999999999,"5min":0.66600000000000004,"15min":0.70599999999999996},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.432,"15min":0.432},"min":{"1min":0.32200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.40200000000000002,"5min":0.32600000000000001,"15min":0.27400000000000002},"max":{"1min":0.66400000000000003,"5min":0.69799999999999995,"15min":0.80900000000000005},"last":0.46400000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.29099999999999998},"max":{"1min":0.623,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.441}]},{"osd":4,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.57799999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78500000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.55100000000000005,"5min":0.45000000000000001,"15min":0.376},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.72299999999999998}]},{"osd":5,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":0.996,"15min":1.0589999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.45700000000000002},"max":{"1min":0.95999999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.92300000000000004}]},{"osd":6,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.39700000000000002},"max":{"1min":0.91800000000000004,"5min":1.006,"15min":1.0740000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.51700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.89200000000000002,"5min":1.0229999999999999,"15min":1.083},"last":0.84699999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96599999999999997,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.75800000000000001},"min":{"1min":0.61299999999999999,"5min":0.52000000000000002,"15min":0.51100000000000001},"max":{"1min":0.92000000000000004,"5min":1.0429999999999999,"15min":1.123},"last":0.89900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575977,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.318,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.438},"min":{"1min":0.35299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.76100000000000001},"last":0.38}]},{"osd":1,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42799999999999999},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.65000000000000002},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.752},"last":0.40100000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.80800000000000005},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.81899999999999995},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66000000000000003,"15min":0.66100000000000003},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.437},"max":{"1min":0.80100000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.93899999999999995,"5min":1.026,"15min":1.026},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.056,"15min":1.056},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.499,"15min":0.47199999999999998},"max":{"1min":0.93700000000000006,"5min":1.095,"15min":1.095},"last":0.56899999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":1.1499999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72099999999999997,"15min":0.72199999999999998},"min":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.0940000000000001},"last":0.54300000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.55000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.018,"5min":1.071,"15min":1.083},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.53600000000000003},"max":{"1min":0.95399999999999996,"5min":1.081,"15min":1.1359999999999999},"last":0.627}]}]},{"osd":4,"up_from":13,"seq":55834575977,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312260,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059580,"statfs":{"total":96636764160,"available":96317009920,"internally_reserved":0,"allocated":2945024,"data_stored":1705967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.46700000000000003,"5min":0.39400000000000002,"15min":0.375},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.80100000000000005,"5min":0.876,"15min":0.876},"last":0.45000000000000001}]},{"osd":1,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.622},"min":{"1min":0.496,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.45900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.379}]},{"osd":2,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.79100000000000004,"5min":0.80400000000000005,"15min":0.82199999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.40500000000000003,"15min":0.38500000000000001},"max":{"1min":0.82899999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.442},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82299999999999995,"5min":0.86799999999999999,"15min":0.87},"last":0.54600000000000004}]},{"osd":5,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.69699999999999995,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.33800000000000002,"5min":0.31900000000000001,"15min":0.308},"max":{"1min":0.66200000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.41399999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.33600000000000002},"max":{"1min":0.61199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.502},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.37},"max":{"1min":0.72599999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51600000000000001}]},{"osd":7,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.34899999999999998},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.375,"15min":0.35799999999999998},"max":{"1min":0.78100000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.52300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575977,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308152,"kb_used_data":2864,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063688,"statfs":{"total":96636764160,"available":96321216512,"internally_reserved":0,"allocated":2932736,"data_stored":1705942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.34799999999999998},"max":{"1min":0.88300000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.432},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.48399999999999999,"5min":0.41999999999999998,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":1.149,"15min":1.149},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.52800000000000002,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":1.125,"15min":1.125},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.38400000000000001,"15min":0.38},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.432,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77900000000000003,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.61599999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.41199999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":1.252},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.51100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.95499999999999996},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.64400000000000002},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":1.196,"15min":1.196},"last":0.52200000000000002}]},{"osd":4,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.58699999999999997,"5min":0.78700000000000003,"15min":0.90500000000000003},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.81299999999999994},"last":0.30299999999999999}]},{"osd":6,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.50600000000000001},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68400000000000005,"5min":0.82199999999999995,"15min":1.0209999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.54200000000000004},"min":{"1min":0.41099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.98399999999999999},"last":0.40000000000000002}]},{"osd":7,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.54200000000000004},"min":{"1min":0.39700000000000002,"5min":0.371,"15min":0.33600000000000002},"max":{"1min":0.629,"5min":0.80100000000000005,"15min":0.86399999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.57799999999999996},"min":{"1min":0.372,"5min":0.371,"15min":0.371},"max":{"1min":0.80900000000000005,"5min":0.83099999999999996,"15min":0.91700000000000004},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:32:48.381 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-24T15:32:48.382 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:32:48.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.510+0000 7f96bc8cd700 1 Processor -- start 2024-01-24T15:32:48.512 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.510+0000 7f96bc8cd700 1 -- start start 2024-01-24T15:32:48.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96bc8cd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f96b8150d40 0x7f96b8069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96bc8cd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96b8155040 0x7f96b80c3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96bc8cd700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b80c2ce0 con 0x7f96b8065ec0 2024-01-24T15:32:48.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96bc8cd700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b8069890 con 0x7f96b8150d40 2024-01-24T15:32:48.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96bc8cd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b8069b70 con 0x7f96b8155040 2024-01-24T15:32:48.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96b659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f96b8150d40 0x7f96b8069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96b659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f96b8150d40 0x7f96b8069240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52480/0 (socket says 172.21.15.29:52480) 2024-01-24T15:32:48.515 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96b659c700 1 -- 172.21.15.29:0/3042955893 learned_addr learned my addr 172.21.15.29:0/3042955893 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:48.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96b6d9d700 1 --2- 172.21.15.29:0/3042955893 >> v2:172.21.15.163:3300/0 conn(0x7f96b8155040 0x7f96b80c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96b659c700 1 -- 172.21.15.29:0/3042955893 >> v2:172.21.15.163:3300/0 conn(0x7f96b8155040 msgr2=0x7f96b80c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:48.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96b659c700 1 --2- 172.21.15.29:0/3042955893 >> v2:172.21.15.163:3300/0 conn(0x7f96b8155040 0x7f96b80c3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96b659c700 1 -- 172.21.15.29:0/3042955893 >> v1:172.21.15.29:6789/0 conn(0x7f96b8065ec0 legacy=0x7f96b8154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.511+0000 7f96b659c700 1 -- 172.21.15.29:0/3042955893 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b8069e50 con 0x7f96b8150d40 2024-01-24T15:32:48.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.512+0000 7f96b659c700 1 --2- 172.21.15.29:0/3042955893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96b8150d40 0x7f96b8069240 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 rx=0x7f96b80c56a0 tx=0x7f96a80090f0).ready entity=mon.0 client_cookie=a139b6f4a350b2be server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:48.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.512+0000 7f96b559a700 1 -- 172.21.15.29:0/3042955893 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a800b520 con 0x7f96b8150d40 2024-01-24T15:32:48.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.512+0000 7f96b559a700 1 -- 172.21.15.29:0/3042955893 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a800b680 con 0x7f96b8150d40 2024-01-24T15:32:48.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.512+0000 7f96b559a700 1 -- 172.21.15.29:0/3042955893 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a800ba00 con 0x7f96b8150d40 2024-01-24T15:32:48.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.512+0000 7f96bc8cd700 1 -- 172.21.15.29:0/3042955893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96b8150d40 msgr2=0x7f96b8069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.512+0000 7f96bc8cd700 1 --2- 172.21.15.29:0/3042955893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96b8150d40 0x7f96b8069240 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 rx=0x7f96b80c56a0 tx=0x7f96a80090f0).stop 2024-01-24T15:32:48.516 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.513+0000 7f96bc8cd700 1 -- 172.21.15.29:0/3042955893 shutdown_connections 2024-01-24T15:32:48.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.513+0000 7f96bc8cd700 1 --2- 172.21.15.29:0/3042955893 >> v2:172.21.15.163:3300/0 conn(0x7f96b8155040 0x7f96b80c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.513+0000 7f96bc8cd700 1 --2- 172.21.15.29:0/3042955893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96b8150d40 0x7f96b8069240 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.513+0000 7f96bc8cd700 1 -- 172.21.15.29:0/3042955893 >> 172.21.15.29:0/3042955893 conn(0x7f96b80c8580 msgr2=0x7f96b80c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:48.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.513+0000 7f96bc8cd700 1 -- 172.21.15.29:0/3042955893 shutdown_connections 2024-01-24T15:32:48.517 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.513+0000 7f96bc8cd700 1 -- 172.21.15.29:0/3042955893 wait complete. 2024-01-24T15:32:48.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.514+0000 7f96bc8cd700 1 Processor -- start 2024-01-24T15:32:48.519 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.514+0000 7f96bc8cd700 1 -- start start 2024-01-24T15:32:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96bc8cd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b8065ec0 0x7f96b80aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96bc8cd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b8150d40 0x7f96b80c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96bc8cd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96b8155040 0x7f96b8151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96bc8cd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80abd40 con 0x7f96b8155040 2024-01-24T15:32:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96bc8cd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80c4500 con 0x7f96b8150d40 2024-01-24T15:32:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96bc8cd700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80ab130 con 0x7f96b8065ec0 2024-01-24T15:32:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96b6d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96b8155040 0x7f96b8151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96b6d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96b8155040 0x7f96b8151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52488/0 (socket says 172.21.15.29:52488) 2024-01-24T15:32:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96b6d9d700 1 -- 172.21.15.29:0/2957225746 learned_addr learned my addr 172.21.15.29:0/2957225746 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96b5d9b700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b8150d40 0x7f96b80c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.520 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96b6d9d700 1 -- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b8065ec0 msgr2=0x7f96b80aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96b6d9d700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b8065ec0 0x7f96b80aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96b6d9d700 1 -- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b8150d40 msgr2=0x7f96b80c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96b6d9d700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b8150d40 0x7f96b80c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96b6d9d700 1 -- 172.21.15.29:0/2957225746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a8009370 con 0x7f96b8155040 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96b6d9d700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96b8155040 0x7f96b8151120 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 rx=0x7f96b0009ef0 tx=0x7f96b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96a67fc700 1 -- 172.21.15.29:0/2957225746 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b000cc50 con 0x7f96b8155040 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96a67fc700 1 -- 172.21.15.29:0/2957225746 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b000f040 con 0x7f96b8155040 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96bc8cd700 1 -- 172.21.15.29:0/2957225746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b80ab410 con 0x7f96b8155040 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.515+0000 7f96a67fc700 1 -- 172.21.15.29:0/2957225746 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0013400 con 0x7f96b8155040 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.516+0000 7f96bc8cd700 1 -- 172.21.15.29:0/2957225746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b80ab9d0 con 0x7f96b8155040 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.517+0000 7f96bc8cd700 1 -- 172.21.15.29:0/2957225746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b80475e0 con 0x7f96b8155040 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.517+0000 7f96a67fc700 1 -- 172.21.15.29:0/2957225746 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f96b0013770 con 0x7f96b8155040 2024-01-24T15:32:48.521 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.517+0000 7f96a67fc700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f969805f240 0x7f9698061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.517+0000 7f96a67fc700 1 -- 172.21.15.29:0/2957225746 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f96b007b060 con 0x7f96b8155040 2024-01-24T15:32:48.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.517+0000 7f96b659c700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f969805f240 0x7f9698061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.522 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.518+0000 7f96b659c700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f969805f240 0x7f9698061700 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f96a800b2e0 tx=0x7f96a8017000).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:48.523 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.522+0000 7f96a67fc700 1 -- 172.21.15.29:0/2957225746 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f96b004be90 con 0x7f96b8155040 2024-01-24T15:32:48.699 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.697+0000 7f96bc8cd700 1 -- 172.21.15.29:0/2957225746 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96b80abe80 con 0x7f969805f240 2024-01-24T15:32:48.702 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.700+0000 7f96a67fc700 1 -- 172.21.15.29:0/2957225746 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70884 (secure 0 0 0) 0x7f96b80abe80 con 0x7f969805f240 2024-01-24T15:32:48.703 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:48.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.702+0000 7f96bc8cd700 1 -- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f969805f240 msgr2=0x7f9698061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.702+0000 7f96bc8cd700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f969805f240 0x7f9698061700 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f96a800b2e0 tx=0x7f96a8017000).stop 2024-01-24T15:32:48.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.702+0000 7f96bc8cd700 1 -- 172.21.15.29:0/2957225746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96b8155040 msgr2=0x7f96b8151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.702+0000 7f96bc8cd700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96b8155040 0x7f96b8151120 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 rx=0x7f96b0009ef0 tx=0x7f96b000b040).stop 2024-01-24T15:32:48.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.702+0000 7f96bc8cd700 1 -- 172.21.15.29:0/2957225746 shutdown_connections 2024-01-24T15:32:48.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.702+0000 7f96bc8cd700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96b8155040 0x7f96b8151120 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.702+0000 7f96bc8cd700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f969805f240 0x7f9698061700 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.702+0000 7f96bc8cd700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b8065ec0 0x7f96b80aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-01-24T15:32:48.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.702+0000 7f96bc8cd700 1 --2- 172.21.15.29:0/2957225746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b8150d40 0x7f96b80c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.702+0000 7f96bc8cd700 1 -- 172.21.15.29:0/2957225746 >> 172.21.15.29:0/2957225746 conn(0x7f96b80c8580 msgr2=0x7f96b80627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:48.705 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.702+0000 7f96bc8cd700 1 -- 172.21.15.29:0/2957225746 shutdown_connections 2024-01-24T15:32:48.706 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.702+0000 7f96bc8cd700 1 -- 172.21.15.29:0/2957225746 wait complete. 2024-01-24T15:32:48.706 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:32:48.718 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-01-24T15:32:48.478833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2473724,"kb_used_data":23228,"kb_used_omap":0,"kb_used_meta":2450432,"kb_avail":752500996,"statfs":{"total":773094113280,"available":770561019904,"internally_reserved":0,"allocated":23785472,"data_stored":13831639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2509242368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13041,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.008062"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":193,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.886293+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:32:43.886293+0000","last_peered":"2024-01-24T15:32:43.886293+0000","last_clean":"2024-01-24T15:32:43.886293+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:32:43.886293+0000","last_undegraded":"2024-01-24T15:32:43.886293+0000","last_fullsized":"2024-01-24T15:32:43.886293+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":219,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.499262+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:32:43.499262+0000","last_peered":"2024-01-24T15:32:43.499262+0000","last_clean":"2024-01-24T15:32:43.499262+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:32:43.499262+0000","last_undegraded":"2024-01-24T15:32:43.499262+0000","last_fullsized":"2024-01-24T15:32:43.499262+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":214,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.493041+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:32:43.493041+0000","last_peered":"2024-01-24T15:32:43.493041+0000","last_clean":"2024-01-24T15:32:43.493041+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:32:43.493041+0000","last_undegraded":"2024-01-24T15:32:43.493041+0000","last_fullsized":"2024-01-24T15:32:43.493041+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":219,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.498006+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:32:43.498006+0000","last_peered":"2024-01-24T15:32:43.498006+0000","last_clean":"2024-01-24T15:32:43.498006+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:32:43.498006+0000","last_undegraded":"2024-01-24T15:32:43.498006+0000","last_fullsized":"2024-01-24T15:32:43.498006+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":196,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.880343+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:32:43.880343+0000","last_peered":"2024-01-24T15:32:43.880343+0000","last_clean":"2024-01-24T15:32:43.880343+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:32:43.880343+0000","last_undegraded":"2024-01-24T15:32:43.880343+0000","last_fullsized":"2024-01-24T15:32:43.880343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.880247+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:32:43.880247+0000","last_peered":"2024-01-24T15:32:43.880247+0000","last_clean":"2024-01-24T15:32:43.880247+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:32:43.880247+0000","last_undegraded":"2024-01-24T15:32:43.880247+0000","last_fullsized":"2024-01-24T15:32:43.880247+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":194,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.881987+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:32:43.881987+0000","last_peered":"2024-01-24T15:32:43.881987+0000","last_clean":"2024-01-24T15:32:43.881987+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:32:43.881987+0000","last_undegraded":"2024-01-24T15:32:43.881987+0000","last_fullsized":"2024-01-24T15:32:43.881987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":193,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.889119+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:32:43.889119+0000","last_peered":"2024-01-24T15:32:43.889119+0000","last_clean":"2024-01-24T15:32:43.889119+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:32:43.889119+0000","last_undegraded":"2024-01-24T15:32:43.889119+0000","last_fullsized":"2024-01-24T15:32:43.889119+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":192,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.882264+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:32:43.882264+0000","last_peered":"2024-01-24T15:32:43.882264+0000","last_clean":"2024-01-24T15:32:43.882264+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:32:43.882264+0000","last_undegraded":"2024-01-24T15:32:43.882264+0000","last_fullsized":"2024-01-24T15:32:43.882264+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575977,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304096,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94067744,"statfs":{"total":96636764160,"available":96325369856,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.47499999999999998,"5min":0.39300000000000002,"15min":0.32000000000000001},"max":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.83199999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.75700000000000001,"5min":0.84799999999999998,"15min":0.85499999999999998},"last":0.622}]},{"osd":1,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.61299999999999999,"5min":0.374,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":0.878,"15min":0.878},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.89600000000000002},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.55900000000000005,"5min":0.374,"15min":0.374},"max":{"1min":0.82899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80800000000000005,"5min":0.83299999999999996,"15min":0.85099999999999998},"last":0.68400000000000005}]},{"osd":3,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":0.82399999999999995,"5min":0.83199999999999996,"15min":0.92600000000000005},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.50700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.88},"last":0.66000000000000003}]},{"osd":4,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51400000000000001,"15min":0.49399999999999999},"min":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.30599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.58099999999999996}]},{"osd":5,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.439,"5min":0.35899999999999999,"15min":0.32300000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.76100000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.73299999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.42399999999999999,"5min":0.39000000000000001,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81899999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.623,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.40999999999999998,"15min":0.35099999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.92700000000000005},"last":0.70999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575978,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.442,"5min":0.377,"15min":0.31900000000000001},"max":{"1min":0.83699999999999997,"5min":0.998,"15min":0.998},"last":0.877},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.61699999999999999},"min":{"1min":0.48599999999999999,"5min":0.44700000000000001,"15min":0.32800000000000001},"max":{"1min":0.82399999999999995,"5min":0.82699999999999996,"15min":0.95599999999999996},"last":0.70999999999999996}]},{"osd":1,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":1.089},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.45000000000000001,"5min":0.41899999999999998,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.85599999999999998,"15min":0.94299999999999995},"last":0.82099999999999995}]},{"osd":2,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.627},"min":{"1min":0.379,"5min":0.379,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.90900000000000003},"last":0.754}]},{"osd":3,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64200000000000002,"15min":0.63900000000000001},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.371},"max":{"1min":0.879,"5min":0.96299999999999997,"15min":1.0980000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.42299999999999999},"max":{"1min":0.78300000000000003,"5min":0.83899999999999997,"15min":1.169},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":0.73599999999999999,"5min":0.753,"15min":0.753},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.33100000000000002,"15min":0.312},"max":{"1min":0.79100000000000004,"5min":0.81399999999999995,"15min":0.83999999999999997},"last":0.65300000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":0.82499999999999996,"15min":0.86499999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69599999999999995,"5min":0.81699999999999995,"15min":0.82799999999999996},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71099999999999997,"5min":0.83699999999999997,"15min":0.88},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.83599999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575977,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.41799999999999998},"min":{"1min":0.28999999999999998,"5min":0.26600000000000001,"15min":0.24299999999999999},"max":{"1min":0.91200000000000003,"5min":3.254,"15min":3.254},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.26100000000000001},"max":{"1min":0.82299999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.57399999999999995}]},{"osd":2,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.34200000000000003,"5min":0.30299999999999999,"15min":0.28499999999999998},"max":{"1min":0.86499999999999999,"5min":1.306,"15min":1.306},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.45200000000000001},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53900000000000003}]},{"osd":3,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.46600000000000003},"min":{"1min":0.31900000000000001,"5min":0.312,"15min":0.308},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.36899999999999999,"5min":0.34100000000000003,"15min":0.32600000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58699999999999997}]},{"osd":4,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73799999999999999,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47299999999999998},"max":{"1min":1.1080000000000001,"5min":5.3630000000000004,"15min":5.3630000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.55500000000000005,"5min":0.46300000000000002,"15min":0.40400000000000003},"max":{"1min":1.244,"5min":1.677,"15min":1.677},"last":0.69199999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.71699999999999997},"min":{"1min":0.53900000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.232,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.56100000000000005,"5min":0.47399999999999998,"15min":0.46100000000000002},"max":{"1min":1.081,"5min":1.117,"15min":1.2989999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.54100000000000004,"5min":0.47899999999999998,"15min":0.46800000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.57599999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1859999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.84799999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.58099999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1759999999999999,"5min":1.704,"15min":1.704},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81100000000000005,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1599999999999999,"5min":5.29,"15min":5.29},"last":0.879}]}]},{"osd":0,"up_from":14,"seq":60129543275,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312284,"kb_used_data":2900,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059556,"statfs":{"total":96636764160,"available":96316985344,"internally_reserved":0,"allocated":2969600,"data_stored":1728114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.40799999999999997},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.26000000000000001},"max":{"1min":0.65600000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.27300000000000002},"max":{"1min":0.66100000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.318,"15min":0.29599999999999999},"max":{"1min":0.73399999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.46899999999999997,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":0.64600000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.499}]},{"osd":3,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.44500000000000001,"15min":0.442},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59299999999999997,"5min":0.67400000000000004,"15min":0.72799999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76600000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.42999999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.98699999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.51900000000000002,"5min":0.45100000000000001,"15min":0.41199999999999998},"max":{"1min":0.92700000000000005,"5min":0.998,"15min":0.998},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67900000000000005,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.40699999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.0029999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.55800000000000005,"5min":0.46700000000000003,"15min":0.46500000000000002},"max":{"1min":1.0960000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97799999999999998,"5min":1.27,"15min":1.27},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.61799999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72799999999999998,"15min":0.73099999999999998},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98699999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.55600000000000005,"5min":0.51000000000000001,"15min":0.46600000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575977,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312288,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059552,"statfs":{"total":96636764160,"available":96316981248,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.69699999999999995},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.60899999999999999,"5min":0.66600000000000004,"15min":0.70599999999999996},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.432,"15min":0.432},"min":{"1min":0.32200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.40200000000000002,"5min":0.32600000000000001,"15min":0.27400000000000002},"max":{"1min":0.66400000000000003,"5min":0.69799999999999995,"15min":0.80900000000000005},"last":0.46400000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.29099999999999998},"max":{"1min":0.623,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.441}]},{"osd":4,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.57799999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78500000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.55100000000000005,"5min":0.45000000000000001,"15min":0.376},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.72299999999999998}]},{"osd":5,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":0.996,"15min":1.0589999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.45700000000000002},"max":{"1min":0.95999999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.92300000000000004}]},{"osd":6,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.39700000000000002},"max":{"1min":0.91800000000000004,"5min":1.006,"15min":1.0740000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.51700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.89200000000000002,"5min":1.0229999999999999,"15min":1.083},"last":0.84699999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96599999999999997,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.75800000000000001},"min":{"1min":0.61299999999999999,"5min":0.52000000000000002,"15min":0.51100000000000001},"max":{"1min":0.92000000000000004,"5min":1.0429999999999999,"15min":1.123},"last":0.89900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575977,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.318,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.438},"min":{"1min":0.35299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.76100000000000001},"last":0.38}]},{"osd":1,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42799999999999999},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.65000000000000002},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.752},"last":0.40100000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.80800000000000005},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.81899999999999995},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66000000000000003,"15min":0.66100000000000003},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.437},"max":{"1min":0.80100000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.93899999999999995,"5min":1.026,"15min":1.026},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.056,"15min":1.056},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.499,"15min":0.47199999999999998},"max":{"1min":0.93700000000000006,"5min":1.095,"15min":1.095},"last":0.56899999999999995}]},{"osd":6,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":1.1499999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72099999999999997,"15min":0.72199999999999998},"min":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.0940000000000001},"last":0.54300000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.55000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.018,"5min":1.071,"15min":1.083},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.53600000000000003},"max":{"1min":0.95399999999999996,"5min":1.081,"15min":1.1359999999999999},"last":0.627}]}]},{"osd":4,"up_from":13,"seq":55834575978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312288,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059552,"statfs":{"total":96636764160,"available":96316981248,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.46700000000000003,"5min":0.39400000000000002,"15min":0.375},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.80100000000000005,"5min":0.876,"15min":0.876},"last":0.50600000000000001}]},{"osd":1,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.622},"min":{"1min":0.496,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.45900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.71199999999999997}]},{"osd":2,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.79100000000000004,"5min":0.80400000000000005,"15min":0.82199999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.40500000000000003,"15min":0.38500000000000001},"max":{"1min":0.82899999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64100000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82299999999999995,"5min":0.86799999999999999,"15min":0.87},"last":0.72199999999999998}]},{"osd":5,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.69699999999999995,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.33800000000000002,"5min":0.31900000000000001,"15min":0.308},"max":{"1min":0.66200000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.60299999999999998}]},{"osd":6,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.33600000000000002},"max":{"1min":0.61199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.37},"max":{"1min":0.72599999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59699999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.34899999999999998},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.375,"15min":0.35799999999999998},"max":{"1min":0.78100000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.74399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575978,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.34799999999999998},"max":{"1min":0.88300000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.48399999999999999,"5min":0.41999999999999998,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":1.149,"15min":1.149},"last":0.72299999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.52800000000000002,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":1.125,"15min":1.125},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.38400000000000001,"15min":0.38},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.77600000000000002}]},{"osd":2,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.432,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77900000000000003,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.61599999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.41199999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":1.252},"last":0.70799999999999996}]},{"osd":3,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.51100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.95499999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.64400000000000002},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":1.196,"15min":1.196},"last":0.84099999999999997}]},{"osd":4,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.58699999999999997,"5min":0.78700000000000003,"15min":0.90500000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.81299999999999994},"last":0.52200000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.50600000000000001},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68400000000000005,"5min":0.82199999999999995,"15min":1.0209999999999999},"last":0.437},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.54200000000000004},"min":{"1min":0.41099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.98399999999999999},"last":0.54700000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.54200000000000004},"min":{"1min":0.39700000000000002,"5min":0.371,"15min":0.33600000000000002},"max":{"1min":0.629,"5min":0.80100000000000005,"15min":0.86399999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.57799999999999996},"min":{"1min":0.372,"5min":0.371,"15min":0.371},"max":{"1min":0.80900000000000005,"5min":0.83099999999999996,"15min":0.91700000000000004},"last":0.56799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:32:48.719 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.848+0000 7f39f48f4700 1 Processor -- start 2024-01-24T15:32:48.850 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.848+0000 7f39f48f4700 1 -- start start 2024-01-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39f48f4700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f39f01540a0 0x7f39f00c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39f48f4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39f0062ec0 0x7f39f0069240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39f48f4700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f0065360 con 0x7f39f01550f0 2024-01-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39f48f4700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f00a74f0 con 0x7f39f01540a0 2024-01-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39f48f4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00a6ce0 con 0x7f39f0062ec0 2024-01-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39ee59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f39f01540a0 0x7f39f00c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39ee59c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f39f01540a0 0x7f39f00c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52498/0 (socket says 172.21.15.29:52498) 2024-01-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39ee59c700 1 -- 172.21.15.29:0/571911232 learned_addr learned my addr 172.21.15.29:0/571911232 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39edd9b700 1 --1- 172.21.15.29:0/571911232 >> v1:172.21.15.29:6789/0 conn(0x7f39f01550f0 0x7f39f00627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53964/0 (socket says 172.21.15.29:53964) 2024-01-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39eed9d700 1 --2- 172.21.15.29:0/571911232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39f0062ec0 0x7f39f0069240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39ee59c700 1 -- 172.21.15.29:0/571911232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39f0062ec0 msgr2=0x7f39f0069240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:48.853 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39ee59c700 1 --2- 172.21.15.29:0/571911232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39f0062ec0 0x7f39f0069240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39ee59c700 1 -- 172.21.15.29:0/571911232 >> v1:172.21.15.29:6789/0 conn(0x7f39f01550f0 legacy=0x7f39f00627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39ee59c700 1 -- 172.21.15.29:0/571911232 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f00636f0 con 0x7f39f01540a0 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.849+0000 7f39ee59c700 1 --2- 172.21.15.29:0/571911232 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f01540a0 0x7f39f00c4ca0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 rx=0x7f39e0006fd0 tx=0x7f39e00090f0).ready entity=mon.0 client_cookie=6c922892ad046975 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.850+0000 7f39ed59a700 1 -- 172.21.15.29:0/571911232 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e000b600 con 0x7f39f01540a0 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.850+0000 7f39ed59a700 1 -- 172.21.15.29:0/571911232 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e000b760 con 0x7f39f01540a0 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.850+0000 7f39ed59a700 1 -- 172.21.15.29:0/571911232 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e000bae0 con 0x7f39f01540a0 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.850+0000 7f39f48f4700 1 -- 172.21.15.29:0/571911232 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f01540a0 msgr2=0x7f39f00c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.850+0000 7f39f48f4700 1 --2- 172.21.15.29:0/571911232 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f01540a0 0x7f39f00c4ca0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 rx=0x7f39e0006fd0 tx=0x7f39e00090f0).stop 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.850+0000 7f39f48f4700 1 -- 172.21.15.29:0/571911232 shutdown_connections 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.850+0000 7f39f48f4700 1 --2- 172.21.15.29:0/571911232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39f0062ec0 0x7f39f0069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.850+0000 7f39f48f4700 1 --2- 172.21.15.29:0/571911232 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f01540a0 0x7f39f00c4ca0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.850+0000 7f39f48f4700 1 -- 172.21.15.29:0/571911232 >> 172.21.15.29:0/571911232 conn(0x7f39f00c85c0 msgr2=0x7f39f00c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.850+0000 7f39f48f4700 1 -- 172.21.15.29:0/571911232 shutdown_connections 2024-01-24T15:32:48.854 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.850+0000 7f39f48f4700 1 -- 172.21.15.29:0/571911232 wait complete. 2024-01-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.851+0000 7f39f48f4700 1 Processor -- start 2024-01-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.852+0000 7f39f48f4700 1 -- start start 2024-01-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.852+0000 7f39f48f4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f0062ec0 0x7f39f00ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.852+0000 7f39f48f4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39f01540a0 0x7f39f0150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.852+0000 7f39f48f4700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39f01550f0 0x7f39f0166ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.852+0000 7f39f48f4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00ad330 con 0x7f39f0062ec0 2024-01-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.852+0000 7f39f48f4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f01513a0 con 0x7f39f01540a0 2024-01-24T15:32:48.857 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.852+0000 7f39f48f4700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00aac20 con 0x7f39f01550f0 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.852+0000 7f39ee59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f0062ec0 0x7f39f00ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.852+0000 7f39ee59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f0062ec0 0x7f39f00ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52502/0 (socket says 172.21.15.29:52502) 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.852+0000 7f39ee59c700 1 -- 172.21.15.29:0/978184032 learned_addr learned my addr 172.21.15.29:0/978184032 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39edd9b700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39f01540a0 0x7f39f0150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39eed9d700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39f01550f0 0x7f39f0166ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39ee59c700 1 -- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39f01550f0 msgr2=0x7f39f0166ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39ee59c700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39f01550f0 0x7f39f0166ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39ee59c700 1 -- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39f01540a0 msgr2=0x7f39f0150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39ee59c700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39f01540a0 0x7f39f0150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39ee59c700 1 -- 172.21.15.29:0/978184032 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0009370 con 0x7f39f0062ec0 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39eed9d700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39f01550f0 0x7f39f0166ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39ee59c700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f0062ec0 0x7f39f00ace40 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 rx=0x7f39f0069030 tx=0x7f39e0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39de7fc700 1 -- 172.21.15.29:0/978184032 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e000b8d0 con 0x7f39f0062ec0 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39de7fc700 1 -- 172.21.15.29:0/978184032 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e000ba30 con 0x7f39f0062ec0 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39f48f4700 1 -- 172.21.15.29:0/978184032 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f00aaf00 con 0x7f39f0062ec0 2024-01-24T15:32:48.858 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39de7fc700 1 -- 172.21.15.29:0/978184032 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e001c670 con 0x7f39f0062ec0 2024-01-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.853+0000 7f39f48f4700 1 -- 172.21.15.29:0/978184032 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f00ab4c0 con 0x7f39f0062ec0 2024-01-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.854+0000 7f39f48f4700 1 -- 172.21.15.29:0/978184032 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f00475e0 con 0x7f39f0062ec0 2024-01-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.855+0000 7f39de7fc700 1 -- 172.21.15.29:0/978184032 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f39e001ca60 con 0x7f39f0062ec0 2024-01-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.855+0000 7f39de7fc700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f39d005f270 0x7f39d0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.855+0000 7f39de7fc700 1 -- 172.21.15.29:0/978184032 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f39e007c330 con 0x7f39f0062ec0 2024-01-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.855+0000 7f39edd9b700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f39d005f270 0x7f39d0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:48.859 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.856+0000 7f39edd9b700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f39d005f270 0x7f39d0061730 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f39f00c6800 tx=0x7f39e4007d30).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:48.861 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:48.859+0000 7f39de7fc700 1 -- 172.21.15.29:0/978184032 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f39e004d160 con 0x7f39f0062ec0 2024-01-24T15:32:49.041 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.039+0000 7f39f48f4700 1 -- 172.21.15.29:0/978184032 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39f006a5f0 con 0x7f39f0062ec0 2024-01-24T15:32:49.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.040+0000 7f39de7fc700 1 -- 172.21.15.29:0/978184032 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v237) v1 ==== 74+0+15898 (secure 0 0 0) 0x7f39e002a090 con 0x7f39f0062ec0 2024-01-24T15:32:49.042 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:49.043 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":237,"fsid":"b1c88d29-1963-443e-8a67-63a822ae75dd","created":"2024-01-24T13:58:00.880204+0000","modified":"2024-01-24T15:32:44.489705+0000","last_up_change":"2024-01-24T13:58:10.738353+0000","last_in_change":"2024-01-24T13:58:03.363720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-01-24T13:58:09.781701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-24T13:58:12.196460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12bc8013-1170-4506-9682-f768047e31e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":43714},{"type":"v1","addr":"172.21.15.29:6825","nonce":43714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":43714},{"type":"v1","addr":"172.21.15.29:6827","nonce":43714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":43714},{"type":"v1","addr":"172.21.15.29:6831","nonce":43714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":43714},{"type":"v1","addr":"172.21.15.29:6829","nonce":43714}]},"public_addr":"172.21.15.29:6825/43714","cluster_addr":"172.21.15.29:6827/43714","heartbeat_back_addr":"172.21.15.29:6831/43714","heartbeat_front_addr":"172.21.15.29:6829/43714","state":["exists","up"]},{"osd":1,"uuid":"998bacf2-ab7f-4cca-8df3-3d88ada9d0d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":43696},{"type":"v1","addr":"172.21.15.29:6801","nonce":43696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":43696},{"type":"v1","addr":"172.21.15.29:6803","nonce":43696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":43696},{"type":"v1","addr":"172.21.15.29:6807","nonce":43696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":43696},{"type":"v1","addr":"172.21.15.29:6805","nonce":43696}]},"public_addr":"172.21.15.29:6801/43696","cluster_addr":"172.21.15.29:6803/43696","heartbeat_back_addr":"172.21.15.29:6807/43696","heartbeat_front_addr":"172.21.15.29:6805/43696","state":["exists","up"]},{"osd":2,"uuid":"f7a975b6-cab3-43be-a47b-178d2b65f18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":43698},{"type":"v1","addr":"172.21.15.29:6817","nonce":43698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":43698},{"type":"v1","addr":"172.21.15.29:6819","nonce":43698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":43698},{"type":"v1","addr":"172.21.15.29:6823","nonce":43698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":43698},{"type":"v1","addr":"172.21.15.29:6821","nonce":43698}]},"public_addr":"172.21.15.29:6817/43698","cluster_addr":"172.21.15.29:6819/43698","heartbeat_back_addr":"172.21.15.29:6823/43698","heartbeat_front_addr":"172.21.15.29:6821/43698","state":["exists","up"]},{"osd":3,"uuid":"9f912c3d-46b4-4cc5-81c5-ebcb5a980ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":43697},{"type":"v1","addr":"172.21.15.29:6809","nonce":43697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":43697},{"type":"v1","addr":"172.21.15.29:6811","nonce":43697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":43697},{"type":"v1","addr":"172.21.15.29:6815","nonce":43697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":43697},{"type":"v1","addr":"172.21.15.29:6813","nonce":43697}]},"public_addr":"172.21.15.29:6809/43697","cluster_addr":"172.21.15.29:6811/43697","heartbeat_back_addr":"172.21.15.29:6815/43697","heartbeat_front_addr":"172.21.15.29:6813/43697","state":["exists","up"]},{"osd":4,"uuid":"a9ba71db-c0f6-4c63-a4e2-cf7f94184237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":43896},{"type":"v1","addr":"172.21.15.163:6801","nonce":43896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":43896},{"type":"v1","addr":"172.21.15.163:6803","nonce":43896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":43896},{"type":"v1","addr":"172.21.15.163:6807","nonce":43896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":43896},{"type":"v1","addr":"172.21.15.163:6805","nonce":43896}]},"public_addr":"172.21.15.163:6801/43896","cluster_addr":"172.21.15.163:6803/43896","heartbeat_back_addr":"172.21.15.163:6807/43896","heartbeat_front_addr":"172.21.15.163:6805/43896","state":["exists","up"]},{"osd":5,"uuid":"3768b6d0-201b-490d-9990-a3312db4d9c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":43926},{"type":"v1","addr":"172.21.15.163:6820","nonce":43926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":43926},{"type":"v1","addr":"172.21.15.163:6825","nonce":43926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":43926},{"type":"v1","addr":"172.21.15.163:6831","nonce":43926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6827","nonce":43926},{"type":"v1","addr":"172.21.15.163:6829","nonce":43926}]},"public_addr":"172.21.15.163:6820/43926","cluster_addr":"172.21.15.163:6825/43926","heartbeat_back_addr":"172.21.15.163:6831/43926","heartbeat_front_addr":"172.21.15.163:6829/43926","state":["exists","up"]},{"osd":6,"uuid":"555eb732-dd59-49c4-a326-e5f75cbb17d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":43897},{"type":"v1","addr":"172.21.15.163:6809","nonce":43897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":43897},{"type":"v1","addr":"172.21.15.163:6811","nonce":43897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":43897},{"type":"v1","addr":"172.21.15.163:6815","nonce":43897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":43897},{"type":"v1","addr":"172.21.15.163:6813","nonce":43897}]},"public_addr":"172.21.15.163:6809/43897","cluster_addr":"172.21.15.163:6811/43897","heartbeat_back_addr":"172.21.15.163:6815/43897","heartbeat_front_addr":"172.21.15.163:6813/43897","state":["exists","up"]},{"osd":7,"uuid":"4935e576-9e41-4af8-9c65-30b80ddb2b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":229,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":43927},{"type":"v1","addr":"172.21.15.163:6818","nonce":43927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6819","nonce":43927},{"type":"v1","addr":"172.21.15.163:6821","nonce":43927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":43927},{"type":"v1","addr":"172.21.15.163:6828","nonce":43927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":43927},{"type":"v1","addr":"172.21.15.163:6824","nonce":43927}]},"public_addr":"172.21.15.163:6818/43927","cluster_addr":"172.21.15.163:6821/43927","heartbeat_back_addr":"172.21.15.163:6828/43927","heartbeat_front_addr":"172.21.15.163:6824/43927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.773618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.752978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.557244+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.523002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.244889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.493701+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.398126+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T13:58:08.494591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/2645482892":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/4093084652":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/3512593770":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/343370470":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/4238824203":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/3387006926":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/3908030498":"2024-01-25T15:32:40.461411+0000","172.21.15.163:0/741220992":"2024-01-25T15:32:33.173546+0000","172.21.15.163:0/2316406386":"2024-01-25T15:32:33.173546+0000","172.21.15.163:6833/57365":"2024-01-25T15:32:33.173546+0000","172.21.15.163:6837/532321390":"2024-01-25T15:32:40.962950+0000","172.21.15.163:6835/532321390":"2024-01-25T15:32:40.962950+0000","172.21.15.163:6832/57365":"2024-01-25T15:32:33.173546+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":3},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":37,"length":1},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":3},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":69,"length":3},{"begin":73,"length":1},{"begin":75,"length":3},{"begin":79,"length":4},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":3},{"begin":94,"length":1},{"begin":96,"length":3},{"begin":100,"length":3},{"begin":104,"length":1},{"begin":106,"length":3},{"begin":110,"length":2},{"begin":152,"length":70}]},{"pool":19,"snaps":[{"begin":20,"length":38},{"begin":66,"length":48}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T15:32:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.042+0000 7f39f48f4700 1 -- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f39d005f270 msgr2=0x7f39d0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.042+0000 7f39f48f4700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f39d005f270 0x7f39d0061730 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f39f00c6800 tx=0x7f39e4007d30).stop 2024-01-24T15:32:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.042+0000 7f39f48f4700 1 -- 172.21.15.29:0/978184032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f0062ec0 msgr2=0x7f39f00ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.042+0000 7f39f48f4700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f0062ec0 0x7f39f00ace40 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 rx=0x7f39f0069030 tx=0x7f39e0017000).stop 2024-01-24T15:32:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.042+0000 7f39f48f4700 1 -- 172.21.15.29:0/978184032 shutdown_connections 2024-01-24T15:32:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.042+0000 7f39f48f4700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f39d005f270 0x7f39d0061730 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.042+0000 7f39f48f4700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39f01550f0 0x7f39f0166ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.042+0000 7f39f48f4700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39f0062ec0 0x7f39f00ace40 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.042+0000 7f39f48f4700 1 --2- 172.21.15.29:0/978184032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39f01540a0 0x7f39f0150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.042+0000 7f39f48f4700 1 -- 172.21.15.29:0/978184032 >> 172.21.15.29:0/978184032 conn(0x7f39f00c85c0 msgr2=0x7f39f00654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.042+0000 7f39f48f4700 1 -- 172.21.15.29:0/978184032 shutdown_connections 2024-01-24T15:32:49.045 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:49.042+0000 7f39f48f4700 1 -- 172.21.15.29:0/978184032 wait complete. 2024-01-24T15:32:50.058 INFO:tasks.ceph:Scrubbing osd.0 2024-01-24T15:32:50.059 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-01-24T15:32:50.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.200+0000 7f017102e700 1 Processor -- start 2024-01-24T15:32:50.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.200+0000 7f017102e700 1 -- start start 2024-01-24T15:32:50.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.201+0000 7f017102e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1517b0 0x7f016c151c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.201+0000 7f017102e700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c069930 con 0x7f016c158950 2024-01-24T15:32:50.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.201+0000 7f017102e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c069dc0 con 0x7f016c1513d0 2024-01-24T15:32:50.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.201+0000 7f017102e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c063890 con 0x7f016c1517b0 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.201+0000 7f016b59e700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f016c158950 0x7f016c15ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53968/0 (socket says 172.21.15.29:53968) 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.201+0000 7f016b59e700 1 -- 172.21.15.29:0/2037239271 learned_addr learned my addr 172.21.15.29:0/2037239271 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f016a59c700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1517b0 0x7f016c151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140476475 0 0) 0x7f016c069930 con 0x7f016c158950 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01540033f0 con 0x7f016c158950 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969590875 0 0) 0x7f016c069dc0 con 0x7f016c1513d0 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f016c069930 con 0x7f016c1513d0 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3661772239 0 0) 0x7f01540033f0 con 0x7f016c158950 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1517b0 msgr2=0x7f016c151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1517b0 0x7f016c151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 >> v1:172.21.15.163:6789/0 conn(0x7f016c1513d0 legacy=0x7f016c063180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 --> v1:172.21.15.29:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c068d20 con 0x7f016c158950 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 <== mon.0 v1:172.21.15.29:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3503764720 0 0) 0x7f01640017f0 con 0x7f016c158950 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 >> v1:172.21.15.29:6789/0 conn(0x7f016c158950 legacy=0x7f016c15ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.207 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0154003aa0 0x7f0154003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0154004690 0x7f0154004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1517b0 0x7f0154005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015400b8e0 con 0x7f0154003aa0 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015400ba50 con 0x7f0154004690 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015400bc10 con 0x7f016c1517b0 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f016ad9d700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0154003aa0 0x7f0154003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f016a59c700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0154004690 0x7f0154004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f016b59e700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1517b0 0x7f0154005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f016ad9d700 1 -- 172.21.15.29:0/2037239271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1517b0 msgr2=0x7f0154005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f016ad9d700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1517b0 0x7f0154005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f016ad9d700 1 -- 172.21.15.29:0/2037239271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0154004690 msgr2=0x7f0154004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f016ad9d700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0154004690 0x7f0154004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f016ad9d700 1 -- 172.21.15.29:0/2037239271 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01540033f0 con 0x7f0154003aa0 2024-01-24T15:32:50.208 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.202+0000 7f016a59c700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0154004690 0x7f0154004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.203+0000 7f016ad9d700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0154003aa0 0x7f0154003ea0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 rx=0x7f0160001890 tx=0x7f0160007bb0).ready entity=mon.0 client_cookie=f890ebe5829e71f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.203+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160007020 con 0x7f0154003aa0 2024-01-24T15:32:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.203+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0160007180 con 0x7f0154003aa0 2024-01-24T15:32:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.203+0000 7f0169d9b700 1 -- 172.21.15.29:0/2037239271 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160007500 con 0x7f0154003aa0 2024-01-24T15:32:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.203+0000 7f017102e700 1 -- 172.21.15.29:0/2037239271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0154003aa0 msgr2=0x7f0154003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.203+0000 7f017102e700 1 --2- 172.21.15.29:0/2037239271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0154003aa0 0x7f0154003ea0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 rx=0x7f0160001890 tx=0x7f0160007bb0).stop 2024-01-24T15:32:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.203+0000 7f016b59e700 1 -- 172.21.15.29:0/2037239271 reap_dead start 2024-01-24T15:32:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.203+0000 7f017102e700 1 -- 172.21.15.29:0/2037239271 shutdown_connections 2024-01-24T15:32:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.203+0000 7f017102e700 1 -- 172.21.15.29:0/2037239271 >> 172.21.15.29:0/2037239271 conn(0x7f016c0c7280 msgr2=0x7f016c0c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.203+0000 7f017102e700 1 -- 172.21.15.29:0/2037239271 shutdown_connections 2024-01-24T15:32:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.203+0000 7f017102e700 1 -- 172.21.15.29:0/2037239271 wait complete. 2024-01-24T15:32:50.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.205+0000 7f017102e700 1 Processor -- start 2024-01-24T15:32:50.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.205+0000 7f017102e700 1 -- start start 2024-01-24T15:32:50.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.206+0000 7f017102e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0154003aa0 0x7f016c154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.206+0000 7f017102e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f016c161e30 0x7f016c148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.206+0000 7f017102e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1653c0 0x7f016c0c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.206+0000 7f017102e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c155ca0 con 0x7f0154003aa0 2024-01-24T15:32:50.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.206+0000 7f017102e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c1552f0 con 0x7f016c161e30 2024-01-24T15:32:50.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.206+0000 7f017102e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c154a50 con 0x7f016c1653c0 2024-01-24T15:32:50.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.206+0000 7f016ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0154003aa0 0x7f016c154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f016ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0154003aa0 0x7f016c154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52508/0 (socket says 172.21.15.29:52508) 2024-01-24T15:32:50.214 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f016ad9d700 1 -- 172.21.15.29:0/2520834647 learned_addr learned my addr 172.21.15.29:0/2520834647 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f016a59c700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f016c161e30 0x7f016c148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f016b59e700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1653c0 0x7f016c0c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f016ad9d700 1 -- 172.21.15.29:0/2520834647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1653c0 msgr2=0x7f016c0c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f016ad9d700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1653c0 0x7f016c0c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f016ad9d700 1 -- 172.21.15.29:0/2520834647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f016c161e30 msgr2=0x7f016c148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f016ad9d700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f016c161e30 0x7f016c148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f016ad9d700 1 -- 172.21.15.29:0/2520834647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c154d30 con 0x7f0154003aa0 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f016b59e700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f016c1653c0 0x7f016c0c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f016ad9d700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0154003aa0 0x7f016c154200 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 rx=0x7f0160001e80 tx=0x7f0160016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f0152ffd700 1 -- 172.21.15.29:0/2520834647 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016001e480 con 0x7f0154003aa0 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f0152ffd700 1 -- 172.21.15.29:0/2520834647 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016001e5e0 con 0x7f0154003aa0 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f017102e700 1 -- 172.21.15.29:0/2520834647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c155010 con 0x7f0154003aa0 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f0152ffd700 1 -- 172.21.15.29:0/2520834647 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016001e9a0 con 0x7f0154003aa0 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.207+0000 7f017102e700 1 -- 172.21.15.29:0/2520834647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c148a40 con 0x7f0154003aa0 2024-01-24T15:32:50.215 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.208+0000 7f017102e700 1 -- 172.21.15.29:0/2520834647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f016c0475e0 con 0x7f0154003aa0 2024-01-24T15:32:50.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.209+0000 7f0152ffd700 1 -- 172.21.15.29:0/2520834647 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f016001ed50 con 0x7f0154003aa0 2024-01-24T15:32:50.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.209+0000 7f0152ffd700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f015805f270 0x7f0158061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.209+0000 7f0152ffd700 1 -- 172.21.15.29:0/2520834647 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f016007b640 con 0x7f0154003aa0 2024-01-24T15:32:50.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.209+0000 7f0152ffd700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] conn(0x7f0158066a90 0x7f0158068f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.209+0000 7f0152ffd700 1 -- 172.21.15.29:0/2520834647 --> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0158069510 con 0x7f0158066a90 2024-01-24T15:32:50.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.209+0000 7f0152ffd700 1 -- 172.21.15.29:0/2520834647 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=237) v2 ==== 24+0+0 (secure 0 0 0) 0x7f016007dc90 con 0x7f0154003aa0 2024-01-24T15:32:50.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.209+0000 7f016a59c700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f015805f270 0x7f0158061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.210+0000 7f016b59e700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] conn(0x7f0158066a90 0x7f0158068f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.210+0000 7f016b59e700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] conn(0x7f0158066a90 0x7f0158068f30 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.210+0000 7f016a59c700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f015805f270 0x7f0158061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f016c150380 tx=0x7f015c008040).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.216 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.211+0000 7f0152ffd700 1 -- 172.21.15.29:0/2520834647 <== osd.0 v2:172.21.15.29:6824/43714 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0158069510 con 0x7f0158066a90 2024-01-24T15:32:50.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.225+0000 7f017102e700 1 -- 172.21.15.29:0/2520834647 --> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f016c155f40 con 0x7f0158066a90 2024-01-24T15:32:50.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.225+0000 7f0152ffd700 1 -- 172.21.15.29:0/2520834647 <== osd.0 v2:172.21.15.29:6824/43714 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f016c155f40 con 0x7f0158066a90 2024-01-24T15:32:50.227 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:32:50.228 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2024-01-24T15:32:50.228 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:32:50.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.225+0000 7f017102e700 1 -- 172.21.15.29:0/2520834647 >> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] conn(0x7f0158066a90 msgr2=0x7f0158068f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.225+0000 7f017102e700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.29:6824/43714,v1:172.21.15.29:6825/43714] conn(0x7f0158066a90 0x7f0158068f30 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.226+0000 7f017102e700 1 -- 172.21.15.29:0/2520834647 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f015805f270 msgr2=0x7f0158061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.226+0000 7f017102e700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f015805f270 0x7f0158061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f016c150380 tx=0x7f015c008040).stop 2024-01-24T15:32:50.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.226+0000 7f017102e700 1 -- 172.21.15.29:0/2520834647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0154003aa0 msgr2=0x7f016c154200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.226+0000 7f017102e700 1 --2- 172.21.15.29:0/2520834647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0154003aa0 0x7f016c154200 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 rx=0x7f0160001e80 tx=0x7f0160016000).stop 2024-01-24T15:32:50.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.226+0000 7f016b59e700 1 -- 172.21.15.29:0/2520834647 reap_dead start 2024-01-24T15:32:50.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.226+0000 7f017102e700 1 -- 172.21.15.29:0/2520834647 shutdown_connections 2024-01-24T15:32:50.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.226+0000 7f017102e700 1 -- 172.21.15.29:0/2520834647 >> 172.21.15.29:0/2520834647 conn(0x7f016c0c7280 msgr2=0x7f016c1508d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:50.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.226+0000 7f017102e700 1 -- 172.21.15.29:0/2520834647 shutdown_connections 2024-01-24T15:32:50.229 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.226+0000 7f017102e700 1 -- 172.21.15.29:0/2520834647 wait complete. 2024-01-24T15:32:50.241 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-01-24T15:32:50.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.370+0000 7f8181605700 1 Processor -- start 2024-01-24T15:32:50.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.370+0000 7f8181605700 1 -- start start 2024-01-24T15:32:50.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f8181605700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f817c154240 0x7f817c064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f8181605700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f817c0a9d20 0x7f817c0aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f8181605700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f817c157cf0 0x7f817c158140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f8181605700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817c1550e0 con 0x7f817c154240 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f8181605700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817c0aa670 con 0x7f817c0a9d20 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f8181605700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817c0650d0 con 0x7f817c157cf0 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f817affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f817c154240 0x7f817c064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f817affd700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f817c154240 0x7f817c064780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52518/0 (socket says 172.21.15.29:52518) 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f817affd700 1 -- 172.21.15.29:0/634786843 learned_addr learned my addr 172.21.15.29:0/634786843 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f817b7fe700 1 --2- 172.21.15.29:0/634786843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f817c157cf0 0x7f817c158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f817a7fc700 1 --2- 172.21.15.29:0/634786843 >> v2:172.21.15.163:3300/0 conn(0x7f817c0a9d20 0x7f817c0aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f817affd700 1 -- 172.21.15.29:0/634786843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f817c157cf0 msgr2=0x7f817c158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.371+0000 7f817affd700 1 --2- 172.21.15.29:0/634786843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f817c157cf0 0x7f817c158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f817affd700 1 -- 172.21.15.29:0/634786843 >> v2:172.21.15.163:3300/0 conn(0x7f817c0a9d20 msgr2=0x7f817c0aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:50.375 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f817affd700 1 --2- 172.21.15.29:0/634786843 >> v2:172.21.15.163:3300/0 conn(0x7f817c0a9d20 0x7f817c0aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f817affd700 1 -- 172.21.15.29:0/634786843 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c154a90 con 0x7f817c154240 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f817affd700 1 --2- 172.21.15.29:0/634786843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f817c154240 0x7f817c064780 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 rx=0x7f817c084090 tx=0x7f816c0090f0).ready entity=mon.0 client_cookie=dcc52795b442a64e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f8179ffb700 1 -- 172.21.15.29:0/634786843 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c00f510 con 0x7f817c154240 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f8179ffb700 1 -- 172.21.15.29:0/634786843 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c00f670 con 0x7f817c154240 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f8179ffb700 1 -- 172.21.15.29:0/634786843 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c00fa10 con 0x7f817c154240 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f8181605700 1 -- 172.21.15.29:0/634786843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f817c154240 msgr2=0x7f817c064780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f8181605700 1 --2- 172.21.15.29:0/634786843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f817c154240 0x7f817c064780 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 rx=0x7f817c084090 tx=0x7f816c0090f0).stop 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f8181605700 1 -- 172.21.15.29:0/634786843 shutdown_connections 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f8181605700 1 --2- 172.21.15.29:0/634786843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f817c157cf0 0x7f817c158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f8181605700 1 --2- 172.21.15.29:0/634786843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f817c154240 0x7f817c064780 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f8181605700 1 --2- 172.21.15.29:0/634786843 >> v2:172.21.15.163:3300/0 conn(0x7f817c0a9d20 0x7f817c0aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f8181605700 1 -- 172.21.15.29:0/634786843 >> 172.21.15.29:0/634786843 conn(0x7f817c0c8310 msgr2=0x7f817c05e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.372+0000 7f8181605700 1 -- 172.21.15.29:0/634786843 shutdown_connections 2024-01-24T15:32:50.376 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.373+0000 7f8181605700 1 -- 172.21.15.29:0/634786843 wait complete. 2024-01-24T15:32:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.374+0000 7f8181605700 1 Processor -- start 2024-01-24T15:32:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.374+0000 7f8181605700 1 -- start start 2024-01-24T15:32:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f8181605700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f817c0a9d20 0x7f817c1645c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f8181605700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f817c154240 0x7f817c0c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f8181605700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f817c157cf0 0x7f817c0c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f8181605700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c0af8c0 con 0x7f817c157cf0 2024-01-24T15:32:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f8181605700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c0c50c0 con 0x7f817c0a9d20 2024-01-24T15:32:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f8181605700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817c0aef90 con 0x7f817c154240 2024-01-24T15:32:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f817b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f817c157cf0 0x7f817c0c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f817b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f817c157cf0 0x7f817c0c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52530/0 (socket says 172.21.15.29:52530) 2024-01-24T15:32:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f817b7fe700 1 -- 172.21.15.29:0/1114484597 learned_addr learned my addr 172.21.15.29:0/1114484597 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:50.379 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f817affd700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f817c0a9d20 0x7f817c1645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f817a7fc700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f817c154240 0x7f817c0c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f817b7fe700 1 -- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f817c154240 msgr2=0x7f817c0c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f817b7fe700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f817c154240 0x7f817c0c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f817b7fe700 1 -- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f817c0a9d20 msgr2=0x7f817c1645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f817b7fe700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f817c0a9d20 0x7f817c1645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f817b7fe700 1 -- 172.21.15.29:0/1114484597 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c009370 con 0x7f817c157cf0 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.375+0000 7f817b7fe700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f817c157cf0 0x7f817c0c45e0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 rx=0x7f8174009df0 tx=0x7f817400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.376+0000 7f8172ffd700 1 -- 172.21.15.29:0/1114484597 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817400caa0 con 0x7f817c157cf0 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.376+0000 7f8172ffd700 1 -- 172.21.15.29:0/1114484597 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817400f040 con 0x7f817c157cf0 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.376+0000 7f8181605700 1 -- 172.21.15.29:0/1114484597 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817c0af270 con 0x7f817c157cf0 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.376+0000 7f8172ffd700 1 -- 172.21.15.29:0/1114484597 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817400ce10 con 0x7f817c157cf0 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.376+0000 7f8181605700 1 -- 172.21.15.29:0/1114484597 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817c164ab0 con 0x7f817c157cf0 2024-01-24T15:32:50.380 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.377+0000 7f8181605700 1 -- 172.21.15.29:0/1114484597 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817c0475e0 con 0x7f817c157cf0 2024-01-24T15:32:50.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.377+0000 7f8172ffd700 1 -- 172.21.15.29:0/1114484597 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8174013610 con 0x7f817c157cf0 2024-01-24T15:32:50.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.378+0000 7f8172ffd700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f815c05f270 0x7f815c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.378+0000 7f8172ffd700 1 -- 172.21.15.29:0/1114484597 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f817407add0 con 0x7f817c157cf0 2024-01-24T15:32:50.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.378+0000 7f817affd700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f815c05f270 0x7f815c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.381 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.378+0000 7f817affd700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f815c05f270 0x7f815c061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f817c084090 tx=0x7f816c017000).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.384 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.382+0000 7f8172ffd700 1 -- 172.21.15.29:0/1114484597 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f817404bc00 con 0x7f817c157cf0 2024-01-24T15:32:50.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.564+0000 7f8181605700 1 -- 172.21.15.29:0/1114484597 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f817c154d70 con 0x7f815c05f270 2024-01-24T15:32:50.566 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.564+0000 7f8172ffd700 1 -- 172.21.15.29:0/1114484597 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f817c154d70 con 0x7f815c05f270 2024-01-24T15:32:50.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.566+0000 7f8181605700 1 -- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f815c05f270 msgr2=0x7f815c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.566+0000 7f8181605700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f815c05f270 0x7f815c061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f817c084090 tx=0x7f816c017000).stop 2024-01-24T15:32:50.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.566+0000 7f8181605700 1 -- 172.21.15.29:0/1114484597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f817c157cf0 msgr2=0x7f817c0c45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.566+0000 7f8181605700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f817c157cf0 0x7f817c0c45e0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 rx=0x7f8174009df0 tx=0x7f817400b040).stop 2024-01-24T15:32:50.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.566+0000 7f8181605700 1 -- 172.21.15.29:0/1114484597 shutdown_connections 2024-01-24T15:32:50.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.566+0000 7f8181605700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f817c157cf0 0x7f817c0c45e0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.566+0000 7f8181605700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f817c0a9d20 0x7f817c1645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.566+0000 7f8181605700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f815c05f270 0x7f815c061730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.566+0000 7f8181605700 1 --2- 172.21.15.29:0/1114484597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f817c154240 0x7f817c0c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.566+0000 7f8181605700 1 -- 172.21.15.29:0/1114484597 >> 172.21.15.29:0/1114484597 conn(0x7f817c0c8310 msgr2=0x7f817c1534d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:50.569 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.566+0000 7f8181605700 1 -- 172.21.15.29:0/1114484597 shutdown_connections 2024-01-24T15:32:50.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.567+0000 7f8181605700 1 -- 172.21.15.29:0/1114484597 wait complete. 2024-01-24T15:32:50.570 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 0 to deep-scrub 2024-01-24T15:32:50.582 INFO:tasks.ceph:Scrubbing osd.1 2024-01-24T15:32:50.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-01-24T15:32:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.713+0000 7fa372840700 1 Processor -- start 2024-01-24T15:32:50.716 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.713+0000 7fa372840700 1 -- start start 2024-01-24T15:32:50.717 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.714+0000 7fa372840700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa36c062970 0x7fa36c062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.714+0000 7fa372840700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36c0c3e00 con 0x7fa36c158720 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.714+0000 7fa372840700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36c0c2900 con 0x7fa36c0c2190 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.714+0000 7fa372840700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36c065720 con 0x7fa36c062970 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa370ddd700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa36c158720 0x7fa36c0a4890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53982/0 (socket says 172.21.15.29:53982) 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa370ddd700 1 -- 172.21.15.29:0/366529055 learned_addr learned my addr 172.21.15.29:0/366529055 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa36bfff700 1 --2- 172.21.15.29:0/366529055 >> v2:172.21.15.163:3300/0 conn(0x7fa36c062970 0x7fa36c062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa36affd700 1 -- 172.21.15.29:0/366529055 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689003387 0 0) 0x7fa36c0c3e00 con 0x7fa36c158720 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa36affd700 1 -- 172.21.15.29:0/366529055 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3540033f0 con 0x7fa36c158720 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa36bfff700 1 -- 172.21.15.29:0/366529055 >> v1:172.21.15.163:6789/0 conn(0x7fa36c0c2190 legacy=0x7fa36c0c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa36affd700 1 -- 172.21.15.29:0/366529055 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760350288 0 0) 0x7fa36c0c2900 con 0x7fa36c0c2190 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa36bfff700 1 -- 172.21.15.29:0/366529055 >> v1:172.21.15.29:6789/0 conn(0x7fa36c158720 legacy=0x7fa36c0a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa36bfff700 1 -- 172.21.15.29:0/366529055 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c158bd0 con 0x7fa36c062970 2024-01-24T15:32:50.718 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa36bfff700 1 --2- 172.21.15.29:0/366529055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36c062970 0x7fa36c062d50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fa35c009760 tx=0x7fa35c009060).ready entity=mon.1 client_cookie=62551965f899f50 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa36affd700 1 -- 172.21.15.29:0/366529055 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00f510 con 0x7fa36c062970 2024-01-24T15:32:50.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa36affd700 1 -- 172.21.15.29:0/366529055 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c00f670 con 0x7fa36c062970 2024-01-24T15:32:50.719 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.715+0000 7fa36affd700 1 -- 172.21.15.29:0/366529055 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00f9f0 con 0x7fa36c062970 2024-01-24T15:32:50.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.716+0000 7fa372840700 1 -- 172.21.15.29:0/366529055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36c062970 msgr2=0x7fa36c062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.716+0000 7fa372840700 1 --2- 172.21.15.29:0/366529055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36c062970 0x7fa36c062d50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fa35c009760 tx=0x7fa35c009060).stop 2024-01-24T15:32:50.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.716+0000 7fa372840700 1 -- 172.21.15.29:0/366529055 shutdown_connections 2024-01-24T15:32:50.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.716+0000 7fa372840700 1 --2- 172.21.15.29:0/366529055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36c062970 0x7fa36c062d50 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.716+0000 7fa372840700 1 -- 172.21.15.29:0/366529055 >> 172.21.15.29:0/366529055 conn(0x7fa36c067b20 msgr2=0x7fa36c0638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:50.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.716+0000 7fa372840700 1 -- 172.21.15.29:0/366529055 shutdown_connections 2024-01-24T15:32:50.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.716+0000 7fa372840700 1 -- 172.21.15.29:0/366529055 wait complete. 2024-01-24T15:32:50.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.717+0000 7fa372840700 1 Processor -- start 2024-01-24T15:32:50.720 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.718+0000 7fa372840700 1 -- start start 2024-01-24T15:32:50.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.718+0000 7fa372840700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa36c062970 0x7fa36c156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.718+0000 7fa372840700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa36c158720 0x7fa36c0a9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.718+0000 7fa372840700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36c1570e0 0x7fa36c0aa120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.722 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa372840700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c0a4930 con 0x7fa36c062970 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa372840700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c0a4bd0 con 0x7fa36c1570e0 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa372840700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c0a4eb0 con 0x7fa36c158720 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa36c062970 0x7fa36c156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa36c062970 0x7fa36c156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52540/0 (socket says 172.21.15.29:52540) 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa36bfff700 1 -- 172.21.15.29:0/2691843097 learned_addr learned my addr 172.21.15.29:0/2691843097 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa370ddd700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36c1570e0 0x7fa36c0aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa36b7fe700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa36c158720 0x7fa36c0a9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa36bfff700 1 -- 172.21.15.29:0/2691843097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa36c158720 msgr2=0x7fa36c0a9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa36bfff700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa36c158720 0x7fa36c0a9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa36bfff700 1 -- 172.21.15.29:0/2691843097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36c1570e0 msgr2=0x7fa36c0aa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa36bfff700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36c1570e0 0x7fa36c0aa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa36bfff700 1 -- 172.21.15.29:0/2691843097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0092a0 con 0x7fa36c062970 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa36bfff700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa36c062970 0x7fa36c156bf0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 rx=0x7fa35c009e00 tx=0x7fa35c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.723 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa353fff700 1 -- 172.21.15.29:0/2691843097 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00f7e0 con 0x7fa36c062970 2024-01-24T15:32:50.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa353fff700 1 -- 172.21.15.29:0/2691843097 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c00f940 con 0x7fa36c062970 2024-01-24T15:32:50.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa372840700 1 -- 172.21.15.29:0/2691843097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36c0a5190 con 0x7fa36c062970 2024-01-24T15:32:50.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa353fff700 1 -- 172.21.15.29:0/2691843097 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c01b660 con 0x7fa36c062970 2024-01-24T15:32:50.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.719+0000 7fa372840700 1 -- 172.21.15.29:0/2691843097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36c0aa610 con 0x7fa36c062970 2024-01-24T15:32:50.724 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.720+0000 7fa372840700 1 -- 172.21.15.29:0/2691843097 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa36c0475e0 con 0x7fa36c062970 2024-01-24T15:32:50.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.721+0000 7fa353fff700 1 -- 172.21.15.29:0/2691843097 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa35c01b9d0 con 0x7fa36c062970 2024-01-24T15:32:50.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.721+0000 7fa353fff700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa35805f270 0x7fa358061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.721+0000 7fa353fff700 1 -- 172.21.15.29:0/2691843097 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fa35c07c0b0 con 0x7fa36c062970 2024-01-24T15:32:50.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.721+0000 7fa353fff700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] conn(0x7fa358066a90 0x7fa358068f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.725 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.721+0000 7fa353fff700 1 -- 172.21.15.29:0/2691843097 --> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa358069510 con 0x7fa358066a90 2024-01-24T15:32:50.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.721+0000 7fa353fff700 1 -- 172.21.15.29:0/2691843097 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=237) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa35c07e700 con 0x7fa36c062970 2024-01-24T15:32:50.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.721+0000 7fa36b7fe700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa35805f270 0x7fa358061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.722+0000 7fa370ddd700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] conn(0x7fa358066a90 0x7fa358068f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.722+0000 7fa36b7fe700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa35805f270 0x7fa358061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fa36c0a7af0 tx=0x7fa360009040).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.722+0000 7fa370ddd700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] conn(0x7fa358066a90 0x7fa358068f30 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.726 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.723+0000 7fa353fff700 1 -- 172.21.15.29:0/2691843097 <== osd.1 v2:172.21.15.29:6800/43696 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa358069510 con 0x7fa358066a90 2024-01-24T15:32:50.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.737+0000 7fa372840700 1 -- 172.21.15.29:0/2691843097 --> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa36c0646d0 con 0x7fa358066a90 2024-01-24T15:32:50.739 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.737+0000 7fa353fff700 1 -- 172.21.15.29:0/2691843097 <== osd.1 v2:172.21.15.29:6800/43696 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa36c0646d0 con 0x7fa358066a90 2024-01-24T15:32:50.739 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:32:50.739 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2024-01-24T15:32:50.739 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:32:50.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.737+0000 7fa372840700 1 -- 172.21.15.29:0/2691843097 >> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] conn(0x7fa358066a90 msgr2=0x7fa358068f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.737+0000 7fa372840700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.29:6800/43696,v1:172.21.15.29:6801/43696] conn(0x7fa358066a90 0x7fa358068f30 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.737+0000 7fa372840700 1 -- 172.21.15.29:0/2691843097 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa35805f270 msgr2=0x7fa358061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.737+0000 7fa372840700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa35805f270 0x7fa358061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fa36c0a7af0 tx=0x7fa360009040).stop 2024-01-24T15:32:50.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.737+0000 7fa372840700 1 -- 172.21.15.29:0/2691843097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa36c062970 msgr2=0x7fa36c156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.737+0000 7fa372840700 1 --2- 172.21.15.29:0/2691843097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa36c062970 0x7fa36c156bf0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 rx=0x7fa35c009e00 tx=0x7fa35c017000).stop 2024-01-24T15:32:50.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.737+0000 7fa370ddd700 1 -- 172.21.15.29:0/2691843097 reap_dead start 2024-01-24T15:32:50.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.737+0000 7fa372840700 1 -- 172.21.15.29:0/2691843097 shutdown_connections 2024-01-24T15:32:50.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.737+0000 7fa372840700 1 -- 172.21.15.29:0/2691843097 >> 172.21.15.29:0/2691843097 conn(0x7fa36c067b20 msgr2=0x7fa36c152250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:50.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.738+0000 7fa372840700 1 -- 172.21.15.29:0/2691843097 shutdown_connections 2024-01-24T15:32:50.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.738+0000 7fa372840700 1 -- 172.21.15.29:0/2691843097 wait complete. 2024-01-24T15:32:50.753 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-01-24T15:32:50.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.881+0000 7fc87d35a700 1 Processor -- start 2024-01-24T15:32:50.883 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.881+0000 7fc87d35a700 1 -- start start 2024-01-24T15:32:50.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc87d35a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc878151de0 0x7fc878155470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc87d35a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8780638c0 0x7fc8780664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc87d35a700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc878158800 con 0x7fc878066990 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc87d35a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc878066e20 con 0x7fc878151de0 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc87d35a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc878157ed0 con 0x7fc8780638c0 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc8777fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fc878066990 0x7fc87813d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:53996/0 (socket says 172.21.15.29:53996) 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc8777fe700 1 -- 172.21.15.29:0/903794495 learned_addr learned my addr 172.21.15.29:0/903794495 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc876ffd700 1 --2- 172.21.15.29:0/903794495 >> v2:172.21.15.163:3300/0 conn(0x7fc878151de0 0x7fc878155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc8767fc700 1 --2- 172.21.15.29:0/903794495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8780638c0 0x7fc8780664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc875ffb700 1 -- 172.21.15.29:0/903794495 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929969337 0 0) 0x7fc878158800 con 0x7fc878066990 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc875ffb700 1 -- 172.21.15.29:0/903794495 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8600033f0 con 0x7fc878066990 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc8767fc700 1 -- 172.21.15.29:0/903794495 >> v2:172.21.15.163:3300/0 conn(0x7fc878151de0 msgr2=0x7fc878155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc8767fc700 1 --2- 172.21.15.29:0/903794495 >> v2:172.21.15.163:3300/0 conn(0x7fc878151de0 0x7fc878155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc8767fc700 1 -- 172.21.15.29:0/903794495 >> v1:172.21.15.29:6789/0 conn(0x7fc878066990 legacy=0x7fc87813d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc8767fc700 1 -- 172.21.15.29:0/903794495 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8781581b0 con 0x7fc8780638c0 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.882+0000 7fc876ffd700 1 --2- 172.21.15.29:0/903794495 >> v2:172.21.15.163:3300/0 conn(0x7fc878151de0 0x7fc878155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.883+0000 7fc8767fc700 1 --2- 172.21.15.29:0/903794495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8780638c0 0x7fc8780664a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fc86c009760 tx=0x7fc86c009060).ready entity=mon.2 client_cookie=2c465a79cca30455 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.883+0000 7fc875ffb700 1 -- 172.21.15.29:0/903794495 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00d4a0 con 0x7fc8780638c0 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.883+0000 7fc875ffb700 1 -- 172.21.15.29:0/903794495 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c00d600 con 0x7fc8780638c0 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.883+0000 7fc875ffb700 1 -- 172.21.15.29:0/903794495 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00d980 con 0x7fc8780638c0 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.883+0000 7fc87d35a700 1 -- 172.21.15.29:0/903794495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8780638c0 msgr2=0x7fc8780664a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.883+0000 7fc87d35a700 1 --2- 172.21.15.29:0/903794495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8780638c0 0x7fc8780664a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fc86c009760 tx=0x7fc86c009060).stop 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.883+0000 7fc87d35a700 1 -- 172.21.15.29:0/903794495 shutdown_connections 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.883+0000 7fc87d35a700 1 --2- 172.21.15.29:0/903794495 >> v2:172.21.15.163:3300/0 conn(0x7fc878151de0 0x7fc878155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.883+0000 7fc87d35a700 1 --2- 172.21.15.29:0/903794495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8780638c0 0x7fc8780664a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.883+0000 7fc87d35a700 1 -- 172.21.15.29:0/903794495 >> 172.21.15.29:0/903794495 conn(0x7fc8780c5640 msgr2=0x7fc8780c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.883+0000 7fc87d35a700 1 -- 172.21.15.29:0/903794495 shutdown_connections 2024-01-24T15:32:50.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.883+0000 7fc87d35a700 1 -- 172.21.15.29:0/903794495 wait complete. 2024-01-24T15:32:50.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.885+0000 7fc87d35a700 1 Processor -- start 2024-01-24T15:32:50.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.885+0000 7fc87d35a700 1 -- start start 2024-01-24T15:32:50.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc87d35a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8780638c0 0x7fc878164640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc87d35a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc878066990 0x7fc8780ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc87d35a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc878151de0 0x7fc8780ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc87d35a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780ac850 con 0x7fc8780638c0 2024-01-24T15:32:50.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc87d35a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8781658a0 con 0x7fc878066990 2024-01-24T15:32:50.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc87d35a700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc878164c20 con 0x7fc878151de0 2024-01-24T15:32:50.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc876ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8780638c0 0x7fc878164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc876ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8780638c0 0x7fc878164640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52546/0 (socket says 172.21.15.29:52546) 2024-01-24T15:32:50.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc876ffd700 1 -- 172.21.15.29:0/4212942482 learned_addr learned my addr 172.21.15.29:0/4212942482 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:50.890 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc8767fc700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc878066990 0x7fc8780ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc8777fe700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc878151de0 0x7fc8780ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc876ffd700 1 -- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc878151de0 msgr2=0x7fc8780ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc876ffd700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc878151de0 0x7fc8780ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc876ffd700 1 -- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc878066990 msgr2=0x7fc8780ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc876ffd700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc878066990 0x7fc8780ac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc876ffd700 1 -- 172.21.15.29:0/4212942482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680092a0 con 0x7fc8780638c0 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc876ffd700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8780638c0 0x7fc878164640 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 rx=0x7fc86800e980 tx=0x7fc868009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc85effd700 1 -- 172.21.15.29:0/4212942482 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868012600 con 0x7fc8780638c0 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc85effd700 1 -- 172.21.15.29:0/4212942482 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc868012760 con 0x7fc8780638c0 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc87d35a700 1 -- 172.21.15.29:0/4212942482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c0092a0 con 0x7fc8780638c0 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc85effd700 1 -- 172.21.15.29:0/4212942482 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868012ae0 con 0x7fc8780638c0 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.886+0000 7fc87d35a700 1 -- 172.21.15.29:0/4212942482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8781651e0 con 0x7fc8780638c0 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.888+0000 7fc87d35a700 1 -- 172.21.15.29:0/4212942482 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8780475e0 con 0x7fc8780638c0 2024-01-24T15:32:50.891 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.888+0000 7fc85effd700 1 -- 172.21.15.29:0/4212942482 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc86801b400 con 0x7fc8780638c0 2024-01-24T15:32:50.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.888+0000 7fc85effd700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc86405f2a0 0x7fc864061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:50.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.888+0000 7fc85effd700 1 -- 172.21.15.29:0/4212942482 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fc86807ab50 con 0x7fc8780638c0 2024-01-24T15:32:50.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.889+0000 7fc8767fc700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc86405f2a0 0x7fc864061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:50.892 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.889+0000 7fc8767fc700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc86405f2a0 0x7fc864061760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc86c00d260 tx=0x7fc86c016000).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:50.894 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:50.893+0000 7fc85effd700 1 -- 172.21.15.29:0/4212942482 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc86804b920 con 0x7fc8780638c0 2024-01-24T15:32:51.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.072+0000 7fc87d35a700 1 -- 172.21.15.29:0/4212942482 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8780665d0 con 0x7fc86405f2a0 2024-01-24T15:32:51.074 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.073+0000 7fc85effd700 1 -- 172.21.15.29:0/4212942482 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc8780665d0 con 0x7fc86405f2a0 2024-01-24T15:32:51.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.074+0000 7fc87d35a700 1 -- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc86405f2a0 msgr2=0x7fc864061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.074+0000 7fc87d35a700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc86405f2a0 0x7fc864061760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc86c00d260 tx=0x7fc86c016000).stop 2024-01-24T15:32:51.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.074+0000 7fc87d35a700 1 -- 172.21.15.29:0/4212942482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8780638c0 msgr2=0x7fc878164640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.074+0000 7fc87d35a700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8780638c0 0x7fc878164640 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 rx=0x7fc86800e980 tx=0x7fc868009060).stop 2024-01-24T15:32:51.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.074+0000 7fc87d35a700 1 -- 172.21.15.29:0/4212942482 shutdown_connections 2024-01-24T15:32:51.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.074+0000 7fc87d35a700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc86405f2a0 0x7fc864061760 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.074+0000 7fc87d35a700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc878151de0 0x7fc8780ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.074+0000 7fc87d35a700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8780638c0 0x7fc878164640 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.074+0000 7fc87d35a700 1 --2- 172.21.15.29:0/4212942482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc878066990 0x7fc8780ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.074+0000 7fc87d35a700 1 -- 172.21.15.29:0/4212942482 >> 172.21.15.29:0/4212942482 conn(0x7fc8780c5640 msgr2=0x7fc8780c7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:51.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.075+0000 7fc87d35a700 1 -- 172.21.15.29:0/4212942482 shutdown_connections 2024-01-24T15:32:51.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.075+0000 7fc87d35a700 1 -- 172.21.15.29:0/4212942482 wait complete. 2024-01-24T15:32:51.078 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 1 to deep-scrub 2024-01-24T15:32:51.090 INFO:tasks.ceph:Scrubbing osd.2 2024-01-24T15:32:51.090 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-01-24T15:32:51.223 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.221+0000 7ff06e044700 1 Processor -- start 2024-01-24T15:32:51.224 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.221+0000 7ff06e044700 1 -- start start 2024-01-24T15:32:51.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.222+0000 7ff06e044700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff06813a6f0 0x7ff068068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.222+0000 7ff06e044700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0680bf870 0x7ff06815de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.222+0000 7ff06e044700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0680643f0 con 0x7ff0680638c0 2024-01-24T15:32:51.225 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.222+0000 7ff06e044700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0680bdb30 con 0x7ff06813a6f0 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.222+0000 7ff06e044700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680be3d0 con 0x7ff0680bf870 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.222+0000 7ff0677fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff06813a6f0 0x7ff068068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.222+0000 7ff0677fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff06813a6f0 0x7ff068068660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52550/0 (socket says 172.21.15.29:52550) 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.222+0000 7ff0677fe700 1 -- 172.21.15.29:0/2422564818 learned_addr learned my addr 172.21.15.29:0/2422564818 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.223+0000 7ff067fff700 1 --2- 172.21.15.29:0/2422564818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0680bf870 0x7ff06815de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.223+0000 7ff0677fe700 1 -- 172.21.15.29:0/2422564818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0680bf870 msgr2=0x7ff06815de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.223+0000 7ff0677fe700 1 --2- 172.21.15.29:0/2422564818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0680bf870 0x7ff06815de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.223+0000 7ff0677fe700 1 -- 172.21.15.29:0/2422564818 >> v1:172.21.15.29:6789/0 conn(0x7ff0680638c0 legacy=0x7ff068066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.223+0000 7ff0677fe700 1 -- 172.21.15.29:0/2422564818 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff068064110 con 0x7ff06813a6f0 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.223+0000 7ff0677fe700 1 --2- 172.21.15.29:0/2422564818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff06813a6f0 0x7ff068068660 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 rx=0x7ff058006fd0 tx=0x7ff0580090f0).ready entity=mon.0 client_cookie=2f1ce43ce725e442 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.223+0000 7ff0667fc700 1 -- 172.21.15.29:0/2422564818 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05800b580 con 0x7ff06813a6f0 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.223+0000 7ff0667fc700 1 -- 172.21.15.29:0/2422564818 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05800b6e0 con 0x7ff06813a6f0 2024-01-24T15:32:51.226 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.223+0000 7ff0667fc700 1 -- 172.21.15.29:0/2422564818 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05800ba60 con 0x7ff06813a6f0 2024-01-24T15:32:51.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.224+0000 7ff06e044700 1 -- 172.21.15.29:0/2422564818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff06813a6f0 msgr2=0x7ff068068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.224+0000 7ff06e044700 1 --2- 172.21.15.29:0/2422564818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff06813a6f0 0x7ff068068660 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 rx=0x7ff058006fd0 tx=0x7ff0580090f0).stop 2024-01-24T15:32:51.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.224+0000 7ff06e044700 1 -- 172.21.15.29:0/2422564818 shutdown_connections 2024-01-24T15:32:51.227 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.224+0000 7ff06e044700 1 --2- 172.21.15.29:0/2422564818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0680bf870 0x7ff06815de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.224+0000 7ff06e044700 1 --2- 172.21.15.29:0/2422564818 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff06813a6f0 0x7ff068068660 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.224+0000 7ff06e044700 1 -- 172.21.15.29:0/2422564818 >> 172.21.15.29:0/2422564818 conn(0x7ff0680c8bd0 msgr2=0x7ff0680c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:51.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.224+0000 7ff06e044700 1 -- 172.21.15.29:0/2422564818 shutdown_connections 2024-01-24T15:32:51.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.224+0000 7ff06e044700 1 -- 172.21.15.29:0/2422564818 wait complete. 2024-01-24T15:32:51.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.226+0000 7ff06e044700 1 Processor -- start 2024-01-24T15:32:51.228 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.226+0000 7ff06e044700 1 -- start start 2024-01-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff06e044700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0680638c0 0x7ff068152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff06e044700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0680bf870 0x7ff0680ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff06e044700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff06813a6f0 0x7ff0680adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff06e044700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680a4a20 con 0x7ff0680638c0 2024-01-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff06e044700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680a4d00 con 0x7ff06813a6f0 2024-01-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff06e044700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680a4fe0 con 0x7ff0680bf870 2024-01-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff0677fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0680638c0 0x7ff068152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff0677fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0680638c0 0x7ff068152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52560/0 (socket says 172.21.15.29:52560) 2024-01-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff0677fe700 1 -- 172.21.15.29:0/1187539221 learned_addr learned my addr 172.21.15.29:0/1187539221 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff067fff700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff06813a6f0 0x7ff0680adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff066ffd700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0680bf870 0x7ff0680ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.231 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff0677fe700 1 -- 172.21.15.29:0/1187539221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0680bf870 msgr2=0x7ff0680ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff0677fe700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0680bf870 0x7ff0680ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff0677fe700 1 -- 172.21.15.29:0/1187539221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff06813a6f0 msgr2=0x7ff0680adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff0677fe700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff06813a6f0 0x7ff0680adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff0677fe700 1 -- 172.21.15.29:0/1187539221 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff058009370 con 0x7ff0680638c0 2024-01-24T15:32:51.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.227+0000 7ff0677fe700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0680638c0 0x7ff068152990 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 rx=0x7ff068068ff0 tx=0x7ff058017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.228+0000 7ff04f7fe700 1 -- 172.21.15.29:0/1187539221 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058000bc0 con 0x7ff0680638c0 2024-01-24T15:32:51.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.228+0000 7ff04f7fe700 1 -- 172.21.15.29:0/1187539221 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05800b850 con 0x7ff0680638c0 2024-01-24T15:32:51.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.228+0000 7ff06e044700 1 -- 172.21.15.29:0/1187539221 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0680a52c0 con 0x7ff0680638c0 2024-01-24T15:32:51.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.228+0000 7ff04f7fe700 1 -- 172.21.15.29:0/1187539221 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058000d20 con 0x7ff0680638c0 2024-01-24T15:32:51.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.228+0000 7ff06e044700 1 -- 172.21.15.29:0/1187539221 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff068152e80 con 0x7ff0680638c0 2024-01-24T15:32:51.232 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.229+0000 7ff06e044700 1 -- 172.21.15.29:0/1187539221 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff06806ae00 con 0x7ff0680638c0 2024-01-24T15:32:51.233 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.229+0000 7ff04f7fe700 1 -- 172.21.15.29:0/1187539221 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff05801ca90 con 0x7ff0680638c0 2024-01-24T15:32:51.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.230+0000 7ff04f7fe700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff05405f270 0x7ff054061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.230+0000 7ff04f7fe700 1 -- 172.21.15.29:0/1187539221 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7ff05807c320 con 0x7ff0680638c0 2024-01-24T15:32:51.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.230+0000 7ff066ffd700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff05405f270 0x7ff054061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.230+0000 7ff04f7fe700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] conn(0x7ff054066a90 0x7ff054068f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.230+0000 7ff04f7fe700 1 -- 172.21.15.29:0/1187539221 --> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff054069510 con 0x7ff054066a90 2024-01-24T15:32:51.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.230+0000 7ff04f7fe700 1 -- 172.21.15.29:0/1187539221 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=237) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff05807c700 con 0x7ff0680638c0 2024-01-24T15:32:51.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.230+0000 7ff067fff700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] conn(0x7ff054066a90 0x7ff054068f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.230+0000 7ff066ffd700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff05405f270 0x7ff054061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff0680aed70 tx=0x7ff05c007d40).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.230+0000 7ff067fff700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] conn(0x7ff054066a90 0x7ff054068f30 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.234 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.231+0000 7ff04f7fe700 1 -- 172.21.15.29:0/1187539221 <== osd.2 v2:172.21.15.29:6816/43698 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff054069510 con 0x7ff054066a90 2024-01-24T15:32:51.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.245+0000 7ff06e044700 1 -- 172.21.15.29:0/1187539221 --> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff0680475e0 con 0x7ff054066a90 2024-01-24T15:32:51.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.245+0000 7ff04f7fe700 1 -- 172.21.15.29:0/1187539221 <== osd.2 v2:172.21.15.29:6816/43698 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff0680475e0 con 0x7ff054066a90 2024-01-24T15:32:51.248 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:32:51.248 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2024-01-24T15:32:51.248 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:32:51.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.246+0000 7ff06e044700 1 -- 172.21.15.29:0/1187539221 >> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] conn(0x7ff054066a90 msgr2=0x7ff054068f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.246+0000 7ff06e044700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.29:6816/43698,v1:172.21.15.29:6817/43698] conn(0x7ff054066a90 0x7ff054068f30 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.246+0000 7ff06e044700 1 -- 172.21.15.29:0/1187539221 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff05405f270 msgr2=0x7ff054061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.246+0000 7ff06e044700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff05405f270 0x7ff054061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff0680aed70 tx=0x7ff05c007d40).stop 2024-01-24T15:32:51.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.246+0000 7ff06e044700 1 -- 172.21.15.29:0/1187539221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0680638c0 msgr2=0x7ff068152990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.246+0000 7ff06e044700 1 --2- 172.21.15.29:0/1187539221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0680638c0 0x7ff068152990 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 rx=0x7ff068068ff0 tx=0x7ff058017000).stop 2024-01-24T15:32:51.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.246+0000 7ff067fff700 1 -- 172.21.15.29:0/1187539221 reap_dead start 2024-01-24T15:32:51.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.246+0000 7ff06e044700 1 -- 172.21.15.29:0/1187539221 shutdown_connections 2024-01-24T15:32:51.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.246+0000 7ff06e044700 1 -- 172.21.15.29:0/1187539221 >> 172.21.15.29:0/1187539221 conn(0x7ff0680c8bd0 msgr2=0x7ff06815a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:51.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.246+0000 7ff06e044700 1 -- 172.21.15.29:0/1187539221 shutdown_connections 2024-01-24T15:32:51.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.246+0000 7ff06e044700 1 -- 172.21.15.29:0/1187539221 wait complete. 2024-01-24T15:32:51.261 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-01-24T15:32:51.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.390+0000 7f4a4d346700 1 Processor -- start 2024-01-24T15:32:51.392 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.390+0000 7f4a4d346700 1 -- start start 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a4d346700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a481540a0 0x7f4a480be570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a4d346700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4a48154480 0x7f4a480a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a4d346700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4a480bca10 0x7f4a480627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a4d346700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4806a920 con 0x7f4a480bca10 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a4d346700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a48069c50 con 0x7f4a48154480 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a4d346700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480c8870 con 0x7f4a481540a0 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a477fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4a480bca10 0x7f4a480627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a477fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4a480bca10 0x7f4a480627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52570/0 (socket says 172.21.15.29:52570) 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a477fe700 1 -- 172.21.15.29:0/1886994356 learned_addr learned my addr 172.21.15.29:0/1886994356 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a46ffd700 1 --2- 172.21.15.29:0/1886994356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a481540a0 0x7f4a480be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a467fc700 1 --2- 172.21.15.29:0/1886994356 >> v2:172.21.15.163:3300/0 conn(0x7f4a48154480 0x7f4a480a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a477fe700 1 -- 172.21.15.29:0/1886994356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a481540a0 msgr2=0x7f4a480be570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:51.395 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a477fe700 1 --2- 172.21.15.29:0/1886994356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a481540a0 0x7f4a480be570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a477fe700 1 -- 172.21.15.29:0/1886994356 >> v2:172.21.15.163:3300/0 conn(0x7f4a48154480 msgr2=0x7f4a480a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a477fe700 1 --2- 172.21.15.29:0/1886994356 >> v2:172.21.15.163:3300/0 conn(0x7f4a48154480 0x7f4a480a6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a477fe700 1 -- 172.21.15.29:0/1886994356 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480c8260 con 0x7f4a480bca10 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.391+0000 7f4a477fe700 1 --2- 172.21.15.29:0/1886994356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a480bca10 0x7f4a480627b0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 rx=0x7f4a4805f390 tx=0x7f4a40009840).ready entity=mon.0 client_cookie=855f24f83d6dec09 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.392+0000 7f4a45ffb700 1 -- 172.21.15.29:0/1886994356 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000e070 con 0x7f4a480bca10 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.392+0000 7f4a45ffb700 1 -- 172.21.15.29:0/1886994356 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4000b970 con 0x7f4a480bca10 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.392+0000 7f4a45ffb700 1 -- 172.21.15.29:0/1886994356 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000bcf0 con 0x7f4a480bca10 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.392+0000 7f4a4d346700 1 -- 172.21.15.29:0/1886994356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a480bca10 msgr2=0x7f4a480627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.392+0000 7f4a4d346700 1 --2- 172.21.15.29:0/1886994356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a480bca10 0x7f4a480627b0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 rx=0x7f4a4805f390 tx=0x7f4a40009840).stop 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.392+0000 7f4a4d346700 1 -- 172.21.15.29:0/1886994356 shutdown_connections 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.392+0000 7f4a4d346700 1 --2- 172.21.15.29:0/1886994356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a481540a0 0x7f4a480be570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.392+0000 7f4a4d346700 1 --2- 172.21.15.29:0/1886994356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a480bca10 0x7f4a480627b0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.392+0000 7f4a4d346700 1 --2- 172.21.15.29:0/1886994356 >> v2:172.21.15.163:3300/0 conn(0x7f4a48154480 0x7f4a480a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.396 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.392+0000 7f4a4d346700 1 -- 172.21.15.29:0/1886994356 >> 172.21.15.29:0/1886994356 conn(0x7f4a480c5020 msgr2=0x7f4a480c7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:51.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.392+0000 7f4a4d346700 1 -- 172.21.15.29:0/1886994356 shutdown_connections 2024-01-24T15:32:51.397 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.392+0000 7f4a4d346700 1 -- 172.21.15.29:0/1886994356 wait complete. 2024-01-24T15:32:51.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.394+0000 7f4a4d346700 1 Processor -- start 2024-01-24T15:32:51.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.394+0000 7f4a4d346700 1 -- start start 2024-01-24T15:32:51.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.394+0000 7f4a4d346700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a480bca10 0x7f4a481502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.394+0000 7f4a4d346700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a481540a0 0x7f4a48152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a4d346700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a48154480 0x7f4a4814dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a4d346700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480bfb20 con 0x7f4a481540a0 2024-01-24T15:32:51.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a4d346700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48152f00 con 0x7f4a480bca10 2024-01-24T15:32:51.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a4d346700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a481531e0 con 0x7f4a48154480 2024-01-24T15:32:51.399 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a467fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a481540a0 0x7f4a48152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a467fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a481540a0 0x7f4a48152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52586/0 (socket says 172.21.15.29:52586) 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a467fc700 1 -- 172.21.15.29:0/1791814376 learned_addr learned my addr 172.21.15.29:0/1791814376 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a46ffd700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a480bca10 0x7f4a481502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a477fe700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a48154480 0x7f4a4814dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a467fc700 1 -- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a48154480 msgr2=0x7f4a4814dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a467fc700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a48154480 0x7f4a4814dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a467fc700 1 -- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a480bca10 msgr2=0x7f4a481502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a467fc700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a480bca10 0x7f4a481502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a467fc700 1 -- 172.21.15.29:0/1791814376 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40009ae0 con 0x7f4a481540a0 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a467fc700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a481540a0 0x7f4a48152720 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 rx=0x7f4a3c00e6f0 tx=0x7f4a3c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a2effd700 1 -- 172.21.15.29:0/1791814376 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c012600 con 0x7f4a481540a0 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a2effd700 1 -- 172.21.15.29:0/1791814376 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c012760 con 0x7f4a481540a0 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a4d346700 1 -- 172.21.15.29:0/1791814376 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a480bfdc0 con 0x7f4a481540a0 2024-01-24T15:32:51.400 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a2effd700 1 -- 172.21.15.29:0/1791814376 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c012ae0 con 0x7f4a481540a0 2024-01-24T15:32:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.395+0000 7f4a4d346700 1 -- 172.21.15.29:0/1791814376 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a48152c10 con 0x7f4a481540a0 2024-01-24T15:32:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.397+0000 7f4a4d346700 1 -- 172.21.15.29:0/1791814376 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a48066e50 con 0x7f4a481540a0 2024-01-24T15:32:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.397+0000 7f4a2effd700 1 -- 172.21.15.29:0/1791814376 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4a3c01b400 con 0x7f4a481540a0 2024-01-24T15:32:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.397+0000 7f4a2effd700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4a3405f270 0x7f4a34061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.401 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.397+0000 7f4a2effd700 1 -- 172.21.15.29:0/1791814376 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f4a3c07a900 con 0x7f4a481540a0 2024-01-24T15:32:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.398+0000 7f4a46ffd700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4a3405f270 0x7f4a34061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.402 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.398+0000 7f4a46ffd700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4a3405f270 0x7f4a34061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f4a4814ff90 tx=0x7f4a38009060).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.403 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.402+0000 7f4a2effd700 1 -- 172.21.15.29:0/1791814376 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f4a3c04b6d0 con 0x7f4a481540a0 2024-01-24T15:32:51.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.585+0000 7f4a4d346700 1 -- 172.21.15.29:0/1791814376 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a481533a0 con 0x7f4a3405f270 2024-01-24T15:32:51.587 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.585+0000 7f4a2effd700 1 -- 172.21.15.29:0/1791814376 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4a38009970 con 0x7f4a3405f270 2024-01-24T15:32:51.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.587+0000 7f4a4d346700 1 -- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4a3405f270 msgr2=0x7f4a34061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.587+0000 7f4a4d346700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4a3405f270 0x7f4a34061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f4a4814ff90 tx=0x7f4a38009060).stop 2024-01-24T15:32:51.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.587+0000 7f4a4d346700 1 -- 172.21.15.29:0/1791814376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a481540a0 msgr2=0x7f4a48152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.587+0000 7f4a4d346700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a481540a0 0x7f4a48152720 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 rx=0x7f4a3c00e6f0 tx=0x7f4a3c009060).stop 2024-01-24T15:32:51.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.587+0000 7f4a4d346700 1 -- 172.21.15.29:0/1791814376 shutdown_connections 2024-01-24T15:32:51.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.587+0000 7f4a4d346700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f4a3405f270 0x7f4a34061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.587+0000 7f4a4d346700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a48154480 0x7f4a4814dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.587+0000 7f4a4d346700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a480bca10 0x7f4a481502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.587+0000 7f4a4d346700 1 --2- 172.21.15.29:0/1791814376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a481540a0 0x7f4a48152720 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.587+0000 7f4a4d346700 1 -- 172.21.15.29:0/1791814376 >> 172.21.15.29:0/1791814376 conn(0x7f4a480c5020 msgr2=0x7f4a480639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:51.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.587+0000 7f4a4d346700 1 -- 172.21.15.29:0/1791814376 shutdown_connections 2024-01-24T15:32:51.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.587+0000 7f4a4d346700 1 -- 172.21.15.29:0/1791814376 wait complete. 2024-01-24T15:32:51.591 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 2 to deep-scrub 2024-01-24T15:32:51.603 INFO:tasks.ceph:Scrubbing osd.3 2024-01-24T15:32:51.603 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-01-24T15:32:51.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.735+0000 7fe155d01700 1 Processor -- start 2024-01-24T15:32:51.737 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.735+0000 7fe155d01700 1 -- start start 2024-01-24T15:32:51.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.736+0000 7fe155d01700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe150062970 0x7fe150062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.740 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.736+0000 7fe155d01700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1500c2190 0x7fe150158720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.736+0000 7fe155d01700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1500c3e00 con 0x7fe1500c4b10 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.736+0000 7fe155d01700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1500c2900 con 0x7fe1500c2190 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.736+0000 7fe155d01700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe150158c10 con 0x7fe150062970 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.736+0000 7fe14effd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1500c2190 0x7fe150158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.736+0000 7fe14f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe150062970 0x7fe150062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.736+0000 7fe14f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe150062970 0x7fe150062d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.29:48106/0 (socket says 172.21.15.29:48106) 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.736+0000 7fe14ffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe1500c4b10 0x7fe1500a4890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:40526/0 (socket says 172.21.15.29:40526) 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.736+0000 7fe14f7fe700 1 -- 172.21.15.29:0/51577542 learned_addr learned my addr 172.21.15.29:0/51577542 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.736+0000 7fe14effd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1500c2190 0x7fe150158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.29:55284/0 (socket says 172.21.15.29:55284) 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.736+0000 7fe14e7fc700 1 -- 172.21.15.29:0/51577542 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220624527 0 0) 0x7fe1500c3e00 con 0x7fe1500c4b10 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe14e7fc700 1 -- 172.21.15.29:0/51577542 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1380033f0 con 0x7fe1500c4b10 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe14effd700 1 -- 172.21.15.29:0/51577542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe150062970 msgr2=0x7fe150062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe14effd700 1 --2- 172.21.15.29:0/51577542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe150062970 0x7fe150062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.741 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe14effd700 1 -- 172.21.15.29:0/51577542 >> v1:172.21.15.163:6789/0 conn(0x7fe1500c4b10 legacy=0x7fe1500a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe14effd700 1 -- 172.21.15.29:0/51577542 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe150065720 con 0x7fe1500c2190 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe14f7fe700 1 --2- 172.21.15.29:0/51577542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe150062970 0x7fe150062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe14effd700 1 --2- 172.21.15.29:0/51577542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500c2190 0x7fe150158720 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7fe144009760 tx=0x7fe144009060).ready entity=mon.1 client_cookie=cf690a9f34cfd433 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe14e7fc700 1 -- 172.21.15.29:0/51577542 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400f510 con 0x7fe1500c2190 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe14e7fc700 1 -- 172.21.15.29:0/51577542 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14400f670 con 0x7fe1500c2190 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe14e7fc700 1 -- 172.21.15.29:0/51577542 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400f9f0 con 0x7fe1500c2190 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe155d01700 1 -- 172.21.15.29:0/51577542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500c2190 msgr2=0x7fe150158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe155d01700 1 --2- 172.21.15.29:0/51577542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500c2190 0x7fe150158720 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7fe144009760 tx=0x7fe144009060).stop 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe155d01700 1 -- 172.21.15.29:0/51577542 shutdown_connections 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.737+0000 7fe155d01700 1 --2- 172.21.15.29:0/51577542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe150062970 0x7fe150062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.738+0000 7fe155d01700 1 --2- 172.21.15.29:0/51577542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500c2190 0x7fe150158720 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.738+0000 7fe155d01700 1 -- 172.21.15.29:0/51577542 >> 172.21.15.29:0/51577542 conn(0x7fe150067b20 msgr2=0x7fe1500638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.738+0000 7fe155d01700 1 -- 172.21.15.29:0/51577542 shutdown_connections 2024-01-24T15:32:51.742 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.738+0000 7fe155d01700 1 -- 172.21.15.29:0/51577542 wait complete. 2024-01-24T15:32:51.745 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.739+0000 7fe155d01700 1 Processor -- start 2024-01-24T15:32:51.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.739+0000 7fe155d01700 1 -- start start 2024-01-24T15:32:51.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe155d01700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe150062970 0x7fe1500bd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe155d01700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1500c4b10 0x7fe1501512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe155d01700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1500ad990 0x7fe150154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe155d01700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe150157890 con 0x7fe1500ad990 2024-01-24T15:32:51.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe155d01700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe150157400 con 0x7fe150062970 2024-01-24T15:32:51.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe155d01700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1501517e0 con 0x7fe1500c4b10 2024-01-24T15:32:51.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe14ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1500ad990 0x7fe150154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe14effd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1500c4b10 0x7fe1501512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe14f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe150062970 0x7fe1500bd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe14ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1500ad990 0x7fe150154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52588/0 (socket says 172.21.15.29:52588) 2024-01-24T15:32:51.746 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe14ffff700 1 -- 172.21.15.29:0/945944989 learned_addr learned my addr 172.21.15.29:0/945944989 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe14ffff700 1 -- 172.21.15.29:0/945944989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1500c4b10 msgr2=0x7fe1501512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe14ffff700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1500c4b10 0x7fe1501512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe14ffff700 1 -- 172.21.15.29:0/945944989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe150062970 msgr2=0x7fe1500bd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe14ffff700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe150062970 0x7fe1500bd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe14ffff700 1 -- 172.21.15.29:0/945944989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe140009370 con 0x7fe1500ad990 2024-01-24T15:32:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.740+0000 7fe14f7fe700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe150062970 0x7fe1500bd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.741+0000 7fe14ffff700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1500ad990 0x7fe150154390 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 rx=0x7fe14800c710 tx=0x7fe148007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.741+0000 7fe1377fe700 1 -- 172.21.15.29:0/945944989 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148005c70 con 0x7fe1500ad990 2024-01-24T15:32:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.741+0000 7fe1377fe700 1 -- 172.21.15.29:0/945944989 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe148005dd0 con 0x7fe1500ad990 2024-01-24T15:32:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.741+0000 7fe155d01700 1 -- 172.21.15.29:0/945944989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1440092a0 con 0x7fe1500ad990 2024-01-24T15:32:51.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.741+0000 7fe1377fe700 1 -- 172.21.15.29:0/945944989 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480072e0 con 0x7fe1500ad990 2024-01-24T15:32:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.741+0000 7fe155d01700 1 -- 172.21.15.29:0/945944989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe150156ad0 con 0x7fe1500ad990 2024-01-24T15:32:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.742+0000 7fe155d01700 1 -- 172.21.15.29:0/945944989 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1500475e0 con 0x7fe1500ad990 2024-01-24T15:32:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.742+0000 7fe1377fe700 1 -- 172.21.15.29:0/945944989 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe148007690 con 0x7fe1500ad990 2024-01-24T15:32:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.743+0000 7fe1377fe700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe13c05f2a0 0x7fe13c061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.743+0000 7fe1377fe700 1 -- 172.21.15.29:0/945944989 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fe148079f30 con 0x7fe1500ad990 2024-01-24T15:32:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.743+0000 7fe1377fe700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] conn(0x7fe13c066a40 0x7fe13c068ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.743+0000 7fe1377fe700 1 -- 172.21.15.29:0/945944989 --> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe13c0694c0 con 0x7fe13c066a40 2024-01-24T15:32:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.743+0000 7fe1377fe700 1 -- 172.21.15.29:0/945944989 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=237) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe14807c580 con 0x7fe1500ad990 2024-01-24T15:32:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.743+0000 7fe14f7fe700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe13c05f2a0 0x7fe13c061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.743+0000 7fe14effd700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] conn(0x7fe13c066a40 0x7fe13c068ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.743+0000 7fe14f7fe700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe13c05f2a0 0x7fe13c061760 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe140006fd0 tx=0x7fe1400090f0).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.743+0000 7fe14effd700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] conn(0x7fe13c066a40 0x7fe13c068ee0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.749 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.744+0000 7fe1377fe700 1 -- 172.21.15.29:0/945944989 <== osd.3 v2:172.21.15.29:6808/43697 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe13c0694c0 con 0x7fe13c066a40 2024-01-24T15:32:51.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.759+0000 7fe155d01700 1 -- 172.21.15.29:0/945944989 --> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe150157570 con 0x7fe13c066a40 2024-01-24T15:32:51.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.759+0000 7fe1377fe700 1 -- 172.21.15.29:0/945944989 <== osd.3 v2:172.21.15.29:6808/43697 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe150157570 con 0x7fe13c066a40 2024-01-24T15:32:51.762 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:32:51.762 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2024-01-24T15:32:51.762 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:32:51.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.759+0000 7fe155d01700 1 -- 172.21.15.29:0/945944989 >> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] conn(0x7fe13c066a40 msgr2=0x7fe13c068ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.759+0000 7fe155d01700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.29:6808/43697,v1:172.21.15.29:6809/43697] conn(0x7fe13c066a40 0x7fe13c068ee0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.760+0000 7fe155d01700 1 -- 172.21.15.29:0/945944989 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe13c05f2a0 msgr2=0x7fe13c061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.760+0000 7fe155d01700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fe13c05f2a0 0x7fe13c061760 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe140006fd0 tx=0x7fe1400090f0).stop 2024-01-24T15:32:51.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.760+0000 7fe155d01700 1 -- 172.21.15.29:0/945944989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1500ad990 msgr2=0x7fe150154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.760+0000 7fe155d01700 1 --2- 172.21.15.29:0/945944989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1500ad990 0x7fe150154390 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 rx=0x7fe14800c710 tx=0x7fe148007f90).stop 2024-01-24T15:32:51.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.760+0000 7fe14ffff700 1 -- 172.21.15.29:0/945944989 reap_dead start 2024-01-24T15:32:51.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.760+0000 7fe155d01700 1 -- 172.21.15.29:0/945944989 shutdown_connections 2024-01-24T15:32:51.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.760+0000 7fe155d01700 1 -- 172.21.15.29:0/945944989 >> 172.21.15.29:0/945944989 conn(0x7fe150067b20 msgr2=0x7fe150152060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:51.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.760+0000 7fe155d01700 1 -- 172.21.15.29:0/945944989 shutdown_connections 2024-01-24T15:32:51.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.760+0000 7fe155d01700 1 -- 172.21.15.29:0/945944989 wait complete. 2024-01-24T15:32:51.775 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-01-24T15:32:51.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.904+0000 7ff6b39d9700 1 Processor -- start 2024-01-24T15:32:51.906 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.904+0000 7ff6b39d9700 1 -- start start 2024-01-24T15:32:51.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6b39d9700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6ac066bb0 0x7ff6ac155d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6b39d9700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac156b40 con 0x7ff6ac135080 2024-01-24T15:32:51.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6b39d9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac153110 con 0x7ff6ac156260 2024-01-24T15:32:51.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6b39d9700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ac14cb90 con 0x7ff6ac066bb0 2024-01-24T15:32:51.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6b0f74700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6ac066bb0 0x7ff6ac155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.908 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6b0f74700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6ac066bb0 0x7ff6ac155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52604/0 (socket says 172.21.15.29:52604) 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6b0f74700 1 -- 172.21.15.29:0/2973162883 learned_addr learned my addr 172.21.15.29:0/2973162883 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6b1775700 1 --1- 172.21.15.29:0/2973162883 >> v1:172.21.15.29:6789/0 conn(0x7ff6ac135080 0x7ff6ac0664a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54012/0 (socket says 172.21.15.29:54012) 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6b0f74700 1 -- 172.21.15.29:0/2973162883 >> v1:172.21.15.163:6789/0 conn(0x7ff6ac156260 legacy=0x7ff6ac15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6b0f74700 1 -- 172.21.15.29:0/2973162883 >> v1:172.21.15.29:6789/0 conn(0x7ff6ac135080 legacy=0x7ff6ac0664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6b0f74700 1 -- 172.21.15.29:0/2973162883 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac1535a0 con 0x7ff6ac066bb0 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6b0f74700 1 --2- 172.21.15.29:0/2973162883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ac066bb0 0x7ff6ac155d70 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 rx=0x7ff69c006fd0 tx=0x7ff69c0090f0).ready entity=mon.0 client_cookie=ae443ab449c6159f server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.905+0000 7ff6a3fff700 1 -- 172.21.15.29:0/2973162883 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00b600 con 0x7ff6ac066bb0 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.906+0000 7ff6a3fff700 1 -- 172.21.15.29:0/2973162883 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c00b760 con 0x7ff6ac066bb0 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.906+0000 7ff6a3fff700 1 -- 172.21.15.29:0/2973162883 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00bae0 con 0x7ff6ac066bb0 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.906+0000 7ff6b39d9700 1 -- 172.21.15.29:0/2973162883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ac066bb0 msgr2=0x7ff6ac155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.906+0000 7ff6b39d9700 1 --2- 172.21.15.29:0/2973162883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ac066bb0 0x7ff6ac155d70 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 rx=0x7ff69c006fd0 tx=0x7ff69c0090f0).stop 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.906+0000 7ff6b39d9700 1 -- 172.21.15.29:0/2973162883 shutdown_connections 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.906+0000 7ff6b39d9700 1 --2- 172.21.15.29:0/2973162883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ac066bb0 0x7ff6ac155d70 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.909 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.906+0000 7ff6b39d9700 1 -- 172.21.15.29:0/2973162883 >> 172.21.15.29:0/2973162883 conn(0x7ff6ac0c5640 msgr2=0x7ff6ac0c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:51.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.906+0000 7ff6b39d9700 1 -- 172.21.15.29:0/2973162883 shutdown_connections 2024-01-24T15:32:51.910 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.906+0000 7ff6b39d9700 1 -- 172.21.15.29:0/2973162883 wait complete. 2024-01-24T15:32:51.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.907+0000 7ff6b39d9700 1 Processor -- start 2024-01-24T15:32:51.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.907+0000 7ff6b39d9700 1 -- start start 2024-01-24T15:32:51.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.908+0000 7ff6b39d9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ac066bb0 0x7ff6ac1356b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.908+0000 7ff6b39d9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6ac135080 0x7ff6ac14b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.908+0000 7ff6b39d9700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ac156260 0x7ff6ac162970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.908+0000 7ff6b39d9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0c0150 con 0x7ff6ac066bb0 2024-01-24T15:32:51.912 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.908+0000 7ff6b39d9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac1633c0 con 0x7ff6ac135080 2024-01-24T15:32:51.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.908+0000 7ff6b39d9700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac1636a0 con 0x7ff6ac156260 2024-01-24T15:32:51.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b1775700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ac066bb0 0x7ff6ac1356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b1775700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ac066bb0 0x7ff6ac1356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52606/0 (socket says 172.21.15.29:52606) 2024-01-24T15:32:51.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b1775700 1 -- 172.21.15.29:0/1795017876 learned_addr learned my addr 172.21.15.29:0/1795017876 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:51.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b0f74700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6ac135080 0x7ff6ac14b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b1f76700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ac156260 0x7ff6ac162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b1775700 1 -- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ac156260 msgr2=0x7ff6ac162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b1775700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ac156260 0x7ff6ac162970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b1775700 1 -- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6ac135080 msgr2=0x7ff6ac14b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:51.913 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b1775700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6ac135080 0x7ff6ac14b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:51.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b1775700 1 -- 172.21.15.29:0/1795017876 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c009370 con 0x7ff6ac066bb0 2024-01-24T15:32:51.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b1775700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ac066bb0 0x7ff6ac1356b0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 rx=0x7ff6a8007fd0 tx=0x7ff6a8007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6a17fa700 1 -- 172.21.15.29:0/1795017876 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800ce10 con 0x7ff6ac066bb0 2024-01-24T15:32:51.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6a17fa700 1 -- 172.21.15.29:0/1795017876 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a80072e0 con 0x7ff6ac066bb0 2024-01-24T15:32:51.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b39d9700 1 -- 172.21.15.29:0/1795017876 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac0bfde0 con 0x7ff6ac066bb0 2024-01-24T15:32:51.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6a17fa700 1 -- 172.21.15.29:0/1795017876 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a80056e0 con 0x7ff6ac066bb0 2024-01-24T15:32:51.914 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.909+0000 7ff6b39d9700 1 -- 172.21.15.29:0/1795017876 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac14b920 con 0x7ff6ac066bb0 2024-01-24T15:32:51.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.911+0000 7ff6b39d9700 1 -- 172.21.15.29:0/1795017876 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac1638c0 con 0x7ff6ac066bb0 2024-01-24T15:32:51.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.911+0000 7ff6a17fa700 1 -- 172.21.15.29:0/1795017876 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff6a8005ad0 con 0x7ff6ac066bb0 2024-01-24T15:32:51.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.911+0000 7ff6a17fa700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff69005f270 0x7ff690061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:51.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.911+0000 7ff6a17fa700 1 -- 172.21.15.29:0/1795017876 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7ff6a8079e30 con 0x7ff6ac066bb0 2024-01-24T15:32:51.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.911+0000 7ff6b0f74700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff69005f270 0x7ff690061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:51.915 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.912+0000 7ff6b0f74700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff69005f270 0x7ff690061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff69c009130 tx=0x7ff69c017000).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:51.918 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:51.916+0000 7ff6a17fa700 1 -- 172.21.15.29:0/1795017876 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7ff6a804ac60 con 0x7ff6ac066bb0 2024-01-24T15:32:52.096 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.094+0000 7ff6b39d9700 1 -- 172.21.15.29:0/1795017876 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6ac14bb10 con 0x7ff69005f270 2024-01-24T15:32:52.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.095+0000 7ff6a17fa700 1 -- 172.21.15.29:0/1795017876 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff69c006b70 con 0x7ff69005f270 2024-01-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.097+0000 7ff6b39d9700 1 -- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff69005f270 msgr2=0x7ff690061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.097+0000 7ff6b39d9700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff69005f270 0x7ff690061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff69c009130 tx=0x7ff69c017000).stop 2024-01-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.097+0000 7ff6b39d9700 1 -- 172.21.15.29:0/1795017876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ac066bb0 msgr2=0x7ff6ac1356b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.097+0000 7ff6b39d9700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ac066bb0 0x7ff6ac1356b0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 rx=0x7ff6a8007fd0 tx=0x7ff6a8007df0).stop 2024-01-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.097+0000 7ff6b39d9700 1 -- 172.21.15.29:0/1795017876 shutdown_connections 2024-01-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.097+0000 7ff6b39d9700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7ff69005f270 0x7ff690061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.097+0000 7ff6b39d9700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6ac156260 0x7ff6ac162970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.097+0000 7ff6b39d9700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6ac066bb0 0x7ff6ac1356b0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.097+0000 7ff6b39d9700 1 --2- 172.21.15.29:0/1795017876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6ac135080 0x7ff6ac14b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.097+0000 7ff6b39d9700 1 -- 172.21.15.29:0/1795017876 >> 172.21.15.29:0/1795017876 conn(0x7ff6ac0c5640 msgr2=0x7ff6ac0c7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.097+0000 7ff6b39d9700 1 -- 172.21.15.29:0/1795017876 shutdown_connections 2024-01-24T15:32:52.100 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.097+0000 7ff6b39d9700 1 -- 172.21.15.29:0/1795017876 wait complete. 2024-01-24T15:32:52.101 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 3 to deep-scrub 2024-01-24T15:32:52.113 INFO:tasks.ceph:Scrubbing osd.4 2024-01-24T15:32:52.113 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-01-24T15:32:52.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.244+0000 7f8fe4edc700 1 Processor -- start 2024-01-24T15:32:52.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.244+0000 7f8fe4edc700 1 -- start start 2024-01-24T15:32:52.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.245+0000 7f8fe4edc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8fe0066b60 0x7f8fe00c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.245+0000 7f8fe4edc700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8fe00684c0 0x7f8fe0153090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.245+0000 7f8fe4edc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe0067a70 con 0x7f8fe014fd10 2024-01-24T15:32:52.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.245+0000 7f8fe4edc700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe00c3d50 con 0x7f8fe00684c0 2024-01-24T15:32:52.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.245+0000 7f8fe4edc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe01500f0 con 0x7f8fe0066b60 2024-01-24T15:32:52.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.245+0000 7f8fded9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8fe00684c0 0x7f8fe0153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.245+0000 7f8fddd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8fe0066b60 0x7f8fe00c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.245+0000 7f8fded9d700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8fe00684c0 0x7f8fe0153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52618/0 (socket says 172.21.15.29:52618) 2024-01-24T15:32:52.249 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.245+0000 7f8fde59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8fe014fd10 0x7f8fe0066450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.29:40548/0 (socket says 172.21.15.29:40548) 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.245+0000 7f8fded9d700 1 -- 172.21.15.29:0/2271301547 learned_addr learned my addr 172.21.15.29:0/2271301547 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.246+0000 7f8fddd9b700 1 -- 172.21.15.29:0/2271301547 >> v2:172.21.15.29:3300/0 conn(0x7f8fe00684c0 msgr2=0x7f8fe0153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.246+0000 7f8fddd9b700 1 --2- 172.21.15.29:0/2271301547 >> v2:172.21.15.29:3300/0 conn(0x7f8fe00684c0 0x7f8fe0153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.246+0000 7f8fddd9b700 1 -- 172.21.15.29:0/2271301547 >> v1:172.21.15.163:6789/0 conn(0x7f8fe014fd10 legacy=0x7f8fe0066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.246+0000 7f8fdd59a700 1 -- 172.21.15.29:0/2271301547 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820272636 0 0) 0x7f8fe0067a70 con 0x7f8fe014fd10 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.246+0000 7f8fddd9b700 1 -- 172.21.15.29:0/2271301547 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe0153580 con 0x7f8fe0066b60 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.246+0000 7f8fded9d700 1 --2- 172.21.15.29:0/2271301547 >> v2:172.21.15.29:3300/0 conn(0x7f8fe00684c0 0x7f8fe0153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.246+0000 7f8fddd9b700 1 --2- 172.21.15.29:0/2271301547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fe0066b60 0x7f8fe00c31d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f8fe00c3180 tx=0x7f8fd0009060).ready entity=mon.1 client_cookie=e6eef5ff08fa235 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.246+0000 7f8fdd59a700 1 -- 172.21.15.29:0/2271301547 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd000f510 con 0x7f8fe0066b60 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.246+0000 7f8fdd59a700 1 -- 172.21.15.29:0/2271301547 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd000f670 con 0x7f8fe0066b60 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.246+0000 7f8fdd59a700 1 -- 172.21.15.29:0/2271301547 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd000f9f0 con 0x7f8fe0066b60 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.246+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2271301547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fe0066b60 msgr2=0x7f8fe00c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.246+0000 7f8fe4edc700 1 --2- 172.21.15.29:0/2271301547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fe0066b60 0x7f8fe00c31d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f8fe00c3180 tx=0x7f8fd0009060).stop 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.247+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2271301547 shutdown_connections 2024-01-24T15:32:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.247+0000 7f8fe4edc700 1 --2- 172.21.15.29:0/2271301547 >> v2:172.21.15.29:3300/0 conn(0x7f8fe00684c0 0x7f8fe0153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.247+0000 7f8fe4edc700 1 --2- 172.21.15.29:0/2271301547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fe0066b60 0x7f8fe00c31d0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.247+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2271301547 >> 172.21.15.29:0/2271301547 conn(0x7f8fe00c8bd0 msgr2=0x7f8fe00c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:52.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.247+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2271301547 shutdown_connections 2024-01-24T15:32:52.251 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.247+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2271301547 wait complete. 2024-01-24T15:32:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.248+0000 7f8fe4edc700 1 Processor -- start 2024-01-24T15:32:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.248+0000 7f8fe4edc700 1 -- start start 2024-01-24T15:32:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fe4edc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fe0066b60 0x7f8fe01521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fe4edc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe00684c0 0x7f8fe01329e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fe4edc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fe014fd10 0x7f8fe0150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fe4edc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe0153160 con 0x7f8fe00684c0 2024-01-24T15:32:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fe4edc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe01526d0 con 0x7f8fe0066b60 2024-01-24T15:32:52.254 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fe4edc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe01657b0 con 0x7f8fe014fd10 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fddd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe00684c0 0x7f8fe01329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fddd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe00684c0 0x7f8fe01329e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52634/0 (socket says 172.21.15.29:52634) 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fddd9b700 1 -- 172.21.15.29:0/2290151522 learned_addr learned my addr 172.21.15.29:0/2290151522 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fded9d700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fe014fd10 0x7f8fe0150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fde59c700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fe0066b60 0x7f8fe01521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fddd9b700 1 -- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fe014fd10 msgr2=0x7f8fe0150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fddd9b700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fe014fd10 0x7f8fe0150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fddd9b700 1 -- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fe0066b60 msgr2=0x7f8fe01521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fddd9b700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fe0066b60 0x7f8fe01521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fddd9b700 1 -- 172.21.15.29:0/2290151522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8009ae0 con 0x7f8fe00684c0 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fddd9b700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe00684c0 0x7f8fe01329e0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 rx=0x7f8fe0068ba0 tx=0x7f8fd0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.249+0000 7f8fd67fc700 1 -- 172.21.15.29:0/2290151522 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd000f7e0 con 0x7f8fe00684c0 2024-01-24T15:32:52.255 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.250+0000 7f8fd67fc700 1 -- 172.21.15.29:0/2290151522 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd000f940 con 0x7f8fe00684c0 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.250+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2290151522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd00092a0 con 0x7f8fe00684c0 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.250+0000 7f8fd67fc700 1 -- 172.21.15.29:0/2290151522 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd001b660 con 0x7f8fe00684c0 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.250+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2290151522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe0152ba0 con 0x7f8fe00684c0 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.250+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2290151522 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8fe0165580 con 0x7f8fe00684c0 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.251+0000 7f8fd67fc700 1 -- 172.21.15.29:0/2290151522 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8fd001b9d0 con 0x7f8fe00684c0 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.251+0000 7f8fd67fc700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f8fc005f270 0x7f8fc0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.251+0000 7f8fd67fc700 1 -- 172.21.15.29:0/2290151522 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f8fd007be50 con 0x7f8fe00684c0 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.251+0000 7f8fde59c700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f8fc005f270 0x7f8fc0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.251+0000 7f8fd67fc700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] conn(0x7f8fc0066a90 0x7f8fc0068f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.251+0000 7f8fd67fc700 1 -- 172.21.15.29:0/2290151522 --> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fc0069510 con 0x7f8fc0066a90 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.252+0000 7f8fd67fc700 1 -- 172.21.15.29:0/2290151522 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=237) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd007e4a0 con 0x7f8fe00684c0 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.252+0000 7f8fded9d700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] conn(0x7f8fc0066a90 0x7f8fc0068f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.252+0000 7f8fde59c700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f8fc005f270 0x7f8fc0061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f8fc8007fd0 tx=0x7f8fc8007d70).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.256 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.252+0000 7f8fded9d700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] conn(0x7f8fc0066a90 0x7f8fc0068f30 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.257 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.253+0000 7f8fd67fc700 1 -- 172.21.15.29:0/2290151522 <== osd.4 v2:172.21.15.163:6800/43896 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8fc0069510 con 0x7f8fc0066a90 2024-01-24T15:32:52.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.267+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2290151522 --> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8fe00ab660 con 0x7f8fc0066a90 2024-01-24T15:32:52.269 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.267+0000 7f8fd67fc700 1 -- 172.21.15.29:0/2290151522 <== osd.4 v2:172.21.15.163:6800/43896 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8fe00ab660 con 0x7f8fc0066a90 2024-01-24T15:32:52.270 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:32:52.270 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2024-01-24T15:32:52.270 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:32:52.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.268+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] conn(0x7f8fc0066a90 msgr2=0x7f8fc0068f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.268+0000 7f8fe4edc700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:6800/43896,v1:172.21.15.163:6801/43896] conn(0x7f8fc0066a90 0x7f8fc0068f30 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.268+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f8fc005f270 msgr2=0x7f8fc0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.268+0000 7f8fe4edc700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f8fc005f270 0x7f8fc0061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f8fc8007fd0 tx=0x7f8fc8007d70).stop 2024-01-24T15:32:52.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.268+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2290151522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe00684c0 msgr2=0x7f8fe01329e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.268+0000 7f8fe4edc700 1 --2- 172.21.15.29:0/2290151522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fe00684c0 0x7f8fe01329e0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 rx=0x7f8fe0068ba0 tx=0x7f8fd0017000).stop 2024-01-24T15:32:52.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.268+0000 7f8fded9d700 1 -- 172.21.15.29:0/2290151522 reap_dead start 2024-01-24T15:32:52.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.268+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2290151522 shutdown_connections 2024-01-24T15:32:52.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.268+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2290151522 >> 172.21.15.29:0/2290151522 conn(0x7f8fe00c8bd0 msgr2=0x7f8fe0066f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:52.271 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.268+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2290151522 shutdown_connections 2024-01-24T15:32:52.272 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.268+0000 7f8fe4edc700 1 -- 172.21.15.29:0/2290151522 wait complete. 2024-01-24T15:32:52.283 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-01-24T15:32:52.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.410+0000 7fa93b7ab700 1 Processor -- start 2024-01-24T15:32:52.413 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.411+0000 7fa93b7ab700 1 -- start start 2024-01-24T15:32:52.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa93b7ab700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa934155470 0x7fa9340664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa93b7ab700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa934066990 0x7fa93413d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.415 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa93b7ab700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa934158800 con 0x7fa934151de0 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa93b7ab700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa934066dd0 con 0x7fa934155470 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa93b7ab700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa934157ed0 con 0x7fa934066990 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa938d46700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa934155470 0x7fa9340664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa938d46700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa934155470 0x7fa9340664a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52644/0 (socket says 172.21.15.29:52644) 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa938d46700 1 -- 172.21.15.29:0/1227626984 learned_addr learned my addr 172.21.15.29:0/1227626984 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa939d48700 1 --2- 172.21.15.29:0/1227626984 >> v2:172.21.15.163:3300/0 conn(0x7fa934066990 0x7fa93413d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa938d46700 1 -- 172.21.15.29:0/1227626984 >> v2:172.21.15.163:3300/0 conn(0x7fa934066990 msgr2=0x7fa93413d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa938d46700 1 --2- 172.21.15.29:0/1227626984 >> v2:172.21.15.163:3300/0 conn(0x7fa934066990 0x7fa93413d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa938d46700 1 -- 172.21.15.29:0/1227626984 >> v1:172.21.15.163:6789/0 conn(0x7fa934151de0 legacy=0x7fa9340638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa938d46700 1 -- 172.21.15.29:0/1227626984 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9341581b0 con 0x7fa934155470 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa938d46700 1 --2- 172.21.15.29:0/1227626984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa934155470 0x7fa9340664a0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 rx=0x7fa9340c6680 tx=0x7fa9240090f0).ready entity=mon.0 client_cookie=d20000263416e4e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.412+0000 7fa92bfff700 1 -- 172.21.15.29:0/1227626984 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92400b520 con 0x7fa934155470 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.413+0000 7fa92bfff700 1 -- 172.21.15.29:0/1227626984 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92400b680 con 0x7fa934155470 2024-01-24T15:32:52.416 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.413+0000 7fa92bfff700 1 -- 172.21.15.29:0/1227626984 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92400ba00 con 0x7fa934155470 2024-01-24T15:32:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.413+0000 7fa93b7ab700 1 -- 172.21.15.29:0/1227626984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa934155470 msgr2=0x7fa9340664a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.413+0000 7fa93b7ab700 1 --2- 172.21.15.29:0/1227626984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa934155470 0x7fa9340664a0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 rx=0x7fa9340c6680 tx=0x7fa9240090f0).stop 2024-01-24T15:32:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.413+0000 7fa93b7ab700 1 -- 172.21.15.29:0/1227626984 shutdown_connections 2024-01-24T15:32:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.413+0000 7fa93b7ab700 1 --2- 172.21.15.29:0/1227626984 >> v2:172.21.15.163:3300/0 conn(0x7fa934066990 0x7fa93413d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.413+0000 7fa93b7ab700 1 --2- 172.21.15.29:0/1227626984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa934155470 0x7fa9340664a0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.413+0000 7fa93b7ab700 1 -- 172.21.15.29:0/1227626984 >> 172.21.15.29:0/1227626984 conn(0x7fa9340c5640 msgr2=0x7fa9340c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.413+0000 7fa93b7ab700 1 -- 172.21.15.29:0/1227626984 shutdown_connections 2024-01-24T15:32:52.417 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.413+0000 7fa93b7ab700 1 -- 172.21.15.29:0/1227626984 wait complete. 2024-01-24T15:32:52.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.414+0000 7fa93b7ab700 1 Processor -- start 2024-01-24T15:32:52.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.414+0000 7fa93b7ab700 1 -- start start 2024-01-24T15:32:52.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.415+0000 7fa93b7ab700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa934066990 0x7fa93413d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.419 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.415+0000 7fa93b7ab700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa934151de0 0x7fa934156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.415+0000 7fa93b7ab700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa934155470 0x7fa93413c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.415+0000 7fa93b7ab700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9341579b0 con 0x7fa934151de0 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.415+0000 7fa93b7ab700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa934157110 con 0x7fa934066990 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.415+0000 7fa93b7ab700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9341573f0 con 0x7fa934155470 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.415+0000 7fa938d46700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa934151de0 0x7fa934156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa938d46700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa934151de0 0x7fa934156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52652/0 (socket says 172.21.15.29:52652) 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa938d46700 1 -- 172.21.15.29:0/2153089216 learned_addr learned my addr 172.21.15.29:0/2153089216 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa939547700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa934066990 0x7fa93413d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa939d48700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa934155470 0x7fa93413c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa938d46700 1 -- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa934155470 msgr2=0x7fa93413c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa938d46700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa934155470 0x7fa93413c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa938d46700 1 -- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa934066990 msgr2=0x7fa93413d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa938d46700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa934066990 0x7fa93413d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.420 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa938d46700 1 -- 172.21.15.29:0/2153089216 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa924009370 con 0x7fa934151de0 2024-01-24T15:32:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa938d46700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa934151de0 0x7fa934156930 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 rx=0x7fa93406a0c0 tx=0x7fa924017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa9297fa700 1 -- 172.21.15.29:0/2153089216 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92400be90 con 0x7fa934151de0 2024-01-24T15:32:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa9297fa700 1 -- 172.21.15.29:0/2153089216 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92400b7f0 con 0x7fa934151de0 2024-01-24T15:32:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa93b7ab700 1 -- 172.21.15.29:0/2153089216 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9341576d0 con 0x7fa934151de0 2024-01-24T15:32:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa9297fa700 1 -- 172.21.15.29:0/2153089216 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92401c670 con 0x7fa934151de0 2024-01-24T15:32:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.416+0000 7fa93b7ab700 1 -- 172.21.15.29:0/2153089216 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa934156e20 con 0x7fa934151de0 2024-01-24T15:32:52.421 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.417+0000 7fa93b7ab700 1 -- 172.21.15.29:0/2153089216 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9340475e0 con 0x7fa934151de0 2024-01-24T15:32:52.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.418+0000 7fa9297fa700 1 -- 172.21.15.29:0/2153089216 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa92401c9e0 con 0x7fa934151de0 2024-01-24T15:32:52.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.418+0000 7fa9297fa700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa91805f270 0x7fa918061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.418+0000 7fa9297fa700 1 -- 172.21.15.29:0/2153089216 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fa92407bee0 con 0x7fa934151de0 2024-01-24T15:32:52.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.419+0000 7fa939547700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa91805f270 0x7fa918061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.422 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.419+0000 7fa939547700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa91805f270 0x7fa918061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fa934157d10 tx=0x7fa930009040).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.424 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.423+0000 7fa9297fa700 1 -- 172.21.15.29:0/2153089216 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fa92404cc50 con 0x7fa934151de0 2024-01-24T15:32:52.606 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.604+0000 7fa93b7ab700 1 -- 172.21.15.29:0/2153089216 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa93413e020 con 0x7fa91805f270 2024-01-24T15:32:52.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.605+0000 7fa9297fa700 1 -- 172.21.15.29:0/2153089216 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa93413e020 con 0x7fa91805f270 2024-01-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.606+0000 7fa93b7ab700 1 -- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa91805f270 msgr2=0x7fa918061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.606+0000 7fa93b7ab700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa91805f270 0x7fa918061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fa934157d10 tx=0x7fa930009040).stop 2024-01-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.606+0000 7fa93b7ab700 1 -- 172.21.15.29:0/2153089216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa934151de0 msgr2=0x7fa934156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.606+0000 7fa93b7ab700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa934151de0 0x7fa934156930 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 rx=0x7fa93406a0c0 tx=0x7fa924017000).stop 2024-01-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.606+0000 7fa93b7ab700 1 -- 172.21.15.29:0/2153089216 shutdown_connections 2024-01-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.606+0000 7fa93b7ab700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fa91805f270 0x7fa918061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.606+0000 7fa93b7ab700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa934155470 0x7fa93413c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.606+0000 7fa93b7ab700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa934066990 0x7fa93413d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.606+0000 7fa93b7ab700 1 --2- 172.21.15.29:0/2153089216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa934151de0 0x7fa934156930 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.607+0000 7fa93b7ab700 1 -- 172.21.15.29:0/2153089216 >> 172.21.15.29:0/2153089216 conn(0x7fa9340c5640 msgr2=0x7fa9340bf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:52.609 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.607+0000 7fa93b7ab700 1 -- 172.21.15.29:0/2153089216 shutdown_connections 2024-01-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.607+0000 7fa93b7ab700 1 -- 172.21.15.29:0/2153089216 wait complete. 2024-01-24T15:32:52.610 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 4 to deep-scrub 2024-01-24T15:32:52.622 INFO:tasks.ceph:Scrubbing osd.5 2024-01-24T15:32:52.622 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-01-24T15:32:52.756 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.754+0000 7f426c913700 1 Processor -- start 2024-01-24T15:32:52.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.754+0000 7f426c913700 1 -- start start 2024-01-24T15:32:52.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.755+0000 7f426c913700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4268066450 0x7f4268066830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.755+0000 7f426c913700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42681530b0 con 0x7f4268159da0 2024-01-24T15:32:52.759 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.755+0000 7f426c913700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4268153300 con 0x7f4268152540 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.755+0000 7f426c913700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4268067b10 con 0x7f4268066450 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f4266d9d700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4268159da0 0x7f426815c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54028/0 (socket says 172.21.15.29:54028) 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f4266d9d700 1 -- 172.21.15.29:0/3665031967 learned_addr learned my addr 172.21.15.29:0/3665031967 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f426659c700 1 --2- 172.21.15.29:0/3665031967 >> v2:172.21.15.163:3300/0 conn(0x7f4268066450 0x7f4268066830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f426559a700 1 -- 172.21.15.29:0/3665031967 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717157262 0 0) 0x7f42681530b0 con 0x7f4268159da0 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f426559a700 1 -- 172.21.15.29:0/3665031967 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42500033f0 con 0x7f4268159da0 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f426559a700 1 -- 172.21.15.29:0/3665031967 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680272691 0 0) 0x7f4268153300 con 0x7f4268152540 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f426559a700 1 -- 172.21.15.29:0/3665031967 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42681530b0 con 0x7f4268152540 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f426659c700 1 -- 172.21.15.29:0/3665031967 >> v1:172.21.15.163:6789/0 conn(0x7f4268152540 legacy=0x7f42681529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f426659c700 1 -- 172.21.15.29:0/3665031967 >> v1:172.21.15.29:6789/0 conn(0x7f4268159da0 legacy=0x7f426815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f426659c700 1 -- 172.21.15.29:0/3665031967 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4268067df0 con 0x7f4268066450 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f426659c700 1 --2- 172.21.15.29:0/3665031967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4268066450 0x7f4268066830 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f425c009760 tx=0x7f425c009060).ready entity=mon.1 client_cookie=8976c73a044a9d50 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f426559a700 1 -- 172.21.15.29:0/3665031967 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c00d4a0 con 0x7f4268066450 2024-01-24T15:32:52.760 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.756+0000 7f426559a700 1 -- 172.21.15.29:0/3665031967 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c00d600 con 0x7f4268066450 2024-01-24T15:32:52.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.757+0000 7f426559a700 1 -- 172.21.15.29:0/3665031967 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c00d980 con 0x7f4268066450 2024-01-24T15:32:52.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.757+0000 7f426c913700 1 -- 172.21.15.29:0/3665031967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4268066450 msgr2=0x7f4268066830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.757+0000 7f426c913700 1 --2- 172.21.15.29:0/3665031967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4268066450 0x7f4268066830 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f425c009760 tx=0x7f425c009060).stop 2024-01-24T15:32:52.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.757+0000 7f426c913700 1 -- 172.21.15.29:0/3665031967 shutdown_connections 2024-01-24T15:32:52.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.757+0000 7f426c913700 1 --2- 172.21.15.29:0/3665031967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4268066450 0x7f4268066830 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.757+0000 7f426c913700 1 -- 172.21.15.29:0/3665031967 >> 172.21.15.29:0/3665031967 conn(0x7f42680c8bd0 msgr2=0x7f42680c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:52.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.757+0000 7f426c913700 1 -- 172.21.15.29:0/3665031967 shutdown_connections 2024-01-24T15:32:52.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.757+0000 7f426c913700 1 -- 172.21.15.29:0/3665031967 wait complete. 2024-01-24T15:32:52.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.758+0000 7f426c913700 1 Processor -- start 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.758+0000 7f426c913700 1 -- start start 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.759+0000 7f426c913700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4268066450 0x7f42680c1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.759+0000 7f426c913700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4268152540 0x7f4268137e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.759+0000 7f426c913700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4268159da0 0x7f4268150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.759+0000 7f426c913700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42680c2500 con 0x7f4268066450 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.759+0000 7f426c913700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426813b8f0 con 0x7f4268152540 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.759+0000 7f426c913700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42680c2990 con 0x7f4268159da0 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.759+0000 7f426659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4268066450 0x7f42680c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.759+0000 7f426659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4268066450 0x7f42680c1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52666/0 (socket says 172.21.15.29:52666) 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.759+0000 7f426659c700 1 -- 172.21.15.29:0/2031630251 learned_addr learned my addr 172.21.15.29:0/2031630251 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f4266d9d700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4268159da0 0x7f4268150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f4265d9b700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4268152540 0x7f4268137e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f426659c700 1 -- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4268159da0 msgr2=0x7f4268150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f426659c700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4268159da0 0x7f4268150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f426659c700 1 -- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4268152540 msgr2=0x7f4268137e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f426659c700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4268152540 0x7f4268137e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f426659c700 1 -- 172.21.15.29:0/2031630251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0092a0 con 0x7f4268066450 2024-01-24T15:32:52.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f426659c700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4268066450 0x7f42680c1260 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 rx=0x7f426806a130 tx=0x7f425c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f42567fc700 1 -- 172.21.15.29:0/2031630251 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c00d770 con 0x7f4268066450 2024-01-24T15:32:52.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f42567fc700 1 -- 172.21.15.29:0/2031630251 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c00d8d0 con 0x7f4268066450 2024-01-24T15:32:52.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f426c913700 1 -- 172.21.15.29:0/2031630251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42680c18f0 con 0x7f4268066450 2024-01-24T15:32:52.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f42567fc700 1 -- 172.21.15.29:0/2031630251 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c01d660 con 0x7f4268066450 2024-01-24T15:32:52.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.760+0000 7f426c913700 1 -- 172.21.15.29:0/2031630251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42680c1eb0 con 0x7f4268066450 2024-01-24T15:32:52.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.761+0000 7f426c913700 1 -- 172.21.15.29:0/2031630251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42680475e0 con 0x7f4268066450 2024-01-24T15:32:52.765 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.762+0000 7f42567fc700 1 -- 172.21.15.29:0/2031630251 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f425c01da50 con 0x7f4268066450 2024-01-24T15:32:52.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.762+0000 7f42567fc700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f424805f270 0x7f4248061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.762+0000 7f42567fc700 1 -- 172.21.15.29:0/2031630251 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f425c07c970 con 0x7f4268066450 2024-01-24T15:32:52.766 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.762+0000 7f42567fc700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] conn(0x7f4248066a90 0x7f4248068f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.762+0000 7f4265d9b700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f424805f270 0x7f4248061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.762+0000 7f42567fc700 1 -- 172.21.15.29:0/2031630251 --> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4248069510 con 0x7f4248066a90 2024-01-24T15:32:52.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.762+0000 7f42567fc700 1 -- 172.21.15.29:0/2031630251 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=237) v2 ==== 24+0+0 (secure 0 0 0) 0x7f425c07cd50 con 0x7f4268066450 2024-01-24T15:32:52.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.762+0000 7f4266d9d700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] conn(0x7f4248066a90 0x7f4248068f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.763+0000 7f4265d9b700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f424805f270 0x7f4248061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f426814c980 tx=0x7f4258009040).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.763+0000 7f4266d9d700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] conn(0x7f4248066a90 0x7f4248068f30 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.763+0000 7f42567fc700 1 -- 172.21.15.29:0/2031630251 <== osd.5 v2:172.21.15.163:6817/43926 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4248069510 con 0x7f4248066a90 2024-01-24T15:32:52.779 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.778+0000 7f426c913700 1 -- 172.21.15.29:0/2031630251 --> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f42680c2640 con 0x7f4248066a90 2024-01-24T15:32:52.780 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.778+0000 7f42567fc700 1 -- 172.21.15.29:0/2031630251 <== osd.5 v2:172.21.15.163:6817/43926 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4260001f00 con 0x7f4248066a90 2024-01-24T15:32:52.780 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:32:52.780 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2024-01-24T15:32:52.780 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:32:52.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.778+0000 7f426c913700 1 -- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] conn(0x7f4248066a90 msgr2=0x7f4248068f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.778+0000 7f426c913700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:6817/43926,v1:172.21.15.163:6820/43926] conn(0x7f4248066a90 0x7f4248068f30 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.781 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.778+0000 7f426c913700 1 -- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f424805f270 msgr2=0x7f4248061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.778+0000 7f426c913700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f424805f270 0x7f4248061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f426814c980 tx=0x7f4258009040).stop 2024-01-24T15:32:52.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.778+0000 7f426c913700 1 -- 172.21.15.29:0/2031630251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4268066450 msgr2=0x7f42680c1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.778+0000 7f426c913700 1 --2- 172.21.15.29:0/2031630251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4268066450 0x7f42680c1260 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 rx=0x7f426806a130 tx=0x7f425c016000).stop 2024-01-24T15:32:52.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.778+0000 7f4266d9d700 1 -- 172.21.15.29:0/2031630251 reap_dead start 2024-01-24T15:32:52.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.779+0000 7f426c913700 1 -- 172.21.15.29:0/2031630251 shutdown_connections 2024-01-24T15:32:52.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.779+0000 7f426c913700 1 -- 172.21.15.29:0/2031630251 >> 172.21.15.29:0/2031630251 conn(0x7f42680c8bd0 msgr2=0x7f42680bdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:52.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.779+0000 7f426c913700 1 -- 172.21.15.29:0/2031630251 shutdown_connections 2024-01-24T15:32:52.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.779+0000 7f426c913700 1 -- 172.21.15.29:0/2031630251 wait complete. 2024-01-24T15:32:52.794 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-01-24T15:32:52.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.929+0000 7efe7d719700 1 Processor -- start 2024-01-24T15:32:52.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.929+0000 7efe7d719700 1 -- start start 2024-01-24T15:32:52.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.930+0000 7efe7d719700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efe78066bb0 0x7efe781525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.930+0000 7efe7d719700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efe78152ae0 0x7efe780bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.930+0000 7efe7d719700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe78156460 con 0x7efe78135350 2024-01-24T15:32:52.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.930+0000 7efe7d719700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe78155c00 con 0x7efe78152ae0 2024-01-24T15:32:52.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.930+0000 7efe7d719700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe78153390 con 0x7efe78066bb0 2024-01-24T15:32:52.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.930+0000 7efe777fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efe78152ae0 0x7efe780bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.930+0000 7efe777fe700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efe78152ae0 0x7efe780bdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52674/0 (socket says 172.21.15.29:52674) 2024-01-24T15:32:52.934 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.930+0000 7efe777fe700 1 -- 172.21.15.29:0/2506605505 learned_addr learned my addr 172.21.15.29:0/2506605505 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.930+0000 7efe767fc700 1 --2- 172.21.15.29:0/2506605505 >> v2:172.21.15.163:3300/0 conn(0x7efe78066bb0 0x7efe781525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.931+0000 7efe777fe700 1 -- 172.21.15.29:0/2506605505 >> v2:172.21.15.163:3300/0 conn(0x7efe78066bb0 msgr2=0x7efe781525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.931+0000 7efe777fe700 1 --2- 172.21.15.29:0/2506605505 >> v2:172.21.15.163:3300/0 conn(0x7efe78066bb0 0x7efe781525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.931+0000 7efe777fe700 1 -- 172.21.15.29:0/2506605505 >> v1:172.21.15.163:6789/0 conn(0x7efe78135350 legacy=0x7efe780664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.931+0000 7efe777fe700 1 -- 172.21.15.29:0/2506605505 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe78155e10 con 0x7efe78152ae0 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.931+0000 7efe767fc700 1 --2- 172.21.15.29:0/2506605505 >> v2:172.21.15.163:3300/0 conn(0x7efe78066bb0 0x7efe781525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.931+0000 7efe777fe700 1 --2- 172.21.15.29:0/2506605505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe78152ae0 0x7efe780bdb00 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 rx=0x7efe7805f220 tx=0x7efe700090f0).ready entity=mon.0 client_cookie=ee15f5290ed09f34 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.931+0000 7efe75ffb700 1 -- 172.21.15.29:0/2506605505 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7000b520 con 0x7efe78152ae0 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.931+0000 7efe75ffb700 1 -- 172.21.15.29:0/2506605505 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7000b680 con 0x7efe78152ae0 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.931+0000 7efe75ffb700 1 -- 172.21.15.29:0/2506605505 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7000ba00 con 0x7efe78152ae0 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.932+0000 7efe7d719700 1 -- 172.21.15.29:0/2506605505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe78152ae0 msgr2=0x7efe780bdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.932+0000 7efe7d719700 1 --2- 172.21.15.29:0/2506605505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe78152ae0 0x7efe780bdb00 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 rx=0x7efe7805f220 tx=0x7efe700090f0).stop 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.932+0000 7efe7d719700 1 -- 172.21.15.29:0/2506605505 shutdown_connections 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.932+0000 7efe7d719700 1 --2- 172.21.15.29:0/2506605505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe78152ae0 0x7efe780bdb00 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.935 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.932+0000 7efe7d719700 1 --2- 172.21.15.29:0/2506605505 >> v2:172.21.15.163:3300/0 conn(0x7efe78066bb0 0x7efe781525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.932+0000 7efe7d719700 1 -- 172.21.15.29:0/2506605505 >> 172.21.15.29:0/2506605505 conn(0x7efe780c5640 msgr2=0x7efe780c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:52.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.932+0000 7efe7d719700 1 -- 172.21.15.29:0/2506605505 shutdown_connections 2024-01-24T15:32:52.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.932+0000 7efe7d719700 1 -- 172.21.15.29:0/2506605505 wait complete. 2024-01-24T15:32:52.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.934+0000 7efe7d719700 1 Processor -- start 2024-01-24T15:32:52.936 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.934+0000 7efe7d719700 1 -- start start 2024-01-24T15:32:52.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.935+0000 7efe7d719700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe78066bb0 0x7efe78151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.935+0000 7efe7d719700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe78135350 0x7efe78137af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.935+0000 7efe7d719700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe78152ae0 0x7efe78137fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.935+0000 7efe7d719700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe781384d0 con 0x7efe78066bb0 2024-01-24T15:32:52.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.935+0000 7efe7d719700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe78135cd0 con 0x7efe78152ae0 2024-01-24T15:32:52.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.935+0000 7efe7d719700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe780ae690 con 0x7efe78135350 2024-01-24T15:32:52.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.935+0000 7efe76ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe78066bb0 0x7efe78151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.935+0000 7efe76ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe78066bb0 0x7efe78151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52678/0 (socket says 172.21.15.29:52678) 2024-01-24T15:32:52.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.935+0000 7efe76ffd700 1 -- 172.21.15.29:0/1525053404 learned_addr learned my addr 172.21.15.29:0/1525053404 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:52.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.935+0000 7efe777fe700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe78152ae0 0x7efe78137fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.939 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.935+0000 7efe767fc700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe78135350 0x7efe78137af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.936+0000 7efe76ffd700 1 -- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe78135350 msgr2=0x7efe78137af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.936+0000 7efe76ffd700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe78135350 0x7efe78137af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.936+0000 7efe76ffd700 1 -- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe78152ae0 msgr2=0x7efe78137fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:52.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.936+0000 7efe76ffd700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe78152ae0 0x7efe78137fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:52.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.936+0000 7efe76ffd700 1 -- 172.21.15.29:0/1525053404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c0092a0 con 0x7efe78066bb0 2024-01-24T15:32:52.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.936+0000 7efe76ffd700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe78066bb0 0x7efe78151520 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 rx=0x7efe6800c840 tx=0x7efe68001660).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.936+0000 7efe5effd700 1 -- 172.21.15.29:0/1525053404 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68006a20 con 0x7efe78066bb0 2024-01-24T15:32:52.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.936+0000 7efe5effd700 1 -- 172.21.15.29:0/1525053404 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe68006b80 con 0x7efe78066bb0 2024-01-24T15:32:52.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.936+0000 7efe7d719700 1 -- 172.21.15.29:0/1525053404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe70009370 con 0x7efe78066bb0 2024-01-24T15:32:52.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.936+0000 7efe5effd700 1 -- 172.21.15.29:0/1525053404 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68007020 con 0x7efe78066bb0 2024-01-24T15:32:52.940 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.936+0000 7efe7d719700 1 -- 172.21.15.29:0/1525053404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe78135960 con 0x7efe78066bb0 2024-01-24T15:32:52.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.937+0000 7efe7d719700 1 -- 172.21.15.29:0/1525053404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe780aebb0 con 0x7efe78066bb0 2024-01-24T15:32:52.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.938+0000 7efe5effd700 1 -- 172.21.15.29:0/1525053404 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7efe68007390 con 0x7efe78066bb0 2024-01-24T15:32:52.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.939+0000 7efe5effd700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efe6405f270 0x7efe64061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:52.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.939+0000 7efe5effd700 1 -- 172.21.15.29:0/1525053404 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7efe68079f50 con 0x7efe78066bb0 2024-01-24T15:32:52.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.939+0000 7efe767fc700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efe6405f270 0x7efe64061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:52.942 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.940+0000 7efe767fc700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efe6405f270 0x7efe64061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7efe78152850 tx=0x7efe6c009060).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:52.945 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:52.943+0000 7efe5effd700 1 -- 172.21.15.29:0/1525053404 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7efe68047b00 con 0x7efe78066bb0 2024-01-24T15:32:53.126 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.124+0000 7efe7d719700 1 -- 172.21.15.29:0/1525053404 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7efe780475e0 con 0x7efe6405f270 2024-01-24T15:32:53.127 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.124+0000 7efe5effd700 1 -- 172.21.15.29:0/1525053404 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efe780475e0 con 0x7efe6405f270 2024-01-24T15:32:53.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.127+0000 7efe7d719700 1 -- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efe6405f270 msgr2=0x7efe64061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.127+0000 7efe7d719700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efe6405f270 0x7efe64061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7efe78152850 tx=0x7efe6c009060).stop 2024-01-24T15:32:53.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.127+0000 7efe7d719700 1 -- 172.21.15.29:0/1525053404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe78066bb0 msgr2=0x7efe78151520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.127+0000 7efe7d719700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe78066bb0 0x7efe78151520 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 rx=0x7efe6800c840 tx=0x7efe68001660).stop 2024-01-24T15:32:53.129 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.127+0000 7efe7d719700 1 -- 172.21.15.29:0/1525053404 shutdown_connections 2024-01-24T15:32:53.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.127+0000 7efe7d719700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe78152ae0 0x7efe78137fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.127+0000 7efe7d719700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe78066bb0 0x7efe78151520 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.127+0000 7efe7d719700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7efe6405f270 0x7efe64061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.127+0000 7efe7d719700 1 --2- 172.21.15.29:0/1525053404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe78135350 0x7efe78137af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.127+0000 7efe7d719700 1 -- 172.21.15.29:0/1525053404 >> 172.21.15.29:0/1525053404 conn(0x7efe780c5640 msgr2=0x7efe780679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:53.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.127+0000 7efe7d719700 1 -- 172.21.15.29:0/1525053404 shutdown_connections 2024-01-24T15:32:53.130 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.127+0000 7efe7d719700 1 -- 172.21.15.29:0/1525053404 wait complete. 2024-01-24T15:32:53.130 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 5 to deep-scrub 2024-01-24T15:32:53.142 INFO:tasks.ceph:Scrubbing osd.6 2024-01-24T15:32:53.142 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-01-24T15:32:53.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.274+0000 7f636207d700 1 Processor -- start 2024-01-24T15:32:53.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.274+0000 7f636207d700 1 -- start start 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.274+0000 7f636207d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f635c0c2190 0x7f635c062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f636207d700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635c0c3e00 con 0x7f635c062970 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f636207d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635c0c2900 con 0x7f635c158720 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f636207d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635c065720 con 0x7f635c0c2190 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f635b7fe700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f635c062970 0x7f635c0c4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:54036/0 (socket says 172.21.15.29:54036) 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f635b7fe700 1 -- 172.21.15.29:0/1988819251 learned_addr learned my addr 172.21.15.29:0/1988819251 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f635affd700 1 --2- 172.21.15.29:0/1988819251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f635c0c2190 0x7f635c062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f635a7fc700 1 -- 172.21.15.29:0/1988819251 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098635887 0 0) 0x7f635c0c3e00 con 0x7f635c062970 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f635a7fc700 1 -- 172.21.15.29:0/1988819251 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63440033f0 con 0x7f635c062970 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f635a7fc700 1 -- 172.21.15.29:0/1988819251 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1710805448 0 0) 0x7f635c0c2900 con 0x7f635c158720 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f635a7fc700 1 -- 172.21.15.29:0/1988819251 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f635c0c3e00 con 0x7f635c158720 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f635affd700 1 -- 172.21.15.29:0/1988819251 >> v1:172.21.15.163:6789/0 conn(0x7f635c158720 legacy=0x7f635c0a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.279 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f635affd700 1 -- 172.21.15.29:0/1988819251 >> v1:172.21.15.29:6789/0 conn(0x7f635c062970 legacy=0x7f635c0c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f635affd700 1 -- 172.21.15.29:0/1988819251 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c158bd0 con 0x7f635c0c2190 2024-01-24T15:32:53.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.275+0000 7f635affd700 1 --2- 172.21.15.29:0/1988819251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f635c0c2190 0x7f635c062d50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f6350009790 tx=0x7f6350009060).ready entity=mon.2 client_cookie=dd39c3d081e9bf2e server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.276+0000 7f635a7fc700 1 -- 172.21.15.29:0/1988819251 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635000f510 con 0x7f635c0c2190 2024-01-24T15:32:53.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.276+0000 7f635a7fc700 1 -- 172.21.15.29:0/1988819251 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635000f670 con 0x7f635c0c2190 2024-01-24T15:32:53.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.276+0000 7f635a7fc700 1 -- 172.21.15.29:0/1988819251 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635000f9f0 con 0x7f635c0c2190 2024-01-24T15:32:53.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.276+0000 7f636207d700 1 -- 172.21.15.29:0/1988819251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f635c0c2190 msgr2=0x7f635c062d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.276+0000 7f636207d700 1 --2- 172.21.15.29:0/1988819251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f635c0c2190 0x7f635c062d50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f6350009790 tx=0x7f6350009060).stop 2024-01-24T15:32:53.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.276+0000 7f636207d700 1 -- 172.21.15.29:0/1988819251 shutdown_connections 2024-01-24T15:32:53.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.276+0000 7f636207d700 1 --2- 172.21.15.29:0/1988819251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f635c0c2190 0x7f635c062d50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.276+0000 7f636207d700 1 -- 172.21.15.29:0/1988819251 >> 172.21.15.29:0/1988819251 conn(0x7f635c067b20 msgr2=0x7f635c0638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:53.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.276+0000 7f636207d700 1 -- 172.21.15.29:0/1988819251 shutdown_connections 2024-01-24T15:32:53.280 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.276+0000 7f636207d700 1 -- 172.21.15.29:0/1988819251 wait complete. 2024-01-24T15:32:53.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.278+0000 7f636207d700 1 Processor -- start 2024-01-24T15:32:53.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.278+0000 7f636207d700 1 -- start start 2024-01-24T15:32:53.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.278+0000 7f636207d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f635c062970 0x7f635c168560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.278+0000 7f636207d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f635c158720 0x7f635c146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.278+0000 7f636207d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c14b210 0x7f635c1501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.278+0000 7f636207d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c14bb00 con 0x7f635c158720 2024-01-24T15:32:53.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.278+0000 7f636207d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c168a50 con 0x7f635c14b210 2024-01-24T15:32:53.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.278+0000 7f636207d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635c146d50 con 0x7f635c062970 2024-01-24T15:32:53.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f635affd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f635c158720 0x7f635c146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f635affd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f635c158720 0x7f635c146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:52682/0 (socket says 172.21.15.29:52682) 2024-01-24T15:32:53.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f635affd700 1 -- 172.21.15.29:0/2922865159 learned_addr learned my addr 172.21.15.29:0/2922865159 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:53.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f635b7fe700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f635c062970 0x7f635c168560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f635bfff700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c14b210 0x7f635c1501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f635affd700 1 -- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f635c062970 msgr2=0x7f635c168560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f635affd700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f635c062970 0x7f635c168560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f635affd700 1 -- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c14b210 msgr2=0x7f635c1501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f635affd700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c14b210 0x7f635c1501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f635affd700 1 -- 172.21.15.29:0/2922865159 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63500092a0 con 0x7f635c158720 2024-01-24T15:32:53.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f635bfff700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c14b210 0x7f635c1501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:32:53.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f635affd700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f635c158720 0x7f635c146860 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 rx=0x7f635c068d70 tx=0x7f6350017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f63437fe700 1 -- 172.21.15.29:0/2922865159 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635000f7e0 con 0x7f635c158720 2024-01-24T15:32:53.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f63437fe700 1 -- 172.21.15.29:0/2922865159 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635000f940 con 0x7f635c158720 2024-01-24T15:32:53.285 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f636207d700 1 -- 172.21.15.29:0/2922865159 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635c0bb5a0 con 0x7f635c158720 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f63437fe700 1 -- 172.21.15.29:0/2922865159 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635001b660 con 0x7f635c158720 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.279+0000 7f636207d700 1 -- 172.21.15.29:0/2922865159 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635c0bbb60 con 0x7f635c158720 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.280+0000 7f636207d700 1 -- 172.21.15.29:0/2922865159 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f635c0475e0 con 0x7f635c158720 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.281+0000 7f63437fe700 1 -- 172.21.15.29:0/2922865159 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f635001b9d0 con 0x7f635c158720 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.281+0000 7f63437fe700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f634805f270 0x7f6348061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.281+0000 7f63437fe700 1 -- 172.21.15.29:0/2922865159 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f635007bb40 con 0x7f635c158720 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.281+0000 7f63437fe700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] conn(0x7f6348066a90 0x7f6348068f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.281+0000 7f635b7fe700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f634805f270 0x7f6348061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.281+0000 7f63437fe700 1 -- 172.21.15.29:0/2922865159 --> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6348069510 con 0x7f6348066a90 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.281+0000 7f63437fe700 1 -- 172.21.15.29:0/2922865159 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=237) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6350080240 con 0x7f635c158720 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.281+0000 7f635bfff700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] conn(0x7f6348066a90 0x7f6348068f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.282+0000 7f635b7fe700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f634805f270 0x7f6348061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f634c005750 tx=0x7f634c009040).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.282+0000 7f635bfff700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] conn(0x7f6348066a90 0x7f6348068f30 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.283+0000 7f63437fe700 1 -- 172.21.15.29:0/2922865159 <== osd.6 v2:172.21.15.163:6808/43897 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6348069510 con 0x7f6348066a90 2024-01-24T15:32:53.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.297+0000 7f636207d700 1 -- 172.21.15.29:0/2922865159 --> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f635c0c2620 con 0x7f6348066a90 2024-01-24T15:32:53.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.297+0000 7f63437fe700 1 -- 172.21.15.29:0/2922865159 <== osd.6 v2:172.21.15.163:6808/43897 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f635c0c2620 con 0x7f6348066a90 2024-01-24T15:32:53.299 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:32:53.300 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2024-01-24T15:32:53.300 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:32:53.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.298+0000 7f636207d700 1 -- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] conn(0x7f6348066a90 msgr2=0x7f6348068f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.298+0000 7f636207d700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:6808/43897,v1:172.21.15.163:6809/43897] conn(0x7f6348066a90 0x7f6348068f30 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.298+0000 7f636207d700 1 -- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f634805f270 msgr2=0x7f6348061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.298+0000 7f636207d700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f634805f270 0x7f6348061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f634c005750 tx=0x7f634c009040).stop 2024-01-24T15:32:53.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.298+0000 7f636207d700 1 -- 172.21.15.29:0/2922865159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f635c158720 msgr2=0x7f635c146860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.298+0000 7f636207d700 1 --2- 172.21.15.29:0/2922865159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f635c158720 0x7f635c146860 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 rx=0x7f635c068d70 tx=0x7f6350017000).stop 2024-01-24T15:32:53.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.298+0000 7f635bfff700 1 -- 172.21.15.29:0/2922865159 reap_dead start 2024-01-24T15:32:53.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.298+0000 7f636207d700 1 -- 172.21.15.29:0/2922865159 shutdown_connections 2024-01-24T15:32:53.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.298+0000 7f636207d700 1 -- 172.21.15.29:0/2922865159 >> 172.21.15.29:0/2922865159 conn(0x7f635c067b20 msgr2=0x7f635c0ab150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:53.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.298+0000 7f636207d700 1 -- 172.21.15.29:0/2922865159 shutdown_connections 2024-01-24T15:32:53.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.298+0000 7f636207d700 1 -- 172.21.15.29:0/2922865159 wait complete. 2024-01-24T15:32:53.313 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-01-24T15:32:53.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.441+0000 7fd4d488b700 1 Processor -- start 2024-01-24T15:32:53.443 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.441+0000 7fd4d488b700 1 -- start start 2024-01-24T15:32:53.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.442+0000 7fd4d488b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4d0155470 0x7fd4d00664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.442+0000 7fd4d488b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd4d0066990 0x7fd4d013d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.442+0000 7fd4d488b700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d0158800 con 0x7fd4d0151de0 2024-01-24T15:32:53.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.442+0000 7fd4d488b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d0066e20 con 0x7fd4d0066990 2024-01-24T15:32:53.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.442+0000 7fd4d488b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d0157ed0 con 0x7fd4d0155470 2024-01-24T15:32:53.445 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.442+0000 7fd4ce59c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fd4d0151de0 0x7fd4d00638c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:60598/0 (socket says 172.21.15.29:60598) 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.442+0000 7fd4ce59c700 1 -- 172.21.15.29:0/1990260638 learned_addr learned my addr 172.21.15.29:0/1990260638 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.442+0000 7fd4cdd9b700 1 --2- 172.21.15.29:0/1990260638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4d0155470 0x7fd4d00664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.442+0000 7fd4ced9d700 1 --2- 172.21.15.29:0/1990260638 >> v2:172.21.15.163:3300/0 conn(0x7fd4d0066990 0x7fd4d013d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.442+0000 7fd4cd59a700 1 -- 172.21.15.29:0/1990260638 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655325984 0 0) 0x7fd4d0158800 con 0x7fd4d0151de0 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.443+0000 7fd4cd59a700 1 -- 172.21.15.29:0/1990260638 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4b80033f0 con 0x7fd4d0151de0 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.443+0000 7fd4cdd9b700 1 -- 172.21.15.29:0/1990260638 >> v2:172.21.15.163:3300/0 conn(0x7fd4d0066990 msgr2=0x7fd4d013d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.443+0000 7fd4cdd9b700 1 --2- 172.21.15.29:0/1990260638 >> v2:172.21.15.163:3300/0 conn(0x7fd4d0066990 0x7fd4d013d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.443+0000 7fd4cdd9b700 1 -- 172.21.15.29:0/1990260638 >> v1:172.21.15.29:6789/0 conn(0x7fd4d0151de0 legacy=0x7fd4d00638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.443+0000 7fd4cdd9b700 1 -- 172.21.15.29:0/1990260638 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d01581b0 con 0x7fd4d0155470 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.443+0000 7fd4cdd9b700 1 --2- 172.21.15.29:0/1990260638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4d0155470 0x7fd4d00664a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fd4c4009760 tx=0x7fd4c4009060).ready entity=mon.2 client_cookie=4708b560fd566080 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.443+0000 7fd4cd59a700 1 -- 172.21.15.29:0/1990260638 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400d4a0 con 0x7fd4d0155470 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.443+0000 7fd4cd59a700 1 -- 172.21.15.29:0/1990260638 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c400d600 con 0x7fd4d0155470 2024-01-24T15:32:53.446 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.443+0000 7fd4cd59a700 1 -- 172.21.15.29:0/1990260638 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400d980 con 0x7fd4d0155470 2024-01-24T15:32:53.449 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.444+0000 7fd4d488b700 1 -- 172.21.15.29:0/1990260638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4d0155470 msgr2=0x7fd4d00664a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.444+0000 7fd4d488b700 1 --2- 172.21.15.29:0/1990260638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4d0155470 0x7fd4d00664a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fd4c4009760 tx=0x7fd4c4009060).stop 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.444+0000 7fd4d488b700 1 -- 172.21.15.29:0/1990260638 shutdown_connections 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.444+0000 7fd4d488b700 1 --2- 172.21.15.29:0/1990260638 >> v2:172.21.15.163:3300/0 conn(0x7fd4d0066990 0x7fd4d013d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.444+0000 7fd4d488b700 1 --2- 172.21.15.29:0/1990260638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4d0155470 0x7fd4d00664a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.444+0000 7fd4d488b700 1 -- 172.21.15.29:0/1990260638 >> 172.21.15.29:0/1990260638 conn(0x7fd4d00c5640 msgr2=0x7fd4d00c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.444+0000 7fd4d488b700 1 -- 172.21.15.29:0/1990260638 shutdown_connections 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.444+0000 7fd4d488b700 1 -- 172.21.15.29:0/1990260638 wait complete. 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.445+0000 7fd4d488b700 1 Processor -- start 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.445+0000 7fd4d488b700 1 -- start start 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4d488b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4d0066990 0x7fd4d013c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4d488b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4d0151de0 0x7fd4d0165680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4d488b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d0155470 0x7fd4d00c0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.450 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4d488b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00c3a80 con 0x7fd4d0155470 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4d488b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00c3150 con 0x7fd4d0066990 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4d488b700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00c3430 con 0x7fd4d0151de0 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4ced9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d0155470 0x7fd4d00c0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4ced9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d0155470 0x7fd4d00c0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49720/0 (socket says 172.21.15.29:49720) 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4ced9d700 1 -- 172.21.15.29:0/631365693 learned_addr learned my addr 172.21.15.29:0/631365693 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4ce59c700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4d0066990 0x7fd4d013c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4cdd9b700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4d0151de0 0x7fd4d0165680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4ced9d700 1 -- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4d0151de0 msgr2=0x7fd4d0165680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4ced9d700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4d0151de0 0x7fd4d0165680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4ced9d700 1 -- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4d0066990 msgr2=0x7fd4d013c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4ced9d700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4d0066990 0x7fd4d013c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4ced9d700 1 -- 172.21.15.29:0/631365693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c40092a0 con 0x7fd4d0155470 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4cdd9b700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4d0151de0 0x7fd4d0165680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:53.451 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.446+0000 7fd4ced9d700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d0155470 0x7fd4d00c0610 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 rx=0x7fd4c8009e90 tx=0x7fd4c800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.447+0000 7fd4be7fc700 1 -- 172.21.15.29:0/631365693 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c800cb50 con 0x7fd4d0155470 2024-01-24T15:32:53.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.447+0000 7fd4be7fc700 1 -- 172.21.15.29:0/631365693 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c800f040 con 0x7fd4d0155470 2024-01-24T15:32:53.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.447+0000 7fd4d488b700 1 -- 172.21.15.29:0/631365693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d00c3710 con 0x7fd4d0155470 2024-01-24T15:32:53.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.447+0000 7fd4be7fc700 1 -- 172.21.15.29:0/631365693 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c800cec0 con 0x7fd4d0155470 2024-01-24T15:32:53.452 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.447+0000 7fd4d488b700 1 -- 172.21.15.29:0/631365693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d0165db0 con 0x7fd4d0155470 2024-01-24T15:32:53.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.448+0000 7fd4d488b700 1 -- 172.21.15.29:0/631365693 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d00c3bf0 con 0x7fd4d0155470 2024-01-24T15:32:53.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.448+0000 7fd4be7fc700 1 -- 172.21.15.29:0/631365693 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd4c8013610 con 0x7fd4d0155470 2024-01-24T15:32:53.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.449+0000 7fd4be7fc700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd4b005f270 0x7fd4b0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.449+0000 7fd4be7fc700 1 -- 172.21.15.29:0/631365693 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fd4c807aff0 con 0x7fd4d0155470 2024-01-24T15:32:53.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.449+0000 7fd4ce59c700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd4b005f270 0x7fd4b0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.453 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.449+0000 7fd4ce59c700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd4b005f270 0x7fd4b0061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd4c0005750 tx=0x7fd4c0009040).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.453+0000 7fd4be7fc700 1 -- 172.21.15.29:0/631365693 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fd4c804f850 con 0x7fd4d0155470 2024-01-24T15:32:53.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.635+0000 7fd4d488b700 1 -- 172.21.15.29:0/631365693 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4d00475e0 con 0x7fd4b005f270 2024-01-24T15:32:53.638 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.636+0000 7fd4be7fc700 1 -- 172.21.15.29:0/631365693 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd4d00475e0 con 0x7fd4b005f270 2024-01-24T15:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.638+0000 7fd4d488b700 1 -- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd4b005f270 msgr2=0x7fd4b0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.638+0000 7fd4d488b700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd4b005f270 0x7fd4b0061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd4c0005750 tx=0x7fd4c0009040).stop 2024-01-24T15:32:53.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.638+0000 7fd4d488b700 1 -- 172.21.15.29:0/631365693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d0155470 msgr2=0x7fd4d00c0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.638+0000 7fd4d488b700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d0155470 0x7fd4d00c0610 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 rx=0x7fd4c8009e90 tx=0x7fd4c800b040).stop 2024-01-24T15:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.638+0000 7fd4d488b700 1 -- 172.21.15.29:0/631365693 shutdown_connections 2024-01-24T15:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.638+0000 7fd4d488b700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4d0155470 0x7fd4d00c0610 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.638+0000 7fd4d488b700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4d0066990 0x7fd4d013c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.638+0000 7fd4d488b700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fd4b005f270 0x7fd4b0061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.638+0000 7fd4d488b700 1 --2- 172.21.15.29:0/631365693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4d0151de0 0x7fd4d0165680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.638+0000 7fd4d488b700 1 -- 172.21.15.29:0/631365693 >> 172.21.15.29:0/631365693 conn(0x7fd4d00c5640 msgr2=0x7fd4d00bdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.638+0000 7fd4d488b700 1 -- 172.21.15.29:0/631365693 shutdown_connections 2024-01-24T15:32:53.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.638+0000 7fd4d488b700 1 -- 172.21.15.29:0/631365693 wait complete. 2024-01-24T15:32:53.642 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 6 to deep-scrub 2024-01-24T15:32:53.653 INFO:tasks.ceph:Scrubbing osd.7 2024-01-24T15:32:53.654 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-01-24T15:32:53.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.786+0000 7f35de0e5700 1 Processor -- start 2024-01-24T15:32:53.788 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.786+0000 7f35de0e5700 1 -- start start 2024-01-24T15:32:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35de0e5700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35d80c31d0 0x7f35d80684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35de0e5700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d8067a70 con 0x7f35d8066b60 2024-01-24T15:32:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35de0e5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d80689b0 con 0x7f35d814fd10 2024-01-24T15:32:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35de0e5700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d80c3d50 con 0x7f35d80c31d0 2024-01-24T15:32:53.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35d7fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35d80c31d0 0x7f35d80684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35d7fff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35d80c31d0 0x7f35d80684c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49728/0 (socket says 172.21.15.29:49728) 2024-01-24T15:32:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35d7fff700 1 -- 172.21.15.29:0/964634602 learned_addr learned my addr 172.21.15.29:0/964634602 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35d6ffd700 1 --1- 172.21.15.29:0/964634602 >> v1:172.21.15.29:6789/0 conn(0x7f35d8066b60 0x7f35d80bdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:60612/0 (socket says 172.21.15.29:60612) 2024-01-24T15:32:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35d7fff700 1 -- 172.21.15.29:0/964634602 >> v1:172.21.15.163:6789/0 conn(0x7f35d814fd10 legacy=0x7f35d8066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35d7fff700 1 -- 172.21.15.29:0/964634602 >> v1:172.21.15.29:6789/0 conn(0x7f35d8066b60 legacy=0x7f35d80bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35d7fff700 1 -- 172.21.15.29:0/964634602 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d80c3790 con 0x7f35d80c31d0 2024-01-24T15:32:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35d7fff700 1 --2- 172.21.15.29:0/964634602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d80c31d0 0x7f35d80684c0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 rx=0x7f35d0009f20 tx=0x7f35d0009840).ready entity=mon.0 client_cookie=7ea750d5babcaa05 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35d67fc700 1 -- 172.21.15.29:0/964634602 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000e070 con 0x7f35d80c31d0 2024-01-24T15:32:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35d67fc700 1 -- 172.21.15.29:0/964634602 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d000ba80 con 0x7f35d80c31d0 2024-01-24T15:32:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.787+0000 7f35d67fc700 1 -- 172.21.15.29:0/964634602 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000be00 con 0x7f35d80c31d0 2024-01-24T15:32:53.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.788+0000 7f35de0e5700 1 -- 172.21.15.29:0/964634602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d80c31d0 msgr2=0x7f35d80684c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.788+0000 7f35de0e5700 1 --2- 172.21.15.29:0/964634602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d80c31d0 0x7f35d80684c0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 rx=0x7f35d0009f20 tx=0x7f35d0009840).stop 2024-01-24T15:32:53.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.788+0000 7f35de0e5700 1 -- 172.21.15.29:0/964634602 shutdown_connections 2024-01-24T15:32:53.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.788+0000 7f35de0e5700 1 --2- 172.21.15.29:0/964634602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d80c31d0 0x7f35d80684c0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.788+0000 7f35de0e5700 1 -- 172.21.15.29:0/964634602 >> 172.21.15.29:0/964634602 conn(0x7f35d80c8bd0 msgr2=0x7f35d80c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:53.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.788+0000 7f35de0e5700 1 -- 172.21.15.29:0/964634602 shutdown_connections 2024-01-24T15:32:53.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.788+0000 7f35de0e5700 1 -- 172.21.15.29:0/964634602 wait complete. 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.789+0000 7f35de0e5700 1 Processor -- start 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.790+0000 7f35de0e5700 1 -- start start 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.790+0000 7f35de0e5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35d8066b60 0x7f35d814d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.790+0000 7f35de0e5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d80c31d0 0x7f35d8138cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.790+0000 7f35de0e5700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35d814fd10 0x7f35d80bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.790+0000 7f35de0e5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c1b80 con 0x7f35d80c31d0 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.790+0000 7f35de0e5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c05e0 con 0x7f35d8066b60 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.790+0000 7f35de0e5700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c08c0 con 0x7f35d814fd10 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.790+0000 7f35d6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d80c31d0 0x7f35d8138cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35d6ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d80c31d0 0x7f35d8138cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49742/0 (socket says 172.21.15.29:49742) 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35d6ffd700 1 -- 172.21.15.29:0/4199297746 learned_addr learned my addr 172.21.15.29:0/4199297746 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35d77fe700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35d8066b60 0x7f35d814d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35d7fff700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35d814fd10 0x7f35d80bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35d6ffd700 1 -- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35d814fd10 msgr2=0x7f35d80bfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35d6ffd700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35d814fd10 0x7f35d80bfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35d6ffd700 1 -- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35d8066b60 msgr2=0x7f35d814d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35d6ffd700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35d8066b60 0x7f35d814d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35d6ffd700 1 -- 172.21.15.29:0/4199297746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d0009ae0 con 0x7f35d80c31d0 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35d6ffd700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d80c31d0 0x7f35d8138cb0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 rx=0x7f35c800c4c0 tx=0x7f35c8007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35cf7fe700 1 -- 172.21.15.29:0/4199297746 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c800ce10 con 0x7f35d80c31d0 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35cf7fe700 1 -- 172.21.15.29:0/4199297746 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c80072e0 con 0x7f35d80c31d0 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35de0e5700 1 -- 172.21.15.29:0/4199297746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d80c0ba0 con 0x7f35d80c31d0 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35cf7fe700 1 -- 172.21.15.29:0/4199297746 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c80056e0 con 0x7f35d80c31d0 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.791+0000 7f35de0e5700 1 -- 172.21.15.29:0/4199297746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d814d680 con 0x7f35d80c31d0 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.792+0000 7f35de0e5700 1 -- 172.21.15.29:0/4199297746 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35d806ae00 con 0x7f35d80c31d0 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.793+0000 7f35cf7fe700 1 -- 172.21.15.29:0/4199297746 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f35c8005b10 con 0x7f35d80c31d0 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.793+0000 7f35cf7fe700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f35b805f270 0x7f35b8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.797 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.793+0000 7f35cf7fe700 1 -- 172.21.15.29:0/4199297746 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f35c8079c90 con 0x7f35d80c31d0 2024-01-24T15:32:53.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.793+0000 7f35cf7fe700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] conn(0x7f35b8066a90 0x7f35b8068f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.793+0000 7f35cf7fe700 1 -- 172.21.15.29:0/4199297746 --> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35b8069510 con 0x7f35b8066a90 2024-01-24T15:32:53.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.793+0000 7f35cf7fe700 1 -- 172.21.15.29:0/4199297746 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=237) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35c807c2e0 con 0x7f35d80c31d0 2024-01-24T15:32:53.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.793+0000 7f35d77fe700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f35b805f270 0x7f35b8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.794+0000 7f35d7fff700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] conn(0x7f35b8066a90 0x7f35b8068f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.794+0000 7f35d77fe700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f35b805f270 0x7f35b8061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f35d80c85c0 tx=0x7f35c0009040).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.794+0000 7f35d7fff700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] conn(0x7f35b8066a90 0x7f35b8068f30 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.795+0000 7f35cf7fe700 1 -- 172.21.15.29:0/4199297746 <== osd.7 v2:172.21.15.163:6816/43927 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f35b8069510 con 0x7f35b8066a90 2024-01-24T15:32:53.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.809+0000 7f35de0e5700 1 -- 172.21.15.29:0/4199297746 --> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f35d80475e0 con 0x7f35b8066a90 2024-01-24T15:32:53.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.809+0000 7f35cf7fe700 1 -- 172.21.15.29:0/4199297746 <== osd.7 v2:172.21.15.163:6816/43927 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f35d80475e0 con 0x7f35b8066a90 2024-01-24T15:32:53.811 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-01-24T15:32:53.812 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2024-01-24T15:32:53.812 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-01-24T15:32:53.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.810+0000 7f35de0e5700 1 -- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] conn(0x7f35b8066a90 msgr2=0x7f35b8068f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.810+0000 7f35de0e5700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:6816/43927,v1:172.21.15.163:6818/43927] conn(0x7f35b8066a90 0x7f35b8068f30 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.810+0000 7f35de0e5700 1 -- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f35b805f270 msgr2=0x7f35b8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.810+0000 7f35de0e5700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f35b805f270 0x7f35b8061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f35d80c85c0 tx=0x7f35c0009040).stop 2024-01-24T15:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.810+0000 7f35de0e5700 1 -- 172.21.15.29:0/4199297746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d80c31d0 msgr2=0x7f35d8138cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.810+0000 7f35de0e5700 1 --2- 172.21.15.29:0/4199297746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35d80c31d0 0x7f35d8138cb0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 rx=0x7f35c800c4c0 tx=0x7f35c8007dc0).stop 2024-01-24T15:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.810+0000 7f35d7fff700 1 -- 172.21.15.29:0/4199297746 reap_dead start 2024-01-24T15:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.810+0000 7f35de0e5700 1 -- 172.21.15.29:0/4199297746 shutdown_connections 2024-01-24T15:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.810+0000 7f35de0e5700 1 -- 172.21.15.29:0/4199297746 >> 172.21.15.29:0/4199297746 conn(0x7f35d80c8bd0 msgr2=0x7f35d8153b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.810+0000 7f35de0e5700 1 -- 172.21.15.29:0/4199297746 shutdown_connections 2024-01-24T15:32:53.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.810+0000 7f35de0e5700 1 -- 172.21.15.29:0/4199297746 wait complete. 2024-01-24T15:32:53.825 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-01-24T15:32:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.953+0000 7fc93d20c700 1 Processor -- start 2024-01-24T15:32:53.956 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.954+0000 7fc93d20c700 1 -- start start 2024-01-24T15:32:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.954+0000 7fc93d20c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc938154240 0x7fc938064780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.957 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.954+0000 7fc93d20c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc9380a9d20 0x7fc9380aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.954+0000 7fc93d20c700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9381550e0 con 0x7fc938157cf0 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.954+0000 7fc93d20c700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9380c0d10 con 0x7fc9380a9d20 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.954+0000 7fc93d20c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380aa670 con 0x7fc938154240 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc93659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc9380a9d20 0x7fc9380aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc93659c700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc9380a9d20 0x7fc9380aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49746/0 (socket says 172.21.15.29:49746) 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc93659c700 1 -- 172.21.15.29:0/2725040752 learned_addr learned my addr 172.21.15.29:0/2725040752 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc936d9d700 1 --2- 172.21.15.29:0/2725040752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc938154240 0x7fc938064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc93659c700 1 -- 172.21.15.29:0/2725040752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc938154240 msgr2=0x7fc938064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc93659c700 1 --2- 172.21.15.29:0/2725040752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc938154240 0x7fc938064780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc93659c700 1 -- 172.21.15.29:0/2725040752 >> v1:172.21.15.29:6789/0 conn(0x7fc938157cf0 legacy=0x7fc938158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc935d9b700 1 -- 172.21.15.29:0/2725040752 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614206408 0 0) 0x7fc9381550e0 con 0x7fc938157cf0 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc93659c700 1 -- 172.21.15.29:0/2725040752 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9380650d0 con 0x7fc9380a9d20 2024-01-24T15:32:53.958 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc936d9d700 1 --2- 172.21.15.29:0/2725040752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc938154240 0x7fc938064780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc93659c700 1 --2- 172.21.15.29:0/2725040752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9380a9d20 0x7fc9380aa180 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 rx=0x7fc92c006fd0 tx=0x7fc92c0090f0).ready entity=mon.0 client_cookie=9141769bfca07da2 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc935d9b700 1 -- 172.21.15.29:0/2725040752 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00b600 con 0x7fc9380a9d20 2024-01-24T15:32:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc935d9b700 1 -- 172.21.15.29:0/2725040752 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c00b760 con 0x7fc9380a9d20 2024-01-24T15:32:53.959 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.955+0000 7fc935d9b700 1 -- 172.21.15.29:0/2725040752 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00bae0 con 0x7fc9380a9d20 2024-01-24T15:32:53.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.956+0000 7fc93d20c700 1 -- 172.21.15.29:0/2725040752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9380a9d20 msgr2=0x7fc9380aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.956+0000 7fc93d20c700 1 --2- 172.21.15.29:0/2725040752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9380a9d20 0x7fc9380aa180 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 rx=0x7fc92c006fd0 tx=0x7fc92c0090f0).stop 2024-01-24T15:32:53.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.956+0000 7fc93d20c700 1 -- 172.21.15.29:0/2725040752 shutdown_connections 2024-01-24T15:32:53.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.956+0000 7fc93d20c700 1 --2- 172.21.15.29:0/2725040752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc938154240 0x7fc938064780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.966 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.956+0000 7fc93d20c700 1 --2- 172.21.15.29:0/2725040752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9380a9d20 0x7fc9380aa180 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.956+0000 7fc93d20c700 1 -- 172.21.15.29:0/2725040752 >> 172.21.15.29:0/2725040752 conn(0x7fc9380c8310 msgr2=0x7fc93805e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:53.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.956+0000 7fc93d20c700 1 -- 172.21.15.29:0/2725040752 shutdown_connections 2024-01-24T15:32:53.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.956+0000 7fc93d20c700 1 -- 172.21.15.29:0/2725040752 wait complete. 2024-01-24T15:32:53.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.957+0000 7fc93d20c700 1 Processor -- start 2024-01-24T15:32:53.967 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.957+0000 7fc93d20c700 1 -- start start 2024-01-24T15:32:53.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.958+0000 7fc93d20c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9380a9d20 0x7fc9380b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.969 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.958+0000 7fc93d20c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc938154240 0x7fc9381529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.958+0000 7fc93d20c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc938157cf0 0x7fc938153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.958+0000 7fc93d20c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380aedd0 con 0x7fc9380a9d20 2024-01-24T15:32:53.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.958+0000 7fc93d20c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380af020 con 0x7fc938154240 2024-01-24T15:32:53.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.958+0000 7fc93d20c700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380af300 con 0x7fc938157cf0 2024-01-24T15:32:53.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.958+0000 7fc936d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9380a9d20 0x7fc9380b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.958+0000 7fc936d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9380a9d20 0x7fc9380b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49758/0 (socket says 172.21.15.29:49758) 2024-01-24T15:32:53.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.958+0000 7fc936d9d700 1 -- 172.21.15.29:0/2978848334 learned_addr learned my addr 172.21.15.29:0/2978848334 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:53.970 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.958+0000 7fc93759e700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc938157cf0 0x7fc938153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.958+0000 7fc93659c700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc938154240 0x7fc9381529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.959+0000 7fc936d9d700 1 -- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc938157cf0 msgr2=0x7fc938153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.959+0000 7fc936d9d700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc938157cf0 0x7fc938153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.959+0000 7fc936d9d700 1 -- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc938154240 msgr2=0x7fc9381529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:53.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.959+0000 7fc936d9d700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc938154240 0x7fc9381529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:53.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.959+0000 7fc936d9d700 1 -- 172.21.15.29:0/2978848334 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c009370 con 0x7fc9380a9d20 2024-01-24T15:32:53.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.959+0000 7fc936d9d700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9380a9d20 0x7fc9380b00c0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 rx=0x7fc92800d860 tx=0x7fc928008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.959+0000 7fc91effd700 1 -- 172.21.15.29:0/2978848334 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92800f520 con 0x7fc9380a9d20 2024-01-24T15:32:53.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.959+0000 7fc91effd700 1 -- 172.21.15.29:0/2978848334 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92800f680 con 0x7fc9380a9d20 2024-01-24T15:32:53.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.959+0000 7fc93d20c700 1 -- 172.21.15.29:0/2978848334 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9380af5e0 con 0x7fc9380a9d20 2024-01-24T15:32:53.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.959+0000 7fc91effd700 1 -- 172.21.15.29:0/2978848334 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92800fa00 con 0x7fc9380a9d20 2024-01-24T15:32:53.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.959+0000 7fc93d20c700 1 -- 172.21.15.29:0/2978848334 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9380afba0 con 0x7fc9380a9d20 2024-01-24T15:32:53.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.960+0000 7fc93d20c700 1 -- 172.21.15.29:0/2978848334 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9380aff10 con 0x7fc9380a9d20 2024-01-24T15:32:53.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.961+0000 7fc91effd700 1 -- 172.21.15.29:0/2978848334 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc9280123e0 con 0x7fc9380a9d20 2024-01-24T15:32:53.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.961+0000 7fc91effd700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc92405f270 0x7fc924061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:53.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.961+0000 7fc91effd700 1 -- 172.21.15.29:0/2978848334 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fc92807a240 con 0x7fc9380a9d20 2024-01-24T15:32:53.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.961+0000 7fc93659c700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc92405f270 0x7fc924061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:53.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.962+0000 7fc93659c700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc92405f270 0x7fc924061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fc92c006fd0 tx=0x7fc92c017000).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:53.973 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:53.965+0000 7fc91effd700 1 -- 172.21.15.29:0/2978848334 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7fc92804b010 con 0x7fc9380a9d20 2024-01-24T15:32:54.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.146+0000 7fc93d20c700 1 -- 172.21.15.29:0/2978848334 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9380475e0 con 0x7fc92405f270 2024-01-24T15:32:54.148 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.146+0000 7fc91effd700 1 -- 172.21.15.29:0/2978848334 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc9380475e0 con 0x7fc92405f270 2024-01-24T15:32:54.150 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.148+0000 7fc93d20c700 1 -- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc92405f270 msgr2=0x7fc924061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:54.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.148+0000 7fc93d20c700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc92405f270 0x7fc924061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fc92c006fd0 tx=0x7fc92c017000).stop 2024-01-24T15:32:54.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.148+0000 7fc93d20c700 1 -- 172.21.15.29:0/2978848334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9380a9d20 msgr2=0x7fc9380b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:54.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.148+0000 7fc93d20c700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9380a9d20 0x7fc9380b00c0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 rx=0x7fc92800d860 tx=0x7fc928008040).stop 2024-01-24T15:32:54.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.148+0000 7fc93d20c700 1 -- 172.21.15.29:0/2978848334 shutdown_connections 2024-01-24T15:32:54.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.148+0000 7fc93d20c700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7fc92405f270 0x7fc924061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:54.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.148+0000 7fc93d20c700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc938157cf0 0x7fc938153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:54.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.148+0000 7fc93d20c700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9380a9d20 0x7fc9380b00c0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:54.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.148+0000 7fc93d20c700 1 --2- 172.21.15.29:0/2978848334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc938154240 0x7fc9381529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:54.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.148+0000 7fc93d20c700 1 -- 172.21.15.29:0/2978848334 >> 172.21.15.29:0/2978848334 conn(0x7fc9380c8310 msgr2=0x7fc9380ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:54.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.148+0000 7fc93d20c700 1 -- 172.21.15.29:0/2978848334 shutdown_connections 2024-01-24T15:32:54.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.148+0000 7fc93d20c700 1 -- 172.21.15.29:0/2978848334 wait complete. 2024-01-24T15:32:54.152 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 7 to deep-scrub 2024-01-24T15:32:54.164 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:32:54.294 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.292+0000 7f7f7730f700 1 Processor -- start 2024-01-24T15:32:54.295 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.292+0000 7f7f7730f700 1 -- start start 2024-01-24T15:32:54.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.293+0000 7f7f7730f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f70138330 0x7f7f70138710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:54.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.293+0000 7f7f7730f700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f70065770 con 0x7f7f700bca50 2024-01-24T15:32:54.297 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.293+0000 7f7f7730f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f70069cf0 con 0x7f7f700627b0 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.293+0000 7f7f7730f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70069f60 con 0x7f7f70138330 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.294+0000 7f7f748aa700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7f700bca50 0x7f7f700be3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:60624/0 (socket says 172.21.15.29:60624) 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.294+0000 7f7f748aa700 1 -- 172.21.15.29:0/1167640240 learned_addr learned my addr 172.21.15.29:0/1167640240 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.294+0000 7f7f750ab700 1 --2- 172.21.15.29:0/1167640240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f70138330 0x7f7f70138710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.294+0000 7f7f67fff700 1 -- 172.21.15.29:0/1167640240 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938216282 0 0) 0x7f7f70065770 con 0x7f7f700bca50 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.294+0000 7f7f67fff700 1 -- 172.21.15.29:0/1167640240 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f5c0033f0 con 0x7f7f700bca50 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.294+0000 7f7f67fff700 1 -- 172.21.15.29:0/1167640240 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166990525 0 0) 0x7f7f70069cf0 con 0x7f7f700627b0 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.294+0000 7f7f750ab700 1 -- 172.21.15.29:0/1167640240 >> v1:172.21.15.163:6789/0 conn(0x7f7f700627b0 legacy=0x7f7f70065c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.294+0000 7f7f750ab700 1 -- 172.21.15.29:0/1167640240 >> v1:172.21.15.29:6789/0 conn(0x7f7f700bca50 legacy=0x7f7f700be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.294+0000 7f7f750ab700 1 -- 172.21.15.29:0/1167640240 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7006a240 con 0x7f7f70138330 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.294+0000 7f7f750ab700 1 --2- 172.21.15.29:0/1167640240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f70138330 0x7f7f70138710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f7f60009790 tx=0x7f7f60009060).ready entity=mon.2 client_cookie=fa9e2cbbe2a0f58d server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.295+0000 7f7f67fff700 1 -- 172.21.15.29:0/1167640240 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c001470 con 0x7f7f70138330 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.295+0000 7f7f67fff700 1 -- 172.21.15.29:0/1167640240 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f60012410 con 0x7f7f70138330 2024-01-24T15:32:54.298 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.295+0000 7f7f67fff700 1 -- 172.21.15.29:0/1167640240 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60012790 con 0x7f7f70138330 2024-01-24T15:32:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.295+0000 7f7f7730f700 1 -- 172.21.15.29:0/1167640240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f70138330 msgr2=0x7f7f70138710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.295+0000 7f7f7730f700 1 --2- 172.21.15.29:0/1167640240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f70138330 0x7f7f70138710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f7f60009790 tx=0x7f7f60009060).stop 2024-01-24T15:32:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.295+0000 7f7f7730f700 1 -- 172.21.15.29:0/1167640240 shutdown_connections 2024-01-24T15:32:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.295+0000 7f7f7730f700 1 --2- 172.21.15.29:0/1167640240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f70138330 0x7f7f70138710 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.295+0000 7f7f7730f700 1 -- 172.21.15.29:0/1167640240 >> 172.21.15.29:0/1167640240 conn(0x7f7f700c8580 msgr2=0x7f7f700c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.295+0000 7f7f7730f700 1 -- 172.21.15.29:0/1167640240 shutdown_connections 2024-01-24T15:32:54.299 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.295+0000 7f7f7730f700 1 -- 172.21.15.29:0/1167640240 wait complete. 2024-01-24T15:32:54.301 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.296+0000 7f7f7730f700 1 Processor -- start 2024-01-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.296+0000 7f7f7730f700 1 -- start start 2024-01-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.297+0000 7f7f7730f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f700627b0 0x7f7f70156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.297+0000 7f7f7730f700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f700bca50 0x7f7f70156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.297+0000 7f7f7730f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f70138330 0x7f7f701587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.297+0000 7f7f7730f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70151620 con 0x7f7f70138330 2024-01-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.297+0000 7f7f7730f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700a9780 con 0x7f7f700627b0 2024-01-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.297+0000 7f7f7730f700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700ab9a0 con 0x7f7f700bca50 2024-01-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.297+0000 7f7f758ac700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f70138330 0x7f7f701587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f758ac700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f70138330 0x7f7f701587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49766/0 (socket says 172.21.15.29:49766) 2024-01-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f758ac700 1 -- 172.21.15.29:0/899808743 learned_addr learned my addr 172.21.15.29:0/899808743 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:32:54.302 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f748aa700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f700bca50 0x7f7f70156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f750ab700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f700627b0 0x7f7f70156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f758ac700 1 -- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f700bca50 msgr2=0x7f7f70156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f758ac700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f700bca50 0x7f7f70156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f758ac700 1 -- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f700627b0 msgr2=0x7f7f70156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f758ac700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f700627b0 0x7f7f70156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f758ac700 1 -- 172.21.15.29:0/899808743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f600092a0 con 0x7f7f70138330 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f748aa700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f700bca50 0x7f7f70156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f758ac700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f70138330 0x7f7f701587f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 rx=0x7f7f6800c760 tx=0x7f7f68007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f657fa700 1 -- 172.21.15.29:0/899808743 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68001630 con 0x7f7f70138330 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f657fa700 1 -- 172.21.15.29:0/899808743 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f68010510 con 0x7f7f70138330 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f7730f700 1 -- 172.21.15.29:0/899808743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f70157360 con 0x7f7f70138330 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f657fa700 1 -- 172.21.15.29:0/899808743 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f680108d0 con 0x7f7f70138330 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.298+0000 7f7f7730f700 1 -- 172.21.15.29:0/899808743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f700abc40 con 0x7f7f70138330 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.299+0000 7f7f7730f700 1 -- 172.21.15.29:0/899808743 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f700475e0 con 0x7f7f70138330 2024-01-24T15:32:54.303 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.300+0000 7f7f657fa700 1 -- 172.21.15.29:0/899808743 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7f68010c80 con 0x7f7f70138330 2024-01-24T15:32:54.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.300+0000 7f7f657fa700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7f5405f240 0x7f7f54061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:32:54.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.300+0000 7f7f657fa700 1 -- 172.21.15.29:0/899808743 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f7f6807a550 con 0x7f7f70138330 2024-01-24T15:32:54.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.301+0000 7f7f750ab700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7f5405f240 0x7f7f54061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:32:54.304 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.301+0000 7f7f750ab700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7f5405f240 0x7f7f54061700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7f600096e0 tx=0x7f7f60009060).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:32:54.307 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.305+0000 7f7f657fa700 1 -- 172.21.15.29:0/899808743 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7f6804b320 con 0x7f7f70138330 2024-01-24T15:32:54.481 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.479+0000 7f7f7730f700 1 -- 172.21.15.29:0/899808743 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f70159c70 con 0x7f7f5405f240 2024-01-24T15:32:54.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.482+0000 7f7f657fa700 1 -- 172.21.15.29:0/899808743 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70610 (secure 0 0 0) 0x7f7f60000ce0 con 0x7f7f5405f240 2024-01-24T15:32:54.486 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:32:54.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.484+0000 7f7f7730f700 1 -- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7f5405f240 msgr2=0x7f7f54061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:54.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.484+0000 7f7f7730f700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7f5405f240 0x7f7f54061700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7f600096e0 tx=0x7f7f60009060).stop 2024-01-24T15:32:54.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.484+0000 7f7f7730f700 1 -- 172.21.15.29:0/899808743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f70138330 msgr2=0x7f7f701587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:32:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.484+0000 7f7f7730f700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f70138330 0x7f7f701587f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 rx=0x7f7f6800c760 tx=0x7f7f68007f10).stop 2024-01-24T15:32:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.484+0000 7f7f7730f700 1 -- 172.21.15.29:0/899808743 shutdown_connections 2024-01-24T15:32:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.484+0000 7f7f7730f700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f70138330 0x7f7f701587f0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.484+0000 7f7f7730f700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f700627b0 0x7f7f70156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.484+0000 7f7f7730f700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f7f5405f240 0x7f7f54061700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.484+0000 7f7f7730f700 1 --2- 172.21.15.29:0/899808743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f700bca50 0x7f7f70156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:32:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.484+0000 7f7f7730f700 1 -- 172.21.15.29:0/899808743 >> 172.21.15.29:0/899808743 conn(0x7f7f700c8580 msgr2=0x7f7f700c5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:32:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.484+0000 7f7f7730f700 1 -- 172.21.15.29:0/899808743 shutdown_connections 2024-01-24T15:32:54.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:32:54.484+0000 7f7f7730f700 1 -- 172.21.15.29:0/899808743 wait complete. 2024-01-24T15:32:54.489 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:32:54.499 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-01-24T15:32:54.479945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2473724,"kb_used_data":23228,"kb_used_omap":0,"kb_used_meta":2450432,"kb_avail":752500996,"statfs":{"total":773094113280,"available":770561019904,"internally_reserved":0,"allocated":23785472,"data_stored":13831639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2509242368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13041,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.008062"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":193,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.886293+0000","last_change":"2024-01-24T13:58:13.871108+0000","last_active":"2024-01-24T15:32:43.886293+0000","last_peered":"2024-01-24T15:32:43.886293+0000","last_clean":"2024-01-24T15:32:43.886293+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:32:43.886293+0000","last_undegraded":"2024-01-24T15:32:43.886293+0000","last_fullsized":"2024-01-24T15:32:43.886293+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":219,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.499262+0000","last_change":"2024-01-24T13:58:13.769799+0000","last_active":"2024-01-24T15:32:43.499262+0000","last_peered":"2024-01-24T15:32:43.499262+0000","last_clean":"2024-01-24T15:32:43.499262+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:32:43.499262+0000","last_undegraded":"2024-01-24T15:32:43.499262+0000","last_fullsized":"2024-01-24T15:32:43.499262+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":214,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.493041+0000","last_change":"2024-01-24T13:58:13.761966+0000","last_active":"2024-01-24T15:32:43.493041+0000","last_peered":"2024-01-24T15:32:43.493041+0000","last_clean":"2024-01-24T15:32:43.493041+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:32:43.493041+0000","last_undegraded":"2024-01-24T15:32:43.493041+0000","last_fullsized":"2024-01-24T15:32:43.493041+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":219,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.498006+0000","last_change":"2024-01-24T13:58:13.767045+0000","last_active":"2024-01-24T15:32:43.498006+0000","last_peered":"2024-01-24T15:32:43.498006+0000","last_clean":"2024-01-24T15:32:43.498006+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:32:43.498006+0000","last_undegraded":"2024-01-24T15:32:43.498006+0000","last_fullsized":"2024-01-24T15:32:43.498006+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":196,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.880343+0000","last_change":"2024-01-24T13:58:11.842761+0000","last_active":"2024-01-24T15:32:43.880343+0000","last_peered":"2024-01-24T15:32:43.880343+0000","last_clean":"2024-01-24T15:32:43.880343+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:32:43.880343+0000","last_undegraded":"2024-01-24T15:32:43.880343+0000","last_fullsized":"2024-01-24T15:32:43.880343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:10.738353+0000","last_clean_scrub_stamp":"2024-01-24T13:58:10.738353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.880247+0000","last_change":"2024-01-24T13:58:13.765039+0000","last_active":"2024-01-24T15:32:43.880247+0000","last_peered":"2024-01-24T15:32:43.880247+0000","last_clean":"2024-01-24T15:32:43.880247+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:32:43.880247+0000","last_undegraded":"2024-01-24T15:32:43.880247+0000","last_fullsized":"2024-01-24T15:32:43.880247+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":194,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.881987+0000","last_change":"2024-01-24T13:58:13.871295+0000","last_active":"2024-01-24T15:32:43.881987+0000","last_peered":"2024-01-24T15:32:43.881987+0000","last_clean":"2024-01-24T15:32:43.881987+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:32:43.881987+0000","last_undegraded":"2024-01-24T15:32:43.881987+0000","last_fullsized":"2024-01-24T15:32:43.881987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":193,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.889119+0000","last_change":"2024-01-24T13:58:13.772779+0000","last_active":"2024-01-24T15:32:43.889119+0000","last_peered":"2024-01-24T15:32:43.889119+0000","last_clean":"2024-01-24T15:32:43.889119+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:32:43.889119+0000","last_undegraded":"2024-01-24T15:32:43.889119+0000","last_fullsized":"2024-01-24T15:32:43.889119+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":192,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-01-24T15:32:43.882264+0000","last_change":"2024-01-24T13:58:13.772743+0000","last_active":"2024-01-24T15:32:43.882264+0000","last_peered":"2024-01-24T15:32:43.882264+0000","last_clean":"2024-01-24T15:32:43.882264+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:32:43.882264+0000","last_undegraded":"2024-01-24T15:32:43.882264+0000","last_fullsized":"2024-01-24T15:32:43.882264+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T13:58:12.746817+0000","last_clean_scrub_stamp":"2024-01-24T13:58:12.746817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575978,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304096,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94067744,"statfs":{"total":96636764160,"available":96325369856,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.47499999999999998,"5min":0.39300000000000002,"15min":0.32000000000000001},"max":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.83199999999999996},"last":0.442},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.75700000000000001,"5min":0.84799999999999998,"15min":0.85499999999999998},"last":0.40300000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.61299999999999999,"5min":0.374,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":0.878,"15min":0.878},"last":0.434},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.89600000000000002},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.55900000000000005,"5min":0.374,"15min":0.374},"max":{"1min":0.82899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80800000000000005,"5min":0.83299999999999996,"15min":0.85099999999999998},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":0.82399999999999995,"5min":0.83199999999999996,"15min":0.92600000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.50700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.88},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51400000000000001,"15min":0.49399999999999999},"min":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.30599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.42299999999999999}]},{"osd":5,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.439,"5min":0.35899999999999999,"15min":0.32300000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.76100000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.49099999999999999}]},{"osd":6,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.42399999999999999,"5min":0.39000000000000001,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81899999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.623,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.40999999999999998,"15min":0.35099999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.92700000000000005},"last":0.50900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575979,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.442,"5min":0.377,"15min":0.31900000000000001},"max":{"1min":0.83699999999999997,"5min":0.998,"15min":0.998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.61699999999999999},"min":{"1min":0.48599999999999999,"5min":0.44700000000000001,"15min":0.32800000000000001},"max":{"1min":0.82399999999999995,"5min":0.82699999999999996,"15min":0.95599999999999996},"last":0.45500000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":1.089},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.45000000000000001,"5min":0.41899999999999998,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.85599999999999998,"15min":0.94299999999999995},"last":0.53300000000000003}]},{"osd":2,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.627},"min":{"1min":0.379,"5min":0.379,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.90900000000000003},"last":0.502}]},{"osd":3,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64200000000000002,"15min":0.63900000000000001},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.371},"max":{"1min":0.879,"5min":0.96299999999999997,"15min":1.0980000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.42299999999999999},"max":{"1min":0.78300000000000003,"5min":0.83899999999999997,"15min":1.169},"last":0.56599999999999995}]},{"osd":4,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":0.73599999999999999,"5min":0.753,"15min":0.753},"last":0.436},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.33100000000000002,"15min":0.312},"max":{"1min":0.79100000000000004,"5min":0.81399999999999995,"15min":0.83999999999999997},"last":0.37}]},{"osd":5,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":0.82499999999999996,"15min":0.86499999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69599999999999995,"5min":0.81699999999999995,"15min":0.82799999999999996},"last":0.41699999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71099999999999997,"5min":0.83699999999999997,"15min":0.88},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.83599999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575978,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.41799999999999998},"min":{"1min":0.28999999999999998,"5min":0.26600000000000001,"15min":0.24299999999999999},"max":{"1min":0.91200000000000003,"5min":3.254,"15min":3.254},"last":0.44},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.26100000000000001},"max":{"1min":0.82299999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.34200000000000003,"5min":0.30299999999999999,"15min":0.28499999999999998},"max":{"1min":0.86499999999999999,"5min":1.306,"15min":1.306},"last":0.498},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.45200000000000001},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.46600000000000003},"min":{"1min":0.31900000000000001,"5min":0.312,"15min":0.308},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.36899999999999999,"5min":0.34100000000000003,"15min":0.32600000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.54700000000000004}]},{"osd":4,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73799999999999999,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47299999999999998},"max":{"1min":1.1080000000000001,"5min":5.3630000000000004,"15min":5.3630000000000004},"last":0.624},{"interface":"front","average":{"1min":0.745,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.55500000000000005,"5min":0.46300000000000002,"15min":0.40400000000000003},"max":{"1min":1.244,"5min":1.677,"15min":1.677},"last":0.68100000000000005}]},{"osd":5,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.71699999999999997},"min":{"1min":0.53900000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.232,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.56100000000000005,"5min":0.47399999999999998,"15min":0.46100000000000002},"max":{"1min":1.081,"5min":1.117,"15min":1.2989999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.54100000000000004,"5min":0.47899999999999998,"15min":0.46800000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.57599999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1859999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.69399999999999995}]},{"osd":7,"last update":"Wed Jan 24 15:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.58099999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1759999999999999,"5min":1.704,"15min":1.704},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81100000000000005,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1599999999999999,"5min":5.29,"15min":5.29},"last":0.71799999999999997}]}]},{"osd":0,"up_from":14,"seq":60129543276,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312284,"kb_used_data":2900,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059556,"statfs":{"total":96636764160,"available":96316985344,"internally_reserved":0,"allocated":2969600,"data_stored":1728114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.40799999999999997},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.26000000000000001},"max":{"1min":0.65600000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.27300000000000002},"max":{"1min":0.66100000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.42999999999999999}]},{"osd":2,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.318,"15min":0.29599999999999999},"max":{"1min":0.73399999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.44},{"interface":"front","average":{"1min":0.495,"5min":0.46899999999999997,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":0.64600000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.379}]},{"osd":3,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.44500000000000001,"15min":0.442},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59299999999999997,"5min":0.67400000000000004,"15min":0.72799999999999998},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76600000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.47499999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.98699999999999999},"last":0.497},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.51900000000000002,"5min":0.45100000000000001,"15min":0.41199999999999998},"max":{"1min":0.92700000000000005,"5min":0.998,"15min":0.998},"last":0.45600000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67900000000000005,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.40699999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.0029999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.55800000000000005,"5min":0.46700000000000003,"15min":0.46500000000000002},"max":{"1min":1.0960000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.53100000000000003}]},{"osd":6,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97799999999999998,"5min":1.27,"15min":1.27},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.61799999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.52200000000000002}]},{"osd":7,"last update":"Wed Jan 24 15:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72799999999999998,"15min":0.73099999999999998},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98699999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.55600000000000005,"5min":0.51000000000000001,"15min":0.46600000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.48899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575978,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312288,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059552,"statfs":{"total":96636764160,"available":96316981248,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.252},"max":{"1min":0.626,"5min":0.626,"15min":0.69699999999999995},"last":0.432},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.60899999999999999,"5min":0.66600000000000004,"15min":0.70599999999999996},"last":0.61299999999999999}]},{"osd":1,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.432,"15min":0.432},"min":{"1min":0.32200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.498},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.40200000000000002,"5min":0.32600000000000001,"15min":0.27400000000000002},"max":{"1min":0.66400000000000003,"5min":0.69799999999999995,"15min":0.80900000000000005},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.29099999999999998},"max":{"1min":0.623,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.58999999999999997}]},{"osd":4,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.57799999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78500000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.55100000000000005,"5min":0.45000000000000001,"15min":0.376},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":0.996,"15min":1.0589999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.45700000000000002},"max":{"1min":0.95999999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.39700000000000002},"max":{"1min":0.91800000000000004,"5min":1.006,"15min":1.0740000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.51700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.89200000000000002,"5min":1.0229999999999999,"15min":1.083},"last":0.79100000000000004}]},{"osd":7,"last update":"Wed Jan 24 15:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96599999999999997,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.75800000000000001},"min":{"1min":0.61299999999999999,"5min":0.52000000000000002,"15min":0.51100000000000001},"max":{"1min":0.92000000000000004,"5min":1.0429999999999999,"15min":1.123},"last":0.76200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575978,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.318,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.373},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.438},"min":{"1min":0.35299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.76100000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42799999999999999},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.65000000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.752},"last":0.435}]},{"osd":2,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.80800000000000005},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.81899999999999995},"last":0.53000000000000003}]},{"osd":4,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66000000000000003,"15min":0.66100000000000003},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.437},"max":{"1min":0.80100000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.93899999999999995,"5min":1.026,"15min":1.026},"last":0.65400000000000003}]},{"osd":5,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.056,"15min":1.056},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.499,"15min":0.47199999999999998},"max":{"1min":0.93700000000000006,"5min":1.095,"15min":1.095},"last":0.56000000000000005}]},{"osd":6,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":1.1499999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72099999999999997,"15min":0.72199999999999998},"min":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.0940000000000001},"last":0.876}]},{"osd":7,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.55000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.018,"5min":1.071,"15min":1.083},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.53600000000000003},"max":{"1min":0.95399999999999996,"5min":1.081,"15min":1.1359999999999999},"last":0.83399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575979,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312288,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059552,"statfs":{"total":96636764160,"available":96316981248,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.46700000000000003,"5min":0.39400000000000002,"15min":0.375},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.629},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.80100000000000005,"5min":0.876,"15min":0.876},"last":0.58699999999999997}]},{"osd":1,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.622},"min":{"1min":0.496,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.45900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.79100000000000004,"5min":0.80400000000000005,"15min":0.82199999999999995},"last":0.499},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.40500000000000003,"15min":0.38500000000000001},"max":{"1min":0.82899999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64500000000000002}]},{"osd":3,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82299999999999995,"5min":0.86799999999999999,"15min":0.87},"last":0.42799999999999999}]},{"osd":5,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.69699999999999995,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.33800000000000002,"5min":0.31900000000000001,"15min":0.308},"max":{"1min":0.66200000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.54100000000000004}]},{"osd":6,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.33600000000000002},"max":{"1min":0.61199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.37},"max":{"1min":0.72599999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52000000000000002}]},{"osd":7,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.34899999999999998},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.375,"15min":0.35799999999999998},"max":{"1min":0.78100000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575979,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.34799999999999998},"max":{"1min":0.88300000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.48399999999999999,"5min":0.41999999999999998,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":1.149,"15min":1.149},"last":0.51200000000000001}]},{"osd":1,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.52800000000000002,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":1.125,"15min":1.125},"last":0.5},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.38400000000000001,"15min":0.38},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.432,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77900000000000003,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.61599999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.41199999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":1.252},"last":0.59199999999999997}]},{"osd":3,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.51100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.95499999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.64400000000000002},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":1.196,"15min":1.196},"last":0.61699999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.58699999999999997,"5min":0.78700000000000003,"15min":0.90500000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.503,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.81299999999999994},"last":0.432}]},{"osd":6,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.50600000000000001},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68400000000000005,"5min":0.82199999999999995,"15min":1.0209999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.54200000000000004},"min":{"1min":0.41099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.98399999999999999},"last":0.65600000000000003}]},{"osd":7,"last update":"Wed Jan 24 15:32:03 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.54200000000000004},"min":{"1min":0.39700000000000002,"5min":0.371,"15min":0.33600000000000002},"max":{"1min":0.629,"5min":0.80100000000000005,"15min":0.86399999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.57799999999999996},"min":{"1min":0.372,"5min":0.371,"15min":0.371},"max":{"1min":0.80900000000000005,"5min":0.83099999999999996,"15min":0.91700000000000004},"last":0.63800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:32:54.504 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-01-24T13:58:12.746817+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=13, tm_min=58, tm_sec=12, tm_wday=2, tm_yday=24, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=49, tm_wday=2, tm_yday=24, tm_isdst=0) 2024-01-24T15:32:54.505 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-01-24T13:58:12.746817+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=13, tm_min=58, tm_sec=12, tm_wday=2, tm_yday=24, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=49, tm_wday=2, tm_yday=24, tm_isdst=0) 2024-01-24T15:32:54.505 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-01-24T13:58:12.746817+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=13, tm_min=58, tm_sec=12, tm_wday=2, tm_yday=24, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=49, tm_wday=2, tm_yday=24, tm_isdst=0) 2024-01-24T15:32:54.505 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-01-24T13:58:12.746817+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=13, tm_min=58, tm_sec=12, tm_wday=2, tm_yday=24, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=49, tm_wday=2, tm_yday=24, tm_isdst=0) 2024-01-24T15:32:54.505 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-01-24T13:58:10.738353+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=13, tm_min=58, tm_sec=10, tm_wday=2, tm_yday=24, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=49, tm_wday=2, tm_yday=24, tm_isdst=0) 2024-01-24T15:32:54.505 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-01-24T13:58:12.746817+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=13, tm_min=58, tm_sec=12, tm_wday=2, tm_yday=24, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=49, tm_wday=2, tm_yday=24, tm_isdst=0) 2024-01-24T15:32:54.506 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-01-24T13:58:12.746817+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=13, tm_min=58, tm_sec=12, tm_wday=2, tm_yday=24, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=49, tm_wday=2, tm_yday=24, tm_isdst=0) 2024-01-24T15:32:54.506 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-01-24T13:58:12.746817+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=13, tm_min=58, tm_sec=12, tm_wday=2, tm_yday=24, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=49, tm_wday=2, tm_yday=24, tm_isdst=0) 2024-01-24T15:32:54.506 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-01-24T13:58:12.746817+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=13, tm_min=58, tm_sec=12, tm_wday=2, tm_yday=24, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=24, tm_hour=15, tm_min=32, tm_sec=49, tm_wday=2, tm_yday=24, tm_isdst=0) 2024-01-24T15:32:54.506 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-24T15:33:13.119 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:33:13.123 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:33:14.507 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T15:33:14.644 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.642+0000 7f7a1cd79700 1 Processor -- start 2024-01-24T15:33:14.645 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.642+0000 7f7a1cd79700 1 -- start start 2024-01-24T15:33:14.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.643+0000 7f7a1cd79700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7a180be520 0x7f7a180664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:14.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.643+0000 7f7a1cd79700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a18064120 con 0x7f7a1813a5c0 2024-01-24T15:33:14.647 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.643+0000 7f7a1cd79700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a180bf6a0 con 0x7f7a180669e0 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.643+0000 7f7a1cd79700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a18066e00 con 0x7f7a180be520 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.644+0000 7f7a1659c700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7a1813a5c0 0x7f7a18063a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40862/0 (socket says 172.21.15.29:40862) 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.644+0000 7f7a1659c700 1 -- 172.21.15.29:0/878260850 learned_addr learned my addr 172.21.15.29:0/878260850 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.644+0000 7f7a15d9b700 1 --2- 172.21.15.29:0/878260850 >> v2:172.21.15.163:3300/0 conn(0x7f7a180be520 0x7f7a180664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.644+0000 7f7a1559a700 1 -- 172.21.15.29:0/878260850 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958206302 0 0) 0x7f7a18064120 con 0x7f7a1813a5c0 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.644+0000 7f7a1559a700 1 -- 172.21.15.29:0/878260850 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a000033f0 con 0x7f7a1813a5c0 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.644+0000 7f7a1559a700 1 -- 172.21.15.29:0/878260850 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862914211 0 0) 0x7f7a180bf6a0 con 0x7f7a180669e0 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.644+0000 7f7a1559a700 1 -- 172.21.15.29:0/878260850 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a18064120 con 0x7f7a180669e0 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.644+0000 7f7a1559a700 1 -- 172.21.15.29:0/878260850 <== mon.0 v1:172.21.15.29:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3866777456 0 0) 0x7f7a000033f0 con 0x7f7a1813a5c0 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.644+0000 7f7a15d9b700 1 -- 172.21.15.29:0/878260850 >> v1:172.21.15.163:6789/0 conn(0x7f7a180669e0 legacy=0x7f7a180c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.644+0000 7f7a15d9b700 1 -- 172.21.15.29:0/878260850 >> v1:172.21.15.29:6789/0 conn(0x7f7a1813a5c0 legacy=0x7f7a18063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.644+0000 7f7a15d9b700 1 -- 172.21.15.29:0/878260850 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a180bea90 con 0x7f7a180be520 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.644+0000 7f7a15d9b700 1 --2- 172.21.15.29:0/878260850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a180be520 0x7f7a180664f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f7a08009760 tx=0x7f7a08009060).ready entity=mon.1 client_cookie=97762d0fc13a9d88 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:33:14.648 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.645+0000 7f7a1559a700 1 -- 172.21.15.29:0/878260850 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c001470 con 0x7f7a180be520 2024-01-24T15:33:14.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.645+0000 7f7a1559a700 1 -- 172.21.15.29:0/878260850 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a08011400 con 0x7f7a180be520 2024-01-24T15:33:14.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.645+0000 7f7a1559a700 1 -- 172.21.15.29:0/878260850 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08011780 con 0x7f7a180be520 2024-01-24T15:33:14.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.645+0000 7f7a1cd79700 1 -- 172.21.15.29:0/878260850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a180be520 msgr2=0x7f7a180664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:14.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.645+0000 7f7a1cd79700 1 --2- 172.21.15.29:0/878260850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a180be520 0x7f7a180664f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f7a08009760 tx=0x7f7a08009060).stop 2024-01-24T15:33:14.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.645+0000 7f7a1cd79700 1 -- 172.21.15.29:0/878260850 shutdown_connections 2024-01-24T15:33:14.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.645+0000 7f7a1cd79700 1 --2- 172.21.15.29:0/878260850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a180be520 0x7f7a180664f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:14.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.645+0000 7f7a1cd79700 1 -- 172.21.15.29:0/878260850 >> 172.21.15.29:0/878260850 conn(0x7f7a180c8bb0 msgr2=0x7f7a180c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:33:14.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.645+0000 7f7a1cd79700 1 -- 172.21.15.29:0/878260850 shutdown_connections 2024-01-24T15:33:14.649 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.645+0000 7f7a1cd79700 1 -- 172.21.15.29:0/878260850 wait complete. 2024-01-24T15:33:14.651 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.647+0000 7f7a1cd79700 1 Processor -- start 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.647+0000 7f7a1cd79700 1 -- start start 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.648+0000 7f7a1cd79700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a180669e0 0x7f7a18148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.648+0000 7f7a1cd79700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a180be520 0x7f7a180aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.648+0000 7f7a1cd79700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a1813a5c0 0x7f7a180aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.648+0000 7f7a1cd79700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a180aafa0 con 0x7f7a180be520 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.648+0000 7f7a1cd79700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a18155c60 con 0x7f7a1813a5c0 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.648+0000 7f7a1cd79700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a18155f40 con 0x7f7a180669e0 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.648+0000 7f7a15d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a180be520 0x7f7a180aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.648+0000 7f7a15d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a180be520 0x7f7a180aa5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60874/0 (socket says 172.21.15.29:60874) 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.648+0000 7f7a15d9b700 1 -- 172.21.15.29:0/1837793858 learned_addr learned my addr 172.21.15.29:0/1837793858 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.648+0000 7f7a1659c700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a180669e0 0x7f7a18148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.648+0000 7f7a16d9d700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a1813a5c0 0x7f7a180aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.649+0000 7f7a15d9b700 1 -- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a180669e0 msgr2=0x7f7a18148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:14.652 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.649+0000 7f7a15d9b700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a180669e0 0x7f7a18148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:14.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.649+0000 7f7a15d9b700 1 -- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a1813a5c0 msgr2=0x7f7a180aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:14.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.649+0000 7f7a15d9b700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a1813a5c0 0x7f7a180aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:14.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.649+0000 7f7a15d9b700 1 -- 172.21.15.29:0/1837793858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a080092a0 con 0x7f7a180be520 2024-01-24T15:33:14.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.649+0000 7f7a15d9b700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a180be520 0x7f7a180aa5c0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 rx=0x7f7a08009d50 tx=0x7f7a08016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:33:14.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.649+0000 7f7a067fc700 1 -- 172.21.15.29:0/1837793858 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a080106f0 con 0x7f7a180be520 2024-01-24T15:33:14.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.649+0000 7f7a067fc700 1 -- 172.21.15.29:0/1837793858 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a08010850 con 0x7f7a180be520 2024-01-24T15:33:14.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.649+0000 7f7a1cd79700 1 -- 172.21.15.29:0/1837793858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a18156220 con 0x7f7a180be520 2024-01-24T15:33:14.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.649+0000 7f7a067fc700 1 -- 172.21.15.29:0/1837793858 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08010bd0 con 0x7f7a180be520 2024-01-24T15:33:14.653 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.649+0000 7f7a1cd79700 1 -- 172.21.15.29:0/1837793858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a181567e0 con 0x7f7a180be520 2024-01-24T15:33:14.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.650+0000 7f7a1cd79700 1 -- 172.21.15.29:0/1837793858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a180475e0 con 0x7f7a180be520 2024-01-24T15:33:14.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.650+0000 7f7a067fc700 1 -- 172.21.15.29:0/1837793858 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7a08027460 con 0x7f7a180be520 2024-01-24T15:33:14.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.651+0000 7f7a067fc700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f79f805f270 0x7f79f8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:14.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.651+0000 7f7a067fc700 1 -- 172.21.15.29:0/1837793858 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f7a0807bb70 con 0x7f7a180be520 2024-01-24T15:33:14.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.651+0000 7f7a1659c700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f79f805f270 0x7f79f8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:14.654 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.652+0000 7f7a1659c700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f79f805f270 0x7f79f8061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f7a0c000fb0 tx=0x7f7a0c0019e0).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:33:14.657 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.655+0000 7f7a067fc700 1 -- 172.21.15.29:0/1837793858 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f7a0804c940 con 0x7f7a180be520 2024-01-24T15:33:14.836 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.834+0000 7f7a1cd79700 1 -- 172.21.15.29:0/1837793858 --> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a180bed70 con 0x7f79f805f270 2024-01-24T15:33:14.840 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.838+0000 7f7a067fc700 1 -- 172.21.15.29:0/1837793858 <== mgr.9763 v2:172.21.15.163:6832/57365 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70674 (secure 0 0 0) 0x7f7a180bed70 con 0x7f79f805f270 2024-01-24T15:33:14.841 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:33:14.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.840+0000 7f7a1cd79700 1 -- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f79f805f270 msgr2=0x7f79f8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:14.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.840+0000 7f7a1cd79700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f79f805f270 0x7f79f8061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f7a0c000fb0 tx=0x7f7a0c0019e0).stop 2024-01-24T15:33:14.842 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.840+0000 7f7a1cd79700 1 -- 172.21.15.29:0/1837793858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a180be520 msgr2=0x7f7a180aa5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:14.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.840+0000 7f7a1cd79700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a180be520 0x7f7a180aa5c0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 rx=0x7f7a08009d50 tx=0x7f7a08016000).stop 2024-01-24T15:33:14.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.840+0000 7f7a1cd79700 1 -- 172.21.15.29:0/1837793858 shutdown_connections 2024-01-24T15:33:14.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.840+0000 7f7a1cd79700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a1813a5c0 0x7f7a180aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:14.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.840+0000 7f7a1cd79700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f79f805f270 0x7f79f8061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:14.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.840+0000 7f7a1cd79700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a180669e0 0x7f7a18148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:14.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.840+0000 7f7a1cd79700 1 --2- 172.21.15.29:0/1837793858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a180be520 0x7f7a180aa5c0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:14.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.840+0000 7f7a1cd79700 1 -- 172.21.15.29:0/1837793858 >> 172.21.15.29:0/1837793858 conn(0x7f7a180c8bb0 msgr2=0x7f7a180ad660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:33:14.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.840+0000 7f7a1cd79700 1 -- 172.21.15.29:0/1837793858 shutdown_connections 2024-01-24T15:33:14.843 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.840+0000 7f7a1cd79700 1 -- 172.21.15.29:0/1837793858 wait complete. 2024-01-24T15:33:14.843 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-01-24T15:33:14.855 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-24T15:33:14.484308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2473724,"kb_used_data":23228,"kb_used_omap":0,"kb_used_meta":2450432,"kb_avail":752500996,"statfs":{"total":773094113280,"available":770561019904,"internally_reserved":0,"allocated":23785472,"data_stored":13831639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2509242368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13041,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.008062"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":198,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-01-24T15:32:53.524650+0000","last_change":"2024-01-24T15:32:53.524650+0000","last_active":"2024-01-24T15:32:53.524650+0000","last_peered":"2024-01-24T15:32:53.524650+0000","last_clean":"2024-01-24T15:32:53.524650+0000","last_became_active":"2024-01-24T13:58:13.870801+0000","last_became_peered":"2024-01-24T13:58:13.870801+0000","last_unstale":"2024-01-24T15:32:53.524650+0000","last_undegraded":"2024-01-24T15:32:53.524650+0000","last_fullsized":"2024-01-24T15:32:53.524650+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:53.524555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:53.524555+0000","last_clean_scrub_stamp":"2024-01-24T15:32:53.524555+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":224,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-01-24T15:32:53.519252+0000","last_change":"2024-01-24T15:32:53.519252+0000","last_active":"2024-01-24T15:32:53.519252+0000","last_peered":"2024-01-24T15:32:53.519252+0000","last_clean":"2024-01-24T15:32:53.519252+0000","last_became_active":"2024-01-24T13:58:13.769511+0000","last_became_peered":"2024-01-24T13:58:13.769511+0000","last_unstale":"2024-01-24T15:32:53.519252+0000","last_undegraded":"2024-01-24T15:32:53.519252+0000","last_fullsized":"2024-01-24T15:32:53.519252+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:53.519120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:53.519120+0000","last_clean_scrub_stamp":"2024-01-24T15:32:53.519120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":219,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-01-24T15:32:51.753609+0000","last_change":"2024-01-24T15:32:51.753609+0000","last_active":"2024-01-24T15:32:51.753609+0000","last_peered":"2024-01-24T15:32:51.753609+0000","last_clean":"2024-01-24T15:32:51.753609+0000","last_became_active":"2024-01-24T13:58:13.761633+0000","last_became_peered":"2024-01-24T13:58:13.761633+0000","last_unstale":"2024-01-24T15:32:51.753609+0000","last_undegraded":"2024-01-24T15:32:51.753609+0000","last_fullsized":"2024-01-24T15:32:51.753609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:51.753524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:51.753524+0000","last_clean_scrub_stamp":"2024-01-24T15:32:51.753524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":224,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-01-24T15:32:56.493270+0000","last_change":"2024-01-24T15:32:56.493270+0000","last_active":"2024-01-24T15:32:56.493270+0000","last_peered":"2024-01-24T15:32:56.493270+0000","last_clean":"2024-01-24T15:32:56.493270+0000","last_became_active":"2024-01-24T13:58:13.766775+0000","last_became_peered":"2024-01-24T13:58:13.766775+0000","last_unstale":"2024-01-24T15:32:56.493270+0000","last_undegraded":"2024-01-24T15:32:56.493270+0000","last_fullsized":"2024-01-24T15:32:56.493270+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:56.493170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:56.493170+0000","last_clean_scrub_stamp":"2024-01-24T15:32:56.493170+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":201,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-01-24T15:32:54.936435+0000","last_change":"2024-01-24T15:32:54.936435+0000","last_active":"2024-01-24T15:32:54.936435+0000","last_peered":"2024-01-24T15:32:54.936435+0000","last_clean":"2024-01-24T15:32:54.936435+0000","last_became_active":"2024-01-24T13:58:11.842431+0000","last_became_peered":"2024-01-24T13:58:11.842431+0000","last_unstale":"2024-01-24T15:32:54.936435+0000","last_undegraded":"2024-01-24T15:32:54.936435+0000","last_fullsized":"2024-01-24T15:32:54.936435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:54.936263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:54.936263+0000","last_clean_scrub_stamp":"2024-01-24T15:32:54.936263+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":199,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-01-24T15:32:56.931495+0000","last_change":"2024-01-24T15:32:56.931495+0000","last_active":"2024-01-24T15:32:56.931495+0000","last_peered":"2024-01-24T15:32:56.931495+0000","last_clean":"2024-01-24T15:32:56.931495+0000","last_became_active":"2024-01-24T13:58:13.764764+0000","last_became_peered":"2024-01-24T13:58:13.764764+0000","last_unstale":"2024-01-24T15:32:56.931495+0000","last_undegraded":"2024-01-24T15:32:56.931495+0000","last_fullsized":"2024-01-24T15:32:56.931495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:56.931385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:56.931385+0000","last_clean_scrub_stamp":"2024-01-24T15:32:56.931385+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":199,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-01-24T15:32:57.943609+0000","last_change":"2024-01-24T15:32:57.943609+0000","last_active":"2024-01-24T15:32:57.943609+0000","last_peered":"2024-01-24T15:32:57.943609+0000","last_clean":"2024-01-24T15:32:57.943609+0000","last_became_active":"2024-01-24T13:58:13.870989+0000","last_became_peered":"2024-01-24T13:58:13.870989+0000","last_unstale":"2024-01-24T15:32:57.943609+0000","last_undegraded":"2024-01-24T15:32:57.943609+0000","last_fullsized":"2024-01-24T15:32:57.943609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:57.943455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:57.943455+0000","last_clean_scrub_stamp":"2024-01-24T15:32:57.943455+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":198,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-01-24T15:32:55.585543+0000","last_change":"2024-01-24T15:32:55.585543+0000","last_active":"2024-01-24T15:32:55.585543+0000","last_peered":"2024-01-24T15:32:55.585543+0000","last_clean":"2024-01-24T15:32:55.585543+0000","last_became_active":"2024-01-24T13:58:13.772445+0000","last_became_peered":"2024-01-24T13:58:13.772445+0000","last_unstale":"2024-01-24T15:32:55.585543+0000","last_undegraded":"2024-01-24T15:32:55.585543+0000","last_fullsized":"2024-01-24T15:32:55.585543+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:55.585441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:55.585441+0000","last_clean_scrub_stamp":"2024-01-24T15:32:55.585441+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":197,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-01-24T15:32:55.146479+0000","last_change":"2024-01-24T15:32:55.146479+0000","last_active":"2024-01-24T15:32:55.146479+0000","last_peered":"2024-01-24T15:32:55.146479+0000","last_clean":"2024-01-24T15:32:55.146479+0000","last_became_active":"2024-01-24T13:58:13.772446+0000","last_became_peered":"2024-01-24T13:58:13.772446+0000","last_unstale":"2024-01-24T15:32:55.146479+0000","last_undegraded":"2024-01-24T15:32:55.146479+0000","last_fullsized":"2024-01-24T15:32:55.146479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T15:32:55.146389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T15:32:55.146389+0000","last_clean_scrub_stamp":"2024-01-24T15:32:55.146389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575982,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304096,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94067744,"statfs":{"total":96636764160,"available":96325369856,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58799999999999997,"15min":0.60099999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.32000000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77400000000000002,"5min":0.81100000000000005,"15min":0.85499999999999998},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed Jan 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75800000000000001,"5min":0.84499999999999997,"15min":0.878},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77800000000000002,"5min":0.84099999999999997,"15min":0.89600000000000002},"last":0.49099999999999999}]},{"osd":2,"last update":"Wed Jan 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.626,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.374,"15min":0.374},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61499999999999999,"15min":0.627},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.85099999999999998},"last":0.45100000000000001}]},{"osd":3,"last update":"Wed Jan 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.625,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.746,"5min":0.83199999999999996,"15min":0.92600000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.61299999999999999,"15min":0.629},"min":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75800000000000001,"5min":0.86199999999999999,"15min":0.88},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Jan 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.503,"15min":0.49399999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.30599999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.376},{"interface":"front","average":{"1min":0.504,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.74299999999999999,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.52400000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52800000000000002,"15min":0.51300000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32300000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.56499999999999995},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.498}]},{"osd":6,"last update":"Wed Jan 24 15:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":0.81899999999999995},"last":0.432},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.60199999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.35099999999999998},"max":{"1min":0.78300000000000003,"5min":0.879,"15min":0.92700000000000005},"last":0.46600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575983,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.442,"5min":0.377,"15min":0.31900000000000001},"max":{"1min":0.83699999999999997,"5min":0.998,"15min":0.998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.61699999999999999},"min":{"1min":0.48599999999999999,"5min":0.44700000000000001,"15min":0.32800000000000001},"max":{"1min":0.82399999999999995,"5min":0.82699999999999996,"15min":0.95599999999999996},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":1.089},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.45000000000000001,"5min":0.41899999999999998,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.85599999999999998,"15min":0.94299999999999995},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.627},"min":{"1min":0.379,"5min":0.379,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.90900000000000003},"last":0.59899999999999998}]},{"osd":3,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64200000000000002,"15min":0.63900000000000001},"min":{"1min":0.495,"5min":0.42099999999999999,"15min":0.371},"max":{"1min":0.879,"5min":0.96299999999999997,"15min":1.0980000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.42299999999999999},"max":{"1min":0.78300000000000003,"5min":0.83899999999999997,"15min":1.169},"last":0.57099999999999995}]},{"osd":4,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":0.73599999999999999,"5min":0.753,"15min":0.753},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.33100000000000002,"15min":0.312},"max":{"1min":0.79100000000000004,"5min":0.81399999999999995,"15min":0.83999999999999997},"last":0.48099999999999998}]},{"osd":5,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":0.82499999999999996,"15min":0.86499999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69599999999999995,"5min":0.81699999999999995,"15min":0.82799999999999996},"last":0.45600000000000002}]},{"osd":7,"last update":"Wed Jan 24 15:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71099999999999997,"5min":0.83699999999999997,"15min":0.88},"last":0.496},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.83599999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575982,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.434,"15min":0.41999999999999998},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.24299999999999999},"max":{"1min":0.58299999999999996,"5min":3.254,"15min":3.254},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.433,"15min":0.443},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26100000000000001},"max":{"1min":0.64000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.44900000000000001}]},{"osd":2,"last update":"Wed Jan 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45500000000000002,"15min":0.46300000000000002},"min":{"1min":0.318,"5min":0.30299999999999999,"15min":0.28499999999999998},"max":{"1min":0.60899999999999999,"5min":1.306,"15min":1.306},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.44800000000000001,"15min":0.45200000000000001},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.28399999999999997},"max":{"1min":0.53900000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.63}]},{"osd":3,"last update":"Wed Jan 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.45100000000000001,"15min":0.46500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55700000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48599999999999999,"15min":0.495},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66000000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52900000000000003}]},{"osd":4,"last update":"Wed Jan 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67900000000000005,"15min":0.70199999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99299999999999999,"5min":1.1639999999999999,"15min":5.3630000000000004},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68200000000000005,"15min":0.69199999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.40400000000000003},"max":{"1min":0.86199999999999999,"5min":1.244,"15min":1.677},"last":0.748}]},{"osd":5,"last update":"Wed Jan 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.53200000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.014,"5min":1.2410000000000001,"15min":1.7330000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68899999999999995,"15min":0.69999999999999996},"min":{"1min":0.48399999999999999,"5min":0.47399999999999998,"15min":0.46100000000000002},"max":{"1min":0.92500000000000004,"5min":1.081,"15min":1.2989999999999999},"last":0.86899999999999999}]},{"osd":6,"last update":"Wed Jan 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.503,"5min":0.47899999999999998,"15min":0.46800000000000003},"max":{"1min":0.82299999999999995,"5min":1.258,"15min":1.258},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73899999999999999,"15min":0.754},"min":{"1min":0.52800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.96899999999999997,"5min":1.2370000000000001,"15min":1.7729999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Wed Jan 24 15:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70999999999999996,"15min":0.72699999999999998},"min":{"1min":0.53100000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91600000000000004,"5min":1.266,"15min":1.704},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.78800000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96099999999999997,"5min":1.2769999999999999,"15min":5.29},"last":0.96699999999999997}]}]},{"osd":0,"up_from":14,"seq":60129543280,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312284,"kb_used_data":2900,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059556,"statfs":{"total":96636764160,"available":96316985344,"internally_reserved":0,"allocated":2969600,"data_stored":1728114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 24 15:33:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42199999999999999,"15min":0.40699999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55900000000000005,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":0.60399999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.495}]},{"osd":2,"last update":"Wed Jan 24 15:33:08 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46400000000000002,"15min":0.45400000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29599999999999999},"max":{"1min":0.71899999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46700000000000003,"15min":0.46400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70399999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.60199999999999998}]},{"osd":3,"last update":"Wed Jan 24 15:33:08 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.443,"15min":0.44},"min":{"1min":0.30199999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.66800000000000004,"5min":0.67400000000000004,"15min":0.72799999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52900000000000003,"15min":0.51700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73499999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:33:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39600000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.98699999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92100000000000004,"5min":0.998,"15min":0.998},"last":0.91600000000000004}]},{"osd":5,"last update":"Wed Jan 24 15:33:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.40699999999999997},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.70399999999999996,"15min":0.70999999999999996},"min":{"1min":0.504,"5min":0.46700000000000003,"15min":0.46500000000000002},"max":{"1min":0.90200000000000002,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.874}]},{"osd":6,"last update":"Wed Jan 24 15:33:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98299999999999998,"5min":1.27,"15min":1.27},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70899999999999996,"15min":0.70999999999999996},"min":{"1min":0.48399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.0680000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Wed Jan 24 15:33:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71599999999999997,"15min":0.72699999999999998},"min":{"1min":0.46800000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.02,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.47699999999999998},"max":{"1min":0.996,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.86399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575982,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312288,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059552,"statfs":{"total":96636764160,"available":96316981248,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.252},"max":{"1min":0.621,"5min":0.626,"15min":0.69699999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.255,"5min":0.253,"15min":0.253},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Jan 24 15:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42499999999999999,"15min":0.42999999999999999},"min":{"1min":0.26700000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.64900000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46800000000000003,"15min":0.47599999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.42399999999999999}]},{"osd":3,"last update":"Wed Jan 24 15:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46800000000000003,"15min":0.46999999999999997},"min":{"1min":0.34899999999999998,"5min":0.32100000000000001,"15min":0.29099999999999998},"max":{"1min":0.66800000000000004,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.61499999999999999,"5min":0.71999999999999997,"15min":0.77100000000000002},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Jan 24 15:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86899999999999999,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.376},"max":{"1min":0.85499999999999998,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Wed Jan 24 15:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69499999999999995,"15min":0.70799999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94999999999999996,"5min":0.996,"15min":1.0589999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45700000000000002},"max":{"1min":0.92300000000000004,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.66000000000000003}]},{"osd":6,"last update":"Wed Jan 24 15:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.39700000000000002},"max":{"1min":0.91600000000000004,"5min":1.006,"15min":1.0720000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72699999999999998,"15min":0.73599999999999999},"min":{"1min":0.505,"5min":0.495,"15min":0.495},"max":{"1min":0.95799999999999996,"5min":1.0229999999999999,"15min":1.083},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed Jan 24 15:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.72599999999999998,"15min":0.73899999999999999},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.006,"5min":1.0429999999999999,"15min":1.123},"last":0.71399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.318,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.438},"min":{"1min":0.35299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.76100000000000001},"last":0.39300000000000002}]},{"osd":1,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42799999999999999},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.65000000000000002},"last":0.376},{"interface":"front","average":{"1min":0.501,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.752},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.80800000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.81899999999999995},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66000000000000003,"15min":0.66100000000000003},"min":{"1min":0.46500000000000002,"5min":0.46400000000000002,"15min":0.437},"max":{"1min":0.80100000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.93899999999999995,"5min":1.026,"15min":1.026},"last":0.60999999999999999}]},{"osd":5,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":1.056,"15min":1.056},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.499,"15min":0.47199999999999998},"max":{"1min":0.93700000000000006,"5min":1.095,"15min":1.095},"last":0.73999999999999999}]},{"osd":6,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":1.1499999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72099999999999997,"15min":0.72199999999999998},"min":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.0940000000000001},"last":0.68700000000000006}]},{"osd":7,"last update":"Wed Jan 24 15:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.55000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.018,"5min":1.071,"15min":1.083},"last":0.748},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.53600000000000003},"max":{"1min":0.95399999999999996,"5min":1.081,"15min":1.1359999999999999},"last":0.71999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312288,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94059552,"statfs":{"total":96636764160,"available":96316981248,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.46700000000000003,"5min":0.39400000000000002,"15min":0.375},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.38500000000000001,"15min":0.36799999999999999},"max":{"1min":0.80100000000000005,"5min":0.876,"15min":0.876},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.622},"min":{"1min":0.496,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80000000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.45900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.79100000000000004,"5min":0.80400000000000005,"15min":0.82199999999999995},"last":0.498},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.40500000000000003,"15min":0.38500000000000001},"max":{"1min":0.82899999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54200000000000004}]},{"osd":3,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82299999999999995,"5min":0.86799999999999999,"15min":0.87},"last":0.52600000000000002}]},{"osd":5,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.69699999999999995,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.33800000000000002,"5min":0.31900000000000001,"15min":0.308},"max":{"1min":0.66200000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46000000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.33600000000000002},"max":{"1min":0.61199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.37},"max":{"1min":0.72599999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48299999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36699999999999999,"15min":0.34899999999999998},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.375,"15min":0.35799999999999998},"max":{"1min":0.78100000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.43099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575983,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308192,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063648,"statfs":{"total":96636764160,"available":96321175552,"internally_reserved":0,"allocated":2973696,"data_stored":1729075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 24 15:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.39500000000000002,"5min":0.377,"15min":0.34799999999999998},"max":{"1min":0.88200000000000001,"5min":0.88300000000000001,"15min":1.1160000000000001},"last":0.627},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.371},"max":{"1min":0.749,"5min":0.85099999999999998,"15min":1.149},"last":0.57199999999999995}]},{"osd":1,"last update":"Wed Jan 24 15:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":1.125},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.58899999999999997,"15min":0.60299999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77600000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed Jan 24 15:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60999999999999999,"15min":0.627},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.2330000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59699999999999998,"15min":0.60999999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41199999999999998},"max":{"1min":0.70799999999999996,"5min":0.83399999999999996,"15min":1.252},"last":0.58699999999999997}]},{"osd":3,"last update":"Wed Jan 24 15:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76000000000000001,"5min":0.93500000000000005,"15min":0.95499999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":0.90500000000000003,"15min":1.196},"last":0.61499999999999999}]},{"osd":4,"last update":"Wed Jan 24 15:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.747,"5min":0.78700000000000003,"15min":0.90500000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.81299999999999994},"last":0.51900000000000002}]},{"osd":6,"last update":"Wed Jan 24 15:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.503,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73599999999999999,"5min":0.76400000000000001,"15min":1.0209999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.37,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75600000000000001,"5min":0.78100000000000003,"15min":0.98399999999999999},"last":0.41099999999999998}]},{"osd":7,"last update":"Wed Jan 24 15:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.33600000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.86399999999999999},"last":0.437},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.746,"5min":0.83099999999999996,"15min":0.91700000000000004},"last":0.49299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T15:33:14.858 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-01-24T15:33:14.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.986+0000 7f3083212700 1 Processor -- start 2024-01-24T15:33:14.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.986+0000 7f3083212700 1 -- start start 2024-01-24T15:33:14.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f3083212700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f307c0a7210 0x7f307c0c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f3083212700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f307c062ec0 0x7f307c065420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f3083212700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307c0a6d80 con 0x7f307c154100 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f3083212700 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c0bcaf0 con 0x7f307c0a7210 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f3083212700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c0bcd90 con 0x7f307c062ec0 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f307bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f307c0a7210 0x7f307c0c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f307bfff700 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f307c0a7210 0x7f307c0c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60886/0 (socket says 172.21.15.29:60886) 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f307bfff700 1 -- 172.21.15.29:0/653300859 learned_addr learned my addr 172.21.15.29:0/653300859 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f30817af700 1 --2- 172.21.15.29:0/653300859 >> v2:172.21.15.163:3300/0 conn(0x7f307c062ec0 0x7f307c065420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f307bfff700 1 -- 172.21.15.29:0/653300859 >> v2:172.21.15.163:3300/0 conn(0x7f307c062ec0 msgr2=0x7f307c065420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f307bfff700 1 --2- 172.21.15.29:0/653300859 >> v2:172.21.15.163:3300/0 conn(0x7f307c062ec0 0x7f307c065420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f307bfff700 1 -- 172.21.15.29:0/653300859 >> v1:172.21.15.29:6789/0 conn(0x7f307c154100 legacy=0x7f307c0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f307bfff700 1 -- 172.21.15.29:0/653300859 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c0bd070 con 0x7f307c0a7210 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f307bfff700 1 --2- 172.21.15.29:0/653300859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f307c0a7210 0x7f307c0c8200 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 rx=0x7f307c0c2080 tx=0x7f30680090f0).ready entity=mon.0 client_cookie=6d488162b12c8650 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.987+0000 7f307b7fe700 1 -- 172.21.15.29:0/653300859 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306800b520 con 0x7f307c0a7210 2024-01-24T15:33:14.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.988+0000 7f307b7fe700 1 -- 172.21.15.29:0/653300859 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306800b680 con 0x7f307c0a7210 2024-01-24T15:33:14.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.988+0000 7f307b7fe700 1 -- 172.21.15.29:0/653300859 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306800ba00 con 0x7f307c0a7210 2024-01-24T15:33:14.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.988+0000 7f3083212700 1 -- 172.21.15.29:0/653300859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f307c0a7210 msgr2=0x7f307c0c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:14.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.988+0000 7f3083212700 1 --2- 172.21.15.29:0/653300859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f307c0a7210 0x7f307c0c8200 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 rx=0x7f307c0c2080 tx=0x7f30680090f0).stop 2024-01-24T15:33:14.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.988+0000 7f3083212700 1 -- 172.21.15.29:0/653300859 shutdown_connections 2024-01-24T15:33:14.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.988+0000 7f3083212700 1 --2- 172.21.15.29:0/653300859 >> v2:172.21.15.163:3300/0 conn(0x7f307c062ec0 0x7f307c065420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:14.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.988+0000 7f3083212700 1 --2- 172.21.15.29:0/653300859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f307c0a7210 0x7f307c0c8200 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:14.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.988+0000 7f3083212700 1 -- 172.21.15.29:0/653300859 >> 172.21.15.29:0/653300859 conn(0x7f307c064390 msgr2=0x7f307c0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:33:14.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.988+0000 7f3083212700 1 -- 172.21.15.29:0/653300859 shutdown_connections 2024-01-24T15:33:14.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.988+0000 7f3083212700 1 -- 172.21.15.29:0/653300859 wait complete. 2024-01-24T15:33:14.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.989+0000 7f3083212700 1 Processor -- start 2024-01-24T15:33:14.992 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.990+0000 7f3083212700 1 -- start start 2024-01-24T15:33:14.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.990+0000 7f3083212700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f307c062ec0 0x7f307c14e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:14.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.990+0000 7f3083212700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f307c0a7210 0x7f307c0aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:14.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.990+0000 7f3083212700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f307c154100 0x7f307c0ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:14.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.990+0000 7f3083212700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0ab730 con 0x7f307c062ec0 2024-01-24T15:33:14.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.990+0000 7f3083212700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0a8be0 con 0x7f307c154100 2024-01-24T15:33:14.994 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.990+0000 7f3083212700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0ad950 con 0x7f307c0a7210 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3080fae700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f307c062ec0 0x7f307c14e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3080fae700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f307c062ec0 0x7f307c14e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60898/0 (socket says 172.21.15.29:60898) 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3080fae700 1 -- 172.21.15.29:0/1071452077 learned_addr learned my addr 172.21.15.29:0/1071452077 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f307bfff700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f307c0a7210 0x7f307c0aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f30817af700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f307c154100 0x7f307c0ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3080fae700 1 -- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f307c0a7210 msgr2=0x7f307c0aad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3080fae700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f307c0a7210 0x7f307c0aad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3080fae700 1 -- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f307c154100 msgr2=0x7f307c0ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3080fae700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f307c154100 0x7f307c0ab240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3080fae700 1 -- 172.21.15.29:0/1071452077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3068009370 con 0x7f307c062ec0 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f30817af700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f307c154100 0x7f307c0ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3080fae700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f307c062ec0 0x7f307c14e880 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 rx=0x7f307000c440 tx=0x7f3070007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3078ff9700 1 -- 172.21.15.29:0/1071452077 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307000ce40 con 0x7f307c062ec0 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3078ff9700 1 -- 172.21.15.29:0/1071452077 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3070007020 con 0x7f307c062ec0 2024-01-24T15:33:14.995 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3078ff9700 1 -- 172.21.15.29:0/1071452077 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30700073c0 con 0x7f307c062ec0 2024-01-24T15:33:14.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3083212700 1 -- 172.21.15.29:0/1071452077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c0ab930 con 0x7f307c062ec0 2024-01-24T15:33:14.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.991+0000 7f3083212700 1 -- 172.21.15.29:0/1071452077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c0abef0 con 0x7f307c062ec0 2024-01-24T15:33:14.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.993+0000 7f3083212700 1 -- 172.21.15.29:0/1071452077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c0475e0 con 0x7f307c062ec0 2024-01-24T15:33:14.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.993+0000 7f3078ff9700 1 -- 172.21.15.29:0/1071452077 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f30700068b0 con 0x7f307c062ec0 2024-01-24T15:33:14.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.993+0000 7f3078ff9700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f306005f270 0x7f3060061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:14.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.993+0000 7f3078ff9700 1 -- 172.21.15.29:0/1071452077 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f3070079ae0 con 0x7f307c062ec0 2024-01-24T15:33:14.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.994+0000 7f307bfff700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f306005f270 0x7f3060061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:14.997 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.994+0000 7f307bfff700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f306005f270 0x7f3060061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f3068009130 tx=0x7f3068017000).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:33:15.000 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:14.998+0000 7f3078ff9700 1 -- 172.21.15.29:0/1071452077 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f3070047b00 con 0x7f307c062ec0 2024-01-24T15:33:15.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.180+0000 7f3083212700 1 -- 172.21.15.29:0/1071452077 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f307c06a1e0 con 0x7f307c062ec0 2024-01-24T15:33:15.186 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.184+0000 7f3078ff9700 1 -- 172.21.15.29:0/1071452077 <== mon.0 v2:172.21.15.29:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f307004a850 con 0x7f307c062ec0 2024-01-24T15:33:15.187 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.185+0000 7f3078ff9700 1 -- 172.21.15.29:0/1071452077 <== mon.0 v2:172.21.15.29:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v46)=0 v46) v1 ==== 123+0+0 (secure 0 0 0) 0x7f307004e050 con 0x7f307c062ec0 2024-01-24T15:33:15.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.188+0000 7f3083212700 1 -- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f306005f270 msgr2=0x7f3060061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:15.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.188+0000 7f3083212700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f306005f270 0x7f3060061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f3068009130 tx=0x7f3068017000).stop 2024-01-24T15:33:15.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.188+0000 7f3083212700 1 -- 172.21.15.29:0/1071452077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f307c062ec0 msgr2=0x7f307c14e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:15.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.188+0000 7f3083212700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f307c062ec0 0x7f307c14e880 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 rx=0x7f307000c440 tx=0x7f3070007e00).stop 2024-01-24T15:33:15.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.188+0000 7f3083212700 1 -- 172.21.15.29:0/1071452077 shutdown_connections 2024-01-24T15:33:15.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.188+0000 7f3083212700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f307c154100 0x7f307c0ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:15.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.188+0000 7f3083212700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f307c062ec0 0x7f307c14e880 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:15.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.188+0000 7f3083212700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f306005f270 0x7f3060061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:15.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.188+0000 7f3083212700 1 --2- 172.21.15.29:0/1071452077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f307c0a7210 0x7f307c0aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:15.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.188+0000 7f3083212700 1 -- 172.21.15.29:0/1071452077 >> 172.21.15.29:0/1071452077 conn(0x7f307c064390 msgr2=0x7f307c0c3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:33:15.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.188+0000 7f3083212700 1 -- 172.21.15.29:0/1071452077 shutdown_connections 2024-01-24T15:33:15.191 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.188+0000 7f3083212700 1 -- 172.21.15.29:0/1071452077 wait complete. 2024-01-24T15:33:15.203 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-24T15:33:15.203 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-24T15:33:15.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.330+0000 7f91881cc700 1 Processor -- start 2024-01-24T15:33:15.333 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.331+0000 7f91881cc700 1 -- start start 2024-01-24T15:33:15.335 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.331+0000 7f91881cc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91800627b0 0x7f9180062c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f91881cc700 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9180065770 con 0x7f9180138330 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f91881cc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9180069cf0 con 0x7f9180138710 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f91881cc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9180063100 con 0x7f91800627b0 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f9185f68700 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f9180138330 0x7f91800bca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:40874/0 (socket says 172.21.15.29:40874) 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f9185f68700 1 -- 172.21.15.29:0/2714923152 learned_addr learned my addr 172.21.15.29:0/2714923152 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f9186769700 1 --2- 172.21.15.29:0/2714923152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91800627b0 0x7f9180062c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f9184f66700 1 -- 172.21.15.29:0/2714923152 <== mon.0 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521274551 0 0) 0x7f9180065770 con 0x7f9180138330 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f9184f66700 1 -- 172.21.15.29:0/2714923152 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f916c0033f0 con 0x7f9180138330 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f9184f66700 1 -- 172.21.15.29:0/2714923152 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197455135 0 0) 0x7f9180069cf0 con 0x7f9180138710 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f9184f66700 1 -- 172.21.15.29:0/2714923152 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9180065770 con 0x7f9180138710 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f9186769700 1 -- 172.21.15.29:0/2714923152 >> v1:172.21.15.163:6789/0 conn(0x7f9180138710 legacy=0x7f91800be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f9186769700 1 -- 172.21.15.29:0/2714923152 >> v1:172.21.15.29:6789/0 conn(0x7f9180138330 legacy=0x7f91800bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f9186769700 1 -- 172.21.15.29:0/2714923152 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9180069f60 con 0x7f91800627b0 2024-01-24T15:33:15.336 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.332+0000 7f9186769700 1 --2- 172.21.15.29:0/2714923152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91800627b0 0x7f9180062c10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f9178009e90 tx=0x7f9178009780).ready entity=mon.2 client_cookie=20795115778fd3ad server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:33:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.333+0000 7f9184f66700 1 -- 172.21.15.29:0/2714923152 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917800e070 con 0x7f91800627b0 2024-01-24T15:33:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.333+0000 7f9184f66700 1 -- 172.21.15.29:0/2714923152 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f917800bab0 con 0x7f91800627b0 2024-01-24T15:33:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.333+0000 7f9184f66700 1 -- 172.21.15.29:0/2714923152 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917800be50 con 0x7f91800627b0 2024-01-24T15:33:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.333+0000 7f91881cc700 1 -- 172.21.15.29:0/2714923152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91800627b0 msgr2=0x7f9180062c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.333+0000 7f91881cc700 1 --2- 172.21.15.29:0/2714923152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91800627b0 0x7f9180062c10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f9178009e90 tx=0x7f9178009780).stop 2024-01-24T15:33:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.333+0000 7f91881cc700 1 -- 172.21.15.29:0/2714923152 shutdown_connections 2024-01-24T15:33:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.333+0000 7f91881cc700 1 --2- 172.21.15.29:0/2714923152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91800627b0 0x7f9180062c10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.333+0000 7f91881cc700 1 -- 172.21.15.29:0/2714923152 >> 172.21.15.29:0/2714923152 conn(0x7f91800c8580 msgr2=0x7f91800c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:33:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.333+0000 7f91881cc700 1 -- 172.21.15.29:0/2714923152 shutdown_connections 2024-01-24T15:33:15.337 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.333+0000 7f91881cc700 1 -- 172.21.15.29:0/2714923152 wait complete. 2024-01-24T15:33:15.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.335+0000 7f91881cc700 1 Processor -- start 2024-01-24T15:33:15.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.335+0000 7f91881cc700 1 -- start start 2024-01-24T15:33:15.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.335+0000 7f91881cc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91800627b0 0x7f91800bf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:15.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.335+0000 7f91881cc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9180138330 0x7f9180163790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:15.340 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.335+0000 7f91881cc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9180138710 0x7f91800c1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.335+0000 7f91881cc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91800c1c40 con 0x7f9180138330 2024-01-24T15:33:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.335+0000 7f91881cc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918014e4e0 con 0x7f91800627b0 2024-01-24T15:33:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.335+0000 7f91881cc700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91800bf530 con 0x7f9180138710 2024-01-24T15:33:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.335+0000 7f9185767700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9180138330 0x7f9180163790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9185767700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9180138330 0x7f9180163790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:60904/0 (socket says 172.21.15.29:60904) 2024-01-24T15:33:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9185767700 1 -- 172.21.15.29:0/3581219614 learned_addr learned my addr 172.21.15.29:0/3581219614 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-01-24T15:33:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9186769700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9180138710 0x7f91800c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9185f68700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91800627b0 0x7f91800bf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:15.341 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9185767700 1 -- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9180138710 msgr2=0x7f91800c1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9185767700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9180138710 0x7f91800c1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9185767700 1 -- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91800627b0 msgr2=0x7f91800bf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9185767700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91800627b0 0x7f91800bf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9185767700 1 -- 172.21.15.29:0/3581219614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9178009a20 con 0x7f9180138330 2024-01-24T15:33:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9186769700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9180138710 0x7f91800c1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:33:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9185f68700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91800627b0 0x7f91800bf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-01-24T15:33:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9185767700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9180138330 0x7f9180163790 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 rx=0x7f917000c4c0 tx=0x7f9170007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:33:15.342 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9175ffb700 1 -- 172.21.15.29:0/3581219614 <== mon.0 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917000f470 con 0x7f9180138330 2024-01-24T15:33:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9175ffb700 1 -- 172.21.15.29:0/3581219614 <== mon.0 v2:172.21.15.29:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f917000f5d0 con 0x7f9180138330 2024-01-24T15:33:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f91881cc700 1 -- 172.21.15.29:0/3581219614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918014dbb0 con 0x7f9180138330 2024-01-24T15:33:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f9175ffb700 1 -- 172.21.15.29:0/3581219614 <== mon.0 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917000f990 con 0x7f9180138330 2024-01-24T15:33:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.336+0000 7f91881cc700 1 -- 172.21.15.29:0/3581219614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918014e1f0 con 0x7f9180138330 2024-01-24T15:33:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.338+0000 7f91881cc700 1 -- 172.21.15.29:0/3581219614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91800475e0 con 0x7f9180138330 2024-01-24T15:33:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.338+0000 7f9175ffb700 1 -- 172.21.15.29:0/3581219614 <== mon.0 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f917000fd40 con 0x7f9180138330 2024-01-24T15:33:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.338+0000 7f9175ffb700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f916405f240 0x7f9164061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-01-24T15:33:15.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.338+0000 7f9175ffb700 1 -- 172.21.15.29:0/3581219614 <== mon.0 v2:172.21.15.29:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f9170079b90 con 0x7f9180138330 2024-01-24T15:33:15.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.339+0000 7f9185f68700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f916405f240 0x7f9164061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-24T15:33:15.344 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.339+0000 7f9185f68700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f916405f240 0x7f9164061700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f917c005750 tx=0x7f917c009040).ready entity=mgr.9763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-24T15:33:15.345 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.343+0000 7f9175ffb700 1 -- 172.21.15.29:0/3581219614 <== mon.0 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162534 (secure 0 0 0) 0x7f9170048b00 con 0x7f9180138330 2024-01-24T15:33:15.570 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.568+0000 7f91881cc700 1 -- 172.21.15.29:0/3581219614 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f918006a240 con 0x7f9180138330 2024-01-24T15:33:15.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.569+0000 7f9175ffb700 1 -- 172.21.15.29:0/3581219614 <== mon.0 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3537 (secure 0 0 0) 0x7f917004ab30 con 0x7f9180138330 2024-01-24T15:33:15.571 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-01-24T15:33:15.571 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":100,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1883755467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1883755467},{"type":"v1","addr":"172.21.15.163:6836","nonce":1883755467}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1773668659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1773668659},{"type":"v1","addr":"172.21.15.29:6834","nonce":1773668659}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3438722509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3438722509},{"type":"v1","addr":"172.21.15.29:6835","nonce":3438722509}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8657,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2573533258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2573533258},{"type":"v1","addr":"172.21.15.163:6837","nonce":2573533258}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[]} 2024-01-24T15:33:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.571+0000 7f91881cc700 1 -- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f916405f240 msgr2=0x7f9164061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.571+0000 7f91881cc700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f916405f240 0x7f9164061700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f917c005750 tx=0x7f917c009040).stop 2024-01-24T15:33:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.571+0000 7f91881cc700 1 -- 172.21.15.29:0/3581219614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9180138330 msgr2=0x7f9180163790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T15:33:15.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.571+0000 7f91881cc700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9180138330 0x7f9180163790 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 rx=0x7f917000c4c0 tx=0x7f9170007c90).stop 2024-01-24T15:33:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.571+0000 7f91881cc700 1 -- 172.21.15.29:0/3581219614 shutdown_connections 2024-01-24T15:33:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.571+0000 7f91881cc700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:6832/57365,v1:172.21.15.163:6833/57365] conn(0x7f916405f240 0x7f9164061700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.571+0000 7f91881cc700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9180138710 0x7f91800c1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.571+0000 7f91881cc700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91800627b0 0x7f91800bf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.571+0000 7f91881cc700 1 --2- 172.21.15.29:0/3581219614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9180138330 0x7f9180163790 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-01-24T15:33:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.571+0000 7f91881cc700 1 -- 172.21.15.29:0/3581219614 >> 172.21.15.29:0/3581219614 conn(0x7f91800c8580 msgr2=0x7f91800c70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T15:33:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.571+0000 7f91881cc700 1 -- 172.21.15.29:0/3581219614 shutdown_connections 2024-01-24T15:33:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:2024-01-24T15:33:15.571+0000 7f91881cc700 1 -- 172.21.15.29:0/3581219614 wait complete. 2024-01-24T15:33:15.574 INFO:teuthology.orchestra.run.smithi029.stderr:dumped fsmap epoch 100 2024-01-24T15:33:15.586 DEBUG:tasks.cephfs.filesystem:already dead... 2024-01-24T15:33:15.586 INFO:teuthology.misc:Shutting down mds daemons... 2024-01-24T15:33:15.587 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-24T15:33:15.587 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:15.596 INFO:tasks.ceph.mds.a:Stopped 2024-01-24T15:33:15.596 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-24T15:33:15.596 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:15.645 INFO:tasks.ceph.mds.c:Stopped 2024-01-24T15:33:15.646 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-24T15:33:15.646 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:15.656 INFO:tasks.ceph.mds.b:Stopped 2024-01-24T15:33:15.657 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-24T15:33:15.657 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:15.708 INFO:tasks.ceph.mds.d:Stopped 2024-01-24T15:33:15.708 INFO:teuthology.misc:Shutting down osd daemons... 2024-01-24T15:33:15.708 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-01-24T15:33:15.709 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:15.748 INFO:tasks.ceph.osd.0:Stopped 2024-01-24T15:33:15.748 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-01-24T15:33:15.748 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:15.824 INFO:tasks.ceph.osd.1:Stopped 2024-01-24T15:33:15.824 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-01-24T15:33:15.824 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:15.893 INFO:tasks.ceph.osd.2:Stopped 2024-01-24T15:33:15.894 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-01-24T15:33:15.894 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:15.969 INFO:tasks.ceph.osd.3:Stopped 2024-01-24T15:33:15.970 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-01-24T15:33:15.970 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:16.003 INFO:tasks.ceph.osd.4:Stopped 2024-01-24T15:33:16.003 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-01-24T15:33:16.003 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:16.078 INFO:tasks.ceph.osd.5:Stopped 2024-01-24T15:33:16.079 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-01-24T15:33:16.079 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:16.154 INFO:tasks.ceph.osd.6:Stopped 2024-01-24T15:33:16.155 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-01-24T15:33:16.155 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:16.226 INFO:tasks.ceph.osd.7:Stopped 2024-01-24T15:33:16.226 INFO:teuthology.misc:Shutting down mgr daemons... 2024-01-24T15:33:16.226 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-01-24T15:33:16.226 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:16.255 INFO:tasks.ceph.mgr.y:Stopped 2024-01-24T15:33:16.256 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-01-24T15:33:16.256 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:16.294 INFO:tasks.ceph.mgr.x:Stopped 2024-01-24T15:33:16.294 INFO:teuthology.misc:Shutting down mon daemons... 2024-01-24T15:33:16.294 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-24T15:33:16.294 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:16.320 INFO:tasks.ceph.mon.a:Stopped 2024-01-24T15:33:16.320 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-24T15:33:16.320 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:16.350 INFO:tasks.ceph.mon.b:Stopped 2024-01-24T15:33:16.350 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-01-24T15:33:16.350 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T15:33:16.407 INFO:tasks.ceph.mon.c:Stopped 2024-01-24T15:33:16.407 INFO:tasks.ceph:Checking cluster log for badness... 2024-01-24T15:33:16.407 DEBUG:teuthology.orchestra.run.smithi029:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-01-24T15:33:16.447 INFO:teuthology.orchestra.run.smithi029.stdout:2024-01-24T14:00:46.712338+0000 mon.a (mon.0) 365 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-01-24T15:33:16.448 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-01-24T15:33:16.448 DEBUG:teuthology.orchestra.run.smithi029:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-01-24T15:33:16.529 DEBUG:teuthology.orchestra.run.smithi029:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-01-24T15:33:16.609 DEBUG:teuthology.orchestra.run.smithi029:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-01-24T15:33:16.690 INFO:teuthology.orchestra.run.smithi029.stdout:2024-01-24T14:00:46.712338+0000 mon.a (mon.0) 365 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-01-24T15:33:16.690 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T15:33:16.691 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-01-24T15:33:16.957 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T15:33:16.958 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-01-24T15:33:17.129 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T15:33:17.129 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-01-24T15:33:17.220 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T15:33:17.220 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-01-24T15:33:17.319 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T15:33:17.320 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-01-24T15:33:17.598 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T15:33:17.598 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-01-24T15:33:17.695 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T15:33:17.695 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-01-24T15:33:17.794 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T15:33:17.795 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-01-24T15:33:17.870 INFO:tasks.ceph:Archiving mon data... 2024-01-24T15:33:17.871 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993/data/mon.a.tgz 2024-01-24T15:33:17.871 DEBUG:teuthology.orchestra.run.smithi029:> mktemp 2024-01-24T15:33:17.893 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/tmp.4hZiouMyvD 2024-01-24T15:33:17.894 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4hZiouMyvD 2024-01-24T15:33:18.469 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0666 /tmp/tmp.4hZiouMyvD 2024-01-24T15:33:18.562 DEBUG:teuthology.orchestra.remote:smithi029:/tmp/tmp.4hZiouMyvD is 2MB 2024-01-24T15:33:18.692 DEBUG:teuthology.orchestra.run.smithi029:> rm -fr /tmp/tmp.4hZiouMyvD 2024-01-24T15:33:18.714 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993/data/mon.b.tgz 2024-01-24T15:33:18.714 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2024-01-24T15:33:18.735 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.QCEa0XaFa9 2024-01-24T15:33:18.736 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.QCEa0XaFa9 2024-01-24T15:33:19.238 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.QCEa0XaFa9 2024-01-24T15:33:19.328 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.QCEa0XaFa9 is 2MB 2024-01-24T15:33:19.428 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.QCEa0XaFa9 2024-01-24T15:33:19.444 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993/data/mon.c.tgz 2024-01-24T15:33:19.445 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2024-01-24T15:33:19.501 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.55N4xNAmsN 2024-01-24T15:33:19.502 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.55N4xNAmsN 2024-01-24T15:33:19.980 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.55N4xNAmsN 2024-01-24T15:33:20.068 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.55N4xNAmsN is 2MB 2024-01-24T15:33:20.196 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.55N4xNAmsN 2024-01-24T15:33:20.213 INFO:tasks.ceph:Cleaning ceph cluster... 2024-01-24T15:33:20.213 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-24T15:33:20.215 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-24T15:33:20.279 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-01-24T15:33:20.279 DEBUG:teuthology.orchestra.run.smithi029:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-01-24T15:33:20.291 DEBUG:teuthology.orchestra.run.smithi163:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-01-24T15:33:20.323 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-01-24T15:33:20.346 INFO:teuthology.orchestra.run.smithi163.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-01-24T15:33:20.356 INFO:tasks.ceph:Archiving crash dumps... 2024-01-24T15:33:20.358 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993/remote/smithi029/crash 2024-01-24T15:33:20.359 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-24T15:33:20.405 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993/remote/smithi163/crash 2024-01-24T15:33:20.406 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-24T15:33:20.432 INFO:tasks.ceph:Shutting down logrotate 2024-01-24T15:33:20.433 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:33:20.450 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-24T15:33:20.543 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-24T15:33:20.578 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-24T15:33:20.604 INFO:tasks.ceph:Compressing logs... 2024-01-24T15:33:20.604 DEBUG:teuthology.orchestra.run.smithi029:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-24T15:33:20.621 DEBUG:teuthology.orchestra.run.smithi163:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-24T15:33:20.654 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80538.log 2024-01-24T15:33:20.654 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70925.log 2024-01-24T15:33:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58614.log 2024-01-24T15:33:20.655 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80538.log.gz 2024-01-24T15:33:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.70925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81431.log 2024-01-24T15:33:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58614.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70925.log.gz 2024-01-24T15:33:20.655 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72348.log 2024-01-24T15:33:20.656 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58614.log.gz 2024-01-24T15:33:20.656 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68356.log 2024-01-24T15:33:20.659 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81431.log: /var/log/ceph/ceph-client.admin.72348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101668.log 2024-01-24T15:33:20.659 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81431.log.gz 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68356.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72348.log.gz 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69962.log 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.101668.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68356.log.gz 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101668.log.gz 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49498.log 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99440.log 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69962.log.gz 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73349.log 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49498.log.gz 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51021.log 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73349.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.99440.log.gz 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60657.log 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73349.log.gz 2024-01-24T15:33:20.660 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78169.log 2024-01-24T15:33:20.661 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51021.log.gz 2024-01-24T15:33:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88386.log 2024-01-24T15:33:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78169.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60657.log.gz 2024-01-24T15:33:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78355.log 2024-01-24T15:33:20.661 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78169.log.gz 2024-01-24T15:33:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100299.log 2024-01-24T15:33:20.661 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88386.log.gz 2024-01-24T15:33:20.661 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81270.log 2024-01-24T15:33:20.668 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100299.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78355.log.gz 2024-01-24T15:33:20.668 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91759.log 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100299.log.gz 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54212.log 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81270.log.gz 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96357.log 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54212.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91759.log.gz 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80733.log 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54212.log.gz 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70536.log 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96357.log.gz 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82883.log 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.70536.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80733.log.gz 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91254.log 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70536.log.gz 2024-01-24T15:33:20.669 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99546.log 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82883.log.gz 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93931.log 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91254.log.gz 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55924.log 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99546.log.gz 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.93931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78215.log 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.93931.log.gz 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89820.log 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78215.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55924.log.gz 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54671.log 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78215.log.gz 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89726.log 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89820.log.gz 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74077.log 2024-01-24T15:33:20.670 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89726.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54671.log.gz 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67861.log 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89726.log.gz 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53276.log 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74077.log.gz 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.67861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54553.log 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67861.log.gz 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53644.log 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54553.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53276.log.gz 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71072.log 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54553.log.gz 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71609.log 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53644.log.gz 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71072.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71609.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71072.log.gz 2024-01-24T15:33:20.671 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99343.log 2024-01-24T15:33:20.672 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71609.log.gz 2024-01-24T15:33:20.672 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63642.log 2024-01-24T15:33:20.672 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69822.log 2024-01-24T15:33:20.680 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-01-24T15:33:20.680 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.48440.log 2024-01-24T15:33:20.680 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43591.log 2024-01-24T15:33:20.680 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.3.48440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47495.log 2024-01-24T15:33:20.680 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.48440.log.gz 2024-01-24T15:33:20.680 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43499.log 2024-01-24T15:33:20.680 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.47495.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43591.log.gz 2024-01-24T15:33:20.680 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-01-24T15:33:20.680 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47495.log.gz 2024-01-24T15:33:20.680 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47977.log 2024-01-24T15:33:20.680 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43499.log: /var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-24T15:33:20.680 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43499.log.gz 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.1.47977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47977.log.gz 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43736.log 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43453.log 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.43736.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43736.log.gz 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43453.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mds.d.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43453.log.gz 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48010.log 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48010.log: /var/log/ceph/ceph-mon.c.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48010.log.gz 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43782.log 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43782.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43782.log.gz 2024-01-24T15:33:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-01-24T15:33:20.691 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mon.b.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-24T15:33:20.691 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43643.log 2024-01-24T15:33:20.691 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43643.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.43643.log.gz 2024-01-24T15:33:20.692 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-01-24T15:33:20.692 INFO:teuthology.orchestra.run.smithi163.stderr: 94.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-24T15:33:20.697 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43690.log 2024-01-24T15:33:20.697 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43690.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43690.log.gz 2024-01-24T15:33:20.697 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-01-24T15:33:20.702 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-01-24T15:33:20.709 INFO:teuthology.orchestra.run.smithi163.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-01-24T15:33:20.723 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.4.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-01-24T15:33:20.723 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43545.log 2024-01-24T15:33:20.723 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43545.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43545.log.gz 2024-01-24T15:33:20.727 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99343.log.gz 2024-01-24T15:33:20.727 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103062.log 2024-01-24T15:33:20.727 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63642.log.gz 2024-01-24T15:33:20.727 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69143.log 2024-01-24T15:33:20.727 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103062.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69822.log.gz 2024-01-24T15:33:20.727 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92743.log 2024-01-24T15:33:20.727 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103062.log.gz 2024-01-24T15:33:20.727 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96173.log 2024-01-24T15:33:20.727 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69143.log.gz 2024-01-24T15:33:20.727 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49268.log 2024-01-24T15:33:20.727 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92743.log.gz 2024-01-24T15:33:20.727 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68658.log 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96173.log.gz 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95558.log 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68658.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49268.log.gz 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49958.log 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68658.log.gz 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76585.log 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95558.log.gz 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53506.log 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49958.log.gz 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57101.log 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76585.log.gz 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88639.log 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53506.log.gz 2024-01-24T15:33:20.728 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100638.log 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57101.log.gz 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69776.log 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100638.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88639.log.gz 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52907.log 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100638.log.gz 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47711.log 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69776.log.gz 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59102.log 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52907.log.gz 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74586.log 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59102.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47711.log.gz 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64265.log 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59102.log.gz 2024-01-24T15:33:20.729 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87794.log 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74586.log.gz 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56393.log 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64265.log.gz 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.87794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50652.log 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87794.log.gz 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82275.log 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56393.log.gz 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92171.log 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50652.log.gz 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47759.log 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82275.log.gz 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97041.log 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7%/var/log/ceph/ceph-client.admin.47759.log: -- replaced with /var/log/ceph/ceph-client.admin.92171.log.gz 2024-01-24T15:33:20.730 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90503.log 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.97041.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47759.log.gz 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63956.log 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97041.log.gz 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82647.log 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63956.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90503.log.gz 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77612.log 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63956.log.gz 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.82647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101766.log 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.82647.log.gz 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103934.log 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101766.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77612.log.gz 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72902.log 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101766.log.gz 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55648.log 2024-01-24T15:33:20.731 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103934.log.gz 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81040.log 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68064.log 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55648.log.gz 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.72902.log.gz 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63342.log 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76836.log 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68064.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81040.log.gz 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96127.log 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68064.log.gz 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63342.log.gz 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53230.log 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96127.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76836.log.gz 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64311.log 2024-01-24T15:33:20.732 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96127.log.gz 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90392.log 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64311.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53230.log.gz 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54830.log 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90392.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64311.log.gz 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92465.log 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90392.log.gz 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64961.log 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54830.log.gz 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101158.log 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.64961.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92465.log.gz 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64961.log.gz 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88038.log 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100460.log 2024-01-24T15:33:20.733 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88038.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101158.log.gz 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83037.log 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88038.log.gz 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51297.log 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100460.log.gz 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.83037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58430.log 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83037.log.gz 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77926.log 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51297.log.gz 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103658.log 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58430.log.gz 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79087.log 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77926.log.gz 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54120.log 2024-01-24T15:33:20.734 INFO:teuthology.orchestra.run.smithi029.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103658.log.gz 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.79087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95605.log 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79087.log.gz 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97440.log 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54120.log.gz 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89772.log 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95605.log.gz 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.97440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76170.log 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97440.log.gz 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75628.log 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95246.log 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.89772.log.gz 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75628.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76170.log.gz 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76121.log 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.75628.log.gz 2024-01-24T15:33:20.735 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88202.log 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95246.log.gz 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68259.log 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76121.log.gz 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88478.log 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68259.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88202.log.gz 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53936.log 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68259.log.gz 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55060.log 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88478.log.gz 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55694.log 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55060.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53936.log.gz 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76401.log 2024-01-24T15:33:20.736 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55060.log.gz 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70194.log 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55694.log.gz 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75352.log 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.70194.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.76401.log.gz 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70194.log.gz 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54074.log 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88340.log 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75352.log.gz 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102084.log 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54074.log.gz 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88549.log 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102084.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88340.log.gz 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75062.log 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.102084.log.gz/var/log/ceph/ceph-client.admin.88549.log: 2024-01-24T15:33:20.737 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77880.log 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88549.log.gz 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71769.log 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75062.log.gz 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102707.log 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71769.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77880.log.gz 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103382.log 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71769.log.gz 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55462.log 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102707.log.gz 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49406.log 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55462.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103382.log.gz 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102170.log 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55462.log.gz 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51506.log 2024-01-24T15:33:20.738 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102170.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49406.log.gz 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88817.log 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102170.log.gz 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77788.log 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90151.log 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88817.log.gz 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103474.log 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77788.log.gz 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92975.log 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90151.log.gz 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91393.log 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103474.log.gz 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.92975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77495.log 2024-01-24T15:33:20.739 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.92975.log.gz 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63035.log 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91393.log.gz 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71379.log 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77495.log.gz 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67597.log 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63035.log.gz/var/log/ceph/ceph-client.admin.71379.log: 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71379.log.gz 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.67597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49544.log 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mds.c.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67597.log.gz 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90795.log 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64359.log 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49544.log.gz 2024-01-24T15:33:20.740 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100692.log 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90795.log.gz 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55340.log 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52794.log /var/log/ceph/ceph-client.admin.100692.log: 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64359.log.gz 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100692.log.gz 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104256.log 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62987.log 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55340.log.gz 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52794.log.gz 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.104256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75444.log 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0%/var/log/ceph/ceph-client.admin.62987.log: -- replaced with /var/log/ceph/ceph-client.admin.104256.log.gz 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100928.log 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62987.log.gz 2024-01-24T15:33:20.741 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69005.log 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75444.log.gz 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76634.log 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97596.log 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100928.log.gz 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76634.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69005.log.gz 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96696.log 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.76634.log.gz 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.97596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74175.log 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97596.log.gz 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97642.log 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96696.log: /var/log/ceph/ceph-client.admin.74175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90993.log 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96696.log.gz 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.97642.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.74175.log.gz 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69477.log 2024-01-24T15:33:20.742 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90993.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97642.log.gz 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50559.log 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90993.log.gz 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81132.log 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69477.log.gz 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73783.log 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50559.log.gz 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79243.log 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73783.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81132.log.gz 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59412.log 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73783.log.gz 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.79243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52953.log 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79243.log.gz 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49912.log 2024-01-24T15:33:20.743 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59412.log: /var/log/ceph/ceph-client.admin.52953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78261.log 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59412.log.gz 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49912.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52953.log.gz 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99955.log 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71855.log 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2%/var/log/ceph/ceph-client.admin.78261.log: -- replaced with /var/log/ceph/ceph-client.admin.49912.log.gz 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99955.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78261.log.gz 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64219.log 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99955.log.gz 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80880.log 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71855.log: /var/log/ceph/ceph-client.admin.64219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70674.log 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71855.log.gz 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64219.log.gz 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53798.log 2024-01-24T15:33:20.744 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100002.log 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80880.log.gz 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.70674.log: /var/log/ceph/ceph-client.admin.53798.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70674.log.gz 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52656.log 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53798.log.gz 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50145.log 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81851.log 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100002.log.gz 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52656.log: /var/log/ceph/ceph-client.admin.50145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63249.log 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52656.log.gz 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50145.log.gz 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68406.log 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63249.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81851.log.gz 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74924.log 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63249.log.gz 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100784.log 2024-01-24T15:33:20.745 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74924.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68406.log.gz 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81951.log 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74924.log.gz 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97370.log 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100784.log: /var/log/ceph/ceph-client.admin.81951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78115.log 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100784.log.gz 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81951.log.gz 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.97370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88248.log 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53414.log 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78115.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97370.log.gz 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88248.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78115.log.gz 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69051.log 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56062.log 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53414.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88248.log.gz 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69051.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53414.log.gz 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95951.log 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69051.log.gz 2024-01-24T15:33:20.746 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74289.log 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101342.log 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95951.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56062.log.gz 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89197.log 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95951.log.gz 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74289.log.gz 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90052.log 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101342.log.gz 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81525.log 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.90052.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89197.log.gz 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.90052.log.gz 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75490.log 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53552.log 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81525.log.gz 2024-01-24T15:33:20.747 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63388.log 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53552.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75490.log.gz 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99594.log 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53552.log.gz 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52564.log 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63388.log.gz 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47519.log 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52564.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99594.log.gz 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55416.log 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52564.log.gz 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100253.log 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55416.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47519.log.gz 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73398.log 2024-01-24T15:33:20.748 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55416.log.gz 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75016.log 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100253.log.gz 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103750.log 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75016.log: 87.5%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.73398.log.gz--verbose -- 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78631.log 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103750.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75016.log.gz 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69281.log 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103750.log.gz 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47665.log 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104026.log 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47665.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69281.log.gz 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57325.log 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.78631.log.gz 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47665.log.gz 2024-01-24T15:33:20.749 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.104026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99496.log 2024-01-24T15:33:20.750 INFO:teuthology.orchestra.run.smithi029.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104026.log.gz 2024-01-24T15:33:20.750 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79156.log 2024-01-24T15:33:20.750 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57325.log: /var/log/ceph/ceph-client.admin.99496.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.57325.log.gz 2024-01-24T15:33:20.750 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99496.log.gz 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.79156.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79156.log.gz 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96742.log 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80635.log 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54784.log 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96742.log.gz 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64729.log 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80635.log.gz 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72202.log 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54784.log.gz 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68113.log 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64729.log: /var/log/ceph/ceph-client.admin.72202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104414.log 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64729.log.gz 2024-01-24T15:33:20.775 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72202.log.gz 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68867.log 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64165.log 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.104414.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68113.log.gz 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68867.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104414.log.gz 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64165.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68867.log.gz 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64165.log.gz 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64497.log 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90691.log 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76051.log 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90691.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64497.log.gz 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54028.log 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90691.log.gz 2024-01-24T15:33:20.776 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69329.log 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97324.log 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54028.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76051.log.gz 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69329.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54028.log.gz 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55970.log 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63732.log 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.97324.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69329.log.gz 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55970.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97324.log.gz 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95881.log 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55970.log.gz 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63732.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78919.log 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63732.log.gz 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95881.log.gz 2024-01-24T15:33:20.777 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72950.log 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.78919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102804.log 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78919.log.gz 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51251.log 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102804.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72950.log.gz 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52748.log 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.102804.log.gz 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89450.log 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52748.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51251.log.gz 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52748.log.gz 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89450.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89450.log.gz 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76738.log 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56837.log 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82045.log 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56837.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76738.log.gz 2024-01-24T15:33:20.778 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56837.log.gz 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47619.log 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90845.log 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82045.log.gz 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47619.log: /var/log/ceph/ceph-client.admin.90845.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47619.log.gz 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63296.log 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90845.log.gz 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47565.log 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91598.log /var/log/ceph/ceph-client.admin.63296.log: 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87941.log 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63296.log.gz 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7%/var/log/ceph/ceph-client.admin.91598.log: -- replaced with /var/log/ceph/ceph-client.admin.47565.log.gz 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91187.log 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91598.log.gz 2024-01-24T15:33:20.779 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.87941.log: /var/log/ceph/ceph-client.admin.91187.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87941.log.gz 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91187.log.gz 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95743.log 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102756.log 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43695.log 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102756.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95743.log.gz 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102756.log.gz 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43695.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43695.log.gz 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72488.log 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58970.log 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71471.log 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72488.log.gz 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58970.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58970.log.gz 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71471.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71471.log.gz 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50467.log 2024-01-24T15:33:20.780 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59168.log 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76447.log 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59168.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50467.log.gz 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59168.log.gz 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76447.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76447.log.gz 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50745.log 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82183.log 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43388.log 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50745.log.gz 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61269.log 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.82183.log.gz 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92649.log 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43388.log.gz 2024-01-24T15:33:20.781 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82415.log 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61269.log.gz 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80335.log 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92649.log.gzgzip 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.91040.log 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82415.log: /var/log/ceph/ceph-client.admin.80335.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82415.log.gz 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91040.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80335.log.gz 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.91040.log.gz 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101720.log 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.97273.log 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51958.log 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101720.log.gz 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.97273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68612.log 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.97273.log.gz 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 /var/log/ceph/ceph-client.0.51958.log: --verbose -- /var/log/ceph/ceph-client.admin.71425.log 2024-01-24T15:33:20.782 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51958.log.gz 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100592.log 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68612.log.gz 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80947.log 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose 89.0%-- /var/log/ceph/ceph-client.admin.50283.log -- replaced with /var/log/ceph/ceph-client.admin.71425.log.gz 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100592.log: /var/log/ceph/ceph-client.admin.80947.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100592.log.gz 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65284.log 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50283.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80947.log.gz 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.65284.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50283.log.gz 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82814.log 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65284.log.gz 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104072.log 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95490.log 2024-01-24T15:33:20.783 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82814.log.gz 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.104072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93977.log 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90642.log 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104072.log.gz 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95490.log: /var/log/ceph/ceph-client.admin.93977.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95490.log.gz 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90642.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93977.log.gz 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90642.log.gz 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72302.log 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76355.log 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89404.log 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72302.log.gz 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76355.log: /var/log/ceph/ceph-client.admin.89404.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76355.log.gz 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89404.log.gz 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62686.log 2024-01-24T15:33:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63434.log 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81178.log 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62686.log.gz 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93232.log 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81178.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63434.log.gz 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81178.log.gz 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.93232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101112.log 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51343.log 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93232.log.gz 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47957.log 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101112.log.gz 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101250.log 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74381.log 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47957.log: /var/log/ceph/ceph-client.admin.101250.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.51343.log.gz 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47957.log.gz 2024-01-24T15:33:20.785 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5%/var/log/ceph/ceph-client.admin.74381.log: -- replaced with /var/log/ceph/ceph-client.admin.101250.log.gz 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74381.log.gz 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52702.log 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58904.log 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96510.log 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52702.log.gz 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89003.log 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96510.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58904.log.gz 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54304.log 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96510.log.gz 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74970.log 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89003.log.gz 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54304.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74970.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54304.log.gz 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72534.log 2024-01-24T15:33:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mds.a.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74970.log.gz 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58838.log 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102461.log 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr: 87.6%/var/log/ceph/ceph-client.admin.58838.log: -- replaced with /var/log/ceph/ceph-client.admin.72534.log.gz 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102461.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58838.log.gz 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102461.log.gz 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53138.log 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68704.log 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52518.log 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68704.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53138.log.gz 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99888.log 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52518.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68704.log.gz 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74519.log 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52518.log.gz 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87404.log 2024-01-24T15:33:20.787 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99888.log.gz 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74519.log: /var/log/ceph/ceph-client.admin.87404.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74519.log.gz 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87404.log.gz 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100048.log 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90893.log 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100048.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100048.log.gz/var/log/ceph/ceph-client.admin.90893.log: 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80685.log 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90893.log.gz 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76258.log 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80685.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80685.log.gz 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76258.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76258.log.gz 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96453.log 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70245.log 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96453.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96453.log.gz 2024-01-24T15:33:20.788 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.70245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92373.log 2024-01-24T15:33:20.789 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70245.log.gz 2024-01-24T15:33:20.796 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92373.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92373.log.gz 2024-01-24T15:33:20.796 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47861.log 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91531.log 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47861.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47861.log.gz 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91531.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91531.log.gz 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70827.log 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82367.log 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.70827.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70827.log.gz 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82367.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.82367.log.gz 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72133.log 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50191.log 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72133.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72133.log.gz 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50191.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50191.log.gz 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76935.log 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr: 89.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49360.log 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62790.log 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.76935.log.gz 2024-01-24T15:33:20.797 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49360.log: /var/log/ceph/ceph-client.admin.62790.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49360.log.gz 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62790.log.gz 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91138.log 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102608.log 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102405.log 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91138.log.gz 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61883.log 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102405.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102608.log.gz 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50513.log 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102405.log.gz 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61883.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61883.log.gz 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50513.log: /var/log/ceph/ceph-osd.3.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50513.log.gz 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88771.log 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77972.log 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88771.log: gzip -5 --verbose -- 88.2%/var/log/ceph/ceph-client.admin.47379.log 2024-01-24T15:33:20.798 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88771.log.gz 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77972.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77972.log.gz 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47205.log 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47379.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47379.log.gz 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47205.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47205.log.gz 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69730.log 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68014.log 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69730.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69730.log.gz 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68014.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68014.log.gz 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60351.log 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87543.log 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60351.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60351.log.gz 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.87543.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87543.log.gz 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100140.log 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64635.log 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100140.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100140.log.gz 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64635.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64635.log.gz 2024-01-24T15:33:20.799 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58660.log 2024-01-24T15:33:20.800 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96556.log 2024-01-24T15:33:20.800 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58660.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58660.log.gz 2024-01-24T15:33:20.800 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96556.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96556.log.gz 2024-01-24T15:33:20.800 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103612.log 2024-01-24T15:33:20.800 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55198.log 2024-01-24T15:33:20.800 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103612.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103612.log.gz 2024-01-24T15:33:20.800 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55198.log: gzip 88.3%-5 -- replaced with /var/log/ceph/ceph-client.admin.55198.log.gz--verbose 2024-01-24T15:33:20.800 INFO:teuthology.orchestra.run.smithi029.stderr:-- /var/log/ceph/ceph-client.admin.92123.log 2024-01-24T15:33:20.801 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48049.log 2024-01-24T15:33:20.801 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92123.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.37075.log 2024-01-24T15:33:20.801 INFO:teuthology.orchestra.run.smithi029.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92123.log.gz 2024-01-24T15:33:20.802 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48049.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48049.log.gz 2024-01-24T15:33:20.802 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.tmp-client.admin.37075.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.37075.log.gz 2024-01-24T15:33:20.802 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-01-24T15:33:20.805 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102903.log 2024-01-24T15:33:20.806 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102903.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102903.log.gz 2024-01-24T15:33:20.807 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87746.log 2024-01-24T15:33:20.808 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.87746.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87746.log.gz 2024-01-24T15:33:20.808 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69523.log 2024-01-24T15:33:20.809 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69523.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69523.log.gz 2024-01-24T15:33:20.809 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59036.log 2024-01-24T15:33:20.810 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59036.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59036.log.gz 2024-01-24T15:33:20.810 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65058.log 2024-01-24T15:33:20.811 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.65058.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65058.log.gz 2024-01-24T15:33:20.811 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71119.log 2024-01-24T15:33:20.812 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71119.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71119.log.gz 2024-01-24T15:33:20.812 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57371.log 2024-01-24T15:33:20.813 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57371.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57371.log.gz 2024-01-24T15:33:20.813 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103704.log 2024-01-24T15:33:20.814 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103704.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103704.log.gz 2024-01-24T15:33:20.814 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103566.log 2024-01-24T15:33:20.815 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103566.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103566.log.gz 2024-01-24T15:33:20.816 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103980.log 2024-01-24T15:33:20.817 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103980.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103980.log.gz 2024-01-24T15:33:20.817 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54461.log 2024-01-24T15:33:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54461.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54461.log.gz 2024-01-24T15:33:20.818 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81755.log 2024-01-24T15:33:20.819 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81755.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81755.log.gz 2024-01-24T15:33:20.819 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101204.log 2024-01-24T15:33:20.820 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101204.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.101204.log.gz 2024-01-24T15:33:20.820 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81709.log 2024-01-24T15:33:20.821 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81709.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81709.log.gz 2024-01-24T15:33:20.821 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50837.log 2024-01-24T15:33:20.822 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50837.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50837.log.gz 2024-01-24T15:33:20.822 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80830.log 2024-01-24T15:33:20.823 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80830.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80830.log.gz 2024-01-24T15:33:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43823.log 2024-01-24T15:33:20.824 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.43823.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43823.log.gz 2024-01-24T15:33:20.825 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77403.log 2024-01-24T15:33:20.825 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71947.log 2024-01-24T15:33:20.826 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77403.log: /var/log/ceph/ceph-client.admin.71947.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77403.log.gz 2024-01-24T15:33:20.826 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71947.log.gz 2024-01-24T15:33:20.826 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94945.log 2024-01-24T15:33:20.827 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.94945.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94945.log.gz 2024-01-24T15:33:20.828 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72856.log 2024-01-24T15:33:20.829 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72856.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72856.log.gz 2024-01-24T15:33:20.829 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91669.log 2024-01-24T15:33:20.830 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91669.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91669.log.gz 2024-01-24T15:33:20.830 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58568.log 2024-01-24T15:33:20.831 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58568.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58568.log.gz 2024-01-24T15:33:20.831 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73980.log 2024-01-24T15:33:20.832 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73980.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73980.log.gz 2024-01-24T15:33:20.832 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58338.log 2024-01-24T15:33:20.833 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49866.log 2024-01-24T15:33:20.834 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58338.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58338.log.gz 2024-01-24T15:33:20.834 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49866.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49866.log.gz 2024-01-24T15:33:20.834 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78447.log 2024-01-24T15:33:20.835 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78447.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78447.log.gz 2024-01-24T15:33:20.835 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70723.log 2024-01-24T15:33:20.836 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.70723.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70723.log.gz 2024-01-24T15:33:20.836 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49176.log 2024-01-24T15:33:20.837 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91301.log 2024-01-24T15:33:20.838 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49176.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49176.log.gz 2024-01-24T15:33:20.838 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91301.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91301.log.gz 2024-01-24T15:33:20.838 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94347.log 2024-01-24T15:33:20.839 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.94347.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94347.log.gz 2024-01-24T15:33:20.839 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87843.log 2024-01-24T15:33:20.840 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.87843.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.87843.log.gz 2024-01-24T15:33:20.840 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72764.log 2024-01-24T15:33:20.841 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72764.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72764.log.gz 2024-01-24T15:33:20.841 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50421.log 2024-01-24T15:33:20.842 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50421.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50421.log.gz 2024-01-24T15:33:20.842 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59300.log 2024-01-24T15:33:20.844 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59300.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59300.log.gz 2024-01-24T15:33:20.844 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89542.log 2024-01-24T15:33:20.845 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89542.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89542.log.gz 2024-01-24T15:33:20.845 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103888.log 2024-01-24T15:33:20.845 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54166.log 2024-01-24T15:33:20.846 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103888.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103888.log.gz 2024-01-24T15:33:20.847 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54166.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54166.log.gz 2024-01-24T15:33:20.847 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88432.log 2024-01-24T15:33:20.848 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88432.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88432.log.gz 2024-01-24T15:33:20.848 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68473.log 2024-01-24T15:33:20.849 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62497.log 2024-01-24T15:33:20.850 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68473.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68473.log.gz 2024-01-24T15:33:20.850 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62497.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62497.log.gz 2024-01-24T15:33:20.850 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81905.log 2024-01-24T15:33:20.851 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81905.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81905.log.gz 2024-01-24T15:33:20.851 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73679.log 2024-01-24T15:33:20.852 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77032.log 2024-01-24T15:33:20.853 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73679.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73679.log.gz 2024-01-24T15:33:20.853 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77032.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77032.log.gz 2024-01-24T15:33:20.853 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71517.log 2024-01-24T15:33:20.854 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71517.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71517.log.gz 2024-01-24T15:33:20.854 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55878.log 2024-01-24T15:33:20.855 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-01-24T15:33:20.861 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55878.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55878.log.gz 2024-01-24T15:33:20.862 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50791.log 2024-01-24T15:33:20.863 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50791.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50791.log.gz 2024-01-24T15:33:20.863 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88955.log 2024-01-24T15:33:20.882 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88955.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88955.log.gz 2024-01-24T15:33:20.882 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53184.log 2024-01-24T15:33:20.884 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53184.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53184.log.gz 2024-01-24T15:33:20.884 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103336.log 2024-01-24T15:33:20.885 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103336.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103336.log.gz 2024-01-24T15:33:20.885 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52610.log 2024-01-24T15:33:20.886 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52610.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52610.log.gz 2024-01-24T15:33:20.886 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100974.log 2024-01-24T15:33:20.887 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100974.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100974.log.gz 2024-01-24T15:33:20.888 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78539.log 2024-01-24T15:33:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78539.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78539.log.gz 2024-01-24T15:33:20.889 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93072.log 2024-01-24T15:33:20.890 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.93072.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93072.log.gz 2024-01-24T15:33:20.890 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52999.log 2024-01-24T15:33:20.891 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52999.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52999.log.gz 2024-01-24T15:33:20.891 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75110.log 2024-01-24T15:33:20.893 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75110.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75110.log.gz 2024-01-24T15:33:20.893 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88725.log 2024-01-24T15:33:20.894 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88725.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88725.log.gz 2024-01-24T15:33:20.894 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88155.log 2024-01-24T15:33:20.895 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88155.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88155.log.gz 2024-01-24T15:33:20.895 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72256.log 2024-01-24T15:33:20.896 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72256.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72256.log.gz 2024-01-24T15:33:20.897 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88294.log 2024-01-24T15:33:20.898 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88294.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88294.log.gz 2024-01-24T15:33:20.898 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65330.log 2024-01-24T15:33:20.899 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.65330.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65330.log.gz 2024-01-24T15:33:20.899 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62937.log 2024-01-24T15:33:20.900 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62937.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62937.log.gz 2024-01-24T15:33:20.901 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92029.log 2024-01-24T15:33:20.902 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92029.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92029.log.gz 2024-01-24T15:33:20.902 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78401.log 2024-01-24T15:33:20.903 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78401.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78401.log.gz 2024-01-24T15:33:20.903 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47471.log 2024-01-24T15:33:20.904 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47471.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47471.log.gz 2024-01-24T15:33:20.904 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55556.log 2024-01-24T15:33:20.906 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55556.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55556.log.gz 2024-01-24T15:33:20.906 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92695.log 2024-01-24T15:33:20.908 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92695.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.92695.log.gz 2024-01-24T15:33:20.908 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68308.log 2024-01-24T15:33:20.909 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68308.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68308.log.gz 2024-01-24T15:33:20.909 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103108.log 2024-01-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103108.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103108.log.gz 2024-01-24T15:33:20.910 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62190.log 2024-01-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62190.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62190.log.gz 2024-01-24T15:33:20.912 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71563.log 2024-01-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71563.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71563.log.gz 2024-01-24T15:33:20.913 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89588.log 2024-01-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89588.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89588.log.gz 2024-01-24T15:33:20.914 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57693.log 2024-01-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57693.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57693.log.gz 2024-01-24T15:33:20.916 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96926.log 2024-01-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96926.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96926.log.gz 2024-01-24T15:33:20.917 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74656.log 2024-01-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74656.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74656.log.gz 2024-01-24T15:33:20.918 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82137.log 2024-01-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82137.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82137.log.gz 2024-01-24T15:33:20.920 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76682.log 2024-01-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76682.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76682.log.gz 2024-01-24T15:33:20.921 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.52421.log 2024-01-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.2.52421.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.52421.log.gz 2024-01-24T15:33:20.922 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102854.log 2024-01-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102854.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102854.log.gz 2024-01-24T15:33:20.924 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55508.log 2024-01-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55508.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55508.log.gz 2024-01-24T15:33:20.933 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49314.log 2024-01-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49314.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49314.log.gz 2024-01-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73930.log 2024-01-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73930.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73930.log.gz 2024-01-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57601.log 2024-01-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57601.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57601.log.gz 2024-01-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71168.log 2024-01-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71168.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71168.log.gz 2024-01-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95098.log 2024-01-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95098.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95098.log.gz 2024-01-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78018.log 2024-01-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78018.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78018.log.gz 2024-01-24T15:33:20.934 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57555.log 2024-01-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57555.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57555.log.gz 2024-01-24T15:33:20.935 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55832.log 2024-01-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55832.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55832.log.gz 2024-01-24T15:33:20.936 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87592.log 2024-01-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.87592.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87592.log.gz 2024-01-24T15:33:20.937 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90105.log 2024-01-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90105.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90105.log.gz 2024-01-24T15:33:20.939 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51205.log 2024-01-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51205.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51205.log.gz 2024-01-24T15:33:20.940 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60045.log 2024-01-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60045.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60045.log.gz 2024-01-24T15:33:20.941 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95697.log 2024-01-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95697.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95697.log.gz 2024-01-24T15:33:20.943 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104210.log 2024-01-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.104210.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104210.log.gz 2024-01-24T15:33:20.944 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95296.log 2024-01-24T15:33:20.945 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95296.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95296.log.gz 2024-01-24T15:33:20.946 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53368.log 2024-01-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53368.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53368.log.gz 2024-01-24T15:33:20.947 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50375.log 2024-01-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50375.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50375.log.gz 2024-01-24T15:33:20.948 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73279.log 2024-01-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73279.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73279.log.gz 2024-01-24T15:33:20.949 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90739.log 2024-01-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90739.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90739.log.gz 2024-01-24T15:33:20.951 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71333.log 2024-01-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71333.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71333.log.gz 2024-01-24T15:33:20.952 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52472.log 2024-01-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52472.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52472.log.gz 2024-01-24T15:33:20.953 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89496.log 2024-01-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89496.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89496.log.gz 2024-01-24T15:33:20.954 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73492.log 2024-01-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73492.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73492.log.gz 2024-01-24T15:33:20.956 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75954.log 2024-01-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.75954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75954.log.gz 2024-01-24T15:33:20.957 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-24T15:33:20.964 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48095.log 2024-01-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48095.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48095.log.gz 2024-01-24T15:33:20.965 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91089.log 2024-01-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91089.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91089.log.gz 2024-01-24T15:33:20.967 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92279.log 2024-01-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92279.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92279.log.gz 2024-01-24T15:33:20.968 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104370.log 2024-01-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.104370.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.104370.log.gz 2024-01-24T15:33:20.969 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77080.log 2024-01-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77080.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77080.log.gz 2024-01-24T15:33:20.971 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102559.log 2024-01-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102559.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102559.log.gz 2024-01-24T15:33:20.972 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64681.log 2024-01-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64681.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.64681.log.gz 2024-01-24T15:33:20.974 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89634.log 2024-01-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89634.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89634.log.gz 2024-01-24T15:33:20.975 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96405.log 2024-01-24T15:33:20.976 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96405.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.96405.log.gz 2024-01-24T15:33:20.977 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51991.log 2024-01-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51991.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51991.log.gz 2024-01-24T15:33:20.978 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56016.log 2024-01-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56016.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56016.log.gz 2024-01-24T15:33:20.979 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95651.log 2024-01-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95651.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95651.log.gz 2024-01-24T15:33:20.980 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74242.log 2024-01-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74242.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74242.log.gz 2024-01-24T15:33:20.982 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54414.log 2024-01-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54414.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54414.log.gz 2024-01-24T15:33:20.983 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54922.log 2024-01-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54922.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54922.log.gz 2024-01-24T15:33:20.984 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53982.log 2024-01-24T15:33:20.985 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53982.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53982.log.gz 2024-01-24T15:33:20.986 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89680.log 2024-01-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89680.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89680.log.gz 2024-01-24T15:33:20.987 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62888.log 2024-01-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62888.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62888.log.gz 2024-01-24T15:33:20.988 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81341.log 2024-01-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81341.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81341.log.gz 2024-01-24T15:33:20.989 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75158.log 2024-01-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75158.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75158.log.gz 2024-01-24T15:33:20.991 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54258.log 2024-01-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54258.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54258.log.gz 2024-01-24T15:33:20.992 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77130.log 2024-01-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77130.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77130.log.gz 2024-01-24T15:33:20.993 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56342.log 2024-01-24T15:33:20.994 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.56342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56342.log.gz 2024-01-24T15:33:20.994 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69684.log 2024-01-24T15:33:20.996 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69684.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69684.log.gz 2024-01-24T15:33:20.996 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64405.log 2024-01-24T15:33:20.997 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64405.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64405.log.gz 2024-01-24T15:33:20.997 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91485.log 2024-01-24T15:33:20.998 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91485.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91485.log.gz 2024-01-24T15:33:20.998 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61577.log 2024-01-24T15:33:21.000 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.61577.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61577.log.gz 2024-01-24T15:33:21.000 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82229.log 2024-01-24T15:33:21.001 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82229.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82229.log.gz 2024-01-24T15:33:21.001 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56903.log 2024-01-24T15:33:21.002 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56903.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56903.log.gz 2024-01-24T15:33:21.002 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52840.log 2024-01-24T15:33:21.006 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.52840.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52840.log.gz 2024-01-24T15:33:21.007 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89312.log 2024-01-24T15:33:21.019 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89312.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89312.log.gz 2024-01-24T15:33:21.019 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82991.log 2024-01-24T15:33:21.020 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82991.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82991.log.gz 2024-01-24T15:33:21.020 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74832.log 2024-01-24T15:33:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74832.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74832.log.gz 2024-01-24T15:33:21.021 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64096.log 2024-01-24T15:33:21.024 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64096.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64096.log.gz 2024-01-24T15:33:21.024 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91891.log 2024-01-24T15:33:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91891.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91891.log.gz 2024-01-24T15:33:21.026 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48141.log 2024-01-24T15:33:21.027 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48141.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48141.log.gz 2024-01-24T15:33:21.027 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87990.log 2024-01-24T15:33:21.028 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.87990.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87990.log.gz 2024-01-24T15:33:21.028 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101905.log 2024-01-24T15:33:21.030 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101905.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101905.log.gz 2024-01-24T15:33:21.030 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77244.log 2024-01-24T15:33:21.031 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77244.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77244.log.gz 2024-01-24T15:33:21.031 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53322.log 2024-01-24T15:33:21.032 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53322.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53322.log.gz 2024-01-24T15:33:21.032 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81803.log 2024-01-24T15:33:21.033 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81803.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81803.log.gz 2024-01-24T15:33:21.034 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97509.log 2024-01-24T15:33:21.035 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.97509.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97509.log.gz 2024-01-24T15:33:21.035 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69592.log 2024-01-24T15:33:21.036 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69592.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69592.log.gz 2024-01-24T15:33:21.036 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96834.log 2024-01-24T15:33:21.038 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96834.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96834.log.gz 2024-01-24T15:33:21.038 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55740.log 2024-01-24T15:33:21.039 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55740.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55740.log.gz 2024-01-24T15:33:21.039 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94994.log 2024-01-24T15:33:21.041 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.94994.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.94994.log.gz 2024-01-24T15:33:21.041 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104460.log 2024-01-24T15:33:21.042 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.104460.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104460.log.gz 2024-01-24T15:33:21.042 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64048.log 2024-01-24T15:33:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64048.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64048.log.gz 2024-01-24T15:33:21.043 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90233.log 2024-01-24T15:33:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90233.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90233.log.gz 2024-01-24T15:33:21.044 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73233.log 2024-01-24T15:33:21.045 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73233.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73233.log.gz 2024-01-24T15:33:21.046 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92603.log 2024-01-24T15:33:21.047 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92603.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92603.log.gz 2024-01-24T15:33:21.047 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81617.log 2024-01-24T15:33:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81617.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81617.log.gz 2024-01-24T15:33:21.048 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63480.log 2024-01-24T15:33:21.050 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63480.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63480.log.gz 2024-01-24T15:33:21.050 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47425.log 2024-01-24T15:33:21.051 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47425.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47425.log.gz 2024-01-24T15:33:21.051 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49682.log 2024-01-24T15:33:21.053 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49682.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49682.log.gz 2024-01-24T15:33:21.053 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72718.log 2024-01-24T15:33:21.054 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72718.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.72718.log.gz 2024-01-24T15:33:21.054 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82321.log 2024-01-24T15:33:21.055 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82321.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82321.log.gz 2024-01-24T15:33:21.055 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78679.log 2024-01-24T15:33:21.056 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78679.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78679.log.gz 2024-01-24T15:33:21.057 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72396.log 2024-01-24T15:33:21.058 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72396.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72396.log.gz 2024-01-24T15:33:21.058 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100738.log 2024-01-24T15:33:21.059 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100738.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100738.log.gz 2024-01-24T15:33:21.059 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96265.log 2024-01-24T15:33:21.060 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96265.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96265.log.gz 2024-01-24T15:33:21.060 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47807.log 2024-01-24T15:33:21.061 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47807.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47807.log.gz 2024-01-24T15:33:21.062 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89051.log 2024-01-24T15:33:21.063 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89051.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89051.log.gz 2024-01-24T15:33:21.063 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56705.log 2024-01-24T15:33:21.064 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56705.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56705.log.gz 2024-01-24T15:33:21.064 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89358.log 2024-01-24T15:33:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89358.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89358.log.gz 2024-01-24T15:33:21.065 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96311.log 2024-01-24T15:33:21.067 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96311.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96311.log.gz 2024-01-24T15:33:21.067 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62636.log 2024-01-24T15:33:21.068 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62636.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62636.log.gz 2024-01-24T15:33:21.069 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77290.log 2024-01-24T15:33:21.070 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77290.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77290.log.gz 2024-01-24T15:33:21.070 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48922.log 2024-01-24T15:33:21.073 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74473.log 2024-01-24T15:33:21.074 INFO:teuthology.orchestra.run.smithi029.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.48922.log.gz 2024-01-24T15:33:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74473.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74473.log.gz 2024-01-24T15:33:21.075 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82744.log 2024-01-24T15:33:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82744.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82744.log.gz 2024-01-24T15:33:21.076 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75582.log 2024-01-24T15:33:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75582.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75582.log.gz 2024-01-24T15:33:21.077 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77357.log 2024-01-24T15:33:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77357.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77357.log.gz 2024-01-24T15:33:21.079 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50883.log 2024-01-24T15:33:21.080 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50883.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50883.log.gz 2024-01-24T15:33:21.080 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78067.log 2024-01-24T15:33:21.081 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78067.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.78067.log.gz 2024-01-24T15:33:21.081 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102511.log 2024-01-24T15:33:21.082 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102511.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102511.log.gz 2024-01-24T15:33:21.083 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47909.log 2024-01-24T15:33:21.084 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47909.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47909.log.gz 2024-01-24T15:33:21.084 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68750.log 2024-01-24T15:33:21.085 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68750.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68750.log.gz 2024-01-24T15:33:21.085 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95835.log 2024-01-24T15:33:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95835.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95835.log.gz 2024-01-24T15:33:21.086 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51113.log 2024-01-24T15:33:21.088 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51113.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51113.log.gz 2024-01-24T15:33:21.088 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100880.log 2024-01-24T15:33:21.089 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100880.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100880.log.gz 2024-01-24T15:33:21.089 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82698.log 2024-01-24T15:33:21.090 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82698.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82698.log.gz 2024-01-24T15:33:21.090 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57509.log 2024-01-24T15:33:21.091 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57509.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.57509.log.gz 2024-01-24T15:33:21.092 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68796.log 2024-01-24T15:33:21.093 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68796.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68796.log.gz 2024-01-24T15:33:21.093 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78493.log 2024-01-24T15:33:21.094 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78493.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78493.log.gz 2024-01-24T15:33:21.094 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80488.log 2024-01-24T15:33:21.095 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80488.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.80488.log.gz 2024-01-24T15:33:21.096 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87696.log 2024-01-24T15:33:21.097 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.87696.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.87696.log.gz 2024-01-24T15:33:21.097 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91937.log 2024-01-24T15:33:21.098 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91937.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91937.log.gz 2024-01-24T15:33:21.098 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63864.log 2024-01-24T15:33:21.100 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63864.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63864.log.gz 2024-01-24T15:33:21.100 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72039.log 2024-01-24T15:33:21.101 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72039.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72039.log.gz 2024-01-24T15:33:21.101 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77834.log 2024-01-24T15:33:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77834.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77834.log.gz 2024-01-24T15:33:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99643.log 2024-01-24T15:33:21.103 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99643.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99643.log.gz 2024-01-24T15:33:21.104 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71286.log 2024-01-24T15:33:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71286.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71286.log.gz 2024-01-24T15:33:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69235.log 2024-01-24T15:33:21.106 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69235.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69235.log.gz 2024-01-24T15:33:21.106 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71679.log 2024-01-24T15:33:21.108 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71679.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71679.log.gz 2024-01-24T15:33:21.108 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71901.log 2024-01-24T15:33:21.109 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71901.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71901.log.gz 2024-01-24T15:33:21.109 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54507.log 2024-01-24T15:33:21.110 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54507.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54507.log.gz 2024-01-24T15:33:21.110 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103428.log 2024-01-24T15:33:21.111 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103428.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103428.log.gz 2024-01-24T15:33:21.112 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94401.log 2024-01-24T15:33:21.113 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.94401.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94401.log.gz 2024-01-24T15:33:21.113 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57035.log 2024-01-24T15:33:21.114 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57035.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.57035.log.gz 2024-01-24T15:33:21.114 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47159.log 2024-01-24T15:33:21.115 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47159.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47159.log.gz 2024-01-24T15:33:21.115 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92511.log 2024-01-24T15:33:21.116 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92511.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.92511.log.gz 2024-01-24T15:33:21.117 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53890.log 2024-01-24T15:33:21.118 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53890.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53890.log.gz 2024-01-24T15:33:21.118 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99098.log 2024-01-24T15:33:21.119 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99098.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99098.log.gz 2024-01-24T15:33:21.119 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74746.log 2024-01-24T15:33:21.120 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74746.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74746.log.gz 2024-01-24T15:33:21.120 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69097.log 2024-01-24T15:33:21.122 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69097.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69097.log.gz 2024-01-24T15:33:21.122 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88863.log 2024-01-24T15:33:21.123 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88863.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88863.log.gz 2024-01-24T15:33:21.123 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54350.log 2024-01-24T15:33:21.124 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54350.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54350.log.gz 2024-01-24T15:33:21.124 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93142.log 2024-01-24T15:33:21.126 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.93142.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93142.log.gz 2024-01-24T15:33:21.126 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101992.log 2024-01-24T15:33:21.127 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101992.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101992.log.gz 2024-01-24T15:33:21.127 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64589.log 2024-01-24T15:33:21.128 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64589.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64589.log.gz 2024-01-24T15:33:21.128 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96650.log 2024-01-24T15:33:21.129 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96650.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96650.log.gz 2024-01-24T15:33:21.130 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81999.log 2024-01-24T15:33:21.131 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81999.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81999.log.gz 2024-01-24T15:33:21.131 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75674.log 2024-01-24T15:33:21.133 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75674.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.75674.log.gz 2024-01-24T15:33:21.133 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68211.log 2024-01-24T15:33:21.134 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68211.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68211.log.gz 2024-01-24T15:33:21.134 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103016.log 2024-01-24T15:33:21.135 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103016.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103016.log.gz 2024-01-24T15:33:21.135 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101836.log 2024-01-24T15:33:21.137 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101836.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101836.log.gz 2024-01-24T15:33:21.137 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95440.log 2024-01-24T15:33:21.138 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95440.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.95440.log.gz 2024-01-24T15:33:21.139 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70771.log 2024-01-24T15:33:21.140 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.70771.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70771.log.gz 2024-01-24T15:33:21.140 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77541.log 2024-01-24T15:33:21.142 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77541.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77541.log.gz 2024-01-24T15:33:21.142 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79017.log 2024-01-24T15:33:21.143 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72442.log 2024-01-24T15:33:21.143 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.79017.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79017.log.gz 2024-01-24T15:33:21.150 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72442.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72442.log.gz 2024-01-24T15:33:21.150 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81086.log 2024-01-24T15:33:21.151 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81086.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81086.log.gz 2024-01-24T15:33:21.151 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89105.log 2024-01-24T15:33:21.153 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89105.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89105.log.gz 2024-01-24T15:33:21.153 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49820.log 2024-01-24T15:33:21.154 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49820.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49820.log.gz 2024-01-24T15:33:21.154 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95042.log 2024-01-24T15:33:21.155 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95042.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95042.log.gz 2024-01-24T15:33:21.155 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101066.log 2024-01-24T15:33:21.157 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101066.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.101066.log.gz 2024-01-24T15:33:21.157 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62840.log 2024-01-24T15:33:21.158 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62840.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62840.log.gz 2024-01-24T15:33:21.158 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100546.log 2024-01-24T15:33:21.159 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100546.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100546.log.gz 2024-01-24T15:33:21.159 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50975.log 2024-01-24T15:33:21.160 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50975.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50975.log.gz 2024-01-24T15:33:21.160 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59234.log 2024-01-24T15:33:21.162 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59234.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.59234.log.gz 2024-01-24T15:33:21.162 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104118.log 2024-01-24T15:33:21.163 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.104118.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104118.log.gz 2024-01-24T15:33:21.163 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73881.log 2024-01-24T15:33:21.164 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73881.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73881.log.gz 2024-01-24T15:33:21.165 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100186.log 2024-01-24T15:33:21.166 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100186.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100186.log.gz 2024-01-24T15:33:21.166 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67910.log 2024-01-24T15:33:21.167 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.67910.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67910.log.gz 2024-01-24T15:33:21.167 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50096.log 2024-01-24T15:33:21.169 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50096.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50096.log.gz 2024-01-24T15:33:21.169 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63526.log 2024-01-24T15:33:21.170 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63526.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63526.log.gz 2024-01-24T15:33:21.170 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96602.log 2024-01-24T15:33:21.171 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96602.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96602.log.gz 2024-01-24T15:33:21.171 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74427.log 2024-01-24T15:33:21.173 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74427.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74427.log.gz 2024-01-24T15:33:21.173 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92557.log 2024-01-24T15:33:21.174 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92557.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92557.log.gz 2024-01-24T15:33:21.174 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73630.log 2024-01-24T15:33:21.175 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73630.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73630.log.gz 2024-01-24T15:33:21.175 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103154.log 2024-01-24T15:33:21.177 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103154.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103154.log.gz 2024-01-24T15:33:21.177 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50004.log 2024-01-24T15:33:21.178 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50004.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50004.log.gz 2024-01-24T15:33:21.178 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54968.log 2024-01-24T15:33:21.179 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54968.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54968.log.gz 2024-01-24T15:33:21.179 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55152.log 2024-01-24T15:33:21.181 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55152.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55152.log.gz 2024-01-24T15:33:21.181 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76983.log 2024-01-24T15:33:21.182 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76983.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76983.log.gz 2024-01-24T15:33:21.182 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70361.log 2024-01-24T15:33:21.183 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.70361.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70361.log.gz 2024-01-24T15:33:21.183 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93318.log 2024-01-24T15:33:21.184 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.93318.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93318.log.gz 2024-01-24T15:33:21.185 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96041.log 2024-01-24T15:33:21.186 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96041.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96041.log.gz 2024-01-24T15:33:21.186 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75258.log 2024-01-24T15:33:21.187 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75258.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75258.log.gz 2024-01-24T15:33:21.187 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62734.log 2024-01-24T15:33:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.62734.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62734.log.gz 2024-01-24T15:33:21.188 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53045.log 2024-01-24T15:33:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53045.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53045.log.gz 2024-01-24T15:33:21.190 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60963.log 2024-01-24T15:33:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.60963.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60963.log.gz 2024-01-24T15:33:21.191 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101436.log 2024-01-24T15:33:21.192 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101436.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101436.log.gz 2024-01-24T15:33:21.192 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99693.log 2024-01-24T15:33:21.193 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99693.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99693.log.gz 2024-01-24T15:33:21.193 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72810.log 2024-01-24T15:33:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72810.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72810.log.gz 2024-01-24T15:33:21.195 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57417.log 2024-01-24T15:33:21.196 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57417.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57417.log.gz 2024-01-24T15:33:21.196 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49728.log 2024-01-24T15:33:21.197 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49728.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49728.log.gz 2024-01-24T15:33:21.198 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73727.log 2024-01-24T15:33:21.199 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73727.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73727.log.gz 2024-01-24T15:33:21.199 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58292.log 2024-01-24T15:33:21.200 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58292.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58292.log.gz 2024-01-24T15:33:21.200 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96788.log 2024-01-24T15:33:21.202 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96788.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.96788.log.gz 2024-01-24T15:33:21.202 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73182.log 2024-01-24T15:33:21.203 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.0.73182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73182.log.gz 2024-01-24T15:33:21.203 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58384.log 2024-01-24T15:33:21.204 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58384.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58384.log.gz 2024-01-24T15:33:21.204 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91347.log 2024-01-24T15:33:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91347.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91347.log.gz 2024-01-24T15:33:21.206 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80384.log 2024-01-24T15:33:21.207 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80384.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80384.log.gz 2024-01-24T15:33:21.207 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51067.log 2024-01-24T15:33:21.208 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51067.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51067.log.gz 2024-01-24T15:33:21.209 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72626.log 2024-01-24T15:33:21.210 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72626.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72626.log.gz 2024-01-24T15:33:21.210 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53460.log 2024-01-24T15:33:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53460.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53460.log.gz 2024-01-24T15:33:21.211 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90438.log 2024-01-24T15:33:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90438.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.90438.log.gz 2024-01-24T15:33:21.212 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101296.log 2024-01-24T15:33:21.213 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101296.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101296.log.gz 2024-01-24T15:33:21.214 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89266.log 2024-01-24T15:33:21.215 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89266.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89266.log.gz 2024-01-24T15:33:21.215 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64451.log 2024-01-24T15:33:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64451.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64451.log.gz 2024-01-24T15:33:21.217 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70291.log 2024-01-24T15:33:21.218 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.70291.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70291.log.gz 2024-01-24T15:33:21.218 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92075.log 2024-01-24T15:33:21.219 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92075.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.92075.log.gz 2024-01-24T15:33:21.219 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81663.log 2024-01-24T15:33:21.220 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81663.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81663.log.gz 2024-01-24T15:33:21.220 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73833.log 2024-01-24T15:33:21.222 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.73833.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73833.log.gz 2024-01-24T15:33:21.222 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80994.log 2024-01-24T15:33:21.223 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80994.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80994.log.gz 2024-01-24T15:33:21.223 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55786.log 2024-01-24T15:33:21.224 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55786.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55786.log.gz 2024-01-24T15:33:21.224 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74335.log 2024-01-24T15:33:21.225 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74335.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74335.log.gz 2024-01-24T15:33:21.226 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94807.log 2024-01-24T15:33:21.227 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.94807.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94807.log.gz 2024-01-24T15:33:21.227 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95344.log 2024-01-24T15:33:21.228 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95344.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95344.log.gz 2024-01-24T15:33:21.228 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57279.log 2024-01-24T15:33:21.229 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57279.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.57279.log.gz 2024-01-24T15:33:21.229 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72672.log 2024-01-24T15:33:21.231 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72672.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72672.log.gz 2024-01-24T15:33:21.231 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102308.log 2024-01-24T15:33:21.232 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102308.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102308.log.gz 2024-01-24T15:33:21.232 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56110.log 2024-01-24T15:33:21.234 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56110.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56110.log.gz 2024-01-24T15:33:21.234 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71993.log 2024-01-24T15:33:21.235 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71993.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71993.log.gz 2024-01-24T15:33:21.235 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102949.log 2024-01-24T15:33:21.236 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102949.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102949.log.gz 2024-01-24T15:33:21.236 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92419.log 2024-01-24T15:33:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92419.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92419.log.gz 2024-01-24T15:33:21.237 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70974.log 2024-01-24T15:33:21.239 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.70974.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70974.log.gz 2024-01-24T15:33:21.239 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87893.log 2024-01-24T15:33:21.240 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.87893.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87893.log.gz 2024-01-24T15:33:21.240 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78971.log 2024-01-24T15:33:21.241 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78971.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78971.log.gz 2024-01-24T15:33:21.241 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54738.log 2024-01-24T15:33:21.242 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54738.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54738.log.gz 2024-01-24T15:33:21.243 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48279.log 2024-01-24T15:33:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48279.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.48279.log.gz 2024-01-24T15:33:21.244 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75304.log 2024-01-24T15:33:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75304.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75304.log.gz 2024-01-24T15:33:21.246 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65012.log 2024-01-24T15:33:21.247 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.65012.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65012.log.gz 2024-01-24T15:33:21.247 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75722.log 2024-01-24T15:33:21.248 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75722.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75722.log.gz 2024-01-24T15:33:21.248 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55292.log 2024-01-24T15:33:21.250 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55292.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55292.log.gz 2024-01-24T15:33:21.250 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89151.log 2024-01-24T15:33:21.251 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.89151.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89151.log.gz 2024-01-24T15:33:21.251 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68566.log 2024-01-24T15:33:21.252 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68566.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68566.log.gz 2024-01-24T15:33:21.252 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58522.log 2024-01-24T15:33:21.253 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58522.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58522.log.gz 2024-01-24T15:33:21.253 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50605.log 2024-01-24T15:33:21.255 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50605.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50605.log.gz 2024-01-24T15:33:21.255 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63084.log 2024-01-24T15:33:21.256 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63084.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63084.log.gz 2024-01-24T15:33:21.256 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55602.log 2024-01-24T15:33:21.257 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55602.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55602.log.gz 2024-01-24T15:33:21.257 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75212.log 2024-01-24T15:33:21.258 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75212.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.75212.log.gz 2024-01-24T15:33:21.258 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53844.log 2024-01-24T15:33:21.260 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53844.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53844.log.gz 2024-01-24T15:33:21.260 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88909.log 2024-01-24T15:33:21.261 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88909.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88909.log.gz 2024-01-24T15:33:21.261 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58706.log 2024-01-24T15:33:21.262 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58706.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58706.log.gz 2024-01-24T15:33:21.263 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92325.log 2024-01-24T15:33:21.264 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92325.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.92325.log.gz 2024-01-24T15:33:21.264 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70877.log 2024-01-24T15:33:21.265 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.70877.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.70877.log.gz 2024-01-24T15:33:21.265 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82091.log 2024-01-24T15:33:21.266 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.82091.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82091.log.gz 2024-01-24T15:33:21.267 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56599.log 2024-01-24T15:33:21.268 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56599.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56599.log.gz 2024-01-24T15:33:21.268 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57647.log 2024-01-24T15:33:21.269 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57647.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57647.log.gz 2024-01-24T15:33:21.270 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102038.log 2024-01-24T15:33:21.271 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102038.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102038.log.gz 2024-01-24T15:33:21.271 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68520.log 2024-01-24T15:33:21.272 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68520.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68520.log.gz 2024-01-24T15:33:21.272 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91983.log 2024-01-24T15:33:21.273 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91983.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.91983.log.gz 2024-01-24T15:33:21.273 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88088.log 2024-01-24T15:33:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.88088.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.88088.log.gz 2024-01-24T15:33:21.275 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91845.log 2024-01-24T15:33:21.276 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91845.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91845.log.gz 2024-01-24T15:33:21.276 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49774.log 2024-01-24T15:33:21.277 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49774.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49774.log.gz 2024-01-24T15:33:21.277 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64002.log 2024-01-24T15:33:21.278 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64002.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64002.log.gz 2024-01-24T15:33:21.279 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80197.log 2024-01-24T15:33:21.280 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80197.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80197.log.gz 2024-01-24T15:33:21.280 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64543.log 2024-01-24T15:33:21.281 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.64543.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64543.log.gz 2024-01-24T15:33:21.281 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99838.log 2024-01-24T15:33:21.282 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99838.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99838.log.gz 2024-01-24T15:33:21.282 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69189.log 2024-01-24T15:33:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69189.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69189.log.gz 2024-01-24T15:33:21.284 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57233.log 2024-01-24T15:33:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57233.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.57233.log.gz 2024-01-24T15:33:21.285 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92225.log 2024-01-24T15:33:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.92225.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92225.log.gz 2024-01-24T15:33:21.286 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50699.log 2024-01-24T15:33:21.287 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50699.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50699.log.gz 2024-01-24T15:33:21.288 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48187.log 2024-01-24T15:33:21.289 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48187.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48187.log.gz 2024-01-24T15:33:21.289 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80782.log 2024-01-24T15:33:21.290 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80782.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80782.log.gz 2024-01-24T15:33:21.290 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102357.log 2024-01-24T15:33:21.291 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102357.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.102357.log.gz 2024-01-24T15:33:21.291 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56969.log 2024-01-24T15:33:21.293 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56969.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56969.log.gz 2024-01-24T15:33:21.293 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99392.log 2024-01-24T15:33:21.294 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99392.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99392.log.gz 2024-01-24T15:33:21.294 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103520.log 2024-01-24T15:33:21.295 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103520.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103520.log.gz 2024-01-24T15:33:21.295 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47096.log 2024-01-24T15:33:21.297 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47096.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47096.log.gz 2024-01-24T15:33:21.297 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48233.log 2024-01-24T15:33:21.298 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48233.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48233.log.gz 2024-01-24T15:33:21.298 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47295.log 2024-01-24T15:33:21.299 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.47295.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47295.log.gz 2024-01-24T15:33:21.299 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-01-24T15:33:21.302 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69377.log 2024-01-24T15:33:21.315 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69377.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69377.log.gz 2024-01-24T15:33:21.315 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63132.log 2024-01-24T15:33:21.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63132.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63132.log.gz 2024-01-24T15:33:21.339 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71218.log 2024-01-24T15:33:21.359 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71218.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71218.log.gz 2024-01-24T15:33:21.360 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69431.log 2024-01-24T15:33:21.372 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53598.log 2024-01-24T15:33:21.372 INFO:teuthology.orchestra.run.smithi029.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69431.log.gz 2024-01-24T15:33:21.384 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99741.log 2024-01-24T15:33:21.393 INFO:teuthology.orchestra.run.smithi029.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53598.log.gz 2024-01-24T15:33:21.409 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99741.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99741.log.gz 2024-01-24T15:33:21.419 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102658.log 2024-01-24T15:33:21.421 INFO:teuthology.orchestra.run.smithi029.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-01-24T15:33:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.102658.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102658.log.gz 2024-01-24T15:33:21.423 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55106.log 2024-01-24T15:33:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55106.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55106.log.gz 2024-01-24T15:33:21.424 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78309.log 2024-01-24T15:33:21.426 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78309.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78309.log.gz 2024-01-24T15:33:21.426 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71024.log 2024-01-24T15:33:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.71024.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71024.log.gz 2024-01-24T15:33:21.427 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59739.log 2024-01-24T15:33:21.428 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59739.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59739.log.gz 2024-01-24T15:33:21.429 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93026.log 2024-01-24T15:33:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.93026.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93026.log.gz 2024-01-24T15:33:21.430 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100094.log 2024-01-24T15:33:21.431 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100094.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100094.log.gz 2024-01-24T15:33:21.431 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48003.log 2024-01-24T15:33:21.432 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48003.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48003.log.gz 2024-01-24T15:33:21.433 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96880.log 2024-01-24T15:33:21.434 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96880.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96880.log.gz 2024-01-24T15:33:21.434 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86162.log 2024-01-24T15:33:21.435 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.86162.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86162.log.gz 2024-01-24T15:33:21.435 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101020.log 2024-01-24T15:33:21.436 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101020.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101020.log.gz 2024-01-24T15:33:21.436 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76885.log 2024-01-24T15:33:21.438 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76885.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76885.log.gz 2024-01-24T15:33:21.438 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101388.log 2024-01-24T15:33:21.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.101388.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.101388.log.gz 2024-01-24T15:33:21.440 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48327.log 2024-01-24T15:33:21.441 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.48327.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48327.log.gz 2024-01-24T15:33:21.441 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72085.log 2024-01-24T15:33:21.442 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72085.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72085.log.gz 2024-01-24T15:33:21.442 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104164.log 2024-01-24T15:33:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.104164.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104164.log.gz 2024-01-24T15:33:21.443 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77702.log 2024-01-24T15:33:21.445 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77702.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77702.log.gz 2024-01-24T15:33:21.445 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55244.log 2024-01-24T15:33:21.446 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55244.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55244.log.gz 2024-01-24T15:33:21.446 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76788.log 2024-01-24T15:33:21.447 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76788.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76788.log.gz 2024-01-24T15:33:21.447 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50929.log 2024-01-24T15:33:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50929.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50929.log.gz 2024-01-24T15:33:21.449 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90304.log 2024-01-24T15:33:21.450 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90304.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90304.log.gz 2024-01-24T15:33:21.450 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69868.log 2024-01-24T15:33:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69868.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69868.log.gz 2024-01-24T15:33:21.451 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77197.log 2024-01-24T15:33:21.452 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77197.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77197.log.gz 2024-01-24T15:33:21.453 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56771.log 2024-01-24T15:33:21.454 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56771.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56771.log.gz 2024-01-24T15:33:21.454 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57463.log 2024-01-24T15:33:21.455 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57463.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57463.log.gz 2024-01-24T15:33:21.455 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56439.log 2024-01-24T15:33:21.456 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56439.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56439.log.gz 2024-01-24T15:33:21.456 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74125.log 2024-01-24T15:33:21.458 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74125.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74125.log.gz 2024-01-24T15:33:21.458 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59366.log 2024-01-24T15:33:21.459 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.59366.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.59366.log.gz 2024-01-24T15:33:21.459 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81571.log 2024-01-24T15:33:21.460 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81571.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81571.log.gz 2024-01-24T15:33:21.460 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56509.log 2024-01-24T15:33:21.461 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.56509.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56509.log.gz 2024-01-24T15:33:21.461 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69638.log 2024-01-24T15:33:21.463 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69638.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69638.log.gz 2024-01-24T15:33:21.463 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95197.log 2024-01-24T15:33:21.464 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95197.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.95197.log.gz 2024-01-24T15:33:21.464 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50050.log 2024-01-24T15:33:21.465 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50050.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50050.log.gz 2024-01-24T15:33:21.465 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75398.log 2024-01-24T15:33:21.467 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75398.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75398.log.gz 2024-01-24T15:33:21.467 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80432.log 2024-01-24T15:33:21.468 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80432.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80432.log.gz 2024-01-24T15:33:21.468 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103842.log 2024-01-24T15:33:21.469 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103842.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103842.log.gz 2024-01-24T15:33:21.469 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63182.log 2024-01-24T15:33:21.471 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63182.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63182.log.gz 2024-01-24T15:33:21.471 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99790.log 2024-01-24T15:33:21.472 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99790.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99790.log.gz 2024-01-24T15:33:21.472 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94281.log 2024-01-24T15:33:21.473 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.94281.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94281.log.gz 2024-01-24T15:33:21.473 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57167.log 2024-01-24T15:33:21.474 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.57167.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57167.log.gz 2024-01-24T15:33:21.475 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68161.log 2024-01-24T15:33:21.476 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.68161.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68161.log.gz 2024-01-24T15:33:21.476 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63818.log 2024-01-24T15:33:21.477 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63818.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63818.log.gz 2024-01-24T15:33:21.477 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49452.log 2024-01-24T15:33:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49452.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49452.log.gz 2024-01-24T15:33:21.478 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63572.log 2024-01-24T15:33:21.479 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63572.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63572.log.gz 2024-01-24T15:33:21.480 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58772.log 2024-01-24T15:33:21.481 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58772.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58772.log.gz 2024-01-24T15:33:21.481 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49222.log 2024-01-24T15:33:21.482 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49222.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49222.log.gz 2024-01-24T15:33:21.482 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95148.log 2024-01-24T15:33:21.484 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95148.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95148.log.gz 2024-01-24T15:33:21.484 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80091.log 2024-01-24T15:33:21.485 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80091.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80091.log.gz 2024-01-24T15:33:21.485 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100370.log 2024-01-24T15:33:21.487 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100370.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100370.log.gz 2024-01-24T15:33:21.487 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95391.log 2024-01-24T15:33:21.488 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95391.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95391.log.gz 2024-01-24T15:33:21.488 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96993.log 2024-01-24T15:33:21.490 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96993.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.96993.log.gz 2024-01-24T15:33:21.490 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63910.log 2024-01-24T15:33:21.491 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.63910.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63910.log.gz 2024-01-24T15:33:21.491 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96219.log 2024-01-24T15:33:21.493 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.96219.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96219.log.gz 2024-01-24T15:33:21.493 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75536.log 2024-01-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.75536.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75536.log.gz 2024-01-24T15:33:21.494 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65197.log 2024-01-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.65197.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.65197.log.gz 2024-01-24T15:33:21.495 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81224.log 2024-01-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.81224.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81224.log.gz 2024-01-24T15:33:21.497 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76005.log 2024-01-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.76005.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.76005.log.gz 2024-01-24T15:33:21.499 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70430.log 2024-01-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.70430.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70430.log.gz 2024-01-24T15:33:21.500 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99204.log 2024-01-24T15:33:21.501 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.99204.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99204.log.gz 2024-01-24T15:33:21.502 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53092.log 2024-01-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.53092.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53092.log.gz 2024-01-24T15:33:21.503 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51159.log 2024-01-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.51159.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51159.log.gz 2024-01-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49590.log 2024-01-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49590.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49590.log.gz 2024-01-24T15:33:21.506 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80586.log 2024-01-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.80586.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80586.log.gz 2024-01-24T15:33:21.507 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74878.log 2024-01-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74878.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74878.log.gz 2024-01-24T15:33:21.509 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86208.log 2024-01-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.86208.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86208.log.gz 2024-01-24T15:33:21.510 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55014.log 2024-01-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.55014.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55014.log.gz 2024-01-24T15:33:21.512 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90942.log 2024-01-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.90942.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90942.log.gz 2024-01-24T15:33:21.513 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65128.log 2024-01-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.65128.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65128.log.gz 2024-01-24T15:33:21.514 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77449.log 2024-01-24T15:33:21.515 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.77449.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77449.log.gz 2024-01-24T15:33:21.516 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87640.log 2024-01-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.87640.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.87640.log.gz 2024-01-24T15:33:21.517 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78585.log 2024-01-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.78585.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78585.log.gz 2024-01-24T15:33:21.518 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58476.log 2024-01-24T15:33:21.519 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.58476.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58476.log.gz 2024-01-24T15:33:21.520 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67723.log 2024-01-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.67723.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67723.log.gz 2024-01-24T15:33:21.521 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54876.log 2024-01-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.54876.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54876.log.gz 2024-01-24T15:33:21.522 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95789.log 2024-01-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.95789.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95789.log.gz 2024-01-24T15:33:21.523 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69914.log 2024-01-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.69914.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.69914.log.gz 2024-01-24T15:33:21.525 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104302.log 2024-01-24T15:33:21.526 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.104302.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104302.log.gz 2024-01-24T15:33:21.527 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50329.log 2024-01-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50329.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50329.log.gz 2024-01-24T15:33:21.528 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50237.log 2024-01-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.50237.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50237.log.gz 2024-01-24T15:33:21.529 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91439.log 2024-01-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.91439.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91439.log.gz 2024-01-24T15:33:21.530 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100832.log 2024-01-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.100832.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100832.log.gz 2024-01-24T15:33:21.532 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103796.log 2024-01-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.103796.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103796.log.gz 2024-01-24T15:33:21.533 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72580.log 2024-01-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.72580.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72580.log.gz 2024-01-24T15:33:21.534 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74028.log 2024-01-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.74028.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74028.log.gz 2024-01-24T15:33:21.535 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67958.log 2024-01-24T15:33:21.536 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.67958.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67958.log.gz 2024-01-24T15:33:21.537 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49636.log 2024-01-24T15:33:21.538 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.49636.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49636.log.gz 2024-01-24T15:33:21.648 INFO:teuthology.orchestra.run.smithi029.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-01-24T15:33:21.884 INFO:teuthology.orchestra.run.smithi029.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-01-24T15:33:22.333 INFO:teuthology.orchestra.run.smithi163.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-01-24T15:33:22.443 INFO:teuthology.orchestra.run.smithi163.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-01-24T15:33:22.624 INFO:teuthology.orchestra.run.smithi163.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-01-24T15:33:23.712 INFO:teuthology.orchestra.run.smithi163.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-01-24T15:33:24.153 INFO:teuthology.orchestra.run.smithi163.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-01-24T15:33:24.198 INFO:teuthology.orchestra.run.smithi029.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-01-24T15:33:24.215 INFO:teuthology.orchestra.run.smithi163.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-01-24T15:33:24.288 INFO:teuthology.orchestra.run.smithi029.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-01-24T15:33:24.371 INFO:teuthology.orchestra.run.smithi163.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-01-24T15:33:24.373 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-01-24T15:33:24.373 INFO:teuthology.orchestra.run.smithi163.stderr:real 0m3.713s 2024-01-24T15:33:24.373 INFO:teuthology.orchestra.run.smithi163.stderr:user 0m18.464s 2024-01-24T15:33:24.374 INFO:teuthology.orchestra.run.smithi163.stderr:sys 0m0.691s 2024-01-24T15:33:24.401 INFO:teuthology.orchestra.run.smithi029.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-01-24T15:33:24.868 INFO:teuthology.orchestra.run.smithi029.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-01-24T15:33:25.472 INFO:teuthology.orchestra.run.smithi029.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-01-24T15:33:25.476 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-01-24T15:33:25.476 INFO:teuthology.orchestra.run.smithi029.stderr:real 0m4.835s 2024-01-24T15:33:25.476 INFO:teuthology.orchestra.run.smithi029.stderr:user 0m21.113s 2024-01-24T15:33:25.476 INFO:teuthology.orchestra.run.smithi029.stderr:sys 0m1.277s 2024-01-24T15:33:25.476 INFO:tasks.ceph:Archiving logs... 2024-01-24T15:33:25.476 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993/remote/smithi029/log 2024-01-24T15:33:25.477 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-24T15:33:28.753 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993/remote/smithi163/log 2024-01-24T15:33:28.754 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-24T15:33:31.129 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-24T15:33:31.139 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-24T15:33:31.139 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-24T15:33:31.142 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-24T15:33:31.227 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-24T15:33:31.237 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-24T15:33:31.237 DEBUG:teuthology.orchestra.run.smithi029:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-24T15:33:31.240 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-24T15:33:31.257 INFO:teuthology.orchestra.run.smithi029.stderr:bash: ntpq: command not found 2024-01-24T15:33:31.259 INFO:teuthology.orchestra.run.smithi163.stderr:bash: ntpq: command not found 2024-01-24T15:33:31.263 INFO:teuthology.orchestra.run.smithi029.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-24T15:33:31.263 INFO:teuthology.orchestra.run.smithi029.stdout:=============================================================================== 2024-01-24T15:33:31.263 INFO:teuthology.orchestra.run.smithi029.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 46 +500us[ +479us] +/- 99ms 2024-01-24T15:33:31.263 INFO:teuthology.orchestra.run.smithi029.stdout:^* hv02.front.sepia.ceph.com 3 7 377 46 -1294us[-1314us] +/- 60ms 2024-01-24T15:33:31.263 INFO:teuthology.orchestra.run.smithi029.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +1335us[+1335us] +/- 81ms 2024-01-24T15:33:31.263 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-01-24T15:33:31.265 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-24T15:33:31.265 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-01-24T15:33:31.265 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 44 +567us[ +543us] +/- 99ms 2024-01-24T15:33:31.265 INFO:teuthology.orchestra.run.smithi163.stdout:^* hv02.front.sepia.ceph.com 3 7 377 42 -1244us[-1267us] +/- 60ms 2024-01-24T15:33:31.265 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +1404us[+1379us] +/- 81ms 2024-01-24T15:33:31.265 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-01-24T15:33:31.265 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-24T15:33:31.274 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-24T15:33:31.275 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-24T15:33:31.301 DEBUG:teuthology.orchestra.run.smithi029:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-24T15:33:31.412 DEBUG:teuthology.orchestra.run.smithi163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-24T15:33:31.488 DEBUG:teuthology.orchestra.run.smithi029:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-24T15:33:31.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T15:33:31.526 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-24T15:33:31.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T15:33:31.558 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-24T15:33:31.569 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-24T15:33:31.592 INFO:teuthology.task.internal:Duration was 6502.062039 seconds 2024-01-24T15:33:31.593 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-24T15:33:31.627 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-24T15:33:31.628 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-24T15:33:31.630 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-24T15:33:31.679 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-24T15:33:31.689 INFO:teuthology.orchestra.run.smithi029.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-24T15:33:32.004 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-24T15:33:32.004 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi029.front.sepia.ceph.com 2024-01-24T15:33:32.004 DEBUG:teuthology.orchestra.run.smithi029:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-24T15:33:32.032 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2024-01-24T15:33:32.032 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-24T15:33:32.054 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-24T15:33:32.054 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-24T15:33:32.074 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-24T15:33:32.140 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-24T15:33:32.140 DEBUG:teuthology.orchestra.run.smithi029:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-24T15:33:32.143 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-24T15:33:32.289 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-24T15:33:32.302 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-24T15:33:32.302 DEBUG:teuthology.orchestra.run.smithi029:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-24T15:33:32.333 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-24T15:33:32.360 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-24T15:33:32.370 DEBUG:teuthology.orchestra.run.smithi029:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-24T15:33:32.375 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-24T15:33:32.404 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = core 2024-01-24T15:33:32.428 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2024-01-24T15:33:32.442 DEBUG:teuthology.orchestra.run.smithi029:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-24T15:33:32.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T15:33:32.482 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-24T15:33:32.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T15:33:32.499 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-24T15:33:32.508 INFO:teuthology.task.internal:Transferring archived files... 2024-01-24T15:33:32.509 DEBUG:teuthology.misc:Transferring archived files from smithi029:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993/remote/smithi029 2024-01-24T15:33:32.509 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-24T15:33:32.573 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993/remote/smithi163 2024-01-24T15:33:32.574 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-24T15:33:32.612 INFO:teuthology.task.internal:Removing archive directory... 2024-01-24T15:33:32.612 DEBUG:teuthology.orchestra.run.smithi029:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-24T15:33:32.615 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-24T15:33:32.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-24T15:33:32.675 INFO:teuthology.task.internal:Not uploading archives. 2024-01-24T15:33:32.675 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-24T15:33:32.694 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-24T15:33:32.694 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-24T15:33:32.697 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-24T15:33:32.716 INFO:teuthology.orchestra.run.smithi029.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 24 15:33 /home/ubuntu/cephtest 2024-01-24T15:33:32.719 INFO:teuthology.orchestra.run.smithi163.stdout: 262183 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 24 15:33 /home/ubuntu/cephtest 2024-01-24T15:33:32.721 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-24T15:33:32.732 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-24T15:33:32.795 INFO:teuthology.nuke:Checking targets against current locks 2024-01-24T15:33:32.819 DEBUG:teuthology.nuke:shortname: smithi029 2024-01-24T15:33:32.820 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-24T15:33:32.846 DEBUG:teuthology.nuke:shortname: smithi163 2024-01-24T15:33:32.846 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-24T15:33:32.849 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-24 13:39:10.629982', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI6PZ2wPbShQTiM/sTKlfYNw8H9btmfys69InEuhp0dLIpq2XsOxNNeYRu+9JOp4KpOsakfk4NvM3/d57tFr91M='} 2024-01-24T15:33:32.872 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-24_13:20:07-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7530993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-24 13:39:10.627900', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG7uo/iwSJdGQ71ymsdoikagQTGpJfY2j1wVDZ+MX9OUV8Jev1J0swzSpQDMJ1RkC3+idQGjoMx1olU7dutxTKM='} 2024-01-24T15:33:32.901 INFO:teuthology.orchestra.console.smithi029:Power off 2024-01-24T15:33:32.902 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-24T15:33:32.929 DEBUG:teuthology.orchestra.console.smithi029:power off output: Chassis Power Control: Down/Off 2024-01-24T15:33:32.930 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T15:33:32.945 INFO:teuthology.orchestra.console.smithi163:Power off 2024-01-24T15:33:32.945 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-24T15:33:32.959 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2024-01-24T15:33:32.967 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off 2024-01-24T15:33:32.967 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T15:33:32.991 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-01-24T15:33:36.960 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T15:33:36.992 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T15:33:37.070 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2024-01-24T15:33:37.102 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-01-24T15:33:41.072 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T15:33:41.102 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T15:33:41.182 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2024-01-24T15:33:41.213 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2024-01-24T15:33:41.284 INFO:teuthology.orchestra.console.smithi029:Power off completed 2024-01-24T15:33:41.314 INFO:teuthology.orchestra.console.smithi163:Power off completed 2024-01-24T15:33:41.419 INFO:teuthology.run:Summary data: description: fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6502.062039375305 failure_reason: '"2024-01-24T14:00:46.712338+0000 mon.a (mon.0) 365 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2024-01-24T15:33:41.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T15:33:41.500 INFO:teuthology.run:FAIL